2023-11-08T19:11:35.636 INFO:root:teuthology version: 0.0.1.dev203+g6899cd2 2023-11-08T19:11:35.637 DEBUG:teuthology.run:Teuthology command: teuthology --name pdonnell-2023-11-08_19:01:12-fs:functional-main-distro-default-smithi --archive /home/teuthworker/archive/pdonnell-2023-11-08_19:01:12-fs:functional-main-distro-default-smithi/7452105 --description fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snapshots} --owner scheduled_pdonnell@teuthology --verbose -- /home/teuthworker/archive/pdonnell-2023-11-08_19:01:12-fs:functional-main-distro-default-smithi/7452105/orig.config.yaml 2023-11-08T19:11:35.685 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-08T19:11:35.779 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2023-11-08_19:01:12-fs:functional-main-distro-default-smithi/7452105 branch: main description: fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snapshots} email: pdonnell@redhat.com first_in_suite: false job_id: '7452105' kernel: client: sha1: distro kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: pdonnell-2023-11-08_19:01:12-fs:functional-main-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: main ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - evicting unresponsive client - RECENT_CRASH sha1: 7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs check-counter: dry_run: true install: ceph: flavor: default sha1: 7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4 kclient: syntax: v1 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-batrick-testing-20231108.185857 sha1: e797689ab88984658d73bbc928b2ed06c676d14a owner: scheduled_pdonnell@teuthology priority: 50 repo: https://github.com/ceph/ceph.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 3044 sha1: 7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4 sleep_before_teardown: 0 suite: fs:functional suite_branch: wip-batrick-testing-20231108.185857 suite_path: /home/teuthworker/src/github.com_batrick_ceph_e797689ab88984658d73bbc928b2ed06c676d14a/qa suite_relpath: qa suite_repo: https://github.com/batrick/ceph.git suite_sha1: e797689ab88984658d73bbc928b2ed06c676d14a syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi143.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= smithi174.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - kclient: null - exec: mon.a: - ceph config set mgr mgr/crash/warn_recent_interval 0 - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.test_snapshots teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 6899cd26fceddb2fec83dc1a1349394b28c8998e timestamp: 2023-11-08_19:01:12 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3590714 2023-11-08T19:11:35.780 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_batrick_ceph_e797689ab88984658d73bbc928b2ed06c676d14a/qa; will attempt to use it 2023-11-08T19:11:35.781 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_batrick_ceph_e797689ab88984658d73bbc928b2ed06c676d14a/qa/tasks 2023-11-08T19:11:35.781 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-11-08T19:11:35.783 INFO:teuthology.task.internal:Checking packages... 2023-11-08T19:11:35.812 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash '7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4' 2023-11-08T19:11:35.813 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-11-08T19:11:35.813 INFO:teuthology.packaging:ref: None 2023-11-08T19:11:35.814 INFO:teuthology.packaging:tag: None 2023-11-08T19:11:35.814 INFO:teuthology.packaging:branch: main 2023-11-08T19:11:35.814 INFO:teuthology.packaging:sha1: 7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4 2023-11-08T19:11:35.815 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=main 2023-11-08T19:11:36.192 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-7224.g8858839c 2023-11-08T19:11:36.193 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-11-08T19:11:36.199 INFO:teuthology.task.internal:no buildpackages task found 2023-11-08T19:11:36.199 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-11-08T19:11:36.218 INFO:teuthology.task.internal:Saving configuration 2023-11-08T19:11:36.240 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-11-08T19:11:36.246 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-08T19:11:36.271 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi143.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2023-11-08_19:01:12-fs:functional-main-distro-default-smithi/7452105', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-08 19:06:33.302270', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-11-08T19:11:36.292 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi174.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2023-11-08_19:01:12-fs:functional-main-distro-default-smithi/7452105', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-08 19:06:33.304851', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-11-08T19:11:36.292 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-11-08T19:11:36.299 INFO:teuthology.task.internal:roles: ubuntu@smithi143.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2023-11-08T19:11:36.299 INFO:teuthology.task.internal:roles: ubuntu@smithi174.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2023-11-08T19:11:36.300 INFO:teuthology.run_tasks:Running task console_log... 2023-11-08T19:11:36.376 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f35a72d5b80>, signals=[15]) 2023-11-08T19:11:36.377 INFO:teuthology.run_tasks:Running task internal.connect... 2023-11-08T19:11:36.383 INFO:teuthology.task.internal:Opening connections... 2023-11-08T19:11:36.383 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi143.front.sepia.ceph.com 2023-11-08T19:11:36.385 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi143.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-08T19:11:36.453 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi174.front.sepia.ceph.com 2023-11-08T19:11:36.454 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi174.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-08T19:11:36.527 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-11-08T19:11:36.533 DEBUG:teuthology.orchestra.run.smithi143:> uname -m 2023-11-08T19:11:36.553 INFO:teuthology.orchestra.run.smithi143.stdout:x86_64 2023-11-08T19:11:36.554 DEBUG:teuthology.orchestra.run.smithi143:> cat /etc/os-release 2023-11-08T19:11:36.613 INFO:teuthology.orchestra.run.smithi143.stdout:NAME="Red Hat Enterprise Linux" 2023-11-08T19:11:36.613 INFO:teuthology.orchestra.run.smithi143.stdout:VERSION="8.6 (Ootpa)" 2023-11-08T19:11:36.613 INFO:teuthology.orchestra.run.smithi143.stdout:ID="rhel" 2023-11-08T19:11:36.614 INFO:teuthology.orchestra.run.smithi143.stdout:ID_LIKE="fedora" 2023-11-08T19:11:36.614 INFO:teuthology.orchestra.run.smithi143.stdout:VERSION_ID="8.6" 2023-11-08T19:11:36.615 INFO:teuthology.orchestra.run.smithi143.stdout:PLATFORM_ID="platform:el8" 2023-11-08T19:11:36.617 INFO:teuthology.orchestra.run.smithi143.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-11-08T19:11:36.618 INFO:teuthology.orchestra.run.smithi143.stdout:ANSI_COLOR="0;31" 2023-11-08T19:11:36.618 INFO:teuthology.orchestra.run.smithi143.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-11-08T19:11:36.618 INFO:teuthology.orchestra.run.smithi143.stdout:HOME_URL="https://www.redhat.com/" 2023-11-08T19:11:36.618 INFO:teuthology.orchestra.run.smithi143.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-11-08T19:11:36.619 INFO:teuthology.orchestra.run.smithi143.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-08T19:11:36.619 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:11:36.619 INFO:teuthology.orchestra.run.smithi143.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-11-08T19:11:36.619 INFO:teuthology.orchestra.run.smithi143.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-11-08T19:11:36.619 INFO:teuthology.orchestra.run.smithi143.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-11-08T19:11:36.619 INFO:teuthology.orchestra.run.smithi143.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-11-08T19:11:36.620 INFO:teuthology.lock.ops:Updating smithi143.front.sepia.ceph.com on lock server 2023-11-08T19:11:36.644 DEBUG:teuthology.orchestra.run.smithi174:> uname -m 2023-11-08T19:11:36.665 INFO:teuthology.orchestra.run.smithi174.stdout:x86_64 2023-11-08T19:11:36.665 DEBUG:teuthology.orchestra.run.smithi174:> cat /etc/os-release 2023-11-08T19:11:36.724 INFO:teuthology.orchestra.run.smithi174.stdout:NAME="Red Hat Enterprise Linux" 2023-11-08T19:11:36.724 INFO:teuthology.orchestra.run.smithi174.stdout:VERSION="8.6 (Ootpa)" 2023-11-08T19:11:36.725 INFO:teuthology.orchestra.run.smithi174.stdout:ID="rhel" 2023-11-08T19:11:36.725 INFO:teuthology.orchestra.run.smithi174.stdout:ID_LIKE="fedora" 2023-11-08T19:11:36.725 INFO:teuthology.orchestra.run.smithi174.stdout:VERSION_ID="8.6" 2023-11-08T19:11:36.725 INFO:teuthology.orchestra.run.smithi174.stdout:PLATFORM_ID="platform:el8" 2023-11-08T19:11:36.725 INFO:teuthology.orchestra.run.smithi174.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-11-08T19:11:36.726 INFO:teuthology.orchestra.run.smithi174.stdout:ANSI_COLOR="0;31" 2023-11-08T19:11:36.726 INFO:teuthology.orchestra.run.smithi174.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-11-08T19:11:36.727 INFO:teuthology.orchestra.run.smithi174.stdout:HOME_URL="https://www.redhat.com/" 2023-11-08T19:11:36.727 INFO:teuthology.orchestra.run.smithi174.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-11-08T19:11:36.727 INFO:teuthology.orchestra.run.smithi174.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-08T19:11:36.727 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-08T19:11:36.727 INFO:teuthology.orchestra.run.smithi174.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-11-08T19:11:36.728 INFO:teuthology.orchestra.run.smithi174.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-11-08T19:11:36.728 INFO:teuthology.orchestra.run.smithi174.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-11-08T19:11:36.728 INFO:teuthology.orchestra.run.smithi174.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-11-08T19:11:36.729 INFO:teuthology.lock.ops:Updating smithi174.front.sepia.ceph.com on lock server 2023-11-08T19:11:36.752 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-11-08T19:11:36.762 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-11-08T19:11:36.768 INFO:teuthology.task.internal:Checking for old test directory... 2023-11-08T19:11:36.768 DEBUG:teuthology.orchestra.run.smithi143:> test '!' -e /home/ubuntu/cephtest 2023-11-08T19:11:36.771 DEBUG:teuthology.orchestra.run.smithi174:> test '!' -e /home/ubuntu/cephtest 2023-11-08T19:11:36.789 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-11-08T19:11:36.796 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-11-08T19:11:36.796 DEBUG:teuthology.orchestra.run.smithi143:> test -z $(ls -A /var/lib/ceph) 2023-11-08T19:11:36.830 DEBUG:teuthology.orchestra.run.smithi174:> test -z $(ls -A /var/lib/ceph) 2023-11-08T19:11:36.862 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-11-08T19:11:36.913 INFO:teuthology.run_tasks:Running task kernel... 2023-11-08T19:11:36.928 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2023-11-08T19:11:36.929 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}, 'client.2': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}, 'client.3': {'sha1': 'distro'}} 2023-11-08T19:11:36.929 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}}, timeout 300 2023-11-08T19:11:36.929 DEBUG:teuthology.orchestra.run.smithi143:> test -f /run/.containerenv -o -f /.dockerenv 2023-11-08T19:11:36.930 DEBUG:teuthology.orchestra.run.smithi174:> test -f /run/.containerenv -o -f /.dockerenv 2023-11-08T19:11:36.948 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-08T19:11:36.949 DEBUG:teuthology.orchestra.run.smithi174:> uname -r 2023-11-08T19:11:36.950 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-08T19:11:36.950 DEBUG:teuthology.orchestra.run.smithi143:> uname -r 2023-11-08T19:11:37.007 INFO:teuthology.orchestra.run.smithi174.stdout:4.18.0-372.9.1.el8.x86_64 2023-11-08T19:11:37.008 INFO:teuthology.task.kernel:Running kernel on smithi174: 4.18.0-372.9.1.el8.x86_64 2023-11-08T19:11:37.008 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum install -y kernel 2023-11-08T19:11:37.010 INFO:teuthology.orchestra.run.smithi143.stdout:4.18.0-372.9.1.el8.x86_64 2023-11-08T19:11:37.011 INFO:teuthology.task.kernel:Running kernel on smithi143: 4.18.0-372.9.1.el8.x86_64 2023-11-08T19:11:37.011 DEBUG:teuthology.orchestra.run.smithi143:> sudo yum install -y kernel 2023-11-08T19:11:38.203 INFO:teuthology.orchestra.run.smithi143.stdout:Updating Subscription Management repositories. 2023-11-08T19:11:38.203 INFO:teuthology.orchestra.run.smithi143.stdout:Unable to read consumer identity 2023-11-08T19:11:38.207 INFO:teuthology.orchestra.run.smithi174.stdout:Updating Subscription Management repositories. 2023-11-08T19:11:38.207 INFO:teuthology.orchestra.run.smithi174.stdout:Unable to read consumer identity 2023-11-08T19:11:39.598 INFO:teuthology.orchestra.run.smithi174.stdout:Copr repo for python3-asyncssh owned by ceph 23 kB/s | 3.5 kB 00:00 2023-11-08T19:11:39.621 INFO:teuthology.orchestra.run.smithi143.stdout:Copr repo for python3-asyncssh owned by ceph 22 kB/s | 3.5 kB 00:00 2023-11-08T19:11:45.931 INFO:teuthology.orchestra.run.smithi174.stdout:Extra Packages for Enterprise Linux 2.5 MB/s | 16 MB 00:06 2023-11-08T19:11:46.671 INFO:teuthology.orchestra.run.smithi143.stdout:Extra Packages for Enterprise Linux 2.3 MB/s | 16 MB 00:07 2023-11-08T19:11:49.525 INFO:teuthology.orchestra.run.smithi174.stdout:lab-extras 418 kB/s | 24 kB 00:00 2023-11-08T19:11:50.188 INFO:teuthology.orchestra.run.smithi143.stdout:lab-extras 413 kB/s | 24 kB 00:00 2023-11-08T19:11:50.261 INFO:teuthology.orchestra.run.smithi174.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2023-11-08T19:11:50.316 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies resolved. 2023-11-08T19:11:50.317 INFO:teuthology.orchestra.run.smithi174.stdout:Nothing to do. 2023-11-08T19:11:50.317 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2023-11-08T19:11:50.430 DEBUG:teuthology.orchestra.run.smithi174:> echo no | sudo yum reinstall kernel || true 2023-11-08T19:11:50.801 INFO:teuthology.orchestra.run.smithi174.stdout:Updating Subscription Management repositories. 2023-11-08T19:11:50.801 INFO:teuthology.orchestra.run.smithi174.stdout:Unable to read consumer identity 2023-11-08T19:11:50.899 INFO:teuthology.orchestra.run.smithi143.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2023-11-08T19:11:50.942 INFO:teuthology.orchestra.run.smithi143.stdout:Dependencies resolved. 2023-11-08T19:11:50.943 INFO:teuthology.orchestra.run.smithi143.stdout:Nothing to do. 2023-11-08T19:11:50.944 INFO:teuthology.orchestra.run.smithi143.stdout:Complete! 2023-11-08T19:11:51.065 INFO:teuthology.orchestra.run.smithi174.stdout:Last metadata expiration check: 0:00:02 ago on Wed 08 Nov 2023 07:11:49 PM UTC. 2023-11-08T19:11:51.081 DEBUG:teuthology.orchestra.run.smithi143:> echo no | sudo yum reinstall kernel || true 2023-11-08T19:11:51.106 INFO:teuthology.orchestra.run.smithi174.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-11-08T19:11:51.145 INFO:teuthology.orchestra.run.smithi174.stderr:Error: No packages marked for reinstall. 2023-11-08T19:11:51.192 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum reinstall -y kernel || true 2023-11-08T19:11:51.411 INFO:teuthology.orchestra.run.smithi143.stdout:Updating Subscription Management repositories. 2023-11-08T19:11:51.412 INFO:teuthology.orchestra.run.smithi143.stdout:Unable to read consumer identity 2023-11-08T19:11:51.518 INFO:teuthology.orchestra.run.smithi174.stdout:Updating Subscription Management repositories. 2023-11-08T19:11:51.519 INFO:teuthology.orchestra.run.smithi174.stdout:Unable to read consumer identity 2023-11-08T19:11:51.680 INFO:teuthology.orchestra.run.smithi143.stdout:Last metadata expiration check: 0:00:01 ago on Wed 08 Nov 2023 07:11:50 PM UTC. 2023-11-08T19:11:51.720 INFO:teuthology.orchestra.run.smithi143.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-11-08T19:11:51.757 INFO:teuthology.orchestra.run.smithi143.stderr:Error: No packages marked for reinstall. 2023-11-08T19:11:51.782 INFO:teuthology.orchestra.run.smithi174.stdout:Last metadata expiration check: 0:00:02 ago on Wed 08 Nov 2023 07:11:49 PM UTC. 2023-11-08T19:11:51.804 DEBUG:teuthology.orchestra.run.smithi143:> sudo yum reinstall -y kernel || true 2023-11-08T19:11:51.821 INFO:teuthology.orchestra.run.smithi174.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-11-08T19:11:51.848 INFO:teuthology.orchestra.run.smithi174.stderr:Error: No packages marked for reinstall. 2023-11-08T19:11:51.893 DEBUG:teuthology.orchestra.run.smithi174:> rpm -q kernel | sort -rV | head -n 1 2023-11-08T19:11:51.973 INFO:teuthology.orchestra.run.smithi174.stdout:kernel-4.18.0-372.9.1.el8.x86_64 2023-11-08T19:11:51.974 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-372.9.1.el8.x86_64 2023-11-08T19:11:51.974 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2023-11-08T19:11:51.975 DEBUG:teuthology.parallel:result is None 2023-11-08T19:11:52.133 INFO:teuthology.orchestra.run.smithi143.stdout:Updating Subscription Management repositories. 2023-11-08T19:11:52.133 INFO:teuthology.orchestra.run.smithi143.stdout:Unable to read consumer identity 2023-11-08T19:11:52.394 INFO:teuthology.orchestra.run.smithi143.stdout:Last metadata expiration check: 0:00:02 ago on Wed 08 Nov 2023 07:11:50 PM UTC. 2023-11-08T19:11:52.433 INFO:teuthology.orchestra.run.smithi143.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-11-08T19:11:52.475 INFO:teuthology.orchestra.run.smithi143.stderr:Error: No packages marked for reinstall. 2023-11-08T19:11:52.525 DEBUG:teuthology.orchestra.run.smithi143:> rpm -q kernel | sort -rV | head -n 1 2023-11-08T19:11:52.607 INFO:teuthology.orchestra.run.smithi143.stdout:kernel-4.18.0-372.9.1.el8.x86_64 2023-11-08T19:11:52.607 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-372.9.1.el8.x86_64 2023-11-08T19:11:52.607 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2023-11-08T19:11:52.608 DEBUG:teuthology.parallel:result is None 2023-11-08T19:11:52.608 INFO:teuthology.run_tasks:Running task internal.base... 2023-11-08T19:11:52.615 INFO:teuthology.task.internal:Creating test directory... 2023-11-08T19:11:52.616 DEBUG:teuthology.orchestra.run.smithi143:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-11-08T19:11:52.618 DEBUG:teuthology.orchestra.run.smithi174:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-11-08T19:11:52.637 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-11-08T19:11:52.649 INFO:teuthology.run_tasks:Running task internal.archive... 2023-11-08T19:11:52.656 INFO:teuthology.task.internal:Creating archive directory... 2023-11-08T19:11:52.656 DEBUG:teuthology.orchestra.run.smithi143:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-11-08T19:11:52.675 DEBUG:teuthology.orchestra.run.smithi174:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-11-08T19:11:52.703 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-11-08T19:11:52.710 INFO:teuthology.task.internal:Enabling coredump saving... 2023-11-08T19:11:52.710 DEBUG:teuthology.orchestra.run.smithi143:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-11-08T19:11:52.733 DEBUG:teuthology.orchestra.run.smithi174:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-11-08T19:11:52.769 INFO:teuthology.orchestra.run.smithi174.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-08T19:11:52.780 INFO:teuthology.orchestra.run.smithi143.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-08T19:11:52.790 INFO:teuthology.orchestra.run.smithi174.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-08T19:11:52.795 INFO:teuthology.orchestra.run.smithi143.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-08T19:11:52.795 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-11-08T19:11:52.806 INFO:teuthology.task.internal:Configuring sudo... 2023-11-08T19:11:52.807 DEBUG:teuthology.orchestra.run.smithi143:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-11-08T19:11:52.809 DEBUG:teuthology.orchestra.run.smithi174:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-11-08T19:11:52.860 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-11-08T19:11:52.873 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-11-08T19:11:52.873 DEBUG:teuthology.orchestra.run.smithi143:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-11-08T19:11:52.876 DEBUG:teuthology.orchestra.run.smithi174:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-11-08T19:11:52.918 DEBUG:teuthology.orchestra.run.smithi143:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-08T19:11:52.976 DEBUG:teuthology.orchestra.run.smithi143:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-08T19:11:53.022 DEBUG:teuthology.orchestra.run.smithi143:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-08T19:11:53.106 DEBUG:teuthology.orchestra.run.smithi143:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-08T19:11:53.146 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-08T19:11:53.147 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-11-08T19:11:53.213 DEBUG:teuthology.orchestra.run.smithi174:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-08T19:11:53.260 DEBUG:teuthology.orchestra.run.smithi174:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-08T19:11:53.318 DEBUG:teuthology.orchestra.run.smithi174:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-08T19:11:53.403 DEBUG:teuthology.orchestra.run.smithi174:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-08T19:11:53.445 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2023-11-08T19:11:53.445 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-11-08T19:11:53.514 DEBUG:teuthology.orchestra.run.smithi143:> sudo service rsyslog restart 2023-11-08T19:11:53.517 DEBUG:teuthology.orchestra.run.smithi174:> sudo service rsyslog restart 2023-11-08T19:11:53.581 INFO:teuthology.orchestra.run.smithi143.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-08T19:11:53.609 INFO:teuthology.orchestra.run.smithi174.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-08T19:11:53.901 INFO:teuthology.run_tasks:Running task internal.timer... 2023-11-08T19:11:53.939 INFO:teuthology.task.internal:Starting timer... 2023-11-08T19:11:53.939 INFO:teuthology.run_tasks:Running task pcp... 2023-11-08T19:11:53.955 INFO:teuthology.run_tasks:Running task selinux... 2023-11-08T19:11:53.971 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2023-11-08T19:11:53.971 DEBUG:teuthology.orchestra.run.smithi143:> sudo service auditd rotate 2023-11-08T19:11:54.048 INFO:teuthology.orchestra.run.smithi143.stdout:Rotating logs: 2023-11-08T19:11:54.049 DEBUG:teuthology.orchestra.run.smithi174:> sudo service auditd rotate 2023-11-08T19:11:54.110 INFO:teuthology.orchestra.run.smithi174.stdout:Rotating logs: 2023-11-08T19:11:54.112 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-11-08T19:11:54.112 DEBUG:teuthology.orchestra.run.smithi143:> /usr/sbin/getenforce 2023-11-08T19:11:54.141 INFO:teuthology.orchestra.run.smithi143.stdout:Permissive 2023-11-08T19:11:54.142 DEBUG:teuthology.orchestra.run.smithi174:> /usr/sbin/getenforce 2023-11-08T19:11:54.171 INFO:teuthology.orchestra.run.smithi174.stdout:Permissive 2023-11-08T19:11:54.171 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi143.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi174.front.sepia.ceph.com': 'permissive'} 2023-11-08T19:11:54.172 DEBUG:teuthology.orchestra.run.smithi143:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-08T19:11:54.213 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-08T19:11:54.214 DEBUG:teuthology.orchestra.run.smithi174:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-08T19:11:54.245 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-08T19:11:54.246 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-11-08T19:11:54.247 DEBUG:teuthology.orchestra.run.smithi143:> sudo /usr/sbin/setenforce permissive 2023-11-08T19:11:54.286 DEBUG:teuthology.orchestra.run.smithi174:> sudo /usr/sbin/setenforce permissive 2023-11-08T19:11:54.317 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-11-08T19:11:54.344 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2023-11-08T19:11:54.345 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-11-08T19:11:54.363 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-11-08T19:11:54.369 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi143.front.sepia.ceph.com,smithi174.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-11-08T19:20:56.393 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi143.front.sepia.ceph.com'), Remote(name='ubuntu@smithi174.front.sepia.ceph.com')] 2023-11-08T19:20:56.394 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi143.front.sepia.ceph.com' 2023-11-08T19:20:56.395 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi143.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-08T19:20:56.464 DEBUG:teuthology.orchestra.run.smithi143:> true 2023-11-08T19:20:56.531 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi143.front.sepia.ceph.com' 2023-11-08T19:20:56.532 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi174.front.sepia.ceph.com' 2023-11-08T19:20:56.532 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi174.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-08T19:20:56.600 DEBUG:teuthology.orchestra.run.smithi174:> true 2023-11-08T19:20:56.667 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi174.front.sepia.ceph.com' 2023-11-08T19:20:56.668 INFO:teuthology.run_tasks:Running task clock... 2023-11-08T19:20:56.681 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-11-08T19:20:56.681 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-11-08T19:20:56.682 DEBUG:teuthology.orchestra.run.smithi143:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-08T19:20:56.684 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-11-08T19:20:56.685 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-08T19:20:56.716 INFO:teuthology.orchestra.run.smithi143.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-11-08T19:20:56.731 INFO:teuthology.orchestra.run.smithi143.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-11-08T19:20:56.739 INFO:teuthology.orchestra.run.smithi174.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-11-08T19:20:56.754 INFO:teuthology.orchestra.run.smithi174.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-11-08T19:20:56.795 INFO:teuthology.orchestra.run.smithi143.stderr:sudo: ntpd: command not found 2023-11-08T19:20:56.806 INFO:teuthology.orchestra.run.smithi143.stdout:506 Cannot talk to daemon 2023-11-08T19:20:56.821 INFO:teuthology.orchestra.run.smithi143.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-11-08T19:20:56.822 INFO:teuthology.orchestra.run.smithi174.stderr:sudo: ntpd: command not found 2023-11-08T19:20:56.834 INFO:teuthology.orchestra.run.smithi174.stdout:506 Cannot talk to daemon 2023-11-08T19:20:56.835 INFO:teuthology.orchestra.run.smithi143.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-11-08T19:20:56.848 INFO:teuthology.orchestra.run.smithi174.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-11-08T19:20:56.861 INFO:teuthology.orchestra.run.smithi174.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-11-08T19:20:56.903 INFO:teuthology.orchestra.run.smithi143.stderr:bash: ntpq: command not found 2023-11-08T19:20:56.907 INFO:teuthology.orchestra.run.smithi143.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-08T19:20:56.907 INFO:teuthology.orchestra.run.smithi143.stdout:=============================================================================== 2023-11-08T19:20:56.907 INFO:teuthology.orchestra.run.smithi143.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-08T19:20:56.907 INFO:teuthology.orchestra.run.smithi143.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-08T19:20:56.908 INFO:teuthology.orchestra.run.smithi143.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-08T19:20:56.908 INFO:teuthology.orchestra.run.smithi143.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-08T19:20:56.934 INFO:teuthology.orchestra.run.smithi174.stderr:bash: ntpq: command not found 2023-11-08T19:20:56.938 INFO:teuthology.orchestra.run.smithi174.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-08T19:20:56.938 INFO:teuthology.orchestra.run.smithi174.stdout:=============================================================================== 2023-11-08T19:20:56.939 INFO:teuthology.orchestra.run.smithi174.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-08T19:20:56.939 INFO:teuthology.orchestra.run.smithi174.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-08T19:20:56.939 INFO:teuthology.orchestra.run.smithi174.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-08T19:20:56.939 INFO:teuthology.orchestra.run.smithi174.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-08T19:20:56.940 INFO:teuthology.run_tasks:Running task install... 2023-11-08T19:20:56.948 DEBUG:teuthology.task.install:project ceph 2023-11-08T19:20:56.949 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4'}} 2023-11-08T19:20:56.949 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4'} 2023-11-08T19:20:56.949 INFO:teuthology.task.install:Using flavor: default 2023-11-08T19:20:56.955 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-11-08T19:20:56.955 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-11-08T19:20:56.956 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4', 'tag': None, 'wait_for_package': False} 2023-11-08T19:20:56.957 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4 2023-11-08T19:20:56.959 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4', 'tag': None, 'wait_for_package': False} 2023-11-08T19:20:56.959 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4 2023-11-08T19:20:57.103 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/main/7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4/centos/8/flavors/default/ 2023-11-08T19:20:57.104 INFO:teuthology.task.install.rpm:Package version is 18.0.0-7035.g7532011a 2023-11-08T19:20:57.198 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/main/7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4/centos/8/flavors/default/ 2023-11-08T19:20:57.198 INFO:teuthology.task.install.rpm:Package version is 18.0.0-7035.g7532011a 2023-11-08T19:20:57.211 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/main/7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/main/7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/main/7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-11-08T19:20:57.211 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2023-11-08T19:20:57.211 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-11-08T19:20:57.240 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-11-08T19:20:57.240 DEBUG:teuthology.orchestra.run.smithi174:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-11-08T19:20:57.308 DEBUG:teuthology.orchestra.run.smithi174:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-11-08T19:20:57.385 DEBUG:teuthology.orchestra.run.smithi174:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-11-08T19:20:57.395 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/main/7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/main/7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/main/7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-11-08T19:20:57.396 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-08T19:20:57.396 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-11-08T19:20:57.426 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-11-08T19:20:57.426 DEBUG:teuthology.orchestra.run.smithi143:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-11-08T19:20:57.451 INFO:teuthology.orchestra.run.smithi174.stdout:check_obsoletes = 1 2023-11-08T19:20:57.453 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum clean all 2023-11-08T19:20:57.495 DEBUG:teuthology.orchestra.run.smithi143:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-11-08T19:20:57.574 DEBUG:teuthology.orchestra.run.smithi143:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-11-08T19:20:57.641 INFO:teuthology.orchestra.run.smithi143.stdout:check_obsoletes = 1 2023-11-08T19:20:57.643 DEBUG:teuthology.orchestra.run.smithi143:> sudo yum clean all 2023-11-08T19:20:57.824 INFO:teuthology.orchestra.run.smithi174.stdout:Updating Subscription Management repositories. 2023-11-08T19:20:58.016 INFO:teuthology.orchestra.run.smithi143.stdout:Updating Subscription Management repositories. 2023-11-08T19:20:58.104 INFO:teuthology.orchestra.run.smithi174.stdout:44 files removed 2023-11-08T19:20:58.150 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-11-08T19:20:58.254 INFO:teuthology.orchestra.run.smithi143.stdout:44 files removed 2023-11-08T19:20:58.307 DEBUG:teuthology.orchestra.run.smithi143:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-11-08T19:20:58.484 INFO:teuthology.orchestra.run.smithi174.stdout:Updating Subscription Management repositories. 2023-11-08T19:20:58.637 INFO:teuthology.orchestra.run.smithi143.stdout:Updating Subscription Management repositories. 2023-11-08T19:20:59.418 INFO:teuthology.orchestra.run.smithi174.stdout:ceph packages for x86_64 167 kB/s | 81 kB 00:00 2023-11-08T19:20:59.487 INFO:teuthology.orchestra.run.smithi143.stdout:ceph packages for x86_64 166 kB/s | 81 kB 00:00 2023-11-08T19:20:59.838 INFO:teuthology.orchestra.run.smithi143.stdout:ceph noarch packages 30 kB/s | 9.4 kB 00:00 2023-11-08T19:20:59.869 INFO:teuthology.orchestra.run.smithi174.stdout:ceph noarch packages 22 kB/s | 9.4 kB 00:00 2023-11-08T19:21:00.136 INFO:teuthology.orchestra.run.smithi174.stdout:ceph source packages 6.1 kB/s | 1.5 kB 00:00 2023-11-08T19:21:00.214 INFO:teuthology.orchestra.run.smithi143.stdout:ceph source packages 4.2 kB/s | 1.5 kB 00:00 2023-11-08T19:21:00.279 INFO:teuthology.orchestra.run.smithi174.stdout:Copr repo for python3-asyncssh owned by ceph 28 kB/s | 3.5 kB 00:00 2023-11-08T19:21:00.356 INFO:teuthology.orchestra.run.smithi143.stdout:Copr repo for python3-asyncssh owned by ceph 28 kB/s | 3.5 kB 00:00 2023-11-08T19:21:02.407 INFO:teuthology.orchestra.run.smithi174.stdout:Extra Packages for Enterprise Linux 7.6 MB/s | 16 MB 00:02 2023-11-08T19:21:05.904 INFO:teuthology.orchestra.run.smithi174.stdout:lab-extras 406 kB/s | 24 kB 00:00 2023-11-08T19:21:06.789 INFO:teuthology.orchestra.run.smithi174.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 65 MB/s | 53 MB 00:00 2023-11-08T19:21:14.907 INFO:teuthology.orchestra.run.smithi174.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 58 MB/s | 47 MB 00:00 2023-11-08T19:21:18.357 INFO:teuthology.orchestra.run.smithi143.stdout:Extra Packages for Enterprise Linux 908 kB/s | 16 MB 00:17 2023-11-08T19:21:21.879 INFO:teuthology.orchestra.run.smithi143.stdout:lab-extras 463 kB/s | 24 kB 00:00 2023-11-08T19:21:22.765 INFO:teuthology.orchestra.run.smithi143.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 58 MB/s | 47 MB 00:00 2023-11-08T19:21:25.830 INFO:teuthology.orchestra.run.smithi174.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-11-08T19:21:25.831 INFO:teuthology.orchestra.run.smithi174.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-11-08T19:21:25.845 INFO:teuthology.orchestra.run.smithi174.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-11-08T19:21:25.847 INFO:teuthology.orchestra.run.smithi174.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-11-08T19:21:25.848 INFO:teuthology.orchestra.run.smithi174.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-11-08T19:21:25.849 INFO:teuthology.orchestra.run.smithi174.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-11-08T19:21:25.851 INFO:teuthology.orchestra.run.smithi174.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-11-08T19:21:25.854 INFO:teuthology.orchestra.run.smithi174.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-11-08T19:21:25.856 INFO:teuthology.orchestra.run.smithi174.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-11-08T19:21:25.985 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies resolved. 2023-11-08T19:21:25.997 INFO:teuthology.orchestra.run.smithi174.stdout:=========================================================================================================================== 2023-11-08T19:21:25.998 INFO:teuthology.orchestra.run.smithi174.stdout: Package Arch Version Repository Size 2023-11-08T19:21:25.998 INFO:teuthology.orchestra.run.smithi174.stdout:=========================================================================================================================== 2023-11-08T19:21:25.998 INFO:teuthology.orchestra.run.smithi174.stdout:Installing: 2023-11-08T19:21:25.998 INFO:teuthology.orchestra.run.smithi174.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-11-08T19:21:25.999 INFO:teuthology.orchestra.run.smithi174.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-11-08T19:21:25.999 INFO:teuthology.orchestra.run.smithi174.stdout: ceph x86_64 2:18.0.0-7035.g7532011a.el8 ceph 6.4 k 2023-11-08T19:21:25.999 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-base x86_64 2:18.0.0-7035.g7532011a.el8 ceph 5.4 M 2023-11-08T19:21:25.999 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-fuse x86_64 2:18.0.0-7035.g7532011a.el8 ceph 919 k 2023-11-08T19:21:25.999 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-7035.g7532011a.el8 ceph 171 k 2023-11-08T19:21:26.000 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr x86_64 2:18.0.0-7035.g7532011a.el8 ceph 1.6 M 2023-11-08T19:21:26.000 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-cephadm noarch 2:18.0.0-7035.g7532011a.el8 ceph-noarch 132 k 2023-11-08T19:21:26.000 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-dashboard noarch 2:18.0.0-7035.g7532011a.el8 ceph-noarch 1.8 M 2023-11-08T19:21:26.000 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-7035.g7532011a.el8 ceph-noarch 7.4 M 2023-11-08T19:21:26.000 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-rook noarch 2:18.0.0-7035.g7532011a.el8 ceph-noarch 48 k 2023-11-08T19:21:26.001 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-radosgw x86_64 2:18.0.0-7035.g7532011a.el8 ceph 14 M 2023-11-08T19:21:26.001 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-test x86_64 2:18.0.0-7035.g7532011a.el8 ceph 49 M 2023-11-08T19:21:26.001 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-volume noarch 2:18.0.0-7035.g7532011a.el8 ceph-noarch 261 k 2023-11-08T19:21:26.001 INFO:teuthology.orchestra.run.smithi174.stdout: cephadm noarch 2:18.0.0-7035.g7532011a.el8 ceph-noarch 191 k 2023-11-08T19:21:26.001 INFO:teuthology.orchestra.run.smithi174.stdout: cephfs-mirror x86_64 2:18.0.0-7035.g7532011a.el8 ceph 262 k 2023-11-08T19:21:26.002 INFO:teuthology.orchestra.run.smithi174.stdout: cephfs-top noarch 2:18.0.0-7035.g7532011a.el8 ceph-noarch 18 k 2023-11-08T19:21:26.002 INFO:teuthology.orchestra.run.smithi174.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-11-08T19:21:26.002 INFO:teuthology.orchestra.run.smithi174.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-11-08T19:21:26.002 INFO:teuthology.orchestra.run.smithi174.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-11-08T19:21:26.002 INFO:teuthology.orchestra.run.smithi174.stdout: libcephfs-devel x86_64 2:18.0.0-7035.g7532011a.el8 ceph 31 k 2023-11-08T19:21:26.003 INFO:teuthology.orchestra.run.smithi174.stdout: libcephfs2 x86_64 2:18.0.0-7035.g7532011a.el8 ceph 838 k 2023-11-08T19:21:26.003 INFO:teuthology.orchestra.run.smithi174.stdout: librados-devel x86_64 2:18.0.0-7035.g7532011a.el8 ceph 139 k 2023-11-08T19:21:26.003 INFO:teuthology.orchestra.run.smithi174.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-11-08T19:21:26.003 INFO:teuthology.orchestra.run.smithi174.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-11-08T19:21:26.003 INFO:teuthology.orchestra.run.smithi174.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-11-08T19:21:26.004 INFO:teuthology.orchestra.run.smithi174.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-11-08T19:21:26.004 INFO:teuthology.orchestra.run.smithi174.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-11-08T19:21:26.004 INFO:teuthology.orchestra.run.smithi174.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-11-08T19:21:26.004 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cephfs x86_64 2:18.0.0-7035.g7532011a.el8 ceph 197 k 2023-11-08T19:21:26.004 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rados x86_64 2:18.0.0-7035.g7532011a.el8 ceph 371 k 2023-11-08T19:21:26.005 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rbd x86_64 2:18.0.0-7035.g7532011a.el8 ceph 364 k 2023-11-08T19:21:26.005 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rgw x86_64 2:18.0.0-7035.g7532011a.el8 ceph 107 k 2023-11-08T19:21:26.005 INFO:teuthology.orchestra.run.smithi174.stdout: rbd-fuse x86_64 2:18.0.0-7035.g7532011a.el8 ceph 94 k 2023-11-08T19:21:26.005 INFO:teuthology.orchestra.run.smithi174.stdout: rbd-mirror x86_64 2:18.0.0-7035.g7532011a.el8 ceph 3.6 M 2023-11-08T19:21:26.005 INFO:teuthology.orchestra.run.smithi174.stdout: rbd-nbd x86_64 2:18.0.0-7035.g7532011a.el8 ceph 181 k 2023-11-08T19:21:26.006 INFO:teuthology.orchestra.run.smithi174.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-11-08T19:21:26.006 INFO:teuthology.orchestra.run.smithi174.stdout:Upgrading: 2023-11-08T19:21:26.006 INFO:teuthology.orchestra.run.smithi174.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-11-08T19:21:26.006 INFO:teuthology.orchestra.run.smithi174.stdout: librados2 x86_64 2:18.0.0-7035.g7532011a.el8 ceph 3.8 M 2023-11-08T19:21:26.006 INFO:teuthology.orchestra.run.smithi174.stdout: librbd1 x86_64 2:18.0.0-7035.g7532011a.el8 ceph 3.7 M 2023-11-08T19:21:26.007 INFO:teuthology.orchestra.run.smithi174.stdout:Installing dependencies: 2023-11-08T19:21:26.007 INFO:teuthology.orchestra.run.smithi174.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-11-08T19:21:26.007 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-common x86_64 2:18.0.0-7035.g7532011a.el8 ceph 23 M 2023-11-08T19:21:26.007 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-grafana-dashboards noarch 2:18.0.0-7035.g7532011a.el8 ceph-noarch 23 k 2023-11-08T19:21:26.007 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mds x86_64 2:18.0.0-7035.g7532011a.el8 ceph 2.3 M 2023-11-08T19:21:26.008 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-modules-core noarch 2:18.0.0-7035.g7532011a.el8 ceph-noarch 242 k 2023-11-08T19:21:26.008 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mon x86_64 2:18.0.0-7035.g7532011a.el8 ceph 4.6 M 2023-11-08T19:21:26.008 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-osd x86_64 2:18.0.0-7035.g7532011a.el8 ceph 18 M 2023-11-08T19:21:26.008 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-prometheus-alerts noarch 2:18.0.0-7035.g7532011a.el8 ceph-noarch 15 k 2023-11-08T19:21:26.008 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-selinux x86_64 2:18.0.0-7035.g7532011a.el8 ceph 24 k 2023-11-08T19:21:26.009 INFO:teuthology.orchestra.run.smithi174.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-11-08T19:21:26.009 INFO:teuthology.orchestra.run.smithi174.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-11-08T19:21:26.009 INFO:teuthology.orchestra.run.smithi174.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-11-08T19:21:26.009 INFO:teuthology.orchestra.run.smithi174.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-11-08T19:21:26.009 INFO:teuthology.orchestra.run.smithi174.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-11-08T19:21:26.009 INFO:teuthology.orchestra.run.smithi174.stdout: libcephsqlite x86_64 2:18.0.0-7035.g7532011a.el8 ceph 182 k 2023-11-08T19:21:26.010 INFO:teuthology.orchestra.run.smithi174.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-11-08T19:21:26.010 INFO:teuthology.orchestra.run.smithi174.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-11-08T19:21:26.010 INFO:teuthology.orchestra.run.smithi174.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-11-08T19:21:26.010 INFO:teuthology.orchestra.run.smithi174.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-11-08T19:21:26.010 INFO:teuthology.orchestra.run.smithi174.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-11-08T19:21:26.011 INFO:teuthology.orchestra.run.smithi174.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-11-08T19:21:26.011 INFO:teuthology.orchestra.run.smithi174.stdout: libradosstriper1 x86_64 2:18.0.0-7035.g7532011a.el8 ceph 518 k 2023-11-08T19:21:26.011 INFO:teuthology.orchestra.run.smithi174.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-11-08T19:21:26.011 INFO:teuthology.orchestra.run.smithi174.stdout: librgw2 x86_64 2:18.0.0-7035.g7532011a.el8 ceph 6.9 M 2023-11-08T19:21:26.011 INFO:teuthology.orchestra.run.smithi174.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-11-08T19:21:26.012 INFO:teuthology.orchestra.run.smithi174.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-11-08T19:21:26.012 INFO:teuthology.orchestra.run.smithi174.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-11-08T19:21:26.012 INFO:teuthology.orchestra.run.smithi174.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-11-08T19:21:26.012 INFO:teuthology.orchestra.run.smithi174.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-11-08T19:21:26.012 INFO:teuthology.orchestra.run.smithi174.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-11-08T19:21:26.013 INFO:teuthology.orchestra.run.smithi174.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-11-08T19:21:26.013 INFO:teuthology.orchestra.run.smithi174.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-11-08T19:21:26.013 INFO:teuthology.orchestra.run.smithi174.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-11-08T19:21:26.013 INFO:teuthology.orchestra.run.smithi174.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-11-08T19:21:26.014 INFO:teuthology.orchestra.run.smithi174.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-11-08T19:21:26.014 INFO:teuthology.orchestra.run.smithi174.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-11-08T19:21:26.014 INFO:teuthology.orchestra.run.smithi174.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-11-08T19:21:26.014 INFO:teuthology.orchestra.run.smithi174.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-11-08T19:21:26.015 INFO:teuthology.orchestra.run.smithi174.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-11-08T19:21:26.015 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-11-08T19:21:26.015 INFO:teuthology.orchestra.run.smithi174.stdout: python3-ceph-argparse x86_64 2:18.0.0-7035.g7532011a.el8 ceph 45 k 2023-11-08T19:21:26.015 INFO:teuthology.orchestra.run.smithi174.stdout: python3-ceph-common x86_64 2:18.0.0-7035.g7532011a.el8 ceph 126 k 2023-11-08T19:21:26.015 INFO:teuthology.orchestra.run.smithi174.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-11-08T19:21:26.015 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-11-08T19:21:26.016 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-11-08T19:21:26.016 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-11-08T19:21:26.016 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-11-08T19:21:26.016 INFO:teuthology.orchestra.run.smithi174.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-11-08T19:21:26.016 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-11-08T19:21:26.017 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-11-08T19:21:26.017 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-11-08T19:21:26.017 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-11-08T19:21:26.017 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-11-08T19:21:26.017 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-11-08T19:21:26.018 INFO:teuthology.orchestra.run.smithi174.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-11-08T19:21:26.018 INFO:teuthology.orchestra.run.smithi174.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-11-08T19:21:26.018 INFO:teuthology.orchestra.run.smithi174.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-11-08T19:21:26.018 INFO:teuthology.orchestra.run.smithi174.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-11-08T19:21:26.018 INFO:teuthology.orchestra.run.smithi174.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-11-08T19:21:26.019 INFO:teuthology.orchestra.run.smithi174.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-11-08T19:21:26.019 INFO:teuthology.orchestra.run.smithi174.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-11-08T19:21:26.019 INFO:teuthology.orchestra.run.smithi174.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-11-08T19:21:26.019 INFO:teuthology.orchestra.run.smithi174.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-11-08T19:21:26.019 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-11-08T19:21:26.020 INFO:teuthology.orchestra.run.smithi174.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-11-08T19:21:26.020 INFO:teuthology.orchestra.run.smithi174.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-11-08T19:21:26.020 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-11-08T19:21:26.020 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-11-08T19:21:26.020 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-11-08T19:21:26.021 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-11-08T19:21:26.021 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-11-08T19:21:26.021 INFO:teuthology.orchestra.run.smithi174.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-11-08T19:21:26.021 INFO:teuthology.orchestra.run.smithi174.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-11-08T19:21:26.021 INFO:teuthology.orchestra.run.smithi174.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-11-08T19:21:26.021 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-11-08T19:21:26.022 INFO:teuthology.orchestra.run.smithi174.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-11-08T19:21:26.022 INFO:teuthology.orchestra.run.smithi174.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-11-08T19:21:26.022 INFO:teuthology.orchestra.run.smithi174.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-11-08T19:21:26.022 INFO:teuthology.orchestra.run.smithi174.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-11-08T19:21:26.023 INFO:teuthology.orchestra.run.smithi174.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-11-08T19:21:26.023 INFO:teuthology.orchestra.run.smithi174.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-11-08T19:21:26.023 INFO:teuthology.orchestra.run.smithi174.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-11-08T19:21:26.023 INFO:teuthology.orchestra.run.smithi174.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-11-08T19:21:26.023 INFO:teuthology.orchestra.run.smithi174.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-11-08T19:21:26.023 INFO:teuthology.orchestra.run.smithi174.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-11-08T19:21:26.024 INFO:teuthology.orchestra.run.smithi174.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-11-08T19:21:26.024 INFO:teuthology.orchestra.run.smithi174.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-11-08T19:21:26.024 INFO:teuthology.orchestra.run.smithi174.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-11-08T19:21:26.024 INFO:teuthology.orchestra.run.smithi174.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-11-08T19:21:26.024 INFO:teuthology.orchestra.run.smithi174.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-11-08T19:21:26.025 INFO:teuthology.orchestra.run.smithi174.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-11-08T19:21:26.025 INFO:teuthology.orchestra.run.smithi174.stdout:Enabling module streams: 2023-11-08T19:21:26.025 INFO:teuthology.orchestra.run.smithi174.stdout: postgresql 10 2023-11-08T19:21:26.025 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-08T19:21:26.025 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction Summary 2023-11-08T19:21:26.026 INFO:teuthology.orchestra.run.smithi174.stdout:=========================================================================================================================== 2023-11-08T19:21:26.026 INFO:teuthology.orchestra.run.smithi174.stdout:Install 128 Packages 2023-11-08T19:21:26.026 INFO:teuthology.orchestra.run.smithi174.stdout:Upgrade 3 Packages 2023-11-08T19:21:26.026 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-08T19:21:26.027 INFO:teuthology.orchestra.run.smithi174.stdout:Total download size: 209 M 2023-11-08T19:21:26.027 INFO:teuthology.orchestra.run.smithi174.stdout:Downloading Packages: 2023-11-08T19:21:26.348 INFO:teuthology.orchestra.run.smithi174.stdout:(1/131): ceph-18.0.0-7035.g7532011a.el8.x86_64. 44 kB/s | 6.4 kB 00:00 2023-11-08T19:21:26.557 INFO:teuthology.orchestra.run.smithi174.stdout:(2/131): ceph-fuse-18.0.0-7035.g7532011a.el8.x8 4.3 MB/s | 919 kB 00:00 2023-11-08T19:21:26.624 INFO:teuthology.orchestra.run.smithi174.stdout:(3/131): ceph-immutable-object-cache-18.0.0-703 2.5 MB/s | 171 kB 00:00 2023-11-08T19:21:26.924 INFO:teuthology.orchestra.run.smithi174.stdout:(4/131): ceph-base-18.0.0-7035.g7532011a.el8.x8 7.4 MB/s | 5.4 MB 00:00 2023-11-08T19:21:26.991 INFO:teuthology.orchestra.run.smithi174.stdout:(5/131): ceph-mds-18.0.0-7035.g7532011a.el8.x86 6.3 MB/s | 2.3 MB 00:00 2023-11-08T19:21:27.124 INFO:teuthology.orchestra.run.smithi174.stdout:(6/131): ceph-mgr-18.0.0-7035.g7532011a.el8.x86 8.1 MB/s | 1.6 MB 00:00 2023-11-08T19:21:27.933 INFO:teuthology.orchestra.run.smithi174.stdout:(7/131): ceph-mon-18.0.0-7035.g7532011a.el8.x86 4.9 MB/s | 4.6 MB 00:00 2023-11-08T19:21:28.292 INFO:teuthology.orchestra.run.smithi174.stdout:(8/131): ceph-common-18.0.0-7035.g7532011a.el8. 11 MB/s | 23 MB 00:02 2023-11-08T19:21:28.350 INFO:teuthology.orchestra.run.smithi174.stdout:(9/131): ceph-selinux-18.0.0-7035.g7532011a.el8 419 kB/s | 24 kB 00:00 2023-11-08T19:21:28.659 INFO:teuthology.orchestra.run.smithi174.stdout:(10/131): ceph-osd-18.0.0-7035.g7532011a.el8.x8 12 MB/s | 18 MB 00:01 2023-11-08T19:21:28.709 INFO:teuthology.orchestra.run.smithi174.stdout:(11/131): cephfs-mirror-18.0.0-7035.g7532011a.e 5.2 MB/s | 262 kB 00:00 2023-11-08T19:21:28.751 INFO:teuthology.orchestra.run.smithi174.stdout:(12/131): libcephfs-devel-18.0.0-7035.g7532011a 745 kB/s | 31 kB 00:00 2023-11-08T19:21:28.843 INFO:teuthology.orchestra.run.smithi174.stdout:(13/131): libcephfs2-18.0.0-7035.g7532011a.el8. 8.9 MB/s | 838 kB 00:00 2023-11-08T19:21:28.902 INFO:teuthology.orchestra.run.smithi174.stdout:(14/131): libcephsqlite-18.0.0-7035.g7532011a.e 3.1 MB/s | 182 kB 00:00 2023-11-08T19:21:28.961 INFO:teuthology.orchestra.run.smithi174.stdout:(15/131): librados-devel-18.0.0-7035.g7532011a. 2.3 MB/s | 139 kB 00:00 2023-11-08T19:21:29.036 INFO:teuthology.orchestra.run.smithi174.stdout:(16/131): libradosstriper1-18.0.0-7035.g7532011 6.8 MB/s | 518 kB 00:00 2023-11-08T19:21:29.544 INFO:teuthology.orchestra.run.smithi174.stdout:(17/131): librgw2-18.0.0-7035.g7532011a.el8.x86 14 MB/s | 6.9 MB 00:00 2023-11-08T19:21:29.595 INFO:teuthology.orchestra.run.smithi174.stdout:(18/131): python3-ceph-argparse-18.0.0-7035.g75 897 kB/s | 45 kB 00:00 2023-11-08T19:21:29.653 INFO:teuthology.orchestra.run.smithi174.stdout:(19/131): python3-ceph-common-18.0.0-7035.g7532 2.1 MB/s | 126 kB 00:00 2023-11-08T19:21:29.720 INFO:teuthology.orchestra.run.smithi174.stdout:(20/131): python3-cephfs-18.0.0-7035.g7532011a. 2.9 MB/s | 197 kB 00:00 2023-11-08T19:21:29.820 INFO:teuthology.orchestra.run.smithi174.stdout:(21/131): python3-rados-18.0.0-7035.g7532011a.e 3.6 MB/s | 371 kB 00:00 2023-11-08T19:21:29.879 INFO:teuthology.orchestra.run.smithi174.stdout:(22/131): python3-rbd-18.0.0-7035.g7532011a.el8 6.1 MB/s | 364 kB 00:00 2023-11-08T19:21:29.946 INFO:teuthology.orchestra.run.smithi174.stdout:(23/131): python3-rgw-18.0.0-7035.g7532011a.el8 1.6 MB/s | 107 kB 00:00 2023-11-08T19:21:30.013 INFO:teuthology.orchestra.run.smithi174.stdout:(24/131): rbd-fuse-18.0.0-7035.g7532011a.el8.x8 1.4 MB/s | 94 kB 00:00 2023-11-08T19:21:30.338 INFO:teuthology.orchestra.run.smithi174.stdout:(25/131): rbd-mirror-18.0.0-7035.g7532011a.el8. 11 MB/s | 3.6 MB 00:00 2023-11-08T19:21:30.397 INFO:teuthology.orchestra.run.smithi174.stdout:(26/131): rbd-nbd-18.0.0-7035.g7532011a.el8.x86 3.0 MB/s | 181 kB 00:00 2023-11-08T19:21:30.464 INFO:teuthology.orchestra.run.smithi174.stdout:(27/131): ceph-grafana-dashboards-18.0.0-7035.g 347 kB/s | 23 kB 00:00 2023-11-08T19:21:30.531 INFO:teuthology.orchestra.run.smithi174.stdout:(28/131): ceph-mgr-cephadm-18.0.0-7035.g7532011 1.9 MB/s | 132 kB 00:00 2023-11-08T19:21:30.591 INFO:teuthology.orchestra.run.smithi143.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 62 MB/s | 53 MB 00:00 2023-11-08T19:21:30.723 INFO:teuthology.orchestra.run.smithi174.stdout:(29/131): ceph-mgr-dashboard-18.0.0-7035.g75320 9.2 MB/s | 1.8 MB 00:00 2023-11-08T19:21:31.348 INFO:teuthology.orchestra.run.smithi174.stdout:(30/131): ceph-mgr-diskprediction-local-18.0.0- 12 MB/s | 7.4 MB 00:00 2023-11-08T19:21:31.536 INFO:teuthology.orchestra.run.smithi174.stdout:(31/131): ceph-radosgw-18.0.0-7035.g7532011a.el 3.9 MB/s | 14 MB 00:03 2023-11-08T19:21:31.565 INFO:teuthology.orchestra.run.smithi174.stdout:(32/131): ceph-mgr-modules-core-18.0.0-7035.g75 1.1 MB/s | 242 kB 00:00 2023-11-08T19:21:31.590 INFO:teuthology.orchestra.run.smithi174.stdout:(33/131): ceph-mgr-rook-18.0.0-7035.g7532011a.e 885 kB/s | 48 kB 00:00 2023-11-08T19:21:31.616 INFO:teuthology.orchestra.run.smithi174.stdout:(34/131): ceph-prometheus-alerts-18.0.0-7035.g7 298 kB/s | 15 kB 00:00 2023-11-08T19:21:31.641 INFO:teuthology.orchestra.run.smithi174.stdout:(35/131): ceph-volume-18.0.0-7035.g7532011a.el8 5.1 MB/s | 261 kB 00:00 2023-11-08T19:21:31.666 INFO:teuthology.orchestra.run.smithi174.stdout:(36/131): cephadm-18.0.0-7035.g7532011a.el8.noa 3.7 MB/s | 191 kB 00:00 2023-11-08T19:21:31.691 INFO:teuthology.orchestra.run.smithi174.stdout:(37/131): cephfs-top-18.0.0-7035.g7532011a.el8. 360 kB/s | 18 kB 00:00 2023-11-08T19:21:32.450 INFO:teuthology.orchestra.run.smithi174.stdout:(38/131): ceph-test-18.0.0-7035.g7532011a.el8.x 12 MB/s | 49 MB 00:04 2023-11-08T19:21:33.209 INFO:teuthology.orchestra.run.smithi174.stdout:(39/131): libunwind-1.3.1-3.el8.x86_64.rpm 99 kB/s | 75 kB 00:00 2023-11-08T19:21:34.118 INFO:teuthology.orchestra.run.smithi174.stdout:(40/131): liboath-2.6.2-3.el8.x86_64.rpm 24 kB/s | 59 kB 00:02 2023-11-08T19:21:35.851 INFO:teuthology.orchestra.run.smithi174.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 25 kB/s | 44 kB 00:01 2023-11-08T19:21:37.868 INFO:teuthology.orchestra.run.smithi174.stdout:(42/131): gperftools-libs-2.7-9.el8.x86_64.rpm 49 kB/s | 306 kB 00:06 2023-11-08T19:21:38.543 INFO:teuthology.orchestra.run.smithi174.stdout:(43/131): python3-cachetools-3.1.1-4.el8.noarch 49 kB/s | 33 kB 00:00 2023-11-08T19:21:38.923 INFO:teuthology.orchestra.run.smithi174.stdout:(44/131): python3-asyncssh-2.7.0-2.el8.noarch.r 83 kB/s | 476 kB 00:05 2023-11-08T19:21:39.177 INFO:teuthology.orchestra.run.smithi174.stdout:(45/131): python3-certifi-2018.10.15-7.el8.noar 26 kB/s | 16 kB 00:00 2023-11-08T19:21:39.485 INFO:teuthology.orchestra.run.smithi174.stdout:(46/131): python3-beautifulsoup4-4.6.3-2.el8.1. 51 kB/s | 185 kB 00:03 2023-11-08T19:21:40.419 INFO:teuthology.orchestra.run.smithi174.stdout:(47/131): python3-google-auth-1.1.1-10.el8.noar 87 kB/s | 81 kB 00:00 2023-11-08T19:21:40.669 INFO:teuthology.orchestra.run.smithi174.stdout:(48/131): python3-cheroot-8.5.2-1.el8.noarch.rp 99 kB/s | 173 kB 00:01 2023-11-08T19:21:40.869 INFO:teuthology.orchestra.run.smithi174.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 25 kB/s | 11 kB 00:00 2023-11-08T19:21:41.153 INFO:teuthology.orchestra.run.smithi174.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 36 kB/s | 18 kB 00:00 2023-11-08T19:21:41.906 INFO:teuthology.orchestra.run.smithi143.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-11-08T19:21:41.907 INFO:teuthology.orchestra.run.smithi143.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-11-08T19:21:41.922 INFO:teuthology.orchestra.run.smithi143.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-11-08T19:21:41.923 INFO:teuthology.orchestra.run.smithi143.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-11-08T19:21:41.924 INFO:teuthology.orchestra.run.smithi143.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-11-08T19:21:41.925 INFO:teuthology.orchestra.run.smithi143.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-11-08T19:21:41.928 INFO:teuthology.orchestra.run.smithi143.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-11-08T19:21:41.931 INFO:teuthology.orchestra.run.smithi143.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-11-08T19:21:41.932 INFO:teuthology.orchestra.run.smithi143.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-11-08T19:21:42.020 INFO:teuthology.orchestra.run.smithi174.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 56 kB/s | 49 kB 00:00 2023-11-08T19:21:42.065 INFO:teuthology.orchestra.run.smithi143.stdout:Dependencies resolved. 2023-11-08T19:21:42.077 INFO:teuthology.orchestra.run.smithi143.stdout:=========================================================================================================================== 2023-11-08T19:21:42.077 INFO:teuthology.orchestra.run.smithi143.stdout: Package Arch Version Repository Size 2023-11-08T19:21:42.078 INFO:teuthology.orchestra.run.smithi143.stdout:=========================================================================================================================== 2023-11-08T19:21:42.078 INFO:teuthology.orchestra.run.smithi143.stdout:Installing: 2023-11-08T19:21:42.078 INFO:teuthology.orchestra.run.smithi143.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-11-08T19:21:42.078 INFO:teuthology.orchestra.run.smithi143.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-11-08T19:21:42.079 INFO:teuthology.orchestra.run.smithi143.stdout: ceph x86_64 2:18.0.0-7035.g7532011a.el8 ceph 6.4 k 2023-11-08T19:21:42.079 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-base x86_64 2:18.0.0-7035.g7532011a.el8 ceph 5.4 M 2023-11-08T19:21:42.079 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-fuse x86_64 2:18.0.0-7035.g7532011a.el8 ceph 919 k 2023-11-08T19:21:42.079 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-7035.g7532011a.el8 ceph 171 k 2023-11-08T19:21:42.079 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr x86_64 2:18.0.0-7035.g7532011a.el8 ceph 1.6 M 2023-11-08T19:21:42.080 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr-cephadm noarch 2:18.0.0-7035.g7532011a.el8 ceph-noarch 132 k 2023-11-08T19:21:42.080 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr-dashboard noarch 2:18.0.0-7035.g7532011a.el8 ceph-noarch 1.8 M 2023-11-08T19:21:42.080 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-7035.g7532011a.el8 ceph-noarch 7.4 M 2023-11-08T19:21:42.080 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr-rook noarch 2:18.0.0-7035.g7532011a.el8 ceph-noarch 48 k 2023-11-08T19:21:42.081 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-radosgw x86_64 2:18.0.0-7035.g7532011a.el8 ceph 14 M 2023-11-08T19:21:42.081 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-test x86_64 2:18.0.0-7035.g7532011a.el8 ceph 49 M 2023-11-08T19:21:42.081 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-volume noarch 2:18.0.0-7035.g7532011a.el8 ceph-noarch 261 k 2023-11-08T19:21:42.081 INFO:teuthology.orchestra.run.smithi143.stdout: cephadm noarch 2:18.0.0-7035.g7532011a.el8 ceph-noarch 191 k 2023-11-08T19:21:42.081 INFO:teuthology.orchestra.run.smithi143.stdout: cephfs-mirror x86_64 2:18.0.0-7035.g7532011a.el8 ceph 262 k 2023-11-08T19:21:42.082 INFO:teuthology.orchestra.run.smithi143.stdout: cephfs-top noarch 2:18.0.0-7035.g7532011a.el8 ceph-noarch 18 k 2023-11-08T19:21:42.082 INFO:teuthology.orchestra.run.smithi143.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-11-08T19:21:42.082 INFO:teuthology.orchestra.run.smithi143.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-11-08T19:21:42.082 INFO:teuthology.orchestra.run.smithi143.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-11-08T19:21:42.082 INFO:teuthology.orchestra.run.smithi143.stdout: libcephfs-devel x86_64 2:18.0.0-7035.g7532011a.el8 ceph 31 k 2023-11-08T19:21:42.083 INFO:teuthology.orchestra.run.smithi143.stdout: libcephfs2 x86_64 2:18.0.0-7035.g7532011a.el8 ceph 838 k 2023-11-08T19:21:42.083 INFO:teuthology.orchestra.run.smithi143.stdout: librados-devel x86_64 2:18.0.0-7035.g7532011a.el8 ceph 139 k 2023-11-08T19:21:42.083 INFO:teuthology.orchestra.run.smithi143.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-11-08T19:21:42.083 INFO:teuthology.orchestra.run.smithi143.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-11-08T19:21:42.083 INFO:teuthology.orchestra.run.smithi143.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-11-08T19:21:42.084 INFO:teuthology.orchestra.run.smithi143.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-11-08T19:21:42.084 INFO:teuthology.orchestra.run.smithi143.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-11-08T19:21:42.084 INFO:teuthology.orchestra.run.smithi143.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-11-08T19:21:42.084 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cephfs x86_64 2:18.0.0-7035.g7532011a.el8 ceph 197 k 2023-11-08T19:21:42.084 INFO:teuthology.orchestra.run.smithi143.stdout: python3-rados x86_64 2:18.0.0-7035.g7532011a.el8 ceph 371 k 2023-11-08T19:21:42.085 INFO:teuthology.orchestra.run.smithi143.stdout: python3-rbd x86_64 2:18.0.0-7035.g7532011a.el8 ceph 364 k 2023-11-08T19:21:42.085 INFO:teuthology.orchestra.run.smithi143.stdout: python3-rgw x86_64 2:18.0.0-7035.g7532011a.el8 ceph 107 k 2023-11-08T19:21:42.085 INFO:teuthology.orchestra.run.smithi143.stdout: rbd-fuse x86_64 2:18.0.0-7035.g7532011a.el8 ceph 94 k 2023-11-08T19:21:42.085 INFO:teuthology.orchestra.run.smithi143.stdout: rbd-mirror x86_64 2:18.0.0-7035.g7532011a.el8 ceph 3.6 M 2023-11-08T19:21:42.085 INFO:teuthology.orchestra.run.smithi143.stdout: rbd-nbd x86_64 2:18.0.0-7035.g7532011a.el8 ceph 181 k 2023-11-08T19:21:42.086 INFO:teuthology.orchestra.run.smithi143.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-11-08T19:21:42.086 INFO:teuthology.orchestra.run.smithi143.stdout:Upgrading: 2023-11-08T19:21:42.086 INFO:teuthology.orchestra.run.smithi143.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-11-08T19:21:42.086 INFO:teuthology.orchestra.run.smithi143.stdout: librados2 x86_64 2:18.0.0-7035.g7532011a.el8 ceph 3.8 M 2023-11-08T19:21:42.086 INFO:teuthology.orchestra.run.smithi143.stdout: librbd1 x86_64 2:18.0.0-7035.g7532011a.el8 ceph 3.7 M 2023-11-08T19:21:42.087 INFO:teuthology.orchestra.run.smithi143.stdout:Installing dependencies: 2023-11-08T19:21:42.087 INFO:teuthology.orchestra.run.smithi143.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-11-08T19:21:42.087 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-common x86_64 2:18.0.0-7035.g7532011a.el8 ceph 23 M 2023-11-08T19:21:42.087 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-grafana-dashboards noarch 2:18.0.0-7035.g7532011a.el8 ceph-noarch 23 k 2023-11-08T19:21:42.087 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mds x86_64 2:18.0.0-7035.g7532011a.el8 ceph 2.3 M 2023-11-08T19:21:42.088 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr-modules-core noarch 2:18.0.0-7035.g7532011a.el8 ceph-noarch 242 k 2023-11-08T19:21:42.088 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mon x86_64 2:18.0.0-7035.g7532011a.el8 ceph 4.6 M 2023-11-08T19:21:42.088 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-osd x86_64 2:18.0.0-7035.g7532011a.el8 ceph 18 M 2023-11-08T19:21:42.088 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-prometheus-alerts noarch 2:18.0.0-7035.g7532011a.el8 ceph-noarch 15 k 2023-11-08T19:21:42.088 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-selinux x86_64 2:18.0.0-7035.g7532011a.el8 ceph 24 k 2023-11-08T19:21:42.089 INFO:teuthology.orchestra.run.smithi143.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-11-08T19:21:42.089 INFO:teuthology.orchestra.run.smithi143.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-11-08T19:21:42.089 INFO:teuthology.orchestra.run.smithi143.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-11-08T19:21:42.089 INFO:teuthology.orchestra.run.smithi143.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-11-08T19:21:42.089 INFO:teuthology.orchestra.run.smithi143.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-11-08T19:21:42.090 INFO:teuthology.orchestra.run.smithi143.stdout: libcephsqlite x86_64 2:18.0.0-7035.g7532011a.el8 ceph 182 k 2023-11-08T19:21:42.090 INFO:teuthology.orchestra.run.smithi143.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-11-08T19:21:42.090 INFO:teuthology.orchestra.run.smithi143.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-11-08T19:21:42.090 INFO:teuthology.orchestra.run.smithi143.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-11-08T19:21:42.090 INFO:teuthology.orchestra.run.smithi143.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-11-08T19:21:42.091 INFO:teuthology.orchestra.run.smithi143.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-11-08T19:21:42.091 INFO:teuthology.orchestra.run.smithi143.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-11-08T19:21:42.091 INFO:teuthology.orchestra.run.smithi143.stdout: libradosstriper1 x86_64 2:18.0.0-7035.g7532011a.el8 ceph 518 k 2023-11-08T19:21:42.091 INFO:teuthology.orchestra.run.smithi143.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-11-08T19:21:42.091 INFO:teuthology.orchestra.run.smithi143.stdout: librgw2 x86_64 2:18.0.0-7035.g7532011a.el8 ceph 6.9 M 2023-11-08T19:21:42.092 INFO:teuthology.orchestra.run.smithi143.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-11-08T19:21:42.092 INFO:teuthology.orchestra.run.smithi143.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-11-08T19:21:42.092 INFO:teuthology.orchestra.run.smithi143.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-11-08T19:21:42.092 INFO:teuthology.orchestra.run.smithi143.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-11-08T19:21:42.092 INFO:teuthology.orchestra.run.smithi143.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-11-08T19:21:42.093 INFO:teuthology.orchestra.run.smithi143.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-11-08T19:21:42.093 INFO:teuthology.orchestra.run.smithi143.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-11-08T19:21:42.093 INFO:teuthology.orchestra.run.smithi143.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-11-08T19:21:42.093 INFO:teuthology.orchestra.run.smithi143.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-11-08T19:21:42.093 INFO:teuthology.orchestra.run.smithi143.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-11-08T19:21:42.094 INFO:teuthology.orchestra.run.smithi143.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-11-08T19:21:42.094 INFO:teuthology.orchestra.run.smithi143.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-11-08T19:21:42.094 INFO:teuthology.orchestra.run.smithi143.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-11-08T19:21:42.094 INFO:teuthology.orchestra.run.smithi143.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-11-08T19:21:42.094 INFO:teuthology.orchestra.run.smithi143.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-11-08T19:21:42.095 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-11-08T19:21:42.095 INFO:teuthology.orchestra.run.smithi143.stdout: python3-ceph-argparse x86_64 2:18.0.0-7035.g7532011a.el8 ceph 45 k 2023-11-08T19:21:42.095 INFO:teuthology.orchestra.run.smithi143.stdout: python3-ceph-common x86_64 2:18.0.0-7035.g7532011a.el8 ceph 126 k 2023-11-08T19:21:42.095 INFO:teuthology.orchestra.run.smithi143.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-11-08T19:21:42.095 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-11-08T19:21:42.096 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-11-08T19:21:42.096 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-11-08T19:21:42.096 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-11-08T19:21:42.096 INFO:teuthology.orchestra.run.smithi143.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-11-08T19:21:42.096 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-11-08T19:21:42.097 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-11-08T19:21:42.097 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-11-08T19:21:42.097 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-11-08T19:21:42.097 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-11-08T19:21:42.097 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-11-08T19:21:42.098 INFO:teuthology.orchestra.run.smithi143.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-11-08T19:21:42.098 INFO:teuthology.orchestra.run.smithi143.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-11-08T19:21:42.098 INFO:teuthology.orchestra.run.smithi143.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-11-08T19:21:42.098 INFO:teuthology.orchestra.run.smithi143.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-11-08T19:21:42.098 INFO:teuthology.orchestra.run.smithi143.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-11-08T19:21:42.099 INFO:teuthology.orchestra.run.smithi143.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-11-08T19:21:42.099 INFO:teuthology.orchestra.run.smithi143.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-11-08T19:21:42.099 INFO:teuthology.orchestra.run.smithi143.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-11-08T19:21:42.099 INFO:teuthology.orchestra.run.smithi143.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-11-08T19:21:42.099 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-11-08T19:21:42.100 INFO:teuthology.orchestra.run.smithi143.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-11-08T19:21:42.100 INFO:teuthology.orchestra.run.smithi143.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-11-08T19:21:42.100 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-11-08T19:21:42.100 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-11-08T19:21:42.101 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-11-08T19:21:42.101 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-11-08T19:21:42.101 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-11-08T19:21:42.101 INFO:teuthology.orchestra.run.smithi143.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-11-08T19:21:42.101 INFO:teuthology.orchestra.run.smithi143.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-11-08T19:21:42.101 INFO:teuthology.orchestra.run.smithi143.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-11-08T19:21:42.102 INFO:teuthology.orchestra.run.smithi143.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-11-08T19:21:42.102 INFO:teuthology.orchestra.run.smithi143.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-11-08T19:21:42.102 INFO:teuthology.orchestra.run.smithi143.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-11-08T19:21:42.102 INFO:teuthology.orchestra.run.smithi143.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-11-08T19:21:42.102 INFO:teuthology.orchestra.run.smithi143.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-11-08T19:21:42.103 INFO:teuthology.orchestra.run.smithi143.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-11-08T19:21:42.103 INFO:teuthology.orchestra.run.smithi143.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-11-08T19:21:42.103 INFO:teuthology.orchestra.run.smithi143.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-11-08T19:21:42.103 INFO:teuthology.orchestra.run.smithi143.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-11-08T19:21:42.103 INFO:teuthology.orchestra.run.smithi143.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-11-08T19:21:42.104 INFO:teuthology.orchestra.run.smithi143.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-11-08T19:21:42.104 INFO:teuthology.orchestra.run.smithi143.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-11-08T19:21:42.104 INFO:teuthology.orchestra.run.smithi143.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-11-08T19:21:42.104 INFO:teuthology.orchestra.run.smithi143.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-11-08T19:21:42.104 INFO:teuthology.orchestra.run.smithi143.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-11-08T19:21:42.105 INFO:teuthology.orchestra.run.smithi143.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-11-08T19:21:42.105 INFO:teuthology.orchestra.run.smithi143.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-11-08T19:21:42.105 INFO:teuthology.orchestra.run.smithi143.stdout:Enabling module streams: 2023-11-08T19:21:42.105 INFO:teuthology.orchestra.run.smithi143.stdout: postgresql 10 2023-11-08T19:21:42.105 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:21:42.106 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction Summary 2023-11-08T19:21:42.106 INFO:teuthology.orchestra.run.smithi143.stdout:=========================================================================================================================== 2023-11-08T19:21:42.106 INFO:teuthology.orchestra.run.smithi143.stdout:Install 128 Packages 2023-11-08T19:21:42.106 INFO:teuthology.orchestra.run.smithi143.stdout:Upgrade 3 Packages 2023-11-08T19:21:42.106 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:21:42.107 INFO:teuthology.orchestra.run.smithi143.stdout:Total download size: 209 M 2023-11-08T19:21:42.107 INFO:teuthology.orchestra.run.smithi143.stdout:Downloading Packages: 2023-11-08T19:21:42.258 INFO:teuthology.orchestra.run.smithi143.stdout:(1/131): ceph-18.0.0-7035.g7532011a.el8.x86_64. 48 kB/s | 6.4 kB 00:00 2023-11-08T19:21:42.441 INFO:teuthology.orchestra.run.smithi143.stdout:(2/131): ceph-fuse-18.0.0-7035.g7532011a.el8.x8 4.9 MB/s | 919 kB 00:00 2023-11-08T19:21:42.500 INFO:teuthology.orchestra.run.smithi143.stdout:(3/131): ceph-immutable-object-cache-18.0.0-703 2.9 MB/s | 171 kB 00:00 2023-11-08T19:21:42.792 INFO:teuthology.orchestra.run.smithi143.stdout:(4/131): ceph-mds-18.0.0-7035.g7532011a.el8.x86 7.9 MB/s | 2.3 MB 00:00 2023-11-08T19:21:42.936 INFO:teuthology.orchestra.run.smithi174.stdout:(52/131): python3-more-itertools-7.2.0-3.el8.no 64 kB/s | 59 kB 00:00 2023-11-08T19:21:43.000 INFO:teuthology.orchestra.run.smithi143.stdout:(5/131): ceph-mgr-18.0.0-7035.g7532011a.el8.x86 7.8 MB/s | 1.6 MB 00:00 2023-11-08T19:21:43.478 INFO:teuthology.orchestra.run.smithi174.stdout:(53/131): python3-cherrypy-18.4.0-1.el8.noarch. 89 kB/s | 384 kB 00:04 2023-11-08T19:21:43.592 INFO:teuthology.orchestra.run.smithi143.stdout:(6/131): ceph-mon-18.0.0-7035.g7532011a.el8.x86 7.8 MB/s | 4.6 MB 00:00 2023-11-08T19:21:43.659 INFO:teuthology.orchestra.run.smithi143.stdout:(7/131): ceph-base-18.0.0-7035.g7532011a.el8.x8 3.5 MB/s | 5.4 MB 00:01 2023-11-08T19:21:43.712 INFO:teuthology.orchestra.run.smithi174.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 78 kB/s | 60 kB 00:00 2023-11-08T19:21:43.951 INFO:teuthology.orchestra.run.smithi143.stdout:(8/131): ceph-common-18.0.0-7035.g7532011a.el8. 12 MB/s | 23 MB 00:01 2023-11-08T19:21:43.970 INFO:teuthology.orchestra.run.smithi174.stdout:(55/131): python3-portend-2.6-1.el8.noarch.rpm 60 kB/s | 16 kB 00:00 2023-11-08T19:21:43.993 INFO:teuthology.orchestra.run.smithi143.stdout:(9/131): ceph-selinux-18.0.0-7035.g7532011a.el8 588 kB/s | 24 kB 00:00 2023-11-08T19:21:44.479 INFO:teuthology.orchestra.run.smithi174.stdout:(56/131): python3-repoze-lru-0.7-6.el8.noarch.r 66 kB/s | 34 kB 00:00 2023-11-08T19:21:45.010 INFO:teuthology.orchestra.run.smithi143.stdout:(10/131): ceph-osd-18.0.0-7035.g7532011a.el8.x8 13 MB/s | 18 MB 00:01 2023-11-08T19:21:45.077 INFO:teuthology.orchestra.run.smithi143.stdout:(11/131): cephfs-mirror-18.0.0-7035.g7532011a.e 3.9 MB/s | 262 kB 00:00 2023-11-08T19:21:45.165 INFO:teuthology.orchestra.run.smithi143.stdout:(12/131): libcephfs-devel-18.0.0-7035.g7532011a 353 kB/s | 31 kB 00:00 2023-11-08T19:21:45.270 INFO:teuthology.orchestra.run.smithi143.stdout:(13/131): libcephfs2-18.0.0-7035.g7532011a.el8. 7.9 MB/s | 838 kB 00:00 2023-11-08T19:21:45.337 INFO:teuthology.orchestra.run.smithi143.stdout:(14/131): libcephsqlite-18.0.0-7035.g7532011a.e 2.7 MB/s | 182 kB 00:00 2023-11-08T19:21:45.403 INFO:teuthology.orchestra.run.smithi143.stdout:(15/131): librados-devel-18.0.0-7035.g7532011a. 2.0 MB/s | 139 kB 00:00 2023-11-08T19:21:45.487 INFO:teuthology.orchestra.run.smithi143.stdout:(16/131): libradosstriper1-18.0.0-7035.g7532011 6.1 MB/s | 518 kB 00:00 2023-11-08T19:21:46.121 INFO:teuthology.orchestra.run.smithi143.stdout:(17/131): librgw2-18.0.0-7035.g7532011a.el8.x86 11 MB/s | 6.9 MB 00:00 2023-11-08T19:21:46.254 INFO:teuthology.orchestra.run.smithi143.stdout:(18/131): ceph-radosgw-18.0.0-7035.g7532011a.el 5.4 MB/s | 14 MB 00:02 2023-11-08T19:21:46.280 INFO:teuthology.orchestra.run.smithi143.stdout:(19/131): python3-ceph-argparse-18.0.0-7035.g75 282 kB/s | 45 kB 00:00 2023-11-08T19:21:46.330 INFO:teuthology.orchestra.run.smithi143.stdout:(20/131): python3-cephfs-18.0.0-7035.g7532011a. 3.8 MB/s | 197 kB 00:00 2023-11-08T19:21:46.355 INFO:teuthology.orchestra.run.smithi143.stdout:(21/131): python3-ceph-common-18.0.0-7035.g7532 1.2 MB/s | 126 kB 00:00 2023-11-08T19:21:46.406 INFO:teuthology.orchestra.run.smithi143.stdout:(22/131): python3-rados-18.0.0-7035.g7532011a.e 4.8 MB/s | 371 kB 00:00 2023-11-08T19:21:46.464 INFO:teuthology.orchestra.run.smithi143.stdout:(23/131): python3-rgw-18.0.0-7035.g7532011a.el8 1.8 MB/s | 107 kB 00:00 2023-11-08T19:21:46.532 INFO:teuthology.orchestra.run.smithi143.stdout:(24/131): python3-rbd-18.0.0-7035.g7532011a.el8 2.0 MB/s | 364 kB 00:00 2023-11-08T19:21:46.557 INFO:teuthology.orchestra.run.smithi143.stdout:(25/131): rbd-fuse-18.0.0-7035.g7532011a.el8.x8 1.0 MB/s | 94 kB 00:00 2023-11-08T19:21:46.632 INFO:teuthology.orchestra.run.smithi143.stdout:(26/131): rbd-nbd-18.0.0-7035.g7532011a.el8.x86 2.4 MB/s | 181 kB 00:00 2023-11-08T19:21:46.691 INFO:teuthology.orchestra.run.smithi143.stdout:(27/131): ceph-grafana-dashboards-18.0.0-7035.g 397 kB/s | 23 kB 00:00 2023-11-08T19:21:46.746 INFO:teuthology.orchestra.run.smithi174.stdout:(57/131): python3-routes-2.4.1-12.el8.noarch.rp 87 kB/s | 196 kB 00:02 2023-11-08T19:21:46.757 INFO:teuthology.orchestra.run.smithi143.stdout:(28/131): ceph-mgr-cephadm-18.0.0-7035.g7532011 1.9 MB/s | 132 kB 00:00 2023-11-08T19:21:46.941 INFO:teuthology.orchestra.run.smithi143.stdout:(29/131): ceph-mgr-dashboard-18.0.0-7035.g75320 9.6 MB/s | 1.8 MB 00:00 2023-11-08T19:21:47.183 INFO:teuthology.orchestra.run.smithi143.stdout:(30/131): rbd-mirror-18.0.0-7035.g7532011a.el8. 5.5 MB/s | 3.6 MB 00:00 2023-11-08T19:21:47.300 INFO:teuthology.orchestra.run.smithi143.stdout:(31/131): ceph-mgr-modules-core-18.0.0-7035.g75 2.0 MB/s | 242 kB 00:00 2023-11-08T19:21:47.400 INFO:teuthology.orchestra.run.smithi143.stdout:(32/131): ceph-mgr-rook-18.0.0-7035.g7532011a.e 477 kB/s | 48 kB 00:00 2023-11-08T19:21:47.534 INFO:teuthology.orchestra.run.smithi143.stdout:(33/131): ceph-mgr-diskprediction-local-18.0.0- 12 MB/s | 7.4 MB 00:00 2023-11-08T19:21:47.559 INFO:teuthology.orchestra.run.smithi143.stdout:(34/131): ceph-prometheus-alerts-18.0.0-7035.g7 94 kB/s | 15 kB 00:00 2023-11-08T19:21:47.572 INFO:teuthology.orchestra.run.smithi174.stdout:(58/131): python3-rsa-4.9-2.el8.noarch.rpm 78 kB/s | 64 kB 00:00 2023-11-08T19:21:47.610 INFO:teuthology.orchestra.run.smithi143.stdout:(35/131): ceph-volume-18.0.0-7035.g7532011a.el8 3.4 MB/s | 261 kB 00:00 2023-11-08T19:21:47.643 INFO:teuthology.orchestra.run.smithi143.stdout:(36/131): cephadm-18.0.0-7035.g7532011a.el8.noa 2.2 MB/s | 191 kB 00:00 2023-11-08T19:21:47.669 INFO:teuthology.orchestra.run.smithi143.stdout:(37/131): cephfs-top-18.0.0-7035.g7532011a.el8. 308 kB/s | 18 kB 00:00 2023-11-08T19:21:48.128 INFO:teuthology.orchestra.run.smithi143.stdout:(38/131): ceph-test-18.0.0-7035.g7532011a.el8.x 12 MB/s | 49 MB 00:04 2023-11-08T19:21:48.129 INFO:teuthology.orchestra.run.smithi174.stdout:(59/131): python3-simplegeneric-0.8.1-17.el8.no 34 kB/s | 19 kB 00:00 2023-11-08T19:21:48.505 INFO:teuthology.orchestra.run.smithi174.stdout:(60/131): python3-singledispatch-3.4.0.3-18.el8 64 kB/s | 24 kB 00:00 2023-11-08T19:21:48.697 INFO:teuthology.orchestra.run.smithi174.stdout:(61/131): python3-pecan-1.3.2-9.el8.noarch.rpm 54 kB/s | 283 kB 00:05 2023-11-08T19:21:49.053 INFO:teuthology.orchestra.run.smithi143.stdout:(39/131): liboath-2.6.2-3.el8.x86_64.rpm 42 kB/s | 59 kB 00:01 2023-11-08T19:21:49.336 INFO:teuthology.orchestra.run.smithi143.stdout:(40/131): libunwind-1.3.1-3.el8.x86_64.rpm 62 kB/s | 75 kB 00:01 2023-11-08T19:21:49.363 INFO:teuthology.orchestra.run.smithi174.stdout:(62/131): python3-trustme-0.6.0-4.el8.noarch.rp 41 kB/s | 27 kB 00:00 2023-11-08T19:21:50.295 INFO:teuthology.orchestra.run.smithi143.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 45 kB/s | 44 kB 00:00 2023-11-08T19:21:50.781 INFO:teuthology.orchestra.run.smithi174.stdout:(63/131): python3-tempora-1.14.1-5.el8.noarch.r 13 kB/s | 29 kB 00:02 2023-11-08T19:21:50.954 INFO:teuthology.orchestra.run.smithi143.stdout:(42/131): python3-beautifulsoup4-4.6.3-2.el8.1. 281 kB/s | 185 kB 00:00 2023-11-08T19:21:51.062 INFO:teuthology.orchestra.run.smithi143.stdout:(43/131): python3-cachetools-3.1.1-4.el8.noarch 305 kB/s | 33 kB 00:00 2023-11-08T19:21:51.146 INFO:teuthology.orchestra.run.smithi143.stdout:(44/131): python3-certifi-2018.10.15-7.el8.noar 195 kB/s | 16 kB 00:00 2023-11-08T19:21:51.197 INFO:teuthology.orchestra.run.smithi143.stdout:(45/131): gperftools-libs-2.7-9.el8.x86_64.rpm 86 kB/s | 306 kB 00:03 2023-11-08T19:21:51.480 INFO:teuthology.orchestra.run.smithi143.stdout:(46/131): python3-cheroot-8.5.2-1.el8.noarch.rp 517 kB/s | 173 kB 00:00 2023-11-08T19:21:51.689 INFO:teuthology.orchestra.run.smithi174.stdout:(64/131): python3-waitress-1.4.3-1.el8.noarch.r 110 kB/s | 255 kB 00:02 2023-11-08T19:21:51.713 INFO:teuthology.orchestra.run.smithi143.stdout:(47/131): python3-google-auth-1.1.1-10.el8.noar 348 kB/s | 81 kB 00:00 2023-11-08T19:21:51.789 INFO:teuthology.orchestra.run.smithi143.stdout:(48/131): python3-jaraco-6.2-6.el8.noarch.rpm 148 kB/s | 11 kB 00:00 2023-11-08T19:21:51.981 INFO:teuthology.orchestra.run.smithi174.stdout:(65/131): python3-websocket-client-0.56.0-5.el8 210 kB/s | 61 kB 00:00 2023-11-08T19:21:52.289 INFO:teuthology.orchestra.run.smithi174.stdout:(66/131): python3-webtest-2.0.33-1.el8.noarch.r 277 kB/s | 85 kB 00:00 2023-11-08T19:21:52.386 INFO:teuthology.orchestra.run.smithi143.stdout:(49/131): python3-jaraco-functools-2.0-4.el8.no 29 kB/s | 18 kB 00:00 2023-11-08T19:21:52.906 INFO:teuthology.orchestra.run.smithi174.stdout:(67/131): python3-webob-1.8.5-1.el8.1.noarch.rp 118 kB/s | 251 kB 00:02 2023-11-08T19:21:52.957 INFO:teuthology.orchestra.run.smithi174.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 34 kB/s | 23 kB 00:00 2023-11-08T19:21:52.990 INFO:teuthology.orchestra.run.smithi174.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.1 MB/s | 70 kB 00:00 2023-11-08T19:21:53.049 INFO:teuthology.orchestra.run.smithi174.stdout:(70/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 803 kB/s | 47 kB 00:00 2023-11-08T19:21:53.082 INFO:teuthology.orchestra.run.smithi174.stdout:(71/131): libselinux-devel-2.9-5.el8.x86_64.rpm 5.9 MB/s | 200 kB 00:00 2023-11-08T19:21:53.124 INFO:teuthology.orchestra.run.smithi174.stdout:(72/131): python3-jwt-1.6.1-2.el8.noarch.rpm 1.0 MB/s | 43 kB 00:00 2023-11-08T19:21:53.166 INFO:teuthology.orchestra.run.smithi174.stdout:(73/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 5.2 MB/s | 220 kB 00:00 2023-11-08T19:21:53.225 INFO:teuthology.orchestra.run.smithi174.stdout:(74/131): keyutils-libs-devel-1.5.10-9.el8.x86_ 823 kB/s | 48 kB 00:00 2023-11-08T19:21:53.267 INFO:teuthology.orchestra.run.smithi174.stdout:(75/131): libsepol-devel-2.9-3.el8.x86_64.rpm 2.0 MB/s | 87 kB 00:00 2023-11-08T19:21:53.309 INFO:teuthology.orchestra.run.smithi174.stdout:(76/131): python3-requests-oauthlib-1.0.0-1.el8 1.0 MB/s | 43 kB 00:00 2023-11-08T19:21:53.351 INFO:teuthology.orchestra.run.smithi174.stdout:(77/131): python3-pycparser-2.14-14.el8.noarch. 2.6 MB/s | 109 kB 00:00 2023-11-08T19:21:53.393 INFO:teuthology.orchestra.run.smithi174.stdout:(78/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 5.4 MB/s | 229 kB 00:00 2023-11-08T19:21:53.443 INFO:teuthology.orchestra.run.smithi174.stdout:(79/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 11 MB/s | 560 kB 00:00 2023-11-08T19:21:53.485 INFO:teuthology.orchestra.run.smithi174.stdout:(80/131): libaio-devel-0.3.112-1.el8.x86_64.rpm 449 kB/s | 19 kB 00:00 2023-11-08T19:21:53.527 INFO:teuthology.orchestra.run.smithi174.stdout:(81/131): python3-oauthlib-2.1.0-1.el8.noarch.r 3.6 MB/s | 155 kB 00:00 2023-11-08T19:21:53.569 INFO:teuthology.orchestra.run.smithi174.stdout:(82/131): python3-cryptography-3.2.1-5.el8.x86_ 13 MB/s | 559 kB 00:00 2023-11-08T19:21:53.602 INFO:teuthology.orchestra.run.smithi174.stdout:(83/131): libverto-devel-0.3.0-5.el8.x86_64.rpm 537 kB/s | 18 kB 00:00 2023-11-08T19:21:53.661 INFO:teuthology.orchestra.run.smithi174.stdout:(84/131): pcre2-devel-10.32-2.el8.x86_64.rpm 10 MB/s | 605 kB 00:00 2023-11-08T19:21:53.703 INFO:teuthology.orchestra.run.smithi174.stdout:(85/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 5.6 MB/s | 238 kB 00:00 2023-11-08T19:21:53.778 INFO:teuthology.orchestra.run.smithi174.stdout:(86/131): openssl-devel-1.1.1k-7.el8_6.x86_64.r 31 MB/s | 2.3 MB 00:00 2023-11-08T19:21:53.820 INFO:teuthology.orchestra.run.smithi174.stdout:(87/131): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 1.7 MB/s | 72 kB 00:00 2023-11-08T19:21:53.839 INFO:teuthology.orchestra.run.smithi143.stdout:(50/131): python3-asyncssh-2.7.0-2.el8.noarch.r 99 kB/s | 476 kB 00:04 2023-11-08T19:21:53.862 INFO:teuthology.orchestra.run.smithi174.stdout:(88/131): fuse-2.9.7-15.el8.x86_64.rpm 1.9 MB/s | 83 kB 00:00 2023-11-08T19:21:53.864 INFO:teuthology.orchestra.run.smithi143.stdout:(51/131): python3-cherrypy-18.4.0-1.el8.noarch. 144 kB/s | 384 kB 00:02 2023-11-08T19:21:53.896 INFO:teuthology.orchestra.run.smithi174.stdout:(89/131): elfutils-libelf-devel-0.186-1.el8.x86 1.8 MB/s | 60 kB 00:00 2023-11-08T19:21:53.938 INFO:teuthology.orchestra.run.smithi174.stdout:(90/131): libuuid-devel-2.32.1-35.el8.x86_64.rp 2.3 MB/s | 98 kB 00:00 2023-11-08T19:21:53.971 INFO:teuthology.orchestra.run.smithi174.stdout:(91/131): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 1.7 MB/s | 58 kB 00:00 2023-11-08T19:21:54.013 INFO:teuthology.orchestra.run.smithi174.stdout:(92/131): libcom_err-devel-1.45.6-4.el8.x86_64. 927 kB/s | 39 kB 00:00 2023-11-08T19:21:54.347 INFO:teuthology.orchestra.run.smithi174.stdout:(93/131): libgfortran-8.5.0-10.1.el8_6.x86_64.r 1.9 MB/s | 644 kB 00:00 2023-11-08T19:21:54.389 INFO:teuthology.orchestra.run.smithi174.stdout:(94/131): libquadmath-8.5.0-10.1.el8_6.x86_64.r 4.0 MB/s | 171 kB 00:00 2023-11-08T19:21:54.431 INFO:teuthology.orchestra.run.smithi174.stdout:(95/131): NetworkManager-initscripts-updown-1.3 3.2 MB/s | 138 kB 00:00 2023-11-08T19:21:54.665 INFO:teuthology.orchestra.run.smithi143.stdout:(52/131): python3-logutils-0.3.5-11.el8.noarch. 59 kB/s | 49 kB 00:00 2023-11-08T19:21:54.715 INFO:teuthology.orchestra.run.smithi174.stdout:(96/131): python3-scipy-1.0.0-21.module+el8.5.0 48 MB/s | 14 MB 00:00 2023-11-08T19:21:54.757 INFO:teuthology.orchestra.run.smithi174.stdout:(97/131): python3-pyasn1-modules-0.3.7-6.el8.no 2.6 MB/s | 110 kB 00:00 2023-11-08T19:21:54.799 INFO:teuthology.orchestra.run.smithi174.stdout:(98/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.3 MB/s | 54 kB 00:00 2023-11-08T19:21:54.841 INFO:teuthology.orchestra.run.smithi174.stdout:(99/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 6.1 MB/s | 259 kB 00:00 2023-11-08T19:21:54.883 INFO:teuthology.orchestra.run.smithi174.stdout:(100/131): socat-1.7.4.1-1.el8.x86_64.rpm 7.6 MB/s | 323 kB 00:00 2023-11-08T19:21:54.924 INFO:teuthology.orchestra.run.smithi174.stdout:(101/131): python3-markupsafe-0.23-19.el8.x86_6 937 kB/s | 39 kB 00:00 2023-11-08T19:21:55.000 INFO:teuthology.orchestra.run.smithi174.stdout:(102/131): bison-3.0.4-10.el8.x86_64.rpm 8.9 MB/s | 688 kB 00:00 2023-11-08T19:21:55.042 INFO:teuthology.orchestra.run.smithi174.stdout:(103/131): python3-jinja2-2.10.1-3.el8.noarch.r 13 MB/s | 538 kB 00:00 2023-11-08T19:21:55.084 INFO:teuthology.orchestra.run.smithi174.stdout:(104/131): python3-mako-1.0.6-13.el8.noarch.rpm 3.7 MB/s | 157 kB 00:00 2023-11-08T19:21:55.126 INFO:teuthology.orchestra.run.smithi174.stdout:(105/131): oniguruma-6.8.2-2.el8.x86_64.rpm 4.4 MB/s | 187 kB 00:00 2023-11-08T19:21:55.176 INFO:teuthology.orchestra.run.smithi174.stdout:(106/131): automake-1.16.1-7.el8.noarch.rpm 14 MB/s | 713 kB 00:00 2023-11-08T19:21:55.293 INFO:teuthology.orchestra.run.smithi174.stdout:(107/131): python3-babel-2.5.1-7.el8.noarch.rpm 41 MB/s | 4.8 MB 00:00 2023-11-08T19:21:55.335 INFO:teuthology.orchestra.run.smithi174.stdout:(108/131): python3-jsonpatch-1.21-2.el8.noarch. 640 kB/s | 27 kB 00:00 2023-11-08T19:21:55.377 INFO:teuthology.orchestra.run.smithi174.stdout:(109/131): flex-2.6.1-9.el8.x86_64.rpm 7.5 MB/s | 320 kB 00:00 2023-11-08T19:21:55.435 INFO:teuthology.orchestra.run.smithi174.stdout:(110/131): python3-prettytable-0.7.2-14.el8.noa 751 kB/s | 44 kB 00:00 2023-11-08T19:21:55.490 INFO:teuthology.orchestra.run.smithi143.stdout:(53/131): python3-more-itertools-7.2.0-3.el8.no 36 kB/s | 59 kB 00:01 2023-11-08T19:21:55.544 INFO:teuthology.orchestra.run.smithi174.stdout:(111/131): python3-numpy-1.14.3-10.el8.x86_64.r 34 MB/s | 3.7 MB 00:00 2023-11-08T19:21:55.586 INFO:teuthology.orchestra.run.smithi174.stdout:(112/131): python3-werkzeug-0.12.2-4.el8.noarch 11 MB/s | 457 kB 00:00 2023-11-08T19:21:55.628 INFO:teuthology.orchestra.run.smithi174.stdout:(113/131): python3-numpy-f2py-1.14.3-10.el8.x86 5.3 MB/s | 225 kB 00:00 2023-11-08T19:21:55.670 INFO:teuthology.orchestra.run.smithi174.stdout:(114/131): uuid-1.6.2-43.el8.x86_64.rpm 1.5 MB/s | 64 kB 00:00 2023-11-08T19:21:55.712 INFO:teuthology.orchestra.run.smithi174.stdout:(115/131): python3-pyasn1-0.3.7-6.el8.noarch.rp 3.0 MB/s | 126 kB 00:00 2023-11-08T19:21:55.745 INFO:teuthology.orchestra.run.smithi174.stdout:(116/131): python3-jsonpointer-1.10-11.el8.noar 599 kB/s | 20 kB 00:00 2023-11-08T19:21:55.788 INFO:teuthology.orchestra.run.smithi174.stdout:(117/131): python3-pyOpenSSL-19.0.0-1.el8.noarc 2.4 MB/s | 103 kB 00:00 2023-11-08T19:21:55.840 INFO:teuthology.orchestra.run.smithi143.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 51 kB/s | 60 kB 00:01 2023-11-08T19:21:55.872 INFO:teuthology.orchestra.run.smithi174.stdout:(118/131): libtool-2.4.6-25.el8.x86_64.rpm 8.2 MB/s | 709 kB 00:00 2023-11-08T19:21:55.997 INFO:teuthology.orchestra.run.smithi174.stdout:(119/131): openblas-threads-0.3.15-3.el8.x86_64 39 MB/s | 4.9 MB 00:00 2023-11-08T19:21:56.038 INFO:teuthology.orchestra.run.smithi174.stdout:(120/131): libpq-13.5-1.el8.x86_64.rpm 4.7 MB/s | 198 kB 00:00 2023-11-08T19:21:56.164 INFO:teuthology.orchestra.run.smithi174.stdout:(121/131): openblas-0.3.15-3.el8.x86_64.rpm 38 MB/s | 4.7 MB 00:00 2023-11-08T19:21:56.214 INFO:teuthology.orchestra.run.smithi174.stdout:(122/131): jq-1.6-3.el8.x86_64.rpm 4.0 MB/s | 202 kB 00:00 2023-11-08T19:21:56.215 INFO:teuthology.orchestra.run.smithi143.stdout:(55/131): python3-portend-2.6-1.el8.noarch.rpm 41 kB/s | 16 kB 00:00 2023-11-08T19:21:56.256 INFO:teuthology.orchestra.run.smithi174.stdout:(123/131): librdkafka-0.11.4-3.el8.x86_64.rpm 8.3 MB/s | 354 kB 00:00 2023-11-08T19:21:56.390 INFO:teuthology.orchestra.run.smithi174.stdout:(124/131): postgresql-server-10.21-2.module+el8 38 MB/s | 5.1 MB 00:00 2023-11-08T19:21:56.449 INFO:teuthology.orchestra.run.smithi174.stdout:(125/131): postgresql-contrib-10.21-2.module+el 14 MB/s | 808 kB 00:00 2023-11-08T19:21:56.515 INFO:teuthology.orchestra.run.smithi174.stdout:(126/131): postgresql-10.21-2.module+el8.6.0+15 23 MB/s | 1.5 MB 00:00 2023-11-08T19:21:56.924 INFO:teuthology.orchestra.run.smithi174.stdout:(127/131): librados2-18.0.0-7035.g7532011a.el8. 9.2 MB/s | 3.8 MB 00:00 2023-11-08T19:21:57.057 INFO:teuthology.orchestra.run.smithi143.stdout:(56/131): python3-repoze-lru-0.7-6.el8.noarch.r 40 kB/s | 34 kB 00:00 2023-11-08T19:21:57.249 INFO:teuthology.orchestra.run.smithi174.stdout:(128/131): librbd1-18.0.0-7035.g7532011a.el8.x8 11 MB/s | 3.7 MB 00:00 2023-11-08T19:21:57.300 INFO:teuthology.orchestra.run.smithi174.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 16 MB/s | 798 kB 00:00 2023-11-08T19:22:01.091 INFO:teuthology.orchestra.run.smithi174.stdout:(130/131): python3-kubernetes-11.0.0-6.el8.noar 59 kB/s | 1.2 MB 00:20 2023-11-08T19:22:01.435 INFO:teuthology.orchestra.run.smithi143.stdout:(57/131): python3-pecan-1.3.2-9.el8.noarch.rpm 48 kB/s | 283 kB 00:05 2023-11-08T19:22:01.474 INFO:teuthology.orchestra.run.smithi143.stdout:(58/131): python3-kubernetes-11.0.0-6.el8.noarc 131 kB/s | 1.2 MB 00:09 2023-11-08T19:22:01.674 INFO:teuthology.orchestra.run.smithi143.stdout:(59/131): python3-simplegeneric-0.8.1-17.el8.no 93 kB/s | 19 kB 00:00 2023-11-08T19:22:02.099 INFO:teuthology.orchestra.run.smithi143.stdout:(60/131): python3-rsa-4.9-2.el8.noarch.rpm 97 kB/s | 64 kB 00:00 2023-11-08T19:22:02.175 INFO:teuthology.orchestra.run.smithi143.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 48 kB/s | 24 kB 00:00 2023-11-08T19:22:02.458 INFO:teuthology.orchestra.run.smithi143.stdout:(62/131): python3-routes-2.4.1-12.el8.noarch.rp 36 kB/s | 196 kB 00:05 2023-11-08T19:22:02.475 INFO:teuthology.orchestra.run.smithi143.stdout:(63/131): python3-tempora-1.14.1-5.el8.noarch.r 78 kB/s | 29 kB 00:00 2023-11-08T19:22:02.551 INFO:teuthology.orchestra.run.smithi143.stdout:(64/131): python3-trustme-0.6.0-4.el8.noarch.rp 73 kB/s | 27 kB 00:00 2023-11-08T19:22:03.559 INFO:teuthology.orchestra.run.smithi143.stdout:(65/131): python3-websocket-client-0.56.0-5.el8 61 kB/s | 61 kB 00:01 2023-11-08T19:22:05.051 INFO:teuthology.orchestra.run.smithi143.stdout:(66/131): python3-webtest-2.0.33-1.el8.noarch.r 57 kB/s | 85 kB 00:01 2023-11-08T19:22:05.551 INFO:teuthology.orchestra.run.smithi143.stdout:(67/131): python3-zc-lockfile-2.0-2.el8.noarch. 46 kB/s | 23 kB 00:00 2023-11-08T19:22:06.593 INFO:teuthology.orchestra.run.smithi143.stdout:(68/131): python3-webob-1.8.5-1.el8.1.noarch.rp 61 kB/s | 251 kB 00:04 2023-11-08T19:22:06.618 INFO:teuthology.orchestra.run.smithi143.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.7 MB/s | 70 kB 00:00 2023-11-08T19:22:06.927 INFO:teuthology.orchestra.run.smithi143.stdout:(70/131): python3-scipy-1.0.0-21.module+el8.5.0 44 MB/s | 14 MB 00:00 2023-11-08T19:22:06.969 INFO:teuthology.orchestra.run.smithi143.stdout:(71/131): python3-pyasn1-modules-0.3.7-6.el8.no 2.6 MB/s | 110 kB 00:00 2023-11-08T19:22:07.011 INFO:teuthology.orchestra.run.smithi143.stdout:(72/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.3 MB/s | 54 kB 00:00 2023-11-08T19:22:07.053 INFO:teuthology.orchestra.run.smithi143.stdout:(73/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 6.1 MB/s | 259 kB 00:00 2023-11-08T19:22:07.095 INFO:teuthology.orchestra.run.smithi143.stdout:(74/131): socat-1.7.4.1-1.el8.x86_64.rpm 7.5 MB/s | 323 kB 00:00 2023-11-08T19:22:07.137 INFO:teuthology.orchestra.run.smithi143.stdout:(75/131): python3-markupsafe-0.23-19.el8.x86_64 930 kB/s | 39 kB 00:00 2023-11-08T19:22:07.196 INFO:teuthology.orchestra.run.smithi143.stdout:(76/131): bison-3.0.4-10.el8.x86_64.rpm 12 MB/s | 688 kB 00:00 2023-11-08T19:22:07.246 INFO:teuthology.orchestra.run.smithi143.stdout:(77/131): python3-jinja2-2.10.1-3.el8.noarch.rp 10 MB/s | 538 kB 00:00 2023-11-08T19:22:07.288 INFO:teuthology.orchestra.run.smithi143.stdout:(78/131): python3-mako-1.0.6-13.el8.noarch.rpm 3.7 MB/s | 157 kB 00:00 2023-11-08T19:22:07.330 INFO:teuthology.orchestra.run.smithi143.stdout:(79/131): oniguruma-6.8.2-2.el8.x86_64.rpm 4.4 MB/s | 187 kB 00:00 2023-11-08T19:22:07.380 INFO:teuthology.orchestra.run.smithi143.stdout:(80/131): automake-1.16.1-7.el8.noarch.rpm 14 MB/s | 713 kB 00:00 2023-11-08T19:22:07.506 INFO:teuthology.orchestra.run.smithi143.stdout:(81/131): python3-babel-2.5.1-7.el8.noarch.rpm 38 MB/s | 4.8 MB 00:00 2023-11-08T19:22:07.547 INFO:teuthology.orchestra.run.smithi143.stdout:(82/131): python3-jsonpatch-1.21-2.el8.noarch.r 639 kB/s | 27 kB 00:00 2023-11-08T19:22:07.589 INFO:teuthology.orchestra.run.smithi143.stdout:(83/131): flex-2.6.1-9.el8.x86_64.rpm 7.5 MB/s | 320 kB 00:00 2023-11-08T19:22:07.631 INFO:teuthology.orchestra.run.smithi143.stdout:(84/131): python3-prettytable-0.7.2-14.el8.noar 1.0 MB/s | 44 kB 00:00 2023-11-08T19:22:07.740 INFO:teuthology.orchestra.run.smithi143.stdout:(85/131): python3-numpy-1.14.3-10.el8.x86_64.rp 34 MB/s | 3.7 MB 00:00 2023-11-08T19:22:07.799 INFO:teuthology.orchestra.run.smithi143.stdout:(86/131): python3-werkzeug-0.12.2-4.el8.noarch. 7.7 MB/s | 457 kB 00:00 2023-11-08T19:22:07.841 INFO:teuthology.orchestra.run.smithi143.stdout:(87/131): python3-numpy-f2py-1.14.3-10.el8.x86_ 5.3 MB/s | 225 kB 00:00 2023-11-08T19:22:07.883 INFO:teuthology.orchestra.run.smithi143.stdout:(88/131): uuid-1.6.2-43.el8.x86_64.rpm 1.5 MB/s | 64 kB 00:00 2023-11-08T19:22:07.924 INFO:teuthology.orchestra.run.smithi143.stdout:(89/131): python3-pyasn1-0.3.7-6.el8.noarch.rpm 3.0 MB/s | 126 kB 00:00 2023-11-08T19:22:07.967 INFO:teuthology.orchestra.run.smithi143.stdout:(90/131): python3-jsonpointer-1.10-11.el8.noarc 479 kB/s | 20 kB 00:00 2023-11-08T19:22:08.017 INFO:teuthology.orchestra.run.smithi143.stdout:(91/131): python3-pyOpenSSL-19.0.0-1.el8.noarch 2.0 MB/s | 103 kB 00:00 2023-11-08T19:22:08.076 INFO:teuthology.orchestra.run.smithi143.stdout:(92/131): libtool-2.4.6-25.el8.x86_64.rpm 12 MB/s | 709 kB 00:00 2023-11-08T19:22:08.201 INFO:teuthology.orchestra.run.smithi143.stdout:(93/131): openblas-threads-0.3.15-3.el8.x86_64. 39 MB/s | 4.9 MB 00:00 2023-11-08T19:22:08.243 INFO:teuthology.orchestra.run.smithi143.stdout:(94/131): libpq-13.5-1.el8.x86_64.rpm 4.6 MB/s | 198 kB 00:00 2023-11-08T19:22:08.368 INFO:teuthology.orchestra.run.smithi143.stdout:(95/131): openblas-0.3.15-3.el8.x86_64.rpm 38 MB/s | 4.7 MB 00:00 2023-11-08T19:22:08.411 INFO:teuthology.orchestra.run.smithi143.stdout:(96/131): jq-1.6-3.el8.x86_64.rpm 4.7 MB/s | 202 kB 00:00 2023-11-08T19:22:08.452 INFO:teuthology.orchestra.run.smithi143.stdout:(97/131): librdkafka-0.11.4-3.el8.x86_64.rpm 8.3 MB/s | 354 kB 00:00 2023-11-08T19:22:08.750 INFO:teuthology.orchestra.run.smithi143.stdout:(98/131): postgresql-server-10.21-2.module+el8. 17 MB/s | 5.1 MB 00:00 2023-11-08T19:22:08.970 INFO:teuthology.orchestra.run.smithi143.stdout:(99/131): python3-waitress-1.4.3-1.el8.noarch.r 39 kB/s | 255 kB 00:06 2023-11-08T19:22:09.003 INFO:teuthology.orchestra.run.smithi143.stdout:(100/131): postgresql-contrib-10.21-2.module+el 3.1 MB/s | 808 kB 00:00 2023-11-08T19:22:09.054 INFO:teuthology.orchestra.run.smithi143.stdout:(101/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 1.0 MB/s | 47 kB 00:00 2023-11-08T19:22:09.096 INFO:teuthology.orchestra.run.smithi143.stdout:(102/131): postgresql-10.21-2.module+el8.6.0+15 12 MB/s | 1.5 MB 00:00 2023-11-08T19:22:09.121 INFO:teuthology.orchestra.run.smithi143.stdout:(103/131): libselinux-devel-2.9-5.el8.x86_64.rp 2.9 MB/s | 200 kB 00:00 2023-11-08T19:22:09.123 INFO:teuthology.orchestra.run.smithi174.stdout:(131/131): thrift-0.13.0-2.el8.x86_64.rpm 110 kB/s | 1.7 MB 00:16 2023-11-08T19:22:09.126 INFO:teuthology.orchestra.run.smithi174.stdout:-------------------------------------------------------------------------------- 2023-11-08T19:22:09.127 INFO:teuthology.orchestra.run.smithi174.stdout:Total 4.9 MB/s | 209 MB 00:43 2023-11-08T19:22:09.146 INFO:teuthology.orchestra.run.smithi143.stdout:(104/131): python3-jwt-1.6.1-2.el8.noarch.rpm 850 kB/s | 43 kB 00:00 2023-11-08T19:22:09.171 INFO:teuthology.orchestra.run.smithi143.stdout:(105/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 4.3 MB/s | 220 kB 00:00 2023-11-08T19:22:09.197 INFO:teuthology.orchestra.run.smithi143.stdout:(106/131): keyutils-libs-devel-1.5.10-9.el8.x86 956 kB/s | 48 kB 00:00 2023-11-08T19:22:09.222 INFO:teuthology.orchestra.run.smithi143.stdout:(107/131): libsepol-devel-2.9-3.el8.x86_64.rpm 1.7 MB/s | 87 kB 00:00 2023-11-08T19:22:09.247 INFO:teuthology.orchestra.run.smithi143.stdout:(108/131): python3-requests-oauthlib-1.0.0-1.el 850 kB/s | 43 kB 00:00 2023-11-08T19:22:09.264 INFO:teuthology.orchestra.run.smithi143.stdout:(109/131): python3-pycparser-2.14-14.el8.noarch 2.5 MB/s | 109 kB 00:00 2023-11-08T19:22:09.290 INFO:teuthology.orchestra.run.smithi143.stdout:(110/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 5.3 MB/s | 229 kB 00:00 2023-11-08T19:22:09.323 INFO:teuthology.orchestra.run.smithi143.stdout:(111/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 9.4 MB/s | 560 kB 00:00 2023-11-08T19:22:09.348 INFO:teuthology.orchestra.run.smithi143.stdout:(112/131): libaio-devel-0.3.112-1.el8.x86_64.rp 320 kB/s | 19 kB 00:00 2023-11-08T19:22:09.374 INFO:teuthology.orchestra.run.smithi143.stdout:(113/131): python3-oauthlib-2.1.0-1.el8.noarch. 3.0 MB/s | 155 kB 00:00 2023-11-08T19:22:09.407 INFO:teuthology.orchestra.run.smithi143.stdout:(114/131): python3-cryptography-3.2.1-5.el8.x86 9.3 MB/s | 559 kB 00:00 2023-11-08T19:22:09.425 INFO:teuthology.orchestra.run.smithi143.stdout:(115/131): libverto-devel-0.3.0-5.el8.x86_64.rp 356 kB/s | 18 kB 00:00 2023-11-08T19:22:09.474 INFO:teuthology.orchestra.run.smithi143.stdout:(116/131): pcre2-devel-10.32-2.el8.x86_64.rpm 9.0 MB/s | 605 kB 00:00 2023-11-08T19:22:09.500 INFO:teuthology.orchestra.run.smithi143.stdout:(117/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 3.1 MB/s | 238 kB 00:00 2023-11-08T19:22:09.543 INFO:teuthology.orchestra.run.smithi143.stdout:(118/131): xfsprogs-devel-5.0.0-10.el8.x86_64.r 1.7 MB/s | 72 kB 00:00 2023-11-08T19:22:09.593 INFO:teuthology.orchestra.run.smithi143.stdout:(119/131): openssl-devel-1.1.1k-7.el8_6.x86_64. 20 MB/s | 2.3 MB 00:00 2023-11-08T19:22:09.618 INFO:teuthology.orchestra.run.smithi143.stdout:(120/131): fuse-2.9.7-15.el8.x86_64.rpm 1.1 MB/s | 83 kB 00:00 2023-11-08T19:22:09.643 INFO:teuthology.orchestra.run.smithi143.stdout:(121/131): elfutils-libelf-devel-0.186-1.el8.x8 1.2 MB/s | 60 kB 00:00 2023-11-08T19:22:09.670 INFO:teuthology.orchestra.run.smithi143.stdout:(122/131): libuuid-devel-2.32.1-35.el8.x86_64.r 1.9 MB/s | 98 kB 00:00 2023-11-08T19:22:09.687 INFO:teuthology.orchestra.run.smithi143.stdout:(123/131): zlib-devel-1.2.11-18.el8_5.x86_64.rp 1.3 MB/s | 58 kB 00:00 2023-11-08T19:22:09.703 INFO:teuthology.orchestra.run.smithi143.stdout:(124/131): libcom_err-devel-1.45.6-4.el8.x86_64 1.1 MB/s | 39 kB 00:00 2023-11-08T19:22:09.737 INFO:teuthology.orchestra.run.smithi143.stdout:(125/131): libgfortran-8.5.0-10.1.el8_6.x86_64. 12 MB/s | 644 kB 00:00 2023-11-08T19:22:09.762 INFO:teuthology.orchestra.run.smithi143.stdout:(126/131): libquadmath-8.5.0-10.1.el8_6.x86_64. 2.8 MB/s | 171 kB 00:00 2023-11-08T19:22:09.787 INFO:teuthology.orchestra.run.smithi143.stdout:(127/131): NetworkManager-initscripts-updown-1. 2.7 MB/s | 138 kB 00:00 2023-11-08T19:22:09.934 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction check 2023-11-08T19:22:10.053 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction check succeeded. 2023-11-08T19:22:10.054 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction test 2023-11-08T19:22:10.062 INFO:teuthology.orchestra.run.smithi143.stdout:(128/131): librados2-18.0.0-7035.g7532011a.el8. 13 MB/s | 3.8 MB 00:00 2023-11-08T19:22:10.112 INFO:teuthology.orchestra.run.smithi143.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 16 MB/s | 798 kB 00:00 2023-11-08T19:22:10.379 INFO:teuthology.orchestra.run.smithi143.stdout:(130/131): librbd1-18.0.0-7035.g7532011a.el8.x8 6.3 MB/s | 3.7 MB 00:00 2023-11-08T19:22:11.175 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction test succeeded. 2023-11-08T19:22:11.179 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction 2023-11-08T19:22:16.170 INFO:teuthology.orchestra.run.smithi174.stdout: Preparing : 1/1 2023-11-08T19:22:16.356 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-11-08T19:22:16.547 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-11-08T19:22:16.733 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 3/134 2023-11-08T19:22:16.866 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 3/134 2023-11-08T19:22:17.066 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-11-08T19:22:17.479 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-11-08T19:22:17.708 INFO:teuthology.orchestra.run.smithi174.stdout: Upgrading : librados2-2:18.0.0-7035.g7532011a.el8.x86_64 5/134 2023-11-08T19:22:18.097 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: librados2-2:18.0.0-7035.g7532011a.el8.x86_64 5/134 2023-11-08T19:22:18.299 INFO:teuthology.orchestra.run.smithi174.stdout: Upgrading : librbd1-2:18.0.0-7035.g7532011a.el8.x86_64 6/134 2023-11-08T19:22:18.445 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: librbd1-2:18.0.0-7035.g7532011a.el8.x86_64 6/134 2023-11-08T19:22:18.624 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libcephfs2-2:18.0.0-7035.g7532011a.el8.x86_64 7/134 2023-11-08T19:22:18.720 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: libcephfs2-2:18.0.0-7035.g7532011a.el8.x86_64 7/134 2023-11-08T19:22:18.929 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-rados-2:18.0.0-7035.g7532011a.el8.x86_64 8/134 2023-11-08T19:22:19.117 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 9/134 2023-11-08T19:22:19.369 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-11-08T19:22:19.560 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-11-08T19:22:19.973 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-11-08T19:22:20.252 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-11-08T19:22:20.449 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libcephsqlite-2:18.0.0-7035.g7532011a.el8.x86_64 14/134 2023-11-08T19:22:20.540 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: libcephsqlite-2:18.0.0-7035.g7532011a.el8.x86_64 14/134 2023-11-08T19:22:20.707 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libradosstriper1-2:18.0.0-7035.g7532011a.el8.x86 15/134 2023-11-08T19:22:20.855 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: libradosstriper1-2:18.0.0-7035.g7532011a.el8.x86 15/134 2023-11-08T19:22:21.061 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 16/134 2023-11-08T19:22:21.230 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 17/134 2023-11-08T19:22:21.434 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 18/134 2023-11-08T19:22:21.557 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-11-08T19:22:21.697 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-11-08T19:22:21.815 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-11-08T19:22:22.453 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-11-08T19:22:22.632 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-11-08T19:22:23.229 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-11-08T19:22:23.431 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : openblas-0.3.15-3.el8.x86_64 22/134 2023-11-08T19:22:24.137 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 22/134 2023-11-08T19:22:24.421 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 23/134 2023-11-08T19:22:24.629 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 24/134 2023-11-08T19:22:24.841 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-11-08T19:22:25.063 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-ceph-common-2:18.0.0-7035.g7532011a.el8. 26/134 2023-11-08T19:22:25.218 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-ceph-argparse-2:18.0.0-7035.g7532011a.el 27/134 2023-11-08T19:22:25.477 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-cephfs-2:18.0.0-7035.g7532011a.el8.x86_6 28/134 2023-11-08T19:22:25.681 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-11-08T19:22:27.111 INFO:teuthology.orchestra.run.smithi143.stdout:(131/131): thrift-0.13.0-2.el8.x86_64.rpm 82 kB/s | 1.7 MB 00:21 2023-11-08T19:22:27.113 INFO:teuthology.orchestra.run.smithi143.stdout:-------------------------------------------------------------------------------- 2023-11-08T19:22:27.113 INFO:teuthology.orchestra.run.smithi143.stdout:Total 4.6 MB/s | 209 MB 00:45 2023-11-08T19:22:27.456 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-11-08T19:22:27.956 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction check 2023-11-08T19:22:28.077 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction check succeeded. 2023-11-08T19:22:28.077 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction test 2023-11-08T19:22:28.485 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 30/134 2023-11-08T19:22:28.803 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 31/134 2023-11-08T19:22:29.083 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 32/134 2023-11-08T19:22:29.197 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction test succeeded. 2023-11-08T19:22:29.201 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction 2023-11-08T19:22:29.325 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/134 2023-11-08T19:22:29.525 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-11-08T19:22:29.741 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-11-08T19:22:29.920 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-rbd-2:18.0.0-7035.g7532011a.el8.x86_64 36/134 2023-11-08T19:22:30.071 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : librados-devel-2:18.0.0-7035.g7532011a.el8.x86_6 37/134 2023-11-08T19:22:30.254 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 38/134 2023-11-08T19:22:30.441 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 39/134 2023-11-08T19:22:30.639 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : uuid-1.6.2-43.el8.x86_64 40/134 2023-11-08T19:22:30.694 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 40/134 2023-11-08T19:22:30.910 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 41/134 2023-11-08T19:22:31.134 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : automake-1.16.1-7.el8.noarch 42/134 2023-11-08T19:22:31.272 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 43/134 2023-11-08T19:22:31.347 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 43/134 2023-11-08T19:22:31.609 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : jq-1.6-3.el8.x86_64 44/134 2023-11-08T19:22:31.768 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 45/134 2023-11-08T19:22:31.959 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 46/134 2023-11-08T19:22:32.132 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 47/134 2023-11-08T19:22:32.324 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : fuse-2.9.7-15.el8.x86_64 48/134 2023-11-08T19:22:32.466 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 49/134 2023-11-08T19:22:32.659 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 50/134 2023-11-08T19:22:32.882 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 51/134 2023-11-08T19:22:33.212 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 52/134 2023-11-08T19:22:33.465 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 53/134 2023-11-08T19:22:33.734 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 54/134 2023-11-08T19:22:33.995 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 55/134 2023-11-08T19:22:34.035 INFO:teuthology.orchestra.run.smithi143.stdout: Preparing : 1/1 2023-11-08T19:22:34.213 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-11-08T19:22:34.243 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 56/134 2023-11-08T19:22:34.387 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-11-08T19:22:34.589 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 57/134 2023-11-08T19:22:34.605 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 3/134 2023-11-08T19:22:34.774 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 4/134 2023-11-08T19:22:34.813 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 58/134 2023-11-08T19:22:34.890 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 4/134 2023-11-08T19:22:35.039 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 59/134 2023-11-08T19:22:35.082 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-11-08T19:22:35.235 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 60/134 2023-11-08T19:22:35.445 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 61/134 2023-11-08T19:22:35.514 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-11-08T19:22:35.724 INFO:teuthology.orchestra.run.smithi143.stdout: Upgrading : librados2-2:18.0.0-7035.g7532011a.el8.x86_64 6/134 2023-11-08T19:22:35.934 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 62/134 2023-11-08T19:22:36.132 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: librados2-2:18.0.0-7035.g7532011a.el8.x86_64 6/134 2023-11-08T19:22:36.246 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 63/134 2023-11-08T19:22:36.324 INFO:teuthology.orchestra.run.smithi143.stdout: Upgrading : librbd1-2:18.0.0-7035.g7532011a.el8.x86_64 7/134 2023-11-08T19:22:36.458 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: librbd1-2:18.0.0-7035.g7532011a.el8.x86_64 7/134 2023-11-08T19:22:36.511 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 64/134 2023-11-08T19:22:36.665 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libcephfs2-2:18.0.0-7035.g7532011a.el8.x86_64 8/134 2023-11-08T19:22:36.759 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: libcephfs2-2:18.0.0-7035.g7532011a.el8.x86_64 8/134 2023-11-08T19:22:36.776 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 65/134 2023-11-08T19:22:36.957 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-rados-2:18.0.0-7035.g7532011a.el8.x86_64 9/134 2023-11-08T19:22:36.998 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 66/134 2023-11-08T19:22:37.187 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-11-08T19:22:37.235 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-11-08T19:22:37.366 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-11-08T19:22:37.434 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-11-08T19:22:37.664 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-11-08T19:22:37.860 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-11-08T19:22:37.878 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-11-08T19:22:38.075 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-11-08T19:22:38.114 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-11-08T19:22:38.248 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-11-08T19:22:38.306 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libcephsqlite-2:18.0.0-7035.g7532011a.el8.x86_64 14/134 2023-11-08T19:22:38.421 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: libcephsqlite-2:18.0.0-7035.g7532011a.el8.x86_64 14/134 2023-11-08T19:22:38.473 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-11-08T19:22:38.606 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libradosstriper1-2:18.0.0-7035.g7532011a.el8.x86 15/134 2023-11-08T19:22:38.669 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: libradosstriper1-2:18.0.0-7035.g7532011a.el8.x86 15/134 2023-11-08T19:22:38.674 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-11-08T19:22:38.822 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-11-08T19:22:38.906 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-11-08T19:22:38.951 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-11-08T19:22:39.074 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-11-08T19:22:39.098 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-11-08T19:22:39.218 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-11-08T19:22:39.485 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-11-08T19:22:39.683 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-11-08T19:22:39.741 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-11-08T19:22:39.902 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-11-08T19:22:40.089 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-11-08T19:22:40.160 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-11-08T19:22:40.395 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-11-08T19:22:40.569 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-11-08T19:22:40.687 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-11-08T19:22:40.819 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-11-08T19:22:40.856 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : openblas-0.3.15-3.el8.x86_64 19/134 2023-11-08T19:22:41.571 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 19/134 2023-11-08T19:22:41.834 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 20/134 2023-11-08T19:22:41.905 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-11-08T19:22:42.090 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 21/134 2023-11-08T19:22:42.191 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-11-08T19:22:42.309 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 22/134 2023-11-08T19:22:42.453 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-11-08T19:22:42.495 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/134 2023-11-08T19:22:42.712 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 24/134 2023-11-08T19:22:42.737 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-11-08T19:22:42.924 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-11-08T19:22:43.046 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-11-08T19:22:43.095 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-ceph-common-2:18.0.0-7035.g7532011a.el8. 26/134 2023-11-08T19:22:43.226 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-11-08T19:22:43.249 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-ceph-argparse-2:18.0.0-7035.g7532011a.el 27/134 2023-11-08T19:22:44.061 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-11-08T19:22:44.141 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-cephfs-2:18.0.0-7035.g7532011a.el8.x86_6 28/134 2023-11-08T19:22:44.287 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : librgw2-2:18.0.0-7035.g7532011a.el8.x86_64 92/134 2023-11-08T19:22:44.360 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: librgw2-2:18.0.0-7035.g7532011a.el8.x86_64 92/134 2023-11-08T19:22:44.444 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 29/134 2023-11-08T19:22:44.571 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-rgw-2:18.0.0-7035.g7532011a.el8.x86_64 93/134 2023-11-08T19:22:44.733 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 30/134 2023-11-08T19:22:44.992 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 31/134 2023-11-08T19:22:45.189 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-11-08T19:22:46.963 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-11-08T19:22:47.306 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 33/134 2023-11-08T19:22:47.499 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-11-08T19:22:47.698 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-11-08T19:22:47.868 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-rbd-2:18.0.0-7035.g7532011a.el8.x86_64 36/134 2023-11-08T19:22:48.025 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : librados-devel-2:18.0.0-7035.g7532011a.el8.x86_6 37/134 2023-11-08T19:22:48.197 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 38/134 2023-11-08T19:22:48.351 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-common-2:18.0.0-7035.g7532011a.el8.x86_64 94/134 2023-11-08T19:22:48.373 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 39/134 2023-11-08T19:22:48.540 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : fuse-2.9.7-15.el8.x86_64 40/134 2023-11-08T19:22:48.586 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-common-2:18.0.0-7035.g7532011a.el8.x86_64 94/134 2023-11-08T19:22:48.691 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 41/134 2023-11-08T19:22:48.895 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 42/134 2023-11-08T19:22:49.102 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 43/134 2023-11-08T19:22:49.200 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-common-2:18.0.0-7035.g7532011a.el8.x86_64 94/134 2023-11-08T19:22:49.461 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 44/134 2023-11-08T19:22:49.513 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-base-2:18.0.0-7035.g7532011a.el8.x86_64 95/134 2023-11-08T19:22:49.578 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-base-2:18.0.0-7035.g7532011a.el8.x86_64 95/134 2023-11-08T19:22:49.703 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-selinux-2:18.0.0-7035.g7532011a.el8.x86_64 96/134 2023-11-08T19:22:49.716 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 45/134 2023-11-08T19:22:49.914 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 46/134 2023-11-08T19:22:50.149 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 47/134 2023-11-08T19:22:50.359 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 48/134 2023-11-08T19:22:50.671 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 49/134 2023-11-08T19:22:50.929 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 50/134 2023-11-08T19:22:51.109 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 51/134 2023-11-08T19:22:51.352 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 52/134 2023-11-08T19:22:51.527 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 53/134 2023-11-08T19:22:51.709 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 54/134 2023-11-08T19:22:52.015 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/134 2023-11-08T19:22:52.286 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/134 2023-11-08T19:22:52.534 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/134 2023-11-08T19:22:52.794 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 58/134 2023-11-08T19:22:52.986 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 59/134 2023-11-08T19:22:53.148 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 60/134 2023-11-08T19:22:53.345 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : uuid-1.6.2-43.el8.x86_64 61/134 2023-11-08T19:22:53.401 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 61/134 2023-11-08T19:22:53.627 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 62/134 2023-11-08T19:22:53.832 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : automake-1.16.1-7.el8.noarch 63/134 2023-11-08T19:22:54.020 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 64/134 2023-11-08T19:22:54.104 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 64/134 2023-11-08T19:22:54.309 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : jq-1.6-3.el8.x86_64 65/134 2023-11-08T19:22:54.488 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 66/134 2023-11-08T19:22:54.661 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-11-08T19:22:54.881 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-11-08T19:22:55.173 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-11-08T19:22:55.411 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-11-08T19:22:55.549 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-11-08T19:22:55.697 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-11-08T19:22:55.879 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-11-08T19:22:56.082 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-11-08T19:22:56.255 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-11-08T19:22:56.406 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-11-08T19:22:56.542 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-11-08T19:22:56.773 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-11-08T19:22:57.006 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-11-08T19:22:57.184 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-11-08T19:22:57.449 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-11-08T19:22:57.677 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-11-08T19:22:57.852 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-11-08T19:22:58.082 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-11-08T19:22:59.154 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-11-08T19:22:59.415 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-11-08T19:22:59.644 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-11-08T19:22:59.920 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-11-08T19:23:00.197 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-11-08T19:23:00.396 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-11-08T19:23:01.241 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-11-08T19:23:01.427 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : librgw2-2:18.0.0-7035.g7532011a.el8.x86_64 92/134 2023-11-08T19:23:01.493 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: librgw2-2:18.0.0-7035.g7532011a.el8.x86_64 92/134 2023-11-08T19:23:01.678 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-rgw-2:18.0.0-7035.g7532011a.el8.x86_64 93/134 2023-11-08T19:23:05.368 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-common-2:18.0.0-7035.g7532011a.el8.x86_64 94/134 2023-11-08T19:23:05.586 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-common-2:18.0.0-7035.g7532011a.el8.x86_64 94/134 2023-11-08T19:23:06.304 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-common-2:18.0.0-7035.g7532011a.el8.x86_64 94/134 2023-11-08T19:23:06.628 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-base-2:18.0.0-7035.g7532011a.el8.x86_64 95/134 2023-11-08T19:23:06.704 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-base-2:18.0.0-7035.g7532011a.el8.x86_64 95/134 2023-11-08T19:23:06.843 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-selinux-2:18.0.0-7035.g7532011a.el8.x86_64 96/134 2023-11-08T19:23:08.537 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-selinux-2:18.0.0-7035.g7532011a.el8.x86_64 96/134 2023-11-08T19:23:08.537 INFO:teuthology.orchestra.run.smithi174.stdout:skipping the directory /sys 2023-11-08T19:23:08.537 INFO:teuthology.orchestra.run.smithi174.stdout:skipping the directory /proc 2023-11-08T19:23:08.538 INFO:teuthology.orchestra.run.smithi174.stdout:skipping the directory /dev 2023-11-08T19:23:08.538 INFO:teuthology.orchestra.run.smithi174.stdout:skipping the directory /run 2023-11-08T19:23:08.538 INFO:teuthology.orchestra.run.smithi174.stdout:skipping the directory /mnt 2023-11-08T19:23:08.538 INFO:teuthology.orchestra.run.smithi174.stdout:skipping the directory /var/tmp 2023-11-08T19:23:08.538 INFO:teuthology.orchestra.run.smithi174.stdout:skipping the directory /home 2023-11-08T19:23:08.539 INFO:teuthology.orchestra.run.smithi174.stdout:skipping the directory /tmp 2023-11-08T19:23:08.539 INFO:teuthology.orchestra.run.smithi174.stdout:skipping the directory /dev 2023-11-08T19:23:08.539 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-08T19:23:08.908 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-volume-2:18.0.0-7035.g7532011a.el8.noarch 97/134 2023-11-08T19:23:10.522 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-volume-2:18.0.0-7035.g7532011a.el8.noarch 97/134 2023-11-08T19:23:10.676 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-osd-2:18.0.0-7035.g7532011a.el8.x86_64 98/134 2023-11-08T19:23:10.896 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-osd-2:18.0.0-7035.g7532011a.el8.x86_64 98/134 2023-11-08T19:23:11.050 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mds-2:18.0.0-7035.g7532011a.el8.x86_64 99/134 2023-11-08T19:23:11.514 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-mds-2:18.0.0-7035.g7532011a.el8.x86_64 99/134 2023-11-08T19:23:11.741 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mon-2:18.0.0-7035.g7532011a.el8.x86_64 100/134 2023-11-08T19:23:11.758 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-mon-2:18.0.0-7035.g7532011a.el8.x86_64 100/134 2023-11-08T19:23:13.629 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: cephadm-2:18.0.0-7035.g7532011a.el8.noarch 101/134 2023-11-08T19:23:13.820 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : cephadm-2:18.0.0-7035.g7532011a.el8.noarch 101/134 2023-11-08T19:23:13.980 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-7035.g7532011a.e 102/134 2023-11-08T19:23:14.165 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-7035.g7532011a. 103/134 2023-11-08T19:23:14.399 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-7035.g7532011a.el8.noa 104/134 2023-11-08T19:23:14.620 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-7035.g7532011a.el8.noa 104/134 2023-11-08T19:23:14.891 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-7035.g7532011a.el8.n 105/134 2023-11-08T19:23:15.853 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-7035.g7532011a.el8.n 105/134 2023-11-08T19:23:16.024 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-7035.g753 106/134 2023-11-08T19:23:16.115 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-7035.g753 106/134 2023-11-08T19:23:16.493 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-7035.g7532011a.el 107/134 2023-11-08T19:23:16.708 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mgr-2:18.0.0-7035.g7532011a.el8.x86_64 108/134 2023-11-08T19:23:16.743 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-mgr-2:18.0.0-7035.g7532011a.el8.x86_64 108/134 2023-11-08T19:23:16.898 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mgr-rook-2:18.0.0-7035.g7532011a.el8.noarch 109/134 2023-11-08T19:23:16.910 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-7035.g7532011a.el8.noarch 109/134 2023-11-08T19:23:18.268 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-2:18.0.0-7035.g7532011a.el8.x86_64 110/134 2023-11-08T19:23:18.512 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-radosgw-2:18.0.0-7035.g7532011a.el8.x86_64 111/134 2023-11-08T19:23:18.555 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-7035.g7532011a.el8.x86_64 111/134 2023-11-08T19:23:18.707 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-7035.g75320 112/134 2023-11-08T19:23:18.749 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-7035.g75320 112/134 2023-11-08T19:23:18.907 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : cephfs-mirror-2:18.0.0-7035.g7532011a.el8.x86_64 113/134 2023-11-08T19:23:19.264 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-7035.g7532011a.el8.x86_64 113/134 2023-11-08T19:23:19.432 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : rbd-mirror-2:18.0.0-7035.g7532011a.el8.x86_64 114/134 2023-11-08T19:23:24.101 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: rbd-mirror-2:18.0.0-7035.g7532011a.el8.x86_64 114/134 2023-11-08T19:23:24.979 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-test-2:18.0.0-7035.g7532011a.el8.x86_64 115/134 2023-11-08T19:23:25.318 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 116/134 2023-11-08T19:23:25.514 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-fuse-2:18.0.0-7035.g7532011a.el8.x86_64 117/134 2023-11-08T19:23:25.796 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 118/134 2023-11-08T19:23:25.814 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-selinux-2:18.0.0-7035.g7532011a.el8.x86_64 96/134 2023-11-08T19:23:25.815 INFO:teuthology.orchestra.run.smithi143.stdout:skipping the directory /sys 2023-11-08T19:23:25.815 INFO:teuthology.orchestra.run.smithi143.stdout:skipping the directory /proc 2023-11-08T19:23:25.815 INFO:teuthology.orchestra.run.smithi143.stdout:skipping the directory /dev 2023-11-08T19:23:25.815 INFO:teuthology.orchestra.run.smithi143.stdout:skipping the directory /run 2023-11-08T19:23:25.816 INFO:teuthology.orchestra.run.smithi143.stdout:skipping the directory /mnt 2023-11-08T19:23:25.816 INFO:teuthology.orchestra.run.smithi143.stdout:skipping the directory /var/tmp 2023-11-08T19:23:25.816 INFO:teuthology.orchestra.run.smithi143.stdout:skipping the directory /home 2023-11-08T19:23:25.816 INFO:teuthology.orchestra.run.smithi143.stdout:skipping the directory /tmp 2023-11-08T19:23:25.816 INFO:teuthology.orchestra.run.smithi143.stdout:skipping the directory /dev 2023-11-08T19:23:25.817 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:23:25.947 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libtool-2.4.6-25.el8.x86_64 119/134 2023-11-08T19:23:26.139 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 119/134 2023-11-08T19:23:26.148 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-volume-2:18.0.0-7035.g7532011a.el8.noarch 97/134 2023-11-08T19:23:26.368 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 120/134 2023-11-08T19:23:26.585 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libcephfs-devel-2:18.0.0-7035.g7532011a.el8.x86_ 121/134 2023-11-08T19:23:26.748 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-11-08T19:23:27.761 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-volume-2:18.0.0-7035.g7532011a.el8.noarch 97/134 2023-11-08T19:23:27.949 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-osd-2:18.0.0-7035.g7532011a.el8.x86_64 98/134 2023-11-08T19:23:28.169 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-osd-2:18.0.0-7035.g7532011a.el8.x86_64 98/134 2023-11-08T19:23:28.324 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-mds-2:18.0.0-7035.g7532011a.el8.x86_64 99/134 2023-11-08T19:23:28.784 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-mds-2:18.0.0-7035.g7532011a.el8.x86_64 99/134 2023-11-08T19:23:28.946 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-08T19:23:28.957 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-mon-2:18.0.0-7035.g7532011a.el8.x86_64 100/134 2023-11-08T19:23:28.974 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-mon-2:18.0.0-7035.g7532011a.el8.x86_64 100/134 2023-11-08T19:23:29.205 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-08T19:23:29.223 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-08T19:23:29.389 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : cephfs-top-2:18.0.0-7035.g7532011a.el8.noarch 124/134 2023-11-08T19:23:29.688 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : rbd-fuse-2:18.0.0-7035.g7532011a.el8.x86_64 125/134 2023-11-08T19:23:29.972 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : rbd-nbd-2:18.0.0-7035.g7532011a.el8.x86_64 126/134 2023-11-08T19:23:30.351 INFO:teuthology.orchestra.run.smithi174.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-11-08T19:23:30.554 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : flex-2.6.1-9.el8.x86_64 128/134 2023-11-08T19:23:30.632 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: cephadm-2:18.0.0-7035.g7532011a.el8.noarch 101/134 2023-11-08T19:23:30.634 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 128/134 2023-11-08T19:23:30.813 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : bison-3.0.4-10.el8.x86_64 129/134 2023-11-08T19:23:30.817 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : cephadm-2:18.0.0-7035.g7532011a.el8.noarch 101/134 2023-11-08T19:23:30.825 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 129/134 2023-11-08T19:23:30.954 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-7035.g7532011a.e 102/134 2023-11-08T19:23:31.013 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-11-08T19:23:31.021 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-11-08T19:23:31.146 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-7035.g7532011a. 103/134 2023-11-08T19:23:31.157 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 131/134 2023-11-08T19:23:31.157 INFO:teuthology.orchestra.run.smithi174.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-08T19:23:31.314 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-7035.g7532011a.el8.noa 104/134 2023-11-08T19:23:31.341 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-08T19:23:31.341 INFO:teuthology.orchestra.run.smithi174.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-11-08T19:23:31.533 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-7035.g7532011a.el8.noa 104/134 2023-11-08T19:23:31.602 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-11-08T19:23:31.770 INFO:teuthology.orchestra.run.smithi174.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-11-08T19:23:31.847 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-7035.g7532011a.el8.n 105/134 2023-11-08T19:23:32.301 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-11-08T19:23:32.301 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-2:18.0.0-7035.g7532011a.el8.x86_64 1/134 2023-11-08T19:23:32.302 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-base-2:18.0.0-7035.g7532011a.el8.x86_64 2/134 2023-11-08T19:23:32.302 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-common-2:18.0.0-7035.g7532011a.el8.x86_64 3/134 2023-11-08T19:23:32.302 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-fuse-2:18.0.0-7035.g7532011a.el8.x86_64 4/134 2023-11-08T19:23:32.302 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-7035.g75320 5/134 2023-11-08T19:23:32.302 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mds-2:18.0.0-7035.g7532011a.el8.x86_64 6/134 2023-11-08T19:23:32.303 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mgr-2:18.0.0-7035.g7532011a.el8.x86_64 7/134 2023-11-08T19:23:32.303 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mon-2:18.0.0-7035.g7532011a.el8.x86_64 8/134 2023-11-08T19:23:32.303 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-osd-2:18.0.0-7035.g7532011a.el8.x86_64 9/134 2023-11-08T19:23:32.303 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-radosgw-2:18.0.0-7035.g7532011a.el8.x86_64 10/134 2023-11-08T19:23:32.303 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-selinux-2:18.0.0-7035.g7532011a.el8.x86_64 11/134 2023-11-08T19:23:32.304 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-test-2:18.0.0-7035.g7532011a.el8.x86_64 12/134 2023-11-08T19:23:32.304 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : cephfs-mirror-2:18.0.0-7035.g7532011a.el8.x86_64 13/134 2023-11-08T19:23:32.304 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libcephfs-devel-2:18.0.0-7035.g7532011a.el8.x86_ 14/134 2023-11-08T19:23:32.304 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libcephfs2-2:18.0.0-7035.g7532011a.el8.x86_64 15/134 2023-11-08T19:23:32.304 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libcephsqlite-2:18.0.0-7035.g7532011a.el8.x86_64 16/134 2023-11-08T19:23:32.304 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librados-devel-2:18.0.0-7035.g7532011a.el8.x86_6 17/134 2023-11-08T19:23:32.305 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libradosstriper1-2:18.0.0-7035.g7532011a.el8.x86 18/134 2023-11-08T19:23:32.305 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librgw2-2:18.0.0-7035.g7532011a.el8.x86_64 19/134 2023-11-08T19:23:32.305 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-ceph-argparse-2:18.0.0-7035.g7532011a.el 20/134 2023-11-08T19:23:32.305 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-ceph-common-2:18.0.0-7035.g7532011a.el8. 21/134 2023-11-08T19:23:32.305 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-cephfs-2:18.0.0-7035.g7532011a.el8.x86_6 22/134 2023-11-08T19:23:32.306 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-rados-2:18.0.0-7035.g7532011a.el8.x86_64 23/134 2023-11-08T19:23:32.306 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-rbd-2:18.0.0-7035.g7532011a.el8.x86_64 24/134 2023-11-08T19:23:32.306 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-rgw-2:18.0.0-7035.g7532011a.el8.x86_64 25/134 2023-11-08T19:23:32.306 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : rbd-fuse-2:18.0.0-7035.g7532011a.el8.x86_64 26/134 2023-11-08T19:23:32.306 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : rbd-mirror-2:18.0.0-7035.g7532011a.el8.x86_64 27/134 2023-11-08T19:23:32.306 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : rbd-nbd-2:18.0.0-7035.g7532011a.el8.x86_64 28/134 2023-11-08T19:23:32.307 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-7035.g7532011a. 29/134 2023-11-08T19:23:32.307 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-7035.g7532011a.el8.noa 30/134 2023-11-08T19:23:32.307 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-7035.g7532011a.el8.n 31/134 2023-11-08T19:23:32.307 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-7035.g753 32/134 2023-11-08T19:23:32.307 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-7035.g7532011a.el 33/134 2023-11-08T19:23:32.308 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mgr-rook-2:18.0.0-7035.g7532011a.el8.noarch 34/134 2023-11-08T19:23:32.308 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-7035.g7532011a.e 35/134 2023-11-08T19:23:32.308 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-volume-2:18.0.0-7035.g7532011a.el8.noarch 36/134 2023-11-08T19:23:32.308 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : cephadm-2:18.0.0-7035.g7532011a.el8.noarch 37/134 2023-11-08T19:23:32.308 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : cephfs-top-2:18.0.0-7035.g7532011a.el8.noarch 38/134 2023-11-08T19:23:32.309 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-11-08T19:23:32.309 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-11-08T19:23:32.309 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-11-08T19:23:32.309 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-11-08T19:23:32.309 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-11-08T19:23:32.310 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-11-08T19:23:32.310 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-11-08T19:23:32.310 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-11-08T19:23:32.310 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-11-08T19:23:32.310 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-11-08T19:23:32.311 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-11-08T19:23:32.311 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-11-08T19:23:32.311 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-11-08T19:23:32.311 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-11-08T19:23:32.311 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-11-08T19:23:32.311 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-11-08T19:23:32.312 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-11-08T19:23:32.312 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-11-08T19:23:32.312 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-11-08T19:23:32.312 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-11-08T19:23:32.312 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-11-08T19:23:32.313 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-11-08T19:23:32.313 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-11-08T19:23:32.313 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-11-08T19:23:32.313 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-11-08T19:23:32.313 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-11-08T19:23:32.313 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-11-08T19:23:32.314 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-11-08T19:23:32.314 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-11-08T19:23:32.314 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-11-08T19:23:32.314 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-11-08T19:23:32.315 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-11-08T19:23:32.315 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-11-08T19:23:32.315 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 72/134 2023-11-08T19:23:32.315 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 73/134 2023-11-08T19:23:32.315 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/134 2023-11-08T19:23:32.315 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/134 2023-11-08T19:23:32.316 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 76/134 2023-11-08T19:23:32.316 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 77/134 2023-11-08T19:23:32.316 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/134 2023-11-08T19:23:32.316 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 79/134 2023-11-08T19:23:32.316 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/134 2023-11-08T19:23:32.317 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 81/134 2023-11-08T19:23:32.317 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 82/134 2023-11-08T19:23:32.317 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 83/134 2023-11-08T19:23:32.317 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 84/134 2023-11-08T19:23:32.317 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 85/134 2023-11-08T19:23:32.318 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 86/134 2023-11-08T19:23:32.318 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 87/134 2023-11-08T19:23:32.318 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 88/134 2023-11-08T19:23:32.318 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 89/134 2023-11-08T19:23:32.318 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 90/134 2023-11-08T19:23:32.319 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 91/134 2023-11-08T19:23:32.319 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 92/134 2023-11-08T19:23:32.319 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 93/134 2023-11-08T19:23:32.319 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 94/134 2023-11-08T19:23:32.319 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 95/134 2023-11-08T19:23:32.320 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 96/134 2023-11-08T19:23:32.320 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 97/134 2023-11-08T19:23:32.320 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 98/134 2023-11-08T19:23:32.320 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 99/134 2023-11-08T19:23:32.320 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 100/134 2023-11-08T19:23:32.320 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 101/134 2023-11-08T19:23:32.321 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 102/134 2023-11-08T19:23:32.321 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 103/134 2023-11-08T19:23:32.321 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : bison-3.0.4-10.el8.x86_64 104/134 2023-11-08T19:23:32.321 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 105/134 2023-11-08T19:23:32.321 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 106/134 2023-11-08T19:23:32.322 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/134 2023-11-08T19:23:32.322 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : automake-1.16.1-7.el8.noarch 108/134 2023-11-08T19:23:32.322 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 109/134 2023-11-08T19:23:32.322 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 110/134 2023-11-08T19:23:32.322 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : flex-2.6.1-9.el8.x86_64 111/134 2023-11-08T19:23:32.323 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 112/134 2023-11-08T19:23:32.323 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 113/134 2023-11-08T19:23:32.323 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 114/134 2023-11-08T19:23:32.323 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 115/134 2023-11-08T19:23:32.323 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 116/134 2023-11-08T19:23:32.324 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 117/134 2023-11-08T19:23:32.324 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 118/134 2023-11-08T19:23:32.324 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 119/134 2023-11-08T19:23:32.324 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 120/134 2023-11-08T19:23:32.324 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 121/134 2023-11-08T19:23:32.325 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libpq-13.5-1.el8.x86_64 122/134 2023-11-08T19:23:32.325 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 123/134 2023-11-08T19:23:32.325 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : jq-1.6-3.el8.x86_64 124/134 2023-11-08T19:23:32.325 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 125/134 2023-11-08T19:23:32.325 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 126/134 2023-11-08T19:23:32.325 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 127/134 2023-11-08T19:23:32.326 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 128/134 2023-11-08T19:23:32.326 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librados2-2:18.0.0-7035.g7532011a.el8.x86_64 129/134 2023-11-08T19:23:32.326 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-11-08T19:23:32.326 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librbd1-2:18.0.0-7035.g7532011a.el8.x86_64 131/134 2023-11-08T19:23:32.326 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-08T19:23:32.327 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-11-08T19:23:32.806 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-7035.g7532011a.el8.n 105/134 2023-11-08T19:23:32.956 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-7035.g753 106/134 2023-11-08T19:23:33.045 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-7035.g753 106/134 2023-11-08T19:23:33.455 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-7035.g7532011a.el 107/134 2023-11-08T19:23:33.631 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-mgr-2:18.0.0-7035.g7532011a.el8.x86_64 108/134 2023-11-08T19:23:33.666 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-mgr-2:18.0.0-7035.g7532011a.el8.x86_64 108/134 2023-11-08T19:23:33.855 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-mgr-rook-2:18.0.0-7035.g7532011a.el8.noarch 109/134 2023-11-08T19:23:33.867 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-7035.g7532011a.el8.noarch 109/134 2023-11-08T19:23:34.997 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-11-08T19:23:34.997 INFO:teuthology.orchestra.run.smithi174.stdout:Last metadata expiration check: 0:02:20 ago on Wed 08 Nov 2023 07:21:14 PM UTC. 2023-11-08T19:23:35.193 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-2:18.0.0-7035.g7532011a.el8.x86_64 110/134 2023-11-08T19:23:35.429 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-radosgw-2:18.0.0-7035.g7532011a.el8.x86_64 111/134 2023-11-08T19:23:35.466 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-7035.g7532011a.el8.x86_64 111/134 2023-11-08T19:23:35.623 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-7035.g75320 112/134 2023-11-08T19:23:35.665 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-7035.g75320 112/134 2023-11-08T19:23:35.856 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : cephfs-mirror-2:18.0.0-7035.g7532011a.el8.x86_64 113/134 2023-11-08T19:23:36.215 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-7035.g7532011a.el8.x86_64 113/134 2023-11-08T19:23:36.364 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : rbd-mirror-2:18.0.0-7035.g7532011a.el8.x86_64 114/134 2023-11-08T19:23:37.697 INFO:teuthology.orchestra.run.smithi174.stdout:Installed products updated. 2023-11-08T19:23:37.917 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-08T19:23:37.917 INFO:teuthology.orchestra.run.smithi174.stdout:Upgraded: 2023-11-08T19:23:37.917 INFO:teuthology.orchestra.run.smithi174.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-11-08T19:23:37.918 INFO:teuthology.orchestra.run.smithi174.stdout: librados2-2:18.0.0-7035.g7532011a.el8.x86_64 2023-11-08T19:23:37.918 INFO:teuthology.orchestra.run.smithi174.stdout: librbd1-2:18.0.0-7035.g7532011a.el8.x86_64 2023-11-08T19:23:37.918 INFO:teuthology.orchestra.run.smithi174.stdout:Installed: 2023-11-08T19:23:37.918 INFO:teuthology.orchestra.run.smithi174.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-11-08T19:23:37.919 INFO:teuthology.orchestra.run.smithi174.stdout: automake-1.16.1-7.el8.noarch 2023-11-08T19:23:37.919 INFO:teuthology.orchestra.run.smithi174.stdout: bison-3.0.4-10.el8.x86_64 2023-11-08T19:23:37.919 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-2:18.0.0-7035.g7532011a.el8.x86_64 2023-11-08T19:23:37.919 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-base-2:18.0.0-7035.g7532011a.el8.x86_64 2023-11-08T19:23:37.919 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-common-2:18.0.0-7035.g7532011a.el8.x86_64 2023-11-08T19:23:37.920 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-fuse-2:18.0.0-7035.g7532011a.el8.x86_64 2023-11-08T19:23:37.920 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-grafana-dashboards-2:18.0.0-7035.g7532011a.el8.noarch 2023-11-08T19:23:37.920 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-immutable-object-cache-2:18.0.0-7035.g7532011a.el8.x86_64 2023-11-08T19:23:37.920 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mds-2:18.0.0-7035.g7532011a.el8.x86_64 2023-11-08T19:23:37.920 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-2:18.0.0-7035.g7532011a.el8.x86_64 2023-11-08T19:23:37.921 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-cephadm-2:18.0.0-7035.g7532011a.el8.noarch 2023-11-08T19:23:37.921 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-dashboard-2:18.0.0-7035.g7532011a.el8.noarch 2023-11-08T19:23:37.921 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-diskprediction-local-2:18.0.0-7035.g7532011a.el8.noarch 2023-11-08T19:23:37.921 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-modules-core-2:18.0.0-7035.g7532011a.el8.noarch 2023-11-08T19:23:37.921 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-rook-2:18.0.0-7035.g7532011a.el8.noarch 2023-11-08T19:23:37.922 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mon-2:18.0.0-7035.g7532011a.el8.x86_64 2023-11-08T19:23:37.922 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-osd-2:18.0.0-7035.g7532011a.el8.x86_64 2023-11-08T19:23:37.922 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-prometheus-alerts-2:18.0.0-7035.g7532011a.el8.noarch 2023-11-08T19:23:37.922 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-radosgw-2:18.0.0-7035.g7532011a.el8.x86_64 2023-11-08T19:23:37.922 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-selinux-2:18.0.0-7035.g7532011a.el8.x86_64 2023-11-08T19:23:37.923 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-test-2:18.0.0-7035.g7532011a.el8.x86_64 2023-11-08T19:23:37.923 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-volume-2:18.0.0-7035.g7532011a.el8.noarch 2023-11-08T19:23:37.923 INFO:teuthology.orchestra.run.smithi174.stdout: cephadm-2:18.0.0-7035.g7532011a.el8.noarch 2023-11-08T19:23:37.923 INFO:teuthology.orchestra.run.smithi174.stdout: cephfs-mirror-2:18.0.0-7035.g7532011a.el8.x86_64 2023-11-08T19:23:37.923 INFO:teuthology.orchestra.run.smithi174.stdout: cephfs-top-2:18.0.0-7035.g7532011a.el8.noarch 2023-11-08T19:23:37.924 INFO:teuthology.orchestra.run.smithi174.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-11-08T19:23:37.924 INFO:teuthology.orchestra.run.smithi174.stdout: flex-2.6.1-9.el8.x86_64 2023-11-08T19:23:37.924 INFO:teuthology.orchestra.run.smithi174.stdout: fuse-2.9.7-15.el8.x86_64 2023-11-08T19:23:37.924 INFO:teuthology.orchestra.run.smithi174.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-11-08T19:23:37.924 INFO:teuthology.orchestra.run.smithi174.stdout: jq-1.6-3.el8.x86_64 2023-11-08T19:23:37.925 INFO:teuthology.orchestra.run.smithi174.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-11-08T19:23:37.925 INFO:teuthology.orchestra.run.smithi174.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-11-08T19:23:37.925 INFO:teuthology.orchestra.run.smithi174.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-11-08T19:23:37.925 INFO:teuthology.orchestra.run.smithi174.stdout: libcephfs-devel-2:18.0.0-7035.g7532011a.el8.x86_64 2023-11-08T19:23:37.925 INFO:teuthology.orchestra.run.smithi174.stdout: libcephfs2-2:18.0.0-7035.g7532011a.el8.x86_64 2023-11-08T19:23:37.925 INFO:teuthology.orchestra.run.smithi174.stdout: libcephsqlite-2:18.0.0-7035.g7532011a.el8.x86_64 2023-11-08T19:23:37.926 INFO:teuthology.orchestra.run.smithi174.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-11-08T19:23:37.926 INFO:teuthology.orchestra.run.smithi174.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-11-08T19:23:37.926 INFO:teuthology.orchestra.run.smithi174.stdout: liboath-2.6.2-3.el8.x86_64 2023-11-08T19:23:37.926 INFO:teuthology.orchestra.run.smithi174.stdout: libpq-13.5-1.el8.x86_64 2023-11-08T19:23:37.927 INFO:teuthology.orchestra.run.smithi174.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-11-08T19:23:37.927 INFO:teuthology.orchestra.run.smithi174.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-11-08T19:23:37.927 INFO:teuthology.orchestra.run.smithi174.stdout: librados-devel-2:18.0.0-7035.g7532011a.el8.x86_64 2023-11-08T19:23:37.927 INFO:teuthology.orchestra.run.smithi174.stdout: libradosstriper1-2:18.0.0-7035.g7532011a.el8.x86_64 2023-11-08T19:23:37.927 INFO:teuthology.orchestra.run.smithi174.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-11-08T19:23:37.928 INFO:teuthology.orchestra.run.smithi174.stdout: librgw2-2:18.0.0-7035.g7532011a.el8.x86_64 2023-11-08T19:23:37.928 INFO:teuthology.orchestra.run.smithi174.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-11-08T19:23:37.928 INFO:teuthology.orchestra.run.smithi174.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-11-08T19:23:37.928 INFO:teuthology.orchestra.run.smithi174.stdout: libtool-2.4.6-25.el8.x86_64 2023-11-08T19:23:37.928 INFO:teuthology.orchestra.run.smithi174.stdout: libunwind-1.3.1-3.el8.x86_64 2023-11-08T19:23:37.928 INFO:teuthology.orchestra.run.smithi174.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-11-08T19:23:37.929 INFO:teuthology.orchestra.run.smithi174.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-11-08T19:23:37.929 INFO:teuthology.orchestra.run.smithi174.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-11-08T19:23:37.929 INFO:teuthology.orchestra.run.smithi174.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-11-08T19:23:37.929 INFO:teuthology.orchestra.run.smithi174.stdout: openblas-0.3.15-3.el8.x86_64 2023-11-08T19:23:37.929 INFO:teuthology.orchestra.run.smithi174.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-11-08T19:23:37.929 INFO:teuthology.orchestra.run.smithi174.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-11-08T19:23:37.930 INFO:teuthology.orchestra.run.smithi174.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-11-08T19:23:37.930 INFO:teuthology.orchestra.run.smithi174.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-11-08T19:23:37.930 INFO:teuthology.orchestra.run.smithi174.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-11-08T19:23:37.930 INFO:teuthology.orchestra.run.smithi174.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-08T19:23:37.931 INFO:teuthology.orchestra.run.smithi174.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-08T19:23:37.931 INFO:teuthology.orchestra.run.smithi174.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-08T19:23:37.931 INFO:teuthology.orchestra.run.smithi174.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-11-08T19:23:37.931 INFO:teuthology.orchestra.run.smithi174.stdout: python3-babel-2.5.1-7.el8.noarch 2023-11-08T19:23:37.932 INFO:teuthology.orchestra.run.smithi174.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-11-08T19:23:37.932 INFO:teuthology.orchestra.run.smithi174.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-11-08T19:23:37.932 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-11-08T19:23:37.932 INFO:teuthology.orchestra.run.smithi174.stdout: python3-ceph-argparse-2:18.0.0-7035.g7532011a.el8.x86_64 2023-11-08T19:23:37.932 INFO:teuthology.orchestra.run.smithi174.stdout: python3-ceph-common-2:18.0.0-7035.g7532011a.el8.x86_64 2023-11-08T19:23:37.933 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cephfs-2:18.0.0-7035.g7532011a.el8.x86_64 2023-11-08T19:23:37.933 INFO:teuthology.orchestra.run.smithi174.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-11-08T19:23:37.933 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-11-08T19:23:37.933 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-11-08T19:23:37.933 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-11-08T19:23:37.934 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-11-08T19:23:37.934 INFO:teuthology.orchestra.run.smithi174.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-11-08T19:23:37.934 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-6.2-6.el8.noarch 2023-11-08T19:23:37.934 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-11-08T19:23:37.934 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-11-08T19:23:37.935 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-11-08T19:23:37.935 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-11-08T19:23:37.935 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-11-08T19:23:37.935 INFO:teuthology.orchestra.run.smithi174.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-11-08T19:23:37.935 INFO:teuthology.orchestra.run.smithi174.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-11-08T19:23:37.935 INFO:teuthology.orchestra.run.smithi174.stdout: python3-mako-1.0.6-13.el8.noarch 2023-11-08T19:23:37.936 INFO:teuthology.orchestra.run.smithi174.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-11-08T19:23:37.936 INFO:teuthology.orchestra.run.smithi174.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-11-08T19:23:37.936 INFO:teuthology.orchestra.run.smithi174.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-11-08T19:23:37.936 INFO:teuthology.orchestra.run.smithi174.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-11-08T19:23:37.936 INFO:teuthology.orchestra.run.smithi174.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-11-08T19:23:37.936 INFO:teuthology.orchestra.run.smithi174.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-11-08T19:23:37.937 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-11-08T19:23:37.937 INFO:teuthology.orchestra.run.smithi174.stdout: python3-portend-2.6-1.el8.noarch 2023-11-08T19:23:37.937 INFO:teuthology.orchestra.run.smithi174.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-11-08T19:23:37.937 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-11-08T19:23:37.937 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-11-08T19:23:37.938 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-11-08T19:23:37.938 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pycparser-2.14-14.el8.noarch 2023-11-08T19:23:37.938 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pytz-2017.2-9.el8.noarch 2023-11-08T19:23:37.938 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rados-2:18.0.0-7035.g7532011a.el8.x86_64 2023-11-08T19:23:37.938 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rbd-2:18.0.0-7035.g7532011a.el8.x86_64 2023-11-08T19:23:37.939 INFO:teuthology.orchestra.run.smithi174.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-11-08T19:23:37.939 INFO:teuthology.orchestra.run.smithi174.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-11-08T19:23:37.939 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rgw-2:18.0.0-7035.g7532011a.el8.x86_64 2023-11-08T19:23:37.939 INFO:teuthology.orchestra.run.smithi174.stdout: python3-routes-2.4.1-12.el8.noarch 2023-11-08T19:23:37.939 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rsa-4.9-2.el8.noarch 2023-11-08T19:23:37.939 INFO:teuthology.orchestra.run.smithi174.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-11-08T19:23:37.940 INFO:teuthology.orchestra.run.smithi174.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-11-08T19:23:37.940 INFO:teuthology.orchestra.run.smithi174.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-11-08T19:23:37.940 INFO:teuthology.orchestra.run.smithi174.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-11-08T19:23:37.940 INFO:teuthology.orchestra.run.smithi174.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-11-08T19:23:37.940 INFO:teuthology.orchestra.run.smithi174.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-11-08T19:23:37.940 INFO:teuthology.orchestra.run.smithi174.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-11-08T19:23:37.941 INFO:teuthology.orchestra.run.smithi174.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-11-08T19:23:37.941 INFO:teuthology.orchestra.run.smithi174.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-11-08T19:23:37.941 INFO:teuthology.orchestra.run.smithi174.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-11-08T19:23:37.941 INFO:teuthology.orchestra.run.smithi174.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-11-08T19:23:37.941 INFO:teuthology.orchestra.run.smithi174.stdout: rbd-fuse-2:18.0.0-7035.g7532011a.el8.x86_64 2023-11-08T19:23:37.942 INFO:teuthology.orchestra.run.smithi174.stdout: rbd-mirror-2:18.0.0-7035.g7532011a.el8.x86_64 2023-11-08T19:23:37.942 INFO:teuthology.orchestra.run.smithi174.stdout: rbd-nbd-2:18.0.0-7035.g7532011a.el8.x86_64 2023-11-08T19:23:37.942 INFO:teuthology.orchestra.run.smithi174.stdout: socat-1.7.4.1-1.el8.x86_64 2023-11-08T19:23:37.942 INFO:teuthology.orchestra.run.smithi174.stdout: thrift-0.13.0-2.el8.x86_64 2023-11-08T19:23:37.942 INFO:teuthology.orchestra.run.smithi174.stdout: uuid-1.6.2-43.el8.x86_64 2023-11-08T19:23:37.942 INFO:teuthology.orchestra.run.smithi174.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-11-08T19:23:37.943 INFO:teuthology.orchestra.run.smithi174.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-11-08T19:23:37.946 INFO:teuthology.orchestra.run.smithi174.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-11-08T19:23:37.946 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-08T19:23:37.946 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2023-11-08T19:23:38.132 DEBUG:teuthology.parallel:result is None 2023-11-08T19:23:41.049 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: rbd-mirror-2:18.0.0-7035.g7532011a.el8.x86_64 114/134 2023-11-08T19:23:41.364 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-test-2:18.0.0-7035.g7532011a.el8.x86_64 115/134 2023-11-08T19:23:41.504 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libtool-2.4.6-25.el8.x86_64 116/134 2023-11-08T19:23:41.694 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 116/134 2023-11-08T19:23:42.573 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 117/134 2023-11-08T19:23:42.892 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 118/134 2023-11-08T19:23:43.079 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-fuse-2:18.0.0-7035.g7532011a.el8.x86_64 119/134 2023-11-08T19:23:43.275 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-11-08T19:23:43.450 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libcephfs-devel-2:18.0.0-7035.g7532011a.el8.x86_ 121/134 2023-11-08T19:23:43.597 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-11-08T19:23:45.764 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-08T19:23:45.995 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-08T19:23:46.014 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-08T19:23:46.179 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : cephfs-top-2:18.0.0-7035.g7532011a.el8.noarch 124/134 2023-11-08T19:23:46.462 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : rbd-fuse-2:18.0.0-7035.g7532011a.el8.x86_64 125/134 2023-11-08T19:23:46.797 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : rbd-nbd-2:18.0.0-7035.g7532011a.el8.x86_64 126/134 2023-11-08T19:23:47.097 INFO:teuthology.orchestra.run.smithi143.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-11-08T19:23:47.237 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-11-08T19:23:47.245 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-11-08T19:23:47.420 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 129/134 2023-11-08T19:23:47.586 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : flex-2.6.1-9.el8.x86_64 130/134 2023-11-08T19:23:47.666 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 130/134 2023-11-08T19:23:47.862 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : bison-3.0.4-10.el8.x86_64 131/134 2023-11-08T19:23:47.874 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 131/134 2023-11-08T19:23:47.874 INFO:teuthology.orchestra.run.smithi143.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-08T19:23:48.098 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-08T19:23:48.098 INFO:teuthology.orchestra.run.smithi143.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-11-08T19:23:48.317 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-11-08T19:23:48.493 INFO:teuthology.orchestra.run.smithi143.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-11-08T19:23:49.005 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-11-08T19:23:49.006 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-2:18.0.0-7035.g7532011a.el8.x86_64 1/134 2023-11-08T19:23:49.006 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-base-2:18.0.0-7035.g7532011a.el8.x86_64 2/134 2023-11-08T19:23:49.006 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-common-2:18.0.0-7035.g7532011a.el8.x86_64 3/134 2023-11-08T19:23:49.006 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-fuse-2:18.0.0-7035.g7532011a.el8.x86_64 4/134 2023-11-08T19:23:49.006 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-7035.g75320 5/134 2023-11-08T19:23:49.007 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-mds-2:18.0.0-7035.g7532011a.el8.x86_64 6/134 2023-11-08T19:23:49.007 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-mgr-2:18.0.0-7035.g7532011a.el8.x86_64 7/134 2023-11-08T19:23:49.007 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-mon-2:18.0.0-7035.g7532011a.el8.x86_64 8/134 2023-11-08T19:23:49.007 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-osd-2:18.0.0-7035.g7532011a.el8.x86_64 9/134 2023-11-08T19:23:49.007 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-radosgw-2:18.0.0-7035.g7532011a.el8.x86_64 10/134 2023-11-08T19:23:49.008 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-selinux-2:18.0.0-7035.g7532011a.el8.x86_64 11/134 2023-11-08T19:23:49.008 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-test-2:18.0.0-7035.g7532011a.el8.x86_64 12/134 2023-11-08T19:23:49.008 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : cephfs-mirror-2:18.0.0-7035.g7532011a.el8.x86_64 13/134 2023-11-08T19:23:49.008 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libcephfs-devel-2:18.0.0-7035.g7532011a.el8.x86_ 14/134 2023-11-08T19:23:49.009 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libcephfs2-2:18.0.0-7035.g7532011a.el8.x86_64 15/134 2023-11-08T19:23:49.009 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libcephsqlite-2:18.0.0-7035.g7532011a.el8.x86_64 16/134 2023-11-08T19:23:49.009 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : librados-devel-2:18.0.0-7035.g7532011a.el8.x86_6 17/134 2023-11-08T19:23:49.009 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libradosstriper1-2:18.0.0-7035.g7532011a.el8.x86 18/134 2023-11-08T19:23:49.009 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : librgw2-2:18.0.0-7035.g7532011a.el8.x86_64 19/134 2023-11-08T19:23:49.010 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-ceph-argparse-2:18.0.0-7035.g7532011a.el 20/134 2023-11-08T19:23:49.010 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-ceph-common-2:18.0.0-7035.g7532011a.el8. 21/134 2023-11-08T19:23:49.010 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-cephfs-2:18.0.0-7035.g7532011a.el8.x86_6 22/134 2023-11-08T19:23:49.010 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-rados-2:18.0.0-7035.g7532011a.el8.x86_64 23/134 2023-11-08T19:23:49.010 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-rbd-2:18.0.0-7035.g7532011a.el8.x86_64 24/134 2023-11-08T19:23:49.011 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-rgw-2:18.0.0-7035.g7532011a.el8.x86_64 25/134 2023-11-08T19:23:49.011 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : rbd-fuse-2:18.0.0-7035.g7532011a.el8.x86_64 26/134 2023-11-08T19:23:49.011 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : rbd-mirror-2:18.0.0-7035.g7532011a.el8.x86_64 27/134 2023-11-08T19:23:49.020 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : rbd-nbd-2:18.0.0-7035.g7532011a.el8.x86_64 28/134 2023-11-08T19:23:49.020 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-7035.g7532011a. 29/134 2023-11-08T19:23:49.020 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-7035.g7532011a.el8.noa 30/134 2023-11-08T19:23:49.021 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-7035.g7532011a.el8.n 31/134 2023-11-08T19:23:49.021 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-7035.g753 32/134 2023-11-08T19:23:49.021 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-7035.g7532011a.el 33/134 2023-11-08T19:23:49.021 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-mgr-rook-2:18.0.0-7035.g7532011a.el8.noarch 34/134 2023-11-08T19:23:49.021 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-7035.g7532011a.e 35/134 2023-11-08T19:23:49.022 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-volume-2:18.0.0-7035.g7532011a.el8.noarch 36/134 2023-11-08T19:23:49.022 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : cephadm-2:18.0.0-7035.g7532011a.el8.noarch 37/134 2023-11-08T19:23:49.022 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : cephfs-top-2:18.0.0-7035.g7532011a.el8.noarch 38/134 2023-11-08T19:23:49.022 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-11-08T19:23:49.022 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-11-08T19:23:49.023 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-11-08T19:23:49.023 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-11-08T19:23:49.023 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-11-08T19:23:49.023 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-11-08T19:23:49.024 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-11-08T19:23:49.024 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-11-08T19:23:49.024 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-11-08T19:23:49.025 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-11-08T19:23:49.025 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-11-08T19:23:49.025 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-11-08T19:23:49.025 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-11-08T19:23:49.026 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-11-08T19:23:49.026 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-11-08T19:23:49.026 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-11-08T19:23:49.026 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-11-08T19:23:49.026 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-11-08T19:23:49.027 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-11-08T19:23:49.027 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-11-08T19:23:49.027 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-11-08T19:23:49.027 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-11-08T19:23:49.027 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-11-08T19:23:49.027 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-11-08T19:23:49.028 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-11-08T19:23:49.028 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-11-08T19:23:49.028 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-11-08T19:23:49.028 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-11-08T19:23:49.028 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-11-08T19:23:49.029 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-11-08T19:23:49.029 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-11-08T19:23:49.029 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-11-08T19:23:49.029 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-11-08T19:23:49.029 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 72/134 2023-11-08T19:23:49.030 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 73/134 2023-11-08T19:23:49.030 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 74/134 2023-11-08T19:23:49.030 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 75/134 2023-11-08T19:23:49.030 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 76/134 2023-11-08T19:23:49.030 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 77/134 2023-11-08T19:23:49.031 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/134 2023-11-08T19:23:49.031 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 79/134 2023-11-08T19:23:49.031 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 80/134 2023-11-08T19:23:49.031 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 81/134 2023-11-08T19:23:49.031 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : automake-1.16.1-7.el8.noarch 82/134 2023-11-08T19:23:49.032 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 83/134 2023-11-08T19:23:49.032 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 84/134 2023-11-08T19:23:49.032 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : flex-2.6.1-9.el8.x86_64 85/134 2023-11-08T19:23:49.032 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 86/134 2023-11-08T19:23:49.032 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/134 2023-11-08T19:23:49.032 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 88/134 2023-11-08T19:23:49.033 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/134 2023-11-08T19:23:49.033 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 90/134 2023-11-08T19:23:49.033 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 91/134 2023-11-08T19:23:49.033 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 92/134 2023-11-08T19:23:49.033 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 93/134 2023-11-08T19:23:49.034 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 94/134 2023-11-08T19:23:49.034 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 95/134 2023-11-08T19:23:49.034 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libpq-13.5-1.el8.x86_64 96/134 2023-11-08T19:23:49.034 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 97/134 2023-11-08T19:23:49.034 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : jq-1.6-3.el8.x86_64 98/134 2023-11-08T19:23:49.035 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 99/134 2023-11-08T19:23:49.035 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 100/134 2023-11-08T19:23:49.035 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 101/134 2023-11-08T19:23:49.035 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 102/134 2023-11-08T19:23:49.035 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 103/134 2023-11-08T19:23:49.035 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 104/134 2023-11-08T19:23:49.036 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/134 2023-11-08T19:23:49.036 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 106/134 2023-11-08T19:23:49.036 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 107/134 2023-11-08T19:23:49.036 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 108/134 2023-11-08T19:23:49.036 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 109/134 2023-11-08T19:23:49.037 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 110/134 2023-11-08T19:23:49.037 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 111/134 2023-11-08T19:23:49.037 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 112/134 2023-11-08T19:23:49.037 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 113/134 2023-11-08T19:23:49.037 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 114/134 2023-11-08T19:23:49.037 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 115/134 2023-11-08T19:23:49.038 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 116/134 2023-11-08T19:23:49.038 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 117/134 2023-11-08T19:23:49.038 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 118/134 2023-11-08T19:23:49.038 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 119/134 2023-11-08T19:23:49.038 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-11-08T19:23:49.039 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 121/134 2023-11-08T19:23:49.039 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-11-08T19:23:49.039 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 123/134 2023-11-08T19:23:49.039 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 124/134 2023-11-08T19:23:49.039 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 125/134 2023-11-08T19:23:49.040 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 126/134 2023-11-08T19:23:49.040 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 127/134 2023-11-08T19:23:49.040 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-11-08T19:23:49.040 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : librados2-2:18.0.0-7035.g7532011a.el8.x86_64 129/134 2023-11-08T19:23:49.040 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-11-08T19:23:49.041 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : librbd1-2:18.0.0-7035.g7532011a.el8.x86_64 131/134 2023-11-08T19:23:49.041 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-08T19:23:49.041 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-11-08T19:23:51.754 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-11-08T19:23:51.754 INFO:teuthology.orchestra.run.smithi143.stdout:Last metadata expiration check: 0:02:21 ago on Wed 08 Nov 2023 07:21:30 PM UTC. 2023-11-08T19:23:54.298 INFO:teuthology.orchestra.run.smithi143.stdout:Installed products updated. 2023-11-08T19:23:54.519 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:23:54.520 INFO:teuthology.orchestra.run.smithi143.stdout:Upgraded: 2023-11-08T19:23:54.520 INFO:teuthology.orchestra.run.smithi143.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-11-08T19:23:54.520 INFO:teuthology.orchestra.run.smithi143.stdout: librados2-2:18.0.0-7035.g7532011a.el8.x86_64 2023-11-08T19:23:54.520 INFO:teuthology.orchestra.run.smithi143.stdout: librbd1-2:18.0.0-7035.g7532011a.el8.x86_64 2023-11-08T19:23:54.521 INFO:teuthology.orchestra.run.smithi143.stdout:Installed: 2023-11-08T19:23:54.521 INFO:teuthology.orchestra.run.smithi143.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-11-08T19:23:54.521 INFO:teuthology.orchestra.run.smithi143.stdout: automake-1.16.1-7.el8.noarch 2023-11-08T19:23:54.521 INFO:teuthology.orchestra.run.smithi143.stdout: bison-3.0.4-10.el8.x86_64 2023-11-08T19:23:54.521 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-2:18.0.0-7035.g7532011a.el8.x86_64 2023-11-08T19:23:54.522 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-base-2:18.0.0-7035.g7532011a.el8.x86_64 2023-11-08T19:23:54.522 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-common-2:18.0.0-7035.g7532011a.el8.x86_64 2023-11-08T19:23:54.522 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-fuse-2:18.0.0-7035.g7532011a.el8.x86_64 2023-11-08T19:23:54.522 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-grafana-dashboards-2:18.0.0-7035.g7532011a.el8.noarch 2023-11-08T19:23:54.522 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-immutable-object-cache-2:18.0.0-7035.g7532011a.el8.x86_64 2023-11-08T19:23:54.523 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mds-2:18.0.0-7035.g7532011a.el8.x86_64 2023-11-08T19:23:54.523 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr-2:18.0.0-7035.g7532011a.el8.x86_64 2023-11-08T19:23:54.523 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr-cephadm-2:18.0.0-7035.g7532011a.el8.noarch 2023-11-08T19:23:54.523 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr-dashboard-2:18.0.0-7035.g7532011a.el8.noarch 2023-11-08T19:23:54.523 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr-diskprediction-local-2:18.0.0-7035.g7532011a.el8.noarch 2023-11-08T19:23:54.523 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr-modules-core-2:18.0.0-7035.g7532011a.el8.noarch 2023-11-08T19:23:54.525 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr-rook-2:18.0.0-7035.g7532011a.el8.noarch 2023-11-08T19:23:54.525 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mon-2:18.0.0-7035.g7532011a.el8.x86_64 2023-11-08T19:23:54.525 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-osd-2:18.0.0-7035.g7532011a.el8.x86_64 2023-11-08T19:23:54.526 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-prometheus-alerts-2:18.0.0-7035.g7532011a.el8.noarch 2023-11-08T19:23:54.526 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-radosgw-2:18.0.0-7035.g7532011a.el8.x86_64 2023-11-08T19:23:54.526 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-selinux-2:18.0.0-7035.g7532011a.el8.x86_64 2023-11-08T19:23:54.526 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-test-2:18.0.0-7035.g7532011a.el8.x86_64 2023-11-08T19:23:54.526 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-volume-2:18.0.0-7035.g7532011a.el8.noarch 2023-11-08T19:23:54.527 INFO:teuthology.orchestra.run.smithi143.stdout: cephadm-2:18.0.0-7035.g7532011a.el8.noarch 2023-11-08T19:23:54.527 INFO:teuthology.orchestra.run.smithi143.stdout: cephfs-mirror-2:18.0.0-7035.g7532011a.el8.x86_64 2023-11-08T19:23:54.527 INFO:teuthology.orchestra.run.smithi143.stdout: cephfs-top-2:18.0.0-7035.g7532011a.el8.noarch 2023-11-08T19:23:54.527 INFO:teuthology.orchestra.run.smithi143.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-11-08T19:23:54.527 INFO:teuthology.orchestra.run.smithi143.stdout: flex-2.6.1-9.el8.x86_64 2023-11-08T19:23:54.528 INFO:teuthology.orchestra.run.smithi143.stdout: fuse-2.9.7-15.el8.x86_64 2023-11-08T19:23:54.528 INFO:teuthology.orchestra.run.smithi143.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-11-08T19:23:54.528 INFO:teuthology.orchestra.run.smithi143.stdout: jq-1.6-3.el8.x86_64 2023-11-08T19:23:54.528 INFO:teuthology.orchestra.run.smithi143.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-11-08T19:23:54.528 INFO:teuthology.orchestra.run.smithi143.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-11-08T19:23:54.528 INFO:teuthology.orchestra.run.smithi143.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-11-08T19:23:54.529 INFO:teuthology.orchestra.run.smithi143.stdout: libcephfs-devel-2:18.0.0-7035.g7532011a.el8.x86_64 2023-11-08T19:23:54.529 INFO:teuthology.orchestra.run.smithi143.stdout: libcephfs2-2:18.0.0-7035.g7532011a.el8.x86_64 2023-11-08T19:23:54.529 INFO:teuthology.orchestra.run.smithi143.stdout: libcephsqlite-2:18.0.0-7035.g7532011a.el8.x86_64 2023-11-08T19:23:54.529 INFO:teuthology.orchestra.run.smithi143.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-11-08T19:23:54.529 INFO:teuthology.orchestra.run.smithi143.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-11-08T19:23:54.530 INFO:teuthology.orchestra.run.smithi143.stdout: liboath-2.6.2-3.el8.x86_64 2023-11-08T19:23:54.530 INFO:teuthology.orchestra.run.smithi143.stdout: libpq-13.5-1.el8.x86_64 2023-11-08T19:23:54.530 INFO:teuthology.orchestra.run.smithi143.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-11-08T19:23:54.530 INFO:teuthology.orchestra.run.smithi143.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-11-08T19:23:54.530 INFO:teuthology.orchestra.run.smithi143.stdout: librados-devel-2:18.0.0-7035.g7532011a.el8.x86_64 2023-11-08T19:23:54.531 INFO:teuthology.orchestra.run.smithi143.stdout: libradosstriper1-2:18.0.0-7035.g7532011a.el8.x86_64 2023-11-08T19:23:54.531 INFO:teuthology.orchestra.run.smithi143.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-11-08T19:23:54.531 INFO:teuthology.orchestra.run.smithi143.stdout: librgw2-2:18.0.0-7035.g7532011a.el8.x86_64 2023-11-08T19:23:54.531 INFO:teuthology.orchestra.run.smithi143.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-11-08T19:23:54.531 INFO:teuthology.orchestra.run.smithi143.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-11-08T19:23:54.532 INFO:teuthology.orchestra.run.smithi143.stdout: libtool-2.4.6-25.el8.x86_64 2023-11-08T19:23:54.532 INFO:teuthology.orchestra.run.smithi143.stdout: libunwind-1.3.1-3.el8.x86_64 2023-11-08T19:23:54.532 INFO:teuthology.orchestra.run.smithi143.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-11-08T19:23:54.532 INFO:teuthology.orchestra.run.smithi143.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-11-08T19:23:54.532 INFO:teuthology.orchestra.run.smithi143.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-11-08T19:23:54.533 INFO:teuthology.orchestra.run.smithi143.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-11-08T19:23:54.533 INFO:teuthology.orchestra.run.smithi143.stdout: openblas-0.3.15-3.el8.x86_64 2023-11-08T19:23:54.533 INFO:teuthology.orchestra.run.smithi143.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-11-08T19:23:54.533 INFO:teuthology.orchestra.run.smithi143.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-11-08T19:23:54.533 INFO:teuthology.orchestra.run.smithi143.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-11-08T19:23:54.533 INFO:teuthology.orchestra.run.smithi143.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-11-08T19:23:54.534 INFO:teuthology.orchestra.run.smithi143.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-11-08T19:23:54.534 INFO:teuthology.orchestra.run.smithi143.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-08T19:23:54.534 INFO:teuthology.orchestra.run.smithi143.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-08T19:23:54.534 INFO:teuthology.orchestra.run.smithi143.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-08T19:23:54.534 INFO:teuthology.orchestra.run.smithi143.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-11-08T19:23:54.535 INFO:teuthology.orchestra.run.smithi143.stdout: python3-babel-2.5.1-7.el8.noarch 2023-11-08T19:23:54.535 INFO:teuthology.orchestra.run.smithi143.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-11-08T19:23:54.535 INFO:teuthology.orchestra.run.smithi143.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-11-08T19:23:54.535 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-11-08T19:23:54.535 INFO:teuthology.orchestra.run.smithi143.stdout: python3-ceph-argparse-2:18.0.0-7035.g7532011a.el8.x86_64 2023-11-08T19:23:54.536 INFO:teuthology.orchestra.run.smithi143.stdout: python3-ceph-common-2:18.0.0-7035.g7532011a.el8.x86_64 2023-11-08T19:23:54.536 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cephfs-2:18.0.0-7035.g7532011a.el8.x86_64 2023-11-08T19:23:54.536 INFO:teuthology.orchestra.run.smithi143.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-11-08T19:23:54.536 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-11-08T19:23:54.536 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-11-08T19:23:54.537 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-11-08T19:23:54.537 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-11-08T19:23:54.537 INFO:teuthology.orchestra.run.smithi143.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-11-08T19:23:54.537 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jaraco-6.2-6.el8.noarch 2023-11-08T19:23:54.537 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-11-08T19:23:54.538 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-11-08T19:23:54.538 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-11-08T19:23:54.538 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-11-08T19:23:54.538 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-11-08T19:23:54.538 INFO:teuthology.orchestra.run.smithi143.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-11-08T19:23:54.539 INFO:teuthology.orchestra.run.smithi143.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-11-08T19:23:54.539 INFO:teuthology.orchestra.run.smithi143.stdout: python3-mako-1.0.6-13.el8.noarch 2023-11-08T19:23:54.539 INFO:teuthology.orchestra.run.smithi143.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-11-08T19:23:54.539 INFO:teuthology.orchestra.run.smithi143.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-11-08T19:23:54.539 INFO:teuthology.orchestra.run.smithi143.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-11-08T19:23:54.539 INFO:teuthology.orchestra.run.smithi143.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-11-08T19:23:54.540 INFO:teuthology.orchestra.run.smithi143.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-11-08T19:23:54.540 INFO:teuthology.orchestra.run.smithi143.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-11-08T19:23:54.540 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-11-08T19:23:54.540 INFO:teuthology.orchestra.run.smithi143.stdout: python3-portend-2.6-1.el8.noarch 2023-11-08T19:23:54.540 INFO:teuthology.orchestra.run.smithi143.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-11-08T19:23:54.541 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-11-08T19:23:54.541 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-11-08T19:23:54.541 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-11-08T19:23:54.541 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pycparser-2.14-14.el8.noarch 2023-11-08T19:23:54.541 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pytz-2017.2-9.el8.noarch 2023-11-08T19:23:54.542 INFO:teuthology.orchestra.run.smithi143.stdout: python3-rados-2:18.0.0-7035.g7532011a.el8.x86_64 2023-11-08T19:23:54.542 INFO:teuthology.orchestra.run.smithi143.stdout: python3-rbd-2:18.0.0-7035.g7532011a.el8.x86_64 2023-11-08T19:23:54.542 INFO:teuthology.orchestra.run.smithi143.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-11-08T19:23:54.542 INFO:teuthology.orchestra.run.smithi143.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-11-08T19:23:54.542 INFO:teuthology.orchestra.run.smithi143.stdout: python3-rgw-2:18.0.0-7035.g7532011a.el8.x86_64 2023-11-08T19:23:54.542 INFO:teuthology.orchestra.run.smithi143.stdout: python3-routes-2.4.1-12.el8.noarch 2023-11-08T19:23:54.543 INFO:teuthology.orchestra.run.smithi143.stdout: python3-rsa-4.9-2.el8.noarch 2023-11-08T19:23:54.543 INFO:teuthology.orchestra.run.smithi143.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-11-08T19:23:54.543 INFO:teuthology.orchestra.run.smithi143.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-11-08T19:23:54.543 INFO:teuthology.orchestra.run.smithi143.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-11-08T19:23:54.543 INFO:teuthology.orchestra.run.smithi143.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-11-08T19:23:54.544 INFO:teuthology.orchestra.run.smithi143.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-11-08T19:23:54.544 INFO:teuthology.orchestra.run.smithi143.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-11-08T19:23:54.544 INFO:teuthology.orchestra.run.smithi143.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-11-08T19:23:54.544 INFO:teuthology.orchestra.run.smithi143.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-11-08T19:23:54.544 INFO:teuthology.orchestra.run.smithi143.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-11-08T19:23:54.545 INFO:teuthology.orchestra.run.smithi143.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-11-08T19:23:54.545 INFO:teuthology.orchestra.run.smithi143.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-11-08T19:23:54.545 INFO:teuthology.orchestra.run.smithi143.stdout: rbd-fuse-2:18.0.0-7035.g7532011a.el8.x86_64 2023-11-08T19:23:54.545 INFO:teuthology.orchestra.run.smithi143.stdout: rbd-mirror-2:18.0.0-7035.g7532011a.el8.x86_64 2023-11-08T19:23:54.545 INFO:teuthology.orchestra.run.smithi143.stdout: rbd-nbd-2:18.0.0-7035.g7532011a.el8.x86_64 2023-11-08T19:23:54.546 INFO:teuthology.orchestra.run.smithi143.stdout: socat-1.7.4.1-1.el8.x86_64 2023-11-08T19:23:54.546 INFO:teuthology.orchestra.run.smithi143.stdout: thrift-0.13.0-2.el8.x86_64 2023-11-08T19:23:54.546 INFO:teuthology.orchestra.run.smithi143.stdout: uuid-1.6.2-43.el8.x86_64 2023-11-08T19:23:54.546 INFO:teuthology.orchestra.run.smithi143.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-11-08T19:23:54.547 INFO:teuthology.orchestra.run.smithi143.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-11-08T19:23:54.547 INFO:teuthology.orchestra.run.smithi143.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-11-08T19:23:54.547 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:23:54.547 INFO:teuthology.orchestra.run.smithi143.stdout:Complete! 2023-11-08T19:23:54.729 DEBUG:teuthology.parallel:result is None 2023-11-08T19:23:54.730 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4 2023-11-08T19:23:54.875 DEBUG:teuthology.orchestra.run.smithi143:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-11-08T19:23:54.898 INFO:teuthology.orchestra.run.smithi143.stdout:18.0.0-7035.g7532011a.el8 2023-11-08T19:23:54.899 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-7035.g7532011a.el8 2023-11-08T19:23:54.899 INFO:teuthology.task.install:The correct ceph version 18.0.0-7035.g7532011a is installed. 2023-11-08T19:23:54.900 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4 2023-11-08T19:23:55.037 DEBUG:teuthology.orchestra.run.smithi174:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-11-08T19:23:55.066 INFO:teuthology.orchestra.run.smithi174.stdout:18.0.0-7035.g7532011a.el8 2023-11-08T19:23:55.067 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-7035.g7532011a.el8 2023-11-08T19:23:55.067 INFO:teuthology.task.install:The correct ceph version 18.0.0-7035.g7532011a is installed. 2023-11-08T19:23:55.068 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-11-08T19:23:55.068 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-08T19:23:55.069 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-11-08T19:23:55.109 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2023-11-08T19:23:55.109 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-11-08T19:23:55.155 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-11-08T19:23:55.155 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-08T19:23:55.156 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd of=/usr/bin/daemon-helper 2023-11-08T19:23:55.183 DEBUG:teuthology.orchestra.run.smithi143:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-11-08T19:23:55.249 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2023-11-08T19:23:55.249 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/usr/bin/daemon-helper 2023-11-08T19:23:55.279 DEBUG:teuthology.orchestra.run.smithi174:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-11-08T19:23:55.346 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-11-08T19:23:55.347 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-08T19:23:55.347 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd of=/usr/bin/adjust-ulimits 2023-11-08T19:23:55.373 DEBUG:teuthology.orchestra.run.smithi143:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-11-08T19:23:55.440 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2023-11-08T19:23:55.441 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/usr/bin/adjust-ulimits 2023-11-08T19:23:55.468 DEBUG:teuthology.orchestra.run.smithi174:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-11-08T19:23:55.535 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-11-08T19:23:55.536 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-08T19:23:55.536 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd of=/usr/bin/stdin-killer 2023-11-08T19:23:55.562 DEBUG:teuthology.orchestra.run.smithi143:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-11-08T19:23:55.629 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2023-11-08T19:23:55.629 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/usr/bin/stdin-killer 2023-11-08T19:23:55.656 DEBUG:teuthology.orchestra.run.smithi174:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-11-08T19:23:55.721 INFO:teuthology.run_tasks:Running task ceph... 2023-11-08T19:23:55.785 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-11-08T19:23:55.786 DEBUG:teuthology.orchestra.run.smithi143:> sudo chmod 777 /var/log/ceph 2023-11-08T19:23:55.789 DEBUG:teuthology.orchestra.run.smithi174:> sudo chmod 777 /var/log/ceph 2023-11-08T19:23:55.815 INFO:tasks.ceph:Disabling ceph logrotate... 2023-11-08T19:23:55.816 DEBUG:teuthology.orchestra.run.smithi143:> sudo rm -f -- /etc/logrotate.d/ceph 2023-11-08T19:23:55.855 DEBUG:teuthology.orchestra.run.smithi174:> sudo rm -f -- /etc/logrotate.d/ceph 2023-11-08T19:23:55.881 INFO:tasks.ceph:Creating extra log directories... 2023-11-08T19:23:55.882 DEBUG:teuthology.orchestra.run.smithi143:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-11-08T19:23:55.922 DEBUG:teuthology.orchestra.run.smithi174:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-11-08T19:23:55.954 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-11-08T19:23:55.955 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-11-08T19:23:55.955 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-11-08T19:23:55.955 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-08T19:23:55.956 DEBUG:teuthology.orchestra.run.smithi143:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-11-08T19:23:56.005 DEBUG:teuthology.orchestra.run.smithi143:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-11-08T19:23:56.147 DEBUG:teuthology.orchestra.run.smithi143:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-11-08T19:23:56.174 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2023-11-08T19:23:56.174 DEBUG:teuthology.orchestra.run.smithi174:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-11-08T19:23:56.191 DEBUG:teuthology.orchestra.run.smithi174:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-11-08T19:23:56.328 DEBUG:teuthology.orchestra.run.smithi174:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-11-08T19:23:56.365 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-11-08T19:23:56.365 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'evicting unresponsive client', 'RECENT_CRASH'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-11-08T19:23:56.366 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/pdonnell-2023-11-08_19:01:12-fs:functional-main-distro-default-smithi/7452105', 'branch': 'main', 'description': 'fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snapshots}', 'email': 'pdonnell@redhat.com', 'first_in_suite': False, 'job_id': '7452105', 'kernel': {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'pdonnell-2023-11-08_19:01:12-fs:functional-main-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'rhel', 'os_version': '8.6', 'overrides': {'admin_socket': {'branch': 'main'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'evicting unresponsive client', 'RECENT_CRASH'], 'sha1': '7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'check-counter': {'dry_run': True}, 'install': {'ceph': {'flavor': 'default', 'sha1': '7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4'}}, 'kclient': {'syntax': 'v1'}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-batrick-testing-20231108.185857', 'sha1': 'e797689ab88984658d73bbc928b2ed06c676d14a'}}, 'owner': 'scheduled_pdonnell@teuthology', 'priority': 50, 'repo': 'https://github.com/ceph/ceph.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 3044, 'sha1': '7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4', 'sleep_before_teardown': 0, 'suite': 'fs:functional', 'suite_branch': 'wip-batrick-testing-20231108.185857', 'suite_path': '/home/teuthworker/src/github.com_batrick_ceph_e797689ab88984658d73bbc928b2ed06c676d14a/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/batrick/ceph.git', 'suite_sha1': 'e797689ab88984658d73bbc928b2ed06c676d14a', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi143.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs=', 'smithi174.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'evicting unresponsive client', 'RECENT_CRASH'], 'sha1': '7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4', 'cluster': 'ceph'}}, {'kclient': None}, {'exec': {'mon.a': ['ceph config set mgr mgr/crash/warn_recent_interval 0']}}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.test_snapshots']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '6899cd26fceddb2fec83dc1a1349394b28c8998e', 'timestamp': '2023-11-08_19:01:12', 'tube': 'smithi', 'user': 'pdonnell', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3590714'} 2023-11-08T19:23:56.367 DEBUG:teuthology.orchestra.run.smithi143:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-11-08T19:23:56.370 DEBUG:teuthology.orchestra.run.smithi174:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-11-08T19:23:56.410 DEBUG:teuthology.orchestra.run.smithi143:> sudo install -d -m0777 -- /var/run/ceph 2023-11-08T19:23:56.427 DEBUG:teuthology.orchestra.run.smithi174:> sudo install -d -m0777 -- /var/run/ceph 2023-11-08T19:23:56.477 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-08T19:23:56.478 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/scratch_devs of=/dev/stdout 2023-11-08T19:23:56.508 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-11-08T19:23:56.508 DEBUG:teuthology.orchestra.run.smithi143:> stat /dev/vg_nvme/lv_1 2023-11-08T19:23:56.564 INFO:teuthology.orchestra.run.smithi143.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-11-08T19:23:56.565 INFO:teuthology.orchestra.run.smithi143.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-08T19:23:56.565 INFO:teuthology.orchestra.run.smithi143.stdout:Device: 6h/6d Inode: 393428 Links: 1 2023-11-08T19:23:56.565 INFO:teuthology.orchestra.run.smithi143.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-08T19:23:56.565 INFO:teuthology.orchestra.run.smithi143.stdout:Context: system_u:object_r:device_t:s0 2023-11-08T19:23:56.565 INFO:teuthology.orchestra.run.smithi143.stdout:Access: 2023-11-08 19:23:48.710040107 +0000 2023-11-08T19:23:56.566 INFO:teuthology.orchestra.run.smithi143.stdout:Modify: 2023-11-08 19:20:26.973297459 +0000 2023-11-08T19:23:56.566 INFO:teuthology.orchestra.run.smithi143.stdout:Change: 2023-11-08 19:20:26.973297459 +0000 2023-11-08T19:23:56.566 INFO:teuthology.orchestra.run.smithi143.stdout: Birth: - 2023-11-08T19:23:56.567 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-11-08T19:23:56.630 INFO:teuthology.orchestra.run.smithi143.stderr:1+0 records in 2023-11-08T19:23:56.631 INFO:teuthology.orchestra.run.smithi143.stderr:1+0 records out 2023-11-08T19:23:56.631 INFO:teuthology.orchestra.run.smithi143.stderr:512 bytes copied, 0.000320664 s, 1.6 MB/s 2023-11-08T19:23:56.632 DEBUG:teuthology.orchestra.run.smithi143:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-11-08T19:23:56.690 DEBUG:teuthology.orchestra.run.smithi143:> stat /dev/vg_nvme/lv_2 2023-11-08T19:23:56.747 INFO:teuthology.orchestra.run.smithi143.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-11-08T19:23:56.747 INFO:teuthology.orchestra.run.smithi143.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-08T19:23:56.747 INFO:teuthology.orchestra.run.smithi143.stdout:Device: 6h/6d Inode: 388975 Links: 1 2023-11-08T19:23:56.748 INFO:teuthology.orchestra.run.smithi143.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-08T19:23:56.748 INFO:teuthology.orchestra.run.smithi143.stdout:Context: system_u:object_r:device_t:s0 2023-11-08T19:23:56.748 INFO:teuthology.orchestra.run.smithi143.stdout:Access: 2023-11-08 19:23:48.710040107 +0000 2023-11-08T19:23:56.748 INFO:teuthology.orchestra.run.smithi143.stdout:Modify: 2023-11-08 19:20:27.473284143 +0000 2023-11-08T19:23:56.748 INFO:teuthology.orchestra.run.smithi143.stdout:Change: 2023-11-08 19:20:27.473284143 +0000 2023-11-08T19:23:56.748 INFO:teuthology.orchestra.run.smithi143.stdout: Birth: - 2023-11-08T19:23:56.749 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-11-08T19:23:56.812 INFO:teuthology.orchestra.run.smithi143.stderr:1+0 records in 2023-11-08T19:23:56.812 INFO:teuthology.orchestra.run.smithi143.stderr:1+0 records out 2023-11-08T19:23:56.812 INFO:teuthology.orchestra.run.smithi143.stderr:512 bytes copied, 0.000241768 s, 2.1 MB/s 2023-11-08T19:23:56.813 DEBUG:teuthology.orchestra.run.smithi143:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-11-08T19:23:56.872 DEBUG:teuthology.orchestra.run.smithi143:> stat /dev/vg_nvme/lv_3 2023-11-08T19:23:56.929 INFO:teuthology.orchestra.run.smithi143.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-11-08T19:23:56.929 INFO:teuthology.orchestra.run.smithi143.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-08T19:23:56.930 INFO:teuthology.orchestra.run.smithi143.stdout:Device: 6h/6d Inode: 392793 Links: 1 2023-11-08T19:23:56.930 INFO:teuthology.orchestra.run.smithi143.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-08T19:23:56.930 INFO:teuthology.orchestra.run.smithi143.stdout:Context: system_u:object_r:device_t:s0 2023-11-08T19:23:56.930 INFO:teuthology.orchestra.run.smithi143.stdout:Access: 2023-11-08 19:23:48.710040107 +0000 2023-11-08T19:23:56.930 INFO:teuthology.orchestra.run.smithi143.stdout:Modify: 2023-11-08 19:20:27.980270640 +0000 2023-11-08T19:23:56.931 INFO:teuthology.orchestra.run.smithi143.stdout:Change: 2023-11-08 19:20:27.980270640 +0000 2023-11-08T19:23:56.931 INFO:teuthology.orchestra.run.smithi143.stdout: Birth: - 2023-11-08T19:23:56.931 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-11-08T19:23:56.994 INFO:teuthology.orchestra.run.smithi143.stderr:1+0 records in 2023-11-08T19:23:56.994 INFO:teuthology.orchestra.run.smithi143.stderr:1+0 records out 2023-11-08T19:23:56.994 INFO:teuthology.orchestra.run.smithi143.stderr:512 bytes copied, 0.000243073 s, 2.1 MB/s 2023-11-08T19:23:56.995 DEBUG:teuthology.orchestra.run.smithi143:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-11-08T19:23:57.052 DEBUG:teuthology.orchestra.run.smithi143:> stat /dev/vg_nvme/lv_4 2023-11-08T19:23:57.109 INFO:teuthology.orchestra.run.smithi143.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-11-08T19:23:57.109 INFO:teuthology.orchestra.run.smithi143.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-08T19:23:57.110 INFO:teuthology.orchestra.run.smithi143.stdout:Device: 6h/6d Inode: 394374 Links: 1 2023-11-08T19:23:57.110 INFO:teuthology.orchestra.run.smithi143.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-08T19:23:57.110 INFO:teuthology.orchestra.run.smithi143.stdout:Context: system_u:object_r:device_t:s0 2023-11-08T19:23:57.110 INFO:teuthology.orchestra.run.smithi143.stdout:Access: 2023-11-08 19:23:48.711040081 +0000 2023-11-08T19:23:57.110 INFO:teuthology.orchestra.run.smithi143.stdout:Modify: 2023-11-08 19:20:28.489257083 +0000 2023-11-08T19:23:57.111 INFO:teuthology.orchestra.run.smithi143.stdout:Change: 2023-11-08 19:20:28.489257083 +0000 2023-11-08T19:23:57.111 INFO:teuthology.orchestra.run.smithi143.stdout: Birth: - 2023-11-08T19:23:57.111 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-11-08T19:23:57.173 INFO:teuthology.orchestra.run.smithi143.stderr:1+0 records in 2023-11-08T19:23:57.174 INFO:teuthology.orchestra.run.smithi143.stderr:1+0 records out 2023-11-08T19:23:57.174 INFO:teuthology.orchestra.run.smithi143.stderr:512 bytes copied, 0.000192973 s, 2.7 MB/s 2023-11-08T19:23:57.175 DEBUG:teuthology.orchestra.run.smithi143:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-11-08T19:23:57.232 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-08T19:23:57.233 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2023-11-08T19:23:57.233 DEBUG:teuthology.orchestra.run.smithi174:> dd if=/scratch_devs of=/dev/stdout 2023-11-08T19:23:57.250 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-11-08T19:23:57.251 DEBUG:teuthology.orchestra.run.smithi174:> stat /dev/vg_nvme/lv_1 2023-11-08T19:23:57.308 INFO:teuthology.orchestra.run.smithi174.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-11-08T19:23:57.308 INFO:teuthology.orchestra.run.smithi174.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-08T19:23:57.308 INFO:teuthology.orchestra.run.smithi174.stdout:Device: 6h/6d Inode: 382556 Links: 1 2023-11-08T19:23:57.309 INFO:teuthology.orchestra.run.smithi174.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-08T19:23:57.309 INFO:teuthology.orchestra.run.smithi174.stdout:Context: system_u:object_r:device_t:s0 2023-11-08T19:23:57.309 INFO:teuthology.orchestra.run.smithi174.stdout:Access: 2023-11-08 19:23:31.997094803 +0000 2023-11-08T19:23:57.309 INFO:teuthology.orchestra.run.smithi174.stdout:Modify: 2023-11-08 19:20:26.955404897 +0000 2023-11-08T19:23:57.310 INFO:teuthology.orchestra.run.smithi174.stdout:Change: 2023-11-08 19:20:26.955404897 +0000 2023-11-08T19:23:57.310 INFO:teuthology.orchestra.run.smithi174.stdout: Birth: - 2023-11-08T19:23:57.310 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-11-08T19:23:57.375 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records in 2023-11-08T19:23:57.375 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records out 2023-11-08T19:23:57.375 INFO:teuthology.orchestra.run.smithi174.stderr:512 bytes copied, 0.000312281 s, 1.6 MB/s 2023-11-08T19:23:57.376 DEBUG:teuthology.orchestra.run.smithi174:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-11-08T19:23:57.434 DEBUG:teuthology.orchestra.run.smithi174:> stat /dev/vg_nvme/lv_2 2023-11-08T19:23:57.491 INFO:teuthology.orchestra.run.smithi174.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-11-08T19:23:57.492 INFO:teuthology.orchestra.run.smithi174.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-08T19:23:57.492 INFO:teuthology.orchestra.run.smithi174.stdout:Device: 6h/6d Inode: 386436 Links: 1 2023-11-08T19:23:57.492 INFO:teuthology.orchestra.run.smithi174.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-08T19:23:57.492 INFO:teuthology.orchestra.run.smithi174.stdout:Context: system_u:object_r:device_t:s0 2023-11-08T19:23:57.493 INFO:teuthology.orchestra.run.smithi174.stdout:Access: 2023-11-08 19:23:31.997094803 +0000 2023-11-08T19:23:57.493 INFO:teuthology.orchestra.run.smithi174.stdout:Modify: 2023-11-08 19:20:27.449398506 +0000 2023-11-08T19:23:57.493 INFO:teuthology.orchestra.run.smithi174.stdout:Change: 2023-11-08 19:20:27.449398506 +0000 2023-11-08T19:23:57.493 INFO:teuthology.orchestra.run.smithi174.stdout: Birth: - 2023-11-08T19:23:57.493 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-11-08T19:23:57.557 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records in 2023-11-08T19:23:57.557 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records out 2023-11-08T19:23:57.557 INFO:teuthology.orchestra.run.smithi174.stderr:512 bytes copied, 0.000242429 s, 2.1 MB/s 2023-11-08T19:23:57.558 DEBUG:teuthology.orchestra.run.smithi174:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-11-08T19:23:57.616 DEBUG:teuthology.orchestra.run.smithi174:> stat /dev/vg_nvme/lv_3 2023-11-08T19:23:57.673 INFO:teuthology.orchestra.run.smithi174.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-11-08T19:23:57.673 INFO:teuthology.orchestra.run.smithi174.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-08T19:23:57.673 INFO:teuthology.orchestra.run.smithi174.stdout:Device: 6h/6d Inode: 395465 Links: 1 2023-11-08T19:23:57.673 INFO:teuthology.orchestra.run.smithi174.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-08T19:23:57.674 INFO:teuthology.orchestra.run.smithi174.stdout:Context: system_u:object_r:device_t:s0 2023-11-08T19:23:57.674 INFO:teuthology.orchestra.run.smithi174.stdout:Access: 2023-11-08 19:23:31.997094803 +0000 2023-11-08T19:23:57.674 INFO:teuthology.orchestra.run.smithi174.stdout:Modify: 2023-11-08 19:20:27.980391635 +0000 2023-11-08T19:23:57.674 INFO:teuthology.orchestra.run.smithi174.stdout:Change: 2023-11-08 19:20:27.980391635 +0000 2023-11-08T19:23:57.674 INFO:teuthology.orchestra.run.smithi174.stdout: Birth: - 2023-11-08T19:23:57.675 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-11-08T19:23:57.738 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records in 2023-11-08T19:23:57.738 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records out 2023-11-08T19:23:57.739 INFO:teuthology.orchestra.run.smithi174.stderr:512 bytes copied, 0.000223432 s, 2.3 MB/s 2023-11-08T19:23:57.740 DEBUG:teuthology.orchestra.run.smithi174:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-11-08T19:23:57.797 DEBUG:teuthology.orchestra.run.smithi174:> stat /dev/vg_nvme/lv_4 2023-11-08T19:23:57.854 INFO:teuthology.orchestra.run.smithi174.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-11-08T19:23:57.854 INFO:teuthology.orchestra.run.smithi174.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-08T19:23:57.855 INFO:teuthology.orchestra.run.smithi174.stdout:Device: 6h/6d Inode: 393834 Links: 1 2023-11-08T19:23:57.855 INFO:teuthology.orchestra.run.smithi174.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-08T19:23:57.855 INFO:teuthology.orchestra.run.smithi174.stdout:Context: system_u:object_r:device_t:s0 2023-11-08T19:23:57.855 INFO:teuthology.orchestra.run.smithi174.stdout:Access: 2023-11-08 19:23:31.997094803 +0000 2023-11-08T19:23:57.855 INFO:teuthology.orchestra.run.smithi174.stdout:Modify: 2023-11-08 19:20:28.472385269 +0000 2023-11-08T19:23:57.856 INFO:teuthology.orchestra.run.smithi174.stdout:Change: 2023-11-08 19:20:28.472385269 +0000 2023-11-08T19:23:57.856 INFO:teuthology.orchestra.run.smithi174.stdout: Birth: - 2023-11-08T19:23:57.856 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-11-08T19:23:57.918 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records in 2023-11-08T19:23:57.919 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records out 2023-11-08T19:23:57.919 INFO:teuthology.orchestra.run.smithi174.stderr:512 bytes copied, 0.000208484 s, 2.5 MB/s 2023-11-08T19:23:57.920 DEBUG:teuthology.orchestra.run.smithi174:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-11-08T19:23:57.977 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-08T19:23:57.978 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi143.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi174.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-11-08T19:23:57.978 INFO:tasks.ceph:Generating config... 2023-11-08T19:23:57.979 INFO:tasks.ceph:[client] client mount timeout = 600 2023-11-08T19:23:57.980 INFO:tasks.ceph:[client] debug client = 20 2023-11-08T19:23:57.980 INFO:tasks.ceph:[client] debug ms = 1 2023-11-08T19:23:57.980 INFO:tasks.ceph:[client] pid file = 2023-11-08T19:23:57.980 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-11-08T19:23:57.980 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-11-08T19:23:57.981 INFO:tasks.ceph:[global] ms die on skipped message = False 2023-11-08T19:23:57.981 INFO:tasks.ceph:[mds] debug mds = 20 2023-11-08T19:23:57.981 INFO:tasks.ceph:[mds] debug mds balancer = 20 2023-11-08T19:23:57.981 INFO:tasks.ceph:[mds] debug ms = 1 2023-11-08T19:23:57.982 INFO:tasks.ceph:[mds] mds debug frag = True 2023-11-08T19:23:57.982 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-11-08T19:23:57.982 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-11-08T19:23:57.982 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-11-08T19:23:57.983 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-11-08T19:23:57.983 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-11-08T19:23:57.983 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-11-08T19:23:57.983 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-11-08T19:23:57.984 INFO:tasks.ceph:[mgr] debug ms = 1 2023-11-08T19:23:57.984 INFO:tasks.ceph:[mon] debug mon = 20 2023-11-08T19:23:57.984 INFO:tasks.ceph:[mon] debug ms = 1 2023-11-08T19:23:57.984 INFO:tasks.ceph:[mon] debug paxos = 20 2023-11-08T19:23:57.985 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-11-08T19:23:57.985 INFO:tasks.ceph:[osd] bdev async discard = True 2023-11-08T19:23:57.985 INFO:tasks.ceph:[osd] bdev enable discard = True 2023-11-08T19:23:57.985 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2023-11-08T19:23:57.986 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-11-08T19:23:57.986 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-11-08T19:23:57.986 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2023-11-08T19:23:57.986 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2023-11-08T19:23:57.987 INFO:tasks.ceph:[osd] debug ms = 1 2023-11-08T19:23:57.987 INFO:tasks.ceph:[osd] debug osd = 20 2023-11-08T19:23:57.987 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2023-11-08T19:23:57.987 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-11-08T19:23:57.988 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-11-08T19:23:57.988 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-11-08T19:23:57.988 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-11-08T19:23:57.988 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-11-08T19:23:57.989 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-11-08T19:23:57.989 INFO:tasks.ceph:Setting up mon.a... 2023-11-08T19:23:57.989 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-11-08T19:23:58.043 INFO:teuthology.orchestra.run.smithi143.stdout:creating /etc/ceph/ceph.keyring 2023-11-08T19:23:58.046 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-11-08T19:23:58.093 DEBUG:teuthology.orchestra.run.smithi143:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-11-08T19:23:58.120 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.143'), ('mon.b', '172.21.15.174'), ('mon.c', '[v2:172.21.15.174:3301,v1:172.21.15.174:6790]')] 2023-11-08T19:23:58.121 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.143,172.21.15.174,[v2:172.21.15.174:3301,v1:172.21.15.174:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-11-08T19:23:58.122 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-08T19:23:58.122 DEBUG:teuthology.orchestra.run.smithi143:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-11-08T19:23:58.178 DEBUG:teuthology.orchestra.run.smithi143:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.143 --add b 172.21.15.174 --addv c '[v2:172.21.15.174:3301,v1:172.21.15.174:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-11-08T19:23:58.268 INFO:teuthology.orchestra.run.smithi143.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-11-08T19:23:58.268 INFO:teuthology.orchestra.run.smithi143.stdout:monmaptool: generated fsid 968483af-77fc-4068-ad7c-5c2a1a4613d7 2023-11-08T19:23:58.268 INFO:teuthology.orchestra.run.smithi143.stdout:setting min_mon_release = pacific 2023-11-08T19:23:58.269 INFO:teuthology.orchestra.run.smithi143.stdout:epoch 0 2023-11-08T19:23:58.269 INFO:teuthology.orchestra.run.smithi143.stdout:fsid 968483af-77fc-4068-ad7c-5c2a1a4613d7 2023-11-08T19:23:58.269 INFO:teuthology.orchestra.run.smithi143.stdout:last_changed 2023-11-08T19:23:58.267909+0000 2023-11-08T19:23:58.269 INFO:teuthology.orchestra.run.smithi143.stdout:created 2023-11-08T19:23:58.267909+0000 2023-11-08T19:23:58.269 INFO:teuthology.orchestra.run.smithi143.stdout:min_mon_release 16 (pacific) 2023-11-08T19:23:58.269 INFO:teuthology.orchestra.run.smithi143.stdout:election_strategy: 1 2023-11-08T19:23:58.270 INFO:teuthology.orchestra.run.smithi143.stdout:0: [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] mon.a 2023-11-08T19:23:58.270 INFO:teuthology.orchestra.run.smithi143.stdout:1: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.b 2023-11-08T19:23:58.270 INFO:teuthology.orchestra.run.smithi143.stdout:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2023-11-08T19:23:58.270 INFO:teuthology.orchestra.run.smithi143.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-11-08T19:23:58.272 DEBUG:teuthology.orchestra.run.smithi143:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-11-08T19:23:58.288 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 968483af-77fc-4068-ad7c-5c2a1a4613d7... 2023-11-08T19:23:58.289 DEBUG:teuthology.orchestra.run.smithi143:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-08T19:23:58.331 DEBUG:teuthology.orchestra.run.smithi174:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-08T19:23:58.373 INFO:teuthology.orchestra.run.smithi143.stdout:[global] 2023-11-08T19:23:58.374 INFO:teuthology.orchestra.run.smithi143.stdout: chdir = "" 2023-11-08T19:23:58.374 INFO:teuthology.orchestra.run.smithi143.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-08T19:23:58.374 INFO:teuthology.orchestra.run.smithi143.stdout: auth supported = cephx 2023-11-08T19:23:58.374 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:23:58.374 INFO:teuthology.orchestra.run.smithi143.stdout: filestore xattr use omap = true 2023-11-08T19:23:58.375 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:23:58.375 INFO:teuthology.orchestra.run.smithi143.stdout: mon clock drift allowed = 1.000 2023-11-08T19:23:58.375 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:23:58.375 INFO:teuthology.orchestra.run.smithi143.stdout: osd crush chooseleaf type = 0 2023-11-08T19:23:58.375 INFO:teuthology.orchestra.run.smithi143.stdout: auth debug = true 2023-11-08T19:23:58.375 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:23:58.376 INFO:teuthology.orchestra.run.smithi143.stdout: ms die on old message = true 2023-11-08T19:23:58.376 INFO:teuthology.orchestra.run.smithi143.stdout: ms die on bug = true 2023-11-08T19:23:58.376 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:23:58.376 INFO:teuthology.orchestra.run.smithi143.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-08T19:23:58.376 INFO:teuthology.orchestra.run.smithi143.stdout: mon pg warn max object skew = 0 2023-11-08T19:23:58.376 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:23:58.377 INFO:teuthology.orchestra.run.smithi143.stdout: # disable pg_autoscaler by default for new pools 2023-11-08T19:23:58.377 INFO:teuthology.orchestra.run.smithi143.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-08T19:23:58.377 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:23:58.377 INFO:teuthology.orchestra.run.smithi143.stdout: osd pool default size = 2 2023-11-08T19:23:58.377 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:23:58.378 INFO:teuthology.orchestra.run.smithi143.stdout: mon osd allow primary affinity = true 2023-11-08T19:23:58.378 INFO:teuthology.orchestra.run.smithi143.stdout: mon osd allow pg remap = true 2023-11-08T19:23:58.378 INFO:teuthology.orchestra.run.smithi143.stdout: mon warn on legacy crush tunables = false 2023-11-08T19:23:58.378 INFO:teuthology.orchestra.run.smithi143.stdout: mon warn on crush straw calc version zero = false 2023-11-08T19:23:58.378 INFO:teuthology.orchestra.run.smithi143.stdout: mon warn on no sortbitwise = false 2023-11-08T19:23:58.378 INFO:teuthology.orchestra.run.smithi143.stdout: mon warn on osd down out interval zero = false 2023-11-08T19:23:58.379 INFO:teuthology.orchestra.run.smithi143.stdout: mon warn on too few osds = false 2023-11-08T19:23:58.379 INFO:teuthology.orchestra.run.smithi143.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-08T19:23:58.379 INFO:teuthology.orchestra.run.smithi143.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-08T19:23:58.379 INFO:teuthology.orchestra.run.smithi143.stdout: mon_allow_pool_size_one = true 2023-11-08T19:23:58.379 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:23:58.379 INFO:teuthology.orchestra.run.smithi143.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-11-08T19:23:58.380 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:23:58.380 INFO:teuthology.orchestra.run.smithi143.stdout: osd default data pool replay window = 5 2023-11-08T19:23:58.380 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:23:58.380 INFO:teuthology.orchestra.run.smithi143.stdout: mon allow pool delete = true 2023-11-08T19:23:58.380 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:23:58.381 INFO:teuthology.orchestra.run.smithi143.stdout: mon cluster log file level = debug 2023-11-08T19:23:58.381 INFO:teuthology.orchestra.run.smithi143.stdout: debug asserts on shutdown = true 2023-11-08T19:23:58.381 INFO:teuthology.orchestra.run.smithi143.stdout: mon health detail to clog = false 2023-11-08T19:23:58.381 INFO:teuthology.orchestra.run.smithi143.stdout: mon host = "172.21.15.143,172.21.15.174,[v2:172.21.15.174:3301,v1:172.21.15.174:6790]" 2023-11-08T19:23:58.381 INFO:teuthology.orchestra.run.smithi143.stdout: ms die on skipped message = False 2023-11-08T19:23:58.381 INFO:teuthology.orchestra.run.smithi143.stdout: fsid = 968483af-77fc-4068-ad7c-5c2a1a4613d7 2023-11-08T19:23:58.382 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:23:58.382 INFO:teuthology.orchestra.run.smithi143.stdout:[osd] 2023-11-08T19:23:58.382 INFO:teuthology.orchestra.run.smithi143.stdout: osd journal size = 100 2023-11-08T19:23:58.382 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:23:58.382 INFO:teuthology.orchestra.run.smithi143.stdout: osd scrub load threshold = 5.0 2023-11-08T19:23:58.382 INFO:teuthology.orchestra.run.smithi143.stdout: osd scrub max interval = 600 2023-11-08T19:23:58.383 INFO:teuthology.orchestra.run.smithi143.stdout: osd mclock profile = high_recovery_ops 2023-11-08T19:23:58.383 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:23:58.383 INFO:teuthology.orchestra.run.smithi143.stdout: osd recover clone overlap = true 2023-11-08T19:23:58.383 INFO:teuthology.orchestra.run.smithi143.stdout: osd recovery max chunk = 1048576 2023-11-08T19:23:58.383 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:23:58.383 INFO:teuthology.orchestra.run.smithi143.stdout: osd debug shutdown = true 2023-11-08T19:23:58.384 INFO:teuthology.orchestra.run.smithi143.stdout: osd debug op order = true 2023-11-08T19:23:58.384 INFO:teuthology.orchestra.run.smithi143.stdout: osd debug verify stray on activate = true 2023-11-08T19:23:58.384 INFO:teuthology.orchestra.run.smithi143.stdout: osd debug trim objects = true 2023-11-08T19:23:58.384 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:23:58.384 INFO:teuthology.orchestra.run.smithi143.stdout: osd open classes on start = true 2023-11-08T19:23:58.385 INFO:teuthology.orchestra.run.smithi143.stdout: osd debug pg log writeout = true 2023-11-08T19:23:58.385 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:23:58.385 INFO:teuthology.orchestra.run.smithi143.stdout: osd deep scrub update digest min age = 30 2023-11-08T19:23:58.385 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:23:58.385 INFO:teuthology.orchestra.run.smithi143.stdout: osd map max advance = 10 2023-11-08T19:23:58.385 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:23:58.386 INFO:teuthology.orchestra.run.smithi143.stdout: journal zero on create = true 2023-11-08T19:23:58.386 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:23:58.386 INFO:teuthology.orchestra.run.smithi143.stdout: filestore ondisk finisher threads = 3 2023-11-08T19:23:58.386 INFO:teuthology.orchestra.run.smithi143.stdout: filestore apply finisher threads = 3 2023-11-08T19:23:58.386 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:23:58.387 INFO:teuthology.orchestra.run.smithi143.stdout: bdev debug aio = true 2023-11-08T19:23:58.387 INFO:teuthology.orchestra.run.smithi143.stdout: osd debug misdirected ops = true 2023-11-08T19:23:58.387 INFO:teuthology.orchestra.run.smithi143.stdout: bdev async discard = True 2023-11-08T19:23:58.387 INFO:teuthology.orchestra.run.smithi143.stdout: bdev enable discard = True 2023-11-08T19:23:58.387 INFO:teuthology.orchestra.run.smithi143.stdout: bluestore allocator = bitmap 2023-11-08T19:23:58.388 INFO:teuthology.orchestra.run.smithi143.stdout: bluestore block size = 96636764160 2023-11-08T19:23:58.388 INFO:teuthology.orchestra.run.smithi143.stdout: bluestore fsck on mount = True 2023-11-08T19:23:58.388 INFO:teuthology.orchestra.run.smithi143.stdout: debug bluefs = 1/20 2023-11-08T19:23:58.388 INFO:teuthology.orchestra.run.smithi143.stdout: debug bluestore = 1/20 2023-11-08T19:23:58.388 INFO:teuthology.orchestra.run.smithi143.stdout: debug ms = 1 2023-11-08T19:23:58.388 INFO:teuthology.orchestra.run.smithi143.stdout: debug osd = 20 2023-11-08T19:23:58.389 INFO:teuthology.orchestra.run.smithi143.stdout: debug rocksdb = 4/10 2023-11-08T19:23:58.389 INFO:teuthology.orchestra.run.smithi143.stdout: mon osd backfillfull_ratio = 0.85 2023-11-08T19:23:58.389 INFO:teuthology.orchestra.run.smithi143.stdout: mon osd full ratio = 0.9 2023-11-08T19:23:58.389 INFO:teuthology.orchestra.run.smithi143.stdout: mon osd nearfull ratio = 0.8 2023-11-08T19:23:58.389 INFO:teuthology.orchestra.run.smithi143.stdout: osd failsafe full ratio = 0.95 2023-11-08T19:23:58.390 INFO:teuthology.orchestra.run.smithi143.stdout: osd objectstore = bluestore 2023-11-08T19:23:58.390 INFO:teuthology.orchestra.run.smithi143.stdout: osd op complaint time = 180 2023-11-08T19:23:58.390 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:23:58.390 INFO:teuthology.orchestra.run.smithi143.stdout:[mgr] 2023-11-08T19:23:58.390 INFO:teuthology.orchestra.run.smithi143.stdout: debug ms = 1 2023-11-08T19:23:58.391 INFO:teuthology.orchestra.run.smithi143.stdout: debug mgr = 20 2023-11-08T19:23:58.391 INFO:teuthology.orchestra.run.smithi143.stdout: debug mon = 20 2023-11-08T19:23:58.391 INFO:teuthology.orchestra.run.smithi143.stdout: debug auth = 20 2023-11-08T19:23:58.391 INFO:teuthology.orchestra.run.smithi143.stdout: mon reweight min pgs per osd = 4 2023-11-08T19:23:58.391 INFO:teuthology.orchestra.run.smithi143.stdout: mon reweight min bytes per osd = 10 2023-11-08T19:23:58.391 INFO:teuthology.orchestra.run.smithi143.stdout: mgr/telemetry/nag = false 2023-11-08T19:23:58.392 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:23:58.392 INFO:teuthology.orchestra.run.smithi143.stdout:[mon] 2023-11-08T19:23:58.392 INFO:teuthology.orchestra.run.smithi143.stdout: debug ms = 1 2023-11-08T19:23:58.392 INFO:teuthology.orchestra.run.smithi143.stdout: debug mon = 20 2023-11-08T19:23:58.392 INFO:teuthology.orchestra.run.smithi143.stdout: debug paxos = 20 2023-11-08T19:23:58.393 INFO:teuthology.orchestra.run.smithi143.stdout: debug auth = 20 2023-11-08T19:23:58.393 INFO:teuthology.orchestra.run.smithi143.stdout: mon data avail warn = 5 2023-11-08T19:23:58.393 INFO:teuthology.orchestra.run.smithi143.stdout: mon mgr mkfs grace = 240 2023-11-08T19:23:58.393 INFO:teuthology.orchestra.run.smithi143.stdout: mon reweight min pgs per osd = 4 2023-11-08T19:23:58.393 INFO:teuthology.orchestra.run.smithi143.stdout: mon osd reporter subtree level = osd 2023-11-08T19:23:58.393 INFO:teuthology.orchestra.run.smithi143.stdout: mon osd prime pg temp = true 2023-11-08T19:23:58.394 INFO:teuthology.orchestra.run.smithi143.stdout: mon reweight min bytes per osd = 10 2023-11-08T19:23:58.394 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:23:58.394 INFO:teuthology.orchestra.run.smithi143.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-08T19:23:58.394 INFO:teuthology.orchestra.run.smithi143.stdout: auth mon ticket ttl = 660 # 11m 2023-11-08T19:23:58.394 INFO:teuthology.orchestra.run.smithi143.stdout: auth service ticket ttl = 240 # 4m 2023-11-08T19:23:58.395 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:23:58.395 INFO:teuthology.orchestra.run.smithi143.stdout: # don't complain about insecure global_id in the test suite 2023-11-08T19:23:58.395 INFO:teuthology.orchestra.run.smithi143.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-08T19:23:58.395 INFO:teuthology.orchestra.run.smithi143.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-08T19:23:58.395 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:23:58.395 INFO:teuthology.orchestra.run.smithi143.stdout: # 1m isn't quite enough 2023-11-08T19:23:58.396 INFO:teuthology.orchestra.run.smithi143.stdout: mon_down_mkfs_grace = 2m 2023-11-08T19:23:58.396 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:23:58.396 INFO:teuthology.orchestra.run.smithi143.stdout: mon_warn_on_filestore_osds = false 2023-11-08T19:23:58.396 INFO:teuthology.orchestra.run.smithi143.stdout: mon op complaint time = 120 2023-11-08T19:23:58.396 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:23:58.397 INFO:teuthology.orchestra.run.smithi143.stdout:[client] 2023-11-08T19:23:58.397 INFO:teuthology.orchestra.run.smithi143.stdout: rgw cache enabled = true 2023-11-08T19:23:58.397 INFO:teuthology.orchestra.run.smithi143.stdout: rgw enable ops log = true 2023-11-08T19:23:58.397 INFO:teuthology.orchestra.run.smithi143.stdout: rgw enable usage log = true 2023-11-08T19:23:58.397 INFO:teuthology.orchestra.run.smithi143.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-08T19:23:58.397 INFO:teuthology.orchestra.run.smithi143.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-08T19:23:58.398 INFO:teuthology.orchestra.run.smithi143.stdout: client mount timeout = 600 2023-11-08T19:23:58.398 INFO:teuthology.orchestra.run.smithi143.stdout: debug client = 20 2023-11-08T19:23:58.398 INFO:teuthology.orchestra.run.smithi143.stdout: debug ms = 1 2023-11-08T19:23:58.398 INFO:teuthology.orchestra.run.smithi143.stdout: pid file = "" 2023-11-08T19:23:58.398 INFO:teuthology.orchestra.run.smithi143.stdout: rados mon op timeout = 900 2023-11-08T19:23:58.399 INFO:teuthology.orchestra.run.smithi143.stdout: rados osd op timeout = 900 2023-11-08T19:23:58.399 INFO:teuthology.orchestra.run.smithi143.stdout:[mon.a] 2023-11-08T19:23:58.399 INFO:teuthology.orchestra.run.smithi143.stdout:[mon.b] 2023-11-08T19:23:58.399 INFO:teuthology.orchestra.run.smithi143.stdout:[mon.c] 2023-11-08T19:23:58.399 INFO:teuthology.orchestra.run.smithi143.stdout:[mds.a] 2023-11-08T19:23:58.399 INFO:teuthology.orchestra.run.smithi143.stdout:[mds.c] 2023-11-08T19:23:58.400 INFO:teuthology.orchestra.run.smithi143.stdout:[mds.b] 2023-11-08T19:23:58.400 INFO:teuthology.orchestra.run.smithi143.stdout:[mds.d] 2023-11-08T19:23:58.400 INFO:teuthology.orchestra.run.smithi143.stdout:[mds] 2023-11-08T19:23:58.400 INFO:teuthology.orchestra.run.smithi143.stdout: debug mds = 20 2023-11-08T19:23:58.400 INFO:teuthology.orchestra.run.smithi143.stdout: debug mds balancer = 20 2023-11-08T19:23:58.400 INFO:teuthology.orchestra.run.smithi143.stdout: debug ms = 1 2023-11-08T19:23:58.401 INFO:teuthology.orchestra.run.smithi143.stdout: mds debug frag = True 2023-11-08T19:23:58.401 INFO:teuthology.orchestra.run.smithi143.stdout: mds debug scatterstat = True 2023-11-08T19:23:58.401 INFO:teuthology.orchestra.run.smithi143.stdout: mds op complaint time = 180 2023-11-08T19:23:58.401 INFO:teuthology.orchestra.run.smithi143.stdout: mds verify scatter = True 2023-11-08T19:23:58.401 INFO:teuthology.orchestra.run.smithi143.stdout: osd op complaint time = 180 2023-11-08T19:23:58.402 INFO:teuthology.orchestra.run.smithi143.stdout: rados mon op timeout = 900 2023-11-08T19:23:58.402 INFO:teuthology.orchestra.run.smithi143.stdout: rados osd op timeout = 900 2023-11-08T19:23:58.403 INFO:teuthology.orchestra.run.smithi174.stdout:[global] 2023-11-08T19:23:58.404 INFO:teuthology.orchestra.run.smithi174.stdout: chdir = "" 2023-11-08T19:23:58.404 INFO:teuthology.orchestra.run.smithi174.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-08T19:23:58.404 INFO:teuthology.orchestra.run.smithi174.stdout: auth supported = cephx 2023-11-08T19:23:58.404 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-08T19:23:58.404 INFO:teuthology.orchestra.run.smithi174.stdout: filestore xattr use omap = true 2023-11-08T19:23:58.405 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-08T19:23:58.405 INFO:teuthology.orchestra.run.smithi174.stdout: mon clock drift allowed = 1.000 2023-11-08T19:23:58.405 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-08T19:23:58.405 INFO:teuthology.orchestra.run.smithi174.stdout: osd crush chooseleaf type = 0 2023-11-08T19:23:58.405 INFO:teuthology.orchestra.run.smithi174.stdout: auth debug = true 2023-11-08T19:23:58.405 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-08T19:23:58.406 INFO:teuthology.orchestra.run.smithi174.stdout: ms die on old message = true 2023-11-08T19:23:58.406 INFO:teuthology.orchestra.run.smithi174.stdout: ms die on bug = true 2023-11-08T19:23:58.406 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-08T19:23:58.406 INFO:teuthology.orchestra.run.smithi174.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-08T19:23:58.406 INFO:teuthology.orchestra.run.smithi174.stdout: mon pg warn max object skew = 0 2023-11-08T19:23:58.407 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-08T19:23:58.407 INFO:teuthology.orchestra.run.smithi174.stdout: # disable pg_autoscaler by default for new pools 2023-11-08T19:23:58.407 INFO:teuthology.orchestra.run.smithi174.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-08T19:23:58.407 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-08T19:23:58.407 INFO:teuthology.orchestra.run.smithi174.stdout: osd pool default size = 2 2023-11-08T19:23:58.408 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-08T19:23:58.408 INFO:teuthology.orchestra.run.smithi174.stdout: mon osd allow primary affinity = true 2023-11-08T19:23:58.408 INFO:teuthology.orchestra.run.smithi174.stdout: mon osd allow pg remap = true 2023-11-08T19:23:58.408 INFO:teuthology.orchestra.run.smithi174.stdout: mon warn on legacy crush tunables = false 2023-11-08T19:23:58.408 INFO:teuthology.orchestra.run.smithi174.stdout: mon warn on crush straw calc version zero = false 2023-11-08T19:23:58.408 INFO:teuthology.orchestra.run.smithi174.stdout: mon warn on no sortbitwise = false 2023-11-08T19:23:58.409 INFO:teuthology.orchestra.run.smithi174.stdout: mon warn on osd down out interval zero = false 2023-11-08T19:23:58.409 INFO:teuthology.orchestra.run.smithi174.stdout: mon warn on too few osds = false 2023-11-08T19:23:58.409 INFO:teuthology.orchestra.run.smithi174.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-08T19:23:58.409 INFO:teuthology.orchestra.run.smithi174.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-08T19:23:58.409 INFO:teuthology.orchestra.run.smithi174.stdout: mon_allow_pool_size_one = true 2023-11-08T19:23:58.410 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-08T19:23:58.410 INFO:teuthology.orchestra.run.smithi174.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-11-08T19:23:58.410 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-08T19:23:58.410 INFO:teuthology.orchestra.run.smithi174.stdout: osd default data pool replay window = 5 2023-11-08T19:23:58.410 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-08T19:23:58.411 INFO:teuthology.orchestra.run.smithi174.stdout: mon allow pool delete = true 2023-11-08T19:23:58.411 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-08T19:23:58.411 INFO:teuthology.orchestra.run.smithi174.stdout: mon cluster log file level = debug 2023-11-08T19:23:58.411 INFO:teuthology.orchestra.run.smithi174.stdout: debug asserts on shutdown = true 2023-11-08T19:23:58.411 INFO:teuthology.orchestra.run.smithi174.stdout: mon health detail to clog = false 2023-11-08T19:23:58.412 INFO:teuthology.orchestra.run.smithi174.stdout: mon host = "172.21.15.143,172.21.15.174,[v2:172.21.15.174:3301,v1:172.21.15.174:6790]" 2023-11-08T19:23:58.412 INFO:teuthology.orchestra.run.smithi174.stdout: ms die on skipped message = False 2023-11-08T19:23:58.412 INFO:teuthology.orchestra.run.smithi174.stdout: fsid = 968483af-77fc-4068-ad7c-5c2a1a4613d7 2023-11-08T19:23:58.412 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-08T19:23:58.412 INFO:teuthology.orchestra.run.smithi174.stdout:[osd] 2023-11-08T19:23:58.412 INFO:teuthology.orchestra.run.smithi174.stdout: osd journal size = 100 2023-11-08T19:23:58.413 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-08T19:23:58.413 INFO:teuthology.orchestra.run.smithi174.stdout: osd scrub load threshold = 5.0 2023-11-08T19:23:58.413 INFO:teuthology.orchestra.run.smithi174.stdout: osd scrub max interval = 600 2023-11-08T19:23:58.413 INFO:teuthology.orchestra.run.smithi174.stdout: osd mclock profile = high_recovery_ops 2023-11-08T19:23:58.413 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-08T19:23:58.414 INFO:teuthology.orchestra.run.smithi174.stdout: osd recover clone overlap = true 2023-11-08T19:23:58.414 INFO:teuthology.orchestra.run.smithi174.stdout: osd recovery max chunk = 1048576 2023-11-08T19:23:58.414 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-08T19:23:58.414 INFO:teuthology.orchestra.run.smithi174.stdout: osd debug shutdown = true 2023-11-08T19:23:58.414 INFO:teuthology.orchestra.run.smithi174.stdout: osd debug op order = true 2023-11-08T19:23:58.414 INFO:teuthology.orchestra.run.smithi174.stdout: osd debug verify stray on activate = true 2023-11-08T19:23:58.415 INFO:teuthology.orchestra.run.smithi174.stdout: osd debug trim objects = true 2023-11-08T19:23:58.415 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-08T19:23:58.415 INFO:teuthology.orchestra.run.smithi174.stdout: osd open classes on start = true 2023-11-08T19:23:58.415 INFO:teuthology.orchestra.run.smithi174.stdout: osd debug pg log writeout = true 2023-11-08T19:23:58.415 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-08T19:23:58.416 INFO:teuthology.orchestra.run.smithi174.stdout: osd deep scrub update digest min age = 30 2023-11-08T19:23:58.416 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-08T19:23:58.416 INFO:teuthology.orchestra.run.smithi174.stdout: osd map max advance = 10 2023-11-08T19:23:58.416 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-08T19:23:58.416 INFO:teuthology.orchestra.run.smithi174.stdout: journal zero on create = true 2023-11-08T19:23:58.416 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-08T19:23:58.417 INFO:teuthology.orchestra.run.smithi174.stdout: filestore ondisk finisher threads = 3 2023-11-08T19:23:58.417 INFO:teuthology.orchestra.run.smithi174.stdout: filestore apply finisher threads = 3 2023-11-08T19:23:58.417 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-08T19:23:58.417 INFO:teuthology.orchestra.run.smithi174.stdout: bdev debug aio = true 2023-11-08T19:23:58.417 INFO:teuthology.orchestra.run.smithi174.stdout: osd debug misdirected ops = true 2023-11-08T19:23:58.417 INFO:teuthology.orchestra.run.smithi174.stdout: bdev async discard = True 2023-11-08T19:23:58.418 INFO:teuthology.orchestra.run.smithi174.stdout: bdev enable discard = True 2023-11-08T19:23:58.418 INFO:teuthology.orchestra.run.smithi174.stdout: bluestore allocator = bitmap 2023-11-08T19:23:58.418 INFO:teuthology.orchestra.run.smithi174.stdout: bluestore block size = 96636764160 2023-11-08T19:23:58.418 INFO:teuthology.orchestra.run.smithi174.stdout: bluestore fsck on mount = True 2023-11-08T19:23:58.418 INFO:teuthology.orchestra.run.smithi174.stdout: debug bluefs = 1/20 2023-11-08T19:23:58.419 INFO:teuthology.orchestra.run.smithi174.stdout: debug bluestore = 1/20 2023-11-08T19:23:58.419 INFO:teuthology.orchestra.run.smithi174.stdout: debug ms = 1 2023-11-08T19:23:58.419 INFO:teuthology.orchestra.run.smithi174.stdout: debug osd = 20 2023-11-08T19:23:58.419 INFO:teuthology.orchestra.run.smithi174.stdout: debug rocksdb = 4/10 2023-11-08T19:23:58.419 INFO:teuthology.orchestra.run.smithi174.stdout: mon osd backfillfull_ratio = 0.85 2023-11-08T19:23:58.420 INFO:teuthology.orchestra.run.smithi174.stdout: mon osd full ratio = 0.9 2023-11-08T19:23:58.420 INFO:teuthology.orchestra.run.smithi174.stdout: mon osd nearfull ratio = 0.8 2023-11-08T19:23:58.420 INFO:teuthology.orchestra.run.smithi174.stdout: osd failsafe full ratio = 0.95 2023-11-08T19:23:58.420 INFO:teuthology.orchestra.run.smithi174.stdout: osd objectstore = bluestore 2023-11-08T19:23:58.420 INFO:teuthology.orchestra.run.smithi174.stdout: osd op complaint time = 180 2023-11-08T19:23:58.420 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-08T19:23:58.421 INFO:teuthology.orchestra.run.smithi174.stdout:[mgr] 2023-11-08T19:23:58.421 INFO:teuthology.orchestra.run.smithi174.stdout: debug ms = 1 2023-11-08T19:23:58.421 INFO:teuthology.orchestra.run.smithi174.stdout: debug mgr = 20 2023-11-08T19:23:58.421 INFO:teuthology.orchestra.run.smithi174.stdout: debug mon = 20 2023-11-08T19:23:58.421 INFO:teuthology.orchestra.run.smithi174.stdout: debug auth = 20 2023-11-08T19:23:58.422 INFO:teuthology.orchestra.run.smithi174.stdout: mon reweight min pgs per osd = 4 2023-11-08T19:23:58.422 INFO:teuthology.orchestra.run.smithi174.stdout: mon reweight min bytes per osd = 10 2023-11-08T19:23:58.422 INFO:teuthology.orchestra.run.smithi174.stdout: mgr/telemetry/nag = false 2023-11-08T19:23:58.422 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-08T19:23:58.422 INFO:teuthology.orchestra.run.smithi174.stdout:[mon] 2023-11-08T19:23:58.423 INFO:teuthology.orchestra.run.smithi174.stdout: debug ms = 1 2023-11-08T19:23:58.423 INFO:teuthology.orchestra.run.smithi174.stdout: debug mon = 20 2023-11-08T19:23:58.423 INFO:teuthology.orchestra.run.smithi174.stdout: debug paxos = 20 2023-11-08T19:23:58.423 INFO:teuthology.orchestra.run.smithi174.stdout: debug auth = 20 2023-11-08T19:23:58.423 INFO:teuthology.orchestra.run.smithi174.stdout: mon data avail warn = 5 2023-11-08T19:23:58.423 INFO:teuthology.orchestra.run.smithi174.stdout: mon mgr mkfs grace = 240 2023-11-08T19:23:58.424 INFO:teuthology.orchestra.run.smithi174.stdout: mon reweight min pgs per osd = 4 2023-11-08T19:23:58.424 INFO:teuthology.orchestra.run.smithi174.stdout: mon osd reporter subtree level = osd 2023-11-08T19:23:58.424 INFO:teuthology.orchestra.run.smithi174.stdout: mon osd prime pg temp = true 2023-11-08T19:23:58.424 INFO:teuthology.orchestra.run.smithi174.stdout: mon reweight min bytes per osd = 10 2023-11-08T19:23:58.424 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-08T19:23:58.424 INFO:teuthology.orchestra.run.smithi174.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-08T19:23:58.425 INFO:teuthology.orchestra.run.smithi174.stdout: auth mon ticket ttl = 660 # 11m 2023-11-08T19:23:58.425 INFO:teuthology.orchestra.run.smithi174.stdout: auth service ticket ttl = 240 # 4m 2023-11-08T19:23:58.425 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-08T19:23:58.425 INFO:teuthology.orchestra.run.smithi174.stdout: # don't complain about insecure global_id in the test suite 2023-11-08T19:23:58.425 INFO:teuthology.orchestra.run.smithi174.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-08T19:23:58.426 INFO:teuthology.orchestra.run.smithi174.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-08T19:23:58.426 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-08T19:23:58.426 INFO:teuthology.orchestra.run.smithi174.stdout: # 1m isn't quite enough 2023-11-08T19:23:58.426 INFO:teuthology.orchestra.run.smithi174.stdout: mon_down_mkfs_grace = 2m 2023-11-08T19:23:58.426 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-08T19:23:58.426 INFO:teuthology.orchestra.run.smithi174.stdout: mon_warn_on_filestore_osds = false 2023-11-08T19:23:58.427 INFO:teuthology.orchestra.run.smithi174.stdout: mon op complaint time = 120 2023-11-08T19:23:58.427 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-08T19:23:58.427 INFO:teuthology.orchestra.run.smithi174.stdout:[client] 2023-11-08T19:23:58.427 INFO:teuthology.orchestra.run.smithi174.stdout: rgw cache enabled = true 2023-11-08T19:23:58.427 INFO:teuthology.orchestra.run.smithi174.stdout: rgw enable ops log = true 2023-11-08T19:23:58.427 INFO:teuthology.orchestra.run.smithi174.stdout: rgw enable usage log = true 2023-11-08T19:23:58.428 INFO:teuthology.orchestra.run.smithi174.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-08T19:23:58.428 INFO:teuthology.orchestra.run.smithi174.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-08T19:23:58.428 INFO:teuthology.orchestra.run.smithi174.stdout: client mount timeout = 600 2023-11-08T19:23:58.428 INFO:teuthology.orchestra.run.smithi174.stdout: debug client = 20 2023-11-08T19:23:58.428 INFO:teuthology.orchestra.run.smithi174.stdout: debug ms = 1 2023-11-08T19:23:58.429 INFO:teuthology.orchestra.run.smithi174.stdout: pid file = "" 2023-11-08T19:23:58.429 INFO:teuthology.orchestra.run.smithi174.stdout: rados mon op timeout = 900 2023-11-08T19:23:58.429 INFO:teuthology.orchestra.run.smithi174.stdout: rados osd op timeout = 900 2023-11-08T19:23:58.429 INFO:teuthology.orchestra.run.smithi174.stdout:[mon.a] 2023-11-08T19:23:58.429 INFO:teuthology.orchestra.run.smithi174.stdout:[mon.b] 2023-11-08T19:23:58.429 INFO:teuthology.orchestra.run.smithi174.stdout:[mon.c] 2023-11-08T19:23:58.430 INFO:teuthology.orchestra.run.smithi174.stdout:[mds.a] 2023-11-08T19:23:58.430 INFO:teuthology.orchestra.run.smithi174.stdout:[mds.c] 2023-11-08T19:23:58.430 INFO:teuthology.orchestra.run.smithi174.stdout:[mds.b] 2023-11-08T19:23:58.430 INFO:teuthology.orchestra.run.smithi174.stdout:[mds.d] 2023-11-08T19:23:58.430 INFO:teuthology.orchestra.run.smithi174.stdout:[mds] 2023-11-08T19:23:58.431 INFO:teuthology.orchestra.run.smithi174.stdout: debug mds = 20 2023-11-08T19:23:58.431 INFO:teuthology.orchestra.run.smithi174.stdout: debug mds balancer = 20 2023-11-08T19:23:58.431 INFO:teuthology.orchestra.run.smithi174.stdout: debug ms = 1 2023-11-08T19:23:58.431 INFO:teuthology.orchestra.run.smithi174.stdout: mds debug frag = True 2023-11-08T19:23:58.431 INFO:teuthology.orchestra.run.smithi174.stdout: mds debug scatterstat = True 2023-11-08T19:23:58.432 INFO:teuthology.orchestra.run.smithi174.stdout: mds op complaint time = 180 2023-11-08T19:23:58.432 INFO:teuthology.orchestra.run.smithi174.stdout: mds verify scatter = True 2023-11-08T19:23:58.432 INFO:teuthology.orchestra.run.smithi174.stdout: osd op complaint time = 180 2023-11-08T19:23:58.432 INFO:teuthology.orchestra.run.smithi174.stdout: rados mon op timeout = 900 2023-11-08T19:23:58.432 INFO:teuthology.orchestra.run.smithi174.stdout: rados osd op timeout = 900 2023-11-08T19:23:58.432 INFO:tasks.ceph:Creating admin key on mon.a... 2023-11-08T19:23:58.433 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-11-08T19:23:58.500 INFO:tasks.ceph:Copying monmap to all nodes... 2023-11-08T19:23:58.501 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-08T19:23:58.501 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-11-08T19:23:58.518 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-08T19:23:58.518 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-11-08T19:23:58.574 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi143.front.sepia.ceph.com 2023-11-08T19:23:58.575 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-08T19:23:58.575 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd of=/etc/ceph/ceph.keyring 2023-11-08T19:23:58.575 DEBUG:teuthology.orchestra.run.smithi143:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-11-08T19:23:58.650 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-08T19:23:58.651 DEBUG:teuthology.orchestra.run.smithi143:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-11-08T19:23:58.707 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi174.front.sepia.ceph.com 2023-11-08T19:23:58.707 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2023-11-08T19:23:58.708 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/etc/ceph/ceph.keyring 2023-11-08T19:23:58.708 DEBUG:teuthology.orchestra.run.smithi174:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-11-08T19:23:58.751 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2023-11-08T19:23:58.752 DEBUG:teuthology.orchestra.run.smithi174:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-11-08T19:23:58.809 INFO:tasks.ceph:Setting up mon nodes... 2023-11-08T19:23:58.810 INFO:tasks.ceph:Setting up mgr nodes... 2023-11-08T19:23:58.810 DEBUG:teuthology.orchestra.run.smithi143:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-11-08T19:23:58.863 INFO:teuthology.orchestra.run.smithi143.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-11-08T19:23:58.866 DEBUG:teuthology.orchestra.run.smithi174:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-11-08T19:23:58.931 INFO:teuthology.orchestra.run.smithi174.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-11-08T19:23:58.934 INFO:tasks.ceph:Setting up mds nodes... 2023-11-08T19:23:58.935 DEBUG:teuthology.orchestra.run.smithi143:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-11-08T19:23:58.988 INFO:teuthology.orchestra.run.smithi143.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-11-08T19:23:58.990 DEBUG:teuthology.orchestra.run.smithi143:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-11-08T19:23:59.016 DEBUG:teuthology.orchestra.run.smithi143:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-11-08T19:23:59.109 INFO:teuthology.orchestra.run.smithi143.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-11-08T19:23:59.112 DEBUG:teuthology.orchestra.run.smithi143:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-11-08T19:23:59.137 DEBUG:teuthology.orchestra.run.smithi174:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-11-08T19:23:59.196 INFO:teuthology.orchestra.run.smithi174.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-11-08T19:23:59.198 DEBUG:teuthology.orchestra.run.smithi174:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-11-08T19:23:59.226 DEBUG:teuthology.orchestra.run.smithi174:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-11-08T19:23:59.319 INFO:teuthology.orchestra.run.smithi174.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-11-08T19:23:59.321 DEBUG:teuthology.orchestra.run.smithi174:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-11-08T19:23:59.347 INFO:tasks.ceph_client:Setting up client nodes... 2023-11-08T19:23:59.348 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-11-08T19:23:59.391 INFO:teuthology.orchestra.run.smithi143.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-11-08T19:23:59.404 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2023-11-08T19:23:59.488 INFO:teuthology.orchestra.run.smithi143.stdout:creating /etc/ceph/ceph.client.2.keyring 2023-11-08T19:23:59.500 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-11-08T19:23:59.544 INFO:teuthology.orchestra.run.smithi174.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-11-08T19:23:59.557 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2023-11-08T19:23:59.599 INFO:teuthology.orchestra.run.smithi174.stdout:creating /etc/ceph/ceph.client.3.keyring 2023-11-08T19:23:59.612 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-11-08T19:23:59.613 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi143.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi174.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-11-08T19:23:59.613 DEBUG:teuthology.orchestra.run.smithi143:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-11-08T19:23:59.639 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-08T19:23:59.640 INFO:tasks.ceph:role: osd.0 2023-11-08T19:23:59.640 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi143.front.sepia.ceph.com 2023-11-08T19:23:59.640 DEBUG:teuthology.orchestra.run.smithi143:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-11-08T19:23:59.713 INFO:teuthology.orchestra.run.smithi143.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-11-08T19:23:59.714 INFO:teuthology.orchestra.run.smithi143.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-08T19:23:59.714 INFO:teuthology.orchestra.run.smithi143.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-08T19:23:59.714 INFO:teuthology.orchestra.run.smithi143.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-08T19:23:59.714 INFO:teuthology.orchestra.run.smithi143.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-08T19:23:59.715 INFO:teuthology.orchestra.run.smithi143.stdout: = sunit=0 swidth=0 blks 2023-11-08T19:23:59.715 INFO:teuthology.orchestra.run.smithi143.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-08T19:23:59.715 INFO:teuthology.orchestra.run.smithi143.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-08T19:23:59.715 INFO:teuthology.orchestra.run.smithi143.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-08T19:23:59.715 INFO:teuthology.orchestra.run.smithi143.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-08T19:24:00.141 INFO:teuthology.orchestra.run.smithi143.stdout:Discarding blocks...Done. 2023-11-08T19:24:00.143 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi143.front.sepia.ceph.com -o noatime 2023-11-08T19:24:00.143 DEBUG:teuthology.orchestra.run.smithi143:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-11-08T19:24:00.302 DEBUG:teuthology.orchestra.run.smithi143:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-11-08T19:24:00.335 DEBUG:teuthology.orchestra.run.smithi143:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-11-08T19:24:00.401 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-08T19:24:00.402 INFO:tasks.ceph:role: osd.1 2023-11-08T19:24:00.402 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi143.front.sepia.ceph.com 2023-11-08T19:24:00.402 DEBUG:teuthology.orchestra.run.smithi143:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-11-08T19:24:00.476 INFO:teuthology.orchestra.run.smithi143.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-11-08T19:24:00.476 INFO:teuthology.orchestra.run.smithi143.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-08T19:24:00.476 INFO:teuthology.orchestra.run.smithi143.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-08T19:24:00.477 INFO:teuthology.orchestra.run.smithi143.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-08T19:24:00.477 INFO:teuthology.orchestra.run.smithi143.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-08T19:24:00.477 INFO:teuthology.orchestra.run.smithi143.stdout: = sunit=0 swidth=0 blks 2023-11-08T19:24:00.477 INFO:teuthology.orchestra.run.smithi143.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-08T19:24:00.477 INFO:teuthology.orchestra.run.smithi143.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-08T19:24:00.478 INFO:teuthology.orchestra.run.smithi143.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-08T19:24:00.478 INFO:teuthology.orchestra.run.smithi143.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-08T19:24:00.903 INFO:teuthology.orchestra.run.smithi143.stdout:Discarding blocks...Done. 2023-11-08T19:24:00.905 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi143.front.sepia.ceph.com -o noatime 2023-11-08T19:24:00.905 DEBUG:teuthology.orchestra.run.smithi143:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-11-08T19:24:01.015 DEBUG:teuthology.orchestra.run.smithi143:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-11-08T19:24:01.048 DEBUG:teuthology.orchestra.run.smithi143:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-11-08T19:24:01.115 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-08T19:24:01.116 INFO:tasks.ceph:role: osd.2 2023-11-08T19:24:01.116 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi143.front.sepia.ceph.com 2023-11-08T19:24:01.116 DEBUG:teuthology.orchestra.run.smithi143:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-11-08T19:24:01.190 INFO:teuthology.orchestra.run.smithi143.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-11-08T19:24:01.191 INFO:teuthology.orchestra.run.smithi143.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-08T19:24:01.191 INFO:teuthology.orchestra.run.smithi143.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-08T19:24:01.191 INFO:teuthology.orchestra.run.smithi143.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-08T19:24:01.191 INFO:teuthology.orchestra.run.smithi143.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-08T19:24:01.192 INFO:teuthology.orchestra.run.smithi143.stdout: = sunit=0 swidth=0 blks 2023-11-08T19:24:01.192 INFO:teuthology.orchestra.run.smithi143.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-08T19:24:01.192 INFO:teuthology.orchestra.run.smithi143.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-08T19:24:01.192 INFO:teuthology.orchestra.run.smithi143.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-08T19:24:01.192 INFO:teuthology.orchestra.run.smithi143.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-08T19:24:01.638 INFO:teuthology.orchestra.run.smithi143.stdout:Discarding blocks...Done. 2023-11-08T19:24:01.640 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi143.front.sepia.ceph.com -o noatime 2023-11-08T19:24:01.641 DEBUG:teuthology.orchestra.run.smithi143:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-11-08T19:24:01.763 DEBUG:teuthology.orchestra.run.smithi143:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-11-08T19:24:01.797 DEBUG:teuthology.orchestra.run.smithi143:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-11-08T19:24:01.865 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-08T19:24:01.865 INFO:tasks.ceph:role: osd.3 2023-11-08T19:24:01.866 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi143.front.sepia.ceph.com 2023-11-08T19:24:01.866 DEBUG:teuthology.orchestra.run.smithi143:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-11-08T19:24:01.941 INFO:teuthology.orchestra.run.smithi143.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-11-08T19:24:01.941 INFO:teuthology.orchestra.run.smithi143.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-08T19:24:01.941 INFO:teuthology.orchestra.run.smithi143.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-08T19:24:01.942 INFO:teuthology.orchestra.run.smithi143.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-08T19:24:01.942 INFO:teuthology.orchestra.run.smithi143.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-08T19:24:01.942 INFO:teuthology.orchestra.run.smithi143.stdout: = sunit=0 swidth=0 blks 2023-11-08T19:24:01.942 INFO:teuthology.orchestra.run.smithi143.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-08T19:24:01.942 INFO:teuthology.orchestra.run.smithi143.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-08T19:24:01.943 INFO:teuthology.orchestra.run.smithi143.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-08T19:24:01.943 INFO:teuthology.orchestra.run.smithi143.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-08T19:24:02.387 INFO:teuthology.orchestra.run.smithi143.stdout:Discarding blocks...Done. 2023-11-08T19:24:02.388 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi143.front.sepia.ceph.com -o noatime 2023-11-08T19:24:02.389 DEBUG:teuthology.orchestra.run.smithi143:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-11-08T19:24:02.548 DEBUG:teuthology.orchestra.run.smithi143:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-11-08T19:24:02.584 DEBUG:teuthology.orchestra.run.smithi143:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-08T19:24:02.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:02.694+0000 7fc91be84640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-11-08T19:24:02.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:02.695+0000 7fc91be84640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-11-08T19:24:02.697 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:02.695+0000 7fc91be84640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-11-08T19:24:02.697 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:02.695+0000 7fc91be84640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-11-08T19:24:02.697 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:02.695+0000 7fc91be84640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-11-08T19:24:02.697 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:02.695+0000 7fc91be84640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-11-08T19:24:05.575 DEBUG:teuthology.orchestra.run.smithi143:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-11-08T19:24:05.606 DEBUG:teuthology.orchestra.run.smithi143:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-08T19:24:05.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:05.697+0000 7f022474f640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-11-08T19:24:05.699 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:05.697+0000 7f022474f640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-11-08T19:24:05.699 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:05.697+0000 7f022474f640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-11-08T19:24:05.699 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:05.697+0000 7f022474f640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-11-08T19:24:05.699 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:05.697+0000 7f022474f640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-11-08T19:24:05.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:05.697+0000 7f022474f640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-11-08T19:24:08.569 DEBUG:teuthology.orchestra.run.smithi143:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-11-08T19:24:08.601 DEBUG:teuthology.orchestra.run.smithi143:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-08T19:24:08.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:08.713+0000 7fdeedec4640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-11-08T19:24:08.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:08.713+0000 7fdeedec4640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-11-08T19:24:08.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:08.713+0000 7fdeedec4640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-11-08T19:24:08.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:08.713+0000 7fdeedec4640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-11-08T19:24:08.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:08.713+0000 7fdeedec4640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-11-08T19:24:08.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:08.714+0000 7fdeedec4640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-11-08T19:24:11.584 DEBUG:teuthology.orchestra.run.smithi143:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-11-08T19:24:11.616 DEBUG:teuthology.orchestra.run.smithi143:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-08T19:24:11.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:11.706+0000 7f4943beb640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-11-08T19:24:11.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:11.706+0000 7f4943beb640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-11-08T19:24:11.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:11.706+0000 7f4943beb640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-11-08T19:24:11.709 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:11.706+0000 7f4943beb640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-11-08T19:24:11.709 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:11.706+0000 7f4943beb640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-11-08T19:24:11.709 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:11.707+0000 7f4943beb640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-11-08T19:24:14.587 DEBUG:teuthology.orchestra.run.smithi143:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-11-08T19:24:14.619 DEBUG:teuthology.orchestra.run.smithi174:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-11-08T19:24:14.651 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-08T19:24:14.652 INFO:tasks.ceph:role: osd.4 2023-11-08T19:24:14.652 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi174.front.sepia.ceph.com 2023-11-08T19:24:14.653 DEBUG:teuthology.orchestra.run.smithi174:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-11-08T19:24:14.728 INFO:teuthology.orchestra.run.smithi174.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-11-08T19:24:14.728 INFO:teuthology.orchestra.run.smithi174.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-08T19:24:14.729 INFO:teuthology.orchestra.run.smithi174.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-08T19:24:14.729 INFO:teuthology.orchestra.run.smithi174.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-08T19:24:14.729 INFO:teuthology.orchestra.run.smithi174.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-08T19:24:14.729 INFO:teuthology.orchestra.run.smithi174.stdout: = sunit=0 swidth=0 blks 2023-11-08T19:24:14.729 INFO:teuthology.orchestra.run.smithi174.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-08T19:24:14.730 INFO:teuthology.orchestra.run.smithi174.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-08T19:24:14.730 INFO:teuthology.orchestra.run.smithi174.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-08T19:24:14.730 INFO:teuthology.orchestra.run.smithi174.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-08T19:24:15.156 INFO:teuthology.orchestra.run.smithi174.stdout:Discarding blocks...Done. 2023-11-08T19:24:15.158 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi174.front.sepia.ceph.com -o noatime 2023-11-08T19:24:15.158 DEBUG:teuthology.orchestra.run.smithi174:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-11-08T19:24:15.343 DEBUG:teuthology.orchestra.run.smithi174:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-11-08T19:24:15.385 DEBUG:teuthology.orchestra.run.smithi174:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-11-08T19:24:15.456 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-08T19:24:15.457 INFO:tasks.ceph:role: osd.5 2023-11-08T19:24:15.457 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi174.front.sepia.ceph.com 2023-11-08T19:24:15.457 DEBUG:teuthology.orchestra.run.smithi174:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-11-08T19:24:15.533 INFO:teuthology.orchestra.run.smithi174.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-11-08T19:24:15.533 INFO:teuthology.orchestra.run.smithi174.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-08T19:24:15.533 INFO:teuthology.orchestra.run.smithi174.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-08T19:24:15.533 INFO:teuthology.orchestra.run.smithi174.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-08T19:24:15.534 INFO:teuthology.orchestra.run.smithi174.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-08T19:24:15.534 INFO:teuthology.orchestra.run.smithi174.stdout: = sunit=0 swidth=0 blks 2023-11-08T19:24:15.534 INFO:teuthology.orchestra.run.smithi174.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-08T19:24:15.534 INFO:teuthology.orchestra.run.smithi174.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-08T19:24:15.534 INFO:teuthology.orchestra.run.smithi174.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-08T19:24:15.535 INFO:teuthology.orchestra.run.smithi174.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-08T19:24:15.953 INFO:teuthology.orchestra.run.smithi174.stdout:Discarding blocks...Done. 2023-11-08T19:24:15.955 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi174.front.sepia.ceph.com -o noatime 2023-11-08T19:24:15.956 DEBUG:teuthology.orchestra.run.smithi174:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-11-08T19:24:16.068 DEBUG:teuthology.orchestra.run.smithi174:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-11-08T19:24:16.101 DEBUG:teuthology.orchestra.run.smithi174:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-11-08T19:24:16.171 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-08T19:24:16.172 INFO:tasks.ceph:role: osd.6 2023-11-08T19:24:16.172 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi174.front.sepia.ceph.com 2023-11-08T19:24:16.173 DEBUG:teuthology.orchestra.run.smithi174:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-11-08T19:24:16.248 INFO:teuthology.orchestra.run.smithi174.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-11-08T19:24:16.248 INFO:teuthology.orchestra.run.smithi174.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-08T19:24:16.249 INFO:teuthology.orchestra.run.smithi174.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-08T19:24:16.249 INFO:teuthology.orchestra.run.smithi174.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-08T19:24:16.249 INFO:teuthology.orchestra.run.smithi174.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-08T19:24:16.249 INFO:teuthology.orchestra.run.smithi174.stdout: = sunit=0 swidth=0 blks 2023-11-08T19:24:16.249 INFO:teuthology.orchestra.run.smithi174.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-08T19:24:16.250 INFO:teuthology.orchestra.run.smithi174.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-08T19:24:16.250 INFO:teuthology.orchestra.run.smithi174.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-08T19:24:16.250 INFO:teuthology.orchestra.run.smithi174.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-08T19:24:16.702 INFO:teuthology.orchestra.run.smithi174.stdout:Discarding blocks...Done. 2023-11-08T19:24:16.703 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi174.front.sepia.ceph.com -o noatime 2023-11-08T19:24:16.703 DEBUG:teuthology.orchestra.run.smithi174:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-11-08T19:24:16.866 DEBUG:teuthology.orchestra.run.smithi174:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-11-08T19:24:16.906 DEBUG:teuthology.orchestra.run.smithi174:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-11-08T19:24:16.976 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-08T19:24:16.976 INFO:tasks.ceph:role: osd.7 2023-11-08T19:24:16.976 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi174.front.sepia.ceph.com 2023-11-08T19:24:16.977 DEBUG:teuthology.orchestra.run.smithi174:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-11-08T19:24:17.052 INFO:teuthology.orchestra.run.smithi174.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-11-08T19:24:17.052 INFO:teuthology.orchestra.run.smithi174.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-08T19:24:17.053 INFO:teuthology.orchestra.run.smithi174.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-08T19:24:17.053 INFO:teuthology.orchestra.run.smithi174.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-08T19:24:17.053 INFO:teuthology.orchestra.run.smithi174.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-08T19:24:17.053 INFO:teuthology.orchestra.run.smithi174.stdout: = sunit=0 swidth=0 blks 2023-11-08T19:24:17.054 INFO:teuthology.orchestra.run.smithi174.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-08T19:24:17.054 INFO:teuthology.orchestra.run.smithi174.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-08T19:24:17.054 INFO:teuthology.orchestra.run.smithi174.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-08T19:24:17.054 INFO:teuthology.orchestra.run.smithi174.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-08T19:24:17.489 INFO:teuthology.orchestra.run.smithi174.stdout:Discarding blocks...Done. 2023-11-08T19:24:17.491 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi174.front.sepia.ceph.com -o noatime 2023-11-08T19:24:17.491 DEBUG:teuthology.orchestra.run.smithi174:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-11-08T19:24:17.669 DEBUG:teuthology.orchestra.run.smithi174:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-11-08T19:24:17.707 DEBUG:teuthology.orchestra.run.smithi174:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-08T19:24:17.820 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:17.818+0000 7f2978485640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-11-08T19:24:17.820 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:17.818+0000 7f2978485640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-11-08T19:24:17.820 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:17.818+0000 7f2978485640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-11-08T19:24:17.820 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:17.818+0000 7f2978485640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-11-08T19:24:17.821 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:17.818+0000 7f2978485640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-11-08T19:24:17.821 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:17.818+0000 7f2978485640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-11-08T19:24:20.696 DEBUG:teuthology.orchestra.run.smithi174:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-11-08T19:24:20.727 DEBUG:teuthology.orchestra.run.smithi174:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-08T19:24:20.820 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:20.817+0000 7f7bdfdaf640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-11-08T19:24:20.820 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:20.818+0000 7f7bdfdaf640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-11-08T19:24:20.820 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:20.818+0000 7f7bdfdaf640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-11-08T19:24:20.821 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:20.818+0000 7f7bdfdaf640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-11-08T19:24:20.821 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:20.818+0000 7f7bdfdaf640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-11-08T19:24:20.821 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:20.818+0000 7f7bdfdaf640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-11-08T19:24:23.687 DEBUG:teuthology.orchestra.run.smithi174:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-11-08T19:24:23.718 DEBUG:teuthology.orchestra.run.smithi174:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-08T19:24:23.809 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:23.807+0000 7f0aa7042640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-11-08T19:24:23.810 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:23.807+0000 7f0aa7042640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-11-08T19:24:23.810 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:23.807+0000 7f0aa7042640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-11-08T19:24:23.810 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:23.807+0000 7f0aa7042640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-11-08T19:24:23.810 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:23.807+0000 7f0aa7042640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-11-08T19:24:23.811 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:23.808+0000 7f0aa7042640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-11-08T19:24:26.368 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T19:24:26.372 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T19:24:26.697 DEBUG:teuthology.orchestra.run.smithi174:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-11-08T19:24:26.728 DEBUG:teuthology.orchestra.run.smithi174:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-08T19:24:26.820 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:26.818+0000 7f78ed754640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-11-08T19:24:26.821 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:26.819+0000 7f78ed754640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-11-08T19:24:26.821 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:26.819+0000 7f78ed754640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-11-08T19:24:26.821 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:26.819+0000 7f78ed754640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-11-08T19:24:26.821 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:26.819+0000 7f78ed754640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-11-08T19:24:26.822 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:26.819+0000 7f78ed754640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-11-08T19:24:29.693 DEBUG:teuthology.orchestra.run.smithi174:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-11-08T19:24:29.726 INFO:tasks.ceph:Reading keys from all nodes... 2023-11-08T19:24:29.727 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-08T19:24:29.727 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-11-08T19:24:29.760 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-08T19:24:29.760 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-11-08T19:24:29.832 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-08T19:24:29.832 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-11-08T19:24:29.902 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-08T19:24:29.902 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-11-08T19:24:29.971 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-08T19:24:29.971 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-11-08T19:24:30.039 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-08T19:24:30.040 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-11-08T19:24:30.107 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-08T19:24:30.108 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-11-08T19:24:30.177 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2023-11-08T19:24:30.177 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-11-08T19:24:30.210 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2023-11-08T19:24:30.210 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-11-08T19:24:30.279 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2023-11-08T19:24:30.279 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-11-08T19:24:30.348 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2023-11-08T19:24:30.349 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-11-08T19:24:30.416 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2023-11-08T19:24:30.417 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-11-08T19:24:30.484 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2023-11-08T19:24:30.484 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-11-08T19:24:30.551 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2023-11-08T19:24:30.552 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-11-08T19:24:30.618 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-08T19:24:30.618 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-11-08T19:24:30.639 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-08T19:24:30.639 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2023-11-08T19:24:30.698 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2023-11-08T19:24:30.699 DEBUG:teuthology.orchestra.run.smithi174:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-11-08T19:24:30.716 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2023-11-08T19:24:30.716 DEBUG:teuthology.orchestra.run.smithi174:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2023-11-08T19:24:30.773 INFO:tasks.ceph:Adding keys to all mons... 2023-11-08T19:24:30.774 DEBUG:teuthology.orchestra.run.smithi143:> sudo tee -a /etc/ceph/ceph.keyring 2023-11-08T19:24:30.776 DEBUG:teuthology.orchestra.run.smithi174:> sudo tee -a /etc/ceph/ceph.keyring 2023-11-08T19:24:30.819 INFO:teuthology.orchestra.run.smithi143.stdout:[mgr.y] 2023-11-08T19:24:30.819 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQBO4Etl4Gl4MxAArxdUnjLDhTLO3feXHV4HyQ== 2023-11-08T19:24:30.819 INFO:teuthology.orchestra.run.smithi143.stdout:[mds.a] 2023-11-08T19:24:30.820 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQBO4EtlqQPkOhAACwt3IOZLFtEQle9/FFliYA== 2023-11-08T19:24:30.820 INFO:teuthology.orchestra.run.smithi143.stdout:[mds.c] 2023-11-08T19:24:30.820 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQBP4EtlGEeDBhAAihr82JiBJZR0rRkQj2n8wA== 2023-11-08T19:24:30.820 INFO:teuthology.orchestra.run.smithi143.stdout:[osd.0] 2023-11-08T19:24:30.820 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQBS4EtloVd4KRAAi5nj/vB9UK/mWZ9lkaFAxg== 2023-11-08T19:24:30.821 INFO:teuthology.orchestra.run.smithi143.stdout:[osd.1] 2023-11-08T19:24:30.821 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQBV4EtlWlOaKRAAooz5P+AoTgyuL/QAv4REKA== 2023-11-08T19:24:30.821 INFO:teuthology.orchestra.run.smithi143.stdout:[osd.2] 2023-11-08T19:24:30.821 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQBY4EtlBpaWKhAAmob9TfEKw5x6dEIu8ALVGA== 2023-11-08T19:24:30.821 INFO:teuthology.orchestra.run.smithi143.stdout:[osd.3] 2023-11-08T19:24:30.821 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQBb4Etl5pkrKhAA38ATMC3NkZ4plRRA69i0vw== 2023-11-08T19:24:30.822 INFO:teuthology.orchestra.run.smithi143.stdout:[mgr.x] 2023-11-08T19:24:30.822 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQBO4EtlzrqBNxAA5rtcgkzroSTKuBBmeQRiig== 2023-11-08T19:24:30.822 INFO:teuthology.orchestra.run.smithi143.stdout:[mds.b] 2023-11-08T19:24:30.822 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQBP4EtlHJqqCxAAxgT7Ce6QICWZ0qebHWZhMQ== 2023-11-08T19:24:30.823 INFO:teuthology.orchestra.run.smithi143.stdout:[mds.d] 2023-11-08T19:24:30.823 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQBP4EtlW5//EhAArVOkzNpCYd6Skl6eh5EYUQ== 2023-11-08T19:24:30.823 INFO:teuthology.orchestra.run.smithi143.stdout:[osd.4] 2023-11-08T19:24:30.823 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQBh4EtlqLnUMBAABIrSskahtrZR0ClfczX5Ag== 2023-11-08T19:24:30.823 INFO:teuthology.orchestra.run.smithi143.stdout:[osd.5] 2023-11-08T19:24:30.824 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQBk4Etlt9PTMBAAz55IHg6Jrin3BUAvhOM/Fg== 2023-11-08T19:24:30.824 INFO:teuthology.orchestra.run.smithi143.stdout:[osd.6] 2023-11-08T19:24:30.824 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQBn4EtlcTkyMBAA5fD3h4s+99rgd6fqHHCFMw== 2023-11-08T19:24:30.824 INFO:teuthology.orchestra.run.smithi143.stdout:[osd.7] 2023-11-08T19:24:30.824 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQBq4EtldEviMBAAyGYrHVGwxHdmtDOwWCSHDQ== 2023-11-08T19:24:30.825 INFO:teuthology.orchestra.run.smithi143.stdout:[client.0] 2023-11-08T19:24:30.825 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQBP4Etl0GdSFxAAVkuHL0EnjrtTUMkj/afPkg== 2023-11-08T19:24:30.825 INFO:teuthology.orchestra.run.smithi143.stdout:[client.2] 2023-11-08T19:24:30.825 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQBP4Etlm+8SHRAAAwxcy2EzUvtFw9/mTzyk7w== 2023-11-08T19:24:30.825 INFO:teuthology.orchestra.run.smithi143.stdout:[client.1] 2023-11-08T19:24:30.826 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQBP4EtlPQ5yIBAAiyQaqCJph2KOPBZPSIh4ZQ== 2023-11-08T19:24:30.826 INFO:teuthology.orchestra.run.smithi143.stdout:[client.3] 2023-11-08T19:24:30.826 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQBP4EtlNDC8IxAACBkCM+3TXhJv5eE9xLvwKA== 2023-11-08T19:24:30.838 INFO:teuthology.orchestra.run.smithi174.stdout:[mgr.y] 2023-11-08T19:24:30.838 INFO:teuthology.orchestra.run.smithi174.stdout: key = AQBO4Etl4Gl4MxAArxdUnjLDhTLO3feXHV4HyQ== 2023-11-08T19:24:30.839 INFO:teuthology.orchestra.run.smithi174.stdout:[mds.a] 2023-11-08T19:24:30.839 INFO:teuthology.orchestra.run.smithi174.stdout: key = AQBO4EtlqQPkOhAACwt3IOZLFtEQle9/FFliYA== 2023-11-08T19:24:30.839 INFO:teuthology.orchestra.run.smithi174.stdout:[mds.c] 2023-11-08T19:24:30.839 INFO:teuthology.orchestra.run.smithi174.stdout: key = AQBP4EtlGEeDBhAAihr82JiBJZR0rRkQj2n8wA== 2023-11-08T19:24:30.839 INFO:teuthology.orchestra.run.smithi174.stdout:[osd.0] 2023-11-08T19:24:30.840 INFO:teuthology.orchestra.run.smithi174.stdout: key = AQBS4EtloVd4KRAAi5nj/vB9UK/mWZ9lkaFAxg== 2023-11-08T19:24:30.840 INFO:teuthology.orchestra.run.smithi174.stdout:[osd.1] 2023-11-08T19:24:30.840 INFO:teuthology.orchestra.run.smithi174.stdout: key = AQBV4EtlWlOaKRAAooz5P+AoTgyuL/QAv4REKA== 2023-11-08T19:24:30.840 INFO:teuthology.orchestra.run.smithi174.stdout:[osd.2] 2023-11-08T19:24:30.840 INFO:teuthology.orchestra.run.smithi174.stdout: key = AQBY4EtlBpaWKhAAmob9TfEKw5x6dEIu8ALVGA== 2023-11-08T19:24:30.841 INFO:teuthology.orchestra.run.smithi174.stdout:[osd.3] 2023-11-08T19:24:30.841 INFO:teuthology.orchestra.run.smithi174.stdout: key = AQBb4Etl5pkrKhAA38ATMC3NkZ4plRRA69i0vw== 2023-11-08T19:24:30.841 INFO:teuthology.orchestra.run.smithi174.stdout:[mgr.x] 2023-11-08T19:24:30.841 INFO:teuthology.orchestra.run.smithi174.stdout: key = AQBO4EtlzrqBNxAA5rtcgkzroSTKuBBmeQRiig== 2023-11-08T19:24:30.841 INFO:teuthology.orchestra.run.smithi174.stdout:[mds.b] 2023-11-08T19:24:30.842 INFO:teuthology.orchestra.run.smithi174.stdout: key = AQBP4EtlHJqqCxAAxgT7Ce6QICWZ0qebHWZhMQ== 2023-11-08T19:24:30.842 INFO:teuthology.orchestra.run.smithi174.stdout:[mds.d] 2023-11-08T19:24:30.842 INFO:teuthology.orchestra.run.smithi174.stdout: key = AQBP4EtlW5//EhAArVOkzNpCYd6Skl6eh5EYUQ== 2023-11-08T19:24:30.842 INFO:teuthology.orchestra.run.smithi174.stdout:[osd.4] 2023-11-08T19:24:30.842 INFO:teuthology.orchestra.run.smithi174.stdout: key = AQBh4EtlqLnUMBAABIrSskahtrZR0ClfczX5Ag== 2023-11-08T19:24:30.843 INFO:teuthology.orchestra.run.smithi174.stdout:[osd.5] 2023-11-08T19:24:30.843 INFO:teuthology.orchestra.run.smithi174.stdout: key = AQBk4Etlt9PTMBAAz55IHg6Jrin3BUAvhOM/Fg== 2023-11-08T19:24:30.843 INFO:teuthology.orchestra.run.smithi174.stdout:[osd.6] 2023-11-08T19:24:30.843 INFO:teuthology.orchestra.run.smithi174.stdout: key = AQBn4EtlcTkyMBAA5fD3h4s+99rgd6fqHHCFMw== 2023-11-08T19:24:30.843 INFO:teuthology.orchestra.run.smithi174.stdout:[osd.7] 2023-11-08T19:24:30.843 INFO:teuthology.orchestra.run.smithi174.stdout: key = AQBq4EtldEviMBAAyGYrHVGwxHdmtDOwWCSHDQ== 2023-11-08T19:24:30.844 INFO:teuthology.orchestra.run.smithi174.stdout:[client.0] 2023-11-08T19:24:30.844 INFO:teuthology.orchestra.run.smithi174.stdout: key = AQBP4Etl0GdSFxAAVkuHL0EnjrtTUMkj/afPkg== 2023-11-08T19:24:30.844 INFO:teuthology.orchestra.run.smithi174.stdout:[client.2] 2023-11-08T19:24:30.844 INFO:teuthology.orchestra.run.smithi174.stdout: key = AQBP4Etlm+8SHRAAAwxcy2EzUvtFw9/mTzyk7w== 2023-11-08T19:24:30.844 INFO:teuthology.orchestra.run.smithi174.stdout:[client.1] 2023-11-08T19:24:30.845 INFO:teuthology.orchestra.run.smithi174.stdout: key = AQBP4EtlPQ5yIBAAiyQaqCJph2KOPBZPSIh4ZQ== 2023-11-08T19:24:30.845 INFO:teuthology.orchestra.run.smithi174.stdout:[client.3] 2023-11-08T19:24:30.845 INFO:teuthology.orchestra.run.smithi174.stdout: key = AQBP4EtlNDC8IxAACBkCM+3TXhJv5eE9xLvwKA== 2023-11-08T19:24:30.846 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-08T19:24:30.869 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-08T19:24:30.939 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-08T19:24:30.942 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-08T19:24:30.998 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-08T19:24:31.000 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-08T19:24:31.056 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-08T19:24:31.059 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-08T19:24:31.112 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-08T19:24:31.114 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-08T19:24:31.167 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-08T19:24:31.170 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-08T19:24:31.223 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-08T19:24:31.226 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-08T19:24:31.279 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-08T19:24:31.281 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-08T19:24:31.335 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-08T19:24:31.337 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-08T19:24:31.393 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-08T19:24:31.396 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-08T19:24:31.452 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-08T19:24:31.455 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-08T19:24:31.508 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-08T19:24:31.511 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-08T19:24:31.565 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-08T19:24:31.568 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-08T19:24:31.622 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-08T19:24:31.624 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-08T19:24:31.677 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-08T19:24:31.680 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-08T19:24:31.736 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-08T19:24:31.739 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-08T19:24:31.795 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-08T19:24:31.797 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-08T19:24:31.853 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-08T19:24:31.856 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-08T19:24:31.912 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-11-08T19:24:31.913 DEBUG:teuthology.orchestra.run.smithi143:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-11-08T19:24:31.939 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-11-08T19:24:32.037 DEBUG:teuthology.orchestra.run.smithi143:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-11-08T19:24:32.064 DEBUG:teuthology.orchestra.run.smithi174:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-11-08T19:24:32.091 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-11-08T19:24:32.190 DEBUG:teuthology.orchestra.run.smithi174:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-11-08T19:24:32.215 DEBUG:teuthology.orchestra.run.smithi174:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-11-08T19:24:32.282 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-11-08T19:24:32.380 DEBUG:teuthology.orchestra.run.smithi174:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-11-08T19:24:32.406 DEBUG:teuthology.orchestra.run.smithi143:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-11-08T19:24:32.408 DEBUG:teuthology.orchestra.run.smithi174:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-11-08T19:24:32.463 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-11-08T19:24:32.463 INFO:tasks.ceph.mon.a:Restarting daemon 2023-11-08T19:24:32.464 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-11-08T19:24:32.466 INFO:tasks.ceph.mon.a:Started 2023-11-08T19:24:32.466 INFO:tasks.ceph.mon.b:Restarting daemon 2023-11-08T19:24:32.467 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-11-08T19:24:32.505 INFO:tasks.ceph.mon.b:Started 2023-11-08T19:24:32.506 INFO:tasks.ceph.mon.c:Restarting daemon 2023-11-08T19:24:32.506 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-11-08T19:24:32.508 INFO:tasks.ceph.mon.c:Started 2023-11-08T19:24:32.509 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-11-08T19:24:32.509 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-11-08T19:24:32.509 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-11-08T19:24:32.512 INFO:tasks.ceph.mgr.y:Started 2023-11-08T19:24:32.512 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-11-08T19:24:32.512 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-11-08T19:24:32.514 INFO:tasks.ceph.mgr.x:Started 2023-11-08T19:24:32.515 INFO:tasks.ceph:Setting crush tunables to default 2023-11-08T19:24:32.515 DEBUG:teuthology.orchestra.run.smithi143:> sudo ceph --cluster ceph osd crush tunables default 2023-11-08T19:24:32.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:32.636+0000 7f119298c700 1 Processor -- start 2023-11-08T19:24:32.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:32.636+0000 7f119298c700 1 -- start start 2023-11-08T19:24:32.639 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:32.636+0000 7f119298c700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f118c0d1d80 0x7f118c0b2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:32.639 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:32.637+0000 7f119298c700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f118c0b1730 0x7f118c0cc4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:32.640 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:32.637+0000 7f119298c700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f118c0d0420 con 0x7f118c15bb20 2023-11-08T19:24:32.640 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:32.637+0000 7f119298c700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f118c0cf600 con 0x7f118c0d1d80 2023-11-08T19:24:32.640 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:32.637+0000 7f119298c700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f118c0cf810 con 0x7f118c0b1730 2023-11-08T19:24:32.640 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:32.637+0000 7f118b7fe700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f118c0d1d80 0x7f118c0b2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:32.641 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:32.637+0000 7f118bfff700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f118c15bb20 0x7f118c0d1670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:44212/0 (socket says 172.21.15.143:44212) 2023-11-08T19:24:32.641 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:32.637+0000 7f118b7fe700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f118c0d1d80 0x7f118c0b2670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48060/0 (socket says 172.21.15.143:48060) 2023-11-08T19:24:32.641 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:32.637+0000 7f118bfff700 1 -- 172.21.15.143:0/4098029018 learned_addr learned my addr 172.21.15.143:0/4098029018 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:24:32.641 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:32.637+0000 7f118b7fe700 1 -- 172.21.15.143:0/4098029018 >> v2:172.21.15.143:3300/0 conn(0x7f118c0d1d80 msgr2=0x7f118c0b2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-11-08T19:24:32.641 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:32.637+0000 7f118b7fe700 1 -- 172.21.15.143:0/4098029018 >> v2:172.21.15.143:3300/0 conn(0x7f118c0d1d80 msgr2=0x7f118c0b2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-08T19:24:32.642 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:32.637+0000 7f118b7fe700 1 --2- 172.21.15.143:0/4098029018 >> v2:172.21.15.143:3300/0 conn(0x7f118c0d1d80 0x7f118c0b2670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-08T19:24:32.642 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:32.637+0000 7f1190f29700 1 --2- 172.21.15.143:0/4098029018 >> v2:172.21.15.174:3300/0 conn(0x7f118c0b1730 0x7f118c0cc4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:32.642 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:32.637+0000 7f118b7fe700 1 --2- 172.21.15.143:0/4098029018 >> v2:172.21.15.143:3300/0 conn(0x7f118c0d1d80 0x7f118c0b2670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-08T19:24:32.642 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:32.637+0000 7f1190f29700 1 -- 172.21.15.143:0/4098029018 >> v2:172.21.15.174:3300/0 conn(0x7f118c0b1730 msgr2=0x7f118c0cc4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-11-08T19:24:32.642 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:32.637+0000 7f1190f29700 1 -- 172.21.15.143:0/4098029018 >> v2:172.21.15.174:3300/0 conn(0x7f118c0b1730 msgr2=0x7f118c0cc4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-08T19:24:32.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:32.637+0000 7f1190f29700 1 --2- 172.21.15.143:0/4098029018 >> v2:172.21.15.174:3300/0 conn(0x7f118c0b1730 0x7f118c0cc4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-08T19:24:32.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:32.637+0000 7f1190f29700 1 --2- 172.21.15.143:0/4098029018 >> v2:172.21.15.174:3300/0 conn(0x7f118c0b1730 0x7f118c0cc4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-08T19:24:32.839 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:32.838+0000 7f118b7fe700 1 --2- 172.21.15.143:0/4098029018 >> v2:172.21.15.143:3300/0 conn(0x7f118c0d1d80 0x7f118c0b2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:32.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:32.838+0000 7f1190f29700 1 --2- 172.21.15.143:0/4098029018 >> v2:172.21.15.174:3300/0 conn(0x7f118c0b1730 0x7f118c0cc4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:32.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:32.838+0000 7f118b7fe700 1 -- 172.21.15.143:0/4098029018 >> v2:172.21.15.143:3300/0 conn(0x7f118c0d1d80 msgr2=0x7f118c0b2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-11-08T19:24:32.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:32.838+0000 7f118b7fe700 1 -- 172.21.15.143:0/4098029018 >> v2:172.21.15.143:3300/0 conn(0x7f118c0d1d80 msgr2=0x7f118c0b2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-08T19:24:32.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:32.838+0000 7f118b7fe700 1 --2- 172.21.15.143:0/4098029018 >> v2:172.21.15.143:3300/0 conn(0x7f118c0d1d80 0x7f118c0b2670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-08T19:24:32.841 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:32.838+0000 7f1190f29700 1 -- 172.21.15.143:0/4098029018 >> v2:172.21.15.174:3300/0 conn(0x7f118c0b1730 msgr2=0x7f118c0cc4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-11-08T19:24:32.841 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:32.838+0000 7f118b7fe700 1 --2- 172.21.15.143:0/4098029018 >> v2:172.21.15.143:3300/0 conn(0x7f118c0d1d80 0x7f118c0b2670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-08T19:24:32.841 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:32.838+0000 7f1190f29700 1 -- 172.21.15.143:0/4098029018 >> v2:172.21.15.174:3300/0 conn(0x7f118c0b1730 msgr2=0x7f118c0cc4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-08T19:24:32.841 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:32.838+0000 7f1190f29700 1 --2- 172.21.15.143:0/4098029018 >> v2:172.21.15.174:3300/0 conn(0x7f118c0b1730 0x7f118c0cc4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-08T19:24:32.842 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:32.838+0000 7f1190f29700 1 --2- 172.21.15.143:0/4098029018 >> v2:172.21.15.174:3300/0 conn(0x7f118c0b1730 0x7f118c0cc4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-08T19:24:32.854 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:32.850+0000 7f118affd700 1 -- 172.21.15.143:0/4098029018 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4284738065 0 0) 0x7f118c0d0420 con 0x7f118c15bb20 2023-11-08T19:24:32.854 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:32.850+0000 7f118affd700 1 -- 172.21.15.143:0/4098029018 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f11740035b0 con 0x7f118c15bb20 2023-11-08T19:24:32.854 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:32.850+0000 7f118affd700 1 -- 172.21.15.143:0/4098029018 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1873164893 0 0) 0x7f11740035b0 con 0x7f118c15bb20 2023-11-08T19:24:32.855 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:32.850+0000 7f118affd700 1 -- 172.21.15.143:0/4098029018 >> v2:172.21.15.174:3300/0 conn(0x7f118c0b1730 msgr2=0x7f118c0cc4d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-08T19:24:32.855 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:32.850+0000 7f118affd700 1 --2- 172.21.15.143:0/4098029018 >> v2:172.21.15.174:3300/0 conn(0x7f118c0b1730 0x7f118c0cc4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:32.855 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:32.850+0000 7f118affd700 1 -- 172.21.15.143:0/4098029018 >> v2:172.21.15.143:3300/0 conn(0x7f118c0d1d80 msgr2=0x7f118c0b2670 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-08T19:24:32.855 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:32.850+0000 7f118affd700 1 --2- 172.21.15.143:0/4098029018 >> v2:172.21.15.143:3300/0 conn(0x7f118c0d1d80 0x7f118c0b2670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:32.855 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:32.850+0000 7f118affd700 1 -- 172.21.15.143:0/4098029018 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f118c0cfaf0 con 0x7f118c15bb20 2023-11-08T19:24:32.856 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:32.850+0000 7f118affd700 1 -- 172.21.15.143:0/4098029018 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4147756303 0 0) 0x7f117c001670 con 0x7f118c15bb20 2023-11-08T19:24:32.856 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:32.850+0000 7f118affd700 1 -- 172.21.15.143:0/4098029018 >> v1:172.21.15.143:6789/0 conn(0x7f118c15bb20 legacy=0x7f118c0d1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:32.856 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:32.850+0000 7f118affd700 1 --2- 172.21.15.143:0/4098029018 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1174003a00 0x7f1174003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:32.856 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:32.850+0000 7f118affd700 1 --2- 172.21.15.143:0/4098029018 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1174004850 0x7f1174004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:32.856 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:32.850+0000 7f118affd700 1 --2- 172.21.15.143:0/4098029018 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1174005210 0x7f117400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:32.857 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:32.850+0000 7f118affd700 1 -- 172.21.15.143:0/4098029018 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f118c0cf600 con 0x7f1174003a00 2023-11-08T19:24:32.857 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:32.850+0000 7f118affd700 1 -- 172.21.15.143:0/4098029018 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f118c0cf810 con 0x7f1174005210 2023-11-08T19:24:32.857 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:32.850+0000 7f118affd700 1 -- 172.21.15.143:0/4098029018 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f117400c110 con 0x7f1174004850 2023-11-08T19:24:32.857 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:32.850+0000 7f118bfff700 1 --2- 172.21.15.143:0/4098029018 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1174003a00 0x7f1174003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:32.857 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:32.851+0000 7f1190f29700 1 --2- 172.21.15.143:0/4098029018 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1174005210 0x7f117400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:32.858 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:32.851+0000 7f118b7fe700 1 --2- 172.21.15.143:0/4098029018 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1174004850 0x7f1174004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:32.858 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:32.851+0000 7f118bfff700 1 -- 172.21.15.143:0/4098029018 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1174004850 msgr2=0x7f1174004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:24:32.858 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:32.851+0000 7f118bfff700 1 --2- 172.21.15.143:0/4098029018 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1174004850 0x7f1174004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:32.858 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:32.851+0000 7f118bfff700 1 -- 172.21.15.143:0/4098029018 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1174005210 msgr2=0x7f117400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:24:32.858 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:32.851+0000 7f118bfff700 1 --2- 172.21.15.143:0/4098029018 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1174005210 0x7f117400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:32.859 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:32.851+0000 7f1190f29700 1 -- 172.21.15.143:0/4098029018 reap_dead start 2023-11-08T19:24:32.859 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:32.851+0000 7f118bfff700 1 -- 172.21.15.143:0/4098029018 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11740035b0 con 0x7f1174003a00 2023-11-08T19:24:32.859 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:32.851+0000 7f118bfff700 1 --2- 172.21.15.143:0/4098029018 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1174003a00 0x7f1174003e20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f117c00c370 tx=0x7f1174005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75dd566ed5378a3b server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:32.859 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:32.851+0000 7f118affd700 1 -- 172.21.15.143:0/4098029018 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f117c007c60 con 0x7f1174003a00 2023-11-08T19:24:32.859 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:32.851+0000 7f118affd700 1 -- 172.21.15.143:0/4098029018 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f117c00ce30 con 0x7f1174003a00 2023-11-08T19:24:32.860 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:32.851+0000 7f118affd700 1 -- 172.21.15.143:0/4098029018 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f117c005ba0 con 0x7f1174003a00 2023-11-08T19:24:32.860 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:32.852+0000 7f119298c700 1 -- 172.21.15.143:0/4098029018 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1174003a00 msgr2=0x7f1174003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:32.860 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:32.852+0000 7f119298c700 1 --2- 172.21.15.143:0/4098029018 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1174003a00 0x7f1174003e20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f117c00c370 tx=0x7f1174005880 comp rx=0 tx=0).stop 2023-11-08T19:24:32.860 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:32.852+0000 7f119298c700 1 -- 172.21.15.143:0/4098029018 shutdown_connections 2023-11-08T19:24:32.860 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:32.852+0000 7f119298c700 1 --2- 172.21.15.143:0/4098029018 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1174003a00 0x7f1174003e20 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:32.861 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:32.852+0000 7f119298c700 1 -- 172.21.15.143:0/4098029018 >> 172.21.15.143:0/4098029018 conn(0x7f118c08b000 msgr2=0x7f118c09ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:24:32.861 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:32.852+0000 7f119298c700 1 -- 172.21.15.143:0/4098029018 shutdown_connections 2023-11-08T19:24:32.861 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:32.852+0000 7f119298c700 1 -- 172.21.15.143:0/4098029018 wait complete. 2023-11-08T19:24:32.864 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:32.854+0000 7f119298c700 1 Processor -- start 2023-11-08T19:24:32.864 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:32.854+0000 7f119298c700 1 -- start start 2023-11-08T19:24:32.864 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:32.855+0000 7f119298c700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1174003a00 0x7f118c14fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:32.864 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:32.855+0000 7f119298c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f118c14f690 0x7f118c0cdfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:32.864 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:32.855+0000 7f119298c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f118c0ce510 0x7f118c0ce9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:32.865 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:32.855+0000 7f119298c700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f118c150c70 con 0x7f118c14f690 2023-11-08T19:24:32.865 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:32.855+0000 7f119298c700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f118c0ceee0 con 0x7f118c0ce510 2023-11-08T19:24:32.865 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:32.855+0000 7f119298c700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f118c0cc7e0 con 0x7f1174003a00 2023-11-08T19:24:32.865 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:32.855+0000 7f118b7fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f118c14f690 0x7f118c0cdfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:32.865 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:32.855+0000 7f118bfff700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1174003a00 0x7f118c14fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:32.866 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:32.855+0000 7f118b7fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f118c14f690 0x7f118c0cdfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48084/0 (socket says 172.21.15.143:48084) 2023-11-08T19:24:32.866 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:32.855+0000 7f118b7fe700 1 -- 172.21.15.143:0/1048110177 learned_addr learned my addr 172.21.15.143:0/1048110177 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:24:32.866 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:32.855+0000 7f1190f29700 1 --2- 172.21.15.143:0/1048110177 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f118c0ce510 0x7f118c0ce9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:32.866 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:32.855+0000 7f118b7fe700 1 -- 172.21.15.143:0/1048110177 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1174003a00 msgr2=0x7f118c14fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:32.866 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:32.855+0000 7f118b7fe700 1 --2- 172.21.15.143:0/1048110177 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1174003a00 0x7f118c14fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:32.867 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:32.855+0000 7f118b7fe700 1 -- 172.21.15.143:0/1048110177 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f118c0ce510 msgr2=0x7f118c0ce9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:32.867 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:32.855+0000 7f118b7fe700 1 --2- 172.21.15.143:0/1048110177 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f118c0ce510 0x7f118c0ce9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:32.867 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:32.855+0000 7f118b7fe700 1 -- 172.21.15.143:0/1048110177 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f118c150f10 con 0x7f118c14f690 2023-11-08T19:24:32.867 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:32.855+0000 7f118b7fe700 1 --2- 172.21.15.143:0/1048110177 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f118c14f690 0x7f118c0cdfd0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f118000e860 tx=0x7f118000eb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:32.867 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:32.855+0000 7f1188ff9700 1 -- 172.21.15.143:0/1048110177 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f118001e410 con 0x7f118c14f690 2023-11-08T19:24:32.868 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:32.855+0000 7f1188ff9700 1 -- 172.21.15.143:0/1048110177 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f118000c040 con 0x7f118c14f690 2023-11-08T19:24:32.868 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:32.855+0000 7f119298c700 1 -- 172.21.15.143:0/1048110177 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f118c1511f0 con 0x7f118c14f690 2023-11-08T19:24:32.868 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:32.855+0000 7f1188ff9700 1 -- 172.21.15.143:0/1048110177 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f118001e7b0 con 0x7f118c14f690 2023-11-08T19:24:32.868 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:32.855+0000 7f119298c700 1 -- 172.21.15.143:0/1048110177 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f118c0cc490 con 0x7f118c14f690 2023-11-08T19:24:32.868 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:32.855+0000 7f1188ff9700 1 -- 172.21.15.143:0/1048110177 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f118001ec10 con 0x7f118c14f690 2023-11-08T19:24:32.869 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:32.855+0000 7f1188ff9700 1 -- 172.21.15.143:0/1048110177 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f118001d700 con 0x7f118c14f690 2023-11-08T19:24:32.869 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:32.856+0000 7f119298c700 1 -- 172.21.15.143:0/1048110177 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f118c0d0060 con 0x7f118c14f690 2023-11-08T19:24:32.869 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:32.858+0000 7f1188ff9700 1 -- 172.21.15.143:0/1048110177 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f118003db40 con 0x7f118c14f690 2023-11-08T19:24:32.933 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:32.932+0000 7f119298c700 1 -- 172.21.15.143:0/1048110177 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f118c04eee0 con 0x7f118c14f690 2023-11-08T19:24:32.934 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:32.933+0000 7f1188ff9700 1 -- 172.21.15.143:0/1048110177 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f118002c070 con 0x7f118c14f690 2023-11-08T19:24:32.936 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:32.934+0000 7f119298c700 1 -- 172.21.15.143:0/1048110177 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f118c14f690 msgr2=0x7f118c0cdfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:32.936 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:32.934+0000 7f119298c700 1 --2- 172.21.15.143:0/1048110177 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f118c14f690 0x7f118c0cdfd0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f118000e860 tx=0x7f118000eb70 comp rx=0 tx=0).stop 2023-11-08T19:24:32.936 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:32.934+0000 7f119298c700 1 -- 172.21.15.143:0/1048110177 shutdown_connections 2023-11-08T19:24:32.937 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:32.934+0000 7f119298c700 1 --2- 172.21.15.143:0/1048110177 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1174003a00 0x7f118c14fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:32.937 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:32.934+0000 7f119298c700 1 --2- 172.21.15.143:0/1048110177 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f118c0ce510 0x7f118c0ce9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:32.937 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:32.934+0000 7f119298c700 1 --2- 172.21.15.143:0/1048110177 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f118c14f690 0x7f118c0cdfd0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:32.937 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:32.934+0000 7f119298c700 1 -- 172.21.15.143:0/1048110177 >> 172.21.15.143:0/1048110177 conn(0x7f118c08b000 msgr2=0x7f118c0b3b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:24:32.937 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:32.934+0000 7f119298c700 1 -- 172.21.15.143:0/1048110177 shutdown_connections 2023-11-08T19:24:32.937 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:32.934+0000 7f119298c700 1 -- 172.21.15.143:0/1048110177 wait complete. 2023-11-08T19:24:32.938 INFO:teuthology.orchestra.run.smithi143.stderr:adjusted tunables profile to default 2023-11-08T19:24:32.947 INFO:tasks.ceph:check_enable_crimson: False 2023-11-08T19:24:32.947 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-11-08T19:24:32.948 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-08T19:24:32.948 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-11-08T19:24:32.965 INFO:tasks.ceph.mgr.x.smithi174.stderr:2023-11-08T19:24:32.964+0000 7fd2ab6d0200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-11-08T19:24:32.974 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-08T19:24:32.974 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-11-08T19:24:33.030 INFO:tasks.ceph.mgr.x.smithi174.stderr:2023-11-08T19:24:33.029+0000 7fd2ab6d0200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-08T19:24:33.040 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-08T19:24:33.041 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-11-08T19:24:33.105 INFO:tasks.ceph.mgr.x.smithi174.stderr:2023-11-08T19:24:33.104+0000 7fd2ab6d0200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-08T19:24:33.107 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-08T19:24:33.108 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-11-08T19:24:33.173 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2023-11-08T19:24:33.173 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-11-08T19:24:33.184 INFO:tasks.ceph.mgr.x.smithi174.stderr:2023-11-08T19:24:33.182+0000 7fd2ab6d0200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-08T19:24:33.200 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2023-11-08T19:24:33.200 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-11-08T19:24:33.266 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2023-11-08T19:24:33.267 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-11-08T19:24:33.304 INFO:tasks.ceph.mgr.y.smithi143.stderr:2023-11-08T19:24:33.303+0000 7fd1e98fa200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-11-08T19:24:33.332 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2023-11-08T19:24:33.332 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-11-08T19:24:33.334 INFO:tasks.ceph.mgr.x.smithi174.stderr:2023-11-08T19:24:33.332+0000 7fd2ab6d0200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-08T19:24:33.358 DEBUG:teuthology.orchestra.run.smithi174:> sudo ceph --cluster ceph osd new bae98ec1-d94e-403d-a220-a522a53ea6f2 0 2023-11-08T19:24:33.370 INFO:tasks.ceph.mgr.y.smithi143.stderr:2023-11-08T19:24:33.369+0000 7fd1e98fa200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-08T19:24:33.447 INFO:tasks.ceph.mgr.y.smithi143.stderr:2023-11-08T19:24:33.445+0000 7fd1e98fa200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-08T19:24:33.519 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.517+0000 7fe53c284700 1 Processor -- start 2023-11-08T19:24:33.519 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.517+0000 7fe53c284700 1 -- start start 2023-11-08T19:24:33.521 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.518+0000 7fe53c284700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe53406b0b0 0x7fe53406c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:33.522 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.518+0000 7fe53c284700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe53406cc50 con 0x7fe53415bc80 2023-11-08T19:24:33.522 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.518+0000 7fe53c284700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe53406b670 con 0x7fe5340b22e0 2023-11-08T19:24:33.522 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.518+0000 7fe53c284700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5340ce290 con 0x7fe53406b0b0 2023-11-08T19:24:33.523 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.518+0000 7fe53a821700 1 --1- >> v1:172.21.15.174:6789/0 conn(0x7fe5340b22e0 0x7fe5340b37b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.174:6789/0 says I am v1:172.21.15.174:52164/0 (socket says 172.21.15.174:52164) 2023-11-08T19:24:33.523 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.518+0000 7fe53a821700 1 -- 172.21.15.174:0/2453639971 learned_addr learned my addr 172.21.15.174:0/2453639971 (peer_addr_for_me v1:172.21.15.174:0/0) 2023-11-08T19:24:33.523 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.518+0000 7fe53981f700 1 --2- 172.21.15.174:0/2453639971 >> v2:172.21.15.143:3300/0 conn(0x7fe53406b0b0 0x7fe53406c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:33.523 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.518+0000 7fe53901e700 1 -- 172.21.15.174:0/2453639971 <== mon.1 v1:172.21.15.174:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 226490077 0 0) 0x7fe53406b670 con 0x7fe5340b22e0 2023-11-08T19:24:33.524 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.518+0000 7fe53901e700 1 -- 172.21.15.174:0/2453639971 --> v1:172.21.15.174:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe5200035b0 con 0x7fe5340b22e0 2023-11-08T19:24:33.524 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.519+0000 7fe53981f700 1 -- 172.21.15.174:0/2453639971 >> v1:172.21.15.174:6789/0 conn(0x7fe5340b22e0 legacy=0x7fe5340b37b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:33.525 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.519+0000 7fe53981f700 1 -- 172.21.15.174:0/2453639971 >> v1:172.21.15.143:6789/0 conn(0x7fe53415bc80 legacy=0x7fe5340b6280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:33.525 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.519+0000 7fe53901e700 1 -- 172.21.15.174:0/2453639971 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3048191175 0 0) 0x7fe53406cc50 con 0x7fe53415bc80 2023-11-08T19:24:33.525 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.519+0000 7fe53981f700 1 -- 172.21.15.174:0/2453639971 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe53406b950 con 0x7fe53406b0b0 2023-11-08T19:24:33.525 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.519+0000 7fe53981f700 1 --2- 172.21.15.174:0/2453639971 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe53406b0b0 0x7fe53406c710 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fe524009840 tx=0x7fe524009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=967ac0dd502f7e6d server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:33.526 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.519+0000 7fe53901e700 1 -- 172.21.15.174:0/2453639971 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe524011830 con 0x7fe53406b0b0 2023-11-08T19:24:33.526 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.519+0000 7fe53901e700 1 -- 172.21.15.174:0/2453639971 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe524011990 con 0x7fe53406b0b0 2023-11-08T19:24:33.526 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.519+0000 7fe53901e700 1 -- 172.21.15.174:0/2453639971 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe524011d20 con 0x7fe53406b0b0 2023-11-08T19:24:33.526 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.519+0000 7fe53c284700 1 -- 172.21.15.174:0/2453639971 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe53406b0b0 msgr2=0x7fe53406c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:33.526 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.519+0000 7fe53c284700 1 --2- 172.21.15.174:0/2453639971 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe53406b0b0 0x7fe53406c710 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fe524009840 tx=0x7fe524009b50 comp rx=0 tx=0).stop 2023-11-08T19:24:33.527 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.520+0000 7fe53c284700 1 -- 172.21.15.174:0/2453639971 shutdown_connections 2023-11-08T19:24:33.527 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.520+0000 7fe53c284700 1 --2- 172.21.15.174:0/2453639971 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe53406b0b0 0x7fe53406c710 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:33.527 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.520+0000 7fe53c284700 1 -- 172.21.15.174:0/2453639971 >> 172.21.15.174:0/2453639971 conn(0x7fe5340cd2e0 msgr2=0x7fe5340cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:24:33.527 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.520+0000 7fe53c284700 1 -- 172.21.15.174:0/2453639971 shutdown_connections 2023-11-08T19:24:33.528 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.520+0000 7fe53c284700 1 -- 172.21.15.174:0/2453639971 wait complete. 2023-11-08T19:24:33.528 INFO:tasks.ceph.mgr.y.smithi143.stderr:2023-11-08T19:24:33.524+0000 7fd1e98fa200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-08T19:24:33.531 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.521+0000 7fe53c284700 1 Processor -- start 2023-11-08T19:24:33.531 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.521+0000 7fe53c284700 1 -- start start 2023-11-08T19:24:33.531 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.522+0000 7fe53c284700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe53406b0b0 0x7fe5340c5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:33.531 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.522+0000 7fe53c284700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe5340b22e0 0x7fe53415a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:33.532 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.522+0000 7fe53c284700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe53415bc80 0x7fe5340c9b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:33.532 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.522+0000 7fe53c284700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5340c5c70 con 0x7fe53406b0b0 2023-11-08T19:24:33.532 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.522+0000 7fe53c284700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5340c8ab0 con 0x7fe5340b22e0 2023-11-08T19:24:33.532 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.522+0000 7fe53c284700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5340c8d90 con 0x7fe53415bc80 2023-11-08T19:24:33.532 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.522+0000 7fe53981f700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe5340b22e0 0x7fe53415a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:33.532 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.522+0000 7fe53a821700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe53415bc80 0x7fe5340c9b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:33.533 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.522+0000 7fe53981f700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe5340b22e0 0x7fe53415a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:47170/0 (socket says 172.21.15.174:47170) 2023-11-08T19:24:33.533 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.522+0000 7fe53981f700 1 -- 172.21.15.174:0/605619944 learned_addr learned my addr 172.21.15.174:0/605619944 (peer_addr_for_me v2:172.21.15.174:0/0) 2023-11-08T19:24:33.533 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.522+0000 7fe53a020700 1 --2- 172.21.15.174:0/605619944 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe53406b0b0 0x7fe5340c5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:33.533 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.523+0000 7fe53a821700 1 -- 172.21.15.174:0/605619944 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe5340b22e0 msgr2=0x7fe53415a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:33.534 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.523+0000 7fe53a821700 1 --2- 172.21.15.174:0/605619944 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe5340b22e0 0x7fe53415a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:33.534 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.523+0000 7fe53a821700 1 -- 172.21.15.174:0/605619944 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe53406b0b0 msgr2=0x7fe5340c5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:33.534 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.523+0000 7fe53a821700 1 --2- 172.21.15.174:0/605619944 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe53406b0b0 0x7fe5340c5730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:33.534 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.523+0000 7fe53a821700 1 -- 172.21.15.174:0/605619944 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5240094a0 con 0x7fe53415bc80 2023-11-08T19:24:33.534 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.523+0000 7fe53a020700 1 --2- 172.21.15.174:0/605619944 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe53406b0b0 0x7fe5340c5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T19:24:33.535 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.523+0000 7fe53981f700 1 --2- 172.21.15.174:0/605619944 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe5340b22e0 0x7fe53415a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T19:24:33.535 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.523+0000 7fe53a821700 1 --2- 172.21.15.174:0/605619944 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe53415bc80 0x7fe5340c9b50 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe52c00ca80 tx=0x7fe52c00cd90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:33.535 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.523+0000 7fe52affd700 1 -- 172.21.15.174:0/605619944 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe52c00a9e0 con 0x7fe53415bc80 2023-11-08T19:24:33.536 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.523+0000 7fe52affd700 1 -- 172.21.15.174:0/605619944 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe52c00ab40 con 0x7fe53415bc80 2023-11-08T19:24:33.536 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.523+0000 7fe53c284700 1 -- 172.21.15.174:0/605619944 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5340c9070 con 0x7fe53415bc80 2023-11-08T19:24:33.536 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.523+0000 7fe52affd700 1 -- 172.21.15.174:0/605619944 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe52c012400 con 0x7fe53415bc80 2023-11-08T19:24:33.536 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.523+0000 7fe53c284700 1 -- 172.21.15.174:0/605619944 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5340c9630 con 0x7fe53415bc80 2023-11-08T19:24:33.536 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.523+0000 7fe52affd700 1 -- 172.21.15.174:0/605619944 <== mon.2 v2:172.21.15.174:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fe52c012560 con 0x7fe53415bc80 2023-11-08T19:24:33.536 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.523+0000 7fe52affd700 1 -- 172.21.15.174:0/605619944 <== mon.2 v2:172.21.15.174:3301/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7fe52c01c8f0 con 0x7fe53415bc80 2023-11-08T19:24:33.537 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.524+0000 7fe53c284700 1 -- 172.21.15.174:0/605619944 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5340c9820 con 0x7fe53415bc80 2023-11-08T19:24:33.537 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.526+0000 7fe52affd700 1 -- 172.21.15.174:0/605619944 <== mon.2 v2:172.21.15.174:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7fe52c021060 con 0x7fe53415bc80 2023-11-08T19:24:33.611 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.609+0000 7fe53c284700 1 -- 172.21.15.174:0/605619944 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_command({"prefix": "osd new", "uuid": "bae98ec1-d94e-403d-a220-a522a53ea6f2", "id": 0} v 0) v1 -- 0x7fe53404eee0 con 0x7fe53415bc80 2023-11-08T19:24:33.616 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.615+0000 7fe52affd700 1 -- 172.21.15.174:0/605619944 <== mon.2 v2:172.21.15.174:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "bae98ec1-d94e-403d-a220-a522a53ea6f2", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7fe52c017070 con 0x7fe53415bc80 2023-11-08T19:24:33.617 INFO:teuthology.orchestra.run.smithi174.stdout:0 2023-11-08T19:24:33.618 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.616+0000 7fe53c284700 1 -- 172.21.15.174:0/605619944 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe53415bc80 msgr2=0x7fe5340c9b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:33.618 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.616+0000 7fe53c284700 1 --2- 172.21.15.174:0/605619944 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe53415bc80 0x7fe5340c9b50 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe52c00ca80 tx=0x7fe52c00cd90 comp rx=0 tx=0).stop 2023-11-08T19:24:33.618 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.616+0000 7fe53c284700 1 -- 172.21.15.174:0/605619944 shutdown_connections 2023-11-08T19:24:33.619 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.616+0000 7fe53c284700 1 --2- 172.21.15.174:0/605619944 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe53415bc80 0x7fe5340c9b50 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:33.619 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.616+0000 7fe53c284700 1 --2- 172.21.15.174:0/605619944 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe53406b0b0 0x7fe5340c5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:33.619 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.616+0000 7fe53c284700 1 --2- 172.21.15.174:0/605619944 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe5340b22e0 0x7fe53415a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:33.619 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.616+0000 7fe53c284700 1 -- 172.21.15.174:0/605619944 >> 172.21.15.174:0/605619944 conn(0x7fe5340cd2e0 msgr2=0x7fe5340afa30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:24:33.619 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.616+0000 7fe53c284700 1 -- 172.21.15.174:0/605619944 shutdown_connections 2023-11-08T19:24:33.619 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.616+0000 7fe53c284700 1 -- 172.21.15.174:0/605619944 wait complete. 2023-11-08T19:24:33.629 DEBUG:teuthology.orchestra.run.smithi174:> sudo ceph --cluster ceph osd new 5cb6538a-1ccb-4322-8633-09f66af0b7e4 1 2023-11-08T19:24:33.669 INFO:tasks.ceph.mgr.y.smithi143.stderr:2023-11-08T19:24:33.668+0000 7fd1e98fa200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-08T19:24:33.749 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.747+0000 7fcdc89dd700 1 Processor -- start 2023-11-08T19:24:33.749 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.747+0000 7fcdc89dd700 1 -- start start 2023-11-08T19:24:33.752 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.748+0000 7fcdc89dd700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fcdc415bb20 0x7fcdc40b2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:33.752 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.748+0000 7fcdc89dd700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fcdc40d1d80 0x7fcdc40cc4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:33.752 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.748+0000 7fcdc89dd700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdc40d0420 con 0x7fcdc40b1730 2023-11-08T19:24:33.752 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.748+0000 7fcdc89dd700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdc40cf600 con 0x7fcdc415bb20 2023-11-08T19:24:33.753 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.748+0000 7fcdc89dd700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdc40cf810 con 0x7fcdc40d1d80 2023-11-08T19:24:33.753 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.748+0000 7fcdc2d9d700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fcdc40d1d80 0x7fcdc40cc4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:33.753 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.748+0000 7fcdc2d9d700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fcdc40d1d80 0x7fcdc40cc4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:47178/0 (socket says 172.21.15.174:47178) 2023-11-08T19:24:33.753 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.748+0000 7fcdc2d9d700 1 -- 172.21.15.174:0/422381426 learned_addr learned my addr 172.21.15.174:0/422381426 (peer_addr_for_me v2:172.21.15.174:0/0) 2023-11-08T19:24:33.754 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.748+0000 7fcdc259c700 1 --2- 172.21.15.174:0/422381426 >> v2:172.21.15.143:3300/0 conn(0x7fcdc415bb20 0x7fcdc40b2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:33.754 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.749+0000 7fcdc2d9d700 1 -- 172.21.15.174:0/422381426 >> v2:172.21.15.143:3300/0 conn(0x7fcdc415bb20 msgr2=0x7fcdc40b2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:24:33.754 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.749+0000 7fcdc2d9d700 1 --2- 172.21.15.174:0/422381426 >> v2:172.21.15.143:3300/0 conn(0x7fcdc415bb20 0x7fcdc40b2670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:33.754 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.749+0000 7fcdc2d9d700 1 -- 172.21.15.174:0/422381426 >> v1:172.21.15.143:6789/0 conn(0x7fcdc40b1730 legacy=0x7fcdc40d1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:33.754 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.749+0000 7fcdc2d9d700 1 -- 172.21.15.174:0/422381426 --> v2:172.21.15.174:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdc40cfaf0 con 0x7fcdc40d1d80 2023-11-08T19:24:33.755 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.749+0000 7fcdc2d9d700 1 --2- 172.21.15.174:0/422381426 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcdc40d1d80 0x7fcdc40cc4d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fcdc40d2b10 tx=0x7fcdbc00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4806ed52dd7321d3 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:33.755 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.749+0000 7fcdc159a700 1 -- 172.21.15.174:0/422381426 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdbc00e070 con 0x7fcdc40d1d80 2023-11-08T19:24:33.755 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.749+0000 7fcdc159a700 1 -- 172.21.15.174:0/422381426 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdbc00b910 con 0x7fcdc40d1d80 2023-11-08T19:24:33.755 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.749+0000 7fcdc159a700 1 -- 172.21.15.174:0/422381426 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdbc00bcc0 con 0x7fcdc40d1d80 2023-11-08T19:24:33.755 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.750+0000 7fcdc89dd700 1 -- 172.21.15.174:0/422381426 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcdc40d1d80 msgr2=0x7fcdc40cc4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:33.756 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.750+0000 7fcdc89dd700 1 --2- 172.21.15.174:0/422381426 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcdc40d1d80 0x7fcdc40cc4d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fcdc40d2b10 tx=0x7fcdbc00b330 comp rx=0 tx=0).stop 2023-11-08T19:24:33.756 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.750+0000 7fcdc89dd700 1 -- 172.21.15.174:0/422381426 shutdown_connections 2023-11-08T19:24:33.756 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.750+0000 7fcdc89dd700 1 --2- 172.21.15.174:0/422381426 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcdc40d1d80 0x7fcdc40cc4d0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:33.756 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.750+0000 7fcdc89dd700 1 --2- 172.21.15.174:0/422381426 >> v2:172.21.15.143:3300/0 conn(0x7fcdc415bb20 0x7fcdc40b2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:33.756 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.750+0000 7fcdc89dd700 1 -- 172.21.15.174:0/422381426 >> 172.21.15.174:0/422381426 conn(0x7fcdc408b000 msgr2=0x7fcdc409ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:24:33.757 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.750+0000 7fcdc89dd700 1 -- 172.21.15.174:0/422381426 shutdown_connections 2023-11-08T19:24:33.757 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.750+0000 7fcdc89dd700 1 -- 172.21.15.174:0/422381426 wait complete. 2023-11-08T19:24:33.757 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.751+0000 7fcdc89dd700 1 Processor -- start 2023-11-08T19:24:33.758 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.751+0000 7fcdc89dd700 1 -- start start 2023-11-08T19:24:33.758 INFO:tasks.ceph.mgr.x.smithi174.stderr:2023-11-08T19:24:33.752+0000 7fd2ab6d0200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-08T19:24:33.760 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.752+0000 7fcdc89dd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcdc40b1730 0x7fcdc40c4590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:33.761 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.752+0000 7fcdc89dd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcdc40d1d80 0x7fcdc40cc4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:33.761 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.752+0000 7fcdc89dd700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcdc415bb20 0x7fcdc40cca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:33.761 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.752+0000 7fcdc89dd700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc40ce090 con 0x7fcdc40d1d80 2023-11-08T19:24:33.761 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.752+0000 7fcdc89dd700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc415d4d0 con 0x7fcdc40b1730 2023-11-08T19:24:33.762 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.752+0000 7fcdc89dd700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc40cf390 con 0x7fcdc415bb20 2023-11-08T19:24:33.762 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.752+0000 7fcdc259c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcdc40b1730 0x7fcdc40c4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:33.762 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.752+0000 7fcdc2d9d700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcdc415bb20 0x7fcdc40cca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:33.762 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.752+0000 7fcdc259c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcdc40b1730 0x7fcdc40c4590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:47180/0 (socket says 172.21.15.174:47180) 2023-11-08T19:24:33.762 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.752+0000 7fcdc259c700 1 -- 172.21.15.174:0/2000306604 learned_addr learned my addr 172.21.15.174:0/2000306604 (peer_addr_for_me v2:172.21.15.174:0/0) 2023-11-08T19:24:33.763 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.752+0000 7fcdc2d9d700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcdc415bb20 0x7fcdc40cca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.174:46948/0 (socket says 172.21.15.174:46948) 2023-11-08T19:24:33.763 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.752+0000 7fcdc1d9b700 1 --2- 172.21.15.174:0/2000306604 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcdc40d1d80 0x7fcdc40cc4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:33.763 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.752+0000 7fcdc2d9d700 1 -- 172.21.15.174:0/2000306604 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcdc40b1730 msgr2=0x7fcdc40c4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:33.763 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.752+0000 7fcdc2d9d700 1 --2- 172.21.15.174:0/2000306604 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcdc40b1730 0x7fcdc40c4590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:33.763 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.752+0000 7fcdc2d9d700 1 -- 172.21.15.174:0/2000306604 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcdc40d1d80 msgr2=0x7fcdc40cc4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:33.764 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.752+0000 7fcdc2d9d700 1 --2- 172.21.15.174:0/2000306604 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcdc40d1d80 0x7fcdc40cc4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:33.764 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.752+0000 7fcdc2d9d700 1 -- 172.21.15.174:0/2000306604 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdbc009b30 con 0x7fcdc415bb20 2023-11-08T19:24:33.764 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.752+0000 7fcdc259c700 1 --2- 172.21.15.174:0/2000306604 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcdc40b1730 0x7fcdc40c4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T19:24:33.764 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.752+0000 7fcdc1d9b700 1 --2- 172.21.15.174:0/2000306604 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcdc40d1d80 0x7fcdc40cc4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T19:24:33.764 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.753+0000 7fcdc2d9d700 1 --2- 172.21.15.174:0/2000306604 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcdc415bb20 0x7fcdc40cca10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc015000 tx=0x7fcdbc004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:33.765 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.753+0000 7fcdb2ffd700 1 -- 172.21.15.174:0/2000306604 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdbc004880 con 0x7fcdc415bb20 2023-11-08T19:24:33.765 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.753+0000 7fcdb2ffd700 1 -- 172.21.15.174:0/2000306604 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdbc0092e0 con 0x7fcdc415bb20 2023-11-08T19:24:33.765 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.753+0000 7fcdb2ffd700 1 -- 172.21.15.174:0/2000306604 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdbc01c660 con 0x7fcdc415bb20 2023-11-08T19:24:33.765 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.753+0000 7fcdc89dd700 1 -- 172.21.15.174:0/2000306604 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdc415c8c0 con 0x7fcdc415bb20 2023-11-08T19:24:33.765 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.753+0000 7fcdc89dd700 1 -- 172.21.15.174:0/2000306604 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdc415ce80 con 0x7fcdc415bb20 2023-11-08T19:24:33.766 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.753+0000 7fcdb2ffd700 1 -- 172.21.15.174:0/2000306604 <== mon.2 v2:172.21.15.174:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fcdbc01cac0 con 0x7fcdc415bb20 2023-11-08T19:24:33.766 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.753+0000 7fcdb2ffd700 1 -- 172.21.15.174:0/2000306604 <== mon.2 v2:172.21.15.174:3301/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7fcdbc025bf0 con 0x7fcdc415bb20 2023-11-08T19:24:33.766 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.754+0000 7fcdc89dd700 1 -- 172.21.15.174:0/2000306604 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdc404eee0 con 0x7fcdc415bb20 2023-11-08T19:24:33.766 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.756+0000 7fcdb2ffd700 1 -- 172.21.15.174:0/2000306604 <== mon.2 v2:172.21.15.174:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7fcdbc043b40 con 0x7fcdc415bb20 2023-11-08T19:24:33.843 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.841+0000 7fcdc89dd700 1 -- 172.21.15.174:0/2000306604 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_command({"prefix": "osd new", "uuid": "5cb6538a-1ccb-4322-8633-09f66af0b7e4", "id": 1} v 0) v1 -- 0x7fcdc415d610 con 0x7fcdc415bb20 2023-11-08T19:24:33.851 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.849+0000 7fcdb2ffd700 1 -- 172.21.15.174:0/2000306604 <== mon.2 v2:172.21.15.174:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "5cb6538a-1ccb-4322-8633-09f66af0b7e4", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7fcdbc032030 con 0x7fcdc415bb20 2023-11-08T19:24:33.851 INFO:teuthology.orchestra.run.smithi174.stdout:1 2023-11-08T19:24:33.852 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.850+0000 7fcdc89dd700 1 -- 172.21.15.174:0/2000306604 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcdc415bb20 msgr2=0x7fcdc40cca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:33.852 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.850+0000 7fcdc89dd700 1 --2- 172.21.15.174:0/2000306604 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcdc415bb20 0x7fcdc40cca10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc015000 tx=0x7fcdbc004ad0 comp rx=0 tx=0).stop 2023-11-08T19:24:33.852 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.850+0000 7fcdc89dd700 1 -- 172.21.15.174:0/2000306604 shutdown_connections 2023-11-08T19:24:33.853 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.850+0000 7fcdc89dd700 1 --2- 172.21.15.174:0/2000306604 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcdc415bb20 0x7fcdc40cca10 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:33.853 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.850+0000 7fcdc89dd700 1 --2- 172.21.15.174:0/2000306604 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcdc40b1730 0x7fcdc40c4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:33.853 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.850+0000 7fcdc89dd700 1 --2- 172.21.15.174:0/2000306604 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcdc40d1d80 0x7fcdc40cc4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:33.853 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.850+0000 7fcdc89dd700 1 -- 172.21.15.174:0/2000306604 >> 172.21.15.174:0/2000306604 conn(0x7fcdc408b000 msgr2=0x7fcdc40b30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:24:33.854 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.850+0000 7fcdc89dd700 1 -- 172.21.15.174:0/2000306604 shutdown_connections 2023-11-08T19:24:33.854 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.850+0000 7fcdc89dd700 1 -- 172.21.15.174:0/2000306604 wait complete. 2023-11-08T19:24:33.864 DEBUG:teuthology.orchestra.run.smithi174:> sudo ceph --cluster ceph osd new cdbf522c-60ef-4d60-9653-8dcb529c101a 2 2023-11-08T19:24:33.878 INFO:tasks.ceph.mgr.x.smithi174.stderr:2023-11-08T19:24:33.877+0000 7fd2ab6d0200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-08T19:24:33.966 INFO:tasks.ceph.mgr.x.smithi174.stderr:2023-11-08T19:24:33.964+0000 7fd2ab6d0200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-08T19:24:33.987 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.985+0000 7fa47d7c3700 1 Processor -- start 2023-11-08T19:24:33.987 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.985+0000 7fa47d7c3700 1 -- start start 2023-11-08T19:24:33.989 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.986+0000 7fa47d7c3700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa4780ce660 0x7fa4781623f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:33.989 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.986+0000 7fa47d7c3700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa478162930 con 0x7fa47815dd60 2023-11-08T19:24:33.989 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.986+0000 7fa47d7c3700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa47806b2b0 con 0x7fa4780b2a10 2023-11-08T19:24:33.990 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.986+0000 7fa47d7c3700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4780cf000 con 0x7fa4780ce660 2023-11-08T19:24:33.990 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.986+0000 7fa4767fc700 1 --1- >> v1:172.21.15.174:6789/0 conn(0x7fa4780b2a10 0x7fa47806bd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.174:6789/0 says I am v1:172.21.15.174:52186/0 (socket says 172.21.15.174:52186) 2023-11-08T19:24:33.990 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.986+0000 7fa4767fc700 1 -- 172.21.15.174:0/3985570112 learned_addr learned my addr 172.21.15.174:0/3985570112 (peer_addr_for_me v1:172.21.15.174:0/0) 2023-11-08T19:24:33.990 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.986+0000 7fa4777fe700 1 --2- 172.21.15.174:0/3985570112 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa4780ce660 0x7fa4781623f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:33.990 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.987+0000 7fa4777fe700 1 -- 172.21.15.174:0/3985570112 >> v1:172.21.15.174:6789/0 conn(0x7fa4780b2a10 legacy=0x7fa47806bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:33.991 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.987+0000 7fa4777fe700 1 -- 172.21.15.174:0/3985570112 >> v1:172.21.15.143:6789/0 conn(0x7fa47815dd60 legacy=0x7fa4780b2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:33.991 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.987+0000 7fa4777fe700 1 -- 172.21.15.174:0/3985570112 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4780cf2e0 con 0x7fa4780ce660 2023-11-08T19:24:33.991 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.987+0000 7fa4777fe700 1 --2- 172.21.15.174:0/3985570112 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa4780ce660 0x7fa4781623f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa470009dc0 tx=0x7fa47000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99827866a355ebc8 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:33.991 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.987+0000 7fa475ffb700 1 -- 172.21.15.174:0/3985570112 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa47000e070 con 0x7fa4780ce660 2023-11-08T19:24:33.991 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.987+0000 7fa475ffb700 1 -- 172.21.15.174:0/3985570112 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa47000bbd0 con 0x7fa4780ce660 2023-11-08T19:24:33.992 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.987+0000 7fa475ffb700 1 -- 172.21.15.174:0/3985570112 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa470012430 con 0x7fa4780ce660 2023-11-08T19:24:33.992 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.987+0000 7fa47d7c3700 1 -- 172.21.15.174:0/3985570112 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa4780ce660 msgr2=0x7fa4781623f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:33.992 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.987+0000 7fa47d7c3700 1 --2- 172.21.15.174:0/3985570112 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa4780ce660 0x7fa4781623f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa470009dc0 tx=0x7fa47000b330 comp rx=0 tx=0).stop 2023-11-08T19:24:33.992 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.987+0000 7fa47d7c3700 1 -- 172.21.15.174:0/3985570112 shutdown_connections 2023-11-08T19:24:33.992 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.987+0000 7fa47d7c3700 1 --2- 172.21.15.174:0/3985570112 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa4780ce660 0x7fa4781623f0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:33.993 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.987+0000 7fa47d7c3700 1 -- 172.21.15.174:0/3985570112 >> 172.21.15.174:0/3985570112 conn(0x7fa4780cd300 msgr2=0x7fa4780cd710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:24:33.993 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.987+0000 7fa47d7c3700 1 -- 172.21.15.174:0/3985570112 shutdown_connections 2023-11-08T19:24:33.993 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.987+0000 7fa47d7c3700 1 -- 172.21.15.174:0/3985570112 wait complete. 2023-11-08T19:24:33.995 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.989+0000 7fa47d7c3700 1 Processor -- start 2023-11-08T19:24:33.995 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.989+0000 7fa47d7c3700 1 -- start start 2023-11-08T19:24:33.995 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.989+0000 7fa47d7c3700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa4780b2a10 0x7fa478156f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:33.996 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.989+0000 7fa47d7c3700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa4780ce660 0x7fa47815f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:33.996 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.989+0000 7fa47d7c3700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa47815dd60 0x7fa4780b0760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:33.996 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.989+0000 7fa47d7c3700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa478160fa0 con 0x7fa4780b2a10 2023-11-08T19:24:33.996 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.989+0000 7fa47d7c3700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa47815ff00 con 0x7fa47815dd60 2023-11-08T19:24:33.996 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.989+0000 7fa47d7c3700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4781601e0 con 0x7fa4780ce660 2023-11-08T19:24:33.997 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.990+0000 7fa4777fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa47815dd60 0x7fa4780b0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:33.997 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.990+0000 7fa4767fc700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa4780ce660 0x7fa47815f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:33.997 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.990+0000 7fa4777fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa47815dd60 0x7fa4780b0760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:47196/0 (socket says 172.21.15.174:47196) 2023-11-08T19:24:33.997 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.990+0000 7fa4767fc700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa4780ce660 0x7fa47815f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.174:46958/0 (socket says 172.21.15.174:46958) 2023-11-08T19:24:33.997 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.990+0000 7fa4777fe700 1 -- 172.21.15.174:0/3261962197 learned_addr learned my addr 172.21.15.174:0/3261962197 (peer_addr_for_me v2:172.21.15.174:0/0) 2023-11-08T19:24:33.998 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.990+0000 7fa476ffd700 1 --2- 172.21.15.174:0/3261962197 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa4780b2a10 0x7fa478156f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:33.998 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.990+0000 7fa4767fc700 1 -- 172.21.15.174:0/3261962197 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa47815dd60 msgr2=0x7fa4780b0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:33.998 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.990+0000 7fa4767fc700 1 --2- 172.21.15.174:0/3261962197 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa47815dd60 0x7fa4780b0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:33.998 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.990+0000 7fa4767fc700 1 -- 172.21.15.174:0/3261962197 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa4780b2a10 msgr2=0x7fa478156f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:33.998 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.990+0000 7fa4767fc700 1 --2- 172.21.15.174:0/3261962197 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa4780b2a10 0x7fa478156f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:33.999 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.990+0000 7fa4767fc700 1 -- 172.21.15.174:0/3261962197 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa470009a70 con 0x7fa4780ce660 2023-11-08T19:24:33.999 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.990+0000 7fa4777fe700 1 --2- 172.21.15.174:0/3261962197 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa47815dd60 0x7fa4780b0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T19:24:33.999 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.990+0000 7fa4767fc700 1 --2- 172.21.15.174:0/3261962197 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa4780ce660 0x7fa47815f4b0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa46c005910 tx=0x7fa46c00c6e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:33.999 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.990+0000 7fa45f7fe700 1 -- 172.21.15.174:0/3261962197 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa46c005d70 con 0x7fa4780ce660 2023-11-08T19:24:33.999 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.990+0000 7fa45f7fe700 1 -- 172.21.15.174:0/3261962197 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa46c00ce90 con 0x7fa4780ce660 2023-11-08T19:24:34.000 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.990+0000 7fa47d7c3700 1 -- 172.21.15.174:0/3261962197 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4781604c0 con 0x7fa4780ce660 2023-11-08T19:24:34.000 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.990+0000 7fa45f7fe700 1 -- 172.21.15.174:0/3261962197 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa46c007310 con 0x7fa4780ce660 2023-11-08T19:24:34.000 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.990+0000 7fa47d7c3700 1 -- 172.21.15.174:0/3261962197 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa478160a80 con 0x7fa4780ce660 2023-11-08T19:24:34.000 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.990+0000 7fa45f7fe700 1 -- 172.21.15.174:0/3261962197 <== mon.2 v2:172.21.15.174:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fa46c01c440 con 0x7fa4780ce660 2023-11-08T19:24:34.000 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.991+0000 7fa45f7fe700 1 -- 172.21.15.174:0/3261962197 <== mon.2 v2:172.21.15.174:3301/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7fa46c007770 con 0x7fa4780ce660 2023-11-08T19:24:34.001 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.991+0000 7fa47d7c3700 1 -- 172.21.15.174:0/3261962197 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa478160ca0 con 0x7fa4780ce660 2023-11-08T19:24:34.001 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:33.993+0000 7fa45f7fe700 1 -- 172.21.15.174:0/3261962197 <== mon.2 v2:172.21.15.174:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7fa46c017070 con 0x7fa4780ce660 2023-11-08T19:24:34.050 INFO:tasks.ceph.mgr.x.smithi174.stderr:2023-11-08T19:24:34.049+0000 7fd2ab6d0200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-08T19:24:34.071 INFO:tasks.ceph.mgr.y.smithi143.stderr:2023-11-08T19:24:34.070+0000 7fd1e98fa200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-08T19:24:34.078 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.076+0000 7fa47d7c3700 1 -- 172.21.15.174:0/3261962197 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_command({"prefix": "osd new", "uuid": "cdbf522c-60ef-4d60-9653-8dcb529c101a", "id": 2} v 0) v1 -- 0x7fa47806b7b0 con 0x7fa4780ce660 2023-11-08T19:24:34.083 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.082+0000 7fa45f7fe700 1 -- 172.21.15.174:0/3261962197 <== mon.2 v2:172.21.15.174:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "cdbf522c-60ef-4d60-9653-8dcb529c101a", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7fa46c01c880 con 0x7fa4780ce660 2023-11-08T19:24:34.084 INFO:teuthology.orchestra.run.smithi174.stdout:2 2023-11-08T19:24:34.085 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.083+0000 7fa47d7c3700 1 -- 172.21.15.174:0/3261962197 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa4780ce660 msgr2=0x7fa47815f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:34.085 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.083+0000 7fa47d7c3700 1 --2- 172.21.15.174:0/3261962197 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa4780ce660 0x7fa47815f4b0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa46c005910 tx=0x7fa46c00c6e0 comp rx=0 tx=0).stop 2023-11-08T19:24:34.085 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.083+0000 7fa47d7c3700 1 -- 172.21.15.174:0/3261962197 shutdown_connections 2023-11-08T19:24:34.085 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.083+0000 7fa47d7c3700 1 --2- 172.21.15.174:0/3261962197 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa47815dd60 0x7fa4780b0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:34.086 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.083+0000 7fa47d7c3700 1 --2- 172.21.15.174:0/3261962197 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa4780b2a10 0x7fa478156f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:34.086 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.083+0000 7fa47d7c3700 1 --2- 172.21.15.174:0/3261962197 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa4780ce660 0x7fa47815f4b0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:34.086 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.083+0000 7fa47d7c3700 1 -- 172.21.15.174:0/3261962197 >> 172.21.15.174:0/3261962197 conn(0x7fa4780cd300 msgr2=0x7fa478074130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:24:34.086 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.083+0000 7fa47d7c3700 1 -- 172.21.15.174:0/3261962197 shutdown_connections 2023-11-08T19:24:34.086 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.083+0000 7fa47d7c3700 1 -- 172.21.15.174:0/3261962197 wait complete. 2023-11-08T19:24:34.096 DEBUG:teuthology.orchestra.run.smithi174:> sudo ceph --cluster ceph osd new 4e2783cf-ebb9-4dd8-8245-bc1d17ce4325 3 2023-11-08T19:24:34.188 INFO:tasks.ceph.mgr.y.smithi143.stderr:2023-11-08T19:24:34.187+0000 7fd1e98fa200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-08T19:24:34.259 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.258+0000 7fbf5b545700 1 Processor -- start 2023-11-08T19:24:34.260 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.258+0000 7fbf5b545700 1 -- start start 2023-11-08T19:24:34.262 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.259+0000 7fbf5b545700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fbf5415bb20 0x7fbf540b2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:34.262 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.259+0000 7fbf5b545700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fbf540d1d80 0x7fbf540cc4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:34.263 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.259+0000 7fbf5b545700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf540d0420 con 0x7fbf540b1730 2023-11-08T19:24:34.263 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.259+0000 7fbf5b545700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf540cf600 con 0x7fbf540d1d80 2023-11-08T19:24:34.263 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.259+0000 7fbf5b545700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf540cf810 con 0x7fbf5415bb20 2023-11-08T19:24:34.263 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.259+0000 7fbf592e1700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fbf5415bb20 0x7fbf540b2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:34.263 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.259+0000 7fbf58ae0700 1 --1- >> v1:172.21.15.174:6789/0 conn(0x7fbf540b1730 0x7fbf540d1670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.174:6789/0 says I am v1:172.21.15.174:52196/0 (socket says 172.21.15.174:52196) 2023-11-08T19:24:34.264 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.259+0000 7fbf58ae0700 1 -- 172.21.15.174:0/1134631383 learned_addr learned my addr 172.21.15.174:0/1134631383 (peer_addr_for_me v1:172.21.15.174:0/0) 2023-11-08T19:24:34.264 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.259+0000 7fbf59ae2700 1 --2- 172.21.15.174:0/1134631383 >> v2:172.21.15.143:3300/0 conn(0x7fbf540d1d80 0x7fbf540cc4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:34.264 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.259+0000 7fbf592e1700 1 -- 172.21.15.174:0/1134631383 >> v2:172.21.15.143:3300/0 conn(0x7fbf540d1d80 msgr2=0x7fbf540cc4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:24:34.264 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.259+0000 7fbf592e1700 1 --2- 172.21.15.174:0/1134631383 >> v2:172.21.15.143:3300/0 conn(0x7fbf540d1d80 0x7fbf540cc4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:34.264 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.259+0000 7fbf592e1700 1 -- 172.21.15.174:0/1134631383 >> v1:172.21.15.174:6789/0 conn(0x7fbf540b1730 legacy=0x7fbf540d1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:34.265 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.259+0000 7fbf592e1700 1 -- 172.21.15.174:0/1134631383 --> v2:172.21.15.174:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf540cfaf0 con 0x7fbf5415bb20 2023-11-08T19:24:34.265 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.259+0000 7fbf4bfff700 1 -- 172.21.15.174:0/1134631383 <== mon.1 v1:172.21.15.174:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4106102352 0 0) 0x7fbf540d0420 con 0x7fbf540b1730 2023-11-08T19:24:34.265 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.259+0000 7fbf59ae2700 1 --2- 172.21.15.174:0/1134631383 >> v2:172.21.15.143:3300/0 conn(0x7fbf540d1d80 0x7fbf540cc4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:24:34.265 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.259+0000 7fbf592e1700 1 --2- 172.21.15.174:0/1134631383 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbf5415bb20 0x7fbf540b2670 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fbf5408b850 tx=0x7fbf50009b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a0ec488b6d85dfd server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:34.265 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.259+0000 7fbf4bfff700 1 -- 172.21.15.174:0/1134631383 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf50011830 con 0x7fbf5415bb20 2023-11-08T19:24:34.266 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.259+0000 7fbf4bfff700 1 -- 172.21.15.174:0/1134631383 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf50011990 con 0x7fbf5415bb20 2023-11-08T19:24:34.266 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.259+0000 7fbf4bfff700 1 -- 172.21.15.174:0/1134631383 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf50011d20 con 0x7fbf5415bb20 2023-11-08T19:24:34.266 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.260+0000 7fbf5b545700 1 -- 172.21.15.174:0/1134631383 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbf5415bb20 msgr2=0x7fbf540b2670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:34.266 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.260+0000 7fbf5b545700 1 --2- 172.21.15.174:0/1134631383 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbf5415bb20 0x7fbf540b2670 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fbf5408b850 tx=0x7fbf50009b20 comp rx=0 tx=0).stop 2023-11-08T19:24:34.266 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.260+0000 7fbf5b545700 1 -- 172.21.15.174:0/1134631383 shutdown_connections 2023-11-08T19:24:34.267 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.260+0000 7fbf5b545700 1 --2- 172.21.15.174:0/1134631383 >> v2:172.21.15.143:3300/0 conn(0x7fbf540d1d80 0x7fbf540cc4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:34.267 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.260+0000 7fbf5b545700 1 --2- 172.21.15.174:0/1134631383 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbf5415bb20 0x7fbf540b2670 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:34.267 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.260+0000 7fbf5b545700 1 -- 172.21.15.174:0/1134631383 >> 172.21.15.174:0/1134631383 conn(0x7fbf5408b000 msgr2=0x7fbf5409ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:24:34.267 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.260+0000 7fbf5b545700 1 -- 172.21.15.174:0/1134631383 shutdown_connections 2023-11-08T19:24:34.267 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.260+0000 7fbf5b545700 1 -- 172.21.15.174:0/1134631383 wait complete. 2023-11-08T19:24:34.270 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.261+0000 7fbf5b545700 1 Processor -- start 2023-11-08T19:24:34.270 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.261+0000 7fbf5b545700 1 -- start start 2023-11-08T19:24:34.270 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.262+0000 7fbf5b545700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbf540b1730 0x7fbf54157fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:34.271 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.262+0000 7fbf5b545700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbf540d1d80 0x7fbf5406c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:34.271 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.262+0000 7fbf5b545700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf5415bb20 0x7fbf54070480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:34.271 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.262+0000 7fbf5b545700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf540cc4d0 con 0x7fbf5415bb20 2023-11-08T19:24:34.271 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.262+0000 7fbf5b545700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf54158500 con 0x7fbf540d1d80 2023-11-08T19:24:34.271 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.262+0000 7fbf5b545700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf5406cb70 con 0x7fbf540b1730 2023-11-08T19:24:34.272 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.262+0000 7fbf592e1700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbf540b1730 0x7fbf54157fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:34.272 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.262+0000 7fbf592e1700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbf540b1730 0x7fbf54157fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.174:46966/0 (socket says 172.21.15.174:46966) 2023-11-08T19:24:34.272 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.262+0000 7fbf592e1700 1 -- 172.21.15.174:0/934314076 learned_addr learned my addr 172.21.15.174:0/934314076 (peer_addr_for_me v2:172.21.15.174:0/0) 2023-11-08T19:24:34.272 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.262+0000 7fbf58ae0700 1 --2- 172.21.15.174:0/934314076 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbf540d1d80 0x7fbf5406c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:34.272 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.262+0000 7fbf59ae2700 1 --2- 172.21.15.174:0/934314076 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf5415bb20 0x7fbf54070480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:34.273 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.262+0000 7fbf592e1700 1 -- 172.21.15.174:0/934314076 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbf540d1d80 msgr2=0x7fbf5406c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:34.273 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.262+0000 7fbf592e1700 1 --2- 172.21.15.174:0/934314076 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbf540d1d80 0x7fbf5406c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:34.273 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.262+0000 7fbf592e1700 1 -- 172.21.15.174:0/934314076 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf5415bb20 msgr2=0x7fbf54070480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:34.273 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.262+0000 7fbf592e1700 1 --2- 172.21.15.174:0/934314076 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf5415bb20 0x7fbf54070480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:34.273 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.262+0000 7fbf592e1700 1 -- 172.21.15.174:0/934314076 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf500094a0 con 0x7fbf540b1730 2023-11-08T19:24:34.274 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.263+0000 7fbf592e1700 1 --2- 172.21.15.174:0/934314076 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbf540b1730 0x7fbf54157fc0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fbf50000c00 tx=0x7fbf50004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:34.274 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.263+0000 7fbf49ffb700 1 -- 172.21.15.174:0/934314076 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf500036a0 con 0x7fbf540b1730 2023-11-08T19:24:34.274 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.263+0000 7fbf49ffb700 1 -- 172.21.15.174:0/934314076 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf5001f070 con 0x7fbf540b1730 2023-11-08T19:24:34.274 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.263+0000 7fbf49ffb700 1 -- 172.21.15.174:0/934314076 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf50003a20 con 0x7fbf540b1730 2023-11-08T19:24:34.274 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.263+0000 7fbf5b545700 1 -- 172.21.15.174:0/934314076 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf5406cdf0 con 0x7fbf540b1730 2023-11-08T19:24:34.275 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.263+0000 7fbf5b545700 1 -- 172.21.15.174:0/934314076 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf5406d3b0 con 0x7fbf540b1730 2023-11-08T19:24:34.275 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.263+0000 7fbf49ffb700 1 -- 172.21.15.174:0/934314076 <== mon.2 v2:172.21.15.174:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fbf50026480 con 0x7fbf540b1730 2023-11-08T19:24:34.275 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.263+0000 7fbf49ffb700 1 -- 172.21.15.174:0/934314076 <== mon.2 v2:172.21.15.174:3301/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7fbf50021070 con 0x7fbf540b1730 2023-11-08T19:24:34.275 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.264+0000 7fbf5b545700 1 -- 172.21.15.174:0/934314076 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf5404eee0 con 0x7fbf540b1730 2023-11-08T19:24:34.275 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.266+0000 7fbf49ffb700 1 -- 172.21.15.174:0/934314076 <== mon.2 v2:172.21.15.174:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7fbf50045b40 con 0x7fbf540b1730 2023-11-08T19:24:34.276 INFO:tasks.ceph.mgr.y.smithi143.stderr:2023-11-08T19:24:34.271+0000 7fd1e98fa200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-08T19:24:34.352 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.350+0000 7fbf5b545700 1 -- 172.21.15.174:0/934314076 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_command({"prefix": "osd new", "uuid": "4e2783cf-ebb9-4dd8-8245-bc1d17ce4325", "id": 3} v 0) v1 -- 0x7fbf5415f2d0 con 0x7fbf540b1730 2023-11-08T19:24:34.354 INFO:tasks.ceph.mgr.y.smithi143.stderr:2023-11-08T19:24:34.353+0000 7fd1e98fa200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-08T19:24:34.357 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.356+0000 7fbf49ffb700 1 -- 172.21.15.174:0/934314076 <== mon.2 v2:172.21.15.174:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "4e2783cf-ebb9-4dd8-8245-bc1d17ce4325", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7fbf50034030 con 0x7fbf540b1730 2023-11-08T19:24:34.357 INFO:teuthology.orchestra.run.smithi174.stdout:3 2023-11-08T19:24:34.359 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.356+0000 7fbf5b545700 1 -- 172.21.15.174:0/934314076 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbf540b1730 msgr2=0x7fbf54157fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:34.359 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.356+0000 7fbf5b545700 1 --2- 172.21.15.174:0/934314076 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbf540b1730 0x7fbf54157fc0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fbf50000c00 tx=0x7fbf50004750 comp rx=0 tx=0).stop 2023-11-08T19:24:34.359 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.356+0000 7fbf5b545700 1 -- 172.21.15.174:0/934314076 shutdown_connections 2023-11-08T19:24:34.359 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.356+0000 7fbf5b545700 1 --2- 172.21.15.174:0/934314076 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbf540b1730 0x7fbf54157fc0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:34.359 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.356+0000 7fbf5b545700 1 --2- 172.21.15.174:0/934314076 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf5415bb20 0x7fbf54070480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:34.360 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.356+0000 7fbf5b545700 1 --2- 172.21.15.174:0/934314076 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbf540d1d80 0x7fbf5406c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:34.360 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.356+0000 7fbf5b545700 1 -- 172.21.15.174:0/934314076 >> 172.21.15.174:0/934314076 conn(0x7fbf5408b000 msgr2=0x7fbf5415a000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:24:34.360 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.357+0000 7fbf5b545700 1 -- 172.21.15.174:0/934314076 shutdown_connections 2023-11-08T19:24:34.360 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.357+0000 7fbf5b545700 1 -- 172.21.15.174:0/934314076 wait complete. 2023-11-08T19:24:34.369 DEBUG:teuthology.orchestra.run.smithi174:> sudo ceph --cluster ceph osd new 691256fa-9b77-49b9-a346-7ae1adf0ca80 4 2023-11-08T19:24:34.491 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.489+0000 7f15a196e700 1 Processor -- start 2023-11-08T19:24:34.491 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.489+0000 7f15a196e700 1 -- start start 2023-11-08T19:24:34.493 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.491+0000 7f15a196e700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f159c072010 0x7f159c15b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:34.494 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.491+0000 7f15a196e700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f159c072c40 con 0x7f159c158610 2023-11-08T19:24:34.494 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.491+0000 7f15a196e700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f159c15be90 con 0x7f159c0cc350 2023-11-08T19:24:34.494 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.491+0000 7f15a196e700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f159c0cf150 con 0x7f159c072010 2023-11-08T19:24:34.494 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.491+0000 7f159a7fc700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f159c072010 0x7f159c15b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:34.494 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.491+0000 7f159b7fe700 1 --1- >> v1:172.21.15.174:6789/0 conn(0x7f159c0cc350 0x7f159c06aff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.174:6789/0 says I am v1:172.21.15.174:52210/0 (socket says 172.21.15.174:52210) 2023-11-08T19:24:34.495 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.491+0000 7f159b7fe700 1 -- 172.21.15.174:0/294126705 learned_addr learned my addr 172.21.15.174:0/294126705 (peer_addr_for_me v1:172.21.15.174:0/0) 2023-11-08T19:24:34.495 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.491+0000 7f159a7fc700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f159c072010 0x7f159c15b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:47210/0 (socket says 172.21.15.174:47210) 2023-11-08T19:24:34.495 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.491+0000 7f159a7fc700 1 -- 172.21.15.174:0/294126705 >> v1:172.21.15.174:6789/0 conn(0x7f159c0cc350 legacy=0x7f159c06aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:34.495 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.491+0000 7f1599ffb700 1 -- 172.21.15.174:0/294126705 <== mon.1 v1:172.21.15.174:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2645745061 0 0) 0x7f1594001490 con 0x7f159c0cc350 2023-11-08T19:24:34.496 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.491+0000 7f159a7fc700 1 -- 172.21.15.174:0/294126705 >> v1:172.21.15.143:6789/0 conn(0x7f159c158610 legacy=0x7f159c071900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:34.496 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.491+0000 7f159a7fc700 1 -- 172.21.15.174:0/294126705 --> v2:172.21.15.174:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f159c0cf360 con 0x7f159c072010 2023-11-08T19:24:34.496 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.491+0000 7f159a7fc700 1 --2- 172.21.15.174:0/294126705 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f159c072010 0x7f159c15b950 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f1590009840 tx=0x7f1590009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d58d0fb4e9d394dc server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:34.496 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.491+0000 7f1599ffb700 1 -- 172.21.15.174:0/294126705 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1590011830 con 0x7f159c072010 2023-11-08T19:24:34.497 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.492+0000 7f1599ffb700 1 -- 172.21.15.174:0/294126705 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1590011990 con 0x7f159c072010 2023-11-08T19:24:34.497 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.492+0000 7f1599ffb700 1 -- 172.21.15.174:0/294126705 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1590011d20 con 0x7f159c072010 2023-11-08T19:24:34.500 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.492+0000 7f15a196e700 1 -- 172.21.15.174:0/294126705 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f159c072010 msgr2=0x7f159c15b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:34.500 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.492+0000 7f15a196e700 1 --2- 172.21.15.174:0/294126705 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f159c072010 0x7f159c15b950 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f1590009840 tx=0x7f1590009b50 comp rx=0 tx=0).stop 2023-11-08T19:24:34.500 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.492+0000 7f15a196e700 1 -- 172.21.15.174:0/294126705 shutdown_connections 2023-11-08T19:24:34.500 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.492+0000 7f15a196e700 1 --2- 172.21.15.174:0/294126705 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f159c072010 0x7f159c15b950 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:34.500 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.492+0000 7f15a196e700 1 -- 172.21.15.174:0/294126705 >> 172.21.15.174:0/294126705 conn(0x7f159c0b1c70 msgr2=0x7f159c0b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:24:34.501 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.492+0000 7f15a196e700 1 -- 172.21.15.174:0/294126705 shutdown_connections 2023-11-08T19:24:34.501 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.493+0000 7f15a196e700 1 -- 172.21.15.174:0/294126705 wait complete. 2023-11-08T19:24:34.501 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.494+0000 7f15a196e700 1 Processor -- start 2023-11-08T19:24:34.501 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.495+0000 7f15a196e700 1 -- start start 2023-11-08T19:24:34.501 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.496+0000 7f15a196e700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f159c072010 0x7f159c0b4af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:34.502 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.496+0000 7f15a196e700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f159c0cc350 0x7f159c0b3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:34.502 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.496+0000 7f15a196e700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f159c158610 0x7f159c0c9830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:34.502 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.496+0000 7f15a196e700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f159c0b2cc0 con 0x7f159c158610 2023-11-08T19:24:34.502 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.496+0000 7f15a196e700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f159c0b3280 con 0x7f159c0cc350 2023-11-08T19:24:34.502 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.496+0000 7f15a196e700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f159c14ce90 con 0x7f159c072010 2023-11-08T19:24:34.503 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.496+0000 7f159affd700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f159c072010 0x7f159c0b4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:34.503 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.496+0000 7f159a7fc700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f159c0cc350 0x7f159c0b3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:34.503 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.496+0000 7f159affd700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f159c072010 0x7f159c0b4af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.174:46978/0 (socket says 172.21.15.174:46978) 2023-11-08T19:24:34.503 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.496+0000 7f159affd700 1 -- 172.21.15.174:0/2403493722 learned_addr learned my addr 172.21.15.174:0/2403493722 (peer_addr_for_me v2:172.21.15.174:0/0) 2023-11-08T19:24:34.503 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.496+0000 7f159b7fe700 1 --2- 172.21.15.174:0/2403493722 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f159c158610 0x7f159c0c9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:34.504 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.496+0000 7f159a7fc700 1 -- 172.21.15.174:0/2403493722 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f159c072010 msgr2=0x7f159c0b4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:34.504 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.496+0000 7f159a7fc700 1 --2- 172.21.15.174:0/2403493722 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f159c072010 0x7f159c0b4af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:34.504 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.496+0000 7f159a7fc700 1 -- 172.21.15.174:0/2403493722 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f159c158610 msgr2=0x7f159c0c9830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:34.504 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.496+0000 7f159a7fc700 1 --2- 172.21.15.174:0/2403493722 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f159c158610 0x7f159c0c9830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:34.504 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.496+0000 7f159a7fc700 1 -- 172.21.15.174:0/2403493722 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15900094a0 con 0x7f159c0cc350 2023-11-08T19:24:34.504 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.496+0000 7f159affd700 1 --2- 172.21.15.174:0/2403493722 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f159c072010 0x7f159c0b4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T19:24:34.505 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.496+0000 7f159a7fc700 1 --2- 172.21.15.174:0/2403493722 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f159c0cc350 0x7f159c0b3710 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1590016000 tx=0x7f1590004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:34.505 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.497+0000 7f1583fff700 1 -- 172.21.15.174:0/2403493722 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f159001a7e0 con 0x7f159c0cc350 2023-11-08T19:24:34.505 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.497+0000 7f1583fff700 1 -- 172.21.15.174:0/2403493722 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f159001a940 con 0x7f159c0cc350 2023-11-08T19:24:34.505 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.497+0000 7f15a196e700 1 -- 172.21.15.174:0/2403493722 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f159c0b5180 con 0x7f159c0cc350 2023-11-08T19:24:34.505 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.497+0000 7f1583fff700 1 -- 172.21.15.174:0/2403493722 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f159001acb0 con 0x7f159c0cc350 2023-11-08T19:24:34.505 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.497+0000 7f15a196e700 1 -- 172.21.15.174:0/2403493722 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f159c0b2f10 con 0x7f159c0cc350 2023-11-08T19:24:34.506 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.497+0000 7f1583fff700 1 -- 172.21.15.174:0/2403493722 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f1590022070 con 0x7f159c0cc350 2023-11-08T19:24:34.506 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.497+0000 7f1583fff700 1 -- 172.21.15.174:0/2403493722 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7f1590024930 con 0x7f159c0cc350 2023-11-08T19:24:34.506 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.498+0000 7f15a196e700 1 -- 172.21.15.174:0/2403493722 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f159c0b56a0 con 0x7f159c0cc350 2023-11-08T19:24:34.507 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.500+0000 7f1583fff700 1 -- 172.21.15.174:0/2403493722 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f1590029070 con 0x7f159c0cc350 2023-11-08T19:24:34.509 INFO:tasks.ceph.mgr.x.smithi174.stderr:2023-11-08T19:24:34.507+0000 7fd2ab6d0200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-08T19:24:34.592 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.590+0000 7f15a196e700 1 -- 172.21.15.174:0/2403493722 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd new", "uuid": "691256fa-9b77-49b9-a346-7ae1adf0ca80", "id": 4} v 0) v1 -- 0x7f159c04eee0 con 0x7f159c0cc350 2023-11-08T19:24:34.597 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.595+0000 7f1583fff700 1 -- 172.21.15.174:0/2403493722 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "691256fa-9b77-49b9-a346-7ae1adf0ca80", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f1590015020 con 0x7f159c0cc350 2023-11-08T19:24:34.597 INFO:teuthology.orchestra.run.smithi174.stdout:4 2023-11-08T19:24:34.598 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.596+0000 7f15a196e700 1 -- 172.21.15.174:0/2403493722 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f159c0cc350 msgr2=0x7f159c0b3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:34.599 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.596+0000 7f15a196e700 1 --2- 172.21.15.174:0/2403493722 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f159c0cc350 0x7f159c0b3710 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1590016000 tx=0x7f1590004750 comp rx=0 tx=0).stop 2023-11-08T19:24:34.599 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.596+0000 7f15a196e700 1 -- 172.21.15.174:0/2403493722 shutdown_connections 2023-11-08T19:24:34.599 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.596+0000 7f15a196e700 1 --2- 172.21.15.174:0/2403493722 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f159c072010 0x7f159c0b4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:34.599 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.596+0000 7f15a196e700 1 --2- 172.21.15.174:0/2403493722 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f159c158610 0x7f159c0c9830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:34.599 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.596+0000 7f15a196e700 1 --2- 172.21.15.174:0/2403493722 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f159c0cc350 0x7f159c0b3710 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:34.600 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.596+0000 7f15a196e700 1 -- 172.21.15.174:0/2403493722 >> 172.21.15.174:0/2403493722 conn(0x7f159c0b1c70 msgr2=0x7f159c0b0bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:24:34.600 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.596+0000 7f15a196e700 1 -- 172.21.15.174:0/2403493722 shutdown_connections 2023-11-08T19:24:34.600 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.596+0000 7f15a196e700 1 -- 172.21.15.174:0/2403493722 wait complete. 2023-11-08T19:24:34.610 DEBUG:teuthology.orchestra.run.smithi174:> sudo ceph --cluster ceph osd new 756fc352-48c9-46ce-b52e-0541fe6f8525 5 2023-11-08T19:24:34.731 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.729+0000 7f60aaeab700 1 Processor -- start 2023-11-08T19:24:34.731 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.729+0000 7f60aaeab700 1 -- start start 2023-11-08T19:24:34.733 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.730+0000 7f60aaeab700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f60a415bb20 0x7f60a40b2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:34.734 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.730+0000 7f60aaeab700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f60a40d1d80 0x7f60a40cc4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:34.734 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.730+0000 7f60aaeab700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60a40d0420 con 0x7f60a40b1730 2023-11-08T19:24:34.734 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.730+0000 7f60aaeab700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60a40cf600 con 0x7f60a415bb20 2023-11-08T19:24:34.734 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.730+0000 7f60aaeab700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60a40cf810 con 0x7f60a40d1d80 2023-11-08T19:24:34.735 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.730+0000 7f60a9448700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f60a40d1d80 0x7f60a40cc4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:34.735 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.730+0000 7f60a8c47700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f60a415bb20 0x7f60a40b2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:34.735 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.730+0000 7f60a9448700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f60a40d1d80 0x7f60a40cc4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.174:46986/0 (socket says 172.21.15.174:46986) 2023-11-08T19:24:34.735 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.730+0000 7f60a8c47700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f60a415bb20 0x7f60a40b2670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:47220/0 (socket says 172.21.15.174:47220) 2023-11-08T19:24:34.735 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.730+0000 7f60a3fff700 1 --1- >> v1:172.21.15.174:6789/0 conn(0x7f60a40b1730 0x7f60a40d1670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.174:6789/0 says I am v1:172.21.15.174:52218/0 (socket says 172.21.15.174:52218) 2023-11-08T19:24:34.736 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.730+0000 7f60a9448700 1 -- 172.21.15.174:0/2891196161 learned_addr learned my addr 172.21.15.174:0/2891196161 (peer_addr_for_me v2:172.21.15.174:0/0) 2023-11-08T19:24:34.736 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.730+0000 7f60a9448700 1 -- 172.21.15.174:0/2891196161 >> v2:172.21.15.174:3300/0 conn(0x7f60a415bb20 msgr2=0x7f60a40b2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:24:34.736 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.730+0000 7f60a9448700 1 --2- 172.21.15.174:0/2891196161 >> v2:172.21.15.174:3300/0 conn(0x7f60a415bb20 0x7f60a40b2670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:34.736 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.730+0000 7f60a9448700 1 -- 172.21.15.174:0/2891196161 >> v1:172.21.15.174:6789/0 conn(0x7f60a40b1730 legacy=0x7f60a40d1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:34.736 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.730+0000 7f60a9448700 1 -- 172.21.15.174:0/2891196161 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60a40cfaf0 con 0x7f60a40d1d80 2023-11-08T19:24:34.736 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.730+0000 7f60a8c47700 1 --2- 172.21.15.174:0/2891196161 >> v2:172.21.15.174:3300/0 conn(0x7f60a415bb20 0x7f60a40b2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T19:24:34.737 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.730+0000 7f60a9448700 1 --2- 172.21.15.174:0/2891196161 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f60a40d1d80 0x7f60a40cc4d0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f609c009e50 tx=0x7f609c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=825a5039e52c785 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:34.737 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.731+0000 7f60a37fe700 1 -- 172.21.15.174:0/2891196161 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f609c00e070 con 0x7f60a40d1d80 2023-11-08T19:24:34.737 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.731+0000 7f60a37fe700 1 -- 172.21.15.174:0/2891196161 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f609c00b910 con 0x7f60a40d1d80 2023-11-08T19:24:34.737 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.731+0000 7f60a37fe700 1 -- 172.21.15.174:0/2891196161 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f609c00bcc0 con 0x7f60a40d1d80 2023-11-08T19:24:34.737 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.731+0000 7f60aaeab700 1 -- 172.21.15.174:0/2891196161 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f60a40d1d80 msgr2=0x7f60a40cc4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:34.738 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.731+0000 7f60aaeab700 1 --2- 172.21.15.174:0/2891196161 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f60a40d1d80 0x7f60a40cc4d0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f609c009e50 tx=0x7f609c00b330 comp rx=0 tx=0).stop 2023-11-08T19:24:34.738 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.731+0000 7f60aaeab700 1 -- 172.21.15.174:0/2891196161 shutdown_connections 2023-11-08T19:24:34.738 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.731+0000 7f60aaeab700 1 --2- 172.21.15.174:0/2891196161 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f60a40d1d80 0x7f60a40cc4d0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:34.738 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.731+0000 7f60aaeab700 1 --2- 172.21.15.174:0/2891196161 >> v2:172.21.15.174:3300/0 conn(0x7f60a415bb20 0x7f60a40b2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:34.738 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.731+0000 7f60aaeab700 1 -- 172.21.15.174:0/2891196161 >> 172.21.15.174:0/2891196161 conn(0x7f60a408b000 msgr2=0x7f60a409ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:24:34.739 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.731+0000 7f60aaeab700 1 -- 172.21.15.174:0/2891196161 shutdown_connections 2023-11-08T19:24:34.739 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.731+0000 7f60aaeab700 1 -- 172.21.15.174:0/2891196161 wait complete. 2023-11-08T19:24:34.741 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.732+0000 7f60aaeab700 1 Processor -- start 2023-11-08T19:24:34.742 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.732+0000 7f60aaeab700 1 -- start start 2023-11-08T19:24:34.742 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.733+0000 7f60aaeab700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f60a40b1730 0x7f60a414b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:34.742 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.733+0000 7f60aaeab700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f60a40d1d80 0x7f60a415d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:34.742 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.733+0000 7f60aaeab700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f60a415bb20 0x7f60a414a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:34.743 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.733+0000 7f60aaeab700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60a415e160 con 0x7f60a40b1730 2023-11-08T19:24:34.743 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.733+0000 7f60aaeab700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60a415e440 con 0x7f60a40d1d80 2023-11-08T19:24:34.743 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.733+0000 7f60aaeab700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60a415e720 con 0x7f60a415bb20 2023-11-08T19:24:34.743 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.733+0000 7f60a3fff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f60a40d1d80 0x7f60a415d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:34.743 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.733+0000 7f60a9448700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f60a415bb20 0x7f60a414a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:34.744 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.733+0000 7f60a3fff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f60a40d1d80 0x7f60a415d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:47226/0 (socket says 172.21.15.174:47226) 2023-11-08T19:24:34.744 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.733+0000 7f60a9448700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f60a415bb20 0x7f60a414a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.174:46992/0 (socket says 172.21.15.174:46992) 2023-11-08T19:24:34.744 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.733+0000 7f60a3fff700 1 -- 172.21.15.174:0/1400588008 learned_addr learned my addr 172.21.15.174:0/1400588008 (peer_addr_for_me v2:172.21.15.174:0/0) 2023-11-08T19:24:34.744 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.734+0000 7f60a8c47700 1 --2- 172.21.15.174:0/1400588008 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f60a40b1730 0x7f60a414b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:34.744 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.734+0000 7f60a9448700 1 -- 172.21.15.174:0/1400588008 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f60a40d1d80 msgr2=0x7f60a415d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:34.744 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.734+0000 7f60a9448700 1 --2- 172.21.15.174:0/1400588008 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f60a40d1d80 0x7f60a415d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:34.745 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.734+0000 7f60a9448700 1 -- 172.21.15.174:0/1400588008 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f60a40b1730 msgr2=0x7f60a414b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:34.745 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.734+0000 7f60a9448700 1 --2- 172.21.15.174:0/1400588008 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f60a40b1730 0x7f60a414b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:34.745 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.734+0000 7f60a9448700 1 -- 172.21.15.174:0/1400588008 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60980094a0 con 0x7f60a415bb20 2023-11-08T19:24:34.745 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.734+0000 7f60a9448700 1 --2- 172.21.15.174:0/1400588008 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f60a415bb20 0x7f60a414a560 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f609c015000 tx=0x7f609c004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:34.745 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.734+0000 7f60a17fa700 1 -- 172.21.15.174:0/1400588008 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f609c01de90 con 0x7f60a415bb20 2023-11-08T19:24:34.746 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.734+0000 7f60a17fa700 1 -- 172.21.15.174:0/1400588008 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f609c0099b0 con 0x7f60a415bb20 2023-11-08T19:24:34.746 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.734+0000 7f60aaeab700 1 -- 172.21.15.174:0/1400588008 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f609c009b30 con 0x7f60a415bb20 2023-11-08T19:24:34.746 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.734+0000 7f60a17fa700 1 -- 172.21.15.174:0/1400588008 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f609c004280 con 0x7f60a415bb20 2023-11-08T19:24:34.746 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.734+0000 7f60aaeab700 1 -- 172.21.15.174:0/1400588008 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60a415ece0 con 0x7f60a415bb20 2023-11-08T19:24:34.746 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.734+0000 7f60a17fa700 1 -- 172.21.15.174:0/1400588008 <== mon.2 v2:172.21.15.174:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f609c0046e0 con 0x7f60a415bb20 2023-11-08T19:24:34.747 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.734+0000 7f60a17fa700 1 -- 172.21.15.174:0/1400588008 <== mon.2 v2:172.21.15.174:3301/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7f609c0123f0 con 0x7f60a415bb20 2023-11-08T19:24:34.747 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.735+0000 7f60aaeab700 1 -- 172.21.15.174:0/1400588008 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60a404eee0 con 0x7f60a415bb20 2023-11-08T19:24:34.747 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.737+0000 7f60a17fa700 1 -- 172.21.15.174:0/1400588008 <== mon.2 v2:172.21.15.174:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f609c02a070 con 0x7f60a415bb20 2023-11-08T19:24:34.779 INFO:tasks.ceph.mgr.y.smithi143.stderr:2023-11-08T19:24:34.778+0000 7fd1e98fa200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-08T19:24:34.823 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.821+0000 7f60aaeab700 1 -- 172.21.15.174:0/1400588008 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_command({"prefix": "osd new", "uuid": "756fc352-48c9-46ce-b52e-0541fe6f8525", "id": 5} v 0) v1 -- 0x7f60a415b690 con 0x7f60a415bb20 2023-11-08T19:24:34.831 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.829+0000 7f60a17fa700 1 -- 172.21.15.174:0/1400588008 <== mon.2 v2:172.21.15.174:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "756fc352-48c9-46ce-b52e-0541fe6f8525", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f609c0253f0 con 0x7f60a415bb20 2023-11-08T19:24:34.831 INFO:teuthology.orchestra.run.smithi174.stdout:5 2023-11-08T19:24:34.832 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.830+0000 7f60aaeab700 1 -- 172.21.15.174:0/1400588008 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f60a415bb20 msgr2=0x7f60a414a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:34.832 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.830+0000 7f60aaeab700 1 --2- 172.21.15.174:0/1400588008 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f60a415bb20 0x7f60a414a560 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f609c015000 tx=0x7f609c004ad0 comp rx=0 tx=0).stop 2023-11-08T19:24:34.833 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.830+0000 7f60aaeab700 1 -- 172.21.15.174:0/1400588008 shutdown_connections 2023-11-08T19:24:34.833 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.830+0000 7f60aaeab700 1 --2- 172.21.15.174:0/1400588008 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f60a415bb20 0x7f60a414a560 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:34.833 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.830+0000 7f60aaeab700 1 --2- 172.21.15.174:0/1400588008 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f60a40b1730 0x7f60a414b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:34.833 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.830+0000 7f60aaeab700 1 --2- 172.21.15.174:0/1400588008 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f60a40d1d80 0x7f60a415d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:34.833 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.830+0000 7f60aaeab700 1 -- 172.21.15.174:0/1400588008 >> 172.21.15.174:0/1400588008 conn(0x7f60a408b000 msgr2=0x7f60a4070c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:24:34.834 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.830+0000 7f60aaeab700 1 -- 172.21.15.174:0/1400588008 shutdown_connections 2023-11-08T19:24:34.834 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.830+0000 7f60aaeab700 1 -- 172.21.15.174:0/1400588008 wait complete. 2023-11-08T19:24:34.846 DEBUG:teuthology.orchestra.run.smithi174:> sudo ceph --cluster ceph osd new fab2ac2b-ed97-4cde-9825-b480eabf8b8d 6 2023-11-08T19:24:34.991 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.990+0000 7f1ba2b3d700 1 Processor -- start 2023-11-08T19:24:34.992 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.990+0000 7f1ba2b3d700 1 -- start start 2023-11-08T19:24:34.994 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.991+0000 7f1ba2b3d700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f1b9c15df20 0x7f1b9c0b25f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:34.994 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.991+0000 7f1ba2b3d700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1b9c0d20d0 0x7f1b9c0cfd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:34.995 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.991+0000 7f1ba2b3d700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b9c0b2b30 con 0x7f1b9c06bd20 2023-11-08T19:24:34.995 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.991+0000 7f1ba2b3d700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b9c06c2c0 con 0x7f1b9c15df20 2023-11-08T19:24:34.995 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.991+0000 7f1ba2b3d700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b9c06c5a0 con 0x7f1b9c0d20d0 2023-11-08T19:24:34.995 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.991+0000 7f1ba08d9700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f1b9c15df20 0x7f1b9c0b25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:34.995 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.991+0000 7f1ba10da700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1b9c0d20d0 0x7f1b9c0cfd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:34.996 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.991+0000 7f1b9bfff700 1 --1- >> v1:172.21.15.174:6789/0 conn(0x7f1b9c06bd20 0x7f1b9c0d19c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.174:6789/0 says I am v1:172.21.15.174:52230/0 (socket says 172.21.15.174:52230) 2023-11-08T19:24:34.996 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.991+0000 7f1b9bfff700 1 -- 172.21.15.174:0/877348085 learned_addr learned my addr 172.21.15.174:0/877348085 (peer_addr_for_me v1:172.21.15.174:0/0) 2023-11-08T19:24:34.996 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.991+0000 7f1ba10da700 1 -- 172.21.15.174:0/877348085 >> v2:172.21.15.174:3300/0 conn(0x7f1b9c15df20 msgr2=0x7f1b9c0b25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:24:34.996 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.991+0000 7f1ba10da700 1 --2- 172.21.15.174:0/877348085 >> v2:172.21.15.174:3300/0 conn(0x7f1b9c15df20 0x7f1b9c0b25f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:34.996 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.991+0000 7f1ba10da700 1 -- 172.21.15.174:0/877348085 >> v1:172.21.15.174:6789/0 conn(0x7f1b9c06bd20 legacy=0x7f1b9c0d19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:34.997 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.991+0000 7f1ba10da700 1 -- 172.21.15.174:0/877348085 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b9c06c880 con 0x7f1b9c0d20d0 2023-11-08T19:24:34.997 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.991+0000 7f1ba08d9700 1 --2- 172.21.15.174:0/877348085 >> v2:172.21.15.174:3300/0 conn(0x7f1b9c15df20 0x7f1b9c0b25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T19:24:34.997 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.991+0000 7f1ba10da700 1 --2- 172.21.15.174:0/877348085 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1b9c0d20d0 0x7f1b9c0cfd30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f1b94009d90 tx=0x7f1b9400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c229f99d9ae4f07 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:34.997 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.991+0000 7f1b9b7fe700 1 -- 172.21.15.174:0/877348085 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b9400e070 con 0x7f1b9c0d20d0 2023-11-08T19:24:34.997 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.991+0000 7f1b9b7fe700 1 -- 172.21.15.174:0/877348085 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b9400bb20 con 0x7f1b9c0d20d0 2023-11-08T19:24:34.998 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.991+0000 7f1b9b7fe700 1 -- 172.21.15.174:0/877348085 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b94012430 con 0x7f1b9c0d20d0 2023-11-08T19:24:34.998 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.992+0000 7f1ba2b3d700 1 -- 172.21.15.174:0/877348085 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1b9c0d20d0 msgr2=0x7f1b9c0cfd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:34.998 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.992+0000 7f1ba2b3d700 1 --2- 172.21.15.174:0/877348085 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1b9c0d20d0 0x7f1b9c0cfd30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f1b94009d90 tx=0x7f1b9400b330 comp rx=0 tx=0).stop 2023-11-08T19:24:34.998 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.992+0000 7f1ba2b3d700 1 -- 172.21.15.174:0/877348085 shutdown_connections 2023-11-08T19:24:34.998 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.992+0000 7f1ba2b3d700 1 --2- 172.21.15.174:0/877348085 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1b9c0d20d0 0x7f1b9c0cfd30 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:34.999 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.992+0000 7f1ba2b3d700 1 --2- 172.21.15.174:0/877348085 >> v2:172.21.15.174:3300/0 conn(0x7f1b9c15df20 0x7f1b9c0b25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:34.999 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.992+0000 7f1ba2b3d700 1 -- 172.21.15.174:0/877348085 >> 172.21.15.174:0/877348085 conn(0x7f1b9c08b000 msgr2=0x7f1b9c09ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:24:34.999 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.992+0000 7f1ba2b3d700 1 -- 172.21.15.174:0/877348085 shutdown_connections 2023-11-08T19:24:34.999 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.992+0000 7f1ba2b3d700 1 -- 172.21.15.174:0/877348085 wait complete. 2023-11-08T19:24:35.002 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.993+0000 7f1ba2b3d700 1 Processor -- start 2023-11-08T19:24:35.002 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.993+0000 7f1ba2b3d700 1 -- start start 2023-11-08T19:24:35.002 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.994+0000 7f1ba2b3d700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1b9c06bd20 0x7f1b9c1f35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:35.002 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.994+0000 7f1ba2b3d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1b9c0d20d0 0x7f1b9c15c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:35.003 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.994+0000 7f1ba2b3d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1b9c15df20 0x7f1b9c0b46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:35.003 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.994+0000 7f1ba2b3d700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b9c15c6b0 con 0x7f1b9c0d20d0 2023-11-08T19:24:35.003 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.994+0000 7f1ba2b3d700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b9c1f3bc0 con 0x7f1b9c15df20 2023-11-08T19:24:35.003 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.994+0000 7f1ba2b3d700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b9c1f3ea0 con 0x7f1b9c06bd20 2023-11-08T19:24:35.003 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.994+0000 7f1ba08d9700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1b9c06bd20 0x7f1b9c1f35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:35.004 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.994+0000 7f1ba08d9700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1b9c06bd20 0x7f1b9c1f35e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.174:47000/0 (socket says 172.21.15.174:47000) 2023-11-08T19:24:35.004 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.994+0000 7f1ba08d9700 1 -- 172.21.15.174:0/4101669002 learned_addr learned my addr 172.21.15.174:0/4101669002 (peer_addr_for_me v2:172.21.15.174:0/0) 2023-11-08T19:24:35.004 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.994+0000 7f1ba10da700 1 --2- 172.21.15.174:0/4101669002 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1b9c15df20 0x7f1b9c0b46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:35.004 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.994+0000 7f1b9bfff700 1 --2- 172.21.15.174:0/4101669002 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1b9c0d20d0 0x7f1b9c15c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:35.004 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.995+0000 7f1ba08d9700 1 -- 172.21.15.174:0/4101669002 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1b9c15df20 msgr2=0x7f1b9c0b46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:35.005 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.995+0000 7f1ba08d9700 1 --2- 172.21.15.174:0/4101669002 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1b9c15df20 0x7f1b9c0b46e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:35.005 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.995+0000 7f1ba08d9700 1 -- 172.21.15.174:0/4101669002 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1b9c0d20d0 msgr2=0x7f1b9c15c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:35.005 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.995+0000 7f1ba08d9700 1 --2- 172.21.15.174:0/4101669002 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1b9c0d20d0 0x7f1b9c15c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:35.005 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.995+0000 7f1ba08d9700 1 -- 172.21.15.174:0/4101669002 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b900094a0 con 0x7f1b9c06bd20 2023-11-08T19:24:35.005 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.995+0000 7f1b9bfff700 1 --2- 172.21.15.174:0/4101669002 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1b9c0d20d0 0x7f1b9c15c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:24:35.006 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.995+0000 7f1ba08d9700 1 --2- 172.21.15.174:0/4101669002 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1b9c06bd20 0x7f1b9c1f35e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f1b9000ebe0 tx=0x7f1b9000eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:35.006 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.995+0000 7f1b997fa700 1 -- 172.21.15.174:0/4101669002 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b9000c850 con 0x7f1b9c06bd20 2023-11-08T19:24:35.006 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.995+0000 7f1b997fa700 1 -- 172.21.15.174:0/4101669002 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b9000c9b0 con 0x7f1b9c06bd20 2023-11-08T19:24:35.006 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.995+0000 7f1ba2b3d700 1 -- 172.21.15.174:0/4101669002 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b94009a70 con 0x7f1b9c06bd20 2023-11-08T19:24:35.006 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.995+0000 7f1b997fa700 1 -- 172.21.15.174:0/4101669002 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b9000cd40 con 0x7f1b9c06bd20 2023-11-08T19:24:35.007 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.995+0000 7f1ba2b3d700 1 -- 172.21.15.174:0/4101669002 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b9c1f4460 con 0x7f1b9c06bd20 2023-11-08T19:24:35.007 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.995+0000 7f1b997fa700 1 -- 172.21.15.174:0/4101669002 <== mon.2 v2:172.21.15.174:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f1b90005d20 con 0x7f1b9c06bd20 2023-11-08T19:24:35.007 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.995+0000 7f1b997fa700 1 -- 172.21.15.174:0/4101669002 <== mon.2 v2:172.21.15.174:3301/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7f1b90006840 con 0x7f1b9c06bd20 2023-11-08T19:24:35.007 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.996+0000 7f1ba2b3d700 1 -- 172.21.15.174:0/4101669002 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b9c0b6680 con 0x7f1b9c06bd20 2023-11-08T19:24:35.007 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:34.998+0000 7f1b997fa700 1 -- 172.21.15.174:0/4101669002 <== mon.2 v2:172.21.15.174:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f1b90021060 con 0x7f1b9c06bd20 2023-11-08T19:24:35.082 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:35.080+0000 7f1ba2b3d700 1 -- 172.21.15.174:0/4101669002 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_command({"prefix": "osd new", "uuid": "fab2ac2b-ed97-4cde-9825-b480eabf8b8d", "id": 6} v 0) v1 -- 0x7f1b9c04eee0 con 0x7f1b9c06bd20 2023-11-08T19:24:35.087 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:35.085+0000 7f1b997fa700 1 -- 172.21.15.174:0/4101669002 <== mon.2 v2:172.21.15.174:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "fab2ac2b-ed97-4cde-9825-b480eabf8b8d", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f1b90017070 con 0x7f1b9c06bd20 2023-11-08T19:24:35.087 INFO:teuthology.orchestra.run.smithi174.stdout:6 2023-11-08T19:24:35.088 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:35.086+0000 7f1ba2b3d700 1 -- 172.21.15.174:0/4101669002 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1b9c06bd20 msgr2=0x7f1b9c1f35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:35.089 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:35.086+0000 7f1ba2b3d700 1 --2- 172.21.15.174:0/4101669002 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1b9c06bd20 0x7f1b9c1f35e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f1b9000ebe0 tx=0x7f1b9000eef0 comp rx=0 tx=0).stop 2023-11-08T19:24:35.089 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:35.086+0000 7f1ba2b3d700 1 -- 172.21.15.174:0/4101669002 shutdown_connections 2023-11-08T19:24:35.089 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:35.086+0000 7f1ba2b3d700 1 --2- 172.21.15.174:0/4101669002 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1b9c06bd20 0x7f1b9c1f35e0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:35.089 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:35.087+0000 7f1ba2b3d700 1 --2- 172.21.15.174:0/4101669002 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1b9c15df20 0x7f1b9c0b46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:35.089 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:35.087+0000 7f1ba2b3d700 1 --2- 172.21.15.174:0/4101669002 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1b9c0d20d0 0x7f1b9c15c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:35.090 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:35.087+0000 7f1ba2b3d700 1 -- 172.21.15.174:0/4101669002 >> 172.21.15.174:0/4101669002 conn(0x7f1b9c08b000 msgr2=0x7f1b9c0b0600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:24:35.090 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:35.087+0000 7f1ba2b3d700 1 -- 172.21.15.174:0/4101669002 shutdown_connections 2023-11-08T19:24:35.090 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:35.087+0000 7f1ba2b3d700 1 -- 172.21.15.174:0/4101669002 wait complete. 2023-11-08T19:24:35.101 DEBUG:teuthology.orchestra.run.smithi174:> sudo ceph --cluster ceph osd new 3de9b797-2ab6-4d41-a863-7e7d638fd703 7 2023-11-08T19:24:35.174 INFO:tasks.ceph.mgr.x.smithi174.stderr:2023-11-08T19:24:35.173+0000 7fd2ab6d0200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-08T19:24:35.222 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:35.220+0000 7f97bde1e700 1 Processor -- start 2023-11-08T19:24:35.222 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:35.220+0000 7f97bde1e700 1 -- start start 2023-11-08T19:24:35.225 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:35.221+0000 7f97bde1e700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f97b806b240 0x7f97b80caff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:35.225 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:35.221+0000 7f97bde1e700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97b8161890 con 0x7f97b815dba0 2023-11-08T19:24:35.225 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:35.221+0000 7f97bde1e700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97b806b740 con 0x7f97b80cb530 2023-11-08T19:24:35.225 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:35.221+0000 7f97bde1e700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97b806cd20 con 0x7f97b806b240 2023-11-08T19:24:35.226 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:35.221+0000 7f97b6ffd700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f97b806b240 0x7f97b80caff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:35.226 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:35.221+0000 7f97b7fff700 1 --1- >> v1:172.21.15.174:6789/0 conn(0x7f97b80cb530 0x7f97b80b22e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.174:6789/0 says I am v1:172.21.15.174:52244/0 (socket says 172.21.15.174:52244) 2023-11-08T19:24:35.226 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:35.221+0000 7f97b7fff700 1 -- 172.21.15.174:0/2601313882 learned_addr learned my addr 172.21.15.174:0/2601313882 (peer_addr_for_me v1:172.21.15.174:0/0) 2023-11-08T19:24:35.226 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:35.221+0000 7f97b6ffd700 1 -- 172.21.15.174:0/2601313882 >> v1:172.21.15.174:6789/0 conn(0x7f97b80cb530 legacy=0x7f97b80b22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:35.227 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:35.221+0000 7f97b6ffd700 1 -- 172.21.15.174:0/2601313882 >> v1:172.21.15.143:6789/0 conn(0x7f97b815dba0 legacy=0x7f97b806c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:35.227 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:35.221+0000 7f97b67fc700 1 -- 172.21.15.174:0/2601313882 <== mon.1 v1:172.21.15.174:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1030843938 0 0) 0x7f97b806b740 con 0x7f97b80cb530 2023-11-08T19:24:35.227 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:35.221+0000 7f97b6ffd700 1 -- 172.21.15.174:0/2601313882 --> v2:172.21.15.174:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97b806ba20 con 0x7f97b806b240 2023-11-08T19:24:35.227 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:35.222+0000 7f97b6ffd700 1 --2- 172.21.15.174:0/2601313882 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f97b806b240 0x7f97b80caff0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f97ac009840 tx=0x7f97ac009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c10243277307b4bd server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:35.227 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:35.222+0000 7f97b67fc700 1 -- 172.21.15.174:0/2601313882 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97ac011830 con 0x7f97b806b240 2023-11-08T19:24:35.228 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:35.222+0000 7f97b67fc700 1 -- 172.21.15.174:0/2601313882 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97ac011990 con 0x7f97b806b240 2023-11-08T19:24:35.228 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:35.222+0000 7f97b67fc700 1 -- 172.21.15.174:0/2601313882 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97ac011d20 con 0x7f97b806b240 2023-11-08T19:24:35.228 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:35.222+0000 7f97bde1e700 1 -- 172.21.15.174:0/2601313882 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f97b806b240 msgr2=0x7f97b80caff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:35.228 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:35.222+0000 7f97bde1e700 1 --2- 172.21.15.174:0/2601313882 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f97b806b240 0x7f97b80caff0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f97ac009840 tx=0x7f97ac009b50 comp rx=0 tx=0).stop 2023-11-08T19:24:35.228 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:35.222+0000 7f97bde1e700 1 -- 172.21.15.174:0/2601313882 shutdown_connections 2023-11-08T19:24:35.228 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:35.222+0000 7f97bde1e700 1 --2- 172.21.15.174:0/2601313882 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f97b806b240 0x7f97b80caff0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:35.229 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:35.222+0000 7f97bde1e700 1 -- 172.21.15.174:0/2601313882 >> 172.21.15.174:0/2601313882 conn(0x7f97b80cd2e0 msgr2=0x7f97b80cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:24:35.229 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:35.222+0000 7f97bde1e700 1 -- 172.21.15.174:0/2601313882 shutdown_connections 2023-11-08T19:24:35.229 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:35.222+0000 7f97bde1e700 1 -- 172.21.15.174:0/2601313882 wait complete. 2023-11-08T19:24:35.232 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:35.224+0000 7f97bde1e700 1 Processor -- start 2023-11-08T19:24:35.232 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:35.224+0000 7f97bde1e700 1 -- start start 2023-11-08T19:24:35.232 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:35.225+0000 7f97bde1e700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f97b806b240 0x7f97b815ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:35.233 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:35.225+0000 7f97bde1e700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f97b80cb530 0x7f97b80c9b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:35.233 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:35.225+0000 7f97bde1e700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f97b815dba0 0x7f97b80aec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:35.233 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:35.225+0000 7f97bde1e700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97b80ca750 con 0x7f97b80cb530 2023-11-08T19:24:35.233 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:35.225+0000 7f97bde1e700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97b80c9560 con 0x7f97b815dba0 2023-11-08T19:24:35.233 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:35.225+0000 7f97bde1e700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97b80c6b50 con 0x7f97b806b240 2023-11-08T19:24:35.234 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:35.225+0000 7f97b77fe700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f97b806b240 0x7f97b815ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:35.234 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:35.225+0000 7f97b7fff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f97b815dba0 0x7f97b80aec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:35.234 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:35.225+0000 7f97b77fe700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f97b806b240 0x7f97b815ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.174:47012/0 (socket says 172.21.15.174:47012) 2023-11-08T19:24:35.234 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:35.225+0000 7f97b7fff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f97b815dba0 0x7f97b80aec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:47252/0 (socket says 172.21.15.174:47252) 2023-11-08T19:24:35.235 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:35.225+0000 7f97b77fe700 1 -- 172.21.15.174:0/870569935 learned_addr learned my addr 172.21.15.174:0/870569935 (peer_addr_for_me v2:172.21.15.174:0/0) 2023-11-08T19:24:35.235 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:35.225+0000 7f97b6ffd700 1 --2- 172.21.15.174:0/870569935 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f97b80cb530 0x7f97b80c9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:35.235 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:35.225+0000 7f97b7fff700 1 -- 172.21.15.174:0/870569935 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f97b806b240 msgr2=0x7f97b815ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:35.235 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:35.225+0000 7f97b7fff700 1 --2- 172.21.15.174:0/870569935 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f97b806b240 0x7f97b815ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:35.235 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:35.225+0000 7f97b7fff700 1 -- 172.21.15.174:0/870569935 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f97b80cb530 msgr2=0x7f97b80c9b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:35.236 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:35.225+0000 7f97b7fff700 1 --2- 172.21.15.174:0/870569935 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f97b80cb530 0x7f97b80c9b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:35.236 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:35.225+0000 7f97b7fff700 1 -- 172.21.15.174:0/870569935 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97ac0094a0 con 0x7f97b815dba0 2023-11-08T19:24:35.236 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:35.225+0000 7f97b77fe700 1 --2- 172.21.15.174:0/870569935 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f97b806b240 0x7f97b815ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T19:24:35.236 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:35.225+0000 7f97b7fff700 1 --2- 172.21.15.174:0/870569935 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f97b815dba0 0x7f97b80aec30 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f97b000cc10 tx=0x7f97b000cf20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:35.236 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:35.225+0000 7f979ffff700 1 -- 172.21.15.174:0/870569935 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97b000aa00 con 0x7f97b815dba0 2023-11-08T19:24:35.236 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:35.225+0000 7f979ffff700 1 -- 172.21.15.174:0/870569935 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97b000ab80 con 0x7f97b815dba0 2023-11-08T19:24:35.237 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:35.225+0000 7f97bde1e700 1 -- 172.21.15.174:0/870569935 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97b80c6e30 con 0x7f97b815dba0 2023-11-08T19:24:35.237 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:35.225+0000 7f979ffff700 1 -- 172.21.15.174:0/870569935 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97b00070d0 con 0x7f97b815dba0 2023-11-08T19:24:35.237 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:35.225+0000 7f97bde1e700 1 -- 172.21.15.174:0/870569935 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97b80c7110 con 0x7f97b815dba0 2023-11-08T19:24:35.237 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:35.226+0000 7f979ffff700 1 -- 172.21.15.174:0/870569935 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f97b0007530 con 0x7f97b815dba0 2023-11-08T19:24:35.237 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:35.226+0000 7f979ffff700 1 -- 172.21.15.174:0/870569935 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7f97b0012a60 con 0x7f97b815dba0 2023-11-08T19:24:35.238 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:35.227+0000 7f97bde1e700 1 -- 172.21.15.174:0/870569935 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97b804eee0 con 0x7f97b815dba0 2023-11-08T19:24:35.238 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:35.229+0000 7f979ffff700 1 -- 172.21.15.174:0/870569935 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f97b0018070 con 0x7f97b815dba0 2023-11-08T19:24:35.246 INFO:tasks.ceph.mgr.x.smithi174.stderr:2023-11-08T19:24:35.245+0000 7fd2ab6d0200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-08T19:24:35.314 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:35.312+0000 7f97bde1e700 1 -- 172.21.15.174:0/870569935 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd new", "uuid": "3de9b797-2ab6-4d41-a863-7e7d638fd703", "id": 7} v 0) v1 -- 0x7f97b80b67e0 con 0x7f97b815dba0 2023-11-08T19:24:35.321 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:35.320+0000 7f979ffff700 1 -- 172.21.15.174:0/870569935 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "3de9b797-2ab6-4d41-a863-7e7d638fd703", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f97b0007970 con 0x7f97b815dba0 2023-11-08T19:24:35.322 INFO:teuthology.orchestra.run.smithi174.stdout:7 2023-11-08T19:24:35.323 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:35.321+0000 7f97bde1e700 1 -- 172.21.15.174:0/870569935 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f97b815dba0 msgr2=0x7f97b80aec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:35.323 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:35.321+0000 7f97bde1e700 1 --2- 172.21.15.174:0/870569935 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f97b815dba0 0x7f97b80aec30 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f97b000cc10 tx=0x7f97b000cf20 comp rx=0 tx=0).stop 2023-11-08T19:24:35.323 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:35.321+0000 7f97bde1e700 1 -- 172.21.15.174:0/870569935 shutdown_connections 2023-11-08T19:24:35.323 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:35.321+0000 7f97bde1e700 1 --2- 172.21.15.174:0/870569935 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f97b806b240 0x7f97b815ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:35.324 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:35.321+0000 7f97bde1e700 1 --2- 172.21.15.174:0/870569935 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f97b815dba0 0x7f97b80aec30 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:35.324 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:35.321+0000 7f97bde1e700 1 --2- 172.21.15.174:0/870569935 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f97b80cb530 0x7f97b80c9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:35.324 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:35.321+0000 7f97bde1e700 1 -- 172.21.15.174:0/870569935 >> 172.21.15.174:0/870569935 conn(0x7f97b80cd2e0 msgr2=0x7f97b815cd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:24:35.324 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:35.321+0000 7f97bde1e700 1 -- 172.21.15.174:0/870569935 shutdown_connections 2023-11-08T19:24:35.324 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-08T19:24:35.321+0000 7f97bde1e700 1 -- 172.21.15.174:0/870569935 wait complete. 2023-11-08T19:24:35.339 INFO:tasks.ceph.osd.0:Restarting daemon 2023-11-08T19:24:35.340 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-11-08T19:24:35.343 INFO:tasks.ceph.osd.0:Started 2023-11-08T19:24:35.343 INFO:tasks.ceph.osd.1:Restarting daemon 2023-11-08T19:24:35.344 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-11-08T19:24:35.347 INFO:tasks.ceph.osd.1:Started 2023-11-08T19:24:35.347 INFO:tasks.ceph.osd.2:Restarting daemon 2023-11-08T19:24:35.348 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-11-08T19:24:35.351 INFO:tasks.ceph.osd.2:Started 2023-11-08T19:24:35.351 INFO:tasks.ceph.osd.3:Restarting daemon 2023-11-08T19:24:35.352 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-11-08T19:24:35.355 INFO:tasks.ceph.osd.3:Started 2023-11-08T19:24:35.355 INFO:tasks.ceph.osd.4:Restarting daemon 2023-11-08T19:24:35.356 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-11-08T19:24:35.359 INFO:tasks.ceph.osd.4:Started 2023-11-08T19:24:35.359 INFO:tasks.ceph.osd.5:Restarting daemon 2023-11-08T19:24:35.360 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-11-08T19:24:35.362 INFO:tasks.ceph.osd.5:Started 2023-11-08T19:24:35.362 INFO:tasks.ceph.osd.6:Restarting daemon 2023-11-08T19:24:35.363 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-11-08T19:24:35.365 INFO:tasks.ceph.osd.6:Started 2023-11-08T19:24:35.366 INFO:tasks.ceph.osd.7:Restarting daemon 2023-11-08T19:24:35.366 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-11-08T19:24:35.368 INFO:tasks.ceph.osd.7:Started 2023-11-08T19:24:35.369 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:24:35.444 INFO:tasks.ceph.mgr.y.smithi143.stderr:2023-11-08T19:24:35.443+0000 7fd1e98fa200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-08T19:24:35.516 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:35.515+0000 7fc363cd7700 1 Processor -- start 2023-11-08T19:24:35.517 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:35.515+0000 7fc363cd7700 1 -- start start 2023-11-08T19:24:35.520 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:35.516+0000 7fc363cd7700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc35c06c030 0x7fc35c0cd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:35.520 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:35.516+0000 7fc363cd7700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc35c15de80 0x7fc35c0cbf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:35.520 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:35.516+0000 7fc363cd7700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc35c0cd480 con 0x7fc35c06c730 2023-11-08T19:24:35.520 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:35.516+0000 7fc363cd7700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc35c0cc870 con 0x7fc35c15de80 2023-11-08T19:24:35.521 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:35.516+0000 7fc363cd7700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc35c0ccb50 con 0x7fc35c06c030 2023-11-08T19:24:35.521 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:35.516+0000 7fc362274700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc35c15de80 0x7fc35c0cbf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:35.521 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:35.516+0000 7fc361a73700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fc35c06c730 0x7fc35c06cb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:44282/0 (socket says 172.21.15.143:44282) 2023-11-08T19:24:35.521 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:35.516+0000 7fc361a73700 1 -- 172.21.15.143:0/2271986763 learned_addr learned my addr 172.21.15.143:0/2271986763 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:24:35.522 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:35.516+0000 7fc361272700 1 --2- 172.21.15.143:0/2271986763 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc35c06c030 0x7fc35c0cd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:35.522 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:35.516+0000 7fc362274700 1 -- 172.21.15.143:0/2271986763 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc35c06c030 msgr2=0x7fc35c0cd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:24:35.522 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:35.516+0000 7fc362274700 1 --2- 172.21.15.143:0/2271986763 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc35c06c030 0x7fc35c0cd910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:35.522 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:35.516+0000 7fc362274700 1 -- 172.21.15.143:0/2271986763 >> v1:172.21.15.143:6789/0 conn(0x7fc35c06c730 legacy=0x7fc35c06cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:35.523 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:35.516+0000 7fc360a71700 1 -- 172.21.15.143:0/2271986763 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2888141430 0 0) 0x7fc35c0cd480 con 0x7fc35c06c730 2023-11-08T19:24:35.523 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:35.516+0000 7fc362274700 1 -- 172.21.15.143:0/2271986763 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc35c0cce30 con 0x7fc35c15de80 2023-11-08T19:24:35.523 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:35.517+0000 7fc362274700 1 --2- 172.21.15.143:0/2271986763 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc35c15de80 0x7fc35c0cbf00 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fc354009d90 tx=0x7fc35400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30fd1361aa13343b server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:35.523 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:35.517+0000 7fc360a71700 1 -- 172.21.15.143:0/2271986763 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc35400e070 con 0x7fc35c15de80 2023-11-08T19:24:35.523 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:35.517+0000 7fc360a71700 1 -- 172.21.15.143:0/2271986763 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc35400bb20 con 0x7fc35c15de80 2023-11-08T19:24:35.524 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:35.517+0000 7fc360a71700 1 -- 172.21.15.143:0/2271986763 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc354012430 con 0x7fc35c15de80 2023-11-08T19:24:35.524 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:35.517+0000 7fc363cd7700 1 -- 172.21.15.143:0/2271986763 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc35c15de80 msgr2=0x7fc35c0cbf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:35.524 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:35.517+0000 7fc363cd7700 1 --2- 172.21.15.143:0/2271986763 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc35c15de80 0x7fc35c0cbf00 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fc354009d90 tx=0x7fc35400b330 comp rx=0 tx=0).stop 2023-11-08T19:24:35.524 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:35.517+0000 7fc363cd7700 1 -- 172.21.15.143:0/2271986763 shutdown_connections 2023-11-08T19:24:35.524 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:35.517+0000 7fc363cd7700 1 --2- 172.21.15.143:0/2271986763 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc35c15de80 0x7fc35c0cbf00 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:35.525 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:35.517+0000 7fc363cd7700 1 --2- 172.21.15.143:0/2271986763 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc35c06c030 0x7fc35c0cd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:35.525 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:35.517+0000 7fc363cd7700 1 -- 172.21.15.143:0/2271986763 >> 172.21.15.143:0/2271986763 conn(0x7fc35c08b000 msgr2=0x7fc35c09ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:24:35.525 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:35.517+0000 7fc363cd7700 1 -- 172.21.15.143:0/2271986763 shutdown_connections 2023-11-08T19:24:35.525 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:35.517+0000 7fc363cd7700 1 -- 172.21.15.143:0/2271986763 wait complete. 2023-11-08T19:24:35.525 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:35.519+0000 7fc363cd7700 1 Processor -- start 2023-11-08T19:24:35.525 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:35.519+0000 7fc363cd7700 1 -- start start 2023-11-08T19:24:35.526 INFO:tasks.ceph.mgr.y.smithi143.stderr:2023-11-08T19:24:35.519+0000 7fd1e98fa200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-08T19:24:35.529 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:35.519+0000 7fc363cd7700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc35c06c030 0x7fc35c14f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:35.529 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:35.519+0000 7fc363cd7700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc35c06c730 0x7fc35c0b6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:35.529 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:35.519+0000 7fc363cd7700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc35c15de80 0x7fc35c1f6b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:35.529 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:35.519+0000 7fc363cd7700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc35c15f100 con 0x7fc35c06c030 2023-11-08T19:24:35.529 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:35.519+0000 7fc363cd7700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc35c0cb3b0 con 0x7fc35c15de80 2023-11-08T19:24:35.530 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:35.519+0000 7fc363cd7700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc35c0cab10 con 0x7fc35c06c730 2023-11-08T19:24:35.530 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:35.519+0000 7fc361a73700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc35c06c030 0x7fc35c14f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:35.530 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:35.519+0000 7fc361a73700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc35c06c030 0x7fc35c14f0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48134/0 (socket says 172.21.15.143:48134) 2023-11-08T19:24:35.530 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:35.519+0000 7fc361a73700 1 -- 172.21.15.143:0/1648853091 learned_addr learned my addr 172.21.15.143:0/1648853091 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:24:35.530 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:35.519+0000 7fc361272700 1 --2- 172.21.15.143:0/1648853091 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc35c06c730 0x7fc35c0b6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:35.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:35.520+0000 7fc362274700 1 --2- 172.21.15.143:0/1648853091 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc35c15de80 0x7fc35c1f6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:35.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:35.520+0000 7fc361272700 1 -- 172.21.15.143:0/1648853091 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc35c15de80 msgr2=0x7fc35c1f6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:35.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:35.520+0000 7fc361272700 1 --2- 172.21.15.143:0/1648853091 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc35c15de80 0x7fc35c1f6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:35.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:35.520+0000 7fc361272700 1 -- 172.21.15.143:0/1648853091 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc35c06c030 msgr2=0x7fc35c14f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:35.532 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:35.520+0000 7fc361272700 1 --2- 172.21.15.143:0/1648853091 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc35c06c030 0x7fc35c14f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:35.532 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:35.520+0000 7fc361272700 1 -- 172.21.15.143:0/1648853091 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc354009a70 con 0x7fc35c06c730 2023-11-08T19:24:35.532 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:35.520+0000 7fc361a73700 1 --2- 172.21.15.143:0/1648853091 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc35c06c030 0x7fc35c14f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T19:24:35.532 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:35.520+0000 7fc361272700 1 --2- 172.21.15.143:0/1648853091 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc35c06c730 0x7fc35c0b6ac0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fc34c00e8d0 tx=0x7fc34c00ebe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:35.532 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:35.520+0000 7fc3527fc700 1 -- 172.21.15.143:0/1648853091 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc34c013680 con 0x7fc35c06c730 2023-11-08T19:24:35.533 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:35.520+0000 7fc3527fc700 1 -- 172.21.15.143:0/1648853091 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc34c0137e0 con 0x7fc35c06c730 2023-11-08T19:24:35.533 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:35.520+0000 7fc3527fc700 1 -- 172.21.15.143:0/1648853091 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc34c013b50 con 0x7fc35c06c730 2023-11-08T19:24:35.533 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:35.520+0000 7fc363cd7700 1 -- 172.21.15.143:0/1648853091 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc35c0cadf0 con 0x7fc35c06c730 2023-11-08T19:24:35.533 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:35.520+0000 7fc363cd7700 1 -- 172.21.15.143:0/1648853091 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc35c1f7850 con 0x7fc35c06c730 2023-11-08T19:24:35.533 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:35.520+0000 7fc3527fc700 1 -- 172.21.15.143:0/1648853091 <== mon.2 v2:172.21.15.174:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fc34c005a20 con 0x7fc35c06c730 2023-11-08T19:24:35.534 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:35.520+0000 7fc3527fc700 1 -- 172.21.15.143:0/1648853091 <== mon.2 v2:172.21.15.174:3301/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7fc34c006700 con 0x7fc35c06c730 2023-11-08T19:24:35.534 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:35.521+0000 7fc363cd7700 1 -- 172.21.15.143:0/1648853091 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc35c15f270 con 0x7fc35c06c730 2023-11-08T19:24:35.534 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:35.523+0000 7fc3527fc700 1 -- 172.21.15.143:0/1648853091 <== mon.2 v2:172.21.15.174:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7fc34c021060 con 0x7fc35c06c730 2023-11-08T19:24:35.594 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:35.593+0000 7fc363cd7700 1 -- 172.21.15.143:0/1648853091 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc35c04eee0 con 0x7fc35c06c730 2023-11-08T19:24:35.595 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:35.593+0000 7fc3527fc700 1 -- 172.21.15.143:0/1648853091 <== mon.2 v2:172.21.15.174:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7fc34c017070 con 0x7fc35c06c730 2023-11-08T19:24:35.595 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:24:35.595 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":10,"fsid":"968483af-77fc-4068-ad7c-5c2a1a4613d7","created":"2023-11-08T19:24:32.826484+0000","modified":"2023-11-08T19:24:35.314525+0000","last_up_change":"0.000000","last_in_change":"2023-11-08T19:24:35.314525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"bae98ec1-d94e-403d-a220-a522a53ea6f2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"5cb6538a-1ccb-4322-8633-09f66af0b7e4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"cdbf522c-60ef-4d60-9653-8dcb529c101a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"4e2783cf-ebb9-4dd8-8245-bc1d17ce4325","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"691256fa-9b77-49b9-a346-7ae1adf0ca80","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"756fc352-48c9-46ce-b52e-0541fe6f8525","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"fab2ac2b-ed97-4cde-9825-b480eabf8b8d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"3de9b797-2ab6-4d41-a863-7e7d638fd703","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:24:35.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:35.594+0000 7fc363cd7700 1 -- 172.21.15.143:0/1648853091 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc35c06c730 msgr2=0x7fc35c0b6ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:35.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:35.594+0000 7fc363cd7700 1 --2- 172.21.15.143:0/1648853091 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc35c06c730 0x7fc35c0b6ac0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fc34c00e8d0 tx=0x7fc34c00ebe0 comp rx=0 tx=0).stop 2023-11-08T19:24:35.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:35.594+0000 7fc363cd7700 1 -- 172.21.15.143:0/1648853091 shutdown_connections 2023-11-08T19:24:35.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:35.594+0000 7fc363cd7700 1 --2- 172.21.15.143:0/1648853091 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc35c15de80 0x7fc35c1f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:35.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:35.594+0000 7fc363cd7700 1 --2- 172.21.15.143:0/1648853091 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc35c06c030 0x7fc35c14f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:35.598 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:35.594+0000 7fc363cd7700 1 --2- 172.21.15.143:0/1648853091 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc35c06c730 0x7fc35c0b6ac0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:35.598 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:35.594+0000 7fc363cd7700 1 -- 172.21.15.143:0/1648853091 >> 172.21.15.143:0/1648853091 conn(0x7fc35c08b000 msgr2=0x7fc35c0d18e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:24:35.598 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:35.594+0000 7fc363cd7700 1 -- 172.21.15.143:0/1648853091 shutdown_connections 2023-11-08T19:24:35.598 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:35.594+0000 7fc363cd7700 1 -- 172.21.15.143:0/1648853091 wait complete. 2023-11-08T19:24:35.608 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-11-08T19:24:35.608 INFO:tasks.ceph:Waiting for OSDs to come up 2023-11-08T19:24:35.964 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-11-08T19:24:35.962+0000 7fbd0dc8f640 -1 Falling back to public interface 2023-11-08T19:24:35.965 INFO:tasks.ceph.osd.4.smithi174.stderr:2023-11-08T19:24:35.963+0000 7f150c4a9640 -1 Falling back to public interface 2023-11-08T19:24:35.970 INFO:tasks.ceph.osd.5.smithi174.stderr:2023-11-08T19:24:35.969+0000 7f13b486e640 -1 Falling back to public interface 2023-11-08T19:24:35.975 INFO:tasks.ceph.osd.7.smithi174.stderr:2023-11-08T19:24:35.973+0000 7fdefa14c640 -1 Falling back to public interface 2023-11-08T19:24:35.976 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-11-08T19:24:35.975+0000 7fd13563d640 -1 Falling back to public interface 2023-11-08T19:24:35.986 INFO:tasks.ceph.osd.6.smithi174.stderr:2023-11-08T19:24:35.984+0000 7f3335113640 -1 Falling back to public interface 2023-11-08T19:24:35.988 INFO:tasks.ceph.osd.3.smithi143.stderr:2023-11-08T19:24:35.987+0000 7f4da9013640 -1 Falling back to public interface 2023-11-08T19:24:35.990 INFO:tasks.ceph.osd.2.smithi143.stderr:2023-11-08T19:24:35.988+0000 7ff089d26640 -1 Falling back to public interface 2023-11-08T19:24:35.995 INFO:tasks.ceph.mgr.x.smithi174.stderr:2023-11-08T19:24:35.994+0000 7fd2ab6d0200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-08T19:24:36.072 INFO:tasks.ceph.mgr.x.smithi174.stderr:2023-11-08T19:24:36.071+0000 7fd2ab6d0200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-11-08T19:24:36.212 INFO:tasks.ceph.mgr.x.smithi174.stderr:2023-11-08T19:24:36.210+0000 7fd2ab6d0200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-08T19:24:36.220 INFO:tasks.ceph.mgr.y.smithi143.stderr:2023-11-08T19:24:36.219+0000 7fd1e98fa200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-08T19:24:36.300 INFO:tasks.ceph.mgr.y.smithi143.stderr:2023-11-08T19:24:36.299+0000 7fd1e98fa200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-11-08T19:24:36.412 DEBUG:teuthology.orchestra.run.smithi143:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-11-08T19:24:36.441 INFO:tasks.ceph.mgr.y.smithi143.stderr:2023-11-08T19:24:36.440+0000 7fd1e98fa200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-08T19:24:36.446 INFO:tasks.ceph.mgr.x.smithi174.stderr:2023-11-08T19:24:36.445+0000 7fd2ab6d0200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-11-08T19:24:36.525 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:36.523+0000 7f112df32700 1 Processor -- start 2023-11-08T19:24:36.525 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:36.523+0000 7f112df32700 1 -- start start 2023-11-08T19:24:36.527 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:36.524+0000 7f112df32700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f112806e120 0x7f11280caf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:36.528 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:36.524+0000 7f112df32700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f112806e500 0x7f112815dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:36.528 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:36.524+0000 7f112df32700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f112815e550 con 0x7f112815e100 2023-11-08T19:24:36.528 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:36.524+0000 7f112df32700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f112806abd0 con 0x7f112806e500 2023-11-08T19:24:36.528 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:36.524+0000 7f112df32700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11280cc340 con 0x7f112806e120 2023-11-08T19:24:36.529 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:36.524+0000 7f1126ffd700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f112806e500 0x7f112815dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:36.529 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:36.524+0000 7f1127fff700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f112815e100 0x7f112815a5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:44298/0 (socket says 172.21.15.143:44298) 2023-11-08T19:24:36.529 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:36.524+0000 7f1126ffd700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f112806e500 0x7f112815dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48142/0 (socket says 172.21.15.143:48142) 2023-11-08T19:24:36.529 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:36.524+0000 7f1127fff700 1 -- 172.21.15.143:0/1461637242 learned_addr learned my addr 172.21.15.143:0/1461637242 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:24:36.529 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:36.524+0000 7f11277fe700 1 --2- 172.21.15.143:0/1461637242 >> v2:172.21.15.174:3300/0 conn(0x7f112806e120 0x7f11280caf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:36.530 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:36.525+0000 7f1126ffd700 1 -- 172.21.15.143:0/1461637242 >> v2:172.21.15.174:3300/0 conn(0x7f112806e120 msgr2=0x7f11280caf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:24:36.530 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:36.525+0000 7f1126ffd700 1 --2- 172.21.15.143:0/1461637242 >> v2:172.21.15.174:3300/0 conn(0x7f112806e120 0x7f11280caf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:36.530 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:36.525+0000 7f11267fc700 1 -- 172.21.15.143:0/1461637242 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4227122462 0 0) 0x7f1120001490 con 0x7f112815e100 2023-11-08T19:24:36.530 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:36.525+0000 7f1126ffd700 1 -- 172.21.15.143:0/1461637242 >> v1:172.21.15.143:6789/0 conn(0x7f112815e100 legacy=0x7f112815a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:36.530 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:36.525+0000 7f1126ffd700 1 -- 172.21.15.143:0/1461637242 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11280cb7c0 con 0x7f112806e500 2023-11-08T19:24:36.531 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:36.525+0000 7f1126ffd700 1 --2- 172.21.15.143:0/1461637242 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f112806e500 0x7f112815dbc0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f111c009840 tx=0x7f111c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a288756acf8ae93 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:36.531 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:36.525+0000 7f11267fc700 1 -- 172.21.15.143:0/1461637242 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f111c011830 con 0x7f112806e500 2023-11-08T19:24:36.531 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:36.525+0000 7f11267fc700 1 -- 172.21.15.143:0/1461637242 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f111c011990 con 0x7f112806e500 2023-11-08T19:24:36.531 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:36.525+0000 7f11267fc700 1 -- 172.21.15.143:0/1461637242 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f111c011d20 con 0x7f112806e500 2023-11-08T19:24:36.531 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:36.526+0000 7f112df32700 1 -- 172.21.15.143:0/1461637242 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f112806e500 msgr2=0x7f112815dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:36.532 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:36.526+0000 7f112df32700 1 --2- 172.21.15.143:0/1461637242 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f112806e500 0x7f112815dbc0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f111c009840 tx=0x7f111c009b50 comp rx=0 tx=0).stop 2023-11-08T19:24:36.532 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:36.526+0000 7f112df32700 1 -- 172.21.15.143:0/1461637242 shutdown_connections 2023-11-08T19:24:36.532 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:36.526+0000 7f112df32700 1 --2- 172.21.15.143:0/1461637242 >> v2:172.21.15.174:3300/0 conn(0x7f112806e120 0x7f11280caf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:36.532 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:36.526+0000 7f112df32700 1 --2- 172.21.15.143:0/1461637242 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f112806e500 0x7f112815dbc0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:36.532 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:36.526+0000 7f112df32700 1 -- 172.21.15.143:0/1461637242 >> 172.21.15.143:0/1461637242 conn(0x7f11280b19e0 msgr2=0x7f11280b1df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:24:36.533 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:36.526+0000 7f112df32700 1 -- 172.21.15.143:0/1461637242 shutdown_connections 2023-11-08T19:24:36.533 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:36.526+0000 7f112df32700 1 -- 172.21.15.143:0/1461637242 wait complete. 2023-11-08T19:24:36.535 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:36.527+0000 7f112df32700 1 Processor -- start 2023-11-08T19:24:36.536 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:36.527+0000 7f112df32700 1 -- start start 2023-11-08T19:24:36.536 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:36.528+0000 7f112df32700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f112806e120 0x7f1128163e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:36.536 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:36.528+0000 7f112df32700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f112806e500 0x7f1128158b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:36.536 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:36.528+0000 7f112df32700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f112815e100 0x7f1128165f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:36.536 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:36.528+0000 7f112df32700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1128164d60 con 0x7f112806e120 2023-11-08T19:24:36.537 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:36.528+0000 7f112df32700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1128165040 con 0x7f112806e500 2023-11-08T19:24:36.537 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:36.528+0000 7f112df32700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1128165320 con 0x7f112815e100 2023-11-08T19:24:36.537 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:36.528+0000 7f11277fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f112806e120 0x7f1128163e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:36.537 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:36.528+0000 7f11277fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f112806e120 0x7f1128163e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48146/0 (socket says 172.21.15.143:48146) 2023-11-08T19:24:36.537 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:36.528+0000 7f11277fe700 1 -- 172.21.15.143:0/1803005629 learned_addr learned my addr 172.21.15.143:0/1803005629 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:24:36.538 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:36.528+0000 7f1126ffd700 1 --2- 172.21.15.143:0/1803005629 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f112806e500 0x7f1128158b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:36.538 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:36.528+0000 7f1127fff700 1 --2- 172.21.15.143:0/1803005629 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f112815e100 0x7f1128165f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:36.538 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:36.528+0000 7f11277fe700 1 -- 172.21.15.143:0/1803005629 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f112815e100 msgr2=0x7f1128165f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:36.538 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:36.528+0000 7f11277fe700 1 --2- 172.21.15.143:0/1803005629 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f112815e100 0x7f1128165f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:36.539 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:36.528+0000 7f11277fe700 1 -- 172.21.15.143:0/1803005629 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f112806e500 msgr2=0x7f1128158b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:36.539 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:36.528+0000 7f11277fe700 1 --2- 172.21.15.143:0/1803005629 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f112806e500 0x7f1128158b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:36.539 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:36.528+0000 7f11277fe700 1 -- 172.21.15.143:0/1803005629 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f111c0094a0 con 0x7f112806e120 2023-11-08T19:24:36.539 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:36.528+0000 7f11277fe700 1 --2- 172.21.15.143:0/1803005629 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f112806e120 0x7f1128163e00 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f111800e870 tx=0x7f111800eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:36.539 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:36.529+0000 7f110ffff700 1 -- 172.21.15.143:0/1803005629 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1118009590 con 0x7f112806e120 2023-11-08T19:24:36.540 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:36.529+0000 7f110ffff700 1 -- 172.21.15.143:0/1803005629 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1118013680 con 0x7f112806e120 2023-11-08T19:24:36.540 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:36.529+0000 7f112df32700 1 -- 172.21.15.143:0/1803005629 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1128165600 con 0x7f112806e120 2023-11-08T19:24:36.540 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:36.529+0000 7f110ffff700 1 -- 172.21.15.143:0/1803005629 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11180139f0 con 0x7f112806e120 2023-11-08T19:24:36.540 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:36.529+0000 7f112df32700 1 -- 172.21.15.143:0/1803005629 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1128164340 con 0x7f112806e120 2023-11-08T19:24:36.540 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:36.529+0000 7f110ffff700 1 -- 172.21.15.143:0/1803005629 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f1118005a20 con 0x7f112806e120 2023-11-08T19:24:36.541 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:36.529+0000 7f110ffff700 1 -- 172.21.15.143:0/1803005629 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f11180066b0 con 0x7f112806e120 2023-11-08T19:24:36.541 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:36.529+0000 7f112df32700 1 -- 172.21.15.143:0/1803005629 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1128165b20 con 0x7f112806e120 2023-11-08T19:24:36.541 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:36.532+0000 7f110ffff700 1 -- 172.21.15.143:0/1803005629 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f1128165b20 con 0x7f112806e120 2023-11-08T19:24:36.603 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:36.602+0000 7f112df32700 1 -- 172.21.15.143:0/1803005629 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1128151120 con 0x7f112806e120 2023-11-08T19:24:36.604 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:36.602+0000 7f110ffff700 1 -- 172.21.15.143:0/1803005629 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f1118021060 con 0x7f112806e120 2023-11-08T19:24:36.604 INFO:teuthology.misc.health.smithi143.stdout: 2023-11-08T19:24:36.605 INFO:teuthology.misc.health.smithi143.stdout:{"epoch":10,"fsid":"968483af-77fc-4068-ad7c-5c2a1a4613d7","created":"2023-11-08T19:24:32.826484+0000","modified":"2023-11-08T19:24:35.314525+0000","last_up_change":"0.000000","last_in_change":"2023-11-08T19:24:35.314525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"bae98ec1-d94e-403d-a220-a522a53ea6f2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"5cb6538a-1ccb-4322-8633-09f66af0b7e4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"cdbf522c-60ef-4d60-9653-8dcb529c101a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"4e2783cf-ebb9-4dd8-8245-bc1d17ce4325","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"691256fa-9b77-49b9-a346-7ae1adf0ca80","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"756fc352-48c9-46ce-b52e-0541fe6f8525","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"fab2ac2b-ed97-4cde-9825-b480eabf8b8d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"3de9b797-2ab6-4d41-a863-7e7d638fd703","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:24:36.605 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:36.603+0000 7f112df32700 1 -- 172.21.15.143:0/1803005629 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f112806e120 msgr2=0x7f1128163e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:36.606 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:36.603+0000 7f112df32700 1 --2- 172.21.15.143:0/1803005629 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f112806e120 0x7f1128163e00 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f111800e870 tx=0x7f111800eb80 comp rx=0 tx=0).stop 2023-11-08T19:24:36.606 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:36.603+0000 7f112df32700 1 -- 172.21.15.143:0/1803005629 shutdown_connections 2023-11-08T19:24:36.606 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:36.603+0000 7f112df32700 1 --2- 172.21.15.143:0/1803005629 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f112815e100 0x7f1128165f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:36.606 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:36.603+0000 7f112df32700 1 --2- 172.21.15.143:0/1803005629 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f112806e120 0x7f1128163e00 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:36.606 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:36.603+0000 7f112df32700 1 --2- 172.21.15.143:0/1803005629 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f112806e500 0x7f1128158b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:36.607 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:36.603+0000 7f112df32700 1 -- 172.21.15.143:0/1803005629 >> 172.21.15.143:0/1803005629 conn(0x7f11280b19e0 msgr2=0x7f112806b490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:24:36.607 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:36.603+0000 7f112df32700 1 -- 172.21.15.143:0/1803005629 shutdown_connections 2023-11-08T19:24:36.607 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:36.603+0000 7f112df32700 1 -- 172.21.15.143:0/1803005629 wait complete. 2023-11-08T19:24:36.615 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-11-08T19:24:36.661 INFO:tasks.ceph.mgr.x.smithi174.stderr:2023-11-08T19:24:36.659+0000 7fd2ab6d0200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-08T19:24:36.685 INFO:tasks.ceph.mgr.y.smithi143.stderr:2023-11-08T19:24:36.683+0000 7fd1e98fa200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-11-08T19:24:36.792 INFO:tasks.ceph.mgr.x.smithi174.stderr:2023-11-08T19:24:36.791+0000 7fd2ab6d0200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-08T19:24:36.901 INFO:tasks.ceph.mgr.y.smithi143.stderr:2023-11-08T19:24:36.900+0000 7fd1e98fa200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-08T19:24:37.011 INFO:tasks.ceph.mgr.x.smithi174.stderr:2023-11-08T19:24:37.009+0000 7fd2ab6d0200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-08T19:24:37.034 INFO:tasks.ceph.mgr.y.smithi143.stderr:2023-11-08T19:24:37.033+0000 7fd1e98fa200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-08T19:24:37.165 INFO:tasks.ceph.mgr.x.smithi174.stderr:2023-11-08T19:24:37.163+0000 7fd2ab6d0200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-08T19:24:37.253 INFO:tasks.ceph.mgr.y.smithi143.stderr:2023-11-08T19:24:37.252+0000 7fd1e98fa200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-08T19:24:37.278 INFO:tasks.ceph.mgr.x.smithi174.stderr:2023-11-08T19:24:37.276+0000 7fd2ab6d0200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-08T19:24:37.409 INFO:tasks.ceph.mgr.y.smithi143.stderr:2023-11-08T19:24:37.408+0000 7fd1e98fa200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-08T19:24:37.523 INFO:tasks.ceph.mgr.y.smithi143.stderr:2023-11-08T19:24:37.522+0000 7fd1e98fa200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-08T19:24:37.602 INFO:tasks.ceph.mgr.x.smithi174.stderr:2023-11-08T19:24:37.600+0000 7fd2ab6d0200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-08T19:24:37.701 INFO:tasks.ceph.mgr.x.smithi174.stderr:2023-11-08T19:24:37.700+0000 7fd2ab6d0200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-08T19:24:37.848 INFO:tasks.ceph.mgr.y.smithi143.stderr:2023-11-08T19:24:37.847+0000 7fd1e98fa200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-08T19:24:37.887 INFO:tasks.ceph.mgr.x.smithi174.stderr:2023-11-08T19:24:37.886+0000 7fd2ab6d0200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-08T19:24:37.949 INFO:tasks.ceph.mgr.y.smithi143.stderr:2023-11-08T19:24:37.948+0000 7fd1e98fa200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-08T19:24:38.139 INFO:tasks.ceph.mgr.y.smithi143.stderr:2023-11-08T19:24:38.138+0000 7fd1e98fa200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-08T19:24:38.488 INFO:tasks.ceph.mgr.x.smithi174.stderr:2023-11-08T19:24:38.486+0000 7fd2ab6d0200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-08T19:24:38.807 INFO:tasks.ceph.mgr.y.smithi143.stderr:2023-11-08T19:24:38.805+0000 7fd1e98fa200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-08T19:24:39.482 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-11-08T19:24:39.480+0000 7fbd0dc8f640 -1 osd.1 0 log_to_monitors true 2023-11-08T19:24:39.492 INFO:tasks.ceph.osd.2.smithi143.stderr:2023-11-08T19:24:39.491+0000 7ff089d26640 -1 osd.2 0 log_to_monitors true 2023-11-08T19:24:39.500 INFO:tasks.ceph.osd.3.smithi143.stderr:2023-11-08T19:24:39.498+0000 7f4da9013640 -1 osd.3 0 log_to_monitors true 2023-11-08T19:24:39.505 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-11-08T19:24:39.504+0000 7fd13563d640 -1 osd.0 0 log_to_monitors true 2023-11-08T19:24:39.516 INFO:tasks.ceph.mgr.x.smithi174.stderr:2023-11-08T19:24:39.515+0000 7fd263637700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-11-08T19:24:39.518 INFO:tasks.ceph.mgr.x.smithi174.stderr:2023-11-08T19:24:39.516+0000 7fd263637700 -1 mgr.server handle_report got status from non-daemon mon.b 2023-11-08T19:24:39.530 INFO:tasks.ceph.osd.6.smithi174.stderr:2023-11-08T19:24:39.528+0000 7f3335113640 -1 osd.6 0 log_to_monitors true 2023-11-08T19:24:39.542 INFO:tasks.ceph.osd.7.smithi174.stderr:2023-11-08T19:24:39.540+0000 7fdefa14c640 -1 osd.7 0 log_to_monitors true 2023-11-08T19:24:39.546 INFO:tasks.ceph.osd.5.smithi174.stderr:2023-11-08T19:24:39.545+0000 7f13b486e640 -1 osd.5 0 log_to_monitors true 2023-11-08T19:24:39.547 INFO:tasks.ceph.osd.4.smithi174.stderr:2023-11-08T19:24:39.545+0000 7f150c4a9640 -1 osd.4 0 log_to_monitors true 2023-11-08T19:24:42.547 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-11-08T19:24:42.546+0000 7fd1291f4700 -1 osd.0 0 waiting for initial osdmap 2023-11-08T19:24:42.566 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-11-08T19:24:42.565+0000 7fd1215d4700 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-08T19:24:42.569 INFO:tasks.ceph.osd.7.smithi174.stderr:2023-11-08T19:24:42.569+0000 7fdeedd03700 -1 osd.7 0 waiting for initial osdmap 2023-11-08T19:24:42.585 INFO:tasks.ceph.osd.5.smithi174.stderr:2023-11-08T19:24:42.585+0000 7f13a8425700 -1 osd.5 0 waiting for initial osdmap 2023-11-08T19:24:42.589 INFO:tasks.ceph.osd.7.smithi174.stderr:2023-11-08T19:24:42.589+0000 7fdee60e3700 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-08T19:24:42.590 INFO:tasks.ceph.osd.2.smithi143.stderr:2023-11-08T19:24:42.589+0000 7ff07d8dd700 -1 osd.2 0 waiting for initial osdmap 2023-11-08T19:24:42.600 INFO:tasks.ceph.osd.6.smithi174.stderr:2023-11-08T19:24:42.600+0000 7f3328cca700 -1 osd.6 0 waiting for initial osdmap 2023-11-08T19:24:42.602 INFO:tasks.ceph.osd.2.smithi143.stderr:2023-11-08T19:24:42.601+0000 7ff075cbd700 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-08T19:24:42.603 INFO:tasks.ceph.osd.5.smithi174.stderr:2023-11-08T19:24:42.603+0000 7f13a0805700 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-08T19:24:42.612 INFO:tasks.ceph.osd.6.smithi174.stderr:2023-11-08T19:24:42.612+0000 7f33210aa700 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-08T19:24:42.617 INFO:tasks.ceph.osd.4.smithi174.stderr:2023-11-08T19:24:42.616+0000 7f1500060700 -1 osd.4 0 waiting for initial osdmap 2023-11-08T19:24:42.633 INFO:tasks.ceph.osd.3.smithi143.stderr:2023-11-08T19:24:42.632+0000 7f4d9cbca700 -1 osd.3 0 waiting for initial osdmap 2023-11-08T19:24:42.633 INFO:tasks.ceph.osd.4.smithi174.stderr:2023-11-08T19:24:42.633+0000 7f14f8440700 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-08T19:24:42.652 INFO:tasks.ceph.osd.3.smithi143.stderr:2023-11-08T19:24:42.651+0000 7f4d94faa700 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-08T19:24:42.725 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-11-08T19:24:42.724+0000 7fbd01846700 -1 osd.1 0 waiting for initial osdmap 2023-11-08T19:24:42.740 INFO:tasks.ceph.osd.1.smithi143.stderr:2023-11-08T19:24:42.739+0000 7fbcf9c26700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-08T19:24:43.423 DEBUG:teuthology.orchestra.run.smithi143:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-11-08T19:24:43.539 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:43.537+0000 7fcd9de3e700 1 Processor -- start 2023-11-08T19:24:43.539 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:43.537+0000 7fcd9de3e700 1 -- start start 2023-11-08T19:24:43.542 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:43.538+0000 7fcd9de3e700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcd980af9b0 0x7fcd980cda00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:43.542 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:43.538+0000 7fcd9de3e700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fcd980afd90 0x7fcd9806abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:43.542 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:43.538+0000 7fcd9de3e700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fcd9806b110 0x7fcd9815d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:43.542 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:43.538+0000 7fcd9de3e700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd98162800 con 0x7fcd980afd90 2023-11-08T19:24:43.543 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:43.538+0000 7fcd9de3e700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd9806b5c0 con 0x7fcd9806b110 2023-11-08T19:24:43.543 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:43.538+0000 7fcd9de3e700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd980cd250 con 0x7fcd980af9b0 2023-11-08T19:24:43.543 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:43.538+0000 7fcd96ffd700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fcd980afd90 0x7fcd9806abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:43.543 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:43.538+0000 7fcd96ffd700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fcd980afd90 0x7fcd9806abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48204/0 (socket says 172.21.15.143:48204) 2023-11-08T19:24:43.543 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:43.538+0000 7fcd96ffd700 1 -- 172.21.15.143:0/4053587920 learned_addr learned my addr 172.21.15.143:0/4053587920 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:24:43.544 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:43.538+0000 7fcd977fe700 1 --2- 172.21.15.143:0/4053587920 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcd980af9b0 0x7fcd980cda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:43.544 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:43.538+0000 7fcd97fff700 1 --2- 172.21.15.143:0/4053587920 >> v2:172.21.15.174:3300/0 conn(0x7fcd9806b110 0x7fcd9815d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:43.544 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:43.538+0000 7fcd96ffd700 1 -- 172.21.15.143:0/4053587920 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcd980af9b0 msgr2=0x7fcd980cda00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:24:43.544 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:43.538+0000 7fcd96ffd700 1 --2- 172.21.15.143:0/4053587920 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcd980af9b0 0x7fcd980cda00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:43.544 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:43.538+0000 7fcd96ffd700 1 -- 172.21.15.143:0/4053587920 >> v2:172.21.15.174:3300/0 conn(0x7fcd9806b110 msgr2=0x7fcd9815d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:24:43.545 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:43.538+0000 7fcd96ffd700 1 --2- 172.21.15.143:0/4053587920 >> v2:172.21.15.174:3300/0 conn(0x7fcd9806b110 0x7fcd9815d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:43.545 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:43.538+0000 7fcd96ffd700 1 -- 172.21.15.143:0/4053587920 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd980cfc70 con 0x7fcd980afd90 2023-11-08T19:24:43.545 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:43.539+0000 7fcd96ffd700 1 --2- 172.21.15.143:0/4053587920 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcd980afd90 0x7fcd9806abd0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fcd980709a0 tx=0x7fcd88009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c23cf577576217f server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:43.545 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:43.539+0000 7fcd967fc700 1 -- 172.21.15.143:0/4053587920 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd88011830 con 0x7fcd980afd90 2023-11-08T19:24:43.545 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:43.539+0000 7fcd967fc700 1 -- 172.21.15.143:0/4053587920 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd88011990 con 0x7fcd980afd90 2023-11-08T19:24:43.546 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:43.539+0000 7fcd967fc700 1 -- 172.21.15.143:0/4053587920 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd88011d20 con 0x7fcd980afd90 2023-11-08T19:24:43.546 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:43.539+0000 7fcd9de3e700 1 -- 172.21.15.143:0/4053587920 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcd980afd90 msgr2=0x7fcd9806abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:43.546 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:43.539+0000 7fcd9de3e700 1 --2- 172.21.15.143:0/4053587920 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcd980afd90 0x7fcd9806abd0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fcd980709a0 tx=0x7fcd88009c00 comp rx=0 tx=0).stop 2023-11-08T19:24:43.546 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:43.539+0000 7fcd9de3e700 1 -- 172.21.15.143:0/4053587920 shutdown_connections 2023-11-08T19:24:43.547 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:43.539+0000 7fcd9de3e700 1 --2- 172.21.15.143:0/4053587920 >> v2:172.21.15.174:3300/0 conn(0x7fcd9806b110 0x7fcd9815d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:43.547 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:43.539+0000 7fcd9de3e700 1 --2- 172.21.15.143:0/4053587920 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcd980af9b0 0x7fcd980cda00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:43.547 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:43.539+0000 7fcd9de3e700 1 --2- 172.21.15.143:0/4053587920 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcd980afd90 0x7fcd9806abd0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:43.547 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:43.539+0000 7fcd9de3e700 1 -- 172.21.15.143:0/4053587920 >> 172.21.15.143:0/4053587920 conn(0x7fcd980725e0 msgr2=0x7fcd98099a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:24:43.547 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:43.539+0000 7fcd9de3e700 1 -- 172.21.15.143:0/4053587920 shutdown_connections 2023-11-08T19:24:43.547 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:43.539+0000 7fcd9de3e700 1 -- 172.21.15.143:0/4053587920 wait complete. 2023-11-08T19:24:43.548 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:43.541+0000 7fcd9de3e700 1 Processor -- start 2023-11-08T19:24:43.548 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:43.541+0000 7fcd9de3e700 1 -- start start 2023-11-08T19:24:43.550 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:43.541+0000 7fcd9de3e700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcd9806b110 0x7fcd980c6b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:43.551 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:43.541+0000 7fcd9de3e700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcd980af9b0 0x7fcd9815ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:43.551 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:43.541+0000 7fcd9de3e700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcd980afd90 0x7fcd981f6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:43.551 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:43.541+0000 7fcd9de3e700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd980c7080 con 0x7fcd980afd90 2023-11-08T19:24:43.551 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:43.541+0000 7fcd9de3e700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd980c5ba0 con 0x7fcd980af9b0 2023-11-08T19:24:43.551 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:43.541+0000 7fcd9de3e700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd9815f270 con 0x7fcd9806b110 2023-11-08T19:24:43.552 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:43.542+0000 7fcd97fff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcd980afd90 0x7fcd981f6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:43.552 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:43.542+0000 7fcd97fff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcd980afd90 0x7fcd981f6cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48212/0 (socket says 172.21.15.143:48212) 2023-11-08T19:24:43.552 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:43.542+0000 7fcd97fff700 1 -- 172.21.15.143:0/1774431240 learned_addr learned my addr 172.21.15.143:0/1774431240 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:24:43.552 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:43.542+0000 7fcd96ffd700 1 --2- 172.21.15.143:0/1774431240 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcd980af9b0 0x7fcd9815ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:43.553 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:43.542+0000 7fcd97fff700 1 -- 172.21.15.143:0/1774431240 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcd9806b110 msgr2=0x7fcd980c6b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:43.553 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:43.542+0000 7fcd97fff700 1 --2- 172.21.15.143:0/1774431240 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcd9806b110 0x7fcd980c6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:43.553 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:43.542+0000 7fcd97fff700 1 -- 172.21.15.143:0/1774431240 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcd980af9b0 msgr2=0x7fcd9815ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:43.553 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:43.542+0000 7fcd97fff700 1 --2- 172.21.15.143:0/1774431240 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcd980af9b0 0x7fcd9815ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:43.553 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:43.542+0000 7fcd97fff700 1 -- 172.21.15.143:0/1774431240 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd88009550 con 0x7fcd980afd90 2023-11-08T19:24:43.554 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:43.542+0000 7fcd97fff700 1 --2- 172.21.15.143:0/1774431240 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcd980afd90 0x7fcd981f6cd0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fcd9000efc0 tx=0x7fcd9000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:43.554 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:43.542+0000 7fcd7ffff700 1 -- 172.21.15.143:0/1774431240 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd9000cd40 con 0x7fcd980afd90 2023-11-08T19:24:43.554 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:43.542+0000 7fcd7ffff700 1 -- 172.21.15.143:0/1774431240 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd9000f040 con 0x7fcd980afd90 2023-11-08T19:24:43.554 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:43.542+0000 7fcd7ffff700 1 -- 172.21.15.143:0/1774431240 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd900146c0 con 0x7fcd980afd90 2023-11-08T19:24:43.554 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:43.542+0000 7fcd9de3e700 1 -- 172.21.15.143:0/1774431240 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd980c5020 con 0x7fcd980afd90 2023-11-08T19:24:43.555 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:43.542+0000 7fcd9de3e700 1 -- 172.21.15.143:0/1774431240 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd980c55e0 con 0x7fcd980afd90 2023-11-08T19:24:43.555 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:43.543+0000 7fcd9de3e700 1 -- 172.21.15.143:0/1774431240 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd98075070 con 0x7fcd980afd90 2023-11-08T19:24:43.555 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:43.543+0000 7fcd7ffff700 1 -- 172.21.15.143:0/1774431240 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 4) v1 ==== 89175+0+0 (secure 0 0 0) 0x7fcd90014820 con 0x7fcd980afd90 2023-11-08T19:24:43.555 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:43.544+0000 7fcd7ffff700 1 --2- 172.21.15.143:0/1774431240 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fcd8406ae70 0x7fcd8406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:43.555 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:43.544+0000 7fcd7ffff700 1 -- 172.21.15.143:0/1774431240 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2231+0+0 (secure 0 0 0) 0x7fcd90089010 con 0x7fcd980afd90 2023-11-08T19:24:43.556 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:43.544+0000 7fcd977fe700 1 --2- 172.21.15.143:0/1774431240 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fcd8406ae70 0x7fcd8406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:43.556 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:43.544+0000 7fcd977fe700 1 --2- 172.21.15.143:0/1774431240 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fcd8406ae70 0x7fcd8406d330 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fcd980c7240 tx=0x7fcd8c009580 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:43.556 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:43.548+0000 7fcd7ffff700 1 -- 172.21.15.143:0/1774431240 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcd90054fc0 con 0x7fcd980afd90 2023-11-08T19:24:43.750 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:43.749+0000 7fcd9de3e700 1 -- 172.21.15.143:0/1774431240 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcd980c7810 con 0x7fcd980afd90 2023-11-08T19:24:43.751 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:43.750+0000 7fcd7ffff700 1 -- 172.21.15.143:0/1774431240 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) v1 ==== 74+0+10956 (secure 0 0 0) 0x7fcd90018090 con 0x7fcd980afd90 2023-11-08T19:24:43.751 INFO:teuthology.misc.health.smithi143.stdout: 2023-11-08T19:24:43.752 INFO:teuthology.misc.health.smithi143.stdout:{"epoch":13,"fsid":"968483af-77fc-4068-ad7c-5c2a1a4613d7","created":"2023-11-08T19:24:32.826484+0000","modified":"2023-11-08T19:24:43.573880+0000","last_up_change":"2023-11-08T19:24:43.573880+0000","last_in_change":"2023-11-08T19:24:35.314525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"bae98ec1-d94e-403d-a220-a522a53ea6f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6809","nonce":1487830566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6811","nonce":1487830566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6815","nonce":1487830566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6813","nonce":1487830566}]},"public_addr":"172.21.15.143:6809/1487830566","cluster_addr":"172.21.15.143:6811/1487830566","heartbeat_back_addr":"172.21.15.143:6815/1487830566","heartbeat_front_addr":"172.21.15.143:6813/1487830566","state":["exists","up"]},{"osd":1,"uuid":"5cb6538a-1ccb-4322-8633-09f66af0b7e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6801","nonce":787341397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6803","nonce":787341397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6807","nonce":787341397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6805","nonce":787341397}]},"public_addr":"172.21.15.143:6801/787341397","cluster_addr":"172.21.15.143:6803/787341397","heartbeat_back_addr":"172.21.15.143:6807/787341397","heartbeat_front_addr":"172.21.15.143:6805/787341397","state":["exists","up"]},{"osd":2,"uuid":"cdbf522c-60ef-4d60-9653-8dcb529c101a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6822","nonce":2301325597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6826","nonce":2301325597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6831","nonce":2301325597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6829","nonce":2301325597}]},"public_addr":"172.21.15.143:6822/2301325597","cluster_addr":"172.21.15.143:6826/2301325597","heartbeat_back_addr":"172.21.15.143:6831/2301325597","heartbeat_front_addr":"172.21.15.143:6829/2301325597","state":["exists","up"]},{"osd":3,"uuid":"4e2783cf-ebb9-4dd8-8245-bc1d17ce4325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6817","nonce":2021504428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6819","nonce":2021504428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6827","nonce":2021504428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6823","nonce":2021504428}]},"public_addr":"172.21.15.143:6817/2021504428","cluster_addr":"172.21.15.143:6819/2021504428","heartbeat_back_addr":"172.21.15.143:6827/2021504428","heartbeat_front_addr":"172.21.15.143:6823/2021504428","state":["exists","up"]},{"osd":4,"uuid":"691256fa-9b77-49b9-a346-7ae1adf0ca80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6801","nonce":1993380595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6803","nonce":1993380595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6807","nonce":1993380595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6805","nonce":1993380595}]},"public_addr":"172.21.15.174:6801/1993380595","cluster_addr":"172.21.15.174:6803/1993380595","heartbeat_back_addr":"172.21.15.174:6807/1993380595","heartbeat_front_addr":"172.21.15.174:6805/1993380595","state":["exists","up"]},{"osd":5,"uuid":"756fc352-48c9-46ce-b52e-0541fe6f8525","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6809","nonce":1777923303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6811","nonce":1777923303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6815","nonce":1777923303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6813","nonce":1777923303}]},"public_addr":"172.21.15.174:6809/1777923303","cluster_addr":"172.21.15.174:6811/1777923303","heartbeat_back_addr":"172.21.15.174:6815/1777923303","heartbeat_front_addr":"172.21.15.174:6813/1777923303","state":["exists","up"]},{"osd":6,"uuid":"fab2ac2b-ed97-4cde-9825-b480eabf8b8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6825","nonce":2755130316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6827","nonce":2755130316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6831","nonce":2755130316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6829","nonce":2755130316}]},"public_addr":"172.21.15.174:6825/2755130316","cluster_addr":"172.21.15.174:6827/2755130316","heartbeat_back_addr":"172.21.15.174:6831/2755130316","heartbeat_front_addr":"172.21.15.174:6829/2755130316","state":["exists","up"]},{"osd":7,"uuid":"3de9b797-2ab6-4d41-a863-7e7d638fd703","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6817","nonce":2847141411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6819","nonce":2847141411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6823","nonce":2847141411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6821","nonce":2847141411}]},"public_addr":"172.21.15.174:6817/2847141411","cluster_addr":"172.21.15.174:6819/2847141411","heartbeat_back_addr":"172.21.15.174:6823/2847141411","heartbeat_front_addr":"172.21.15.174:6821/2847141411","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:24:43.753 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:43.752+0000 7fcd9de3e700 1 -- 172.21.15.143:0/1774431240 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fcd8406ae70 msgr2=0x7fcd8406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:43.753 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:43.752+0000 7fcd9de3e700 1 --2- 172.21.15.143:0/1774431240 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fcd8406ae70 0x7fcd8406d330 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fcd980c7240 tx=0x7fcd8c009580 comp rx=0 tx=0).stop 2023-11-08T19:24:43.754 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:43.752+0000 7fcd9de3e700 1 -- 172.21.15.143:0/1774431240 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcd980afd90 msgr2=0x7fcd981f6cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:43.754 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:43.752+0000 7fcd9de3e700 1 --2- 172.21.15.143:0/1774431240 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcd980afd90 0x7fcd981f6cd0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fcd9000efc0 tx=0x7fcd9000c3a0 comp rx=0 tx=0).stop 2023-11-08T19:24:43.754 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:43.752+0000 7fcd9de3e700 1 -- 172.21.15.143:0/1774431240 shutdown_connections 2023-11-08T19:24:43.754 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:43.752+0000 7fcd9de3e700 1 --2- 172.21.15.143:0/1774431240 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fcd8406ae70 0x7fcd8406d330 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:43.755 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:43.752+0000 7fcd9de3e700 1 --2- 172.21.15.143:0/1774431240 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcd9806b110 0x7fcd980c6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:43.755 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:43.752+0000 7fcd9de3e700 1 --2- 172.21.15.143:0/1774431240 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcd980afd90 0x7fcd981f6cd0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:43.755 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:43.752+0000 7fcd9de3e700 1 --2- 172.21.15.143:0/1774431240 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcd980af9b0 0x7fcd9815ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:43.755 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:43.752+0000 7fcd9de3e700 1 -- 172.21.15.143:0/1774431240 >> 172.21.15.143:0/1774431240 conn(0x7fcd980725e0 msgr2=0x7fcd980d0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:24:43.755 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:43.752+0000 7fcd9de3e700 1 -- 172.21.15.143:0/1774431240 shutdown_connections 2023-11-08T19:24:43.755 INFO:teuthology.misc.health.smithi143.stderr:2023-11-08T19:24:43.752+0000 7fcd9de3e700 1 -- 172.21.15.143:0/1774431240 wait complete. 2023-11-08T19:24:43.764 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-11-08T19:24:43.765 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-11-08T19:24:43.766 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-08T19:24:43.766 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-08T19:24:43.768 INFO:tasks.ceph.mds.a:Started 2023-11-08T19:24:43.769 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-08T19:24:43.769 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-08T19:24:43.771 INFO:tasks.ceph.mds.c:Started 2023-11-08T19:24:43.771 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-08T19:24:43.772 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-08T19:24:43.775 INFO:tasks.ceph.mds.b:Started 2023-11-08T19:24:43.775 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-08T19:24:43.775 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-08T19:24:43.777 INFO:tasks.ceph.mds.d:Started 2023-11-08T19:24:43.778 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-11-08T19:24:43.778 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:24:43.849 INFO:tasks.ceph.mds.c.smithi143.stdout:starting mds.c at 2023-11-08T19:24:43.849 INFO:tasks.ceph.mds.a.smithi143.stdout:starting mds.a at 2023-11-08T19:24:43.864 INFO:tasks.ceph.mds.d.smithi174.stdout:starting mds.d at 2023-11-08T19:24:43.865 INFO:tasks.ceph.mds.b.smithi174.stdout:starting mds.b at 2023-11-08T19:24:43.907 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:43.905+0000 7f76a4edc700 1 Processor -- start 2023-11-08T19:24:43.907 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:43.906+0000 7f76a4edc700 1 -- start start 2023-11-08T19:24:43.910 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:43.906+0000 7f76a4edc700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f76a015aa30 0x7f76a00cf510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:43.910 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:43.906+0000 7f76a4edc700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f76a00b2210 0x7f76a00b2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:43.910 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:43.906+0000 7f76a4edc700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f76a00cbe20 0x7f76a00cc270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:43.910 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:43.906+0000 7f76a4edc700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76a00b2bb0 con 0x7f76a00cbe20 2023-11-08T19:24:43.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:43.906+0000 7f76a4edc700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76a00cf080 con 0x7f76a015aa30 2023-11-08T19:24:43.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:43.906+0000 7f76a4edc700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76a00ce470 con 0x7f76a00b2210 2023-11-08T19:24:43.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:43.907+0000 7f769ed9d700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f76a00cbe20 0x7f76a00cc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:43.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:43.907+0000 7f769ed9d700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f76a00cbe20 0x7f76a00cc270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48394/0 (socket says 172.21.15.143:48394) 2023-11-08T19:24:43.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:43.907+0000 7f769ed9d700 1 -- 172.21.15.143:0/3150283710 learned_addr learned my addr 172.21.15.143:0/3150283710 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:24:43.912 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:43.907+0000 7f769e59c700 1 --2- 172.21.15.143:0/3150283710 >> v2:172.21.15.174:3300/0 conn(0x7f76a015aa30 0x7f76a00cf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:43.912 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:43.907+0000 7f769dd9b700 1 --2- 172.21.15.143:0/3150283710 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f76a00b2210 0x7f76a00b2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:43.912 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:43.907+0000 7f769ed9d700 1 -- 172.21.15.143:0/3150283710 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f76a00b2210 msgr2=0x7f76a00b2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:24:43.912 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:43.907+0000 7f769ed9d700 1 --2- 172.21.15.143:0/3150283710 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f76a00b2210 0x7f76a00b2670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:43.913 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:43.907+0000 7f769ed9d700 1 -- 172.21.15.143:0/3150283710 >> v2:172.21.15.174:3300/0 conn(0x7f76a015aa30 msgr2=0x7f76a00cf510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:24:43.913 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:43.907+0000 7f769ed9d700 1 --2- 172.21.15.143:0/3150283710 >> v2:172.21.15.174:3300/0 conn(0x7f76a015aa30 0x7f76a00cf510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:43.913 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:43.907+0000 7f769ed9d700 1 -- 172.21.15.143:0/3150283710 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76a00ce750 con 0x7f76a00cbe20 2023-11-08T19:24:43.913 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:43.907+0000 7f769ed9d700 1 --2- 172.21.15.143:0/3150283710 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f76a00cbe20 0x7f76a00cc270 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f76a0071c90 tx=0x7f769800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9af50c2f30143eb9 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:43.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:43.907+0000 7f769d59a700 1 -- 172.21.15.143:0/3150283710 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f769800e070 con 0x7f76a00cbe20 2023-11-08T19:24:43.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:43.907+0000 7f769d59a700 1 -- 172.21.15.143:0/3150283710 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f769800b910 con 0x7f76a00cbe20 2023-11-08T19:24:43.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:43.907+0000 7f769d59a700 1 -- 172.21.15.143:0/3150283710 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f769800bcc0 con 0x7f76a00cbe20 2023-11-08T19:24:43.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:43.908+0000 7f76a4edc700 1 -- 172.21.15.143:0/3150283710 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f76a00cbe20 msgr2=0x7f76a00cc270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:43.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:43.908+0000 7f76a4edc700 1 --2- 172.21.15.143:0/3150283710 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f76a00cbe20 0x7f76a00cc270 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f76a0071c90 tx=0x7f769800b330 comp rx=0 tx=0).stop 2023-11-08T19:24:43.915 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:43.908+0000 7f76a4edc700 1 -- 172.21.15.143:0/3150283710 shutdown_connections 2023-11-08T19:24:43.915 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:43.908+0000 7f76a4edc700 1 --2- 172.21.15.143:0/3150283710 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f76a00cbe20 0x7f76a00cc270 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:43.915 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:43.908+0000 7f76a4edc700 1 --2- 172.21.15.143:0/3150283710 >> v2:172.21.15.174:3300/0 conn(0x7f76a015aa30 0x7f76a00cf510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:43.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:43.908+0000 7f76a4edc700 1 --2- 172.21.15.143:0/3150283710 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f76a00b2210 0x7f76a00b2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:43.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:43.908+0000 7f76a4edc700 1 -- 172.21.15.143:0/3150283710 >> 172.21.15.143:0/3150283710 conn(0x7f76a0099ab0 msgr2=0x7f76a00d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:24:43.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:43.908+0000 7f76a4edc700 1 -- 172.21.15.143:0/3150283710 shutdown_connections 2023-11-08T19:24:43.917 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:43.908+0000 7f76a4edc700 1 -- 172.21.15.143:0/3150283710 wait complete. 2023-11-08T19:24:43.921 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:43.909+0000 7f76a4edc700 1 Processor -- start 2023-11-08T19:24:43.921 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:43.909+0000 7f76a4edc700 1 -- start start 2023-11-08T19:24:43.921 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:43.910+0000 7f76a4edc700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f76a00b2210 0x7f76a00c7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:43.921 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:43.910+0000 7f76a4edc700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f76a00cbe20 0x7f76a00b6010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:43.921 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:43.910+0000 7f76a4edc700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f76a015aa30 0x7f76a006ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:43.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:43.910+0000 7f76a4edc700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76a00c7660 con 0x7f76a00b2210 2023-11-08T19:24:43.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:43.910+0000 7f76a4edc700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76a014d8b0 con 0x7f76a015aa30 2023-11-08T19:24:43.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:43.910+0000 7f76a4edc700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76a014d5d0 con 0x7f76a00cbe20 2023-11-08T19:24:43.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:43.910+0000 7f769e59c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f76a00b2210 0x7f76a00c7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:43.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:43.910+0000 7f769e59c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f76a00b2210 0x7f76a00c7120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48396/0 (socket says 172.21.15.143:48396) 2023-11-08T19:24:43.923 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:43.910+0000 7f769e59c700 1 -- 172.21.15.143:0/1380740814 learned_addr learned my addr 172.21.15.143:0/1380740814 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:24:43.923 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:43.910+0000 7f769ed9d700 1 --2- 172.21.15.143:0/1380740814 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f76a015aa30 0x7f76a006ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:43.923 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:43.910+0000 7f769dd9b700 1 --2- 172.21.15.143:0/1380740814 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f76a00cbe20 0x7f76a00b6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:43.924 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:43.910+0000 7f769e59c700 1 -- 172.21.15.143:0/1380740814 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f76a00cbe20 msgr2=0x7f76a00b6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:43.924 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:43.910+0000 7f769e59c700 1 --2- 172.21.15.143:0/1380740814 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f76a00cbe20 0x7f76a00b6010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:43.924 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:43.910+0000 7f769e59c700 1 -- 172.21.15.143:0/1380740814 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f76a015aa30 msgr2=0x7f76a006ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:43.924 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:43.910+0000 7f769e59c700 1 --2- 172.21.15.143:0/1380740814 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f76a015aa30 0x7f76a006ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:43.925 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:43.910+0000 7f769e59c700 1 -- 172.21.15.143:0/1380740814 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7698009b30 con 0x7f76a00b2210 2023-11-08T19:24:43.925 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:43.910+0000 7f769e59c700 1 --2- 172.21.15.143:0/1380740814 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f76a00b2210 0x7f76a00c7120 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f769000e900 tx=0x7f769000ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:43.925 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:43.911+0000 7f768effd700 1 -- 172.21.15.143:0/1380740814 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7690013680 con 0x7f76a00b2210 2023-11-08T19:24:43.925 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:43.911+0000 7f768effd700 1 -- 172.21.15.143:0/1380740814 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76900137e0 con 0x7f76a00b2210 2023-11-08T19:24:43.925 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:43.911+0000 7f768effd700 1 -- 172.21.15.143:0/1380740814 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7690013b50 con 0x7f76a00b2210 2023-11-08T19:24:43.926 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:43.911+0000 7f76a4edc700 1 -- 172.21.15.143:0/1380740814 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76a01f4d00 con 0x7f76a00b2210 2023-11-08T19:24:43.926 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:43.911+0000 7f76a4edc700 1 -- 172.21.15.143:0/1380740814 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76a01f52c0 con 0x7f76a00b2210 2023-11-08T19:24:43.926 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:43.912+0000 7f76a4edc700 1 -- 172.21.15.143:0/1380740814 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76a01f5450 con 0x7f76a00b2210 2023-11-08T19:24:43.927 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:43.912+0000 7f768effd700 1 -- 172.21.15.143:0/1380740814 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 4) v1 ==== 89175+0+0 (secure 0 0 0) 0x7f7690013cb0 con 0x7f76a00b2210 2023-11-08T19:24:43.927 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:43.912+0000 7f768effd700 1 --2- 172.21.15.143:0/1380740814 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f768006ae70 0x7f768006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:43.927 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:43.912+0000 7f768effd700 1 -- 172.21.15.143:0/1380740814 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 4471+0+0 (secure 0 0 0) 0x7f7690089200 con 0x7f76a00b2210 2023-11-08T19:24:43.927 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:43.913+0000 7f769dd9b700 1 --2- 172.21.15.143:0/1380740814 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f768006ae70 0x7f768006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:43.928 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:43.913+0000 7f769dd9b700 1 --2- 172.21.15.143:0/1380740814 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f768006ae70 0x7f768006d330 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f76a00b3610 tx=0x7f7694009210 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:43.928 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:43.917+0000 7f768effd700 1 -- 172.21.15.143:0/1380740814 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f76900548f0 con 0x7f76a00b2210 2023-11-08T19:24:44.107 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:44.106+0000 7f76a4edc700 1 -- 172.21.15.143:0/1380740814 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f76a004eee0 con 0x7f76a00b2210 2023-11-08T19:24:44.108 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:44.106+0000 7f768effd700 1 -- 172.21.15.143:0/1380740814 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) v1 ==== 74+0+10956 (secure 0 0 0) 0x7f7690021090 con 0x7f76a00b2210 2023-11-08T19:24:44.108 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:24:44.108 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":13,"fsid":"968483af-77fc-4068-ad7c-5c2a1a4613d7","created":"2023-11-08T19:24:32.826484+0000","modified":"2023-11-08T19:24:43.573880+0000","last_up_change":"2023-11-08T19:24:43.573880+0000","last_in_change":"2023-11-08T19:24:35.314525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"bae98ec1-d94e-403d-a220-a522a53ea6f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6809","nonce":1487830566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6811","nonce":1487830566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6815","nonce":1487830566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6813","nonce":1487830566}]},"public_addr":"172.21.15.143:6809/1487830566","cluster_addr":"172.21.15.143:6811/1487830566","heartbeat_back_addr":"172.21.15.143:6815/1487830566","heartbeat_front_addr":"172.21.15.143:6813/1487830566","state":["exists","up"]},{"osd":1,"uuid":"5cb6538a-1ccb-4322-8633-09f66af0b7e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6801","nonce":787341397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6803","nonce":787341397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6807","nonce":787341397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6805","nonce":787341397}]},"public_addr":"172.21.15.143:6801/787341397","cluster_addr":"172.21.15.143:6803/787341397","heartbeat_back_addr":"172.21.15.143:6807/787341397","heartbeat_front_addr":"172.21.15.143:6805/787341397","state":["exists","up"]},{"osd":2,"uuid":"cdbf522c-60ef-4d60-9653-8dcb529c101a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6822","nonce":2301325597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6826","nonce":2301325597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6831","nonce":2301325597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6829","nonce":2301325597}]},"public_addr":"172.21.15.143:6822/2301325597","cluster_addr":"172.21.15.143:6826/2301325597","heartbeat_back_addr":"172.21.15.143:6831/2301325597","heartbeat_front_addr":"172.21.15.143:6829/2301325597","state":["exists","up"]},{"osd":3,"uuid":"4e2783cf-ebb9-4dd8-8245-bc1d17ce4325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6817","nonce":2021504428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6819","nonce":2021504428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6827","nonce":2021504428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6823","nonce":2021504428}]},"public_addr":"172.21.15.143:6817/2021504428","cluster_addr":"172.21.15.143:6819/2021504428","heartbeat_back_addr":"172.21.15.143:6827/2021504428","heartbeat_front_addr":"172.21.15.143:6823/2021504428","state":["exists","up"]},{"osd":4,"uuid":"691256fa-9b77-49b9-a346-7ae1adf0ca80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6801","nonce":1993380595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6803","nonce":1993380595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6807","nonce":1993380595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6805","nonce":1993380595}]},"public_addr":"172.21.15.174:6801/1993380595","cluster_addr":"172.21.15.174:6803/1993380595","heartbeat_back_addr":"172.21.15.174:6807/1993380595","heartbeat_front_addr":"172.21.15.174:6805/1993380595","state":["exists","up"]},{"osd":5,"uuid":"756fc352-48c9-46ce-b52e-0541fe6f8525","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6809","nonce":1777923303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6811","nonce":1777923303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6815","nonce":1777923303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6813","nonce":1777923303}]},"public_addr":"172.21.15.174:6809/1777923303","cluster_addr":"172.21.15.174:6811/1777923303","heartbeat_back_addr":"172.21.15.174:6815/1777923303","heartbeat_front_addr":"172.21.15.174:6813/1777923303","state":["exists","up"]},{"osd":6,"uuid":"fab2ac2b-ed97-4cde-9825-b480eabf8b8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6825","nonce":2755130316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6827","nonce":2755130316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6831","nonce":2755130316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6829","nonce":2755130316}]},"public_addr":"172.21.15.174:6825/2755130316","cluster_addr":"172.21.15.174:6827/2755130316","heartbeat_back_addr":"172.21.15.174:6831/2755130316","heartbeat_front_addr":"172.21.15.174:6829/2755130316","state":["exists","up"]},{"osd":7,"uuid":"3de9b797-2ab6-4d41-a863-7e7d638fd703","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6817","nonce":2847141411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6819","nonce":2847141411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6823","nonce":2847141411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6821","nonce":2847141411}]},"public_addr":"172.21.15.174:6817/2847141411","cluster_addr":"172.21.15.174:6819/2847141411","heartbeat_back_addr":"172.21.15.174:6823/2847141411","heartbeat_front_addr":"172.21.15.174:6821/2847141411","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:24:44.110 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:44.109+0000 7f76a4edc700 1 -- 172.21.15.143:0/1380740814 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f768006ae70 msgr2=0x7f768006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:44.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:44.109+0000 7f76a4edc700 1 --2- 172.21.15.143:0/1380740814 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f768006ae70 0x7f768006d330 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f76a00b3610 tx=0x7f7694009210 comp rx=0 tx=0).stop 2023-11-08T19:24:44.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:44.109+0000 7f76a4edc700 1 -- 172.21.15.143:0/1380740814 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f76a00b2210 msgr2=0x7f76a00c7120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:44.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:44.109+0000 7f76a4edc700 1 --2- 172.21.15.143:0/1380740814 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f76a00b2210 0x7f76a00c7120 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f769000e900 tx=0x7f769000ec10 comp rx=0 tx=0).stop 2023-11-08T19:24:44.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:44.109+0000 7f76a4edc700 1 -- 172.21.15.143:0/1380740814 shutdown_connections 2023-11-08T19:24:44.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:44.109+0000 7f76a4edc700 1 --2- 172.21.15.143:0/1380740814 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f768006ae70 0x7f768006d330 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:44.112 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:44.109+0000 7f76a4edc700 1 --2- 172.21.15.143:0/1380740814 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f76a015aa30 0x7f76a006ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:44.112 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:44.109+0000 7f76a4edc700 1 --2- 172.21.15.143:0/1380740814 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f76a00b2210 0x7f76a00c7120 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:44.112 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:44.109+0000 7f76a4edc700 1 --2- 172.21.15.143:0/1380740814 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f76a00cbe20 0x7f76a00b6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:44.113 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:44.109+0000 7f76a4edc700 1 -- 172.21.15.143:0/1380740814 >> 172.21.15.143:0/1380740814 conn(0x7f76a0099ab0 msgr2=0x7f76a00cf7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:24:44.113 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:44.109+0000 7f76a4edc700 1 -- 172.21.15.143:0/1380740814 shutdown_connections 2023-11-08T19:24:44.113 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:44.109+0000 7f76a4edc700 1 -- 172.21.15.143:0/1380740814 wait complete. 2023-11-08T19:24:44.122 INFO:tasks.cephfs.filesystem.ceph_manager:[] 2023-11-08T19:24:44.123 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:24:44.243 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:44.241+0000 7f547b5f5700 1 Processor -- start 2023-11-08T19:24:44.243 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:44.242+0000 7f547b5f5700 1 -- start start 2023-11-08T19:24:44.246 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:44.242+0000 7f547b5f5700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f54740668e0 0x7f54740cab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:44.246 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:44.242+0000 7f547b5f5700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f547406df90 0x7f547406aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:44.246 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:44.242+0000 7f547b5f5700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54740ce7f0 con 0x7f547406dbb0 2023-11-08T19:24:44.246 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:44.242+0000 7f547b5f5700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5474066db0 con 0x7f54740668e0 2023-11-08T19:24:44.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:44.242+0000 7f547b5f5700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54740af5e0 con 0x7f547406df90 2023-11-08T19:24:44.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:44.242+0000 7f5479391700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f547406dbb0 0x7f54740ca470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:44558/0 (socket says 172.21.15.143:44558) 2023-11-08T19:24:44.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:44.242+0000 7f5479391700 1 -- 172.21.15.143:0/3753914794 learned_addr learned my addr 172.21.15.143:0/3753914794 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:24:44.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:44.243+0000 7f5478b90700 1 --2- 172.21.15.143:0/3753914794 >> v2:172.21.15.174:3300/0 conn(0x7f54740668e0 0x7f54740cab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:44.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:44.243+0000 7f5479b92700 1 --2- 172.21.15.143:0/3753914794 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f547406df90 0x7f547406aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:44.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:44.243+0000 7f546bfff700 1 -- 172.21.15.143:0/3753914794 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3521344542 0 0) 0x7f54740ce7f0 con 0x7f547406dbb0 2023-11-08T19:24:44.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:44.243+0000 7f546bfff700 1 -- 172.21.15.143:0/3753914794 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f54600035b0 con 0x7f547406dbb0 2023-11-08T19:24:44.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:44.243+0000 7f546bfff700 1 -- 172.21.15.143:0/3753914794 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2128327626 0 0) 0x7f54600035b0 con 0x7f547406dbb0 2023-11-08T19:24:44.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:44.243+0000 7f5479b92700 1 -- 172.21.15.143:0/3753914794 >> v2:172.21.15.174:3300/0 conn(0x7f54740668e0 msgr2=0x7f54740cab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:24:44.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:44.243+0000 7f5479b92700 1 --2- 172.21.15.143:0/3753914794 >> v2:172.21.15.174:3300/0 conn(0x7f54740668e0 0x7f54740cab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:44.249 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:44.243+0000 7f5479b92700 1 -- 172.21.15.143:0/3753914794 >> v1:172.21.15.143:6789/0 conn(0x7f547406dbb0 legacy=0x7f54740ca470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:44.249 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:44.243+0000 7f5479b92700 1 -- 172.21.15.143:0/3753914794 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54740cb520 con 0x7f547406df90 2023-11-08T19:24:44.249 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:44.243+0000 7f5479b92700 1 --2- 172.21.15.143:0/3753914794 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f547406df90 0x7f547406aff0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f546c009d90 tx=0x7f546c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cfda40bbc43a500a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:44.249 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:44.243+0000 7f546bfff700 1 -- 172.21.15.143:0/3753914794 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f546c00e070 con 0x7f547406df90 2023-11-08T19:24:44.249 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:44.243+0000 7f546bfff700 1 -- 172.21.15.143:0/3753914794 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f546c003a20 con 0x7f547406df90 2023-11-08T19:24:44.249 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:44.243+0000 7f546bfff700 1 -- 172.21.15.143:0/3753914794 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5464001670 con 0x7f547406df90 2023-11-08T19:24:44.250 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:44.244+0000 7f547b5f5700 1 -- 172.21.15.143:0/3753914794 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f547406df90 msgr2=0x7f547406aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:44.250 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:44.244+0000 7f547b5f5700 1 --2- 172.21.15.143:0/3753914794 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f547406df90 0x7f547406aff0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f546c009d90 tx=0x7f546c00b330 comp rx=0 tx=0).stop 2023-11-08T19:24:44.250 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:44.244+0000 7f547b5f5700 1 -- 172.21.15.143:0/3753914794 shutdown_connections 2023-11-08T19:24:44.250 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:44.244+0000 7f547b5f5700 1 --2- 172.21.15.143:0/3753914794 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f547406df90 0x7f547406aff0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:44.250 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:44.244+0000 7f547b5f5700 1 --2- 172.21.15.143:0/3753914794 >> v2:172.21.15.174:3300/0 conn(0x7f54740668e0 0x7f54740cab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:44.251 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:44.244+0000 7f547b5f5700 1 -- 172.21.15.143:0/3753914794 >> 172.21.15.143:0/3753914794 conn(0x7f54740b1a20 msgr2=0x7f54740b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:24:44.251 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:44.244+0000 7f547b5f5700 1 -- 172.21.15.143:0/3753914794 shutdown_connections 2023-11-08T19:24:44.251 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:44.244+0000 7f547b5f5700 1 -- 172.21.15.143:0/3753914794 wait complete. 2023-11-08T19:24:44.254 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:44.245+0000 7f547b5f5700 1 Processor -- start 2023-11-08T19:24:44.254 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:44.245+0000 7f547b5f5700 1 -- start start 2023-11-08T19:24:44.254 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:44.246+0000 7f547b5f5700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f54740668e0 0x7f547415a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:44.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:44.246+0000 7f547b5f5700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f547406dbb0 0x7f54740b40a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:44.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:44.246+0000 7f547b5f5700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f547406df90 0x7f547414c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:44.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:44.246+0000 7f547b5f5700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f547414c8c0 con 0x7f547406dbb0 2023-11-08T19:24:44.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:44.246+0000 7f547b5f5700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54740b3000 con 0x7f547406df90 2023-11-08T19:24:44.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:44.246+0000 7f547b5f5700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54740b32e0 con 0x7f54740668e0 2023-11-08T19:24:44.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:44.246+0000 7f5478b90700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f547406dbb0 0x7f54740b40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:44.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:44.246+0000 7f5478b90700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f547406dbb0 0x7f54740b40a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48412/0 (socket says 172.21.15.143:48412) 2023-11-08T19:24:44.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:44.246+0000 7f5478b90700 1 -- 172.21.15.143:0/169633935 learned_addr learned my addr 172.21.15.143:0/169633935 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:24:44.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:44.246+0000 7f5479391700 1 --2- 172.21.15.143:0/169633935 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f54740668e0 0x7f547415a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:44.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:44.246+0000 7f5479b92700 1 --2- 172.21.15.143:0/169633935 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f547406df90 0x7f547414c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:44.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:44.246+0000 7f5478b90700 1 -- 172.21.15.143:0/169633935 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f54740668e0 msgr2=0x7f547415a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:44.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:44.246+0000 7f5478b90700 1 --2- 172.21.15.143:0/169633935 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f54740668e0 0x7f547415a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:44.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:44.246+0000 7f5478b90700 1 -- 172.21.15.143:0/169633935 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f547406df90 msgr2=0x7f547414c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:44.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:44.246+0000 7f5478b90700 1 --2- 172.21.15.143:0/169633935 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f547406df90 0x7f547414c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:44.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:44.246+0000 7f5478b90700 1 -- 172.21.15.143:0/169633935 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f546c009a70 con 0x7f547406dbb0 2023-11-08T19:24:44.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:44.246+0000 7f5478b90700 1 --2- 172.21.15.143:0/169633935 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f547406dbb0 0x7f54740b40a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f547000e9b0 tx=0x7f547000ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:44.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:44.246+0000 7f5469ffb700 1 -- 172.21.15.143:0/169633935 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5470009710 con 0x7f547406dbb0 2023-11-08T19:24:44.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:44.246+0000 7f5469ffb700 1 -- 172.21.15.143:0/169633935 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f547000c850 con 0x7f547406dbb0 2023-11-08T19:24:44.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:44.247+0000 7f5469ffb700 1 -- 172.21.15.143:0/169633935 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f547000cbc0 con 0x7f547406dbb0 2023-11-08T19:24:44.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:44.247+0000 7f547b5f5700 1 -- 172.21.15.143:0/169633935 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54740b35c0 con 0x7f547406dbb0 2023-11-08T19:24:44.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:44.247+0000 7f547b5f5700 1 -- 172.21.15.143:0/169633935 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54740b3b80 con 0x7f547406dbb0 2023-11-08T19:24:44.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:44.248+0000 7f547b5f5700 1 -- 172.21.15.143:0/169633935 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f547406d1a0 con 0x7f547406dbb0 2023-11-08T19:24:44.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:44.248+0000 7f5469ffb700 1 -- 172.21.15.143:0/169633935 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 4) v1 ==== 89175+0+0 (secure 0 0 0) 0x7f547000cd20 con 0x7f547406dbb0 2023-11-08T19:24:44.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:44.248+0000 7f5469ffb700 1 --2- 172.21.15.143:0/169633935 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f545806ae70 0x7f545806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:44.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:44.248+0000 7f5469ffb700 1 -- 172.21.15.143:0/169633935 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 4471+0+0 (secure 0 0 0) 0x7f5470089040 con 0x7f547406dbb0 2023-11-08T19:24:44.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:44.249+0000 7f5479391700 1 --2- 172.21.15.143:0/169633935 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f545806ae70 0x7f545806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:44.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:44.249+0000 7f5479391700 1 --2- 172.21.15.143:0/169633935 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f545806ae70 0x7f545806d330 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5464001eb0 tx=0x7f5464001e40 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:44.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:44.253+0000 7f5469ffb700 1 -- 172.21.15.143:0/169633935 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f54700546a0 con 0x7f547406dbb0 2023-11-08T19:24:44.490 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:44.488+0000 7f547b5f5700 1 -- 172.21.15.143:0/169633935 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f547404eee0 con 0x7f547406dbb0 2023-11-08T19:24:44.490 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:44.488+0000 7f5469ffb700 1 -- 172.21.15.143:0/169633935 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1327 (secure 0 0 0) 0x7f5470021540 con 0x7f547406dbb0 2023-11-08T19:24:44.491 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:24:44.491 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/2364281303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":2364281303},{"type":"v1","addr":"172.21.15.143:6835","nonce":2364281303}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2023-11-08T19:24:44.493 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:44.491+0000 7f547b5f5700 1 -- 172.21.15.143:0/169633935 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f545806ae70 msgr2=0x7f545806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:44.493 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:44.491+0000 7f547b5f5700 1 --2- 172.21.15.143:0/169633935 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f545806ae70 0x7f545806d330 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5464001eb0 tx=0x7f5464001e40 comp rx=0 tx=0).stop 2023-11-08T19:24:44.493 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:44.491+0000 7f547b5f5700 1 -- 172.21.15.143:0/169633935 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f547406dbb0 msgr2=0x7f54740b40a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:44.494 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:44.491+0000 7f547b5f5700 1 --2- 172.21.15.143:0/169633935 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f547406dbb0 0x7f54740b40a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f547000e9b0 tx=0x7f547000ecc0 comp rx=0 tx=0).stop 2023-11-08T19:24:44.494 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:44.491+0000 7f547b5f5700 1 -- 172.21.15.143:0/169633935 shutdown_connections 2023-11-08T19:24:44.494 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:44.491+0000 7f547b5f5700 1 --2- 172.21.15.143:0/169633935 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f545806ae70 0x7f545806d330 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:44.495 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:44.491+0000 7f547b5f5700 1 --2- 172.21.15.143:0/169633935 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f54740668e0 0x7f547415a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:44.495 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:44.491+0000 7f547b5f5700 1 --2- 172.21.15.143:0/169633935 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f547406df90 0x7f547414c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:44.495 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:44.491+0000 7f547b5f5700 1 --2- 172.21.15.143:0/169633935 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f547406dbb0 0x7f54740b40a0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:44.495 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:44.491+0000 7f547b5f5700 1 -- 172.21.15.143:0/169633935 >> 172.21.15.143:0/169633935 conn(0x7f54740b1a20 msgr2=0x7f547406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:24:44.495 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:44.491+0000 7f547b5f5700 1 -- 172.21.15.143:0/169633935 shutdown_connections 2023-11-08T19:24:44.496 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:44.491+0000 7f547b5f5700 1 -- 172.21.15.143:0/169633935 wait complete. 2023-11-08T19:24:44.496 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 2 2023-11-08T19:24:46.505 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:24:46.627 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:46.625+0000 7fa1846d8700 1 Processor -- start 2023-11-08T19:24:46.627 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:46.625+0000 7fa1846d8700 1 -- start start 2023-11-08T19:24:46.629 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:46.626+0000 7fa1846d8700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fa17c0d1c70 0x7fa17c0b14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:46.630 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:46.626+0000 7fa1846d8700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa17c0d03a0 0x7fa17c0cc5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:46.630 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:46.626+0000 7fa1846d8700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa17c0b1bc0 con 0x7fa17c15ba30 2023-11-08T19:24:46.630 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:46.626+0000 7fa1846d8700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa17c0b1ea0 con 0x7fa17c0d03a0 2023-11-08T19:24:46.630 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:46.626+0000 7fa1846d8700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa17c0b2180 con 0x7fa17c0d1c70 2023-11-08T19:24:46.630 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:46.626+0000 7fa182c75700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa17c0d03a0 0x7fa17c0cc5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:46.631 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:46.626+0000 7fa182474700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fa17c15ba30 0x7fa17c0d1560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:44614/0 (socket says 172.21.15.143:44614) 2023-11-08T19:24:46.631 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:46.626+0000 7fa182474700 1 -- 172.21.15.143:0/1790619751 learned_addr learned my addr 172.21.15.143:0/1790619751 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:24:46.631 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:46.627+0000 7fa181c73700 1 --2- 172.21.15.143:0/1790619751 >> v2:172.21.15.174:3300/0 conn(0x7fa17c0d1c70 0x7fa17c0b14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:46.631 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:46.627+0000 7fa182c75700 1 -- 172.21.15.143:0/1790619751 >> v2:172.21.15.174:3300/0 conn(0x7fa17c0d1c70 msgr2=0x7fa17c0b14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:24:46.631 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:46.627+0000 7fa182c75700 1 --2- 172.21.15.143:0/1790619751 >> v2:172.21.15.174:3300/0 conn(0x7fa17c0d1c70 0x7fa17c0b14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:46.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:46.627+0000 7fa182c75700 1 -- 172.21.15.143:0/1790619751 >> v1:172.21.15.143:6789/0 conn(0x7fa17c15ba30 legacy=0x7fa17c0d1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:46.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:46.627+0000 7fa181472700 1 -- 172.21.15.143:0/1790619751 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1568048022 0 0) 0x7fa17c0b1bc0 con 0x7fa17c15ba30 2023-11-08T19:24:46.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:46.627+0000 7fa182c75700 1 -- 172.21.15.143:0/1790619751 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa17c0b2460 con 0x7fa17c0d03a0 2023-11-08T19:24:46.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:46.627+0000 7fa182c75700 1 --2- 172.21.15.143:0/1790619751 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa17c0d03a0 0x7fa17c0cc5a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa17c0cbe60 tx=0x7fa17400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a51504d0f98d0df0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:46.633 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:46.627+0000 7fa181472700 1 -- 172.21.15.143:0/1790619751 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa17400e070 con 0x7fa17c0d03a0 2023-11-08T19:24:46.633 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:46.627+0000 7fa181472700 1 -- 172.21.15.143:0/1790619751 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa17400bb20 con 0x7fa17c0d03a0 2023-11-08T19:24:46.633 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:46.627+0000 7fa181472700 1 -- 172.21.15.143:0/1790619751 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa174012430 con 0x7fa17c0d03a0 2023-11-08T19:24:46.633 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:46.628+0000 7fa1846d8700 1 -- 172.21.15.143:0/1790619751 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa17c0d03a0 msgr2=0x7fa17c0cc5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:46.633 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:46.628+0000 7fa1846d8700 1 --2- 172.21.15.143:0/1790619751 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa17c0d03a0 0x7fa17c0cc5a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa17c0cbe60 tx=0x7fa17400b330 comp rx=0 tx=0).stop 2023-11-08T19:24:46.634 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:46.628+0000 7fa1846d8700 1 -- 172.21.15.143:0/1790619751 shutdown_connections 2023-11-08T19:24:46.634 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:46.628+0000 7fa1846d8700 1 --2- 172.21.15.143:0/1790619751 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa17c0d03a0 0x7fa17c0cc5a0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:46.634 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:46.628+0000 7fa1846d8700 1 --2- 172.21.15.143:0/1790619751 >> v2:172.21.15.174:3300/0 conn(0x7fa17c0d1c70 0x7fa17c0b14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:46.634 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:46.628+0000 7fa1846d8700 1 -- 172.21.15.143:0/1790619751 >> 172.21.15.143:0/1790619751 conn(0x7fa17c08b000 msgr2=0x7fa17c09ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:24:46.634 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:46.628+0000 7fa1846d8700 1 -- 172.21.15.143:0/1790619751 shutdown_connections 2023-11-08T19:24:46.635 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:46.628+0000 7fa1846d8700 1 -- 172.21.15.143:0/1790619751 wait complete. 2023-11-08T19:24:46.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:46.629+0000 7fa1846d8700 1 Processor -- start 2023-11-08T19:24:46.638 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:46.629+0000 7fa1846d8700 1 -- start start 2023-11-08T19:24:46.638 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:46.630+0000 7fa1846d8700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa17c0d03a0 0x7fa17c06d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:46.638 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:46.630+0000 7fa1846d8700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa17c0d1c70 0x7fa17c157710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:46.638 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:46.630+0000 7fa1846d8700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa17c15ba30 0x7fa17c06bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:46.638 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:46.630+0000 7fa1846d8700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa17c06dc70 con 0x7fa17c15ba30 2023-11-08T19:24:46.639 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:46.630+0000 7fa1846d8700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa17c06c690 con 0x7fa17c0d03a0 2023-11-08T19:24:46.639 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:46.630+0000 7fa1846d8700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa17c06c970 con 0x7fa17c0d1c70 2023-11-08T19:24:46.639 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:46.630+0000 7fa182c75700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa17c15ba30 0x7fa17c06bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:46.639 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:46.630+0000 7fa182c75700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa17c15ba30 0x7fa17c06bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48470/0 (socket says 172.21.15.143:48470) 2023-11-08T19:24:46.639 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:46.630+0000 7fa182c75700 1 -- 172.21.15.143:0/3422891194 learned_addr learned my addr 172.21.15.143:0/3422891194 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:24:46.640 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:46.630+0000 7fa182474700 1 --2- 172.21.15.143:0/3422891194 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa17c0d03a0 0x7fa17c06d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:46.640 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:46.630+0000 7fa181c73700 1 --2- 172.21.15.143:0/3422891194 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa17c0d1c70 0x7fa17c157710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:46.640 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:46.630+0000 7fa182c75700 1 -- 172.21.15.143:0/3422891194 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa17c0d1c70 msgr2=0x7fa17c157710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:46.640 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:46.630+0000 7fa182c75700 1 --2- 172.21.15.143:0/3422891194 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa17c0d1c70 0x7fa17c157710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:46.640 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:46.630+0000 7fa182c75700 1 -- 172.21.15.143:0/3422891194 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa17c0d03a0 msgr2=0x7fa17c06d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:46.641 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:46.630+0000 7fa182c75700 1 --2- 172.21.15.143:0/3422891194 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa17c0d03a0 0x7fa17c06d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:46.641 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:46.630+0000 7fa182c75700 1 -- 172.21.15.143:0/3422891194 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa174009a70 con 0x7fa17c15ba30 2023-11-08T19:24:46.641 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:46.630+0000 7fa182c75700 1 --2- 172.21.15.143:0/3422891194 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa17c15ba30 0x7fa17c06bd20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa17c08c000 tx=0x7fa174004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:46.641 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:46.630+0000 7fa172ffd700 1 -- 172.21.15.143:0/3422891194 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa174012430 con 0x7fa17c15ba30 2023-11-08T19:24:46.641 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:46.630+0000 7fa172ffd700 1 -- 172.21.15.143:0/3422891194 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa174020070 con 0x7fa17c15ba30 2023-11-08T19:24:46.641 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:46.630+0000 7fa1846d8700 1 -- 172.21.15.143:0/3422891194 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa17c06cc50 con 0x7fa17c15ba30 2023-11-08T19:24:46.642 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:46.630+0000 7fa172ffd700 1 -- 172.21.15.143:0/3422891194 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa174004280 con 0x7fa17c15ba30 2023-11-08T19:24:46.642 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:46.630+0000 7fa1846d8700 1 -- 172.21.15.143:0/3422891194 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa17c06d210 con 0x7fa17c15ba30 2023-11-08T19:24:46.642 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:46.631+0000 7fa1846d8700 1 -- 172.21.15.143:0/3422891194 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa17c04eee0 con 0x7fa17c15ba30 2023-11-08T19:24:46.642 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:46.632+0000 7fa172ffd700 1 -- 172.21.15.143:0/3422891194 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 4) v1 ==== 89175+0+0 (secure 0 0 0) 0x7fa1740043e0 con 0x7fa17c15ba30 2023-11-08T19:24:46.642 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:46.633+0000 7fa172ffd700 1 --2- 172.21.15.143:0/3422891194 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa16006ae70 0x7fa16006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:46.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:46.633+0000 7fa172ffd700 1 -- 172.21.15.143:0/3422891194 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fa174022070 con 0x7fa17c15ba30 2023-11-08T19:24:46.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:46.633+0000 7fa182474700 1 --2- 172.21.15.143:0/3422891194 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa16006ae70 0x7fa16006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:46.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:46.633+0000 7fa182474700 1 --2- 172.21.15.143:0/3422891194 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa16006ae70 0x7fa16006d330 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa16c000f60 tx=0x7fa16c007e70 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:46.644 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:46.637+0000 7fa172ffd700 1 -- 172.21.15.143:0/3422891194 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa174056930 con 0x7fa17c15ba30 2023-11-08T19:24:46.874 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:46.873+0000 7fa1846d8700 1 -- 172.21.15.143:0/3422891194 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa17c0cee70 con 0x7fa17c15ba30 2023-11-08T19:24:46.875 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:46.873+0000 7fa172ffd700 1 -- 172.21.15.143:0/3422891194 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3700 (secure 0 0 0) 0x7fa17402c090 con 0x7fa17c15ba30 2023-11-08T19:24:46.875 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:24:46.876 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4208,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6834/37280614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":37280614},{"type":"v1","addr":"172.21.15.143:6834","nonce":37280614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/2364281303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":2364281303},{"type":"v1","addr":"172.21.15.143:6835","nonce":2364281303}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6836/4066129908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":4066129908},{"type":"v1","addr":"172.21.15.174:6836","nonce":4066129908}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4282,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/2124972405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6835","nonce":2124972405},{"type":"v1","addr":"172.21.15.174:6837","nonce":2124972405}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2023-11-08T19:24:46.877 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:46.876+0000 7fa1846d8700 1 -- 172.21.15.143:0/3422891194 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa16006ae70 msgr2=0x7fa16006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:46.877 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:46.876+0000 7fa1846d8700 1 --2- 172.21.15.143:0/3422891194 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa16006ae70 0x7fa16006d330 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa16c000f60 tx=0x7fa16c007e70 comp rx=0 tx=0).stop 2023-11-08T19:24:46.878 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:46.876+0000 7fa1846d8700 1 -- 172.21.15.143:0/3422891194 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa17c15ba30 msgr2=0x7fa17c06bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:46.878 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:46.876+0000 7fa1846d8700 1 --2- 172.21.15.143:0/3422891194 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa17c15ba30 0x7fa17c06bd20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa17c08c000 tx=0x7fa174004ad0 comp rx=0 tx=0).stop 2023-11-08T19:24:46.878 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:46.876+0000 7fa1846d8700 1 -- 172.21.15.143:0/3422891194 shutdown_connections 2023-11-08T19:24:46.879 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:46.876+0000 7fa1846d8700 1 --2- 172.21.15.143:0/3422891194 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa16006ae70 0x7fa16006d330 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:46.879 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:46.876+0000 7fa1846d8700 1 --2- 172.21.15.143:0/3422891194 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa17c15ba30 0x7fa17c06bd20 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:46.879 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:46.876+0000 7fa1846d8700 1 --2- 172.21.15.143:0/3422891194 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa17c0d03a0 0x7fa17c06d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:46.879 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:46.876+0000 7fa1846d8700 1 --2- 172.21.15.143:0/3422891194 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa17c0d1c70 0x7fa17c157710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:46.879 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:46.876+0000 7fa1846d8700 1 -- 172.21.15.143:0/3422891194 >> 172.21.15.143:0/3422891194 conn(0x7fa17c08b000 msgr2=0x7fa17c0ae9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:24:46.880 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:46.876+0000 7fa1846d8700 1 -- 172.21.15.143:0/3422891194 shutdown_connections 2023-11-08T19:24:46.880 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:46.876+0000 7fa1846d8700 1 -- 172.21.15.143:0/3422891194 wait complete. 2023-11-08T19:24:46.880 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 3 2023-11-08T19:24:46.889 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:24:47.010 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.009+0000 7f7ee19d8700 1 Processor -- start 2023-11-08T19:24:47.011 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.009+0000 7f7ee19d8700 1 -- start start 2023-11-08T19:24:47.017 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.009+0000 7f7ee19d8700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7edc0668e0 0x7f7edc0cab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:47.018 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.009+0000 7f7ee19d8700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f7edc06df90 0x7f7edc06aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:47.018 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.009+0000 7f7ee19d8700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7edc0ce7f0 con 0x7f7edc06dbb0 2023-11-08T19:24:47.018 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.009+0000 7f7ee19d8700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7edc066db0 con 0x7f7edc06df90 2023-11-08T19:24:47.018 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.009+0000 7f7ee19d8700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7edc0af5e0 con 0x7f7edc0668e0 2023-11-08T19:24:47.019 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.010+0000 7f7edaffd700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f7edc06dbb0 0x7f7edc0ca470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:44628/0 (socket says 172.21.15.143:44628) 2023-11-08T19:24:47.019 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.010+0000 7f7edaffd700 1 -- 172.21.15.143:0/2656960594 learned_addr learned my addr 172.21.15.143:0/2656960594 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:24:47.019 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.010+0000 7f7ed9ffb700 1 -- 172.21.15.143:0/2656960594 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3616365269 0 0) 0x7f7edc0ce7f0 con 0x7f7edc06dbb0 2023-11-08T19:24:47.019 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.010+0000 7f7ed9ffb700 1 -- 172.21.15.143:0/2656960594 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ec40035b0 con 0x7f7edc06dbb0 2023-11-08T19:24:47.019 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.010+0000 7f7eda7fc700 1 --2- 172.21.15.143:0/2656960594 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7edc0668e0 0x7f7edc0cab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:47.020 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.010+0000 7f7edb7fe700 1 --2- 172.21.15.143:0/2656960594 >> v2:172.21.15.174:3300/0 conn(0x7f7edc06df90 0x7f7edc06aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:47.020 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.010+0000 7f7ed9ffb700 1 -- 172.21.15.143:0/2656960594 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 487577675 0 0) 0x7f7ec40035b0 con 0x7f7edc06dbb0 2023-11-08T19:24:47.020 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.010+0000 7f7ed9ffb700 1 -- 172.21.15.143:0/2656960594 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7edc0668e0 msgr2=0x7f7edc0cab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:24:47.020 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.010+0000 7f7ed9ffb700 1 --2- 172.21.15.143:0/2656960594 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7edc0668e0 0x7f7edc0cab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:47.021 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.010+0000 7f7ed9ffb700 1 -- 172.21.15.143:0/2656960594 >> v2:172.21.15.174:3300/0 conn(0x7f7edc06df90 msgr2=0x7f7edc06aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:24:47.021 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.010+0000 7f7ed9ffb700 1 --2- 172.21.15.143:0/2656960594 >> v2:172.21.15.174:3300/0 conn(0x7f7edc06df90 0x7f7edc06aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:47.021 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.010+0000 7f7ed9ffb700 1 -- 172.21.15.143:0/2656960594 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7edc0cb520 con 0x7f7edc06dbb0 2023-11-08T19:24:47.021 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.010+0000 7f7ed9ffb700 1 -- 172.21.15.143:0/2656960594 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4147756303 0 0) 0x7f7ecc001670 con 0x7f7edc06dbb0 2023-11-08T19:24:47.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.010+0000 7f7edb7fe700 1 --2- 172.21.15.143:0/2656960594 >> v2:172.21.15.174:3300/0 conn(0x7f7edc06df90 0x7f7edc06aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:24:47.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.010+0000 7f7ed9ffb700 1 -- 172.21.15.143:0/2656960594 >> v1:172.21.15.143:6789/0 conn(0x7f7edc06dbb0 legacy=0x7f7edc0ca470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:47.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.010+0000 7f7ed9ffb700 1 --2- 172.21.15.143:0/2656960594 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7ec4003a00 0x7f7ec4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:47.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.010+0000 7f7ed9ffb700 1 --2- 172.21.15.143:0/2656960594 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7ec4004850 0x7f7ec4004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:47.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.010+0000 7f7ed9ffb700 1 --2- 172.21.15.143:0/2656960594 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7edc0668e0 0x7f7ec4005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:47.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.010+0000 7f7ed9ffb700 1 -- 172.21.15.143:0/2656960594 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7edc066db0 con 0x7f7ec4004850 2023-11-08T19:24:47.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.010+0000 7f7ed9ffb700 1 -- 172.21.15.143:0/2656960594 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7edc0af5e0 con 0x7f7ec4003a00 2023-11-08T19:24:47.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.010+0000 7f7ed9ffb700 1 -- 172.21.15.143:0/2656960594 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec400baf0 con 0x7f7edc0668e0 2023-11-08T19:24:47.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.010+0000 7f7eda7fc700 1 --2- 172.21.15.143:0/2656960594 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7ec4004850 0x7f7ec4004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:47.024 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.010+0000 7f7edaffd700 1 --2- 172.21.15.143:0/2656960594 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7ec4003a00 0x7f7ec4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:47.024 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.010+0000 7f7edb7fe700 1 --2- 172.21.15.143:0/2656960594 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7edc0668e0 0x7f7ec4005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:47.024 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.011+0000 7f7eda7fc700 1 -- 172.21.15.143:0/2656960594 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7edc0668e0 msgr2=0x7f7ec4005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:24:47.024 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.011+0000 7f7eda7fc700 1 --2- 172.21.15.143:0/2656960594 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7edc0668e0 0x7f7ec4005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:47.025 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.011+0000 7f7eda7fc700 1 -- 172.21.15.143:0/2656960594 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7ec4003a00 msgr2=0x7f7ec4003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:24:47.025 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.011+0000 7f7eda7fc700 1 --2- 172.21.15.143:0/2656960594 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7ec4003a00 0x7f7ec4003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:47.025 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.011+0000 7f7eda7fc700 1 -- 172.21.15.143:0/2656960594 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ec40035b0 con 0x7f7ec4004850 2023-11-08T19:24:47.025 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.011+0000 7f7edaffd700 1 --2- 172.21.15.143:0/2656960594 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7ec4003a00 0x7f7ec4003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T19:24:47.025 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.011+0000 7f7eda7fc700 1 --2- 172.21.15.143:0/2656960594 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7ec4004850 0x7f7ec4004cd0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f7edc066d00 tx=0x7f7ec4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b20a1bbca32b44b server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:47.026 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.011+0000 7f7ed9ffb700 1 -- 172.21.15.143:0/2656960594 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ed0009e70 con 0x7f7ec4004850 2023-11-08T19:24:47.026 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.011+0000 7f7ed9ffb700 1 -- 172.21.15.143:0/2656960594 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ed000e6c0 con 0x7f7ec4004850 2023-11-08T19:24:47.026 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.011+0000 7f7ed9ffb700 1 -- 172.21.15.143:0/2656960594 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ed000ea30 con 0x7f7ec4004850 2023-11-08T19:24:47.026 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.011+0000 7f7ee19d8700 1 -- 172.21.15.143:0/2656960594 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7ec4004850 msgr2=0x7f7ec4004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:47.026 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.011+0000 7f7ee19d8700 1 --2- 172.21.15.143:0/2656960594 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7ec4004850 0x7f7ec4004cd0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f7edc066d00 tx=0x7f7ec4005880 comp rx=0 tx=0).stop 2023-11-08T19:24:47.027 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.011+0000 7f7edb7fe700 1 -- 172.21.15.143:0/2656960594 reap_dead start 2023-11-08T19:24:47.027 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.011+0000 7f7ee19d8700 1 -- 172.21.15.143:0/2656960594 shutdown_connections 2023-11-08T19:24:47.027 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.011+0000 7f7ee19d8700 1 -- 172.21.15.143:0/2656960594 >> 172.21.15.143:0/2656960594 conn(0x7f7edc0b1a20 msgr2=0x7f7edc0b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:24:47.027 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.011+0000 7f7ee19d8700 1 -- 172.21.15.143:0/2656960594 shutdown_connections 2023-11-08T19:24:47.027 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.011+0000 7f7ee19d8700 1 -- 172.21.15.143:0/2656960594 wait complete. 2023-11-08T19:24:47.027 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.013+0000 7f7ee19d8700 1 Processor -- start 2023-11-08T19:24:47.028 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.013+0000 7f7ee19d8700 1 -- start start 2023-11-08T19:24:47.028 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.014+0000 7f7ee19d8700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7ec4004850 0x7f7edc158f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:47.028 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.014+0000 7f7ee19d8700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7edc156710 0x7f7edc06aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:47.028 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.014+0000 7f7ee19d8700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7edc06b530 0x7f7edc0c56d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:47.028 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.014+0000 7f7ee19d8700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7edc06b9b0 con 0x7f7edc156710 2023-11-08T19:24:47.029 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.014+0000 7f7ee19d8700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7edc159440 con 0x7f7edc06b530 2023-11-08T19:24:47.029 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.014+0000 7f7ee19d8700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7edc0c6a00 con 0x7f7ec4004850 2023-11-08T19:24:47.029 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.014+0000 7f7eda7fc700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7edc156710 0x7f7edc06aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:47.029 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.014+0000 7f7eda7fc700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7edc156710 0x7f7edc06aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48488/0 (socket says 172.21.15.143:48488) 2023-11-08T19:24:47.029 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.014+0000 7f7eda7fc700 1 -- 172.21.15.143:0/2529018961 learned_addr learned my addr 172.21.15.143:0/2529018961 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:24:47.030 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.014+0000 7f7edaffd700 1 --2- 172.21.15.143:0/2529018961 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7ec4004850 0x7f7edc158f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:47.030 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.014+0000 7f7edb7fe700 1 --2- 172.21.15.143:0/2529018961 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7edc06b530 0x7f7edc0c56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:47.030 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.014+0000 7f7eda7fc700 1 -- 172.21.15.143:0/2529018961 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7ec4004850 msgr2=0x7f7edc158f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:47.030 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.014+0000 7f7eda7fc700 1 --2- 172.21.15.143:0/2529018961 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7ec4004850 0x7f7edc158f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:47.030 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.014+0000 7f7eda7fc700 1 -- 172.21.15.143:0/2529018961 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7edc06b530 msgr2=0x7f7edc0c56d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:47.031 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.014+0000 7f7eda7fc700 1 --2- 172.21.15.143:0/2529018961 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7edc06b530 0x7f7edc0c56d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:47.031 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.014+0000 7f7eda7fc700 1 -- 172.21.15.143:0/2529018961 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7edc0c6440 con 0x7f7edc156710 2023-11-08T19:24:47.031 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.014+0000 7f7edaffd700 1 --2- 172.21.15.143:0/2529018961 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7ec4004850 0x7f7edc158f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:24:47.031 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.014+0000 7f7eda7fc700 1 --2- 172.21.15.143:0/2529018961 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7edc156710 0x7f7edc06aff0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7ed00090b0 tx=0x7f7ed00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:47.031 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.014+0000 7f7ec3fff700 1 -- 172.21.15.143:0/2529018961 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ed0003ab0 con 0x7f7edc156710 2023-11-08T19:24:47.032 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.014+0000 7f7ec3fff700 1 -- 172.21.15.143:0/2529018961 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ed0003c50 con 0x7f7edc156710 2023-11-08T19:24:47.032 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.014+0000 7f7ec3fff700 1 -- 172.21.15.143:0/2529018961 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ed0009e70 con 0x7f7edc156710 2023-11-08T19:24:47.032 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.014+0000 7f7ee19d8700 1 -- 172.21.15.143:0/2529018961 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7edc0c6720 con 0x7f7edc156710 2023-11-08T19:24:47.032 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.014+0000 7f7ee19d8700 1 -- 172.21.15.143:0/2529018961 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7edc0c3cd0 con 0x7f7edc156710 2023-11-08T19:24:47.032 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.015+0000 7f7ee19d8700 1 -- 172.21.15.143:0/2529018961 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7edc0c4040 con 0x7f7edc156710 2023-11-08T19:24:47.033 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.016+0000 7f7ec3fff700 1 -- 172.21.15.143:0/2529018961 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 4) v1 ==== 89175+0+0 (secure 0 0 0) 0x7f7ed0013470 con 0x7f7edc156710 2023-11-08T19:24:47.033 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.016+0000 7f7ec3fff700 1 --2- 172.21.15.143:0/2529018961 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f7ec806ae70 0x7f7ec806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:47.033 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.016+0000 7f7ec3fff700 1 -- 172.21.15.143:0/2529018961 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f7ed008ab40 con 0x7f7edc156710 2023-11-08T19:24:47.033 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.016+0000 7f7edaffd700 1 --2- 172.21.15.143:0/2529018961 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f7ec806ae70 0x7f7ec806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:47.034 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.016+0000 7f7edaffd700 1 --2- 172.21.15.143:0/2529018961 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f7ec806ae70 0x7f7ec806d330 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc000fb0 tx=0x7f7ecc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:47.034 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.020+0000 7f7ec3fff700 1 -- 172.21.15.143:0/2529018961 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7ed00561d0 con 0x7f7edc156710 2023-11-08T19:24:47.212 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.210+0000 7f7ee19d8700 1 -- 172.21.15.143:0/2529018961 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7edc06d1a0 con 0x7f7edc156710 2023-11-08T19:24:47.212 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.211+0000 7f7ec3fff700 1 -- 172.21.15.143:0/2529018961 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12898 (secure 0 0 0) 0x7f7ed00597f0 con 0x7f7edc156710 2023-11-08T19:24:47.213 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:24:47.213 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":16,"fsid":"968483af-77fc-4068-ad7c-5c2a1a4613d7","created":"2023-11-08T19:24:32.826484+0000","modified":"2023-11-08T19:24:46.578565+0000","last_up_change":"2023-11-08T19:24:43.573880+0000","last_in_change":"2023-11-08T19:24:35.314525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:44.551530+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bae98ec1-d94e-403d-a220-a522a53ea6f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6809","nonce":1487830566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6811","nonce":1487830566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6815","nonce":1487830566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6813","nonce":1487830566}]},"public_addr":"172.21.15.143:6809/1487830566","cluster_addr":"172.21.15.143:6811/1487830566","heartbeat_back_addr":"172.21.15.143:6815/1487830566","heartbeat_front_addr":"172.21.15.143:6813/1487830566","state":["exists","up"]},{"osd":1,"uuid":"5cb6538a-1ccb-4322-8633-09f66af0b7e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6801","nonce":787341397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6803","nonce":787341397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6807","nonce":787341397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6805","nonce":787341397}]},"public_addr":"172.21.15.143:6801/787341397","cluster_addr":"172.21.15.143:6803/787341397","heartbeat_back_addr":"172.21.15.143:6807/787341397","heartbeat_front_addr":"172.21.15.143:6805/787341397","state":["exists","up"]},{"osd":2,"uuid":"cdbf522c-60ef-4d60-9653-8dcb529c101a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6822","nonce":2301325597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6826","nonce":2301325597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6831","nonce":2301325597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6829","nonce":2301325597}]},"public_addr":"172.21.15.143:6822/2301325597","cluster_addr":"172.21.15.143:6826/2301325597","heartbeat_back_addr":"172.21.15.143:6831/2301325597","heartbeat_front_addr":"172.21.15.143:6829/2301325597","state":["exists","up"]},{"osd":3,"uuid":"4e2783cf-ebb9-4dd8-8245-bc1d17ce4325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6817","nonce":2021504428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6819","nonce":2021504428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6827","nonce":2021504428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6823","nonce":2021504428}]},"public_addr":"172.21.15.143:6817/2021504428","cluster_addr":"172.21.15.143:6819/2021504428","heartbeat_back_addr":"172.21.15.143:6827/2021504428","heartbeat_front_addr":"172.21.15.143:6823/2021504428","state":["exists","up"]},{"osd":4,"uuid":"691256fa-9b77-49b9-a346-7ae1adf0ca80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6801","nonce":1993380595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6803","nonce":1993380595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6807","nonce":1993380595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6805","nonce":1993380595}]},"public_addr":"172.21.15.174:6801/1993380595","cluster_addr":"172.21.15.174:6803/1993380595","heartbeat_back_addr":"172.21.15.174:6807/1993380595","heartbeat_front_addr":"172.21.15.174:6805/1993380595","state":["exists","up"]},{"osd":5,"uuid":"756fc352-48c9-46ce-b52e-0541fe6f8525","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6809","nonce":1777923303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6811","nonce":1777923303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6815","nonce":1777923303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6813","nonce":1777923303}]},"public_addr":"172.21.15.174:6809/1777923303","cluster_addr":"172.21.15.174:6811/1777923303","heartbeat_back_addr":"172.21.15.174:6815/1777923303","heartbeat_front_addr":"172.21.15.174:6813/1777923303","state":["exists","up"]},{"osd":6,"uuid":"fab2ac2b-ed97-4cde-9825-b480eabf8b8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6825","nonce":2755130316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6827","nonce":2755130316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6831","nonce":2755130316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6829","nonce":2755130316}]},"public_addr":"172.21.15.174:6825/2755130316","cluster_addr":"172.21.15.174:6827/2755130316","heartbeat_back_addr":"172.21.15.174:6831/2755130316","heartbeat_front_addr":"172.21.15.174:6829/2755130316","state":["exists","up"]},{"osd":7,"uuid":"3de9b797-2ab6-4d41-a863-7e7d638fd703","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6817","nonce":2847141411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6819","nonce":2847141411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6823","nonce":2847141411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6821","nonce":2847141411}]},"public_addr":"172.21.15.174:6817/2847141411","cluster_addr":"172.21.15.174:6819/2847141411","heartbeat_back_addr":"172.21.15.174:6823/2847141411","heartbeat_front_addr":"172.21.15.174:6821/2847141411","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.477076+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.496890+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.539305+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.454673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.538881+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.521294+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.520712+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.540404+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:24:47.215 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.213+0000 7f7ee19d8700 1 -- 172.21.15.143:0/2529018961 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f7ec806ae70 msgr2=0x7f7ec806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:47.215 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.213+0000 7f7ee19d8700 1 --2- 172.21.15.143:0/2529018961 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f7ec806ae70 0x7f7ec806d330 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc000fb0 tx=0x7f7ecc009040 comp rx=0 tx=0).stop 2023-11-08T19:24:47.216 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.213+0000 7f7ee19d8700 1 -- 172.21.15.143:0/2529018961 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7edc156710 msgr2=0x7f7edc06aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:47.216 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.213+0000 7f7ee19d8700 1 --2- 172.21.15.143:0/2529018961 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7edc156710 0x7f7edc06aff0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7ed00090b0 tx=0x7f7ed00047c0 comp rx=0 tx=0).stop 2023-11-08T19:24:47.217 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.214+0000 7f7ee19d8700 1 -- 172.21.15.143:0/2529018961 shutdown_connections 2023-11-08T19:24:47.217 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.214+0000 7f7ee19d8700 1 --2- 172.21.15.143:0/2529018961 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f7ec806ae70 0x7f7ec806d330 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:47.217 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.214+0000 7f7ee19d8700 1 --2- 172.21.15.143:0/2529018961 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7ec4004850 0x7f7edc158f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:47.217 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.214+0000 7f7ee19d8700 1 --2- 172.21.15.143:0/2529018961 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7edc06b530 0x7f7edc0c56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:47.217 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.214+0000 7f7ee19d8700 1 --2- 172.21.15.143:0/2529018961 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7edc156710 0x7f7edc06aff0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:47.218 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.214+0000 7f7ee19d8700 1 -- 172.21.15.143:0/2529018961 >> 172.21.15.143:0/2529018961 conn(0x7f7edc0b1a20 msgr2=0x7f7edc06bf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:24:47.218 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.214+0000 7f7ee19d8700 1 -- 172.21.15.143:0/2529018961 shutdown_connections 2023-11-08T19:24:47.218 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.214+0000 7f7ee19d8700 1 -- 172.21.15.143:0/2529018961 wait complete. 2023-11-08T19:24:47.227 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T19:24:44.551530+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-08T19:24:47.228 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T19:24:47.351 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.349+0000 7fea8d542700 1 Processor -- start 2023-11-08T19:24:47.351 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.349+0000 7fea8d542700 1 -- start start 2023-11-08T19:24:47.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.350+0000 7fea8d542700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fea8815bd10 0x7fea880d0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:47.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.350+0000 7fea8d542700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fea880b0800 0x7fea880b0c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:47.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.350+0000 7fea8d542700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea880d0580 con 0x7fea880cc1f0 2023-11-08T19:24:47.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.350+0000 7fea8d542700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea880cceb0 con 0x7fea880b0800 2023-11-08T19:24:47.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.350+0000 7fea8d542700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea880b11c0 con 0x7fea8815bd10 2023-11-08T19:24:47.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.350+0000 7fea86ffd700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fea8815bd10 0x7fea880d0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:47.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.350+0000 7fea867fc700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fea880b0800 0x7fea880b0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:47.356 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.350+0000 7fea877fe700 1 --1- >> v1:172.21.15.174:6789/0 conn(0x7fea880cc1f0 0x7fea880cc680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.174:6789/0 says I am v1:172.21.15.143:55584/0 (socket says 172.21.15.143:55584) 2023-11-08T19:24:47.356 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.350+0000 7fea86ffd700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fea8815bd10 0x7fea880d0a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.143:36132/0 (socket says 172.21.15.143:36132) 2023-11-08T19:24:47.356 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.350+0000 7fea877fe700 1 -- 172.21.15.143:0/3062912060 learned_addr learned my addr 172.21.15.143:0/3062912060 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:24:47.356 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.350+0000 7fea867fc700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fea880b0800 0x7fea880b0c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.143:60224/0 (socket says 172.21.15.143:60224) 2023-11-08T19:24:47.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.351+0000 7fea867fc700 1 -- 172.21.15.143:0/3062912060 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fea8815bd10 msgr2=0x7fea880d0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:24:47.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.351+0000 7fea867fc700 1 --2- 172.21.15.143:0/3062912060 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fea8815bd10 0x7fea880d0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:47.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.351+0000 7fea867fc700 1 -- 172.21.15.143:0/3062912060 >> v1:172.21.15.174:6789/0 conn(0x7fea880cc1f0 legacy=0x7fea880cc680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:47.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.351+0000 7fea85ffb700 1 -- 172.21.15.143:0/3062912060 <== mon.1 v1:172.21.15.174:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1656041976 0 0) 0x7fea880d0580 con 0x7fea880cc1f0 2023-11-08T19:24:47.358 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.351+0000 7fea867fc700 1 -- 172.21.15.143:0/3062912060 --> v2:172.21.15.174:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea880cfc50 con 0x7fea880b0800 2023-11-08T19:24:47.358 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.351+0000 7fea867fc700 1 --2- 172.21.15.143:0/3062912060 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fea880b0800 0x7fea880b0c80 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fea78009840 tx=0x7fea78009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f430fef7a8ef7f server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:47.358 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.351+0000 7fea85ffb700 1 -- 172.21.15.143:0/3062912060 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea78011830 con 0x7fea880b0800 2023-11-08T19:24:47.358 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.351+0000 7fea85ffb700 1 -- 172.21.15.143:0/3062912060 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea78011990 con 0x7fea880b0800 2023-11-08T19:24:47.358 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.351+0000 7fea85ffb700 1 -- 172.21.15.143:0/3062912060 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea78011d20 con 0x7fea880b0800 2023-11-08T19:24:47.359 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.351+0000 7fea8d542700 1 -- 172.21.15.143:0/3062912060 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fea880b0800 msgr2=0x7fea880b0c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:47.359 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.351+0000 7fea8d542700 1 --2- 172.21.15.143:0/3062912060 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fea880b0800 0x7fea880b0c80 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fea78009840 tx=0x7fea78009b50 comp rx=0 tx=0).stop 2023-11-08T19:24:47.359 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.352+0000 7fea8d542700 1 -- 172.21.15.143:0/3062912060 shutdown_connections 2023-11-08T19:24:47.359 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.352+0000 7fea8d542700 1 --2- 172.21.15.143:0/3062912060 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fea8815bd10 0x7fea880d0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:47.359 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.352+0000 7fea8d542700 1 --2- 172.21.15.143:0/3062912060 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fea880b0800 0x7fea880b0c80 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:47.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.352+0000 7fea8d542700 1 -- 172.21.15.143:0/3062912060 >> 172.21.15.143:0/3062912060 conn(0x7fea8806f2e0 msgr2=0x7fea8806f6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:24:47.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.352+0000 7fea8d542700 1 -- 172.21.15.143:0/3062912060 shutdown_connections 2023-11-08T19:24:47.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.352+0000 7fea8d542700 1 -- 172.21.15.143:0/3062912060 wait complete. 2023-11-08T19:24:47.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.353+0000 7fea8d542700 1 Processor -- start 2023-11-08T19:24:47.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.353+0000 7fea8d542700 1 -- start start 2023-11-08T19:24:47.364 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.354+0000 7fea8d542700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fea880b0800 0x7fea881509f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:47.364 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.354+0000 7fea8d542700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fea880cc1f0 0x7fea880c7300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:47.364 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.354+0000 7fea8d542700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fea8815bd10 0x7fea88159970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:47.364 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.354+0000 7fea8d542700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea881512f0 con 0x7fea880cc1f0 2023-11-08T19:24:47.364 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.354+0000 7fea8d542700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea8815ca20 con 0x7fea880b0800 2023-11-08T19:24:47.365 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.354+0000 7fea8d542700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea880c79e0 con 0x7fea8815bd10 2023-11-08T19:24:47.365 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.354+0000 7fea867fc700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fea880cc1f0 0x7fea880c7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:47.365 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.354+0000 7fea867fc700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fea880cc1f0 0x7fea880c7300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48500/0 (socket says 172.21.15.143:48500) 2023-11-08T19:24:47.365 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.354+0000 7fea867fc700 1 -- 172.21.15.143:0/3617515619 learned_addr learned my addr 172.21.15.143:0/3617515619 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:24:47.366 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.354+0000 7fea877fe700 1 --2- 172.21.15.143:0/3617515619 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fea8815bd10 0x7fea88159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:47.366 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.354+0000 7fea86ffd700 1 --2- 172.21.15.143:0/3617515619 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fea880b0800 0x7fea881509f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:47.366 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.354+0000 7fea867fc700 1 -- 172.21.15.143:0/3617515619 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fea8815bd10 msgr2=0x7fea88159970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:47.366 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.354+0000 7fea867fc700 1 --2- 172.21.15.143:0/3617515619 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fea8815bd10 0x7fea88159970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:47.366 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.354+0000 7fea867fc700 1 -- 172.21.15.143:0/3617515619 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fea880b0800 msgr2=0x7fea881509f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:47.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.354+0000 7fea867fc700 1 --2- 172.21.15.143:0/3617515619 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fea880b0800 0x7fea881509f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:47.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.354+0000 7fea867fc700 1 -- 172.21.15.143:0/3617515619 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea780094a0 con 0x7fea880cc1f0 2023-11-08T19:24:47.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.354+0000 7fea867fc700 1 --2- 172.21.15.143:0/3617515619 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fea880cc1f0 0x7fea880c7300 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fea780090b0 tx=0x7fea78004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:47.368 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.354+0000 7fea6f7fe700 1 -- 172.21.15.143:0/3617515619 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea780045d0 con 0x7fea880cc1f0 2023-11-08T19:24:47.368 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.354+0000 7fea6f7fe700 1 -- 172.21.15.143:0/3617515619 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea7801e070 con 0x7fea880cc1f0 2023-11-08T19:24:47.369 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.354+0000 7fea8d542700 1 -- 172.21.15.143:0/3617515619 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea8815c0f0 con 0x7fea880cc1f0 2023-11-08T19:24:47.369 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.354+0000 7fea6f7fe700 1 -- 172.21.15.143:0/3617515619 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea780038c0 con 0x7fea880cc1f0 2023-11-08T19:24:47.369 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.355+0000 7fea8d542700 1 -- 172.21.15.143:0/3617515619 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea8815c6b0 con 0x7fea880cc1f0 2023-11-08T19:24:47.370 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.355+0000 7fea8d542700 1 -- 172.21.15.143:0/3617515619 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea8804eee0 con 0x7fea880cc1f0 2023-11-08T19:24:47.370 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.356+0000 7fea6f7fe700 1 -- 172.21.15.143:0/3617515619 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 4) v1 ==== 89175+0+0 (secure 0 0 0) 0x7fea78003a20 con 0x7fea880cc1f0 2023-11-08T19:24:47.371 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.356+0000 7fea6f7fe700 1 --2- 172.21.15.143:0/3617515619 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fea7406ae70 0x7fea7406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:47.371 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.356+0000 7fea6f7fe700 1 -- 172.21.15.143:0/3617515619 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fea78089ff0 con 0x7fea880cc1f0 2023-11-08T19:24:47.371 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.357+0000 7fea86ffd700 1 --2- 172.21.15.143:0/3617515619 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fea7406ae70 0x7fea7406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:47.372 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.357+0000 7fea86ffd700 1 --2- 172.21.15.143:0/3617515619 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fea7406ae70 0x7fea7406d330 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fea881e65d0 tx=0x7fea7c009210 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:47.372 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.361+0000 7fea6f7fe700 1 -- 172.21.15.143:0/3617515619 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fea780bcca0 con 0x7fea880cc1f0 2023-11-08T19:24:47.482 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.481+0000 7fea6f7fe700 1 -- 172.21.15.143:0/3617515619 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fea78003cd0 con 0x7fea880cc1f0 2023-11-08T19:24:47.557 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.555+0000 7fea8d542700 1 -- 172.21.15.143:0/3617515619 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fea880c8790 con 0x7fea880cc1f0 2023-11-08T19:24:47.558 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.557+0000 7fea6f7fe700 1 -- 172.21.15.143:0/3617515619 <== mon.0 v2:172.21.15.143:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7fea7806f680 con 0x7fea880cc1f0 2023-11-08T19:24:47.559 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2023-11-08T19:24:47.561 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.559+0000 7fea8d542700 1 -- 172.21.15.143:0/3617515619 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fea7406ae70 msgr2=0x7fea7406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:47.561 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.559+0000 7fea8d542700 1 --2- 172.21.15.143:0/3617515619 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fea7406ae70 0x7fea7406d330 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fea881e65d0 tx=0x7fea7c009210 comp rx=0 tx=0).stop 2023-11-08T19:24:47.561 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.559+0000 7fea8d542700 1 -- 172.21.15.143:0/3617515619 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fea880cc1f0 msgr2=0x7fea880c7300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:47.562 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.559+0000 7fea8d542700 1 --2- 172.21.15.143:0/3617515619 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fea880cc1f0 0x7fea880c7300 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fea780090b0 tx=0x7fea78004750 comp rx=0 tx=0).stop 2023-11-08T19:24:47.562 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.559+0000 7fea8d542700 1 -- 172.21.15.143:0/3617515619 shutdown_connections 2023-11-08T19:24:47.562 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.559+0000 7fea8d542700 1 --2- 172.21.15.143:0/3617515619 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fea7406ae70 0x7fea7406d330 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:47.562 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.559+0000 7fea8d542700 1 --2- 172.21.15.143:0/3617515619 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fea8815bd10 0x7fea88159970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:47.562 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.559+0000 7fea8d542700 1 --2- 172.21.15.143:0/3617515619 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fea880b0800 0x7fea881509f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:47.563 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.559+0000 7fea8d542700 1 --2- 172.21.15.143:0/3617515619 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fea880cc1f0 0x7fea880c7300 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:47.563 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.559+0000 7fea8d542700 1 -- 172.21.15.143:0/3617515619 >> 172.21.15.143:0/3617515619 conn(0x7fea8806f2e0 msgr2=0x7fea880b1300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:24:47.563 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.559+0000 7fea8d542700 1 -- 172.21.15.143:0/3617515619 shutdown_connections 2023-11-08T19:24:47.563 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.559+0000 7fea8d542700 1 -- 172.21.15.143:0/3617515619 wait complete. 2023-11-08T19:24:47.573 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-08T19:24:47.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.691+0000 7fb275973700 1 Processor -- start 2023-11-08T19:24:47.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.691+0000 7fb275973700 1 -- start start 2023-11-08T19:24:47.695 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.692+0000 7fb275973700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb2700cf090 0x7fb2700d1f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:47.695 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.692+0000 7fb275973700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb2700d3690 0x7fb2700b9690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:47.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.692+0000 7fb275973700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2700d3200 con 0x7fb27015dc80 2023-11-08T19:24:47.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.692+0000 7fb275973700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2700b9bd0 con 0x7fb2700d3690 2023-11-08T19:24:47.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.692+0000 7fb275973700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2700b7cf0 con 0x7fb2700cf090 2023-11-08T19:24:47.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.692+0000 7fb26f7fe700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb2700d3690 0x7fb2700b9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:47.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.692+0000 7fb26effd700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fb27015dc80 0x7fb2700ce980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:44660/0 (socket says 172.21.15.143:44660) 2023-11-08T19:24:47.697 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.692+0000 7fb26effd700 1 -- 172.21.15.143:0/2702859835 learned_addr learned my addr 172.21.15.143:0/2702859835 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:24:47.697 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.692+0000 7fb26e7fc700 1 --2- 172.21.15.143:0/2702859835 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb2700cf090 0x7fb2700d1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:47.697 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.692+0000 7fb26f7fe700 1 -- 172.21.15.143:0/2702859835 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb2700cf090 msgr2=0x7fb2700d1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:24:47.697 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.692+0000 7fb26f7fe700 1 --2- 172.21.15.143:0/2702859835 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb2700cf090 0x7fb2700d1f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:47.697 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.692+0000 7fb26f7fe700 1 -- 172.21.15.143:0/2702859835 >> v1:172.21.15.143:6789/0 conn(0x7fb27015dc80 legacy=0x7fb2700ce980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:47.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.692+0000 7fb26f7fe700 1 -- 172.21.15.143:0/2702859835 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb27015e060 con 0x7fb2700d3690 2023-11-08T19:24:47.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.692+0000 7fb26dffb700 1 -- 172.21.15.143:0/2702859835 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3108715327 0 0) 0x7fb2700d3200 con 0x7fb27015dc80 2023-11-08T19:24:47.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.693+0000 7fb26f7fe700 1 --2- 172.21.15.143:0/2702859835 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb2700d3690 0x7fb2700b9690 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb268009d90 tx=0x7fb26800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25a9ac94365c48b6 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:47.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.693+0000 7fb26dffb700 1 -- 172.21.15.143:0/2702859835 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb26800e070 con 0x7fb2700d3690 2023-11-08T19:24:47.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.693+0000 7fb26dffb700 1 -- 172.21.15.143:0/2702859835 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb26800bb20 con 0x7fb2700d3690 2023-11-08T19:24:47.699 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.693+0000 7fb26dffb700 1 -- 172.21.15.143:0/2702859835 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb268012430 con 0x7fb2700d3690 2023-11-08T19:24:47.699 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.693+0000 7fb275973700 1 -- 172.21.15.143:0/2702859835 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb2700d3690 msgr2=0x7fb2700b9690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:47.699 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.693+0000 7fb275973700 1 --2- 172.21.15.143:0/2702859835 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb2700d3690 0x7fb2700b9690 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb268009d90 tx=0x7fb26800b330 comp rx=0 tx=0).stop 2023-11-08T19:24:47.699 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.693+0000 7fb275973700 1 -- 172.21.15.143:0/2702859835 shutdown_connections 2023-11-08T19:24:47.699 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.693+0000 7fb275973700 1 --2- 172.21.15.143:0/2702859835 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb2700d3690 0x7fb2700b9690 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:47.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.693+0000 7fb275973700 1 --2- 172.21.15.143:0/2702859835 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb2700cf090 0x7fb2700d1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:47.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.693+0000 7fb275973700 1 -- 172.21.15.143:0/2702859835 >> 172.21.15.143:0/2702859835 conn(0x7fb2700668e0 msgr2=0x7fb270066cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:24:47.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.693+0000 7fb275973700 1 -- 172.21.15.143:0/2702859835 shutdown_connections 2023-11-08T19:24:47.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.693+0000 7fb275973700 1 -- 172.21.15.143:0/2702859835 wait complete. 2023-11-08T19:24:47.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.694+0000 7fb275973700 1 Processor -- start 2023-11-08T19:24:47.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.695+0000 7fb275973700 1 -- start start 2023-11-08T19:24:47.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.695+0000 7fb275973700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb2700cf090 0x7fb2700cdac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:47.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.695+0000 7fb275973700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb2700d3690 0x7fb2701f3040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:47.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.695+0000 7fb275973700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb27015dc80 0x7fb270157140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:47.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.695+0000 7fb275973700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2700cd370 con 0x7fb27015dc80 2023-11-08T19:24:47.705 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.695+0000 7fb275973700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2700caeb0 con 0x7fb2700d3690 2023-11-08T19:24:47.705 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.695+0000 7fb275973700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2700cb5a0 con 0x7fb2700cf090 2023-11-08T19:24:47.705 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.696+0000 7fb26f7fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb27015dc80 0x7fb270157140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:47.705 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.696+0000 7fb26f7fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb27015dc80 0x7fb270157140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48516/0 (socket says 172.21.15.143:48516) 2023-11-08T19:24:47.705 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.696+0000 7fb26f7fe700 1 -- 172.21.15.143:0/443183468 learned_addr learned my addr 172.21.15.143:0/443183468 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:24:47.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.696+0000 7fb26e7fc700 1 --2- 172.21.15.143:0/443183468 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb2700d3690 0x7fb2701f3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:47.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.696+0000 7fb26effd700 1 --2- 172.21.15.143:0/443183468 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb2700cf090 0x7fb2700cdac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:47.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.696+0000 7fb26f7fe700 1 -- 172.21.15.143:0/443183468 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb2700cf090 msgr2=0x7fb2700cdac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:47.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.696+0000 7fb26f7fe700 1 --2- 172.21.15.143:0/443183468 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb2700cf090 0x7fb2700cdac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:47.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.696+0000 7fb26f7fe700 1 -- 172.21.15.143:0/443183468 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb2700d3690 msgr2=0x7fb2701f3040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:47.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.696+0000 7fb26f7fe700 1 --2- 172.21.15.143:0/443183468 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb2700d3690 0x7fb2701f3040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:47.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.696+0000 7fb26f7fe700 1 -- 172.21.15.143:0/443183468 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb268009a70 con 0x7fb27015dc80 2023-11-08T19:24:47.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.696+0000 7fb26e7fc700 1 --2- 172.21.15.143:0/443183468 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb2700d3690 0x7fb2701f3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:24:47.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.696+0000 7fb26f7fe700 1 --2- 172.21.15.143:0/443183468 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb27015dc80 0x7fb270157140 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb268013000 tx=0x7fb268004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:47.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.696+0000 7fb257fff700 1 -- 172.21.15.143:0/443183468 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb268012430 con 0x7fb27015dc80 2023-11-08T19:24:47.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.696+0000 7fb257fff700 1 -- 172.21.15.143:0/443183468 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb268020070 con 0x7fb27015dc80 2023-11-08T19:24:47.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.696+0000 7fb275973700 1 -- 172.21.15.143:0/443183468 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2700cb880 con 0x7fb27015dc80 2023-11-08T19:24:47.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.696+0000 7fb257fff700 1 -- 172.21.15.143:0/443183468 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb268004280 con 0x7fb27015dc80 2023-11-08T19:24:47.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.696+0000 7fb275973700 1 -- 172.21.15.143:0/443183468 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2700cd5a0 con 0x7fb27015dc80 2023-11-08T19:24:47.709 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.697+0000 7fb275973700 1 -- 172.21.15.143:0/443183468 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb27004eee0 con 0x7fb27015dc80 2023-11-08T19:24:47.709 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.697+0000 7fb257fff700 1 -- 172.21.15.143:0/443183468 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb2680043e0 con 0x7fb27015dc80 2023-11-08T19:24:47.709 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.698+0000 7fb257fff700 1 --2- 172.21.15.143:0/443183468 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fb25c06af00 0x7fb25c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:47.709 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.698+0000 7fb257fff700 1 -- 172.21.15.143:0/443183468 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fb268022070 con 0x7fb27015dc80 2023-11-08T19:24:47.709 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.698+0000 7fb26effd700 1 --2- 172.21.15.143:0/443183468 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fb25c06af00 0x7fb25c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:47.710 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.698+0000 7fb26effd700 1 --2- 172.21.15.143:0/443183468 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fb25c06af00 0x7fb25c06d3c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb264000f60 tx=0x7fb264007c90 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:47.710 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.702+0000 7fb257fff700 1 -- 172.21.15.143:0/443183468 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb268056780 con 0x7fb27015dc80 2023-11-08T19:24:47.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.898+0000 7fb275973700 1 -- 172.21.15.143:0/443183468 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fb2700d20d0 con 0x7fb27015dc80 2023-11-08T19:24:47.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.898+0000 7fb257fff700 1 -- 172.21.15.143:0/443183468 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) v1 ==== 84+0+65 (secure 0 0 0) 0x7fb26801ae60 con 0x7fb27015dc80 2023-11-08T19:24:47.900 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:24:47.901 INFO:teuthology.orchestra.run.smithi143.stdout:[ 2023-11-08T19:24:47.901 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-08T19:24:47.901 INFO:teuthology.orchestra.run.smithi143.stdout: "poolnum": 1, 2023-11-08T19:24:47.901 INFO:teuthology.orchestra.run.smithi143.stdout: "poolname": ".mgr" 2023-11-08T19:24:47.901 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-08T19:24:47.902 INFO:teuthology.orchestra.run.smithi143.stdout:] 2023-11-08T19:24:47.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.900+0000 7fb275973700 1 -- 172.21.15.143:0/443183468 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fb25c06af00 msgr2=0x7fb25c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:47.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.900+0000 7fb275973700 1 --2- 172.21.15.143:0/443183468 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fb25c06af00 0x7fb25c06d3c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb264000f60 tx=0x7fb264007c90 comp rx=0 tx=0).stop 2023-11-08T19:24:47.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.900+0000 7fb275973700 1 -- 172.21.15.143:0/443183468 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb27015dc80 msgr2=0x7fb270157140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:47.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.900+0000 7fb275973700 1 --2- 172.21.15.143:0/443183468 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb27015dc80 0x7fb270157140 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb268013000 tx=0x7fb268004ad0 comp rx=0 tx=0).stop 2023-11-08T19:24:47.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.901+0000 7fb275973700 1 -- 172.21.15.143:0/443183468 shutdown_connections 2023-11-08T19:24:47.904 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.901+0000 7fb275973700 1 --2- 172.21.15.143:0/443183468 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fb25c06af00 0x7fb25c06d3c0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:47.904 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.901+0000 7fb275973700 1 --2- 172.21.15.143:0/443183468 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb2700cf090 0x7fb2700cdac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:47.904 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.901+0000 7fb275973700 1 --2- 172.21.15.143:0/443183468 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb27015dc80 0x7fb270157140 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:47.904 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.901+0000 7fb275973700 1 --2- 172.21.15.143:0/443183468 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb2700d3690 0x7fb2701f3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:47.904 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.901+0000 7fb275973700 1 -- 172.21.15.143:0/443183468 >> 172.21.15.143:0/443183468 conn(0x7fb2700668e0 msgr2=0x7fb2700b8a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:24:47.905 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.901+0000 7fb275973700 1 -- 172.21.15.143:0/443183468 shutdown_connections 2023-11-08T19:24:47.905 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:47.901+0000 7fb275973700 1 -- 172.21.15.143:0/443183468 wait complete. 2023-11-08T19:24:47.914 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-08T19:24:47.914 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-08T19:24:48.039 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.038+0000 7fc2759a9700 1 Processor -- start 2023-11-08T19:24:48.040 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.038+0000 7fc2759a9700 1 -- start start 2023-11-08T19:24:48.043 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.039+0000 7fc2759a9700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fc27015dca0 0x7fc2700b93f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:48.043 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.039+0000 7fc2759a9700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc2700b5630 0x7fc2700b5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:48.043 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.039+0000 7fc2759a9700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2700b8f50 con 0x7fc2700b6b70 2023-11-08T19:24:48.044 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.039+0000 7fc2759a9700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2700b6020 con 0x7fc2700b5630 2023-11-08T19:24:48.044 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.039+0000 7fc2759a9700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2700b3360 con 0x7fc27015dca0 2023-11-08T19:24:48.044 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.039+0000 7fc26f7fe700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fc2700b6b70 0x7fc2700b6f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:44678/0 (socket says 172.21.15.143:44678) 2023-11-08T19:24:48.044 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.039+0000 7fc26e7fc700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc2700b5630 0x7fc2700b5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:48.044 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.039+0000 7fc26f7fe700 1 -- 172.21.15.143:0/2375845492 learned_addr learned my addr 172.21.15.143:0/2375845492 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:24:48.045 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.039+0000 7fc26effd700 1 --2- 172.21.15.143:0/2375845492 >> v2:172.21.15.174:3300/0 conn(0x7fc27015dca0 0x7fc2700b93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:48.045 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.039+0000 7fc26dffb700 1 -- 172.21.15.143:0/2375845492 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1861720556 0 0) 0x7fc2700b8f50 con 0x7fc2700b6b70 2023-11-08T19:24:48.045 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.039+0000 7fc26dffb700 1 -- 172.21.15.143:0/2375845492 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc2580035b0 con 0x7fc2700b6b70 2023-11-08T19:24:48.045 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.039+0000 7fc26e7fc700 1 -- 172.21.15.143:0/2375845492 >> v2:172.21.15.174:3300/0 conn(0x7fc27015dca0 msgr2=0x7fc2700b93f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:24:48.046 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.039+0000 7fc26e7fc700 1 --2- 172.21.15.143:0/2375845492 >> v2:172.21.15.174:3300/0 conn(0x7fc27015dca0 0x7fc2700b93f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:48.046 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.039+0000 7fc26e7fc700 1 -- 172.21.15.143:0/2375845492 >> v1:172.21.15.143:6789/0 conn(0x7fc2700b6b70 legacy=0x7fc2700b6f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:48.046 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.039+0000 7fc26e7fc700 1 -- 172.21.15.143:0/2375845492 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2700b7eb0 con 0x7fc2700b5630 2023-11-08T19:24:48.046 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.039+0000 7fc26e7fc700 1 --2- 172.21.15.143:0/2375845492 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc2700b5630 0x7fc2700b5ae0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc2700cf650 tx=0x7fc264009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fef04fb6aa45878a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:48.046 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.039+0000 7fc26dffb700 1 -- 172.21.15.143:0/2375845492 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc264011830 con 0x7fc2700b5630 2023-11-08T19:24:48.047 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.039+0000 7fc26dffb700 1 -- 172.21.15.143:0/2375845492 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc264011990 con 0x7fc2700b5630 2023-11-08T19:24:48.047 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.039+0000 7fc26dffb700 1 -- 172.21.15.143:0/2375845492 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc264011d20 con 0x7fc2700b5630 2023-11-08T19:24:48.047 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.040+0000 7fc2759a9700 1 -- 172.21.15.143:0/2375845492 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc2700b5630 msgr2=0x7fc2700b5ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:48.047 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.040+0000 7fc2759a9700 1 --2- 172.21.15.143:0/2375845492 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc2700b5630 0x7fc2700b5ae0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc2700cf650 tx=0x7fc264009b20 comp rx=0 tx=0).stop 2023-11-08T19:24:48.047 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.040+0000 7fc2759a9700 1 -- 172.21.15.143:0/2375845492 shutdown_connections 2023-11-08T19:24:48.048 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.040+0000 7fc2759a9700 1 --2- 172.21.15.143:0/2375845492 >> v2:172.21.15.174:3300/0 conn(0x7fc27015dca0 0x7fc2700b93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:48.048 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.040+0000 7fc2759a9700 1 --2- 172.21.15.143:0/2375845492 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc2700b5630 0x7fc2700b5ae0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:48.048 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.040+0000 7fc2759a9700 1 -- 172.21.15.143:0/2375845492 >> 172.21.15.143:0/2375845492 conn(0x7fc270066d00 msgr2=0x7fc270067110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:24:48.048 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.040+0000 7fc2759a9700 1 -- 172.21.15.143:0/2375845492 shutdown_connections 2023-11-08T19:24:48.048 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.040+0000 7fc2759a9700 1 -- 172.21.15.143:0/2375845492 wait complete. 2023-11-08T19:24:48.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.041+0000 7fc2759a9700 1 Processor -- start 2023-11-08T19:24:48.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.041+0000 7fc2759a9700 1 -- start start 2023-11-08T19:24:48.052 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.042+0000 7fc2759a9700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc2700b5630 0x7fc270159cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:48.052 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.042+0000 7fc2759a9700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc2700b6b70 0x7fc2700c8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:48.052 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.042+0000 7fc2759a9700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc27015dca0 0x7fc2700c9430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:48.052 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.042+0000 7fc2759a9700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2700cc2a0 con 0x7fc27015dca0 2023-11-08T19:24:48.053 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.042+0000 7fc2759a9700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc27015a230 con 0x7fc2700b5630 2023-11-08T19:24:48.053 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.042+0000 7fc2759a9700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2700cb970 con 0x7fc2700b6b70 2023-11-08T19:24:48.053 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.042+0000 7fc26f7fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc27015dca0 0x7fc2700c9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:48.053 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.042+0000 7fc26f7fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc27015dca0 0x7fc2700c9430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48528/0 (socket says 172.21.15.143:48528) 2023-11-08T19:24:48.053 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.042+0000 7fc26f7fe700 1 -- 172.21.15.143:0/1958986159 learned_addr learned my addr 172.21.15.143:0/1958986159 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:24:48.054 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.042+0000 7fc26effd700 1 --2- 172.21.15.143:0/1958986159 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc2700b5630 0x7fc270159cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:48.054 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.042+0000 7fc26e7fc700 1 --2- 172.21.15.143:0/1958986159 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc2700b6b70 0x7fc2700c8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:48.054 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.042+0000 7fc26f7fe700 1 -- 172.21.15.143:0/1958986159 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc2700b6b70 msgr2=0x7fc2700c8ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:48.054 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.042+0000 7fc26f7fe700 1 --2- 172.21.15.143:0/1958986159 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc2700b6b70 0x7fc2700c8ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:48.055 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.042+0000 7fc26f7fe700 1 -- 172.21.15.143:0/1958986159 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc2700b5630 msgr2=0x7fc270159cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:48.055 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.042+0000 7fc26f7fe700 1 --2- 172.21.15.143:0/1958986159 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc2700b5630 0x7fc270159cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:48.055 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.043+0000 7fc26f7fe700 1 -- 172.21.15.143:0/1958986159 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2640094a0 con 0x7fc27015dca0 2023-11-08T19:24:48.055 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.043+0000 7fc26effd700 1 --2- 172.21.15.143:0/1958986159 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc2700b5630 0x7fc270159cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:24:48.055 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.043+0000 7fc26f7fe700 1 --2- 172.21.15.143:0/1958986159 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc27015dca0 0x7fc2700c9430 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc26800cb40 tx=0x7fc26800cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:48.056 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.043+0000 7fc257fff700 1 -- 172.21.15.143:0/1958986159 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc26800ab50 con 0x7fc27015dca0 2023-11-08T19:24:48.056 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.043+0000 7fc257fff700 1 -- 172.21.15.143:0/1958986159 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc26800acb0 con 0x7fc27015dca0 2023-11-08T19:24:48.056 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.043+0000 7fc2759a9700 1 -- 172.21.15.143:0/1958986159 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2700cbc50 con 0x7fc27015dca0 2023-11-08T19:24:48.056 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.043+0000 7fc257fff700 1 -- 172.21.15.143:0/1958986159 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc268012620 con 0x7fc27015dca0 2023-11-08T19:24:48.056 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.043+0000 7fc2759a9700 1 -- 172.21.15.143:0/1958986159 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2700cb630 con 0x7fc27015dca0 2023-11-08T19:24:48.056 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.044+0000 7fc2759a9700 1 -- 172.21.15.143:0/1958986159 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2700cc630 con 0x7fc27015dca0 2023-11-08T19:24:48.057 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.044+0000 7fc257fff700 1 -- 172.21.15.143:0/1958986159 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc26801b3b0 con 0x7fc27015dca0 2023-11-08T19:24:48.057 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.044+0000 7fc257fff700 1 --2- 172.21.15.143:0/1958986159 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc25c06af00 0x7fc25c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:48.057 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.045+0000 7fc257fff700 1 -- 172.21.15.143:0/1958986159 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fc268089030 con 0x7fc27015dca0 2023-11-08T19:24:48.057 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.045+0000 7fc26effd700 1 --2- 172.21.15.143:0/1958986159 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc25c06af00 0x7fc25c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:48.058 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.045+0000 7fc26effd700 1 --2- 172.21.15.143:0/1958986159 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc25c06af00 0x7fc25c06d3c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc260009b90 tx=0x7fc260008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:48.058 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.049+0000 7fc257fff700 1 -- 172.21.15.143:0/1958986159 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc2680544c0 con 0x7fc27015dca0 2023-11-08T19:24:48.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.238+0000 7fc2759a9700 1 -- 172.21.15.143:0/1958986159 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fc27004eee0 con 0x7fc27015dca0 2023-11-08T19:24:48.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.587+0000 7fc257fff700 1 -- 172.21.15.143:0/1958986159 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) v1 ==== 138+0+0 (secure 0 0 0) 0x7fc268057ae0 con 0x7fc27015dca0 2023-11-08T19:24:48.592 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.590+0000 7fc2759a9700 1 -- 172.21.15.143:0/1958986159 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc25c06af00 msgr2=0x7fc25c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:48.592 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.590+0000 7fc2759a9700 1 --2- 172.21.15.143:0/1958986159 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc25c06af00 0x7fc25c06d3c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc260009b90 tx=0x7fc260008040 comp rx=0 tx=0).stop 2023-11-08T19:24:48.592 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.590+0000 7fc2759a9700 1 -- 172.21.15.143:0/1958986159 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc27015dca0 msgr2=0x7fc2700c9430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:48.592 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.590+0000 7fc2759a9700 1 --2- 172.21.15.143:0/1958986159 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc27015dca0 0x7fc2700c9430 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc26800cb40 tx=0x7fc26800cf00 comp rx=0 tx=0).stop 2023-11-08T19:24:48.593 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.590+0000 7fc2759a9700 1 -- 172.21.15.143:0/1958986159 shutdown_connections 2023-11-08T19:24:48.593 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.590+0000 7fc2759a9700 1 --2- 172.21.15.143:0/1958986159 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc25c06af00 0x7fc25c06d3c0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:48.593 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.590+0000 7fc2759a9700 1 --2- 172.21.15.143:0/1958986159 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc27015dca0 0x7fc2700c9430 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:48.593 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.590+0000 7fc2759a9700 1 --2- 172.21.15.143:0/1958986159 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc2700b5630 0x7fc270159cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:48.593 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.590+0000 7fc2759a9700 1 --2- 172.21.15.143:0/1958986159 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc2700b6b70 0x7fc2700c8ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:48.594 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.590+0000 7fc2759a9700 1 -- 172.21.15.143:0/1958986159 >> 172.21.15.143:0/1958986159 conn(0x7fc270066d00 msgr2=0x7fc2700b6f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:24:48.594 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.590+0000 7fc2759a9700 1 -- 172.21.15.143:0/1958986159 shutdown_connections 2023-11-08T19:24:48.594 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.590+0000 7fc2759a9700 1 -- 172.21.15.143:0/1958986159 wait complete. 2023-11-08T19:24:48.595 INFO:teuthology.orchestra.run.smithi143.stderr:pool 'cephfs_metadata' created 2023-11-08T19:24:48.608 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-08T19:24:48.736 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.734+0000 7fa2783ac700 1 Processor -- start 2023-11-08T19:24:48.736 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.734+0000 7fa2783ac700 1 -- start start 2023-11-08T19:24:48.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.735+0000 7fa2783ac700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa27006bd20 0x7fa2700d01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:48.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.735+0000 7fa2783ac700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fa2700b61e0 0x7fa2700cc730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:48.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.735+0000 7fa2783ac700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2700cc2a0 con 0x7fa27015a950 2023-11-08T19:24:48.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.735+0000 7fa2783ac700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2700cc130 con 0x7fa27006bd20 2023-11-08T19:24:48.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.735+0000 7fa2783ac700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa27006c1b0 con 0x7fa2700b61e0 2023-11-08T19:24:48.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.735+0000 7fa275947700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa27006bd20 0x7fa2700d01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:48.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.735+0000 7fa275947700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa27006bd20 0x7fa2700d01b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48536/0 (socket says 172.21.15.143:48536) 2023-11-08T19:24:48.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.735+0000 7fa275947700 1 -- 172.21.15.143:0/2641838358 learned_addr learned my addr 172.21.15.143:0/2641838358 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:24:48.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.735+0000 7fa276949700 1 --2- 172.21.15.143:0/2641838358 >> v2:172.21.15.174:3300/0 conn(0x7fa2700b61e0 0x7fa2700cc730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:48.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.736+0000 7fa275947700 1 -- 172.21.15.143:0/2641838358 >> v2:172.21.15.174:3300/0 conn(0x7fa2700b61e0 msgr2=0x7fa2700cc730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:24:48.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.736+0000 7fa275947700 1 --2- 172.21.15.143:0/2641838358 >> v2:172.21.15.174:3300/0 conn(0x7fa2700b61e0 0x7fa2700cc730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:48.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.736+0000 7fa275947700 1 -- 172.21.15.143:0/2641838358 >> v1:172.21.15.174:6789/0 conn(0x7fa27015a950 legacy=0x7fa270072370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:48.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.736+0000 7fa275947700 1 -- 172.21.15.143:0/2641838358 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa27006c490 con 0x7fa27006bd20 2023-11-08T19:24:48.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.736+0000 7fa276949700 1 --2- 172.21.15.143:0/2641838358 >> v2:172.21.15.174:3300/0 conn(0x7fa2700b61e0 0x7fa2700cc730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:24:48.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.736+0000 7fa275947700 1 --2- 172.21.15.143:0/2641838358 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa27006bd20 0x7fa2700d01b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa2700c7890 tx=0x7fa26c00be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e82115ce07e0a04 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:48.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.736+0000 7fa275146700 1 -- 172.21.15.143:0/2641838358 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa26c00dd30 con 0x7fa27006bd20 2023-11-08T19:24:48.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.736+0000 7fa275146700 1 -- 172.21.15.143:0/2641838358 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa26c00de90 con 0x7fa27006bd20 2023-11-08T19:24:48.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.736+0000 7fa275146700 1 -- 172.21.15.143:0/2641838358 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa26c010690 con 0x7fa27006bd20 2023-11-08T19:24:48.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.736+0000 7fa2783ac700 1 -- 172.21.15.143:0/2641838358 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa27006bd20 msgr2=0x7fa2700d01b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:48.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.736+0000 7fa2783ac700 1 --2- 172.21.15.143:0/2641838358 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa27006bd20 0x7fa2700d01b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa2700c7890 tx=0x7fa26c00be70 comp rx=0 tx=0).stop 2023-11-08T19:24:48.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.736+0000 7fa2783ac700 1 -- 172.21.15.143:0/2641838358 shutdown_connections 2023-11-08T19:24:48.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.736+0000 7fa2783ac700 1 --2- 172.21.15.143:0/2641838358 >> v2:172.21.15.174:3300/0 conn(0x7fa2700b61e0 0x7fa2700cc730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:48.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.736+0000 7fa2783ac700 1 --2- 172.21.15.143:0/2641838358 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa27006bd20 0x7fa2700d01b0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:48.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.736+0000 7fa2783ac700 1 -- 172.21.15.143:0/2641838358 >> 172.21.15.143:0/2641838358 conn(0x7fa2700cd2e0 msgr2=0x7fa2700cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:24:48.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.737+0000 7fa2783ac700 1 -- 172.21.15.143:0/2641838358 shutdown_connections 2023-11-08T19:24:48.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.737+0000 7fa2783ac700 1 -- 172.21.15.143:0/2641838358 wait complete. 2023-11-08T19:24:48.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.738+0000 7fa2783ac700 1 Processor -- start 2023-11-08T19:24:48.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.738+0000 7fa2783ac700 1 -- start start 2023-11-08T19:24:48.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.738+0000 7fa2783ac700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2700b61e0 0x7fa2701581d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:48.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.738+0000 7fa2783ac700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa27015a950 0x7fa270156330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:48.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.738+0000 7fa2783ac700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa270156fd0 0x7fa2700ce3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:48.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.738+0000 7fa2783ac700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa270156870 con 0x7fa27015a950 2023-11-08T19:24:48.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.738+0000 7fa2783ac700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2700c4330 con 0x7fa2700b61e0 2023-11-08T19:24:48.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.738+0000 7fa2783ac700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2700c4610 con 0x7fa270156fd0 2023-11-08T19:24:48.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.738+0000 7fa275947700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa27015a950 0x7fa270156330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:48.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.738+0000 7fa275947700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa27015a950 0x7fa270156330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48542/0 (socket says 172.21.15.143:48542) 2023-11-08T19:24:48.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.738+0000 7fa275947700 1 -- 172.21.15.143:0/2469940353 learned_addr learned my addr 172.21.15.143:0/2469940353 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:24:48.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.739+0000 7fa276148700 1 --2- 172.21.15.143:0/2469940353 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2700b61e0 0x7fa2701581d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:48.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.739+0000 7fa275947700 1 -- 172.21.15.143:0/2469940353 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa270156fd0 msgr2=0x7fa2700ce3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:48.750 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.739+0000 7fa276949700 1 --2- 172.21.15.143:0/2469940353 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa270156fd0 0x7fa2700ce3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:48.750 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.739+0000 7fa275947700 1 --2- 172.21.15.143:0/2469940353 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa270156fd0 0x7fa2700ce3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:48.750 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.739+0000 7fa275947700 1 -- 172.21.15.143:0/2469940353 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2700b61e0 msgr2=0x7fa2701581d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:48.750 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.739+0000 7fa275947700 1 --2- 172.21.15.143:0/2469940353 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2700b61e0 0x7fa2701581d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:48.750 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.739+0000 7fa275947700 1 -- 172.21.15.143:0/2469940353 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa26c00b7f0 con 0x7fa27015a950 2023-11-08T19:24:48.751 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.739+0000 7fa276148700 1 --2- 172.21.15.143:0/2469940353 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2700b61e0 0x7fa2701581d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:24:48.751 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.739+0000 7fa275947700 1 --2- 172.21.15.143:0/2469940353 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa27015a950 0x7fa270156330 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa26c000c00 tx=0x7fa26c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:48.751 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.739+0000 7fa266ffd700 1 -- 172.21.15.143:0/2469940353 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa26c01f070 con 0x7fa27015a950 2023-11-08T19:24:48.751 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.739+0000 7fa266ffd700 1 -- 172.21.15.143:0/2469940353 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa26c003ba0 con 0x7fa27015a950 2023-11-08T19:24:48.751 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.739+0000 7fa266ffd700 1 -- 172.21.15.143:0/2469940353 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa26c010470 con 0x7fa27015a950 2023-11-08T19:24:48.752 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.739+0000 7fa2783ac700 1 -- 172.21.15.143:0/2469940353 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2700c48f0 con 0x7fa27015a950 2023-11-08T19:24:48.752 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.739+0000 7fa2783ac700 1 -- 172.21.15.143:0/2469940353 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2700c4eb0 con 0x7fa27015a950 2023-11-08T19:24:48.752 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.740+0000 7fa2783ac700 1 -- 172.21.15.143:0/2469940353 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa27004eee0 con 0x7fa27015a950 2023-11-08T19:24:48.752 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.741+0000 7fa266ffd700 1 -- 172.21.15.143:0/2469940353 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa26c019400 con 0x7fa27015a950 2023-11-08T19:24:48.752 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.741+0000 7fa266ffd700 1 --2- 172.21.15.143:0/2469940353 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa25406af00 0x7fa25406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:48.753 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.741+0000 7fa266ffd700 1 -- 172.21.15.143:0/2469940353 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5227+0+0 (secure 0 0 0) 0x7fa26c08c870 con 0x7fa27015a950 2023-11-08T19:24:48.753 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.741+0000 7fa276148700 1 --2- 172.21.15.143:0/2469940353 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa25406af00 0x7fa25406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:48.753 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.742+0000 7fa276148700 1 --2- 172.21.15.143:0/2469940353 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa25406af00 0x7fa25406d3c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa2700c5dc0 tx=0x7fa260009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:48.753 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.746+0000 7fa266ffd700 1 -- 172.21.15.143:0/2469940353 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa26c057b90 con 0x7fa27015a950 2023-11-08T19:24:48.938 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:48.936+0000 7fa2783ac700 1 -- 172.21.15.143:0/2469940353 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fa270072150 con 0x7fa27015a950 2023-11-08T19:24:49.591 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:49.590+0000 7fa266ffd700 1 -- 172.21.15.143:0/2469940353 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v18) v1 ==== 170+0+0 (secure 0 0 0) 0x7fa26c05b1b0 con 0x7fa27015a950 2023-11-08T19:24:49.594 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:49.593+0000 7fa2783ac700 1 -- 172.21.15.143:0/2469940353 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa25406af00 msgr2=0x7fa25406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:49.595 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:49.593+0000 7fa2783ac700 1 --2- 172.21.15.143:0/2469940353 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa25406af00 0x7fa25406d3c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa2700c5dc0 tx=0x7fa260009040 comp rx=0 tx=0).stop 2023-11-08T19:24:49.595 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:49.593+0000 7fa2783ac700 1 -- 172.21.15.143:0/2469940353 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa27015a950 msgr2=0x7fa270156330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:49.596 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:49.593+0000 7fa2783ac700 1 --2- 172.21.15.143:0/2469940353 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa27015a950 0x7fa270156330 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa26c000c00 tx=0x7fa26c004750 comp rx=0 tx=0).stop 2023-11-08T19:24:49.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:49.594+0000 7fa2783ac700 1 -- 172.21.15.143:0/2469940353 shutdown_connections 2023-11-08T19:24:49.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:49.594+0000 7fa2783ac700 1 --2- 172.21.15.143:0/2469940353 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa25406af00 0x7fa25406d3c0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:49.598 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:49.594+0000 7fa2783ac700 1 --2- 172.21.15.143:0/2469940353 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa270156fd0 0x7fa2700ce3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:49.598 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:49.594+0000 7fa2783ac700 1 --2- 172.21.15.143:0/2469940353 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2700b61e0 0x7fa2701581d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:49.599 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:49.594+0000 7fa2783ac700 1 --2- 172.21.15.143:0/2469940353 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa27015a950 0x7fa270156330 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:49.599 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:49.594+0000 7fa2783ac700 1 -- 172.21.15.143:0/2469940353 >> 172.21.15.143:0/2469940353 conn(0x7fa2700cd2e0 msgr2=0x7fa2700cc630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:24:49.599 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:49.594+0000 7fa2783ac700 1 -- 172.21.15.143:0/2469940353 shutdown_connections 2023-11-08T19:24:49.599 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:49.594+0000 7fa2783ac700 1 -- 172.21.15.143:0/2469940353 wait complete. 2023-11-08T19:24:49.604 INFO:teuthology.orchestra.run.smithi143.stderr:pool 'cephfs_data' created 2023-11-08T19:24:49.624 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-08T19:24:49.751 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:49.750+0000 7f1f1c3d8700 1 Processor -- start 2023-11-08T19:24:49.752 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:49.750+0000 7f1f1c3d8700 1 -- start start 2023-11-08T19:24:49.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:49.751+0000 7f1f1c3d8700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1f1415bb20 0x7f1f140b2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:49.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:49.751+0000 7f1f1c3d8700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f1f140b1730 0x7f1f140d1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:49.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:49.751+0000 7f1f1c3d8700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f140d0420 con 0x7f1f140d1bb0 2023-11-08T19:24:49.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:49.751+0000 7f1f1c3d8700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f140cf600 con 0x7f1f140b1730 2023-11-08T19:24:49.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:49.751+0000 7f1f1c3d8700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f140d2040 con 0x7f1f1415bb20 2023-11-08T19:24:49.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:49.751+0000 7f1f19973700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f1f140b1730 0x7f1f140d1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:49.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:49.751+0000 7f1f1a975700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f1f140d1bb0 0x7f1f140ce420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:44704/0 (socket says 172.21.15.143:44704) 2023-11-08T19:24:49.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:49.751+0000 7f1f19973700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f1f140b1730 0x7f1f140d1670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48548/0 (socket says 172.21.15.143:48548) 2023-11-08T19:24:49.756 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:49.751+0000 7f1f1a975700 1 -- 172.21.15.143:0/1606369411 learned_addr learned my addr 172.21.15.143:0/1606369411 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:24:49.756 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:49.751+0000 7f1f1a174700 1 --2- 172.21.15.143:0/1606369411 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1f1415bb20 0x7f1f140b2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:49.756 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:49.751+0000 7f1f19973700 1 -- 172.21.15.143:0/1606369411 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1f1415bb20 msgr2=0x7f1f140b2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:24:49.756 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:49.751+0000 7f1f19973700 1 --2- 172.21.15.143:0/1606369411 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1f1415bb20 0x7f1f140b2670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:49.756 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:49.751+0000 7f1f19973700 1 -- 172.21.15.143:0/1606369411 >> v1:172.21.15.143:6789/0 conn(0x7f1f140d1bb0 legacy=0x7f1f140ce420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:49.757 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:49.751+0000 7f1f19172700 1 -- 172.21.15.143:0/1606369411 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1240244921 0 0) 0x7f1f140d0420 con 0x7f1f140d1bb0 2023-11-08T19:24:49.757 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:49.751+0000 7f1f19973700 1 -- 172.21.15.143:0/1606369411 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f140cf810 con 0x7f1f140b1730 2023-11-08T19:24:49.757 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:49.751+0000 7f1f19973700 1 --2- 172.21.15.143:0/1606369411 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1f140b1730 0x7f1f140d1670 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1f10009840 tx=0x7f1f10009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a434a9dcea27a4dc server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:49.757 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:49.752+0000 7f1f19172700 1 -- 172.21.15.143:0/1606369411 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f10011830 con 0x7f1f140b1730 2023-11-08T19:24:49.757 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:49.752+0000 7f1f19172700 1 -- 172.21.15.143:0/1606369411 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f10011990 con 0x7f1f140b1730 2023-11-08T19:24:49.758 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:49.752+0000 7f1f19172700 1 -- 172.21.15.143:0/1606369411 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f10011d20 con 0x7f1f140b1730 2023-11-08T19:24:49.758 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:49.752+0000 7f1f1c3d8700 1 -- 172.21.15.143:0/1606369411 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1f140b1730 msgr2=0x7f1f140d1670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:49.758 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:49.752+0000 7f1f1c3d8700 1 --2- 172.21.15.143:0/1606369411 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1f140b1730 0x7f1f140d1670 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1f10009840 tx=0x7f1f10009b50 comp rx=0 tx=0).stop 2023-11-08T19:24:49.758 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:49.752+0000 7f1f1c3d8700 1 -- 172.21.15.143:0/1606369411 shutdown_connections 2023-11-08T19:24:49.758 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:49.752+0000 7f1f1c3d8700 1 --2- 172.21.15.143:0/1606369411 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1f1415bb20 0x7f1f140b2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:49.759 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:49.752+0000 7f1f1c3d8700 1 --2- 172.21.15.143:0/1606369411 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1f140b1730 0x7f1f140d1670 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:49.759 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:49.752+0000 7f1f1c3d8700 1 -- 172.21.15.143:0/1606369411 >> 172.21.15.143:0/1606369411 conn(0x7f1f1408b000 msgr2=0x7f1f1409ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:24:49.759 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:49.752+0000 7f1f1c3d8700 1 -- 172.21.15.143:0/1606369411 shutdown_connections 2023-11-08T19:24:49.759 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:49.752+0000 7f1f1c3d8700 1 -- 172.21.15.143:0/1606369411 wait complete. 2023-11-08T19:24:49.764 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:49.753+0000 7f1f1c3d8700 1 Processor -- start 2023-11-08T19:24:49.765 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:49.754+0000 7f1f1c3d8700 1 -- start start 2023-11-08T19:24:49.765 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:49.754+0000 7f1f1c3d8700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1f140b1730 0x7f1f140c6960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:49.765 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:49.754+0000 7f1f1c3d8700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1f140d1bb0 0x7f1f140b5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:49.765 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:49.754+0000 7f1f1c3d8700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1f1415bb20 0x7f1f140b67e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:49.766 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:49.754+0000 7f1f1c3d8700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f140b4650 con 0x7f1f140d1bb0 2023-11-08T19:24:49.766 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:49.754+0000 7f1f1c3d8700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f140cc890 con 0x7f1f1415bb20 2023-11-08T19:24:49.766 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:49.754+0000 7f1f1c3d8700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f140b48a0 con 0x7f1f140b1730 2023-11-08T19:24:49.766 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:49.754+0000 7f1f19973700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1f140d1bb0 0x7f1f140b5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:49.766 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:49.754+0000 7f1f19973700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1f140d1bb0 0x7f1f140b5660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48552/0 (socket says 172.21.15.143:48552) 2023-11-08T19:24:49.767 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:49.754+0000 7f1f19973700 1 -- 172.21.15.143:0/3924752113 learned_addr learned my addr 172.21.15.143:0/3924752113 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:24:49.767 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:49.755+0000 7f1f19973700 1 -- 172.21.15.143:0/3924752113 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1f140b1730 msgr2=0x7f1f140c6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:49.767 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:49.755+0000 7f1f19973700 1 --2- 172.21.15.143:0/3924752113 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1f140b1730 0x7f1f140c6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:49.767 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:49.755+0000 7f1f19973700 1 -- 172.21.15.143:0/3924752113 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1f1415bb20 msgr2=0x7f1f140b67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:49.767 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:49.755+0000 7f1f1a975700 1 --2- 172.21.15.143:0/3924752113 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1f1415bb20 0x7f1f140b67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:49.768 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:49.755+0000 7f1f19973700 1 --2- 172.21.15.143:0/3924752113 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1f1415bb20 0x7f1f140b67e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:49.768 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:49.755+0000 7f1f19973700 1 -- 172.21.15.143:0/3924752113 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f100094a0 con 0x7f1f140d1bb0 2023-11-08T19:24:49.768 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:49.755+0000 7f1f19973700 1 --2- 172.21.15.143:0/3924752113 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1f140d1bb0 0x7f1f140b5660 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1f100090b0 tx=0x7f1f10004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:49.768 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:49.755+0000 7f1f0affd700 1 -- 172.21.15.143:0/3924752113 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f100038a0 con 0x7f1f140d1bb0 2023-11-08T19:24:49.769 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:49.755+0000 7f1f0affd700 1 -- 172.21.15.143:0/3924752113 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f1001f070 con 0x7f1f140d1bb0 2023-11-08T19:24:49.769 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:49.755+0000 7f1f1c3d8700 1 -- 172.21.15.143:0/3924752113 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f140b4b80 con 0x7f1f140d1bb0 2023-11-08T19:24:49.769 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:49.755+0000 7f1f0affd700 1 -- 172.21.15.143:0/3924752113 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f10003c20 con 0x7f1f140d1bb0 2023-11-08T19:24:49.769 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:49.755+0000 7f1f1c3d8700 1 -- 172.21.15.143:0/3924752113 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f140b5140 con 0x7f1f140d1bb0 2023-11-08T19:24:49.769 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:49.756+0000 7f1f1c3d8700 1 -- 172.21.15.143:0/3924752113 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f140b2c10 con 0x7f1f140d1bb0 2023-11-08T19:24:49.770 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:49.756+0000 7f1f0affd700 1 -- 172.21.15.143:0/3924752113 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1f1001a430 con 0x7f1f140d1bb0 2023-11-08T19:24:49.770 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:49.757+0000 7f1f0affd700 1 --2- 172.21.15.143:0/3924752113 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f1ef806af00 0x7f1ef806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:49.770 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:49.757+0000 7f1f0affd700 1 -- 172.21.15.143:0/3924752113 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f1f10021070 con 0x7f1f140d1bb0 2023-11-08T19:24:49.770 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:49.757+0000 7f1f1a174700 1 --2- 172.21.15.143:0/3924752113 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f1ef806af00 0x7f1ef806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:49.771 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:49.757+0000 7f1f1a174700 1 --2- 172.21.15.143:0/3924752113 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f1ef806af00 0x7f1ef806d3c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1f140c7ca0 tx=0x7f1f04008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:49.771 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:49.761+0000 7f1f0affd700 1 -- 172.21.15.143:0/3924752113 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1f10056120 con 0x7f1f140d1bb0 2023-11-08T19:24:49.995 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:49.994+0000 7f1f1c3d8700 1 -- 172.21.15.143:0/3924752113 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f1f140cfdd0 con 0x7f1f140d1bb0 2023-11-08T19:24:50.619 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:50.617+0000 7f1f0affd700 1 -- 172.21.15.143:0/3924752113 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7f1f100150c0 con 0x7f1f140d1bb0 2023-11-08T19:24:50.627 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:50.625+0000 7f1f1c3d8700 1 -- 172.21.15.143:0/3924752113 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f1ef806af00 msgr2=0x7f1ef806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:50.627 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:50.625+0000 7f1f1c3d8700 1 --2- 172.21.15.143:0/3924752113 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f1ef806af00 0x7f1ef806d3c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1f140c7ca0 tx=0x7f1f04008040 comp rx=0 tx=0).stop 2023-11-08T19:24:50.627 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:50.625+0000 7f1f1c3d8700 1 -- 172.21.15.143:0/3924752113 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1f140d1bb0 msgr2=0x7f1f140b5660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:50.628 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:50.625+0000 7f1f1c3d8700 1 --2- 172.21.15.143:0/3924752113 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1f140d1bb0 0x7f1f140b5660 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1f100090b0 tx=0x7f1f10004750 comp rx=0 tx=0).stop 2023-11-08T19:24:50.628 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:50.626+0000 7f1f1c3d8700 1 -- 172.21.15.143:0/3924752113 shutdown_connections 2023-11-08T19:24:50.628 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:50.626+0000 7f1f1c3d8700 1 --2- 172.21.15.143:0/3924752113 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f1ef806af00 0x7f1ef806d3c0 secure :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1f140c7ca0 tx=0x7f1f04008040 comp rx=0 tx=0).stop 2023-11-08T19:24:50.628 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:50.626+0000 7f1f1c3d8700 1 --2- 172.21.15.143:0/3924752113 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1f140b1730 0x7f1f140c6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:50.628 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:50.626+0000 7f1f1c3d8700 1 --2- 172.21.15.143:0/3924752113 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1f1415bb20 0x7f1f140b67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:50.629 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:50.626+0000 7f1f1c3d8700 1 --2- 172.21.15.143:0/3924752113 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1f140d1bb0 0x7f1f140b5660 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:50.629 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:50.626+0000 7f1f1c3d8700 1 -- 172.21.15.143:0/3924752113 >> 172.21.15.143:0/3924752113 conn(0x7f1f1408b000 msgr2=0x7f1f140b3960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:24:50.629 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:50.626+0000 7f1f1c3d8700 1 -- 172.21.15.143:0/3924752113 shutdown_connections 2023-11-08T19:24:50.629 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:50.626+0000 7f1f1c3d8700 1 -- 172.21.15.143:0/3924752113 wait complete. 2023-11-08T19:24:50.632 INFO:teuthology.orchestra.run.smithi143.stderr:new fs with metadata pool 2 and data pool 3 2023-11-08T19:24:50.649 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:24:50.773 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:50.771+0000 7f59a118b700 1 Processor -- start 2023-11-08T19:24:50.773 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:50.771+0000 7f59a118b700 1 -- start start 2023-11-08T19:24:50.775 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:50.772+0000 7f59a118b700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f599c15aa30 0x7f599c0cf510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:50.776 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:50.772+0000 7f59a118b700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f599c0cc530 0x7f599c0b26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:50.776 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:50.772+0000 7f59a118b700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f599c0cf080 con 0x7f599c0b2210 2023-11-08T19:24:50.776 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:50.772+0000 7f59a118b700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f599c0b2be0 con 0x7f599c15aa30 2023-11-08T19:24:50.776 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:50.772+0000 7f59a118b700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f599c0ce470 con 0x7f599c0cc530 2023-11-08T19:24:50.776 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:50.772+0000 7f599ad9d700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f599c15aa30 0x7f599c0cf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:50.777 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:50.772+0000 7f599ad9d700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f599c15aa30 0x7f599c0cf510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48560/0 (socket says 172.21.15.143:48560) 2023-11-08T19:24:50.777 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:50.772+0000 7f599ad9d700 1 -- 172.21.15.143:0/1512578692 learned_addr learned my addr 172.21.15.143:0/1512578692 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:24:50.777 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:50.772+0000 7f599ad9d700 1 -- 172.21.15.143:0/1512578692 >> v2:172.21.15.174:3300/0 conn(0x7f599c0cc530 msgr2=0x7f599c0b26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:24:50.777 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:50.772+0000 7f599ad9d700 1 --2- 172.21.15.143:0/1512578692 >> v2:172.21.15.174:3300/0 conn(0x7f599c0cc530 0x7f599c0b26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:50.777 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:50.772+0000 7f599ad9d700 1 -- 172.21.15.143:0/1512578692 >> v1:172.21.15.174:6789/0 conn(0x7f599c0b2210 legacy=0x7f599c0cbe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:24:50.778 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:50.772+0000 7f599ad9d700 1 -- 172.21.15.143:0/1512578692 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f599c0ce750 con 0x7f599c15aa30 2023-11-08T19:24:50.778 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:50.773+0000 7f599ad9d700 1 --2- 172.21.15.143:0/1512578692 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f599c15aa30 0x7f599c0cf510 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f599c0709e0 tx=0x7f599000be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=448ed0f06634824b server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:50.778 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:50.773+0000 7f5999d9b700 1 -- 172.21.15.143:0/1512578692 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5990005e30 con 0x7f599c15aa30 2023-11-08T19:24:50.778 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:50.773+0000 7f5999d9b700 1 -- 172.21.15.143:0/1512578692 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f599000d8b0 con 0x7f599c15aa30 2023-11-08T19:24:50.778 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:50.773+0000 7f5999d9b700 1 -- 172.21.15.143:0/1512578692 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f599000dc40 con 0x7f599c15aa30 2023-11-08T19:24:50.779 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:50.773+0000 7f59a118b700 1 -- 172.21.15.143:0/1512578692 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f599c15aa30 msgr2=0x7f599c0cf510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:50.779 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:50.773+0000 7f59a118b700 1 --2- 172.21.15.143:0/1512578692 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f599c15aa30 0x7f599c0cf510 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f599c0709e0 tx=0x7f599000be70 comp rx=0 tx=0).stop 2023-11-08T19:24:50.779 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:50.773+0000 7f59a118b700 1 -- 172.21.15.143:0/1512578692 shutdown_connections 2023-11-08T19:24:50.779 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:50.773+0000 7f59a118b700 1 --2- 172.21.15.143:0/1512578692 >> v2:172.21.15.174:3300/0 conn(0x7f599c0cc530 0x7f599c0b26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:50.779 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:50.773+0000 7f59a118b700 1 --2- 172.21.15.143:0/1512578692 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f599c15aa30 0x7f599c0cf510 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:50.779 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:50.773+0000 7f59a118b700 1 -- 172.21.15.143:0/1512578692 >> 172.21.15.143:0/1512578692 conn(0x7f599c099ab0 msgr2=0x7f599c0d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:24:50.780 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:50.773+0000 7f59a118b700 1 -- 172.21.15.143:0/1512578692 shutdown_connections 2023-11-08T19:24:50.780 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:50.773+0000 7f59a118b700 1 -- 172.21.15.143:0/1512578692 wait complete. 2023-11-08T19:24:50.783 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:50.775+0000 7f59a118b700 1 Processor -- start 2023-11-08T19:24:50.783 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:50.775+0000 7f59a118b700 1 -- start start 2023-11-08T19:24:50.783 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:50.775+0000 7f59a118b700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f599c0b2210 0x7f599c14ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:50.783 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:50.775+0000 7f59a118b700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f599c0cc530 0x7f599c0c77d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:50.783 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:50.775+0000 7f59a118b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f599c15aa30 0x7f599c0b15f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:50.784 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:50.776+0000 7f59a118b700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f599c0c8850 con 0x7f599c0b2210 2023-11-08T19:24:50.784 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:50.776+0000 7f59a118b700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f599c15b0e0 con 0x7f599c15aa30 2023-11-08T19:24:50.784 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:50.776+0000 7f59a118b700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f599c0c8290 con 0x7f599c0cc530 2023-11-08T19:24:50.784 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:50.776+0000 7f599ad9d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f599c0b2210 0x7f599c14ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:50.784 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:50.776+0000 7f599ad9d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f599c0b2210 0x7f599c14ffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48566/0 (socket says 172.21.15.143:48566) 2023-11-08T19:24:50.785 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:50.776+0000 7f599ad9d700 1 -- 172.21.15.143:0/340154129 learned_addr learned my addr 172.21.15.143:0/340154129 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:24:50.785 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:50.776+0000 7f599b59e700 1 --2- 172.21.15.143:0/340154129 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f599c15aa30 0x7f599c0b15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:50.785 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:50.776+0000 7f599a59c700 1 --2- 172.21.15.143:0/340154129 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f599c0cc530 0x7f599c0c77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:50.785 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:50.776+0000 7f599ad9d700 1 -- 172.21.15.143:0/340154129 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f599c0cc530 msgr2=0x7f599c0c77d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:50.785 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:50.776+0000 7f599ad9d700 1 --2- 172.21.15.143:0/340154129 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f599c0cc530 0x7f599c0c77d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:50.786 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:50.776+0000 7f599ad9d700 1 -- 172.21.15.143:0/340154129 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f599c15aa30 msgr2=0x7f599c0b15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:50.786 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:50.776+0000 7f599ad9d700 1 --2- 172.21.15.143:0/340154129 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f599c15aa30 0x7f599c0b15f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:50.786 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:50.776+0000 7f599ad9d700 1 -- 172.21.15.143:0/340154129 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f599000b7a0 con 0x7f599c0b2210 2023-11-08T19:24:50.786 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:50.776+0000 7f599ad9d700 1 --2- 172.21.15.143:0/340154129 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f599c0b2210 0x7f599c14ffb0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5990000c00 tx=0x7f5990004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:50.786 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:50.776+0000 7f59837fe700 1 -- 172.21.15.143:0/340154129 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f599001a650 con 0x7f599c0b2210 2023-11-08T19:24:50.787 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:50.776+0000 7f59837fe700 1 -- 172.21.15.143:0/340154129 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f599001a7d0 con 0x7f599c0b2210 2023-11-08T19:24:50.787 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:50.776+0000 7f59a118b700 1 -- 172.21.15.143:0/340154129 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f599c0c8570 con 0x7f599c0b2210 2023-11-08T19:24:50.787 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:50.776+0000 7f59837fe700 1 -- 172.21.15.143:0/340154129 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f599001ab80 con 0x7f599c0b2210 2023-11-08T19:24:50.787 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:50.776+0000 7f59a118b700 1 -- 172.21.15.143:0/340154129 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f599c0c9030 con 0x7f599c0b2210 2023-11-08T19:24:50.787 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:50.777+0000 7f59a118b700 1 -- 172.21.15.143:0/340154129 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f599c0c8bb0 con 0x7f599c0b2210 2023-11-08T19:24:50.788 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:50.778+0000 7f59837fe700 1 -- 172.21.15.143:0/340154129 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f599001ace0 con 0x7f599c0b2210 2023-11-08T19:24:50.788 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:50.778+0000 7f59837fe700 1 --2- 172.21.15.143:0/340154129 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f598806af00 0x7f598806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:50.788 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:50.778+0000 7f59837fe700 1 -- 172.21.15.143:0/340154129 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f599008b020 con 0x7f599c0b2210 2023-11-08T19:24:50.788 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:50.778+0000 7f599a59c700 1 --2- 172.21.15.143:0/340154129 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f598806af00 0x7f598806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:50.788 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:50.779+0000 7f599a59c700 1 --2- 172.21.15.143:0/340154129 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f598806af00 0x7f598806d3c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f599c0afbc0 tx=0x7f598c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:50.789 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:50.783+0000 7f59837fe700 1 -- 172.21.15.143:0/340154129 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f599c0c8bb0 con 0x7f599c0b2210 2023-11-08T19:24:50.976 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:50.975+0000 7f59a118b700 1 -- 172.21.15.143:0/340154129 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f599c0c8bb0 con 0x7f599c0b2210 2023-11-08T19:24:50.977 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:50.976+0000 7f59837fe700 1 -- 172.21.15.143:0/340154129 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+16566 (secure 0 0 0) 0x7f599c0c8bb0 con 0x7f599c0b2210 2023-11-08T19:24:50.978 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:24:50.978 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":19,"fsid":"968483af-77fc-4068-ad7c-5c2a1a4613d7","created":"2023-11-08T19:24:32.826484+0000","modified":"2023-11-08T19:24:49.995234+0000","last_up_change":"2023-11-08T19:24:43.573880+0000","last_in_change":"2023-11-08T19:24:35.314525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:44.551530+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-08T19:24:48.256781+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-08T19:24:48.954990+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bae98ec1-d94e-403d-a220-a522a53ea6f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6809","nonce":1487830566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6811","nonce":1487830566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6815","nonce":1487830566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6813","nonce":1487830566}]},"public_addr":"172.21.15.143:6809/1487830566","cluster_addr":"172.21.15.143:6811/1487830566","heartbeat_back_addr":"172.21.15.143:6815/1487830566","heartbeat_front_addr":"172.21.15.143:6813/1487830566","state":["exists","up"]},{"osd":1,"uuid":"5cb6538a-1ccb-4322-8633-09f66af0b7e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6801","nonce":787341397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6803","nonce":787341397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6807","nonce":787341397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6805","nonce":787341397}]},"public_addr":"172.21.15.143:6801/787341397","cluster_addr":"172.21.15.143:6803/787341397","heartbeat_back_addr":"172.21.15.143:6807/787341397","heartbeat_front_addr":"172.21.15.143:6805/787341397","state":["exists","up"]},{"osd":2,"uuid":"cdbf522c-60ef-4d60-9653-8dcb529c101a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6822","nonce":2301325597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6826","nonce":2301325597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6831","nonce":2301325597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6829","nonce":2301325597}]},"public_addr":"172.21.15.143:6822/2301325597","cluster_addr":"172.21.15.143:6826/2301325597","heartbeat_back_addr":"172.21.15.143:6831/2301325597","heartbeat_front_addr":"172.21.15.143:6829/2301325597","state":["exists","up"]},{"osd":3,"uuid":"4e2783cf-ebb9-4dd8-8245-bc1d17ce4325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6817","nonce":2021504428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6819","nonce":2021504428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6827","nonce":2021504428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6823","nonce":2021504428}]},"public_addr":"172.21.15.143:6817/2021504428","cluster_addr":"172.21.15.143:6819/2021504428","heartbeat_back_addr":"172.21.15.143:6827/2021504428","heartbeat_front_addr":"172.21.15.143:6823/2021504428","state":["exists","up"]},{"osd":4,"uuid":"691256fa-9b77-49b9-a346-7ae1adf0ca80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6801","nonce":1993380595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6803","nonce":1993380595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6807","nonce":1993380595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6805","nonce":1993380595}]},"public_addr":"172.21.15.174:6801/1993380595","cluster_addr":"172.21.15.174:6803/1993380595","heartbeat_back_addr":"172.21.15.174:6807/1993380595","heartbeat_front_addr":"172.21.15.174:6805/1993380595","state":["exists","up"]},{"osd":5,"uuid":"756fc352-48c9-46ce-b52e-0541fe6f8525","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6809","nonce":1777923303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6811","nonce":1777923303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6815","nonce":1777923303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6813","nonce":1777923303}]},"public_addr":"172.21.15.174:6809/1777923303","cluster_addr":"172.21.15.174:6811/1777923303","heartbeat_back_addr":"172.21.15.174:6815/1777923303","heartbeat_front_addr":"172.21.15.174:6813/1777923303","state":["exists","up"]},{"osd":6,"uuid":"fab2ac2b-ed97-4cde-9825-b480eabf8b8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6825","nonce":2755130316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6827","nonce":2755130316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6831","nonce":2755130316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6829","nonce":2755130316}]},"public_addr":"172.21.15.174:6825/2755130316","cluster_addr":"172.21.15.174:6827/2755130316","heartbeat_back_addr":"172.21.15.174:6831/2755130316","heartbeat_front_addr":"172.21.15.174:6829/2755130316","state":["exists","up"]},{"osd":7,"uuid":"3de9b797-2ab6-4d41-a863-7e7d638fd703","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6817","nonce":2847141411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6819","nonce":2847141411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6823","nonce":2847141411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6821","nonce":2847141411}]},"public_addr":"172.21.15.174:6817/2847141411","cluster_addr":"172.21.15.174:6819/2847141411","heartbeat_back_addr":"172.21.15.174:6823/2847141411","heartbeat_front_addr":"172.21.15.174:6821/2847141411","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.477076+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.496890+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.539305+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.454673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.538881+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.521294+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.520712+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.540404+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:24:50.980 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:50.978+0000 7f59a118b700 1 -- 172.21.15.143:0/340154129 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f598806af00 msgr2=0x7f598806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:50.980 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:50.978+0000 7f59a118b700 1 --2- 172.21.15.143:0/340154129 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f598806af00 0x7f598806d3c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f599c0afbc0 tx=0x7f598c009040 comp rx=0 tx=0).stop 2023-11-08T19:24:50.980 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:50.978+0000 7f59a118b700 1 -- 172.21.15.143:0/340154129 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f599c0b2210 msgr2=0x7f599c14ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:50.980 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:50.978+0000 7f59a118b700 1 --2- 172.21.15.143:0/340154129 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f599c0b2210 0x7f599c14ffb0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5990000c00 tx=0x7f5990004750 comp rx=0 tx=0).stop 2023-11-08T19:24:50.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:50.978+0000 7f59a118b700 1 -- 172.21.15.143:0/340154129 shutdown_connections 2023-11-08T19:24:50.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:50.978+0000 7f59a118b700 1 --2- 172.21.15.143:0/340154129 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f598806af00 0x7f598806d3c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:50.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:50.978+0000 7f59a118b700 1 --2- 172.21.15.143:0/340154129 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f599c15aa30 0x7f599c0b15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:50.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:50.978+0000 7f59a118b700 1 --2- 172.21.15.143:0/340154129 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f599c0b2210 0x7f599c14ffb0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:50.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:50.978+0000 7f59a118b700 1 --2- 172.21.15.143:0/340154129 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f599c0cc530 0x7f599c0c77d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:50.982 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:50.978+0000 7f59a118b700 1 -- 172.21.15.143:0/340154129 >> 172.21.15.143:0/340154129 conn(0x7f599c099ab0 msgr2=0x7f599c0cb2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:24:50.982 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:50.978+0000 7f59a118b700 1 -- 172.21.15.143:0/340154129 shutdown_connections 2023-11-08T19:24:50.982 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:50.978+0000 7f59a118b700 1 -- 172.21.15.143:0/340154129 wait complete. 2023-11-08T19:24:50.992 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:24:51.112 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.110+0000 7f05795f3700 1 Processor -- start 2023-11-08T19:24:51.112 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.110+0000 7f05795f3700 1 -- start start 2023-11-08T19:24:51.114 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.111+0000 7f05795f3700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f05740cf090 0x7f05740d1f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:51.114 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.111+0000 7f05795f3700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05740d3200 con 0x7f057415dc80 2023-11-08T19:24:51.114 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.111+0000 7f05795f3700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f057415e060 con 0x7f05740d3690 2023-11-08T19:24:51.114 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.111+0000 7f05795f3700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05740d25f0 con 0x7f05740cf090 2023-11-08T19:24:51.115 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.111+0000 7f0572ffd700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f057415dc80 0x7f05740ce980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:44728/0 (socket says 172.21.15.143:44728) 2023-11-08T19:24:51.115 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.111+0000 7f0572ffd700 1 -- 172.21.15.143:0/2878247857 learned_addr learned my addr 172.21.15.143:0/2878247857 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:24:51.115 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.111+0000 7f05727fc700 1 --2- 172.21.15.143:0/2878247857 >> v2:172.21.15.174:3300/0 conn(0x7f05740cf090 0x7f05740d1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:51.115 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.111+0000 7f0571ffb700 1 -- 172.21.15.143:0/2878247857 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 233412224 0 0) 0x7f05740d3200 con 0x7f057415dc80 2023-11-08T19:24:51.115 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.111+0000 7f0571ffb700 1 -- 172.21.15.143:0/2878247857 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f055c0035b0 con 0x7f057415dc80 2023-11-08T19:24:51.116 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.111+0000 7f05727fc700 1 -- 172.21.15.143:0/2878247857 >> v1:172.21.15.174:6789/0 conn(0x7f05740d3690 legacy=0x7f05740b6ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:51.116 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.111+0000 7f05727fc700 1 -- 172.21.15.143:0/2878247857 >> v1:172.21.15.143:6789/0 conn(0x7f057415dc80 legacy=0x7f05740ce980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:51.116 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.111+0000 7f05727fc700 1 -- 172.21.15.143:0/2878247857 --> v2:172.21.15.174:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05740d28d0 con 0x7f05740cf090 2023-11-08T19:24:51.116 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.112+0000 7f05727fc700 1 --2- 172.21.15.143:0/2878247857 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f05740cf090 0x7f05740d1f80 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0564009840 tx=0x7f0564009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c18a0f1197b2d51 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:51.116 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.112+0000 7f0571ffb700 1 -- 172.21.15.143:0/2878247857 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0564011830 con 0x7f05740cf090 2023-11-08T19:24:51.116 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.112+0000 7f0571ffb700 1 -- 172.21.15.143:0/2878247857 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0564011990 con 0x7f05740cf090 2023-11-08T19:24:51.117 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.112+0000 7f0571ffb700 1 -- 172.21.15.143:0/2878247857 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0564011d20 con 0x7f05740cf090 2023-11-08T19:24:51.117 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.112+0000 7f05795f3700 1 -- 172.21.15.143:0/2878247857 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f05740cf090 msgr2=0x7f05740d1f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:51.117 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.112+0000 7f05795f3700 1 --2- 172.21.15.143:0/2878247857 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f05740cf090 0x7f05740d1f80 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0564009840 tx=0x7f0564009b50 comp rx=0 tx=0).stop 2023-11-08T19:24:51.117 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.112+0000 7f05795f3700 1 -- 172.21.15.143:0/2878247857 shutdown_connections 2023-11-08T19:24:51.117 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.112+0000 7f05795f3700 1 --2- 172.21.15.143:0/2878247857 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f05740cf090 0x7f05740d1f80 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:51.118 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.112+0000 7f05795f3700 1 -- 172.21.15.143:0/2878247857 >> 172.21.15.143:0/2878247857 conn(0x7f05740668e0 msgr2=0x7f0574066cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:24:51.118 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.112+0000 7f05795f3700 1 -- 172.21.15.143:0/2878247857 shutdown_connections 2023-11-08T19:24:51.118 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.112+0000 7f05795f3700 1 -- 172.21.15.143:0/2878247857 wait complete. 2023-11-08T19:24:51.121 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.114+0000 7f05795f3700 1 Processor -- start 2023-11-08T19:24:51.121 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.114+0000 7f05795f3700 1 -- start start 2023-11-08T19:24:51.121 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.114+0000 7f05795f3700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f05740cf090 0x7f05740cc540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:51.121 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.114+0000 7f05795f3700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f05740d3690 0x7f05740cca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:51.122 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.114+0000 7f05795f3700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f057415dc80 0x7f05741f3120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:51.122 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.114+0000 7f05795f3700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05740ccfc0 con 0x7f057415dc80 2023-11-08T19:24:51.122 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.114+0000 7f05795f3700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05740cdb10 con 0x7f05740d3690 2023-11-08T19:24:51.122 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.114+0000 7f05795f3700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05740cd270 con 0x7f05740cf090 2023-11-08T19:24:51.122 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.115+0000 7f05737fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f057415dc80 0x7f05741f3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:51.123 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.115+0000 7f05737fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f057415dc80 0x7f05741f3120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48584/0 (socket says 172.21.15.143:48584) 2023-11-08T19:24:51.123 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.115+0000 7f05737fe700 1 -- 172.21.15.143:0/3504726873 learned_addr learned my addr 172.21.15.143:0/3504726873 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:24:51.123 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.115+0000 7f05727fc700 1 --2- 172.21.15.143:0/3504726873 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f05740d3690 0x7f05740cca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:51.123 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.115+0000 7f0572ffd700 1 --2- 172.21.15.143:0/3504726873 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f05740cf090 0x7f05740cc540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:51.123 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.115+0000 7f05737fe700 1 -- 172.21.15.143:0/3504726873 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f05740cf090 msgr2=0x7f05740cc540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:51.124 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.115+0000 7f05737fe700 1 --2- 172.21.15.143:0/3504726873 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f05740cf090 0x7f05740cc540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:51.124 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.115+0000 7f05737fe700 1 -- 172.21.15.143:0/3504726873 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f05740d3690 msgr2=0x7f05740cca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:51.124 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.115+0000 7f05737fe700 1 --2- 172.21.15.143:0/3504726873 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f05740d3690 0x7f05740cca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:51.124 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.115+0000 7f05737fe700 1 -- 172.21.15.143:0/3504726873 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05640094a0 con 0x7f057415dc80 2023-11-08T19:24:51.124 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.115+0000 7f05727fc700 1 --2- 172.21.15.143:0/3504726873 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f05740d3690 0x7f05740cca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:24:51.125 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.115+0000 7f05737fe700 1 --2- 172.21.15.143:0/3504726873 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f057415dc80 0x7f05741f3120 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f056c00cb10 tx=0x7f056c00ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:51.125 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.115+0000 7f055b7fe700 1 -- 172.21.15.143:0/3504726873 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f056c003c00 con 0x7f057415dc80 2023-11-08T19:24:51.125 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.115+0000 7f055b7fe700 1 -- 172.21.15.143:0/3504726873 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f056c003d60 con 0x7f057415dc80 2023-11-08T19:24:51.125 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.115+0000 7f055b7fe700 1 -- 172.21.15.143:0/3504726873 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f056c00ab40 con 0x7f057415dc80 2023-11-08T19:24:51.125 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.115+0000 7f05795f3700 1 -- 172.21.15.143:0/3504726873 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05740cd550 con 0x7f057415dc80 2023-11-08T19:24:51.126 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.115+0000 7f05795f3700 1 -- 172.21.15.143:0/3504726873 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05740c87d0 con 0x7f057415dc80 2023-11-08T19:24:51.126 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.116+0000 7f05795f3700 1 -- 172.21.15.143:0/3504726873 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f057404eee0 con 0x7f057415dc80 2023-11-08T19:24:51.126 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.117+0000 7f055b7fe700 1 -- 172.21.15.143:0/3504726873 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f056c00aca0 con 0x7f057415dc80 2023-11-08T19:24:51.126 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.118+0000 7f055b7fe700 1 --2- 172.21.15.143:0/3504726873 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f056006af00 0x7f056006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:51.126 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.118+0000 7f055b7fe700 1 -- 172.21.15.143:0/3504726873 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f056c08a080 con 0x7f057415dc80 2023-11-08T19:24:51.127 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.118+0000 7f0572ffd700 1 --2- 172.21.15.143:0/3504726873 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f056006af00 0x7f056006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:51.127 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.118+0000 7f0572ffd700 1 --2- 172.21.15.143:0/3504726873 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f056006af00 0x7f056006d3c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f05680011e0 tx=0x7f0568009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:51.127 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.121+0000 7f055b7fe700 1 -- 172.21.15.143:0/3504726873 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f056c0551a0 con 0x7f057415dc80 2023-11-08T19:24:51.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.311+0000 7f05795f3700 1 -- 172.21.15.143:0/3504726873 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f05740cc080 con 0x7f057415dc80 2023-11-08T19:24:51.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.312+0000 7f055b7fe700 1 -- 172.21.15.143:0/3504726873 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+16566 (secure 0 0 0) 0x7f056c0587c0 con 0x7f057415dc80 2023-11-08T19:24:51.314 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:24:51.314 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":19,"fsid":"968483af-77fc-4068-ad7c-5c2a1a4613d7","created":"2023-11-08T19:24:32.826484+0000","modified":"2023-11-08T19:24:49.995234+0000","last_up_change":"2023-11-08T19:24:43.573880+0000","last_in_change":"2023-11-08T19:24:35.314525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:44.551530+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-08T19:24:48.256781+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-08T19:24:48.954990+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bae98ec1-d94e-403d-a220-a522a53ea6f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6809","nonce":1487830566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6811","nonce":1487830566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6815","nonce":1487830566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6813","nonce":1487830566}]},"public_addr":"172.21.15.143:6809/1487830566","cluster_addr":"172.21.15.143:6811/1487830566","heartbeat_back_addr":"172.21.15.143:6815/1487830566","heartbeat_front_addr":"172.21.15.143:6813/1487830566","state":["exists","up"]},{"osd":1,"uuid":"5cb6538a-1ccb-4322-8633-09f66af0b7e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6801","nonce":787341397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6803","nonce":787341397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6807","nonce":787341397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6805","nonce":787341397}]},"public_addr":"172.21.15.143:6801/787341397","cluster_addr":"172.21.15.143:6803/787341397","heartbeat_back_addr":"172.21.15.143:6807/787341397","heartbeat_front_addr":"172.21.15.143:6805/787341397","state":["exists","up"]},{"osd":2,"uuid":"cdbf522c-60ef-4d60-9653-8dcb529c101a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6822","nonce":2301325597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6826","nonce":2301325597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6831","nonce":2301325597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6829","nonce":2301325597}]},"public_addr":"172.21.15.143:6822/2301325597","cluster_addr":"172.21.15.143:6826/2301325597","heartbeat_back_addr":"172.21.15.143:6831/2301325597","heartbeat_front_addr":"172.21.15.143:6829/2301325597","state":["exists","up"]},{"osd":3,"uuid":"4e2783cf-ebb9-4dd8-8245-bc1d17ce4325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6817","nonce":2021504428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6819","nonce":2021504428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6827","nonce":2021504428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6823","nonce":2021504428}]},"public_addr":"172.21.15.143:6817/2021504428","cluster_addr":"172.21.15.143:6819/2021504428","heartbeat_back_addr":"172.21.15.143:6827/2021504428","heartbeat_front_addr":"172.21.15.143:6823/2021504428","state":["exists","up"]},{"osd":4,"uuid":"691256fa-9b77-49b9-a346-7ae1adf0ca80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6801","nonce":1993380595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6803","nonce":1993380595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6807","nonce":1993380595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6805","nonce":1993380595}]},"public_addr":"172.21.15.174:6801/1993380595","cluster_addr":"172.21.15.174:6803/1993380595","heartbeat_back_addr":"172.21.15.174:6807/1993380595","heartbeat_front_addr":"172.21.15.174:6805/1993380595","state":["exists","up"]},{"osd":5,"uuid":"756fc352-48c9-46ce-b52e-0541fe6f8525","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6809","nonce":1777923303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6811","nonce":1777923303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6815","nonce":1777923303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6813","nonce":1777923303}]},"public_addr":"172.21.15.174:6809/1777923303","cluster_addr":"172.21.15.174:6811/1777923303","heartbeat_back_addr":"172.21.15.174:6815/1777923303","heartbeat_front_addr":"172.21.15.174:6813/1777923303","state":["exists","up"]},{"osd":6,"uuid":"fab2ac2b-ed97-4cde-9825-b480eabf8b8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6825","nonce":2755130316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6827","nonce":2755130316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6831","nonce":2755130316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6829","nonce":2755130316}]},"public_addr":"172.21.15.174:6825/2755130316","cluster_addr":"172.21.15.174:6827/2755130316","heartbeat_back_addr":"172.21.15.174:6831/2755130316","heartbeat_front_addr":"172.21.15.174:6829/2755130316","state":["exists","up"]},{"osd":7,"uuid":"3de9b797-2ab6-4d41-a863-7e7d638fd703","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6817","nonce":2847141411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6819","nonce":2847141411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6823","nonce":2847141411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6821","nonce":2847141411}]},"public_addr":"172.21.15.174:6817/2847141411","cluster_addr":"172.21.15.174:6819/2847141411","heartbeat_back_addr":"172.21.15.174:6823/2847141411","heartbeat_front_addr":"172.21.15.174:6821/2847141411","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.477076+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.496890+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.539305+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.454673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.538881+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.521294+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.520712+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.540404+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:24:51.316 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.315+0000 7f05795f3700 1 -- 172.21.15.143:0/3504726873 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f056006af00 msgr2=0x7f056006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:51.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.315+0000 7f05795f3700 1 --2- 172.21.15.143:0/3504726873 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f056006af00 0x7f056006d3c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f05680011e0 tx=0x7f0568009040 comp rx=0 tx=0).stop 2023-11-08T19:24:51.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.315+0000 7f05795f3700 1 -- 172.21.15.143:0/3504726873 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f057415dc80 msgr2=0x7f05741f3120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:51.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.315+0000 7f05795f3700 1 --2- 172.21.15.143:0/3504726873 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f057415dc80 0x7f05741f3120 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f056c00cb10 tx=0x7f056c00ced0 comp rx=0 tx=0).stop 2023-11-08T19:24:51.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.315+0000 7f05795f3700 1 -- 172.21.15.143:0/3504726873 shutdown_connections 2023-11-08T19:24:51.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.315+0000 7f05795f3700 1 --2- 172.21.15.143:0/3504726873 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f056006af00 0x7f056006d3c0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:51.318 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.315+0000 7f05795f3700 1 --2- 172.21.15.143:0/3504726873 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f05740cf090 0x7f05740cc540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:51.318 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.315+0000 7f05795f3700 1 --2- 172.21.15.143:0/3504726873 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f057415dc80 0x7f05741f3120 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:51.318 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.315+0000 7f05795f3700 1 --2- 172.21.15.143:0/3504726873 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f05740d3690 0x7f05740cca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:51.318 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.315+0000 7f05795f3700 1 -- 172.21.15.143:0/3504726873 >> 172.21.15.143:0/3504726873 conn(0x7f05740668e0 msgr2=0x7f057415bf00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:24:51.318 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.315+0000 7f05795f3700 1 -- 172.21.15.143:0/3504726873 shutdown_connections 2023-11-08T19:24:51.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.315+0000 7f05795f3700 1 -- 172.21.15.143:0/3504726873 wait complete. 2023-11-08T19:24:51.328 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-08T19:24:51.451 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.450+0000 7f8f78ed3700 1 Processor -- start 2023-11-08T19:24:51.452 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.450+0000 7f8f78ed3700 1 -- start start 2023-11-08T19:24:51.455 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.451+0000 7f8f78ed3700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8f7406b240 0x7f8f740caff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:51.455 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.451+0000 7f8f78ed3700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f8f740cb530 0x7f8f740b22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:51.455 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.451+0000 7f8f78ed3700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f74161890 con 0x7f8f7415dba0 2023-11-08T19:24:51.455 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.451+0000 7f8f78ed3700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f7406cd20 con 0x7f8f740cb530 2023-11-08T19:24:51.456 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.451+0000 7f8f78ed3700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f7406b740 con 0x7f8f7406b240 2023-11-08T19:24:51.456 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.451+0000 7f8f7259c700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f8f7415dba0 0x7f8f7406c610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:44742/0 (socket says 172.21.15.143:44742) 2023-11-08T19:24:51.456 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.451+0000 7f8f7259c700 1 -- 172.21.15.143:0/898805387 learned_addr learned my addr 172.21.15.143:0/898805387 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:24:51.456 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.451+0000 7f8f72d9d700 1 --2- 172.21.15.143:0/898805387 >> v2:172.21.15.174:3300/0 conn(0x7f8f740cb530 0x7f8f740b22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:51.456 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.451+0000 7f8f71d9b700 1 --2- 172.21.15.143:0/898805387 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8f7406b240 0x7f8f740caff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:51.457 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.451+0000 7f8f7159a700 1 -- 172.21.15.143:0/898805387 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 839679071 0 0) 0x7f8f74161890 con 0x7f8f7415dba0 2023-11-08T19:24:51.457 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.451+0000 7f8f7159a700 1 -- 172.21.15.143:0/898805387 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f5c0035b0 con 0x7f8f7415dba0 2023-11-08T19:24:51.457 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.451+0000 7f8f72d9d700 1 -- 172.21.15.143:0/898805387 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8f7406b240 msgr2=0x7f8f740caff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:24:51.457 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.451+0000 7f8f72d9d700 1 --2- 172.21.15.143:0/898805387 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8f7406b240 0x7f8f740caff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:51.457 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.451+0000 7f8f72d9d700 1 -- 172.21.15.143:0/898805387 >> v1:172.21.15.143:6789/0 conn(0x7f8f7415dba0 legacy=0x7f8f7406c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:51.458 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.451+0000 7f8f72d9d700 1 -- 172.21.15.143:0/898805387 --> v2:172.21.15.174:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f7406ba20 con 0x7f8f740cb530 2023-11-08T19:24:51.458 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.451+0000 7f8f72d9d700 1 --2- 172.21.15.143:0/898805387 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8f740cb530 0x7f8f740b22e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c009d90 tx=0x7f8f6c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b7665560614aaae server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:51.458 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.452+0000 7f8f7159a700 1 -- 172.21.15.143:0/898805387 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f6c00e070 con 0x7f8f740cb530 2023-11-08T19:24:51.458 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.452+0000 7f8f7159a700 1 -- 172.21.15.143:0/898805387 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f6c00bb20 con 0x7f8f740cb530 2023-11-08T19:24:51.458 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.452+0000 7f8f7159a700 1 -- 172.21.15.143:0/898805387 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f6c012430 con 0x7f8f740cb530 2023-11-08T19:24:51.459 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.452+0000 7f8f78ed3700 1 -- 172.21.15.143:0/898805387 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8f740cb530 msgr2=0x7f8f740b22e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:51.459 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.452+0000 7f8f78ed3700 1 --2- 172.21.15.143:0/898805387 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8f740cb530 0x7f8f740b22e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c009d90 tx=0x7f8f6c00b330 comp rx=0 tx=0).stop 2023-11-08T19:24:51.459 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.452+0000 7f8f78ed3700 1 -- 172.21.15.143:0/898805387 shutdown_connections 2023-11-08T19:24:51.459 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.452+0000 7f8f78ed3700 1 --2- 172.21.15.143:0/898805387 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8f740cb530 0x7f8f740b22e0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:51.459 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.452+0000 7f8f78ed3700 1 --2- 172.21.15.143:0/898805387 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8f7406b240 0x7f8f740caff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:51.460 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.452+0000 7f8f78ed3700 1 -- 172.21.15.143:0/898805387 >> 172.21.15.143:0/898805387 conn(0x7f8f740cd2e0 msgr2=0x7f8f740cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:24:51.460 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.452+0000 7f8f78ed3700 1 -- 172.21.15.143:0/898805387 shutdown_connections 2023-11-08T19:24:51.460 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.452+0000 7f8f78ed3700 1 -- 172.21.15.143:0/898805387 wait complete. 2023-11-08T19:24:51.460 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.454+0000 7f8f78ed3700 1 Processor -- start 2023-11-08T19:24:51.460 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.454+0000 7f8f78ed3700 1 -- start start 2023-11-08T19:24:51.463 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.454+0000 7f8f78ed3700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8f7406b240 0x7f8f7415c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:51.463 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.454+0000 7f8f78ed3700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f740cb530 0x7f8f740c4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:51.463 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.454+0000 7f8f78ed3700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8f7415dba0 0x7f8f740c48d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:51.464 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.454+0000 7f8f78ed3700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f7415d2a0 con 0x7f8f740cb530 2023-11-08T19:24:51.464 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.454+0000 7f8f78ed3700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f740c4f00 con 0x7f8f7406b240 2023-11-08T19:24:51.464 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.454+0000 7f8f78ed3700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f740c51e0 con 0x7f8f7415dba0 2023-11-08T19:24:51.464 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.454+0000 7f8f71d9b700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f740cb530 0x7f8f740c4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:51.464 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.454+0000 7f8f71d9b700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f740cb530 0x7f8f740c4390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48596/0 (socket says 172.21.15.143:48596) 2023-11-08T19:24:51.465 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.454+0000 7f8f71d9b700 1 -- 172.21.15.143:0/26803860 learned_addr learned my addr 172.21.15.143:0/26803860 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:24:51.465 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.455+0000 7f8f7259c700 1 --2- 172.21.15.143:0/26803860 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8f7406b240 0x7f8f7415c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:51.465 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.455+0000 7f8f72d9d700 1 --2- 172.21.15.143:0/26803860 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8f7415dba0 0x7f8f740c48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:51.465 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.455+0000 7f8f71d9b700 1 -- 172.21.15.143:0/26803860 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8f7415dba0 msgr2=0x7f8f740c48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:51.465 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.455+0000 7f8f71d9b700 1 --2- 172.21.15.143:0/26803860 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8f7415dba0 0x7f8f740c48d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:51.466 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.455+0000 7f8f71d9b700 1 -- 172.21.15.143:0/26803860 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8f7406b240 msgr2=0x7f8f7415c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:51.466 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.455+0000 7f8f71d9b700 1 --2- 172.21.15.143:0/26803860 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8f7406b240 0x7f8f7415c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:51.466 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.455+0000 7f8f71d9b700 1 -- 172.21.15.143:0/26803860 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f6c009a70 con 0x7f8f740cb530 2023-11-08T19:24:51.466 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.455+0000 7f8f7259c700 1 --2- 172.21.15.143:0/26803860 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8f7406b240 0x7f8f7415c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:24:51.466 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.455+0000 7f8f71d9b700 1 --2- 172.21.15.143:0/26803860 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f740cb530 0x7f8f740c4390 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8f6800e9b0 tx=0x7f8f6800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:51.467 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.455+0000 7f8f62ffd700 1 -- 172.21.15.143:0/26803860 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f68009710 con 0x7f8f740cb530 2023-11-08T19:24:51.467 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.455+0000 7f8f62ffd700 1 -- 172.21.15.143:0/26803860 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f6800c850 con 0x7f8f740cb530 2023-11-08T19:24:51.467 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.455+0000 7f8f78ed3700 1 -- 172.21.15.143:0/26803860 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f740c54c0 con 0x7f8f740cb530 2023-11-08T19:24:51.467 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.455+0000 7f8f62ffd700 1 -- 172.21.15.143:0/26803860 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f6800cbc0 con 0x7f8f740cb530 2023-11-08T19:24:51.467 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.455+0000 7f8f78ed3700 1 -- 172.21.15.143:0/26803860 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f740c5a80 con 0x7f8f740cb530 2023-11-08T19:24:51.467 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.456+0000 7f8f78ed3700 1 -- 172.21.15.143:0/26803860 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f7404eee0 con 0x7f8f740cb530 2023-11-08T19:24:51.468 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.456+0000 7f8f62ffd700 1 -- 172.21.15.143:0/26803860 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8f6800cd20 con 0x7f8f740cb530 2023-11-08T19:24:51.468 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.457+0000 7f8f62ffd700 1 --2- 172.21.15.143:0/26803860 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f8f5406af00 0x7f8f5406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:51.468 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.457+0000 7f8f62ffd700 1 -- 172.21.15.143:0/26803860 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8f68089780 con 0x7f8f740cb530 2023-11-08T19:24:51.468 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.457+0000 7f8f7259c700 1 --2- 172.21.15.143:0/26803860 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f8f5406af00 0x7f8f5406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:51.468 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.458+0000 7f8f7259c700 1 --2- 172.21.15.143:0/26803860 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f8f5406af00 0x7f8f5406d3c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8f640011e0 tx=0x7f8f64009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:51.469 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.461+0000 7f8f62ffd700 1 -- 172.21.15.143:0/26803860 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8f68054830 con 0x7f8f740cb530 2023-11-08T19:24:51.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:51.702+0000 7f8f78ed3700 1 -- 172.21.15.143:0/26803860 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f8f740b67e0 con 0x7f8f740cb530 2023-11-08T19:24:52.635 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:52.634+0000 7f8f62ffd700 1 -- 172.21.15.143:0/26803860 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) v1 ==== 118+0+0 (secure 0 0 0) 0x7f8f68057e50 con 0x7f8f740cb530 2023-11-08T19:24:52.638 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:52.636+0000 7f8f78ed3700 1 -- 172.21.15.143:0/26803860 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f8f5406af00 msgr2=0x7f8f5406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:52.638 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:52.636+0000 7f8f78ed3700 1 --2- 172.21.15.143:0/26803860 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f8f5406af00 0x7f8f5406d3c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8f640011e0 tx=0x7f8f64009040 comp rx=0 tx=0).stop 2023-11-08T19:24:52.639 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:52.636+0000 7f8f78ed3700 1 -- 172.21.15.143:0/26803860 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f740cb530 msgr2=0x7f8f740c4390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:52.639 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:52.636+0000 7f8f78ed3700 1 --2- 172.21.15.143:0/26803860 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f740cb530 0x7f8f740c4390 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8f6800e9b0 tx=0x7f8f6800ed70 comp rx=0 tx=0).stop 2023-11-08T19:24:52.639 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:52.636+0000 7f8f78ed3700 1 -- 172.21.15.143:0/26803860 shutdown_connections 2023-11-08T19:24:52.639 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:52.636+0000 7f8f78ed3700 1 --2- 172.21.15.143:0/26803860 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f8f5406af00 0x7f8f5406d3c0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:52.639 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:52.637+0000 7f8f78ed3700 1 --2- 172.21.15.143:0/26803860 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8f7415dba0 0x7f8f740c48d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:52.640 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:52.637+0000 7f8f78ed3700 1 --2- 172.21.15.143:0/26803860 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8f7406b240 0x7f8f7415c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:52.640 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:52.637+0000 7f8f78ed3700 1 --2- 172.21.15.143:0/26803860 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f740cb530 0x7f8f740c4390 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:52.640 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:52.637+0000 7f8f78ed3700 1 -- 172.21.15.143:0/26803860 >> 172.21.15.143:0/26803860 conn(0x7f8f740cd2e0 msgr2=0x7f8f7406c2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:24:52.640 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:52.637+0000 7f8f78ed3700 1 -- 172.21.15.143:0/26803860 shutdown_connections 2023-11-08T19:24:52.640 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:52.637+0000 7f8f78ed3700 1 -- 172.21.15.143:0/26803860 wait complete. 2023-11-08T19:24:52.653 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-08T19:24:52.653 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-08T19:24:52.781 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:52.779+0000 7fafe7779700 1 Processor -- start 2023-11-08T19:24:52.781 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:52.780+0000 7fafe7779700 1 -- start start 2023-11-08T19:24:52.783 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:52.780+0000 7fafe7779700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fafe015bb20 0x7fafe00b2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:52.783 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:52.780+0000 7fafe7779700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fafe00d1d80 0x7fafe00cc4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:52.784 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:52.780+0000 7fafe7779700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafe00d0420 con 0x7fafe00b1730 2023-11-08T19:24:52.784 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:52.780+0000 7fafe7779700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafe00cf600 con 0x7fafe015bb20 2023-11-08T19:24:52.784 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:52.780+0000 7fafe7779700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafe00cf810 con 0x7fafe00d1d80 2023-11-08T19:24:52.784 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:52.781+0000 7fafe5515700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fafe015bb20 0x7fafe00b2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:52.784 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:52.781+0000 7fafe4d14700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fafe00b1730 0x7fafe00d1670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:44756/0 (socket says 172.21.15.143:44756) 2023-11-08T19:24:52.785 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:52.781+0000 7fafe4d14700 1 -- 172.21.15.143:0/2041011583 learned_addr learned my addr 172.21.15.143:0/2041011583 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:24:52.785 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:52.781+0000 7fafe5515700 1 -- 172.21.15.143:0/2041011583 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fafe00d1d80 msgr2=0x7fafe00cc4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:24:52.785 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:52.781+0000 7fafe5515700 1 --2- 172.21.15.143:0/2041011583 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fafe00d1d80 0x7fafe00cc4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:52.785 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:52.781+0000 7fafe5515700 1 -- 172.21.15.143:0/2041011583 >> v1:172.21.15.143:6789/0 conn(0x7fafe00b1730 legacy=0x7fafe00d1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:52.785 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:52.781+0000 7fafe5515700 1 -- 172.21.15.143:0/2041011583 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafe00cfaf0 con 0x7fafe015bb20 2023-11-08T19:24:52.786 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:52.781+0000 7fafd7fff700 1 -- 172.21.15.143:0/2041011583 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 537534209 0 0) 0x7fafe00d0420 con 0x7fafe00b1730 2023-11-08T19:24:52.786 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:52.781+0000 7fafe5515700 1 --2- 172.21.15.143:0/2041011583 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fafe015bb20 0x7fafe00b2670 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fafd0009840 tx=0x7fafd0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5422924082d6a50b server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:52.786 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:52.781+0000 7fafd7fff700 1 -- 172.21.15.143:0/2041011583 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafd0011830 con 0x7fafe015bb20 2023-11-08T19:24:52.786 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:52.781+0000 7fafd7fff700 1 -- 172.21.15.143:0/2041011583 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafd0011990 con 0x7fafe015bb20 2023-11-08T19:24:52.786 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:52.781+0000 7fafd7fff700 1 -- 172.21.15.143:0/2041011583 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafd0011d20 con 0x7fafe015bb20 2023-11-08T19:24:52.787 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:52.782+0000 7fafe7779700 1 -- 172.21.15.143:0/2041011583 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fafe015bb20 msgr2=0x7fafe00b2670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:52.787 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:52.782+0000 7fafe7779700 1 --2- 172.21.15.143:0/2041011583 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fafe015bb20 0x7fafe00b2670 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fafd0009840 tx=0x7fafd0009b50 comp rx=0 tx=0).stop 2023-11-08T19:24:52.787 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:52.782+0000 7fafe7779700 1 -- 172.21.15.143:0/2041011583 shutdown_connections 2023-11-08T19:24:52.787 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:52.782+0000 7fafe7779700 1 --2- 172.21.15.143:0/2041011583 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fafe00d1d80 0x7fafe00cc4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:52.787 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:52.782+0000 7fafe7779700 1 --2- 172.21.15.143:0/2041011583 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fafe015bb20 0x7fafe00b2670 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:52.787 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:52.782+0000 7fafe7779700 1 -- 172.21.15.143:0/2041011583 >> 172.21.15.143:0/2041011583 conn(0x7fafe008b000 msgr2=0x7fafe009ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:24:52.788 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:52.782+0000 7fafe7779700 1 -- 172.21.15.143:0/2041011583 shutdown_connections 2023-11-08T19:24:52.788 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:52.782+0000 7fafe7779700 1 -- 172.21.15.143:0/2041011583 wait complete. 2023-11-08T19:24:52.790 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:52.783+0000 7fafe7779700 1 Processor -- start 2023-11-08T19:24:52.791 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:52.783+0000 7fafe7779700 1 -- start start 2023-11-08T19:24:52.791 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:52.784+0000 7fafe7779700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fafe00b1730 0x7fafe00c4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:52.791 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:52.784+0000 7fafe7779700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fafe00d1d80 0x7fafe00ccaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:52.791 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:52.784+0000 7fafe7779700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fafe015bb20 0x7fafe00ccfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:52.791 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:52.784+0000 7fafe7779700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafe006c860 con 0x7fafe00b1730 2023-11-08T19:24:52.792 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:52.784+0000 7fafe7779700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafe006cb40 con 0x7fafe015bb20 2023-11-08T19:24:52.792 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:52.784+0000 7fafe7779700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafe006ce20 con 0x7fafe00d1d80 2023-11-08T19:24:52.792 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:52.784+0000 7fafe5515700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fafe00b1730 0x7fafe00c4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:52.792 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:52.784+0000 7fafe5515700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fafe00b1730 0x7fafe00c4730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48608/0 (socket says 172.21.15.143:48608) 2023-11-08T19:24:52.792 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:52.784+0000 7fafe5515700 1 -- 172.21.15.143:0/1150221984 learned_addr learned my addr 172.21.15.143:0/1150221984 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:24:52.792 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:52.784+0000 7fafe5515700 1 -- 172.21.15.143:0/1150221984 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fafe00d1d80 msgr2=0x7fafe00ccaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:52.793 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:52.784+0000 7fafe5515700 1 --2- 172.21.15.143:0/1150221984 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fafe00d1d80 0x7fafe00ccaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:52.793 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:52.784+0000 7fafe5515700 1 -- 172.21.15.143:0/1150221984 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fafe015bb20 msgr2=0x7fafe00ccfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:52.793 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:52.784+0000 7fafe5515700 1 --2- 172.21.15.143:0/1150221984 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fafe015bb20 0x7fafe00ccfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:52.793 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:52.784+0000 7fafe5515700 1 -- 172.21.15.143:0/1150221984 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafd00094a0 con 0x7fafe00b1730 2023-11-08T19:24:52.793 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:52.784+0000 7fafe5515700 1 --2- 172.21.15.143:0/1150221984 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fafe00b1730 0x7fafe00c4730 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fafd0014040 tx=0x7fafd0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:52.794 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:52.784+0000 7fafd5ffb700 1 -- 172.21.15.143:0/1150221984 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafd00045f0 con 0x7fafe00b1730 2023-11-08T19:24:52.794 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:52.784+0000 7fafd5ffb700 1 -- 172.21.15.143:0/1150221984 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafd001e070 con 0x7fafe00b1730 2023-11-08T19:24:52.794 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:52.784+0000 7fafe7779700 1 -- 172.21.15.143:0/1150221984 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafe006d100 con 0x7fafe00b1730 2023-11-08T19:24:52.794 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:52.784+0000 7fafd5ffb700 1 -- 172.21.15.143:0/1150221984 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafd0003b00 con 0x7fafe00b1730 2023-11-08T19:24:52.794 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:52.785+0000 7fafe7779700 1 -- 172.21.15.143:0/1150221984 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafe00cd520 con 0x7fafe00b1730 2023-11-08T19:24:52.795 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:52.785+0000 7fafe7779700 1 -- 172.21.15.143:0/1150221984 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafe00d1ad0 con 0x7fafe00b1730 2023-11-08T19:24:52.795 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:52.786+0000 7fafd5ffb700 1 -- 172.21.15.143:0/1150221984 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fafd0003c60 con 0x7fafe00b1730 2023-11-08T19:24:52.795 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:52.786+0000 7fafd5ffb700 1 --2- 172.21.15.143:0/1150221984 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fafc406af00 0x7fafc406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:52.795 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:52.786+0000 7fafd5ffb700 1 -- 172.21.15.143:0/1150221984 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fafd008a740 con 0x7fafe00b1730 2023-11-08T19:24:52.795 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:52.787+0000 7fafe4d14700 1 --2- 172.21.15.143:0/1150221984 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fafc406af00 0x7fafc406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:52.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:52.787+0000 7fafe4d14700 1 --2- 172.21.15.143:0/1150221984 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fafc406af00 0x7fafc406d3c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fafdc0011e0 tx=0x7fafdc0079b0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:52.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:52.791+0000 7fafd5ffb700 1 -- 172.21.15.143:0/1150221984 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fafd0055880 con 0x7fafe00b1730 2023-11-08T19:24:53.027 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:53.025+0000 7fafe7779700 1 -- 172.21.15.143:0/1150221984 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fafe004eee0 con 0x7fafe00b1730 2023-11-08T19:24:53.642 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:53.640+0000 7fafd5ffb700 1 -- 172.21.15.143:0/1150221984 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) v1 ==== 122+0+0 (secure 0 0 0) 0x7fafd0020020 con 0x7fafe00b1730 2023-11-08T19:24:53.645 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:53.644+0000 7fafe7779700 1 -- 172.21.15.143:0/1150221984 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fafc406af00 msgr2=0x7fafc406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:53.645 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:53.644+0000 7fafe7779700 1 --2- 172.21.15.143:0/1150221984 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fafc406af00 0x7fafc406d3c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fafdc0011e0 tx=0x7fafdc0079b0 comp rx=0 tx=0).stop 2023-11-08T19:24:53.646 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:53.644+0000 7fafe7779700 1 -- 172.21.15.143:0/1150221984 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fafe00b1730 msgr2=0x7fafe00c4730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:53.646 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:53.644+0000 7fafe7779700 1 --2- 172.21.15.143:0/1150221984 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fafe00b1730 0x7fafe00c4730 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fafd0014040 tx=0x7fafd0004750 comp rx=0 tx=0).stop 2023-11-08T19:24:53.646 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:53.644+0000 7fafe7779700 1 -- 172.21.15.143:0/1150221984 shutdown_connections 2023-11-08T19:24:53.646 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:53.644+0000 7fafe7779700 1 --2- 172.21.15.143:0/1150221984 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fafc406af00 0x7fafc406d3c0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:53.647 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:53.644+0000 7fafe7779700 1 --2- 172.21.15.143:0/1150221984 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fafe015bb20 0x7fafe00ccfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:53.647 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:53.644+0000 7fafe7779700 1 --2- 172.21.15.143:0/1150221984 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fafe00b1730 0x7fafe00c4730 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:53.647 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:53.644+0000 7fafe7779700 1 --2- 172.21.15.143:0/1150221984 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fafe00d1d80 0x7fafe00ccaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:53.647 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:53.644+0000 7fafe7779700 1 -- 172.21.15.143:0/1150221984 >> 172.21.15.143:0/1150221984 conn(0x7fafe008b000 msgr2=0x7fafe00b3de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:24:53.647 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:53.644+0000 7fafe7779700 1 -- 172.21.15.143:0/1150221984 shutdown_connections 2023-11-08T19:24:53.648 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:53.644+0000 7fafe7779700 1 -- 172.21.15.143:0/1150221984 wait complete. 2023-11-08T19:24:53.658 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-08T19:24:53.658 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:24:53.780 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:53.779+0000 7fb06ac3d700 1 Processor -- start 2023-11-08T19:24:53.780 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:53.779+0000 7fb06ac3d700 1 -- start start 2023-11-08T19:24:53.784 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:53.779+0000 7fb06ac3d700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb06415dd00 0x7fb0640cca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:53.785 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:53.780+0000 7fb06ac3d700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0640ccfb0 con 0x7fb0641618f0 2023-11-08T19:24:53.785 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:53.780+0000 7fb06ac3d700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb064161df0 con 0x7fb06406b320 2023-11-08T19:24:53.785 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:53.780+0000 7fb06ac3d700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0640cad50 con 0x7fb06415dd00 2023-11-08T19:24:53.785 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:53.780+0000 7fb063fff700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fb0641618f0 0x7fb06406ac10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:44772/0 (socket says 172.21.15.143:44772) 2023-11-08T19:24:53.785 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:53.780+0000 7fb063fff700 1 -- 172.21.15.143:0/1308488673 learned_addr learned my addr 172.21.15.143:0/1308488673 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:24:53.786 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:53.780+0000 7fb0637fe700 1 -- 172.21.15.143:0/1308488673 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2538109328 0 0) 0x7fb0640ccfb0 con 0x7fb0641618f0 2023-11-08T19:24:53.786 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:53.780+0000 7fb0637fe700 1 -- 172.21.15.143:0/1308488673 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb0540035b0 con 0x7fb0641618f0 2023-11-08T19:24:53.789 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:53.780+0000 7fb0689d9700 1 --2- 172.21.15.143:0/1308488673 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb06415dd00 0x7fb0640cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:53.789 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:53.780+0000 7fb0637fe700 1 -- 172.21.15.143:0/1308488673 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2138588192 0 0) 0x7fb0540035b0 con 0x7fb0641618f0 2023-11-08T19:24:53.790 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:53.780+0000 7fb0637fe700 1 -- 172.21.15.143:0/1308488673 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb06415dd00 msgr2=0x7fb0640cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:24:53.790 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:53.780+0000 7fb0637fe700 1 --2- 172.21.15.143:0/1308488673 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb06415dd00 0x7fb0640cca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:53.790 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:53.780+0000 7fb0637fe700 1 -- 172.21.15.143:0/1308488673 >> v1:172.21.15.174:6789/0 conn(0x7fb06406b320 legacy=0x7fb0640cb1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:53.790 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:53.780+0000 7fb0637fe700 1 -- 172.21.15.143:0/1308488673 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0641620d0 con 0x7fb0641618f0 2023-11-08T19:24:53.791 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:53.780+0000 7fb0637fe700 1 -- 172.21.15.143:0/1308488673 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4147756303 0 0) 0x7fb050001670 con 0x7fb0641618f0 2023-11-08T19:24:53.791 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:53.780+0000 7fb0689d9700 1 --2- 172.21.15.143:0/1308488673 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb06415dd00 0x7fb0640cca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:24:53.791 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:53.780+0000 7fb0637fe700 1 -- 172.21.15.143:0/1308488673 >> v1:172.21.15.143:6789/0 conn(0x7fb0641618f0 legacy=0x7fb06406ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:53.791 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:53.780+0000 7fb0637fe700 1 --2- 172.21.15.143:0/1308488673 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb054003a30 0x7fb054003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:53.791 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:53.780+0000 7fb0637fe700 1 --2- 172.21.15.143:0/1308488673 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb06415dd00 0x7fb054004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:53.792 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:53.780+0000 7fb0637fe700 1 --2- 172.21.15.143:0/1308488673 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb054004d90 0x7fb054005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:53.792 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:53.780+0000 7fb0637fe700 1 -- 172.21.15.143:0/1308488673 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb05400baf0 con 0x7fb054004d90 2023-11-08T19:24:53.792 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:53.780+0000 7fb0637fe700 1 -- 172.21.15.143:0/1308488673 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb05400bc60 con 0x7fb054003a30 2023-11-08T19:24:53.792 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:53.780+0000 7fb0637fe700 1 -- 172.21.15.143:0/1308488673 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb05400be20 con 0x7fb06415dd00 2023-11-08T19:24:53.793 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:53.780+0000 7fb0691da700 1 --2- 172.21.15.143:0/1308488673 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb054004d90 0x7fb054005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:53.793 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:53.780+0000 7fb0689d9700 1 --2- 172.21.15.143:0/1308488673 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb054003a30 0x7fb054003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:53.793 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:53.780+0000 7fb063fff700 1 --2- 172.21.15.143:0/1308488673 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb06415dd00 0x7fb054004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:53.793 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:53.781+0000 7fb0691da700 1 -- 172.21.15.143:0/1308488673 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb06415dd00 msgr2=0x7fb054004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:24:53.794 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:53.781+0000 7fb0691da700 1 --2- 172.21.15.143:0/1308488673 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb06415dd00 0x7fb054004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:53.794 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:53.781+0000 7fb0691da700 1 -- 172.21.15.143:0/1308488673 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb054003a30 msgr2=0x7fb054003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:24:53.794 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:53.781+0000 7fb0691da700 1 --2- 172.21.15.143:0/1308488673 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb054003a30 0x7fb054003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:53.794 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:53.781+0000 7fb0691da700 1 -- 172.21.15.143:0/1308488673 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0540035b0 con 0x7fb054004d90 2023-11-08T19:24:53.794 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:53.781+0000 7fb0691da700 1 --2- 172.21.15.143:0/1308488673 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb054004d90 0x7fb054005210 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb05c00c160 tx=0x7fb054005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db197fb161ba2f0a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:53.795 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:53.781+0000 7fb0637fe700 1 -- 172.21.15.143:0/1308488673 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb05c001db0 con 0x7fb054004d90 2023-11-08T19:24:53.795 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:53.781+0000 7fb0637fe700 1 -- 172.21.15.143:0/1308488673 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb05c00cca0 con 0x7fb054004d90 2023-11-08T19:24:53.795 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:53.781+0000 7fb0637fe700 1 -- 172.21.15.143:0/1308488673 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb05c003c40 con 0x7fb054004d90 2023-11-08T19:24:53.795 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:53.781+0000 7fb06ac3d700 1 -- 172.21.15.143:0/1308488673 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb054004d90 msgr2=0x7fb054005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:53.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:53.781+0000 7fb06ac3d700 1 --2- 172.21.15.143:0/1308488673 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb054004d90 0x7fb054005210 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb05c00c160 tx=0x7fb054005880 comp rx=0 tx=0).stop 2023-11-08T19:24:53.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:53.781+0000 7fb0691da700 1 -- 172.21.15.143:0/1308488673 reap_dead start 2023-11-08T19:24:53.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:53.781+0000 7fb06ac3d700 1 -- 172.21.15.143:0/1308488673 shutdown_connections 2023-11-08T19:24:53.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:53.781+0000 7fb06ac3d700 1 -- 172.21.15.143:0/1308488673 >> 172.21.15.143:0/1308488673 conn(0x7fb064099ab0 msgr2=0x7fb0640d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:24:53.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:53.782+0000 7fb06ac3d700 1 -- 172.21.15.143:0/1308488673 shutdown_connections 2023-11-08T19:24:53.797 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:53.782+0000 7fb06ac3d700 1 -- 172.21.15.143:0/1308488673 wait complete. 2023-11-08T19:24:53.797 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:53.783+0000 7fb06ac3d700 1 Processor -- start 2023-11-08T19:24:53.797 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:53.783+0000 7fb06ac3d700 1 -- start start 2023-11-08T19:24:53.800 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:53.784+0000 7fb06ac3d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb0640b02e0 0x7fb064157a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:53.800 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:53.784+0000 7fb06ac3d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb064156b10 0x7fb0641f29f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:53.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:53.784+0000 7fb06ac3d700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb0640b06c0 0x7fb0641f2f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:53.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:53.784+0000 7fb06ac3d700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0641f3470 con 0x7fb0640b02e0 2023-11-08T19:24:53.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:53.784+0000 7fb06ac3d700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0640b64d0 con 0x7fb064156b10 2023-11-08T19:24:53.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:53.784+0000 7fb06ac3d700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0640b3b10 con 0x7fb0640b06c0 2023-11-08T19:24:53.802 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:53.784+0000 7fb0689d9700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb0640b02e0 0x7fb064157a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:53.802 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:53.784+0000 7fb0689d9700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb0640b02e0 0x7fb064157a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48628/0 (socket says 172.21.15.143:48628) 2023-11-08T19:24:53.802 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:53.784+0000 7fb0689d9700 1 -- 172.21.15.143:0/1090528057 learned_addr learned my addr 172.21.15.143:0/1090528057 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:24:53.802 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:53.784+0000 7fb063fff700 1 --2- 172.21.15.143:0/1090528057 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb064156b10 0x7fb0641f29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:53.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:53.784+0000 7fb0691da700 1 --2- 172.21.15.143:0/1090528057 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb0640b06c0 0x7fb0641f2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:53.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:53.785+0000 7fb0689d9700 1 -- 172.21.15.143:0/1090528057 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb0640b06c0 msgr2=0x7fb0641f2f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:53.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:53.785+0000 7fb0689d9700 1 --2- 172.21.15.143:0/1090528057 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb0640b06c0 0x7fb0641f2f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:53.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:53.785+0000 7fb0689d9700 1 -- 172.21.15.143:0/1090528057 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb064156b10 msgr2=0x7fb0641f29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:53.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:53.785+0000 7fb0689d9700 1 --2- 172.21.15.143:0/1090528057 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb064156b10 0x7fb0641f29f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:53.804 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:53.785+0000 7fb0689d9700 1 -- 172.21.15.143:0/1090528057 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb064157f70 con 0x7fb0640b02e0 2023-11-08T19:24:53.804 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:53.785+0000 7fb063fff700 1 --2- 172.21.15.143:0/1090528057 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb064156b10 0x7fb0641f29f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:24:53.804 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:53.785+0000 7fb0691da700 1 --2- 172.21.15.143:0/1090528057 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb0640b06c0 0x7fb0641f2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:24:53.804 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:53.785+0000 7fb0689d9700 1 --2- 172.21.15.143:0/1090528057 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb0640b02e0 0x7fb064157a30 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb0580116a0 tx=0x7fb058011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:53.804 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:53.785+0000 7fb0617fa700 1 -- 172.21.15.143:0/1090528057 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb05800c070 con 0x7fb0640b02e0 2023-11-08T19:24:53.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:53.785+0000 7fb0617fa700 1 -- 172.21.15.143:0/1090528057 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb058015870 con 0x7fb0640b02e0 2023-11-08T19:24:53.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:53.785+0000 7fb0617fa700 1 -- 172.21.15.143:0/1090528057 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb058015c00 con 0x7fb0640b02e0 2023-11-08T19:24:53.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:53.785+0000 7fb06ac3d700 1 -- 172.21.15.143:0/1090528057 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb064158480 con 0x7fb0640b02e0 2023-11-08T19:24:53.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:53.785+0000 7fb06ac3d700 1 -- 172.21.15.143:0/1090528057 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0640b34c0 con 0x7fb0640b02e0 2023-11-08T19:24:53.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:53.786+0000 7fb06ac3d700 1 -- 172.21.15.143:0/1090528057 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb06404eee0 con 0x7fb0640b02e0 2023-11-08T19:24:53.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:53.786+0000 7fb0617fa700 1 -- 172.21.15.143:0/1090528057 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb058013400 con 0x7fb0640b02e0 2023-11-08T19:24:53.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:53.787+0000 7fb0617fa700 1 --2- 172.21.15.143:0/1090528057 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fb04806af00 0x7fb04806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:53.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:53.787+0000 7fb0617fa700 1 -- 172.21.15.143:0/1090528057 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb058089840 con 0x7fb0640b02e0 2023-11-08T19:24:53.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:53.787+0000 7fb063fff700 1 --2- 172.21.15.143:0/1090528057 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fb04806af00 0x7fb04806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:53.807 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:53.787+0000 7fb063fff700 1 --2- 172.21.15.143:0/1090528057 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fb04806af00 0x7fb04806d3c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb0641f6a20 tx=0x7fb050009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:53.807 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:53.791+0000 7fb0617fa700 1 -- 172.21.15.143:0/1090528057 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb058054980 con 0x7fb0640b02e0 2023-11-08T19:24:54.026 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.024+0000 7fb06ac3d700 1 -- 172.21.15.143:0/1090528057 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb0640b3860 con 0x7fb0640b02e0 2023-11-08T19:24:54.027 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.024+0000 7fb0617fa700 1 -- 172.21.15.143:0/1090528057 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4913 (secure 0 0 0) 0x7fb058023090 con 0x7fb0640b02e0 2023-11-08T19:24:54.027 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:24:54.027 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4208,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6834/37280614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":37280614},{"type":"v1","addr":"172.21.15.143:6834","nonce":37280614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/2364281303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":2364281303},{"type":"v1","addr":"172.21.15.143:6835","nonce":2364281303}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6836/4066129908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":4066129908},{"type":"v1","addr":"172.21.15.174:6836","nonce":4066129908}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T19:24:49.995547+0000","modified":"2023-11-08T19:24:53.027089+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.174:6837/2124972405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6835","nonce":2124972405},{"type":"v1","addr":"172.21.15.174:6837","nonce":2124972405}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-08T19:24:54.029 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.026+0000 7fb06ac3d700 1 -- 172.21.15.143:0/1090528057 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fb04806af00 msgr2=0x7fb04806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:54.029 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.026+0000 7fb06ac3d700 1 --2- 172.21.15.143:0/1090528057 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fb04806af00 0x7fb04806d3c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb0641f6a20 tx=0x7fb050009040 comp rx=0 tx=0).stop 2023-11-08T19:24:54.029 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.026+0000 7fb06ac3d700 1 -- 172.21.15.143:0/1090528057 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb0640b02e0 msgr2=0x7fb064157a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:54.030 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.026+0000 7fb06ac3d700 1 --2- 172.21.15.143:0/1090528057 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb0640b02e0 0x7fb064157a30 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb0580116a0 tx=0x7fb058011a60 comp rx=0 tx=0).stop 2023-11-08T19:24:54.030 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.027+0000 7fb06ac3d700 1 -- 172.21.15.143:0/1090528057 shutdown_connections 2023-11-08T19:24:54.030 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.027+0000 7fb06ac3d700 1 --2- 172.21.15.143:0/1090528057 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fb04806af00 0x7fb04806d3c0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:54.030 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.027+0000 7fb06ac3d700 1 --2- 172.21.15.143:0/1090528057 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb0640b06c0 0x7fb0641f2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:54.031 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.027+0000 7fb06ac3d700 1 --2- 172.21.15.143:0/1090528057 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb0640b02e0 0x7fb064157a30 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:54.031 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.027+0000 7fb06ac3d700 1 --2- 172.21.15.143:0/1090528057 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb064156b10 0x7fb0641f29f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:54.031 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.027+0000 7fb06ac3d700 1 -- 172.21.15.143:0/1090528057 >> 172.21.15.143:0/1090528057 conn(0x7fb064099ab0 msgr2=0x7fb06406f520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:24:54.031 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.027+0000 7fb06ac3d700 1 -- 172.21.15.143:0/1090528057 shutdown_connections 2023-11-08T19:24:54.031 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.027+0000 7fb06ac3d700 1 -- 172.21.15.143:0/1090528057 wait complete. 2023-11-08T19:24:54.032 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 8 2023-11-08T19:24:54.040 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:24:54.160 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.158+0000 7f2a400df700 1 Processor -- start 2023-11-08T19:24:54.160 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.159+0000 7f2a400df700 1 -- start start 2023-11-08T19:24:54.167 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.159+0000 7f2a400df700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2a380cc530 0x7f2a380cf510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:54.167 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.160+0000 7f2a400df700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a380cf080 con 0x7f2a3815aa30 2023-11-08T19:24:54.168 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.160+0000 7f2a400df700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a380ce470 con 0x7f2a380b2920 2023-11-08T19:24:54.168 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.160+0000 7f2a400df700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a380ce750 con 0x7f2a380cc530 2023-11-08T19:24:54.168 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.160+0000 7f2a3de7b700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f2a3815aa30 0x7f2a380b2210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:44790/0 (socket says 172.21.15.143:44790) 2023-11-08T19:24:54.168 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.160+0000 7f2a3de7b700 1 -- 172.21.15.143:0/2441951955 learned_addr learned my addr 172.21.15.143:0/2441951955 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:24:54.169 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.160+0000 7f2a3e67c700 1 --2- 172.21.15.143:0/2441951955 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2a380cc530 0x7f2a380cf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:54.169 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.160+0000 7f2a3ce79700 1 -- 172.21.15.143:0/2441951955 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 224621617 0 0) 0x7f2a380cf080 con 0x7f2a3815aa30 2023-11-08T19:24:54.169 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.160+0000 7f2a3ce79700 1 -- 172.21.15.143:0/2441951955 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a240035b0 con 0x7f2a3815aa30 2023-11-08T19:24:54.169 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.160+0000 7f2a3ce79700 1 -- 172.21.15.143:0/2441951955 <== mon.1 v1:172.21.15.174:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1639206356 0 0) 0x7f2a380ce470 con 0x7f2a380b2920 2023-11-08T19:24:54.169 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.160+0000 7f2a3ce79700 1 -- 172.21.15.143:0/2441951955 --> v1:172.21.15.174:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a380cf080 con 0x7f2a380b2920 2023-11-08T19:24:54.170 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.160+0000 7f2a3ce79700 1 -- 172.21.15.143:0/2441951955 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2799798131 0 0) 0x7f2a240035b0 con 0x7f2a3815aa30 2023-11-08T19:24:54.170 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.160+0000 7f2a3ce79700 1 -- 172.21.15.143:0/2441951955 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2a380cc530 msgr2=0x7f2a380cf510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:24:54.170 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.160+0000 7f2a3ce79700 1 --2- 172.21.15.143:0/2441951955 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2a380cc530 0x7f2a380cf510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:54.170 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.160+0000 7f2a3ce79700 1 -- 172.21.15.143:0/2441951955 >> v1:172.21.15.174:6789/0 conn(0x7f2a380b2920 legacy=0x7f2a380cbe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:54.171 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.160+0000 7f2a3ce79700 1 -- 172.21.15.143:0/2441951955 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a380cea30 con 0x7f2a3815aa30 2023-11-08T19:24:54.171 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.160+0000 7f2a3ce79700 1 -- 172.21.15.143:0/2441951955 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4147756303 0 0) 0x7f2a280017f0 con 0x7f2a3815aa30 2023-11-08T19:24:54.171 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.160+0000 7f2a3ce79700 1 -- 172.21.15.143:0/2441951955 >> v1:172.21.15.143:6789/0 conn(0x7f2a3815aa30 legacy=0x7f2a380b2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:54.171 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.160+0000 7f2a3ce79700 1 --2- 172.21.15.143:0/2441951955 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2a380cc530 0x7f2a24003bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:54.172 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.160+0000 7f2a3ce79700 1 --2- 172.21.15.143:0/2441951955 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a24004120 0x7f2a24004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:54.172 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.160+0000 7f2a3ce79700 1 --2- 172.21.15.143:0/2441951955 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2a24004d90 0x7f2a24005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:54.172 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.160+0000 7f2a3ce79700 1 -- 172.21.15.143:0/2441951955 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a380ce750 con 0x7f2a24004d90 2023-11-08T19:24:54.172 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.160+0000 7f2a3ce79700 1 -- 172.21.15.143:0/2441951955 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a2400baf0 con 0x7f2a24004120 2023-11-08T19:24:54.172 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.160+0000 7f2a3ce79700 1 -- 172.21.15.143:0/2441951955 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a2400bcb0 con 0x7f2a380cc530 2023-11-08T19:24:54.173 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.160+0000 7f2a3e67c700 1 --2- 172.21.15.143:0/2441951955 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2a24004d90 0x7f2a24005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:54.173 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.160+0000 7f2a3de7b700 1 --2- 172.21.15.143:0/2441951955 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2a380cc530 0x7f2a24003bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:54.173 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.160+0000 7f2a3d67a700 1 --2- 172.21.15.143:0/2441951955 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a24004120 0x7f2a24004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:54.173 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.161+0000 7f2a3e67c700 1 -- 172.21.15.143:0/2441951955 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2a380cc530 msgr2=0x7f2a24003bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:24:54.174 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.161+0000 7f2a3e67c700 1 --2- 172.21.15.143:0/2441951955 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2a380cc530 0x7f2a24003bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:54.174 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.161+0000 7f2a3e67c700 1 -- 172.21.15.143:0/2441951955 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a24004120 msgr2=0x7f2a24004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:24:54.174 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.161+0000 7f2a3e67c700 1 --2- 172.21.15.143:0/2441951955 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a24004120 0x7f2a24004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:54.174 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.161+0000 7f2a3e67c700 1 -- 172.21.15.143:0/2441951955 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a240035b0 con 0x7f2a24004d90 2023-11-08T19:24:54.175 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.161+0000 7f2a3e67c700 1 --2- 172.21.15.143:0/2441951955 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2a24004d90 0x7f2a24005210 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2a30009fc0 tx=0x7f2a24005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61685026092c4925 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:54.175 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.161+0000 7f2a3ce79700 1 -- 172.21.15.143:0/2441951955 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a3000ec30 con 0x7f2a24004d90 2023-11-08T19:24:54.175 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.161+0000 7f2a3ce79700 1 -- 172.21.15.143:0/2441951955 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a3000f040 con 0x7f2a24004d90 2023-11-08T19:24:54.176 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.161+0000 7f2a3ce79700 1 -- 172.21.15.143:0/2441951955 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a30014480 con 0x7f2a24004d90 2023-11-08T19:24:54.176 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.161+0000 7f2a400df700 1 -- 172.21.15.143:0/2441951955 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2a24004d90 msgr2=0x7f2a24005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:54.176 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.161+0000 7f2a400df700 1 --2- 172.21.15.143:0/2441951955 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2a24004d90 0x7f2a24005210 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2a30009fc0 tx=0x7f2a24005880 comp rx=0 tx=0).stop 2023-11-08T19:24:54.176 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.161+0000 7f2a3e67c700 1 -- 172.21.15.143:0/2441951955 reap_dead start 2023-11-08T19:24:54.177 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.161+0000 7f2a400df700 1 -- 172.21.15.143:0/2441951955 shutdown_connections 2023-11-08T19:24:54.177 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.161+0000 7f2a400df700 1 -- 172.21.15.143:0/2441951955 >> 172.21.15.143:0/2441951955 conn(0x7f2a38099ab0 msgr2=0x7f2a380d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:24:54.177 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.161+0000 7f2a400df700 1 -- 172.21.15.143:0/2441951955 shutdown_connections 2023-11-08T19:24:54.177 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.161+0000 7f2a400df700 1 -- 172.21.15.143:0/2441951955 wait complete. 2023-11-08T19:24:54.178 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.163+0000 7f2a400df700 1 Processor -- start 2023-11-08T19:24:54.178 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.163+0000 7f2a400df700 1 -- start start 2023-11-08T19:24:54.178 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.163+0000 7f2a400df700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a24004d90 0x7f2a38156140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:54.178 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.163+0000 7f2a400df700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2a380c41d0 0x7f2a380c4650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:54.179 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.163+0000 7f2a400df700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2a380c4f70 0x7f2a380c53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:54.179 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.163+0000 7f2a400df700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a380c4b90 con 0x7f2a380c41d0 2023-11-08T19:24:54.179 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.163+0000 7f2a400df700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a381f4540 con 0x7f2a24004d90 2023-11-08T19:24:54.179 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.163+0000 7f2a400df700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a38156680 con 0x7f2a380c4f70 2023-11-08T19:24:54.180 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.163+0000 7f2a3d67a700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2a380c41d0 0x7f2a380c4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:54.180 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.163+0000 7f2a3d67a700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2a380c41d0 0x7f2a380c4650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48650/0 (socket says 172.21.15.143:48650) 2023-11-08T19:24:54.180 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.164+0000 7f2a3d67a700 1 -- 172.21.15.143:0/448683108 learned_addr learned my addr 172.21.15.143:0/448683108 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:24:54.180 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.164+0000 7f2a3de7b700 1 --2- 172.21.15.143:0/448683108 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a24004d90 0x7f2a38156140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:54.180 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.164+0000 7f2a3e67c700 1 --2- 172.21.15.143:0/448683108 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2a380c4f70 0x7f2a380c53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:54.181 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.164+0000 7f2a3d67a700 1 -- 172.21.15.143:0/448683108 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2a380c4f70 msgr2=0x7f2a380c53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:54.181 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.164+0000 7f2a3d67a700 1 --2- 172.21.15.143:0/448683108 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2a380c4f70 0x7f2a380c53f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:54.181 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.164+0000 7f2a3d67a700 1 -- 172.21.15.143:0/448683108 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a24004d90 msgr2=0x7f2a38156140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:54.181 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.164+0000 7f2a3d67a700 1 --2- 172.21.15.143:0/448683108 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a24004d90 0x7f2a38156140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:54.181 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.164+0000 7f2a3d67a700 1 -- 172.21.15.143:0/448683108 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a381f3970 con 0x7f2a380c41d0 2023-11-08T19:24:54.182 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.164+0000 7f2a3e67c700 1 --2- 172.21.15.143:0/448683108 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2a380c4f70 0x7f2a380c53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:24:54.182 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.164+0000 7f2a3d67a700 1 --2- 172.21.15.143:0/448683108 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2a380c41d0 0x7f2a380c4650 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2a3400f7e0 tx=0x7f2a3400fba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:54.182 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.164+0000 7f2a2e7fc700 1 -- 172.21.15.143:0/448683108 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a34001520 con 0x7f2a380c41d0 2023-11-08T19:24:54.182 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.164+0000 7f2a2e7fc700 1 -- 172.21.15.143:0/448683108 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a340052e0 con 0x7f2a380c41d0 2023-11-08T19:24:54.183 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.164+0000 7f2a400df700 1 -- 172.21.15.143:0/448683108 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a381f3c10 con 0x7f2a380c41d0 2023-11-08T19:24:54.183 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.164+0000 7f2a2e7fc700 1 -- 172.21.15.143:0/448683108 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a34003680 con 0x7f2a380c41d0 2023-11-08T19:24:54.183 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.164+0000 7f2a400df700 1 -- 172.21.15.143:0/448683108 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a381f41d0 con 0x7f2a380c41d0 2023-11-08T19:24:54.183 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.165+0000 7f2a400df700 1 -- 172.21.15.143:0/448683108 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a3804eee0 con 0x7f2a380c41d0 2023-11-08T19:24:54.183 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.165+0000 7f2a2e7fc700 1 -- 172.21.15.143:0/448683108 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2a34003800 con 0x7f2a380c41d0 2023-11-08T19:24:54.184 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.166+0000 7f2a2e7fc700 1 --2- 172.21.15.143:0/448683108 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f2a1c06af00 0x7f2a1c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:54.184 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.166+0000 7f2a2e7fc700 1 -- 172.21.15.143:0/448683108 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2a34089500 con 0x7f2a380c41d0 2023-11-08T19:24:54.184 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.166+0000 7f2a3de7b700 1 --2- 172.21.15.143:0/448683108 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f2a1c06af00 0x7f2a1c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:54.185 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.166+0000 7f2a3de7b700 1 --2- 172.21.15.143:0/448683108 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f2a1c06af00 0x7f2a1c06d3c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2a28000ef0 tx=0x7f2a28009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:54.185 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.170+0000 7f2a2e7fc700 1 -- 172.21.15.143:0/448683108 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2a34054640 con 0x7f2a380c41d0 2023-11-08T19:24:54.364 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.362+0000 7f2a400df700 1 -- 172.21.15.143:0/448683108 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2a38158dd0 con 0x7f2a380c41d0 2023-11-08T19:24:54.365 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.364+0000 7f2a2e7fc700 1 -- 172.21.15.143:0/448683108 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16553 (secure 0 0 0) 0x7f2a34057c60 con 0x7f2a380c41d0 2023-11-08T19:24:54.365 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:24:54.366 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":20,"fsid":"968483af-77fc-4068-ad7c-5c2a1a4613d7","created":"2023-11-08T19:24:32.826484+0000","modified":"2023-11-08T19:24:51.618072+0000","last_up_change":"2023-11-08T19:24:43.573880+0000","last_in_change":"2023-11-08T19:24:35.314525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:44.551530+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-08T19:24:48.256781+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-08T19:24:48.954990+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bae98ec1-d94e-403d-a220-a522a53ea6f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6809","nonce":1487830566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6811","nonce":1487830566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6815","nonce":1487830566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6813","nonce":1487830566}]},"public_addr":"172.21.15.143:6809/1487830566","cluster_addr":"172.21.15.143:6811/1487830566","heartbeat_back_addr":"172.21.15.143:6815/1487830566","heartbeat_front_addr":"172.21.15.143:6813/1487830566","state":["exists","up"]},{"osd":1,"uuid":"5cb6538a-1ccb-4322-8633-09f66af0b7e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6801","nonce":787341397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6803","nonce":787341397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6807","nonce":787341397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6805","nonce":787341397}]},"public_addr":"172.21.15.143:6801/787341397","cluster_addr":"172.21.15.143:6803/787341397","heartbeat_back_addr":"172.21.15.143:6807/787341397","heartbeat_front_addr":"172.21.15.143:6805/787341397","state":["exists","up"]},{"osd":2,"uuid":"cdbf522c-60ef-4d60-9653-8dcb529c101a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6822","nonce":2301325597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6826","nonce":2301325597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6831","nonce":2301325597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6829","nonce":2301325597}]},"public_addr":"172.21.15.143:6822/2301325597","cluster_addr":"172.21.15.143:6826/2301325597","heartbeat_back_addr":"172.21.15.143:6831/2301325597","heartbeat_front_addr":"172.21.15.143:6829/2301325597","state":["exists","up"]},{"osd":3,"uuid":"4e2783cf-ebb9-4dd8-8245-bc1d17ce4325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6817","nonce":2021504428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6819","nonce":2021504428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6827","nonce":2021504428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6823","nonce":2021504428}]},"public_addr":"172.21.15.143:6817/2021504428","cluster_addr":"172.21.15.143:6819/2021504428","heartbeat_back_addr":"172.21.15.143:6827/2021504428","heartbeat_front_addr":"172.21.15.143:6823/2021504428","state":["exists","up"]},{"osd":4,"uuid":"691256fa-9b77-49b9-a346-7ae1adf0ca80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6801","nonce":1993380595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6803","nonce":1993380595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6807","nonce":1993380595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6805","nonce":1993380595}]},"public_addr":"172.21.15.174:6801/1993380595","cluster_addr":"172.21.15.174:6803/1993380595","heartbeat_back_addr":"172.21.15.174:6807/1993380595","heartbeat_front_addr":"172.21.15.174:6805/1993380595","state":["exists","up"]},{"osd":5,"uuid":"756fc352-48c9-46ce-b52e-0541fe6f8525","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6809","nonce":1777923303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6811","nonce":1777923303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6815","nonce":1777923303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6813","nonce":1777923303}]},"public_addr":"172.21.15.174:6809/1777923303","cluster_addr":"172.21.15.174:6811/1777923303","heartbeat_back_addr":"172.21.15.174:6815/1777923303","heartbeat_front_addr":"172.21.15.174:6813/1777923303","state":["exists","up"]},{"osd":6,"uuid":"fab2ac2b-ed97-4cde-9825-b480eabf8b8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6825","nonce":2755130316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6827","nonce":2755130316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6831","nonce":2755130316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6829","nonce":2755130316}]},"public_addr":"172.21.15.174:6825/2755130316","cluster_addr":"172.21.15.174:6827/2755130316","heartbeat_back_addr":"172.21.15.174:6831/2755130316","heartbeat_front_addr":"172.21.15.174:6829/2755130316","state":["exists","up"]},{"osd":7,"uuid":"3de9b797-2ab6-4d41-a863-7e7d638fd703","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6817","nonce":2847141411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6819","nonce":2847141411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6823","nonce":2847141411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6821","nonce":2847141411}]},"public_addr":"172.21.15.174:6817/2847141411","cluster_addr":"172.21.15.174:6819/2847141411","heartbeat_back_addr":"172.21.15.174:6823/2847141411","heartbeat_front_addr":"172.21.15.174:6821/2847141411","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.477076+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.496890+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.539305+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.454673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.538881+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.521294+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.520712+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.540404+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:24:54.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.366+0000 7f2a400df700 1 -- 172.21.15.143:0/448683108 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f2a1c06af00 msgr2=0x7f2a1c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:54.368 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.366+0000 7f2a400df700 1 --2- 172.21.15.143:0/448683108 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f2a1c06af00 0x7f2a1c06d3c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2a28000ef0 tx=0x7f2a28009040 comp rx=0 tx=0).stop 2023-11-08T19:24:54.368 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.366+0000 7f2a400df700 1 -- 172.21.15.143:0/448683108 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2a380c41d0 msgr2=0x7f2a380c4650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:54.368 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.366+0000 7f2a400df700 1 --2- 172.21.15.143:0/448683108 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2a380c41d0 0x7f2a380c4650 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2a3400f7e0 tx=0x7f2a3400fba0 comp rx=0 tx=0).stop 2023-11-08T19:24:54.368 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.366+0000 7f2a400df700 1 -- 172.21.15.143:0/448683108 shutdown_connections 2023-11-08T19:24:54.369 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.366+0000 7f2a400df700 1 --2- 172.21.15.143:0/448683108 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f2a1c06af00 0x7f2a1c06d3c0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:54.369 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.366+0000 7f2a400df700 1 --2- 172.21.15.143:0/448683108 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2a380c4f70 0x7f2a380c53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:54.369 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.366+0000 7f2a400df700 1 --2- 172.21.15.143:0/448683108 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a24004d90 0x7f2a38156140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:54.369 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.366+0000 7f2a400df700 1 --2- 172.21.15.143:0/448683108 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2a380c41d0 0x7f2a380c4650 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:54.369 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.366+0000 7f2a400df700 1 -- 172.21.15.143:0/448683108 >> 172.21.15.143:0/448683108 conn(0x7f2a38099ab0 msgr2=0x7f2a3806db40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:24:54.370 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.366+0000 7f2a400df700 1 -- 172.21.15.143:0/448683108 shutdown_connections 2023-11-08T19:24:54.370 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.366+0000 7f2a400df700 1 -- 172.21.15.143:0/448683108 wait complete. 2023-11-08T19:24:54.380 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-08T19:24:54.380 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-08T19:24:54.500 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.498+0000 7f5ab117c700 1 Processor -- start 2023-11-08T19:24:54.500 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.498+0000 7f5ab117c700 1 -- start start 2023-11-08T19:24:54.503 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.499+0000 7f5ab117c700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f5aac0b2920 0x7f5aac0cf510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:54.503 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.499+0000 7f5ab117c700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5aac0cbe20 0x7f5aac0cc290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:54.503 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.499+0000 7f5ab117c700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5aac0cf080 con 0x7f5aac15aa30 2023-11-08T19:24:54.504 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.499+0000 7f5ab117c700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5aac0cc7d0 con 0x7f5aac0b2920 2023-11-08T19:24:54.504 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.499+0000 7f5ab117c700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5aac0ce470 con 0x7f5aac0cbe20 2023-11-08T19:24:54.504 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.500+0000 7f5aaa59c700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f5aac0b2920 0x7f5aac0cf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:54.504 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.500+0000 7f5aab59e700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5aac0cbe20 0x7f5aac0cc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:54.504 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.500+0000 7f5aaa59c700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f5aac0b2920 0x7f5aac0cf510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.143:60388/0 (socket says 172.21.15.143:60388) 2023-11-08T19:24:54.505 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.500+0000 7f5aaa59c700 1 -- 172.21.15.143:0/3487548212 learned_addr learned my addr 172.21.15.143:0/3487548212 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:24:54.505 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.500+0000 7f5aaad9d700 1 --1- >> v1:172.21.15.174:6789/0 conn(0x7f5aac15aa30 0x7f5aac0b2210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.174:6789/0 says I am v1:172.21.15.143:55744/0 (socket says 172.21.15.143:55744) 2023-11-08T19:24:54.505 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.500+0000 7f5aaa59c700 1 -- 172.21.15.143:0/3487548212 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5aac0cbe20 msgr2=0x7f5aac0cc290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:24:54.505 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.500+0000 7f5aaa59c700 1 --2- 172.21.15.143:0/3487548212 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5aac0cbe20 0x7f5aac0cc290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:54.505 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.500+0000 7f5aaa59c700 1 -- 172.21.15.143:0/3487548212 >> v1:172.21.15.174:6789/0 conn(0x7f5aac15aa30 legacy=0x7f5aac0b2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:54.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.500+0000 7f5aaa59c700 1 -- 172.21.15.143:0/3487548212 --> v2:172.21.15.174:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5aac0ce750 con 0x7f5aac0b2920 2023-11-08T19:24:54.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.500+0000 7f5aa9d9b700 1 -- 172.21.15.143:0/3487548212 <== mon.1 v1:172.21.15.174:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2587531295 0 0) 0x7f5aac0cf080 con 0x7f5aac15aa30 2023-11-08T19:24:54.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.500+0000 7f5aaa59c700 1 --2- 172.21.15.143:0/3487548212 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5aac0b2920 0x7f5aac0cf510 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f5aa00098f0 tx=0x7f5aa0009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85fa76da653a3dfc server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:54.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.500+0000 7f5aa9d9b700 1 -- 172.21.15.143:0/3487548212 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aa0011830 con 0x7f5aac0b2920 2023-11-08T19:24:54.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.500+0000 7f5aa9d9b700 1 -- 172.21.15.143:0/3487548212 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5aa0011990 con 0x7f5aac0b2920 2023-11-08T19:24:54.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.500+0000 7f5aa9d9b700 1 -- 172.21.15.143:0/3487548212 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aa0011d20 con 0x7f5aac0b2920 2023-11-08T19:24:54.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.501+0000 7f5ab117c700 1 -- 172.21.15.143:0/3487548212 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5aac0b2920 msgr2=0x7f5aac0cf510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:54.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.501+0000 7f5ab117c700 1 --2- 172.21.15.143:0/3487548212 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5aac0b2920 0x7f5aac0cf510 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f5aa00098f0 tx=0x7f5aa0009c00 comp rx=0 tx=0).stop 2023-11-08T19:24:54.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.501+0000 7f5ab117c700 1 -- 172.21.15.143:0/3487548212 shutdown_connections 2023-11-08T19:24:54.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.501+0000 7f5ab117c700 1 --2- 172.21.15.143:0/3487548212 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5aac0cbe20 0x7f5aac0cc290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:54.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.501+0000 7f5ab117c700 1 --2- 172.21.15.143:0/3487548212 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5aac0b2920 0x7f5aac0cf510 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:54.508 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.501+0000 7f5ab117c700 1 -- 172.21.15.143:0/3487548212 >> 172.21.15.143:0/3487548212 conn(0x7f5aac099ab0 msgr2=0x7f5aac0d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:24:54.508 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.501+0000 7f5ab117c700 1 -- 172.21.15.143:0/3487548212 shutdown_connections 2023-11-08T19:24:54.508 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.501+0000 7f5ab117c700 1 -- 172.21.15.143:0/3487548212 wait complete. 2023-11-08T19:24:54.511 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.502+0000 7f5ab117c700 1 Processor -- start 2023-11-08T19:24:54.511 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.502+0000 7f5ab117c700 1 -- start start 2023-11-08T19:24:54.511 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.503+0000 7f5ab117c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5aac0b2920 0x7f5aac14bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:54.511 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.503+0000 7f5ab117c700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5aac0cbe20 0x7f5aac155200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:54.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.503+0000 7f5ab117c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5aac15aa30 0x7f5aac15cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:54.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.503+0000 7f5ab117c700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aac155740 con 0x7f5aac15aa30 2023-11-08T19:24:54.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.503+0000 7f5ab117c700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aac154d70 con 0x7f5aac0b2920 2023-11-08T19:24:54.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.503+0000 7f5ab117c700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5aac15d6a0 con 0x7f5aac0cbe20 2023-11-08T19:24:54.513 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.503+0000 7f5aab59e700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5aac15aa30 0x7f5aac15cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:54.513 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.503+0000 7f5aab59e700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5aac15aa30 0x7f5aac15cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48664/0 (socket says 172.21.15.143:48664) 2023-11-08T19:24:54.513 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.503+0000 7f5aab59e700 1 -- 172.21.15.143:0/2431286679 learned_addr learned my addr 172.21.15.143:0/2431286679 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:24:54.513 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.503+0000 7f5aaad9d700 1 --2- 172.21.15.143:0/2431286679 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5aac0b2920 0x7f5aac14bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:54.513 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.503+0000 7f5aaa59c700 1 --2- 172.21.15.143:0/2431286679 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5aac0cbe20 0x7f5aac155200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:54.514 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.503+0000 7f5aab59e700 1 -- 172.21.15.143:0/2431286679 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5aac0cbe20 msgr2=0x7f5aac155200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:54.514 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.503+0000 7f5aab59e700 1 --2- 172.21.15.143:0/2431286679 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5aac0cbe20 0x7f5aac155200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:54.514 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.503+0000 7f5aab59e700 1 -- 172.21.15.143:0/2431286679 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5aac0b2920 msgr2=0x7f5aac14bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:54.514 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.503+0000 7f5aab59e700 1 --2- 172.21.15.143:0/2431286679 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5aac0b2920 0x7f5aac14bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:54.514 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.503+0000 7f5aab59e700 1 -- 172.21.15.143:0/2431286679 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5aa0009550 con 0x7f5aac15aa30 2023-11-08T19:24:54.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.504+0000 7f5aab59e700 1 --2- 172.21.15.143:0/2431286679 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5aac15aa30 0x7f5aac15cbe0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5aa400efc0 tx=0x7f5aa400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:54.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.504+0000 7f5a937fe700 1 -- 172.21.15.143:0/2431286679 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aa4009ea0 con 0x7f5aac15aa30 2023-11-08T19:24:54.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.504+0000 7f5a937fe700 1 -- 172.21.15.143:0/2431286679 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5aa400f040 con 0x7f5aac15aa30 2023-11-08T19:24:54.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.504+0000 7f5ab117c700 1 -- 172.21.15.143:0/2431286679 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5aac154720 con 0x7f5aac15aa30 2023-11-08T19:24:54.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.504+0000 7f5a937fe700 1 -- 172.21.15.143:0/2431286679 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aa400ce70 con 0x7f5aac15aa30 2023-11-08T19:24:54.516 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.504+0000 7f5ab117c700 1 -- 172.21.15.143:0/2431286679 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5aac153f30 con 0x7f5aac15aa30 2023-11-08T19:24:54.516 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.505+0000 7f5ab117c700 1 -- 172.21.15.143:0/2431286679 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5aac0cfd30 con 0x7f5aac15aa30 2023-11-08T19:24:54.516 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.505+0000 7f5a937fe700 1 -- 172.21.15.143:0/2431286679 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5aa4014480 con 0x7f5aac15aa30 2023-11-08T19:24:54.516 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.505+0000 7f5a937fe700 1 --2- 172.21.15.143:0/2431286679 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f5a9806af00 0x7f5a9806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:54.516 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.506+0000 7f5a937fe700 1 -- 172.21.15.143:0/2431286679 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5aa4089e40 con 0x7f5aac15aa30 2023-11-08T19:24:54.517 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.506+0000 7f5aaad9d700 1 --2- 172.21.15.143:0/2431286679 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f5a9806af00 0x7f5a9806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:54.517 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.506+0000 7f5aaad9d700 1 --2- 172.21.15.143:0/2431286679 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f5a9806af00 0x7f5a9806d3c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c000fb0 tx=0x7f5a9c007ec0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:54.517 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.510+0000 7f5a937fe700 1 -- 172.21.15.143:0/2431286679 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5aa4054e50 con 0x7f5aac15aa30 2023-11-08T19:24:54.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.698+0000 7f5ab117c700 1 -- 172.21.15.143:0/2431286679 --> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5aac1545a0 con 0x7f5a9806af00 2023-11-08T19:24:54.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.707+0000 7f5a937fe700 1 -- 172.21.15.143:0/2431286679 <== mgr.4099 v2:172.21.15.174:6832/4037280178 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+261184 (secure 0 0 0) 0x7f5a9c001340 con 0x7f5a9806af00 2023-11-08T19:24:54.711 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:24:54.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.709+0000 7f5ab117c700 1 -- 172.21.15.143:0/2431286679 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f5a9806af00 msgr2=0x7f5a9806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:54.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.709+0000 7f5ab117c700 1 --2- 172.21.15.143:0/2431286679 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f5a9806af00 0x7f5a9806d3c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c000fb0 tx=0x7f5a9c007ec0 comp rx=0 tx=0).stop 2023-11-08T19:24:54.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.709+0000 7f5ab117c700 1 -- 172.21.15.143:0/2431286679 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5aac15aa30 msgr2=0x7f5aac15cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:54.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.709+0000 7f5ab117c700 1 --2- 172.21.15.143:0/2431286679 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5aac15aa30 0x7f5aac15cbe0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5aa400efc0 tx=0x7f5aa400c3a0 comp rx=0 tx=0).stop 2023-11-08T19:24:54.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.710+0000 7f5ab117c700 1 -- 172.21.15.143:0/2431286679 shutdown_connections 2023-11-08T19:24:54.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.710+0000 7f5ab117c700 1 --2- 172.21.15.143:0/2431286679 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f5a9806af00 0x7f5a9806d3c0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:54.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.710+0000 7f5ab117c700 1 --2- 172.21.15.143:0/2431286679 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5aac15aa30 0x7f5aac15cbe0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:54.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.710+0000 7f5ab117c700 1 --2- 172.21.15.143:0/2431286679 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5aac0b2920 0x7f5aac14bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:54.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.710+0000 7f5ab117c700 1 --2- 172.21.15.143:0/2431286679 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5aac0cbe20 0x7f5aac155200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:54.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.710+0000 7f5ab117c700 1 -- 172.21.15.143:0/2431286679 >> 172.21.15.143:0/2431286679 conn(0x7f5aac099ab0 msgr2=0x7f5aac0b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:24:54.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.710+0000 7f5ab117c700 1 -- 172.21.15.143:0/2431286679 shutdown_connections 2023-11-08T19:24:54.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.710+0000 7f5ab117c700 1 -- 172.21.15.143:0/2431286679 wait complete. 2023-11-08T19:24:54.716 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2023-11-08T19:24:54.723 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":18,"stamp":"2023-11-08T19:24:54.509481+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591166,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":187,"num_write_kb":2427,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":140,"ondisk_log_size":140,"up":158,"acting":158,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":143,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218904,"kb_used_data":2724,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755816,"statfs":{"total":773094113280,"available":772869955584,"internally_reserved":0,"allocated":2789376,"data_stored":1705642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":798,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.917376"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:24:49.588741+0000","last_change":"2023-11-08T19:24:49.588741+0000","last_active":"2023-11-08T19:24:49.588741+0000","last_peered":"2023-11-08T19:24:49.588741+0000","last_clean":"2023-11-08T19:24:49.588741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:24:49.588741+0000","last_undegraded":"2023-11-08T19:24:49.588741+0000","last_fullsized":"2023-11-08T19:24:49.588741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.805606+0000","last_change":"2023-11-08T19:24:50.623985+0000","last_active":"2023-11-08T19:24:52.805606+0000","last_peered":"2023-11-08T19:24:52.805606+0000","last_clean":"2023-11-08T19:24:52.805606+0000","last_became_active":"2023-11-08T19:24:50.623553+0000","last_became_peered":"2023-11-08T19:24:50.623553+0000","last_unstale":"2023-11-08T19:24:52.805606+0000","last_undegraded":"2023-11-08T19:24:52.805606+0000","last_fullsized":"2023-11-08T19:24:52.805606+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:13:58.948371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.309774+0000","last_change":"2023-11-08T19:24:50.617914+0000","last_active":"2023-11-08T19:24:52.309774+0000","last_peered":"2023-11-08T19:24:52.309774+0000","last_clean":"2023-11-08T19:24:52.309774+0000","last_became_active":"2023-11-08T19:24:50.617611+0000","last_became_peered":"2023-11-08T19:24:50.617611+0000","last_unstale":"2023-11-08T19:24:52.309774+0000","last_undegraded":"2023-11-08T19:24:52.309774+0000","last_fullsized":"2023-11-08T19:24:52.309774+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:30:36.041820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.352504+0000","last_change":"2023-11-08T19:24:50.640039+0000","last_active":"2023-11-08T19:24:52.352504+0000","last_peered":"2023-11-08T19:24:52.352504+0000","last_clean":"2023-11-08T19:24:52.352504+0000","last_became_active":"2023-11-08T19:24:50.639560+0000","last_became_peered":"2023-11-08T19:24:50.639560+0000","last_unstale":"2023-11-08T19:24:52.352504+0000","last_undegraded":"2023-11-08T19:24:52.352504+0000","last_fullsized":"2023-11-08T19:24:52.352504+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:01:53.410545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:24:49.588741+0000","last_change":"2023-11-08T19:24:49.588741+0000","last_active":"2023-11-08T19:24:49.588741+0000","last_peered":"2023-11-08T19:24:49.588741+0000","last_clean":"2023-11-08T19:24:49.588741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:24:49.588741+0000","last_undegraded":"2023-11-08T19:24:49.588741+0000","last_fullsized":"2023-11-08T19:24:49.588741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.351953+0000","last_change":"2023-11-08T19:24:50.641005+0000","last_active":"2023-11-08T19:24:52.351953+0000","last_peered":"2023-11-08T19:24:52.351953+0000","last_clean":"2023-11-08T19:24:52.351953+0000","last_became_active":"2023-11-08T19:24:50.640727+0000","last_became_peered":"2023-11-08T19:24:50.640727+0000","last_unstale":"2023-11-08T19:24:52.351953+0000","last_undegraded":"2023-11-08T19:24:52.351953+0000","last_fullsized":"2023-11-08T19:24:52.351953+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:36:53.967796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:24:49.588741+0000","last_change":"2023-11-08T19:24:49.588741+0000","last_active":"2023-11-08T19:24:49.588741+0000","last_peered":"2023-11-08T19:24:49.588741+0000","last_clean":"2023-11-08T19:24:49.588741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:24:49.588741+0000","last_undegraded":"2023-11-08T19:24:49.588741+0000","last_fullsized":"2023-11-08T19:24:49.588741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.626351+0000","last_change":"2023-11-08T19:24:50.639325+0000","last_active":"2023-11-08T19:24:51.626351+0000","last_peered":"2023-11-08T19:24:51.626351+0000","last_clean":"2023-11-08T19:24:51.626351+0000","last_became_active":"2023-11-08T19:24:50.638993+0000","last_became_peered":"2023-11-08T19:24:50.638993+0000","last_unstale":"2023-11-08T19:24:51.626351+0000","last_undegraded":"2023-11-08T19:24:51.626351+0000","last_fullsized":"2023-11-08T19:24:51.626351+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:30:32.602976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:24:49.588741+0000","last_change":"2023-11-08T19:24:49.588741+0000","last_active":"2023-11-08T19:24:49.588741+0000","last_peered":"2023-11-08T19:24:49.588741+0000","last_clean":"2023-11-08T19:24:49.588741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:24:49.588741+0000","last_undegraded":"2023-11-08T19:24:49.588741+0000","last_fullsized":"2023-11-08T19:24:49.588741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:24:49.588741+0000","last_change":"2023-11-08T19:24:49.588741+0000","last_active":"2023-11-08T19:24:49.588741+0000","last_peered":"2023-11-08T19:24:49.588741+0000","last_clean":"2023-11-08T19:24:49.588741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:24:49.588741+0000","last_undegraded":"2023-11-08T19:24:49.588741+0000","last_fullsized":"2023-11-08T19:24:49.588741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:24:49.588741+0000","last_change":"2023-11-08T19:24:49.588741+0000","last_active":"2023-11-08T19:24:49.588741+0000","last_peered":"2023-11-08T19:24:49.588741+0000","last_clean":"2023-11-08T19:24:49.588741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:24:49.588741+0000","last_undegraded":"2023-11-08T19:24:49.588741+0000","last_fullsized":"2023-11-08T19:24:49.588741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.805767+0000","last_change":"2023-11-08T19:24:50.621511+0000","last_active":"2023-11-08T19:24:52.805767+0000","last_peered":"2023-11-08T19:24:52.805767+0000","last_clean":"2023-11-08T19:24:52.805767+0000","last_became_active":"2023-11-08T19:24:50.621142+0000","last_became_peered":"2023-11-08T19:24:50.621142+0000","last_unstale":"2023-11-08T19:24:52.805767+0000","last_undegraded":"2023-11-08T19:24:52.805767+0000","last_fullsized":"2023-11-08T19:24:52.805767+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:14:01.459274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.626625+0000","last_change":"2023-11-08T19:24:50.625632+0000","last_active":"2023-11-08T19:24:51.626625+0000","last_peered":"2023-11-08T19:24:51.626625+0000","last_clean":"2023-11-08T19:24:51.626625+0000","last_became_active":"2023-11-08T19:24:50.625370+0000","last_became_peered":"2023-11-08T19:24:50.625370+0000","last_unstale":"2023-11-08T19:24:51.626625+0000","last_undegraded":"2023-11-08T19:24:51.626625+0000","last_fullsized":"2023-11-08T19:24:51.626625+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:13:58.948371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.309818+0000","last_change":"2023-11-08T19:24:50.625350+0000","last_active":"2023-11-08T19:24:52.309818+0000","last_peered":"2023-11-08T19:24:52.309818+0000","last_clean":"2023-11-08T19:24:52.309818+0000","last_became_active":"2023-11-08T19:24:50.625078+0000","last_became_peered":"2023-11-08T19:24:50.625078+0000","last_unstale":"2023-11-08T19:24:52.309818+0000","last_undegraded":"2023-11-08T19:24:52.309818+0000","last_fullsized":"2023-11-08T19:24:52.309818+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:42:42.512786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.804817+0000","last_change":"2023-11-08T19:24:50.629187+0000","last_active":"2023-11-08T19:24:52.804817+0000","last_peered":"2023-11-08T19:24:52.804817+0000","last_clean":"2023-11-08T19:24:52.804817+0000","last_became_active":"2023-11-08T19:24:50.628898+0000","last_became_peered":"2023-11-08T19:24:50.628898+0000","last_unstale":"2023-11-08T19:24:52.804817+0000","last_undegraded":"2023-11-08T19:24:52.804817+0000","last_fullsized":"2023-11-08T19:24:52.804817+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:15:33.285449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.351906+0000","last_change":"2023-11-08T19:24:50.638579+0000","last_active":"2023-11-08T19:24:52.351906+0000","last_peered":"2023-11-08T19:24:52.351906+0000","last_clean":"2023-11-08T19:24:52.351906+0000","last_became_active":"2023-11-08T19:24:50.638297+0000","last_became_peered":"2023-11-08T19:24:50.638297+0000","last_unstale":"2023-11-08T19:24:52.351906+0000","last_undegraded":"2023-11-08T19:24:52.351906+0000","last_fullsized":"2023-11-08T19:24:52.351906+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:13:58.948371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.352572+0000","last_change":"2023-11-08T19:24:50.641446+0000","last_active":"2023-11-08T19:24:52.352572+0000","last_peered":"2023-11-08T19:24:52.352572+0000","last_clean":"2023-11-08T19:24:52.352572+0000","last_became_active":"2023-11-08T19:24:50.639709+0000","last_became_peered":"2023-11-08T19:24:50.639709+0000","last_unstale":"2023-11-08T19:24:52.352572+0000","last_undegraded":"2023-11-08T19:24:52.352572+0000","last_fullsized":"2023-11-08T19:24:52.352572+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:09:01.749686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.804922+0000","last_change":"2023-11-08T19:24:50.619241+0000","last_active":"2023-11-08T19:24:52.804922+0000","last_peered":"2023-11-08T19:24:52.804922+0000","last_clean":"2023-11-08T19:24:52.804922+0000","last_became_active":"2023-11-08T19:24:50.618962+0000","last_became_peered":"2023-11-08T19:24:50.618962+0000","last_unstale":"2023-11-08T19:24:52.804922+0000","last_undegraded":"2023-11-08T19:24:52.804922+0000","last_fullsized":"2023-11-08T19:24:52.804922+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:14:01.459274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:24:49.588741+0000","last_change":"2023-11-08T19:24:49.588741+0000","last_active":"2023-11-08T19:24:49.588741+0000","last_peered":"2023-11-08T19:24:49.588741+0000","last_clean":"2023-11-08T19:24:49.588741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:24:49.588741+0000","last_undegraded":"2023-11-08T19:24:49.588741+0000","last_fullsized":"2023-11-08T19:24:49.588741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.309784+0000","last_change":"2023-11-08T19:24:49.608481+0000","last_active":"2023-11-08T19:24:52.309784+0000","last_peered":"2023-11-08T19:24:52.309784+0000","last_clean":"2023-11-08T19:24:52.309784+0000","last_became_active":"2023-11-08T19:24:49.607778+0000","last_became_peered":"2023-11-08T19:24:49.607778+0000","last_unstale":"2023-11-08T19:24:52.309784+0000","last_undegraded":"2023-11-08T19:24:52.309784+0000","last_fullsized":"2023-11-08T19:24:52.309784+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:01:09.884262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.626900+0000","last_change":"2023-11-08T19:24:50.624852+0000","last_active":"2023-11-08T19:24:51.626900+0000","last_peered":"2023-11-08T19:24:51.626900+0000","last_clean":"2023-11-08T19:24:51.626900+0000","last_became_active":"2023-11-08T19:24:50.624551+0000","last_became_peered":"2023-11-08T19:24:50.624551+0000","last_unstale":"2023-11-08T19:24:51.626900+0000","last_undegraded":"2023-11-08T19:24:51.626900+0000","last_fullsized":"2023-11-08T19:24:51.626900+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:12:54.641952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.626938+0000","last_change":"2023-11-08T19:24:49.610418+0000","last_active":"2023-11-08T19:24:51.626938+0000","last_peered":"2023-11-08T19:24:51.626938+0000","last_clean":"2023-11-08T19:24:51.626938+0000","last_became_active":"2023-11-08T19:24:49.610041+0000","last_became_peered":"2023-11-08T19:24:49.610041+0000","last_unstale":"2023-11-08T19:24:51.626938+0000","last_undegraded":"2023-11-08T19:24:51.626938+0000","last_fullsized":"2023-11-08T19:24:51.626938+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:00:12.067877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.309779+0000","last_change":"2023-11-08T19:24:50.627052+0000","last_active":"2023-11-08T19:24:52.309779+0000","last_peered":"2023-11-08T19:24:52.309779+0000","last_clean":"2023-11-08T19:24:52.309779+0000","last_became_active":"2023-11-08T19:24:50.626751+0000","last_became_peered":"2023-11-08T19:24:50.626751+0000","last_unstale":"2023-11-08T19:24:52.309779+0000","last_undegraded":"2023-11-08T19:24:52.309779+0000","last_fullsized":"2023-11-08T19:24:52.309779+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:46:19.375897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-08T19:24:48.592048+0000","last_change":"2023-11-08T19:24:48.592048+0000","last_active":"2023-11-08T19:24:48.584965+0000","last_peered":"2023-11-08T19:24:48.584965+0000","last_clean":"2023-11-08T19:24:48.584965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:24:48.592048+0000","last_undegraded":"2023-11-08T19:24:48.592048+0000","last_fullsized":"2023-11-08T19:24:48.592048+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.352125+0000","last_change":"2023-11-08T19:24:50.641183+0000","last_active":"2023-11-08T19:24:52.352125+0000","last_peered":"2023-11-08T19:24:52.352125+0000","last_clean":"2023-11-08T19:24:52.352125+0000","last_became_active":"2023-11-08T19:24:50.640910+0000","last_became_peered":"2023-11-08T19:24:50.640910+0000","last_unstale":"2023-11-08T19:24:52.352125+0000","last_undegraded":"2023-11-08T19:24:52.352125+0000","last_fullsized":"2023-11-08T19:24:52.352125+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:49:19.100370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-08T19:24:48.591551+0000","last_change":"2023-11-08T19:24:48.591551+0000","last_active":"2023-11-08T19:24:48.584965+0000","last_peered":"2023-11-08T19:24:48.584965+0000","last_clean":"2023-11-08T19:24:48.584965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:24:48.591551+0000","last_undegraded":"2023-11-08T19:24:48.591551+0000","last_fullsized":"2023-11-08T19:24:48.591551+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.352284+0000","last_change":"2023-11-08T19:24:50.640245+0000","last_active":"2023-11-08T19:24:52.352284+0000","last_peered":"2023-11-08T19:24:52.352284+0000","last_clean":"2023-11-08T19:24:52.352284+0000","last_became_active":"2023-11-08T19:24:50.639668+0000","last_became_peered":"2023-11-08T19:24:50.639668+0000","last_unstale":"2023-11-08T19:24:52.352284+0000","last_undegraded":"2023-11-08T19:24:52.352284+0000","last_fullsized":"2023-11-08T19:24:52.352284+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:42:27.150066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.626599+0000","last_change":"2023-11-08T19:24:49.612665+0000","last_active":"2023-11-08T19:24:51.626599+0000","last_peered":"2023-11-08T19:24:51.626599+0000","last_clean":"2023-11-08T19:24:51.626599+0000","last_became_active":"2023-11-08T19:24:49.612362+0000","last_became_peered":"2023-11-08T19:24:49.612362+0000","last_unstale":"2023-11-08T19:24:51.626599+0000","last_undegraded":"2023-11-08T19:24:51.626599+0000","last_fullsized":"2023-11-08T19:24:51.626599+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:46:12.493411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.805419+0000","last_change":"2023-11-08T19:24:50.627537+0000","last_active":"2023-11-08T19:24:52.805419+0000","last_peered":"2023-11-08T19:24:52.805419+0000","last_clean":"2023-11-08T19:24:52.805419+0000","last_became_active":"2023-11-08T19:24:50.626489+0000","last_became_peered":"2023-11-08T19:24:50.626489+0000","last_unstale":"2023-11-08T19:24:52.805419+0000","last_undegraded":"2023-11-08T19:24:52.805419+0000","last_fullsized":"2023-11-08T19:24:52.805419+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:01:10.888039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-08T19:24:48.592162+0000","last_change":"2023-11-08T19:24:48.592162+0000","last_active":"2023-11-08T19:24:48.584965+0000","last_peered":"2023-11-08T19:24:48.584965+0000","last_clean":"2023-11-08T19:24:48.584965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:24:48.592162+0000","last_undegraded":"2023-11-08T19:24:48.592162+0000","last_fullsized":"2023-11-08T19:24:48.592162+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.309935+0000","last_change":"2023-11-08T19:24:50.627987+0000","last_active":"2023-11-08T19:24:52.309935+0000","last_peered":"2023-11-08T19:24:52.309935+0000","last_clean":"2023-11-08T19:24:52.309935+0000","last_became_active":"2023-11-08T19:24:50.627402+0000","last_became_peered":"2023-11-08T19:24:50.627402+0000","last_unstale":"2023-11-08T19:24:52.309935+0000","last_undegraded":"2023-11-08T19:24:52.309935+0000","last_fullsized":"2023-11-08T19:24:52.309935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:36:53.967796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.626523+0000","last_change":"2023-11-08T19:24:49.606482+0000","last_active":"2023-11-08T19:24:51.626523+0000","last_peered":"2023-11-08T19:24:51.626523+0000","last_clean":"2023-11-08T19:24:51.626523+0000","last_became_active":"2023-11-08T19:24:49.606211+0000","last_became_peered":"2023-11-08T19:24:49.606211+0000","last_unstale":"2023-11-08T19:24:51.626523+0000","last_undegraded":"2023-11-08T19:24:51.626523+0000","last_fullsized":"2023-11-08T19:24:51.626523+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:42:59.845581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.626483+0000","last_change":"2023-11-08T19:24:50.625917+0000","last_active":"2023-11-08T19:24:51.626483+0000","last_peered":"2023-11-08T19:24:51.626483+0000","last_clean":"2023-11-08T19:24:51.626483+0000","last_became_active":"2023-11-08T19:24:50.625398+0000","last_became_peered":"2023-11-08T19:24:50.625398+0000","last_unstale":"2023-11-08T19:24:51.626483+0000","last_undegraded":"2023-11-08T19:24:51.626483+0000","last_fullsized":"2023-11-08T19:24:51.626483+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:40:11.978983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.351905+0000","last_change":"2023-11-08T19:24:49.616558+0000","last_active":"2023-11-08T19:24:52.351905+0000","last_peered":"2023-11-08T19:24:52.351905+0000","last_clean":"2023-11-08T19:24:52.351905+0000","last_became_active":"2023-11-08T19:24:49.616277+0000","last_became_peered":"2023-11-08T19:24:49.616277+0000","last_unstale":"2023-11-08T19:24:52.351905+0000","last_undegraded":"2023-11-08T19:24:52.351905+0000","last_fullsized":"2023-11-08T19:24:52.351905+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:01:09.884262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.805048+0000","last_change":"2023-11-08T19:24:50.623658+0000","last_active":"2023-11-08T19:24:52.805048+0000","last_peered":"2023-11-08T19:24:52.805048+0000","last_clean":"2023-11-08T19:24:52.805048+0000","last_became_active":"2023-11-08T19:24:50.623399+0000","last_became_peered":"2023-11-08T19:24:50.623399+0000","last_unstale":"2023-11-08T19:24:52.805048+0000","last_undegraded":"2023-11-08T19:24:52.805048+0000","last_fullsized":"2023-11-08T19:24:52.805048+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:40:11.978983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-08T19:24:48.593392+0000","last_change":"2023-11-08T19:24:48.593392+0000","last_active":"2023-11-08T19:24:48.584965+0000","last_peered":"2023-11-08T19:24:48.584965+0000","last_clean":"2023-11-08T19:24:48.584965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:24:48.593392+0000","last_undegraded":"2023-11-08T19:24:48.593392+0000","last_fullsized":"2023-11-08T19:24:48.593392+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.352305+0000","last_change":"2023-11-08T19:24:50.640397+0000","last_active":"2023-11-08T19:24:52.352305+0000","last_peered":"2023-11-08T19:24:52.352305+0000","last_clean":"2023-11-08T19:24:52.352305+0000","last_became_active":"2023-11-08T19:24:50.639730+0000","last_became_peered":"2023-11-08T19:24:50.639730+0000","last_unstale":"2023-11-08T19:24:52.352305+0000","last_undegraded":"2023-11-08T19:24:52.352305+0000","last_fullsized":"2023-11-08T19:24:52.352305+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:08:22.499163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.352212+0000","last_change":"2023-11-08T19:24:49.611202+0000","last_active":"2023-11-08T19:24:52.352212+0000","last_peered":"2023-11-08T19:24:52.352212+0000","last_clean":"2023-11-08T19:24:52.352212+0000","last_became_active":"2023-11-08T19:24:49.610894+0000","last_became_peered":"2023-11-08T19:24:49.610894+0000","last_unstale":"2023-11-08T19:24:52.352212+0000","last_undegraded":"2023-11-08T19:24:52.352212+0000","last_fullsized":"2023-11-08T19:24:52.352212+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:48:25.546248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.309935+0000","last_change":"2023-11-08T19:24:50.638901+0000","last_active":"2023-11-08T19:24:52.309935+0000","last_peered":"2023-11-08T19:24:52.309935+0000","last_clean":"2023-11-08T19:24:52.309935+0000","last_became_active":"2023-11-08T19:24:50.638587+0000","last_became_peered":"2023-11-08T19:24:50.638587+0000","last_unstale":"2023-11-08T19:24:52.309935+0000","last_undegraded":"2023-11-08T19:24:52.309935+0000","last_fullsized":"2023-11-08T19:24:52.309935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:13:58.948371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.805313+0000","last_change":"2023-11-08T19:24:49.612170+0000","last_active":"2023-11-08T19:24:52.805313+0000","last_peered":"2023-11-08T19:24:52.805313+0000","last_clean":"2023-11-08T19:24:52.805313+0000","last_became_active":"2023-11-08T19:24:49.611895+0000","last_became_peered":"2023-11-08T19:24:49.611895+0000","last_unstale":"2023-11-08T19:24:52.805313+0000","last_undegraded":"2023-11-08T19:24:52.805313+0000","last_fullsized":"2023-11-08T19:24:52.805313+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:38:24.246638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.805721+0000","last_change":"2023-11-08T19:24:50.627060+0000","last_active":"2023-11-08T19:24:52.805721+0000","last_peered":"2023-11-08T19:24:52.805721+0000","last_clean":"2023-11-08T19:24:52.805721+0000","last_became_active":"2023-11-08T19:24:50.626468+0000","last_became_peered":"2023-11-08T19:24:50.626468+0000","last_unstale":"2023-11-08T19:24:52.805721+0000","last_undegraded":"2023-11-08T19:24:52.805721+0000","last_fullsized":"2023-11-08T19:24:52.805721+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:00:13.071654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-08T19:24:48.591494+0000","last_change":"2023-11-08T19:24:48.591494+0000","last_active":"2023-11-08T19:24:48.584965+0000","last_peered":"2023-11-08T19:24:48.584965+0000","last_clean":"2023-11-08T19:24:48.584965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:24:48.591494+0000","last_undegraded":"2023-11-08T19:24:48.591494+0000","last_fullsized":"2023-11-08T19:24:48.591494+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:24:49.588741+0000","last_change":"2023-11-08T19:24:49.588741+0000","last_active":"2023-11-08T19:24:49.588741+0000","last_peered":"2023-11-08T19:24:49.588741+0000","last_clean":"2023-11-08T19:24:49.588741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:24:49.588741+0000","last_undegraded":"2023-11-08T19:24:49.588741+0000","last_fullsized":"2023-11-08T19:24:49.588741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.309884+0000","last_change":"2023-11-08T19:24:49.610471+0000","last_active":"2023-11-08T19:24:52.309884+0000","last_peered":"2023-11-08T19:24:52.309884+0000","last_clean":"2023-11-08T19:24:52.309884+0000","last_became_active":"2023-11-08T19:24:49.610182+0000","last_became_peered":"2023-11-08T19:24:49.610182+0000","last_unstale":"2023-11-08T19:24:52.309884+0000","last_undegraded":"2023-11-08T19:24:52.309884+0000","last_fullsized":"2023-11-08T19:24:52.309884+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:14:00.455497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.309602+0000","last_change":"2023-11-08T19:24:50.619498+0000","last_active":"2023-11-08T19:24:52.309602+0000","last_peered":"2023-11-08T19:24:52.309602+0000","last_clean":"2023-11-08T19:24:52.309602+0000","last_became_active":"2023-11-08T19:24:50.619231+0000","last_became_peered":"2023-11-08T19:24:50.619231+0000","last_unstale":"2023-11-08T19:24:52.309602+0000","last_undegraded":"2023-11-08T19:24:52.309602+0000","last_fullsized":"2023-11-08T19:24:52.309602+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:38:25.250414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.352455+0000","last_change":"2023-11-08T19:24:49.612336+0000","last_active":"2023-11-08T19:24:52.352455+0000","last_peered":"2023-11-08T19:24:52.352455+0000","last_clean":"2023-11-08T19:24:52.352455+0000","last_became_active":"2023-11-08T19:24:49.611885+0000","last_became_peered":"2023-11-08T19:24:49.611885+0000","last_unstale":"2023-11-08T19:24:52.352455+0000","last_undegraded":"2023-11-08T19:24:52.352455+0000","last_fullsized":"2023-11-08T19:24:52.352455+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:46:18.372121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:24:49.588741+0000","last_change":"2023-11-08T19:24:49.588741+0000","last_active":"2023-11-08T19:24:49.588741+0000","last_peered":"2023-11-08T19:24:49.588741+0000","last_clean":"2023-11-08T19:24:49.588741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:24:49.588741+0000","last_undegraded":"2023-11-08T19:24:49.588741+0000","last_fullsized":"2023-11-08T19:24:49.588741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-08T19:24:48.591465+0000","last_change":"2023-11-08T19:24:48.591465+0000","last_active":"2023-11-08T19:24:48.584965+0000","last_peered":"2023-11-08T19:24:48.584965+0000","last_clean":"2023-11-08T19:24:48.584965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:24:48.591465+0000","last_undegraded":"2023-11-08T19:24:48.591465+0000","last_fullsized":"2023-11-08T19:24:48.591465+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:24:49.588741+0000","last_change":"2023-11-08T19:24:49.588741+0000","last_active":"2023-11-08T19:24:49.588741+0000","last_peered":"2023-11-08T19:24:49.588741+0000","last_clean":"2023-11-08T19:24:49.588741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:24:49.588741+0000","last_undegraded":"2023-11-08T19:24:49.588741+0000","last_fullsized":"2023-11-08T19:24:49.588741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.804637+0000","last_change":"2023-11-08T19:24:49.608503+0000","last_active":"2023-11-08T19:24:52.804637+0000","last_peered":"2023-11-08T19:24:52.804637+0000","last_clean":"2023-11-08T19:24:52.804637+0000","last_became_active":"2023-11-08T19:24:49.608176+0000","last_became_peered":"2023-11-08T19:24:49.608176+0000","last_unstale":"2023-11-08T19:24:52.804637+0000","last_undegraded":"2023-11-08T19:24:52.804637+0000","last_fullsized":"2023-11-08T19:24:52.804637+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:17:50.005667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:24:49.588741+0000","last_change":"2023-11-08T19:24:49.588741+0000","last_active":"2023-11-08T19:24:49.588741+0000","last_peered":"2023-11-08T19:24:49.588741+0000","last_clean":"2023-11-08T19:24:49.588741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:24:49.588741+0000","last_undegraded":"2023-11-08T19:24:49.588741+0000","last_fullsized":"2023-11-08T19:24:49.588741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.805855+0000","last_change":"2023-11-08T19:24:49.610638+0000","last_active":"2023-11-08T19:24:52.805855+0000","last_peered":"2023-11-08T19:24:52.805855+0000","last_clean":"2023-11-08T19:24:52.805855+0000","last_became_active":"2023-11-08T19:24:49.610362+0000","last_became_peered":"2023-11-08T19:24:49.610362+0000","last_unstale":"2023-11-08T19:24:52.805855+0000","last_undegraded":"2023-11-08T19:24:52.805855+0000","last_fullsized":"2023-11-08T19:24:52.805855+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:15:32.030617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:24:49.588741+0000","last_change":"2023-11-08T19:24:49.588741+0000","last_active":"2023-11-08T19:24:49.588741+0000","last_peered":"2023-11-08T19:24:49.588741+0000","last_clean":"2023-11-08T19:24:49.588741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:24:49.588741+0000","last_undegraded":"2023-11-08T19:24:49.588741+0000","last_fullsized":"2023-11-08T19:24:49.588741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.805154+0000","last_change":"2023-11-08T19:24:49.613413+0000","last_active":"2023-11-08T19:24:52.805154+0000","last_peered":"2023-11-08T19:24:52.805154+0000","last_clean":"2023-11-08T19:24:52.805154+0000","last_became_active":"2023-11-08T19:24:49.613131+0000","last_became_peered":"2023-11-08T19:24:49.613131+0000","last_unstale":"2023-11-08T19:24:52.805154+0000","last_undegraded":"2023-11-08T19:24:52.805154+0000","last_fullsized":"2023-11-08T19:24:52.805154+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:46:12.493411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.352374+0000","last_change":"2023-11-08T19:24:50.640031+0000","last_active":"2023-11-08T19:24:52.352374+0000","last_peered":"2023-11-08T19:24:52.352374+0000","last_clean":"2023-11-08T19:24:52.352374+0000","last_became_active":"2023-11-08T19:24:50.639718+0000","last_became_peered":"2023-11-08T19:24:50.639718+0000","last_unstale":"2023-11-08T19:24:52.352374+0000","last_undegraded":"2023-11-08T19:24:52.352374+0000","last_fullsized":"2023-11-08T19:24:52.352374+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:08:17.096077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.352346+0000","last_change":"2023-11-08T19:24:49.612137+0000","last_active":"2023-11-08T19:24:52.352346+0000","last_peered":"2023-11-08T19:24:52.352346+0000","last_clean":"2023-11-08T19:24:52.352346+0000","last_became_active":"2023-11-08T19:24:49.611532+0000","last_became_peered":"2023-11-08T19:24:49.611532+0000","last_unstale":"2023-11-08T19:24:52.352346+0000","last_undegraded":"2023-11-08T19:24:52.352346+0000","last_fullsized":"2023-11-08T19:24:52.352346+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:40:10.975206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.805536+0000","last_change":"2023-11-08T19:24:50.630854+0000","last_active":"2023-11-08T19:24:52.805536+0000","last_peered":"2023-11-08T19:24:52.805536+0000","last_clean":"2023-11-08T19:24:52.805536+0000","last_became_active":"2023-11-08T19:24:50.630549+0000","last_became_peered":"2023-11-08T19:24:50.630549+0000","last_unstale":"2023-11-08T19:24:52.805536+0000","last_undegraded":"2023-11-08T19:24:52.805536+0000","last_fullsized":"2023-11-08T19:24:52.805536+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:48:26.550025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.804950+0000","last_change":"2023-11-08T19:24:50.620897+0000","last_active":"2023-11-08T19:24:52.804950+0000","last_peered":"2023-11-08T19:24:52.804950+0000","last_clean":"2023-11-08T19:24:52.804950+0000","last_became_active":"2023-11-08T19:24:50.620606+0000","last_became_peered":"2023-11-08T19:24:50.620606+0000","last_unstale":"2023-11-08T19:24:52.804950+0000","last_undegraded":"2023-11-08T19:24:52.804950+0000","last_fullsized":"2023-11-08T19:24:52.804950+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:30:36.041820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.804625+0000","last_change":"2023-11-08T19:24:49.612840+0000","last_active":"2023-11-08T19:24:52.804625+0000","last_peered":"2023-11-08T19:24:52.804625+0000","last_clean":"2023-11-08T19:24:52.804625+0000","last_became_active":"2023-11-08T19:24:49.612652+0000","last_became_peered":"2023-11-08T19:24:49.612652+0000","last_unstale":"2023-11-08T19:24:52.804625+0000","last_undegraded":"2023-11-08T19:24:52.804625+0000","last_fullsized":"2023-11-08T19:24:52.804625+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:48:25.546248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.626593+0000","last_change":"2023-11-08T19:24:50.642515+0000","last_active":"2023-11-08T19:24:51.626593+0000","last_peered":"2023-11-08T19:24:51.626593+0000","last_clean":"2023-11-08T19:24:51.626593+0000","last_became_active":"2023-11-08T19:24:50.642204+0000","last_became_peered":"2023-11-08T19:24:50.642204+0000","last_unstale":"2023-11-08T19:24:51.626593+0000","last_undegraded":"2023-11-08T19:24:51.626593+0000","last_fullsized":"2023-11-08T19:24:51.626593+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:42:42.512786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.351949+0000","last_change":"2023-11-08T19:24:49.605594+0000","last_active":"2023-11-08T19:24:52.351949+0000","last_peered":"2023-11-08T19:24:52.351949+0000","last_clean":"2023-11-08T19:24:52.351949+0000","last_became_active":"2023-11-08T19:24:49.605208+0000","last_became_peered":"2023-11-08T19:24:49.605208+0000","last_unstale":"2023-11-08T19:24:52.351949+0000","last_undegraded":"2023-11-08T19:24:52.351949+0000","last_fullsized":"2023-11-08T19:24:52.351949+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:17:50.005667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":162,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-11-08T19:24:48.592313+0000","last_change":"2023-11-08T19:24:45.888122+0000","last_active":"2023-11-08T19:24:48.592313+0000","last_peered":"2023-11-08T19:24:48.592313+0000","last_clean":"2023-11-08T19:24:48.592313+0000","last_became_active":"2023-11-08T19:24:45.887048+0000","last_became_peered":"2023-11-08T19:24:45.887048+0000","last_unstale":"2023-11-08T19:24:48.592313+0000","last_undegraded":"2023-11-08T19:24:48.592313+0000","last_fullsized":"2023-11-08T19:24:48.592313+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:44.575133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:44.575133+0000","last_clean_scrub_stamp":"2023-11-08T19:24:44.575133+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:19:20.202062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.309622+0000","last_change":"2023-11-08T19:24:50.628353+0000","last_active":"2023-11-08T19:24:52.309622+0000","last_peered":"2023-11-08T19:24:52.309622+0000","last_clean":"2023-11-08T19:24:52.309622+0000","last_became_active":"2023-11-08T19:24:50.628008+0000","last_became_peered":"2023-11-08T19:24:50.628008+0000","last_unstale":"2023-11-08T19:24:52.309622+0000","last_undegraded":"2023-11-08T19:24:52.309622+0000","last_fullsized":"2023-11-08T19:24:52.309622+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:46:13.497187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-08T19:24:48.591491+0000","last_change":"2023-11-08T19:24:48.591491+0000","last_active":"2023-11-08T19:24:48.584965+0000","last_peered":"2023-11-08T19:24:48.584965+0000","last_clean":"2023-11-08T19:24:48.584965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:24:48.591491+0000","last_undegraded":"2023-11-08T19:24:48.591491+0000","last_fullsized":"2023-11-08T19:24:48.591491+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-08T19:24:48.591465+0000","last_change":"2023-11-08T19:24:48.591465+0000","last_active":"2023-11-08T19:24:48.584965+0000","last_peered":"2023-11-08T19:24:48.584965+0000","last_clean":"2023-11-08T19:24:48.584965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:24:48.591465+0000","last_undegraded":"2023-11-08T19:24:48.591465+0000","last_fullsized":"2023-11-08T19:24:48.591465+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.805231+0000","last_change":"2023-11-08T19:24:50.624279+0000","last_active":"2023-11-08T19:24:52.805231+0000","last_peered":"2023-11-08T19:24:52.805231+0000","last_clean":"2023-11-08T19:24:52.805231+0000","last_became_active":"2023-11-08T19:24:50.623920+0000","last_became_peered":"2023-11-08T19:24:50.623920+0000","last_unstale":"2023-11-08T19:24:52.805231+0000","last_undegraded":"2023-11-08T19:24:52.805231+0000","last_fullsized":"2023-11-08T19:24:52.805231+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:42:42.512786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.805402+0000","last_change":"2023-11-08T19:24:49.606910+0000","last_active":"2023-11-08T19:24:52.805402+0000","last_peered":"2023-11-08T19:24:52.805402+0000","last_clean":"2023-11-08T19:24:52.805402+0000","last_became_active":"2023-11-08T19:24:49.606627+0000","last_became_peered":"2023-11-08T19:24:49.606627+0000","last_unstale":"2023-11-08T19:24:52.805402+0000","last_undegraded":"2023-11-08T19:24:52.805402+0000","last_fullsized":"2023-11-08T19:24:52.805402+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:56:00.148860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.627087+0000","last_change":"2023-11-08T19:24:50.622398+0000","last_active":"2023-11-08T19:24:51.627087+0000","last_peered":"2023-11-08T19:24:51.627087+0000","last_clean":"2023-11-08T19:24:51.627087+0000","last_became_active":"2023-11-08T19:24:50.622092+0000","last_became_peered":"2023-11-08T19:24:50.622092+0000","last_unstale":"2023-11-08T19:24:51.627087+0000","last_undegraded":"2023-11-08T19:24:51.627087+0000","last_fullsized":"2023-11-08T19:24:51.627087+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:42:27.150066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-08T19:24:48.592846+0000","last_change":"2023-11-08T19:24:48.592846+0000","last_active":"2023-11-08T19:24:48.584965+0000","last_peered":"2023-11-08T19:24:48.584965+0000","last_clean":"2023-11-08T19:24:48.584965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:24:48.592846+0000","last_undegraded":"2023-11-08T19:24:48.592846+0000","last_fullsized":"2023-11-08T19:24:48.592846+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.309770+0000","last_change":"2023-11-08T19:24:50.624848+0000","last_active":"2023-11-08T19:24:52.309770+0000","last_peered":"2023-11-08T19:24:52.309770+0000","last_clean":"2023-11-08T19:24:52.309770+0000","last_became_active":"2023-11-08T19:24:50.624518+0000","last_became_peered":"2023-11-08T19:24:50.624518+0000","last_unstale":"2023-11-08T19:24:52.309770+0000","last_undegraded":"2023-11-08T19:24:52.309770+0000","last_fullsized":"2023-11-08T19:24:52.309770+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:15:33.285449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-08T19:24:48.592059+0000","last_change":"2023-11-08T19:24:48.592059+0000","last_active":"2023-11-08T19:24:48.584965+0000","last_peered":"2023-11-08T19:24:48.584965+0000","last_clean":"2023-11-08T19:24:48.584965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:24:48.592059+0000","last_undegraded":"2023-11-08T19:24:48.592059+0000","last_fullsized":"2023-11-08T19:24:48.592059+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.805558+0000","last_change":"2023-11-08T19:24:50.627488+0000","last_active":"2023-11-08T19:24:52.805558+0000","last_peered":"2023-11-08T19:24:52.805558+0000","last_clean":"2023-11-08T19:24:52.805558+0000","last_became_active":"2023-11-08T19:24:50.626571+0000","last_became_peered":"2023-11-08T19:24:50.626571+0000","last_unstale":"2023-11-08T19:24:52.805558+0000","last_undegraded":"2023-11-08T19:24:52.805558+0000","last_fullsized":"2023-11-08T19:24:52.805558+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:43:00.849357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.804911+0000","last_change":"2023-11-08T19:24:49.606579+0000","last_active":"2023-11-08T19:24:52.804911+0000","last_peered":"2023-11-08T19:24:52.804911+0000","last_clean":"2023-11-08T19:24:52.804911+0000","last_became_active":"2023-11-08T19:24:49.606283+0000","last_became_peered":"2023-11-08T19:24:49.606283+0000","last_unstale":"2023-11-08T19:24:52.804911+0000","last_undegraded":"2023-11-08T19:24:52.804911+0000","last_fullsized":"2023-11-08T19:24:52.804911+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:14:00.455497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.626421+0000","last_change":"2023-11-08T19:24:50.625692+0000","last_active":"2023-11-08T19:24:51.626421+0000","last_peered":"2023-11-08T19:24:51.626421+0000","last_clean":"2023-11-08T19:24:51.626421+0000","last_became_active":"2023-11-08T19:24:50.625014+0000","last_became_peered":"2023-11-08T19:24:50.625014+0000","last_unstale":"2023-11-08T19:24:51.626421+0000","last_undegraded":"2023-11-08T19:24:51.626421+0000","last_fullsized":"2023-11-08T19:24:51.626421+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:15:33.285449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.805081+0000","last_change":"2023-11-08T19:24:49.611023+0000","last_active":"2023-11-08T19:24:52.805081+0000","last_peered":"2023-11-08T19:24:52.805081+0000","last_clean":"2023-11-08T19:24:52.805081+0000","last_became_active":"2023-11-08T19:24:49.610735+0000","last_became_peered":"2023-11-08T19:24:49.610735+0000","last_unstale":"2023-11-08T19:24:52.805081+0000","last_undegraded":"2023-11-08T19:24:52.805081+0000","last_fullsized":"2023-11-08T19:24:52.805081+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:30:31.599200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.626730+0000","last_change":"2023-11-08T19:24:50.631491+0000","last_active":"2023-11-08T19:24:51.626730+0000","last_peered":"2023-11-08T19:24:51.626730+0000","last_clean":"2023-11-08T19:24:51.626730+0000","last_became_active":"2023-11-08T19:24:50.630872+0000","last_became_peered":"2023-11-08T19:24:50.630872+0000","last_unstale":"2023-11-08T19:24:51.626730+0000","last_undegraded":"2023-11-08T19:24:51.626730+0000","last_fullsized":"2023-11-08T19:24:51.626730+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:36:53.967796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-08T19:24:48.592103+0000","last_change":"2023-11-08T19:24:48.592103+0000","last_active":"2023-11-08T19:24:48.584965+0000","last_peered":"2023-11-08T19:24:48.584965+0000","last_clean":"2023-11-08T19:24:48.584965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:24:48.592103+0000","last_undegraded":"2023-11-08T19:24:48.592103+0000","last_fullsized":"2023-11-08T19:24:48.592103+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:24:49.588741+0000","last_change":"2023-11-08T19:24:49.588741+0000","last_active":"2023-11-08T19:24:49.588741+0000","last_peered":"2023-11-08T19:24:49.588741+0000","last_clean":"2023-11-08T19:24:49.588741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:24:49.588741+0000","last_undegraded":"2023-11-08T19:24:49.588741+0000","last_fullsized":"2023-11-08T19:24:49.588741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.805091+0000","last_change":"2023-11-08T19:24:49.613365+0000","last_active":"2023-11-08T19:24:52.805091+0000","last_peered":"2023-11-08T19:24:52.805091+0000","last_clean":"2023-11-08T19:24:52.805091+0000","last_became_active":"2023-11-08T19:24:49.613052+0000","last_became_peered":"2023-11-08T19:24:49.613052+0000","last_unstale":"2023-11-08T19:24:52.805091+0000","last_undegraded":"2023-11-08T19:24:52.805091+0000","last_fullsized":"2023-11-08T19:24:52.805091+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:41:24.551269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:24:49.588741+0000","last_change":"2023-11-08T19:24:49.588741+0000","last_active":"2023-11-08T19:24:49.588741+0000","last_peered":"2023-11-08T19:24:49.588741+0000","last_clean":"2023-11-08T19:24:49.588741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:24:49.588741+0000","last_undegraded":"2023-11-08T19:24:49.588741+0000","last_fullsized":"2023-11-08T19:24:49.588741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.805219+0000","last_change":"2023-11-08T19:24:49.606271+0000","last_active":"2023-11-08T19:24:52.805219+0000","last_peered":"2023-11-08T19:24:52.805219+0000","last_clean":"2023-11-08T19:24:52.805219+0000","last_became_active":"2023-11-08T19:24:49.605802+0000","last_became_peered":"2023-11-08T19:24:49.605802+0000","last_unstale":"2023-11-08T19:24:52.805219+0000","last_undegraded":"2023-11-08T19:24:52.805219+0000","last_fullsized":"2023-11-08T19:24:52.805219+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:07:00.575561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.352338+0000","last_change":"2023-11-08T19:24:50.640594+0000","last_active":"2023-11-08T19:24:52.352338+0000","last_peered":"2023-11-08T19:24:52.352338+0000","last_clean":"2023-11-08T19:24:52.352338+0000","last_became_active":"2023-11-08T19:24:50.640295+0000","last_became_peered":"2023-11-08T19:24:50.640295+0000","last_unstale":"2023-11-08T19:24:52.352338+0000","last_undegraded":"2023-11-08T19:24:52.352338+0000","last_fullsized":"2023-11-08T19:24:52.352338+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:34:42.075394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.805026+0000","last_change":"2023-11-08T19:24:49.610782+0000","last_active":"2023-11-08T19:24:52.805026+0000","last_peered":"2023-11-08T19:24:52.805026+0000","last_clean":"2023-11-08T19:24:52.805026+0000","last_became_active":"2023-11-08T19:24:49.610493+0000","last_became_peered":"2023-11-08T19:24:49.610493+0000","last_unstale":"2023-11-08T19:24:52.805026+0000","last_undegraded":"2023-11-08T19:24:52.805026+0000","last_fullsized":"2023-11-08T19:24:52.805026+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:42:59.845581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:24:49.588741+0000","last_change":"2023-11-08T19:24:49.588741+0000","last_active":"2023-11-08T19:24:49.588741+0000","last_peered":"2023-11-08T19:24:49.588741+0000","last_clean":"2023-11-08T19:24:49.588741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:24:49.588741+0000","last_undegraded":"2023-11-08T19:24:49.588741+0000","last_fullsized":"2023-11-08T19:24:49.588741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:24:49.588741+0000","last_change":"2023-11-08T19:24:49.588741+0000","last_active":"2023-11-08T19:24:49.588741+0000","last_peered":"2023-11-08T19:24:49.588741+0000","last_clean":"2023-11-08T19:24:49.588741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:24:49.588741+0000","last_undegraded":"2023-11-08T19:24:49.588741+0000","last_fullsized":"2023-11-08T19:24:49.588741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.804775+0000","last_change":"2023-11-08T19:24:50.625948+0000","last_active":"2023-11-08T19:24:52.804775+0000","last_peered":"2023-11-08T19:24:52.804775+0000","last_clean":"2023-11-08T19:24:52.804775+0000","last_became_active":"2023-11-08T19:24:50.625633+0000","last_became_peered":"2023-11-08T19:24:50.625633+0000","last_unstale":"2023-11-08T19:24:52.804775+0000","last_undegraded":"2023-11-08T19:24:52.804775+0000","last_fullsized":"2023-11-08T19:24:52.804775+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:00:13.071654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.352423+0000","last_change":"2023-11-08T19:24:50.640919+0000","last_active":"2023-11-08T19:24:52.352423+0000","last_peered":"2023-11-08T19:24:52.352423+0000","last_clean":"2023-11-08T19:24:52.352423+0000","last_became_active":"2023-11-08T19:24:50.640496+0000","last_became_peered":"2023-11-08T19:24:50.640496+0000","last_unstale":"2023-11-08T19:24:52.352423+0000","last_undegraded":"2023-11-08T19:24:52.352423+0000","last_fullsized":"2023-11-08T19:24:52.352423+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:59:37.592538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.309782+0000","last_change":"2023-11-08T19:24:50.625916+0000","last_active":"2023-11-08T19:24:52.309782+0000","last_peered":"2023-11-08T19:24:52.309782+0000","last_clean":"2023-11-08T19:24:52.309782+0000","last_became_active":"2023-11-08T19:24:50.625625+0000","last_became_peered":"2023-11-08T19:24:50.625625+0000","last_unstale":"2023-11-08T19:24:52.309782+0000","last_undegraded":"2023-11-08T19:24:52.309782+0000","last_fullsized":"2023-11-08T19:24:52.309782+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:40:11.978983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.626555+0000","last_change":"2023-11-08T19:24:50.628686+0000","last_active":"2023-11-08T19:24:51.626555+0000","last_peered":"2023-11-08T19:24:51.626555+0000","last_clean":"2023-11-08T19:24:51.626555+0000","last_became_active":"2023-11-08T19:24:50.628289+0000","last_became_peered":"2023-11-08T19:24:50.628289+0000","last_unstale":"2023-11-08T19:24:51.626555+0000","last_undegraded":"2023-11-08T19:24:51.626555+0000","last_fullsized":"2023-11-08T19:24:51.626555+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:46:19.375897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.805231+0000","last_change":"2023-11-08T19:24:50.624352+0000","last_active":"2023-11-08T19:24:52.805231+0000","last_peered":"2023-11-08T19:24:52.805231+0000","last_clean":"2023-11-08T19:24:52.805231+0000","last_became_active":"2023-11-08T19:24:50.624035+0000","last_became_peered":"2023-11-08T19:24:50.624035+0000","last_unstale":"2023-11-08T19:24:52.805231+0000","last_undegraded":"2023-11-08T19:24:52.805231+0000","last_fullsized":"2023-11-08T19:24:52.805231+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:46:19.375897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:24:49.588741+0000","last_change":"2023-11-08T19:24:49.588741+0000","last_active":"2023-11-08T19:24:49.588741+0000","last_peered":"2023-11-08T19:24:49.588741+0000","last_clean":"2023-11-08T19:24:49.588741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:24:49.588741+0000","last_undegraded":"2023-11-08T19:24:49.588741+0000","last_fullsized":"2023-11-08T19:24:49.588741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.626991+0000","last_change":"2023-11-08T19:24:50.628009+0000","last_active":"2023-11-08T19:24:51.626991+0000","last_peered":"2023-11-08T19:24:51.626991+0000","last_clean":"2023-11-08T19:24:51.626991+0000","last_became_active":"2023-11-08T19:24:50.627668+0000","last_became_peered":"2023-11-08T19:24:50.627668+0000","last_unstale":"2023-11-08T19:24:51.626991+0000","last_undegraded":"2023-11-08T19:24:51.626991+0000","last_fullsized":"2023-11-08T19:24:51.626991+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:49:19.100370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.352100+0000","last_change":"2023-11-08T19:24:50.641255+0000","last_active":"2023-11-08T19:24:52.352100+0000","last_peered":"2023-11-08T19:24:52.352100+0000","last_clean":"2023-11-08T19:24:52.352100+0000","last_became_active":"2023-11-08T19:24:50.640164+0000","last_became_peered":"2023-11-08T19:24:50.640164+0000","last_unstale":"2023-11-08T19:24:52.352100+0000","last_undegraded":"2023-11-08T19:24:52.352100+0000","last_fullsized":"2023-11-08T19:24:52.352100+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:12:54.641952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.626435+0000","last_change":"2023-11-08T19:24:50.629006+0000","last_active":"2023-11-08T19:24:51.626435+0000","last_peered":"2023-11-08T19:24:51.626435+0000","last_clean":"2023-11-08T19:24:51.626435+0000","last_became_active":"2023-11-08T19:24:50.628685+0000","last_became_peered":"2023-11-08T19:24:50.628685+0000","last_unstale":"2023-11-08T19:24:51.626435+0000","last_undegraded":"2023-11-08T19:24:51.626435+0000","last_fullsized":"2023-11-08T19:24:51.626435+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:30:36.041820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.352270+0000","last_change":"2023-11-08T19:24:50.640737+0000","last_active":"2023-11-08T19:24:52.352270+0000","last_peered":"2023-11-08T19:24:52.352270+0000","last_clean":"2023-11-08T19:24:52.352270+0000","last_became_active":"2023-11-08T19:24:50.640255+0000","last_became_peered":"2023-11-08T19:24:50.640255+0000","last_unstale":"2023-11-08T19:24:52.352270+0000","last_undegraded":"2023-11-08T19:24:52.352270+0000","last_fullsized":"2023-11-08T19:24:52.352270+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:13:41.469149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.309753+0000","last_change":"2023-11-08T19:24:50.620645+0000","last_active":"2023-11-08T19:24:52.309753+0000","last_peered":"2023-11-08T19:24:52.309753+0000","last_clean":"2023-11-08T19:24:52.309753+0000","last_became_active":"2023-11-08T19:24:50.620349+0000","last_became_peered":"2023-11-08T19:24:50.620349+0000","last_unstale":"2023-11-08T19:24:52.309753+0000","last_undegraded":"2023-11-08T19:24:52.309753+0000","last_fullsized":"2023-11-08T19:24:52.309753+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:30:32.602976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:24:49.588741+0000","last_change":"2023-11-08T19:24:49.588741+0000","last_active":"2023-11-08T19:24:49.588741+0000","last_peered":"2023-11-08T19:24:49.588741+0000","last_clean":"2023-11-08T19:24:49.588741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:24:49.588741+0000","last_undegraded":"2023-11-08T19:24:49.588741+0000","last_fullsized":"2023-11-08T19:24:49.588741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":92,"acting":92,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":798,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":2626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":23,"ondisk_log_size":23,"up":64,"acting":64,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27760,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344080,"statfs":{"total":96636764160,"available":96608337920,"internally_reserved":0,"allocated":761856,"data_stored":641928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47099999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52000000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.432}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56200000000000006}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.372}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67900000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57599999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":167936,"data_stored":51560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47599999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.495}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51900000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56200000000000006}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39400000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":71070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36299999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23499999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26500000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45200000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47699999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":811008,"data_stored":661916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35099999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27400000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40000000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73199999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71099999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68300000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81000000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":192512,"data_stored":64610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46000000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.443}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82899999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81599999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84799999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.878}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.308}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27000000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.379}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59399999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50900000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54100000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70099999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":71036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46000000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55800000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65400000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75800000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67200000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.746}]}]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":208896,"data_stored":71000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39800000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55300000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61099999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.439}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66700000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69499999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-08T19:24:54.731 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-08T19:24:54.862 INFO:tasks.ceph.mds.b.smithi174.stderr:2023-11-08T19:24:54.862+0000 7f9d74556700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-08T19:24:54.883 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.882+0000 7fcb6088f700 1 Processor -- start 2023-11-08T19:24:54.884 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.882+0000 7fcb6088f700 1 -- start start 2023-11-08T19:24:54.886 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.883+0000 7fcb6088f700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fcb5c15dd00 0x7fcb5c0cca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:54.887 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.883+0000 7fcb6088f700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fcb5c06b320 0x7fcb5c0cb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:54.887 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.883+0000 7fcb6088f700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb5c0ccfb0 con 0x7fcb5c1618f0 2023-11-08T19:24:54.887 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.883+0000 7fcb6088f700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb5c0cad50 con 0x7fcb5c06b320 2023-11-08T19:24:54.887 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.883+0000 7fcb6088f700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb5c161df0 con 0x7fcb5c15dd00 2023-11-08T19:24:54.887 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.883+0000 7fcb5ad9d700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fcb5c06b320 0x7fcb5c0cb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:54.888 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.883+0000 7fcb59d9b700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fcb5c1618f0 0x7fcb5c06ac10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:44824/0 (socket says 172.21.15.143:44824) 2023-11-08T19:24:54.888 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.883+0000 7fcb59d9b700 1 -- 172.21.15.143:0/4009257742 learned_addr learned my addr 172.21.15.143:0/4009257742 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:24:54.888 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.883+0000 7fcb5a59c700 1 --2- 172.21.15.143:0/4009257742 >> v2:172.21.15.174:3300/0 conn(0x7fcb5c15dd00 0x7fcb5c0cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:54.888 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.884+0000 7fcb5ad9d700 1 -- 172.21.15.143:0/4009257742 >> v2:172.21.15.174:3300/0 conn(0x7fcb5c15dd00 msgr2=0x7fcb5c0cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:24:54.888 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.884+0000 7fcb5ad9d700 1 --2- 172.21.15.143:0/4009257742 >> v2:172.21.15.174:3300/0 conn(0x7fcb5c15dd00 0x7fcb5c0cca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:54.889 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.884+0000 7fcb5ad9d700 1 -- 172.21.15.143:0/4009257742 >> v1:172.21.15.143:6789/0 conn(0x7fcb5c1618f0 legacy=0x7fcb5c06ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:54.889 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.884+0000 7fcb5ad9d700 1 -- 172.21.15.143:0/4009257742 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb5c1620d0 con 0x7fcb5c06b320 2023-11-08T19:24:54.889 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.884+0000 7fcb5a59c700 1 --2- 172.21.15.143:0/4009257742 >> v2:172.21.15.174:3300/0 conn(0x7fcb5c15dd00 0x7fcb5c0cca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:24:54.889 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.884+0000 7fcb5ad9d700 1 --2- 172.21.15.143:0/4009257742 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcb5c06b320 0x7fcb5c0cb1a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c070a60 tx=0x7fcb5400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f77a6bbd2e8c846 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:54.889 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.884+0000 7fcb5959a700 1 -- 172.21.15.143:0/4009257742 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb5400e070 con 0x7fcb5c06b320 2023-11-08T19:24:54.889 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.884+0000 7fcb5959a700 1 -- 172.21.15.143:0/4009257742 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb5400bbd0 con 0x7fcb5c06b320 2023-11-08T19:24:54.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.884+0000 7fcb5959a700 1 -- 172.21.15.143:0/4009257742 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb54012430 con 0x7fcb5c06b320 2023-11-08T19:24:54.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.884+0000 7fcb6088f700 1 -- 172.21.15.143:0/4009257742 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcb5c06b320 msgr2=0x7fcb5c0cb1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:54.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.884+0000 7fcb6088f700 1 --2- 172.21.15.143:0/4009257742 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcb5c06b320 0x7fcb5c0cb1a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c070a60 tx=0x7fcb5400b330 comp rx=0 tx=0).stop 2023-11-08T19:24:54.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.884+0000 7fcb6088f700 1 -- 172.21.15.143:0/4009257742 shutdown_connections 2023-11-08T19:24:54.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.884+0000 7fcb6088f700 1 --2- 172.21.15.143:0/4009257742 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcb5c06b320 0x7fcb5c0cb1a0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:54.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.884+0000 7fcb6088f700 1 --2- 172.21.15.143:0/4009257742 >> v2:172.21.15.174:3300/0 conn(0x7fcb5c15dd00 0x7fcb5c0cca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:54.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.884+0000 7fcb6088f700 1 -- 172.21.15.143:0/4009257742 >> 172.21.15.143:0/4009257742 conn(0x7fcb5c099ab0 msgr2=0x7fcb5c0d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:24:54.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.884+0000 7fcb6088f700 1 -- 172.21.15.143:0/4009257742 shutdown_connections 2023-11-08T19:24:54.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.884+0000 7fcb6088f700 1 -- 172.21.15.143:0/4009257742 wait complete. 2023-11-08T19:24:54.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.886+0000 7fcb6088f700 1 Processor -- start 2023-11-08T19:24:54.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.886+0000 7fcb6088f700 1 -- start start 2023-11-08T19:24:54.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.887+0000 7fcb6088f700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcb5c06b320 0x7fcb5c14e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:54.895 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.887+0000 7fcb6088f700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcb5c15dd00 0x7fcb5c14dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:54.895 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.887+0000 7fcb6088f700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcb5c1618f0 0x7fcb5c14a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:54.895 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.887+0000 7fcb6088f700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb5c14ead0 con 0x7fcb5c15dd00 2023-11-08T19:24:54.895 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.887+0000 7fcb6088f700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb5c14e3e0 con 0x7fcb5c06b320 2023-11-08T19:24:54.896 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.887+0000 7fcb6088f700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb5c0b5980 con 0x7fcb5c1618f0 2023-11-08T19:24:54.896 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.887+0000 7fcb59d9b700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcb5c15dd00 0x7fcb5c14dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:54.896 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.887+0000 7fcb59d9b700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcb5c15dd00 0x7fcb5c14dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48678/0 (socket says 172.21.15.143:48678) 2023-11-08T19:24:54.896 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.887+0000 7fcb59d9b700 1 -- 172.21.15.143:0/962147673 learned_addr learned my addr 172.21.15.143:0/962147673 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:24:54.896 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.887+0000 7fcb5a59c700 1 --2- 172.21.15.143:0/962147673 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcb5c06b320 0x7fcb5c14e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:54.897 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.887+0000 7fcb5ad9d700 1 --2- 172.21.15.143:0/962147673 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcb5c1618f0 0x7fcb5c14a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:54.897 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.887+0000 7fcb59d9b700 1 -- 172.21.15.143:0/962147673 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcb5c1618f0 msgr2=0x7fcb5c14a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:54.897 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.887+0000 7fcb59d9b700 1 --2- 172.21.15.143:0/962147673 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcb5c1618f0 0x7fcb5c14a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:54.897 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.887+0000 7fcb59d9b700 1 -- 172.21.15.143:0/962147673 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcb5c06b320 msgr2=0x7fcb5c14e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:54.897 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.887+0000 7fcb59d9b700 1 --2- 172.21.15.143:0/962147673 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcb5c06b320 0x7fcb5c14e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:54.897 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.887+0000 7fcb59d9b700 1 -- 172.21.15.143:0/962147673 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb54009a70 con 0x7fcb5c15dd00 2023-11-08T19:24:54.898 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.887+0000 7fcb59d9b700 1 --2- 172.21.15.143:0/962147673 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcb5c15dd00 0x7fcb5c14dea0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c00c480 tx=0x7fcb4c00c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:54.898 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.887+0000 7fcb4affd700 1 -- 172.21.15.143:0/962147673 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb4c005d70 con 0x7fcb5c15dd00 2023-11-08T19:24:54.898 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.887+0000 7fcb4affd700 1 -- 172.21.15.143:0/962147673 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb4c0070d0 con 0x7fcb5c15dd00 2023-11-08T19:24:54.898 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.887+0000 7fcb6088f700 1 -- 172.21.15.143:0/962147673 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb5c0b5c60 con 0x7fcb5c15dd00 2023-11-08T19:24:54.898 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.887+0000 7fcb4affd700 1 -- 172.21.15.143:0/962147673 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb4c007440 con 0x7fcb5c15dd00 2023-11-08T19:24:54.899 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.887+0000 7fcb6088f700 1 -- 172.21.15.143:0/962147673 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb5c0b6220 con 0x7fcb5c15dd00 2023-11-08T19:24:54.899 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.888+0000 7fcb6088f700 1 -- 172.21.15.143:0/962147673 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb5c0aff60 con 0x7fcb5c15dd00 2023-11-08T19:24:54.899 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.889+0000 7fcb4affd700 1 -- 172.21.15.143:0/962147673 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fcb4c0075a0 con 0x7fcb5c15dd00 2023-11-08T19:24:54.899 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.889+0000 7fcb4affd700 1 --2- 172.21.15.143:0/962147673 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fcb3c06af00 0x7fcb3c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:54.899 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.889+0000 7fcb4affd700 1 -- 172.21.15.143:0/962147673 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcb4c088e30 con 0x7fcb5c15dd00 2023-11-08T19:24:54.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.890+0000 7fcb5a59c700 1 --2- 172.21.15.143:0/962147673 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fcb3c06af00 0x7fcb3c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:54.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.890+0000 7fcb5a59c700 1 --2- 172.21.15.143:0/962147673 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fcb3c06af00 0x7fcb3c06d3c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fcb5000c920 tx=0x7fcb50008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:54.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:54.893+0000 7fcb4affd700 1 -- 172.21.15.143:0/962147673 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcb4c054070 con 0x7fcb5c15dd00 2023-11-08T19:24:55.079 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.077+0000 7fcb6088f700 1 -- 172.21.15.143:0/962147673 --> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcb5c162690 con 0x7fcb3c06af00 2023-11-08T19:24:55.087 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.086+0000 7fcb4affd700 1 -- 172.21.15.143:0/962147673 <== mgr.4099 v2:172.21.15.174:6832/4037280178 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+261184 (secure 0 0 0) 0x7fcb5000c5b0 con 0x7fcb3c06af00 2023-11-08T19:24:55.091 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:24:55.093 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.089+0000 7fcb6088f700 1 -- 172.21.15.143:0/962147673 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fcb3c06af00 msgr2=0x7fcb3c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:55.093 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.089+0000 7fcb6088f700 1 --2- 172.21.15.143:0/962147673 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fcb3c06af00 0x7fcb3c06d3c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fcb5000c920 tx=0x7fcb50008040 comp rx=0 tx=0).stop 2023-11-08T19:24:55.093 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.089+0000 7fcb6088f700 1 -- 172.21.15.143:0/962147673 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcb5c15dd00 msgr2=0x7fcb5c14dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:55.094 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.089+0000 7fcb6088f700 1 --2- 172.21.15.143:0/962147673 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcb5c15dd00 0x7fcb5c14dea0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c00c480 tx=0x7fcb4c00c840 comp rx=0 tx=0).stop 2023-11-08T19:24:55.094 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.089+0000 7fcb6088f700 1 -- 172.21.15.143:0/962147673 shutdown_connections 2023-11-08T19:24:55.094 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.089+0000 7fcb6088f700 1 --2- 172.21.15.143:0/962147673 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fcb3c06af00 0x7fcb3c06d3c0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:55.094 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.089+0000 7fcb6088f700 1 --2- 172.21.15.143:0/962147673 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcb5c1618f0 0x7fcb5c14a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:55.094 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.089+0000 7fcb6088f700 1 --2- 172.21.15.143:0/962147673 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcb5c06b320 0x7fcb5c14e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:55.095 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.089+0000 7fcb6088f700 1 --2- 172.21.15.143:0/962147673 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcb5c15dd00 0x7fcb5c14dea0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:55.095 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.089+0000 7fcb6088f700 1 -- 172.21.15.143:0/962147673 >> 172.21.15.143:0/962147673 conn(0x7fcb5c099ab0 msgr2=0x7fcb5c0ad380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:24:55.095 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.089+0000 7fcb6088f700 1 -- 172.21.15.143:0/962147673 shutdown_connections 2023-11-08T19:24:55.095 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.089+0000 7fcb6088f700 1 -- 172.21.15.143:0/962147673 wait complete. 2023-11-08T19:24:55.095 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2023-11-08T19:24:55.103 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":18,"stamp":"2023-11-08T19:24:54.509481+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591166,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":187,"num_write_kb":2427,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":140,"ondisk_log_size":140,"up":158,"acting":158,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":143,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218904,"kb_used_data":2724,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755816,"statfs":{"total":773094113280,"available":772869955584,"internally_reserved":0,"allocated":2789376,"data_stored":1705642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":798,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.917376"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:24:49.588741+0000","last_change":"2023-11-08T19:24:49.588741+0000","last_active":"2023-11-08T19:24:49.588741+0000","last_peered":"2023-11-08T19:24:49.588741+0000","last_clean":"2023-11-08T19:24:49.588741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:24:49.588741+0000","last_undegraded":"2023-11-08T19:24:49.588741+0000","last_fullsized":"2023-11-08T19:24:49.588741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.805606+0000","last_change":"2023-11-08T19:24:50.623985+0000","last_active":"2023-11-08T19:24:52.805606+0000","last_peered":"2023-11-08T19:24:52.805606+0000","last_clean":"2023-11-08T19:24:52.805606+0000","last_became_active":"2023-11-08T19:24:50.623553+0000","last_became_peered":"2023-11-08T19:24:50.623553+0000","last_unstale":"2023-11-08T19:24:52.805606+0000","last_undegraded":"2023-11-08T19:24:52.805606+0000","last_fullsized":"2023-11-08T19:24:52.805606+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:13:58.948371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.309774+0000","last_change":"2023-11-08T19:24:50.617914+0000","last_active":"2023-11-08T19:24:52.309774+0000","last_peered":"2023-11-08T19:24:52.309774+0000","last_clean":"2023-11-08T19:24:52.309774+0000","last_became_active":"2023-11-08T19:24:50.617611+0000","last_became_peered":"2023-11-08T19:24:50.617611+0000","last_unstale":"2023-11-08T19:24:52.309774+0000","last_undegraded":"2023-11-08T19:24:52.309774+0000","last_fullsized":"2023-11-08T19:24:52.309774+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:30:36.041820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.352504+0000","last_change":"2023-11-08T19:24:50.640039+0000","last_active":"2023-11-08T19:24:52.352504+0000","last_peered":"2023-11-08T19:24:52.352504+0000","last_clean":"2023-11-08T19:24:52.352504+0000","last_became_active":"2023-11-08T19:24:50.639560+0000","last_became_peered":"2023-11-08T19:24:50.639560+0000","last_unstale":"2023-11-08T19:24:52.352504+0000","last_undegraded":"2023-11-08T19:24:52.352504+0000","last_fullsized":"2023-11-08T19:24:52.352504+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:01:53.410545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:24:49.588741+0000","last_change":"2023-11-08T19:24:49.588741+0000","last_active":"2023-11-08T19:24:49.588741+0000","last_peered":"2023-11-08T19:24:49.588741+0000","last_clean":"2023-11-08T19:24:49.588741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:24:49.588741+0000","last_undegraded":"2023-11-08T19:24:49.588741+0000","last_fullsized":"2023-11-08T19:24:49.588741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.351953+0000","last_change":"2023-11-08T19:24:50.641005+0000","last_active":"2023-11-08T19:24:52.351953+0000","last_peered":"2023-11-08T19:24:52.351953+0000","last_clean":"2023-11-08T19:24:52.351953+0000","last_became_active":"2023-11-08T19:24:50.640727+0000","last_became_peered":"2023-11-08T19:24:50.640727+0000","last_unstale":"2023-11-08T19:24:52.351953+0000","last_undegraded":"2023-11-08T19:24:52.351953+0000","last_fullsized":"2023-11-08T19:24:52.351953+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:36:53.967796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:24:49.588741+0000","last_change":"2023-11-08T19:24:49.588741+0000","last_active":"2023-11-08T19:24:49.588741+0000","last_peered":"2023-11-08T19:24:49.588741+0000","last_clean":"2023-11-08T19:24:49.588741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:24:49.588741+0000","last_undegraded":"2023-11-08T19:24:49.588741+0000","last_fullsized":"2023-11-08T19:24:49.588741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.626351+0000","last_change":"2023-11-08T19:24:50.639325+0000","last_active":"2023-11-08T19:24:51.626351+0000","last_peered":"2023-11-08T19:24:51.626351+0000","last_clean":"2023-11-08T19:24:51.626351+0000","last_became_active":"2023-11-08T19:24:50.638993+0000","last_became_peered":"2023-11-08T19:24:50.638993+0000","last_unstale":"2023-11-08T19:24:51.626351+0000","last_undegraded":"2023-11-08T19:24:51.626351+0000","last_fullsized":"2023-11-08T19:24:51.626351+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:30:32.602976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:24:49.588741+0000","last_change":"2023-11-08T19:24:49.588741+0000","last_active":"2023-11-08T19:24:49.588741+0000","last_peered":"2023-11-08T19:24:49.588741+0000","last_clean":"2023-11-08T19:24:49.588741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:24:49.588741+0000","last_undegraded":"2023-11-08T19:24:49.588741+0000","last_fullsized":"2023-11-08T19:24:49.588741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:24:49.588741+0000","last_change":"2023-11-08T19:24:49.588741+0000","last_active":"2023-11-08T19:24:49.588741+0000","last_peered":"2023-11-08T19:24:49.588741+0000","last_clean":"2023-11-08T19:24:49.588741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:24:49.588741+0000","last_undegraded":"2023-11-08T19:24:49.588741+0000","last_fullsized":"2023-11-08T19:24:49.588741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:24:49.588741+0000","last_change":"2023-11-08T19:24:49.588741+0000","last_active":"2023-11-08T19:24:49.588741+0000","last_peered":"2023-11-08T19:24:49.588741+0000","last_clean":"2023-11-08T19:24:49.588741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:24:49.588741+0000","last_undegraded":"2023-11-08T19:24:49.588741+0000","last_fullsized":"2023-11-08T19:24:49.588741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.805767+0000","last_change":"2023-11-08T19:24:50.621511+0000","last_active":"2023-11-08T19:24:52.805767+0000","last_peered":"2023-11-08T19:24:52.805767+0000","last_clean":"2023-11-08T19:24:52.805767+0000","last_became_active":"2023-11-08T19:24:50.621142+0000","last_became_peered":"2023-11-08T19:24:50.621142+0000","last_unstale":"2023-11-08T19:24:52.805767+0000","last_undegraded":"2023-11-08T19:24:52.805767+0000","last_fullsized":"2023-11-08T19:24:52.805767+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:14:01.459274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.626625+0000","last_change":"2023-11-08T19:24:50.625632+0000","last_active":"2023-11-08T19:24:51.626625+0000","last_peered":"2023-11-08T19:24:51.626625+0000","last_clean":"2023-11-08T19:24:51.626625+0000","last_became_active":"2023-11-08T19:24:50.625370+0000","last_became_peered":"2023-11-08T19:24:50.625370+0000","last_unstale":"2023-11-08T19:24:51.626625+0000","last_undegraded":"2023-11-08T19:24:51.626625+0000","last_fullsized":"2023-11-08T19:24:51.626625+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:13:58.948371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.309818+0000","last_change":"2023-11-08T19:24:50.625350+0000","last_active":"2023-11-08T19:24:52.309818+0000","last_peered":"2023-11-08T19:24:52.309818+0000","last_clean":"2023-11-08T19:24:52.309818+0000","last_became_active":"2023-11-08T19:24:50.625078+0000","last_became_peered":"2023-11-08T19:24:50.625078+0000","last_unstale":"2023-11-08T19:24:52.309818+0000","last_undegraded":"2023-11-08T19:24:52.309818+0000","last_fullsized":"2023-11-08T19:24:52.309818+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:42:42.512786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.804817+0000","last_change":"2023-11-08T19:24:50.629187+0000","last_active":"2023-11-08T19:24:52.804817+0000","last_peered":"2023-11-08T19:24:52.804817+0000","last_clean":"2023-11-08T19:24:52.804817+0000","last_became_active":"2023-11-08T19:24:50.628898+0000","last_became_peered":"2023-11-08T19:24:50.628898+0000","last_unstale":"2023-11-08T19:24:52.804817+0000","last_undegraded":"2023-11-08T19:24:52.804817+0000","last_fullsized":"2023-11-08T19:24:52.804817+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:15:33.285449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.351906+0000","last_change":"2023-11-08T19:24:50.638579+0000","last_active":"2023-11-08T19:24:52.351906+0000","last_peered":"2023-11-08T19:24:52.351906+0000","last_clean":"2023-11-08T19:24:52.351906+0000","last_became_active":"2023-11-08T19:24:50.638297+0000","last_became_peered":"2023-11-08T19:24:50.638297+0000","last_unstale":"2023-11-08T19:24:52.351906+0000","last_undegraded":"2023-11-08T19:24:52.351906+0000","last_fullsized":"2023-11-08T19:24:52.351906+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:13:58.948371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.352572+0000","last_change":"2023-11-08T19:24:50.641446+0000","last_active":"2023-11-08T19:24:52.352572+0000","last_peered":"2023-11-08T19:24:52.352572+0000","last_clean":"2023-11-08T19:24:52.352572+0000","last_became_active":"2023-11-08T19:24:50.639709+0000","last_became_peered":"2023-11-08T19:24:50.639709+0000","last_unstale":"2023-11-08T19:24:52.352572+0000","last_undegraded":"2023-11-08T19:24:52.352572+0000","last_fullsized":"2023-11-08T19:24:52.352572+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:09:01.749686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.804922+0000","last_change":"2023-11-08T19:24:50.619241+0000","last_active":"2023-11-08T19:24:52.804922+0000","last_peered":"2023-11-08T19:24:52.804922+0000","last_clean":"2023-11-08T19:24:52.804922+0000","last_became_active":"2023-11-08T19:24:50.618962+0000","last_became_peered":"2023-11-08T19:24:50.618962+0000","last_unstale":"2023-11-08T19:24:52.804922+0000","last_undegraded":"2023-11-08T19:24:52.804922+0000","last_fullsized":"2023-11-08T19:24:52.804922+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:14:01.459274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:24:49.588741+0000","last_change":"2023-11-08T19:24:49.588741+0000","last_active":"2023-11-08T19:24:49.588741+0000","last_peered":"2023-11-08T19:24:49.588741+0000","last_clean":"2023-11-08T19:24:49.588741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:24:49.588741+0000","last_undegraded":"2023-11-08T19:24:49.588741+0000","last_fullsized":"2023-11-08T19:24:49.588741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.309784+0000","last_change":"2023-11-08T19:24:49.608481+0000","last_active":"2023-11-08T19:24:52.309784+0000","last_peered":"2023-11-08T19:24:52.309784+0000","last_clean":"2023-11-08T19:24:52.309784+0000","last_became_active":"2023-11-08T19:24:49.607778+0000","last_became_peered":"2023-11-08T19:24:49.607778+0000","last_unstale":"2023-11-08T19:24:52.309784+0000","last_undegraded":"2023-11-08T19:24:52.309784+0000","last_fullsized":"2023-11-08T19:24:52.309784+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:01:09.884262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.626900+0000","last_change":"2023-11-08T19:24:50.624852+0000","last_active":"2023-11-08T19:24:51.626900+0000","last_peered":"2023-11-08T19:24:51.626900+0000","last_clean":"2023-11-08T19:24:51.626900+0000","last_became_active":"2023-11-08T19:24:50.624551+0000","last_became_peered":"2023-11-08T19:24:50.624551+0000","last_unstale":"2023-11-08T19:24:51.626900+0000","last_undegraded":"2023-11-08T19:24:51.626900+0000","last_fullsized":"2023-11-08T19:24:51.626900+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:12:54.641952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.626938+0000","last_change":"2023-11-08T19:24:49.610418+0000","last_active":"2023-11-08T19:24:51.626938+0000","last_peered":"2023-11-08T19:24:51.626938+0000","last_clean":"2023-11-08T19:24:51.626938+0000","last_became_active":"2023-11-08T19:24:49.610041+0000","last_became_peered":"2023-11-08T19:24:49.610041+0000","last_unstale":"2023-11-08T19:24:51.626938+0000","last_undegraded":"2023-11-08T19:24:51.626938+0000","last_fullsized":"2023-11-08T19:24:51.626938+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:00:12.067877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.309779+0000","last_change":"2023-11-08T19:24:50.627052+0000","last_active":"2023-11-08T19:24:52.309779+0000","last_peered":"2023-11-08T19:24:52.309779+0000","last_clean":"2023-11-08T19:24:52.309779+0000","last_became_active":"2023-11-08T19:24:50.626751+0000","last_became_peered":"2023-11-08T19:24:50.626751+0000","last_unstale":"2023-11-08T19:24:52.309779+0000","last_undegraded":"2023-11-08T19:24:52.309779+0000","last_fullsized":"2023-11-08T19:24:52.309779+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:46:19.375897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-08T19:24:48.592048+0000","last_change":"2023-11-08T19:24:48.592048+0000","last_active":"2023-11-08T19:24:48.584965+0000","last_peered":"2023-11-08T19:24:48.584965+0000","last_clean":"2023-11-08T19:24:48.584965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:24:48.592048+0000","last_undegraded":"2023-11-08T19:24:48.592048+0000","last_fullsized":"2023-11-08T19:24:48.592048+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.352125+0000","last_change":"2023-11-08T19:24:50.641183+0000","last_active":"2023-11-08T19:24:52.352125+0000","last_peered":"2023-11-08T19:24:52.352125+0000","last_clean":"2023-11-08T19:24:52.352125+0000","last_became_active":"2023-11-08T19:24:50.640910+0000","last_became_peered":"2023-11-08T19:24:50.640910+0000","last_unstale":"2023-11-08T19:24:52.352125+0000","last_undegraded":"2023-11-08T19:24:52.352125+0000","last_fullsized":"2023-11-08T19:24:52.352125+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:49:19.100370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-08T19:24:48.591551+0000","last_change":"2023-11-08T19:24:48.591551+0000","last_active":"2023-11-08T19:24:48.584965+0000","last_peered":"2023-11-08T19:24:48.584965+0000","last_clean":"2023-11-08T19:24:48.584965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:24:48.591551+0000","last_undegraded":"2023-11-08T19:24:48.591551+0000","last_fullsized":"2023-11-08T19:24:48.591551+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.352284+0000","last_change":"2023-11-08T19:24:50.640245+0000","last_active":"2023-11-08T19:24:52.352284+0000","last_peered":"2023-11-08T19:24:52.352284+0000","last_clean":"2023-11-08T19:24:52.352284+0000","last_became_active":"2023-11-08T19:24:50.639668+0000","last_became_peered":"2023-11-08T19:24:50.639668+0000","last_unstale":"2023-11-08T19:24:52.352284+0000","last_undegraded":"2023-11-08T19:24:52.352284+0000","last_fullsized":"2023-11-08T19:24:52.352284+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:42:27.150066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.626599+0000","last_change":"2023-11-08T19:24:49.612665+0000","last_active":"2023-11-08T19:24:51.626599+0000","last_peered":"2023-11-08T19:24:51.626599+0000","last_clean":"2023-11-08T19:24:51.626599+0000","last_became_active":"2023-11-08T19:24:49.612362+0000","last_became_peered":"2023-11-08T19:24:49.612362+0000","last_unstale":"2023-11-08T19:24:51.626599+0000","last_undegraded":"2023-11-08T19:24:51.626599+0000","last_fullsized":"2023-11-08T19:24:51.626599+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:46:12.493411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.805419+0000","last_change":"2023-11-08T19:24:50.627537+0000","last_active":"2023-11-08T19:24:52.805419+0000","last_peered":"2023-11-08T19:24:52.805419+0000","last_clean":"2023-11-08T19:24:52.805419+0000","last_became_active":"2023-11-08T19:24:50.626489+0000","last_became_peered":"2023-11-08T19:24:50.626489+0000","last_unstale":"2023-11-08T19:24:52.805419+0000","last_undegraded":"2023-11-08T19:24:52.805419+0000","last_fullsized":"2023-11-08T19:24:52.805419+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:01:10.888039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-08T19:24:48.592162+0000","last_change":"2023-11-08T19:24:48.592162+0000","last_active":"2023-11-08T19:24:48.584965+0000","last_peered":"2023-11-08T19:24:48.584965+0000","last_clean":"2023-11-08T19:24:48.584965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:24:48.592162+0000","last_undegraded":"2023-11-08T19:24:48.592162+0000","last_fullsized":"2023-11-08T19:24:48.592162+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.309935+0000","last_change":"2023-11-08T19:24:50.627987+0000","last_active":"2023-11-08T19:24:52.309935+0000","last_peered":"2023-11-08T19:24:52.309935+0000","last_clean":"2023-11-08T19:24:52.309935+0000","last_became_active":"2023-11-08T19:24:50.627402+0000","last_became_peered":"2023-11-08T19:24:50.627402+0000","last_unstale":"2023-11-08T19:24:52.309935+0000","last_undegraded":"2023-11-08T19:24:52.309935+0000","last_fullsized":"2023-11-08T19:24:52.309935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:36:53.967796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.626523+0000","last_change":"2023-11-08T19:24:49.606482+0000","last_active":"2023-11-08T19:24:51.626523+0000","last_peered":"2023-11-08T19:24:51.626523+0000","last_clean":"2023-11-08T19:24:51.626523+0000","last_became_active":"2023-11-08T19:24:49.606211+0000","last_became_peered":"2023-11-08T19:24:49.606211+0000","last_unstale":"2023-11-08T19:24:51.626523+0000","last_undegraded":"2023-11-08T19:24:51.626523+0000","last_fullsized":"2023-11-08T19:24:51.626523+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:42:59.845581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.626483+0000","last_change":"2023-11-08T19:24:50.625917+0000","last_active":"2023-11-08T19:24:51.626483+0000","last_peered":"2023-11-08T19:24:51.626483+0000","last_clean":"2023-11-08T19:24:51.626483+0000","last_became_active":"2023-11-08T19:24:50.625398+0000","last_became_peered":"2023-11-08T19:24:50.625398+0000","last_unstale":"2023-11-08T19:24:51.626483+0000","last_undegraded":"2023-11-08T19:24:51.626483+0000","last_fullsized":"2023-11-08T19:24:51.626483+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:40:11.978983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.351905+0000","last_change":"2023-11-08T19:24:49.616558+0000","last_active":"2023-11-08T19:24:52.351905+0000","last_peered":"2023-11-08T19:24:52.351905+0000","last_clean":"2023-11-08T19:24:52.351905+0000","last_became_active":"2023-11-08T19:24:49.616277+0000","last_became_peered":"2023-11-08T19:24:49.616277+0000","last_unstale":"2023-11-08T19:24:52.351905+0000","last_undegraded":"2023-11-08T19:24:52.351905+0000","last_fullsized":"2023-11-08T19:24:52.351905+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:01:09.884262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.805048+0000","last_change":"2023-11-08T19:24:50.623658+0000","last_active":"2023-11-08T19:24:52.805048+0000","last_peered":"2023-11-08T19:24:52.805048+0000","last_clean":"2023-11-08T19:24:52.805048+0000","last_became_active":"2023-11-08T19:24:50.623399+0000","last_became_peered":"2023-11-08T19:24:50.623399+0000","last_unstale":"2023-11-08T19:24:52.805048+0000","last_undegraded":"2023-11-08T19:24:52.805048+0000","last_fullsized":"2023-11-08T19:24:52.805048+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:40:11.978983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-08T19:24:48.593392+0000","last_change":"2023-11-08T19:24:48.593392+0000","last_active":"2023-11-08T19:24:48.584965+0000","last_peered":"2023-11-08T19:24:48.584965+0000","last_clean":"2023-11-08T19:24:48.584965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:24:48.593392+0000","last_undegraded":"2023-11-08T19:24:48.593392+0000","last_fullsized":"2023-11-08T19:24:48.593392+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.352305+0000","last_change":"2023-11-08T19:24:50.640397+0000","last_active":"2023-11-08T19:24:52.352305+0000","last_peered":"2023-11-08T19:24:52.352305+0000","last_clean":"2023-11-08T19:24:52.352305+0000","last_became_active":"2023-11-08T19:24:50.639730+0000","last_became_peered":"2023-11-08T19:24:50.639730+0000","last_unstale":"2023-11-08T19:24:52.352305+0000","last_undegraded":"2023-11-08T19:24:52.352305+0000","last_fullsized":"2023-11-08T19:24:52.352305+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:08:22.499163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.352212+0000","last_change":"2023-11-08T19:24:49.611202+0000","last_active":"2023-11-08T19:24:52.352212+0000","last_peered":"2023-11-08T19:24:52.352212+0000","last_clean":"2023-11-08T19:24:52.352212+0000","last_became_active":"2023-11-08T19:24:49.610894+0000","last_became_peered":"2023-11-08T19:24:49.610894+0000","last_unstale":"2023-11-08T19:24:52.352212+0000","last_undegraded":"2023-11-08T19:24:52.352212+0000","last_fullsized":"2023-11-08T19:24:52.352212+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:48:25.546248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.309935+0000","last_change":"2023-11-08T19:24:50.638901+0000","last_active":"2023-11-08T19:24:52.309935+0000","last_peered":"2023-11-08T19:24:52.309935+0000","last_clean":"2023-11-08T19:24:52.309935+0000","last_became_active":"2023-11-08T19:24:50.638587+0000","last_became_peered":"2023-11-08T19:24:50.638587+0000","last_unstale":"2023-11-08T19:24:52.309935+0000","last_undegraded":"2023-11-08T19:24:52.309935+0000","last_fullsized":"2023-11-08T19:24:52.309935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:13:58.948371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.805313+0000","last_change":"2023-11-08T19:24:49.612170+0000","last_active":"2023-11-08T19:24:52.805313+0000","last_peered":"2023-11-08T19:24:52.805313+0000","last_clean":"2023-11-08T19:24:52.805313+0000","last_became_active":"2023-11-08T19:24:49.611895+0000","last_became_peered":"2023-11-08T19:24:49.611895+0000","last_unstale":"2023-11-08T19:24:52.805313+0000","last_undegraded":"2023-11-08T19:24:52.805313+0000","last_fullsized":"2023-11-08T19:24:52.805313+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:38:24.246638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.805721+0000","last_change":"2023-11-08T19:24:50.627060+0000","last_active":"2023-11-08T19:24:52.805721+0000","last_peered":"2023-11-08T19:24:52.805721+0000","last_clean":"2023-11-08T19:24:52.805721+0000","last_became_active":"2023-11-08T19:24:50.626468+0000","last_became_peered":"2023-11-08T19:24:50.626468+0000","last_unstale":"2023-11-08T19:24:52.805721+0000","last_undegraded":"2023-11-08T19:24:52.805721+0000","last_fullsized":"2023-11-08T19:24:52.805721+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:00:13.071654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-08T19:24:48.591494+0000","last_change":"2023-11-08T19:24:48.591494+0000","last_active":"2023-11-08T19:24:48.584965+0000","last_peered":"2023-11-08T19:24:48.584965+0000","last_clean":"2023-11-08T19:24:48.584965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:24:48.591494+0000","last_undegraded":"2023-11-08T19:24:48.591494+0000","last_fullsized":"2023-11-08T19:24:48.591494+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:24:49.588741+0000","last_change":"2023-11-08T19:24:49.588741+0000","last_active":"2023-11-08T19:24:49.588741+0000","last_peered":"2023-11-08T19:24:49.588741+0000","last_clean":"2023-11-08T19:24:49.588741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:24:49.588741+0000","last_undegraded":"2023-11-08T19:24:49.588741+0000","last_fullsized":"2023-11-08T19:24:49.588741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.309884+0000","last_change":"2023-11-08T19:24:49.610471+0000","last_active":"2023-11-08T19:24:52.309884+0000","last_peered":"2023-11-08T19:24:52.309884+0000","last_clean":"2023-11-08T19:24:52.309884+0000","last_became_active":"2023-11-08T19:24:49.610182+0000","last_became_peered":"2023-11-08T19:24:49.610182+0000","last_unstale":"2023-11-08T19:24:52.309884+0000","last_undegraded":"2023-11-08T19:24:52.309884+0000","last_fullsized":"2023-11-08T19:24:52.309884+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:14:00.455497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.309602+0000","last_change":"2023-11-08T19:24:50.619498+0000","last_active":"2023-11-08T19:24:52.309602+0000","last_peered":"2023-11-08T19:24:52.309602+0000","last_clean":"2023-11-08T19:24:52.309602+0000","last_became_active":"2023-11-08T19:24:50.619231+0000","last_became_peered":"2023-11-08T19:24:50.619231+0000","last_unstale":"2023-11-08T19:24:52.309602+0000","last_undegraded":"2023-11-08T19:24:52.309602+0000","last_fullsized":"2023-11-08T19:24:52.309602+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:38:25.250414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.352455+0000","last_change":"2023-11-08T19:24:49.612336+0000","last_active":"2023-11-08T19:24:52.352455+0000","last_peered":"2023-11-08T19:24:52.352455+0000","last_clean":"2023-11-08T19:24:52.352455+0000","last_became_active":"2023-11-08T19:24:49.611885+0000","last_became_peered":"2023-11-08T19:24:49.611885+0000","last_unstale":"2023-11-08T19:24:52.352455+0000","last_undegraded":"2023-11-08T19:24:52.352455+0000","last_fullsized":"2023-11-08T19:24:52.352455+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:46:18.372121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:24:49.588741+0000","last_change":"2023-11-08T19:24:49.588741+0000","last_active":"2023-11-08T19:24:49.588741+0000","last_peered":"2023-11-08T19:24:49.588741+0000","last_clean":"2023-11-08T19:24:49.588741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:24:49.588741+0000","last_undegraded":"2023-11-08T19:24:49.588741+0000","last_fullsized":"2023-11-08T19:24:49.588741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-08T19:24:48.591465+0000","last_change":"2023-11-08T19:24:48.591465+0000","last_active":"2023-11-08T19:24:48.584965+0000","last_peered":"2023-11-08T19:24:48.584965+0000","last_clean":"2023-11-08T19:24:48.584965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:24:48.591465+0000","last_undegraded":"2023-11-08T19:24:48.591465+0000","last_fullsized":"2023-11-08T19:24:48.591465+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:24:49.588741+0000","last_change":"2023-11-08T19:24:49.588741+0000","last_active":"2023-11-08T19:24:49.588741+0000","last_peered":"2023-11-08T19:24:49.588741+0000","last_clean":"2023-11-08T19:24:49.588741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:24:49.588741+0000","last_undegraded":"2023-11-08T19:24:49.588741+0000","last_fullsized":"2023-11-08T19:24:49.588741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.804637+0000","last_change":"2023-11-08T19:24:49.608503+0000","last_active":"2023-11-08T19:24:52.804637+0000","last_peered":"2023-11-08T19:24:52.804637+0000","last_clean":"2023-11-08T19:24:52.804637+0000","last_became_active":"2023-11-08T19:24:49.608176+0000","last_became_peered":"2023-11-08T19:24:49.608176+0000","last_unstale":"2023-11-08T19:24:52.804637+0000","last_undegraded":"2023-11-08T19:24:52.804637+0000","last_fullsized":"2023-11-08T19:24:52.804637+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:17:50.005667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:24:49.588741+0000","last_change":"2023-11-08T19:24:49.588741+0000","last_active":"2023-11-08T19:24:49.588741+0000","last_peered":"2023-11-08T19:24:49.588741+0000","last_clean":"2023-11-08T19:24:49.588741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:24:49.588741+0000","last_undegraded":"2023-11-08T19:24:49.588741+0000","last_fullsized":"2023-11-08T19:24:49.588741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.805855+0000","last_change":"2023-11-08T19:24:49.610638+0000","last_active":"2023-11-08T19:24:52.805855+0000","last_peered":"2023-11-08T19:24:52.805855+0000","last_clean":"2023-11-08T19:24:52.805855+0000","last_became_active":"2023-11-08T19:24:49.610362+0000","last_became_peered":"2023-11-08T19:24:49.610362+0000","last_unstale":"2023-11-08T19:24:52.805855+0000","last_undegraded":"2023-11-08T19:24:52.805855+0000","last_fullsized":"2023-11-08T19:24:52.805855+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:15:32.030617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:24:49.588741+0000","last_change":"2023-11-08T19:24:49.588741+0000","last_active":"2023-11-08T19:24:49.588741+0000","last_peered":"2023-11-08T19:24:49.588741+0000","last_clean":"2023-11-08T19:24:49.588741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:24:49.588741+0000","last_undegraded":"2023-11-08T19:24:49.588741+0000","last_fullsized":"2023-11-08T19:24:49.588741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.805154+0000","last_change":"2023-11-08T19:24:49.613413+0000","last_active":"2023-11-08T19:24:52.805154+0000","last_peered":"2023-11-08T19:24:52.805154+0000","last_clean":"2023-11-08T19:24:52.805154+0000","last_became_active":"2023-11-08T19:24:49.613131+0000","last_became_peered":"2023-11-08T19:24:49.613131+0000","last_unstale":"2023-11-08T19:24:52.805154+0000","last_undegraded":"2023-11-08T19:24:52.805154+0000","last_fullsized":"2023-11-08T19:24:52.805154+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:46:12.493411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.352374+0000","last_change":"2023-11-08T19:24:50.640031+0000","last_active":"2023-11-08T19:24:52.352374+0000","last_peered":"2023-11-08T19:24:52.352374+0000","last_clean":"2023-11-08T19:24:52.352374+0000","last_became_active":"2023-11-08T19:24:50.639718+0000","last_became_peered":"2023-11-08T19:24:50.639718+0000","last_unstale":"2023-11-08T19:24:52.352374+0000","last_undegraded":"2023-11-08T19:24:52.352374+0000","last_fullsized":"2023-11-08T19:24:52.352374+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:08:17.096077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.352346+0000","last_change":"2023-11-08T19:24:49.612137+0000","last_active":"2023-11-08T19:24:52.352346+0000","last_peered":"2023-11-08T19:24:52.352346+0000","last_clean":"2023-11-08T19:24:52.352346+0000","last_became_active":"2023-11-08T19:24:49.611532+0000","last_became_peered":"2023-11-08T19:24:49.611532+0000","last_unstale":"2023-11-08T19:24:52.352346+0000","last_undegraded":"2023-11-08T19:24:52.352346+0000","last_fullsized":"2023-11-08T19:24:52.352346+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:40:10.975206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.805536+0000","last_change":"2023-11-08T19:24:50.630854+0000","last_active":"2023-11-08T19:24:52.805536+0000","last_peered":"2023-11-08T19:24:52.805536+0000","last_clean":"2023-11-08T19:24:52.805536+0000","last_became_active":"2023-11-08T19:24:50.630549+0000","last_became_peered":"2023-11-08T19:24:50.630549+0000","last_unstale":"2023-11-08T19:24:52.805536+0000","last_undegraded":"2023-11-08T19:24:52.805536+0000","last_fullsized":"2023-11-08T19:24:52.805536+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:48:26.550025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.804950+0000","last_change":"2023-11-08T19:24:50.620897+0000","last_active":"2023-11-08T19:24:52.804950+0000","last_peered":"2023-11-08T19:24:52.804950+0000","last_clean":"2023-11-08T19:24:52.804950+0000","last_became_active":"2023-11-08T19:24:50.620606+0000","last_became_peered":"2023-11-08T19:24:50.620606+0000","last_unstale":"2023-11-08T19:24:52.804950+0000","last_undegraded":"2023-11-08T19:24:52.804950+0000","last_fullsized":"2023-11-08T19:24:52.804950+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:30:36.041820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.804625+0000","last_change":"2023-11-08T19:24:49.612840+0000","last_active":"2023-11-08T19:24:52.804625+0000","last_peered":"2023-11-08T19:24:52.804625+0000","last_clean":"2023-11-08T19:24:52.804625+0000","last_became_active":"2023-11-08T19:24:49.612652+0000","last_became_peered":"2023-11-08T19:24:49.612652+0000","last_unstale":"2023-11-08T19:24:52.804625+0000","last_undegraded":"2023-11-08T19:24:52.804625+0000","last_fullsized":"2023-11-08T19:24:52.804625+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:48:25.546248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.626593+0000","last_change":"2023-11-08T19:24:50.642515+0000","last_active":"2023-11-08T19:24:51.626593+0000","last_peered":"2023-11-08T19:24:51.626593+0000","last_clean":"2023-11-08T19:24:51.626593+0000","last_became_active":"2023-11-08T19:24:50.642204+0000","last_became_peered":"2023-11-08T19:24:50.642204+0000","last_unstale":"2023-11-08T19:24:51.626593+0000","last_undegraded":"2023-11-08T19:24:51.626593+0000","last_fullsized":"2023-11-08T19:24:51.626593+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:42:42.512786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.351949+0000","last_change":"2023-11-08T19:24:49.605594+0000","last_active":"2023-11-08T19:24:52.351949+0000","last_peered":"2023-11-08T19:24:52.351949+0000","last_clean":"2023-11-08T19:24:52.351949+0000","last_became_active":"2023-11-08T19:24:49.605208+0000","last_became_peered":"2023-11-08T19:24:49.605208+0000","last_unstale":"2023-11-08T19:24:52.351949+0000","last_undegraded":"2023-11-08T19:24:52.351949+0000","last_fullsized":"2023-11-08T19:24:52.351949+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:17:50.005667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":162,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-11-08T19:24:48.592313+0000","last_change":"2023-11-08T19:24:45.888122+0000","last_active":"2023-11-08T19:24:48.592313+0000","last_peered":"2023-11-08T19:24:48.592313+0000","last_clean":"2023-11-08T19:24:48.592313+0000","last_became_active":"2023-11-08T19:24:45.887048+0000","last_became_peered":"2023-11-08T19:24:45.887048+0000","last_unstale":"2023-11-08T19:24:48.592313+0000","last_undegraded":"2023-11-08T19:24:48.592313+0000","last_fullsized":"2023-11-08T19:24:48.592313+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:44.575133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:44.575133+0000","last_clean_scrub_stamp":"2023-11-08T19:24:44.575133+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:19:20.202062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.309622+0000","last_change":"2023-11-08T19:24:50.628353+0000","last_active":"2023-11-08T19:24:52.309622+0000","last_peered":"2023-11-08T19:24:52.309622+0000","last_clean":"2023-11-08T19:24:52.309622+0000","last_became_active":"2023-11-08T19:24:50.628008+0000","last_became_peered":"2023-11-08T19:24:50.628008+0000","last_unstale":"2023-11-08T19:24:52.309622+0000","last_undegraded":"2023-11-08T19:24:52.309622+0000","last_fullsized":"2023-11-08T19:24:52.309622+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:46:13.497187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-08T19:24:48.591491+0000","last_change":"2023-11-08T19:24:48.591491+0000","last_active":"2023-11-08T19:24:48.584965+0000","last_peered":"2023-11-08T19:24:48.584965+0000","last_clean":"2023-11-08T19:24:48.584965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:24:48.591491+0000","last_undegraded":"2023-11-08T19:24:48.591491+0000","last_fullsized":"2023-11-08T19:24:48.591491+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-08T19:24:48.591465+0000","last_change":"2023-11-08T19:24:48.591465+0000","last_active":"2023-11-08T19:24:48.584965+0000","last_peered":"2023-11-08T19:24:48.584965+0000","last_clean":"2023-11-08T19:24:48.584965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:24:48.591465+0000","last_undegraded":"2023-11-08T19:24:48.591465+0000","last_fullsized":"2023-11-08T19:24:48.591465+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.805231+0000","last_change":"2023-11-08T19:24:50.624279+0000","last_active":"2023-11-08T19:24:52.805231+0000","last_peered":"2023-11-08T19:24:52.805231+0000","last_clean":"2023-11-08T19:24:52.805231+0000","last_became_active":"2023-11-08T19:24:50.623920+0000","last_became_peered":"2023-11-08T19:24:50.623920+0000","last_unstale":"2023-11-08T19:24:52.805231+0000","last_undegraded":"2023-11-08T19:24:52.805231+0000","last_fullsized":"2023-11-08T19:24:52.805231+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:42:42.512786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.805402+0000","last_change":"2023-11-08T19:24:49.606910+0000","last_active":"2023-11-08T19:24:52.805402+0000","last_peered":"2023-11-08T19:24:52.805402+0000","last_clean":"2023-11-08T19:24:52.805402+0000","last_became_active":"2023-11-08T19:24:49.606627+0000","last_became_peered":"2023-11-08T19:24:49.606627+0000","last_unstale":"2023-11-08T19:24:52.805402+0000","last_undegraded":"2023-11-08T19:24:52.805402+0000","last_fullsized":"2023-11-08T19:24:52.805402+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:56:00.148860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.627087+0000","last_change":"2023-11-08T19:24:50.622398+0000","last_active":"2023-11-08T19:24:51.627087+0000","last_peered":"2023-11-08T19:24:51.627087+0000","last_clean":"2023-11-08T19:24:51.627087+0000","last_became_active":"2023-11-08T19:24:50.622092+0000","last_became_peered":"2023-11-08T19:24:50.622092+0000","last_unstale":"2023-11-08T19:24:51.627087+0000","last_undegraded":"2023-11-08T19:24:51.627087+0000","last_fullsized":"2023-11-08T19:24:51.627087+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:42:27.150066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-08T19:24:48.592846+0000","last_change":"2023-11-08T19:24:48.592846+0000","last_active":"2023-11-08T19:24:48.584965+0000","last_peered":"2023-11-08T19:24:48.584965+0000","last_clean":"2023-11-08T19:24:48.584965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:24:48.592846+0000","last_undegraded":"2023-11-08T19:24:48.592846+0000","last_fullsized":"2023-11-08T19:24:48.592846+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.309770+0000","last_change":"2023-11-08T19:24:50.624848+0000","last_active":"2023-11-08T19:24:52.309770+0000","last_peered":"2023-11-08T19:24:52.309770+0000","last_clean":"2023-11-08T19:24:52.309770+0000","last_became_active":"2023-11-08T19:24:50.624518+0000","last_became_peered":"2023-11-08T19:24:50.624518+0000","last_unstale":"2023-11-08T19:24:52.309770+0000","last_undegraded":"2023-11-08T19:24:52.309770+0000","last_fullsized":"2023-11-08T19:24:52.309770+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:15:33.285449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-08T19:24:48.592059+0000","last_change":"2023-11-08T19:24:48.592059+0000","last_active":"2023-11-08T19:24:48.584965+0000","last_peered":"2023-11-08T19:24:48.584965+0000","last_clean":"2023-11-08T19:24:48.584965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:24:48.592059+0000","last_undegraded":"2023-11-08T19:24:48.592059+0000","last_fullsized":"2023-11-08T19:24:48.592059+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.805558+0000","last_change":"2023-11-08T19:24:50.627488+0000","last_active":"2023-11-08T19:24:52.805558+0000","last_peered":"2023-11-08T19:24:52.805558+0000","last_clean":"2023-11-08T19:24:52.805558+0000","last_became_active":"2023-11-08T19:24:50.626571+0000","last_became_peered":"2023-11-08T19:24:50.626571+0000","last_unstale":"2023-11-08T19:24:52.805558+0000","last_undegraded":"2023-11-08T19:24:52.805558+0000","last_fullsized":"2023-11-08T19:24:52.805558+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:43:00.849357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.804911+0000","last_change":"2023-11-08T19:24:49.606579+0000","last_active":"2023-11-08T19:24:52.804911+0000","last_peered":"2023-11-08T19:24:52.804911+0000","last_clean":"2023-11-08T19:24:52.804911+0000","last_became_active":"2023-11-08T19:24:49.606283+0000","last_became_peered":"2023-11-08T19:24:49.606283+0000","last_unstale":"2023-11-08T19:24:52.804911+0000","last_undegraded":"2023-11-08T19:24:52.804911+0000","last_fullsized":"2023-11-08T19:24:52.804911+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:14:00.455497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.626421+0000","last_change":"2023-11-08T19:24:50.625692+0000","last_active":"2023-11-08T19:24:51.626421+0000","last_peered":"2023-11-08T19:24:51.626421+0000","last_clean":"2023-11-08T19:24:51.626421+0000","last_became_active":"2023-11-08T19:24:50.625014+0000","last_became_peered":"2023-11-08T19:24:50.625014+0000","last_unstale":"2023-11-08T19:24:51.626421+0000","last_undegraded":"2023-11-08T19:24:51.626421+0000","last_fullsized":"2023-11-08T19:24:51.626421+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:15:33.285449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.805081+0000","last_change":"2023-11-08T19:24:49.611023+0000","last_active":"2023-11-08T19:24:52.805081+0000","last_peered":"2023-11-08T19:24:52.805081+0000","last_clean":"2023-11-08T19:24:52.805081+0000","last_became_active":"2023-11-08T19:24:49.610735+0000","last_became_peered":"2023-11-08T19:24:49.610735+0000","last_unstale":"2023-11-08T19:24:52.805081+0000","last_undegraded":"2023-11-08T19:24:52.805081+0000","last_fullsized":"2023-11-08T19:24:52.805081+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:30:31.599200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.626730+0000","last_change":"2023-11-08T19:24:50.631491+0000","last_active":"2023-11-08T19:24:51.626730+0000","last_peered":"2023-11-08T19:24:51.626730+0000","last_clean":"2023-11-08T19:24:51.626730+0000","last_became_active":"2023-11-08T19:24:50.630872+0000","last_became_peered":"2023-11-08T19:24:50.630872+0000","last_unstale":"2023-11-08T19:24:51.626730+0000","last_undegraded":"2023-11-08T19:24:51.626730+0000","last_fullsized":"2023-11-08T19:24:51.626730+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:36:53.967796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-08T19:24:48.592103+0000","last_change":"2023-11-08T19:24:48.592103+0000","last_active":"2023-11-08T19:24:48.584965+0000","last_peered":"2023-11-08T19:24:48.584965+0000","last_clean":"2023-11-08T19:24:48.584965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:24:48.592103+0000","last_undegraded":"2023-11-08T19:24:48.592103+0000","last_fullsized":"2023-11-08T19:24:48.592103+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:24:49.588741+0000","last_change":"2023-11-08T19:24:49.588741+0000","last_active":"2023-11-08T19:24:49.588741+0000","last_peered":"2023-11-08T19:24:49.588741+0000","last_clean":"2023-11-08T19:24:49.588741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:24:49.588741+0000","last_undegraded":"2023-11-08T19:24:49.588741+0000","last_fullsized":"2023-11-08T19:24:49.588741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.805091+0000","last_change":"2023-11-08T19:24:49.613365+0000","last_active":"2023-11-08T19:24:52.805091+0000","last_peered":"2023-11-08T19:24:52.805091+0000","last_clean":"2023-11-08T19:24:52.805091+0000","last_became_active":"2023-11-08T19:24:49.613052+0000","last_became_peered":"2023-11-08T19:24:49.613052+0000","last_unstale":"2023-11-08T19:24:52.805091+0000","last_undegraded":"2023-11-08T19:24:52.805091+0000","last_fullsized":"2023-11-08T19:24:52.805091+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:41:24.551269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:24:49.588741+0000","last_change":"2023-11-08T19:24:49.588741+0000","last_active":"2023-11-08T19:24:49.588741+0000","last_peered":"2023-11-08T19:24:49.588741+0000","last_clean":"2023-11-08T19:24:49.588741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:24:49.588741+0000","last_undegraded":"2023-11-08T19:24:49.588741+0000","last_fullsized":"2023-11-08T19:24:49.588741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.805219+0000","last_change":"2023-11-08T19:24:49.606271+0000","last_active":"2023-11-08T19:24:52.805219+0000","last_peered":"2023-11-08T19:24:52.805219+0000","last_clean":"2023-11-08T19:24:52.805219+0000","last_became_active":"2023-11-08T19:24:49.605802+0000","last_became_peered":"2023-11-08T19:24:49.605802+0000","last_unstale":"2023-11-08T19:24:52.805219+0000","last_undegraded":"2023-11-08T19:24:52.805219+0000","last_fullsized":"2023-11-08T19:24:52.805219+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:07:00.575561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.352338+0000","last_change":"2023-11-08T19:24:50.640594+0000","last_active":"2023-11-08T19:24:52.352338+0000","last_peered":"2023-11-08T19:24:52.352338+0000","last_clean":"2023-11-08T19:24:52.352338+0000","last_became_active":"2023-11-08T19:24:50.640295+0000","last_became_peered":"2023-11-08T19:24:50.640295+0000","last_unstale":"2023-11-08T19:24:52.352338+0000","last_undegraded":"2023-11-08T19:24:52.352338+0000","last_fullsized":"2023-11-08T19:24:52.352338+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:34:42.075394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.805026+0000","last_change":"2023-11-08T19:24:49.610782+0000","last_active":"2023-11-08T19:24:52.805026+0000","last_peered":"2023-11-08T19:24:52.805026+0000","last_clean":"2023-11-08T19:24:52.805026+0000","last_became_active":"2023-11-08T19:24:49.610493+0000","last_became_peered":"2023-11-08T19:24:49.610493+0000","last_unstale":"2023-11-08T19:24:52.805026+0000","last_undegraded":"2023-11-08T19:24:52.805026+0000","last_fullsized":"2023-11-08T19:24:52.805026+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:42:59.845581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:24:49.588741+0000","last_change":"2023-11-08T19:24:49.588741+0000","last_active":"2023-11-08T19:24:49.588741+0000","last_peered":"2023-11-08T19:24:49.588741+0000","last_clean":"2023-11-08T19:24:49.588741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:24:49.588741+0000","last_undegraded":"2023-11-08T19:24:49.588741+0000","last_fullsized":"2023-11-08T19:24:49.588741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:24:49.588741+0000","last_change":"2023-11-08T19:24:49.588741+0000","last_active":"2023-11-08T19:24:49.588741+0000","last_peered":"2023-11-08T19:24:49.588741+0000","last_clean":"2023-11-08T19:24:49.588741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:24:49.588741+0000","last_undegraded":"2023-11-08T19:24:49.588741+0000","last_fullsized":"2023-11-08T19:24:49.588741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.804775+0000","last_change":"2023-11-08T19:24:50.625948+0000","last_active":"2023-11-08T19:24:52.804775+0000","last_peered":"2023-11-08T19:24:52.804775+0000","last_clean":"2023-11-08T19:24:52.804775+0000","last_became_active":"2023-11-08T19:24:50.625633+0000","last_became_peered":"2023-11-08T19:24:50.625633+0000","last_unstale":"2023-11-08T19:24:52.804775+0000","last_undegraded":"2023-11-08T19:24:52.804775+0000","last_fullsized":"2023-11-08T19:24:52.804775+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:00:13.071654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.352423+0000","last_change":"2023-11-08T19:24:50.640919+0000","last_active":"2023-11-08T19:24:52.352423+0000","last_peered":"2023-11-08T19:24:52.352423+0000","last_clean":"2023-11-08T19:24:52.352423+0000","last_became_active":"2023-11-08T19:24:50.640496+0000","last_became_peered":"2023-11-08T19:24:50.640496+0000","last_unstale":"2023-11-08T19:24:52.352423+0000","last_undegraded":"2023-11-08T19:24:52.352423+0000","last_fullsized":"2023-11-08T19:24:52.352423+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:59:37.592538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.309782+0000","last_change":"2023-11-08T19:24:50.625916+0000","last_active":"2023-11-08T19:24:52.309782+0000","last_peered":"2023-11-08T19:24:52.309782+0000","last_clean":"2023-11-08T19:24:52.309782+0000","last_became_active":"2023-11-08T19:24:50.625625+0000","last_became_peered":"2023-11-08T19:24:50.625625+0000","last_unstale":"2023-11-08T19:24:52.309782+0000","last_undegraded":"2023-11-08T19:24:52.309782+0000","last_fullsized":"2023-11-08T19:24:52.309782+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:40:11.978983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.626555+0000","last_change":"2023-11-08T19:24:50.628686+0000","last_active":"2023-11-08T19:24:51.626555+0000","last_peered":"2023-11-08T19:24:51.626555+0000","last_clean":"2023-11-08T19:24:51.626555+0000","last_became_active":"2023-11-08T19:24:50.628289+0000","last_became_peered":"2023-11-08T19:24:50.628289+0000","last_unstale":"2023-11-08T19:24:51.626555+0000","last_undegraded":"2023-11-08T19:24:51.626555+0000","last_fullsized":"2023-11-08T19:24:51.626555+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:46:19.375897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.805231+0000","last_change":"2023-11-08T19:24:50.624352+0000","last_active":"2023-11-08T19:24:52.805231+0000","last_peered":"2023-11-08T19:24:52.805231+0000","last_clean":"2023-11-08T19:24:52.805231+0000","last_became_active":"2023-11-08T19:24:50.624035+0000","last_became_peered":"2023-11-08T19:24:50.624035+0000","last_unstale":"2023-11-08T19:24:52.805231+0000","last_undegraded":"2023-11-08T19:24:52.805231+0000","last_fullsized":"2023-11-08T19:24:52.805231+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:46:19.375897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:24:49.588741+0000","last_change":"2023-11-08T19:24:49.588741+0000","last_active":"2023-11-08T19:24:49.588741+0000","last_peered":"2023-11-08T19:24:49.588741+0000","last_clean":"2023-11-08T19:24:49.588741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:24:49.588741+0000","last_undegraded":"2023-11-08T19:24:49.588741+0000","last_fullsized":"2023-11-08T19:24:49.588741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.626991+0000","last_change":"2023-11-08T19:24:50.628009+0000","last_active":"2023-11-08T19:24:51.626991+0000","last_peered":"2023-11-08T19:24:51.626991+0000","last_clean":"2023-11-08T19:24:51.626991+0000","last_became_active":"2023-11-08T19:24:50.627668+0000","last_became_peered":"2023-11-08T19:24:50.627668+0000","last_unstale":"2023-11-08T19:24:51.626991+0000","last_undegraded":"2023-11-08T19:24:51.626991+0000","last_fullsized":"2023-11-08T19:24:51.626991+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:49:19.100370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.352100+0000","last_change":"2023-11-08T19:24:50.641255+0000","last_active":"2023-11-08T19:24:52.352100+0000","last_peered":"2023-11-08T19:24:52.352100+0000","last_clean":"2023-11-08T19:24:52.352100+0000","last_became_active":"2023-11-08T19:24:50.640164+0000","last_became_peered":"2023-11-08T19:24:50.640164+0000","last_unstale":"2023-11-08T19:24:52.352100+0000","last_undegraded":"2023-11-08T19:24:52.352100+0000","last_fullsized":"2023-11-08T19:24:52.352100+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:12:54.641952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.626435+0000","last_change":"2023-11-08T19:24:50.629006+0000","last_active":"2023-11-08T19:24:51.626435+0000","last_peered":"2023-11-08T19:24:51.626435+0000","last_clean":"2023-11-08T19:24:51.626435+0000","last_became_active":"2023-11-08T19:24:50.628685+0000","last_became_peered":"2023-11-08T19:24:50.628685+0000","last_unstale":"2023-11-08T19:24:51.626435+0000","last_undegraded":"2023-11-08T19:24:51.626435+0000","last_fullsized":"2023-11-08T19:24:51.626435+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:30:36.041820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.352270+0000","last_change":"2023-11-08T19:24:50.640737+0000","last_active":"2023-11-08T19:24:52.352270+0000","last_peered":"2023-11-08T19:24:52.352270+0000","last_clean":"2023-11-08T19:24:52.352270+0000","last_became_active":"2023-11-08T19:24:50.640255+0000","last_became_peered":"2023-11-08T19:24:50.640255+0000","last_unstale":"2023-11-08T19:24:52.352270+0000","last_undegraded":"2023-11-08T19:24:52.352270+0000","last_fullsized":"2023-11-08T19:24:52.352270+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:13:41.469149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.309753+0000","last_change":"2023-11-08T19:24:50.620645+0000","last_active":"2023-11-08T19:24:52.309753+0000","last_peered":"2023-11-08T19:24:52.309753+0000","last_clean":"2023-11-08T19:24:52.309753+0000","last_became_active":"2023-11-08T19:24:50.620349+0000","last_became_peered":"2023-11-08T19:24:50.620349+0000","last_unstale":"2023-11-08T19:24:52.309753+0000","last_undegraded":"2023-11-08T19:24:52.309753+0000","last_fullsized":"2023-11-08T19:24:52.309753+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:30:32.602976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:24:49.588741+0000","last_change":"2023-11-08T19:24:49.588741+0000","last_active":"2023-11-08T19:24:49.588741+0000","last_peered":"2023-11-08T19:24:49.588741+0000","last_clean":"2023-11-08T19:24:49.588741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:24:49.588741+0000","last_undegraded":"2023-11-08T19:24:49.588741+0000","last_fullsized":"2023-11-08T19:24:49.588741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":92,"acting":92,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":798,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":2626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":23,"ondisk_log_size":23,"up":64,"acting":64,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27760,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344080,"statfs":{"total":96636764160,"available":96608337920,"internally_reserved":0,"allocated":761856,"data_stored":641928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47099999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52000000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.432}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56200000000000006}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.372}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67900000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57599999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":167936,"data_stored":51560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47599999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.495}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51900000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56200000000000006}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39400000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":71070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36299999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23499999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26500000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45200000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47699999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":811008,"data_stored":661916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35099999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27400000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40000000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73199999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71099999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68300000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81000000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":192512,"data_stored":64610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46000000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.443}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82899999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81599999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84799999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.878}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.308}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27000000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.379}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59399999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50900000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54100000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70099999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":71036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46000000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55800000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65400000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75800000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67200000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.746}]}]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":208896,"data_stored":71000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39800000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55300000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61099999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.439}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66700000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69499999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-08T19:24:55.108 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-08T19:24:55.264 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.262+0000 7f9897a5d700 1 Processor -- start 2023-11-08T19:24:55.264 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.262+0000 7f9897a5d700 1 -- start start 2023-11-08T19:24:55.266 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.263+0000 7f9897a5d700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f98900cf090 0x7f98900d1f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:55.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.263+0000 7f9897a5d700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f98900d3690 0x7f98900b9690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:55.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.263+0000 7f9897a5d700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98900d3200 con 0x7f989015dc80 2023-11-08T19:24:55.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.263+0000 7f9897a5d700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98900b9bd0 con 0x7f98900cf090 2023-11-08T19:24:55.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.263+0000 7f9897a5d700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98900b7cf0 con 0x7f98900d3690 2023-11-08T19:24:55.268 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.263+0000 7f9894ff8700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f98900cf090 0x7f98900d1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:55.268 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.263+0000 7f9894ff8700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f98900cf090 0x7f98900d1f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48686/0 (socket says 172.21.15.143:48686) 2023-11-08T19:24:55.268 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.263+0000 7f9894ff8700 1 -- 172.21.15.143:0/2165472855 learned_addr learned my addr 172.21.15.143:0/2165472855 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:24:55.268 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.264+0000 7f9895ffa700 1 --2- 172.21.15.143:0/2165472855 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f98900d3690 0x7f98900b9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:55.268 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.264+0000 7f9894ff8700 1 -- 172.21.15.143:0/2165472855 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f98900d3690 msgr2=0x7f98900b9690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:24:55.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.264+0000 7f9894ff8700 1 --2- 172.21.15.143:0/2165472855 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f98900d3690 0x7f98900b9690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:55.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.264+0000 7f9894ff8700 1 -- 172.21.15.143:0/2165472855 >> v1:172.21.15.174:6789/0 conn(0x7f989015dc80 legacy=0x7f98900ce980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:55.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.264+0000 7f9894ff8700 1 -- 172.21.15.143:0/2165472855 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f989015e060 con 0x7f98900cf090 2023-11-08T19:24:55.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.264+0000 7f9894ff8700 1 --2- 172.21.15.143:0/2165472855 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98900cf090 0x7f98900d1f80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f988c0098f0 tx=0x7f988c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1ca2b02b875695e server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:55.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.264+0000 7f9887fff700 1 -- 172.21.15.143:0/2165472855 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f988c00b8a0 con 0x7f98900cf090 2023-11-08T19:24:55.270 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.264+0000 7f9887fff700 1 -- 172.21.15.143:0/2165472855 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f988c00ba00 con 0x7f98900cf090 2023-11-08T19:24:55.270 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.264+0000 7f9887fff700 1 -- 172.21.15.143:0/2165472855 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f988c00bd90 con 0x7f98900cf090 2023-11-08T19:24:55.270 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.265+0000 7f9897a5d700 1 -- 172.21.15.143:0/2165472855 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98900cf090 msgr2=0x7f98900d1f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:55.270 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.265+0000 7f9897a5d700 1 --2- 172.21.15.143:0/2165472855 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98900cf090 0x7f98900d1f80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f988c0098f0 tx=0x7f988c009c00 comp rx=0 tx=0).stop 2023-11-08T19:24:55.270 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.265+0000 7f9897a5d700 1 -- 172.21.15.143:0/2165472855 shutdown_connections 2023-11-08T19:24:55.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.265+0000 7f9897a5d700 1 --2- 172.21.15.143:0/2165472855 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f98900d3690 0x7f98900b9690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:55.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.265+0000 7f9897a5d700 1 --2- 172.21.15.143:0/2165472855 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98900cf090 0x7f98900d1f80 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:55.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.265+0000 7f9897a5d700 1 -- 172.21.15.143:0/2165472855 >> 172.21.15.143:0/2165472855 conn(0x7f98900668e0 msgr2=0x7f9890066cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:24:55.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.265+0000 7f9897a5d700 1 -- 172.21.15.143:0/2165472855 shutdown_connections 2023-11-08T19:24:55.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.265+0000 7f9897a5d700 1 -- 172.21.15.143:0/2165472855 wait complete. 2023-11-08T19:24:55.274 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.266+0000 7f9897a5d700 1 Processor -- start 2023-11-08T19:24:55.274 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.266+0000 7f9897a5d700 1 -- start start 2023-11-08T19:24:55.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.267+0000 7f9897a5d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98900cf090 0x7f9890160b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:55.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.267+0000 7f9897a5d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f98900d3690 0x7f98901f32a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:55.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.267+0000 7f9897a5d700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f989015dc80 0x7f989015ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:55.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.267+0000 7f9897a5d700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98901f6f80 con 0x7f98900cf090 2023-11-08T19:24:55.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.267+0000 7f9897a5d700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9890161060 con 0x7f98900d3690 2023-11-08T19:24:55.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.267+0000 7f9897a5d700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98901f71d0 con 0x7f989015dc80 2023-11-08T19:24:55.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.267+0000 7f98957f9700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98900cf090 0x7f9890160b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:55.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.267+0000 7f98957f9700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98900cf090 0x7f9890160b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48690/0 (socket says 172.21.15.143:48690) 2023-11-08T19:24:55.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.267+0000 7f98957f9700 1 -- 172.21.15.143:0/4284691432 learned_addr learned my addr 172.21.15.143:0/4284691432 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:24:55.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.267+0000 7f9894ff8700 1 --2- 172.21.15.143:0/4284691432 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f98900d3690 0x7f98901f32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:55.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.267+0000 7f9895ffa700 1 --2- 172.21.15.143:0/4284691432 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f989015dc80 0x7f989015ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:55.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.267+0000 7f98957f9700 1 -- 172.21.15.143:0/4284691432 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f989015dc80 msgr2=0x7f989015ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:55.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.267+0000 7f98957f9700 1 --2- 172.21.15.143:0/4284691432 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f989015dc80 0x7f989015ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:55.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.267+0000 7f98957f9700 1 -- 172.21.15.143:0/4284691432 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f98900d3690 msgr2=0x7f98901f32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:55.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.267+0000 7f98957f9700 1 --2- 172.21.15.143:0/4284691432 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f98900d3690 0x7f98901f32a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:55.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.267+0000 7f98957f9700 1 -- 172.21.15.143:0/4284691432 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f988c009550 con 0x7f98900cf090 2023-11-08T19:24:55.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.267+0000 7f98957f9700 1 --2- 172.21.15.143:0/4284691432 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98900cf090 0x7f9890160b20 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f988000c450 tx=0x7f988000c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:55.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.267+0000 7f9885ffb700 1 -- 172.21.15.143:0/4284691432 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98800052d0 con 0x7f98900cf090 2023-11-08T19:24:55.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.267+0000 7f9885ffb700 1 -- 172.21.15.143:0/4284691432 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9880005430 con 0x7f98900cf090 2023-11-08T19:24:55.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.267+0000 7f9897a5d700 1 -- 172.21.15.143:0/4284691432 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98901f74b0 con 0x7f98900cf090 2023-11-08T19:24:55.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.267+0000 7f9885ffb700 1 -- 172.21.15.143:0/4284691432 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98800038c0 con 0x7f98900cf090 2023-11-08T19:24:55.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.267+0000 7f9897a5d700 1 -- 172.21.15.143:0/4284691432 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98901f7a70 con 0x7f98900cf090 2023-11-08T19:24:55.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.269+0000 7f9897a5d700 1 -- 172.21.15.143:0/4284691432 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f989004eee0 con 0x7f98900cf090 2023-11-08T19:24:55.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.269+0000 7f9885ffb700 1 -- 172.21.15.143:0/4284691432 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9880003a20 con 0x7f98900cf090 2023-11-08T19:24:55.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.269+0000 7f9885ffb700 1 --2- 172.21.15.143:0/4284691432 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f987406af00 0x7f987406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:55.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.269+0000 7f9885ffb700 1 -- 172.21.15.143:0/4284691432 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f98800888f0 con 0x7f98900cf090 2023-11-08T19:24:55.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.269+0000 7f9894ff8700 1 --2- 172.21.15.143:0/4284691432 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f987406af00 0x7f987406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:55.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.270+0000 7f9894ff8700 1 --2- 172.21.15.143:0/4284691432 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f987406af00 0x7f987406d3c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f988c009c40 tx=0x7f988c008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:55.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.273+0000 7f9885ffb700 1 -- 172.21.15.143:0/4284691432 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9880053a30 con 0x7f98900cf090 2023-11-08T19:24:55.458 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.456+0000 7f9897a5d700 1 -- 172.21.15.143:0/4284691432 --> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f98900d28d0 con 0x7f987406af00 2023-11-08T19:24:55.465 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.464+0000 7f9885ffb700 1 -- 172.21.15.143:0/4284691432 <== mgr.4099 v2:172.21.15.174:6832/4037280178 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+261184 (secure 0 0 0) 0x7f988c00bb70 con 0x7f987406af00 2023-11-08T19:24:55.468 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:24:55.470 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.467+0000 7f9897a5d700 1 -- 172.21.15.143:0/4284691432 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f987406af00 msgr2=0x7f987406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:55.471 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.467+0000 7f9897a5d700 1 --2- 172.21.15.143:0/4284691432 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f987406af00 0x7f987406d3c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f988c009c40 tx=0x7f988c008030 comp rx=0 tx=0).stop 2023-11-08T19:24:55.471 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.467+0000 7f9897a5d700 1 -- 172.21.15.143:0/4284691432 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98900cf090 msgr2=0x7f9890160b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:55.471 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.467+0000 7f9897a5d700 1 --2- 172.21.15.143:0/4284691432 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98900cf090 0x7f9890160b20 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f988000c450 tx=0x7f988000c760 comp rx=0 tx=0).stop 2023-11-08T19:24:55.471 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.467+0000 7f9897a5d700 1 -- 172.21.15.143:0/4284691432 shutdown_connections 2023-11-08T19:24:55.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.467+0000 7f9897a5d700 1 --2- 172.21.15.143:0/4284691432 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f987406af00 0x7f987406d3c0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:55.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.467+0000 7f9897a5d700 1 --2- 172.21.15.143:0/4284691432 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f989015dc80 0x7f989015ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:55.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.467+0000 7f9897a5d700 1 --2- 172.21.15.143:0/4284691432 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98900cf090 0x7f9890160b20 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:55.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.467+0000 7f9897a5d700 1 --2- 172.21.15.143:0/4284691432 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f98900d3690 0x7f98901f32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:55.473 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.467+0000 7f9897a5d700 1 -- 172.21.15.143:0/4284691432 >> 172.21.15.143:0/4284691432 conn(0x7f98900668e0 msgr2=0x7f989015bf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:24:55.473 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.467+0000 7f9897a5d700 1 -- 172.21.15.143:0/4284691432 shutdown_connections 2023-11-08T19:24:55.473 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.467+0000 7f9897a5d700 1 -- 172.21.15.143:0/4284691432 wait complete. 2023-11-08T19:24:55.473 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2023-11-08T19:24:55.480 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":18,"stamp":"2023-11-08T19:24:54.509481+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591166,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":187,"num_write_kb":2427,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":140,"ondisk_log_size":140,"up":158,"acting":158,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":143,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218904,"kb_used_data":2724,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755816,"statfs":{"total":773094113280,"available":772869955584,"internally_reserved":0,"allocated":2789376,"data_stored":1705642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":798,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.917376"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:24:49.588741+0000","last_change":"2023-11-08T19:24:49.588741+0000","last_active":"2023-11-08T19:24:49.588741+0000","last_peered":"2023-11-08T19:24:49.588741+0000","last_clean":"2023-11-08T19:24:49.588741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:24:49.588741+0000","last_undegraded":"2023-11-08T19:24:49.588741+0000","last_fullsized":"2023-11-08T19:24:49.588741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.805606+0000","last_change":"2023-11-08T19:24:50.623985+0000","last_active":"2023-11-08T19:24:52.805606+0000","last_peered":"2023-11-08T19:24:52.805606+0000","last_clean":"2023-11-08T19:24:52.805606+0000","last_became_active":"2023-11-08T19:24:50.623553+0000","last_became_peered":"2023-11-08T19:24:50.623553+0000","last_unstale":"2023-11-08T19:24:52.805606+0000","last_undegraded":"2023-11-08T19:24:52.805606+0000","last_fullsized":"2023-11-08T19:24:52.805606+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:13:58.948371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.309774+0000","last_change":"2023-11-08T19:24:50.617914+0000","last_active":"2023-11-08T19:24:52.309774+0000","last_peered":"2023-11-08T19:24:52.309774+0000","last_clean":"2023-11-08T19:24:52.309774+0000","last_became_active":"2023-11-08T19:24:50.617611+0000","last_became_peered":"2023-11-08T19:24:50.617611+0000","last_unstale":"2023-11-08T19:24:52.309774+0000","last_undegraded":"2023-11-08T19:24:52.309774+0000","last_fullsized":"2023-11-08T19:24:52.309774+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:30:36.041820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.352504+0000","last_change":"2023-11-08T19:24:50.640039+0000","last_active":"2023-11-08T19:24:52.352504+0000","last_peered":"2023-11-08T19:24:52.352504+0000","last_clean":"2023-11-08T19:24:52.352504+0000","last_became_active":"2023-11-08T19:24:50.639560+0000","last_became_peered":"2023-11-08T19:24:50.639560+0000","last_unstale":"2023-11-08T19:24:52.352504+0000","last_undegraded":"2023-11-08T19:24:52.352504+0000","last_fullsized":"2023-11-08T19:24:52.352504+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:01:53.410545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:24:49.588741+0000","last_change":"2023-11-08T19:24:49.588741+0000","last_active":"2023-11-08T19:24:49.588741+0000","last_peered":"2023-11-08T19:24:49.588741+0000","last_clean":"2023-11-08T19:24:49.588741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:24:49.588741+0000","last_undegraded":"2023-11-08T19:24:49.588741+0000","last_fullsized":"2023-11-08T19:24:49.588741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.351953+0000","last_change":"2023-11-08T19:24:50.641005+0000","last_active":"2023-11-08T19:24:52.351953+0000","last_peered":"2023-11-08T19:24:52.351953+0000","last_clean":"2023-11-08T19:24:52.351953+0000","last_became_active":"2023-11-08T19:24:50.640727+0000","last_became_peered":"2023-11-08T19:24:50.640727+0000","last_unstale":"2023-11-08T19:24:52.351953+0000","last_undegraded":"2023-11-08T19:24:52.351953+0000","last_fullsized":"2023-11-08T19:24:52.351953+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:36:53.967796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:24:49.588741+0000","last_change":"2023-11-08T19:24:49.588741+0000","last_active":"2023-11-08T19:24:49.588741+0000","last_peered":"2023-11-08T19:24:49.588741+0000","last_clean":"2023-11-08T19:24:49.588741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:24:49.588741+0000","last_undegraded":"2023-11-08T19:24:49.588741+0000","last_fullsized":"2023-11-08T19:24:49.588741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.626351+0000","last_change":"2023-11-08T19:24:50.639325+0000","last_active":"2023-11-08T19:24:51.626351+0000","last_peered":"2023-11-08T19:24:51.626351+0000","last_clean":"2023-11-08T19:24:51.626351+0000","last_became_active":"2023-11-08T19:24:50.638993+0000","last_became_peered":"2023-11-08T19:24:50.638993+0000","last_unstale":"2023-11-08T19:24:51.626351+0000","last_undegraded":"2023-11-08T19:24:51.626351+0000","last_fullsized":"2023-11-08T19:24:51.626351+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:30:32.602976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:24:49.588741+0000","last_change":"2023-11-08T19:24:49.588741+0000","last_active":"2023-11-08T19:24:49.588741+0000","last_peered":"2023-11-08T19:24:49.588741+0000","last_clean":"2023-11-08T19:24:49.588741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:24:49.588741+0000","last_undegraded":"2023-11-08T19:24:49.588741+0000","last_fullsized":"2023-11-08T19:24:49.588741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:24:49.588741+0000","last_change":"2023-11-08T19:24:49.588741+0000","last_active":"2023-11-08T19:24:49.588741+0000","last_peered":"2023-11-08T19:24:49.588741+0000","last_clean":"2023-11-08T19:24:49.588741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:24:49.588741+0000","last_undegraded":"2023-11-08T19:24:49.588741+0000","last_fullsized":"2023-11-08T19:24:49.588741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:24:49.588741+0000","last_change":"2023-11-08T19:24:49.588741+0000","last_active":"2023-11-08T19:24:49.588741+0000","last_peered":"2023-11-08T19:24:49.588741+0000","last_clean":"2023-11-08T19:24:49.588741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:24:49.588741+0000","last_undegraded":"2023-11-08T19:24:49.588741+0000","last_fullsized":"2023-11-08T19:24:49.588741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.805767+0000","last_change":"2023-11-08T19:24:50.621511+0000","last_active":"2023-11-08T19:24:52.805767+0000","last_peered":"2023-11-08T19:24:52.805767+0000","last_clean":"2023-11-08T19:24:52.805767+0000","last_became_active":"2023-11-08T19:24:50.621142+0000","last_became_peered":"2023-11-08T19:24:50.621142+0000","last_unstale":"2023-11-08T19:24:52.805767+0000","last_undegraded":"2023-11-08T19:24:52.805767+0000","last_fullsized":"2023-11-08T19:24:52.805767+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:14:01.459274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.626625+0000","last_change":"2023-11-08T19:24:50.625632+0000","last_active":"2023-11-08T19:24:51.626625+0000","last_peered":"2023-11-08T19:24:51.626625+0000","last_clean":"2023-11-08T19:24:51.626625+0000","last_became_active":"2023-11-08T19:24:50.625370+0000","last_became_peered":"2023-11-08T19:24:50.625370+0000","last_unstale":"2023-11-08T19:24:51.626625+0000","last_undegraded":"2023-11-08T19:24:51.626625+0000","last_fullsized":"2023-11-08T19:24:51.626625+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:13:58.948371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.309818+0000","last_change":"2023-11-08T19:24:50.625350+0000","last_active":"2023-11-08T19:24:52.309818+0000","last_peered":"2023-11-08T19:24:52.309818+0000","last_clean":"2023-11-08T19:24:52.309818+0000","last_became_active":"2023-11-08T19:24:50.625078+0000","last_became_peered":"2023-11-08T19:24:50.625078+0000","last_unstale":"2023-11-08T19:24:52.309818+0000","last_undegraded":"2023-11-08T19:24:52.309818+0000","last_fullsized":"2023-11-08T19:24:52.309818+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:42:42.512786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.804817+0000","last_change":"2023-11-08T19:24:50.629187+0000","last_active":"2023-11-08T19:24:52.804817+0000","last_peered":"2023-11-08T19:24:52.804817+0000","last_clean":"2023-11-08T19:24:52.804817+0000","last_became_active":"2023-11-08T19:24:50.628898+0000","last_became_peered":"2023-11-08T19:24:50.628898+0000","last_unstale":"2023-11-08T19:24:52.804817+0000","last_undegraded":"2023-11-08T19:24:52.804817+0000","last_fullsized":"2023-11-08T19:24:52.804817+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:15:33.285449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.351906+0000","last_change":"2023-11-08T19:24:50.638579+0000","last_active":"2023-11-08T19:24:52.351906+0000","last_peered":"2023-11-08T19:24:52.351906+0000","last_clean":"2023-11-08T19:24:52.351906+0000","last_became_active":"2023-11-08T19:24:50.638297+0000","last_became_peered":"2023-11-08T19:24:50.638297+0000","last_unstale":"2023-11-08T19:24:52.351906+0000","last_undegraded":"2023-11-08T19:24:52.351906+0000","last_fullsized":"2023-11-08T19:24:52.351906+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:13:58.948371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.352572+0000","last_change":"2023-11-08T19:24:50.641446+0000","last_active":"2023-11-08T19:24:52.352572+0000","last_peered":"2023-11-08T19:24:52.352572+0000","last_clean":"2023-11-08T19:24:52.352572+0000","last_became_active":"2023-11-08T19:24:50.639709+0000","last_became_peered":"2023-11-08T19:24:50.639709+0000","last_unstale":"2023-11-08T19:24:52.352572+0000","last_undegraded":"2023-11-08T19:24:52.352572+0000","last_fullsized":"2023-11-08T19:24:52.352572+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:09:01.749686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.804922+0000","last_change":"2023-11-08T19:24:50.619241+0000","last_active":"2023-11-08T19:24:52.804922+0000","last_peered":"2023-11-08T19:24:52.804922+0000","last_clean":"2023-11-08T19:24:52.804922+0000","last_became_active":"2023-11-08T19:24:50.618962+0000","last_became_peered":"2023-11-08T19:24:50.618962+0000","last_unstale":"2023-11-08T19:24:52.804922+0000","last_undegraded":"2023-11-08T19:24:52.804922+0000","last_fullsized":"2023-11-08T19:24:52.804922+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:14:01.459274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:24:49.588741+0000","last_change":"2023-11-08T19:24:49.588741+0000","last_active":"2023-11-08T19:24:49.588741+0000","last_peered":"2023-11-08T19:24:49.588741+0000","last_clean":"2023-11-08T19:24:49.588741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:24:49.588741+0000","last_undegraded":"2023-11-08T19:24:49.588741+0000","last_fullsized":"2023-11-08T19:24:49.588741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.309784+0000","last_change":"2023-11-08T19:24:49.608481+0000","last_active":"2023-11-08T19:24:52.309784+0000","last_peered":"2023-11-08T19:24:52.309784+0000","last_clean":"2023-11-08T19:24:52.309784+0000","last_became_active":"2023-11-08T19:24:49.607778+0000","last_became_peered":"2023-11-08T19:24:49.607778+0000","last_unstale":"2023-11-08T19:24:52.309784+0000","last_undegraded":"2023-11-08T19:24:52.309784+0000","last_fullsized":"2023-11-08T19:24:52.309784+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:01:09.884262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.626900+0000","last_change":"2023-11-08T19:24:50.624852+0000","last_active":"2023-11-08T19:24:51.626900+0000","last_peered":"2023-11-08T19:24:51.626900+0000","last_clean":"2023-11-08T19:24:51.626900+0000","last_became_active":"2023-11-08T19:24:50.624551+0000","last_became_peered":"2023-11-08T19:24:50.624551+0000","last_unstale":"2023-11-08T19:24:51.626900+0000","last_undegraded":"2023-11-08T19:24:51.626900+0000","last_fullsized":"2023-11-08T19:24:51.626900+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:12:54.641952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.626938+0000","last_change":"2023-11-08T19:24:49.610418+0000","last_active":"2023-11-08T19:24:51.626938+0000","last_peered":"2023-11-08T19:24:51.626938+0000","last_clean":"2023-11-08T19:24:51.626938+0000","last_became_active":"2023-11-08T19:24:49.610041+0000","last_became_peered":"2023-11-08T19:24:49.610041+0000","last_unstale":"2023-11-08T19:24:51.626938+0000","last_undegraded":"2023-11-08T19:24:51.626938+0000","last_fullsized":"2023-11-08T19:24:51.626938+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:00:12.067877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.309779+0000","last_change":"2023-11-08T19:24:50.627052+0000","last_active":"2023-11-08T19:24:52.309779+0000","last_peered":"2023-11-08T19:24:52.309779+0000","last_clean":"2023-11-08T19:24:52.309779+0000","last_became_active":"2023-11-08T19:24:50.626751+0000","last_became_peered":"2023-11-08T19:24:50.626751+0000","last_unstale":"2023-11-08T19:24:52.309779+0000","last_undegraded":"2023-11-08T19:24:52.309779+0000","last_fullsized":"2023-11-08T19:24:52.309779+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:46:19.375897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-08T19:24:48.592048+0000","last_change":"2023-11-08T19:24:48.592048+0000","last_active":"2023-11-08T19:24:48.584965+0000","last_peered":"2023-11-08T19:24:48.584965+0000","last_clean":"2023-11-08T19:24:48.584965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:24:48.592048+0000","last_undegraded":"2023-11-08T19:24:48.592048+0000","last_fullsized":"2023-11-08T19:24:48.592048+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.352125+0000","last_change":"2023-11-08T19:24:50.641183+0000","last_active":"2023-11-08T19:24:52.352125+0000","last_peered":"2023-11-08T19:24:52.352125+0000","last_clean":"2023-11-08T19:24:52.352125+0000","last_became_active":"2023-11-08T19:24:50.640910+0000","last_became_peered":"2023-11-08T19:24:50.640910+0000","last_unstale":"2023-11-08T19:24:52.352125+0000","last_undegraded":"2023-11-08T19:24:52.352125+0000","last_fullsized":"2023-11-08T19:24:52.352125+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:49:19.100370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-08T19:24:48.591551+0000","last_change":"2023-11-08T19:24:48.591551+0000","last_active":"2023-11-08T19:24:48.584965+0000","last_peered":"2023-11-08T19:24:48.584965+0000","last_clean":"2023-11-08T19:24:48.584965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:24:48.591551+0000","last_undegraded":"2023-11-08T19:24:48.591551+0000","last_fullsized":"2023-11-08T19:24:48.591551+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.352284+0000","last_change":"2023-11-08T19:24:50.640245+0000","last_active":"2023-11-08T19:24:52.352284+0000","last_peered":"2023-11-08T19:24:52.352284+0000","last_clean":"2023-11-08T19:24:52.352284+0000","last_became_active":"2023-11-08T19:24:50.639668+0000","last_became_peered":"2023-11-08T19:24:50.639668+0000","last_unstale":"2023-11-08T19:24:52.352284+0000","last_undegraded":"2023-11-08T19:24:52.352284+0000","last_fullsized":"2023-11-08T19:24:52.352284+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:42:27.150066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.626599+0000","last_change":"2023-11-08T19:24:49.612665+0000","last_active":"2023-11-08T19:24:51.626599+0000","last_peered":"2023-11-08T19:24:51.626599+0000","last_clean":"2023-11-08T19:24:51.626599+0000","last_became_active":"2023-11-08T19:24:49.612362+0000","last_became_peered":"2023-11-08T19:24:49.612362+0000","last_unstale":"2023-11-08T19:24:51.626599+0000","last_undegraded":"2023-11-08T19:24:51.626599+0000","last_fullsized":"2023-11-08T19:24:51.626599+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:46:12.493411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.805419+0000","last_change":"2023-11-08T19:24:50.627537+0000","last_active":"2023-11-08T19:24:52.805419+0000","last_peered":"2023-11-08T19:24:52.805419+0000","last_clean":"2023-11-08T19:24:52.805419+0000","last_became_active":"2023-11-08T19:24:50.626489+0000","last_became_peered":"2023-11-08T19:24:50.626489+0000","last_unstale":"2023-11-08T19:24:52.805419+0000","last_undegraded":"2023-11-08T19:24:52.805419+0000","last_fullsized":"2023-11-08T19:24:52.805419+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:01:10.888039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-08T19:24:48.592162+0000","last_change":"2023-11-08T19:24:48.592162+0000","last_active":"2023-11-08T19:24:48.584965+0000","last_peered":"2023-11-08T19:24:48.584965+0000","last_clean":"2023-11-08T19:24:48.584965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:24:48.592162+0000","last_undegraded":"2023-11-08T19:24:48.592162+0000","last_fullsized":"2023-11-08T19:24:48.592162+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.309935+0000","last_change":"2023-11-08T19:24:50.627987+0000","last_active":"2023-11-08T19:24:52.309935+0000","last_peered":"2023-11-08T19:24:52.309935+0000","last_clean":"2023-11-08T19:24:52.309935+0000","last_became_active":"2023-11-08T19:24:50.627402+0000","last_became_peered":"2023-11-08T19:24:50.627402+0000","last_unstale":"2023-11-08T19:24:52.309935+0000","last_undegraded":"2023-11-08T19:24:52.309935+0000","last_fullsized":"2023-11-08T19:24:52.309935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:36:53.967796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.626523+0000","last_change":"2023-11-08T19:24:49.606482+0000","last_active":"2023-11-08T19:24:51.626523+0000","last_peered":"2023-11-08T19:24:51.626523+0000","last_clean":"2023-11-08T19:24:51.626523+0000","last_became_active":"2023-11-08T19:24:49.606211+0000","last_became_peered":"2023-11-08T19:24:49.606211+0000","last_unstale":"2023-11-08T19:24:51.626523+0000","last_undegraded":"2023-11-08T19:24:51.626523+0000","last_fullsized":"2023-11-08T19:24:51.626523+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:42:59.845581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.626483+0000","last_change":"2023-11-08T19:24:50.625917+0000","last_active":"2023-11-08T19:24:51.626483+0000","last_peered":"2023-11-08T19:24:51.626483+0000","last_clean":"2023-11-08T19:24:51.626483+0000","last_became_active":"2023-11-08T19:24:50.625398+0000","last_became_peered":"2023-11-08T19:24:50.625398+0000","last_unstale":"2023-11-08T19:24:51.626483+0000","last_undegraded":"2023-11-08T19:24:51.626483+0000","last_fullsized":"2023-11-08T19:24:51.626483+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:40:11.978983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.351905+0000","last_change":"2023-11-08T19:24:49.616558+0000","last_active":"2023-11-08T19:24:52.351905+0000","last_peered":"2023-11-08T19:24:52.351905+0000","last_clean":"2023-11-08T19:24:52.351905+0000","last_became_active":"2023-11-08T19:24:49.616277+0000","last_became_peered":"2023-11-08T19:24:49.616277+0000","last_unstale":"2023-11-08T19:24:52.351905+0000","last_undegraded":"2023-11-08T19:24:52.351905+0000","last_fullsized":"2023-11-08T19:24:52.351905+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:01:09.884262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.805048+0000","last_change":"2023-11-08T19:24:50.623658+0000","last_active":"2023-11-08T19:24:52.805048+0000","last_peered":"2023-11-08T19:24:52.805048+0000","last_clean":"2023-11-08T19:24:52.805048+0000","last_became_active":"2023-11-08T19:24:50.623399+0000","last_became_peered":"2023-11-08T19:24:50.623399+0000","last_unstale":"2023-11-08T19:24:52.805048+0000","last_undegraded":"2023-11-08T19:24:52.805048+0000","last_fullsized":"2023-11-08T19:24:52.805048+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:40:11.978983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-08T19:24:48.593392+0000","last_change":"2023-11-08T19:24:48.593392+0000","last_active":"2023-11-08T19:24:48.584965+0000","last_peered":"2023-11-08T19:24:48.584965+0000","last_clean":"2023-11-08T19:24:48.584965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:24:48.593392+0000","last_undegraded":"2023-11-08T19:24:48.593392+0000","last_fullsized":"2023-11-08T19:24:48.593392+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.352305+0000","last_change":"2023-11-08T19:24:50.640397+0000","last_active":"2023-11-08T19:24:52.352305+0000","last_peered":"2023-11-08T19:24:52.352305+0000","last_clean":"2023-11-08T19:24:52.352305+0000","last_became_active":"2023-11-08T19:24:50.639730+0000","last_became_peered":"2023-11-08T19:24:50.639730+0000","last_unstale":"2023-11-08T19:24:52.352305+0000","last_undegraded":"2023-11-08T19:24:52.352305+0000","last_fullsized":"2023-11-08T19:24:52.352305+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:08:22.499163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.352212+0000","last_change":"2023-11-08T19:24:49.611202+0000","last_active":"2023-11-08T19:24:52.352212+0000","last_peered":"2023-11-08T19:24:52.352212+0000","last_clean":"2023-11-08T19:24:52.352212+0000","last_became_active":"2023-11-08T19:24:49.610894+0000","last_became_peered":"2023-11-08T19:24:49.610894+0000","last_unstale":"2023-11-08T19:24:52.352212+0000","last_undegraded":"2023-11-08T19:24:52.352212+0000","last_fullsized":"2023-11-08T19:24:52.352212+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:48:25.546248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.309935+0000","last_change":"2023-11-08T19:24:50.638901+0000","last_active":"2023-11-08T19:24:52.309935+0000","last_peered":"2023-11-08T19:24:52.309935+0000","last_clean":"2023-11-08T19:24:52.309935+0000","last_became_active":"2023-11-08T19:24:50.638587+0000","last_became_peered":"2023-11-08T19:24:50.638587+0000","last_unstale":"2023-11-08T19:24:52.309935+0000","last_undegraded":"2023-11-08T19:24:52.309935+0000","last_fullsized":"2023-11-08T19:24:52.309935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:13:58.948371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.805313+0000","last_change":"2023-11-08T19:24:49.612170+0000","last_active":"2023-11-08T19:24:52.805313+0000","last_peered":"2023-11-08T19:24:52.805313+0000","last_clean":"2023-11-08T19:24:52.805313+0000","last_became_active":"2023-11-08T19:24:49.611895+0000","last_became_peered":"2023-11-08T19:24:49.611895+0000","last_unstale":"2023-11-08T19:24:52.805313+0000","last_undegraded":"2023-11-08T19:24:52.805313+0000","last_fullsized":"2023-11-08T19:24:52.805313+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:38:24.246638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.805721+0000","last_change":"2023-11-08T19:24:50.627060+0000","last_active":"2023-11-08T19:24:52.805721+0000","last_peered":"2023-11-08T19:24:52.805721+0000","last_clean":"2023-11-08T19:24:52.805721+0000","last_became_active":"2023-11-08T19:24:50.626468+0000","last_became_peered":"2023-11-08T19:24:50.626468+0000","last_unstale":"2023-11-08T19:24:52.805721+0000","last_undegraded":"2023-11-08T19:24:52.805721+0000","last_fullsized":"2023-11-08T19:24:52.805721+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:00:13.071654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-08T19:24:48.591494+0000","last_change":"2023-11-08T19:24:48.591494+0000","last_active":"2023-11-08T19:24:48.584965+0000","last_peered":"2023-11-08T19:24:48.584965+0000","last_clean":"2023-11-08T19:24:48.584965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:24:48.591494+0000","last_undegraded":"2023-11-08T19:24:48.591494+0000","last_fullsized":"2023-11-08T19:24:48.591494+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:24:49.588741+0000","last_change":"2023-11-08T19:24:49.588741+0000","last_active":"2023-11-08T19:24:49.588741+0000","last_peered":"2023-11-08T19:24:49.588741+0000","last_clean":"2023-11-08T19:24:49.588741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:24:49.588741+0000","last_undegraded":"2023-11-08T19:24:49.588741+0000","last_fullsized":"2023-11-08T19:24:49.588741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.309884+0000","last_change":"2023-11-08T19:24:49.610471+0000","last_active":"2023-11-08T19:24:52.309884+0000","last_peered":"2023-11-08T19:24:52.309884+0000","last_clean":"2023-11-08T19:24:52.309884+0000","last_became_active":"2023-11-08T19:24:49.610182+0000","last_became_peered":"2023-11-08T19:24:49.610182+0000","last_unstale":"2023-11-08T19:24:52.309884+0000","last_undegraded":"2023-11-08T19:24:52.309884+0000","last_fullsized":"2023-11-08T19:24:52.309884+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:14:00.455497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.309602+0000","last_change":"2023-11-08T19:24:50.619498+0000","last_active":"2023-11-08T19:24:52.309602+0000","last_peered":"2023-11-08T19:24:52.309602+0000","last_clean":"2023-11-08T19:24:52.309602+0000","last_became_active":"2023-11-08T19:24:50.619231+0000","last_became_peered":"2023-11-08T19:24:50.619231+0000","last_unstale":"2023-11-08T19:24:52.309602+0000","last_undegraded":"2023-11-08T19:24:52.309602+0000","last_fullsized":"2023-11-08T19:24:52.309602+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:38:25.250414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.352455+0000","last_change":"2023-11-08T19:24:49.612336+0000","last_active":"2023-11-08T19:24:52.352455+0000","last_peered":"2023-11-08T19:24:52.352455+0000","last_clean":"2023-11-08T19:24:52.352455+0000","last_became_active":"2023-11-08T19:24:49.611885+0000","last_became_peered":"2023-11-08T19:24:49.611885+0000","last_unstale":"2023-11-08T19:24:52.352455+0000","last_undegraded":"2023-11-08T19:24:52.352455+0000","last_fullsized":"2023-11-08T19:24:52.352455+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:46:18.372121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:24:49.588741+0000","last_change":"2023-11-08T19:24:49.588741+0000","last_active":"2023-11-08T19:24:49.588741+0000","last_peered":"2023-11-08T19:24:49.588741+0000","last_clean":"2023-11-08T19:24:49.588741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:24:49.588741+0000","last_undegraded":"2023-11-08T19:24:49.588741+0000","last_fullsized":"2023-11-08T19:24:49.588741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-08T19:24:48.591465+0000","last_change":"2023-11-08T19:24:48.591465+0000","last_active":"2023-11-08T19:24:48.584965+0000","last_peered":"2023-11-08T19:24:48.584965+0000","last_clean":"2023-11-08T19:24:48.584965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:24:48.591465+0000","last_undegraded":"2023-11-08T19:24:48.591465+0000","last_fullsized":"2023-11-08T19:24:48.591465+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:24:49.588741+0000","last_change":"2023-11-08T19:24:49.588741+0000","last_active":"2023-11-08T19:24:49.588741+0000","last_peered":"2023-11-08T19:24:49.588741+0000","last_clean":"2023-11-08T19:24:49.588741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:24:49.588741+0000","last_undegraded":"2023-11-08T19:24:49.588741+0000","last_fullsized":"2023-11-08T19:24:49.588741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.804637+0000","last_change":"2023-11-08T19:24:49.608503+0000","last_active":"2023-11-08T19:24:52.804637+0000","last_peered":"2023-11-08T19:24:52.804637+0000","last_clean":"2023-11-08T19:24:52.804637+0000","last_became_active":"2023-11-08T19:24:49.608176+0000","last_became_peered":"2023-11-08T19:24:49.608176+0000","last_unstale":"2023-11-08T19:24:52.804637+0000","last_undegraded":"2023-11-08T19:24:52.804637+0000","last_fullsized":"2023-11-08T19:24:52.804637+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:17:50.005667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:24:49.588741+0000","last_change":"2023-11-08T19:24:49.588741+0000","last_active":"2023-11-08T19:24:49.588741+0000","last_peered":"2023-11-08T19:24:49.588741+0000","last_clean":"2023-11-08T19:24:49.588741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:24:49.588741+0000","last_undegraded":"2023-11-08T19:24:49.588741+0000","last_fullsized":"2023-11-08T19:24:49.588741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.805855+0000","last_change":"2023-11-08T19:24:49.610638+0000","last_active":"2023-11-08T19:24:52.805855+0000","last_peered":"2023-11-08T19:24:52.805855+0000","last_clean":"2023-11-08T19:24:52.805855+0000","last_became_active":"2023-11-08T19:24:49.610362+0000","last_became_peered":"2023-11-08T19:24:49.610362+0000","last_unstale":"2023-11-08T19:24:52.805855+0000","last_undegraded":"2023-11-08T19:24:52.805855+0000","last_fullsized":"2023-11-08T19:24:52.805855+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:15:32.030617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:24:49.588741+0000","last_change":"2023-11-08T19:24:49.588741+0000","last_active":"2023-11-08T19:24:49.588741+0000","last_peered":"2023-11-08T19:24:49.588741+0000","last_clean":"2023-11-08T19:24:49.588741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:24:49.588741+0000","last_undegraded":"2023-11-08T19:24:49.588741+0000","last_fullsized":"2023-11-08T19:24:49.588741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.805154+0000","last_change":"2023-11-08T19:24:49.613413+0000","last_active":"2023-11-08T19:24:52.805154+0000","last_peered":"2023-11-08T19:24:52.805154+0000","last_clean":"2023-11-08T19:24:52.805154+0000","last_became_active":"2023-11-08T19:24:49.613131+0000","last_became_peered":"2023-11-08T19:24:49.613131+0000","last_unstale":"2023-11-08T19:24:52.805154+0000","last_undegraded":"2023-11-08T19:24:52.805154+0000","last_fullsized":"2023-11-08T19:24:52.805154+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:46:12.493411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.352374+0000","last_change":"2023-11-08T19:24:50.640031+0000","last_active":"2023-11-08T19:24:52.352374+0000","last_peered":"2023-11-08T19:24:52.352374+0000","last_clean":"2023-11-08T19:24:52.352374+0000","last_became_active":"2023-11-08T19:24:50.639718+0000","last_became_peered":"2023-11-08T19:24:50.639718+0000","last_unstale":"2023-11-08T19:24:52.352374+0000","last_undegraded":"2023-11-08T19:24:52.352374+0000","last_fullsized":"2023-11-08T19:24:52.352374+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:08:17.096077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.352346+0000","last_change":"2023-11-08T19:24:49.612137+0000","last_active":"2023-11-08T19:24:52.352346+0000","last_peered":"2023-11-08T19:24:52.352346+0000","last_clean":"2023-11-08T19:24:52.352346+0000","last_became_active":"2023-11-08T19:24:49.611532+0000","last_became_peered":"2023-11-08T19:24:49.611532+0000","last_unstale":"2023-11-08T19:24:52.352346+0000","last_undegraded":"2023-11-08T19:24:52.352346+0000","last_fullsized":"2023-11-08T19:24:52.352346+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:40:10.975206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.805536+0000","last_change":"2023-11-08T19:24:50.630854+0000","last_active":"2023-11-08T19:24:52.805536+0000","last_peered":"2023-11-08T19:24:52.805536+0000","last_clean":"2023-11-08T19:24:52.805536+0000","last_became_active":"2023-11-08T19:24:50.630549+0000","last_became_peered":"2023-11-08T19:24:50.630549+0000","last_unstale":"2023-11-08T19:24:52.805536+0000","last_undegraded":"2023-11-08T19:24:52.805536+0000","last_fullsized":"2023-11-08T19:24:52.805536+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:48:26.550025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.804950+0000","last_change":"2023-11-08T19:24:50.620897+0000","last_active":"2023-11-08T19:24:52.804950+0000","last_peered":"2023-11-08T19:24:52.804950+0000","last_clean":"2023-11-08T19:24:52.804950+0000","last_became_active":"2023-11-08T19:24:50.620606+0000","last_became_peered":"2023-11-08T19:24:50.620606+0000","last_unstale":"2023-11-08T19:24:52.804950+0000","last_undegraded":"2023-11-08T19:24:52.804950+0000","last_fullsized":"2023-11-08T19:24:52.804950+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:30:36.041820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.804625+0000","last_change":"2023-11-08T19:24:49.612840+0000","last_active":"2023-11-08T19:24:52.804625+0000","last_peered":"2023-11-08T19:24:52.804625+0000","last_clean":"2023-11-08T19:24:52.804625+0000","last_became_active":"2023-11-08T19:24:49.612652+0000","last_became_peered":"2023-11-08T19:24:49.612652+0000","last_unstale":"2023-11-08T19:24:52.804625+0000","last_undegraded":"2023-11-08T19:24:52.804625+0000","last_fullsized":"2023-11-08T19:24:52.804625+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:48:25.546248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.626593+0000","last_change":"2023-11-08T19:24:50.642515+0000","last_active":"2023-11-08T19:24:51.626593+0000","last_peered":"2023-11-08T19:24:51.626593+0000","last_clean":"2023-11-08T19:24:51.626593+0000","last_became_active":"2023-11-08T19:24:50.642204+0000","last_became_peered":"2023-11-08T19:24:50.642204+0000","last_unstale":"2023-11-08T19:24:51.626593+0000","last_undegraded":"2023-11-08T19:24:51.626593+0000","last_fullsized":"2023-11-08T19:24:51.626593+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:42:42.512786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.351949+0000","last_change":"2023-11-08T19:24:49.605594+0000","last_active":"2023-11-08T19:24:52.351949+0000","last_peered":"2023-11-08T19:24:52.351949+0000","last_clean":"2023-11-08T19:24:52.351949+0000","last_became_active":"2023-11-08T19:24:49.605208+0000","last_became_peered":"2023-11-08T19:24:49.605208+0000","last_unstale":"2023-11-08T19:24:52.351949+0000","last_undegraded":"2023-11-08T19:24:52.351949+0000","last_fullsized":"2023-11-08T19:24:52.351949+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:17:50.005667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":162,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-11-08T19:24:48.592313+0000","last_change":"2023-11-08T19:24:45.888122+0000","last_active":"2023-11-08T19:24:48.592313+0000","last_peered":"2023-11-08T19:24:48.592313+0000","last_clean":"2023-11-08T19:24:48.592313+0000","last_became_active":"2023-11-08T19:24:45.887048+0000","last_became_peered":"2023-11-08T19:24:45.887048+0000","last_unstale":"2023-11-08T19:24:48.592313+0000","last_undegraded":"2023-11-08T19:24:48.592313+0000","last_fullsized":"2023-11-08T19:24:48.592313+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:44.575133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:44.575133+0000","last_clean_scrub_stamp":"2023-11-08T19:24:44.575133+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:19:20.202062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.309622+0000","last_change":"2023-11-08T19:24:50.628353+0000","last_active":"2023-11-08T19:24:52.309622+0000","last_peered":"2023-11-08T19:24:52.309622+0000","last_clean":"2023-11-08T19:24:52.309622+0000","last_became_active":"2023-11-08T19:24:50.628008+0000","last_became_peered":"2023-11-08T19:24:50.628008+0000","last_unstale":"2023-11-08T19:24:52.309622+0000","last_undegraded":"2023-11-08T19:24:52.309622+0000","last_fullsized":"2023-11-08T19:24:52.309622+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:46:13.497187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-08T19:24:48.591491+0000","last_change":"2023-11-08T19:24:48.591491+0000","last_active":"2023-11-08T19:24:48.584965+0000","last_peered":"2023-11-08T19:24:48.584965+0000","last_clean":"2023-11-08T19:24:48.584965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:24:48.591491+0000","last_undegraded":"2023-11-08T19:24:48.591491+0000","last_fullsized":"2023-11-08T19:24:48.591491+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-08T19:24:48.591465+0000","last_change":"2023-11-08T19:24:48.591465+0000","last_active":"2023-11-08T19:24:48.584965+0000","last_peered":"2023-11-08T19:24:48.584965+0000","last_clean":"2023-11-08T19:24:48.584965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:24:48.591465+0000","last_undegraded":"2023-11-08T19:24:48.591465+0000","last_fullsized":"2023-11-08T19:24:48.591465+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.805231+0000","last_change":"2023-11-08T19:24:50.624279+0000","last_active":"2023-11-08T19:24:52.805231+0000","last_peered":"2023-11-08T19:24:52.805231+0000","last_clean":"2023-11-08T19:24:52.805231+0000","last_became_active":"2023-11-08T19:24:50.623920+0000","last_became_peered":"2023-11-08T19:24:50.623920+0000","last_unstale":"2023-11-08T19:24:52.805231+0000","last_undegraded":"2023-11-08T19:24:52.805231+0000","last_fullsized":"2023-11-08T19:24:52.805231+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:42:42.512786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.805402+0000","last_change":"2023-11-08T19:24:49.606910+0000","last_active":"2023-11-08T19:24:52.805402+0000","last_peered":"2023-11-08T19:24:52.805402+0000","last_clean":"2023-11-08T19:24:52.805402+0000","last_became_active":"2023-11-08T19:24:49.606627+0000","last_became_peered":"2023-11-08T19:24:49.606627+0000","last_unstale":"2023-11-08T19:24:52.805402+0000","last_undegraded":"2023-11-08T19:24:52.805402+0000","last_fullsized":"2023-11-08T19:24:52.805402+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:56:00.148860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.627087+0000","last_change":"2023-11-08T19:24:50.622398+0000","last_active":"2023-11-08T19:24:51.627087+0000","last_peered":"2023-11-08T19:24:51.627087+0000","last_clean":"2023-11-08T19:24:51.627087+0000","last_became_active":"2023-11-08T19:24:50.622092+0000","last_became_peered":"2023-11-08T19:24:50.622092+0000","last_unstale":"2023-11-08T19:24:51.627087+0000","last_undegraded":"2023-11-08T19:24:51.627087+0000","last_fullsized":"2023-11-08T19:24:51.627087+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:42:27.150066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-08T19:24:48.592846+0000","last_change":"2023-11-08T19:24:48.592846+0000","last_active":"2023-11-08T19:24:48.584965+0000","last_peered":"2023-11-08T19:24:48.584965+0000","last_clean":"2023-11-08T19:24:48.584965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:24:48.592846+0000","last_undegraded":"2023-11-08T19:24:48.592846+0000","last_fullsized":"2023-11-08T19:24:48.592846+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.309770+0000","last_change":"2023-11-08T19:24:50.624848+0000","last_active":"2023-11-08T19:24:52.309770+0000","last_peered":"2023-11-08T19:24:52.309770+0000","last_clean":"2023-11-08T19:24:52.309770+0000","last_became_active":"2023-11-08T19:24:50.624518+0000","last_became_peered":"2023-11-08T19:24:50.624518+0000","last_unstale":"2023-11-08T19:24:52.309770+0000","last_undegraded":"2023-11-08T19:24:52.309770+0000","last_fullsized":"2023-11-08T19:24:52.309770+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:15:33.285449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-08T19:24:48.592059+0000","last_change":"2023-11-08T19:24:48.592059+0000","last_active":"2023-11-08T19:24:48.584965+0000","last_peered":"2023-11-08T19:24:48.584965+0000","last_clean":"2023-11-08T19:24:48.584965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:24:48.592059+0000","last_undegraded":"2023-11-08T19:24:48.592059+0000","last_fullsized":"2023-11-08T19:24:48.592059+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.805558+0000","last_change":"2023-11-08T19:24:50.627488+0000","last_active":"2023-11-08T19:24:52.805558+0000","last_peered":"2023-11-08T19:24:52.805558+0000","last_clean":"2023-11-08T19:24:52.805558+0000","last_became_active":"2023-11-08T19:24:50.626571+0000","last_became_peered":"2023-11-08T19:24:50.626571+0000","last_unstale":"2023-11-08T19:24:52.805558+0000","last_undegraded":"2023-11-08T19:24:52.805558+0000","last_fullsized":"2023-11-08T19:24:52.805558+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:43:00.849357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.804911+0000","last_change":"2023-11-08T19:24:49.606579+0000","last_active":"2023-11-08T19:24:52.804911+0000","last_peered":"2023-11-08T19:24:52.804911+0000","last_clean":"2023-11-08T19:24:52.804911+0000","last_became_active":"2023-11-08T19:24:49.606283+0000","last_became_peered":"2023-11-08T19:24:49.606283+0000","last_unstale":"2023-11-08T19:24:52.804911+0000","last_undegraded":"2023-11-08T19:24:52.804911+0000","last_fullsized":"2023-11-08T19:24:52.804911+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:14:00.455497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.626421+0000","last_change":"2023-11-08T19:24:50.625692+0000","last_active":"2023-11-08T19:24:51.626421+0000","last_peered":"2023-11-08T19:24:51.626421+0000","last_clean":"2023-11-08T19:24:51.626421+0000","last_became_active":"2023-11-08T19:24:50.625014+0000","last_became_peered":"2023-11-08T19:24:50.625014+0000","last_unstale":"2023-11-08T19:24:51.626421+0000","last_undegraded":"2023-11-08T19:24:51.626421+0000","last_fullsized":"2023-11-08T19:24:51.626421+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:15:33.285449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.805081+0000","last_change":"2023-11-08T19:24:49.611023+0000","last_active":"2023-11-08T19:24:52.805081+0000","last_peered":"2023-11-08T19:24:52.805081+0000","last_clean":"2023-11-08T19:24:52.805081+0000","last_became_active":"2023-11-08T19:24:49.610735+0000","last_became_peered":"2023-11-08T19:24:49.610735+0000","last_unstale":"2023-11-08T19:24:52.805081+0000","last_undegraded":"2023-11-08T19:24:52.805081+0000","last_fullsized":"2023-11-08T19:24:52.805081+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:30:31.599200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.626730+0000","last_change":"2023-11-08T19:24:50.631491+0000","last_active":"2023-11-08T19:24:51.626730+0000","last_peered":"2023-11-08T19:24:51.626730+0000","last_clean":"2023-11-08T19:24:51.626730+0000","last_became_active":"2023-11-08T19:24:50.630872+0000","last_became_peered":"2023-11-08T19:24:50.630872+0000","last_unstale":"2023-11-08T19:24:51.626730+0000","last_undegraded":"2023-11-08T19:24:51.626730+0000","last_fullsized":"2023-11-08T19:24:51.626730+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:36:53.967796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-08T19:24:48.592103+0000","last_change":"2023-11-08T19:24:48.592103+0000","last_active":"2023-11-08T19:24:48.584965+0000","last_peered":"2023-11-08T19:24:48.584965+0000","last_clean":"2023-11-08T19:24:48.584965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:24:48.592103+0000","last_undegraded":"2023-11-08T19:24:48.592103+0000","last_fullsized":"2023-11-08T19:24:48.592103+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:24:49.588741+0000","last_change":"2023-11-08T19:24:49.588741+0000","last_active":"2023-11-08T19:24:49.588741+0000","last_peered":"2023-11-08T19:24:49.588741+0000","last_clean":"2023-11-08T19:24:49.588741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:24:49.588741+0000","last_undegraded":"2023-11-08T19:24:49.588741+0000","last_fullsized":"2023-11-08T19:24:49.588741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.805091+0000","last_change":"2023-11-08T19:24:49.613365+0000","last_active":"2023-11-08T19:24:52.805091+0000","last_peered":"2023-11-08T19:24:52.805091+0000","last_clean":"2023-11-08T19:24:52.805091+0000","last_became_active":"2023-11-08T19:24:49.613052+0000","last_became_peered":"2023-11-08T19:24:49.613052+0000","last_unstale":"2023-11-08T19:24:52.805091+0000","last_undegraded":"2023-11-08T19:24:52.805091+0000","last_fullsized":"2023-11-08T19:24:52.805091+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:41:24.551269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:24:49.588741+0000","last_change":"2023-11-08T19:24:49.588741+0000","last_active":"2023-11-08T19:24:49.588741+0000","last_peered":"2023-11-08T19:24:49.588741+0000","last_clean":"2023-11-08T19:24:49.588741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:24:49.588741+0000","last_undegraded":"2023-11-08T19:24:49.588741+0000","last_fullsized":"2023-11-08T19:24:49.588741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.805219+0000","last_change":"2023-11-08T19:24:49.606271+0000","last_active":"2023-11-08T19:24:52.805219+0000","last_peered":"2023-11-08T19:24:52.805219+0000","last_clean":"2023-11-08T19:24:52.805219+0000","last_became_active":"2023-11-08T19:24:49.605802+0000","last_became_peered":"2023-11-08T19:24:49.605802+0000","last_unstale":"2023-11-08T19:24:52.805219+0000","last_undegraded":"2023-11-08T19:24:52.805219+0000","last_fullsized":"2023-11-08T19:24:52.805219+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:07:00.575561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.352338+0000","last_change":"2023-11-08T19:24:50.640594+0000","last_active":"2023-11-08T19:24:52.352338+0000","last_peered":"2023-11-08T19:24:52.352338+0000","last_clean":"2023-11-08T19:24:52.352338+0000","last_became_active":"2023-11-08T19:24:50.640295+0000","last_became_peered":"2023-11-08T19:24:50.640295+0000","last_unstale":"2023-11-08T19:24:52.352338+0000","last_undegraded":"2023-11-08T19:24:52.352338+0000","last_fullsized":"2023-11-08T19:24:52.352338+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:34:42.075394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.805026+0000","last_change":"2023-11-08T19:24:49.610782+0000","last_active":"2023-11-08T19:24:52.805026+0000","last_peered":"2023-11-08T19:24:52.805026+0000","last_clean":"2023-11-08T19:24:52.805026+0000","last_became_active":"2023-11-08T19:24:49.610493+0000","last_became_peered":"2023-11-08T19:24:49.610493+0000","last_unstale":"2023-11-08T19:24:52.805026+0000","last_undegraded":"2023-11-08T19:24:52.805026+0000","last_fullsized":"2023-11-08T19:24:52.805026+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:42:59.845581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:24:49.588741+0000","last_change":"2023-11-08T19:24:49.588741+0000","last_active":"2023-11-08T19:24:49.588741+0000","last_peered":"2023-11-08T19:24:49.588741+0000","last_clean":"2023-11-08T19:24:49.588741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:24:49.588741+0000","last_undegraded":"2023-11-08T19:24:49.588741+0000","last_fullsized":"2023-11-08T19:24:49.588741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:24:49.588741+0000","last_change":"2023-11-08T19:24:49.588741+0000","last_active":"2023-11-08T19:24:49.588741+0000","last_peered":"2023-11-08T19:24:49.588741+0000","last_clean":"2023-11-08T19:24:49.588741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:24:49.588741+0000","last_undegraded":"2023-11-08T19:24:49.588741+0000","last_fullsized":"2023-11-08T19:24:49.588741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.804775+0000","last_change":"2023-11-08T19:24:50.625948+0000","last_active":"2023-11-08T19:24:52.804775+0000","last_peered":"2023-11-08T19:24:52.804775+0000","last_clean":"2023-11-08T19:24:52.804775+0000","last_became_active":"2023-11-08T19:24:50.625633+0000","last_became_peered":"2023-11-08T19:24:50.625633+0000","last_unstale":"2023-11-08T19:24:52.804775+0000","last_undegraded":"2023-11-08T19:24:52.804775+0000","last_fullsized":"2023-11-08T19:24:52.804775+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:00:13.071654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.352423+0000","last_change":"2023-11-08T19:24:50.640919+0000","last_active":"2023-11-08T19:24:52.352423+0000","last_peered":"2023-11-08T19:24:52.352423+0000","last_clean":"2023-11-08T19:24:52.352423+0000","last_became_active":"2023-11-08T19:24:50.640496+0000","last_became_peered":"2023-11-08T19:24:50.640496+0000","last_unstale":"2023-11-08T19:24:52.352423+0000","last_undegraded":"2023-11-08T19:24:52.352423+0000","last_fullsized":"2023-11-08T19:24:52.352423+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:59:37.592538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.309782+0000","last_change":"2023-11-08T19:24:50.625916+0000","last_active":"2023-11-08T19:24:52.309782+0000","last_peered":"2023-11-08T19:24:52.309782+0000","last_clean":"2023-11-08T19:24:52.309782+0000","last_became_active":"2023-11-08T19:24:50.625625+0000","last_became_peered":"2023-11-08T19:24:50.625625+0000","last_unstale":"2023-11-08T19:24:52.309782+0000","last_undegraded":"2023-11-08T19:24:52.309782+0000","last_fullsized":"2023-11-08T19:24:52.309782+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:40:11.978983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.626555+0000","last_change":"2023-11-08T19:24:50.628686+0000","last_active":"2023-11-08T19:24:51.626555+0000","last_peered":"2023-11-08T19:24:51.626555+0000","last_clean":"2023-11-08T19:24:51.626555+0000","last_became_active":"2023-11-08T19:24:50.628289+0000","last_became_peered":"2023-11-08T19:24:50.628289+0000","last_unstale":"2023-11-08T19:24:51.626555+0000","last_undegraded":"2023-11-08T19:24:51.626555+0000","last_fullsized":"2023-11-08T19:24:51.626555+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:46:19.375897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.805231+0000","last_change":"2023-11-08T19:24:50.624352+0000","last_active":"2023-11-08T19:24:52.805231+0000","last_peered":"2023-11-08T19:24:52.805231+0000","last_clean":"2023-11-08T19:24:52.805231+0000","last_became_active":"2023-11-08T19:24:50.624035+0000","last_became_peered":"2023-11-08T19:24:50.624035+0000","last_unstale":"2023-11-08T19:24:52.805231+0000","last_undegraded":"2023-11-08T19:24:52.805231+0000","last_fullsized":"2023-11-08T19:24:52.805231+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:46:19.375897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:24:49.588741+0000","last_change":"2023-11-08T19:24:49.588741+0000","last_active":"2023-11-08T19:24:49.588741+0000","last_peered":"2023-11-08T19:24:49.588741+0000","last_clean":"2023-11-08T19:24:49.588741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:24:49.588741+0000","last_undegraded":"2023-11-08T19:24:49.588741+0000","last_fullsized":"2023-11-08T19:24:49.588741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.626991+0000","last_change":"2023-11-08T19:24:50.628009+0000","last_active":"2023-11-08T19:24:51.626991+0000","last_peered":"2023-11-08T19:24:51.626991+0000","last_clean":"2023-11-08T19:24:51.626991+0000","last_became_active":"2023-11-08T19:24:50.627668+0000","last_became_peered":"2023-11-08T19:24:50.627668+0000","last_unstale":"2023-11-08T19:24:51.626991+0000","last_undegraded":"2023-11-08T19:24:51.626991+0000","last_fullsized":"2023-11-08T19:24:51.626991+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:49:19.100370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.352100+0000","last_change":"2023-11-08T19:24:50.641255+0000","last_active":"2023-11-08T19:24:52.352100+0000","last_peered":"2023-11-08T19:24:52.352100+0000","last_clean":"2023-11-08T19:24:52.352100+0000","last_became_active":"2023-11-08T19:24:50.640164+0000","last_became_peered":"2023-11-08T19:24:50.640164+0000","last_unstale":"2023-11-08T19:24:52.352100+0000","last_undegraded":"2023-11-08T19:24:52.352100+0000","last_fullsized":"2023-11-08T19:24:52.352100+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:12:54.641952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.626435+0000","last_change":"2023-11-08T19:24:50.629006+0000","last_active":"2023-11-08T19:24:51.626435+0000","last_peered":"2023-11-08T19:24:51.626435+0000","last_clean":"2023-11-08T19:24:51.626435+0000","last_became_active":"2023-11-08T19:24:50.628685+0000","last_became_peered":"2023-11-08T19:24:50.628685+0000","last_unstale":"2023-11-08T19:24:51.626435+0000","last_undegraded":"2023-11-08T19:24:51.626435+0000","last_fullsized":"2023-11-08T19:24:51.626435+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:30:36.041820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.352270+0000","last_change":"2023-11-08T19:24:50.640737+0000","last_active":"2023-11-08T19:24:52.352270+0000","last_peered":"2023-11-08T19:24:52.352270+0000","last_clean":"2023-11-08T19:24:52.352270+0000","last_became_active":"2023-11-08T19:24:50.640255+0000","last_became_peered":"2023-11-08T19:24:50.640255+0000","last_unstale":"2023-11-08T19:24:52.352270+0000","last_undegraded":"2023-11-08T19:24:52.352270+0000","last_fullsized":"2023-11-08T19:24:52.352270+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:13:41.469149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.309753+0000","last_change":"2023-11-08T19:24:50.620645+0000","last_active":"2023-11-08T19:24:52.309753+0000","last_peered":"2023-11-08T19:24:52.309753+0000","last_clean":"2023-11-08T19:24:52.309753+0000","last_became_active":"2023-11-08T19:24:50.620349+0000","last_became_peered":"2023-11-08T19:24:50.620349+0000","last_unstale":"2023-11-08T19:24:52.309753+0000","last_undegraded":"2023-11-08T19:24:52.309753+0000","last_fullsized":"2023-11-08T19:24:52.309753+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:30:32.602976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:24:49.588741+0000","last_change":"2023-11-08T19:24:49.588741+0000","last_active":"2023-11-08T19:24:49.588741+0000","last_peered":"2023-11-08T19:24:49.588741+0000","last_clean":"2023-11-08T19:24:49.588741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:24:49.588741+0000","last_undegraded":"2023-11-08T19:24:49.588741+0000","last_fullsized":"2023-11-08T19:24:49.588741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":92,"acting":92,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":798,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":2626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":23,"ondisk_log_size":23,"up":64,"acting":64,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27760,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344080,"statfs":{"total":96636764160,"available":96608337920,"internally_reserved":0,"allocated":761856,"data_stored":641928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47099999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52000000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.432}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56200000000000006}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.372}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67900000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57599999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":167936,"data_stored":51560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47599999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.495}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51900000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56200000000000006}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39400000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":71070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36299999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23499999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26500000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45200000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47699999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":811008,"data_stored":661916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35099999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27400000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40000000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73199999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71099999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68300000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81000000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":192512,"data_stored":64610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46000000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.443}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82899999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81599999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84799999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.878}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.308}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27000000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.379}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59399999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50900000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54100000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70099999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":71036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46000000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55800000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65400000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75800000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67200000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.746}]}]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":208896,"data_stored":71000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39800000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55300000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61099999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.439}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66700000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69499999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-08T19:24:55.486 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3f is not active+clean 2023-11-08T19:24:55.486 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-08T19:24:49.588741+0000', 'last_change': '2023-11-08T19:24:49.588741+0000', 'last_active': '2023-11-08T19:24:49.588741+0000', 'last_peered': '2023-11-08T19:24:49.588741+0000', 'last_clean': '2023-11-08T19:24:49.588741+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:24:49.588741+0000', 'last_undegraded': '2023-11-08T19:24:49.588741+0000', 'last_fullsized': '2023-11-08T19:24:49.588741+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:24:49.588741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:24:49.588741+0000', 'last_clean_scrub_stamp': '2023-11-08T19:24:49.588741+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-08T19:24:55.487 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3b is not active+clean 2023-11-08T19:24:55.487 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-08T19:24:49.588741+0000', 'last_change': '2023-11-08T19:24:49.588741+0000', 'last_active': '2023-11-08T19:24:49.588741+0000', 'last_peered': '2023-11-08T19:24:49.588741+0000', 'last_clean': '2023-11-08T19:24:49.588741+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:24:49.588741+0000', 'last_undegraded': '2023-11-08T19:24:49.588741+0000', 'last_fullsized': '2023-11-08T19:24:49.588741+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:24:49.588741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:24:49.588741+0000', 'last_clean_scrub_stamp': '2023-11-08T19:24:49.588741+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-08T19:24:55.488 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.39 is not active+clean 2023-11-08T19:24:55.488 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-08T19:24:49.588741+0000', 'last_change': '2023-11-08T19:24:49.588741+0000', 'last_active': '2023-11-08T19:24:49.588741+0000', 'last_peered': '2023-11-08T19:24:49.588741+0000', 'last_clean': '2023-11-08T19:24:49.588741+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:24:49.588741+0000', 'last_undegraded': '2023-11-08T19:24:49.588741+0000', 'last_fullsized': '2023-11-08T19:24:49.588741+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:24:49.588741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:24:49.588741+0000', 'last_clean_scrub_stamp': '2023-11-08T19:24:49.588741+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-08T19:24:55.488 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.37 is not active+clean 2023-11-08T19:24:55.489 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-08T19:24:49.588741+0000', 'last_change': '2023-11-08T19:24:49.588741+0000', 'last_active': '2023-11-08T19:24:49.588741+0000', 'last_peered': '2023-11-08T19:24:49.588741+0000', 'last_clean': '2023-11-08T19:24:49.588741+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:24:49.588741+0000', 'last_undegraded': '2023-11-08T19:24:49.588741+0000', 'last_fullsized': '2023-11-08T19:24:49.588741+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:24:49.588741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:24:49.588741+0000', 'last_clean_scrub_stamp': '2023-11-08T19:24:49.588741+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-08T19:24:55.489 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.36 is not active+clean 2023-11-08T19:24:55.489 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-08T19:24:49.588741+0000', 'last_change': '2023-11-08T19:24:49.588741+0000', 'last_active': '2023-11-08T19:24:49.588741+0000', 'last_peered': '2023-11-08T19:24:49.588741+0000', 'last_clean': '2023-11-08T19:24:49.588741+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:24:49.588741+0000', 'last_undegraded': '2023-11-08T19:24:49.588741+0000', 'last_fullsized': '2023-11-08T19:24:49.588741+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:24:49.588741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:24:49.588741+0000', 'last_clean_scrub_stamp': '2023-11-08T19:24:49.588741+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-08T19:24:55.490 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.35 is not active+clean 2023-11-08T19:24:55.490 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-08T19:24:49.588741+0000', 'last_change': '2023-11-08T19:24:49.588741+0000', 'last_active': '2023-11-08T19:24:49.588741+0000', 'last_peered': '2023-11-08T19:24:49.588741+0000', 'last_clean': '2023-11-08T19:24:49.588741+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:24:49.588741+0000', 'last_undegraded': '2023-11-08T19:24:49.588741+0000', 'last_fullsized': '2023-11-08T19:24:49.588741+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:24:49.588741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:24:49.588741+0000', 'last_clean_scrub_stamp': '2023-11-08T19:24:49.588741+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-08T19:24:55.490 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2d is not active+clean 2023-11-08T19:24:55.491 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-08T19:24:49.588741+0000', 'last_change': '2023-11-08T19:24:49.588741+0000', 'last_active': '2023-11-08T19:24:49.588741+0000', 'last_peered': '2023-11-08T19:24:49.588741+0000', 'last_clean': '2023-11-08T19:24:49.588741+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:24:49.588741+0000', 'last_undegraded': '2023-11-08T19:24:49.588741+0000', 'last_fullsized': '2023-11-08T19:24:49.588741+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:24:49.588741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:24:49.588741+0000', 'last_clean_scrub_stamp': '2023-11-08T19:24:49.588741+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-08T19:24:55.491 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1d is not active+clean 2023-11-08T19:24:55.491 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2023-11-08T19:24:48.592048+0000', 'last_change': '2023-11-08T19:24:48.592048+0000', 'last_active': '2023-11-08T19:24:48.584965+0000', 'last_peered': '2023-11-08T19:24:48.584965+0000', 'last_clean': '2023-11-08T19:24:48.584965+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:24:48.592048+0000', 'last_undegraded': '2023-11-08T19:24:48.592048+0000', 'last_fullsized': '2023-11-08T19:24:48.592048+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:24:48.584965+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:24:48.584965+0000', 'last_clean_scrub_stamp': '2023-11-08T19:24:48.584965+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-08T19:24:55.492 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1c is not active+clean 2023-11-08T19:24:55.492 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2023-11-08T19:24:48.591551+0000', 'last_change': '2023-11-08T19:24:48.591551+0000', 'last_active': '2023-11-08T19:24:48.584965+0000', 'last_peered': '2023-11-08T19:24:48.584965+0000', 'last_clean': '2023-11-08T19:24:48.584965+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:24:48.591551+0000', 'last_undegraded': '2023-11-08T19:24:48.591551+0000', 'last_fullsized': '2023-11-08T19:24:48.591551+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:24:48.584965+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:24:48.584965+0000', 'last_clean_scrub_stamp': '2023-11-08T19:24:48.584965+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-08T19:24:55.493 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1a is not active+clean 2023-11-08T19:24:55.493 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2023-11-08T19:24:48.592162+0000', 'last_change': '2023-11-08T19:24:48.592162+0000', 'last_active': '2023-11-08T19:24:48.584965+0000', 'last_peered': '2023-11-08T19:24:48.584965+0000', 'last_clean': '2023-11-08T19:24:48.584965+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:24:48.592162+0000', 'last_undegraded': '2023-11-08T19:24:48.592162+0000', 'last_fullsized': '2023-11-08T19:24:48.592162+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:24:48.584965+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:24:48.584965+0000', 'last_clean_scrub_stamp': '2023-11-08T19:24:48.584965+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-08T19:24:55.493 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.17 is not active+clean 2023-11-08T19:24:55.494 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2023-11-08T19:24:48.593392+0000', 'last_change': '2023-11-08T19:24:48.593392+0000', 'last_active': '2023-11-08T19:24:48.584965+0000', 'last_peered': '2023-11-08T19:24:48.584965+0000', 'last_clean': '2023-11-08T19:24:48.584965+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:24:48.593392+0000', 'last_undegraded': '2023-11-08T19:24:48.593392+0000', 'last_fullsized': '2023-11-08T19:24:48.593392+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:24:48.584965+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:24:48.584965+0000', 'last_clean_scrub_stamp': '2023-11-08T19:24:48.584965+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-08T19:24:55.494 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.14 is not active+clean 2023-11-08T19:24:55.494 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2023-11-08T19:24:48.591494+0000', 'last_change': '2023-11-08T19:24:48.591494+0000', 'last_active': '2023-11-08T19:24:48.584965+0000', 'last_peered': '2023-11-08T19:24:48.584965+0000', 'last_clean': '2023-11-08T19:24:48.584965+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:24:48.591494+0000', 'last_undegraded': '2023-11-08T19:24:48.591494+0000', 'last_fullsized': '2023-11-08T19:24:48.591494+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:24:48.584965+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:24:48.584965+0000', 'last_clean_scrub_stamp': '2023-11-08T19:24:48.584965+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-08T19:24:55.495 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.15 is not active+clean 2023-11-08T19:24:55.495 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-08T19:24:49.588741+0000', 'last_change': '2023-11-08T19:24:49.588741+0000', 'last_active': '2023-11-08T19:24:49.588741+0000', 'last_peered': '2023-11-08T19:24:49.588741+0000', 'last_clean': '2023-11-08T19:24:49.588741+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:24:49.588741+0000', 'last_undegraded': '2023-11-08T19:24:49.588741+0000', 'last_fullsized': '2023-11-08T19:24:49.588741+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:24:49.588741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:24:49.588741+0000', 'last_clean_scrub_stamp': '2023-11-08T19:24:49.588741+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-08T19:24:55.496 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.13 is not active+clean 2023-11-08T19:24:55.496 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-08T19:24:49.588741+0000', 'last_change': '2023-11-08T19:24:49.588741+0000', 'last_active': '2023-11-08T19:24:49.588741+0000', 'last_peered': '2023-11-08T19:24:49.588741+0000', 'last_clean': '2023-11-08T19:24:49.588741+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:24:49.588741+0000', 'last_undegraded': '2023-11-08T19:24:49.588741+0000', 'last_fullsized': '2023-11-08T19:24:49.588741+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:24:49.588741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:24:49.588741+0000', 'last_clean_scrub_stamp': '2023-11-08T19:24:49.588741+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-08T19:24:55.496 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.11 is not active+clean 2023-11-08T19:24:55.497 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2023-11-08T19:24:48.591465+0000', 'last_change': '2023-11-08T19:24:48.591465+0000', 'last_active': '2023-11-08T19:24:48.584965+0000', 'last_peered': '2023-11-08T19:24:48.584965+0000', 'last_clean': '2023-11-08T19:24:48.584965+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:24:48.591465+0000', 'last_undegraded': '2023-11-08T19:24:48.591465+0000', 'last_fullsized': '2023-11-08T19:24:48.591465+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:24:48.584965+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:24:48.584965+0000', 'last_clean_scrub_stamp': '2023-11-08T19:24:48.584965+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-08T19:24:55.497 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.10 is not active+clean 2023-11-08T19:24:55.497 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-08T19:24:49.588741+0000', 'last_change': '2023-11-08T19:24:49.588741+0000', 'last_active': '2023-11-08T19:24:49.588741+0000', 'last_peered': '2023-11-08T19:24:49.588741+0000', 'last_clean': '2023-11-08T19:24:49.588741+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:24:49.588741+0000', 'last_undegraded': '2023-11-08T19:24:49.588741+0000', 'last_fullsized': '2023-11-08T19:24:49.588741+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:24:49.588741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:24:49.588741+0000', 'last_clean_scrub_stamp': '2023-11-08T19:24:49.588741+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-08T19:24:55.498 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.11 is not active+clean 2023-11-08T19:24:55.498 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-08T19:24:49.588741+0000', 'last_change': '2023-11-08T19:24:49.588741+0000', 'last_active': '2023-11-08T19:24:49.588741+0000', 'last_peered': '2023-11-08T19:24:49.588741+0000', 'last_clean': '2023-11-08T19:24:49.588741+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:24:49.588741+0000', 'last_undegraded': '2023-11-08T19:24:49.588741+0000', 'last_fullsized': '2023-11-08T19:24:49.588741+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:24:49.588741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:24:49.588741+0000', 'last_clean_scrub_stamp': '2023-11-08T19:24:49.588741+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-08T19:24:55.498 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.e is not active+clean 2023-11-08T19:24:55.499 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-08T19:24:49.588741+0000', 'last_change': '2023-11-08T19:24:49.588741+0000', 'last_active': '2023-11-08T19:24:49.588741+0000', 'last_peered': '2023-11-08T19:24:49.588741+0000', 'last_clean': '2023-11-08T19:24:49.588741+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:24:49.588741+0000', 'last_undegraded': '2023-11-08T19:24:49.588741+0000', 'last_fullsized': '2023-11-08T19:24:49.588741+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:24:49.588741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:24:49.588741+0000', 'last_clean_scrub_stamp': '2023-11-08T19:24:49.588741+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-08T19:24:55.499 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.0 is not active+clean 2023-11-08T19:24:55.499 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2023-11-08T19:24:48.591491+0000', 'last_change': '2023-11-08T19:24:48.591491+0000', 'last_active': '2023-11-08T19:24:48.584965+0000', 'last_peered': '2023-11-08T19:24:48.584965+0000', 'last_clean': '2023-11-08T19:24:48.584965+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:24:48.591491+0000', 'last_undegraded': '2023-11-08T19:24:48.591491+0000', 'last_fullsized': '2023-11-08T19:24:48.591491+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:24:48.584965+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:24:48.584965+0000', 'last_clean_scrub_stamp': '2023-11-08T19:24:48.584965+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-08T19:24:55.500 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.5 is not active+clean 2023-11-08T19:24:55.500 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2023-11-08T19:24:48.591465+0000', 'last_change': '2023-11-08T19:24:48.591465+0000', 'last_active': '2023-11-08T19:24:48.584965+0000', 'last_peered': '2023-11-08T19:24:48.584965+0000', 'last_clean': '2023-11-08T19:24:48.584965+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:24:48.591465+0000', 'last_undegraded': '2023-11-08T19:24:48.591465+0000', 'last_fullsized': '2023-11-08T19:24:48.591465+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:24:48.584965+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:24:48.584965+0000', 'last_clean_scrub_stamp': '2023-11-08T19:24:48.584965+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-08T19:24:55.500 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.7 is not active+clean 2023-11-08T19:24:55.501 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2023-11-08T19:24:48.592846+0000', 'last_change': '2023-11-08T19:24:48.592846+0000', 'last_active': '2023-11-08T19:24:48.584965+0000', 'last_peered': '2023-11-08T19:24:48.584965+0000', 'last_clean': '2023-11-08T19:24:48.584965+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:24:48.592846+0000', 'last_undegraded': '2023-11-08T19:24:48.592846+0000', 'last_fullsized': '2023-11-08T19:24:48.592846+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:24:48.584965+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:24:48.584965+0000', 'last_clean_scrub_stamp': '2023-11-08T19:24:48.584965+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-08T19:24:55.501 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.8 is not active+clean 2023-11-08T19:24:55.501 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2023-11-08T19:24:48.592059+0000', 'last_change': '2023-11-08T19:24:48.592059+0000', 'last_active': '2023-11-08T19:24:48.584965+0000', 'last_peered': '2023-11-08T19:24:48.584965+0000', 'last_clean': '2023-11-08T19:24:48.584965+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:24:48.592059+0000', 'last_undegraded': '2023-11-08T19:24:48.592059+0000', 'last_fullsized': '2023-11-08T19:24:48.592059+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:24:48.584965+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:24:48.584965+0000', 'last_clean_scrub_stamp': '2023-11-08T19:24:48.584965+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-08T19:24:55.502 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.b is not active+clean 2023-11-08T19:24:55.502 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2023-11-08T19:24:48.592103+0000', 'last_change': '2023-11-08T19:24:48.592103+0000', 'last_active': '2023-11-08T19:24:48.584965+0000', 'last_peered': '2023-11-08T19:24:48.584965+0000', 'last_clean': '2023-11-08T19:24:48.584965+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:24:48.592103+0000', 'last_undegraded': '2023-11-08T19:24:48.592103+0000', 'last_fullsized': '2023-11-08T19:24:48.592103+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:24:48.584965+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:24:48.584965+0000', 'last_clean_scrub_stamp': '2023-11-08T19:24:48.584965+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-08T19:24:55.503 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.a is not active+clean 2023-11-08T19:24:55.503 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-08T19:24:49.588741+0000', 'last_change': '2023-11-08T19:24:49.588741+0000', 'last_active': '2023-11-08T19:24:49.588741+0000', 'last_peered': '2023-11-08T19:24:49.588741+0000', 'last_clean': '2023-11-08T19:24:49.588741+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:24:49.588741+0000', 'last_undegraded': '2023-11-08T19:24:49.588741+0000', 'last_fullsized': '2023-11-08T19:24:49.588741+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:24:49.588741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:24:49.588741+0000', 'last_clean_scrub_stamp': '2023-11-08T19:24:49.588741+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-08T19:24:55.503 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.d is not active+clean 2023-11-08T19:24:55.504 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-08T19:24:49.588741+0000', 'last_change': '2023-11-08T19:24:49.588741+0000', 'last_active': '2023-11-08T19:24:49.588741+0000', 'last_peered': '2023-11-08T19:24:49.588741+0000', 'last_clean': '2023-11-08T19:24:49.588741+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:24:49.588741+0000', 'last_undegraded': '2023-11-08T19:24:49.588741+0000', 'last_fullsized': '2023-11-08T19:24:49.588741+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:24:49.588741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:24:49.588741+0000', 'last_clean_scrub_stamp': '2023-11-08T19:24:49.588741+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-08T19:24:55.504 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.f is not active+clean 2023-11-08T19:24:55.504 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-08T19:24:49.588741+0000', 'last_change': '2023-11-08T19:24:49.588741+0000', 'last_active': '2023-11-08T19:24:49.588741+0000', 'last_peered': '2023-11-08T19:24:49.588741+0000', 'last_clean': '2023-11-08T19:24:49.588741+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:24:49.588741+0000', 'last_undegraded': '2023-11-08T19:24:49.588741+0000', 'last_fullsized': '2023-11-08T19:24:49.588741+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:24:49.588741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:24:49.588741+0000', 'last_clean_scrub_stamp': '2023-11-08T19:24:49.588741+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-08T19:24:55.505 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.20 is not active+clean 2023-11-08T19:24:55.505 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-08T19:24:49.588741+0000', 'last_change': '2023-11-08T19:24:49.588741+0000', 'last_active': '2023-11-08T19:24:49.588741+0000', 'last_peered': '2023-11-08T19:24:49.588741+0000', 'last_clean': '2023-11-08T19:24:49.588741+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:24:49.588741+0000', 'last_undegraded': '2023-11-08T19:24:49.588741+0000', 'last_fullsized': '2023-11-08T19:24:49.588741+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:24:49.588741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:24:49.588741+0000', 'last_clean_scrub_stamp': '2023-11-08T19:24:49.588741+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-08T19:24:55.506 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.26 is not active+clean 2023-11-08T19:24:55.506 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-08T19:24:49.588741+0000', 'last_change': '2023-11-08T19:24:49.588741+0000', 'last_active': '2023-11-08T19:24:49.588741+0000', 'last_peered': '2023-11-08T19:24:49.588741+0000', 'last_clean': '2023-11-08T19:24:49.588741+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:24:49.588741+0000', 'last_undegraded': '2023-11-08T19:24:49.588741+0000', 'last_fullsized': '2023-11-08T19:24:49.588741+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:24:49.588741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:24:49.588741+0000', 'last_clean_scrub_stamp': '2023-11-08T19:24:49.588741+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-08T19:24:55.506 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2c is not active+clean 2023-11-08T19:24:55.507 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-08T19:24:49.588741+0000', 'last_change': '2023-11-08T19:24:49.588741+0000', 'last_active': '2023-11-08T19:24:49.588741+0000', 'last_peered': '2023-11-08T19:24:49.588741+0000', 'last_clean': '2023-11-08T19:24:49.588741+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:24:49.588741+0000', 'last_undegraded': '2023-11-08T19:24:49.588741+0000', 'last_fullsized': '2023-11-08T19:24:49.588741+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:24:49.588741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:24:49.588741+0000', 'last_clean_scrub_stamp': '2023-11-08T19:24:49.588741+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-08T19:24:55.507 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-11-08T19:24:55.639 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.638+0000 7f84f586c700 1 Processor -- start 2023-11-08T19:24:55.640 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.638+0000 7f84f586c700 1 -- start start 2023-11-08T19:24:55.642 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.639+0000 7f84f586c700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f84f015db10 0x7f84f00d19b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:55.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.639+0000 7f84f586c700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f84f00d1ef0 0x7f84f00d0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:55.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.639+0000 7f84f586c700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f84f00d08a0 0x7f84f00b5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:55.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.639+0000 7f84f586c700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84f00d0d30 con 0x7f84f00d08a0 2023-11-08T19:24:55.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.639+0000 7f84f586c700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84f00b94d0 con 0x7f84f015db10 2023-11-08T19:24:55.644 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.639+0000 7f84f586c700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84f00d2520 con 0x7f84f00d1ef0 2023-11-08T19:24:55.644 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.639+0000 7f84ef7fe700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f84f00d08a0 0x7f84f00b5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:55.644 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.639+0000 7f84ef7fe700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f84f00d08a0 0x7f84f00b5410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48700/0 (socket says 172.21.15.143:48700) 2023-11-08T19:24:55.644 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.639+0000 7f84ef7fe700 1 -- 172.21.15.143:0/3459352825 learned_addr learned my addr 172.21.15.143:0/3459352825 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:24:55.644 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.639+0000 7f84eeffd700 1 --2- 172.21.15.143:0/3459352825 >> v2:172.21.15.174:3300/0 conn(0x7f84f015db10 0x7f84f00d19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:55.645 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.639+0000 7f84ee7fc700 1 --2- 172.21.15.143:0/3459352825 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f84f00d1ef0 0x7f84f00d0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:55.645 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.639+0000 7f84ef7fe700 1 -- 172.21.15.143:0/3459352825 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f84f00d1ef0 msgr2=0x7f84f00d0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:24:55.645 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.639+0000 7f84ef7fe700 1 --2- 172.21.15.143:0/3459352825 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f84f00d1ef0 0x7f84f00d0360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:55.645 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.639+0000 7f84ef7fe700 1 -- 172.21.15.143:0/3459352825 >> v2:172.21.15.174:3300/0 conn(0x7f84f015db10 msgr2=0x7f84f00d19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:24:55.645 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.639+0000 7f84ef7fe700 1 --2- 172.21.15.143:0/3459352825 >> v2:172.21.15.174:3300/0 conn(0x7f84f015db10 0x7f84f00d19b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:55.646 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.639+0000 7f84ef7fe700 1 -- 172.21.15.143:0/3459352825 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84f00d2800 con 0x7f84f00d08a0 2023-11-08T19:24:55.646 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.639+0000 7f84ee7fc700 1 --2- 172.21.15.143:0/3459352825 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f84f00d1ef0 0x7f84f00d0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:24:55.646 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.639+0000 7f84ef7fe700 1 --2- 172.21.15.143:0/3459352825 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f84f00d08a0 0x7f84f00b5410 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f84f008bd10 tx=0x7f84e800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec69e9ac0934545b server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:55.646 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.639+0000 7f84edffb700 1 -- 172.21.15.143:0/3459352825 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e800e070 con 0x7f84f00d08a0 2023-11-08T19:24:55.646 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.640+0000 7f84edffb700 1 -- 172.21.15.143:0/3459352825 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84e800b910 con 0x7f84f00d08a0 2023-11-08T19:24:55.647 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.640+0000 7f84edffb700 1 -- 172.21.15.143:0/3459352825 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e800bcc0 con 0x7f84f00d08a0 2023-11-08T19:24:55.647 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.640+0000 7f84f586c700 1 -- 172.21.15.143:0/3459352825 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f84f00d08a0 msgr2=0x7f84f00b5410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:55.647 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.640+0000 7f84f586c700 1 --2- 172.21.15.143:0/3459352825 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f84f00d08a0 0x7f84f00b5410 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f84f008bd10 tx=0x7f84e800b330 comp rx=0 tx=0).stop 2023-11-08T19:24:55.647 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.640+0000 7f84f586c700 1 -- 172.21.15.143:0/3459352825 shutdown_connections 2023-11-08T19:24:55.647 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.640+0000 7f84f586c700 1 --2- 172.21.15.143:0/3459352825 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f84f00d08a0 0x7f84f00b5410 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:55.648 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.640+0000 7f84f586c700 1 --2- 172.21.15.143:0/3459352825 >> v2:172.21.15.174:3300/0 conn(0x7f84f015db10 0x7f84f00d19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:55.648 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.640+0000 7f84f586c700 1 --2- 172.21.15.143:0/3459352825 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f84f00d1ef0 0x7f84f00d0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:55.648 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.640+0000 7f84f586c700 1 -- 172.21.15.143:0/3459352825 >> 172.21.15.143:0/3459352825 conn(0x7f84f006be10 msgr2=0x7f84f006c1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:24:55.648 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.640+0000 7f84f586c700 1 -- 172.21.15.143:0/3459352825 shutdown_connections 2023-11-08T19:24:55.648 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.640+0000 7f84f586c700 1 -- 172.21.15.143:0/3459352825 wait complete. 2023-11-08T19:24:55.649 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.641+0000 7f84f586c700 1 Processor -- start 2023-11-08T19:24:55.649 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.642+0000 7f84f586c700 1 -- start start 2023-11-08T19:24:55.651 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.642+0000 7f84f586c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f84f00d08a0 0x7f84f00caa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:55.652 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.642+0000 7f84f586c700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f84f00d1ef0 0x7f84f00b4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:55.652 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.642+0000 7f84f586c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f84f015db10 0x7f84f015f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:55.652 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.642+0000 7f84f586c700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84f015a740 con 0x7f84f015db10 2023-11-08T19:24:55.652 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.642+0000 7f84f586c700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84f00b29a0 con 0x7f84f00d08a0 2023-11-08T19:24:55.652 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.642+0000 7f84f586c700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84f00cd4a0 con 0x7f84f00d1ef0 2023-11-08T19:24:55.653 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.642+0000 7f84ef7fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f84f015db10 0x7f84f015f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:55.653 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.642+0000 7f84ef7fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f84f015db10 0x7f84f015f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48704/0 (socket says 172.21.15.143:48704) 2023-11-08T19:24:55.653 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.642+0000 7f84ef7fe700 1 -- 172.21.15.143:0/2881813287 learned_addr learned my addr 172.21.15.143:0/2881813287 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:24:55.653 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.643+0000 7f84eeffd700 1 --2- 172.21.15.143:0/2881813287 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f84f00d08a0 0x7f84f00caa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:55.653 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.643+0000 7f84ee7fc700 1 --2- 172.21.15.143:0/2881813287 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f84f00d1ef0 0x7f84f00b4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:55.654 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.643+0000 7f84ef7fe700 1 -- 172.21.15.143:0/2881813287 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f84f00d1ef0 msgr2=0x7f84f00b4780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:55.654 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.643+0000 7f84ef7fe700 1 --2- 172.21.15.143:0/2881813287 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f84f00d1ef0 0x7f84f00b4780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:55.654 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.643+0000 7f84ef7fe700 1 -- 172.21.15.143:0/2881813287 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f84f00d08a0 msgr2=0x7f84f00caa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:55.654 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.643+0000 7f84ef7fe700 1 --2- 172.21.15.143:0/2881813287 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f84f00d08a0 0x7f84f00caa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:55.655 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.643+0000 7f84ef7fe700 1 -- 172.21.15.143:0/2881813287 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84e8009b30 con 0x7f84f015db10 2023-11-08T19:24:55.656 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.643+0000 7f84ef7fe700 1 --2- 172.21.15.143:0/2881813287 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f84f015db10 0x7f84f015f4e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f84f006d370 tx=0x7f84e8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:55.656 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.643+0000 7f84d7fff700 1 -- 172.21.15.143:0/2881813287 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e80099b0 con 0x7f84f015db10 2023-11-08T19:24:55.656 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.643+0000 7f84d7fff700 1 -- 172.21.15.143:0/2881813287 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84e8004250 con 0x7f84f015db10 2023-11-08T19:24:55.656 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.643+0000 7f84f586c700 1 -- 172.21.15.143:0/2881813287 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84f00cd780 con 0x7f84f015db10 2023-11-08T19:24:55.656 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.643+0000 7f84d7fff700 1 -- 172.21.15.143:0/2881813287 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e8004600 con 0x7f84f015db10 2023-11-08T19:24:55.657 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.643+0000 7f84f586c700 1 -- 172.21.15.143:0/2881813287 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84f00b4cc0 con 0x7f84f015db10 2023-11-08T19:24:55.657 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.644+0000 7f84f586c700 1 -- 172.21.15.143:0/2881813287 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84f00cd9c0 con 0x7f84f015db10 2023-11-08T19:24:55.657 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.644+0000 7f84d7fff700 1 -- 172.21.15.143:0/2881813287 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f84e8004760 con 0x7f84f015db10 2023-11-08T19:24:55.657 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.645+0000 7f84d7fff700 1 --2- 172.21.15.143:0/2881813287 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f84dc06af00 0x7f84dc06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:55.657 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.645+0000 7f84d7fff700 1 -- 172.21.15.143:0/2881813287 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f84e808ad50 con 0x7f84f015db10 2023-11-08T19:24:55.658 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.645+0000 7f84eeffd700 1 --2- 172.21.15.143:0/2881813287 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f84dc06af00 0x7f84dc06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:55.658 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.645+0000 7f84eeffd700 1 --2- 172.21.15.143:0/2881813287 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f84dc06af00 0x7f84dc06d3c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f84f015c600 tx=0x7f84e4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:55.658 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.649+0000 7f84d7fff700 1 -- 172.21.15.143:0/2881813287 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f84f00cd9c0 con 0x7f84f015db10 2023-11-08T19:24:55.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.920+0000 7f84f586c700 1 -- 172.21.15.143:0/2881813287 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f84f00b32f0 con 0x7f84f015db10 2023-11-08T19:24:55.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.921+0000 7f84d7fff700 1 -- 172.21.15.143:0/2881813287 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1172 (secure 0 0 0) 0x7f84e80204b0 con 0x7f84f015db10 2023-11-08T19:24:55.923 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:24:55.923 INFO:teuthology.orchestra.run.smithi143.stdout:{"fsid":"968483af-77fc-4068-ad7c-5c2a1a4613d7","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":23,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":20,"num_osds":8,"num_up_osds":8,"osd_up_since":1699471483,"num_in_osds":8,"osd_in_since":1699471475,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":68},{"state_name":"unknown","count":18},{"state_name":"creating+peering","count":11}],"num_pgs":97,"num_pools":3,"num_objects":15,"data_bytes":591166,"bytes_used":224157696,"bytes_avail":772869955584,"bytes_total":773094113280,"unknown_pgs_ratio":0.18556700646877289,"inactive_pgs_ratio":0.11340206116437912,"write_bytes_sec":2290,"read_op_per_sec":0,"write_op_per_sec":5},"fsmap":{"epoch":8,"id":1,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":1,"rank":0,"name":"b","status":"up:active","gid":4282}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":1,"modified":"2023-11-08T19:24:32.827594+0000","services":{}},"progress_events":{}} 2023-11-08T19:24:55.925 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.923+0000 7f84f586c700 1 -- 172.21.15.143:0/2881813287 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f84dc06af00 msgr2=0x7f84dc06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:55.925 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.923+0000 7f84f586c700 1 --2- 172.21.15.143:0/2881813287 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f84dc06af00 0x7f84dc06d3c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f84f015c600 tx=0x7f84e4008040 comp rx=0 tx=0).stop 2023-11-08T19:24:55.925 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.923+0000 7f84f586c700 1 -- 172.21.15.143:0/2881813287 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f84f015db10 msgr2=0x7f84f015f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:55.926 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.923+0000 7f84f586c700 1 --2- 172.21.15.143:0/2881813287 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f84f015db10 0x7f84f015f4e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f84f006d370 tx=0x7f84e8004ad0 comp rx=0 tx=0).stop 2023-11-08T19:24:55.926 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.923+0000 7f84f586c700 1 -- 172.21.15.143:0/2881813287 shutdown_connections 2023-11-08T19:24:55.926 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.923+0000 7f84f586c700 1 --2- 172.21.15.143:0/2881813287 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f84dc06af00 0x7f84dc06d3c0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:55.926 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.923+0000 7f84f586c700 1 --2- 172.21.15.143:0/2881813287 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f84f015db10 0x7f84f015f4e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:55.926 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.923+0000 7f84f586c700 1 --2- 172.21.15.143:0/2881813287 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f84f00d08a0 0x7f84f00caa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:55.927 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.923+0000 7f84f586c700 1 --2- 172.21.15.143:0/2881813287 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f84f00d1ef0 0x7f84f00b4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:55.927 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.924+0000 7f84f586c700 1 -- 172.21.15.143:0/2881813287 >> 172.21.15.143:0/2881813287 conn(0x7f84f006be10 msgr2=0x7f84f00b8550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:24:55.927 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.924+0000 7f84f586c700 1 -- 172.21.15.143:0/2881813287 shutdown_connections 2023-11-08T19:24:55.927 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:55.924+0000 7f84f586c700 1 -- 172.21.15.143:0/2881813287 wait complete. 2023-11-08T19:24:55.937 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-11-08T19:24:55.937 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-08T19:24:56.058 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:56.057+0000 7f52e60e9700 1 Processor -- start 2023-11-08T19:24:56.059 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:56.057+0000 7f52e60e9700 1 -- start start 2023-11-08T19:24:56.064 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:56.057+0000 7f52e60e9700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f52e006b320 0x7f52e0168c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:56.065 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:56.057+0000 7f52e60e9700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52e0161df0 con 0x7f52e015dd00 2023-11-08T19:24:56.065 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:56.057+0000 7f52e60e9700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52e01620d0 con 0x7f52e01618f0 2023-11-08T19:24:56.065 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:56.057+0000 7f52e60e9700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52e00cad50 con 0x7f52e006b320 2023-11-08T19:24:56.065 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:56.058+0000 7f52df7fe700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f52e015dd00 0x7f52e00cca70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:44864/0 (socket says 172.21.15.143:44864) 2023-11-08T19:24:56.065 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:56.058+0000 7f52df7fe700 1 -- 172.21.15.143:0/344371330 learned_addr learned my addr 172.21.15.143:0/344371330 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:24:56.066 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:56.058+0000 7f52dffff700 1 --2- 172.21.15.143:0/344371330 >> v2:172.21.15.174:3300/0 conn(0x7f52e006b320 0x7f52e0168c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:56.066 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:56.058+0000 7f52de7fc700 1 -- 172.21.15.143:0/344371330 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2084266903 0 0) 0x7f52e0161df0 con 0x7f52e015dd00 2023-11-08T19:24:56.066 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:56.058+0000 7f52de7fc700 1 -- 172.21.15.143:0/344371330 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52c80035b0 con 0x7f52e015dd00 2023-11-08T19:24:56.066 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:56.058+0000 7f52de7fc700 1 -- 172.21.15.143:0/344371330 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1532630853 0 0) 0x7f52c80035b0 con 0x7f52e015dd00 2023-11-08T19:24:56.066 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:56.058+0000 7f52de7fc700 1 -- 172.21.15.143:0/344371330 >> v2:172.21.15.174:3300/0 conn(0x7f52e006b320 msgr2=0x7f52e0168c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:24:56.067 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:56.058+0000 7f52de7fc700 1 --2- 172.21.15.143:0/344371330 >> v2:172.21.15.174:3300/0 conn(0x7f52e006b320 0x7f52e0168c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:56.067 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:56.058+0000 7f52de7fc700 1 -- 172.21.15.143:0/344371330 >> v1:172.21.15.174:6789/0 conn(0x7f52e01618f0 legacy=0x7f52e006ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:56.067 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:56.058+0000 7f52de7fc700 1 -- 172.21.15.143:0/344371330 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52e01623b0 con 0x7f52e015dd00 2023-11-08T19:24:56.067 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:56.058+0000 7f52de7fc700 1 -- 172.21.15.143:0/344371330 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4147756303 0 0) 0x7f52d00017f0 con 0x7f52e015dd00 2023-11-08T19:24:56.067 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:56.058+0000 7f52de7fc700 1 -- 172.21.15.143:0/344371330 >> v1:172.21.15.143:6789/0 conn(0x7f52e015dd00 legacy=0x7f52e00cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:56.068 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:56.058+0000 7f52de7fc700 1 --2- 172.21.15.143:0/344371330 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f52c8003a00 0x7f52c8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:56.068 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:56.058+0000 7f52de7fc700 1 --2- 172.21.15.143:0/344371330 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f52c8004850 0x7f52c8004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:56.068 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:56.058+0000 7f52de7fc700 1 --2- 172.21.15.143:0/344371330 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f52c8005210 0x7f52c800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:56.068 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:56.058+0000 7f52de7fc700 1 -- 172.21.15.143:0/344371330 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52c800c080 con 0x7f52c8004850 2023-11-08T19:24:56.068 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:56.058+0000 7f52de7fc700 1 -- 172.21.15.143:0/344371330 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52c800c1f0 con 0x7f52c8005210 2023-11-08T19:24:56.068 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:56.058+0000 7f52de7fc700 1 -- 172.21.15.143:0/344371330 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52c800c3b0 con 0x7f52c8003a00 2023-11-08T19:24:56.069 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:56.058+0000 7f52deffd700 1 --2- 172.21.15.143:0/344371330 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f52c8004850 0x7f52c8004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:56.069 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:56.058+0000 7f52dffff700 1 --2- 172.21.15.143:0/344371330 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f52c8005210 0x7f52c800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:56.069 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:56.058+0000 7f52df7fe700 1 --2- 172.21.15.143:0/344371330 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f52c8003a00 0x7f52c8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:56.069 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:56.058+0000 7f52deffd700 1 -- 172.21.15.143:0/344371330 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f52c8003a00 msgr2=0x7f52c8003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:24:56.069 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:56.058+0000 7f52deffd700 1 --2- 172.21.15.143:0/344371330 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f52c8003a00 0x7f52c8003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:56.069 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:56.058+0000 7f52deffd700 1 -- 172.21.15.143:0/344371330 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f52c8005210 msgr2=0x7f52c800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:24:56.070 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:56.058+0000 7f52deffd700 1 --2- 172.21.15.143:0/344371330 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f52c8005210 0x7f52c800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:56.070 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:56.058+0000 7f52deffd700 1 -- 172.21.15.143:0/344371330 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52c80035b0 con 0x7f52c8004850 2023-11-08T19:24:56.070 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:56.058+0000 7f52dffff700 1 -- 172.21.15.143:0/344371330 reap_dead start 2023-11-08T19:24:56.070 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:56.058+0000 7f52df7fe700 1 --2- 172.21.15.143:0/344371330 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f52c8003a00 0x7f52c8003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:24:56.070 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:56.059+0000 7f52deffd700 1 --2- 172.21.15.143:0/344371330 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f52c8004850 0x7f52c8004cd0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f52d40017c0 tx=0x7f52c8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=556bbdf2d3599f76 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:56.071 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:56.059+0000 7f52de7fc700 1 -- 172.21.15.143:0/344371330 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d40052d0 con 0x7f52c8004850 2023-11-08T19:24:56.071 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:56.059+0000 7f52de7fc700 1 -- 172.21.15.143:0/344371330 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52d4005450 con 0x7f52c8004850 2023-11-08T19:24:56.071 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:56.059+0000 7f52de7fc700 1 -- 172.21.15.143:0/344371330 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d40038c0 con 0x7f52c8004850 2023-11-08T19:24:56.071 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:56.059+0000 7f52e60e9700 1 -- 172.21.15.143:0/344371330 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f52c8004850 msgr2=0x7f52c8004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:56.071 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:56.059+0000 7f52e60e9700 1 --2- 172.21.15.143:0/344371330 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f52c8004850 0x7f52c8004cd0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f52d40017c0 tx=0x7f52c8005880 comp rx=0 tx=0).stop 2023-11-08T19:24:56.072 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:56.059+0000 7f52e60e9700 1 -- 172.21.15.143:0/344371330 shutdown_connections 2023-11-08T19:24:56.072 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:56.059+0000 7f52e60e9700 1 --2- 172.21.15.143:0/344371330 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f52c8004850 0x7f52c8004cd0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:56.072 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:56.059+0000 7f52e60e9700 1 -- 172.21.15.143:0/344371330 >> 172.21.15.143:0/344371330 conn(0x7f52e0099ab0 msgr2=0x7f52e00d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:24:56.072 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:56.059+0000 7f52e60e9700 1 -- 172.21.15.143:0/344371330 shutdown_connections 2023-11-08T19:24:56.072 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:56.059+0000 7f52e60e9700 1 -- 172.21.15.143:0/344371330 wait complete. 2023-11-08T19:24:56.073 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:56.061+0000 7f52e60e9700 1 Processor -- start 2023-11-08T19:24:56.073 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:56.061+0000 7f52e60e9700 1 -- start start 2023-11-08T19:24:56.073 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:56.061+0000 7f52e60e9700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f52c8004850 0x7f52e0151d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:56.073 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:56.061+0000 7f52e60e9700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f52e00b1c80 0x7f52e0155e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:56.073 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:56.062+0000 7f52e60e9700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f52e0156370 0x7f52e00c5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:56.073 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:56.062+0000 7f52e60e9700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52e00c4230 con 0x7f52c8004850 2023-11-08T19:24:56.074 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:56.062+0000 7f52e60e9700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52e00c4db0 con 0x7f52e0156370 2023-11-08T19:24:56.074 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:56.062+0000 7f52e60e9700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52e00c4480 con 0x7f52e00b1c80 2023-11-08T19:24:56.074 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:56.062+0000 7f52df7fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f52c8004850 0x7f52e0151d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:56.074 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:56.062+0000 7f52df7fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f52c8004850 0x7f52e0151d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48722/0 (socket says 172.21.15.143:48722) 2023-11-08T19:24:56.074 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:56.062+0000 7f52df7fe700 1 -- 172.21.15.143:0/913329695 learned_addr learned my addr 172.21.15.143:0/913329695 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:24:56.075 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:56.062+0000 7f52dffff700 1 --2- 172.21.15.143:0/913329695 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f52e0156370 0x7f52e00c5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:56.075 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:56.062+0000 7f52deffd700 1 --2- 172.21.15.143:0/913329695 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f52e00b1c80 0x7f52e0155e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:56.075 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:56.062+0000 7f52df7fe700 1 -- 172.21.15.143:0/913329695 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f52e00b1c80 msgr2=0x7f52e0155e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:56.075 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:56.062+0000 7f52df7fe700 1 --2- 172.21.15.143:0/913329695 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f52e00b1c80 0x7f52e0155e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:56.075 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:56.062+0000 7f52df7fe700 1 -- 172.21.15.143:0/913329695 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f52e0156370 msgr2=0x7f52e00c5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:56.076 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:56.062+0000 7f52df7fe700 1 --2- 172.21.15.143:0/913329695 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f52e0156370 0x7f52e00c5240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:56.076 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:56.062+0000 7f52df7fe700 1 -- 172.21.15.143:0/913329695 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52e015e260 con 0x7f52c8004850 2023-11-08T19:24:56.076 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:56.062+0000 7f52dffff700 1 --2- 172.21.15.143:0/913329695 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f52e0156370 0x7f52e00c5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:24:56.076 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:56.062+0000 7f52df7fe700 1 --2- 172.21.15.143:0/913329695 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f52c8004850 0x7f52e0151d60 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f52d0001e10 tx=0x7f52d000eb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:56.076 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:56.062+0000 7f52c7fff700 1 -- 172.21.15.143:0/913329695 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d00072e0 con 0x7f52c8004850 2023-11-08T19:24:56.077 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:56.062+0000 7f52c7fff700 1 -- 172.21.15.143:0/913329695 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52d0007440 con 0x7f52c8004850 2023-11-08T19:24:56.077 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:56.062+0000 7f52c7fff700 1 -- 172.21.15.143:0/913329695 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d0005830 con 0x7f52c8004850 2023-11-08T19:24:56.077 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:56.062+0000 7f52e60e9700 1 -- 172.21.15.143:0/913329695 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52e00c4760 con 0x7f52c8004850 2023-11-08T19:24:56.077 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:56.062+0000 7f52e60e9700 1 -- 172.21.15.143:0/913329695 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52e00c6d30 con 0x7f52c8004850 2023-11-08T19:24:56.077 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:56.063+0000 7f52e60e9700 1 -- 172.21.15.143:0/913329695 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52e00c5090 con 0x7f52c8004850 2023-11-08T19:24:56.078 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:56.064+0000 7f52c7fff700 1 -- 172.21.15.143:0/913329695 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f52d0005990 con 0x7f52c8004850 2023-11-08T19:24:56.078 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:56.064+0000 7f52c7fff700 1 --2- 172.21.15.143:0/913329695 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f52cc06af00 0x7f52cc06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:56.079 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:56.064+0000 7f52c7fff700 1 -- 172.21.15.143:0/913329695 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f52d0089530 con 0x7f52c8004850 2023-11-08T19:24:56.079 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:56.064+0000 7f52deffd700 1 --2- 172.21.15.143:0/913329695 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f52cc06af00 0x7f52cc06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:56.079 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:56.065+0000 7f52deffd700 1 --2- 172.21.15.143:0/913329695 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f52cc06af00 0x7f52cc06d3c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f52d4000c00 tx=0x7f52d4009030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:56.079 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:56.068+0000 7f52c7fff700 1 -- 172.21.15.143:0/913329695 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f52d0057fe0 con 0x7f52c8004850 2023-11-08T19:24:56.253 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:56.250+0000 7f52e60e9700 1 -- 172.21.15.143:0/913329695 --> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f52e015e0e0 con 0x7f52cc06af00 2023-11-08T19:24:56.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:56.258+0000 7f52c7fff700 1 -- 172.21.15.143:0/913329695 <== mgr.4099 v2:172.21.15.174:6832/4037280178 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+261184 (secure 0 0 0) 0x7f52d40051d0 con 0x7f52cc06af00 2023-11-08T19:24:56.262 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:24:56.264 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:56.261+0000 7f52e60e9700 1 -- 172.21.15.143:0/913329695 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f52cc06af00 msgr2=0x7f52cc06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:56.264 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:56.261+0000 7f52e60e9700 1 --2- 172.21.15.143:0/913329695 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f52cc06af00 0x7f52cc06d3c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f52d4000c00 tx=0x7f52d4009030 comp rx=0 tx=0).stop 2023-11-08T19:24:56.265 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:56.261+0000 7f52e60e9700 1 -- 172.21.15.143:0/913329695 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f52c8004850 msgr2=0x7f52e0151d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:56.265 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:56.261+0000 7f52e60e9700 1 --2- 172.21.15.143:0/913329695 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f52c8004850 0x7f52e0151d60 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f52d0001e10 tx=0x7f52d000eb10 comp rx=0 tx=0).stop 2023-11-08T19:24:56.265 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:56.261+0000 7f52e60e9700 1 -- 172.21.15.143:0/913329695 shutdown_connections 2023-11-08T19:24:56.265 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:56.261+0000 7f52e60e9700 1 --2- 172.21.15.143:0/913329695 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f52cc06af00 0x7f52cc06d3c0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:56.265 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:56.261+0000 7f52e60e9700 1 --2- 172.21.15.143:0/913329695 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f52e0156370 0x7f52e00c5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:56.266 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:56.261+0000 7f52e60e9700 1 --2- 172.21.15.143:0/913329695 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f52c8004850 0x7f52e0151d60 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:56.266 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:56.261+0000 7f52e60e9700 1 --2- 172.21.15.143:0/913329695 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f52e00b1c80 0x7f52e0155e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:56.266 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:56.261+0000 7f52e60e9700 1 -- 172.21.15.143:0/913329695 >> 172.21.15.143:0/913329695 conn(0x7f52e0099ab0 msgr2=0x7f52e015b800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:24:56.266 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:56.261+0000 7f52e60e9700 1 -- 172.21.15.143:0/913329695 shutdown_connections 2023-11-08T19:24:56.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:56.261+0000 7f52e60e9700 1 -- 172.21.15.143:0/913329695 wait complete. 2023-11-08T19:24:56.267 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2023-11-08T19:24:56.274 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":18,"stamp":"2023-11-08T19:24:54.509481+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591166,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":187,"num_write_kb":2427,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":140,"ondisk_log_size":140,"up":158,"acting":158,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":143,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218904,"kb_used_data":2724,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755816,"statfs":{"total":773094113280,"available":772869955584,"internally_reserved":0,"allocated":2789376,"data_stored":1705642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":798,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.917376"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:24:49.588741+0000","last_change":"2023-11-08T19:24:49.588741+0000","last_active":"2023-11-08T19:24:49.588741+0000","last_peered":"2023-11-08T19:24:49.588741+0000","last_clean":"2023-11-08T19:24:49.588741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:24:49.588741+0000","last_undegraded":"2023-11-08T19:24:49.588741+0000","last_fullsized":"2023-11-08T19:24:49.588741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.805606+0000","last_change":"2023-11-08T19:24:50.623985+0000","last_active":"2023-11-08T19:24:52.805606+0000","last_peered":"2023-11-08T19:24:52.805606+0000","last_clean":"2023-11-08T19:24:52.805606+0000","last_became_active":"2023-11-08T19:24:50.623553+0000","last_became_peered":"2023-11-08T19:24:50.623553+0000","last_unstale":"2023-11-08T19:24:52.805606+0000","last_undegraded":"2023-11-08T19:24:52.805606+0000","last_fullsized":"2023-11-08T19:24:52.805606+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:13:58.948371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.309774+0000","last_change":"2023-11-08T19:24:50.617914+0000","last_active":"2023-11-08T19:24:52.309774+0000","last_peered":"2023-11-08T19:24:52.309774+0000","last_clean":"2023-11-08T19:24:52.309774+0000","last_became_active":"2023-11-08T19:24:50.617611+0000","last_became_peered":"2023-11-08T19:24:50.617611+0000","last_unstale":"2023-11-08T19:24:52.309774+0000","last_undegraded":"2023-11-08T19:24:52.309774+0000","last_fullsized":"2023-11-08T19:24:52.309774+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:30:36.041820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.352504+0000","last_change":"2023-11-08T19:24:50.640039+0000","last_active":"2023-11-08T19:24:52.352504+0000","last_peered":"2023-11-08T19:24:52.352504+0000","last_clean":"2023-11-08T19:24:52.352504+0000","last_became_active":"2023-11-08T19:24:50.639560+0000","last_became_peered":"2023-11-08T19:24:50.639560+0000","last_unstale":"2023-11-08T19:24:52.352504+0000","last_undegraded":"2023-11-08T19:24:52.352504+0000","last_fullsized":"2023-11-08T19:24:52.352504+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:01:53.410545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:24:49.588741+0000","last_change":"2023-11-08T19:24:49.588741+0000","last_active":"2023-11-08T19:24:49.588741+0000","last_peered":"2023-11-08T19:24:49.588741+0000","last_clean":"2023-11-08T19:24:49.588741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:24:49.588741+0000","last_undegraded":"2023-11-08T19:24:49.588741+0000","last_fullsized":"2023-11-08T19:24:49.588741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.351953+0000","last_change":"2023-11-08T19:24:50.641005+0000","last_active":"2023-11-08T19:24:52.351953+0000","last_peered":"2023-11-08T19:24:52.351953+0000","last_clean":"2023-11-08T19:24:52.351953+0000","last_became_active":"2023-11-08T19:24:50.640727+0000","last_became_peered":"2023-11-08T19:24:50.640727+0000","last_unstale":"2023-11-08T19:24:52.351953+0000","last_undegraded":"2023-11-08T19:24:52.351953+0000","last_fullsized":"2023-11-08T19:24:52.351953+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:36:53.967796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:24:49.588741+0000","last_change":"2023-11-08T19:24:49.588741+0000","last_active":"2023-11-08T19:24:49.588741+0000","last_peered":"2023-11-08T19:24:49.588741+0000","last_clean":"2023-11-08T19:24:49.588741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:24:49.588741+0000","last_undegraded":"2023-11-08T19:24:49.588741+0000","last_fullsized":"2023-11-08T19:24:49.588741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.626351+0000","last_change":"2023-11-08T19:24:50.639325+0000","last_active":"2023-11-08T19:24:51.626351+0000","last_peered":"2023-11-08T19:24:51.626351+0000","last_clean":"2023-11-08T19:24:51.626351+0000","last_became_active":"2023-11-08T19:24:50.638993+0000","last_became_peered":"2023-11-08T19:24:50.638993+0000","last_unstale":"2023-11-08T19:24:51.626351+0000","last_undegraded":"2023-11-08T19:24:51.626351+0000","last_fullsized":"2023-11-08T19:24:51.626351+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:30:32.602976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:24:49.588741+0000","last_change":"2023-11-08T19:24:49.588741+0000","last_active":"2023-11-08T19:24:49.588741+0000","last_peered":"2023-11-08T19:24:49.588741+0000","last_clean":"2023-11-08T19:24:49.588741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:24:49.588741+0000","last_undegraded":"2023-11-08T19:24:49.588741+0000","last_fullsized":"2023-11-08T19:24:49.588741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:24:49.588741+0000","last_change":"2023-11-08T19:24:49.588741+0000","last_active":"2023-11-08T19:24:49.588741+0000","last_peered":"2023-11-08T19:24:49.588741+0000","last_clean":"2023-11-08T19:24:49.588741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:24:49.588741+0000","last_undegraded":"2023-11-08T19:24:49.588741+0000","last_fullsized":"2023-11-08T19:24:49.588741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:24:49.588741+0000","last_change":"2023-11-08T19:24:49.588741+0000","last_active":"2023-11-08T19:24:49.588741+0000","last_peered":"2023-11-08T19:24:49.588741+0000","last_clean":"2023-11-08T19:24:49.588741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:24:49.588741+0000","last_undegraded":"2023-11-08T19:24:49.588741+0000","last_fullsized":"2023-11-08T19:24:49.588741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.805767+0000","last_change":"2023-11-08T19:24:50.621511+0000","last_active":"2023-11-08T19:24:52.805767+0000","last_peered":"2023-11-08T19:24:52.805767+0000","last_clean":"2023-11-08T19:24:52.805767+0000","last_became_active":"2023-11-08T19:24:50.621142+0000","last_became_peered":"2023-11-08T19:24:50.621142+0000","last_unstale":"2023-11-08T19:24:52.805767+0000","last_undegraded":"2023-11-08T19:24:52.805767+0000","last_fullsized":"2023-11-08T19:24:52.805767+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:14:01.459274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.626625+0000","last_change":"2023-11-08T19:24:50.625632+0000","last_active":"2023-11-08T19:24:51.626625+0000","last_peered":"2023-11-08T19:24:51.626625+0000","last_clean":"2023-11-08T19:24:51.626625+0000","last_became_active":"2023-11-08T19:24:50.625370+0000","last_became_peered":"2023-11-08T19:24:50.625370+0000","last_unstale":"2023-11-08T19:24:51.626625+0000","last_undegraded":"2023-11-08T19:24:51.626625+0000","last_fullsized":"2023-11-08T19:24:51.626625+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:13:58.948371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.309818+0000","last_change":"2023-11-08T19:24:50.625350+0000","last_active":"2023-11-08T19:24:52.309818+0000","last_peered":"2023-11-08T19:24:52.309818+0000","last_clean":"2023-11-08T19:24:52.309818+0000","last_became_active":"2023-11-08T19:24:50.625078+0000","last_became_peered":"2023-11-08T19:24:50.625078+0000","last_unstale":"2023-11-08T19:24:52.309818+0000","last_undegraded":"2023-11-08T19:24:52.309818+0000","last_fullsized":"2023-11-08T19:24:52.309818+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:42:42.512786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.804817+0000","last_change":"2023-11-08T19:24:50.629187+0000","last_active":"2023-11-08T19:24:52.804817+0000","last_peered":"2023-11-08T19:24:52.804817+0000","last_clean":"2023-11-08T19:24:52.804817+0000","last_became_active":"2023-11-08T19:24:50.628898+0000","last_became_peered":"2023-11-08T19:24:50.628898+0000","last_unstale":"2023-11-08T19:24:52.804817+0000","last_undegraded":"2023-11-08T19:24:52.804817+0000","last_fullsized":"2023-11-08T19:24:52.804817+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:15:33.285449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.351906+0000","last_change":"2023-11-08T19:24:50.638579+0000","last_active":"2023-11-08T19:24:52.351906+0000","last_peered":"2023-11-08T19:24:52.351906+0000","last_clean":"2023-11-08T19:24:52.351906+0000","last_became_active":"2023-11-08T19:24:50.638297+0000","last_became_peered":"2023-11-08T19:24:50.638297+0000","last_unstale":"2023-11-08T19:24:52.351906+0000","last_undegraded":"2023-11-08T19:24:52.351906+0000","last_fullsized":"2023-11-08T19:24:52.351906+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:13:58.948371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.352572+0000","last_change":"2023-11-08T19:24:50.641446+0000","last_active":"2023-11-08T19:24:52.352572+0000","last_peered":"2023-11-08T19:24:52.352572+0000","last_clean":"2023-11-08T19:24:52.352572+0000","last_became_active":"2023-11-08T19:24:50.639709+0000","last_became_peered":"2023-11-08T19:24:50.639709+0000","last_unstale":"2023-11-08T19:24:52.352572+0000","last_undegraded":"2023-11-08T19:24:52.352572+0000","last_fullsized":"2023-11-08T19:24:52.352572+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:09:01.749686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.804922+0000","last_change":"2023-11-08T19:24:50.619241+0000","last_active":"2023-11-08T19:24:52.804922+0000","last_peered":"2023-11-08T19:24:52.804922+0000","last_clean":"2023-11-08T19:24:52.804922+0000","last_became_active":"2023-11-08T19:24:50.618962+0000","last_became_peered":"2023-11-08T19:24:50.618962+0000","last_unstale":"2023-11-08T19:24:52.804922+0000","last_undegraded":"2023-11-08T19:24:52.804922+0000","last_fullsized":"2023-11-08T19:24:52.804922+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:14:01.459274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:24:49.588741+0000","last_change":"2023-11-08T19:24:49.588741+0000","last_active":"2023-11-08T19:24:49.588741+0000","last_peered":"2023-11-08T19:24:49.588741+0000","last_clean":"2023-11-08T19:24:49.588741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:24:49.588741+0000","last_undegraded":"2023-11-08T19:24:49.588741+0000","last_fullsized":"2023-11-08T19:24:49.588741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.309784+0000","last_change":"2023-11-08T19:24:49.608481+0000","last_active":"2023-11-08T19:24:52.309784+0000","last_peered":"2023-11-08T19:24:52.309784+0000","last_clean":"2023-11-08T19:24:52.309784+0000","last_became_active":"2023-11-08T19:24:49.607778+0000","last_became_peered":"2023-11-08T19:24:49.607778+0000","last_unstale":"2023-11-08T19:24:52.309784+0000","last_undegraded":"2023-11-08T19:24:52.309784+0000","last_fullsized":"2023-11-08T19:24:52.309784+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:01:09.884262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.626900+0000","last_change":"2023-11-08T19:24:50.624852+0000","last_active":"2023-11-08T19:24:51.626900+0000","last_peered":"2023-11-08T19:24:51.626900+0000","last_clean":"2023-11-08T19:24:51.626900+0000","last_became_active":"2023-11-08T19:24:50.624551+0000","last_became_peered":"2023-11-08T19:24:50.624551+0000","last_unstale":"2023-11-08T19:24:51.626900+0000","last_undegraded":"2023-11-08T19:24:51.626900+0000","last_fullsized":"2023-11-08T19:24:51.626900+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:12:54.641952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.626938+0000","last_change":"2023-11-08T19:24:49.610418+0000","last_active":"2023-11-08T19:24:51.626938+0000","last_peered":"2023-11-08T19:24:51.626938+0000","last_clean":"2023-11-08T19:24:51.626938+0000","last_became_active":"2023-11-08T19:24:49.610041+0000","last_became_peered":"2023-11-08T19:24:49.610041+0000","last_unstale":"2023-11-08T19:24:51.626938+0000","last_undegraded":"2023-11-08T19:24:51.626938+0000","last_fullsized":"2023-11-08T19:24:51.626938+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:00:12.067877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.309779+0000","last_change":"2023-11-08T19:24:50.627052+0000","last_active":"2023-11-08T19:24:52.309779+0000","last_peered":"2023-11-08T19:24:52.309779+0000","last_clean":"2023-11-08T19:24:52.309779+0000","last_became_active":"2023-11-08T19:24:50.626751+0000","last_became_peered":"2023-11-08T19:24:50.626751+0000","last_unstale":"2023-11-08T19:24:52.309779+0000","last_undegraded":"2023-11-08T19:24:52.309779+0000","last_fullsized":"2023-11-08T19:24:52.309779+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:46:19.375897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-08T19:24:48.592048+0000","last_change":"2023-11-08T19:24:48.592048+0000","last_active":"2023-11-08T19:24:48.584965+0000","last_peered":"2023-11-08T19:24:48.584965+0000","last_clean":"2023-11-08T19:24:48.584965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:24:48.592048+0000","last_undegraded":"2023-11-08T19:24:48.592048+0000","last_fullsized":"2023-11-08T19:24:48.592048+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.352125+0000","last_change":"2023-11-08T19:24:50.641183+0000","last_active":"2023-11-08T19:24:52.352125+0000","last_peered":"2023-11-08T19:24:52.352125+0000","last_clean":"2023-11-08T19:24:52.352125+0000","last_became_active":"2023-11-08T19:24:50.640910+0000","last_became_peered":"2023-11-08T19:24:50.640910+0000","last_unstale":"2023-11-08T19:24:52.352125+0000","last_undegraded":"2023-11-08T19:24:52.352125+0000","last_fullsized":"2023-11-08T19:24:52.352125+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:49:19.100370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-08T19:24:48.591551+0000","last_change":"2023-11-08T19:24:48.591551+0000","last_active":"2023-11-08T19:24:48.584965+0000","last_peered":"2023-11-08T19:24:48.584965+0000","last_clean":"2023-11-08T19:24:48.584965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:24:48.591551+0000","last_undegraded":"2023-11-08T19:24:48.591551+0000","last_fullsized":"2023-11-08T19:24:48.591551+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.352284+0000","last_change":"2023-11-08T19:24:50.640245+0000","last_active":"2023-11-08T19:24:52.352284+0000","last_peered":"2023-11-08T19:24:52.352284+0000","last_clean":"2023-11-08T19:24:52.352284+0000","last_became_active":"2023-11-08T19:24:50.639668+0000","last_became_peered":"2023-11-08T19:24:50.639668+0000","last_unstale":"2023-11-08T19:24:52.352284+0000","last_undegraded":"2023-11-08T19:24:52.352284+0000","last_fullsized":"2023-11-08T19:24:52.352284+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:42:27.150066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.626599+0000","last_change":"2023-11-08T19:24:49.612665+0000","last_active":"2023-11-08T19:24:51.626599+0000","last_peered":"2023-11-08T19:24:51.626599+0000","last_clean":"2023-11-08T19:24:51.626599+0000","last_became_active":"2023-11-08T19:24:49.612362+0000","last_became_peered":"2023-11-08T19:24:49.612362+0000","last_unstale":"2023-11-08T19:24:51.626599+0000","last_undegraded":"2023-11-08T19:24:51.626599+0000","last_fullsized":"2023-11-08T19:24:51.626599+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:46:12.493411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.805419+0000","last_change":"2023-11-08T19:24:50.627537+0000","last_active":"2023-11-08T19:24:52.805419+0000","last_peered":"2023-11-08T19:24:52.805419+0000","last_clean":"2023-11-08T19:24:52.805419+0000","last_became_active":"2023-11-08T19:24:50.626489+0000","last_became_peered":"2023-11-08T19:24:50.626489+0000","last_unstale":"2023-11-08T19:24:52.805419+0000","last_undegraded":"2023-11-08T19:24:52.805419+0000","last_fullsized":"2023-11-08T19:24:52.805419+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:01:10.888039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-08T19:24:48.592162+0000","last_change":"2023-11-08T19:24:48.592162+0000","last_active":"2023-11-08T19:24:48.584965+0000","last_peered":"2023-11-08T19:24:48.584965+0000","last_clean":"2023-11-08T19:24:48.584965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:24:48.592162+0000","last_undegraded":"2023-11-08T19:24:48.592162+0000","last_fullsized":"2023-11-08T19:24:48.592162+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.309935+0000","last_change":"2023-11-08T19:24:50.627987+0000","last_active":"2023-11-08T19:24:52.309935+0000","last_peered":"2023-11-08T19:24:52.309935+0000","last_clean":"2023-11-08T19:24:52.309935+0000","last_became_active":"2023-11-08T19:24:50.627402+0000","last_became_peered":"2023-11-08T19:24:50.627402+0000","last_unstale":"2023-11-08T19:24:52.309935+0000","last_undegraded":"2023-11-08T19:24:52.309935+0000","last_fullsized":"2023-11-08T19:24:52.309935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:36:53.967796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.626523+0000","last_change":"2023-11-08T19:24:49.606482+0000","last_active":"2023-11-08T19:24:51.626523+0000","last_peered":"2023-11-08T19:24:51.626523+0000","last_clean":"2023-11-08T19:24:51.626523+0000","last_became_active":"2023-11-08T19:24:49.606211+0000","last_became_peered":"2023-11-08T19:24:49.606211+0000","last_unstale":"2023-11-08T19:24:51.626523+0000","last_undegraded":"2023-11-08T19:24:51.626523+0000","last_fullsized":"2023-11-08T19:24:51.626523+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:42:59.845581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.626483+0000","last_change":"2023-11-08T19:24:50.625917+0000","last_active":"2023-11-08T19:24:51.626483+0000","last_peered":"2023-11-08T19:24:51.626483+0000","last_clean":"2023-11-08T19:24:51.626483+0000","last_became_active":"2023-11-08T19:24:50.625398+0000","last_became_peered":"2023-11-08T19:24:50.625398+0000","last_unstale":"2023-11-08T19:24:51.626483+0000","last_undegraded":"2023-11-08T19:24:51.626483+0000","last_fullsized":"2023-11-08T19:24:51.626483+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:40:11.978983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.351905+0000","last_change":"2023-11-08T19:24:49.616558+0000","last_active":"2023-11-08T19:24:52.351905+0000","last_peered":"2023-11-08T19:24:52.351905+0000","last_clean":"2023-11-08T19:24:52.351905+0000","last_became_active":"2023-11-08T19:24:49.616277+0000","last_became_peered":"2023-11-08T19:24:49.616277+0000","last_unstale":"2023-11-08T19:24:52.351905+0000","last_undegraded":"2023-11-08T19:24:52.351905+0000","last_fullsized":"2023-11-08T19:24:52.351905+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:01:09.884262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.805048+0000","last_change":"2023-11-08T19:24:50.623658+0000","last_active":"2023-11-08T19:24:52.805048+0000","last_peered":"2023-11-08T19:24:52.805048+0000","last_clean":"2023-11-08T19:24:52.805048+0000","last_became_active":"2023-11-08T19:24:50.623399+0000","last_became_peered":"2023-11-08T19:24:50.623399+0000","last_unstale":"2023-11-08T19:24:52.805048+0000","last_undegraded":"2023-11-08T19:24:52.805048+0000","last_fullsized":"2023-11-08T19:24:52.805048+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:40:11.978983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-08T19:24:48.593392+0000","last_change":"2023-11-08T19:24:48.593392+0000","last_active":"2023-11-08T19:24:48.584965+0000","last_peered":"2023-11-08T19:24:48.584965+0000","last_clean":"2023-11-08T19:24:48.584965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:24:48.593392+0000","last_undegraded":"2023-11-08T19:24:48.593392+0000","last_fullsized":"2023-11-08T19:24:48.593392+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.352305+0000","last_change":"2023-11-08T19:24:50.640397+0000","last_active":"2023-11-08T19:24:52.352305+0000","last_peered":"2023-11-08T19:24:52.352305+0000","last_clean":"2023-11-08T19:24:52.352305+0000","last_became_active":"2023-11-08T19:24:50.639730+0000","last_became_peered":"2023-11-08T19:24:50.639730+0000","last_unstale":"2023-11-08T19:24:52.352305+0000","last_undegraded":"2023-11-08T19:24:52.352305+0000","last_fullsized":"2023-11-08T19:24:52.352305+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:08:22.499163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.352212+0000","last_change":"2023-11-08T19:24:49.611202+0000","last_active":"2023-11-08T19:24:52.352212+0000","last_peered":"2023-11-08T19:24:52.352212+0000","last_clean":"2023-11-08T19:24:52.352212+0000","last_became_active":"2023-11-08T19:24:49.610894+0000","last_became_peered":"2023-11-08T19:24:49.610894+0000","last_unstale":"2023-11-08T19:24:52.352212+0000","last_undegraded":"2023-11-08T19:24:52.352212+0000","last_fullsized":"2023-11-08T19:24:52.352212+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:48:25.546248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.309935+0000","last_change":"2023-11-08T19:24:50.638901+0000","last_active":"2023-11-08T19:24:52.309935+0000","last_peered":"2023-11-08T19:24:52.309935+0000","last_clean":"2023-11-08T19:24:52.309935+0000","last_became_active":"2023-11-08T19:24:50.638587+0000","last_became_peered":"2023-11-08T19:24:50.638587+0000","last_unstale":"2023-11-08T19:24:52.309935+0000","last_undegraded":"2023-11-08T19:24:52.309935+0000","last_fullsized":"2023-11-08T19:24:52.309935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:13:58.948371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.805313+0000","last_change":"2023-11-08T19:24:49.612170+0000","last_active":"2023-11-08T19:24:52.805313+0000","last_peered":"2023-11-08T19:24:52.805313+0000","last_clean":"2023-11-08T19:24:52.805313+0000","last_became_active":"2023-11-08T19:24:49.611895+0000","last_became_peered":"2023-11-08T19:24:49.611895+0000","last_unstale":"2023-11-08T19:24:52.805313+0000","last_undegraded":"2023-11-08T19:24:52.805313+0000","last_fullsized":"2023-11-08T19:24:52.805313+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:38:24.246638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.805721+0000","last_change":"2023-11-08T19:24:50.627060+0000","last_active":"2023-11-08T19:24:52.805721+0000","last_peered":"2023-11-08T19:24:52.805721+0000","last_clean":"2023-11-08T19:24:52.805721+0000","last_became_active":"2023-11-08T19:24:50.626468+0000","last_became_peered":"2023-11-08T19:24:50.626468+0000","last_unstale":"2023-11-08T19:24:52.805721+0000","last_undegraded":"2023-11-08T19:24:52.805721+0000","last_fullsized":"2023-11-08T19:24:52.805721+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:00:13.071654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-08T19:24:48.591494+0000","last_change":"2023-11-08T19:24:48.591494+0000","last_active":"2023-11-08T19:24:48.584965+0000","last_peered":"2023-11-08T19:24:48.584965+0000","last_clean":"2023-11-08T19:24:48.584965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:24:48.591494+0000","last_undegraded":"2023-11-08T19:24:48.591494+0000","last_fullsized":"2023-11-08T19:24:48.591494+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:24:49.588741+0000","last_change":"2023-11-08T19:24:49.588741+0000","last_active":"2023-11-08T19:24:49.588741+0000","last_peered":"2023-11-08T19:24:49.588741+0000","last_clean":"2023-11-08T19:24:49.588741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:24:49.588741+0000","last_undegraded":"2023-11-08T19:24:49.588741+0000","last_fullsized":"2023-11-08T19:24:49.588741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.309884+0000","last_change":"2023-11-08T19:24:49.610471+0000","last_active":"2023-11-08T19:24:52.309884+0000","last_peered":"2023-11-08T19:24:52.309884+0000","last_clean":"2023-11-08T19:24:52.309884+0000","last_became_active":"2023-11-08T19:24:49.610182+0000","last_became_peered":"2023-11-08T19:24:49.610182+0000","last_unstale":"2023-11-08T19:24:52.309884+0000","last_undegraded":"2023-11-08T19:24:52.309884+0000","last_fullsized":"2023-11-08T19:24:52.309884+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:14:00.455497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.309602+0000","last_change":"2023-11-08T19:24:50.619498+0000","last_active":"2023-11-08T19:24:52.309602+0000","last_peered":"2023-11-08T19:24:52.309602+0000","last_clean":"2023-11-08T19:24:52.309602+0000","last_became_active":"2023-11-08T19:24:50.619231+0000","last_became_peered":"2023-11-08T19:24:50.619231+0000","last_unstale":"2023-11-08T19:24:52.309602+0000","last_undegraded":"2023-11-08T19:24:52.309602+0000","last_fullsized":"2023-11-08T19:24:52.309602+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:38:25.250414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.352455+0000","last_change":"2023-11-08T19:24:49.612336+0000","last_active":"2023-11-08T19:24:52.352455+0000","last_peered":"2023-11-08T19:24:52.352455+0000","last_clean":"2023-11-08T19:24:52.352455+0000","last_became_active":"2023-11-08T19:24:49.611885+0000","last_became_peered":"2023-11-08T19:24:49.611885+0000","last_unstale":"2023-11-08T19:24:52.352455+0000","last_undegraded":"2023-11-08T19:24:52.352455+0000","last_fullsized":"2023-11-08T19:24:52.352455+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:46:18.372121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:24:49.588741+0000","last_change":"2023-11-08T19:24:49.588741+0000","last_active":"2023-11-08T19:24:49.588741+0000","last_peered":"2023-11-08T19:24:49.588741+0000","last_clean":"2023-11-08T19:24:49.588741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:24:49.588741+0000","last_undegraded":"2023-11-08T19:24:49.588741+0000","last_fullsized":"2023-11-08T19:24:49.588741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-08T19:24:48.591465+0000","last_change":"2023-11-08T19:24:48.591465+0000","last_active":"2023-11-08T19:24:48.584965+0000","last_peered":"2023-11-08T19:24:48.584965+0000","last_clean":"2023-11-08T19:24:48.584965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:24:48.591465+0000","last_undegraded":"2023-11-08T19:24:48.591465+0000","last_fullsized":"2023-11-08T19:24:48.591465+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:24:49.588741+0000","last_change":"2023-11-08T19:24:49.588741+0000","last_active":"2023-11-08T19:24:49.588741+0000","last_peered":"2023-11-08T19:24:49.588741+0000","last_clean":"2023-11-08T19:24:49.588741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:24:49.588741+0000","last_undegraded":"2023-11-08T19:24:49.588741+0000","last_fullsized":"2023-11-08T19:24:49.588741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.804637+0000","last_change":"2023-11-08T19:24:49.608503+0000","last_active":"2023-11-08T19:24:52.804637+0000","last_peered":"2023-11-08T19:24:52.804637+0000","last_clean":"2023-11-08T19:24:52.804637+0000","last_became_active":"2023-11-08T19:24:49.608176+0000","last_became_peered":"2023-11-08T19:24:49.608176+0000","last_unstale":"2023-11-08T19:24:52.804637+0000","last_undegraded":"2023-11-08T19:24:52.804637+0000","last_fullsized":"2023-11-08T19:24:52.804637+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:17:50.005667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:24:49.588741+0000","last_change":"2023-11-08T19:24:49.588741+0000","last_active":"2023-11-08T19:24:49.588741+0000","last_peered":"2023-11-08T19:24:49.588741+0000","last_clean":"2023-11-08T19:24:49.588741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:24:49.588741+0000","last_undegraded":"2023-11-08T19:24:49.588741+0000","last_fullsized":"2023-11-08T19:24:49.588741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.805855+0000","last_change":"2023-11-08T19:24:49.610638+0000","last_active":"2023-11-08T19:24:52.805855+0000","last_peered":"2023-11-08T19:24:52.805855+0000","last_clean":"2023-11-08T19:24:52.805855+0000","last_became_active":"2023-11-08T19:24:49.610362+0000","last_became_peered":"2023-11-08T19:24:49.610362+0000","last_unstale":"2023-11-08T19:24:52.805855+0000","last_undegraded":"2023-11-08T19:24:52.805855+0000","last_fullsized":"2023-11-08T19:24:52.805855+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:15:32.030617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:24:49.588741+0000","last_change":"2023-11-08T19:24:49.588741+0000","last_active":"2023-11-08T19:24:49.588741+0000","last_peered":"2023-11-08T19:24:49.588741+0000","last_clean":"2023-11-08T19:24:49.588741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:24:49.588741+0000","last_undegraded":"2023-11-08T19:24:49.588741+0000","last_fullsized":"2023-11-08T19:24:49.588741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.805154+0000","last_change":"2023-11-08T19:24:49.613413+0000","last_active":"2023-11-08T19:24:52.805154+0000","last_peered":"2023-11-08T19:24:52.805154+0000","last_clean":"2023-11-08T19:24:52.805154+0000","last_became_active":"2023-11-08T19:24:49.613131+0000","last_became_peered":"2023-11-08T19:24:49.613131+0000","last_unstale":"2023-11-08T19:24:52.805154+0000","last_undegraded":"2023-11-08T19:24:52.805154+0000","last_fullsized":"2023-11-08T19:24:52.805154+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:46:12.493411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.352374+0000","last_change":"2023-11-08T19:24:50.640031+0000","last_active":"2023-11-08T19:24:52.352374+0000","last_peered":"2023-11-08T19:24:52.352374+0000","last_clean":"2023-11-08T19:24:52.352374+0000","last_became_active":"2023-11-08T19:24:50.639718+0000","last_became_peered":"2023-11-08T19:24:50.639718+0000","last_unstale":"2023-11-08T19:24:52.352374+0000","last_undegraded":"2023-11-08T19:24:52.352374+0000","last_fullsized":"2023-11-08T19:24:52.352374+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:08:17.096077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.352346+0000","last_change":"2023-11-08T19:24:49.612137+0000","last_active":"2023-11-08T19:24:52.352346+0000","last_peered":"2023-11-08T19:24:52.352346+0000","last_clean":"2023-11-08T19:24:52.352346+0000","last_became_active":"2023-11-08T19:24:49.611532+0000","last_became_peered":"2023-11-08T19:24:49.611532+0000","last_unstale":"2023-11-08T19:24:52.352346+0000","last_undegraded":"2023-11-08T19:24:52.352346+0000","last_fullsized":"2023-11-08T19:24:52.352346+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:40:10.975206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.805536+0000","last_change":"2023-11-08T19:24:50.630854+0000","last_active":"2023-11-08T19:24:52.805536+0000","last_peered":"2023-11-08T19:24:52.805536+0000","last_clean":"2023-11-08T19:24:52.805536+0000","last_became_active":"2023-11-08T19:24:50.630549+0000","last_became_peered":"2023-11-08T19:24:50.630549+0000","last_unstale":"2023-11-08T19:24:52.805536+0000","last_undegraded":"2023-11-08T19:24:52.805536+0000","last_fullsized":"2023-11-08T19:24:52.805536+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:48:26.550025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.804950+0000","last_change":"2023-11-08T19:24:50.620897+0000","last_active":"2023-11-08T19:24:52.804950+0000","last_peered":"2023-11-08T19:24:52.804950+0000","last_clean":"2023-11-08T19:24:52.804950+0000","last_became_active":"2023-11-08T19:24:50.620606+0000","last_became_peered":"2023-11-08T19:24:50.620606+0000","last_unstale":"2023-11-08T19:24:52.804950+0000","last_undegraded":"2023-11-08T19:24:52.804950+0000","last_fullsized":"2023-11-08T19:24:52.804950+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:30:36.041820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.804625+0000","last_change":"2023-11-08T19:24:49.612840+0000","last_active":"2023-11-08T19:24:52.804625+0000","last_peered":"2023-11-08T19:24:52.804625+0000","last_clean":"2023-11-08T19:24:52.804625+0000","last_became_active":"2023-11-08T19:24:49.612652+0000","last_became_peered":"2023-11-08T19:24:49.612652+0000","last_unstale":"2023-11-08T19:24:52.804625+0000","last_undegraded":"2023-11-08T19:24:52.804625+0000","last_fullsized":"2023-11-08T19:24:52.804625+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:48:25.546248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.626593+0000","last_change":"2023-11-08T19:24:50.642515+0000","last_active":"2023-11-08T19:24:51.626593+0000","last_peered":"2023-11-08T19:24:51.626593+0000","last_clean":"2023-11-08T19:24:51.626593+0000","last_became_active":"2023-11-08T19:24:50.642204+0000","last_became_peered":"2023-11-08T19:24:50.642204+0000","last_unstale":"2023-11-08T19:24:51.626593+0000","last_undegraded":"2023-11-08T19:24:51.626593+0000","last_fullsized":"2023-11-08T19:24:51.626593+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:42:42.512786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.351949+0000","last_change":"2023-11-08T19:24:49.605594+0000","last_active":"2023-11-08T19:24:52.351949+0000","last_peered":"2023-11-08T19:24:52.351949+0000","last_clean":"2023-11-08T19:24:52.351949+0000","last_became_active":"2023-11-08T19:24:49.605208+0000","last_became_peered":"2023-11-08T19:24:49.605208+0000","last_unstale":"2023-11-08T19:24:52.351949+0000","last_undegraded":"2023-11-08T19:24:52.351949+0000","last_fullsized":"2023-11-08T19:24:52.351949+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:17:50.005667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":162,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-11-08T19:24:48.592313+0000","last_change":"2023-11-08T19:24:45.888122+0000","last_active":"2023-11-08T19:24:48.592313+0000","last_peered":"2023-11-08T19:24:48.592313+0000","last_clean":"2023-11-08T19:24:48.592313+0000","last_became_active":"2023-11-08T19:24:45.887048+0000","last_became_peered":"2023-11-08T19:24:45.887048+0000","last_unstale":"2023-11-08T19:24:48.592313+0000","last_undegraded":"2023-11-08T19:24:48.592313+0000","last_fullsized":"2023-11-08T19:24:48.592313+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:44.575133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:44.575133+0000","last_clean_scrub_stamp":"2023-11-08T19:24:44.575133+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:19:20.202062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.309622+0000","last_change":"2023-11-08T19:24:50.628353+0000","last_active":"2023-11-08T19:24:52.309622+0000","last_peered":"2023-11-08T19:24:52.309622+0000","last_clean":"2023-11-08T19:24:52.309622+0000","last_became_active":"2023-11-08T19:24:50.628008+0000","last_became_peered":"2023-11-08T19:24:50.628008+0000","last_unstale":"2023-11-08T19:24:52.309622+0000","last_undegraded":"2023-11-08T19:24:52.309622+0000","last_fullsized":"2023-11-08T19:24:52.309622+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:46:13.497187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-08T19:24:48.591491+0000","last_change":"2023-11-08T19:24:48.591491+0000","last_active":"2023-11-08T19:24:48.584965+0000","last_peered":"2023-11-08T19:24:48.584965+0000","last_clean":"2023-11-08T19:24:48.584965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:24:48.591491+0000","last_undegraded":"2023-11-08T19:24:48.591491+0000","last_fullsized":"2023-11-08T19:24:48.591491+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-08T19:24:48.591465+0000","last_change":"2023-11-08T19:24:48.591465+0000","last_active":"2023-11-08T19:24:48.584965+0000","last_peered":"2023-11-08T19:24:48.584965+0000","last_clean":"2023-11-08T19:24:48.584965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:24:48.591465+0000","last_undegraded":"2023-11-08T19:24:48.591465+0000","last_fullsized":"2023-11-08T19:24:48.591465+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.805231+0000","last_change":"2023-11-08T19:24:50.624279+0000","last_active":"2023-11-08T19:24:52.805231+0000","last_peered":"2023-11-08T19:24:52.805231+0000","last_clean":"2023-11-08T19:24:52.805231+0000","last_became_active":"2023-11-08T19:24:50.623920+0000","last_became_peered":"2023-11-08T19:24:50.623920+0000","last_unstale":"2023-11-08T19:24:52.805231+0000","last_undegraded":"2023-11-08T19:24:52.805231+0000","last_fullsized":"2023-11-08T19:24:52.805231+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:42:42.512786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.805402+0000","last_change":"2023-11-08T19:24:49.606910+0000","last_active":"2023-11-08T19:24:52.805402+0000","last_peered":"2023-11-08T19:24:52.805402+0000","last_clean":"2023-11-08T19:24:52.805402+0000","last_became_active":"2023-11-08T19:24:49.606627+0000","last_became_peered":"2023-11-08T19:24:49.606627+0000","last_unstale":"2023-11-08T19:24:52.805402+0000","last_undegraded":"2023-11-08T19:24:52.805402+0000","last_fullsized":"2023-11-08T19:24:52.805402+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:56:00.148860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.627087+0000","last_change":"2023-11-08T19:24:50.622398+0000","last_active":"2023-11-08T19:24:51.627087+0000","last_peered":"2023-11-08T19:24:51.627087+0000","last_clean":"2023-11-08T19:24:51.627087+0000","last_became_active":"2023-11-08T19:24:50.622092+0000","last_became_peered":"2023-11-08T19:24:50.622092+0000","last_unstale":"2023-11-08T19:24:51.627087+0000","last_undegraded":"2023-11-08T19:24:51.627087+0000","last_fullsized":"2023-11-08T19:24:51.627087+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:42:27.150066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-08T19:24:48.592846+0000","last_change":"2023-11-08T19:24:48.592846+0000","last_active":"2023-11-08T19:24:48.584965+0000","last_peered":"2023-11-08T19:24:48.584965+0000","last_clean":"2023-11-08T19:24:48.584965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:24:48.592846+0000","last_undegraded":"2023-11-08T19:24:48.592846+0000","last_fullsized":"2023-11-08T19:24:48.592846+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.309770+0000","last_change":"2023-11-08T19:24:50.624848+0000","last_active":"2023-11-08T19:24:52.309770+0000","last_peered":"2023-11-08T19:24:52.309770+0000","last_clean":"2023-11-08T19:24:52.309770+0000","last_became_active":"2023-11-08T19:24:50.624518+0000","last_became_peered":"2023-11-08T19:24:50.624518+0000","last_unstale":"2023-11-08T19:24:52.309770+0000","last_undegraded":"2023-11-08T19:24:52.309770+0000","last_fullsized":"2023-11-08T19:24:52.309770+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:15:33.285449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-08T19:24:48.592059+0000","last_change":"2023-11-08T19:24:48.592059+0000","last_active":"2023-11-08T19:24:48.584965+0000","last_peered":"2023-11-08T19:24:48.584965+0000","last_clean":"2023-11-08T19:24:48.584965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:24:48.592059+0000","last_undegraded":"2023-11-08T19:24:48.592059+0000","last_fullsized":"2023-11-08T19:24:48.592059+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.805558+0000","last_change":"2023-11-08T19:24:50.627488+0000","last_active":"2023-11-08T19:24:52.805558+0000","last_peered":"2023-11-08T19:24:52.805558+0000","last_clean":"2023-11-08T19:24:52.805558+0000","last_became_active":"2023-11-08T19:24:50.626571+0000","last_became_peered":"2023-11-08T19:24:50.626571+0000","last_unstale":"2023-11-08T19:24:52.805558+0000","last_undegraded":"2023-11-08T19:24:52.805558+0000","last_fullsized":"2023-11-08T19:24:52.805558+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:43:00.849357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.804911+0000","last_change":"2023-11-08T19:24:49.606579+0000","last_active":"2023-11-08T19:24:52.804911+0000","last_peered":"2023-11-08T19:24:52.804911+0000","last_clean":"2023-11-08T19:24:52.804911+0000","last_became_active":"2023-11-08T19:24:49.606283+0000","last_became_peered":"2023-11-08T19:24:49.606283+0000","last_unstale":"2023-11-08T19:24:52.804911+0000","last_undegraded":"2023-11-08T19:24:52.804911+0000","last_fullsized":"2023-11-08T19:24:52.804911+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:14:00.455497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.626421+0000","last_change":"2023-11-08T19:24:50.625692+0000","last_active":"2023-11-08T19:24:51.626421+0000","last_peered":"2023-11-08T19:24:51.626421+0000","last_clean":"2023-11-08T19:24:51.626421+0000","last_became_active":"2023-11-08T19:24:50.625014+0000","last_became_peered":"2023-11-08T19:24:50.625014+0000","last_unstale":"2023-11-08T19:24:51.626421+0000","last_undegraded":"2023-11-08T19:24:51.626421+0000","last_fullsized":"2023-11-08T19:24:51.626421+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:15:33.285449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.805081+0000","last_change":"2023-11-08T19:24:49.611023+0000","last_active":"2023-11-08T19:24:52.805081+0000","last_peered":"2023-11-08T19:24:52.805081+0000","last_clean":"2023-11-08T19:24:52.805081+0000","last_became_active":"2023-11-08T19:24:49.610735+0000","last_became_peered":"2023-11-08T19:24:49.610735+0000","last_unstale":"2023-11-08T19:24:52.805081+0000","last_undegraded":"2023-11-08T19:24:52.805081+0000","last_fullsized":"2023-11-08T19:24:52.805081+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:30:31.599200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.626730+0000","last_change":"2023-11-08T19:24:50.631491+0000","last_active":"2023-11-08T19:24:51.626730+0000","last_peered":"2023-11-08T19:24:51.626730+0000","last_clean":"2023-11-08T19:24:51.626730+0000","last_became_active":"2023-11-08T19:24:50.630872+0000","last_became_peered":"2023-11-08T19:24:50.630872+0000","last_unstale":"2023-11-08T19:24:51.626730+0000","last_undegraded":"2023-11-08T19:24:51.626730+0000","last_fullsized":"2023-11-08T19:24:51.626730+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:36:53.967796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-08T19:24:48.592103+0000","last_change":"2023-11-08T19:24:48.592103+0000","last_active":"2023-11-08T19:24:48.584965+0000","last_peered":"2023-11-08T19:24:48.584965+0000","last_clean":"2023-11-08T19:24:48.584965+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:24:48.592103+0000","last_undegraded":"2023-11-08T19:24:48.592103+0000","last_fullsized":"2023-11-08T19:24:48.592103+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:24:49.588741+0000","last_change":"2023-11-08T19:24:49.588741+0000","last_active":"2023-11-08T19:24:49.588741+0000","last_peered":"2023-11-08T19:24:49.588741+0000","last_clean":"2023-11-08T19:24:49.588741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:24:49.588741+0000","last_undegraded":"2023-11-08T19:24:49.588741+0000","last_fullsized":"2023-11-08T19:24:49.588741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.805091+0000","last_change":"2023-11-08T19:24:49.613365+0000","last_active":"2023-11-08T19:24:52.805091+0000","last_peered":"2023-11-08T19:24:52.805091+0000","last_clean":"2023-11-08T19:24:52.805091+0000","last_became_active":"2023-11-08T19:24:49.613052+0000","last_became_peered":"2023-11-08T19:24:49.613052+0000","last_unstale":"2023-11-08T19:24:52.805091+0000","last_undegraded":"2023-11-08T19:24:52.805091+0000","last_fullsized":"2023-11-08T19:24:52.805091+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:41:24.551269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:24:49.588741+0000","last_change":"2023-11-08T19:24:49.588741+0000","last_active":"2023-11-08T19:24:49.588741+0000","last_peered":"2023-11-08T19:24:49.588741+0000","last_clean":"2023-11-08T19:24:49.588741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:24:49.588741+0000","last_undegraded":"2023-11-08T19:24:49.588741+0000","last_fullsized":"2023-11-08T19:24:49.588741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.805219+0000","last_change":"2023-11-08T19:24:49.606271+0000","last_active":"2023-11-08T19:24:52.805219+0000","last_peered":"2023-11-08T19:24:52.805219+0000","last_clean":"2023-11-08T19:24:52.805219+0000","last_became_active":"2023-11-08T19:24:49.605802+0000","last_became_peered":"2023-11-08T19:24:49.605802+0000","last_unstale":"2023-11-08T19:24:52.805219+0000","last_undegraded":"2023-11-08T19:24:52.805219+0000","last_fullsized":"2023-11-08T19:24:52.805219+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:07:00.575561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.352338+0000","last_change":"2023-11-08T19:24:50.640594+0000","last_active":"2023-11-08T19:24:52.352338+0000","last_peered":"2023-11-08T19:24:52.352338+0000","last_clean":"2023-11-08T19:24:52.352338+0000","last_became_active":"2023-11-08T19:24:50.640295+0000","last_became_peered":"2023-11-08T19:24:50.640295+0000","last_unstale":"2023-11-08T19:24:52.352338+0000","last_undegraded":"2023-11-08T19:24:52.352338+0000","last_fullsized":"2023-11-08T19:24:52.352338+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:34:42.075394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.805026+0000","last_change":"2023-11-08T19:24:49.610782+0000","last_active":"2023-11-08T19:24:52.805026+0000","last_peered":"2023-11-08T19:24:52.805026+0000","last_clean":"2023-11-08T19:24:52.805026+0000","last_became_active":"2023-11-08T19:24:49.610493+0000","last_became_peered":"2023-11-08T19:24:49.610493+0000","last_unstale":"2023-11-08T19:24:52.805026+0000","last_undegraded":"2023-11-08T19:24:52.805026+0000","last_fullsized":"2023-11-08T19:24:52.805026+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:42:59.845581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:24:49.588741+0000","last_change":"2023-11-08T19:24:49.588741+0000","last_active":"2023-11-08T19:24:49.588741+0000","last_peered":"2023-11-08T19:24:49.588741+0000","last_clean":"2023-11-08T19:24:49.588741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:24:49.588741+0000","last_undegraded":"2023-11-08T19:24:49.588741+0000","last_fullsized":"2023-11-08T19:24:49.588741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:24:49.588741+0000","last_change":"2023-11-08T19:24:49.588741+0000","last_active":"2023-11-08T19:24:49.588741+0000","last_peered":"2023-11-08T19:24:49.588741+0000","last_clean":"2023-11-08T19:24:49.588741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:24:49.588741+0000","last_undegraded":"2023-11-08T19:24:49.588741+0000","last_fullsized":"2023-11-08T19:24:49.588741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.804775+0000","last_change":"2023-11-08T19:24:50.625948+0000","last_active":"2023-11-08T19:24:52.804775+0000","last_peered":"2023-11-08T19:24:52.804775+0000","last_clean":"2023-11-08T19:24:52.804775+0000","last_became_active":"2023-11-08T19:24:50.625633+0000","last_became_peered":"2023-11-08T19:24:50.625633+0000","last_unstale":"2023-11-08T19:24:52.804775+0000","last_undegraded":"2023-11-08T19:24:52.804775+0000","last_fullsized":"2023-11-08T19:24:52.804775+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:00:13.071654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.352423+0000","last_change":"2023-11-08T19:24:50.640919+0000","last_active":"2023-11-08T19:24:52.352423+0000","last_peered":"2023-11-08T19:24:52.352423+0000","last_clean":"2023-11-08T19:24:52.352423+0000","last_became_active":"2023-11-08T19:24:50.640496+0000","last_became_peered":"2023-11-08T19:24:50.640496+0000","last_unstale":"2023-11-08T19:24:52.352423+0000","last_undegraded":"2023-11-08T19:24:52.352423+0000","last_fullsized":"2023-11-08T19:24:52.352423+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:59:37.592538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.309782+0000","last_change":"2023-11-08T19:24:50.625916+0000","last_active":"2023-11-08T19:24:52.309782+0000","last_peered":"2023-11-08T19:24:52.309782+0000","last_clean":"2023-11-08T19:24:52.309782+0000","last_became_active":"2023-11-08T19:24:50.625625+0000","last_became_peered":"2023-11-08T19:24:50.625625+0000","last_unstale":"2023-11-08T19:24:52.309782+0000","last_undegraded":"2023-11-08T19:24:52.309782+0000","last_fullsized":"2023-11-08T19:24:52.309782+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:40:11.978983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.626555+0000","last_change":"2023-11-08T19:24:50.628686+0000","last_active":"2023-11-08T19:24:51.626555+0000","last_peered":"2023-11-08T19:24:51.626555+0000","last_clean":"2023-11-08T19:24:51.626555+0000","last_became_active":"2023-11-08T19:24:50.628289+0000","last_became_peered":"2023-11-08T19:24:50.628289+0000","last_unstale":"2023-11-08T19:24:51.626555+0000","last_undegraded":"2023-11-08T19:24:51.626555+0000","last_fullsized":"2023-11-08T19:24:51.626555+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:46:19.375897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.805231+0000","last_change":"2023-11-08T19:24:50.624352+0000","last_active":"2023-11-08T19:24:52.805231+0000","last_peered":"2023-11-08T19:24:52.805231+0000","last_clean":"2023-11-08T19:24:52.805231+0000","last_became_active":"2023-11-08T19:24:50.624035+0000","last_became_peered":"2023-11-08T19:24:50.624035+0000","last_unstale":"2023-11-08T19:24:52.805231+0000","last_undegraded":"2023-11-08T19:24:52.805231+0000","last_fullsized":"2023-11-08T19:24:52.805231+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:46:19.375897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:24:49.588741+0000","last_change":"2023-11-08T19:24:49.588741+0000","last_active":"2023-11-08T19:24:49.588741+0000","last_peered":"2023-11-08T19:24:49.588741+0000","last_clean":"2023-11-08T19:24:49.588741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:24:49.588741+0000","last_undegraded":"2023-11-08T19:24:49.588741+0000","last_fullsized":"2023-11-08T19:24:49.588741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.626991+0000","last_change":"2023-11-08T19:24:50.628009+0000","last_active":"2023-11-08T19:24:51.626991+0000","last_peered":"2023-11-08T19:24:51.626991+0000","last_clean":"2023-11-08T19:24:51.626991+0000","last_became_active":"2023-11-08T19:24:50.627668+0000","last_became_peered":"2023-11-08T19:24:50.627668+0000","last_unstale":"2023-11-08T19:24:51.626991+0000","last_undegraded":"2023-11-08T19:24:51.626991+0000","last_fullsized":"2023-11-08T19:24:51.626991+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:49:19.100370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.352100+0000","last_change":"2023-11-08T19:24:50.641255+0000","last_active":"2023-11-08T19:24:52.352100+0000","last_peered":"2023-11-08T19:24:52.352100+0000","last_clean":"2023-11-08T19:24:52.352100+0000","last_became_active":"2023-11-08T19:24:50.640164+0000","last_became_peered":"2023-11-08T19:24:50.640164+0000","last_unstale":"2023-11-08T19:24:52.352100+0000","last_undegraded":"2023-11-08T19:24:52.352100+0000","last_fullsized":"2023-11-08T19:24:52.352100+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:12:54.641952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.626435+0000","last_change":"2023-11-08T19:24:50.629006+0000","last_active":"2023-11-08T19:24:51.626435+0000","last_peered":"2023-11-08T19:24:51.626435+0000","last_clean":"2023-11-08T19:24:51.626435+0000","last_became_active":"2023-11-08T19:24:50.628685+0000","last_became_peered":"2023-11-08T19:24:50.628685+0000","last_unstale":"2023-11-08T19:24:51.626435+0000","last_undegraded":"2023-11-08T19:24:51.626435+0000","last_fullsized":"2023-11-08T19:24:51.626435+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:30:36.041820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.352270+0000","last_change":"2023-11-08T19:24:50.640737+0000","last_active":"2023-11-08T19:24:52.352270+0000","last_peered":"2023-11-08T19:24:52.352270+0000","last_clean":"2023-11-08T19:24:52.352270+0000","last_became_active":"2023-11-08T19:24:50.640255+0000","last_became_peered":"2023-11-08T19:24:50.640255+0000","last_unstale":"2023-11-08T19:24:52.352270+0000","last_undegraded":"2023-11-08T19:24:52.352270+0000","last_fullsized":"2023-11-08T19:24:52.352270+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:13:41.469149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.309753+0000","last_change":"2023-11-08T19:24:50.620645+0000","last_active":"2023-11-08T19:24:52.309753+0000","last_peered":"2023-11-08T19:24:52.309753+0000","last_clean":"2023-11-08T19:24:52.309753+0000","last_became_active":"2023-11-08T19:24:50.620349+0000","last_became_peered":"2023-11-08T19:24:50.620349+0000","last_unstale":"2023-11-08T19:24:52.309753+0000","last_undegraded":"2023-11-08T19:24:52.309753+0000","last_fullsized":"2023-11-08T19:24:52.309753+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:30:32.602976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:24:49.588741+0000","last_change":"2023-11-08T19:24:49.588741+0000","last_active":"2023-11-08T19:24:49.588741+0000","last_peered":"2023-11-08T19:24:49.588741+0000","last_clean":"2023-11-08T19:24:49.588741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:24:49.588741+0000","last_undegraded":"2023-11-08T19:24:49.588741+0000","last_fullsized":"2023-11-08T19:24:49.588741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":92,"acting":92,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":798,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":2626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":23,"ondisk_log_size":23,"up":64,"acting":64,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27760,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344080,"statfs":{"total":96636764160,"available":96608337920,"internally_reserved":0,"allocated":761856,"data_stored":641928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47099999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52000000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.432}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56200000000000006}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.372}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67900000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57599999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":167936,"data_stored":51560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47599999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.495}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51900000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56200000000000006}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39400000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":71070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36299999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23499999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26500000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45200000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47699999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":811008,"data_stored":661916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35099999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27400000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40000000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73199999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71099999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68300000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81000000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":192512,"data_stored":64610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46000000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.443}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82899999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81599999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84799999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.878}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.308}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27000000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.379}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59399999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50900000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54100000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70099999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":71036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46000000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55800000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65400000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75800000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67200000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.746}]}]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":208896,"data_stored":71000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39800000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55300000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61099999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.439}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66700000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69499999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-08T19:24:56.462 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T19:24:56.465 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T19:24:59.281 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-08T19:24:59.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.413+0000 7f71b4b26700 1 Processor -- start 2023-11-08T19:24:59.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.413+0000 7f71b4b26700 1 -- start start 2023-11-08T19:24:59.417 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.414+0000 7f71b4b26700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f71b01618f0 0x7f71b006ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:59.417 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.414+0000 7f71b4b26700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f71b006b150 0x7f71b00cb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:59.418 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.414+0000 7f71b4b26700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71b0161df0 con 0x7f71b015dd00 2023-11-08T19:24:59.418 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.414+0000 7f71b4b26700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71b006b600 con 0x7f71b01618f0 2023-11-08T19:24:59.418 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.414+0000 7f71b4b26700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71b00cad50 con 0x7f71b006b150 2023-11-08T19:24:59.418 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.414+0000 7f71add9b700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f71b01618f0 0x7f71b006ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:59.419 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.414+0000 7f71ae59c700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f71b015dd00 0x7f71b00cca70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:44884/0 (socket says 172.21.15.143:44884) 2023-11-08T19:24:59.419 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.414+0000 7f71ae59c700 1 -- 172.21.15.143:0/1766386427 learned_addr learned my addr 172.21.15.143:0/1766386427 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:24:59.419 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.414+0000 7f71ad59a700 1 -- 172.21.15.143:0/1766386427 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4175394923 0 0) 0x7f71b0161df0 con 0x7f71b015dd00 2023-11-08T19:24:59.419 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.414+0000 7f71ad59a700 1 -- 172.21.15.143:0/1766386427 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f71980035b0 con 0x7f71b015dd00 2023-11-08T19:24:59.419 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.414+0000 7f71aed9d700 1 --2- 172.21.15.143:0/1766386427 >> v2:172.21.15.174:3300/0 conn(0x7f71b006b150 0x7f71b00cb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:59.419 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.415+0000 7f71add9b700 1 -- 172.21.15.143:0/1766386427 >> v2:172.21.15.174:3300/0 conn(0x7f71b006b150 msgr2=0x7f71b00cb1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:24:59.420 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.415+0000 7f71add9b700 1 --2- 172.21.15.143:0/1766386427 >> v2:172.21.15.174:3300/0 conn(0x7f71b006b150 0x7f71b00cb1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:59.420 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.415+0000 7f71add9b700 1 -- 172.21.15.143:0/1766386427 >> v1:172.21.15.143:6789/0 conn(0x7f71b015dd00 legacy=0x7f71b00cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:59.420 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.415+0000 7f71add9b700 1 -- 172.21.15.143:0/1766386427 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71b01620d0 con 0x7f71b01618f0 2023-11-08T19:24:59.420 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.415+0000 7f71add9b700 1 --2- 172.21.15.143:0/1766386427 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f71b01618f0 0x7f71b006ac10 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f71b0099f70 tx=0x7f71a0009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=686bc8fed704086b server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:59.420 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.415+0000 7f71ad59a700 1 -- 172.21.15.143:0/1766386427 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a0011830 con 0x7f71b01618f0 2023-11-08T19:24:59.421 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.415+0000 7f71ad59a700 1 -- 172.21.15.143:0/1766386427 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71a0011990 con 0x7f71b01618f0 2023-11-08T19:24:59.421 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.415+0000 7f71ad59a700 1 -- 172.21.15.143:0/1766386427 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a0011d20 con 0x7f71b01618f0 2023-11-08T19:24:59.421 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.415+0000 7f71b4b26700 1 -- 172.21.15.143:0/1766386427 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f71b01618f0 msgr2=0x7f71b006ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:59.421 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.415+0000 7f71b4b26700 1 --2- 172.21.15.143:0/1766386427 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f71b01618f0 0x7f71b006ac10 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f71b0099f70 tx=0x7f71a0009b20 comp rx=0 tx=0).stop 2023-11-08T19:24:59.421 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.416+0000 7f71b4b26700 1 -- 172.21.15.143:0/1766386427 shutdown_connections 2023-11-08T19:24:59.422 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.416+0000 7f71b4b26700 1 --2- 172.21.15.143:0/1766386427 >> v2:172.21.15.174:3300/0 conn(0x7f71b006b150 0x7f71b00cb1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:59.422 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.416+0000 7f71b4b26700 1 --2- 172.21.15.143:0/1766386427 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f71b01618f0 0x7f71b006ac10 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:59.422 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.416+0000 7f71b4b26700 1 -- 172.21.15.143:0/1766386427 >> 172.21.15.143:0/1766386427 conn(0x7f71b0099ab0 msgr2=0x7f71b00d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:24:59.422 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.416+0000 7f71b4b26700 1 -- 172.21.15.143:0/1766386427 shutdown_connections 2023-11-08T19:24:59.422 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.416+0000 7f71b4b26700 1 -- 172.21.15.143:0/1766386427 wait complete. 2023-11-08T19:24:59.425 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.417+0000 7f71b4b26700 1 Processor -- start 2023-11-08T19:24:59.426 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.417+0000 7f71b4b26700 1 -- start start 2023-11-08T19:24:59.426 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.418+0000 7f71b4b26700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f71b006b150 0x7f71b014f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:59.426 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.418+0000 7f71b4b26700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f71b015dd00 0x7f71b00c77d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:59.426 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.418+0000 7f71b4b26700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f71b01618f0 0x7f71b01f2d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:59.426 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.418+0000 7f71b4b26700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71b00b1100 con 0x7f71b006b150 2023-11-08T19:24:59.427 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.418+0000 7f71b4b26700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71b00b00f0 con 0x7f71b01618f0 2023-11-08T19:24:59.427 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.418+0000 7f71b4b26700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71b00b0340 con 0x7f71b015dd00 2023-11-08T19:24:59.427 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.418+0000 7f71ae59c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f71b006b150 0x7f71b014f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:59.427 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.418+0000 7f71ae59c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f71b006b150 0x7f71b014f980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48736/0 (socket says 172.21.15.143:48736) 2023-11-08T19:24:59.427 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.418+0000 7f71ae59c700 1 -- 172.21.15.143:0/1681604039 learned_addr learned my addr 172.21.15.143:0/1681604039 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:24:59.428 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.419+0000 7f71aed9d700 1 --2- 172.21.15.143:0/1681604039 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f71b01618f0 0x7f71b01f2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:59.428 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.419+0000 7f71ae59c700 1 -- 172.21.15.143:0/1681604039 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f71b015dd00 msgr2=0x7f71b00c77d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:59.428 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.419+0000 7f71add9b700 1 --2- 172.21.15.143:0/1681604039 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f71b015dd00 0x7f71b00c77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:59.428 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.419+0000 7f71ae59c700 1 --2- 172.21.15.143:0/1681604039 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f71b015dd00 0x7f71b00c77d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:59.428 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.419+0000 7f71ae59c700 1 -- 172.21.15.143:0/1681604039 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f71b01618f0 msgr2=0x7f71b01f2d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:59.429 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.419+0000 7f71ae59c700 1 --2- 172.21.15.143:0/1681604039 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f71b01618f0 0x7f71b01f2d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:59.429 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.419+0000 7f71ae59c700 1 -- 172.21.15.143:0/1681604039 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71a00094a0 con 0x7f71b006b150 2023-11-08T19:24:59.429 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.419+0000 7f71ae59c700 1 --2- 172.21.15.143:0/1681604039 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f71b006b150 0x7f71b014f980 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f71a4001260 tx=0x7f71a400ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:59.429 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.419+0000 7f719effd700 1 -- 172.21.15.143:0/1681604039 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a40074e0 con 0x7f71b006b150 2023-11-08T19:24:59.429 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.419+0000 7f719effd700 1 -- 172.21.15.143:0/1681604039 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71a4007640 con 0x7f71b006b150 2023-11-08T19:24:59.430 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.419+0000 7f71b4b26700 1 -- 172.21.15.143:0/1681604039 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71b00b0620 con 0x7f71b006b150 2023-11-08T19:24:59.430 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.419+0000 7f719effd700 1 -- 172.21.15.143:0/1681604039 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a4005a50 con 0x7f71b006b150 2023-11-08T19:24:59.430 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.419+0000 7f71b4b26700 1 -- 172.21.15.143:0/1681604039 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71b00b0be0 con 0x7f71b006b150 2023-11-08T19:24:59.430 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.420+0000 7f71b4b26700 1 -- 172.21.15.143:0/1681604039 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71b00aff60 con 0x7f71b006b150 2023-11-08T19:24:59.430 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.420+0000 7f719effd700 1 -- 172.21.15.143:0/1681604039 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f71a4005bb0 con 0x7f71b006b150 2023-11-08T19:24:59.431 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.421+0000 7f719effd700 1 --2- 172.21.15.143:0/1681604039 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f719006af00 0x7f719006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:59.431 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.421+0000 7f719effd700 1 -- 172.21.15.143:0/1681604039 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f71a4088c10 con 0x7f71b006b150 2023-11-08T19:24:59.431 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.421+0000 7f71add9b700 1 --2- 172.21.15.143:0/1681604039 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f719006af00 0x7f719006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:59.431 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.421+0000 7f71add9b700 1 --2- 172.21.15.143:0/1681604039 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f719006af00 0x7f719006d3c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f71b00c5810 tx=0x7f71a0008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:59.432 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.425+0000 7f719effd700 1 -- 172.21.15.143:0/1681604039 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f71a4053c20 con 0x7f71b006b150 2023-11-08T19:24:59.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.611+0000 7f71b4b26700 1 -- 172.21.15.143:0/1681604039 --> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f71b00ca470 con 0x7f719006af00 2023-11-08T19:24:59.620 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.619+0000 7f719effd700 1 -- 172.21.15.143:0/1681604039 <== mgr.4099 v2:172.21.15.174:6832/4037280178 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264645 (secure 0 0 0) 0x7f71b00ca470 con 0x7f719006af00 2023-11-08T19:24:59.623 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:24:59.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.622+0000 7f71b4b26700 1 -- 172.21.15.143:0/1681604039 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f719006af00 msgr2=0x7f719006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:59.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.622+0000 7f71b4b26700 1 --2- 172.21.15.143:0/1681604039 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f719006af00 0x7f719006d3c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f71b00c5810 tx=0x7f71a0008030 comp rx=0 tx=0).stop 2023-11-08T19:24:59.626 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.622+0000 7f71b4b26700 1 -- 172.21.15.143:0/1681604039 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f71b006b150 msgr2=0x7f71b014f980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:59.626 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.622+0000 7f71b4b26700 1 --2- 172.21.15.143:0/1681604039 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f71b006b150 0x7f71b014f980 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f71a4001260 tx=0x7f71a400ca90 comp rx=0 tx=0).stop 2023-11-08T19:24:59.626 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.622+0000 7f71b4b26700 1 -- 172.21.15.143:0/1681604039 shutdown_connections 2023-11-08T19:24:59.626 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.622+0000 7f71b4b26700 1 --2- 172.21.15.143:0/1681604039 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f719006af00 0x7f719006d3c0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:59.627 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.622+0000 7f71b4b26700 1 --2- 172.21.15.143:0/1681604039 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f71b01618f0 0x7f71b01f2d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:59.627 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.622+0000 7f71b4b26700 1 --2- 172.21.15.143:0/1681604039 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f71b006b150 0x7f71b014f980 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:59.627 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.622+0000 7f71b4b26700 1 --2- 172.21.15.143:0/1681604039 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f71b015dd00 0x7f71b00c77d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:59.627 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.622+0000 7f71b4b26700 1 -- 172.21.15.143:0/1681604039 >> 172.21.15.143:0/1681604039 conn(0x7f71b0099ab0 msgr2=0x7f71b006f4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:24:59.627 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.622+0000 7f71b4b26700 1 -- 172.21.15.143:0/1681604039 shutdown_connections 2023-11-08T19:24:59.627 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.622+0000 7f71b4b26700 1 -- 172.21.15.143:0/1681604039 wait complete. 2023-11-08T19:24:59.628 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2023-11-08T19:24:59.635 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2023-11-08T19:24:58.510488+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":188,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219008,"kb_used_data":2812,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755712,"statfs":{"total":773094113280,"available":772869849088,"internally_reserved":0,"allocated":2879488,"data_stored":1746536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001827"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.631402+0000","last_change":"2023-11-08T19:24:50.618820+0000","last_active":"2023-11-08T19:24:51.631402+0000","last_peered":"2023-11-08T19:24:51.631402+0000","last_clean":"2023-11-08T19:24:51.631402+0000","last_became_active":"2023-11-08T19:24:50.618531+0000","last_became_peered":"2023-11-08T19:24:50.618531+0000","last_unstale":"2023-11-08T19:24:51.631402+0000","last_undegraded":"2023-11-08T19:24:51.631402+0000","last_fullsized":"2023-11-08T19:24:51.631402+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:33:19.902860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.805606+0000","last_change":"2023-11-08T19:24:50.623985+0000","last_active":"2023-11-08T19:24:52.805606+0000","last_peered":"2023-11-08T19:24:52.805606+0000","last_clean":"2023-11-08T19:24:52.805606+0000","last_became_active":"2023-11-08T19:24:50.623553+0000","last_became_peered":"2023-11-08T19:24:50.623553+0000","last_unstale":"2023-11-08T19:24:52.805606+0000","last_undegraded":"2023-11-08T19:24:52.805606+0000","last_fullsized":"2023-11-08T19:24:52.805606+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:13:58.948371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.309774+0000","last_change":"2023-11-08T19:24:50.617914+0000","last_active":"2023-11-08T19:24:52.309774+0000","last_peered":"2023-11-08T19:24:52.309774+0000","last_clean":"2023-11-08T19:24:52.309774+0000","last_became_active":"2023-11-08T19:24:50.617611+0000","last_became_peered":"2023-11-08T19:24:50.617611+0000","last_unstale":"2023-11-08T19:24:52.309774+0000","last_undegraded":"2023-11-08T19:24:52.309774+0000","last_fullsized":"2023-11-08T19:24:52.309774+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:30:36.041820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.352504+0000","last_change":"2023-11-08T19:24:50.640039+0000","last_active":"2023-11-08T19:24:52.352504+0000","last_peered":"2023-11-08T19:24:52.352504+0000","last_clean":"2023-11-08T19:24:52.352504+0000","last_became_active":"2023-11-08T19:24:50.639560+0000","last_became_peered":"2023-11-08T19:24:50.639560+0000","last_unstale":"2023-11-08T19:24:52.352504+0000","last_undegraded":"2023-11-08T19:24:52.352504+0000","last_fullsized":"2023-11-08T19:24:52.352504+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:01:53.410545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.631276+0000","last_change":"2023-11-08T19:24:50.617837+0000","last_active":"2023-11-08T19:24:51.631276+0000","last_peered":"2023-11-08T19:24:51.631276+0000","last_clean":"2023-11-08T19:24:51.631276+0000","last_became_active":"2023-11-08T19:24:50.617537+0000","last_became_peered":"2023-11-08T19:24:50.617537+0000","last_unstale":"2023-11-08T19:24:51.631276+0000","last_undegraded":"2023-11-08T19:24:51.631276+0000","last_fullsized":"2023-11-08T19:24:51.631276+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:59:37.592538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.351953+0000","last_change":"2023-11-08T19:24:50.641005+0000","last_active":"2023-11-08T19:24:52.351953+0000","last_peered":"2023-11-08T19:24:52.351953+0000","last_clean":"2023-11-08T19:24:52.351953+0000","last_became_active":"2023-11-08T19:24:50.640727+0000","last_became_peered":"2023-11-08T19:24:50.640727+0000","last_unstale":"2023-11-08T19:24:52.351953+0000","last_undegraded":"2023-11-08T19:24:52.351953+0000","last_fullsized":"2023-11-08T19:24:52.351953+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:36:53.967796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.630219+0000","last_change":"2023-11-08T19:24:50.625311+0000","last_active":"2023-11-08T19:24:51.630219+0000","last_peered":"2023-11-08T19:24:51.630219+0000","last_clean":"2023-11-08T19:24:51.630219+0000","last_became_active":"2023-11-08T19:24:50.625012+0000","last_became_peered":"2023-11-08T19:24:50.625012+0000","last_unstale":"2023-11-08T19:24:51.630219+0000","last_undegraded":"2023-11-08T19:24:51.630219+0000","last_fullsized":"2023-11-08T19:24:51.630219+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:49:19.100370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.626351+0000","last_change":"2023-11-08T19:24:50.639325+0000","last_active":"2023-11-08T19:24:51.626351+0000","last_peered":"2023-11-08T19:24:51.626351+0000","last_clean":"2023-11-08T19:24:51.626351+0000","last_became_active":"2023-11-08T19:24:50.638993+0000","last_became_peered":"2023-11-08T19:24:50.638993+0000","last_unstale":"2023-11-08T19:24:51.626351+0000","last_undegraded":"2023-11-08T19:24:51.626351+0000","last_fullsized":"2023-11-08T19:24:51.626351+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:30:32.602976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.631383+0000","last_change":"2023-11-08T19:24:50.638920+0000","last_active":"2023-11-08T19:24:51.631383+0000","last_peered":"2023-11-08T19:24:51.631383+0000","last_clean":"2023-11-08T19:24:51.631383+0000","last_became_active":"2023-11-08T19:24:50.638607+0000","last_became_peered":"2023-11-08T19:24:50.638607+0000","last_unstale":"2023-11-08T19:24:51.631383+0000","last_undegraded":"2023-11-08T19:24:51.631383+0000","last_fullsized":"2023-11-08T19:24:51.631383+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:09:01.749686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.630901+0000","last_change":"2023-11-08T19:24:50.637631+0000","last_active":"2023-11-08T19:24:51.630901+0000","last_peered":"2023-11-08T19:24:51.630901+0000","last_clean":"2023-11-08T19:24:51.630901+0000","last_became_active":"2023-11-08T19:24:50.637346+0000","last_became_peered":"2023-11-08T19:24:50.637346+0000","last_unstale":"2023-11-08T19:24:51.630901+0000","last_undegraded":"2023-11-08T19:24:51.630901+0000","last_fullsized":"2023-11-08T19:24:51.630901+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:13:58.948371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.630498+0000","last_change":"2023-11-08T19:24:50.642227+0000","last_active":"2023-11-08T19:24:51.630498+0000","last_peered":"2023-11-08T19:24:51.630498+0000","last_clean":"2023-11-08T19:24:51.630498+0000","last_became_active":"2023-11-08T19:24:50.641955+0000","last_became_peered":"2023-11-08T19:24:50.641955+0000","last_unstale":"2023-11-08T19:24:51.630498+0000","last_undegraded":"2023-11-08T19:24:51.630498+0000","last_fullsized":"2023-11-08T19:24:51.630498+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:40:11.978983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.805767+0000","last_change":"2023-11-08T19:24:50.621511+0000","last_active":"2023-11-08T19:24:52.805767+0000","last_peered":"2023-11-08T19:24:52.805767+0000","last_clean":"2023-11-08T19:24:52.805767+0000","last_became_active":"2023-11-08T19:24:50.621142+0000","last_became_peered":"2023-11-08T19:24:50.621142+0000","last_unstale":"2023-11-08T19:24:52.805767+0000","last_undegraded":"2023-11-08T19:24:52.805767+0000","last_fullsized":"2023-11-08T19:24:52.805767+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:14:01.459274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.626625+0000","last_change":"2023-11-08T19:24:50.625632+0000","last_active":"2023-11-08T19:24:51.626625+0000","last_peered":"2023-11-08T19:24:51.626625+0000","last_clean":"2023-11-08T19:24:51.626625+0000","last_became_active":"2023-11-08T19:24:50.625370+0000","last_became_peered":"2023-11-08T19:24:50.625370+0000","last_unstale":"2023-11-08T19:24:51.626625+0000","last_undegraded":"2023-11-08T19:24:51.626625+0000","last_fullsized":"2023-11-08T19:24:51.626625+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:13:58.948371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.309818+0000","last_change":"2023-11-08T19:24:50.625350+0000","last_active":"2023-11-08T19:24:52.309818+0000","last_peered":"2023-11-08T19:24:52.309818+0000","last_clean":"2023-11-08T19:24:52.309818+0000","last_became_active":"2023-11-08T19:24:50.625078+0000","last_became_peered":"2023-11-08T19:24:50.625078+0000","last_unstale":"2023-11-08T19:24:52.309818+0000","last_undegraded":"2023-11-08T19:24:52.309818+0000","last_fullsized":"2023-11-08T19:24:52.309818+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:42:42.512786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.804817+0000","last_change":"2023-11-08T19:24:50.629187+0000","last_active":"2023-11-08T19:24:52.804817+0000","last_peered":"2023-11-08T19:24:52.804817+0000","last_clean":"2023-11-08T19:24:52.804817+0000","last_became_active":"2023-11-08T19:24:50.628898+0000","last_became_peered":"2023-11-08T19:24:50.628898+0000","last_unstale":"2023-11-08T19:24:52.804817+0000","last_undegraded":"2023-11-08T19:24:52.804817+0000","last_fullsized":"2023-11-08T19:24:52.804817+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:15:33.285449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.351906+0000","last_change":"2023-11-08T19:24:50.638579+0000","last_active":"2023-11-08T19:24:52.351906+0000","last_peered":"2023-11-08T19:24:52.351906+0000","last_clean":"2023-11-08T19:24:52.351906+0000","last_became_active":"2023-11-08T19:24:50.638297+0000","last_became_peered":"2023-11-08T19:24:50.638297+0000","last_unstale":"2023-11-08T19:24:52.351906+0000","last_undegraded":"2023-11-08T19:24:52.351906+0000","last_fullsized":"2023-11-08T19:24:52.351906+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:13:58.948371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.352572+0000","last_change":"2023-11-08T19:24:50.641446+0000","last_active":"2023-11-08T19:24:52.352572+0000","last_peered":"2023-11-08T19:24:52.352572+0000","last_clean":"2023-11-08T19:24:52.352572+0000","last_became_active":"2023-11-08T19:24:50.639709+0000","last_became_peered":"2023-11-08T19:24:50.639709+0000","last_unstale":"2023-11-08T19:24:52.352572+0000","last_undegraded":"2023-11-08T19:24:52.352572+0000","last_fullsized":"2023-11-08T19:24:52.352572+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:09:01.749686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.804922+0000","last_change":"2023-11-08T19:24:50.619241+0000","last_active":"2023-11-08T19:24:52.804922+0000","last_peered":"2023-11-08T19:24:52.804922+0000","last_clean":"2023-11-08T19:24:52.804922+0000","last_became_active":"2023-11-08T19:24:50.618962+0000","last_became_peered":"2023-11-08T19:24:50.618962+0000","last_unstale":"2023-11-08T19:24:52.804922+0000","last_undegraded":"2023-11-08T19:24:52.804922+0000","last_fullsized":"2023-11-08T19:24:52.804922+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:14:01.459274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.631027+0000","last_change":"2023-11-08T19:24:50.618043+0000","last_active":"2023-11-08T19:24:51.631027+0000","last_peered":"2023-11-08T19:24:51.631027+0000","last_clean":"2023-11-08T19:24:51.631027+0000","last_became_active":"2023-11-08T19:24:50.617727+0000","last_became_peered":"2023-11-08T19:24:50.617727+0000","last_unstale":"2023-11-08T19:24:51.631027+0000","last_undegraded":"2023-11-08T19:24:51.631027+0000","last_fullsized":"2023-11-08T19:24:51.631027+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:13:41.469149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.309784+0000","last_change":"2023-11-08T19:24:49.608481+0000","last_active":"2023-11-08T19:24:52.309784+0000","last_peered":"2023-11-08T19:24:52.309784+0000","last_clean":"2023-11-08T19:24:52.309784+0000","last_became_active":"2023-11-08T19:24:49.607778+0000","last_became_peered":"2023-11-08T19:24:49.607778+0000","last_unstale":"2023-11-08T19:24:52.309784+0000","last_undegraded":"2023-11-08T19:24:52.309784+0000","last_fullsized":"2023-11-08T19:24:52.309784+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:01:09.884262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.626900+0000","last_change":"2023-11-08T19:24:50.624852+0000","last_active":"2023-11-08T19:24:51.626900+0000","last_peered":"2023-11-08T19:24:51.626900+0000","last_clean":"2023-11-08T19:24:51.626900+0000","last_became_active":"2023-11-08T19:24:50.624551+0000","last_became_peered":"2023-11-08T19:24:50.624551+0000","last_unstale":"2023-11-08T19:24:51.626900+0000","last_undegraded":"2023-11-08T19:24:51.626900+0000","last_fullsized":"2023-11-08T19:24:51.626900+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:12:54.641952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.626938+0000","last_change":"2023-11-08T19:24:49.610418+0000","last_active":"2023-11-08T19:24:51.626938+0000","last_peered":"2023-11-08T19:24:51.626938+0000","last_clean":"2023-11-08T19:24:51.626938+0000","last_became_active":"2023-11-08T19:24:49.610041+0000","last_became_peered":"2023-11-08T19:24:49.610041+0000","last_unstale":"2023-11-08T19:24:51.626938+0000","last_undegraded":"2023-11-08T19:24:51.626938+0000","last_fullsized":"2023-11-08T19:24:51.626938+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:00:12.067877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.309779+0000","last_change":"2023-11-08T19:24:50.627052+0000","last_active":"2023-11-08T19:24:52.309779+0000","last_peered":"2023-11-08T19:24:52.309779+0000","last_clean":"2023-11-08T19:24:52.309779+0000","last_became_active":"2023-11-08T19:24:50.626751+0000","last_became_peered":"2023-11-08T19:24:50.626751+0000","last_unstale":"2023-11-08T19:24:52.309779+0000","last_undegraded":"2023-11-08T19:24:52.309779+0000","last_fullsized":"2023-11-08T19:24:52.309779+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:46:19.375897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-08T19:24:50.639795+0000","last_change":"2023-11-08T19:24:49.605677+0000","last_active":"2023-11-08T19:24:50.639795+0000","last_peered":"2023-11-08T19:24:50.639795+0000","last_clean":"2023-11-08T19:24:50.639795+0000","last_became_active":"2023-11-08T19:24:49.605399+0000","last_became_peered":"2023-11-08T19:24:49.605399+0000","last_unstale":"2023-11-08T19:24:50.639795+0000","last_undegraded":"2023-11-08T19:24:50.639795+0000","last_fullsized":"2023-11-08T19:24:50.639795+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:46:18.372121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.352125+0000","last_change":"2023-11-08T19:24:50.641183+0000","last_active":"2023-11-08T19:24:52.352125+0000","last_peered":"2023-11-08T19:24:52.352125+0000","last_clean":"2023-11-08T19:24:52.352125+0000","last_became_active":"2023-11-08T19:24:50.640910+0000","last_became_peered":"2023-11-08T19:24:50.640910+0000","last_unstale":"2023-11-08T19:24:52.352125+0000","last_undegraded":"2023-11-08T19:24:52.352125+0000","last_fullsized":"2023-11-08T19:24:52.352125+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:49:19.100370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-08T19:24:50.642030+0000","last_change":"2023-11-08T19:24:49.614714+0000","last_active":"2023-11-08T19:24:50.642030+0000","last_peered":"2023-11-08T19:24:50.642030+0000","last_clean":"2023-11-08T19:24:50.642030+0000","last_became_active":"2023-11-08T19:24:49.614390+0000","last_became_peered":"2023-11-08T19:24:49.614390+0000","last_unstale":"2023-11-08T19:24:50.642030+0000","last_undegraded":"2023-11-08T19:24:50.642030+0000","last_fullsized":"2023-11-08T19:24:50.642030+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:46:12.493411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.352284+0000","last_change":"2023-11-08T19:24:50.640245+0000","last_active":"2023-11-08T19:24:52.352284+0000","last_peered":"2023-11-08T19:24:52.352284+0000","last_clean":"2023-11-08T19:24:52.352284+0000","last_became_active":"2023-11-08T19:24:50.639668+0000","last_became_peered":"2023-11-08T19:24:50.639668+0000","last_unstale":"2023-11-08T19:24:52.352284+0000","last_undegraded":"2023-11-08T19:24:52.352284+0000","last_fullsized":"2023-11-08T19:24:52.352284+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:42:27.150066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.626599+0000","last_change":"2023-11-08T19:24:49.612665+0000","last_active":"2023-11-08T19:24:51.626599+0000","last_peered":"2023-11-08T19:24:51.626599+0000","last_clean":"2023-11-08T19:24:51.626599+0000","last_became_active":"2023-11-08T19:24:49.612362+0000","last_became_peered":"2023-11-08T19:24:49.612362+0000","last_unstale":"2023-11-08T19:24:51.626599+0000","last_undegraded":"2023-11-08T19:24:51.626599+0000","last_fullsized":"2023-11-08T19:24:51.626599+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:46:12.493411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.805419+0000","last_change":"2023-11-08T19:24:50.627537+0000","last_active":"2023-11-08T19:24:52.805419+0000","last_peered":"2023-11-08T19:24:52.805419+0000","last_clean":"2023-11-08T19:24:52.805419+0000","last_became_active":"2023-11-08T19:24:50.626489+0000","last_became_peered":"2023-11-08T19:24:50.626489+0000","last_unstale":"2023-11-08T19:24:52.805419+0000","last_undegraded":"2023-11-08T19:24:52.805419+0000","last_fullsized":"2023-11-08T19:24:52.805419+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:01:10.888039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-08T19:24:50.637474+0000","last_change":"2023-11-08T19:24:49.606613+0000","last_active":"2023-11-08T19:24:50.637474+0000","last_peered":"2023-11-08T19:24:50.637474+0000","last_clean":"2023-11-08T19:24:50.637474+0000","last_became_active":"2023-11-08T19:24:49.606324+0000","last_became_peered":"2023-11-08T19:24:49.606324+0000","last_unstale":"2023-11-08T19:24:50.637474+0000","last_undegraded":"2023-11-08T19:24:50.637474+0000","last_fullsized":"2023-11-08T19:24:50.637474+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:07:00.575561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.309935+0000","last_change":"2023-11-08T19:24:50.627987+0000","last_active":"2023-11-08T19:24:52.309935+0000","last_peered":"2023-11-08T19:24:52.309935+0000","last_clean":"2023-11-08T19:24:52.309935+0000","last_became_active":"2023-11-08T19:24:50.627402+0000","last_became_peered":"2023-11-08T19:24:50.627402+0000","last_unstale":"2023-11-08T19:24:52.309935+0000","last_undegraded":"2023-11-08T19:24:52.309935+0000","last_fullsized":"2023-11-08T19:24:52.309935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:36:53.967796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.626523+0000","last_change":"2023-11-08T19:24:49.606482+0000","last_active":"2023-11-08T19:24:51.626523+0000","last_peered":"2023-11-08T19:24:51.626523+0000","last_clean":"2023-11-08T19:24:51.626523+0000","last_became_active":"2023-11-08T19:24:49.606211+0000","last_became_peered":"2023-11-08T19:24:49.606211+0000","last_unstale":"2023-11-08T19:24:51.626523+0000","last_undegraded":"2023-11-08T19:24:51.626523+0000","last_fullsized":"2023-11-08T19:24:51.626523+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:42:59.845581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.626483+0000","last_change":"2023-11-08T19:24:50.625917+0000","last_active":"2023-11-08T19:24:51.626483+0000","last_peered":"2023-11-08T19:24:51.626483+0000","last_clean":"2023-11-08T19:24:51.626483+0000","last_became_active":"2023-11-08T19:24:50.625398+0000","last_became_peered":"2023-11-08T19:24:50.625398+0000","last_unstale":"2023-11-08T19:24:51.626483+0000","last_undegraded":"2023-11-08T19:24:51.626483+0000","last_fullsized":"2023-11-08T19:24:51.626483+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:40:11.978983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.351905+0000","last_change":"2023-11-08T19:24:49.616558+0000","last_active":"2023-11-08T19:24:52.351905+0000","last_peered":"2023-11-08T19:24:52.351905+0000","last_clean":"2023-11-08T19:24:52.351905+0000","last_became_active":"2023-11-08T19:24:49.616277+0000","last_became_peered":"2023-11-08T19:24:49.616277+0000","last_unstale":"2023-11-08T19:24:52.351905+0000","last_undegraded":"2023-11-08T19:24:52.351905+0000","last_fullsized":"2023-11-08T19:24:52.351905+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:01:09.884262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.805048+0000","last_change":"2023-11-08T19:24:50.623658+0000","last_active":"2023-11-08T19:24:52.805048+0000","last_peered":"2023-11-08T19:24:52.805048+0000","last_clean":"2023-11-08T19:24:52.805048+0000","last_became_active":"2023-11-08T19:24:50.623399+0000","last_became_peered":"2023-11-08T19:24:50.623399+0000","last_unstale":"2023-11-08T19:24:52.805048+0000","last_undegraded":"2023-11-08T19:24:52.805048+0000","last_fullsized":"2023-11-08T19:24:52.805048+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:40:11.978983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-08T19:24:50.610454+0000","last_change":"2023-11-08T19:24:49.608767+0000","last_active":"2023-11-08T19:24:50.610454+0000","last_peered":"2023-11-08T19:24:50.610454+0000","last_clean":"2023-11-08T19:24:50.610454+0000","last_became_active":"2023-11-08T19:24:49.608450+0000","last_became_peered":"2023-11-08T19:24:49.608450+0000","last_unstale":"2023-11-08T19:24:50.610454+0000","last_undegraded":"2023-11-08T19:24:50.610454+0000","last_fullsized":"2023-11-08T19:24:50.610454+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:42:59.845581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.352305+0000","last_change":"2023-11-08T19:24:50.640397+0000","last_active":"2023-11-08T19:24:52.352305+0000","last_peered":"2023-11-08T19:24:52.352305+0000","last_clean":"2023-11-08T19:24:52.352305+0000","last_became_active":"2023-11-08T19:24:50.639730+0000","last_became_peered":"2023-11-08T19:24:50.639730+0000","last_unstale":"2023-11-08T19:24:52.352305+0000","last_undegraded":"2023-11-08T19:24:52.352305+0000","last_fullsized":"2023-11-08T19:24:52.352305+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:08:22.499163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.352212+0000","last_change":"2023-11-08T19:24:49.611202+0000","last_active":"2023-11-08T19:24:52.352212+0000","last_peered":"2023-11-08T19:24:52.352212+0000","last_clean":"2023-11-08T19:24:52.352212+0000","last_became_active":"2023-11-08T19:24:49.610894+0000","last_became_peered":"2023-11-08T19:24:49.610894+0000","last_unstale":"2023-11-08T19:24:52.352212+0000","last_undegraded":"2023-11-08T19:24:52.352212+0000","last_fullsized":"2023-11-08T19:24:52.352212+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:48:25.546248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.309935+0000","last_change":"2023-11-08T19:24:50.638901+0000","last_active":"2023-11-08T19:24:52.309935+0000","last_peered":"2023-11-08T19:24:52.309935+0000","last_clean":"2023-11-08T19:24:52.309935+0000","last_became_active":"2023-11-08T19:24:50.638587+0000","last_became_peered":"2023-11-08T19:24:50.638587+0000","last_unstale":"2023-11-08T19:24:52.309935+0000","last_undegraded":"2023-11-08T19:24:52.309935+0000","last_fullsized":"2023-11-08T19:24:52.309935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:13:58.948371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.805313+0000","last_change":"2023-11-08T19:24:49.612170+0000","last_active":"2023-11-08T19:24:52.805313+0000","last_peered":"2023-11-08T19:24:52.805313+0000","last_clean":"2023-11-08T19:24:52.805313+0000","last_became_active":"2023-11-08T19:24:49.611895+0000","last_became_peered":"2023-11-08T19:24:49.611895+0000","last_unstale":"2023-11-08T19:24:52.805313+0000","last_undegraded":"2023-11-08T19:24:52.805313+0000","last_fullsized":"2023-11-08T19:24:52.805313+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:38:24.246638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.805721+0000","last_change":"2023-11-08T19:24:50.627060+0000","last_active":"2023-11-08T19:24:52.805721+0000","last_peered":"2023-11-08T19:24:52.805721+0000","last_clean":"2023-11-08T19:24:52.805721+0000","last_became_active":"2023-11-08T19:24:50.626468+0000","last_became_peered":"2023-11-08T19:24:50.626468+0000","last_unstale":"2023-11-08T19:24:52.805721+0000","last_undegraded":"2023-11-08T19:24:52.805721+0000","last_fullsized":"2023-11-08T19:24:52.805721+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:00:13.071654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":18,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-08T19:24:50.645041+0000","last_change":"2023-11-08T19:24:49.610676+0000","last_active":"2023-11-08T19:24:50.645041+0000","last_peered":"2023-11-08T19:24:50.645041+0000","last_clean":"2023-11-08T19:24:50.645041+0000","last_became_active":"2023-11-08T19:24:49.610344+0000","last_became_peered":"2023-11-08T19:24:49.610344+0000","last_unstale":"2023-11-08T19:24:50.645041+0000","last_undegraded":"2023-11-08T19:24:50.645041+0000","last_fullsized":"2023-11-08T19:24:50.645041+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:46:12.493411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.631028+0000","last_change":"2023-11-08T19:24:50.626565+0000","last_active":"2023-11-08T19:24:51.631028+0000","last_peered":"2023-11-08T19:24:51.631028+0000","last_clean":"2023-11-08T19:24:51.631028+0000","last_became_active":"2023-11-08T19:24:50.626280+0000","last_became_peered":"2023-11-08T19:24:50.626280+0000","last_unstale":"2023-11-08T19:24:51.631028+0000","last_undegraded":"2023-11-08T19:24:51.631028+0000","last_fullsized":"2023-11-08T19:24:51.631028+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:08:22.499163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.309884+0000","last_change":"2023-11-08T19:24:49.610471+0000","last_active":"2023-11-08T19:24:52.309884+0000","last_peered":"2023-11-08T19:24:52.309884+0000","last_clean":"2023-11-08T19:24:52.309884+0000","last_became_active":"2023-11-08T19:24:49.610182+0000","last_became_peered":"2023-11-08T19:24:49.610182+0000","last_unstale":"2023-11-08T19:24:52.309884+0000","last_undegraded":"2023-11-08T19:24:52.309884+0000","last_fullsized":"2023-11-08T19:24:52.309884+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:14:00.455497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.309602+0000","last_change":"2023-11-08T19:24:50.619498+0000","last_active":"2023-11-08T19:24:52.309602+0000","last_peered":"2023-11-08T19:24:52.309602+0000","last_clean":"2023-11-08T19:24:52.309602+0000","last_became_active":"2023-11-08T19:24:50.619231+0000","last_became_peered":"2023-11-08T19:24:50.619231+0000","last_unstale":"2023-11-08T19:24:52.309602+0000","last_undegraded":"2023-11-08T19:24:52.309602+0000","last_fullsized":"2023-11-08T19:24:52.309602+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:38:25.250414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.352455+0000","last_change":"2023-11-08T19:24:49.612336+0000","last_active":"2023-11-08T19:24:52.352455+0000","last_peered":"2023-11-08T19:24:52.352455+0000","last_clean":"2023-11-08T19:24:52.352455+0000","last_became_active":"2023-11-08T19:24:49.611885+0000","last_became_peered":"2023-11-08T19:24:49.611885+0000","last_unstale":"2023-11-08T19:24:52.352455+0000","last_undegraded":"2023-11-08T19:24:52.352455+0000","last_fullsized":"2023-11-08T19:24:52.352455+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:46:18.372121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.631204+0000","last_change":"2023-11-08T19:24:50.629107+0000","last_active":"2023-11-08T19:24:51.631204+0000","last_peered":"2023-11-08T19:24:51.631204+0000","last_clean":"2023-11-08T19:24:51.631204+0000","last_became_active":"2023-11-08T19:24:50.628005+0000","last_became_peered":"2023-11-08T19:24:50.628005+0000","last_unstale":"2023-11-08T19:24:51.631204+0000","last_undegraded":"2023-11-08T19:24:51.631204+0000","last_fullsized":"2023-11-08T19:24:51.631204+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:34:42.075394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-08T19:24:50.609767+0000","last_change":"2023-11-08T19:24:49.605863+0000","last_active":"2023-11-08T19:24:50.609767+0000","last_peered":"2023-11-08T19:24:50.609767+0000","last_clean":"2023-11-08T19:24:50.609767+0000","last_became_active":"2023-11-08T19:24:49.604986+0000","last_became_peered":"2023-11-08T19:24:49.604986+0000","last_unstale":"2023-11-08T19:24:50.609767+0000","last_undegraded":"2023-11-08T19:24:50.609767+0000","last_fullsized":"2023-11-08T19:24:50.609767+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:15:32.030617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.629786+0000","last_change":"2023-11-08T19:24:50.618716+0000","last_active":"2023-11-08T19:24:51.629786+0000","last_peered":"2023-11-08T19:24:51.629786+0000","last_clean":"2023-11-08T19:24:51.629786+0000","last_became_active":"2023-11-08T19:24:50.618425+0000","last_became_peered":"2023-11-08T19:24:50.618425+0000","last_unstale":"2023-11-08T19:24:51.629786+0000","last_undegraded":"2023-11-08T19:24:51.629786+0000","last_fullsized":"2023-11-08T19:24:51.629786+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:15:33.285449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.804637+0000","last_change":"2023-11-08T19:24:49.608503+0000","last_active":"2023-11-08T19:24:52.804637+0000","last_peered":"2023-11-08T19:24:52.804637+0000","last_clean":"2023-11-08T19:24:52.804637+0000","last_became_active":"2023-11-08T19:24:49.608176+0000","last_became_peered":"2023-11-08T19:24:49.608176+0000","last_unstale":"2023-11-08T19:24:52.804637+0000","last_undegraded":"2023-11-08T19:24:52.804637+0000","last_fullsized":"2023-11-08T19:24:52.804637+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:17:50.005667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.630752+0000","last_change":"2023-11-08T19:24:50.629255+0000","last_active":"2023-11-08T19:24:51.630752+0000","last_peered":"2023-11-08T19:24:51.630752+0000","last_clean":"2023-11-08T19:24:51.630752+0000","last_became_active":"2023-11-08T19:24:50.628763+0000","last_became_peered":"2023-11-08T19:24:50.628763+0000","last_unstale":"2023-11-08T19:24:51.630752+0000","last_undegraded":"2023-11-08T19:24:51.630752+0000","last_fullsized":"2023-11-08T19:24:51.630752+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:42:27.150066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.805855+0000","last_change":"2023-11-08T19:24:49.610638+0000","last_active":"2023-11-08T19:24:52.805855+0000","last_peered":"2023-11-08T19:24:52.805855+0000","last_clean":"2023-11-08T19:24:52.805855+0000","last_became_active":"2023-11-08T19:24:49.610362+0000","last_became_peered":"2023-11-08T19:24:49.610362+0000","last_unstale":"2023-11-08T19:24:52.805855+0000","last_undegraded":"2023-11-08T19:24:52.805855+0000","last_fullsized":"2023-11-08T19:24:52.805855+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:15:32.030617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.631275+0000","last_change":"2023-11-08T19:24:50.627667+0000","last_active":"2023-11-08T19:24:51.631275+0000","last_peered":"2023-11-08T19:24:51.631275+0000","last_clean":"2023-11-08T19:24:51.631275+0000","last_became_active":"2023-11-08T19:24:50.627390+0000","last_became_peered":"2023-11-08T19:24:50.627390+0000","last_unstale":"2023-11-08T19:24:51.631275+0000","last_undegraded":"2023-11-08T19:24:51.631275+0000","last_fullsized":"2023-11-08T19:24:51.631275+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:01:53.410545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.805154+0000","last_change":"2023-11-08T19:24:49.613413+0000","last_active":"2023-11-08T19:24:52.805154+0000","last_peered":"2023-11-08T19:24:52.805154+0000","last_clean":"2023-11-08T19:24:52.805154+0000","last_became_active":"2023-11-08T19:24:49.613131+0000","last_became_peered":"2023-11-08T19:24:49.613131+0000","last_unstale":"2023-11-08T19:24:52.805154+0000","last_undegraded":"2023-11-08T19:24:52.805154+0000","last_fullsized":"2023-11-08T19:24:52.805154+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:46:12.493411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.352374+0000","last_change":"2023-11-08T19:24:50.640031+0000","last_active":"2023-11-08T19:24:52.352374+0000","last_peered":"2023-11-08T19:24:52.352374+0000","last_clean":"2023-11-08T19:24:52.352374+0000","last_became_active":"2023-11-08T19:24:50.639718+0000","last_became_peered":"2023-11-08T19:24:50.639718+0000","last_unstale":"2023-11-08T19:24:52.352374+0000","last_undegraded":"2023-11-08T19:24:52.352374+0000","last_fullsized":"2023-11-08T19:24:52.352374+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:08:17.096077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.352346+0000","last_change":"2023-11-08T19:24:49.612137+0000","last_active":"2023-11-08T19:24:52.352346+0000","last_peered":"2023-11-08T19:24:52.352346+0000","last_clean":"2023-11-08T19:24:52.352346+0000","last_became_active":"2023-11-08T19:24:49.611532+0000","last_became_peered":"2023-11-08T19:24:49.611532+0000","last_unstale":"2023-11-08T19:24:52.352346+0000","last_undegraded":"2023-11-08T19:24:52.352346+0000","last_fullsized":"2023-11-08T19:24:52.352346+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:40:10.975206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.805536+0000","last_change":"2023-11-08T19:24:50.630854+0000","last_active":"2023-11-08T19:24:52.805536+0000","last_peered":"2023-11-08T19:24:52.805536+0000","last_clean":"2023-11-08T19:24:52.805536+0000","last_became_active":"2023-11-08T19:24:50.630549+0000","last_became_peered":"2023-11-08T19:24:50.630549+0000","last_unstale":"2023-11-08T19:24:52.805536+0000","last_undegraded":"2023-11-08T19:24:52.805536+0000","last_fullsized":"2023-11-08T19:24:52.805536+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:48:26.550025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.804950+0000","last_change":"2023-11-08T19:24:50.620897+0000","last_active":"2023-11-08T19:24:52.804950+0000","last_peered":"2023-11-08T19:24:52.804950+0000","last_clean":"2023-11-08T19:24:52.804950+0000","last_became_active":"2023-11-08T19:24:50.620606+0000","last_became_peered":"2023-11-08T19:24:50.620606+0000","last_unstale":"2023-11-08T19:24:52.804950+0000","last_undegraded":"2023-11-08T19:24:52.804950+0000","last_fullsized":"2023-11-08T19:24:52.804950+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:30:36.041820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.804625+0000","last_change":"2023-11-08T19:24:49.612840+0000","last_active":"2023-11-08T19:24:52.804625+0000","last_peered":"2023-11-08T19:24:52.804625+0000","last_clean":"2023-11-08T19:24:52.804625+0000","last_became_active":"2023-11-08T19:24:49.612652+0000","last_became_peered":"2023-11-08T19:24:49.612652+0000","last_unstale":"2023-11-08T19:24:52.804625+0000","last_undegraded":"2023-11-08T19:24:52.804625+0000","last_fullsized":"2023-11-08T19:24:52.804625+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:48:25.546248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.626593+0000","last_change":"2023-11-08T19:24:50.642515+0000","last_active":"2023-11-08T19:24:51.626593+0000","last_peered":"2023-11-08T19:24:51.626593+0000","last_clean":"2023-11-08T19:24:51.626593+0000","last_became_active":"2023-11-08T19:24:50.642204+0000","last_became_peered":"2023-11-08T19:24:50.642204+0000","last_unstale":"2023-11-08T19:24:51.626593+0000","last_undegraded":"2023-11-08T19:24:51.626593+0000","last_fullsized":"2023-11-08T19:24:51.626593+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:42:42.512786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.351949+0000","last_change":"2023-11-08T19:24:49.605594+0000","last_active":"2023-11-08T19:24:52.351949+0000","last_peered":"2023-11-08T19:24:52.351949+0000","last_clean":"2023-11-08T19:24:52.351949+0000","last_became_active":"2023-11-08T19:24:49.605208+0000","last_became_peered":"2023-11-08T19:24:49.605208+0000","last_unstale":"2023-11-08T19:24:52.351949+0000","last_undegraded":"2023-11-08T19:24:52.351949+0000","last_fullsized":"2023-11-08T19:24:52.351949+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:17:50.005667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":165,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-08T19:24:50.613237+0000","last_change":"2023-11-08T19:24:45.888122+0000","last_active":"2023-11-08T19:24:50.613237+0000","last_peered":"2023-11-08T19:24:50.613237+0000","last_clean":"2023-11-08T19:24:50.613237+0000","last_became_active":"2023-11-08T19:24:45.887048+0000","last_became_peered":"2023-11-08T19:24:45.887048+0000","last_unstale":"2023-11-08T19:24:50.613237+0000","last_undegraded":"2023-11-08T19:24:50.613237+0000","last_fullsized":"2023-11-08T19:24:50.613237+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:44.575133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:44.575133+0000","last_clean_scrub_stamp":"2023-11-08T19:24:44.575133+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:19:20.202062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.309622+0000","last_change":"2023-11-08T19:24:50.628353+0000","last_active":"2023-11-08T19:24:52.309622+0000","last_peered":"2023-11-08T19:24:52.309622+0000","last_clean":"2023-11-08T19:24:52.309622+0000","last_became_active":"2023-11-08T19:24:50.628008+0000","last_became_peered":"2023-11-08T19:24:50.628008+0000","last_unstale":"2023-11-08T19:24:52.309622+0000","last_undegraded":"2023-11-08T19:24:52.309622+0000","last_fullsized":"2023-11-08T19:24:52.309622+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:46:13.497187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-08T19:24:50.612010+0000","last_change":"2023-11-08T19:24:49.608659+0000","last_active":"2023-11-08T19:24:50.612010+0000","last_peered":"2023-11-08T19:24:50.612010+0000","last_clean":"2023-11-08T19:24:50.612010+0000","last_became_active":"2023-11-08T19:24:49.608243+0000","last_became_peered":"2023-11-08T19:24:49.608243+0000","last_unstale":"2023-11-08T19:24:50.612010+0000","last_undegraded":"2023-11-08T19:24:50.612010+0000","last_fullsized":"2023-11-08T19:24:50.612010+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:42:41.509009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-08T19:24:50.611395+0000","last_change":"2023-11-08T19:24:49.611793+0000","last_active":"2023-11-08T19:24:50.611395+0000","last_peered":"2023-11-08T19:24:50.611395+0000","last_clean":"2023-11-08T19:24:50.611395+0000","last_became_active":"2023-11-08T19:24:49.611489+0000","last_became_peered":"2023-11-08T19:24:49.611489+0000","last_unstale":"2023-11-08T19:24:50.611395+0000","last_undegraded":"2023-11-08T19:24:50.611395+0000","last_fullsized":"2023-11-08T19:24:50.611395+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:15:32.281672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.805231+0000","last_change":"2023-11-08T19:24:50.624279+0000","last_active":"2023-11-08T19:24:52.805231+0000","last_peered":"2023-11-08T19:24:52.805231+0000","last_clean":"2023-11-08T19:24:52.805231+0000","last_became_active":"2023-11-08T19:24:50.623920+0000","last_became_peered":"2023-11-08T19:24:50.623920+0000","last_unstale":"2023-11-08T19:24:52.805231+0000","last_undegraded":"2023-11-08T19:24:52.805231+0000","last_fullsized":"2023-11-08T19:24:52.805231+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:42:42.512786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.805402+0000","last_change":"2023-11-08T19:24:49.606910+0000","last_active":"2023-11-08T19:24:52.805402+0000","last_peered":"2023-11-08T19:24:52.805402+0000","last_clean":"2023-11-08T19:24:52.805402+0000","last_became_active":"2023-11-08T19:24:49.606627+0000","last_became_peered":"2023-11-08T19:24:49.606627+0000","last_unstale":"2023-11-08T19:24:52.805402+0000","last_undegraded":"2023-11-08T19:24:52.805402+0000","last_fullsized":"2023-11-08T19:24:52.805402+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:56:00.148860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.627087+0000","last_change":"2023-11-08T19:24:50.622398+0000","last_active":"2023-11-08T19:24:51.627087+0000","last_peered":"2023-11-08T19:24:51.627087+0000","last_clean":"2023-11-08T19:24:51.627087+0000","last_became_active":"2023-11-08T19:24:50.622092+0000","last_became_peered":"2023-11-08T19:24:50.622092+0000","last_unstale":"2023-11-08T19:24:51.627087+0000","last_undegraded":"2023-11-08T19:24:51.627087+0000","last_fullsized":"2023-11-08T19:24:51.627087+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:42:27.150066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-08T19:24:50.638180+0000","last_change":"2023-11-08T19:24:49.604285+0000","last_active":"2023-11-08T19:24:50.638180+0000","last_peered":"2023-11-08T19:24:50.638180+0000","last_clean":"2023-11-08T19:24:50.638180+0000","last_became_active":"2023-11-08T19:24:49.603965+0000","last_became_peered":"2023-11-08T19:24:49.603965+0000","last_unstale":"2023-11-08T19:24:50.638180+0000","last_undegraded":"2023-11-08T19:24:50.638180+0000","last_fullsized":"2023-11-08T19:24:50.638180+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:01:09.884262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.309770+0000","last_change":"2023-11-08T19:24:50.624848+0000","last_active":"2023-11-08T19:24:52.309770+0000","last_peered":"2023-11-08T19:24:52.309770+0000","last_clean":"2023-11-08T19:24:52.309770+0000","last_became_active":"2023-11-08T19:24:50.624518+0000","last_became_peered":"2023-11-08T19:24:50.624518+0000","last_unstale":"2023-11-08T19:24:52.309770+0000","last_undegraded":"2023-11-08T19:24:52.309770+0000","last_fullsized":"2023-11-08T19:24:52.309770+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:15:33.285449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-08T19:24:50.611239+0000","last_change":"2023-11-08T19:24:49.611779+0000","last_active":"2023-11-08T19:24:50.611239+0000","last_peered":"2023-11-08T19:24:50.611239+0000","last_clean":"2023-11-08T19:24:50.611239+0000","last_became_active":"2023-11-08T19:24:49.611422+0000","last_became_peered":"2023-11-08T19:24:49.611422+0000","last_unstale":"2023-11-08T19:24:50.611239+0000","last_undegraded":"2023-11-08T19:24:50.611239+0000","last_fullsized":"2023-11-08T19:24:50.611239+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:30:31.599200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.805558+0000","last_change":"2023-11-08T19:24:50.627488+0000","last_active":"2023-11-08T19:24:52.805558+0000","last_peered":"2023-11-08T19:24:52.805558+0000","last_clean":"2023-11-08T19:24:52.805558+0000","last_became_active":"2023-11-08T19:24:50.626571+0000","last_became_peered":"2023-11-08T19:24:50.626571+0000","last_unstale":"2023-11-08T19:24:52.805558+0000","last_undegraded":"2023-11-08T19:24:52.805558+0000","last_fullsized":"2023-11-08T19:24:52.805558+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:43:00.849357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.804911+0000","last_change":"2023-11-08T19:24:49.606579+0000","last_active":"2023-11-08T19:24:52.804911+0000","last_peered":"2023-11-08T19:24:52.804911+0000","last_clean":"2023-11-08T19:24:52.804911+0000","last_became_active":"2023-11-08T19:24:49.606283+0000","last_became_peered":"2023-11-08T19:24:49.606283+0000","last_unstale":"2023-11-08T19:24:52.804911+0000","last_undegraded":"2023-11-08T19:24:52.804911+0000","last_fullsized":"2023-11-08T19:24:52.804911+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:14:00.455497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.626421+0000","last_change":"2023-11-08T19:24:50.625692+0000","last_active":"2023-11-08T19:24:51.626421+0000","last_peered":"2023-11-08T19:24:51.626421+0000","last_clean":"2023-11-08T19:24:51.626421+0000","last_became_active":"2023-11-08T19:24:50.625014+0000","last_became_peered":"2023-11-08T19:24:50.625014+0000","last_unstale":"2023-11-08T19:24:51.626421+0000","last_undegraded":"2023-11-08T19:24:51.626421+0000","last_fullsized":"2023-11-08T19:24:51.626421+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:15:33.285449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.805081+0000","last_change":"2023-11-08T19:24:49.611023+0000","last_active":"2023-11-08T19:24:52.805081+0000","last_peered":"2023-11-08T19:24:52.805081+0000","last_clean":"2023-11-08T19:24:52.805081+0000","last_became_active":"2023-11-08T19:24:49.610735+0000","last_became_peered":"2023-11-08T19:24:49.610735+0000","last_unstale":"2023-11-08T19:24:52.805081+0000","last_undegraded":"2023-11-08T19:24:52.805081+0000","last_fullsized":"2023-11-08T19:24:52.805081+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:30:31.599200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.626730+0000","last_change":"2023-11-08T19:24:50.631491+0000","last_active":"2023-11-08T19:24:51.626730+0000","last_peered":"2023-11-08T19:24:51.626730+0000","last_clean":"2023-11-08T19:24:51.626730+0000","last_became_active":"2023-11-08T19:24:50.630872+0000","last_became_peered":"2023-11-08T19:24:50.630872+0000","last_unstale":"2023-11-08T19:24:51.626730+0000","last_undegraded":"2023-11-08T19:24:51.626730+0000","last_fullsized":"2023-11-08T19:24:51.626730+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:36:53.967796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-08T19:24:50.640533+0000","last_change":"2023-11-08T19:24:49.613965+0000","last_active":"2023-11-08T19:24:50.640533+0000","last_peered":"2023-11-08T19:24:50.640533+0000","last_clean":"2023-11-08T19:24:50.640533+0000","last_became_active":"2023-11-08T19:24:49.613629+0000","last_became_peered":"2023-11-08T19:24:49.613629+0000","last_unstale":"2023-11-08T19:24:50.640533+0000","last_undegraded":"2023-11-08T19:24:50.640533+0000","last_fullsized":"2023-11-08T19:24:50.640533+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:56:00.148860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.629891+0000","last_change":"2023-11-08T19:24:50.627696+0000","last_active":"2023-11-08T19:24:51.629891+0000","last_peered":"2023-11-08T19:24:51.629891+0000","last_clean":"2023-11-08T19:24:51.629891+0000","last_became_active":"2023-11-08T19:24:50.627346+0000","last_became_peered":"2023-11-08T19:24:50.627346+0000","last_unstale":"2023-11-08T19:24:51.629891+0000","last_undegraded":"2023-11-08T19:24:51.629891+0000","last_fullsized":"2023-11-08T19:24:51.629891+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:30:36.041820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.805091+0000","last_change":"2023-11-08T19:24:49.613365+0000","last_active":"2023-11-08T19:24:52.805091+0000","last_peered":"2023-11-08T19:24:52.805091+0000","last_clean":"2023-11-08T19:24:52.805091+0000","last_became_active":"2023-11-08T19:24:49.613052+0000","last_became_peered":"2023-11-08T19:24:49.613052+0000","last_unstale":"2023-11-08T19:24:52.805091+0000","last_undegraded":"2023-11-08T19:24:52.805091+0000","last_fullsized":"2023-11-08T19:24:52.805091+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:41:24.551269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.631232+0000","last_change":"2023-11-08T19:24:50.642477+0000","last_active":"2023-11-08T19:24:51.631232+0000","last_peered":"2023-11-08T19:24:51.631232+0000","last_clean":"2023-11-08T19:24:51.631232+0000","last_became_active":"2023-11-08T19:24:50.642186+0000","last_became_peered":"2023-11-08T19:24:50.642186+0000","last_unstale":"2023-11-08T19:24:51.631232+0000","last_undegraded":"2023-11-08T19:24:51.631232+0000","last_fullsized":"2023-11-08T19:24:51.631232+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:08:17.096077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.805219+0000","last_change":"2023-11-08T19:24:49.606271+0000","last_active":"2023-11-08T19:24:52.805219+0000","last_peered":"2023-11-08T19:24:52.805219+0000","last_clean":"2023-11-08T19:24:52.805219+0000","last_became_active":"2023-11-08T19:24:49.605802+0000","last_became_peered":"2023-11-08T19:24:49.605802+0000","last_unstale":"2023-11-08T19:24:52.805219+0000","last_undegraded":"2023-11-08T19:24:52.805219+0000","last_fullsized":"2023-11-08T19:24:52.805219+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:07:00.575561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.352338+0000","last_change":"2023-11-08T19:24:50.640594+0000","last_active":"2023-11-08T19:24:52.352338+0000","last_peered":"2023-11-08T19:24:52.352338+0000","last_clean":"2023-11-08T19:24:52.352338+0000","last_became_active":"2023-11-08T19:24:50.640295+0000","last_became_peered":"2023-11-08T19:24:50.640295+0000","last_unstale":"2023-11-08T19:24:52.352338+0000","last_undegraded":"2023-11-08T19:24:52.352338+0000","last_fullsized":"2023-11-08T19:24:52.352338+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:34:42.075394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.805026+0000","last_change":"2023-11-08T19:24:49.610782+0000","last_active":"2023-11-08T19:24:52.805026+0000","last_peered":"2023-11-08T19:24:52.805026+0000","last_clean":"2023-11-08T19:24:52.805026+0000","last_became_active":"2023-11-08T19:24:49.610493+0000","last_became_peered":"2023-11-08T19:24:49.610493+0000","last_unstale":"2023-11-08T19:24:52.805026+0000","last_undegraded":"2023-11-08T19:24:52.805026+0000","last_fullsized":"2023-11-08T19:24:52.805026+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:42:59.845581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.631518+0000","last_change":"2023-11-08T19:24:50.630352+0000","last_active":"2023-11-08T19:24:51.631518+0000","last_peered":"2023-11-08T19:24:51.631518+0000","last_clean":"2023-11-08T19:24:51.631518+0000","last_became_active":"2023-11-08T19:24:50.628170+0000","last_became_peered":"2023-11-08T19:24:50.628170+0000","last_unstale":"2023-11-08T19:24:51.631518+0000","last_undegraded":"2023-11-08T19:24:51.631518+0000","last_fullsized":"2023-11-08T19:24:51.631518+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:18:29.376849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.629782+0000","last_change":"2023-11-08T19:24:50.616189+0000","last_active":"2023-11-08T19:24:51.629782+0000","last_peered":"2023-11-08T19:24:51.629782+0000","last_clean":"2023-11-08T19:24:51.629782+0000","last_became_active":"2023-11-08T19:24:50.615704+0000","last_became_peered":"2023-11-08T19:24:50.615704+0000","last_unstale":"2023-11-08T19:24:51.629782+0000","last_undegraded":"2023-11-08T19:24:51.629782+0000","last_fullsized":"2023-11-08T19:24:51.629782+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:30:32.602976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.804775+0000","last_change":"2023-11-08T19:24:50.625948+0000","last_active":"2023-11-08T19:24:52.804775+0000","last_peered":"2023-11-08T19:24:52.804775+0000","last_clean":"2023-11-08T19:24:52.804775+0000","last_became_active":"2023-11-08T19:24:50.625633+0000","last_became_peered":"2023-11-08T19:24:50.625633+0000","last_unstale":"2023-11-08T19:24:52.804775+0000","last_undegraded":"2023-11-08T19:24:52.804775+0000","last_fullsized":"2023-11-08T19:24:52.804775+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:00:13.071654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.352423+0000","last_change":"2023-11-08T19:24:50.640919+0000","last_active":"2023-11-08T19:24:52.352423+0000","last_peered":"2023-11-08T19:24:52.352423+0000","last_clean":"2023-11-08T19:24:52.352423+0000","last_became_active":"2023-11-08T19:24:50.640496+0000","last_became_peered":"2023-11-08T19:24:50.640496+0000","last_unstale":"2023-11-08T19:24:52.352423+0000","last_undegraded":"2023-11-08T19:24:52.352423+0000","last_fullsized":"2023-11-08T19:24:52.352423+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:59:37.592538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.309782+0000","last_change":"2023-11-08T19:24:50.625916+0000","last_active":"2023-11-08T19:24:52.309782+0000","last_peered":"2023-11-08T19:24:52.309782+0000","last_clean":"2023-11-08T19:24:52.309782+0000","last_became_active":"2023-11-08T19:24:50.625625+0000","last_became_peered":"2023-11-08T19:24:50.625625+0000","last_unstale":"2023-11-08T19:24:52.309782+0000","last_undegraded":"2023-11-08T19:24:52.309782+0000","last_fullsized":"2023-11-08T19:24:52.309782+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:40:11.978983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.626555+0000","last_change":"2023-11-08T19:24:50.628686+0000","last_active":"2023-11-08T19:24:51.626555+0000","last_peered":"2023-11-08T19:24:51.626555+0000","last_clean":"2023-11-08T19:24:51.626555+0000","last_became_active":"2023-11-08T19:24:50.628289+0000","last_became_peered":"2023-11-08T19:24:50.628289+0000","last_unstale":"2023-11-08T19:24:51.626555+0000","last_undegraded":"2023-11-08T19:24:51.626555+0000","last_fullsized":"2023-11-08T19:24:51.626555+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:46:19.375897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.805231+0000","last_change":"2023-11-08T19:24:50.624352+0000","last_active":"2023-11-08T19:24:52.805231+0000","last_peered":"2023-11-08T19:24:52.805231+0000","last_clean":"2023-11-08T19:24:52.805231+0000","last_became_active":"2023-11-08T19:24:50.624035+0000","last_became_peered":"2023-11-08T19:24:50.624035+0000","last_unstale":"2023-11-08T19:24:52.805231+0000","last_undegraded":"2023-11-08T19:24:52.805231+0000","last_fullsized":"2023-11-08T19:24:52.805231+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:46:19.375897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.630792+0000","last_change":"2023-11-08T19:24:50.616410+0000","last_active":"2023-11-08T19:24:51.630792+0000","last_peered":"2023-11-08T19:24:51.630792+0000","last_clean":"2023-11-08T19:24:51.630792+0000","last_became_active":"2023-11-08T19:24:50.616118+0000","last_became_peered":"2023-11-08T19:24:50.616118+0000","last_unstale":"2023-11-08T19:24:51.630792+0000","last_undegraded":"2023-11-08T19:24:51.630792+0000","last_fullsized":"2023-11-08T19:24:51.630792+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:42:42.512786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.626991+0000","last_change":"2023-11-08T19:24:50.628009+0000","last_active":"2023-11-08T19:24:51.626991+0000","last_peered":"2023-11-08T19:24:51.626991+0000","last_clean":"2023-11-08T19:24:51.626991+0000","last_became_active":"2023-11-08T19:24:50.627668+0000","last_became_peered":"2023-11-08T19:24:50.627668+0000","last_unstale":"2023-11-08T19:24:51.626991+0000","last_undegraded":"2023-11-08T19:24:51.626991+0000","last_fullsized":"2023-11-08T19:24:51.626991+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:49:19.100370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.352100+0000","last_change":"2023-11-08T19:24:50.641255+0000","last_active":"2023-11-08T19:24:52.352100+0000","last_peered":"2023-11-08T19:24:52.352100+0000","last_clean":"2023-11-08T19:24:52.352100+0000","last_became_active":"2023-11-08T19:24:50.640164+0000","last_became_peered":"2023-11-08T19:24:50.640164+0000","last_unstale":"2023-11-08T19:24:52.352100+0000","last_undegraded":"2023-11-08T19:24:52.352100+0000","last_fullsized":"2023-11-08T19:24:52.352100+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:12:54.641952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.626435+0000","last_change":"2023-11-08T19:24:50.629006+0000","last_active":"2023-11-08T19:24:51.626435+0000","last_peered":"2023-11-08T19:24:51.626435+0000","last_clean":"2023-11-08T19:24:51.626435+0000","last_became_active":"2023-11-08T19:24:50.628685+0000","last_became_peered":"2023-11-08T19:24:50.628685+0000","last_unstale":"2023-11-08T19:24:51.626435+0000","last_undegraded":"2023-11-08T19:24:51.626435+0000","last_fullsized":"2023-11-08T19:24:51.626435+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:30:36.041820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.352270+0000","last_change":"2023-11-08T19:24:50.640737+0000","last_active":"2023-11-08T19:24:52.352270+0000","last_peered":"2023-11-08T19:24:52.352270+0000","last_clean":"2023-11-08T19:24:52.352270+0000","last_became_active":"2023-11-08T19:24:50.640255+0000","last_became_peered":"2023-11-08T19:24:50.640255+0000","last_unstale":"2023-11-08T19:24:52.352270+0000","last_undegraded":"2023-11-08T19:24:52.352270+0000","last_fullsized":"2023-11-08T19:24:52.352270+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:13:41.469149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.309753+0000","last_change":"2023-11-08T19:24:50.620645+0000","last_active":"2023-11-08T19:24:52.309753+0000","last_peered":"2023-11-08T19:24:52.309753+0000","last_clean":"2023-11-08T19:24:52.309753+0000","last_became_active":"2023-11-08T19:24:50.620349+0000","last_became_peered":"2023-11-08T19:24:50.620349+0000","last_unstale":"2023-11-08T19:24:52.309753+0000","last_undegraded":"2023-11-08T19:24:52.309753+0000","last_fullsized":"2023-11-08T19:24:52.309753+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:30:32.602976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.630586+0000","last_change":"2023-11-08T19:24:50.629751+0000","last_active":"2023-11-08T19:24:51.630586+0000","last_peered":"2023-11-08T19:24:51.630586+0000","last_clean":"2023-11-08T19:24:51.630586+0000","last_became_active":"2023-11-08T19:24:50.629453+0000","last_became_peered":"2023-11-08T19:24:50.629453+0000","last_unstale":"2023-11-08T19:24:51.630586+0000","last_undegraded":"2023-11-08T19:24:51.630586+0000","last_fullsized":"2023-11-08T19:24:51.630586+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:46:19.375897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":802816,"data_stored":661860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.433}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45900000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56799999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72599999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67600000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75800000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56599999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54500000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29599999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41599999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47799999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":71070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36299999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23499999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26500000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45200000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47699999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":811008,"data_stored":661916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35099999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27400000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40000000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73199999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71099999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68300000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81000000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":192512,"data_stored":64610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46000000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.443}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82899999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81599999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84799999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.878}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.308}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27000000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.379}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59399999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50900000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54100000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70099999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":71036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46000000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55800000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65400000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75800000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67200000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.746}]}]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":208896,"data_stored":71000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39800000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55300000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61099999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.439}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66700000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69499999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-08T19:24:59.640 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-08T19:24:59.641 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-11-08T19:24:59.641 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-11-08T19:24:59.642 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-11-08T19:24:59.642 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-11-08T19:24:59.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.794+0000 7fe6a806a700 1 Processor -- start 2023-11-08T19:24:59.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.795+0000 7fe6a806a700 1 -- start start 2023-11-08T19:24:59.798 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.795+0000 7fe6a806a700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe6a015dd00 0x7fe6a00cca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:59.799 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.795+0000 7fe6a806a700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe6a01618f0 0x7fe6a006ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:59.799 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.795+0000 7fe6a806a700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6a00ccfb0 con 0x7fe6a006b150 2023-11-08T19:24:59.799 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.795+0000 7fe6a806a700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6a006b600 con 0x7fe6a015dd00 2023-11-08T19:24:59.799 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.795+0000 7fe6a806a700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a00cad50 con 0x7fe6a01618f0 2023-11-08T19:24:59.799 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.796+0000 7fe6a5e06700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe6a015dd00 0x7fe6a00cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:59.800 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.796+0000 7fe6a5e06700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe6a015dd00 0x7fe6a00cca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48744/0 (socket says 172.21.15.143:48744) 2023-11-08T19:24:59.800 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.796+0000 7fe6a5e06700 1 -- 172.21.15.143:0/3373827001 learned_addr learned my addr 172.21.15.143:0/3373827001 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:24:59.800 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.796+0000 7fe6a5e06700 1 -- 172.21.15.143:0/3373827001 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe6a01618f0 msgr2=0x7fe6a006ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:24:59.800 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.796+0000 7fe6a5e06700 1 --2- 172.21.15.143:0/3373827001 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe6a01618f0 0x7fe6a006ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:59.800 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.796+0000 7fe6a5e06700 1 -- 172.21.15.143:0/3373827001 >> v1:172.21.15.174:6789/0 conn(0x7fe6a006b150 legacy=0x7fe6a00b2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:24:59.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.796+0000 7fe6a5e06700 1 -- 172.21.15.143:0/3373827001 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6a0161df0 con 0x7fe6a015dd00 2023-11-08T19:24:59.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.796+0000 7fe6a5e06700 1 --2- 172.21.15.143:0/3373827001 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe6a015dd00 0x7fe6a00cca70 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe690006fd0 tx=0x7fe690009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a05689b921802c22 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:59.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.796+0000 7fe6a4e04700 1 -- 172.21.15.143:0/3373827001 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe690005e30 con 0x7fe6a015dd00 2023-11-08T19:24:59.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.796+0000 7fe6a4e04700 1 -- 172.21.15.143:0/3373827001 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe690006030 con 0x7fe6a015dd00 2023-11-08T19:24:59.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.796+0000 7fe6a4e04700 1 -- 172.21.15.143:0/3373827001 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6900063c0 con 0x7fe6a015dd00 2023-11-08T19:24:59.802 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.797+0000 7fe6a806a700 1 -- 172.21.15.143:0/3373827001 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe6a015dd00 msgr2=0x7fe6a00cca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:59.802 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.797+0000 7fe6a806a700 1 --2- 172.21.15.143:0/3373827001 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe6a015dd00 0x7fe6a00cca70 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe690006fd0 tx=0x7fe690009bd0 comp rx=0 tx=0).stop 2023-11-08T19:24:59.802 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.797+0000 7fe6a806a700 1 -- 172.21.15.143:0/3373827001 shutdown_connections 2023-11-08T19:24:59.802 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.797+0000 7fe6a806a700 1 --2- 172.21.15.143:0/3373827001 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe6a015dd00 0x7fe6a00cca70 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:59.802 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.797+0000 7fe6a806a700 1 --2- 172.21.15.143:0/3373827001 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe6a01618f0 0x7fe6a006ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:59.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.797+0000 7fe6a806a700 1 -- 172.21.15.143:0/3373827001 >> 172.21.15.143:0/3373827001 conn(0x7fe6a0099ab0 msgr2=0x7fe6a00d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:24:59.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.797+0000 7fe6a806a700 1 -- 172.21.15.143:0/3373827001 shutdown_connections 2023-11-08T19:24:59.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.797+0000 7fe6a806a700 1 -- 172.21.15.143:0/3373827001 wait complete. 2023-11-08T19:24:59.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.798+0000 7fe6a806a700 1 Processor -- start 2023-11-08T19:24:59.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.798+0000 7fe6a806a700 1 -- start start 2023-11-08T19:24:59.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.799+0000 7fe6a806a700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe6a006b150 0x7fe6a00c77d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:59.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.799+0000 7fe6a806a700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe6a015dd00 0x7fe6a00c6990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:59.807 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.799+0000 7fe6a806a700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe6a01618f0 0x7fe6a00b1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:59.807 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.799+0000 7fe6a806a700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a00b29e0 con 0x7fe6a015dd00 2023-11-08T19:24:59.807 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.799+0000 7fe6a806a700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a00c44d0 con 0x7fe6a006b150 2023-11-08T19:24:59.807 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.799+0000 7fe6a806a700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a00b2140 con 0x7fe6a01618f0 2023-11-08T19:24:59.807 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.799+0000 7fe6a5605700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe6a015dd00 0x7fe6a00c6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:59.808 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.799+0000 7fe6a5605700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe6a015dd00 0x7fe6a00c6990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48752/0 (socket says 172.21.15.143:48752) 2023-11-08T19:24:59.808 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.799+0000 7fe6a5605700 1 -- 172.21.15.143:0/1689699962 learned_addr learned my addr 172.21.15.143:0/1689699962 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:24:59.808 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.800+0000 7fe6a5e06700 1 --2- 172.21.15.143:0/1689699962 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe6a006b150 0x7fe6a00c77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:59.808 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.800+0000 7fe6a6607700 1 --2- 172.21.15.143:0/1689699962 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe6a01618f0 0x7fe6a00b1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:59.808 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.800+0000 7fe6a5605700 1 -- 172.21.15.143:0/1689699962 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe6a01618f0 msgr2=0x7fe6a00b1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:59.809 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.800+0000 7fe6a5605700 1 --2- 172.21.15.143:0/1689699962 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe6a01618f0 0x7fe6a00b1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:59.809 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.800+0000 7fe6a5605700 1 -- 172.21.15.143:0/1689699962 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe6a006b150 msgr2=0x7fe6a00c77d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:59.809 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.800+0000 7fe6a5605700 1 --2- 172.21.15.143:0/1689699962 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe6a006b150 0x7fe6a00c77d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:59.809 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.800+0000 7fe6a5605700 1 -- 172.21.15.143:0/1689699962 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe690009500 con 0x7fe6a015dd00 2023-11-08T19:24:59.809 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.800+0000 7fe6a5e06700 1 --2- 172.21.15.143:0/1689699962 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe6a006b150 0x7fe6a00c77d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:24:59.810 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.800+0000 7fe6a5605700 1 --2- 172.21.15.143:0/1689699962 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe6a015dd00 0x7fe6a00c6990 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe69c00b4a0 tx=0x7fe69c00b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:59.810 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.800+0000 7fe6967fc700 1 -- 172.21.15.143:0/1689699962 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe69c011830 con 0x7fe6a015dd00 2023-11-08T19:24:59.810 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.800+0000 7fe6967fc700 1 -- 172.21.15.143:0/1689699962 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe69c011990 con 0x7fe6a015dd00 2023-11-08T19:24:59.810 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.800+0000 7fe6967fc700 1 -- 172.21.15.143:0/1689699962 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe69c011d40 con 0x7fe6a015dd00 2023-11-08T19:24:59.810 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.800+0000 7fe6a806a700 1 -- 172.21.15.143:0/1689699962 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6a00b2420 con 0x7fe6a015dd00 2023-11-08T19:24:59.811 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.800+0000 7fe6a806a700 1 -- 172.21.15.143:0/1689699962 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6a00ca530 con 0x7fe6a015dd00 2023-11-08T19:24:59.811 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.801+0000 7fe6a806a700 1 -- 172.21.15.143:0/1689699962 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6a00c4230 con 0x7fe6a015dd00 2023-11-08T19:24:59.811 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.801+0000 7fe6967fc700 1 -- 172.21.15.143:0/1689699962 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe69c01a400 con 0x7fe6a015dd00 2023-11-08T19:24:59.811 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.802+0000 7fe6967fc700 1 --2- 172.21.15.143:0/1689699962 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fe68406af00 0x7fe68406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:59.811 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.802+0000 7fe6967fc700 1 -- 172.21.15.143:0/1689699962 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe69c0895f0 con 0x7fe6a015dd00 2023-11-08T19:24:59.812 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.802+0000 7fe6a5e06700 1 --2- 172.21.15.143:0/1689699962 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fe68406af00 0x7fe68406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:59.812 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.802+0000 7fe6a5e06700 1 --2- 172.21.15.143:0/1689699962 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fe68406af00 0x7fe68406d3c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe690013040 tx=0x7fe690008070 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:59.812 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:24:59.806+0000 7fe6967fc700 1 -- 172.21.15.143:0/1689699962 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe6a00c4230 con 0x7fe6a015dd00 2023-11-08T19:25:00.051 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.049+0000 7fe6a806a700 1 -- 172.21.15.143:0/1689699962 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fe6a00c4610 con 0x7fe6a015dd00 2023-11-08T19:25:00.054 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.052+0000 7fe6967fc700 1 -- 172.21.15.143:0/1689699962 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+171039 (secure 0 0 0) 0x7fe6a00c4610 con 0x7fe6a015dd00 2023-11-08T19:25:00.056 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:25:00.057 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.055+0000 7fe6a806a700 1 -- 172.21.15.143:0/1689699962 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fe68406af00 msgr2=0x7fe68406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:00.057 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.055+0000 7fe6a806a700 1 --2- 172.21.15.143:0/1689699962 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fe68406af00 0x7fe68406d3c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe690013040 tx=0x7fe690008070 comp rx=0 tx=0).stop 2023-11-08T19:25:00.058 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.055+0000 7fe6a806a700 1 -- 172.21.15.143:0/1689699962 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe6a015dd00 msgr2=0x7fe6a00c6990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:00.058 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.055+0000 7fe6a806a700 1 --2- 172.21.15.143:0/1689699962 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe6a015dd00 0x7fe6a00c6990 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe69c00b4a0 tx=0x7fe69c00b860 comp rx=0 tx=0).stop 2023-11-08T19:25:00.058 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.055+0000 7fe6a806a700 1 -- 172.21.15.143:0/1689699962 shutdown_connections 2023-11-08T19:25:00.058 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.055+0000 7fe6a806a700 1 --2- 172.21.15.143:0/1689699962 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fe68406af00 0x7fe68406d3c0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:00.058 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.055+0000 7fe6a806a700 1 --2- 172.21.15.143:0/1689699962 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe6a01618f0 0x7fe6a00b1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:00.059 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.055+0000 7fe6a806a700 1 --2- 172.21.15.143:0/1689699962 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe6a006b150 0x7fe6a00c77d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:00.059 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.055+0000 7fe6a806a700 1 --2- 172.21.15.143:0/1689699962 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe6a015dd00 0x7fe6a00c6990 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:00.059 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.055+0000 7fe6a806a700 1 -- 172.21.15.143:0/1689699962 >> 172.21.15.143:0/1689699962 conn(0x7fe6a0099ab0 msgr2=0x7fe6a00ad380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:00.059 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.055+0000 7fe6a806a700 1 -- 172.21.15.143:0/1689699962 shutdown_connections 2023-11-08T19:25:00.059 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.055+0000 7fe6a806a700 1 -- 172.21.15.143:0/1689699962 wait complete. 2023-11-08T19:25:00.068 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":5,"active_gid":4099,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":4037280178},{"type":"v1","addr":"172.21.15.174:6833","nonce":4037280178}]},"active_addr":"172.21.15.174:6833/4037280178","active_change":"2023-11-08T19:24:38.502017+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4110,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:v0.0.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:v0.0.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.174:0","nonce":1832728653}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.174:0","nonce":2664141320}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.174:0","nonce":1537638593}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.174:0","nonce":1411402224}]}]} 2023-11-08T19:25:00.072 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-11-08T19:25:00.072 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-11-08T19:25:00.073 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:25:00.192 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.190+0000 7fd34c38f700 1 Processor -- start 2023-11-08T19:25:00.192 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.190+0000 7fd34c38f700 1 -- start start 2023-11-08T19:25:00.198 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.191+0000 7fd34c38f700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fd3440cc410 0x7fd3440cc820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:00.198 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.191+0000 7fd34c38f700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3440b2a60 con 0x7fd344159870 2023-11-08T19:25:00.198 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.191+0000 7fd34c38f700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3440b2d40 con 0x7fd34406ed60 2023-11-08T19:25:00.199 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.191+0000 7fd34c38f700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd34406b090 con 0x7fd3440cc410 2023-11-08T19:25:00.199 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.191+0000 7fd34a12b700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fd344159870 0x7fd3440b2210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:44912/0 (socket says 172.21.15.143:44912) 2023-11-08T19:25:00.199 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.191+0000 7fd34a12b700 1 -- 172.21.15.143:0/2368914512 learned_addr learned my addr 172.21.15.143:0/2368914512 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:25:00.199 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.192+0000 7fd34a92c700 1 --2- 172.21.15.143:0/2368914512 >> v2:172.21.15.174:3300/0 conn(0x7fd3440cc410 0x7fd3440cc820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:00.199 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.192+0000 7fd349129700 1 -- 172.21.15.143:0/2368914512 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2792448044 0 0) 0x7fd3440b2a60 con 0x7fd344159870 2023-11-08T19:25:00.199 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.192+0000 7fd349129700 1 -- 172.21.15.143:0/2368914512 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd3300035b0 con 0x7fd344159870 2023-11-08T19:25:00.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.192+0000 7fd349129700 1 -- 172.21.15.143:0/2368914512 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 564525584 0 0) 0x7fd3300035b0 con 0x7fd344159870 2023-11-08T19:25:00.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.192+0000 7fd349129700 1 -- 172.21.15.143:0/2368914512 >> v2:172.21.15.174:3300/0 conn(0x7fd3440cc410 msgr2=0x7fd3440cc820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:00.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.192+0000 7fd349129700 1 --2- 172.21.15.143:0/2368914512 >> v2:172.21.15.174:3300/0 conn(0x7fd3440cc410 0x7fd3440cc820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:00.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.192+0000 7fd349129700 1 -- 172.21.15.143:0/2368914512 >> v1:172.21.15.174:6789/0 conn(0x7fd34406ed60 legacy=0x7fd34406b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:00.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.192+0000 7fd349129700 1 -- 172.21.15.143:0/2368914512 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3440b3020 con 0x7fd344159870 2023-11-08T19:25:00.201 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.192+0000 7fd349129700 1 -- 172.21.15.143:0/2368914512 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4147756303 0 0) 0x7fd3340017f0 con 0x7fd344159870 2023-11-08T19:25:00.201 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.192+0000 7fd349129700 1 -- 172.21.15.143:0/2368914512 >> v1:172.21.15.143:6789/0 conn(0x7fd344159870 legacy=0x7fd3440b2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:00.201 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.192+0000 7fd349129700 1 --2- 172.21.15.143:0/2368914512 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd330003a00 0x7fd330003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:00.201 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.192+0000 7fd349129700 1 --2- 172.21.15.143:0/2368914512 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd330004850 0x7fd330004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:00.201 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.192+0000 7fd349129700 1 --2- 172.21.15.143:0/2368914512 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd330005330 0x7fd33000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:00.202 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.192+0000 7fd349129700 1 -- 172.21.15.143:0/2368914512 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd34406b090 con 0x7fd330005330 2023-11-08T19:25:00.202 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.192+0000 7fd349129700 1 -- 172.21.15.143:0/2368914512 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd33000c200 con 0x7fd330003a00 2023-11-08T19:25:00.202 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.192+0000 7fd349129700 1 -- 172.21.15.143:0/2368914512 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd33000c3c0 con 0x7fd330004850 2023-11-08T19:25:00.202 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.192+0000 7fd34a92c700 1 --2- 172.21.15.143:0/2368914512 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd330005330 0x7fd33000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:00.202 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.192+0000 7fd34a12b700 1 --2- 172.21.15.143:0/2368914512 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd330003a00 0x7fd330003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:00.202 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.192+0000 7fd34992a700 1 --2- 172.21.15.143:0/2368914512 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd330004850 0x7fd330004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:00.203 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.192+0000 7fd34a92c700 1 -- 172.21.15.143:0/2368914512 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd330004850 msgr2=0x7fd330004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:00.203 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.192+0000 7fd34a92c700 1 --2- 172.21.15.143:0/2368914512 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd330004850 0x7fd330004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:00.203 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.192+0000 7fd34a92c700 1 -- 172.21.15.143:0/2368914512 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd330003a00 msgr2=0x7fd330003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:00.203 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.192+0000 7fd34a92c700 1 --2- 172.21.15.143:0/2368914512 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd330003a00 0x7fd330003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:00.203 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.192+0000 7fd34a92c700 1 -- 172.21.15.143:0/2368914512 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3300035b0 con 0x7fd330005330 2023-11-08T19:25:00.204 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.192+0000 7fd34a92c700 1 -- 172.21.15.143:0/2368914512 reap_dead start 2023-11-08T19:25:00.204 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.193+0000 7fd34a92c700 1 --2- 172.21.15.143:0/2368914512 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd330005330 0x7fd33000ba10 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd33c009f90 tx=0x7fd330005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47dd3b16fb3129f6 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:00.204 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.193+0000 7fd349129700 1 -- 172.21.15.143:0/2368914512 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd33c003a20 con 0x7fd330005330 2023-11-08T19:25:00.204 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.193+0000 7fd349129700 1 -- 172.21.15.143:0/2368914512 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd33c00f040 con 0x7fd330005330 2023-11-08T19:25:00.204 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.193+0000 7fd349129700 1 -- 172.21.15.143:0/2368914512 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd33c003dc0 con 0x7fd330005330 2023-11-08T19:25:00.205 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.193+0000 7fd34c38f700 1 -- 172.21.15.143:0/2368914512 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd330005330 msgr2=0x7fd33000ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:00.205 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.193+0000 7fd34c38f700 1 --2- 172.21.15.143:0/2368914512 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd330005330 0x7fd33000ba10 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd33c009f90 tx=0x7fd330005880 comp rx=0 tx=0).stop 2023-11-08T19:25:00.205 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.193+0000 7fd34c38f700 1 -- 172.21.15.143:0/2368914512 shutdown_connections 2023-11-08T19:25:00.205 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.193+0000 7fd34c38f700 1 --2- 172.21.15.143:0/2368914512 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd330005330 0x7fd33000ba10 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:00.205 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.193+0000 7fd34c38f700 1 -- 172.21.15.143:0/2368914512 >> 172.21.15.143:0/2368914512 conn(0x7fd344099ab0 msgr2=0x7fd3440d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:00.205 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.193+0000 7fd34c38f700 1 -- 172.21.15.143:0/2368914512 shutdown_connections 2023-11-08T19:25:00.206 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.193+0000 7fd34c38f700 1 -- 172.21.15.143:0/2368914512 wait complete. 2023-11-08T19:25:00.206 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.194+0000 7fd34c38f700 1 Processor -- start 2023-11-08T19:25:00.206 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.194+0000 7fd34c38f700 1 -- start start 2023-11-08T19:25:00.206 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.195+0000 7fd34c38f700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd330005330 0x7fd34415a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:00.206 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.195+0000 7fd34c38f700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd3440ae640 0x7fd3440c5d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:00.207 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.195+0000 7fd34c38f700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd3440aeac0 0x7fd3440c6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:00.207 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.195+0000 7fd34c38f700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3441f6bd0 con 0x7fd330005330 2023-11-08T19:25:00.207 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.195+0000 7fd34c38f700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd34415aae0 con 0x7fd3440ae640 2023-11-08T19:25:00.207 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.195+0000 7fd34c38f700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd344151730 con 0x7fd3440aeac0 2023-11-08T19:25:00.207 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.195+0000 7fd34a12b700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd330005330 0x7fd34415a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:00.208 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.195+0000 7fd34a12b700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd330005330 0x7fd34415a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48770/0 (socket says 172.21.15.143:48770) 2023-11-08T19:25:00.208 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.195+0000 7fd34a12b700 1 -- 172.21.15.143:0/1247772327 learned_addr learned my addr 172.21.15.143:0/1247772327 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:00.208 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.195+0000 7fd34a12b700 1 -- 172.21.15.143:0/1247772327 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd3440aeac0 msgr2=0x7fd3440c6290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:00.208 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.195+0000 7fd34a12b700 1 --2- 172.21.15.143:0/1247772327 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd3440aeac0 0x7fd3440c6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:00.208 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.195+0000 7fd34a12b700 1 -- 172.21.15.143:0/1247772327 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd3440ae640 msgr2=0x7fd3440c5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:00.208 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.195+0000 7fd34a12b700 1 --2- 172.21.15.143:0/1247772327 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd3440ae640 0x7fd3440c5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:00.209 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.195+0000 7fd34a12b700 1 -- 172.21.15.143:0/1247772327 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3441f6e20 con 0x7fd330005330 2023-11-08T19:25:00.209 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.196+0000 7fd34a12b700 1 --2- 172.21.15.143:0/1247772327 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd330005330 0x7fd34415a5a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fd33400f800 tx=0x7fd33400fbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:00.209 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.196+0000 7fd33affd700 1 -- 172.21.15.143:0/1247772327 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd334001780 con 0x7fd330005330 2023-11-08T19:25:00.209 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.196+0000 7fd33affd700 1 -- 172.21.15.143:0/1247772327 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3340072e0 con 0x7fd330005330 2023-11-08T19:25:00.209 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.196+0000 7fd33affd700 1 -- 172.21.15.143:0/1247772327 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd334007690 con 0x7fd330005330 2023-11-08T19:25:00.210 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.196+0000 7fd34c38f700 1 -- 172.21.15.143:0/1247772327 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3441f7100 con 0x7fd330005330 2023-11-08T19:25:00.210 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.196+0000 7fd34c38f700 1 -- 172.21.15.143:0/1247772327 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3441f76c0 con 0x7fd330005330 2023-11-08T19:25:00.210 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.197+0000 7fd34c38f700 1 -- 172.21.15.143:0/1247772327 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3440b0c10 con 0x7fd330005330 2023-11-08T19:25:00.211 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.197+0000 7fd33affd700 1 -- 172.21.15.143:0/1247772327 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd334005830 con 0x7fd330005330 2023-11-08T19:25:00.211 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.197+0000 7fd33affd700 1 --2- 172.21.15.143:0/1247772327 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd32806af00 0x7fd32806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:00.211 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.198+0000 7fd33affd700 1 -- 172.21.15.143:0/1247772327 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd3340895f0 con 0x7fd330005330 2023-11-08T19:25:00.212 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.198+0000 7fd34992a700 1 --2- 172.21.15.143:0/1247772327 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd32806af00 0x7fd32806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:00.212 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.198+0000 7fd34992a700 1 --2- 172.21.15.143:0/1247772327 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd32806af00 0x7fd32806d3c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd340000c00 tx=0x7fd340009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:00.212 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.202+0000 7fd33affd700 1 -- 172.21.15.143:0/1247772327 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd3340546a0 con 0x7fd330005330 2023-11-08T19:25:00.394 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.393+0000 7fd34c38f700 1 -- 172.21.15.143:0/1247772327 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd34404eee0 con 0x7fd330005330 2023-11-08T19:25:00.395 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.394+0000 7fd33affd700 1 -- 172.21.15.143:0/1247772327 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16553 (secure 0 0 0) 0x7fd334057cc0 con 0x7fd330005330 2023-11-08T19:25:00.396 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:25:00.396 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":20,"fsid":"968483af-77fc-4068-ad7c-5c2a1a4613d7","created":"2023-11-08T19:24:32.826484+0000","modified":"2023-11-08T19:24:51.618072+0000","last_up_change":"2023-11-08T19:24:43.573880+0000","last_in_change":"2023-11-08T19:24:35.314525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:44.551530+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-08T19:24:48.256781+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-08T19:24:48.954990+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bae98ec1-d94e-403d-a220-a522a53ea6f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6809","nonce":1487830566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6811","nonce":1487830566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6815","nonce":1487830566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6813","nonce":1487830566}]},"public_addr":"172.21.15.143:6809/1487830566","cluster_addr":"172.21.15.143:6811/1487830566","heartbeat_back_addr":"172.21.15.143:6815/1487830566","heartbeat_front_addr":"172.21.15.143:6813/1487830566","state":["exists","up"]},{"osd":1,"uuid":"5cb6538a-1ccb-4322-8633-09f66af0b7e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6801","nonce":787341397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6803","nonce":787341397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6807","nonce":787341397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6805","nonce":787341397}]},"public_addr":"172.21.15.143:6801/787341397","cluster_addr":"172.21.15.143:6803/787341397","heartbeat_back_addr":"172.21.15.143:6807/787341397","heartbeat_front_addr":"172.21.15.143:6805/787341397","state":["exists","up"]},{"osd":2,"uuid":"cdbf522c-60ef-4d60-9653-8dcb529c101a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6822","nonce":2301325597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6826","nonce":2301325597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6831","nonce":2301325597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6829","nonce":2301325597}]},"public_addr":"172.21.15.143:6822/2301325597","cluster_addr":"172.21.15.143:6826/2301325597","heartbeat_back_addr":"172.21.15.143:6831/2301325597","heartbeat_front_addr":"172.21.15.143:6829/2301325597","state":["exists","up"]},{"osd":3,"uuid":"4e2783cf-ebb9-4dd8-8245-bc1d17ce4325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6817","nonce":2021504428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6819","nonce":2021504428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6827","nonce":2021504428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6823","nonce":2021504428}]},"public_addr":"172.21.15.143:6817/2021504428","cluster_addr":"172.21.15.143:6819/2021504428","heartbeat_back_addr":"172.21.15.143:6827/2021504428","heartbeat_front_addr":"172.21.15.143:6823/2021504428","state":["exists","up"]},{"osd":4,"uuid":"691256fa-9b77-49b9-a346-7ae1adf0ca80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6801","nonce":1993380595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6803","nonce":1993380595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6807","nonce":1993380595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6805","nonce":1993380595}]},"public_addr":"172.21.15.174:6801/1993380595","cluster_addr":"172.21.15.174:6803/1993380595","heartbeat_back_addr":"172.21.15.174:6807/1993380595","heartbeat_front_addr":"172.21.15.174:6805/1993380595","state":["exists","up"]},{"osd":5,"uuid":"756fc352-48c9-46ce-b52e-0541fe6f8525","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6809","nonce":1777923303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6811","nonce":1777923303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6815","nonce":1777923303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6813","nonce":1777923303}]},"public_addr":"172.21.15.174:6809/1777923303","cluster_addr":"172.21.15.174:6811/1777923303","heartbeat_back_addr":"172.21.15.174:6815/1777923303","heartbeat_front_addr":"172.21.15.174:6813/1777923303","state":["exists","up"]},{"osd":6,"uuid":"fab2ac2b-ed97-4cde-9825-b480eabf8b8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6825","nonce":2755130316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6827","nonce":2755130316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6831","nonce":2755130316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6829","nonce":2755130316}]},"public_addr":"172.21.15.174:6825/2755130316","cluster_addr":"172.21.15.174:6827/2755130316","heartbeat_back_addr":"172.21.15.174:6831/2755130316","heartbeat_front_addr":"172.21.15.174:6829/2755130316","state":["exists","up"]},{"osd":7,"uuid":"3de9b797-2ab6-4d41-a863-7e7d638fd703","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6817","nonce":2847141411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6819","nonce":2847141411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6823","nonce":2847141411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6821","nonce":2847141411}]},"public_addr":"172.21.15.174:6817/2847141411","cluster_addr":"172.21.15.174:6819/2847141411","heartbeat_back_addr":"172.21.15.174:6823/2847141411","heartbeat_front_addr":"172.21.15.174:6821/2847141411","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.477076+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.496890+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.539305+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.454673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.538881+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.521294+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.520712+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.540404+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:25:00.397 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.396+0000 7fd34c38f700 1 -- 172.21.15.143:0/1247772327 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd32806af00 msgr2=0x7fd32806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:00.398 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.396+0000 7fd34c38f700 1 --2- 172.21.15.143:0/1247772327 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd32806af00 0x7fd32806d3c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd340000c00 tx=0x7fd340009040 comp rx=0 tx=0).stop 2023-11-08T19:25:00.398 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.396+0000 7fd34c38f700 1 -- 172.21.15.143:0/1247772327 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd330005330 msgr2=0x7fd34415a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:00.398 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.396+0000 7fd34c38f700 1 --2- 172.21.15.143:0/1247772327 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd330005330 0x7fd34415a5a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fd33400f800 tx=0x7fd33400fbc0 comp rx=0 tx=0).stop 2023-11-08T19:25:00.398 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.396+0000 7fd34c38f700 1 -- 172.21.15.143:0/1247772327 shutdown_connections 2023-11-08T19:25:00.399 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.396+0000 7fd34c38f700 1 --2- 172.21.15.143:0/1247772327 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd32806af00 0x7fd32806d3c0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:00.399 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.396+0000 7fd34c38f700 1 --2- 172.21.15.143:0/1247772327 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd3440aeac0 0x7fd3440c6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:00.399 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.396+0000 7fd34c38f700 1 --2- 172.21.15.143:0/1247772327 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd330005330 0x7fd34415a5a0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:00.399 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.396+0000 7fd34c38f700 1 --2- 172.21.15.143:0/1247772327 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd3440ae640 0x7fd3440c5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:00.399 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.396+0000 7fd34c38f700 1 -- 172.21.15.143:0/1247772327 >> 172.21.15.143:0/1247772327 conn(0x7fd344099ab0 msgr2=0x7fd344158e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:00.399 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.396+0000 7fd34c38f700 1 -- 172.21.15.143:0/1247772327 shutdown_connections 2023-11-08T19:25:00.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.396+0000 7fd34c38f700 1 -- 172.21.15.143:0/1247772327 wait complete. 2023-11-08T19:25:00.410 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-11-08T19:25:00.410 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:25:00.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.529+0000 7fe2dc9ea700 1 Processor -- start 2023-11-08T19:25:00.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.529+0000 7fe2dc9ea700 1 -- start start 2023-11-08T19:25:00.535 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.530+0000 7fe2dc9ea700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe2d80d1c70 0x7fe2d80b14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:00.535 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.530+0000 7fe2dc9ea700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2d80b1bc0 con 0x7fe2d815ba30 2023-11-08T19:25:00.535 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.530+0000 7fe2dc9ea700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2d80b1ea0 con 0x7fe2d80d03a0 2023-11-08T19:25:00.535 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.530+0000 7fe2dc9ea700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2d80b2180 con 0x7fe2d80d1c70 2023-11-08T19:25:00.536 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.530+0000 7fe2d659c700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fe2d815ba30 0x7fe2d80d1560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:44932/0 (socket says 172.21.15.143:44932) 2023-11-08T19:25:00.536 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.530+0000 7fe2d659c700 1 -- 172.21.15.143:0/1265258087 learned_addr learned my addr 172.21.15.143:0/1265258087 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:25:00.536 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.530+0000 7fe2d5d9b700 1 --2- 172.21.15.143:0/1265258087 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe2d80d1c70 0x7fe2d80b14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:00.536 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.530+0000 7fe2d559a700 1 -- 172.21.15.143:0/1265258087 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2450891215 0 0) 0x7fe2d80b1bc0 con 0x7fe2d815ba30 2023-11-08T19:25:00.536 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.530+0000 7fe2d559a700 1 -- 172.21.15.143:0/1265258087 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2c00035b0 con 0x7fe2d815ba30 2023-11-08T19:25:00.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.531+0000 7fe2d559a700 1 -- 172.21.15.143:0/1265258087 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3988322022 0 0) 0x7fe2c00035b0 con 0x7fe2d815ba30 2023-11-08T19:25:00.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.531+0000 7fe2d559a700 1 -- 172.21.15.143:0/1265258087 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe2d80d1c70 msgr2=0x7fe2d80b14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:00.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.531+0000 7fe2d559a700 1 --2- 172.21.15.143:0/1265258087 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe2d80d1c70 0x7fe2d80b14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:00.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.531+0000 7fe2d559a700 1 -- 172.21.15.143:0/1265258087 >> v1:172.21.15.174:6789/0 conn(0x7fe2d80d03a0 legacy=0x7fe2d80cc5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:00.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.531+0000 7fe2d559a700 1 -- 172.21.15.143:0/1265258087 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2d80b2460 con 0x7fe2d815ba30 2023-11-08T19:25:00.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.531+0000 7fe2d559a700 1 -- 172.21.15.143:0/1265258087 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4147756303 0 0) 0x7fe2cc001670 con 0x7fe2d815ba30 2023-11-08T19:25:00.538 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.531+0000 7fe2d559a700 1 -- 172.21.15.143:0/1265258087 >> v1:172.21.15.143:6789/0 conn(0x7fe2d815ba30 legacy=0x7fe2d80d1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:00.538 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.531+0000 7fe2d559a700 1 --2- 172.21.15.143:0/1265258087 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe2c0003a30 0x7fe2c0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:00.538 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.531+0000 7fe2d559a700 1 --2- 172.21.15.143:0/1265258087 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe2d80d1c70 0x7fe2c0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:00.538 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.531+0000 7fe2d559a700 1 --2- 172.21.15.143:0/1265258087 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe2c0004d90 0x7fe2c0005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:00.538 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.531+0000 7fe2d559a700 1 -- 172.21.15.143:0/1265258087 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2d80b2180 con 0x7fe2c0003a30 2023-11-08T19:25:00.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.531+0000 7fe2d559a700 1 -- 172.21.15.143:0/1265258087 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2c000ba10 con 0x7fe2c0004d90 2023-11-08T19:25:00.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.531+0000 7fe2d559a700 1 -- 172.21.15.143:0/1265258087 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2c000bbd0 con 0x7fe2d80d1c70 2023-11-08T19:25:00.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.531+0000 7fe2d659c700 1 --2- 172.21.15.143:0/1265258087 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe2c0003a30 0x7fe2c0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:00.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.531+0000 7fe2d5d9b700 1 --2- 172.21.15.143:0/1265258087 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe2d80d1c70 0x7fe2c0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:00.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.531+0000 7fe2d6d9d700 1 --2- 172.21.15.143:0/1265258087 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe2c0004d90 0x7fe2c0005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:00.540 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.531+0000 7fe2d659c700 1 -- 172.21.15.143:0/1265258087 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe2d80d1c70 msgr2=0x7fe2c0004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:00.540 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.531+0000 7fe2d659c700 1 --2- 172.21.15.143:0/1265258087 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe2d80d1c70 0x7fe2c0004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:00.540 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.531+0000 7fe2d659c700 1 -- 172.21.15.143:0/1265258087 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe2c0004d90 msgr2=0x7fe2c0005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:00.540 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.531+0000 7fe2d659c700 1 --2- 172.21.15.143:0/1265258087 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe2c0004d90 0x7fe2c0005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:00.540 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.531+0000 7fe2d659c700 1 -- 172.21.15.143:0/1265258087 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2c00035b0 con 0x7fe2c0003a30 2023-11-08T19:25:00.541 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.531+0000 7fe2d6d9d700 1 --2- 172.21.15.143:0/1265258087 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe2c0004d90 0x7fe2c0005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:25:00.541 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.531+0000 7fe2d659c700 1 --2- 172.21.15.143:0/1265258087 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe2c0003a30 0x7fe2c0003e10 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe2cc00c2f0 tx=0x7fe2c0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27af2f729debfa9a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:00.541 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.531+0000 7fe2d559a700 1 -- 172.21.15.143:0/1265258087 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2cc00cd70 con 0x7fe2c0003a30 2023-11-08T19:25:00.541 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.531+0000 7fe2d559a700 1 -- 172.21.15.143:0/1265258087 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2cc005960 con 0x7fe2c0003a30 2023-11-08T19:25:00.541 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.531+0000 7fe2d559a700 1 -- 172.21.15.143:0/1265258087 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2cc005cf0 con 0x7fe2c0003a30 2023-11-08T19:25:00.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.532+0000 7fe2dc9ea700 1 -- 172.21.15.143:0/1265258087 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe2c0003a30 msgr2=0x7fe2c0003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:00.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.532+0000 7fe2dc9ea700 1 --2- 172.21.15.143:0/1265258087 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe2c0003a30 0x7fe2c0003e10 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe2cc00c2f0 tx=0x7fe2c0005880 comp rx=0 tx=0).stop 2023-11-08T19:25:00.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.532+0000 7fe2d6d9d700 1 -- 172.21.15.143:0/1265258087 reap_dead start 2023-11-08T19:25:00.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.532+0000 7fe2dc9ea700 1 -- 172.21.15.143:0/1265258087 shutdown_connections 2023-11-08T19:25:00.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.532+0000 7fe2dc9ea700 1 -- 172.21.15.143:0/1265258087 >> 172.21.15.143:0/1265258087 conn(0x7fe2d808b000 msgr2=0x7fe2d809ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:00.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.532+0000 7fe2dc9ea700 1 -- 172.21.15.143:0/1265258087 shutdown_connections 2023-11-08T19:25:00.543 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.532+0000 7fe2dc9ea700 1 -- 172.21.15.143:0/1265258087 wait complete. 2023-11-08T19:25:00.543 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.533+0000 7fe2dc9ea700 1 Processor -- start 2023-11-08T19:25:00.543 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.533+0000 7fe2dc9ea700 1 -- start start 2023-11-08T19:25:00.545 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.534+0000 7fe2dc9ea700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe2c0003a30 0x7fe2d806d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:00.546 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.534+0000 7fe2dc9ea700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe2d806c5b0 0x7fe2d815c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:00.546 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.534+0000 7fe2dc9ea700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe2d815c620 0x7fe2d80b63d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:00.546 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.534+0000 7fe2dc9ea700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2d81555f0 con 0x7fe2d806c5b0 2023-11-08T19:25:00.546 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.534+0000 7fe2dc9ea700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2d806dd40 con 0x7fe2c0003a30 2023-11-08T19:25:00.546 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.534+0000 7fe2dc9ea700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2d80b5650 con 0x7fe2d815c620 2023-11-08T19:25:00.547 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.534+0000 7fe2d5d9b700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe2d806c5b0 0x7fe2d815c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:00.547 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.534+0000 7fe2d5d9b700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe2d806c5b0 0x7fe2d815c0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48792/0 (socket says 172.21.15.143:48792) 2023-11-08T19:25:00.547 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.534+0000 7fe2d5d9b700 1 -- 172.21.15.143:0/1523596212 learned_addr learned my addr 172.21.15.143:0/1523596212 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:00.547 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.534+0000 7fe2d659c700 1 --2- 172.21.15.143:0/1523596212 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe2c0003a30 0x7fe2d806d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:00.547 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.534+0000 7fe2d6d9d700 1 --2- 172.21.15.143:0/1523596212 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe2d815c620 0x7fe2d80b63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:00.548 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.535+0000 7fe2d5d9b700 1 -- 172.21.15.143:0/1523596212 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe2d815c620 msgr2=0x7fe2d80b63d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:00.548 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.535+0000 7fe2d5d9b700 1 --2- 172.21.15.143:0/1523596212 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe2d815c620 0x7fe2d80b63d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:00.548 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.535+0000 7fe2d5d9b700 1 -- 172.21.15.143:0/1523596212 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe2c0003a30 msgr2=0x7fe2d806d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:00.548 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.535+0000 7fe2d5d9b700 1 --2- 172.21.15.143:0/1523596212 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe2c0003a30 0x7fe2d806d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:00.548 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.535+0000 7fe2d5d9b700 1 -- 172.21.15.143:0/1523596212 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2d80b58f0 con 0x7fe2d806c5b0 2023-11-08T19:25:00.549 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.535+0000 7fe2d5d9b700 1 --2- 172.21.15.143:0/1523596212 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe2d806c5b0 0x7fe2d815c0e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe2c8011aa0 tx=0x7fe2c8011db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:00.549 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.535+0000 7fe2c6ffd700 1 -- 172.21.15.143:0/1523596212 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c8011ee0 con 0x7fe2d806c5b0 2023-11-08T19:25:00.549 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.535+0000 7fe2c6ffd700 1 -- 172.21.15.143:0/1523596212 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2c800f940 con 0x7fe2d806c5b0 2023-11-08T19:25:00.549 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.535+0000 7fe2dc9ea700 1 -- 172.21.15.143:0/1523596212 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2d80b5bd0 con 0x7fe2d806c5b0 2023-11-08T19:25:00.549 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.535+0000 7fe2c6ffd700 1 -- 172.21.15.143:0/1523596212 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c800fcf0 con 0x7fe2d806c5b0 2023-11-08T19:25:00.550 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.535+0000 7fe2dc9ea700 1 -- 172.21.15.143:0/1523596212 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2d806d970 con 0x7fe2d806c5b0 2023-11-08T19:25:00.550 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.536+0000 7fe2dc9ea700 1 -- 172.21.15.143:0/1523596212 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2d804eee0 con 0x7fe2d806c5b0 2023-11-08T19:25:00.550 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.536+0000 7fe2c6ffd700 1 -- 172.21.15.143:0/1523596212 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe2c8014440 con 0x7fe2d806c5b0 2023-11-08T19:25:00.550 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.537+0000 7fe2c6ffd700 1 --2- 172.21.15.143:0/1523596212 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fe2b806af00 0x7fe2b806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:00.550 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.537+0000 7fe2c6ffd700 1 -- 172.21.15.143:0/1523596212 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe2c8089b30 con 0x7fe2d806c5b0 2023-11-08T19:25:00.550 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.537+0000 7fe2d659c700 1 --2- 172.21.15.143:0/1523596212 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fe2b806af00 0x7fe2b806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:00.551 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.537+0000 7fe2d659c700 1 --2- 172.21.15.143:0/1523596212 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fe2b806af00 0x7fe2b806d3c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe2cc001930 tx=0x7fe2cc007000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:00.551 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.541+0000 7fe2c6ffd700 1 -- 172.21.15.143:0/1523596212 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe2c8054d30 con 0x7fe2d806c5b0 2023-11-08T19:25:00.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.732+0000 7fe2dc9ea700 1 -- 172.21.15.143:0/1523596212 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe2d80af050 con 0x7fe2d806c5b0 2023-11-08T19:25:00.735 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.734+0000 7fe2c6ffd700 1 -- 172.21.15.143:0/1523596212 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16553 (secure 0 0 0) 0x7fe2c8058350 con 0x7fe2d806c5b0 2023-11-08T19:25:00.735 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:25:00.736 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":20,"fsid":"968483af-77fc-4068-ad7c-5c2a1a4613d7","created":"2023-11-08T19:24:32.826484+0000","modified":"2023-11-08T19:24:51.618072+0000","last_up_change":"2023-11-08T19:24:43.573880+0000","last_in_change":"2023-11-08T19:24:35.314525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:44.551530+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-08T19:24:48.256781+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-08T19:24:48.954990+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bae98ec1-d94e-403d-a220-a522a53ea6f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6809","nonce":1487830566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6811","nonce":1487830566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6815","nonce":1487830566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6813","nonce":1487830566}]},"public_addr":"172.21.15.143:6809/1487830566","cluster_addr":"172.21.15.143:6811/1487830566","heartbeat_back_addr":"172.21.15.143:6815/1487830566","heartbeat_front_addr":"172.21.15.143:6813/1487830566","state":["exists","up"]},{"osd":1,"uuid":"5cb6538a-1ccb-4322-8633-09f66af0b7e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6801","nonce":787341397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6803","nonce":787341397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6807","nonce":787341397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6805","nonce":787341397}]},"public_addr":"172.21.15.143:6801/787341397","cluster_addr":"172.21.15.143:6803/787341397","heartbeat_back_addr":"172.21.15.143:6807/787341397","heartbeat_front_addr":"172.21.15.143:6805/787341397","state":["exists","up"]},{"osd":2,"uuid":"cdbf522c-60ef-4d60-9653-8dcb529c101a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6822","nonce":2301325597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6826","nonce":2301325597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6831","nonce":2301325597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6829","nonce":2301325597}]},"public_addr":"172.21.15.143:6822/2301325597","cluster_addr":"172.21.15.143:6826/2301325597","heartbeat_back_addr":"172.21.15.143:6831/2301325597","heartbeat_front_addr":"172.21.15.143:6829/2301325597","state":["exists","up"]},{"osd":3,"uuid":"4e2783cf-ebb9-4dd8-8245-bc1d17ce4325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6817","nonce":2021504428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6819","nonce":2021504428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6827","nonce":2021504428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6823","nonce":2021504428}]},"public_addr":"172.21.15.143:6817/2021504428","cluster_addr":"172.21.15.143:6819/2021504428","heartbeat_back_addr":"172.21.15.143:6827/2021504428","heartbeat_front_addr":"172.21.15.143:6823/2021504428","state":["exists","up"]},{"osd":4,"uuid":"691256fa-9b77-49b9-a346-7ae1adf0ca80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6801","nonce":1993380595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6803","nonce":1993380595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6807","nonce":1993380595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6805","nonce":1993380595}]},"public_addr":"172.21.15.174:6801/1993380595","cluster_addr":"172.21.15.174:6803/1993380595","heartbeat_back_addr":"172.21.15.174:6807/1993380595","heartbeat_front_addr":"172.21.15.174:6805/1993380595","state":["exists","up"]},{"osd":5,"uuid":"756fc352-48c9-46ce-b52e-0541fe6f8525","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6809","nonce":1777923303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6811","nonce":1777923303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6815","nonce":1777923303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6813","nonce":1777923303}]},"public_addr":"172.21.15.174:6809/1777923303","cluster_addr":"172.21.15.174:6811/1777923303","heartbeat_back_addr":"172.21.15.174:6815/1777923303","heartbeat_front_addr":"172.21.15.174:6813/1777923303","state":["exists","up"]},{"osd":6,"uuid":"fab2ac2b-ed97-4cde-9825-b480eabf8b8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6825","nonce":2755130316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6827","nonce":2755130316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6831","nonce":2755130316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6829","nonce":2755130316}]},"public_addr":"172.21.15.174:6825/2755130316","cluster_addr":"172.21.15.174:6827/2755130316","heartbeat_back_addr":"172.21.15.174:6831/2755130316","heartbeat_front_addr":"172.21.15.174:6829/2755130316","state":["exists","up"]},{"osd":7,"uuid":"3de9b797-2ab6-4d41-a863-7e7d638fd703","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6817","nonce":2847141411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6819","nonce":2847141411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6823","nonce":2847141411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6821","nonce":2847141411}]},"public_addr":"172.21.15.174:6817/2847141411","cluster_addr":"172.21.15.174:6819/2847141411","heartbeat_back_addr":"172.21.15.174:6823/2847141411","heartbeat_front_addr":"172.21.15.174:6821/2847141411","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.477076+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.496890+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.539305+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.454673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.538881+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.521294+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.520712+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.540404+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:25:00.737 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.736+0000 7fe2dc9ea700 1 -- 172.21.15.143:0/1523596212 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fe2b806af00 msgr2=0x7fe2b806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:00.737 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.736+0000 7fe2dc9ea700 1 --2- 172.21.15.143:0/1523596212 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fe2b806af00 0x7fe2b806d3c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe2cc001930 tx=0x7fe2cc007000 comp rx=0 tx=0).stop 2023-11-08T19:25:00.738 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.736+0000 7fe2dc9ea700 1 -- 172.21.15.143:0/1523596212 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe2d806c5b0 msgr2=0x7fe2d815c0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:00.738 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.736+0000 7fe2dc9ea700 1 --2- 172.21.15.143:0/1523596212 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe2d806c5b0 0x7fe2d815c0e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe2c8011aa0 tx=0x7fe2c8011db0 comp rx=0 tx=0).stop 2023-11-08T19:25:00.738 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.736+0000 7fe2dc9ea700 1 -- 172.21.15.143:0/1523596212 shutdown_connections 2023-11-08T19:25:00.738 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.736+0000 7fe2dc9ea700 1 --2- 172.21.15.143:0/1523596212 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fe2b806af00 0x7fe2b806d3c0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:00.738 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.736+0000 7fe2dc9ea700 1 --2- 172.21.15.143:0/1523596212 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe2d815c620 0x7fe2d80b63d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:00.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.736+0000 7fe2dc9ea700 1 --2- 172.21.15.143:0/1523596212 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe2c0003a30 0x7fe2d806d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:00.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.736+0000 7fe2dc9ea700 1 --2- 172.21.15.143:0/1523596212 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe2d806c5b0 0x7fe2d815c0e0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:00.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.736+0000 7fe2dc9ea700 1 -- 172.21.15.143:0/1523596212 >> 172.21.15.143:0/1523596212 conn(0x7fe2d808b000 msgr2=0x7fe2d80ae480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:00.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.736+0000 7fe2dc9ea700 1 -- 172.21.15.143:0/1523596212 shutdown_connections 2023-11-08T19:25:00.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.736+0000 7fe2dc9ea700 1 -- 172.21.15.143:0/1523596212 wait complete. 2023-11-08T19:25:00.750 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-11-08T19:25:00.750 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-11-08T19:25:00.751 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-11-08T19:25:00.752 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-11-08T19:25:00.752 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-11-08T19:25:00.753 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-11-08T19:25:00.753 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-11-08T19:25:00.754 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-11-08T19:25:00.960 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.958+0000 7f0cfc5a1700 1 Processor -- start 2023-11-08T19:25:00.960 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.959+0000 7f0cfc5a1700 1 -- start start 2023-11-08T19:25:00.966 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.960+0000 7f0cfc5a1700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f0cf415ba20 0x7f0cf40b24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:00.966 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.960+0000 7f0cfc5a1700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cf406cd60 con 0x7f0cf415f660 2023-11-08T19:25:00.966 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.960+0000 7f0cfc5a1700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cf415fb60 con 0x7f0cf40b1810 2023-11-08T19:25:00.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.960+0000 7f0cfc5a1700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cf415fe40 con 0x7f0cf415ba20 2023-11-08T19:25:00.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.963+0000 7f0cfa33d700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f0cf415ba20 0x7f0cf40b24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:00.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.963+0000 7f0cfa33d700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f0cf415ba20 0x7f0cf40b24d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48798/0 (socket says 172.21.15.143:48798) 2023-11-08T19:25:00.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.963+0000 7f0cfa33d700 1 -- 172.21.15.143:0/677745760 learned_addr learned my addr 172.21.15.143:0/677745760 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:00.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.963+0000 7f0cfa33d700 1 -- 172.21.15.143:0/677745760 >> v1:172.21.15.174:6789/0 conn(0x7f0cf40b1810 legacy=0x7f0cf406d1f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-08T19:25:00.968 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.963+0000 7f0cfa33d700 1 -- 172.21.15.143:0/677745760 >> v1:172.21.15.143:6789/0 conn(0x7f0cf415f660 legacy=0x7f0cf40b1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:00.968 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.963+0000 7f0cfa33d700 1 -- 172.21.15.143:0/677745760 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cf4160120 con 0x7f0cf415ba20 2023-11-08T19:25:00.968 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.963+0000 7f0cfa33d700 1 --2- 172.21.15.143:0/677745760 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0cf415ba20 0x7f0cf40b24d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f0cec009e80 tx=0x7f0cec00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a06754553e0d2f0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:00.968 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.964+0000 7f0cf933b700 1 -- 172.21.15.143:0/677745760 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cec00e070 con 0x7f0cf415ba20 2023-11-08T19:25:00.968 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.964+0000 7f0cf933b700 1 -- 172.21.15.143:0/677745760 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cec003a20 con 0x7f0cf415ba20 2023-11-08T19:25:00.969 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.964+0000 7f0cf933b700 1 -- 172.21.15.143:0/677745760 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cec003dd0 con 0x7f0cf415ba20 2023-11-08T19:25:00.969 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.964+0000 7f0da849a700 1 Processor -- start 2023-11-08T19:25:00.970 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.965+0000 7f0cfc5a1700 1 -- 172.21.15.143:0/677745760 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0cf415ba20 msgr2=0x7f0cf40b24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:00.970 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.965+0000 7f0cfc5a1700 1 --2- 172.21.15.143:0/677745760 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0cf415ba20 0x7f0cf40b24d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f0cec009e80 tx=0x7f0cec00b330 comp rx=0 tx=0).stop 2023-11-08T19:25:00.970 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.965+0000 7f0cfc5a1700 1 -- 172.21.15.143:0/677745760 shutdown_connections 2023-11-08T19:25:00.970 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.965+0000 7f0cfc5a1700 1 --2- 172.21.15.143:0/677745760 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0cf415ba20 0x7f0cf40b24d0 secure :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f0cec009e80 tx=0x7f0cec00b330 comp rx=0 tx=0).stop 2023-11-08T19:25:00.971 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.965+0000 7f0cfc5a1700 1 -- 172.21.15.143:0/677745760 >> 172.21.15.143:0/677745760 conn(0x7f0cf406b060 msgr2=0x7f0cf406b470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:00.971 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.965+0000 7f0da849a700 1 -- start start 2023-11-08T19:25:00.972 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.966+0000 7f0da849a700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f0da0159e00 0x7f0da006ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:00.972 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.966+0000 7f0da849a700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0da00cd1c0 con 0x7f0da00b1810 2023-11-08T19:25:00.972 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.966+0000 7f0da849a700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0da015da60 con 0x7f0da0159a20 2023-11-08T19:25:00.972 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.966+0000 7f0da849a700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0da00cd510 con 0x7f0da0159e00 2023-11-08T19:25:00.973 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.966+0000 7f0cfc5a1700 1 -- 172.21.15.143:0/677745760 shutdown_connections 2023-11-08T19:25:00.973 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.966+0000 7f0da5a35700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f0da00b1810 0x7f0da00ca100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:44956/0 (socket says 172.21.15.143:44956) 2023-11-08T19:25:00.973 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.966+0000 7f0da5a35700 1 -- 172.21.15.143:0/1808885908 learned_addr learned my addr 172.21.15.143:0/1808885908 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:25:00.974 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.965+0000 7fcfeb995700 1 Processor -- start 2023-11-08T19:25:00.974 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.966+0000 7fb50800c700 1 Processor -- start 2023-11-08T19:25:00.974 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.966+0000 7fb50800c700 1 -- start start 2023-11-08T19:25:00.975 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.966+0000 7f0cfc5a1700 1 -- 172.21.15.143:0/677745760 wait complete. 2023-11-08T19:25:00.975 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.968+0000 7fb50800c700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb5000cbe00 0x7fb5000cb1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:00.975 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.968+0000 7fb50800c700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fb500072130 0x7fb50006daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:00.976 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.968+0000 7fb50800c700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5000ce790 con 0x7fb500071d50 2023-11-08T19:25:00.976 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.968+0000 7fb50800c700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5000c69a0 con 0x7fb500072130 2023-11-08T19:25:00.976 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.968+0000 7fb50800c700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb50006f970 con 0x7fb5000cbe00 2023-11-08T19:25:00.976 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.968+0000 7f0cfc5a1700 1 Processor -- start 2023-11-08T19:25:00.977 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.968+0000 7f0cfc5a1700 1 -- start start 2023-11-08T19:25:00.977 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.966+0000 7f0da5234700 1 -- 172.21.15.143:0/1808885908 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1834493100 0 0) 0x7f0da00cd1c0 con 0x7f0da00b1810 2023-11-08T19:25:00.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.966+0000 7f0da5234700 1 -- 172.21.15.143:0/1808885908 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d8c0035b0 con 0x7f0da00b1810 2023-11-08T19:25:00.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.967+0000 7f0da5234700 1 -- 172.21.15.143:0/1808885908 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1435430016 0 0) 0x7f0d8c0035b0 con 0x7f0da00b1810 2023-11-08T19:25:00.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.967+0000 7f0da5234700 1 -- 172.21.15.143:0/1808885908 >> v2:172.21.15.174:3300/0 conn(0x7f0da0159e00 msgr2=0x7f0da006ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:00.979 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.967+0000 7f0da5234700 1 --2- 172.21.15.143:0/1808885908 >> v2:172.21.15.174:3300/0 conn(0x7f0da0159e00 0x7f0da006ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:00.979 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.967+0000 7f0da5234700 1 -- 172.21.15.143:0/1808885908 >> v1:172.21.15.174:6789/0 conn(0x7f0da0159a20 legacy=0x7f0da00b1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:00.979 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.967+0000 7f0da5234700 1 -- 172.21.15.143:0/1808885908 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0da00cad70 con 0x7f0da00b1810 2023-11-08T19:25:00.980 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.967+0000 7f0da5234700 1 -- 172.21.15.143:0/1808885908 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4147756303 0 0) 0x7f0d9c001670 con 0x7f0da00b1810 2023-11-08T19:25:00.980 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.967+0000 7f0da5234700 1 -- 172.21.15.143:0/1808885908 >> v1:172.21.15.143:6789/0 conn(0x7f0da00b1810 legacy=0x7f0da00ca100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:00.980 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.967+0000 7f0da5234700 1 --2- 172.21.15.143:0/1808885908 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0d8c003a00 0x7f0d8c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:00.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.967+0000 7f0da5234700 1 --2- 172.21.15.143:0/1808885908 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0d8c004850 0x7f0d8c004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:00.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.967+0000 7f0da5234700 1 --2- 172.21.15.143:0/1808885908 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d8c005410 0x7f0d8c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:00.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.967+0000 7f0da5234700 1 -- 172.21.15.143:0/1808885908 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0da00cd510 con 0x7f0d8c003a00 2023-11-08T19:25:00.982 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.967+0000 7f0da5234700 1 -- 172.21.15.143:0/1808885908 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d8c00c2f0 con 0x7f0d8c005410 2023-11-08T19:25:00.982 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.967+0000 7f0da5234700 1 -- 172.21.15.143:0/1808885908 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d8c00c4b0 con 0x7f0d8c004850 2023-11-08T19:25:00.982 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.967+0000 7f0da6236700 1 --2- 172.21.15.143:0/1808885908 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0d8c003a00 0x7f0d8c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:00.983 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.968+0000 7f0da6236700 1 -- 172.21.15.143:0/1808885908 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0d8c004850 msgr2=0x7f0d8c004cd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-08T19:25:00.983 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.968+0000 7f0da6236700 1 --2- 172.21.15.143:0/1808885908 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0d8c004850 0x7f0d8c004cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:00.983 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.968+0000 7f0da6236700 1 -- 172.21.15.143:0/1808885908 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d8c005410 msgr2=0x7f0d8c00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:00.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.968+0000 7f0da6a37700 1 --2- 172.21.15.143:0/1808885908 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d8c005410 0x7f0d8c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:00.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.968+0000 7f0da6236700 1 --2- 172.21.15.143:0/1808885908 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d8c005410 0x7f0d8c00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:00.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.968+0000 7f0da6236700 1 -- 172.21.15.143:0/1808885908 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d8c0035b0 con 0x7f0d8c003a00 2023-11-08T19:25:00.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.968+0000 7f0da6a37700 1 --2- 172.21.15.143:0/1808885908 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d8c005410 0x7f0d8c00ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:25:00.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.968+0000 7f0da6a37700 1 -- 172.21.15.143:0/1808885908 reap_dead start 2023-11-08T19:25:00.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.968+0000 7f0da6236700 1 --2- 172.21.15.143:0/1808885908 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0d8c003a00 0x7f0d8c003de0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f0d90001900 tx=0x7f0d8c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f079ac9bc6a3900 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:00.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.968+0000 7f0da5234700 1 -- 172.21.15.143:0/1808885908 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d900052d0 con 0x7f0d8c003a00 2023-11-08T19:25:00.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.968+0000 7f0da5234700 1 -- 172.21.15.143:0/1808885908 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d90005430 con 0x7f0d8c003a00 2023-11-08T19:25:00.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.968+0000 7f0da5234700 1 -- 172.21.15.143:0/1808885908 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d900038c0 con 0x7f0d8c003a00 2023-11-08T19:25:00.986 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.968+0000 7fb505da8700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fb500071d50 0x7fb50006d390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:44968/0 (socket says 172.21.15.143:44968) 2023-11-08T19:25:00.986 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.968+0000 7fb505da8700 1 -- 172.21.15.143:0/4058188630 learned_addr learned my addr 172.21.15.143:0/4058188630 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:25:00.987 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.967+0000 7fcfeb995700 1 -- start start 2023-11-08T19:25:00.987 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.968+0000 7fcfeb995700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcfe40b1810 0x7fcfe40aea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:00.987 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.968+0000 7fcfeb995700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fcfe406d420 0x7fcfe406d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:00.988 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.969+0000 7fcfeb995700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfe406cf90 con 0x7fcfe4162060 2023-11-08T19:25:00.988 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.969+0000 7fcfeb995700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfe40c9c40 con 0x7fcfe406d420 2023-11-08T19:25:00.988 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.969+0000 7fcfeb995700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfe40cdb60 con 0x7fcfe40b1810 2023-11-08T19:25:00.988 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.969+0000 7fcfe9f32700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fcfe406d420 0x7fcfe406d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:00.988 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.969+0000 7fcfe9f32700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fcfe406d420 0x7fcfe406d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48820/0 (socket says 172.21.15.143:48820) 2023-11-08T19:25:00.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.969+0000 7fcfe9f32700 1 -- 172.21.15.143:0/958443314 learned_addr learned my addr 172.21.15.143:0/958443314 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:00.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.969+0000 7fcfe8f30700 1 --2- 172.21.15.143:0/958443314 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcfe40b1810 0x7fcfe40aea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:00.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.969+0000 7f0cfc5a1700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0cf40b1810 0x7f0cf40c7240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:00.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.969+0000 7f0cfc5a1700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0cf415f660 0x7f0cf414e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:00.990 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.969+0000 7f0cfc5a1700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0cf40cee90 0x7f0cf415c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:00.990 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.969+0000 7f0cfc5a1700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf40c6e20 con 0x7f0cf415f660 2023-11-08T19:25:00.990 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.969+0000 7f0cfc5a1700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf415ec40 con 0x7f0cf40b1810 2023-11-08T19:25:00.990 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.969+0000 7f0cfc5a1700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf415e960 con 0x7f0cf40cee90 2023-11-08T19:25:00.991 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.969+0000 7fcfe9f32700 1 -- 172.21.15.143:0/958443314 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcfe40b1810 msgr2=0x7fcfe40aea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:00.991 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.969+0000 7fcfe9f32700 1 --2- 172.21.15.143:0/958443314 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcfe40b1810 0x7fcfe40aea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:00.991 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.969+0000 7fcfe9f32700 1 -- 172.21.15.143:0/958443314 >> v1:172.21.15.143:6789/0 conn(0x7fcfe4162060 legacy=0x7fcfe40b1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:00.992 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.969+0000 7fcfe9f32700 1 -- 172.21.15.143:0/958443314 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfe4071be0 con 0x7fcfe406d420 2023-11-08T19:25:00.992 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.970+0000 7fcfe8f30700 1 --2- 172.21.15.143:0/958443314 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcfe40b1810 0x7fcfe40aea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:25:00.992 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.969+0000 7fb504da6700 1 -- 172.21.15.143:0/4058188630 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 432970322 0 0) 0x7fb5000ce790 con 0x7fb500071d50 2023-11-08T19:25:00.993 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.969+0000 7fb504da6700 1 -- 172.21.15.143:0/4058188630 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb4ec0035b0 con 0x7fb500071d50 2023-11-08T19:25:00.993 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.969+0000 7fb5055a7700 1 --2- 172.21.15.143:0/4058188630 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb5000cbe00 0x7fb5000cb1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:00.993 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.970+0000 7fb5065a9700 1 --2- 172.21.15.143:0/4058188630 >> v2:172.21.15.174:3300/0 conn(0x7fb500072130 0x7fb50006daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:00.993 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.970+0000 7f0da849a700 1 -- 172.21.15.143:0/1808885908 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0d8c003a00 msgr2=0x7f0d8c003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:00.994 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.970+0000 7f0da849a700 1 --2- 172.21.15.143:0/1808885908 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0d8c003a00 0x7f0d8c003de0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f0d90001900 tx=0x7f0d8c005880 comp rx=0 tx=0).stop 2023-11-08T19:25:00.994 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.970+0000 7fb5055a7700 1 -- 172.21.15.143:0/4058188630 >> v2:172.21.15.174:3300/0 conn(0x7fb500072130 msgr2=0x7fb50006daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:00.995 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.970+0000 7fb5055a7700 1 --2- 172.21.15.143:0/4058188630 >> v2:172.21.15.174:3300/0 conn(0x7fb500072130 0x7fb50006daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:00.995 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.970+0000 7fb5055a7700 1 -- 172.21.15.143:0/4058188630 >> v1:172.21.15.143:6789/0 conn(0x7fb500071d50 legacy=0x7fb50006d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:00.995 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.970+0000 7fb5055a7700 1 -- 172.21.15.143:0/4058188630 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb50006fc50 con 0x7fb5000cbe00 2023-11-08T19:25:00.995 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.970+0000 7fcfe9f32700 1 --2- 172.21.15.143:0/958443314 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcfe406d420 0x7fcfe406d8b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fcfdc009e50 tx=0x7fcfdc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78c95dd4753aa111 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:00.996 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.970+0000 7f0da849a700 1 -- 172.21.15.143:0/1808885908 shutdown_connections 2023-11-08T19:25:00.996 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.970+0000 7f0da849a700 1 --2- 172.21.15.143:0/1808885908 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0d8c003a00 0x7f0d8c003de0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:00.996 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.970+0000 7f0da849a700 1 -- 172.21.15.143:0/1808885908 >> 172.21.15.143:0/1808885908 conn(0x7f0da006b060 msgr2=0x7f0da006b470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:00.997 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.970+0000 7f2a725c5700 1 Processor -- start 2023-11-08T19:25:00.997 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.970+0000 7f2a725c5700 1 -- start start 2023-11-08T19:25:00.997 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.970+0000 7f0cf9b3c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0cf415f660 0x7f0cf414e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:00.997 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.970+0000 7f0cf9b3c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0cf415f660 0x7f0cf414e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48826/0 (socket says 172.21.15.143:48826) 2023-11-08T19:25:00.998 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.970+0000 7f0cfa33d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0cf40b1810 0x7f0cf40c7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:00.998 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.970+0000 7f0cf9b3c700 1 -- 172.21.15.143:0/3426336670 learned_addr learned my addr 172.21.15.143:0/3426336670 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:00.998 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.970+0000 7f0cfab3e700 1 --2- 172.21.15.143:0/3426336670 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0cf40cee90 0x7f0cf415c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:00.999 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.970+0000 7fb5055a7700 1 --2- 172.21.15.143:0/4058188630 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb5000cbe00 0x7fb5000cb1c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fb4f0009870 tx=0x7fb4f0009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a46aeb663d6d4a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:00.999 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.970+0000 7f0cfab3e700 1 -- 172.21.15.143:0/3426336670 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0cf40b1810 msgr2=0x7f0cf40c7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:00.999 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.970+0000 7f0cfab3e700 1 --2- 172.21.15.143:0/3426336670 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0cf40b1810 0x7f0cf40c7240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:00.999 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.970+0000 7f0cfab3e700 1 -- 172.21.15.143:0/3426336670 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0cf415f660 msgr2=0x7f0cf414e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:01.000 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.970+0000 7f0cfab3e700 1 --2- 172.21.15.143:0/3426336670 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0cf415f660 0x7f0cf414e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:01.000 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.970+0000 7f0cfab3e700 1 -- 172.21.15.143:0/3426336670 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cec009b30 con 0x7f0cf40cee90 2023-11-08T19:25:01.000 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.970+0000 7fcfdbfff700 1 -- 172.21.15.143:0/958443314 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfdc00e070 con 0x7fcfe406d420 2023-11-08T19:25:01.001 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.970+0000 7f0cf9b3c700 1 --2- 172.21.15.143:0/3426336670 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0cf415f660 0x7f0cf414e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:25:01.001 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.971+0000 7f0cfab3e700 1 --2- 172.21.15.143:0/3426336670 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0cf40cee90 0x7f0cf415c130 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f0cf40723b0 tx=0x7f0cf000b720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:01.001 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.971+0000 7f2a725c5700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f2a6c0c2a50 0x7f2a6c0cd2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:01.002 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.971+0000 7f2a725c5700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2a6c0b1810 0x7f2a6c0c9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:01.002 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.971+0000 7f2a725c5700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a6c070e60 con 0x7f2a6c0cd820 2023-11-08T19:25:01.002 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.971+0000 7f2a725c5700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a6c0cddf0 con 0x7f2a6c0c2a50 2023-11-08T19:25:01.002 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.971+0000 7f2a725c5700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a6c0710b0 con 0x7f2a6c0b1810 2023-11-08T19:25:01.003 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.971+0000 7f2a70b62700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2a6c0b1810 0x7f2a6c0c9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:01.003 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.971+0000 7f2a6bfff700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f2a6c0c2a50 0x7f2a6c0cd2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:01.003 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.971+0000 7f2a70b62700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2a6c0b1810 0x7f2a6c0c9cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.143:36474/0 (socket says 172.21.15.143:36474) 2023-11-08T19:25:01.003 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.971+0000 7f2a70b62700 1 -- 172.21.15.143:0/1790216944 learned_addr learned my addr 172.21.15.143:0/1790216944 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:01.004 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.970+0000 7fcfdbfff700 1 -- 172.21.15.143:0/958443314 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfdc003a20 con 0x7fcfe406d420 2023-11-08T19:25:01.004 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.970+0000 7fcfdbfff700 1 -- 172.21.15.143:0/958443314 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfdc003dd0 con 0x7fcfe406d420 2023-11-08T19:25:01.004 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.971+0000 7fcfeb995700 1 -- 172.21.15.143:0/958443314 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcfe406d420 msgr2=0x7fcfe406d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:01.004 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.971+0000 7fcfeb995700 1 --2- 172.21.15.143:0/958443314 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcfe406d420 0x7fcfe406d8b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fcfdc009e50 tx=0x7fcfdc00b330 comp rx=0 tx=0).stop 2023-11-08T19:25:01.005 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.971+0000 7fcfeb995700 1 -- 172.21.15.143:0/958443314 shutdown_connections 2023-11-08T19:25:01.005 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.971+0000 7fcfeb995700 1 --2- 172.21.15.143:0/958443314 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcfe406d420 0x7fcfe406d8b0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:01.005 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.971+0000 7fcfeb995700 1 --2- 172.21.15.143:0/958443314 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcfe40b1810 0x7fcfe40aea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:01.005 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.971+0000 7fcfeb995700 1 -- 172.21.15.143:0/958443314 >> 172.21.15.143:0/958443314 conn(0x7fcfe406b4c0 msgr2=0x7fcfe40cf680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:01.006 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.971+0000 7fb504da6700 1 -- 172.21.15.143:0/4058188630 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f0011830 con 0x7fb5000cbe00 2023-11-08T19:25:01.006 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.971+0000 7fb504da6700 1 -- 172.21.15.143:0/4058188630 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4f0011990 con 0x7fb5000cbe00 2023-11-08T19:25:01.006 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.971+0000 7fb504da6700 1 -- 172.21.15.143:0/4058188630 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f0011d20 con 0x7fb5000cbe00 2023-11-08T19:25:01.006 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.971+0000 7fb50800c700 1 -- 172.21.15.143:0/4058188630 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb5000cbe00 msgr2=0x7fb5000cb1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:01.006 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.972+0000 7fb50800c700 1 --2- 172.21.15.143:0/4058188630 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb5000cbe00 0x7fb5000cb1c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fb4f0009870 tx=0x7fb4f0009b80 comp rx=0 tx=0).stop 2023-11-08T19:25:01.007 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.970+0000 7f0da849a700 1 -- 172.21.15.143:0/1808885908 shutdown_connections 2023-11-08T19:25:01.007 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.970+0000 7f0da849a700 1 -- 172.21.15.143:0/1808885908 wait complete. 2023-11-08T19:25:01.007 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.972+0000 7f0ceaffd700 1 -- 172.21.15.143:0/3426336670 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf0011830 con 0x7f0cf40cee90 2023-11-08T19:25:01.008 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.972+0000 7fb50800c700 1 -- 172.21.15.143:0/4058188630 shutdown_connections 2023-11-08T19:25:01.008 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.972+0000 7fb50800c700 1 --2- 172.21.15.143:0/4058188630 >> v2:172.21.15.174:3300/0 conn(0x7fb500072130 0x7fb50006daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:01.008 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.972+0000 7fb50800c700 1 --2- 172.21.15.143:0/4058188630 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb5000cbe00 0x7fb5000cb1c0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:01.008 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.972+0000 7fb50800c700 1 -- 172.21.15.143:0/4058188630 >> 172.21.15.143:0/4058188630 conn(0x7fb5000b1650 msgr2=0x7fb5000b1a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:01.009 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.972+0000 7f0ceaffd700 1 -- 172.21.15.143:0/3426336670 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cf000be90 con 0x7f0cf40cee90 2023-11-08T19:25:01.009 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.972+0000 7f0cfc5a1700 1 -- 172.21.15.143:0/3426336670 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cf40c6760 con 0x7f0cf40cee90 2023-11-08T19:25:01.009 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.972+0000 7f0cfc5a1700 1 -- 172.21.15.143:0/3426336670 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cf40c7780 con 0x7f0cf40cee90 2023-11-08T19:25:01.009 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.972+0000 7f0ceaffd700 1 -- 172.21.15.143:0/3426336670 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf0011bd0 con 0x7f0cf40cee90 2023-11-08T19:25:01.010 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.972+0000 7f2a70b62700 1 -- 172.21.15.143:0/1790216944 >> v2:172.21.15.174:3300/0 conn(0x7f2a6c0c2a50 msgr2=0x7f2a6c0cd2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:01.010 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.972+0000 7f2a70b62700 1 --2- 172.21.15.143:0/1790216944 >> v2:172.21.15.174:3300/0 conn(0x7f2a6c0c2a50 0x7f2a6c0cd2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:01.010 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.972+0000 7f2a70b62700 1 -- 172.21.15.143:0/1790216944 >> v1:172.21.15.143:6789/0 conn(0x7f2a6c0cd820 legacy=0x7f2a6c0b1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:01.011 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.972+0000 7f2a70b62700 1 -- 172.21.15.143:0/1790216944 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a6c071390 con 0x7f2a6c0b1810 2023-11-08T19:25:01.011 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.972+0000 7fcfeb995700 1 -- 172.21.15.143:0/958443314 shutdown_connections 2023-11-08T19:25:01.011 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.972+0000 7f0da849a700 1 Processor -- start 2023-11-08T19:25:01.012 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.972+0000 7f2a70b62700 1 --2- 172.21.15.143:0/1790216944 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2a6c0b1810 0x7f2a6c0c9cf0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c009920 tx=0x7f2a5c009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17a61120d0c885f server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:01.012 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.973+0000 7fb50800c700 1 -- 172.21.15.143:0/4058188630 shutdown_connections 2023-11-08T19:25:01.013 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.972+0000 7f0da849a700 1 -- start start 2023-11-08T19:25:01.013 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.973+0000 7f0da849a700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0d8c003a00 0x7f0da00b33c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:01.013 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.973+0000 7f0da849a700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0da015f4f0 0x7f0da01562a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:01.013 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.973+0000 7f0da849a700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0da01581c0 0x7f0da0158670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:01.014 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.973+0000 7f0da849a700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0da0158bb0 con 0x7f0da01581c0 2023-11-08T19:25:01.014 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.973+0000 7f0da849a700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0da00b45b0 con 0x7f0da015f4f0 2023-11-08T19:25:01.014 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.973+0000 7f0da849a700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0da00b39a0 con 0x7f0d8c003a00 2023-11-08T19:25:01.014 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.974+0000 7f0da5a35700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0da015f4f0 0x7f0da01562a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:01.015 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.973+0000 7fb50800c700 1 -- 172.21.15.143:0/4058188630 wait complete. 2023-11-08T19:25:01.015 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.974+0000 7f0cfc5a1700 1 -- 172.21.15.143:0/3426336670 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0cdc000fc0 con 0x7f0cf40cee90 2023-11-08T19:25:01.016 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.974+0000 7f0da5a35700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0da015f4f0 0x7f0da01562a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.143:60570/0 (socket says 172.21.15.143:60570) 2023-11-08T19:25:01.016 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.974+0000 7f0da5a35700 1 -- 172.21.15.143:0/4282759718 learned_addr learned my addr 172.21.15.143:0/4282759718 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:01.016 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.974+0000 7f0da6a37700 1 --2- 172.21.15.143:0/4282759718 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0da01581c0 0x7f0da0158670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:01.016 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.974+0000 7f3e25fce700 1 Processor -- start 2023-11-08T19:25:01.017 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.974+0000 7f0da5a35700 1 -- 172.21.15.143:0/4282759718 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0d8c003a00 msgr2=0x7f0da00b33c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:01.017 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.974+0000 7f0da5a35700 1 --2- 172.21.15.143:0/4282759718 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0d8c003a00 0x7f0da00b33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:01.017 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.974+0000 7f0da5a35700 1 -- 172.21.15.143:0/4282759718 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0da01581c0 msgr2=0x7f0da0158670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:01.018 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.974+0000 7f0da5a35700 1 --2- 172.21.15.143:0/4282759718 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0da01581c0 0x7f0da0158670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:01.018 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.974+0000 7f0da5a35700 1 -- 172.21.15.143:0/4282759718 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0da00b3c80 con 0x7f0da015f4f0 2023-11-08T19:25:01.018 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.974+0000 7f3e25fce700 1 -- start start 2023-11-08T19:25:01.019 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.975+0000 7f0ceaffd700 1 -- 172.21.15.143:0/3426336670 <== mon.2 v2:172.21.15.174:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0cf0011d30 con 0x7f0cf40cee90 2023-11-08T19:25:01.019 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.973+0000 7f2a6affd700 1 -- 172.21.15.143:0/1790216944 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a5c004230 con 0x7f2a6c0b1810 2023-11-08T19:25:01.019 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.973+0000 7f2a6affd700 1 -- 172.21.15.143:0/1790216944 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a5c004390 con 0x7f2a6c0b1810 2023-11-08T19:25:01.020 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.973+0000 7f2a6affd700 1 -- 172.21.15.143:0/1790216944 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a5c0036a0 con 0x7f2a6c0b1810 2023-11-08T19:25:01.020 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.975+0000 7f2a725c5700 1 -- 172.21.15.143:0/1790216944 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2a6c0b1810 msgr2=0x7f2a6c0c9cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:01.020 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.975+0000 7f2a725c5700 1 --2- 172.21.15.143:0/1790216944 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2a6c0b1810 0x7f2a6c0c9cf0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c009920 tx=0x7f2a5c009c30 comp rx=0 tx=0).stop 2023-11-08T19:25:01.020 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.975+0000 7f3e25fce700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f3e20159a20 0x7f3e2006ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:01.021 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.975+0000 7f3e25fce700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f3e20159e00 0x7f3e200b1100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:01.021 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.975+0000 7f3e25fce700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e200cd1c0 con 0x7f3e200cc100 2023-11-08T19:25:01.021 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.975+0000 7f3e25fce700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e200cd510 con 0x7f3e20159a20 2023-11-08T19:25:01.021 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.975+0000 7f3e25fce700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e2015da60 con 0x7f3e20159e00 2023-11-08T19:25:01.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.975+0000 7f3e1effd700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f3e20159e00 0x7f3e200b1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:01.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.976+0000 7f3e1effd700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f3e20159e00 0x7f3e200b1100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.143:60578/0 (socket says 172.21.15.143:60578) 2023-11-08T19:25:01.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.976+0000 7f3e1effd700 1 -- 172.21.15.143:0/876775502 learned_addr learned my addr 172.21.15.143:0/876775502 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:01.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.973+0000 7fcfeb995700 1 -- 172.21.15.143:0/958443314 wait complete. 2023-11-08T19:25:01.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.975+0000 7fcfeb995700 1 Processor -- start 2023-11-08T19:25:01.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.975+0000 7fcfeb995700 1 -- start start 2023-11-08T19:25:01.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.975+0000 7fb50800c700 1 Processor -- start 2023-11-08T19:25:01.024 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.975+0000 7fb50800c700 1 -- start start 2023-11-08T19:25:01.024 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.974+0000 7f0da5a35700 1 --2- 172.21.15.143:0/4282759718 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0da015f4f0 0x7f0da01562a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c00c730 tx=0x7f0d9c00ca40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:01.024 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.974+0000 7f0d96ffd700 1 -- 172.21.15.143:0/4282759718 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d9c004c80 con 0x7f0da015f4f0 2023-11-08T19:25:01.025 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.975+0000 7f0d96ffd700 1 -- 172.21.15.143:0/4282759718 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d9c004de0 con 0x7f0da015f4f0 2023-11-08T19:25:01.025 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.975+0000 7f0d96ffd700 1 -- 172.21.15.143:0/4282759718 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d9c007310 con 0x7f0da015f4f0 2023-11-08T19:25:01.025 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.975+0000 7f0da849a700 1 -- 172.21.15.143:0/4282759718 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0da00b3f60 con 0x7f0da015f4f0 2023-11-08T19:25:01.025 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.975+0000 7f0da849a700 1 -- 172.21.15.143:0/4282759718 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0da015f970 con 0x7f0da015f4f0 2023-11-08T19:25:01.026 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.975+0000 7f3a49fce700 1 Processor -- start 2023-11-08T19:25:01.026 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.976+0000 7f3e1f7fe700 1 --2- 172.21.15.143:0/876775502 >> v2:172.21.15.143:3300/0 conn(0x7f3e20159a20 0x7f3e2006ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:01.027 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.975+0000 7f0ceaffd700 1 --2- 172.21.15.143:0/3426336670 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f0cd806afd0 0x7f0cd806d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:01.027 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.976+0000 7f0ceaffd700 1 -- 172.21.15.143:0/3426336670 <== mon.2 v2:172.21.15.174:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0cf00894d0 con 0x7f0cf40cee90 2023-11-08T19:25:01.027 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.976+0000 7f0ceaffd700 1 --2- 172.21.15.143:0/3426336670 >> [v2:172.21.15.143:6808/1487830566,v1:172.21.15.143:6809/1487830566] conn(0x7f0cd80731f0 0x7f0cd8075610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:01.027 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.976+0000 7f0ceaffd700 1 -- 172.21.15.143:0/3426336670 --> [v2:172.21.15.143:6808/1487830566,v1:172.21.15.143:6809/1487830566] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0cd8075cc0 con 0x7f0cd80731f0 2023-11-08T19:25:01.027 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.976+0000 7f0ceaffd700 1 -- 172.21.15.143:0/3426336670 <== mon.2 v2:172.21.15.174:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0cf008baf0 con 0x7f0cf40cee90 2023-11-08T19:25:01.028 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.976+0000 7f3e1effd700 1 -- 172.21.15.143:0/876775502 >> v2:172.21.15.143:3300/0 conn(0x7f3e20159a20 msgr2=0x7f3e2006ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:01.028 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.976+0000 7f3e1effd700 1 --2- 172.21.15.143:0/876775502 >> v2:172.21.15.143:3300/0 conn(0x7f3e20159a20 0x7f3e2006ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:01.028 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.976+0000 7f3e1effd700 1 -- 172.21.15.143:0/876775502 >> v1:172.21.15.143:6789/0 conn(0x7f3e200cc100 legacy=0x7f3e200ca100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:01.028 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.976+0000 7f3e1effd700 1 -- 172.21.15.143:0/876775502 --> v2:172.21.15.174:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e200cad70 con 0x7f3e20159e00 2023-11-08T19:25:01.029 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.976+0000 7fcfeb995700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcfe406d420 0x7fcfe40c4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:01.029 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.976+0000 7fcfeb995700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcfe40b1810 0x7fcfe4149ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:01.029 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.976+0000 7fcfeb995700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcfe4162060 0x7fcfe4160590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:01.029 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.976+0000 7fcfeb995700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfe4147700 con 0x7fcfe406d420 2023-11-08T19:25:01.030 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.976+0000 7fcfeb995700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfe41479e0 con 0x7fcfe40b1810 2023-11-08T19:25:01.030 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.976+0000 7fcfeb995700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfe4147cc0 con 0x7fcfe4162060 2023-11-08T19:25:01.030 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.976+0000 7f0cf9b3c700 1 --2- 172.21.15.143:0/3426336670 >> [v2:172.21.15.143:6808/1487830566,v1:172.21.15.143:6809/1487830566] conn(0x7f0cd80731f0 0x7f0cd8075610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:01.031 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.976+0000 7f0cfa33d700 1 --2- 172.21.15.143:0/3426336670 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f0cd806afd0 0x7f0cd806d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:01.031 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.976+0000 7f3e1effd700 1 --2- 172.21.15.143:0/876775502 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3e20159e00 0x7f3e200b1100 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3e200cf5e0 tx=0x7f3e14009bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=107d77ccf892dad3 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:01.031 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.976+0000 7f2a725c5700 1 -- 172.21.15.143:0/1790216944 shutdown_connections 2023-11-08T19:25:01.032 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.976+0000 7f2a725c5700 1 --2- 172.21.15.143:0/1790216944 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2a6c0b1810 0x7f2a6c0c9cf0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:01.032 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.976+0000 7f2a725c5700 1 --2- 172.21.15.143:0/1790216944 >> v2:172.21.15.174:3300/0 conn(0x7f2a6c0c2a50 0x7f2a6c0cd2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:01.032 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.976+0000 7f2a725c5700 1 -- 172.21.15.143:0/1790216944 >> 172.21.15.143:0/1790216944 conn(0x7f2a6c06b060 msgr2=0x7f2a6c06b470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:01.032 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.976+0000 7f7dc54a9700 1 Processor -- start 2023-11-08T19:25:01.033 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.976+0000 7fb50800c700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb500071d50 0x7fb5001ebdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:01.033 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.976+0000 7fb50800c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb500072130 0x7fb5001dbe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:01.034 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.976+0000 7fb50800c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb5000cbe00 0x7fb5001f8340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:01.034 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.976+0000 7fb50800c700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5001dc340 con 0x7fb500072130 2023-11-08T19:25:01.034 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.976+0000 7fb50800c700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5000c3590 con 0x7fb5000cbe00 2023-11-08T19:25:01.034 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.976+0000 7fb50800c700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5000c5230 con 0x7fb500071d50 2023-11-08T19:25:01.034 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.976+0000 7fb5055a7700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb500072130 0x7fb5001dbe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:01.035 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.976+0000 7fb5055a7700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb500072130 0x7fb5001dbe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48860/0 (socket says 172.21.15.143:48860) 2023-11-08T19:25:01.035 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.976+0000 7fb5055a7700 1 -- 172.21.15.143:0/2072945897 learned_addr learned my addr 172.21.15.143:0/2072945897 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:01.035 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.977+0000 7f2a725c5700 1 -- 172.21.15.143:0/1790216944 shutdown_connections 2023-11-08T19:25:01.036 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.977+0000 7f7dc54a9700 1 -- start start 2023-11-08T19:25:01.036 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.976+0000 7f3a49fce700 1 -- start start 2023-11-08T19:25:01.036 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.977+0000 7f3a49fce700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f3a44159a20 0x7f3a4406ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:01.037 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.977+0000 7f3a49fce700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a440cd1c0 con 0x7f3a440b1810 2023-11-08T19:25:01.037 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.977+0000 7f3a49fce700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a4415da60 con 0x7f3a44159e00 2023-11-08T19:25:01.037 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.977+0000 7f3a49fce700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a440cd510 con 0x7f3a44159a20 2023-11-08T19:25:01.038 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.977+0000 7f3e1e7fc700 1 -- 172.21.15.143:0/876775502 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e14004230 con 0x7f3e20159e00 2023-11-08T19:25:01.038 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.976+0000 7fcfe8f30700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcfe40b1810 0x7fcfe4149ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:01.039 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.977+0000 7fcfe8f30700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcfe40b1810 0x7fcfe4149ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.143:60586/0 (socket says 172.21.15.143:60586) 2023-11-08T19:25:01.039 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.977+0000 7fcfe8f30700 1 -- 172.21.15.143:0/1174702490 learned_addr learned my addr 172.21.15.143:0/1174702490 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:01.039 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.977+0000 7fcfe9f32700 1 --2- 172.21.15.143:0/1174702490 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcfe4162060 0x7fcfe4160590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:01.039 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.977+0000 7fcfe9731700 1 --2- 172.21.15.143:0/1174702490 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcfe406d420 0x7fcfe40c4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:01.039 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.977+0000 7fcfe9f32700 1 -- 172.21.15.143:0/1174702490 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcfe40b1810 msgr2=0x7fcfe4149ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:01.040 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.977+0000 7fcfe9f32700 1 --2- 172.21.15.143:0/1174702490 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcfe40b1810 0x7fcfe4149ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:01.040 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.977+0000 7fcfe9f32700 1 -- 172.21.15.143:0/1174702490 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcfe406d420 msgr2=0x7fcfe40c4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:01.040 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.977+0000 7fcfe9f32700 1 --2- 172.21.15.143:0/1174702490 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcfe406d420 0x7fcfe40c4fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:01.040 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.977+0000 7fcfe9f32700 1 -- 172.21.15.143:0/1174702490 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfdc009b30 con 0x7fcfe4162060 2023-11-08T19:25:01.040 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.978+0000 7fcfe9f32700 1 --2- 172.21.15.143:0/1174702490 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcfe4162060 0x7fcfe4160590 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fcfdc014000 tx=0x7fcfdc004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:01.041 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.978+0000 7f7dc54a9700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7dc015dc80 0x7f7dc00d1f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:01.041 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.978+0000 7f7dc54a9700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f7dc00cf090 0x7f7dc00b9690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:01.041 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.978+0000 7f7dc54a9700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7dc00d3200 con 0x7f7dc00d3690 2023-11-08T19:25:01.041 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.978+0000 7f7dc54a9700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7dc00b9bd0 con 0x7f7dc00cf090 2023-11-08T19:25:01.042 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.978+0000 7f7dc54a9700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7dc00b7cf0 con 0x7f7dc015dc80 2023-11-08T19:25:01.042 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.977+0000 7f0cfa33d700 1 --2- 172.21.15.143:0/3426336670 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f0cd806afd0 0x7f0cd806d490 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0cf40cf4c0 tx=0x7f0cec009000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:01.043 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.977+0000 7f0cf9b3c700 1 --2- 172.21.15.143:0/3426336670 >> [v2:172.21.15.143:6808/1487830566,v1:172.21.15.143:6809/1487830566] conn(0x7f0cd80731f0 0x7f0cd8075610 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:01.043 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.978+0000 7f0ceaffd700 1 -- 172.21.15.143:0/3426336670 <== osd.0 v2:172.21.15.143:6808/1487830566 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26745 (crc 0 0 0) 0x7f0cd8075cc0 con 0x7f0cd80731f0 2023-11-08T19:25:01.043 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.978+0000 7fcfd9ffb700 1 -- 172.21.15.143:0/1174702490 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfdc0092e0 con 0x7fcfe4162060 2023-11-08T19:25:01.044 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.978+0000 7fb5055a7700 1 -- 172.21.15.143:0/2072945897 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb500071d50 msgr2=0x7fb5001ebdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:01.044 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.978+0000 7fb5055a7700 1 --2- 172.21.15.143:0/2072945897 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb500071d50 0x7fb5001ebdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:01.044 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.978+0000 7fb5055a7700 1 -- 172.21.15.143:0/2072945897 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb5000cbe00 msgr2=0x7fb5001f8340 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-08T19:25:01.044 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.978+0000 7fb5055a7700 1 --2- 172.21.15.143:0/2072945897 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb5000cbe00 0x7fb5001f8340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:01.044 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.978+0000 7fb5055a7700 1 -- 172.21.15.143:0/2072945897 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4f00094a0 con 0x7fb500072130 2023-11-08T19:25:01.045 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.978+0000 7fb5055a7700 1 --2- 172.21.15.143:0/2072945897 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb500072130 0x7fb5001dbe00 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb4f0014040 tx=0x7fb4f0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:01.045 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.978+0000 7f0d96ffd700 1 -- 172.21.15.143:0/4282759718 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0d9c007470 con 0x7f0da015f4f0 2023-11-08T19:25:01.045 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.978+0000 7f0d96ffd700 1 --2- 172.21.15.143:0/4282759718 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f0d8406af00 0x7f0d8406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:01.045 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.978+0000 7f0d96ffd700 1 -- 172.21.15.143:0/4282759718 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0d9c088870 con 0x7f0da015f4f0 2023-11-08T19:25:01.046 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.978+0000 7fcfd9ffb700 1 -- 172.21.15.143:0/1174702490 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfdc00be70 con 0x7fcfe4162060 2023-11-08T19:25:01.046 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.978+0000 7fcfd9ffb700 1 -- 172.21.15.143:0/1174702490 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfdc004460 con 0x7fcfe4162060 2023-11-08T19:25:01.046 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.978+0000 7f3a43fff700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f3a440b1810 0x7f3a440ca100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:45020/0 (socket says 172.21.15.143:45020) 2023-11-08T19:25:01.047 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.978+0000 7f3a43fff700 1 -- 172.21.15.143:0/2297274895 learned_addr learned my addr 172.21.15.143:0/2297274895 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:25:01.047 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.977+0000 7f3e1e7fc700 1 -- 172.21.15.143:0/876775502 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e14004390 con 0x7f3e20159e00 2023-11-08T19:25:01.048 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.977+0000 7f3e1e7fc700 1 -- 172.21.15.143:0/876775502 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e140036a0 con 0x7f3e20159e00 2023-11-08T19:25:01.048 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.978+0000 7f3e25fce700 1 -- 172.21.15.143:0/876775502 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3e20159e00 msgr2=0x7f3e200b1100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:01.048 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.978+0000 7f3e25fce700 1 --2- 172.21.15.143:0/876775502 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3e20159e00 0x7f3e200b1100 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3e200cf5e0 tx=0x7f3e14009bd0 comp rx=0 tx=0).stop 2023-11-08T19:25:01.048 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.978+0000 7f3e25fce700 1 -- 172.21.15.143:0/876775502 shutdown_connections 2023-11-08T19:25:01.048 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.978+0000 7f3e25fce700 1 --2- 172.21.15.143:0/876775502 >> v2:172.21.15.143:3300/0 conn(0x7f3e20159a20 0x7f3e2006ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:01.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.978+0000 7f3e25fce700 1 --2- 172.21.15.143:0/876775502 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3e20159e00 0x7f3e200b1100 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:01.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.978+0000 7f3e25fce700 1 -- 172.21.15.143:0/876775502 >> 172.21.15.143:0/876775502 conn(0x7f3e2006b060 msgr2=0x7f3e2006b470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:01.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.977+0000 7f2a725c5700 1 -- 172.21.15.143:0/1790216944 wait complete. 2023-11-08T19:25:01.050 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.978+0000 7f2a725c5700 1 Processor -- start 2023-11-08T19:25:01.050 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.978+0000 7f0da6236700 1 --2- 172.21.15.143:0/4282759718 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f0d8406af00 0x7f0d8406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:01.051 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.978+0000 7fcfeb995700 1 -- 172.21.15.143:0/1174702490 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfe4147fa0 con 0x7fcfe4162060 2023-11-08T19:25:01.051 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.979+0000 7fcfeb995700 1 -- 172.21.15.143:0/1174702490 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfe4160ad0 con 0x7fcfe4162060 2023-11-08T19:25:01.052 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.978+0000 7f7dbeffd700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7dc015dc80 0x7f7dc00d1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:01.052 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.978+0000 7f7dbeffd700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7dc015dc80 0x7f7dc00d1f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.143:36508/0 (socket says 172.21.15.143:36508) 2023-11-08T19:25:01.052 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.978+0000 7f7dbeffd700 1 -- 172.21.15.143:0/2980713306 learned_addr learned my addr 172.21.15.143:0/2980713306 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:01.052 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.979+0000 7f7dbf7fe700 1 --2- 172.21.15.143:0/2980713306 >> v2:172.21.15.143:3300/0 conn(0x7f7dc00cf090 0x7f7dc00b9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:01.052 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.979+0000 7f7dbeffd700 1 -- 172.21.15.143:0/2980713306 >> v2:172.21.15.143:3300/0 conn(0x7f7dc00cf090 msgr2=0x7f7dc00b9690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:01.053 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.979+0000 7f7dbeffd700 1 --2- 172.21.15.143:0/2980713306 >> v2:172.21.15.143:3300/0 conn(0x7f7dc00cf090 0x7f7dc00b9690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:01.053 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.979+0000 7f7dbeffd700 1 -- 172.21.15.143:0/2980713306 >> v1:172.21.15.174:6789/0 conn(0x7f7dc00d3690 legacy=0x7f7dc00ce980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:01.053 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.979+0000 7f7dbeffd700 1 -- 172.21.15.143:0/2980713306 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dc015e060 con 0x7f7dc015dc80 2023-11-08T19:25:01.053 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.979+0000 7f7dbeffd700 1 --2- 172.21.15.143:0/2980713306 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7dc015dc80 0x7f7dc00d1f80 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f7db40098f0 tx=0x7f7db4009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3cbd41586725eeaf server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:01.054 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.979+0000 7fb4f67fc700 1 -- 172.21.15.143:0/2072945897 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f00036a0 con 0x7fb500072130 2023-11-08T19:25:01.054 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.979+0000 7fb4f67fc700 1 -- 172.21.15.143:0/2072945897 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4f001f070 con 0x7fb500072130 2023-11-08T19:25:01.054 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.979+0000 7fb4f67fc700 1 -- 172.21.15.143:0/2072945897 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f0003a00 con 0x7fb500072130 2023-11-08T19:25:01.054 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.979+0000 7fb50800c700 1 -- 172.21.15.143:0/2072945897 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5000c4960 con 0x7fb500072130 2023-11-08T19:25:01.054 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.979+0000 7fb50800c700 1 -- 172.21.15.143:0/2072945897 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5001ec390 con 0x7fb500072130 2023-11-08T19:25:01.055 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.979+0000 7f3a437fe700 1 --2- 172.21.15.143:0/2297274895 >> v2:172.21.15.174:3300/0 conn(0x7f3a44159a20 0x7f3a4406ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:01.055 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.979+0000 7f3a437fe700 1 -- 172.21.15.143:0/2297274895 >> v1:172.21.15.174:6789/0 conn(0x7f3a44159e00 legacy=0x7f3a440b1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:01.055 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.979+0000 7f3a437fe700 1 -- 172.21.15.143:0/2297274895 >> v1:172.21.15.143:6789/0 conn(0x7f3a440b1810 legacy=0x7f3a440ca100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:01.056 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.979+0000 7f3a437fe700 1 -- 172.21.15.143:0/2297274895 --> v2:172.21.15.174:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a440cad70 con 0x7f3a44159a20 2023-11-08T19:25:01.056 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.979+0000 7f3a437fe700 1 --2- 172.21.15.143:0/2297274895 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a44159a20 0x7f3a4406ce00 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3a38009840 tx=0x7f3a38009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c740a9d72f9d7a05 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:01.056 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.980+0000 7f3a427fc700 1 -- 172.21.15.143:0/2297274895 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a38004230 con 0x7f3a44159a20 2023-11-08T19:25:01.056 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.980+0000 7f3a427fc700 1 -- 172.21.15.143:0/2297274895 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a38004390 con 0x7f3a44159a20 2023-11-08T19:25:01.056 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.980+0000 7f3a427fc700 1 -- 172.21.15.143:0/2297274895 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a380036a0 con 0x7f3a44159a20 2023-11-08T19:25:01.057 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.980+0000 7fcfeb995700 1 -- 172.21.15.143:0/1174702490 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fcfe4147b50 con 0x7fcfe4162060 2023-11-08T19:25:01.057 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.980+0000 7f3e25fce700 1 -- 172.21.15.143:0/876775502 shutdown_connections 2023-11-08T19:25:01.058 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.980+0000 7fb50800c700 1 -- 172.21.15.143:0/2072945897 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb5000ce270 con 0x7fb500072130 2023-11-08T19:25:01.058 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.980+0000 7f7dbdffb700 1 -- 172.21.15.143:0/2980713306 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7db4004230 con 0x7f7dc015dc80 2023-11-08T19:25:01.058 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.981+0000 7f3a49fce700 1 -- 172.21.15.143:0/2297274895 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a44159a20 msgr2=0x7f3a4406ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:01.059 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.981+0000 7f3a49fce700 1 --2- 172.21.15.143:0/2297274895 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a44159a20 0x7f3a4406ce00 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3a38009840 tx=0x7f3a38009b50 comp rx=0 tx=0).stop 2023-11-08T19:25:01.059 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.981+0000 7f0da6236700 1 --2- 172.21.15.143:0/4282759718 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f0d8406af00 0x7f0d8406d3c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0d90000c00 tx=0x7f0d90009070 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:01.060 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.979+0000 7f2a725c5700 1 -- start start 2023-11-08T19:25:01.060 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.980+0000 7f2a725c5700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2a6c0b1810 0x7f2a6c203690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:01.060 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.980+0000 7f2a725c5700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a6c0c2a50 0x7f2a6c0b08e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:01.060 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.980+0000 7f2a725c5700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2a6c0cd820 0x7f2a6c0b0e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:01.060 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.980+0000 7f2a725c5700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a6c0c66c0 con 0x7f2a6c0cd820 2023-11-08T19:25:01.061 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.981+0000 7f0da849a700 1 --2- 172.21.15.143:0/4282759718 >> [v2:172.21.15.174:6824/2755130316,v1:172.21.15.174:6825/2755130316] conn(0x7f0da00b21a0 0x7f0da00b2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:01.061 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.981+0000 7f0da849a700 1 -- 172.21.15.143:0/4282759718 --> [v2:172.21.15.174:6824/2755130316,v1:172.21.15.174:6825/2755130316] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0da00cb050 con 0x7f0da00b21a0 2023-11-08T19:25:01.062 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.980+0000 7f2a725c5700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a6c201570 con 0x7f2a6c0c2a50 2023-11-08T19:25:01.062 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.980+0000 7f2a725c5700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a6c0c6960 con 0x7f2a6c0b1810 2023-11-08T19:25:01.063 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.980+0000 7f2a70b62700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2a6c0cd820 0x7f2a6c0b0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:01.063 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.981+0000 7f2a70b62700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2a6c0cd820 0x7f2a6c0b0e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48882/0 (socket says 172.21.15.143:48882) 2023-11-08T19:25:01.063 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.981+0000 7f2a70b62700 1 -- 172.21.15.143:0/1080853847 learned_addr learned my addr 172.21.15.143:0/1080853847 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:01.063 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.981+0000 7f2a70b62700 1 -- 172.21.15.143:0/1080853847 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2a6c0b1810 msgr2=0x7f2a6c203690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:01.064 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.981+0000 7f2a70b62700 1 --2- 172.21.15.143:0/1080853847 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2a6c0b1810 0x7f2a6c203690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:01.064 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.981+0000 7f2a70b62700 1 -- 172.21.15.143:0/1080853847 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a6c0c2a50 msgr2=0x7f2a6c0b08e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-08T19:25:01.064 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.981+0000 7f2a70b62700 1 --2- 172.21.15.143:0/1080853847 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a6c0c2a50 0x7f2a6c0b08e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:01.065 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.981+0000 7f2a70b62700 1 -- 172.21.15.143:0/1080853847 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a5c009550 con 0x7f2a6c0cd820 2023-11-08T19:25:01.065 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.981+0000 7f0da6a37700 1 --2- 172.21.15.143:0/4282759718 >> [v2:172.21.15.174:6824/2755130316,v1:172.21.15.174:6825/2755130316] conn(0x7f0da00b21a0 0x7f0da00b2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:01.066 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.980+0000 7f3e25fce700 1 -- 172.21.15.143:0/876775502 wait complete. 2023-11-08T19:25:01.066 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.980+0000 7f7dbdffb700 1 -- 172.21.15.143:0/2980713306 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7db4004390 con 0x7f7dc015dc80 2023-11-08T19:25:01.066 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.980+0000 7f7dbdffb700 1 -- 172.21.15.143:0/2980713306 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7db40036a0 con 0x7f7dc015dc80 2023-11-08T19:25:01.066 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.981+0000 7f7dc54a9700 1 -- 172.21.15.143:0/2980713306 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7dc015dc80 msgr2=0x7f7dc00d1f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:01.067 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.981+0000 7f7dc54a9700 1 --2- 172.21.15.143:0/2980713306 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7dc015dc80 0x7f7dc00d1f80 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f7db40098f0 tx=0x7f7db4009c00 comp rx=0 tx=0).stop 2023-11-08T19:25:01.067 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.981+0000 7f3a49fce700 1 -- 172.21.15.143:0/2297274895 shutdown_connections 2023-11-08T19:25:01.067 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.981+0000 7f3a49fce700 1 --2- 172.21.15.143:0/2297274895 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a44159a20 0x7f3a4406ce00 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:01.068 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.981+0000 7f3a49fce700 1 -- 172.21.15.143:0/2297274895 >> 172.21.15.143:0/2297274895 conn(0x7f3a4406b060 msgr2=0x7f3a4406b470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:01.068 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.982+0000 7f2a70b62700 1 --2- 172.21.15.143:0/1080853847 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2a6c0cd820 0x7f2a6c0b0e20 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c0711f0 tx=0x7f2a5c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:01.068 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.982+0000 7f2a68ff9700 1 -- 172.21.15.143:0/1080853847 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a5c0045d0 con 0x7f2a6c0cd820 2023-11-08T19:25:01.068 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.982+0000 7f2a68ff9700 1 -- 172.21.15.143:0/1080853847 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a5c011630 con 0x7f2a6c0cd820 2023-11-08T19:25:01.069 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.982+0000 7f2a725c5700 1 -- 172.21.15.143:0/1080853847 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a6c0c6c40 con 0x7f2a6c0cd820 2023-11-08T19:25:01.069 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.982+0000 7f3e25fce700 1 Processor -- start 2023-11-08T19:25:01.069 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.982+0000 7f0da6a37700 1 --2- 172.21.15.143:0/4282759718 >> [v2:172.21.15.174:6824/2755130316,v1:172.21.15.174:6825/2755130316] conn(0x7f0da00b21a0 0x7f0da00b2580 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:01.070 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.982+0000 7f3a49fce700 1 -- 172.21.15.143:0/2297274895 shutdown_connections 2023-11-08T19:25:01.070 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.983+0000 7fcfd9ffb700 1 -- 172.21.15.143:0/1174702490 <== mon.2 v2:172.21.15.174:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fcfdc0045c0 con 0x7fcfe4162060 2023-11-08T19:25:01.071 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.983+0000 7fb4f67fc700 1 -- 172.21.15.143:0/2072945897 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb4f0003b60 con 0x7fb500072130 2023-11-08T19:25:01.071 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.983+0000 7fcfd9ffb700 1 --2- 172.21.15.143:0/1174702490 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fcfc806af00 0x7fcfc806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:01.071 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.983+0000 7fb4f67fc700 1 --2- 172.21.15.143:0/2072945897 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fb4e406af00 0x7fb4e406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:01.072 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.983+0000 7fcfd9ffb700 1 -- 172.21.15.143:0/1174702490 <== mon.2 v2:172.21.15.174:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcfdc08aef0 con 0x7fcfe4162060 2023-11-08T19:25:01.072 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.983+0000 7fcfd9ffb700 1 --2- 172.21.15.143:0/1174702490 >> [v2:172.21.15.143:6816/2021504428,v1:172.21.15.143:6817/2021504428] conn(0x7fcfc8072f30 0x7fcfc8075350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:01.073 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.983+0000 7fcfd9ffb700 1 -- 172.21.15.143:0/1174702490 --> [v2:172.21.15.143:6816/2021504428,v1:172.21.15.143:6817/2021504428] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcfc8075a00 con 0x7fcfc8072f30 2023-11-08T19:25:01.073 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.983+0000 7fcfd9ffb700 1 -- 172.21.15.143:0/1174702490 <== mon.2 v2:172.21.15.174:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcfdc08b260 con 0x7fcfe4162060 2023-11-08T19:25:01.073 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.983+0000 7f7dc54a9700 1 -- 172.21.15.143:0/2980713306 shutdown_connections 2023-11-08T19:25:01.073 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.983+0000 7f7dc54a9700 1 --2- 172.21.15.143:0/2980713306 >> v2:172.21.15.143:3300/0 conn(0x7f7dc00cf090 0x7f7dc00b9690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:01.074 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.983+0000 7f7dc54a9700 1 --2- 172.21.15.143:0/2980713306 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7dc015dc80 0x7f7dc00d1f80 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:01.074 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.983+0000 7f7dc54a9700 1 -- 172.21.15.143:0/2980713306 >> 172.21.15.143:0/2980713306 conn(0x7f7dc00668e0 msgr2=0x7f7dc0066cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:01.074 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.983+0000 7fb4f67fc700 1 -- 172.21.15.143:0/2072945897 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb4f0021070 con 0x7fb500072130 2023-11-08T19:25:01.074 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.983+0000 7fb4f67fc700 1 --2- 172.21.15.143:0/2072945897 >> [v2:172.21.15.174:6808/1777923303,v1:172.21.15.174:6809/1777923303] conn(0x7fb4e40730a0 0x7fb4e40754c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:01.075 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.983+0000 7f0d96ffd700 1 -- 172.21.15.143:0/4282759718 <== osd.6 v2:172.21.15.174:6824/2755130316 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26745 (crc 0 0 0) 0x7f0d980078a0 con 0x7f0da00b21a0 2023-11-08T19:25:01.075 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.982+0000 7f2a68ff9700 1 -- 172.21.15.143:0/1080853847 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a5c0119a0 con 0x7f2a6c0cd820 2023-11-08T19:25:01.076 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.982+0000 7f2a725c5700 1 -- 172.21.15.143:0/1080853847 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a6c2018b0 con 0x7f2a6c0cd820 2023-11-08T19:25:01.076 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.984+0000 7f7dc54a9700 1 -- 172.21.15.143:0/2980713306 shutdown_connections 2023-11-08T19:25:01.077 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.984+0000 7f7dc54a9700 1 -- 172.21.15.143:0/2980713306 wait complete. 2023-11-08T19:25:01.077 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.983+0000 7fb4f67fc700 1 -- 172.21.15.143:0/2072945897 --> [v2:172.21.15.174:6808/1777923303,v1:172.21.15.174:6809/1777923303] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb4e4075b70 con 0x7fb4e40730a0 2023-11-08T19:25:01.078 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.983+0000 7fb4f67fc700 1 -- 172.21.15.143:0/2072945897 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb4f008b0c0 con 0x7fb500072130 2023-11-08T19:25:01.078 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.984+0000 7fb5065a9700 1 --2- 172.21.15.143:0/2072945897 >> [v2:172.21.15.174:6808/1777923303,v1:172.21.15.174:6809/1777923303] conn(0x7fb4e40730a0 0x7fb4e40754c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:01.079 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.982+0000 7f3e25fce700 1 -- start start 2023-11-08T19:25:01.079 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.983+0000 7f3e25fce700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e200cc100 0x7f3e200c3100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:01.080 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.983+0000 7f3e25fce700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3e20159a20 0x7f3e200c5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:01.080 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.983+0000 7f3e25fce700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3e20159e00 0x7f3e2014a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:01.080 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.983+0000 7f3e25fce700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e200c5fe0 con 0x7f3e200cc100 2023-11-08T19:25:01.081 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.983+0000 7f3e25fce700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e2014a5b0 con 0x7f3e20159e00 2023-11-08T19:25:01.081 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.983+0000 7f3e25fce700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e20149340 con 0x7f3e20159a20 2023-11-08T19:25:01.081 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.984+0000 7f3e1f7fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e200cc100 0x7f3e200c3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:01.082 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.984+0000 7f3e1f7fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e200cc100 0x7f3e200c3100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48888/0 (socket says 172.21.15.143:48888) 2023-11-08T19:25:01.082 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.984+0000 7f3e1f7fe700 1 -- 172.21.15.143:0/4135659825 learned_addr learned my addr 172.21.15.143:0/4135659825 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:01.082 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.984+0000 7f3e1ffff700 1 --2- 172.21.15.143:0/4135659825 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3e20159e00 0x7f3e2014a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:01.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.987+0000 7f2a68ff9700 1 -- 172.21.15.143:0/1080853847 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2a5c022070 con 0x7f2a6c0cd820 2023-11-08T19:25:01.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.987+0000 7f2a68ff9700 1 --2- 172.21.15.143:0/1080853847 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f2a5806af00 0x7f2a5806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:01.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.987+0000 7f2a68ff9700 1 -- 172.21.15.143:0/1080853847 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2a5c08b090 con 0x7f2a6c0cd820 2023-11-08T19:25:01.085 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.984+0000 7f3e1f7fe700 1 -- 172.21.15.143:0/4135659825 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3e20159a20 msgr2=0x7f3e200c5aa0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-08T19:25:01.085 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.984+0000 7f3e1f7fe700 1 --2- 172.21.15.143:0/4135659825 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3e20159a20 0x7f3e200c5aa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:01.085 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.984+0000 7f3e1f7fe700 1 -- 172.21.15.143:0/4135659825 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3e20159e00 msgr2=0x7f3e2014a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:01.086 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.984+0000 7f3e1f7fe700 1 --2- 172.21.15.143:0/4135659825 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3e20159e00 0x7f3e2014a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:01.088 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.984+0000 7f3e1f7fe700 1 -- 172.21.15.143:0/4135659825 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e14009550 con 0x7f3e200cc100 2023-11-08T19:25:01.088 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.984+0000 7f3e1ffff700 1 --2- 172.21.15.143:0/4135659825 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3e20159e00 0x7f3e2014a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:25:01.089 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.986+0000 7f3e1f7fe700 1 --2- 172.21.15.143:0/4135659825 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e200cc100 0x7f3e200c3100 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f3e1000e840 tx=0x7f3e1000ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:01.089 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.985+0000 7fcfe8f30700 1 --2- 172.21.15.143:0/1174702490 >> [v2:172.21.15.143:6816/2021504428,v1:172.21.15.143:6817/2021504428] conn(0x7fcfc8072f30 0x7fcfc8075350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:01.090 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.985+0000 7fcfe9731700 1 --2- 172.21.15.143:0/1174702490 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fcfc806af00 0x7fcfc806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:01.090 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.985+0000 7fcfe8f30700 1 --2- 172.21.15.143:0/1174702490 >> [v2:172.21.15.143:6816/2021504428,v1:172.21.15.143:6817/2021504428] conn(0x7fcfc8072f30 0x7fcfc8075350 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:01.091 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.985+0000 7fcfe9731700 1 --2- 172.21.15.143:0/1174702490 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fcfc806af00 0x7fcfc806d3c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fcfe0007fd0 tx=0x7fcfe0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:01.091 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.986+0000 7f7dc54a9700 1 Processor -- start 2023-11-08T19:25:01.092 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.987+0000 7f7dc54a9700 1 -- start start 2023-11-08T19:25:01.092 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.984+0000 7fb5065a9700 1 --2- 172.21.15.143:0/2072945897 >> [v2:172.21.15.174:6808/1777923303,v1:172.21.15.174:6809/1777923303] conn(0x7fb4e40730a0 0x7fb4e40754c0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:01.093 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.986+0000 7fb4f67fc700 1 -- 172.21.15.143:0/2072945897 <== osd.5 v2:172.21.15.174:6808/1777923303 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26745 (crc 0 0 0) 0x7fb4e4075b70 con 0x7fb4e40730a0 2023-11-08T19:25:01.094 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.983+0000 7f3a49fce700 1 -- 172.21.15.143:0/2297274895 wait complete. 2023-11-08T19:25:01.095 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.984+0000 7f3a49fce700 1 Processor -- start 2023-11-08T19:25:01.095 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.985+0000 7f3a49fce700 1 -- start start 2023-11-08T19:25:01.095 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.986+0000 7f3a49fce700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a440b1810 0x7f3a441fb760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:01.096 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.986+0000 7f3a49fce700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3a44159a20 0x7f3a440af3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:01.096 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.986+0000 7f3a49fce700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3a44159e00 0x7f3a441557e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:01.096 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.986+0000 7f3a49fce700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a441fbff0 con 0x7f3a44159e00 2023-11-08T19:25:01.096 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.986+0000 7f3a49fce700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a441fc2d0 con 0x7f3a440b1810 2023-11-08T19:25:01.096 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.986+0000 7f3a49fce700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a441fc5b0 con 0x7f3a44159a20 2023-11-08T19:25:01.097 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.986+0000 7f3a42ffd700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3a44159a20 0x7f3a440af3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:01.097 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.986+0000 7f3a42ffd700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3a44159a20 0x7f3a440af3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.143:36538/0 (socket says 172.21.15.143:36538) 2023-11-08T19:25:01.097 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.986+0000 7f3a42ffd700 1 -- 172.21.15.143:0/2834563838 learned_addr learned my addr 172.21.15.143:0/2834563838 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:01.098 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.986+0000 7f3a437fe700 1 --2- 172.21.15.143:0/2834563838 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a440b1810 0x7f3a441fb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:01.098 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.986+0000 7f3a43fff700 1 --2- 172.21.15.143:0/2834563838 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3a44159e00 0x7f3a441557e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:01.098 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.987+0000 7f3a43fff700 1 -- 172.21.15.143:0/2834563838 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3a44159a20 msgr2=0x7f3a440af3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:01.098 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.987+0000 7f3a43fff700 1 --2- 172.21.15.143:0/2834563838 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3a44159a20 0x7f3a440af3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:01.099 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.987+0000 7f3a43fff700 1 -- 172.21.15.143:0/2834563838 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a440b1810 msgr2=0x7f3a441fb760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:01.099 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.987+0000 7f3a43fff700 1 --2- 172.21.15.143:0/2834563838 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a440b1810 0x7f3a441fb760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:01.099 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.987+0000 7f3a43fff700 1 -- 172.21.15.143:0/2834563838 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a380094a0 con 0x7f3a44159e00 2023-11-08T19:25:01.099 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.988+0000 7f2a6bfff700 1 --2- 172.21.15.143:0/1080853847 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f2a5806af00 0x7f2a5806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:01.100 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.988+0000 7f2a6bfff700 1 --2- 172.21.15.143:0/1080853847 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f2a5806af00 0x7f2a5806d3c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c0c58a0 tx=0x7f2a6400b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:01.100 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.992+0000 7f2a725c5700 1 --2- 172.21.15.143:0/1080853847 >> [v2:172.21.15.143:6800/787341397,v1:172.21.15.143:6801/787341397] conn(0x7f2a6c200840 0x7f2a6c200c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:01.100 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.992+0000 7f2a725c5700 1 -- 172.21.15.143:0/1080853847 --> [v2:172.21.15.143:6800/787341397,v1:172.21.15.143:6801/787341397] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2a6c071670 con 0x7f2a6c200840 2023-11-08T19:25:01.100 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.992+0000 7f2a6b7fe700 1 --2- 172.21.15.143:0/1080853847 >> [v2:172.21.15.143:6800/787341397,v1:172.21.15.143:6801/787341397] conn(0x7f2a6c200840 0x7f2a6c200c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:01.100 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.992+0000 7f2a6b7fe700 1 --2- 172.21.15.143:0/1080853847 >> [v2:172.21.15.143:6800/787341397,v1:172.21.15.143:6801/787341397] conn(0x7f2a6c200840 0x7f2a6c200c20 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:01.101 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.993+0000 7f2a68ff9700 1 -- 172.21.15.143:0/1080853847 <== osd.1 v2:172.21.15.143:6800/787341397 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26745 (crc 0 0 0) 0x7f2a60009280 con 0x7f2a6c200840 2023-11-08T19:25:01.101 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.994+0000 7f3e07fff700 1 -- 172.21.15.143:0/4135659825 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e10013610 con 0x7f3e200cc100 2023-11-08T19:25:01.101 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.994+0000 7f3e07fff700 1 -- 172.21.15.143:0/4135659825 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e10013770 con 0x7f3e200cc100 2023-11-08T19:25:01.101 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.994+0000 7f3e07fff700 1 -- 172.21.15.143:0/4135659825 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e10013b20 con 0x7f3e200cc100 2023-11-08T19:25:01.102 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.988+0000 7f7dc54a9700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7dc00cf090 0x7f7dc00c8730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:01.102 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.988+0000 7f7dc54a9700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7dc00d3690 0x7f7dc0200ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:01.102 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.988+0000 7f7dc54a9700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7dc015dc80 0x7f7dc0150850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:01.103 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.988+0000 7f7dc54a9700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dc01f7710 con 0x7f7dc00cf090 2023-11-08T19:25:01.103 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.988+0000 7f7dc54a9700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dc0201cc0 con 0x7f7dc00d3690 2023-11-08T19:25:01.103 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.988+0000 7f7dc54a9700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7dc00c9080 con 0x7f7dc015dc80 2023-11-08T19:25:01.103 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.988+0000 7f7dbeffd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7dc00cf090 0x7f7dc00c8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:01.103 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.988+0000 7f7dbe7fc700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7dc00d3690 0x7f7dc0200ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:01.104 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.988+0000 7f7dbeffd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7dc00cf090 0x7f7dc00c8730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48908/0 (socket says 172.21.15.143:48908) 2023-11-08T19:25:01.104 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.988+0000 7f7dbeffd700 1 -- 172.21.15.143:0/896618954 learned_addr learned my addr 172.21.15.143:0/896618954 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:01.104 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.988+0000 7f7dbe7fc700 1 -- 172.21.15.143:0/896618954 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7dc015dc80 msgr2=0x7f7dc0150850 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-08T19:25:01.104 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.988+0000 7f7dbe7fc700 1 --2- 172.21.15.143:0/896618954 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7dc015dc80 0x7f7dc0150850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:01.104 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.988+0000 7f7dbe7fc700 1 -- 172.21.15.143:0/896618954 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7dc00cf090 msgr2=0x7f7dc00c8730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:01.104 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.988+0000 7f7dbe7fc700 1 --2- 172.21.15.143:0/896618954 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7dc00cf090 0x7f7dc00c8730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:01.105 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.988+0000 7f7dbe7fc700 1 -- 172.21.15.143:0/896618954 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7db4009550 con 0x7f7dc00d3690 2023-11-08T19:25:01.105 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.989+0000 7f7dbe7fc700 1 --2- 172.21.15.143:0/896618954 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7dc00d3690 0x7f7dc0200ed0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7dc00d3a90 tx=0x7f7db000c6e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:01.105 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.989+0000 7f7da77fe700 1 -- 172.21.15.143:0/896618954 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7db0011790 con 0x7f7dc00d3690 2023-11-08T19:25:01.105 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.989+0000 7f7da77fe700 1 -- 172.21.15.143:0/896618954 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7db000ce90 con 0x7f7dc00d3690 2023-11-08T19:25:01.106 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.989+0000 7f7da77fe700 1 -- 172.21.15.143:0/896618954 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7db0011b30 con 0x7f7dc00d3690 2023-11-08T19:25:01.106 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.989+0000 7f7dc54a9700 1 -- 172.21.15.143:0/896618954 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7dc00c9360 con 0x7f7dc00d3690 2023-11-08T19:25:01.106 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.989+0000 7f7dc54a9700 1 -- 172.21.15.143:0/896618954 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dc0201950 con 0x7f7dc00d3690 2023-11-08T19:25:01.106 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.991+0000 7f7da77fe700 1 -- 172.21.15.143:0/896618954 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7db0011c90 con 0x7f7dc00d3690 2023-11-08T19:25:01.106 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.991+0000 7f7da77fe700 1 --2- 172.21.15.143:0/896618954 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f7dac06af00 0x7f7dac06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:01.107 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.992+0000 7f7da77fe700 1 -- 172.21.15.143:0/896618954 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7db0089280 con 0x7f7dc00d3690 2023-11-08T19:25:01.107 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.992+0000 7f7dbeffd700 1 --2- 172.21.15.143:0/896618954 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f7dac06af00 0x7f7dac06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:01.107 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.993+0000 7fb505da8700 1 --2- 172.21.15.143:0/2072945897 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fb4e406af00 0x7fb4e406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:01.108 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.988+0000 7f3a43fff700 1 --2- 172.21.15.143:0/2834563838 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3a44159e00 0x7f3a441557e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c00d750 tx=0x7f3a3c00da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:01.108 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.988+0000 7f3a2bfff700 1 -- 172.21.15.143:0/2834563838 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a3c0055e0 con 0x7f3a44159e00 2023-11-08T19:25:01.108 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.988+0000 7f3a2bfff700 1 -- 172.21.15.143:0/2834563838 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a3c005760 con 0x7f3a44159e00 2023-11-08T19:25:01.108 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.988+0000 7f3a49fce700 1 -- 172.21.15.143:0/2834563838 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a4415b950 con 0x7f3a44159e00 2023-11-08T19:25:01.108 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.988+0000 7f3a49fce700 1 -- 172.21.15.143:0/2834563838 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a440c66c0 con 0x7f3a44159e00 2023-11-08T19:25:01.109 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.989+0000 7f3a2bfff700 1 -- 172.21.15.143:0/2834563838 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a3c005b10 con 0x7f3a44159e00 2023-11-08T19:25:01.109 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.991+0000 7f3a2bfff700 1 -- 172.21.15.143:0/2834563838 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3a3c005c70 con 0x7f3a44159e00 2023-11-08T19:25:01.109 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.992+0000 7f3a2bfff700 1 --2- 172.21.15.143:0/2834563838 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f3a2c06b070 0x7f3a2c06d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:01.109 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.992+0000 7f3a2bfff700 1 -- 172.21.15.143:0/2834563838 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3a3c088ad0 con 0x7f3a44159e00 2023-11-08T19:25:01.110 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.996+0000 7fb505da8700 1 --2- 172.21.15.143:0/2072945897 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fb4e406af00 0x7fb4e406d3c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb5000c9240 tx=0x7fb4fc0079c0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:01.110 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.986+0000 7fcfd9ffb700 1 -- 172.21.15.143:0/1174702490 <== osd.3 v2:172.21.15.143:6816/2021504428 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26745 (crc 0 0 0) 0x7fcfc8075a00 con 0x7fcfc8072f30 2023-11-08T19:25:01.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.997+0000 7f3a437fe700 1 --2- 172.21.15.143:0/2834563838 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f3a2c06b070 0x7f3a2c06d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:01.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.997+0000 7f7dbeffd700 1 --2- 172.21.15.143:0/896618954 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f7dac06af00 0x7f7dac06d3c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7db4009c40 tx=0x7f7db4008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:01.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.997+0000 7f3a437fe700 1 --2- 172.21.15.143:0/2834563838 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f3a2c06b070 0x7f3a2c06d530 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3a38009b90 tx=0x7f3a38008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:01.112 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.998+0000 7f3e25fce700 1 -- 172.21.15.143:0/4135659825 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e200c3790 con 0x7f3e200cc100 2023-11-08T19:25:01.112 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.998+0000 7f3e25fce700 1 -- 172.21.15.143:0/4135659825 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e20149870 con 0x7f3e200cc100 2023-11-08T19:25:01.113 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:00.998+0000 7f3e25fce700 1 -- 172.21.15.143:0/4135659825 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3e20159640 con 0x7f3e200cc100 2023-11-08T19:25:01.113 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.000+0000 7f7dc54a9700 1 --2- 172.21.15.143:0/896618954 >> [v2:172.21.15.174:6800/1993380595,v1:172.21.15.174:6801/1993380595] conn(0x7f7dc0150d90 0x7f7dc00b8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:01.114 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.000+0000 7f7dc54a9700 1 -- 172.21.15.143:0/896618954 --> [v2:172.21.15.174:6800/1993380595,v1:172.21.15.174:6801/1993380595] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7dc00d25f0 con 0x7f7dc0150d90 2023-11-08T19:25:01.115 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.000+0000 7f3a49fce700 1 --2- 172.21.15.143:0/2834563838 >> [v2:172.21.15.174:6816/2847141411,v1:172.21.15.174:6817/2847141411] conn(0x7f3a240015e0 0x7f3a24003aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:01.115 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.000+0000 7f3a49fce700 1 -- 172.21.15.143:0/2834563838 --> [v2:172.21.15.174:6816/2847141411,v1:172.21.15.174:6817/2847141411] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3a24006bd0 con 0x7f3a240015e0 2023-11-08T19:25:01.115 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.000+0000 7f3a42ffd700 1 --2- 172.21.15.143:0/2834563838 >> [v2:172.21.15.174:6816/2847141411,v1:172.21.15.174:6817/2847141411] conn(0x7f3a240015e0 0x7f3a24003aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:01.116 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.000+0000 7f3a42ffd700 1 --2- 172.21.15.143:0/2834563838 >> [v2:172.21.15.174:6816/2847141411,v1:172.21.15.174:6817/2847141411] conn(0x7f3a240015e0 0x7f3a24003aa0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:01.116 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.000+0000 7f3e07fff700 1 -- 172.21.15.143:0/4135659825 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3e10013c80 con 0x7f3e200cc100 2023-11-08T19:25:01.117 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.000+0000 7f3e07fff700 1 --2- 172.21.15.143:0/4135659825 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f3e0c06af50 0x7f3e0c06d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:01.117 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.000+0000 7f3e07fff700 1 -- 172.21.15.143:0/4135659825 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3e10089570 con 0x7f3e200cc100 2023-11-08T19:25:01.117 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.000+0000 7f3e1effd700 1 --2- 172.21.15.143:0/4135659825 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f3e0c06af50 0x7f3e0c06d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:01.118 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.000+0000 7f3e07fff700 1 --2- 172.21.15.143:0/4135659825 >> [v2:172.21.15.143:6820/2301325597,v1:172.21.15.143:6822/2301325597] conn(0x7f3e0c073170 0x7f3e0c075590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:01.118 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.001+0000 7f3e07fff700 1 -- 172.21.15.143:0/4135659825 --> [v2:172.21.15.143:6820/2301325597,v1:172.21.15.143:6822/2301325597] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3e0c075c40 con 0x7f3e0c073170 2023-11-08T19:25:01.118 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.001+0000 7f3e07fff700 1 -- 172.21.15.143:0/4135659825 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3e10089a10 con 0x7f3e200cc100 2023-11-08T19:25:01.118 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.001+0000 7f3e1ffff700 1 --2- 172.21.15.143:0/4135659825 >> [v2:172.21.15.143:6820/2301325597,v1:172.21.15.143:6822/2301325597] conn(0x7f3e0c073170 0x7f3e0c075590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:01.118 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.001+0000 7f3e1effd700 1 --2- 172.21.15.143:0/4135659825 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f3e0c06af50 0x7f3e0c06d410 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3e2014e220 tx=0x7f3e14008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:01.119 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.001+0000 7f3e1ffff700 1 --2- 172.21.15.143:0/4135659825 >> [v2:172.21.15.143:6820/2301325597,v1:172.21.15.143:6822/2301325597] conn(0x7f3e0c073170 0x7f3e0c075590 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:01.119 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.001+0000 7f3a2bfff700 1 -- 172.21.15.143:0/2834563838 <== osd.7 v2:172.21.15.174:6816/2847141411 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26745 (crc 0 0 0) 0x7f3a24006bd0 con 0x7f3a240015e0 2023-11-08T19:25:01.120 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.002+0000 7f3e07fff700 1 -- 172.21.15.143:0/4135659825 <== osd.2 v2:172.21.15.143:6820/2301325597 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26745 (crc 0 0 0) 0x7f3e0c075c40 con 0x7f3e0c073170 2023-11-08T19:25:01.120 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.010+0000 7f0da849a700 1 -- 172.21.15.143:0/4282759718 --> [v2:172.21.15.174:6824/2755130316,v1:172.21.15.174:6825/2755130316] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0da00cc100 con 0x7f0da00b21a0 2023-11-08T19:25:01.120 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.010+0000 7f0d96ffd700 1 -- 172.21.15.143:0/4282759718 <== osd.6 v2:172.21.15.174:6824/2755130316 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f0d98007c80 con 0x7f0da00b21a0 2023-11-08T19:25:01.121 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.011+0000 7f0da849a700 1 -- 172.21.15.143:0/4282759718 >> [v2:172.21.15.174:6824/2755130316,v1:172.21.15.174:6825/2755130316] conn(0x7f0da00b21a0 msgr2=0x7f0da00b2580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:01.122 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.011+0000 7f0da849a700 1 --2- 172.21.15.143:0/4282759718 >> [v2:172.21.15.174:6824/2755130316,v1:172.21.15.174:6825/2755130316] conn(0x7f0da00b21a0 0x7f0da00b2580 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:01.122 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.011+0000 7f0da849a700 1 -- 172.21.15.143:0/4282759718 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f0d8406af00 msgr2=0x7f0d8406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:01.122 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.011+0000 7f0da849a700 1 --2- 172.21.15.143:0/4282759718 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f0d8406af00 0x7f0d8406d3c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0d90000c00 tx=0x7f0d90009070 comp rx=0 tx=0).stop 2023-11-08T19:25:01.122 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.011+0000 7f0da849a700 1 -- 172.21.15.143:0/4282759718 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0da015f4f0 msgr2=0x7f0da01562a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:01.123 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.011+0000 7f0da849a700 1 --2- 172.21.15.143:0/4282759718 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0da015f4f0 0x7f0da01562a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c00c730 tx=0x7f0d9c00ca40 comp rx=0 tx=0).stop 2023-11-08T19:25:01.123 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.011+0000 7f0da6a37700 1 -- 172.21.15.143:0/4282759718 reap_dead start 2023-11-08T19:25:01.123 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.011+0000 7f0da849a700 1 -- 172.21.15.143:0/4282759718 shutdown_connections 2023-11-08T19:25:01.123 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.011+0000 7f0da849a700 1 -- 172.21.15.143:0/4282759718 >> 172.21.15.143:0/4282759718 conn(0x7f0da006b060 msgr2=0x7f0da00cd780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:01.123 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.011+0000 7f0da849a700 1 -- 172.21.15.143:0/4282759718 shutdown_connections 2023-11-08T19:25:01.124 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.011+0000 7f0da849a700 1 -- 172.21.15.143:0/4282759718 wait complete. 2023-11-08T19:25:01.124 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.014+0000 7f0cfc5a1700 1 -- 172.21.15.143:0/3426336670 --> [v2:172.21.15.143:6808/1487830566,v1:172.21.15.143:6809/1487830566] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0cdc002d40 con 0x7f0cd80731f0 2023-11-08T19:25:01.124 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.014+0000 7f0ceaffd700 1 -- 172.21.15.143:0/3426336670 <== osd.0 v2:172.21.15.143:6808/1487830566 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f0cdc002d40 con 0x7f0cd80731f0 2023-11-08T19:25:01.125 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.019+0000 7f7dbf7fe700 1 --2- 172.21.15.143:0/896618954 >> [v2:172.21.15.174:6800/1993380595,v1:172.21.15.174:6801/1993380595] conn(0x7f7dc0150d90 0x7f7dc00b8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:01.125 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.019+0000 7f0cfc5a1700 1 -- 172.21.15.143:0/3426336670 >> [v2:172.21.15.143:6808/1487830566,v1:172.21.15.143:6809/1487830566] conn(0x7f0cd80731f0 msgr2=0x7f0cd8075610 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:01.126 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.019+0000 7f0cfc5a1700 1 --2- 172.21.15.143:0/3426336670 >> [v2:172.21.15.143:6808/1487830566,v1:172.21.15.143:6809/1487830566] conn(0x7f0cd80731f0 0x7f0cd8075610 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:01.126 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.019+0000 7fcfeb995700 1 -- 172.21.15.143:0/1174702490 --> [v2:172.21.15.143:6816/2021504428,v1:172.21.15.143:6817/2021504428] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fcfe40ce050 con 0x7fcfc8072f30 2023-11-08T19:25:01.127 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.019+0000 7f0cfc5a1700 1 -- 172.21.15.143:0/3426336670 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f0cd806afd0 msgr2=0x7f0cd806d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:01.127 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.019+0000 7f0cfc5a1700 1 --2- 172.21.15.143:0/3426336670 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f0cd806afd0 0x7f0cd806d490 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0cf40cf4c0 tx=0x7f0cec009000 comp rx=0 tx=0).stop 2023-11-08T19:25:01.127 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.019+0000 7f0cfc5a1700 1 -- 172.21.15.143:0/3426336670 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0cf40cee90 msgr2=0x7f0cf415c130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:01.127 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.019+0000 7f0cfc5a1700 1 --2- 172.21.15.143:0/3426336670 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0cf40cee90 0x7f0cf415c130 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f0cf40723b0 tx=0x7f0cf000b720 comp rx=0 tx=0).stop 2023-11-08T19:25:01.128 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.019+0000 7f0cfab3e700 1 -- 172.21.15.143:0/3426336670 reap_dead start 2023-11-08T19:25:01.128 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.019+0000 7f2a725c5700 1 -- 172.21.15.143:0/1080853847 --> [v2:172.21.15.143:6800/787341397,v1:172.21.15.143:6801/787341397] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f2a6c0c2e30 con 0x7f2a6c200840 2023-11-08T19:25:01.128 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.019+0000 7f0cfc5a1700 1 -- 172.21.15.143:0/3426336670 shutdown_connections 2023-11-08T19:25:01.129 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.019+0000 7f0cfc5a1700 1 -- 172.21.15.143:0/3426336670 >> 172.21.15.143:0/3426336670 conn(0x7f0cf406b060 msgr2=0x7f0cf40af4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:01.129 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.020+0000 7fcfd9ffb700 1 -- 172.21.15.143:0/1174702490 <== osd.3 v2:172.21.15.143:6816/2021504428 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fcfd4009720 con 0x7fcfc8072f30 2023-11-08T19:25:01.129 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.020+0000 7f0cfc5a1700 1 -- 172.21.15.143:0/3426336670 shutdown_connections 2023-11-08T19:25:01.130 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.020+0000 7f2a68ff9700 1 -- 172.21.15.143:0/1080853847 <== osd.1 v2:172.21.15.143:6800/787341397 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f2a60009780 con 0x7f2a6c200840 2023-11-08T19:25:01.131 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.020+0000 7fb50800c700 1 -- 172.21.15.143:0/2072945897 --> [v2:172.21.15.174:6808/1777923303,v1:172.21.15.174:6809/1777923303] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb5000cdf90 con 0x7fb4e40730a0 2023-11-08T19:25:01.132 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.020+0000 7f0cfc5a1700 1 -- 172.21.15.143:0/3426336670 wait complete. 2023-11-08T19:25:01.132 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.020+0000 7f2a725c5700 1 -- 172.21.15.143:0/1080853847 >> [v2:172.21.15.143:6800/787341397,v1:172.21.15.143:6801/787341397] conn(0x7f2a6c200840 msgr2=0x7f2a6c200c20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:01.133 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.020+0000 7f2a725c5700 1 --2- 172.21.15.143:0/1080853847 >> [v2:172.21.15.143:6800/787341397,v1:172.21.15.143:6801/787341397] conn(0x7f2a6c200840 0x7f2a6c200c20 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:01.133 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.020+0000 7f2a725c5700 1 -- 172.21.15.143:0/1080853847 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f2a5806af00 msgr2=0x7f2a5806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:01.133 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.020+0000 7f2a725c5700 1 --2- 172.21.15.143:0/1080853847 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f2a5806af00 0x7f2a5806d3c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c0c58a0 tx=0x7f2a6400b040 comp rx=0 tx=0).stop 2023-11-08T19:25:01.133 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.020+0000 7f2a725c5700 1 -- 172.21.15.143:0/1080853847 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2a6c0cd820 msgr2=0x7f2a6c0b0e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:01.133 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.020+0000 7f2a725c5700 1 --2- 172.21.15.143:0/1080853847 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2a6c0cd820 0x7f2a6c0b0e20 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c0711f0 tx=0x7f2a5c004750 comp rx=0 tx=0).stop 2023-11-08T19:25:01.134 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.020+0000 7f7dbf7fe700 1 --2- 172.21.15.143:0/896618954 >> [v2:172.21.15.174:6800/1993380595,v1:172.21.15.174:6801/1993380595] conn(0x7f7dc0150d90 0x7f7dc00b8ea0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:01.134 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.020+0000 7fb4f67fc700 1 -- 172.21.15.143:0/2072945897 <== osd.5 v2:172.21.15.174:6808/1777923303 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fb4f8009e80 con 0x7fb4e40730a0 2023-11-08T19:25:01.135 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.021+0000 7fb50800c700 1 -- 172.21.15.143:0/2072945897 >> [v2:172.21.15.174:6808/1777923303,v1:172.21.15.174:6809/1777923303] conn(0x7fb4e40730a0 msgr2=0x7fb4e40754c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:01.135 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.021+0000 7fb50800c700 1 --2- 172.21.15.143:0/2072945897 >> [v2:172.21.15.174:6808/1777923303,v1:172.21.15.174:6809/1777923303] conn(0x7fb4e40730a0 0x7fb4e40754c0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:01.136 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.021+0000 7fb50800c700 1 -- 172.21.15.143:0/2072945897 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fb4e406af00 msgr2=0x7fb4e406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:01.136 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.021+0000 7fb50800c700 1 --2- 172.21.15.143:0/2072945897 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fb4e406af00 0x7fb4e406d3c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb5000c9240 tx=0x7fb4fc0079c0 comp rx=0 tx=0).stop 2023-11-08T19:25:01.136 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.021+0000 7fb50800c700 1 -- 172.21.15.143:0/2072945897 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb500072130 msgr2=0x7fb5001dbe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:01.136 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.021+0000 7fb50800c700 1 --2- 172.21.15.143:0/2072945897 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb500072130 0x7fb5001dbe00 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb4f0014040 tx=0x7fb4f0004750 comp rx=0 tx=0).stop 2023-11-08T19:25:01.136 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.021+0000 7fb5065a9700 1 -- 172.21.15.143:0/2072945897 reap_dead start 2023-11-08T19:25:01.137 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.021+0000 7f2a70b62700 1 -- 172.21.15.143:0/1080853847 reap_dead start 2023-11-08T19:25:01.137 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.021+0000 7f3e25fce700 1 -- 172.21.15.143:0/4135659825 --> [v2:172.21.15.143:6820/2301325597,v1:172.21.15.143:6822/2301325597] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f3e200cb330 con 0x7f3e0c073170 2023-11-08T19:25:01.138 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.021+0000 7f2a725c5700 1 -- 172.21.15.143:0/1080853847 shutdown_connections 2023-11-08T19:25:01.138 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.021+0000 7f2a725c5700 1 -- 172.21.15.143:0/1080853847 >> 172.21.15.143:0/1080853847 conn(0x7f2a6c06b060 msgr2=0x7f2a6c1657d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:01.139 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.022+0000 7fb50800c700 1 -- 172.21.15.143:0/2072945897 shutdown_connections 2023-11-08T19:25:01.139 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.022+0000 7fb50800c700 1 -- 172.21.15.143:0/2072945897 >> 172.21.15.143:0/2072945897 conn(0x7fb5000b1650 msgr2=0x7fb5001ecbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:01.139 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.022+0000 7f7da77fe700 1 -- 172.21.15.143:0/896618954 <== osd.4 v2:172.21.15.174:6800/1993380595 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26745 (crc 0 0 0) 0x7f7db8009e80 con 0x7f7dc0150d90 2023-11-08T19:25:01.140 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.022+0000 7f2a725c5700 1 -- 172.21.15.143:0/1080853847 shutdown_connections 2023-11-08T19:25:01.140 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.022+0000 7fb50800c700 1 -- 172.21.15.143:0/2072945897 shutdown_connections 2023-11-08T19:25:01.141 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.022+0000 7fb50800c700 1 -- 172.21.15.143:0/2072945897 wait complete. 2023-11-08T19:25:01.141 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.022+0000 7f2a725c5700 1 -- 172.21.15.143:0/1080853847 wait complete. 2023-11-08T19:25:01.142 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.022+0000 7f3e07fff700 1 -- 172.21.15.143:0/4135659825 <== osd.2 v2:172.21.15.143:6820/2301325597 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f3e18001a40 con 0x7f3e0c073170 2023-11-08T19:25:01.144 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.023+0000 7f3e25fce700 1 -- 172.21.15.143:0/4135659825 >> [v2:172.21.15.143:6820/2301325597,v1:172.21.15.143:6822/2301325597] conn(0x7f3e0c073170 msgr2=0x7f3e0c075590 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:01.144 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.023+0000 7f3e25fce700 1 --2- 172.21.15.143:0/4135659825 >> [v2:172.21.15.143:6820/2301325597,v1:172.21.15.143:6822/2301325597] conn(0x7f3e0c073170 0x7f3e0c075590 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:01.144 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.023+0000 7f3e25fce700 1 -- 172.21.15.143:0/4135659825 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f3e0c06af50 msgr2=0x7f3e0c06d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:01.145 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.023+0000 7f3e25fce700 1 --2- 172.21.15.143:0/4135659825 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f3e0c06af50 0x7f3e0c06d410 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3e2014e220 tx=0x7f3e14008030 comp rx=0 tx=0).stop 2023-11-08T19:25:01.145 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.023+0000 7f3e25fce700 1 -- 172.21.15.143:0/4135659825 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e200cc100 msgr2=0x7f3e200c3100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:01.145 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.023+0000 7f3e25fce700 1 --2- 172.21.15.143:0/4135659825 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e200cc100 0x7f3e200c3100 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f3e1000e840 tx=0x7f3e1000ec00 comp rx=0 tx=0).stop 2023-11-08T19:25:01.145 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.023+0000 7f3e1ffff700 1 -- 172.21.15.143:0/4135659825 reap_dead start 2023-11-08T19:25:01.146 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.023+0000 7f3e25fce700 1 -- 172.21.15.143:0/4135659825 shutdown_connections 2023-11-08T19:25:01.146 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.023+0000 7f3e25fce700 1 -- 172.21.15.143:0/4135659825 >> 172.21.15.143:0/4135659825 conn(0x7f3e2006b060 msgr2=0x7f3e2014e300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:01.147 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.028+0000 7fcfeb995700 1 -- 172.21.15.143:0/1174702490 >> [v2:172.21.15.143:6816/2021504428,v1:172.21.15.143:6817/2021504428] conn(0x7fcfc8072f30 msgr2=0x7fcfc8075350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:01.147 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.028+0000 7fcfeb995700 1 --2- 172.21.15.143:0/1174702490 >> [v2:172.21.15.143:6816/2021504428,v1:172.21.15.143:6817/2021504428] conn(0x7fcfc8072f30 0x7fcfc8075350 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:01.148 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.029+0000 7fcfeb995700 1 -- 172.21.15.143:0/1174702490 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fcfc806af00 msgr2=0x7fcfc806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:01.148 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.029+0000 7fcfeb995700 1 --2- 172.21.15.143:0/1174702490 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fcfc806af00 0x7fcfc806d3c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fcfe0007fd0 tx=0x7fcfe0009040 comp rx=0 tx=0).stop 2023-11-08T19:25:01.148 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.029+0000 7fcfeb995700 1 -- 172.21.15.143:0/1174702490 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcfe4162060 msgr2=0x7fcfe4160590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:01.148 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.029+0000 7fcfeb995700 1 --2- 172.21.15.143:0/1174702490 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcfe4162060 0x7fcfe4160590 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fcfdc014000 tx=0x7fcfdc004ad0 comp rx=0 tx=0).stop 2023-11-08T19:25:01.149 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.029+0000 7fcfe9f32700 1 -- 172.21.15.143:0/1174702490 reap_dead start 2023-11-08T19:25:01.149 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.029+0000 7fcfeb995700 1 -- 172.21.15.143:0/1174702490 shutdown_connections 2023-11-08T19:25:01.149 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.029+0000 7fcfeb995700 1 -- 172.21.15.143:0/1174702490 >> 172.21.15.143:0/1174702490 conn(0x7fcfe406b4c0 msgr2=0x7fcfe415f620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:01.150 INFO:teuthology.orchestra.run.smithi143.stdout:55834574855 2023-11-08T19:25:01.150 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-11-08T19:25:01.151 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.031+0000 7f3a49fce700 1 -- 172.21.15.143:0/2834563838 --> [v2:172.21.15.174:6816/2847141411,v1:172.21.15.174:6817/2847141411] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f3a24005cb0 con 0x7f3a240015e0 2023-11-08T19:25:01.151 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.032+0000 7fcfeb995700 1 -- 172.21.15.143:0/1174702490 shutdown_connections 2023-11-08T19:25:01.152 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.032+0000 7fcfeb995700 1 -- 172.21.15.143:0/1174702490 wait complete. 2023-11-08T19:25:01.152 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.032+0000 7f3a2bfff700 1 -- 172.21.15.143:0/2834563838 <== osd.7 v2:172.21.15.174:6816/2847141411 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f3a24005cb0 con 0x7f3a240015e0 2023-11-08T19:25:01.153 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.033+0000 7f3a49fce700 1 -- 172.21.15.143:0/2834563838 >> [v2:172.21.15.174:6816/2847141411,v1:172.21.15.174:6817/2847141411] conn(0x7f3a240015e0 msgr2=0x7f3a24003aa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:01.153 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.033+0000 7f3a49fce700 1 --2- 172.21.15.143:0/2834563838 >> [v2:172.21.15.174:6816/2847141411,v1:172.21.15.174:6817/2847141411] conn(0x7f3a240015e0 0x7f3a24003aa0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:01.153 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.034+0000 7f3e25fce700 1 -- 172.21.15.143:0/4135659825 shutdown_connections 2023-11-08T19:25:01.154 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.037+0000 7f3e25fce700 1 -- 172.21.15.143:0/4135659825 wait complete. 2023-11-08T19:25:01.155 INFO:teuthology.orchestra.run.smithi143.stdout:55834574855 2023-11-08T19:25:01.156 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-11-08T19:25:01.157 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.040+0000 7f3a49fce700 1 -- 172.21.15.143:0/2834563838 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f3a2c06b070 msgr2=0x7f3a2c06d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:01.157 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.040+0000 7f3a49fce700 1 --2- 172.21.15.143:0/2834563838 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f3a2c06b070 0x7f3a2c06d530 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3a38009b90 tx=0x7f3a38008030 comp rx=0 tx=0).stop 2023-11-08T19:25:01.157 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.040+0000 7f3a49fce700 1 -- 172.21.15.143:0/2834563838 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3a44159e00 msgr2=0x7f3a441557e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:01.158 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.040+0000 7f3a49fce700 1 --2- 172.21.15.143:0/2834563838 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3a44159e00 0x7f3a441557e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c00d750 tx=0x7f3a3c00da60 comp rx=0 tx=0).stop 2023-11-08T19:25:01.158 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.040+0000 7f3a43fff700 1 -- 172.21.15.143:0/2834563838 reap_dead start 2023-11-08T19:25:01.159 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.041+0000 7f3a49fce700 1 -- 172.21.15.143:0/2834563838 shutdown_connections 2023-11-08T19:25:01.159 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.041+0000 7f3a49fce700 1 -- 172.21.15.143:0/2834563838 >> 172.21.15.143:0/2834563838 conn(0x7f3a4406b060 msgr2=0x7f3a440cdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:01.160 INFO:teuthology.orchestra.run.smithi143.stdout:55834574855 2023-11-08T19:25:01.161 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-11-08T19:25:01.162 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.045+0000 7f3a49fce700 1 -- 172.21.15.143:0/2834563838 shutdown_connections 2023-11-08T19:25:01.163 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.045+0000 7f3a49fce700 1 -- 172.21.15.143:0/2834563838 wait complete. 2023-11-08T19:25:01.163 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.049+0000 7f7dc54a9700 1 -- 172.21.15.143:0/896618954 --> [v2:172.21.15.174:6800/1993380595,v1:172.21.15.174:6801/1993380595] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f7dc00d2bb0 con 0x7f7dc0150d90 2023-11-08T19:25:01.165 INFO:teuthology.orchestra.run.smithi143.stdout:55834574855 2023-11-08T19:25:01.165 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-11-08T19:25:01.166 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.049+0000 7f7da77fe700 1 -- 172.21.15.143:0/896618954 <== osd.4 v2:172.21.15.174:6800/1993380595 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f7db8013310 con 0x7f7dc0150d90 2023-11-08T19:25:01.168 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.050+0000 7f7dc54a9700 1 -- 172.21.15.143:0/896618954 >> [v2:172.21.15.174:6800/1993380595,v1:172.21.15.174:6801/1993380595] conn(0x7f7dc0150d90 msgr2=0x7f7dc00b8ea0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:01.168 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.050+0000 7f7dc54a9700 1 --2- 172.21.15.143:0/896618954 >> [v2:172.21.15.174:6800/1993380595,v1:172.21.15.174:6801/1993380595] conn(0x7f7dc0150d90 0x7f7dc00b8ea0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:01.168 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.050+0000 7f7dc54a9700 1 -- 172.21.15.143:0/896618954 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f7dac06af00 msgr2=0x7f7dac06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:01.169 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.050+0000 7f7dc54a9700 1 --2- 172.21.15.143:0/896618954 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f7dac06af00 0x7f7dac06d3c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7db4009c40 tx=0x7f7db4008030 comp rx=0 tx=0).stop 2023-11-08T19:25:01.169 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.050+0000 7f7dc54a9700 1 -- 172.21.15.143:0/896618954 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7dc00d3690 msgr2=0x7f7dc0200ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:01.170 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.050+0000 7f7dc54a9700 1 --2- 172.21.15.143:0/896618954 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7dc00d3690 0x7f7dc0200ed0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7dc00d3a90 tx=0x7f7db000c6e0 comp rx=0 tx=0).stop 2023-11-08T19:25:01.170 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.050+0000 7f7dbf7fe700 1 -- 172.21.15.143:0/896618954 reap_dead start 2023-11-08T19:25:01.171 INFO:teuthology.orchestra.run.smithi143.stdout:55834574856 2023-11-08T19:25:01.172 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-11-08T19:25:01.173 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.051+0000 7f7dc54a9700 1 -- 172.21.15.143:0/896618954 shutdown_connections 2023-11-08T19:25:01.174 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.051+0000 7f7dc54a9700 1 -- 172.21.15.143:0/896618954 >> 172.21.15.143:0/896618954 conn(0x7f7dc00668e0 msgr2=0x7f7dc015c910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:01.174 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.051+0000 7f7dc54a9700 1 -- 172.21.15.143:0/896618954 shutdown_connections 2023-11-08T19:25:01.174 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.051+0000 7f7dc54a9700 1 -- 172.21.15.143:0/896618954 wait complete. 2023-11-08T19:25:01.176 INFO:teuthology.orchestra.run.smithi143.stdout:55834574855 2023-11-08T19:25:01.176 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-11-08T19:25:01.178 INFO:teuthology.orchestra.run.smithi143.stdout:55834574855 2023-11-08T19:25:01.179 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-11-08T19:25:01.180 INFO:teuthology.orchestra.run.smithi143.stdout:55834574855 2023-11-08T19:25:01.181 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-11-08T19:25:01.399 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.397+0000 7f144cff0700 1 Processor -- start 2023-11-08T19:25:01.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.397+0000 7f144cff0700 1 -- start start 2023-11-08T19:25:01.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.399+0000 7f144cff0700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f1448162060 0x7f14480aea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:01.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.399+0000 7f144cff0700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f14480b1100 0x7f14480b1540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:01.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.399+0000 7f144cff0700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f144806cf90 con 0x7f1448165c50 2023-11-08T19:25:01.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.399+0000 7f144cff0700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14480c9c40 con 0x7f1448162060 2023-11-08T19:25:01.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.399+0000 7f144cff0700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14480cdb60 con 0x7f14480b1100 2023-11-08T19:25:01.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.397+0000 7fa60b72d700 1 Processor -- start 2023-11-08T19:25:01.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.397+0000 7fa60b72d700 1 -- start start 2023-11-08T19:25:01.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.398+0000 7fa60b72d700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa6040b1810 0x7fa6040cb2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:01.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.398+0000 7fa60b72d700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6040ca100 con 0x7fa604159cf0 2023-11-08T19:25:01.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.398+0000 7fa60b72d700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6040ce330 con 0x7fa6040cae60 2023-11-08T19:25:01.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.398+0000 7fa60b72d700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6040ce610 con 0x7fa6040b1810 2023-11-08T19:25:01.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.399+0000 7fa6094c9700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fa604159cf0 0x7fa60406d0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:45086/0 (socket says 172.21.15.143:45086) 2023-11-08T19:25:01.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.399+0000 7fa6094c9700 1 -- 172.21.15.143:0/3606436659 learned_addr learned my addr 172.21.15.143:0/3606436659 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:25:01.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.400+0000 7f1445d9b700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f14480b1100 0x7f14480b1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:01.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.400+0000 7f144659c700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f1448162060 0x7f14480aea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:01.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.400+0000 7f144659c700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f1448162060 0x7f14480aea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.143:60668/0 (socket says 172.21.15.143:60668) 2023-11-08T19:25:01.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.400+0000 7f144659c700 1 -- 172.21.15.143:0/1656884998 learned_addr learned my addr 172.21.15.143:0/1656884998 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:01.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.400+0000 7fa609cca700 1 --2- 172.21.15.143:0/3606436659 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa6040b1810 0x7fa6040cb2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:01.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.400+0000 7f1445d9b700 1 -- 172.21.15.143:0/1656884998 >> v2:172.21.15.174:3300/0 conn(0x7f1448162060 msgr2=0x7f14480aea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:01.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.400+0000 7f1445d9b700 1 --2- 172.21.15.143:0/1656884998 >> v2:172.21.15.174:3300/0 conn(0x7f1448162060 0x7f14480aea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:01.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.400+0000 7f1445d9b700 1 -- 172.21.15.143:0/1656884998 >> v1:172.21.15.174:6789/0 conn(0x7f1448165c50 legacy=0x7f14481660c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:01.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.400+0000 7f144559a700 1 -- 172.21.15.143:0/1656884998 <== mon.1 v1:172.21.15.174:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1089940053 0 0) 0x7f144806cf90 con 0x7f1448165c50 2023-11-08T19:25:01.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.401+0000 7f1445d9b700 1 -- 172.21.15.143:0/1656884998 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14480b1a80 con 0x7f14480b1100 2023-11-08T19:25:01.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.401+0000 7f144659c700 1 --2- 172.21.15.143:0/1656884998 >> v2:172.21.15.174:3300/0 conn(0x7f1448162060 0x7f14480aea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T19:25:01.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.401+0000 7f1445d9b700 1 --2- 172.21.15.143:0/1656884998 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f14480b1100 0x7f14480b1540 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f143c009840 tx=0x7f143c009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53abf7bd2f7cb00e server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:01.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.401+0000 7fa609cca700 1 -- 172.21.15.143:0/3606436659 >> v1:172.21.15.174:6789/0 conn(0x7fa6040cae60 legacy=0x7fa6040b1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:01.409 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.401+0000 7fa609cca700 1 -- 172.21.15.143:0/3606436659 >> v1:172.21.15.143:6789/0 conn(0x7fa604159cf0 legacy=0x7fa60406d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:01.409 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.401+0000 7fa609cca700 1 -- 172.21.15.143:0/3606436659 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6040ca3a0 con 0x7fa6040b1810 2023-11-08T19:25:01.410 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.401+0000 7f144559a700 1 -- 172.21.15.143:0/1656884998 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f143c011830 con 0x7f14480b1100 2023-11-08T19:25:01.410 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.401+0000 7f144559a700 1 -- 172.21.15.143:0/1656884998 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f143c011990 con 0x7f14480b1100 2023-11-08T19:25:01.410 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.401+0000 7f144559a700 1 -- 172.21.15.143:0/1656884998 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f143c011d20 con 0x7f14480b1100 2023-11-08T19:25:01.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.402+0000 7f144cff0700 1 -- 172.21.15.143:0/1656884998 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f14480b1100 msgr2=0x7f14480b1540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:01.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.402+0000 7f144cff0700 1 --2- 172.21.15.143:0/1656884998 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f14480b1100 0x7f14480b1540 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f143c009840 tx=0x7f143c009b50 comp rx=0 tx=0).stop 2023-11-08T19:25:01.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.401+0000 7fa609cca700 1 --2- 172.21.15.143:0/3606436659 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa6040b1810 0x7fa6040cb2f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fa5fc00d210 tx=0x7fa5fc00d520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=60b7dbb38a9407d7 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:01.412 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.402+0000 7fa5fbfff700 1 -- 172.21.15.143:0/3606436659 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5fc010070 con 0x7fa6040b1810 2023-11-08T19:25:01.412 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.402+0000 7f144cff0700 1 -- 172.21.15.143:0/1656884998 shutdown_connections 2023-11-08T19:25:01.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.402+0000 7f144cff0700 1 --2- 172.21.15.143:0/1656884998 >> v2:172.21.15.174:3300/0 conn(0x7f1448162060 0x7f14480aea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:01.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.402+0000 7f144cff0700 1 --2- 172.21.15.143:0/1656884998 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f14480b1100 0x7f14480b1540 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:01.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.402+0000 7f144cff0700 1 -- 172.21.15.143:0/1656884998 >> 172.21.15.143:0/1656884998 conn(0x7f144806b4c0 msgr2=0x7f14480cf680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:01.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.403+0000 7fa5fbfff700 1 -- 172.21.15.143:0/3606436659 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5fc00dcd0 con 0x7fa6040b1810 2023-11-08T19:25:01.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.403+0000 7fa5fbfff700 1 -- 172.21.15.143:0/3606436659 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5fc007cc0 con 0x7fa6040b1810 2023-11-08T19:25:01.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.403+0000 7fa60b72d700 1 -- 172.21.15.143:0/3606436659 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa6040b1810 msgr2=0x7fa6040cb2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:01.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.403+0000 7fa60b72d700 1 --2- 172.21.15.143:0/3606436659 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa6040b1810 0x7fa6040cb2f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fa5fc00d210 tx=0x7fa5fc00d520 comp rx=0 tx=0).stop 2023-11-08T19:25:01.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.403+0000 7f144cff0700 1 -- 172.21.15.143:0/1656884998 shutdown_connections 2023-11-08T19:25:01.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.403+0000 7f144cff0700 1 -- 172.21.15.143:0/1656884998 wait complete. 2023-11-08T19:25:01.417 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.404+0000 7fa60b72d700 1 -- 172.21.15.143:0/3606436659 shutdown_connections 2023-11-08T19:25:01.418 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.404+0000 7fa60b72d700 1 --2- 172.21.15.143:0/3606436659 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa6040b1810 0x7fa6040cb2f0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:01.418 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.404+0000 7fa60b72d700 1 -- 172.21.15.143:0/3606436659 >> 172.21.15.143:0/3606436659 conn(0x7fa60406b4c0 msgr2=0x7fa6040cf680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:01.418 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.404+0000 7fa60b72d700 1 -- 172.21.15.143:0/3606436659 shutdown_connections 2023-11-08T19:25:01.419 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.404+0000 7fa60b72d700 1 -- 172.21.15.143:0/3606436659 wait complete. 2023-11-08T19:25:01.419 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.405+0000 7f144cff0700 1 Processor -- start 2023-11-08T19:25:01.420 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.406+0000 7fa60b72d700 1 Processor -- start 2023-11-08T19:25:01.421 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.405+0000 7f144cff0700 1 -- start start 2023-11-08T19:25:01.421 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.406+0000 7f144cff0700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f14480b1100 0x7f1448166060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:01.421 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.406+0000 7f144cff0700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1448162060 0x7f14481665a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:01.422 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.406+0000 7f144cff0700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1448165c50 0x7f144815d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:01.422 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.406+0000 7f144cff0700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f144815e430 con 0x7f1448165c50 2023-11-08T19:25:01.422 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.406+0000 7f144cff0700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1448166ae0 con 0x7f1448162060 2023-11-08T19:25:01.422 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.406+0000 7f144cff0700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f144815de70 con 0x7f14480b1100 2023-11-08T19:25:01.423 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.407+0000 7f1445d9b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1448162060 0x7f14481665a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:01.423 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.407+0000 7f1446d9d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1448165c50 0x7f144815d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:01.423 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.407+0000 7f1445d9b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1448162060 0x7f14481665a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.143:60672/0 (socket says 172.21.15.143:60672) 2023-11-08T19:25:01.423 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.407+0000 7f1445d9b700 1 -- 172.21.15.143:0/2524222030 learned_addr learned my addr 172.21.15.143:0/2524222030 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:01.423 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.407+0000 7f144659c700 1 --2- 172.21.15.143:0/2524222030 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f14480b1100 0x7f1448166060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:01.424 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.406+0000 7fa60b72d700 1 -- start start 2023-11-08T19:25:01.424 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.407+0000 7fa60b72d700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa6040b1810 0x7fa6040b46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:01.424 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.407+0000 7fa60b72d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6040cae60 0x7fa60415aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:01.425 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.407+0000 7fa60b72d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa604159cf0 0x7fa6040c8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:01.425 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.407+0000 7fa60b72d700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6041f7f50 con 0x7fa6040cae60 2023-11-08T19:25:01.425 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.407+0000 7fa60b72d700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6041f87b0 con 0x7fa604159cf0 2023-11-08T19:25:01.425 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.407+0000 7fa60b72d700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa60415b730 con 0x7fa6040b1810 2023-11-08T19:25:01.425 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.408+0000 7f1446d9d700 1 -- 172.21.15.143:0/2524222030 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f14480b1100 msgr2=0x7f1448166060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:01.426 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.408+0000 7f1446d9d700 1 --2- 172.21.15.143:0/2524222030 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f14480b1100 0x7f1448166060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:01.426 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.408+0000 7f1446d9d700 1 -- 172.21.15.143:0/2524222030 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1448162060 msgr2=0x7f14481665a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:01.426 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.408+0000 7f1446d9d700 1 --2- 172.21.15.143:0/2524222030 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1448162060 0x7f14481665a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:01.426 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.408+0000 7f1446d9d700 1 -- 172.21.15.143:0/2524222030 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1438009550 con 0x7f1448165c50 2023-11-08T19:25:01.427 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.408+0000 7fa6094c9700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa6040b1810 0x7fa6040b46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:01.427 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.408+0000 7fa6094c9700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa6040b1810 0x7fa6040b46e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.143:36600/0 (socket says 172.21.15.143:36600) 2023-11-08T19:25:01.427 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.408+0000 7fa6094c9700 1 -- 172.21.15.143:0/785107892 learned_addr learned my addr 172.21.15.143:0/785107892 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:01.428 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.408+0000 7fa609cca700 1 --2- 172.21.15.143:0/785107892 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa604159cf0 0x7fa6040c8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:01.428 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.408+0000 7f1446d9d700 1 --2- 172.21.15.143:0/2524222030 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1448165c50 0x7f144815d1d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f144000c780 tx=0x7f144000ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:01.429 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.405+0000 7fa5c7e8d700 1 Processor -- start 2023-11-08T19:25:01.429 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.406+0000 7fa5c7e8d700 1 -- start start 2023-11-08T19:25:01.430 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.407+0000 7fa5c7e8d700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fa5c00b1370 0x7fa5c016eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:01.430 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.407+0000 7fa5c7e8d700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa5c00cda90 0x7fa5c016d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:01.430 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.407+0000 7fa5c7e8d700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5c016f3e0 con 0x7fa5c00c9c40 2023-11-08T19:25:01.430 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.407+0000 7fa5c7e8d700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5c006d550 con 0x7fa5c00b1370 2023-11-08T19:25:01.431 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.407+0000 7fa5c7e8d700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c008a660 con 0x7fa5c00cda90 2023-11-08T19:25:01.431 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.407+0000 7fa5c5428700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa5c00cda90 0x7fa5c016d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:01.431 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.407+0000 7fa5c5428700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa5c00cda90 0x7fa5c016d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.143:36592/0 (socket says 172.21.15.143:36592) 2023-11-08T19:25:01.431 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.408+0000 7fa5c5428700 1 -- 172.21.15.143:0/764260538 learned_addr learned my addr 172.21.15.143:0/764260538 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:01.432 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.408+0000 7fa5c5c29700 1 --2- 172.21.15.143:0/764260538 >> v2:172.21.15.174:3300/0 conn(0x7fa5c00b1370 0x7fa5c016eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:01.432 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.408+0000 7fa5c4c27700 1 -- 172.21.15.143:0/764260538 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1728840165 0 0) 0x7fa5b8001490 con 0x7fa5c00c9c40 2023-11-08T19:25:01.432 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.408+0000 7fa5c4c27700 1 -- 172.21.15.143:0/764260538 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5ac003590 con 0x7fa5c00c9c40 2023-11-08T19:25:01.432 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.409+0000 7fa5c5428700 1 -- 172.21.15.143:0/764260538 >> v2:172.21.15.174:3300/0 conn(0x7fa5c00b1370 msgr2=0x7fa5c016eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:01.433 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.409+0000 7fa5c5428700 1 --2- 172.21.15.143:0/764260538 >> v2:172.21.15.174:3300/0 conn(0x7fa5c00b1370 0x7fa5c016eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:01.433 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.409+0000 7fa5c5428700 1 -- 172.21.15.143:0/764260538 >> v1:172.21.15.143:6789/0 conn(0x7fa5c00c9c40 legacy=0x7fa5c0070e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:01.433 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.409+0000 7fa5c5428700 1 -- 172.21.15.143:0/764260538 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5c00ce200 con 0x7fa5c00cda90 2023-11-08T19:25:01.434 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.409+0000 7fa6094c9700 1 -- 172.21.15.143:0/785107892 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa604159cf0 msgr2=0x7fa6040c8d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:01.434 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.409+0000 7fa6094c9700 1 --2- 172.21.15.143:0/785107892 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa604159cf0 0x7fa6040c8d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:01.434 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.409+0000 7fa6094c9700 1 -- 172.21.15.143:0/785107892 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6040cae60 msgr2=0x7fa60415aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:01.434 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.409+0000 7fa6094c9700 1 --2- 172.21.15.143:0/785107892 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6040cae60 0x7fa60415aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:01.435 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.409+0000 7fa6094c9700 1 -- 172.21.15.143:0/785107892 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5fc007910 con 0x7fa6040b1810 2023-11-08T19:25:01.435 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.409+0000 7fa5c5428700 1 --2- 172.21.15.143:0/764260538 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa5c00cda90 0x7fa5c016d8f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa5b0008370 tx=0x7fa5b000ac60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=36c8178db9e5348a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:01.435 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.406+0000 7fdb81b53700 1 Processor -- start 2023-11-08T19:25:01.436 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.406+0000 7fdb81b53700 1 -- start start 2023-11-08T19:25:01.436 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.407+0000 7fdb81b53700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fdb7c0b1370 0x7fdb7c16eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:01.436 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.407+0000 7fdb81b53700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fdb7c0cda90 0x7fdb7c16d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:01.436 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.407+0000 7fdb81b53700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb7c16f3e0 con 0x7fdb7c0c9c40 2023-11-08T19:25:01.437 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.407+0000 7fdb81b53700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb7c06d550 con 0x7fdb7c0b1370 2023-11-08T19:25:01.437 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.407+0000 7fdb81b53700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb7c08a660 con 0x7fdb7c0cda90 2023-11-08T19:25:01.437 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.408+0000 7fdb7affd700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fdb7c0cda90 0x7fdb7c16d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:01.437 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.408+0000 7fdb7bfff700 1 --1- >> v1:172.21.15.174:6789/0 conn(0x7fdb7c0c9c40 0x7fdb7c070e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.174:6789/0 says I am v1:172.21.15.143:56046/0 (socket says 172.21.15.143:56046) 2023-11-08T19:25:01.438 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.408+0000 7fdb7bfff700 1 -- 172.21.15.143:0/1782242200 learned_addr learned my addr 172.21.15.143:0/1782242200 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:25:01.438 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.408+0000 7fdb7a7fc700 1 -- 172.21.15.143:0/1782242200 <== mon.1 v1:172.21.15.174:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 528395966 0 0) 0x7fdb74001490 con 0x7fdb7c0c9c40 2023-11-08T19:25:01.438 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.408+0000 7fdb7b7fe700 1 --2- 172.21.15.143:0/1782242200 >> v2:172.21.15.143:3300/0 conn(0x7fdb7c0b1370 0x7fdb7c16eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:01.439 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.409+0000 7fdb7a7fc700 1 -- 172.21.15.143:0/1782242200 --> v1:172.21.15.174:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb64003590 con 0x7fdb7c0c9c40 2023-11-08T19:25:01.439 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.409+0000 7f1436ffd700 1 -- 172.21.15.143:0/2524222030 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1440003e30 con 0x7f1448165c50 2023-11-08T19:25:01.439 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.409+0000 7f1436ffd700 1 -- 172.21.15.143:0/2524222030 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14400050d0 con 0x7f1448165c50 2023-11-08T19:25:01.439 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.409+0000 7f1436ffd700 1 -- 172.21.15.143:0/2524222030 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1440005480 con 0x7f1448165c50 2023-11-08T19:25:01.440 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.409+0000 7fa6094c9700 1 --2- 172.21.15.143:0/785107892 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa6040b1810 0x7fa6040b46e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa60000c520 tx=0x7fa60000c830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:01.440 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.409+0000 7fa5f9ffb700 1 -- 172.21.15.143:0/785107892 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa600005d70 con 0x7fa6040b1810 2023-11-08T19:25:01.441 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.409+0000 7fa60b72d700 1 -- 172.21.15.143:0/785107892 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa60415b450 con 0x7fa6040b1810 2023-11-08T19:25:01.441 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.409+0000 7fa60b72d700 1 -- 172.21.15.143:0/785107892 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6041f8440 con 0x7fa6040b1810 2023-11-08T19:25:01.441 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.410+0000 7fa5f9ffb700 1 -- 172.21.15.143:0/785107892 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6000070d0 con 0x7fa6040b1810 2023-11-08T19:25:01.441 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.410+0000 7fa5f9ffb700 1 -- 172.21.15.143:0/785107892 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa600007440 con 0x7fa6040b1810 2023-11-08T19:25:01.442 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.409+0000 7f144cff0700 1 -- 172.21.15.143:0/2524222030 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f143c0094a0 con 0x7f1448165c50 2023-11-08T19:25:01.442 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.409+0000 7f144cff0700 1 -- 172.21.15.143:0/2524222030 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1448163630 con 0x7f1448165c50 2023-11-08T19:25:01.443 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.409+0000 7fa5c4c27700 1 -- 172.21.15.143:0/764260538 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5b0008df0 con 0x7fa5c00cda90 2023-11-08T19:25:01.443 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.409+0000 7fa5c4c27700 1 -- 172.21.15.143:0/764260538 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5b000a0d0 con 0x7fa5c00cda90 2023-11-08T19:25:01.443 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.409+0000 7fa5c4c27700 1 -- 172.21.15.143:0/764260538 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5b000a460 con 0x7fa5c00cda90 2023-11-08T19:25:01.443 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.410+0000 7fa5c7e8d700 1 -- 172.21.15.143:0/764260538 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa5c00cda90 msgr2=0x7fa5c016d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:01.444 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.410+0000 7fa5c7e8d700 1 --2- 172.21.15.143:0/764260538 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa5c00cda90 0x7fa5c016d8f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa5b0008370 tx=0x7fa5b000ac60 comp rx=0 tx=0).stop 2023-11-08T19:25:01.444 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.410+0000 7fa5c7e8d700 1 -- 172.21.15.143:0/764260538 shutdown_connections 2023-11-08T19:25:01.444 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.410+0000 7fa5c7e8d700 1 --2- 172.21.15.143:0/764260538 >> v2:172.21.15.174:3300/0 conn(0x7fa5c00b1370 0x7fa5c016eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:01.444 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.410+0000 7fa5c7e8d700 1 --2- 172.21.15.143:0/764260538 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa5c00cda90 0x7fa5c016d8f0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:01.444 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.410+0000 7fa5c7e8d700 1 -- 172.21.15.143:0/764260538 >> 172.21.15.143:0/764260538 conn(0x7fa5c006b060 msgr2=0x7fa5c006b470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:01.445 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.411+0000 7fa5c7e8d700 1 -- 172.21.15.143:0/764260538 shutdown_connections 2023-11-08T19:25:01.445 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.411+0000 7fa5f9ffb700 1 -- 172.21.15.143:0/785107892 <== mon.2 v2:172.21.15.174:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa6000075a0 con 0x7fa6040b1810 2023-11-08T19:25:01.445 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.411+0000 7fa5f9ffb700 1 --2- 172.21.15.143:0/785107892 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa5f006b010 0x7fa5f006d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:01.446 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.410+0000 7fdb7affd700 1 -- 172.21.15.143:0/1782242200 >> v2:172.21.15.143:3300/0 conn(0x7fdb7c0b1370 msgr2=0x7fdb7c16eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:01.446 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.410+0000 7fdb7affd700 1 --2- 172.21.15.143:0/1782242200 >> v2:172.21.15.143:3300/0 conn(0x7fdb7c0b1370 0x7fdb7c16eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:01.447 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.410+0000 7fdb7affd700 1 -- 172.21.15.143:0/1782242200 >> v1:172.21.15.174:6789/0 conn(0x7fdb7c0c9c40 legacy=0x7fdb7c070e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:01.447 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.410+0000 7fdb7affd700 1 -- 172.21.15.143:0/1782242200 --> v2:172.21.15.174:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb7c0ce200 con 0x7fdb7c0cda90 2023-11-08T19:25:01.447 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.410+0000 7fdb7affd700 1 --2- 172.21.15.143:0/1782242200 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdb7c0cda90 0x7fdb7c16d8f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c08afd0 tx=0x7fdb7000bbd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=37169bbbfe3af426 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:01.447 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.411+0000 7fdb7a7fc700 1 -- 172.21.15.143:0/1782242200 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb74001b80 con 0x7fdb7c0cda90 2023-11-08T19:25:01.447 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.411+0000 7fdb7a7fc700 1 -- 172.21.15.143:0/1782242200 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb70004030 con 0x7fdb7c0cda90 2023-11-08T19:25:01.448 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.411+0000 7fdb7a7fc700 1 -- 172.21.15.143:0/1782242200 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb700043a0 con 0x7fdb7c0cda90 2023-11-08T19:25:01.448 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.412+0000 7fa5f9ffb700 1 -- 172.21.15.143:0/785107892 <== mon.2 v2:172.21.15.174:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa600088950 con 0x7fa6040b1810 2023-11-08T19:25:01.448 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.411+0000 7fa5c7e8d700 1 -- 172.21.15.143:0/764260538 wait complete. 2023-11-08T19:25:01.449 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.412+0000 7fa608cc8700 1 --2- 172.21.15.143:0/785107892 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa5f006b010 0x7fa5f006d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:01.450 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.412+0000 7fdb81b53700 1 -- 172.21.15.143:0/1782242200 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdb7c0cda90 msgr2=0x7fdb7c16d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:01.450 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.412+0000 7fdb81b53700 1 --2- 172.21.15.143:0/1782242200 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdb7c0cda90 0x7fdb7c16d8f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c08afd0 tx=0x7fdb7000bbd0 comp rx=0 tx=0).stop 2023-11-08T19:25:01.450 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.412+0000 7fdb81b53700 1 -- 172.21.15.143:0/1782242200 shutdown_connections 2023-11-08T19:25:01.450 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.412+0000 7fdb81b53700 1 --2- 172.21.15.143:0/1782242200 >> v2:172.21.15.143:3300/0 conn(0x7fdb7c0b1370 0x7fdb7c16eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:01.451 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.412+0000 7fdb81b53700 1 --2- 172.21.15.143:0/1782242200 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdb7c0cda90 0x7fdb7c16d8f0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:01.451 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.412+0000 7fdb81b53700 1 -- 172.21.15.143:0/1782242200 >> 172.21.15.143:0/1782242200 conn(0x7fdb7c06b060 msgr2=0x7fdb7c06b470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:01.451 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.412+0000 7fdb81b53700 1 -- 172.21.15.143:0/1782242200 shutdown_connections 2023-11-08T19:25:01.451 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.412+0000 7fdb81b53700 1 -- 172.21.15.143:0/1782242200 wait complete. 2023-11-08T19:25:01.452 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.412+0000 7fa608cc8700 1 --2- 172.21.15.143:0/785107892 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa5f006b010 0x7fa5f006d4d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa6040c7910 tx=0x7fa5f4007a20 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:01.452 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.413+0000 7f1436ffd700 1 -- 172.21.15.143:0/2524222030 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f14400055e0 con 0x7f1448165c50 2023-11-08T19:25:01.453 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.413+0000 7fa5c7e8d700 1 Processor -- start 2023-11-08T19:25:01.453 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.413+0000 7fa5c7e8d700 1 -- start start 2023-11-08T19:25:01.453 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.413+0000 7fa5c7e8d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa5c00b1370 0x7fa5c00c91f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:01.453 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.413+0000 7fa5c7e8d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa5c00c9c40 0x7fa5c00b2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:01.454 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.413+0000 7fa5c7e8d700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa5c0165960 0x7fa5c0089740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:01.454 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.413+0000 7fa5c7e8d700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c00b3100 con 0x7fa5c00b1370 2023-11-08T19:25:01.454 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.413+0000 7fa5c7e8d700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c00b3360 con 0x7fa5c00c9c40 2023-11-08T19:25:01.454 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.413+0000 7fa5c7e8d700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c00b3640 con 0x7fa5c0165960 2023-11-08T19:25:01.455 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.414+0000 7fa5c5428700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa5c00c9c40 0x7fa5c00b2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:01.455 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.414+0000 7fa5c642a700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa5c0165960 0x7fa5c0089740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:01.455 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.414+0000 7fa5c5428700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa5c00c9c40 0x7fa5c00b2bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.143:60700/0 (socket says 172.21.15.143:60700) 2023-11-08T19:25:01.455 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.414+0000 7fa5c5428700 1 -- 172.21.15.143:0/1695703496 learned_addr learned my addr 172.21.15.143:0/1695703496 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:01.455 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.414+0000 7fa5c642a700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa5c0165960 0x7fa5c0089740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.143:36610/0 (socket says 172.21.15.143:36610) 2023-11-08T19:25:01.456 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.414+0000 7fa5c5c29700 1 --2- 172.21.15.143:0/1695703496 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa5c00b1370 0x7fa5c00c91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:01.456 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.413+0000 7f1436ffd700 1 --2- 172.21.15.143:0/2524222030 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f142806afd0 0x7f142806d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:01.456 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.414+0000 7f1436ffd700 1 -- 172.21.15.143:0/2524222030 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1440088be0 con 0x7f1448165c50 2023-11-08T19:25:01.457 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.414+0000 7f144cff0700 1 -- 172.21.15.143:0/2524222030 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f144804eee0 con 0x7f1448165c50 2023-11-08T19:25:01.457 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.414+0000 7f144659c700 1 --2- 172.21.15.143:0/2524222030 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f142806afd0 0x7f142806d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:01.457 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.414+0000 7fdb81b53700 1 Processor -- start 2023-11-08T19:25:01.458 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.414+0000 7fa60b72d700 1 -- 172.21.15.143:0/785107892 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5e40052f0 con 0x7fa6040b1810 2023-11-08T19:25:01.458 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.414+0000 7f22e6ef3700 1 Processor -- start 2023-11-08T19:25:01.459 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.414+0000 7fa5c5428700 1 -- 172.21.15.143:0/1695703496 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa5c0165960 msgr2=0x7fa5c0089740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:01.459 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.414+0000 7fa5c5428700 1 --2- 172.21.15.143:0/1695703496 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa5c0165960 0x7fa5c0089740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:01.459 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.414+0000 7fa5c5428700 1 -- 172.21.15.143:0/1695703496 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa5c00b1370 msgr2=0x7fa5c00c91f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:01.459 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.414+0000 7fa5c5428700 1 --2- 172.21.15.143:0/1695703496 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa5c00b1370 0x7fa5c00c91f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:01.460 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.414+0000 7fa5c5428700 1 -- 172.21.15.143:0/1695703496 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5b0008050 con 0x7fa5c00c9c40 2023-11-08T19:25:01.460 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.414+0000 7fa5c5c29700 1 --2- 172.21.15.143:0/1695703496 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa5c00b1370 0x7fa5c00c91f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T19:25:01.460 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.415+0000 7fa5c5428700 1 --2- 172.21.15.143:0/1695703496 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa5c00c9c40 0x7fa5c00b2bc0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa5b0000c00 tx=0x7fa5b0004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:01.460 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.415+0000 7f144659c700 1 --2- 172.21.15.143:0/2524222030 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f142806afd0 0x7f142806d490 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1438006d30 tx=0x7f14380092c0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:01.461 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.415+0000 7fa5b67fc700 1 -- 172.21.15.143:0/1695703496 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5b000a440 con 0x7fa5c00c9c40 2023-11-08T19:25:01.461 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.415+0000 7fa5b67fc700 1 -- 172.21.15.143:0/1695703496 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5b000a5a0 con 0x7fa5c00c9c40 2023-11-08T19:25:01.462 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.415+0000 7fa5c7e8d700 1 -- 172.21.15.143:0/1695703496 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5c00b3920 con 0x7fa5c00c9c40 2023-11-08T19:25:01.462 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.415+0000 7fa5c7e8d700 1 -- 172.21.15.143:0/1695703496 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5c00b3ee0 con 0x7fa5c00c9c40 2023-11-08T19:25:01.462 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.415+0000 7fa5b67fc700 1 -- 172.21.15.143:0/1695703496 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5b0009e50 con 0x7fa5c00c9c40 2023-11-08T19:25:01.462 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.414+0000 7fdb81b53700 1 -- start start 2023-11-08T19:25:01.463 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.416+0000 7fdb81b53700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb7c0b1370 0x7fdb7c208870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:01.463 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.416+0000 7fdb81b53700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdb7c0c9c40 0x7fdb7c0c70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:01.463 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.416+0000 7fdb81b53700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdb7c0b0580 0x7fdb7c16a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:01.463 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.416+0000 7fdb81b53700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb7c0c7a40 con 0x7fdb7c0b1370 2023-11-08T19:25:01.464 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.416+0000 7fdb81b53700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb7c16c700 con 0x7fdb7c0c9c40 2023-11-08T19:25:01.464 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.416+0000 7fdb81b53700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb7c0b0a30 con 0x7fdb7c0b0580 2023-11-08T19:25:01.464 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.415+0000 7f22e6ef3700 1 -- start start 2023-11-08T19:25:01.465 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.416+0000 7f22e6ef3700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f22e0159a20 0x7f22e006ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:01.465 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.416+0000 7f22e6ef3700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f22e00b1810 0x7f22e00ca100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:01.465 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.416+0000 7f22e6ef3700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22e00cd1c0 con 0x7f22e0159e00 2023-11-08T19:25:01.465 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.416+0000 7f22e6ef3700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22e00cd510 con 0x7f22e0159a20 2023-11-08T19:25:01.466 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.416+0000 7f22e6ef3700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22e015da60 con 0x7f22e00b1810 2023-11-08T19:25:01.466 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.415+0000 7ff7f81a8700 1 Processor -- start 2023-11-08T19:25:01.466 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.415+0000 7ff7f81a8700 1 -- start start 2023-11-08T19:25:01.467 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.416+0000 7fa5c7e8d700 1 -- 172.21.15.143:0/1695703496 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5c00b40f0 con 0x7fa5c00c9c40 2023-11-08T19:25:01.467 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.416+0000 7fdb7b7fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb7c0b1370 0x7fdb7c208870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:01.468 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.416+0000 7fdb7b7fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb7c0b1370 0x7fdb7c208870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48976/0 (socket says 172.21.15.143:48976) 2023-11-08T19:25:01.468 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.416+0000 7fdb7b7fe700 1 -- 172.21.15.143:0/2489207518 learned_addr learned my addr 172.21.15.143:0/2489207518 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:01.468 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.416+0000 7fdb7affd700 1 --2- 172.21.15.143:0/2489207518 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdb7c0c9c40 0x7fdb7c0c70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:01.468 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.417+0000 7fdb7affd700 1 -- 172.21.15.143:0/2489207518 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdb7c0b0580 msgr2=0x7fdb7c16a1d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-08T19:25:01.468 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.417+0000 7fdb7affd700 1 --2- 172.21.15.143:0/2489207518 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdb7c0b0580 0x7fdb7c16a1d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:01.469 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.417+0000 7fdb7affd700 1 -- 172.21.15.143:0/2489207518 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb7c0b1370 msgr2=0x7fdb7c208870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:01.469 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.417+0000 7fdb7affd700 1 --2- 172.21.15.143:0/2489207518 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb7c0b1370 0x7fdb7c208870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:01.469 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.417+0000 7fdb7affd700 1 -- 172.21.15.143:0/2489207518 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb7000b550 con 0x7fdb7c0c9c40 2023-11-08T19:25:01.469 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.417+0000 7fdb7affd700 1 --2- 172.21.15.143:0/2489207518 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdb7c0c9c40 0x7fdb7c0c70d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fdb70014000 tx=0x7fdb70004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:01.470 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.416+0000 7f22e4c8f700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f22e0159a20 0x7f22e006ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:01.470 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.416+0000 7f22e4c8f700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f22e0159a20 0x7f22e006ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48982/0 (socket says 172.21.15.143:48982) 2023-11-08T19:25:01.470 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.416+0000 7f22e4c8f700 1 -- 172.21.15.143:0/2196754278 learned_addr learned my addr 172.21.15.143:0/2196754278 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:01.471 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.416+0000 7f22e5490700 1 --2- 172.21.15.143:0/2196754278 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f22e00b1810 0x7f22e00ca100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:01.471 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.417+0000 7f22e4c8f700 1 -- 172.21.15.143:0/2196754278 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f22e00b1810 msgr2=0x7f22e00ca100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:01.471 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.417+0000 7f22e4c8f700 1 --2- 172.21.15.143:0/2196754278 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f22e00b1810 0x7f22e00ca100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:01.471 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.417+0000 7f22e4c8f700 1 -- 172.21.15.143:0/2196754278 >> v1:172.21.15.174:6789/0 conn(0x7f22e0159e00 legacy=0x7f22e00b1100 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-08T19:25:01.471 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.417+0000 7f22e4c8f700 1 -- 172.21.15.143:0/2196754278 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22e00cad70 con 0x7f22e0159a20 2023-11-08T19:25:01.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.417+0000 7f22e4c8f700 1 --2- 172.21.15.143:0/2196754278 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f22e0159a20 0x7f22e006ce00 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f22d40088f0 tx=0x7f22d400d330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e7bdc7d91a45279 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:01.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.417+0000 7ff7f81a8700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff7f00c2bd0 0x7ff7f00cdb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:01.473 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.417+0000 7ff7f81a8700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff7f00b1810 0x7ff7f006cf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:01.473 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.417+0000 7ff7f81a8700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7f00ce180 con 0x7ff7f0163240 2023-11-08T19:25:01.473 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.417+0000 7ff7f81a8700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7f00c2fb0 con 0x7ff7f00c2bd0 2023-11-08T19:25:01.473 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.417+0000 7ff7f81a8700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f00c9c40 con 0x7ff7f00b1810 2023-11-08T19:25:01.473 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.417+0000 7ff7f6745700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff7f00b1810 0x7ff7f006cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:01.474 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.417+0000 7ff7f6745700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff7f00b1810 0x7ff7f006cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.143:36626/0 (socket says 172.21.15.143:36626) 2023-11-08T19:25:01.474 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.417+0000 7ff7f6745700 1 -- 172.21.15.143:0/3221226473 learned_addr learned my addr 172.21.15.143:0/3221226473 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:01.474 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.417+0000 7ff7f5f44700 1 --2- 172.21.15.143:0/3221226473 >> v2:172.21.15.143:3300/0 conn(0x7ff7f00c2bd0 0x7ff7f00cdb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:01.474 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.417+0000 7ff7f4f42700 1 -- 172.21.15.143:0/3221226473 <== mon.1 v1:172.21.15.174:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2640508475 0 0) 0x7ff7f00ce180 con 0x7ff7f0163240 2023-11-08T19:25:01.474 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.418+0000 7ff7f5f44700 1 -- 172.21.15.143:0/3221226473 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff7f00b1810 msgr2=0x7ff7f006cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:01.475 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.418+0000 7ff7f5f44700 1 --2- 172.21.15.143:0/3221226473 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff7f00b1810 0x7ff7f006cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:01.475 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.418+0000 7ff7f5f44700 1 -- 172.21.15.143:0/3221226473 >> v1:172.21.15.174:6789/0 conn(0x7ff7f0163240 legacy=0x7ff7f00b1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:01.475 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.417+0000 7fa5b67fc700 1 -- 172.21.15.143:0/1695703496 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa5b001a070 con 0x7fa5c00c9c40 2023-11-08T19:25:01.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.418+0000 7fa5b67fc700 1 --2- 172.21.15.143:0/1695703496 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa5a406af50 0x7fa5a406d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:01.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.418+0000 7ff7f5f44700 1 -- 172.21.15.143:0/3221226473 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7f0070f90 con 0x7ff7f00c2bd0 2023-11-08T19:25:01.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.418+0000 7ff7f5f44700 1 --2- 172.21.15.143:0/3221226473 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff7f00c2bd0 0x7ff7f00cdb30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff7e0009840 tx=0x7ff7e0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df213d650209e38f server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:01.477 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.418+0000 7fdb63fff700 1 -- 172.21.15.143:0/2489207518 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb7001e070 con 0x7fdb7c0c9c40 2023-11-08T19:25:01.477 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.418+0000 7fdb63fff700 1 -- 172.21.15.143:0/2489207518 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb70004550 con 0x7fdb7c0c9c40 2023-11-08T19:25:01.477 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.418+0000 7fdb63fff700 1 -- 172.21.15.143:0/2489207518 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb70003ad0 con 0x7fdb7c0c9c40 2023-11-08T19:25:01.478 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.418+0000 7fa5b67fc700 1 -- 172.21.15.143:0/1695703496 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa5b008a1e0 con 0x7fa5c00c9c40 2023-11-08T19:25:01.478 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.418+0000 7fdb81b53700 1 -- 172.21.15.143:0/2489207518 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb7c16a710 con 0x7fdb7c0c9c40 2023-11-08T19:25:01.479 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.418+0000 7fdb81b53700 1 -- 172.21.15.143:0/2489207518 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb7c0b3490 con 0x7fdb7c0c9c40 2023-11-08T19:25:01.479 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.418+0000 7fa5c5c29700 1 --2- 172.21.15.143:0/1695703496 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa5a406af50 0x7fa5a406d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:01.479 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.419+0000 7fa5c5c29700 1 --2- 172.21.15.143:0/1695703496 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa5a406af50 0x7fa5a406d410 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc00d610 tx=0x7fa5bc00f040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:01.480 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.419+0000 7fdb81b53700 1 -- 172.21.15.143:0/2489207518 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb7c0b32f0 con 0x7fdb7c0c9c40 2023-11-08T19:25:01.480 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.420+0000 7fa5f9ffb700 1 -- 172.21.15.143:0/785107892 <== mon.2 v2:172.21.15.174:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa600053ab0 con 0x7fa6040b1810 2023-11-08T19:25:01.481 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.421+0000 7f1436ffd700 1 -- 172.21.15.143:0/2524222030 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1440017080 con 0x7f1448165c50 2023-11-08T19:25:01.481 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.421+0000 7f22df7fe700 1 -- 172.21.15.143:0/2196754278 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d4010070 con 0x7f22e0159a20 2023-11-08T19:25:01.482 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.421+0000 7f22df7fe700 1 -- 172.21.15.143:0/2196754278 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22d400d990 con 0x7f22e0159a20 2023-11-08T19:25:01.482 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.421+0000 7f22df7fe700 1 -- 172.21.15.143:0/2196754278 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d400dd00 con 0x7f22e0159a20 2023-11-08T19:25:01.482 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.422+0000 7f22e6ef3700 1 -- 172.21.15.143:0/2196754278 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f22e0159a20 msgr2=0x7f22e006ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:01.482 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.422+0000 7f22e6ef3700 1 --2- 172.21.15.143:0/2196754278 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f22e0159a20 0x7f22e006ce00 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f22d40088f0 tx=0x7f22d400d330 comp rx=0 tx=0).stop 2023-11-08T19:25:01.483 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.423+0000 7fa5b67fc700 1 -- 172.21.15.143:0/1695703496 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa5b0055400 con 0x7fa5c00c9c40 2023-11-08T19:25:01.483 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.424+0000 7fdb63fff700 1 -- 172.21.15.143:0/2489207518 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fdb70003c30 con 0x7fdb7c0c9c40 2023-11-08T19:25:01.483 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.425+0000 7fdb63fff700 1 --2- 172.21.15.143:0/2489207518 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fdb6806af80 0x7fdb6806d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:01.484 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.425+0000 7fdb63fff700 1 -- 172.21.15.143:0/2489207518 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdb7008a7a0 con 0x7fdb7c0c9c40 2023-11-08T19:25:01.484 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.426+0000 7fdb7b7fe700 1 --2- 172.21.15.143:0/2489207518 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fdb6806af80 0x7fdb6806d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:01.486 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.426+0000 7f9418684700 1 Processor -- start 2023-11-08T19:25:01.486 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.426+0000 7f9418684700 1 -- start start 2023-11-08T19:25:01.487 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.427+0000 7f9418684700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f941006e4d0 0x7f9410070e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:01.487 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.427+0000 7f9418684700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9410071d50 con 0x7f94100b1100 2023-11-08T19:25:01.487 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.427+0000 7f9418684700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9410071790 con 0x7f941015dad0 2023-11-08T19:25:01.488 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.427+0000 7f9418684700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9410071a70 con 0x7f941006e4d0 2023-11-08T19:25:01.488 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.427+0000 7f9416c21700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f94100b1100 0x7f941015deb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:45158/0 (socket says 172.21.15.143:45158) 2023-11-08T19:25:01.488 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.427+0000 7f9416c21700 1 -- 172.21.15.143:0/1008763271 learned_addr learned my addr 172.21.15.143:0/1008763271 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:25:01.488 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.428+0000 7f941541e700 1 -- 172.21.15.143:0/1008763271 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1004555821 0 0) 0x7f9410071d50 con 0x7f94100b1100 2023-11-08T19:25:01.488 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.428+0000 7f941541e700 1 -- 172.21.15.143:0/1008763271 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f93fc0035b0 con 0x7f94100b1100 2023-11-08T19:25:01.489 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.429+0000 7f941541e700 1 -- 172.21.15.143:0/1008763271 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1076159006 0 0) 0x7f93fc0035b0 con 0x7f94100b1100 2023-11-08T19:25:01.489 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.429+0000 7f941541e700 1 -- 172.21.15.143:0/1008763271 >> v2:172.21.15.174:3300/0 conn(0x7f941006e4d0 msgr2=0x7f9410070e60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-08T19:25:01.489 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.429+0000 7f941541e700 1 --2- 172.21.15.143:0/1008763271 >> v2:172.21.15.174:3300/0 conn(0x7f941006e4d0 0x7f9410070e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:01.489 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.429+0000 7f941541e700 1 -- 172.21.15.143:0/1008763271 >> v1:172.21.15.174:6789/0 conn(0x7f941015dad0 legacy=0x7f94101620a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:01.490 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.429+0000 7f941541e700 1 -- 172.21.15.143:0/1008763271 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94101618a0 con 0x7f94100b1100 2023-11-08T19:25:01.490 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.429+0000 7f941541e700 1 -- 172.21.15.143:0/1008763271 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4147756303 0 0) 0x7f94080019f0 con 0x7f94100b1100 2023-11-08T19:25:01.490 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.429+0000 7f941541e700 1 -- 172.21.15.143:0/1008763271 >> v1:172.21.15.143:6789/0 conn(0x7f94100b1100 legacy=0x7f941015deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:01.490 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.429+0000 7f941541e700 1 --2- 172.21.15.143:0/1008763271 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f93fc003a00 0x7f93fc003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:01.490 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.429+0000 7f941541e700 1 --2- 172.21.15.143:0/1008763271 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f93fc004850 0x7f93fc004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:01.491 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.429+0000 7f941541e700 1 --2- 172.21.15.143:0/1008763271 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f93fc005330 0x7f93fc00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:01.491 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.429+0000 7f941541e700 1 -- 172.21.15.143:0/1008763271 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9410071a70 con 0x7f93fc005330 2023-11-08T19:25:01.491 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.429+0000 7f941541e700 1 -- 172.21.15.143:0/1008763271 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93fc00c280 con 0x7f93fc004850 2023-11-08T19:25:01.491 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.429+0000 7f941541e700 1 -- 172.21.15.143:0/1008763271 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93fc00c440 con 0x7f93fc003a00 2023-11-08T19:25:01.492 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.429+0000 7f9416c21700 1 --2- 172.21.15.143:0/1008763271 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f93fc005330 0x7f93fc00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:01.492 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.429+0000 7f9416c21700 1 -- 172.21.15.143:0/1008763271 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f93fc003a00 msgr2=0x7f93fc003e00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-08T19:25:01.492 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.429+0000 7f9416c21700 1 --2- 172.21.15.143:0/1008763271 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f93fc003a00 0x7f93fc003e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:01.492 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.429+0000 7f9416c21700 1 -- 172.21.15.143:0/1008763271 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f93fc004850 msgr2=0x7f93fc004cd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-08T19:25:01.492 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.429+0000 7f9416c21700 1 --2- 172.21.15.143:0/1008763271 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f93fc004850 0x7f93fc004cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:01.493 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.429+0000 7f9416c21700 1 -- 172.21.15.143:0/1008763271 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94101618a0 con 0x7f93fc005330 2023-11-08T19:25:01.493 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.429+0000 7f9416c21700 1 -- 172.21.15.143:0/1008763271 reap_dead start 2023-11-08T19:25:01.493 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.429+0000 7f9416c21700 1 --2- 172.21.15.143:0/1008763271 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f93fc005330 0x7f93fc00ba10 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f940800a970 tx=0x7f93fc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=299d7afe9d76770d server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:01.493 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.430+0000 7f941541e700 1 -- 172.21.15.143:0/1008763271 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f940800c620 con 0x7f93fc005330 2023-11-08T19:25:01.494 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.430+0000 7f941541e700 1 -- 172.21.15.143:0/1008763271 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f940800c7a0 con 0x7f93fc005330 2023-11-08T19:25:01.494 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.430+0000 7f941541e700 1 -- 172.21.15.143:0/1008763271 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f940800cb50 con 0x7f93fc005330 2023-11-08T19:25:01.494 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.430+0000 7fdb7b7fe700 1 --2- 172.21.15.143:0/2489207518 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fdb6806af80 0x7fdb6806d440 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fdb6c004530 tx=0x7fdb6c00f040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:01.495 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.430+0000 7f9418684700 1 -- 172.21.15.143:0/1008763271 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f93fc005330 msgr2=0x7f93fc00ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:01.495 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.430+0000 7f9418684700 1 --2- 172.21.15.143:0/1008763271 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f93fc005330 0x7f93fc00ba10 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f940800a970 tx=0x7f93fc005880 comp rx=0 tx=0).stop 2023-11-08T19:25:01.496 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.421+0000 7ff7f4f42700 1 -- 172.21.15.143:0/3221226473 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7e0011830 con 0x7ff7f00c2bd0 2023-11-08T19:25:01.496 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.421+0000 7ff7f4f42700 1 -- 172.21.15.143:0/3221226473 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7e0011990 con 0x7ff7f00c2bd0 2023-11-08T19:25:01.496 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.421+0000 7ff7f4f42700 1 -- 172.21.15.143:0/3221226473 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7e0011d00 con 0x7ff7f00c2bd0 2023-11-08T19:25:01.496 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.422+0000 7ff7f81a8700 1 -- 172.21.15.143:0/3221226473 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff7f00c2bd0 msgr2=0x7ff7f00cdb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:01.497 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.422+0000 7ff7f81a8700 1 --2- 172.21.15.143:0/3221226473 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff7f00c2bd0 0x7ff7f00cdb30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff7e0009840 tx=0x7ff7e0009b50 comp rx=0 tx=0).stop 2023-11-08T19:25:01.497 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.422+0000 7ff7f81a8700 1 -- 172.21.15.143:0/3221226473 shutdown_connections 2023-11-08T19:25:01.497 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.422+0000 7ff7f81a8700 1 --2- 172.21.15.143:0/3221226473 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff7f00b1810 0x7ff7f006cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:01.497 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.432+0000 7fdb63fff700 1 -- 172.21.15.143:0/2489207518 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdb70055a50 con 0x7fdb7c0c9c40 2023-11-08T19:25:01.499 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.422+0000 7ff7f81a8700 1 --2- 172.21.15.143:0/3221226473 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff7f00c2bd0 0x7ff7f00cdb30 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:01.499 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.422+0000 7ff7f81a8700 1 -- 172.21.15.143:0/3221226473 >> 172.21.15.143:0/3221226473 conn(0x7ff7f006b4c0 msgr2=0x7ff7f00cf680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:01.499 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.422+0000 7ff7f81a8700 1 -- 172.21.15.143:0/3221226473 shutdown_connections 2023-11-08T19:25:01.500 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.422+0000 7ff7f81a8700 1 -- 172.21.15.143:0/3221226473 wait complete. 2023-11-08T19:25:01.500 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.424+0000 7ff7f81a8700 1 Processor -- start 2023-11-08T19:25:01.500 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.424+0000 7ff7f81a8700 1 -- start start 2023-11-08T19:25:01.500 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.425+0000 7ff7f81a8700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff7f00b1810 0x7ff7f01febc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:01.500 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.425+0000 7ff7f81a8700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff7f00c2bd0 0x7ff7f00c03a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:01.501 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.425+0000 7ff7f81a8700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff7f0163240 0x7ff7f00c08e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:01.501 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.425+0000 7ff7f81a8700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f00c0e20 con 0x7ff7f0163240 2023-11-08T19:25:01.501 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.425+0000 7ff7f81a8700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f00c2150 con 0x7ff7f00c2bd0 2023-11-08T19:25:01.501 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.425+0000 7ff7f81a8700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f00c10b0 con 0x7ff7f00b1810 2023-11-08T19:25:01.502 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.425+0000 7ff7f6745700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff7f0163240 0x7ff7f00c08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:01.502 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.425+0000 7ff7f6745700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff7f0163240 0x7ff7f00c08e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48998/0 (socket says 172.21.15.143:48998) 2023-11-08T19:25:01.502 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.425+0000 7ff7f6745700 1 -- 172.21.15.143:0/989353387 learned_addr learned my addr 172.21.15.143:0/989353387 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:01.502 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.425+0000 7ff7f6745700 1 -- 172.21.15.143:0/989353387 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff7f00b1810 msgr2=0x7ff7f01febc0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-08T19:25:01.503 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.425+0000 7ff7f6745700 1 --2- 172.21.15.143:0/989353387 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff7f00b1810 0x7ff7f01febc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:01.503 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.426+0000 7ff7f6745700 1 -- 172.21.15.143:0/989353387 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff7f00c2bd0 msgr2=0x7ff7f00c03a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-08T19:25:01.503 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.426+0000 7ff7f6745700 1 --2- 172.21.15.143:0/989353387 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff7f00c2bd0 0x7ff7f00c03a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:01.503 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.426+0000 7ff7f6745700 1 -- 172.21.15.143:0/989353387 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7e00094a0 con 0x7ff7f0163240 2023-11-08T19:25:01.504 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.426+0000 7ff7f6745700 1 --2- 172.21.15.143:0/989353387 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff7f0163240 0x7ff7f00c08e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff7e8007ad0 tx=0x7ff7e8007e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:01.504 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.426+0000 7ff7e67fc700 1 -- 172.21.15.143:0/989353387 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7e8003c20 con 0x7ff7f0163240 2023-11-08T19:25:01.504 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.426+0000 7ff7e67fc700 1 -- 172.21.15.143:0/989353387 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7e8003da0 con 0x7ff7f0163240 2023-11-08T19:25:01.504 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.426+0000 7ff7e67fc700 1 -- 172.21.15.143:0/989353387 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7e800dbf0 con 0x7ff7f0163240 2023-11-08T19:25:01.504 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.427+0000 7ff7f81a8700 1 -- 172.21.15.143:0/989353387 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7f00c1390 con 0x7ff7f0163240 2023-11-08T19:25:01.505 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.427+0000 7ff7f81a8700 1 -- 172.21.15.143:0/989353387 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7f00c1950 con 0x7ff7f0163240 2023-11-08T19:25:01.505 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.429+0000 7ff7e67fc700 1 -- 172.21.15.143:0/989353387 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff7e800dd50 con 0x7ff7f0163240 2023-11-08T19:25:01.505 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.430+0000 7ff7e67fc700 1 --2- 172.21.15.143:0/989353387 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7ff7d406af30 0x7ff7d406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:01.505 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.430+0000 7ff7e67fc700 1 -- 172.21.15.143:0/989353387 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff7e80cd5a0 con 0x7ff7f0163240 2023-11-08T19:25:01.505 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.430+0000 7ff7f81a8700 1 -- 172.21.15.143:0/989353387 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7f00b2110 con 0x7ff7f0163240 2023-11-08T19:25:01.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.435+0000 7f9418684700 1 -- 172.21.15.143:0/1008763271 shutdown_connections 2023-11-08T19:25:01.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.435+0000 7f9418684700 1 --2- 172.21.15.143:0/1008763271 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f93fc005330 0x7f93fc00ba10 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:01.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.435+0000 7f9418684700 1 -- 172.21.15.143:0/1008763271 >> 172.21.15.143:0/1008763271 conn(0x7f941006b060 msgr2=0x7f941006b470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:01.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.435+0000 7f9418684700 1 -- 172.21.15.143:0/1008763271 shutdown_connections 2023-11-08T19:25:01.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.435+0000 7f9418684700 1 -- 172.21.15.143:0/1008763271 wait complete. 2023-11-08T19:25:01.508 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.437+0000 7f9418684700 1 Processor -- start 2023-11-08T19:25:01.508 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.437+0000 7f9418684700 1 -- start start 2023-11-08T19:25:01.508 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.439+0000 7f9418684700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f93fc005330 0x7f9410160b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:01.508 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.439+0000 7f9418684700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94100c7fd0 0x7f94100c8450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:01.508 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.439+0000 7f9418684700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f94101ffc20 0x7f94102000a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:01.509 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.439+0000 7f9418684700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94100c6a50 con 0x7f94100c7fd0 2023-11-08T19:25:01.509 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.439+0000 7f9418684700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94100a8550 con 0x7f94101ffc20 2023-11-08T19:25:01.509 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.439+0000 7f9418684700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94100c8990 con 0x7f93fc005330 2023-11-08T19:25:01.509 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.439+0000 7f9415c1f700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94100c7fd0 0x7f94100c8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:01.509 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.439+0000 7f9415c1f700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94100c7fd0 0x7f94100c8450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49014/0 (socket says 172.21.15.143:49014) 2023-11-08T19:25:01.510 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.439+0000 7f9415c1f700 1 -- 172.21.15.143:0/1244684595 learned_addr learned my addr 172.21.15.143:0/1244684595 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:01.510 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.435+0000 7ff7f5f44700 1 --2- 172.21.15.143:0/989353387 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7ff7d406af30 0x7ff7d406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:01.510 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.435+0000 7ff7e67fc700 1 -- 172.21.15.143:0/989353387 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff7e80985b0 con 0x7ff7f0163240 2023-11-08T19:25:01.510 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.437+0000 7ff7f5f44700 1 --2- 172.21.15.143:0/989353387 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7ff7d406af30 0x7ff7d406d3f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff7f00c5d70 tx=0x7ff7e0008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:01.511 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.439+0000 7f22e6ef3700 1 -- 172.21.15.143:0/2196754278 shutdown_connections 2023-11-08T19:25:01.511 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.439+0000 7f22e6ef3700 1 --2- 172.21.15.143:0/2196754278 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f22e00b1810 0x7f22e00ca100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:01.511 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.439+0000 7f22e6ef3700 1 --2- 172.21.15.143:0/2196754278 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f22e0159a20 0x7f22e006ce00 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:01.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.439+0000 7f22e6ef3700 1 -- 172.21.15.143:0/2196754278 >> 172.21.15.143:0/2196754278 conn(0x7f22e006b060 msgr2=0x7f22e006b470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:01.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.439+0000 7f9415c1f700 1 -- 172.21.15.143:0/1244684595 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f93fc005330 msgr2=0x7f9410160b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:01.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.439+0000 7f9415c1f700 1 --2- 172.21.15.143:0/1244684595 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f93fc005330 0x7f9410160b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:01.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.439+0000 7f9415c1f700 1 -- 172.21.15.143:0/1244684595 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f94101ffc20 msgr2=0x7f94102000a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:01.513 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.439+0000 7f9415c1f700 1 --2- 172.21.15.143:0/1244684595 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f94101ffc20 0x7f94102000a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:01.513 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.439+0000 7f9415c1f700 1 -- 172.21.15.143:0/1244684595 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94102005e0 con 0x7f94100c7fd0 2023-11-08T19:25:01.513 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.440+0000 7f22e6ef3700 1 -- 172.21.15.143:0/2196754278 shutdown_connections 2023-11-08T19:25:01.514 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.440+0000 7f9415c1f700 1 --2- 172.21.15.143:0/1244684595 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94100c7fd0 0x7f94100c8450 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f940000b3f0 tx=0x7f940000b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:01.514 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.440+0000 7f22e6ef3700 1 -- 172.21.15.143:0/2196754278 wait complete. 2023-11-08T19:25:01.514 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.442+0000 7f22e6ef3700 1 Processor -- start 2023-11-08T19:25:01.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.440+0000 7f9406ffd700 1 -- 172.21.15.143:0/1244684595 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9400011830 con 0x7f94100c7fd0 2023-11-08T19:25:01.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.440+0000 7f9406ffd700 1 -- 172.21.15.143:0/1244684595 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94000119b0 con 0x7f94100c7fd0 2023-11-08T19:25:01.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.440+0000 7f9406ffd700 1 -- 172.21.15.143:0/1244684595 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9400011d60 con 0x7f94100c7fd0 2023-11-08T19:25:01.516 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.440+0000 7f9418684700 1 -- 172.21.15.143:0/1244684595 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94100c8be0 con 0x7f94100c7fd0 2023-11-08T19:25:01.516 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.440+0000 7f9418684700 1 -- 172.21.15.143:0/1244684595 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94100c91a0 con 0x7f94100c7fd0 2023-11-08T19:25:01.518 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.442+0000 7f22e6ef3700 1 -- start start 2023-11-08T19:25:01.519 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.443+0000 7f22e6ef3700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f22e00b1810 0x7f22e01fac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:01.519 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.443+0000 7f22e6ef3700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f22e0159a20 0x7f22e006d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:01.519 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.443+0000 7f22e6ef3700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f22e0159e00 0x7f22e006dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:01.519 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.443+0000 7f22e6ef3700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e01fb170 con 0x7f22e0159a20 2023-11-08T19:25:01.520 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.443+0000 7f22e6ef3700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e015d690 con 0x7f22e00b1810 2023-11-08T19:25:01.520 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.443+0000 7f22e6ef3700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22e02024d0 con 0x7f22e0159e00 2023-11-08T19:25:01.520 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.444+0000 7f22dffff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f22e0159a20 0x7f22e006d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:01.520 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.444+0000 7f22dffff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f22e0159a20 0x7f22e006d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49024/0 (socket says 172.21.15.143:49024) 2023-11-08T19:25:01.520 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.444+0000 7f22dffff700 1 -- 172.21.15.143:0/3602248697 learned_addr learned my addr 172.21.15.143:0/3602248697 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:01.521 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.444+0000 7f22e5490700 1 --2- 172.21.15.143:0/3602248697 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f22e0159e00 0x7f22e006dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:01.521 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.444+0000 7f22e4c8f700 1 --2- 172.21.15.143:0/3602248697 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f22e00b1810 0x7f22e01fac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:01.521 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.444+0000 7f22dffff700 1 -- 172.21.15.143:0/3602248697 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f22e0159e00 msgr2=0x7f22e006dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:01.521 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.444+0000 7f22dffff700 1 --2- 172.21.15.143:0/3602248697 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f22e0159e00 0x7f22e006dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:01.522 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.444+0000 7f22dffff700 1 -- 172.21.15.143:0/3602248697 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f22e00b1810 msgr2=0x7f22e01fac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:01.522 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.444+0000 7f22dffff700 1 --2- 172.21.15.143:0/3602248697 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f22e00b1810 0x7f22e01fac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:01.522 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.444+0000 7f22dffff700 1 -- 172.21.15.143:0/3602248697 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22d40085d0 con 0x7f22e0159a20 2023-11-08T19:25:01.522 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.445+0000 7f22dffff700 1 --2- 172.21.15.143:0/3602248697 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f22e0159a20 0x7f22e006d6b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f22d800ba50 tx=0x7f22d800be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:01.523 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.445+0000 7f22dd7fa700 1 -- 172.21.15.143:0/3602248697 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d800cab0 con 0x7f22e0159a20 2023-11-08T19:25:01.523 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.445+0000 7f22dd7fa700 1 -- 172.21.15.143:0/3602248697 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22d800cc30 con 0x7f22e0159a20 2023-11-08T19:25:01.523 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.445+0000 7f22dd7fa700 1 -- 172.21.15.143:0/3602248697 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d8014440 con 0x7f22e0159a20 2023-11-08T19:25:01.523 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.446+0000 7f22e6ef3700 1 -- 172.21.15.143:0/3602248697 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22e0201870 con 0x7f22e0159a20 2023-11-08T19:25:01.523 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.446+0000 7f22e6ef3700 1 -- 172.21.15.143:0/3602248697 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22e0201e80 con 0x7f22e0159a20 2023-11-08T19:25:01.524 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.447+0000 7f22e6ef3700 1 -- 172.21.15.143:0/3602248697 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22e004eee0 con 0x7f22e0159a20 2023-11-08T19:25:01.524 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.448+0000 7f22dd7fa700 1 -- 172.21.15.143:0/3602248697 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f22d80145a0 con 0x7f22e0159a20 2023-11-08T19:25:01.524 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.448+0000 7f22dd7fa700 1 --2- 172.21.15.143:0/3602248697 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f22cc06af00 0x7f22cc06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:01.525 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.449+0000 7f22e4c8f700 1 --2- 172.21.15.143:0/3602248697 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f22cc06af00 0x7f22cc06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:01.525 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.449+0000 7f22e4c8f700 1 --2- 172.21.15.143:0/3602248697 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f22cc06af00 0x7f22cc06d3c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f22d4000c00 tx=0x7f22d4012000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:01.525 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.450+0000 7f22dd7fa700 1 -- 172.21.15.143:0/3602248697 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f22d80585a0 con 0x7f22e0159a20 2023-11-08T19:25:01.526 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.440+0000 7f94f6721700 1 Processor -- start 2023-11-08T19:25:01.526 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.441+0000 7f94f6721700 1 -- start start 2023-11-08T19:25:01.526 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.442+0000 7f94f6721700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f94f00b1370 0x7f94f016eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:01.527 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.442+0000 7f94f6721700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f94f016e000 0x7f94f006d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:01.527 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.442+0000 7f94f6721700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94f016f3e0 con 0x7f94f00cda90 2023-11-08T19:25:01.527 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.442+0000 7f94f6721700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94f006d550 con 0x7f94f016e000 2023-11-08T19:25:01.527 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.442+0000 7f94f6721700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94f008a660 con 0x7f94f00b1370 2023-11-08T19:25:01.527 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.442+0000 7f94effff700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f94f00b1370 0x7f94f016eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:01.528 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.442+0000 7f94effff700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f94f00b1370 0x7f94f016eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.143:36656/0 (socket says 172.21.15.143:36656) 2023-11-08T19:25:01.528 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.442+0000 7f94effff700 1 -- 172.21.15.143:0/4216087546 learned_addr learned my addr 172.21.15.143:0/4216087546 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:01.528 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.444+0000 7f94eeffd700 1 -- 172.21.15.143:0/4216087546 <== mon.1 v1:172.21.15.174:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2940158075 0 0) 0x7f94e8001490 con 0x7f94f00cda90 2023-11-08T19:25:01.528 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.452+0000 7f94eeffd700 1 -- 172.21.15.143:0/4216087546 --> v1:172.21.15.174:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94d8003590 con 0x7f94f00cda90 2023-11-08T19:25:01.528 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.443+0000 7f94f4cbe700 1 --2- 172.21.15.143:0/4216087546 >> v2:172.21.15.174:3300/0 conn(0x7f94f016e000 0x7f94f006d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:01.529 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.452+0000 7f94effff700 1 -- 172.21.15.143:0/4216087546 >> v2:172.21.15.174:3300/0 conn(0x7f94f016e000 msgr2=0x7f94f006d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:01.529 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.452+0000 7f94effff700 1 --2- 172.21.15.143:0/4216087546 >> v2:172.21.15.174:3300/0 conn(0x7f94f016e000 0x7f94f006d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:01.529 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.452+0000 7f94effff700 1 -- 172.21.15.143:0/4216087546 >> v1:172.21.15.174:6789/0 conn(0x7f94f00cda90 legacy=0x7f94f016d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:01.529 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.452+0000 7f94effff700 1 -- 172.21.15.143:0/4216087546 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94f00ce200 con 0x7f94f00b1370 2023-11-08T19:25:01.529 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.453+0000 7f94effff700 1 --2- 172.21.15.143:0/4216087546 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f94f00b1370 0x7f94f016eea0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f94e0013b70 tx=0x7f94e0013e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc768daca4c8501c server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:01.530 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.453+0000 7f94eeffd700 1 -- 172.21.15.143:0/4216087546 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e000a2e0 con 0x7f94f00b1370 2023-11-08T19:25:01.530 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.453+0000 7f94eeffd700 1 -- 172.21.15.143:0/4216087546 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94e000a440 con 0x7f94f00b1370 2023-11-08T19:25:01.530 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.453+0000 7f94eeffd700 1 -- 172.21.15.143:0/4216087546 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e000a7b0 con 0x7f94f00b1370 2023-11-08T19:25:01.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.453+0000 7f9406ffd700 1 -- 172.21.15.143:0/1244684595 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f940001a400 con 0x7f94100c7fd0 2023-11-08T19:25:01.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.454+0000 7f9406ffd700 1 --2- 172.21.15.143:0/1244684595 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f93f406af50 0x7f93f406d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:01.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.454+0000 7f9406ffd700 1 -- 172.21.15.143:0/1244684595 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f94000893b0 con 0x7f94100c7fd0 2023-11-08T19:25:01.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.454+0000 7f9416420700 1 --2- 172.21.15.143:0/1244684595 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f93f406af50 0x7f93f406d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:01.532 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.454+0000 7f9416420700 1 --2- 172.21.15.143:0/1244684595 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f93f406af50 0x7f93f406d410 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f940c007fd0 tx=0x7f940c007a20 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:01.532 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.455+0000 7f94f6721700 1 -- 172.21.15.143:0/4216087546 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f94f00b1370 msgr2=0x7f94f016eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:01.532 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.455+0000 7f94f6721700 1 --2- 172.21.15.143:0/4216087546 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f94f00b1370 0x7f94f016eea0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f94e0013b70 tx=0x7f94e0013e80 comp rx=0 tx=0).stop 2023-11-08T19:25:01.533 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.455+0000 7f22dd7fa700 1 -- 172.21.15.143:0/3602248697 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f22d808e060 con 0x7f22e0159a20 2023-11-08T19:25:01.533 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.456+0000 7f9418684700 1 -- 172.21.15.143:0/1244684595 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93f80052f0 con 0x7f94100c7fd0 2023-11-08T19:25:01.533 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.461+0000 7f9406ffd700 1 -- 172.21.15.143:0/1244684595 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9400015080 con 0x7f94100c7fd0 2023-11-08T19:25:01.535 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.460+0000 7f94f6721700 1 -- 172.21.15.143:0/4216087546 shutdown_connections 2023-11-08T19:25:01.535 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.460+0000 7f94f6721700 1 --2- 172.21.15.143:0/4216087546 >> v2:172.21.15.174:3300/0 conn(0x7f94f016e000 0x7f94f006d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:01.535 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.460+0000 7f94f6721700 1 --2- 172.21.15.143:0/4216087546 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f94f00b1370 0x7f94f016eea0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:01.535 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.460+0000 7f94f6721700 1 -- 172.21.15.143:0/4216087546 >> 172.21.15.143:0/4216087546 conn(0x7f94f006b060 msgr2=0x7f94f006b470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:01.536 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.471+0000 7f94f6721700 1 -- 172.21.15.143:0/4216087546 shutdown_connections 2023-11-08T19:25:01.536 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.471+0000 7f94f6721700 1 -- 172.21.15.143:0/4216087546 wait complete. 2023-11-08T19:25:01.536 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.473+0000 7f94f6721700 1 Processor -- start 2023-11-08T19:25:01.536 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.473+0000 7f94f6721700 1 -- start start 2023-11-08T19:25:01.536 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.474+0000 7f94f6721700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f94f00b1370 0x7f94f0206970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:01.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.474+0000 7f94f6721700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94f016e000 0x7f94f0204bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:01.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.474+0000 7f94f6721700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f94f0206eb0 0x7f94f0205100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:01.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.474+0000 7f94f6721700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94f016ad90 con 0x7f94f016e000 2023-11-08T19:25:01.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.474+0000 7f94f6721700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94f0207c80 con 0x7f94f00b1370 2023-11-08T19:25:01.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.474+0000 7f94f6721700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94f02073e0 con 0x7f94f0206eb0 2023-11-08T19:25:01.538 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.475+0000 7f94ef7fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94f016e000 0x7f94f0204bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:01.538 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.475+0000 7f94ef7fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94f016e000 0x7f94f0204bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49036/0 (socket says 172.21.15.143:49036) 2023-11-08T19:25:01.538 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.475+0000 7f94ef7fe700 1 -- 172.21.15.143:0/2858259955 learned_addr learned my addr 172.21.15.143:0/2858259955 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:01.538 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.475+0000 7f94ef7fe700 1 -- 172.21.15.143:0/2858259955 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f94f0206eb0 msgr2=0x7f94f0205100 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-08T19:25:01.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.475+0000 7f94ef7fe700 1 --2- 172.21.15.143:0/2858259955 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f94f0206eb0 0x7f94f0205100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:01.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.475+0000 7f94ef7fe700 1 -- 172.21.15.143:0/2858259955 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f94f00b1370 msgr2=0x7f94f0206970 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-08T19:25:01.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.475+0000 7f94ef7fe700 1 --2- 172.21.15.143:0/2858259955 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f94f00b1370 0x7f94f0206970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:01.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.475+0000 7f94ef7fe700 1 -- 172.21.15.143:0/2858259955 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94e0013850 con 0x7f94f016e000 2023-11-08T19:25:01.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.475+0000 7f94ef7fe700 1 --2- 172.21.15.143:0/2858259955 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94f016e000 0x7f94f0204bc0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f94e8001f10 tx=0x7f94e800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:01.540 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.476+0000 7f94ecff9700 1 -- 172.21.15.143:0/2858259955 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e800ade0 con 0x7f94f016e000 2023-11-08T19:25:01.540 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.476+0000 7f94ecff9700 1 -- 172.21.15.143:0/2858259955 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94e8003a00 con 0x7f94f016e000 2023-11-08T19:25:01.540 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.476+0000 7f94ecff9700 1 -- 172.21.15.143:0/2858259955 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e8003d90 con 0x7f94f016e000 2023-11-08T19:25:01.540 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.476+0000 7f94f6721700 1 -- 172.21.15.143:0/2858259955 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94f02076c0 con 0x7f94f016e000 2023-11-08T19:25:01.540 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.476+0000 7f94f6721700 1 -- 172.21.15.143:0/2858259955 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94f01660e0 con 0x7f94f016e000 2023-11-08T19:25:01.541 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.478+0000 7f94ecff9700 1 -- 172.21.15.143:0/2858259955 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f94e80050d0 con 0x7f94f016e000 2023-11-08T19:25:01.541 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.479+0000 7f94ecff9700 1 --2- 172.21.15.143:0/2858259955 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f94d406afa0 0x7f94d406d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:01.541 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.479+0000 7f94ecff9700 1 -- 172.21.15.143:0/2858259955 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f94e8089680 con 0x7f94f016e000 2023-11-08T19:25:01.541 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.486+0000 7f94f6721700 1 -- 172.21.15.143:0/2858259955 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94cc0052f0 con 0x7f94f016e000 2023-11-08T19:25:01.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.491+0000 7f94effff700 1 --2- 172.21.15.143:0/2858259955 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f94d406afa0 0x7f94d406d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:01.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.496+0000 7f94ecff9700 1 -- 172.21.15.143:0/2858259955 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f94e80547a0 con 0x7f94f016e000 2023-11-08T19:25:01.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.533+0000 7f94effff700 1 --2- 172.21.15.143:0/2858259955 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f94d406afa0 0x7f94d406d460 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f94e001e040 tx=0x7f94e000c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:01.825 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.824+0000 7f144cff0700 1 -- 172.21.15.143:0/2524222030 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f14480d0490 con 0x7f1448165c50 2023-11-08T19:25:01.826 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.824+0000 7f1436ffd700 1 -- 172.21.15.143:0/2524222030 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1440053c70 con 0x7f1448165c50 2023-11-08T19:25:01.829 INFO:teuthology.orchestra.run.smithi143.stdout:55834574854 2023-11-08T19:25:01.829 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.828+0000 7f144cff0700 1 -- 172.21.15.143:0/2524222030 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f142806afd0 msgr2=0x7f142806d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:01.829 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.828+0000 7f144cff0700 1 --2- 172.21.15.143:0/2524222030 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f142806afd0 0x7f142806d490 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1438006d30 tx=0x7f14380092c0 comp rx=0 tx=0).stop 2023-11-08T19:25:01.830 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.828+0000 7f144cff0700 1 -- 172.21.15.143:0/2524222030 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1448165c50 msgr2=0x7f144815d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:01.830 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.828+0000 7f144cff0700 1 --2- 172.21.15.143:0/2524222030 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1448165c50 0x7f144815d1d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f144000c780 tx=0x7f144000ca90 comp rx=0 tx=0).stop 2023-11-08T19:25:01.831 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.828+0000 7f144cff0700 1 -- 172.21.15.143:0/2524222030 shutdown_connections 2023-11-08T19:25:01.831 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.828+0000 7f144cff0700 1 --2- 172.21.15.143:0/2524222030 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f142806afd0 0x7f142806d490 secure :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1438006d30 tx=0x7f14380092c0 comp rx=0 tx=0).stop 2023-11-08T19:25:01.831 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.828+0000 7f144cff0700 1 --2- 172.21.15.143:0/2524222030 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f14480b1100 0x7f1448166060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:01.831 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.828+0000 7f144cff0700 1 --2- 172.21.15.143:0/2524222030 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1448165c50 0x7f144815d1d0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:01.831 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.828+0000 7f144cff0700 1 --2- 172.21.15.143:0/2524222030 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1448162060 0x7f14481665a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:01.832 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.828+0000 7f144cff0700 1 -- 172.21.15.143:0/2524222030 >> 172.21.15.143:0/2524222030 conn(0x7f144806b4c0 msgr2=0x7f144815f210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:01.832 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.829+0000 7f144cff0700 1 -- 172.21.15.143:0/2524222030 shutdown_connections 2023-11-08T19:25:01.832 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.829+0000 7f144cff0700 1 -- 172.21.15.143:0/2524222030 wait complete. 2023-11-08T19:25:01.832 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.831+0000 7fa60b72d700 1 -- 172.21.15.143:0/785107892 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fa5e4005870 con 0x7fa6040b1810 2023-11-08T19:25:01.833 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.832+0000 7fa5c7e8d700 1 -- 172.21.15.143:0/1695703496 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fa5c0067c00 con 0x7fa5c00c9c40 2023-11-08T19:25:01.833 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.832+0000 7fa5b67fc700 1 -- 172.21.15.143:0/1695703496 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa5b0058a20 con 0x7fa5c00c9c40 2023-11-08T19:25:01.834 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.832+0000 7fa5f9ffb700 1 -- 172.21.15.143:0/785107892 <== mon.2 v2:172.21.15.174:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa600017030 con 0x7fa6040b1810 2023-11-08T19:25:01.834 INFO:teuthology.orchestra.run.smithi143.stdout:55834574855 2023-11-08T19:25:01.840 INFO:teuthology.orchestra.run.smithi143.stdout:55834574854 2023-11-08T19:25:01.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.836+0000 7fa60b72d700 1 -- 172.21.15.143:0/785107892 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa5f006b010 msgr2=0x7fa5f006d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:01.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.836+0000 7fa60b72d700 1 --2- 172.21.15.143:0/785107892 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa5f006b010 0x7fa5f006d4d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa6040c7910 tx=0x7fa5f4007a20 comp rx=0 tx=0).stop 2023-11-08T19:25:01.841 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.836+0000 7fa60b72d700 1 -- 172.21.15.143:0/785107892 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa6040b1810 msgr2=0x7fa6040b46e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:01.841 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.836+0000 7fa60b72d700 1 --2- 172.21.15.143:0/785107892 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa6040b1810 0x7fa6040b46e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa60000c520 tx=0x7fa60000c830 comp rx=0 tx=0).stop 2023-11-08T19:25:01.841 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.836+0000 7fa60b72d700 1 -- 172.21.15.143:0/785107892 shutdown_connections 2023-11-08T19:25:01.841 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.836+0000 7fa60b72d700 1 --2- 172.21.15.143:0/785107892 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa5f006b010 0x7fa5f006d4d0 secure :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa6040c7910 tx=0x7fa5f4007a20 comp rx=0 tx=0).stop 2023-11-08T19:25:01.841 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.836+0000 7fa60b72d700 1 --2- 172.21.15.143:0/785107892 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa6040b1810 0x7fa6040b46e0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:01.842 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.836+0000 7fa60b72d700 1 --2- 172.21.15.143:0/785107892 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa604159cf0 0x7fa6040c8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:01.842 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.836+0000 7fa60b72d700 1 --2- 172.21.15.143:0/785107892 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6040cae60 0x7fa60415aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:01.842 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.836+0000 7fa60b72d700 1 -- 172.21.15.143:0/785107892 >> 172.21.15.143:0/785107892 conn(0x7fa60406b4c0 msgr2=0x7fa6040aed40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:01.842 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.839+0000 7fa60b72d700 1 -- 172.21.15.143:0/785107892 shutdown_connections 2023-11-08T19:25:01.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.839+0000 7fdb81b53700 1 -- 172.21.15.143:0/2489207518 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fdb7c0b32f0 con 0x7fdb7c0c9c40 2023-11-08T19:25:01.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.839+0000 7fa60b72d700 1 -- 172.21.15.143:0/785107892 wait complete. 2023-11-08T19:25:01.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.839+0000 7fa5c7e8d700 1 -- 172.21.15.143:0/1695703496 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa5a406af50 msgr2=0x7fa5a406d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:01.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.839+0000 7fa5c7e8d700 1 --2- 172.21.15.143:0/1695703496 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa5a406af50 0x7fa5a406d410 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc00d610 tx=0x7fa5bc00f040 comp rx=0 tx=0).stop 2023-11-08T19:25:01.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.839+0000 7fa5c7e8d700 1 -- 172.21.15.143:0/1695703496 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa5c00c9c40 msgr2=0x7fa5c00b2bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:01.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.839+0000 7fa5c7e8d700 1 --2- 172.21.15.143:0/1695703496 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa5c00c9c40 0x7fa5c00b2bc0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa5b0000c00 tx=0x7fa5b0004750 comp rx=0 tx=0).stop 2023-11-08T19:25:01.845 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.839+0000 7fdb63fff700 1 -- 172.21.15.143:0/2489207518 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fdb7c0b32f0 con 0x7fdb7c0c9c40 2023-11-08T19:25:01.845 INFO:teuthology.orchestra.run.smithi143.stdout:55834574854 2023-11-08T19:25:01.846 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.844+0000 7fa5c7e8d700 1 -- 172.21.15.143:0/1695703496 shutdown_connections 2023-11-08T19:25:01.846 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.844+0000 7fa5c7e8d700 1 --2- 172.21.15.143:0/1695703496 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa5a406af50 0x7fa5a406d410 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:01.846 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.844+0000 7fa5c7e8d700 1 --2- 172.21.15.143:0/1695703496 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa5c0165960 0x7fa5c0089740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:01.846 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.844+0000 7fa5c7e8d700 1 --2- 172.21.15.143:0/1695703496 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa5c00b1370 0x7fa5c00c91f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:01.846 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.844+0000 7fa5c7e8d700 1 --2- 172.21.15.143:0/1695703496 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa5c00c9c40 0x7fa5c00b2bc0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:01.847 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.844+0000 7fa5c7e8d700 1 -- 172.21.15.143:0/1695703496 >> 172.21.15.143:0/1695703496 conn(0x7fa5c006b060 msgr2=0x7fa5c00ca970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:01.847 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.844+0000 7ff7f81a8700 1 -- 172.21.15.143:0/989353387 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7ff7f004eee0 con 0x7ff7f0163240 2023-11-08T19:25:01.847 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.844+0000 7ff7e67fc700 1 -- 172.21.15.143:0/989353387 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff7e805d020 con 0x7ff7f0163240 2023-11-08T19:25:01.848 INFO:teuthology.orchestra.run.smithi143.stdout:55834574854 2023-11-08T19:25:01.848 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.847+0000 7ff7f81a8700 1 -- 172.21.15.143:0/989353387 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7ff7d406af30 msgr2=0x7ff7d406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:01.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.847+0000 7ff7f81a8700 1 --2- 172.21.15.143:0/989353387 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7ff7d406af30 0x7ff7d406d3f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff7f00c5d70 tx=0x7ff7e0008030 comp rx=0 tx=0).stop 2023-11-08T19:25:01.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.847+0000 7ff7f81a8700 1 -- 172.21.15.143:0/989353387 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff7f0163240 msgr2=0x7ff7f00c08e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:01.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.847+0000 7ff7f81a8700 1 --2- 172.21.15.143:0/989353387 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff7f0163240 0x7ff7f00c08e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff7e8007ad0 tx=0x7ff7e8007e90 comp rx=0 tx=0).stop 2023-11-08T19:25:01.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.847+0000 7ff7f81a8700 1 -- 172.21.15.143:0/989353387 shutdown_connections 2023-11-08T19:25:01.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.847+0000 7ff7f81a8700 1 --2- 172.21.15.143:0/989353387 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7ff7d406af30 0x7ff7d406d3f0 secure :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff7f00c5d70 tx=0x7ff7e0008030 comp rx=0 tx=0).stop 2023-11-08T19:25:01.850 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.847+0000 7ff7f81a8700 1 --2- 172.21.15.143:0/989353387 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff7f00b1810 0x7ff7f01febc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:01.850 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.847+0000 7ff7f81a8700 1 --2- 172.21.15.143:0/989353387 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff7f0163240 0x7ff7f00c08e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:01.850 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.847+0000 7ff7f81a8700 1 --2- 172.21.15.143:0/989353387 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff7f00c2bd0 0x7ff7f00c03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:01.850 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.847+0000 7ff7f81a8700 1 -- 172.21.15.143:0/989353387 >> 172.21.15.143:0/989353387 conn(0x7ff7f006b4c0 msgr2=0x7ff7f01624f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:01.851 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.848+0000 7fa5c7e8d700 1 -- 172.21.15.143:0/1695703496 shutdown_connections 2023-11-08T19:25:01.851 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.848+0000 7fa5c7e8d700 1 -- 172.21.15.143:0/1695703496 wait complete. 2023-11-08T19:25:01.852 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.848+0000 7fdb81b53700 1 -- 172.21.15.143:0/2489207518 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fdb6806af80 msgr2=0x7fdb6806d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:01.852 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.848+0000 7fdb81b53700 1 --2- 172.21.15.143:0/2489207518 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fdb6806af80 0x7fdb6806d440 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fdb6c004530 tx=0x7fdb6c00f040 comp rx=0 tx=0).stop 2023-11-08T19:25:01.852 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.848+0000 7fdb81b53700 1 -- 172.21.15.143:0/2489207518 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdb7c0c9c40 msgr2=0x7fdb7c0c70d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:01.852 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.848+0000 7fdb81b53700 1 --2- 172.21.15.143:0/2489207518 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdb7c0c9c40 0x7fdb7c0c70d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fdb70014000 tx=0x7fdb70004750 comp rx=0 tx=0).stop 2023-11-08T19:25:01.852 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.849+0000 7fdb81b53700 1 -- 172.21.15.143:0/2489207518 shutdown_connections 2023-11-08T19:25:01.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.849+0000 7fdb81b53700 1 --2- 172.21.15.143:0/2489207518 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fdb6806af80 0x7fdb6806d440 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:01.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.849+0000 7fdb81b53700 1 --2- 172.21.15.143:0/2489207518 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdb7c0b0580 0x7fdb7c16a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:01.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.849+0000 7fdb81b53700 1 --2- 172.21.15.143:0/2489207518 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb7c0b1370 0x7fdb7c208870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:01.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.849+0000 7fdb81b53700 1 --2- 172.21.15.143:0/2489207518 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdb7c0c9c40 0x7fdb7c0c70d0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:01.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.849+0000 7fdb81b53700 1 -- 172.21.15.143:0/2489207518 >> 172.21.15.143:0/2489207518 conn(0x7fdb7c06b060 msgr2=0x7fdb7c0cb4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:01.854 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.849+0000 7ff7f81a8700 1 -- 172.21.15.143:0/989353387 shutdown_connections 2023-11-08T19:25:01.854 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.849+0000 7ff7f81a8700 1 -- 172.21.15.143:0/989353387 wait complete. 2023-11-08T19:25:01.855 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.850+0000 7fdb81b53700 1 -- 172.21.15.143:0/2489207518 shutdown_connections 2023-11-08T19:25:01.855 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.850+0000 7fdb81b53700 1 -- 172.21.15.143:0/2489207518 wait complete. 2023-11-08T19:25:01.856 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.0 2023-11-08T19:25:01.856 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.855+0000 7f9418684700 1 -- 172.21.15.143:0/1244684595 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f93f8005870 con 0x7f94100c7fd0 2023-11-08T19:25:01.857 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.855+0000 7f9406ffd700 1 -- 172.21.15.143:0/1244684595 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f94000544b0 con 0x7f94100c7fd0 2023-11-08T19:25:01.863 INFO:teuthology.orchestra.run.smithi143.stdout:55834574854 2023-11-08T19:25:01.863 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.861+0000 7f9418684700 1 -- 172.21.15.143:0/1244684595 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f93f406af50 msgr2=0x7f93f406d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:01.863 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.861+0000 7f9418684700 1 --2- 172.21.15.143:0/1244684595 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f93f406af50 0x7f93f406d410 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f940c007fd0 tx=0x7f940c007a20 comp rx=0 tx=0).stop 2023-11-08T19:25:01.864 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.861+0000 7f9418684700 1 -- 172.21.15.143:0/1244684595 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94100c7fd0 msgr2=0x7f94100c8450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:01.864 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.861+0000 7f9418684700 1 --2- 172.21.15.143:0/1244684595 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94100c7fd0 0x7f94100c8450 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f940000b3f0 tx=0x7f940000b7b0 comp rx=0 tx=0).stop 2023-11-08T19:25:01.864 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.861+0000 7f9418684700 1 -- 172.21.15.143:0/1244684595 shutdown_connections 2023-11-08T19:25:01.864 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.861+0000 7f9418684700 1 --2- 172.21.15.143:0/1244684595 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f93f406af50 0x7f93f406d410 secure :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f940c007fd0 tx=0x7f940c007a20 comp rx=0 tx=0).stop 2023-11-08T19:25:01.865 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.861+0000 7f9418684700 1 --2- 172.21.15.143:0/1244684595 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f93fc005330 0x7f9410160b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:01.865 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.861+0000 7f9418684700 1 --2- 172.21.15.143:0/1244684595 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f94101ffc20 0x7f94102000a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:01.865 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.861+0000 7f9418684700 1 --2- 172.21.15.143:0/1244684595 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94100c7fd0 0x7f94100c8450 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:01.865 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.861+0000 7f9418684700 1 -- 172.21.15.143:0/1244684595 >> 172.21.15.143:0/1244684595 conn(0x7f941006b060 msgr2=0x7f941015b840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:01.866 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.3 2023-11-08T19:25:01.867 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.862+0000 7f9418684700 1 -- 172.21.15.143:0/1244684595 shutdown_connections 2023-11-08T19:25:01.867 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.862+0000 7f9418684700 1 -- 172.21.15.143:0/1244684595 wait complete. 2023-11-08T19:25:01.867 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.862+0000 7f22e6ef3700 1 -- 172.21.15.143:0/3602248697 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f22e00cbe80 con 0x7f22e0159a20 2023-11-08T19:25:01.868 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.863+0000 7f22dd7fa700 1 -- 172.21.15.143:0/3602248697 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f22d808e020 con 0x7f22e0159a20 2023-11-08T19:25:01.868 INFO:teuthology.orchestra.run.smithi143.stdout:55834574854 2023-11-08T19:25:01.868 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.866+0000 7f22e6ef3700 1 -- 172.21.15.143:0/3602248697 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f22cc06af00 msgr2=0x7f22cc06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:01.869 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.866+0000 7f22e6ef3700 1 --2- 172.21.15.143:0/3602248697 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f22cc06af00 0x7f22cc06d3c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f22d4000c00 tx=0x7f22d4012000 comp rx=0 tx=0).stop 2023-11-08T19:25:01.869 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.866+0000 7f22e6ef3700 1 -- 172.21.15.143:0/3602248697 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f22e0159a20 msgr2=0x7f22e006d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:01.869 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.866+0000 7f22e6ef3700 1 --2- 172.21.15.143:0/3602248697 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f22e0159a20 0x7f22e006d6b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f22d800ba50 tx=0x7f22d800be10 comp rx=0 tx=0).stop 2023-11-08T19:25:01.870 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.868+0000 7f22e6ef3700 1 -- 172.21.15.143:0/3602248697 shutdown_connections 2023-11-08T19:25:01.870 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.868+0000 7f22e6ef3700 1 --2- 172.21.15.143:0/3602248697 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f22cc06af00 0x7f22cc06d3c0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:01.870 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.868+0000 7f22e6ef3700 1 --2- 172.21.15.143:0/3602248697 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f22e0159e00 0x7f22e006dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:01.870 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.868+0000 7f22e6ef3700 1 --2- 172.21.15.143:0/3602248697 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f22e00b1810 0x7f22e01fac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:01.871 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.868+0000 7f22e6ef3700 1 --2- 172.21.15.143:0/3602248697 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f22e0159a20 0x7f22e006d6b0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:01.871 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.868+0000 7f22e6ef3700 1 -- 172.21.15.143:0/3602248697 >> 172.21.15.143:0/3602248697 conn(0x7f22e006b060 msgr2=0x7f22e0158370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:01.871 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.868+0000 7f22e6ef3700 1 -- 172.21.15.143:0/3602248697 shutdown_connections 2023-11-08T19:25:01.871 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.868+0000 7f22e6ef3700 1 -- 172.21.15.143:0/3602248697 wait complete. 2023-11-08T19:25:01.876 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.2 2023-11-08T19:25:01.877 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.5 2023-11-08T19:25:01.878 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.6 2023-11-08T19:25:01.881 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.7 2023-11-08T19:25:01.882 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.881+0000 7f94f6721700 1 -- 172.21.15.143:0/2858259955 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f94cc0055a0 con 0x7f94f016e000 2023-11-08T19:25:01.882 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.881+0000 7f94ecff9700 1 -- 172.21.15.143:0/2858259955 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f94e8001d70 con 0x7f94f016e000 2023-11-08T19:25:01.883 INFO:teuthology.orchestra.run.smithi143.stdout:55834574854 2023-11-08T19:25:01.885 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.883+0000 7f94f6721700 1 -- 172.21.15.143:0/2858259955 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f94d406afa0 msgr2=0x7f94d406d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:01.886 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.883+0000 7f94f6721700 1 --2- 172.21.15.143:0/2858259955 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f94d406afa0 0x7f94d406d460 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f94e001e040 tx=0x7f94e000c040 comp rx=0 tx=0).stop 2023-11-08T19:25:01.886 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.883+0000 7f94f6721700 1 -- 172.21.15.143:0/2858259955 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94f016e000 msgr2=0x7f94f0204bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:01.886 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.883+0000 7f94f6721700 1 --2- 172.21.15.143:0/2858259955 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94f016e000 0x7f94f0204bc0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f94e8001f10 tx=0x7f94e800a340 comp rx=0 tx=0).stop 2023-11-08T19:25:01.886 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.883+0000 7f94f6721700 1 -- 172.21.15.143:0/2858259955 shutdown_connections 2023-11-08T19:25:01.886 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.883+0000 7f94f6721700 1 --2- 172.21.15.143:0/2858259955 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f94d406afa0 0x7f94d406d460 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:01.887 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.883+0000 7f94f6721700 1 --2- 172.21.15.143:0/2858259955 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f94f0206eb0 0x7f94f0205100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:01.887 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.883+0000 7f94f6721700 1 --2- 172.21.15.143:0/2858259955 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f94f00b1370 0x7f94f0206970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:01.887 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.883+0000 7f94f6721700 1 --2- 172.21.15.143:0/2858259955 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94f016e000 0x7f94f0204bc0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:01.887 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.883+0000 7f94f6721700 1 -- 172.21.15.143:0/2858259955 >> 172.21.15.143:0/2858259955 conn(0x7f94f006b060 msgr2=0x7f94f00b0090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:01.887 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.883+0000 7f94f6721700 1 -- 172.21.15.143:0/2858259955 shutdown_connections 2023-11-08T19:25:01.888 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:01.883+0000 7f94f6721700 1 -- 172.21.15.143:0/2858259955 wait complete. 2023-11-08T19:25:01.888 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.1 2023-11-08T19:25:01.897 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.4 2023-11-08T19:25:02.857 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-11-08T19:25:02.868 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-11-08T19:25:02.876 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-11-08T19:25:02.877 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-11-08T19:25:02.879 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-11-08T19:25:02.882 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-11-08T19:25:02.890 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-11-08T19:25:02.898 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-11-08T19:25:03.059 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.055+0000 7f0928b6d700 1 Processor -- start 2023-11-08T19:25:03.060 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.056+0000 7f0928b6d700 1 -- start start 2023-11-08T19:25:03.060 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.057+0000 7f0928b6d700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f092416e000 0x7f092416eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:03.060 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.057+0000 7f0928b6d700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f092416f3e0 con 0x7f09240cda90 2023-11-08T19:25:03.060 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.057+0000 7f0928b6d700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f092406d550 con 0x7f09240b1370 2023-11-08T19:25:03.061 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.057+0000 7f0928b6d700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f092408a660 con 0x7f092416e000 2023-11-08T19:25:03.061 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.058+0000 7f0921d9b700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f09240cda90 0x7f092406d9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:45198/0 (socket says 172.21.15.143:45198) 2023-11-08T19:25:03.061 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.058+0000 7f0921d9b700 1 -- 172.21.15.143:0/155864192 learned_addr learned my addr 172.21.15.143:0/155864192 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:25:03.061 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.058+0000 7f0922d9d700 1 --2- 172.21.15.143:0/155864192 >> v2:172.21.15.143:3300/0 conn(0x7f092416e000 0x7f092416eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:03.062 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.059+0000 7f0922d9d700 1 -- 172.21.15.143:0/155864192 >> v1:172.21.15.174:6789/0 conn(0x7f09240b1370 legacy=0x7f092416d8f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-08T19:25:03.063 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.059+0000 7f0922d9d700 1 -- 172.21.15.143:0/155864192 >> v1:172.21.15.143:6789/0 conn(0x7f09240cda90 legacy=0x7f092406d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:03.063 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.059+0000 7f0922d9d700 1 -- 172.21.15.143:0/155864192 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09240ce200 con 0x7f092416e000 2023-11-08T19:25:03.063 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.059+0000 7f0922d9d700 1 --2- 172.21.15.143:0/155864192 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f092416e000 0x7f092416eea0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f0918009ca0 tx=0x7f0918007040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91e5e216cf2780d9 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:03.064 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.060+0000 7f092159a700 1 -- 172.21.15.143:0/155864192 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f091800f6a0 con 0x7f092416e000 2023-11-08T19:25:03.064 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.060+0000 7f092159a700 1 -- 172.21.15.143:0/155864192 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f091800f800 con 0x7f092416e000 2023-11-08T19:25:03.064 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.060+0000 7f092159a700 1 -- 172.21.15.143:0/155864192 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f091800fb90 con 0x7f092416e000 2023-11-08T19:25:03.066 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.062+0000 7f0928b6d700 1 -- 172.21.15.143:0/155864192 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f092416e000 msgr2=0x7f092416eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:03.066 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.062+0000 7f0928b6d700 1 --2- 172.21.15.143:0/155864192 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f092416e000 0x7f092416eea0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f0918009ca0 tx=0x7f0918007040 comp rx=0 tx=0).stop 2023-11-08T19:25:03.067 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.062+0000 7f0928b6d700 1 -- 172.21.15.143:0/155864192 shutdown_connections 2023-11-08T19:25:03.067 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.062+0000 7f0928b6d700 1 --2- 172.21.15.143:0/155864192 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f092416e000 0x7f092416eea0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:03.067 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.062+0000 7f0928b6d700 1 -- 172.21.15.143:0/155864192 >> 172.21.15.143:0/155864192 conn(0x7f092406b060 msgr2=0x7f092406b470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:03.067 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.062+0000 7f0928b6d700 1 -- 172.21.15.143:0/155864192 shutdown_connections 2023-11-08T19:25:03.068 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.062+0000 7f0928b6d700 1 -- 172.21.15.143:0/155864192 wait complete. 2023-11-08T19:25:03.069 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.065+0000 7f0928b6d700 1 Processor -- start 2023-11-08T19:25:03.069 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.065+0000 7f0928b6d700 1 -- start start 2023-11-08T19:25:03.069 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.066+0000 7f0928b6d700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f09240b1370 0x7f09240b2e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:03.070 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.066+0000 7f0928b6d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f092416e000 0x7f0924089680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:03.070 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.066+0000 7f0928b6d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f09240b4e00 0x7f09240c8890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:03.070 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.066+0000 7f0928b6d700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09240b40c0 con 0x7f09240b4e00 2023-11-08T19:25:03.071 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.066+0000 7f0928b6d700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09240b3440 con 0x7f092416e000 2023-11-08T19:25:03.071 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.066+0000 7f0928b6d700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09240b3720 con 0x7f09240b1370 2023-11-08T19:25:03.071 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.067+0000 7f0922d9d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f09240b4e00 0x7f09240c8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:03.071 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.067+0000 7f0922d9d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f09240b4e00 0x7f09240c8890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49052/0 (socket says 172.21.15.143:49052) 2023-11-08T19:25:03.071 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.067+0000 7f0922d9d700 1 -- 172.21.15.143:0/3084546754 learned_addr learned my addr 172.21.15.143:0/3084546754 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:03.072 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.068+0000 7f0921d9b700 1 --2- 172.21.15.143:0/3084546754 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f092416e000 0x7f0924089680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:03.072 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.068+0000 7f092259c700 1 --2- 172.21.15.143:0/3084546754 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f09240b1370 0x7f09240b2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:03.073 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.068+0000 7f0922d9d700 1 -- 172.21.15.143:0/3084546754 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f09240b1370 msgr2=0x7f09240b2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:03.073 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.068+0000 7f0922d9d700 1 --2- 172.21.15.143:0/3084546754 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f09240b1370 0x7f09240b2e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:03.073 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.068+0000 7f0922d9d700 1 -- 172.21.15.143:0/3084546754 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f092416e000 msgr2=0x7f0924089680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:03.073 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.068+0000 7f0922d9d700 1 --2- 172.21.15.143:0/3084546754 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f092416e000 0x7f0924089680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:03.073 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.068+0000 7f0922d9d700 1 -- 172.21.15.143:0/3084546754 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0918009980 con 0x7f09240b4e00 2023-11-08T19:25:03.074 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.068+0000 7f0922d9d700 1 --2- 172.21.15.143:0/3084546754 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f09240b4e00 0x7f09240c8890 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f0924072360 tx=0x7f0918004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:03.074 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.069+0000 7f092259c700 1 --2- 172.21.15.143:0/3084546754 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f09240b1370 0x7f09240b2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:25:03.074 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.069+0000 7f0912ffd700 1 -- 172.21.15.143:0/3084546754 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0918010950 con 0x7f09240b4e00 2023-11-08T19:25:03.074 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.069+0000 7f0912ffd700 1 -- 172.21.15.143:0/3084546754 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f091801f070 con 0x7f09240b4e00 2023-11-08T19:25:03.075 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.069+0000 7f0912ffd700 1 -- 172.21.15.143:0/3084546754 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0918010cf0 con 0x7f09240b4e00 2023-11-08T19:25:03.075 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.069+0000 7f0928b6d700 1 -- 172.21.15.143:0/3084546754 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09240b3a00 con 0x7f09240b4e00 2023-11-08T19:25:03.075 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.069+0000 7f0928b6d700 1 -- 172.21.15.143:0/3084546754 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09240b5290 con 0x7f09240b4e00 2023-11-08T19:25:03.075 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.073+0000 7f0912ffd700 1 -- 172.21.15.143:0/3084546754 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0918007440 con 0x7f09240b4e00 2023-11-08T19:25:03.076 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.074+0000 7f0912ffd700 1 --2- 172.21.15.143:0/3084546754 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f090c06b150 0x7f090c06d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:03.076 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.074+0000 7f0912ffd700 1 -- 172.21.15.143:0/3084546754 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0918021070 con 0x7f09240b4e00 2023-11-08T19:25:03.077 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.075+0000 7f092259c700 1 --2- 172.21.15.143:0/3084546754 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f090c06b150 0x7f090c06d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:03.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.083+0000 7f0928b6d700 1 -- 172.21.15.143:0/3084546754 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09080052f0 con 0x7f09240b4e00 2023-11-08T19:25:03.094 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.093+0000 7f0912ffd700 1 -- 172.21.15.143:0/3084546754 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0918056080 con 0x7f09240b4e00 2023-11-08T19:25:03.094 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.093+0000 7f092259c700 1 --2- 172.21.15.143:0/3084546754 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f090c06b150 0x7f090c06d610 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f091c007eb0 tx=0x7f091c0073c0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:03.114 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.113+0000 7fc192da1700 1 Processor -- start 2023-11-08T19:25:03.116 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.113+0000 7fc192da1700 1 -- start start 2023-11-08T19:25:03.116 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.114+0000 7fc192da1700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fc18c15dc80 0x7fc18c0d1f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:03.116 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.114+0000 7fc192da1700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc18c0cf090 0x7fc18c0b9690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:03.116 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.114+0000 7fc192da1700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc18c0d3200 con 0x7fc18c0d3690 2023-11-08T19:25:03.117 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.114+0000 7fc192da1700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc18c0b9bd0 con 0x7fc18c15dc80 2023-11-08T19:25:03.117 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.114+0000 7fc192da1700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc18c0b7cf0 con 0x7fc18c0cf090 2023-11-08T19:25:03.117 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.115+0000 7fc18bfff700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fc18c0d3690 0x7fc18c0ce980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:45212/0 (socket says 172.21.15.143:45212) 2023-11-08T19:25:03.117 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.115+0000 7fc18bfff700 1 -- 172.21.15.143:0/2396517299 learned_addr learned my addr 172.21.15.143:0/2396517299 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:25:03.118 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.115+0000 7fc190b3d700 1 --2- 172.21.15.143:0/2396517299 >> v2:172.21.15.174:3300/0 conn(0x7fc18c15dc80 0x7fc18c0d1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:03.119 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.115+0000 7fc19133e700 1 --2- 172.21.15.143:0/2396517299 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc18c0cf090 0x7fc18c0b9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:03.119 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.116+0000 7fc190b3d700 1 -- 172.21.15.143:0/2396517299 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc18c0cf090 msgr2=0x7fc18c0b9690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:03.119 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.116+0000 7fc190b3d700 1 --2- 172.21.15.143:0/2396517299 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc18c0cf090 0x7fc18c0b9690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:03.119 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.116+0000 7fc190b3d700 1 -- 172.21.15.143:0/2396517299 >> v1:172.21.15.143:6789/0 conn(0x7fc18c0d3690 legacy=0x7fc18c0ce980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:03.119 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.116+0000 7fc190b3d700 1 -- 172.21.15.143:0/2396517299 --> v2:172.21.15.174:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc18c15e060 con 0x7fc18c15dc80 2023-11-08T19:25:03.120 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.116+0000 7fc190b3d700 1 --2- 172.21.15.143:0/2396517299 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc18c15dc80 0x7fc18c0d1f80 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc184009d90 tx=0x7fc18400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7bff30096d5a86ae server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:03.120 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.117+0000 7fc18b7fe700 1 -- 172.21.15.143:0/2396517299 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc18400e070 con 0x7fc18c15dc80 2023-11-08T19:25:03.120 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.117+0000 7fc18b7fe700 1 -- 172.21.15.143:0/2396517299 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc18400bad0 con 0x7fc18c15dc80 2023-11-08T19:25:03.120 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.117+0000 7fc18b7fe700 1 -- 172.21.15.143:0/2396517299 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc18400be60 con 0x7fc18c15dc80 2023-11-08T19:25:03.121 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.118+0000 7fc192da1700 1 -- 172.21.15.143:0/2396517299 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc18c15dc80 msgr2=0x7fc18c0d1f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:03.121 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.118+0000 7fc192da1700 1 --2- 172.21.15.143:0/2396517299 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc18c15dc80 0x7fc18c0d1f80 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc184009d90 tx=0x7fc18400b330 comp rx=0 tx=0).stop 2023-11-08T19:25:03.122 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.118+0000 7f40e5a16700 1 Processor -- start 2023-11-08T19:25:03.122 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.118+0000 7f40e5a16700 1 -- start start 2023-11-08T19:25:03.122 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.118+0000 7f1454ff8700 1 Processor -- start 2023-11-08T19:25:03.123 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.120+0000 7f40e5a16700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f40e00b1810 0x7f40e00aea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:03.123 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.120+0000 7f40e5a16700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f40e006d420 0x7f40e006d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:03.123 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.120+0000 7f40e5a16700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40e006cf90 con 0x7f40e0162060 2023-11-08T19:25:03.123 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.120+0000 7f40e5a16700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40e00c9c40 con 0x7f40e006d420 2023-11-08T19:25:03.124 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.120+0000 7f40e5a16700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40e00cdb60 con 0x7f40e00b1810 2023-11-08T19:25:03.124 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.119+0000 7f1454ff8700 1 -- start start 2023-11-08T19:25:03.124 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.119+0000 7f1b7892f700 1 Processor -- start 2023-11-08T19:25:03.125 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.119+0000 7f1b7892f700 1 -- start start 2023-11-08T19:25:03.125 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.120+0000 7f756080e700 1 Processor -- start 2023-11-08T19:25:03.125 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.120+0000 7f756080e700 1 -- start start 2023-11-08T19:25:03.126 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.120+0000 7fc192da1700 1 -- 172.21.15.143:0/2396517299 shutdown_connections 2023-11-08T19:25:03.126 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.120+0000 7fc192da1700 1 --2- 172.21.15.143:0/2396517299 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc18c0cf090 0x7fc18c0b9690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:03.126 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.120+0000 7fc192da1700 1 --2- 172.21.15.143:0/2396517299 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc18c15dc80 0x7fc18c0d1f80 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:03.127 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.120+0000 7fc192da1700 1 -- 172.21.15.143:0/2396517299 >> 172.21.15.143:0/2396517299 conn(0x7fc18c0668e0 msgr2=0x7fc18c066cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:03.127 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.120+0000 7f1b7892f700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1b740b1810 0x7f1b7416eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:03.127 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.120+0000 7f1b7892f700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b7416f3e0 con 0x7f1b7416d8f0 2023-11-08T19:25:03.128 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.120+0000 7f1b7892f700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b7406d550 con 0x7f1b740cda90 2023-11-08T19:25:03.128 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.120+0000 7f1b7892f700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b7408a660 con 0x7f1b740b1810 2023-11-08T19:25:03.128 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.120+0000 7f1454ff8700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f1450162060 0x7f14500aea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:03.128 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.121+0000 7f1454ff8700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f14500b1100 0x7f14500b1540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:03.129 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.121+0000 7f1454ff8700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f145006cf90 con 0x7f1450165c50 2023-11-08T19:25:03.129 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.121+0000 7f1454ff8700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14500c9c40 con 0x7f1450162060 2023-11-08T19:25:03.129 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.121+0000 7f1454ff8700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14500cdb60 con 0x7f14500b1100 2023-11-08T19:25:03.129 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.120+0000 7f40deffd700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f40e0162060 0x7f40e00b1100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:45216/0 (socket says 172.21.15.143:45216) 2023-11-08T19:25:03.130 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.121+0000 7f40deffd700 1 -- 172.21.15.143:0/2862005778 learned_addr learned my addr 172.21.15.143:0/2862005778 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:25:03.130 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.121+0000 7f40de7fc700 1 --2- 172.21.15.143:0/2862005778 >> v2:172.21.15.174:3300/0 conn(0x7f40e00b1810 0x7f40e00aea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:03.130 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.121+0000 7f40df7fe700 1 --2- 172.21.15.143:0/2862005778 >> v2:172.21.15.143:3300/0 conn(0x7f40e006d420 0x7f40e006d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:03.130 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.121+0000 7f1445d9b700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f14500b1100 0x7f14500b1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:03.131 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.121+0000 7f40df7fe700 1 -- 172.21.15.143:0/2862005778 >> v2:172.21.15.174:3300/0 conn(0x7f40e00b1810 msgr2=0x7f40e00aea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:03.131 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.121+0000 7f40df7fe700 1 --2- 172.21.15.143:0/2862005778 >> v2:172.21.15.174:3300/0 conn(0x7f40e00b1810 0x7f40e00aea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:03.131 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.121+0000 7f40df7fe700 1 -- 172.21.15.143:0/2862005778 >> v1:172.21.15.143:6789/0 conn(0x7f40e0162060 legacy=0x7f40e00b1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:03.132 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.121+0000 7f40df7fe700 1 -- 172.21.15.143:0/2862005778 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40e0071be0 con 0x7f40e006d420 2023-11-08T19:25:03.132 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.121+0000 7f1b7259c700 1 --1- >> v1:172.21.15.174:6789/0 conn(0x7f1b740cda90 0x7f1b740b1100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.174:6789/0 says I am v1:172.21.15.143:56158/0 (socket says 172.21.15.143:56158) 2023-11-08T19:25:03.132 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.121+0000 7f1b7259c700 1 -- 172.21.15.143:0/1985099023 learned_addr learned my addr 172.21.15.143:0/1985099023 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:25:03.133 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.121+0000 7f1445d9b700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f14500b1100 0x7f14500b1540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.143:36712/0 (socket says 172.21.15.143:36712) 2023-11-08T19:25:03.133 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.121+0000 7f1445d9b700 1 -- 172.21.15.143:0/91061159 learned_addr learned my addr 172.21.15.143:0/91061159 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:03.134 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.121+0000 7f40df7fe700 1 --2- 172.21.15.143:0/2862005778 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f40e006d420 0x7f40e006d830 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f40e0165b90 tx=0x7f40d4009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f9cd57324909270 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:03.134 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.121+0000 7f1b71d9b700 1 --2- 172.21.15.143:0/1985099023 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1b740b1810 0x7f1b7416eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:03.135 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.121+0000 7f144e59c700 1 --2- 172.21.15.143:0/91061159 >> v2:172.21.15.143:3300/0 conn(0x7f1450162060 0x7f14500aea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:03.135 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.121+0000 7f756080e700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f755c162060 0x7f755c0aea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:03.136 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.122+0000 7f756080e700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f755c06cf90 con 0x7f755c0b1100 2023-11-08T19:25:03.136 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.122+0000 7f756080e700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f755c166a70 con 0x7f755c06e430 2023-11-08T19:25:03.136 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.122+0000 7f756080e700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f755c0c9c40 con 0x7f755c162060 2023-11-08T19:25:03.137 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.122+0000 7f7559d9b700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f755c0b1100 0x7f755c165c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:45234/0 (socket says 172.21.15.143:45234) 2023-11-08T19:25:03.137 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.122+0000 7f7559d9b700 1 -- 172.21.15.143:0/2865026870 learned_addr learned my addr 172.21.15.143:0/2865026870 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:25:03.138 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.122+0000 7f1b7159a700 1 -- 172.21.15.143:0/1985099023 <== mon.1 v1:172.21.15.174:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 417791998 0 0) 0x7f1b7406d550 con 0x7f1b740cda90 2023-11-08T19:25:03.138 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.122+0000 7f1b7159a700 1 -- 172.21.15.143:0/1985099023 --> v1:172.21.15.174:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b5c0057c0 con 0x7f1b740cda90 2023-11-08T19:25:03.139 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.122+0000 7f144e59c700 1 -- 172.21.15.143:0/91061159 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f14500b1100 msgr2=0x7f14500b1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:03.139 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.122+0000 7f144e59c700 1 --2- 172.21.15.143:0/91061159 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f14500b1100 0x7f14500b1540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:03.139 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.122+0000 7f144e59c700 1 -- 172.21.15.143:0/91061159 >> v1:172.21.15.143:6789/0 conn(0x7f1450165c50 legacy=0x7f14501660c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:03.140 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.122+0000 7f144e59c700 1 -- 172.21.15.143:0/91061159 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14500b1a80 con 0x7f1450162060 2023-11-08T19:25:03.140 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.122+0000 7f1b71d9b700 1 -- 172.21.15.143:0/1985099023 >> v1:172.21.15.174:6789/0 conn(0x7f1b740cda90 legacy=0x7f1b740b1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:03.141 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.122+0000 7f1b71d9b700 1 -- 172.21.15.143:0/1985099023 >> v1:172.21.15.143:6789/0 conn(0x7f1b7416d8f0 legacy=0x7f1b7406d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:03.141 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.122+0000 7f1b71d9b700 1 -- 172.21.15.143:0/1985099023 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b740cdf20 con 0x7f1b740b1810 2023-11-08T19:25:03.142 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.122+0000 7f144e59c700 1 --2- 172.21.15.143:0/91061159 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1450162060 0x7f14500aea70 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f1440009d90 tx=0x7f144000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d13e3d07d53c236f server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:03.142 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.122+0000 7f755a59c700 1 --2- 172.21.15.143:0/2865026870 >> v2:172.21.15.174:3300/0 conn(0x7f755c162060 0x7f755c0aea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:03.143 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.122+0000 7f1b71d9b700 1 --2- 172.21.15.143:0/1985099023 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1b740b1810 0x7f1b7416eea0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1b6400aa10 tx=0x7f1b6400ad20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=950312a9a57285e3 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:03.143 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.122+0000 7f755959a700 1 -- 172.21.15.143:0/2865026870 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 496764124 0 0) 0x7f755c06cf90 con 0x7f755c0b1100 2023-11-08T19:25:03.144 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.122+0000 7f755959a700 1 -- 172.21.15.143:0/2865026870 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f75440035b0 con 0x7f755c0b1100 2023-11-08T19:25:03.145 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.122+0000 7fc192da1700 1 -- 172.21.15.143:0/2396517299 shutdown_connections 2023-11-08T19:25:03.145 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.123+0000 7fc192da1700 1 -- 172.21.15.143:0/2396517299 wait complete. 2023-11-08T19:25:03.146 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.123+0000 7f1b7159a700 1 -- 172.21.15.143:0/1985099023 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b6400cb70 con 0x7f1b740b1810 2023-11-08T19:25:03.146 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.123+0000 7f1b7159a700 1 -- 172.21.15.143:0/1985099023 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b6400ccd0 con 0x7f1b740b1810 2023-11-08T19:25:03.146 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.123+0000 7f1b7159a700 1 -- 172.21.15.143:0/1985099023 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b64010640 con 0x7f1b740b1810 2023-11-08T19:25:03.147 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.123+0000 7f40ddffb700 1 -- 172.21.15.143:0/2862005778 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40d4011830 con 0x7f40e006d420 2023-11-08T19:25:03.147 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.123+0000 7f40ddffb700 1 -- 172.21.15.143:0/2862005778 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40d4011990 con 0x7f40e006d420 2023-11-08T19:25:03.148 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.123+0000 7f40ddffb700 1 -- 172.21.15.143:0/2862005778 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40d4011d00 con 0x7f40e006d420 2023-11-08T19:25:03.148 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.123+0000 7f144dd9b700 1 -- 172.21.15.143:0/91061159 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f144000e070 con 0x7f1450162060 2023-11-08T19:25:03.148 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.123+0000 7f144dd9b700 1 -- 172.21.15.143:0/91061159 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f144000bb20 con 0x7f1450162060 2023-11-08T19:25:03.149 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.123+0000 7f144dd9b700 1 -- 172.21.15.143:0/91061159 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1440012430 con 0x7f1450162060 2023-11-08T19:25:03.149 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.123+0000 7f755959a700 1 -- 172.21.15.143:0/2865026870 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4137648795 0 0) 0x7f75440035b0 con 0x7f755c0b1100 2023-11-08T19:25:03.150 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.123+0000 7f755959a700 1 -- 172.21.15.143:0/2865026870 >> v2:172.21.15.174:3300/0 conn(0x7f755c162060 msgr2=0x7f755c0aea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:03.150 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.123+0000 7f755959a700 1 --2- 172.21.15.143:0/2865026870 >> v2:172.21.15.174:3300/0 conn(0x7f755c162060 0x7f755c0aea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:03.150 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.123+0000 7f755959a700 1 -- 172.21.15.143:0/2865026870 >> v1:172.21.15.174:6789/0 conn(0x7f755c06e430 legacy=0x7f755c166360 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-08T19:25:03.151 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.123+0000 7f755959a700 1 -- 172.21.15.143:0/2865026870 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f755c071be0 con 0x7f755c0b1100 2023-11-08T19:25:03.151 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.123+0000 7f755959a700 1 -- 172.21.15.143:0/2865026870 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4147756303 0 0) 0x7f7550001670 con 0x7f755c0b1100 2023-11-08T19:25:03.151 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.123+0000 7f755959a700 1 -- 172.21.15.143:0/2865026870 >> v1:172.21.15.143:6789/0 conn(0x7f755c0b1100 legacy=0x7f755c165c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:03.151 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.123+0000 7f755959a700 1 --2- 172.21.15.143:0/2865026870 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7544003a00 0x7f7544003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:03.152 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.123+0000 7f755959a700 1 --2- 172.21.15.143:0/2865026870 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7544004850 0x7f7544004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:03.152 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.123+0000 7f755959a700 1 --2- 172.21.15.143:0/2865026870 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7544005330 0x7f754400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:03.152 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.123+0000 7f755959a700 1 -- 172.21.15.143:0/2865026870 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f754400c1f0 con 0x7f7544003a00 2023-11-08T19:25:03.152 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.123+0000 7f755959a700 1 -- 172.21.15.143:0/2865026870 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f754400c360 con 0x7f7544004850 2023-11-08T19:25:03.152 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.123+0000 7f755959a700 1 -- 172.21.15.143:0/2865026870 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f754400c520 con 0x7f7544005330 2023-11-08T19:25:03.153 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.123+0000 7f7559d9b700 1 --2- 172.21.15.143:0/2865026870 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7544004850 0x7f7544004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:03.153 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.123+0000 7f7559d9b700 1 -- 172.21.15.143:0/2865026870 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7544005330 msgr2=0x7f754400ba10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-08T19:25:03.153 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.123+0000 7f7559d9b700 1 --2- 172.21.15.143:0/2865026870 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7544005330 0x7f754400ba10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:03.153 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.123+0000 7f7559d9b700 1 -- 172.21.15.143:0/2865026870 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7544003a00 msgr2=0x7f7544003e00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-08T19:25:03.153 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.123+0000 7f7559d9b700 1 --2- 172.21.15.143:0/2865026870 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7544003a00 0x7f7544003e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:03.154 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.123+0000 7f7559d9b700 1 -- 172.21.15.143:0/2865026870 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f755c071be0 con 0x7f7544004850 2023-11-08T19:25:03.154 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.124+0000 7f1b7892f700 1 -- 172.21.15.143:0/1985099023 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1b740b1810 msgr2=0x7f1b7416eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:03.154 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.124+0000 7f1b7892f700 1 --2- 172.21.15.143:0/1985099023 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1b740b1810 0x7f1b7416eea0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1b6400aa10 tx=0x7f1b6400ad20 comp rx=0 tx=0).stop 2023-11-08T19:25:03.155 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.124+0000 7f7559d9b700 1 --2- 172.21.15.143:0/2865026870 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7544004850 0x7f7544004cd0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f755000c340 tx=0x7f7544005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb4d5213555d152a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:03.155 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.124+0000 7f40e5a16700 1 -- 172.21.15.143:0/2862005778 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f40e006d420 msgr2=0x7f40e006d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:03.155 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.124+0000 7f40e5a16700 1 --2- 172.21.15.143:0/2862005778 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f40e006d420 0x7f40e006d830 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f40e0165b90 tx=0x7f40d4009b20 comp rx=0 tx=0).stop 2023-11-08T19:25:03.156 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.124+0000 7f40e5a16700 1 -- 172.21.15.143:0/2862005778 shutdown_connections 2023-11-08T19:25:03.156 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.124+0000 7f40e5a16700 1 --2- 172.21.15.143:0/2862005778 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f40e006d420 0x7f40e006d830 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:03.156 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.124+0000 7f40e5a16700 1 --2- 172.21.15.143:0/2862005778 >> v2:172.21.15.174:3300/0 conn(0x7f40e00b1810 0x7f40e00aea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:03.156 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.124+0000 7f1b7892f700 1 -- 172.21.15.143:0/1985099023 shutdown_connections 2023-11-08T19:25:03.157 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.124+0000 7f1b7892f700 1 --2- 172.21.15.143:0/1985099023 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1b740b1810 0x7f1b7416eea0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:03.157 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.124+0000 7f40e5a16700 1 -- 172.21.15.143:0/2862005778 >> 172.21.15.143:0/2862005778 conn(0x7f40e006b4c0 msgr2=0x7f40e00cf680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:03.157 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.124+0000 7f1b7892f700 1 -- 172.21.15.143:0/1985099023 >> 172.21.15.143:0/1985099023 conn(0x7f1b7406b060 msgr2=0x7f1b7406b470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:03.158 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.124+0000 7f1454ff8700 1 -- 172.21.15.143:0/91061159 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1450162060 msgr2=0x7f14500aea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:03.158 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.124+0000 7f1454ff8700 1 --2- 172.21.15.143:0/91061159 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1450162060 0x7f14500aea70 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f1440009d90 tx=0x7f144000b330 comp rx=0 tx=0).stop 2023-11-08T19:25:03.159 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.124+0000 7f755959a700 1 -- 172.21.15.143:0/2865026870 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f755000ce80 con 0x7f7544004850 2023-11-08T19:25:03.159 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.124+0000 7f755959a700 1 -- 172.21.15.143:0/2865026870 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7550005960 con 0x7f7544004850 2023-11-08T19:25:03.159 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.124+0000 7f755959a700 1 -- 172.21.15.143:0/2865026870 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7550005cf0 con 0x7f7544004850 2023-11-08T19:25:03.160 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.124+0000 7f40e5a16700 1 -- 172.21.15.143:0/2862005778 shutdown_connections 2023-11-08T19:25:03.160 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.124+0000 7f40e5a16700 1 -- 172.21.15.143:0/2862005778 wait complete. 2023-11-08T19:25:03.160 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.124+0000 7f1454ff8700 1 -- 172.21.15.143:0/91061159 shutdown_connections 2023-11-08T19:25:03.161 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.124+0000 7f1454ff8700 1 --2- 172.21.15.143:0/91061159 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1450162060 0x7f14500aea70 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:03.161 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.124+0000 7f1454ff8700 1 --2- 172.21.15.143:0/91061159 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f14500b1100 0x7f14500b1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:03.161 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.124+0000 7f1454ff8700 1 -- 172.21.15.143:0/91061159 >> 172.21.15.143:0/91061159 conn(0x7f145006b4c0 msgr2=0x7f14500cf680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:03.162 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.124+0000 7fc192da1700 1 Processor -- start 2023-11-08T19:25:03.162 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.124+0000 7fc192da1700 1 -- start start 2023-11-08T19:25:03.162 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.125+0000 7f756080e700 1 -- 172.21.15.143:0/2865026870 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7544004850 msgr2=0x7f7544004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:03.162 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.125+0000 7f756080e700 1 --2- 172.21.15.143:0/2865026870 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7544004850 0x7f7544004cd0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f755000c340 tx=0x7f7544005880 comp rx=0 tx=0).stop 2023-11-08T19:25:03.163 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.125+0000 7f1b7892f700 1 -- 172.21.15.143:0/1985099023 shutdown_connections 2023-11-08T19:25:03.163 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.125+0000 7f1b7892f700 1 -- 172.21.15.143:0/1985099023 wait complete. 2023-11-08T19:25:03.164 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.125+0000 7fc192da1700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc18c0cf090 0x7fc18c152150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:03.164 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.125+0000 7fc192da1700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc18c0d3690 0x7fc18c156b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:03.164 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.125+0000 7fc192da1700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc18c15dc80 0x7fc18c155380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:03.164 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.125+0000 7fc192da1700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc18c155d40 con 0x7fc18c0cf090 2023-11-08T19:25:03.165 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.125+0000 7fc192da1700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc18c0c81b0 con 0x7fc18c0d3690 2023-11-08T19:25:03.165 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.125+0000 7fc192da1700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc18c0c7880 con 0x7fc18c15dc80 2023-11-08T19:25:03.165 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.124+0000 7f1454ff8700 1 -- 172.21.15.143:0/91061159 shutdown_connections 2023-11-08T19:25:03.166 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.126+0000 7f1454ff8700 1 -- 172.21.15.143:0/91061159 wait complete. 2023-11-08T19:25:03.166 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.126+0000 7fc19133e700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc18c15dc80 0x7fc18c155380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:03.166 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.126+0000 7fc19133e700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc18c15dc80 0x7fc18c155380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.143:36728/0 (socket says 172.21.15.143:36728) 2023-11-08T19:25:03.167 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.126+0000 7fc19133e700 1 -- 172.21.15.143:0/1574468572 learned_addr learned my addr 172.21.15.143:0/1574468572 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:03.167 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.126+0000 7fc190b3d700 1 --2- 172.21.15.143:0/1574468572 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc18c0cf090 0x7fc18c152150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:03.167 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.126+0000 7fc18bfff700 1 --2- 172.21.15.143:0/1574468572 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc18c0d3690 0x7fc18c156b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:03.167 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.126+0000 7f40e5a16700 1 Processor -- start 2023-11-08T19:25:03.168 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.126+0000 7fc190b3d700 1 -- 172.21.15.143:0/1574468572 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc18c15dc80 msgr2=0x7fc18c155380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:03.168 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.126+0000 7fc190b3d700 1 --2- 172.21.15.143:0/1574468572 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc18c15dc80 0x7fc18c155380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:03.168 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.126+0000 7fc190b3d700 1 -- 172.21.15.143:0/1574468572 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc18c0d3690 msgr2=0x7fc18c156b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:03.169 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.126+0000 7fc190b3d700 1 --2- 172.21.15.143:0/1574468572 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc18c0d3690 0x7fc18c156b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:03.169 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.126+0000 7fc190b3d700 1 -- 172.21.15.143:0/1574468572 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc184009a70 con 0x7fc18c0cf090 2023-11-08T19:25:03.169 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.126+0000 7fc190b3d700 1 --2- 172.21.15.143:0/1574468572 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc18c0cf090 0x7fc18c152150 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fc184015000 tx=0x7fc184004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:03.169 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.127+0000 7f1b7892f700 1 Processor -- start 2023-11-08T19:25:03.170 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.127+0000 7fc1897fa700 1 -- 172.21.15.143:0/1574468572 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1840047b0 con 0x7fc18c0cf090 2023-11-08T19:25:03.170 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.127+0000 7fc1897fa700 1 -- 172.21.15.143:0/1574468572 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc184004950 con 0x7fc18c0cf090 2023-11-08T19:25:03.171 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.127+0000 7fc192da1700 1 -- 172.21.15.143:0/1574468572 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc18c0c7b60 con 0x7fc18c0cf090 2023-11-08T19:25:03.171 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.127+0000 7fc192da1700 1 -- 172.21.15.143:0/1574468572 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc18c1570a0 con 0x7fc18c0cf090 2023-11-08T19:25:03.171 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.127+0000 7fc1897fa700 1 -- 172.21.15.143:0/1574468572 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc18401c680 con 0x7fc18c0cf090 2023-11-08T19:25:03.171 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.126+0000 7f40e5a16700 1 -- start start 2023-11-08T19:25:03.172 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.127+0000 7f40e5a16700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f40e006d420 0x7f40e014d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:03.172 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.127+0000 7f40e5a16700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f40e00b1810 0x7f40e00c4b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:03.172 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.127+0000 7f40e5a16700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f40e0162060 0x7f40e00c50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:03.172 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.127+0000 7f40e5a16700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40e00c56b0 con 0x7f40e0162060 2023-11-08T19:25:03.172 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.127+0000 7f40e5a16700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40e00c5990 con 0x7f40e00b1810 2023-11-08T19:25:03.173 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.127+0000 7f40e5a16700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40e00c5c70 con 0x7f40e006d420 2023-11-08T19:25:03.173 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.128+0000 7f1454ff8700 1 Processor -- start 2023-11-08T19:25:03.173 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.128+0000 7f40df7fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f40e0162060 0x7f40e00c50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:03.174 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.128+0000 7f40df7fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f40e0162060 0x7f40e00c50a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49096/0 (socket says 172.21.15.143:49096) 2023-11-08T19:25:03.174 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.128+0000 7f40df7fe700 1 -- 172.21.15.143:0/974277949 learned_addr learned my addr 172.21.15.143:0/974277949 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:03.174 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.127+0000 7f1b7892f700 1 -- start start 2023-11-08T19:25:03.175 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.128+0000 7f40de7fc700 1 --2- 172.21.15.143:0/974277949 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f40e00b1810 0x7f40e00c4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:03.175 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.128+0000 7f1b7892f700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1b740b1810 0x7f1b7416ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:03.175 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.128+0000 7f1b7892f700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1b7416d8f0 0x7f1b7420ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:03.175 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.128+0000 7f1b7892f700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1b7420d410 0x7f1b740b4990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:03.176 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.128+0000 7f1b7892f700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b7416b0c0 con 0x7f1b7416d8f0 2023-11-08T19:25:03.176 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.128+0000 7f1b7892f700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b740c59d0 con 0x7f1b7420d410 2023-11-08T19:25:03.176 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.128+0000 7f1b7892f700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b740c4930 con 0x7f1b740b1810 2023-11-08T19:25:03.176 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.128+0000 7f40df7fe700 1 -- 172.21.15.143:0/974277949 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f40e006d420 msgr2=0x7f40e014d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:03.177 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.128+0000 7f40df7fe700 1 --2- 172.21.15.143:0/974277949 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f40e006d420 0x7f40e014d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:03.177 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.128+0000 7f40df7fe700 1 -- 172.21.15.143:0/974277949 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f40e00b1810 msgr2=0x7f40e00c4b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:03.177 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.128+0000 7f40df7fe700 1 --2- 172.21.15.143:0/974277949 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f40e00b1810 0x7f40e00c4b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:03.177 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.128+0000 7f40df7fe700 1 -- 172.21.15.143:0/974277949 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40d40094a0 con 0x7f40e0162060 2023-11-08T19:25:03.178 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.128+0000 7f1b7259c700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1b740b1810 0x7f1b7416ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:03.178 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.128+0000 7f1b7259c700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1b740b1810 0x7f1b7416ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.143:36738/0 (socket says 172.21.15.143:36738) 2023-11-08T19:25:03.178 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.128+0000 7f1b7259c700 1 -- 172.21.15.143:0/610929395 learned_addr learned my addr 172.21.15.143:0/610929395 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:03.178 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.128+0000 7f1b72d9d700 1 --2- 172.21.15.143:0/610929395 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1b7420d410 0x7f1b740b4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:03.179 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.128+0000 7f1454ff8700 1 -- start start 2023-11-08T19:25:03.179 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.129+0000 7f1b7259c700 1 -- 172.21.15.143:0/610929395 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1b7420d410 msgr2=0x7f1b740b4990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:03.180 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.129+0000 7f1b7259c700 1 --2- 172.21.15.143:0/610929395 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1b7420d410 0x7f1b740b4990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:03.180 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.129+0000 7f1b7259c700 1 -- 172.21.15.143:0/610929395 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1b7416d8f0 msgr2=0x7f1b7420ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:03.180 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.129+0000 7f1b7259c700 1 --2- 172.21.15.143:0/610929395 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1b7416d8f0 0x7f1b7420ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:03.180 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.129+0000 7f1b7259c700 1 -- 172.21.15.143:0/610929395 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b6400a640 con 0x7f1b740b1810 2023-11-08T19:25:03.180 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.129+0000 7f1b72d9d700 1 --2- 172.21.15.143:0/610929395 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1b7420d410 0x7f1b740b4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T19:25:03.181 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.129+0000 7f40df7fe700 1 --2- 172.21.15.143:0/974277949 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f40e0162060 0x7f40e00c50a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f40d40090b0 tx=0x7f40d4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:03.181 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.129+0000 7f1b7259c700 1 --2- 172.21.15.143:0/610929395 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1b740b1810 0x7f1b7416ab80 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1b6800c990 tx=0x7f1b6800cca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:03.182 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.129+0000 7fc1897fa700 1 -- 172.21.15.143:0/1574468572 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc18401c7e0 con 0x7fc18c0cf090 2023-11-08T19:25:03.182 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.129+0000 7f1454ff8700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f14500b1100 0x7f145015e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:03.182 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.129+0000 7f1454ff8700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1450162060 0x7f14500b4820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:03.182 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.129+0000 7f1454ff8700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1450165c50 0x7f14500b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:03.183 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.129+0000 7f1454ff8700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14501f8bf0 con 0x7f1450165c50 2023-11-08T19:25:03.183 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.129+0000 7f1454ff8700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14501625e0 con 0x7f1450162060 2023-11-08T19:25:03.183 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.129+0000 7f1454ff8700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14501628c0 con 0x7f14500b1100 2023-11-08T19:25:03.183 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.129+0000 7f40c7fff700 1 -- 172.21.15.143:0/974277949 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40d4004590 con 0x7f40e0162060 2023-11-08T19:25:03.184 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.129+0000 7f40c7fff700 1 -- 172.21.15.143:0/974277949 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40d401f070 con 0x7f40e0162060 2023-11-08T19:25:03.184 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.129+0000 7f40c7fff700 1 -- 172.21.15.143:0/974277949 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40d40038e0 con 0x7f40e0162060 2023-11-08T19:25:03.185 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.129+0000 7f144ed9d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1450165c50 0x7f14500b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:03.185 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.129+0000 7f144ed9d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1450165c50 0x7f14500b4d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49108/0 (socket says 172.21.15.143:49108) 2023-11-08T19:25:03.185 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.129+0000 7f144ed9d700 1 -- 172.21.15.143:0/108331573 learned_addr learned my addr 172.21.15.143:0/108331573 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:03.185 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.130+0000 7f144e59c700 1 --2- 172.21.15.143:0/108331573 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f14500b1100 0x7f145015e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:03.185 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.130+0000 7f144ed9d700 1 -- 172.21.15.143:0/108331573 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f14500b1100 msgr2=0x7f145015e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:03.186 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.130+0000 7f144ed9d700 1 --2- 172.21.15.143:0/108331573 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f14500b1100 0x7f145015e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:03.186 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.130+0000 7f144ed9d700 1 -- 172.21.15.143:0/108331573 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1450162060 msgr2=0x7f14500b4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:03.186 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.130+0000 7f144ed9d700 1 --2- 172.21.15.143:0/108331573 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1450162060 0x7f14500b4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:03.186 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.130+0000 7f144ed9d700 1 -- 172.21.15.143:0/108331573 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1440009a70 con 0x7f1450165c50 2023-11-08T19:25:03.186 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.130+0000 7f144e59c700 1 --2- 172.21.15.143:0/108331573 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f14500b1100 0x7f145015e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:25:03.187 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.130+0000 7fc1897fa700 1 --2- 172.21.15.143:0/1574468572 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc17406b090 0x7fc17406d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:03.187 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.130+0000 7fc1897fa700 1 -- 172.21.15.143:0/1574468572 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc18408ae90 con 0x7fc18c0cf090 2023-11-08T19:25:03.188 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.130+0000 7f40e5a16700 1 -- 172.21.15.143:0/974277949 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40e00c5f50 con 0x7f40e0162060 2023-11-08T19:25:03.188 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.130+0000 7f40e5a16700 1 -- 172.21.15.143:0/974277949 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40e015cec0 con 0x7f40e0162060 2023-11-08T19:25:03.188 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.130+0000 7f144ed9d700 1 --2- 172.21.15.143:0/108331573 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1450165c50 0x7f14500b4d60 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f144800c3c0 tx=0x7f144800c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:03.189 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.130+0000 7fc18bfff700 1 --2- 172.21.15.143:0/1574468572 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc17406b090 0x7fc17406d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:03.189 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.129+0000 7f1b62ffd700 1 -- 172.21.15.143:0/610929395 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b68007930 con 0x7f1b740b1810 2023-11-08T19:25:03.189 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.129+0000 7f1b62ffd700 1 -- 172.21.15.143:0/610929395 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b68007a90 con 0x7f1b740b1810 2023-11-08T19:25:03.190 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.129+0000 7f1b62ffd700 1 -- 172.21.15.143:0/610929395 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b68007e40 con 0x7f1b740b1810 2023-11-08T19:25:03.190 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.130+0000 7f1b7892f700 1 -- 172.21.15.143:0/610929395 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b740c4c10 con 0x7f1b740b1810 2023-11-08T19:25:03.190 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.130+0000 7f1b7892f700 1 -- 172.21.15.143:0/610929395 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b740c51d0 con 0x7f1b740b1810 2023-11-08T19:25:03.191 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.131+0000 7f14477fe700 1 -- 172.21.15.143:0/108331573 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14480052d0 con 0x7f1450165c50 2023-11-08T19:25:03.191 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.131+0000 7f14477fe700 1 -- 172.21.15.143:0/108331573 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1448005430 con 0x7f1450165c50 2023-11-08T19:25:03.191 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.131+0000 7f14477fe700 1 -- 172.21.15.143:0/108331573 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14480038a0 con 0x7f1450165c50 2023-11-08T19:25:03.191 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.131+0000 7f1454ff8700 1 -- 172.21.15.143:0/108331573 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1450162ba0 con 0x7f1450165c50 2023-11-08T19:25:03.192 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.131+0000 7f40e5a16700 1 -- 172.21.15.143:0/974277949 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40e00c6470 con 0x7f40e0162060 2023-11-08T19:25:03.192 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.131+0000 7f1b7892f700 1 -- 172.21.15.143:0/610929395 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b7404eee0 con 0x7f1b740b1810 2023-11-08T19:25:03.193 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.132+0000 7fc18bfff700 1 --2- 172.21.15.143:0/1574468572 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc17406b090 0x7fc17406d550 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc17c0013c0 tx=0x7fc17c001350 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:03.193 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.132+0000 7f40c7fff700 1 -- 172.21.15.143:0/974277949 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f40d4003a40 con 0x7f40e0162060 2023-11-08T19:25:03.193 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.131+0000 7f1454ff8700 1 -- 172.21.15.143:0/108331573 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f145015e700 con 0x7f1450165c50 2023-11-08T19:25:03.194 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.132+0000 7f1454ff8700 1 -- 172.21.15.143:0/108331573 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14501630c0 con 0x7f1450165c50 2023-11-08T19:25:03.194 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.132+0000 7f40c7fff700 1 --2- 172.21.15.143:0/974277949 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f40cc06af00 0x7f40cc06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:03.194 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.132+0000 7f14477fe700 1 -- 172.21.15.143:0/108331573 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1448003a00 con 0x7f1450165c50 2023-11-08T19:25:03.195 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.133+0000 7f40c7fff700 1 -- 172.21.15.143:0/974277949 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f40d4021070 con 0x7f40e0162060 2023-11-08T19:25:03.195 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.133+0000 7f40deffd700 1 --2- 172.21.15.143:0/974277949 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f40cc06af00 0x7f40cc06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:03.196 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.133+0000 7f14477fe700 1 --2- 172.21.15.143:0/108331573 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f143c06b010 0x7f143c06d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:03.196 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.133+0000 7f14477fe700 1 -- 172.21.15.143:0/108331573 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1448088910 con 0x7f1450165c50 2023-11-08T19:25:03.196 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.133+0000 7f144e59c700 1 --2- 172.21.15.143:0/108331573 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f143c06b010 0x7f143c06d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:03.197 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.133+0000 7f40deffd700 1 --2- 172.21.15.143:0/974277949 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f40cc06af00 0x7f40cc06d3c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f40d80012e0 tx=0x7f40d8001590 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:03.197 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.133+0000 7f1b62ffd700 1 -- 172.21.15.143:0/610929395 <== mon.2 v2:172.21.15.174:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1b68016420 con 0x7f1b740b1810 2023-11-08T19:25:03.198 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.134+0000 7f1b62ffd700 1 --2- 172.21.15.143:0/610929395 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f1b5c06b3a0 0x7f1b5c06d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:03.198 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.134+0000 7f1b62ffd700 1 -- 172.21.15.143:0/610929395 <== mon.2 v2:172.21.15.174:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1b68012070 con 0x7f1b740b1810 2023-11-08T19:25:03.198 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.134+0000 7f1b71d9b700 1 --2- 172.21.15.143:0/610929395 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f1b5c06b3a0 0x7f1b5c06d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:03.199 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.134+0000 7f144e59c700 1 --2- 172.21.15.143:0/108331573 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f143c06b010 0x7f143c06d4d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f14500be630 tx=0x7f1440011000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:03.199 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.134+0000 7f1b71d9b700 1 --2- 172.21.15.143:0/610929395 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f1b5c06b3a0 0x7f1b5c06d860 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1b6400ad60 tx=0x7f1b6400e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:03.199 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.135+0000 7fdcac4bc700 1 Processor -- start 2023-11-08T19:25:03.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.135+0000 7fdcac4bc700 1 -- start start 2023-11-08T19:25:03.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.136+0000 7f755ad9d700 1 -- 172.21.15.143:0/2865026870 reap_dead start 2023-11-08T19:25:03.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.136+0000 7f756080e700 1 -- 172.21.15.143:0/2865026870 shutdown_connections 2023-11-08T19:25:03.201 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.136+0000 7f756080e700 1 -- 172.21.15.143:0/2865026870 >> 172.21.15.143:0/2865026870 conn(0x7f755c06b4c0 msgr2=0x7f755c0cf680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:03.201 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.136+0000 7f756080e700 1 -- 172.21.15.143:0/2865026870 shutdown_connections 2023-11-08T19:25:03.201 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.136+0000 7fc192da1700 1 -- 172.21.15.143:0/1574468572 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc16c0052f0 con 0x7fc18c0cf090 2023-11-08T19:25:03.202 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.136+0000 7f756080e700 1 -- 172.21.15.143:0/2865026870 wait complete. 2023-11-08T19:25:03.203 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.136+0000 7fdcac4bc700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fdca415dc80 0x7fdca40d1f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:03.203 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.136+0000 7fdcac4bc700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fdca40d3690 0x7fdca40b9690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:03.203 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.136+0000 7fdcac4bc700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdca40b7cf0 con 0x7fdca40ce980 2023-11-08T19:25:03.203 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.136+0000 7fdcac4bc700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdca40b9bd0 con 0x7fdca40d3690 2023-11-08T19:25:03.204 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.136+0000 7fdcac4bc700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdca40d3200 con 0x7fdca415dc80 2023-11-08T19:25:03.204 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.137+0000 7fdcaa258700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fdca415dc80 0x7fdca40d1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:03.204 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.137+0000 7fdcaa258700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fdca415dc80 0x7fdca40d1f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.143:60850/0 (socket says 172.21.15.143:60850) 2023-11-08T19:25:03.204 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.137+0000 7fdcaa258700 1 -- 172.21.15.143:0/3127531294 learned_addr learned my addr 172.21.15.143:0/3127531294 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:03.204 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.137+0000 7fdca9a57700 1 --2- 172.21.15.143:0/3127531294 >> v2:172.21.15.143:3300/0 conn(0x7fdca40d3690 0x7fdca40b9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:03.205 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.137+0000 7fdca9a57700 1 -- 172.21.15.143:0/3127531294 >> v2:172.21.15.174:3300/0 conn(0x7fdca415dc80 msgr2=0x7fdca40d1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:03.205 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.137+0000 7fdca9a57700 1 --2- 172.21.15.143:0/3127531294 >> v2:172.21.15.174:3300/0 conn(0x7fdca415dc80 0x7fdca40d1f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:03.205 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.137+0000 7fdca9a57700 1 -- 172.21.15.143:0/3127531294 >> v1:172.21.15.143:6789/0 conn(0x7fdca40ce980 legacy=0x7fdca40b5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:03.205 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.137+0000 7fdca9a57700 1 -- 172.21.15.143:0/3127531294 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdca415e060 con 0x7fdca40d3690 2023-11-08T19:25:03.205 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.137+0000 7fdca9a57700 1 --2- 172.21.15.143:0/3127531294 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdca40d3690 0x7fdca40b9690 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fdca40b6710 tx=0x7fdc94009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d0f6f058c47f243 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:03.206 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.138+0000 7f756080e700 1 Processor -- start 2023-11-08T19:25:03.206 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.138+0000 7f756080e700 1 -- start start 2023-11-08T19:25:03.209 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.139+0000 7f1b62ffd700 1 -- 172.21.15.143:0/610929395 <== mon.2 v2:172.21.15.174:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1b680549a0 con 0x7f1b740b1810 2023-11-08T19:25:03.210 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.139+0000 7f40c7fff700 1 -- 172.21.15.143:0/974277949 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f40e00c6470 con 0x7f40e0162060 2023-11-08T19:25:03.210 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.139+0000 7f756080e700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f755c15ce10 0x7f755c0c3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:03.210 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.139+0000 7f756080e700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f755c0c3b90 0x7f755c160b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:03.211 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.139+0000 7f756080e700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f755c15d1f0 0x7f755c161060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:03.211 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.139+0000 7f756080e700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f755c0c9670 con 0x7f755c15ce10 2023-11-08T19:25:03.211 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.139+0000 7f756080e700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f755c0c2b90 con 0x7f755c15d1f0 2023-11-08T19:25:03.211 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.139+0000 7f756080e700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f755c1f8560 con 0x7f755c0c3b90 2023-11-08T19:25:03.212 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.139+0000 7f755a59c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f755c15ce10 0x7f755c0c3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:03.212 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.139+0000 7f755a59c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f755c15ce10 0x7f755c0c3650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49126/0 (socket says 172.21.15.143:49126) 2023-11-08T19:25:03.212 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.139+0000 7f755a59c700 1 -- 172.21.15.143:0/3586610471 learned_addr learned my addr 172.21.15.143:0/3586610471 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:03.212 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.139+0000 7f755a59c700 1 -- 172.21.15.143:0/3586610471 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f755c0c3b90 msgr2=0x7f755c160b20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-08T19:25:03.212 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.139+0000 7f755a59c700 1 --2- 172.21.15.143:0/3586610471 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f755c0c3b90 0x7f755c160b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:03.212 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.139+0000 7f755a59c700 1 -- 172.21.15.143:0/3586610471 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f755c15d1f0 msgr2=0x7f755c161060 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-08T19:25:03.213 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.139+0000 7f755a59c700 1 --2- 172.21.15.143:0/3586610471 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f755c15d1f0 0x7f755c161060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:03.213 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.139+0000 7f755a59c700 1 -- 172.21.15.143:0/3586610471 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f755c1f8840 con 0x7f755c15ce10 2023-11-08T19:25:03.213 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.140+0000 7f755a59c700 1 --2- 172.21.15.143:0/3586610471 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f755c15ce10 0x7f755c0c3650 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f754c00ae90 tx=0x7f754c00cad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:03.214 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.148+0000 7fdca9256700 1 -- 172.21.15.143:0/3127531294 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc94011830 con 0x7fdca40d3690 2023-11-08T19:25:03.214 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.148+0000 7f14477fe700 1 -- 172.21.15.143:0/108331573 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f14501630c0 con 0x7f1450165c50 2023-11-08T19:25:03.215 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.148+0000 7f754affd700 1 -- 172.21.15.143:0/3586610471 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f754c004020 con 0x7f755c15ce10 2023-11-08T19:25:03.215 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.149+0000 7f756080e700 1 -- 172.21.15.143:0/3586610471 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f755c0c2e50 con 0x7f755c15ce10 2023-11-08T19:25:03.215 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.149+0000 7f756080e700 1 -- 172.21.15.143:0/3586610471 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f755c1f7940 con 0x7f755c15ce10 2023-11-08T19:25:03.215 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.152+0000 7f756080e700 1 -- 172.21.15.143:0/3586610471 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f755c0c3370 con 0x7f755c15ce10 2023-11-08T19:25:03.216 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.156+0000 7f65a17ab700 1 Processor -- start 2023-11-08T19:25:03.216 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.156+0000 7f65a17ab700 1 -- start start 2023-11-08T19:25:03.217 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.157+0000 7fdca9256700 1 -- 172.21.15.143:0/3127531294 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc94011990 con 0x7fdca40d3690 2023-11-08T19:25:03.217 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.157+0000 7fdca9256700 1 -- 172.21.15.143:0/3127531294 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc94011d00 con 0x7fdca40d3690 2023-11-08T19:25:03.217 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.157+0000 7f65a17ab700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f659c1618f0 0x7f659c06ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:03.217 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.157+0000 7f65a17ab700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f659c161df0 con 0x7f659c06b150 2023-11-08T19:25:03.218 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.157+0000 7f65a17ab700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f659c1620d0 con 0x7f659c15dd00 2023-11-08T19:25:03.218 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.157+0000 7f65a17ab700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f659c06b600 con 0x7f659c1618f0 2023-11-08T19:25:03.218 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.157+0000 7fc1897fa700 1 -- 172.21.15.143:0/1574468572 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc184055f30 con 0x7fc18c0cf090 2023-11-08T19:25:03.219 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.158+0000 7fdcac4bc700 1 -- 172.21.15.143:0/3127531294 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdca40d3690 msgr2=0x7fdca40b9690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:03.219 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.158+0000 7fdcac4bc700 1 --2- 172.21.15.143:0/3127531294 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdca40d3690 0x7fdca40b9690 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fdca40b6710 tx=0x7fdc94009b20 comp rx=0 tx=0).stop 2023-11-08T19:25:03.220 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.159+0000 7f659affd700 1 --1- >> v1:172.21.15.174:6789/0 conn(0x7f659c15dd00 0x7f659c0cca70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.174:6789/0 says I am v1:172.21.15.143:56216/0 (socket says 172.21.15.143:56216) 2023-11-08T19:25:03.220 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.159+0000 7f659affd700 1 -- 172.21.15.143:0/771289090 learned_addr learned my addr 172.21.15.143:0/771289090 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:25:03.220 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.159+0000 7f659a7fc700 1 --2- 172.21.15.143:0/771289090 >> v2:172.21.15.174:3300/0 conn(0x7f659c1618f0 0x7f659c06ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:03.221 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.159+0000 7f659a7fc700 1 -- 172.21.15.143:0/771289090 >> v1:172.21.15.174:6789/0 conn(0x7f659c15dd00 legacy=0x7f659c0cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:03.221 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.159+0000 7f659a7fc700 1 -- 172.21.15.143:0/771289090 >> v1:172.21.15.143:6789/0 conn(0x7f659c06b150 legacy=0x7f659c0cb1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:03.221 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.159+0000 7f659a7fc700 1 -- 172.21.15.143:0/771289090 --> v2:172.21.15.174:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f659c1623b0 con 0x7f659c1618f0 2023-11-08T19:25:03.223 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.160+0000 7fdcac4bc700 1 -- 172.21.15.143:0/3127531294 shutdown_connections 2023-11-08T19:25:03.224 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.160+0000 7fdcac4bc700 1 --2- 172.21.15.143:0/3127531294 >> v2:172.21.15.174:3300/0 conn(0x7fdca415dc80 0x7fdca40d1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:03.224 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.160+0000 7fdcac4bc700 1 --2- 172.21.15.143:0/3127531294 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdca40d3690 0x7fdca40b9690 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:03.224 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.160+0000 7fdcac4bc700 1 -- 172.21.15.143:0/3127531294 >> 172.21.15.143:0/3127531294 conn(0x7fdca40668e0 msgr2=0x7fdca4066cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:03.225 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.160+0000 7fdcac4bc700 1 -- 172.21.15.143:0/3127531294 shutdown_connections 2023-11-08T19:25:03.225 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.160+0000 7fdcac4bc700 1 -- 172.21.15.143:0/3127531294 wait complete. 2023-11-08T19:25:03.225 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.162+0000 7fdcac4bc700 1 Processor -- start 2023-11-08T19:25:03.225 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.162+0000 7fdcac4bc700 1 -- start start 2023-11-08T19:25:03.226 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.163+0000 7fdcac4bc700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdca40ce980 0x7fdca40b3610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:03.226 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.163+0000 7fdcac4bc700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdca40d3690 0x7fdca41fd370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:03.226 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.163+0000 7fdcac4bc700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdca415dc80 0x7fdca40cd490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:03.227 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.163+0000 7fdcac4bc700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdca41fd8b0 con 0x7fdca415dc80 2023-11-08T19:25:03.227 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.163+0000 7fdcac4bc700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdca40cdf00 con 0x7fdca40ce980 2023-11-08T19:25:03.227 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.163+0000 7fdcac4bc700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdca40ce1e0 con 0x7fdca40d3690 2023-11-08T19:25:03.228 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.163+0000 7fdcaa258700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdca40ce980 0x7fdca40b3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:03.228 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.163+0000 7fdcaa258700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdca40ce980 0x7fdca40b3610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.143:60864/0 (socket says 172.21.15.143:60864) 2023-11-08T19:25:03.228 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.163+0000 7fdcaa258700 1 -- 172.21.15.143:0/3889314570 learned_addr learned my addr 172.21.15.143:0/3889314570 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:03.229 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.163+0000 7fdca9a57700 1 --2- 172.21.15.143:0/3889314570 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdca40d3690 0x7fdca41fd370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:03.229 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.164+0000 7fdcaa258700 1 -- 172.21.15.143:0/3889314570 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdca40d3690 msgr2=0x7fdca41fd370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:03.229 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.164+0000 7fdcaa258700 1 --2- 172.21.15.143:0/3889314570 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdca40d3690 0x7fdca41fd370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:03.229 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.164+0000 7fdcaa258700 1 -- 172.21.15.143:0/3889314570 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdca415dc80 msgr2=0x7fdca40cd490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:03.230 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.164+0000 7fdcaa258700 1 --2- 172.21.15.143:0/3889314570 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdca415dc80 0x7fdca40cd490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:03.230 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.164+0000 7fdcaa258700 1 -- 172.21.15.143:0/3889314570 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc940094a0 con 0x7fdca40ce980 2023-11-08T19:25:03.230 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.164+0000 7fdca9a57700 1 --2- 172.21.15.143:0/3889314570 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdca40d3690 0x7fdca41fd370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T19:25:03.231 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.164+0000 7fdcaa258700 1 --2- 172.21.15.143:0/3889314570 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdca40ce980 0x7fdca40b3610 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fdca000e910 tx=0x7fdca000ec20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:03.231 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.164+0000 7fdc9affd700 1 -- 172.21.15.143:0/3889314570 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdca00096a0 con 0x7fdca40ce980 2023-11-08T19:25:03.231 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.164+0000 7fdc9affd700 1 -- 172.21.15.143:0/3889314570 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdca000ee60 con 0x7fdca40ce980 2023-11-08T19:25:03.232 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.164+0000 7fdc9affd700 1 -- 172.21.15.143:0/3889314570 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdca000ca90 con 0x7fdca40ce980 2023-11-08T19:25:03.232 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.164+0000 7fdcac4bc700 1 -- 172.21.15.143:0/3889314570 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdca40ce4c0 con 0x7fdca40ce980 2023-11-08T19:25:03.232 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.164+0000 7fdcac4bc700 1 -- 172.21.15.143:0/3889314570 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdca4200c00 con 0x7fdca40ce980 2023-11-08T19:25:03.233 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.166+0000 7fdcac4bc700 1 -- 172.21.15.143:0/3889314570 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdca40cb040 con 0x7fdca40ce980 2023-11-08T19:25:03.233 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.166+0000 7f754affd700 1 -- 172.21.15.143:0/3586610471 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f754c004180 con 0x7f755c15ce10 2023-11-08T19:25:03.234 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.166+0000 7f754affd700 1 -- 172.21.15.143:0/3586610471 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f754c0044f0 con 0x7f755c15ce10 2023-11-08T19:25:03.234 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.167+0000 7fdc9affd700 1 -- 172.21.15.143:0/3889314570 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fdca000cbf0 con 0x7fdca40ce980 2023-11-08T19:25:03.235 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.167+0000 7fdc9affd700 1 --2- 172.21.15.143:0/3889314570 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fdc9006b010 0x7fdc9006d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:03.235 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.167+0000 7fdc9affd700 1 -- 172.21.15.143:0/3889314570 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdca0089570 con 0x7fdca40ce980 2023-11-08T19:25:03.235 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.167+0000 7fdca9a57700 1 --2- 172.21.15.143:0/3889314570 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fdc9006b010 0x7fdc9006d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:03.236 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.168+0000 7f754affd700 1 -- 172.21.15.143:0/3586610471 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f754c0036a0 con 0x7f755c15ce10 2023-11-08T19:25:03.236 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.168+0000 7f754affd700 1 --2- 172.21.15.143:0/3586610471 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f753c06af00 0x7f753c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:03.236 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.168+0000 7f754affd700 1 -- 172.21.15.143:0/3586610471 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f754c09a2b0 con 0x7f755c15ce10 2023-11-08T19:25:03.237 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.170+0000 7f7559d9b700 1 --2- 172.21.15.143:0/3586610471 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f753c06af00 0x7f753c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:03.237 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.170+0000 7f754affd700 1 -- 172.21.15.143:0/3586610471 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f754c065350 con 0x7f755c15ce10 2023-11-08T19:25:03.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.172+0000 7fdca9a57700 1 --2- 172.21.15.143:0/3889314570 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fdc9006b010 0x7fdc9006d4d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fdca40d3a90 tx=0x7fdc94008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:03.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.172+0000 7fdc9affd700 1 -- 172.21.15.143:0/3889314570 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdca40cb040 con 0x7fdca40ce980 2023-11-08T19:25:03.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.178+0000 7f7559d9b700 1 --2- 172.21.15.143:0/3586610471 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f753c06af00 0x7f753c06d3c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f755c0c9ba0 tx=0x7f7550007000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:03.240 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.178+0000 7f659a7fc700 1 --2- 172.21.15.143:0/771289090 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f659c1618f0 0x7f659c06ac10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f65900097c0 tx=0x7f6590009ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=361663f697a0c388 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:03.241 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.184+0000 7f6599ffb700 1 -- 172.21.15.143:0/771289090 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6590004230 con 0x7f659c1618f0 2023-11-08T19:25:03.241 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.184+0000 7f6599ffb700 1 -- 172.21.15.143:0/771289090 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6590004390 con 0x7f659c1618f0 2023-11-08T19:25:03.241 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.184+0000 7f6599ffb700 1 -- 172.21.15.143:0/771289090 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65900036a0 con 0x7f659c1618f0 2023-11-08T19:25:03.241 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.185+0000 7f65a17ab700 1 -- 172.21.15.143:0/771289090 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f659c1618f0 msgr2=0x7f659c06ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:03.242 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.185+0000 7f65a17ab700 1 --2- 172.21.15.143:0/771289090 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f659c1618f0 0x7f659c06ac10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f65900097c0 tx=0x7f6590009ad0 comp rx=0 tx=0).stop 2023-11-08T19:25:03.242 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.185+0000 7f65a17ab700 1 -- 172.21.15.143:0/771289090 shutdown_connections 2023-11-08T19:25:03.242 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.185+0000 7f65a17ab700 1 --2- 172.21.15.143:0/771289090 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f659c1618f0 0x7f659c06ac10 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:03.242 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.185+0000 7f65a17ab700 1 -- 172.21.15.143:0/771289090 >> 172.21.15.143:0/771289090 conn(0x7f659c099ab0 msgr2=0x7f659c0d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:03.242 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.185+0000 7f65a17ab700 1 -- 172.21.15.143:0/771289090 shutdown_connections 2023-11-08T19:25:03.243 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.185+0000 7f65a17ab700 1 -- 172.21.15.143:0/771289090 wait complete. 2023-11-08T19:25:03.243 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.188+0000 7f65a17ab700 1 Processor -- start 2023-11-08T19:25:03.243 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.188+0000 7f65a17ab700 1 -- start start 2023-11-08T19:25:03.243 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.189+0000 7f65a17ab700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f659c06b150 0x7f659c2041e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:03.243 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.189+0000 7f65a17ab700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f659c15dd00 0x7f659c0b33b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:03.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.189+0000 7f65a17ab700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f659c1618f0 0x7f659c1ffde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:03.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.189+0000 7f65a17ab700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f659c0c5d30 con 0x7f659c1618f0 2023-11-08T19:25:03.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.189+0000 7f65a17ab700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f659c0c6010 con 0x7f659c06b150 2023-11-08T19:25:03.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.189+0000 7f65a17ab700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f659c0c62f0 con 0x7f659c15dd00 2023-11-08T19:25:03.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.190+0000 7f659affd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f659c06b150 0x7f659c2041e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:03.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.190+0000 7f659affd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f659c06b150 0x7f659c2041e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.143:60874/0 (socket says 172.21.15.143:60874) 2023-11-08T19:25:03.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.190+0000 7f659affd700 1 -- 172.21.15.143:0/3911280795 learned_addr learned my addr 172.21.15.143:0/3911280795 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:03.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.190+0000 7f659a7fc700 1 --2- 172.21.15.143:0/3911280795 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f659c15dd00 0x7f659c0b33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:03.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.191+0000 7f659affd700 1 -- 172.21.15.143:0/3911280795 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f659c15dd00 msgr2=0x7f659c0b33b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:03.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.191+0000 7f659affd700 1 --2- 172.21.15.143:0/3911280795 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f659c15dd00 0x7f659c0b33b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:03.246 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.191+0000 7f659affd700 1 -- 172.21.15.143:0/3911280795 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f659c1618f0 msgr2=0x7f659c1ffde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:03.246 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.191+0000 7f659affd700 1 --2- 172.21.15.143:0/3911280795 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f659c1618f0 0x7f659c1ffde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:03.246 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.191+0000 7f659affd700 1 -- 172.21.15.143:0/3911280795 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65900094a0 con 0x7f659c06b150 2023-11-08T19:25:03.246 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.192+0000 7f659affd700 1 --2- 172.21.15.143:0/3911280795 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f659c06b150 0x7f659c2041e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f658c00c750 tx=0x7f658c00ca60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:03.246 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.192+0000 7f65837fe700 1 -- 172.21.15.143:0/3911280795 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f658c004c80 con 0x7f659c06b150 2023-11-08T19:25:03.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.192+0000 7f65837fe700 1 -- 172.21.15.143:0/3911280795 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f658c004e00 con 0x7f659c06b150 2023-11-08T19:25:03.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.192+0000 7f65837fe700 1 -- 172.21.15.143:0/3911280795 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f658c007310 con 0x7f659c06b150 2023-11-08T19:25:03.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.192+0000 7f65a17ab700 1 -- 172.21.15.143:0/3911280795 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f659c15e260 con 0x7f659c06b150 2023-11-08T19:25:03.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.192+0000 7f65a17ab700 1 -- 172.21.15.143:0/3911280795 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f659c204720 con 0x7f659c06b150 2023-11-08T19:25:03.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.195+0000 7f65837fe700 1 -- 172.21.15.143:0/3911280795 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f658c007470 con 0x7f659c06b150 2023-11-08T19:25:03.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.196+0000 7f65837fe700 1 --2- 172.21.15.143:0/3911280795 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f658406b130 0x7f658406d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:03.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.196+0000 7f65837fe700 1 -- 172.21.15.143:0/3911280795 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f658c0892e0 con 0x7f659c06b150 2023-11-08T19:25:03.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.201+0000 7f659a7fc700 1 --2- 172.21.15.143:0/3911280795 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f658406b130 0x7f658406d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:03.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.201+0000 7f659a7fc700 1 --2- 172.21.15.143:0/3911280795 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f658406b130 0x7f658406d5f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f65900090b0 tx=0x7f6590008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:03.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.207+0000 7f65a17ab700 1 -- 172.21.15.143:0/3911280795 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f657c0052f0 con 0x7f659c06b150 2023-11-08T19:25:03.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.212+0000 7f65837fe700 1 -- 172.21.15.143:0/3911280795 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f658c054400 con 0x7f659c06b150 2023-11-08T19:25:03.500 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.491+0000 7f0928b6d700 1 -- 172.21.15.143:0/3084546754 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f09080055a0 con 0x7f09240b4e00 2023-11-08T19:25:03.501 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.499+0000 7f0912ffd700 1 -- 172.21.15.143:0/3084546754 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f09180596a0 con 0x7f09240b4e00 2023-11-08T19:25:03.504 INFO:teuthology.orchestra.run.smithi143.stdout:55834574855 2023-11-08T19:25:03.505 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.503+0000 7f0928b6d700 1 -- 172.21.15.143:0/3084546754 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f090c06b150 msgr2=0x7f090c06d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:03.505 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.503+0000 7f0928b6d700 1 --2- 172.21.15.143:0/3084546754 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f090c06b150 0x7f090c06d610 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f091c007eb0 tx=0x7f091c0073c0 comp rx=0 tx=0).stop 2023-11-08T19:25:03.505 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.503+0000 7f0928b6d700 1 -- 172.21.15.143:0/3084546754 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f09240b4e00 msgr2=0x7f09240c8890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:03.505 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.503+0000 7f0928b6d700 1 --2- 172.21.15.143:0/3084546754 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f09240b4e00 0x7f09240c8890 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f0924072360 tx=0x7f0918004750 comp rx=0 tx=0).stop 2023-11-08T19:25:03.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.504+0000 7f0928b6d700 1 -- 172.21.15.143:0/3084546754 shutdown_connections 2023-11-08T19:25:03.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.504+0000 7f0928b6d700 1 --2- 172.21.15.143:0/3084546754 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f090c06b150 0x7f090c06d610 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:03.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.504+0000 7f0928b6d700 1 --2- 172.21.15.143:0/3084546754 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f09240b1370 0x7f09240b2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:03.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.504+0000 7f0928b6d700 1 --2- 172.21.15.143:0/3084546754 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f09240b4e00 0x7f09240c8890 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:03.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.504+0000 7f0928b6d700 1 --2- 172.21.15.143:0/3084546754 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f092416e000 0x7f0924089680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:03.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.504+0000 7f0928b6d700 1 -- 172.21.15.143:0/3084546754 >> 172.21.15.143:0/3084546754 conn(0x7f092406b060 msgr2=0x7f092416b6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:03.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.504+0000 7f0928b6d700 1 -- 172.21.15.143:0/3084546754 shutdown_connections 2023-11-08T19:25:03.508 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.504+0000 7f0928b6d700 1 -- 172.21.15.143:0/3084546754 wait complete. 2023-11-08T19:25:03.510 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.509+0000 7f1454ff8700 1 -- 172.21.15.143:0/108331573 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f14501630c0 con 0x7f1450165c50 2023-11-08T19:25:03.511 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.510+0000 7f14477fe700 1 -- 172.21.15.143:0/108331573 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f14501630c0 con 0x7f1450165c50 2023-11-08T19:25:03.514 INFO:teuthology.orchestra.run.smithi143.stdout:55834574855 2023-11-08T19:25:03.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.514+0000 7f1454ff8700 1 -- 172.21.15.143:0/108331573 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f143c06b010 msgr2=0x7f143c06d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:03.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.514+0000 7f1454ff8700 1 --2- 172.21.15.143:0/108331573 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f143c06b010 0x7f143c06d4d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f14500be630 tx=0x7f1440011000 comp rx=0 tx=0).stop 2023-11-08T19:25:03.516 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.514+0000 7f1454ff8700 1 -- 172.21.15.143:0/108331573 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1450165c50 msgr2=0x7f14500b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:03.516 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.514+0000 7f1454ff8700 1 --2- 172.21.15.143:0/108331573 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1450165c50 0x7f14500b4d60 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f144800c3c0 tx=0x7f144800c780 comp rx=0 tx=0).stop 2023-11-08T19:25:03.517 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.514+0000 7f1454ff8700 1 -- 172.21.15.143:0/108331573 shutdown_connections 2023-11-08T19:25:03.517 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.514+0000 7f1454ff8700 1 --2- 172.21.15.143:0/108331573 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f143c06b010 0x7f143c06d4d0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:03.517 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.514+0000 7f1454ff8700 1 --2- 172.21.15.143:0/108331573 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f14500b1100 0x7f145015e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:03.517 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.514+0000 7f1454ff8700 1 --2- 172.21.15.143:0/108331573 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1450165c50 0x7f14500b4d60 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:03.517 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.514+0000 7f1454ff8700 1 --2- 172.21.15.143:0/108331573 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1450162060 0x7f14500b4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:03.518 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.514+0000 7f1454ff8700 1 -- 172.21.15.143:0/108331573 >> 172.21.15.143:0/108331573 conn(0x7f145006b4c0 msgr2=0x7f1450160840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:03.518 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.515+0000 7f1454ff8700 1 -- 172.21.15.143:0/108331573 shutdown_connections 2023-11-08T19:25:03.518 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.515+0000 7f1454ff8700 1 -- 172.21.15.143:0/108331573 wait complete. 2023-11-08T19:25:03.526 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.0 2023-11-08T19:25:03.526 DEBUG:teuthology.parallel:result is None 2023-11-08T19:25:03.534 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.6 2023-11-08T19:25:03.535 DEBUG:teuthology.parallel:result is None 2023-11-08T19:25:03.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.536+0000 7f40e5a16700 1 -- 172.21.15.143:0/974277949 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f40e00c6470 con 0x7f40e0162060 2023-11-08T19:25:03.538 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.536+0000 7f40c7fff700 1 -- 172.21.15.143:0/974277949 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f40e00c6470 con 0x7f40e0162060 2023-11-08T19:25:03.538 INFO:teuthology.orchestra.run.smithi143.stdout:55834574855 2023-11-08T19:25:03.541 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.540+0000 7f40e5a16700 1 -- 172.21.15.143:0/974277949 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f40cc06af00 msgr2=0x7f40cc06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:03.541 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.540+0000 7f40e5a16700 1 --2- 172.21.15.143:0/974277949 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f40cc06af00 0x7f40cc06d3c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f40d80012e0 tx=0x7f40d8001590 comp rx=0 tx=0).stop 2023-11-08T19:25:03.541 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.540+0000 7f40e5a16700 1 -- 172.21.15.143:0/974277949 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f40e0162060 msgr2=0x7f40e00c50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:03.541 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.540+0000 7f40e5a16700 1 --2- 172.21.15.143:0/974277949 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f40e0162060 0x7f40e00c50a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f40d40090b0 tx=0x7f40d4004750 comp rx=0 tx=0).stop 2023-11-08T19:25:03.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.541+0000 7f40e5a16700 1 -- 172.21.15.143:0/974277949 shutdown_connections 2023-11-08T19:25:03.543 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.541+0000 7f40e5a16700 1 --2- 172.21.15.143:0/974277949 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f40cc06af00 0x7f40cc06d3c0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:03.543 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.541+0000 7f40e5a16700 1 --2- 172.21.15.143:0/974277949 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f40e006d420 0x7f40e014d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:03.543 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.541+0000 7f40e5a16700 1 --2- 172.21.15.143:0/974277949 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f40e0162060 0x7f40e00c50a0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:03.543 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.541+0000 7f40e5a16700 1 --2- 172.21.15.143:0/974277949 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f40e00b1810 0x7f40e00c4b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:03.544 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.541+0000 7f40e5a16700 1 -- 172.21.15.143:0/974277949 >> 172.21.15.143:0/974277949 conn(0x7f40e006b4c0 msgr2=0x7f40e0160b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:03.544 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.541+0000 7f40e5a16700 1 -- 172.21.15.143:0/974277949 shutdown_connections 2023-11-08T19:25:03.544 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.541+0000 7f40e5a16700 1 -- 172.21.15.143:0/974277949 wait complete. 2023-11-08T19:25:03.546 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.545+0000 7f1b7892f700 1 -- 172.21.15.143:0/610929395 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f1b740ccf40 con 0x7f1b740b1810 2023-11-08T19:25:03.546 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.545+0000 7f1b62ffd700 1 -- 172.21.15.143:0/610929395 <== mon.2 v2:172.21.15.174:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1b68007020 con 0x7f1b740b1810 2023-11-08T19:25:03.547 INFO:teuthology.orchestra.run.smithi143.stdout:55834574855 2023-11-08T19:25:03.550 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.549+0000 7f1b7892f700 1 -- 172.21.15.143:0/610929395 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f1b5c06b3a0 msgr2=0x7f1b5c06d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:03.550 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.549+0000 7f1b7892f700 1 --2- 172.21.15.143:0/610929395 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f1b5c06b3a0 0x7f1b5c06d860 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1b6400ad60 tx=0x7f1b6400e000 comp rx=0 tx=0).stop 2023-11-08T19:25:03.550 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.549+0000 7f1b7892f700 1 -- 172.21.15.143:0/610929395 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1b740b1810 msgr2=0x7f1b7416ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:03.551 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.549+0000 7f1b7892f700 1 --2- 172.21.15.143:0/610929395 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1b740b1810 0x7f1b7416ab80 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1b6800c990 tx=0x7f1b6800cca0 comp rx=0 tx=0).stop 2023-11-08T19:25:03.551 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.549+0000 7f1b7892f700 1 -- 172.21.15.143:0/610929395 shutdown_connections 2023-11-08T19:25:03.551 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.549+0000 7f1b7892f700 1 --2- 172.21.15.143:0/610929395 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f1b5c06b3a0 0x7f1b5c06d860 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:03.552 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.549+0000 7f1b7892f700 1 --2- 172.21.15.143:0/610929395 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1b740b1810 0x7f1b7416ab80 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:03.552 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.549+0000 7f1b7892f700 1 --2- 172.21.15.143:0/610929395 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1b7420d410 0x7f1b740b4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:03.552 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.549+0000 7f1b7892f700 1 --2- 172.21.15.143:0/610929395 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1b7416d8f0 0x7f1b7420ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:03.552 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.549+0000 7f1b7892f700 1 -- 172.21.15.143:0/610929395 >> 172.21.15.143:0/610929395 conn(0x7f1b7406b060 msgr2=0x7f1b7416dd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:03.553 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.549+0000 7fdcac4bc700 1 -- 172.21.15.143:0/3889314570 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fdca40cb040 con 0x7fdca40ce980 2023-11-08T19:25:03.553 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.549+0000 7f1b7892f700 1 -- 172.21.15.143:0/610929395 shutdown_connections 2023-11-08T19:25:03.553 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.550+0000 7f1b7892f700 1 -- 172.21.15.143:0/610929395 wait complete. 2023-11-08T19:25:03.554 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.550+0000 7fdc9affd700 1 -- 172.21.15.143:0/3889314570 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fdca40cb040 con 0x7fdca40ce980 2023-11-08T19:25:03.554 INFO:teuthology.orchestra.run.smithi143.stdout:55834574855 2023-11-08T19:25:03.557 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.552+0000 7fdcac4bc700 1 -- 172.21.15.143:0/3889314570 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fdc9006b010 msgr2=0x7fdc9006d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:03.558 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.552+0000 7fdcac4bc700 1 --2- 172.21.15.143:0/3889314570 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fdc9006b010 0x7fdc9006d4d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fdca40d3a90 tx=0x7fdc94008030 comp rx=0 tx=0).stop 2023-11-08T19:25:03.558 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.552+0000 7fdcac4bc700 1 -- 172.21.15.143:0/3889314570 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdca40ce980 msgr2=0x7fdca40b3610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:03.559 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.552+0000 7fdcac4bc700 1 --2- 172.21.15.143:0/3889314570 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdca40ce980 0x7fdca40b3610 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fdca000e910 tx=0x7fdca000ec20 comp rx=0 tx=0).stop 2023-11-08T19:25:03.559 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.552+0000 7fdcac4bc700 1 -- 172.21.15.143:0/3889314570 shutdown_connections 2023-11-08T19:25:03.559 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.552+0000 7fdcac4bc700 1 --2- 172.21.15.143:0/3889314570 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fdc9006b010 0x7fdc9006d4d0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:03.559 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.552+0000 7fdcac4bc700 1 --2- 172.21.15.143:0/3889314570 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdca415dc80 0x7fdca40cd490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:03.559 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.552+0000 7fdcac4bc700 1 --2- 172.21.15.143:0/3889314570 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdca40ce980 0x7fdca40b3610 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:03.560 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.552+0000 7fdcac4bc700 1 --2- 172.21.15.143:0/3889314570 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdca40d3690 0x7fdca41fd370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:03.560 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.552+0000 7fdcac4bc700 1 -- 172.21.15.143:0/3889314570 >> 172.21.15.143:0/3889314570 conn(0x7fdca40668e0 msgr2=0x7fdca415bde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:03.560 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.557+0000 7fdcac4bc700 1 -- 172.21.15.143:0/3889314570 shutdown_connections 2023-11-08T19:25:03.561 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.557+0000 7fdcac4bc700 1 -- 172.21.15.143:0/3889314570 wait complete. 2023-11-08T19:25:03.561 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.560+0000 7fc192da1700 1 -- 172.21.15.143:0/1574468572 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fc16c0055a0 con 0x7fc18c0cf090 2023-11-08T19:25:03.562 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.7 2023-11-08T19:25:03.563 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.560+0000 7fc1897fa700 1 -- 172.21.15.143:0/1574468572 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc184059550 con 0x7fc18c0cf090 2023-11-08T19:25:03.563 INFO:teuthology.orchestra.run.smithi143.stdout:55834574856 2023-11-08T19:25:03.563 DEBUG:teuthology.parallel:result is None 2023-11-08T19:25:03.564 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.562+0000 7fc192da1700 1 -- 172.21.15.143:0/1574468572 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc17406b090 msgr2=0x7fc17406d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:03.564 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.562+0000 7fc192da1700 1 --2- 172.21.15.143:0/1574468572 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc17406b090 0x7fc17406d550 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc17c0013c0 tx=0x7fc17c001350 comp rx=0 tx=0).stop 2023-11-08T19:25:03.565 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.562+0000 7fc192da1700 1 -- 172.21.15.143:0/1574468572 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc18c0cf090 msgr2=0x7fc18c152150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:03.565 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.562+0000 7fc192da1700 1 --2- 172.21.15.143:0/1574468572 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc18c0cf090 0x7fc18c152150 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fc184015000 tx=0x7fc184004ad0 comp rx=0 tx=0).stop 2023-11-08T19:25:03.565 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.563+0000 7fc192da1700 1 -- 172.21.15.143:0/1574468572 shutdown_connections 2023-11-08T19:25:03.565 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.563+0000 7fc192da1700 1 --2- 172.21.15.143:0/1574468572 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc17406b090 0x7fc17406d550 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:03.566 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.563+0000 7fc192da1700 1 --2- 172.21.15.143:0/1574468572 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc18c15dc80 0x7fc18c155380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:03.566 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.563+0000 7fc192da1700 1 --2- 172.21.15.143:0/1574468572 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc18c0cf090 0x7fc18c152150 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:03.566 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.563+0000 7fc192da1700 1 --2- 172.21.15.143:0/1574468572 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc18c0d3690 0x7fc18c156b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:03.566 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.563+0000 7fc192da1700 1 -- 172.21.15.143:0/1574468572 >> 172.21.15.143:0/1574468572 conn(0x7fc18c0668e0 msgr2=0x7fc18c15a9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:03.566 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.563+0000 7fc192da1700 1 -- 172.21.15.143:0/1574468572 shutdown_connections 2023-11-08T19:25:03.567 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.563+0000 7fc192da1700 1 -- 172.21.15.143:0/1574468572 wait complete. 2023-11-08T19:25:03.569 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.567+0000 7f756080e700 1 -- 172.21.15.143:0/3586610471 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f755c0c3370 con 0x7f755c15ce10 2023-11-08T19:25:03.569 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.568+0000 7f754affd700 1 -- 172.21.15.143:0/3586610471 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f754c02f020 con 0x7f755c15ce10 2023-11-08T19:25:03.569 INFO:teuthology.orchestra.run.smithi143.stdout:55834574855 2023-11-08T19:25:03.572 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.5 2023-11-08T19:25:03.572 DEBUG:teuthology.parallel:result is None 2023-11-08T19:25:03.573 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.572+0000 7f756080e700 1 -- 172.21.15.143:0/3586610471 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f753c06af00 msgr2=0x7f753c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:03.573 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.572+0000 7f756080e700 1 --2- 172.21.15.143:0/3586610471 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f753c06af00 0x7f753c06d3c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f755c0c9ba0 tx=0x7f7550007000 comp rx=0 tx=0).stop 2023-11-08T19:25:03.573 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.572+0000 7f756080e700 1 -- 172.21.15.143:0/3586610471 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f755c15ce10 msgr2=0x7f755c0c3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:03.573 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.572+0000 7f756080e700 1 --2- 172.21.15.143:0/3586610471 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f755c15ce10 0x7f755c0c3650 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f754c00ae90 tx=0x7f754c00cad0 comp rx=0 tx=0).stop 2023-11-08T19:25:03.574 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.573+0000 7f756080e700 1 -- 172.21.15.143:0/3586610471 shutdown_connections 2023-11-08T19:25:03.574 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.573+0000 7f756080e700 1 --2- 172.21.15.143:0/3586610471 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f753c06af00 0x7f753c06d3c0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:03.575 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.573+0000 7f756080e700 1 --2- 172.21.15.143:0/3586610471 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f755c15d1f0 0x7f755c161060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:03.575 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.573+0000 7f756080e700 1 --2- 172.21.15.143:0/3586610471 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f755c15ce10 0x7f755c0c3650 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:03.575 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.573+0000 7f756080e700 1 --2- 172.21.15.143:0/3586610471 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f755c0c3b90 0x7f755c160b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:03.575 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.573+0000 7f756080e700 1 -- 172.21.15.143:0/3586610471 >> 172.21.15.143:0/3586610471 conn(0x7f755c06b4c0 msgr2=0x7f755c0cd1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:03.575 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.573+0000 7f756080e700 1 -- 172.21.15.143:0/3586610471 shutdown_connections 2023-11-08T19:25:03.576 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.573+0000 7f756080e700 1 -- 172.21.15.143:0/3586610471 wait complete. 2023-11-08T19:25:03.576 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.4 2023-11-08T19:25:03.577 DEBUG:teuthology.parallel:result is None 2023-11-08T19:25:03.579 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.3 2023-11-08T19:25:03.579 DEBUG:teuthology.parallel:result is None 2023-11-08T19:25:03.592 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.1 2023-11-08T19:25:03.593 DEBUG:teuthology.parallel:result is None 2023-11-08T19:25:03.593 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.591+0000 7f65a17ab700 1 -- 172.21.15.143:0/3911280795 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f657c0055a0 con 0x7f659c06b150 2023-11-08T19:25:03.594 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.592+0000 7f65837fe700 1 -- 172.21.15.143:0/3911280795 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f658c057a20 con 0x7f659c06b150 2023-11-08T19:25:03.594 INFO:teuthology.orchestra.run.smithi143.stdout:55834574855 2023-11-08T19:25:03.596 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.594+0000 7f65a17ab700 1 -- 172.21.15.143:0/3911280795 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f658406b130 msgr2=0x7f658406d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:03.596 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.594+0000 7f65a17ab700 1 --2- 172.21.15.143:0/3911280795 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f658406b130 0x7f658406d5f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f65900090b0 tx=0x7f6590008030 comp rx=0 tx=0).stop 2023-11-08T19:25:03.596 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.594+0000 7f65a17ab700 1 -- 172.21.15.143:0/3911280795 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f659c06b150 msgr2=0x7f659c2041e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:03.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.594+0000 7f65a17ab700 1 --2- 172.21.15.143:0/3911280795 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f659c06b150 0x7f659c2041e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f658c00c750 tx=0x7f658c00ca60 comp rx=0 tx=0).stop 2023-11-08T19:25:03.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.594+0000 7f65a17ab700 1 -- 172.21.15.143:0/3911280795 shutdown_connections 2023-11-08T19:25:03.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.594+0000 7f65a17ab700 1 --2- 172.21.15.143:0/3911280795 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f658406b130 0x7f658406d5f0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:03.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.594+0000 7f65a17ab700 1 --2- 172.21.15.143:0/3911280795 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f659c1618f0 0x7f659c1ffde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:03.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.594+0000 7f65a17ab700 1 --2- 172.21.15.143:0/3911280795 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f659c06b150 0x7f659c2041e0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:03.598 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.594+0000 7f65a17ab700 1 --2- 172.21.15.143:0/3911280795 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f659c15dd00 0x7f659c0b33b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:03.598 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.594+0000 7f65a17ab700 1 -- 172.21.15.143:0/3911280795 >> 172.21.15.143:0/3911280795 conn(0x7f659c099ab0 msgr2=0x7f659c0aef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:03.598 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.594+0000 7f65a17ab700 1 -- 172.21.15.143:0/3911280795 shutdown_connections 2023-11-08T19:25:03.598 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.594+0000 7f65a17ab700 1 -- 172.21.15.143:0/3911280795 wait complete. 2023-11-08T19:25:03.607 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.2 2023-11-08T19:25:03.608 DEBUG:teuthology.parallel:result is None 2023-11-08T19:25:03.608 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-11-08T19:25:03.608 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-08T19:25:03.769 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.767+0000 7fc3a941d700 1 Processor -- start 2023-11-08T19:25:03.769 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.768+0000 7fc3a941d700 1 -- start start 2023-11-08T19:25:03.772 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.768+0000 7fc3a941d700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fc3a415dc80 0x7fc3a40d1f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:03.772 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.768+0000 7fc3a941d700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc3a40d3690 0x7fc3a40b9690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:03.772 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.768+0000 7fc3a941d700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3a40b7cf0 con 0x7fc3a40ce980 2023-11-08T19:25:03.772 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.768+0000 7fc3a941d700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3a40b9bd0 con 0x7fc3a40d3690 2023-11-08T19:25:03.772 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.768+0000 7fc3a941d700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3a40d3200 con 0x7fc3a415dc80 2023-11-08T19:25:03.773 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.768+0000 7fc3a27fc700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc3a40d3690 0x7fc3a40b9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:03.773 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.769+0000 7fc3a27fc700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc3a40d3690 0x7fc3a40b9690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49154/0 (socket says 172.21.15.143:49154) 2023-11-08T19:25:03.773 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.769+0000 7fc3a27fc700 1 -- 172.21.15.143:0/2550286746 learned_addr learned my addr 172.21.15.143:0/2550286746 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:03.773 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.769+0000 7fc3a2ffd700 1 --2- 172.21.15.143:0/2550286746 >> v2:172.21.15.174:3300/0 conn(0x7fc3a415dc80 0x7fc3a40d1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:03.773 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.769+0000 7fc3a27fc700 1 -- 172.21.15.143:0/2550286746 >> v2:172.21.15.174:3300/0 conn(0x7fc3a415dc80 msgr2=0x7fc3a40d1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:03.774 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.769+0000 7fc3a27fc700 1 --2- 172.21.15.143:0/2550286746 >> v2:172.21.15.174:3300/0 conn(0x7fc3a415dc80 0x7fc3a40d1f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:03.774 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.769+0000 7fc3a27fc700 1 -- 172.21.15.143:0/2550286746 >> v1:172.21.15.174:6789/0 conn(0x7fc3a40ce980 legacy=0x7fc3a40b5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:03.774 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.769+0000 7fc3a27fc700 1 -- 172.21.15.143:0/2550286746 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3a415e060 con 0x7fc3a40d3690 2023-11-08T19:25:03.775 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.769+0000 7fc3a2ffd700 1 --2- 172.21.15.143:0/2550286746 >> v2:172.21.15.174:3300/0 conn(0x7fc3a415dc80 0x7fc3a40d1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:25:03.775 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.769+0000 7fc3a27fc700 1 --2- 172.21.15.143:0/2550286746 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc3a40d3690 0x7fc3a40b9690 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc3a40b6710 tx=0x7fc394009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc744b729c3cc8ce server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:03.775 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.769+0000 7fc3a1ffb700 1 -- 172.21.15.143:0/2550286746 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc394004230 con 0x7fc3a40d3690 2023-11-08T19:25:03.775 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.769+0000 7fc3a1ffb700 1 -- 172.21.15.143:0/2550286746 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc394004390 con 0x7fc3a40d3690 2023-11-08T19:25:03.775 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.769+0000 7fc3a1ffb700 1 -- 172.21.15.143:0/2550286746 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3940036a0 con 0x7fc3a40d3690 2023-11-08T19:25:03.775 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.770+0000 7fc3a941d700 1 -- 172.21.15.143:0/2550286746 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc3a40d3690 msgr2=0x7fc3a40b9690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:03.776 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.770+0000 7fc3a941d700 1 --2- 172.21.15.143:0/2550286746 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc3a40d3690 0x7fc3a40b9690 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc3a40b6710 tx=0x7fc394009bd0 comp rx=0 tx=0).stop 2023-11-08T19:25:03.776 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.770+0000 7fc3a941d700 1 -- 172.21.15.143:0/2550286746 shutdown_connections 2023-11-08T19:25:03.776 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.770+0000 7fc3a941d700 1 --2- 172.21.15.143:0/2550286746 >> v2:172.21.15.174:3300/0 conn(0x7fc3a415dc80 0x7fc3a40d1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:03.776 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.770+0000 7fc3a941d700 1 --2- 172.21.15.143:0/2550286746 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc3a40d3690 0x7fc3a40b9690 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:03.776 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.770+0000 7fc3a941d700 1 -- 172.21.15.143:0/2550286746 >> 172.21.15.143:0/2550286746 conn(0x7fc3a40668e0 msgr2=0x7fc3a4066cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:03.777 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.770+0000 7fc3a941d700 1 -- 172.21.15.143:0/2550286746 shutdown_connections 2023-11-08T19:25:03.777 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.770+0000 7fc3a941d700 1 -- 172.21.15.143:0/2550286746 wait complete. 2023-11-08T19:25:03.780 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.771+0000 7fc3a941d700 1 Processor -- start 2023-11-08T19:25:03.780 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.771+0000 7fc3a941d700 1 -- start start 2023-11-08T19:25:03.780 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.772+0000 7fc3a941d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc3a40ce980 0x7fc3a40c9310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:03.780 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.772+0000 7fc3a941d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc3a40d3690 0x7fc3a40b3610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:03.780 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.772+0000 7fc3a941d700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc3a415dc80 0x7fc3a414fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:03.781 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.772+0000 7fc3a941d700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a40cd9a0 con 0x7fc3a40d3690 2023-11-08T19:25:03.781 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.772+0000 7fc3a941d700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a40b3d10 con 0x7fc3a40ce980 2023-11-08T19:25:03.781 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.772+0000 7fc3a941d700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a40b3ff0 con 0x7fc3a415dc80 2023-11-08T19:25:03.781 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.773+0000 7fc3a27fc700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc3a40d3690 0x7fc3a40b3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:03.782 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.773+0000 7fc3a27fc700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc3a40d3690 0x7fc3a40b3610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49160/0 (socket says 172.21.15.143:49160) 2023-11-08T19:25:03.782 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.773+0000 7fc3a27fc700 1 -- 172.21.15.143:0/450447653 learned_addr learned my addr 172.21.15.143:0/450447653 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:03.782 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.773+0000 7fc3a2ffd700 1 --2- 172.21.15.143:0/450447653 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc3a40ce980 0x7fc3a40c9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:03.782 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.773+0000 7fc3a37fe700 1 --2- 172.21.15.143:0/450447653 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc3a415dc80 0x7fc3a414fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:03.782 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.773+0000 7fc3a27fc700 1 -- 172.21.15.143:0/450447653 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc3a415dc80 msgr2=0x7fc3a414fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:03.783 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.773+0000 7fc3a27fc700 1 --2- 172.21.15.143:0/450447653 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc3a415dc80 0x7fc3a414fbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:03.783 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.773+0000 7fc3a27fc700 1 -- 172.21.15.143:0/450447653 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc3a40ce980 msgr2=0x7fc3a40c9310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:03.783 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.773+0000 7fc3a27fc700 1 --2- 172.21.15.143:0/450447653 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc3a40ce980 0x7fc3a40c9310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:03.783 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.773+0000 7fc3a27fc700 1 -- 172.21.15.143:0/450447653 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc394009550 con 0x7fc3a40d3690 2023-11-08T19:25:03.783 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.773+0000 7fc3a27fc700 1 --2- 172.21.15.143:0/450447653 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc3a40d3690 0x7fc3a40b3610 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc394004000 tx=0x7fc394004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:03.784 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.773+0000 7fc38b7fe700 1 -- 172.21.15.143:0/450447653 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc394003d80 con 0x7fc3a40d3690 2023-11-08T19:25:03.784 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.773+0000 7fc38b7fe700 1 -- 172.21.15.143:0/450447653 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc39401f070 con 0x7fc3a40d3690 2023-11-08T19:25:03.784 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.773+0000 7fc3a941d700 1 -- 172.21.15.143:0/450447653 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3a40b42d0 con 0x7fc3a40d3690 2023-11-08T19:25:03.784 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.773+0000 7fc38b7fe700 1 -- 172.21.15.143:0/450447653 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc394011870 con 0x7fc3a40d3690 2023-11-08T19:25:03.784 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.773+0000 7fc3a941d700 1 -- 172.21.15.143:0/450447653 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3a40b4890 con 0x7fc3a40d3690 2023-11-08T19:25:03.785 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.774+0000 7fc3a941d700 1 -- 172.21.15.143:0/450447653 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3a404eee0 con 0x7fc3a40d3690 2023-11-08T19:25:03.785 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.774+0000 7fc38b7fe700 1 -- 172.21.15.143:0/450447653 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc394021030 con 0x7fc3a40d3690 2023-11-08T19:25:03.785 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.775+0000 7fc38b7fe700 1 --2- 172.21.15.143:0/450447653 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc39006af00 0x7fc39006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:03.785 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.775+0000 7fc38b7fe700 1 -- 172.21.15.143:0/450447653 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc39408a9f0 con 0x7fc3a40d3690 2023-11-08T19:25:03.785 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.775+0000 7fc3a2ffd700 1 --2- 172.21.15.143:0/450447653 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc39006af00 0x7fc39006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:03.786 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.775+0000 7fc3a2ffd700 1 --2- 172.21.15.143:0/450447653 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc39006af00 0x7fc39006d3c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc398006fd0 tx=0x7fc398008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:03.786 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.779+0000 7fc38b7fe700 1 -- 172.21.15.143:0/450447653 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc394055b30 con 0x7fc3a40d3690 2023-11-08T19:25:03.965 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.964+0000 7fc3a941d700 1 -- 172.21.15.143:0/450447653 --> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc3a40d28d0 con 0x7fc39006af00 2023-11-08T19:25:03.972 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.971+0000 7fc38b7fe700 1 -- 172.21.15.143:0/450447653 <== mgr.4099 v2:172.21.15.174:6832/4037280178 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264590 (secure 0 0 0) 0x7fc39800c050 con 0x7fc39006af00 2023-11-08T19:25:03.976 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:25:03.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.974+0000 7fc3a941d700 1 -- 172.21.15.143:0/450447653 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc39006af00 msgr2=0x7fc39006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:03.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.974+0000 7fc3a941d700 1 --2- 172.21.15.143:0/450447653 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc39006af00 0x7fc39006d3c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc398006fd0 tx=0x7fc398008040 comp rx=0 tx=0).stop 2023-11-08T19:25:03.979 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.974+0000 7fc3a941d700 1 -- 172.21.15.143:0/450447653 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc3a40d3690 msgr2=0x7fc3a40b3610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:03.979 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.974+0000 7fc3a941d700 1 --2- 172.21.15.143:0/450447653 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc3a40d3690 0x7fc3a40b3610 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc394004000 tx=0x7fc394004750 comp rx=0 tx=0).stop 2023-11-08T19:25:03.979 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.974+0000 7fc3a941d700 1 -- 172.21.15.143:0/450447653 shutdown_connections 2023-11-08T19:25:03.979 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.974+0000 7fc3a941d700 1 --2- 172.21.15.143:0/450447653 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc39006af00 0x7fc39006d3c0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:03.980 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.974+0000 7fc3a941d700 1 --2- 172.21.15.143:0/450447653 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc3a415dc80 0x7fc3a414fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:03.980 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.974+0000 7fc3a941d700 1 --2- 172.21.15.143:0/450447653 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc3a40ce980 0x7fc3a40c9310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:03.980 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.974+0000 7fc3a941d700 1 --2- 172.21.15.143:0/450447653 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc3a40d3690 0x7fc3a40b3610 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:03.980 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.974+0000 7fc3a941d700 1 -- 172.21.15.143:0/450447653 >> 172.21.15.143:0/450447653 conn(0x7fc3a40668e0 msgr2=0x7fc3a4066cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:03.980 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.974+0000 7fc3a941d700 1 -- 172.21.15.143:0/450447653 shutdown_connections 2023-11-08T19:25:03.980 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:03.974+0000 7fc3a941d700 1 -- 172.21.15.143:0/450447653 wait complete. 2023-11-08T19:25:03.981 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2023-11-08T19:25:03.988 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2023-11-08T19:25:02.511635+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219024,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755696,"statfs":{"total":773094113280,"available":772869832704,"internally_reserved":0,"allocated":2891776,"data_stored":1752836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.883726"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.631402+0000","last_change":"2023-11-08T19:24:50.618820+0000","last_active":"2023-11-08T19:24:51.631402+0000","last_peered":"2023-11-08T19:24:51.631402+0000","last_clean":"2023-11-08T19:24:51.631402+0000","last_became_active":"2023-11-08T19:24:50.618531+0000","last_became_peered":"2023-11-08T19:24:50.618531+0000","last_unstale":"2023-11-08T19:24:51.631402+0000","last_undegraded":"2023-11-08T19:24:51.631402+0000","last_fullsized":"2023-11-08T19:24:51.631402+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:33:19.902860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.805606+0000","last_change":"2023-11-08T19:24:50.623985+0000","last_active":"2023-11-08T19:24:52.805606+0000","last_peered":"2023-11-08T19:24:52.805606+0000","last_clean":"2023-11-08T19:24:52.805606+0000","last_became_active":"2023-11-08T19:24:50.623553+0000","last_became_peered":"2023-11-08T19:24:50.623553+0000","last_unstale":"2023-11-08T19:24:52.805606+0000","last_undegraded":"2023-11-08T19:24:52.805606+0000","last_fullsized":"2023-11-08T19:24:52.805606+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:13:58.948371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.309774+0000","last_change":"2023-11-08T19:24:50.617914+0000","last_active":"2023-11-08T19:24:52.309774+0000","last_peered":"2023-11-08T19:24:52.309774+0000","last_clean":"2023-11-08T19:24:52.309774+0000","last_became_active":"2023-11-08T19:24:50.617611+0000","last_became_peered":"2023-11-08T19:24:50.617611+0000","last_unstale":"2023-11-08T19:24:52.309774+0000","last_undegraded":"2023-11-08T19:24:52.309774+0000","last_fullsized":"2023-11-08T19:24:52.309774+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:30:36.041820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.352504+0000","last_change":"2023-11-08T19:24:50.640039+0000","last_active":"2023-11-08T19:24:52.352504+0000","last_peered":"2023-11-08T19:24:52.352504+0000","last_clean":"2023-11-08T19:24:52.352504+0000","last_became_active":"2023-11-08T19:24:50.639560+0000","last_became_peered":"2023-11-08T19:24:50.639560+0000","last_unstale":"2023-11-08T19:24:52.352504+0000","last_undegraded":"2023-11-08T19:24:52.352504+0000","last_fullsized":"2023-11-08T19:24:52.352504+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:01:53.410545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.631276+0000","last_change":"2023-11-08T19:24:50.617837+0000","last_active":"2023-11-08T19:24:51.631276+0000","last_peered":"2023-11-08T19:24:51.631276+0000","last_clean":"2023-11-08T19:24:51.631276+0000","last_became_active":"2023-11-08T19:24:50.617537+0000","last_became_peered":"2023-11-08T19:24:50.617537+0000","last_unstale":"2023-11-08T19:24:51.631276+0000","last_undegraded":"2023-11-08T19:24:51.631276+0000","last_fullsized":"2023-11-08T19:24:51.631276+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:59:37.592538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.351953+0000","last_change":"2023-11-08T19:24:50.641005+0000","last_active":"2023-11-08T19:24:52.351953+0000","last_peered":"2023-11-08T19:24:52.351953+0000","last_clean":"2023-11-08T19:24:52.351953+0000","last_became_active":"2023-11-08T19:24:50.640727+0000","last_became_peered":"2023-11-08T19:24:50.640727+0000","last_unstale":"2023-11-08T19:24:52.351953+0000","last_undegraded":"2023-11-08T19:24:52.351953+0000","last_fullsized":"2023-11-08T19:24:52.351953+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:36:53.967796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.630219+0000","last_change":"2023-11-08T19:24:50.625311+0000","last_active":"2023-11-08T19:24:51.630219+0000","last_peered":"2023-11-08T19:24:51.630219+0000","last_clean":"2023-11-08T19:24:51.630219+0000","last_became_active":"2023-11-08T19:24:50.625012+0000","last_became_peered":"2023-11-08T19:24:50.625012+0000","last_unstale":"2023-11-08T19:24:51.630219+0000","last_undegraded":"2023-11-08T19:24:51.630219+0000","last_fullsized":"2023-11-08T19:24:51.630219+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:49:19.100370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.626351+0000","last_change":"2023-11-08T19:24:50.639325+0000","last_active":"2023-11-08T19:24:51.626351+0000","last_peered":"2023-11-08T19:24:51.626351+0000","last_clean":"2023-11-08T19:24:51.626351+0000","last_became_active":"2023-11-08T19:24:50.638993+0000","last_became_peered":"2023-11-08T19:24:50.638993+0000","last_unstale":"2023-11-08T19:24:51.626351+0000","last_undegraded":"2023-11-08T19:24:51.626351+0000","last_fullsized":"2023-11-08T19:24:51.626351+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:30:32.602976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.631383+0000","last_change":"2023-11-08T19:24:50.638920+0000","last_active":"2023-11-08T19:24:51.631383+0000","last_peered":"2023-11-08T19:24:51.631383+0000","last_clean":"2023-11-08T19:24:51.631383+0000","last_became_active":"2023-11-08T19:24:50.638607+0000","last_became_peered":"2023-11-08T19:24:50.638607+0000","last_unstale":"2023-11-08T19:24:51.631383+0000","last_undegraded":"2023-11-08T19:24:51.631383+0000","last_fullsized":"2023-11-08T19:24:51.631383+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:09:01.749686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.630901+0000","last_change":"2023-11-08T19:24:50.637631+0000","last_active":"2023-11-08T19:24:51.630901+0000","last_peered":"2023-11-08T19:24:51.630901+0000","last_clean":"2023-11-08T19:24:51.630901+0000","last_became_active":"2023-11-08T19:24:50.637346+0000","last_became_peered":"2023-11-08T19:24:50.637346+0000","last_unstale":"2023-11-08T19:24:51.630901+0000","last_undegraded":"2023-11-08T19:24:51.630901+0000","last_fullsized":"2023-11-08T19:24:51.630901+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:13:58.948371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.630498+0000","last_change":"2023-11-08T19:24:50.642227+0000","last_active":"2023-11-08T19:24:51.630498+0000","last_peered":"2023-11-08T19:24:51.630498+0000","last_clean":"2023-11-08T19:24:51.630498+0000","last_became_active":"2023-11-08T19:24:50.641955+0000","last_became_peered":"2023-11-08T19:24:50.641955+0000","last_unstale":"2023-11-08T19:24:51.630498+0000","last_undegraded":"2023-11-08T19:24:51.630498+0000","last_fullsized":"2023-11-08T19:24:51.630498+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:40:11.978983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.805767+0000","last_change":"2023-11-08T19:24:50.621511+0000","last_active":"2023-11-08T19:24:52.805767+0000","last_peered":"2023-11-08T19:24:52.805767+0000","last_clean":"2023-11-08T19:24:52.805767+0000","last_became_active":"2023-11-08T19:24:50.621142+0000","last_became_peered":"2023-11-08T19:24:50.621142+0000","last_unstale":"2023-11-08T19:24:52.805767+0000","last_undegraded":"2023-11-08T19:24:52.805767+0000","last_fullsized":"2023-11-08T19:24:52.805767+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:14:01.459274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.626625+0000","last_change":"2023-11-08T19:24:50.625632+0000","last_active":"2023-11-08T19:24:51.626625+0000","last_peered":"2023-11-08T19:24:51.626625+0000","last_clean":"2023-11-08T19:24:51.626625+0000","last_became_active":"2023-11-08T19:24:50.625370+0000","last_became_peered":"2023-11-08T19:24:50.625370+0000","last_unstale":"2023-11-08T19:24:51.626625+0000","last_undegraded":"2023-11-08T19:24:51.626625+0000","last_fullsized":"2023-11-08T19:24:51.626625+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:13:58.948371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.309818+0000","last_change":"2023-11-08T19:24:50.625350+0000","last_active":"2023-11-08T19:24:52.309818+0000","last_peered":"2023-11-08T19:24:52.309818+0000","last_clean":"2023-11-08T19:24:52.309818+0000","last_became_active":"2023-11-08T19:24:50.625078+0000","last_became_peered":"2023-11-08T19:24:50.625078+0000","last_unstale":"2023-11-08T19:24:52.309818+0000","last_undegraded":"2023-11-08T19:24:52.309818+0000","last_fullsized":"2023-11-08T19:24:52.309818+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:42:42.512786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.804817+0000","last_change":"2023-11-08T19:24:50.629187+0000","last_active":"2023-11-08T19:24:52.804817+0000","last_peered":"2023-11-08T19:24:52.804817+0000","last_clean":"2023-11-08T19:24:52.804817+0000","last_became_active":"2023-11-08T19:24:50.628898+0000","last_became_peered":"2023-11-08T19:24:50.628898+0000","last_unstale":"2023-11-08T19:24:52.804817+0000","last_undegraded":"2023-11-08T19:24:52.804817+0000","last_fullsized":"2023-11-08T19:24:52.804817+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:15:33.285449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.351906+0000","last_change":"2023-11-08T19:24:50.638579+0000","last_active":"2023-11-08T19:24:52.351906+0000","last_peered":"2023-11-08T19:24:52.351906+0000","last_clean":"2023-11-08T19:24:52.351906+0000","last_became_active":"2023-11-08T19:24:50.638297+0000","last_became_peered":"2023-11-08T19:24:50.638297+0000","last_unstale":"2023-11-08T19:24:52.351906+0000","last_undegraded":"2023-11-08T19:24:52.351906+0000","last_fullsized":"2023-11-08T19:24:52.351906+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:13:58.948371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.352572+0000","last_change":"2023-11-08T19:24:50.641446+0000","last_active":"2023-11-08T19:24:52.352572+0000","last_peered":"2023-11-08T19:24:52.352572+0000","last_clean":"2023-11-08T19:24:52.352572+0000","last_became_active":"2023-11-08T19:24:50.639709+0000","last_became_peered":"2023-11-08T19:24:50.639709+0000","last_unstale":"2023-11-08T19:24:52.352572+0000","last_undegraded":"2023-11-08T19:24:52.352572+0000","last_fullsized":"2023-11-08T19:24:52.352572+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:09:01.749686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.804922+0000","last_change":"2023-11-08T19:24:50.619241+0000","last_active":"2023-11-08T19:24:52.804922+0000","last_peered":"2023-11-08T19:24:52.804922+0000","last_clean":"2023-11-08T19:24:52.804922+0000","last_became_active":"2023-11-08T19:24:50.618962+0000","last_became_peered":"2023-11-08T19:24:50.618962+0000","last_unstale":"2023-11-08T19:24:52.804922+0000","last_undegraded":"2023-11-08T19:24:52.804922+0000","last_fullsized":"2023-11-08T19:24:52.804922+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:14:01.459274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.631027+0000","last_change":"2023-11-08T19:24:50.618043+0000","last_active":"2023-11-08T19:24:51.631027+0000","last_peered":"2023-11-08T19:24:51.631027+0000","last_clean":"2023-11-08T19:24:51.631027+0000","last_became_active":"2023-11-08T19:24:50.617727+0000","last_became_peered":"2023-11-08T19:24:50.617727+0000","last_unstale":"2023-11-08T19:24:51.631027+0000","last_undegraded":"2023-11-08T19:24:51.631027+0000","last_fullsized":"2023-11-08T19:24:51.631027+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:13:41.469149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.309784+0000","last_change":"2023-11-08T19:24:49.608481+0000","last_active":"2023-11-08T19:24:52.309784+0000","last_peered":"2023-11-08T19:24:52.309784+0000","last_clean":"2023-11-08T19:24:52.309784+0000","last_became_active":"2023-11-08T19:24:49.607778+0000","last_became_peered":"2023-11-08T19:24:49.607778+0000","last_unstale":"2023-11-08T19:24:52.309784+0000","last_undegraded":"2023-11-08T19:24:52.309784+0000","last_fullsized":"2023-11-08T19:24:52.309784+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:01:09.884262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.626900+0000","last_change":"2023-11-08T19:24:50.624852+0000","last_active":"2023-11-08T19:24:51.626900+0000","last_peered":"2023-11-08T19:24:51.626900+0000","last_clean":"2023-11-08T19:24:51.626900+0000","last_became_active":"2023-11-08T19:24:50.624551+0000","last_became_peered":"2023-11-08T19:24:50.624551+0000","last_unstale":"2023-11-08T19:24:51.626900+0000","last_undegraded":"2023-11-08T19:24:51.626900+0000","last_fullsized":"2023-11-08T19:24:51.626900+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:12:54.641952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.626938+0000","last_change":"2023-11-08T19:24:49.610418+0000","last_active":"2023-11-08T19:24:51.626938+0000","last_peered":"2023-11-08T19:24:51.626938+0000","last_clean":"2023-11-08T19:24:51.626938+0000","last_became_active":"2023-11-08T19:24:49.610041+0000","last_became_peered":"2023-11-08T19:24:49.610041+0000","last_unstale":"2023-11-08T19:24:51.626938+0000","last_undegraded":"2023-11-08T19:24:51.626938+0000","last_fullsized":"2023-11-08T19:24:51.626938+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:00:12.067877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.309779+0000","last_change":"2023-11-08T19:24:50.627052+0000","last_active":"2023-11-08T19:24:52.309779+0000","last_peered":"2023-11-08T19:24:52.309779+0000","last_clean":"2023-11-08T19:24:52.309779+0000","last_became_active":"2023-11-08T19:24:50.626751+0000","last_became_peered":"2023-11-08T19:24:50.626751+0000","last_unstale":"2023-11-08T19:24:52.309779+0000","last_undegraded":"2023-11-08T19:24:52.309779+0000","last_fullsized":"2023-11-08T19:24:52.309779+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:46:19.375897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-08T19:24:50.639795+0000","last_change":"2023-11-08T19:24:49.605677+0000","last_active":"2023-11-08T19:24:50.639795+0000","last_peered":"2023-11-08T19:24:50.639795+0000","last_clean":"2023-11-08T19:24:50.639795+0000","last_became_active":"2023-11-08T19:24:49.605399+0000","last_became_peered":"2023-11-08T19:24:49.605399+0000","last_unstale":"2023-11-08T19:24:50.639795+0000","last_undegraded":"2023-11-08T19:24:50.639795+0000","last_fullsized":"2023-11-08T19:24:50.639795+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:46:18.372121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.352125+0000","last_change":"2023-11-08T19:24:50.641183+0000","last_active":"2023-11-08T19:24:52.352125+0000","last_peered":"2023-11-08T19:24:52.352125+0000","last_clean":"2023-11-08T19:24:52.352125+0000","last_became_active":"2023-11-08T19:24:50.640910+0000","last_became_peered":"2023-11-08T19:24:50.640910+0000","last_unstale":"2023-11-08T19:24:52.352125+0000","last_undegraded":"2023-11-08T19:24:52.352125+0000","last_fullsized":"2023-11-08T19:24:52.352125+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:49:19.100370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-08T19:24:50.642030+0000","last_change":"2023-11-08T19:24:49.614714+0000","last_active":"2023-11-08T19:24:50.642030+0000","last_peered":"2023-11-08T19:24:50.642030+0000","last_clean":"2023-11-08T19:24:50.642030+0000","last_became_active":"2023-11-08T19:24:49.614390+0000","last_became_peered":"2023-11-08T19:24:49.614390+0000","last_unstale":"2023-11-08T19:24:50.642030+0000","last_undegraded":"2023-11-08T19:24:50.642030+0000","last_fullsized":"2023-11-08T19:24:50.642030+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:46:12.493411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.352284+0000","last_change":"2023-11-08T19:24:50.640245+0000","last_active":"2023-11-08T19:24:52.352284+0000","last_peered":"2023-11-08T19:24:52.352284+0000","last_clean":"2023-11-08T19:24:52.352284+0000","last_became_active":"2023-11-08T19:24:50.639668+0000","last_became_peered":"2023-11-08T19:24:50.639668+0000","last_unstale":"2023-11-08T19:24:52.352284+0000","last_undegraded":"2023-11-08T19:24:52.352284+0000","last_fullsized":"2023-11-08T19:24:52.352284+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:42:27.150066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.626599+0000","last_change":"2023-11-08T19:24:49.612665+0000","last_active":"2023-11-08T19:24:51.626599+0000","last_peered":"2023-11-08T19:24:51.626599+0000","last_clean":"2023-11-08T19:24:51.626599+0000","last_became_active":"2023-11-08T19:24:49.612362+0000","last_became_peered":"2023-11-08T19:24:49.612362+0000","last_unstale":"2023-11-08T19:24:51.626599+0000","last_undegraded":"2023-11-08T19:24:51.626599+0000","last_fullsized":"2023-11-08T19:24:51.626599+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:46:12.493411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.805419+0000","last_change":"2023-11-08T19:24:50.627537+0000","last_active":"2023-11-08T19:24:52.805419+0000","last_peered":"2023-11-08T19:24:52.805419+0000","last_clean":"2023-11-08T19:24:52.805419+0000","last_became_active":"2023-11-08T19:24:50.626489+0000","last_became_peered":"2023-11-08T19:24:50.626489+0000","last_unstale":"2023-11-08T19:24:52.805419+0000","last_undegraded":"2023-11-08T19:24:52.805419+0000","last_fullsized":"2023-11-08T19:24:52.805419+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:01:10.888039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-08T19:24:50.637474+0000","last_change":"2023-11-08T19:24:49.606613+0000","last_active":"2023-11-08T19:24:50.637474+0000","last_peered":"2023-11-08T19:24:50.637474+0000","last_clean":"2023-11-08T19:24:50.637474+0000","last_became_active":"2023-11-08T19:24:49.606324+0000","last_became_peered":"2023-11-08T19:24:49.606324+0000","last_unstale":"2023-11-08T19:24:50.637474+0000","last_undegraded":"2023-11-08T19:24:50.637474+0000","last_fullsized":"2023-11-08T19:24:50.637474+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:07:00.575561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.309935+0000","last_change":"2023-11-08T19:24:50.627987+0000","last_active":"2023-11-08T19:24:52.309935+0000","last_peered":"2023-11-08T19:24:52.309935+0000","last_clean":"2023-11-08T19:24:52.309935+0000","last_became_active":"2023-11-08T19:24:50.627402+0000","last_became_peered":"2023-11-08T19:24:50.627402+0000","last_unstale":"2023-11-08T19:24:52.309935+0000","last_undegraded":"2023-11-08T19:24:52.309935+0000","last_fullsized":"2023-11-08T19:24:52.309935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:36:53.967796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.626523+0000","last_change":"2023-11-08T19:24:49.606482+0000","last_active":"2023-11-08T19:24:51.626523+0000","last_peered":"2023-11-08T19:24:51.626523+0000","last_clean":"2023-11-08T19:24:51.626523+0000","last_became_active":"2023-11-08T19:24:49.606211+0000","last_became_peered":"2023-11-08T19:24:49.606211+0000","last_unstale":"2023-11-08T19:24:51.626523+0000","last_undegraded":"2023-11-08T19:24:51.626523+0000","last_fullsized":"2023-11-08T19:24:51.626523+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:42:59.845581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.626483+0000","last_change":"2023-11-08T19:24:50.625917+0000","last_active":"2023-11-08T19:24:51.626483+0000","last_peered":"2023-11-08T19:24:51.626483+0000","last_clean":"2023-11-08T19:24:51.626483+0000","last_became_active":"2023-11-08T19:24:50.625398+0000","last_became_peered":"2023-11-08T19:24:50.625398+0000","last_unstale":"2023-11-08T19:24:51.626483+0000","last_undegraded":"2023-11-08T19:24:51.626483+0000","last_fullsized":"2023-11-08T19:24:51.626483+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:40:11.978983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.351905+0000","last_change":"2023-11-08T19:24:49.616558+0000","last_active":"2023-11-08T19:24:52.351905+0000","last_peered":"2023-11-08T19:24:52.351905+0000","last_clean":"2023-11-08T19:24:52.351905+0000","last_became_active":"2023-11-08T19:24:49.616277+0000","last_became_peered":"2023-11-08T19:24:49.616277+0000","last_unstale":"2023-11-08T19:24:52.351905+0000","last_undegraded":"2023-11-08T19:24:52.351905+0000","last_fullsized":"2023-11-08T19:24:52.351905+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:01:09.884262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.805048+0000","last_change":"2023-11-08T19:24:50.623658+0000","last_active":"2023-11-08T19:24:52.805048+0000","last_peered":"2023-11-08T19:24:52.805048+0000","last_clean":"2023-11-08T19:24:52.805048+0000","last_became_active":"2023-11-08T19:24:50.623399+0000","last_became_peered":"2023-11-08T19:24:50.623399+0000","last_unstale":"2023-11-08T19:24:52.805048+0000","last_undegraded":"2023-11-08T19:24:52.805048+0000","last_fullsized":"2023-11-08T19:24:52.805048+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:40:11.978983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-08T19:24:50.610454+0000","last_change":"2023-11-08T19:24:49.608767+0000","last_active":"2023-11-08T19:24:50.610454+0000","last_peered":"2023-11-08T19:24:50.610454+0000","last_clean":"2023-11-08T19:24:50.610454+0000","last_became_active":"2023-11-08T19:24:49.608450+0000","last_became_peered":"2023-11-08T19:24:49.608450+0000","last_unstale":"2023-11-08T19:24:50.610454+0000","last_undegraded":"2023-11-08T19:24:50.610454+0000","last_fullsized":"2023-11-08T19:24:50.610454+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:42:59.845581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.352305+0000","last_change":"2023-11-08T19:24:50.640397+0000","last_active":"2023-11-08T19:24:52.352305+0000","last_peered":"2023-11-08T19:24:52.352305+0000","last_clean":"2023-11-08T19:24:52.352305+0000","last_became_active":"2023-11-08T19:24:50.639730+0000","last_became_peered":"2023-11-08T19:24:50.639730+0000","last_unstale":"2023-11-08T19:24:52.352305+0000","last_undegraded":"2023-11-08T19:24:52.352305+0000","last_fullsized":"2023-11-08T19:24:52.352305+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:08:22.499163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.352212+0000","last_change":"2023-11-08T19:24:49.611202+0000","last_active":"2023-11-08T19:24:52.352212+0000","last_peered":"2023-11-08T19:24:52.352212+0000","last_clean":"2023-11-08T19:24:52.352212+0000","last_became_active":"2023-11-08T19:24:49.610894+0000","last_became_peered":"2023-11-08T19:24:49.610894+0000","last_unstale":"2023-11-08T19:24:52.352212+0000","last_undegraded":"2023-11-08T19:24:52.352212+0000","last_fullsized":"2023-11-08T19:24:52.352212+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:48:25.546248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.309935+0000","last_change":"2023-11-08T19:24:50.638901+0000","last_active":"2023-11-08T19:24:52.309935+0000","last_peered":"2023-11-08T19:24:52.309935+0000","last_clean":"2023-11-08T19:24:52.309935+0000","last_became_active":"2023-11-08T19:24:50.638587+0000","last_became_peered":"2023-11-08T19:24:50.638587+0000","last_unstale":"2023-11-08T19:24:52.309935+0000","last_undegraded":"2023-11-08T19:24:52.309935+0000","last_fullsized":"2023-11-08T19:24:52.309935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:13:58.948371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.805313+0000","last_change":"2023-11-08T19:24:49.612170+0000","last_active":"2023-11-08T19:24:52.805313+0000","last_peered":"2023-11-08T19:24:52.805313+0000","last_clean":"2023-11-08T19:24:52.805313+0000","last_became_active":"2023-11-08T19:24:49.611895+0000","last_became_peered":"2023-11-08T19:24:49.611895+0000","last_unstale":"2023-11-08T19:24:52.805313+0000","last_undegraded":"2023-11-08T19:24:52.805313+0000","last_fullsized":"2023-11-08T19:24:52.805313+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:38:24.246638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.805721+0000","last_change":"2023-11-08T19:24:50.627060+0000","last_active":"2023-11-08T19:24:52.805721+0000","last_peered":"2023-11-08T19:24:52.805721+0000","last_clean":"2023-11-08T19:24:52.805721+0000","last_became_active":"2023-11-08T19:24:50.626468+0000","last_became_peered":"2023-11-08T19:24:50.626468+0000","last_unstale":"2023-11-08T19:24:52.805721+0000","last_undegraded":"2023-11-08T19:24:52.805721+0000","last_fullsized":"2023-11-08T19:24:52.805721+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:00:13.071654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":18,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-08T19:24:50.645041+0000","last_change":"2023-11-08T19:24:49.610676+0000","last_active":"2023-11-08T19:24:50.645041+0000","last_peered":"2023-11-08T19:24:50.645041+0000","last_clean":"2023-11-08T19:24:50.645041+0000","last_became_active":"2023-11-08T19:24:49.610344+0000","last_became_peered":"2023-11-08T19:24:49.610344+0000","last_unstale":"2023-11-08T19:24:50.645041+0000","last_undegraded":"2023-11-08T19:24:50.645041+0000","last_fullsized":"2023-11-08T19:24:50.645041+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:46:12.493411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.631028+0000","last_change":"2023-11-08T19:24:50.626565+0000","last_active":"2023-11-08T19:24:51.631028+0000","last_peered":"2023-11-08T19:24:51.631028+0000","last_clean":"2023-11-08T19:24:51.631028+0000","last_became_active":"2023-11-08T19:24:50.626280+0000","last_became_peered":"2023-11-08T19:24:50.626280+0000","last_unstale":"2023-11-08T19:24:51.631028+0000","last_undegraded":"2023-11-08T19:24:51.631028+0000","last_fullsized":"2023-11-08T19:24:51.631028+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:08:22.499163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.309884+0000","last_change":"2023-11-08T19:24:49.610471+0000","last_active":"2023-11-08T19:24:52.309884+0000","last_peered":"2023-11-08T19:24:52.309884+0000","last_clean":"2023-11-08T19:24:52.309884+0000","last_became_active":"2023-11-08T19:24:49.610182+0000","last_became_peered":"2023-11-08T19:24:49.610182+0000","last_unstale":"2023-11-08T19:24:52.309884+0000","last_undegraded":"2023-11-08T19:24:52.309884+0000","last_fullsized":"2023-11-08T19:24:52.309884+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:14:00.455497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.309602+0000","last_change":"2023-11-08T19:24:50.619498+0000","last_active":"2023-11-08T19:24:52.309602+0000","last_peered":"2023-11-08T19:24:52.309602+0000","last_clean":"2023-11-08T19:24:52.309602+0000","last_became_active":"2023-11-08T19:24:50.619231+0000","last_became_peered":"2023-11-08T19:24:50.619231+0000","last_unstale":"2023-11-08T19:24:52.309602+0000","last_undegraded":"2023-11-08T19:24:52.309602+0000","last_fullsized":"2023-11-08T19:24:52.309602+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:38:25.250414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.352455+0000","last_change":"2023-11-08T19:24:49.612336+0000","last_active":"2023-11-08T19:24:52.352455+0000","last_peered":"2023-11-08T19:24:52.352455+0000","last_clean":"2023-11-08T19:24:52.352455+0000","last_became_active":"2023-11-08T19:24:49.611885+0000","last_became_peered":"2023-11-08T19:24:49.611885+0000","last_unstale":"2023-11-08T19:24:52.352455+0000","last_undegraded":"2023-11-08T19:24:52.352455+0000","last_fullsized":"2023-11-08T19:24:52.352455+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:46:18.372121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.631204+0000","last_change":"2023-11-08T19:24:50.629107+0000","last_active":"2023-11-08T19:24:51.631204+0000","last_peered":"2023-11-08T19:24:51.631204+0000","last_clean":"2023-11-08T19:24:51.631204+0000","last_became_active":"2023-11-08T19:24:50.628005+0000","last_became_peered":"2023-11-08T19:24:50.628005+0000","last_unstale":"2023-11-08T19:24:51.631204+0000","last_undegraded":"2023-11-08T19:24:51.631204+0000","last_fullsized":"2023-11-08T19:24:51.631204+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:34:42.075394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-08T19:24:50.609767+0000","last_change":"2023-11-08T19:24:49.605863+0000","last_active":"2023-11-08T19:24:50.609767+0000","last_peered":"2023-11-08T19:24:50.609767+0000","last_clean":"2023-11-08T19:24:50.609767+0000","last_became_active":"2023-11-08T19:24:49.604986+0000","last_became_peered":"2023-11-08T19:24:49.604986+0000","last_unstale":"2023-11-08T19:24:50.609767+0000","last_undegraded":"2023-11-08T19:24:50.609767+0000","last_fullsized":"2023-11-08T19:24:50.609767+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:15:32.030617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.629786+0000","last_change":"2023-11-08T19:24:50.618716+0000","last_active":"2023-11-08T19:24:51.629786+0000","last_peered":"2023-11-08T19:24:51.629786+0000","last_clean":"2023-11-08T19:24:51.629786+0000","last_became_active":"2023-11-08T19:24:50.618425+0000","last_became_peered":"2023-11-08T19:24:50.618425+0000","last_unstale":"2023-11-08T19:24:51.629786+0000","last_undegraded":"2023-11-08T19:24:51.629786+0000","last_fullsized":"2023-11-08T19:24:51.629786+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:15:33.285449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.804637+0000","last_change":"2023-11-08T19:24:49.608503+0000","last_active":"2023-11-08T19:24:52.804637+0000","last_peered":"2023-11-08T19:24:52.804637+0000","last_clean":"2023-11-08T19:24:52.804637+0000","last_became_active":"2023-11-08T19:24:49.608176+0000","last_became_peered":"2023-11-08T19:24:49.608176+0000","last_unstale":"2023-11-08T19:24:52.804637+0000","last_undegraded":"2023-11-08T19:24:52.804637+0000","last_fullsized":"2023-11-08T19:24:52.804637+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:17:50.005667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.630752+0000","last_change":"2023-11-08T19:24:50.629255+0000","last_active":"2023-11-08T19:24:51.630752+0000","last_peered":"2023-11-08T19:24:51.630752+0000","last_clean":"2023-11-08T19:24:51.630752+0000","last_became_active":"2023-11-08T19:24:50.628763+0000","last_became_peered":"2023-11-08T19:24:50.628763+0000","last_unstale":"2023-11-08T19:24:51.630752+0000","last_undegraded":"2023-11-08T19:24:51.630752+0000","last_fullsized":"2023-11-08T19:24:51.630752+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:42:27.150066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.805855+0000","last_change":"2023-11-08T19:24:49.610638+0000","last_active":"2023-11-08T19:24:52.805855+0000","last_peered":"2023-11-08T19:24:52.805855+0000","last_clean":"2023-11-08T19:24:52.805855+0000","last_became_active":"2023-11-08T19:24:49.610362+0000","last_became_peered":"2023-11-08T19:24:49.610362+0000","last_unstale":"2023-11-08T19:24:52.805855+0000","last_undegraded":"2023-11-08T19:24:52.805855+0000","last_fullsized":"2023-11-08T19:24:52.805855+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:15:32.030617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.631275+0000","last_change":"2023-11-08T19:24:50.627667+0000","last_active":"2023-11-08T19:24:51.631275+0000","last_peered":"2023-11-08T19:24:51.631275+0000","last_clean":"2023-11-08T19:24:51.631275+0000","last_became_active":"2023-11-08T19:24:50.627390+0000","last_became_peered":"2023-11-08T19:24:50.627390+0000","last_unstale":"2023-11-08T19:24:51.631275+0000","last_undegraded":"2023-11-08T19:24:51.631275+0000","last_fullsized":"2023-11-08T19:24:51.631275+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:01:53.410545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.805154+0000","last_change":"2023-11-08T19:24:49.613413+0000","last_active":"2023-11-08T19:24:52.805154+0000","last_peered":"2023-11-08T19:24:52.805154+0000","last_clean":"2023-11-08T19:24:52.805154+0000","last_became_active":"2023-11-08T19:24:49.613131+0000","last_became_peered":"2023-11-08T19:24:49.613131+0000","last_unstale":"2023-11-08T19:24:52.805154+0000","last_undegraded":"2023-11-08T19:24:52.805154+0000","last_fullsized":"2023-11-08T19:24:52.805154+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:46:12.493411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.352374+0000","last_change":"2023-11-08T19:24:50.640031+0000","last_active":"2023-11-08T19:24:52.352374+0000","last_peered":"2023-11-08T19:24:52.352374+0000","last_clean":"2023-11-08T19:24:52.352374+0000","last_became_active":"2023-11-08T19:24:50.639718+0000","last_became_peered":"2023-11-08T19:24:50.639718+0000","last_unstale":"2023-11-08T19:24:52.352374+0000","last_undegraded":"2023-11-08T19:24:52.352374+0000","last_fullsized":"2023-11-08T19:24:52.352374+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:08:17.096077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.352346+0000","last_change":"2023-11-08T19:24:49.612137+0000","last_active":"2023-11-08T19:24:52.352346+0000","last_peered":"2023-11-08T19:24:52.352346+0000","last_clean":"2023-11-08T19:24:52.352346+0000","last_became_active":"2023-11-08T19:24:49.611532+0000","last_became_peered":"2023-11-08T19:24:49.611532+0000","last_unstale":"2023-11-08T19:24:52.352346+0000","last_undegraded":"2023-11-08T19:24:52.352346+0000","last_fullsized":"2023-11-08T19:24:52.352346+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:40:10.975206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.805536+0000","last_change":"2023-11-08T19:24:50.630854+0000","last_active":"2023-11-08T19:24:52.805536+0000","last_peered":"2023-11-08T19:24:52.805536+0000","last_clean":"2023-11-08T19:24:52.805536+0000","last_became_active":"2023-11-08T19:24:50.630549+0000","last_became_peered":"2023-11-08T19:24:50.630549+0000","last_unstale":"2023-11-08T19:24:52.805536+0000","last_undegraded":"2023-11-08T19:24:52.805536+0000","last_fullsized":"2023-11-08T19:24:52.805536+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:48:26.550025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.804950+0000","last_change":"2023-11-08T19:24:50.620897+0000","last_active":"2023-11-08T19:24:52.804950+0000","last_peered":"2023-11-08T19:24:52.804950+0000","last_clean":"2023-11-08T19:24:52.804950+0000","last_became_active":"2023-11-08T19:24:50.620606+0000","last_became_peered":"2023-11-08T19:24:50.620606+0000","last_unstale":"2023-11-08T19:24:52.804950+0000","last_undegraded":"2023-11-08T19:24:52.804950+0000","last_fullsized":"2023-11-08T19:24:52.804950+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:30:36.041820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.804625+0000","last_change":"2023-11-08T19:24:49.612840+0000","last_active":"2023-11-08T19:24:52.804625+0000","last_peered":"2023-11-08T19:24:52.804625+0000","last_clean":"2023-11-08T19:24:52.804625+0000","last_became_active":"2023-11-08T19:24:49.612652+0000","last_became_peered":"2023-11-08T19:24:49.612652+0000","last_unstale":"2023-11-08T19:24:52.804625+0000","last_undegraded":"2023-11-08T19:24:52.804625+0000","last_fullsized":"2023-11-08T19:24:52.804625+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:48:25.546248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.626593+0000","last_change":"2023-11-08T19:24:50.642515+0000","last_active":"2023-11-08T19:24:51.626593+0000","last_peered":"2023-11-08T19:24:51.626593+0000","last_clean":"2023-11-08T19:24:51.626593+0000","last_became_active":"2023-11-08T19:24:50.642204+0000","last_became_peered":"2023-11-08T19:24:50.642204+0000","last_unstale":"2023-11-08T19:24:51.626593+0000","last_undegraded":"2023-11-08T19:24:51.626593+0000","last_fullsized":"2023-11-08T19:24:51.626593+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:42:42.512786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.351949+0000","last_change":"2023-11-08T19:24:49.605594+0000","last_active":"2023-11-08T19:24:52.351949+0000","last_peered":"2023-11-08T19:24:52.351949+0000","last_clean":"2023-11-08T19:24:52.351949+0000","last_became_active":"2023-11-08T19:24:49.605208+0000","last_became_peered":"2023-11-08T19:24:49.605208+0000","last_unstale":"2023-11-08T19:24:52.351949+0000","last_undegraded":"2023-11-08T19:24:52.351949+0000","last_fullsized":"2023-11-08T19:24:52.351949+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:17:50.005667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":165,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-08T19:24:50.613237+0000","last_change":"2023-11-08T19:24:45.888122+0000","last_active":"2023-11-08T19:24:50.613237+0000","last_peered":"2023-11-08T19:24:50.613237+0000","last_clean":"2023-11-08T19:24:50.613237+0000","last_became_active":"2023-11-08T19:24:45.887048+0000","last_became_peered":"2023-11-08T19:24:45.887048+0000","last_unstale":"2023-11-08T19:24:50.613237+0000","last_undegraded":"2023-11-08T19:24:50.613237+0000","last_fullsized":"2023-11-08T19:24:50.613237+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:44.575133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:44.575133+0000","last_clean_scrub_stamp":"2023-11-08T19:24:44.575133+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:19:20.202062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.309622+0000","last_change":"2023-11-08T19:24:50.628353+0000","last_active":"2023-11-08T19:24:52.309622+0000","last_peered":"2023-11-08T19:24:52.309622+0000","last_clean":"2023-11-08T19:24:52.309622+0000","last_became_active":"2023-11-08T19:24:50.628008+0000","last_became_peered":"2023-11-08T19:24:50.628008+0000","last_unstale":"2023-11-08T19:24:52.309622+0000","last_undegraded":"2023-11-08T19:24:52.309622+0000","last_fullsized":"2023-11-08T19:24:52.309622+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:46:13.497187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-08T19:24:50.612010+0000","last_change":"2023-11-08T19:24:49.608659+0000","last_active":"2023-11-08T19:24:50.612010+0000","last_peered":"2023-11-08T19:24:50.612010+0000","last_clean":"2023-11-08T19:24:50.612010+0000","last_became_active":"2023-11-08T19:24:49.608243+0000","last_became_peered":"2023-11-08T19:24:49.608243+0000","last_unstale":"2023-11-08T19:24:50.612010+0000","last_undegraded":"2023-11-08T19:24:50.612010+0000","last_fullsized":"2023-11-08T19:24:50.612010+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:42:41.509009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-08T19:24:50.611395+0000","last_change":"2023-11-08T19:24:49.611793+0000","last_active":"2023-11-08T19:24:50.611395+0000","last_peered":"2023-11-08T19:24:50.611395+0000","last_clean":"2023-11-08T19:24:50.611395+0000","last_became_active":"2023-11-08T19:24:49.611489+0000","last_became_peered":"2023-11-08T19:24:49.611489+0000","last_unstale":"2023-11-08T19:24:50.611395+0000","last_undegraded":"2023-11-08T19:24:50.611395+0000","last_fullsized":"2023-11-08T19:24:50.611395+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:15:32.281672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.805231+0000","last_change":"2023-11-08T19:24:50.624279+0000","last_active":"2023-11-08T19:24:52.805231+0000","last_peered":"2023-11-08T19:24:52.805231+0000","last_clean":"2023-11-08T19:24:52.805231+0000","last_became_active":"2023-11-08T19:24:50.623920+0000","last_became_peered":"2023-11-08T19:24:50.623920+0000","last_unstale":"2023-11-08T19:24:52.805231+0000","last_undegraded":"2023-11-08T19:24:52.805231+0000","last_fullsized":"2023-11-08T19:24:52.805231+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:42:42.512786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:53.590119+0000","last_change":"2023-11-08T19:24:49.606910+0000","last_active":"2023-11-08T19:24:53.590119+0000","last_peered":"2023-11-08T19:24:53.590119+0000","last_clean":"2023-11-08T19:24:53.590119+0000","last_became_active":"2023-11-08T19:24:49.606627+0000","last_became_peered":"2023-11-08T19:24:49.606627+0000","last_unstale":"2023-11-08T19:24:53.590119+0000","last_undegraded":"2023-11-08T19:24:53.590119+0000","last_fullsized":"2023-11-08T19:24:53.590119+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:56:00.148860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.627087+0000","last_change":"2023-11-08T19:24:50.622398+0000","last_active":"2023-11-08T19:24:51.627087+0000","last_peered":"2023-11-08T19:24:51.627087+0000","last_clean":"2023-11-08T19:24:51.627087+0000","last_became_active":"2023-11-08T19:24:50.622092+0000","last_became_peered":"2023-11-08T19:24:50.622092+0000","last_unstale":"2023-11-08T19:24:51.627087+0000","last_undegraded":"2023-11-08T19:24:51.627087+0000","last_fullsized":"2023-11-08T19:24:51.627087+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:42:27.150066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-08T19:24:50.638180+0000","last_change":"2023-11-08T19:24:49.604285+0000","last_active":"2023-11-08T19:24:50.638180+0000","last_peered":"2023-11-08T19:24:50.638180+0000","last_clean":"2023-11-08T19:24:50.638180+0000","last_became_active":"2023-11-08T19:24:49.603965+0000","last_became_peered":"2023-11-08T19:24:49.603965+0000","last_unstale":"2023-11-08T19:24:50.638180+0000","last_undegraded":"2023-11-08T19:24:50.638180+0000","last_fullsized":"2023-11-08T19:24:50.638180+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:01:09.884262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.309770+0000","last_change":"2023-11-08T19:24:50.624848+0000","last_active":"2023-11-08T19:24:52.309770+0000","last_peered":"2023-11-08T19:24:52.309770+0000","last_clean":"2023-11-08T19:24:52.309770+0000","last_became_active":"2023-11-08T19:24:50.624518+0000","last_became_peered":"2023-11-08T19:24:50.624518+0000","last_unstale":"2023-11-08T19:24:52.309770+0000","last_undegraded":"2023-11-08T19:24:52.309770+0000","last_fullsized":"2023-11-08T19:24:52.309770+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:15:33.285449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-08T19:24:50.611239+0000","last_change":"2023-11-08T19:24:49.611779+0000","last_active":"2023-11-08T19:24:50.611239+0000","last_peered":"2023-11-08T19:24:50.611239+0000","last_clean":"2023-11-08T19:24:50.611239+0000","last_became_active":"2023-11-08T19:24:49.611422+0000","last_became_peered":"2023-11-08T19:24:49.611422+0000","last_unstale":"2023-11-08T19:24:50.611239+0000","last_undegraded":"2023-11-08T19:24:50.611239+0000","last_fullsized":"2023-11-08T19:24:50.611239+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:30:31.599200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.805558+0000","last_change":"2023-11-08T19:24:50.627488+0000","last_active":"2023-11-08T19:24:52.805558+0000","last_peered":"2023-11-08T19:24:52.805558+0000","last_clean":"2023-11-08T19:24:52.805558+0000","last_became_active":"2023-11-08T19:24:50.626571+0000","last_became_peered":"2023-11-08T19:24:50.626571+0000","last_unstale":"2023-11-08T19:24:52.805558+0000","last_undegraded":"2023-11-08T19:24:52.805558+0000","last_fullsized":"2023-11-08T19:24:52.805558+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:43:00.849357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.804911+0000","last_change":"2023-11-08T19:24:49.606579+0000","last_active":"2023-11-08T19:24:52.804911+0000","last_peered":"2023-11-08T19:24:52.804911+0000","last_clean":"2023-11-08T19:24:52.804911+0000","last_became_active":"2023-11-08T19:24:49.606283+0000","last_became_peered":"2023-11-08T19:24:49.606283+0000","last_unstale":"2023-11-08T19:24:52.804911+0000","last_undegraded":"2023-11-08T19:24:52.804911+0000","last_fullsized":"2023-11-08T19:24:52.804911+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:14:00.455497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.626421+0000","last_change":"2023-11-08T19:24:50.625692+0000","last_active":"2023-11-08T19:24:51.626421+0000","last_peered":"2023-11-08T19:24:51.626421+0000","last_clean":"2023-11-08T19:24:51.626421+0000","last_became_active":"2023-11-08T19:24:50.625014+0000","last_became_peered":"2023-11-08T19:24:50.625014+0000","last_unstale":"2023-11-08T19:24:51.626421+0000","last_undegraded":"2023-11-08T19:24:51.626421+0000","last_fullsized":"2023-11-08T19:24:51.626421+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:15:33.285449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.805081+0000","last_change":"2023-11-08T19:24:49.611023+0000","last_active":"2023-11-08T19:24:52.805081+0000","last_peered":"2023-11-08T19:24:52.805081+0000","last_clean":"2023-11-08T19:24:52.805081+0000","last_became_active":"2023-11-08T19:24:49.610735+0000","last_became_peered":"2023-11-08T19:24:49.610735+0000","last_unstale":"2023-11-08T19:24:52.805081+0000","last_undegraded":"2023-11-08T19:24:52.805081+0000","last_fullsized":"2023-11-08T19:24:52.805081+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:30:31.599200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.626730+0000","last_change":"2023-11-08T19:24:50.631491+0000","last_active":"2023-11-08T19:24:51.626730+0000","last_peered":"2023-11-08T19:24:51.626730+0000","last_clean":"2023-11-08T19:24:51.626730+0000","last_became_active":"2023-11-08T19:24:50.630872+0000","last_became_peered":"2023-11-08T19:24:50.630872+0000","last_unstale":"2023-11-08T19:24:51.626730+0000","last_undegraded":"2023-11-08T19:24:51.626730+0000","last_fullsized":"2023-11-08T19:24:51.626730+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:36:53.967796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-08T19:24:50.640533+0000","last_change":"2023-11-08T19:24:49.613965+0000","last_active":"2023-11-08T19:24:50.640533+0000","last_peered":"2023-11-08T19:24:50.640533+0000","last_clean":"2023-11-08T19:24:50.640533+0000","last_became_active":"2023-11-08T19:24:49.613629+0000","last_became_peered":"2023-11-08T19:24:49.613629+0000","last_unstale":"2023-11-08T19:24:50.640533+0000","last_undegraded":"2023-11-08T19:24:50.640533+0000","last_fullsized":"2023-11-08T19:24:50.640533+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:56:00.148860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.629891+0000","last_change":"2023-11-08T19:24:50.627696+0000","last_active":"2023-11-08T19:24:51.629891+0000","last_peered":"2023-11-08T19:24:51.629891+0000","last_clean":"2023-11-08T19:24:51.629891+0000","last_became_active":"2023-11-08T19:24:50.627346+0000","last_became_peered":"2023-11-08T19:24:50.627346+0000","last_unstale":"2023-11-08T19:24:51.629891+0000","last_undegraded":"2023-11-08T19:24:51.629891+0000","last_fullsized":"2023-11-08T19:24:51.629891+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:30:36.041820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.805091+0000","last_change":"2023-11-08T19:24:49.613365+0000","last_active":"2023-11-08T19:24:52.805091+0000","last_peered":"2023-11-08T19:24:52.805091+0000","last_clean":"2023-11-08T19:24:52.805091+0000","last_became_active":"2023-11-08T19:24:49.613052+0000","last_became_peered":"2023-11-08T19:24:49.613052+0000","last_unstale":"2023-11-08T19:24:52.805091+0000","last_undegraded":"2023-11-08T19:24:52.805091+0000","last_fullsized":"2023-11-08T19:24:52.805091+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:41:24.551269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.631232+0000","last_change":"2023-11-08T19:24:50.642477+0000","last_active":"2023-11-08T19:24:51.631232+0000","last_peered":"2023-11-08T19:24:51.631232+0000","last_clean":"2023-11-08T19:24:51.631232+0000","last_became_active":"2023-11-08T19:24:50.642186+0000","last_became_peered":"2023-11-08T19:24:50.642186+0000","last_unstale":"2023-11-08T19:24:51.631232+0000","last_undegraded":"2023-11-08T19:24:51.631232+0000","last_fullsized":"2023-11-08T19:24:51.631232+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:08:17.096077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.805219+0000","last_change":"2023-11-08T19:24:49.606271+0000","last_active":"2023-11-08T19:24:52.805219+0000","last_peered":"2023-11-08T19:24:52.805219+0000","last_clean":"2023-11-08T19:24:52.805219+0000","last_became_active":"2023-11-08T19:24:49.605802+0000","last_became_peered":"2023-11-08T19:24:49.605802+0000","last_unstale":"2023-11-08T19:24:52.805219+0000","last_undegraded":"2023-11-08T19:24:52.805219+0000","last_fullsized":"2023-11-08T19:24:52.805219+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:07:00.575561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.352338+0000","last_change":"2023-11-08T19:24:50.640594+0000","last_active":"2023-11-08T19:24:52.352338+0000","last_peered":"2023-11-08T19:24:52.352338+0000","last_clean":"2023-11-08T19:24:52.352338+0000","last_became_active":"2023-11-08T19:24:50.640295+0000","last_became_peered":"2023-11-08T19:24:50.640295+0000","last_unstale":"2023-11-08T19:24:52.352338+0000","last_undegraded":"2023-11-08T19:24:52.352338+0000","last_fullsized":"2023-11-08T19:24:52.352338+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:34:42.075394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.805026+0000","last_change":"2023-11-08T19:24:49.610782+0000","last_active":"2023-11-08T19:24:52.805026+0000","last_peered":"2023-11-08T19:24:52.805026+0000","last_clean":"2023-11-08T19:24:52.805026+0000","last_became_active":"2023-11-08T19:24:49.610493+0000","last_became_peered":"2023-11-08T19:24:49.610493+0000","last_unstale":"2023-11-08T19:24:52.805026+0000","last_undegraded":"2023-11-08T19:24:52.805026+0000","last_fullsized":"2023-11-08T19:24:52.805026+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:42:59.845581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.631518+0000","last_change":"2023-11-08T19:24:50.630352+0000","last_active":"2023-11-08T19:24:51.631518+0000","last_peered":"2023-11-08T19:24:51.631518+0000","last_clean":"2023-11-08T19:24:51.631518+0000","last_became_active":"2023-11-08T19:24:50.628170+0000","last_became_peered":"2023-11-08T19:24:50.628170+0000","last_unstale":"2023-11-08T19:24:51.631518+0000","last_undegraded":"2023-11-08T19:24:51.631518+0000","last_fullsized":"2023-11-08T19:24:51.631518+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:18:29.376849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.629782+0000","last_change":"2023-11-08T19:24:50.616189+0000","last_active":"2023-11-08T19:24:51.629782+0000","last_peered":"2023-11-08T19:24:51.629782+0000","last_clean":"2023-11-08T19:24:51.629782+0000","last_became_active":"2023-11-08T19:24:50.615704+0000","last_became_peered":"2023-11-08T19:24:50.615704+0000","last_unstale":"2023-11-08T19:24:51.629782+0000","last_undegraded":"2023-11-08T19:24:51.629782+0000","last_fullsized":"2023-11-08T19:24:51.629782+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:30:32.602976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.804775+0000","last_change":"2023-11-08T19:24:50.625948+0000","last_active":"2023-11-08T19:24:52.804775+0000","last_peered":"2023-11-08T19:24:52.804775+0000","last_clean":"2023-11-08T19:24:52.804775+0000","last_became_active":"2023-11-08T19:24:50.625633+0000","last_became_peered":"2023-11-08T19:24:50.625633+0000","last_unstale":"2023-11-08T19:24:52.804775+0000","last_undegraded":"2023-11-08T19:24:52.804775+0000","last_fullsized":"2023-11-08T19:24:52.804775+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:00:13.071654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.352423+0000","last_change":"2023-11-08T19:24:50.640919+0000","last_active":"2023-11-08T19:24:52.352423+0000","last_peered":"2023-11-08T19:24:52.352423+0000","last_clean":"2023-11-08T19:24:52.352423+0000","last_became_active":"2023-11-08T19:24:50.640496+0000","last_became_peered":"2023-11-08T19:24:50.640496+0000","last_unstale":"2023-11-08T19:24:52.352423+0000","last_undegraded":"2023-11-08T19:24:52.352423+0000","last_fullsized":"2023-11-08T19:24:52.352423+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:59:37.592538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.309782+0000","last_change":"2023-11-08T19:24:50.625916+0000","last_active":"2023-11-08T19:24:52.309782+0000","last_peered":"2023-11-08T19:24:52.309782+0000","last_clean":"2023-11-08T19:24:52.309782+0000","last_became_active":"2023-11-08T19:24:50.625625+0000","last_became_peered":"2023-11-08T19:24:50.625625+0000","last_unstale":"2023-11-08T19:24:52.309782+0000","last_undegraded":"2023-11-08T19:24:52.309782+0000","last_fullsized":"2023-11-08T19:24:52.309782+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:40:11.978983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.626555+0000","last_change":"2023-11-08T19:24:50.628686+0000","last_active":"2023-11-08T19:24:51.626555+0000","last_peered":"2023-11-08T19:24:51.626555+0000","last_clean":"2023-11-08T19:24:51.626555+0000","last_became_active":"2023-11-08T19:24:50.628289+0000","last_became_peered":"2023-11-08T19:24:50.628289+0000","last_unstale":"2023-11-08T19:24:51.626555+0000","last_undegraded":"2023-11-08T19:24:51.626555+0000","last_fullsized":"2023-11-08T19:24:51.626555+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:46:19.375897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.805231+0000","last_change":"2023-11-08T19:24:50.624352+0000","last_active":"2023-11-08T19:24:52.805231+0000","last_peered":"2023-11-08T19:24:52.805231+0000","last_clean":"2023-11-08T19:24:52.805231+0000","last_became_active":"2023-11-08T19:24:50.624035+0000","last_became_peered":"2023-11-08T19:24:50.624035+0000","last_unstale":"2023-11-08T19:24:52.805231+0000","last_undegraded":"2023-11-08T19:24:52.805231+0000","last_fullsized":"2023-11-08T19:24:52.805231+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:46:19.375897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.630792+0000","last_change":"2023-11-08T19:24:50.616410+0000","last_active":"2023-11-08T19:24:51.630792+0000","last_peered":"2023-11-08T19:24:51.630792+0000","last_clean":"2023-11-08T19:24:51.630792+0000","last_became_active":"2023-11-08T19:24:50.616118+0000","last_became_peered":"2023-11-08T19:24:50.616118+0000","last_unstale":"2023-11-08T19:24:51.630792+0000","last_undegraded":"2023-11-08T19:24:51.630792+0000","last_fullsized":"2023-11-08T19:24:51.630792+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:42:42.512786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.626991+0000","last_change":"2023-11-08T19:24:50.628009+0000","last_active":"2023-11-08T19:24:51.626991+0000","last_peered":"2023-11-08T19:24:51.626991+0000","last_clean":"2023-11-08T19:24:51.626991+0000","last_became_active":"2023-11-08T19:24:50.627668+0000","last_became_peered":"2023-11-08T19:24:50.627668+0000","last_unstale":"2023-11-08T19:24:51.626991+0000","last_undegraded":"2023-11-08T19:24:51.626991+0000","last_fullsized":"2023-11-08T19:24:51.626991+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:49:19.100370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.352100+0000","last_change":"2023-11-08T19:24:50.641255+0000","last_active":"2023-11-08T19:24:52.352100+0000","last_peered":"2023-11-08T19:24:52.352100+0000","last_clean":"2023-11-08T19:24:52.352100+0000","last_became_active":"2023-11-08T19:24:50.640164+0000","last_became_peered":"2023-11-08T19:24:50.640164+0000","last_unstale":"2023-11-08T19:24:52.352100+0000","last_undegraded":"2023-11-08T19:24:52.352100+0000","last_fullsized":"2023-11-08T19:24:52.352100+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:12:54.641952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.626435+0000","last_change":"2023-11-08T19:24:50.629006+0000","last_active":"2023-11-08T19:24:51.626435+0000","last_peered":"2023-11-08T19:24:51.626435+0000","last_clean":"2023-11-08T19:24:51.626435+0000","last_became_active":"2023-11-08T19:24:50.628685+0000","last_became_peered":"2023-11-08T19:24:50.628685+0000","last_unstale":"2023-11-08T19:24:51.626435+0000","last_undegraded":"2023-11-08T19:24:51.626435+0000","last_fullsized":"2023-11-08T19:24:51.626435+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:30:36.041820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.352270+0000","last_change":"2023-11-08T19:24:50.640737+0000","last_active":"2023-11-08T19:24:52.352270+0000","last_peered":"2023-11-08T19:24:52.352270+0000","last_clean":"2023-11-08T19:24:52.352270+0000","last_became_active":"2023-11-08T19:24:50.640255+0000","last_became_peered":"2023-11-08T19:24:50.640255+0000","last_unstale":"2023-11-08T19:24:52.352270+0000","last_undegraded":"2023-11-08T19:24:52.352270+0000","last_fullsized":"2023-11-08T19:24:52.352270+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:13:41.469149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.309753+0000","last_change":"2023-11-08T19:24:50.620645+0000","last_active":"2023-11-08T19:24:52.309753+0000","last_peered":"2023-11-08T19:24:52.309753+0000","last_clean":"2023-11-08T19:24:52.309753+0000","last_became_active":"2023-11-08T19:24:50.620349+0000","last_became_peered":"2023-11-08T19:24:50.620349+0000","last_unstale":"2023-11-08T19:24:52.309753+0000","last_undegraded":"2023-11-08T19:24:52.309753+0000","last_fullsized":"2023-11-08T19:24:52.309753+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:30:32.602976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.630586+0000","last_change":"2023-11-08T19:24:50.629751+0000","last_active":"2023-11-08T19:24:51.630586+0000","last_peered":"2023-11-08T19:24:51.630586+0000","last_clean":"2023-11-08T19:24:51.630586+0000","last_became_active":"2023-11-08T19:24:50.629453+0000","last_became_peered":"2023-11-08T19:24:50.629453+0000","last_unstale":"2023-11-08T19:24:51.630586+0000","last_undegraded":"2023-11-08T19:24:51.630586+0000","last_fullsized":"2023-11-08T19:24:51.630586+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:46:19.375897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":802816,"data_stored":661860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67000000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54300000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68500000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70599999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60699999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.498}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45100000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55000000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41099999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46600000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34699999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":71070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29999999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32300000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38100000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52700000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55700000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":811008,"data_stored":661916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39600000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.432}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79300000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77900000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67200000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83699999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":204800,"data_stored":70910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76200000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80600000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83299999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574856,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38700000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40699999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34599999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67800000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56200000000000006}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53700000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":71036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61799999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.504}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68899999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67400000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39700000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48899999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":208896,"data_stored":71000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61299999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59899999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42099999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39800000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36899999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-08T19:25:03.993 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-08T19:25:04.147 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.146+0000 7fa4842c0700 1 Processor -- start 2023-11-08T19:25:04.148 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.146+0000 7fa4842c0700 1 -- start start 2023-11-08T19:25:04.150 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.147+0000 7fa4842c0700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa47c06ed60 0x7fa47c06b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:04.150 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.147+0000 7fa4842c0700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fa47c0cc410 0x7fa47c0cc820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:04.150 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.147+0000 7fa4842c0700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa47c0b2a60 con 0x7fa47c159870 2023-11-08T19:25:04.151 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.147+0000 7fa4842c0700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa47c06b090 con 0x7fa47c06ed60 2023-11-08T19:25:04.151 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.147+0000 7fa4842c0700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa47c0b2d40 con 0x7fa47c0cc410 2023-11-08T19:25:04.151 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.147+0000 7fa48185b700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa47c06ed60 0x7fa47c06b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:04.151 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.147+0000 7fa48205c700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fa47c159870 0x7fa47c0b2210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:45320/0 (socket says 172.21.15.143:45320) 2023-11-08T19:25:04.152 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.147+0000 7fa48205c700 1 -- 172.21.15.143:0/1094962167 learned_addr learned my addr 172.21.15.143:0/1094962167 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:25:04.152 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.147+0000 7fa48285d700 1 --2- 172.21.15.143:0/1094962167 >> v2:172.21.15.174:3300/0 conn(0x7fa47c0cc410 0x7fa47c0cc820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:04.152 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.147+0000 7fa48105a700 1 -- 172.21.15.143:0/1094962167 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2550114876 0 0) 0x7fa47c0b2a60 con 0x7fa47c159870 2023-11-08T19:25:04.152 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.147+0000 7fa48105a700 1 -- 172.21.15.143:0/1094962167 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa4680035b0 con 0x7fa47c159870 2023-11-08T19:25:04.152 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.147+0000 7fa48185b700 1 -- 172.21.15.143:0/1094962167 >> v2:172.21.15.174:3300/0 conn(0x7fa47c0cc410 msgr2=0x7fa47c0cc820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:04.153 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.147+0000 7fa48185b700 1 --2- 172.21.15.143:0/1094962167 >> v2:172.21.15.174:3300/0 conn(0x7fa47c0cc410 0x7fa47c0cc820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:04.153 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.147+0000 7fa48185b700 1 -- 172.21.15.143:0/1094962167 >> v1:172.21.15.143:6789/0 conn(0x7fa47c159870 legacy=0x7fa47c0b2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:04.153 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.147+0000 7fa48185b700 1 -- 172.21.15.143:0/1094962167 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa47c0b3020 con 0x7fa47c06ed60 2023-11-08T19:25:04.153 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.147+0000 7fa48285d700 1 --2- 172.21.15.143:0/1094962167 >> v2:172.21.15.174:3300/0 conn(0x7fa47c0cc410 0x7fa47c0cc820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:25:04.153 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.148+0000 7fa48185b700 1 --2- 172.21.15.143:0/1094962167 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa47c06ed60 0x7fa47c06b4b0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa47c0c41c0 tx=0x7fa478009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6288302b2c059abe server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:04.153 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.148+0000 7fa48105a700 1 -- 172.21.15.143:0/1094962167 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa478011830 con 0x7fa47c06ed60 2023-11-08T19:25:04.154 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.148+0000 7fa48105a700 1 -- 172.21.15.143:0/1094962167 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa478011990 con 0x7fa47c06ed60 2023-11-08T19:25:04.154 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.148+0000 7fa48105a700 1 -- 172.21.15.143:0/1094962167 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa478011d20 con 0x7fa47c06ed60 2023-11-08T19:25:04.154 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.148+0000 7fa4842c0700 1 -- 172.21.15.143:0/1094962167 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa47c06ed60 msgr2=0x7fa47c06b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:04.154 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.148+0000 7fa4842c0700 1 --2- 172.21.15.143:0/1094962167 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa47c06ed60 0x7fa47c06b4b0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa47c0c41c0 tx=0x7fa478009b20 comp rx=0 tx=0).stop 2023-11-08T19:25:04.154 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.148+0000 7fa4842c0700 1 -- 172.21.15.143:0/1094962167 shutdown_connections 2023-11-08T19:25:04.155 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.148+0000 7fa4842c0700 1 --2- 172.21.15.143:0/1094962167 >> v2:172.21.15.174:3300/0 conn(0x7fa47c0cc410 0x7fa47c0cc820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:04.155 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.148+0000 7fa4842c0700 1 --2- 172.21.15.143:0/1094962167 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa47c06ed60 0x7fa47c06b4b0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:04.155 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.148+0000 7fa4842c0700 1 -- 172.21.15.143:0/1094962167 >> 172.21.15.143:0/1094962167 conn(0x7fa47c099ab0 msgr2=0x7fa47c0d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:04.155 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.148+0000 7fa4842c0700 1 -- 172.21.15.143:0/1094962167 shutdown_connections 2023-11-08T19:25:04.156 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.148+0000 7fa4842c0700 1 -- 172.21.15.143:0/1094962167 wait complete. 2023-11-08T19:25:04.160 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.150+0000 7fa4842c0700 1 Processor -- start 2023-11-08T19:25:04.161 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.150+0000 7fa4842c0700 1 -- start start 2023-11-08T19:25:04.161 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.150+0000 7fa4842c0700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa47c06ed60 0x7fa47c0c9d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:04.161 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.150+0000 7fa4842c0700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa47c0cc410 0x7fa47c1f49d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:04.162 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.150+0000 7fa4842c0700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa47c159870 0x7fa47c1f4f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:04.162 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.150+0000 7fa4842c0700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa47c0c6120 con 0x7fa47c159870 2023-11-08T19:25:04.162 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.150+0000 7fa4842c0700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa47c0c6400 con 0x7fa47c0cc410 2023-11-08T19:25:04.162 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.150+0000 7fa4842c0700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa47c0c66e0 con 0x7fa47c06ed60 2023-11-08T19:25:04.163 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.150+0000 7fa48285d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa47c159870 0x7fa47c1f4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:04.163 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.150+0000 7fa48285d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa47c159870 0x7fa47c1f4f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49176/0 (socket says 172.21.15.143:49176) 2023-11-08T19:25:04.163 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.150+0000 7fa48285d700 1 -- 172.21.15.143:0/2645273491 learned_addr learned my addr 172.21.15.143:0/2645273491 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:04.163 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.150+0000 7fa48185b700 1 --2- 172.21.15.143:0/2645273491 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa47c0cc410 0x7fa47c1f49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:04.163 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.150+0000 7fa48205c700 1 --2- 172.21.15.143:0/2645273491 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa47c06ed60 0x7fa47c0c9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:04.164 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.151+0000 7fa48285d700 1 -- 172.21.15.143:0/2645273491 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa47c06ed60 msgr2=0x7fa47c0c9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:04.164 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.151+0000 7fa48285d700 1 --2- 172.21.15.143:0/2645273491 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa47c06ed60 0x7fa47c0c9d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:04.164 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.151+0000 7fa48285d700 1 -- 172.21.15.143:0/2645273491 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa47c0cc410 msgr2=0x7fa47c1f49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:04.164 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.151+0000 7fa48285d700 1 --2- 172.21.15.143:0/2645273491 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa47c0cc410 0x7fa47c1f49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:04.164 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.151+0000 7fa48285d700 1 -- 172.21.15.143:0/2645273491 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4780094a0 con 0x7fa47c159870 2023-11-08T19:25:04.165 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.151+0000 7fa48285d700 1 --2- 172.21.15.143:0/2645273491 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa47c159870 0x7fa47c1f4f10 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa474007df0 tx=0x7fa47400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:04.165 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.151+0000 7fa472ffd700 1 -- 172.21.15.143:0/2645273491 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa47400dd20 con 0x7fa47c159870 2023-11-08T19:25:04.165 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.151+0000 7fa472ffd700 1 -- 172.21.15.143:0/2645273491 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa47400f040 con 0x7fa47c159870 2023-11-08T19:25:04.165 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.151+0000 7fa472ffd700 1 -- 172.21.15.143:0/2645273491 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa474003c60 con 0x7fa47c159870 2023-11-08T19:25:04.166 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.151+0000 7fa4842c0700 1 -- 172.21.15.143:0/2645273491 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa47c159c50 con 0x7fa47c159870 2023-11-08T19:25:04.166 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.151+0000 7fa4842c0700 1 -- 172.21.15.143:0/2645273491 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa47c0ca260 con 0x7fa47c159870 2023-11-08T19:25:04.166 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.152+0000 7fa4842c0700 1 -- 172.21.15.143:0/2645273491 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa47c04eee0 con 0x7fa47c159870 2023-11-08T19:25:04.166 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.152+0000 7fa472ffd700 1 -- 172.21.15.143:0/2645273491 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa474004030 con 0x7fa47c159870 2023-11-08T19:25:04.166 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.153+0000 7fa472ffd700 1 --2- 172.21.15.143:0/2645273491 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa46006af00 0x7fa46006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:04.167 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.153+0000 7fa472ffd700 1 -- 172.21.15.143:0/2645273491 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa474089560 con 0x7fa47c159870 2023-11-08T19:25:04.167 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.153+0000 7fa48205c700 1 --2- 172.21.15.143:0/2645273491 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa46006af00 0x7fa46006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:04.167 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.153+0000 7fa48205c700 1 --2- 172.21.15.143:0/2645273491 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa46006af00 0x7fa46006d3c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa46c0010e0 tx=0x7fa46c007c70 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:04.167 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.157+0000 7fa472ffd700 1 -- 172.21.15.143:0/2645273491 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa474054760 con 0x7fa47c159870 2023-11-08T19:25:04.344 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.342+0000 7fa4842c0700 1 -- 172.21.15.143:0/2645273491 --> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa47c0ccd60 con 0x7fa46006af00 2023-11-08T19:25:04.351 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.350+0000 7fa472ffd700 1 -- 172.21.15.143:0/2645273491 <== mgr.4099 v2:172.21.15.174:6832/4037280178 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264590 (secure 0 0 0) 0x7fa46c005830 con 0x7fa46006af00 2023-11-08T19:25:04.354 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:25:04.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.353+0000 7fa4842c0700 1 -- 172.21.15.143:0/2645273491 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa46006af00 msgr2=0x7fa46006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:04.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.353+0000 7fa4842c0700 1 --2- 172.21.15.143:0/2645273491 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa46006af00 0x7fa46006d3c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa46c0010e0 tx=0x7fa46c007c70 comp rx=0 tx=0).stop 2023-11-08T19:25:04.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.353+0000 7fa4842c0700 1 -- 172.21.15.143:0/2645273491 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa47c159870 msgr2=0x7fa47c1f4f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:04.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.353+0000 7fa4842c0700 1 --2- 172.21.15.143:0/2645273491 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa47c159870 0x7fa47c1f4f10 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa474007df0 tx=0x7fa47400d3b0 comp rx=0 tx=0).stop 2023-11-08T19:25:04.358 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.353+0000 7fa4842c0700 1 -- 172.21.15.143:0/2645273491 shutdown_connections 2023-11-08T19:25:04.358 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.353+0000 7fa4842c0700 1 --2- 172.21.15.143:0/2645273491 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa46006af00 0x7fa46006d3c0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:04.358 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.353+0000 7fa4842c0700 1 --2- 172.21.15.143:0/2645273491 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa47c06ed60 0x7fa47c0c9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:04.358 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.353+0000 7fa4842c0700 1 --2- 172.21.15.143:0/2645273491 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa47c159870 0x7fa47c1f4f10 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:04.358 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.353+0000 7fa4842c0700 1 --2- 172.21.15.143:0/2645273491 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa47c0cc410 0x7fa47c1f49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:04.359 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.353+0000 7fa4842c0700 1 -- 172.21.15.143:0/2645273491 >> 172.21.15.143:0/2645273491 conn(0x7fa47c099ab0 msgr2=0x7fa47c0ccee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:04.359 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.353+0000 7fa4842c0700 1 -- 172.21.15.143:0/2645273491 shutdown_connections 2023-11-08T19:25:04.359 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.353+0000 7fa4842c0700 1 -- 172.21.15.143:0/2645273491 wait complete. 2023-11-08T19:25:04.359 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2023-11-08T19:25:04.366 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2023-11-08T19:25:02.511635+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219024,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755696,"statfs":{"total":773094113280,"available":772869832704,"internally_reserved":0,"allocated":2891776,"data_stored":1752836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.883726"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.631402+0000","last_change":"2023-11-08T19:24:50.618820+0000","last_active":"2023-11-08T19:24:51.631402+0000","last_peered":"2023-11-08T19:24:51.631402+0000","last_clean":"2023-11-08T19:24:51.631402+0000","last_became_active":"2023-11-08T19:24:50.618531+0000","last_became_peered":"2023-11-08T19:24:50.618531+0000","last_unstale":"2023-11-08T19:24:51.631402+0000","last_undegraded":"2023-11-08T19:24:51.631402+0000","last_fullsized":"2023-11-08T19:24:51.631402+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:33:19.902860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.805606+0000","last_change":"2023-11-08T19:24:50.623985+0000","last_active":"2023-11-08T19:24:52.805606+0000","last_peered":"2023-11-08T19:24:52.805606+0000","last_clean":"2023-11-08T19:24:52.805606+0000","last_became_active":"2023-11-08T19:24:50.623553+0000","last_became_peered":"2023-11-08T19:24:50.623553+0000","last_unstale":"2023-11-08T19:24:52.805606+0000","last_undegraded":"2023-11-08T19:24:52.805606+0000","last_fullsized":"2023-11-08T19:24:52.805606+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:13:58.948371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.309774+0000","last_change":"2023-11-08T19:24:50.617914+0000","last_active":"2023-11-08T19:24:52.309774+0000","last_peered":"2023-11-08T19:24:52.309774+0000","last_clean":"2023-11-08T19:24:52.309774+0000","last_became_active":"2023-11-08T19:24:50.617611+0000","last_became_peered":"2023-11-08T19:24:50.617611+0000","last_unstale":"2023-11-08T19:24:52.309774+0000","last_undegraded":"2023-11-08T19:24:52.309774+0000","last_fullsized":"2023-11-08T19:24:52.309774+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:30:36.041820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.352504+0000","last_change":"2023-11-08T19:24:50.640039+0000","last_active":"2023-11-08T19:24:52.352504+0000","last_peered":"2023-11-08T19:24:52.352504+0000","last_clean":"2023-11-08T19:24:52.352504+0000","last_became_active":"2023-11-08T19:24:50.639560+0000","last_became_peered":"2023-11-08T19:24:50.639560+0000","last_unstale":"2023-11-08T19:24:52.352504+0000","last_undegraded":"2023-11-08T19:24:52.352504+0000","last_fullsized":"2023-11-08T19:24:52.352504+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:01:53.410545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.631276+0000","last_change":"2023-11-08T19:24:50.617837+0000","last_active":"2023-11-08T19:24:51.631276+0000","last_peered":"2023-11-08T19:24:51.631276+0000","last_clean":"2023-11-08T19:24:51.631276+0000","last_became_active":"2023-11-08T19:24:50.617537+0000","last_became_peered":"2023-11-08T19:24:50.617537+0000","last_unstale":"2023-11-08T19:24:51.631276+0000","last_undegraded":"2023-11-08T19:24:51.631276+0000","last_fullsized":"2023-11-08T19:24:51.631276+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:59:37.592538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.351953+0000","last_change":"2023-11-08T19:24:50.641005+0000","last_active":"2023-11-08T19:24:52.351953+0000","last_peered":"2023-11-08T19:24:52.351953+0000","last_clean":"2023-11-08T19:24:52.351953+0000","last_became_active":"2023-11-08T19:24:50.640727+0000","last_became_peered":"2023-11-08T19:24:50.640727+0000","last_unstale":"2023-11-08T19:24:52.351953+0000","last_undegraded":"2023-11-08T19:24:52.351953+0000","last_fullsized":"2023-11-08T19:24:52.351953+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:36:53.967796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.630219+0000","last_change":"2023-11-08T19:24:50.625311+0000","last_active":"2023-11-08T19:24:51.630219+0000","last_peered":"2023-11-08T19:24:51.630219+0000","last_clean":"2023-11-08T19:24:51.630219+0000","last_became_active":"2023-11-08T19:24:50.625012+0000","last_became_peered":"2023-11-08T19:24:50.625012+0000","last_unstale":"2023-11-08T19:24:51.630219+0000","last_undegraded":"2023-11-08T19:24:51.630219+0000","last_fullsized":"2023-11-08T19:24:51.630219+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:49:19.100370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.626351+0000","last_change":"2023-11-08T19:24:50.639325+0000","last_active":"2023-11-08T19:24:51.626351+0000","last_peered":"2023-11-08T19:24:51.626351+0000","last_clean":"2023-11-08T19:24:51.626351+0000","last_became_active":"2023-11-08T19:24:50.638993+0000","last_became_peered":"2023-11-08T19:24:50.638993+0000","last_unstale":"2023-11-08T19:24:51.626351+0000","last_undegraded":"2023-11-08T19:24:51.626351+0000","last_fullsized":"2023-11-08T19:24:51.626351+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:30:32.602976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.631383+0000","last_change":"2023-11-08T19:24:50.638920+0000","last_active":"2023-11-08T19:24:51.631383+0000","last_peered":"2023-11-08T19:24:51.631383+0000","last_clean":"2023-11-08T19:24:51.631383+0000","last_became_active":"2023-11-08T19:24:50.638607+0000","last_became_peered":"2023-11-08T19:24:50.638607+0000","last_unstale":"2023-11-08T19:24:51.631383+0000","last_undegraded":"2023-11-08T19:24:51.631383+0000","last_fullsized":"2023-11-08T19:24:51.631383+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:09:01.749686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.630901+0000","last_change":"2023-11-08T19:24:50.637631+0000","last_active":"2023-11-08T19:24:51.630901+0000","last_peered":"2023-11-08T19:24:51.630901+0000","last_clean":"2023-11-08T19:24:51.630901+0000","last_became_active":"2023-11-08T19:24:50.637346+0000","last_became_peered":"2023-11-08T19:24:50.637346+0000","last_unstale":"2023-11-08T19:24:51.630901+0000","last_undegraded":"2023-11-08T19:24:51.630901+0000","last_fullsized":"2023-11-08T19:24:51.630901+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:13:58.948371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.630498+0000","last_change":"2023-11-08T19:24:50.642227+0000","last_active":"2023-11-08T19:24:51.630498+0000","last_peered":"2023-11-08T19:24:51.630498+0000","last_clean":"2023-11-08T19:24:51.630498+0000","last_became_active":"2023-11-08T19:24:50.641955+0000","last_became_peered":"2023-11-08T19:24:50.641955+0000","last_unstale":"2023-11-08T19:24:51.630498+0000","last_undegraded":"2023-11-08T19:24:51.630498+0000","last_fullsized":"2023-11-08T19:24:51.630498+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:40:11.978983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.805767+0000","last_change":"2023-11-08T19:24:50.621511+0000","last_active":"2023-11-08T19:24:52.805767+0000","last_peered":"2023-11-08T19:24:52.805767+0000","last_clean":"2023-11-08T19:24:52.805767+0000","last_became_active":"2023-11-08T19:24:50.621142+0000","last_became_peered":"2023-11-08T19:24:50.621142+0000","last_unstale":"2023-11-08T19:24:52.805767+0000","last_undegraded":"2023-11-08T19:24:52.805767+0000","last_fullsized":"2023-11-08T19:24:52.805767+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:14:01.459274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.626625+0000","last_change":"2023-11-08T19:24:50.625632+0000","last_active":"2023-11-08T19:24:51.626625+0000","last_peered":"2023-11-08T19:24:51.626625+0000","last_clean":"2023-11-08T19:24:51.626625+0000","last_became_active":"2023-11-08T19:24:50.625370+0000","last_became_peered":"2023-11-08T19:24:50.625370+0000","last_unstale":"2023-11-08T19:24:51.626625+0000","last_undegraded":"2023-11-08T19:24:51.626625+0000","last_fullsized":"2023-11-08T19:24:51.626625+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:13:58.948371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.309818+0000","last_change":"2023-11-08T19:24:50.625350+0000","last_active":"2023-11-08T19:24:52.309818+0000","last_peered":"2023-11-08T19:24:52.309818+0000","last_clean":"2023-11-08T19:24:52.309818+0000","last_became_active":"2023-11-08T19:24:50.625078+0000","last_became_peered":"2023-11-08T19:24:50.625078+0000","last_unstale":"2023-11-08T19:24:52.309818+0000","last_undegraded":"2023-11-08T19:24:52.309818+0000","last_fullsized":"2023-11-08T19:24:52.309818+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:42:42.512786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.804817+0000","last_change":"2023-11-08T19:24:50.629187+0000","last_active":"2023-11-08T19:24:52.804817+0000","last_peered":"2023-11-08T19:24:52.804817+0000","last_clean":"2023-11-08T19:24:52.804817+0000","last_became_active":"2023-11-08T19:24:50.628898+0000","last_became_peered":"2023-11-08T19:24:50.628898+0000","last_unstale":"2023-11-08T19:24:52.804817+0000","last_undegraded":"2023-11-08T19:24:52.804817+0000","last_fullsized":"2023-11-08T19:24:52.804817+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:15:33.285449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.351906+0000","last_change":"2023-11-08T19:24:50.638579+0000","last_active":"2023-11-08T19:24:52.351906+0000","last_peered":"2023-11-08T19:24:52.351906+0000","last_clean":"2023-11-08T19:24:52.351906+0000","last_became_active":"2023-11-08T19:24:50.638297+0000","last_became_peered":"2023-11-08T19:24:50.638297+0000","last_unstale":"2023-11-08T19:24:52.351906+0000","last_undegraded":"2023-11-08T19:24:52.351906+0000","last_fullsized":"2023-11-08T19:24:52.351906+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:13:58.948371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.352572+0000","last_change":"2023-11-08T19:24:50.641446+0000","last_active":"2023-11-08T19:24:52.352572+0000","last_peered":"2023-11-08T19:24:52.352572+0000","last_clean":"2023-11-08T19:24:52.352572+0000","last_became_active":"2023-11-08T19:24:50.639709+0000","last_became_peered":"2023-11-08T19:24:50.639709+0000","last_unstale":"2023-11-08T19:24:52.352572+0000","last_undegraded":"2023-11-08T19:24:52.352572+0000","last_fullsized":"2023-11-08T19:24:52.352572+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:09:01.749686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.804922+0000","last_change":"2023-11-08T19:24:50.619241+0000","last_active":"2023-11-08T19:24:52.804922+0000","last_peered":"2023-11-08T19:24:52.804922+0000","last_clean":"2023-11-08T19:24:52.804922+0000","last_became_active":"2023-11-08T19:24:50.618962+0000","last_became_peered":"2023-11-08T19:24:50.618962+0000","last_unstale":"2023-11-08T19:24:52.804922+0000","last_undegraded":"2023-11-08T19:24:52.804922+0000","last_fullsized":"2023-11-08T19:24:52.804922+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:14:01.459274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.631027+0000","last_change":"2023-11-08T19:24:50.618043+0000","last_active":"2023-11-08T19:24:51.631027+0000","last_peered":"2023-11-08T19:24:51.631027+0000","last_clean":"2023-11-08T19:24:51.631027+0000","last_became_active":"2023-11-08T19:24:50.617727+0000","last_became_peered":"2023-11-08T19:24:50.617727+0000","last_unstale":"2023-11-08T19:24:51.631027+0000","last_undegraded":"2023-11-08T19:24:51.631027+0000","last_fullsized":"2023-11-08T19:24:51.631027+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:13:41.469149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.309784+0000","last_change":"2023-11-08T19:24:49.608481+0000","last_active":"2023-11-08T19:24:52.309784+0000","last_peered":"2023-11-08T19:24:52.309784+0000","last_clean":"2023-11-08T19:24:52.309784+0000","last_became_active":"2023-11-08T19:24:49.607778+0000","last_became_peered":"2023-11-08T19:24:49.607778+0000","last_unstale":"2023-11-08T19:24:52.309784+0000","last_undegraded":"2023-11-08T19:24:52.309784+0000","last_fullsized":"2023-11-08T19:24:52.309784+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:01:09.884262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.626900+0000","last_change":"2023-11-08T19:24:50.624852+0000","last_active":"2023-11-08T19:24:51.626900+0000","last_peered":"2023-11-08T19:24:51.626900+0000","last_clean":"2023-11-08T19:24:51.626900+0000","last_became_active":"2023-11-08T19:24:50.624551+0000","last_became_peered":"2023-11-08T19:24:50.624551+0000","last_unstale":"2023-11-08T19:24:51.626900+0000","last_undegraded":"2023-11-08T19:24:51.626900+0000","last_fullsized":"2023-11-08T19:24:51.626900+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:12:54.641952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.626938+0000","last_change":"2023-11-08T19:24:49.610418+0000","last_active":"2023-11-08T19:24:51.626938+0000","last_peered":"2023-11-08T19:24:51.626938+0000","last_clean":"2023-11-08T19:24:51.626938+0000","last_became_active":"2023-11-08T19:24:49.610041+0000","last_became_peered":"2023-11-08T19:24:49.610041+0000","last_unstale":"2023-11-08T19:24:51.626938+0000","last_undegraded":"2023-11-08T19:24:51.626938+0000","last_fullsized":"2023-11-08T19:24:51.626938+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:00:12.067877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.309779+0000","last_change":"2023-11-08T19:24:50.627052+0000","last_active":"2023-11-08T19:24:52.309779+0000","last_peered":"2023-11-08T19:24:52.309779+0000","last_clean":"2023-11-08T19:24:52.309779+0000","last_became_active":"2023-11-08T19:24:50.626751+0000","last_became_peered":"2023-11-08T19:24:50.626751+0000","last_unstale":"2023-11-08T19:24:52.309779+0000","last_undegraded":"2023-11-08T19:24:52.309779+0000","last_fullsized":"2023-11-08T19:24:52.309779+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:46:19.375897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-08T19:24:50.639795+0000","last_change":"2023-11-08T19:24:49.605677+0000","last_active":"2023-11-08T19:24:50.639795+0000","last_peered":"2023-11-08T19:24:50.639795+0000","last_clean":"2023-11-08T19:24:50.639795+0000","last_became_active":"2023-11-08T19:24:49.605399+0000","last_became_peered":"2023-11-08T19:24:49.605399+0000","last_unstale":"2023-11-08T19:24:50.639795+0000","last_undegraded":"2023-11-08T19:24:50.639795+0000","last_fullsized":"2023-11-08T19:24:50.639795+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:46:18.372121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.352125+0000","last_change":"2023-11-08T19:24:50.641183+0000","last_active":"2023-11-08T19:24:52.352125+0000","last_peered":"2023-11-08T19:24:52.352125+0000","last_clean":"2023-11-08T19:24:52.352125+0000","last_became_active":"2023-11-08T19:24:50.640910+0000","last_became_peered":"2023-11-08T19:24:50.640910+0000","last_unstale":"2023-11-08T19:24:52.352125+0000","last_undegraded":"2023-11-08T19:24:52.352125+0000","last_fullsized":"2023-11-08T19:24:52.352125+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:49:19.100370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-08T19:24:50.642030+0000","last_change":"2023-11-08T19:24:49.614714+0000","last_active":"2023-11-08T19:24:50.642030+0000","last_peered":"2023-11-08T19:24:50.642030+0000","last_clean":"2023-11-08T19:24:50.642030+0000","last_became_active":"2023-11-08T19:24:49.614390+0000","last_became_peered":"2023-11-08T19:24:49.614390+0000","last_unstale":"2023-11-08T19:24:50.642030+0000","last_undegraded":"2023-11-08T19:24:50.642030+0000","last_fullsized":"2023-11-08T19:24:50.642030+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:46:12.493411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.352284+0000","last_change":"2023-11-08T19:24:50.640245+0000","last_active":"2023-11-08T19:24:52.352284+0000","last_peered":"2023-11-08T19:24:52.352284+0000","last_clean":"2023-11-08T19:24:52.352284+0000","last_became_active":"2023-11-08T19:24:50.639668+0000","last_became_peered":"2023-11-08T19:24:50.639668+0000","last_unstale":"2023-11-08T19:24:52.352284+0000","last_undegraded":"2023-11-08T19:24:52.352284+0000","last_fullsized":"2023-11-08T19:24:52.352284+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:42:27.150066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.626599+0000","last_change":"2023-11-08T19:24:49.612665+0000","last_active":"2023-11-08T19:24:51.626599+0000","last_peered":"2023-11-08T19:24:51.626599+0000","last_clean":"2023-11-08T19:24:51.626599+0000","last_became_active":"2023-11-08T19:24:49.612362+0000","last_became_peered":"2023-11-08T19:24:49.612362+0000","last_unstale":"2023-11-08T19:24:51.626599+0000","last_undegraded":"2023-11-08T19:24:51.626599+0000","last_fullsized":"2023-11-08T19:24:51.626599+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:46:12.493411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.805419+0000","last_change":"2023-11-08T19:24:50.627537+0000","last_active":"2023-11-08T19:24:52.805419+0000","last_peered":"2023-11-08T19:24:52.805419+0000","last_clean":"2023-11-08T19:24:52.805419+0000","last_became_active":"2023-11-08T19:24:50.626489+0000","last_became_peered":"2023-11-08T19:24:50.626489+0000","last_unstale":"2023-11-08T19:24:52.805419+0000","last_undegraded":"2023-11-08T19:24:52.805419+0000","last_fullsized":"2023-11-08T19:24:52.805419+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:01:10.888039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-08T19:24:50.637474+0000","last_change":"2023-11-08T19:24:49.606613+0000","last_active":"2023-11-08T19:24:50.637474+0000","last_peered":"2023-11-08T19:24:50.637474+0000","last_clean":"2023-11-08T19:24:50.637474+0000","last_became_active":"2023-11-08T19:24:49.606324+0000","last_became_peered":"2023-11-08T19:24:49.606324+0000","last_unstale":"2023-11-08T19:24:50.637474+0000","last_undegraded":"2023-11-08T19:24:50.637474+0000","last_fullsized":"2023-11-08T19:24:50.637474+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:07:00.575561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.309935+0000","last_change":"2023-11-08T19:24:50.627987+0000","last_active":"2023-11-08T19:24:52.309935+0000","last_peered":"2023-11-08T19:24:52.309935+0000","last_clean":"2023-11-08T19:24:52.309935+0000","last_became_active":"2023-11-08T19:24:50.627402+0000","last_became_peered":"2023-11-08T19:24:50.627402+0000","last_unstale":"2023-11-08T19:24:52.309935+0000","last_undegraded":"2023-11-08T19:24:52.309935+0000","last_fullsized":"2023-11-08T19:24:52.309935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:36:53.967796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.626523+0000","last_change":"2023-11-08T19:24:49.606482+0000","last_active":"2023-11-08T19:24:51.626523+0000","last_peered":"2023-11-08T19:24:51.626523+0000","last_clean":"2023-11-08T19:24:51.626523+0000","last_became_active":"2023-11-08T19:24:49.606211+0000","last_became_peered":"2023-11-08T19:24:49.606211+0000","last_unstale":"2023-11-08T19:24:51.626523+0000","last_undegraded":"2023-11-08T19:24:51.626523+0000","last_fullsized":"2023-11-08T19:24:51.626523+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:42:59.845581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.626483+0000","last_change":"2023-11-08T19:24:50.625917+0000","last_active":"2023-11-08T19:24:51.626483+0000","last_peered":"2023-11-08T19:24:51.626483+0000","last_clean":"2023-11-08T19:24:51.626483+0000","last_became_active":"2023-11-08T19:24:50.625398+0000","last_became_peered":"2023-11-08T19:24:50.625398+0000","last_unstale":"2023-11-08T19:24:51.626483+0000","last_undegraded":"2023-11-08T19:24:51.626483+0000","last_fullsized":"2023-11-08T19:24:51.626483+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:40:11.978983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.351905+0000","last_change":"2023-11-08T19:24:49.616558+0000","last_active":"2023-11-08T19:24:52.351905+0000","last_peered":"2023-11-08T19:24:52.351905+0000","last_clean":"2023-11-08T19:24:52.351905+0000","last_became_active":"2023-11-08T19:24:49.616277+0000","last_became_peered":"2023-11-08T19:24:49.616277+0000","last_unstale":"2023-11-08T19:24:52.351905+0000","last_undegraded":"2023-11-08T19:24:52.351905+0000","last_fullsized":"2023-11-08T19:24:52.351905+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:01:09.884262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.805048+0000","last_change":"2023-11-08T19:24:50.623658+0000","last_active":"2023-11-08T19:24:52.805048+0000","last_peered":"2023-11-08T19:24:52.805048+0000","last_clean":"2023-11-08T19:24:52.805048+0000","last_became_active":"2023-11-08T19:24:50.623399+0000","last_became_peered":"2023-11-08T19:24:50.623399+0000","last_unstale":"2023-11-08T19:24:52.805048+0000","last_undegraded":"2023-11-08T19:24:52.805048+0000","last_fullsized":"2023-11-08T19:24:52.805048+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:40:11.978983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-08T19:24:50.610454+0000","last_change":"2023-11-08T19:24:49.608767+0000","last_active":"2023-11-08T19:24:50.610454+0000","last_peered":"2023-11-08T19:24:50.610454+0000","last_clean":"2023-11-08T19:24:50.610454+0000","last_became_active":"2023-11-08T19:24:49.608450+0000","last_became_peered":"2023-11-08T19:24:49.608450+0000","last_unstale":"2023-11-08T19:24:50.610454+0000","last_undegraded":"2023-11-08T19:24:50.610454+0000","last_fullsized":"2023-11-08T19:24:50.610454+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:42:59.845581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.352305+0000","last_change":"2023-11-08T19:24:50.640397+0000","last_active":"2023-11-08T19:24:52.352305+0000","last_peered":"2023-11-08T19:24:52.352305+0000","last_clean":"2023-11-08T19:24:52.352305+0000","last_became_active":"2023-11-08T19:24:50.639730+0000","last_became_peered":"2023-11-08T19:24:50.639730+0000","last_unstale":"2023-11-08T19:24:52.352305+0000","last_undegraded":"2023-11-08T19:24:52.352305+0000","last_fullsized":"2023-11-08T19:24:52.352305+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:08:22.499163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.352212+0000","last_change":"2023-11-08T19:24:49.611202+0000","last_active":"2023-11-08T19:24:52.352212+0000","last_peered":"2023-11-08T19:24:52.352212+0000","last_clean":"2023-11-08T19:24:52.352212+0000","last_became_active":"2023-11-08T19:24:49.610894+0000","last_became_peered":"2023-11-08T19:24:49.610894+0000","last_unstale":"2023-11-08T19:24:52.352212+0000","last_undegraded":"2023-11-08T19:24:52.352212+0000","last_fullsized":"2023-11-08T19:24:52.352212+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:48:25.546248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.309935+0000","last_change":"2023-11-08T19:24:50.638901+0000","last_active":"2023-11-08T19:24:52.309935+0000","last_peered":"2023-11-08T19:24:52.309935+0000","last_clean":"2023-11-08T19:24:52.309935+0000","last_became_active":"2023-11-08T19:24:50.638587+0000","last_became_peered":"2023-11-08T19:24:50.638587+0000","last_unstale":"2023-11-08T19:24:52.309935+0000","last_undegraded":"2023-11-08T19:24:52.309935+0000","last_fullsized":"2023-11-08T19:24:52.309935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:13:58.948371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.805313+0000","last_change":"2023-11-08T19:24:49.612170+0000","last_active":"2023-11-08T19:24:52.805313+0000","last_peered":"2023-11-08T19:24:52.805313+0000","last_clean":"2023-11-08T19:24:52.805313+0000","last_became_active":"2023-11-08T19:24:49.611895+0000","last_became_peered":"2023-11-08T19:24:49.611895+0000","last_unstale":"2023-11-08T19:24:52.805313+0000","last_undegraded":"2023-11-08T19:24:52.805313+0000","last_fullsized":"2023-11-08T19:24:52.805313+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:38:24.246638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.805721+0000","last_change":"2023-11-08T19:24:50.627060+0000","last_active":"2023-11-08T19:24:52.805721+0000","last_peered":"2023-11-08T19:24:52.805721+0000","last_clean":"2023-11-08T19:24:52.805721+0000","last_became_active":"2023-11-08T19:24:50.626468+0000","last_became_peered":"2023-11-08T19:24:50.626468+0000","last_unstale":"2023-11-08T19:24:52.805721+0000","last_undegraded":"2023-11-08T19:24:52.805721+0000","last_fullsized":"2023-11-08T19:24:52.805721+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:00:13.071654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":18,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-08T19:24:50.645041+0000","last_change":"2023-11-08T19:24:49.610676+0000","last_active":"2023-11-08T19:24:50.645041+0000","last_peered":"2023-11-08T19:24:50.645041+0000","last_clean":"2023-11-08T19:24:50.645041+0000","last_became_active":"2023-11-08T19:24:49.610344+0000","last_became_peered":"2023-11-08T19:24:49.610344+0000","last_unstale":"2023-11-08T19:24:50.645041+0000","last_undegraded":"2023-11-08T19:24:50.645041+0000","last_fullsized":"2023-11-08T19:24:50.645041+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:46:12.493411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.631028+0000","last_change":"2023-11-08T19:24:50.626565+0000","last_active":"2023-11-08T19:24:51.631028+0000","last_peered":"2023-11-08T19:24:51.631028+0000","last_clean":"2023-11-08T19:24:51.631028+0000","last_became_active":"2023-11-08T19:24:50.626280+0000","last_became_peered":"2023-11-08T19:24:50.626280+0000","last_unstale":"2023-11-08T19:24:51.631028+0000","last_undegraded":"2023-11-08T19:24:51.631028+0000","last_fullsized":"2023-11-08T19:24:51.631028+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:08:22.499163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.309884+0000","last_change":"2023-11-08T19:24:49.610471+0000","last_active":"2023-11-08T19:24:52.309884+0000","last_peered":"2023-11-08T19:24:52.309884+0000","last_clean":"2023-11-08T19:24:52.309884+0000","last_became_active":"2023-11-08T19:24:49.610182+0000","last_became_peered":"2023-11-08T19:24:49.610182+0000","last_unstale":"2023-11-08T19:24:52.309884+0000","last_undegraded":"2023-11-08T19:24:52.309884+0000","last_fullsized":"2023-11-08T19:24:52.309884+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:14:00.455497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.309602+0000","last_change":"2023-11-08T19:24:50.619498+0000","last_active":"2023-11-08T19:24:52.309602+0000","last_peered":"2023-11-08T19:24:52.309602+0000","last_clean":"2023-11-08T19:24:52.309602+0000","last_became_active":"2023-11-08T19:24:50.619231+0000","last_became_peered":"2023-11-08T19:24:50.619231+0000","last_unstale":"2023-11-08T19:24:52.309602+0000","last_undegraded":"2023-11-08T19:24:52.309602+0000","last_fullsized":"2023-11-08T19:24:52.309602+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:38:25.250414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.352455+0000","last_change":"2023-11-08T19:24:49.612336+0000","last_active":"2023-11-08T19:24:52.352455+0000","last_peered":"2023-11-08T19:24:52.352455+0000","last_clean":"2023-11-08T19:24:52.352455+0000","last_became_active":"2023-11-08T19:24:49.611885+0000","last_became_peered":"2023-11-08T19:24:49.611885+0000","last_unstale":"2023-11-08T19:24:52.352455+0000","last_undegraded":"2023-11-08T19:24:52.352455+0000","last_fullsized":"2023-11-08T19:24:52.352455+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:46:18.372121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.631204+0000","last_change":"2023-11-08T19:24:50.629107+0000","last_active":"2023-11-08T19:24:51.631204+0000","last_peered":"2023-11-08T19:24:51.631204+0000","last_clean":"2023-11-08T19:24:51.631204+0000","last_became_active":"2023-11-08T19:24:50.628005+0000","last_became_peered":"2023-11-08T19:24:50.628005+0000","last_unstale":"2023-11-08T19:24:51.631204+0000","last_undegraded":"2023-11-08T19:24:51.631204+0000","last_fullsized":"2023-11-08T19:24:51.631204+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:34:42.075394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-08T19:24:50.609767+0000","last_change":"2023-11-08T19:24:49.605863+0000","last_active":"2023-11-08T19:24:50.609767+0000","last_peered":"2023-11-08T19:24:50.609767+0000","last_clean":"2023-11-08T19:24:50.609767+0000","last_became_active":"2023-11-08T19:24:49.604986+0000","last_became_peered":"2023-11-08T19:24:49.604986+0000","last_unstale":"2023-11-08T19:24:50.609767+0000","last_undegraded":"2023-11-08T19:24:50.609767+0000","last_fullsized":"2023-11-08T19:24:50.609767+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:15:32.030617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.629786+0000","last_change":"2023-11-08T19:24:50.618716+0000","last_active":"2023-11-08T19:24:51.629786+0000","last_peered":"2023-11-08T19:24:51.629786+0000","last_clean":"2023-11-08T19:24:51.629786+0000","last_became_active":"2023-11-08T19:24:50.618425+0000","last_became_peered":"2023-11-08T19:24:50.618425+0000","last_unstale":"2023-11-08T19:24:51.629786+0000","last_undegraded":"2023-11-08T19:24:51.629786+0000","last_fullsized":"2023-11-08T19:24:51.629786+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:15:33.285449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.804637+0000","last_change":"2023-11-08T19:24:49.608503+0000","last_active":"2023-11-08T19:24:52.804637+0000","last_peered":"2023-11-08T19:24:52.804637+0000","last_clean":"2023-11-08T19:24:52.804637+0000","last_became_active":"2023-11-08T19:24:49.608176+0000","last_became_peered":"2023-11-08T19:24:49.608176+0000","last_unstale":"2023-11-08T19:24:52.804637+0000","last_undegraded":"2023-11-08T19:24:52.804637+0000","last_fullsized":"2023-11-08T19:24:52.804637+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:17:50.005667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.630752+0000","last_change":"2023-11-08T19:24:50.629255+0000","last_active":"2023-11-08T19:24:51.630752+0000","last_peered":"2023-11-08T19:24:51.630752+0000","last_clean":"2023-11-08T19:24:51.630752+0000","last_became_active":"2023-11-08T19:24:50.628763+0000","last_became_peered":"2023-11-08T19:24:50.628763+0000","last_unstale":"2023-11-08T19:24:51.630752+0000","last_undegraded":"2023-11-08T19:24:51.630752+0000","last_fullsized":"2023-11-08T19:24:51.630752+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:42:27.150066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.805855+0000","last_change":"2023-11-08T19:24:49.610638+0000","last_active":"2023-11-08T19:24:52.805855+0000","last_peered":"2023-11-08T19:24:52.805855+0000","last_clean":"2023-11-08T19:24:52.805855+0000","last_became_active":"2023-11-08T19:24:49.610362+0000","last_became_peered":"2023-11-08T19:24:49.610362+0000","last_unstale":"2023-11-08T19:24:52.805855+0000","last_undegraded":"2023-11-08T19:24:52.805855+0000","last_fullsized":"2023-11-08T19:24:52.805855+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:15:32.030617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.631275+0000","last_change":"2023-11-08T19:24:50.627667+0000","last_active":"2023-11-08T19:24:51.631275+0000","last_peered":"2023-11-08T19:24:51.631275+0000","last_clean":"2023-11-08T19:24:51.631275+0000","last_became_active":"2023-11-08T19:24:50.627390+0000","last_became_peered":"2023-11-08T19:24:50.627390+0000","last_unstale":"2023-11-08T19:24:51.631275+0000","last_undegraded":"2023-11-08T19:24:51.631275+0000","last_fullsized":"2023-11-08T19:24:51.631275+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:01:53.410545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.805154+0000","last_change":"2023-11-08T19:24:49.613413+0000","last_active":"2023-11-08T19:24:52.805154+0000","last_peered":"2023-11-08T19:24:52.805154+0000","last_clean":"2023-11-08T19:24:52.805154+0000","last_became_active":"2023-11-08T19:24:49.613131+0000","last_became_peered":"2023-11-08T19:24:49.613131+0000","last_unstale":"2023-11-08T19:24:52.805154+0000","last_undegraded":"2023-11-08T19:24:52.805154+0000","last_fullsized":"2023-11-08T19:24:52.805154+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:46:12.493411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.352374+0000","last_change":"2023-11-08T19:24:50.640031+0000","last_active":"2023-11-08T19:24:52.352374+0000","last_peered":"2023-11-08T19:24:52.352374+0000","last_clean":"2023-11-08T19:24:52.352374+0000","last_became_active":"2023-11-08T19:24:50.639718+0000","last_became_peered":"2023-11-08T19:24:50.639718+0000","last_unstale":"2023-11-08T19:24:52.352374+0000","last_undegraded":"2023-11-08T19:24:52.352374+0000","last_fullsized":"2023-11-08T19:24:52.352374+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:08:17.096077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.352346+0000","last_change":"2023-11-08T19:24:49.612137+0000","last_active":"2023-11-08T19:24:52.352346+0000","last_peered":"2023-11-08T19:24:52.352346+0000","last_clean":"2023-11-08T19:24:52.352346+0000","last_became_active":"2023-11-08T19:24:49.611532+0000","last_became_peered":"2023-11-08T19:24:49.611532+0000","last_unstale":"2023-11-08T19:24:52.352346+0000","last_undegraded":"2023-11-08T19:24:52.352346+0000","last_fullsized":"2023-11-08T19:24:52.352346+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:40:10.975206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.805536+0000","last_change":"2023-11-08T19:24:50.630854+0000","last_active":"2023-11-08T19:24:52.805536+0000","last_peered":"2023-11-08T19:24:52.805536+0000","last_clean":"2023-11-08T19:24:52.805536+0000","last_became_active":"2023-11-08T19:24:50.630549+0000","last_became_peered":"2023-11-08T19:24:50.630549+0000","last_unstale":"2023-11-08T19:24:52.805536+0000","last_undegraded":"2023-11-08T19:24:52.805536+0000","last_fullsized":"2023-11-08T19:24:52.805536+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:48:26.550025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.804950+0000","last_change":"2023-11-08T19:24:50.620897+0000","last_active":"2023-11-08T19:24:52.804950+0000","last_peered":"2023-11-08T19:24:52.804950+0000","last_clean":"2023-11-08T19:24:52.804950+0000","last_became_active":"2023-11-08T19:24:50.620606+0000","last_became_peered":"2023-11-08T19:24:50.620606+0000","last_unstale":"2023-11-08T19:24:52.804950+0000","last_undegraded":"2023-11-08T19:24:52.804950+0000","last_fullsized":"2023-11-08T19:24:52.804950+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:30:36.041820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.804625+0000","last_change":"2023-11-08T19:24:49.612840+0000","last_active":"2023-11-08T19:24:52.804625+0000","last_peered":"2023-11-08T19:24:52.804625+0000","last_clean":"2023-11-08T19:24:52.804625+0000","last_became_active":"2023-11-08T19:24:49.612652+0000","last_became_peered":"2023-11-08T19:24:49.612652+0000","last_unstale":"2023-11-08T19:24:52.804625+0000","last_undegraded":"2023-11-08T19:24:52.804625+0000","last_fullsized":"2023-11-08T19:24:52.804625+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:48:25.546248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.626593+0000","last_change":"2023-11-08T19:24:50.642515+0000","last_active":"2023-11-08T19:24:51.626593+0000","last_peered":"2023-11-08T19:24:51.626593+0000","last_clean":"2023-11-08T19:24:51.626593+0000","last_became_active":"2023-11-08T19:24:50.642204+0000","last_became_peered":"2023-11-08T19:24:50.642204+0000","last_unstale":"2023-11-08T19:24:51.626593+0000","last_undegraded":"2023-11-08T19:24:51.626593+0000","last_fullsized":"2023-11-08T19:24:51.626593+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:42:42.512786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.351949+0000","last_change":"2023-11-08T19:24:49.605594+0000","last_active":"2023-11-08T19:24:52.351949+0000","last_peered":"2023-11-08T19:24:52.351949+0000","last_clean":"2023-11-08T19:24:52.351949+0000","last_became_active":"2023-11-08T19:24:49.605208+0000","last_became_peered":"2023-11-08T19:24:49.605208+0000","last_unstale":"2023-11-08T19:24:52.351949+0000","last_undegraded":"2023-11-08T19:24:52.351949+0000","last_fullsized":"2023-11-08T19:24:52.351949+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:17:50.005667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":165,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-08T19:24:50.613237+0000","last_change":"2023-11-08T19:24:45.888122+0000","last_active":"2023-11-08T19:24:50.613237+0000","last_peered":"2023-11-08T19:24:50.613237+0000","last_clean":"2023-11-08T19:24:50.613237+0000","last_became_active":"2023-11-08T19:24:45.887048+0000","last_became_peered":"2023-11-08T19:24:45.887048+0000","last_unstale":"2023-11-08T19:24:50.613237+0000","last_undegraded":"2023-11-08T19:24:50.613237+0000","last_fullsized":"2023-11-08T19:24:50.613237+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:44.575133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:44.575133+0000","last_clean_scrub_stamp":"2023-11-08T19:24:44.575133+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:19:20.202062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.309622+0000","last_change":"2023-11-08T19:24:50.628353+0000","last_active":"2023-11-08T19:24:52.309622+0000","last_peered":"2023-11-08T19:24:52.309622+0000","last_clean":"2023-11-08T19:24:52.309622+0000","last_became_active":"2023-11-08T19:24:50.628008+0000","last_became_peered":"2023-11-08T19:24:50.628008+0000","last_unstale":"2023-11-08T19:24:52.309622+0000","last_undegraded":"2023-11-08T19:24:52.309622+0000","last_fullsized":"2023-11-08T19:24:52.309622+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:46:13.497187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-08T19:24:50.612010+0000","last_change":"2023-11-08T19:24:49.608659+0000","last_active":"2023-11-08T19:24:50.612010+0000","last_peered":"2023-11-08T19:24:50.612010+0000","last_clean":"2023-11-08T19:24:50.612010+0000","last_became_active":"2023-11-08T19:24:49.608243+0000","last_became_peered":"2023-11-08T19:24:49.608243+0000","last_unstale":"2023-11-08T19:24:50.612010+0000","last_undegraded":"2023-11-08T19:24:50.612010+0000","last_fullsized":"2023-11-08T19:24:50.612010+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:42:41.509009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-08T19:24:50.611395+0000","last_change":"2023-11-08T19:24:49.611793+0000","last_active":"2023-11-08T19:24:50.611395+0000","last_peered":"2023-11-08T19:24:50.611395+0000","last_clean":"2023-11-08T19:24:50.611395+0000","last_became_active":"2023-11-08T19:24:49.611489+0000","last_became_peered":"2023-11-08T19:24:49.611489+0000","last_unstale":"2023-11-08T19:24:50.611395+0000","last_undegraded":"2023-11-08T19:24:50.611395+0000","last_fullsized":"2023-11-08T19:24:50.611395+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:15:32.281672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.805231+0000","last_change":"2023-11-08T19:24:50.624279+0000","last_active":"2023-11-08T19:24:52.805231+0000","last_peered":"2023-11-08T19:24:52.805231+0000","last_clean":"2023-11-08T19:24:52.805231+0000","last_became_active":"2023-11-08T19:24:50.623920+0000","last_became_peered":"2023-11-08T19:24:50.623920+0000","last_unstale":"2023-11-08T19:24:52.805231+0000","last_undegraded":"2023-11-08T19:24:52.805231+0000","last_fullsized":"2023-11-08T19:24:52.805231+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:42:42.512786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:53.590119+0000","last_change":"2023-11-08T19:24:49.606910+0000","last_active":"2023-11-08T19:24:53.590119+0000","last_peered":"2023-11-08T19:24:53.590119+0000","last_clean":"2023-11-08T19:24:53.590119+0000","last_became_active":"2023-11-08T19:24:49.606627+0000","last_became_peered":"2023-11-08T19:24:49.606627+0000","last_unstale":"2023-11-08T19:24:53.590119+0000","last_undegraded":"2023-11-08T19:24:53.590119+0000","last_fullsized":"2023-11-08T19:24:53.590119+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:56:00.148860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.627087+0000","last_change":"2023-11-08T19:24:50.622398+0000","last_active":"2023-11-08T19:24:51.627087+0000","last_peered":"2023-11-08T19:24:51.627087+0000","last_clean":"2023-11-08T19:24:51.627087+0000","last_became_active":"2023-11-08T19:24:50.622092+0000","last_became_peered":"2023-11-08T19:24:50.622092+0000","last_unstale":"2023-11-08T19:24:51.627087+0000","last_undegraded":"2023-11-08T19:24:51.627087+0000","last_fullsized":"2023-11-08T19:24:51.627087+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:42:27.150066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-08T19:24:50.638180+0000","last_change":"2023-11-08T19:24:49.604285+0000","last_active":"2023-11-08T19:24:50.638180+0000","last_peered":"2023-11-08T19:24:50.638180+0000","last_clean":"2023-11-08T19:24:50.638180+0000","last_became_active":"2023-11-08T19:24:49.603965+0000","last_became_peered":"2023-11-08T19:24:49.603965+0000","last_unstale":"2023-11-08T19:24:50.638180+0000","last_undegraded":"2023-11-08T19:24:50.638180+0000","last_fullsized":"2023-11-08T19:24:50.638180+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:01:09.884262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.309770+0000","last_change":"2023-11-08T19:24:50.624848+0000","last_active":"2023-11-08T19:24:52.309770+0000","last_peered":"2023-11-08T19:24:52.309770+0000","last_clean":"2023-11-08T19:24:52.309770+0000","last_became_active":"2023-11-08T19:24:50.624518+0000","last_became_peered":"2023-11-08T19:24:50.624518+0000","last_unstale":"2023-11-08T19:24:52.309770+0000","last_undegraded":"2023-11-08T19:24:52.309770+0000","last_fullsized":"2023-11-08T19:24:52.309770+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:15:33.285449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-08T19:24:50.611239+0000","last_change":"2023-11-08T19:24:49.611779+0000","last_active":"2023-11-08T19:24:50.611239+0000","last_peered":"2023-11-08T19:24:50.611239+0000","last_clean":"2023-11-08T19:24:50.611239+0000","last_became_active":"2023-11-08T19:24:49.611422+0000","last_became_peered":"2023-11-08T19:24:49.611422+0000","last_unstale":"2023-11-08T19:24:50.611239+0000","last_undegraded":"2023-11-08T19:24:50.611239+0000","last_fullsized":"2023-11-08T19:24:50.611239+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:30:31.599200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.805558+0000","last_change":"2023-11-08T19:24:50.627488+0000","last_active":"2023-11-08T19:24:52.805558+0000","last_peered":"2023-11-08T19:24:52.805558+0000","last_clean":"2023-11-08T19:24:52.805558+0000","last_became_active":"2023-11-08T19:24:50.626571+0000","last_became_peered":"2023-11-08T19:24:50.626571+0000","last_unstale":"2023-11-08T19:24:52.805558+0000","last_undegraded":"2023-11-08T19:24:52.805558+0000","last_fullsized":"2023-11-08T19:24:52.805558+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:43:00.849357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.804911+0000","last_change":"2023-11-08T19:24:49.606579+0000","last_active":"2023-11-08T19:24:52.804911+0000","last_peered":"2023-11-08T19:24:52.804911+0000","last_clean":"2023-11-08T19:24:52.804911+0000","last_became_active":"2023-11-08T19:24:49.606283+0000","last_became_peered":"2023-11-08T19:24:49.606283+0000","last_unstale":"2023-11-08T19:24:52.804911+0000","last_undegraded":"2023-11-08T19:24:52.804911+0000","last_fullsized":"2023-11-08T19:24:52.804911+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:14:00.455497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.626421+0000","last_change":"2023-11-08T19:24:50.625692+0000","last_active":"2023-11-08T19:24:51.626421+0000","last_peered":"2023-11-08T19:24:51.626421+0000","last_clean":"2023-11-08T19:24:51.626421+0000","last_became_active":"2023-11-08T19:24:50.625014+0000","last_became_peered":"2023-11-08T19:24:50.625014+0000","last_unstale":"2023-11-08T19:24:51.626421+0000","last_undegraded":"2023-11-08T19:24:51.626421+0000","last_fullsized":"2023-11-08T19:24:51.626421+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:15:33.285449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.805081+0000","last_change":"2023-11-08T19:24:49.611023+0000","last_active":"2023-11-08T19:24:52.805081+0000","last_peered":"2023-11-08T19:24:52.805081+0000","last_clean":"2023-11-08T19:24:52.805081+0000","last_became_active":"2023-11-08T19:24:49.610735+0000","last_became_peered":"2023-11-08T19:24:49.610735+0000","last_unstale":"2023-11-08T19:24:52.805081+0000","last_undegraded":"2023-11-08T19:24:52.805081+0000","last_fullsized":"2023-11-08T19:24:52.805081+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:30:31.599200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.626730+0000","last_change":"2023-11-08T19:24:50.631491+0000","last_active":"2023-11-08T19:24:51.626730+0000","last_peered":"2023-11-08T19:24:51.626730+0000","last_clean":"2023-11-08T19:24:51.626730+0000","last_became_active":"2023-11-08T19:24:50.630872+0000","last_became_peered":"2023-11-08T19:24:50.630872+0000","last_unstale":"2023-11-08T19:24:51.626730+0000","last_undegraded":"2023-11-08T19:24:51.626730+0000","last_fullsized":"2023-11-08T19:24:51.626730+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:36:53.967796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-08T19:24:50.640533+0000","last_change":"2023-11-08T19:24:49.613965+0000","last_active":"2023-11-08T19:24:50.640533+0000","last_peered":"2023-11-08T19:24:50.640533+0000","last_clean":"2023-11-08T19:24:50.640533+0000","last_became_active":"2023-11-08T19:24:49.613629+0000","last_became_peered":"2023-11-08T19:24:49.613629+0000","last_unstale":"2023-11-08T19:24:50.640533+0000","last_undegraded":"2023-11-08T19:24:50.640533+0000","last_fullsized":"2023-11-08T19:24:50.640533+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:56:00.148860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.629891+0000","last_change":"2023-11-08T19:24:50.627696+0000","last_active":"2023-11-08T19:24:51.629891+0000","last_peered":"2023-11-08T19:24:51.629891+0000","last_clean":"2023-11-08T19:24:51.629891+0000","last_became_active":"2023-11-08T19:24:50.627346+0000","last_became_peered":"2023-11-08T19:24:50.627346+0000","last_unstale":"2023-11-08T19:24:51.629891+0000","last_undegraded":"2023-11-08T19:24:51.629891+0000","last_fullsized":"2023-11-08T19:24:51.629891+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:30:36.041820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.805091+0000","last_change":"2023-11-08T19:24:49.613365+0000","last_active":"2023-11-08T19:24:52.805091+0000","last_peered":"2023-11-08T19:24:52.805091+0000","last_clean":"2023-11-08T19:24:52.805091+0000","last_became_active":"2023-11-08T19:24:49.613052+0000","last_became_peered":"2023-11-08T19:24:49.613052+0000","last_unstale":"2023-11-08T19:24:52.805091+0000","last_undegraded":"2023-11-08T19:24:52.805091+0000","last_fullsized":"2023-11-08T19:24:52.805091+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:41:24.551269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.631232+0000","last_change":"2023-11-08T19:24:50.642477+0000","last_active":"2023-11-08T19:24:51.631232+0000","last_peered":"2023-11-08T19:24:51.631232+0000","last_clean":"2023-11-08T19:24:51.631232+0000","last_became_active":"2023-11-08T19:24:50.642186+0000","last_became_peered":"2023-11-08T19:24:50.642186+0000","last_unstale":"2023-11-08T19:24:51.631232+0000","last_undegraded":"2023-11-08T19:24:51.631232+0000","last_fullsized":"2023-11-08T19:24:51.631232+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:08:17.096077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.805219+0000","last_change":"2023-11-08T19:24:49.606271+0000","last_active":"2023-11-08T19:24:52.805219+0000","last_peered":"2023-11-08T19:24:52.805219+0000","last_clean":"2023-11-08T19:24:52.805219+0000","last_became_active":"2023-11-08T19:24:49.605802+0000","last_became_peered":"2023-11-08T19:24:49.605802+0000","last_unstale":"2023-11-08T19:24:52.805219+0000","last_undegraded":"2023-11-08T19:24:52.805219+0000","last_fullsized":"2023-11-08T19:24:52.805219+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:07:00.575561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.352338+0000","last_change":"2023-11-08T19:24:50.640594+0000","last_active":"2023-11-08T19:24:52.352338+0000","last_peered":"2023-11-08T19:24:52.352338+0000","last_clean":"2023-11-08T19:24:52.352338+0000","last_became_active":"2023-11-08T19:24:50.640295+0000","last_became_peered":"2023-11-08T19:24:50.640295+0000","last_unstale":"2023-11-08T19:24:52.352338+0000","last_undegraded":"2023-11-08T19:24:52.352338+0000","last_fullsized":"2023-11-08T19:24:52.352338+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:34:42.075394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.805026+0000","last_change":"2023-11-08T19:24:49.610782+0000","last_active":"2023-11-08T19:24:52.805026+0000","last_peered":"2023-11-08T19:24:52.805026+0000","last_clean":"2023-11-08T19:24:52.805026+0000","last_became_active":"2023-11-08T19:24:49.610493+0000","last_became_peered":"2023-11-08T19:24:49.610493+0000","last_unstale":"2023-11-08T19:24:52.805026+0000","last_undegraded":"2023-11-08T19:24:52.805026+0000","last_fullsized":"2023-11-08T19:24:52.805026+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:48.584965+0000","last_clean_scrub_stamp":"2023-11-08T19:24:48.584965+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:42:59.845581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.631518+0000","last_change":"2023-11-08T19:24:50.630352+0000","last_active":"2023-11-08T19:24:51.631518+0000","last_peered":"2023-11-08T19:24:51.631518+0000","last_clean":"2023-11-08T19:24:51.631518+0000","last_became_active":"2023-11-08T19:24:50.628170+0000","last_became_peered":"2023-11-08T19:24:50.628170+0000","last_unstale":"2023-11-08T19:24:51.631518+0000","last_undegraded":"2023-11-08T19:24:51.631518+0000","last_fullsized":"2023-11-08T19:24:51.631518+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:18:29.376849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.629782+0000","last_change":"2023-11-08T19:24:50.616189+0000","last_active":"2023-11-08T19:24:51.629782+0000","last_peered":"2023-11-08T19:24:51.629782+0000","last_clean":"2023-11-08T19:24:51.629782+0000","last_became_active":"2023-11-08T19:24:50.615704+0000","last_became_peered":"2023-11-08T19:24:50.615704+0000","last_unstale":"2023-11-08T19:24:51.629782+0000","last_undegraded":"2023-11-08T19:24:51.629782+0000","last_fullsized":"2023-11-08T19:24:51.629782+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:30:32.602976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.804775+0000","last_change":"2023-11-08T19:24:50.625948+0000","last_active":"2023-11-08T19:24:52.804775+0000","last_peered":"2023-11-08T19:24:52.804775+0000","last_clean":"2023-11-08T19:24:52.804775+0000","last_became_active":"2023-11-08T19:24:50.625633+0000","last_became_peered":"2023-11-08T19:24:50.625633+0000","last_unstale":"2023-11-08T19:24:52.804775+0000","last_undegraded":"2023-11-08T19:24:52.804775+0000","last_fullsized":"2023-11-08T19:24:52.804775+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:00:13.071654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.352423+0000","last_change":"2023-11-08T19:24:50.640919+0000","last_active":"2023-11-08T19:24:52.352423+0000","last_peered":"2023-11-08T19:24:52.352423+0000","last_clean":"2023-11-08T19:24:52.352423+0000","last_became_active":"2023-11-08T19:24:50.640496+0000","last_became_peered":"2023-11-08T19:24:50.640496+0000","last_unstale":"2023-11-08T19:24:52.352423+0000","last_undegraded":"2023-11-08T19:24:52.352423+0000","last_fullsized":"2023-11-08T19:24:52.352423+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:59:37.592538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.309782+0000","last_change":"2023-11-08T19:24:50.625916+0000","last_active":"2023-11-08T19:24:52.309782+0000","last_peered":"2023-11-08T19:24:52.309782+0000","last_clean":"2023-11-08T19:24:52.309782+0000","last_became_active":"2023-11-08T19:24:50.625625+0000","last_became_peered":"2023-11-08T19:24:50.625625+0000","last_unstale":"2023-11-08T19:24:52.309782+0000","last_undegraded":"2023-11-08T19:24:52.309782+0000","last_fullsized":"2023-11-08T19:24:52.309782+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:40:11.978983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.626555+0000","last_change":"2023-11-08T19:24:50.628686+0000","last_active":"2023-11-08T19:24:51.626555+0000","last_peered":"2023-11-08T19:24:51.626555+0000","last_clean":"2023-11-08T19:24:51.626555+0000","last_became_active":"2023-11-08T19:24:50.628289+0000","last_became_peered":"2023-11-08T19:24:50.628289+0000","last_unstale":"2023-11-08T19:24:51.626555+0000","last_undegraded":"2023-11-08T19:24:51.626555+0000","last_fullsized":"2023-11-08T19:24:51.626555+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:46:19.375897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.805231+0000","last_change":"2023-11-08T19:24:50.624352+0000","last_active":"2023-11-08T19:24:52.805231+0000","last_peered":"2023-11-08T19:24:52.805231+0000","last_clean":"2023-11-08T19:24:52.805231+0000","last_became_active":"2023-11-08T19:24:50.624035+0000","last_became_peered":"2023-11-08T19:24:50.624035+0000","last_unstale":"2023-11-08T19:24:52.805231+0000","last_undegraded":"2023-11-08T19:24:52.805231+0000","last_fullsized":"2023-11-08T19:24:52.805231+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:46:19.375897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.630792+0000","last_change":"2023-11-08T19:24:50.616410+0000","last_active":"2023-11-08T19:24:51.630792+0000","last_peered":"2023-11-08T19:24:51.630792+0000","last_clean":"2023-11-08T19:24:51.630792+0000","last_became_active":"2023-11-08T19:24:50.616118+0000","last_became_peered":"2023-11-08T19:24:50.616118+0000","last_unstale":"2023-11-08T19:24:51.630792+0000","last_undegraded":"2023-11-08T19:24:51.630792+0000","last_fullsized":"2023-11-08T19:24:51.630792+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:42:42.512786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.626991+0000","last_change":"2023-11-08T19:24:50.628009+0000","last_active":"2023-11-08T19:24:51.626991+0000","last_peered":"2023-11-08T19:24:51.626991+0000","last_clean":"2023-11-08T19:24:51.626991+0000","last_became_active":"2023-11-08T19:24:50.627668+0000","last_became_peered":"2023-11-08T19:24:50.627668+0000","last_unstale":"2023-11-08T19:24:51.626991+0000","last_undegraded":"2023-11-08T19:24:51.626991+0000","last_fullsized":"2023-11-08T19:24:51.626991+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:49:19.100370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.352100+0000","last_change":"2023-11-08T19:24:50.641255+0000","last_active":"2023-11-08T19:24:52.352100+0000","last_peered":"2023-11-08T19:24:52.352100+0000","last_clean":"2023-11-08T19:24:52.352100+0000","last_became_active":"2023-11-08T19:24:50.640164+0000","last_became_peered":"2023-11-08T19:24:50.640164+0000","last_unstale":"2023-11-08T19:24:52.352100+0000","last_undegraded":"2023-11-08T19:24:52.352100+0000","last_fullsized":"2023-11-08T19:24:52.352100+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:12:54.641952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.626435+0000","last_change":"2023-11-08T19:24:50.629006+0000","last_active":"2023-11-08T19:24:51.626435+0000","last_peered":"2023-11-08T19:24:51.626435+0000","last_clean":"2023-11-08T19:24:51.626435+0000","last_became_active":"2023-11-08T19:24:50.628685+0000","last_became_peered":"2023-11-08T19:24:50.628685+0000","last_unstale":"2023-11-08T19:24:51.626435+0000","last_undegraded":"2023-11-08T19:24:51.626435+0000","last_fullsized":"2023-11-08T19:24:51.626435+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:30:36.041820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.352270+0000","last_change":"2023-11-08T19:24:50.640737+0000","last_active":"2023-11-08T19:24:52.352270+0000","last_peered":"2023-11-08T19:24:52.352270+0000","last_clean":"2023-11-08T19:24:52.352270+0000","last_became_active":"2023-11-08T19:24:50.640255+0000","last_became_peered":"2023-11-08T19:24:50.640255+0000","last_unstale":"2023-11-08T19:24:52.352270+0000","last_undegraded":"2023-11-08T19:24:52.352270+0000","last_fullsized":"2023-11-08T19:24:52.352270+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:13:41.469149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:52.309753+0000","last_change":"2023-11-08T19:24:50.620645+0000","last_active":"2023-11-08T19:24:52.309753+0000","last_peered":"2023-11-08T19:24:52.309753+0000","last_clean":"2023-11-08T19:24:52.309753+0000","last_became_active":"2023-11-08T19:24:50.620349+0000","last_became_peered":"2023-11-08T19:24:50.620349+0000","last_unstale":"2023-11-08T19:24:52.309753+0000","last_undegraded":"2023-11-08T19:24:52.309753+0000","last_fullsized":"2023-11-08T19:24:52.309753+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:30:32.602976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T19:24:51.630586+0000","last_change":"2023-11-08T19:24:50.629751+0000","last_active":"2023-11-08T19:24:51.630586+0000","last_peered":"2023-11-08T19:24:51.630586+0000","last_clean":"2023-11-08T19:24:51.630586+0000","last_became_active":"2023-11-08T19:24:50.629453+0000","last_became_peered":"2023-11-08T19:24:50.629453+0000","last_unstale":"2023-11-08T19:24:51.630586+0000","last_undegraded":"2023-11-08T19:24:51.630586+0000","last_fullsized":"2023-11-08T19:24:51.630586+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:49.588741+0000","last_clean_scrub_stamp":"2023-11-08T19:24:49.588741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:46:19.375897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":802816,"data_stored":661860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67000000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54300000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68500000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70599999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60699999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.498}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45100000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55000000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41099999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46600000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34699999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":71070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29999999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32300000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38100000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52700000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55700000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":811008,"data_stored":661916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39600000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.432}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79300000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77900000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67200000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83699999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":204800,"data_stored":70910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76200000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80600000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83299999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574856,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38700000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40699999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34599999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67800000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56200000000000006}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53700000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":71036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61799999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.504}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68899999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67400000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39700000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48899999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":208896,"data_stored":71000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61299999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59899999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42099999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39800000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36899999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-08T19:25:04.372 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-11-08T19:25:04.372 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-11-08T19:25:04.373 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-11-08T19:25:04.373 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-08T19:25:04.525 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.524+0000 7fc9126be700 1 Processor -- start 2023-11-08T19:25:04.525 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.524+0000 7fc9126be700 1 -- start start 2023-11-08T19:25:04.528 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.525+0000 7fc9126be700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc90c15adb0 0x7fc90c0cb340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:04.528 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.525+0000 7fc9126be700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc90c0ced10 0x7fc90c0cf160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:04.529 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.525+0000 7fc9126be700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc90c0cf6a0 con 0x7fc90c15a9d0 2023-11-08T19:25:04.529 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.525+0000 7fc9126be700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc90c0ce8d0 con 0x7fc90c15adb0 2023-11-08T19:25:04.529 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.525+0000 7fc9126be700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc90c0cb940 con 0x7fc90c0ced10 2023-11-08T19:25:04.530 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.525+0000 7fc90b7fe700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc90c15adb0 0x7fc90c0cb340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:04.530 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.525+0000 7fc90bfff700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fc90c15a9d0 0x7fc90c0d10e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:45334/0 (socket says 172.21.15.143:45334) 2023-11-08T19:25:04.530 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.525+0000 7fc90bfff700 1 -- 172.21.15.143:0/3300211557 learned_addr learned my addr 172.21.15.143:0/3300211557 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:25:04.530 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.525+0000 7fc910c5b700 1 --2- 172.21.15.143:0/3300211557 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc90c0ced10 0x7fc90c0cf160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:04.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.525+0000 7fc90b7fe700 1 -- 172.21.15.143:0/3300211557 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc90c0ced10 msgr2=0x7fc90c0cf160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:04.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.525+0000 7fc90b7fe700 1 --2- 172.21.15.143:0/3300211557 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc90c0ced10 0x7fc90c0cf160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:04.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.525+0000 7fc90b7fe700 1 -- 172.21.15.143:0/3300211557 >> v1:172.21.15.143:6789/0 conn(0x7fc90c15a9d0 legacy=0x7fc90c0d10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:04.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.525+0000 7fc90b7fe700 1 -- 172.21.15.143:0/3300211557 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc90c0cbc20 con 0x7fc90c15adb0 2023-11-08T19:25:04.532 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.525+0000 7fc90affd700 1 -- 172.21.15.143:0/3300211557 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1789014504 0 0) 0x7fc8fc001290 con 0x7fc90c15a9d0 2023-11-08T19:25:04.532 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.525+0000 7fc90b7fe700 1 --2- 172.21.15.143:0/3300211557 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc90c15adb0 0x7fc90c0cb340 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc900009840 tx=0x7fc900009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4e3e633c8b0a24e server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:04.532 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.525+0000 7fc90affd700 1 -- 172.21.15.143:0/3300211557 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc900011830 con 0x7fc90c15adb0 2023-11-08T19:25:04.532 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.525+0000 7fc90affd700 1 -- 172.21.15.143:0/3300211557 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc900011990 con 0x7fc90c15adb0 2023-11-08T19:25:04.532 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.525+0000 7fc90affd700 1 -- 172.21.15.143:0/3300211557 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc900011d20 con 0x7fc90c15adb0 2023-11-08T19:25:04.533 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.526+0000 7fc9126be700 1 -- 172.21.15.143:0/3300211557 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc90c15adb0 msgr2=0x7fc90c0cb340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:04.533 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.526+0000 7fc9126be700 1 --2- 172.21.15.143:0/3300211557 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc90c15adb0 0x7fc90c0cb340 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc900009840 tx=0x7fc900009b50 comp rx=0 tx=0).stop 2023-11-08T19:25:04.533 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.526+0000 7fc9126be700 1 -- 172.21.15.143:0/3300211557 shutdown_connections 2023-11-08T19:25:04.534 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.526+0000 7fc9126be700 1 --2- 172.21.15.143:0/3300211557 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc90c0ced10 0x7fc90c0cf160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:04.534 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.526+0000 7fc9126be700 1 --2- 172.21.15.143:0/3300211557 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc90c15adb0 0x7fc90c0cb340 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:04.534 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.526+0000 7fc9126be700 1 -- 172.21.15.143:0/3300211557 >> 172.21.15.143:0/3300211557 conn(0x7fc90c099a70 msgr2=0x7fc90c0d07c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:04.534 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.526+0000 7fc9126be700 1 -- 172.21.15.143:0/3300211557 shutdown_connections 2023-11-08T19:25:04.535 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.526+0000 7fc9126be700 1 -- 172.21.15.143:0/3300211557 wait complete. 2023-11-08T19:25:04.535 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.527+0000 7fc9126be700 1 Processor -- start 2023-11-08T19:25:04.538 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.527+0000 7fc9126be700 1 -- start start 2023-11-08T19:25:04.538 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.528+0000 7fc9126be700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc90c0ced10 0x7fc90c09a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:04.538 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.528+0000 7fc9126be700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc90c15a9d0 0x7fc90c0c8b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:04.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.528+0000 7fc9126be700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc90c15adb0 0x7fc90c1595c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:04.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.528+0000 7fc9126be700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc90c159b00 con 0x7fc90c15adb0 2023-11-08T19:25:04.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.528+0000 7fc9126be700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc90c09aa30 con 0x7fc90c15a9d0 2023-11-08T19:25:04.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.528+0000 7fc9126be700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc90c09ad10 con 0x7fc90c0ced10 2023-11-08T19:25:04.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.528+0000 7fc910c5b700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc90c15adb0 0x7fc90c1595c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:04.540 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.528+0000 7fc910c5b700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc90c15adb0 0x7fc90c1595c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49190/0 (socket says 172.21.15.143:49190) 2023-11-08T19:25:04.540 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.528+0000 7fc910c5b700 1 -- 172.21.15.143:0/558952960 learned_addr learned my addr 172.21.15.143:0/558952960 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:04.540 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.528+0000 7fc90b7fe700 1 --2- 172.21.15.143:0/558952960 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc90c15a9d0 0x7fc90c0c8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:04.540 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.528+0000 7fc90bfff700 1 --2- 172.21.15.143:0/558952960 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc90c0ced10 0x7fc90c09a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:04.540 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.528+0000 7fc910c5b700 1 -- 172.21.15.143:0/558952960 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc90c0ced10 msgr2=0x7fc90c09a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:04.541 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.528+0000 7fc910c5b700 1 --2- 172.21.15.143:0/558952960 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc90c0ced10 0x7fc90c09a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:04.541 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.528+0000 7fc910c5b700 1 -- 172.21.15.143:0/558952960 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc90c15a9d0 msgr2=0x7fc90c0c8b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:04.541 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.528+0000 7fc910c5b700 1 --2- 172.21.15.143:0/558952960 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc90c15a9d0 0x7fc90c0c8b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:04.541 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.528+0000 7fc910c5b700 1 -- 172.21.15.143:0/558952960 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9000094a0 con 0x7fc90c15adb0 2023-11-08T19:25:04.541 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.529+0000 7fc910c5b700 1 --2- 172.21.15.143:0/558952960 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc90c15adb0 0x7fc90c1595c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc90400cd70 tx=0x7fc90400a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:04.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.529+0000 7fc908ff9700 1 -- 172.21.15.143:0/558952960 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc904003a20 con 0x7fc90c15adb0 2023-11-08T19:25:04.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.529+0000 7fc908ff9700 1 -- 172.21.15.143:0/558952960 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc904003b80 con 0x7fc90c15adb0 2023-11-08T19:25:04.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.529+0000 7fc9126be700 1 -- 172.21.15.143:0/558952960 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc90c09aff0 con 0x7fc90c15adb0 2023-11-08T19:25:04.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.529+0000 7fc908ff9700 1 -- 172.21.15.143:0/558952960 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc904004020 con 0x7fc90c15adb0 2023-11-08T19:25:04.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.529+0000 7fc9126be700 1 -- 172.21.15.143:0/558952960 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc90c09b5b0 con 0x7fc90c15adb0 2023-11-08T19:25:04.543 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.530+0000 7fc9126be700 1 -- 172.21.15.143:0/558952960 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc90c09b7a0 con 0x7fc90c15adb0 2023-11-08T19:25:04.543 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.530+0000 7fc908ff9700 1 -- 172.21.15.143:0/558952960 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc904004180 con 0x7fc90c15adb0 2023-11-08T19:25:04.543 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.530+0000 7fc908ff9700 1 --2- 172.21.15.143:0/558952960 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc8f806af00 0x7fc8f806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:04.543 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.530+0000 7fc908ff9700 1 -- 172.21.15.143:0/558952960 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc904012070 con 0x7fc90c15adb0 2023-11-08T19:25:04.543 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.531+0000 7fc90bfff700 1 --2- 172.21.15.143:0/558952960 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc8f806af00 0x7fc8f806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:04.544 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.531+0000 7fc90bfff700 1 --2- 172.21.15.143:0/558952960 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc8f806af00 0x7fc8f806d3c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc0014f0 tx=0x7fc8fc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:04.544 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.535+0000 7fc908ff9700 1 -- 172.21.15.143:0/558952960 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc904054e50 con 0x7fc90c15adb0 2023-11-08T19:25:04.768 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.766+0000 7fc9126be700 1 -- 172.21.15.143:0/558952960 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fc90c04eee0 con 0x7fc90c15adb0 2023-11-08T19:25:04.768 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.767+0000 7fc908ff9700 1 -- 172.21.15.143:0/558952960 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fc904058470 con 0x7fc90c15adb0 2023-11-08T19:25:04.768 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:25:04.769 INFO:teuthology.orchestra.run.smithi143.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-11-08T19:25:04.770 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.769+0000 7fc9126be700 1 -- 172.21.15.143:0/558952960 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc8f806af00 msgr2=0x7fc8f806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:04.771 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.769+0000 7fc9126be700 1 --2- 172.21.15.143:0/558952960 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc8f806af00 0x7fc8f806d3c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc0014f0 tx=0x7fc8fc009040 comp rx=0 tx=0).stop 2023-11-08T19:25:04.771 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.769+0000 7fc9126be700 1 -- 172.21.15.143:0/558952960 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc90c15adb0 msgr2=0x7fc90c1595c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:04.771 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.769+0000 7fc9126be700 1 --2- 172.21.15.143:0/558952960 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc90c15adb0 0x7fc90c1595c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc90400cd70 tx=0x7fc90400a570 comp rx=0 tx=0).stop 2023-11-08T19:25:04.771 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.769+0000 7fc9126be700 1 -- 172.21.15.143:0/558952960 shutdown_connections 2023-11-08T19:25:04.771 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.769+0000 7fc9126be700 1 --2- 172.21.15.143:0/558952960 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc8f806af00 0x7fc8f806d3c0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:04.772 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.769+0000 7fc9126be700 1 --2- 172.21.15.143:0/558952960 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc90c0ced10 0x7fc90c09a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:04.772 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.769+0000 7fc9126be700 1 --2- 172.21.15.143:0/558952960 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc90c15adb0 0x7fc90c1595c0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:04.772 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.769+0000 7fc9126be700 1 --2- 172.21.15.143:0/558952960 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc90c15a9d0 0x7fc90c0c8b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:04.772 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.769+0000 7fc9126be700 1 -- 172.21.15.143:0/558952960 >> 172.21.15.143:0/558952960 conn(0x7fc90c099a70 msgr2=0x7fc90c06ede0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:04.772 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.769+0000 7fc9126be700 1 -- 172.21.15.143:0/558952960 shutdown_connections 2023-11-08T19:25:04.772 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.769+0000 7fc9126be700 1 -- 172.21.15.143:0/558952960 wait complete. 2023-11-08T19:25:04.782 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-11-08T19:25:04.783 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:25:04.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.902+0000 7fc8c6dc3700 1 Processor -- start 2023-11-08T19:25:04.904 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.902+0000 7fc8c6dc3700 1 -- start start 2023-11-08T19:25:04.906 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.903+0000 7fc8c6dc3700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fc8c00b2920 0x7fc8c00cf510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:04.907 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.903+0000 7fc8c6dc3700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc8c00cbe20 0x7fc8c00cc290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:04.907 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.903+0000 7fc8c6dc3700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8c00cf080 con 0x7fc8c015aa30 2023-11-08T19:25:04.907 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.903+0000 7fc8c6dc3700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8c00cc7d0 con 0x7fc8c00b2920 2023-11-08T19:25:04.907 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.903+0000 7fc8c6dc3700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8c00ce470 con 0x7fc8c00cbe20 2023-11-08T19:25:04.907 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.903+0000 7fc8c5360700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc8c00cbe20 0x7fc8c00cc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:04.908 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.903+0000 7fc8bffff700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fc8c00b2920 0x7fc8c00cf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:04.908 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.903+0000 7fc8c4b5f700 1 --1- >> v1:172.21.15.174:6789/0 conn(0x7fc8c015aa30 0x7fc8c00b2210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.174:6789/0 says I am v1:172.21.15.143:56282/0 (socket says 172.21.15.143:56282) 2023-11-08T19:25:04.908 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.903+0000 7fc8c4b5f700 1 -- 172.21.15.143:0/3430184916 learned_addr learned my addr 172.21.15.143:0/3430184916 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:25:04.908 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.904+0000 7fc8bf7fe700 1 -- 172.21.15.143:0/3430184916 <== mon.1 v1:172.21.15.174:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4166061500 0 0) 0x7fc8c00cf080 con 0x7fc8c015aa30 2023-11-08T19:25:04.908 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.904+0000 7fc8bf7fe700 1 -- 172.21.15.143:0/3430184916 --> v1:172.21.15.174:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc8b00035b0 con 0x7fc8c015aa30 2023-11-08T19:25:04.909 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.904+0000 7fc8c5360700 1 -- 172.21.15.143:0/3430184916 >> v2:172.21.15.174:3300/0 conn(0x7fc8c00b2920 msgr2=0x7fc8c00cf510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:04.909 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.904+0000 7fc8c5360700 1 --2- 172.21.15.143:0/3430184916 >> v2:172.21.15.174:3300/0 conn(0x7fc8c00b2920 0x7fc8c00cf510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:04.909 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.904+0000 7fc8c5360700 1 -- 172.21.15.143:0/3430184916 >> v1:172.21.15.174:6789/0 conn(0x7fc8c015aa30 legacy=0x7fc8c00b2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:04.909 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.904+0000 7fc8c5360700 1 -- 172.21.15.143:0/3430184916 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8c00ce750 con 0x7fc8c00cbe20 2023-11-08T19:25:04.909 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.904+0000 7fc8bffff700 1 --2- 172.21.15.143:0/3430184916 >> v2:172.21.15.174:3300/0 conn(0x7fc8c00b2920 0x7fc8c00cf510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T19:25:04.910 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.904+0000 7fc8c5360700 1 --2- 172.21.15.143:0/3430184916 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc8c00cbe20 0x7fc8c00cc290 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc8b8009d90 tx=0x7fc8b800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8fd8ee6e16b994e4 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:04.910 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.904+0000 7fc8bf7fe700 1 -- 172.21.15.143:0/3430184916 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b800e070 con 0x7fc8c00cbe20 2023-11-08T19:25:04.910 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.904+0000 7fc8bf7fe700 1 -- 172.21.15.143:0/3430184916 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8b800bb20 con 0x7fc8c00cbe20 2023-11-08T19:25:04.910 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.904+0000 7fc8bf7fe700 1 -- 172.21.15.143:0/3430184916 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b800be90 con 0x7fc8c00cbe20 2023-11-08T19:25:04.910 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.905+0000 7fc8c6dc3700 1 -- 172.21.15.143:0/3430184916 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc8c00cbe20 msgr2=0x7fc8c00cc290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:04.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.905+0000 7fc8c6dc3700 1 --2- 172.21.15.143:0/3430184916 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc8c00cbe20 0x7fc8c00cc290 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc8b8009d90 tx=0x7fc8b800b330 comp rx=0 tx=0).stop 2023-11-08T19:25:04.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.905+0000 7fc8c6dc3700 1 -- 172.21.15.143:0/3430184916 shutdown_connections 2023-11-08T19:25:04.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.905+0000 7fc8c6dc3700 1 --2- 172.21.15.143:0/3430184916 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc8c00cbe20 0x7fc8c00cc290 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:04.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.905+0000 7fc8c6dc3700 1 --2- 172.21.15.143:0/3430184916 >> v2:172.21.15.174:3300/0 conn(0x7fc8c00b2920 0x7fc8c00cf510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:04.912 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.905+0000 7fc8c6dc3700 1 -- 172.21.15.143:0/3430184916 >> 172.21.15.143:0/3430184916 conn(0x7fc8c0099ab0 msgr2=0x7fc8c00d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:04.912 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.905+0000 7fc8c6dc3700 1 -- 172.21.15.143:0/3430184916 shutdown_connections 2023-11-08T19:25:04.912 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.905+0000 7fc8c6dc3700 1 -- 172.21.15.143:0/3430184916 wait complete. 2023-11-08T19:25:04.915 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.906+0000 7fc8c6dc3700 1 Processor -- start 2023-11-08T19:25:04.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.906+0000 7fc8c6dc3700 1 -- start start 2023-11-08T19:25:04.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.907+0000 7fc8c6dc3700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc8c00b2920 0x7fc8c0158260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:04.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.907+0000 7fc8c6dc3700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc8c00cbe20 0x7fc8c015c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:04.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.907+0000 7fc8c6dc3700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc8c015aa30 0x7fc8c00c94e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:04.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.907+0000 7fc8c6dc3700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8c01587a0 con 0x7fc8c00b2920 2023-11-08T19:25:04.917 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.907+0000 7fc8c6dc3700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8c00c7c30 con 0x7fc8c015aa30 2023-11-08T19:25:04.917 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.907+0000 7fc8c6dc3700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8c00c7900 con 0x7fc8c00cbe20 2023-11-08T19:25:04.917 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.907+0000 7fc8c4b5f700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc8c00b2920 0x7fc8c0158260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:04.917 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.907+0000 7fc8c4b5f700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc8c00b2920 0x7fc8c0158260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49200/0 (socket says 172.21.15.143:49200) 2023-11-08T19:25:04.918 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.907+0000 7fc8c4b5f700 1 -- 172.21.15.143:0/3479571413 learned_addr learned my addr 172.21.15.143:0/3479571413 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:04.918 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.907+0000 7fc8bffff700 1 --2- 172.21.15.143:0/3479571413 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc8c00cbe20 0x7fc8c015c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:04.918 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.907+0000 7fc8c5360700 1 --2- 172.21.15.143:0/3479571413 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc8c015aa30 0x7fc8c00c94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:04.918 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.907+0000 7fc8c4b5f700 1 -- 172.21.15.143:0/3479571413 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc8c00cbe20 msgr2=0x7fc8c015c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:04.918 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.907+0000 7fc8c4b5f700 1 --2- 172.21.15.143:0/3479571413 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc8c00cbe20 0x7fc8c015c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:04.919 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.907+0000 7fc8c4b5f700 1 -- 172.21.15.143:0/3479571413 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc8c015aa30 msgr2=0x7fc8c00c94e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:04.919 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.907+0000 7fc8c4b5f700 1 --2- 172.21.15.143:0/3479571413 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc8c015aa30 0x7fc8c00c94e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:04.919 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.907+0000 7fc8c4b5f700 1 -- 172.21.15.143:0/3479571413 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8ac0094a0 con 0x7fc8c00b2920 2023-11-08T19:25:04.919 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.908+0000 7fc8c4b5f700 1 --2- 172.21.15.143:0/3479571413 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc8c00b2920 0x7fc8c0158260 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc8b400c4d0 tx=0x7fc8b400c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:04.920 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.908+0000 7fc8bd7fa700 1 -- 172.21.15.143:0/3479571413 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b4005dc0 con 0x7fc8c00b2920 2023-11-08T19:25:04.920 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.908+0000 7fc8bd7fa700 1 -- 172.21.15.143:0/3479571413 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8b400ce80 con 0x7fc8c00b2920 2023-11-08T19:25:04.920 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.908+0000 7fc8c6dc3700 1 -- 172.21.15.143:0/3479571413 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8b8009a70 con 0x7fc8c00b2920 2023-11-08T19:25:04.920 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.908+0000 7fc8bd7fa700 1 -- 172.21.15.143:0/3479571413 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b40072d0 con 0x7fc8c00b2920 2023-11-08T19:25:04.921 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.908+0000 7fc8c6dc3700 1 -- 172.21.15.143:0/3479571413 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8c00ca210 con 0x7fc8c00b2920 2023-11-08T19:25:04.921 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.909+0000 7fc8c6dc3700 1 -- 172.21.15.143:0/3479571413 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8c004eee0 con 0x7fc8c00b2920 2023-11-08T19:25:04.921 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.909+0000 7fc8bd7fa700 1 -- 172.21.15.143:0/3479571413 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc8b4007430 con 0x7fc8c00b2920 2023-11-08T19:25:04.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.909+0000 7fc8bd7fa700 1 --2- 172.21.15.143:0/3479571413 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc8a406af00 0x7fc8a406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:04.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.909+0000 7fc8bd7fa700 1 -- 172.21.15.143:0/3479571413 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc8b4088780 con 0x7fc8c00b2920 2023-11-08T19:25:04.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.910+0000 7fc8bffff700 1 --2- 172.21.15.143:0/3479571413 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc8a406af00 0x7fc8a406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:04.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.910+0000 7fc8bffff700 1 --2- 172.21.15.143:0/3479571413 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc8a406af00 0x7fc8a406d3c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc8c00aec90 tx=0x7fc8ac009210 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:04.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:04.914+0000 7fc8bd7fa700 1 -- 172.21.15.143:0/3479571413 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc8b4053970 con 0x7fc8c00b2920 2023-11-08T19:25:05.105 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.104+0000 7fc8c6dc3700 1 -- 172.21.15.143:0/3479571413 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc8c00b2ff0 con 0x7fc8c00b2920 2023-11-08T19:25:05.106 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.105+0000 7fc8bd7fa700 1 -- 172.21.15.143:0/3479571413 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16553 (secure 0 0 0) 0x7fc8b4056f90 con 0x7fc8c00b2920 2023-11-08T19:25:05.107 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:25:05.108 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":20,"fsid":"968483af-77fc-4068-ad7c-5c2a1a4613d7","created":"2023-11-08T19:24:32.826484+0000","modified":"2023-11-08T19:24:51.618072+0000","last_up_change":"2023-11-08T19:24:43.573880+0000","last_in_change":"2023-11-08T19:24:35.314525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:44.551530+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-08T19:24:48.256781+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-08T19:24:48.954990+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bae98ec1-d94e-403d-a220-a522a53ea6f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6809","nonce":1487830566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6811","nonce":1487830566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6815","nonce":1487830566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6813","nonce":1487830566}]},"public_addr":"172.21.15.143:6809/1487830566","cluster_addr":"172.21.15.143:6811/1487830566","heartbeat_back_addr":"172.21.15.143:6815/1487830566","heartbeat_front_addr":"172.21.15.143:6813/1487830566","state":["exists","up"]},{"osd":1,"uuid":"5cb6538a-1ccb-4322-8633-09f66af0b7e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6801","nonce":787341397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6803","nonce":787341397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6807","nonce":787341397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6805","nonce":787341397}]},"public_addr":"172.21.15.143:6801/787341397","cluster_addr":"172.21.15.143:6803/787341397","heartbeat_back_addr":"172.21.15.143:6807/787341397","heartbeat_front_addr":"172.21.15.143:6805/787341397","state":["exists","up"]},{"osd":2,"uuid":"cdbf522c-60ef-4d60-9653-8dcb529c101a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6822","nonce":2301325597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6826","nonce":2301325597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6831","nonce":2301325597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6829","nonce":2301325597}]},"public_addr":"172.21.15.143:6822/2301325597","cluster_addr":"172.21.15.143:6826/2301325597","heartbeat_back_addr":"172.21.15.143:6831/2301325597","heartbeat_front_addr":"172.21.15.143:6829/2301325597","state":["exists","up"]},{"osd":3,"uuid":"4e2783cf-ebb9-4dd8-8245-bc1d17ce4325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6817","nonce":2021504428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6819","nonce":2021504428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6827","nonce":2021504428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6823","nonce":2021504428}]},"public_addr":"172.21.15.143:6817/2021504428","cluster_addr":"172.21.15.143:6819/2021504428","heartbeat_back_addr":"172.21.15.143:6827/2021504428","heartbeat_front_addr":"172.21.15.143:6823/2021504428","state":["exists","up"]},{"osd":4,"uuid":"691256fa-9b77-49b9-a346-7ae1adf0ca80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6801","nonce":1993380595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6803","nonce":1993380595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6807","nonce":1993380595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6805","nonce":1993380595}]},"public_addr":"172.21.15.174:6801/1993380595","cluster_addr":"172.21.15.174:6803/1993380595","heartbeat_back_addr":"172.21.15.174:6807/1993380595","heartbeat_front_addr":"172.21.15.174:6805/1993380595","state":["exists","up"]},{"osd":5,"uuid":"756fc352-48c9-46ce-b52e-0541fe6f8525","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6809","nonce":1777923303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6811","nonce":1777923303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6815","nonce":1777923303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6813","nonce":1777923303}]},"public_addr":"172.21.15.174:6809/1777923303","cluster_addr":"172.21.15.174:6811/1777923303","heartbeat_back_addr":"172.21.15.174:6815/1777923303","heartbeat_front_addr":"172.21.15.174:6813/1777923303","state":["exists","up"]},{"osd":6,"uuid":"fab2ac2b-ed97-4cde-9825-b480eabf8b8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6825","nonce":2755130316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6827","nonce":2755130316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6831","nonce":2755130316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6829","nonce":2755130316}]},"public_addr":"172.21.15.174:6825/2755130316","cluster_addr":"172.21.15.174:6827/2755130316","heartbeat_back_addr":"172.21.15.174:6831/2755130316","heartbeat_front_addr":"172.21.15.174:6829/2755130316","state":["exists","up"]},{"osd":7,"uuid":"3de9b797-2ab6-4d41-a863-7e7d638fd703","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6817","nonce":2847141411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6819","nonce":2847141411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6823","nonce":2847141411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6821","nonce":2847141411}]},"public_addr":"172.21.15.174:6817/2847141411","cluster_addr":"172.21.15.174:6819/2847141411","heartbeat_back_addr":"172.21.15.174:6823/2847141411","heartbeat_front_addr":"172.21.15.174:6821/2847141411","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.477076+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.496890+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.539305+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.454673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.538881+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.521294+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.520712+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.540404+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:25:05.109 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.107+0000 7fc8c6dc3700 1 -- 172.21.15.143:0/3479571413 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc8a406af00 msgr2=0x7fc8a406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:05.109 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.107+0000 7fc8c6dc3700 1 --2- 172.21.15.143:0/3479571413 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc8a406af00 0x7fc8a406d3c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc8c00aec90 tx=0x7fc8ac009210 comp rx=0 tx=0).stop 2023-11-08T19:25:05.109 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.107+0000 7fc8c6dc3700 1 -- 172.21.15.143:0/3479571413 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc8c00b2920 msgr2=0x7fc8c0158260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:05.110 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.107+0000 7fc8c6dc3700 1 --2- 172.21.15.143:0/3479571413 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc8c00b2920 0x7fc8c0158260 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc8b400c4d0 tx=0x7fc8b400c7e0 comp rx=0 tx=0).stop 2023-11-08T19:25:05.110 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.107+0000 7fc8c6dc3700 1 -- 172.21.15.143:0/3479571413 shutdown_connections 2023-11-08T19:25:05.110 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.107+0000 7fc8c6dc3700 1 --2- 172.21.15.143:0/3479571413 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc8a406af00 0x7fc8a406d3c0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:05.110 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.107+0000 7fc8c6dc3700 1 --2- 172.21.15.143:0/3479571413 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc8c015aa30 0x7fc8c00c94e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:05.110 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.107+0000 7fc8c6dc3700 1 --2- 172.21.15.143:0/3479571413 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc8c00b2920 0x7fc8c0158260 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:05.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.107+0000 7fc8c6dc3700 1 --2- 172.21.15.143:0/3479571413 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc8c00cbe20 0x7fc8c015c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:05.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.107+0000 7fc8c6dc3700 1 -- 172.21.15.143:0/3479571413 >> 172.21.15.143:0/3479571413 conn(0x7fc8c0099ab0 msgr2=0x7fc8c0158a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:05.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.107+0000 7fc8c6dc3700 1 -- 172.21.15.143:0/3479571413 shutdown_connections 2023-11-08T19:25:05.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.108+0000 7fc8c6dc3700 1 -- 172.21.15.143:0/3479571413 wait complete. 2023-11-08T19:25:05.121 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T19:24:44.551530+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-08T19:24:48.256781+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-08T19:24:48.954990+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-08T19:25:05.122 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T19:25:05.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.244+0000 7ff40bba2700 1 Processor -- start 2023-11-08T19:25:05.246 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.244+0000 7ff40bba2700 1 -- start start 2023-11-08T19:25:05.252 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.245+0000 7ff40bba2700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff40406bd20 0x7ff4040d01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:05.252 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.245+0000 7ff40bba2700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7ff4040b61e0 0x7ff4040cc730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:05.253 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.245+0000 7ff40bba2700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4040cc2a0 con 0x7ff40415a950 2023-11-08T19:25:05.253 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.245+0000 7ff40bba2700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4040cc130 con 0x7ff40406bd20 2023-11-08T19:25:05.253 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.245+0000 7ff40bba2700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff40406c1b0 con 0x7ff4040b61e0 2023-11-08T19:25:05.253 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.245+0000 7ff40913d700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff40406bd20 0x7ff4040d01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:05.254 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.245+0000 7ff40913d700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff40406bd20 0x7ff4040d01b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49210/0 (socket says 172.21.15.143:49210) 2023-11-08T19:25:05.254 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.245+0000 7ff40913d700 1 -- 172.21.15.143:0/1557770356 learned_addr learned my addr 172.21.15.143:0/1557770356 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:05.254 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.245+0000 7ff40a13f700 1 --2- 172.21.15.143:0/1557770356 >> v2:172.21.15.174:3300/0 conn(0x7ff4040b61e0 0x7ff4040cc730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:05.254 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.246+0000 7ff40913d700 1 -- 172.21.15.143:0/1557770356 >> v2:172.21.15.174:3300/0 conn(0x7ff4040b61e0 msgr2=0x7ff4040cc730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:05.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.246+0000 7ff40913d700 1 --2- 172.21.15.143:0/1557770356 >> v2:172.21.15.174:3300/0 conn(0x7ff4040b61e0 0x7ff4040cc730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:05.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.246+0000 7ff40913d700 1 -- 172.21.15.143:0/1557770356 >> v1:172.21.15.174:6789/0 conn(0x7ff40415a950 legacy=0x7ff404072370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:05.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.246+0000 7ff40913d700 1 -- 172.21.15.143:0/1557770356 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff40406c490 con 0x7ff40406bd20 2023-11-08T19:25:05.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.246+0000 7ff40893c700 1 -- 172.21.15.143:0/1557770356 <== mon.1 v1:172.21.15.174:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3237932846 0 0) 0x7ff4040cc2a0 con 0x7ff40415a950 2023-11-08T19:25:05.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.246+0000 7ff40913d700 1 --2- 172.21.15.143:0/1557770356 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff40406bd20 0x7ff4040d01b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff4040c7890 tx=0x7ff400009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11ecaa8b49af02dd server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:05.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.246+0000 7ff40893c700 1 -- 172.21.15.143:0/1557770356 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff400011830 con 0x7ff40406bd20 2023-11-08T19:25:05.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.246+0000 7ff40893c700 1 -- 172.21.15.143:0/1557770356 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff400011990 con 0x7ff40406bd20 2023-11-08T19:25:05.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.246+0000 7ff40893c700 1 -- 172.21.15.143:0/1557770356 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff400011d20 con 0x7ff40406bd20 2023-11-08T19:25:05.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.246+0000 7ff40bba2700 1 -- 172.21.15.143:0/1557770356 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff40406bd20 msgr2=0x7ff4040d01b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:05.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.246+0000 7ff40bba2700 1 --2- 172.21.15.143:0/1557770356 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff40406bd20 0x7ff4040d01b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff4040c7890 tx=0x7ff400009bd0 comp rx=0 tx=0).stop 2023-11-08T19:25:05.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.246+0000 7ff40bba2700 1 -- 172.21.15.143:0/1557770356 shutdown_connections 2023-11-08T19:25:05.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.246+0000 7ff40bba2700 1 --2- 172.21.15.143:0/1557770356 >> v2:172.21.15.174:3300/0 conn(0x7ff4040b61e0 0x7ff4040cc730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:05.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.246+0000 7ff40bba2700 1 --2- 172.21.15.143:0/1557770356 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff40406bd20 0x7ff4040d01b0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:05.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.246+0000 7ff40bba2700 1 -- 172.21.15.143:0/1557770356 >> 172.21.15.143:0/1557770356 conn(0x7ff4040cd2e0 msgr2=0x7ff4040cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:05.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.247+0000 7ff40bba2700 1 -- 172.21.15.143:0/1557770356 shutdown_connections 2023-11-08T19:25:05.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.247+0000 7ff40bba2700 1 -- 172.21.15.143:0/1557770356 wait complete. 2023-11-08T19:25:05.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.248+0000 7ff40bba2700 1 Processor -- start 2023-11-08T19:25:05.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.248+0000 7ff40bba2700 1 -- start start 2023-11-08T19:25:05.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.248+0000 7ff40bba2700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff4040b61e0 0x7ff40414fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:05.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.248+0000 7ff40bba2700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff40415a950 0x7ff4040c6340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:05.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.248+0000 7ff40bba2700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff4040c6880 0x7ff4040b0560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:05.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.248+0000 7ff40bba2700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4040c6d30 con 0x7ff40415a950 2023-11-08T19:25:05.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.248+0000 7ff40bba2700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4040b2060 con 0x7ff4040c6880 2023-11-08T19:25:05.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.248+0000 7ff40bba2700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4040b1290 con 0x7ff4040b61e0 2023-11-08T19:25:05.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.248+0000 7ff40913d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff40415a950 0x7ff4040c6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:05.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.248+0000 7ff40913d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff40415a950 0x7ff4040c6340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49216/0 (socket says 172.21.15.143:49216) 2023-11-08T19:25:05.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.248+0000 7ff40913d700 1 -- 172.21.15.143:0/2518519653 learned_addr learned my addr 172.21.15.143:0/2518519653 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:05.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.249+0000 7ff40a13f700 1 --2- 172.21.15.143:0/2518519653 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff4040c6880 0x7ff4040b0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:05.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.249+0000 7ff40993e700 1 --2- 172.21.15.143:0/2518519653 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff4040b61e0 0x7ff40414fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:05.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.249+0000 7ff40913d700 1 -- 172.21.15.143:0/2518519653 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff4040b61e0 msgr2=0x7ff40414fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:05.261 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.249+0000 7ff40913d700 1 --2- 172.21.15.143:0/2518519653 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff4040b61e0 0x7ff40414fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:05.261 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.249+0000 7ff40913d700 1 -- 172.21.15.143:0/2518519653 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff4040c6880 msgr2=0x7ff4040b0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:05.261 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.249+0000 7ff40913d700 1 --2- 172.21.15.143:0/2518519653 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff4040c6880 0x7ff4040b0560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:05.261 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.249+0000 7ff40913d700 1 -- 172.21.15.143:0/2518519653 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff400009550 con 0x7ff40415a950 2023-11-08T19:25:05.261 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.249+0000 7ff40913d700 1 --2- 172.21.15.143:0/2518519653 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff40415a950 0x7ff4040c6340 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff400009c10 tx=0x7ff400004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:05.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.249+0000 7ff3fa7fc700 1 -- 172.21.15.143:0/2518519653 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4000036a0 con 0x7ff40415a950 2023-11-08T19:25:05.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.249+0000 7ff3fa7fc700 1 -- 172.21.15.143:0/2518519653 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff400003800 con 0x7ff40415a950 2023-11-08T19:25:05.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.249+0000 7ff40bba2700 1 -- 172.21.15.143:0/2518519653 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4040b22b0 con 0x7ff40415a950 2023-11-08T19:25:05.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.249+0000 7ff3fa7fc700 1 -- 172.21.15.143:0/2518519653 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff400003b70 con 0x7ff40415a950 2023-11-08T19:25:05.265 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.249+0000 7ff40bba2700 1 -- 172.21.15.143:0/2518519653 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4040b2870 con 0x7ff40415a950 2023-11-08T19:25:05.265 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.250+0000 7ff40bba2700 1 -- 172.21.15.143:0/2518519653 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff40404eee0 con 0x7ff40415a950 2023-11-08T19:25:05.266 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.250+0000 7ff3fa7fc700 1 -- 172.21.15.143:0/2518519653 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff400003cd0 con 0x7ff40415a950 2023-11-08T19:25:05.266 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.251+0000 7ff3fa7fc700 1 --2- 172.21.15.143:0/2518519653 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7ff3e806af00 0x7ff3e806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:05.266 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.251+0000 7ff3fa7fc700 1 -- 172.21.15.143:0/2518519653 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff400021030 con 0x7ff40415a950 2023-11-08T19:25:05.266 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.251+0000 7ff40993e700 1 --2- 172.21.15.143:0/2518519653 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7ff3e806af00 0x7ff3e806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:05.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.251+0000 7ff40993e700 1 --2- 172.21.15.143:0/2518519653 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7ff3e806af00 0x7ff3e806d3c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff3f4000f60 tx=0x7ff3f4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:05.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.255+0000 7ff3fa7fc700 1 -- 172.21.15.143:0/2518519653 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff400055e30 con 0x7ff40415a950 2023-11-08T19:25:05.446 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.444+0000 7ff40bba2700 1 -- 172.21.15.143:0/2518519653 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff4040cf590 con 0x7ff40415a950 2023-11-08T19:25:05.446 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.445+0000 7ff3fa7fc700 1 -- 172.21.15.143:0/2518519653 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff40002b020 con 0x7ff40415a950 2023-11-08T19:25:05.447 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2023-11-08T19:25:05.448 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.447+0000 7ff40bba2700 1 -- 172.21.15.143:0/2518519653 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7ff3e806af00 msgr2=0x7ff3e806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:05.449 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.447+0000 7ff40bba2700 1 --2- 172.21.15.143:0/2518519653 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7ff3e806af00 0x7ff3e806d3c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff3f4000f60 tx=0x7ff3f4009040 comp rx=0 tx=0).stop 2023-11-08T19:25:05.449 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.447+0000 7ff40bba2700 1 -- 172.21.15.143:0/2518519653 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff40415a950 msgr2=0x7ff4040c6340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:05.449 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.447+0000 7ff40bba2700 1 --2- 172.21.15.143:0/2518519653 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff40415a950 0x7ff4040c6340 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff400009c10 tx=0x7ff400004750 comp rx=0 tx=0).stop 2023-11-08T19:25:05.449 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.447+0000 7ff40bba2700 1 -- 172.21.15.143:0/2518519653 shutdown_connections 2023-11-08T19:25:05.449 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.447+0000 7ff40bba2700 1 --2- 172.21.15.143:0/2518519653 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7ff3e806af00 0x7ff3e806d3c0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:05.450 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.447+0000 7ff40bba2700 1 --2- 172.21.15.143:0/2518519653 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff4040b61e0 0x7ff40414fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:05.450 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.447+0000 7ff40bba2700 1 --2- 172.21.15.143:0/2518519653 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff4040c6880 0x7ff4040b0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:05.450 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.447+0000 7ff40bba2700 1 --2- 172.21.15.143:0/2518519653 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff40415a950 0x7ff4040c6340 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:05.450 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.447+0000 7ff40bba2700 1 -- 172.21.15.143:0/2518519653 >> 172.21.15.143:0/2518519653 conn(0x7ff4040cd2e0 msgr2=0x7ff4040cc730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:05.450 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.447+0000 7ff40bba2700 1 -- 172.21.15.143:0/2518519653 shutdown_connections 2023-11-08T19:25:05.451 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.447+0000 7ff40bba2700 1 -- 172.21.15.143:0/2518519653 wait complete. 2023-11-08T19:25:05.460 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-08T19:25:05.584 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.583+0000 7fab89519700 1 Processor -- start 2023-11-08T19:25:05.584 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.583+0000 7fab89519700 1 -- start start 2023-11-08T19:25:05.587 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.583+0000 7fab89519700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fab8415dca0 0x7fab840b93f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:05.587 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.583+0000 7fab89519700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fab840b5ae0 0x7fab840b7280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:05.587 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.583+0000 7fab89519700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab840b8f50 con 0x7fab840b5630 2023-11-08T19:25:05.587 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.583+0000 7fab89519700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab840b3360 con 0x7fab8415dca0 2023-11-08T19:25:05.587 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.583+0000 7fab89519700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab840b7eb0 con 0x7fab840b5ae0 2023-11-08T19:25:05.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.584+0000 7fab82ffd700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fab8415dca0 0x7fab840b93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:05.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.584+0000 7fab827fc700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fab840b5630 0x7fab840b6b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:45376/0 (socket says 172.21.15.143:45376) 2023-11-08T19:25:05.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.584+0000 7fab827fc700 1 -- 172.21.15.143:0/1859573573 learned_addr learned my addr 172.21.15.143:0/1859573573 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:25:05.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.584+0000 7fab837fe700 1 --2- 172.21.15.143:0/1859573573 >> v2:172.21.15.174:3300/0 conn(0x7fab840b5ae0 0x7fab840b7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:05.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.584+0000 7fab82ffd700 1 -- 172.21.15.143:0/1859573573 >> v2:172.21.15.174:3300/0 conn(0x7fab840b5ae0 msgr2=0x7fab840b7280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:05.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.584+0000 7fab82ffd700 1 --2- 172.21.15.143:0/1859573573 >> v2:172.21.15.174:3300/0 conn(0x7fab840b5ae0 0x7fab840b7280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:05.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.584+0000 7fab82ffd700 1 -- 172.21.15.143:0/1859573573 >> v1:172.21.15.143:6789/0 conn(0x7fab840b5630 legacy=0x7fab840b6b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:05.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.584+0000 7fab82ffd700 1 -- 172.21.15.143:0/1859573573 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab840b8190 con 0x7fab8415dca0 2023-11-08T19:25:05.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.584+0000 7fab82ffd700 1 --2- 172.21.15.143:0/1859573573 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fab8415dca0 0x7fab840b93f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fab840b35b0 tx=0x7fab78009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d0fd824823194b3 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:05.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.584+0000 7fab81ffb700 1 -- 172.21.15.143:0/1859573573 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab78011830 con 0x7fab8415dca0 2023-11-08T19:25:05.590 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.584+0000 7fab81ffb700 1 -- 172.21.15.143:0/1859573573 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab78011990 con 0x7fab8415dca0 2023-11-08T19:25:05.590 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.584+0000 7fab81ffb700 1 -- 172.21.15.143:0/1859573573 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab78011d20 con 0x7fab8415dca0 2023-11-08T19:25:05.590 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.585+0000 7fab89519700 1 -- 172.21.15.143:0/1859573573 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fab8415dca0 msgr2=0x7fab840b93f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:05.590 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.585+0000 7fab89519700 1 --2- 172.21.15.143:0/1859573573 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fab8415dca0 0x7fab840b93f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fab840b35b0 tx=0x7fab78009b20 comp rx=0 tx=0).stop 2023-11-08T19:25:05.590 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.585+0000 7fab89519700 1 -- 172.21.15.143:0/1859573573 shutdown_connections 2023-11-08T19:25:05.591 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.585+0000 7fab89519700 1 --2- 172.21.15.143:0/1859573573 >> v2:172.21.15.174:3300/0 conn(0x7fab840b5ae0 0x7fab840b7280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:05.591 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.585+0000 7fab89519700 1 --2- 172.21.15.143:0/1859573573 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fab8415dca0 0x7fab840b93f0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:05.591 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.585+0000 7fab89519700 1 -- 172.21.15.143:0/1859573573 >> 172.21.15.143:0/1859573573 conn(0x7fab84066d00 msgr2=0x7fab84067110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:05.591 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.585+0000 7fab89519700 1 -- 172.21.15.143:0/1859573573 shutdown_connections 2023-11-08T19:25:05.591 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.585+0000 7fab89519700 1 -- 172.21.15.143:0/1859573573 wait complete. 2023-11-08T19:25:05.594 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.586+0000 7fab89519700 1 Processor -- start 2023-11-08T19:25:05.595 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.586+0000 7fab89519700 1 -- start start 2023-11-08T19:25:05.595 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.587+0000 7fab89519700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fab840b5630 0x7fab840c8820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:05.595 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.587+0000 7fab89519700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fab840b5ae0 0x7fab840c9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:05.595 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.587+0000 7fab89519700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fab8415dca0 0x7fab840cac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:05.595 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.587+0000 7fab89519700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab840c8400 con 0x7fab8415dca0 2023-11-08T19:25:05.596 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.587+0000 7fab89519700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab8414e8a0 con 0x7fab840b5ae0 2023-11-08T19:25:05.596 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.587+0000 7fab89519700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab840c7780 con 0x7fab840b5630 2023-11-08T19:25:05.596 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.587+0000 7fab837fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fab8415dca0 0x7fab840cac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:05.596 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.587+0000 7fab837fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fab8415dca0 0x7fab840cac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49232/0 (socket says 172.21.15.143:49232) 2023-11-08T19:25:05.596 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.587+0000 7fab837fe700 1 -- 172.21.15.143:0/2182764278 learned_addr learned my addr 172.21.15.143:0/2182764278 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:05.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.587+0000 7fab827fc700 1 --2- 172.21.15.143:0/2182764278 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fab840b5ae0 0x7fab840c9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:05.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.587+0000 7fab82ffd700 1 --2- 172.21.15.143:0/2182764278 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fab840b5630 0x7fab840c8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:05.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.587+0000 7fab837fe700 1 -- 172.21.15.143:0/2182764278 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fab840b5630 msgr2=0x7fab840c8820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:05.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.587+0000 7fab837fe700 1 --2- 172.21.15.143:0/2182764278 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fab840b5630 0x7fab840c8820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:05.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.587+0000 7fab837fe700 1 -- 172.21.15.143:0/2182764278 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fab840b5ae0 msgr2=0x7fab840c9ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:05.598 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.587+0000 7fab837fe700 1 --2- 172.21.15.143:0/2182764278 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fab840b5ae0 0x7fab840c9ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:05.598 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.587+0000 7fab837fe700 1 -- 172.21.15.143:0/2182764278 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab780094a0 con 0x7fab8415dca0 2023-11-08T19:25:05.598 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.587+0000 7fab827fc700 1 --2- 172.21.15.143:0/2182764278 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fab840b5ae0 0x7fab840c9ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:25:05.598 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.587+0000 7fab837fe700 1 --2- 172.21.15.143:0/2182764278 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fab8415dca0 0x7fab840cac80 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fab7c007e40 tx=0x7fab7c00e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:05.599 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.587+0000 7fab6b7fe700 1 -- 172.21.15.143:0/2182764278 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab7c00eea0 con 0x7fab8415dca0 2023-11-08T19:25:05.599 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.587+0000 7fab6b7fe700 1 -- 172.21.15.143:0/2182764278 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab7c014420 con 0x7fab8415dca0 2023-11-08T19:25:05.599 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.587+0000 7fab89519700 1 -- 172.21.15.143:0/2182764278 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab840c7a60 con 0x7fab8415dca0 2023-11-08T19:25:05.599 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.587+0000 7fab6b7fe700 1 -- 172.21.15.143:0/2182764278 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab7c0147b0 con 0x7fab8415dca0 2023-11-08T19:25:05.599 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.587+0000 7fab89519700 1 -- 172.21.15.143:0/2182764278 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab840c8020 con 0x7fab8415dca0 2023-11-08T19:25:05.600 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.588+0000 7fab89519700 1 -- 172.21.15.143:0/2182764278 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab840c8540 con 0x7fab8415dca0 2023-11-08T19:25:05.600 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.589+0000 7fab6b7fe700 1 -- 172.21.15.143:0/2182764278 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fab7c014910 con 0x7fab8415dca0 2023-11-08T19:25:05.600 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.589+0000 7fab6b7fe700 1 --2- 172.21.15.143:0/2182764278 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fab7006af00 0x7fab7006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:05.600 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.589+0000 7fab6b7fe700 1 -- 172.21.15.143:0/2182764278 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fab7c089da0 con 0x7fab8415dca0 2023-11-08T19:25:05.600 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.589+0000 7fab82ffd700 1 --2- 172.21.15.143:0/2182764278 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fab7006af00 0x7fab7006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:05.600 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.590+0000 7fab82ffd700 1 --2- 172.21.15.143:0/2182764278 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fab7006af00 0x7fab7006d3c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fab780090b0 tx=0x7fab78008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:05.601 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.593+0000 7fab6b7fe700 1 -- 172.21.15.143:0/2182764278 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fab840c8540 con 0x7fab8415dca0 2023-11-08T19:25:05.787 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.785+0000 7fab89519700 1 -- 172.21.15.143:0/2182764278 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fab840c8540 con 0x7fab8415dca0 2023-11-08T19:25:05.787 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.786+0000 7fab6b7fe700 1 -- 172.21.15.143:0/2182764278 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7fab840c8540 con 0x7fab8415dca0 2023-11-08T19:25:05.788 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2023-11-08T19:25:05.789 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.787+0000 7fab89519700 1 -- 172.21.15.143:0/2182764278 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fab7006af00 msgr2=0x7fab7006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:05.789 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.788+0000 7fab89519700 1 --2- 172.21.15.143:0/2182764278 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fab7006af00 0x7fab7006d3c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fab780090b0 tx=0x7fab78008030 comp rx=0 tx=0).stop 2023-11-08T19:25:05.790 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.788+0000 7fab89519700 1 -- 172.21.15.143:0/2182764278 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fab8415dca0 msgr2=0x7fab840cac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:05.790 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.788+0000 7fab89519700 1 --2- 172.21.15.143:0/2182764278 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fab8415dca0 0x7fab840cac80 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fab7c007e40 tx=0x7fab7c00e3c0 comp rx=0 tx=0).stop 2023-11-08T19:25:05.790 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.788+0000 7fab89519700 1 -- 172.21.15.143:0/2182764278 shutdown_connections 2023-11-08T19:25:05.790 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.788+0000 7fab89519700 1 --2- 172.21.15.143:0/2182764278 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fab7006af00 0x7fab7006d3c0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:05.791 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.788+0000 7fab89519700 1 --2- 172.21.15.143:0/2182764278 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fab840b5630 0x7fab840c8820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:05.791 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.788+0000 7fab89519700 1 --2- 172.21.15.143:0/2182764278 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fab8415dca0 0x7fab840cac80 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:05.791 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.788+0000 7fab89519700 1 --2- 172.21.15.143:0/2182764278 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fab840b5ae0 0x7fab840c9ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:05.791 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.788+0000 7fab89519700 1 -- 172.21.15.143:0/2182764278 >> 172.21.15.143:0/2182764278 conn(0x7fab84066d00 msgr2=0x7fab840b35e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:05.792 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.788+0000 7fab89519700 1 -- 172.21.15.143:0/2182764278 shutdown_connections 2023-11-08T19:25:05.792 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.788+0000 7fab89519700 1 -- 172.21.15.143:0/2182764278 wait complete. 2023-11-08T19:25:05.801 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-08T19:25:05.923 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.922+0000 7f04b2478700 1 Processor -- start 2023-11-08T19:25:05.924 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.922+0000 7f04b2478700 1 -- start start 2023-11-08T19:25:05.929 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.923+0000 7f04b2478700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f04ac15bb20 0x7f04ac0b2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:05.930 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.923+0000 7f04b2478700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04ac0d0420 con 0x7f04ac0d1d80 2023-11-08T19:25:05.930 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.923+0000 7f04b2478700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04ac0cf810 con 0x7f04ac0b1730 2023-11-08T19:25:05.930 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.923+0000 7f04b2478700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04ac0cf600 con 0x7f04ac15bb20 2023-11-08T19:25:05.930 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.923+0000 7f04b0a15700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f04ac0d1d80 0x7f04ac0cc4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:45392/0 (socket says 172.21.15.143:45392) 2023-11-08T19:25:05.931 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.923+0000 7f04b0a15700 1 -- 172.21.15.143:0/563671869 learned_addr learned my addr 172.21.15.143:0/563671869 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:25:05.931 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.923+0000 7f04abfff700 1 --2- 172.21.15.143:0/563671869 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f04ac15bb20 0x7f04ac0b2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:05.931 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.923+0000 7f04aaffd700 1 -- 172.21.15.143:0/563671869 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1696852397 0 0) 0x7f04ac0d0420 con 0x7f04ac0d1d80 2023-11-08T19:25:05.931 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.923+0000 7f04aaffd700 1 -- 172.21.15.143:0/563671869 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f04940035b0 con 0x7f04ac0d1d80 2023-11-08T19:25:05.931 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.923+0000 7f04aaffd700 1 -- 172.21.15.143:0/563671869 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2874760918 0 0) 0x7f04940035b0 con 0x7f04ac0d1d80 2023-11-08T19:25:05.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.923+0000 7f04aaffd700 1 -- 172.21.15.143:0/563671869 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f04ac15bb20 msgr2=0x7f04ac0b2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:05.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.923+0000 7f04aaffd700 1 --2- 172.21.15.143:0/563671869 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f04ac15bb20 0x7f04ac0b2670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:05.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.923+0000 7f04aaffd700 1 -- 172.21.15.143:0/563671869 >> v1:172.21.15.174:6789/0 conn(0x7f04ac0b1730 legacy=0x7f04ac0d1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:05.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.923+0000 7f04aaffd700 1 -- 172.21.15.143:0/563671869 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04ac0cfaf0 con 0x7f04ac0d1d80 2023-11-08T19:25:05.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.923+0000 7f04aaffd700 1 -- 172.21.15.143:0/563671869 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4147756303 0 0) 0x7f04a40019f0 con 0x7f04ac0d1d80 2023-11-08T19:25:05.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.923+0000 7f04aaffd700 1 -- 172.21.15.143:0/563671869 >> v1:172.21.15.143:6789/0 conn(0x7f04ac0d1d80 legacy=0x7f04ac0cc4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:05.933 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.923+0000 7f04aaffd700 1 --2- 172.21.15.143:0/563671869 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0494003a30 0x7f0494003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:05.933 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.924+0000 7f04aaffd700 1 --2- 172.21.15.143:0/563671869 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f04940043e0 0x7f04940048b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:05.933 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.924+0000 7f04aaffd700 1 --2- 172.21.15.143:0/563671869 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f04ac15bb20 0x7f0494005110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:05.933 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.924+0000 7f04aaffd700 1 -- 172.21.15.143:0/563671869 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04ac0cf600 con 0x7f0494003a30 2023-11-08T19:25:05.933 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.924+0000 7f04aaffd700 1 -- 172.21.15.143:0/563671869 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f049400ba10 con 0x7f04940043e0 2023-11-08T19:25:05.934 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.924+0000 7f04aaffd700 1 -- 172.21.15.143:0/563671869 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f049400bbd0 con 0x7f04ac15bb20 2023-11-08T19:25:05.934 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.924+0000 7f04abfff700 1 --2- 172.21.15.143:0/563671869 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0494003a30 0x7f0494003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:05.934 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.924+0000 7f04ab7fe700 1 --2- 172.21.15.143:0/563671869 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f04940043e0 0x7f04940048b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:05.934 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.924+0000 7f04b0a15700 1 --2- 172.21.15.143:0/563671869 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f04ac15bb20 0x7f0494005110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:05.934 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.924+0000 7f04abfff700 1 -- 172.21.15.143:0/563671869 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f04ac15bb20 msgr2=0x7f0494005110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:05.935 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.924+0000 7f04abfff700 1 --2- 172.21.15.143:0/563671869 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f04ac15bb20 0x7f0494005110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:05.935 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.924+0000 7f04abfff700 1 -- 172.21.15.143:0/563671869 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f04940043e0 msgr2=0x7f04940048b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:05.935 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.924+0000 7f04abfff700 1 --2- 172.21.15.143:0/563671869 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f04940043e0 0x7f04940048b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:05.935 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.924+0000 7f04abfff700 1 -- 172.21.15.143:0/563671869 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04940035b0 con 0x7f0494003a30 2023-11-08T19:25:05.935 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.924+0000 7f04ab7fe700 1 --2- 172.21.15.143:0/563671869 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f04940043e0 0x7f04940048b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:25:05.935 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.924+0000 7f04abfff700 1 --2- 172.21.15.143:0/563671869 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0494003a30 0x7f0494003e10 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f04a0009b60 tx=0x7f0494005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2d4e4e162acf66f server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:05.936 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.924+0000 7f04aaffd700 1 -- 172.21.15.143:0/563671869 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04a000e6c0 con 0x7f0494003a30 2023-11-08T19:25:05.936 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.924+0000 7f04aaffd700 1 -- 172.21.15.143:0/563671869 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04a000e820 con 0x7f0494003a30 2023-11-08T19:25:05.936 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.924+0000 7f04aaffd700 1 -- 172.21.15.143:0/563671869 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04a000eb90 con 0x7f0494003a30 2023-11-08T19:25:05.936 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.925+0000 7f04b2478700 1 -- 172.21.15.143:0/563671869 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0494003a30 msgr2=0x7f0494003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:05.936 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.925+0000 7f04b2478700 1 --2- 172.21.15.143:0/563671869 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0494003a30 0x7f0494003e10 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f04a0009b60 tx=0x7f0494005880 comp rx=0 tx=0).stop 2023-11-08T19:25:05.937 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.925+0000 7f04b0a15700 1 -- 172.21.15.143:0/563671869 reap_dead start 2023-11-08T19:25:05.937 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.925+0000 7f04b2478700 1 -- 172.21.15.143:0/563671869 shutdown_connections 2023-11-08T19:25:05.937 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.925+0000 7f04b2478700 1 -- 172.21.15.143:0/563671869 >> 172.21.15.143:0/563671869 conn(0x7f04ac08b000 msgr2=0x7f04ac09ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:05.937 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.925+0000 7f04b2478700 1 -- 172.21.15.143:0/563671869 shutdown_connections 2023-11-08T19:25:05.937 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.925+0000 7f04b2478700 1 -- 172.21.15.143:0/563671869 wait complete. 2023-11-08T19:25:05.938 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.926+0000 7f04b2478700 1 Processor -- start 2023-11-08T19:25:05.938 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.926+0000 7f04b2478700 1 -- start start 2023-11-08T19:25:05.938 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.927+0000 7f04b2478700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0494003a30 0x7f04ac0cd4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:05.938 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.927+0000 7f04b2478700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f04ac1f4bd0 0x7f04ac1f0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:05.938 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.927+0000 7f04b2478700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f04ac0b4650 0x7f04ac0b4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:05.939 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.927+0000 7f04b2478700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04ac0b5040 con 0x7f04ac1f4bd0 2023-11-08T19:25:05.939 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.927+0000 7f04b2478700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04ac0cdd10 con 0x7f04ac0b4650 2023-11-08T19:25:05.939 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.927+0000 7f04b2478700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04ac0cdff0 con 0x7f0494003a30 2023-11-08T19:25:05.939 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.927+0000 7f04ab7fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f04ac1f4bd0 0x7f04ac1f0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:05.939 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.927+0000 7f04ab7fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f04ac1f4bd0 0x7f04ac1f0ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49248/0 (socket says 172.21.15.143:49248) 2023-11-08T19:25:05.939 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.927+0000 7f04ab7fe700 1 -- 172.21.15.143:0/1453440380 learned_addr learned my addr 172.21.15.143:0/1453440380 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:05.940 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.927+0000 7f04b0a15700 1 --2- 172.21.15.143:0/1453440380 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f04ac0b4650 0x7f04ac0b4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:05.940 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.927+0000 7f04abfff700 1 --2- 172.21.15.143:0/1453440380 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0494003a30 0x7f04ac0cd4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:05.940 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.927+0000 7f04ab7fe700 1 -- 172.21.15.143:0/1453440380 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0494003a30 msgr2=0x7f04ac0cd4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:05.940 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.927+0000 7f04ab7fe700 1 --2- 172.21.15.143:0/1453440380 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0494003a30 0x7f04ac0cd4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:05.940 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.927+0000 7f04ab7fe700 1 -- 172.21.15.143:0/1453440380 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f04ac0b4650 msgr2=0x7f04ac0b4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:05.941 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.927+0000 7f04ab7fe700 1 --2- 172.21.15.143:0/1453440380 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f04ac0b4650 0x7f04ac0b4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:05.941 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.927+0000 7f04ab7fe700 1 -- 172.21.15.143:0/1453440380 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04ac0ce2d0 con 0x7f04ac1f4bd0 2023-11-08T19:25:05.941 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.927+0000 7f04ab7fe700 1 --2- 172.21.15.143:0/1453440380 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f04ac1f4bd0 0x7f04ac1f0ea0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f049c00e7d0 tx=0x7f049c00eae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:05.941 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.927+0000 7f04a8ff9700 1 -- 172.21.15.143:0/1453440380 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f049c0052e0 con 0x7f04ac1f4bd0 2023-11-08T19:25:05.941 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.927+0000 7f04a8ff9700 1 -- 172.21.15.143:0/1453440380 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f049c005440 con 0x7f04ac1f4bd0 2023-11-08T19:25:05.941 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.927+0000 7f04b2478700 1 -- 172.21.15.143:0/1453440380 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04ac0ce5b0 con 0x7f04ac1f4bd0 2023-11-08T19:25:05.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.927+0000 7f04a8ff9700 1 -- 172.21.15.143:0/1453440380 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f049c003880 con 0x7f04ac1f4bd0 2023-11-08T19:25:05.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.928+0000 7f04b2478700 1 -- 172.21.15.143:0/1453440380 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04ac0cda20 con 0x7f04ac1f4bd0 2023-11-08T19:25:05.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.928+0000 7f04b2478700 1 -- 172.21.15.143:0/1453440380 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04ac0cead0 con 0x7f04ac1f4bd0 2023-11-08T19:25:05.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.929+0000 7f04a8ff9700 1 -- 172.21.15.143:0/1453440380 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f049c0039e0 con 0x7f04ac1f4bd0 2023-11-08T19:25:05.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.929+0000 7f04a8ff9700 1 --2- 172.21.15.143:0/1453440380 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f049806af00 0x7f049806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:05.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.929+0000 7f04a8ff9700 1 -- 172.21.15.143:0/1453440380 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f049c088c60 con 0x7f04ac1f4bd0 2023-11-08T19:25:05.944 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.929+0000 7f04abfff700 1 --2- 172.21.15.143:0/1453440380 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f049806af00 0x7f049806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:05.944 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.930+0000 7f04abfff700 1 --2- 172.21.15.143:0/1453440380 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f049806af00 0x7f049806d3c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f04a0000c00 tx=0x7f04a001f040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:05.944 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:05.933+0000 7f04a8ff9700 1 -- 172.21.15.143:0/1453440380 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f04ac0cead0 con 0x7f04ac1f4bd0 2023-11-08T19:25:06.124 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.123+0000 7f04b2478700 1 -- 172.21.15.143:0/1453440380 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f04ac04eee0 con 0x7f04ac1f4bd0 2023-11-08T19:25:06.125 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.123+0000 7f04a8ff9700 1 -- 172.21.15.143:0/1453440380 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f049c019020 con 0x7f04ac1f4bd0 2023-11-08T19:25:06.125 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 64 2023-11-08T19:25:06.127 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.125+0000 7f04b2478700 1 -- 172.21.15.143:0/1453440380 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f049806af00 msgr2=0x7f049806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:06.127 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.125+0000 7f04b2478700 1 --2- 172.21.15.143:0/1453440380 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f049806af00 0x7f049806d3c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f04a0000c00 tx=0x7f04a001f040 comp rx=0 tx=0).stop 2023-11-08T19:25:06.127 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.125+0000 7f04b2478700 1 -- 172.21.15.143:0/1453440380 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f04ac1f4bd0 msgr2=0x7f04ac1f0ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:06.127 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.125+0000 7f04b2478700 1 --2- 172.21.15.143:0/1453440380 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f04ac1f4bd0 0x7f04ac1f0ea0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f049c00e7d0 tx=0x7f049c00eae0 comp rx=0 tx=0).stop 2023-11-08T19:25:06.127 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.125+0000 7f04b2478700 1 -- 172.21.15.143:0/1453440380 shutdown_connections 2023-11-08T19:25:06.128 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.125+0000 7f04b2478700 1 --2- 172.21.15.143:0/1453440380 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f049806af00 0x7f049806d3c0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:06.128 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.125+0000 7f04b2478700 1 --2- 172.21.15.143:0/1453440380 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0494003a30 0x7f04ac0cd4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:06.128 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.125+0000 7f04b2478700 1 --2- 172.21.15.143:0/1453440380 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f04ac0b4650 0x7f04ac0b4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:06.128 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.125+0000 7f04b2478700 1 --2- 172.21.15.143:0/1453440380 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f04ac1f4bd0 0x7f04ac1f0ea0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:06.128 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.125+0000 7f04b2478700 1 -- 172.21.15.143:0/1453440380 >> 172.21.15.143:0/1453440380 conn(0x7f04ac08b000 msgr2=0x7f04ac0b3b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:06.129 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.125+0000 7f04b2478700 1 -- 172.21.15.143:0/1453440380 shutdown_connections 2023-11-08T19:25:06.129 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.125+0000 7f04b2478700 1 -- 172.21.15.143:0/1453440380 wait complete. 2023-11-08T19:25:06.139 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:25:06.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.258+0000 7f1f2fb22700 1 Processor -- start 2023-11-08T19:25:06.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.258+0000 7f1f2fb22700 1 -- start start 2023-11-08T19:25:06.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.259+0000 7f1f2fb22700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1f2815aa30 0x7f1f280cf510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:06.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.259+0000 7f1f2fb22700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f1f280b2210 0x7f1f280b2690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:06.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.259+0000 7f1f2fb22700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f280cf080 con 0x7f1f280cbe20 2023-11-08T19:25:06.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.259+0000 7f1f2fb22700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f280b2bd0 con 0x7f1f280b2210 2023-11-08T19:25:06.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.259+0000 7f1f2fb22700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f280ce470 con 0x7f1f2815aa30 2023-11-08T19:25:06.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.259+0000 7f1f2e0bf700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f1f280cbe20 0x7f1f280cc270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:45412/0 (socket says 172.21.15.143:45412) 2023-11-08T19:25:06.264 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.259+0000 7f1f2e0bf700 1 -- 172.21.15.143:0/1660581701 learned_addr learned my addr 172.21.15.143:0/1660581701 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:25:06.264 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.259+0000 7f1f2d8be700 1 --2- 172.21.15.143:0/1660581701 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1f2815aa30 0x7f1f280cf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:06.264 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.259+0000 7f1f2d0bd700 1 --2- 172.21.15.143:0/1660581701 >> v2:172.21.15.174:3300/0 conn(0x7f1f280b2210 0x7f1f280b2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:06.264 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.259+0000 7f1f2c8bc700 1 -- 172.21.15.143:0/1660581701 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2646800538 0 0) 0x7f1f280cf080 con 0x7f1f280cbe20 2023-11-08T19:25:06.264 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.259+0000 7f1f2c8bc700 1 -- 172.21.15.143:0/1660581701 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f140035b0 con 0x7f1f280cbe20 2023-11-08T19:25:06.265 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.260+0000 7f1f2c8bc700 1 -- 172.21.15.143:0/1660581701 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1181060974 0 0) 0x7f1f140035b0 con 0x7f1f280cbe20 2023-11-08T19:25:06.265 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.260+0000 7f1f2d0bd700 1 -- 172.21.15.143:0/1660581701 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1f2815aa30 msgr2=0x7f1f280cf510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:06.265 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.260+0000 7f1f2d0bd700 1 --2- 172.21.15.143:0/1660581701 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1f2815aa30 0x7f1f280cf510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:06.265 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.260+0000 7f1f2d0bd700 1 -- 172.21.15.143:0/1660581701 >> v1:172.21.15.143:6789/0 conn(0x7f1f280cbe20 legacy=0x7f1f280cc270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:06.265 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.260+0000 7f1f2d0bd700 1 -- 172.21.15.143:0/1660581701 --> v2:172.21.15.174:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f280ce750 con 0x7f1f280b2210 2023-11-08T19:25:06.266 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.260+0000 7f1f2d0bd700 1 --2- 172.21.15.143:0/1660581701 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1f280b2210 0x7f1f280b2690 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1f24009840 tx=0x7f1f24009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f40613bca5def66e server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:06.266 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.260+0000 7f1f2c8bc700 1 -- 172.21.15.143:0/1660581701 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f200019f0 con 0x7f1f280b2210 2023-11-08T19:25:06.266 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.260+0000 7f1f2c8bc700 1 -- 172.21.15.143:0/1660581701 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f24004030 con 0x7f1f280b2210 2023-11-08T19:25:06.266 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.260+0000 7f1f2c8bc700 1 -- 172.21.15.143:0/1660581701 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f240043a0 con 0x7f1f280b2210 2023-11-08T19:25:06.266 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.261+0000 7f1f2fb22700 1 -- 172.21.15.143:0/1660581701 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1f280b2210 msgr2=0x7f1f280b2690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:06.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.261+0000 7f1f2fb22700 1 --2- 172.21.15.143:0/1660581701 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1f280b2210 0x7f1f280b2690 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1f24009840 tx=0x7f1f24009b50 comp rx=0 tx=0).stop 2023-11-08T19:25:06.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.261+0000 7f1f2fb22700 1 -- 172.21.15.143:0/1660581701 shutdown_connections 2023-11-08T19:25:06.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.261+0000 7f1f2fb22700 1 --2- 172.21.15.143:0/1660581701 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1f2815aa30 0x7f1f280cf510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:06.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.261+0000 7f1f2fb22700 1 --2- 172.21.15.143:0/1660581701 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1f280b2210 0x7f1f280b2690 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:06.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.261+0000 7f1f2fb22700 1 -- 172.21.15.143:0/1660581701 >> 172.21.15.143:0/1660581701 conn(0x7f1f28099ab0 msgr2=0x7f1f280d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:06.268 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.261+0000 7f1f2fb22700 1 -- 172.21.15.143:0/1660581701 shutdown_connections 2023-11-08T19:25:06.268 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.261+0000 7f1f2fb22700 1 -- 172.21.15.143:0/1660581701 wait complete. 2023-11-08T19:25:06.270 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.262+0000 7f1f2fb22700 1 Processor -- start 2023-11-08T19:25:06.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.262+0000 7f1f2fb22700 1 -- start start 2023-11-08T19:25:06.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.263+0000 7f1f2fb22700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1f280b2210 0x7f1f280c9e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:06.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.263+0000 7f1f2fb22700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1f280cbe20 0x7f1f2814f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:06.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.263+0000 7f1f2fb22700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1f2815aa30 0x7f1f2814f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:06.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.263+0000 7f1f2fb22700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f2814f570 con 0x7f1f280b2210 2023-11-08T19:25:06.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.263+0000 7f1f2fb22700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f2814ffb0 con 0x7f1f280cbe20 2023-11-08T19:25:06.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.263+0000 7f1f2fb22700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f28150290 con 0x7f1f2815aa30 2023-11-08T19:25:06.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.263+0000 7f1f2d8be700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1f280b2210 0x7f1f280c9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:06.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.263+0000 7f1f2d8be700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1f280b2210 0x7f1f280c9e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49260/0 (socket says 172.21.15.143:49260) 2023-11-08T19:25:06.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.263+0000 7f1f2d8be700 1 -- 172.21.15.143:0/4184297517 learned_addr learned my addr 172.21.15.143:0/4184297517 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:06.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.263+0000 7f1f2d0bd700 1 --2- 172.21.15.143:0/4184297517 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1f280cbe20 0x7f1f2814f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:06.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.263+0000 7f1f2e0bf700 1 --2- 172.21.15.143:0/4184297517 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1f2815aa30 0x7f1f2814f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:06.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.264+0000 7f1f2d8be700 1 -- 172.21.15.143:0/4184297517 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1f2815aa30 msgr2=0x7f1f2814f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:06.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.264+0000 7f1f2d8be700 1 --2- 172.21.15.143:0/4184297517 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1f2815aa30 0x7f1f2814f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:06.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.264+0000 7f1f2d8be700 1 -- 172.21.15.143:0/4184297517 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1f280cbe20 msgr2=0x7f1f2814f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:06.274 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.264+0000 7f1f2d8be700 1 --2- 172.21.15.143:0/4184297517 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1f280cbe20 0x7f1f2814f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:06.274 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.264+0000 7f1f2d8be700 1 -- 172.21.15.143:0/4184297517 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f240094a0 con 0x7f1f280b2210 2023-11-08T19:25:06.274 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.264+0000 7f1f2d0bd700 1 --2- 172.21.15.143:0/4184297517 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1f280cbe20 0x7f1f2814f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:25:06.274 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.264+0000 7f1f2e0bf700 1 --2- 172.21.15.143:0/4184297517 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1f2815aa30 0x7f1f2814f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:25:06.274 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.264+0000 7f1f2d8be700 1 --2- 172.21.15.143:0/4184297517 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1f280b2210 0x7f1f280c9e50 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1f1800e900 tx=0x7f1f1800ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:06.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.264+0000 7f1f1e7fc700 1 -- 172.21.15.143:0/4184297517 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f18009650 con 0x7f1f280b2210 2023-11-08T19:25:06.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.264+0000 7f1f1e7fc700 1 -- 172.21.15.143:0/4184297517 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f1800c850 con 0x7f1f280b2210 2023-11-08T19:25:06.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.264+0000 7f1f2fb22700 1 -- 172.21.15.143:0/4184297517 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f28150570 con 0x7f1f280b2210 2023-11-08T19:25:06.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.264+0000 7f1f1e7fc700 1 -- 172.21.15.143:0/4184297517 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f1800cbc0 con 0x7f1f280b2210 2023-11-08T19:25:06.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.264+0000 7f1f2fb22700 1 -- 172.21.15.143:0/4184297517 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f2814fca0 con 0x7f1f280b2210 2023-11-08T19:25:06.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.265+0000 7f1f2fb22700 1 -- 172.21.15.143:0/4184297517 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f280c5250 con 0x7f1f280b2210 2023-11-08T19:25:06.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.265+0000 7f1f1e7fc700 1 -- 172.21.15.143:0/4184297517 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1f1800cd20 con 0x7f1f280b2210 2023-11-08T19:25:06.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.266+0000 7f1f1e7fc700 1 --2- 172.21.15.143:0/4184297517 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f1f0c06af00 0x7f1f0c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:06.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.266+0000 7f1f1e7fc700 1 -- 172.21.15.143:0/4184297517 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1f18089a50 con 0x7f1f280b2210 2023-11-08T19:25:06.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.266+0000 7f1f2d0bd700 1 --2- 172.21.15.143:0/4184297517 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f1f0c06af00 0x7f1f0c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:06.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.266+0000 7f1f2d0bd700 1 --2- 172.21.15.143:0/4184297517 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f1f0c06af00 0x7f1f0c06d3c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1f240090b0 tx=0x7f1f24008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:06.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.270+0000 7f1f1e7fc700 1 -- 172.21.15.143:0/4184297517 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1f18054b00 con 0x7f1f280b2210 2023-11-08T19:25:06.500 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.499+0000 7f1f2fb22700 1 -- 172.21.15.143:0/4184297517 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1f28150a90 con 0x7f1f280b2210 2023-11-08T19:25:06.501 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.500+0000 7f1f1e7fc700 1 -- 172.21.15.143:0/4184297517 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4913 (secure 0 0 0) 0x7f1f18021090 con 0x7f1f280b2210 2023-11-08T19:25:06.502 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:25:06.502 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4208,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6834/37280614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":37280614},{"type":"v1","addr":"172.21.15.143:6834","nonce":37280614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/2364281303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":2364281303},{"type":"v1","addr":"172.21.15.143:6835","nonce":2364281303}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6836/4066129908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":4066129908},{"type":"v1","addr":"172.21.15.174:6836","nonce":4066129908}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T19:24:49.995547+0000","modified":"2023-11-08T19:24:53.027089+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.174:6837/2124972405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6835","nonce":2124972405},{"type":"v1","addr":"172.21.15.174:6837","nonce":2124972405}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-08T19:25:06.503 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.502+0000 7f1f2fb22700 1 -- 172.21.15.143:0/4184297517 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f1f0c06af00 msgr2=0x7f1f0c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:06.503 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.502+0000 7f1f2fb22700 1 --2- 172.21.15.143:0/4184297517 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f1f0c06af00 0x7f1f0c06d3c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1f240090b0 tx=0x7f1f24008030 comp rx=0 tx=0).stop 2023-11-08T19:25:06.504 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.502+0000 7f1f2fb22700 1 -- 172.21.15.143:0/4184297517 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1f280b2210 msgr2=0x7f1f280c9e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:06.504 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.502+0000 7f1f2fb22700 1 --2- 172.21.15.143:0/4184297517 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1f280b2210 0x7f1f280c9e50 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1f1800e900 tx=0x7f1f1800ecc0 comp rx=0 tx=0).stop 2023-11-08T19:25:06.504 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.502+0000 7f1f2fb22700 1 -- 172.21.15.143:0/4184297517 shutdown_connections 2023-11-08T19:25:06.504 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.502+0000 7f1f2fb22700 1 --2- 172.21.15.143:0/4184297517 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f1f0c06af00 0x7f1f0c06d3c0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:06.504 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.502+0000 7f1f2fb22700 1 --2- 172.21.15.143:0/4184297517 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1f2815aa30 0x7f1f2814f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:06.505 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.502+0000 7f1f2fb22700 1 --2- 172.21.15.143:0/4184297517 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1f280b2210 0x7f1f280c9e50 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:06.505 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.502+0000 7f1f2fb22700 1 --2- 172.21.15.143:0/4184297517 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1f280cbe20 0x7f1f2814f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:06.505 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.502+0000 7f1f2fb22700 1 -- 172.21.15.143:0/4184297517 >> 172.21.15.143:0/4184297517 conn(0x7f1f28099ab0 msgr2=0x7f1f280b12f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:06.505 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.502+0000 7f1f2fb22700 1 -- 172.21.15.143:0/4184297517 shutdown_connections 2023-11-08T19:25:06.505 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.502+0000 7f1f2fb22700 1 -- 172.21.15.143:0/4184297517 wait complete. 2023-11-08T19:25:06.506 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 8 2023-11-08T19:25:06.515 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:25:06.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.636+0000 7f35626a8700 1 Processor -- start 2023-11-08T19:25:06.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.636+0000 7f35626a8700 1 -- start start 2023-11-08T19:25:06.640 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.636+0000 7f35626a8700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f355c0b2920 0x7f355c0cf510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:06.640 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.636+0000 7f35626a8700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f355c0cbe20 0x7f355c0cc290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:06.640 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.636+0000 7f35626a8700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f355c0cf080 con 0x7f355c15aa30 2023-11-08T19:25:06.640 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.636+0000 7f35626a8700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f355c0cc7d0 con 0x7f355c0cbe20 2023-11-08T19:25:06.641 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.636+0000 7f35626a8700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f355c0ce470 con 0x7f355c0b2920 2023-11-08T19:25:06.641 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.637+0000 7f3560c45700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f355c0cbe20 0x7f355c0cc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:06.641 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.637+0000 7f3560c45700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f355c0cbe20 0x7f355c0cc290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49272/0 (socket says 172.21.15.143:49272) 2023-11-08T19:25:06.641 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.637+0000 7f3560c45700 1 -- 172.21.15.143:0/2903655598 learned_addr learned my addr 172.21.15.143:0/2903655598 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:06.641 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.637+0000 7f355b7fe700 1 --2- 172.21.15.143:0/2903655598 >> v2:172.21.15.174:3300/0 conn(0x7f355c0b2920 0x7f355c0cf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:06.642 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.637+0000 7f3560c45700 1 -- 172.21.15.143:0/2903655598 >> v2:172.21.15.174:3300/0 conn(0x7f355c0b2920 msgr2=0x7f355c0cf510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:06.642 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.637+0000 7f3560c45700 1 --2- 172.21.15.143:0/2903655598 >> v2:172.21.15.174:3300/0 conn(0x7f355c0b2920 0x7f355c0cf510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:06.642 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.637+0000 7f3560c45700 1 -- 172.21.15.143:0/2903655598 >> v1:172.21.15.174:6789/0 conn(0x7f355c15aa30 legacy=0x7f355c0b2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:06.642 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.637+0000 7f3560c45700 1 -- 172.21.15.143:0/2903655598 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f355c0ce750 con 0x7f355c0cbe20 2023-11-08T19:25:06.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.637+0000 7f3560c45700 1 --2- 172.21.15.143:0/2903655598 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f355c0cbe20 0x7f355c0cc290 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f355c0ce260 tx=0x7f355400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b5686f3ae14496a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:06.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.637+0000 7f355affd700 1 -- 172.21.15.143:0/2903655598 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f355400e070 con 0x7f355c0cbe20 2023-11-08T19:25:06.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.637+0000 7f355affd700 1 -- 172.21.15.143:0/2903655598 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f355400b910 con 0x7f355c0cbe20 2023-11-08T19:25:06.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.637+0000 7f355affd700 1 -- 172.21.15.143:0/2903655598 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f355400bcc0 con 0x7f355c0cbe20 2023-11-08T19:25:06.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.638+0000 7f35626a8700 1 -- 172.21.15.143:0/2903655598 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f355c0cbe20 msgr2=0x7f355c0cc290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:06.644 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.638+0000 7f35626a8700 1 --2- 172.21.15.143:0/2903655598 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f355c0cbe20 0x7f355c0cc290 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f355c0ce260 tx=0x7f355400b330 comp rx=0 tx=0).stop 2023-11-08T19:25:06.644 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.638+0000 7f35626a8700 1 -- 172.21.15.143:0/2903655598 shutdown_connections 2023-11-08T19:25:06.644 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.638+0000 7f35626a8700 1 --2- 172.21.15.143:0/2903655598 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f355c0cbe20 0x7f355c0cc290 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:06.644 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.638+0000 7f35626a8700 1 --2- 172.21.15.143:0/2903655598 >> v2:172.21.15.174:3300/0 conn(0x7f355c0b2920 0x7f355c0cf510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:06.644 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.638+0000 7f35626a8700 1 -- 172.21.15.143:0/2903655598 >> 172.21.15.143:0/2903655598 conn(0x7f355c099ab0 msgr2=0x7f355c0d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:06.645 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.638+0000 7f35626a8700 1 -- 172.21.15.143:0/2903655598 shutdown_connections 2023-11-08T19:25:06.645 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.638+0000 7f35626a8700 1 -- 172.21.15.143:0/2903655598 wait complete. 2023-11-08T19:25:06.647 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.639+0000 7f35626a8700 1 Processor -- start 2023-11-08T19:25:06.648 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.639+0000 7f35626a8700 1 -- start start 2023-11-08T19:25:06.648 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.640+0000 7f35626a8700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f355c0b2920 0x7f355c157040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:06.648 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.640+0000 7f35626a8700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f355c0cbe20 0x7f355c1e3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:06.648 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.640+0000 7f35626a8700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f355c15aa30 0x7f355c0b52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:06.649 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.640+0000 7f35626a8700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f355c0c5300 con 0x7f355c0b2920 2023-11-08T19:25:06.649 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.640+0000 7f35626a8700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f355c1e4ea0 con 0x7f355c15aa30 2023-11-08T19:25:06.649 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.640+0000 7f35626a8700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f355c1e4600 con 0x7f355c0cbe20 2023-11-08T19:25:06.649 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.640+0000 7f355bfff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f355c0b2920 0x7f355c157040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:06.649 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.640+0000 7f355bfff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f355c0b2920 0x7f355c157040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49274/0 (socket says 172.21.15.143:49274) 2023-11-08T19:25:06.650 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.640+0000 7f355bfff700 1 -- 172.21.15.143:0/595509496 learned_addr learned my addr 172.21.15.143:0/595509496 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:06.650 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.640+0000 7f3560c45700 1 --2- 172.21.15.143:0/595509496 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f355c15aa30 0x7f355c0b52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:06.650 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.640+0000 7f355b7fe700 1 --2- 172.21.15.143:0/595509496 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f355c0cbe20 0x7f355c1e3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:06.650 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.640+0000 7f355bfff700 1 -- 172.21.15.143:0/595509496 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f355c0cbe20 msgr2=0x7f355c1e3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:06.650 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.640+0000 7f355bfff700 1 --2- 172.21.15.143:0/595509496 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f355c0cbe20 0x7f355c1e3cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:06.651 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.640+0000 7f355bfff700 1 -- 172.21.15.143:0/595509496 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f355c15aa30 msgr2=0x7f355c0b52c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:06.651 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.640+0000 7f355bfff700 1 --2- 172.21.15.143:0/595509496 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f355c15aa30 0x7f355c0b52c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:06.651 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.640+0000 7f355bfff700 1 -- 172.21.15.143:0/595509496 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3554009b30 con 0x7f355c0b2920 2023-11-08T19:25:06.651 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.641+0000 7f355bfff700 1 --2- 172.21.15.143:0/595509496 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f355c0b2920 0x7f355c157040 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f354c00c430 tx=0x7f354c00c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:06.651 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.641+0000 7f3558ff9700 1 -- 172.21.15.143:0/595509496 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f354c0052d0 con 0x7f355c0b2920 2023-11-08T19:25:06.652 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.641+0000 7f3558ff9700 1 -- 172.21.15.143:0/595509496 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f354c005450 con 0x7f355c0b2920 2023-11-08T19:25:06.652 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.641+0000 7f3558ff9700 1 -- 172.21.15.143:0/595509496 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f354c006af0 con 0x7f355c0b2920 2023-11-08T19:25:06.652 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.641+0000 7f35626a8700 1 -- 172.21.15.143:0/595509496 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f355c1e48e0 con 0x7f355c0b2920 2023-11-08T19:25:06.652 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.641+0000 7f35626a8700 1 -- 172.21.15.143:0/595509496 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f355c0c5530 con 0x7f355c0b2920 2023-11-08T19:25:06.652 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.642+0000 7f35626a8700 1 -- 172.21.15.143:0/595509496 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f355c04eee0 con 0x7f355c0b2920 2023-11-08T19:25:06.653 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.642+0000 7f3558ff9700 1 -- 172.21.15.143:0/595509496 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f354c0036d0 con 0x7f355c0b2920 2023-11-08T19:25:06.653 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.642+0000 7f3558ff9700 1 --2- 172.21.15.143:0/595509496 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f354806af00 0x7f354806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:06.653 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.643+0000 7f3558ff9700 1 -- 172.21.15.143:0/595509496 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f354c088710 con 0x7f355c0b2920 2023-11-08T19:25:06.653 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.643+0000 7f355b7fe700 1 --2- 172.21.15.143:0/595509496 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f354806af00 0x7f354806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:06.653 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.643+0000 7f355b7fe700 1 --2- 172.21.15.143:0/595509496 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f354806af00 0x7f354806d3c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f355c1e5770 tx=0x7f3550009210 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:06.654 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.647+0000 7f3558ff9700 1 -- 172.21.15.143:0/595509496 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f354c053720 con 0x7f355c0b2920 2023-11-08T19:25:06.842 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.840+0000 7f35626a8700 1 -- 172.21.15.143:0/595509496 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f355c0b2ff0 con 0x7f355c0b2920 2023-11-08T19:25:06.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.841+0000 7f3558ff9700 1 -- 172.21.15.143:0/595509496 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16553 (secure 0 0 0) 0x7f354c056d40 con 0x7f355c0b2920 2023-11-08T19:25:06.843 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:25:06.844 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":20,"fsid":"968483af-77fc-4068-ad7c-5c2a1a4613d7","created":"2023-11-08T19:24:32.826484+0000","modified":"2023-11-08T19:24:51.618072+0000","last_up_change":"2023-11-08T19:24:43.573880+0000","last_in_change":"2023-11-08T19:24:35.314525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:44.551530+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-08T19:24:48.256781+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-08T19:24:48.954990+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bae98ec1-d94e-403d-a220-a522a53ea6f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6809","nonce":1487830566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6811","nonce":1487830566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6815","nonce":1487830566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6813","nonce":1487830566}]},"public_addr":"172.21.15.143:6809/1487830566","cluster_addr":"172.21.15.143:6811/1487830566","heartbeat_back_addr":"172.21.15.143:6815/1487830566","heartbeat_front_addr":"172.21.15.143:6813/1487830566","state":["exists","up"]},{"osd":1,"uuid":"5cb6538a-1ccb-4322-8633-09f66af0b7e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6801","nonce":787341397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6803","nonce":787341397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6807","nonce":787341397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6805","nonce":787341397}]},"public_addr":"172.21.15.143:6801/787341397","cluster_addr":"172.21.15.143:6803/787341397","heartbeat_back_addr":"172.21.15.143:6807/787341397","heartbeat_front_addr":"172.21.15.143:6805/787341397","state":["exists","up"]},{"osd":2,"uuid":"cdbf522c-60ef-4d60-9653-8dcb529c101a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6822","nonce":2301325597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6826","nonce":2301325597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6831","nonce":2301325597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6829","nonce":2301325597}]},"public_addr":"172.21.15.143:6822/2301325597","cluster_addr":"172.21.15.143:6826/2301325597","heartbeat_back_addr":"172.21.15.143:6831/2301325597","heartbeat_front_addr":"172.21.15.143:6829/2301325597","state":["exists","up"]},{"osd":3,"uuid":"4e2783cf-ebb9-4dd8-8245-bc1d17ce4325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6817","nonce":2021504428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6819","nonce":2021504428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6827","nonce":2021504428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6823","nonce":2021504428}]},"public_addr":"172.21.15.143:6817/2021504428","cluster_addr":"172.21.15.143:6819/2021504428","heartbeat_back_addr":"172.21.15.143:6827/2021504428","heartbeat_front_addr":"172.21.15.143:6823/2021504428","state":["exists","up"]},{"osd":4,"uuid":"691256fa-9b77-49b9-a346-7ae1adf0ca80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6801","nonce":1993380595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6803","nonce":1993380595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6807","nonce":1993380595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6805","nonce":1993380595}]},"public_addr":"172.21.15.174:6801/1993380595","cluster_addr":"172.21.15.174:6803/1993380595","heartbeat_back_addr":"172.21.15.174:6807/1993380595","heartbeat_front_addr":"172.21.15.174:6805/1993380595","state":["exists","up"]},{"osd":5,"uuid":"756fc352-48c9-46ce-b52e-0541fe6f8525","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6809","nonce":1777923303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6811","nonce":1777923303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6815","nonce":1777923303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6813","nonce":1777923303}]},"public_addr":"172.21.15.174:6809/1777923303","cluster_addr":"172.21.15.174:6811/1777923303","heartbeat_back_addr":"172.21.15.174:6815/1777923303","heartbeat_front_addr":"172.21.15.174:6813/1777923303","state":["exists","up"]},{"osd":6,"uuid":"fab2ac2b-ed97-4cde-9825-b480eabf8b8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6825","nonce":2755130316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6827","nonce":2755130316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6831","nonce":2755130316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6829","nonce":2755130316}]},"public_addr":"172.21.15.174:6825/2755130316","cluster_addr":"172.21.15.174:6827/2755130316","heartbeat_back_addr":"172.21.15.174:6831/2755130316","heartbeat_front_addr":"172.21.15.174:6829/2755130316","state":["exists","up"]},{"osd":7,"uuid":"3de9b797-2ab6-4d41-a863-7e7d638fd703","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6817","nonce":2847141411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6819","nonce":2847141411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6823","nonce":2847141411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6821","nonce":2847141411}]},"public_addr":"172.21.15.174:6817/2847141411","cluster_addr":"172.21.15.174:6819/2847141411","heartbeat_back_addr":"172.21.15.174:6823/2847141411","heartbeat_front_addr":"172.21.15.174:6821/2847141411","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.477076+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.496890+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.539305+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.454673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.538881+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.521294+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.520712+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.540404+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:25:06.845 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.843+0000 7f35626a8700 1 -- 172.21.15.143:0/595509496 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f354806af00 msgr2=0x7f354806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:06.846 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.843+0000 7f35626a8700 1 --2- 172.21.15.143:0/595509496 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f354806af00 0x7f354806d3c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f355c1e5770 tx=0x7f3550009210 comp rx=0 tx=0).stop 2023-11-08T19:25:06.846 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.843+0000 7f35626a8700 1 -- 172.21.15.143:0/595509496 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f355c0b2920 msgr2=0x7f355c157040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:06.846 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.843+0000 7f35626a8700 1 --2- 172.21.15.143:0/595509496 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f355c0b2920 0x7f355c157040 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f354c00c430 tx=0x7f354c00c740 comp rx=0 tx=0).stop 2023-11-08T19:25:06.846 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.843+0000 7f35626a8700 1 -- 172.21.15.143:0/595509496 shutdown_connections 2023-11-08T19:25:06.847 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.843+0000 7f35626a8700 1 --2- 172.21.15.143:0/595509496 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f354806af00 0x7f354806d3c0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:06.847 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.843+0000 7f35626a8700 1 --2- 172.21.15.143:0/595509496 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f355c15aa30 0x7f355c0b52c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:06.847 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.843+0000 7f35626a8700 1 --2- 172.21.15.143:0/595509496 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f355c0b2920 0x7f355c157040 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:06.847 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.843+0000 7f35626a8700 1 --2- 172.21.15.143:0/595509496 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f355c0cbe20 0x7f355c1e3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:06.847 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.843+0000 7f35626a8700 1 -- 172.21.15.143:0/595509496 >> 172.21.15.143:0/595509496 conn(0x7f355c099ab0 msgr2=0x7f355c0b19b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:06.848 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.843+0000 7f35626a8700 1 -- 172.21.15.143:0/595509496 shutdown_connections 2023-11-08T19:25:06.848 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.843+0000 7f35626a8700 1 -- 172.21.15.143:0/595509496 wait complete. 2023-11-08T19:25:06.858 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T19:24:44.551530+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-08T19:24:48.256781+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-08T19:24:48.954990+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-08T19:25:06.858 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T19:25:06.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.979+0000 7f8c97f2b700 1 Processor -- start 2023-11-08T19:25:06.982 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.979+0000 7f8c97f2b700 1 -- start start 2023-11-08T19:25:06.988 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.980+0000 7f8c97f2b700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8c9015dba0 0x7f8c9006c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:06.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.980+0000 7f8c97f2b700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f8c900b22e0 0x7f8c900b2770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:06.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.980+0000 7f8c97f2b700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c90161890 con 0x7f8c9006b240 2023-11-08T19:25:06.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.980+0000 7f8c97f2b700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c900b2cb0 con 0x7f8c900b22e0 2023-11-08T19:25:06.990 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.980+0000 7f8c97f2b700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c9006b740 con 0x7f8c9015dba0 2023-11-08T19:25:06.990 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.980+0000 7f8c964c8700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f8c900b22e0 0x7f8c900b2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:06.990 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.980+0000 7f8c964c8700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f8c900b22e0 0x7f8c900b2770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49284/0 (socket says 172.21.15.143:49284) 2023-11-08T19:25:06.991 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.980+0000 7f8c964c8700 1 -- 172.21.15.143:0/3738556186 learned_addr learned my addr 172.21.15.143:0/3738556186 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:06.991 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.980+0000 7f8c95cc7700 1 --2- 172.21.15.143:0/3738556186 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8c9015dba0 0x7f8c9006c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:06.992 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.980+0000 7f8c964c8700 1 -- 172.21.15.143:0/3738556186 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8c9015dba0 msgr2=0x7f8c9006c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:06.992 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.980+0000 7f8c964c8700 1 --2- 172.21.15.143:0/3738556186 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8c9015dba0 0x7f8c9006c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:06.992 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.980+0000 7f8c964c8700 1 -- 172.21.15.143:0/3738556186 >> v1:172.21.15.174:6789/0 conn(0x7f8c9006b240 legacy=0x7f8c900caff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:06.993 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.980+0000 7f8c964c8700 1 -- 172.21.15.143:0/3738556186 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c9006ba20 con 0x7f8c900b22e0 2023-11-08T19:25:06.993 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.980+0000 7f8c964c8700 1 --2- 172.21.15.143:0/3738556186 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8c900b22e0 0x7f8c900b2770 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c0098f0 tx=0x7f8c8c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cac2ebd75b0ad344 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:06.994 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.980+0000 7f8c94cc5700 1 -- 172.21.15.143:0/3738556186 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c8c00b8a0 con 0x7f8c900b22e0 2023-11-08T19:25:06.994 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.980+0000 7f8c94cc5700 1 -- 172.21.15.143:0/3738556186 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c8c00ba00 con 0x7f8c900b22e0 2023-11-08T19:25:06.994 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.980+0000 7f8c94cc5700 1 -- 172.21.15.143:0/3738556186 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c8c00bd90 con 0x7f8c900b22e0 2023-11-08T19:25:06.995 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.981+0000 7f8c97f2b700 1 -- 172.21.15.143:0/3738556186 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8c900b22e0 msgr2=0x7f8c900b2770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:06.995 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.981+0000 7f8c97f2b700 1 --2- 172.21.15.143:0/3738556186 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8c900b22e0 0x7f8c900b2770 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c0098f0 tx=0x7f8c8c009c00 comp rx=0 tx=0).stop 2023-11-08T19:25:06.996 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.981+0000 7f8c97f2b700 1 -- 172.21.15.143:0/3738556186 shutdown_connections 2023-11-08T19:25:06.996 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.981+0000 7f8c97f2b700 1 --2- 172.21.15.143:0/3738556186 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8c900b22e0 0x7f8c900b2770 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:06.996 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.981+0000 7f8c97f2b700 1 --2- 172.21.15.143:0/3738556186 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8c9015dba0 0x7f8c9006c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:06.997 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.981+0000 7f8c97f2b700 1 -- 172.21.15.143:0/3738556186 >> 172.21.15.143:0/3738556186 conn(0x7f8c900cd2e0 msgr2=0x7f8c900cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:06.997 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.981+0000 7f8c97f2b700 1 -- 172.21.15.143:0/3738556186 shutdown_connections 2023-11-08T19:25:06.997 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.981+0000 7f8c97f2b700 1 -- 172.21.15.143:0/3738556186 wait complete. 2023-11-08T19:25:06.998 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.982+0000 7f8c97f2b700 1 Processor -- start 2023-11-08T19:25:06.998 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.982+0000 7f8c97f2b700 1 -- start start 2023-11-08T19:25:06.999 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.983+0000 7f8c97f2b700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8c9006b240 0x7f8c900c48f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:06.999 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.983+0000 7f8c97f2b700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8c900b22e0 0x7f8c900c5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:06.999 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.983+0000 7f8c97f2b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c9015dba0 0x7f8c900cc2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:07.000 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.983+0000 7f8c97f2b700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c900ca3f0 con 0x7f8c900b22e0 2023-11-08T19:25:07.000 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.983+0000 7f8c97f2b700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c900c4e30 con 0x7f8c9015dba0 2023-11-08T19:25:07.000 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.983+0000 7f8c97f2b700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c900aeea0 con 0x7f8c9006b240 2023-11-08T19:25:07.001 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.983+0000 7f8c954c6700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8c900b22e0 0x7f8c900c5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:07.001 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.983+0000 7f8c954c6700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8c900b22e0 0x7f8c900c5040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49288/0 (socket says 172.21.15.143:49288) 2023-11-08T19:25:07.002 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.983+0000 7f8c954c6700 1 -- 172.21.15.143:0/2808991038 learned_addr learned my addr 172.21.15.143:0/2808991038 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:07.002 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.983+0000 7f8c95cc7700 1 --2- 172.21.15.143:0/2808991038 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8c9006b240 0x7f8c900c48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:07.002 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.983+0000 7f8c964c8700 1 --2- 172.21.15.143:0/2808991038 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c9015dba0 0x7f8c900cc2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:07.003 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.983+0000 7f8c954c6700 1 -- 172.21.15.143:0/2808991038 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8c9006b240 msgr2=0x7f8c900c48f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:07.003 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.983+0000 7f8c954c6700 1 --2- 172.21.15.143:0/2808991038 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8c9006b240 0x7f8c900c48f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:07.003 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.983+0000 7f8c954c6700 1 -- 172.21.15.143:0/2808991038 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c9015dba0 msgr2=0x7f8c900cc2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:07.004 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.983+0000 7f8c954c6700 1 --2- 172.21.15.143:0/2808991038 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c9015dba0 0x7f8c900cc2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:07.004 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.983+0000 7f8c954c6700 1 -- 172.21.15.143:0/2808991038 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c8c009550 con 0x7f8c900b22e0 2023-11-08T19:25:07.004 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.983+0000 7f8c964c8700 1 --2- 172.21.15.143:0/2808991038 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c9015dba0 0x7f8c900cc2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:25:07.005 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.983+0000 7f8c954c6700 1 --2- 172.21.15.143:0/2808991038 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8c900b22e0 0x7f8c900c5040 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8c8400c4e0 tx=0x7f8c8400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:07.005 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.984+0000 7f8c827fc700 1 -- 172.21.15.143:0/2808991038 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c840052d0 con 0x7f8c900b22e0 2023-11-08T19:25:07.006 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.984+0000 7f8c827fc700 1 -- 172.21.15.143:0/2808991038 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c84005430 con 0x7f8c900b22e0 2023-11-08T19:25:07.006 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.984+0000 7f8c97f2b700 1 -- 172.21.15.143:0/2808991038 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c900af180 con 0x7f8c900b22e0 2023-11-08T19:25:07.006 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.984+0000 7f8c827fc700 1 -- 172.21.15.143:0/2808991038 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c840038c0 con 0x7f8c900b22e0 2023-11-08T19:25:07.007 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.984+0000 7f8c97f2b700 1 -- 172.21.15.143:0/2808991038 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c900af740 con 0x7f8c900b22e0 2023-11-08T19:25:07.007 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.985+0000 7f8c97f2b700 1 -- 172.21.15.143:0/2808991038 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c9004eee0 con 0x7f8c900b22e0 2023-11-08T19:25:07.007 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.985+0000 7f8c827fc700 1 -- 172.21.15.143:0/2808991038 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8c84003a20 con 0x7f8c900b22e0 2023-11-08T19:25:07.008 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.985+0000 7f8c827fc700 1 --2- 172.21.15.143:0/2808991038 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f8c7406af00 0x7f8c7406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:07.008 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.986+0000 7f8c827fc700 1 -- 172.21.15.143:0/2808991038 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8c84088ab0 con 0x7f8c900b22e0 2023-11-08T19:25:07.009 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.986+0000 7f8c95cc7700 1 --2- 172.21.15.143:0/2808991038 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f8c7406af00 0x7f8c7406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:07.009 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.986+0000 7f8c95cc7700 1 --2- 172.21.15.143:0/2808991038 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f8c7406af00 0x7f8c7406d3c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f8c88009f60 tx=0x7f8c8800b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:07.010 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:06.990+0000 7f8c827fc700 1 -- 172.21.15.143:0/2808991038 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8c84053b60 con 0x7f8c900b22e0 2023-11-08T19:25:07.178 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.175+0000 7f8c97f2b700 1 -- 172.21.15.143:0/2808991038 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8c900b67e0 con 0x7f8c900b22e0 2023-11-08T19:25:07.178 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.175+0000 7f8c827fc700 1 -- 172.21.15.143:0/2808991038 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8c840170a0 con 0x7f8c900b22e0 2023-11-08T19:25:07.178 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2023-11-08T19:25:07.181 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.178+0000 7f8c97f2b700 1 -- 172.21.15.143:0/2808991038 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f8c7406af00 msgr2=0x7f8c7406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:07.181 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.178+0000 7f8c97f2b700 1 --2- 172.21.15.143:0/2808991038 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f8c7406af00 0x7f8c7406d3c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f8c88009f60 tx=0x7f8c8800b040 comp rx=0 tx=0).stop 2023-11-08T19:25:07.181 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.178+0000 7f8c97f2b700 1 -- 172.21.15.143:0/2808991038 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8c900b22e0 msgr2=0x7f8c900c5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:07.181 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.178+0000 7f8c97f2b700 1 --2- 172.21.15.143:0/2808991038 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8c900b22e0 0x7f8c900c5040 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8c8400c4e0 tx=0x7f8c8400c8a0 comp rx=0 tx=0).stop 2023-11-08T19:25:07.182 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.178+0000 7f8c97f2b700 1 -- 172.21.15.143:0/2808991038 shutdown_connections 2023-11-08T19:25:07.182 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.178+0000 7f8c97f2b700 1 --2- 172.21.15.143:0/2808991038 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f8c7406af00 0x7f8c7406d3c0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:07.182 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.178+0000 7f8c97f2b700 1 --2- 172.21.15.143:0/2808991038 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8c9006b240 0x7f8c900c48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:07.182 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.178+0000 7f8c97f2b700 1 --2- 172.21.15.143:0/2808991038 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c9015dba0 0x7f8c900cc2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:07.182 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.178+0000 7f8c97f2b700 1 --2- 172.21.15.143:0/2808991038 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8c900b22e0 0x7f8c900c5040 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:07.183 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.178+0000 7f8c97f2b700 1 -- 172.21.15.143:0/2808991038 >> 172.21.15.143:0/2808991038 conn(0x7f8c900cd2e0 msgr2=0x7f8c900b02c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:07.183 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.178+0000 7f8c97f2b700 1 -- 172.21.15.143:0/2808991038 shutdown_connections 2023-11-08T19:25:07.183 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.178+0000 7f8c97f2b700 1 -- 172.21.15.143:0/2808991038 wait complete. 2023-11-08T19:25:07.192 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-08T19:25:07.316 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.314+0000 7fb87a8e4700 1 Processor -- start 2023-11-08T19:25:07.316 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.314+0000 7fb87a8e4700 1 -- start start 2023-11-08T19:25:07.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.314+0000 7fb87a8e4700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fb87406b240 0x7fb8740caff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:07.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.314+0000 7fb87a8e4700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb8740cb530 0x7fb8740b22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:07.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.314+0000 7fb87a8e4700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb874161890 con 0x7fb87415dba0 2023-11-08T19:25:07.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.314+0000 7fb87a8e4700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb87406cd20 con 0x7fb87406b240 2023-11-08T19:25:07.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.314+0000 7fb87a8e4700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb87406b740 con 0x7fb8740cb530 2023-11-08T19:25:07.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.315+0000 7fb878e81700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb8740cb530 0x7fb8740b22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:07.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.315+0000 7fb8737fe700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fb87406b240 0x7fb8740caff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:07.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.315+0000 7fb8737fe700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fb87406b240 0x7fb8740caff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.143:32794/0 (socket says 172.21.15.143:32794) 2023-11-08T19:25:07.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.315+0000 7fb878e81700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb8740cb530 0x7fb8740b22e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.143:36936/0 (socket says 172.21.15.143:36936) 2023-11-08T19:25:07.321 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.315+0000 7fb873fff700 1 --1- >> v1:172.21.15.174:6789/0 conn(0x7fb87415dba0 0x7fb87406c610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.174:6789/0 says I am v1:172.21.15.143:56382/0 (socket says 172.21.15.143:56382) 2023-11-08T19:25:07.321 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.315+0000 7fb878e81700 1 -- 172.21.15.143:0/4071697863 learned_addr learned my addr 172.21.15.143:0/4071697863 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:07.321 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.315+0000 7fb878e81700 1 -- 172.21.15.143:0/4071697863 >> v2:172.21.15.174:3300/0 conn(0x7fb87406b240 msgr2=0x7fb8740caff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:07.321 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.315+0000 7fb878e81700 1 --2- 172.21.15.143:0/4071697863 >> v2:172.21.15.174:3300/0 conn(0x7fb87406b240 0x7fb8740caff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:07.321 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.315+0000 7fb878e81700 1 -- 172.21.15.143:0/4071697863 >> v1:172.21.15.174:6789/0 conn(0x7fb87415dba0 legacy=0x7fb87406c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:07.322 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.315+0000 7fb878e81700 1 -- 172.21.15.143:0/4071697863 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb87406ba20 con 0x7fb8740cb530 2023-11-08T19:25:07.322 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.315+0000 7fb8737fe700 1 --2- 172.21.15.143:0/4071697863 >> v2:172.21.15.174:3300/0 conn(0x7fb87406b240 0x7fb8740caff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T19:25:07.322 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.315+0000 7fb878e81700 1 --2- 172.21.15.143:0/4071697863 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb8740cb530 0x7fb8740b22e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb86c009e50 tx=0x7fb86c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e563010b782aed41 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:07.322 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.315+0000 7fb872ffd700 1 -- 172.21.15.143:0/4071697863 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb86c00e070 con 0x7fb8740cb530 2023-11-08T19:25:07.322 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.315+0000 7fb872ffd700 1 -- 172.21.15.143:0/4071697863 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb86c00b910 con 0x7fb8740cb530 2023-11-08T19:25:07.323 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.315+0000 7fb872ffd700 1 -- 172.21.15.143:0/4071697863 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb86c00bcc0 con 0x7fb8740cb530 2023-11-08T19:25:07.323 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.316+0000 7fb87a8e4700 1 -- 172.21.15.143:0/4071697863 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb8740cb530 msgr2=0x7fb8740b22e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:07.323 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.316+0000 7fb87a8e4700 1 --2- 172.21.15.143:0/4071697863 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb8740cb530 0x7fb8740b22e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb86c009e50 tx=0x7fb86c00b330 comp rx=0 tx=0).stop 2023-11-08T19:25:07.323 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.316+0000 7fb87a8e4700 1 -- 172.21.15.143:0/4071697863 shutdown_connections 2023-11-08T19:25:07.323 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.316+0000 7fb87a8e4700 1 --2- 172.21.15.143:0/4071697863 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb8740cb530 0x7fb8740b22e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:07.324 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.316+0000 7fb87a8e4700 1 --2- 172.21.15.143:0/4071697863 >> v2:172.21.15.174:3300/0 conn(0x7fb87406b240 0x7fb8740caff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:07.324 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.316+0000 7fb87a8e4700 1 -- 172.21.15.143:0/4071697863 >> 172.21.15.143:0/4071697863 conn(0x7fb8740cd2e0 msgr2=0x7fb8740cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:07.324 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.316+0000 7fb87a8e4700 1 -- 172.21.15.143:0/4071697863 shutdown_connections 2023-11-08T19:25:07.324 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.316+0000 7fb87a8e4700 1 -- 172.21.15.143:0/4071697863 wait complete. 2023-11-08T19:25:07.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.317+0000 7fb87a8e4700 1 Processor -- start 2023-11-08T19:25:07.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.318+0000 7fb87a8e4700 1 -- start start 2023-11-08T19:25:07.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.318+0000 7fb87a8e4700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb87406b240 0x7fb87415e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:07.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.319+0000 7fb87a8e4700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb8740cb530 0x7fb87415b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:07.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.319+0000 7fb87a8e4700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb87415dba0 0x7fb8740c8e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:07.329 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.319+0000 7fb87a8e4700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8740c9cd0 con 0x7fb87406b240 2023-11-08T19:25:07.329 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.319+0000 7fb87a8e4700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8740c87b0 con 0x7fb8740cb530 2023-11-08T19:25:07.329 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.319+0000 7fb87a8e4700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8740c99f0 con 0x7fb87415dba0 2023-11-08T19:25:07.329 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.319+0000 7fb873fff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb87406b240 0x7fb87415e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:07.329 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.319+0000 7fb873fff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb87406b240 0x7fb87415e2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49300/0 (socket says 172.21.15.143:49300) 2023-11-08T19:25:07.330 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.319+0000 7fb873fff700 1 -- 172.21.15.143:0/3501519273 learned_addr learned my addr 172.21.15.143:0/3501519273 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:07.330 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.319+0000 7fb878e81700 1 --2- 172.21.15.143:0/3501519273 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb87415dba0 0x7fb8740c8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:07.330 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.319+0000 7fb8737fe700 1 --2- 172.21.15.143:0/3501519273 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb8740cb530 0x7fb87415b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:07.330 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.319+0000 7fb873fff700 1 -- 172.21.15.143:0/3501519273 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb87415dba0 msgr2=0x7fb8740c8e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:07.330 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.319+0000 7fb873fff700 1 --2- 172.21.15.143:0/3501519273 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb87415dba0 0x7fb8740c8e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:07.331 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.319+0000 7fb873fff700 1 -- 172.21.15.143:0/3501519273 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb8740cb530 msgr2=0x7fb87415b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:07.331 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.319+0000 7fb873fff700 1 --2- 172.21.15.143:0/3501519273 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb8740cb530 0x7fb87415b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:07.331 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.319+0000 7fb873fff700 1 -- 172.21.15.143:0/3501519273 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8640094a0 con 0x7fb87406b240 2023-11-08T19:25:07.331 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.319+0000 7fb8737fe700 1 --2- 172.21.15.143:0/3501519273 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb8740cb530 0x7fb87415b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:25:07.331 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.319+0000 7fb878e81700 1 --2- 172.21.15.143:0/3501519273 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb87415dba0 0x7fb8740c8e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:25:07.332 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.319+0000 7fb873fff700 1 --2- 172.21.15.143:0/3501519273 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb87406b240 0x7fb87415e2e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb86800c500 tx=0x7fb86800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:07.332 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.319+0000 7fb870ff9700 1 -- 172.21.15.143:0/3501519273 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb868005cb0 con 0x7fb87406b240 2023-11-08T19:25:07.332 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.319+0000 7fb870ff9700 1 -- 172.21.15.143:0/3501519273 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb868005e10 con 0x7fb87406b240 2023-11-08T19:25:07.332 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.319+0000 7fb87a8e4700 1 -- 172.21.15.143:0/3501519273 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb86c009b30 con 0x7fb87406b240 2023-11-08T19:25:07.332 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.319+0000 7fb870ff9700 1 -- 172.21.15.143:0/3501519273 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8680072d0 con 0x7fb87406b240 2023-11-08T19:25:07.332 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.319+0000 7fb87a8e4700 1 -- 172.21.15.143:0/3501519273 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8741e6e20 con 0x7fb87406b240 2023-11-08T19:25:07.333 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.320+0000 7fb87a8e4700 1 -- 172.21.15.143:0/3501519273 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb87404eee0 con 0x7fb87406b240 2023-11-08T19:25:07.333 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.321+0000 7fb870ff9700 1 -- 172.21.15.143:0/3501519273 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb868007430 con 0x7fb87406b240 2023-11-08T19:25:07.333 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.321+0000 7fb870ff9700 1 --2- 172.21.15.143:0/3501519273 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fb86006af00 0x7fb86006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:07.333 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.321+0000 7fb870ff9700 1 -- 172.21.15.143:0/3501519273 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb868088880 con 0x7fb87406b240 2023-11-08T19:25:07.333 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.321+0000 7fb8737fe700 1 --2- 172.21.15.143:0/3501519273 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fb86006af00 0x7fb86006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:07.334 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.322+0000 7fb8737fe700 1 --2- 172.21.15.143:0/3501519273 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fb86006af00 0x7fb86006d3c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb8740b53f0 tx=0x7fb864008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:07.334 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.326+0000 7fb870ff9700 1 -- 172.21.15.143:0/3501519273 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb868053a70 con 0x7fb87406b240 2023-11-08T19:25:07.520 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.518+0000 7fb87a8e4700 1 -- 172.21.15.143:0/3501519273 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb8740b67e0 con 0x7fb87406b240 2023-11-08T19:25:07.521 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.518+0000 7fb870ff9700 1 -- 172.21.15.143:0/3501519273 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb8680170a0 con 0x7fb87406b240 2023-11-08T19:25:07.521 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2023-11-08T19:25:07.523 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.520+0000 7fb87a8e4700 1 -- 172.21.15.143:0/3501519273 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fb86006af00 msgr2=0x7fb86006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:07.523 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.520+0000 7fb87a8e4700 1 --2- 172.21.15.143:0/3501519273 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fb86006af00 0x7fb86006d3c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb8740b53f0 tx=0x7fb864008040 comp rx=0 tx=0).stop 2023-11-08T19:25:07.523 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.520+0000 7fb87a8e4700 1 -- 172.21.15.143:0/3501519273 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb87406b240 msgr2=0x7fb87415e2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:07.524 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.520+0000 7fb87a8e4700 1 --2- 172.21.15.143:0/3501519273 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb87406b240 0x7fb87415e2e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb86800c500 tx=0x7fb86800c8c0 comp rx=0 tx=0).stop 2023-11-08T19:25:07.524 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.520+0000 7fb87a8e4700 1 -- 172.21.15.143:0/3501519273 shutdown_connections 2023-11-08T19:25:07.524 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.520+0000 7fb87a8e4700 1 --2- 172.21.15.143:0/3501519273 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fb86006af00 0x7fb86006d3c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:07.524 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.520+0000 7fb87a8e4700 1 --2- 172.21.15.143:0/3501519273 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb87415dba0 0x7fb8740c8e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:07.524 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.520+0000 7fb87a8e4700 1 --2- 172.21.15.143:0/3501519273 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb87406b240 0x7fb87415e2e0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:07.525 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.520+0000 7fb87a8e4700 1 --2- 172.21.15.143:0/3501519273 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb8740cb530 0x7fb87415b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:07.525 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.520+0000 7fb87a8e4700 1 -- 172.21.15.143:0/3501519273 >> 172.21.15.143:0/3501519273 conn(0x7fb8740cd2e0 msgr2=0x7fb87415cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:07.525 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.520+0000 7fb87a8e4700 1 -- 172.21.15.143:0/3501519273 shutdown_connections 2023-11-08T19:25:07.525 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.520+0000 7fb87a8e4700 1 -- 172.21.15.143:0/3501519273 wait complete. 2023-11-08T19:25:07.534 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-08T19:25:07.657 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.654+0000 7fb057b3a700 1 Processor -- start 2023-11-08T19:25:07.657 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.655+0000 7fb057b3a700 1 -- start start 2023-11-08T19:25:07.659 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.655+0000 7fb057b3a700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fb0500d1d80 0x7fb0500b2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:07.660 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.655+0000 7fb057b3a700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb0500b1730 0x7fb0500cc4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:07.660 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.655+0000 7fb057b3a700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0500d0420 con 0x7fb05015bb20 2023-11-08T19:25:07.660 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.655+0000 7fb057b3a700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0500cf600 con 0x7fb0500b1730 2023-11-08T19:25:07.660 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.655+0000 7fb057b3a700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0500cf810 con 0x7fb0500d1d80 2023-11-08T19:25:07.661 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.655+0000 7fb0560d7700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb0500b1730 0x7fb0500cc4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:07.661 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.656+0000 7fb0560d7700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb0500b1730 0x7fb0500cc4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49312/0 (socket says 172.21.15.143:49312) 2023-11-08T19:25:07.661 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.656+0000 7fb0560d7700 1 -- 172.21.15.143:0/2983518184 learned_addr learned my addr 172.21.15.143:0/2983518184 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:07.661 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.656+0000 7fb0550d5700 1 --2- 172.21.15.143:0/2983518184 >> v2:172.21.15.174:3300/0 conn(0x7fb0500d1d80 0x7fb0500b2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:07.662 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.656+0000 7fb0560d7700 1 -- 172.21.15.143:0/2983518184 >> v2:172.21.15.174:3300/0 conn(0x7fb0500d1d80 msgr2=0x7fb0500b2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:07.662 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.656+0000 7fb0560d7700 1 --2- 172.21.15.143:0/2983518184 >> v2:172.21.15.174:3300/0 conn(0x7fb0500d1d80 0x7fb0500b2670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:07.662 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.656+0000 7fb0560d7700 1 -- 172.21.15.143:0/2983518184 >> v1:172.21.15.174:6789/0 conn(0x7fb05015bb20 legacy=0x7fb0500d1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:07.662 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.656+0000 7fb0560d7700 1 -- 172.21.15.143:0/2983518184 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0500cfaf0 con 0x7fb0500b1730 2023-11-08T19:25:07.662 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.656+0000 7fb0560d7700 1 --2- 172.21.15.143:0/2983518184 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb0500b1730 0x7fb0500cc4d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb05015f5f0 tx=0x7fb04800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4d28069fc92234f server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:07.663 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.656+0000 7fb0548d4700 1 -- 172.21.15.143:0/2983518184 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb04800e070 con 0x7fb0500b1730 2023-11-08T19:25:07.663 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.656+0000 7fb0548d4700 1 -- 172.21.15.143:0/2983518184 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb048003a20 con 0x7fb0500b1730 2023-11-08T19:25:07.663 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.656+0000 7fb0548d4700 1 -- 172.21.15.143:0/2983518184 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb048003dd0 con 0x7fb0500b1730 2023-11-08T19:25:07.663 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.657+0000 7fb057b3a700 1 -- 172.21.15.143:0/2983518184 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb0500b1730 msgr2=0x7fb0500cc4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:07.663 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.657+0000 7fb057b3a700 1 --2- 172.21.15.143:0/2983518184 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb0500b1730 0x7fb0500cc4d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb05015f5f0 tx=0x7fb04800b330 comp rx=0 tx=0).stop 2023-11-08T19:25:07.664 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.657+0000 7fb057b3a700 1 -- 172.21.15.143:0/2983518184 shutdown_connections 2023-11-08T19:25:07.664 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.657+0000 7fb057b3a700 1 --2- 172.21.15.143:0/2983518184 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb0500b1730 0x7fb0500cc4d0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:07.664 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.657+0000 7fb057b3a700 1 --2- 172.21.15.143:0/2983518184 >> v2:172.21.15.174:3300/0 conn(0x7fb0500d1d80 0x7fb0500b2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:07.664 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.657+0000 7fb057b3a700 1 -- 172.21.15.143:0/2983518184 >> 172.21.15.143:0/2983518184 conn(0x7fb05008b000 msgr2=0x7fb05009ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:07.664 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.657+0000 7fb057b3a700 1 -- 172.21.15.143:0/2983518184 shutdown_connections 2023-11-08T19:25:07.664 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.657+0000 7fb057b3a700 1 -- 172.21.15.143:0/2983518184 wait complete. 2023-11-08T19:25:07.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.658+0000 7fb057b3a700 1 Processor -- start 2023-11-08T19:25:07.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.658+0000 7fb057b3a700 1 -- start start 2023-11-08T19:25:07.668 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.659+0000 7fb057b3a700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0500b1730 0x7fb0500ceeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:07.668 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.659+0000 7fb057b3a700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb0500d1d80 0x7fb0501f47e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:07.668 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.659+0000 7fb057b3a700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb05015bb20 0x7fb0501f1b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:07.668 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.659+0000 7fb057b3a700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0500cf3f0 con 0x7fb0500d1d80 2023-11-08T19:25:07.669 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.659+0000 7fb057b3a700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0501f50e0 con 0x7fb0500b1730 2023-11-08T19:25:07.669 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.659+0000 7fb057b3a700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0501f53c0 con 0x7fb05015bb20 2023-11-08T19:25:07.669 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.659+0000 7fb0550d5700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb0500d1d80 0x7fb0501f47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:07.669 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.659+0000 7fb0550d5700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb0500d1d80 0x7fb0501f47e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49316/0 (socket says 172.21.15.143:49316) 2023-11-08T19:25:07.669 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.659+0000 7fb0550d5700 1 -- 172.21.15.143:0/921795468 learned_addr learned my addr 172.21.15.143:0/921795468 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:07.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.659+0000 7fb0560d7700 1 --2- 172.21.15.143:0/921795468 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb05015bb20 0x7fb0501f1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:07.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.659+0000 7fb0558d6700 1 --2- 172.21.15.143:0/921795468 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0500b1730 0x7fb0500ceeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:07.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.659+0000 7fb0550d5700 1 -- 172.21.15.143:0/921795468 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb05015bb20 msgr2=0x7fb0501f1b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:07.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.659+0000 7fb0550d5700 1 --2- 172.21.15.143:0/921795468 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb05015bb20 0x7fb0501f1b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:07.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.659+0000 7fb0550d5700 1 -- 172.21.15.143:0/921795468 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0500b1730 msgr2=0x7fb0500ceeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:07.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.659+0000 7fb0550d5700 1 --2- 172.21.15.143:0/921795468 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0500b1730 0x7fb0500ceeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:07.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.659+0000 7fb0550d5700 1 -- 172.21.15.143:0/921795468 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb048009b30 con 0x7fb0500d1d80 2023-11-08T19:25:07.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.659+0000 7fb0560d7700 1 --2- 172.21.15.143:0/921795468 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb05015bb20 0x7fb0501f1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:25:07.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.659+0000 7fb0550d5700 1 --2- 172.21.15.143:0/921795468 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb0500d1d80 0x7fb0501f47e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb04c00c7f0 tx=0x7fb04c00cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:07.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.659+0000 7fb0427fc700 1 -- 172.21.15.143:0/921795468 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb04c011800 con 0x7fb0500d1d80 2023-11-08T19:25:07.672 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.659+0000 7fb0427fc700 1 -- 172.21.15.143:0/921795468 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb04c011960 con 0x7fb0500d1d80 2023-11-08T19:25:07.672 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.659+0000 7fb0427fc700 1 -- 172.21.15.143:0/921795468 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb04c011cf0 con 0x7fb0500d1d80 2023-11-08T19:25:07.672 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.659+0000 7fb057b3a700 1 -- 172.21.15.143:0/921795468 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0501f56a0 con 0x7fb0500d1d80 2023-11-08T19:25:07.672 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.659+0000 7fb057b3a700 1 -- 172.21.15.143:0/921795468 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0501f15e0 con 0x7fb0500d1d80 2023-11-08T19:25:07.672 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.660+0000 7fb057b3a700 1 -- 172.21.15.143:0/921795468 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0501f1800 con 0x7fb0500d1d80 2023-11-08T19:25:07.672 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.661+0000 7fb0427fc700 1 -- 172.21.15.143:0/921795468 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb04c018360 con 0x7fb0500d1d80 2023-11-08T19:25:07.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.661+0000 7fb0427fc700 1 --2- 172.21.15.143:0/921795468 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fb03406af00 0x7fb03406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:07.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.661+0000 7fb0427fc700 1 -- 172.21.15.143:0/921795468 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb04c089610 con 0x7fb0500d1d80 2023-11-08T19:25:07.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.661+0000 7fb0558d6700 1 --2- 172.21.15.143:0/921795468 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fb03406af00 0x7fb03406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:07.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.662+0000 7fb0558d6700 1 --2- 172.21.15.143:0/921795468 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fb03406af00 0x7fb03406d3c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb05015a970 tx=0x7fb044008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:07.674 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.666+0000 7fb0427fc700 1 -- 172.21.15.143:0/921795468 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb04c0546d0 con 0x7fb0500d1d80 2023-11-08T19:25:07.857 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.855+0000 7fb057b3a700 1 -- 172.21.15.143:0/921795468 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb05004eee0 con 0x7fb0500d1d80 2023-11-08T19:25:07.858 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.855+0000 7fb0427fc700 1 -- 172.21.15.143:0/921795468 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb04c057cf0 con 0x7fb0500d1d80 2023-11-08T19:25:07.858 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 64 2023-11-08T19:25:07.860 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.857+0000 7fb057b3a700 1 -- 172.21.15.143:0/921795468 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fb03406af00 msgr2=0x7fb03406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:07.860 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.857+0000 7fb057b3a700 1 --2- 172.21.15.143:0/921795468 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fb03406af00 0x7fb03406d3c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb05015a970 tx=0x7fb044008040 comp rx=0 tx=0).stop 2023-11-08T19:25:07.860 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.857+0000 7fb057b3a700 1 -- 172.21.15.143:0/921795468 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb0500d1d80 msgr2=0x7fb0501f47e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:07.861 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.857+0000 7fb057b3a700 1 --2- 172.21.15.143:0/921795468 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb0500d1d80 0x7fb0501f47e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb04c00c7f0 tx=0x7fb04c00cbb0 comp rx=0 tx=0).stop 2023-11-08T19:25:07.861 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.857+0000 7fb057b3a700 1 -- 172.21.15.143:0/921795468 shutdown_connections 2023-11-08T19:25:07.861 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.857+0000 7fb057b3a700 1 --2- 172.21.15.143:0/921795468 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fb03406af00 0x7fb03406d3c0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:07.861 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.857+0000 7fb057b3a700 1 --2- 172.21.15.143:0/921795468 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb05015bb20 0x7fb0501f1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:07.861 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.857+0000 7fb057b3a700 1 --2- 172.21.15.143:0/921795468 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0500b1730 0x7fb0500ceeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:07.862 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.857+0000 7fb057b3a700 1 --2- 172.21.15.143:0/921795468 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb0500d1d80 0x7fb0501f47e0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:07.862 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.857+0000 7fb057b3a700 1 -- 172.21.15.143:0/921795468 >> 172.21.15.143:0/921795468 conn(0x7fb05008b000 msgr2=0x7fb050070a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:07.862 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.857+0000 7fb057b3a700 1 -- 172.21.15.143:0/921795468 shutdown_connections 2023-11-08T19:25:07.862 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.857+0000 7fb057b3a700 1 -- 172.21.15.143:0/921795468 wait complete. 2023-11-08T19:25:07.872 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:25:07.992 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.990+0000 7f4bfaa15700 1 Processor -- start 2023-11-08T19:25:07.993 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.990+0000 7f4bfaa15700 1 -- start start 2023-11-08T19:25:07.995 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.991+0000 7f4bfaa15700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4bf415ba30 0x7f4bf40b14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:07.996 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.991+0000 7f4bfaa15700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f4bf40d1c70 0x7f4bf40cc5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:07.996 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.991+0000 7f4bfaa15700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bf40b1bc0 con 0x7f4bf40d03a0 2023-11-08T19:25:07.996 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.991+0000 7f4bfaa15700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bf40b1ea0 con 0x7f4bf40d1c70 2023-11-08T19:25:07.996 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.991+0000 7f4bfaa15700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf40b2180 con 0x7f4bf415ba30 2023-11-08T19:25:07.997 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.991+0000 7f4bf8fb2700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f4bf40d1c70 0x7f4bf40cc5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:07.997 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.991+0000 7f4bf37fe700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f4bf40d03a0 0x7f4bf40d1560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:45478/0 (socket says 172.21.15.143:45478) 2023-11-08T19:25:07.997 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.991+0000 7f4bf37fe700 1 -- 172.21.15.143:0/1309969009 learned_addr learned my addr 172.21.15.143:0/1309969009 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:25:07.997 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.991+0000 7f4bf8fb2700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f4bf40d1c70 0x7f4bf40cc5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49326/0 (socket says 172.21.15.143:49326) 2023-11-08T19:25:07.997 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.991+0000 7f4bf3fff700 1 --2- 172.21.15.143:0/1309969009 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4bf415ba30 0x7f4bf40b14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:07.998 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.991+0000 7f4bf8fb2700 1 -- 172.21.15.143:0/1309969009 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4bf415ba30 msgr2=0x7f4bf40b14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:07.998 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.991+0000 7f4bf8fb2700 1 --2- 172.21.15.143:0/1309969009 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4bf415ba30 0x7f4bf40b14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:07.998 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.991+0000 7f4bf8fb2700 1 -- 172.21.15.143:0/1309969009 >> v1:172.21.15.143:6789/0 conn(0x7f4bf40d03a0 legacy=0x7f4bf40d1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:07.998 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.991+0000 7f4bf2ffd700 1 -- 172.21.15.143:0/1309969009 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 656806040 0 0) 0x7f4bf40b1bc0 con 0x7f4bf40d03a0 2023-11-08T19:25:07.998 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.991+0000 7f4bf8fb2700 1 -- 172.21.15.143:0/1309969009 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bf40b2460 con 0x7f4bf40d1c70 2023-11-08T19:25:07.999 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.991+0000 7f4bf3fff700 1 --2- 172.21.15.143:0/1309969009 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4bf415ba30 0x7f4bf40b14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:25:07.999 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.991+0000 7f4bf8fb2700 1 --2- 172.21.15.143:0/1309969009 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4bf40d1c70 0x7f4bf40cc5a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4bec009d90 tx=0x7f4bec00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=748eca7642e2e98f server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:07.999 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.992+0000 7f4bf2ffd700 1 -- 172.21.15.143:0/1309969009 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bec00e070 con 0x7f4bf40d1c70 2023-11-08T19:25:07.999 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.992+0000 7f4bf2ffd700 1 -- 172.21.15.143:0/1309969009 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bec00bbd0 con 0x7f4bf40d1c70 2023-11-08T19:25:07.999 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.992+0000 7f4bf2ffd700 1 -- 172.21.15.143:0/1309969009 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bec012430 con 0x7f4bf40d1c70 2023-11-08T19:25:07.999 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.992+0000 7f4bfaa15700 1 -- 172.21.15.143:0/1309969009 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4bf40d1c70 msgr2=0x7f4bf40cc5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:08.000 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.992+0000 7f4bfaa15700 1 --2- 172.21.15.143:0/1309969009 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4bf40d1c70 0x7f4bf40cc5a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4bec009d90 tx=0x7f4bec00b330 comp rx=0 tx=0).stop 2023-11-08T19:25:08.000 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.992+0000 7f4bfaa15700 1 -- 172.21.15.143:0/1309969009 shutdown_connections 2023-11-08T19:25:08.000 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.992+0000 7f4bfaa15700 1 --2- 172.21.15.143:0/1309969009 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4bf40d1c70 0x7f4bf40cc5a0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:08.000 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.992+0000 7f4bfaa15700 1 --2- 172.21.15.143:0/1309969009 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4bf415ba30 0x7f4bf40b14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:08.000 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.992+0000 7f4bfaa15700 1 -- 172.21.15.143:0/1309969009 >> 172.21.15.143:0/1309969009 conn(0x7f4bf408b000 msgr2=0x7f4bf409ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:08.001 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.992+0000 7f4bfaa15700 1 -- 172.21.15.143:0/1309969009 shutdown_connections 2023-11-08T19:25:08.001 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.992+0000 7f4bfaa15700 1 -- 172.21.15.143:0/1309969009 wait complete. 2023-11-08T19:25:08.001 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.993+0000 7f4bfaa15700 1 Processor -- start 2023-11-08T19:25:08.001 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.994+0000 7f4bfaa15700 1 -- start start 2023-11-08T19:25:08.004 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.994+0000 7f4bfaa15700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4bf40d03a0 0x7f4bf4141940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:08.004 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.994+0000 7f4bfaa15700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4bf40d1c70 0x7f4bf4155980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:08.004 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.994+0000 7f4bfaa15700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4bf415ba30 0x7f4bf4155ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:08.005 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.994+0000 7f4bfaa15700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf40cd290 con 0x7f4bf415ba30 2023-11-08T19:25:08.005 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.994+0000 7f4bfaa15700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf40cde10 con 0x7f4bf40d1c70 2023-11-08T19:25:08.005 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.994+0000 7f4bfaa15700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf40cd470 con 0x7f4bf40d03a0 2023-11-08T19:25:08.005 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.994+0000 7f4bf8fb2700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4bf415ba30 0x7f4bf4155ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:08.005 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.994+0000 7f4bf8fb2700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4bf415ba30 0x7f4bf4155ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49332/0 (socket says 172.21.15.143:49332) 2023-11-08T19:25:08.006 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.994+0000 7f4bf8fb2700 1 -- 172.21.15.143:0/3435421378 learned_addr learned my addr 172.21.15.143:0/3435421378 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:08.006 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.995+0000 7f4bf37fe700 1 --2- 172.21.15.143:0/3435421378 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4bf40d1c70 0x7f4bf4155980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:08.006 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.995+0000 7f4bf3fff700 1 --2- 172.21.15.143:0/3435421378 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4bf40d03a0 0x7f4bf4141940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:08.006 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.995+0000 7f4bf8fb2700 1 -- 172.21.15.143:0/3435421378 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4bf40d03a0 msgr2=0x7f4bf4141940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:08.006 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.995+0000 7f4bf8fb2700 1 --2- 172.21.15.143:0/3435421378 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4bf40d03a0 0x7f4bf4141940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:08.007 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.995+0000 7f4bf8fb2700 1 -- 172.21.15.143:0/3435421378 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4bf40d1c70 msgr2=0x7f4bf4155980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:08.007 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.995+0000 7f4bf8fb2700 1 --2- 172.21.15.143:0/3435421378 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4bf40d1c70 0x7f4bf4155980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:08.007 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.995+0000 7f4bf8fb2700 1 -- 172.21.15.143:0/3435421378 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bec009a70 con 0x7f4bf415ba30 2023-11-08T19:25:08.007 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.995+0000 7f4bf8fb2700 1 --2- 172.21.15.143:0/3435421378 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4bf415ba30 0x7f4bf4155ec0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4bf408c000 tx=0x7f4bec004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:08.007 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.995+0000 7f4bf0ff9700 1 -- 172.21.15.143:0/3435421378 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bec003ea0 con 0x7f4bf415ba30 2023-11-08T19:25:08.008 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.995+0000 7f4bf0ff9700 1 -- 172.21.15.143:0/3435421378 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bec020070 con 0x7f4bf415ba30 2023-11-08T19:25:08.008 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.995+0000 7f4bf0ff9700 1 -- 172.21.15.143:0/3435421378 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bec004280 con 0x7f4bf415ba30 2023-11-08T19:25:08.008 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.995+0000 7f4bfaa15700 1 -- 172.21.15.143:0/3435421378 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bf40cd750 con 0x7f4bf415ba30 2023-11-08T19:25:08.008 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.995+0000 7f4bfaa15700 1 -- 172.21.15.143:0/3435421378 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bf4141e80 con 0x7f4bf415ba30 2023-11-08T19:25:08.008 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.996+0000 7f4bfaa15700 1 -- 172.21.15.143:0/3435421378 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bf40cdf50 con 0x7f4bf415ba30 2023-11-08T19:25:08.008 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.996+0000 7f4bf0ff9700 1 -- 172.21.15.143:0/3435421378 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4bec0043e0 con 0x7f4bf415ba30 2023-11-08T19:25:08.009 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.997+0000 7f4bf0ff9700 1 --2- 172.21.15.143:0/3435421378 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f4be006af00 0x7f4be006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:08.009 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.997+0000 7f4bf0ff9700 1 -- 172.21.15.143:0/3435421378 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4bec022070 con 0x7f4bf415ba30 2023-11-08T19:25:08.009 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.997+0000 7f4bf3fff700 1 --2- 172.21.15.143:0/3435421378 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f4be006af00 0x7f4be006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:08.009 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:07.997+0000 7f4bf3fff700 1 --2- 172.21.15.143:0/3435421378 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f4be006af00 0x7f4be006d3c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f4be8006fd0 tx=0x7f4be8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:08.009 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.001+0000 7f4bf0ff9700 1 -- 172.21.15.143:0/3435421378 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4bec056470 con 0x7f4bf415ba30 2023-11-08T19:25:08.235 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.233+0000 7f4bfaa15700 1 -- 172.21.15.143:0/3435421378 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4bf404eee0 con 0x7f4bf415ba30 2023-11-08T19:25:08.236 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.234+0000 7f4bf0ff9700 1 -- 172.21.15.143:0/3435421378 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4913 (secure 0 0 0) 0x7f4bec059a90 con 0x7f4bf415ba30 2023-11-08T19:25:08.237 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:25:08.237 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4208,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6834/37280614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":37280614},{"type":"v1","addr":"172.21.15.143:6834","nonce":37280614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/2364281303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":2364281303},{"type":"v1","addr":"172.21.15.143:6835","nonce":2364281303}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6836/4066129908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":4066129908},{"type":"v1","addr":"172.21.15.174:6836","nonce":4066129908}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T19:24:49.995547+0000","modified":"2023-11-08T19:24:53.027089+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.174:6837/2124972405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6835","nonce":2124972405},{"type":"v1","addr":"172.21.15.174:6837","nonce":2124972405}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-08T19:25:08.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.236+0000 7f4bfaa15700 1 -- 172.21.15.143:0/3435421378 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f4be006af00 msgr2=0x7f4be006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:08.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.236+0000 7f4bfaa15700 1 --2- 172.21.15.143:0/3435421378 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f4be006af00 0x7f4be006d3c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f4be8006fd0 tx=0x7f4be8008040 comp rx=0 tx=0).stop 2023-11-08T19:25:08.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.236+0000 7f4bfaa15700 1 -- 172.21.15.143:0/3435421378 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4bf415ba30 msgr2=0x7f4bf4155ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:08.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.236+0000 7f4bfaa15700 1 --2- 172.21.15.143:0/3435421378 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4bf415ba30 0x7f4bf4155ec0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4bf408c000 tx=0x7f4bec004ad0 comp rx=0 tx=0).stop 2023-11-08T19:25:08.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.236+0000 7f4bfaa15700 1 -- 172.21.15.143:0/3435421378 shutdown_connections 2023-11-08T19:25:08.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.236+0000 7f4bfaa15700 1 --2- 172.21.15.143:0/3435421378 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f4be006af00 0x7f4be006d3c0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:08.240 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.236+0000 7f4bfaa15700 1 --2- 172.21.15.143:0/3435421378 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4bf40d03a0 0x7f4bf4141940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:08.240 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.236+0000 7f4bfaa15700 1 --2- 172.21.15.143:0/3435421378 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4bf415ba30 0x7f4bf4155ec0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:08.240 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.236+0000 7f4bfaa15700 1 --2- 172.21.15.143:0/3435421378 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4bf40d1c70 0x7f4bf4155980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:08.240 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.236+0000 7f4bfaa15700 1 -- 172.21.15.143:0/3435421378 >> 172.21.15.143:0/3435421378 conn(0x7f4bf408b000 msgr2=0x7f4bf40cc8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:08.241 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.236+0000 7f4bfaa15700 1 -- 172.21.15.143:0/3435421378 shutdown_connections 2023-11-08T19:25:08.241 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.236+0000 7f4bfaa15700 1 -- 172.21.15.143:0/3435421378 wait complete. 2023-11-08T19:25:08.241 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 8 2023-11-08T19:25:08.251 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:25:08.370 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.368+0000 7f3f3ba8b700 1 Processor -- start 2023-11-08T19:25:08.370 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.368+0000 7f3f3ba8b700 1 -- start start 2023-11-08T19:25:08.373 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.369+0000 7f3f3ba8b700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f3f3415aa30 0x7f3f340cf510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:08.373 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.369+0000 7f3f3ba8b700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f3f340cc530 0x7f3f340b26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:08.373 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.369+0000 7f3f3ba8b700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f340cf080 con 0x7f3f340b2210 2023-11-08T19:25:08.373 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.369+0000 7f3f3ba8b700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f340b2be0 con 0x7f3f3415aa30 2023-11-08T19:25:08.374 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.369+0000 7f3f3ba8b700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f340ce470 con 0x7f3f340cc530 2023-11-08T19:25:08.374 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.369+0000 7f3f39827700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f3f3415aa30 0x7f3f340cf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:08.374 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.369+0000 7f3f39827700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f3f3415aa30 0x7f3f340cf510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49336/0 (socket says 172.21.15.143:49336) 2023-11-08T19:25:08.374 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.369+0000 7f3f39827700 1 -- 172.21.15.143:0/194817407 learned_addr learned my addr 172.21.15.143:0/194817407 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:08.374 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.369+0000 7f3f3a028700 1 --2- 172.21.15.143:0/194817407 >> v2:172.21.15.174:3300/0 conn(0x7f3f340cc530 0x7f3f340b26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:08.375 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.369+0000 7f3f39827700 1 -- 172.21.15.143:0/194817407 >> v2:172.21.15.174:3300/0 conn(0x7f3f340cc530 msgr2=0x7f3f340b26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:08.375 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.369+0000 7f3f39827700 1 --2- 172.21.15.143:0/194817407 >> v2:172.21.15.174:3300/0 conn(0x7f3f340cc530 0x7f3f340b26a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:08.375 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.369+0000 7f3f39827700 1 -- 172.21.15.143:0/194817407 >> v1:172.21.15.174:6789/0 conn(0x7f3f340b2210 legacy=0x7f3f340cbe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:08.375 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.369+0000 7f3f39827700 1 -- 172.21.15.143:0/194817407 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f340ce750 con 0x7f3f3415aa30 2023-11-08T19:25:08.375 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.369+0000 7f3f39827700 1 --2- 172.21.15.143:0/194817407 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3f3415aa30 0x7f3f340cf510 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3f340709e0 tx=0x7f3f30009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a24694b106a29ec server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:08.376 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.369+0000 7f3f38825700 1 -- 172.21.15.143:0/194817407 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f30005e30 con 0x7f3f3415aa30 2023-11-08T19:25:08.376 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.369+0000 7f3f38825700 1 -- 172.21.15.143:0/194817407 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f30006030 con 0x7f3f3415aa30 2023-11-08T19:25:08.376 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.369+0000 7f3f38825700 1 -- 172.21.15.143:0/194817407 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f300063e0 con 0x7f3f3415aa30 2023-11-08T19:25:08.376 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.370+0000 7f3f3ba8b700 1 -- 172.21.15.143:0/194817407 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3f3415aa30 msgr2=0x7f3f340cf510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:08.376 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.370+0000 7f3f3ba8b700 1 --2- 172.21.15.143:0/194817407 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3f3415aa30 0x7f3f340cf510 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3f340709e0 tx=0x7f3f30009bd0 comp rx=0 tx=0).stop 2023-11-08T19:25:08.377 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.370+0000 7f3f3ba8b700 1 -- 172.21.15.143:0/194817407 shutdown_connections 2023-11-08T19:25:08.377 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.370+0000 7f3f3ba8b700 1 --2- 172.21.15.143:0/194817407 >> v2:172.21.15.174:3300/0 conn(0x7f3f340cc530 0x7f3f340b26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:08.377 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.370+0000 7f3f3ba8b700 1 --2- 172.21.15.143:0/194817407 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3f3415aa30 0x7f3f340cf510 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:08.377 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.370+0000 7f3f3ba8b700 1 -- 172.21.15.143:0/194817407 >> 172.21.15.143:0/194817407 conn(0x7f3f34099ab0 msgr2=0x7f3f340d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:08.377 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.370+0000 7f3f3ba8b700 1 -- 172.21.15.143:0/194817407 shutdown_connections 2023-11-08T19:25:08.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.370+0000 7f3f3ba8b700 1 -- 172.21.15.143:0/194817407 wait complete. 2023-11-08T19:25:08.380 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.372+0000 7f3f3ba8b700 1 Processor -- start 2023-11-08T19:25:08.380 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.372+0000 7f3f3ba8b700 1 -- start start 2023-11-08T19:25:08.380 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.372+0000 7f3f3ba8b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3f340b2210 0x7f3f34153f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:08.381 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.373+0000 7f3f3ba8b700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3f340cc530 0x7f3f340c8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:08.381 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.373+0000 7f3f3ba8b700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3f3415aa30 0x7f3f340c96c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:08.381 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.373+0000 7f3f3ba8b700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f341f5a50 con 0x7f3f340cc530 2023-11-08T19:25:08.381 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.373+0000 7f3f3ba8b700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f340c94e0 con 0x7f3f340b2210 2023-11-08T19:25:08.381 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.373+0000 7f3f3ba8b700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f340c9c00 con 0x7f3f3415aa30 2023-11-08T19:25:08.382 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.373+0000 7f3f39026700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3f340cc530 0x7f3f340c8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:08.382 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.373+0000 7f3f39026700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3f340cc530 0x7f3f340c8fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49342/0 (socket says 172.21.15.143:49342) 2023-11-08T19:25:08.382 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.373+0000 7f3f39026700 1 -- 172.21.15.143:0/3929531296 learned_addr learned my addr 172.21.15.143:0/3929531296 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:08.382 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.373+0000 7f3f3a028700 1 --2- 172.21.15.143:0/3929531296 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3f3415aa30 0x7f3f340c96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:08.382 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.373+0000 7f3f39827700 1 --2- 172.21.15.143:0/3929531296 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3f340b2210 0x7f3f34153f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:08.383 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.373+0000 7f3f39026700 1 -- 172.21.15.143:0/3929531296 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3f3415aa30 msgr2=0x7f3f340c96c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:08.383 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.373+0000 7f3f39026700 1 --2- 172.21.15.143:0/3929531296 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3f3415aa30 0x7f3f340c96c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:08.383 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.373+0000 7f3f39026700 1 -- 172.21.15.143:0/3929531296 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3f340b2210 msgr2=0x7f3f34153f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:08.383 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.373+0000 7f3f39026700 1 --2- 172.21.15.143:0/3929531296 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3f340b2210 0x7f3f34153f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:08.384 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.373+0000 7f3f39026700 1 -- 172.21.15.143:0/3929531296 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f30009500 con 0x7f3f340cc530 2023-11-08T19:25:08.384 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.373+0000 7f3f39026700 1 --2- 172.21.15.143:0/3929531296 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3f340cc530 0x7f3f340c8fa0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3f2400c4d0 tx=0x7f3f2400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:08.384 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.373+0000 7f3f2a7fc700 1 -- 172.21.15.143:0/3929531296 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f24011790 con 0x7f3f340cc530 2023-11-08T19:25:08.384 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.373+0000 7f3f2a7fc700 1 -- 172.21.15.143:0/3929531296 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f240118f0 con 0x7f3f340cc530 2023-11-08T19:25:08.384 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.373+0000 7f3f2a7fc700 1 -- 172.21.15.143:0/3929531296 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f24011ca0 con 0x7f3f340cc530 2023-11-08T19:25:08.385 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.373+0000 7f3f3ba8b700 1 -- 172.21.15.143:0/3929531296 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f341f4e40 con 0x7f3f340cc530 2023-11-08T19:25:08.385 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.373+0000 7f3f3ba8b700 1 -- 172.21.15.143:0/3929531296 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f341f5400 con 0x7f3f340cc530 2023-11-08T19:25:08.385 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.374+0000 7f3f3ba8b700 1 -- 172.21.15.143:0/3929531296 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f3415bc60 con 0x7f3f340cc530 2023-11-08T19:25:08.385 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.375+0000 7f3f2a7fc700 1 -- 172.21.15.143:0/3929531296 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3f2401c460 con 0x7f3f340cc530 2023-11-08T19:25:08.385 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.375+0000 7f3f2a7fc700 1 --2- 172.21.15.143:0/3929531296 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f3f1806af00 0x7f3f1806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:08.386 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.375+0000 7f3f2a7fc700 1 -- 172.21.15.143:0/3929531296 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3f24089440 con 0x7f3f340cc530 2023-11-08T19:25:08.386 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.375+0000 7f3f39827700 1 --2- 172.21.15.143:0/3929531296 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f3f1806af00 0x7f3f1806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:08.386 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.376+0000 7f3f39827700 1 --2- 172.21.15.143:0/3929531296 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f3f1806af00 0x7f3f1806d3c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f3f340709e0 tx=0x7f3f30008070 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:08.386 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.379+0000 7f3f2a7fc700 1 -- 172.21.15.143:0/3929531296 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3f240544f0 con 0x7f3f340cc530 2023-11-08T19:25:08.571 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.568+0000 7f3f3ba8b700 1 -- 172.21.15.143:0/3929531296 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3f341544b0 con 0x7f3f340cc530 2023-11-08T19:25:08.572 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.570+0000 7f3f2a7fc700 1 -- 172.21.15.143:0/3929531296 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16553 (secure 0 0 0) 0x7f3f24057b10 con 0x7f3f340cc530 2023-11-08T19:25:08.572 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:25:08.573 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":20,"fsid":"968483af-77fc-4068-ad7c-5c2a1a4613d7","created":"2023-11-08T19:24:32.826484+0000","modified":"2023-11-08T19:24:51.618072+0000","last_up_change":"2023-11-08T19:24:43.573880+0000","last_in_change":"2023-11-08T19:24:35.314525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:44.551530+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-08T19:24:48.256781+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-08T19:24:48.954990+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bae98ec1-d94e-403d-a220-a522a53ea6f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6809","nonce":1487830566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6811","nonce":1487830566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6815","nonce":1487830566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6813","nonce":1487830566}]},"public_addr":"172.21.15.143:6809/1487830566","cluster_addr":"172.21.15.143:6811/1487830566","heartbeat_back_addr":"172.21.15.143:6815/1487830566","heartbeat_front_addr":"172.21.15.143:6813/1487830566","state":["exists","up"]},{"osd":1,"uuid":"5cb6538a-1ccb-4322-8633-09f66af0b7e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6801","nonce":787341397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6803","nonce":787341397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6807","nonce":787341397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6805","nonce":787341397}]},"public_addr":"172.21.15.143:6801/787341397","cluster_addr":"172.21.15.143:6803/787341397","heartbeat_back_addr":"172.21.15.143:6807/787341397","heartbeat_front_addr":"172.21.15.143:6805/787341397","state":["exists","up"]},{"osd":2,"uuid":"cdbf522c-60ef-4d60-9653-8dcb529c101a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6822","nonce":2301325597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6826","nonce":2301325597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6831","nonce":2301325597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6829","nonce":2301325597}]},"public_addr":"172.21.15.143:6822/2301325597","cluster_addr":"172.21.15.143:6826/2301325597","heartbeat_back_addr":"172.21.15.143:6831/2301325597","heartbeat_front_addr":"172.21.15.143:6829/2301325597","state":["exists","up"]},{"osd":3,"uuid":"4e2783cf-ebb9-4dd8-8245-bc1d17ce4325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6817","nonce":2021504428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6819","nonce":2021504428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6827","nonce":2021504428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6823","nonce":2021504428}]},"public_addr":"172.21.15.143:6817/2021504428","cluster_addr":"172.21.15.143:6819/2021504428","heartbeat_back_addr":"172.21.15.143:6827/2021504428","heartbeat_front_addr":"172.21.15.143:6823/2021504428","state":["exists","up"]},{"osd":4,"uuid":"691256fa-9b77-49b9-a346-7ae1adf0ca80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6801","nonce":1993380595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6803","nonce":1993380595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6807","nonce":1993380595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6805","nonce":1993380595}]},"public_addr":"172.21.15.174:6801/1993380595","cluster_addr":"172.21.15.174:6803/1993380595","heartbeat_back_addr":"172.21.15.174:6807/1993380595","heartbeat_front_addr":"172.21.15.174:6805/1993380595","state":["exists","up"]},{"osd":5,"uuid":"756fc352-48c9-46ce-b52e-0541fe6f8525","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6809","nonce":1777923303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6811","nonce":1777923303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6815","nonce":1777923303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6813","nonce":1777923303}]},"public_addr":"172.21.15.174:6809/1777923303","cluster_addr":"172.21.15.174:6811/1777923303","heartbeat_back_addr":"172.21.15.174:6815/1777923303","heartbeat_front_addr":"172.21.15.174:6813/1777923303","state":["exists","up"]},{"osd":6,"uuid":"fab2ac2b-ed97-4cde-9825-b480eabf8b8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6825","nonce":2755130316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6827","nonce":2755130316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6831","nonce":2755130316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6829","nonce":2755130316}]},"public_addr":"172.21.15.174:6825/2755130316","cluster_addr":"172.21.15.174:6827/2755130316","heartbeat_back_addr":"172.21.15.174:6831/2755130316","heartbeat_front_addr":"172.21.15.174:6829/2755130316","state":["exists","up"]},{"osd":7,"uuid":"3de9b797-2ab6-4d41-a863-7e7d638fd703","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6817","nonce":2847141411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6819","nonce":2847141411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6823","nonce":2847141411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6821","nonce":2847141411}]},"public_addr":"172.21.15.174:6817/2847141411","cluster_addr":"172.21.15.174:6819/2847141411","heartbeat_back_addr":"172.21.15.174:6823/2847141411","heartbeat_front_addr":"172.21.15.174:6821/2847141411","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.477076+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.496890+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.539305+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.454673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.538881+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.521294+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.520712+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.540404+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:25:08.574 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.572+0000 7f3f3ba8b700 1 -- 172.21.15.143:0/3929531296 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f3f1806af00 msgr2=0x7f3f1806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:08.575 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.572+0000 7f3f3ba8b700 1 --2- 172.21.15.143:0/3929531296 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f3f1806af00 0x7f3f1806d3c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f3f340709e0 tx=0x7f3f30008070 comp rx=0 tx=0).stop 2023-11-08T19:25:08.575 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.572+0000 7f3f3ba8b700 1 -- 172.21.15.143:0/3929531296 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3f340cc530 msgr2=0x7f3f340c8fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:08.575 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.572+0000 7f3f3ba8b700 1 --2- 172.21.15.143:0/3929531296 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3f340cc530 0x7f3f340c8fa0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3f2400c4d0 tx=0x7f3f2400c890 comp rx=0 tx=0).stop 2023-11-08T19:25:08.575 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.572+0000 7f3f3ba8b700 1 -- 172.21.15.143:0/3929531296 shutdown_connections 2023-11-08T19:25:08.575 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.572+0000 7f3f3ba8b700 1 --2- 172.21.15.143:0/3929531296 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f3f1806af00 0x7f3f1806d3c0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:08.576 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.572+0000 7f3f3ba8b700 1 --2- 172.21.15.143:0/3929531296 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3f3415aa30 0x7f3f340c96c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:08.576 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.572+0000 7f3f3ba8b700 1 --2- 172.21.15.143:0/3929531296 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3f340b2210 0x7f3f34153f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:08.576 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.572+0000 7f3f3ba8b700 1 --2- 172.21.15.143:0/3929531296 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3f340cc530 0x7f3f340c8fa0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:08.576 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.572+0000 7f3f3ba8b700 1 -- 172.21.15.143:0/3929531296 >> 172.21.15.143:0/3929531296 conn(0x7f3f34099ab0 msgr2=0x7f3f340ccf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:08.576 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.572+0000 7f3f3ba8b700 1 -- 172.21.15.143:0/3929531296 shutdown_connections 2023-11-08T19:25:08.577 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.572+0000 7f3f3ba8b700 1 -- 172.21.15.143:0/3929531296 wait complete. 2023-11-08T19:25:08.586 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:25:08.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.704+0000 7f88d51a7700 1 Processor -- start 2023-11-08T19:25:08.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.704+0000 7f88d51a7700 1 -- start start 2023-11-08T19:25:08.709 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.705+0000 7f88d51a7700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f88d00d1c70 0x7f88d00b14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:08.709 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.705+0000 7f88d51a7700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f88d00d03a0 0x7f88d00cc5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:08.710 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.705+0000 7f88d51a7700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88d00b1bc0 con 0x7f88d015ba30 2023-11-08T19:25:08.710 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.705+0000 7f88d51a7700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88d00b1ea0 con 0x7f88d00d03a0 2023-11-08T19:25:08.710 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.705+0000 7f88d51a7700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88d00b2180 con 0x7f88d00d1c70 2023-11-08T19:25:08.710 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.705+0000 7f88ce59c700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f88d00d1c70 0x7f88d00b14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:08.710 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.705+0000 7f88cf59e700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f88d00d03a0 0x7f88d00cc5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:08.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.705+0000 7f88ced9d700 1 --1- >> v1:172.21.15.174:6789/0 conn(0x7f88d015ba30 0x7f88d00d1560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.174:6789/0 says I am v1:172.21.15.143:56436/0 (socket says 172.21.15.143:56436) 2023-11-08T19:25:08.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.705+0000 7f88ced9d700 1 -- 172.21.15.143:0/3181629362 learned_addr learned my addr 172.21.15.143:0/3181629362 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:25:08.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.705+0000 7f88cdd9b700 1 -- 172.21.15.143:0/3181629362 <== mon.1 v1:172.21.15.174:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3623535221 0 0) 0x7f88d00b1bc0 con 0x7f88d015ba30 2023-11-08T19:25:08.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.705+0000 7f88cdd9b700 1 -- 172.21.15.143:0/3181629362 --> v1:172.21.15.174:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f88b80035b0 con 0x7f88d015ba30 2023-11-08T19:25:08.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.705+0000 7f88ce59c700 1 -- 172.21.15.143:0/3181629362 >> v2:172.21.15.174:3300/0 conn(0x7f88d00d03a0 msgr2=0x7f88d00cc5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:08.712 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.705+0000 7f88ce59c700 1 --2- 172.21.15.143:0/3181629362 >> v2:172.21.15.174:3300/0 conn(0x7f88d00d03a0 0x7f88d00cc5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:08.712 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.705+0000 7f88ce59c700 1 -- 172.21.15.143:0/3181629362 >> v1:172.21.15.174:6789/0 conn(0x7f88d015ba30 legacy=0x7f88d00d1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:08.712 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.705+0000 7f88ce59c700 1 -- 172.21.15.143:0/3181629362 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88d00b2460 con 0x7f88d00d1c70 2023-11-08T19:25:08.712 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.705+0000 7f88cf59e700 1 --2- 172.21.15.143:0/3181629362 >> v2:172.21.15.174:3300/0 conn(0x7f88d00d03a0 0x7f88d00cc5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T19:25:08.712 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.706+0000 7f88ce59c700 1 --2- 172.21.15.143:0/3181629362 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f88d00d1c70 0x7f88d00b14c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f88c4009840 tx=0x7f88c4009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5de530c76c379714 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:08.713 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.706+0000 7f88cdd9b700 1 -- 172.21.15.143:0/3181629362 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88c4011830 con 0x7f88d00d1c70 2023-11-08T19:25:08.713 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.706+0000 7f88cdd9b700 1 -- 172.21.15.143:0/3181629362 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88c4011990 con 0x7f88d00d1c70 2023-11-08T19:25:08.713 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.706+0000 7f88cdd9b700 1 -- 172.21.15.143:0/3181629362 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88c4011d00 con 0x7f88d00d1c70 2023-11-08T19:25:08.713 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.706+0000 7f88d51a7700 1 -- 172.21.15.143:0/3181629362 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f88d00d1c70 msgr2=0x7f88d00b14c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:08.713 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.706+0000 7f88d51a7700 1 --2- 172.21.15.143:0/3181629362 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f88d00d1c70 0x7f88d00b14c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f88c4009840 tx=0x7f88c4009b50 comp rx=0 tx=0).stop 2023-11-08T19:25:08.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.706+0000 7f88d51a7700 1 -- 172.21.15.143:0/3181629362 shutdown_connections 2023-11-08T19:25:08.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.706+0000 7f88d51a7700 1 --2- 172.21.15.143:0/3181629362 >> v2:172.21.15.174:3300/0 conn(0x7f88d00d03a0 0x7f88d00cc5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:08.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.706+0000 7f88d51a7700 1 --2- 172.21.15.143:0/3181629362 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f88d00d1c70 0x7f88d00b14c0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:08.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.706+0000 7f88d51a7700 1 -- 172.21.15.143:0/3181629362 >> 172.21.15.143:0/3181629362 conn(0x7f88d008b000 msgr2=0x7f88d009ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:08.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.706+0000 7f88d51a7700 1 -- 172.21.15.143:0/3181629362 shutdown_connections 2023-11-08T19:25:08.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.706+0000 7f88d51a7700 1 -- 172.21.15.143:0/3181629362 wait complete. 2023-11-08T19:25:08.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.708+0000 7f88d51a7700 1 Processor -- start 2023-11-08T19:25:08.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.708+0000 7f88d51a7700 1 -- start start 2023-11-08T19:25:08.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.709+0000 7f88d51a7700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f88d00d03a0 0x7f88d01f10c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:08.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.709+0000 7f88d51a7700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f88d00d1c70 0x7f88d01f50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:08.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.709+0000 7f88d51a7700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f88d015ba30 0x7f88d015e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:08.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.709+0000 7f88d51a7700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88d00cd1f0 con 0x7f88d015ba30 2023-11-08T19:25:08.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.709+0000 7f88d51a7700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88d01f55e0 con 0x7f88d00d1c70 2023-11-08T19:25:08.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.709+0000 7f88d51a7700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88d01f16f0 con 0x7f88d00d03a0 2023-11-08T19:25:08.721 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.709+0000 7f88cf59e700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f88d015ba30 0x7f88d015e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:08.721 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.709+0000 7f88cf59e700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f88d015ba30 0x7f88d015e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49358/0 (socket says 172.21.15.143:49358) 2023-11-08T19:25:08.722 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.709+0000 7f88cf59e700 1 -- 172.21.15.143:0/4010488991 learned_addr learned my addr 172.21.15.143:0/4010488991 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:08.722 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.709+0000 7f88ced9d700 1 --2- 172.21.15.143:0/4010488991 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f88d00d03a0 0x7f88d01f10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:08.722 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.709+0000 7f88ce59c700 1 --2- 172.21.15.143:0/4010488991 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f88d00d1c70 0x7f88d01f50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:08.722 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.710+0000 7f88cf59e700 1 -- 172.21.15.143:0/4010488991 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f88d00d03a0 msgr2=0x7f88d01f10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:08.723 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.710+0000 7f88cf59e700 1 --2- 172.21.15.143:0/4010488991 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f88d00d03a0 0x7f88d01f10c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:08.723 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.710+0000 7f88cf59e700 1 -- 172.21.15.143:0/4010488991 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f88d00d1c70 msgr2=0x7f88d01f50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:08.723 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.710+0000 7f88cf59e700 1 --2- 172.21.15.143:0/4010488991 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f88d00d1c70 0x7f88d01f50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:08.723 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.710+0000 7f88cf59e700 1 -- 172.21.15.143:0/4010488991 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88c8009a70 con 0x7f88d015ba30 2023-11-08T19:25:08.723 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.710+0000 7f88cf59e700 1 --2- 172.21.15.143:0/4010488991 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f88d015ba30 0x7f88d015e420 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f88c800efc0 tx=0x7f88c800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:08.723 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.710+0000 7f88b77fe700 1 -- 172.21.15.143:0/4010488991 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88c800ca90 con 0x7f88d015ba30 2023-11-08T19:25:08.724 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.711+0000 7f88b77fe700 1 -- 172.21.15.143:0/4010488991 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88c800f040 con 0x7f88d015ba30 2023-11-08T19:25:08.724 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.711+0000 7f88d51a7700 1 -- 172.21.15.143:0/4010488991 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88c40094a0 con 0x7f88d015ba30 2023-11-08T19:25:08.724 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.711+0000 7f88b77fe700 1 -- 172.21.15.143:0/4010488991 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88c800ce10 con 0x7f88d015ba30 2023-11-08T19:25:08.724 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.711+0000 7f88d51a7700 1 -- 172.21.15.143:0/4010488991 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88d01f1cb0 con 0x7f88d015ba30 2023-11-08T19:25:08.724 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.712+0000 7f88d51a7700 1 -- 172.21.15.143:0/4010488991 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88d00cf160 con 0x7f88d015ba30 2023-11-08T19:25:08.725 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.712+0000 7f88b77fe700 1 -- 172.21.15.143:0/4010488991 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f88c8014480 con 0x7f88d015ba30 2023-11-08T19:25:08.725 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.712+0000 7f88b77fe700 1 --2- 172.21.15.143:0/4010488991 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f88bc06af00 0x7f88bc06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:08.725 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.712+0000 7f88b77fe700 1 -- 172.21.15.143:0/4010488991 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f88c8089ca0 con 0x7f88d015ba30 2023-11-08T19:25:08.725 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.713+0000 7f88ced9d700 1 --2- 172.21.15.143:0/4010488991 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f88bc06af00 0x7f88bc06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:08.726 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.713+0000 7f88ced9d700 1 --2- 172.21.15.143:0/4010488991 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f88bc06af00 0x7f88bc06d3c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f88c0007fd0 tx=0x7f88c0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:08.726 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.719+0000 7f88b77fe700 1 -- 172.21.15.143:0/4010488991 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f88c8054cd0 con 0x7f88d015ba30 2023-11-08T19:25:08.954 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.952+0000 7f88d51a7700 1 -- 172.21.15.143:0/4010488991 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f88d004eee0 con 0x7f88d015ba30 2023-11-08T19:25:08.955 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.953+0000 7f88b77fe700 1 -- 172.21.15.143:0/4010488991 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4913 (secure 0 0 0) 0x7f88c8009d70 con 0x7f88d015ba30 2023-11-08T19:25:08.956 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:25:08.956 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4208,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6834/37280614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":37280614},{"type":"v1","addr":"172.21.15.143:6834","nonce":37280614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/2364281303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":2364281303},{"type":"v1","addr":"172.21.15.143:6835","nonce":2364281303}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6836/4066129908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":4066129908},{"type":"v1","addr":"172.21.15.174:6836","nonce":4066129908}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T19:24:49.995547+0000","modified":"2023-11-08T19:24:53.027089+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.174:6837/2124972405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6835","nonce":2124972405},{"type":"v1","addr":"172.21.15.174:6837","nonce":2124972405}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-08T19:25:08.957 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.956+0000 7f88d51a7700 1 -- 172.21.15.143:0/4010488991 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f88bc06af00 msgr2=0x7f88bc06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:08.958 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.956+0000 7f88d51a7700 1 --2- 172.21.15.143:0/4010488991 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f88bc06af00 0x7f88bc06d3c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f88c0007fd0 tx=0x7f88c0009040 comp rx=0 tx=0).stop 2023-11-08T19:25:08.958 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.956+0000 7f88d51a7700 1 -- 172.21.15.143:0/4010488991 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f88d015ba30 msgr2=0x7f88d015e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:08.958 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.956+0000 7f88d51a7700 1 --2- 172.21.15.143:0/4010488991 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f88d015ba30 0x7f88d015e420 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f88c800efc0 tx=0x7f88c800c3a0 comp rx=0 tx=0).stop 2023-11-08T19:25:08.958 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.956+0000 7f88d51a7700 1 -- 172.21.15.143:0/4010488991 shutdown_connections 2023-11-08T19:25:08.958 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.956+0000 7f88d51a7700 1 --2- 172.21.15.143:0/4010488991 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f88bc06af00 0x7f88bc06d3c0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:08.959 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.956+0000 7f88d51a7700 1 --2- 172.21.15.143:0/4010488991 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f88d00d03a0 0x7f88d01f10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:08.959 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.956+0000 7f88d51a7700 1 --2- 172.21.15.143:0/4010488991 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f88d015ba30 0x7f88d015e420 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:08.959 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.956+0000 7f88d51a7700 1 --2- 172.21.15.143:0/4010488991 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f88d00d1c70 0x7f88d01f50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:08.959 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.956+0000 7f88d51a7700 1 -- 172.21.15.143:0/4010488991 >> 172.21.15.143:0/4010488991 conn(0x7f88d008b000 msgr2=0x7f88d00af260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:08.959 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.956+0000 7f88d51a7700 1 -- 172.21.15.143:0/4010488991 shutdown_connections 2023-11-08T19:25:08.960 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:08.956+0000 7f88d51a7700 1 -- 172.21.15.143:0/4010488991 wait complete. 2023-11-08T19:25:08.960 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 8 2023-11-08T19:25:09.371 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-08T19:24:49.995547+0000', 'modified': '2023-11-08T19:24:53.027089+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4282}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4282': {'gid': 4282, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.174:6837/2124972405', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.174:6835', 'nonce': 2124972405}, {'type': 'v1', 'addr': '172.21.15.174:6837', 'nonce': 2124972405}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-08T19:25:09.372 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-08T19:25:09.372 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-11-08T19:25:09.497 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.495+0000 7fa2078f0700 1 Processor -- start 2023-11-08T19:25:09.498 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.495+0000 7fa2078f0700 1 -- start start 2023-11-08T19:25:09.500 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.497+0000 7fa2078f0700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fa20015dc80 0x7fa2000d1f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:09.500 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.497+0000 7fa2078f0700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa2000cf090 0x7fa2000b9690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:09.500 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.497+0000 7fa2078f0700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2000d3200 con 0x7fa2000d3690 2023-11-08T19:25:09.500 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.497+0000 7fa2078f0700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2000b9bd0 con 0x7fa2000cf090 2023-11-08T19:25:09.501 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.497+0000 7fa2078f0700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2000b7cf0 con 0x7fa20015dc80 2023-11-08T19:25:09.501 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.497+0000 7fa205e8d700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa2000cf090 0x7fa2000b9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:09.501 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.497+0000 7fa205e8d700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa2000cf090 0x7fa2000b9690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49364/0 (socket says 172.21.15.143:49364) 2023-11-08T19:25:09.501 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.497+0000 7fa205e8d700 1 -- 172.21.15.143:0/1609094028 learned_addr learned my addr 172.21.15.143:0/1609094028 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:09.501 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.497+0000 7fa20568c700 1 --2- 172.21.15.143:0/1609094028 >> v2:172.21.15.174:3300/0 conn(0x7fa20015dc80 0x7fa2000d1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:09.502 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.497+0000 7fa205e8d700 1 -- 172.21.15.143:0/1609094028 >> v2:172.21.15.174:3300/0 conn(0x7fa20015dc80 msgr2=0x7fa2000d1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:09.502 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.497+0000 7fa205e8d700 1 --2- 172.21.15.143:0/1609094028 >> v2:172.21.15.174:3300/0 conn(0x7fa20015dc80 0x7fa2000d1f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:09.502 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.497+0000 7fa205e8d700 1 -- 172.21.15.143:0/1609094028 >> v1:172.21.15.174:6789/0 conn(0x7fa2000d3690 legacy=0x7fa2000ce980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:09.502 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.497+0000 7fa205e8d700 1 -- 172.21.15.143:0/1609094028 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa20015e060 con 0x7fa2000cf090 2023-11-08T19:25:09.502 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.497+0000 7fa205e8d700 1 --2- 172.21.15.143:0/1609094028 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa2000cf090 0x7fa2000b9690 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa2000b6710 tx=0x7fa1f800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d4f451309431035 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:09.503 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.497+0000 7fa1f7fff700 1 -- 172.21.15.143:0/1609094028 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f800e070 con 0x7fa2000cf090 2023-11-08T19:25:09.503 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.497+0000 7fa1f7fff700 1 -- 172.21.15.143:0/1609094028 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1f800b910 con 0x7fa2000cf090 2023-11-08T19:25:09.503 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.497+0000 7fa1f7fff700 1 -- 172.21.15.143:0/1609094028 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f800bcc0 con 0x7fa2000cf090 2023-11-08T19:25:09.503 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.497+0000 7fa2078f0700 1 -- 172.21.15.143:0/1609094028 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa2000cf090 msgr2=0x7fa2000b9690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:09.503 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.497+0000 7fa2078f0700 1 --2- 172.21.15.143:0/1609094028 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa2000cf090 0x7fa2000b9690 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa2000b6710 tx=0x7fa1f800b330 comp rx=0 tx=0).stop 2023-11-08T19:25:09.504 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.497+0000 7fa2078f0700 1 -- 172.21.15.143:0/1609094028 shutdown_connections 2023-11-08T19:25:09.504 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.497+0000 7fa2078f0700 1 --2- 172.21.15.143:0/1609094028 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa2000cf090 0x7fa2000b9690 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:09.504 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.497+0000 7fa2078f0700 1 --2- 172.21.15.143:0/1609094028 >> v2:172.21.15.174:3300/0 conn(0x7fa20015dc80 0x7fa2000d1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:09.504 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.497+0000 7fa2078f0700 1 -- 172.21.15.143:0/1609094028 >> 172.21.15.143:0/1609094028 conn(0x7fa2000668e0 msgr2=0x7fa200066cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:09.504 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.497+0000 7fa2078f0700 1 -- 172.21.15.143:0/1609094028 shutdown_connections 2023-11-08T19:25:09.505 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.497+0000 7fa2078f0700 1 -- 172.21.15.143:0/1609094028 wait complete. 2023-11-08T19:25:09.509 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.499+0000 7fa2078f0700 1 Processor -- start 2023-11-08T19:25:09.510 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.499+0000 7fa2078f0700 1 -- start start 2023-11-08T19:25:09.510 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.499+0000 7fa2078f0700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa2000cf090 0x7fa2001f3270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:09.510 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.499+0000 7fa2078f0700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa2000d3690 0x7fa2000c76e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:09.510 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.499+0000 7fa2078f0700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa20015dc80 0x7fa2000c7c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:09.510 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.499+0000 7fa2078f0700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2000cbad0 con 0x7fa2000d3690 2023-11-08T19:25:09.511 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.499+0000 7fa2078f0700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa200161040 con 0x7fa20015dc80 2023-11-08T19:25:09.511 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.499+0000 7fa2078f0700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa200160d60 con 0x7fa2000cf090 2023-11-08T19:25:09.511 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.499+0000 7fa204e8b700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa2000d3690 0x7fa2000c76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:09.511 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.499+0000 7fa204e8b700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa2000d3690 0x7fa2000c76e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49368/0 (socket says 172.21.15.143:49368) 2023-11-08T19:25:09.511 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.499+0000 7fa204e8b700 1 -- 172.21.15.143:0/3535133102 learned_addr learned my addr 172.21.15.143:0/3535133102 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:09.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.500+0000 7fa205e8d700 1 --2- 172.21.15.143:0/3535133102 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa20015dc80 0x7fa2000c7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:09.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.500+0000 7fa20568c700 1 --2- 172.21.15.143:0/3535133102 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa2000cf090 0x7fa2001f3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:09.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.500+0000 7fa204e8b700 1 -- 172.21.15.143:0/3535133102 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa2000cf090 msgr2=0x7fa2001f3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:09.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.500+0000 7fa204e8b700 1 --2- 172.21.15.143:0/3535133102 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa2000cf090 0x7fa2001f3270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:09.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.500+0000 7fa204e8b700 1 -- 172.21.15.143:0/3535133102 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa20015dc80 msgr2=0x7fa2000c7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:09.513 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.500+0000 7fa204e8b700 1 --2- 172.21.15.143:0/3535133102 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa20015dc80 0x7fa2000c7c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:09.513 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.500+0000 7fa204e8b700 1 -- 172.21.15.143:0/3535133102 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1f8009b30 con 0x7fa2000d3690 2023-11-08T19:25:09.513 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.500+0000 7fa20568c700 1 --2- 172.21.15.143:0/3535133102 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa2000cf090 0x7fa2001f3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:25:09.513 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.500+0000 7fa205e8d700 1 --2- 172.21.15.143:0/3535133102 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa20015dc80 0x7fa2000c7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:25:09.513 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.500+0000 7fa204e8b700 1 --2- 172.21.15.143:0/3535133102 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa2000d3690 0x7fa2000c76e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc00c4b0 tx=0x7fa1fc00c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:09.514 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.500+0000 7fa1f5ffb700 1 -- 172.21.15.143:0/3535133102 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1fc0052d0 con 0x7fa2000d3690 2023-11-08T19:25:09.514 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.500+0000 7fa1f5ffb700 1 -- 172.21.15.143:0/3535133102 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1fc005430 con 0x7fa2000d3690 2023-11-08T19:25:09.514 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.500+0000 7fa2078f0700 1 -- 172.21.15.143:0/3535133102 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2000cb410 con 0x7fa2000d3690 2023-11-08T19:25:09.514 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.500+0000 7fa1f5ffb700 1 -- 172.21.15.143:0/3535133102 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1fc0038c0 con 0x7fa2000d3690 2023-11-08T19:25:09.514 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.500+0000 7fa2078f0700 1 -- 172.21.15.143:0/3535133102 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa200150790 con 0x7fa2000d3690 2023-11-08T19:25:09.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.502+0000 7fa1f5ffb700 1 -- 172.21.15.143:0/3535133102 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa1fc003a20 con 0x7fa2000d3690 2023-11-08T19:25:09.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.502+0000 7fa1f5ffb700 1 --2- 172.21.15.143:0/3535133102 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa1e406af00 0x7fa1e406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:09.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.502+0000 7fa1f5ffb700 1 -- 172.21.15.143:0/3535133102 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa1fc088a00 con 0x7fa2000d3690 2023-11-08T19:25:09.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.502+0000 7fa20568c700 1 --2- 172.21.15.143:0/3535133102 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa1e406af00 0x7fa1e406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:09.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.504+0000 7fa20568c700 1 --2- 172.21.15.143:0/3535133102 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa1e406af00 0x7fa1e406d3c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa1f0009ac0 tx=0x7fa1f0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:09.516 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.505+0000 7fa213ae9b80 1 Processor -- start 2023-11-08T19:25:09.516 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.505+0000 7fa213ae9b80 1 -- start start 2023-11-08T19:25:09.516 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.506+0000 7fa213ae9b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5615553376f0 0x561555367450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:09.516 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.506+0000 7fa213ae9b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561555339c30 0x561555367990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:09.516 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.506+0000 7fa213ae9b80 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x561555337ad0 0x561555351f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:09.517 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.506+0000 7fa213ae9b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615552a36e0 con 0x5615553376f0 2023-11-08T19:25:09.517 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.506+0000 7fa213ae9b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615552973b0 con 0x561555339c30 2023-11-08T19:25:09.517 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.506+0000 7fa213ae9b80 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x561555294cd0 con 0x561555337ad0 2023-11-08T19:25:09.517 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.506+0000 7fa20568c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5615553376f0 0x561555367450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:09.517 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.506+0000 7fa20568c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5615553376f0 0x561555367450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49374/0 (socket says 172.21.15.143:49374) 2023-11-08T19:25:09.518 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.506+0000 7fa20568c700 1 -- 172.21.15.143:0/458386115 learned_addr learned my addr 172.21.15.143:0/458386115 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:09.518 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.506+0000 7fa204e8b700 1 --2- 172.21.15.143:0/458386115 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561555339c30 0x561555367990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:09.518 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.506+0000 7fa205e8d700 1 --2- 172.21.15.143:0/458386115 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x561555337ad0 0x561555351f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:09.518 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.506+0000 7fa20568c700 1 -- 172.21.15.143:0/458386115 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x561555337ad0 msgr2=0x561555351f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:09.518 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.506+0000 7fa20568c700 1 --2- 172.21.15.143:0/458386115 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x561555337ad0 0x561555351f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:09.519 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.506+0000 7fa20568c700 1 -- 172.21.15.143:0/458386115 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561555339c30 msgr2=0x561555367990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:09.519 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.506+0000 7fa20568c700 1 --2- 172.21.15.143:0/458386115 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561555339c30 0x561555367990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:09.519 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.506+0000 7fa20568c700 1 -- 172.21.15.143:0/458386115 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56155510d5e0 con 0x5615553376f0 2023-11-08T19:25:09.519 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.506+0000 7fa20568c700 1 --2- 172.21.15.143:0/458386115 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5615553376f0 0x561555367450 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa1f0010980 tx=0x7fa1f0013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd66ad776a75ae39 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:09.519 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.507+0000 7fa1eaffd700 1 -- 172.21.15.143:0/458386115 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f0004230 con 0x5615553376f0 2023-11-08T19:25:09.519 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.507+0000 7fa1eaffd700 1 -- 172.21.15.143:0/458386115 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1f001a070 con 0x5615553376f0 2023-11-08T19:25:09.520 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.507+0000 7fa1eaffd700 1 -- 172.21.15.143:0/458386115 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f0004590 con 0x5615553376f0 2023-11-08T19:25:09.520 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.507+0000 7fa213ae9b80 1 -- 172.21.15.143:0/458386115 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5615553376f0 msgr2=0x561555367450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:09.520 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.507+0000 7fa213ae9b80 1 --2- 172.21.15.143:0/458386115 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5615553376f0 0x561555367450 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa1f0010980 tx=0x7fa1f0013000 comp rx=0 tx=0).stop 2023-11-08T19:25:09.520 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.507+0000 7fa213ae9b80 1 -- 172.21.15.143:0/458386115 shutdown_connections 2023-11-08T19:25:09.520 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.507+0000 7fa213ae9b80 1 --2- 172.21.15.143:0/458386115 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x561555337ad0 0x561555351f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:09.521 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.507+0000 7fa213ae9b80 1 --2- 172.21.15.143:0/458386115 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5615553376f0 0x561555367450 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:09.521 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.507+0000 7fa213ae9b80 1 --2- 172.21.15.143:0/458386115 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561555339c30 0x561555367990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:09.521 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.507+0000 7fa213ae9b80 1 -- 172.21.15.143:0/458386115 >> 172.21.15.143:0/458386115 conn(0x5615552b2660 msgr2=0x56155533d3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:09.521 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.507+0000 7fa213ae9b80 1 -- 172.21.15.143:0/458386115 shutdown_connections 2023-11-08T19:25:09.521 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.507+0000 7fa213ae9b80 1 -- 172.21.15.143:0/458386115 wait complete. 2023-11-08T19:25:09.521 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.508+0000 7fa213ae9b80 1 -- start start 2023-11-08T19:25:09.522 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.508+0000 7fa213ae9b80 1 Processor -- start 2023-11-08T19:25:09.526 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.509+0000 7fa213ae9b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5615553376f0 0x56155537b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:09.526 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.509+0000 7fa213ae9b80 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x561555337ad0 0x56155536a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:09.526 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.509+0000 7fa213ae9b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56155537cb80 0x561555368d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:09.526 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.509+0000 7fa213ae9b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615552905c0 con 0x5615553376f0 2023-11-08T19:25:09.526 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.509+0000 7fa213ae9b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x56155522c0d0 con 0x56155537cb80 2023-11-08T19:25:09.527 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.509+0000 7fa213ae9b80 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x56155522bd70 con 0x561555337ad0 2023-11-08T19:25:09.527 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.509+0000 7fa20568c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5615553376f0 0x56155537b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:09.527 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.509+0000 7fa213ae9b80 10 client.0 _ll_register_callbacks cb 0x5615550887b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T19:25:09.527 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.509+0000 7fa20568c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5615553376f0 0x56155537b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49380/0 (socket says 172.21.15.143:49380) 2023-11-08T19:25:09.527 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.509+0000 7fa20568c700 1 -- 172.21.15.143:0/1058302959 learned_addr learned my addr 172.21.15.143:0/1058302959 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:09.528 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.510+0000 7fa204e8b700 1 --2- 172.21.15.143:0/1058302959 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x561555337ad0 0x56155536a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:09.528 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.510+0000 7fa205e8d700 1 --2- 172.21.15.143:0/1058302959 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56155537cb80 0x561555368d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:09.528 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.510+0000 7fa20568c700 1 -- 172.21.15.143:0/1058302959 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x561555337ad0 msgr2=0x56155536a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:09.528 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.510+0000 7fa20568c700 1 --2- 172.21.15.143:0/1058302959 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x561555337ad0 0x56155536a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:09.528 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.510+0000 7fa20568c700 1 -- 172.21.15.143:0/1058302959 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56155537cb80 msgr2=0x561555368d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:09.529 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.510+0000 7fa20568c700 1 --2- 172.21.15.143:0/1058302959 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56155537cb80 0x561555368d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:09.529 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.510+0000 7fa20568c700 1 -- 172.21.15.143:0/1058302959 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1f000e050 con 0x5615553376f0 2023-11-08T19:25:09.529 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.510+0000 7fa20568c700 1 --2- 172.21.15.143:0/1058302959 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5615553376f0 0x56155537b5b0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa1f0024040 tx=0x7fa1f0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3d80415268b4672 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:09.529 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.510+0000 7fa1dffff700 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2023-11-08T19:25:09.529 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.510+0000 7fa1dffff700 1 -- 172.21.15.143:0/1058302959 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f0015ad0 con 0x5615553376f0 2023-11-08T19:25:09.529 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.510+0000 7fa1dffff700 1 -- 172.21.15.143:0/1058302959 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa1f0015c30 con 0x5615553376f0 2023-11-08T19:25:09.530 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.510+0000 7fa213ae9b80 1 -- 172.21.15.143:0/1058302959 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56155532f880 con 0x5615553376f0 2023-11-08T19:25:09.530 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.510+0000 7fa1dffff700 1 -- 172.21.15.143:0/1058302959 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa1f0015df0 con 0x5615553376f0 2023-11-08T19:25:09.530 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.510+0000 7fa213ae9b80 10 client.4656 fetch_fsmap learned FSMap version 8 2023-11-08T19:25:09.530 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.510+0000 7fa213ae9b80 1 -- 172.21.15.143:0/1058302959 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x5615550b9d90 con 0x5615553376f0 2023-11-08T19:25:09.530 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.510+0000 7fa1dffff700 1 -- 172.21.15.143:0/1058302959 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56155532f880 con 0x5615553376f0 2023-11-08T19:25:09.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.510+0000 7fa1dffff700 1 -- 172.21.15.143:0/1058302959 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 8) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fa1f00036a0 con 0x5615553376f0 2023-11-08T19:25:09.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.510+0000 7fa213ae9b80 10 client.4656 fetch_fsmap finished waiting for FSMap version 8 2023-11-08T19:25:09.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.510+0000 7fa213ae9b80 10 client.4656 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-08T19:25:09.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.510+0000 7fa213ae9b80 20 client.4656 populate_metadata read hostname 'smithi143' 2023-11-08T19:25:09.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.510+0000 7fa213ae9b80 1 --2- 172.21.15.143:0/1058302959 >> [v2:172.21.15.174:6835/2124972405,v1:172.21.15.174:6837/2124972405] conn(0x561555395480 0x561555391940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:09.532 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.510+0000 7fa213ae9b80 4 client.4656 mds_command: new command op to 4282 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-08T19:25:09.532 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.510+0000 7fa213ae9b80 1 -- 172.21.15.143:0/1058302959 --> [v2:172.21.15.174:6835/2124972405,v1:172.21.15.174:6837/2124972405] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56155530ba30 con 0x561555395480 2023-11-08T19:25:09.532 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.511+0000 7fa204e8b700 1 --2- 172.21.15.143:0/1058302959 >> [v2:172.21.15.174:6835/2124972405,v1:172.21.15.174:6837/2124972405] conn(0x561555395480 0x561555391940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:09.532 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.511+0000 7fa204e8b700 1 --2- 172.21.15.143:0/1058302959 >> [v2:172.21.15.174:6835/2124972405,v1:172.21.15.174:6837/2124972405] conn(0x561555395480 0x561555391940 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36c36b695474c478 server_cookie=dd5e80b8d9335409 in_seq=0 out_seq=0 2023-11-08T19:25:09.532 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.511+0000 7fa1dffff700 10 client.4656 ms_handle_connect on v2:172.21.15.174:6835/2124972405 2023-11-08T19:25:09.533 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.511+0000 7fa1dffff700 1 -- 172.21.15.143:0/1058302959 <== mds.0 v2:172.21.15.174:6835/2124972405 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56155530ba30 con 0x561555395480 2023-11-08T19:25:09.533 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.511+0000 7fa1dffff700 10 client.4656 handle_command_reply: tid=0 2023-11-08T19:25:09.533 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.511+0000 7fa213ae9b80 1 client.4656 shutdown 2023-11-08T19:25:09.533 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.511+0000 7fa213ae9b80 1 -- 172.21.15.143:0/1058302959 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5615553376f0 msgr2=0x56155537b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:09.533 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.511+0000 7fa213ae9b80 1 --2- 172.21.15.143:0/1058302959 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5615553376f0 0x56155537b5b0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa1f0024040 tx=0x7fa1f0004750 comp rx=0 tx=0).stop 2023-11-08T19:25:09.534 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.511+0000 7fa213ae9b80 1 -- 172.21.15.143:0/1058302959 shutdown_connections 2023-11-08T19:25:09.534 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.511+0000 7fa213ae9b80 1 --2- 172.21.15.143:0/1058302959 >> [v2:172.21.15.174:6835/2124972405,v1:172.21.15.174:6837/2124972405] conn(0x561555395480 0x561555391940 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:09.534 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.511+0000 7fa213ae9b80 1 --2- 172.21.15.143:0/1058302959 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56155537cb80 0x561555368d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:09.534 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.511+0000 7fa1dffff700 0 client.4656 ms_handle_reset on v2:172.21.15.174:6835/2124972405 2023-11-08T19:25:09.534 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.511+0000 7fa213ae9b80 1 --2- 172.21.15.143:0/1058302959 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5615553376f0 0x56155537b5b0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:09.535 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.511+0000 7fa213ae9b80 1 --2- 172.21.15.143:0/1058302959 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x561555337ad0 0x56155536a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:09.535 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.511+0000 7fa213ae9b80 1 -- 172.21.15.143:0/1058302959 >> 172.21.15.143:0/1058302959 conn(0x5615552b2660 msgr2=0x561555383580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:09.535 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.511+0000 7fa213ae9b80 1 -- 172.21.15.143:0/1058302959 shutdown_connections 2023-11-08T19:25:09.535 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.511+0000 7fa213ae9b80 1 -- 172.21.15.143:0/1058302959 wait complete. 2023-11-08T19:25:09.535 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.511+0000 7fa213ae9b80 20 client.4656 trim_cache size 0 max 16384 2023-11-08T19:25:09.541 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.530+0000 7fa213ae9b80 1 Processor -- start 2023-11-08T19:25:09.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.530+0000 7fa213ae9b80 1 -- start start 2023-11-08T19:25:09.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.531+0000 7fa213ae9b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615553376f0 0x5615553a4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:09.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.531+0000 7fa213ae9b80 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x561555337ad0 0x56155537bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:09.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.531+0000 7fa213ae9b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56155537cb80 0x56155537c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:09.543 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.531+0000 7fa213ae9b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x56155510f8d0 con 0x56155537cb80 2023-11-08T19:25:09.543 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.531+0000 7fa213ae9b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615553675c0 con 0x5615553376f0 2023-11-08T19:25:09.543 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.531+0000 7fa213ae9b80 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x56155533a950 con 0x561555337ad0 2023-11-08T19:25:09.543 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.531+0000 7fa205e8d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56155537cb80 0x56155537c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:09.543 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.531+0000 7fa205e8d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56155537cb80 0x56155537c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49392/0 (socket says 172.21.15.143:49392) 2023-11-08T19:25:09.544 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.531+0000 7fa205e8d700 1 -- 172.21.15.143:0/1678301659 learned_addr learned my addr 172.21.15.143:0/1678301659 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:09.544 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.531+0000 7fa20568c700 1 --2- 172.21.15.143:0/1678301659 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615553376f0 0x5615553a4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:09.544 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.531+0000 7fa204e8b700 1 --2- 172.21.15.143:0/1678301659 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x561555337ad0 0x56155537bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:09.544 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.531+0000 7fa205e8d700 1 -- 172.21.15.143:0/1678301659 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x561555337ad0 msgr2=0x56155537bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:09.544 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.531+0000 7fa205e8d700 1 --2- 172.21.15.143:0/1678301659 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x561555337ad0 0x56155537bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:09.545 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.531+0000 7fa205e8d700 1 -- 172.21.15.143:0/1678301659 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615553376f0 msgr2=0x5615553a4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:09.545 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.531+0000 7fa205e8d700 1 --2- 172.21.15.143:0/1678301659 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615553376f0 0x5615553a4430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:09.545 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.531+0000 7fa205e8d700 1 -- 172.21.15.143:0/1678301659 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5615550a2c10 con 0x56155537cb80 2023-11-08T19:25:09.545 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.531+0000 7fa204e8b700 1 --2- 172.21.15.143:0/1678301659 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x561555337ad0 0x56155537bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:25:09.545 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.531+0000 7fa205e8d700 1 --2- 172.21.15.143:0/1678301659 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56155537cb80 0x56155537c240 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa1f8012000 tx=0x7fa1f8010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bd91a52453a41fa server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:09.546 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.531+0000 7fa1eb7fe700 1 -- 172.21.15.143:0/1678301659 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f8022070 con 0x56155537cb80 2023-11-08T19:25:09.546 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.531+0000 7fa1eb7fe700 1 -- 172.21.15.143:0/1678301659 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1f800bb00 con 0x56155537cb80 2023-11-08T19:25:09.546 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.531+0000 7fa1eb7fe700 1 -- 172.21.15.143:0/1678301659 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f8003c60 con 0x56155537cb80 2023-11-08T19:25:09.546 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.532+0000 7fa213ae9b80 1 -- 172.21.15.143:0/1678301659 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56155537cb80 msgr2=0x56155537c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:09.546 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.532+0000 7fa213ae9b80 1 --2- 172.21.15.143:0/1678301659 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56155537cb80 0x56155537c240 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa1f8012000 tx=0x7fa1f8010000 comp rx=0 tx=0).stop 2023-11-08T19:25:09.547 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.532+0000 7fa213ae9b80 1 -- 172.21.15.143:0/1678301659 shutdown_connections 2023-11-08T19:25:09.547 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.532+0000 7fa213ae9b80 1 --2- 172.21.15.143:0/1678301659 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56155537cb80 0x56155537c240 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:09.547 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.532+0000 7fa213ae9b80 1 --2- 172.21.15.143:0/1678301659 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615553376f0 0x5615553a4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:09.547 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.532+0000 7fa213ae9b80 1 --2- 172.21.15.143:0/1678301659 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x561555337ad0 0x56155537bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:09.547 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.532+0000 7fa213ae9b80 1 -- 172.21.15.143:0/1678301659 >> 172.21.15.143:0/1678301659 conn(0x5615552b2660 msgr2=0x561555367b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:09.548 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.532+0000 7fa213ae9b80 1 -- 172.21.15.143:0/1678301659 shutdown_connections 2023-11-08T19:25:09.548 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.532+0000 7fa213ae9b80 1 -- 172.21.15.143:0/1678301659 wait complete. 2023-11-08T19:25:09.548 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.533+0000 7fa213ae9b80 1 -- start start 2023-11-08T19:25:09.548 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.533+0000 7fa213ae9b80 1 Processor -- start 2023-11-08T19:25:09.548 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.533+0000 7fa213ae9b80 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x5615553376f0 0x5615553d0250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:09.549 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.533+0000 7fa213ae9b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561555337ad0 0x56155536aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:09.549 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.533+0000 7fa213ae9b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56155537cb80 0x56155536a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:09.549 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.533+0000 7fa213ae9b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615553d0da0 con 0x561555337ad0 2023-11-08T19:25:09.549 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.533+0000 7fa213ae9b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615553685e0 con 0x56155537cb80 2023-11-08T19:25:09.549 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.534+0000 7fa213ae9b80 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x5615553688c0 con 0x5615553376f0 2023-11-08T19:25:09.549 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.534+0000 7fa213ae9b80 10 client.0 _ll_register_callbacks cb 0x7fa1f0024070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T19:25:09.550 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.534+0000 7fa204e8b700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561555337ad0 0x56155536aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:09.550 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.534+0000 7fa204e8b700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561555337ad0 0x56155536aea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49396/0 (socket says 172.21.15.143:49396) 2023-11-08T19:25:09.550 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.534+0000 7fa204e8b700 1 -- 172.21.15.143:0/1025413230 learned_addr learned my addr 172.21.15.143:0/1025413230 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:09.550 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.534+0000 7fa20568c700 1 --2- 172.21.15.143:0/1025413230 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x5615553376f0 0x5615553d0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:09.550 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.534+0000 7fa205e8d700 1 --2- 172.21.15.143:0/1025413230 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56155537cb80 0x56155536a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:09.551 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.534+0000 7fa204e8b700 1 -- 172.21.15.143:0/1025413230 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x5615553376f0 msgr2=0x5615553d0250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:09.551 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.534+0000 7fa204e8b700 1 --2- 172.21.15.143:0/1025413230 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x5615553376f0 0x5615553d0250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:09.551 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.534+0000 7fa204e8b700 1 -- 172.21.15.143:0/1025413230 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56155537cb80 msgr2=0x56155536a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:09.551 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.534+0000 7fa204e8b700 1 --2- 172.21.15.143:0/1025413230 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56155537cb80 0x56155536a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:09.551 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.534+0000 7fa204e8b700 1 -- 172.21.15.143:0/1025413230 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1f800be70 con 0x561555337ad0 2023-11-08T19:25:09.552 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.534+0000 7fa205e8d700 1 --2- 172.21.15.143:0/1025413230 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56155537cb80 0x56155536a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:25:09.552 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.534+0000 7fa204e8b700 1 --2- 172.21.15.143:0/1025413230 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561555337ad0 0x56155536aea0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc088ce0 tx=0x7fa1fc001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b5ed31c4f71ae00 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:09.552 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.534+0000 7fa1dffff700 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2023-11-08T19:25:09.552 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.534+0000 7fa1dffff700 1 -- 172.21.15.143:0/1025413230 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1fc05b950 con 0x561555337ad0 2023-11-08T19:25:09.552 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.534+0000 7fa1dffff700 1 -- 172.21.15.143:0/1025413230 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa1fc05fb00 con 0x561555337ad0 2023-11-08T19:25:09.553 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.534+0000 7fa213ae9b80 1 -- 172.21.15.143:0/1025413230 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa1f001a070 con 0x561555337ad0 2023-11-08T19:25:09.553 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.534+0000 7fa1dffff700 1 -- 172.21.15.143:0/1025413230 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa1fc0761a0 con 0x561555337ad0 2023-11-08T19:25:09.553 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.534+0000 7fa213ae9b80 10 client.4662 fetch_fsmap learned FSMap version 8 2023-11-08T19:25:09.553 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.534+0000 7fa213ae9b80 1 -- 172.21.15.143:0/1025413230 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x5615550e0200 con 0x561555337ad0 2023-11-08T19:25:09.553 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.534+0000 7fa1dffff700 1 -- 172.21.15.143:0/1025413230 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1f001a070 con 0x561555337ad0 2023-11-08T19:25:09.553 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.534+0000 7fa1dffff700 1 -- 172.21.15.143:0/1025413230 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 8) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fa1fc056cf0 con 0x561555337ad0 2023-11-08T19:25:09.554 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.534+0000 7fa213ae9b80 10 client.4662 fetch_fsmap finished waiting for FSMap version 8 2023-11-08T19:25:09.554 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.534+0000 7fa213ae9b80 10 client.4662 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-08T19:25:09.554 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.534+0000 7fa213ae9b80 20 client.4662 populate_metadata read hostname 'smithi143' 2023-11-08T19:25:09.554 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.534+0000 7fa213ae9b80 1 --2- 172.21.15.143:0/1025413230 >> [v2:172.21.15.174:6835/2124972405,v1:172.21.15.174:6837/2124972405] conn(0x56155537bda0 0x56155537c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:09.554 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.535+0000 7fa213ae9b80 4 client.4662 mds_command: new command op to 4282 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-08T19:25:09.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.535+0000 7fa213ae9b80 1 -- 172.21.15.143:0/1025413230 --> [v2:172.21.15.174:6835/2124972405,v1:172.21.15.174:6837/2124972405] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5615552895a0 con 0x56155537bda0 2023-11-08T19:25:09.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.535+0000 7fa20568c700 1 --2- 172.21.15.143:0/1025413230 >> [v2:172.21.15.174:6835/2124972405,v1:172.21.15.174:6837/2124972405] conn(0x56155537bda0 0x56155537c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:09.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.535+0000 7fa1dffff700 10 client.4662 ms_handle_connect on v2:172.21.15.174:6835/2124972405 2023-11-08T19:25:09.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.535+0000 7fa20568c700 1 --2- 172.21.15.143:0/1025413230 >> [v2:172.21.15.174:6835/2124972405,v1:172.21.15.174:6837/2124972405] conn(0x56155537bda0 0x56155537c180 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14ae8e4ecd47e6b4 server_cookie=d2b151a1a40aa5ca in_seq=0 out_seq=0 2023-11-08T19:25:09.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.535+0000 7fa1dffff700 1 -- 172.21.15.143:0/1025413230 <== mds.0 v2:172.21.15.174:6835/2124972405 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (crc 0 0 0) 0x5615552895a0 con 0x56155537bda0 2023-11-08T19:25:09.556 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.535+0000 7fa1dffff700 10 client.4662 handle_command_reply: tid=0 2023-11-08T19:25:09.556 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.535+0000 7fa213ae9b80 1 client.4662 shutdown 2023-11-08T19:25:09.556 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.535+0000 7fa213ae9b80 1 -- 172.21.15.143:0/1025413230 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561555337ad0 msgr2=0x56155536aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:09.556 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.535+0000 7fa213ae9b80 1 --2- 172.21.15.143:0/1025413230 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561555337ad0 0x56155536aea0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc088ce0 tx=0x7fa1fc001120 comp rx=0 tx=0).stop 2023-11-08T19:25:09.556 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.535+0000 7fa213ae9b80 1 -- 172.21.15.143:0/1025413230 shutdown_connections 2023-11-08T19:25:09.557 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.535+0000 7fa213ae9b80 1 --2- 172.21.15.143:0/1025413230 >> [v2:172.21.15.174:6835/2124972405,v1:172.21.15.174:6837/2124972405] conn(0x56155537bda0 0x56155537c180 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:09.557 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.535+0000 7fa213ae9b80 1 --2- 172.21.15.143:0/1025413230 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x5615553376f0 0x5615553d0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:09.557 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.535+0000 7fa213ae9b80 1 --2- 172.21.15.143:0/1025413230 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56155537cb80 0x56155536a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:09.557 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.535+0000 7fa1dffff700 0 client.4662 ms_handle_reset on v2:172.21.15.174:6835/2124972405 2023-11-08T19:25:09.557 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.535+0000 7fa213ae9b80 1 --2- 172.21.15.143:0/1025413230 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561555337ad0 0x56155536aea0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:09.558 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.535+0000 7fa213ae9b80 1 -- 172.21.15.143:0/1025413230 >> 172.21.15.143:0/1025413230 conn(0x5615552b2660 msgr2=0x56155538e8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:09.558 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.535+0000 7fa213ae9b80 1 -- 172.21.15.143:0/1025413230 shutdown_connections 2023-11-08T19:25:09.558 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.536+0000 7fa213ae9b80 1 -- 172.21.15.143:0/1025413230 wait complete. 2023-11-08T19:25:09.558 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.536+0000 7fa213ae9b80 20 client.4662 trim_cache size 0 max 16384 2023-11-08T19:25:09.559 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2023-11-08T19:25:09.559 INFO:teuthology.orchestra.run.smithi143.stdout: "cluster_fsid": "968483af-77fc-4068-ad7c-5c2a1a4613d7", 2023-11-08T19:25:09.559 INFO:teuthology.orchestra.run.smithi143.stdout: "whoami": 0, 2023-11-08T19:25:09.559 INFO:teuthology.orchestra.run.smithi143.stdout: "id": 4282, 2023-11-08T19:25:09.559 INFO:teuthology.orchestra.run.smithi143.stdout: "want_state": "up:active", 2023-11-08T19:25:09.560 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "up:active", 2023-11-08T19:25:09.560 INFO:teuthology.orchestra.run.smithi143.stdout: "fs_name": "cephfs", 2023-11-08T19:25:09.560 INFO:teuthology.orchestra.run.smithi143.stdout: "rank_uptime": 18.914858520999999, 2023-11-08T19:25:09.560 INFO:teuthology.orchestra.run.smithi143.stdout: "mdsmap_epoch": 8, 2023-11-08T19:25:09.560 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_epoch": 20, 2023-11-08T19:25:09.561 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_epoch_barrier": 20, 2023-11-08T19:25:09.561 INFO:teuthology.orchestra.run.smithi143.stdout: "uptime": 25.669805754999999 2023-11-08T19:25:09.561 INFO:teuthology.orchestra.run.smithi143.stdout:} 2023-11-08T19:25:09.562 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.536+0000 7fa2078f0700 1 -- 172.21.15.143:0/3535133102 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa1e406af00 msgr2=0x7fa1e406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:09.563 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.536+0000 7fa2078f0700 1 --2- 172.21.15.143:0/3535133102 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa1e406af00 0x7fa1e406d3c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa1f0009ac0 tx=0x7fa1f0008040 comp rx=0 tx=0).stop 2023-11-08T19:25:09.563 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.536+0000 7fa2078f0700 1 -- 172.21.15.143:0/3535133102 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa2000d3690 msgr2=0x7fa2000c76e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:09.563 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.536+0000 7fa2078f0700 1 --2- 172.21.15.143:0/3535133102 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa2000d3690 0x7fa2000c76e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc00c4b0 tx=0x7fa1fc00c870 comp rx=0 tx=0).stop 2023-11-08T19:25:09.563 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.536+0000 7fa2078f0700 1 -- 172.21.15.143:0/3535133102 shutdown_connections 2023-11-08T19:25:09.563 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.536+0000 7fa2078f0700 1 --2- 172.21.15.143:0/3535133102 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa1e406af00 0x7fa1e406d3c0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:09.564 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.536+0000 7fa2078f0700 1 --2- 172.21.15.143:0/3535133102 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa2000cf090 0x7fa2001f3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:09.564 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.536+0000 7fa2078f0700 1 --2- 172.21.15.143:0/3535133102 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa20015dc80 0x7fa2000c7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:09.564 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.536+0000 7fa2078f0700 1 --2- 172.21.15.143:0/3535133102 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa2000d3690 0x7fa2000c76e0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:09.564 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.536+0000 7fa2078f0700 1 -- 172.21.15.143:0/3535133102 >> 172.21.15.143:0/3535133102 conn(0x7fa2000668e0 msgr2=0x7fa20015c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:09.564 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.536+0000 7fa2078f0700 1 -- 172.21.15.143:0/3535133102 shutdown_connections 2023-11-08T19:25:09.564 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:09.536+0000 7fa2078f0700 1 -- 172.21.15.143:0/3535133102 wait complete. 2023-11-08T19:25:09.565 INFO:teuthology.run_tasks:Running task kclient... 2023-11-08T19:25:09.588 INFO:tasks.kclient:Mounting kernel clients... 2023-11-08T19:25:09.588 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2023-11-08T19:25:09.589 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:09.589 DEBUG:teuthology.orchestra.run.smithi143:> ip netns list 2023-11-08T19:25:09.606 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:09.607 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip link delete ceph-brx 2023-11-08T19:25:09.675 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-08T19:25:09.676 INFO:teuthology.orchestra.run.smithi143.stderr:Cannot find device "ceph-brx" 2023-11-08T19:25:09.677 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:09.677 DEBUG:teuthology.orchestra.run.smithi143:> ip netns list 2023-11-08T19:25:09.734 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:09.734 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip link delete ceph-brx 2023-11-08T19:25:09.803 INFO:teuthology.orchestra.run.smithi143.stderr:Cannot find device "ceph-brx" 2023-11-08T19:25:09.804 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-08T19:25:09.805 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:09.805 DEBUG:teuthology.orchestra.run.smithi174:> ip netns list 2023-11-08T19:25:09.826 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:09.827 DEBUG:teuthology.orchestra.run.smithi174:> sudo ip link delete ceph-brx 2023-11-08T19:25:09.901 INFO:teuthology.orchestra.run.smithi174.stderr:Cannot find device "ceph-brx" 2023-11-08T19:25:09.902 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-08T19:25:09.903 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:09.904 DEBUG:teuthology.orchestra.run.smithi174:> ip netns list 2023-11-08T19:25:09.963 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:09.963 DEBUG:teuthology.orchestra.run.smithi174:> sudo ip link delete ceph-brx 2023-11-08T19:25:10.033 INFO:teuthology.orchestra.run.smithi174.stderr:Cannot find device "ceph-brx" 2023-11-08T19:25:10.034 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-08T19:25:10.035 INFO:tasks.kclient:client.0 config is {'syntax': 'v1'} 2023-11-08T19:25:10.035 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-08T19:25:10.036 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-08T19:25:10.036 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-08T19:25:10.157 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:10.155+0000 7f1daef43700 1 Processor -- start 2023-11-08T19:25:10.157 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:10.156+0000 7f1daef43700 1 -- start start 2023-11-08T19:25:10.160 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:10.156+0000 7f1daef43700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f1da80d1c70 0x7f1da80b14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:10.160 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:10.156+0000 7f1daef43700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1da80b1bc0 con 0x7f1da815ba30 2023-11-08T19:25:10.160 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:10.156+0000 7f1daef43700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1da80b1ea0 con 0x7f1da80d03a0 2023-11-08T19:25:10.161 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:10.156+0000 7f1daef43700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1da80b2180 con 0x7f1da80d1c70 2023-11-08T19:25:10.161 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:10.156+0000 7f1daccdf700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f1da815ba30 0x7f1da80d1560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:45556/0 (socket says 172.21.15.143:45556) 2023-11-08T19:25:10.161 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:10.156+0000 7f1daccdf700 1 -- 172.21.15.143:0/2958209024 learned_addr learned my addr 172.21.15.143:0/2958209024 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:25:10.161 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:10.156+0000 7f1da7fff700 1 --2- 172.21.15.143:0/2958209024 >> v2:172.21.15.143:3300/0 conn(0x7f1da80d1c70 0x7f1da80b14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:10.162 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:10.157+0000 7f1da77fe700 1 -- 172.21.15.143:0/2958209024 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 341673072 0 0) 0x7f1da80b1bc0 con 0x7f1da815ba30 2023-11-08T19:25:10.162 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:10.157+0000 7f1da77fe700 1 -- 172.21.15.143:0/2958209024 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d980035b0 con 0x7f1da815ba30 2023-11-08T19:25:10.162 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:10.157+0000 7f1da7fff700 1 -- 172.21.15.143:0/2958209024 >> v1:172.21.15.174:6789/0 conn(0x7f1da80d03a0 legacy=0x7f1da80cc5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:10.162 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:10.157+0000 7f1da7fff700 1 -- 172.21.15.143:0/2958209024 >> v1:172.21.15.143:6789/0 conn(0x7f1da815ba30 legacy=0x7f1da80d1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:10.162 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:10.157+0000 7f1da7fff700 1 -- 172.21.15.143:0/2958209024 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1da80b2460 con 0x7f1da80d1c70 2023-11-08T19:25:10.163 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:10.157+0000 7f1da7fff700 1 --2- 172.21.15.143:0/2958209024 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1da80d1c70 0x7f1da80b14c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1d94009840 tx=0x7f1d94009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9de5c775397f35f9 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:10.163 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:10.157+0000 7f1da77fe700 1 -- 172.21.15.143:0/2958209024 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d94004230 con 0x7f1da80d1c70 2023-11-08T19:25:10.163 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:10.157+0000 7f1da77fe700 1 -- 172.21.15.143:0/2958209024 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d94004390 con 0x7f1da80d1c70 2023-11-08T19:25:10.163 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:10.157+0000 7f1da77fe700 1 -- 172.21.15.143:0/2958209024 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d940036a0 con 0x7f1da80d1c70 2023-11-08T19:25:10.163 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:10.158+0000 7f1daef43700 1 -- 172.21.15.143:0/2958209024 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1da80d1c70 msgr2=0x7f1da80b14c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:10.164 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:10.158+0000 7f1daef43700 1 --2- 172.21.15.143:0/2958209024 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1da80d1c70 0x7f1da80b14c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1d94009840 tx=0x7f1d94009b50 comp rx=0 tx=0).stop 2023-11-08T19:25:10.164 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:10.158+0000 7f1daef43700 1 -- 172.21.15.143:0/2958209024 shutdown_connections 2023-11-08T19:25:10.164 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:10.158+0000 7f1daef43700 1 --2- 172.21.15.143:0/2958209024 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1da80d1c70 0x7f1da80b14c0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:10.164 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:10.158+0000 7f1daef43700 1 -- 172.21.15.143:0/2958209024 >> 172.21.15.143:0/2958209024 conn(0x7f1da808b000 msgr2=0x7f1da809ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:10.165 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:10.158+0000 7f1daef43700 1 -- 172.21.15.143:0/2958209024 shutdown_connections 2023-11-08T19:25:10.165 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:10.158+0000 7f1daef43700 1 -- 172.21.15.143:0/2958209024 wait complete. 2023-11-08T19:25:10.165 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:10.159+0000 7f1daef43700 1 Processor -- start 2023-11-08T19:25:10.165 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:10.159+0000 7f1daef43700 1 -- start start 2023-11-08T19:25:10.169 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:10.160+0000 7f1daef43700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1da80d03a0 0x7f1da815a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:10.169 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:10.160+0000 7f1daef43700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1da80d1c70 0x7f1da815aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:10.169 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:10.160+0000 7f1daef43700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1da815ba30 0x7f1da815dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:10.169 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:10.160+0000 7f1daef43700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1da815af50 con 0x7f1da80d03a0 2023-11-08T19:25:10.170 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:10.160+0000 7f1daef43700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1da80c5640 con 0x7f1da815ba30 2023-11-08T19:25:10.170 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:10.160+0000 7f1daef43700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1da815c080 con 0x7f1da80d1c70 2023-11-08T19:25:10.170 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:10.160+0000 7f1daccdf700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1da80d03a0 0x7f1da815a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:10.170 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:10.160+0000 7f1daccdf700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1da80d03a0 0x7f1da815a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49408/0 (socket says 172.21.15.143:49408) 2023-11-08T19:25:10.171 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:10.160+0000 7f1daccdf700 1 -- 172.21.15.143:0/929771988 learned_addr learned my addr 172.21.15.143:0/929771988 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:10.171 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:10.160+0000 7f1dad4e0700 1 --2- 172.21.15.143:0/929771988 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1da815ba30 0x7f1da815dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:10.171 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:10.160+0000 7f1da7fff700 1 --2- 172.21.15.143:0/929771988 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1da80d1c70 0x7f1da815aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:10.171 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:10.160+0000 7f1daccdf700 1 -- 172.21.15.143:0/929771988 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1da80d1c70 msgr2=0x7f1da815aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:10.172 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:10.160+0000 7f1daccdf700 1 --2- 172.21.15.143:0/929771988 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1da80d1c70 0x7f1da815aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:10.172 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:10.160+0000 7f1daccdf700 1 -- 172.21.15.143:0/929771988 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1da815ba30 msgr2=0x7f1da815dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:10.172 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:10.160+0000 7f1daccdf700 1 --2- 172.21.15.143:0/929771988 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1da815ba30 0x7f1da815dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:10.172 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:10.160+0000 7f1daccdf700 1 -- 172.21.15.143:0/929771988 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d940094a0 con 0x7f1da80d03a0 2023-11-08T19:25:10.173 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:10.160+0000 7f1da7fff700 1 --2- 172.21.15.143:0/929771988 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1da80d1c70 0x7f1da815aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:25:10.173 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:10.160+0000 7f1daccdf700 1 --2- 172.21.15.143:0/929771988 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1da80d03a0 0x7f1da815a4d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c00c700 tx=0x7f1d9c00cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:10.173 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:10.160+0000 7f1da57fa700 1 -- 172.21.15.143:0/929771988 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d9c005d70 con 0x7f1da80d03a0 2023-11-08T19:25:10.173 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:10.160+0000 7f1da57fa700 1 -- 172.21.15.143:0/929771988 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d9c0070d0 con 0x7f1da80d03a0 2023-11-08T19:25:10.174 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:10.160+0000 7f1da57fa700 1 -- 172.21.15.143:0/929771988 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d9c007460 con 0x7f1da80d03a0 2023-11-08T19:25:10.174 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:10.160+0000 7f1daef43700 1 -- 172.21.15.143:0/929771988 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1da80c8260 con 0x7f1da80d03a0 2023-11-08T19:25:10.174 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:10.160+0000 7f1daef43700 1 -- 172.21.15.143:0/929771988 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1da80c5b70 con 0x7f1da80d03a0 2023-11-08T19:25:10.175 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:10.161+0000 7f1daef43700 1 -- 172.21.15.143:0/929771988 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1da804eee0 con 0x7f1da80d03a0 2023-11-08T19:25:10.175 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:10.162+0000 7f1da57fa700 1 -- 172.21.15.143:0/929771988 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1d9c0075c0 con 0x7f1da80d03a0 2023-11-08T19:25:10.175 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:10.162+0000 7f1da57fa700 1 --2- 172.21.15.143:0/929771988 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f1d8c06af00 0x7f1d8c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:10.176 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:10.162+0000 7f1da57fa700 1 -- 172.21.15.143:0/929771988 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1d9c088bb0 con 0x7f1da80d03a0 2023-11-08T19:25:10.176 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:10.162+0000 7f1da7fff700 1 --2- 172.21.15.143:0/929771988 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f1d8c06af00 0x7f1d8c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:10.176 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:10.163+0000 7f1da7fff700 1 --2- 172.21.15.143:0/929771988 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f1d8c06af00 0x7f1d8c06d3c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f1d94009840 tx=0x7f1d94008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:10.177 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:10.166+0000 7f1da57fa700 1 -- 172.21.15.143:0/929771988 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1d9c053c70 con 0x7f1da80d03a0 2023-11-08T19:25:10.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:10.399+0000 7f1daef43700 1 -- 172.21.15.143:0/929771988 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f1da80d0ff0 con 0x7f1da80d03a0 2023-11-08T19:25:10.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:10.399+0000 7f1da57fa700 1 -- 172.21.15.143:0/929771988 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f1d9c057290 con 0x7f1da80d03a0 2023-11-08T19:25:10.401 INFO:teuthology.orchestra.run.smithi143.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-08T19:25:10.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:10.401+0000 7f1daef43700 1 -- 172.21.15.143:0/929771988 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f1d8c06af00 msgr2=0x7f1d8c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:10.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:10.401+0000 7f1daef43700 1 --2- 172.21.15.143:0/929771988 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f1d8c06af00 0x7f1d8c06d3c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f1d94009840 tx=0x7f1d94008030 comp rx=0 tx=0).stop 2023-11-08T19:25:10.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:10.401+0000 7f1daef43700 1 -- 172.21.15.143:0/929771988 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1da80d03a0 msgr2=0x7f1da815a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:10.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:10.401+0000 7f1daef43700 1 --2- 172.21.15.143:0/929771988 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1da80d03a0 0x7f1da815a4d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c00c700 tx=0x7f1d9c00cac0 comp rx=0 tx=0).stop 2023-11-08T19:25:10.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:10.401+0000 7f1daef43700 1 -- 172.21.15.143:0/929771988 shutdown_connections 2023-11-08T19:25:10.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:10.401+0000 7f1daef43700 1 --2- 172.21.15.143:0/929771988 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f1d8c06af00 0x7f1d8c06d3c0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:10.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:10.401+0000 7f1daef43700 1 --2- 172.21.15.143:0/929771988 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1da815ba30 0x7f1da815dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:10.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:10.401+0000 7f1daef43700 1 --2- 172.21.15.143:0/929771988 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1da80d03a0 0x7f1da815a4d0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:10.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:10.401+0000 7f1daef43700 1 --2- 172.21.15.143:0/929771988 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1da80d1c70 0x7f1da815aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:10.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:10.401+0000 7f1daef43700 1 -- 172.21.15.143:0/929771988 >> 172.21.15.143:0/929771988 conn(0x7f1da808b000 msgr2=0x7f1da80ae480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:10.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:10.401+0000 7f1daef43700 1 -- 172.21.15.143:0/929771988 shutdown_connections 2023-11-08T19:25:10.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:10.401+0000 7f1daef43700 1 -- 172.21.15.143:0/929771988 wait complete. 2023-11-08T19:25:10.415 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-08T19:25:10.415 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-08T19:25:10.416 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi143.front.sepia.ceph.com 2023-11-08T19:25:10.416 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-08T19:25:10.416 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-08T19:25:10.417 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-08T19:25:10.417 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-08T19:25:10.417 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-08T19:25:10.418 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-08T19:25:10.418 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:10.419 DEBUG:teuthology.orchestra.run.smithi143:> ip addr 2023-11-08T19:25:10.435 INFO:teuthology.orchestra.run.smithi143.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-08T19:25:10.436 INFO:teuthology.orchestra.run.smithi143.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-08T19:25:10.436 INFO:teuthology.orchestra.run.smithi143.stdout: inet 127.0.0.1/8 scope host lo 2023-11-08T19:25:10.436 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:25:10.436 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 ::1/128 scope host 2023-11-08T19:25:10.437 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:25:10.437 INFO:teuthology.orchestra.run.smithi143.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-08T19:25:10.437 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:d9:3b:94 brd ff:ff:ff:ff:ff:ff 2023-11-08T19:25:10.437 INFO:teuthology.orchestra.run.smithi143.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-08T19:25:10.438 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:d9:3b:95 brd ff:ff:ff:ff:ff:ff 2023-11-08T19:25:10.438 INFO:teuthology.orchestra.run.smithi143.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-08T19:25:10.438 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:88:77:82 brd ff:ff:ff:ff:ff:ff 2023-11-08T19:25:10.438 INFO:teuthology.orchestra.run.smithi143.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-08T19:25:10.439 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:88:77:83 brd ff:ff:ff:ff:ff:ff 2023-11-08T19:25:10.439 INFO:teuthology.orchestra.run.smithi143.stdout: inet 172.21.15.143/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-08T19:25:10.439 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft 42357sec preferred_lft 42357sec 2023-11-08T19:25:10.439 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 fe80::e628:d259:2f76:52a6/64 scope link noprefixroute 2023-11-08T19:25:10.440 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:25:10.440 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-11-08T19:25:10.441 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:10.441 DEBUG:teuthology.orchestra.run.smithi143:> (cd / && exec bash -c ' 2023-11-08T19:25:10.441 DEBUG:teuthology.orchestra.run.smithi143:> set -e 2023-11-08T19:25:10.442 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip link add name ceph-brx type bridge 2023-11-08T19:25:10.442 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip addr flush dev ceph-brx 2023-11-08T19:25:10.443 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip link set ceph-brx up 2023-11-08T19:25:10.443 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-11-08T19:25:10.443 DEBUG:teuthology.orchestra.run.smithi143:> ') 2023-11-08T19:25:10.609 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:10.610 DEBUG:teuthology.orchestra.run.smithi143:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-08T19:25:10.637 INFO:teuthology.orchestra.run.smithi143.stdout:1 2023-11-08T19:25:10.639 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:10.639 DEBUG:teuthology.orchestra.run.smithi143:> route 2023-11-08T19:25:10.714 INFO:teuthology.orchestra.run.smithi143.stdout:Kernel IP routing table 2023-11-08T19:25:10.714 INFO:teuthology.orchestra.run.smithi143.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-08T19:25:10.715 INFO:teuthology.orchestra.run.smithi143.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-08T19:25:10.715 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-08T19:25:10.715 INFO:teuthology.orchestra.run.smithi143.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-08T19:25:10.716 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:10.716 DEBUG:teuthology.orchestra.run.smithi143:> (cd / && exec bash -c ' 2023-11-08T19:25:10.716 DEBUG:teuthology.orchestra.run.smithi143:> set -e 2023-11-08T19:25:10.717 DEBUG:teuthology.orchestra.run.smithi143:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-08T19:25:10.717 DEBUG:teuthology.orchestra.run.smithi143:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-08T19:25:10.717 DEBUG:teuthology.orchestra.run.smithi143:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-08T19:25:10.718 DEBUG:teuthology.orchestra.run.smithi143:> ') 2023-11-08T19:25:11.021 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:11.022 DEBUG:teuthology.orchestra.run.smithi143:> ip netns list 2023-11-08T19:25:11.039 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:11.039 DEBUG:teuthology.orchestra.run.smithi143:> ip netns list-id 2023-11-08T19:25:11.095 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:11.096 DEBUG:teuthology.orchestra.run.smithi143:> (cd / && exec bash -c ' 2023-11-08T19:25:11.096 DEBUG:teuthology.orchestra.run.smithi143:> set -e 2023-11-08T19:25:11.096 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-08T19:25:11.097 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-11-08T19:25:11.097 DEBUG:teuthology.orchestra.run.smithi143:> ') 2023-11-08T19:25:11.190 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-11-08T19:25:11.191 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:11.191 DEBUG:teuthology.orchestra.run.smithi143:> (cd / && exec bash -c ' 2023-11-08T19:25:11.192 DEBUG:teuthology.orchestra.run.smithi143:> set -e 2023-11-08T19:25:11.192 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-11-08T19:25:11.192 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-11-08T19:25:11.193 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-11-08T19:25:11.193 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-11-08T19:25:11.193 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-11-08T19:25:11.193 DEBUG:teuthology.orchestra.run.smithi143:> ') 2023-11-08T19:25:11.397 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:11.397 DEBUG:teuthology.orchestra.run.smithi143:> (cd / && exec bash -c ' 2023-11-08T19:25:11.397 DEBUG:teuthology.orchestra.run.smithi143:> set -e 2023-11-08T19:25:11.398 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip link set brx.0 up 2023-11-08T19:25:11.398 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip link set dev brx.0 master ceph-brx 2023-11-08T19:25:11.398 DEBUG:teuthology.orchestra.run.smithi143:> ') 2023-11-08T19:25:11.507 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-08T19:25:11.507 DEBUG:teuthology.orchestra.run.smithi143:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-08T19:25:11.528 INFO:teuthology.orchestra.run.smithi143.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-08T19:25:11.529 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-08T19:25:11.529 DEBUG:teuthology.orchestra.run.smithi143:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-08T19:25:11.585 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-11-08T19:25:11.585 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:11.586 DEBUG:teuthology.orchestra.run.smithi143:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-08T19:25:11.696 INFO:teuthology.orchestra.run.smithi143.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-08T19:25:11.697 INFO:teuthology.orchestra.run.smithi143.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-11-08T19:25:11.697 INFO:teuthology.orchestra.run.smithi143.stdout:invalid new device string format 2023-11-08T19:25:11.795 INFO:teuthology.orchestra.run.smithi143.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-08T19:25:11.795 INFO:teuthology.orchestra.run.smithi143.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-11-08T19:25:11.795 INFO:teuthology.orchestra.run.smithi143.stdout:invalid new device string format 2023-11-08T19:25:11.796 INFO:teuthology.orchestra.run.smithi143.stdout:mount.ceph: resolved to: "172.21.15.143:3300,172.21.15.174:3300,172.21.15.174:3301" 2023-11-08T19:25:11.796 INFO:teuthology.orchestra.run.smithi143.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-08T19:25:11.828 INFO:teuthology.orchestra.run.smithi143.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-08T19:25:11.828 INFO:teuthology.orchestra.run.smithi143.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-11-08T19:25:11.828 INFO:teuthology.orchestra.run.smithi143.stdout:invalid new device string format 2023-11-08T19:25:11.829 INFO:teuthology.orchestra.run.smithi143.stdout:mount.ceph: resolved to: "172.21.15.143:3300,172.21.15.174:3300,172.21.15.174:3301" 2023-11-08T19:25:11.829 INFO:teuthology.orchestra.run.smithi143.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-08T19:25:11.829 INFO:teuthology.orchestra.run.smithi143.stdout:mount.ceph: resolved to: "172.21.15.143:6789,172.21.15.174:6789,172.21.15.174:6790" 2023-11-08T19:25:11.829 INFO:teuthology.orchestra.run.smithi143.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.143:6789,172.21.15.174:6789,172.21.15.174:6790:/ 2023-11-08T19:25:11.830 INFO:teuthology.orchestra.run.smithi143.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=968483af-77fc-4068-ad7c-5c2a1a4613d7" will pass to kernel 2023-11-08T19:25:11.832 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-08T19:25:11.832 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:11.833 DEBUG:teuthology.orchestra.run.smithi143:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-08T19:25:11.865 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:11.866 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-08T19:25:11.954 INFO:teuthology.orchestra.run.smithi143.stdout:client4674 2023-11-08T19:25:11.955 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-08T19:25:11.955 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:25:11.973 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:11.973 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-08T19:25:12.061 INFO:teuthology.orchestra.run.smithi143.stdout:client4674 2023-11-08T19:25:12.061 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:12.062 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-08T19:25:12.110 INFO:teuthology.orchestra.run.smithi143.stdout:client4674 2023-11-08T19:25:12.111 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:12.111 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/968483af-77fc-4068-ad7c-5c2a1a4613d7.client4674/status') 2023-11-08T19:25:12.164 INFO:teuthology.orchestra.run.smithi143.stdout:instance: client.4674 (3)192.168.0.1:0/1090043387 2023-11-08T19:25:12.165 INFO:teuthology.orchestra.run.smithi143.stdout:blocklisted: false 2023-11-08T19:25:12.166 INFO:teuthology.orchestra.run.smithi143.stderr:0+1 records in 2023-11-08T19:25:12.166 INFO:teuthology.orchestra.run.smithi143.stderr:0+1 records out 2023-11-08T19:25:12.166 INFO:teuthology.orchestra.run.smithi143.stderr:69 bytes copied, 2.1805e-05 s, 3.2 MB/s 2023-11-08T19:25:12.166 INFO:tasks.kclient:client.2 config is {'syntax': 'v1'} 2023-11-08T19:25:12.167 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-08T19:25:12.167 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-11-08T19:25:12.167 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-08T19:25:12.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:12.287+0000 7f8af21f5700 1 Processor -- start 2023-11-08T19:25:12.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:12.287+0000 7f8af21f5700 1 -- start start 2023-11-08T19:25:12.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:12.288+0000 7f8af21f5700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f8aec15dd00 0x7f8aec0cca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:12.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:12.288+0000 7f8af21f5700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8aec06b320 0x7f8aec0cb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:12.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:12.288+0000 7f8af21f5700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8aec0ccfb0 con 0x7f8aec1618f0 2023-11-08T19:25:12.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:12.288+0000 7f8af21f5700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8aec0cad50 con 0x7f8aec15dd00 2023-11-08T19:25:12.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:12.288+0000 7f8af21f5700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8aec161df0 con 0x7f8aec06b320 2023-11-08T19:25:12.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:12.288+0000 7f8aeb7fe700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f8aec15dd00 0x7f8aec0cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:12.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:12.288+0000 7f8aeaffd700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f8aec1618f0 0x7f8aec06ac10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:45580/0 (socket says 172.21.15.143:45580) 2023-11-08T19:25:12.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:12.288+0000 7f8aeaffd700 1 -- 172.21.15.143:0/287410098 learned_addr learned my addr 172.21.15.143:0/287410098 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:25:12.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:12.288+0000 7f8aebfff700 1 --2- 172.21.15.143:0/287410098 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8aec06b320 0x7f8aec0cb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:12.293 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:12.288+0000 7f8aeb7fe700 1 -- 172.21.15.143:0/287410098 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8aec06b320 msgr2=0x7f8aec0cb1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:12.293 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:12.288+0000 7f8aeb7fe700 1 --2- 172.21.15.143:0/287410098 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8aec06b320 0x7f8aec0cb1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:12.293 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:12.288+0000 7f8aeb7fe700 1 -- 172.21.15.143:0/287410098 >> v1:172.21.15.143:6789/0 conn(0x7f8aec1618f0 legacy=0x7f8aec06ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:12.294 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:12.288+0000 7f8aeb7fe700 1 -- 172.21.15.143:0/287410098 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8aec1620d0 con 0x7f8aec15dd00 2023-11-08T19:25:12.294 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:12.288+0000 7f8aea7fc700 1 -- 172.21.15.143:0/287410098 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2217985051 0 0) 0x7f8aec0ccfb0 con 0x7f8aec1618f0 2023-11-08T19:25:12.294 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:12.288+0000 7f8aeb7fe700 1 --2- 172.21.15.143:0/287410098 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8aec15dd00 0x7f8aec0cca70 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f8ad4009820 tx=0x7f8ad4009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=766035a873467de0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:12.294 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:12.289+0000 7f8aea7fc700 1 -- 172.21.15.143:0/287410098 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad4005e30 con 0x7f8aec15dd00 2023-11-08T19:25:12.294 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:12.289+0000 7f8aea7fc700 1 -- 172.21.15.143:0/287410098 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ad4006030 con 0x7f8aec15dd00 2023-11-08T19:25:12.295 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:12.289+0000 7f8aea7fc700 1 -- 172.21.15.143:0/287410098 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad40063e0 con 0x7f8aec15dd00 2023-11-08T19:25:12.295 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:12.289+0000 7f8af21f5700 1 -- 172.21.15.143:0/287410098 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8aec15dd00 msgr2=0x7f8aec0cca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:12.295 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:12.289+0000 7f8af21f5700 1 --2- 172.21.15.143:0/287410098 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8aec15dd00 0x7f8aec0cca70 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f8ad4009820 tx=0x7f8ad4009b30 comp rx=0 tx=0).stop 2023-11-08T19:25:12.295 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:12.289+0000 7f8af21f5700 1 -- 172.21.15.143:0/287410098 shutdown_connections 2023-11-08T19:25:12.295 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:12.289+0000 7f8af21f5700 1 --2- 172.21.15.143:0/287410098 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8aec06b320 0x7f8aec0cb1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:12.296 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:12.289+0000 7f8af21f5700 1 --2- 172.21.15.143:0/287410098 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8aec15dd00 0x7f8aec0cca70 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:12.296 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:12.289+0000 7f8af21f5700 1 -- 172.21.15.143:0/287410098 >> 172.21.15.143:0/287410098 conn(0x7f8aec099ab0 msgr2=0x7f8aec0d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:12.296 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:12.289+0000 7f8af21f5700 1 -- 172.21.15.143:0/287410098 shutdown_connections 2023-11-08T19:25:12.296 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:12.289+0000 7f8af21f5700 1 -- 172.21.15.143:0/287410098 wait complete. 2023-11-08T19:25:12.299 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:12.290+0000 7f8af21f5700 1 Processor -- start 2023-11-08T19:25:12.300 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:12.291+0000 7f8af21f5700 1 -- start start 2023-11-08T19:25:12.300 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:12.291+0000 7f8af21f5700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8aec06b320 0x7f8aec1f4960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:12.300 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:12.291+0000 7f8af21f5700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8aec15dd00 0x7f8aec154c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:12.300 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:12.291+0000 7f8af21f5700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8aec1618f0 0x7f8aec0c6600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:12.300 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:12.291+0000 7f8af21f5700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aec155140 con 0x7f8aec06b320 2023-11-08T19:25:12.301 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:12.291+0000 7f8af21f5700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aec1f4690 con 0x7f8aec15dd00 2023-11-08T19:25:12.301 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:12.291+0000 7f8af21f5700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8aec1f3a80 con 0x7f8aec1618f0 2023-11-08T19:25:12.301 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:12.291+0000 7f8aeb7fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8aec06b320 0x7f8aec1f4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:12.301 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:12.291+0000 7f8aeb7fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8aec06b320 0x7f8aec1f4960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49428/0 (socket says 172.21.15.143:49428) 2023-11-08T19:25:12.301 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:12.291+0000 7f8aeb7fe700 1 -- 172.21.15.143:0/2555903764 learned_addr learned my addr 172.21.15.143:0/2555903764 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:12.302 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:12.292+0000 7f8aebfff700 1 --2- 172.21.15.143:0/2555903764 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8aec1618f0 0x7f8aec0c6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:12.302 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:12.292+0000 7f8aeb7fe700 1 -- 172.21.15.143:0/2555903764 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8aec1618f0 msgr2=0x7f8aec0c6600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:12.302 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:12.292+0000 7f8aeb7fe700 1 --2- 172.21.15.143:0/2555903764 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8aec1618f0 0x7f8aec0c6600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:12.302 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:12.292+0000 7f8aeb7fe700 1 -- 172.21.15.143:0/2555903764 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8aec15dd00 msgr2=0x7f8aec154c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:12.302 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:12.292+0000 7f8aeaffd700 1 --2- 172.21.15.143:0/2555903764 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8aec15dd00 0x7f8aec154c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:12.303 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:12.292+0000 7f8aeb7fe700 1 --2- 172.21.15.143:0/2555903764 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8aec15dd00 0x7f8aec154c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:12.303 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:12.292+0000 7f8aeb7fe700 1 -- 172.21.15.143:0/2555903764 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ad4009430 con 0x7f8aec06b320 2023-11-08T19:25:12.303 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:12.292+0000 7f8aeb7fe700 1 --2- 172.21.15.143:0/2555903764 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8aec06b320 0x7f8aec1f4960 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8ad4013040 tx=0x7f8ad4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:12.303 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:12.292+0000 7f8ae3fff700 1 -- 172.21.15.143:0/2555903764 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad4025430 con 0x7f8aec06b320 2023-11-08T19:25:12.303 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:12.292+0000 7f8ae3fff700 1 -- 172.21.15.143:0/2555903764 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ad4020070 con 0x7f8aec06b320 2023-11-08T19:25:12.304 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:12.292+0000 7f8af21f5700 1 -- 172.21.15.143:0/2555903764 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8aec1f3d60 con 0x7f8aec06b320 2023-11-08T19:25:12.304 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:12.292+0000 7f8ae3fff700 1 -- 172.21.15.143:0/2555903764 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad4025810 con 0x7f8aec06b320 2023-11-08T19:25:12.304 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:12.292+0000 7f8af21f5700 1 -- 172.21.15.143:0/2555903764 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8aec1f4320 con 0x7f8aec06b320 2023-11-08T19:25:12.304 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:12.293+0000 7f8af21f5700 1 -- 172.21.15.143:0/2555903764 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8aec0aff60 con 0x7f8aec06b320 2023-11-08T19:25:12.304 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:12.293+0000 7f8ae3fff700 1 -- 172.21.15.143:0/2555903764 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8ad4022030 con 0x7f8aec06b320 2023-11-08T19:25:12.304 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:12.294+0000 7f8ae3fff700 1 --2- 172.21.15.143:0/2555903764 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f8acc06af00 0x7f8acc06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:12.305 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:12.294+0000 7f8ae3fff700 1 -- 172.21.15.143:0/2555903764 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8ad408abd0 con 0x7f8aec06b320 2023-11-08T19:25:12.305 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:12.294+0000 7f8aeaffd700 1 --2- 172.21.15.143:0/2555903764 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f8acc06af00 0x7f8acc06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:12.305 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:12.295+0000 7f8aeaffd700 1 --2- 172.21.15.143:0/2555903764 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f8acc06af00 0x7f8acc06d3c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8aec06f4e0 tx=0x7f8adc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:12.305 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:12.298+0000 7f8ae3fff700 1 -- 172.21.15.143:0/2555903764 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8ad4055d10 con 0x7f8aec06b320 2023-11-08T19:25:12.532 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:12.530+0000 7f8af21f5700 1 -- 172.21.15.143:0/2555903764 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f8aec04eee0 con 0x7f8aec06b320 2023-11-08T19:25:12.532 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:12.531+0000 7f8ae3fff700 1 -- 172.21.15.143:0/2555903764 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f8ad4059330 con 0x7f8aec06b320 2023-11-08T19:25:12.532 INFO:teuthology.orchestra.run.smithi143.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-08T19:25:12.534 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:12.533+0000 7f8af21f5700 1 -- 172.21.15.143:0/2555903764 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f8acc06af00 msgr2=0x7f8acc06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:12.534 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:12.533+0000 7f8af21f5700 1 --2- 172.21.15.143:0/2555903764 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f8acc06af00 0x7f8acc06d3c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8aec06f4e0 tx=0x7f8adc009040 comp rx=0 tx=0).stop 2023-11-08T19:25:12.535 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:12.533+0000 7f8af21f5700 1 -- 172.21.15.143:0/2555903764 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8aec06b320 msgr2=0x7f8aec1f4960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:12.535 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:12.533+0000 7f8af21f5700 1 --2- 172.21.15.143:0/2555903764 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8aec06b320 0x7f8aec1f4960 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8ad4013040 tx=0x7f8ad4004750 comp rx=0 tx=0).stop 2023-11-08T19:25:12.535 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:12.533+0000 7f8af21f5700 1 -- 172.21.15.143:0/2555903764 shutdown_connections 2023-11-08T19:25:12.535 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:12.533+0000 7f8af21f5700 1 --2- 172.21.15.143:0/2555903764 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f8acc06af00 0x7f8acc06d3c0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:12.535 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:12.533+0000 7f8af21f5700 1 --2- 172.21.15.143:0/2555903764 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8aec1618f0 0x7f8aec0c6600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:12.536 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:12.533+0000 7f8af21f5700 1 --2- 172.21.15.143:0/2555903764 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8aec06b320 0x7f8aec1f4960 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:12.536 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:12.533+0000 7f8af21f5700 1 --2- 172.21.15.143:0/2555903764 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8aec15dd00 0x7f8aec154c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:12.536 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:12.533+0000 7f8af21f5700 1 -- 172.21.15.143:0/2555903764 >> 172.21.15.143:0/2555903764 conn(0x7f8aec099ab0 msgr2=0x7f8aec0ad380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:12.536 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:12.533+0000 7f8af21f5700 1 -- 172.21.15.143:0/2555903764 shutdown_connections 2023-11-08T19:25:12.536 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:12.533+0000 7f8af21f5700 1 -- 172.21.15.143:0/2555903764 wait complete. 2023-11-08T19:25:12.546 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-08T19:25:12.547 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-08T19:25:12.547 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi143.front.sepia.ceph.com 2023-11-08T19:25:12.547 INFO:tasks.cephfs.mount:self.client.name = client.2 2023-11-08T19:25:12.547 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-11-08T19:25:12.548 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-08T19:25:12.548 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-08T19:25:12.548 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-08T19:25:12.548 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2023-11-08T19:25:12.549 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:12.549 DEBUG:teuthology.orchestra.run.smithi143:> ip addr 2023-11-08T19:25:12.566 INFO:teuthology.orchestra.run.smithi143.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-08T19:25:12.567 INFO:teuthology.orchestra.run.smithi143.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-08T19:25:12.567 INFO:teuthology.orchestra.run.smithi143.stdout: inet 127.0.0.1/8 scope host lo 2023-11-08T19:25:12.567 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:25:12.567 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 ::1/128 scope host 2023-11-08T19:25:12.567 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:25:12.568 INFO:teuthology.orchestra.run.smithi143.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-08T19:25:12.568 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:d9:3b:94 brd ff:ff:ff:ff:ff:ff 2023-11-08T19:25:12.568 INFO:teuthology.orchestra.run.smithi143.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-08T19:25:12.568 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:d9:3b:95 brd ff:ff:ff:ff:ff:ff 2023-11-08T19:25:12.568 INFO:teuthology.orchestra.run.smithi143.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-08T19:25:12.569 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:88:77:82 brd ff:ff:ff:ff:ff:ff 2023-11-08T19:25:12.569 INFO:teuthology.orchestra.run.smithi143.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-08T19:25:12.569 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:88:77:83 brd ff:ff:ff:ff:ff:ff 2023-11-08T19:25:12.569 INFO:teuthology.orchestra.run.smithi143.stdout: inet 172.21.15.143/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-08T19:25:12.569 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft 42355sec preferred_lft 42355sec 2023-11-08T19:25:12.569 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 fe80::e628:d259:2f76:52a6/64 scope link noprefixroute 2023-11-08T19:25:12.570 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:25:12.570 INFO:teuthology.orchestra.run.smithi143.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-08T19:25:12.570 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 46:e3:3e:66:77:ea brd ff:ff:ff:ff:ff:ff 2023-11-08T19:25:12.570 INFO:teuthology.orchestra.run.smithi143.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-08T19:25:12.571 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:25:12.571 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 fe80::749a:8aff:fe59:325c/64 scope link 2023-11-08T19:25:12.571 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:25:12.571 INFO:teuthology.orchestra.run.smithi143.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-08T19:25:12.572 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 46:e3:3e:66:77:ea brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-08T19:25:12.572 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 fe80::44e3:3eff:fe66:77ea/64 scope link tentative 2023-11-08T19:25:12.572 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:25:12.572 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:12.573 DEBUG:teuthology.orchestra.run.smithi143:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-08T19:25:12.631 INFO:teuthology.orchestra.run.smithi143.stdout:1 2023-11-08T19:25:12.632 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:12.633 DEBUG:teuthology.orchestra.run.smithi143:> route 2023-11-08T19:25:12.691 INFO:teuthology.orchestra.run.smithi143.stdout:Kernel IP routing table 2023-11-08T19:25:12.691 INFO:teuthology.orchestra.run.smithi143.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-08T19:25:12.692 INFO:teuthology.orchestra.run.smithi143.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-08T19:25:12.692 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-08T19:25:12.692 INFO:teuthology.orchestra.run.smithi143.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-08T19:25:12.692 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:12.693 DEBUG:teuthology.orchestra.run.smithi143:> (cd / && exec bash -c ' 2023-11-08T19:25:12.693 DEBUG:teuthology.orchestra.run.smithi143:> set -e 2023-11-08T19:25:12.694 DEBUG:teuthology.orchestra.run.smithi143:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-08T19:25:12.694 DEBUG:teuthology.orchestra.run.smithi143:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-08T19:25:12.694 DEBUG:teuthology.orchestra.run.smithi143:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-08T19:25:12.695 DEBUG:teuthology.orchestra.run.smithi143:> ') 2023-11-08T19:25:12.796 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:12.796 DEBUG:teuthology.orchestra.run.smithi143:> ip netns list 2023-11-08T19:25:12.813 INFO:teuthology.orchestra.run.smithi143.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-08T19:25:12.813 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:12.814 DEBUG:teuthology.orchestra.run.smithi143:> ip netns list-id 2023-11-08T19:25:12.870 INFO:teuthology.orchestra.run.smithi143.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2023-11-08T19:25:12.871 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:12.871 DEBUG:teuthology.orchestra.run.smithi143:> (cd / && exec bash -c ' 2023-11-08T19:25:12.872 DEBUG:teuthology.orchestra.run.smithi143:> set -e 2023-11-08T19:25:12.872 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-08T19:25:12.872 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2023-11-08T19:25:12.872 DEBUG:teuthology.orchestra.run.smithi143:> ') 2023-11-08T19:25:12.963 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:12.963 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-11-08T19:25:13.008 INFO:teuthology.orchestra.run.smithi143.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-08T19:25:13.008 INFO:teuthology.orchestra.run.smithi143.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-08T19:25:13.008 INFO:teuthology.orchestra.run.smithi143.stdout: inet 127.0.0.1/8 scope host lo 2023-11-08T19:25:13.009 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:25:13.009 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 ::1/128 scope host 2023-11-08T19:25:13.009 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:25:13.009 INFO:teuthology.orchestra.run.smithi143.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-08T19:25:13.009 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 7a:c0:fb:f0:e1:c8 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-08T19:25:13.010 INFO:teuthology.orchestra.run.smithi143.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-08T19:25:13.010 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:25:13.010 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 fe80::78c0:fbff:fef0:e1c8/64 scope link tentative 2023-11-08T19:25:13.010 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:25:13.017 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:13.018 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-11-08T19:25:13.060 INFO:teuthology.orchestra.run.smithi143.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-08T19:25:13.060 INFO:teuthology.orchestra.run.smithi143.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-08T19:25:13.061 INFO:teuthology.orchestra.run.smithi143.stdout: inet 127.0.0.1/8 scope host lo 2023-11-08T19:25:13.061 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:25:13.061 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 ::1/128 scope host 2023-11-08T19:25:13.061 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:25:13.062 INFO:teuthology.orchestra.run.smithi143.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-08T19:25:13.062 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 7a:c0:fb:f0:e1:c8 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-08T19:25:13.062 INFO:teuthology.orchestra.run.smithi143.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-08T19:25:13.062 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:25:13.063 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 fe80::78c0:fbff:fef0:e1c8/64 scope link tentative 2023-11-08T19:25:13.063 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:25:13.075 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2023-11-08T19:25:13.076 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:13.076 DEBUG:teuthology.orchestra.run.smithi143:> (cd / && exec bash -c ' 2023-11-08T19:25:13.076 DEBUG:teuthology.orchestra.run.smithi143:> set -e 2023-11-08T19:25:13.077 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2023-11-08T19:25:13.077 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-11-08T19:25:13.077 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2023-11-08T19:25:13.078 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2023-11-08T19:25:13.078 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2023-11-08T19:25:13.078 DEBUG:teuthology.orchestra.run.smithi143:> ') 2023-11-08T19:25:13.324 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:13.324 DEBUG:teuthology.orchestra.run.smithi143:> (cd / && exec bash -c ' 2023-11-08T19:25:13.324 DEBUG:teuthology.orchestra.run.smithi143:> set -e 2023-11-08T19:25:13.325 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip link set brx.1 up 2023-11-08T19:25:13.325 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip link set dev brx.1 master ceph-brx 2023-11-08T19:25:13.325 DEBUG:teuthology.orchestra.run.smithi143:> ') 2023-11-08T19:25:13.434 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-08T19:25:13.434 DEBUG:teuthology.orchestra.run.smithi143:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2023-11-08T19:25:13.454 INFO:teuthology.orchestra.run.smithi143.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2023-11-08T19:25:13.454 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-08T19:25:13.455 DEBUG:teuthology.orchestra.run.smithi143:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2023-11-08T19:25:13.510 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-11-08T19:25:13.510 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:13.511 DEBUG:teuthology.orchestra.run.smithi143:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2023-11-08T19:25:13.596 INFO:teuthology.orchestra.run.smithi143.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2023-11-08T19:25:13.597 INFO:teuthology.orchestra.run.smithi143.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2023-11-08T19:25:13.597 INFO:teuthology.orchestra.run.smithi143.stdout:invalid new device string format 2023-11-08T19:25:13.605 INFO:teuthology.orchestra.run.smithi143.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2023-11-08T19:25:13.606 INFO:teuthology.orchestra.run.smithi143.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2023-11-08T19:25:13.606 INFO:teuthology.orchestra.run.smithi143.stdout:invalid new device string format 2023-11-08T19:25:13.606 INFO:teuthology.orchestra.run.smithi143.stdout:mount.ceph: resolved to: "172.21.15.143:3300,172.21.15.174:3300,172.21.15.174:3301" 2023-11-08T19:25:13.607 INFO:teuthology.orchestra.run.smithi143.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-08T19:25:13.680 INFO:teuthology.orchestra.run.smithi143.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2023-11-08T19:25:13.680 INFO:teuthology.orchestra.run.smithi143.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2023-11-08T19:25:13.680 INFO:teuthology.orchestra.run.smithi143.stdout:invalid new device string format 2023-11-08T19:25:13.681 INFO:teuthology.orchestra.run.smithi143.stdout:mount.ceph: resolved to: "172.21.15.143:3300,172.21.15.174:3300,172.21.15.174:3301" 2023-11-08T19:25:13.681 INFO:teuthology.orchestra.run.smithi143.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-08T19:25:13.681 INFO:teuthology.orchestra.run.smithi143.stdout:mount.ceph: resolved to: "172.21.15.143:6789,172.21.15.174:6789,172.21.15.174:6790" 2023-11-08T19:25:13.681 INFO:teuthology.orchestra.run.smithi143.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.143:6789,172.21.15.174:6789,172.21.15.174:6790:/ 2023-11-08T19:25:13.682 INFO:teuthology.orchestra.run.smithi143.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs,key=2,fsid=968483af-77fc-4068-ad7c-5c2a1a4613d7" will pass to kernel 2023-11-08T19:25:13.683 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-08T19:25:13.683 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:13.683 DEBUG:teuthology.orchestra.run.smithi143:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2023-11-08T19:25:13.710 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:13.710 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-08T19:25:13.786 INFO:teuthology.orchestra.run.smithi143.stdout:client4642 2023-11-08T19:25:13.786 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-08T19:25:13.787 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:25:13.849 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:13.849 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-08T19:25:13.922 INFO:teuthology.orchestra.run.smithi143.stdout:client4642 2023-11-08T19:25:13.922 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:13.923 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-08T19:25:13.998 INFO:teuthology.orchestra.run.smithi143.stdout:client4642 2023-11-08T19:25:13.999 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:13.999 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/968483af-77fc-4068-ad7c-5c2a1a4613d7.client4642/status') 2023-11-08T19:25:14.083 INFO:teuthology.orchestra.run.smithi143.stdout:instance: client.4642 (3)172.21.15.143:0/2066548757 2023-11-08T19:25:14.083 INFO:teuthology.orchestra.run.smithi143.stdout:blocklisted: false 2023-11-08T19:25:14.087 INFO:teuthology.orchestra.run.smithi143.stderr:0+1 records in 2023-11-08T19:25:14.088 INFO:teuthology.orchestra.run.smithi143.stderr:0+1 records out 2023-11-08T19:25:14.088 INFO:teuthology.orchestra.run.smithi143.stderr:71 bytes copied, 2.3042e-05 s, 3.1 MB/s 2023-11-08T19:25:14.088 INFO:tasks.kclient:client.1 config is {'syntax': 'v1'} 2023-11-08T19:25:14.089 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-08T19:25:14.089 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-08T19:25:14.090 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-08T19:25:14.251 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:14.250+0000 7fa5ccfde700 1 Processor -- start 2023-11-08T19:25:14.252 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:14.250+0000 7fa5ccfde700 1 -- start start 2023-11-08T19:25:14.254 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:14.251+0000 7fa5ccfde700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa5c81618f0 0x7fa5c806ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:14.254 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:14.251+0000 7fa5ccfde700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fa5c806b150 0x7fa5c80cb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:14.254 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:14.251+0000 7fa5ccfde700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5c8161df0 con 0x7fa5c815dd00 2023-11-08T19:25:14.254 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:14.251+0000 7fa5ccfde700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5c806b600 con 0x7fa5c81618f0 2023-11-08T19:25:14.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:14.251+0000 7fa5ccfde700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5c80cad50 con 0x7fa5c806b150 2023-11-08T19:25:14.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:14.251+0000 7fa5c5d9b700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa5c81618f0 0x7fa5c806ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:14.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:14.251+0000 7fa5c5d9b700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa5c81618f0 0x7fa5c806ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49448/0 (socket says 172.21.15.143:49448) 2023-11-08T19:25:14.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:14.251+0000 7fa5c5d9b700 1 -- 172.21.15.143:0/3231704629 learned_addr learned my addr 172.21.15.143:0/3231704629 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:14.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:14.251+0000 7fa5c6d9d700 1 --2- 172.21.15.143:0/3231704629 >> v2:172.21.15.174:3300/0 conn(0x7fa5c806b150 0x7fa5c80cb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:14.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:14.251+0000 7fa5c5d9b700 1 -- 172.21.15.143:0/3231704629 >> v2:172.21.15.174:3300/0 conn(0x7fa5c806b150 msgr2=0x7fa5c80cb1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:14.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:14.251+0000 7fa5c5d9b700 1 --2- 172.21.15.143:0/3231704629 >> v2:172.21.15.174:3300/0 conn(0x7fa5c806b150 0x7fa5c80cb1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:14.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:14.251+0000 7fa5c5d9b700 1 -- 172.21.15.143:0/3231704629 >> v1:172.21.15.174:6789/0 conn(0x7fa5c815dd00 legacy=0x7fa5c80cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:14.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:14.251+0000 7fa5c5d9b700 1 -- 172.21.15.143:0/3231704629 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5c81620d0 con 0x7fa5c81618f0 2023-11-08T19:25:14.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:14.251+0000 7fa5c5d9b700 1 --2- 172.21.15.143:0/3231704629 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa5c81618f0 0x7fa5c806ac10 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa5c8099f70 tx=0x7fa5bc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=292044b7e67eb2ac server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:14.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:14.251+0000 7fa5c559a700 1 -- 172.21.15.143:0/3231704629 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5bc004230 con 0x7fa5c81618f0 2023-11-08T19:25:14.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:14.252+0000 7fa5c559a700 1 -- 172.21.15.143:0/3231704629 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5bc004390 con 0x7fa5c81618f0 2023-11-08T19:25:14.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:14.252+0000 7fa5c559a700 1 -- 172.21.15.143:0/3231704629 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5bc0036a0 con 0x7fa5c81618f0 2023-11-08T19:25:14.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:14.252+0000 7fa5ccfde700 1 -- 172.21.15.143:0/3231704629 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa5c81618f0 msgr2=0x7fa5c806ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:14.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:14.252+0000 7fa5ccfde700 1 --2- 172.21.15.143:0/3231704629 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa5c81618f0 0x7fa5c806ac10 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa5c8099f70 tx=0x7fa5bc009bd0 comp rx=0 tx=0).stop 2023-11-08T19:25:14.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:14.252+0000 7fa5ccfde700 1 -- 172.21.15.143:0/3231704629 shutdown_connections 2023-11-08T19:25:14.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:14.252+0000 7fa5ccfde700 1 --2- 172.21.15.143:0/3231704629 >> v2:172.21.15.174:3300/0 conn(0x7fa5c806b150 0x7fa5c80cb1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:14.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:14.252+0000 7fa5ccfde700 1 --2- 172.21.15.143:0/3231704629 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa5c81618f0 0x7fa5c806ac10 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:14.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:14.252+0000 7fa5ccfde700 1 -- 172.21.15.143:0/3231704629 >> 172.21.15.143:0/3231704629 conn(0x7fa5c8099ab0 msgr2=0x7fa5c80d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:14.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:14.252+0000 7fa5ccfde700 1 -- 172.21.15.143:0/3231704629 shutdown_connections 2023-11-08T19:25:14.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:14.252+0000 7fa5ccfde700 1 -- 172.21.15.143:0/3231704629 wait complete. 2023-11-08T19:25:14.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:14.254+0000 7fa5ccfde700 1 Processor -- start 2023-11-08T19:25:14.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:14.254+0000 7fa5ccfde700 1 -- start start 2023-11-08T19:25:14.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:14.254+0000 7fa5ccfde700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa5c806b150 0x7fa5c81588f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:14.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:14.254+0000 7fa5ccfde700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa5c815dd00 0x7fa5c80b4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:14.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:14.254+0000 7fa5ccfde700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa5c81618f0 0x7fa5c8160740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:14.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:14.254+0000 7fa5ccfde700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c80b54f0 con 0x7fa5c806b150 2023-11-08T19:25:14.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:14.254+0000 7fa5ccfde700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c80b4b20 con 0x7fa5c81618f0 2023-11-08T19:25:14.264 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:14.254+0000 7fa5ccfde700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c81f4630 con 0x7fa5c815dd00 2023-11-08T19:25:14.264 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:14.254+0000 7fa5c659c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa5c806b150 0x7fa5c81588f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:14.264 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:14.254+0000 7fa5c659c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa5c806b150 0x7fa5c81588f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49452/0 (socket says 172.21.15.143:49452) 2023-11-08T19:25:14.264 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:14.254+0000 7fa5c659c700 1 -- 172.21.15.143:0/1886933979 learned_addr learned my addr 172.21.15.143:0/1886933979 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:14.264 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:14.255+0000 7fa5c6d9d700 1 --2- 172.21.15.143:0/1886933979 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa5c81618f0 0x7fa5c8160740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:14.265 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:14.255+0000 7fa5c5d9b700 1 --2- 172.21.15.143:0/1886933979 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa5c815dd00 0x7fa5c80b4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:14.265 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:14.255+0000 7fa5c659c700 1 -- 172.21.15.143:0/1886933979 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa5c815dd00 msgr2=0x7fa5c80b4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:14.265 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:14.255+0000 7fa5c659c700 1 --2- 172.21.15.143:0/1886933979 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa5c815dd00 0x7fa5c80b4fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:14.265 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:14.255+0000 7fa5c659c700 1 -- 172.21.15.143:0/1886933979 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa5c81618f0 msgr2=0x7fa5c8160740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:14.265 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:14.255+0000 7fa5c659c700 1 --2- 172.21.15.143:0/1886933979 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa5c81618f0 0x7fa5c8160740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:14.265 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:14.255+0000 7fa5c659c700 1 -- 172.21.15.143:0/1886933979 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5bc009550 con 0x7fa5c806b150 2023-11-08T19:25:14.266 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:14.255+0000 7fa5c6d9d700 1 --2- 172.21.15.143:0/1886933979 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa5c81618f0 0x7fa5c8160740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:25:14.266 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:14.255+0000 7fa5c659c700 1 --2- 172.21.15.143:0/1886933979 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa5c806b150 0x7fa5c81588f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa5b800c520 tx=0x7fa5b800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:14.266 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:14.255+0000 7fa5b6ffd700 1 -- 172.21.15.143:0/1886933979 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5b80054e0 con 0x7fa5c806b150 2023-11-08T19:25:14.266 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:14.255+0000 7fa5b6ffd700 1 -- 172.21.15.143:0/1886933979 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5b80036d0 con 0x7fa5c806b150 2023-11-08T19:25:14.266 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:14.255+0000 7fa5ccfde700 1 -- 172.21.15.143:0/1886933979 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5c80b41f0 con 0x7fa5c806b150 2023-11-08T19:25:14.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:14.255+0000 7fa5b6ffd700 1 -- 172.21.15.143:0/1886933979 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5b8003a80 con 0x7fa5c806b150 2023-11-08T19:25:14.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:14.255+0000 7fa5ccfde700 1 -- 172.21.15.143:0/1886933979 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5c80b47b0 con 0x7fa5c806b150 2023-11-08T19:25:14.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:14.256+0000 7fa5ccfde700 1 -- 172.21.15.143:0/1886933979 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5c80aff60 con 0x7fa5c806b150 2023-11-08T19:25:14.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:14.256+0000 7fa5b6ffd700 1 -- 172.21.15.143:0/1886933979 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa5b8003be0 con 0x7fa5c806b150 2023-11-08T19:25:14.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:14.257+0000 7fa5b6ffd700 1 --2- 172.21.15.143:0/1886933979 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa5a806af00 0x7fa5a806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:14.268 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:14.257+0000 7fa5b6ffd700 1 -- 172.21.15.143:0/1886933979 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa5b8088a30 con 0x7fa5c806b150 2023-11-08T19:25:14.268 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:14.257+0000 7fa5c5d9b700 1 --2- 172.21.15.143:0/1886933979 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa5a806af00 0x7fa5a806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:14.268 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:14.257+0000 7fa5c5d9b700 1 --2- 172.21.15.143:0/1886933979 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa5a806af00 0x7fa5a806d3c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc009130 tx=0x7fa5bc008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:14.268 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:14.261+0000 7fa5b6ffd700 1 -- 172.21.15.143:0/1886933979 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa5b8053b70 con 0x7fa5c806b150 2023-11-08T19:25:14.498 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:14.496+0000 7fa5ccfde700 1 -- 172.21.15.143:0/1886933979 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fa5c80b4e00 con 0x7fa5c806b150 2023-11-08T19:25:14.498 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:14.496+0000 7fa5b6ffd700 1 -- 172.21.15.143:0/1886933979 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7fa5b8017020 con 0x7fa5c806b150 2023-11-08T19:25:14.499 INFO:teuthology.orchestra.run.smithi143.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-08T19:25:14.500 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:14.498+0000 7fa5ccfde700 1 -- 172.21.15.143:0/1886933979 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa5a806af00 msgr2=0x7fa5a806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:14.500 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:14.498+0000 7fa5ccfde700 1 --2- 172.21.15.143:0/1886933979 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa5a806af00 0x7fa5a806d3c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc009130 tx=0x7fa5bc008030 comp rx=0 tx=0).stop 2023-11-08T19:25:14.501 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:14.498+0000 7fa5ccfde700 1 -- 172.21.15.143:0/1886933979 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa5c806b150 msgr2=0x7fa5c81588f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:14.501 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:14.498+0000 7fa5ccfde700 1 --2- 172.21.15.143:0/1886933979 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa5c806b150 0x7fa5c81588f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa5b800c520 tx=0x7fa5b800c8e0 comp rx=0 tx=0).stop 2023-11-08T19:25:14.501 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:14.498+0000 7fa5ccfde700 1 -- 172.21.15.143:0/1886933979 shutdown_connections 2023-11-08T19:25:14.501 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:14.498+0000 7fa5ccfde700 1 --2- 172.21.15.143:0/1886933979 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa5a806af00 0x7fa5a806d3c0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:14.501 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:14.498+0000 7fa5ccfde700 1 --2- 172.21.15.143:0/1886933979 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa5c81618f0 0x7fa5c8160740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:14.502 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:14.498+0000 7fa5ccfde700 1 --2- 172.21.15.143:0/1886933979 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa5c806b150 0x7fa5c81588f0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:14.502 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:14.498+0000 7fa5ccfde700 1 --2- 172.21.15.143:0/1886933979 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa5c815dd00 0x7fa5c80b4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:14.502 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:14.498+0000 7fa5ccfde700 1 -- 172.21.15.143:0/1886933979 >> 172.21.15.143:0/1886933979 conn(0x7fa5c8099ab0 msgr2=0x7fa5c806f4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:14.502 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:14.498+0000 7fa5ccfde700 1 -- 172.21.15.143:0/1886933979 shutdown_connections 2023-11-08T19:25:14.502 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:14.498+0000 7fa5ccfde700 1 -- 172.21.15.143:0/1886933979 wait complete. 2023-11-08T19:25:14.512 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-08T19:25:14.512 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-08T19:25:14.513 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi174.front.sepia.ceph.com 2023-11-08T19:25:14.513 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-11-08T19:25:14.513 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-08T19:25:14.514 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-08T19:25:14.514 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-08T19:25:14.514 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-08T19:25:14.515 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-08T19:25:14.515 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:14.515 DEBUG:teuthology.orchestra.run.smithi174:> ip addr 2023-11-08T19:25:14.537 INFO:teuthology.orchestra.run.smithi174.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-08T19:25:14.537 INFO:teuthology.orchestra.run.smithi174.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-08T19:25:14.537 INFO:teuthology.orchestra.run.smithi174.stdout: inet 127.0.0.1/8 scope host lo 2023-11-08T19:25:14.538 INFO:teuthology.orchestra.run.smithi174.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:25:14.538 INFO:teuthology.orchestra.run.smithi174.stdout: inet6 ::1/128 scope host 2023-11-08T19:25:14.538 INFO:teuthology.orchestra.run.smithi174.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:25:14.538 INFO:teuthology.orchestra.run.smithi174.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-08T19:25:14.539 INFO:teuthology.orchestra.run.smithi174.stdout: link/ether 0c:c4:7a:d9:36:e0 brd ff:ff:ff:ff:ff:ff 2023-11-08T19:25:14.539 INFO:teuthology.orchestra.run.smithi174.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-08T19:25:14.539 INFO:teuthology.orchestra.run.smithi174.stdout: link/ether 0c:c4:7a:d9:36:e1 brd ff:ff:ff:ff:ff:ff 2023-11-08T19:25:14.539 INFO:teuthology.orchestra.run.smithi174.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-08T19:25:14.539 INFO:teuthology.orchestra.run.smithi174.stdout: link/ether 0c:c4:7a:88:77:1a brd ff:ff:ff:ff:ff:ff 2023-11-08T19:25:14.540 INFO:teuthology.orchestra.run.smithi174.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-08T19:25:14.540 INFO:teuthology.orchestra.run.smithi174.stdout: link/ether 0c:c4:7a:88:77:1b brd ff:ff:ff:ff:ff:ff 2023-11-08T19:25:14.540 INFO:teuthology.orchestra.run.smithi174.stdout: inet 172.21.15.174/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-08T19:25:14.540 INFO:teuthology.orchestra.run.smithi174.stdout: valid_lft 42345sec preferred_lft 42345sec 2023-11-08T19:25:14.541 INFO:teuthology.orchestra.run.smithi174.stdout: inet6 fe80::a48d:9583:b83f:827c/64 scope link noprefixroute 2023-11-08T19:25:14.541 INFO:teuthology.orchestra.run.smithi174.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:25:14.541 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-11-08T19:25:14.542 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:14.542 DEBUG:teuthology.orchestra.run.smithi174:> (cd / && exec bash -c ' 2023-11-08T19:25:14.542 DEBUG:teuthology.orchestra.run.smithi174:> set -e 2023-11-08T19:25:14.543 DEBUG:teuthology.orchestra.run.smithi174:> sudo ip link add name ceph-brx type bridge 2023-11-08T19:25:14.543 DEBUG:teuthology.orchestra.run.smithi174:> sudo ip addr flush dev ceph-brx 2023-11-08T19:25:14.543 DEBUG:teuthology.orchestra.run.smithi174:> sudo ip link set ceph-brx up 2023-11-08T19:25:14.544 DEBUG:teuthology.orchestra.run.smithi174:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-11-08T19:25:14.544 DEBUG:teuthology.orchestra.run.smithi174:> ') 2023-11-08T19:25:14.712 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:14.713 DEBUG:teuthology.orchestra.run.smithi174:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-08T19:25:14.742 INFO:teuthology.orchestra.run.smithi174.stdout:1 2023-11-08T19:25:14.743 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:14.743 DEBUG:teuthology.orchestra.run.smithi174:> route 2023-11-08T19:25:14.812 INFO:teuthology.orchestra.run.smithi174.stdout:Kernel IP routing table 2023-11-08T19:25:14.812 INFO:teuthology.orchestra.run.smithi174.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-08T19:25:14.812 INFO:teuthology.orchestra.run.smithi174.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-08T19:25:14.813 INFO:teuthology.orchestra.run.smithi174.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-08T19:25:14.813 INFO:teuthology.orchestra.run.smithi174.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-08T19:25:14.813 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:14.814 DEBUG:teuthology.orchestra.run.smithi174:> (cd / && exec bash -c ' 2023-11-08T19:25:14.814 DEBUG:teuthology.orchestra.run.smithi174:> set -e 2023-11-08T19:25:14.814 DEBUG:teuthology.orchestra.run.smithi174:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-08T19:25:14.814 DEBUG:teuthology.orchestra.run.smithi174:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-08T19:25:14.815 DEBUG:teuthology.orchestra.run.smithi174:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-08T19:25:14.815 DEBUG:teuthology.orchestra.run.smithi174:> ') 2023-11-08T19:25:15.073 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:15.074 DEBUG:teuthology.orchestra.run.smithi174:> ip netns list 2023-11-08T19:25:15.092 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:15.093 DEBUG:teuthology.orchestra.run.smithi174:> ip netns list-id 2023-11-08T19:25:15.149 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:15.150 DEBUG:teuthology.orchestra.run.smithi174:> (cd / && exec bash -c ' 2023-11-08T19:25:15.150 DEBUG:teuthology.orchestra.run.smithi174:> set -e 2023-11-08T19:25:15.150 DEBUG:teuthology.orchestra.run.smithi174:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-08T19:25:15.151 DEBUG:teuthology.orchestra.run.smithi174:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2023-11-08T19:25:15.152 DEBUG:teuthology.orchestra.run.smithi174:> ') 2023-11-08T19:25:15.243 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2023-11-08T19:25:15.244 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:15.244 DEBUG:teuthology.orchestra.run.smithi174:> (cd / && exec bash -c ' 2023-11-08T19:25:15.244 DEBUG:teuthology.orchestra.run.smithi174:> set -e 2023-11-08T19:25:15.245 DEBUG:teuthology.orchestra.run.smithi174:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2023-11-08T19:25:15.245 DEBUG:teuthology.orchestra.run.smithi174:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-11-08T19:25:15.245 DEBUG:teuthology.orchestra.run.smithi174:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2023-11-08T19:25:15.245 DEBUG:teuthology.orchestra.run.smithi174:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2023-11-08T19:25:15.246 DEBUG:teuthology.orchestra.run.smithi174:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2023-11-08T19:25:15.246 DEBUG:teuthology.orchestra.run.smithi174:> ') 2023-11-08T19:25:15.464 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:15.465 DEBUG:teuthology.orchestra.run.smithi174:> (cd / && exec bash -c ' 2023-11-08T19:25:15.465 DEBUG:teuthology.orchestra.run.smithi174:> set -e 2023-11-08T19:25:15.465 DEBUG:teuthology.orchestra.run.smithi174:> sudo ip link set brx.0 up 2023-11-08T19:25:15.465 DEBUG:teuthology.orchestra.run.smithi174:> sudo ip link set dev brx.0 master ceph-brx 2023-11-08T19:25:15.466 DEBUG:teuthology.orchestra.run.smithi174:> ') 2023-11-08T19:25:15.565 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-08T19:25:15.566 DEBUG:teuthology.orchestra.run.smithi174:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-11-08T19:25:15.585 INFO:teuthology.orchestra.run.smithi174.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-11-08T19:25:15.586 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-08T19:25:15.586 DEBUG:teuthology.orchestra.run.smithi174:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-11-08T19:25:15.641 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-11-08T19:25:15.641 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:15.642 DEBUG:teuthology.orchestra.run.smithi174:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-11-08T19:25:15.743 INFO:teuthology.orchestra.run.smithi174.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-11-08T19:25:15.743 INFO:teuthology.orchestra.run.smithi174.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-11-08T19:25:15.744 INFO:teuthology.orchestra.run.smithi174.stdout:invalid new device string format 2023-11-08T19:25:15.834 INFO:teuthology.orchestra.run.smithi174.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-11-08T19:25:15.835 INFO:teuthology.orchestra.run.smithi174.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-11-08T19:25:15.835 INFO:teuthology.orchestra.run.smithi174.stdout:invalid new device string format 2023-11-08T19:25:15.835 INFO:teuthology.orchestra.run.smithi174.stdout:mount.ceph: resolved to: "172.21.15.143:3300,172.21.15.174:3300,172.21.15.174:3301" 2023-11-08T19:25:15.835 INFO:teuthology.orchestra.run.smithi174.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-08T19:25:15.913 INFO:teuthology.orchestra.run.smithi174.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-11-08T19:25:15.913 INFO:teuthology.orchestra.run.smithi174.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-11-08T19:25:15.914 INFO:teuthology.orchestra.run.smithi174.stdout:invalid new device string format 2023-11-08T19:25:15.914 INFO:teuthology.orchestra.run.smithi174.stdout:mount.ceph: resolved to: "172.21.15.143:3300,172.21.15.174:3300,172.21.15.174:3301" 2023-11-08T19:25:15.914 INFO:teuthology.orchestra.run.smithi174.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-08T19:25:15.914 INFO:teuthology.orchestra.run.smithi174.stdout:mount.ceph: resolved to: "172.21.15.143:6789,172.21.15.174:6789,172.21.15.174:6790" 2023-11-08T19:25:15.914 INFO:teuthology.orchestra.run.smithi174.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.143:6789,172.21.15.174:6789,172.21.15.174:6790:/ 2023-11-08T19:25:15.915 INFO:teuthology.orchestra.run.smithi174.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=968483af-77fc-4068-ad7c-5c2a1a4613d7" will pass to kernel 2023-11-08T19:25:15.916 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-08T19:25:15.916 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:15.917 DEBUG:teuthology.orchestra.run.smithi174:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-11-08T19:25:15.943 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:15.943 DEBUG:teuthology.orchestra.run.smithi174:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-08T19:25:16.015 INFO:teuthology.orchestra.run.smithi174.stdout:client4692 2023-11-08T19:25:16.015 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2023-11-08T19:25:16.016 DEBUG:teuthology.orchestra.run.smithi174:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:25:16.072 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:16.072 DEBUG:teuthology.orchestra.run.smithi174:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-08T19:25:16.144 INFO:teuthology.orchestra.run.smithi174.stdout:client4692 2023-11-08T19:25:16.145 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:16.145 DEBUG:teuthology.orchestra.run.smithi174:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-08T19:25:16.216 INFO:teuthology.orchestra.run.smithi174.stdout:client4692 2023-11-08T19:25:16.217 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:16.217 DEBUG:teuthology.orchestra.run.smithi174:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/968483af-77fc-4068-ad7c-5c2a1a4613d7.client4692/status') 2023-11-08T19:25:16.296 INFO:teuthology.orchestra.run.smithi174.stdout:instance: client.4692 (3)172.21.15.174:0/3242009724 2023-11-08T19:25:16.297 INFO:teuthology.orchestra.run.smithi174.stdout:blocklisted: false 2023-11-08T19:25:16.298 INFO:teuthology.orchestra.run.smithi174.stderr:0+1 records in 2023-11-08T19:25:16.298 INFO:teuthology.orchestra.run.smithi174.stderr:0+1 records out 2023-11-08T19:25:16.298 INFO:teuthology.orchestra.run.smithi174.stderr:71 bytes copied, 2.3773e-05 s, 3.0 MB/s 2023-11-08T19:25:16.299 INFO:tasks.kclient:client.3 config is {'syntax': 'v1'} 2023-11-08T19:25:16.299 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-08T19:25:16.299 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-11-08T19:25:16.300 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-08T19:25:16.433 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:16.432+0000 7f1b639b7700 1 Processor -- start 2023-11-08T19:25:16.433 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:16.432+0000 7f1b639b7700 1 -- start start 2023-11-08T19:25:16.436 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:16.433+0000 7f1b639b7700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f1b5c071aa0 0x7f1b5c0cd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:16.436 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:16.433+0000 7f1b639b7700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1b5c0cd700 0x7f1b5c0ca590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:16.436 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:16.433+0000 7f1b639b7700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b5c0ceb30 con 0x7f1b5c157e30 2023-11-08T19:25:16.436 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:16.433+0000 7f1b639b7700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b5c071680 con 0x7f1b5c071aa0 2023-11-08T19:25:16.437 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:16.433+0000 7f1b639b7700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b5c071f30 con 0x7f1b5c0cd700 2023-11-08T19:25:16.437 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:16.433+0000 7f1b60f52700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f1b5c071aa0 0x7f1b5c0cd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:16.437 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:16.433+0000 7f1b60f52700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f1b5c071aa0 0x7f1b5c0cd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49468/0 (socket says 172.21.15.143:49468) 2023-11-08T19:25:16.437 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:16.433+0000 7f1b60f52700 1 -- 172.21.15.143:0/1749847491 learned_addr learned my addr 172.21.15.143:0/1749847491 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:16.437 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:16.433+0000 7f1b61f54700 1 --2- 172.21.15.143:0/1749847491 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1b5c0cd700 0x7f1b5c0ca590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:16.438 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:16.433+0000 7f1b60f52700 1 -- 172.21.15.143:0/1749847491 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1b5c0cd700 msgr2=0x7f1b5c0ca590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:16.438 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:16.433+0000 7f1b60f52700 1 --2- 172.21.15.143:0/1749847491 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1b5c0cd700 0x7f1b5c0ca590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:16.438 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:16.433+0000 7f1b60f52700 1 -- 172.21.15.143:0/1749847491 >> v1:172.21.15.174:6789/0 conn(0x7f1b5c157e30 legacy=0x7f1b5c0cde00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:16.438 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:16.433+0000 7f1b60f52700 1 -- 172.21.15.143:0/1749847491 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b5c070fe0 con 0x7f1b5c071aa0 2023-11-08T19:25:16.438 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:16.433+0000 7f1b60f52700 1 --2- 172.21.15.143:0/1749847491 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1b5c071aa0 0x7f1b5c0cd1c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1b5800bb90 tx=0x7f1b5800bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc5eda8abd43bfb7 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:16.439 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:16.433+0000 7f1b53fff700 1 -- 172.21.15.143:0/1749847491 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b5800dc00 con 0x7f1b5c071aa0 2023-11-08T19:25:16.439 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:16.433+0000 7f1b53fff700 1 -- 172.21.15.143:0/1749847491 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b5800dd60 con 0x7f1b5c071aa0 2023-11-08T19:25:16.439 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:16.433+0000 7f1b53fff700 1 -- 172.21.15.143:0/1749847491 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b58004270 con 0x7f1b5c071aa0 2023-11-08T19:25:16.439 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:16.434+0000 7f1b639b7700 1 -- 172.21.15.143:0/1749847491 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1b5c071aa0 msgr2=0x7f1b5c0cd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:16.439 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:16.434+0000 7f1b639b7700 1 --2- 172.21.15.143:0/1749847491 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1b5c071aa0 0x7f1b5c0cd1c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1b5800bb90 tx=0x7f1b5800bea0 comp rx=0 tx=0).stop 2023-11-08T19:25:16.440 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:16.434+0000 7f1b639b7700 1 -- 172.21.15.143:0/1749847491 shutdown_connections 2023-11-08T19:25:16.440 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:16.434+0000 7f1b639b7700 1 --2- 172.21.15.143:0/1749847491 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1b5c0cd700 0x7f1b5c0ca590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:16.440 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:16.434+0000 7f1b639b7700 1 --2- 172.21.15.143:0/1749847491 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1b5c071aa0 0x7f1b5c0cd1c0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:16.441 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:16.434+0000 7f1b639b7700 1 -- 172.21.15.143:0/1749847491 >> 172.21.15.143:0/1749847491 conn(0x7f1b5c06b4c0 msgr2=0x7f1b5c0cf680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:16.441 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:16.434+0000 7f1b639b7700 1 -- 172.21.15.143:0/1749847491 shutdown_connections 2023-11-08T19:25:16.441 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:16.434+0000 7f1b639b7700 1 -- 172.21.15.143:0/1749847491 wait complete. 2023-11-08T19:25:16.444 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:16.435+0000 7f1b639b7700 1 Processor -- start 2023-11-08T19:25:16.445 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:16.435+0000 7f1b639b7700 1 -- start start 2023-11-08T19:25:16.445 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:16.436+0000 7f1b639b7700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1b5c071aa0 0x7f1b5c14b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:16.445 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:16.436+0000 7f1b639b7700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1b5c0cd700 0x7f1b5c154a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:16.445 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:16.436+0000 7f1b639b7700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1b5c157e30 0x7f1b5c0b39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:16.445 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:16.436+0000 7f1b639b7700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b5c0ae1d0 con 0x7f1b5c157e30 2023-11-08T19:25:16.446 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:16.436+0000 7f1b639b7700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b5c0ad300 con 0x7f1b5c071aa0 2023-11-08T19:25:16.446 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:16.436+0000 7f1b639b7700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b5c0ad550 con 0x7f1b5c0cd700 2023-11-08T19:25:16.446 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:16.436+0000 7f1b61f54700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1b5c157e30 0x7f1b5c0b39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:16.446 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:16.436+0000 7f1b61f54700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1b5c157e30 0x7f1b5c0b39b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49474/0 (socket says 172.21.15.143:49474) 2023-11-08T19:25:16.446 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:16.436+0000 7f1b61f54700 1 -- 172.21.15.143:0/1493775832 learned_addr learned my addr 172.21.15.143:0/1493775832 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:16.447 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:16.436+0000 7f1b60f52700 1 --2- 172.21.15.143:0/1493775832 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1b5c0cd700 0x7f1b5c154a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:16.447 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:16.436+0000 7f1b61753700 1 --2- 172.21.15.143:0/1493775832 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1b5c071aa0 0x7f1b5c14b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:16.447 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:16.436+0000 7f1b61f54700 1 -- 172.21.15.143:0/1493775832 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1b5c0cd700 msgr2=0x7f1b5c154a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:16.447 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:16.436+0000 7f1b61f54700 1 --2- 172.21.15.143:0/1493775832 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1b5c0cd700 0x7f1b5c154a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:16.447 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:16.436+0000 7f1b61f54700 1 -- 172.21.15.143:0/1493775832 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1b5c071aa0 msgr2=0x7f1b5c14b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:16.447 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:16.436+0000 7f1b61f54700 1 --2- 172.21.15.143:0/1493775832 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1b5c071aa0 0x7f1b5c14b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:16.448 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:16.436+0000 7f1b61f54700 1 -- 172.21.15.143:0/1493775832 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b5800b7f0 con 0x7f1b5c157e30 2023-11-08T19:25:16.448 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:16.436+0000 7f1b61753700 1 --2- 172.21.15.143:0/1493775832 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1b5c071aa0 0x7f1b5c14b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:25:16.448 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:16.437+0000 7f1b61f54700 1 --2- 172.21.15.143:0/1493775832 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1b5c157e30 0x7f1b5c0b39b0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1b5400cd70 tx=0x7f1b5400a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:16.448 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:16.437+0000 7f1b51ffb700 1 -- 172.21.15.143:0/1493775832 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b54003a20 con 0x7f1b5c157e30 2023-11-08T19:25:16.448 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:16.437+0000 7f1b51ffb700 1 -- 172.21.15.143:0/1493775832 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b54003b80 con 0x7f1b5c157e30 2023-11-08T19:25:16.449 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:16.437+0000 7f1b639b7700 1 -- 172.21.15.143:0/1493775832 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b5c0ad830 con 0x7f1b5c157e30 2023-11-08T19:25:16.449 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:16.437+0000 7f1b51ffb700 1 -- 172.21.15.143:0/1493775832 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b54004020 con 0x7f1b5c157e30 2023-11-08T19:25:16.449 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:16.437+0000 7f1b639b7700 1 -- 172.21.15.143:0/1493775832 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b5c0addf0 con 0x7f1b5c157e30 2023-11-08T19:25:16.449 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:16.438+0000 7f1b639b7700 1 -- 172.21.15.143:0/1493775832 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b5c0cf030 con 0x7f1b5c157e30 2023-11-08T19:25:16.449 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:16.438+0000 7f1b51ffb700 1 -- 172.21.15.143:0/1493775832 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1b54004180 con 0x7f1b5c157e30 2023-11-08T19:25:16.450 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:16.438+0000 7f1b51ffb700 1 --2- 172.21.15.143:0/1493775832 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f1b4006af00 0x7f1b4006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:16.450 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:16.439+0000 7f1b51ffb700 1 -- 172.21.15.143:0/1493775832 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1b54012070 con 0x7f1b5c157e30 2023-11-08T19:25:16.450 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:16.439+0000 7f1b61753700 1 --2- 172.21.15.143:0/1493775832 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f1b4006af00 0x7f1b4006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:16.450 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:16.439+0000 7f1b61753700 1 --2- 172.21.15.143:0/1493775832 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f1b4006af00 0x7f1b4006d3c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c009930 tx=0x7f1b4c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:16.450 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:16.443+0000 7f1b51ffb700 1 -- 172.21.15.143:0/1493775832 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1b54054e30 con 0x7f1b5c157e30 2023-11-08T19:25:16.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:16.676+0000 7f1b639b7700 1 -- 172.21.15.143:0/1493775832 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f1b5c0c6e60 con 0x7f1b5c157e30 2023-11-08T19:25:16.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:16.676+0000 7f1b51ffb700 1 -- 172.21.15.143:0/1493775832 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f1b54058450 con 0x7f1b5c157e30 2023-11-08T19:25:16.679 INFO:teuthology.orchestra.run.smithi143.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-08T19:25:16.681 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:16.678+0000 7f1b639b7700 1 -- 172.21.15.143:0/1493775832 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f1b4006af00 msgr2=0x7f1b4006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:16.681 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:16.678+0000 7f1b639b7700 1 --2- 172.21.15.143:0/1493775832 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f1b4006af00 0x7f1b4006d3c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c009930 tx=0x7f1b4c008040 comp rx=0 tx=0).stop 2023-11-08T19:25:16.682 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:16.678+0000 7f1b639b7700 1 -- 172.21.15.143:0/1493775832 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1b5c157e30 msgr2=0x7f1b5c0b39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:16.682 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:16.678+0000 7f1b639b7700 1 --2- 172.21.15.143:0/1493775832 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1b5c157e30 0x7f1b5c0b39b0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1b5400cd70 tx=0x7f1b5400a570 comp rx=0 tx=0).stop 2023-11-08T19:25:16.682 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:16.678+0000 7f1b639b7700 1 -- 172.21.15.143:0/1493775832 shutdown_connections 2023-11-08T19:25:16.682 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:16.678+0000 7f1b639b7700 1 --2- 172.21.15.143:0/1493775832 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f1b4006af00 0x7f1b4006d3c0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:16.683 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:16.678+0000 7f1b639b7700 1 --2- 172.21.15.143:0/1493775832 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1b5c157e30 0x7f1b5c0b39b0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:16.683 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:16.678+0000 7f1b639b7700 1 --2- 172.21.15.143:0/1493775832 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1b5c071aa0 0x7f1b5c14b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:16.683 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:16.678+0000 7f1b639b7700 1 --2- 172.21.15.143:0/1493775832 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1b5c0cd700 0x7f1b5c154a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:16.683 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:16.678+0000 7f1b639b7700 1 -- 172.21.15.143:0/1493775832 >> 172.21.15.143:0/1493775832 conn(0x7f1b5c06b4c0 msgr2=0x7f1b5c155c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:16.683 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:16.679+0000 7f1b639b7700 1 -- 172.21.15.143:0/1493775832 shutdown_connections 2023-11-08T19:25:16.684 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:16.679+0000 7f1b639b7700 1 -- 172.21.15.143:0/1493775832 wait complete. 2023-11-08T19:25:16.692 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-08T19:25:16.692 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-08T19:25:16.693 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi174.front.sepia.ceph.com 2023-11-08T19:25:16.693 INFO:tasks.cephfs.mount:self.client.name = client.3 2023-11-08T19:25:16.693 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-11-08T19:25:16.694 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-08T19:25:16.694 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-08T19:25:16.695 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-08T19:25:16.695 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2023-11-08T19:25:16.696 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:16.696 DEBUG:teuthology.orchestra.run.smithi174:> ip addr 2023-11-08T19:25:16.715 INFO:teuthology.orchestra.run.smithi174.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-08T19:25:16.715 INFO:teuthology.orchestra.run.smithi174.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-08T19:25:16.715 INFO:teuthology.orchestra.run.smithi174.stdout: inet 127.0.0.1/8 scope host lo 2023-11-08T19:25:16.716 INFO:teuthology.orchestra.run.smithi174.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:25:16.716 INFO:teuthology.orchestra.run.smithi174.stdout: inet6 ::1/128 scope host 2023-11-08T19:25:16.716 INFO:teuthology.orchestra.run.smithi174.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:25:16.716 INFO:teuthology.orchestra.run.smithi174.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-08T19:25:16.716 INFO:teuthology.orchestra.run.smithi174.stdout: link/ether 0c:c4:7a:d9:36:e0 brd ff:ff:ff:ff:ff:ff 2023-11-08T19:25:16.717 INFO:teuthology.orchestra.run.smithi174.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-08T19:25:16.717 INFO:teuthology.orchestra.run.smithi174.stdout: link/ether 0c:c4:7a:d9:36:e1 brd ff:ff:ff:ff:ff:ff 2023-11-08T19:25:16.717 INFO:teuthology.orchestra.run.smithi174.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-08T19:25:16.717 INFO:teuthology.orchestra.run.smithi174.stdout: link/ether 0c:c4:7a:88:77:1a brd ff:ff:ff:ff:ff:ff 2023-11-08T19:25:16.718 INFO:teuthology.orchestra.run.smithi174.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-08T19:25:16.718 INFO:teuthology.orchestra.run.smithi174.stdout: link/ether 0c:c4:7a:88:77:1b brd ff:ff:ff:ff:ff:ff 2023-11-08T19:25:16.718 INFO:teuthology.orchestra.run.smithi174.stdout: inet 172.21.15.174/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-08T19:25:16.718 INFO:teuthology.orchestra.run.smithi174.stdout: valid_lft 42343sec preferred_lft 42343sec 2023-11-08T19:25:16.719 INFO:teuthology.orchestra.run.smithi174.stdout: inet6 fe80::a48d:9583:b83f:827c/64 scope link noprefixroute 2023-11-08T19:25:16.719 INFO:teuthology.orchestra.run.smithi174.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:25:16.719 INFO:teuthology.orchestra.run.smithi174.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-08T19:25:16.719 INFO:teuthology.orchestra.run.smithi174.stdout: link/ether 02:24:aa:d4:f1:e3 brd ff:ff:ff:ff:ff:ff 2023-11-08T19:25:16.719 INFO:teuthology.orchestra.run.smithi174.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-08T19:25:16.720 INFO:teuthology.orchestra.run.smithi174.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:25:16.720 INFO:teuthology.orchestra.run.smithi174.stdout: inet6 fe80::80e4:ecff:fe70:1583/64 scope link 2023-11-08T19:25:16.720 INFO:teuthology.orchestra.run.smithi174.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:25:16.720 INFO:teuthology.orchestra.run.smithi174.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-08T19:25:16.721 INFO:teuthology.orchestra.run.smithi174.stdout: link/ether 02:24:aa:d4:f1:e3 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-08T19:25:16.721 INFO:teuthology.orchestra.run.smithi174.stdout: inet6 fe80::24:aaff:fed4:f1e3/64 scope link tentative 2023-11-08T19:25:16.721 INFO:teuthology.orchestra.run.smithi174.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:25:16.721 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:16.722 DEBUG:teuthology.orchestra.run.smithi174:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-08T19:25:16.781 INFO:teuthology.orchestra.run.smithi174.stdout:1 2023-11-08T19:25:16.783 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:16.783 DEBUG:teuthology.orchestra.run.smithi174:> route 2023-11-08T19:25:16.841 INFO:teuthology.orchestra.run.smithi174.stdout:Kernel IP routing table 2023-11-08T19:25:16.842 INFO:teuthology.orchestra.run.smithi174.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-08T19:25:16.842 INFO:teuthology.orchestra.run.smithi174.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-08T19:25:16.842 INFO:teuthology.orchestra.run.smithi174.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-08T19:25:16.842 INFO:teuthology.orchestra.run.smithi174.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-08T19:25:16.843 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:16.844 DEBUG:teuthology.orchestra.run.smithi174:> (cd / && exec bash -c ' 2023-11-08T19:25:16.844 DEBUG:teuthology.orchestra.run.smithi174:> set -e 2023-11-08T19:25:16.845 DEBUG:teuthology.orchestra.run.smithi174:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-08T19:25:16.845 DEBUG:teuthology.orchestra.run.smithi174:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-08T19:25:16.845 DEBUG:teuthology.orchestra.run.smithi174:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-08T19:25:16.846 DEBUG:teuthology.orchestra.run.smithi174:> ') 2023-11-08T19:25:16.948 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:16.949 DEBUG:teuthology.orchestra.run.smithi174:> ip netns list 2023-11-08T19:25:16.965 INFO:teuthology.orchestra.run.smithi174.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-08T19:25:16.966 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:16.966 DEBUG:teuthology.orchestra.run.smithi174:> ip netns list-id 2023-11-08T19:25:17.021 INFO:teuthology.orchestra.run.smithi174.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2023-11-08T19:25:17.022 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:17.022 DEBUG:teuthology.orchestra.run.smithi174:> (cd / && exec bash -c ' 2023-11-08T19:25:17.023 DEBUG:teuthology.orchestra.run.smithi174:> set -e 2023-11-08T19:25:17.023 DEBUG:teuthology.orchestra.run.smithi174:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-08T19:25:17.023 DEBUG:teuthology.orchestra.run.smithi174:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2023-11-08T19:25:17.023 DEBUG:teuthology.orchestra.run.smithi174:> ') 2023-11-08T19:25:17.114 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:17.114 DEBUG:teuthology.orchestra.run.smithi174:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-11-08T19:25:17.149 INFO:teuthology.orchestra.run.smithi174.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-08T19:25:17.150 INFO:teuthology.orchestra.run.smithi174.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-08T19:25:17.150 INFO:teuthology.orchestra.run.smithi174.stdout: inet 127.0.0.1/8 scope host lo 2023-11-08T19:25:17.150 INFO:teuthology.orchestra.run.smithi174.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:25:17.150 INFO:teuthology.orchestra.run.smithi174.stdout: inet6 ::1/128 scope host 2023-11-08T19:25:17.150 INFO:teuthology.orchestra.run.smithi174.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:25:17.151 INFO:teuthology.orchestra.run.smithi174.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-08T19:25:17.151 INFO:teuthology.orchestra.run.smithi174.stdout: link/ether 22:78:e2:80:47:06 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-08T19:25:17.151 INFO:teuthology.orchestra.run.smithi174.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-08T19:25:17.151 INFO:teuthology.orchestra.run.smithi174.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:25:17.151 INFO:teuthology.orchestra.run.smithi174.stdout: inet6 fe80::2078:e2ff:fe80:4706/64 scope link tentative 2023-11-08T19:25:17.152 INFO:teuthology.orchestra.run.smithi174.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:25:17.163 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:17.164 DEBUG:teuthology.orchestra.run.smithi174:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-11-08T19:25:17.248 INFO:teuthology.orchestra.run.smithi174.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-08T19:25:17.248 INFO:teuthology.orchestra.run.smithi174.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-08T19:25:17.249 INFO:teuthology.orchestra.run.smithi174.stdout: inet 127.0.0.1/8 scope host lo 2023-11-08T19:25:17.249 INFO:teuthology.orchestra.run.smithi174.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:25:17.249 INFO:teuthology.orchestra.run.smithi174.stdout: inet6 ::1/128 scope host 2023-11-08T19:25:17.249 INFO:teuthology.orchestra.run.smithi174.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:25:17.249 INFO:teuthology.orchestra.run.smithi174.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-08T19:25:17.250 INFO:teuthology.orchestra.run.smithi174.stdout: link/ether 22:78:e2:80:47:06 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-08T19:25:17.250 INFO:teuthology.orchestra.run.smithi174.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-08T19:25:17.250 INFO:teuthology.orchestra.run.smithi174.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:25:17.250 INFO:teuthology.orchestra.run.smithi174.stdout: inet6 fe80::2078:e2ff:fe80:4706/64 scope link 2023-11-08T19:25:17.251 INFO:teuthology.orchestra.run.smithi174.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:25:17.265 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2023-11-08T19:25:17.266 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:17.266 DEBUG:teuthology.orchestra.run.smithi174:> (cd / && exec bash -c ' 2023-11-08T19:25:17.267 DEBUG:teuthology.orchestra.run.smithi174:> set -e 2023-11-08T19:25:17.267 DEBUG:teuthology.orchestra.run.smithi174:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2023-11-08T19:25:17.267 DEBUG:teuthology.orchestra.run.smithi174:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-11-08T19:25:17.268 DEBUG:teuthology.orchestra.run.smithi174:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2023-11-08T19:25:17.268 DEBUG:teuthology.orchestra.run.smithi174:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2023-11-08T19:25:17.268 DEBUG:teuthology.orchestra.run.smithi174:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2023-11-08T19:25:17.269 DEBUG:teuthology.orchestra.run.smithi174:> ') 2023-11-08T19:25:17.485 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:17.485 DEBUG:teuthology.orchestra.run.smithi174:> (cd / && exec bash -c ' 2023-11-08T19:25:17.486 DEBUG:teuthology.orchestra.run.smithi174:> set -e 2023-11-08T19:25:17.486 DEBUG:teuthology.orchestra.run.smithi174:> sudo ip link set brx.1 up 2023-11-08T19:25:17.486 DEBUG:teuthology.orchestra.run.smithi174:> sudo ip link set dev brx.1 master ceph-brx 2023-11-08T19:25:17.487 DEBUG:teuthology.orchestra.run.smithi174:> ') 2023-11-08T19:25:17.596 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-08T19:25:17.597 DEBUG:teuthology.orchestra.run.smithi174:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2023-11-08T19:25:17.619 INFO:teuthology.orchestra.run.smithi174.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2023-11-08T19:25:17.619 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-08T19:25:17.620 DEBUG:teuthology.orchestra.run.smithi174:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2023-11-08T19:25:17.675 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-11-08T19:25:17.676 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:17.676 DEBUG:teuthology.orchestra.run.smithi174:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2023-11-08T19:25:17.761 INFO:teuthology.orchestra.run.smithi174.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2023-11-08T19:25:17.761 INFO:teuthology.orchestra.run.smithi174.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2023-11-08T19:25:17.761 INFO:teuthology.orchestra.run.smithi174.stdout:invalid new device string format 2023-11-08T19:25:17.770 INFO:teuthology.orchestra.run.smithi174.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2023-11-08T19:25:17.770 INFO:teuthology.orchestra.run.smithi174.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2023-11-08T19:25:17.770 INFO:teuthology.orchestra.run.smithi174.stdout:invalid new device string format 2023-11-08T19:25:17.770 INFO:teuthology.orchestra.run.smithi174.stdout:mount.ceph: resolved to: "172.21.15.143:3300,172.21.15.174:3300,172.21.15.174:3301" 2023-11-08T19:25:17.771 INFO:teuthology.orchestra.run.smithi174.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-08T19:25:17.798 INFO:teuthology.orchestra.run.smithi174.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2023-11-08T19:25:17.799 INFO:teuthology.orchestra.run.smithi174.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2023-11-08T19:25:17.799 INFO:teuthology.orchestra.run.smithi174.stdout:invalid new device string format 2023-11-08T19:25:17.799 INFO:teuthology.orchestra.run.smithi174.stdout:mount.ceph: resolved to: "172.21.15.143:3300,172.21.15.174:3300,172.21.15.174:3301" 2023-11-08T19:25:17.800 INFO:teuthology.orchestra.run.smithi174.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-08T19:25:17.800 INFO:teuthology.orchestra.run.smithi174.stdout:mount.ceph: resolved to: "172.21.15.143:6789,172.21.15.174:6789,172.21.15.174:6790" 2023-11-08T19:25:17.800 INFO:teuthology.orchestra.run.smithi174.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.143:6789,172.21.15.174:6789,172.21.15.174:6790:/ 2023-11-08T19:25:17.801 INFO:teuthology.orchestra.run.smithi174.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs,key=3,fsid=968483af-77fc-4068-ad7c-5c2a1a4613d7" will pass to kernel 2023-11-08T19:25:17.802 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-08T19:25:17.803 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:17.803 DEBUG:teuthology.orchestra.run.smithi174:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2023-11-08T19:25:17.837 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:17.838 DEBUG:teuthology.orchestra.run.smithi174:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-08T19:25:17.909 INFO:teuthology.orchestra.run.smithi174.stdout:client4457 2023-11-08T19:25:17.910 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2023-11-08T19:25:17.910 DEBUG:teuthology.orchestra.run.smithi174:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:25:17.967 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:17.968 DEBUG:teuthology.orchestra.run.smithi174:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-08T19:25:18.039 INFO:teuthology.orchestra.run.smithi174.stdout:client4457 2023-11-08T19:25:18.039 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:18.040 DEBUG:teuthology.orchestra.run.smithi174:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-08T19:25:18.111 INFO:teuthology.orchestra.run.smithi174.stdout:client4457 2023-11-08T19:25:18.111 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:18.112 DEBUG:teuthology.orchestra.run.smithi174:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/968483af-77fc-4068-ad7c-5c2a1a4613d7.client4457/status') 2023-11-08T19:25:18.191 INFO:teuthology.orchestra.run.smithi174.stdout:instance: client.4457 (3)192.168.0.2:0/2811799056 2023-11-08T19:25:18.192 INFO:teuthology.orchestra.run.smithi174.stdout:blocklisted: false 2023-11-08T19:25:18.193 INFO:teuthology.orchestra.run.smithi174.stderr:0+1 records in 2023-11-08T19:25:18.193 INFO:teuthology.orchestra.run.smithi174.stderr:0+1 records out 2023-11-08T19:25:18.193 INFO:teuthology.orchestra.run.smithi174.stderr:69 bytes copied, 2.5977e-05 s, 2.7 MB/s 2023-11-08T19:25:18.194 INFO:teuthology.run_tasks:Running task exec... 2023-11-08T19:25:18.204 INFO:teuthology.task.exec:Executing custom commands... 2023-11-08T19:25:18.205 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi143.front.sepia.ceph.com 2023-11-08T19:25:18.205 DEBUG:teuthology.orchestra.run.smithi143:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/crash/warn_recent_interval 0' 2023-11-08T19:25:18.339 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.338+0000 7f179142a700 1 Processor -- start 2023-11-08T19:25:18.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.338+0000 7f179142a700 1 -- start start 2023-11-08T19:25:18.342 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.339+0000 7f179142a700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f178c0d3470 0x7f178c0d2e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:18.342 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.339+0000 7f179142a700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f178c06afe0 0x7f178c06b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:18.343 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.339+0000 7f179142a700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f178c06b970 con 0x7f178c15dcd0 2023-11-08T19:25:18.343 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.339+0000 7f179142a700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f178c0b9b90 con 0x7f178c06afe0 2023-11-08T19:25:18.343 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.339+0000 7f179142a700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f178c0ceaa0 con 0x7f178c0d3470 2023-11-08T19:25:18.343 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.339+0000 7f178b7fe700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f178c06afe0 0x7f178c06b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:18.344 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.339+0000 7f178b7fe700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f178c06afe0 0x7f178c06b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49484/0 (socket says 172.21.15.143:49484) 2023-11-08T19:25:18.344 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.339+0000 7f178b7fe700 1 -- 172.21.15.143:0/2897058053 learned_addr learned my addr 172.21.15.143:0/2897058053 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:18.344 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.339+0000 7f178a7fc700 1 --2- 172.21.15.143:0/2897058053 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f178c0d3470 0x7f178c0d2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:18.344 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.339+0000 7f178b7fe700 1 -- 172.21.15.143:0/2897058053 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f178c0d3470 msgr2=0x7f178c0d2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:18.344 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.339+0000 7f178b7fe700 1 --2- 172.21.15.143:0/2897058053 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f178c0d3470 0x7f178c0d2e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:18.344 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.339+0000 7f178b7fe700 1 -- 172.21.15.143:0/2897058053 >> v1:172.21.15.174:6789/0 conn(0x7f178c15dcd0 legacy=0x7f178c0b9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:18.345 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.339+0000 7f178b7fe700 1 -- 172.21.15.143:0/2897058053 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f178c0ced80 con 0x7f178c06afe0 2023-11-08T19:25:18.345 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.339+0000 7f178b7fe700 1 --2- 172.21.15.143:0/2897058053 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f178c06afe0 0x7f178c06b430 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f1784009e50 tx=0x7f178400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=290cce646b51c3f5 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:18.345 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.340+0000 7f1789ffb700 1 -- 172.21.15.143:0/2897058053 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f178400e070 con 0x7f178c06afe0 2023-11-08T19:25:18.345 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.340+0000 7f1789ffb700 1 -- 172.21.15.143:0/2897058053 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f178400b910 con 0x7f178c06afe0 2023-11-08T19:25:18.346 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.340+0000 7f1789ffb700 1 -- 172.21.15.143:0/2897058053 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f178400bcc0 con 0x7f178c06afe0 2023-11-08T19:25:18.346 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.340+0000 7f179142a700 1 -- 172.21.15.143:0/2897058053 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f178c06afe0 msgr2=0x7f178c06b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:18.346 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.340+0000 7f179142a700 1 --2- 172.21.15.143:0/2897058053 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f178c06afe0 0x7f178c06b430 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f1784009e50 tx=0x7f178400b330 comp rx=0 tx=0).stop 2023-11-08T19:25:18.346 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.340+0000 7f179142a700 1 -- 172.21.15.143:0/2897058053 shutdown_connections 2023-11-08T19:25:18.346 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.340+0000 7f179142a700 1 --2- 172.21.15.143:0/2897058053 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f178c06afe0 0x7f178c06b430 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:18.347 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.340+0000 7f179142a700 1 --2- 172.21.15.143:0/2897058053 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f178c0d3470 0x7f178c0d2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:18.347 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.340+0000 7f179142a700 1 -- 172.21.15.143:0/2897058053 >> 172.21.15.143:0/2897058053 conn(0x7f178c0d3b50 msgr2=0x7f178c0d3f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:18.347 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.340+0000 7f179142a700 1 -- 172.21.15.143:0/2897058053 shutdown_connections 2023-11-08T19:25:18.347 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.340+0000 7f179142a700 1 -- 172.21.15.143:0/2897058053 wait complete. 2023-11-08T19:25:18.350 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.342+0000 7f179142a700 1 Processor -- start 2023-11-08T19:25:18.350 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.342+0000 7f179142a700 1 -- start start 2023-11-08T19:25:18.350 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.342+0000 7f179142a700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f178c06afe0 0x7f178c1553c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:18.350 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.342+0000 7f179142a700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f178c0d3470 0x7f178c1572b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:18.351 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.342+0000 7f179142a700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f178c15dcd0 0x7f178c15ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:18.351 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.342+0000 7f179142a700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f178c0cb840 con 0x7f178c0d3470 2023-11-08T19:25:18.351 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.342+0000 7f179142a700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f178c0cac30 con 0x7f178c15dcd0 2023-11-08T19:25:18.351 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.342+0000 7f179142a700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f178c0caf10 con 0x7f178c06afe0 2023-11-08T19:25:18.351 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.342+0000 7f178a7fc700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f178c0d3470 0x7f178c1572b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:18.352 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.342+0000 7f178a7fc700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f178c0d3470 0x7f178c1572b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49488/0 (socket says 172.21.15.143:49488) 2023-11-08T19:25:18.352 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.342+0000 7f178a7fc700 1 -- 172.21.15.143:0/2621435045 learned_addr learned my addr 172.21.15.143:0/2621435045 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:18.352 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.342+0000 7f178b7fe700 1 --2- 172.21.15.143:0/2621435045 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f178c15dcd0 0x7f178c15ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:18.352 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.342+0000 7f178affd700 1 --2- 172.21.15.143:0/2621435045 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f178c06afe0 0x7f178c1553c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:18.352 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.343+0000 7f178a7fc700 1 -- 172.21.15.143:0/2621435045 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f178c06afe0 msgr2=0x7f178c1553c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:18.353 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.343+0000 7f178a7fc700 1 --2- 172.21.15.143:0/2621435045 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f178c06afe0 0x7f178c1553c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:18.353 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.343+0000 7f178a7fc700 1 -- 172.21.15.143:0/2621435045 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f178c15dcd0 msgr2=0x7f178c15ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:18.353 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.343+0000 7f178a7fc700 1 --2- 172.21.15.143:0/2621435045 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f178c15dcd0 0x7f178c15ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:18.353 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.343+0000 7f178a7fc700 1 -- 172.21.15.143:0/2621435045 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1784009b30 con 0x7f178c0d3470 2023-11-08T19:25:18.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.343+0000 7f178a7fc700 1 --2- 172.21.15.143:0/2621435045 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f178c0d3470 0x7f178c1572b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f177c00e9b0 tx=0x7f177c00ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:18.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.343+0000 7f17737fe700 1 -- 172.21.15.143:0/2621435045 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f177c009710 con 0x7f178c0d3470 2023-11-08T19:25:18.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.343+0000 7f17737fe700 1 -- 172.21.15.143:0/2621435045 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f177c00ee50 con 0x7f178c0d3470 2023-11-08T19:25:18.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.343+0000 7f17737fe700 1 -- 172.21.15.143:0/2621435045 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f177c00ca50 con 0x7f178c0d3470 2023-11-08T19:25:18.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.343+0000 7f179142a700 1 -- 172.21.15.143:0/2621435045 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f178c0cb1f0 con 0x7f178c0d3470 2023-11-08T19:25:18.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.343+0000 7f179142a700 1 -- 172.21.15.143:0/2621435045 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f178c15f1a0 con 0x7f178c0d3470 2023-11-08T19:25:18.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.344+0000 7f179142a700 1 -- 172.21.15.143:0/2621435045 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f178c0d1d00 con 0x7f178c0d3470 2023-11-08T19:25:18.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.344+0000 7f17737fe700 1 -- 172.21.15.143:0/2621435045 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f177c00cbb0 con 0x7f178c0d3470 2023-11-08T19:25:18.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.345+0000 7f17737fe700 1 --2- 172.21.15.143:0/2621435045 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f177806af00 0x7f177806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:18.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.345+0000 7f17737fe700 1 -- 172.21.15.143:0/2621435045 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f177c089570 con 0x7f178c0d3470 2023-11-08T19:25:18.356 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.345+0000 7f178affd700 1 --2- 172.21.15.143:0/2621435045 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f177806af00 0x7f177806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:18.356 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.345+0000 7f178affd700 1 --2- 172.21.15.143:0/2621435045 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f177806af00 0x7f177806d3c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f1780007fd0 tx=0x7f1780009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:18.356 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.349+0000 7f17737fe700 1 -- 172.21.15.143:0/2621435045 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f177c0546b0 con 0x7f178c0d3470 2023-11-08T19:25:18.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.538+0000 7f179142a700 1 -- 172.21.15.143:0/2621435045 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command([{prefix=config set, name=mgr/crash/warn_recent_interval}] v 0) v1 -- 0x7f178c04f730 con 0x7f178c0d3470 2023-11-08T19:25:18.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.541+0000 7f17737fe700 1 -- 172.21.15.143:0/2621435045 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/crash/warn_recent_interval}]=0 v10)=0 v10) v1 ==== 128+0+0 (secure 0 0 0) 0x7f177c00ce60 con 0x7f178c0d3470 2023-11-08T19:25:18.545 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.543+0000 7f179142a700 1 -- 172.21.15.143:0/2621435045 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f177806af00 msgr2=0x7f177806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:18.545 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.543+0000 7f179142a700 1 --2- 172.21.15.143:0/2621435045 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f177806af00 0x7f177806d3c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f1780007fd0 tx=0x7f1780009040 comp rx=0 tx=0).stop 2023-11-08T19:25:18.545 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.543+0000 7f179142a700 1 -- 172.21.15.143:0/2621435045 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f178c0d3470 msgr2=0x7f178c1572b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:18.546 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.543+0000 7f179142a700 1 --2- 172.21.15.143:0/2621435045 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f178c0d3470 0x7f178c1572b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f177c00e9b0 tx=0x7f177c00ecc0 comp rx=0 tx=0).stop 2023-11-08T19:25:18.546 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.543+0000 7f179142a700 1 -- 172.21.15.143:0/2621435045 shutdown_connections 2023-11-08T19:25:18.546 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.543+0000 7f179142a700 1 --2- 172.21.15.143:0/2621435045 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f177806af00 0x7f177806d3c0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:18.546 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.543+0000 7f179142a700 1 --2- 172.21.15.143:0/2621435045 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f178c06afe0 0x7f178c1553c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:18.546 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.543+0000 7f179142a700 1 --2- 172.21.15.143:0/2621435045 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f178c15dcd0 0x7f178c15ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:18.547 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.543+0000 7f179142a700 1 --2- 172.21.15.143:0/2621435045 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f178c0d3470 0x7f178c1572b0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:18.547 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.543+0000 7f179142a700 1 -- 172.21.15.143:0/2621435045 >> 172.21.15.143:0/2621435045 conn(0x7f178c0d3b50 msgr2=0x7f178c0d3f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:18.547 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.543+0000 7f179142a700 1 -- 172.21.15.143:0/2621435045 shutdown_connections 2023-11-08T19:25:18.547 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.543+0000 7f179142a700 1 -- 172.21.15.143:0/2621435045 wait complete. 2023-11-08T19:25:18.557 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2023-11-08T19:25:18.572 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:25:18.694 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.692+0000 7f1d94441700 1 Processor -- start 2023-11-08T19:25:18.694 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.692+0000 7f1d94441700 1 -- start start 2023-11-08T19:25:18.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.693+0000 7f1d94441700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f1d8c15ba30 0x7f1d8c0b14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:18.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.693+0000 7f1d94441700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f1d8c0d1c70 0x7f1d8c0cc5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:18.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.693+0000 7f1d94441700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d8c0b1bc0 con 0x7f1d8c0d03a0 2023-11-08T19:25:18.697 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.693+0000 7f1d94441700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d8c0b1ea0 con 0x7f1d8c0d1c70 2023-11-08T19:25:18.697 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.693+0000 7f1d94441700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d8c0b2180 con 0x7f1d8c15ba30 2023-11-08T19:25:18.697 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.693+0000 7f1d929de700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f1d8c0d1c70 0x7f1d8c0cc5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:18.697 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.693+0000 7f1d929de700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f1d8c0d1c70 0x7f1d8c0cc5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49496/0 (socket says 172.21.15.143:49496) 2023-11-08T19:25:18.697 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.693+0000 7f1d929de700 1 -- 172.21.15.143:0/3324896560 learned_addr learned my addr 172.21.15.143:0/3324896560 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:18.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.694+0000 7f1d921dd700 1 --2- 172.21.15.143:0/3324896560 >> v2:172.21.15.174:3300/0 conn(0x7f1d8c15ba30 0x7f1d8c0b14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:18.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.694+0000 7f1d929de700 1 -- 172.21.15.143:0/3324896560 >> v2:172.21.15.174:3300/0 conn(0x7f1d8c15ba30 msgr2=0x7f1d8c0b14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:18.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.694+0000 7f1d929de700 1 --2- 172.21.15.143:0/3324896560 >> v2:172.21.15.174:3300/0 conn(0x7f1d8c15ba30 0x7f1d8c0b14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:18.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.694+0000 7f1d929de700 1 -- 172.21.15.143:0/3324896560 >> v1:172.21.15.174:6789/0 conn(0x7f1d8c0d03a0 legacy=0x7f1d8c0d1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:18.699 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.694+0000 7f1d929de700 1 -- 172.21.15.143:0/3324896560 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d8c0b2460 con 0x7f1d8c0d1c70 2023-11-08T19:25:18.699 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.694+0000 7f1d929de700 1 --2- 172.21.15.143:0/3324896560 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1d8c0d1c70 0x7f1d8c0cc5a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1d8c0cf3d0 tx=0x7f1d8400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90a8c7c8af07b962 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:18.699 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.694+0000 7f1d911db700 1 -- 172.21.15.143:0/3324896560 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d8400e070 con 0x7f1d8c0d1c70 2023-11-08T19:25:18.699 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.694+0000 7f1d911db700 1 -- 172.21.15.143:0/3324896560 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d8400b9c0 con 0x7f1d8c0d1c70 2023-11-08T19:25:18.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.694+0000 7f1d911db700 1 -- 172.21.15.143:0/3324896560 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d8400bd70 con 0x7f1d8c0d1c70 2023-11-08T19:25:18.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.695+0000 7f1d94441700 1 -- 172.21.15.143:0/3324896560 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1d8c0d1c70 msgr2=0x7f1d8c0cc5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:18.701 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.695+0000 7f1d94441700 1 --2- 172.21.15.143:0/3324896560 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1d8c0d1c70 0x7f1d8c0cc5a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1d8c0cf3d0 tx=0x7f1d8400b330 comp rx=0 tx=0).stop 2023-11-08T19:25:18.701 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.695+0000 7f1d94441700 1 -- 172.21.15.143:0/3324896560 shutdown_connections 2023-11-08T19:25:18.701 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.695+0000 7f1d94441700 1 --2- 172.21.15.143:0/3324896560 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1d8c0d1c70 0x7f1d8c0cc5a0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:18.701 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.695+0000 7f1d94441700 1 --2- 172.21.15.143:0/3324896560 >> v2:172.21.15.174:3300/0 conn(0x7f1d8c15ba30 0x7f1d8c0b14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:18.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.695+0000 7f1d94441700 1 -- 172.21.15.143:0/3324896560 >> 172.21.15.143:0/3324896560 conn(0x7f1d8c08b000 msgr2=0x7f1d8c09ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:18.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.695+0000 7f1d94441700 1 -- 172.21.15.143:0/3324896560 shutdown_connections 2023-11-08T19:25:18.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.695+0000 7f1d94441700 1 -- 172.21.15.143:0/3324896560 wait complete. 2023-11-08T19:25:18.705 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.696+0000 7f1d94441700 1 Processor -- start 2023-11-08T19:25:18.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.696+0000 7f1d94441700 1 -- start start 2023-11-08T19:25:18.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.697+0000 7f1d94441700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1d8c0d03a0 0x7f1d8c0c7e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:18.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.697+0000 7f1d94441700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1d8c0d1c70 0x7f1d8c14cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:18.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.697+0000 7f1d94441700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1d8c15ba30 0x7f1d8c0c8980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:18.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.697+0000 7f1d94441700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d8c0b6b60 con 0x7f1d8c15ba30 2023-11-08T19:25:18.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.697+0000 7f1d94441700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d8c0b6880 con 0x7f1d8c0d03a0 2023-11-08T19:25:18.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.697+0000 7f1d94441700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d8c0c5f20 con 0x7f1d8c0d1c70 2023-11-08T19:25:18.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.697+0000 7f1d929de700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1d8c15ba30 0x7f1d8c0c8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:18.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.697+0000 7f1d929de700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1d8c15ba30 0x7f1d8c0c8980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49502/0 (socket says 172.21.15.143:49502) 2023-11-08T19:25:18.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.697+0000 7f1d921dd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1d8c0d03a0 0x7f1d8c0c7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:18.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.697+0000 7f1d929de700 1 -- 172.21.15.143:0/2469343759 learned_addr learned my addr 172.21.15.143:0/2469343759 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:18.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.697+0000 7f1d919dc700 1 --2- 172.21.15.143:0/2469343759 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1d8c0d1c70 0x7f1d8c14cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:18.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.697+0000 7f1d929de700 1 -- 172.21.15.143:0/2469343759 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1d8c0d1c70 msgr2=0x7f1d8c14cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:18.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.697+0000 7f1d929de700 1 --2- 172.21.15.143:0/2469343759 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1d8c0d1c70 0x7f1d8c14cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:18.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.697+0000 7f1d929de700 1 -- 172.21.15.143:0/2469343759 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1d8c0d03a0 msgr2=0x7f1d8c0c7e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:18.709 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.697+0000 7f1d929de700 1 --2- 172.21.15.143:0/2469343759 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1d8c0d03a0 0x7f1d8c0c7e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:18.709 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.697+0000 7f1d929de700 1 -- 172.21.15.143:0/2469343759 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d84009b30 con 0x7f1d8c15ba30 2023-11-08T19:25:18.709 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.697+0000 7f1d929de700 1 --2- 172.21.15.143:0/2469343759 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1d8c15ba30 0x7f1d8c0c8980 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1d8c08c000 tx=0x7f1d84004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:18.709 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.697+0000 7f1d82ffd700 1 -- 172.21.15.143:0/2469343759 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d84004880 con 0x7f1d8c15ba30 2023-11-08T19:25:18.709 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.697+0000 7f1d82ffd700 1 -- 172.21.15.143:0/2469343759 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d840099b0 con 0x7f1d8c15ba30 2023-11-08T19:25:18.710 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.697+0000 7f1d82ffd700 1 -- 172.21.15.143:0/2469343759 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d8401c680 con 0x7f1d8c15ba30 2023-11-08T19:25:18.710 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.698+0000 7f1d94441700 1 -- 172.21.15.143:0/2469343759 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d8c0c6200 con 0x7f1d8c15ba30 2023-11-08T19:25:18.710 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.698+0000 7f1d94441700 1 -- 172.21.15.143:0/2469343759 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d8c14d460 con 0x7f1d8c15ba30 2023-11-08T19:25:18.710 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.698+0000 7f1d94441700 1 -- 172.21.15.143:0/2469343759 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d8c0d0780 con 0x7f1d8c15ba30 2023-11-08T19:25:18.710 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.699+0000 7f1d82ffd700 1 -- 172.21.15.143:0/2469343759 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1d8401c7e0 con 0x7f1d8c15ba30 2023-11-08T19:25:18.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.699+0000 7f1d82ffd700 1 --2- 172.21.15.143:0/2469343759 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f1d7006af00 0x7f1d7006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:18.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.700+0000 7f1d82ffd700 1 -- 172.21.15.143:0/2469343759 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1d8408b150 con 0x7f1d8c15ba30 2023-11-08T19:25:18.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.700+0000 7f1d921dd700 1 --2- 172.21.15.143:0/2469343759 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f1d7006af00 0x7f1d7006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:18.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.700+0000 7f1d921dd700 1 --2- 172.21.15.143:0/2469343759 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f1d7006af00 0x7f1d7006d3c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f1d8c0c7e00 tx=0x7f1d7c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:18.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.704+0000 7f1d82ffd700 1 -- 172.21.15.143:0/2469343759 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1d8c0d0780 con 0x7f1d8c15ba30 2023-11-08T19:25:18.896 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.893+0000 7f1d94441700 1 -- 172.21.15.143:0/2469343759 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1d8c0d0780 con 0x7f1d8c15ba30 2023-11-08T19:25:18.896 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.894+0000 7f1d82ffd700 1 -- 172.21.15.143:0/2469343759 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16553 (secure 0 0 0) 0x7f1d8c0d0780 con 0x7f1d8c15ba30 2023-11-08T19:25:18.897 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:25:18.897 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":20,"fsid":"968483af-77fc-4068-ad7c-5c2a1a4613d7","created":"2023-11-08T19:24:32.826484+0000","modified":"2023-11-08T19:24:51.618072+0000","last_up_change":"2023-11-08T19:24:43.573880+0000","last_in_change":"2023-11-08T19:24:35.314525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:44.551530+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-08T19:24:48.256781+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-08T19:24:48.954990+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bae98ec1-d94e-403d-a220-a522a53ea6f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6809","nonce":1487830566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6811","nonce":1487830566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6815","nonce":1487830566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6813","nonce":1487830566}]},"public_addr":"172.21.15.143:6809/1487830566","cluster_addr":"172.21.15.143:6811/1487830566","heartbeat_back_addr":"172.21.15.143:6815/1487830566","heartbeat_front_addr":"172.21.15.143:6813/1487830566","state":["exists","up"]},{"osd":1,"uuid":"5cb6538a-1ccb-4322-8633-09f66af0b7e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6801","nonce":787341397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6803","nonce":787341397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6807","nonce":787341397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6805","nonce":787341397}]},"public_addr":"172.21.15.143:6801/787341397","cluster_addr":"172.21.15.143:6803/787341397","heartbeat_back_addr":"172.21.15.143:6807/787341397","heartbeat_front_addr":"172.21.15.143:6805/787341397","state":["exists","up"]},{"osd":2,"uuid":"cdbf522c-60ef-4d60-9653-8dcb529c101a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6822","nonce":2301325597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6826","nonce":2301325597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6831","nonce":2301325597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6829","nonce":2301325597}]},"public_addr":"172.21.15.143:6822/2301325597","cluster_addr":"172.21.15.143:6826/2301325597","heartbeat_back_addr":"172.21.15.143:6831/2301325597","heartbeat_front_addr":"172.21.15.143:6829/2301325597","state":["exists","up"]},{"osd":3,"uuid":"4e2783cf-ebb9-4dd8-8245-bc1d17ce4325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6817","nonce":2021504428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6819","nonce":2021504428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6827","nonce":2021504428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6823","nonce":2021504428}]},"public_addr":"172.21.15.143:6817/2021504428","cluster_addr":"172.21.15.143:6819/2021504428","heartbeat_back_addr":"172.21.15.143:6827/2021504428","heartbeat_front_addr":"172.21.15.143:6823/2021504428","state":["exists","up"]},{"osd":4,"uuid":"691256fa-9b77-49b9-a346-7ae1adf0ca80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6801","nonce":1993380595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6803","nonce":1993380595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6807","nonce":1993380595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6805","nonce":1993380595}]},"public_addr":"172.21.15.174:6801/1993380595","cluster_addr":"172.21.15.174:6803/1993380595","heartbeat_back_addr":"172.21.15.174:6807/1993380595","heartbeat_front_addr":"172.21.15.174:6805/1993380595","state":["exists","up"]},{"osd":5,"uuid":"756fc352-48c9-46ce-b52e-0541fe6f8525","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6809","nonce":1777923303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6811","nonce":1777923303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6815","nonce":1777923303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6813","nonce":1777923303}]},"public_addr":"172.21.15.174:6809/1777923303","cluster_addr":"172.21.15.174:6811/1777923303","heartbeat_back_addr":"172.21.15.174:6815/1777923303","heartbeat_front_addr":"172.21.15.174:6813/1777923303","state":["exists","up"]},{"osd":6,"uuid":"fab2ac2b-ed97-4cde-9825-b480eabf8b8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6825","nonce":2755130316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6827","nonce":2755130316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6831","nonce":2755130316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6829","nonce":2755130316}]},"public_addr":"172.21.15.174:6825/2755130316","cluster_addr":"172.21.15.174:6827/2755130316","heartbeat_back_addr":"172.21.15.174:6831/2755130316","heartbeat_front_addr":"172.21.15.174:6829/2755130316","state":["exists","up"]},{"osd":7,"uuid":"3de9b797-2ab6-4d41-a863-7e7d638fd703","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6817","nonce":2847141411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6819","nonce":2847141411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6823","nonce":2847141411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6821","nonce":2847141411}]},"public_addr":"172.21.15.174:6817/2847141411","cluster_addr":"172.21.15.174:6819/2847141411","heartbeat_back_addr":"172.21.15.174:6823/2847141411","heartbeat_front_addr":"172.21.15.174:6821/2847141411","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.477076+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.496890+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.539305+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.454673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.538881+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.521294+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.520712+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.540404+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:25:18.898 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.896+0000 7f1d94441700 1 -- 172.21.15.143:0/2469343759 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f1d7006af00 msgr2=0x7f1d7006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:18.898 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.896+0000 7f1d94441700 1 --2- 172.21.15.143:0/2469343759 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f1d7006af00 0x7f1d7006d3c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f1d8c0c7e00 tx=0x7f1d7c008040 comp rx=0 tx=0).stop 2023-11-08T19:25:18.899 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.896+0000 7f1d94441700 1 -- 172.21.15.143:0/2469343759 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1d8c15ba30 msgr2=0x7f1d8c0c8980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:18.899 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.896+0000 7f1d94441700 1 --2- 172.21.15.143:0/2469343759 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1d8c15ba30 0x7f1d8c0c8980 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1d8c08c000 tx=0x7f1d84004ad0 comp rx=0 tx=0).stop 2023-11-08T19:25:18.899 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.896+0000 7f1d94441700 1 -- 172.21.15.143:0/2469343759 shutdown_connections 2023-11-08T19:25:18.899 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.896+0000 7f1d94441700 1 --2- 172.21.15.143:0/2469343759 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f1d7006af00 0x7f1d7006d3c0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:18.899 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.896+0000 7f1d94441700 1 --2- 172.21.15.143:0/2469343759 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1d8c15ba30 0x7f1d8c0c8980 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:18.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.896+0000 7f1d94441700 1 --2- 172.21.15.143:0/2469343759 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1d8c0d03a0 0x7f1d8c0c7e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:18.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.896+0000 7f1d94441700 1 --2- 172.21.15.143:0/2469343759 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1d8c0d1c70 0x7f1d8c14cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:18.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.896+0000 7f1d94441700 1 -- 172.21.15.143:0/2469343759 >> 172.21.15.143:0/2469343759 conn(0x7f1d8c08b000 msgr2=0x7f1d8c0af5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:18.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.896+0000 7f1d94441700 1 -- 172.21.15.143:0/2469343759 shutdown_connections 2023-11-08T19:25:18.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:18.897+0000 7f1d94441700 1 -- 172.21.15.143:0/2469343759 wait complete. 2023-11-08T19:25:18.910 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T19:24:44.551530+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-08T19:24:48.256781+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-08T19:24:48.954990+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-08T19:25:18.911 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T19:25:19.034 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.033+0000 7f9d94be3700 1 Processor -- start 2023-11-08T19:25:19.035 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.033+0000 7f9d94be3700 1 -- start start 2023-11-08T19:25:19.037 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.034+0000 7f9d94be3700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f9d9006ac10 0x7f9d9006b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:19.038 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.034+0000 7f9d94be3700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f9d900b3870 0x7f9d900b3c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:19.038 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.034+0000 7f9d94be3700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d90162690 con 0x7f9d9015dd40 2023-11-08T19:25:19.038 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.034+0000 7f9d94be3700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d9015e120 con 0x7f9d900b3870 2023-11-08T19:25:19.038 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.034+0000 7f9d94be3700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d9006b5c0 con 0x7f9d9006ac10 2023-11-08T19:25:19.039 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.034+0000 7f9d8ed9d700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f9d900b3870 0x7f9d900b3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:19.039 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.034+0000 7f9d8ed9d700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f9d900b3870 0x7f9d900b3c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49510/0 (socket says 172.21.15.143:49510) 2023-11-08T19:25:19.039 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.034+0000 7f9d8ed9d700 1 -- 172.21.15.143:0/1143137772 learned_addr learned my addr 172.21.15.143:0/1143137772 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:19.039 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.034+0000 7f9d8dd9b700 1 --2- 172.21.15.143:0/1143137772 >> v2:172.21.15.174:3300/0 conn(0x7f9d9006ac10 0x7f9d9006b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:19.040 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.034+0000 7f9d8ed9d700 1 -- 172.21.15.143:0/1143137772 >> v2:172.21.15.174:3300/0 conn(0x7f9d9006ac10 msgr2=0x7f9d9006b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:19.040 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.034+0000 7f9d8ed9d700 1 --2- 172.21.15.143:0/1143137772 >> v2:172.21.15.174:3300/0 conn(0x7f9d9006ac10 0x7f9d9006b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:19.040 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.034+0000 7f9d8ed9d700 1 -- 172.21.15.143:0/1143137772 >> v1:172.21.15.174:6789/0 conn(0x7f9d9015dd40 legacy=0x7f9d900b21c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:19.040 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.035+0000 7f9d8ed9d700 1 -- 172.21.15.143:0/1143137772 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d901617a0 con 0x7f9d900b3870 2023-11-08T19:25:19.041 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.035+0000 7f9d8ed9d700 1 --2- 172.21.15.143:0/1143137772 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9d900b3870 0x7f9d900b3c50 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9d9006ee70 tx=0x7f9d8800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=797cf4b88f5aebb1 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:19.041 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.035+0000 7f9d8d59a700 1 -- 172.21.15.143:0/1143137772 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d8800e070 con 0x7f9d900b3870 2023-11-08T19:25:19.041 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.035+0000 7f9d8d59a700 1 -- 172.21.15.143:0/1143137772 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d88003a20 con 0x7f9d900b3870 2023-11-08T19:25:19.041 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.035+0000 7f9d8d59a700 1 -- 172.21.15.143:0/1143137772 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d88003dd0 con 0x7f9d900b3870 2023-11-08T19:25:19.042 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.035+0000 7f9d94be3700 1 -- 172.21.15.143:0/1143137772 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9d900b3870 msgr2=0x7f9d900b3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:19.042 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.035+0000 7f9d94be3700 1 --2- 172.21.15.143:0/1143137772 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9d900b3870 0x7f9d900b3c50 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9d9006ee70 tx=0x7f9d8800b330 comp rx=0 tx=0).stop 2023-11-08T19:25:19.042 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.035+0000 7f9d94be3700 1 -- 172.21.15.143:0/1143137772 shutdown_connections 2023-11-08T19:25:19.042 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.035+0000 7f9d94be3700 1 --2- 172.21.15.143:0/1143137772 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9d900b3870 0x7f9d900b3c50 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:19.042 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.035+0000 7f9d94be3700 1 --2- 172.21.15.143:0/1143137772 >> v2:172.21.15.174:3300/0 conn(0x7f9d9006ac10 0x7f9d9006b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:19.043 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.035+0000 7f9d94be3700 1 -- 172.21.15.143:0/1143137772 >> 172.21.15.143:0/1143137772 conn(0x7f9d9006dfe0 msgr2=0x7f9d9006e3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:19.043 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.035+0000 7f9d94be3700 1 -- 172.21.15.143:0/1143137772 shutdown_connections 2023-11-08T19:25:19.043 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.035+0000 7f9d94be3700 1 -- 172.21.15.143:0/1143137772 wait complete. 2023-11-08T19:25:19.046 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.037+0000 7f9d94be3700 1 Processor -- start 2023-11-08T19:25:19.046 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.037+0000 7f9d94be3700 1 -- start start 2023-11-08T19:25:19.046 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.037+0000 7f9d94be3700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9d9006ac10 0x7f9d9015bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:19.046 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.038+0000 7f9d94be3700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9d900b3870 0x7f9d900c8c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:19.047 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.038+0000 7f9d94be3700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9d9015dd40 0x7f9d900af0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:19.047 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.038+0000 7f9d94be3700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d900ad6a0 con 0x7f9d9015dd40 2023-11-08T19:25:19.047 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.038+0000 7f9d94be3700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d900aca20 con 0x7f9d900b3870 2023-11-08T19:25:19.047 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.038+0000 7f9d94be3700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d900acd00 con 0x7f9d9006ac10 2023-11-08T19:25:19.047 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.038+0000 7f9d8ed9d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9d9015dd40 0x7f9d900af0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:19.048 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.038+0000 7f9d8ed9d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9d9015dd40 0x7f9d900af0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49516/0 (socket says 172.21.15.143:49516) 2023-11-08T19:25:19.048 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.038+0000 7f9d8ed9d700 1 -- 172.21.15.143:0/2177895862 learned_addr learned my addr 172.21.15.143:0/2177895862 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:19.048 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.038+0000 7f9d8e59c700 1 --2- 172.21.15.143:0/2177895862 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9d9006ac10 0x7f9d9015bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:19.048 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.038+0000 7f9d8dd9b700 1 --2- 172.21.15.143:0/2177895862 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9d900b3870 0x7f9d900c8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:19.048 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.038+0000 7f9d8ed9d700 1 -- 172.21.15.143:0/2177895862 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9d9006ac10 msgr2=0x7f9d9015bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:19.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.038+0000 7f9d8ed9d700 1 --2- 172.21.15.143:0/2177895862 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9d9006ac10 0x7f9d9015bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:19.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.038+0000 7f9d8ed9d700 1 -- 172.21.15.143:0/2177895862 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9d900b3870 msgr2=0x7f9d900c8c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:19.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.038+0000 7f9d8ed9d700 1 --2- 172.21.15.143:0/2177895862 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9d900b3870 0x7f9d900c8c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:19.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.038+0000 7f9d8ed9d700 1 -- 172.21.15.143:0/2177895862 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d88009b30 con 0x7f9d9015dd40 2023-11-08T19:25:19.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.038+0000 7f9d8ed9d700 1 --2- 172.21.15.143:0/2177895862 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9d9015dd40 0x7f9d900af0f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9d90068920 tx=0x7f9d88004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:19.050 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.038+0000 7f9d7effd700 1 -- 172.21.15.143:0/2177895862 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d88004440 con 0x7f9d9015dd40 2023-11-08T19:25:19.050 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.038+0000 7f9d7effd700 1 -- 172.21.15.143:0/2177895862 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d880045c0 con 0x7f9d9015dd40 2023-11-08T19:25:19.050 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.038+0000 7f9d7effd700 1 -- 172.21.15.143:0/2177895862 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d88004970 con 0x7f9d9015dd40 2023-11-08T19:25:19.050 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.038+0000 7f9d94be3700 1 -- 172.21.15.143:0/2177895862 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d900acfe0 con 0x7f9d9015dd40 2023-11-08T19:25:19.050 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.038+0000 7f9d94be3700 1 -- 172.21.15.143:0/2177895862 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d9015c480 con 0x7f9d9015dd40 2023-11-08T19:25:19.051 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.039+0000 7f9d94be3700 1 -- 172.21.15.143:0/2177895862 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d9004eee0 con 0x7f9d9015dd40 2023-11-08T19:25:19.051 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.040+0000 7f9d7effd700 1 -- 172.21.15.143:0/2177895862 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9d88012430 con 0x7f9d9015dd40 2023-11-08T19:25:19.051 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.040+0000 7f9d7effd700 1 --2- 172.21.15.143:0/2177895862 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f9d7006af00 0x7f9d7006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:19.051 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.040+0000 7f9d7effd700 1 -- 172.21.15.143:0/2177895862 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9d8808a8b0 con 0x7f9d9015dd40 2023-11-08T19:25:19.052 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.040+0000 7f9d8e59c700 1 --2- 172.21.15.143:0/2177895862 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f9d7006af00 0x7f9d7006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:19.052 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.040+0000 7f9d8e59c700 1 --2- 172.21.15.143:0/2177895862 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f9d7006af00 0x7f9d7006d3c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9d900d07e0 tx=0x7f9d840090e0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:19.052 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.044+0000 7f9d7effd700 1 -- 172.21.15.143:0/2177895862 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9d88059480 con 0x7f9d9015dd40 2023-11-08T19:25:19.234 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.233+0000 7f9d94be3700 1 -- 172.21.15.143:0/2177895862 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9d900b66c0 con 0x7f9d9015dd40 2023-11-08T19:25:19.235 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.233+0000 7f9d7effd700 1 -- 172.21.15.143:0/2177895862 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9d880170a0 con 0x7f9d9015dd40 2023-11-08T19:25:19.235 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2023-11-08T19:25:19.237 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.235+0000 7f9d94be3700 1 -- 172.21.15.143:0/2177895862 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f9d7006af00 msgr2=0x7f9d7006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:19.237 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.235+0000 7f9d94be3700 1 --2- 172.21.15.143:0/2177895862 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f9d7006af00 0x7f9d7006d3c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9d900d07e0 tx=0x7f9d840090e0 comp rx=0 tx=0).stop 2023-11-08T19:25:19.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.235+0000 7f9d94be3700 1 -- 172.21.15.143:0/2177895862 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9d9015dd40 msgr2=0x7f9d900af0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:19.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.235+0000 7f9d94be3700 1 --2- 172.21.15.143:0/2177895862 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9d9015dd40 0x7f9d900af0f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9d90068920 tx=0x7f9d88004ad0 comp rx=0 tx=0).stop 2023-11-08T19:25:19.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.235+0000 7f9d94be3700 1 -- 172.21.15.143:0/2177895862 shutdown_connections 2023-11-08T19:25:19.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.235+0000 7f9d94be3700 1 --2- 172.21.15.143:0/2177895862 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f9d7006af00 0x7f9d7006d3c0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:19.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.235+0000 7f9d94be3700 1 --2- 172.21.15.143:0/2177895862 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9d9006ac10 0x7f9d9015bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:19.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.235+0000 7f9d94be3700 1 --2- 172.21.15.143:0/2177895862 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9d9015dd40 0x7f9d900af0f0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:19.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.235+0000 7f9d94be3700 1 --2- 172.21.15.143:0/2177895862 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9d900b3870 0x7f9d900c8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:19.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.236+0000 7f9d94be3700 1 -- 172.21.15.143:0/2177895862 >> 172.21.15.143:0/2177895862 conn(0x7f9d9006dfe0 msgr2=0x7f9d900ce570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:19.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.236+0000 7f9d94be3700 1 -- 172.21.15.143:0/2177895862 shutdown_connections 2023-11-08T19:25:19.240 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.236+0000 7f9d94be3700 1 -- 172.21.15.143:0/2177895862 wait complete. 2023-11-08T19:25:19.249 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-08T19:25:19.372 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.371+0000 7f9d1128d700 1 Processor -- start 2023-11-08T19:25:19.373 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.371+0000 7f9d1128d700 1 -- start start 2023-11-08T19:25:19.375 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.372+0000 7f9d1128d700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f9d0c15dca0 0x7f9d0c0b93f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:19.375 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.372+0000 7f9d1128d700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9d0c0b5ae0 0x7f9d0c0b7280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:19.376 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.372+0000 7f9d1128d700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d0c0b8f50 con 0x7f9d0c0b5630 2023-11-08T19:25:19.376 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.372+0000 7f9d1128d700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d0c0b3360 con 0x7f9d0c15dca0 2023-11-08T19:25:19.376 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.372+0000 7f9d1128d700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d0c0b7eb0 con 0x7f9d0c0b5ae0 2023-11-08T19:25:19.376 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.372+0000 7f9d0affd700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f9d0c15dca0 0x7f9d0c0b93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:19.377 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.372+0000 7f9d0affd700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f9d0c15dca0 0x7f9d0c0b93f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49520/0 (socket says 172.21.15.143:49520) 2023-11-08T19:25:19.377 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.372+0000 7f9d0affd700 1 -- 172.21.15.143:0/1464391918 learned_addr learned my addr 172.21.15.143:0/1464391918 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:19.377 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.372+0000 7f9d0b7fe700 1 --2- 172.21.15.143:0/1464391918 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9d0c0b5ae0 0x7f9d0c0b7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:19.377 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.372+0000 7f9d0affd700 1 -- 172.21.15.143:0/1464391918 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9d0c0b5ae0 msgr2=0x7f9d0c0b7280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:19.377 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.372+0000 7f9d0affd700 1 --2- 172.21.15.143:0/1464391918 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9d0c0b5ae0 0x7f9d0c0b7280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:19.377 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.372+0000 7f9d0affd700 1 -- 172.21.15.143:0/1464391918 >> v1:172.21.15.174:6789/0 conn(0x7f9d0c0b5630 legacy=0x7f9d0c0b6b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:19.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.372+0000 7f9d0affd700 1 -- 172.21.15.143:0/1464391918 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d0c0b8190 con 0x7f9d0c15dca0 2023-11-08T19:25:19.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.372+0000 7f9d0affd700 1 --2- 172.21.15.143:0/1464391918 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9d0c15dca0 0x7f9d0c0b93f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9d000098f0 tx=0x7f9d00009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea2d9270831fa85d server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:19.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.373+0000 7f9d09ffb700 1 -- 172.21.15.143:0/1464391918 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d00011830 con 0x7f9d0c15dca0 2023-11-08T19:25:19.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.373+0000 7f9d09ffb700 1 -- 172.21.15.143:0/1464391918 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d00011990 con 0x7f9d0c15dca0 2023-11-08T19:25:19.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.373+0000 7f9d09ffb700 1 -- 172.21.15.143:0/1464391918 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d00011d20 con 0x7f9d0c15dca0 2023-11-08T19:25:19.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.373+0000 7f9d1128d700 1 -- 172.21.15.143:0/1464391918 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9d0c15dca0 msgr2=0x7f9d0c0b93f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:19.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.373+0000 7f9d1128d700 1 --2- 172.21.15.143:0/1464391918 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9d0c15dca0 0x7f9d0c0b93f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9d000098f0 tx=0x7f9d00009c00 comp rx=0 tx=0).stop 2023-11-08T19:25:19.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.373+0000 7f9d1128d700 1 -- 172.21.15.143:0/1464391918 shutdown_connections 2023-11-08T19:25:19.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.373+0000 7f9d1128d700 1 --2- 172.21.15.143:0/1464391918 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9d0c0b5ae0 0x7f9d0c0b7280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:19.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.373+0000 7f9d1128d700 1 --2- 172.21.15.143:0/1464391918 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9d0c15dca0 0x7f9d0c0b93f0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:19.380 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.373+0000 7f9d1128d700 1 -- 172.21.15.143:0/1464391918 >> 172.21.15.143:0/1464391918 conn(0x7f9d0c066d00 msgr2=0x7f9d0c067110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:19.380 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.373+0000 7f9d1128d700 1 -- 172.21.15.143:0/1464391918 shutdown_connections 2023-11-08T19:25:19.380 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.373+0000 7f9d1128d700 1 -- 172.21.15.143:0/1464391918 wait complete. 2023-11-08T19:25:19.383 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.375+0000 7f9d1128d700 1 Processor -- start 2023-11-08T19:25:19.383 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.375+0000 7f9d1128d700 1 -- start start 2023-11-08T19:25:19.383 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.376+0000 7f9d1128d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9d0c0b5630 0x7f9d0c1f4340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:19.383 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.376+0000 7f9d1128d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9d0c0b5ae0 0x7f9d0c0c7f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:19.383 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.376+0000 7f9d1128d700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9d0c15dca0 0x7f9d0c15ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:19.384 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.376+0000 7f9d1128d700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d0c0c9e40 con 0x7f9d0c0b5630 2023-11-08T19:25:19.384 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.376+0000 7f9d1128d700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d0c1f3a70 con 0x7f9d0c0b5ae0 2023-11-08T19:25:19.384 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.376+0000 7f9d1128d700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d0c0c91c0 con 0x7f9d0c15dca0 2023-11-08T19:25:19.384 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.376+0000 7f9d0affd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9d0c0b5630 0x7f9d0c1f4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:19.385 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.376+0000 7f9d0affd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9d0c0b5630 0x7f9d0c1f4340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49526/0 (socket says 172.21.15.143:49526) 2023-11-08T19:25:19.385 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.376+0000 7f9d0affd700 1 -- 172.21.15.143:0/1521565468 learned_addr learned my addr 172.21.15.143:0/1521565468 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:19.385 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.376+0000 7f9d0b7fe700 1 --2- 172.21.15.143:0/1521565468 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9d0c15dca0 0x7f9d0c15ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:19.385 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.376+0000 7f9d0a7fc700 1 --2- 172.21.15.143:0/1521565468 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9d0c0b5ae0 0x7f9d0c0c7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:19.386 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.376+0000 7f9d0affd700 1 -- 172.21.15.143:0/1521565468 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9d0c15dca0 msgr2=0x7f9d0c15ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:19.386 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.376+0000 7f9d0affd700 1 --2- 172.21.15.143:0/1521565468 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9d0c15dca0 0x7f9d0c15ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:19.386 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.376+0000 7f9d0affd700 1 -- 172.21.15.143:0/1521565468 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9d0c0b5ae0 msgr2=0x7f9d0c0c7f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:19.386 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.376+0000 7f9d0affd700 1 --2- 172.21.15.143:0/1521565468 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9d0c0b5ae0 0x7f9d0c0c7f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:19.386 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.376+0000 7f9d0affd700 1 -- 172.21.15.143:0/1521565468 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d00009550 con 0x7f9d0c0b5630 2023-11-08T19:25:19.387 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.376+0000 7f9d0b7fe700 1 --2- 172.21.15.143:0/1521565468 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9d0c15dca0 0x7f9d0c15ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:25:19.387 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.376+0000 7f9d0affd700 1 --2- 172.21.15.143:0/1521565468 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9d0c0b5630 0x7f9d0c1f4340 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9d00009c40 tx=0x7f9d00004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:19.387 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.377+0000 7f9cf37fe700 1 -- 172.21.15.143:0/1521565468 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d000045d0 con 0x7f9d0c0b5630 2023-11-08T19:25:19.387 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.377+0000 7f9cf37fe700 1 -- 172.21.15.143:0/1521565468 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d0001e070 con 0x7f9d0c0b5630 2023-11-08T19:25:19.387 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.377+0000 7f9d1128d700 1 -- 172.21.15.143:0/1521565468 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d0c0c94a0 con 0x7f9d0c0b5630 2023-11-08T19:25:19.388 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.377+0000 7f9cf37fe700 1 -- 172.21.15.143:0/1521565468 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d000038e0 con 0x7f9d0c0b5630 2023-11-08T19:25:19.388 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.377+0000 7f9d1128d700 1 -- 172.21.15.143:0/1521565468 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d0c0c9a60 con 0x7f9d0c0b5630 2023-11-08T19:25:19.388 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.378+0000 7f9d1128d700 1 -- 172.21.15.143:0/1521565468 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d0c0c9f80 con 0x7f9d0c0b5630 2023-11-08T19:25:19.388 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.378+0000 7f9cf37fe700 1 -- 172.21.15.143:0/1521565468 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9d00003a40 con 0x7f9d0c0b5630 2023-11-08T19:25:19.388 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.378+0000 7f9cf37fe700 1 --2- 172.21.15.143:0/1521565468 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f9cf806af00 0x7f9cf806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:19.389 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.378+0000 7f9cf37fe700 1 -- 172.21.15.143:0/1521565468 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9d0008a500 con 0x7f9d0c0b5630 2023-11-08T19:25:19.389 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.379+0000 7f9d0a7fc700 1 --2- 172.21.15.143:0/1521565468 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f9cf806af00 0x7f9cf806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:19.389 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.379+0000 7f9d0a7fc700 1 --2- 172.21.15.143:0/1521565468 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f9cf806af00 0x7f9cf806d3c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c157050 tx=0x7f9cfc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:19.389 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.383+0000 7f9cf37fe700 1 -- 172.21.15.143:0/1521565468 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9d00055640 con 0x7f9d0c0b5630 2023-11-08T19:25:19.572 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.571+0000 7f9d1128d700 1 -- 172.21.15.143:0/1521565468 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9d0c04eee0 con 0x7f9d0c0b5630 2023-11-08T19:25:19.572 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.571+0000 7f9cf37fe700 1 -- 172.21.15.143:0/1521565468 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9d00058c60 con 0x7f9d0c0b5630 2023-11-08T19:25:19.573 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2023-11-08T19:25:19.575 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.573+0000 7f9d1128d700 1 -- 172.21.15.143:0/1521565468 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f9cf806af00 msgr2=0x7f9cf806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:19.575 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.573+0000 7f9d1128d700 1 --2- 172.21.15.143:0/1521565468 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f9cf806af00 0x7f9cf806d3c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c157050 tx=0x7f9cfc009040 comp rx=0 tx=0).stop 2023-11-08T19:25:19.575 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.573+0000 7f9d1128d700 1 -- 172.21.15.143:0/1521565468 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9d0c0b5630 msgr2=0x7f9d0c1f4340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:19.575 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.573+0000 7f9d1128d700 1 --2- 172.21.15.143:0/1521565468 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9d0c0b5630 0x7f9d0c1f4340 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9d00009c40 tx=0x7f9d00004750 comp rx=0 tx=0).stop 2023-11-08T19:25:19.575 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.573+0000 7f9d1128d700 1 -- 172.21.15.143:0/1521565468 shutdown_connections 2023-11-08T19:25:19.576 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.573+0000 7f9d1128d700 1 --2- 172.21.15.143:0/1521565468 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f9cf806af00 0x7f9cf806d3c0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:19.576 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.573+0000 7f9d1128d700 1 --2- 172.21.15.143:0/1521565468 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9d0c15dca0 0x7f9d0c15ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:19.576 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.573+0000 7f9d1128d700 1 --2- 172.21.15.143:0/1521565468 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9d0c0b5630 0x7f9d0c1f4340 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:19.576 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.573+0000 7f9d1128d700 1 --2- 172.21.15.143:0/1521565468 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9d0c0b5ae0 0x7f9d0c0c7f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:19.576 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.573+0000 7f9d1128d700 1 -- 172.21.15.143:0/1521565468 >> 172.21.15.143:0/1521565468 conn(0x7f9d0c066d00 msgr2=0x7f9d0c0b2a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:19.577 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.573+0000 7f9d1128d700 1 -- 172.21.15.143:0/1521565468 shutdown_connections 2023-11-08T19:25:19.577 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.573+0000 7f9d1128d700 1 -- 172.21.15.143:0/1521565468 wait complete. 2023-11-08T19:25:19.587 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-08T19:25:19.710 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.708+0000 7fa2e008c700 1 Processor -- start 2023-11-08T19:25:19.710 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.709+0000 7fa2e008c700 1 -- start start 2023-11-08T19:25:19.713 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.709+0000 7fa2e008c700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa2d80d1d80 0x7fa2d80b2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:19.713 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.709+0000 7fa2e008c700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2d80d0420 con 0x7fa2d80b1730 2023-11-08T19:25:19.713 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.709+0000 7fa2e008c700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2d80cf810 con 0x7fa2d815bb20 2023-11-08T19:25:19.713 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.709+0000 7fa2e008c700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2d80cf600 con 0x7fa2d80d1d80 2023-11-08T19:25:19.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.710+0000 7fa2dd627700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa2d80d1d80 0x7fa2d80b2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:19.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.710+0000 7fa2de629700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fa2d80b1730 0x7fa2d80cc4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:45692/0 (socket says 172.21.15.143:45692) 2023-11-08T19:25:19.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.710+0000 7fa2de629700 1 -- 172.21.15.143:0/3732550264 learned_addr learned my addr 172.21.15.143:0/3732550264 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:25:19.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.710+0000 7fa2dd627700 1 -- 172.21.15.143:0/3732550264 >> v1:172.21.15.174:6789/0 conn(0x7fa2d815bb20 legacy=0x7fa2d80d1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:19.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.710+0000 7fa2dd627700 1 -- 172.21.15.143:0/3732550264 >> v1:172.21.15.143:6789/0 conn(0x7fa2d80b1730 legacy=0x7fa2d80cc4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:19.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.710+0000 7fa2dd627700 1 -- 172.21.15.143:0/3732550264 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2d80cfaf0 con 0x7fa2d80d1d80 2023-11-08T19:25:19.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.710+0000 7fa2dd627700 1 --2- 172.21.15.143:0/3732550264 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa2d80d1d80 0x7fa2d80b2670 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa2d4009840 tx=0x7fa2d4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3019e77b28f18f31 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:19.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.710+0000 7fa2dce26700 1 -- 172.21.15.143:0/3732550264 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d4011830 con 0x7fa2d80d1d80 2023-11-08T19:25:19.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.710+0000 7fa2dce26700 1 -- 172.21.15.143:0/3732550264 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2d4011990 con 0x7fa2d80d1d80 2023-11-08T19:25:19.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.710+0000 7fa2dce26700 1 -- 172.21.15.143:0/3732550264 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d4011d20 con 0x7fa2d80d1d80 2023-11-08T19:25:19.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.711+0000 7fa2e008c700 1 -- 172.21.15.143:0/3732550264 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa2d80d1d80 msgr2=0x7fa2d80b2670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:19.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.711+0000 7fa2e008c700 1 --2- 172.21.15.143:0/3732550264 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa2d80d1d80 0x7fa2d80b2670 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa2d4009840 tx=0x7fa2d4009b50 comp rx=0 tx=0).stop 2023-11-08T19:25:19.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.711+0000 7fa2e008c700 1 -- 172.21.15.143:0/3732550264 shutdown_connections 2023-11-08T19:25:19.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.711+0000 7fa2e008c700 1 --2- 172.21.15.143:0/3732550264 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa2d80d1d80 0x7fa2d80b2670 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:19.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.711+0000 7fa2e008c700 1 -- 172.21.15.143:0/3732550264 >> 172.21.15.143:0/3732550264 conn(0x7fa2d808b000 msgr2=0x7fa2d809ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:19.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.711+0000 7fa2e008c700 1 -- 172.21.15.143:0/3732550264 shutdown_connections 2023-11-08T19:25:19.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.711+0000 7fa2e008c700 1 -- 172.21.15.143:0/3732550264 wait complete. 2023-11-08T19:25:19.721 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.712+0000 7fa2e008c700 1 Processor -- start 2023-11-08T19:25:19.721 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.712+0000 7fa2e008c700 1 -- start start 2023-11-08T19:25:19.721 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.713+0000 7fa2e008c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2d80b1730 0x7fa2d81e5800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:19.722 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.713+0000 7fa2e008c700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa2d80d1d80 0x7fa2d80b6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:19.722 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.713+0000 7fa2e008c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa2d815bb20 0x7fa2d806d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:19.722 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.713+0000 7fa2e008c700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d806dd80 con 0x7fa2d815bb20 2023-11-08T19:25:19.722 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.713+0000 7fa2e008c700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d81f3ca0 con 0x7fa2d80b1730 2023-11-08T19:25:19.722 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.713+0000 7fa2e008c700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d81f3f10 con 0x7fa2d80d1d80 2023-11-08T19:25:19.723 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.713+0000 7fa2de629700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa2d815bb20 0x7fa2d806d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:19.723 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.713+0000 7fa2de629700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa2d815bb20 0x7fa2d806d840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49544/0 (socket says 172.21.15.143:49544) 2023-11-08T19:25:19.723 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.713+0000 7fa2de629700 1 -- 172.21.15.143:0/1084042237 learned_addr learned my addr 172.21.15.143:0/1084042237 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:19.723 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.713+0000 7fa2dde28700 1 --2- 172.21.15.143:0/1084042237 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2d80b1730 0x7fa2d81e5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:19.724 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.713+0000 7fa2dd627700 1 --2- 172.21.15.143:0/1084042237 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa2d80d1d80 0x7fa2d80b6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:19.724 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.713+0000 7fa2de629700 1 -- 172.21.15.143:0/1084042237 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa2d80d1d80 msgr2=0x7fa2d80b6cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:19.724 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.713+0000 7fa2de629700 1 --2- 172.21.15.143:0/1084042237 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa2d80d1d80 0x7fa2d80b6cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:19.724 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.713+0000 7fa2de629700 1 -- 172.21.15.143:0/1084042237 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2d80b1730 msgr2=0x7fa2d81e5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:19.725 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.713+0000 7fa2de629700 1 --2- 172.21.15.143:0/1084042237 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2d80b1730 0x7fa2d81e5800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:19.725 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.713+0000 7fa2de629700 1 -- 172.21.15.143:0/1084042237 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2d40094a0 con 0x7fa2d815bb20 2023-11-08T19:25:19.725 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.713+0000 7fa2de629700 1 --2- 172.21.15.143:0/1084042237 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa2d815bb20 0x7fa2d806d840 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa2d000cbf0 tx=0x7fa2d000cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:19.725 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.713+0000 7fa2ce7fc700 1 -- 172.21.15.143:0/1084042237 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d000aa60 con 0x7fa2d815bb20 2023-11-08T19:25:19.726 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.713+0000 7fa2ce7fc700 1 -- 172.21.15.143:0/1084042237 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2d000abc0 con 0x7fa2d815bb20 2023-11-08T19:25:19.726 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.713+0000 7fa2ce7fc700 1 -- 172.21.15.143:0/1084042237 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d0012400 con 0x7fa2d815bb20 2023-11-08T19:25:19.726 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.714+0000 7fa2e008c700 1 -- 172.21.15.143:0/1084042237 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2d81f41f0 con 0x7fa2d815bb20 2023-11-08T19:25:19.726 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.714+0000 7fa2e008c700 1 -- 172.21.15.143:0/1084042237 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2d80b73d0 con 0x7fa2d815bb20 2023-11-08T19:25:19.727 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.714+0000 7fa2e008c700 1 -- 172.21.15.143:0/1084042237 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2d81f4710 con 0x7fa2d815bb20 2023-11-08T19:25:19.727 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.715+0000 7fa2ce7fc700 1 -- 172.21.15.143:0/1084042237 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa2d001b3b0 con 0x7fa2d815bb20 2023-11-08T19:25:19.727 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.715+0000 7fa2ce7fc700 1 --2- 172.21.15.143:0/1084042237 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa2bc06af00 0x7fa2bc06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:19.727 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.715+0000 7fa2ce7fc700 1 -- 172.21.15.143:0/1084042237 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa2d0089040 con 0x7fa2d815bb20 2023-11-08T19:25:19.728 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.715+0000 7fa2dde28700 1 --2- 172.21.15.143:0/1084042237 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa2bc06af00 0x7fa2bc06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:19.728 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.716+0000 7fa2dde28700 1 --2- 172.21.15.143:0/1084042237 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa2bc06af00 0x7fa2bc06d3c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa2c8000f60 tx=0x7fa2c8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:19.728 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.720+0000 7fa2ce7fc700 1 -- 172.21.15.143:0/1084042237 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa2d81f4710 con 0x7fa2d815bb20 2023-11-08T19:25:19.910 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.909+0000 7fa2e008c700 1 -- 172.21.15.143:0/1084042237 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa2d81f4710 con 0x7fa2d815bb20 2023-11-08T19:25:19.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.909+0000 7fa2ce7fc700 1 -- 172.21.15.143:0/1084042237 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa2d81f4710 con 0x7fa2d815bb20 2023-11-08T19:25:19.911 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 64 2023-11-08T19:25:19.913 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.911+0000 7fa2e008c700 1 -- 172.21.15.143:0/1084042237 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa2bc06af00 msgr2=0x7fa2bc06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:19.913 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.911+0000 7fa2e008c700 1 --2- 172.21.15.143:0/1084042237 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa2bc06af00 0x7fa2bc06d3c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa2c8000f60 tx=0x7fa2c8009040 comp rx=0 tx=0).stop 2023-11-08T19:25:19.913 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.911+0000 7fa2e008c700 1 -- 172.21.15.143:0/1084042237 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa2d815bb20 msgr2=0x7fa2d806d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:19.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.911+0000 7fa2e008c700 1 --2- 172.21.15.143:0/1084042237 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa2d815bb20 0x7fa2d806d840 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa2d000cbf0 tx=0x7fa2d000cf00 comp rx=0 tx=0).stop 2023-11-08T19:25:19.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.911+0000 7fa2e008c700 1 -- 172.21.15.143:0/1084042237 shutdown_connections 2023-11-08T19:25:19.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.911+0000 7fa2e008c700 1 --2- 172.21.15.143:0/1084042237 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa2bc06af00 0x7fa2bc06d3c0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:19.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.911+0000 7fa2e008c700 1 --2- 172.21.15.143:0/1084042237 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa2d815bb20 0x7fa2d806d840 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:19.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.911+0000 7fa2e008c700 1 --2- 172.21.15.143:0/1084042237 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2d80b1730 0x7fa2d81e5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:19.915 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.911+0000 7fa2e008c700 1 --2- 172.21.15.143:0/1084042237 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa2d80d1d80 0x7fa2d80b6cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:19.915 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.911+0000 7fa2e008c700 1 -- 172.21.15.143:0/1084042237 >> 172.21.15.143:0/1084042237 conn(0x7fa2d808b000 msgr2=0x7fa2d8070c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:19.915 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.911+0000 7fa2e008c700 1 -- 172.21.15.143:0/1084042237 shutdown_connections 2023-11-08T19:25:19.915 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:19.911+0000 7fa2e008c700 1 -- 172.21.15.143:0/1084042237 wait complete. 2023-11-08T19:25:19.925 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:25:20.045 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.043+0000 7fdd9ccec700 1 Processor -- start 2023-11-08T19:25:20.045 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.044+0000 7fdd9ccec700 1 -- start start 2023-11-08T19:25:20.048 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.044+0000 7fdd9ccec700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fdd9815dd00 0x7fdd980cca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:20.048 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.044+0000 7fdd9ccec700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fdd981618f0 0x7fdd9806ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:20.048 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.044+0000 7fdd9ccec700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd980ccfb0 con 0x7fdd9806b150 2023-11-08T19:25:20.048 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.044+0000 7fdd9ccec700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd9806b600 con 0x7fdd981618f0 2023-11-08T19:25:20.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.044+0000 7fdd9ccec700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd980cad50 con 0x7fdd9815dd00 2023-11-08T19:25:20.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.045+0000 7fdd95d9b700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fdd981618f0 0x7fdd9806ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:20.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.045+0000 7fdd95d9b700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fdd981618f0 0x7fdd9806ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49548/0 (socket says 172.21.15.143:49548) 2023-11-08T19:25:20.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.045+0000 7fdd95d9b700 1 -- 172.21.15.143:0/3427327949 learned_addr learned my addr 172.21.15.143:0/3427327949 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:20.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.045+0000 7fdd9659c700 1 --2- 172.21.15.143:0/3427327949 >> v2:172.21.15.174:3300/0 conn(0x7fdd9815dd00 0x7fdd980cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:20.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.045+0000 7fdd95d9b700 1 -- 172.21.15.143:0/3427327949 >> v2:172.21.15.174:3300/0 conn(0x7fdd9815dd00 msgr2=0x7fdd980cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:20.050 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.045+0000 7fdd95d9b700 1 --2- 172.21.15.143:0/3427327949 >> v2:172.21.15.174:3300/0 conn(0x7fdd9815dd00 0x7fdd980cca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:20.050 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.045+0000 7fdd95d9b700 1 -- 172.21.15.143:0/3427327949 >> v1:172.21.15.174:6789/0 conn(0x7fdd9806b150 legacy=0x7fdd980b2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:20.050 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.045+0000 7fdd95d9b700 1 -- 172.21.15.143:0/3427327949 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd98161df0 con 0x7fdd981618f0 2023-11-08T19:25:20.050 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.045+0000 7fdd95d9b700 1 --2- 172.21.15.143:0/3427327949 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd981618f0 0x7fdd9806ac10 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fdd98070a60 tx=0x7fdd80009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8584092228d95e53 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:20.051 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.045+0000 7fdd9559a700 1 -- 172.21.15.143:0/3427327949 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd80004230 con 0x7fdd981618f0 2023-11-08T19:25:20.051 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.045+0000 7fdd9559a700 1 -- 172.21.15.143:0/3427327949 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd80004390 con 0x7fdd981618f0 2023-11-08T19:25:20.051 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.045+0000 7fdd9559a700 1 -- 172.21.15.143:0/3427327949 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd800036a0 con 0x7fdd981618f0 2023-11-08T19:25:20.051 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.046+0000 7fdd9ccec700 1 -- 172.21.15.143:0/3427327949 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd981618f0 msgr2=0x7fdd9806ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:20.051 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.046+0000 7fdd9ccec700 1 --2- 172.21.15.143:0/3427327949 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd981618f0 0x7fdd9806ac10 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fdd98070a60 tx=0x7fdd80009bd0 comp rx=0 tx=0).stop 2023-11-08T19:25:20.051 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.046+0000 7fdd9ccec700 1 -- 172.21.15.143:0/3427327949 shutdown_connections 2023-11-08T19:25:20.052 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.046+0000 7fdd9ccec700 1 --2- 172.21.15.143:0/3427327949 >> v2:172.21.15.174:3300/0 conn(0x7fdd9815dd00 0x7fdd980cca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:20.052 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.046+0000 7fdd9ccec700 1 --2- 172.21.15.143:0/3427327949 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd981618f0 0x7fdd9806ac10 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:20.052 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.046+0000 7fdd9ccec700 1 -- 172.21.15.143:0/3427327949 >> 172.21.15.143:0/3427327949 conn(0x7fdd98099ab0 msgr2=0x7fdd980d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:20.052 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.046+0000 7fdd9ccec700 1 -- 172.21.15.143:0/3427327949 shutdown_connections 2023-11-08T19:25:20.052 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.046+0000 7fdd9ccec700 1 -- 172.21.15.143:0/3427327949 wait complete. 2023-11-08T19:25:20.055 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.047+0000 7fdd9ccec700 1 Processor -- start 2023-11-08T19:25:20.056 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.047+0000 7fdd9ccec700 1 -- start start 2023-11-08T19:25:20.056 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.048+0000 7fdd9ccec700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdd9806b150 0x7fdd980c62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:20.056 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.048+0000 7fdd9ccec700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdd9815dd00 0x7fdd981f4400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:20.056 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.048+0000 7fdd9ccec700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd981618f0 0x7fdd980b24d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:20.057 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.048+0000 7fdd9ccec700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd980b0de0 con 0x7fdd981618f0 2023-11-08T19:25:20.057 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.048+0000 7fdd9ccec700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd980b4490 con 0x7fdd9806b150 2023-11-08T19:25:20.057 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.048+0000 7fdd9ccec700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd981f4940 con 0x7fdd9815dd00 2023-11-08T19:25:20.057 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.048+0000 7fdd96d9d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd981618f0 0x7fdd980b24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:20.057 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.048+0000 7fdd96d9d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd981618f0 0x7fdd980b24d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49556/0 (socket says 172.21.15.143:49556) 2023-11-08T19:25:20.058 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.048+0000 7fdd96d9d700 1 -- 172.21.15.143:0/1847971666 learned_addr learned my addr 172.21.15.143:0/1847971666 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:20.058 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.048+0000 7fdd95d9b700 1 --2- 172.21.15.143:0/1847971666 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdd9815dd00 0x7fdd981f4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:20.058 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.048+0000 7fdd9659c700 1 --2- 172.21.15.143:0/1847971666 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdd9806b150 0x7fdd980c62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:20.058 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.048+0000 7fdd96d9d700 1 -- 172.21.15.143:0/1847971666 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdd9815dd00 msgr2=0x7fdd981f4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:20.058 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.048+0000 7fdd96d9d700 1 --2- 172.21.15.143:0/1847971666 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdd9815dd00 0x7fdd981f4400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:20.059 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.048+0000 7fdd96d9d700 1 -- 172.21.15.143:0/1847971666 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdd9806b150 msgr2=0x7fdd980c62d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:20.059 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.048+0000 7fdd96d9d700 1 --2- 172.21.15.143:0/1847971666 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdd9806b150 0x7fdd980c62d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:20.059 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.048+0000 7fdd96d9d700 1 -- 172.21.15.143:0/1847971666 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd80009550 con 0x7fdd981618f0 2023-11-08T19:25:20.059 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.048+0000 7fdd95d9b700 1 --2- 172.21.15.143:0/1847971666 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdd9815dd00 0x7fdd981f4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:25:20.060 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.048+0000 7fdd9659c700 1 --2- 172.21.15.143:0/1847971666 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdd9806b150 0x7fdd980c62d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:25:20.060 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.049+0000 7fdd96d9d700 1 --2- 172.21.15.143:0/1847971666 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd981618f0 0x7fdd980b24d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fdd9000b990 tx=0x7fdd9000bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:20.060 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.049+0000 7fdd8effd700 1 -- 172.21.15.143:0/1847971666 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd9000c9d0 con 0x7fdd981618f0 2023-11-08T19:25:20.060 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.049+0000 7fdd8effd700 1 -- 172.21.15.143:0/1847971666 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd9000cb30 con 0x7fdd981618f0 2023-11-08T19:25:20.060 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.049+0000 7fdd9ccec700 1 -- 172.21.15.143:0/1847971666 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd980c7010 con 0x7fdd981618f0 2023-11-08T19:25:20.061 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.049+0000 7fdd8effd700 1 -- 172.21.15.143:0/1847971666 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd90014440 con 0x7fdd981618f0 2023-11-08T19:25:20.061 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.049+0000 7fdd9ccec700 1 -- 172.21.15.143:0/1847971666 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd980b0470 con 0x7fdd981618f0 2023-11-08T19:25:20.061 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.050+0000 7fdd9ccec700 1 -- 172.21.15.143:0/1847971666 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd9804eee0 con 0x7fdd981618f0 2023-11-08T19:25:20.061 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.050+0000 7fdd8effd700 1 -- 172.21.15.143:0/1847971666 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fdd90006d50 con 0x7fdd981618f0 2023-11-08T19:25:20.062 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.050+0000 7fdd8effd700 1 --2- 172.21.15.143:0/1847971666 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fdd7806af00 0x7fdd7806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:20.062 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.051+0000 7fdd8effd700 1 -- 172.21.15.143:0/1847971666 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdd90089170 con 0x7fdd981618f0 2023-11-08T19:25:20.062 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.051+0000 7fdd9659c700 1 --2- 172.21.15.143:0/1847971666 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fdd7806af00 0x7fdd7806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:20.062 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.051+0000 7fdd9659c700 1 --2- 172.21.15.143:0/1847971666 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fdd7806af00 0x7fdd7806d3c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fdd88006fd0 tx=0x7fdd88008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:20.063 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.055+0000 7fdd8effd700 1 -- 172.21.15.143:0/1847971666 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdd900541a0 con 0x7fdd981618f0 2023-11-08T19:25:20.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.242+0000 7fdd9ccec700 1 -- 172.21.15.143:0/1847971666 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdd980b45d0 con 0x7fdd981618f0 2023-11-08T19:25:20.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.244+0000 7fdd8effd700 1 -- 172.21.15.143:0/1847971666 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16553 (secure 0 0 0) 0x7fdd900577c0 con 0x7fdd981618f0 2023-11-08T19:25:20.245 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:25:20.246 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":20,"fsid":"968483af-77fc-4068-ad7c-5c2a1a4613d7","created":"2023-11-08T19:24:32.826484+0000","modified":"2023-11-08T19:24:51.618072+0000","last_up_change":"2023-11-08T19:24:43.573880+0000","last_in_change":"2023-11-08T19:24:35.314525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:44.551530+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-08T19:24:48.256781+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-08T19:24:48.954990+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bae98ec1-d94e-403d-a220-a522a53ea6f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6809","nonce":1487830566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6811","nonce":1487830566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6815","nonce":1487830566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6813","nonce":1487830566}]},"public_addr":"172.21.15.143:6809/1487830566","cluster_addr":"172.21.15.143:6811/1487830566","heartbeat_back_addr":"172.21.15.143:6815/1487830566","heartbeat_front_addr":"172.21.15.143:6813/1487830566","state":["exists","up"]},{"osd":1,"uuid":"5cb6538a-1ccb-4322-8633-09f66af0b7e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6801","nonce":787341397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6803","nonce":787341397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6807","nonce":787341397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6805","nonce":787341397}]},"public_addr":"172.21.15.143:6801/787341397","cluster_addr":"172.21.15.143:6803/787341397","heartbeat_back_addr":"172.21.15.143:6807/787341397","heartbeat_front_addr":"172.21.15.143:6805/787341397","state":["exists","up"]},{"osd":2,"uuid":"cdbf522c-60ef-4d60-9653-8dcb529c101a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6822","nonce":2301325597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6826","nonce":2301325597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6831","nonce":2301325597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6829","nonce":2301325597}]},"public_addr":"172.21.15.143:6822/2301325597","cluster_addr":"172.21.15.143:6826/2301325597","heartbeat_back_addr":"172.21.15.143:6831/2301325597","heartbeat_front_addr":"172.21.15.143:6829/2301325597","state":["exists","up"]},{"osd":3,"uuid":"4e2783cf-ebb9-4dd8-8245-bc1d17ce4325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6817","nonce":2021504428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6819","nonce":2021504428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6827","nonce":2021504428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6823","nonce":2021504428}]},"public_addr":"172.21.15.143:6817/2021504428","cluster_addr":"172.21.15.143:6819/2021504428","heartbeat_back_addr":"172.21.15.143:6827/2021504428","heartbeat_front_addr":"172.21.15.143:6823/2021504428","state":["exists","up"]},{"osd":4,"uuid":"691256fa-9b77-49b9-a346-7ae1adf0ca80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6801","nonce":1993380595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6803","nonce":1993380595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6807","nonce":1993380595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6805","nonce":1993380595}]},"public_addr":"172.21.15.174:6801/1993380595","cluster_addr":"172.21.15.174:6803/1993380595","heartbeat_back_addr":"172.21.15.174:6807/1993380595","heartbeat_front_addr":"172.21.15.174:6805/1993380595","state":["exists","up"]},{"osd":5,"uuid":"756fc352-48c9-46ce-b52e-0541fe6f8525","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6809","nonce":1777923303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6811","nonce":1777923303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6815","nonce":1777923303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6813","nonce":1777923303}]},"public_addr":"172.21.15.174:6809/1777923303","cluster_addr":"172.21.15.174:6811/1777923303","heartbeat_back_addr":"172.21.15.174:6815/1777923303","heartbeat_front_addr":"172.21.15.174:6813/1777923303","state":["exists","up"]},{"osd":6,"uuid":"fab2ac2b-ed97-4cde-9825-b480eabf8b8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6825","nonce":2755130316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6827","nonce":2755130316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6831","nonce":2755130316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6829","nonce":2755130316}]},"public_addr":"172.21.15.174:6825/2755130316","cluster_addr":"172.21.15.174:6827/2755130316","heartbeat_back_addr":"172.21.15.174:6831/2755130316","heartbeat_front_addr":"172.21.15.174:6829/2755130316","state":["exists","up"]},{"osd":7,"uuid":"3de9b797-2ab6-4d41-a863-7e7d638fd703","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6817","nonce":2847141411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6819","nonce":2847141411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6823","nonce":2847141411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6821","nonce":2847141411}]},"public_addr":"172.21.15.174:6817/2847141411","cluster_addr":"172.21.15.174:6819/2847141411","heartbeat_back_addr":"172.21.15.174:6823/2847141411","heartbeat_front_addr":"172.21.15.174:6821/2847141411","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.477076+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.496890+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.539305+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.454673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.538881+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.521294+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.520712+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.540404+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:25:20.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.246+0000 7fdd9ccec700 1 -- 172.21.15.143:0/1847971666 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fdd7806af00 msgr2=0x7fdd7806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:20.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.246+0000 7fdd9ccec700 1 --2- 172.21.15.143:0/1847971666 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fdd7806af00 0x7fdd7806d3c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fdd88006fd0 tx=0x7fdd88008040 comp rx=0 tx=0).stop 2023-11-08T19:25:20.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.246+0000 7fdd9ccec700 1 -- 172.21.15.143:0/1847971666 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd981618f0 msgr2=0x7fdd980b24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:20.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.246+0000 7fdd9ccec700 1 --2- 172.21.15.143:0/1847971666 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd981618f0 0x7fdd980b24d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fdd9000b990 tx=0x7fdd9000bd50 comp rx=0 tx=0).stop 2023-11-08T19:25:20.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.246+0000 7fdd9ccec700 1 -- 172.21.15.143:0/1847971666 shutdown_connections 2023-11-08T19:25:20.249 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.246+0000 7fdd9ccec700 1 --2- 172.21.15.143:0/1847971666 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fdd7806af00 0x7fdd7806d3c0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:20.249 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.246+0000 7fdd9ccec700 1 --2- 172.21.15.143:0/1847971666 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd981618f0 0x7fdd980b24d0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:20.249 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.246+0000 7fdd9ccec700 1 --2- 172.21.15.143:0/1847971666 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdd9806b150 0x7fdd980c62d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:20.249 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.246+0000 7fdd9ccec700 1 --2- 172.21.15.143:0/1847971666 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdd9815dd00 0x7fdd981f4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:20.249 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.246+0000 7fdd9ccec700 1 -- 172.21.15.143:0/1847971666 >> 172.21.15.143:0/1847971666 conn(0x7fdd98099ab0 msgr2=0x7fdd9815d030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:20.250 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.246+0000 7fdd9ccec700 1 -- 172.21.15.143:0/1847971666 shutdown_connections 2023-11-08T19:25:20.250 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.246+0000 7fdd9ccec700 1 -- 172.21.15.143:0/1847971666 wait complete. 2023-11-08T19:25:20.260 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T19:24:44.551530+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-08T19:24:48.256781+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-08T19:24:48.954990+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-08T19:25:20.260 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T19:25:20.384 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.382+0000 7f61dd590700 1 Processor -- start 2023-11-08T19:25:20.384 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.382+0000 7f61dd590700 1 -- start start 2023-11-08T19:25:20.390 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.383+0000 7f61dd590700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f61d806c0c0 0x7f61d80ccfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:20.391 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.383+0000 7f61dd590700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f61d815c9b0 0x7f61d8071900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:20.391 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.383+0000 7f61dd590700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61d80ced30 con 0x7f61d815ce60 2023-11-08T19:25:20.391 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.383+0000 7f61dd590700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61d80cd4f0 con 0x7f61d806c0c0 2023-11-08T19:25:20.391 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.383+0000 7f61dd590700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61d80cdf60 con 0x7f61d815c9b0 2023-11-08T19:25:20.392 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.383+0000 7f61d6ffd700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f61d806c0c0 0x7f61d80ccfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:20.392 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.383+0000 7f61d77fe700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f61d815ce60 0x7f61d81522a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:45716/0 (socket says 172.21.15.143:45716) 2023-11-08T19:25:20.392 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.383+0000 7f61d77fe700 1 -- 172.21.15.143:0/1325581274 learned_addr learned my addr 172.21.15.143:0/1325581274 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:25:20.392 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.383+0000 7f61d67fc700 1 --2- 172.21.15.143:0/1325581274 >> v2:172.21.15.174:3300/0 conn(0x7f61d815c9b0 0x7f61d8071900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:20.392 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.384+0000 7f61d6ffd700 1 -- 172.21.15.143:0/1325581274 >> v2:172.21.15.174:3300/0 conn(0x7f61d815c9b0 msgr2=0x7f61d8071900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:20.393 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.384+0000 7f61d6ffd700 1 --2- 172.21.15.143:0/1325581274 >> v2:172.21.15.174:3300/0 conn(0x7f61d815c9b0 0x7f61d8071900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:20.393 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.384+0000 7f61d6ffd700 1 -- 172.21.15.143:0/1325581274 >> v1:172.21.15.143:6789/0 conn(0x7f61d815ce60 legacy=0x7f61d81522a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:20.393 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.384+0000 7f61d6ffd700 1 -- 172.21.15.143:0/1325581274 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61d80cea50 con 0x7f61d806c0c0 2023-11-08T19:25:20.393 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.384+0000 7f61d5ffb700 1 -- 172.21.15.143:0/1325581274 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2853433787 0 0) 0x7f61d80ced30 con 0x7f61d815ce60 2023-11-08T19:25:20.393 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.384+0000 7f61d6ffd700 1 --2- 172.21.15.143:0/1325581274 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61d806c0c0 0x7f61d80ccfb0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f61d8067070 tx=0x7f61cc009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a150d37cfd9d698 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:20.394 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.384+0000 7f61d5ffb700 1 -- 172.21.15.143:0/1325581274 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61cc011830 con 0x7f61d806c0c0 2023-11-08T19:25:20.394 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.384+0000 7f61d5ffb700 1 -- 172.21.15.143:0/1325581274 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61cc011990 con 0x7f61d806c0c0 2023-11-08T19:25:20.394 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.384+0000 7f61d5ffb700 1 -- 172.21.15.143:0/1325581274 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61cc011d20 con 0x7f61d806c0c0 2023-11-08T19:25:20.394 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.384+0000 7f61dd590700 1 -- 172.21.15.143:0/1325581274 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61d806c0c0 msgr2=0x7f61d80ccfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:20.394 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.384+0000 7f61dd590700 1 --2- 172.21.15.143:0/1325581274 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61d806c0c0 0x7f61d80ccfb0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f61d8067070 tx=0x7f61cc009b20 comp rx=0 tx=0).stop 2023-11-08T19:25:20.395 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.384+0000 7f61dd590700 1 -- 172.21.15.143:0/1325581274 shutdown_connections 2023-11-08T19:25:20.395 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.384+0000 7f61dd590700 1 --2- 172.21.15.143:0/1325581274 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61d806c0c0 0x7f61d80ccfb0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:20.395 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.384+0000 7f61dd590700 1 --2- 172.21.15.143:0/1325581274 >> v2:172.21.15.174:3300/0 conn(0x7f61d815c9b0 0x7f61d8071900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:20.395 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.384+0000 7f61dd590700 1 -- 172.21.15.143:0/1325581274 >> 172.21.15.143:0/1325581274 conn(0x7f61d80b1c70 msgr2=0x7f61d80b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:20.395 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.384+0000 7f61dd590700 1 -- 172.21.15.143:0/1325581274 shutdown_connections 2023-11-08T19:25:20.396 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.384+0000 7f61dd590700 1 -- 172.21.15.143:0/1325581274 wait complete. 2023-11-08T19:25:20.396 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.386+0000 7f61dd590700 1 Processor -- start 2023-11-08T19:25:20.396 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.386+0000 7f61dd590700 1 -- start start 2023-11-08T19:25:20.396 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.387+0000 7f61dd590700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f61d806c0c0 0x7f61d80c8720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:20.396 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.387+0000 7f61dd590700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61d815c9b0 0x7f61d80b5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:20.397 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.387+0000 7f61dd590700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f61d815ce60 0x7f61d81517a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:20.397 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.387+0000 7f61dd590700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61d80c9a80 con 0x7f61d815c9b0 2023-11-08T19:25:20.397 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.387+0000 7f61dd590700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61d80b5ec0 con 0x7f61d815ce60 2023-11-08T19:25:20.397 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.387+0000 7f61dd590700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61d80c8e00 con 0x7f61d806c0c0 2023-11-08T19:25:20.397 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.387+0000 7f61d67fc700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61d815c9b0 0x7f61d80b5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:20.397 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.387+0000 7f61d67fc700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61d815c9b0 0x7f61d80b5980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49568/0 (socket says 172.21.15.143:49568) 2023-11-08T19:25:20.398 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.387+0000 7f61d67fc700 1 -- 172.21.15.143:0/124549996 learned_addr learned my addr 172.21.15.143:0/124549996 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:20.398 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.387+0000 7f61d6ffd700 1 --2- 172.21.15.143:0/124549996 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f61d806c0c0 0x7f61d80c8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:20.398 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.387+0000 7f61d77fe700 1 --2- 172.21.15.143:0/124549996 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f61d815ce60 0x7f61d81517a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:20.398 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.387+0000 7f61d67fc700 1 -- 172.21.15.143:0/124549996 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f61d806c0c0 msgr2=0x7f61d80c8720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:20.399 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.387+0000 7f61d67fc700 1 --2- 172.21.15.143:0/124549996 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f61d806c0c0 0x7f61d80c8720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:20.399 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.387+0000 7f61d67fc700 1 -- 172.21.15.143:0/124549996 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f61d815ce60 msgr2=0x7f61d81517a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:20.399 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.387+0000 7f61d67fc700 1 --2- 172.21.15.143:0/124549996 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f61d815ce60 0x7f61d81517a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:20.399 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.387+0000 7f61d67fc700 1 -- 172.21.15.143:0/124549996 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61cc0094a0 con 0x7f61d815c9b0 2023-11-08T19:25:20.399 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.387+0000 7f61d6ffd700 1 --2- 172.21.15.143:0/124549996 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f61d806c0c0 0x7f61d80c8720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:25:20.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.387+0000 7f61d67fc700 1 --2- 172.21.15.143:0/124549996 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61d815c9b0 0x7f61d80b5980 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f61c800e9e0 tx=0x7f61c800eda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:20.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.387+0000 7f61bf7fe700 1 -- 172.21.15.143:0/124549996 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c8009710 con 0x7f61d815c9b0 2023-11-08T19:25:20.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.387+0000 7f61bf7fe700 1 -- 172.21.15.143:0/124549996 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61c800c850 con 0x7f61d815c9b0 2023-11-08T19:25:20.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.387+0000 7f61dd590700 1 -- 172.21.15.143:0/124549996 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61d80c90e0 con 0x7f61d815c9b0 2023-11-08T19:25:20.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.387+0000 7f61bf7fe700 1 -- 172.21.15.143:0/124549996 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c800cbc0 con 0x7f61d815c9b0 2023-11-08T19:25:20.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.387+0000 7f61dd590700 1 -- 172.21.15.143:0/124549996 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61d80c96a0 con 0x7f61d815c9b0 2023-11-08T19:25:20.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.388+0000 7f61dd590700 1 -- 172.21.15.143:0/124549996 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61d80c9bc0 con 0x7f61d815c9b0 2023-11-08T19:25:20.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.389+0000 7f61bf7fe700 1 -- 172.21.15.143:0/124549996 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f61c800cd20 con 0x7f61d815c9b0 2023-11-08T19:25:20.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.389+0000 7f61bf7fe700 1 --2- 172.21.15.143:0/124549996 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f61c406af00 0x7f61c406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:20.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.389+0000 7f61bf7fe700 1 -- 172.21.15.143:0/124549996 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f61c80897b0 con 0x7f61d815c9b0 2023-11-08T19:25:20.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.389+0000 7f61d6ffd700 1 --2- 172.21.15.143:0/124549996 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f61c406af00 0x7f61c406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:20.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.390+0000 7f61d6ffd700 1 --2- 172.21.15.143:0/124549996 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f61c406af00 0x7f61c406d3c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f61cc0090b0 tx=0x7f61cc008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:20.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.393+0000 7f61bf7fe700 1 -- 172.21.15.143:0/124549996 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f61d80c9bc0 con 0x7f61d815c9b0 2023-11-08T19:25:20.583 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.582+0000 7f61dd590700 1 -- 172.21.15.143:0/124549996 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f61d80c9bc0 con 0x7f61d815c9b0 2023-11-08T19:25:20.584 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.582+0000 7f61bf7fe700 1 -- 172.21.15.143:0/124549996 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f61c8021020 con 0x7f61d815c9b0 2023-11-08T19:25:20.584 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2023-11-08T19:25:20.586 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.585+0000 7f61dd590700 1 -- 172.21.15.143:0/124549996 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f61c406af00 msgr2=0x7f61c406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:20.586 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.585+0000 7f61dd590700 1 --2- 172.21.15.143:0/124549996 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f61c406af00 0x7f61c406d3c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f61cc0090b0 tx=0x7f61cc008030 comp rx=0 tx=0).stop 2023-11-08T19:25:20.587 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.585+0000 7f61dd590700 1 -- 172.21.15.143:0/124549996 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61d815c9b0 msgr2=0x7f61d80b5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:20.587 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.585+0000 7f61dd590700 1 --2- 172.21.15.143:0/124549996 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61d815c9b0 0x7f61d80b5980 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f61c800e9e0 tx=0x7f61c800eda0 comp rx=0 tx=0).stop 2023-11-08T19:25:20.587 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.585+0000 7f61dd590700 1 -- 172.21.15.143:0/124549996 shutdown_connections 2023-11-08T19:25:20.587 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.585+0000 7f61dd590700 1 --2- 172.21.15.143:0/124549996 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f61c406af00 0x7f61c406d3c0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:20.587 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.585+0000 7f61dd590700 1 --2- 172.21.15.143:0/124549996 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f61d806c0c0 0x7f61d80c8720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:20.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.585+0000 7f61dd590700 1 --2- 172.21.15.143:0/124549996 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f61d815ce60 0x7f61d81517a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:20.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.585+0000 7f61dd590700 1 --2- 172.21.15.143:0/124549996 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61d815c9b0 0x7f61d80b5980 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:20.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.585+0000 7f61dd590700 1 -- 172.21.15.143:0/124549996 >> 172.21.15.143:0/124549996 conn(0x7f61d80b1c70 msgr2=0x7f61d806b6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:20.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.585+0000 7f61dd590700 1 -- 172.21.15.143:0/124549996 shutdown_connections 2023-11-08T19:25:20.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.585+0000 7f61dd590700 1 -- 172.21.15.143:0/124549996 wait complete. 2023-11-08T19:25:20.598 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-08T19:25:20.728 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.723+0000 7f998b3cd700 1 Processor -- start 2023-11-08T19:25:20.729 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.723+0000 7f998b3cd700 1 -- start start 2023-11-08T19:25:20.729 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.724+0000 7f998b3cd700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f998406b320 0x7f99841655b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:20.729 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.724+0000 7f998b3cd700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f998406d510 con 0x7f998415a680 2023-11-08T19:25:20.729 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.724+0000 7f998b3cd700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f998406d7f0 con 0x7f998415aa60 2023-11-08T19:25:20.730 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.724+0000 7f998b3cd700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f998406dad0 con 0x7f998406b320 2023-11-08T19:25:20.730 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.724+0000 7f9989169700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f998415a680 0x7f998406cac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:45728/0 (socket says 172.21.15.143:45728) 2023-11-08T19:25:20.730 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.724+0000 7f9989169700 1 -- 172.21.15.143:0/248830364 learned_addr learned my addr 172.21.15.143:0/248830364 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:25:20.730 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.724+0000 7f998996a700 1 --2- 172.21.15.143:0/248830364 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f998406b320 0x7f99841655b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:20.730 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.724+0000 7f997bfff700 1 -- 172.21.15.143:0/248830364 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2995942535 0 0) 0x7f998406d510 con 0x7f998415a680 2023-11-08T19:25:20.731 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.724+0000 7f997bfff700 1 -- 172.21.15.143:0/248830364 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f99700035b0 con 0x7f998415a680 2023-11-08T19:25:20.731 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.725+0000 7f997bfff700 1 -- 172.21.15.143:0/248830364 <== mon.1 v1:172.21.15.174:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1075829583 0 0) 0x7f998406d7f0 con 0x7f998415aa60 2023-11-08T19:25:20.731 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.725+0000 7f997bfff700 1 -- 172.21.15.143:0/248830364 --> v1:172.21.15.174:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f998406d510 con 0x7f998415aa60 2023-11-08T19:25:20.731 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.725+0000 7f997bfff700 1 -- 172.21.15.143:0/248830364 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 802583885 0 0) 0x7f99700035b0 con 0x7f998415a680 2023-11-08T19:25:20.731 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.725+0000 7f997bfff700 1 -- 172.21.15.143:0/248830364 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f998406b320 msgr2=0x7f99841655b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:20.732 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.725+0000 7f997bfff700 1 --2- 172.21.15.143:0/248830364 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f998406b320 0x7f99841655b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:20.732 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.725+0000 7f997bfff700 1 -- 172.21.15.143:0/248830364 >> v1:172.21.15.174:6789/0 conn(0x7f998415aa60 legacy=0x7f998406ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:20.732 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.725+0000 7f997bfff700 1 -- 172.21.15.143:0/248830364 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99840cd480 con 0x7f998415a680 2023-11-08T19:25:20.732 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.725+0000 7f997bfff700 1 -- 172.21.15.143:0/248830364 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4147756303 0 0) 0x7f9974001670 con 0x7f998415a680 2023-11-08T19:25:20.733 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.725+0000 7f997bfff700 1 -- 172.21.15.143:0/248830364 >> v1:172.21.15.143:6789/0 conn(0x7f998415a680 legacy=0x7f998406cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:20.733 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.725+0000 7f997bfff700 1 --2- 172.21.15.143:0/248830364 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9970003bc0 0x7f9970003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:20.733 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.725+0000 7f997bfff700 1 --2- 172.21.15.143:0/248830364 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f998406b320 0x7f9970004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:20.733 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.725+0000 7f997bfff700 1 --2- 172.21.15.143:0/248830364 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9970004d90 0x7f9970005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:20.733 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.725+0000 7f997bfff700 1 -- 172.21.15.143:0/248830364 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f998406dad0 con 0x7f9970004d90 2023-11-08T19:25:20.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.725+0000 7f997bfff700 1 -- 172.21.15.143:0/248830364 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f997000baf0 con 0x7f9970003bc0 2023-11-08T19:25:20.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.725+0000 7f997bfff700 1 -- 172.21.15.143:0/248830364 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f997000bcb0 con 0x7f998406b320 2023-11-08T19:25:20.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.725+0000 7f998996a700 1 --2- 172.21.15.143:0/248830364 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9970004d90 0x7f9970005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:20.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.725+0000 7f9989169700 1 --2- 172.21.15.143:0/248830364 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9970003bc0 0x7f9970003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:20.735 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.725+0000 7f9988968700 1 --2- 172.21.15.143:0/248830364 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f998406b320 0x7f9970004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:20.735 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.725+0000 7f998996a700 1 -- 172.21.15.143:0/248830364 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f998406b320 msgr2=0x7f9970004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:20.735 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.725+0000 7f998996a700 1 --2- 172.21.15.143:0/248830364 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f998406b320 0x7f9970004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:20.735 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.725+0000 7f998996a700 1 -- 172.21.15.143:0/248830364 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9970003bc0 msgr2=0x7f9970003fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:20.735 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.725+0000 7f998996a700 1 --2- 172.21.15.143:0/248830364 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9970003bc0 0x7f9970003fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:20.736 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.725+0000 7f998996a700 1 -- 172.21.15.143:0/248830364 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99700035b0 con 0x7f9970004d90 2023-11-08T19:25:20.736 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.725+0000 7f998996a700 1 --2- 172.21.15.143:0/248830364 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9970004d90 0x7f9970005210 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f997c009f90 tx=0x7f9970005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce79e2a88e3b227d server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:20.736 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.725+0000 7f997bfff700 1 -- 172.21.15.143:0/248830364 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f997c00ec30 con 0x7f9970004d90 2023-11-08T19:25:20.736 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.725+0000 7f997bfff700 1 -- 172.21.15.143:0/248830364 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f997c00f040 con 0x7f9970004d90 2023-11-08T19:25:20.737 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.726+0000 7f997bfff700 1 -- 172.21.15.143:0/248830364 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f997c014480 con 0x7f9970004d90 2023-11-08T19:25:20.737 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.726+0000 7f998b3cd700 1 -- 172.21.15.143:0/248830364 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9970004d90 msgr2=0x7f9970005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:20.737 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.726+0000 7f998b3cd700 1 --2- 172.21.15.143:0/248830364 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9970004d90 0x7f9970005210 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f997c009f90 tx=0x7f9970005880 comp rx=0 tx=0).stop 2023-11-08T19:25:20.737 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.726+0000 7f998996a700 1 -- 172.21.15.143:0/248830364 reap_dead start 2023-11-08T19:25:20.737 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.726+0000 7f998b3cd700 1 -- 172.21.15.143:0/248830364 shutdown_connections 2023-11-08T19:25:20.738 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.726+0000 7f998b3cd700 1 -- 172.21.15.143:0/248830364 >> 172.21.15.143:0/248830364 conn(0x7f99840cf710 msgr2=0x7f99840cfb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:20.738 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.726+0000 7f998b3cd700 1 -- 172.21.15.143:0/248830364 shutdown_connections 2023-11-08T19:25:20.738 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.726+0000 7f998b3cd700 1 -- 172.21.15.143:0/248830364 wait complete. 2023-11-08T19:25:20.738 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.727+0000 7f998b3cd700 1 Processor -- start 2023-11-08T19:25:20.738 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.727+0000 7f998b3cd700 1 -- start start 2023-11-08T19:25:20.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.728+0000 7f998b3cd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f99840c8150 0x7f99840c33f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:20.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.728+0000 7f998b3cd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f99840c3930 0x7f9984151a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:20.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.728+0000 7f998b3cd700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f99840c8530 0x7f9984151f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:20.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.728+0000 7f998b3cd700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99841524c0 con 0x7f99840c8150 2023-11-08T19:25:20.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.728+0000 7f998b3cd700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99840c4970 con 0x7f99840c3930 2023-11-08T19:25:20.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.728+0000 7f998b3cd700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99840c4c50 con 0x7f99840c8530 2023-11-08T19:25:20.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.728+0000 7f9989169700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f99840c8150 0x7f99840c33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:20.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.728+0000 7f9989169700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f99840c8150 0x7f99840c33f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49586/0 (socket says 172.21.15.143:49586) 2023-11-08T19:25:20.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.728+0000 7f9989169700 1 -- 172.21.15.143:0/3926664114 learned_addr learned my addr 172.21.15.143:0/3926664114 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:20.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.728+0000 7f9988968700 1 --2- 172.21.15.143:0/3926664114 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f99840c3930 0x7f9984151a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:20.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.728+0000 7f998996a700 1 --2- 172.21.15.143:0/3926664114 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f99840c8530 0x7f9984151f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:20.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.728+0000 7f9989169700 1 -- 172.21.15.143:0/3926664114 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f99840c8530 msgr2=0x7f9984151f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:20.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.728+0000 7f9989169700 1 --2- 172.21.15.143:0/3926664114 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f99840c8530 0x7f9984151f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:20.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.728+0000 7f9989169700 1 -- 172.21.15.143:0/3926664114 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f99840c3930 msgr2=0x7f9984151a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:20.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.728+0000 7f9989169700 1 --2- 172.21.15.143:0/3926664114 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f99840c3930 0x7f9984151a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:20.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.728+0000 7f9989169700 1 -- 172.21.15.143:0/3926664114 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99840c3dd0 con 0x7f99840c8150 2023-11-08T19:25:20.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.729+0000 7f9989169700 1 --2- 172.21.15.143:0/3926664114 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f99840c8150 0x7f99840c33f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f9974009730 tx=0x7f9974009af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:20.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.729+0000 7f9979ffb700 1 -- 172.21.15.143:0/3926664114 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f997400f9c0 con 0x7f99840c8150 2023-11-08T19:25:20.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.729+0000 7f9979ffb700 1 -- 172.21.15.143:0/3926664114 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f997400fb40 con 0x7f99840c8150 2023-11-08T19:25:20.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.729+0000 7f998b3cd700 1 -- 172.21.15.143:0/3926664114 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99840c40b0 con 0x7f99840c8150 2023-11-08T19:25:20.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.729+0000 7f9979ffb700 1 -- 172.21.15.143:0/3926664114 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9974016430 con 0x7f99840c8150 2023-11-08T19:25:20.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.729+0000 7f998b3cd700 1 -- 172.21.15.143:0/3926664114 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99840c9330 con 0x7f99840c8150 2023-11-08T19:25:20.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.730+0000 7f998b3cd700 1 -- 172.21.15.143:0/3926664114 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99840c94f0 con 0x7f99840c8150 2023-11-08T19:25:20.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.730+0000 7f9979ffb700 1 -- 172.21.15.143:0/3926664114 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9974016590 con 0x7f99840c8150 2023-11-08T19:25:20.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.730+0000 7f9979ffb700 1 --2- 172.21.15.143:0/3926664114 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f996806af00 0x7f996806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:20.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.730+0000 7f9979ffb700 1 -- 172.21.15.143:0/3926664114 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f99740897e0 con 0x7f99840c8150 2023-11-08T19:25:20.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.731+0000 7f9988968700 1 --2- 172.21.15.143:0/3926664114 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f996806af00 0x7f996806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:20.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.731+0000 7f9988968700 1 --2- 172.21.15.143:0/3926664114 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f996806af00 0x7f996806d3c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f99840ae840 tx=0x7f9980009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:20.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.735+0000 7f9979ffb700 1 -- 172.21.15.143:0/3926664114 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9974054920 con 0x7f99840c8150 2023-11-08T19:25:20.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.920+0000 7f998b3cd700 1 -- 172.21.15.143:0/3926664114 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f998404eee0 con 0x7f99840c8150 2023-11-08T19:25:20.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.921+0000 7f9979ffb700 1 -- 172.21.15.143:0/3926664114 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9974057f40 con 0x7f99840c8150 2023-11-08T19:25:20.923 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2023-11-08T19:25:20.924 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.923+0000 7f998b3cd700 1 -- 172.21.15.143:0/3926664114 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f996806af00 msgr2=0x7f996806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:20.925 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.923+0000 7f998b3cd700 1 --2- 172.21.15.143:0/3926664114 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f996806af00 0x7f996806d3c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f99840ae840 tx=0x7f9980009040 comp rx=0 tx=0).stop 2023-11-08T19:25:20.925 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.923+0000 7f998b3cd700 1 -- 172.21.15.143:0/3926664114 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f99840c8150 msgr2=0x7f99840c33f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:20.925 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.923+0000 7f998b3cd700 1 --2- 172.21.15.143:0/3926664114 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f99840c8150 0x7f99840c33f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f9974009730 tx=0x7f9974009af0 comp rx=0 tx=0).stop 2023-11-08T19:25:20.925 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.923+0000 7f998b3cd700 1 -- 172.21.15.143:0/3926664114 shutdown_connections 2023-11-08T19:25:20.926 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.923+0000 7f998b3cd700 1 --2- 172.21.15.143:0/3926664114 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f996806af00 0x7f996806d3c0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:20.926 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.923+0000 7f998b3cd700 1 --2- 172.21.15.143:0/3926664114 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f99840c8530 0x7f9984151f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:20.926 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.923+0000 7f998b3cd700 1 --2- 172.21.15.143:0/3926664114 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f99840c8150 0x7f99840c33f0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:20.926 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.923+0000 7f998b3cd700 1 --2- 172.21.15.143:0/3926664114 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f99840c3930 0x7f9984151a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:20.926 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.923+0000 7f998b3cd700 1 -- 172.21.15.143:0/3926664114 >> 172.21.15.143:0/3926664114 conn(0x7f99840cf710 msgr2=0x7f998415c800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:20.927 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.923+0000 7f998b3cd700 1 -- 172.21.15.143:0/3926664114 shutdown_connections 2023-11-08T19:25:20.927 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:20.923+0000 7f998b3cd700 1 -- 172.21.15.143:0/3926664114 wait complete. 2023-11-08T19:25:20.937 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-08T19:25:21.061 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.059+0000 7fc01af24700 1 Processor -- start 2023-11-08T19:25:21.061 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.059+0000 7fc01af24700 1 -- start start 2023-11-08T19:25:21.063 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.060+0000 7fc01af24700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fc01406b240 0x7fc0140caff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:21.063 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.060+0000 7fc01af24700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc0140cb530 0x7fc0140b22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:21.063 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.060+0000 7fc01af24700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc014161890 con 0x7fc01415dba0 2023-11-08T19:25:21.064 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.060+0000 7fc01af24700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc01406cd20 con 0x7fc01406b240 2023-11-08T19:25:21.064 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.060+0000 7fc01af24700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc01406b740 con 0x7fc0140cb530 2023-11-08T19:25:21.064 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.060+0000 7fc018cc0700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fc01415dba0 0x7fc01406c610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:45748/0 (socket says 172.21.15.143:45748) 2023-11-08T19:25:21.064 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.060+0000 7fc018cc0700 1 -- 172.21.15.143:0/1774012229 learned_addr learned my addr 172.21.15.143:0/1774012229 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:25:21.065 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.060+0000 7fc013fff700 1 --2- 172.21.15.143:0/1774012229 >> v2:172.21.15.174:3300/0 conn(0x7fc01406b240 0x7fc0140caff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:21.065 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.060+0000 7fc0194c1700 1 --2- 172.21.15.143:0/1774012229 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc0140cb530 0x7fc0140b22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:21.065 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.060+0000 7fc0137fe700 1 -- 172.21.15.143:0/1774012229 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1319498056 0 0) 0x7fc014161890 con 0x7fc01415dba0 2023-11-08T19:25:21.065 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.061+0000 7fc0137fe700 1 -- 172.21.15.143:0/1774012229 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc0040035b0 con 0x7fc01415dba0 2023-11-08T19:25:21.065 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.061+0000 7fc013fff700 1 -- 172.21.15.143:0/1774012229 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc0140cb530 msgr2=0x7fc0140b22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:21.066 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.061+0000 7fc013fff700 1 --2- 172.21.15.143:0/1774012229 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc0140cb530 0x7fc0140b22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:21.066 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.061+0000 7fc013fff700 1 -- 172.21.15.143:0/1774012229 >> v1:172.21.15.143:6789/0 conn(0x7fc01415dba0 legacy=0x7fc01406c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:21.066 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.061+0000 7fc013fff700 1 -- 172.21.15.143:0/1774012229 --> v2:172.21.15.174:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc01406ba20 con 0x7fc01406b240 2023-11-08T19:25:21.066 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.061+0000 7fc013fff700 1 --2- 172.21.15.143:0/1774012229 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc01406b240 0x7fc0140caff0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc000009840 tx=0x7fc000009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7165a7f5cc572bac server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:21.066 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.061+0000 7fc0137fe700 1 -- 172.21.15.143:0/1774012229 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc000011830 con 0x7fc01406b240 2023-11-08T19:25:21.066 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.061+0000 7fc0137fe700 1 -- 172.21.15.143:0/1774012229 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc000011990 con 0x7fc01406b240 2023-11-08T19:25:21.067 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.061+0000 7fc0137fe700 1 -- 172.21.15.143:0/1774012229 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc000011d20 con 0x7fc01406b240 2023-11-08T19:25:21.067 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.062+0000 7fc01af24700 1 -- 172.21.15.143:0/1774012229 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc01406b240 msgr2=0x7fc0140caff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:21.067 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.062+0000 7fc01af24700 1 --2- 172.21.15.143:0/1774012229 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc01406b240 0x7fc0140caff0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc000009840 tx=0x7fc000009b50 comp rx=0 tx=0).stop 2023-11-08T19:25:21.067 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.062+0000 7fc01af24700 1 -- 172.21.15.143:0/1774012229 shutdown_connections 2023-11-08T19:25:21.067 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.062+0000 7fc01af24700 1 --2- 172.21.15.143:0/1774012229 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc0140cb530 0x7fc0140b22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:21.068 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.062+0000 7fc01af24700 1 --2- 172.21.15.143:0/1774012229 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc01406b240 0x7fc0140caff0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:21.068 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.062+0000 7fc01af24700 1 -- 172.21.15.143:0/1774012229 >> 172.21.15.143:0/1774012229 conn(0x7fc0140cd2e0 msgr2=0x7fc0140cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:21.068 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.062+0000 7fc01af24700 1 -- 172.21.15.143:0/1774012229 shutdown_connections 2023-11-08T19:25:21.068 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.062+0000 7fc01af24700 1 -- 172.21.15.143:0/1774012229 wait complete. 2023-11-08T19:25:21.071 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.063+0000 7fc01af24700 1 Processor -- start 2023-11-08T19:25:21.071 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.063+0000 7fc01af24700 1 -- start start 2023-11-08T19:25:21.071 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.064+0000 7fc01af24700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc01406b240 0x7fc0141f7730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:21.072 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.064+0000 7fc01af24700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc0140cb530 0x7fc01415ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:21.072 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.064+0000 7fc01af24700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc01415dba0 0x7fc0140b0880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:21.072 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.064+0000 7fc01af24700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0141f72a0 con 0x7fc01415dba0 2023-11-08T19:25:21.072 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.064+0000 7fc01af24700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0140b32a0 con 0x7fc0140cb530 2023-11-08T19:25:21.072 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.064+0000 7fc01af24700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0141f41e0 con 0x7fc01406b240 2023-11-08T19:25:21.073 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.064+0000 7fc0194c1700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc01415dba0 0x7fc0140b0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:21.073 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.064+0000 7fc0194c1700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc01415dba0 0x7fc0140b0880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49604/0 (socket says 172.21.15.143:49604) 2023-11-08T19:25:21.073 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.064+0000 7fc0194c1700 1 -- 172.21.15.143:0/2557314538 learned_addr learned my addr 172.21.15.143:0/2557314538 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:21.073 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.064+0000 7fc013fff700 1 --2- 172.21.15.143:0/2557314538 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc0140cb530 0x7fc01415ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:21.073 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.064+0000 7fc018cc0700 1 --2- 172.21.15.143:0/2557314538 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc01406b240 0x7fc0141f7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:21.074 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.064+0000 7fc0194c1700 1 -- 172.21.15.143:0/2557314538 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc01406b240 msgr2=0x7fc0141f7730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:21.074 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.064+0000 7fc0194c1700 1 --2- 172.21.15.143:0/2557314538 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc01406b240 0x7fc0141f7730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:21.074 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.064+0000 7fc0194c1700 1 -- 172.21.15.143:0/2557314538 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc0140cb530 msgr2=0x7fc01415ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:21.074 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.064+0000 7fc0194c1700 1 --2- 172.21.15.143:0/2557314538 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc0140cb530 0x7fc01415ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:21.074 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.064+0000 7fc0194c1700 1 -- 172.21.15.143:0/2557314538 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0000094a0 con 0x7fc01415dba0 2023-11-08T19:25:21.075 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.064+0000 7fc018cc0700 1 --2- 172.21.15.143:0/2557314538 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc01406b240 0x7fc0141f7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:25:21.075 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.064+0000 7fc0194c1700 1 --2- 172.21.15.143:0/2557314538 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc01415dba0 0x7fc0140b0880 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc00c00efc0 tx=0x7fc00c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:21.075 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.064+0000 7fc0117fa700 1 -- 172.21.15.143:0/2557314538 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc00c009ea0 con 0x7fc01415dba0 2023-11-08T19:25:21.075 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.064+0000 7fc0117fa700 1 -- 172.21.15.143:0/2557314538 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc00c00f040 con 0x7fc01415dba0 2023-11-08T19:25:21.075 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.064+0000 7fc01af24700 1 -- 172.21.15.143:0/2557314538 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0141f6c50 con 0x7fc01415dba0 2023-11-08T19:25:21.076 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.064+0000 7fc0117fa700 1 -- 172.21.15.143:0/2557314538 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc00c014480 con 0x7fc01415dba0 2023-11-08T19:25:21.076 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.064+0000 7fc01af24700 1 -- 172.21.15.143:0/2557314538 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0141f44c0 con 0x7fc01415dba0 2023-11-08T19:25:21.076 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.065+0000 7fc01af24700 1 -- 172.21.15.143:0/2557314538 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc01404eee0 con 0x7fc01415dba0 2023-11-08T19:25:21.076 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.066+0000 7fc0117fa700 1 -- 172.21.15.143:0/2557314538 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc00c0145e0 con 0x7fc01415dba0 2023-11-08T19:25:21.076 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.066+0000 7fc0117fa700 1 --2- 172.21.15.143:0/2557314538 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fbff806af00 0x7fbff806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:21.076 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.066+0000 7fc0117fa700 1 -- 172.21.15.143:0/2557314538 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc00c089ed0 con 0x7fc01415dba0 2023-11-08T19:25:21.077 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.066+0000 7fc018cc0700 1 --2- 172.21.15.143:0/2557314538 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fbff806af00 0x7fbff806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:21.077 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.067+0000 7fc018cc0700 1 --2- 172.21.15.143:0/2557314538 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fbff806af00 0x7fbff806d3c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fc0080011e0 tx=0x7fc008009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:21.077 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.071+0000 7fc0117fa700 1 -- 172.21.15.143:0/2557314538 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc00c054f80 con 0x7fc01415dba0 2023-11-08T19:25:21.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.258+0000 7fc01af24700 1 -- 172.21.15.143:0/2557314538 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc0140b67e0 con 0x7fc01415dba0 2023-11-08T19:25:21.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.258+0000 7fc0117fa700 1 -- 172.21.15.143:0/2557314538 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc00c0180a0 con 0x7fc01415dba0 2023-11-08T19:25:21.260 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 64 2023-11-08T19:25:21.261 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.260+0000 7fc01af24700 1 -- 172.21.15.143:0/2557314538 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fbff806af00 msgr2=0x7fbff806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:21.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.260+0000 7fc01af24700 1 --2- 172.21.15.143:0/2557314538 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fbff806af00 0x7fbff806d3c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fc0080011e0 tx=0x7fc008009040 comp rx=0 tx=0).stop 2023-11-08T19:25:21.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.260+0000 7fc01af24700 1 -- 172.21.15.143:0/2557314538 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc01415dba0 msgr2=0x7fc0140b0880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:21.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.260+0000 7fc01af24700 1 --2- 172.21.15.143:0/2557314538 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc01415dba0 0x7fc0140b0880 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc00c00efc0 tx=0x7fc00c00c3a0 comp rx=0 tx=0).stop 2023-11-08T19:25:21.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.260+0000 7fc01af24700 1 -- 172.21.15.143:0/2557314538 shutdown_connections 2023-11-08T19:25:21.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.260+0000 7fc01af24700 1 --2- 172.21.15.143:0/2557314538 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fbff806af00 0x7fbff806d3c0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:21.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.260+0000 7fc01af24700 1 --2- 172.21.15.143:0/2557314538 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc01406b240 0x7fc0141f7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:21.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.260+0000 7fc01af24700 1 --2- 172.21.15.143:0/2557314538 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc01415dba0 0x7fc0140b0880 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:21.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.260+0000 7fc01af24700 1 --2- 172.21.15.143:0/2557314538 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc0140cb530 0x7fc01415ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:21.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.260+0000 7fc01af24700 1 -- 172.21.15.143:0/2557314538 >> 172.21.15.143:0/2557314538 conn(0x7fc0140cd2e0 msgr2=0x7fc0140b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:21.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.260+0000 7fc01af24700 1 -- 172.21.15.143:0/2557314538 shutdown_connections 2023-11-08T19:25:21.264 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.260+0000 7fc01af24700 1 -- 172.21.15.143:0/2557314538 wait complete. 2023-11-08T19:25:21.274 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:25:21.394 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.393+0000 7f4892e11700 1 Processor -- start 2023-11-08T19:25:21.395 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.393+0000 7f4892e11700 1 -- start start 2023-11-08T19:25:21.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.394+0000 7f4892e11700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f488c1618f0 0x7f488c06ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:21.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.394+0000 7f4892e11700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f488c06b150 0x7f488c0cb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:21.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.394+0000 7f4892e11700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f488c161df0 con 0x7f488c15dd00 2023-11-08T19:25:21.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.394+0000 7f4892e11700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f488c06b600 con 0x7f488c06b150 2023-11-08T19:25:21.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.394+0000 7f4892e11700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f488c0cad50 con 0x7f488c1618f0 2023-11-08T19:25:21.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.394+0000 7f48913ae700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f488c06b150 0x7f488c0cb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:21.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.394+0000 7f4890bad700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f488c15dd00 0x7f488c0cca70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:45762/0 (socket says 172.21.15.143:45762) 2023-11-08T19:25:21.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.394+0000 7f4890bad700 1 -- 172.21.15.143:0/2167284788 learned_addr learned my addr 172.21.15.143:0/2167284788 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:25:21.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.394+0000 7f48913ae700 1 -- 172.21.15.143:0/2167284788 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f488c1618f0 msgr2=0x7f488c06ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:21.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.394+0000 7f48913ae700 1 --2- 172.21.15.143:0/2167284788 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f488c1618f0 0x7f488c06ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:21.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.394+0000 7f48913ae700 1 -- 172.21.15.143:0/2167284788 >> v1:172.21.15.143:6789/0 conn(0x7f488c15dd00 legacy=0x7f488c0cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:21.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.394+0000 7f48913ae700 1 -- 172.21.15.143:0/2167284788 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f488c1620d0 con 0x7f488c06b150 2023-11-08T19:25:21.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.394+0000 7f488b7fe700 1 -- 172.21.15.143:0/2167284788 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 987039311 0 0) 0x7f488c161df0 con 0x7f488c15dd00 2023-11-08T19:25:21.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.394+0000 7f48913ae700 1 --2- 172.21.15.143:0/2167284788 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f488c06b150 0x7f488c0cb1a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4884009d90 tx=0x7f488400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51943bbaef6b4f55 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:21.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.394+0000 7f488b7fe700 1 -- 172.21.15.143:0/2167284788 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f488400e070 con 0x7f488c06b150 2023-11-08T19:25:21.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.394+0000 7f488b7fe700 1 -- 172.21.15.143:0/2167284788 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f488400bbd0 con 0x7f488c06b150 2023-11-08T19:25:21.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.394+0000 7f488b7fe700 1 -- 172.21.15.143:0/2167284788 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4884012430 con 0x7f488c06b150 2023-11-08T19:25:21.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.395+0000 7f4892e11700 1 -- 172.21.15.143:0/2167284788 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f488c06b150 msgr2=0x7f488c0cb1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:21.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.395+0000 7f4892e11700 1 --2- 172.21.15.143:0/2167284788 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f488c06b150 0x7f488c0cb1a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4884009d90 tx=0x7f488400b330 comp rx=0 tx=0).stop 2023-11-08T19:25:21.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.395+0000 7f4892e11700 1 -- 172.21.15.143:0/2167284788 shutdown_connections 2023-11-08T19:25:21.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.395+0000 7f4892e11700 1 --2- 172.21.15.143:0/2167284788 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f488c06b150 0x7f488c0cb1a0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:21.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.395+0000 7f4892e11700 1 --2- 172.21.15.143:0/2167284788 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f488c1618f0 0x7f488c06ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:21.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.395+0000 7f4892e11700 1 -- 172.21.15.143:0/2167284788 >> 172.21.15.143:0/2167284788 conn(0x7f488c099ab0 msgr2=0x7f488c0d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:21.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.395+0000 7f4892e11700 1 -- 172.21.15.143:0/2167284788 shutdown_connections 2023-11-08T19:25:21.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.395+0000 7f4892e11700 1 -- 172.21.15.143:0/2167284788 wait complete. 2023-11-08T19:25:21.409 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.396+0000 7f4892e11700 1 Processor -- start 2023-11-08T19:25:21.409 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.396+0000 7f4892e11700 1 -- start start 2023-11-08T19:25:21.409 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.397+0000 7f4892e11700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f488c06b150 0x7f488c154de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:21.409 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.397+0000 7f4892e11700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f488c15dd00 0x7f488c0b0f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:21.410 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.397+0000 7f4892e11700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f488c1618f0 0x7f488c0b14d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:21.410 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.397+0000 7f4892e11700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f488c0b2750 con 0x7f488c06b150 2023-11-08T19:25:21.410 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.397+0000 7f4892e11700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f488c0b1af0 con 0x7f488c15dd00 2023-11-08T19:25:21.410 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.397+0000 7f4892e11700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f488c0b1db0 con 0x7f488c1618f0 2023-11-08T19:25:21.410 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.397+0000 7f4890bad700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f488c06b150 0x7f488c154de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:21.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.397+0000 7f4890bad700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f488c06b150 0x7f488c154de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49614/0 (socket says 172.21.15.143:49614) 2023-11-08T19:25:21.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.397+0000 7f4890bad700 1 -- 172.21.15.143:0/2702218012 learned_addr learned my addr 172.21.15.143:0/2702218012 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:21.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.397+0000 7f488bfff700 1 --2- 172.21.15.143:0/2702218012 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f488c15dd00 0x7f488c0b0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:21.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.397+0000 7f48913ae700 1 --2- 172.21.15.143:0/2702218012 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f488c1618f0 0x7f488c0b14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:21.412 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.397+0000 7f4890bad700 1 -- 172.21.15.143:0/2702218012 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f488c1618f0 msgr2=0x7f488c0b14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:21.412 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.397+0000 7f4890bad700 1 --2- 172.21.15.143:0/2702218012 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f488c1618f0 0x7f488c0b14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:21.412 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.397+0000 7f4890bad700 1 -- 172.21.15.143:0/2702218012 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f488c15dd00 msgr2=0x7f488c0b0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:21.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.397+0000 7f4890bad700 1 --2- 172.21.15.143:0/2702218012 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f488c15dd00 0x7f488c0b0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:21.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.397+0000 7f4890bad700 1 -- 172.21.15.143:0/2702218012 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4884009a70 con 0x7f488c06b150 2023-11-08T19:25:21.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.397+0000 7f488bfff700 1 --2- 172.21.15.143:0/2702218012 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f488c15dd00 0x7f488c0b0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:25:21.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.397+0000 7f48913ae700 1 --2- 172.21.15.143:0/2702218012 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f488c1618f0 0x7f488c0b14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:25:21.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.398+0000 7f4890bad700 1 --2- 172.21.15.143:0/2702218012 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f488c06b150 0x7f488c154de0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f48800010e0 tx=0x7f488000cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:21.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.398+0000 7f48897fa700 1 -- 172.21.15.143:0/2702218012 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48800074e0 con 0x7f488c06b150 2023-11-08T19:25:21.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.398+0000 7f48897fa700 1 -- 172.21.15.143:0/2702218012 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4880007640 con 0x7f488c06b150 2023-11-08T19:25:21.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.398+0000 7f4892e11700 1 -- 172.21.15.143:0/2702218012 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f488c0b2090 con 0x7f488c06b150 2023-11-08T19:25:21.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.398+0000 7f48897fa700 1 -- 172.21.15.143:0/2702218012 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4880005a70 con 0x7f488c06b150 2023-11-08T19:25:21.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.398+0000 7f4892e11700 1 -- 172.21.15.143:0/2702218012 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f488c0c6ab0 con 0x7f488c06b150 2023-11-08T19:25:21.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.399+0000 7f4892e11700 1 -- 172.21.15.143:0/2702218012 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f488c04eee0 con 0x7f488c06b150 2023-11-08T19:25:21.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.399+0000 7f48897fa700 1 -- 172.21.15.143:0/2702218012 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4880005bd0 con 0x7f488c06b150 2023-11-08T19:25:21.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.399+0000 7f48897fa700 1 --2- 172.21.15.143:0/2702218012 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f487006af00 0x7f487006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:21.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.399+0000 7f48897fa700 1 -- 172.21.15.143:0/2702218012 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4880088aa0 con 0x7f488c06b150 2023-11-08T19:25:21.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.400+0000 7f488bfff700 1 --2- 172.21.15.143:0/2702218012 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f487006af00 0x7f487006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:21.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.400+0000 7f488bfff700 1 --2- 172.21.15.143:0/2702218012 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f487006af00 0x7f487006d3c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f488c06f000 tx=0x7f4878008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:21.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.404+0000 7f48897fa700 1 -- 172.21.15.143:0/2702218012 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4880053be0 con 0x7f488c06b150 2023-11-08T19:25:21.598 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.597+0000 7f4892e11700 1 -- 172.21.15.143:0/2702218012 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f488c06b920 con 0x7f488c06b150 2023-11-08T19:25:21.599 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.598+0000 7f48897fa700 1 -- 172.21.15.143:0/2702218012 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16553 (secure 0 0 0) 0x7f4880017070 con 0x7f488c06b150 2023-11-08T19:25:21.599 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:25:21.600 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":20,"fsid":"968483af-77fc-4068-ad7c-5c2a1a4613d7","created":"2023-11-08T19:24:32.826484+0000","modified":"2023-11-08T19:24:51.618072+0000","last_up_change":"2023-11-08T19:24:43.573880+0000","last_in_change":"2023-11-08T19:24:35.314525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:44.551530+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-08T19:24:48.256781+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-08T19:24:48.954990+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bae98ec1-d94e-403d-a220-a522a53ea6f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6809","nonce":1487830566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6811","nonce":1487830566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6815","nonce":1487830566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6813","nonce":1487830566}]},"public_addr":"172.21.15.143:6809/1487830566","cluster_addr":"172.21.15.143:6811/1487830566","heartbeat_back_addr":"172.21.15.143:6815/1487830566","heartbeat_front_addr":"172.21.15.143:6813/1487830566","state":["exists","up"]},{"osd":1,"uuid":"5cb6538a-1ccb-4322-8633-09f66af0b7e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6801","nonce":787341397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6803","nonce":787341397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6807","nonce":787341397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6805","nonce":787341397}]},"public_addr":"172.21.15.143:6801/787341397","cluster_addr":"172.21.15.143:6803/787341397","heartbeat_back_addr":"172.21.15.143:6807/787341397","heartbeat_front_addr":"172.21.15.143:6805/787341397","state":["exists","up"]},{"osd":2,"uuid":"cdbf522c-60ef-4d60-9653-8dcb529c101a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6822","nonce":2301325597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6826","nonce":2301325597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6831","nonce":2301325597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6829","nonce":2301325597}]},"public_addr":"172.21.15.143:6822/2301325597","cluster_addr":"172.21.15.143:6826/2301325597","heartbeat_back_addr":"172.21.15.143:6831/2301325597","heartbeat_front_addr":"172.21.15.143:6829/2301325597","state":["exists","up"]},{"osd":3,"uuid":"4e2783cf-ebb9-4dd8-8245-bc1d17ce4325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6817","nonce":2021504428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6819","nonce":2021504428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6827","nonce":2021504428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6823","nonce":2021504428}]},"public_addr":"172.21.15.143:6817/2021504428","cluster_addr":"172.21.15.143:6819/2021504428","heartbeat_back_addr":"172.21.15.143:6827/2021504428","heartbeat_front_addr":"172.21.15.143:6823/2021504428","state":["exists","up"]},{"osd":4,"uuid":"691256fa-9b77-49b9-a346-7ae1adf0ca80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6801","nonce":1993380595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6803","nonce":1993380595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6807","nonce":1993380595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6805","nonce":1993380595}]},"public_addr":"172.21.15.174:6801/1993380595","cluster_addr":"172.21.15.174:6803/1993380595","heartbeat_back_addr":"172.21.15.174:6807/1993380595","heartbeat_front_addr":"172.21.15.174:6805/1993380595","state":["exists","up"]},{"osd":5,"uuid":"756fc352-48c9-46ce-b52e-0541fe6f8525","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6809","nonce":1777923303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6811","nonce":1777923303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6815","nonce":1777923303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6813","nonce":1777923303}]},"public_addr":"172.21.15.174:6809/1777923303","cluster_addr":"172.21.15.174:6811/1777923303","heartbeat_back_addr":"172.21.15.174:6815/1777923303","heartbeat_front_addr":"172.21.15.174:6813/1777923303","state":["exists","up"]},{"osd":6,"uuid":"fab2ac2b-ed97-4cde-9825-b480eabf8b8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6825","nonce":2755130316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6827","nonce":2755130316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6831","nonce":2755130316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6829","nonce":2755130316}]},"public_addr":"172.21.15.174:6825/2755130316","cluster_addr":"172.21.15.174:6827/2755130316","heartbeat_back_addr":"172.21.15.174:6831/2755130316","heartbeat_front_addr":"172.21.15.174:6829/2755130316","state":["exists","up"]},{"osd":7,"uuid":"3de9b797-2ab6-4d41-a863-7e7d638fd703","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6817","nonce":2847141411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6819","nonce":2847141411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6823","nonce":2847141411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6821","nonce":2847141411}]},"public_addr":"172.21.15.174:6817/2847141411","cluster_addr":"172.21.15.174:6819/2847141411","heartbeat_back_addr":"172.21.15.174:6823/2847141411","heartbeat_front_addr":"172.21.15.174:6821/2847141411","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.477076+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.496890+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.539305+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.454673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.538881+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.521294+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.520712+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.540404+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:25:21.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.600+0000 7f4892e11700 1 -- 172.21.15.143:0/2702218012 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f487006af00 msgr2=0x7f487006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:21.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.600+0000 7f4892e11700 1 --2- 172.21.15.143:0/2702218012 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f487006af00 0x7f487006d3c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f488c06f000 tx=0x7f4878008040 comp rx=0 tx=0).stop 2023-11-08T19:25:21.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.600+0000 7f4892e11700 1 -- 172.21.15.143:0/2702218012 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f488c06b150 msgr2=0x7f488c154de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:21.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.600+0000 7f4892e11700 1 --2- 172.21.15.143:0/2702218012 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f488c06b150 0x7f488c154de0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f48800010e0 tx=0x7f488000cae0 comp rx=0 tx=0).stop 2023-11-08T19:25:21.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.600+0000 7f4892e11700 1 -- 172.21.15.143:0/2702218012 shutdown_connections 2023-11-08T19:25:21.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.600+0000 7f4892e11700 1 --2- 172.21.15.143:0/2702218012 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f487006af00 0x7f487006d3c0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:21.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.600+0000 7f4892e11700 1 --2- 172.21.15.143:0/2702218012 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f488c1618f0 0x7f488c0b14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:21.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.600+0000 7f4892e11700 1 --2- 172.21.15.143:0/2702218012 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f488c06b150 0x7f488c154de0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:21.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.600+0000 7f4892e11700 1 --2- 172.21.15.143:0/2702218012 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f488c15dd00 0x7f488c0b0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:21.604 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.600+0000 7f4892e11700 1 -- 172.21.15.143:0/2702218012 >> 172.21.15.143:0/2702218012 conn(0x7f488c099ab0 msgr2=0x7f488c0d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:21.604 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.600+0000 7f4892e11700 1 -- 172.21.15.143:0/2702218012 shutdown_connections 2023-11-08T19:25:21.604 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.600+0000 7f4892e11700 1 -- 172.21.15.143:0/2702218012 wait complete. 2023-11-08T19:25:21.615 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T19:24:44.551530+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-08T19:24:48.256781+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-08T19:24:48.954990+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-08T19:25:21.615 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T19:25:21.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.738+0000 7fbec5d67700 1 Processor -- start 2023-11-08T19:25:21.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.738+0000 7fbec5d67700 1 -- start start 2023-11-08T19:25:21.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.739+0000 7fbec5d67700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fbec00cc8c0 0x7fbec00ccd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:21.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.739+0000 7fbec5d67700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbec0143e10 con 0x7fbec0143a30 2023-11-08T19:25:21.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.739+0000 7fbec5d67700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbec00c6460 con 0x7fbec00c5880 2023-11-08T19:25:21.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.739+0000 7fbec5d67700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbec015e110 con 0x7fbec00cc8c0 2023-11-08T19:25:21.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.739+0000 7fbebf7fe700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fbec0143a30 0x7fbec006bfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:45776/0 (socket says 172.21.15.143:45776) 2023-11-08T19:25:21.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.739+0000 7fbebf7fe700 1 -- 172.21.15.143:0/287332140 learned_addr learned my addr 172.21.15.143:0/287332140 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:25:21.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.739+0000 7fbebeffd700 1 --2- 172.21.15.143:0/287332140 >> v2:172.21.15.143:3300/0 conn(0x7fbec00cc8c0 0x7fbec00ccd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:21.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.740+0000 7fbebe7fc700 1 -- 172.21.15.143:0/287332140 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1159114958 0 0) 0x7fbec0143e10 con 0x7fbec0143a30 2023-11-08T19:25:21.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.740+0000 7fbebe7fc700 1 -- 172.21.15.143:0/287332140 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbea80046c0 con 0x7fbec0143a30 2023-11-08T19:25:21.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.740+0000 7fbebeffd700 1 -- 172.21.15.143:0/287332140 >> v1:172.21.15.174:6789/0 conn(0x7fbec00c5880 legacy=0x7fbec00c5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:21.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.740+0000 7fbebeffd700 1 -- 172.21.15.143:0/287332140 >> v1:172.21.15.143:6789/0 conn(0x7fbec0143a30 legacy=0x7fbec006bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:21.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.740+0000 7fbebeffd700 1 -- 172.21.15.143:0/287332140 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbec006b4f0 con 0x7fbec00cc8c0 2023-11-08T19:25:21.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.740+0000 7fbebeffd700 1 --2- 172.21.15.143:0/287332140 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbec00cc8c0 0x7fbec00ccd50 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fbeb0009840 tx=0x7fbeb0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9ad5cca429aeba0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:21.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.740+0000 7fbebe7fc700 1 -- 172.21.15.143:0/287332140 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeb0004230 con 0x7fbec00cc8c0 2023-11-08T19:25:21.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.740+0000 7fbebe7fc700 1 -- 172.21.15.143:0/287332140 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbeb0004390 con 0x7fbec00cc8c0 2023-11-08T19:25:21.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.740+0000 7fbebe7fc700 1 -- 172.21.15.143:0/287332140 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeb00036a0 con 0x7fbec00cc8c0 2023-11-08T19:25:21.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.741+0000 7fbec5d67700 1 -- 172.21.15.143:0/287332140 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbec00cc8c0 msgr2=0x7fbec00ccd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:21.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.741+0000 7fbec5d67700 1 --2- 172.21.15.143:0/287332140 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbec00cc8c0 0x7fbec00ccd50 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fbeb0009840 tx=0x7fbeb0009b50 comp rx=0 tx=0).stop 2023-11-08T19:25:21.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.741+0000 7fbec5d67700 1 -- 172.21.15.143:0/287332140 shutdown_connections 2023-11-08T19:25:21.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.741+0000 7fbec5d67700 1 --2- 172.21.15.143:0/287332140 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbec00cc8c0 0x7fbec00ccd50 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:21.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.741+0000 7fbec5d67700 1 -- 172.21.15.143:0/287332140 >> 172.21.15.143:0/287332140 conn(0x7fbec00b1c70 msgr2=0x7fbec00b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:21.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.741+0000 7fbec5d67700 1 -- 172.21.15.143:0/287332140 shutdown_connections 2023-11-08T19:25:21.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.741+0000 7fbec5d67700 1 -- 172.21.15.143:0/287332140 wait complete. 2023-11-08T19:25:21.750 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.742+0000 7fbec5d67700 1 Processor -- start 2023-11-08T19:25:21.751 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.742+0000 7fbec5d67700 1 -- start start 2023-11-08T19:25:21.751 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.743+0000 7fbec5d67700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbec00c5880 0x7fbec01e3c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:21.751 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.743+0000 7fbec5d67700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbec00cc8c0 0x7fbec01e5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:21.751 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.743+0000 7fbec5d67700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbec0143a30 0x7fbec015b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:21.752 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.743+0000 7fbec5d67700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbec01318c0 con 0x7fbec00c5880 2023-11-08T19:25:21.752 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.743+0000 7fbec5d67700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbec015c8d0 con 0x7fbec0143a30 2023-11-08T19:25:21.752 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.743+0000 7fbec5d67700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbec0131b10 con 0x7fbec00cc8c0 2023-11-08T19:25:21.752 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.743+0000 7fbebf7fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbec00c5880 0x7fbec01e3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:21.753 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.743+0000 7fbebf7fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbec00c5880 0x7fbec01e3c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49628/0 (socket says 172.21.15.143:49628) 2023-11-08T19:25:21.753 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.743+0000 7fbebf7fe700 1 -- 172.21.15.143:0/1429440659 learned_addr learned my addr 172.21.15.143:0/1429440659 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:21.753 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.743+0000 7fbebffff700 1 --2- 172.21.15.143:0/1429440659 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbec0143a30 0x7fbec015b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:21.753 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.743+0000 7fbebeffd700 1 --2- 172.21.15.143:0/1429440659 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbec00cc8c0 0x7fbec01e5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:21.753 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.743+0000 7fbebf7fe700 1 -- 172.21.15.143:0/1429440659 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbec00cc8c0 msgr2=0x7fbec01e5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:21.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.743+0000 7fbebf7fe700 1 --2- 172.21.15.143:0/1429440659 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbec00cc8c0 0x7fbec01e5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:21.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.743+0000 7fbebf7fe700 1 -- 172.21.15.143:0/1429440659 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbec0143a30 msgr2=0x7fbec015b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:21.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.743+0000 7fbebf7fe700 1 --2- 172.21.15.143:0/1429440659 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbec0143a30 0x7fbec015b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:21.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.743+0000 7fbebf7fe700 1 -- 172.21.15.143:0/1429440659 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbeb00094a0 con 0x7fbec00c5880 2023-11-08T19:25:21.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.743+0000 7fbebf7fe700 1 --2- 172.21.15.143:0/1429440659 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbec00c5880 0x7fbec01e3c10 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fbeb400c500 tx=0x7fbeb400c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:21.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.743+0000 7fbea7fff700 1 -- 172.21.15.143:0/1429440659 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeb4005d70 con 0x7fbec00c5880 2023-11-08T19:25:21.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.743+0000 7fbea7fff700 1 -- 172.21.15.143:0/1429440659 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbeb40070d0 con 0x7fbec00c5880 2023-11-08T19:25:21.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.743+0000 7fbec5d67700 1 -- 172.21.15.143:0/1429440659 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbec0131df0 con 0x7fbec00c5880 2023-11-08T19:25:21.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.743+0000 7fbea7fff700 1 -- 172.21.15.143:0/1429440659 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeb4007440 con 0x7fbec00c5880 2023-11-08T19:25:21.756 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.744+0000 7fbec5d67700 1 -- 172.21.15.143:0/1429440659 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbec01323b0 con 0x7fbec00c5880 2023-11-08T19:25:21.756 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.744+0000 7fbec5d67700 1 -- 172.21.15.143:0/1429440659 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbec004eee0 con 0x7fbec00c5880 2023-11-08T19:25:21.756 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.745+0000 7fbea7fff700 1 -- 172.21.15.143:0/1429440659 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbeb40075a0 con 0x7fbec00c5880 2023-11-08T19:25:21.756 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.745+0000 7fbea7fff700 1 --2- 172.21.15.143:0/1429440659 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fbeac06af00 0x7fbeac06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:21.757 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.746+0000 7fbea7fff700 1 -- 172.21.15.143:0/1429440659 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbeb4088da0 con 0x7fbec00c5880 2023-11-08T19:25:21.757 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.746+0000 7fbebeffd700 1 --2- 172.21.15.143:0/1429440659 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fbeac06af00 0x7fbeac06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:21.757 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.746+0000 7fbebeffd700 1 --2- 172.21.15.143:0/1429440659 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fbeac06af00 0x7fbeac06d3c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fbeb0000c00 tx=0x7fbeb0010000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:21.757 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.749+0000 7fbea7fff700 1 -- 172.21.15.143:0/1429440659 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbeb4053fe0 con 0x7fbec00c5880 2023-11-08T19:25:21.939 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.937+0000 7fbec5d67700 1 -- 172.21.15.143:0/1429440659 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbec006bab0 con 0x7fbec00c5880 2023-11-08T19:25:21.939 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.937+0000 7fbea7fff700 1 -- 172.21.15.143:0/1429440659 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbeb40170a0 con 0x7fbec00c5880 2023-11-08T19:25:21.939 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2023-11-08T19:25:21.941 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.939+0000 7fbec5d67700 1 -- 172.21.15.143:0/1429440659 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fbeac06af00 msgr2=0x7fbeac06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:21.941 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.939+0000 7fbec5d67700 1 --2- 172.21.15.143:0/1429440659 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fbeac06af00 0x7fbeac06d3c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fbeb0000c00 tx=0x7fbeb0010000 comp rx=0 tx=0).stop 2023-11-08T19:25:21.941 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.939+0000 7fbec5d67700 1 -- 172.21.15.143:0/1429440659 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbec00c5880 msgr2=0x7fbec01e3c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:21.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.939+0000 7fbec5d67700 1 --2- 172.21.15.143:0/1429440659 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbec00c5880 0x7fbec01e3c10 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fbeb400c500 tx=0x7fbeb400c810 comp rx=0 tx=0).stop 2023-11-08T19:25:21.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.939+0000 7fbec5d67700 1 -- 172.21.15.143:0/1429440659 shutdown_connections 2023-11-08T19:25:21.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.939+0000 7fbec5d67700 1 --2- 172.21.15.143:0/1429440659 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fbeac06af00 0x7fbeac06d3c0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:21.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.939+0000 7fbec5d67700 1 --2- 172.21.15.143:0/1429440659 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbec0143a30 0x7fbec015b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:21.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.939+0000 7fbec5d67700 1 --2- 172.21.15.143:0/1429440659 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbec00c5880 0x7fbec01e3c10 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:21.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.939+0000 7fbec5d67700 1 --2- 172.21.15.143:0/1429440659 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbec00cc8c0 0x7fbec01e5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:21.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.939+0000 7fbec5d67700 1 -- 172.21.15.143:0/1429440659 >> 172.21.15.143:0/1429440659 conn(0x7fbec00b1c70 msgr2=0x7fbec00ce7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:21.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.939+0000 7fbec5d67700 1 -- 172.21.15.143:0/1429440659 shutdown_connections 2023-11-08T19:25:21.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:21.939+0000 7fbec5d67700 1 -- 172.21.15.143:0/1429440659 wait complete. 2023-11-08T19:25:21.953 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-08T19:25:22.076 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.075+0000 7fdbaa2a6700 1 Processor -- start 2023-11-08T19:25:22.077 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.075+0000 7fdbaa2a6700 1 -- start start 2023-11-08T19:25:22.079 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.076+0000 7fdbaa2a6700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fdba406b240 0x7fdba40caff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:22.079 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.076+0000 7fdbaa2a6700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fdba40cb530 0x7fdba40b22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:22.079 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.076+0000 7fdbaa2a6700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdba4161890 con 0x7fdba415dba0 2023-11-08T19:25:22.080 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.076+0000 7fdbaa2a6700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdba406cd20 con 0x7fdba40cb530 2023-11-08T19:25:22.080 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.076+0000 7fdbaa2a6700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdba406b740 con 0x7fdba406b240 2023-11-08T19:25:22.080 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.076+0000 7fdba8843700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fdba40cb530 0x7fdba40b22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:22.081 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.076+0000 7fdba3fff700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fdba415dba0 0x7fdba406c610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:45790/0 (socket says 172.21.15.143:45790) 2023-11-08T19:25:22.081 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.076+0000 7fdba3fff700 1 -- 172.21.15.143:0/2911423420 learned_addr learned my addr 172.21.15.143:0/2911423420 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:25:22.081 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.076+0000 7fdba8843700 1 -- 172.21.15.143:0/2911423420 >> v2:172.21.15.174:3300/0 conn(0x7fdba406b240 msgr2=0x7fdba40caff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:22.081 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.076+0000 7fdba8843700 1 --2- 172.21.15.143:0/2911423420 >> v2:172.21.15.174:3300/0 conn(0x7fdba406b240 0x7fdba40caff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:22.082 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.076+0000 7fdba8843700 1 -- 172.21.15.143:0/2911423420 >> v1:172.21.15.143:6789/0 conn(0x7fdba415dba0 legacy=0x7fdba406c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:22.082 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.076+0000 7fdba8843700 1 -- 172.21.15.143:0/2911423420 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdba406ba20 con 0x7fdba40cb530 2023-11-08T19:25:22.082 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.076+0000 7fdba8843700 1 --2- 172.21.15.143:0/2911423420 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdba40cb530 0x7fdba40b22e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fdba40d0810 tx=0x7fdb9c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8489d11255d4f8b2 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:22.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.076+0000 7fdba2ffd700 1 -- 172.21.15.143:0/2911423420 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb9c00e070 con 0x7fdba40cb530 2023-11-08T19:25:22.085 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.076+0000 7fdba2ffd700 1 -- 172.21.15.143:0/2911423420 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb9c00bbd0 con 0x7fdba40cb530 2023-11-08T19:25:22.086 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.076+0000 7fdba2ffd700 1 -- 172.21.15.143:0/2911423420 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb9c012430 con 0x7fdba40cb530 2023-11-08T19:25:22.086 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.077+0000 7fdbaa2a6700 1 -- 172.21.15.143:0/2911423420 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdba40cb530 msgr2=0x7fdba40b22e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:22.086 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.077+0000 7fdbaa2a6700 1 --2- 172.21.15.143:0/2911423420 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdba40cb530 0x7fdba40b22e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fdba40d0810 tx=0x7fdb9c00b330 comp rx=0 tx=0).stop 2023-11-08T19:25:22.087 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.077+0000 7fdbaa2a6700 1 -- 172.21.15.143:0/2911423420 shutdown_connections 2023-11-08T19:25:22.087 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.077+0000 7fdbaa2a6700 1 --2- 172.21.15.143:0/2911423420 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdba40cb530 0x7fdba40b22e0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:22.087 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.077+0000 7fdbaa2a6700 1 --2- 172.21.15.143:0/2911423420 >> v2:172.21.15.174:3300/0 conn(0x7fdba406b240 0x7fdba40caff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:22.087 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.077+0000 7fdbaa2a6700 1 -- 172.21.15.143:0/2911423420 >> 172.21.15.143:0/2911423420 conn(0x7fdba40cd2e0 msgr2=0x7fdba40cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:22.087 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.077+0000 7fdbaa2a6700 1 -- 172.21.15.143:0/2911423420 shutdown_connections 2023-11-08T19:25:22.088 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.077+0000 7fdbaa2a6700 1 -- 172.21.15.143:0/2911423420 wait complete. 2023-11-08T19:25:22.091 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.078+0000 7fdbaa2a6700 1 Processor -- start 2023-11-08T19:25:22.091 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.078+0000 7fdbaa2a6700 1 -- start start 2023-11-08T19:25:22.091 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.079+0000 7fdbaa2a6700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdba406b240 0x7fdba41f36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:22.092 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.079+0000 7fdbaa2a6700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdba40cb530 0x7fdba40b2350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:22.092 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.079+0000 7fdbaa2a6700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdba415dba0 0x7fdba40b2890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:22.092 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.079+0000 7fdbaa2a6700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdba40c7930 con 0x7fdba406b240 2023-11-08T19:25:22.093 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.079+0000 7fdbaa2a6700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdba40c5090 con 0x7fdba415dba0 2023-11-08T19:25:22.093 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.079+0000 7fdbaa2a6700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdba40c52e0 con 0x7fdba40cb530 2023-11-08T19:25:22.093 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.079+0000 7fdba3fff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdba406b240 0x7fdba41f36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:22.093 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.079+0000 7fdba3fff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdba406b240 0x7fdba41f36b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49642/0 (socket says 172.21.15.143:49642) 2023-11-08T19:25:22.094 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.079+0000 7fdba3fff700 1 -- 172.21.15.143:0/3777879711 learned_addr learned my addr 172.21.15.143:0/3777879711 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:22.094 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.079+0000 7fdba37fe700 1 --2- 172.21.15.143:0/3777879711 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdba40cb530 0x7fdba40b2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:22.094 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.079+0000 7fdba8843700 1 --2- 172.21.15.143:0/3777879711 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdba415dba0 0x7fdba40b2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:22.094 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.080+0000 7fdba3fff700 1 -- 172.21.15.143:0/3777879711 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdba40cb530 msgr2=0x7fdba40b2350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:22.094 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.080+0000 7fdba3fff700 1 --2- 172.21.15.143:0/3777879711 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdba40cb530 0x7fdba40b2350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:22.095 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.080+0000 7fdba3fff700 1 -- 172.21.15.143:0/3777879711 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdba415dba0 msgr2=0x7fdba40b2890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:22.095 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.080+0000 7fdba3fff700 1 --2- 172.21.15.143:0/3777879711 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdba415dba0 0x7fdba40b2890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:22.095 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.080+0000 7fdba3fff700 1 -- 172.21.15.143:0/3777879711 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb9c009a70 con 0x7fdba406b240 2023-11-08T19:25:22.095 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.080+0000 7fdba3fff700 1 --2- 172.21.15.143:0/3777879711 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdba406b240 0x7fdba41f36b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fdb9800c520 tx=0x7fdb9800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:22.095 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.080+0000 7fdba0ff9700 1 -- 172.21.15.143:0/3777879711 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb98005d70 con 0x7fdba406b240 2023-11-08T19:25:22.096 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.080+0000 7fdba0ff9700 1 -- 172.21.15.143:0/3777879711 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb980070d0 con 0x7fdba406b240 2023-11-08T19:25:22.096 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.080+0000 7fdbaa2a6700 1 -- 172.21.15.143:0/3777879711 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdba40c55c0 con 0x7fdba406b240 2023-11-08T19:25:22.096 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.080+0000 7fdba0ff9700 1 -- 172.21.15.143:0/3777879711 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb98007440 con 0x7fdba406b240 2023-11-08T19:25:22.096 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.080+0000 7fdbaa2a6700 1 -- 172.21.15.143:0/3777879711 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdba40c5b80 con 0x7fdba406b240 2023-11-08T19:25:22.097 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.081+0000 7fdbaa2a6700 1 -- 172.21.15.143:0/3777879711 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdba404eee0 con 0x7fdba406b240 2023-11-08T19:25:22.097 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.081+0000 7fdba0ff9700 1 -- 172.21.15.143:0/3777879711 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fdb980075a0 con 0x7fdba406b240 2023-11-08T19:25:22.097 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.082+0000 7fdba0ff9700 1 --2- 172.21.15.143:0/3777879711 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fdb9006af30 0x7fdb9006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:22.097 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.082+0000 7fdba0ff9700 1 -- 172.21.15.143:0/3777879711 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdb980888c0 con 0x7fdba406b240 2023-11-08T19:25:22.097 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.082+0000 7fdba37fe700 1 --2- 172.21.15.143:0/3777879711 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fdb9006af30 0x7fdb9006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:22.098 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.082+0000 7fdba37fe700 1 --2- 172.21.15.143:0/3777879711 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fdb9006af30 0x7fdb9006d3f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fdba40c5040 tx=0x7fdb94009110 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:22.098 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.086+0000 7fdba0ff9700 1 -- 172.21.15.143:0/3777879711 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdb98053990 con 0x7fdba406b240 2023-11-08T19:25:22.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.272+0000 7fdbaa2a6700 1 -- 172.21.15.143:0/3777879711 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdba40c7220 con 0x7fdba406b240 2023-11-08T19:25:22.274 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.272+0000 7fdba0ff9700 1 -- 172.21.15.143:0/3777879711 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7fdb980170a0 con 0x7fdba406b240 2023-11-08T19:25:22.274 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2023-11-08T19:25:22.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.274+0000 7fdbaa2a6700 1 -- 172.21.15.143:0/3777879711 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fdb9006af30 msgr2=0x7fdb9006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:22.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.274+0000 7fdbaa2a6700 1 --2- 172.21.15.143:0/3777879711 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fdb9006af30 0x7fdb9006d3f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fdba40c5040 tx=0x7fdb94009110 comp rx=0 tx=0).stop 2023-11-08T19:25:22.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.274+0000 7fdbaa2a6700 1 -- 172.21.15.143:0/3777879711 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdba406b240 msgr2=0x7fdba41f36b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:22.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.274+0000 7fdbaa2a6700 1 --2- 172.21.15.143:0/3777879711 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdba406b240 0x7fdba41f36b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fdb9800c520 tx=0x7fdb9800c830 comp rx=0 tx=0).stop 2023-11-08T19:25:22.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.274+0000 7fdbaa2a6700 1 -- 172.21.15.143:0/3777879711 shutdown_connections 2023-11-08T19:25:22.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.274+0000 7fdbaa2a6700 1 --2- 172.21.15.143:0/3777879711 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fdb9006af30 0x7fdb9006d3f0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:22.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.274+0000 7fdbaa2a6700 1 --2- 172.21.15.143:0/3777879711 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdba415dba0 0x7fdba40b2890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:22.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.274+0000 7fdbaa2a6700 1 --2- 172.21.15.143:0/3777879711 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdba406b240 0x7fdba41f36b0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:22.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.274+0000 7fdbaa2a6700 1 --2- 172.21.15.143:0/3777879711 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdba40cb530 0x7fdba40b2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:22.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.274+0000 7fdbaa2a6700 1 -- 172.21.15.143:0/3777879711 >> 172.21.15.143:0/3777879711 conn(0x7fdba40cd2e0 msgr2=0x7fdba40726f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:22.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.274+0000 7fdbaa2a6700 1 -- 172.21.15.143:0/3777879711 shutdown_connections 2023-11-08T19:25:22.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.274+0000 7fdbaa2a6700 1 -- 172.21.15.143:0/3777879711 wait complete. 2023-11-08T19:25:22.290 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-08T19:25:22.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.411+0000 7f6ff4e6e700 1 Processor -- start 2023-11-08T19:25:22.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.411+0000 7f6ff4e6e700 1 -- start start 2023-11-08T19:25:22.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.412+0000 7f6ff4e6e700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f6ff00d1d80 0x7f6ff00b2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:22.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.412+0000 7f6ff4e6e700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f6ff00b1730 0x7f6ff00cc4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:22.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.412+0000 7f6ff4e6e700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ff00d0420 con 0x7f6ff015bb20 2023-11-08T19:25:22.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.412+0000 7f6ff4e6e700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ff00cf600 con 0x7f6ff00b1730 2023-11-08T19:25:22.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.412+0000 7f6ff4e6e700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ff00cf810 con 0x7f6ff00d1d80 2023-11-08T19:25:22.417 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.413+0000 7f6feed9d700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f6ff00b1730 0x7f6ff00cc4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:22.417 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.413+0000 7f6feed9d700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f6ff00b1730 0x7f6ff00cc4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49654/0 (socket says 172.21.15.143:49654) 2023-11-08T19:25:22.417 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.413+0000 7f6feed9d700 1 -- 172.21.15.143:0/2846250584 learned_addr learned my addr 172.21.15.143:0/2846250584 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:22.417 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.413+0000 7f6fedd9b700 1 --2- 172.21.15.143:0/2846250584 >> v2:172.21.15.174:3300/0 conn(0x7f6ff00d1d80 0x7f6ff00b2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:22.418 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.413+0000 7f6feed9d700 1 -- 172.21.15.143:0/2846250584 >> v2:172.21.15.174:3300/0 conn(0x7f6ff00d1d80 msgr2=0x7f6ff00b2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:22.418 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.413+0000 7f6feed9d700 1 --2- 172.21.15.143:0/2846250584 >> v2:172.21.15.174:3300/0 conn(0x7f6ff00d1d80 0x7f6ff00b2670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:22.418 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.413+0000 7f6feed9d700 1 -- 172.21.15.143:0/2846250584 >> v1:172.21.15.174:6789/0 conn(0x7f6ff015bb20 legacy=0x7f6ff00d1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:22.418 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.413+0000 7f6feed9d700 1 -- 172.21.15.143:0/2846250584 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ff00cfaf0 con 0x7f6ff00b1730 2023-11-08T19:25:22.419 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.413+0000 7f6feed9d700 1 --2- 172.21.15.143:0/2846250584 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6ff00b1730 0x7f6ff00cc4d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6ff015f5f0 tx=0x7f6fe800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3be5c6f8653d6e12 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:22.419 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.413+0000 7f6fed59a700 1 -- 172.21.15.143:0/2846250584 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fe800e070 con 0x7f6ff00b1730 2023-11-08T19:25:22.419 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.413+0000 7f6fed59a700 1 -- 172.21.15.143:0/2846250584 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fe8003a20 con 0x7f6ff00b1730 2023-11-08T19:25:22.419 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.413+0000 7f6fed59a700 1 -- 172.21.15.143:0/2846250584 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fe8003dd0 con 0x7f6ff00b1730 2023-11-08T19:25:22.419 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.414+0000 7f6ff4e6e700 1 -- 172.21.15.143:0/2846250584 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6ff00b1730 msgr2=0x7f6ff00cc4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:22.420 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.414+0000 7f6ff4e6e700 1 --2- 172.21.15.143:0/2846250584 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6ff00b1730 0x7f6ff00cc4d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6ff015f5f0 tx=0x7f6fe800b330 comp rx=0 tx=0).stop 2023-11-08T19:25:22.420 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.414+0000 7f6ff4e6e700 1 -- 172.21.15.143:0/2846250584 shutdown_connections 2023-11-08T19:25:22.420 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.414+0000 7f6ff4e6e700 1 --2- 172.21.15.143:0/2846250584 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6ff00b1730 0x7f6ff00cc4d0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:22.420 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.414+0000 7f6ff4e6e700 1 --2- 172.21.15.143:0/2846250584 >> v2:172.21.15.174:3300/0 conn(0x7f6ff00d1d80 0x7f6ff00b2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:22.420 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.414+0000 7f6ff4e6e700 1 -- 172.21.15.143:0/2846250584 >> 172.21.15.143:0/2846250584 conn(0x7f6ff008b000 msgr2=0x7f6ff009ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:22.421 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.414+0000 7f6ff4e6e700 1 -- 172.21.15.143:0/2846250584 shutdown_connections 2023-11-08T19:25:22.421 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.414+0000 7f6ff4e6e700 1 -- 172.21.15.143:0/2846250584 wait complete. 2023-11-08T19:25:22.424 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.415+0000 7f6ff4e6e700 1 Processor -- start 2023-11-08T19:25:22.424 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.415+0000 7f6ff4e6e700 1 -- start start 2023-11-08T19:25:22.424 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.416+0000 7f6ff4e6e700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6ff00b1730 0x7f6ff00b50b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:22.424 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.416+0000 7f6ff4e6e700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6ff00d1d80 0x7f6ff00ce3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:22.424 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.416+0000 7f6ff4e6e700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6ff015bb20 0x7f6ff01519a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:22.425 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.416+0000 7f6ff4e6e700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ff00cd350 con 0x7f6ff015bb20 2023-11-08T19:25:22.425 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.416+0000 7f6ff4e6e700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ff00cd630 con 0x7f6ff00b1730 2023-11-08T19:25:22.425 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.416+0000 7f6ff4e6e700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ff00cd910 con 0x7f6ff00d1d80 2023-11-08T19:25:22.425 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.416+0000 7f6feed9d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6ff015bb20 0x7f6ff01519a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:22.425 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.416+0000 7f6feed9d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6ff015bb20 0x7f6ff01519a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49660/0 (socket says 172.21.15.143:49660) 2023-11-08T19:25:22.426 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.416+0000 7f6feed9d700 1 -- 172.21.15.143:0/2800974781 learned_addr learned my addr 172.21.15.143:0/2800974781 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:22.426 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.416+0000 7f6fee59c700 1 --2- 172.21.15.143:0/2800974781 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6ff00b1730 0x7f6ff00b50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:22.426 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.416+0000 7f6fedd9b700 1 --2- 172.21.15.143:0/2800974781 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6ff00d1d80 0x7f6ff00ce3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:22.426 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.416+0000 7f6feed9d700 1 -- 172.21.15.143:0/2800974781 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6ff00d1d80 msgr2=0x7f6ff00ce3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:22.426 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.416+0000 7f6feed9d700 1 --2- 172.21.15.143:0/2800974781 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6ff00d1d80 0x7f6ff00ce3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:22.427 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.416+0000 7f6feed9d700 1 -- 172.21.15.143:0/2800974781 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6ff00b1730 msgr2=0x7f6ff00b50b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:22.427 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.416+0000 7f6feed9d700 1 --2- 172.21.15.143:0/2800974781 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6ff00b1730 0x7f6ff00b50b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:22.427 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.416+0000 7f6feed9d700 1 -- 172.21.15.143:0/2800974781 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fe8009b30 con 0x7f6ff015bb20 2023-11-08T19:25:22.427 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.416+0000 7f6fee59c700 1 --2- 172.21.15.143:0/2800974781 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6ff00b1730 0x7f6ff00b50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:25:22.427 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.416+0000 7f6fedd9b700 1 --2- 172.21.15.143:0/2800974781 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6ff00d1d80 0x7f6ff00ce3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:25:22.428 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.416+0000 7f6feed9d700 1 --2- 172.21.15.143:0/2800974781 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6ff015bb20 0x7f6ff01519a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f6fe8015040 tx=0x7f6fe8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:22.428 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.416+0000 7f6fdeffd700 1 -- 172.21.15.143:0/2800974781 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fe8004440 con 0x7f6ff015bb20 2023-11-08T19:25:22.428 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.416+0000 7f6fdeffd700 1 -- 172.21.15.143:0/2800974781 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fe80045a0 con 0x7f6ff015bb20 2023-11-08T19:25:22.428 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.416+0000 7f6ff4e6e700 1 -- 172.21.15.143:0/2800974781 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ff00cdbf0 con 0x7f6ff015bb20 2023-11-08T19:25:22.428 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.417+0000 7f6fdeffd700 1 -- 172.21.15.143:0/2800974781 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fe8004950 con 0x7f6ff015bb20 2023-11-08T19:25:22.429 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.417+0000 7f6ff4e6e700 1 -- 172.21.15.143:0/2800974781 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ff00b55f0 con 0x7f6ff015bb20 2023-11-08T19:25:22.429 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.418+0000 7f6ff4e6e700 1 -- 172.21.15.143:0/2800974781 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ff00ce0b0 con 0x7f6ff015bb20 2023-11-08T19:25:22.429 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.418+0000 7f6fdeffd700 1 -- 172.21.15.143:0/2800974781 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6fe8012430 con 0x7f6ff015bb20 2023-11-08T19:25:22.429 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.418+0000 7f6fdeffd700 1 --2- 172.21.15.143:0/2800974781 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f6fd006af00 0x7f6fd006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:22.429 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.419+0000 7f6fdeffd700 1 -- 172.21.15.143:0/2800974781 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6fe808ac10 con 0x7f6ff015bb20 2023-11-08T19:25:22.430 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.419+0000 7f6fee59c700 1 --2- 172.21.15.143:0/2800974781 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f6fd006af00 0x7f6fd006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:22.430 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.419+0000 7f6fee59c700 1 --2- 172.21.15.143:0/2800974781 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f6fd006af00 0x7f6fd006d3c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f6fe4006fd0 tx=0x7f6fe4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:22.430 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.423+0000 7f6fdeffd700 1 -- 172.21.15.143:0/2800974781 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6fe8059760 con 0x7f6ff015bb20 2023-11-08T19:25:22.615 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.614+0000 7f6ff4e6e700 1 -- 172.21.15.143:0/2800974781 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6ff004eee0 con 0x7f6ff015bb20 2023-11-08T19:25:22.616 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.614+0000 7f6fdeffd700 1 -- 172.21.15.143:0/2800974781 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6fe80592f0 con 0x7f6ff015bb20 2023-11-08T19:25:22.616 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 64 2023-11-08T19:25:22.618 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.616+0000 7f6ff4e6e700 1 -- 172.21.15.143:0/2800974781 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f6fd006af00 msgr2=0x7f6fd006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:22.618 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.616+0000 7f6ff4e6e700 1 --2- 172.21.15.143:0/2800974781 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f6fd006af00 0x7f6fd006d3c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f6fe4006fd0 tx=0x7f6fe4008040 comp rx=0 tx=0).stop 2023-11-08T19:25:22.618 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.616+0000 7f6ff4e6e700 1 -- 172.21.15.143:0/2800974781 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6ff015bb20 msgr2=0x7f6ff01519a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:22.618 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.616+0000 7f6ff4e6e700 1 --2- 172.21.15.143:0/2800974781 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6ff015bb20 0x7f6ff01519a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f6fe8015040 tx=0x7f6fe8004ad0 comp rx=0 tx=0).stop 2023-11-08T19:25:22.619 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.616+0000 7f6ff4e6e700 1 -- 172.21.15.143:0/2800974781 shutdown_connections 2023-11-08T19:25:22.619 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.616+0000 7f6ff4e6e700 1 --2- 172.21.15.143:0/2800974781 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f6fd006af00 0x7f6fd006d3c0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:22.619 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.616+0000 7f6ff4e6e700 1 --2- 172.21.15.143:0/2800974781 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6ff015bb20 0x7f6ff01519a0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:22.619 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.616+0000 7f6ff4e6e700 1 --2- 172.21.15.143:0/2800974781 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6ff00b1730 0x7f6ff00b50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:22.619 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.616+0000 7f6ff4e6e700 1 --2- 172.21.15.143:0/2800974781 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6ff00d1d80 0x7f6ff00ce3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:22.620 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.616+0000 7f6ff4e6e700 1 -- 172.21.15.143:0/2800974781 >> 172.21.15.143:0/2800974781 conn(0x7f6ff008b000 msgr2=0x7f6ff0070c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:22.620 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.616+0000 7f6ff4e6e700 1 -- 172.21.15.143:0/2800974781 shutdown_connections 2023-11-08T19:25:22.620 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.616+0000 7f6ff4e6e700 1 -- 172.21.15.143:0/2800974781 wait complete. 2023-11-08T19:25:22.631 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:25:22.751 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.749+0000 7f61d8817700 1 Processor -- start 2023-11-08T19:25:22.751 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.749+0000 7f61d8817700 1 -- start start 2023-11-08T19:25:22.753 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.750+0000 7f61d8817700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f61d406b320 0x7f61d4168c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:22.753 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.750+0000 7f61d8817700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61d4161df0 con 0x7f61d415dd00 2023-11-08T19:25:22.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.750+0000 7f61d8817700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61d41620d0 con 0x7f61d41618f0 2023-11-08T19:25:22.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.750+0000 7f61d8817700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61d40cad50 con 0x7f61d406b320 2023-11-08T19:25:22.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.750+0000 7f61d2d9d700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f61d406b320 0x7f61d4168c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:22.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.750+0000 7f61d259c700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f61d415dd00 0x7f61d40cca70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:45818/0 (socket says 172.21.15.143:45818) 2023-11-08T19:25:22.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.750+0000 7f61d259c700 1 -- 172.21.15.143:0/1314175119 learned_addr learned my addr 172.21.15.143:0/1314175119 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:25:22.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.751+0000 7f61d2d9d700 1 -- 172.21.15.143:0/1314175119 >> v1:172.21.15.174:6789/0 conn(0x7f61d41618f0 legacy=0x7f61d406ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:22.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.751+0000 7f61d2d9d700 1 -- 172.21.15.143:0/1314175119 >> v1:172.21.15.143:6789/0 conn(0x7f61d415dd00 legacy=0x7f61d40cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:22.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.751+0000 7f61d2d9d700 1 -- 172.21.15.143:0/1314175119 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61d41623b0 con 0x7f61d406b320 2023-11-08T19:25:22.756 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.751+0000 7f61d159a700 1 -- 172.21.15.143:0/1314175119 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1484757106 0 0) 0x7f61d4161df0 con 0x7f61d415dd00 2023-11-08T19:25:22.756 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.751+0000 7f61d2d9d700 1 --2- 172.21.15.143:0/1314175119 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61d406b320 0x7f61d4168c60 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f61cc009d90 tx=0x7f61cc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc62e46a2f7ba6c7 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:22.756 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.751+0000 7f61d159a700 1 -- 172.21.15.143:0/1314175119 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61cc00e070 con 0x7f61d406b320 2023-11-08T19:25:22.756 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.751+0000 7f61d159a700 1 -- 172.21.15.143:0/1314175119 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61cc00bb90 con 0x7f61d406b320 2023-11-08T19:25:22.756 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.751+0000 7f61d159a700 1 -- 172.21.15.143:0/1314175119 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61cc003a20 con 0x7f61d406b320 2023-11-08T19:25:22.757 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.751+0000 7f61d8817700 1 -- 172.21.15.143:0/1314175119 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61d406b320 msgr2=0x7f61d4168c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:22.757 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.751+0000 7f61d8817700 1 --2- 172.21.15.143:0/1314175119 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61d406b320 0x7f61d4168c60 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f61cc009d90 tx=0x7f61cc00b330 comp rx=0 tx=0).stop 2023-11-08T19:25:22.757 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.751+0000 7f61d8817700 1 -- 172.21.15.143:0/1314175119 shutdown_connections 2023-11-08T19:25:22.757 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.752+0000 7f61d8817700 1 --2- 172.21.15.143:0/1314175119 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61d406b320 0x7f61d4168c60 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:22.757 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.752+0000 7f61d8817700 1 -- 172.21.15.143:0/1314175119 >> 172.21.15.143:0/1314175119 conn(0x7f61d4099ab0 msgr2=0x7f61d40d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:22.758 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.752+0000 7f61d8817700 1 -- 172.21.15.143:0/1314175119 shutdown_connections 2023-11-08T19:25:22.758 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.752+0000 7f61d8817700 1 -- 172.21.15.143:0/1314175119 wait complete. 2023-11-08T19:25:22.760 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.753+0000 7f61d8817700 1 Processor -- start 2023-11-08T19:25:22.761 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.753+0000 7f61d8817700 1 -- start start 2023-11-08T19:25:22.761 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.754+0000 7f61d8817700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f61d406b320 0x7f61d415e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:22.761 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.754+0000 7f61d8817700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f61d415dd00 0x7f61d4160c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:22.761 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.754+0000 7f61d8817700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61d41618f0 0x7f61d41f2f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:22.762 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.754+0000 7f61d8817700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61d40af6a0 con 0x7f61d41618f0 2023-11-08T19:25:22.762 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.754+0000 7f61d8817700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61d40aea90 con 0x7f61d415dd00 2023-11-08T19:25:22.762 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.754+0000 7f61d8817700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61d40aed70 con 0x7f61d406b320 2023-11-08T19:25:22.762 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.754+0000 7f61d2d9d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61d41618f0 0x7f61d41f2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:22.762 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.754+0000 7f61d2d9d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61d41618f0 0x7f61d41f2f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49674/0 (socket says 172.21.15.143:49674) 2023-11-08T19:25:22.763 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.754+0000 7f61d2d9d700 1 -- 172.21.15.143:0/3080918614 learned_addr learned my addr 172.21.15.143:0/3080918614 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:22.763 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.754+0000 7f61d1d9b700 1 --2- 172.21.15.143:0/3080918614 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f61d415dd00 0x7f61d4160c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:22.763 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.754+0000 7f61d259c700 1 --2- 172.21.15.143:0/3080918614 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f61d406b320 0x7f61d415e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:22.763 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.754+0000 7f61d2d9d700 1 -- 172.21.15.143:0/3080918614 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f61d406b320 msgr2=0x7f61d415e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:22.763 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.754+0000 7f61d2d9d700 1 --2- 172.21.15.143:0/3080918614 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f61d406b320 0x7f61d415e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:22.763 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.754+0000 7f61d2d9d700 1 -- 172.21.15.143:0/3080918614 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f61d415dd00 msgr2=0x7f61d4160c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:22.764 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.754+0000 7f61d2d9d700 1 --2- 172.21.15.143:0/3080918614 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f61d415dd00 0x7f61d4160c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:22.764 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.754+0000 7f61d2d9d700 1 -- 172.21.15.143:0/3080918614 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61cc009a70 con 0x7f61d41618f0 2023-11-08T19:25:22.764 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.754+0000 7f61d2d9d700 1 --2- 172.21.15.143:0/3080918614 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61d41618f0 0x7f61d41f2f10 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f61cc004000 tx=0x7f61cc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:22.764 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.754+0000 7f61c2ffd700 1 -- 172.21.15.143:0/3080918614 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61cc012630 con 0x7f61d41618f0 2023-11-08T19:25:22.764 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.754+0000 7f61c2ffd700 1 -- 172.21.15.143:0/3080918614 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61cc0127b0 con 0x7f61d41618f0 2023-11-08T19:25:22.765 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.754+0000 7f61d8817700 1 -- 172.21.15.143:0/3080918614 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61d415e260 con 0x7f61d41618f0 2023-11-08T19:25:22.765 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.754+0000 7f61c2ffd700 1 -- 172.21.15.143:0/3080918614 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61cc012b60 con 0x7f61d41618f0 2023-11-08T19:25:22.765 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.755+0000 7f61d8817700 1 -- 172.21.15.143:0/3080918614 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61d40af330 con 0x7f61d41618f0 2023-11-08T19:25:22.765 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.756+0000 7f61d8817700 1 -- 172.21.15.143:0/3080918614 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61d404eee0 con 0x7f61d41618f0 2023-11-08T19:25:22.765 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.756+0000 7f61c2ffd700 1 -- 172.21.15.143:0/3080918614 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f61cc012cc0 con 0x7f61d41618f0 2023-11-08T19:25:22.766 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.756+0000 7f61c2ffd700 1 --2- 172.21.15.143:0/3080918614 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f61b406af00 0x7f61b406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:22.766 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.756+0000 7f61c2ffd700 1 -- 172.21.15.143:0/3080918614 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f61cc08bb30 con 0x7f61d41618f0 2023-11-08T19:25:22.766 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.756+0000 7f61d259c700 1 --2- 172.21.15.143:0/3080918614 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f61b406af00 0x7f61b406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:22.766 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.757+0000 7f61d259c700 1 --2- 172.21.15.143:0/3080918614 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f61b406af00 0x7f61b406d3c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f61c80010e0 tx=0x7f61c8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:22.767 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.761+0000 7f61c2ffd700 1 -- 172.21.15.143:0/3080918614 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f61cc056c70 con 0x7f61d41618f0 2023-11-08T19:25:22.953 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.951+0000 7f61d8817700 1 -- 172.21.15.143:0/3080918614 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f61d40af7e0 con 0x7f61d41618f0 2023-11-08T19:25:22.954 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.953+0000 7f61c2ffd700 1 -- 172.21.15.143:0/3080918614 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16553 (secure 0 0 0) 0x7f61cc05a290 con 0x7f61d41618f0 2023-11-08T19:25:22.954 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:25:22.955 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":20,"fsid":"968483af-77fc-4068-ad7c-5c2a1a4613d7","created":"2023-11-08T19:24:32.826484+0000","modified":"2023-11-08T19:24:51.618072+0000","last_up_change":"2023-11-08T19:24:43.573880+0000","last_in_change":"2023-11-08T19:24:35.314525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:44.551530+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-08T19:24:48.256781+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-08T19:24:48.954990+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bae98ec1-d94e-403d-a220-a522a53ea6f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6809","nonce":1487830566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6811","nonce":1487830566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6815","nonce":1487830566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6813","nonce":1487830566}]},"public_addr":"172.21.15.143:6809/1487830566","cluster_addr":"172.21.15.143:6811/1487830566","heartbeat_back_addr":"172.21.15.143:6815/1487830566","heartbeat_front_addr":"172.21.15.143:6813/1487830566","state":["exists","up"]},{"osd":1,"uuid":"5cb6538a-1ccb-4322-8633-09f66af0b7e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6801","nonce":787341397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6803","nonce":787341397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6807","nonce":787341397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6805","nonce":787341397}]},"public_addr":"172.21.15.143:6801/787341397","cluster_addr":"172.21.15.143:6803/787341397","heartbeat_back_addr":"172.21.15.143:6807/787341397","heartbeat_front_addr":"172.21.15.143:6805/787341397","state":["exists","up"]},{"osd":2,"uuid":"cdbf522c-60ef-4d60-9653-8dcb529c101a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6822","nonce":2301325597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6826","nonce":2301325597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6831","nonce":2301325597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6829","nonce":2301325597}]},"public_addr":"172.21.15.143:6822/2301325597","cluster_addr":"172.21.15.143:6826/2301325597","heartbeat_back_addr":"172.21.15.143:6831/2301325597","heartbeat_front_addr":"172.21.15.143:6829/2301325597","state":["exists","up"]},{"osd":3,"uuid":"4e2783cf-ebb9-4dd8-8245-bc1d17ce4325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6817","nonce":2021504428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6819","nonce":2021504428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6827","nonce":2021504428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6823","nonce":2021504428}]},"public_addr":"172.21.15.143:6817/2021504428","cluster_addr":"172.21.15.143:6819/2021504428","heartbeat_back_addr":"172.21.15.143:6827/2021504428","heartbeat_front_addr":"172.21.15.143:6823/2021504428","state":["exists","up"]},{"osd":4,"uuid":"691256fa-9b77-49b9-a346-7ae1adf0ca80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6801","nonce":1993380595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6803","nonce":1993380595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6807","nonce":1993380595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6805","nonce":1993380595}]},"public_addr":"172.21.15.174:6801/1993380595","cluster_addr":"172.21.15.174:6803/1993380595","heartbeat_back_addr":"172.21.15.174:6807/1993380595","heartbeat_front_addr":"172.21.15.174:6805/1993380595","state":["exists","up"]},{"osd":5,"uuid":"756fc352-48c9-46ce-b52e-0541fe6f8525","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6809","nonce":1777923303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6811","nonce":1777923303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6815","nonce":1777923303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6813","nonce":1777923303}]},"public_addr":"172.21.15.174:6809/1777923303","cluster_addr":"172.21.15.174:6811/1777923303","heartbeat_back_addr":"172.21.15.174:6815/1777923303","heartbeat_front_addr":"172.21.15.174:6813/1777923303","state":["exists","up"]},{"osd":6,"uuid":"fab2ac2b-ed97-4cde-9825-b480eabf8b8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6825","nonce":2755130316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6827","nonce":2755130316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6831","nonce":2755130316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6829","nonce":2755130316}]},"public_addr":"172.21.15.174:6825/2755130316","cluster_addr":"172.21.15.174:6827/2755130316","heartbeat_back_addr":"172.21.15.174:6831/2755130316","heartbeat_front_addr":"172.21.15.174:6829/2755130316","state":["exists","up"]},{"osd":7,"uuid":"3de9b797-2ab6-4d41-a863-7e7d638fd703","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6817","nonce":2847141411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6819","nonce":2847141411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6823","nonce":2847141411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6821","nonce":2847141411}]},"public_addr":"172.21.15.174:6817/2847141411","cluster_addr":"172.21.15.174:6819/2847141411","heartbeat_back_addr":"172.21.15.174:6823/2847141411","heartbeat_front_addr":"172.21.15.174:6821/2847141411","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.477076+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.496890+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.539305+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.454673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.538881+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.521294+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.520712+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.540404+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:25:22.956 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.955+0000 7f61d8817700 1 -- 172.21.15.143:0/3080918614 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f61b406af00 msgr2=0x7f61b406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:22.957 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.955+0000 7f61d8817700 1 --2- 172.21.15.143:0/3080918614 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f61b406af00 0x7f61b406d3c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f61c80010e0 tx=0x7f61c8009040 comp rx=0 tx=0).stop 2023-11-08T19:25:22.957 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.955+0000 7f61d8817700 1 -- 172.21.15.143:0/3080918614 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61d41618f0 msgr2=0x7f61d41f2f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:22.957 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.955+0000 7f61d8817700 1 --2- 172.21.15.143:0/3080918614 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61d41618f0 0x7f61d41f2f10 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f61cc004000 tx=0x7f61cc004ad0 comp rx=0 tx=0).stop 2023-11-08T19:25:22.957 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.955+0000 7f61d8817700 1 -- 172.21.15.143:0/3080918614 shutdown_connections 2023-11-08T19:25:22.958 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.955+0000 7f61d8817700 1 --2- 172.21.15.143:0/3080918614 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f61b406af00 0x7f61b406d3c0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:22.958 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.955+0000 7f61d8817700 1 --2- 172.21.15.143:0/3080918614 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f61d406b320 0x7f61d415e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:22.958 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.955+0000 7f61d8817700 1 --2- 172.21.15.143:0/3080918614 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61d41618f0 0x7f61d41f2f10 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:22.959 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.955+0000 7f61d8817700 1 --2- 172.21.15.143:0/3080918614 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f61d415dd00 0x7f61d4160c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:22.959 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.955+0000 7f61d8817700 1 -- 172.21.15.143:0/3080918614 >> 172.21.15.143:0/3080918614 conn(0x7f61d4099ab0 msgr2=0x7f61d40cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:22.959 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.955+0000 7f61d8817700 1 -- 172.21.15.143:0/3080918614 shutdown_connections 2023-11-08T19:25:22.960 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:22.955+0000 7f61d8817700 1 -- 172.21.15.143:0/3080918614 wait complete. 2023-11-08T19:25:22.969 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T19:24:44.551530+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-08T19:24:48.256781+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-08T19:24:48.954990+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-08T19:25:22.970 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T19:25:23.093 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.092+0000 7f2b6da54700 1 Processor -- start 2023-11-08T19:25:23.094 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.092+0000 7f2b6da54700 1 -- start start 2023-11-08T19:25:23.097 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.093+0000 7f2b6da54700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f2b6815dca0 0x7f2b680b93f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:23.097 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.093+0000 7f2b6da54700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b680b8f50 con 0x7f2b680b5ae0 2023-11-08T19:25:23.097 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.093+0000 7f2b6da54700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b680b7eb0 con 0x7f2b680b5630 2023-11-08T19:25:23.098 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.093+0000 7f2b6da54700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b680b3360 con 0x7f2b6815dca0 2023-11-08T19:25:23.098 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.093+0000 7f2b677fe700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f2b680b5ae0 0x7f2b680b7280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:45836/0 (socket says 172.21.15.143:45836) 2023-11-08T19:25:23.098 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.093+0000 7f2b677fe700 1 -- 172.21.15.143:0/2737688645 learned_addr learned my addr 172.21.15.143:0/2737688645 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:25:23.098 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.093+0000 7f2b66ffd700 1 --2- 172.21.15.143:0/2737688645 >> v2:172.21.15.174:3300/0 conn(0x7f2b6815dca0 0x7f2b680b93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:23.099 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.093+0000 7f2b65ffb700 1 -- 172.21.15.143:0/2737688645 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1825118806 0 0) 0x7f2b680b8f50 con 0x7f2b680b5ae0 2023-11-08T19:25:23.099 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.093+0000 7f2b65ffb700 1 -- 172.21.15.143:0/2737688645 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b500035b0 con 0x7f2b680b5ae0 2023-11-08T19:25:23.099 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.093+0000 7f2b66ffd700 1 -- 172.21.15.143:0/2737688645 >> v1:172.21.15.174:6789/0 conn(0x7f2b680b5630 legacy=0x7f2b680b6b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:23.099 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.093+0000 7f2b66ffd700 1 -- 172.21.15.143:0/2737688645 >> v1:172.21.15.143:6789/0 conn(0x7f2b680b5ae0 legacy=0x7f2b680b7280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:23.100 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.093+0000 7f2b66ffd700 1 -- 172.21.15.143:0/2737688645 --> v2:172.21.15.174:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b680b8190 con 0x7f2b6815dca0 2023-11-08T19:25:23.100 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.094+0000 7f2b66ffd700 1 --2- 172.21.15.143:0/2737688645 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2b6815dca0 0x7f2b680b93f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2b58009840 tx=0x7f2b58009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40da163458685026 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:23.100 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.094+0000 7f2b65ffb700 1 -- 172.21.15.143:0/2737688645 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b58011830 con 0x7f2b6815dca0 2023-11-08T19:25:23.101 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.094+0000 7f2b65ffb700 1 -- 172.21.15.143:0/2737688645 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b58011990 con 0x7f2b6815dca0 2023-11-08T19:25:23.101 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.094+0000 7f2b65ffb700 1 -- 172.21.15.143:0/2737688645 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b58011d20 con 0x7f2b6815dca0 2023-11-08T19:25:23.101 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.094+0000 7f2b6da54700 1 -- 172.21.15.143:0/2737688645 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2b6815dca0 msgr2=0x7f2b680b93f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:23.101 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.094+0000 7f2b6da54700 1 --2- 172.21.15.143:0/2737688645 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2b6815dca0 0x7f2b680b93f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2b58009840 tx=0x7f2b58009b50 comp rx=0 tx=0).stop 2023-11-08T19:25:23.101 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.094+0000 7f2b6da54700 1 -- 172.21.15.143:0/2737688645 shutdown_connections 2023-11-08T19:25:23.102 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.094+0000 7f2b6da54700 1 --2- 172.21.15.143:0/2737688645 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2b6815dca0 0x7f2b680b93f0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:23.102 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.094+0000 7f2b6da54700 1 -- 172.21.15.143:0/2737688645 >> 172.21.15.143:0/2737688645 conn(0x7f2b68066d00 msgr2=0x7f2b68067110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:23.102 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.094+0000 7f2b6da54700 1 -- 172.21.15.143:0/2737688645 shutdown_connections 2023-11-08T19:25:23.102 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.094+0000 7f2b6da54700 1 -- 172.21.15.143:0/2737688645 wait complete. 2023-11-08T19:25:23.102 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.096+0000 7f2b6da54700 1 Processor -- start 2023-11-08T19:25:23.103 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.096+0000 7f2b6da54700 1 -- start start 2023-11-08T19:25:23.105 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.096+0000 7f2b6da54700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2b680b5630 0x7f2b68156860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:23.106 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.096+0000 7f2b6da54700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2b680b5ae0 0x7f2b681504d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:23.106 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.096+0000 7f2b6da54700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2b6815dca0 0x7f2b6815aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:23.106 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.096+0000 7f2b6da54700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b681551f0 con 0x7f2b6815dca0 2023-11-08T19:25:23.106 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.096+0000 7f2b6da54700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b68156da0 con 0x7f2b680b5630 2023-11-08T19:25:23.107 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.097+0000 7f2b6da54700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b68153fb0 con 0x7f2b680b5ae0 2023-11-08T19:25:23.107 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.097+0000 7f2b677fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2b6815dca0 0x7f2b6815aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:23.107 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.097+0000 7f2b677fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2b6815dca0 0x7f2b6815aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49686/0 (socket says 172.21.15.143:49686) 2023-11-08T19:25:23.107 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.097+0000 7f2b677fe700 1 -- 172.21.15.143:0/780074295 learned_addr learned my addr 172.21.15.143:0/780074295 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:23.107 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.097+0000 7f2b667fc700 1 --2- 172.21.15.143:0/780074295 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2b680b5ae0 0x7f2b681504d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:23.108 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.097+0000 7f2b66ffd700 1 --2- 172.21.15.143:0/780074295 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2b680b5630 0x7f2b68156860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:23.108 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.097+0000 7f2b677fe700 1 -- 172.21.15.143:0/780074295 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2b680b5ae0 msgr2=0x7f2b681504d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:23.108 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.097+0000 7f2b677fe700 1 --2- 172.21.15.143:0/780074295 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2b680b5ae0 0x7f2b681504d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:23.108 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.097+0000 7f2b677fe700 1 -- 172.21.15.143:0/780074295 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2b680b5630 msgr2=0x7f2b68156860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:23.108 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.097+0000 7f2b677fe700 1 --2- 172.21.15.143:0/780074295 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2b680b5630 0x7f2b68156860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:23.109 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.097+0000 7f2b677fe700 1 -- 172.21.15.143:0/780074295 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b580094a0 con 0x7f2b6815dca0 2023-11-08T19:25:23.109 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.097+0000 7f2b667fc700 1 --2- 172.21.15.143:0/780074295 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2b680b5ae0 0x7f2b681504d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:25:23.109 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.097+0000 7f2b66ffd700 1 --2- 172.21.15.143:0/780074295 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2b680b5630 0x7f2b68156860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:25:23.109 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.097+0000 7f2b677fe700 1 --2- 172.21.15.143:0/780074295 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2b6815dca0 0x7f2b6815aa00 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2b6000cb40 tx=0x7f2b6000cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:23.109 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.097+0000 7f2b4ffff700 1 -- 172.21.15.143:0/780074295 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b6000ab50 con 0x7f2b6815dca0 2023-11-08T19:25:23.109 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.097+0000 7f2b4ffff700 1 -- 172.21.15.143:0/780074295 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b6000acb0 con 0x7f2b6815dca0 2023-11-08T19:25:23.110 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.097+0000 7f2b6da54700 1 -- 172.21.15.143:0/780074295 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b68154290 con 0x7f2b6815dca0 2023-11-08T19:25:23.110 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.097+0000 7f2b4ffff700 1 -- 172.21.15.143:0/780074295 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b60012620 con 0x7f2b6815dca0 2023-11-08T19:25:23.110 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.097+0000 7f2b6da54700 1 -- 172.21.15.143:0/780074295 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b68154850 con 0x7f2b6815dca0 2023-11-08T19:25:23.110 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.098+0000 7f2b6da54700 1 -- 172.21.15.143:0/780074295 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b68155050 con 0x7f2b6815dca0 2023-11-08T19:25:23.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.099+0000 7f2b4ffff700 1 -- 172.21.15.143:0/780074295 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2b6001b3b0 con 0x7f2b6815dca0 2023-11-08T19:25:23.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.099+0000 7f2b4ffff700 1 --2- 172.21.15.143:0/780074295 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f2b5406af00 0x7f2b5406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:23.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.099+0000 7f2b4ffff700 1 -- 172.21.15.143:0/780074295 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2b60089340 con 0x7f2b6815dca0 2023-11-08T19:25:23.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.099+0000 7f2b66ffd700 1 --2- 172.21.15.143:0/780074295 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f2b5406af00 0x7f2b5406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:23.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.100+0000 7f2b66ffd700 1 --2- 172.21.15.143:0/780074295 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f2b5406af00 0x7f2b5406d3c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2b580090b0 tx=0x7f2b5801c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:23.112 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.103+0000 7f2b4ffff700 1 -- 172.21.15.143:0/780074295 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2b68155050 con 0x7f2b6815dca0 2023-11-08T19:25:23.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.291+0000 7f2b6da54700 1 -- 172.21.15.143:0/780074295 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2b68155050 con 0x7f2b6815dca0 2023-11-08T19:25:23.293 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.291+0000 7f2b4ffff700 1 -- 172.21.15.143:0/780074295 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2b68155050 con 0x7f2b6815dca0 2023-11-08T19:25:23.293 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2023-11-08T19:25:23.295 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.294+0000 7f2b6da54700 1 -- 172.21.15.143:0/780074295 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f2b5406af00 msgr2=0x7f2b5406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:23.295 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.294+0000 7f2b6da54700 1 --2- 172.21.15.143:0/780074295 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f2b5406af00 0x7f2b5406d3c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2b580090b0 tx=0x7f2b5801c040 comp rx=0 tx=0).stop 2023-11-08T19:25:23.296 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.294+0000 7f2b6da54700 1 -- 172.21.15.143:0/780074295 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2b6815dca0 msgr2=0x7f2b6815aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:23.296 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.294+0000 7f2b6da54700 1 --2- 172.21.15.143:0/780074295 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2b6815dca0 0x7f2b6815aa00 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2b6000cb40 tx=0x7f2b6000cf00 comp rx=0 tx=0).stop 2023-11-08T19:25:23.296 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.294+0000 7f2b6da54700 1 -- 172.21.15.143:0/780074295 shutdown_connections 2023-11-08T19:25:23.296 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.294+0000 7f2b6da54700 1 --2- 172.21.15.143:0/780074295 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f2b5406af00 0x7f2b5406d3c0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:23.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.294+0000 7f2b6da54700 1 --2- 172.21.15.143:0/780074295 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2b6815dca0 0x7f2b6815aa00 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:23.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.294+0000 7f2b6da54700 1 --2- 172.21.15.143:0/780074295 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2b680b5630 0x7f2b68156860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:23.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.294+0000 7f2b6da54700 1 --2- 172.21.15.143:0/780074295 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2b680b5ae0 0x7f2b681504d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:23.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.294+0000 7f2b6da54700 1 -- 172.21.15.143:0/780074295 >> 172.21.15.143:0/780074295 conn(0x7f2b68066d00 msgr2=0x7f2b680d19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:23.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.294+0000 7f2b6da54700 1 -- 172.21.15.143:0/780074295 shutdown_connections 2023-11-08T19:25:23.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.294+0000 7f2b6da54700 1 -- 172.21.15.143:0/780074295 wait complete. 2023-11-08T19:25:23.308 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-08T19:25:23.431 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.430+0000 7fecaedda700 1 Processor -- start 2023-11-08T19:25:23.431 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.430+0000 7fecaedda700 1 -- start start 2023-11-08T19:25:23.434 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.430+0000 7fecaedda700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7feca815bb20 0x7feca80b2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:23.434 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.430+0000 7fecaedda700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7feca80d1d80 0x7feca80cc4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:23.434 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.430+0000 7fecaedda700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feca80d0420 con 0x7feca80b1730 2023-11-08T19:25:23.435 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.430+0000 7fecaedda700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feca80cf600 con 0x7feca815bb20 2023-11-08T19:25:23.435 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.430+0000 7fecaedda700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feca80cf810 con 0x7feca80d1d80 2023-11-08T19:25:23.435 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.431+0000 7feca7fff700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7feca80b1730 0x7feca80d1670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:45846/0 (socket says 172.21.15.143:45846) 2023-11-08T19:25:23.435 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.431+0000 7fecacb76700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7feca815bb20 0x7feca80b2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:23.435 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.431+0000 7feca7fff700 1 -- 172.21.15.143:0/1825319464 learned_addr learned my addr 172.21.15.143:0/1825319464 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:25:23.436 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.431+0000 7fecad377700 1 --2- 172.21.15.143:0/1825319464 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7feca80d1d80 0x7feca80cc4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:23.436 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.431+0000 7fecacb76700 1 -- 172.21.15.143:0/1825319464 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7feca80d1d80 msgr2=0x7feca80cc4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:23.436 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.431+0000 7fecacb76700 1 --2- 172.21.15.143:0/1825319464 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7feca80d1d80 0x7feca80cc4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:23.436 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.431+0000 7fecacb76700 1 -- 172.21.15.143:0/1825319464 >> v1:172.21.15.143:6789/0 conn(0x7feca80b1730 legacy=0x7feca80d1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:23.436 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.431+0000 7fecacb76700 1 -- 172.21.15.143:0/1825319464 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feca80cfaf0 con 0x7feca815bb20 2023-11-08T19:25:23.437 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.431+0000 7feca77fe700 1 -- 172.21.15.143:0/1825319464 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1604000186 0 0) 0x7feca80d0420 con 0x7feca80b1730 2023-11-08T19:25:23.437 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.431+0000 7fecacb76700 1 --2- 172.21.15.143:0/1825319464 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feca815bb20 0x7feca80b2670 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fec9c009840 tx=0x7fec9c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3c6cc943229e606 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:23.437 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.431+0000 7feca77fe700 1 -- 172.21.15.143:0/1825319464 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec9c011830 con 0x7feca815bb20 2023-11-08T19:25:23.437 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.431+0000 7feca77fe700 1 -- 172.21.15.143:0/1825319464 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec9c011990 con 0x7feca815bb20 2023-11-08T19:25:23.437 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.431+0000 7feca77fe700 1 -- 172.21.15.143:0/1825319464 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec9c011d20 con 0x7feca815bb20 2023-11-08T19:25:23.438 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.432+0000 7fecaedda700 1 -- 172.21.15.143:0/1825319464 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feca815bb20 msgr2=0x7feca80b2670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:23.438 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.432+0000 7fecaedda700 1 --2- 172.21.15.143:0/1825319464 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feca815bb20 0x7feca80b2670 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fec9c009840 tx=0x7fec9c009b50 comp rx=0 tx=0).stop 2023-11-08T19:25:23.438 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.432+0000 7fecaedda700 1 -- 172.21.15.143:0/1825319464 shutdown_connections 2023-11-08T19:25:23.438 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.432+0000 7fecaedda700 1 --2- 172.21.15.143:0/1825319464 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7feca80d1d80 0x7feca80cc4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:23.438 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.432+0000 7fecaedda700 1 --2- 172.21.15.143:0/1825319464 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feca815bb20 0x7feca80b2670 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:23.439 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.432+0000 7fecaedda700 1 -- 172.21.15.143:0/1825319464 >> 172.21.15.143:0/1825319464 conn(0x7feca808b000 msgr2=0x7feca809ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:23.439 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.432+0000 7fecaedda700 1 -- 172.21.15.143:0/1825319464 shutdown_connections 2023-11-08T19:25:23.439 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.432+0000 7fecaedda700 1 -- 172.21.15.143:0/1825319464 wait complete. 2023-11-08T19:25:23.439 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.433+0000 7fecaedda700 1 Processor -- start 2023-11-08T19:25:23.439 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.433+0000 7fecaedda700 1 -- start start 2023-11-08T19:25:23.442 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.434+0000 7fecaedda700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feca80b1730 0x7feca81f0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:23.442 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.434+0000 7fecaedda700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feca80d1d80 0x7feca815e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:23.443 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.434+0000 7fecaedda700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7feca815bb20 0x7feca815e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:23.443 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.434+0000 7fecaedda700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feca815db00 con 0x7feca80d1d80 2023-11-08T19:25:23.443 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.434+0000 7fecaedda700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feca81f1110 con 0x7feca80b1730 2023-11-08T19:25:23.443 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.434+0000 7fecaedda700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feca815ce80 con 0x7feca815bb20 2023-11-08T19:25:23.443 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.434+0000 7feca7fff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feca80d1d80 0x7feca815e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:23.444 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.434+0000 7feca7fff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feca80d1d80 0x7feca815e2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49698/0 (socket says 172.21.15.143:49698) 2023-11-08T19:25:23.444 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.434+0000 7feca7fff700 1 -- 172.21.15.143:0/4185402761 learned_addr learned my addr 172.21.15.143:0/4185402761 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:23.444 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.434+0000 7fecacb76700 1 --2- 172.21.15.143:0/4185402761 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feca80b1730 0x7feca81f0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:23.444 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.434+0000 7fecad377700 1 --2- 172.21.15.143:0/4185402761 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7feca815bb20 0x7feca815e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:23.444 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.434+0000 7feca7fff700 1 -- 172.21.15.143:0/4185402761 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7feca815bb20 msgr2=0x7feca815e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:23.445 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.434+0000 7feca7fff700 1 --2- 172.21.15.143:0/4185402761 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7feca815bb20 0x7feca815e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:23.445 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.434+0000 7feca7fff700 1 -- 172.21.15.143:0/4185402761 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feca80b1730 msgr2=0x7feca81f0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:23.445 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.434+0000 7feca7fff700 1 --2- 172.21.15.143:0/4185402761 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feca80b1730 0x7feca81f0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:23.445 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.434+0000 7feca7fff700 1 -- 172.21.15.143:0/4185402761 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec9c0094a0 con 0x7feca80d1d80 2023-11-08T19:25:23.445 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.434+0000 7feca7fff700 1 --2- 172.21.15.143:0/4185402761 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feca80d1d80 0x7feca815e2a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fec940011e0 tx=0x7fec9400ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:23.446 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.434+0000 7feca57fa700 1 -- 172.21.15.143:0/4185402761 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec94004c80 con 0x7feca80d1d80 2023-11-08T19:25:23.446 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.435+0000 7feca57fa700 1 -- 172.21.15.143:0/4185402761 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec94004e00 con 0x7feca80d1d80 2023-11-08T19:25:23.446 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.435+0000 7feca57fa700 1 -- 172.21.15.143:0/4185402761 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec94007310 con 0x7feca80d1d80 2023-11-08T19:25:23.446 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.435+0000 7fecaedda700 1 -- 172.21.15.143:0/4185402761 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feca815d160 con 0x7feca80d1d80 2023-11-08T19:25:23.446 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.435+0000 7fecaedda700 1 -- 172.21.15.143:0/4185402761 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feca815d720 con 0x7feca80d1d80 2023-11-08T19:25:23.447 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.435+0000 7fecaedda700 1 -- 172.21.15.143:0/4185402761 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feca815dc40 con 0x7feca80d1d80 2023-11-08T19:25:23.447 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.436+0000 7feca57fa700 1 -- 172.21.15.143:0/4185402761 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fec94007470 con 0x7feca80d1d80 2023-11-08T19:25:23.447 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.436+0000 7feca57fa700 1 --2- 172.21.15.143:0/4185402761 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fec8c06af00 0x7fec8c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:23.447 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.436+0000 7feca57fa700 1 -- 172.21.15.143:0/4185402761 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fec94089250 con 0x7feca80d1d80 2023-11-08T19:25:23.447 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.437+0000 7fecacb76700 1 --2- 172.21.15.143:0/4185402761 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fec8c06af00 0x7fec8c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:23.448 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.437+0000 7fecacb76700 1 --2- 172.21.15.143:0/4185402761 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fec8c06af00 0x7fec8c06d3c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fec9c0090b0 tx=0x7fec9c01c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:23.448 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.441+0000 7feca57fa700 1 -- 172.21.15.143:0/4185402761 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fec94054300 con 0x7feca80d1d80 2023-11-08T19:25:23.633 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.631+0000 7fecaedda700 1 -- 172.21.15.143:0/4185402761 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7feca804eee0 con 0x7feca80d1d80 2023-11-08T19:25:23.633 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.632+0000 7feca57fa700 1 -- 172.21.15.143:0/4185402761 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7fec94057920 con 0x7feca80d1d80 2023-11-08T19:25:23.633 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2023-11-08T19:25:23.635 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.634+0000 7fecaedda700 1 -- 172.21.15.143:0/4185402761 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fec8c06af00 msgr2=0x7fec8c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:23.636 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.634+0000 7fecaedda700 1 --2- 172.21.15.143:0/4185402761 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fec8c06af00 0x7fec8c06d3c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fec9c0090b0 tx=0x7fec9c01c040 comp rx=0 tx=0).stop 2023-11-08T19:25:23.636 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.634+0000 7fecaedda700 1 -- 172.21.15.143:0/4185402761 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feca80d1d80 msgr2=0x7feca815e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:23.636 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.634+0000 7fecaedda700 1 --2- 172.21.15.143:0/4185402761 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feca80d1d80 0x7feca815e2a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fec940011e0 tx=0x7fec9400ca30 comp rx=0 tx=0).stop 2023-11-08T19:25:23.636 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.634+0000 7fecaedda700 1 -- 172.21.15.143:0/4185402761 shutdown_connections 2023-11-08T19:25:23.636 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.634+0000 7fecaedda700 1 --2- 172.21.15.143:0/4185402761 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fec8c06af00 0x7fec8c06d3c0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:23.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.634+0000 7fecaedda700 1 --2- 172.21.15.143:0/4185402761 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7feca815bb20 0x7feca815e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:23.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.634+0000 7fecaedda700 1 --2- 172.21.15.143:0/4185402761 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feca80b1730 0x7feca81f0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:23.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.634+0000 7fecaedda700 1 --2- 172.21.15.143:0/4185402761 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feca80d1d80 0x7feca815e2a0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:23.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.634+0000 7fecaedda700 1 -- 172.21.15.143:0/4185402761 >> 172.21.15.143:0/4185402761 conn(0x7feca808b000 msgr2=0x7feca8070a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:23.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.634+0000 7fecaedda700 1 -- 172.21.15.143:0/4185402761 shutdown_connections 2023-11-08T19:25:23.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.634+0000 7fecaedda700 1 -- 172.21.15.143:0/4185402761 wait complete. 2023-11-08T19:25:23.647 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-08T19:25:23.771 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.769+0000 7f4d55f5a700 1 Processor -- start 2023-11-08T19:25:23.771 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.769+0000 7f4d55f5a700 1 -- start start 2023-11-08T19:25:23.774 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.770+0000 7f4d55f5a700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f4d5015dba0 0x7f4d5006c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:23.774 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.770+0000 7f4d55f5a700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f4d5006b240 0x7f4d500caff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:23.774 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.770+0000 7f4d55f5a700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d50161890 con 0x7f4d500cb530 2023-11-08T19:25:23.775 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.770+0000 7f4d55f5a700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d5006b740 con 0x7f4d5015dba0 2023-11-08T19:25:23.775 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.770+0000 7f4d55f5a700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d5006ba20 con 0x7f4d5006b240 2023-11-08T19:25:23.775 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.770+0000 7f4d4f7fe700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f4d5015dba0 0x7f4d5006c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:23.775 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.770+0000 7f4d4f7fe700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f4d5015dba0 0x7f4d5006c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49704/0 (socket says 172.21.15.143:49704) 2023-11-08T19:25:23.775 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.770+0000 7f4d4f7fe700 1 -- 172.21.15.143:0/2291666150 learned_addr learned my addr 172.21.15.143:0/2291666150 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:23.776 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.771+0000 7f4d4effd700 1 --2- 172.21.15.143:0/2291666150 >> v2:172.21.15.174:3300/0 conn(0x7f4d5006b240 0x7f4d500caff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:23.776 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.771+0000 7f4d4f7fe700 1 -- 172.21.15.143:0/2291666150 >> v2:172.21.15.174:3300/0 conn(0x7f4d5006b240 msgr2=0x7f4d500caff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:23.776 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.771+0000 7f4d4f7fe700 1 --2- 172.21.15.143:0/2291666150 >> v2:172.21.15.174:3300/0 conn(0x7f4d5006b240 0x7f4d500caff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:23.776 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.771+0000 7f4d4f7fe700 1 -- 172.21.15.143:0/2291666150 >> v1:172.21.15.174:6789/0 conn(0x7f4d500cb530 legacy=0x7f4d500b22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:23.776 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.771+0000 7f4d4f7fe700 1 -- 172.21.15.143:0/2291666150 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d5006bd00 con 0x7f4d5015dba0 2023-11-08T19:25:23.777 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.771+0000 7f4d4f7fe700 1 --2- 172.21.15.143:0/2291666150 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4d5015dba0 0x7f4d5006c610 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f4d500d0810 tx=0x7f4d44009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7ac7b2c3ae5d7e6 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:23.777 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.771+0000 7f4d4e7fc700 1 -- 172.21.15.143:0/2291666150 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d4400b870 con 0x7f4d5015dba0 2023-11-08T19:25:23.777 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.771+0000 7f4d4e7fc700 1 -- 172.21.15.143:0/2291666150 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d4400b9d0 con 0x7f4d5015dba0 2023-11-08T19:25:23.777 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.771+0000 7f4d4e7fc700 1 -- 172.21.15.143:0/2291666150 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d4400bd60 con 0x7f4d5015dba0 2023-11-08T19:25:23.777 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.771+0000 7f4d55f5a700 1 -- 172.21.15.143:0/2291666150 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4d5015dba0 msgr2=0x7f4d5006c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:23.778 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.772+0000 7f4d55f5a700 1 --2- 172.21.15.143:0/2291666150 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4d5015dba0 0x7f4d5006c610 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f4d500d0810 tx=0x7f4d44009bd0 comp rx=0 tx=0).stop 2023-11-08T19:25:23.778 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.772+0000 7f4d55f5a700 1 -- 172.21.15.143:0/2291666150 shutdown_connections 2023-11-08T19:25:23.778 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.772+0000 7f4d55f5a700 1 --2- 172.21.15.143:0/2291666150 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4d5015dba0 0x7f4d5006c610 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:23.778 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.772+0000 7f4d55f5a700 1 --2- 172.21.15.143:0/2291666150 >> v2:172.21.15.174:3300/0 conn(0x7f4d5006b240 0x7f4d500caff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:23.779 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.772+0000 7f4d55f5a700 1 -- 172.21.15.143:0/2291666150 >> 172.21.15.143:0/2291666150 conn(0x7f4d500cd2e0 msgr2=0x7f4d500cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:23.779 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.772+0000 7f4d55f5a700 1 -- 172.21.15.143:0/2291666150 shutdown_connections 2023-11-08T19:25:23.779 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.772+0000 7f4d55f5a700 1 -- 172.21.15.143:0/2291666150 wait complete. 2023-11-08T19:25:23.779 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.773+0000 7f4d55f5a700 1 Processor -- start 2023-11-08T19:25:23.782 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.773+0000 7f4d55f5a700 1 -- start start 2023-11-08T19:25:23.782 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.774+0000 7f4d55f5a700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4d5006b240 0x7f4d501e8030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:23.783 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.774+0000 7f4d55f5a700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4d500cb530 0x7f4d500ad430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:23.783 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.774+0000 7f4d55f5a700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4d5015dba0 0x7f4d500ad970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:23.783 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.774+0000 7f4d55f5a700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d501598f0 con 0x7f4d500cb530 2023-11-08T19:25:23.783 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.774+0000 7f4d55f5a700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d500adeb0 con 0x7f4d5015dba0 2023-11-08T19:25:23.783 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.774+0000 7f4d55f5a700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d501e7ba0 con 0x7f4d5006b240 2023-11-08T19:25:23.784 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.774+0000 7f4d4effd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4d500cb530 0x7f4d500ad430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:23.784 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.774+0000 7f4d4effd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4d500cb530 0x7f4d500ad430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49712/0 (socket says 172.21.15.143:49712) 2023-11-08T19:25:23.784 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.774+0000 7f4d4effd700 1 -- 172.21.15.143:0/3457187490 learned_addr learned my addr 172.21.15.143:0/3457187490 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:23.784 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.774+0000 7f4d4f7fe700 1 --2- 172.21.15.143:0/3457187490 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4d5006b240 0x7f4d501e8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:23.785 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.774+0000 7f4d4ffff700 1 --2- 172.21.15.143:0/3457187490 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4d5015dba0 0x7f4d500ad970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:23.785 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.774+0000 7f4d4effd700 1 -- 172.21.15.143:0/3457187490 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4d5006b240 msgr2=0x7f4d501e8030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:23.785 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.774+0000 7f4d4effd700 1 --2- 172.21.15.143:0/3457187490 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4d5006b240 0x7f4d501e8030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:23.785 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.774+0000 7f4d4effd700 1 -- 172.21.15.143:0/3457187490 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4d5015dba0 msgr2=0x7f4d500ad970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:23.785 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.774+0000 7f4d4effd700 1 --2- 172.21.15.143:0/3457187490 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4d5015dba0 0x7f4d500ad970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:23.786 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.774+0000 7f4d4effd700 1 -- 172.21.15.143:0/3457187490 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d44009550 con 0x7f4d500cb530 2023-11-08T19:25:23.786 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.774+0000 7f4d4effd700 1 --2- 172.21.15.143:0/3457187490 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4d500cb530 0x7f4d500ad430 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f4d4000e810 tx=0x7f4d4000eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:23.786 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.774+0000 7f4d37fff700 1 -- 172.21.15.143:0/3457187490 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d4000c070 con 0x7f4d500cb530 2023-11-08T19:25:23.786 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.774+0000 7f4d37fff700 1 -- 172.21.15.143:0/3457187490 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d40013610 con 0x7f4d500cb530 2023-11-08T19:25:23.786 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.774+0000 7f4d55f5a700 1 -- 172.21.15.143:0/3457187490 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d501e6f90 con 0x7f4d500cb530 2023-11-08T19:25:23.787 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.774+0000 7f4d37fff700 1 -- 172.21.15.143:0/3457187490 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d400139c0 con 0x7f4d500cb530 2023-11-08T19:25:23.787 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.775+0000 7f4d55f5a700 1 -- 172.21.15.143:0/3457187490 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d501e7550 con 0x7f4d500cb530 2023-11-08T19:25:23.787 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.775+0000 7f4d55f5a700 1 -- 172.21.15.143:0/3457187490 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d501e7d50 con 0x7f4d500cb530 2023-11-08T19:25:23.787 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.776+0000 7f4d37fff700 1 -- 172.21.15.143:0/3457187490 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4d40013b20 con 0x7f4d500cb530 2023-11-08T19:25:23.787 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.776+0000 7f4d37fff700 1 --2- 172.21.15.143:0/3457187490 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f4d3c06af00 0x7f4d3c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:23.788 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.776+0000 7f4d37fff700 1 -- 172.21.15.143:0/3457187490 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4d400892f0 con 0x7f4d500cb530 2023-11-08T19:25:23.788 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.777+0000 7f4d4f7fe700 1 --2- 172.21.15.143:0/3457187490 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f4d3c06af00 0x7f4d3c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:23.788 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.777+0000 7f4d4f7fe700 1 --2- 172.21.15.143:0/3457187490 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f4d3c06af00 0x7f4d3c06d3c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4d44014040 tx=0x7f4d44008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:23.788 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.781+0000 7f4d37fff700 1 -- 172.21.15.143:0/3457187490 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4d501e7d50 con 0x7f4d500cb530 2023-11-08T19:25:23.974 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.972+0000 7f4d55f5a700 1 -- 172.21.15.143:0/3457187490 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4d501e7d10 con 0x7f4d500cb530 2023-11-08T19:25:23.974 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.973+0000 7f4d37fff700 1 -- 172.21.15.143:0/3457187490 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4d40057e40 con 0x7f4d500cb530 2023-11-08T19:25:23.974 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 64 2023-11-08T19:25:23.976 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.975+0000 7f4d55f5a700 1 -- 172.21.15.143:0/3457187490 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f4d3c06af00 msgr2=0x7f4d3c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:23.977 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.975+0000 7f4d55f5a700 1 --2- 172.21.15.143:0/3457187490 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f4d3c06af00 0x7f4d3c06d3c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4d44014040 tx=0x7f4d44008030 comp rx=0 tx=0).stop 2023-11-08T19:25:23.977 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.975+0000 7f4d55f5a700 1 -- 172.21.15.143:0/3457187490 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4d500cb530 msgr2=0x7f4d500ad430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:23.977 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.975+0000 7f4d55f5a700 1 --2- 172.21.15.143:0/3457187490 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4d500cb530 0x7f4d500ad430 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f4d4000e810 tx=0x7f4d4000eb20 comp rx=0 tx=0).stop 2023-11-08T19:25:23.977 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.975+0000 7f4d55f5a700 1 -- 172.21.15.143:0/3457187490 shutdown_connections 2023-11-08T19:25:23.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.975+0000 7f4d55f5a700 1 --2- 172.21.15.143:0/3457187490 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f4d3c06af00 0x7f4d3c06d3c0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:23.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.975+0000 7f4d55f5a700 1 --2- 172.21.15.143:0/3457187490 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4d5006b240 0x7f4d501e8030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:23.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.975+0000 7f4d55f5a700 1 --2- 172.21.15.143:0/3457187490 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4d5015dba0 0x7f4d500ad970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:23.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.975+0000 7f4d55f5a700 1 --2- 172.21.15.143:0/3457187490 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4d500cb530 0x7f4d500ad430 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:23.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.975+0000 7f4d55f5a700 1 -- 172.21.15.143:0/3457187490 >> 172.21.15.143:0/3457187490 conn(0x7f4d500cd2e0 msgr2=0x7f4d500b1b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:23.979 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.975+0000 7f4d55f5a700 1 -- 172.21.15.143:0/3457187490 shutdown_connections 2023-11-08T19:25:23.979 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:23.975+0000 7f4d55f5a700 1 -- 172.21.15.143:0/3457187490 wait complete. 2023-11-08T19:25:24.001 INFO:tasks.cephfs_test_runner:Starting test: test_attaching_pools_with_snaps_to_fs_fails (tasks.cephfs.test_snapshots.TestMonSnapsAndFsPools) Test that attempt to attach pool with snapshots to an fs fails 2023-11-08T19:25:24.002 INFO:tasks.cephfs_test_runner:test_attaching_pools_with_snaps_to_fs_fails (tasks.cephfs.test_snapshots.TestMonSnapsAndFsPools) 2023-11-08T19:25:24.003 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:25:24.122 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.121+0000 7f9fcfbd6700 1 Processor -- start 2023-11-08T19:25:24.123 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.121+0000 7f9fcfbd6700 1 -- start start 2023-11-08T19:25:24.126 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.122+0000 7f9fcfbd6700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9fc815aa30 0x7f9fc80cf510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:24.126 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.122+0000 7f9fcfbd6700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fc80cf080 con 0x7f9fc80b2210 2023-11-08T19:25:24.127 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.122+0000 7f9fcfbd6700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fc80ce470 con 0x7f9fc80cc530 2023-11-08T19:25:24.127 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.122+0000 7f9fcfbd6700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fc80ce750 con 0x7f9fc815aa30 2023-11-08T19:25:24.127 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.122+0000 7f9fcd171700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f9fc80b2210 0x7f9fc80cbe20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:45872/0 (socket says 172.21.15.143:45872) 2023-11-08T19:25:24.128 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.122+0000 7f9fcd171700 1 -- 172.21.15.143:0/1695164128 learned_addr learned my addr 172.21.15.143:0/1695164128 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:25:24.128 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.122+0000 7f9fcc970700 1 -- 172.21.15.143:0/1695164128 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 284589567 0 0) 0x7f9fc80cf080 con 0x7f9fc80b2210 2023-11-08T19:25:24.128 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.122+0000 7f9fcd972700 1 --2- 172.21.15.143:0/1695164128 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9fc815aa30 0x7f9fc80cf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:24.128 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.122+0000 7f9fcc970700 1 -- 172.21.15.143:0/1695164128 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9fb40035b0 con 0x7f9fc80b2210 2023-11-08T19:25:24.128 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.123+0000 7f9fcc970700 1 -- 172.21.15.143:0/1695164128 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3658340377 0 0) 0x7f9fb40035b0 con 0x7f9fc80b2210 2023-11-08T19:25:24.129 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.123+0000 7f9fcc970700 1 -- 172.21.15.143:0/1695164128 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9fc815aa30 msgr2=0x7f9fc80cf510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:24.129 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.123+0000 7f9fcc970700 1 --2- 172.21.15.143:0/1695164128 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9fc815aa30 0x7f9fc80cf510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:24.129 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.123+0000 7f9fcc970700 1 -- 172.21.15.143:0/1695164128 >> v1:172.21.15.174:6789/0 conn(0x7f9fc80cc530 legacy=0x7f9fc806db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:24.129 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.123+0000 7f9fcc970700 1 -- 172.21.15.143:0/1695164128 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fc80cea30 con 0x7f9fc80b2210 2023-11-08T19:25:24.129 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.123+0000 7f9fcc970700 1 -- 172.21.15.143:0/1695164128 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4147756303 0 0) 0x7f9fc4001670 con 0x7f9fc80b2210 2023-11-08T19:25:24.130 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.123+0000 7f9fcc970700 1 -- 172.21.15.143:0/1695164128 >> v1:172.21.15.143:6789/0 conn(0x7f9fc80b2210 legacy=0x7f9fc80cbe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:24.130 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.123+0000 7f9fcc970700 1 --2- 172.21.15.143:0/1695164128 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9fb4003a30 0x7f9fb4003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:24.130 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.123+0000 7f9fcc970700 1 --2- 172.21.15.143:0/1695164128 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9fc815aa30 0x7f9fb4004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:24.130 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.123+0000 7f9fcc970700 1 --2- 172.21.15.143:0/1695164128 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9fb4004d90 0x7f9fb4005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:24.130 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.123+0000 7f9fcc970700 1 -- 172.21.15.143:0/1695164128 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fc80ce750 con 0x7f9fb4003a30 2023-11-08T19:25:24.131 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.123+0000 7f9fcc970700 1 -- 172.21.15.143:0/1695164128 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb400bac0 con 0x7f9fb4004d90 2023-11-08T19:25:24.131 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.123+0000 7f9fcc970700 1 -- 172.21.15.143:0/1695164128 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb400bc80 con 0x7f9fc815aa30 2023-11-08T19:25:24.131 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.123+0000 7f9fcd972700 1 --2- 172.21.15.143:0/1695164128 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9fb4003a30 0x7f9fb4003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:24.131 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.123+0000 7f9fcd171700 1 --2- 172.21.15.143:0/1695164128 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9fc815aa30 0x7f9fb4004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:24.131 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.123+0000 7f9fce173700 1 --2- 172.21.15.143:0/1695164128 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9fb4004d90 0x7f9fb4005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:24.132 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.123+0000 7f9fcd972700 1 -- 172.21.15.143:0/1695164128 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9fc815aa30 msgr2=0x7f9fb4004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:24.134 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.123+0000 7f9fcd972700 1 --2- 172.21.15.143:0/1695164128 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9fc815aa30 0x7f9fb4004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:24.134 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.123+0000 7f9fcd972700 1 -- 172.21.15.143:0/1695164128 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9fb4004d90 msgr2=0x7f9fb4005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:24.134 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.123+0000 7f9fcd972700 1 --2- 172.21.15.143:0/1695164128 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9fb4004d90 0x7f9fb4005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:24.134 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.123+0000 7f9fcd972700 1 -- 172.21.15.143:0/1695164128 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fb40035b0 con 0x7f9fb4003a30 2023-11-08T19:25:24.135 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.123+0000 7f9fcd171700 1 --2- 172.21.15.143:0/1695164128 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9fc815aa30 0x7f9fb4004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:25:24.135 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.123+0000 7f9fcd972700 1 --2- 172.21.15.143:0/1695164128 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9fb4003a30 0x7f9fb4003e50 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f9fb8010140 tx=0x7f9fb4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24193a8f5842c2a9 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:24.135 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.123+0000 7f9fcc970700 1 -- 172.21.15.143:0/1695164128 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fb8010c60 con 0x7f9fb4003a30 2023-11-08T19:25:24.135 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.123+0000 7f9fcc970700 1 -- 172.21.15.143:0/1695164128 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fb8011040 con 0x7f9fb4003a30 2023-11-08T19:25:24.135 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.123+0000 7f9fcc970700 1 -- 172.21.15.143:0/1695164128 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fb8016480 con 0x7f9fb4003a30 2023-11-08T19:25:24.136 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.124+0000 7f9fcfbd6700 1 -- 172.21.15.143:0/1695164128 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9fb4003a30 msgr2=0x7f9fb4003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:24.136 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.124+0000 7f9fcfbd6700 1 --2- 172.21.15.143:0/1695164128 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9fb4003a30 0x7f9fb4003e50 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f9fb8010140 tx=0x7f9fb4005880 comp rx=0 tx=0).stop 2023-11-08T19:25:24.136 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.124+0000 7f9fce173700 1 -- 172.21.15.143:0/1695164128 reap_dead start 2023-11-08T19:25:24.136 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.124+0000 7f9fcfbd6700 1 -- 172.21.15.143:0/1695164128 shutdown_connections 2023-11-08T19:25:24.136 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.124+0000 7f9fcfbd6700 1 -- 172.21.15.143:0/1695164128 >> 172.21.15.143:0/1695164128 conn(0x7f9fc8099ab0 msgr2=0x7f9fc80d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:24.136 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.124+0000 7f9fcfbd6700 1 -- 172.21.15.143:0/1695164128 shutdown_connections 2023-11-08T19:25:24.137 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.124+0000 7f9fcfbd6700 1 -- 172.21.15.143:0/1695164128 wait complete. 2023-11-08T19:25:24.140 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.126+0000 7f9fcfbd6700 1 Processor -- start 2023-11-08T19:25:24.140 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.126+0000 7f9fcfbd6700 1 -- start start 2023-11-08T19:25:24.141 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.126+0000 7f9fcfbd6700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9fb4003a30 0x7f9fc815dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:24.141 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.126+0000 7f9fcfbd6700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9fc80cd740 0x7f9fc80c6970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:24.141 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.127+0000 7f9fcfbd6700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9fc80c5a30 0x7f9fc81585c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:24.141 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.127+0000 7f9fcfbd6700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fc815e110 con 0x7f9fc80c5a30 2023-11-08T19:25:24.141 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.127+0000 7f9fcfbd6700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fc815c940 con 0x7f9fc80cd740 2023-11-08T19:25:24.142 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.127+0000 7f9fcfbd6700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fc815cb70 con 0x7f9fb4003a30 2023-11-08T19:25:24.142 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.127+0000 7f9fce173700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9fc80c5a30 0x7f9fc81585c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:24.142 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.127+0000 7f9fce173700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9fc80c5a30 0x7f9fc81585c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49732/0 (socket says 172.21.15.143:49732) 2023-11-08T19:25:24.142 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.127+0000 7f9fce173700 1 -- 172.21.15.143:0/3803048862 learned_addr learned my addr 172.21.15.143:0/3803048862 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:24.142 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.127+0000 7f9fcd972700 1 --2- 172.21.15.143:0/3803048862 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9fb4003a30 0x7f9fc815dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:24.143 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.127+0000 7f9fcd171700 1 --2- 172.21.15.143:0/3803048862 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9fc80cd740 0x7f9fc80c6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:24.143 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.127+0000 7f9fce173700 1 -- 172.21.15.143:0/3803048862 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9fb4003a30 msgr2=0x7f9fc815dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:24.143 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.127+0000 7f9fce173700 1 --2- 172.21.15.143:0/3803048862 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9fb4003a30 0x7f9fc815dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:24.143 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.127+0000 7f9fce173700 1 -- 172.21.15.143:0/3803048862 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9fc80cd740 msgr2=0x7f9fc80c6970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:24.144 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.127+0000 7f9fce173700 1 --2- 172.21.15.143:0/3803048862 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9fc80cd740 0x7f9fc80c6970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:24.144 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.127+0000 7f9fce173700 1 -- 172.21.15.143:0/3803048862 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fc815ce10 con 0x7f9fc80c5a30 2023-11-08T19:25:24.144 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.127+0000 7f9fce173700 1 --2- 172.21.15.143:0/3803048862 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9fc80c5a30 0x7f9fc81585c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f9fc0000f80 tx=0x7f9fc000ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:24.144 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.127+0000 7f9fbe7fc700 1 -- 172.21.15.143:0/3803048862 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fc0003c10 con 0x7f9fc80c5a30 2023-11-08T19:25:24.144 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.127+0000 7f9fbe7fc700 1 -- 172.21.15.143:0/3803048862 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fc0003d90 con 0x7f9fc80c5a30 2023-11-08T19:25:24.145 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.127+0000 7f9fcfbd6700 1 -- 172.21.15.143:0/3803048862 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fc815d0f0 con 0x7f9fc80c5a30 2023-11-08T19:25:24.145 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.127+0000 7f9fbe7fc700 1 -- 172.21.15.143:0/3803048862 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fc0005310 con 0x7f9fc80c5a30 2023-11-08T19:25:24.145 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.127+0000 7f9fcfbd6700 1 -- 172.21.15.143:0/3803048862 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fc815d6b0 con 0x7f9fc80c5a30 2023-11-08T19:25:24.145 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.128+0000 7f9fcfbd6700 1 -- 172.21.15.143:0/3803048862 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fc804eee0 con 0x7f9fc80c5a30 2023-11-08T19:25:24.145 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.129+0000 7f9fbe7fc700 1 -- 172.21.15.143:0/3803048862 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9fc0005470 con 0x7f9fc80c5a30 2023-11-08T19:25:24.146 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.129+0000 7f9fbe7fc700 1 --2- 172.21.15.143:0/3803048862 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f9fac06af00 0x7f9fac06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:24.146 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.129+0000 7f9fbe7fc700 1 -- 172.21.15.143:0/3803048862 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9fc0089630 con 0x7f9fc80c5a30 2023-11-08T19:25:24.146 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.129+0000 7f9fcd972700 1 --2- 172.21.15.143:0/3803048862 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f9fac06af00 0x7f9fac06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:24.146 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.130+0000 7f9fcd972700 1 --2- 172.21.15.143:0/3803048862 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f9fac06af00 0x7f9fac06d3c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9fb8000c00 tx=0x7f9fb8021040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:24.146 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.133+0000 7f9fbe7fc700 1 -- 172.21.15.143:0/3803048862 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9fc00546e0 con 0x7f9fc80c5a30 2023-11-08T19:25:24.323 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.322+0000 7f9fcfbd6700 1 -- 172.21.15.143:0/3803048862 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9fc80b2ff0 con 0x7f9fc80c5a30 2023-11-08T19:25:24.325 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.323+0000 7f9fbe7fc700 1 -- 172.21.15.143:0/3803048862 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16553 (secure 0 0 0) 0x7f9fc0057d00 con 0x7f9fc80c5a30 2023-11-08T19:25:24.325 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:25:24.325 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":20,"fsid":"968483af-77fc-4068-ad7c-5c2a1a4613d7","created":"2023-11-08T19:24:32.826484+0000","modified":"2023-11-08T19:24:51.618072+0000","last_up_change":"2023-11-08T19:24:43.573880+0000","last_in_change":"2023-11-08T19:24:35.314525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:44.551530+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-08T19:24:48.256781+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-08T19:24:48.954990+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bae98ec1-d94e-403d-a220-a522a53ea6f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6809","nonce":1487830566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6811","nonce":1487830566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6815","nonce":1487830566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6813","nonce":1487830566}]},"public_addr":"172.21.15.143:6809/1487830566","cluster_addr":"172.21.15.143:6811/1487830566","heartbeat_back_addr":"172.21.15.143:6815/1487830566","heartbeat_front_addr":"172.21.15.143:6813/1487830566","state":["exists","up"]},{"osd":1,"uuid":"5cb6538a-1ccb-4322-8633-09f66af0b7e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6801","nonce":787341397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6803","nonce":787341397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6807","nonce":787341397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6805","nonce":787341397}]},"public_addr":"172.21.15.143:6801/787341397","cluster_addr":"172.21.15.143:6803/787341397","heartbeat_back_addr":"172.21.15.143:6807/787341397","heartbeat_front_addr":"172.21.15.143:6805/787341397","state":["exists","up"]},{"osd":2,"uuid":"cdbf522c-60ef-4d60-9653-8dcb529c101a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6822","nonce":2301325597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6826","nonce":2301325597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6831","nonce":2301325597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6829","nonce":2301325597}]},"public_addr":"172.21.15.143:6822/2301325597","cluster_addr":"172.21.15.143:6826/2301325597","heartbeat_back_addr":"172.21.15.143:6831/2301325597","heartbeat_front_addr":"172.21.15.143:6829/2301325597","state":["exists","up"]},{"osd":3,"uuid":"4e2783cf-ebb9-4dd8-8245-bc1d17ce4325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6817","nonce":2021504428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6819","nonce":2021504428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6827","nonce":2021504428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6823","nonce":2021504428}]},"public_addr":"172.21.15.143:6817/2021504428","cluster_addr":"172.21.15.143:6819/2021504428","heartbeat_back_addr":"172.21.15.143:6827/2021504428","heartbeat_front_addr":"172.21.15.143:6823/2021504428","state":["exists","up"]},{"osd":4,"uuid":"691256fa-9b77-49b9-a346-7ae1adf0ca80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6801","nonce":1993380595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6803","nonce":1993380595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6807","nonce":1993380595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6805","nonce":1993380595}]},"public_addr":"172.21.15.174:6801/1993380595","cluster_addr":"172.21.15.174:6803/1993380595","heartbeat_back_addr":"172.21.15.174:6807/1993380595","heartbeat_front_addr":"172.21.15.174:6805/1993380595","state":["exists","up"]},{"osd":5,"uuid":"756fc352-48c9-46ce-b52e-0541fe6f8525","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6809","nonce":1777923303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6811","nonce":1777923303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6815","nonce":1777923303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6813","nonce":1777923303}]},"public_addr":"172.21.15.174:6809/1777923303","cluster_addr":"172.21.15.174:6811/1777923303","heartbeat_back_addr":"172.21.15.174:6815/1777923303","heartbeat_front_addr":"172.21.15.174:6813/1777923303","state":["exists","up"]},{"osd":6,"uuid":"fab2ac2b-ed97-4cde-9825-b480eabf8b8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6825","nonce":2755130316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6827","nonce":2755130316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6831","nonce":2755130316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6829","nonce":2755130316}]},"public_addr":"172.21.15.174:6825/2755130316","cluster_addr":"172.21.15.174:6827/2755130316","heartbeat_back_addr":"172.21.15.174:6831/2755130316","heartbeat_front_addr":"172.21.15.174:6829/2755130316","state":["exists","up"]},{"osd":7,"uuid":"3de9b797-2ab6-4d41-a863-7e7d638fd703","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6817","nonce":2847141411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6819","nonce":2847141411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6823","nonce":2847141411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6821","nonce":2847141411}]},"public_addr":"172.21.15.174:6817/2847141411","cluster_addr":"172.21.15.174:6819/2847141411","heartbeat_back_addr":"172.21.15.174:6823/2847141411","heartbeat_front_addr":"172.21.15.174:6821/2847141411","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.477076+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.496890+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.539305+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.454673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.538881+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.521294+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.520712+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.540404+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:25:24.327 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.326+0000 7f9fcfbd6700 1 -- 172.21.15.143:0/3803048862 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f9fac06af00 msgr2=0x7f9fac06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:24.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.326+0000 7f9fcfbd6700 1 --2- 172.21.15.143:0/3803048862 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f9fac06af00 0x7f9fac06d3c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9fb8000c00 tx=0x7f9fb8021040 comp rx=0 tx=0).stop 2023-11-08T19:25:24.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.326+0000 7f9fcfbd6700 1 -- 172.21.15.143:0/3803048862 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9fc80c5a30 msgr2=0x7f9fc81585c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:24.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.326+0000 7f9fcfbd6700 1 --2- 172.21.15.143:0/3803048862 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9fc80c5a30 0x7f9fc81585c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f9fc0000f80 tx=0x7f9fc000ee00 comp rx=0 tx=0).stop 2023-11-08T19:25:24.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.326+0000 7f9fcfbd6700 1 -- 172.21.15.143:0/3803048862 shutdown_connections 2023-11-08T19:25:24.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.326+0000 7f9fcfbd6700 1 --2- 172.21.15.143:0/3803048862 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f9fac06af00 0x7f9fac06d3c0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:24.329 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.326+0000 7f9fcfbd6700 1 --2- 172.21.15.143:0/3803048862 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9fb4003a30 0x7f9fc815dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:24.329 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.326+0000 7f9fcfbd6700 1 --2- 172.21.15.143:0/3803048862 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9fc80c5a30 0x7f9fc81585c0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:24.329 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.326+0000 7f9fcfbd6700 1 --2- 172.21.15.143:0/3803048862 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9fc80cd740 0x7f9fc80c6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:24.329 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.326+0000 7f9fcfbd6700 1 -- 172.21.15.143:0/3803048862 >> 172.21.15.143:0/3803048862 conn(0x7f9fc8099ab0 msgr2=0x7f9fc80ae6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:24.329 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.326+0000 7f9fcfbd6700 1 -- 172.21.15.143:0/3803048862 shutdown_connections 2023-11-08T19:25:24.330 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.326+0000 7f9fcfbd6700 1 -- 172.21.15.143:0/3803048862 wait complete. 2023-11-08T19:25:24.342 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T19:24:44.551530+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-08T19:24:48.256781+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-08T19:24:48.954990+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-08T19:25:24.343 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T19:25:24.467 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.465+0000 7f2a83e6b700 1 Processor -- start 2023-11-08T19:25:24.467 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.466+0000 7f2a83e6b700 1 -- start start 2023-11-08T19:25:24.470 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.466+0000 7f2a83e6b700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f2a7c15dba0 0x7f2a7c06c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:24.470 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.466+0000 7f2a83e6b700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2a7c06b240 0x7f2a7c0caff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:24.470 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.466+0000 7f2a83e6b700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f2a7c0cb530 0x7f2a7c0b22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:24.471 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.466+0000 7f2a83e6b700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a7c161890 con 0x7f2a7c15dba0 2023-11-08T19:25:24.471 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.466+0000 7f2a83e6b700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a7c06b740 con 0x7f2a7c0cb530 2023-11-08T19:25:24.471 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.466+0000 7f2a83e6b700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a7c06ba20 con 0x7f2a7c06b240 2023-11-08T19:25:24.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.467+0000 7f2a81c07700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f2a7c15dba0 0x7f2a7c06c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:24.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.467+0000 7f2a81c07700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f2a7c15dba0 0x7f2a7c06c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49736/0 (socket says 172.21.15.143:49736) 2023-11-08T19:25:24.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.467+0000 7f2a81c07700 1 -- 172.21.15.143:0/967803889 learned_addr learned my addr 172.21.15.143:0/967803889 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:24.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.467+0000 7f2a81406700 1 --2- 172.21.15.143:0/967803889 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2a7c06b240 0x7f2a7c0caff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:24.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.467+0000 7f2a82408700 1 --2- 172.21.15.143:0/967803889 >> v2:172.21.15.174:3300/0 conn(0x7f2a7c0cb530 0x7f2a7c0b22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:24.473 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.467+0000 7f2a81c07700 1 -- 172.21.15.143:0/967803889 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2a7c06b240 msgr2=0x7f2a7c0caff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:24.473 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.467+0000 7f2a81c07700 1 --2- 172.21.15.143:0/967803889 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2a7c06b240 0x7f2a7c0caff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:24.473 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.467+0000 7f2a81c07700 1 -- 172.21.15.143:0/967803889 >> v2:172.21.15.174:3300/0 conn(0x7f2a7c0cb530 msgr2=0x7f2a7c0b22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:24.473 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.467+0000 7f2a81c07700 1 --2- 172.21.15.143:0/967803889 >> v2:172.21.15.174:3300/0 conn(0x7f2a7c0cb530 0x7f2a7c0b22e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:24.473 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.467+0000 7f2a81c07700 1 -- 172.21.15.143:0/967803889 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a7c06bd00 con 0x7f2a7c15dba0 2023-11-08T19:25:24.474 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.467+0000 7f2a81406700 1 --2- 172.21.15.143:0/967803889 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2a7c06b240 0x7f2a7c0caff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:25:24.474 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.467+0000 7f2a81c07700 1 --2- 172.21.15.143:0/967803889 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2a7c15dba0 0x7f2a7c06c610 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c06f460 tx=0x7f2a6c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0d02e10e3c36a98 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:24.474 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.467+0000 7f2a80c05700 1 -- 172.21.15.143:0/967803889 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a6c00b870 con 0x7f2a7c15dba0 2023-11-08T19:25:24.474 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.467+0000 7f2a80c05700 1 -- 172.21.15.143:0/967803889 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a6c00b9d0 con 0x7f2a7c15dba0 2023-11-08T19:25:24.474 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.467+0000 7f2a80c05700 1 -- 172.21.15.143:0/967803889 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a6c00bd60 con 0x7f2a7c15dba0 2023-11-08T19:25:24.475 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.468+0000 7f2a83e6b700 1 -- 172.21.15.143:0/967803889 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2a7c15dba0 msgr2=0x7f2a7c06c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:24.475 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.468+0000 7f2a83e6b700 1 --2- 172.21.15.143:0/967803889 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2a7c15dba0 0x7f2a7c06c610 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c06f460 tx=0x7f2a6c009bd0 comp rx=0 tx=0).stop 2023-11-08T19:25:24.475 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.468+0000 7f2a83e6b700 1 -- 172.21.15.143:0/967803889 shutdown_connections 2023-11-08T19:25:24.475 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.468+0000 7f2a83e6b700 1 --2- 172.21.15.143:0/967803889 >> v2:172.21.15.174:3300/0 conn(0x7f2a7c0cb530 0x7f2a7c0b22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:24.475 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.468+0000 7f2a83e6b700 1 --2- 172.21.15.143:0/967803889 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2a7c15dba0 0x7f2a7c06c610 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:24.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.468+0000 7f2a83e6b700 1 --2- 172.21.15.143:0/967803889 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2a7c06b240 0x7f2a7c0caff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:24.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.468+0000 7f2a83e6b700 1 -- 172.21.15.143:0/967803889 >> 172.21.15.143:0/967803889 conn(0x7f2a7c0cd2e0 msgr2=0x7f2a7c0cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:24.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.468+0000 7f2a83e6b700 1 -- 172.21.15.143:0/967803889 shutdown_connections 2023-11-08T19:25:24.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.468+0000 7f2a83e6b700 1 -- 172.21.15.143:0/967803889 wait complete. 2023-11-08T19:25:24.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.469+0000 7f2a83e6b700 1 Processor -- start 2023-11-08T19:25:24.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.469+0000 7f2a83e6b700 1 -- start start 2023-11-08T19:25:24.479 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.470+0000 7f2a83e6b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a7c06b240 0x7f2a7c15a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:24.479 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.470+0000 7f2a83e6b700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2a7c0cb530 0x7f2a7c0b3480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:24.480 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.470+0000 7f2a83e6b700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2a7c15dba0 0x7f2a7c0b39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:24.480 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.470+0000 7f2a83e6b700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a7c1f7830 con 0x7f2a7c0cb530 2023-11-08T19:25:24.480 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.470+0000 7f2a83e6b700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a7c0ae6b0 con 0x7f2a7c06b240 2023-11-08T19:25:24.480 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.470+0000 7f2a83e6b700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a7c1f7190 con 0x7f2a7c15dba0 2023-11-08T19:25:24.480 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.470+0000 7f2a81406700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2a7c0cb530 0x7f2a7c0b3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:24.481 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.470+0000 7f2a81406700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2a7c0cb530 0x7f2a7c0b3480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49744/0 (socket says 172.21.15.143:49744) 2023-11-08T19:25:24.481 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.470+0000 7f2a81406700 1 -- 172.21.15.143:0/1318223939 learned_addr learned my addr 172.21.15.143:0/1318223939 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:24.481 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.470+0000 7f2a81c07700 1 --2- 172.21.15.143:0/1318223939 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a7c06b240 0x7f2a7c15a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:24.481 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.470+0000 7f2a82408700 1 --2- 172.21.15.143:0/1318223939 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2a7c15dba0 0x7f2a7c0b39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:24.481 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.470+0000 7f2a81406700 1 -- 172.21.15.143:0/1318223939 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2a7c15dba0 msgr2=0x7f2a7c0b39c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:24.482 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.470+0000 7f2a81406700 1 --2- 172.21.15.143:0/1318223939 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2a7c15dba0 0x7f2a7c0b39c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:24.482 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.470+0000 7f2a81406700 1 -- 172.21.15.143:0/1318223939 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a7c06b240 msgr2=0x7f2a7c15a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:24.482 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.470+0000 7f2a81406700 1 --2- 172.21.15.143:0/1318223939 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a7c06b240 0x7f2a7c15a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:24.482 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.470+0000 7f2a81406700 1 -- 172.21.15.143:0/1318223939 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a6c009550 con 0x7f2a7c0cb530 2023-11-08T19:25:24.483 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.470+0000 7f2a81406700 1 --2- 172.21.15.143:0/1318223939 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2a7c0cb530 0x7f2a7c0b3480 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f2a7800c740 tx=0x7f2a7800ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:24.483 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.470+0000 7f2a727fc700 1 -- 172.21.15.143:0/1318223939 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a78011800 con 0x7f2a7c0cb530 2023-11-08T19:25:24.483 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.470+0000 7f2a727fc700 1 -- 172.21.15.143:0/1318223939 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a78011960 con 0x7f2a7c0cb530 2023-11-08T19:25:24.483 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.470+0000 7f2a83e6b700 1 -- 172.21.15.143:0/1318223939 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a7c1f47c0 con 0x7f2a7c0cb530 2023-11-08T19:25:24.483 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.470+0000 7f2a727fc700 1 -- 172.21.15.143:0/1318223939 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a78011cd0 con 0x7f2a7c0cb530 2023-11-08T19:25:24.483 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.470+0000 7f2a83e6b700 1 -- 172.21.15.143:0/1318223939 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a7c15a980 con 0x7f2a7c0cb530 2023-11-08T19:25:24.484 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.471+0000 7f2a83e6b700 1 -- 172.21.15.143:0/1318223939 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a7c15aba0 con 0x7f2a7c0cb530 2023-11-08T19:25:24.484 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.472+0000 7f2a727fc700 1 -- 172.21.15.143:0/1318223939 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2a78021350 con 0x7f2a7c0cb530 2023-11-08T19:25:24.484 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.472+0000 7f2a727fc700 1 --2- 172.21.15.143:0/1318223939 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f2a6006af00 0x7f2a6006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:24.484 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.472+0000 7f2a727fc700 1 -- 172.21.15.143:0/1318223939 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2a78089320 con 0x7f2a7c0cb530 2023-11-08T19:25:24.484 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.472+0000 7f2a81c07700 1 --2- 172.21.15.143:0/1318223939 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f2a6006af00 0x7f2a6006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:24.485 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.473+0000 7f2a81c07700 1 --2- 172.21.15.143:0/1318223939 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f2a6006af00 0x7f2a6006d3c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c014040 tx=0x7f2a6c008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:24.485 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.477+0000 7f2a727fc700 1 -- 172.21.15.143:0/1318223939 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2a78054460 con 0x7f2a7c0cb530 2023-11-08T19:25:24.669 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.668+0000 7f2a83e6b700 1 -- 172.21.15.143:0/1318223939 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2a7c04eee0 con 0x7f2a7c0cb530 2023-11-08T19:25:24.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.668+0000 7f2a727fc700 1 -- 172.21.15.143:0/1318223939 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2a78057a80 con 0x7f2a7c0cb530 2023-11-08T19:25:24.670 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2023-11-08T19:25:24.672 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.671+0000 7f2a83e6b700 1 -- 172.21.15.143:0/1318223939 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f2a6006af00 msgr2=0x7f2a6006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:24.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.671+0000 7f2a83e6b700 1 --2- 172.21.15.143:0/1318223939 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f2a6006af00 0x7f2a6006d3c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c014040 tx=0x7f2a6c008030 comp rx=0 tx=0).stop 2023-11-08T19:25:24.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.671+0000 7f2a83e6b700 1 -- 172.21.15.143:0/1318223939 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2a7c0cb530 msgr2=0x7f2a7c0b3480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:24.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.671+0000 7f2a83e6b700 1 --2- 172.21.15.143:0/1318223939 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2a7c0cb530 0x7f2a7c0b3480 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f2a7800c740 tx=0x7f2a7800ca50 comp rx=0 tx=0).stop 2023-11-08T19:25:24.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.671+0000 7f2a83e6b700 1 -- 172.21.15.143:0/1318223939 shutdown_connections 2023-11-08T19:25:24.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.671+0000 7f2a83e6b700 1 --2- 172.21.15.143:0/1318223939 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f2a6006af00 0x7f2a6006d3c0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:24.674 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.671+0000 7f2a83e6b700 1 --2- 172.21.15.143:0/1318223939 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2a7c15dba0 0x7f2a7c0b39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:24.674 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.671+0000 7f2a83e6b700 1 --2- 172.21.15.143:0/1318223939 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a7c06b240 0x7f2a7c15a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:24.674 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.671+0000 7f2a83e6b700 1 --2- 172.21.15.143:0/1318223939 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2a7c0cb530 0x7f2a7c0b3480 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:24.674 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.671+0000 7f2a83e6b700 1 -- 172.21.15.143:0/1318223939 >> 172.21.15.143:0/1318223939 conn(0x7f2a7c0cd2e0 msgr2=0x7f2a7c15cbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:24.674 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.671+0000 7f2a83e6b700 1 -- 172.21.15.143:0/1318223939 shutdown_connections 2023-11-08T19:25:24.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.671+0000 7f2a83e6b700 1 -- 172.21.15.143:0/1318223939 wait complete. 2023-11-08T19:25:24.684 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-08T19:25:24.816 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.807+0000 7f24a21c6700 1 Processor -- start 2023-11-08T19:25:24.816 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.807+0000 7f24a21c6700 1 -- start start 2023-11-08T19:25:24.817 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.807+0000 7f24a21c6700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f249c15dba0 0x7f249c06c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:24.817 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.808+0000 7f24a21c6700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f249c161890 con 0x7f249c0b22e0 2023-11-08T19:25:24.817 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.808+0000 7f24a21c6700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f249c06b740 con 0x7f249c06b240 2023-11-08T19:25:24.817 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.808+0000 7f24a21c6700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f249c06ba20 con 0x7f249c15dba0 2023-11-08T19:25:24.817 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.808+0000 7f249b7fe700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f249c15dba0 0x7f249c06c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:24.818 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.808+0000 7f249bfff700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f249c0b22e0 0x7f249c0b2700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:45904/0 (socket says 172.21.15.143:45904) 2023-11-08T19:25:24.818 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.808+0000 7f249bfff700 1 -- 172.21.15.143:0/670878610 learned_addr learned my addr 172.21.15.143:0/670878610 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:25:24.818 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.808+0000 7f249b7fe700 1 -- 172.21.15.143:0/670878610 >> v1:172.21.15.174:6789/0 conn(0x7f249c06b240 legacy=0x7f249c0caff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:24.818 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.808+0000 7f249b7fe700 1 -- 172.21.15.143:0/670878610 >> v1:172.21.15.143:6789/0 conn(0x7f249c0b22e0 legacy=0x7f249c0b2700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:24.819 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.808+0000 7f249a7fc700 1 -- 172.21.15.143:0/670878610 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2370489965 0 0) 0x7f249c161890 con 0x7f249c0b22e0 2023-11-08T19:25:24.819 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.808+0000 7f249b7fe700 1 -- 172.21.15.143:0/670878610 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f249c06bd00 con 0x7f249c15dba0 2023-11-08T19:25:24.819 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.808+0000 7f249b7fe700 1 --2- 172.21.15.143:0/670878610 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f249c15dba0 0x7f249c06c610 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f248c009840 tx=0x7f248c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f440521f84b9c34 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:24.819 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.808+0000 7f249a7fc700 1 -- 172.21.15.143:0/670878610 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f248c004230 con 0x7f249c15dba0 2023-11-08T19:25:24.819 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.808+0000 7f249a7fc700 1 -- 172.21.15.143:0/670878610 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f248c004390 con 0x7f249c15dba0 2023-11-08T19:25:24.820 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.808+0000 7f249a7fc700 1 -- 172.21.15.143:0/670878610 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f248c0036a0 con 0x7f249c15dba0 2023-11-08T19:25:24.820 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.809+0000 7f24a21c6700 1 -- 172.21.15.143:0/670878610 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f249c15dba0 msgr2=0x7f249c06c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:24.820 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.809+0000 7f24a21c6700 1 --2- 172.21.15.143:0/670878610 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f249c15dba0 0x7f249c06c610 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f248c009840 tx=0x7f248c009b50 comp rx=0 tx=0).stop 2023-11-08T19:25:24.820 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.809+0000 7f24a21c6700 1 -- 172.21.15.143:0/670878610 shutdown_connections 2023-11-08T19:25:24.820 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.809+0000 7f24a21c6700 1 --2- 172.21.15.143:0/670878610 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f249c15dba0 0x7f249c06c610 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:24.820 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.809+0000 7f24a21c6700 1 -- 172.21.15.143:0/670878610 >> 172.21.15.143:0/670878610 conn(0x7f249c0cd2e0 msgr2=0x7f249c0cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:24.821 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.809+0000 7f24a21c6700 1 -- 172.21.15.143:0/670878610 shutdown_connections 2023-11-08T19:25:24.821 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.809+0000 7f24a21c6700 1 -- 172.21.15.143:0/670878610 wait complete. 2023-11-08T19:25:24.821 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.810+0000 7f24a21c6700 1 Processor -- start 2023-11-08T19:25:24.821 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.810+0000 7f24a21c6700 1 -- start start 2023-11-08T19:25:24.821 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.811+0000 7f24a21c6700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f249c06b240 0x7f249c0ca5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:24.822 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.811+0000 7f24a21c6700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f249c0b22e0 0x7f249c0c5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:24.822 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.811+0000 7f24a21c6700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f249c15dba0 0x7f249c1e6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:24.822 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.811+0000 7f24a21c6700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f249c0b4b70 con 0x7f249c06b240 2023-11-08T19:25:24.822 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.811+0000 7f24a21c6700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f249c0b55c0 con 0x7f249c15dba0 2023-11-08T19:25:24.823 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.811+0000 7f24a21c6700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f249c0b58a0 con 0x7f249c0b22e0 2023-11-08T19:25:24.823 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.811+0000 7f249b7fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f249c06b240 0x7f249c0ca5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:24.823 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.811+0000 7f249b7fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f249c06b240 0x7f249c0ca5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49756/0 (socket says 172.21.15.143:49756) 2023-11-08T19:25:24.823 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.811+0000 7f249b7fe700 1 -- 172.21.15.143:0/3485541221 learned_addr learned my addr 172.21.15.143:0/3485541221 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:24.823 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.811+0000 7f249affd700 1 --2- 172.21.15.143:0/3485541221 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f249c0b22e0 0x7f249c0c5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:24.823 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.811+0000 7f249bfff700 1 --2- 172.21.15.143:0/3485541221 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f249c15dba0 0x7f249c1e6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:24.824 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.811+0000 7f249b7fe700 1 -- 172.21.15.143:0/3485541221 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f249c0b22e0 msgr2=0x7f249c0c5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:24.824 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.811+0000 7f249b7fe700 1 --2- 172.21.15.143:0/3485541221 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f249c0b22e0 0x7f249c0c5500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:24.824 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.811+0000 7f249b7fe700 1 -- 172.21.15.143:0/3485541221 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f249c15dba0 msgr2=0x7f249c1e6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:24.825 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.811+0000 7f249b7fe700 1 --2- 172.21.15.143:0/3485541221 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f249c15dba0 0x7f249c1e6ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:24.825 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.811+0000 7f249b7fe700 1 -- 172.21.15.143:0/3485541221 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f248c0094a0 con 0x7f249c06b240 2023-11-08T19:25:24.825 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.811+0000 7f249bfff700 1 --2- 172.21.15.143:0/3485541221 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f249c15dba0 0x7f249c1e6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:25:24.825 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.812+0000 7f249b7fe700 1 --2- 172.21.15.143:0/3485541221 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f249c06b240 0x7f249c0ca5b0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f248c0090b0 tx=0x7f248c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:24.825 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.812+0000 7f2483fff700 1 -- 172.21.15.143:0/3485541221 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f248c011830 con 0x7f249c06b240 2023-11-08T19:25:24.826 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.812+0000 7f2483fff700 1 -- 172.21.15.143:0/3485541221 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f248c011990 con 0x7f249c06b240 2023-11-08T19:25:24.826 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.812+0000 7f2483fff700 1 -- 172.21.15.143:0/3485541221 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f248c011d40 con 0x7f249c06b240 2023-11-08T19:25:24.826 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.812+0000 7f24a21c6700 1 -- 172.21.15.143:0/3485541221 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f249c0b5b80 con 0x7f249c06b240 2023-11-08T19:25:24.826 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.812+0000 7f24a21c6700 1 -- 172.21.15.143:0/3485541221 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f249c0b6140 con 0x7f249c06b240 2023-11-08T19:25:24.827 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.813+0000 7f24a21c6700 1 -- 172.21.15.143:0/3485541221 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f249c04eee0 con 0x7f249c06b240 2023-11-08T19:25:24.827 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.813+0000 7f2483fff700 1 -- 172.21.15.143:0/3485541221 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f248c01b440 con 0x7f249c06b240 2023-11-08T19:25:24.827 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.814+0000 7f2483fff700 1 --2- 172.21.15.143:0/3485541221 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f248806af00 0x7f248806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:24.827 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.814+0000 7f2483fff700 1 -- 172.21.15.143:0/3485541221 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f248c08ae60 con 0x7f249c06b240 2023-11-08T19:25:24.827 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.814+0000 7f249affd700 1 --2- 172.21.15.143:0/3485541221 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f248806af00 0x7f248806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:24.828 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.814+0000 7f249affd700 1 --2- 172.21.15.143:0/3485541221 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f248806af00 0x7f248806d3c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f249c0b4f00 tx=0x7f2490009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:24.828 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:24.818+0000 7f2483fff700 1 -- 172.21.15.143:0/3485541221 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f248c055fa0 con 0x7f249c06b240 2023-11-08T19:25:25.007 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.006+0000 7f24a21c6700 1 -- 172.21.15.143:0/3485541221 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f249c161610 con 0x7f249c06b240 2023-11-08T19:25:25.008 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.006+0000 7f2483fff700 1 -- 172.21.15.143:0/3485541221 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f248c0595c0 con 0x7f249c06b240 2023-11-08T19:25:25.008 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2023-11-08T19:25:25.010 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.008+0000 7f24a21c6700 1 -- 172.21.15.143:0/3485541221 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f248806af00 msgr2=0x7f248806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:25.011 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.008+0000 7f24a21c6700 1 --2- 172.21.15.143:0/3485541221 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f248806af00 0x7f248806d3c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f249c0b4f00 tx=0x7f2490009040 comp rx=0 tx=0).stop 2023-11-08T19:25:25.011 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.008+0000 7f24a21c6700 1 -- 172.21.15.143:0/3485541221 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f249c06b240 msgr2=0x7f249c0ca5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:25.011 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.008+0000 7f24a21c6700 1 --2- 172.21.15.143:0/3485541221 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f249c06b240 0x7f249c0ca5b0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f248c0090b0 tx=0x7f248c0047c0 comp rx=0 tx=0).stop 2023-11-08T19:25:25.011 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.008+0000 7f24a21c6700 1 -- 172.21.15.143:0/3485541221 shutdown_connections 2023-11-08T19:25:25.012 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.008+0000 7f24a21c6700 1 --2- 172.21.15.143:0/3485541221 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f248806af00 0x7f248806d3c0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:25.012 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.008+0000 7f24a21c6700 1 --2- 172.21.15.143:0/3485541221 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f249c15dba0 0x7f249c1e6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:25.012 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.008+0000 7f24a21c6700 1 --2- 172.21.15.143:0/3485541221 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f249c06b240 0x7f249c0ca5b0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:25.012 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.008+0000 7f24a21c6700 1 --2- 172.21.15.143:0/3485541221 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f249c0b22e0 0x7f249c0c5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:25.012 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.008+0000 7f24a21c6700 1 -- 172.21.15.143:0/3485541221 >> 172.21.15.143:0/3485541221 conn(0x7f249c0cd2e0 msgr2=0x7f249c0b10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:25.013 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.008+0000 7f24a21c6700 1 -- 172.21.15.143:0/3485541221 shutdown_connections 2023-11-08T19:25:25.013 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.008+0000 7f24a21c6700 1 -- 172.21.15.143:0/3485541221 wait complete. 2023-11-08T19:25:25.022 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-08T19:25:25.146 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.145+0000 7fee0afac700 1 Processor -- start 2023-11-08T19:25:25.147 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.145+0000 7fee0afac700 1 -- start start 2023-11-08T19:25:25.150 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.146+0000 7fee0afac700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fee0406b240 0x7fee040caff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:25.150 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.146+0000 7fee0afac700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee04161890 con 0x7fee0415dba0 2023-11-08T19:25:25.150 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.146+0000 7fee0afac700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee0406b740 con 0x7fee040cb530 2023-11-08T19:25:25.150 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.146+0000 7fee0afac700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee0406cd20 con 0x7fee0406b240 2023-11-08T19:25:25.151 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.146+0000 7fee08d48700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fee0415dba0 0x7fee0406c610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:45918/0 (socket says 172.21.15.143:45918) 2023-11-08T19:25:25.151 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.146+0000 7fee08d48700 1 -- 172.21.15.143:0/41314035 learned_addr learned my addr 172.21.15.143:0/41314035 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:25:25.151 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.146+0000 7fee03fff700 1 --2- 172.21.15.143:0/41314035 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fee0406b240 0x7fee040caff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:25.151 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.146+0000 7fee037fe700 1 -- 172.21.15.143:0/41314035 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2765469643 0 0) 0x7fee04161890 con 0x7fee0415dba0 2023-11-08T19:25:25.152 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.146+0000 7fee037fe700 1 -- 172.21.15.143:0/41314035 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fedf40035b0 con 0x7fee0415dba0 2023-11-08T19:25:25.152 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.146+0000 7fee037fe700 1 -- 172.21.15.143:0/41314035 <== mon.1 v1:172.21.15.174:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1843911888 0 0) 0x7fee0406b740 con 0x7fee040cb530 2023-11-08T19:25:25.152 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.146+0000 7fee037fe700 1 -- 172.21.15.143:0/41314035 --> v1:172.21.15.174:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee04161890 con 0x7fee040cb530 2023-11-08T19:25:25.152 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.146+0000 7fee037fe700 1 -- 172.21.15.143:0/41314035 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1466555540 0 0) 0x7fedf40035b0 con 0x7fee0415dba0 2023-11-08T19:25:25.152 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.146+0000 7fee03fff700 1 -- 172.21.15.143:0/41314035 >> v1:172.21.15.174:6789/0 conn(0x7fee040cb530 legacy=0x7fee040b22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:25.153 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.146+0000 7fee03fff700 1 -- 172.21.15.143:0/41314035 >> v1:172.21.15.143:6789/0 conn(0x7fee0415dba0 legacy=0x7fee0406c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:25.153 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.146+0000 7fee03fff700 1 -- 172.21.15.143:0/41314035 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee0406ba20 con 0x7fee0406b240 2023-11-08T19:25:25.153 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.147+0000 7fee03fff700 1 --2- 172.21.15.143:0/41314035 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fee0406b240 0x7fee040caff0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fedf0009870 tx=0x7fedf0009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=98a9f2569824b8a4 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:25.153 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.147+0000 7fee037fe700 1 -- 172.21.15.143:0/41314035 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedf8001670 con 0x7fee0406b240 2023-11-08T19:25:25.154 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.147+0000 7fee037fe700 1 -- 172.21.15.143:0/41314035 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedf0004030 con 0x7fee0406b240 2023-11-08T19:25:25.154 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.147+0000 7fee037fe700 1 -- 172.21.15.143:0/41314035 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedf00043a0 con 0x7fee0406b240 2023-11-08T19:25:25.154 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.147+0000 7fee0afac700 1 -- 172.21.15.143:0/41314035 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fee0406b240 msgr2=0x7fee040caff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:25.154 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.147+0000 7fee0afac700 1 --2- 172.21.15.143:0/41314035 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fee0406b240 0x7fee040caff0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fedf0009870 tx=0x7fedf0009b80 comp rx=0 tx=0).stop 2023-11-08T19:25:25.155 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.147+0000 7fee0afac700 1 -- 172.21.15.143:0/41314035 shutdown_connections 2023-11-08T19:25:25.155 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.147+0000 7fee0afac700 1 --2- 172.21.15.143:0/41314035 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fee0406b240 0x7fee040caff0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:25.155 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.147+0000 7fee0afac700 1 -- 172.21.15.143:0/41314035 >> 172.21.15.143:0/41314035 conn(0x7fee040cd2e0 msgr2=0x7fee040cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:25.155 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.147+0000 7fee0afac700 1 -- 172.21.15.143:0/41314035 shutdown_connections 2023-11-08T19:25:25.155 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.147+0000 7fee0afac700 1 -- 172.21.15.143:0/41314035 wait complete. 2023-11-08T19:25:25.156 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.149+0000 7fee0afac700 1 Processor -- start 2023-11-08T19:25:25.156 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.149+0000 7fee0afac700 1 -- start start 2023-11-08T19:25:25.159 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.149+0000 7fee0afac700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fee0406b240 0x7fee04160b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:25.159 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.149+0000 7fee0afac700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fee040cb530 0x7fee040c8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:25.159 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.149+0000 7fee0afac700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fee0415dba0 0x7fee0415b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:25.159 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.149+0000 7fee0afac700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee040c9dc0 con 0x7fee0406b240 2023-11-08T19:25:25.160 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.149+0000 7fee0afac700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee040ca1e0 con 0x7fee040cb530 2023-11-08T19:25:25.160 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.149+0000 7fee0afac700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee040c9140 con 0x7fee0415dba0 2023-11-08T19:25:25.160 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.150+0000 7fee08d48700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fee0406b240 0x7fee04160b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:25.160 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.150+0000 7fee08d48700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fee0406b240 0x7fee04160b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49770/0 (socket says 172.21.15.143:49770) 2023-11-08T19:25:25.161 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.150+0000 7fee08d48700 1 -- 172.21.15.143:0/367861068 learned_addr learned my addr 172.21.15.143:0/367861068 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:25.161 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.150+0000 7fee08d48700 1 -- 172.21.15.143:0/367861068 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fee0415dba0 msgr2=0x7fee0415b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:25.161 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.150+0000 7fee09549700 1 --2- 172.21.15.143:0/367861068 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fee0415dba0 0x7fee0415b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:25.161 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.150+0000 7fee08d48700 1 --2- 172.21.15.143:0/367861068 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fee0415dba0 0x7fee0415b030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:25.161 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.150+0000 7fee08d48700 1 -- 172.21.15.143:0/367861068 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fee040cb530 msgr2=0x7fee040c8ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:25.162 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.150+0000 7fee08d48700 1 --2- 172.21.15.143:0/367861068 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fee040cb530 0x7fee040c8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:25.162 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.150+0000 7fee08d48700 1 -- 172.21.15.143:0/367861068 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedf00094a0 con 0x7fee0406b240 2023-11-08T19:25:25.162 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.150+0000 7fee08d48700 1 --2- 172.21.15.143:0/367861068 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fee0406b240 0x7fee04160b40 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fedf8001eb0 tx=0x7fedf800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:25.162 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.150+0000 7fee017fa700 1 -- 172.21.15.143:0/367861068 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedf8004c80 con 0x7fee0406b240 2023-11-08T19:25:25.163 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.150+0000 7fee017fa700 1 -- 172.21.15.143:0/367861068 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedf8004de0 con 0x7fee0406b240 2023-11-08T19:25:25.163 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.150+0000 7fee0afac700 1 -- 172.21.15.143:0/367861068 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee040c9420 con 0x7fee0406b240 2023-11-08T19:25:25.163 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.150+0000 7fee017fa700 1 -- 172.21.15.143:0/367861068 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedf80072d0 con 0x7fee0406b240 2023-11-08T19:25:25.163 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.150+0000 7fee0afac700 1 -- 172.21.15.143:0/367861068 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee040c99e0 con 0x7fee0406b240 2023-11-08T19:25:25.163 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.151+0000 7fee0afac700 1 -- 172.21.15.143:0/367861068 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee0404eee0 con 0x7fee0406b240 2023-11-08T19:25:25.164 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.152+0000 7fee017fa700 1 -- 172.21.15.143:0/367861068 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fedf8007430 con 0x7fee0406b240 2023-11-08T19:25:25.164 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.152+0000 7fee017fa700 1 --2- 172.21.15.143:0/367861068 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fede806af00 0x7fede806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:25.164 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.152+0000 7fee017fa700 1 -- 172.21.15.143:0/367861068 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fedf8089420 con 0x7fee0406b240 2023-11-08T19:25:25.164 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.152+0000 7fee03fff700 1 --2- 172.21.15.143:0/367861068 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fede806af00 0x7fede806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:25.165 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.153+0000 7fee03fff700 1 --2- 172.21.15.143:0/367861068 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fede806af00 0x7fede806d3c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fee041e7b40 tx=0x7fedf0010000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:25.165 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.156+0000 7fee017fa700 1 -- 172.21.15.143:0/367861068 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fedf8054450 con 0x7fee0406b240 2023-11-08T19:25:25.345 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.343+0000 7fee0afac700 1 -- 172.21.15.143:0/367861068 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fee041e8930 con 0x7fee0406b240 2023-11-08T19:25:25.345 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.344+0000 7fee017fa700 1 -- 172.21.15.143:0/367861068 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fedf8057a70 con 0x7fee0406b240 2023-11-08T19:25:25.346 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 64 2023-11-08T19:25:25.348 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.346+0000 7fee0afac700 1 -- 172.21.15.143:0/367861068 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fede806af00 msgr2=0x7fede806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:25.348 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.346+0000 7fee0afac700 1 --2- 172.21.15.143:0/367861068 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fede806af00 0x7fede806d3c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fee041e7b40 tx=0x7fedf0010000 comp rx=0 tx=0).stop 2023-11-08T19:25:25.348 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.346+0000 7fee0afac700 1 -- 172.21.15.143:0/367861068 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fee0406b240 msgr2=0x7fee04160b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:25.349 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.346+0000 7fee0afac700 1 --2- 172.21.15.143:0/367861068 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fee0406b240 0x7fee04160b40 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fedf8001eb0 tx=0x7fedf800c8b0 comp rx=0 tx=0).stop 2023-11-08T19:25:25.349 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.346+0000 7fee0afac700 1 -- 172.21.15.143:0/367861068 shutdown_connections 2023-11-08T19:25:25.349 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.346+0000 7fee0afac700 1 --2- 172.21.15.143:0/367861068 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fede806af00 0x7fede806d3c0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:25.349 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.346+0000 7fee0afac700 1 --2- 172.21.15.143:0/367861068 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fee0415dba0 0x7fee0415b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:25.349 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.346+0000 7fee0afac700 1 --2- 172.21.15.143:0/367861068 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fee0406b240 0x7fee04160b40 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:25.350 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.346+0000 7fee0afac700 1 --2- 172.21.15.143:0/367861068 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fee040cb530 0x7fee040c8ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:25.350 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.346+0000 7fee0afac700 1 -- 172.21.15.143:0/367861068 >> 172.21.15.143:0/367861068 conn(0x7fee040cd2e0 msgr2=0x7fee040b27b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:25.350 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.346+0000 7fee0afac700 1 -- 172.21.15.143:0/367861068 shutdown_connections 2023-11-08T19:25:25.350 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.346+0000 7fee0afac700 1 -- 172.21.15.143:0/367861068 wait complete. 2023-11-08T19:25:25.360 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snapshots.TestMonSnapsAndFsPools.test_attaching_pools_with_snaps_to_fs_fails' 2023-11-08T19:25:25.481 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.479+0000 7f1ab71ca700 1 Processor -- start 2023-11-08T19:25:25.481 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.479+0000 7f1ab71ca700 1 -- start start 2023-11-08T19:25:25.485 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.480+0000 7f1ab71ca700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f1ab015dd00 0x7f1ab00cca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:25.485 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.480+0000 7f1ab71ca700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f1ab006b320 0x7f1ab00cb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:25.485 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.480+0000 7f1ab71ca700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ab00ccfb0 con 0x7f1ab01618f0 2023-11-08T19:25:25.486 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.480+0000 7f1ab71ca700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ab00cad50 con 0x7f1ab015dd00 2023-11-08T19:25:25.486 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.480+0000 7f1ab71ca700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ab0161df0 con 0x7f1ab006b320 2023-11-08T19:25:25.486 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.480+0000 7f1ab4f66700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f1ab015dd00 0x7f1ab00cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:25.486 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.480+0000 7f1aaffff700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f1ab01618f0 0x7f1ab006ac10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:45932/0 (socket says 172.21.15.143:45932) 2023-11-08T19:25:25.486 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.480+0000 7f1aaffff700 1 -- 172.21.15.143:0/3116223853 learned_addr learned my addr 172.21.15.143:0/3116223853 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:25:25.487 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.480+0000 7f1ab5767700 1 --2- 172.21.15.143:0/3116223853 >> v2:172.21.15.174:3300/0 conn(0x7f1ab006b320 0x7f1ab00cb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:25.487 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.480+0000 7f1ab4f66700 1 -- 172.21.15.143:0/3116223853 >> v2:172.21.15.174:3300/0 conn(0x7f1ab006b320 msgr2=0x7f1ab00cb1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:25.487 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.480+0000 7f1ab4f66700 1 --2- 172.21.15.143:0/3116223853 >> v2:172.21.15.174:3300/0 conn(0x7f1ab006b320 0x7f1ab00cb1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:25.487 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.480+0000 7f1ab4f66700 1 -- 172.21.15.143:0/3116223853 >> v1:172.21.15.143:6789/0 conn(0x7f1ab01618f0 legacy=0x7f1ab006ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:25.488 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.480+0000 7f1aaf7fe700 1 -- 172.21.15.143:0/3116223853 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1562470711 0 0) 0x7f1ab00ccfb0 con 0x7f1ab01618f0 2023-11-08T19:25:25.488 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.480+0000 7f1ab4f66700 1 -- 172.21.15.143:0/3116223853 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ab01620d0 con 0x7f1ab015dd00 2023-11-08T19:25:25.488 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.480+0000 7f1ab5767700 1 --2- 172.21.15.143:0/3116223853 >> v2:172.21.15.174:3300/0 conn(0x7f1ab006b320 0x7f1ab00cb1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:25:25.488 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.481+0000 7f1ab4f66700 1 --2- 172.21.15.143:0/3116223853 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1ab015dd00 0x7f1ab00cca70 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f1ab00ce8d0 tx=0x7f1aa4009b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67bcbb54106bd4ef server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:25.489 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.481+0000 7f1aaf7fe700 1 -- 172.21.15.143:0/3116223853 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aa4005e30 con 0x7f1ab015dd00 2023-11-08T19:25:25.489 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.481+0000 7f1aaf7fe700 1 -- 172.21.15.143:0/3116223853 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1aa4006030 con 0x7f1ab015dd00 2023-11-08T19:25:25.489 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.481+0000 7f1aaf7fe700 1 -- 172.21.15.143:0/3116223853 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aa40063c0 con 0x7f1ab015dd00 2023-11-08T19:25:25.489 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.481+0000 7f1ab71ca700 1 -- 172.21.15.143:0/3116223853 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1ab015dd00 msgr2=0x7f1ab00cca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:25.489 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.481+0000 7f1ab71ca700 1 --2- 172.21.15.143:0/3116223853 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1ab015dd00 0x7f1ab00cca70 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f1ab00ce8d0 tx=0x7f1aa4009b00 comp rx=0 tx=0).stop 2023-11-08T19:25:25.490 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.481+0000 7f1ab71ca700 1 -- 172.21.15.143:0/3116223853 shutdown_connections 2023-11-08T19:25:25.490 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.481+0000 7f1ab71ca700 1 --2- 172.21.15.143:0/3116223853 >> v2:172.21.15.174:3300/0 conn(0x7f1ab006b320 0x7f1ab00cb1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:25.490 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.481+0000 7f1ab71ca700 1 --2- 172.21.15.143:0/3116223853 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1ab015dd00 0x7f1ab00cca70 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:25.491 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.481+0000 7f1ab71ca700 1 -- 172.21.15.143:0/3116223853 >> 172.21.15.143:0/3116223853 conn(0x7f1ab0099ab0 msgr2=0x7f1ab00d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:25.491 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.481+0000 7f1ab71ca700 1 -- 172.21.15.143:0/3116223853 shutdown_connections 2023-11-08T19:25:25.491 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.481+0000 7f1ab71ca700 1 -- 172.21.15.143:0/3116223853 wait complete. 2023-11-08T19:25:25.491 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.483+0000 7f1ab71ca700 1 Processor -- start 2023-11-08T19:25:25.491 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.483+0000 7f1ab71ca700 1 -- start start 2023-11-08T19:25:25.495 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.484+0000 7f1ab71ca700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1ab006b320 0x7f1ab00b5b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:25.495 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.484+0000 7f1ab71ca700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1ab015dd00 0x7f1ab01545f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:25.495 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.484+0000 7f1ab71ca700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1ab01618f0 0x7f1ab00c8f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:25.495 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.484+0000 7f1ab71ca700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ab00c67b0 con 0x7f1ab015dd00 2023-11-08T19:25:25.495 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.484+0000 7f1ab71ca700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ab0155220 con 0x7f1ab01618f0 2023-11-08T19:25:25.496 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.484+0000 7f1ab71ca700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ab00ca4b0 con 0x7f1ab006b320 2023-11-08T19:25:25.496 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.484+0000 7f1aaffff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1ab015dd00 0x7f1ab01545f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:25.496 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.484+0000 7f1aaffff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1ab015dd00 0x7f1ab01545f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49784/0 (socket says 172.21.15.143:49784) 2023-11-08T19:25:25.496 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.484+0000 7f1aaffff700 1 -- 172.21.15.143:0/2981196311 learned_addr learned my addr 172.21.15.143:0/2981196311 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:25.497 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.484+0000 7f1ab4f66700 1 --2- 172.21.15.143:0/2981196311 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1ab006b320 0x7f1ab00b5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:25.497 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.484+0000 7f1ab5767700 1 --2- 172.21.15.143:0/2981196311 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1ab01618f0 0x7f1ab00c8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:25.497 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.484+0000 7f1aaffff700 1 -- 172.21.15.143:0/2981196311 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1ab006b320 msgr2=0x7f1ab00b5b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:25.497 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.484+0000 7f1aaffff700 1 --2- 172.21.15.143:0/2981196311 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1ab006b320 0x7f1ab00b5b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:25.497 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.484+0000 7f1aaffff700 1 -- 172.21.15.143:0/2981196311 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1ab01618f0 msgr2=0x7f1ab00c8f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:25.498 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.484+0000 7f1aaffff700 1 --2- 172.21.15.143:0/2981196311 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1ab01618f0 0x7f1ab00c8f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:25.498 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.484+0000 7f1aaffff700 1 -- 172.21.15.143:0/2981196311 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1aa4009430 con 0x7f1ab015dd00 2023-11-08T19:25:25.498 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.484+0000 7f1ab5767700 1 --2- 172.21.15.143:0/2981196311 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1ab01618f0 0x7f1ab00c8f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:25:25.498 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.485+0000 7f1aaffff700 1 --2- 172.21.15.143:0/2981196311 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1ab015dd00 0x7f1ab01545f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c00c550 tx=0x7f1a9c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:25.498 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.485+0000 7f1aad7fa700 1 -- 172.21.15.143:0/2981196311 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a9c004c80 con 0x7f1ab015dd00 2023-11-08T19:25:25.499 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.485+0000 7f1aad7fa700 1 -- 172.21.15.143:0/2981196311 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a9c004e00 con 0x7f1ab015dd00 2023-11-08T19:25:25.499 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.485+0000 7f1aad7fa700 1 -- 172.21.15.143:0/2981196311 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a9c007310 con 0x7f1ab015dd00 2023-11-08T19:25:25.499 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.485+0000 7f1ab71ca700 1 -- 172.21.15.143:0/2981196311 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ab00c61f0 con 0x7f1ab015dd00 2023-11-08T19:25:25.499 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.485+0000 7f1ab71ca700 1 -- 172.21.15.143:0/2981196311 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ab00c7970 con 0x7f1ab015dd00 2023-11-08T19:25:25.500 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.486+0000 7f1ab71ca700 1 -- 172.21.15.143:0/2981196311 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ab00aff60 con 0x7f1ab015dd00 2023-11-08T19:25:25.500 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.486+0000 7f1aad7fa700 1 -- 172.21.15.143:0/2981196311 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1a9c007470 con 0x7f1ab015dd00 2023-11-08T19:25:25.500 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.487+0000 7f1aad7fa700 1 --2- 172.21.15.143:0/2981196311 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f1a9406af00 0x7f1a9406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:25.500 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.487+0000 7f1aad7fa700 1 -- 172.21.15.143:0/2981196311 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1a9c089200 con 0x7f1ab015dd00 2023-11-08T19:25:25.501 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.487+0000 7f1ab4f66700 1 --2- 172.21.15.143:0/2981196311 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f1a9406af00 0x7f1a9406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:25.501 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.487+0000 7f1ab4f66700 1 --2- 172.21.15.143:0/2981196311 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f1a9406af00 0x7f1a9406d3c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f1aa4009b40 tx=0x7f1aa4008070 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:25.501 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.491+0000 7f1aad7fa700 1 -- 172.21.15.143:0/2981196311 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1a9c054340 con 0x7f1ab015dd00 2023-11-08T19:25:25.759 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:25.758+0000 7f1ab71ca700 1 -- 172.21.15.143:0/2981196311 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snapshots.TestMonSnapsAndFsPools.test_attaching_pools_with_snaps_to_fs_fails"]} v 0) v1 -- 0x7f1ab00c77d0 con 0x7f1ab015dd00 2023-11-08T19:25:26.579 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:26.578+0000 7f1aad7fa700 1 -- 172.21.15.143:0/2981196311 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snapshots.TestMonSnapsAndFsPools.test_attaching_pools_with_snaps_to_fs_fails"]}]=0 v51) v1 ==== 176+0+0 (secure 0 0 0) 0x7f1ab00c77d0 con 0x7f1ab015dd00 2023-11-08T19:25:26.582 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:26.580+0000 7f1ab71ca700 1 -- 172.21.15.143:0/2981196311 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f1a9406af00 msgr2=0x7f1a9406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:26.582 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:26.580+0000 7f1ab71ca700 1 --2- 172.21.15.143:0/2981196311 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f1a9406af00 0x7f1a9406d3c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f1aa4009b40 tx=0x7f1aa4008070 comp rx=0 tx=0).stop 2023-11-08T19:25:26.583 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:26.580+0000 7f1ab71ca700 1 -- 172.21.15.143:0/2981196311 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1ab015dd00 msgr2=0x7f1ab01545f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:26.583 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:26.580+0000 7f1ab71ca700 1 --2- 172.21.15.143:0/2981196311 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1ab015dd00 0x7f1ab01545f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c00c550 tx=0x7f1a9c00c910 comp rx=0 tx=0).stop 2023-11-08T19:25:26.583 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:26.581+0000 7f1ab71ca700 1 -- 172.21.15.143:0/2981196311 shutdown_connections 2023-11-08T19:25:26.583 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:26.581+0000 7f1ab71ca700 1 --2- 172.21.15.143:0/2981196311 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f1a9406af00 0x7f1a9406d3c0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:26.583 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:26.581+0000 7f1ab71ca700 1 --2- 172.21.15.143:0/2981196311 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1ab006b320 0x7f1ab00b5b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:26.584 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:26.581+0000 7f1ab71ca700 1 --2- 172.21.15.143:0/2981196311 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1ab01618f0 0x7f1ab00c8f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:26.584 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:26.581+0000 7f1ab71ca700 1 --2- 172.21.15.143:0/2981196311 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1ab015dd00 0x7f1ab01545f0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:26.584 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:26.581+0000 7f1ab71ca700 1 -- 172.21.15.143:0/2981196311 >> 172.21.15.143:0/2981196311 conn(0x7f1ab0099ab0 msgr2=0x7f1ab006f330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:26.584 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:26.581+0000 7f1ab71ca700 1 -- 172.21.15.143:0/2981196311 shutdown_connections 2023-11-08T19:25:26.585 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:26.581+0000 7f1ab71ca700 1 -- 172.21.15.143:0/2981196311 wait complete. 2023-11-08T19:25:26.597 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-08T19:25:26.603 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T19:25:26.645 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T19:25:26.730 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:26.728+0000 7f98d461a700 1 Processor -- start 2023-11-08T19:25:26.730 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:26.728+0000 7f98d461a700 1 -- start start 2023-11-08T19:25:26.733 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:26.729+0000 7f98d461a700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f98cc06bd20 0x7f98cc0d19c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:26.733 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:26.729+0000 7f98d461a700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98cc06c2c0 con 0x7f98cc0d1f00 2023-11-08T19:25:26.733 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:26.729+0000 7f98d461a700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98cc06c5a0 con 0x7f98cc15df20 2023-11-08T19:25:26.733 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:26.729+0000 7f98d461a700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98cc0d2390 con 0x7f98cc06bd20 2023-11-08T19:25:26.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:26.729+0000 7f98d1bb5700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f98cc06bd20 0x7f98cc0d19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:26.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:26.730+0000 7f98d2bb7700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f98cc0d1f00 0x7f98cc0cfd30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:45948/0 (socket says 172.21.15.143:45948) 2023-11-08T19:25:26.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:26.730+0000 7f98d2bb7700 1 -- 172.21.15.143:0/1372834295 learned_addr learned my addr 172.21.15.143:0/1372834295 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:25:26.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:26.730+0000 7f98d1bb5700 1 -- 172.21.15.143:0/1372834295 >> v1:172.21.15.174:6789/0 conn(0x7f98cc15df20 legacy=0x7f98cc0b25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:26.735 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:26.730+0000 7f98d1bb5700 1 -- 172.21.15.143:0/1372834295 >> v1:172.21.15.143:6789/0 conn(0x7f98cc0d1f00 legacy=0x7f98cc0cfd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:26.735 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:26.730+0000 7f98d1bb5700 1 -- 172.21.15.143:0/1372834295 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98cc06c880 con 0x7f98cc06bd20 2023-11-08T19:25:26.735 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:26.730+0000 7f98d13b4700 1 -- 172.21.15.143:0/1372834295 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1325925121 0 0) 0x7f98cc06c2c0 con 0x7f98cc0d1f00 2023-11-08T19:25:26.735 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:26.730+0000 7f98d1bb5700 1 --2- 172.21.15.143:0/1372834295 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98cc06bd20 0x7f98cc0d19c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f98c8009840 tx=0x7f98c8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2007641e0a5eeed3 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:26.736 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:26.730+0000 7f98d13b4700 1 -- 172.21.15.143:0/1372834295 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c8004230 con 0x7f98cc06bd20 2023-11-08T19:25:26.736 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:26.730+0000 7f98d13b4700 1 -- 172.21.15.143:0/1372834295 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98c8004390 con 0x7f98cc06bd20 2023-11-08T19:25:26.736 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:26.730+0000 7f98d13b4700 1 -- 172.21.15.143:0/1372834295 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c80036a0 con 0x7f98cc06bd20 2023-11-08T19:25:26.736 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:26.731+0000 7f98d461a700 1 -- 172.21.15.143:0/1372834295 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98cc06bd20 msgr2=0x7f98cc0d19c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:26.737 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:26.731+0000 7f98d461a700 1 --2- 172.21.15.143:0/1372834295 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98cc06bd20 0x7f98cc0d19c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f98c8009840 tx=0x7f98c8009b50 comp rx=0 tx=0).stop 2023-11-08T19:25:26.737 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:26.731+0000 7f98d461a700 1 -- 172.21.15.143:0/1372834295 shutdown_connections 2023-11-08T19:25:26.737 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:26.731+0000 7f98d461a700 1 --2- 172.21.15.143:0/1372834295 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98cc06bd20 0x7f98cc0d19c0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:26.737 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:26.731+0000 7f98d461a700 1 -- 172.21.15.143:0/1372834295 >> 172.21.15.143:0/1372834295 conn(0x7f98cc08b000 msgr2=0x7f98cc09ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:26.737 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:26.731+0000 7f98d461a700 1 -- 172.21.15.143:0/1372834295 shutdown_connections 2023-11-08T19:25:26.738 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:26.731+0000 7f98d461a700 1 -- 172.21.15.143:0/1372834295 wait complete. 2023-11-08T19:25:26.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:26.732+0000 7f98d461a700 1 Processor -- start 2023-11-08T19:25:26.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:26.732+0000 7f98d461a700 1 -- start start 2023-11-08T19:25:26.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:26.733+0000 7f98d461a700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98cc06bd20 0x7f98cc1f2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:26.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:26.733+0000 7f98d461a700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f98cc0d1f00 0x7f98cc15f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:26.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:26.733+0000 7f98d461a700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f98cc15df20 0x7f98cc0c9510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:26.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:26.733+0000 7f98d461a700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98cc15e3a0 con 0x7f98cc06bd20 2023-11-08T19:25:26.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:26.733+0000 7f98d461a700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98cc15e680 con 0x7f98cc15df20 2023-11-08T19:25:26.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:26.733+0000 7f98d461a700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98cc15e960 con 0x7f98cc0d1f00 2023-11-08T19:25:26.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:26.733+0000 7f98d23b6700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98cc06bd20 0x7f98cc1f2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:26.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:26.733+0000 7f98d23b6700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98cc06bd20 0x7f98cc1f2a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49796/0 (socket says 172.21.15.143:49796) 2023-11-08T19:25:26.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:26.733+0000 7f98d23b6700 1 -- 172.21.15.143:0/510699690 learned_addr learned my addr 172.21.15.143:0/510699690 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:26.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:26.733+0000 7f98d2bb7700 1 --2- 172.21.15.143:0/510699690 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f98cc15df20 0x7f98cc0c9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:26.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:26.733+0000 7f98d1bb5700 1 --2- 172.21.15.143:0/510699690 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f98cc0d1f00 0x7f98cc15f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:26.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:26.733+0000 7f98d23b6700 1 -- 172.21.15.143:0/510699690 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f98cc0d1f00 msgr2=0x7f98cc15f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:26.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:26.733+0000 7f98d23b6700 1 --2- 172.21.15.143:0/510699690 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f98cc0d1f00 0x7f98cc15f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:26.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:26.733+0000 7f98d23b6700 1 -- 172.21.15.143:0/510699690 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f98cc15df20 msgr2=0x7f98cc0c9510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:26.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:26.733+0000 7f98d23b6700 1 --2- 172.21.15.143:0/510699690 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f98cc15df20 0x7f98cc0c9510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:26.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:26.733+0000 7f98d23b6700 1 -- 172.21.15.143:0/510699690 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98c80094a0 con 0x7f98cc06bd20 2023-11-08T19:25:26.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:26.733+0000 7f98d2bb7700 1 --2- 172.21.15.143:0/510699690 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f98cc15df20 0x7f98cc0c9510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:25:26.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:26.733+0000 7f98d1bb5700 1 --2- 172.21.15.143:0/510699690 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f98cc0d1f00 0x7f98cc15f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:25:26.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:26.733+0000 7f98d23b6700 1 --2- 172.21.15.143:0/510699690 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98cc06bd20 0x7f98cc1f2a70 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f98bc00b4a0 tx=0x7f98bc00b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:26.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:26.733+0000 7f98c2ffd700 1 -- 172.21.15.143:0/510699690 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98bc011830 con 0x7f98cc06bd20 2023-11-08T19:25:26.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:26.733+0000 7f98c2ffd700 1 -- 172.21.15.143:0/510699690 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98bc011990 con 0x7f98cc06bd20 2023-11-08T19:25:26.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:26.733+0000 7f98c2ffd700 1 -- 172.21.15.143:0/510699690 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98bc011d40 con 0x7f98cc06bd20 2023-11-08T19:25:26.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:26.733+0000 7f98d461a700 1 -- 172.21.15.143:0/510699690 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98cc15ec40 con 0x7f98cc06bd20 2023-11-08T19:25:26.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:26.733+0000 7f98d461a700 1 -- 172.21.15.143:0/510699690 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98cc151ff0 con 0x7f98cc06bd20 2023-11-08T19:25:26.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:26.734+0000 7f98d461a700 1 -- 172.21.15.143:0/510699690 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98cc15f160 con 0x7f98cc06bd20 2023-11-08T19:25:26.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:26.735+0000 7f98c2ffd700 1 -- 172.21.15.143:0/510699690 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f98bc01a400 con 0x7f98cc06bd20 2023-11-08T19:25:26.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:26.735+0000 7f98c2ffd700 1 --2- 172.21.15.143:0/510699690 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f98b006af00 0x7f98b006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:26.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:26.735+0000 7f98c2ffd700 1 -- 172.21.15.143:0/510699690 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f98bc089320 con 0x7f98cc06bd20 2023-11-08T19:25:26.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:26.736+0000 7f98d1bb5700 1 --2- 172.21.15.143:0/510699690 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f98b006af00 0x7f98b006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:26.750 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:26.736+0000 7f98d1bb5700 1 --2- 172.21.15.143:0/510699690 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f98b006af00 0x7f98b006d3c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f98c8000c00 tx=0x7f98c801c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:26.750 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:26.740+0000 7f98c2ffd700 1 -- 172.21.15.143:0/510699690 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f98cc15f160 con 0x7f98cc06bd20 2023-11-08T19:25:26.931 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:26.929+0000 7f98d461a700 1 -- 172.21.15.143:0/510699690 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f98cc04eee0 con 0x7f98cc06bd20 2023-11-08T19:25:26.934 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:26.933+0000 7f98c2ffd700 1 -- 172.21.15.143:0/510699690 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v11)=0 v11) v1 ==== 122+0+0 (secure 0 0 0) 0x7f98bc0543d0 con 0x7f98cc06bd20 2023-11-08T19:25:26.936 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:26.935+0000 7f98d461a700 1 -- 172.21.15.143:0/510699690 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f98b006af00 msgr2=0x7f98b006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:26.937 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:26.935+0000 7f98d461a700 1 --2- 172.21.15.143:0/510699690 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f98b006af00 0x7f98b006d3c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f98c8000c00 tx=0x7f98c801c040 comp rx=0 tx=0).stop 2023-11-08T19:25:26.937 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:26.935+0000 7f98d461a700 1 -- 172.21.15.143:0/510699690 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98cc06bd20 msgr2=0x7f98cc1f2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:26.937 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:26.935+0000 7f98d461a700 1 --2- 172.21.15.143:0/510699690 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98cc06bd20 0x7f98cc1f2a70 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f98bc00b4a0 tx=0x7f98bc00b860 comp rx=0 tx=0).stop 2023-11-08T19:25:26.938 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:26.935+0000 7f98d461a700 1 -- 172.21.15.143:0/510699690 shutdown_connections 2023-11-08T19:25:26.938 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:26.935+0000 7f98d461a700 1 --2- 172.21.15.143:0/510699690 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f98b006af00 0x7f98b006d3c0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:26.938 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:26.935+0000 7f98d461a700 1 --2- 172.21.15.143:0/510699690 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f98cc15df20 0x7f98cc0c9510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:26.938 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:26.935+0000 7f98d461a700 1 --2- 172.21.15.143:0/510699690 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98cc06bd20 0x7f98cc1f2a70 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:26.938 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:26.935+0000 7f98d461a700 1 --2- 172.21.15.143:0/510699690 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f98cc0d1f00 0x7f98cc15f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:26.939 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:26.935+0000 7f98d461a700 1 -- 172.21.15.143:0/510699690 >> 172.21.15.143:0/510699690 conn(0x7f98cc08b000 msgr2=0x7f98cc0b25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:26.939 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:26.935+0000 7f98d461a700 1 -- 172.21.15.143:0/510699690 shutdown_connections 2023-11-08T19:25:26.939 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:26.935+0000 7f98d461a700 1 -- 172.21.15.143:0/510699690 wait complete. 2023-11-08T19:25:26.948 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-08T19:25:26.949 DEBUG:teuthology.orchestra.run.smithi143:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-08T19:25:26.991 DEBUG:teuthology.orchestra.run.smithi174:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-08T19:25:27.046 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-08T19:25:27.047 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-08T19:25:27.048 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:25:27.066 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-08T19:25:27.067 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:25:27.125 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-08T19:25:27.126 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:25:27.184 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:25:27.344 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.342+0000 7f281c08a700 1 Processor -- start 2023-11-08T19:25:27.344 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.343+0000 7f281c08a700 1 -- start start 2023-11-08T19:25:27.353 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.343+0000 7f281c08a700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f281406c730 0x7f28140cd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:27.353 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.343+0000 7f281c08a700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f281415de80 0x7f28140cbf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:27.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.343+0000 7f281c08a700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28140cd480 con 0x7f281406c030 2023-11-08T19:25:27.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.343+0000 7f281c08a700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28140cc870 con 0x7f281406c730 2023-11-08T19:25:27.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.343+0000 7f281c08a700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28140ccb50 con 0x7f281415de80 2023-11-08T19:25:27.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.343+0000 7f2819625700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f281406c030 0x7f281406cb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:45958/0 (socket says 172.21.15.143:45958) 2023-11-08T19:25:27.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.343+0000 7f2819625700 1 -- 172.21.15.143:0/3108750966 learned_addr learned my addr 172.21.15.143:0/3108750966 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:25:27.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.344+0000 7f2818e24700 1 -- 172.21.15.143:0/3108750966 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3352746481 0 0) 0x7f28140cd480 con 0x7f281406c030 2023-11-08T19:25:27.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.344+0000 7f2818e24700 1 -- 172.21.15.143:0/3108750966 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f28000035b0 con 0x7f281406c030 2023-11-08T19:25:27.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.344+0000 7f2819e26700 1 --2- 172.21.15.143:0/3108750966 >> v2:172.21.15.174:3300/0 conn(0x7f281406c730 0x7f28140cd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:27.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.344+0000 7f281a627700 1 --2- 172.21.15.143:0/3108750966 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f281415de80 0x7f28140cbf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:27.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.344+0000 7f2818e24700 1 -- 172.21.15.143:0/3108750966 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2836412455 0 0) 0x7f28000035b0 con 0x7f281406c030 2023-11-08T19:25:27.356 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.344+0000 7f2818e24700 1 -- 172.21.15.143:0/3108750966 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f281415de80 msgr2=0x7f28140cbf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:27.356 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.344+0000 7f2818e24700 1 --2- 172.21.15.143:0/3108750966 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f281415de80 0x7f28140cbf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:27.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.344+0000 7f2818e24700 1 -- 172.21.15.143:0/3108750966 >> v2:172.21.15.174:3300/0 conn(0x7f281406c730 msgr2=0x7f28140cd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:27.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.344+0000 7f2818e24700 1 --2- 172.21.15.143:0/3108750966 >> v2:172.21.15.174:3300/0 conn(0x7f281406c730 0x7f28140cd910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:27.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.344+0000 7f2818e24700 1 -- 172.21.15.143:0/3108750966 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28140cce30 con 0x7f281406c030 2023-11-08T19:25:27.358 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.344+0000 7f2819e26700 1 --2- 172.21.15.143:0/3108750966 >> v2:172.21.15.174:3300/0 conn(0x7f281406c730 0x7f28140cd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:25:27.358 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.344+0000 7f2818e24700 1 -- 172.21.15.143:0/3108750966 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4147756303 0 0) 0x7f2810001670 con 0x7f281406c030 2023-11-08T19:25:27.358 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.344+0000 7f2818e24700 1 -- 172.21.15.143:0/3108750966 >> v1:172.21.15.143:6789/0 conn(0x7f281406c030 legacy=0x7f281406cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:27.358 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.344+0000 7f2818e24700 1 --2- 172.21.15.143:0/3108750966 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2800003a00 0x7f2800003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:27.358 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.344+0000 7f2818e24700 1 --2- 172.21.15.143:0/3108750966 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f281415de80 0x7f2800004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:27.359 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.344+0000 7f2818e24700 1 --2- 172.21.15.143:0/3108750966 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2800004d90 0x7f2800005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:27.359 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.344+0000 7f2818e24700 1 -- 172.21.15.143:0/3108750966 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28140cc870 con 0x7f2800004d90 2023-11-08T19:25:27.359 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.344+0000 7f2818e24700 1 -- 172.21.15.143:0/3108750966 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28140ccb50 con 0x7f2800003a00 2023-11-08T19:25:27.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.344+0000 7f2818e24700 1 -- 172.21.15.143:0/3108750966 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f280000baf0 con 0x7f281415de80 2023-11-08T19:25:27.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.344+0000 7f281a627700 1 --2- 172.21.15.143:0/3108750966 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2800004d90 0x7f2800005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:27.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.344+0000 7f2819625700 1 --2- 172.21.15.143:0/3108750966 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f281415de80 0x7f2800004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:27.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.344+0000 7f2819e26700 1 --2- 172.21.15.143:0/3108750966 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2800003a00 0x7f2800003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:27.361 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.344+0000 7f281a627700 1 -- 172.21.15.143:0/3108750966 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f281415de80 msgr2=0x7f2800004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:27.361 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.344+0000 7f281a627700 1 --2- 172.21.15.143:0/3108750966 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f281415de80 0x7f2800004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:27.362 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.344+0000 7f281a627700 1 -- 172.21.15.143:0/3108750966 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2800003a00 msgr2=0x7f2800003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:27.362 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.344+0000 7f281a627700 1 --2- 172.21.15.143:0/3108750966 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2800003a00 0x7f2800003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:27.362 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.345+0000 7f281a627700 1 -- 172.21.15.143:0/3108750966 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28000035b0 con 0x7f2800004d90 2023-11-08T19:25:27.363 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.345+0000 7f281a627700 1 --2- 172.21.15.143:0/3108750966 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2800004d90 0x7f2800005210 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f281406ef90 tx=0x7f2800005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7e6ab00b231bb1e server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:27.363 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.345+0000 7f2818e24700 1 -- 172.21.15.143:0/3108750966 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f280c007dc0 con 0x7f2800004d90 2023-11-08T19:25:27.364 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.345+0000 7f2818e24700 1 -- 172.21.15.143:0/3108750966 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f280c003a20 con 0x7f2800004d90 2023-11-08T19:25:27.364 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.345+0000 7f2818e24700 1 -- 172.21.15.143:0/3108750966 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f280c003dd0 con 0x7f2800004d90 2023-11-08T19:25:27.364 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.345+0000 7f281c08a700 1 -- 172.21.15.143:0/3108750966 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2800004d90 msgr2=0x7f2800005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:27.365 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.345+0000 7f281c08a700 1 --2- 172.21.15.143:0/3108750966 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2800004d90 0x7f2800005210 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f281406ef90 tx=0x7f2800005880 comp rx=0 tx=0).stop 2023-11-08T19:25:27.365 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.345+0000 7f281a627700 1 -- 172.21.15.143:0/3108750966 reap_dead start 2023-11-08T19:25:27.365 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.345+0000 7f281c08a700 1 -- 172.21.15.143:0/3108750966 shutdown_connections 2023-11-08T19:25:27.365 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.345+0000 7f281c08a700 1 -- 172.21.15.143:0/3108750966 >> 172.21.15.143:0/3108750966 conn(0x7f281408b000 msgr2=0x7f281409ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:27.365 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.345+0000 7f281c08a700 1 -- 172.21.15.143:0/3108750966 shutdown_connections 2023-11-08T19:25:27.366 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.345+0000 7f281c08a700 1 -- 172.21.15.143:0/3108750966 wait complete. 2023-11-08T19:25:27.366 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.347+0000 7f281c08a700 1 Processor -- start 2023-11-08T19:25:27.366 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.347+0000 7f281c08a700 1 -- start start 2023-11-08T19:25:27.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.347+0000 7f281c08a700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2800004d90 0x7f281415b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:27.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.347+0000 7f281c08a700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f281415e260 0x7f28140c9d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:27.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.347+0000 7f281c08a700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f281414f990 0x7f281414fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:27.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.347+0000 7f281c08a700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f281415f360 con 0x7f281414f990 2023-11-08T19:25:27.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.347+0000 7f281c08a700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2814150330 con 0x7f281415e260 2023-11-08T19:25:27.368 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.347+0000 7f281c08a700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f281415e750 con 0x7f2800004d90 2023-11-08T19:25:27.368 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.347+0000 7f281a627700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f281414f990 0x7f281414fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:27.368 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.348+0000 7f281a627700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f281414f990 0x7f281414fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49818/0 (socket says 172.21.15.143:49818) 2023-11-08T19:25:27.369 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.348+0000 7f281a627700 1 -- 172.21.15.143:0/1669325251 learned_addr learned my addr 172.21.15.143:0/1669325251 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:27.369 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.348+0000 7f2819625700 1 --2- 172.21.15.143:0/1669325251 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f281415e260 0x7f28140c9d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:27.369 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.348+0000 7f2819e26700 1 --2- 172.21.15.143:0/1669325251 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2800004d90 0x7f281415b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:27.370 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.348+0000 7f281a627700 1 -- 172.21.15.143:0/1669325251 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2800004d90 msgr2=0x7f281415b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:27.370 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.348+0000 7f281a627700 1 --2- 172.21.15.143:0/1669325251 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2800004d90 0x7f281415b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:27.370 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.348+0000 7f281a627700 1 -- 172.21.15.143:0/1669325251 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f281415e260 msgr2=0x7f28140c9d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:27.370 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.348+0000 7f281a627700 1 --2- 172.21.15.143:0/1669325251 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f281415e260 0x7f28140c9d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:27.371 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.348+0000 7f281a627700 1 -- 172.21.15.143:0/1669325251 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f281415ea30 con 0x7f281414f990 2023-11-08T19:25:27.371 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.348+0000 7f281a627700 1 --2- 172.21.15.143:0/1669325251 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f281414f990 0x7f281414fdf0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f280c00afd0 tx=0x7f280c004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:27.371 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.348+0000 7f280a7fc700 1 -- 172.21.15.143:0/1669325251 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f280c004240 con 0x7f281414f990 2023-11-08T19:25:27.371 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.348+0000 7f280a7fc700 1 -- 172.21.15.143:0/1669325251 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f280c0043a0 con 0x7f281414f990 2023-11-08T19:25:27.372 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.348+0000 7f281c08a700 1 -- 172.21.15.143:0/1669325251 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f281415ed10 con 0x7f281414f990 2023-11-08T19:25:27.372 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.348+0000 7f280a7fc700 1 -- 172.21.15.143:0/1669325251 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f280c004790 con 0x7f281414f990 2023-11-08T19:25:27.372 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.348+0000 7f281c08a700 1 -- 172.21.15.143:0/1669325251 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f281415a0f0 con 0x7f281414f990 2023-11-08T19:25:27.372 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.349+0000 7f281c08a700 1 -- 172.21.15.143:0/1669325251 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f281415cb90 con 0x7f281414f990 2023-11-08T19:25:27.373 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.349+0000 7f280a7fc700 1 -- 172.21.15.143:0/1669325251 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f280c01f420 con 0x7f281414f990 2023-11-08T19:25:27.373 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.350+0000 7f280a7fc700 1 --2- 172.21.15.143:0/1669325251 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f27f806af00 0x7f27f806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:27.373 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.350+0000 7f280a7fc700 1 -- 172.21.15.143:0/1669325251 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f280c08abe0 con 0x7f281414f990 2023-11-08T19:25:27.373 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.350+0000 7f2819e26700 1 --2- 172.21.15.143:0/1669325251 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f27f806af00 0x7f27f806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:27.373 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.350+0000 7f2819e26700 1 --2- 172.21.15.143:0/1669325251 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f27f806af00 0x7f27f806d3c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2804009870 tx=0x7f2804008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:27.374 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.354+0000 7f280a7fc700 1 -- 172.21.15.143:0/1669325251 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f280c055d20 con 0x7f281414f990 2023-11-08T19:25:27.544 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.542+0000 7f281c08a700 1 -- 172.21.15.143:0/1669325251 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f281404eee0 con 0x7f281414f990 2023-11-08T19:25:27.545 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.544+0000 7f280a7fc700 1 -- 172.21.15.143:0/1669325251 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16553 (secure 0 0 0) 0x7f280c023090 con 0x7f281414f990 2023-11-08T19:25:27.546 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:25:27.546 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":20,"fsid":"968483af-77fc-4068-ad7c-5c2a1a4613d7","created":"2023-11-08T19:24:32.826484+0000","modified":"2023-11-08T19:24:51.618072+0000","last_up_change":"2023-11-08T19:24:43.573880+0000","last_in_change":"2023-11-08T19:24:35.314525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:44.551530+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-08T19:24:48.256781+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-08T19:24:48.954990+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bae98ec1-d94e-403d-a220-a522a53ea6f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6809","nonce":1487830566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6811","nonce":1487830566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6815","nonce":1487830566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6813","nonce":1487830566}]},"public_addr":"172.21.15.143:6809/1487830566","cluster_addr":"172.21.15.143:6811/1487830566","heartbeat_back_addr":"172.21.15.143:6815/1487830566","heartbeat_front_addr":"172.21.15.143:6813/1487830566","state":["exists","up"]},{"osd":1,"uuid":"5cb6538a-1ccb-4322-8633-09f66af0b7e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6801","nonce":787341397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6803","nonce":787341397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6807","nonce":787341397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6805","nonce":787341397}]},"public_addr":"172.21.15.143:6801/787341397","cluster_addr":"172.21.15.143:6803/787341397","heartbeat_back_addr":"172.21.15.143:6807/787341397","heartbeat_front_addr":"172.21.15.143:6805/787341397","state":["exists","up"]},{"osd":2,"uuid":"cdbf522c-60ef-4d60-9653-8dcb529c101a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6822","nonce":2301325597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6826","nonce":2301325597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6831","nonce":2301325597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6829","nonce":2301325597}]},"public_addr":"172.21.15.143:6822/2301325597","cluster_addr":"172.21.15.143:6826/2301325597","heartbeat_back_addr":"172.21.15.143:6831/2301325597","heartbeat_front_addr":"172.21.15.143:6829/2301325597","state":["exists","up"]},{"osd":3,"uuid":"4e2783cf-ebb9-4dd8-8245-bc1d17ce4325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6817","nonce":2021504428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6819","nonce":2021504428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6827","nonce":2021504428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6823","nonce":2021504428}]},"public_addr":"172.21.15.143:6817/2021504428","cluster_addr":"172.21.15.143:6819/2021504428","heartbeat_back_addr":"172.21.15.143:6827/2021504428","heartbeat_front_addr":"172.21.15.143:6823/2021504428","state":["exists","up"]},{"osd":4,"uuid":"691256fa-9b77-49b9-a346-7ae1adf0ca80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6801","nonce":1993380595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6803","nonce":1993380595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6807","nonce":1993380595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6805","nonce":1993380595}]},"public_addr":"172.21.15.174:6801/1993380595","cluster_addr":"172.21.15.174:6803/1993380595","heartbeat_back_addr":"172.21.15.174:6807/1993380595","heartbeat_front_addr":"172.21.15.174:6805/1993380595","state":["exists","up"]},{"osd":5,"uuid":"756fc352-48c9-46ce-b52e-0541fe6f8525","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6809","nonce":1777923303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6811","nonce":1777923303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6815","nonce":1777923303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6813","nonce":1777923303}]},"public_addr":"172.21.15.174:6809/1777923303","cluster_addr":"172.21.15.174:6811/1777923303","heartbeat_back_addr":"172.21.15.174:6815/1777923303","heartbeat_front_addr":"172.21.15.174:6813/1777923303","state":["exists","up"]},{"osd":6,"uuid":"fab2ac2b-ed97-4cde-9825-b480eabf8b8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6825","nonce":2755130316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6827","nonce":2755130316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6831","nonce":2755130316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6829","nonce":2755130316}]},"public_addr":"172.21.15.174:6825/2755130316","cluster_addr":"172.21.15.174:6827/2755130316","heartbeat_back_addr":"172.21.15.174:6831/2755130316","heartbeat_front_addr":"172.21.15.174:6829/2755130316","state":["exists","up"]},{"osd":7,"uuid":"3de9b797-2ab6-4d41-a863-7e7d638fd703","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6817","nonce":2847141411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6819","nonce":2847141411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6823","nonce":2847141411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6821","nonce":2847141411}]},"public_addr":"172.21.15.174:6817/2847141411","cluster_addr":"172.21.15.174:6819/2847141411","heartbeat_back_addr":"172.21.15.174:6823/2847141411","heartbeat_front_addr":"172.21.15.174:6821/2847141411","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.477076+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.496890+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.539305+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.454673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.538881+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.521294+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.520712+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.540404+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:25:27.547 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.546+0000 7f281c08a700 1 -- 172.21.15.143:0/1669325251 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f27f806af00 msgr2=0x7f27f806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:27.548 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.546+0000 7f281c08a700 1 --2- 172.21.15.143:0/1669325251 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f27f806af00 0x7f27f806d3c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2804009870 tx=0x7f2804008040 comp rx=0 tx=0).stop 2023-11-08T19:25:27.548 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.546+0000 7f281c08a700 1 -- 172.21.15.143:0/1669325251 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f281414f990 msgr2=0x7f281414fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:27.548 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.546+0000 7f281c08a700 1 --2- 172.21.15.143:0/1669325251 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f281414f990 0x7f281414fdf0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f280c00afd0 tx=0x7f280c004b40 comp rx=0 tx=0).stop 2023-11-08T19:25:27.549 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.546+0000 7f281c08a700 1 -- 172.21.15.143:0/1669325251 shutdown_connections 2023-11-08T19:25:27.549 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.546+0000 7f281c08a700 1 --2- 172.21.15.143:0/1669325251 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f27f806af00 0x7f27f806d3c0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:27.549 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.546+0000 7f281c08a700 1 --2- 172.21.15.143:0/1669325251 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2800004d90 0x7f281415b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:27.549 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.546+0000 7f281c08a700 1 --2- 172.21.15.143:0/1669325251 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f281414f990 0x7f281414fdf0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:27.550 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.546+0000 7f281c08a700 1 --2- 172.21.15.143:0/1669325251 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f281415e260 0x7f28140c9d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:27.550 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.546+0000 7f281c08a700 1 -- 172.21.15.143:0/1669325251 >> 172.21.15.143:0/1669325251 conn(0x7f281408b000 msgr2=0x7f28140d1820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:27.550 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.546+0000 7f281c08a700 1 -- 172.21.15.143:0/1669325251 shutdown_connections 2023-11-08T19:25:27.551 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.546+0000 7f281c08a700 1 -- 172.21.15.143:0/1669325251 wait complete. 2023-11-08T19:25:27.560 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T19:24:44.551530+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-08T19:24:48.256781+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-08T19:24:48.954990+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-08T19:25:27.561 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T19:25:27.685 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.683+0000 7fecdba0d700 1 Processor -- start 2023-11-08T19:25:27.685 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.683+0000 7fecdba0d700 1 -- start start 2023-11-08T19:25:27.691 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.684+0000 7fecdba0d700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fecd40b1730 0x7fecd40b2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:27.691 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.684+0000 7fecdba0d700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecd40d0420 con 0x7fecd415bb20 2023-11-08T19:25:27.691 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.684+0000 7fecdba0d700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecd40cf810 con 0x7fecd40d1d80 2023-11-08T19:25:27.691 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.684+0000 7fecdba0d700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecd40cf600 con 0x7fecd40b1730 2023-11-08T19:25:27.692 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.684+0000 7fecd97a9700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fecd415bb20 0x7fecd40d1670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:45976/0 (socket says 172.21.15.143:45976) 2023-11-08T19:25:27.692 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.684+0000 7fecd97a9700 1 -- 172.21.15.143:0/1161863117 learned_addr learned my addr 172.21.15.143:0/1161863117 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:25:27.692 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.685+0000 7feccbfff700 1 -- 172.21.15.143:0/1161863117 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1496367829 0 0) 0x7fecd40d0420 con 0x7fecd415bb20 2023-11-08T19:25:27.692 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.685+0000 7fecd9faa700 1 --2- 172.21.15.143:0/1161863117 >> v2:172.21.15.174:3300/0 conn(0x7fecd40b1730 0x7fecd40b2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:27.692 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.685+0000 7feccbfff700 1 -- 172.21.15.143:0/1161863117 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fecc00035b0 con 0x7fecd415bb20 2023-11-08T19:25:27.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.685+0000 7feccbfff700 1 -- 172.21.15.143:0/1161863117 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1427745714 0 0) 0x7fecc00035b0 con 0x7fecd415bb20 2023-11-08T19:25:27.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.685+0000 7feccbfff700 1 -- 172.21.15.143:0/1161863117 >> v2:172.21.15.174:3300/0 conn(0x7fecd40b1730 msgr2=0x7fecd40b2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:27.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.685+0000 7feccbfff700 1 --2- 172.21.15.143:0/1161863117 >> v2:172.21.15.174:3300/0 conn(0x7fecd40b1730 0x7fecd40b2670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:27.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.685+0000 7feccbfff700 1 -- 172.21.15.143:0/1161863117 >> v1:172.21.15.174:6789/0 conn(0x7fecd40d1d80 legacy=0x7fecd40cc4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:27.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.685+0000 7feccbfff700 1 -- 172.21.15.143:0/1161863117 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecd40cfaf0 con 0x7fecd415bb20 2023-11-08T19:25:27.694 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.685+0000 7feccbfff700 1 -- 172.21.15.143:0/1161863117 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4147756303 0 0) 0x7fecc4001670 con 0x7fecd415bb20 2023-11-08T19:25:27.694 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.685+0000 7feccbfff700 1 -- 172.21.15.143:0/1161863117 >> v1:172.21.15.143:6789/0 conn(0x7fecd415bb20 legacy=0x7fecd40d1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:27.694 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.685+0000 7feccbfff700 1 --2- 172.21.15.143:0/1161863117 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fecc0003a00 0x7fecc0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:27.694 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.685+0000 7feccbfff700 1 --2- 172.21.15.143:0/1161863117 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fecc0004850 0x7fecc0004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:27.695 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.685+0000 7feccbfff700 1 --2- 172.21.15.143:0/1161863117 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fecc0005210 0x7fecc000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:27.695 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.685+0000 7feccbfff700 1 -- 172.21.15.143:0/1161863117 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecd40cf600 con 0x7fecc0005210 2023-11-08T19:25:27.695 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.685+0000 7feccbfff700 1 -- 172.21.15.143:0/1161863117 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecc000c0b0 con 0x7fecc0003a00 2023-11-08T19:25:27.695 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.685+0000 7feccbfff700 1 -- 172.21.15.143:0/1161863117 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecc000c270 con 0x7fecc0004850 2023-11-08T19:25:27.695 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.685+0000 7fecd9faa700 1 --2- 172.21.15.143:0/1161863117 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fecc0005210 0x7fecc000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:27.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.685+0000 7fecd97a9700 1 --2- 172.21.15.143:0/1161863117 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fecc0003a00 0x7fecc0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:27.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.685+0000 7fecd8fa8700 1 --2- 172.21.15.143:0/1161863117 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fecc0004850 0x7fecc0004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:27.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.685+0000 7fecd9faa700 1 -- 172.21.15.143:0/1161863117 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fecc0004850 msgr2=0x7fecc0004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:27.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.685+0000 7fecd9faa700 1 --2- 172.21.15.143:0/1161863117 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fecc0004850 0x7fecc0004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:27.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.685+0000 7fecd9faa700 1 -- 172.21.15.143:0/1161863117 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fecc0003a00 msgr2=0x7fecc0003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:27.697 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.685+0000 7fecd9faa700 1 --2- 172.21.15.143:0/1161863117 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fecc0003a00 0x7fecc0003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:27.697 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.685+0000 7fecd9faa700 1 -- 172.21.15.143:0/1161863117 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecc00035b0 con 0x7fecc0005210 2023-11-08T19:25:27.697 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.685+0000 7fecd9faa700 1 -- 172.21.15.143:0/1161863117 reap_dead start 2023-11-08T19:25:27.697 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.686+0000 7fecd9faa700 1 --2- 172.21.15.143:0/1161863117 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fecc0005210 0x7fecc000ba10 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7feccc009fc0 tx=0x7fecc0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1eb824951c1b761 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:27.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.686+0000 7feccbfff700 1 -- 172.21.15.143:0/1161863117 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feccc003a20 con 0x7fecc0005210 2023-11-08T19:25:27.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.686+0000 7feccbfff700 1 -- 172.21.15.143:0/1161863117 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feccc00f040 con 0x7fecc0005210 2023-11-08T19:25:27.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.686+0000 7feccbfff700 1 -- 172.21.15.143:0/1161863117 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feccc003de0 con 0x7fecc0005210 2023-11-08T19:25:27.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.686+0000 7fecdba0d700 1 -- 172.21.15.143:0/1161863117 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fecc0005210 msgr2=0x7fecc000ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:27.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.686+0000 7fecdba0d700 1 --2- 172.21.15.143:0/1161863117 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fecc0005210 0x7fecc000ba10 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7feccc009fc0 tx=0x7fecc0005880 comp rx=0 tx=0).stop 2023-11-08T19:25:27.699 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.686+0000 7fecdba0d700 1 -- 172.21.15.143:0/1161863117 shutdown_connections 2023-11-08T19:25:27.699 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.686+0000 7fecdba0d700 1 --2- 172.21.15.143:0/1161863117 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fecc0005210 0x7fecc000ba10 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:27.699 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.686+0000 7fecdba0d700 1 -- 172.21.15.143:0/1161863117 >> 172.21.15.143:0/1161863117 conn(0x7fecd408b000 msgr2=0x7fecd409ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:27.699 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.686+0000 7fecdba0d700 1 -- 172.21.15.143:0/1161863117 shutdown_connections 2023-11-08T19:25:27.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.686+0000 7fecdba0d700 1 -- 172.21.15.143:0/1161863117 wait complete. 2023-11-08T19:25:27.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.687+0000 7fecdba0d700 1 Processor -- start 2023-11-08T19:25:27.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.687+0000 7fecdba0d700 1 -- start start 2023-11-08T19:25:27.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.688+0000 7fecdba0d700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fecc0005210 0x7fecd41f0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:27.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.688+0000 7fecdba0d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fecd40ca390 0x7fecd40ca810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:27.701 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.688+0000 7fecdba0d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fecd40cd5d0 0x7fecd40cda30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:27.701 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.688+0000 7fecdba0d700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecd40cad50 con 0x7fecd40cd5d0 2023-11-08T19:25:27.701 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.688+0000 7fecdba0d700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecd40b4d40 con 0x7fecd40ca390 2023-11-08T19:25:27.701 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.688+0000 7fecdba0d700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecd40b6480 con 0x7fecc0005210 2023-11-08T19:25:27.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.688+0000 7fecd9faa700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fecd40cd5d0 0x7fecd40cda30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:27.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.688+0000 7fecd9faa700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fecd40cd5d0 0x7fecd40cda30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49838/0 (socket says 172.21.15.143:49838) 2023-11-08T19:25:27.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.688+0000 7fecd9faa700 1 -- 172.21.15.143:0/620270761 learned_addr learned my addr 172.21.15.143:0/620270761 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:27.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.688+0000 7fecd97a9700 1 --2- 172.21.15.143:0/620270761 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fecc0005210 0x7fecd41f0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:27.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.688+0000 7fecd9faa700 1 -- 172.21.15.143:0/620270761 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fecc0005210 msgr2=0x7fecd41f0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:27.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.688+0000 7fecd8fa8700 1 --2- 172.21.15.143:0/620270761 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fecd40ca390 0x7fecd40ca810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:27.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.688+0000 7fecd9faa700 1 --2- 172.21.15.143:0/620270761 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fecc0005210 0x7fecd41f0c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:27.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.688+0000 7fecd9faa700 1 -- 172.21.15.143:0/620270761 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fecd40ca390 msgr2=0x7fecd40ca810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:27.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.689+0000 7fecd9faa700 1 --2- 172.21.15.143:0/620270761 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fecd40ca390 0x7fecd40ca810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:27.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.689+0000 7fecd9faa700 1 -- 172.21.15.143:0/620270761 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecd40ce350 con 0x7fecd40cd5d0 2023-11-08T19:25:27.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.689+0000 7fecd9faa700 1 --2- 172.21.15.143:0/620270761 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fecd40cd5d0 0x7fecd40cda30 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fecd40cbff0 tx=0x7feccc009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:27.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.689+0000 7fecc9ffb700 1 -- 172.21.15.143:0/620270761 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feccc003e10 con 0x7fecd40cd5d0 2023-11-08T19:25:27.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.689+0000 7fecc9ffb700 1 -- 172.21.15.143:0/620270761 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feccc004230 con 0x7fecd40cd5d0 2023-11-08T19:25:27.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.689+0000 7fecdba0d700 1 -- 172.21.15.143:0/620270761 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecd40ce630 con 0x7fecd40cd5d0 2023-11-08T19:25:27.705 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.689+0000 7fecc9ffb700 1 -- 172.21.15.143:0/620270761 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feccc0045e0 con 0x7fecd40cd5d0 2023-11-08T19:25:27.705 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.689+0000 7fecdba0d700 1 -- 172.21.15.143:0/620270761 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecd40b49d0 con 0x7fecd40cd5d0 2023-11-08T19:25:27.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.690+0000 7fecdba0d700 1 -- 172.21.15.143:0/620270761 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecd40cfdb0 con 0x7fecd40cd5d0 2023-11-08T19:25:27.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.690+0000 7fecc9ffb700 1 -- 172.21.15.143:0/620270761 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7feccc004740 con 0x7fecd40cd5d0 2023-11-08T19:25:27.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.691+0000 7fecc9ffb700 1 --2- 172.21.15.143:0/620270761 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fecb806af00 0x7fecb806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:27.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.691+0000 7fecc9ffb700 1 -- 172.21.15.143:0/620270761 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7feccc08a7c0 con 0x7fecd40cd5d0 2023-11-08T19:25:27.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.691+0000 7fecd97a9700 1 --2- 172.21.15.143:0/620270761 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fecb806af00 0x7fecb806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:27.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.691+0000 7fecd97a9700 1 --2- 172.21.15.143:0/620270761 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fecb806af00 0x7fecb806d3c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fecc4001b40 tx=0x7fecc4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:27.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.695+0000 7fecc9ffb700 1 -- 172.21.15.143:0/620270761 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fecd40cfdb0 con 0x7fecd40cd5d0 2023-11-08T19:25:27.886 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.884+0000 7fecdba0d700 1 -- 172.21.15.143:0/620270761 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fecd40cfdb0 con 0x7fecd40cd5d0 2023-11-08T19:25:27.886 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.885+0000 7fecc9ffb700 1 -- 172.21.15.143:0/620270761 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7fecd40cfdb0 con 0x7fecd40cd5d0 2023-11-08T19:25:27.887 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2023-11-08T19:25:27.888 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.887+0000 7fecdba0d700 1 -- 172.21.15.143:0/620270761 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fecb806af00 msgr2=0x7fecb806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:27.889 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.887+0000 7fecdba0d700 1 --2- 172.21.15.143:0/620270761 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fecb806af00 0x7fecb806d3c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fecc4001b40 tx=0x7fecc4009040 comp rx=0 tx=0).stop 2023-11-08T19:25:27.889 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.887+0000 7fecdba0d700 1 -- 172.21.15.143:0/620270761 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fecd40cd5d0 msgr2=0x7fecd40cda30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:27.889 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.887+0000 7fecdba0d700 1 --2- 172.21.15.143:0/620270761 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fecd40cd5d0 0x7fecd40cda30 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fecd40cbff0 tx=0x7feccc009750 comp rx=0 tx=0).stop 2023-11-08T19:25:27.889 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.887+0000 7fecdba0d700 1 -- 172.21.15.143:0/620270761 shutdown_connections 2023-11-08T19:25:27.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.887+0000 7fecdba0d700 1 --2- 172.21.15.143:0/620270761 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fecb806af00 0x7fecb806d3c0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:27.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.887+0000 7fecdba0d700 1 --2- 172.21.15.143:0/620270761 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fecc0005210 0x7fecd41f0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:27.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.887+0000 7fecdba0d700 1 --2- 172.21.15.143:0/620270761 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fecd40cd5d0 0x7fecd40cda30 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:27.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.887+0000 7fecdba0d700 1 --2- 172.21.15.143:0/620270761 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fecd40ca390 0x7fecd40ca810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:27.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.887+0000 7fecdba0d700 1 -- 172.21.15.143:0/620270761 >> 172.21.15.143:0/620270761 conn(0x7fecd408b000 msgr2=0x7fecd41e3190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:27.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.887+0000 7fecdba0d700 1 -- 172.21.15.143:0/620270761 shutdown_connections 2023-11-08T19:25:27.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:27.887+0000 7fecdba0d700 1 -- 172.21.15.143:0/620270761 wait complete. 2023-11-08T19:25:27.900 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-08T19:25:28.024 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.022+0000 7f36b3b96700 1 Processor -- start 2023-11-08T19:25:28.024 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.023+0000 7f36b3b96700 1 -- start start 2023-11-08T19:25:28.026 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.023+0000 7f36b3b96700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f36ac06b240 0x7f36ac0caff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:28.026 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.023+0000 7f36b3b96700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36ac161890 con 0x7f36ac0cb530 2023-11-08T19:25:28.027 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.023+0000 7f36b3b96700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36ac06b740 con 0x7f36ac15dba0 2023-11-08T19:25:28.027 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.023+0000 7f36b3b96700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36ac06cd20 con 0x7f36ac06b240 2023-11-08T19:25:28.027 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.023+0000 7f36b1131700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f36ac06b240 0x7f36ac0caff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:28.027 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.023+0000 7f36b2133700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f36ac0cb530 0x7f36ac0b22e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:46000/0 (socket says 172.21.15.143:46000) 2023-11-08T19:25:28.028 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.023+0000 7f36b2133700 1 -- 172.21.15.143:0/220222086 learned_addr learned my addr 172.21.15.143:0/220222086 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:25:28.028 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.024+0000 7f36b0930700 1 -- 172.21.15.143:0/220222086 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 600119539 0 0) 0x7f36ac161890 con 0x7f36ac0cb530 2023-11-08T19:25:28.028 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.024+0000 7f36b0930700 1 -- 172.21.15.143:0/220222086 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f36980035b0 con 0x7f36ac0cb530 2023-11-08T19:25:28.028 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.024+0000 7f36b1131700 1 -- 172.21.15.143:0/220222086 >> v1:172.21.15.174:6789/0 conn(0x7f36ac15dba0 legacy=0x7f36ac06c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:28.029 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.024+0000 7f36b1131700 1 -- 172.21.15.143:0/220222086 >> v1:172.21.15.143:6789/0 conn(0x7f36ac0cb530 legacy=0x7f36ac0b22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:28.029 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.024+0000 7f36b1131700 1 -- 172.21.15.143:0/220222086 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36ac06ba20 con 0x7f36ac06b240 2023-11-08T19:25:28.029 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.024+0000 7f36b1131700 1 --2- 172.21.15.143:0/220222086 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f36ac06b240 0x7f36ac0caff0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f36a8009840 tx=0x7f36a8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67afc7f56c242011 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:28.029 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.024+0000 7f36b0930700 1 -- 172.21.15.143:0/220222086 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36a8004230 con 0x7f36ac06b240 2023-11-08T19:25:28.029 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.024+0000 7f36b0930700 1 -- 172.21.15.143:0/220222086 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36a8004390 con 0x7f36ac06b240 2023-11-08T19:25:28.030 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.024+0000 7f36b0930700 1 -- 172.21.15.143:0/220222086 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36a80036a0 con 0x7f36ac06b240 2023-11-08T19:25:28.030 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.025+0000 7f36b3b96700 1 -- 172.21.15.143:0/220222086 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f36ac06b240 msgr2=0x7f36ac0caff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:28.033 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.025+0000 7f36b3b96700 1 --2- 172.21.15.143:0/220222086 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f36ac06b240 0x7f36ac0caff0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f36a8009840 tx=0x7f36a8009b50 comp rx=0 tx=0).stop 2023-11-08T19:25:28.034 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.025+0000 7f36b3b96700 1 -- 172.21.15.143:0/220222086 shutdown_connections 2023-11-08T19:25:28.034 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.025+0000 7f36b3b96700 1 --2- 172.21.15.143:0/220222086 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f36ac06b240 0x7f36ac0caff0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:28.034 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.025+0000 7f36b3b96700 1 -- 172.21.15.143:0/220222086 >> 172.21.15.143:0/220222086 conn(0x7f36ac0cd2e0 msgr2=0x7f36ac0cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:28.034 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.025+0000 7f36b3b96700 1 -- 172.21.15.143:0/220222086 shutdown_connections 2023-11-08T19:25:28.034 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.025+0000 7f36b3b96700 1 -- 172.21.15.143:0/220222086 wait complete. 2023-11-08T19:25:28.037 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.026+0000 7f36b3b96700 1 Processor -- start 2023-11-08T19:25:28.038 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.026+0000 7f36b3b96700 1 -- start start 2023-11-08T19:25:28.038 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.027+0000 7f36b3b96700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f36ac06b240 0x7f36ac0acf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:28.038 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.027+0000 7f36b3b96700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f36ac0cb530 0x7f36ac0d07a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:28.038 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.027+0000 7f36b3b96700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f36ac15dba0 0x7f36ac0d0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:28.038 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.027+0000 7f36b3b96700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36ac14b640 con 0x7f36ac06b240 2023-11-08T19:25:28.039 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.027+0000 7f36b3b96700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36ac14acb0 con 0x7f36ac15dba0 2023-11-08T19:25:28.039 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.027+0000 7f36b3b96700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36ac0af7f0 con 0x7f36ac0cb530 2023-11-08T19:25:28.039 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.027+0000 7f36b1932700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f36ac06b240 0x7f36ac0acf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:28.039 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.027+0000 7f36b1932700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f36ac06b240 0x7f36ac0acf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49848/0 (socket says 172.21.15.143:49848) 2023-11-08T19:25:28.039 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.027+0000 7f36b1932700 1 -- 172.21.15.143:0/1054171803 learned_addr learned my addr 172.21.15.143:0/1054171803 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:28.040 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.027+0000 7f36b1131700 1 --2- 172.21.15.143:0/1054171803 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f36ac0cb530 0x7f36ac0d07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:28.040 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.027+0000 7f36b2133700 1 --2- 172.21.15.143:0/1054171803 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f36ac15dba0 0x7f36ac0d0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:28.040 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.028+0000 7f36b1932700 1 -- 172.21.15.143:0/1054171803 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f36ac0cb530 msgr2=0x7f36ac0d07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:28.040 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.028+0000 7f36b1932700 1 --2- 172.21.15.143:0/1054171803 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f36ac0cb530 0x7f36ac0d07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:28.040 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.028+0000 7f36b1932700 1 -- 172.21.15.143:0/1054171803 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f36ac15dba0 msgr2=0x7f36ac0d0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:28.041 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.028+0000 7f36b1932700 1 --2- 172.21.15.143:0/1054171803 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f36ac15dba0 0x7f36ac0d0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:28.041 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.028+0000 7f36b1932700 1 -- 172.21.15.143:0/1054171803 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36a80094a0 con 0x7f36ac06b240 2023-11-08T19:25:28.041 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.028+0000 7f36b1932700 1 --2- 172.21.15.143:0/1054171803 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f36ac06b240 0x7f36ac0acf20 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f369c00b4a0 tx=0x7f369c00b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:28.041 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.028+0000 7f36a27fc700 1 -- 172.21.15.143:0/1054171803 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f369c011830 con 0x7f36ac06b240 2023-11-08T19:25:28.041 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.028+0000 7f36a27fc700 1 -- 172.21.15.143:0/1054171803 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f369c011990 con 0x7f36ac06b240 2023-11-08T19:25:28.042 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.028+0000 7f36b3b96700 1 -- 172.21.15.143:0/1054171803 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36ac14af80 con 0x7f36ac06b240 2023-11-08T19:25:28.042 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.028+0000 7f36a27fc700 1 -- 172.21.15.143:0/1054171803 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f369c011d40 con 0x7f36ac06b240 2023-11-08T19:25:28.042 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.028+0000 7f36b3b96700 1 -- 172.21.15.143:0/1054171803 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36ac0ad460 con 0x7f36ac06b240 2023-11-08T19:25:28.042 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.029+0000 7f36b3b96700 1 -- 172.21.15.143:0/1054171803 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36ac04eee0 con 0x7f36ac06b240 2023-11-08T19:25:28.043 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.029+0000 7f36a27fc700 1 -- 172.21.15.143:0/1054171803 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f369c01a400 con 0x7f36ac06b240 2023-11-08T19:25:28.043 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.030+0000 7f36a27fc700 1 --2- 172.21.15.143:0/1054171803 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f369006af00 0x7f369006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:28.043 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.030+0000 7f36a27fc700 1 -- 172.21.15.143:0/1054171803 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f369c089500 con 0x7f36ac06b240 2023-11-08T19:25:28.043 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.030+0000 7f36b1131700 1 --2- 172.21.15.143:0/1054171803 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f369006af00 0x7f369006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:28.043 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.030+0000 7f36b1131700 1 --2- 172.21.15.143:0/1054171803 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f369006af00 0x7f369006d3c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f36a80090b0 tx=0x7f36a8010000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:28.043 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.034+0000 7f36a27fc700 1 -- 172.21.15.143:0/1054171803 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f369c054640 con 0x7f36ac06b240 2023-11-08T19:25:28.225 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.224+0000 7f36b3b96700 1 -- 172.21.15.143:0/1054171803 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f36ac0b67e0 con 0x7f36ac06b240 2023-11-08T19:25:28.226 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.224+0000 7f36a27fc700 1 -- 172.21.15.143:0/1054171803 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f369c057c60 con 0x7f36ac06b240 2023-11-08T19:25:28.226 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2023-11-08T19:25:28.228 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.226+0000 7f36b3b96700 1 -- 172.21.15.143:0/1054171803 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f369006af00 msgr2=0x7f369006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:28.228 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.226+0000 7f36b3b96700 1 --2- 172.21.15.143:0/1054171803 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f369006af00 0x7f369006d3c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f36a80090b0 tx=0x7f36a8010000 comp rx=0 tx=0).stop 2023-11-08T19:25:28.228 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.226+0000 7f36b3b96700 1 -- 172.21.15.143:0/1054171803 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f36ac06b240 msgr2=0x7f36ac0acf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:28.228 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.226+0000 7f36b3b96700 1 --2- 172.21.15.143:0/1054171803 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f36ac06b240 0x7f36ac0acf20 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f369c00b4a0 tx=0x7f369c00b7b0 comp rx=0 tx=0).stop 2023-11-08T19:25:28.229 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.226+0000 7f36b3b96700 1 -- 172.21.15.143:0/1054171803 shutdown_connections 2023-11-08T19:25:28.229 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.226+0000 7f36b3b96700 1 --2- 172.21.15.143:0/1054171803 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f369006af00 0x7f369006d3c0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:28.229 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.226+0000 7f36b3b96700 1 --2- 172.21.15.143:0/1054171803 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f36ac15dba0 0x7f36ac0d0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:28.229 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.226+0000 7f36b3b96700 1 --2- 172.21.15.143:0/1054171803 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f36ac06b240 0x7f36ac0acf20 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:28.229 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.226+0000 7f36b3b96700 1 --2- 172.21.15.143:0/1054171803 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f36ac0cb530 0x7f36ac0d07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:28.230 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.226+0000 7f36b3b96700 1 -- 172.21.15.143:0/1054171803 >> 172.21.15.143:0/1054171803 conn(0x7f36ac0cd2e0 msgr2=0x7f36ac15ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:28.230 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.226+0000 7f36b3b96700 1 -- 172.21.15.143:0/1054171803 shutdown_connections 2023-11-08T19:25:28.230 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.226+0000 7f36b3b96700 1 -- 172.21.15.143:0/1054171803 wait complete. 2023-11-08T19:25:28.240 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-08T19:25:28.363 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.362+0000 7fd62f08b700 1 Processor -- start 2023-11-08T19:25:28.363 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.362+0000 7fd62f08b700 1 -- start start 2023-11-08T19:25:28.366 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.362+0000 7fd62f08b700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fd62806b240 0x7fd6280caff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:28.366 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.363+0000 7fd62f08b700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd6280cb530 0x7fd6280b22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:28.366 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.363+0000 7fd62f08b700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd628161890 con 0x7fd62815dba0 2023-11-08T19:25:28.366 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.363+0000 7fd62f08b700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd62806cd20 con 0x7fd62806b240 2023-11-08T19:25:28.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.363+0000 7fd62f08b700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd62806b740 con 0x7fd6280cb530 2023-11-08T19:25:28.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.363+0000 7fd627fff700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fd62806b240 0x7fd6280caff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:28.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.363+0000 7fd62d628700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd6280cb530 0x7fd6280b22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:28.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.363+0000 7fd627fff700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fd62806b240 0x7fd6280caff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.143:33356/0 (socket says 172.21.15.143:33356) 2023-11-08T19:25:28.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.363+0000 7fd627fff700 1 -- 172.21.15.143:0/3790308286 learned_addr learned my addr 172.21.15.143:0/3790308286 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:28.368 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.363+0000 7fd627fff700 1 -- 172.21.15.143:0/3790308286 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd6280cb530 msgr2=0x7fd6280b22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:28.368 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.363+0000 7fd627fff700 1 --2- 172.21.15.143:0/3790308286 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd6280cb530 0x7fd6280b22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:28.368 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.363+0000 7fd627fff700 1 -- 172.21.15.143:0/3790308286 >> v1:172.21.15.174:6789/0 conn(0x7fd62815dba0 legacy=0x7fd62806c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:28.368 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.363+0000 7fd627fff700 1 -- 172.21.15.143:0/3790308286 --> v2:172.21.15.174:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd62806ba20 con 0x7fd62806b240 2023-11-08T19:25:28.368 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.363+0000 7fd627fff700 1 --2- 172.21.15.143:0/3790308286 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd62806b240 0x7fd6280caff0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd6140098f0 tx=0x7fd614009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=adad3135d4419af3 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:28.368 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.364+0000 7fd6277fe700 1 -- 172.21.15.143:0/3790308286 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd614011830 con 0x7fd62806b240 2023-11-08T19:25:28.369 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.364+0000 7fd6277fe700 1 -- 172.21.15.143:0/3790308286 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd614011990 con 0x7fd62806b240 2023-11-08T19:25:28.369 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.364+0000 7fd6277fe700 1 -- 172.21.15.143:0/3790308286 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd614011d20 con 0x7fd62806b240 2023-11-08T19:25:28.369 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.364+0000 7fd62f08b700 1 -- 172.21.15.143:0/3790308286 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd62806b240 msgr2=0x7fd6280caff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:28.369 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.364+0000 7fd62f08b700 1 --2- 172.21.15.143:0/3790308286 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd62806b240 0x7fd6280caff0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd6140098f0 tx=0x7fd614009c00 comp rx=0 tx=0).stop 2023-11-08T19:25:28.369 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.364+0000 7fd62f08b700 1 -- 172.21.15.143:0/3790308286 shutdown_connections 2023-11-08T19:25:28.370 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.364+0000 7fd62f08b700 1 --2- 172.21.15.143:0/3790308286 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd6280cb530 0x7fd6280b22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:28.370 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.364+0000 7fd62f08b700 1 --2- 172.21.15.143:0/3790308286 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd62806b240 0x7fd6280caff0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:28.370 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.364+0000 7fd62f08b700 1 -- 172.21.15.143:0/3790308286 >> 172.21.15.143:0/3790308286 conn(0x7fd6280cd2e0 msgr2=0x7fd6280cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:28.370 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.364+0000 7fd62f08b700 1 -- 172.21.15.143:0/3790308286 shutdown_connections 2023-11-08T19:25:28.370 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.364+0000 7fd62f08b700 1 -- 172.21.15.143:0/3790308286 wait complete. 2023-11-08T19:25:28.373 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.366+0000 7fd62f08b700 1 Processor -- start 2023-11-08T19:25:28.374 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.366+0000 7fd62f08b700 1 -- start start 2023-11-08T19:25:28.374 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.366+0000 7fd62f08b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd62806b240 0x7fd6280adde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:28.374 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.366+0000 7fd62f08b700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd6280cb530 0x7fd6280c6f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:28.374 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.366+0000 7fd62f08b700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd62815dba0 0x7fd6280ad420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:28.374 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.366+0000 7fd62f08b700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6280af510 con 0x7fd6280cb530 2023-11-08T19:25:28.375 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.366+0000 7fd62f08b700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6280c74c0 con 0x7fd62806b240 2023-11-08T19:25:28.375 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.366+0000 7fd62f08b700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6280ad960 con 0x7fd62815dba0 2023-11-08T19:25:28.375 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.367+0000 7fd627fff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd6280cb530 0x7fd6280c6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:28.375 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.367+0000 7fd627fff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd6280cb530 0x7fd6280c6f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49864/0 (socket says 172.21.15.143:49864) 2023-11-08T19:25:28.375 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.367+0000 7fd627fff700 1 -- 172.21.15.143:0/847650853 learned_addr learned my addr 172.21.15.143:0/847650853 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:28.376 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.367+0000 7fd62d628700 1 --2- 172.21.15.143:0/847650853 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd62815dba0 0x7fd6280ad420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:28.376 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.367+0000 7fd62ce27700 1 --2- 172.21.15.143:0/847650853 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd62806b240 0x7fd6280adde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:28.376 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.367+0000 7fd627fff700 1 -- 172.21.15.143:0/847650853 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd62815dba0 msgr2=0x7fd6280ad420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:28.376 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.367+0000 7fd627fff700 1 --2- 172.21.15.143:0/847650853 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd62815dba0 0x7fd6280ad420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:28.376 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.367+0000 7fd627fff700 1 -- 172.21.15.143:0/847650853 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd62806b240 msgr2=0x7fd6280adde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:28.377 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.367+0000 7fd627fff700 1 --2- 172.21.15.143:0/847650853 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd62806b240 0x7fd6280adde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:28.377 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.367+0000 7fd627fff700 1 -- 172.21.15.143:0/847650853 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd614009550 con 0x7fd6280cb530 2023-11-08T19:25:28.377 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.367+0000 7fd62d628700 1 --2- 172.21.15.143:0/847650853 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd62815dba0 0x7fd6280ad420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:25:28.377 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.367+0000 7fd627fff700 1 --2- 172.21.15.143:0/847650853 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd6280cb530 0x7fd6280c6f80 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd614000c00 tx=0x7fd614004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:28.377 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.367+0000 7fd6257fa700 1 -- 172.21.15.143:0/847650853 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd614004570 con 0x7fd6280cb530 2023-11-08T19:25:28.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.367+0000 7fd6257fa700 1 -- 172.21.15.143:0/847650853 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd61401f070 con 0x7fd6280cb530 2023-11-08T19:25:28.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.367+0000 7fd62f08b700 1 -- 172.21.15.143:0/847650853 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6280ae470 con 0x7fd6280cb530 2023-11-08T19:25:28.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.367+0000 7fd6257fa700 1 -- 172.21.15.143:0/847650853 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6140038c0 con 0x7fd6280cb530 2023-11-08T19:25:28.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.367+0000 7fd62f08b700 1 -- 172.21.15.143:0/847650853 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6280aea30 con 0x7fd6280cb530 2023-11-08T19:25:28.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.368+0000 7fd62f08b700 1 -- 172.21.15.143:0/847650853 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd62804eee0 con 0x7fd6280cb530 2023-11-08T19:25:28.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.369+0000 7fd6257fa700 1 -- 172.21.15.143:0/847650853 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd614021030 con 0x7fd6280cb530 2023-11-08T19:25:28.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.369+0000 7fd6257fa700 1 --2- 172.21.15.143:0/847650853 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd60c06af00 0x7fd60c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:28.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.369+0000 7fd6257fa700 1 -- 172.21.15.143:0/847650853 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd61408aae0 con 0x7fd6280cb530 2023-11-08T19:25:28.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.369+0000 7fd62ce27700 1 --2- 172.21.15.143:0/847650853 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd60c06af00 0x7fd60c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:28.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.370+0000 7fd62ce27700 1 --2- 172.21.15.143:0/847650853 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd60c06af00 0x7fd60c06d3c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd61c007fd0 tx=0x7fd61c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:28.380 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.373+0000 7fd6257fa700 1 -- 172.21.15.143:0/847650853 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd614055ca0 con 0x7fd6280cb530 2023-11-08T19:25:28.564 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.562+0000 7fd62f08b700 1 -- 172.21.15.143:0/847650853 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd6280b67e0 con 0x7fd6280cb530 2023-11-08T19:25:28.564 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.563+0000 7fd6257fa700 1 -- 172.21.15.143:0/847650853 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd614015020 con 0x7fd6280cb530 2023-11-08T19:25:28.565 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 64 2023-11-08T19:25:28.567 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.565+0000 7fd62f08b700 1 -- 172.21.15.143:0/847650853 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd60c06af00 msgr2=0x7fd60c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:28.567 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.565+0000 7fd62f08b700 1 --2- 172.21.15.143:0/847650853 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd60c06af00 0x7fd60c06d3c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd61c007fd0 tx=0x7fd61c009040 comp rx=0 tx=0).stop 2023-11-08T19:25:28.567 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.565+0000 7fd62f08b700 1 -- 172.21.15.143:0/847650853 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd6280cb530 msgr2=0x7fd6280c6f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:28.567 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.565+0000 7fd62f08b700 1 --2- 172.21.15.143:0/847650853 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd6280cb530 0x7fd6280c6f80 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd614000c00 tx=0x7fd614004750 comp rx=0 tx=0).stop 2023-11-08T19:25:28.568 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.565+0000 7fd62f08b700 1 -- 172.21.15.143:0/847650853 shutdown_connections 2023-11-08T19:25:28.568 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.565+0000 7fd62f08b700 1 --2- 172.21.15.143:0/847650853 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd60c06af00 0x7fd60c06d3c0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:28.568 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.565+0000 7fd62f08b700 1 --2- 172.21.15.143:0/847650853 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd62815dba0 0x7fd6280ad420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:28.568 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.565+0000 7fd62f08b700 1 --2- 172.21.15.143:0/847650853 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd62806b240 0x7fd6280adde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:28.568 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.565+0000 7fd62f08b700 1 --2- 172.21.15.143:0/847650853 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd6280cb530 0x7fd6280c6f80 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:28.569 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.565+0000 7fd62f08b700 1 -- 172.21.15.143:0/847650853 >> 172.21.15.143:0/847650853 conn(0x7fd6280cd2e0 msgr2=0x7fd62815ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:28.569 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.565+0000 7fd62f08b700 1 -- 172.21.15.143:0/847650853 shutdown_connections 2023-11-08T19:25:28.569 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.565+0000 7fd62f08b700 1 -- 172.21.15.143:0/847650853 wait complete. 2023-11-08T19:25:28.581 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-08T19:25:28.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.701+0000 7ffa0228a700 1 Processor -- start 2023-11-08T19:25:28.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.701+0000 7ffa0228a700 1 -- start start 2023-11-08T19:25:28.705 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.701+0000 7ffa0228a700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff9fc06c730 0x7ff9fc0cd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:28.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.701+0000 7ffa0228a700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff9fc06c030 0x7ff9fc06cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:28.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.701+0000 7ffa0228a700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9fc0cd480 con 0x7ff9fc15de80 2023-11-08T19:25:28.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.701+0000 7ffa0228a700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9fc0cc610 con 0x7ff9fc06c030 2023-11-08T19:25:28.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.701+0000 7ffa0228a700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9fc06d050 con 0x7ff9fc06c730 2023-11-08T19:25:28.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.702+0000 7ff9fb7fe700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff9fc06c030 0x7ff9fc06cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:28.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.702+0000 7ff9fb7fe700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff9fc06c030 0x7ff9fc06cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49870/0 (socket says 172.21.15.143:49870) 2023-11-08T19:25:28.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.702+0000 7ff9fb7fe700 1 -- 172.21.15.143:0/1981731951 learned_addr learned my addr 172.21.15.143:0/1981731951 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:28.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.702+0000 7ff9fbfff700 1 --2- 172.21.15.143:0/1981731951 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff9fc06c730 0x7ff9fc0cd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:28.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.702+0000 7ff9fb7fe700 1 -- 172.21.15.143:0/1981731951 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff9fc06c730 msgr2=0x7ff9fc0cd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:28.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.702+0000 7ff9fb7fe700 1 --2- 172.21.15.143:0/1981731951 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff9fc06c730 0x7ff9fc0cd910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:28.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.702+0000 7ff9fb7fe700 1 -- 172.21.15.143:0/1981731951 >> v1:172.21.15.174:6789/0 conn(0x7ff9fc15de80 legacy=0x7ff9fc0cbf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:28.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.702+0000 7ff9fb7fe700 1 -- 172.21.15.143:0/1981731951 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9fc0cc870 con 0x7ff9fc06c030 2023-11-08T19:25:28.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.702+0000 7ff9fbfff700 1 --2- 172.21.15.143:0/1981731951 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff9fc06c730 0x7ff9fc0cd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:25:28.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.702+0000 7ff9fb7fe700 1 --2- 172.21.15.143:0/1981731951 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff9fc06c030 0x7ff9fc06cb10 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ff9f00098f0 tx=0x7ff9f0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34c42f34a4de3c19 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:28.709 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.702+0000 7ff9faffd700 1 -- 172.21.15.143:0/1981731951 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f0004230 con 0x7ff9fc06c030 2023-11-08T19:25:28.709 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.702+0000 7ff9faffd700 1 -- 172.21.15.143:0/1981731951 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9f0004390 con 0x7ff9fc06c030 2023-11-08T19:25:28.709 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.702+0000 7ff9faffd700 1 -- 172.21.15.143:0/1981731951 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f00036a0 con 0x7ff9fc06c030 2023-11-08T19:25:28.709 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.703+0000 7ffa0228a700 1 -- 172.21.15.143:0/1981731951 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff9fc06c030 msgr2=0x7ff9fc06cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:28.710 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.703+0000 7ffa0228a700 1 --2- 172.21.15.143:0/1981731951 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff9fc06c030 0x7ff9fc06cb10 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ff9f00098f0 tx=0x7ff9f0009c00 comp rx=0 tx=0).stop 2023-11-08T19:25:28.710 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.703+0000 7ffa0228a700 1 -- 172.21.15.143:0/1981731951 shutdown_connections 2023-11-08T19:25:28.710 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.703+0000 7ffa0228a700 1 --2- 172.21.15.143:0/1981731951 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff9fc06c730 0x7ff9fc0cd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:28.710 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.703+0000 7ffa0228a700 1 --2- 172.21.15.143:0/1981731951 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff9fc06c030 0x7ff9fc06cb10 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:28.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.703+0000 7ffa0228a700 1 -- 172.21.15.143:0/1981731951 >> 172.21.15.143:0/1981731951 conn(0x7ff9fc08b000 msgr2=0x7ff9fc09ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:28.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.703+0000 7ffa0228a700 1 -- 172.21.15.143:0/1981731951 shutdown_connections 2023-11-08T19:25:28.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.703+0000 7ffa0228a700 1 -- 172.21.15.143:0/1981731951 wait complete. 2023-11-08T19:25:28.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.704+0000 7ffa0228a700 1 Processor -- start 2023-11-08T19:25:28.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.704+0000 7ffa0228a700 1 -- start start 2023-11-08T19:25:28.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.705+0000 7ffa0228a700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff9fc06c030 0x7ff9fc15e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:28.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.705+0000 7ffa0228a700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff9fc06c730 0x7ff9fc15eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:28.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.705+0000 7ffa0228a700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff9fc15de80 0x7ff9fc15f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:28.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.705+0000 7ffa0228a700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9fc15d4e0 con 0x7ff9fc15de80 2023-11-08T19:25:28.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.705+0000 7ffa0228a700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9fc0b5060 con 0x7ff9fc06c030 2023-11-08T19:25:28.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.705+0000 7ffa0228a700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9fc0b5340 con 0x7ff9fc06c730 2023-11-08T19:25:28.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.705+0000 7ffa00827700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff9fc15de80 0x7ff9fc15f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:28.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.705+0000 7ffa00827700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff9fc15de80 0x7ff9fc15f410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49876/0 (socket says 172.21.15.143:49876) 2023-11-08T19:25:28.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.705+0000 7ffa00827700 1 -- 172.21.15.143:0/2207373222 learned_addr learned my addr 172.21.15.143:0/2207373222 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:28.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.705+0000 7ff9fb7fe700 1 --2- 172.21.15.143:0/2207373222 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff9fc06c730 0x7ff9fc15eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:28.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.705+0000 7ff9fbfff700 1 --2- 172.21.15.143:0/2207373222 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff9fc06c030 0x7ff9fc15e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:28.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.705+0000 7ffa00827700 1 -- 172.21.15.143:0/2207373222 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff9fc06c730 msgr2=0x7ff9fc15eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:28.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.705+0000 7ffa00827700 1 --2- 172.21.15.143:0/2207373222 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff9fc06c730 0x7ff9fc15eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:28.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.705+0000 7ffa00827700 1 -- 172.21.15.143:0/2207373222 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff9fc06c030 msgr2=0x7ff9fc15e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:28.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.705+0000 7ffa00827700 1 --2- 172.21.15.143:0/2207373222 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff9fc06c030 0x7ff9fc15e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:28.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.705+0000 7ffa00827700 1 -- 172.21.15.143:0/2207373222 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9f0009550 con 0x7ff9fc15de80 2023-11-08T19:25:28.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.705+0000 7ff9fbfff700 1 --2- 172.21.15.143:0/2207373222 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff9fc06c030 0x7ff9fc15e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:25:28.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.705+0000 7ffa00827700 1 --2- 172.21.15.143:0/2207373222 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff9fc15de80 0x7ff9fc15f410 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff9f400cc10 tx=0x7ff9f400cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:28.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.705+0000 7ff9f8ff9700 1 -- 172.21.15.143:0/2207373222 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f4004b90 con 0x7ff9fc15de80 2023-11-08T19:25:28.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.706+0000 7ff9f8ff9700 1 -- 172.21.15.143:0/2207373222 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9f4004cf0 con 0x7ff9fc15de80 2023-11-08T19:25:28.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.706+0000 7ffa0228a700 1 -- 172.21.15.143:0/2207373222 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9fc0b5620 con 0x7ff9fc15de80 2023-11-08T19:25:28.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.706+0000 7ff9f8ff9700 1 -- 172.21.15.143:0/2207373222 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f400ab90 con 0x7ff9fc15de80 2023-11-08T19:25:28.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.706+0000 7ffa0228a700 1 -- 172.21.15.143:0/2207373222 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9fc15d100 con 0x7ff9fc15de80 2023-11-08T19:25:28.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.706+0000 7ffa0228a700 1 -- 172.21.15.143:0/2207373222 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9fc04eee0 con 0x7ff9fc15de80 2023-11-08T19:25:28.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.707+0000 7ff9f8ff9700 1 -- 172.21.15.143:0/2207373222 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff9f400acf0 con 0x7ff9fc15de80 2023-11-08T19:25:28.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.707+0000 7ff9f8ff9700 1 --2- 172.21.15.143:0/2207373222 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7ff9e806af00 0x7ff9e806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:28.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.707+0000 7ff9f8ff9700 1 -- 172.21.15.143:0/2207373222 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff9f4089050 con 0x7ff9fc15de80 2023-11-08T19:25:28.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.707+0000 7ff9fbfff700 1 --2- 172.21.15.143:0/2207373222 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7ff9e806af00 0x7ff9e806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:28.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.708+0000 7ff9fbfff700 1 --2- 172.21.15.143:0/2207373222 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7ff9e806af00 0x7ff9e806d3c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec00aa20 tx=0x7ff9ec008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:28.721 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.711+0000 7ff9f8ff9700 1 -- 172.21.15.143:0/2207373222 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff9f40bbca0 con 0x7ff9fc15de80 2023-11-08T19:25:28.918 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.917+0000 7ffa0228a700 1 -- 172.21.15.143:0/2207373222 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ff9fc0c8710 con 0x7ff9fc15de80 2023-11-08T19:25:28.919 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.917+0000 7ff9f8ff9700 1 -- 172.21.15.143:0/2207373222 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) v1 ==== 80+0+0 (secure 0 0 0) 0x7ff9f4017020 con 0x7ff9fc15de80 2023-11-08T19:25:28.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.919+0000 7ffa0228a700 1 -- 172.21.15.143:0/2207373222 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7ff9e806af00 msgr2=0x7ff9e806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:28.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.919+0000 7ffa0228a700 1 --2- 172.21.15.143:0/2207373222 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7ff9e806af00 0x7ff9e806d3c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec00aa20 tx=0x7ff9ec008040 comp rx=0 tx=0).stop 2023-11-08T19:25:28.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.919+0000 7ffa0228a700 1 -- 172.21.15.143:0/2207373222 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff9fc15de80 msgr2=0x7ff9fc15f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:28.923 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.919+0000 7ffa0228a700 1 --2- 172.21.15.143:0/2207373222 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff9fc15de80 0x7ff9fc15f410 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff9f400cc10 tx=0x7ff9f400cf20 comp rx=0 tx=0).stop 2023-11-08T19:25:28.923 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.919+0000 7ffa0228a700 1 -- 172.21.15.143:0/2207373222 shutdown_connections 2023-11-08T19:25:28.923 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.919+0000 7ffa0228a700 1 --2- 172.21.15.143:0/2207373222 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7ff9e806af00 0x7ff9e806d3c0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:28.923 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.919+0000 7ffa0228a700 1 --2- 172.21.15.143:0/2207373222 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff9fc15de80 0x7ff9fc15f410 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:28.924 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.919+0000 7ffa0228a700 1 --2- 172.21.15.143:0/2207373222 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff9fc06c030 0x7ff9fc15e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:28.924 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.919+0000 7ffa0228a700 1 --2- 172.21.15.143:0/2207373222 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff9fc06c730 0x7ff9fc15eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:28.924 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.919+0000 7ffa0228a700 1 -- 172.21.15.143:0/2207373222 >> 172.21.15.143:0/2207373222 conn(0x7ff9fc08b000 msgr2=0x7ff9fc0d19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:28.925 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.919+0000 7ffa0228a700 1 -- 172.21.15.143:0/2207373222 shutdown_connections 2023-11-08T19:25:28.925 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:28.919+0000 7ffa0228a700 1 -- 172.21.15.143:0/2207373222 wait complete. 2023-11-08T19:25:28.925 INFO:teuthology.orchestra.run.smithi143.stderr:listed 0 entries 2023-11-08T19:25:28.933 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-08T19:25:28.933 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:28.934 DEBUG:teuthology.orchestra.run.smithi143:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2023-11-08T19:25:28.977 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi143.front.sepia.ceph.com 2023-11-08T19:25:28.977 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:28.978 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-08T19:25:29.046 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2023-11-08T19:25:29.047 DEBUG:teuthology.orchestra.run.smithi174:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:25:29.070 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2023-11-08T19:25:29.071 DEBUG:teuthology.orchestra.run.smithi174:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:25:29.131 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2023-11-08T19:25:29.132 DEBUG:teuthology.orchestra.run.smithi174:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:25:29.193 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:25:29.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.312+0000 7fb31b5a3700 1 Processor -- start 2023-11-08T19:25:29.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.312+0000 7fb31b5a3700 1 -- start start 2023-11-08T19:25:29.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.313+0000 7fb31b5a3700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb314157e30 0x7fb314071aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:29.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.313+0000 7fb31b5a3700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fb3140cd8d0 0x7fb3140ca590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:29.318 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.313+0000 7fb31b5a3700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3140ceb30 con 0x7fb3140cde00 2023-11-08T19:25:29.318 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.313+0000 7fb31b5a3700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb314071680 con 0x7fb3140cd8d0 2023-11-08T19:25:29.318 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.313+0000 7fb31b5a3700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb314070fe0 con 0x7fb314157e30 2023-11-08T19:25:29.318 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.313+0000 7fb318b3e700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fb3140cde00 0x7fb3140cd1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:46034/0 (socket says 172.21.15.143:46034) 2023-11-08T19:25:29.318 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.313+0000 7fb318b3e700 1 -- 172.21.15.143:0/2848337285 learned_addr learned my addr 172.21.15.143:0/2848337285 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:25:29.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.313+0000 7fb30bfff700 1 -- 172.21.15.143:0/2848337285 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1361137113 0 0) 0x7fb3140ceb30 con 0x7fb3140cde00 2023-11-08T19:25:29.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.313+0000 7fb30bfff700 1 -- 172.21.15.143:0/2848337285 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3000046c0 con 0x7fb3140cde00 2023-11-08T19:25:29.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.313+0000 7fb31933f700 1 --2- 172.21.15.143:0/2848337285 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb314157e30 0x7fb314071aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:29.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.313+0000 7fb319b40700 1 --2- 172.21.15.143:0/2848337285 >> v2:172.21.15.174:3300/0 conn(0x7fb3140cd8d0 0x7fb3140ca590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:29.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.313+0000 7fb30bfff700 1 -- 172.21.15.143:0/2848337285 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1389802332 0 0) 0x7fb3000046c0 con 0x7fb3140cde00 2023-11-08T19:25:29.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.313+0000 7fb30bfff700 1 -- 172.21.15.143:0/2848337285 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb314157e30 msgr2=0x7fb314071aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:29.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.313+0000 7fb30bfff700 1 --2- 172.21.15.143:0/2848337285 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb314157e30 0x7fb314071aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:29.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.313+0000 7fb30bfff700 1 -- 172.21.15.143:0/2848337285 >> v2:172.21.15.174:3300/0 conn(0x7fb3140cd8d0 msgr2=0x7fb3140ca590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:29.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.313+0000 7fb30bfff700 1 --2- 172.21.15.143:0/2848337285 >> v2:172.21.15.174:3300/0 conn(0x7fb3140cd8d0 0x7fb3140ca590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:29.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.313+0000 7fb30bfff700 1 -- 172.21.15.143:0/2848337285 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3140ce850 con 0x7fb3140cde00 2023-11-08T19:25:29.321 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.314+0000 7fb30bfff700 1 -- 172.21.15.143:0/2848337285 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4147756303 0 0) 0x7fb304001670 con 0x7fb3140cde00 2023-11-08T19:25:29.321 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.314+0000 7fb30bfff700 1 -- 172.21.15.143:0/2848337285 >> v1:172.21.15.143:6789/0 conn(0x7fb3140cde00 legacy=0x7fb3140cd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:29.321 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.314+0000 7fb30bfff700 1 --2- 172.21.15.143:0/2848337285 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb300004b10 0x7fb300004f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:29.321 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.314+0000 7fb30bfff700 1 --2- 172.21.15.143:0/2848337285 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb300005960 0x7fb300005de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:29.321 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.314+0000 7fb30bfff700 1 --2- 172.21.15.143:0/2848337285 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb314157e30 0x7fb300006320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:29.322 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.314+0000 7fb30bfff700 1 -- 172.21.15.143:0/2848337285 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb314071680 con 0x7fb300005960 2023-11-08T19:25:29.322 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.314+0000 7fb30bfff700 1 -- 172.21.15.143:0/2848337285 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb314070fe0 con 0x7fb300004b10 2023-11-08T19:25:29.322 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.314+0000 7fb30bfff700 1 -- 172.21.15.143:0/2848337285 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3000034d0 con 0x7fb314157e30 2023-11-08T19:25:29.322 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.314+0000 7fb318b3e700 1 --2- 172.21.15.143:0/2848337285 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb300005960 0x7fb300005de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:29.322 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.314+0000 7fb31933f700 1 --2- 172.21.15.143:0/2848337285 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb300004b10 0x7fb300004f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:29.323 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.314+0000 7fb319b40700 1 --2- 172.21.15.143:0/2848337285 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb314157e30 0x7fb300006320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:29.323 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.314+0000 7fb318b3e700 1 -- 172.21.15.143:0/2848337285 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb314157e30 msgr2=0x7fb300006320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:29.323 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.314+0000 7fb318b3e700 1 --2- 172.21.15.143:0/2848337285 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb314157e30 0x7fb300006320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:29.323 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.314+0000 7fb318b3e700 1 -- 172.21.15.143:0/2848337285 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb300004b10 msgr2=0x7fb300004f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:29.323 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.314+0000 7fb318b3e700 1 --2- 172.21.15.143:0/2848337285 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb300004b10 0x7fb300004f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:29.324 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.314+0000 7fb318b3e700 1 -- 172.21.15.143:0/2848337285 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3000046c0 con 0x7fb300005960 2023-11-08T19:25:29.324 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.314+0000 7fb319b40700 1 --2- 172.21.15.143:0/2848337285 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb314157e30 0x7fb300006320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:25:29.324 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.314+0000 7fb318b3e700 1 --2- 172.21.15.143:0/2848337285 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb300005960 0x7fb300005de0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fb30400c190 tx=0x7fb300006990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6efa6abdb9038d5b server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:29.325 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.314+0000 7fb30bfff700 1 -- 172.21.15.143:0/2848337285 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb304007b70 con 0x7fb300005960 2023-11-08T19:25:29.325 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.314+0000 7fb30bfff700 1 -- 172.21.15.143:0/2848337285 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb30400cd30 con 0x7fb300005960 2023-11-08T19:25:29.325 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.314+0000 7fb30bfff700 1 -- 172.21.15.143:0/2848337285 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb30400cea0 con 0x7fb300005960 2023-11-08T19:25:29.325 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.315+0000 7fb31b5a3700 1 -- 172.21.15.143:0/2848337285 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb300005960 msgr2=0x7fb300005de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:29.325 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.315+0000 7fb31b5a3700 1 --2- 172.21.15.143:0/2848337285 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb300005960 0x7fb300005de0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fb30400c190 tx=0x7fb300006990 comp rx=0 tx=0).stop 2023-11-08T19:25:29.326 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.315+0000 7fb319b40700 1 -- 172.21.15.143:0/2848337285 reap_dead start 2023-11-08T19:25:29.326 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.315+0000 7fb31b5a3700 1 -- 172.21.15.143:0/2848337285 shutdown_connections 2023-11-08T19:25:29.326 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.315+0000 7fb31b5a3700 1 -- 172.21.15.143:0/2848337285 >> 172.21.15.143:0/2848337285 conn(0x7fb31406b4c0 msgr2=0x7fb3140cf680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:29.326 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.315+0000 7fb31b5a3700 1 -- 172.21.15.143:0/2848337285 shutdown_connections 2023-11-08T19:25:29.327 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.315+0000 7fb31b5a3700 1 -- 172.21.15.143:0/2848337285 wait complete. 2023-11-08T19:25:29.329 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.316+0000 7fb31b5a3700 1 Processor -- start 2023-11-08T19:25:29.330 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.317+0000 7fb31b5a3700 1 -- start start 2023-11-08T19:25:29.330 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.317+0000 7fb31b5a3700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb300005960 0x7fb3140b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:29.330 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.317+0000 7fb31b5a3700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb3140c8150 0x7fb31414ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:29.330 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.317+0000 7fb31b5a3700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb3140b0110 0x7fb3140b05a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:29.330 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.318+0000 7fb31b5a3700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3140b0ae0 con 0x7fb3140b0110 2023-11-08T19:25:29.331 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.318+0000 7fb31b5a3700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3140b5280 con 0x7fb3140c8150 2023-11-08T19:25:29.331 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.318+0000 7fb31b5a3700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3140b2f90 con 0x7fb300005960 2023-11-08T19:25:29.331 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.318+0000 7fb319b40700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb3140b0110 0x7fb3140b05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:29.331 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.318+0000 7fb319b40700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb3140b0110 0x7fb3140b05a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49896/0 (socket says 172.21.15.143:49896) 2023-11-08T19:25:29.331 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.318+0000 7fb319b40700 1 -- 172.21.15.143:0/1424455822 learned_addr learned my addr 172.21.15.143:0/1424455822 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:29.332 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.318+0000 7fb31933f700 1 --2- 172.21.15.143:0/1424455822 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb300005960 0x7fb3140b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:29.332 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.318+0000 7fb318b3e700 1 --2- 172.21.15.143:0/1424455822 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb3140c8150 0x7fb31414ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:29.332 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.318+0000 7fb319b40700 1 -- 172.21.15.143:0/1424455822 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb300005960 msgr2=0x7fb3140b4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:29.332 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.318+0000 7fb319b40700 1 --2- 172.21.15.143:0/1424455822 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb300005960 0x7fb3140b4d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:29.332 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.318+0000 7fb319b40700 1 -- 172.21.15.143:0/1424455822 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb3140c8150 msgr2=0x7fb31414ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:29.333 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.318+0000 7fb319b40700 1 --2- 172.21.15.143:0/1424455822 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb3140c8150 0x7fb31414ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:29.333 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.318+0000 7fb319b40700 1 -- 172.21.15.143:0/1424455822 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3140b3270 con 0x7fb3140b0110 2023-11-08T19:25:29.333 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.318+0000 7fb31933f700 1 --2- 172.21.15.143:0/1424455822 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb300005960 0x7fb3140b4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:25:29.333 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.318+0000 7fb319b40700 1 --2- 172.21.15.143:0/1424455822 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb3140b0110 0x7fb3140b05a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fb3140722b0 tx=0x7fb30c0114b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:29.334 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.318+0000 7fb309ffb700 1 -- 172.21.15.143:0/1424455822 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb30c00e030 con 0x7fb3140b0110 2023-11-08T19:25:29.334 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.318+0000 7fb309ffb700 1 -- 172.21.15.143:0/1424455822 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb30c011c80 con 0x7fb3140b0110 2023-11-08T19:25:29.334 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.318+0000 7fb309ffb700 1 -- 172.21.15.143:0/1424455822 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb30c016470 con 0x7fb3140b0110 2023-11-08T19:25:29.334 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.318+0000 7fb31b5a3700 1 -- 172.21.15.143:0/1424455822 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3140b3550 con 0x7fb3140b0110 2023-11-08T19:25:29.335 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.318+0000 7fb31b5a3700 1 -- 172.21.15.143:0/1424455822 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb31414bf90 con 0x7fb3140b0110 2023-11-08T19:25:29.335 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.319+0000 7fb31b5a3700 1 -- 172.21.15.143:0/1424455822 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb31404eee0 con 0x7fb3140b0110 2023-11-08T19:25:29.335 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.320+0000 7fb309ffb700 1 -- 172.21.15.143:0/1424455822 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb30c0165f0 con 0x7fb3140b0110 2023-11-08T19:25:29.335 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.320+0000 7fb309ffb700 1 --2- 172.21.15.143:0/1424455822 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fb2f806af00 0x7fb2f806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:29.335 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.320+0000 7fb309ffb700 1 -- 172.21.15.143:0/1424455822 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb30c00f070 con 0x7fb3140b0110 2023-11-08T19:25:29.335 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.321+0000 7fb31933f700 1 --2- 172.21.15.143:0/1424455822 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fb2f806af00 0x7fb2f806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:29.336 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.321+0000 7fb31933f700 1 --2- 172.21.15.143:0/1424455822 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fb2f806af00 0x7fb2f806d3c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb3140b0de0 tx=0x7fb310008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:29.336 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.325+0000 7fb309ffb700 1 -- 172.21.15.143:0/1424455822 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb30c054f90 con 0x7fb3140b0110 2023-11-08T19:25:29.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.513+0000 7fb31b5a3700 1 -- 172.21.15.143:0/1424455822 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb3140b5600 con 0x7fb3140b0110 2023-11-08T19:25:29.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.514+0000 7fb309ffb700 1 -- 172.21.15.143:0/1424455822 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16553 (secure 0 0 0) 0x7fb30c01a090 con 0x7fb3140b0110 2023-11-08T19:25:29.516 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:25:29.516 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":20,"fsid":"968483af-77fc-4068-ad7c-5c2a1a4613d7","created":"2023-11-08T19:24:32.826484+0000","modified":"2023-11-08T19:24:51.618072+0000","last_up_change":"2023-11-08T19:24:43.573880+0000","last_in_change":"2023-11-08T19:24:35.314525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:44.551530+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-08T19:24:48.256781+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-08T19:24:48.954990+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bae98ec1-d94e-403d-a220-a522a53ea6f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6809","nonce":1487830566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6811","nonce":1487830566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6815","nonce":1487830566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6813","nonce":1487830566}]},"public_addr":"172.21.15.143:6809/1487830566","cluster_addr":"172.21.15.143:6811/1487830566","heartbeat_back_addr":"172.21.15.143:6815/1487830566","heartbeat_front_addr":"172.21.15.143:6813/1487830566","state":["exists","up"]},{"osd":1,"uuid":"5cb6538a-1ccb-4322-8633-09f66af0b7e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6801","nonce":787341397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6803","nonce":787341397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6807","nonce":787341397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6805","nonce":787341397}]},"public_addr":"172.21.15.143:6801/787341397","cluster_addr":"172.21.15.143:6803/787341397","heartbeat_back_addr":"172.21.15.143:6807/787341397","heartbeat_front_addr":"172.21.15.143:6805/787341397","state":["exists","up"]},{"osd":2,"uuid":"cdbf522c-60ef-4d60-9653-8dcb529c101a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6822","nonce":2301325597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6826","nonce":2301325597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6831","nonce":2301325597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6829","nonce":2301325597}]},"public_addr":"172.21.15.143:6822/2301325597","cluster_addr":"172.21.15.143:6826/2301325597","heartbeat_back_addr":"172.21.15.143:6831/2301325597","heartbeat_front_addr":"172.21.15.143:6829/2301325597","state":["exists","up"]},{"osd":3,"uuid":"4e2783cf-ebb9-4dd8-8245-bc1d17ce4325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6817","nonce":2021504428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6819","nonce":2021504428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6827","nonce":2021504428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6823","nonce":2021504428}]},"public_addr":"172.21.15.143:6817/2021504428","cluster_addr":"172.21.15.143:6819/2021504428","heartbeat_back_addr":"172.21.15.143:6827/2021504428","heartbeat_front_addr":"172.21.15.143:6823/2021504428","state":["exists","up"]},{"osd":4,"uuid":"691256fa-9b77-49b9-a346-7ae1adf0ca80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6801","nonce":1993380595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6803","nonce":1993380595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6807","nonce":1993380595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6805","nonce":1993380595}]},"public_addr":"172.21.15.174:6801/1993380595","cluster_addr":"172.21.15.174:6803/1993380595","heartbeat_back_addr":"172.21.15.174:6807/1993380595","heartbeat_front_addr":"172.21.15.174:6805/1993380595","state":["exists","up"]},{"osd":5,"uuid":"756fc352-48c9-46ce-b52e-0541fe6f8525","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6809","nonce":1777923303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6811","nonce":1777923303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6815","nonce":1777923303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6813","nonce":1777923303}]},"public_addr":"172.21.15.174:6809/1777923303","cluster_addr":"172.21.15.174:6811/1777923303","heartbeat_back_addr":"172.21.15.174:6815/1777923303","heartbeat_front_addr":"172.21.15.174:6813/1777923303","state":["exists","up"]},{"osd":6,"uuid":"fab2ac2b-ed97-4cde-9825-b480eabf8b8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6825","nonce":2755130316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6827","nonce":2755130316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6831","nonce":2755130316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6829","nonce":2755130316}]},"public_addr":"172.21.15.174:6825/2755130316","cluster_addr":"172.21.15.174:6827/2755130316","heartbeat_back_addr":"172.21.15.174:6831/2755130316","heartbeat_front_addr":"172.21.15.174:6829/2755130316","state":["exists","up"]},{"osd":7,"uuid":"3de9b797-2ab6-4d41-a863-7e7d638fd703","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6817","nonce":2847141411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6819","nonce":2847141411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6823","nonce":2847141411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6821","nonce":2847141411}]},"public_addr":"172.21.15.174:6817/2847141411","cluster_addr":"172.21.15.174:6819/2847141411","heartbeat_back_addr":"172.21.15.174:6823/2847141411","heartbeat_front_addr":"172.21.15.174:6821/2847141411","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.477076+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.496890+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.539305+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.454673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.538881+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.521294+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.520712+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.540404+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:25:29.518 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.516+0000 7fb31b5a3700 1 -- 172.21.15.143:0/1424455822 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fb2f806af00 msgr2=0x7fb2f806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:29.518 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.516+0000 7fb31b5a3700 1 --2- 172.21.15.143:0/1424455822 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fb2f806af00 0x7fb2f806d3c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb3140b0de0 tx=0x7fb310008040 comp rx=0 tx=0).stop 2023-11-08T19:25:29.519 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.516+0000 7fb31b5a3700 1 -- 172.21.15.143:0/1424455822 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb3140b0110 msgr2=0x7fb3140b05a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:29.519 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.516+0000 7fb31b5a3700 1 --2- 172.21.15.143:0/1424455822 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb3140b0110 0x7fb3140b05a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fb3140722b0 tx=0x7fb30c0114b0 comp rx=0 tx=0).stop 2023-11-08T19:25:29.519 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.516+0000 7fb31b5a3700 1 -- 172.21.15.143:0/1424455822 shutdown_connections 2023-11-08T19:25:29.519 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.516+0000 7fb31b5a3700 1 --2- 172.21.15.143:0/1424455822 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fb2f806af00 0x7fb2f806d3c0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:29.519 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.516+0000 7fb31b5a3700 1 --2- 172.21.15.143:0/1424455822 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb300005960 0x7fb3140b4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:29.520 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.516+0000 7fb31b5a3700 1 --2- 172.21.15.143:0/1424455822 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb3140b0110 0x7fb3140b05a0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:29.520 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.516+0000 7fb31b5a3700 1 --2- 172.21.15.143:0/1424455822 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb3140c8150 0x7fb31414ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:29.520 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.516+0000 7fb31b5a3700 1 -- 172.21.15.143:0/1424455822 >> 172.21.15.143:0/1424455822 conn(0x7fb31406b4c0 msgr2=0x7fb3140c5d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:29.520 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.516+0000 7fb31b5a3700 1 -- 172.21.15.143:0/1424455822 shutdown_connections 2023-11-08T19:25:29.520 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.516+0000 7fb31b5a3700 1 -- 172.21.15.143:0/1424455822 wait complete. 2023-11-08T19:25:29.530 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T19:24:44.551530+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-08T19:24:48.256781+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-08T19:24:48.954990+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-08T19:25:29.531 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T19:25:29.660 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.658+0000 7f15e51f6700 1 Processor -- start 2023-11-08T19:25:29.660 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.658+0000 7f15e51f6700 1 -- start start 2023-11-08T19:25:29.663 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.659+0000 7f15e51f6700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f15e015dba0 0x7f15e006c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:29.665 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.659+0000 7f15e51f6700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f15e006b240 0x7f15e00caff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:29.666 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.659+0000 7f15e51f6700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15e0161890 con 0x7f15e00cb530 2023-11-08T19:25:29.666 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.659+0000 7f15e51f6700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15e006b740 con 0x7f15e006b240 2023-11-08T19:25:29.666 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.659+0000 7f15e51f6700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15e006ba20 con 0x7f15e015dba0 2023-11-08T19:25:29.666 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.659+0000 7f15de59c700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f15e006b240 0x7f15e00caff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:29.666 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.659+0000 7f15de59c700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f15e006b240 0x7f15e00caff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49900/0 (socket says 172.21.15.143:49900) 2023-11-08T19:25:29.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.659+0000 7f15de59c700 1 -- 172.21.15.143:0/1950749354 learned_addr learned my addr 172.21.15.143:0/1950749354 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:29.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.659+0000 7f15ded9d700 1 --2- 172.21.15.143:0/1950749354 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f15e015dba0 0x7f15e006c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:29.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.659+0000 7f15de59c700 1 -- 172.21.15.143:0/1950749354 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f15e015dba0 msgr2=0x7f15e006c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:29.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.659+0000 7f15de59c700 1 --2- 172.21.15.143:0/1950749354 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f15e015dba0 0x7f15e006c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:29.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.659+0000 7f15de59c700 1 -- 172.21.15.143:0/1950749354 >> v1:172.21.15.174:6789/0 conn(0x7f15e00cb530 legacy=0x7f15e00b22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:29.668 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.659+0000 7f15de59c700 1 -- 172.21.15.143:0/1950749354 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15e006bd00 con 0x7f15e006b240 2023-11-08T19:25:29.668 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.659+0000 7f15de59c700 1 --2- 172.21.15.143:0/1950749354 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f15e006b240 0x7f15e00caff0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f15d40098f0 tx=0x7f15d4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51606d855dec5329 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:29.668 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.659+0000 7f15ddd9b700 1 -- 172.21.15.143:0/1950749354 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15d4004230 con 0x7f15e006b240 2023-11-08T19:25:29.668 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.659+0000 7f15ddd9b700 1 -- 172.21.15.143:0/1950749354 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15d4004390 con 0x7f15e006b240 2023-11-08T19:25:29.668 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.659+0000 7f15ddd9b700 1 -- 172.21.15.143:0/1950749354 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15d40036a0 con 0x7f15e006b240 2023-11-08T19:25:29.669 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.660+0000 7f15e51f6700 1 -- 172.21.15.143:0/1950749354 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f15e006b240 msgr2=0x7f15e00caff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:29.669 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.660+0000 7f15e51f6700 1 --2- 172.21.15.143:0/1950749354 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f15e006b240 0x7f15e00caff0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f15d40098f0 tx=0x7f15d4009c00 comp rx=0 tx=0).stop 2023-11-08T19:25:29.669 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.660+0000 7f15e51f6700 1 -- 172.21.15.143:0/1950749354 shutdown_connections 2023-11-08T19:25:29.669 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.660+0000 7f15e51f6700 1 --2- 172.21.15.143:0/1950749354 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f15e015dba0 0x7f15e006c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:29.669 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.660+0000 7f15e51f6700 1 --2- 172.21.15.143:0/1950749354 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f15e006b240 0x7f15e00caff0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:29.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.660+0000 7f15e51f6700 1 -- 172.21.15.143:0/1950749354 >> 172.21.15.143:0/1950749354 conn(0x7f15e00cd2e0 msgr2=0x7f15e00cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:29.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.660+0000 7f15e51f6700 1 -- 172.21.15.143:0/1950749354 shutdown_connections 2023-11-08T19:25:29.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.660+0000 7f15e51f6700 1 -- 172.21.15.143:0/1950749354 wait complete. 2023-11-08T19:25:29.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.661+0000 7f15e51f6700 1 Processor -- start 2023-11-08T19:25:29.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.661+0000 7f15e51f6700 1 -- start start 2023-11-08T19:25:29.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.662+0000 7f15e51f6700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f15e006b240 0x7f15e0160aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:29.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.662+0000 7f15e51f6700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f15e00cb530 0x7f15e0156470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:29.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.662+0000 7f15e51f6700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f15e015dba0 0x7f15e01f39f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:29.674 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.662+0000 7f15e51f6700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15e01f70d0 con 0x7f15e00cb530 2023-11-08T19:25:29.674 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.662+0000 7f15e51f6700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15e01f43a0 con 0x7f15e006b240 2023-11-08T19:25:29.674 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.662+0000 7f15e51f6700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15e01f4680 con 0x7f15e015dba0 2023-11-08T19:25:29.674 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.662+0000 7f15de59c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f15e00cb530 0x7f15e0156470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:29.674 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.662+0000 7f15de59c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f15e00cb530 0x7f15e0156470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49906/0 (socket says 172.21.15.143:49906) 2023-11-08T19:25:29.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.662+0000 7f15de59c700 1 -- 172.21.15.143:0/3901640855 learned_addr learned my addr 172.21.15.143:0/3901640855 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:29.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.663+0000 7f15ded9d700 1 --2- 172.21.15.143:0/3901640855 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f15e006b240 0x7f15e0160aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:29.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.663+0000 7f15df59e700 1 --2- 172.21.15.143:0/3901640855 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f15e015dba0 0x7f15e01f39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:29.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.663+0000 7f15de59c700 1 -- 172.21.15.143:0/3901640855 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f15e015dba0 msgr2=0x7f15e01f39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:29.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.663+0000 7f15de59c700 1 --2- 172.21.15.143:0/3901640855 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f15e015dba0 0x7f15e01f39f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:29.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.663+0000 7f15de59c700 1 -- 172.21.15.143:0/3901640855 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f15e006b240 msgr2=0x7f15e0160aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:29.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.663+0000 7f15de59c700 1 --2- 172.21.15.143:0/3901640855 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f15e006b240 0x7f15e0160aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:29.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.663+0000 7f15de59c700 1 -- 172.21.15.143:0/3901640855 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15d4009550 con 0x7f15e00cb530 2023-11-08T19:25:29.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.663+0000 7f15df59e700 1 --2- 172.21.15.143:0/3901640855 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f15e015dba0 0x7f15e01f39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:25:29.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.663+0000 7f15de59c700 1 --2- 172.21.15.143:0/3901640855 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f15e00cb530 0x7f15e0156470 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f15d4009130 tx=0x7f15d4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:29.677 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.663+0000 7f15c77fe700 1 -- 172.21.15.143:0/3901640855 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15d4003c00 con 0x7f15e00cb530 2023-11-08T19:25:29.677 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.663+0000 7f15c77fe700 1 -- 172.21.15.143:0/3901640855 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15d401f070 con 0x7f15e00cb530 2023-11-08T19:25:29.677 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.663+0000 7f15e51f6700 1 -- 172.21.15.143:0/3901640855 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15e01f4960 con 0x7f15e00cb530 2023-11-08T19:25:29.677 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.663+0000 7f15c77fe700 1 -- 172.21.15.143:0/3901640855 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15d400b660 con 0x7f15e00cb530 2023-11-08T19:25:29.677 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.663+0000 7f15e51f6700 1 -- 172.21.15.143:0/3901640855 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15e01569b0 con 0x7f15e00cb530 2023-11-08T19:25:29.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.664+0000 7f15e51f6700 1 -- 172.21.15.143:0/3901640855 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15e01f8060 con 0x7f15e00cb530 2023-11-08T19:25:29.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.664+0000 7f15c77fe700 1 -- 172.21.15.143:0/3901640855 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f15d4021030 con 0x7f15e00cb530 2023-11-08T19:25:29.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.665+0000 7f15c77fe700 1 --2- 172.21.15.143:0/3901640855 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f15cc06af00 0x7f15cc06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:29.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.665+0000 7f15c77fe700 1 -- 172.21.15.143:0/3901640855 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f15d408aba0 con 0x7f15e00cb530 2023-11-08T19:25:29.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.665+0000 7f15ded9d700 1 --2- 172.21.15.143:0/3901640855 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f15cc06af00 0x7f15cc06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:29.679 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.665+0000 7f15ded9d700 1 --2- 172.21.15.143:0/3901640855 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f15cc06af00 0x7f15cc06d3c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f15d000b9b0 tx=0x7f15d0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:29.679 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.669+0000 7f15c77fe700 1 -- 172.21.15.143:0/3901640855 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f15e01f8060 con 0x7f15e00cb530 2023-11-08T19:25:29.860 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.858+0000 7f15e51f6700 1 -- 172.21.15.143:0/3901640855 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f15e01f8060 con 0x7f15e00cb530 2023-11-08T19:25:29.860 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.859+0000 7f15c77fe700 1 -- 172.21.15.143:0/3901640855 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f15e01f8060 con 0x7f15e00cb530 2023-11-08T19:25:29.861 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2023-11-08T19:25:29.862 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.861+0000 7f15e51f6700 1 -- 172.21.15.143:0/3901640855 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f15cc06af00 msgr2=0x7f15cc06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:29.863 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.861+0000 7f15e51f6700 1 --2- 172.21.15.143:0/3901640855 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f15cc06af00 0x7f15cc06d3c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f15d000b9b0 tx=0x7f15d0008040 comp rx=0 tx=0).stop 2023-11-08T19:25:29.863 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.861+0000 7f15e51f6700 1 -- 172.21.15.143:0/3901640855 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f15e00cb530 msgr2=0x7f15e0156470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:29.863 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.861+0000 7f15e51f6700 1 --2- 172.21.15.143:0/3901640855 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f15e00cb530 0x7f15e0156470 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f15d4009130 tx=0x7f15d4004750 comp rx=0 tx=0).stop 2023-11-08T19:25:29.863 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.861+0000 7f15e51f6700 1 -- 172.21.15.143:0/3901640855 shutdown_connections 2023-11-08T19:25:29.864 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.861+0000 7f15e51f6700 1 --2- 172.21.15.143:0/3901640855 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f15cc06af00 0x7f15cc06d3c0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:29.864 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.861+0000 7f15e51f6700 1 --2- 172.21.15.143:0/3901640855 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f15e015dba0 0x7f15e01f39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:29.864 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.861+0000 7f15e51f6700 1 --2- 172.21.15.143:0/3901640855 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f15e006b240 0x7f15e0160aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:29.864 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.861+0000 7f15e51f6700 1 --2- 172.21.15.143:0/3901640855 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f15e00cb530 0x7f15e0156470 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:29.864 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.861+0000 7f15e51f6700 1 -- 172.21.15.143:0/3901640855 >> 172.21.15.143:0/3901640855 conn(0x7f15e00cd2e0 msgr2=0x7f15e015cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:29.864 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.861+0000 7f15e51f6700 1 -- 172.21.15.143:0/3901640855 shutdown_connections 2023-11-08T19:25:29.865 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.861+0000 7f15e51f6700 1 -- 172.21.15.143:0/3901640855 wait complete. 2023-11-08T19:25:29.875 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-08T19:25:29.999 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.997+0000 7f19ad649700 1 Processor -- start 2023-11-08T19:25:29.999 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.997+0000 7f19ad649700 1 -- start start 2023-11-08T19:25:30.009 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.998+0000 7f19ad649700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f19a815bc80 0x7f19a80b6280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:30.009 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.998+0000 7f19ad649700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f19a806b0b0 0x7f19a806c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:30.009 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.998+0000 7f19ad649700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19a80b67c0 con 0x7f19a80b22e0 2023-11-08T19:25:30.010 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.998+0000 7f19ad649700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19a80ce290 con 0x7f19a806b0b0 2023-11-08T19:25:30.010 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.998+0000 7f19ad649700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19a815c060 con 0x7f19a815bc80 2023-11-08T19:25:30.011 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.998+0000 7f19a67fc700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f19a806b0b0 0x7f19a806c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:30.011 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.998+0000 7f19a67fc700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f19a806b0b0 0x7f19a806c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49916/0 (socket says 172.21.15.143:49916) 2023-11-08T19:25:30.011 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.998+0000 7f19a67fc700 1 -- 172.21.15.143:0/4083373668 learned_addr learned my addr 172.21.15.143:0/4083373668 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:30.011 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.999+0000 7f19a6ffd700 1 --2- 172.21.15.143:0/4083373668 >> v2:172.21.15.174:3300/0 conn(0x7f19a815bc80 0x7f19a80b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:30.012 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.999+0000 7f19a67fc700 1 -- 172.21.15.143:0/4083373668 >> v2:172.21.15.174:3300/0 conn(0x7f19a815bc80 msgr2=0x7f19a80b6280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:30.012 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.999+0000 7f19a67fc700 1 --2- 172.21.15.143:0/4083373668 >> v2:172.21.15.174:3300/0 conn(0x7f19a815bc80 0x7f19a80b6280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:30.012 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.999+0000 7f19a67fc700 1 -- 172.21.15.143:0/4083373668 >> v1:172.21.15.174:6789/0 conn(0x7f19a80b22e0 legacy=0x7f19a80b26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:30.012 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.999+0000 7f19a67fc700 1 -- 172.21.15.143:0/4083373668 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19a806cc50 con 0x7f19a806b0b0 2023-11-08T19:25:30.013 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.999+0000 7f19a67fc700 1 --2- 172.21.15.143:0/4083373668 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f19a806b0b0 0x7f19a806c710 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f19a80673b0 tx=0x7f1998009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e183cab4a0d6af22 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:30.013 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.999+0000 7f19a5ffb700 1 -- 172.21.15.143:0/4083373668 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1998004230 con 0x7f19a806b0b0 2023-11-08T19:25:30.013 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.999+0000 7f19a5ffb700 1 -- 172.21.15.143:0/4083373668 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1998004390 con 0x7f19a806b0b0 2023-11-08T19:25:30.014 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:29.999+0000 7f19a5ffb700 1 -- 172.21.15.143:0/4083373668 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19980036a0 con 0x7f19a806b0b0 2023-11-08T19:25:30.014 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.000+0000 7f19ad649700 1 -- 172.21.15.143:0/4083373668 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f19a806b0b0 msgr2=0x7f19a806c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:30.014 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.000+0000 7f19ad649700 1 --2- 172.21.15.143:0/4083373668 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f19a806b0b0 0x7f19a806c710 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f19a80673b0 tx=0x7f1998009bd0 comp rx=0 tx=0).stop 2023-11-08T19:25:30.015 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.000+0000 7f19ad649700 1 -- 172.21.15.143:0/4083373668 shutdown_connections 2023-11-08T19:25:30.015 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.000+0000 7f19ad649700 1 --2- 172.21.15.143:0/4083373668 >> v2:172.21.15.174:3300/0 conn(0x7f19a815bc80 0x7f19a80b6280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:30.015 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.000+0000 7f19ad649700 1 --2- 172.21.15.143:0/4083373668 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f19a806b0b0 0x7f19a806c710 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:30.015 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.000+0000 7f19ad649700 1 -- 172.21.15.143:0/4083373668 >> 172.21.15.143:0/4083373668 conn(0x7f19a80cd2e0 msgr2=0x7f19a80cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:30.015 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.000+0000 7f19ad649700 1 -- 172.21.15.143:0/4083373668 shutdown_connections 2023-11-08T19:25:30.016 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.000+0000 7f19ad649700 1 -- 172.21.15.143:0/4083373668 wait complete. 2023-11-08T19:25:30.016 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.001+0000 7f19ad649700 1 Processor -- start 2023-11-08T19:25:30.016 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.001+0000 7f19ad649700 1 -- start start 2023-11-08T19:25:30.016 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.002+0000 7f19ad649700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f19a806b0b0 0x7f19a81f7580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:30.016 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.002+0000 7f19ad649700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f19a80b22e0 0x7f19a80cf6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:30.017 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.002+0000 7f19ad649700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f19a815bc80 0x7f19a81527f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:30.017 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.002+0000 7f19ad649700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19a80c5aa0 con 0x7f19a806b0b0 2023-11-08T19:25:30.017 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.002+0000 7f19ad649700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19a80d0080 con 0x7f19a80b22e0 2023-11-08T19:25:30.017 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.002+0000 7f19ad649700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19a80d0360 con 0x7f19a815bc80 2023-11-08T19:25:30.018 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.002+0000 7f19a6ffd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f19a806b0b0 0x7f19a81f7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:30.018 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.002+0000 7f19a6ffd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f19a806b0b0 0x7f19a81f7580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49918/0 (socket says 172.21.15.143:49918) 2023-11-08T19:25:30.018 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.002+0000 7f19a6ffd700 1 -- 172.21.15.143:0/2114392600 learned_addr learned my addr 172.21.15.143:0/2114392600 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:30.018 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.002+0000 7f19a67fc700 1 --2- 172.21.15.143:0/2114392600 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f19a80b22e0 0x7f19a80cf6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:30.018 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.002+0000 7f19a77fe700 1 --2- 172.21.15.143:0/2114392600 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f19a815bc80 0x7f19a81527f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:30.019 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.002+0000 7f19a6ffd700 1 -- 172.21.15.143:0/2114392600 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f19a815bc80 msgr2=0x7f19a81527f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:30.019 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.002+0000 7f19a6ffd700 1 --2- 172.21.15.143:0/2114392600 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f19a815bc80 0x7f19a81527f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:30.019 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.002+0000 7f19a6ffd700 1 -- 172.21.15.143:0/2114392600 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f19a80b22e0 msgr2=0x7f19a80cf6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:30.019 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.002+0000 7f19a6ffd700 1 --2- 172.21.15.143:0/2114392600 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f19a80b22e0 0x7f19a80cf6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:30.019 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.002+0000 7f19a6ffd700 1 -- 172.21.15.143:0/2114392600 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1998009550 con 0x7f19a806b0b0 2023-11-08T19:25:30.020 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.002+0000 7f19a67fc700 1 --2- 172.21.15.143:0/2114392600 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f19a80b22e0 0x7f19a80cf6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:25:30.020 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.002+0000 7f19a6ffd700 1 --2- 172.21.15.143:0/2114392600 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f19a806b0b0 0x7f19a81f7580 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f199c00c7f0 tx=0x7f199c00cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:30.020 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.002+0000 7f198f7fe700 1 -- 172.21.15.143:0/2114392600 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f199c011800 con 0x7f19a806b0b0 2023-11-08T19:25:30.020 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.002+0000 7f198f7fe700 1 -- 172.21.15.143:0/2114392600 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f199c011960 con 0x7f19a806b0b0 2023-11-08T19:25:30.020 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.002+0000 7f19ad649700 1 -- 172.21.15.143:0/2114392600 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19a80c53e0 con 0x7f19a806b0b0 2023-11-08T19:25:30.021 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.002+0000 7f198f7fe700 1 -- 172.21.15.143:0/2114392600 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f199c011cf0 con 0x7f19a806b0b0 2023-11-08T19:25:30.021 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.002+0000 7f19ad649700 1 -- 172.21.15.143:0/2114392600 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19a81f7ac0 con 0x7f19a806b0b0 2023-11-08T19:25:30.021 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.004+0000 7f19ad649700 1 -- 172.21.15.143:0/2114392600 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19a81f7cb0 con 0x7f19a806b0b0 2023-11-08T19:25:30.021 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.004+0000 7f198f7fe700 1 -- 172.21.15.143:0/2114392600 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f199c018360 con 0x7f19a806b0b0 2023-11-08T19:25:30.021 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.004+0000 7f198f7fe700 1 --2- 172.21.15.143:0/2114392600 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f199406af00 0x7f199406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:30.021 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.004+0000 7f198f7fe700 1 -- 172.21.15.143:0/2114392600 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f199c089570 con 0x7f19a806b0b0 2023-11-08T19:25:30.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.004+0000 7f19a67fc700 1 --2- 172.21.15.143:0/2114392600 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f199406af00 0x7f199406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:30.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.005+0000 7f19a67fc700 1 --2- 172.21.15.143:0/2114392600 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f199406af00 0x7f199406d3c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f1998009130 tx=0x7f1998008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:30.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.009+0000 7f198f7fe700 1 -- 172.21.15.143:0/2114392600 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f199c0545a0 con 0x7f19a806b0b0 2023-11-08T19:25:30.199 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.197+0000 7f19ad649700 1 -- 172.21.15.143:0/2114392600 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f19a804eee0 con 0x7f19a806b0b0 2023-11-08T19:25:30.199 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.197+0000 7f198f7fe700 1 -- 172.21.15.143:0/2114392600 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f199c057bc0 con 0x7f19a806b0b0 2023-11-08T19:25:30.199 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2023-11-08T19:25:30.201 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.199+0000 7f19ad649700 1 -- 172.21.15.143:0/2114392600 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f199406af00 msgr2=0x7f199406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:30.201 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.199+0000 7f19ad649700 1 --2- 172.21.15.143:0/2114392600 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f199406af00 0x7f199406d3c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f1998009130 tx=0x7f1998008030 comp rx=0 tx=0).stop 2023-11-08T19:25:30.201 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.199+0000 7f19ad649700 1 -- 172.21.15.143:0/2114392600 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f19a806b0b0 msgr2=0x7f19a81f7580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:30.202 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.199+0000 7f19ad649700 1 --2- 172.21.15.143:0/2114392600 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f19a806b0b0 0x7f19a81f7580 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f199c00c7f0 tx=0x7f199c00cbb0 comp rx=0 tx=0).stop 2023-11-08T19:25:30.202 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.199+0000 7f19ad649700 1 -- 172.21.15.143:0/2114392600 shutdown_connections 2023-11-08T19:25:30.202 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.199+0000 7f19ad649700 1 --2- 172.21.15.143:0/2114392600 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f199406af00 0x7f199406d3c0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:30.202 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.199+0000 7f19ad649700 1 --2- 172.21.15.143:0/2114392600 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f19a815bc80 0x7f19a81527f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:30.202 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.199+0000 7f19ad649700 1 --2- 172.21.15.143:0/2114392600 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f19a806b0b0 0x7f19a81f7580 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:30.203 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.199+0000 7f19ad649700 1 --2- 172.21.15.143:0/2114392600 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f19a80b22e0 0x7f19a80cf6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:30.203 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.199+0000 7f19ad649700 1 -- 172.21.15.143:0/2114392600 >> 172.21.15.143:0/2114392600 conn(0x7f19a80cd2e0 msgr2=0x7f19a80b3620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:30.203 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.200+0000 7f19ad649700 1 -- 172.21.15.143:0/2114392600 shutdown_connections 2023-11-08T19:25:30.203 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.200+0000 7f19ad649700 1 -- 172.21.15.143:0/2114392600 wait complete. 2023-11-08T19:25:30.213 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-08T19:25:30.337 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.335+0000 7fb79b924700 1 Processor -- start 2023-11-08T19:25:30.337 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.335+0000 7fb79b924700 1 -- start start 2023-11-08T19:25:30.339 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.336+0000 7fb79b924700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb79406b240 0x7fb7940caff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:30.339 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.336+0000 7fb79b924700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fb7940cb530 0x7fb7940b22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:30.339 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.336+0000 7fb79b924700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb794161890 con 0x7fb79415dba0 2023-11-08T19:25:30.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.336+0000 7fb79b924700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb79406cd20 con 0x7fb79406b240 2023-11-08T19:25:30.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.336+0000 7fb79b924700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb79406b740 con 0x7fb7940cb530 2023-11-08T19:25:30.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.336+0000 7fb798ebf700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb79406b240 0x7fb7940caff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:30.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.336+0000 7fb798ebf700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb79406b240 0x7fb7940caff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49928/0 (socket says 172.21.15.143:49928) 2023-11-08T19:25:30.341 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.336+0000 7fb798ebf700 1 -- 172.21.15.143:0/3516743078 learned_addr learned my addr 172.21.15.143:0/3516743078 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:30.341 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.337+0000 7fb799ec1700 1 --2- 172.21.15.143:0/3516743078 >> v2:172.21.15.174:3300/0 conn(0x7fb7940cb530 0x7fb7940b22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:30.341 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.337+0000 7fb798ebf700 1 -- 172.21.15.143:0/3516743078 >> v2:172.21.15.174:3300/0 conn(0x7fb7940cb530 msgr2=0x7fb7940b22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:30.341 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.337+0000 7fb798ebf700 1 --2- 172.21.15.143:0/3516743078 >> v2:172.21.15.174:3300/0 conn(0x7fb7940cb530 0x7fb7940b22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:30.341 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.337+0000 7fb798ebf700 1 -- 172.21.15.143:0/3516743078 >> v1:172.21.15.174:6789/0 conn(0x7fb79415dba0 legacy=0x7fb79406c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:30.342 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.337+0000 7fb798ebf700 1 -- 172.21.15.143:0/3516743078 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb79406ba20 con 0x7fb79406b240 2023-11-08T19:25:30.342 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.337+0000 7fb798ebf700 1 --2- 172.21.15.143:0/3516743078 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb79406b240 0x7fb7940caff0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fb79406c2c0 tx=0x7fb790009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aba6cbee7894bbb4 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:30.342 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.337+0000 7fb78bfff700 1 -- 172.21.15.143:0/3516743078 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb790004230 con 0x7fb79406b240 2023-11-08T19:25:30.342 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.337+0000 7fb78bfff700 1 -- 172.21.15.143:0/3516743078 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb790004390 con 0x7fb79406b240 2023-11-08T19:25:30.343 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.337+0000 7fb78bfff700 1 -- 172.21.15.143:0/3516743078 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7900036a0 con 0x7fb79406b240 2023-11-08T19:25:30.343 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.337+0000 7fb79b924700 1 -- 172.21.15.143:0/3516743078 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb79406b240 msgr2=0x7fb7940caff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:30.343 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.337+0000 7fb79b924700 1 --2- 172.21.15.143:0/3516743078 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb79406b240 0x7fb7940caff0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fb79406c2c0 tx=0x7fb790009bd0 comp rx=0 tx=0).stop 2023-11-08T19:25:30.343 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.337+0000 7fb79b924700 1 -- 172.21.15.143:0/3516743078 shutdown_connections 2023-11-08T19:25:30.343 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.337+0000 7fb79b924700 1 --2- 172.21.15.143:0/3516743078 >> v2:172.21.15.174:3300/0 conn(0x7fb7940cb530 0x7fb7940b22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:30.344 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.337+0000 7fb79b924700 1 --2- 172.21.15.143:0/3516743078 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb79406b240 0x7fb7940caff0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:30.344 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.337+0000 7fb79b924700 1 -- 172.21.15.143:0/3516743078 >> 172.21.15.143:0/3516743078 conn(0x7fb7940cd2e0 msgr2=0x7fb7940cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:30.344 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.338+0000 7fb79b924700 1 -- 172.21.15.143:0/3516743078 shutdown_connections 2023-11-08T19:25:30.344 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.338+0000 7fb79b924700 1 -- 172.21.15.143:0/3516743078 wait complete. 2023-11-08T19:25:30.347 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.339+0000 7fb79b924700 1 Processor -- start 2023-11-08T19:25:30.347 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.339+0000 7fb79b924700 1 -- start start 2023-11-08T19:25:30.348 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.339+0000 7fb79b924700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb79406b240 0x7fb79415ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:30.348 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.339+0000 7fb79b924700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb7940cb530 0x7fb794160080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:30.348 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.339+0000 7fb79b924700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb79415dba0 0x7fb7941605c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:30.348 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.339+0000 7fb79b924700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7941613b0 con 0x7fb7940cb530 2023-11-08T19:25:30.348 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.340+0000 7fb79b924700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb79415d390 con 0x7fb79406b240 2023-11-08T19:25:30.349 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.340+0000 7fb79b924700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7940c6300 con 0x7fb79415dba0 2023-11-08T19:25:30.349 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.340+0000 7fb798ebf700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb7940cb530 0x7fb794160080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:30.349 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.340+0000 7fb798ebf700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb7940cb530 0x7fb794160080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49934/0 (socket says 172.21.15.143:49934) 2023-11-08T19:25:30.349 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.340+0000 7fb798ebf700 1 -- 172.21.15.143:0/2566807828 learned_addr learned my addr 172.21.15.143:0/2566807828 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:30.349 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.340+0000 7fb799ec1700 1 --2- 172.21.15.143:0/2566807828 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb79415dba0 0x7fb7941605c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:30.350 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.340+0000 7fb7996c0700 1 --2- 172.21.15.143:0/2566807828 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb79406b240 0x7fb79415ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:30.350 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.340+0000 7fb798ebf700 1 -- 172.21.15.143:0/2566807828 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb79415dba0 msgr2=0x7fb7941605c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:30.350 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.340+0000 7fb798ebf700 1 --2- 172.21.15.143:0/2566807828 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb79415dba0 0x7fb7941605c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:30.350 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.340+0000 7fb798ebf700 1 -- 172.21.15.143:0/2566807828 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb79406b240 msgr2=0x7fb79415ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:30.350 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.340+0000 7fb798ebf700 1 --2- 172.21.15.143:0/2566807828 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb79406b240 0x7fb79415ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:30.351 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.340+0000 7fb798ebf700 1 -- 172.21.15.143:0/2566807828 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb790009550 con 0x7fb7940cb530 2023-11-08T19:25:30.351 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.340+0000 7fb7996c0700 1 --2- 172.21.15.143:0/2566807828 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb79406b240 0x7fb79415ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:25:30.351 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.340+0000 7fb798ebf700 1 --2- 172.21.15.143:0/2566807828 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb7940cb530 0x7fb794160080 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb794067530 tx=0x7fb790004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:30.351 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.340+0000 7fb789ffb700 1 -- 172.21.15.143:0/2566807828 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb790011630 con 0x7fb7940cb530 2023-11-08T19:25:30.351 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.340+0000 7fb789ffb700 1 -- 172.21.15.143:0/2566807828 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb790011790 con 0x7fb7940cb530 2023-11-08T19:25:30.352 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.340+0000 7fb79b924700 1 -- 172.21.15.143:0/2566807828 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7940c6550 con 0x7fb7940cb530 2023-11-08T19:25:30.352 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.340+0000 7fb789ffb700 1 -- 172.21.15.143:0/2566807828 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb790011b40 con 0x7fb7940cb530 2023-11-08T19:25:30.352 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.340+0000 7fb79b924700 1 -- 172.21.15.143:0/2566807828 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7940c6b10 con 0x7fb7940cb530 2023-11-08T19:25:30.352 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.341+0000 7fb79b924700 1 -- 172.21.15.143:0/2566807828 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7940c7910 con 0x7fb7940cb530 2023-11-08T19:25:30.353 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.342+0000 7fb789ffb700 1 -- 172.21.15.143:0/2566807828 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb790011ca0 con 0x7fb7940cb530 2023-11-08T19:25:30.353 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.342+0000 7fb789ffb700 1 --2- 172.21.15.143:0/2566807828 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fb77806af00 0x7fb77806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:30.353 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.342+0000 7fb789ffb700 1 -- 172.21.15.143:0/2566807828 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb79008ab10 con 0x7fb7940cb530 2023-11-08T19:25:30.353 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.342+0000 7fb7996c0700 1 --2- 172.21.15.143:0/2566807828 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fb77806af00 0x7fb77806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:30.353 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.343+0000 7fb7996c0700 1 --2- 172.21.15.143:0/2566807828 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fb77806af00 0x7fb77806d3c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fb7940c94c0 tx=0x7fb784009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:30.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.347+0000 7fb789ffb700 1 -- 172.21.15.143:0/2566807828 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb790055d00 con 0x7fb7940cb530 2023-11-08T19:25:30.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.536+0000 7fb79b924700 1 -- 172.21.15.143:0/2566807828 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb79404eee0 con 0x7fb7940cb530 2023-11-08T19:25:30.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.536+0000 7fb789ffb700 1 -- 172.21.15.143:0/2566807828 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb790059320 con 0x7fb7940cb530 2023-11-08T19:25:30.538 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 64 2023-11-08T19:25:30.540 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.538+0000 7fb79b924700 1 -- 172.21.15.143:0/2566807828 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fb77806af00 msgr2=0x7fb77806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:30.540 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.538+0000 7fb79b924700 1 --2- 172.21.15.143:0/2566807828 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fb77806af00 0x7fb77806d3c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fb7940c94c0 tx=0x7fb784009040 comp rx=0 tx=0).stop 2023-11-08T19:25:30.540 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.538+0000 7fb79b924700 1 -- 172.21.15.143:0/2566807828 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb7940cb530 msgr2=0x7fb794160080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:30.541 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.538+0000 7fb79b924700 1 --2- 172.21.15.143:0/2566807828 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb7940cb530 0x7fb794160080 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb794067530 tx=0x7fb790004750 comp rx=0 tx=0).stop 2023-11-08T19:25:30.541 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.538+0000 7fb79b924700 1 -- 172.21.15.143:0/2566807828 shutdown_connections 2023-11-08T19:25:30.541 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.538+0000 7fb79b924700 1 --2- 172.21.15.143:0/2566807828 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fb77806af00 0x7fb77806d3c0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:30.541 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.538+0000 7fb79b924700 1 --2- 172.21.15.143:0/2566807828 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb79415dba0 0x7fb7941605c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:30.541 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.538+0000 7fb79b924700 1 --2- 172.21.15.143:0/2566807828 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb79406b240 0x7fb79415ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:30.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.538+0000 7fb79b924700 1 --2- 172.21.15.143:0/2566807828 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb7940cb530 0x7fb794160080 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:30.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.538+0000 7fb79b924700 1 -- 172.21.15.143:0/2566807828 >> 172.21.15.143:0/2566807828 conn(0x7fb7940cd2e0 msgr2=0x7fb7941e83d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:30.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.538+0000 7fb79b924700 1 -- 172.21.15.143:0/2566807828 shutdown_connections 2023-11-08T19:25:30.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.538+0000 7fb79b924700 1 -- 172.21.15.143:0/2566807828 wait complete. 2023-11-08T19:25:30.552 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-08T19:25:30.677 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.676+0000 7f6836889700 1 Processor -- start 2023-11-08T19:25:30.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.676+0000 7f6836889700 1 -- start start 2023-11-08T19:25:30.681 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.677+0000 7f6836889700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f683015dd00 0x7f68300cca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:30.682 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.677+0000 7f6836889700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f683006b320 0x7f68300cb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:30.682 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.677+0000 7f6836889700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68300ccfb0 con 0x7f68301618f0 2023-11-08T19:25:30.682 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.677+0000 7f6836889700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68300cad50 con 0x7f683015dd00 2023-11-08T19:25:30.682 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.677+0000 7f6836889700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6830161df0 con 0x7f683006b320 2023-11-08T19:25:30.682 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.677+0000 7f682f7fe700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f68301618f0 0x7f683006ac10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:46094/0 (socket says 172.21.15.143:46094) 2023-11-08T19:25:30.683 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.677+0000 7f682f7fe700 1 -- 172.21.15.143:0/3494148611 learned_addr learned my addr 172.21.15.143:0/3494148611 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:25:30.683 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.677+0000 7f6834e26700 1 --2- 172.21.15.143:0/3494148611 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f683006b320 0x7f68300cb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:30.683 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.677+0000 7f682effd700 1 -- 172.21.15.143:0/3494148611 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4285992022 0 0) 0x7f68300ccfb0 con 0x7f68301618f0 2023-11-08T19:25:30.683 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.677+0000 7f682effd700 1 -- 172.21.15.143:0/3494148611 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68180035b0 con 0x7f68301618f0 2023-11-08T19:25:30.683 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.677+0000 7f682ffff700 1 --2- 172.21.15.143:0/3494148611 >> v2:172.21.15.174:3300/0 conn(0x7f683015dd00 0x7f68300cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:30.684 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.677+0000 7f682effd700 1 -- 172.21.15.143:0/3494148611 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1473307595 0 0) 0x7f68180035b0 con 0x7f68301618f0 2023-11-08T19:25:30.685 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.677+0000 7f682effd700 1 -- 172.21.15.143:0/3494148611 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f683006b320 msgr2=0x7f68300cb1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:30.686 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.677+0000 7f682effd700 1 --2- 172.21.15.143:0/3494148611 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f683006b320 0x7f68300cb1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:30.686 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.677+0000 7f682effd700 1 -- 172.21.15.143:0/3494148611 >> v2:172.21.15.174:3300/0 conn(0x7f683015dd00 msgr2=0x7f68300cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:30.686 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.677+0000 7f682effd700 1 --2- 172.21.15.143:0/3494148611 >> v2:172.21.15.174:3300/0 conn(0x7f683015dd00 0x7f68300cca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:30.686 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.677+0000 7f682effd700 1 -- 172.21.15.143:0/3494148611 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68301620d0 con 0x7f68301618f0 2023-11-08T19:25:30.686 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.677+0000 7f682effd700 1 -- 172.21.15.143:0/3494148611 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4147756303 0 0) 0x7f6820001670 con 0x7f68301618f0 2023-11-08T19:25:30.687 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.677+0000 7f682effd700 1 -- 172.21.15.143:0/3494148611 >> v1:172.21.15.143:6789/0 conn(0x7f68301618f0 legacy=0x7f683006ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:30.687 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.677+0000 7f682effd700 1 --2- 172.21.15.143:0/3494148611 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f683006b320 0x7f6818003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:30.687 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.677+0000 7f682effd700 1 --2- 172.21.15.143:0/3494148611 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6818003f80 0x7f6818004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:30.687 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.677+0000 7f682effd700 1 --2- 172.21.15.143:0/3494148611 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6818004d90 0x7f6818005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:30.687 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.678+0000 7f682effd700 1 -- 172.21.15.143:0/3494148611 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6830161df0 con 0x7f6818004d90 2023-11-08T19:25:30.687 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.678+0000 7f682effd700 1 -- 172.21.15.143:0/3494148611 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f681800ba10 con 0x7f6818003f80 2023-11-08T19:25:30.688 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.678+0000 7f682effd700 1 -- 172.21.15.143:0/3494148611 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f681800bb50 con 0x7f683006b320 2023-11-08T19:25:30.688 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.678+0000 7f6834e26700 1 --2- 172.21.15.143:0/3494148611 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6818004d90 0x7f6818005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:30.688 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.678+0000 7f682f7fe700 1 --2- 172.21.15.143:0/3494148611 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6818003f80 0x7f6818004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:30.688 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.678+0000 7f682ffff700 1 --2- 172.21.15.143:0/3494148611 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f683006b320 0x7f6818003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:30.688 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.678+0000 7f6834e26700 1 -- 172.21.15.143:0/3494148611 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f683006b320 msgr2=0x7f6818003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:30.689 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.678+0000 7f6834e26700 1 --2- 172.21.15.143:0/3494148611 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f683006b320 0x7f6818003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:30.689 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.678+0000 7f6834e26700 1 -- 172.21.15.143:0/3494148611 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6818003f80 msgr2=0x7f6818004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:30.689 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.678+0000 7f6834e26700 1 --2- 172.21.15.143:0/3494148611 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6818003f80 0x7f6818004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:30.689 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.678+0000 7f6834e26700 1 -- 172.21.15.143:0/3494148611 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68180035b0 con 0x7f6818004d90 2023-11-08T19:25:30.689 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.678+0000 7f6834e26700 1 --2- 172.21.15.143:0/3494148611 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6818004d90 0x7f6818005210 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f6828009f90 tx=0x7f6818005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91f2bd2be6fa93f8 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:30.690 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.678+0000 7f682effd700 1 -- 172.21.15.143:0/3494148611 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f682800eb70 con 0x7f6818004d90 2023-11-08T19:25:30.690 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.678+0000 7f682effd700 1 -- 172.21.15.143:0/3494148611 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f682800f040 con 0x7f6818004d90 2023-11-08T19:25:30.690 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.678+0000 7f682effd700 1 -- 172.21.15.143:0/3494148611 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6828014480 con 0x7f6818004d90 2023-11-08T19:25:30.690 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.679+0000 7f6836889700 1 -- 172.21.15.143:0/3494148611 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6818004d90 msgr2=0x7f6818005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:30.690 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.679+0000 7f6836889700 1 --2- 172.21.15.143:0/3494148611 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6818004d90 0x7f6818005210 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f6828009f90 tx=0x7f6818005880 comp rx=0 tx=0).stop 2023-11-08T19:25:30.691 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.679+0000 7f6834e26700 1 -- 172.21.15.143:0/3494148611 reap_dead start 2023-11-08T19:25:30.691 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.679+0000 7f6836889700 1 -- 172.21.15.143:0/3494148611 shutdown_connections 2023-11-08T19:25:30.691 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.679+0000 7f6836889700 1 -- 172.21.15.143:0/3494148611 >> 172.21.15.143:0/3494148611 conn(0x7f6830099ab0 msgr2=0x7f68300d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:30.691 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.679+0000 7f6836889700 1 -- 172.21.15.143:0/3494148611 shutdown_connections 2023-11-08T19:25:30.691 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.679+0000 7f6836889700 1 -- 172.21.15.143:0/3494148611 wait complete. 2023-11-08T19:25:30.692 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.680+0000 7f6836889700 1 Processor -- start 2023-11-08T19:25:30.692 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.680+0000 7f6836889700 1 -- start start 2023-11-08T19:25:30.694 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.681+0000 7f6836889700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f68300cb590 0x7f68300b0070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:30.694 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.681+0000 7f6836889700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f68300b1900 0x7f68300cb970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:30.695 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.681+0000 7f6836889700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f68300c5cd0 0x7f683015f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:30.695 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.681+0000 7f6836889700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68300b2f80 con 0x7f68300c5cd0 2023-11-08T19:25:30.695 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.681+0000 7f6836889700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68300c5280 con 0x7f68300b1900 2023-11-08T19:25:30.695 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.681+0000 7f6836889700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68300cbeb0 con 0x7f68300cb590 2023-11-08T19:25:30.695 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.681+0000 7f6834e26700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f68300c5cd0 0x7f683015f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:30.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.681+0000 7f6834e26700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f68300c5cd0 0x7f683015f8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49956/0 (socket says 172.21.15.143:49956) 2023-11-08T19:25:30.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.681+0000 7f6834e26700 1 -- 172.21.15.143:0/3747717286 learned_addr learned my addr 172.21.15.143:0/3747717286 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:30.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.681+0000 7f682f7fe700 1 --2- 172.21.15.143:0/3747717286 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f68300b1900 0x7f68300cb970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:30.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.681+0000 7f682ffff700 1 --2- 172.21.15.143:0/3747717286 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f68300cb590 0x7f68300b0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:30.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.681+0000 7f6834e26700 1 -- 172.21.15.143:0/3747717286 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f68300cb590 msgr2=0x7f68300b0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:30.697 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.681+0000 7f6834e26700 1 --2- 172.21.15.143:0/3747717286 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f68300cb590 0x7f68300b0070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:30.697 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.681+0000 7f6834e26700 1 -- 172.21.15.143:0/3747717286 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f68300b1900 msgr2=0x7f68300cb970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:30.697 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.681+0000 7f6834e26700 1 --2- 172.21.15.143:0/3747717286 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f68300b1900 0x7f68300cb970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:30.697 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.681+0000 7f6834e26700 1 -- 172.21.15.143:0/3747717286 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68300b1e00 con 0x7f68300c5cd0 2023-11-08T19:25:30.697 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.681+0000 7f6834e26700 1 --2- 172.21.15.143:0/3747717286 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f68300c5cd0 0x7f683015f8f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f682800e700 tx=0x7f6828009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:30.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.682+0000 7f682cff9700 1 -- 172.21.15.143:0/3747717286 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6828007000 con 0x7f68300c5cd0 2023-11-08T19:25:30.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.682+0000 7f682cff9700 1 -- 172.21.15.143:0/3747717286 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6828015ab0 con 0x7f68300c5cd0 2023-11-08T19:25:30.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.682+0000 7f6836889700 1 -- 172.21.15.143:0/3747717286 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68300b20e0 con 0x7f68300c5cd0 2023-11-08T19:25:30.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.682+0000 7f682cff9700 1 -- 172.21.15.143:0/3747717286 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6828015e60 con 0x7f68300c5cd0 2023-11-08T19:25:30.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.682+0000 7f6836889700 1 -- 172.21.15.143:0/3747717286 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68300c57b0 con 0x7f68300c5cd0 2023-11-08T19:25:30.699 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.683+0000 7f6836889700 1 -- 172.21.15.143:0/3747717286 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f683004eee0 con 0x7f68300c5cd0 2023-11-08T19:25:30.699 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.683+0000 7f682cff9700 1 -- 172.21.15.143:0/3747717286 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6828024030 con 0x7f68300c5cd0 2023-11-08T19:25:30.699 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.684+0000 7f682cff9700 1 --2- 172.21.15.143:0/3747717286 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f681c06af00 0x7f681c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:30.699 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.684+0000 7f682cff9700 1 -- 172.21.15.143:0/3747717286 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f682808aad0 con 0x7f68300c5cd0 2023-11-08T19:25:30.699 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.684+0000 7f682ffff700 1 --2- 172.21.15.143:0/3747717286 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f681c06af00 0x7f681c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:30.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.684+0000 7f682ffff700 1 --2- 172.21.15.143:0/3747717286 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f681c06af00 0x7f681c06d3c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6824009cc0 tx=0x7f68240091c0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:30.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.687+0000 7f682cff9700 1 -- 172.21.15.143:0/3747717286 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6828055db0 con 0x7f68300c5cd0 2023-11-08T19:25:30.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.890+0000 7f6836889700 1 -- 172.21.15.143:0/3747717286 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f68300cc440 con 0x7f68300c5cd0 2023-11-08T19:25:30.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.891+0000 7f682cff9700 1 -- 172.21.15.143:0/3747717286 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) v1 ==== 80+0+0 (secure 0 0 0) 0x7f6828018080 con 0x7f68300c5cd0 2023-11-08T19:25:30.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.892+0000 7f6836889700 1 -- 172.21.15.143:0/3747717286 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f681c06af00 msgr2=0x7f681c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:30.895 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.892+0000 7f6836889700 1 --2- 172.21.15.143:0/3747717286 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f681c06af00 0x7f681c06d3c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6824009cc0 tx=0x7f68240091c0 comp rx=0 tx=0).stop 2023-11-08T19:25:30.895 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.892+0000 7f6836889700 1 -- 172.21.15.143:0/3747717286 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f68300c5cd0 msgr2=0x7f683015f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:30.895 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.892+0000 7f6836889700 1 --2- 172.21.15.143:0/3747717286 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f68300c5cd0 0x7f683015f8f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f682800e700 tx=0x7f6828009750 comp rx=0 tx=0).stop 2023-11-08T19:25:30.895 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.892+0000 7f6836889700 1 -- 172.21.15.143:0/3747717286 shutdown_connections 2023-11-08T19:25:30.895 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.892+0000 7f6836889700 1 --2- 172.21.15.143:0/3747717286 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f681c06af00 0x7f681c06d3c0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:30.896 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.892+0000 7f6836889700 1 --2- 172.21.15.143:0/3747717286 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f68300cb590 0x7f68300b0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:30.896 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.892+0000 7f6836889700 1 --2- 172.21.15.143:0/3747717286 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f68300c5cd0 0x7f683015f8f0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:30.896 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.892+0000 7f6836889700 1 --2- 172.21.15.143:0/3747717286 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f68300b1900 0x7f68300cb970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:30.896 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.893+0000 7f6836889700 1 -- 172.21.15.143:0/3747717286 >> 172.21.15.143:0/3747717286 conn(0x7f6830099ab0 msgr2=0x7f68300ad380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:30.896 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.893+0000 7f6836889700 1 -- 172.21.15.143:0/3747717286 shutdown_connections 2023-11-08T19:25:30.897 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:30.893+0000 7f6836889700 1 -- 172.21.15.143:0/3747717286 wait complete. 2023-11-08T19:25:30.897 INFO:teuthology.orchestra.run.smithi143.stderr:listed 0 entries 2023-11-08T19:25:30.906 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-11-08T19:25:30.906 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:30.907 DEBUG:teuthology.orchestra.run.smithi174:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2023-11-08T19:25:30.962 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi174.front.sepia.ceph.com 2023-11-08T19:25:30.963 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:30.963 DEBUG:teuthology.orchestra.run.smithi174:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-08T19:25:30.992 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-08T19:25:30.992 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:25:31.010 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-08T19:25:31.011 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:25:31.067 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-08T19:25:31.068 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:25:31.124 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:25:31.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.283+0000 7ff61d833700 1 Processor -- start 2023-11-08T19:25:31.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.283+0000 7ff61d833700 1 -- start start 2023-11-08T19:25:31.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.283+0000 7ff61d833700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff61806c730 0x7ff6180cd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:31.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.283+0000 7ff61d833700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6180cd480 con 0x7ff61815de80 2023-11-08T19:25:31.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.283+0000 7ff61d833700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6180cc610 con 0x7ff61806c030 2023-11-08T19:25:31.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.283+0000 7ff61d833700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6180cc870 con 0x7ff61806c730 2023-11-08T19:25:31.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.284+0000 7ff6177fe700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7ff61815de80 0x7ff6180cbf00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:46116/0 (socket says 172.21.15.143:46116) 2023-11-08T19:25:31.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.284+0000 7ff6177fe700 1 -- 172.21.15.143:0/2522653811 learned_addr learned my addr 172.21.15.143:0/2522653811 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:25:31.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.284+0000 7ff616ffd700 1 --2- 172.21.15.143:0/2522653811 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff61806c730 0x7ff6180cd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:31.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.284+0000 7ff615ffb700 1 -- 172.21.15.143:0/2522653811 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3175518183 0 0) 0x7ff6180cd480 con 0x7ff61815de80 2023-11-08T19:25:31.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.284+0000 7ff615ffb700 1 -- 172.21.15.143:0/2522653811 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff6000035b0 con 0x7ff61815de80 2023-11-08T19:25:31.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.284+0000 7ff615ffb700 1 -- 172.21.15.143:0/2522653811 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1198638329 0 0) 0x7ff6000035b0 con 0x7ff61815de80 2023-11-08T19:25:31.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.284+0000 7ff615ffb700 1 -- 172.21.15.143:0/2522653811 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff61806c730 msgr2=0x7ff6180cd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:31.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.284+0000 7ff615ffb700 1 --2- 172.21.15.143:0/2522653811 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff61806c730 0x7ff6180cd910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:31.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.284+0000 7ff615ffb700 1 -- 172.21.15.143:0/2522653811 >> v1:172.21.15.174:6789/0 conn(0x7ff61806c030 legacy=0x7ff61806cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:31.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.284+0000 7ff615ffb700 1 -- 172.21.15.143:0/2522653811 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6180ccb50 con 0x7ff61815de80 2023-11-08T19:25:31.293 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.284+0000 7ff615ffb700 1 -- 172.21.15.143:0/2522653811 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4147756303 0 0) 0x7ff6100019f0 con 0x7ff61815de80 2023-11-08T19:25:31.293 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.284+0000 7ff615ffb700 1 -- 172.21.15.143:0/2522653811 >> v1:172.21.15.143:6789/0 conn(0x7ff61815de80 legacy=0x7ff6180cbf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:31.294 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.284+0000 7ff615ffb700 1 --2- 172.21.15.143:0/2522653811 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff600003a30 0x7ff600003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:31.294 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.284+0000 7ff615ffb700 1 --2- 172.21.15.143:0/2522653811 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff600004850 0x7ff600004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:31.294 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.284+0000 7ff615ffb700 1 --2- 172.21.15.143:0/2522653811 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff61806c730 0x7ff600005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:31.294 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.284+0000 7ff615ffb700 1 -- 172.21.15.143:0/2522653811 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6180cc870 con 0x7ff600003a30 2023-11-08T19:25:31.294 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.284+0000 7ff615ffb700 1 -- 172.21.15.143:0/2522653811 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff60000bac0 con 0x7ff600004850 2023-11-08T19:25:31.295 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.284+0000 7ff615ffb700 1 -- 172.21.15.143:0/2522653811 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff60000bc80 con 0x7ff61806c730 2023-11-08T19:25:31.295 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.284+0000 7ff616ffd700 1 --2- 172.21.15.143:0/2522653811 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff600003a30 0x7ff600003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:31.295 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.284+0000 7ff6177fe700 1 --2- 172.21.15.143:0/2522653811 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff61806c730 0x7ff600005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:31.295 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.284+0000 7ff6167fc700 1 --2- 172.21.15.143:0/2522653811 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff600004850 0x7ff600004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:31.295 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.284+0000 7ff616ffd700 1 -- 172.21.15.143:0/2522653811 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff61806c730 msgr2=0x7ff600005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:31.295 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.284+0000 7ff616ffd700 1 --2- 172.21.15.143:0/2522653811 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff61806c730 0x7ff600005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:31.296 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.284+0000 7ff616ffd700 1 -- 172.21.15.143:0/2522653811 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff600004850 msgr2=0x7ff600004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:31.296 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.284+0000 7ff616ffd700 1 --2- 172.21.15.143:0/2522653811 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff600004850 0x7ff600004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:31.296 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.284+0000 7ff616ffd700 1 -- 172.21.15.143:0/2522653811 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6000035b0 con 0x7ff600003a30 2023-11-08T19:25:31.296 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.285+0000 7ff616ffd700 1 --2- 172.21.15.143:0/2522653811 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff600003a30 0x7ff600003e50 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ff608009b30 tx=0x7ff600005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64218ede1dd09520 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:31.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.285+0000 7ff615ffb700 1 -- 172.21.15.143:0/2522653811 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff60800e6c0 con 0x7ff600003a30 2023-11-08T19:25:31.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.285+0000 7ff615ffb700 1 -- 172.21.15.143:0/2522653811 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff60800e820 con 0x7ff600003a30 2023-11-08T19:25:31.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.285+0000 7ff615ffb700 1 -- 172.21.15.143:0/2522653811 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff60800eb90 con 0x7ff600003a30 2023-11-08T19:25:31.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.285+0000 7ff61d833700 1 -- 172.21.15.143:0/2522653811 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff600003a30 msgr2=0x7ff600003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:31.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.285+0000 7ff61d833700 1 --2- 172.21.15.143:0/2522653811 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff600003a30 0x7ff600003e50 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ff608009b30 tx=0x7ff600005880 comp rx=0 tx=0).stop 2023-11-08T19:25:31.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.285+0000 7ff6177fe700 1 -- 172.21.15.143:0/2522653811 reap_dead start 2023-11-08T19:25:31.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.285+0000 7ff61d833700 1 -- 172.21.15.143:0/2522653811 shutdown_connections 2023-11-08T19:25:31.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.285+0000 7ff61d833700 1 -- 172.21.15.143:0/2522653811 >> 172.21.15.143:0/2522653811 conn(0x7ff61808b000 msgr2=0x7ff61809ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:31.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.285+0000 7ff61d833700 1 -- 172.21.15.143:0/2522653811 shutdown_connections 2023-11-08T19:25:31.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.285+0000 7ff61d833700 1 -- 172.21.15.143:0/2522653811 wait complete. 2023-11-08T19:25:31.299 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.287+0000 7ff61d833700 1 Processor -- start 2023-11-08T19:25:31.299 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.287+0000 7ff61d833700 1 -- start start 2023-11-08T19:25:31.302 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.288+0000 7ff61d833700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff600003a30 0x7ff61815eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:31.302 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.288+0000 7ff61d833700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff61815feb0 0x7ff618160500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:31.302 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.288+0000 7ff61d833700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff6180ca480 0x7ff61814f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:31.302 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.288+0000 7ff61d833700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6180b60b0 con 0x7ff61815feb0 2023-11-08T19:25:31.303 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.288+0000 7ff61d833700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff618161110 con 0x7ff6180ca480 2023-11-08T19:25:31.303 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.288+0000 7ff61d833700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff618160e30 con 0x7ff600003a30 2023-11-08T19:25:31.303 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.288+0000 7ff6167fc700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff61815feb0 0x7ff618160500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:31.303 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.288+0000 7ff6167fc700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff61815feb0 0x7ff618160500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49972/0 (socket says 172.21.15.143:49972) 2023-11-08T19:25:31.303 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.288+0000 7ff6167fc700 1 -- 172.21.15.143:0/210964769 learned_addr learned my addr 172.21.15.143:0/210964769 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:31.304 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.288+0000 7ff616ffd700 1 --2- 172.21.15.143:0/210964769 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff600003a30 0x7ff61815eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:31.304 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.288+0000 7ff6177fe700 1 --2- 172.21.15.143:0/210964769 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff6180ca480 0x7ff61814f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:31.304 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.288+0000 7ff6167fc700 1 -- 172.21.15.143:0/210964769 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff600003a30 msgr2=0x7ff61815eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:31.304 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.288+0000 7ff6167fc700 1 --2- 172.21.15.143:0/210964769 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff600003a30 0x7ff61815eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:31.304 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.288+0000 7ff6167fc700 1 -- 172.21.15.143:0/210964769 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff6180ca480 msgr2=0x7ff61814f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:31.304 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.288+0000 7ff6167fc700 1 --2- 172.21.15.143:0/210964769 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff6180ca480 0x7ff61814f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:31.305 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.288+0000 7ff6167fc700 1 -- 172.21.15.143:0/210964769 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff61815e3c0 con 0x7ff61815feb0 2023-11-08T19:25:31.305 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.288+0000 7ff6177fe700 1 --2- 172.21.15.143:0/210964769 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff6180ca480 0x7ff61814f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:25:31.305 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.289+0000 7ff6167fc700 1 --2- 172.21.15.143:0/210964769 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff61815feb0 0x7ff618160500 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7ff60c00f640 tx=0x7ff60c00fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:31.306 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.289+0000 7ff5fffff700 1 -- 172.21.15.143:0/210964769 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff60c0052e0 con 0x7ff61815feb0 2023-11-08T19:25:31.306 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.289+0000 7ff5fffff700 1 -- 172.21.15.143:0/210964769 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff60c005440 con 0x7ff61815feb0 2023-11-08T19:25:31.306 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.289+0000 7ff5fffff700 1 -- 172.21.15.143:0/210964769 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff60c0038a0 con 0x7ff61815feb0 2023-11-08T19:25:31.306 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.289+0000 7ff61d833700 1 -- 172.21.15.143:0/210964769 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff61815e6a0 con 0x7ff61815feb0 2023-11-08T19:25:31.306 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.289+0000 7ff61d833700 1 -- 172.21.15.143:0/210964769 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff618160a40 con 0x7ff61815feb0 2023-11-08T19:25:31.307 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.290+0000 7ff61d833700 1 -- 172.21.15.143:0/210964769 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff618160c00 con 0x7ff61815feb0 2023-11-08T19:25:31.307 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.290+0000 7ff5fffff700 1 -- 172.21.15.143:0/210964769 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff60c003a00 con 0x7ff61815feb0 2023-11-08T19:25:31.307 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.290+0000 7ff5fffff700 1 --2- 172.21.15.143:0/210964769 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7ff60406af00 0x7ff60406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:31.307 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.290+0000 7ff5fffff700 1 -- 172.21.15.143:0/210964769 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff60c089340 con 0x7ff61815feb0 2023-11-08T19:25:31.308 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.291+0000 7ff616ffd700 1 --2- 172.21.15.143:0/210964769 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7ff60406af00 0x7ff60406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:31.308 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.291+0000 7ff616ffd700 1 --2- 172.21.15.143:0/210964769 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7ff60406af00 0x7ff60406d3c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff6080090b0 tx=0x7ff608008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:31.308 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.295+0000 7ff5fffff700 1 -- 172.21.15.143:0/210964769 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff60c0543f0 con 0x7ff61815feb0 2023-11-08T19:25:31.486 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.485+0000 7ff61d833700 1 -- 172.21.15.143:0/210964769 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff61804eee0 con 0x7ff61815feb0 2023-11-08T19:25:31.487 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.486+0000 7ff5fffff700 1 -- 172.21.15.143:0/210964769 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16553 (secure 0 0 0) 0x7ff60c003cb0 con 0x7ff61815feb0 2023-11-08T19:25:31.488 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:25:31.488 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":20,"fsid":"968483af-77fc-4068-ad7c-5c2a1a4613d7","created":"2023-11-08T19:24:32.826484+0000","modified":"2023-11-08T19:24:51.618072+0000","last_up_change":"2023-11-08T19:24:43.573880+0000","last_in_change":"2023-11-08T19:24:35.314525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:44.551530+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-08T19:24:48.256781+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-08T19:24:48.954990+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bae98ec1-d94e-403d-a220-a522a53ea6f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6809","nonce":1487830566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6811","nonce":1487830566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6815","nonce":1487830566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6813","nonce":1487830566}]},"public_addr":"172.21.15.143:6809/1487830566","cluster_addr":"172.21.15.143:6811/1487830566","heartbeat_back_addr":"172.21.15.143:6815/1487830566","heartbeat_front_addr":"172.21.15.143:6813/1487830566","state":["exists","up"]},{"osd":1,"uuid":"5cb6538a-1ccb-4322-8633-09f66af0b7e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6801","nonce":787341397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6803","nonce":787341397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6807","nonce":787341397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6805","nonce":787341397}]},"public_addr":"172.21.15.143:6801/787341397","cluster_addr":"172.21.15.143:6803/787341397","heartbeat_back_addr":"172.21.15.143:6807/787341397","heartbeat_front_addr":"172.21.15.143:6805/787341397","state":["exists","up"]},{"osd":2,"uuid":"cdbf522c-60ef-4d60-9653-8dcb529c101a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6822","nonce":2301325597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6826","nonce":2301325597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6831","nonce":2301325597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6829","nonce":2301325597}]},"public_addr":"172.21.15.143:6822/2301325597","cluster_addr":"172.21.15.143:6826/2301325597","heartbeat_back_addr":"172.21.15.143:6831/2301325597","heartbeat_front_addr":"172.21.15.143:6829/2301325597","state":["exists","up"]},{"osd":3,"uuid":"4e2783cf-ebb9-4dd8-8245-bc1d17ce4325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6817","nonce":2021504428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6819","nonce":2021504428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6827","nonce":2021504428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6823","nonce":2021504428}]},"public_addr":"172.21.15.143:6817/2021504428","cluster_addr":"172.21.15.143:6819/2021504428","heartbeat_back_addr":"172.21.15.143:6827/2021504428","heartbeat_front_addr":"172.21.15.143:6823/2021504428","state":["exists","up"]},{"osd":4,"uuid":"691256fa-9b77-49b9-a346-7ae1adf0ca80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6801","nonce":1993380595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6803","nonce":1993380595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6807","nonce":1993380595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6805","nonce":1993380595}]},"public_addr":"172.21.15.174:6801/1993380595","cluster_addr":"172.21.15.174:6803/1993380595","heartbeat_back_addr":"172.21.15.174:6807/1993380595","heartbeat_front_addr":"172.21.15.174:6805/1993380595","state":["exists","up"]},{"osd":5,"uuid":"756fc352-48c9-46ce-b52e-0541fe6f8525","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6809","nonce":1777923303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6811","nonce":1777923303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6815","nonce":1777923303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6813","nonce":1777923303}]},"public_addr":"172.21.15.174:6809/1777923303","cluster_addr":"172.21.15.174:6811/1777923303","heartbeat_back_addr":"172.21.15.174:6815/1777923303","heartbeat_front_addr":"172.21.15.174:6813/1777923303","state":["exists","up"]},{"osd":6,"uuid":"fab2ac2b-ed97-4cde-9825-b480eabf8b8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6825","nonce":2755130316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6827","nonce":2755130316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6831","nonce":2755130316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6829","nonce":2755130316}]},"public_addr":"172.21.15.174:6825/2755130316","cluster_addr":"172.21.15.174:6827/2755130316","heartbeat_back_addr":"172.21.15.174:6831/2755130316","heartbeat_front_addr":"172.21.15.174:6829/2755130316","state":["exists","up"]},{"osd":7,"uuid":"3de9b797-2ab6-4d41-a863-7e7d638fd703","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6817","nonce":2847141411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6819","nonce":2847141411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6823","nonce":2847141411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6821","nonce":2847141411}]},"public_addr":"172.21.15.174:6817/2847141411","cluster_addr":"172.21.15.174:6819/2847141411","heartbeat_back_addr":"172.21.15.174:6823/2847141411","heartbeat_front_addr":"172.21.15.174:6821/2847141411","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.477076+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.496890+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.539305+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.454673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.538881+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.521294+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.520712+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.540404+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:25:31.490 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.489+0000 7ff61d833700 1 -- 172.21.15.143:0/210964769 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7ff60406af00 msgr2=0x7ff60406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:31.490 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.489+0000 7ff61d833700 1 --2- 172.21.15.143:0/210964769 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7ff60406af00 0x7ff60406d3c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff6080090b0 tx=0x7ff608008030 comp rx=0 tx=0).stop 2023-11-08T19:25:31.491 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.489+0000 7ff61d833700 1 -- 172.21.15.143:0/210964769 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff61815feb0 msgr2=0x7ff618160500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:31.491 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.489+0000 7ff61d833700 1 --2- 172.21.15.143:0/210964769 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff61815feb0 0x7ff618160500 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7ff60c00f640 tx=0x7ff60c00fa00 comp rx=0 tx=0).stop 2023-11-08T19:25:31.491 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.489+0000 7ff61d833700 1 -- 172.21.15.143:0/210964769 shutdown_connections 2023-11-08T19:25:31.491 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.489+0000 7ff61d833700 1 --2- 172.21.15.143:0/210964769 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7ff60406af00 0x7ff60406d3c0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:31.492 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.489+0000 7ff61d833700 1 --2- 172.21.15.143:0/210964769 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff600003a30 0x7ff61815eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:31.492 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.489+0000 7ff61d833700 1 --2- 172.21.15.143:0/210964769 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff6180ca480 0x7ff61814f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:31.492 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.489+0000 7ff61d833700 1 --2- 172.21.15.143:0/210964769 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff61815feb0 0x7ff618160500 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:31.492 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.489+0000 7ff61d833700 1 -- 172.21.15.143:0/210964769 >> 172.21.15.143:0/210964769 conn(0x7ff61808b000 msgr2=0x7ff618070b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:31.493 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.489+0000 7ff61d833700 1 -- 172.21.15.143:0/210964769 shutdown_connections 2023-11-08T19:25:31.493 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.489+0000 7ff61d833700 1 -- 172.21.15.143:0/210964769 wait complete. 2023-11-08T19:25:31.503 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T19:24:44.551530+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-08T19:24:48.256781+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-08T19:24:48.954990+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-08T19:25:31.503 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T19:25:31.627 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.626+0000 7f8ca3d05700 1 Processor -- start 2023-11-08T19:25:31.628 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.626+0000 7f8ca3d05700 1 -- start start 2023-11-08T19:25:31.630 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.627+0000 7f8ca3d05700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f8c9c15dba0 0x7f8c9c06c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:31.631 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.627+0000 7f8ca3d05700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f8c9c06b240 0x7f8c9c0caff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:31.631 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.627+0000 7f8ca3d05700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8c9c0cb530 0x7f8c9c0b22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:31.631 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.627+0000 7f8ca3d05700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c9c161890 con 0x7f8c9c15dba0 2023-11-08T19:25:31.631 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.627+0000 7f8ca3d05700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c9c06b740 con 0x7f8c9c06b240 2023-11-08T19:25:31.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.627+0000 7f8ca3d05700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c9c06ba20 con 0x7f8c9c0cb530 2023-11-08T19:25:31.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.627+0000 7f8ca1aa1700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f8c9c15dba0 0x7f8c9c06c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:31.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.627+0000 7f8ca1aa1700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f8c9c15dba0 0x7f8c9c06c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49978/0 (socket says 172.21.15.143:49978) 2023-11-08T19:25:31.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.627+0000 7f8ca1aa1700 1 -- 172.21.15.143:0/1663099664 learned_addr learned my addr 172.21.15.143:0/1663099664 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:31.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.627+0000 7f8ca22a2700 1 --2- 172.21.15.143:0/1663099664 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8c9c0cb530 0x7f8c9c0b22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:31.633 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.627+0000 7f8ca12a0700 1 --2- 172.21.15.143:0/1663099664 >> v2:172.21.15.174:3300/0 conn(0x7f8c9c06b240 0x7f8c9c0caff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:31.633 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.627+0000 7f8ca1aa1700 1 -- 172.21.15.143:0/1663099664 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8c9c0cb530 msgr2=0x7f8c9c0b22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:31.633 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.627+0000 7f8ca1aa1700 1 --2- 172.21.15.143:0/1663099664 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8c9c0cb530 0x7f8c9c0b22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:31.633 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.627+0000 7f8ca1aa1700 1 -- 172.21.15.143:0/1663099664 >> v2:172.21.15.174:3300/0 conn(0x7f8c9c06b240 msgr2=0x7f8c9c0caff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:31.633 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.627+0000 7f8ca1aa1700 1 --2- 172.21.15.143:0/1663099664 >> v2:172.21.15.174:3300/0 conn(0x7f8c9c06b240 0x7f8c9c0caff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:31.634 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.627+0000 7f8ca1aa1700 1 -- 172.21.15.143:0/1663099664 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c9c06bd00 con 0x7f8c9c15dba0 2023-11-08T19:25:31.634 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.627+0000 7f8ca22a2700 1 --2- 172.21.15.143:0/1663099664 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8c9c0cb530 0x7f8c9c0b22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:25:31.634 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.627+0000 7f8ca12a0700 1 --2- 172.21.15.143:0/1663099664 >> v2:172.21.15.174:3300/0 conn(0x7f8c9c06b240 0x7f8c9c0caff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:25:31.634 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.628+0000 7f8ca1aa1700 1 --2- 172.21.15.143:0/1663099664 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8c9c15dba0 0x7f8c9c06c610 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c0d0810 tx=0x7f8c8c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b87310dc32e67fa1 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:31.634 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.628+0000 7f8ca0a9f700 1 -- 172.21.15.143:0/1663099664 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c8c00b870 con 0x7f8c9c15dba0 2023-11-08T19:25:31.635 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.628+0000 7f8ca0a9f700 1 -- 172.21.15.143:0/1663099664 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c8c00b9d0 con 0x7f8c9c15dba0 2023-11-08T19:25:31.635 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.628+0000 7f8ca0a9f700 1 -- 172.21.15.143:0/1663099664 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c8c00bd60 con 0x7f8c9c15dba0 2023-11-08T19:25:31.635 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.628+0000 7f8ca3d05700 1 -- 172.21.15.143:0/1663099664 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8c9c15dba0 msgr2=0x7f8c9c06c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:31.635 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.628+0000 7f8ca3d05700 1 --2- 172.21.15.143:0/1663099664 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8c9c15dba0 0x7f8c9c06c610 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c0d0810 tx=0x7f8c8c009bd0 comp rx=0 tx=0).stop 2023-11-08T19:25:31.635 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.628+0000 7f8ca3d05700 1 -- 172.21.15.143:0/1663099664 shutdown_connections 2023-11-08T19:25:31.636 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.628+0000 7f8ca3d05700 1 --2- 172.21.15.143:0/1663099664 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8c9c0cb530 0x7f8c9c0b22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:31.636 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.628+0000 7f8ca3d05700 1 --2- 172.21.15.143:0/1663099664 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8c9c15dba0 0x7f8c9c06c610 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:31.636 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.628+0000 7f8ca3d05700 1 --2- 172.21.15.143:0/1663099664 >> v2:172.21.15.174:3300/0 conn(0x7f8c9c06b240 0x7f8c9c0caff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:31.636 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.628+0000 7f8ca3d05700 1 -- 172.21.15.143:0/1663099664 >> 172.21.15.143:0/1663099664 conn(0x7f8c9c0cd2e0 msgr2=0x7f8c9c0cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:31.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.628+0000 7f8ca3d05700 1 -- 172.21.15.143:0/1663099664 shutdown_connections 2023-11-08T19:25:31.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.628+0000 7f8ca3d05700 1 -- 172.21.15.143:0/1663099664 wait complete. 2023-11-08T19:25:31.640 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.630+0000 7f8ca3d05700 1 Processor -- start 2023-11-08T19:25:31.640 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.630+0000 7f8ca3d05700 1 -- start start 2023-11-08T19:25:31.640 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.630+0000 7f8ca3d05700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8c9c06b240 0x7f8c9c15a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:31.640 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.630+0000 7f8ca3d05700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c9c0cb530 0x7f8c9c15a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:31.641 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.630+0000 7f8ca3d05700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8c9c15dba0 0x7f8c9c0b3e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:31.641 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.630+0000 7f8ca3d05700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c9c1524a0 con 0x7f8c9c15dba0 2023-11-08T19:25:31.641 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.630+0000 7f8ca3d05700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c9c151c00 con 0x7f8c9c0cb530 2023-11-08T19:25:31.641 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.630+0000 7f8ca3d05700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c9c151ee0 con 0x7f8c9c06b240 2023-11-08T19:25:31.641 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.630+0000 7f8ca22a2700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8c9c15dba0 0x7f8c9c0b3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:31.642 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.630+0000 7f8ca22a2700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8c9c15dba0 0x7f8c9c0b3e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49988/0 (socket says 172.21.15.143:49988) 2023-11-08T19:25:31.642 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.630+0000 7f8ca22a2700 1 -- 172.21.15.143:0/2864278123 learned_addr learned my addr 172.21.15.143:0/2864278123 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:31.642 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.631+0000 7f8ca12a0700 1 --2- 172.21.15.143:0/2864278123 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c9c0cb530 0x7f8c9c15a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:31.642 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.631+0000 7f8ca1aa1700 1 --2- 172.21.15.143:0/2864278123 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8c9c06b240 0x7f8c9c15a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:31.642 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.631+0000 7f8ca22a2700 1 -- 172.21.15.143:0/2864278123 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8c9c06b240 msgr2=0x7f8c9c15a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:31.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.631+0000 7f8ca22a2700 1 --2- 172.21.15.143:0/2864278123 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8c9c06b240 0x7f8c9c15a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:31.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.631+0000 7f8ca22a2700 1 -- 172.21.15.143:0/2864278123 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c9c0cb530 msgr2=0x7f8c9c15a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:31.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.631+0000 7f8ca22a2700 1 --2- 172.21.15.143:0/2864278123 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c9c0cb530 0x7f8c9c15a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:31.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.631+0000 7f8ca22a2700 1 -- 172.21.15.143:0/2864278123 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c8c009550 con 0x7f8c9c15dba0 2023-11-08T19:25:31.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.631+0000 7f8ca12a0700 1 --2- 172.21.15.143:0/2864278123 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c9c0cb530 0x7f8c9c15a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:25:31.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.631+0000 7f8ca1aa1700 1 --2- 172.21.15.143:0/2864278123 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8c9c06b240 0x7f8c9c15a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:25:31.644 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.631+0000 7f8ca22a2700 1 --2- 172.21.15.143:0/2864278123 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8c9c15dba0 0x7f8c9c0b3e70 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f8c94007ef0 tx=0x7f8c9400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:31.644 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.631+0000 7f8c927fc700 1 -- 172.21.15.143:0/2864278123 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c9400de60 con 0x7f8c9c15dba0 2023-11-08T19:25:31.644 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.631+0000 7f8c927fc700 1 -- 172.21.15.143:0/2864278123 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c9400f040 con 0x7f8c9c15dba0 2023-11-08T19:25:31.644 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.631+0000 7f8ca3d05700 1 -- 172.21.15.143:0/2864278123 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c9c1521c0 con 0x7f8c9c15dba0 2023-11-08T19:25:31.644 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.631+0000 7f8c927fc700 1 -- 172.21.15.143:0/2864278123 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c94003c60 con 0x7f8c9c15dba0 2023-11-08T19:25:31.645 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.631+0000 7f8ca3d05700 1 -- 172.21.15.143:0/2864278123 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c9c0c7b50 con 0x7f8c9c15dba0 2023-11-08T19:25:31.645 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.632+0000 7f8ca3d05700 1 -- 172.21.15.143:0/2864278123 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c9c0c9320 con 0x7f8c9c15dba0 2023-11-08T19:25:31.645 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.632+0000 7f8c927fc700 1 -- 172.21.15.143:0/2864278123 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8c94004030 con 0x7f8c9c15dba0 2023-11-08T19:25:31.645 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.633+0000 7f8c927fc700 1 --2- 172.21.15.143:0/2864278123 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f8c8006af00 0x7f8c8006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:31.645 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.633+0000 7f8c927fc700 1 -- 172.21.15.143:0/2864278123 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8c94089740 con 0x7f8c9c15dba0 2023-11-08T19:25:31.646 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.633+0000 7f8ca1aa1700 1 --2- 172.21.15.143:0/2864278123 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f8c8006af00 0x7f8c8006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:31.646 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.634+0000 7f8ca1aa1700 1 --2- 172.21.15.143:0/2864278123 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f8c8006af00 0x7f8c8006d3c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c009130 tx=0x7f8c8c008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:31.646 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.637+0000 7f8c927fc700 1 -- 172.21.15.143:0/2864278123 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8c9c0c9320 con 0x7f8c9c15dba0 2023-11-08T19:25:31.829 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.827+0000 7f8ca3d05700 1 -- 172.21.15.143:0/2864278123 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8c9c0c9320 con 0x7f8c9c15dba0 2023-11-08T19:25:31.829 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.827+0000 7f8c927fc700 1 -- 172.21.15.143:0/2864278123 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8c9401a020 con 0x7f8c9c15dba0 2023-11-08T19:25:31.829 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2023-11-08T19:25:31.831 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.830+0000 7f8ca3d05700 1 -- 172.21.15.143:0/2864278123 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f8c8006af00 msgr2=0x7f8c8006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:31.832 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.830+0000 7f8ca3d05700 1 --2- 172.21.15.143:0/2864278123 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f8c8006af00 0x7f8c8006d3c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c009130 tx=0x7f8c8c008030 comp rx=0 tx=0).stop 2023-11-08T19:25:31.832 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.830+0000 7f8ca3d05700 1 -- 172.21.15.143:0/2864278123 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8c9c15dba0 msgr2=0x7f8c9c0b3e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:31.832 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.830+0000 7f8ca3d05700 1 --2- 172.21.15.143:0/2864278123 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8c9c15dba0 0x7f8c9c0b3e70 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f8c94007ef0 tx=0x7f8c9400d3b0 comp rx=0 tx=0).stop 2023-11-08T19:25:31.832 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.830+0000 7f8ca3d05700 1 -- 172.21.15.143:0/2864278123 shutdown_connections 2023-11-08T19:25:31.832 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.830+0000 7f8ca3d05700 1 --2- 172.21.15.143:0/2864278123 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f8c8006af00 0x7f8c8006d3c0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:31.833 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.830+0000 7f8ca3d05700 1 --2- 172.21.15.143:0/2864278123 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8c9c06b240 0x7f8c9c15a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:31.833 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.830+0000 7f8ca3d05700 1 --2- 172.21.15.143:0/2864278123 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8c9c15dba0 0x7f8c9c0b3e70 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:31.833 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.830+0000 7f8ca3d05700 1 --2- 172.21.15.143:0/2864278123 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c9c0cb530 0x7f8c9c15a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:31.833 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.830+0000 7f8ca3d05700 1 -- 172.21.15.143:0/2864278123 >> 172.21.15.143:0/2864278123 conn(0x7f8c9c0cd2e0 msgr2=0x7f8c9c15ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:31.834 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.830+0000 7f8ca3d05700 1 -- 172.21.15.143:0/2864278123 shutdown_connections 2023-11-08T19:25:31.834 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.830+0000 7f8ca3d05700 1 -- 172.21.15.143:0/2864278123 wait complete. 2023-11-08T19:25:31.843 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-08T19:25:31.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.965+0000 7f3e8b86d700 1 Processor -- start 2023-11-08T19:25:31.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.966+0000 7f3e8b86d700 1 -- start start 2023-11-08T19:25:31.970 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.966+0000 7f3e8b86d700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f3e8415a950 0x7f3e840d01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:31.970 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.966+0000 7f3e8b86d700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f3e8406bd20 0x7f3e84072370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:31.970 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.966+0000 7f3e8b86d700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e840cc2a0 con 0x7f3e840728b0 2023-11-08T19:25:31.970 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.966+0000 7f3e8b86d700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e840cc130 con 0x7f3e8406bd20 2023-11-08T19:25:31.971 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.966+0000 7f3e8b86d700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e8406c1b0 con 0x7f3e8415a950 2023-11-08T19:25:31.971 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.967+0000 7f3e88e08700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f3e8406bd20 0x7f3e84072370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:31.971 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.967+0000 7f3e89e0a700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f3e840728b0 0x7f3e840cc730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:46150/0 (socket says 172.21.15.143:46150) 2023-11-08T19:25:31.971 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.967+0000 7f3e89e0a700 1 -- 172.21.15.143:0/1988910201 learned_addr learned my addr 172.21.15.143:0/1988910201 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:25:31.971 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.967+0000 7f3e89609700 1 --2- 172.21.15.143:0/1988910201 >> v2:172.21.15.174:3300/0 conn(0x7f3e8415a950 0x7f3e840d01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:31.972 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.967+0000 7f3e88e08700 1 -- 172.21.15.143:0/1988910201 >> v2:172.21.15.174:3300/0 conn(0x7f3e8415a950 msgr2=0x7f3e840d01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:31.972 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.967+0000 7f3e88e08700 1 --2- 172.21.15.143:0/1988910201 >> v2:172.21.15.174:3300/0 conn(0x7f3e8415a950 0x7f3e840d01b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:31.972 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.967+0000 7f3e88e08700 1 -- 172.21.15.143:0/1988910201 >> v1:172.21.15.143:6789/0 conn(0x7f3e840728b0 legacy=0x7f3e840cc730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:31.972 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.967+0000 7f3e88e08700 1 -- 172.21.15.143:0/1988910201 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e8406c490 con 0x7f3e8406bd20 2023-11-08T19:25:31.972 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.967+0000 7f3e7bfff700 1 -- 172.21.15.143:0/1988910201 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2660150449 0 0) 0x7f3e840cc2a0 con 0x7f3e840728b0 2023-11-08T19:25:31.972 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.967+0000 7f3e88e08700 1 --2- 172.21.15.143:0/1988910201 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e8406bd20 0x7f3e84072370 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f3e840cdf90 tx=0x7f3e80009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d9888c8fc18c1e3 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:31.973 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.967+0000 7f3e7bfff700 1 -- 172.21.15.143:0/1988910201 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e80011830 con 0x7f3e8406bd20 2023-11-08T19:25:31.973 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.967+0000 7f3e7bfff700 1 -- 172.21.15.143:0/1988910201 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e80011990 con 0x7f3e8406bd20 2023-11-08T19:25:31.973 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.967+0000 7f3e7bfff700 1 -- 172.21.15.143:0/1988910201 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e80011d20 con 0x7f3e8406bd20 2023-11-08T19:25:31.973 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.968+0000 7f3e8b86d700 1 -- 172.21.15.143:0/1988910201 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e8406bd20 msgr2=0x7f3e84072370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:31.973 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.968+0000 7f3e8b86d700 1 --2- 172.21.15.143:0/1988910201 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e8406bd20 0x7f3e84072370 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f3e840cdf90 tx=0x7f3e80009b20 comp rx=0 tx=0).stop 2023-11-08T19:25:31.974 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.968+0000 7f3e8b86d700 1 -- 172.21.15.143:0/1988910201 shutdown_connections 2023-11-08T19:25:31.974 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.968+0000 7f3e8b86d700 1 --2- 172.21.15.143:0/1988910201 >> v2:172.21.15.174:3300/0 conn(0x7f3e8415a950 0x7f3e840d01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:31.974 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.968+0000 7f3e8b86d700 1 --2- 172.21.15.143:0/1988910201 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e8406bd20 0x7f3e84072370 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:31.974 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.968+0000 7f3e8b86d700 1 -- 172.21.15.143:0/1988910201 >> 172.21.15.143:0/1988910201 conn(0x7f3e840cd2e0 msgr2=0x7f3e840cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:31.974 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.968+0000 7f3e8b86d700 1 -- 172.21.15.143:0/1988910201 shutdown_connections 2023-11-08T19:25:31.975 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.968+0000 7f3e8b86d700 1 -- 172.21.15.143:0/1988910201 wait complete. 2023-11-08T19:25:31.977 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.969+0000 7f3e8b86d700 1 Processor -- start 2023-11-08T19:25:31.977 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.969+0000 7f3e8b86d700 1 -- start start 2023-11-08T19:25:31.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.970+0000 7f3e8b86d700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3e840728b0 0x7f3e8414ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:31.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.970+0000 7f3e8b86d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e8415a950 0x7f3e841f25a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:31.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.970+0000 7f3e8b86d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3e840c6770 0x7f3e8415d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:31.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.970+0000 7f3e8b86d700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e840b2190 con 0x7f3e8415a950 2023-11-08T19:25:31.979 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.970+0000 7f3e8b86d700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e840b2470 con 0x7f3e840c6770 2023-11-08T19:25:31.979 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.970+0000 7f3e8b86d700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e840b2750 con 0x7f3e840728b0 2023-11-08T19:25:31.979 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.970+0000 7f3e88e08700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e8415a950 0x7f3e841f25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:31.979 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.970+0000 7f3e88e08700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e8415a950 0x7f3e841f25a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50000/0 (socket says 172.21.15.143:50000) 2023-11-08T19:25:31.979 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.970+0000 7f3e88e08700 1 -- 172.21.15.143:0/4196947652 learned_addr learned my addr 172.21.15.143:0/4196947652 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:31.980 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.970+0000 7f3e89e0a700 1 --2- 172.21.15.143:0/4196947652 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3e840c6770 0x7f3e8415d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:31.980 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.970+0000 7f3e89609700 1 --2- 172.21.15.143:0/4196947652 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3e840728b0 0x7f3e8414ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:31.980 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.970+0000 7f3e88e08700 1 -- 172.21.15.143:0/4196947652 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3e840728b0 msgr2=0x7f3e8414ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:31.980 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.970+0000 7f3e88e08700 1 --2- 172.21.15.143:0/4196947652 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3e840728b0 0x7f3e8414ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:31.980 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.970+0000 7f3e88e08700 1 -- 172.21.15.143:0/4196947652 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3e840c6770 msgr2=0x7f3e8415d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:31.980 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.970+0000 7f3e88e08700 1 --2- 172.21.15.143:0/4196947652 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3e840c6770 0x7f3e8415d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:31.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.970+0000 7f3e88e08700 1 -- 172.21.15.143:0/4196947652 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e800094a0 con 0x7f3e8415a950 2023-11-08T19:25:31.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.970+0000 7f3e88e08700 1 --2- 172.21.15.143:0/4196947652 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e8415a950 0x7f3e841f25a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f3e80000c00 tx=0x7f3e80004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:31.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.971+0000 7f3e79ffb700 1 -- 172.21.15.143:0/4196947652 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e800036a0 con 0x7f3e8415a950 2023-11-08T19:25:31.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.971+0000 7f3e79ffb700 1 -- 172.21.15.143:0/4196947652 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e8001f070 con 0x7f3e8415a950 2023-11-08T19:25:31.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.971+0000 7f3e8b86d700 1 -- 172.21.15.143:0/4196947652 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e840b2a30 con 0x7f3e8415a950 2023-11-08T19:25:31.982 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.971+0000 7f3e79ffb700 1 -- 172.21.15.143:0/4196947652 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e80003a20 con 0x7f3e8415a950 2023-11-08T19:25:31.982 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.971+0000 7f3e8b86d700 1 -- 172.21.15.143:0/4196947652 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e840b2ff0 con 0x7f3e8415a950 2023-11-08T19:25:31.982 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.972+0000 7f3e8b86d700 1 -- 172.21.15.143:0/4196947652 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e840b3210 con 0x7f3e8415a950 2023-11-08T19:25:31.982 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.972+0000 7f3e79ffb700 1 -- 172.21.15.143:0/4196947652 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3e80003b80 con 0x7f3e8415a950 2023-11-08T19:25:31.982 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.972+0000 7f3e79ffb700 1 --2- 172.21.15.143:0/4196947652 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f3e6806af50 0x7f3e6806d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:31.983 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.973+0000 7f3e79ffb700 1 -- 172.21.15.143:0/4196947652 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3e80021070 con 0x7f3e8415a950 2023-11-08T19:25:31.983 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.973+0000 7f3e89609700 1 --2- 172.21.15.143:0/4196947652 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f3e6806af50 0x7f3e6806d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:31.983 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.973+0000 7f3e89609700 1 --2- 172.21.15.143:0/4196947652 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f3e6806af50 0x7f3e6806d410 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f3e74006fd0 tx=0x7f3e74008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:31.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:31.977+0000 7f3e79ffb700 1 -- 172.21.15.143:0/4196947652 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3e800560d0 con 0x7f3e8415a950 2023-11-08T19:25:32.167 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.166+0000 7f3e8b86d700 1 -- 172.21.15.143:0/4196947652 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3e8404eee0 con 0x7f3e8415a950 2023-11-08T19:25:32.167 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.166+0000 7f3e79ffb700 1 -- 172.21.15.143:0/4196947652 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3e80015020 con 0x7f3e8415a950 2023-11-08T19:25:32.168 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2023-11-08T19:25:32.170 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.168+0000 7f3e8b86d700 1 -- 172.21.15.143:0/4196947652 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f3e6806af50 msgr2=0x7f3e6806d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:32.170 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.168+0000 7f3e8b86d700 1 --2- 172.21.15.143:0/4196947652 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f3e6806af50 0x7f3e6806d410 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f3e74006fd0 tx=0x7f3e74008040 comp rx=0 tx=0).stop 2023-11-08T19:25:32.171 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.168+0000 7f3e8b86d700 1 -- 172.21.15.143:0/4196947652 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e8415a950 msgr2=0x7f3e841f25a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:32.171 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.168+0000 7f3e8b86d700 1 --2- 172.21.15.143:0/4196947652 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e8415a950 0x7f3e841f25a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f3e80000c00 tx=0x7f3e80004750 comp rx=0 tx=0).stop 2023-11-08T19:25:32.171 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.168+0000 7f3e8b86d700 1 -- 172.21.15.143:0/4196947652 shutdown_connections 2023-11-08T19:25:32.171 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.168+0000 7f3e8b86d700 1 --2- 172.21.15.143:0/4196947652 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f3e6806af50 0x7f3e6806d410 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:32.172 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.168+0000 7f3e8b86d700 1 --2- 172.21.15.143:0/4196947652 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3e840728b0 0x7f3e8414ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:32.172 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.168+0000 7f3e8b86d700 1 --2- 172.21.15.143:0/4196947652 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3e840c6770 0x7f3e8415d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:32.172 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.168+0000 7f3e8b86d700 1 --2- 172.21.15.143:0/4196947652 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e8415a950 0x7f3e841f25a0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:32.172 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.168+0000 7f3e8b86d700 1 -- 172.21.15.143:0/4196947652 >> 172.21.15.143:0/4196947652 conn(0x7f3e840cd2e0 msgr2=0x7f3e840cee00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:32.172 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.168+0000 7f3e8b86d700 1 -- 172.21.15.143:0/4196947652 shutdown_connections 2023-11-08T19:25:32.173 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.168+0000 7f3e8b86d700 1 -- 172.21.15.143:0/4196947652 wait complete. 2023-11-08T19:25:32.182 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-08T19:25:32.304 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.303+0000 7f70724e0700 1 Processor -- start 2023-11-08T19:25:32.305 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.303+0000 7f70724e0700 1 -- start start 2023-11-08T19:25:32.308 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.304+0000 7f70724e0700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f706c06c0c0 0x7f706c0ccfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:32.308 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.304+0000 7f70724e0700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f706c15c9b0 0x7f706c071900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:32.308 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.304+0000 7f70724e0700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f706c0ced30 con 0x7f706c15ce60 2023-11-08T19:25:32.308 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.304+0000 7f70724e0700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f706c0cd4f0 con 0x7f706c06c0c0 2023-11-08T19:25:32.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.304+0000 7f70724e0700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f706c0cdf60 con 0x7f706c15c9b0 2023-11-08T19:25:32.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.304+0000 7f706bfff700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f706c06c0c0 0x7f706c0ccfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:32.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.304+0000 7f7070a7d700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f706c15ce60 0x7f706c1522a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:46164/0 (socket says 172.21.15.143:46164) 2023-11-08T19:25:32.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.304+0000 7f7070a7d700 1 -- 172.21.15.143:0/4190659598 learned_addr learned my addr 172.21.15.143:0/4190659598 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:25:32.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.304+0000 7f706b7fe700 1 --2- 172.21.15.143:0/4190659598 >> v2:172.21.15.174:3300/0 conn(0x7f706c15c9b0 0x7f706c071900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:32.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.304+0000 7f706bfff700 1 -- 172.21.15.143:0/4190659598 >> v2:172.21.15.174:3300/0 conn(0x7f706c15c9b0 msgr2=0x7f706c071900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:32.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.304+0000 7f706bfff700 1 --2- 172.21.15.143:0/4190659598 >> v2:172.21.15.174:3300/0 conn(0x7f706c15c9b0 0x7f706c071900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:32.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.304+0000 7f706bfff700 1 -- 172.21.15.143:0/4190659598 >> v1:172.21.15.143:6789/0 conn(0x7f706c15ce60 legacy=0x7f706c1522a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:32.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.304+0000 7f706affd700 1 -- 172.21.15.143:0/4190659598 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2614809589 0 0) 0x7f706c0ced30 con 0x7f706c15ce60 2023-11-08T19:25:32.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.304+0000 7f706bfff700 1 -- 172.21.15.143:0/4190659598 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f706c0cea50 con 0x7f706c06c0c0 2023-11-08T19:25:32.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.304+0000 7f706bfff700 1 --2- 172.21.15.143:0/4190659598 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f706c06c0c0 0x7f706c0ccfb0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f706c067070 tx=0x7f7060009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50c5aab37f7e8bd6 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:32.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.305+0000 7f706affd700 1 -- 172.21.15.143:0/4190659598 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7060011830 con 0x7f706c06c0c0 2023-11-08T19:25:32.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.305+0000 7f706affd700 1 -- 172.21.15.143:0/4190659598 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7060011990 con 0x7f706c06c0c0 2023-11-08T19:25:32.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.305+0000 7f706affd700 1 -- 172.21.15.143:0/4190659598 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7060011d20 con 0x7f706c06c0c0 2023-11-08T19:25:32.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.305+0000 7f70724e0700 1 -- 172.21.15.143:0/4190659598 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f706c06c0c0 msgr2=0x7f706c0ccfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:32.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.305+0000 7f70724e0700 1 --2- 172.21.15.143:0/4190659598 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f706c06c0c0 0x7f706c0ccfb0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f706c067070 tx=0x7f7060009b20 comp rx=0 tx=0).stop 2023-11-08T19:25:32.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.305+0000 7f70724e0700 1 -- 172.21.15.143:0/4190659598 shutdown_connections 2023-11-08T19:25:32.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.305+0000 7f70724e0700 1 --2- 172.21.15.143:0/4190659598 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f706c06c0c0 0x7f706c0ccfb0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:32.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.305+0000 7f70724e0700 1 --2- 172.21.15.143:0/4190659598 >> v2:172.21.15.174:3300/0 conn(0x7f706c15c9b0 0x7f706c071900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:32.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.305+0000 7f70724e0700 1 -- 172.21.15.143:0/4190659598 >> 172.21.15.143:0/4190659598 conn(0x7f706c0b1c70 msgr2=0x7f706c0b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:32.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.305+0000 7f70724e0700 1 -- 172.21.15.143:0/4190659598 shutdown_connections 2023-11-08T19:25:32.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.305+0000 7f70724e0700 1 -- 172.21.15.143:0/4190659598 wait complete. 2023-11-08T19:25:32.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.307+0000 7f70724e0700 1 Processor -- start 2023-11-08T19:25:32.316 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.307+0000 7f70724e0700 1 -- start start 2023-11-08T19:25:32.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.308+0000 7f70724e0700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f706c06c0c0 0x7f706c14b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:32.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.308+0000 7f70724e0700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f706c15c9b0 0x7f706c150ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:32.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.308+0000 7f70724e0700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f706c15ce60 0x7f706c1e6a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:32.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.308+0000 7f70724e0700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f706c1e5bf0 con 0x7f706c06c0c0 2023-11-08T19:25:32.318 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.308+0000 7f70724e0700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f706c1e72c0 con 0x7f706c15ce60 2023-11-08T19:25:32.318 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.308+0000 7f70724e0700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f706c1e55a0 con 0x7f706c15c9b0 2023-11-08T19:25:32.318 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.308+0000 7f706bfff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f706c06c0c0 0x7f706c14b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:32.318 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.308+0000 7f706bfff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f706c06c0c0 0x7f706c14b7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50012/0 (socket says 172.21.15.143:50012) 2023-11-08T19:25:32.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.308+0000 7f706bfff700 1 -- 172.21.15.143:0/3579454247 learned_addr learned my addr 172.21.15.143:0/3579454247 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:32.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.308+0000 7f706b7fe700 1 --2- 172.21.15.143:0/3579454247 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f706c15c9b0 0x7f706c150ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:32.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.308+0000 7f7070a7d700 1 --2- 172.21.15.143:0/3579454247 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f706c15ce60 0x7f706c1e6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:32.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.308+0000 7f706bfff700 1 -- 172.21.15.143:0/3579454247 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f706c15c9b0 msgr2=0x7f706c150ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:32.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.308+0000 7f706bfff700 1 --2- 172.21.15.143:0/3579454247 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f706c15c9b0 0x7f706c150ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:32.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.308+0000 7f706bfff700 1 -- 172.21.15.143:0/3579454247 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f706c15ce60 msgr2=0x7f706c1e6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:32.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.308+0000 7f706bfff700 1 --2- 172.21.15.143:0/3579454247 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f706c15ce60 0x7f706c1e6a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:32.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.308+0000 7f706bfff700 1 -- 172.21.15.143:0/3579454247 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70600094a0 con 0x7f706c06c0c0 2023-11-08T19:25:32.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.308+0000 7f706bfff700 1 --2- 172.21.15.143:0/3579454247 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f706c06c0c0 0x7f706c14b7d0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f7060000c00 tx=0x7f7060004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:32.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.308+0000 7f7068ff9700 1 -- 172.21.15.143:0/3579454247 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70600038a0 con 0x7f706c06c0c0 2023-11-08T19:25:32.321 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.308+0000 7f7068ff9700 1 -- 172.21.15.143:0/3579454247 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f706001f070 con 0x7f706c06c0c0 2023-11-08T19:25:32.321 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.308+0000 7f70724e0700 1 -- 172.21.15.143:0/3579454247 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f706c1e5880 con 0x7f706c06c0c0 2023-11-08T19:25:32.321 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.309+0000 7f7068ff9700 1 -- 172.21.15.143:0/3579454247 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7060003c40 con 0x7f706c06c0c0 2023-11-08T19:25:32.321 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.309+0000 7f70724e0700 1 -- 172.21.15.143:0/3579454247 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f706c1e75a0 con 0x7f706c06c0c0 2023-11-08T19:25:32.322 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.309+0000 7f70724e0700 1 -- 172.21.15.143:0/3579454247 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f706c04eee0 con 0x7f706c06c0c0 2023-11-08T19:25:32.322 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.310+0000 7f7068ff9700 1 -- 172.21.15.143:0/3579454247 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7060026480 con 0x7f706c06c0c0 2023-11-08T19:25:32.322 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.310+0000 7f7068ff9700 1 --2- 172.21.15.143:0/3579454247 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f705806af00 0x7f705806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:32.323 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.310+0000 7f7068ff9700 1 -- 172.21.15.143:0/3579454247 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7060021070 con 0x7f706c06c0c0 2023-11-08T19:25:32.323 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.311+0000 7f706b7fe700 1 --2- 172.21.15.143:0/3579454247 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f705806af00 0x7f705806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:32.323 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.311+0000 7f706b7fe700 1 --2- 172.21.15.143:0/3579454247 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f705806af00 0x7f705806d3c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f706c14bd10 tx=0x7f705c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:32.324 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.315+0000 7f7068ff9700 1 -- 172.21.15.143:0/3579454247 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7060056090 con 0x7f706c06c0c0 2023-11-08T19:25:32.504 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.503+0000 7f70724e0700 1 -- 172.21.15.143:0/3579454247 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f706c0b6010 con 0x7f706c06c0c0 2023-11-08T19:25:32.504 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.503+0000 7f7068ff9700 1 -- 172.21.15.143:0/3579454247 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f706002b020 con 0x7f706c06c0c0 2023-11-08T19:25:32.505 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 64 2023-11-08T19:25:32.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.505+0000 7f70724e0700 1 -- 172.21.15.143:0/3579454247 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f705806af00 msgr2=0x7f705806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:32.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.505+0000 7f70724e0700 1 --2- 172.21.15.143:0/3579454247 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f705806af00 0x7f705806d3c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f706c14bd10 tx=0x7f705c008040 comp rx=0 tx=0).stop 2023-11-08T19:25:32.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.505+0000 7f70724e0700 1 -- 172.21.15.143:0/3579454247 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f706c06c0c0 msgr2=0x7f706c14b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:32.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.505+0000 7f70724e0700 1 --2- 172.21.15.143:0/3579454247 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f706c06c0c0 0x7f706c14b7d0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f7060000c00 tx=0x7f7060004750 comp rx=0 tx=0).stop 2023-11-08T19:25:32.508 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.505+0000 7f70724e0700 1 -- 172.21.15.143:0/3579454247 shutdown_connections 2023-11-08T19:25:32.508 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.505+0000 7f70724e0700 1 --2- 172.21.15.143:0/3579454247 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f705806af00 0x7f705806d3c0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:32.508 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.505+0000 7f70724e0700 1 --2- 172.21.15.143:0/3579454247 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f706c15ce60 0x7f706c1e6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:32.508 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.505+0000 7f70724e0700 1 --2- 172.21.15.143:0/3579454247 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f706c06c0c0 0x7f706c14b7d0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:32.508 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.505+0000 7f70724e0700 1 --2- 172.21.15.143:0/3579454247 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f706c15c9b0 0x7f706c150ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:32.509 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.505+0000 7f70724e0700 1 -- 172.21.15.143:0/3579454247 >> 172.21.15.143:0/3579454247 conn(0x7f706c0b1c70 msgr2=0x7f706c06b740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:32.509 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.505+0000 7f70724e0700 1 -- 172.21.15.143:0/3579454247 shutdown_connections 2023-11-08T19:25:32.509 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.505+0000 7f70724e0700 1 -- 172.21.15.143:0/3579454247 wait complete. 2023-11-08T19:25:32.519 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-08T19:25:32.651 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.641+0000 7ff8fb1ed700 1 Processor -- start 2023-11-08T19:25:32.652 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.641+0000 7ff8fb1ed700 1 -- start start 2023-11-08T19:25:32.652 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.641+0000 7ff8fb1ed700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff8f415dd00 0x7ff8f40cca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:32.652 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.641+0000 7ff8fb1ed700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7ff8f406b320 0x7ff8f40cb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:32.653 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.641+0000 7ff8fb1ed700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8f40ccfb0 con 0x7ff8f41618f0 2023-11-08T19:25:32.653 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.641+0000 7ff8fb1ed700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8f40cad50 con 0x7ff8f415dd00 2023-11-08T19:25:32.653 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.641+0000 7ff8fb1ed700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8f4161df0 con 0x7ff8f406b320 2023-11-08T19:25:32.654 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.642+0000 7ff8f8f89700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff8f415dd00 0x7ff8f40cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:32.654 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.642+0000 7ff8f8f89700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff8f415dd00 0x7ff8f40cca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50020/0 (socket says 172.21.15.143:50020) 2023-11-08T19:25:32.655 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.642+0000 7ff8f8f89700 1 -- 172.21.15.143:0/2331982181 learned_addr learned my addr 172.21.15.143:0/2331982181 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:32.655 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.642+0000 7ff8f8f89700 1 -- 172.21.15.143:0/2331982181 >> v2:172.21.15.174:3300/0 conn(0x7ff8f406b320 msgr2=0x7ff8f40cb1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:32.655 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.642+0000 7ff8f978a700 1 --2- 172.21.15.143:0/2331982181 >> v2:172.21.15.174:3300/0 conn(0x7ff8f406b320 0x7ff8f40cb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:32.656 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.642+0000 7ff8f8f89700 1 --2- 172.21.15.143:0/2331982181 >> v2:172.21.15.174:3300/0 conn(0x7ff8f406b320 0x7ff8f40cb1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:32.656 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.642+0000 7ff8f8f89700 1 -- 172.21.15.143:0/2331982181 >> v1:172.21.15.174:6789/0 conn(0x7ff8f41618f0 legacy=0x7ff8f406ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:32.657 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.642+0000 7ff8f8f89700 1 -- 172.21.15.143:0/2331982181 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8f41620d0 con 0x7ff8f415dd00 2023-11-08T19:25:32.657 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.642+0000 7ff8f8f89700 1 --2- 172.21.15.143:0/2331982181 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff8f415dd00 0x7ff8f40cca70 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7ff8f40ce8d0 tx=0x7ff8e8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6c548a669379002 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:32.657 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.642+0000 7ff8f37fe700 1 -- 172.21.15.143:0/2331982181 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e8005e30 con 0x7ff8f415dd00 2023-11-08T19:25:32.658 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.642+0000 7ff8f37fe700 1 -- 172.21.15.143:0/2331982181 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8e8006030 con 0x7ff8f415dd00 2023-11-08T19:25:32.658 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.642+0000 7ff8f37fe700 1 -- 172.21.15.143:0/2331982181 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e80063e0 con 0x7ff8f415dd00 2023-11-08T19:25:32.659 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.643+0000 7ff8fb1ed700 1 -- 172.21.15.143:0/2331982181 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff8f415dd00 msgr2=0x7ff8f40cca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:32.659 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.643+0000 7ff8fb1ed700 1 --2- 172.21.15.143:0/2331982181 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff8f415dd00 0x7ff8f40cca70 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7ff8f40ce8d0 tx=0x7ff8e8009bd0 comp rx=0 tx=0).stop 2023-11-08T19:25:32.659 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.643+0000 7ff8fb1ed700 1 -- 172.21.15.143:0/2331982181 shutdown_connections 2023-11-08T19:25:32.660 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.643+0000 7ff8fb1ed700 1 --2- 172.21.15.143:0/2331982181 >> v2:172.21.15.174:3300/0 conn(0x7ff8f406b320 0x7ff8f40cb1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:32.660 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.643+0000 7ff8fb1ed700 1 --2- 172.21.15.143:0/2331982181 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff8f415dd00 0x7ff8f40cca70 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:32.660 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.643+0000 7ff8fb1ed700 1 -- 172.21.15.143:0/2331982181 >> 172.21.15.143:0/2331982181 conn(0x7ff8f4099ab0 msgr2=0x7ff8f40d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:32.660 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.643+0000 7ff8fb1ed700 1 -- 172.21.15.143:0/2331982181 shutdown_connections 2023-11-08T19:25:32.661 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.643+0000 7ff8fb1ed700 1 -- 172.21.15.143:0/2331982181 wait complete. 2023-11-08T19:25:32.661 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.644+0000 7ff8fb1ed700 1 Processor -- start 2023-11-08T19:25:32.661 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.644+0000 7ff8fb1ed700 1 -- start start 2023-11-08T19:25:32.662 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.645+0000 7ff8fb1ed700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff8f406b320 0x7ff8f40b5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:32.662 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.645+0000 7ff8fb1ed700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff8f415dd00 0x7ff8f40b00f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:32.662 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.645+0000 7ff8fb1ed700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff8f41618f0 0x7ff8f40b0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:32.662 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.646+0000 7ff8fb1ed700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f40b18f0 con 0x7ff8f415dd00 2023-11-08T19:25:32.663 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.646+0000 7ff8fb1ed700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f40c4ff0 con 0x7ff8f406b320 2023-11-08T19:25:32.663 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.646+0000 7ff8fb1ed700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f4156050 con 0x7ff8f41618f0 2023-11-08T19:25:32.663 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.646+0000 7ff8f3fff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff8f415dd00 0x7ff8f40b00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:32.664 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.646+0000 7ff8f3fff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff8f415dd00 0x7ff8f40b00f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50026/0 (socket says 172.21.15.143:50026) 2023-11-08T19:25:32.664 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.646+0000 7ff8f3fff700 1 -- 172.21.15.143:0/3685877906 learned_addr learned my addr 172.21.15.143:0/3685877906 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:32.664 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.646+0000 7ff8f978a700 1 --2- 172.21.15.143:0/3685877906 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff8f41618f0 0x7ff8f40b0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:32.664 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.646+0000 7ff8f8f89700 1 --2- 172.21.15.143:0/3685877906 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff8f406b320 0x7ff8f40b5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:32.665 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.646+0000 7ff8f3fff700 1 -- 172.21.15.143:0/3685877906 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff8f41618f0 msgr2=0x7ff8f40b0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:32.665 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.646+0000 7ff8f3fff700 1 --2- 172.21.15.143:0/3685877906 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff8f41618f0 0x7ff8f40b0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:32.665 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.646+0000 7ff8f3fff700 1 -- 172.21.15.143:0/3685877906 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff8f406b320 msgr2=0x7ff8f40b5870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:32.665 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.646+0000 7ff8f3fff700 1 --2- 172.21.15.143:0/3685877906 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff8f406b320 0x7ff8f40b5870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:32.665 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.646+0000 7ff8f3fff700 1 -- 172.21.15.143:0/3685877906 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8e8009500 con 0x7ff8f415dd00 2023-11-08T19:25:32.666 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.646+0000 7ff8f3fff700 1 --2- 172.21.15.143:0/3685877906 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff8f415dd00 0x7ff8f40b00f0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ff8e000b470 tx=0x7ff8e000b780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:32.666 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.646+0000 7ff8f17fa700 1 -- 172.21.15.143:0/3685877906 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e0011830 con 0x7ff8f415dd00 2023-11-08T19:25:32.666 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.646+0000 7ff8f17fa700 1 -- 172.21.15.143:0/3685877906 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8e00119b0 con 0x7ff8f415dd00 2023-11-08T19:25:32.666 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.646+0000 7ff8fb1ed700 1 -- 172.21.15.143:0/3685877906 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8f4157cf0 con 0x7ff8f415dd00 2023-11-08T19:25:32.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.646+0000 7ff8f17fa700 1 -- 172.21.15.143:0/3685877906 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e0011d60 con 0x7ff8f415dd00 2023-11-08T19:25:32.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.646+0000 7ff8fb1ed700 1 -- 172.21.15.143:0/3685877906 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8f40c4930 con 0x7ff8f415dd00 2023-11-08T19:25:32.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.647+0000 7ff8fb1ed700 1 -- 172.21.15.143:0/3685877906 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8f40aff60 con 0x7ff8f415dd00 2023-11-08T19:25:32.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.648+0000 7ff8f17fa700 1 -- 172.21.15.143:0/3685877906 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff8e001a400 con 0x7ff8f415dd00 2023-11-08T19:25:32.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.648+0000 7ff8f17fa700 1 --2- 172.21.15.143:0/3685877906 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7ff8d806af00 0x7ff8d806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:32.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.648+0000 7ff8f17fa700 1 -- 172.21.15.143:0/3685877906 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff8e00894a0 con 0x7ff8f415dd00 2023-11-08T19:25:32.668 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.648+0000 7ff8f8f89700 1 --2- 172.21.15.143:0/3685877906 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7ff8d806af00 0x7ff8d806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:32.668 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.649+0000 7ff8f8f89700 1 --2- 172.21.15.143:0/3685877906 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7ff8d806af00 0x7ff8d806d3c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff8f40ce8d0 tx=0x7ff8e8008070 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:32.668 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.652+0000 7ff8f17fa700 1 -- 172.21.15.143:0/3685877906 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff8e00545e0 con 0x7ff8f415dd00 2023-11-08T19:25:32.859 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.858+0000 7ff8fb1ed700 1 -- 172.21.15.143:0/3685877906 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ff8f40c5130 con 0x7ff8f415dd00 2023-11-08T19:25:32.860 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.858+0000 7ff8f17fa700 1 -- 172.21.15.143:0/3685877906 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) v1 ==== 80+0+0 (secure 0 0 0) 0x7ff8f40c5130 con 0x7ff8f415dd00 2023-11-08T19:25:32.862 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.860+0000 7ff8fb1ed700 1 -- 172.21.15.143:0/3685877906 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7ff8d806af00 msgr2=0x7ff8d806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:32.862 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.860+0000 7ff8fb1ed700 1 --2- 172.21.15.143:0/3685877906 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7ff8d806af00 0x7ff8d806d3c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff8f40ce8d0 tx=0x7ff8e8008070 comp rx=0 tx=0).stop 2023-11-08T19:25:32.862 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.860+0000 7ff8fb1ed700 1 -- 172.21.15.143:0/3685877906 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff8f415dd00 msgr2=0x7ff8f40b00f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:32.862 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.860+0000 7ff8fb1ed700 1 --2- 172.21.15.143:0/3685877906 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff8f415dd00 0x7ff8f40b00f0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ff8e000b470 tx=0x7ff8e000b780 comp rx=0 tx=0).stop 2023-11-08T19:25:32.863 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.860+0000 7ff8fb1ed700 1 -- 172.21.15.143:0/3685877906 shutdown_connections 2023-11-08T19:25:32.863 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.860+0000 7ff8fb1ed700 1 --2- 172.21.15.143:0/3685877906 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7ff8d806af00 0x7ff8d806d3c0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:32.863 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.860+0000 7ff8fb1ed700 1 --2- 172.21.15.143:0/3685877906 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff8f41618f0 0x7ff8f40b0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:32.863 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.860+0000 7ff8fb1ed700 1 --2- 172.21.15.143:0/3685877906 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff8f406b320 0x7ff8f40b5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:32.863 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.860+0000 7ff8fb1ed700 1 --2- 172.21.15.143:0/3685877906 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff8f415dd00 0x7ff8f40b00f0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:32.864 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.860+0000 7ff8fb1ed700 1 -- 172.21.15.143:0/3685877906 >> 172.21.15.143:0/3685877906 conn(0x7ff8f4099ab0 msgr2=0x7ff8f406f370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:32.864 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.860+0000 7ff8fb1ed700 1 -- 172.21.15.143:0/3685877906 shutdown_connections 2023-11-08T19:25:32.864 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:32.860+0000 7ff8fb1ed700 1 -- 172.21.15.143:0/3685877906 wait complete. 2023-11-08T19:25:32.865 INFO:teuthology.orchestra.run.smithi143.stderr:listed 0 entries 2023-11-08T19:25:32.874 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2023-11-08T19:25:32.874 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:32.874 DEBUG:teuthology.orchestra.run.smithi143:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2023-11-08T19:25:32.912 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi143.front.sepia.ceph.com 2023-11-08T19:25:32.912 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:32.912 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-11-08T19:25:32.969 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2023-11-08T19:25:32.970 DEBUG:teuthology.orchestra.run.smithi174:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:25:32.992 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2023-11-08T19:25:32.993 DEBUG:teuthology.orchestra.run.smithi174:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:25:33.053 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2023-11-08T19:25:33.054 DEBUG:teuthology.orchestra.run.smithi174:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:25:33.113 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:25:33.234 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.233+0000 7f4e56476700 1 Processor -- start 2023-11-08T19:25:33.235 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.233+0000 7f4e56476700 1 -- start start 2023-11-08T19:25:33.237 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.234+0000 7f4e56476700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f4e501618f0 0x7f4e5006ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:33.237 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.234+0000 7f4e56476700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4e5006b150 0x7f4e500cb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:33.237 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.234+0000 7f4e56476700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e50161df0 con 0x7f4e5015dd00 2023-11-08T19:25:33.237 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.234+0000 7f4e56476700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e5006b600 con 0x7f4e501618f0 2023-11-08T19:25:33.237 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.234+0000 7f4e56476700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e500cad50 con 0x7f4e5006b150 2023-11-08T19:25:33.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.234+0000 7f4e4f7fe700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f4e501618f0 0x7f4e5006ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:33.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.234+0000 7f4e4ffff700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f4e5015dd00 0x7f4e500cca70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:46186/0 (socket says 172.21.15.143:46186) 2023-11-08T19:25:33.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.234+0000 7f4e4ffff700 1 -- 172.21.15.143:0/168251910 learned_addr learned my addr 172.21.15.143:0/168251910 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:25:33.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.234+0000 7f4e54a13700 1 --2- 172.21.15.143:0/168251910 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4e5006b150 0x7f4e500cb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:33.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.234+0000 7f4e4f7fe700 1 -- 172.21.15.143:0/168251910 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4e5006b150 msgr2=0x7f4e500cb1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:33.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.234+0000 7f4e4f7fe700 1 --2- 172.21.15.143:0/168251910 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4e5006b150 0x7f4e500cb1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:33.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.234+0000 7f4e4f7fe700 1 -- 172.21.15.143:0/168251910 >> v1:172.21.15.143:6789/0 conn(0x7f4e5015dd00 legacy=0x7f4e500cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:33.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.234+0000 7f4e4f7fe700 1 -- 172.21.15.143:0/168251910 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e501620d0 con 0x7f4e501618f0 2023-11-08T19:25:33.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.234+0000 7f4e4effd700 1 -- 172.21.15.143:0/168251910 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1813974879 0 0) 0x7f4e50161df0 con 0x7f4e5015dd00 2023-11-08T19:25:33.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.234+0000 7f4e4f7fe700 1 --2- 172.21.15.143:0/168251910 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4e501618f0 0x7f4e5006ac10 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f4e44009840 tx=0x7f4e44009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bad6c33467fb1784 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:33.240 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.234+0000 7f4e4effd700 1 -- 172.21.15.143:0/168251910 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e44011830 con 0x7f4e501618f0 2023-11-08T19:25:33.240 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.234+0000 7f4e4effd700 1 -- 172.21.15.143:0/168251910 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e44011990 con 0x7f4e501618f0 2023-11-08T19:25:33.240 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.234+0000 7f4e4effd700 1 -- 172.21.15.143:0/168251910 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e44011d20 con 0x7f4e501618f0 2023-11-08T19:25:33.240 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.235+0000 7f4e56476700 1 -- 172.21.15.143:0/168251910 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4e501618f0 msgr2=0x7f4e5006ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:33.240 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.235+0000 7f4e56476700 1 --2- 172.21.15.143:0/168251910 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4e501618f0 0x7f4e5006ac10 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f4e44009840 tx=0x7f4e44009b50 comp rx=0 tx=0).stop 2023-11-08T19:25:33.241 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.235+0000 7f4e56476700 1 -- 172.21.15.143:0/168251910 shutdown_connections 2023-11-08T19:25:33.241 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.235+0000 7f4e56476700 1 --2- 172.21.15.143:0/168251910 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4e5006b150 0x7f4e500cb1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:33.241 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.235+0000 7f4e56476700 1 --2- 172.21.15.143:0/168251910 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4e501618f0 0x7f4e5006ac10 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:33.241 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.235+0000 7f4e56476700 1 -- 172.21.15.143:0/168251910 >> 172.21.15.143:0/168251910 conn(0x7f4e50099ab0 msgr2=0x7f4e500d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:33.241 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.235+0000 7f4e56476700 1 -- 172.21.15.143:0/168251910 shutdown_connections 2023-11-08T19:25:33.242 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.235+0000 7f4e56476700 1 -- 172.21.15.143:0/168251910 wait complete. 2023-11-08T19:25:33.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.236+0000 7f4e56476700 1 Processor -- start 2023-11-08T19:25:33.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.236+0000 7f4e56476700 1 -- start start 2023-11-08T19:25:33.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.237+0000 7f4e56476700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4e5006b150 0x7f4e500b5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:33.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.237+0000 7f4e56476700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4e5015dd00 0x7f4e50158050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:33.246 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.237+0000 7f4e56476700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4e501618f0 0x7f4e500c5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:33.246 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.237+0000 7f4e56476700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e500b6300 con 0x7f4e5015dd00 2023-11-08T19:25:33.246 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.237+0000 7f4e56476700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e501f7ba0 con 0x7f4e5006b150 2023-11-08T19:25:33.246 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.237+0000 7f4e56476700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e501f7020 con 0x7f4e501618f0 2023-11-08T19:25:33.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.237+0000 7f4e4f7fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4e5015dd00 0x7f4e50158050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:33.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.237+0000 7f4e4f7fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4e5015dd00 0x7f4e50158050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50042/0 (socket says 172.21.15.143:50042) 2023-11-08T19:25:33.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.237+0000 7f4e4f7fe700 1 -- 172.21.15.143:0/2430624651 learned_addr learned my addr 172.21.15.143:0/2430624651 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:33.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.237+0000 7f4e54a13700 1 --2- 172.21.15.143:0/2430624651 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4e501618f0 0x7f4e500c5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:33.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.237+0000 7f4e4ffff700 1 --2- 172.21.15.143:0/2430624651 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4e5006b150 0x7f4e500b5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:33.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.237+0000 7f4e4f7fe700 1 -- 172.21.15.143:0/2430624651 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4e501618f0 msgr2=0x7f4e500c5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:33.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.237+0000 7f4e4f7fe700 1 --2- 172.21.15.143:0/2430624651 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4e501618f0 0x7f4e500c5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:33.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.237+0000 7f4e4f7fe700 1 -- 172.21.15.143:0/2430624651 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4e5006b150 msgr2=0x7f4e500b5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:33.249 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.237+0000 7f4e4f7fe700 1 --2- 172.21.15.143:0/2430624651 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4e5006b150 0x7f4e500b5dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:33.249 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.237+0000 7f4e4f7fe700 1 -- 172.21.15.143:0/2430624651 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e440094a0 con 0x7f4e5015dd00 2023-11-08T19:25:33.249 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.237+0000 7f4e54a13700 1 --2- 172.21.15.143:0/2430624651 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4e501618f0 0x7f4e500c5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:25:33.249 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.238+0000 7f4e4f7fe700 1 --2- 172.21.15.143:0/2430624651 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4e5015dd00 0x7f4e50158050 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f4e440090b0 tx=0x7f4e44004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:33.249 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.238+0000 7f4e4cff9700 1 -- 172.21.15.143:0/2430624651 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e440038a0 con 0x7f4e5015dd00 2023-11-08T19:25:33.250 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.238+0000 7f4e4cff9700 1 -- 172.21.15.143:0/2430624651 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e4401f070 con 0x7f4e5015dd00 2023-11-08T19:25:33.250 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.238+0000 7f4e56476700 1 -- 172.21.15.143:0/2430624651 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e501f7300 con 0x7f4e5015dd00 2023-11-08T19:25:33.250 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.238+0000 7f4e4cff9700 1 -- 172.21.15.143:0/2430624651 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e44003c00 con 0x7f4e5015dd00 2023-11-08T19:25:33.250 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.238+0000 7f4e56476700 1 -- 172.21.15.143:0/2430624651 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e501f78c0 con 0x7f4e5015dd00 2023-11-08T19:25:33.251 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.239+0000 7f4e56476700 1 -- 172.21.15.143:0/2430624651 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e500aff60 con 0x7f4e5015dd00 2023-11-08T19:25:33.251 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.239+0000 7f4e4cff9700 1 -- 172.21.15.143:0/2430624651 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4e4401a430 con 0x7f4e5015dd00 2023-11-08T19:25:33.251 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.240+0000 7f4e4cff9700 1 --2- 172.21.15.143:0/2430624651 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f4e3c06af00 0x7f4e3c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:33.251 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.240+0000 7f4e4cff9700 1 -- 172.21.15.143:0/2430624651 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4e44021070 con 0x7f4e5015dd00 2023-11-08T19:25:33.251 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.240+0000 7f4e4ffff700 1 --2- 172.21.15.143:0/2430624651 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f4e3c06af00 0x7f4e3c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:33.252 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.240+0000 7f4e4ffff700 1 --2- 172.21.15.143:0/2430624651 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f4e3c06af00 0x7f4e3c06d3c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f4e40007fd0 tx=0x7f4e40009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:33.252 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.244+0000 7f4e4cff9700 1 -- 172.21.15.143:0/2430624651 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4e4408e030 con 0x7f4e5015dd00 2023-11-08T19:25:33.437 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.436+0000 7f4e56476700 1 -- 172.21.15.143:0/2430624651 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4e500b3420 con 0x7f4e5015dd00 2023-11-08T19:25:33.438 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.437+0000 7f4e4cff9700 1 -- 172.21.15.143:0/2430624651 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16553 (secure 0 0 0) 0x7f4e500b3420 con 0x7f4e5015dd00 2023-11-08T19:25:33.439 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:25:33.439 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":20,"fsid":"968483af-77fc-4068-ad7c-5c2a1a4613d7","created":"2023-11-08T19:24:32.826484+0000","modified":"2023-11-08T19:24:51.618072+0000","last_up_change":"2023-11-08T19:24:43.573880+0000","last_in_change":"2023-11-08T19:24:35.314525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:44.551530+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-08T19:24:48.256781+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-08T19:24:48.954990+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bae98ec1-d94e-403d-a220-a522a53ea6f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6809","nonce":1487830566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6811","nonce":1487830566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6815","nonce":1487830566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6813","nonce":1487830566}]},"public_addr":"172.21.15.143:6809/1487830566","cluster_addr":"172.21.15.143:6811/1487830566","heartbeat_back_addr":"172.21.15.143:6815/1487830566","heartbeat_front_addr":"172.21.15.143:6813/1487830566","state":["exists","up"]},{"osd":1,"uuid":"5cb6538a-1ccb-4322-8633-09f66af0b7e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6801","nonce":787341397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6803","nonce":787341397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6807","nonce":787341397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6805","nonce":787341397}]},"public_addr":"172.21.15.143:6801/787341397","cluster_addr":"172.21.15.143:6803/787341397","heartbeat_back_addr":"172.21.15.143:6807/787341397","heartbeat_front_addr":"172.21.15.143:6805/787341397","state":["exists","up"]},{"osd":2,"uuid":"cdbf522c-60ef-4d60-9653-8dcb529c101a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6822","nonce":2301325597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6826","nonce":2301325597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6831","nonce":2301325597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6829","nonce":2301325597}]},"public_addr":"172.21.15.143:6822/2301325597","cluster_addr":"172.21.15.143:6826/2301325597","heartbeat_back_addr":"172.21.15.143:6831/2301325597","heartbeat_front_addr":"172.21.15.143:6829/2301325597","state":["exists","up"]},{"osd":3,"uuid":"4e2783cf-ebb9-4dd8-8245-bc1d17ce4325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6817","nonce":2021504428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6819","nonce":2021504428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6827","nonce":2021504428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6823","nonce":2021504428}]},"public_addr":"172.21.15.143:6817/2021504428","cluster_addr":"172.21.15.143:6819/2021504428","heartbeat_back_addr":"172.21.15.143:6827/2021504428","heartbeat_front_addr":"172.21.15.143:6823/2021504428","state":["exists","up"]},{"osd":4,"uuid":"691256fa-9b77-49b9-a346-7ae1adf0ca80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6801","nonce":1993380595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6803","nonce":1993380595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6807","nonce":1993380595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6805","nonce":1993380595}]},"public_addr":"172.21.15.174:6801/1993380595","cluster_addr":"172.21.15.174:6803/1993380595","heartbeat_back_addr":"172.21.15.174:6807/1993380595","heartbeat_front_addr":"172.21.15.174:6805/1993380595","state":["exists","up"]},{"osd":5,"uuid":"756fc352-48c9-46ce-b52e-0541fe6f8525","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6809","nonce":1777923303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6811","nonce":1777923303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6815","nonce":1777923303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6813","nonce":1777923303}]},"public_addr":"172.21.15.174:6809/1777923303","cluster_addr":"172.21.15.174:6811/1777923303","heartbeat_back_addr":"172.21.15.174:6815/1777923303","heartbeat_front_addr":"172.21.15.174:6813/1777923303","state":["exists","up"]},{"osd":6,"uuid":"fab2ac2b-ed97-4cde-9825-b480eabf8b8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6825","nonce":2755130316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6827","nonce":2755130316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6831","nonce":2755130316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6829","nonce":2755130316}]},"public_addr":"172.21.15.174:6825/2755130316","cluster_addr":"172.21.15.174:6827/2755130316","heartbeat_back_addr":"172.21.15.174:6831/2755130316","heartbeat_front_addr":"172.21.15.174:6829/2755130316","state":["exists","up"]},{"osd":7,"uuid":"3de9b797-2ab6-4d41-a863-7e7d638fd703","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6817","nonce":2847141411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6819","nonce":2847141411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6823","nonce":2847141411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6821","nonce":2847141411}]},"public_addr":"172.21.15.174:6817/2847141411","cluster_addr":"172.21.15.174:6819/2847141411","heartbeat_back_addr":"172.21.15.174:6823/2847141411","heartbeat_front_addr":"172.21.15.174:6821/2847141411","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.477076+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.496890+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.539305+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.454673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.538881+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.521294+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.520712+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.540404+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:25:33.441 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.439+0000 7f4e56476700 1 -- 172.21.15.143:0/2430624651 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f4e3c06af00 msgr2=0x7f4e3c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:33.441 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.439+0000 7f4e56476700 1 --2- 172.21.15.143:0/2430624651 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f4e3c06af00 0x7f4e3c06d3c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f4e40007fd0 tx=0x7f4e40009040 comp rx=0 tx=0).stop 2023-11-08T19:25:33.441 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.439+0000 7f4e56476700 1 -- 172.21.15.143:0/2430624651 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4e5015dd00 msgr2=0x7f4e50158050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:33.441 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.439+0000 7f4e56476700 1 --2- 172.21.15.143:0/2430624651 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4e5015dd00 0x7f4e50158050 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f4e440090b0 tx=0x7f4e44004750 comp rx=0 tx=0).stop 2023-11-08T19:25:33.442 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.439+0000 7f4e56476700 1 -- 172.21.15.143:0/2430624651 shutdown_connections 2023-11-08T19:25:33.442 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.439+0000 7f4e56476700 1 --2- 172.21.15.143:0/2430624651 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f4e3c06af00 0x7f4e3c06d3c0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:33.442 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.439+0000 7f4e56476700 1 --2- 172.21.15.143:0/2430624651 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4e501618f0 0x7f4e500c5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:33.442 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.439+0000 7f4e56476700 1 --2- 172.21.15.143:0/2430624651 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4e5006b150 0x7f4e500b5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:33.443 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.439+0000 7f4e56476700 1 --2- 172.21.15.143:0/2430624651 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4e5015dd00 0x7f4e50158050 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:33.443 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.439+0000 7f4e56476700 1 -- 172.21.15.143:0/2430624651 >> 172.21.15.143:0/2430624651 conn(0x7f4e50099ab0 msgr2=0x7f4e500ad410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:33.443 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.439+0000 7f4e56476700 1 -- 172.21.15.143:0/2430624651 shutdown_connections 2023-11-08T19:25:33.443 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.439+0000 7f4e56476700 1 -- 172.21.15.143:0/2430624651 wait complete. 2023-11-08T19:25:33.453 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T19:24:44.551530+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-08T19:24:48.256781+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-08T19:24:48.954990+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-08T19:25:33.454 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T19:25:33.577 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.576+0000 7f5bdf95f700 1 Processor -- start 2023-11-08T19:25:33.577 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.576+0000 7f5bdf95f700 1 -- start start 2023-11-08T19:25:33.580 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.577+0000 7f5bdf95f700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f5bd80b61e0 0x7f5bd80d01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:33.580 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.577+0000 7f5bdf95f700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bd80cc2a0 con 0x7f5bd815a950 2023-11-08T19:25:33.580 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.577+0000 7f5bdf95f700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bd806c1b0 con 0x7f5bd806bd20 2023-11-08T19:25:33.580 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.577+0000 7f5bdf95f700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bd80cc130 con 0x7f5bd80b61e0 2023-11-08T19:25:33.581 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.577+0000 7f5bddefc700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f5bd80b61e0 0x7f5bd80d01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:33.581 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.577+0000 7f5bdd6fb700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f5bd815a950 0x7f5bd8072370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:46200/0 (socket says 172.21.15.143:46200) 2023-11-08T19:25:33.581 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.577+0000 7f5bdd6fb700 1 -- 172.21.15.143:0/2919185362 learned_addr learned my addr 172.21.15.143:0/2919185362 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:25:33.581 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.577+0000 7f5bddefc700 1 -- 172.21.15.143:0/2919185362 >> v1:172.21.15.174:6789/0 conn(0x7f5bd806bd20 legacy=0x7f5bd80cc730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:33.581 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.577+0000 7f5bddefc700 1 -- 172.21.15.143:0/2919185362 >> v1:172.21.15.143:6789/0 conn(0x7f5bd815a950 legacy=0x7f5bd8072370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:33.582 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.577+0000 7f5bcffff700 1 -- 172.21.15.143:0/2919185362 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4118122926 0 0) 0x7f5bd80cc2a0 con 0x7f5bd815a950 2023-11-08T19:25:33.582 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.577+0000 7f5bddefc700 1 -- 172.21.15.143:0/2919185362 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bd806c490 con 0x7f5bd80b61e0 2023-11-08T19:25:33.582 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.577+0000 7f5bddefc700 1 --2- 172.21.15.143:0/2919185362 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5bd80b61e0 0x7f5bd80d01b0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f5bd0009d90 tx=0x7f5bd000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e1cc3a4ec706a6c server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:33.582 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.577+0000 7f5bcffff700 1 -- 172.21.15.143:0/2919185362 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd000e070 con 0x7f5bd80b61e0 2023-11-08T19:25:33.582 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.577+0000 7f5bcffff700 1 -- 172.21.15.143:0/2919185362 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bd000bbd0 con 0x7f5bd80b61e0 2023-11-08T19:25:33.583 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.577+0000 7f5bcffff700 1 -- 172.21.15.143:0/2919185362 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd0012430 con 0x7f5bd80b61e0 2023-11-08T19:25:33.583 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.578+0000 7f5bdf95f700 1 -- 172.21.15.143:0/2919185362 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5bd80b61e0 msgr2=0x7f5bd80d01b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:33.583 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.578+0000 7f5bdf95f700 1 --2- 172.21.15.143:0/2919185362 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5bd80b61e0 0x7f5bd80d01b0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f5bd0009d90 tx=0x7f5bd000b330 comp rx=0 tx=0).stop 2023-11-08T19:25:33.583 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.578+0000 7f5bdf95f700 1 -- 172.21.15.143:0/2919185362 shutdown_connections 2023-11-08T19:25:33.583 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.578+0000 7f5bdf95f700 1 --2- 172.21.15.143:0/2919185362 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5bd80b61e0 0x7f5bd80d01b0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:33.584 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.578+0000 7f5bdf95f700 1 -- 172.21.15.143:0/2919185362 >> 172.21.15.143:0/2919185362 conn(0x7f5bd80cd2e0 msgr2=0x7f5bd80cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:33.585 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.578+0000 7f5bdf95f700 1 -- 172.21.15.143:0/2919185362 shutdown_connections 2023-11-08T19:25:33.585 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.578+0000 7f5bdf95f700 1 -- 172.21.15.143:0/2919185362 wait complete. 2023-11-08T19:25:33.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.579+0000 7f5bdf95f700 1 Processor -- start 2023-11-08T19:25:33.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.579+0000 7f5bdf95f700 1 -- start start 2023-11-08T19:25:33.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.580+0000 7f5bdf95f700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5bd80b61e0 0x7f5bd80b2040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:33.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.580+0000 7f5bdf95f700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5bd815a950 0x7f5bd814dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:33.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.580+0000 7f5bdf95f700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5bd81588c0 0x7f5bd8158d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:33.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.580+0000 7f5bdf95f700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd81592b0 con 0x7f5bd80b61e0 2023-11-08T19:25:33.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.580+0000 7f5bdf95f700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd81f0470 con 0x7f5bd81588c0 2023-11-08T19:25:33.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.580+0000 7f5bdf95f700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd80b2580 con 0x7f5bd815a950 2023-11-08T19:25:33.590 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.580+0000 7f5bdd6fb700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5bd80b61e0 0x7f5bd80b2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:33.590 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.580+0000 7f5bdd6fb700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5bd80b61e0 0x7f5bd80b2040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50052/0 (socket says 172.21.15.143:50052) 2023-11-08T19:25:33.590 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.580+0000 7f5bdd6fb700 1 -- 172.21.15.143:0/1957108260 learned_addr learned my addr 172.21.15.143:0/1957108260 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:33.590 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.580+0000 7f5bdcefa700 1 --2- 172.21.15.143:0/1957108260 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5bd815a950 0x7f5bd814dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:33.591 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.580+0000 7f5bddefc700 1 --2- 172.21.15.143:0/1957108260 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5bd81588c0 0x7f5bd8158d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:33.591 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.580+0000 7f5bdd6fb700 1 -- 172.21.15.143:0/1957108260 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5bd815a950 msgr2=0x7f5bd814dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:33.591 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.580+0000 7f5bdd6fb700 1 --2- 172.21.15.143:0/1957108260 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5bd815a950 0x7f5bd814dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:33.591 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.580+0000 7f5bdd6fb700 1 -- 172.21.15.143:0/1957108260 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5bd81588c0 msgr2=0x7f5bd8158d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:33.591 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.580+0000 7f5bdd6fb700 1 --2- 172.21.15.143:0/1957108260 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5bd81588c0 0x7f5bd8158d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:33.592 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.580+0000 7f5bdd6fb700 1 -- 172.21.15.143:0/1957108260 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bd0009a70 con 0x7f5bd80b61e0 2023-11-08T19:25:33.592 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.580+0000 7f5bdcefa700 1 --2- 172.21.15.143:0/1957108260 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5bd815a950 0x7f5bd814dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:25:33.592 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.580+0000 7f5bddefc700 1 --2- 172.21.15.143:0/1957108260 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5bd81588c0 0x7f5bd8158d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:25:33.592 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.580+0000 7f5bdd6fb700 1 --2- 172.21.15.143:0/1957108260 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5bd80b61e0 0x7f5bd80b2040 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f5bc800c550 tx=0x7f5bc800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:33.592 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.581+0000 7f5bcdffb700 1 -- 172.21.15.143:0/1957108260 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bc8004c80 con 0x7f5bd80b61e0 2023-11-08T19:25:33.593 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.581+0000 7f5bcdffb700 1 -- 172.21.15.143:0/1957108260 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bc8004e00 con 0x7f5bd80b61e0 2023-11-08T19:25:33.593 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.581+0000 7f5bdf95f700 1 -- 172.21.15.143:0/1957108260 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bd80b03a0 con 0x7f5bd80b61e0 2023-11-08T19:25:33.593 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.581+0000 7f5bcdffb700 1 -- 172.21.15.143:0/1957108260 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bc8007310 con 0x7f5bd80b61e0 2023-11-08T19:25:33.593 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.581+0000 7f5bdf95f700 1 -- 172.21.15.143:0/1957108260 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bd80b0960 con 0x7f5bd80b61e0 2023-11-08T19:25:33.593 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.582+0000 7f5bdf95f700 1 -- 172.21.15.143:0/1957108260 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bd80b3b80 con 0x7f5bd80b61e0 2023-11-08T19:25:33.593 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.582+0000 7f5bcdffb700 1 -- 172.21.15.143:0/1957108260 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5bc8007470 con 0x7f5bd80b61e0 2023-11-08T19:25:33.594 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.582+0000 7f5bcdffb700 1 --2- 172.21.15.143:0/1957108260 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f5bbc06af00 0x7f5bbc06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:33.594 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.583+0000 7f5bcdffb700 1 -- 172.21.15.143:0/1957108260 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5bc8089330 con 0x7f5bd80b61e0 2023-11-08T19:25:33.594 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.583+0000 7f5bdcefa700 1 --2- 172.21.15.143:0/1957108260 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f5bbc06af00 0x7f5bbc06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:33.594 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.583+0000 7f5bdcefa700 1 --2- 172.21.15.143:0/1957108260 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f5bbc06af00 0x7f5bbc06d3c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f5bd4000f60 tx=0x7f5bd4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:33.595 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.587+0000 7f5bcdffb700 1 -- 172.21.15.143:0/1957108260 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5bc80543e0 con 0x7f5bd80b61e0 2023-11-08T19:25:33.777 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.776+0000 7f5bdf95f700 1 -- 172.21.15.143:0/1957108260 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5bd804eee0 con 0x7f5bd80b61e0 2023-11-08T19:25:33.778 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.776+0000 7f5bcdffb700 1 -- 172.21.15.143:0/1957108260 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5bc8057a00 con 0x7f5bd80b61e0 2023-11-08T19:25:33.778 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2023-11-08T19:25:33.780 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.778+0000 7f5bdf95f700 1 -- 172.21.15.143:0/1957108260 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f5bbc06af00 msgr2=0x7f5bbc06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:33.780 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.778+0000 7f5bdf95f700 1 --2- 172.21.15.143:0/1957108260 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f5bbc06af00 0x7f5bbc06d3c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f5bd4000f60 tx=0x7f5bd4009040 comp rx=0 tx=0).stop 2023-11-08T19:25:33.781 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.778+0000 7f5bdf95f700 1 -- 172.21.15.143:0/1957108260 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5bd80b61e0 msgr2=0x7f5bd80b2040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:33.781 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.778+0000 7f5bdf95f700 1 --2- 172.21.15.143:0/1957108260 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5bd80b61e0 0x7f5bd80b2040 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f5bc800c550 tx=0x7f5bc800c910 comp rx=0 tx=0).stop 2023-11-08T19:25:33.781 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.778+0000 7f5bdf95f700 1 -- 172.21.15.143:0/1957108260 shutdown_connections 2023-11-08T19:25:33.781 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.778+0000 7f5bdf95f700 1 --2- 172.21.15.143:0/1957108260 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f5bbc06af00 0x7f5bbc06d3c0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:33.781 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.778+0000 7f5bdf95f700 1 --2- 172.21.15.143:0/1957108260 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5bd81588c0 0x7f5bd8158d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:33.782 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.778+0000 7f5bdf95f700 1 --2- 172.21.15.143:0/1957108260 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5bd80b61e0 0x7f5bd80b2040 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:33.782 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.778+0000 7f5bdf95f700 1 --2- 172.21.15.143:0/1957108260 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5bd815a950 0x7f5bd814dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:33.782 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.778+0000 7f5bdf95f700 1 -- 172.21.15.143:0/1957108260 >> 172.21.15.143:0/1957108260 conn(0x7f5bd80cd2e0 msgr2=0x7f5bd815c160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:33.782 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.778+0000 7f5bdf95f700 1 -- 172.21.15.143:0/1957108260 shutdown_connections 2023-11-08T19:25:33.782 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.778+0000 7f5bdf95f700 1 -- 172.21.15.143:0/1957108260 wait complete. 2023-11-08T19:25:33.792 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-08T19:25:33.915 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.914+0000 7fa097319700 1 Processor -- start 2023-11-08T19:25:33.915 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.914+0000 7fa097319700 1 -- start start 2023-11-08T19:25:33.917 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.915+0000 7fa097319700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fa09015df20 0x7fa0900b25f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:33.918 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.915+0000 7fa097319700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa09006bd20 0x7fa0900d19c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:33.918 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.915+0000 7fa097319700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0900b2b30 con 0x7fa0900d1f00 2023-11-08T19:25:33.918 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.915+0000 7fa097319700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0900d2390 con 0x7fa09006bd20 2023-11-08T19:25:33.918 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.915+0000 7fa097319700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa09006c2c0 con 0x7fa09015df20 2023-11-08T19:25:33.919 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.915+0000 7fa0948b4700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa09006bd20 0x7fa0900d19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:33.919 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.915+0000 7fa0958b6700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fa0900d1f00 0x7fa0900ce2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:46218/0 (socket says 172.21.15.143:46218) 2023-11-08T19:25:33.919 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.915+0000 7fa0958b6700 1 -- 172.21.15.143:0/177650232 learned_addr learned my addr 172.21.15.143:0/177650232 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:25:33.919 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.915+0000 7fa0948b4700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa09006bd20 0x7fa0900d19c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50060/0 (socket says 172.21.15.143:50060) 2023-11-08T19:25:33.919 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.915+0000 7fa0950b5700 1 --2- 172.21.15.143:0/177650232 >> v2:172.21.15.174:3300/0 conn(0x7fa09015df20 0x7fa0900b25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:33.920 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.915+0000 7fa0948b4700 1 -- 172.21.15.143:0/177650232 >> v2:172.21.15.174:3300/0 conn(0x7fa09015df20 msgr2=0x7fa0900b25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:33.920 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.915+0000 7fa0948b4700 1 --2- 172.21.15.143:0/177650232 >> v2:172.21.15.174:3300/0 conn(0x7fa09015df20 0x7fa0900b25f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:33.920 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.915+0000 7fa0948b4700 1 -- 172.21.15.143:0/177650232 >> v1:172.21.15.143:6789/0 conn(0x7fa0900d1f00 legacy=0x7fa0900ce2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:33.920 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.915+0000 7fa0948b4700 1 -- 172.21.15.143:0/177650232 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa09006c5a0 con 0x7fa09006bd20 2023-11-08T19:25:33.920 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.915+0000 7fa0950b5700 1 --2- 172.21.15.143:0/177650232 >> v2:172.21.15.174:3300/0 conn(0x7fa09015df20 0x7fa0900b25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:25:33.921 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.915+0000 7fa0948b4700 1 --2- 172.21.15.143:0/177650232 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa09006bd20 0x7fa0900d19c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fa09009f940 tx=0x7fa08c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff582c176e109d59 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:33.921 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.915+0000 7fa087fff700 1 -- 172.21.15.143:0/177650232 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa08c011830 con 0x7fa09006bd20 2023-11-08T19:25:33.921 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.915+0000 7fa087fff700 1 -- 172.21.15.143:0/177650232 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa08c011990 con 0x7fa09006bd20 2023-11-08T19:25:33.921 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.915+0000 7fa087fff700 1 -- 172.21.15.143:0/177650232 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa08c011d20 con 0x7fa09006bd20 2023-11-08T19:25:33.921 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.916+0000 7fa097319700 1 -- 172.21.15.143:0/177650232 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa09006bd20 msgr2=0x7fa0900d19c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:33.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.916+0000 7fa097319700 1 --2- 172.21.15.143:0/177650232 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa09006bd20 0x7fa0900d19c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fa09009f940 tx=0x7fa08c009b20 comp rx=0 tx=0).stop 2023-11-08T19:25:33.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.916+0000 7fa097319700 1 -- 172.21.15.143:0/177650232 shutdown_connections 2023-11-08T19:25:33.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.916+0000 7fa097319700 1 --2- 172.21.15.143:0/177650232 >> v2:172.21.15.174:3300/0 conn(0x7fa09015df20 0x7fa0900b25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:33.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.916+0000 7fa097319700 1 --2- 172.21.15.143:0/177650232 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa09006bd20 0x7fa0900d19c0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:33.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.916+0000 7fa097319700 1 -- 172.21.15.143:0/177650232 >> 172.21.15.143:0/177650232 conn(0x7fa09008b000 msgr2=0x7fa09009ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:33.923 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.916+0000 7fa097319700 1 -- 172.21.15.143:0/177650232 shutdown_connections 2023-11-08T19:25:33.923 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.916+0000 7fa097319700 1 -- 172.21.15.143:0/177650232 wait complete. 2023-11-08T19:25:33.926 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.917+0000 7fa097319700 1 Processor -- start 2023-11-08T19:25:33.926 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.918+0000 7fa097319700 1 -- start start 2023-11-08T19:25:33.926 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.918+0000 7fa097319700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa09006bd20 0x7fa090159360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:33.926 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.918+0000 7fa097319700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa0900d1f00 0x7fa090160ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:33.927 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.918+0000 7fa097319700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa09015df20 0x7fa0900b0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:33.927 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.918+0000 7fa097319700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa09015f050 con 0x7fa09015df20 2023-11-08T19:25:33.927 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.918+0000 7fa097319700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa09015f2f0 con 0x7fa0900d1f00 2023-11-08T19:25:33.927 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.918+0000 7fa097319700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa09015f5d0 con 0x7fa09006bd20 2023-11-08T19:25:33.927 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.919+0000 7fa0958b6700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa09015df20 0x7fa0900b0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:33.928 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.919+0000 7fa0958b6700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa09015df20 0x7fa0900b0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50070/0 (socket says 172.21.15.143:50070) 2023-11-08T19:25:33.928 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.919+0000 7fa0958b6700 1 -- 172.21.15.143:0/2646406769 learned_addr learned my addr 172.21.15.143:0/2646406769 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:33.928 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.919+0000 7fa0950b5700 1 --2- 172.21.15.143:0/2646406769 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa09006bd20 0x7fa090159360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:33.928 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.919+0000 7fa0948b4700 1 --2- 172.21.15.143:0/2646406769 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa0900d1f00 0x7fa090160ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:33.928 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.919+0000 7fa0958b6700 1 -- 172.21.15.143:0/2646406769 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa09006bd20 msgr2=0x7fa090159360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:33.929 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.919+0000 7fa0958b6700 1 --2- 172.21.15.143:0/2646406769 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa09006bd20 0x7fa090159360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:33.929 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.919+0000 7fa0958b6700 1 -- 172.21.15.143:0/2646406769 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa0900d1f00 msgr2=0x7fa090160ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:33.929 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.919+0000 7fa0958b6700 1 --2- 172.21.15.143:0/2646406769 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa0900d1f00 0x7fa090160ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:33.929 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.919+0000 7fa0958b6700 1 -- 172.21.15.143:0/2646406769 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa08c0094a0 con 0x7fa09015df20 2023-11-08T19:25:33.929 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.919+0000 7fa0948b4700 1 --2- 172.21.15.143:0/2646406769 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa0900d1f00 0x7fa090160ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:25:33.930 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.919+0000 7fa0958b6700 1 --2- 172.21.15.143:0/2646406769 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa09015df20 0x7fa0900b0ab0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fa08800ca80 tx=0x7fa08800cd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:33.930 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.919+0000 7fa085ffb700 1 -- 172.21.15.143:0/2646406769 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa08800a9e0 con 0x7fa09015df20 2023-11-08T19:25:33.930 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.919+0000 7fa085ffb700 1 -- 172.21.15.143:0/2646406769 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa08800ab40 con 0x7fa09015df20 2023-11-08T19:25:33.930 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.919+0000 7fa097319700 1 -- 172.21.15.143:0/2646406769 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa09015f8b0 con 0x7fa09015df20 2023-11-08T19:25:33.930 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.919+0000 7fa085ffb700 1 -- 172.21.15.143:0/2646406769 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa088012400 con 0x7fa09015df20 2023-11-08T19:25:33.931 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.919+0000 7fa097319700 1 -- 172.21.15.143:0/2646406769 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa09015fe70 con 0x7fa09015df20 2023-11-08T19:25:33.931 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.920+0000 7fa097319700 1 -- 172.21.15.143:0/2646406769 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa09004eee0 con 0x7fa09015df20 2023-11-08T19:25:33.931 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.921+0000 7fa085ffb700 1 -- 172.21.15.143:0/2646406769 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa08801b3b0 con 0x7fa09015df20 2023-11-08T19:25:33.931 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.921+0000 7fa085ffb700 1 --2- 172.21.15.143:0/2646406769 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa07406af00 0x7fa07406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:33.931 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.921+0000 7fa085ffb700 1 -- 172.21.15.143:0/2646406769 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa088088fe0 con 0x7fa09015df20 2023-11-08T19:25:33.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.921+0000 7fa0950b5700 1 --2- 172.21.15.143:0/2646406769 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa07406af00 0x7fa07406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:33.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.922+0000 7fa0950b5700 1 --2- 172.21.15.143:0/2646406769 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa07406af00 0x7fa07406d3c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa080006fd0 tx=0x7fa080008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:33.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:33.925+0000 7fa085ffb700 1 -- 172.21.15.143:0/2646406769 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa0880541d0 con 0x7fa09015df20 2023-11-08T19:25:34.115 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.114+0000 7fa097319700 1 -- 172.21.15.143:0/2646406769 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa0900cdbd0 con 0x7fa09015df20 2023-11-08T19:25:34.116 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.114+0000 7fa085ffb700 1 -- 172.21.15.143:0/2646406769 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa0880577f0 con 0x7fa09015df20 2023-11-08T19:25:34.116 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2023-11-08T19:25:34.118 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.116+0000 7fa097319700 1 -- 172.21.15.143:0/2646406769 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa07406af00 msgr2=0x7fa07406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:34.118 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.116+0000 7fa097319700 1 --2- 172.21.15.143:0/2646406769 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa07406af00 0x7fa07406d3c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa080006fd0 tx=0x7fa080008040 comp rx=0 tx=0).stop 2023-11-08T19:25:34.118 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.116+0000 7fa097319700 1 -- 172.21.15.143:0/2646406769 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa09015df20 msgr2=0x7fa0900b0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:34.119 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.116+0000 7fa097319700 1 --2- 172.21.15.143:0/2646406769 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa09015df20 0x7fa0900b0ab0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fa08800ca80 tx=0x7fa08800cd90 comp rx=0 tx=0).stop 2023-11-08T19:25:34.119 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.116+0000 7fa097319700 1 -- 172.21.15.143:0/2646406769 shutdown_connections 2023-11-08T19:25:34.119 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.116+0000 7fa097319700 1 --2- 172.21.15.143:0/2646406769 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa07406af00 0x7fa07406d3c0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:34.119 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.116+0000 7fa097319700 1 --2- 172.21.15.143:0/2646406769 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa09006bd20 0x7fa090159360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:34.120 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.116+0000 7fa097319700 1 --2- 172.21.15.143:0/2646406769 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa09015df20 0x7fa0900b0ab0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:34.120 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.116+0000 7fa097319700 1 --2- 172.21.15.143:0/2646406769 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa0900d1f00 0x7fa090160ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:34.120 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.116+0000 7fa097319700 1 -- 172.21.15.143:0/2646406769 >> 172.21.15.143:0/2646406769 conn(0x7fa09008b000 msgr2=0x7fa0900b6660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:34.120 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.116+0000 7fa097319700 1 -- 172.21.15.143:0/2646406769 shutdown_connections 2023-11-08T19:25:34.120 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.116+0000 7fa097319700 1 -- 172.21.15.143:0/2646406769 wait complete. 2023-11-08T19:25:34.130 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-08T19:25:34.253 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.252+0000 7f54d5acc700 1 Processor -- start 2023-11-08T19:25:34.253 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.252+0000 7f54d5acc700 1 -- start start 2023-11-08T19:25:34.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.252+0000 7f54d5acc700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f54d006b240 0x7f54d00caff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:34.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.252+0000 7f54d5acc700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f54d00cb530 0x7f54d00b22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:34.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.252+0000 7f54d5acc700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54d0161890 con 0x7f54d015dba0 2023-11-08T19:25:34.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.252+0000 7f54d5acc700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54d006cd20 con 0x7f54d00cb530 2023-11-08T19:25:34.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.252+0000 7f54d5acc700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54d006b740 con 0x7f54d006b240 2023-11-08T19:25:34.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.253+0000 7f54cffff700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f54d00cb530 0x7f54d00b22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:34.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.253+0000 7f54cf7fe700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f54d015dba0 0x7f54d006c610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:46228/0 (socket says 172.21.15.143:46228) 2023-11-08T19:25:34.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.253+0000 7f54cf7fe700 1 -- 172.21.15.143:0/1299081637 learned_addr learned my addr 172.21.15.143:0/1299081637 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:25:34.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.253+0000 7f54cffff700 1 --2- 172.21.15.143:0/1299081637 >> v2:172.21.15.143:3300/0 conn(0x7f54d00cb530 0x7f54d00b22e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50078/0 (socket says 172.21.15.143:50078) 2023-11-08T19:25:34.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.253+0000 7f54ceffd700 1 --2- 172.21.15.143:0/1299081637 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f54d006b240 0x7f54d00caff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:34.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.253+0000 7f54cffff700 1 -- 172.21.15.143:0/1299081637 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f54d006b240 msgr2=0x7f54d00caff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:34.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.253+0000 7f54cffff700 1 --2- 172.21.15.143:0/1299081637 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f54d006b240 0x7f54d00caff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:34.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.253+0000 7f54cffff700 1 -- 172.21.15.143:0/1299081637 >> v1:172.21.15.143:6789/0 conn(0x7f54d015dba0 legacy=0x7f54d006c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:34.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.253+0000 7f54cffff700 1 -- 172.21.15.143:0/1299081637 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54d006ba20 con 0x7f54d00cb530 2023-11-08T19:25:34.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.253+0000 7f54ce7fc700 1 -- 172.21.15.143:0/1299081637 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2712781636 0 0) 0x7f54d0161890 con 0x7f54d015dba0 2023-11-08T19:25:34.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.253+0000 7f54ceffd700 1 --2- 172.21.15.143:0/1299081637 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f54d006b240 0x7f54d00caff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:25:34.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.253+0000 7f54cffff700 1 --2- 172.21.15.143:0/1299081637 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f54d00cb530 0x7f54d00b22e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f54c8009d90 tx=0x7f54c800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77c4d3d9d74f8e1d server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:34.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.253+0000 7f54ce7fc700 1 -- 172.21.15.143:0/1299081637 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54c800e070 con 0x7f54d00cb530 2023-11-08T19:25:34.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.253+0000 7f54ce7fc700 1 -- 172.21.15.143:0/1299081637 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54c800bbd0 con 0x7f54d00cb530 2023-11-08T19:25:34.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.253+0000 7f54ce7fc700 1 -- 172.21.15.143:0/1299081637 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54c8012430 con 0x7f54d00cb530 2023-11-08T19:25:34.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.254+0000 7f54d5acc700 1 -- 172.21.15.143:0/1299081637 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f54d00cb530 msgr2=0x7f54d00b22e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:34.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.254+0000 7f54d5acc700 1 --2- 172.21.15.143:0/1299081637 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f54d00cb530 0x7f54d00b22e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f54c8009d90 tx=0x7f54c800b330 comp rx=0 tx=0).stop 2023-11-08T19:25:34.261 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.254+0000 7f54d5acc700 1 -- 172.21.15.143:0/1299081637 shutdown_connections 2023-11-08T19:25:34.261 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.254+0000 7f54d5acc700 1 --2- 172.21.15.143:0/1299081637 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f54d00cb530 0x7f54d00b22e0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:34.261 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.254+0000 7f54d5acc700 1 --2- 172.21.15.143:0/1299081637 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f54d006b240 0x7f54d00caff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:34.261 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.254+0000 7f54d5acc700 1 -- 172.21.15.143:0/1299081637 >> 172.21.15.143:0/1299081637 conn(0x7f54d00cd2e0 msgr2=0x7f54d00cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:34.261 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.254+0000 7f54d5acc700 1 -- 172.21.15.143:0/1299081637 shutdown_connections 2023-11-08T19:25:34.261 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.254+0000 7f54d5acc700 1 -- 172.21.15.143:0/1299081637 wait complete. 2023-11-08T19:25:34.264 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.255+0000 7f54d5acc700 1 Processor -- start 2023-11-08T19:25:34.265 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.256+0000 7f54d5acc700 1 -- start start 2023-11-08T19:25:34.265 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.256+0000 7f54d5acc700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f54d006b240 0x7f54d00ca0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:34.265 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.256+0000 7f54d5acc700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f54d00cb530 0x7f54d0149210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:34.265 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.256+0000 7f54d5acc700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f54d015dba0 0x7f54d00c7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:34.265 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.256+0000 7f54d5acc700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54d0149750 con 0x7f54d015dba0 2023-11-08T19:25:34.266 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.256+0000 7f54d5acc700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54d01e7a80 con 0x7f54d00cb530 2023-11-08T19:25:34.266 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.256+0000 7f54d5acc700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54d01e8320 con 0x7f54d006b240 2023-11-08T19:25:34.266 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.257+0000 7f54cffff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f54d015dba0 0x7f54d00c7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:34.266 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.257+0000 7f54cffff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f54d015dba0 0x7f54d00c7bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50084/0 (socket says 172.21.15.143:50084) 2023-11-08T19:25:34.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.257+0000 7f54cffff700 1 -- 172.21.15.143:0/2897823598 learned_addr learned my addr 172.21.15.143:0/2897823598 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:34.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.257+0000 7f54ceffd700 1 --2- 172.21.15.143:0/2897823598 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f54d00cb530 0x7f54d0149210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:34.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.257+0000 7f54cf7fe700 1 --2- 172.21.15.143:0/2897823598 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f54d006b240 0x7f54d00ca0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:34.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.257+0000 7f54cffff700 1 -- 172.21.15.143:0/2897823598 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f54d006b240 msgr2=0x7f54d00ca0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:34.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.257+0000 7f54cffff700 1 --2- 172.21.15.143:0/2897823598 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f54d006b240 0x7f54d00ca0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:34.268 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.257+0000 7f54cffff700 1 -- 172.21.15.143:0/2897823598 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f54d00cb530 msgr2=0x7f54d0149210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:34.268 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.257+0000 7f54cffff700 1 --2- 172.21.15.143:0/2897823598 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f54d00cb530 0x7f54d0149210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:34.268 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.257+0000 7f54cffff700 1 -- 172.21.15.143:0/2897823598 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54c8009a70 con 0x7f54d015dba0 2023-11-08T19:25:34.268 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.257+0000 7f54ceffd700 1 --2- 172.21.15.143:0/2897823598 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f54d00cb530 0x7f54d0149210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:25:34.268 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.257+0000 7f54cf7fe700 1 --2- 172.21.15.143:0/2897823598 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f54d006b240 0x7f54d00ca0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:25:34.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.257+0000 7f54cffff700 1 --2- 172.21.15.143:0/2897823598 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f54d015dba0 0x7f54d00c7bd0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f54c8013000 tx=0x7f54c8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:34.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.257+0000 7f54b7fff700 1 -- 172.21.15.143:0/2897823598 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54c8012430 con 0x7f54d015dba0 2023-11-08T19:25:34.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.257+0000 7f54b7fff700 1 -- 172.21.15.143:0/2897823598 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54c8020070 con 0x7f54d015dba0 2023-11-08T19:25:34.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.257+0000 7f54b7fff700 1 -- 172.21.15.143:0/2897823598 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54c8004280 con 0x7f54d015dba0 2023-11-08T19:25:34.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.257+0000 7f54d5acc700 1 -- 172.21.15.143:0/2897823598 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54d00c8110 con 0x7f54d015dba0 2023-11-08T19:25:34.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.257+0000 7f54d5acc700 1 -- 172.21.15.143:0/2897823598 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54d01e7cd0 con 0x7f54d015dba0 2023-11-08T19:25:34.270 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.258+0000 7f54d5acc700 1 -- 172.21.15.143:0/2897823598 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54d01e8490 con 0x7f54d015dba0 2023-11-08T19:25:34.270 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.258+0000 7f54b7fff700 1 -- 172.21.15.143:0/2897823598 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f54c80043e0 con 0x7f54d015dba0 2023-11-08T19:25:34.270 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.259+0000 7f54b7fff700 1 --2- 172.21.15.143:0/2897823598 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f54bc06af00 0x7f54bc06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:34.270 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.259+0000 7f54b7fff700 1 -- 172.21.15.143:0/2897823598 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f54c8022070 con 0x7f54d015dba0 2023-11-08T19:25:34.270 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.259+0000 7f54cf7fe700 1 --2- 172.21.15.143:0/2897823598 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f54bc06af00 0x7f54bc06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:34.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.259+0000 7f54cf7fe700 1 --2- 172.21.15.143:0/2897823598 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f54bc06af00 0x7f54bc06d3c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f54c0007fd0 tx=0x7f54c0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:34.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.263+0000 7f54b7fff700 1 -- 172.21.15.143:0/2897823598 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f54c80568b0 con 0x7f54d015dba0 2023-11-08T19:25:34.452 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.450+0000 7f54d5acc700 1 -- 172.21.15.143:0/2897823598 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f54d004eee0 con 0x7f54d015dba0 2023-11-08T19:25:34.453 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.451+0000 7f54b7fff700 1 -- 172.21.15.143:0/2897823598 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f54c802c020 con 0x7f54d015dba0 2023-11-08T19:25:34.453 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 64 2023-11-08T19:25:34.455 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.453+0000 7f54d5acc700 1 -- 172.21.15.143:0/2897823598 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f54bc06af00 msgr2=0x7f54bc06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:34.455 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.453+0000 7f54d5acc700 1 --2- 172.21.15.143:0/2897823598 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f54bc06af00 0x7f54bc06d3c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f54c0007fd0 tx=0x7f54c0009040 comp rx=0 tx=0).stop 2023-11-08T19:25:34.455 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.453+0000 7f54d5acc700 1 -- 172.21.15.143:0/2897823598 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f54d015dba0 msgr2=0x7f54d00c7bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:34.455 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.453+0000 7f54d5acc700 1 --2- 172.21.15.143:0/2897823598 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f54d015dba0 0x7f54d00c7bd0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f54c8013000 tx=0x7f54c8004ad0 comp rx=0 tx=0).stop 2023-11-08T19:25:34.456 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.453+0000 7f54d5acc700 1 -- 172.21.15.143:0/2897823598 shutdown_connections 2023-11-08T19:25:34.456 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.453+0000 7f54d5acc700 1 --2- 172.21.15.143:0/2897823598 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f54bc06af00 0x7f54bc06d3c0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:34.456 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.453+0000 7f54d5acc700 1 --2- 172.21.15.143:0/2897823598 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f54d006b240 0x7f54d00ca0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:34.457 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.453+0000 7f54d5acc700 1 --2- 172.21.15.143:0/2897823598 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f54d015dba0 0x7f54d00c7bd0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:34.457 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.453+0000 7f54d5acc700 1 --2- 172.21.15.143:0/2897823598 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f54d00cb530 0x7f54d0149210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:34.457 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.453+0000 7f54d5acc700 1 -- 172.21.15.143:0/2897823598 >> 172.21.15.143:0/2897823598 conn(0x7f54d00cd2e0 msgr2=0x7f54d00723a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:34.457 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.453+0000 7f54d5acc700 1 -- 172.21.15.143:0/2897823598 shutdown_connections 2023-11-08T19:25:34.458 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.453+0000 7f54d5acc700 1 -- 172.21.15.143:0/2897823598 wait complete. 2023-11-08T19:25:34.467 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-08T19:25:34.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.588+0000 7fc102c95700 1 Processor -- start 2023-11-08T19:25:34.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.588+0000 7fc102c95700 1 -- start start 2023-11-08T19:25:34.593 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.588+0000 7fc102c95700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc0fc06b320 0x7fc0fc168c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:34.593 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.588+0000 7fc102c95700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0fc161df0 con 0x7fc0fc15dd00 2023-11-08T19:25:34.594 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.588+0000 7fc102c95700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0fc1620d0 con 0x7fc0fc1618f0 2023-11-08T19:25:34.594 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.588+0000 7fc102c95700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0fc0cad50 con 0x7fc0fc06b320 2023-11-08T19:25:34.594 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.589+0000 7fc100a31700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fc0fc15dd00 0x7fc0fc0cca70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:46242/0 (socket says 172.21.15.143:46242) 2023-11-08T19:25:34.594 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.589+0000 7fc100a31700 1 -- 172.21.15.143:0/17802297 learned_addr learned my addr 172.21.15.143:0/17802297 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:25:34.594 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.589+0000 7fc101232700 1 --2- 172.21.15.143:0/17802297 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc0fc06b320 0x7fc0fc168c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:34.595 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.589+0000 7fc0fb7fe700 1 -- 172.21.15.143:0/17802297 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2372815818 0 0) 0x7fc0fc161df0 con 0x7fc0fc15dd00 2023-11-08T19:25:34.595 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.589+0000 7fc0fb7fe700 1 -- 172.21.15.143:0/17802297 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc0ec0035b0 con 0x7fc0fc15dd00 2023-11-08T19:25:34.595 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.589+0000 7fc0fb7fe700 1 -- 172.21.15.143:0/17802297 <== mon.1 v1:172.21.15.174:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2901115063 0 0) 0x7fc0fc1620d0 con 0x7fc0fc1618f0 2023-11-08T19:25:34.595 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.589+0000 7fc0fb7fe700 1 -- 172.21.15.143:0/17802297 --> v1:172.21.15.174:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc0fc161df0 con 0x7fc0fc1618f0 2023-11-08T19:25:34.596 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.589+0000 7fc0fb7fe700 1 -- 172.21.15.143:0/17802297 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 527964519 0 0) 0x7fc0ec0035b0 con 0x7fc0fc15dd00 2023-11-08T19:25:34.596 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.589+0000 7fc0fb7fe700 1 -- 172.21.15.143:0/17802297 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc0fc06b320 msgr2=0x7fc0fc168c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:34.596 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.589+0000 7fc0fb7fe700 1 --2- 172.21.15.143:0/17802297 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc0fc06b320 0x7fc0fc168c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:34.596 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.589+0000 7fc0fb7fe700 1 -- 172.21.15.143:0/17802297 >> v1:172.21.15.174:6789/0 conn(0x7fc0fc1618f0 legacy=0x7fc0fc06ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:34.596 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.589+0000 7fc0fb7fe700 1 -- 172.21.15.143:0/17802297 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0fc1623b0 con 0x7fc0fc15dd00 2023-11-08T19:25:34.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.589+0000 7fc0fb7fe700 1 -- 172.21.15.143:0/17802297 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4147756303 0 0) 0x7fc0f00017f0 con 0x7fc0fc15dd00 2023-11-08T19:25:34.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.589+0000 7fc0fb7fe700 1 -- 172.21.15.143:0/17802297 >> v1:172.21.15.143:6789/0 conn(0x7fc0fc15dd00 legacy=0x7fc0fc0cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:34.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.589+0000 7fc0fb7fe700 1 --2- 172.21.15.143:0/17802297 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc0fc06b320 0x7fc0ec003bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:34.598 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.589+0000 7fc0fb7fe700 1 --2- 172.21.15.143:0/17802297 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc0ec004170 0x7fc0ec004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:34.598 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.589+0000 7fc0fb7fe700 1 --2- 172.21.15.143:0/17802297 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc0ec004d90 0x7fc0ec005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:34.598 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.589+0000 7fc0fb7fe700 1 -- 172.21.15.143:0/17802297 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0ec00bb20 con 0x7fc0ec004d90 2023-11-08T19:25:34.598 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.589+0000 7fc0fb7fe700 1 -- 172.21.15.143:0/17802297 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0ec00bc90 con 0x7fc0ec004170 2023-11-08T19:25:34.598 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.589+0000 7fc0fb7fe700 1 -- 172.21.15.143:0/17802297 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0ec00be50 con 0x7fc0fc06b320 2023-11-08T19:25:34.599 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.589+0000 7fc101232700 1 --2- 172.21.15.143:0/17802297 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc0ec004d90 0x7fc0ec005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:34.599 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.589+0000 7fc100a31700 1 --2- 172.21.15.143:0/17802297 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc0fc06b320 0x7fc0ec003bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:34.599 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.589+0000 7fc0fbfff700 1 --2- 172.21.15.143:0/17802297 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc0ec004170 0x7fc0ec004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:34.599 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.590+0000 7fc101232700 1 -- 172.21.15.143:0/17802297 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc0fc06b320 msgr2=0x7fc0ec003bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:34.600 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.590+0000 7fc101232700 1 --2- 172.21.15.143:0/17802297 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc0fc06b320 0x7fc0ec003bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:34.600 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.590+0000 7fc101232700 1 -- 172.21.15.143:0/17802297 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc0ec004170 msgr2=0x7fc0ec004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:34.600 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.590+0000 7fc101232700 1 --2- 172.21.15.143:0/17802297 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc0ec004170 0x7fc0ec004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:34.600 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.590+0000 7fc101232700 1 -- 172.21.15.143:0/17802297 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0ec0035b0 con 0x7fc0ec004d90 2023-11-08T19:25:34.600 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.590+0000 7fc101232700 1 --2- 172.21.15.143:0/17802297 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc0ec004d90 0x7fc0ec005210 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fc0f4009fc0 tx=0x7fc0ec005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c4e83647bef194c server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:34.601 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.590+0000 7fc0fb7fe700 1 -- 172.21.15.143:0/17802297 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0f400ec30 con 0x7fc0ec004d90 2023-11-08T19:25:34.601 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.590+0000 7fc0fb7fe700 1 -- 172.21.15.143:0/17802297 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0f400f040 con 0x7fc0ec004d90 2023-11-08T19:25:34.601 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.590+0000 7fc0fb7fe700 1 -- 172.21.15.143:0/17802297 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0f4014480 con 0x7fc0ec004d90 2023-11-08T19:25:34.601 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.590+0000 7fc102c95700 1 -- 172.21.15.143:0/17802297 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc0ec004d90 msgr2=0x7fc0ec005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:34.601 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.590+0000 7fc102c95700 1 --2- 172.21.15.143:0/17802297 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc0ec004d90 0x7fc0ec005210 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fc0f4009fc0 tx=0x7fc0ec005880 comp rx=0 tx=0).stop 2023-11-08T19:25:34.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.590+0000 7fc101232700 1 -- 172.21.15.143:0/17802297 reap_dead start 2023-11-08T19:25:34.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.590+0000 7fc102c95700 1 -- 172.21.15.143:0/17802297 shutdown_connections 2023-11-08T19:25:34.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.590+0000 7fc102c95700 1 -- 172.21.15.143:0/17802297 >> 172.21.15.143:0/17802297 conn(0x7fc0fc099ab0 msgr2=0x7fc0fc0d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:34.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.590+0000 7fc102c95700 1 -- 172.21.15.143:0/17802297 shutdown_connections 2023-11-08T19:25:34.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.590+0000 7fc102c95700 1 -- 172.21.15.143:0/17802297 wait complete. 2023-11-08T19:25:34.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.592+0000 7fc102c95700 1 Processor -- start 2023-11-08T19:25:34.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.592+0000 7fc102c95700 1 -- start start 2023-11-08T19:25:34.606 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.592+0000 7fc102c95700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc0fc14a9a0 0x7fc0fc0b5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:34.606 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.592+0000 7fc102c95700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc0fc0b1d70 0x7fc0fc06ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:34.606 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.592+0000 7fc102c95700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc0fc14ad80 0x7fc0fc06f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:34.606 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.592+0000 7fc102c95700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0fc1550c0 con 0x7fc0fc14ad80 2023-11-08T19:25:34.607 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.592+0000 7fc102c95700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0fc0b6200 con 0x7fc0fc14a9a0 2023-11-08T19:25:34.607 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.592+0000 7fc102c95700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0fc0b2830 con 0x7fc0fc0b1d70 2023-11-08T19:25:34.607 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.593+0000 7fc101232700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc0fc14ad80 0x7fc0fc06f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:34.607 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.593+0000 7fc101232700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc0fc14ad80 0x7fc0fc06f200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50104/0 (socket says 172.21.15.143:50104) 2023-11-08T19:25:34.608 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.593+0000 7fc101232700 1 -- 172.21.15.143:0/3971311522 learned_addr learned my addr 172.21.15.143:0/3971311522 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:34.608 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.593+0000 7fc101232700 1 -- 172.21.15.143:0/3971311522 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc0fc0b1d70 msgr2=0x7fc0fc06ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:34.608 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.593+0000 7fc0fbfff700 1 --2- 172.21.15.143:0/3971311522 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc0fc0b1d70 0x7fc0fc06ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:34.608 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.593+0000 7fc100a31700 1 --2- 172.21.15.143:0/3971311522 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc0fc14a9a0 0x7fc0fc0b5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:34.609 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.593+0000 7fc101232700 1 --2- 172.21.15.143:0/3971311522 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc0fc0b1d70 0x7fc0fc06ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:34.609 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.593+0000 7fc101232700 1 -- 172.21.15.143:0/3971311522 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc0fc14a9a0 msgr2=0x7fc0fc0b5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:34.609 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.593+0000 7fc101232700 1 --2- 172.21.15.143:0/3971311522 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc0fc14a9a0 0x7fc0fc0b5cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:34.609 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.593+0000 7fc101232700 1 -- 172.21.15.143:0/3971311522 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0fc15e260 con 0x7fc0fc14ad80 2023-11-08T19:25:34.609 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.593+0000 7fc101232700 1 --2- 172.21.15.143:0/3971311522 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc0fc14ad80 0x7fc0fc06f200 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fc0f4016000 tx=0x7fc0f4009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:34.610 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.593+0000 7fc0f97fa700 1 -- 172.21.15.143:0/3971311522 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0f4007000 con 0x7fc0fc14ad80 2023-11-08T19:25:34.610 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.593+0000 7fc0f97fa700 1 -- 172.21.15.143:0/3971311522 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0f4022070 con 0x7fc0fc14ad80 2023-11-08T19:25:34.610 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.593+0000 7fc0f97fa700 1 -- 172.21.15.143:0/3971311522 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0f4029480 con 0x7fc0fc14ad80 2023-11-08T19:25:34.610 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.593+0000 7fc102c95700 1 -- 172.21.15.143:0/3971311522 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0fc0b2270 con 0x7fc0fc14ad80 2023-11-08T19:25:34.611 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.593+0000 7fc102c95700 1 -- 172.21.15.143:0/3971311522 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0fc0c42d0 con 0x7fc0fc14ad80 2023-11-08T19:25:34.611 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.594+0000 7fc102c95700 1 -- 172.21.15.143:0/3971311522 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0fc04eee0 con 0x7fc0fc14ad80 2023-11-08T19:25:34.611 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.595+0000 7fc0f97fa700 1 -- 172.21.15.143:0/3971311522 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc0f401e430 con 0x7fc0fc14ad80 2023-11-08T19:25:34.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.595+0000 7fc0f97fa700 1 --2- 172.21.15.143:0/3971311522 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc0e006af00 0x7fc0e006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:34.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.595+0000 7fc0f97fa700 1 -- 172.21.15.143:0/3971311522 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc0f4024070 con 0x7fc0fc14ad80 2023-11-08T19:25:34.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.595+0000 7fc100a31700 1 --2- 172.21.15.143:0/3971311522 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc0e006af00 0x7fc0e006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:34.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.596+0000 7fc100a31700 1 --2- 172.21.15.143:0/3971311522 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc0e006af00 0x7fc0e006d3c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc0f00011d0 tx=0x7fc0f0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:34.613 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.599+0000 7fc0f97fa700 1 -- 172.21.15.143:0/3971311522 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc0f40562c0 con 0x7fc0fc14ad80 2023-11-08T19:25:34.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.804+0000 7fc102c95700 1 -- 172.21.15.143:0/3971311522 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fc0fc06b920 con 0x7fc0fc14ad80 2023-11-08T19:25:34.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.804+0000 7fc0f97fa700 1 -- 172.21.15.143:0/3971311522 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) v1 ==== 80+0+0 (secure 0 0 0) 0x7fc0f40598e0 con 0x7fc0fc14ad80 2023-11-08T19:25:34.808 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.806+0000 7fc102c95700 1 -- 172.21.15.143:0/3971311522 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc0e006af00 msgr2=0x7fc0e006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:34.808 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.806+0000 7fc102c95700 1 --2- 172.21.15.143:0/3971311522 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc0e006af00 0x7fc0e006d3c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc0f00011d0 tx=0x7fc0f0009040 comp rx=0 tx=0).stop 2023-11-08T19:25:34.809 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.806+0000 7fc102c95700 1 -- 172.21.15.143:0/3971311522 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc0fc14ad80 msgr2=0x7fc0fc06f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:34.809 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.806+0000 7fc102c95700 1 --2- 172.21.15.143:0/3971311522 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc0fc14ad80 0x7fc0fc06f200 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fc0f4016000 tx=0x7fc0f4009750 comp rx=0 tx=0).stop 2023-11-08T19:25:34.809 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.806+0000 7fc102c95700 1 -- 172.21.15.143:0/3971311522 shutdown_connections 2023-11-08T19:25:34.809 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.806+0000 7fc102c95700 1 --2- 172.21.15.143:0/3971311522 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc0e006af00 0x7fc0e006d3c0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:34.810 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.806+0000 7fc102c95700 1 --2- 172.21.15.143:0/3971311522 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc0fc14ad80 0x7fc0fc06f200 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:34.810 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.806+0000 7fc102c95700 1 --2- 172.21.15.143:0/3971311522 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc0fc14a9a0 0x7fc0fc0b5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:34.810 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.806+0000 7fc102c95700 1 --2- 172.21.15.143:0/3971311522 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc0fc0b1d70 0x7fc0fc06ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:34.811 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.806+0000 7fc102c95700 1 -- 172.21.15.143:0/3971311522 >> 172.21.15.143:0/3971311522 conn(0x7fc0fc099ab0 msgr2=0x7fc0fc15d030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:34.811 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.806+0000 7fc102c95700 1 -- 172.21.15.143:0/3971311522 shutdown_connections 2023-11-08T19:25:34.811 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:34.806+0000 7fc102c95700 1 -- 172.21.15.143:0/3971311522 wait complete. 2023-11-08T19:25:34.812 INFO:teuthology.orchestra.run.smithi143.stderr:listed 0 entries 2023-11-08T19:25:34.820 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2023-11-08T19:25:34.820 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:34.821 DEBUG:teuthology.orchestra.run.smithi174:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2023-11-08T19:25:34.867 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi174.front.sepia.ceph.com 2023-11-08T19:25:34.868 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:34.868 DEBUG:teuthology.orchestra.run.smithi174:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-11-08T19:25:34.888 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-08T19:25:34.888 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:25:35.008 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.007+0000 7f2e8ca96700 1 Processor -- start 2023-11-08T19:25:35.009 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.007+0000 7f2e8ca96700 1 -- start start 2023-11-08T19:25:35.011 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.008+0000 7f2e8ca96700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2e88159870 0x7f2e880b2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:35.011 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.008+0000 7f2e8ca96700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f2e880cc410 0x7f2e880cc820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:35.012 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.008+0000 7f2e8ca96700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e880b2a60 con 0x7f2e8806ed60 2023-11-08T19:25:35.012 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.008+0000 7f2e8ca96700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e8806b090 con 0x7f2e880cc410 2023-11-08T19:25:35.012 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.008+0000 7f2e8ca96700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e8806bbc0 con 0x7f2e88159870 2023-11-08T19:25:35.012 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.008+0000 7f2e8659c700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2e88159870 0x7f2e880b2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:35.012 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.008+0000 7f2e86d9d700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f2e880cc410 0x7f2e880cc820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:35.013 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.008+0000 7f2e85d9b700 1 --1- >> v1:172.21.15.174:6789/0 conn(0x7f2e8806ed60 0x7f2e8806b4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.174:6789/0 says I am v1:172.21.15.143:57196/0 (socket says 172.21.15.143:57196) 2023-11-08T19:25:35.013 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.008+0000 7f2e85d9b700 1 -- 172.21.15.143:0/1013142521 learned_addr learned my addr 172.21.15.143:0/1013142521 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:25:35.013 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.009+0000 7f2e8559a700 1 -- 172.21.15.143:0/1013142521 <== mon.1 v1:172.21.15.174:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1676949922 0 0) 0x7f2e880b2a60 con 0x7f2e8806ed60 2023-11-08T19:25:35.013 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.009+0000 7f2e8559a700 1 -- 172.21.15.143:0/1013142521 --> v1:172.21.15.174:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2e700035b0 con 0x7f2e8806ed60 2023-11-08T19:25:35.014 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.009+0000 7f2e86d9d700 1 -- 172.21.15.143:0/1013142521 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2e88159870 msgr2=0x7f2e880b2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:35.014 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.009+0000 7f2e86d9d700 1 --2- 172.21.15.143:0/1013142521 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2e88159870 0x7f2e880b2210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:35.014 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.009+0000 7f2e86d9d700 1 -- 172.21.15.143:0/1013142521 >> v1:172.21.15.174:6789/0 conn(0x7f2e8806ed60 legacy=0x7f2e8806b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:35.014 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.009+0000 7f2e86d9d700 1 -- 172.21.15.143:0/1013142521 --> v2:172.21.15.174:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e880b2d40 con 0x7f2e880cc410 2023-11-08T19:25:35.014 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.009+0000 7f2e8659c700 1 --2- 172.21.15.143:0/1013142521 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2e88159870 0x7f2e880b2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T19:25:35.015 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.009+0000 7f2e86d9d700 1 --2- 172.21.15.143:0/1013142521 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2e880cc410 0x7f2e880cc820 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2e80009d90 tx=0x7f2e8000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d632a3bd5470b97b server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:35.015 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.009+0000 7f2e8559a700 1 -- 172.21.15.143:0/1013142521 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e8000e070 con 0x7f2e880cc410 2023-11-08T19:25:35.015 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.009+0000 7f2e8559a700 1 -- 172.21.15.143:0/1013142521 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e8000bb20 con 0x7f2e880cc410 2023-11-08T19:25:35.015 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.009+0000 7f2e8559a700 1 -- 172.21.15.143:0/1013142521 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e80012430 con 0x7f2e880cc410 2023-11-08T19:25:35.015 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.010+0000 7f2e8ca96700 1 -- 172.21.15.143:0/1013142521 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2e880cc410 msgr2=0x7f2e880cc820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:35.016 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.010+0000 7f2e8ca96700 1 --2- 172.21.15.143:0/1013142521 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2e880cc410 0x7f2e880cc820 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2e80009d90 tx=0x7f2e8000b330 comp rx=0 tx=0).stop 2023-11-08T19:25:35.016 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.010+0000 7f2e8ca96700 1 -- 172.21.15.143:0/1013142521 shutdown_connections 2023-11-08T19:25:35.016 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.010+0000 7f2e8ca96700 1 --2- 172.21.15.143:0/1013142521 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2e880cc410 0x7f2e880cc820 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:35.016 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.010+0000 7f2e8ca96700 1 --2- 172.21.15.143:0/1013142521 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2e88159870 0x7f2e880b2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:35.016 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.010+0000 7f2e8ca96700 1 -- 172.21.15.143:0/1013142521 >> 172.21.15.143:0/1013142521 conn(0x7f2e88099ab0 msgr2=0x7f2e880d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:35.016 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.010+0000 7f2e8ca96700 1 -- 172.21.15.143:0/1013142521 shutdown_connections 2023-11-08T19:25:35.017 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.010+0000 7f2e8ca96700 1 -- 172.21.15.143:0/1013142521 wait complete. 2023-11-08T19:25:35.019 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.011+0000 7f2e8ca96700 1 Processor -- start 2023-11-08T19:25:35.019 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.011+0000 7f2e8ca96700 1 -- start start 2023-11-08T19:25:35.020 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.012+0000 7f2e8ca96700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2e8806ed60 0x7f2e881f09d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:35.020 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.012+0000 7f2e8ca96700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2e880cc410 0x7f2e880b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:35.020 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.012+0000 7f2e8ca96700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2e88159870 0x7f2e880b17b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:35.020 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.012+0000 7f2e8ca96700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e880b54c0 con 0x7f2e88159870 2023-11-08T19:25:35.020 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.012+0000 7f2e8ca96700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e880b5df0 con 0x7f2e880cc410 2023-11-08T19:25:35.021 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.012+0000 7f2e8ca96700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e881f3fd0 con 0x7f2e8806ed60 2023-11-08T19:25:35.021 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.012+0000 7f2e86d9d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2e88159870 0x7f2e880b17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:35.021 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.012+0000 7f2e86d9d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2e88159870 0x7f2e880b17b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50116/0 (socket says 172.21.15.143:50116) 2023-11-08T19:25:35.021 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.012+0000 7f2e86d9d700 1 -- 172.21.15.143:0/3121668131 learned_addr learned my addr 172.21.15.143:0/3121668131 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:35.021 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.012+0000 7f2e8659c700 1 --2- 172.21.15.143:0/3121668131 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2e8806ed60 0x7f2e881f09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:35.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.012+0000 7f2e85d9b700 1 --2- 172.21.15.143:0/3121668131 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2e880cc410 0x7f2e880b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:35.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.013+0000 7f2e86d9d700 1 -- 172.21.15.143:0/3121668131 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2e8806ed60 msgr2=0x7f2e881f09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:35.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.013+0000 7f2e86d9d700 1 --2- 172.21.15.143:0/3121668131 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2e8806ed60 0x7f2e881f09d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:35.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.013+0000 7f2e86d9d700 1 -- 172.21.15.143:0/3121668131 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2e880cc410 msgr2=0x7f2e880b4f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:35.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.013+0000 7f2e86d9d700 1 --2- 172.21.15.143:0/3121668131 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2e880cc410 0x7f2e880b4f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:35.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.013+0000 7f2e86d9d700 1 -- 172.21.15.143:0/3121668131 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e7c009430 con 0x7f2e88159870 2023-11-08T19:25:35.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.013+0000 7f2e86d9d700 1 --2- 172.21.15.143:0/3121668131 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2e88159870 0x7f2e880b17b0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f2e80014040 tx=0x7f2e80004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:35.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.013+0000 7f2e76ffd700 1 -- 172.21.15.143:0/3121668131 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e80012a90 con 0x7f2e88159870 2023-11-08T19:25:35.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.013+0000 7f2e76ffd700 1 -- 172.21.15.143:0/3121668131 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e80020070 con 0x7f2e88159870 2023-11-08T19:25:35.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.013+0000 7f2e8ca96700 1 -- 172.21.15.143:0/3121668131 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e80009a70 con 0x7f2e88159870 2023-11-08T19:25:35.024 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.013+0000 7f2e76ffd700 1 -- 172.21.15.143:0/3121668131 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e80012e50 con 0x7f2e88159870 2023-11-08T19:25:35.024 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.013+0000 7f2e8ca96700 1 -- 172.21.15.143:0/3121668131 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e880b44a0 con 0x7f2e88159870 2023-11-08T19:25:35.024 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.014+0000 7f2e8ca96700 1 -- 172.21.15.143:0/3121668131 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e8804eee0 con 0x7f2e88159870 2023-11-08T19:25:35.024 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.014+0000 7f2e76ffd700 1 -- 172.21.15.143:0/3121668131 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2e80004040 con 0x7f2e88159870 2023-11-08T19:25:35.024 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.015+0000 7f2e76ffd700 1 --2- 172.21.15.143:0/3121668131 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f2e6806af00 0x7f2e6806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:35.024 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.015+0000 7f2e76ffd700 1 -- 172.21.15.143:0/3121668131 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2e80022070 con 0x7f2e88159870 2023-11-08T19:25:35.025 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.015+0000 7f2e8659c700 1 --2- 172.21.15.143:0/3121668131 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f2e6806af00 0x7f2e6806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:35.025 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.015+0000 7f2e8659c700 1 --2- 172.21.15.143:0/3121668131 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f2e6806af00 0x7f2e6806d3c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f2e880cbc30 tx=0x7f2e7c0091c0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:35.025 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.019+0000 7f2e76ffd700 1 -- 172.21.15.143:0/3121668131 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2e800569f0 con 0x7f2e88159870 2023-11-08T19:25:35.249 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.247+0000 7f2e8ca96700 1 -- 172.21.15.143:0/3121668131 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2e880af390 con 0x7f2e88159870 2023-11-08T19:25:35.250 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.248+0000 7f2e76ffd700 1 -- 172.21.15.143:0/3121668131 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4913 (secure 0 0 0) 0x7f2e8005a010 con 0x7f2e88159870 2023-11-08T19:25:35.250 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:25:35.250 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4208,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6834/37280614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":37280614},{"type":"v1","addr":"172.21.15.143:6834","nonce":37280614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/2364281303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":2364281303},{"type":"v1","addr":"172.21.15.143:6835","nonce":2364281303}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6836/4066129908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":4066129908},{"type":"v1","addr":"172.21.15.174:6836","nonce":4066129908}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T19:24:49.995547+0000","modified":"2023-11-08T19:24:53.027089+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.174:6837/2124972405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6835","nonce":2124972405},{"type":"v1","addr":"172.21.15.174:6837","nonce":2124972405}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-08T19:25:35.252 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.250+0000 7f2e8ca96700 1 -- 172.21.15.143:0/3121668131 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f2e6806af00 msgr2=0x7f2e6806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:35.252 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.250+0000 7f2e8ca96700 1 --2- 172.21.15.143:0/3121668131 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f2e6806af00 0x7f2e6806d3c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f2e880cbc30 tx=0x7f2e7c0091c0 comp rx=0 tx=0).stop 2023-11-08T19:25:35.252 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.250+0000 7f2e8ca96700 1 -- 172.21.15.143:0/3121668131 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2e88159870 msgr2=0x7f2e880b17b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:35.253 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.250+0000 7f2e8ca96700 1 --2- 172.21.15.143:0/3121668131 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2e88159870 0x7f2e880b17b0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f2e80014040 tx=0x7f2e80004ad0 comp rx=0 tx=0).stop 2023-11-08T19:25:35.253 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.250+0000 7f2e8ca96700 1 -- 172.21.15.143:0/3121668131 shutdown_connections 2023-11-08T19:25:35.253 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.250+0000 7f2e8ca96700 1 --2- 172.21.15.143:0/3121668131 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f2e6806af00 0x7f2e6806d3c0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:35.253 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.250+0000 7f2e8ca96700 1 --2- 172.21.15.143:0/3121668131 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2e8806ed60 0x7f2e881f09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:35.254 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.250+0000 7f2e8ca96700 1 --2- 172.21.15.143:0/3121668131 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2e88159870 0x7f2e880b17b0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:35.254 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.250+0000 7f2e8ca96700 1 --2- 172.21.15.143:0/3121668131 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2e880cc410 0x7f2e880b4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:35.254 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.250+0000 7f2e8ca96700 1 -- 172.21.15.143:0/3121668131 >> 172.21.15.143:0/3121668131 conn(0x7f2e88099ab0 msgr2=0x7f2e8815e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:35.254 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.250+0000 7f2e8ca96700 1 -- 172.21.15.143:0/3121668131 shutdown_connections 2023-11-08T19:25:35.254 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.250+0000 7f2e8ca96700 1 -- 172.21.15.143:0/3121668131 wait complete. 2023-11-08T19:25:35.255 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 8 2023-11-08T19:25:35.264 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:25:35.385 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.383+0000 7ff465261700 1 Processor -- start 2023-11-08T19:25:35.385 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.384+0000 7ff465261700 1 -- start start 2023-11-08T19:25:35.388 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.384+0000 7ff465261700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff46006dbb0 0x7ff4600ca470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:35.389 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.384+0000 7ff465261700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7ff46006df90 0x7ff46006aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:35.389 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.384+0000 7ff465261700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4600ce7f0 con 0x7ff4600668e0 2023-11-08T19:25:35.389 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.384+0000 7ff465261700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff460066db0 con 0x7ff46006df90 2023-11-08T19:25:35.389 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.384+0000 7ff465261700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4600af5e0 con 0x7ff46006dbb0 2023-11-08T19:25:35.390 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.384+0000 7ff45e7fc700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7ff4600668e0 0x7ff4600ca9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:46278/0 (socket says 172.21.15.143:46278) 2023-11-08T19:25:35.390 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.384+0000 7ff45e7fc700 1 -- 172.21.15.143:0/1324223328 learned_addr learned my addr 172.21.15.143:0/1324223328 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:25:35.390 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.385+0000 7ff45f7fe700 1 --2- 172.21.15.143:0/1324223328 >> v2:172.21.15.174:3300/0 conn(0x7ff46006df90 0x7ff46006aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:35.390 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.385+0000 7ff45effd700 1 --2- 172.21.15.143:0/1324223328 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff46006dbb0 0x7ff4600ca470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:35.391 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.385+0000 7ff45dffb700 1 -- 172.21.15.143:0/1324223328 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3969669621 0 0) 0x7ff4600ce7f0 con 0x7ff4600668e0 2023-11-08T19:25:35.391 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.385+0000 7ff45dffb700 1 -- 172.21.15.143:0/1324223328 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff4480035b0 con 0x7ff4600668e0 2023-11-08T19:25:35.391 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.385+0000 7ff45f7fe700 1 -- 172.21.15.143:0/1324223328 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff46006dbb0 msgr2=0x7ff4600ca470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:35.391 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.385+0000 7ff45f7fe700 1 --2- 172.21.15.143:0/1324223328 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff46006dbb0 0x7ff4600ca470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:35.392 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.385+0000 7ff45f7fe700 1 -- 172.21.15.143:0/1324223328 >> v1:172.21.15.143:6789/0 conn(0x7ff4600668e0 legacy=0x7ff4600ca9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:35.392 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.385+0000 7ff45dffb700 1 -- 172.21.15.143:0/1324223328 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2951932189 0 0) 0x7ff4480035b0 con 0x7ff4600668e0 2023-11-08T19:25:35.392 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.385+0000 7ff45f7fe700 1 -- 172.21.15.143:0/1324223328 --> v2:172.21.15.174:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4600cb520 con 0x7ff46006df90 2023-11-08T19:25:35.393 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.385+0000 7ff45f7fe700 1 --2- 172.21.15.143:0/1324223328 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff46006df90 0x7ff46006aff0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff458009d90 tx=0x7ff45800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9548dd39f0470570 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:35.393 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.385+0000 7ff45dffb700 1 -- 172.21.15.143:0/1324223328 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff45800e070 con 0x7ff46006df90 2023-11-08T19:25:35.393 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.385+0000 7ff45dffb700 1 -- 172.21.15.143:0/1324223328 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff45800bb20 con 0x7ff46006df90 2023-11-08T19:25:35.393 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.385+0000 7ff45dffb700 1 -- 172.21.15.143:0/1324223328 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff458012430 con 0x7ff46006df90 2023-11-08T19:25:35.393 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.386+0000 7ff465261700 1 -- 172.21.15.143:0/1324223328 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff46006df90 msgr2=0x7ff46006aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:35.394 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.386+0000 7ff465261700 1 --2- 172.21.15.143:0/1324223328 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff46006df90 0x7ff46006aff0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff458009d90 tx=0x7ff45800b330 comp rx=0 tx=0).stop 2023-11-08T19:25:35.394 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.386+0000 7ff465261700 1 -- 172.21.15.143:0/1324223328 shutdown_connections 2023-11-08T19:25:35.394 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.386+0000 7ff465261700 1 --2- 172.21.15.143:0/1324223328 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff46006df90 0x7ff46006aff0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:35.395 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.386+0000 7ff465261700 1 --2- 172.21.15.143:0/1324223328 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff46006dbb0 0x7ff4600ca470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:35.395 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.386+0000 7ff465261700 1 -- 172.21.15.143:0/1324223328 >> 172.21.15.143:0/1324223328 conn(0x7ff4600b1a20 msgr2=0x7ff4600b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:35.395 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.386+0000 7ff465261700 1 -- 172.21.15.143:0/1324223328 shutdown_connections 2023-11-08T19:25:35.395 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.386+0000 7ff465261700 1 -- 172.21.15.143:0/1324223328 wait complete. 2023-11-08T19:25:35.395 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.387+0000 7ff465261700 1 Processor -- start 2023-11-08T19:25:35.396 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.387+0000 7ff465261700 1 -- start start 2023-11-08T19:25:35.398 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.388+0000 7ff465261700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff4600668e0 0x7ff46015a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:35.399 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.388+0000 7ff465261700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff46006dbb0 0x7ff4600b3f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:35.399 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.388+0000 7ff465261700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff46006df90 0x7ff4601e7580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:35.399 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.388+0000 7ff465261700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4600b5a20 con 0x7ff46006df90 2023-11-08T19:25:35.399 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.388+0000 7ff465261700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4600b4980 con 0x7ff46006dbb0 2023-11-08T19:25:35.399 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.388+0000 7ff465261700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4600b4c60 con 0x7ff4600668e0 2023-11-08T19:25:35.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.388+0000 7ff45f7fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff46006df90 0x7ff4601e7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:35.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.388+0000 7ff45f7fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff46006df90 0x7ff4601e7580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50132/0 (socket says 172.21.15.143:50132) 2023-11-08T19:25:35.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.388+0000 7ff45f7fe700 1 -- 172.21.15.143:0/1239113361 learned_addr learned my addr 172.21.15.143:0/1239113361 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:35.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.388+0000 7ff45e7fc700 1 --2- 172.21.15.143:0/1239113361 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff46006dbb0 0x7ff4600b3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:35.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.388+0000 7ff45effd700 1 --2- 172.21.15.143:0/1239113361 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff4600668e0 0x7ff46015a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:35.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.388+0000 7ff45f7fe700 1 -- 172.21.15.143:0/1239113361 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff4600668e0 msgr2=0x7ff46015a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:35.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.388+0000 7ff45f7fe700 1 --2- 172.21.15.143:0/1239113361 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff4600668e0 0x7ff46015a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:35.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.388+0000 7ff45f7fe700 1 -- 172.21.15.143:0/1239113361 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff46006dbb0 msgr2=0x7ff4600b3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:35.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.388+0000 7ff45f7fe700 1 --2- 172.21.15.143:0/1239113361 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff46006dbb0 0x7ff4600b3f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:35.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.388+0000 7ff45f7fe700 1 -- 172.21.15.143:0/1239113361 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff458009a70 con 0x7ff46006df90 2023-11-08T19:25:35.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.389+0000 7ff45e7fc700 1 --2- 172.21.15.143:0/1239113361 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff46006dbb0 0x7ff4600b3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:25:35.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.389+0000 7ff45f7fe700 1 --2- 172.21.15.143:0/1239113361 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff46006df90 0x7ff4601e7580 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7ff4600d0db0 tx=0x7ff458004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:35.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.389+0000 7ff4477fe700 1 -- 172.21.15.143:0/1239113361 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff45801b810 con 0x7ff46006df90 2023-11-08T19:25:35.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.389+0000 7ff4477fe700 1 -- 172.21.15.143:0/1239113361 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff458020070 con 0x7ff46006df90 2023-11-08T19:25:35.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.389+0000 7ff465261700 1 -- 172.21.15.143:0/1239113361 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4600b4f40 con 0x7ff46006df90 2023-11-08T19:25:35.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.389+0000 7ff4477fe700 1 -- 172.21.15.143:0/1239113361 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff45801bbb0 con 0x7ff46006df90 2023-11-08T19:25:35.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.389+0000 7ff465261700 1 -- 172.21.15.143:0/1239113361 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4600b5500 con 0x7ff46006df90 2023-11-08T19:25:35.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.390+0000 7ff465261700 1 -- 172.21.15.143:0/1239113361 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff46006d1a0 con 0x7ff46006df90 2023-11-08T19:25:35.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.390+0000 7ff4477fe700 1 -- 172.21.15.143:0/1239113361 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff45801bd10 con 0x7ff46006df90 2023-11-08T19:25:35.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.391+0000 7ff4477fe700 1 --2- 172.21.15.143:0/1239113361 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7ff44c06af00 0x7ff44c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:35.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.391+0000 7ff4477fe700 1 -- 172.21.15.143:0/1239113361 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff458022070 con 0x7ff46006df90 2023-11-08T19:25:35.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.391+0000 7ff45effd700 1 --2- 172.21.15.143:0/1239113361 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7ff44c06af00 0x7ff44c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:35.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.391+0000 7ff45effd700 1 --2- 172.21.15.143:0/1239113361 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7ff44c06af00 0x7ff44c06d3c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff4601f6d50 tx=0x7ff454008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:35.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.395+0000 7ff4477fe700 1 -- 172.21.15.143:0/1239113361 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff458056a40 con 0x7ff46006df90 2023-11-08T19:25:35.586 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.585+0000 7ff465261700 1 -- 172.21.15.143:0/1239113361 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff46004eee0 con 0x7ff46006df90 2023-11-08T19:25:35.587 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.586+0000 7ff4477fe700 1 -- 172.21.15.143:0/1239113361 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16553 (secure 0 0 0) 0x7ff45805a060 con 0x7ff46006df90 2023-11-08T19:25:35.588 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:25:35.588 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":20,"fsid":"968483af-77fc-4068-ad7c-5c2a1a4613d7","created":"2023-11-08T19:24:32.826484+0000","modified":"2023-11-08T19:24:51.618072+0000","last_up_change":"2023-11-08T19:24:43.573880+0000","last_in_change":"2023-11-08T19:24:35.314525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:44.551530+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-08T19:24:48.256781+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-08T19:24:48.954990+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bae98ec1-d94e-403d-a220-a522a53ea6f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6809","nonce":1487830566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6811","nonce":1487830566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6815","nonce":1487830566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6813","nonce":1487830566}]},"public_addr":"172.21.15.143:6809/1487830566","cluster_addr":"172.21.15.143:6811/1487830566","heartbeat_back_addr":"172.21.15.143:6815/1487830566","heartbeat_front_addr":"172.21.15.143:6813/1487830566","state":["exists","up"]},{"osd":1,"uuid":"5cb6538a-1ccb-4322-8633-09f66af0b7e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6801","nonce":787341397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6803","nonce":787341397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6807","nonce":787341397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6805","nonce":787341397}]},"public_addr":"172.21.15.143:6801/787341397","cluster_addr":"172.21.15.143:6803/787341397","heartbeat_back_addr":"172.21.15.143:6807/787341397","heartbeat_front_addr":"172.21.15.143:6805/787341397","state":["exists","up"]},{"osd":2,"uuid":"cdbf522c-60ef-4d60-9653-8dcb529c101a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6822","nonce":2301325597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6826","nonce":2301325597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6831","nonce":2301325597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6829","nonce":2301325597}]},"public_addr":"172.21.15.143:6822/2301325597","cluster_addr":"172.21.15.143:6826/2301325597","heartbeat_back_addr":"172.21.15.143:6831/2301325597","heartbeat_front_addr":"172.21.15.143:6829/2301325597","state":["exists","up"]},{"osd":3,"uuid":"4e2783cf-ebb9-4dd8-8245-bc1d17ce4325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6817","nonce":2021504428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6819","nonce":2021504428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6827","nonce":2021504428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6823","nonce":2021504428}]},"public_addr":"172.21.15.143:6817/2021504428","cluster_addr":"172.21.15.143:6819/2021504428","heartbeat_back_addr":"172.21.15.143:6827/2021504428","heartbeat_front_addr":"172.21.15.143:6823/2021504428","state":["exists","up"]},{"osd":4,"uuid":"691256fa-9b77-49b9-a346-7ae1adf0ca80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6801","nonce":1993380595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6803","nonce":1993380595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6807","nonce":1993380595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6805","nonce":1993380595}]},"public_addr":"172.21.15.174:6801/1993380595","cluster_addr":"172.21.15.174:6803/1993380595","heartbeat_back_addr":"172.21.15.174:6807/1993380595","heartbeat_front_addr":"172.21.15.174:6805/1993380595","state":["exists","up"]},{"osd":5,"uuid":"756fc352-48c9-46ce-b52e-0541fe6f8525","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6809","nonce":1777923303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6811","nonce":1777923303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6815","nonce":1777923303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6813","nonce":1777923303}]},"public_addr":"172.21.15.174:6809/1777923303","cluster_addr":"172.21.15.174:6811/1777923303","heartbeat_back_addr":"172.21.15.174:6815/1777923303","heartbeat_front_addr":"172.21.15.174:6813/1777923303","state":["exists","up"]},{"osd":6,"uuid":"fab2ac2b-ed97-4cde-9825-b480eabf8b8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6825","nonce":2755130316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6827","nonce":2755130316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6831","nonce":2755130316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6829","nonce":2755130316}]},"public_addr":"172.21.15.174:6825/2755130316","cluster_addr":"172.21.15.174:6827/2755130316","heartbeat_back_addr":"172.21.15.174:6831/2755130316","heartbeat_front_addr":"172.21.15.174:6829/2755130316","state":["exists","up"]},{"osd":7,"uuid":"3de9b797-2ab6-4d41-a863-7e7d638fd703","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6817","nonce":2847141411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6819","nonce":2847141411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6823","nonce":2847141411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6821","nonce":2847141411}]},"public_addr":"172.21.15.174:6817/2847141411","cluster_addr":"172.21.15.174:6819/2847141411","heartbeat_back_addr":"172.21.15.174:6823/2847141411","heartbeat_front_addr":"172.21.15.174:6821/2847141411","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.477076+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.496890+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.539305+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.454673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.538881+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.521294+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.520712+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.540404+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:25:35.590 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.588+0000 7ff465261700 1 -- 172.21.15.143:0/1239113361 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7ff44c06af00 msgr2=0x7ff44c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:35.590 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.588+0000 7ff465261700 1 --2- 172.21.15.143:0/1239113361 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7ff44c06af00 0x7ff44c06d3c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff4601f6d50 tx=0x7ff454008040 comp rx=0 tx=0).stop 2023-11-08T19:25:35.590 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.588+0000 7ff465261700 1 -- 172.21.15.143:0/1239113361 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff46006df90 msgr2=0x7ff4601e7580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:35.591 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.588+0000 7ff465261700 1 --2- 172.21.15.143:0/1239113361 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff46006df90 0x7ff4601e7580 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7ff4600d0db0 tx=0x7ff458004ad0 comp rx=0 tx=0).stop 2023-11-08T19:25:35.591 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.588+0000 7ff465261700 1 -- 172.21.15.143:0/1239113361 shutdown_connections 2023-11-08T19:25:35.591 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.588+0000 7ff465261700 1 --2- 172.21.15.143:0/1239113361 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7ff44c06af00 0x7ff44c06d3c0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:35.591 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.588+0000 7ff465261700 1 --2- 172.21.15.143:0/1239113361 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff4600668e0 0x7ff46015a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:35.592 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.588+0000 7ff465261700 1 --2- 172.21.15.143:0/1239113361 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff46006df90 0x7ff4601e7580 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:35.592 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.588+0000 7ff465261700 1 --2- 172.21.15.143:0/1239113361 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff46006dbb0 0x7ff4600b3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:35.592 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.588+0000 7ff465261700 1 -- 172.21.15.143:0/1239113361 >> 172.21.15.143:0/1239113361 conn(0x7ff4600b1a20 msgr2=0x7ff46006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:35.592 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.588+0000 7ff465261700 1 -- 172.21.15.143:0/1239113361 shutdown_connections 2023-11-08T19:25:35.592 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.588+0000 7ff465261700 1 -- 172.21.15.143:0/1239113361 wait complete. 2023-11-08T19:25:35.605 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T19:24:44.551530+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-08T19:24:48.256781+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-08T19:24:48.954990+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-08T19:25:35.606 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T19:25:35.729 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.727+0000 7efdc0232700 1 Processor -- start 2023-11-08T19:25:35.729 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.728+0000 7efdc0232700 1 -- start start 2023-11-08T19:25:35.736 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.728+0000 7efdc0232700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efdb806bd20 0x7efdb80d01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:35.736 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.728+0000 7efdc0232700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdb80cc2a0 con 0x7efdb815a950 2023-11-08T19:25:35.736 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.728+0000 7efdc0232700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdb806c1b0 con 0x7efdb80b61e0 2023-11-08T19:25:35.736 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.728+0000 7efdc0232700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdb80cc130 con 0x7efdb806bd20 2023-11-08T19:25:35.737 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.729+0000 7efdbdfce700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7efdb815a950 0x7efdb8072370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:46290/0 (socket says 172.21.15.143:46290) 2023-11-08T19:25:35.738 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.729+0000 7efdbdfce700 1 -- 172.21.15.143:0/592269292 learned_addr learned my addr 172.21.15.143:0/592269292 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:25:35.738 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.729+0000 7efdbcfcc700 1 -- 172.21.15.143:0/592269292 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2371316799 0 0) 0x7efdb80cc2a0 con 0x7efdb815a950 2023-11-08T19:25:35.738 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.729+0000 7efdbd7cd700 1 --2- 172.21.15.143:0/592269292 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efdb806bd20 0x7efdb80d01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:35.738 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.729+0000 7efdbcfcc700 1 -- 172.21.15.143:0/592269292 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efda40035b0 con 0x7efdb815a950 2023-11-08T19:25:35.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.729+0000 7efdbcfcc700 1 -- 172.21.15.143:0/592269292 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 824449461 0 0) 0x7efda40035b0 con 0x7efdb815a950 2023-11-08T19:25:35.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.729+0000 7efdbcfcc700 1 -- 172.21.15.143:0/592269292 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efdb806bd20 msgr2=0x7efdb80d01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:35.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.729+0000 7efdbcfcc700 1 --2- 172.21.15.143:0/592269292 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efdb806bd20 0x7efdb80d01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:35.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.729+0000 7efdbcfcc700 1 -- 172.21.15.143:0/592269292 >> v1:172.21.15.174:6789/0 conn(0x7efdb80b61e0 legacy=0x7efdb80cc730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:35.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.729+0000 7efdbcfcc700 1 -- 172.21.15.143:0/592269292 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdb806c490 con 0x7efdb815a950 2023-11-08T19:25:35.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.729+0000 7efdbcfcc700 1 -- 172.21.15.143:0/592269292 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4147756303 0 0) 0x7efda8001670 con 0x7efdb815a950 2023-11-08T19:25:35.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.729+0000 7efdbcfcc700 1 -- 172.21.15.143:0/592269292 >> v1:172.21.15.143:6789/0 conn(0x7efdb815a950 legacy=0x7efdb8072370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:35.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.729+0000 7efdbcfcc700 1 --2- 172.21.15.143:0/592269292 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efda4003a30 0x7efda4003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:35.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.729+0000 7efdbcfcc700 1 --2- 172.21.15.143:0/592269292 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efda4004850 0x7efda4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:35.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.729+0000 7efdbcfcc700 1 --2- 172.21.15.143:0/592269292 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efda40051d0 0x7efda400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:35.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.729+0000 7efdbcfcc700 1 -- 172.21.15.143:0/592269292 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efda400bf50 con 0x7efda4003a30 2023-11-08T19:25:35.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.729+0000 7efdbcfcc700 1 -- 172.21.15.143:0/592269292 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efda400c090 con 0x7efda40051d0 2023-11-08T19:25:35.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.729+0000 7efdbcfcc700 1 -- 172.21.15.143:0/592269292 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efda400c220 con 0x7efda4004850 2023-11-08T19:25:35.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.729+0000 7efdbdfce700 1 --2- 172.21.15.143:0/592269292 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efda4003a30 0x7efda4003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:35.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.729+0000 7efdbd7cd700 1 --2- 172.21.15.143:0/592269292 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efda4004850 0x7efda4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:35.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.729+0000 7efdbe7cf700 1 --2- 172.21.15.143:0/592269292 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efda40051d0 0x7efda400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:35.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.730+0000 7efdbdfce700 1 -- 172.21.15.143:0/592269292 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efda4004850 msgr2=0x7efda4004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:35.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.730+0000 7efdbdfce700 1 --2- 172.21.15.143:0/592269292 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efda4004850 0x7efda4004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:35.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.730+0000 7efdbdfce700 1 -- 172.21.15.143:0/592269292 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efda40051d0 msgr2=0x7efda400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:35.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.730+0000 7efdbdfce700 1 --2- 172.21.15.143:0/592269292 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efda40051d0 0x7efda400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:35.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.730+0000 7efdbdfce700 1 -- 172.21.15.143:0/592269292 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efda40035b0 con 0x7efda4003a30 2023-11-08T19:25:35.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.730+0000 7efdbdfce700 1 --2- 172.21.15.143:0/592269292 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efda4003a30 0x7efda4003e50 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7efda8001050 tx=0x7efda4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=382451a3d047424b server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:35.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.730+0000 7efdbcfcc700 1 -- 172.21.15.143:0/592269292 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efda800cd80 con 0x7efda4003a30 2023-11-08T19:25:35.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.730+0000 7efdbcfcc700 1 -- 172.21.15.143:0/592269292 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efda8005960 con 0x7efda4003a30 2023-11-08T19:25:35.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.730+0000 7efdbcfcc700 1 -- 172.21.15.143:0/592269292 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efda8005d10 con 0x7efda4003a30 2023-11-08T19:25:35.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.730+0000 7efdc0232700 1 -- 172.21.15.143:0/592269292 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efda4003a30 msgr2=0x7efda4003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:35.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.730+0000 7efdc0232700 1 --2- 172.21.15.143:0/592269292 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efda4003a30 0x7efda4003e50 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7efda8001050 tx=0x7efda4005880 comp rx=0 tx=0).stop 2023-11-08T19:25:35.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.730+0000 7efdbe7cf700 1 -- 172.21.15.143:0/592269292 reap_dead start 2023-11-08T19:25:35.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.730+0000 7efdc0232700 1 -- 172.21.15.143:0/592269292 shutdown_connections 2023-11-08T19:25:35.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.730+0000 7efdc0232700 1 -- 172.21.15.143:0/592269292 >> 172.21.15.143:0/592269292 conn(0x7efdb80cd2e0 msgr2=0x7efdb80cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:35.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.731+0000 7efdc0232700 1 -- 172.21.15.143:0/592269292 shutdown_connections 2023-11-08T19:25:35.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.731+0000 7efdc0232700 1 -- 172.21.15.143:0/592269292 wait complete. 2023-11-08T19:25:35.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.732+0000 7efdc0232700 1 Processor -- start 2023-11-08T19:25:35.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.732+0000 7efdc0232700 1 -- start start 2023-11-08T19:25:35.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.733+0000 7efdc0232700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efda4003a30 0x7efdb8158bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:35.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.733+0000 7efdc0232700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efdb80af990 0x7efdb815ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:35.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.733+0000 7efdc0232700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efdb815d370 0x7efdb81f69c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:35.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.733+0000 7efdc0232700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdb815d800 con 0x7efda4003a30 2023-11-08T19:25:35.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.733+0000 7efdc0232700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdb81f7730 con 0x7efdb815d370 2023-11-08T19:25:35.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.733+0000 7efdc0232700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdb80aebd0 con 0x7efdb80af990 2023-11-08T19:25:35.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.733+0000 7efdbdfce700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efda4003a30 0x7efdb8158bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:35.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.733+0000 7efdbdfce700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efda4003a30 0x7efdb8158bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50148/0 (socket says 172.21.15.143:50148) 2023-11-08T19:25:35.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.733+0000 7efdbdfce700 1 -- 172.21.15.143:0/1351647049 learned_addr learned my addr 172.21.15.143:0/1351647049 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:35.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.733+0000 7efdbd7cd700 1 --2- 172.21.15.143:0/1351647049 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efdb80af990 0x7efdb815ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:35.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.733+0000 7efdbe7cf700 1 --2- 172.21.15.143:0/1351647049 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efdb815d370 0x7efdb81f69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:35.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.733+0000 7efdbdfce700 1 -- 172.21.15.143:0/1351647049 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efdb80af990 msgr2=0x7efdb815ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:35.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.733+0000 7efdbdfce700 1 --2- 172.21.15.143:0/1351647049 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efdb80af990 0x7efdb815ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:35.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.733+0000 7efdbdfce700 1 -- 172.21.15.143:0/1351647049 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efdb815d370 msgr2=0x7efdb81f69c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:35.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.733+0000 7efdbdfce700 1 --2- 172.21.15.143:0/1351647049 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efdb815d370 0x7efdb81f69c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:35.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.733+0000 7efdbdfce700 1 -- 172.21.15.143:0/1351647049 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdb80aeeb0 con 0x7efda4003a30 2023-11-08T19:25:35.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.733+0000 7efdbe7cf700 1 --2- 172.21.15.143:0/1351647049 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efdb815d370 0x7efdb81f69c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:25:35.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.733+0000 7efdbdfce700 1 --2- 172.21.15.143:0/1351647049 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efda4003a30 0x7efdb8158bc0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7efda4005800 tx=0x7efda8000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:35.750 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.733+0000 7efdae7fc700 1 -- 172.21.15.143:0/1351647049 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efda8007c00 con 0x7efda4003a30 2023-11-08T19:25:35.750 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.733+0000 7efdae7fc700 1 -- 172.21.15.143:0/1351647049 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efda8004870 con 0x7efda4003a30 2023-11-08T19:25:35.750 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.733+0000 7efdc0232700 1 -- 172.21.15.143:0/1351647049 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdb80af190 con 0x7efda4003a30 2023-11-08T19:25:35.750 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.733+0000 7efdae7fc700 1 -- 172.21.15.143:0/1351647049 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efda8004c20 con 0x7efda4003a30 2023-11-08T19:25:35.750 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.733+0000 7efdc0232700 1 -- 172.21.15.143:0/1351647049 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdb8159100 con 0x7efda4003a30 2023-11-08T19:25:35.751 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.734+0000 7efdc0232700 1 -- 172.21.15.143:0/1351647049 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdb81592c0 con 0x7efda4003a30 2023-11-08T19:25:35.751 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.735+0000 7efdae7fc700 1 -- 172.21.15.143:0/1351647049 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7efda8019070 con 0x7efda4003a30 2023-11-08T19:25:35.752 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.735+0000 7efdae7fc700 1 --2- 172.21.15.143:0/1351647049 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7efd9c06af00 0x7efd9c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:35.752 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.735+0000 7efdae7fc700 1 -- 172.21.15.143:0/1351647049 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efda808a7a0 con 0x7efda4003a30 2023-11-08T19:25:35.752 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.735+0000 7efdbd7cd700 1 --2- 172.21.15.143:0/1351647049 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7efd9c06af00 0x7efd9c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:35.752 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.736+0000 7efdbd7cd700 1 --2- 172.21.15.143:0/1351647049 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7efd9c06af00 0x7efd9c06d3c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7efdb80b59a0 tx=0x7efdb4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:35.753 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.739+0000 7efdae7fc700 1 -- 172.21.15.143:0/1351647049 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7efdb81592c0 con 0x7efda4003a30 2023-11-08T19:25:35.927 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.926+0000 7efdc0232700 1 -- 172.21.15.143:0/1351647049 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7efdb80af6b0 con 0x7efda4003a30 2023-11-08T19:25:35.928 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.927+0000 7efdae7fc700 1 -- 172.21.15.143:0/1351647049 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7efdb80af6b0 con 0x7efda4003a30 2023-11-08T19:25:35.928 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2023-11-08T19:25:35.930 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.928+0000 7efdc0232700 1 -- 172.21.15.143:0/1351647049 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7efd9c06af00 msgr2=0x7efd9c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:35.930 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.928+0000 7efdc0232700 1 --2- 172.21.15.143:0/1351647049 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7efd9c06af00 0x7efd9c06d3c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7efdb80b59a0 tx=0x7efdb4008040 comp rx=0 tx=0).stop 2023-11-08T19:25:35.931 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.928+0000 7efdc0232700 1 -- 172.21.15.143:0/1351647049 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efda4003a30 msgr2=0x7efdb8158bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:35.931 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.929+0000 7efdc0232700 1 --2- 172.21.15.143:0/1351647049 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efda4003a30 0x7efdb8158bc0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7efda4005800 tx=0x7efda8000fe0 comp rx=0 tx=0).stop 2023-11-08T19:25:35.931 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.929+0000 7efdc0232700 1 -- 172.21.15.143:0/1351647049 shutdown_connections 2023-11-08T19:25:35.931 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.929+0000 7efdc0232700 1 --2- 172.21.15.143:0/1351647049 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7efd9c06af00 0x7efd9c06d3c0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:35.931 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.929+0000 7efdc0232700 1 --2- 172.21.15.143:0/1351647049 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efdb815d370 0x7efdb81f69c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:35.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.929+0000 7efdc0232700 1 --2- 172.21.15.143:0/1351647049 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efda4003a30 0x7efdb8158bc0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:35.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.929+0000 7efdc0232700 1 --2- 172.21.15.143:0/1351647049 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efdb80af990 0x7efdb815ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:35.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.929+0000 7efdc0232700 1 -- 172.21.15.143:0/1351647049 >> 172.21.15.143:0/1351647049 conn(0x7efdb80cd2e0 msgr2=0x7efdb8159510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:35.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.929+0000 7efdc0232700 1 -- 172.21.15.143:0/1351647049 shutdown_connections 2023-11-08T19:25:35.933 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:35.929+0000 7efdc0232700 1 -- 172.21.15.143:0/1351647049 wait complete. 2023-11-08T19:25:35.942 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-08T19:25:36.066 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.065+0000 7fd20711e700 1 Processor -- start 2023-11-08T19:25:36.066 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.065+0000 7fd20711e700 1 -- start start 2023-11-08T19:25:36.069 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.065+0000 7fd20711e700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd20006c0c0 0x7fd2000ccfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:36.070 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.065+0000 7fd20711e700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd20015c9b0 0x7fd200071900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:36.070 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.065+0000 7fd20711e700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2000ced30 con 0x7fd20015ce30 2023-11-08T19:25:36.070 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.065+0000 7fd20711e700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2000cd4f0 con 0x7fd20015c9b0 2023-11-08T19:25:36.070 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.065+0000 7fd20711e700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2000cdf60 con 0x7fd20006c0c0 2023-11-08T19:25:36.071 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.066+0000 7fd1fffff700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd20015c9b0 0x7fd200071900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:36.071 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.066+0000 7fd2056bb700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fd20015ce30 0x7fd2001522a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:46314/0 (socket says 172.21.15.143:46314) 2023-11-08T19:25:36.071 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.066+0000 7fd2056bb700 1 -- 172.21.15.143:0/2942322347 learned_addr learned my addr 172.21.15.143:0/2942322347 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:25:36.071 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.066+0000 7fd204eba700 1 --2- 172.21.15.143:0/2942322347 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd20006c0c0 0x7fd2000ccfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:36.072 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.066+0000 7fd1fffff700 1 -- 172.21.15.143:0/2942322347 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd20006c0c0 msgr2=0x7fd2000ccfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:36.072 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.066+0000 7fd1fffff700 1 --2- 172.21.15.143:0/2942322347 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd20006c0c0 0x7fd2000ccfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:36.072 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.066+0000 7fd1fffff700 1 -- 172.21.15.143:0/2942322347 >> v1:172.21.15.143:6789/0 conn(0x7fd20015ce30 legacy=0x7fd2001522a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:36.072 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.066+0000 7fd1ff7fe700 1 -- 172.21.15.143:0/2942322347 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1979781485 0 0) 0x7fd2000ced30 con 0x7fd20015ce30 2023-11-08T19:25:36.073 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.066+0000 7fd1fffff700 1 -- 172.21.15.143:0/2942322347 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2000cea50 con 0x7fd20015c9b0 2023-11-08T19:25:36.073 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.066+0000 7fd204eba700 1 --2- 172.21.15.143:0/2942322347 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd20006c0c0 0x7fd2000ccfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:25:36.073 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.066+0000 7fd1fffff700 1 --2- 172.21.15.143:0/2942322347 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd20015c9b0 0x7fd200071900 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec009840 tx=0x7fd1ec009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8da2d127c21a08be server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:36.073 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.066+0000 7fd1ff7fe700 1 -- 172.21.15.143:0/2942322347 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1ec011830 con 0x7fd20015c9b0 2023-11-08T19:25:36.074 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.066+0000 7fd1ff7fe700 1 -- 172.21.15.143:0/2942322347 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1ec011990 con 0x7fd20015c9b0 2023-11-08T19:25:36.074 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.066+0000 7fd1ff7fe700 1 -- 172.21.15.143:0/2942322347 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1ec011d20 con 0x7fd20015c9b0 2023-11-08T19:25:36.074 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.067+0000 7fd20711e700 1 -- 172.21.15.143:0/2942322347 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd20015c9b0 msgr2=0x7fd200071900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:36.074 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.067+0000 7fd20711e700 1 --2- 172.21.15.143:0/2942322347 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd20015c9b0 0x7fd200071900 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec009840 tx=0x7fd1ec009b50 comp rx=0 tx=0).stop 2023-11-08T19:25:36.075 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.067+0000 7fd20711e700 1 -- 172.21.15.143:0/2942322347 shutdown_connections 2023-11-08T19:25:36.075 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.067+0000 7fd20711e700 1 --2- 172.21.15.143:0/2942322347 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd20006c0c0 0x7fd2000ccfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:36.075 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.067+0000 7fd20711e700 1 --2- 172.21.15.143:0/2942322347 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd20015c9b0 0x7fd200071900 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:36.075 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.067+0000 7fd20711e700 1 -- 172.21.15.143:0/2942322347 >> 172.21.15.143:0/2942322347 conn(0x7fd2000b1c70 msgr2=0x7fd2000b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:36.075 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.067+0000 7fd20711e700 1 -- 172.21.15.143:0/2942322347 shutdown_connections 2023-11-08T19:25:36.076 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.067+0000 7fd20711e700 1 -- 172.21.15.143:0/2942322347 wait complete. 2023-11-08T19:25:36.076 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.068+0000 7fd20711e700 1 Processor -- start 2023-11-08T19:25:36.076 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.068+0000 7fd20711e700 1 -- start start 2023-11-08T19:25:36.080 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.069+0000 7fd20711e700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd20006c0c0 0x7fd2000c8620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:36.081 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.069+0000 7fd20711e700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd20015c9b0 0x7fd2000b37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:36.081 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.069+0000 7fd20711e700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd20015ce30 0x7fd200154ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:36.081 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.069+0000 7fd20711e700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2000b2d60 con 0x7fd20006c0c0 2023-11-08T19:25:36.081 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.069+0000 7fd20711e700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2000b3cf0 con 0x7fd20015c9b0 2023-11-08T19:25:36.082 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.069+0000 7fd20711e700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2000c8c00 con 0x7fd20015ce30 2023-11-08T19:25:36.082 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.069+0000 7fd204eba700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd20006c0c0 0x7fd2000c8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:36.082 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.069+0000 7fd204eba700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd20006c0c0 0x7fd2000c8620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50162/0 (socket says 172.21.15.143:50162) 2023-11-08T19:25:36.082 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.069+0000 7fd204eba700 1 -- 172.21.15.143:0/2017168964 learned_addr learned my addr 172.21.15.143:0/2017168964 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:36.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.070+0000 7fd2056bb700 1 --2- 172.21.15.143:0/2017168964 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd20015ce30 0x7fd200154ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:36.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.070+0000 7fd1fffff700 1 --2- 172.21.15.143:0/2017168964 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd20015c9b0 0x7fd2000b37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:36.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.070+0000 7fd204eba700 1 -- 172.21.15.143:0/2017168964 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd20015ce30 msgr2=0x7fd200154ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:36.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.070+0000 7fd204eba700 1 --2- 172.21.15.143:0/2017168964 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd20015ce30 0x7fd200154ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:36.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.070+0000 7fd204eba700 1 -- 172.21.15.143:0/2017168964 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd20015c9b0 msgr2=0x7fd2000b37b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:36.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.070+0000 7fd204eba700 1 --2- 172.21.15.143:0/2017168964 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd20015c9b0 0x7fd2000b37b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:36.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.070+0000 7fd204eba700 1 -- 172.21.15.143:0/2017168964 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1ec0094a0 con 0x7fd20006c0c0 2023-11-08T19:25:36.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.070+0000 7fd2056bb700 1 --2- 172.21.15.143:0/2017168964 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd20015ce30 0x7fd200154ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:25:36.085 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.070+0000 7fd1fffff700 1 --2- 172.21.15.143:0/2017168964 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd20015c9b0 0x7fd2000b37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:25:36.085 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.070+0000 7fd204eba700 1 --2- 172.21.15.143:0/2017168964 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd20006c0c0 0x7fd2000c8620 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fd1f400c750 tx=0x7fd1f400cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:36.085 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.070+0000 7fd1fd7fa700 1 -- 172.21.15.143:0/2017168964 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1f4011800 con 0x7fd20006c0c0 2023-11-08T19:25:36.085 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.070+0000 7fd1fd7fa700 1 -- 172.21.15.143:0/2017168964 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1f4011960 con 0x7fd20006c0c0 2023-11-08T19:25:36.086 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.070+0000 7fd1fd7fa700 1 -- 172.21.15.143:0/2017168964 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1f4011cd0 con 0x7fd20006c0c0 2023-11-08T19:25:36.086 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.070+0000 7fd20711e700 1 -- 172.21.15.143:0/2017168964 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2000c8ee0 con 0x7fd20006c0c0 2023-11-08T19:25:36.086 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.070+0000 7fd20711e700 1 -- 172.21.15.143:0/2017168964 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd200151790 con 0x7fd20006c0c0 2023-11-08T19:25:36.086 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.071+0000 7fd20711e700 1 -- 172.21.15.143:0/2017168964 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd20004eee0 con 0x7fd20006c0c0 2023-11-08T19:25:36.086 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.071+0000 7fd1fd7fa700 1 -- 172.21.15.143:0/2017168964 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd1f4021350 con 0x7fd20006c0c0 2023-11-08T19:25:36.087 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.072+0000 7fd1fd7fa700 1 --2- 172.21.15.143:0/2017168964 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd1e406af00 0x7fd1e406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:36.087 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.072+0000 7fd1fd7fa700 1 -- 172.21.15.143:0/2017168964 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd1f40894e0 con 0x7fd20006c0c0 2023-11-08T19:25:36.087 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.072+0000 7fd1fffff700 1 --2- 172.21.15.143:0/2017168964 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd1e406af00 0x7fd1e406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:36.087 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.072+0000 7fd1fffff700 1 --2- 172.21.15.143:0/2017168964 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd1e406af00 0x7fd1e406d3c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec0090b0 tx=0x7fd1ec008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:36.088 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.076+0000 7fd1fd7fa700 1 -- 172.21.15.143:0/2017168964 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd1f4054620 con 0x7fd20006c0c0 2023-11-08T19:25:36.265 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.264+0000 7fd20711e700 1 -- 172.21.15.143:0/2017168964 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd2000cc4a0 con 0x7fd20006c0c0 2023-11-08T19:25:36.266 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.264+0000 7fd1fd7fa700 1 -- 172.21.15.143:0/2017168964 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd1f405e020 con 0x7fd20006c0c0 2023-11-08T19:25:36.266 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2023-11-08T19:25:36.268 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.266+0000 7fd20711e700 1 -- 172.21.15.143:0/2017168964 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd1e406af00 msgr2=0x7fd1e406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:36.268 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.266+0000 7fd20711e700 1 --2- 172.21.15.143:0/2017168964 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd1e406af00 0x7fd1e406d3c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec0090b0 tx=0x7fd1ec008030 comp rx=0 tx=0).stop 2023-11-08T19:25:36.268 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.266+0000 7fd20711e700 1 -- 172.21.15.143:0/2017168964 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd20006c0c0 msgr2=0x7fd2000c8620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:36.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.266+0000 7fd20711e700 1 --2- 172.21.15.143:0/2017168964 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd20006c0c0 0x7fd2000c8620 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fd1f400c750 tx=0x7fd1f400cb10 comp rx=0 tx=0).stop 2023-11-08T19:25:36.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.266+0000 7fd20711e700 1 -- 172.21.15.143:0/2017168964 shutdown_connections 2023-11-08T19:25:36.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.266+0000 7fd20711e700 1 --2- 172.21.15.143:0/2017168964 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd1e406af00 0x7fd1e406d3c0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:36.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.266+0000 7fd20711e700 1 --2- 172.21.15.143:0/2017168964 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd20015ce30 0x7fd200154ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:36.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.266+0000 7fd20711e700 1 --2- 172.21.15.143:0/2017168964 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd20006c0c0 0x7fd2000c8620 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:36.270 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.266+0000 7fd20711e700 1 --2- 172.21.15.143:0/2017168964 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd20015c9b0 0x7fd2000b37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:36.270 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.266+0000 7fd20711e700 1 -- 172.21.15.143:0/2017168964 >> 172.21.15.143:0/2017168964 conn(0x7fd2000b1c70 msgr2=0x7fd20015f650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:36.270 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.266+0000 7fd20711e700 1 -- 172.21.15.143:0/2017168964 shutdown_connections 2023-11-08T19:25:36.270 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.267+0000 7fd20711e700 1 -- 172.21.15.143:0/2017168964 wait complete. 2023-11-08T19:25:36.280 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-08T19:25:36.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.402+0000 7ff91d5ff700 1 Processor -- start 2023-11-08T19:25:36.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.402+0000 7ff91d5ff700 1 -- start start 2023-11-08T19:25:36.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.403+0000 7ff91d5ff700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff9181610b0 0x7ff9180cf560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:36.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.403+0000 7ff91d5ff700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7ff9180cc120 0x7ff9181622a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:36.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.403+0000 7ff91d5ff700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9181627e0 con 0x7ff9180ce3a0 2023-11-08T19:25:36.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.403+0000 7ff91d5ff700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9180d0350 con 0x7ff9181610b0 2023-11-08T19:25:36.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.403+0000 7ff91d5ff700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9180cfb40 con 0x7ff9180cc120 2023-11-08T19:25:36.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.403+0000 7ff916ffd700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff9181610b0 0x7ff9180cf560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:36.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.403+0000 7ff9177fe700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7ff9180ce3a0 0x7ff9180b3410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:46328/0 (socket says 172.21.15.143:46328) 2023-11-08T19:25:36.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.403+0000 7ff9177fe700 1 -- 172.21.15.143:0/3678400575 learned_addr learned my addr 172.21.15.143:0/3678400575 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:25:36.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.403+0000 7ff9167fc700 1 --2- 172.21.15.143:0/3678400575 >> v2:172.21.15.174:3300/0 conn(0x7ff9180cc120 0x7ff9181622a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:36.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.403+0000 7ff916ffd700 1 -- 172.21.15.143:0/3678400575 >> v2:172.21.15.174:3300/0 conn(0x7ff9180cc120 msgr2=0x7ff9181622a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:36.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.403+0000 7ff916ffd700 1 --2- 172.21.15.143:0/3678400575 >> v2:172.21.15.174:3300/0 conn(0x7ff9180cc120 0x7ff9181622a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:36.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.403+0000 7ff915ffb700 1 -- 172.21.15.143:0/3678400575 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3156162692 0 0) 0x7ff910001490 con 0x7ff9180ce3a0 2023-11-08T19:25:36.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.403+0000 7ff916ffd700 1 -- 172.21.15.143:0/3678400575 >> v1:172.21.15.143:6789/0 conn(0x7ff9180ce3a0 legacy=0x7ff9180b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:36.409 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.403+0000 7ff916ffd700 1 -- 172.21.15.143:0/3678400575 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff91806c3c0 con 0x7ff9181610b0 2023-11-08T19:25:36.409 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.403+0000 7ff9167fc700 1 --2- 172.21.15.143:0/3678400575 >> v2:172.21.15.174:3300/0 conn(0x7ff9180cc120 0x7ff9181622a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:25:36.409 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.403+0000 7ff916ffd700 1 --2- 172.21.15.143:0/3678400575 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff9181610b0 0x7ff9180cf560 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7ff91806c000 tx=0x7ff90c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1505bbec02d26d6f server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:36.409 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.404+0000 7ff915ffb700 1 -- 172.21.15.143:0/3678400575 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff90c011830 con 0x7ff9181610b0 2023-11-08T19:25:36.409 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.404+0000 7ff915ffb700 1 -- 172.21.15.143:0/3678400575 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff90c011990 con 0x7ff9181610b0 2023-11-08T19:25:36.409 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.404+0000 7ff915ffb700 1 -- 172.21.15.143:0/3678400575 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff90c011d20 con 0x7ff9181610b0 2023-11-08T19:25:36.410 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.404+0000 7ff91d5ff700 1 -- 172.21.15.143:0/3678400575 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff9181610b0 msgr2=0x7ff9180cf560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:36.410 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.404+0000 7ff91d5ff700 1 --2- 172.21.15.143:0/3678400575 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff9181610b0 0x7ff9180cf560 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7ff91806c000 tx=0x7ff90c009b20 comp rx=0 tx=0).stop 2023-11-08T19:25:36.410 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.404+0000 7ff91d5ff700 1 -- 172.21.15.143:0/3678400575 shutdown_connections 2023-11-08T19:25:36.410 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.404+0000 7ff91d5ff700 1 --2- 172.21.15.143:0/3678400575 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff9181610b0 0x7ff9180cf560 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:36.410 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.404+0000 7ff91d5ff700 1 --2- 172.21.15.143:0/3678400575 >> v2:172.21.15.174:3300/0 conn(0x7ff9180cc120 0x7ff9181622a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:36.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.404+0000 7ff91d5ff700 1 -- 172.21.15.143:0/3678400575 >> 172.21.15.143:0/3678400575 conn(0x7ff91806f2e0 msgr2=0x7ff91806f6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:36.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.404+0000 7ff91d5ff700 1 -- 172.21.15.143:0/3678400575 shutdown_connections 2023-11-08T19:25:36.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.404+0000 7ff91d5ff700 1 -- 172.21.15.143:0/3678400575 wait complete. 2023-11-08T19:25:36.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.405+0000 7ff91d5ff700 1 Processor -- start 2023-11-08T19:25:36.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.406+0000 7ff91d5ff700 1 -- start start 2023-11-08T19:25:36.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.406+0000 7ff91d5ff700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff9180cc120 0x7ff9181531d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:36.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.406+0000 7ff91d5ff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff9180ce3a0 0x7ff9180b0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:36.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.406+0000 7ff91d5ff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff9181610b0 0x7ff9180c6000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:36.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.406+0000 7ff91d5ff700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9180b1090 con 0x7ff9181610b0 2023-11-08T19:25:36.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.406+0000 7ff91d5ff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9180c6540 con 0x7ff9180ce3a0 2023-11-08T19:25:36.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.406+0000 7ff91d5ff700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9180b1280 con 0x7ff9180cc120 2023-11-08T19:25:36.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.406+0000 7ff9177fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff9181610b0 0x7ff9180c6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:36.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.406+0000 7ff9177fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff9181610b0 0x7ff9180c6000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50180/0 (socket says 172.21.15.143:50180) 2023-11-08T19:25:36.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.406+0000 7ff9177fe700 1 -- 172.21.15.143:0/3319119225 learned_addr learned my addr 172.21.15.143:0/3319119225 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:36.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.407+0000 7ff9167fc700 1 --2- 172.21.15.143:0/3319119225 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff9180ce3a0 0x7ff9180b0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:36.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.407+0000 7ff916ffd700 1 --2- 172.21.15.143:0/3319119225 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff9180cc120 0x7ff9181531d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:36.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.407+0000 7ff9177fe700 1 -- 172.21.15.143:0/3319119225 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff9180cc120 msgr2=0x7ff9181531d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:36.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.407+0000 7ff9177fe700 1 --2- 172.21.15.143:0/3319119225 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff9180cc120 0x7ff9181531d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:36.417 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.407+0000 7ff9177fe700 1 -- 172.21.15.143:0/3319119225 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff9180ce3a0 msgr2=0x7ff9180b0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:36.417 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.407+0000 7ff9177fe700 1 --2- 172.21.15.143:0/3319119225 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff9180ce3a0 0x7ff9180b0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:36.417 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.407+0000 7ff9177fe700 1 -- 172.21.15.143:0/3319119225 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff90c0094a0 con 0x7ff9181610b0 2023-11-08T19:25:36.417 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.407+0000 7ff9167fc700 1 --2- 172.21.15.143:0/3319119225 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff9180ce3a0 0x7ff9180b0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:25:36.417 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.407+0000 7ff916ffd700 1 --2- 172.21.15.143:0/3319119225 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff9180cc120 0x7ff9181531d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:25:36.418 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.407+0000 7ff9177fe700 1 --2- 172.21.15.143:0/3319119225 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff9181610b0 0x7ff9180c6000 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff91000cca0 tx=0x7ff91000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:36.418 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.407+0000 7ff8ff7fe700 1 -- 172.21.15.143:0/3319119225 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff91000ad80 con 0x7ff9181610b0 2023-11-08T19:25:36.418 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.407+0000 7ff8ff7fe700 1 -- 172.21.15.143:0/3319119225 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff91000ce40 con 0x7ff9181610b0 2023-11-08T19:25:36.418 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.407+0000 7ff91d5ff700 1 -- 172.21.15.143:0/3319119225 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9180b1560 con 0x7ff9181610b0 2023-11-08T19:25:36.423 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.407+0000 7ff8ff7fe700 1 -- 172.21.15.143:0/3319119225 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff910007310 con 0x7ff9181610b0 2023-11-08T19:25:36.423 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.407+0000 7ff91d5ff700 1 -- 172.21.15.143:0/3319119225 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9180b1b20 con 0x7ff9181610b0 2023-11-08T19:25:36.424 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.408+0000 7ff91d5ff700 1 -- 172.21.15.143:0/3319119225 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff91804eee0 con 0x7ff9181610b0 2023-11-08T19:25:36.424 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.408+0000 7ff8ff7fe700 1 -- 172.21.15.143:0/3319119225 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff910007470 con 0x7ff9181610b0 2023-11-08T19:25:36.424 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.409+0000 7ff8ff7fe700 1 --2- 172.21.15.143:0/3319119225 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7ff90406af00 0x7ff90406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:36.424 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.409+0000 7ff8ff7fe700 1 -- 172.21.15.143:0/3319119225 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff910089250 con 0x7ff9181610b0 2023-11-08T19:25:36.425 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.409+0000 7ff916ffd700 1 --2- 172.21.15.143:0/3319119225 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7ff90406af00 0x7ff90406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:36.425 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.410+0000 7ff916ffd700 1 --2- 172.21.15.143:0/3319119225 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7ff90406af00 0x7ff90406d3c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff90c0090b0 tx=0x7ff90c008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:36.426 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.413+0000 7ff8ff7fe700 1 -- 172.21.15.143:0/3319119225 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff910054390 con 0x7ff9181610b0 2023-11-08T19:25:36.605 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.603+0000 7ff91d5ff700 1 -- 172.21.15.143:0/3319119225 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff9181627e0 con 0x7ff9181610b0 2023-11-08T19:25:36.605 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.603+0000 7ff8ff7fe700 1 -- 172.21.15.143:0/3319119225 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff9181627e0 con 0x7ff9181610b0 2023-11-08T19:25:36.605 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 64 2023-11-08T19:25:36.607 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.606+0000 7ff91d5ff700 1 -- 172.21.15.143:0/3319119225 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7ff90406af00 msgr2=0x7ff90406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:36.607 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.606+0000 7ff91d5ff700 1 --2- 172.21.15.143:0/3319119225 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7ff90406af00 0x7ff90406d3c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff90c0090b0 tx=0x7ff90c008030 comp rx=0 tx=0).stop 2023-11-08T19:25:36.608 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.606+0000 7ff91d5ff700 1 -- 172.21.15.143:0/3319119225 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff9181610b0 msgr2=0x7ff9180c6000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:36.608 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.606+0000 7ff91d5ff700 1 --2- 172.21.15.143:0/3319119225 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff9181610b0 0x7ff9180c6000 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff91000cca0 tx=0x7ff91000a340 comp rx=0 tx=0).stop 2023-11-08T19:25:36.608 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.606+0000 7ff91d5ff700 1 -- 172.21.15.143:0/3319119225 shutdown_connections 2023-11-08T19:25:36.608 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.606+0000 7ff91d5ff700 1 --2- 172.21.15.143:0/3319119225 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7ff90406af00 0x7ff90406d3c0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:36.609 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.606+0000 7ff91d5ff700 1 --2- 172.21.15.143:0/3319119225 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff9180cc120 0x7ff9181531d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:36.609 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.606+0000 7ff91d5ff700 1 --2- 172.21.15.143:0/3319119225 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff9181610b0 0x7ff9180c6000 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:36.609 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.606+0000 7ff91d5ff700 1 --2- 172.21.15.143:0/3319119225 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff9180ce3a0 0x7ff9180b0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:36.609 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.606+0000 7ff91d5ff700 1 -- 172.21.15.143:0/3319119225 >> 172.21.15.143:0/3319119225 conn(0x7ff91806f2e0 msgr2=0x7ff9180b65c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:36.609 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.606+0000 7ff91d5ff700 1 -- 172.21.15.143:0/3319119225 shutdown_connections 2023-11-08T19:25:36.610 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.606+0000 7ff91d5ff700 1 -- 172.21.15.143:0/3319119225 wait complete. 2023-11-08T19:25:36.620 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:25:36.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.738+0000 7ff798af5700 1 Processor -- start 2023-11-08T19:25:36.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.739+0000 7ff798af5700 1 -- start start 2023-11-08T19:25:36.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.739+0000 7ff798af5700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7ff7941618f0 0x7ff79406ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:36.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.739+0000 7ff798af5700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff794161df0 con 0x7ff79415dd00 2023-11-08T19:25:36.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.739+0000 7ff798af5700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7941620d0 con 0x7ff79406b150 2023-11-08T19:25:36.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.739+0000 7ff798af5700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff79406b600 con 0x7ff7941618f0 2023-11-08T19:25:36.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.740+0000 7ff79259c700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7ff79415dd00 0x7ff7940cca70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:46338/0 (socket says 172.21.15.143:46338) 2023-11-08T19:25:36.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.740+0000 7ff79259c700 1 -- 172.21.15.143:0/1489285451 learned_addr learned my addr 172.21.15.143:0/1489285451 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:25:36.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.740+0000 7ff791d9b700 1 --2- 172.21.15.143:0/1489285451 >> v2:172.21.15.174:3300/0 conn(0x7ff7941618f0 0x7ff79406ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:36.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.740+0000 7ff79159a700 1 -- 172.21.15.143:0/1489285451 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 278711885 0 0) 0x7ff794161df0 con 0x7ff79415dd00 2023-11-08T19:25:36.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.740+0000 7ff79159a700 1 -- 172.21.15.143:0/1489285451 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff77c0035b0 con 0x7ff79415dd00 2023-11-08T19:25:36.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.740+0000 7ff79159a700 1 -- 172.21.15.143:0/1489285451 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1411158319 0 0) 0x7ff77c0035b0 con 0x7ff79415dd00 2023-11-08T19:25:36.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.740+0000 7ff79159a700 1 -- 172.21.15.143:0/1489285451 >> v2:172.21.15.174:3300/0 conn(0x7ff7941618f0 msgr2=0x7ff79406ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:36.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.740+0000 7ff79159a700 1 --2- 172.21.15.143:0/1489285451 >> v2:172.21.15.174:3300/0 conn(0x7ff7941618f0 0x7ff79406ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:36.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.740+0000 7ff79159a700 1 -- 172.21.15.143:0/1489285451 >> v1:172.21.15.174:6789/0 conn(0x7ff79406b150 legacy=0x7ff7940cb1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:36.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.740+0000 7ff79159a700 1 -- 172.21.15.143:0/1489285451 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7941623b0 con 0x7ff79415dd00 2023-11-08T19:25:36.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.740+0000 7ff79159a700 1 -- 172.21.15.143:0/1489285451 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4147756303 0 0) 0x7ff7880017f0 con 0x7ff79415dd00 2023-11-08T19:25:36.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.740+0000 7ff79159a700 1 -- 172.21.15.143:0/1489285451 >> v1:172.21.15.143:6789/0 conn(0x7ff79415dd00 legacy=0x7ff7940cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:36.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.740+0000 7ff79159a700 1 --2- 172.21.15.143:0/1489285451 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff77c003a00 0x7ff77c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:36.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.740+0000 7ff79159a700 1 --2- 172.21.15.143:0/1489285451 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff77c004400 0x7ff77c0048d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:36.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.740+0000 7ff79159a700 1 --2- 172.21.15.143:0/1489285451 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff77c0050d0 0x7ff77c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:36.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.740+0000 7ff79159a700 1 -- 172.21.15.143:0/1489285451 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff79406b600 con 0x7ff77c004400 2023-11-08T19:25:36.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.740+0000 7ff79159a700 1 -- 172.21.15.143:0/1489285451 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff77c00c030 con 0x7ff77c003a00 2023-11-08T19:25:36.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.740+0000 7ff79159a700 1 -- 172.21.15.143:0/1489285451 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff77c00c1f0 con 0x7ff77c0050d0 2023-11-08T19:25:36.750 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.740+0000 7ff791d9b700 1 --2- 172.21.15.143:0/1489285451 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff77c004400 0x7ff77c0048d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:36.750 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.740+0000 7ff79259c700 1 --2- 172.21.15.143:0/1489285451 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff77c003a00 0x7ff77c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:36.750 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.740+0000 7ff792d9d700 1 --2- 172.21.15.143:0/1489285451 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff77c0050d0 0x7ff77c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:36.751 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.741+0000 7ff791d9b700 1 -- 172.21.15.143:0/1489285451 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff77c0050d0 msgr2=0x7ff77c00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:36.751 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.741+0000 7ff791d9b700 1 --2- 172.21.15.143:0/1489285451 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff77c0050d0 0x7ff77c00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:36.751 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.741+0000 7ff791d9b700 1 -- 172.21.15.143:0/1489285451 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff77c003a00 msgr2=0x7ff77c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:36.751 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.741+0000 7ff791d9b700 1 --2- 172.21.15.143:0/1489285451 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff77c003a00 0x7ff77c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:36.752 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.741+0000 7ff791d9b700 1 -- 172.21.15.143:0/1489285451 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff77c0035b0 con 0x7ff77c004400 2023-11-08T19:25:36.752 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.741+0000 7ff792d9d700 1 -- 172.21.15.143:0/1489285451 reap_dead start 2023-11-08T19:25:36.752 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.741+0000 7ff791d9b700 1 --2- 172.21.15.143:0/1489285451 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff77c004400 0x7ff77c0048d0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ff784009b60 tx=0x7ff77c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a67942172be19799 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:36.752 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.741+0000 7ff79159a700 1 -- 172.21.15.143:0/1489285451 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff78400e6c0 con 0x7ff77c004400 2023-11-08T19:25:36.752 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.741+0000 7ff79159a700 1 -- 172.21.15.143:0/1489285451 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff78400e820 con 0x7ff77c004400 2023-11-08T19:25:36.753 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.741+0000 7ff79159a700 1 -- 172.21.15.143:0/1489285451 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff78400ebb0 con 0x7ff77c004400 2023-11-08T19:25:36.753 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.741+0000 7ff798af5700 1 -- 172.21.15.143:0/1489285451 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff77c004400 msgr2=0x7ff77c0048d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:36.753 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.741+0000 7ff798af5700 1 --2- 172.21.15.143:0/1489285451 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff77c004400 0x7ff77c0048d0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ff784009b60 tx=0x7ff77c005880 comp rx=0 tx=0).stop 2023-11-08T19:25:36.753 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.742+0000 7ff798af5700 1 -- 172.21.15.143:0/1489285451 shutdown_connections 2023-11-08T19:25:36.753 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.742+0000 7ff798af5700 1 --2- 172.21.15.143:0/1489285451 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff77c004400 0x7ff77c0048d0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:36.753 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.742+0000 7ff798af5700 1 -- 172.21.15.143:0/1489285451 >> 172.21.15.143:0/1489285451 conn(0x7ff794099ab0 msgr2=0x7ff7940d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:36.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.742+0000 7ff798af5700 1 -- 172.21.15.143:0/1489285451 shutdown_connections 2023-11-08T19:25:36.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.742+0000 7ff798af5700 1 -- 172.21.15.143:0/1489285451 wait complete. 2023-11-08T19:25:36.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.743+0000 7ff798af5700 1 Processor -- start 2023-11-08T19:25:36.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.743+0000 7ff798af5700 1 -- start start 2023-11-08T19:25:36.757 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.744+0000 7ff798af5700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff77c004400 0x7ff7940b2280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:36.757 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.744+0000 7ff798af5700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff7940b0070 0x7ff7940cfdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:36.758 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.744+0000 7ff798af5700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff7941518a0 0x7ff794151d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:36.758 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.744+0000 7ff798af5700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff794152290 con 0x7ff77c004400 2023-11-08T19:25:36.758 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.744+0000 7ff798af5700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff794153380 con 0x7ff7941518a0 2023-11-08T19:25:36.758 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.744+0000 7ff798af5700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7940b05a0 con 0x7ff7940b0070 2023-11-08T19:25:36.758 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.744+0000 7ff79259c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff77c004400 0x7ff7940b2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:36.759 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.744+0000 7ff79259c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff77c004400 0x7ff7940b2280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50196/0 (socket says 172.21.15.143:50196) 2023-11-08T19:25:36.759 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.744+0000 7ff79259c700 1 -- 172.21.15.143:0/3718522186 learned_addr learned my addr 172.21.15.143:0/3718522186 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:36.759 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.744+0000 7ff791d9b700 1 --2- 172.21.15.143:0/3718522186 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff7940b0070 0x7ff7940cfdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:36.759 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.744+0000 7ff792d9d700 1 --2- 172.21.15.143:0/3718522186 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff7941518a0 0x7ff794151d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:36.759 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.744+0000 7ff79259c700 1 -- 172.21.15.143:0/3718522186 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff7940b0070 msgr2=0x7ff7940cfdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:36.760 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.744+0000 7ff79259c700 1 --2- 172.21.15.143:0/3718522186 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff7940b0070 0x7ff7940cfdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:36.760 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.744+0000 7ff79259c700 1 -- 172.21.15.143:0/3718522186 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff7941518a0 msgr2=0x7ff794151d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:36.760 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.744+0000 7ff79259c700 1 --2- 172.21.15.143:0/3718522186 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff7941518a0 0x7ff794151d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:36.760 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.744+0000 7ff79259c700 1 -- 172.21.15.143:0/3718522186 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff79415e260 con 0x7ff77c004400 2023-11-08T19:25:36.760 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.744+0000 7ff792d9d700 1 --2- 172.21.15.143:0/3718522186 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff7941518a0 0x7ff794151d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:25:36.761 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.744+0000 7ff791d9b700 1 --2- 172.21.15.143:0/3718522186 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff7940b0070 0x7ff7940cfdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:25:36.761 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.744+0000 7ff79259c700 1 --2- 172.21.15.143:0/3718522186 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff77c004400 0x7ff7940b2280 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ff788001920 tx=0x7ff78800fc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:36.761 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.744+0000 7ff782ffd700 1 -- 172.21.15.143:0/3718522186 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff78800fe80 con 0x7ff77c004400 2023-11-08T19:25:36.761 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.744+0000 7ff782ffd700 1 -- 172.21.15.143:0/3718522186 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7880072e0 con 0x7ff77c004400 2023-11-08T19:25:36.761 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.744+0000 7ff798af5700 1 -- 172.21.15.143:0/3718522186 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7940b0880 con 0x7ff77c004400 2023-11-08T19:25:36.762 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.744+0000 7ff782ffd700 1 -- 172.21.15.143:0/3718522186 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff788007670 con 0x7ff77c004400 2023-11-08T19:25:36.762 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.744+0000 7ff798af5700 1 -- 172.21.15.143:0/3718522186 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7940b0e40 con 0x7ff77c004400 2023-11-08T19:25:36.762 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.745+0000 7ff798af5700 1 -- 172.21.15.143:0/3718522186 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7941531c0 con 0x7ff77c004400 2023-11-08T19:25:36.762 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.746+0000 7ff782ffd700 1 -- 172.21.15.143:0/3718522186 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff788005830 con 0x7ff77c004400 2023-11-08T19:25:36.762 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.746+0000 7ff782ffd700 1 --2- 172.21.15.143:0/3718522186 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7ff77406af00 0x7ff77406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:36.763 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.746+0000 7ff782ffd700 1 -- 172.21.15.143:0/3718522186 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff788089600 con 0x7ff77c004400 2023-11-08T19:25:36.763 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.747+0000 7ff791d9b700 1 --2- 172.21.15.143:0/3718522186 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7ff77406af00 0x7ff77406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:36.763 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.747+0000 7ff791d9b700 1 --2- 172.21.15.143:0/3718522186 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7ff77406af00 0x7ff77406d3c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff7840090b0 tx=0x7ff784008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:36.763 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.751+0000 7ff782ffd700 1 -- 172.21.15.143:0/3718522186 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff7941531c0 con 0x7ff77c004400 2023-11-08T19:25:36.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.984+0000 7ff798af5700 1 -- 172.21.15.143:0/3718522186 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff7941531c0 con 0x7ff77c004400 2023-11-08T19:25:36.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.984+0000 7ff782ffd700 1 -- 172.21.15.143:0/3718522186 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4913 (secure 0 0 0) 0x7ff7941531c0 con 0x7ff77c004400 2023-11-08T19:25:36.986 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:25:36.986 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4208,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6834/37280614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":37280614},{"type":"v1","addr":"172.21.15.143:6834","nonce":37280614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/2364281303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":2364281303},{"type":"v1","addr":"172.21.15.143:6835","nonce":2364281303}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6836/4066129908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":4066129908},{"type":"v1","addr":"172.21.15.174:6836","nonce":4066129908}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T19:24:49.995547+0000","modified":"2023-11-08T19:24:53.027089+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.174:6837/2124972405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6835","nonce":2124972405},{"type":"v1","addr":"172.21.15.174:6837","nonce":2124972405}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-08T19:25:36.988 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.986+0000 7ff798af5700 1 -- 172.21.15.143:0/3718522186 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7ff77406af00 msgr2=0x7ff77406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:36.988 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.986+0000 7ff798af5700 1 --2- 172.21.15.143:0/3718522186 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7ff77406af00 0x7ff77406d3c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff7840090b0 tx=0x7ff784008030 comp rx=0 tx=0).stop 2023-11-08T19:25:36.988 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.986+0000 7ff798af5700 1 -- 172.21.15.143:0/3718522186 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff77c004400 msgr2=0x7ff7940b2280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:36.988 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.986+0000 7ff798af5700 1 --2- 172.21.15.143:0/3718522186 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff77c004400 0x7ff7940b2280 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ff788001920 tx=0x7ff78800fc40 comp rx=0 tx=0).stop 2023-11-08T19:25:36.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.986+0000 7ff798af5700 1 -- 172.21.15.143:0/3718522186 shutdown_connections 2023-11-08T19:25:36.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.986+0000 7ff798af5700 1 --2- 172.21.15.143:0/3718522186 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7ff77406af00 0x7ff77406d3c0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:36.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.986+0000 7ff798af5700 1 --2- 172.21.15.143:0/3718522186 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff7941518a0 0x7ff794151d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:36.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.986+0000 7ff798af5700 1 --2- 172.21.15.143:0/3718522186 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff77c004400 0x7ff7940b2280 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:36.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.986+0000 7ff798af5700 1 --2- 172.21.15.143:0/3718522186 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff7940b0070 0x7ff7940cfdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:36.990 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.986+0000 7ff798af5700 1 -- 172.21.15.143:0/3718522186 >> 172.21.15.143:0/3718522186 conn(0x7ff794099ab0 msgr2=0x7ff79415c030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:36.990 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.986+0000 7ff798af5700 1 -- 172.21.15.143:0/3718522186 shutdown_connections 2023-11-08T19:25:36.990 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:36.986+0000 7ff798af5700 1 -- 172.21.15.143:0/3718522186 wait complete. 2023-11-08T19:25:36.990 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 8 2023-11-08T19:25:37.000 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:25:37.120 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.119+0000 7f0ef4e5b700 1 Processor -- start 2023-11-08T19:25:37.120 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.119+0000 7f0ef4e5b700 1 -- start start 2023-11-08T19:25:37.124 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.120+0000 7f0ef4e5b700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0ef00cc530 0x7f0ef00cf510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:37.125 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.120+0000 7f0ef4e5b700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ef00cf080 con 0x7f0ef00b2920 2023-11-08T19:25:37.125 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.120+0000 7f0ef4e5b700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ef00ce470 con 0x7f0ef015aa30 2023-11-08T19:25:37.125 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.120+0000 7f0ef4e5b700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ef00ce750 con 0x7f0ef00cc530 2023-11-08T19:25:37.125 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.120+0000 7f0eedd9b700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f0ef00b2920 0x7f0ef00cbe20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:46358/0 (socket says 172.21.15.143:46358) 2023-11-08T19:25:37.125 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.120+0000 7f0eedd9b700 1 -- 172.21.15.143:0/1880372631 learned_addr learned my addr 172.21.15.143:0/1880372631 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:25:37.126 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.120+0000 7f0eeed9d700 1 --2- 172.21.15.143:0/1880372631 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0ef00cc530 0x7f0ef00cf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:37.126 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.120+0000 7f0eed59a700 1 -- 172.21.15.143:0/1880372631 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 101247469 0 0) 0x7f0ef00cf080 con 0x7f0ef00b2920 2023-11-08T19:25:37.126 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.120+0000 7f0eed59a700 1 -- 172.21.15.143:0/1880372631 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ed80035b0 con 0x7f0ef00b2920 2023-11-08T19:25:37.126 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.120+0000 7f0eed59a700 1 -- 172.21.15.143:0/1880372631 <== mon.1 v1:172.21.15.174:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3566623017 0 0) 0x7f0ef00ce470 con 0x7f0ef015aa30 2023-11-08T19:25:37.126 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.120+0000 7f0eed59a700 1 -- 172.21.15.143:0/1880372631 --> v1:172.21.15.174:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ef00cf080 con 0x7f0ef015aa30 2023-11-08T19:25:37.127 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.120+0000 7f0eed59a700 1 -- 172.21.15.143:0/1880372631 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2204175895 0 0) 0x7f0ed80035b0 con 0x7f0ef00b2920 2023-11-08T19:25:37.127 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.120+0000 7f0eed59a700 1 -- 172.21.15.143:0/1880372631 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0ef00cc530 msgr2=0x7f0ef00cf510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:37.127 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.120+0000 7f0eed59a700 1 --2- 172.21.15.143:0/1880372631 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0ef00cc530 0x7f0ef00cf510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:37.127 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.120+0000 7f0eed59a700 1 -- 172.21.15.143:0/1880372631 >> v1:172.21.15.174:6789/0 conn(0x7f0ef015aa30 legacy=0x7f0ef00b2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:37.127 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.120+0000 7f0eed59a700 1 -- 172.21.15.143:0/1880372631 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ef00cea30 con 0x7f0ef00b2920 2023-11-08T19:25:37.127 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.120+0000 7f0eed59a700 1 -- 172.21.15.143:0/1880372631 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4147756303 0 0) 0x7f0ee4001670 con 0x7f0ef00b2920 2023-11-08T19:25:37.128 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.120+0000 7f0eed59a700 1 -- 172.21.15.143:0/1880372631 >> v1:172.21.15.143:6789/0 conn(0x7f0ef00b2920 legacy=0x7f0ef00cbe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:37.128 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.120+0000 7f0eed59a700 1 --2- 172.21.15.143:0/1880372631 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0ef00cc530 0x7f0ed8003bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:37.128 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.120+0000 7f0eed59a700 1 --2- 172.21.15.143:0/1880372631 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0ed8004120 0x7f0ed8004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:37.128 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.120+0000 7f0eed59a700 1 --2- 172.21.15.143:0/1880372631 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0ed8004d90 0x7f0ed8005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:37.128 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.120+0000 7f0eed59a700 1 -- 172.21.15.143:0/1880372631 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ef00ce750 con 0x7f0ed8004120 2023-11-08T19:25:37.129 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.121+0000 7f0eed59a700 1 -- 172.21.15.143:0/1880372631 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed800baf0 con 0x7f0ed8004d90 2023-11-08T19:25:37.129 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.121+0000 7f0eed59a700 1 -- 172.21.15.143:0/1880372631 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed800bcb0 con 0x7f0ef00cc530 2023-11-08T19:25:37.129 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.121+0000 7f0eedd9b700 1 --2- 172.21.15.143:0/1880372631 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0ed8004120 0x7f0ed8004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:37.129 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.121+0000 7f0eee59c700 1 --2- 172.21.15.143:0/1880372631 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0ef00cc530 0x7f0ed8003bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:37.129 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.121+0000 7f0eeed9d700 1 --2- 172.21.15.143:0/1880372631 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0ed8004d90 0x7f0ed8005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:37.130 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.121+0000 7f0eedd9b700 1 -- 172.21.15.143:0/1880372631 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0ef00cc530 msgr2=0x7f0ed8003bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:37.130 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.121+0000 7f0eedd9b700 1 --2- 172.21.15.143:0/1880372631 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0ef00cc530 0x7f0ed8003bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:37.130 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.121+0000 7f0eedd9b700 1 -- 172.21.15.143:0/1880372631 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0ed8004d90 msgr2=0x7f0ed8005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:37.130 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.121+0000 7f0eedd9b700 1 --2- 172.21.15.143:0/1880372631 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0ed8004d90 0x7f0ed8005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:37.130 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.121+0000 7f0eedd9b700 1 -- 172.21.15.143:0/1880372631 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ed80035b0 con 0x7f0ed8004120 2023-11-08T19:25:37.131 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.121+0000 7f0eeed9d700 1 --2- 172.21.15.143:0/1880372631 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0ed8004d90 0x7f0ed8005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:25:37.131 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.121+0000 7f0eedd9b700 1 --2- 172.21.15.143:0/1880372631 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0ed8004120 0x7f0ed8004850 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f0ee400c210 tx=0x7f0ed8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76023abe40b637dc server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:37.131 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.121+0000 7f0eed59a700 1 -- 172.21.15.143:0/1880372631 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ee400cd50 con 0x7f0ed8004120 2023-11-08T19:25:37.131 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.121+0000 7f0eed59a700 1 -- 172.21.15.143:0/1880372631 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ee4001af0 con 0x7f0ed8004120 2023-11-08T19:25:37.131 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.121+0000 7f0eed59a700 1 -- 172.21.15.143:0/1880372631 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ee4005ba0 con 0x7f0ed8004120 2023-11-08T19:25:37.132 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.122+0000 7f0ef4e5b700 1 -- 172.21.15.143:0/1880372631 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0ed8004120 msgr2=0x7f0ed8004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:37.132 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.122+0000 7f0ef4e5b700 1 --2- 172.21.15.143:0/1880372631 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0ed8004120 0x7f0ed8004850 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f0ee400c210 tx=0x7f0ed8005880 comp rx=0 tx=0).stop 2023-11-08T19:25:37.132 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.122+0000 7f0eeed9d700 1 -- 172.21.15.143:0/1880372631 reap_dead start 2023-11-08T19:25:37.132 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.122+0000 7f0ef4e5b700 1 -- 172.21.15.143:0/1880372631 shutdown_connections 2023-11-08T19:25:37.132 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.122+0000 7f0ef4e5b700 1 -- 172.21.15.143:0/1880372631 >> 172.21.15.143:0/1880372631 conn(0x7f0ef0099ab0 msgr2=0x7f0ef00d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:37.132 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.122+0000 7f0ef4e5b700 1 -- 172.21.15.143:0/1880372631 shutdown_connections 2023-11-08T19:25:37.133 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.122+0000 7f0ef4e5b700 1 -- 172.21.15.143:0/1880372631 wait complete. 2023-11-08T19:25:37.133 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.123+0000 7f0ef4e5b700 1 Processor -- start 2023-11-08T19:25:37.133 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.123+0000 7f0ef4e5b700 1 -- start start 2023-11-08T19:25:37.135 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.124+0000 7f0ef4e5b700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0ed8004120 0x7f0ef00c66e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:37.136 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.124+0000 7f0ef4e5b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0ef015c0c0 0x7f0ef015d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:37.136 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.124+0000 7f0ef4e5b700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0ef015dcc0 0x7f0ef00b4020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:37.136 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.124+0000 7f0ef4e5b700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ef01f4960 con 0x7f0ed8004120 2023-11-08T19:25:37.136 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.124+0000 7f0ef4e5b700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ef01f3d20 con 0x7f0ef015c0c0 2023-11-08T19:25:37.136 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.124+0000 7f0ef4e5b700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ef01f3fc0 con 0x7f0ef015dcc0 2023-11-08T19:25:37.137 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.124+0000 7f0eee59c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0ed8004120 0x7f0ef00c66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:37.137 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.124+0000 7f0eee59c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0ed8004120 0x7f0ef00c66e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50216/0 (socket says 172.21.15.143:50216) 2023-11-08T19:25:37.137 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.124+0000 7f0eee59c700 1 -- 172.21.15.143:0/2394450311 learned_addr learned my addr 172.21.15.143:0/2394450311 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:37.137 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.124+0000 7f0eedd9b700 1 --2- 172.21.15.143:0/2394450311 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0ef015c0c0 0x7f0ef015d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:37.137 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.124+0000 7f0eeed9d700 1 --2- 172.21.15.143:0/2394450311 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0ef015dcc0 0x7f0ef00b4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:37.138 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.124+0000 7f0eee59c700 1 -- 172.21.15.143:0/2394450311 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0ef015dcc0 msgr2=0x7f0ef00b4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:37.138 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.124+0000 7f0eee59c700 1 --2- 172.21.15.143:0/2394450311 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0ef015dcc0 0x7f0ef00b4020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:37.138 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.124+0000 7f0eee59c700 1 -- 172.21.15.143:0/2394450311 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0ef015c0c0 msgr2=0x7f0ef015d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:37.138 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.124+0000 7f0eee59c700 1 --2- 172.21.15.143:0/2394450311 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0ef015c0c0 0x7f0ef015d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:37.138 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.124+0000 7f0eee59c700 1 -- 172.21.15.143:0/2394450311 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ef01f42a0 con 0x7f0ed8004120 2023-11-08T19:25:37.139 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.124+0000 7f0eee59c700 1 --2- 172.21.15.143:0/2394450311 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0ed8004120 0x7f0ef00c66e0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f0ee000f7e0 tx=0x7f0ee000faf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:37.139 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.124+0000 7f0edeffd700 1 -- 172.21.15.143:0/2394450311 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ee00052e0 con 0x7f0ed8004120 2023-11-08T19:25:37.139 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.125+0000 7f0edeffd700 1 -- 172.21.15.143:0/2394450311 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ee0005440 con 0x7f0ed8004120 2023-11-08T19:25:37.139 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.125+0000 7f0edeffd700 1 -- 172.21.15.143:0/2394450311 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ee00055b0 con 0x7f0ed8004120 2023-11-08T19:25:37.139 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.125+0000 7f0ef4e5b700 1 -- 172.21.15.143:0/2394450311 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ef01f4580 con 0x7f0ed8004120 2023-11-08T19:25:37.140 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.125+0000 7f0ef4e5b700 1 -- 172.21.15.143:0/2394450311 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ef00b6240 con 0x7f0ed8004120 2023-11-08T19:25:37.140 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.125+0000 7f0ef4e5b700 1 -- 172.21.15.143:0/2394450311 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ef01f4710 con 0x7f0ed8004120 2023-11-08T19:25:37.140 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.126+0000 7f0edeffd700 1 -- 172.21.15.143:0/2394450311 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0ee0003910 con 0x7f0ed8004120 2023-11-08T19:25:37.140 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.126+0000 7f0edeffd700 1 --2- 172.21.15.143:0/2394450311 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f0ed006af00 0x7f0ed006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:37.140 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.126+0000 7f0edeffd700 1 -- 172.21.15.143:0/2394450311 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0ee00891e0 con 0x7f0ed8004120 2023-11-08T19:25:37.140 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.127+0000 7f0eedd9b700 1 --2- 172.21.15.143:0/2394450311 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f0ed006af00 0x7f0ed006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:37.141 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.127+0000 7f0eedd9b700 1 --2- 172.21.15.143:0/2394450311 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f0ed006af00 0x7f0ed006d3c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f0ee4000c00 tx=0x7f0ee400a000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:37.141 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.131+0000 7f0edeffd700 1 -- 172.21.15.143:0/2394450311 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0ee0057d20 con 0x7f0ed8004120 2023-11-08T19:25:37.324 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.323+0000 7f0ef4e5b700 1 -- 172.21.15.143:0/2394450311 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0ef00b6240 con 0x7f0ed8004120 2023-11-08T19:25:37.325 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.324+0000 7f0edeffd700 1 -- 172.21.15.143:0/2394450311 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16553 (secure 0 0 0) 0x7f0ef00b6240 con 0x7f0ed8004120 2023-11-08T19:25:37.326 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:25:37.326 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":20,"fsid":"968483af-77fc-4068-ad7c-5c2a1a4613d7","created":"2023-11-08T19:24:32.826484+0000","modified":"2023-11-08T19:24:51.618072+0000","last_up_change":"2023-11-08T19:24:43.573880+0000","last_in_change":"2023-11-08T19:24:35.314525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:44.551530+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-08T19:24:48.256781+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-08T19:24:48.954990+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bae98ec1-d94e-403d-a220-a522a53ea6f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6809","nonce":1487830566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6811","nonce":1487830566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6815","nonce":1487830566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6813","nonce":1487830566}]},"public_addr":"172.21.15.143:6809/1487830566","cluster_addr":"172.21.15.143:6811/1487830566","heartbeat_back_addr":"172.21.15.143:6815/1487830566","heartbeat_front_addr":"172.21.15.143:6813/1487830566","state":["exists","up"]},{"osd":1,"uuid":"5cb6538a-1ccb-4322-8633-09f66af0b7e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6801","nonce":787341397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6803","nonce":787341397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6807","nonce":787341397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6805","nonce":787341397}]},"public_addr":"172.21.15.143:6801/787341397","cluster_addr":"172.21.15.143:6803/787341397","heartbeat_back_addr":"172.21.15.143:6807/787341397","heartbeat_front_addr":"172.21.15.143:6805/787341397","state":["exists","up"]},{"osd":2,"uuid":"cdbf522c-60ef-4d60-9653-8dcb529c101a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6822","nonce":2301325597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6826","nonce":2301325597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6831","nonce":2301325597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6829","nonce":2301325597}]},"public_addr":"172.21.15.143:6822/2301325597","cluster_addr":"172.21.15.143:6826/2301325597","heartbeat_back_addr":"172.21.15.143:6831/2301325597","heartbeat_front_addr":"172.21.15.143:6829/2301325597","state":["exists","up"]},{"osd":3,"uuid":"4e2783cf-ebb9-4dd8-8245-bc1d17ce4325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6817","nonce":2021504428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6819","nonce":2021504428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6827","nonce":2021504428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6823","nonce":2021504428}]},"public_addr":"172.21.15.143:6817/2021504428","cluster_addr":"172.21.15.143:6819/2021504428","heartbeat_back_addr":"172.21.15.143:6827/2021504428","heartbeat_front_addr":"172.21.15.143:6823/2021504428","state":["exists","up"]},{"osd":4,"uuid":"691256fa-9b77-49b9-a346-7ae1adf0ca80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6801","nonce":1993380595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6803","nonce":1993380595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6807","nonce":1993380595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6805","nonce":1993380595}]},"public_addr":"172.21.15.174:6801/1993380595","cluster_addr":"172.21.15.174:6803/1993380595","heartbeat_back_addr":"172.21.15.174:6807/1993380595","heartbeat_front_addr":"172.21.15.174:6805/1993380595","state":["exists","up"]},{"osd":5,"uuid":"756fc352-48c9-46ce-b52e-0541fe6f8525","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6809","nonce":1777923303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6811","nonce":1777923303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6815","nonce":1777923303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6813","nonce":1777923303}]},"public_addr":"172.21.15.174:6809/1777923303","cluster_addr":"172.21.15.174:6811/1777923303","heartbeat_back_addr":"172.21.15.174:6815/1777923303","heartbeat_front_addr":"172.21.15.174:6813/1777923303","state":["exists","up"]},{"osd":6,"uuid":"fab2ac2b-ed97-4cde-9825-b480eabf8b8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6825","nonce":2755130316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6827","nonce":2755130316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6831","nonce":2755130316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6829","nonce":2755130316}]},"public_addr":"172.21.15.174:6825/2755130316","cluster_addr":"172.21.15.174:6827/2755130316","heartbeat_back_addr":"172.21.15.174:6831/2755130316","heartbeat_front_addr":"172.21.15.174:6829/2755130316","state":["exists","up"]},{"osd":7,"uuid":"3de9b797-2ab6-4d41-a863-7e7d638fd703","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6817","nonce":2847141411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6819","nonce":2847141411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6823","nonce":2847141411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6821","nonce":2847141411}]},"public_addr":"172.21.15.174:6817/2847141411","cluster_addr":"172.21.15.174:6819/2847141411","heartbeat_back_addr":"172.21.15.174:6823/2847141411","heartbeat_front_addr":"172.21.15.174:6821/2847141411","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.477076+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.496890+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.539305+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.454673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.538881+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.521294+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.520712+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.540404+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:25:37.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.326+0000 7f0ef4e5b700 1 -- 172.21.15.143:0/2394450311 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f0ed006af00 msgr2=0x7f0ed006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:37.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.326+0000 7f0ef4e5b700 1 --2- 172.21.15.143:0/2394450311 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f0ed006af00 0x7f0ed006d3c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f0ee4000c00 tx=0x7f0ee400a000 comp rx=0 tx=0).stop 2023-11-08T19:25:37.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.326+0000 7f0ef4e5b700 1 -- 172.21.15.143:0/2394450311 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0ed8004120 msgr2=0x7f0ef00c66e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:37.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.326+0000 7f0ef4e5b700 1 --2- 172.21.15.143:0/2394450311 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0ed8004120 0x7f0ef00c66e0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f0ee000f7e0 tx=0x7f0ee000faf0 comp rx=0 tx=0).stop 2023-11-08T19:25:37.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.326+0000 7f0ef4e5b700 1 -- 172.21.15.143:0/2394450311 shutdown_connections 2023-11-08T19:25:37.329 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.326+0000 7f0ef4e5b700 1 --2- 172.21.15.143:0/2394450311 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f0ed006af00 0x7f0ed006d3c0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:37.329 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.326+0000 7f0ef4e5b700 1 --2- 172.21.15.143:0/2394450311 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0ef015dcc0 0x7f0ef00b4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:37.329 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.326+0000 7f0ef4e5b700 1 --2- 172.21.15.143:0/2394450311 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0ed8004120 0x7f0ef00c66e0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:37.330 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.326+0000 7f0ef4e5b700 1 --2- 172.21.15.143:0/2394450311 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0ef015c0c0 0x7f0ef015d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:37.330 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.326+0000 7f0ef4e5b700 1 -- 172.21.15.143:0/2394450311 >> 172.21.15.143:0/2394450311 conn(0x7f0ef0099ab0 msgr2=0x7f0ef006f0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:37.330 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.326+0000 7f0ef4e5b700 1 -- 172.21.15.143:0/2394450311 shutdown_connections 2023-11-08T19:25:37.330 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.326+0000 7f0ef4e5b700 1 -- 172.21.15.143:0/2394450311 wait complete. 2023-11-08T19:25:37.340 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-08T19:25:37.340 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:25:37.460 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.459+0000 7fbebf071700 1 Processor -- start 2023-11-08T19:25:37.461 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.459+0000 7fbebf071700 1 -- start start 2023-11-08T19:25:37.467 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.460+0000 7fbebf071700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fbeb815a2d0 0x7fbeb806ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:37.467 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.460+0000 7fbebf071700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbeb80ce500 0x7fbeb80ce9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:37.467 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.460+0000 7fbebf071700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbeb80cb5e0 con 0x7fbeb806c200 2023-11-08T19:25:37.467 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.460+0000 7fbebf071700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbeb80ceef0 con 0x7fbeb815a2d0 2023-11-08T19:25:37.468 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.460+0000 7fbebf071700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb80cb8c0 con 0x7fbeb80ce500 2023-11-08T19:25:37.468 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.460+0000 7fbeb7fff700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fbeb806c200 0x7fbeb80cab70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:46376/0 (socket says 172.21.15.143:46376) 2023-11-08T19:25:37.468 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.460+0000 7fbeb7fff700 1 -- 172.21.15.143:0/65630119 learned_addr learned my addr 172.21.15.143:0/65630119 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:25:37.468 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.460+0000 7fbeb77fe700 1 -- 172.21.15.143:0/65630119 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2410609679 0 0) 0x7fbeb80cb5e0 con 0x7fbeb806c200 2023-11-08T19:25:37.468 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.460+0000 7fbeb77fe700 1 -- 172.21.15.143:0/65630119 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbea80035b0 con 0x7fbeb806c200 2023-11-08T19:25:37.469 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.460+0000 7fbebce0d700 1 --2- 172.21.15.143:0/65630119 >> v2:172.21.15.174:3300/0 conn(0x7fbeb815a2d0 0x7fbeb806ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:37.469 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.460+0000 7fbebd60e700 1 --2- 172.21.15.143:0/65630119 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbeb80ce500 0x7fbeb80ce9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:37.469 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.460+0000 7fbeb77fe700 1 -- 172.21.15.143:0/65630119 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1787894638 0 0) 0x7fbea80035b0 con 0x7fbeb806c200 2023-11-08T19:25:37.469 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.460+0000 7fbeb77fe700 1 -- 172.21.15.143:0/65630119 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbeb80ce500 msgr2=0x7fbeb80ce9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:37.469 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.460+0000 7fbeb77fe700 1 --2- 172.21.15.143:0/65630119 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbeb80ce500 0x7fbeb80ce9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:37.470 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.460+0000 7fbeb77fe700 1 -- 172.21.15.143:0/65630119 >> v2:172.21.15.174:3300/0 conn(0x7fbeb815a2d0 msgr2=0x7fbeb806ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:37.470 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.460+0000 7fbeb77fe700 1 --2- 172.21.15.143:0/65630119 >> v2:172.21.15.174:3300/0 conn(0x7fbeb815a2d0 0x7fbeb806ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:37.470 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.460+0000 7fbeb77fe700 1 -- 172.21.15.143:0/65630119 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbeb80cbba0 con 0x7fbeb806c200 2023-11-08T19:25:37.470 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.460+0000 7fbeb77fe700 1 -- 172.21.15.143:0/65630119 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4147756303 0 0) 0x7fbea4001670 con 0x7fbeb806c200 2023-11-08T19:25:37.470 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.460+0000 7fbebce0d700 1 --2- 172.21.15.143:0/65630119 >> v2:172.21.15.174:3300/0 conn(0x7fbeb815a2d0 0x7fbeb806ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:25:37.471 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.460+0000 7fbebd60e700 1 --2- 172.21.15.143:0/65630119 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbeb80ce500 0x7fbeb80ce9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:25:37.471 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.460+0000 7fbeb77fe700 1 -- 172.21.15.143:0/65630119 >> v1:172.21.15.143:6789/0 conn(0x7fbeb806c200 legacy=0x7fbeb80cab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:37.471 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.460+0000 7fbeb77fe700 1 --2- 172.21.15.143:0/65630119 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbeb80ce500 0x7fbea8003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:37.471 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.460+0000 7fbeb77fe700 1 --2- 172.21.15.143:0/65630119 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbea8003ff0 0x7fbea8004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:37.471 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.460+0000 7fbeb77fe700 1 --2- 172.21.15.143:0/65630119 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbea8004d90 0x7fbea8005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:37.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.460+0000 7fbeb77fe700 1 -- 172.21.15.143:0/65630119 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb80ceef0 con 0x7fbea8004d90 2023-11-08T19:25:37.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.460+0000 7fbeb77fe700 1 -- 172.21.15.143:0/65630119 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb80cb8c0 con 0x7fbea8003ff0 2023-11-08T19:25:37.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.460+0000 7fbeb77fe700 1 -- 172.21.15.143:0/65630119 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbea800bb80 con 0x7fbeb80ce500 2023-11-08T19:25:37.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.460+0000 7fbebd60e700 1 --2- 172.21.15.143:0/65630119 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbea8004d90 0x7fbea8005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:37.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.461+0000 7fbebce0d700 1 --2- 172.21.15.143:0/65630119 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbeb80ce500 0x7fbea8003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:37.473 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.461+0000 7fbeb7fff700 1 --2- 172.21.15.143:0/65630119 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbea8003ff0 0x7fbea8004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:37.473 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.461+0000 7fbebd60e700 1 -- 172.21.15.143:0/65630119 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbeb80ce500 msgr2=0x7fbea8003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:37.473 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.461+0000 7fbebd60e700 1 --2- 172.21.15.143:0/65630119 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbeb80ce500 0x7fbea8003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:37.474 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.461+0000 7fbebd60e700 1 -- 172.21.15.143:0/65630119 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbea8003ff0 msgr2=0x7fbea8004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:37.474 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.461+0000 7fbebd60e700 1 --2- 172.21.15.143:0/65630119 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbea8003ff0 0x7fbea8004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:37.474 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.461+0000 7fbebd60e700 1 -- 172.21.15.143:0/65630119 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbea80035b0 con 0x7fbea8004d90 2023-11-08T19:25:37.474 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.461+0000 7fbebd60e700 1 --2- 172.21.15.143:0/65630119 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbea8004d90 0x7fbea8005210 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fbeb00073e0 tx=0x7fbea8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b25bdeb4919dd14 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:37.474 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.461+0000 7fbeb77fe700 1 -- 172.21.15.143:0/65630119 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeb0007ea0 con 0x7fbea8004d90 2023-11-08T19:25:37.475 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.461+0000 7fbeb77fe700 1 -- 172.21.15.143:0/65630119 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbeb0014400 con 0x7fbea8004d90 2023-11-08T19:25:37.475 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.461+0000 7fbeb77fe700 1 -- 172.21.15.143:0/65630119 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeb0014790 con 0x7fbea8004d90 2023-11-08T19:25:37.475 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.461+0000 7fbebf071700 1 -- 172.21.15.143:0/65630119 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbea8004d90 msgr2=0x7fbea8005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:37.475 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.461+0000 7fbebf071700 1 --2- 172.21.15.143:0/65630119 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbea8004d90 0x7fbea8005210 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fbeb00073e0 tx=0x7fbea8005880 comp rx=0 tx=0).stop 2023-11-08T19:25:37.475 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.461+0000 7fbebd60e700 1 -- 172.21.15.143:0/65630119 reap_dead start 2023-11-08T19:25:37.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.462+0000 7fbebf071700 1 -- 172.21.15.143:0/65630119 shutdown_connections 2023-11-08T19:25:37.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.462+0000 7fbebf071700 1 -- 172.21.15.143:0/65630119 >> 172.21.15.143:0/65630119 conn(0x7fbeb80b1a20 msgr2=0x7fbeb80b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:37.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.462+0000 7fbebf071700 1 -- 172.21.15.143:0/65630119 shutdown_connections 2023-11-08T19:25:37.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.462+0000 7fbebf071700 1 -- 172.21.15.143:0/65630119 wait complete. 2023-11-08T19:25:37.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.463+0000 7fbebf071700 1 Processor -- start 2023-11-08T19:25:37.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.463+0000 7fbebf071700 1 -- start start 2023-11-08T19:25:37.477 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.464+0000 7fbebf071700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbeb8166c30 0x7fbeb8151c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:37.477 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.464+0000 7fbebf071700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbeb8155600 0x7fbeb80b3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:37.477 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.464+0000 7fbebf071700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbeb8071a30 0x7fbeb8071eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:37.477 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.464+0000 7fbebf071700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb80723f0 con 0x7fbeb8155600 2023-11-08T19:25:37.477 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.464+0000 7fbebf071700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb81517e0 con 0x7fbeb8071a30 2023-11-08T19:25:37.478 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.464+0000 7fbebf071700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb80b3f50 con 0x7fbeb8166c30 2023-11-08T19:25:37.478 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.464+0000 7fbeb7fff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbeb8155600 0x7fbeb80b3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:37.478 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.464+0000 7fbeb7fff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbeb8155600 0x7fbeb80b3a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50236/0 (socket says 172.21.15.143:50236) 2023-11-08T19:25:37.478 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.464+0000 7fbeb7fff700 1 -- 172.21.15.143:0/3129072770 learned_addr learned my addr 172.21.15.143:0/3129072770 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:37.478 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.464+0000 7fbebd60e700 1 --2- 172.21.15.143:0/3129072770 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbeb8071a30 0x7fbeb8071eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:37.479 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.464+0000 7fbebce0d700 1 --2- 172.21.15.143:0/3129072770 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbeb8166c30 0x7fbeb8151c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:37.479 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.464+0000 7fbeb7fff700 1 -- 172.21.15.143:0/3129072770 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbeb8166c30 msgr2=0x7fbeb8151c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:37.479 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.464+0000 7fbeb7fff700 1 --2- 172.21.15.143:0/3129072770 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbeb8166c30 0x7fbeb8151c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:37.479 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.464+0000 7fbeb7fff700 1 -- 172.21.15.143:0/3129072770 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbeb8071a30 msgr2=0x7fbeb8071eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:37.479 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.464+0000 7fbeb7fff700 1 --2- 172.21.15.143:0/3129072770 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbeb8071a30 0x7fbeb8071eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:37.479 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.464+0000 7fbeb7fff700 1 -- 172.21.15.143:0/3129072770 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbeb8150eb0 con 0x7fbeb8155600 2023-11-08T19:25:37.480 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.464+0000 7fbebd60e700 1 --2- 172.21.15.143:0/3129072770 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbeb8071a30 0x7fbeb8071eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:25:37.480 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.464+0000 7fbebce0d700 1 --2- 172.21.15.143:0/3129072770 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbeb8166c30 0x7fbeb8151c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:25:37.480 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.464+0000 7fbeb7fff700 1 --2- 172.21.15.143:0/3129072770 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbeb8155600 0x7fbeb80b3a10 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fbea4007fd0 tx=0x7fbea40076a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:37.480 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.464+0000 7fbeb57fa700 1 -- 172.21.15.143:0/3129072770 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea400f840 con 0x7fbeb8155600 2023-11-08T19:25:37.480 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.464+0000 7fbeb57fa700 1 -- 172.21.15.143:0/3129072770 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbea400f9a0 con 0x7fbeb8155600 2023-11-08T19:25:37.481 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.464+0000 7fbebf071700 1 -- 172.21.15.143:0/3129072770 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbeb8151190 con 0x7fbeb8155600 2023-11-08T19:25:37.481 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.464+0000 7fbeb57fa700 1 -- 172.21.15.143:0/3129072770 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea400fd50 con 0x7fbeb8155600 2023-11-08T19:25:37.481 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.464+0000 7fbebf071700 1 -- 172.21.15.143:0/3129072770 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbeb8167010 con 0x7fbeb8155600 2023-11-08T19:25:37.481 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.465+0000 7fbebf071700 1 -- 172.21.15.143:0/3129072770 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbeb804eee0 con 0x7fbeb8155600 2023-11-08T19:25:37.481 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.466+0000 7fbeb57fa700 1 -- 172.21.15.143:0/3129072770 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbea401d460 con 0x7fbeb8155600 2023-11-08T19:25:37.482 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.466+0000 7fbeb57fa700 1 --2- 172.21.15.143:0/3129072770 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fbe9c06af00 0x7fbe9c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:37.483 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.466+0000 7fbeb57fa700 1 -- 172.21.15.143:0/3129072770 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbea4089780 con 0x7fbeb8155600 2023-11-08T19:25:37.483 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.466+0000 7fbebce0d700 1 --2- 172.21.15.143:0/3129072770 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fbe9c06af00 0x7fbe9c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:37.483 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.467+0000 7fbebce0d700 1 --2- 172.21.15.143:0/3129072770 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fbe9c06af00 0x7fbe9c06d3c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fbeac009a90 tx=0x7fbeac008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:37.483 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.470+0000 7fbeb57fa700 1 -- 172.21.15.143:0/3129072770 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbea4054830 con 0x7fbeb8155600 2023-11-08T19:25:37.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.706+0000 7fbebf071700 1 -- 172.21.15.143:0/3129072770 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbeb8151920 con 0x7fbeb8155600 2023-11-08T19:25:37.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.706+0000 7fbeb57fa700 1 -- 172.21.15.143:0/3129072770 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4913 (secure 0 0 0) 0x7fbea4057e50 con 0x7fbeb8155600 2023-11-08T19:25:37.708 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:25:37.708 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4208,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6834/37280614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":37280614},{"type":"v1","addr":"172.21.15.143:6834","nonce":37280614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/2364281303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":2364281303},{"type":"v1","addr":"172.21.15.143:6835","nonce":2364281303}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6836/4066129908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":4066129908},{"type":"v1","addr":"172.21.15.174:6836","nonce":4066129908}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T19:24:49.995547+0000","modified":"2023-11-08T19:24:53.027089+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.174:6837/2124972405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6835","nonce":2124972405},{"type":"v1","addr":"172.21.15.174:6837","nonce":2124972405}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-08T19:25:37.710 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.708+0000 7fbebf071700 1 -- 172.21.15.143:0/3129072770 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fbe9c06af00 msgr2=0x7fbe9c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:37.710 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.708+0000 7fbebf071700 1 --2- 172.21.15.143:0/3129072770 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fbe9c06af00 0x7fbe9c06d3c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fbeac009a90 tx=0x7fbeac008040 comp rx=0 tx=0).stop 2023-11-08T19:25:37.710 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.708+0000 7fbebf071700 1 -- 172.21.15.143:0/3129072770 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbeb8155600 msgr2=0x7fbeb80b3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:37.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.708+0000 7fbebf071700 1 --2- 172.21.15.143:0/3129072770 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbeb8155600 0x7fbeb80b3a10 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fbea4007fd0 tx=0x7fbea40076a0 comp rx=0 tx=0).stop 2023-11-08T19:25:37.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.708+0000 7fbebf071700 1 -- 172.21.15.143:0/3129072770 shutdown_connections 2023-11-08T19:25:37.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.708+0000 7fbebf071700 1 --2- 172.21.15.143:0/3129072770 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fbe9c06af00 0x7fbe9c06d3c0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:37.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.708+0000 7fbebf071700 1 --2- 172.21.15.143:0/3129072770 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbeb8166c30 0x7fbeb8151c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:37.712 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.708+0000 7fbebf071700 1 --2- 172.21.15.143:0/3129072770 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbeb8071a30 0x7fbeb8071eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:37.712 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.708+0000 7fbebf071700 1 --2- 172.21.15.143:0/3129072770 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbeb8155600 0x7fbeb80b3a10 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:37.712 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.708+0000 7fbebf071700 1 -- 172.21.15.143:0/3129072770 >> 172.21.15.143:0/3129072770 conn(0x7fbeb80b1a20 msgr2=0x7fbeb80c4010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:37.712 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.708+0000 7fbebf071700 1 -- 172.21.15.143:0/3129072770 shutdown_connections 2023-11-08T19:25:37.712 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.708+0000 7fbebf071700 1 -- 172.21.15.143:0/3129072770 wait complete. 2023-11-08T19:25:37.713 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 8 2023-11-08T19:25:37.722 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:25:37.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.841+0000 7f7e85d49700 1 Processor -- start 2023-11-08T19:25:37.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.842+0000 7f7e85d49700 1 -- start start 2023-11-08T19:25:37.845 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.842+0000 7f7e85d49700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f7e8015a2d0 0x7f7e8006ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:37.846 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.842+0000 7f7e85d49700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7e800ce500 0x7f7e800ce9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:37.846 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.842+0000 7f7e85d49700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e800cb5e0 con 0x7f7e8006c200 2023-11-08T19:25:37.846 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.842+0000 7f7e85d49700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e800ceef0 con 0x7f7e8015a2d0 2023-11-08T19:25:37.846 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.842+0000 7f7e85d49700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e800cb8c0 con 0x7f7e800ce500 2023-11-08T19:25:37.847 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.843+0000 7f7e7ffff700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7e800ce500 0x7f7e800ce9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:37.847 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.843+0000 7f7e7f7fe700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f7e8015a2d0 0x7f7e8006ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:37.847 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.843+0000 7f7e7f7fe700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f7e8015a2d0 0x7f7e8006ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.143:33740/0 (socket says 172.21.15.143:33740) 2023-11-08T19:25:37.847 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.843+0000 7f7e7ffff700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7e800ce500 0x7f7e800ce9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.143:37884/0 (socket says 172.21.15.143:37884) 2023-11-08T19:25:37.847 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.843+0000 7f7e7effd700 1 --1- >> v1:172.21.15.174:6789/0 conn(0x7f7e8006c200 0x7f7e800cab70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.174:6789/0 says I am v1:172.21.15.143:57332/0 (socket says 172.21.15.143:57332) 2023-11-08T19:25:37.847 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.843+0000 7f7e7f7fe700 1 -- 172.21.15.143:0/3928440610 learned_addr learned my addr 172.21.15.143:0/3928440610 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:37.848 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.843+0000 7f7e7ffff700 1 -- 172.21.15.143:0/3928440610 >> v2:172.21.15.174:3300/0 conn(0x7f7e8015a2d0 msgr2=0x7f7e8006ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:37.848 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.843+0000 7f7e7ffff700 1 --2- 172.21.15.143:0/3928440610 >> v2:172.21.15.174:3300/0 conn(0x7f7e8015a2d0 0x7f7e8006ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:37.848 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.843+0000 7f7e7ffff700 1 -- 172.21.15.143:0/3928440610 >> v1:172.21.15.174:6789/0 conn(0x7f7e8006c200 legacy=0x7f7e800cab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:37.848 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.843+0000 7f7e7ffff700 1 -- 172.21.15.143:0/3928440610 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e800cbba0 con 0x7f7e800ce500 2023-11-08T19:25:37.848 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.843+0000 7f7e7f7fe700 1 --2- 172.21.15.143:0/3928440610 >> v2:172.21.15.174:3300/0 conn(0x7f7e8015a2d0 0x7f7e8006ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T19:25:37.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.843+0000 7f7e7ffff700 1 --2- 172.21.15.143:0/3928440610 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7e800ce500 0x7f7e800ce9b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7e78009d90 tx=0x7f7e7800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce522c57b074fbea server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:37.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.843+0000 7f7e7e7fc700 1 -- 172.21.15.143:0/3928440610 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e7800e070 con 0x7f7e800ce500 2023-11-08T19:25:37.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.843+0000 7f7e7e7fc700 1 -- 172.21.15.143:0/3928440610 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e7800bb20 con 0x7f7e800ce500 2023-11-08T19:25:37.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.843+0000 7f7e7e7fc700 1 -- 172.21.15.143:0/3928440610 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e78012430 con 0x7f7e800ce500 2023-11-08T19:25:37.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.844+0000 7f7e85d49700 1 -- 172.21.15.143:0/3928440610 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7e800ce500 msgr2=0x7f7e800ce9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:37.850 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.844+0000 7f7e85d49700 1 --2- 172.21.15.143:0/3928440610 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7e800ce500 0x7f7e800ce9b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7e78009d90 tx=0x7f7e7800b330 comp rx=0 tx=0).stop 2023-11-08T19:25:37.850 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.844+0000 7f7e85d49700 1 -- 172.21.15.143:0/3928440610 shutdown_connections 2023-11-08T19:25:37.850 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.844+0000 7f7e85d49700 1 --2- 172.21.15.143:0/3928440610 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7e800ce500 0x7f7e800ce9b0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:37.850 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.844+0000 7f7e85d49700 1 --2- 172.21.15.143:0/3928440610 >> v2:172.21.15.174:3300/0 conn(0x7f7e8015a2d0 0x7f7e8006ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:37.851 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.844+0000 7f7e85d49700 1 -- 172.21.15.143:0/3928440610 >> 172.21.15.143:0/3928440610 conn(0x7f7e800b1a20 msgr2=0x7f7e800b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:37.851 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.844+0000 7f7e85d49700 1 -- 172.21.15.143:0/3928440610 shutdown_connections 2023-11-08T19:25:37.851 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.844+0000 7f7e85d49700 1 -- 172.21.15.143:0/3928440610 wait complete. 2023-11-08T19:25:37.854 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.845+0000 7f7e85d49700 1 Processor -- start 2023-11-08T19:25:37.854 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.845+0000 7f7e85d49700 1 -- start start 2023-11-08T19:25:37.854 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.846+0000 7f7e85d49700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7e8006c200 0x7f7e800c80e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:37.854 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.846+0000 7f7e85d49700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7e800ce500 0x7f7e80157780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:37.854 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.846+0000 7f7e85d49700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7e8015a2d0 0x7f7e80157cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:37.855 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.846+0000 7f7e85d49700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e80153950 con 0x7f7e8015a2d0 2023-11-08T19:25:37.855 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.846+0000 7f7e85d49700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e801572f0 con 0x7f7e8006c200 2023-11-08T19:25:37.855 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.846+0000 7f7e85d49700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e80158200 con 0x7f7e800ce500 2023-11-08T19:25:37.856 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.846+0000 7f7e7ffff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7e8015a2d0 0x7f7e80157cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:37.856 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.846+0000 7f7e7ffff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7e8015a2d0 0x7f7e80157cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50252/0 (socket says 172.21.15.143:50252) 2023-11-08T19:25:37.856 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.846+0000 7f7e7ffff700 1 -- 172.21.15.143:0/1093118651 learned_addr learned my addr 172.21.15.143:0/1093118651 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:37.856 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.846+0000 7f7e7f7fe700 1 --2- 172.21.15.143:0/1093118651 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7e8006c200 0x7f7e800c80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:37.856 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.846+0000 7f7e7effd700 1 --2- 172.21.15.143:0/1093118651 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7e800ce500 0x7f7e80157780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:37.856 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.847+0000 7f7e7ffff700 1 -- 172.21.15.143:0/1093118651 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7e800ce500 msgr2=0x7f7e80157780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:37.857 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.847+0000 7f7e7ffff700 1 --2- 172.21.15.143:0/1093118651 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7e800ce500 0x7f7e80157780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:37.857 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.847+0000 7f7e7ffff700 1 -- 172.21.15.143:0/1093118651 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7e8006c200 msgr2=0x7f7e800c80e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:37.857 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.847+0000 7f7e7ffff700 1 --2- 172.21.15.143:0/1093118651 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7e8006c200 0x7f7e800c80e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:37.857 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.847+0000 7f7e7ffff700 1 -- 172.21.15.143:0/1093118651 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e7000b7f0 con 0x7f7e8015a2d0 2023-11-08T19:25:37.857 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.847+0000 7f7e7effd700 1 --2- 172.21.15.143:0/1093118651 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7e800ce500 0x7f7e80157780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:25:37.858 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.847+0000 7f7e7ffff700 1 --2- 172.21.15.143:0/1093118651 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7e8015a2d0 0x7f7e80157cc0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f7e800cba00 tx=0x7f7e78004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:37.858 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.847+0000 7f7e77fff700 1 -- 172.21.15.143:0/1093118651 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e78012430 con 0x7f7e8015a2d0 2023-11-08T19:25:37.858 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.847+0000 7f7e77fff700 1 -- 172.21.15.143:0/1093118651 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e78020070 con 0x7f7e8015a2d0 2023-11-08T19:25:37.858 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.847+0000 7f7e85d49700 1 -- 172.21.15.143:0/1093118651 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e78009a70 con 0x7f7e8015a2d0 2023-11-08T19:25:37.858 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.847+0000 7f7e77fff700 1 -- 172.21.15.143:0/1093118651 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e78003ea0 con 0x7f7e8015a2d0 2023-11-08T19:25:37.859 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.847+0000 7f7e85d49700 1 -- 172.21.15.143:0/1093118651 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e80156ca0 con 0x7f7e8015a2d0 2023-11-08T19:25:37.859 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.848+0000 7f7e85d49700 1 -- 172.21.15.143:0/1093118651 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e801574a0 con 0x7f7e8015a2d0 2023-11-08T19:25:37.859 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.848+0000 7f7e77fff700 1 -- 172.21.15.143:0/1093118651 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7e78004280 con 0x7f7e8015a2d0 2023-11-08T19:25:37.859 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.848+0000 7f7e77fff700 1 --2- 172.21.15.143:0/1093118651 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f7e6006af00 0x7f7e6006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:37.859 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.849+0000 7f7e77fff700 1 -- 172.21.15.143:0/1093118651 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7e78022070 con 0x7f7e8015a2d0 2023-11-08T19:25:37.860 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.849+0000 7f7e7f7fe700 1 --2- 172.21.15.143:0/1093118651 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f7e6006af00 0x7f7e6006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:37.860 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.849+0000 7f7e7f7fe700 1 --2- 172.21.15.143:0/1093118651 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f7e6006af00 0x7f7e6006d3c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7e70006d30 tx=0x7f7e7000b4d0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:37.860 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:37.853+0000 7f7e77fff700 1 -- 172.21.15.143:0/1093118651 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7e801574a0 con 0x7f7e8015a2d0 2023-11-08T19:25:38.087 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.086+0000 7f7e85d49700 1 -- 172.21.15.143:0/1093118651 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7e801574a0 con 0x7f7e8015a2d0 2023-11-08T19:25:38.088 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.086+0000 7f7e77fff700 1 -- 172.21.15.143:0/1093118651 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4913 (secure 0 0 0) 0x7f7e801574a0 con 0x7f7e8015a2d0 2023-11-08T19:25:38.088 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:25:38.088 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4208,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6834/37280614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":37280614},{"type":"v1","addr":"172.21.15.143:6834","nonce":37280614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/2364281303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":2364281303},{"type":"v1","addr":"172.21.15.143:6835","nonce":2364281303}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6836/4066129908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":4066129908},{"type":"v1","addr":"172.21.15.174:6836","nonce":4066129908}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T19:24:49.995547+0000","modified":"2023-11-08T19:24:53.027089+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.174:6837/2124972405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6835","nonce":2124972405},{"type":"v1","addr":"172.21.15.174:6837","nonce":2124972405}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-08T19:25:38.090 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.088+0000 7f7e85d49700 1 -- 172.21.15.143:0/1093118651 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f7e6006af00 msgr2=0x7f7e6006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:38.090 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.088+0000 7f7e85d49700 1 --2- 172.21.15.143:0/1093118651 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f7e6006af00 0x7f7e6006d3c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7e70006d30 tx=0x7f7e7000b4d0 comp rx=0 tx=0).stop 2023-11-08T19:25:38.090 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.088+0000 7f7e85d49700 1 -- 172.21.15.143:0/1093118651 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7e8015a2d0 msgr2=0x7f7e80157cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:38.091 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.088+0000 7f7e85d49700 1 --2- 172.21.15.143:0/1093118651 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7e8015a2d0 0x7f7e80157cc0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f7e800cba00 tx=0x7f7e78004ad0 comp rx=0 tx=0).stop 2023-11-08T19:25:38.091 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.088+0000 7f7e85d49700 1 -- 172.21.15.143:0/1093118651 shutdown_connections 2023-11-08T19:25:38.091 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.088+0000 7f7e85d49700 1 --2- 172.21.15.143:0/1093118651 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f7e6006af00 0x7f7e6006d3c0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:38.091 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.088+0000 7f7e85d49700 1 --2- 172.21.15.143:0/1093118651 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7e8015a2d0 0x7f7e80157cc0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:38.092 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.088+0000 7f7e85d49700 1 --2- 172.21.15.143:0/1093118651 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7e8006c200 0x7f7e800c80e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:38.092 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.088+0000 7f7e85d49700 1 --2- 172.21.15.143:0/1093118651 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7e800ce500 0x7f7e80157780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:38.092 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.088+0000 7f7e85d49700 1 -- 172.21.15.143:0/1093118651 >> 172.21.15.143:0/1093118651 conn(0x7f7e800b1a20 msgr2=0x7f7e80074130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:38.092 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.088+0000 7f7e85d49700 1 -- 172.21.15.143:0/1093118651 shutdown_connections 2023-11-08T19:25:38.093 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.088+0000 7f7e85d49700 1 -- 172.21.15.143:0/1093118651 wait complete. 2023-11-08T19:25:38.093 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 8 2023-11-08T19:25:38.102 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:25:38.223 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.221+0000 7f5235987700 1 Processor -- start 2023-11-08T19:25:38.223 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.221+0000 7f5235987700 1 -- start start 2023-11-08T19:25:38.225 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.222+0000 7f5235987700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f523015dc80 0x7f52300d1f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:38.226 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.222+0000 7f5235987700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52300d3200 con 0x7f52300cf090 2023-11-08T19:25:38.226 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.222+0000 7f5235987700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f523015e060 con 0x7f52300d3690 2023-11-08T19:25:38.226 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.222+0000 7f5235987700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52300d25f0 con 0x7f523015dc80 2023-11-08T19:25:38.226 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.222+0000 7f522f7fe700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f52300cf090 0x7f52300b6ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:46410/0 (socket says 172.21.15.143:46410) 2023-11-08T19:25:38.227 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.222+0000 7f522f7fe700 1 -- 172.21.15.143:0/3180707538 learned_addr learned my addr 172.21.15.143:0/3180707538 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:25:38.227 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.222+0000 7f522effd700 1 --2- 172.21.15.143:0/3180707538 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f523015dc80 0x7f52300d1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:38.227 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.222+0000 7f522dffb700 1 -- 172.21.15.143:0/3180707538 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 296821425 0 0) 0x7f52300d3200 con 0x7f52300cf090 2023-11-08T19:25:38.227 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.222+0000 7f522dffb700 1 -- 172.21.15.143:0/3180707538 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52180035b0 con 0x7f52300cf090 2023-11-08T19:25:38.227 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.223+0000 7f522dffb700 1 -- 172.21.15.143:0/3180707538 <== mon.1 v1:172.21.15.174:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2197055119 0 0) 0x7f523015e060 con 0x7f52300d3690 2023-11-08T19:25:38.228 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.223+0000 7f522dffb700 1 -- 172.21.15.143:0/3180707538 --> v1:172.21.15.174:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52300d3200 con 0x7f52300d3690 2023-11-08T19:25:38.228 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.223+0000 7f522effd700 1 -- 172.21.15.143:0/3180707538 >> v1:172.21.15.174:6789/0 conn(0x7f52300d3690 legacy=0x7f52300ce980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:38.228 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.223+0000 7f522dffb700 1 -- 172.21.15.143:0/3180707538 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 152644403 0 0) 0x7f52180035b0 con 0x7f52300cf090 2023-11-08T19:25:38.228 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.223+0000 7f522effd700 1 -- 172.21.15.143:0/3180707538 >> v1:172.21.15.143:6789/0 conn(0x7f52300cf090 legacy=0x7f52300b6ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:38.228 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.223+0000 7f522effd700 1 -- 172.21.15.143:0/3180707538 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52300d28d0 con 0x7f523015dc80 2023-11-08T19:25:38.229 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.223+0000 7f522effd700 1 --2- 172.21.15.143:0/3180707538 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f523015dc80 0x7f52300d1f80 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f5224009870 tx=0x7f5224009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bcebf4ccda7c793c server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:38.229 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.223+0000 7f522dffb700 1 -- 172.21.15.143:0/3180707538 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52280019f0 con 0x7f523015dc80 2023-11-08T19:25:38.229 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.223+0000 7f522dffb700 1 -- 172.21.15.143:0/3180707538 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5224004030 con 0x7f523015dc80 2023-11-08T19:25:38.229 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.223+0000 7f522dffb700 1 -- 172.21.15.143:0/3180707538 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52240043c0 con 0x7f523015dc80 2023-11-08T19:25:38.229 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.224+0000 7f5235987700 1 -- 172.21.15.143:0/3180707538 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f523015dc80 msgr2=0x7f52300d1f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:38.230 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.224+0000 7f5235987700 1 --2- 172.21.15.143:0/3180707538 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f523015dc80 0x7f52300d1f80 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f5224009870 tx=0x7f5224009b80 comp rx=0 tx=0).stop 2023-11-08T19:25:38.230 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.224+0000 7f5235987700 1 -- 172.21.15.143:0/3180707538 shutdown_connections 2023-11-08T19:25:38.230 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.224+0000 7f5235987700 1 --2- 172.21.15.143:0/3180707538 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f523015dc80 0x7f52300d1f80 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:38.230 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.224+0000 7f5235987700 1 -- 172.21.15.143:0/3180707538 >> 172.21.15.143:0/3180707538 conn(0x7f52300668e0 msgr2=0x7f5230066cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:38.231 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.224+0000 7f5235987700 1 -- 172.21.15.143:0/3180707538 shutdown_connections 2023-11-08T19:25:38.231 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.224+0000 7f5235987700 1 -- 172.21.15.143:0/3180707538 wait complete. 2023-11-08T19:25:38.234 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.225+0000 7f5235987700 1 Processor -- start 2023-11-08T19:25:38.234 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.225+0000 7f5235987700 1 -- start start 2023-11-08T19:25:38.234 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.226+0000 7f5235987700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f52300cf090 0x7f52301f86a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:38.234 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.226+0000 7f5235987700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f52300d3690 0x7f52300c9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:38.234 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.226+0000 7f5235987700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f523015dc80 0x7f52301faad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:38.235 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.226+0000 7f5235987700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f523015ef20 con 0x7f52300d3690 2023-11-08T19:25:38.235 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.226+0000 7f5235987700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f523015e610 con 0x7f52300cf090 2023-11-08T19:25:38.235 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.226+0000 7f5235987700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f523015e8d0 con 0x7f523015dc80 2023-11-08T19:25:38.235 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.226+0000 7f522e7fc700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f52300d3690 0x7f52300c9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:38.236 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.226+0000 7f522e7fc700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f52300d3690 0x7f52300c9bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50262/0 (socket says 172.21.15.143:50262) 2023-11-08T19:25:38.236 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.226+0000 7f522e7fc700 1 -- 172.21.15.143:0/2129008914 learned_addr learned my addr 172.21.15.143:0/2129008914 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:38.236 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.226+0000 7f522effd700 1 --2- 172.21.15.143:0/2129008914 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f52300cf090 0x7f52301f86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:38.236 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.226+0000 7f522f7fe700 1 --2- 172.21.15.143:0/2129008914 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f523015dc80 0x7f52301faad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:38.236 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.226+0000 7f522e7fc700 1 -- 172.21.15.143:0/2129008914 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f523015dc80 msgr2=0x7f52301faad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:38.237 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.226+0000 7f522e7fc700 1 --2- 172.21.15.143:0/2129008914 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f523015dc80 0x7f52301faad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:38.237 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.226+0000 7f522e7fc700 1 -- 172.21.15.143:0/2129008914 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f52300cf090 msgr2=0x7f52301f86a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:38.237 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.226+0000 7f522e7fc700 1 --2- 172.21.15.143:0/2129008914 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f52300cf090 0x7f52301f86a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:38.237 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.226+0000 7f522e7fc700 1 -- 172.21.15.143:0/2129008914 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52240094a0 con 0x7f52300d3690 2023-11-08T19:25:38.237 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.226+0000 7f522effd700 1 --2- 172.21.15.143:0/2129008914 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f52300cf090 0x7f52301f86a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:25:38.237 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.226+0000 7f522e7fc700 1 --2- 172.21.15.143:0/2129008914 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f52300d3690 0x7f52300c9bc0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f522000c490 tx=0x7f522000c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:38.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.226+0000 7f5217fff700 1 -- 172.21.15.143:0/2129008914 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52200052d0 con 0x7f52300d3690 2023-11-08T19:25:38.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.226+0000 7f5217fff700 1 -- 172.21.15.143:0/2129008914 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5220005450 con 0x7f52300d3690 2023-11-08T19:25:38.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.226+0000 7f5235987700 1 -- 172.21.15.143:0/2129008914 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f523015ebb0 con 0x7f52300d3690 2023-11-08T19:25:38.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.227+0000 7f5217fff700 1 -- 172.21.15.143:0/2129008914 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52200038c0 con 0x7f52300d3690 2023-11-08T19:25:38.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.227+0000 7f5235987700 1 -- 172.21.15.143:0/2129008914 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5230153ea0 con 0x7f52300d3690 2023-11-08T19:25:38.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.228+0000 7f5235987700 1 -- 172.21.15.143:0/2129008914 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f523015f090 con 0x7f52300d3690 2023-11-08T19:25:38.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.228+0000 7f5217fff700 1 -- 172.21.15.143:0/2129008914 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5220003a20 con 0x7f52300d3690 2023-11-08T19:25:38.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.228+0000 7f5217fff700 1 --2- 172.21.15.143:0/2129008914 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f521c06af00 0x7f521c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:38.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.228+0000 7f5217fff700 1 -- 172.21.15.143:0/2129008914 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5220088960 con 0x7f52300d3690 2023-11-08T19:25:38.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.228+0000 7f522effd700 1 --2- 172.21.15.143:0/2129008914 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f521c06af00 0x7f521c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:38.240 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.229+0000 7f522effd700 1 --2- 172.21.15.143:0/2129008914 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f521c06af00 0x7f521c06d3c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f52240097c0 tx=0x7f5224013040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:38.240 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.233+0000 7f5217fff700 1 -- 172.21.15.143:0/2129008914 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5220053aa0 con 0x7f52300d3690 2023-11-08T19:25:38.423 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.422+0000 7f5235987700 1 -- 172.21.15.143:0/2129008914 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f523004eee0 con 0x7f52300d3690 2023-11-08T19:25:38.424 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.423+0000 7f5217fff700 1 -- 172.21.15.143:0/2129008914 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16553 (secure 0 0 0) 0x7f5220017090 con 0x7f52300d3690 2023-11-08T19:25:38.425 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:25:38.425 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":20,"fsid":"968483af-77fc-4068-ad7c-5c2a1a4613d7","created":"2023-11-08T19:24:32.826484+0000","modified":"2023-11-08T19:24:51.618072+0000","last_up_change":"2023-11-08T19:24:43.573880+0000","last_in_change":"2023-11-08T19:24:35.314525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:44.551530+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-08T19:24:48.256781+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-08T19:24:48.954990+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bae98ec1-d94e-403d-a220-a522a53ea6f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6809","nonce":1487830566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6811","nonce":1487830566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6815","nonce":1487830566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6813","nonce":1487830566}]},"public_addr":"172.21.15.143:6809/1487830566","cluster_addr":"172.21.15.143:6811/1487830566","heartbeat_back_addr":"172.21.15.143:6815/1487830566","heartbeat_front_addr":"172.21.15.143:6813/1487830566","state":["exists","up"]},{"osd":1,"uuid":"5cb6538a-1ccb-4322-8633-09f66af0b7e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6801","nonce":787341397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6803","nonce":787341397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6807","nonce":787341397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6805","nonce":787341397}]},"public_addr":"172.21.15.143:6801/787341397","cluster_addr":"172.21.15.143:6803/787341397","heartbeat_back_addr":"172.21.15.143:6807/787341397","heartbeat_front_addr":"172.21.15.143:6805/787341397","state":["exists","up"]},{"osd":2,"uuid":"cdbf522c-60ef-4d60-9653-8dcb529c101a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6822","nonce":2301325597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6826","nonce":2301325597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6831","nonce":2301325597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6829","nonce":2301325597}]},"public_addr":"172.21.15.143:6822/2301325597","cluster_addr":"172.21.15.143:6826/2301325597","heartbeat_back_addr":"172.21.15.143:6831/2301325597","heartbeat_front_addr":"172.21.15.143:6829/2301325597","state":["exists","up"]},{"osd":3,"uuid":"4e2783cf-ebb9-4dd8-8245-bc1d17ce4325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6817","nonce":2021504428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6819","nonce":2021504428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6827","nonce":2021504428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6823","nonce":2021504428}]},"public_addr":"172.21.15.143:6817/2021504428","cluster_addr":"172.21.15.143:6819/2021504428","heartbeat_back_addr":"172.21.15.143:6827/2021504428","heartbeat_front_addr":"172.21.15.143:6823/2021504428","state":["exists","up"]},{"osd":4,"uuid":"691256fa-9b77-49b9-a346-7ae1adf0ca80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6801","nonce":1993380595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6803","nonce":1993380595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6807","nonce":1993380595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6805","nonce":1993380595}]},"public_addr":"172.21.15.174:6801/1993380595","cluster_addr":"172.21.15.174:6803/1993380595","heartbeat_back_addr":"172.21.15.174:6807/1993380595","heartbeat_front_addr":"172.21.15.174:6805/1993380595","state":["exists","up"]},{"osd":5,"uuid":"756fc352-48c9-46ce-b52e-0541fe6f8525","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6809","nonce":1777923303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6811","nonce":1777923303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6815","nonce":1777923303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6813","nonce":1777923303}]},"public_addr":"172.21.15.174:6809/1777923303","cluster_addr":"172.21.15.174:6811/1777923303","heartbeat_back_addr":"172.21.15.174:6815/1777923303","heartbeat_front_addr":"172.21.15.174:6813/1777923303","state":["exists","up"]},{"osd":6,"uuid":"fab2ac2b-ed97-4cde-9825-b480eabf8b8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6825","nonce":2755130316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6827","nonce":2755130316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6831","nonce":2755130316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6829","nonce":2755130316}]},"public_addr":"172.21.15.174:6825/2755130316","cluster_addr":"172.21.15.174:6827/2755130316","heartbeat_back_addr":"172.21.15.174:6831/2755130316","heartbeat_front_addr":"172.21.15.174:6829/2755130316","state":["exists","up"]},{"osd":7,"uuid":"3de9b797-2ab6-4d41-a863-7e7d638fd703","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6817","nonce":2847141411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6819","nonce":2847141411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6823","nonce":2847141411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6821","nonce":2847141411}]},"public_addr":"172.21.15.174:6817/2847141411","cluster_addr":"172.21.15.174:6819/2847141411","heartbeat_back_addr":"172.21.15.174:6823/2847141411","heartbeat_front_addr":"172.21.15.174:6821/2847141411","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.477076+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.496890+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.539305+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.454673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.538881+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.521294+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.520712+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.540404+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:25:38.427 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.425+0000 7f5235987700 1 -- 172.21.15.143:0/2129008914 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f521c06af00 msgr2=0x7f521c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:38.427 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.425+0000 7f5235987700 1 --2- 172.21.15.143:0/2129008914 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f521c06af00 0x7f521c06d3c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f52240097c0 tx=0x7f5224013040 comp rx=0 tx=0).stop 2023-11-08T19:25:38.427 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.425+0000 7f5235987700 1 -- 172.21.15.143:0/2129008914 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f52300d3690 msgr2=0x7f52300c9bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:38.428 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.425+0000 7f5235987700 1 --2- 172.21.15.143:0/2129008914 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f52300d3690 0x7f52300c9bc0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f522000c490 tx=0x7f522000c7a0 comp rx=0 tx=0).stop 2023-11-08T19:25:38.428 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.425+0000 7f5235987700 1 -- 172.21.15.143:0/2129008914 shutdown_connections 2023-11-08T19:25:38.428 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.425+0000 7f5235987700 1 --2- 172.21.15.143:0/2129008914 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f521c06af00 0x7f521c06d3c0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:38.428 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.425+0000 7f5235987700 1 --2- 172.21.15.143:0/2129008914 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f523015dc80 0x7f52301faad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:38.428 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.425+0000 7f5235987700 1 --2- 172.21.15.143:0/2129008914 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f52300cf090 0x7f52301f86a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:38.429 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.425+0000 7f5235987700 1 --2- 172.21.15.143:0/2129008914 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f52300d3690 0x7f52300c9bc0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:38.429 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.425+0000 7f5235987700 1 -- 172.21.15.143:0/2129008914 >> 172.21.15.143:0/2129008914 conn(0x7f52300668e0 msgr2=0x7f523015cfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:38.429 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.426+0000 7f5235987700 1 -- 172.21.15.143:0/2129008914 shutdown_connections 2023-11-08T19:25:38.429 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.426+0000 7f5235987700 1 -- 172.21.15.143:0/2129008914 wait complete. 2023-11-08T19:25:38.439 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-08T19:25:38.561 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.560+0000 7f6fae2d7700 1 Processor -- start 2023-11-08T19:25:38.562 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.560+0000 7f6fae2d7700 1 -- start start 2023-11-08T19:25:38.564 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.561+0000 7f6fae2d7700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f6fa80b2920 0x7f6fa80cf510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:38.564 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.561+0000 7f6fae2d7700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6fa80cbe20 0x7f6fa80cc290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:38.565 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.561+0000 7f6fae2d7700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fa80cf080 con 0x7f6fa815aa30 2023-11-08T19:25:38.565 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.561+0000 7f6fae2d7700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fa80cc7d0 con 0x7f6fa80b2920 2023-11-08T19:25:38.565 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.561+0000 7f6fae2d7700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fa80ce470 con 0x7f6fa80cbe20 2023-11-08T19:25:38.565 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.561+0000 7f6fa7fff700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f6fa815aa30 0x7f6fa80b2210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:46422/0 (socket says 172.21.15.143:46422) 2023-11-08T19:25:38.565 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.561+0000 7f6fa7fff700 1 -- 172.21.15.143:0/3785906186 learned_addr learned my addr 172.21.15.143:0/3785906186 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:25:38.566 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.561+0000 7f6fac874700 1 --2- 172.21.15.143:0/3785906186 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6fa80cbe20 0x7f6fa80cc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:38.566 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.561+0000 7f6fa77fe700 1 --2- 172.21.15.143:0/3785906186 >> v2:172.21.15.174:3300/0 conn(0x7f6fa80b2920 0x7f6fa80cf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:38.566 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.561+0000 7f6fa6ffd700 1 -- 172.21.15.143:0/3785906186 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2268661569 0 0) 0x7f6fa80cf080 con 0x7f6fa815aa30 2023-11-08T19:25:38.566 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.561+0000 7f6fa6ffd700 1 -- 172.21.15.143:0/3785906186 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f900035b0 con 0x7f6fa815aa30 2023-11-08T19:25:38.566 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.561+0000 7f6fac874700 1 -- 172.21.15.143:0/3785906186 >> v2:172.21.15.174:3300/0 conn(0x7f6fa80b2920 msgr2=0x7f6fa80cf510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:38.567 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.561+0000 7f6fac874700 1 --2- 172.21.15.143:0/3785906186 >> v2:172.21.15.174:3300/0 conn(0x7f6fa80b2920 0x7f6fa80cf510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:38.567 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.561+0000 7f6fac874700 1 -- 172.21.15.143:0/3785906186 >> v1:172.21.15.143:6789/0 conn(0x7f6fa815aa30 legacy=0x7f6fa80b2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:38.567 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.561+0000 7f6fac874700 1 -- 172.21.15.143:0/3785906186 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fa80ce750 con 0x7f6fa80cbe20 2023-11-08T19:25:38.567 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.562+0000 7f6fac874700 1 --2- 172.21.15.143:0/3785906186 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6fa80cbe20 0x7f6fa80cc290 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6fa0009d90 tx=0x7f6fa000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76733f8836d4f8b4 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:38.567 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.562+0000 7f6fa6ffd700 1 -- 172.21.15.143:0/3785906186 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa000e070 con 0x7f6fa80cbe20 2023-11-08T19:25:38.568 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.562+0000 7f6fa6ffd700 1 -- 172.21.15.143:0/3785906186 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fa000bb20 con 0x7f6fa80cbe20 2023-11-08T19:25:38.568 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.562+0000 7f6fa6ffd700 1 -- 172.21.15.143:0/3785906186 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa0012430 con 0x7f6fa80cbe20 2023-11-08T19:25:38.568 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.562+0000 7f6fae2d7700 1 -- 172.21.15.143:0/3785906186 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6fa80cbe20 msgr2=0x7f6fa80cc290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:38.568 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.562+0000 7f6fae2d7700 1 --2- 172.21.15.143:0/3785906186 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6fa80cbe20 0x7f6fa80cc290 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6fa0009d90 tx=0x7f6fa000b330 comp rx=0 tx=0).stop 2023-11-08T19:25:38.568 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.562+0000 7f6fae2d7700 1 -- 172.21.15.143:0/3785906186 shutdown_connections 2023-11-08T19:25:38.569 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.562+0000 7f6fae2d7700 1 --2- 172.21.15.143:0/3785906186 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6fa80cbe20 0x7f6fa80cc290 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:38.569 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.562+0000 7f6fae2d7700 1 --2- 172.21.15.143:0/3785906186 >> v2:172.21.15.174:3300/0 conn(0x7f6fa80b2920 0x7f6fa80cf510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:38.569 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.562+0000 7f6fae2d7700 1 -- 172.21.15.143:0/3785906186 >> 172.21.15.143:0/3785906186 conn(0x7f6fa8099ab0 msgr2=0x7f6fa80d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:38.569 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.562+0000 7f6fae2d7700 1 -- 172.21.15.143:0/3785906186 shutdown_connections 2023-11-08T19:25:38.569 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.562+0000 7f6fae2d7700 1 -- 172.21.15.143:0/3785906186 wait complete. 2023-11-08T19:25:38.572 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.564+0000 7f6fae2d7700 1 Processor -- start 2023-11-08T19:25:38.572 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.564+0000 7f6fae2d7700 1 -- start start 2023-11-08T19:25:38.573 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.565+0000 7f6fae2d7700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6fa80b2920 0x7f6fa815da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:38.573 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.565+0000 7f6fae2d7700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6fa80cbe20 0x7f6fa80c69b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:38.573 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.565+0000 7f6fae2d7700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6fa815aa30 0x7f6fa80c6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:38.573 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.565+0000 7f6fae2d7700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fa8152720 con 0x7f6fa815aa30 2023-11-08T19:25:38.573 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.565+0000 7f6fae2d7700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fa8156180 con 0x7f6fa80cbe20 2023-11-08T19:25:38.574 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.565+0000 7f6fae2d7700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fa80b3dc0 con 0x7f6fa80b2920 2023-11-08T19:25:38.574 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.565+0000 7f6fac874700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6fa815aa30 0x7f6fa80c6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:38.574 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.565+0000 7f6fa7fff700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6fa80b2920 0x7f6fa815da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:38.574 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.565+0000 7f6fa77fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6fa80cbe20 0x7f6fa80c69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:38.574 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.565+0000 7f6fac874700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6fa815aa30 0x7f6fa80c6ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50278/0 (socket says 172.21.15.143:50278) 2023-11-08T19:25:38.575 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.565+0000 7f6fac874700 1 -- 172.21.15.143:0/3760947408 learned_addr learned my addr 172.21.15.143:0/3760947408 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:38.575 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.565+0000 7f6fa7fff700 1 -- 172.21.15.143:0/3760947408 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6fa80cbe20 msgr2=0x7f6fa80c69b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:38.575 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.565+0000 7f6fa7fff700 1 --2- 172.21.15.143:0/3760947408 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6fa80cbe20 0x7f6fa80c69b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:38.575 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.565+0000 7f6fa7fff700 1 -- 172.21.15.143:0/3760947408 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6fa815aa30 msgr2=0x7f6fa80c6ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:38.575 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.566+0000 7f6fa7fff700 1 --2- 172.21.15.143:0/3760947408 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6fa815aa30 0x7f6fa80c6ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:38.576 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.566+0000 7f6fa7fff700 1 -- 172.21.15.143:0/3760947408 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fa0009a70 con 0x7f6fa80b2920 2023-11-08T19:25:38.576 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.566+0000 7f6fac874700 1 --2- 172.21.15.143:0/3760947408 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6fa815aa30 0x7f6fa80c6ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T19:25:38.576 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.566+0000 7f6fa7fff700 1 --2- 172.21.15.143:0/3760947408 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6fa80b2920 0x7f6fa815da80 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c001360 tx=0x7f6f9c00c6e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:38.576 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.566+0000 7f6fa4ff9700 1 -- 172.21.15.143:0/3760947408 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f9c0072d0 con 0x7f6fa80b2920 2023-11-08T19:25:38.576 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.566+0000 7f6fa4ff9700 1 -- 172.21.15.143:0/3760947408 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f9c007430 con 0x7f6fa80b2920 2023-11-08T19:25:38.577 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.566+0000 7f6fae2d7700 1 -- 172.21.15.143:0/3760947408 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fa80b4010 con 0x7f6fa80b2920 2023-11-08T19:25:38.577 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.566+0000 7f6fa4ff9700 1 -- 172.21.15.143:0/3760947408 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f9c005830 con 0x7f6fa80b2920 2023-11-08T19:25:38.577 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.566+0000 7f6fae2d7700 1 -- 172.21.15.143:0/3760947408 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fa80b45d0 con 0x7f6fa80b2920 2023-11-08T19:25:38.577 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.567+0000 7f6fae2d7700 1 -- 172.21.15.143:0/3760947408 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fa804eee0 con 0x7f6fa80b2920 2023-11-08T19:25:38.577 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.567+0000 7f6fa4ff9700 1 -- 172.21.15.143:0/3760947408 <== mon.2 v2:172.21.15.174:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6f9c005ba0 con 0x7f6fa80b2920 2023-11-08T19:25:38.578 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.568+0000 7f6fa4ff9700 1 --2- 172.21.15.143:0/3760947408 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f6f9406af00 0x7f6f9406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:38.578 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.568+0000 7f6fa4ff9700 1 -- 172.21.15.143:0/3760947408 <== mon.2 v2:172.21.15.174:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6f9c088ae0 con 0x7f6fa80b2920 2023-11-08T19:25:38.578 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.568+0000 7f6fa77fe700 1 --2- 172.21.15.143:0/3760947408 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f6f9406af00 0x7f6f9406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:38.578 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.568+0000 7f6fa77fe700 1 --2- 172.21.15.143:0/3760947408 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f6f9406af00 0x7f6f9406d3c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6fa81f21c0 tx=0x7f6f98009210 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:38.579 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.573+0000 7f6fa4ff9700 1 -- 172.21.15.143:0/3760947408 <== mon.2 v2:172.21.15.174:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6f9c053ba0 con 0x7f6fa80b2920 2023-11-08T19:25:38.814 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.813+0000 7f6fae2d7700 1 -- 172.21.15.143:0/3760947408 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f6fa80b2ff0 con 0x7f6fa80b2920 2023-11-08T19:25:38.815 INFO:tasks.ceph.mon.a.smithi143.stderr:2023-11-08T19:25:38.814+0000 7f65b584e700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-08T19:25:38.821 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.820+0000 7f6fa4ff9700 1 -- 172.21.15.143:0/3760947408 <== mon.2 v2:172.21.15.174:3301/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) v1 ==== 161+0+0 (secure 0 0 0) 0x7f6f9c007da0 con 0x7f6fa80b2920 2023-11-08T19:25:38.824 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.822+0000 7f6fae2d7700 1 -- 172.21.15.143:0/3760947408 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f6f9406af00 msgr2=0x7f6f9406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:38.824 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.822+0000 7f6fae2d7700 1 --2- 172.21.15.143:0/3760947408 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f6f9406af00 0x7f6f9406d3c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6fa81f21c0 tx=0x7f6f98009210 comp rx=0 tx=0).stop 2023-11-08T19:25:38.824 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.822+0000 7f6fae2d7700 1 -- 172.21.15.143:0/3760947408 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6fa80b2920 msgr2=0x7f6fa815da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:38.825 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.822+0000 7f6fae2d7700 1 --2- 172.21.15.143:0/3760947408 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6fa80b2920 0x7f6fa815da80 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c001360 tx=0x7f6f9c00c6e0 comp rx=0 tx=0).stop 2023-11-08T19:25:38.825 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.822+0000 7f6fae2d7700 1 -- 172.21.15.143:0/3760947408 shutdown_connections 2023-11-08T19:25:38.825 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.822+0000 7f6fae2d7700 1 --2- 172.21.15.143:0/3760947408 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f6f9406af00 0x7f6f9406d3c0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:38.825 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.822+0000 7f6fae2d7700 1 --2- 172.21.15.143:0/3760947408 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6fa80b2920 0x7f6fa815da80 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:38.825 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.822+0000 7f6fae2d7700 1 --2- 172.21.15.143:0/3760947408 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6fa815aa30 0x7f6fa80c6ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:38.826 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.822+0000 7f6fae2d7700 1 --2- 172.21.15.143:0/3760947408 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6fa80cbe20 0x7f6fa80c69b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:38.826 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.822+0000 7f6fae2d7700 1 -- 172.21.15.143:0/3760947408 >> 172.21.15.143:0/3760947408 conn(0x7f6fa8099ab0 msgr2=0x7f6fa806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:38.826 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.822+0000 7f6fae2d7700 1 -- 172.21.15.143:0/3760947408 shutdown_connections 2023-11-08T19:25:38.826 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.822+0000 7f6fae2d7700 1 -- 172.21.15.143:0/3760947408 wait complete. 2023-11-08T19:25:38.827 INFO:teuthology.orchestra.run.smithi143.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-08T19:25:38.836 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-08T19:25:38.860 INFO:tasks.ceph.mds.b.smithi174.stdout:starting mds.b at 2023-11-08T19:25:38.961 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.959+0000 7f22d9231700 1 Processor -- start 2023-11-08T19:25:38.961 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.959+0000 7f22d9231700 1 -- start start 2023-11-08T19:25:38.963 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.960+0000 7f22d9231700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f22d415dba0 0x7f22d406c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:38.964 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.960+0000 7f22d9231700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f22d40b22e0 0x7f22d40b2750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:38.964 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.960+0000 7f22d9231700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22d4161890 con 0x7f22d406b240 2023-11-08T19:25:38.964 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.960+0000 7f22d9231700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22d40b2c90 con 0x7f22d415dba0 2023-11-08T19:25:38.964 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.960+0000 7f22d9231700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22d406b740 con 0x7f22d40b22e0 2023-11-08T19:25:38.964 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.960+0000 7f22d2d9d700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f22d415dba0 0x7f22d406c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:38.965 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.960+0000 7f22d259c700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f22d406b240 0x7f22d40caff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:46438/0 (socket says 172.21.15.143:46438) 2023-11-08T19:25:38.965 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.960+0000 7f22d259c700 1 -- 172.21.15.143:0/3713572214 learned_addr learned my addr 172.21.15.143:0/3713572214 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:25:38.965 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.961+0000 7f22d359e700 1 --2- 172.21.15.143:0/3713572214 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f22d40b22e0 0x7f22d40b2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:38.965 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.961+0000 7f22d2d9d700 1 -- 172.21.15.143:0/3713572214 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f22d40b22e0 msgr2=0x7f22d40b2750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:38.965 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.961+0000 7f22d2d9d700 1 --2- 172.21.15.143:0/3713572214 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f22d40b22e0 0x7f22d40b2750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:38.966 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.961+0000 7f22d2d9d700 1 -- 172.21.15.143:0/3713572214 >> v1:172.21.15.143:6789/0 conn(0x7f22d406b240 legacy=0x7f22d40caff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:38.966 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.961+0000 7f22d1d9b700 1 -- 172.21.15.143:0/3713572214 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3125449781 0 0) 0x7f22d4161890 con 0x7f22d406b240 2023-11-08T19:25:38.966 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.961+0000 7f22d2d9d700 1 -- 172.21.15.143:0/3713572214 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22d406ba20 con 0x7f22d415dba0 2023-11-08T19:25:38.966 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.961+0000 7f22d359e700 1 --2- 172.21.15.143:0/3713572214 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f22d40b22e0 0x7f22d40b2750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:25:38.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.961+0000 7f22d2d9d700 1 --2- 172.21.15.143:0/3713572214 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f22d415dba0 0x7f22d406c610 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f22c4009840 tx=0x7f22c4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87366404eae9ceaf server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:38.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.961+0000 7f22d1d9b700 1 -- 172.21.15.143:0/3713572214 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22c4011830 con 0x7f22d415dba0 2023-11-08T19:25:38.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.961+0000 7f22d1d9b700 1 -- 172.21.15.143:0/3713572214 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22c4011990 con 0x7f22d415dba0 2023-11-08T19:25:38.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.961+0000 7f22d1d9b700 1 -- 172.21.15.143:0/3713572214 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22c4011d20 con 0x7f22d415dba0 2023-11-08T19:25:38.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.962+0000 7f22d9231700 1 -- 172.21.15.143:0/3713572214 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f22d415dba0 msgr2=0x7f22d406c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:38.968 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.962+0000 7f22d9231700 1 --2- 172.21.15.143:0/3713572214 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f22d415dba0 0x7f22d406c610 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f22c4009840 tx=0x7f22c4009b50 comp rx=0 tx=0).stop 2023-11-08T19:25:38.968 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.962+0000 7f22d9231700 1 -- 172.21.15.143:0/3713572214 shutdown_connections 2023-11-08T19:25:38.968 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.962+0000 7f22d9231700 1 --2- 172.21.15.143:0/3713572214 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f22d40b22e0 0x7f22d40b2750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:38.968 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.962+0000 7f22d9231700 1 --2- 172.21.15.143:0/3713572214 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f22d415dba0 0x7f22d406c610 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:38.968 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.962+0000 7f22d9231700 1 -- 172.21.15.143:0/3713572214 >> 172.21.15.143:0/3713572214 conn(0x7f22d40cd2e0 msgr2=0x7f22d40cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:38.969 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.962+0000 7f22d9231700 1 -- 172.21.15.143:0/3713572214 shutdown_connections 2023-11-08T19:25:38.969 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.962+0000 7f22d9231700 1 -- 172.21.15.143:0/3713572214 wait complete. 2023-11-08T19:25:38.971 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.963+0000 7f22d9231700 1 Processor -- start 2023-11-08T19:25:38.972 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.963+0000 7f22d9231700 1 -- start start 2023-11-08T19:25:38.972 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.964+0000 7f22d9231700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f22d406b240 0x7f22d4155fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:38.972 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.964+0000 7f22d9231700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f22d40b22e0 0x7f22d40b5b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:38.972 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.964+0000 7f22d9231700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f22d415dba0 0x7f22d415ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:38.972 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.964+0000 7f22d9231700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22d415e640 con 0x7f22d40b22e0 2023-11-08T19:25:38.973 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.964+0000 7f22d9231700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22d4156520 con 0x7f22d415dba0 2023-11-08T19:25:38.973 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.964+0000 7f22d9231700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22d40b5120 con 0x7f22d406b240 2023-11-08T19:25:38.973 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.964+0000 7f22d259c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f22d40b22e0 0x7f22d40b5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:38.973 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.964+0000 7f22d2d9d700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f22d406b240 0x7f22d4155fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:38.973 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.964+0000 7f22d259c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f22d40b22e0 0x7f22d40b5b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50290/0 (socket says 172.21.15.143:50290) 2023-11-08T19:25:38.974 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.964+0000 7f22d259c700 1 -- 172.21.15.143:0/2644779866 learned_addr learned my addr 172.21.15.143:0/2644779866 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:38.974 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.964+0000 7f22d359e700 1 --2- 172.21.15.143:0/2644779866 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f22d415dba0 0x7f22d415ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:38.974 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.964+0000 7f22d259c700 1 -- 172.21.15.143:0/2644779866 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f22d406b240 msgr2=0x7f22d4155fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:38.974 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.964+0000 7f22d259c700 1 --2- 172.21.15.143:0/2644779866 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f22d406b240 0x7f22d4155fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:38.974 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.964+0000 7f22d259c700 1 -- 172.21.15.143:0/2644779866 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f22d415dba0 msgr2=0x7f22d415ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:38.975 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.964+0000 7f22d259c700 1 --2- 172.21.15.143:0/2644779866 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f22d415dba0 0x7f22d415ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:38.975 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.964+0000 7f22d259c700 1 -- 172.21.15.143:0/2644779866 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22c40094a0 con 0x7f22d40b22e0 2023-11-08T19:25:38.975 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.965+0000 7f22d259c700 1 --2- 172.21.15.143:0/2644779866 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f22d40b22e0 0x7f22d40b5b30 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f22c80011e0 tx=0x7f22c800ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:38.975 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.965+0000 7f22bb7fe700 1 -- 172.21.15.143:0/2644779866 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22c8004c80 con 0x7f22d40b22e0 2023-11-08T19:25:38.975 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.965+0000 7f22bb7fe700 1 -- 172.21.15.143:0/2644779866 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22c8004e00 con 0x7f22d40b22e0 2023-11-08T19:25:38.976 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.965+0000 7f22d9231700 1 -- 172.21.15.143:0/2644779866 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22d40b4510 con 0x7f22d40b22e0 2023-11-08T19:25:38.976 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.965+0000 7f22bb7fe700 1 -- 172.21.15.143:0/2644779866 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22c8007310 con 0x7f22d40b22e0 2023-11-08T19:25:38.976 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.965+0000 7f22d9231700 1 -- 172.21.15.143:0/2644779866 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22d40b4ad0 con 0x7f22d40b22e0 2023-11-08T19:25:38.976 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.966+0000 7f22d9231700 1 -- 172.21.15.143:0/2644779866 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22d404eee0 con 0x7f22d40b22e0 2023-11-08T19:25:38.976 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.966+0000 7f22bb7fe700 1 -- 172.21.15.143:0/2644779866 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f22c8007470 con 0x7f22d40b22e0 2023-11-08T19:25:38.976 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.967+0000 7f22bb7fe700 1 --2- 172.21.15.143:0/2644779866 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f22c006af00 0x7f22c006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:38.977 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.967+0000 7f22bb7fe700 1 -- 172.21.15.143:0/2644779866 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5911+0+0 (secure 0 0 0) 0x7f22c80892d0 con 0x7f22d40b22e0 2023-11-08T19:25:38.977 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.967+0000 7f22d2d9d700 1 --2- 172.21.15.143:0/2644779866 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f22c006af00 0x7f22c006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:38.977 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.967+0000 7f22d2d9d700 1 --2- 172.21.15.143:0/2644779866 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f22c006af00 0x7f22c006d3c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f22c40090b0 tx=0x7f22c401c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:38.977 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:38.971+0000 7f22bb7fe700 1 -- 172.21.15.143:0/2644779866 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f22c80542b0 con 0x7f22d40b22e0 2023-11-08T19:25:39.201 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:39.200+0000 7f22d9231700 1 -- 172.21.15.143:0/2644779866 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f22d40b3130 con 0x7f22d40b22e0 2023-11-08T19:25:39.818 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:39.817+0000 7f22bb7fe700 1 -- 172.21.15.143:0/2644779866 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) v1 ==== 104+0+0 (secure 0 0 0) 0x7f22c80578d0 con 0x7f22d40b22e0 2023-11-08T19:25:39.821 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:39.819+0000 7f22d9231700 1 -- 172.21.15.143:0/2644779866 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f22c006af00 msgr2=0x7f22c006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:39.822 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:39.819+0000 7f22d9231700 1 --2- 172.21.15.143:0/2644779866 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f22c006af00 0x7f22c006d3c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f22c40090b0 tx=0x7f22c401c040 comp rx=0 tx=0).stop 2023-11-08T19:25:39.822 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:39.819+0000 7f22d9231700 1 -- 172.21.15.143:0/2644779866 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f22d40b22e0 msgr2=0x7f22d40b5b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:39.822 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:39.819+0000 7f22d9231700 1 --2- 172.21.15.143:0/2644779866 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f22d40b22e0 0x7f22d40b5b30 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f22c80011e0 tx=0x7f22c800ca30 comp rx=0 tx=0).stop 2023-11-08T19:25:39.822 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:39.820+0000 7f22d9231700 1 -- 172.21.15.143:0/2644779866 shutdown_connections 2023-11-08T19:25:39.822 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:39.820+0000 7f22d9231700 1 --2- 172.21.15.143:0/2644779866 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f22c006af00 0x7f22c006d3c0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:39.823 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:39.820+0000 7f22d9231700 1 --2- 172.21.15.143:0/2644779866 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f22d406b240 0x7f22d4155fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:39.823 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:39.820+0000 7f22d9231700 1 --2- 172.21.15.143:0/2644779866 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f22d415dba0 0x7f22d415ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:39.823 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:39.820+0000 7f22d9231700 1 --2- 172.21.15.143:0/2644779866 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f22d40b22e0 0x7f22d40b5b30 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:39.823 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:39.820+0000 7f22d9231700 1 -- 172.21.15.143:0/2644779866 >> 172.21.15.143:0/2644779866 conn(0x7f22d40cd2e0 msgr2=0x7f22d40cd6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:39.823 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:39.820+0000 7f22d9231700 1 -- 172.21.15.143:0/2644779866 shutdown_connections 2023-11-08T19:25:39.824 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:39.820+0000 7f22d9231700 1 -- 172.21.15.143:0/2644779866 wait complete. 2023-11-08T19:25:39.835 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-08T19:25:39.963 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:39.962+0000 7f4c49b22700 1 Processor -- start 2023-11-08T19:25:39.964 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:39.962+0000 7f4c49b22700 1 -- start start 2023-11-08T19:25:39.965 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:39.963+0000 7f4c49b22700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f4c4415a950 0x7f4c440d01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:39.966 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:39.963+0000 7f4c49b22700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4c4406bd20 0x7f4c44072370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:39.966 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:39.963+0000 7f4c49b22700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c440cc2a0 con 0x7f4c440728b0 2023-11-08T19:25:39.966 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:39.963+0000 7f4c49b22700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c440cc130 con 0x7f4c4415a950 2023-11-08T19:25:39.966 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:39.963+0000 7f4c49b22700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c4406c1b0 con 0x7f4c4406bd20 2023-11-08T19:25:39.966 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:39.963+0000 7f4c43fff700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f4c440728b0 0x7f4c440cc730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:46454/0 (socket says 172.21.15.143:46454) 2023-11-08T19:25:39.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:39.963+0000 7f4c43fff700 1 -- 172.21.15.143:0/3538858514 learned_addr learned my addr 172.21.15.143:0/3538858514 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:25:39.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:39.963+0000 7f4c437fe700 1 --2- 172.21.15.143:0/3538858514 >> v2:172.21.15.174:3300/0 conn(0x7f4c4415a950 0x7f4c440d01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:39.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:39.963+0000 7f4c42ffd700 1 --2- 172.21.15.143:0/3538858514 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4c4406bd20 0x7f4c44072370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:39.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:39.963+0000 7f4c427fc700 1 -- 172.21.15.143:0/3538858514 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3787750436 0 0) 0x7f4c440cc2a0 con 0x7f4c440728b0 2023-11-08T19:25:39.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:39.963+0000 7f4c427fc700 1 -- 172.21.15.143:0/3538858514 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c2c0057c0 con 0x7f4c440728b0 2023-11-08T19:25:39.968 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:39.963+0000 7f4c427fc700 1 -- 172.21.15.143:0/3538858514 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 742237415 0 0) 0x7f4c2c0057c0 con 0x7f4c440728b0 2023-11-08T19:25:39.968 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:39.963+0000 7f4c427fc700 1 -- 172.21.15.143:0/3538858514 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4c4406bd20 msgr2=0x7f4c44072370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:39.968 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:39.963+0000 7f4c427fc700 1 --2- 172.21.15.143:0/3538858514 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4c4406bd20 0x7f4c44072370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:39.968 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:39.963+0000 7f4c427fc700 1 -- 172.21.15.143:0/3538858514 >> v2:172.21.15.174:3300/0 conn(0x7f4c4415a950 msgr2=0x7f4c440d01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:39.968 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:39.963+0000 7f4c427fc700 1 --2- 172.21.15.143:0/3538858514 >> v2:172.21.15.174:3300/0 conn(0x7f4c4415a950 0x7f4c440d01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:39.969 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:39.963+0000 7f4c427fc700 1 -- 172.21.15.143:0/3538858514 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c4406c490 con 0x7f4c440728b0 2023-11-08T19:25:39.969 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:39.963+0000 7f4c427fc700 1 -- 172.21.15.143:0/3538858514 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4147756303 0 0) 0x7f4c3c0019f0 con 0x7f4c440728b0 2023-11-08T19:25:39.970 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:39.963+0000 7f4c427fc700 1 -- 172.21.15.143:0/3538858514 >> v1:172.21.15.143:6789/0 conn(0x7f4c440728b0 legacy=0x7f4c440cc730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:39.970 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:39.963+0000 7f4c427fc700 1 --2- 172.21.15.143:0/3538858514 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4c2c005c10 0x7f4c2c005ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:39.970 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:39.963+0000 7f4c427fc700 1 --2- 172.21.15.143:0/3538858514 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4c2c006570 0x7f4c2c006a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:39.970 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:39.963+0000 7f4c427fc700 1 --2- 172.21.15.143:0/3538858514 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4c2c0070b0 0x7f4c2c007530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:39.975 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:39.963+0000 7f4c427fc700 1 -- 172.21.15.143:0/3538858514 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c4406c1b0 con 0x7f4c2c006570 2023-11-08T19:25:39.975 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:39.963+0000 7f4c427fc700 1 -- 172.21.15.143:0/3538858514 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c2c00dfc0 con 0x7f4c2c0070b0 2023-11-08T19:25:39.976 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:39.963+0000 7f4c427fc700 1 -- 172.21.15.143:0/3538858514 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c2c00e180 con 0x7f4c2c005c10 2023-11-08T19:25:39.976 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:39.963+0000 7f4c42ffd700 1 --2- 172.21.15.143:0/3538858514 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4c2c006570 0x7f4c2c006a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:39.976 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:39.963+0000 7f4c437fe700 1 --2- 172.21.15.143:0/3538858514 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4c2c005c10 0x7f4c2c005ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:39.976 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:39.963+0000 7f4c43fff700 1 --2- 172.21.15.143:0/3538858514 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4c2c0070b0 0x7f4c2c007530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:39.977 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:39.964+0000 7f4c42ffd700 1 -- 172.21.15.143:0/3538858514 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4c2c005c10 msgr2=0x7f4c2c005ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:39.977 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:39.964+0000 7f4c42ffd700 1 --2- 172.21.15.143:0/3538858514 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4c2c005c10 0x7f4c2c005ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:39.977 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:39.964+0000 7f4c42ffd700 1 -- 172.21.15.143:0/3538858514 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4c2c0070b0 msgr2=0x7f4c2c007530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:39.977 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:39.964+0000 7f4c42ffd700 1 --2- 172.21.15.143:0/3538858514 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4c2c0070b0 0x7f4c2c007530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:39.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:39.964+0000 7f4c42ffd700 1 -- 172.21.15.143:0/3538858514 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c2c0057c0 con 0x7f4c2c006570 2023-11-08T19:25:39.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:39.964+0000 7f4c43fff700 1 --2- 172.21.15.143:0/3538858514 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4c2c0070b0 0x7f4c2c007530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:25:39.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:39.964+0000 7f4c42ffd700 1 --2- 172.21.15.143:0/3538858514 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4c2c006570 0x7f4c2c006a60 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f4c38009b60 tx=0x7f4c2c007a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f790ff78c2cf4d0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:39.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:39.964+0000 7f4c427fc700 1 -- 172.21.15.143:0/3538858514 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c3800e6c0 con 0x7f4c2c006570 2023-11-08T19:25:39.979 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:39.964+0000 7f4c427fc700 1 -- 172.21.15.143:0/3538858514 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c3800e820 con 0x7f4c2c006570 2023-11-08T19:25:39.979 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:39.964+0000 7f4c427fc700 1 -- 172.21.15.143:0/3538858514 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c3800eb90 con 0x7f4c2c006570 2023-11-08T19:25:39.979 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:39.964+0000 7f4c49b22700 1 -- 172.21.15.143:0/3538858514 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4c2c006570 msgr2=0x7f4c2c006a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:39.979 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:39.964+0000 7f4c49b22700 1 --2- 172.21.15.143:0/3538858514 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4c2c006570 0x7f4c2c006a60 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f4c38009b60 tx=0x7f4c2c007a90 comp rx=0 tx=0).stop 2023-11-08T19:25:39.979 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:39.964+0000 7f4c43fff700 1 -- 172.21.15.143:0/3538858514 reap_dead start 2023-11-08T19:25:39.980 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:39.964+0000 7f4c49b22700 1 -- 172.21.15.143:0/3538858514 shutdown_connections 2023-11-08T19:25:39.980 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:39.964+0000 7f4c49b22700 1 -- 172.21.15.143:0/3538858514 >> 172.21.15.143:0/3538858514 conn(0x7f4c440cd2e0 msgr2=0x7f4c440cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:39.980 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:39.964+0000 7f4c49b22700 1 -- 172.21.15.143:0/3538858514 shutdown_connections 2023-11-08T19:25:39.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:39.964+0000 7f4c49b22700 1 -- 172.21.15.143:0/3538858514 wait complete. 2023-11-08T19:25:39.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:39.966+0000 7f4c49b22700 1 Processor -- start 2023-11-08T19:25:39.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:39.966+0000 7f4c49b22700 1 -- start start 2023-11-08T19:25:39.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:39.966+0000 7f4c49b22700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4c2c006570 0x7f4c441f2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:39.982 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:39.967+0000 7f4c49b22700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4c440c9b60 0x7f4c4414a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:39.982 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:39.967+0000 7f4c49b22700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4c44152290 0x7f4c4406b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:39.982 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:39.967+0000 7f4c49b22700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c441f2b40 con 0x7f4c44152290 2023-11-08T19:25:39.982 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:39.967+0000 7f4c49b22700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c44154e60 con 0x7f4c440c9b60 2023-11-08T19:25:39.983 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:39.967+0000 7f4c49b22700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c441f2d30 con 0x7f4c2c006570 2023-11-08T19:25:39.983 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:39.967+0000 7f4c43fff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4c44152290 0x7f4c4406b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:39.983 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:39.967+0000 7f4c43fff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4c44152290 0x7f4c4406b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50312/0 (socket says 172.21.15.143:50312) 2023-11-08T19:25:39.983 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:39.967+0000 7f4c43fff700 1 -- 172.21.15.143:0/3216957700 learned_addr learned my addr 172.21.15.143:0/3216957700 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:39.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:39.967+0000 7f4c42ffd700 1 --2- 172.21.15.143:0/3216957700 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4c440c9b60 0x7f4c4414a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:39.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:39.967+0000 7f4c437fe700 1 --2- 172.21.15.143:0/3216957700 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4c2c006570 0x7f4c441f2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:39.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:39.967+0000 7f4c43fff700 1 -- 172.21.15.143:0/3216957700 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4c2c006570 msgr2=0x7f4c441f2600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:39.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:39.967+0000 7f4c43fff700 1 --2- 172.21.15.143:0/3216957700 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4c2c006570 0x7f4c441f2600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:39.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:39.967+0000 7f4c43fff700 1 -- 172.21.15.143:0/3216957700 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4c440c9b60 msgr2=0x7f4c4414a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:39.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:39.967+0000 7f4c43fff700 1 --2- 172.21.15.143:0/3216957700 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4c440c9b60 0x7f4c4414a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:39.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:39.967+0000 7f4c43fff700 1 -- 172.21.15.143:0/3216957700 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c441f3010 con 0x7f4c44152290 2023-11-08T19:25:39.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:39.967+0000 7f4c43fff700 1 --2- 172.21.15.143:0/3216957700 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4c44152290 0x7f4c4406b380 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c000f80 tx=0x7f4c3c00e440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:39.986 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:39.967+0000 7f4c2bfff700 1 -- 172.21.15.143:0/3216957700 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c3c00ebb0 con 0x7f4c44152290 2023-11-08T19:25:39.986 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:39.967+0000 7f4c2bfff700 1 -- 172.21.15.143:0/3216957700 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c3c00ed30 con 0x7f4c44152290 2023-11-08T19:25:39.986 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:39.967+0000 7f4c49b22700 1 -- 172.21.15.143:0/3216957700 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c441f32f0 con 0x7f4c44152290 2023-11-08T19:25:39.986 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:39.967+0000 7f4c2bfff700 1 -- 172.21.15.143:0/3216957700 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c3c013660 con 0x7f4c44152290 2023-11-08T19:25:39.986 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:39.967+0000 7f4c49b22700 1 -- 172.21.15.143:0/3216957700 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c4414a8f0 con 0x7f4c44152290 2023-11-08T19:25:39.987 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:39.968+0000 7f4c49b22700 1 -- 172.21.15.143:0/3216957700 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c44154fd0 con 0x7f4c44152290 2023-11-08T19:25:39.987 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:39.969+0000 7f4c2bfff700 1 -- 172.21.15.143:0/3216957700 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4c3c0137c0 con 0x7f4c44152290 2023-11-08T19:25:39.987 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:39.969+0000 7f4c2bfff700 1 --2- 172.21.15.143:0/3216957700 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f4c3006af00 0x7f4c3006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:39.987 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:39.969+0000 7f4c2bfff700 1 -- 172.21.15.143:0/3216957700 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5911+0+0 (secure 0 0 0) 0x7f4c3c089bb0 con 0x7f4c44152290 2023-11-08T19:25:39.987 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:39.969+0000 7f4c437fe700 1 --2- 172.21.15.143:0/3216957700 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f4c3006af00 0x7f4c3006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:39.988 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:39.970+0000 7f4c437fe700 1 --2- 172.21.15.143:0/3216957700 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f4c3006af00 0x7f4c3006d3c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4c34009ce0 tx=0x7f4c34009210 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:39.988 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:39.973+0000 7f4c2bfff700 1 -- 172.21.15.143:0/3216957700 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4c3c054b90 con 0x7f4c44152290 2023-11-08T19:25:40.163 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:40.162+0000 7f4c49b22700 1 -- 172.21.15.143:0/3216957700 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f4c441f34e0 con 0x7f4c44152290 2023-11-08T19:25:40.822 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:40.821+0000 7f4c2bfff700 1 -- 172.21.15.143:0/3216957700 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v23) v1 ==== 181+0+0 (secure 0 0 0) 0x7f4c3c00eea0 con 0x7f4c44152290 2023-11-08T19:25:40.826 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:40.825+0000 7f4c49b22700 1 -- 172.21.15.143:0/3216957700 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f4c3006af00 msgr2=0x7f4c3006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:40.826 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:40.825+0000 7f4c49b22700 1 --2- 172.21.15.143:0/3216957700 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f4c3006af00 0x7f4c3006d3c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4c34009ce0 tx=0x7f4c34009210 comp rx=0 tx=0).stop 2023-11-08T19:25:40.827 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:40.825+0000 7f4c49b22700 1 -- 172.21.15.143:0/3216957700 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4c44152290 msgr2=0x7f4c4406b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:40.827 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:40.825+0000 7f4c49b22700 1 --2- 172.21.15.143:0/3216957700 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4c44152290 0x7f4c4406b380 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c000f80 tx=0x7f4c3c00e440 comp rx=0 tx=0).stop 2023-11-08T19:25:40.827 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:40.825+0000 7f4c49b22700 1 -- 172.21.15.143:0/3216957700 shutdown_connections 2023-11-08T19:25:40.827 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:40.825+0000 7f4c49b22700 1 --2- 172.21.15.143:0/3216957700 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f4c3006af00 0x7f4c3006d3c0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:40.827 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:40.825+0000 7f4c49b22700 1 --2- 172.21.15.143:0/3216957700 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4c2c006570 0x7f4c441f2600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:40.828 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:40.825+0000 7f4c49b22700 1 --2- 172.21.15.143:0/3216957700 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4c44152290 0x7f4c4406b380 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:40.828 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:40.825+0000 7f4c49b22700 1 --2- 172.21.15.143:0/3216957700 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4c440c9b60 0x7f4c4414a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:40.828 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:40.825+0000 7f4c49b22700 1 -- 172.21.15.143:0/3216957700 >> 172.21.15.143:0/3216957700 conn(0x7f4c440cd2e0 msgr2=0x7f4c44150eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:40.828 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:40.825+0000 7f4c49b22700 1 -- 172.21.15.143:0/3216957700 shutdown_connections 2023-11-08T19:25:40.828 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:40.825+0000 7f4c49b22700 1 -- 172.21.15.143:0/3216957700 wait complete. 2023-11-08T19:25:40.829 INFO:teuthology.orchestra.run.smithi143.stderr:pool 'cephfs_metadata' removed 2023-11-08T19:25:40.843 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-08T19:25:40.974 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:40.973+0000 7f5c0b09f700 1 Processor -- start 2023-11-08T19:25:40.974 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:40.973+0000 7f5c0b09f700 1 -- start start 2023-11-08T19:25:40.977 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:40.973+0000 7f5c0b09f700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5c0415dba0 0x7f5c0406c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:40.977 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:40.973+0000 7f5c0b09f700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f5c0406b240 0x7f5c040caff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:40.977 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:40.973+0000 7f5c0b09f700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c04161890 con 0x7f5c040cb530 2023-11-08T19:25:40.977 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:40.973+0000 7f5c0b09f700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c0406b740 con 0x7f5c0406b240 2023-11-08T19:25:40.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:40.973+0000 7f5c0b09f700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c0406ba20 con 0x7f5c0415dba0 2023-11-08T19:25:40.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:40.974+0000 7f5c03fff700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f5c0406b240 0x7f5c040caff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:40.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:40.974+0000 7f5c03fff700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f5c0406b240 0x7f5c040caff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50318/0 (socket says 172.21.15.143:50318) 2023-11-08T19:25:40.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:40.974+0000 7f5c03fff700 1 -- 172.21.15.143:0/2670043199 learned_addr learned my addr 172.21.15.143:0/2670043199 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:40.979 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:40.974+0000 7f5c08e3b700 1 --2- 172.21.15.143:0/2670043199 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5c0415dba0 0x7f5c0406c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:40.979 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:40.974+0000 7f5c03fff700 1 -- 172.21.15.143:0/2670043199 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5c0415dba0 msgr2=0x7f5c0406c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:40.979 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:40.974+0000 7f5c03fff700 1 --2- 172.21.15.143:0/2670043199 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5c0415dba0 0x7f5c0406c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:40.979 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:40.974+0000 7f5c03fff700 1 -- 172.21.15.143:0/2670043199 >> v1:172.21.15.174:6789/0 conn(0x7f5c040cb530 legacy=0x7f5c040b22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:40.980 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:40.974+0000 7f5c03fff700 1 -- 172.21.15.143:0/2670043199 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c0406bd00 con 0x7f5c0406b240 2023-11-08T19:25:40.980 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:40.974+0000 7f5c03fff700 1 --2- 172.21.15.143:0/2670043199 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c0406b240 0x7f5c040caff0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f5bf00098f0 tx=0x7f5bf0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee63b7846fbfa077 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:40.980 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:40.974+0000 7f5c037fe700 1 -- 172.21.15.143:0/2670043199 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bf000b8a0 con 0x7f5c0406b240 2023-11-08T19:25:40.980 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:40.974+0000 7f5c037fe700 1 -- 172.21.15.143:0/2670043199 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bf000ba00 con 0x7f5c0406b240 2023-11-08T19:25:40.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:40.974+0000 7f5c037fe700 1 -- 172.21.15.143:0/2670043199 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bf000bd90 con 0x7f5c0406b240 2023-11-08T19:25:40.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:40.975+0000 7f5c0b09f700 1 -- 172.21.15.143:0/2670043199 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c0406b240 msgr2=0x7f5c040caff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:40.982 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:40.975+0000 7f5c0b09f700 1 --2- 172.21.15.143:0/2670043199 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c0406b240 0x7f5c040caff0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f5bf00098f0 tx=0x7f5bf0009c00 comp rx=0 tx=0).stop 2023-11-08T19:25:40.982 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:40.975+0000 7f5c0b09f700 1 -- 172.21.15.143:0/2670043199 shutdown_connections 2023-11-08T19:25:40.982 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:40.975+0000 7f5c0b09f700 1 --2- 172.21.15.143:0/2670043199 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5c0415dba0 0x7f5c0406c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:40.983 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:40.975+0000 7f5c0b09f700 1 --2- 172.21.15.143:0/2670043199 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c0406b240 0x7f5c040caff0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:40.983 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:40.975+0000 7f5c0b09f700 1 -- 172.21.15.143:0/2670043199 >> 172.21.15.143:0/2670043199 conn(0x7f5c040cd2e0 msgr2=0x7f5c040cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:40.983 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:40.975+0000 7f5c0b09f700 1 -- 172.21.15.143:0/2670043199 shutdown_connections 2023-11-08T19:25:40.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:40.975+0000 7f5c0b09f700 1 -- 172.21.15.143:0/2670043199 wait complete. 2023-11-08T19:25:40.988 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:40.976+0000 7f5c0b09f700 1 Processor -- start 2023-11-08T19:25:40.988 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:40.976+0000 7f5c0b09f700 1 -- start start 2023-11-08T19:25:40.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:40.977+0000 7f5c0b09f700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c0406b240 0x7f5c040addf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:40.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:40.977+0000 7f5c0b09f700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5c040cb530 0x7f5c040ae3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:40.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:40.977+0000 7f5c0b09f700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5c0415dba0 0x7f5c040ae8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:40.990 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:40.977+0000 7f5c0b09f700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c040c5eb0 con 0x7f5c0406b240 2023-11-08T19:25:40.990 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:40.977+0000 7f5c0b09f700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c040c5230 con 0x7f5c040cb530 2023-11-08T19:25:40.990 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:40.977+0000 7f5c0b09f700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c040c5510 con 0x7f5c0415dba0 2023-11-08T19:25:40.990 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:40.977+0000 7f5c08e3b700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c0406b240 0x7f5c040addf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:40.990 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:40.977+0000 7f5c08e3b700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c0406b240 0x7f5c040addf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50322/0 (socket says 172.21.15.143:50322) 2023-11-08T19:25:40.991 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:40.977+0000 7f5c08e3b700 1 -- 172.21.15.143:0/212598738 learned_addr learned my addr 172.21.15.143:0/212598738 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:40.991 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:40.977+0000 7f5c03fff700 1 --2- 172.21.15.143:0/212598738 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5c040cb530 0x7f5c040ae3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:40.991 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:40.977+0000 7f5c0963c700 1 --2- 172.21.15.143:0/212598738 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5c0415dba0 0x7f5c040ae8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:40.991 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:40.977+0000 7f5c08e3b700 1 -- 172.21.15.143:0/212598738 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5c0415dba0 msgr2=0x7f5c040ae8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:40.991 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:40.977+0000 7f5c08e3b700 1 --2- 172.21.15.143:0/212598738 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5c0415dba0 0x7f5c040ae8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:40.992 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:40.977+0000 7f5c08e3b700 1 -- 172.21.15.143:0/212598738 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5c040cb530 msgr2=0x7f5c040ae3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:40.992 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:40.977+0000 7f5c08e3b700 1 --2- 172.21.15.143:0/212598738 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5c040cb530 0x7f5c040ae3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:40.992 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:40.977+0000 7f5c08e3b700 1 -- 172.21.15.143:0/212598738 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bf0009550 con 0x7f5c0406b240 2023-11-08T19:25:40.992 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:40.977+0000 7f5c08e3b700 1 --2- 172.21.15.143:0/212598738 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c0406b240 0x7f5c040addf0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f5bf800d800 tx=0x7f5bf800db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:40.992 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:40.978+0000 7f5c017fa700 1 -- 172.21.15.143:0/212598738 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bf80125f0 con 0x7f5c0406b240 2023-11-08T19:25:40.993 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:40.978+0000 7f5c017fa700 1 -- 172.21.15.143:0/212598738 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bf8012750 con 0x7f5c0406b240 2023-11-08T19:25:40.993 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:40.978+0000 7f5c0b09f700 1 -- 172.21.15.143:0/212598738 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c040c57f0 con 0x7f5c0406b240 2023-11-08T19:25:40.993 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:40.978+0000 7f5c017fa700 1 -- 172.21.15.143:0/212598738 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bf8012b00 con 0x7f5c0406b240 2023-11-08T19:25:40.993 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:40.978+0000 7f5c0b09f700 1 -- 172.21.15.143:0/212598738 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c040c72b0 con 0x7f5c0406b240 2023-11-08T19:25:40.994 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:40.979+0000 7f5c0b09f700 1 -- 172.21.15.143:0/212598738 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c0404eee0 con 0x7f5c0406b240 2023-11-08T19:25:40.994 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:40.979+0000 7f5c017fa700 1 -- 172.21.15.143:0/212598738 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5bf8012c60 con 0x7f5c0406b240 2023-11-08T19:25:40.994 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:40.979+0000 7f5c017fa700 1 --2- 172.21.15.143:0/212598738 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f5be806af00 0x7f5be806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:40.994 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:40.980+0000 7f5c017fa700 1 -- 172.21.15.143:0/212598738 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5473+0+0 (secure 0 0 0) 0x7f5bf8089570 con 0x7f5c0406b240 2023-11-08T19:25:40.995 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:40.980+0000 7f5c03fff700 1 --2- 172.21.15.143:0/212598738 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f5be806af00 0x7f5be806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:40.995 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:40.980+0000 7f5c03fff700 1 --2- 172.21.15.143:0/212598738 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f5be806af00 0x7f5be806d3c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f5bf0009c40 tx=0x7f5bf0008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:40.995 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:40.984+0000 7f5c017fa700 1 -- 172.21.15.143:0/212598738 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5bf8054700 con 0x7f5c0406b240 2023-11-08T19:25:41.183 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:41.181+0000 7f5c0b09f700 1 -- 172.21.15.143:0/212598738 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f5c040c6120 con 0x7f5c0406b240 2023-11-08T19:25:41.829 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:41.828+0000 7f5c017fa700 1 -- 172.21.15.143:0/212598738 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v24) v1 ==== 169+0+0 (secure 0 0 0) 0x7f5c040c6120 con 0x7f5c0406b240 2023-11-08T19:25:41.834 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:41.831+0000 7f5c0b09f700 1 -- 172.21.15.143:0/212598738 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f5be806af00 msgr2=0x7f5be806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:41.834 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:41.831+0000 7f5c0b09f700 1 --2- 172.21.15.143:0/212598738 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f5be806af00 0x7f5be806d3c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f5bf0009c40 tx=0x7f5bf0008030 comp rx=0 tx=0).stop 2023-11-08T19:25:41.834 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:41.831+0000 7f5c0b09f700 1 -- 172.21.15.143:0/212598738 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c0406b240 msgr2=0x7f5c040addf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:41.834 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:41.831+0000 7f5c0b09f700 1 --2- 172.21.15.143:0/212598738 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c0406b240 0x7f5c040addf0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f5bf800d800 tx=0x7f5bf800db10 comp rx=0 tx=0).stop 2023-11-08T19:25:41.834 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:41.831+0000 7f5c0b09f700 1 -- 172.21.15.143:0/212598738 shutdown_connections 2023-11-08T19:25:41.835 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:41.831+0000 7f5c0b09f700 1 --2- 172.21.15.143:0/212598738 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f5be806af00 0x7f5be806d3c0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:41.835 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:41.831+0000 7f5c0b09f700 1 --2- 172.21.15.143:0/212598738 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5c0415dba0 0x7f5c040ae8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:41.835 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:41.831+0000 7f5c0b09f700 1 --2- 172.21.15.143:0/212598738 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c0406b240 0x7f5c040addf0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:41.835 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:41.831+0000 7f5c0b09f700 1 --2- 172.21.15.143:0/212598738 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5c040cb530 0x7f5c040ae3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:41.835 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:41.831+0000 7f5c0b09f700 1 -- 172.21.15.143:0/212598738 >> 172.21.15.143:0/212598738 conn(0x7f5c040cd2e0 msgr2=0x7f5c0415ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:41.836 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:41.831+0000 7f5c0b09f700 1 -- 172.21.15.143:0/212598738 shutdown_connections 2023-11-08T19:25:41.836 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:41.831+0000 7f5c0b09f700 1 -- 172.21.15.143:0/212598738 wait complete. 2023-11-08T19:25:41.836 INFO:teuthology.orchestra.run.smithi143.stderr:pool 'cephfs_data' removed 2023-11-08T19:25:41.848 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-08T19:25:41.848 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-08T19:25:41.848 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-08T19:25:41.848 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T19:25:41.849 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-08T19:25:41.849 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-08T19:25:41.849 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-08T19:25:41.850 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T19:25:41.850 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-08T19:25:41.850 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-08T19:25:41.850 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-08T19:25:41.851 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T19:25:41.851 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-08T19:25:41.851 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-08T19:25:41.852 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-08T19:25:41.852 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T19:25:41.860 INFO:tasks.ceph.mds.a:Stopped 2023-11-08T19:25:41.860 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-08T19:25:41.862 INFO:tasks.ceph.mds.c:Stopped 2023-11-08T19:25:41.862 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-08T19:25:41.864 INFO:tasks.ceph.mds.b:Stopped 2023-11-08T19:25:41.865 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-08T19:25:41.866 INFO:tasks.ceph.mds.d:Stopped 2023-11-08T19:25:41.866 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-08T19:25:41.904 INFO:tasks.ceph.mds.a:Started 2023-11-08T19:25:41.905 DEBUG:teuthology.parallel:result is None 2023-11-08T19:25:41.906 INFO:tasks.ceph.mds.c:Started 2023-11-08T19:25:41.907 DEBUG:teuthology.parallel:result is None 2023-11-08T19:25:41.907 INFO:tasks.ceph.mds.b:Started 2023-11-08T19:25:41.908 DEBUG:teuthology.parallel:result is None 2023-11-08T19:25:41.909 INFO:tasks.ceph.mds.d:Started 2023-11-08T19:25:41.909 DEBUG:teuthology.parallel:result is None 2023-11-08T19:25:41.909 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-08T19:25:41.984 INFO:tasks.ceph.mds.a.smithi143.stdout:starting mds.a at 2023-11-08T19:25:41.986 INFO:tasks.ceph.mds.c.smithi143.stdout:starting mds.c at 2023-11-08T19:25:41.997 INFO:tasks.ceph.mds.b.smithi174.stdout:starting mds.b at 2023-11-08T19:25:41.998 INFO:tasks.ceph.mds.d.smithi174.stdout:starting mds.d at 2023-11-08T19:25:42.038 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:42.037+0000 7f255563a700 1 Processor -- start 2023-11-08T19:25:42.038 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:42.037+0000 7f255563a700 1 -- start start 2023-11-08T19:25:42.041 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:42.038+0000 7f255563a700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f25500cf090 0x7f25500d1f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:42.041 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:42.038+0000 7f255563a700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f25500d3690 0x7f25500b9690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:42.041 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:42.038+0000 7f255563a700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25500d3200 con 0x7f255015dc80 2023-11-08T19:25:42.041 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:42.038+0000 7f255563a700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25500b9bd0 con 0x7f25500d3690 2023-11-08T19:25:42.041 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:42.038+0000 7f255563a700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25500b7cf0 con 0x7f25500cf090 2023-11-08T19:25:42.042 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:42.038+0000 7f254f7fe700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f25500d3690 0x7f25500b9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:42.042 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:42.038+0000 7f254f7fe700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f25500d3690 0x7f25500b9690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50356/0 (socket says 172.21.15.143:50356) 2023-11-08T19:25:42.042 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:42.038+0000 7f254f7fe700 1 -- 172.21.15.143:0/1756484748 learned_addr learned my addr 172.21.15.143:0/1756484748 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:42.042 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:42.038+0000 7f254e7fc700 1 --2- 172.21.15.143:0/1756484748 >> v2:172.21.15.174:3300/0 conn(0x7f25500cf090 0x7f25500d1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:42.042 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:42.038+0000 7f254f7fe700 1 -- 172.21.15.143:0/1756484748 >> v2:172.21.15.174:3300/0 conn(0x7f25500cf090 msgr2=0x7f25500d1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:42.043 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:42.038+0000 7f254f7fe700 1 --2- 172.21.15.143:0/1756484748 >> v2:172.21.15.174:3300/0 conn(0x7f25500cf090 0x7f25500d1f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:42.043 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:42.038+0000 7f254f7fe700 1 -- 172.21.15.143:0/1756484748 >> v1:172.21.15.174:6789/0 conn(0x7f255015dc80 legacy=0x7f25500ce980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:42.043 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:42.038+0000 7f254f7fe700 1 -- 172.21.15.143:0/1756484748 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f255015e060 con 0x7f25500d3690 2023-11-08T19:25:42.043 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:42.038+0000 7f254e7fc700 1 --2- 172.21.15.143:0/1756484748 >> v2:172.21.15.174:3300/0 conn(0x7f25500cf090 0x7f25500d1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:25:42.043 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:42.038+0000 7f254f7fe700 1 --2- 172.21.15.143:0/1756484748 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f25500d3690 0x7f25500b9690 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f2550161760 tx=0x7f254800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=678e936b64a8520b server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:42.044 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:42.038+0000 7f254dffb700 1 -- 172.21.15.143:0/1756484748 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f254800e070 con 0x7f25500d3690 2023-11-08T19:25:42.044 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:42.038+0000 7f254dffb700 1 -- 172.21.15.143:0/1756484748 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2548003a20 con 0x7f25500d3690 2023-11-08T19:25:42.044 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:42.038+0000 7f254dffb700 1 -- 172.21.15.143:0/1756484748 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2548003dd0 con 0x7f25500d3690 2023-11-08T19:25:42.044 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:42.039+0000 7f255563a700 1 -- 172.21.15.143:0/1756484748 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f25500d3690 msgr2=0x7f25500b9690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:42.044 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:42.039+0000 7f255563a700 1 --2- 172.21.15.143:0/1756484748 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f25500d3690 0x7f25500b9690 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f2550161760 tx=0x7f254800b330 comp rx=0 tx=0).stop 2023-11-08T19:25:42.045 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:42.039+0000 7f255563a700 1 -- 172.21.15.143:0/1756484748 shutdown_connections 2023-11-08T19:25:42.045 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:42.039+0000 7f255563a700 1 --2- 172.21.15.143:0/1756484748 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f25500d3690 0x7f25500b9690 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:42.045 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:42.039+0000 7f255563a700 1 --2- 172.21.15.143:0/1756484748 >> v2:172.21.15.174:3300/0 conn(0x7f25500cf090 0x7f25500d1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:42.045 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:42.039+0000 7f255563a700 1 -- 172.21.15.143:0/1756484748 >> 172.21.15.143:0/1756484748 conn(0x7f25500668e0 msgr2=0x7f2550066cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:42.045 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:42.039+0000 7f255563a700 1 -- 172.21.15.143:0/1756484748 shutdown_connections 2023-11-08T19:25:42.046 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:42.039+0000 7f255563a700 1 -- 172.21.15.143:0/1756484748 wait complete. 2023-11-08T19:25:42.048 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:42.041+0000 7f255563a700 1 Processor -- start 2023-11-08T19:25:42.048 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:42.041+0000 7f255563a700 1 -- start start 2023-11-08T19:25:42.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:42.042+0000 7f255563a700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f25500cf090 0x7f255014cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:42.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:42.042+0000 7f255563a700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f25500d3690 0x7f255014e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:42.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:42.042+0000 7f255563a700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f255015dc80 0x7f255014eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:42.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:42.042+0000 7f255563a700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f255014d570 con 0x7f25500cf090 2023-11-08T19:25:42.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:42.042+0000 7f255563a700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f255014d850 con 0x7f255015dc80 2023-11-08T19:25:42.050 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:42.042+0000 7f255563a700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f255014db30 con 0x7f25500d3690 2023-11-08T19:25:42.050 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:42.042+0000 7f254effd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f25500cf090 0x7f255014cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:42.050 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:42.042+0000 7f254effd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f25500cf090 0x7f255014cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50358/0 (socket says 172.21.15.143:50358) 2023-11-08T19:25:42.050 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:42.042+0000 7f254effd700 1 -- 172.21.15.143:0/1356228698 learned_addr learned my addr 172.21.15.143:0/1356228698 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:42.050 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:42.042+0000 7f254f7fe700 1 --2- 172.21.15.143:0/1356228698 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f255015dc80 0x7f255014eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:42.051 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:42.042+0000 7f254e7fc700 1 --2- 172.21.15.143:0/1356228698 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f25500d3690 0x7f255014e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:42.051 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:42.042+0000 7f254effd700 1 -- 172.21.15.143:0/1356228698 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f25500d3690 msgr2=0x7f255014e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:42.051 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:42.042+0000 7f254effd700 1 --2- 172.21.15.143:0/1356228698 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f25500d3690 0x7f255014e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:42.051 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:42.042+0000 7f254effd700 1 -- 172.21.15.143:0/1356228698 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f255015dc80 msgr2=0x7f255014eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:42.051 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:42.042+0000 7f254effd700 1 --2- 172.21.15.143:0/1356228698 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f255015dc80 0x7f255014eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:42.052 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:42.042+0000 7f254effd700 1 -- 172.21.15.143:0/1356228698 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2548009b30 con 0x7f25500cf090 2023-11-08T19:25:42.052 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:42.042+0000 7f254effd700 1 --2- 172.21.15.143:0/1356228698 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f25500cf090 0x7f255014cab0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f254000c4d0 tx=0x7f254000c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:42.052 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:42.042+0000 7f25377fe700 1 -- 172.21.15.143:0/1356228698 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2540011790 con 0x7f25500cf090 2023-11-08T19:25:42.052 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:42.042+0000 7f25377fe700 1 -- 172.21.15.143:0/1356228698 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25400118f0 con 0x7f25500cf090 2023-11-08T19:25:42.052 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:42.042+0000 7f255563a700 1 -- 172.21.15.143:0/1356228698 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f255014de10 con 0x7f25500cf090 2023-11-08T19:25:42.053 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:42.042+0000 7f25377fe700 1 -- 172.21.15.143:0/1356228698 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2540011ca0 con 0x7f25500cf090 2023-11-08T19:25:42.053 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:42.042+0000 7f255563a700 1 -- 172.21.15.143:0/1356228698 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25500ce5c0 con 0x7f25500cf090 2023-11-08T19:25:42.053 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:42.043+0000 7f255563a700 1 -- 172.21.15.143:0/1356228698 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f255014e330 con 0x7f25500cf090 2023-11-08T19:25:42.053 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:42.044+0000 7f25377fe700 1 -- 172.21.15.143:0/1356228698 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f254001c460 con 0x7f25500cf090 2023-11-08T19:25:42.053 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:42.044+0000 7f25377fe700 1 --2- 172.21.15.143:0/1356228698 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f253c06af00 0x7f253c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:42.054 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:42.044+0000 7f25377fe700 1 -- 172.21.15.143:0/1356228698 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f25400890a0 con 0x7f25500cf090 2023-11-08T19:25:42.054 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:42.045+0000 7f254e7fc700 1 --2- 172.21.15.143:0/1356228698 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f253c06af00 0x7f253c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:42.054 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:42.045+0000 7f254e7fc700 1 --2- 172.21.15.143:0/1356228698 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f253c06af00 0x7f253c06d3c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f255015a5b0 tx=0x7f2544008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:42.054 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:42.049+0000 7f25377fe700 1 -- 172.21.15.143:0/1356228698 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f255014e330 con 0x7f25500cf090 2023-11-08T19:25:42.254 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:42.253+0000 7f255563a700 1 -- 172.21.15.143:0/1356228698 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f255014e330 con 0x7f25500cf090 2023-11-08T19:25:43.590 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:43.585+0000 7f25377fe700 1 -- 172.21.15.143:0/1356228698 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v29) v1 ==== 97+0+0 (secure 0 0 0) 0x7f255014e330 con 0x7f25500cf090 2023-11-08T19:25:43.590 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:43.588+0000 7f255563a700 1 -- 172.21.15.143:0/1356228698 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f253c06af00 msgr2=0x7f253c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:43.591 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:43.588+0000 7f255563a700 1 --2- 172.21.15.143:0/1356228698 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f253c06af00 0x7f253c06d3c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f255015a5b0 tx=0x7f2544008040 comp rx=0 tx=0).stop 2023-11-08T19:25:43.591 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:43.588+0000 7f255563a700 1 -- 172.21.15.143:0/1356228698 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f25500cf090 msgr2=0x7f255014cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:43.591 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:43.588+0000 7f255563a700 1 --2- 172.21.15.143:0/1356228698 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f25500cf090 0x7f255014cab0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f254000c4d0 tx=0x7f254000c7e0 comp rx=0 tx=0).stop 2023-11-08T19:25:43.591 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:43.588+0000 7f255563a700 1 -- 172.21.15.143:0/1356228698 shutdown_connections 2023-11-08T19:25:43.591 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:43.588+0000 7f255563a700 1 --2- 172.21.15.143:0/1356228698 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f253c06af00 0x7f253c06d3c0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:43.592 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:43.588+0000 7f255563a700 1 --2- 172.21.15.143:0/1356228698 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f255015dc80 0x7f255014eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:43.592 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:43.588+0000 7f255563a700 1 --2- 172.21.15.143:0/1356228698 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f25500cf090 0x7f255014cab0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:43.592 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:43.588+0000 7f255563a700 1 --2- 172.21.15.143:0/1356228698 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f25500d3690 0x7f255014e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:43.592 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:43.588+0000 7f255563a700 1 -- 172.21.15.143:0/1356228698 >> 172.21.15.143:0/1356228698 conn(0x7f25500668e0 msgr2=0x7f255015a970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:43.592 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:43.588+0000 7f255563a700 1 -- 172.21.15.143:0/1356228698 shutdown_connections 2023-11-08T19:25:43.593 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:43.588+0000 7f255563a700 1 -- 172.21.15.143:0/1356228698 wait complete. 2023-11-08T19:25:43.593 INFO:teuthology.orchestra.run.smithi143.stderr: removed all blocklist entries 2023-11-08T19:25:43.605 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-08T19:25:43.730 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:43.729+0000 7fd7c69ec700 1 Processor -- start 2023-11-08T19:25:43.731 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:43.729+0000 7fd7c69ec700 1 -- start start 2023-11-08T19:25:43.733 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:43.730+0000 7fd7c69ec700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd7c015f510 0x7fd7c00b24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:43.733 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:43.730+0000 7fd7c69ec700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd7c00b1100 0x7fd7c00b1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:43.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:43.730+0000 7fd7c69ec700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7c00b1a90 con 0x7fd7c015ba40 2023-11-08T19:25:43.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:43.730+0000 7fd7c69ec700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7c015f970 con 0x7fd7c00b1100 2023-11-08T19:25:43.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:43.730+0000 7fd7c69ec700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c015be20 con 0x7fd7c015f510 2023-11-08T19:25:43.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:43.730+0000 7fd7c4f89700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd7c00b1100 0x7fd7c00b1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:43.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:43.730+0000 7fd7c4f89700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd7c00b1100 0x7fd7c00b1550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50374/0 (socket says 172.21.15.143:50374) 2023-11-08T19:25:43.735 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:43.730+0000 7fd7c4f89700 1 -- 172.21.15.143:0/605759593 learned_addr learned my addr 172.21.15.143:0/605759593 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:43.735 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:43.730+0000 7fd7bf7fe700 1 --2- 172.21.15.143:0/605759593 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd7c015f510 0x7fd7c00b24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:43.735 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:43.730+0000 7fd7c4f89700 1 -- 172.21.15.143:0/605759593 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd7c015f510 msgr2=0x7fd7c00b24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:43.735 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:43.730+0000 7fd7c4f89700 1 --2- 172.21.15.143:0/605759593 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd7c015f510 0x7fd7c00b24d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:43.736 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:43.730+0000 7fd7c4f89700 1 -- 172.21.15.143:0/605759593 >> v1:172.21.15.174:6789/0 conn(0x7fd7c015ba40 legacy=0x7fd7c006e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:43.736 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:43.730+0000 7fd7c4f89700 1 -- 172.21.15.143:0/605759593 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7c0070f00 con 0x7fd7c00b1100 2023-11-08T19:25:43.736 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:43.730+0000 7fd7c4f89700 1 --2- 172.21.15.143:0/605759593 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd7c00b1100 0x7fd7c00b1550 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fd7b8009e50 tx=0x7fd7b800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d32169dfa0d96229 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:43.736 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:43.730+0000 7fd7beffd700 1 -- 172.21.15.143:0/605759593 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b800e070 con 0x7fd7c00b1100 2023-11-08T19:25:43.736 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:43.730+0000 7fd7beffd700 1 -- 172.21.15.143:0/605759593 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7b8003a20 con 0x7fd7c00b1100 2023-11-08T19:25:43.737 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:43.730+0000 7fd7beffd700 1 -- 172.21.15.143:0/605759593 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b8003dd0 con 0x7fd7c00b1100 2023-11-08T19:25:43.737 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:43.731+0000 7fd7c69ec700 1 -- 172.21.15.143:0/605759593 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd7c00b1100 msgr2=0x7fd7c00b1550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:43.737 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:43.731+0000 7fd7c69ec700 1 --2- 172.21.15.143:0/605759593 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd7c00b1100 0x7fd7c00b1550 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fd7b8009e50 tx=0x7fd7b800b330 comp rx=0 tx=0).stop 2023-11-08T19:25:43.737 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:43.731+0000 7fd7c69ec700 1 -- 172.21.15.143:0/605759593 shutdown_connections 2023-11-08T19:25:43.737 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:43.731+0000 7fd7c69ec700 1 --2- 172.21.15.143:0/605759593 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd7c00b1100 0x7fd7c00b1550 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:43.738 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:43.731+0000 7fd7c69ec700 1 --2- 172.21.15.143:0/605759593 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd7c015f510 0x7fd7c00b24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:43.738 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:43.731+0000 7fd7c69ec700 1 -- 172.21.15.143:0/605759593 >> 172.21.15.143:0/605759593 conn(0x7fd7c006b4c0 msgr2=0x7fd7c00cf680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:43.738 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:43.731+0000 7fd7c69ec700 1 -- 172.21.15.143:0/605759593 shutdown_connections 2023-11-08T19:25:43.738 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:43.731+0000 7fd7c69ec700 1 -- 172.21.15.143:0/605759593 wait complete. 2023-11-08T19:25:43.738 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:43.732+0000 7fd7c69ec700 1 Processor -- start 2023-11-08T19:25:43.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:43.732+0000 7fd7c69ec700 1 -- start start 2023-11-08T19:25:43.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:43.733+0000 7fd7c69ec700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd7c00b1100 0x7fd7c00b4940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:43.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:43.733+0000 7fd7c69ec700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd7c015ba40 0x7fd7c00c8890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:43.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:43.733+0000 7fd7c69ec700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd7c015f510 0x7fd7c00c8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:43.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:43.733+0000 7fd7c69ec700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c00b3970 con 0x7fd7c00b1100 2023-11-08T19:25:43.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:43.733+0000 7fd7c69ec700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c01f4790 con 0x7fd7c015f510 2023-11-08T19:25:43.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:43.733+0000 7fd7c69ec700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c00b3b80 con 0x7fd7c015ba40 2023-11-08T19:25:43.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:43.733+0000 7fd7bffff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd7c00b1100 0x7fd7c00b4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:43.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:43.733+0000 7fd7bffff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd7c00b1100 0x7fd7c00b4940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50376/0 (socket says 172.21.15.143:50376) 2023-11-08T19:25:43.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:43.733+0000 7fd7bffff700 1 -- 172.21.15.143:0/4145258812 learned_addr learned my addr 172.21.15.143:0/4145258812 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:43.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:43.733+0000 7fd7bf7fe700 1 --2- 172.21.15.143:0/4145258812 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd7c015ba40 0x7fd7c00c8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:43.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:43.733+0000 7fd7c4f89700 1 --2- 172.21.15.143:0/4145258812 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd7c015f510 0x7fd7c00c8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:43.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:43.733+0000 7fd7bffff700 1 -- 172.21.15.143:0/4145258812 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd7c015ba40 msgr2=0x7fd7c00c8890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:43.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:43.733+0000 7fd7bffff700 1 --2- 172.21.15.143:0/4145258812 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd7c015ba40 0x7fd7c00c8890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:43.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:43.733+0000 7fd7bffff700 1 -- 172.21.15.143:0/4145258812 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd7c015f510 msgr2=0x7fd7c00c8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:43.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:43.733+0000 7fd7bffff700 1 --2- 172.21.15.143:0/4145258812 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd7c015f510 0x7fd7c00c8dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:43.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:43.733+0000 7fd7bffff700 1 -- 172.21.15.143:0/4145258812 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7b8009b30 con 0x7fd7c00b1100 2023-11-08T19:25:43.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:43.733+0000 7fd7c4f89700 1 --2- 172.21.15.143:0/4145258812 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd7c015f510 0x7fd7c00c8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:25:43.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:43.734+0000 7fd7bf7fe700 1 --2- 172.21.15.143:0/4145258812 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd7c015ba40 0x7fd7c00c8890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:25:43.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:43.734+0000 7fd7bffff700 1 --2- 172.21.15.143:0/4145258812 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd7c00b1100 0x7fd7c00b4940 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fd7a800c4d0 tx=0x7fd7a800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:43.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:43.734+0000 7fd7bcff9700 1 -- 172.21.15.143:0/4145258812 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a8011790 con 0x7fd7c00b1100 2023-11-08T19:25:43.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:43.734+0000 7fd7bcff9700 1 -- 172.21.15.143:0/4145258812 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7a80118f0 con 0x7fd7c00b1100 2023-11-08T19:25:43.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:43.734+0000 7fd7bcff9700 1 -- 172.21.15.143:0/4145258812 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a8011ca0 con 0x7fd7c00b1100 2023-11-08T19:25:43.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:43.734+0000 7fd7c69ec700 1 -- 172.21.15.143:0/4145258812 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7c00b3e60 con 0x7fd7c00b1100 2023-11-08T19:25:43.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:43.734+0000 7fd7c69ec700 1 -- 172.21.15.143:0/4145258812 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7c00b4420 con 0x7fd7c00b1100 2023-11-08T19:25:43.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:43.735+0000 7fd7c69ec700 1 -- 172.21.15.143:0/4145258812 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7c004eee0 con 0x7fd7c00b1100 2023-11-08T19:25:43.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:43.735+0000 7fd7bcff9700 1 -- 172.21.15.143:0/4145258812 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd7a801c460 con 0x7fd7c00b1100 2023-11-08T19:25:43.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:43.735+0000 7fd7bcff9700 1 --2- 172.21.15.143:0/4145258812 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd7a006af00 0x7fd7a006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:43.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:43.736+0000 7fd7bcff9700 1 -- 172.21.15.143:0/4145258812 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fd7a8089150 con 0x7fd7c00b1100 2023-11-08T19:25:43.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:43.736+0000 7fd7bf7fe700 1 --2- 172.21.15.143:0/4145258812 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd7a006af00 0x7fd7a006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:43.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:43.736+0000 7fd7bf7fe700 1 --2- 172.21.15.143:0/4145258812 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd7a006af00 0x7fd7a006d3c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd7b0006fd0 tx=0x7fd7b0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:43.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:43.740+0000 7fd7bcff9700 1 -- 172.21.15.143:0/4145258812 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd7a80544d0 con 0x7fd7c00b1100 2023-11-08T19:25:44.008 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.006+0000 7fd7c69ec700 1 -- 172.21.15.143:0/4145258812 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fd7c00ae390 con 0x7fd7c00b1100 2023-11-08T19:25:44.009 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.007+0000 7fd7bcff9700 1 -- 172.21.15.143:0/4145258812 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v2) v1 ==== 80+0+6645 (secure 0 0 0) 0x7fd7a8057af0 con 0x7fd7c00b1100 2023-11-08T19:25:44.009 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:25:44.009 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2023-11-08T19:25:44.009 INFO:teuthology.orchestra.run.smithi143.stdout: "auth_dump": [ 2023-11-08T19:25:44.010 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-08T19:25:44.010 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "mds.a", 2023-11-08T19:25:44.010 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQBO4EtlqQPkOhAACwt3IOZLFtEQle9/FFliYA==", 2023-11-08T19:25:44.010 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-08T19:25:44.010 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2023-11-08T19:25:44.011 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow *", 2023-11-08T19:25:44.011 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow *", 2023-11-08T19:25:44.011 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-08T19:25:44.011 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-08T19:25:44.011 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-08T19:25:44.011 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-08T19:25:44.012 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "mds.b", 2023-11-08T19:25:44.012 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQBP4EtlHJqqCxAAxgT7Ce6QICWZ0qebHWZhMQ==", 2023-11-08T19:25:44.012 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-08T19:25:44.012 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2023-11-08T19:25:44.012 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow *", 2023-11-08T19:25:44.013 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow *", 2023-11-08T19:25:44.013 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-08T19:25:44.013 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-08T19:25:44.014 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-08T19:25:44.014 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-08T19:25:44.014 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "mds.c", 2023-11-08T19:25:44.014 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQBP4EtlGEeDBhAAihr82JiBJZR0rRkQj2n8wA==", 2023-11-08T19:25:44.015 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-08T19:25:44.015 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2023-11-08T19:25:44.015 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow *", 2023-11-08T19:25:44.015 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow *", 2023-11-08T19:25:44.015 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-08T19:25:44.016 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-08T19:25:44.016 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-08T19:25:44.016 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-08T19:25:44.016 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "mds.d", 2023-11-08T19:25:44.017 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQBP4EtlW5//EhAArVOkzNpCYd6Skl6eh5EYUQ==", 2023-11-08T19:25:44.017 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-08T19:25:44.017 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2023-11-08T19:25:44.017 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow *", 2023-11-08T19:25:44.017 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow *", 2023-11-08T19:25:44.018 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-08T19:25:44.018 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-08T19:25:44.018 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-08T19:25:44.018 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-08T19:25:44.018 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.0", 2023-11-08T19:25:44.019 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQBS4EtloVd4KRAAi5nj/vB9UK/mWZ9lkaFAxg==", 2023-11-08T19:25:44.019 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-08T19:25:44.019 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2023-11-08T19:25:44.019 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2023-11-08T19:25:44.020 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-08T19:25:44.020 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-08T19:25:44.020 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-08T19:25:44.020 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-08T19:25:44.020 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.1", 2023-11-08T19:25:44.021 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQBV4EtlWlOaKRAAooz5P+AoTgyuL/QAv4REKA==", 2023-11-08T19:25:44.021 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-08T19:25:44.021 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2023-11-08T19:25:44.021 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2023-11-08T19:25:44.021 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-08T19:25:44.021 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-08T19:25:44.022 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-08T19:25:44.022 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-08T19:25:44.022 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.2", 2023-11-08T19:25:44.022 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQBY4EtlBpaWKhAAmob9TfEKw5x6dEIu8ALVGA==", 2023-11-08T19:25:44.022 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-08T19:25:44.023 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2023-11-08T19:25:44.023 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2023-11-08T19:25:44.023 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-08T19:25:44.023 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-08T19:25:44.023 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-08T19:25:44.024 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-08T19:25:44.024 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.3", 2023-11-08T19:25:44.024 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQBb4Etl5pkrKhAA38ATMC3NkZ4plRRA69i0vw==", 2023-11-08T19:25:44.025 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-08T19:25:44.025 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2023-11-08T19:25:44.025 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2023-11-08T19:25:44.025 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-08T19:25:44.025 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-08T19:25:44.025 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-08T19:25:44.026 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-08T19:25:44.026 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.4", 2023-11-08T19:25:44.026 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQBh4EtlqLnUMBAABIrSskahtrZR0ClfczX5Ag==", 2023-11-08T19:25:44.026 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-08T19:25:44.026 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2023-11-08T19:25:44.027 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2023-11-08T19:25:44.027 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-08T19:25:44.027 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-08T19:25:44.027 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-08T19:25:44.027 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-08T19:25:44.028 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.5", 2023-11-08T19:25:44.028 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQBk4Etlt9PTMBAAz55IHg6Jrin3BUAvhOM/Fg==", 2023-11-08T19:25:44.028 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-08T19:25:44.028 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2023-11-08T19:25:44.028 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2023-11-08T19:25:44.028 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-08T19:25:44.029 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-08T19:25:44.029 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-08T19:25:44.029 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-08T19:25:44.029 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.6", 2023-11-08T19:25:44.029 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQBn4EtlcTkyMBAA5fD3h4s+99rgd6fqHHCFMw==", 2023-11-08T19:25:44.030 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-08T19:25:44.030 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2023-11-08T19:25:44.030 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2023-11-08T19:25:44.030 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-08T19:25:44.030 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-08T19:25:44.031 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-08T19:25:44.031 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-08T19:25:44.031 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.7", 2023-11-08T19:25:44.031 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQBq4EtldEviMBAAyGYrHVGwxHdmtDOwWCSHDQ==", 2023-11-08T19:25:44.031 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-08T19:25:44.032 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2023-11-08T19:25:44.032 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2023-11-08T19:25:44.032 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-08T19:25:44.032 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-08T19:25:44.032 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-08T19:25:44.033 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-08T19:25:44.033 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "client.0", 2023-11-08T19:25:44.033 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQBP4Etl0GdSFxAAVkuHL0EnjrtTUMkj/afPkg==", 2023-11-08T19:25:44.033 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-08T19:25:44.033 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2023-11-08T19:25:44.034 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow r", 2023-11-08T19:25:44.034 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow rw", 2023-11-08T19:25:44.034 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow rwx" 2023-11-08T19:25:44.035 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-08T19:25:44.035 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-08T19:25:44.035 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-08T19:25:44.035 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "client.1", 2023-11-08T19:25:44.035 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQBP4EtlPQ5yIBAAiyQaqCJph2KOPBZPSIh4ZQ==", 2023-11-08T19:25:44.036 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-08T19:25:44.036 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2023-11-08T19:25:44.036 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow r", 2023-11-08T19:25:44.036 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow rw", 2023-11-08T19:25:44.036 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow rwx" 2023-11-08T19:25:44.037 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-08T19:25:44.037 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-08T19:25:44.037 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-08T19:25:44.037 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "client.2", 2023-11-08T19:25:44.037 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQBP4Etlm+8SHRAAAwxcy2EzUvtFw9/mTzyk7w==", 2023-11-08T19:25:44.037 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-08T19:25:44.038 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2023-11-08T19:25:44.038 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow r", 2023-11-08T19:25:44.038 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow rw", 2023-11-08T19:25:44.038 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow rwx" 2023-11-08T19:25:44.038 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-08T19:25:44.039 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-08T19:25:44.039 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-08T19:25:44.039 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "client.3", 2023-11-08T19:25:44.039 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQBP4EtlNDC8IxAACBkCM+3TXhJv5eE9xLvwKA==", 2023-11-08T19:25:44.039 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-08T19:25:44.040 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2023-11-08T19:25:44.040 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow r", 2023-11-08T19:25:44.040 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow rw", 2023-11-08T19:25:44.040 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow rwx" 2023-11-08T19:25:44.040 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-08T19:25:44.040 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-08T19:25:44.041 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-08T19:25:44.041 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "client.admin", 2023-11-08T19:25:44.041 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQBO4Etlr5yuHRAAblpqD18z9bT3Wmgn74dehg==", 2023-11-08T19:25:44.041 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-08T19:25:44.041 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow *", 2023-11-08T19:25:44.042 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow *", 2023-11-08T19:25:44.042 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow *", 2023-11-08T19:25:44.042 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-08T19:25:44.042 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-08T19:25:44.042 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-08T19:25:44.043 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-08T19:25:44.043 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "client.bootstrap-mds", 2023-11-08T19:25:44.043 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQBw4Etl+KVKMRAAWKHuOTb9O6KeWqRPOAQOUg==", 2023-11-08T19:25:44.043 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-08T19:25:44.043 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile bootstrap-mds" 2023-11-08T19:25:44.044 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-08T19:25:44.044 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-08T19:25:44.044 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-08T19:25:44.044 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "client.bootstrap-mgr", 2023-11-08T19:25:44.045 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQBw4EtlQ7BKMRAAbQVRkNQDRT8J+RPqIM29Cw==", 2023-11-08T19:25:44.045 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-08T19:25:44.045 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile bootstrap-mgr" 2023-11-08T19:25:44.045 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-08T19:25:44.045 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-08T19:25:44.046 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-08T19:25:44.046 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "client.bootstrap-osd", 2023-11-08T19:25:44.046 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQBw4Etl4bdKMRAA/QBwTfm7Styt9fy59fUWIw==", 2023-11-08T19:25:44.046 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-08T19:25:44.046 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile bootstrap-osd" 2023-11-08T19:25:44.047 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-08T19:25:44.047 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-08T19:25:44.047 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-08T19:25:44.047 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "client.bootstrap-rbd", 2023-11-08T19:25:44.047 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQBw4EtlL79KMRAAwefjy7zrKoLXa2AU+DhfkQ==", 2023-11-08T19:25:44.047 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-08T19:25:44.048 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile bootstrap-rbd" 2023-11-08T19:25:44.048 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-08T19:25:44.048 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-08T19:25:44.048 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-08T19:25:44.048 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "client.bootstrap-rbd-mirror", 2023-11-08T19:25:44.049 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQBw4EtlscZKMRAASoZI4J38XG4xT3fwAA7xww==", 2023-11-08T19:25:44.049 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-08T19:25:44.049 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2023-11-08T19:25:44.049 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-08T19:25:44.049 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-08T19:25:44.050 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-08T19:25:44.050 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "client.bootstrap-rgw", 2023-11-08T19:25:44.050 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQBw4EtlQ85KMRAATDp4o3jHlftcy7oRPolfLA==", 2023-11-08T19:25:44.050 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-08T19:25:44.050 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile bootstrap-rgw" 2023-11-08T19:25:44.051 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-08T19:25:44.051 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-08T19:25:44.051 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-08T19:25:44.051 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "mgr.x", 2023-11-08T19:25:44.051 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQBO4EtlzrqBNxAA5rtcgkzroSTKuBBmeQRiig==", 2023-11-08T19:25:44.051 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-08T19:25:44.052 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow *", 2023-11-08T19:25:44.052 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile mgr", 2023-11-08T19:25:44.052 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-08T19:25:44.052 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-08T19:25:44.052 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-08T19:25:44.053 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-08T19:25:44.053 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "mgr.y", 2023-11-08T19:25:44.053 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQBO4Etl4Gl4MxAArxdUnjLDhTLO3feXHV4HyQ==", 2023-11-08T19:25:44.053 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-08T19:25:44.053 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow *", 2023-11-08T19:25:44.053 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile mgr", 2023-11-08T19:25:44.054 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-08T19:25:44.054 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-08T19:25:44.054 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-08T19:25:44.054 INFO:teuthology.orchestra.run.smithi143.stdout: ] 2023-11-08T19:25:44.054 INFO:teuthology.orchestra.run.smithi143.stdout:} 2023-11-08T19:25:44.056 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.009+0000 7fd7c69ec700 1 -- 172.21.15.143:0/4145258812 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd7a006af00 msgr2=0x7fd7a006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:44.057 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.009+0000 7fd7c69ec700 1 --2- 172.21.15.143:0/4145258812 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd7a006af00 0x7fd7a006d3c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd7b0006fd0 tx=0x7fd7b0008040 comp rx=0 tx=0).stop 2023-11-08T19:25:44.057 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.009+0000 7fd7c69ec700 1 -- 172.21.15.143:0/4145258812 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd7c00b1100 msgr2=0x7fd7c00b4940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:44.057 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.009+0000 7fd7c69ec700 1 --2- 172.21.15.143:0/4145258812 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd7c00b1100 0x7fd7c00b4940 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fd7a800c4d0 tx=0x7fd7a800c890 comp rx=0 tx=0).stop 2023-11-08T19:25:44.057 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.009+0000 7fd7c69ec700 1 -- 172.21.15.143:0/4145258812 shutdown_connections 2023-11-08T19:25:44.057 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.009+0000 7fd7c69ec700 1 --2- 172.21.15.143:0/4145258812 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd7a006af00 0x7fd7a006d3c0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:44.058 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.009+0000 7fd7c69ec700 1 --2- 172.21.15.143:0/4145258812 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd7c015f510 0x7fd7c00c8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:44.058 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.009+0000 7fd7c69ec700 1 --2- 172.21.15.143:0/4145258812 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd7c00b1100 0x7fd7c00b4940 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:44.058 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.009+0000 7fd7c69ec700 1 --2- 172.21.15.143:0/4145258812 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd7c015ba40 0x7fd7c00c8890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:44.058 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.009+0000 7fd7c69ec700 1 -- 172.21.15.143:0/4145258812 >> 172.21.15.143:0/4145258812 conn(0x7fd7c006b4c0 msgr2=0x7fd7c00712e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:44.058 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.009+0000 7fd7c69ec700 1 -- 172.21.15.143:0/4145258812 shutdown_connections 2023-11-08T19:25:44.059 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.009+0000 7fd7c69ec700 1 -- 172.21.15.143:0/4145258812 wait complete. 2023-11-08T19:25:44.059 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2023-11-08T19:25:44.182 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.181+0000 7fe3a4f7b700 1 Processor -- start 2023-11-08T19:25:44.183 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.181+0000 7fe3a4f7b700 1 -- start start 2023-11-08T19:25:44.186 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.182+0000 7fe3a4f7b700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fe3a00cc530 0x7fe3a00cf510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:44.186 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.182+0000 7fe3a4f7b700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3a00cf080 con 0x7fe3a00b2920 2023-11-08T19:25:44.187 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.182+0000 7fe3a4f7b700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3a00ce470 con 0x7fe3a015aa30 2023-11-08T19:25:44.187 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.182+0000 7fe3a4f7b700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3a00ce750 con 0x7fe3a00cc530 2023-11-08T19:25:44.187 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.182+0000 7fe39dd9b700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fe3a00b2920 0x7fe3a00cbe20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:46540/0 (socket says 172.21.15.143:46540) 2023-11-08T19:25:44.187 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.182+0000 7fe39dd9b700 1 -- 172.21.15.143:0/403289621 learned_addr learned my addr 172.21.15.143:0/403289621 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:25:44.188 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.182+0000 7fe39ed9d700 1 --2- 172.21.15.143:0/403289621 >> v2:172.21.15.174:3300/0 conn(0x7fe3a00cc530 0x7fe3a00cf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:44.188 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.182+0000 7fe39d59a700 1 -- 172.21.15.143:0/403289621 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1125985420 0 0) 0x7fe3a00cf080 con 0x7fe3a00b2920 2023-11-08T19:25:44.188 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.182+0000 7fe39d59a700 1 -- 172.21.15.143:0/403289621 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe3880035b0 con 0x7fe3a00b2920 2023-11-08T19:25:44.188 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.182+0000 7fe39d59a700 1 -- 172.21.15.143:0/403289621 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2355224767 0 0) 0x7fe3880035b0 con 0x7fe3a00b2920 2023-11-08T19:25:44.188 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.182+0000 7fe39d59a700 1 -- 172.21.15.143:0/403289621 >> v2:172.21.15.174:3300/0 conn(0x7fe3a00cc530 msgr2=0x7fe3a00cf510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:44.188 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.182+0000 7fe39d59a700 1 --2- 172.21.15.143:0/403289621 >> v2:172.21.15.174:3300/0 conn(0x7fe3a00cc530 0x7fe3a00cf510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:44.189 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.182+0000 7fe39d59a700 1 -- 172.21.15.143:0/403289621 >> v1:172.21.15.174:6789/0 conn(0x7fe3a015aa30 legacy=0x7fe3a00b2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:44.189 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.182+0000 7fe39d59a700 1 -- 172.21.15.143:0/403289621 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3a00cea30 con 0x7fe3a00b2920 2023-11-08T19:25:44.189 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.182+0000 7fe39d59a700 1 -- 172.21.15.143:0/403289621 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4147756303 0 0) 0x7fe394001670 con 0x7fe3a00b2920 2023-11-08T19:25:44.189 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.182+0000 7fe39d59a700 1 -- 172.21.15.143:0/403289621 >> v1:172.21.15.143:6789/0 conn(0x7fe3a00b2920 legacy=0x7fe3a00cbe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:44.189 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.182+0000 7fe39d59a700 1 --2- 172.21.15.143:0/403289621 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe388003a00 0x7fe388003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:44.190 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.182+0000 7fe39d59a700 1 --2- 172.21.15.143:0/403289621 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe388004400 0x7fe3880048d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:44.190 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.183+0000 7fe39d59a700 1 --2- 172.21.15.143:0/403289621 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe388005170 0x7fe38800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:44.190 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.183+0000 7fe39d59a700 1 -- 172.21.15.143:0/403289621 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a00ce750 con 0x7fe388005170 2023-11-08T19:25:44.190 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.183+0000 7fe39d59a700 1 -- 172.21.15.143:0/403289621 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe38800c0b0 con 0x7fe388004400 2023-11-08T19:25:44.190 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.183+0000 7fe39d59a700 1 -- 172.21.15.143:0/403289621 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe38800c270 con 0x7fe388003a00 2023-11-08T19:25:44.191 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.183+0000 7fe39ed9d700 1 --2- 172.21.15.143:0/403289621 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe388005170 0x7fe38800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:44.191 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.183+0000 7fe39dd9b700 1 --2- 172.21.15.143:0/403289621 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe388004400 0x7fe3880048d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:44.191 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.183+0000 7fe39e59c700 1 --2- 172.21.15.143:0/403289621 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe388003a00 0x7fe388003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:44.191 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.183+0000 7fe39ed9d700 1 -- 172.21.15.143:0/403289621 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe388003a00 msgr2=0x7fe388003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:44.191 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.183+0000 7fe39ed9d700 1 --2- 172.21.15.143:0/403289621 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe388003a00 0x7fe388003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:44.192 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.183+0000 7fe39ed9d700 1 -- 172.21.15.143:0/403289621 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe388004400 msgr2=0x7fe3880048d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:44.192 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.183+0000 7fe39ed9d700 1 --2- 172.21.15.143:0/403289621 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe388004400 0x7fe3880048d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:44.192 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.183+0000 7fe39ed9d700 1 -- 172.21.15.143:0/403289621 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3880035b0 con 0x7fe388005170 2023-11-08T19:25:44.192 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.183+0000 7fe39dd9b700 1 --2- 172.21.15.143:0/403289621 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe388004400 0x7fe3880048d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:25:44.192 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.183+0000 7fe39ed9d700 1 -- 172.21.15.143:0/403289621 reap_dead start 2023-11-08T19:25:44.193 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.183+0000 7fe39ed9d700 1 --2- 172.21.15.143:0/403289621 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe388005170 0x7fe38800ba10 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fe398009fc0 tx=0x7fe388005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afa911b4caf17eb2 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:44.193 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.183+0000 7fe39d59a700 1 -- 172.21.15.143:0/403289621 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe398003a20 con 0x7fe388005170 2023-11-08T19:25:44.193 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.183+0000 7fe39d59a700 1 -- 172.21.15.143:0/403289621 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe39800f040 con 0x7fe388005170 2023-11-08T19:25:44.193 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.183+0000 7fe39d59a700 1 -- 172.21.15.143:0/403289621 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe398003de0 con 0x7fe388005170 2023-11-08T19:25:44.193 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.184+0000 7fe3a4f7b700 1 -- 172.21.15.143:0/403289621 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe388005170 msgr2=0x7fe38800ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:44.194 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.184+0000 7fe3a4f7b700 1 --2- 172.21.15.143:0/403289621 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe388005170 0x7fe38800ba10 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fe398009fc0 tx=0x7fe388005880 comp rx=0 tx=0).stop 2023-11-08T19:25:44.194 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.184+0000 7fe3a4f7b700 1 -- 172.21.15.143:0/403289621 shutdown_connections 2023-11-08T19:25:44.194 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.184+0000 7fe3a4f7b700 1 --2- 172.21.15.143:0/403289621 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe388005170 0x7fe38800ba10 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:44.194 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.184+0000 7fe3a4f7b700 1 -- 172.21.15.143:0/403289621 >> 172.21.15.143:0/403289621 conn(0x7fe3a0099ab0 msgr2=0x7fe3a00d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:44.194 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.184+0000 7fe3a4f7b700 1 -- 172.21.15.143:0/403289621 shutdown_connections 2023-11-08T19:25:44.195 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.184+0000 7fe3a4f7b700 1 -- 172.21.15.143:0/403289621 wait complete. 2023-11-08T19:25:44.195 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.185+0000 7fe3a4f7b700 1 Processor -- start 2023-11-08T19:25:44.195 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.185+0000 7fe3a4f7b700 1 -- start start 2023-11-08T19:25:44.198 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.186+0000 7fe3a4f7b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe388005170 0x7fe3a01f10c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:44.198 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.186+0000 7fe3a4f7b700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe3a00c9770 0x7fe3a01f4da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:44.198 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.186+0000 7fe3a4f7b700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe3a01f2080 0x7fe3a006ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:44.198 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.186+0000 7fe3a4f7b700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a01f18c0 con 0x7fe3a00c9770 2023-11-08T19:25:44.198 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.186+0000 7fe3a4f7b700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a01f1600 con 0x7fe388005170 2023-11-08T19:25:44.199 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.186+0000 7fe3a4f7b700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a01f56b0 con 0x7fe3a01f2080 2023-11-08T19:25:44.199 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.186+0000 7fe39dd9b700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe3a00c9770 0x7fe3a01f4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:44.199 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.186+0000 7fe39dd9b700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe3a00c9770 0x7fe3a01f4da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50398/0 (socket says 172.21.15.143:50398) 2023-11-08T19:25:44.199 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.186+0000 7fe39dd9b700 1 -- 172.21.15.143:0/982612533 learned_addr learned my addr 172.21.15.143:0/982612533 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:44.199 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.186+0000 7fe39e59c700 1 --2- 172.21.15.143:0/982612533 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe388005170 0x7fe3a01f10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:44.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.186+0000 7fe39ed9d700 1 --2- 172.21.15.143:0/982612533 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe3a01f2080 0x7fe3a006ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:44.203 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.186+0000 7fe39dd9b700 1 -- 172.21.15.143:0/982612533 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe3a01f2080 msgr2=0x7fe3a006ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:44.203 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.186+0000 7fe39dd9b700 1 --2- 172.21.15.143:0/982612533 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe3a01f2080 0x7fe3a006ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:44.203 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.186+0000 7fe39dd9b700 1 -- 172.21.15.143:0/982612533 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe388005170 msgr2=0x7fe3a01f10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:44.203 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.186+0000 7fe39dd9b700 1 --2- 172.21.15.143:0/982612533 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe388005170 0x7fe3a01f10c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:44.204 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.186+0000 7fe39dd9b700 1 -- 172.21.15.143:0/982612533 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3a01f5990 con 0x7fe3a00c9770 2023-11-08T19:25:44.204 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.186+0000 7fe39e59c700 1 --2- 172.21.15.143:0/982612533 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe388005170 0x7fe3a01f10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:25:44.204 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.186+0000 7fe39dd9b700 1 --2- 172.21.15.143:0/982612533 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe3a00c9770 0x7fe3a01f4da0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fe394000fd0 tx=0x7fe39400eb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:44.205 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.187+0000 7fe38effd700 1 -- 172.21.15.143:0/982612533 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe394012590 con 0x7fe3a00c9770 2023-11-08T19:25:44.205 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.187+0000 7fe38effd700 1 -- 172.21.15.143:0/982612533 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3940126f0 con 0x7fe3a00c9770 2023-11-08T19:25:44.205 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.187+0000 7fe3a4f7b700 1 -- 172.21.15.143:0/982612533 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3a01f1b60 con 0x7fe3a00c9770 2023-11-08T19:25:44.205 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.187+0000 7fe38effd700 1 -- 172.21.15.143:0/982612533 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe394012a60 con 0x7fe3a00c9770 2023-11-08T19:25:44.205 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.187+0000 7fe3a4f7b700 1 -- 172.21.15.143:0/982612533 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3a01f52e0 con 0x7fe3a00c9770 2023-11-08T19:25:44.206 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.187+0000 7fe3a4f7b700 1 -- 172.21.15.143:0/982612533 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3a004eee0 con 0x7fe3a00c9770 2023-11-08T19:25:44.206 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.188+0000 7fe38effd700 1 -- 172.21.15.143:0/982612533 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe394012bc0 con 0x7fe3a00c9770 2023-11-08T19:25:44.206 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.188+0000 7fe38effd700 1 --2- 172.21.15.143:0/982612533 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fe38006af30 0x7fe38006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:44.206 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.188+0000 7fe38effd700 1 -- 172.21.15.143:0/982612533 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fe394088f80 con 0x7fe3a00c9770 2023-11-08T19:25:44.206 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.189+0000 7fe39e59c700 1 --2- 172.21.15.143:0/982612533 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fe38006af30 0x7fe38006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:44.207 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.189+0000 7fe39e59c700 1 --2- 172.21.15.143:0/982612533 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fe38006af30 0x7fe38006d3f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe390007fd0 tx=0x7fe390009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:44.207 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.193+0000 7fe38effd700 1 -- 172.21.15.143:0/982612533 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe3940bbca0 con 0x7fe3a00c9770 2023-11-08T19:25:44.460 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.459+0000 7fe3a4f7b700 1 -- 172.21.15.143:0/982612533 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7fe3a00b2ff0 con 0x7fe3a00c9770 2023-11-08T19:25:44.462 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.461+0000 7fe38effd700 1 -- 172.21.15.143:0/982612533 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v3) v1 ==== 90+0+0 (secure 0 0 0) 0x7fe394057e10 con 0x7fe3a00c9770 2023-11-08T19:25:44.465 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.463+0000 7fe3a4f7b700 1 -- 172.21.15.143:0/982612533 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fe38006af30 msgr2=0x7fe38006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:44.465 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.463+0000 7fe3a4f7b700 1 --2- 172.21.15.143:0/982612533 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fe38006af30 0x7fe38006d3f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe390007fd0 tx=0x7fe390009040 comp rx=0 tx=0).stop 2023-11-08T19:25:44.465 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.463+0000 7fe3a4f7b700 1 -- 172.21.15.143:0/982612533 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe3a00c9770 msgr2=0x7fe3a01f4da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:44.466 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.463+0000 7fe3a4f7b700 1 --2- 172.21.15.143:0/982612533 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe3a00c9770 0x7fe3a01f4da0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fe394000fd0 tx=0x7fe39400eb10 comp rx=0 tx=0).stop 2023-11-08T19:25:44.466 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.463+0000 7fe3a4f7b700 1 -- 172.21.15.143:0/982612533 shutdown_connections 2023-11-08T19:25:44.466 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.463+0000 7fe3a4f7b700 1 --2- 172.21.15.143:0/982612533 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fe38006af30 0x7fe38006d3f0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:44.466 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.463+0000 7fe3a4f7b700 1 --2- 172.21.15.143:0/982612533 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe3a01f2080 0x7fe3a006ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:44.466 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.463+0000 7fe3a4f7b700 1 --2- 172.21.15.143:0/982612533 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe388005170 0x7fe3a01f10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:44.467 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.463+0000 7fe3a4f7b700 1 --2- 172.21.15.143:0/982612533 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe3a00c9770 0x7fe3a01f4da0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:44.467 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.463+0000 7fe3a4f7b700 1 -- 172.21.15.143:0/982612533 >> 172.21.15.143:0/982612533 conn(0x7fe3a0099ab0 msgr2=0x7fe3a0158a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:44.467 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.463+0000 7fe3a4f7b700 1 -- 172.21.15.143:0/982612533 shutdown_connections 2023-11-08T19:25:44.467 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.463+0000 7fe3a4f7b700 1 -- 172.21.15.143:0/982612533 wait complete. 2023-11-08T19:25:44.477 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2023-11-08T19:25:44.598 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.596+0000 7f5a20256700 1 Processor -- start 2023-11-08T19:25:44.598 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.597+0000 7f5a20256700 1 -- start start 2023-11-08T19:25:44.600 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.597+0000 7f5a20256700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f5a180cf090 0x7f5a180d1f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:44.600 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.597+0000 7f5a20256700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5a180d3690 0x7f5a180b9690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:44.600 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.597+0000 7f5a20256700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a180d3200 con 0x7f5a1815dc80 2023-11-08T19:25:44.600 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.597+0000 7f5a20256700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a180b9bd0 con 0x7f5a180cf090 2023-11-08T19:25:44.600 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.597+0000 7f5a20256700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a180b7cf0 con 0x7f5a180d3690 2023-11-08T19:25:44.601 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.598+0000 7f5a1e7f3700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5a180d3690 0x7f5a180b9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:44.601 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.598+0000 7f5a1d7f1700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f5a180cf090 0x7f5a180d1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:44.601 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.598+0000 7f5a1dff2700 1 --1- >> v1:172.21.15.174:6789/0 conn(0x7f5a1815dc80 0x7f5a180ce980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.174:6789/0 says I am v1:172.21.15.143:57492/0 (socket says 172.21.15.143:57492) 2023-11-08T19:25:44.601 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.598+0000 7f5a1dff2700 1 -- 172.21.15.143:0/2223284678 learned_addr learned my addr 172.21.15.143:0/2223284678 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:25:44.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.598+0000 7f5a1d7f1700 1 -- 172.21.15.143:0/2223284678 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5a180d3690 msgr2=0x7f5a180b9690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:44.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.598+0000 7f5a1d7f1700 1 --2- 172.21.15.143:0/2223284678 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5a180d3690 0x7f5a180b9690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:44.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.598+0000 7f5a1d7f1700 1 -- 172.21.15.143:0/2223284678 >> v1:172.21.15.174:6789/0 conn(0x7f5a1815dc80 legacy=0x7f5a180ce980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:44.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.598+0000 7f5a1d7f1700 1 -- 172.21.15.143:0/2223284678 --> v2:172.21.15.174:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a1815e060 con 0x7f5a180cf090 2023-11-08T19:25:44.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.598+0000 7f5a1d7f1700 1 --2- 172.21.15.143:0/2223284678 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5a180cf090 0x7f5a180d1f80 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5a14009840 tx=0x7f5a14009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=68bf52112e4e1f9a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:44.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.598+0000 7f5a1cff0700 1 -- 172.21.15.143:0/2223284678 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a14011830 con 0x7f5a180cf090 2023-11-08T19:25:44.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.599+0000 7f5a1cff0700 1 -- 172.21.15.143:0/2223284678 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a14011990 con 0x7f5a180cf090 2023-11-08T19:25:44.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.599+0000 7f5a1cff0700 1 -- 172.21.15.143:0/2223284678 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a14011d20 con 0x7f5a180cf090 2023-11-08T19:25:44.606 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.599+0000 7f5a20256700 1 -- 172.21.15.143:0/2223284678 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5a180cf090 msgr2=0x7f5a180d1f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:44.609 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.599+0000 7f5a20256700 1 --2- 172.21.15.143:0/2223284678 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5a180cf090 0x7f5a180d1f80 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5a14009840 tx=0x7f5a14009b50 comp rx=0 tx=0).stop 2023-11-08T19:25:44.609 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.599+0000 7f5a20256700 1 -- 172.21.15.143:0/2223284678 shutdown_connections 2023-11-08T19:25:44.609 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.599+0000 7f5a20256700 1 --2- 172.21.15.143:0/2223284678 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5a180d3690 0x7f5a180b9690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:44.610 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.599+0000 7f5a20256700 1 --2- 172.21.15.143:0/2223284678 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5a180cf090 0x7f5a180d1f80 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:44.610 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.599+0000 7f5a20256700 1 -- 172.21.15.143:0/2223284678 >> 172.21.15.143:0/2223284678 conn(0x7f5a180668e0 msgr2=0x7f5a18066cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:44.610 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.599+0000 7f5a20256700 1 -- 172.21.15.143:0/2223284678 shutdown_connections 2023-11-08T19:25:44.610 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.599+0000 7f5a20256700 1 -- 172.21.15.143:0/2223284678 wait complete. 2023-11-08T19:25:44.610 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.600+0000 7f5a20256700 1 Processor -- start 2023-11-08T19:25:44.611 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.600+0000 7f5a20256700 1 -- start start 2023-11-08T19:25:44.611 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.601+0000 7f5a20256700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5a180cf090 0x7f5a1815f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:44.611 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.601+0000 7f5a20256700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5a180d3690 0x7f5a181f2f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:44.611 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.601+0000 7f5a20256700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5a1815dc80 0x7f5a180c8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:44.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.601+0000 7f5a20256700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a1815e7f0 con 0x7f5a180cf090 2023-11-08T19:25:44.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.601+0000 7f5a20256700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a1815ead0 con 0x7f5a1815dc80 2023-11-08T19:25:44.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.601+0000 7f5a20256700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a1815edb0 con 0x7f5a180d3690 2023-11-08T19:25:44.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.601+0000 7f5a1dff2700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5a180cf090 0x7f5a1815f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:44.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.601+0000 7f5a1dff2700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5a180cf090 0x7f5a1815f890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50410/0 (socket says 172.21.15.143:50410) 2023-11-08T19:25:44.613 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.601+0000 7f5a1dff2700 1 -- 172.21.15.143:0/396543034 learned_addr learned my addr 172.21.15.143:0/396543034 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:44.613 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.601+0000 7f5a1d7f1700 1 --2- 172.21.15.143:0/396543034 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5a180d3690 0x7f5a181f2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:44.613 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.601+0000 7f5a1e7f3700 1 --2- 172.21.15.143:0/396543034 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5a1815dc80 0x7f5a180c8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:44.613 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.602+0000 7f5a1dff2700 1 -- 172.21.15.143:0/396543034 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5a180d3690 msgr2=0x7f5a181f2f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:44.613 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.602+0000 7f5a1dff2700 1 --2- 172.21.15.143:0/396543034 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5a180d3690 0x7f5a181f2f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:44.614 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.602+0000 7f5a1dff2700 1 -- 172.21.15.143:0/396543034 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5a1815dc80 msgr2=0x7f5a180c8c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:44.614 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.602+0000 7f5a1dff2700 1 --2- 172.21.15.143:0/396543034 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5a1815dc80 0x7f5a180c8c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:44.614 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.602+0000 7f5a1dff2700 1 -- 172.21.15.143:0/396543034 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a140094a0 con 0x7f5a180cf090 2023-11-08T19:25:44.614 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.602+0000 7f5a1e7f3700 1 --2- 172.21.15.143:0/396543034 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5a1815dc80 0x7f5a180c8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:25:44.614 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.602+0000 7f5a1dff2700 1 --2- 172.21.15.143:0/396543034 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5a180cf090 0x7f5a1815f890 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f5a0800c480 tx=0x7f5a0800c790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:44.615 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.602+0000 7f5a0e7fc700 1 -- 172.21.15.143:0/396543034 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a08005d70 con 0x7f5a180cf090 2023-11-08T19:25:44.615 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.602+0000 7f5a0e7fc700 1 -- 172.21.15.143:0/396543034 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a080070d0 con 0x7f5a180cf090 2023-11-08T19:25:44.615 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.602+0000 7f5a20256700 1 -- 172.21.15.143:0/396543034 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a1815f090 con 0x7f5a180cf090 2023-11-08T19:25:44.615 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.602+0000 7f5a0e7fc700 1 -- 172.21.15.143:0/396543034 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a08007440 con 0x7f5a180cf090 2023-11-08T19:25:44.615 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.602+0000 7f5a20256700 1 -- 172.21.15.143:0/396543034 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a18160040 con 0x7f5a180cf090 2023-11-08T19:25:44.615 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.603+0000 7f5a20256700 1 -- 172.21.15.143:0/396543034 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a181601d0 con 0x7f5a180cf090 2023-11-08T19:25:44.616 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.603+0000 7f5a0e7fc700 1 -- 172.21.15.143:0/396543034 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5a080075a0 con 0x7f5a180cf090 2023-11-08T19:25:44.616 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.604+0000 7f5a0e7fc700 1 --2- 172.21.15.143:0/396543034 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f59fc06af30 0x7f59fc06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:44.616 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.604+0000 7f5a0e7fc700 1 -- 172.21.15.143:0/396543034 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f5a08088520 con 0x7f5a180cf090 2023-11-08T19:25:44.616 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.604+0000 7f5a1d7f1700 1 --2- 172.21.15.143:0/396543034 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f59fc06af30 0x7f59fc06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:44.616 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.604+0000 7f5a1d7f1700 1 --2- 172.21.15.143:0/396543034 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f59fc06af30 0x7f59fc06d3f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f5a14000c00 tx=0x7f5a14008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:44.617 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.608+0000 7f5a0e7fc700 1 -- 172.21.15.143:0/396543034 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5a08053a90 con 0x7f5a180cf090 2023-11-08T19:25:44.875 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.874+0000 7f5a20256700 1 -- 172.21.15.143:0/396543034 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f5a1804eee0 con 0x7f5a180cf090 2023-11-08T19:25:44.877 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.876+0000 7f5a0e7fc700 1 -- 172.21.15.143:0/396543034 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v4) v1 ==== 90+0+0 (secure 0 0 0) 0x7f5a080570b0 con 0x7f5a180cf090 2023-11-08T19:25:44.881 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.879+0000 7f5a20256700 1 -- 172.21.15.143:0/396543034 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f59fc06af30 msgr2=0x7f59fc06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:44.881 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.879+0000 7f5a20256700 1 --2- 172.21.15.143:0/396543034 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f59fc06af30 0x7f59fc06d3f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f5a14000c00 tx=0x7f5a14008030 comp rx=0 tx=0).stop 2023-11-08T19:25:44.882 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.879+0000 7f5a20256700 1 -- 172.21.15.143:0/396543034 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5a180cf090 msgr2=0x7f5a1815f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:44.882 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.879+0000 7f5a20256700 1 --2- 172.21.15.143:0/396543034 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5a180cf090 0x7f5a1815f890 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f5a0800c480 tx=0x7f5a0800c790 comp rx=0 tx=0).stop 2023-11-08T19:25:44.882 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.879+0000 7f5a20256700 1 -- 172.21.15.143:0/396543034 shutdown_connections 2023-11-08T19:25:44.882 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.879+0000 7f5a20256700 1 --2- 172.21.15.143:0/396543034 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f59fc06af30 0x7f59fc06d3f0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:44.883 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.879+0000 7f5a20256700 1 --2- 172.21.15.143:0/396543034 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5a1815dc80 0x7f5a180c8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:44.883 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.879+0000 7f5a20256700 1 --2- 172.21.15.143:0/396543034 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5a180cf090 0x7f5a1815f890 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:44.883 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.879+0000 7f5a20256700 1 --2- 172.21.15.143:0/396543034 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5a180d3690 0x7f5a181f2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:44.883 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.879+0000 7f5a20256700 1 -- 172.21.15.143:0/396543034 >> 172.21.15.143:0/396543034 conn(0x7f5a180668e0 msgr2=0x7f5a1815cfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:44.883 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.879+0000 7f5a20256700 1 -- 172.21.15.143:0/396543034 shutdown_connections 2023-11-08T19:25:44.884 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:44.879+0000 7f5a20256700 1 -- 172.21.15.143:0/396543034 wait complete. 2023-11-08T19:25:44.893 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2023-11-08T19:25:45.014 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.012+0000 7f77cdec5700 1 Processor -- start 2023-11-08T19:25:45.014 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.012+0000 7f77cdec5700 1 -- start start 2023-11-08T19:25:45.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.012+0000 7f77cdec5700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f77c815dd00 0x7f77c80cca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:45.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.012+0000 7f77cdec5700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f77c806b320 0x7f77c80cb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:45.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.012+0000 7f77cdec5700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77c80ccfb0 con 0x7f77c81618f0 2023-11-08T19:25:45.024 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.012+0000 7f77cdec5700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77c80cad50 con 0x7f77c806b320 2023-11-08T19:25:45.024 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.013+0000 7f77cdec5700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77c8161df0 con 0x7f77c815dd00 2023-11-08T19:25:45.024 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.013+0000 7f77c6ffd700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f77c81618f0 0x7f77c806ac10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:46574/0 (socket says 172.21.15.143:46574) 2023-11-08T19:25:45.024 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.013+0000 7f77c6ffd700 1 -- 172.21.15.143:0/418737189 learned_addr learned my addr 172.21.15.143:0/418737189 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:25:45.025 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.013+0000 7f77c77fe700 1 --2- 172.21.15.143:0/418737189 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f77c815dd00 0x7f77c80cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:45.025 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.013+0000 7f77c7fff700 1 --2- 172.21.15.143:0/418737189 >> v2:172.21.15.174:3300/0 conn(0x7f77c806b320 0x7f77c80cb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:45.025 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.013+0000 7f77c67fc700 1 -- 172.21.15.143:0/418737189 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4034333174 0 0) 0x7f77c80ccfb0 con 0x7f77c81618f0 2023-11-08T19:25:45.025 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.013+0000 7f77c67fc700 1 -- 172.21.15.143:0/418737189 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f77b00035b0 con 0x7f77c81618f0 2023-11-08T19:25:45.026 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.013+0000 7f77c67fc700 1 -- 172.21.15.143:0/418737189 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4009734482 0 0) 0x7f77b00035b0 con 0x7f77c81618f0 2023-11-08T19:25:45.026 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.013+0000 7f77c67fc700 1 -- 172.21.15.143:0/418737189 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f77c815dd00 msgr2=0x7f77c80cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:45.026 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.013+0000 7f77c67fc700 1 --2- 172.21.15.143:0/418737189 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f77c815dd00 0x7f77c80cca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:45.026 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.013+0000 7f77c67fc700 1 -- 172.21.15.143:0/418737189 >> v2:172.21.15.174:3300/0 conn(0x7f77c806b320 msgr2=0x7f77c80cb1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:45.027 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.013+0000 7f77c67fc700 1 --2- 172.21.15.143:0/418737189 >> v2:172.21.15.174:3300/0 conn(0x7f77c806b320 0x7f77c80cb1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:45.027 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.013+0000 7f77c67fc700 1 -- 172.21.15.143:0/418737189 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77c81620d0 con 0x7f77c81618f0 2023-11-08T19:25:45.027 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.013+0000 7f77c67fc700 1 -- 172.21.15.143:0/418737189 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4147756303 0 0) 0x7f77bc001670 con 0x7f77c81618f0 2023-11-08T19:25:45.027 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.013+0000 7f77c67fc700 1 -- 172.21.15.143:0/418737189 >> v1:172.21.15.143:6789/0 conn(0x7f77c81618f0 legacy=0x7f77c806ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:45.027 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.013+0000 7f77c67fc700 1 --2- 172.21.15.143:0/418737189 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f77b0003a00 0x7f77b0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:45.027 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.013+0000 7f77c67fc700 1 --2- 172.21.15.143:0/418737189 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f77b0004850 0x7f77b0004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:45.028 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.013+0000 7f77c67fc700 1 --2- 172.21.15.143:0/418737189 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f77c815dd00 0x7f77b0005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:45.028 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.013+0000 7f77c67fc700 1 -- 172.21.15.143:0/418737189 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77c8161df0 con 0x7f77b0003a00 2023-11-08T19:25:45.028 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.013+0000 7f77c67fc700 1 -- 172.21.15.143:0/418737189 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77b000ba10 con 0x7f77b0004850 2023-11-08T19:25:45.029 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.013+0000 7f77c67fc700 1 -- 172.21.15.143:0/418737189 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77b000bb50 con 0x7f77c815dd00 2023-11-08T19:25:45.029 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.013+0000 7f77c77fe700 1 --2- 172.21.15.143:0/418737189 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f77b0003a00 0x7f77b0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:45.029 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.013+0000 7f77c6ffd700 1 --2- 172.21.15.143:0/418737189 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f77b0004850 0x7f77b0004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:45.030 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.013+0000 7f77c7fff700 1 --2- 172.21.15.143:0/418737189 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f77c815dd00 0x7f77b0005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:45.030 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.014+0000 7f77c77fe700 1 -- 172.21.15.143:0/418737189 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f77c815dd00 msgr2=0x7f77b0005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:45.030 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.014+0000 7f77c77fe700 1 --2- 172.21.15.143:0/418737189 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f77c815dd00 0x7f77b0005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:45.030 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.014+0000 7f77c77fe700 1 -- 172.21.15.143:0/418737189 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f77b0004850 msgr2=0x7f77b0004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:45.030 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.014+0000 7f77c77fe700 1 --2- 172.21.15.143:0/418737189 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f77b0004850 0x7f77b0004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:45.031 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.014+0000 7f77c77fe700 1 -- 172.21.15.143:0/418737189 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77b00035b0 con 0x7f77b0003a00 2023-11-08T19:25:45.031 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.014+0000 7f77c6ffd700 1 --2- 172.21.15.143:0/418737189 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f77b0004850 0x7f77b0004cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:25:45.031 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.014+0000 7f77c77fe700 1 --2- 172.21.15.143:0/418737189 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f77b0003a00 0x7f77b0003e20 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f77b8009b40 tx=0x7f77b0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de06715479de7428 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:45.031 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.014+0000 7f77c67fc700 1 -- 172.21.15.143:0/418737189 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77b8005a20 con 0x7f77b0003a00 2023-11-08T19:25:45.031 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.014+0000 7f77c67fc700 1 -- 172.21.15.143:0/418737189 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77b8005b80 con 0x7f77b0003a00 2023-11-08T19:25:45.032 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.014+0000 7f77c67fc700 1 -- 172.21.15.143:0/418737189 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77b800e6c0 con 0x7f77b0003a00 2023-11-08T19:25:45.032 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.014+0000 7f77cdec5700 1 -- 172.21.15.143:0/418737189 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f77b0003a00 msgr2=0x7f77b0003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:45.032 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.014+0000 7f77cdec5700 1 --2- 172.21.15.143:0/418737189 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f77b0003a00 0x7f77b0003e20 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f77b8009b40 tx=0x7f77b0005880 comp rx=0 tx=0).stop 2023-11-08T19:25:45.032 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.014+0000 7f77c7fff700 1 -- 172.21.15.143:0/418737189 reap_dead start 2023-11-08T19:25:45.032 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.014+0000 7f77cdec5700 1 -- 172.21.15.143:0/418737189 shutdown_connections 2023-11-08T19:25:45.033 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.014+0000 7f77cdec5700 1 -- 172.21.15.143:0/418737189 >> 172.21.15.143:0/418737189 conn(0x7f77c8099ab0 msgr2=0x7f77c80d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:45.033 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.014+0000 7f77cdec5700 1 -- 172.21.15.143:0/418737189 shutdown_connections 2023-11-08T19:25:45.033 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.014+0000 7f77cdec5700 1 -- 172.21.15.143:0/418737189 wait complete. 2023-11-08T19:25:45.033 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.016+0000 7f77cdec5700 1 Processor -- start 2023-11-08T19:25:45.033 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.016+0000 7f77cdec5700 1 -- start start 2023-11-08T19:25:45.034 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.017+0000 7f77cdec5700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f77b0003a00 0x7f77c814b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:45.034 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.017+0000 7f77cdec5700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f77c80b29e0 0x7f77c8147830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:45.034 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.017+0000 7f77cdec5700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f77c80b2e60 0x7f77c8147d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:45.034 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.017+0000 7f77cdec5700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77c80b25c0 con 0x7f77c80b2e60 2023-11-08T19:25:45.035 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.017+0000 7f77cdec5700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77c814b6f0 con 0x7f77c80b29e0 2023-11-08T19:25:45.035 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.017+0000 7f77cdec5700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77c8148400 con 0x7f77b0003a00 2023-11-08T19:25:45.035 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.017+0000 7f77c7fff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f77c80b2e60 0x7f77c8147d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:45.036 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.017+0000 7f77c7fff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f77c80b2e60 0x7f77c8147d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50434/0 (socket says 172.21.15.143:50434) 2023-11-08T19:25:45.036 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.017+0000 7f77c7fff700 1 -- 172.21.15.143:0/2310167540 learned_addr learned my addr 172.21.15.143:0/2310167540 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:45.036 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.017+0000 7f77c6ffd700 1 --2- 172.21.15.143:0/2310167540 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f77c80b29e0 0x7f77c8147830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:45.037 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.017+0000 7f77c77fe700 1 --2- 172.21.15.143:0/2310167540 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f77b0003a00 0x7f77c814b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:45.037 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.018+0000 7f77c7fff700 1 -- 172.21.15.143:0/2310167540 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f77b0003a00 msgr2=0x7f77c814b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:45.037 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.018+0000 7f77c7fff700 1 --2- 172.21.15.143:0/2310167540 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f77b0003a00 0x7f77c814b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:45.037 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.018+0000 7f77c7fff700 1 -- 172.21.15.143:0/2310167540 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f77c80b29e0 msgr2=0x7f77c8147830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:45.038 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.018+0000 7f77c7fff700 1 --2- 172.21.15.143:0/2310167540 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f77c80b29e0 0x7f77c8147830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:45.038 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.018+0000 7f77c7fff700 1 -- 172.21.15.143:0/2310167540 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77c81486e0 con 0x7f77c80b2e60 2023-11-08T19:25:45.038 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.018+0000 7f77c7fff700 1 --2- 172.21.15.143:0/2310167540 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f77c80b2e60 0x7f77c8147d70 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f77c8073130 tx=0x7f77c00114b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:45.039 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.018+0000 7f77affff700 1 -- 172.21.15.143:0/2310167540 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c000e070 con 0x7f77c80b2e60 2023-11-08T19:25:45.039 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.018+0000 7f77affff700 1 -- 172.21.15.143:0/2310167540 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77c0003cf0 con 0x7f77c80b2e60 2023-11-08T19:25:45.039 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.018+0000 7f77cdec5700 1 -- 172.21.15.143:0/2310167540 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77c80b21e0 con 0x7f77c80b2e60 2023-11-08T19:25:45.039 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.018+0000 7f77affff700 1 -- 172.21.15.143:0/2310167540 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c0003ea0 con 0x7f77c80b2e60 2023-11-08T19:25:45.039 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.018+0000 7f77cdec5700 1 -- 172.21.15.143:0/2310167540 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77c80b1e00 con 0x7f77c80b2e60 2023-11-08T19:25:45.040 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.019+0000 7f77cdec5700 1 -- 172.21.15.143:0/2310167540 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77c80b2700 con 0x7f77c80b2e60 2023-11-08T19:25:45.040 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.019+0000 7f77affff700 1 -- 172.21.15.143:0/2310167540 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f77c0011960 con 0x7f77c80b2e60 2023-11-08T19:25:45.040 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.020+0000 7f77affff700 1 --2- 172.21.15.143:0/2310167540 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f77b406af00 0x7f77b406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:45.040 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.020+0000 7f77affff700 1 -- 172.21.15.143:0/2310167540 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f77c0089500 con 0x7f77c80b2e60 2023-11-08T19:25:45.041 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.020+0000 7f77c77fe700 1 --2- 172.21.15.143:0/2310167540 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f77b406af00 0x7f77b406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:45.042 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.020+0000 7f77c77fe700 1 --2- 172.21.15.143:0/2310167540 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f77b406af00 0x7f77b406d3c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f77b8000c00 tx=0x7f77b8008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:45.042 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.024+0000 7f77affff700 1 -- 172.21.15.143:0/2310167540 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f77c80b2700 con 0x7f77c80b2e60 2023-11-08T19:25:45.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.289+0000 7f77cdec5700 1 -- 172.21.15.143:0/2310167540 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f77c80b2700 con 0x7f77c80b2e60 2023-11-08T19:25:45.293 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.291+0000 7f77affff700 1 -- 172.21.15.143:0/2310167540 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v5) v1 ==== 90+0+0 (secure 0 0 0) 0x7f77c80b2700 con 0x7f77c80b2e60 2023-11-08T19:25:45.296 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.293+0000 7f77cdec5700 1 -- 172.21.15.143:0/2310167540 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f77b406af00 msgr2=0x7f77b406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:45.296 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.294+0000 7f77cdec5700 1 --2- 172.21.15.143:0/2310167540 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f77b406af00 0x7f77b406d3c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f77b8000c00 tx=0x7f77b8008030 comp rx=0 tx=0).stop 2023-11-08T19:25:45.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.294+0000 7f77cdec5700 1 -- 172.21.15.143:0/2310167540 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f77c80b2e60 msgr2=0x7f77c8147d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:45.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.294+0000 7f77cdec5700 1 --2- 172.21.15.143:0/2310167540 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f77c80b2e60 0x7f77c8147d70 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f77c8073130 tx=0x7f77c00114b0 comp rx=0 tx=0).stop 2023-11-08T19:25:45.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.294+0000 7f77cdec5700 1 -- 172.21.15.143:0/2310167540 shutdown_connections 2023-11-08T19:25:45.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.294+0000 7f77cdec5700 1 --2- 172.21.15.143:0/2310167540 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f77b406af00 0x7f77b406d3c0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:45.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.294+0000 7f77cdec5700 1 --2- 172.21.15.143:0/2310167540 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f77b0003a00 0x7f77c814b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:45.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.294+0000 7f77cdec5700 1 --2- 172.21.15.143:0/2310167540 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f77c80b2e60 0x7f77c8147d70 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:45.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.294+0000 7f77cdec5700 1 --2- 172.21.15.143:0/2310167540 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f77c80b29e0 0x7f77c8147830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:45.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.294+0000 7f77cdec5700 1 -- 172.21.15.143:0/2310167540 >> 172.21.15.143:0/2310167540 conn(0x7f77c8099ab0 msgr2=0x7f77c80ad380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:45.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.294+0000 7f77cdec5700 1 -- 172.21.15.143:0/2310167540 shutdown_connections 2023-11-08T19:25:45.299 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.294+0000 7f77cdec5700 1 -- 172.21.15.143:0/2310167540 wait complete. 2023-11-08T19:25:45.308 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2023-11-08T19:25:45.429 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.427+0000 7fc9c3054700 1 Processor -- start 2023-11-08T19:25:45.430 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.427+0000 7fc9c3054700 1 -- start start 2023-11-08T19:25:45.433 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.428+0000 7fc9c3054700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fc9bc15a2d0 0x7fc9bc06ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:45.433 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.428+0000 7fc9c3054700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc9bc0ce500 0x7fc9bc0ce990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:45.433 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.428+0000 7fc9c3054700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9bc0cb5e0 con 0x7fc9bc06c200 2023-11-08T19:25:45.433 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.428+0000 7fc9c3054700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9bc0ceed0 con 0x7fc9bc0ce500 2023-11-08T19:25:45.433 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.428+0000 7fc9c3054700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9bc0cb8c0 con 0x7fc9bc15a2d0 2023-11-08T19:25:45.434 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.428+0000 7fc9c15f1700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc9bc0ce500 0x7fc9bc0ce990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:45.434 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.428+0000 7fc9c15f1700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc9bc0ce500 0x7fc9bc0ce990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50442/0 (socket says 172.21.15.143:50442) 2023-11-08T19:25:45.435 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.428+0000 7fc9c15f1700 1 -- 172.21.15.143:0/2232650270 learned_addr learned my addr 172.21.15.143:0/2232650270 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:45.435 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.428+0000 7fc9bbfff700 1 --1- 172.21.15.143:0/2232650270 >> v1:172.21.15.143:6789/0 conn(0x7fc9bc06c200 0x7fc9bc0cab70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:46594/0 (socket says 172.21.15.143:46594) 2023-11-08T19:25:45.435 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.428+0000 7fc9c0df0700 1 --2- 172.21.15.143:0/2232650270 >> v2:172.21.15.174:3300/0 conn(0x7fc9bc15a2d0 0x7fc9bc06ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:45.436 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.428+0000 7fc9c15f1700 1 -- 172.21.15.143:0/2232650270 >> v2:172.21.15.174:3300/0 conn(0x7fc9bc15a2d0 msgr2=0x7fc9bc06ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:45.436 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.428+0000 7fc9c15f1700 1 --2- 172.21.15.143:0/2232650270 >> v2:172.21.15.174:3300/0 conn(0x7fc9bc15a2d0 0x7fc9bc06ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:45.436 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.428+0000 7fc9c15f1700 1 -- 172.21.15.143:0/2232650270 >> v1:172.21.15.143:6789/0 conn(0x7fc9bc06c200 legacy=0x7fc9bc0cab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:45.436 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.428+0000 7fc9c15f1700 1 -- 172.21.15.143:0/2232650270 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9bc0cbba0 con 0x7fc9bc0ce500 2023-11-08T19:25:45.437 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.428+0000 7fc9bb7fe700 1 -- 172.21.15.143:0/2232650270 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1978015067 0 0) 0x7fc9bc0cb5e0 con 0x7fc9bc06c200 2023-11-08T19:25:45.437 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.428+0000 7fc9c15f1700 1 --2- 172.21.15.143:0/2232650270 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc9bc0ce500 0x7fc9bc0ce990 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fc9bc071820 tx=0x7fc9b400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28aec0b6af21c548 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:45.437 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.428+0000 7fc9bb7fe700 1 -- 172.21.15.143:0/2232650270 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9b400e070 con 0x7fc9bc0ce500 2023-11-08T19:25:45.437 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.429+0000 7fc9bb7fe700 1 -- 172.21.15.143:0/2232650270 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9b400b9c0 con 0x7fc9bc0ce500 2023-11-08T19:25:45.437 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.429+0000 7fc9bb7fe700 1 -- 172.21.15.143:0/2232650270 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9b400bd70 con 0x7fc9bc0ce500 2023-11-08T19:25:45.438 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.429+0000 7fc9c3054700 1 -- 172.21.15.143:0/2232650270 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc9bc0ce500 msgr2=0x7fc9bc0ce990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:45.438 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.429+0000 7fc9c3054700 1 --2- 172.21.15.143:0/2232650270 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc9bc0ce500 0x7fc9bc0ce990 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fc9bc071820 tx=0x7fc9b400b330 comp rx=0 tx=0).stop 2023-11-08T19:25:45.438 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.429+0000 7fc9c3054700 1 -- 172.21.15.143:0/2232650270 shutdown_connections 2023-11-08T19:25:45.438 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.429+0000 7fc9c3054700 1 --2- 172.21.15.143:0/2232650270 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc9bc0ce500 0x7fc9bc0ce990 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:45.439 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.429+0000 7fc9c3054700 1 --2- 172.21.15.143:0/2232650270 >> v2:172.21.15.174:3300/0 conn(0x7fc9bc15a2d0 0x7fc9bc06ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:45.439 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.429+0000 7fc9c3054700 1 -- 172.21.15.143:0/2232650270 >> 172.21.15.143:0/2232650270 conn(0x7fc9bc0b1a20 msgr2=0x7fc9bc0b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:45.439 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.429+0000 7fc9c3054700 1 -- 172.21.15.143:0/2232650270 shutdown_connections 2023-11-08T19:25:45.439 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.429+0000 7fc9c3054700 1 -- 172.21.15.143:0/2232650270 wait complete. 2023-11-08T19:25:45.439 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.430+0000 7fc9c3054700 1 Processor -- start 2023-11-08T19:25:45.439 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.430+0000 7fc9c3054700 1 -- start start 2023-11-08T19:25:45.442 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.431+0000 7fc9c3054700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc9bc06c200 0x7fc9bc146320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:45.443 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.431+0000 7fc9c3054700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc9bc0ce500 0x7fc9bc0c9500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:45.444 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.431+0000 7fc9c3054700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc9bc15a2d0 0x7fc9bc0c9a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:45.444 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.431+0000 7fc9c3054700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9bc0c7900 con 0x7fc9bc06c200 2023-11-08T19:25:45.444 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.431+0000 7fc9c3054700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9bc0c7ba0 con 0x7fc9bc15a2d0 2023-11-08T19:25:45.444 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.431+0000 7fc9c3054700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9bc0c7e80 con 0x7fc9bc0ce500 2023-11-08T19:25:45.444 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.431+0000 7fc9c0df0700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc9bc06c200 0x7fc9bc146320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:45.445 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.431+0000 7fc9c0df0700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc9bc06c200 0x7fc9bc146320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50444/0 (socket says 172.21.15.143:50444) 2023-11-08T19:25:45.445 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.431+0000 7fc9c0df0700 1 -- 172.21.15.143:0/741659174 learned_addr learned my addr 172.21.15.143:0/741659174 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:45.445 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.432+0000 7fc9c15f1700 1 --2- 172.21.15.143:0/741659174 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc9bc15a2d0 0x7fc9bc0c9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:45.445 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.432+0000 7fc9bbfff700 1 --2- 172.21.15.143:0/741659174 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc9bc0ce500 0x7fc9bc0c9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:45.445 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.432+0000 7fc9c0df0700 1 -- 172.21.15.143:0/741659174 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc9bc0ce500 msgr2=0x7fc9bc0c9500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:45.446 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.432+0000 7fc9c0df0700 1 --2- 172.21.15.143:0/741659174 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc9bc0ce500 0x7fc9bc0c9500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:45.446 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.432+0000 7fc9c0df0700 1 -- 172.21.15.143:0/741659174 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc9bc15a2d0 msgr2=0x7fc9bc0c9a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:45.446 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.432+0000 7fc9c0df0700 1 --2- 172.21.15.143:0/741659174 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc9bc15a2d0 0x7fc9bc0c9a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:45.446 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.432+0000 7fc9c0df0700 1 -- 172.21.15.143:0/741659174 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9b4009b30 con 0x7fc9bc06c200 2023-11-08T19:25:45.446 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.432+0000 7fc9c0df0700 1 --2- 172.21.15.143:0/741659174 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc9bc06c200 0x7fc9bc146320 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fc9b000d800 tx=0x7fc9b000dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:45.447 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.432+0000 7fc9b97fa700 1 -- 172.21.15.143:0/741659174 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9b00125f0 con 0x7fc9bc06c200 2023-11-08T19:25:45.447 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.432+0000 7fc9b97fa700 1 -- 172.21.15.143:0/741659174 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9b0012750 con 0x7fc9bc06c200 2023-11-08T19:25:45.447 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.432+0000 7fc9c3054700 1 -- 172.21.15.143:0/741659174 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9bc0c8160 con 0x7fc9bc06c200 2023-11-08T19:25:45.447 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.432+0000 7fc9b97fa700 1 -- 172.21.15.143:0/741659174 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9b0012b00 con 0x7fc9bc06c200 2023-11-08T19:25:45.447 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.432+0000 7fc9c3054700 1 -- 172.21.15.143:0/741659174 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9bc0c8720 con 0x7fc9bc06c200 2023-11-08T19:25:45.448 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.433+0000 7fc9c3054700 1 -- 172.21.15.143:0/741659174 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9bc0c8a90 con 0x7fc9bc06c200 2023-11-08T19:25:45.448 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.433+0000 7fc9b97fa700 1 -- 172.21.15.143:0/741659174 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc9b0012c60 con 0x7fc9bc06c200 2023-11-08T19:25:45.448 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.434+0000 7fc9b97fa700 1 --2- 172.21.15.143:0/741659174 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc9a006af00 0x7fc9a006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:45.448 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.434+0000 7fc9b97fa700 1 -- 172.21.15.143:0/741659174 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fc9b00895a0 con 0x7fc9bc06c200 2023-11-08T19:25:45.448 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.434+0000 7fc9bbfff700 1 --2- 172.21.15.143:0/741659174 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc9a006af00 0x7fc9a006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:45.449 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.434+0000 7fc9bbfff700 1 --2- 172.21.15.143:0/741659174 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc9a006af00 0x7fc9a006d3c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc9bc0b4150 tx=0x7fc9a8007be0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:45.449 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.438+0000 7fc9b97fa700 1 -- 172.21.15.143:0/741659174 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc9b0054a30 con 0x7fc9bc06c200 2023-11-08T19:25:45.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.712+0000 7fc9c3054700 1 -- 172.21.15.143:0/741659174 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7fc9bc0c88b0 con 0x7fc9bc06c200 2023-11-08T19:25:45.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.714+0000 7fc9b97fa700 1 -- 172.21.15.143:0/741659174 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v6) v1 ==== 90+0+0 (secure 0 0 0) 0x7fc9b0058050 con 0x7fc9bc06c200 2023-11-08T19:25:45.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.717+0000 7fc9c3054700 1 -- 172.21.15.143:0/741659174 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc9a006af00 msgr2=0x7fc9a006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:45.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.717+0000 7fc9c3054700 1 --2- 172.21.15.143:0/741659174 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc9a006af00 0x7fc9a006d3c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc9bc0b4150 tx=0x7fc9a8007be0 comp rx=0 tx=0).stop 2023-11-08T19:25:45.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.717+0000 7fc9c3054700 1 -- 172.21.15.143:0/741659174 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc9bc06c200 msgr2=0x7fc9bc146320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:45.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.717+0000 7fc9c3054700 1 --2- 172.21.15.143:0/741659174 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc9bc06c200 0x7fc9bc146320 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fc9b000d800 tx=0x7fc9b000dbc0 comp rx=0 tx=0).stop 2023-11-08T19:25:45.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.717+0000 7fc9c3054700 1 -- 172.21.15.143:0/741659174 shutdown_connections 2023-11-08T19:25:45.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.717+0000 7fc9c3054700 1 --2- 172.21.15.143:0/741659174 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc9a006af00 0x7fc9a006d3c0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:45.721 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.717+0000 7fc9c3054700 1 --2- 172.21.15.143:0/741659174 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc9bc15a2d0 0x7fc9bc0c9a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:45.721 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.717+0000 7fc9c3054700 1 --2- 172.21.15.143:0/741659174 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc9bc06c200 0x7fc9bc146320 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:45.721 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.717+0000 7fc9c3054700 1 --2- 172.21.15.143:0/741659174 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc9bc0ce500 0x7fc9bc0c9500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:45.721 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.717+0000 7fc9c3054700 1 -- 172.21.15.143:0/741659174 >> 172.21.15.143:0/741659174 conn(0x7fc9bc0b1a20 msgr2=0x7fc9bc0c5ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:45.721 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.717+0000 7fc9c3054700 1 -- 172.21.15.143:0/741659174 shutdown_connections 2023-11-08T19:25:45.722 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.717+0000 7fc9c3054700 1 -- 172.21.15.143:0/741659174 wait complete. 2023-11-08T19:25:45.732 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2023-11-08T19:25:45.856 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.854+0000 7f21d0c78700 1 Processor -- start 2023-11-08T19:25:45.857 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.854+0000 7f21d0c78700 1 -- start start 2023-11-08T19:25:45.860 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.855+0000 7f21d0c78700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f21cc1618f0 0x7f21cc06ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:45.860 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.855+0000 7f21d0c78700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f21cc06b150 0x7f21cc0cb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:45.861 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.855+0000 7f21d0c78700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21cc161df0 con 0x7f21cc15dd00 2023-11-08T19:25:45.861 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.855+0000 7f21d0c78700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21cc06b600 con 0x7f21cc06b150 2023-11-08T19:25:45.861 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.855+0000 7f21d0c78700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21cc0cad50 con 0x7f21cc1618f0 2023-11-08T19:25:45.861 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.855+0000 7f21cad9d700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f21cc06b150 0x7f21cc0cb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:45.862 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.855+0000 7f21ca59c700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f21cc15dd00 0x7f21cc0cca70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:46606/0 (socket says 172.21.15.143:46606) 2023-11-08T19:25:45.862 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.855+0000 7f21cad9d700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f21cc06b150 0x7f21cc0cb1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50456/0 (socket says 172.21.15.143:50456) 2023-11-08T19:25:45.862 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.855+0000 7f21ca59c700 1 -- 172.21.15.143:0/792433958 learned_addr learned my addr 172.21.15.143:0/792433958 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:25:45.862 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.855+0000 7f21c9d9b700 1 --2- 172.21.15.143:0/792433958 >> v2:172.21.15.174:3300/0 conn(0x7f21cc1618f0 0x7f21cc06ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:45.863 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.855+0000 7f21cad9d700 1 -- 172.21.15.143:0/792433958 >> v2:172.21.15.174:3300/0 conn(0x7f21cc1618f0 msgr2=0x7f21cc06ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:45.863 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.855+0000 7f21cad9d700 1 --2- 172.21.15.143:0/792433958 >> v2:172.21.15.174:3300/0 conn(0x7f21cc1618f0 0x7f21cc06ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:45.863 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.855+0000 7f21cad9d700 1 -- 172.21.15.143:0/792433958 >> v1:172.21.15.143:6789/0 conn(0x7f21cc15dd00 legacy=0x7f21cc0cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:45.863 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.855+0000 7f21cad9d700 1 -- 172.21.15.143:0/792433958 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21cc1620d0 con 0x7f21cc06b150 2023-11-08T19:25:45.863 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.855+0000 7f21c959a700 1 -- 172.21.15.143:0/792433958 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 182511797 0 0) 0x7f21cc161df0 con 0x7f21cc15dd00 2023-11-08T19:25:45.864 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.855+0000 7f21cad9d700 1 --2- 172.21.15.143:0/792433958 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21cc06b150 0x7f21cc0cb1a0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f21cc161850 tx=0x7f21c400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6346e81a7b46553d server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:45.864 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.855+0000 7f21c959a700 1 -- 172.21.15.143:0/792433958 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21c400e070 con 0x7f21cc06b150 2023-11-08T19:25:45.864 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.856+0000 7f21c959a700 1 -- 172.21.15.143:0/792433958 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21c400bbd0 con 0x7f21cc06b150 2023-11-08T19:25:45.864 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.856+0000 7f21c959a700 1 -- 172.21.15.143:0/792433958 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21c4012430 con 0x7f21cc06b150 2023-11-08T19:25:45.864 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.856+0000 7f21d0c78700 1 -- 172.21.15.143:0/792433958 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21cc06b150 msgr2=0x7f21cc0cb1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:45.865 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.856+0000 7f21d0c78700 1 --2- 172.21.15.143:0/792433958 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21cc06b150 0x7f21cc0cb1a0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f21cc161850 tx=0x7f21c400b330 comp rx=0 tx=0).stop 2023-11-08T19:25:45.865 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.856+0000 7f21d0c78700 1 -- 172.21.15.143:0/792433958 shutdown_connections 2023-11-08T19:25:45.865 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.856+0000 7f21d0c78700 1 --2- 172.21.15.143:0/792433958 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21cc06b150 0x7f21cc0cb1a0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:45.865 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.856+0000 7f21d0c78700 1 --2- 172.21.15.143:0/792433958 >> v2:172.21.15.174:3300/0 conn(0x7f21cc1618f0 0x7f21cc06ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:45.865 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.856+0000 7f21d0c78700 1 -- 172.21.15.143:0/792433958 >> 172.21.15.143:0/792433958 conn(0x7f21cc099ab0 msgr2=0x7f21cc0d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:45.866 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.856+0000 7f21d0c78700 1 -- 172.21.15.143:0/792433958 shutdown_connections 2023-11-08T19:25:45.866 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.856+0000 7f21d0c78700 1 -- 172.21.15.143:0/792433958 wait complete. 2023-11-08T19:25:45.866 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.858+0000 7f21d0c78700 1 Processor -- start 2023-11-08T19:25:45.866 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.858+0000 7f21d0c78700 1 -- start start 2023-11-08T19:25:45.869 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.858+0000 7f21d0c78700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f21cc06b150 0x7f21cc0c65a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:45.869 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.858+0000 7f21d0c78700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f21cc15dd00 0x7f21cc0c4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:45.870 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.858+0000 7f21d0c78700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21cc1618f0 0x7f21cc1f7000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:45.870 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.858+0000 7f21d0c78700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21cc1f4ee0 con 0x7f21cc1618f0 2023-11-08T19:25:45.870 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.858+0000 7f21d0c78700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21cc1f4bd0 con 0x7f21cc15dd00 2023-11-08T19:25:45.870 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.858+0000 7f21d0c78700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21cc0c5420 con 0x7f21cc06b150 2023-11-08T19:25:45.870 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.858+0000 7f21cad9d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21cc1618f0 0x7f21cc1f7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:45.871 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.858+0000 7f21cad9d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21cc1618f0 0x7f21cc1f7000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50462/0 (socket says 172.21.15.143:50462) 2023-11-08T19:25:45.871 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.859+0000 7f21cad9d700 1 -- 172.21.15.143:0/4233580944 learned_addr learned my addr 172.21.15.143:0/4233580944 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:45.871 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.859+0000 7f21c9d9b700 1 --2- 172.21.15.143:0/4233580944 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f21cc15dd00 0x7f21cc0c4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:45.871 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.859+0000 7f21ca59c700 1 --2- 172.21.15.143:0/4233580944 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f21cc06b150 0x7f21cc0c65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:45.871 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.859+0000 7f21cad9d700 1 -- 172.21.15.143:0/4233580944 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f21cc06b150 msgr2=0x7f21cc0c65a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:45.872 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.859+0000 7f21cad9d700 1 --2- 172.21.15.143:0/4233580944 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f21cc06b150 0x7f21cc0c65a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:45.872 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.859+0000 7f21cad9d700 1 -- 172.21.15.143:0/4233580944 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f21cc15dd00 msgr2=0x7f21cc0c4ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:45.872 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.859+0000 7f21cad9d700 1 --2- 172.21.15.143:0/4233580944 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f21cc15dd00 0x7f21cc0c4ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:45.872 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.859+0000 7f21cad9d700 1 -- 172.21.15.143:0/4233580944 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21c4009a70 con 0x7f21cc1618f0 2023-11-08T19:25:45.872 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.859+0000 7f21c9d9b700 1 --2- 172.21.15.143:0/4233580944 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f21cc15dd00 0x7f21cc0c4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:25:45.873 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.859+0000 7f21ca59c700 1 --2- 172.21.15.143:0/4233580944 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f21cc06b150 0x7f21cc0c65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:25:45.873 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.859+0000 7f21cad9d700 1 --2- 172.21.15.143:0/4233580944 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21cc1618f0 0x7f21cc1f7000 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f21cc070a90 tx=0x7f21c4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:45.873 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.859+0000 7f21baffd700 1 -- 172.21.15.143:0/4233580944 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21c4012430 con 0x7f21cc1618f0 2023-11-08T19:25:45.873 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.859+0000 7f21baffd700 1 -- 172.21.15.143:0/4233580944 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21c4020070 con 0x7f21cc1618f0 2023-11-08T19:25:45.873 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.859+0000 7f21d0c78700 1 -- 172.21.15.143:0/4233580944 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21cc0c5700 con 0x7f21cc1618f0 2023-11-08T19:25:45.874 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.859+0000 7f21baffd700 1 -- 172.21.15.143:0/4233580944 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21c4004280 con 0x7f21cc1618f0 2023-11-08T19:25:45.874 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.859+0000 7f21d0c78700 1 -- 172.21.15.143:0/4233580944 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21cc0c5010 con 0x7f21cc1618f0 2023-11-08T19:25:45.874 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.860+0000 7f21d0c78700 1 -- 172.21.15.143:0/4233580944 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21cc0aff60 con 0x7f21cc1618f0 2023-11-08T19:25:45.874 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.860+0000 7f21baffd700 1 -- 172.21.15.143:0/4233580944 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f21c40043e0 con 0x7f21cc1618f0 2023-11-08T19:25:45.874 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.861+0000 7f21baffd700 1 --2- 172.21.15.143:0/4233580944 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f21ac06af00 0x7f21ac06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:45.875 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.861+0000 7f21baffd700 1 -- 172.21.15.143:0/4233580944 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f21c4022070 con 0x7f21cc1618f0 2023-11-08T19:25:45.875 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.861+0000 7f21ca59c700 1 --2- 172.21.15.143:0/4233580944 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f21ac06af00 0x7f21ac06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:45.875 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.862+0000 7f21ca59c700 1 --2- 172.21.15.143:0/4233580944 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f21ac06af00 0x7f21ac06d3c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f21cc0c7250 tx=0x7f21bc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:45.875 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:45.865+0000 7f21baffd700 1 -- 172.21.15.143:0/4233580944 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f21c40bdca0 con 0x7f21cc1618f0 2023-11-08T19:25:46.136 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.134+0000 7f21d0c78700 1 -- 172.21.15.143:0/4233580944 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f21cc0c51a0 con 0x7f21cc1618f0 2023-11-08T19:25:46.138 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.136+0000 7f21baffd700 1 -- 172.21.15.143:0/4233580944 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v7) v1 ==== 97+0+0 (secure 0 0 0) 0x7f21c40566a0 con 0x7f21cc1618f0 2023-11-08T19:25:46.140 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.138+0000 7f21d0c78700 1 -- 172.21.15.143:0/4233580944 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f21ac06af00 msgr2=0x7f21ac06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:46.141 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.138+0000 7f21d0c78700 1 --2- 172.21.15.143:0/4233580944 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f21ac06af00 0x7f21ac06d3c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f21cc0c7250 tx=0x7f21bc009040 comp rx=0 tx=0).stop 2023-11-08T19:25:46.141 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.138+0000 7f21d0c78700 1 -- 172.21.15.143:0/4233580944 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21cc1618f0 msgr2=0x7f21cc1f7000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:46.141 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.138+0000 7f21d0c78700 1 --2- 172.21.15.143:0/4233580944 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21cc1618f0 0x7f21cc1f7000 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f21cc070a90 tx=0x7f21c4004ad0 comp rx=0 tx=0).stop 2023-11-08T19:25:46.141 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.138+0000 7f21d0c78700 1 -- 172.21.15.143:0/4233580944 shutdown_connections 2023-11-08T19:25:46.141 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.138+0000 7f21d0c78700 1 --2- 172.21.15.143:0/4233580944 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f21ac06af00 0x7f21ac06d3c0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:46.142 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.138+0000 7f21d0c78700 1 --2- 172.21.15.143:0/4233580944 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f21cc06b150 0x7f21cc0c65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:46.142 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.138+0000 7f21d0c78700 1 --2- 172.21.15.143:0/4233580944 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21cc1618f0 0x7f21cc1f7000 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:46.142 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.138+0000 7f21d0c78700 1 --2- 172.21.15.143:0/4233580944 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f21cc15dd00 0x7f21cc0c4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:46.142 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.138+0000 7f21d0c78700 1 -- 172.21.15.143:0/4233580944 >> 172.21.15.143:0/4233580944 conn(0x7f21cc099ab0 msgr2=0x7f21cc06f550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:46.143 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.138+0000 7f21d0c78700 1 -- 172.21.15.143:0/4233580944 shutdown_connections 2023-11-08T19:25:46.143 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.138+0000 7f21d0c78700 1 -- 172.21.15.143:0/4233580944 wait complete. 2023-11-08T19:25:46.152 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2023-11-08T19:25:46.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.272+0000 7f5821e2c700 1 Processor -- start 2023-11-08T19:25:46.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.272+0000 7f5821e2c700 1 -- start start 2023-11-08T19:25:46.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.273+0000 7f5821e2c700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f581c0cf320 0x7f581c06ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:46.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.273+0000 7f5821e2c700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f581c1589c0 con 0x7f581c1585e0 2023-11-08T19:25:46.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.273+0000 7f5821e2c700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f581c0cf9f0 con 0x7f581c0b2510 2023-11-08T19:25:46.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.273+0000 7f5821e2c700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f581c15bf30 con 0x7f581c0cf320 2023-11-08T19:25:46.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.273+0000 7f581bfff700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f581c0cf320 0x7f581c06ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:46.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.273+0000 7f581b7fe700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f581c1585e0 0x7f581c0cec10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:46620/0 (socket says 172.21.15.143:46620) 2023-11-08T19:25:46.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.273+0000 7f581b7fe700 1 -- 172.21.15.143:0/175164547 learned_addr learned my addr 172.21.15.143:0/175164547 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:25:46.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.273+0000 7f581bfff700 1 -- 172.21.15.143:0/175164547 >> v1:172.21.15.174:6789/0 conn(0x7f581c0b2510 legacy=0x7f581c0cb900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:46.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.273+0000 7f581bfff700 1 -- 172.21.15.143:0/175164547 >> v1:172.21.15.143:6789/0 conn(0x7f581c1585e0 legacy=0x7f581c0cec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:46.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.273+0000 7f581bfff700 1 -- 172.21.15.143:0/175164547 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f581c15b320 con 0x7f581c0cf320 2023-11-08T19:25:46.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.273+0000 7f581a7fc700 1 -- 172.21.15.143:0/175164547 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1745924315 0 0) 0x7f5804001110 con 0x7f581c1585e0 2023-11-08T19:25:46.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.273+0000 7f581bfff700 1 --2- 172.21.15.143:0/175164547 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f581c0cf320 0x7f581c06ac10 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f5814009d90 tx=0x7f581400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4d715a1b641a20d server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:46.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.273+0000 7f581a7fc700 1 -- 172.21.15.143:0/175164547 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f581400e070 con 0x7f581c0cf320 2023-11-08T19:25:46.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.273+0000 7f581a7fc700 1 -- 172.21.15.143:0/175164547 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f581400bbd0 con 0x7f581c0cf320 2023-11-08T19:25:46.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.274+0000 7f581a7fc700 1 -- 172.21.15.143:0/175164547 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5814012430 con 0x7f581c0cf320 2023-11-08T19:25:46.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.274+0000 7f5821e2c700 1 -- 172.21.15.143:0/175164547 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f581c0cf320 msgr2=0x7f581c06ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:46.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.274+0000 7f5821e2c700 1 --2- 172.21.15.143:0/175164547 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f581c0cf320 0x7f581c06ac10 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f5814009d90 tx=0x7f581400b330 comp rx=0 tx=0).stop 2023-11-08T19:25:46.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.274+0000 7f5821e2c700 1 -- 172.21.15.143:0/175164547 shutdown_connections 2023-11-08T19:25:46.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.274+0000 7f5821e2c700 1 --2- 172.21.15.143:0/175164547 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f581c0cf320 0x7f581c06ac10 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:46.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.274+0000 7f5821e2c700 1 -- 172.21.15.143:0/175164547 >> 172.21.15.143:0/175164547 conn(0x7f581c0b1a20 msgr2=0x7f581c0b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:46.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.274+0000 7f5821e2c700 1 -- 172.21.15.143:0/175164547 shutdown_connections 2023-11-08T19:25:46.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.274+0000 7f5821e2c700 1 -- 172.21.15.143:0/175164547 wait complete. 2023-11-08T19:25:46.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.275+0000 7f5821e2c700 1 Processor -- start 2023-11-08T19:25:46.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.275+0000 7f5821e2c700 1 -- start start 2023-11-08T19:25:46.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.276+0000 7f5821e2c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f581c0b2510 0x7f581c15eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:46.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.276+0000 7f5821e2c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f581c0cf320 0x7f581c152b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:46.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.276+0000 7f5821e2c700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f581c1585e0 0x7f581c0b4d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:46.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.276+0000 7f5821e2c700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f581c0c6ae0 con 0x7f581c0cf320 2023-11-08T19:25:46.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.276+0000 7f5821e2c700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f581c0c5f60 con 0x7f581c0b2510 2023-11-08T19:25:46.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.276+0000 7f5821e2c700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f581c0c6240 con 0x7f581c1585e0 2023-11-08T19:25:46.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.276+0000 7f581affd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f581c0cf320 0x7f581c152b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:46.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.276+0000 7f581affd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f581c0cf320 0x7f581c152b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50474/0 (socket says 172.21.15.143:50474) 2023-11-08T19:25:46.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.276+0000 7f581affd700 1 -- 172.21.15.143:0/1872111797 learned_addr learned my addr 172.21.15.143:0/1872111797 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:46.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.276+0000 7f581b7fe700 1 --2- 172.21.15.143:0/1872111797 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f581c0b2510 0x7f581c15eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:46.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.276+0000 7f581bfff700 1 --2- 172.21.15.143:0/1872111797 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f581c1585e0 0x7f581c0b4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:46.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.277+0000 7f581affd700 1 -- 172.21.15.143:0/1872111797 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f581c1585e0 msgr2=0x7f581c0b4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:46.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.277+0000 7f581affd700 1 --2- 172.21.15.143:0/1872111797 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f581c1585e0 0x7f581c0b4d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:46.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.277+0000 7f581affd700 1 -- 172.21.15.143:0/1872111797 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f581c0b2510 msgr2=0x7f581c15eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:46.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.277+0000 7f581affd700 1 --2- 172.21.15.143:0/1872111797 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f581c0b2510 0x7f581c15eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:46.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.277+0000 7f581affd700 1 -- 172.21.15.143:0/1872111797 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5814009a70 con 0x7f581c0cf320 2023-11-08T19:25:46.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.277+0000 7f581b7fe700 1 --2- 172.21.15.143:0/1872111797 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f581c0b2510 0x7f581c15eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:25:46.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.277+0000 7f581affd700 1 --2- 172.21.15.143:0/1872111797 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f581c0cf320 0x7f581c152b60 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f580c00c4e0 tx=0x7f580c00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:46.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.277+0000 7f5813fff700 1 -- 172.21.15.143:0/1872111797 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f580c0052d0 con 0x7f581c0cf320 2023-11-08T19:25:46.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.277+0000 7f5813fff700 1 -- 172.21.15.143:0/1872111797 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f580c005430 con 0x7f581c0cf320 2023-11-08T19:25:46.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.277+0000 7f5821e2c700 1 -- 172.21.15.143:0/1872111797 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f581c0c6520 con 0x7f581c0cf320 2023-11-08T19:25:46.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.277+0000 7f5813fff700 1 -- 172.21.15.143:0/1872111797 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f580c0038c0 con 0x7f581c0cf320 2023-11-08T19:25:46.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.277+0000 7f5821e2c700 1 -- 172.21.15.143:0/1872111797 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f581c15f0c0 con 0x7f581c0cf320 2023-11-08T19:25:46.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.278+0000 7f5821e2c700 1 -- 172.21.15.143:0/1872111797 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f581c15c2b0 con 0x7f581c0cf320 2023-11-08T19:25:46.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.278+0000 7f5813fff700 1 -- 172.21.15.143:0/1872111797 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f580c003a20 con 0x7f581c0cf320 2023-11-08T19:25:46.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.279+0000 7f5813fff700 1 --2- 172.21.15.143:0/1872111797 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f57fc06af30 0x7f57fc06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:46.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.279+0000 7f5813fff700 1 -- 172.21.15.143:0/1872111797 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f580c0887f0 con 0x7f581c0cf320 2023-11-08T19:25:46.293 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.279+0000 7f581b7fe700 1 --2- 172.21.15.143:0/1872111797 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f57fc06af30 0x7f57fc06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:46.293 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.279+0000 7f581b7fe700 1 --2- 172.21.15.143:0/1872111797 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f57fc06af30 0x7f57fc06d3f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5804000f60 tx=0x7f5804007e50 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:46.293 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.283+0000 7f5813fff700 1 -- 172.21.15.143:0/1872111797 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f581c15c2b0 con 0x7f581c0cf320 2023-11-08T19:25:46.550 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.547+0000 7f5821e2c700 1 -- 172.21.15.143:0/1872111797 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7f581c15c2b0 con 0x7f581c0cf320 2023-11-08T19:25:46.552 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.550+0000 7f5813fff700 1 -- 172.21.15.143:0/1872111797 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7f581c15c2b0 con 0x7f581c0cf320 2023-11-08T19:25:46.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.552+0000 7f5821e2c700 1 -- 172.21.15.143:0/1872111797 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f57fc06af30 msgr2=0x7f57fc06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:46.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.552+0000 7f5821e2c700 1 --2- 172.21.15.143:0/1872111797 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f57fc06af30 0x7f57fc06d3f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5804000f60 tx=0x7f5804007e50 comp rx=0 tx=0).stop 2023-11-08T19:25:46.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.552+0000 7f5821e2c700 1 -- 172.21.15.143:0/1872111797 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f581c0cf320 msgr2=0x7f581c152b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:46.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.552+0000 7f5821e2c700 1 --2- 172.21.15.143:0/1872111797 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f581c0cf320 0x7f581c152b60 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f580c00c4e0 tx=0x7f580c00c8a0 comp rx=0 tx=0).stop 2023-11-08T19:25:46.556 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.552+0000 7f5821e2c700 1 -- 172.21.15.143:0/1872111797 shutdown_connections 2023-11-08T19:25:46.556 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.552+0000 7f5821e2c700 1 --2- 172.21.15.143:0/1872111797 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f57fc06af30 0x7f57fc06d3f0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:46.556 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.552+0000 7f5821e2c700 1 --2- 172.21.15.143:0/1872111797 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f581c1585e0 0x7f581c0b4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:46.556 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.552+0000 7f5821e2c700 1 --2- 172.21.15.143:0/1872111797 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f581c0b2510 0x7f581c15eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:46.556 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.552+0000 7f5821e2c700 1 --2- 172.21.15.143:0/1872111797 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f581c0cf320 0x7f581c152b60 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:46.557 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.552+0000 7f5821e2c700 1 -- 172.21.15.143:0/1872111797 >> 172.21.15.143:0/1872111797 conn(0x7f581c0b1a20 msgr2=0x7f581c156300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:46.557 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.552+0000 7f5821e2c700 1 -- 172.21.15.143:0/1872111797 shutdown_connections 2023-11-08T19:25:46.557 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.552+0000 7f5821e2c700 1 -- 172.21.15.143:0/1872111797 wait complete. 2023-11-08T19:25:46.567 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:25:46.688 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.685+0000 7fa4501d4700 1 Processor -- start 2023-11-08T19:25:46.688 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.686+0000 7fa4501d4700 1 -- start start 2023-11-08T19:25:46.691 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.686+0000 7fa4501d4700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fa4480d1c70 0x7fa4480b14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:46.692 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.686+0000 7fa4501d4700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa4480d03a0 0x7fa4480cc5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:46.692 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.686+0000 7fa4501d4700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4480b1bc0 con 0x7fa44815ba30 2023-11-08T19:25:46.692 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.686+0000 7fa4501d4700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4480b1ea0 con 0x7fa4480d03a0 2023-11-08T19:25:46.692 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.686+0000 7fa4501d4700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4480b2180 con 0x7fa4480d1c70 2023-11-08T19:25:46.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.686+0000 7fa44e771700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa4480d03a0 0x7fa4480cc5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:46.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.686+0000 7fa44df70700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fa44815ba30 0x7fa4480d1560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:46634/0 (socket says 172.21.15.143:46634) 2023-11-08T19:25:46.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.687+0000 7fa44e771700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa4480d03a0 0x7fa4480cc5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50484/0 (socket says 172.21.15.143:50484) 2023-11-08T19:25:46.694 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.687+0000 7fa44df70700 1 -- 172.21.15.143:0/3840244330 learned_addr learned my addr 172.21.15.143:0/3840244330 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:25:46.694 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.687+0000 7fa44d76f700 1 --2- 172.21.15.143:0/3840244330 >> v2:172.21.15.174:3300/0 conn(0x7fa4480d1c70 0x7fa4480b14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:46.694 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.687+0000 7fa44e771700 1 -- 172.21.15.143:0/3840244330 >> v2:172.21.15.174:3300/0 conn(0x7fa4480d1c70 msgr2=0x7fa4480b14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:46.695 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.687+0000 7fa44e771700 1 --2- 172.21.15.143:0/3840244330 >> v2:172.21.15.174:3300/0 conn(0x7fa4480d1c70 0x7fa4480b14c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:46.695 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.687+0000 7fa44e771700 1 -- 172.21.15.143:0/3840244330 >> v1:172.21.15.143:6789/0 conn(0x7fa44815ba30 legacy=0x7fa4480d1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:46.695 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.687+0000 7fa44e771700 1 -- 172.21.15.143:0/3840244330 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4480b2460 con 0x7fa4480d03a0 2023-11-08T19:25:46.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.687+0000 7fa44cf6e700 1 -- 172.21.15.143:0/3840244330 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3188236514 0 0) 0x7fa4480b1bc0 con 0x7fa44815ba30 2023-11-08T19:25:46.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.687+0000 7fa44e771700 1 --2- 172.21.15.143:0/3840244330 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa4480d03a0 0x7fa4480cc5a0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fa4480cbe60 tx=0x7fa44000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19eb1c20cd5b60c7 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:46.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.687+0000 7fa44cf6e700 1 -- 172.21.15.143:0/3840244330 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa44000e070 con 0x7fa4480d03a0 2023-11-08T19:25:46.697 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.687+0000 7fa44cf6e700 1 -- 172.21.15.143:0/3840244330 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa44000bbd0 con 0x7fa4480d03a0 2023-11-08T19:25:46.697 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.687+0000 7fa44cf6e700 1 -- 172.21.15.143:0/3840244330 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa440012430 con 0x7fa4480d03a0 2023-11-08T19:25:46.697 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.688+0000 7fa4501d4700 1 -- 172.21.15.143:0/3840244330 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa4480d03a0 msgr2=0x7fa4480cc5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:46.697 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.688+0000 7fa4501d4700 1 --2- 172.21.15.143:0/3840244330 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa4480d03a0 0x7fa4480cc5a0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fa4480cbe60 tx=0x7fa44000b330 comp rx=0 tx=0).stop 2023-11-08T19:25:46.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.688+0000 7fa4501d4700 1 -- 172.21.15.143:0/3840244330 shutdown_connections 2023-11-08T19:25:46.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.688+0000 7fa4501d4700 1 --2- 172.21.15.143:0/3840244330 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa4480d03a0 0x7fa4480cc5a0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:46.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.688+0000 7fa4501d4700 1 --2- 172.21.15.143:0/3840244330 >> v2:172.21.15.174:3300/0 conn(0x7fa4480d1c70 0x7fa4480b14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:46.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.688+0000 7fa4501d4700 1 -- 172.21.15.143:0/3840244330 >> 172.21.15.143:0/3840244330 conn(0x7fa44808b000 msgr2=0x7fa44809ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:46.699 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.688+0000 7fa4501d4700 1 -- 172.21.15.143:0/3840244330 shutdown_connections 2023-11-08T19:25:46.699 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.688+0000 7fa4501d4700 1 -- 172.21.15.143:0/3840244330 wait complete. 2023-11-08T19:25:46.699 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.689+0000 7fa4501d4700 1 Processor -- start 2023-11-08T19:25:46.699 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.689+0000 7fa4501d4700 1 -- start start 2023-11-08T19:25:46.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.690+0000 7fa4501d4700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa4480d03a0 0x7fa44806d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:46.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.690+0000 7fa4501d4700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa4480d1c70 0x7fa448157710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:46.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.690+0000 7fa4501d4700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa44815ba30 0x7fa44806bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:46.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.690+0000 7fa4501d4700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa44806dc70 con 0x7fa4480d1c70 2023-11-08T19:25:46.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.690+0000 7fa4501d4700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa44806c690 con 0x7fa4480d03a0 2023-11-08T19:25:46.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.690+0000 7fa4501d4700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa44806c970 con 0x7fa44815ba30 2023-11-08T19:25:46.705 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.690+0000 7fa44d76f700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa4480d1c70 0x7fa448157710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:46.705 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.690+0000 7fa44d76f700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa4480d1c70 0x7fa448157710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50488/0 (socket says 172.21.15.143:50488) 2023-11-08T19:25:46.705 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.690+0000 7fa44d76f700 1 -- 172.21.15.143:0/4221181823 learned_addr learned my addr 172.21.15.143:0/4221181823 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:46.705 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.690+0000 7fa44df70700 1 --2- 172.21.15.143:0/4221181823 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa4480d03a0 0x7fa44806d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:46.705 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.690+0000 7fa44d76f700 1 -- 172.21.15.143:0/4221181823 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa44815ba30 msgr2=0x7fa44806bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:46.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.690+0000 7fa44e771700 1 --2- 172.21.15.143:0/4221181823 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa44815ba30 0x7fa44806bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:46.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.690+0000 7fa44d76f700 1 --2- 172.21.15.143:0/4221181823 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa44815ba30 0x7fa44806bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:46.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.690+0000 7fa44d76f700 1 -- 172.21.15.143:0/4221181823 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa4480d03a0 msgr2=0x7fa44806d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:46.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.690+0000 7fa44d76f700 1 --2- 172.21.15.143:0/4221181823 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa4480d03a0 0x7fa44806d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:46.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.690+0000 7fa44d76f700 1 -- 172.21.15.143:0/4221181823 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa440009a70 con 0x7fa4480d1c70 2023-11-08T19:25:46.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.690+0000 7fa44df70700 1 --2- 172.21.15.143:0/4221181823 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa4480d03a0 0x7fa44806d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:25:46.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.690+0000 7fa44d76f700 1 --2- 172.21.15.143:0/4221181823 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa4480d1c70 0x7fa448157710 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fa44400c7f0 tx=0x7fa44400cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:46.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.690+0000 7fa43e7fc700 1 -- 172.21.15.143:0/4221181823 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa444011800 con 0x7fa4480d1c70 2023-11-08T19:25:46.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.690+0000 7fa43e7fc700 1 -- 172.21.15.143:0/4221181823 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa444011960 con 0x7fa4480d1c70 2023-11-08T19:25:46.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.690+0000 7fa4501d4700 1 -- 172.21.15.143:0/4221181823 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa44806cc50 con 0x7fa4480d1c70 2023-11-08T19:25:46.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.690+0000 7fa43e7fc700 1 -- 172.21.15.143:0/4221181823 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa444011cf0 con 0x7fa4480d1c70 2023-11-08T19:25:46.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.690+0000 7fa4501d4700 1 -- 172.21.15.143:0/4221181823 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa44806d210 con 0x7fa4480d1c70 2023-11-08T19:25:46.709 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.691+0000 7fa4501d4700 1 -- 172.21.15.143:0/4221181823 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa44804eee0 con 0x7fa4480d1c70 2023-11-08T19:25:46.709 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.692+0000 7fa43e7fc700 1 -- 172.21.15.143:0/4221181823 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa444018360 con 0x7fa4480d1c70 2023-11-08T19:25:46.709 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.692+0000 7fa43e7fc700 1 --2- 172.21.15.143:0/4221181823 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa42c06af00 0x7fa42c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:46.709 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.692+0000 7fa43e7fc700 1 -- 172.21.15.143:0/4221181823 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fa444089210 con 0x7fa4480d1c70 2023-11-08T19:25:46.710 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.692+0000 7fa44df70700 1 --2- 172.21.15.143:0/4221181823 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa42c06af00 0x7fa42c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:46.710 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.693+0000 7fa44df70700 1 --2- 172.21.15.143:0/4221181823 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa42c06af00 0x7fa42c06d3c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa438000f60 tx=0x7fa438009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:46.710 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.696+0000 7fa43e7fc700 1 -- 172.21.15.143:0/4221181823 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa444054620 con 0x7fa4480d1c70 2023-11-08T19:25:46.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.889+0000 7fa4501d4700 1 -- 172.21.15.143:0/4221181823 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa4480cee70 con 0x7fa4480d1c70 2023-11-08T19:25:46.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.890+0000 7fa43e7fc700 1 -- 172.21.15.143:0/4221181823 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+12905 (secure 0 0 0) 0x7fa44405e090 con 0x7fa4480d1c70 2023-11-08T19:25:46.892 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:25:46.892 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":29,"fsid":"968483af-77fc-4068-ad7c-5c2a1a4613d7","created":"2023-11-08T19:24:32.826484+0000","modified":"2023-11-08T19:25:43.585094+0000","last_up_change":"2023-11-08T19:24:43.573880+0000","last_in_change":"2023-11-08T19:24:35.314525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:44.551530+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bae98ec1-d94e-403d-a220-a522a53ea6f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6809","nonce":1487830566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6811","nonce":1487830566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6815","nonce":1487830566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6813","nonce":1487830566}]},"public_addr":"172.21.15.143:6809/1487830566","cluster_addr":"172.21.15.143:6811/1487830566","heartbeat_back_addr":"172.21.15.143:6815/1487830566","heartbeat_front_addr":"172.21.15.143:6813/1487830566","state":["exists","up"]},{"osd":1,"uuid":"5cb6538a-1ccb-4322-8633-09f66af0b7e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6801","nonce":787341397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6803","nonce":787341397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6807","nonce":787341397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6805","nonce":787341397}]},"public_addr":"172.21.15.143:6801/787341397","cluster_addr":"172.21.15.143:6803/787341397","heartbeat_back_addr":"172.21.15.143:6807/787341397","heartbeat_front_addr":"172.21.15.143:6805/787341397","state":["exists","up"]},{"osd":2,"uuid":"cdbf522c-60ef-4d60-9653-8dcb529c101a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6822","nonce":2301325597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6826","nonce":2301325597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6831","nonce":2301325597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6829","nonce":2301325597}]},"public_addr":"172.21.15.143:6822/2301325597","cluster_addr":"172.21.15.143:6826/2301325597","heartbeat_back_addr":"172.21.15.143:6831/2301325597","heartbeat_front_addr":"172.21.15.143:6829/2301325597","state":["exists","up"]},{"osd":3,"uuid":"4e2783cf-ebb9-4dd8-8245-bc1d17ce4325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6817","nonce":2021504428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6819","nonce":2021504428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6827","nonce":2021504428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6823","nonce":2021504428}]},"public_addr":"172.21.15.143:6817/2021504428","cluster_addr":"172.21.15.143:6819/2021504428","heartbeat_back_addr":"172.21.15.143:6827/2021504428","heartbeat_front_addr":"172.21.15.143:6823/2021504428","state":["exists","up"]},{"osd":4,"uuid":"691256fa-9b77-49b9-a346-7ae1adf0ca80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6801","nonce":1993380595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6803","nonce":1993380595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6807","nonce":1993380595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6805","nonce":1993380595}]},"public_addr":"172.21.15.174:6801/1993380595","cluster_addr":"172.21.15.174:6803/1993380595","heartbeat_back_addr":"172.21.15.174:6807/1993380595","heartbeat_front_addr":"172.21.15.174:6805/1993380595","state":["exists","up"]},{"osd":5,"uuid":"756fc352-48c9-46ce-b52e-0541fe6f8525","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6809","nonce":1777923303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6811","nonce":1777923303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6815","nonce":1777923303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6813","nonce":1777923303}]},"public_addr":"172.21.15.174:6809/1777923303","cluster_addr":"172.21.15.174:6811/1777923303","heartbeat_back_addr":"172.21.15.174:6815/1777923303","heartbeat_front_addr":"172.21.15.174:6813/1777923303","state":["exists","up"]},{"osd":6,"uuid":"fab2ac2b-ed97-4cde-9825-b480eabf8b8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6825","nonce":2755130316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6827","nonce":2755130316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6831","nonce":2755130316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6829","nonce":2755130316}]},"public_addr":"172.21.15.174:6825/2755130316","cluster_addr":"172.21.15.174:6827/2755130316","heartbeat_back_addr":"172.21.15.174:6831/2755130316","heartbeat_front_addr":"172.21.15.174:6829/2755130316","state":["exists","up"]},{"osd":7,"uuid":"3de9b797-2ab6-4d41-a863-7e7d638fd703","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6817","nonce":2847141411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6819","nonce":2847141411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6823","nonce":2847141411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6821","nonce":2847141411}]},"public_addr":"172.21.15.174:6817/2847141411","cluster_addr":"172.21.15.174:6819/2847141411","heartbeat_back_addr":"172.21.15.174:6823/2847141411","heartbeat_front_addr":"172.21.15.174:6821/2847141411","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.477076+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.496890+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.539305+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.454673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.538881+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.521294+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.520712+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.540404+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:25:46.895 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.892+0000 7fa4501d4700 1 -- 172.21.15.143:0/4221181823 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa42c06af00 msgr2=0x7fa42c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:46.895 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.892+0000 7fa4501d4700 1 --2- 172.21.15.143:0/4221181823 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa42c06af00 0x7fa42c06d3c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa438000f60 tx=0x7fa438009040 comp rx=0 tx=0).stop 2023-11-08T19:25:46.895 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.892+0000 7fa4501d4700 1 -- 172.21.15.143:0/4221181823 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa4480d1c70 msgr2=0x7fa448157710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:46.895 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.892+0000 7fa4501d4700 1 --2- 172.21.15.143:0/4221181823 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa4480d1c70 0x7fa448157710 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fa44400c7f0 tx=0x7fa44400cbb0 comp rx=0 tx=0).stop 2023-11-08T19:25:46.895 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.892+0000 7fa4501d4700 1 -- 172.21.15.143:0/4221181823 shutdown_connections 2023-11-08T19:25:46.896 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.892+0000 7fa4501d4700 1 --2- 172.21.15.143:0/4221181823 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa42c06af00 0x7fa42c06d3c0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:46.896 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.892+0000 7fa4501d4700 1 --2- 172.21.15.143:0/4221181823 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa44815ba30 0x7fa44806bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:46.896 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.892+0000 7fa4501d4700 1 --2- 172.21.15.143:0/4221181823 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa4480d03a0 0x7fa44806d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:46.896 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.892+0000 7fa4501d4700 1 --2- 172.21.15.143:0/4221181823 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa4480d1c70 0x7fa448157710 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:46.896 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.892+0000 7fa4501d4700 1 -- 172.21.15.143:0/4221181823 >> 172.21.15.143:0/4221181823 conn(0x7fa44808b000 msgr2=0x7fa4480ae9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:46.897 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.892+0000 7fa4501d4700 1 -- 172.21.15.143:0/4221181823 shutdown_connections 2023-11-08T19:25:46.897 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:46.892+0000 7fa4501d4700 1 -- 172.21.15.143:0/4221181823 wait complete. 2023-11-08T19:25:46.907 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T19:24:44.551530+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-08T19:25:46.907 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T19:25:47.031 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.028+0000 7f67037a6700 1 Processor -- start 2023-11-08T19:25:47.031 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.028+0000 7f67037a6700 1 -- start start 2023-11-08T19:25:47.034 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.029+0000 7f67037a6700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f66fc0b5ae0 0x7f66fc0d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:47.034 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.029+0000 7f67037a6700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66fc0b9b00 con 0x7f66fc15dca0 2023-11-08T19:25:47.034 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.029+0000 7f67037a6700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66fc0b8f50 con 0x7f66fc0b5630 2023-11-08T19:25:47.035 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.029+0000 7f67037a6700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66fc0b3360 con 0x7f66fc0b5ae0 2023-11-08T19:25:47.035 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.029+0000 7f6701542700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f66fc15dca0 0x7f66fc0b93f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:46648/0 (socket says 172.21.15.143:46648) 2023-11-08T19:25:47.035 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.029+0000 7f6701542700 1 -- 172.21.15.143:0/1062477637 learned_addr learned my addr 172.21.15.143:0/1062477637 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:25:47.035 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.030+0000 7f66f3fff700 1 -- 172.21.15.143:0/1062477637 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3181894510 0 0) 0x7f66fc0b9b00 con 0x7f66fc15dca0 2023-11-08T19:25:47.035 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.030+0000 7f6701d43700 1 --2- 172.21.15.143:0/1062477637 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f66fc0b5ae0 0x7f66fc0d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:47.036 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.030+0000 7f66f3fff700 1 -- 172.21.15.143:0/1062477637 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f66e80035b0 con 0x7f66fc15dca0 2023-11-08T19:25:47.036 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.030+0000 7f66f3fff700 1 -- 172.21.15.143:0/1062477637 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3307436565 0 0) 0x7f66e80035b0 con 0x7f66fc15dca0 2023-11-08T19:25:47.036 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.030+0000 7f66f3fff700 1 -- 172.21.15.143:0/1062477637 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f66fc0b5ae0 msgr2=0x7f66fc0d22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:47.036 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.030+0000 7f66f3fff700 1 --2- 172.21.15.143:0/1062477637 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f66fc0b5ae0 0x7f66fc0d22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:47.036 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.030+0000 7f66f3fff700 1 -- 172.21.15.143:0/1062477637 >> v1:172.21.15.174:6789/0 conn(0x7f66fc0b5630 legacy=0x7f66fc0d1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:47.037 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.030+0000 7f66f3fff700 1 -- 172.21.15.143:0/1062477637 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66fc0b7eb0 con 0x7f66fc15dca0 2023-11-08T19:25:47.037 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.030+0000 7f66f3fff700 1 -- 172.21.15.143:0/1062477637 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4147756303 0 0) 0x7f66f8001670 con 0x7f66fc15dca0 2023-11-08T19:25:47.037 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.030+0000 7f66f3fff700 1 -- 172.21.15.143:0/1062477637 >> v1:172.21.15.143:6789/0 conn(0x7f66fc15dca0 legacy=0x7f66fc0b93f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:47.037 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.030+0000 7f66f3fff700 1 --2- 172.21.15.143:0/1062477637 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f66e8003a30 0x7f66e8003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:47.037 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.030+0000 7f66f3fff700 1 --2- 172.21.15.143:0/1062477637 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f66fc0b5ae0 0x7f66e8004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:47.038 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.030+0000 7f66f3fff700 1 --2- 172.21.15.143:0/1062477637 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f66e8004d90 0x7f66e8005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:47.038 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.030+0000 7f66f3fff700 1 -- 172.21.15.143:0/1062477637 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66e800bb50 con 0x7f66e8004d90 2023-11-08T19:25:47.038 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.030+0000 7f66f3fff700 1 -- 172.21.15.143:0/1062477637 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66e800bcc0 con 0x7f66e8003a30 2023-11-08T19:25:47.038 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.030+0000 7f66f3fff700 1 -- 172.21.15.143:0/1062477637 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66e800be80 con 0x7f66fc0b5ae0 2023-11-08T19:25:47.038 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.030+0000 7f6701d43700 1 --2- 172.21.15.143:0/1062477637 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f66e8004d90 0x7f66e8005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:47.039 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.030+0000 7f6701542700 1 --2- 172.21.15.143:0/1062477637 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f66e8003a30 0x7f66e8003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:47.039 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.030+0000 7f6700d41700 1 --2- 172.21.15.143:0/1062477637 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f66fc0b5ae0 0x7f66e8004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:47.039 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.030+0000 7f6701d43700 1 -- 172.21.15.143:0/1062477637 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f66fc0b5ae0 msgr2=0x7f66e8004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:47.039 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.030+0000 7f6701d43700 1 --2- 172.21.15.143:0/1062477637 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f66fc0b5ae0 0x7f66e8004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:47.039 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.030+0000 7f6701d43700 1 -- 172.21.15.143:0/1062477637 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f66e8003a30 msgr2=0x7f66e8003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:47.040 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.030+0000 7f6701d43700 1 --2- 172.21.15.143:0/1062477637 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f66e8003a30 0x7f66e8003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:47.040 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.030+0000 7f6701d43700 1 -- 172.21.15.143:0/1062477637 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66e80035b0 con 0x7f66e8004d90 2023-11-08T19:25:47.040 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.031+0000 7f6701d43700 1 --2- 172.21.15.143:0/1062477637 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f66e8004d90 0x7f66e8005210 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f66f4009f90 tx=0x7f66e8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cecdaacd05e7ecd server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:47.040 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.031+0000 7f66f3fff700 1 -- 172.21.15.143:0/1062477637 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f400ec30 con 0x7f66e8004d90 2023-11-08T19:25:47.040 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.031+0000 7f66f3fff700 1 -- 172.21.15.143:0/1062477637 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66f400f040 con 0x7f66e8004d90 2023-11-08T19:25:47.041 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.031+0000 7f66f3fff700 1 -- 172.21.15.143:0/1062477637 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f4014480 con 0x7f66e8004d90 2023-11-08T19:25:47.041 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.031+0000 7f67037a6700 1 -- 172.21.15.143:0/1062477637 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f66e8004d90 msgr2=0x7f66e8005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:47.041 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.031+0000 7f67037a6700 1 --2- 172.21.15.143:0/1062477637 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f66e8004d90 0x7f66e8005210 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f66f4009f90 tx=0x7f66e8005880 comp rx=0 tx=0).stop 2023-11-08T19:25:47.041 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.031+0000 7f6701d43700 1 -- 172.21.15.143:0/1062477637 reap_dead start 2023-11-08T19:25:47.041 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.031+0000 7f67037a6700 1 -- 172.21.15.143:0/1062477637 shutdown_connections 2023-11-08T19:25:47.041 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.031+0000 7f67037a6700 1 -- 172.21.15.143:0/1062477637 >> 172.21.15.143:0/1062477637 conn(0x7f66fc066d00 msgr2=0x7f66fc067110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:47.042 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.031+0000 7f67037a6700 1 -- 172.21.15.143:0/1062477637 shutdown_connections 2023-11-08T19:25:47.042 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.031+0000 7f67037a6700 1 -- 172.21.15.143:0/1062477637 wait complete. 2023-11-08T19:25:47.045 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.032+0000 7f67037a6700 1 Processor -- start 2023-11-08T19:25:47.045 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.033+0000 7f67037a6700 1 -- start start 2023-11-08T19:25:47.045 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.033+0000 7f67037a6700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f66fc158a20 0x7f66fc159e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:47.045 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.033+0000 7f67037a6700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f66fc15a3a0 0x7f66fc0b46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:47.046 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.033+0000 7f67037a6700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f66fc158e00 0x7f66fc0b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:47.046 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.033+0000 7f67037a6700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66fc0c9080 con 0x7f66fc158a20 2023-11-08T19:25:47.046 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.033+0000 7f67037a6700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66fc0c8da0 con 0x7f66fc15a3a0 2023-11-08T19:25:47.046 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.033+0000 7f67037a6700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66fc159380 con 0x7f66fc158e00 2023-11-08T19:25:47.046 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.033+0000 7f6701542700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f66fc158a20 0x7f66fc159e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:47.047 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.033+0000 7f6701542700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f66fc158a20 0x7f66fc159e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50506/0 (socket says 172.21.15.143:50506) 2023-11-08T19:25:47.047 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.033+0000 7f6701542700 1 -- 172.21.15.143:0/3643787177 learned_addr learned my addr 172.21.15.143:0/3643787177 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:47.047 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.034+0000 7f6701d43700 1 --2- 172.21.15.143:0/3643787177 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f66fc158e00 0x7f66fc0b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:47.047 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.034+0000 7f6701542700 1 -- 172.21.15.143:0/3643787177 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f66fc158e00 msgr2=0x7f66fc0b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:47.047 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.034+0000 7f6700d41700 1 --2- 172.21.15.143:0/3643787177 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f66fc15a3a0 0x7f66fc0b46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:47.048 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.034+0000 7f6701542700 1 --2- 172.21.15.143:0/3643787177 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f66fc158e00 0x7f66fc0b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:47.048 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.034+0000 7f6701542700 1 -- 172.21.15.143:0/3643787177 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f66fc15a3a0 msgr2=0x7f66fc0b46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:47.048 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.034+0000 7f6701542700 1 --2- 172.21.15.143:0/3643787177 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f66fc15a3a0 0x7f66fc0b46b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:47.048 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.034+0000 7f6701542700 1 -- 172.21.15.143:0/3643787177 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66fc159660 con 0x7f66fc158a20 2023-11-08T19:25:47.048 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.034+0000 7f6701d43700 1 --2- 172.21.15.143:0/3643787177 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f66fc158e00 0x7f66fc0b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:25:47.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.034+0000 7f6700d41700 1 --2- 172.21.15.143:0/3643787177 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f66fc15a3a0 0x7f66fc0b46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:25:47.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.034+0000 7f6701542700 1 --2- 172.21.15.143:0/3643787177 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f66fc158a20 0x7f66fc159e60 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f66f8007290 tx=0x7f66f8007650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:47.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.034+0000 7f66f1ffb700 1 -- 172.21.15.143:0/3643787177 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f800f7d0 con 0x7f66fc158a20 2023-11-08T19:25:47.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.034+0000 7f66f1ffb700 1 -- 172.21.15.143:0/3643787177 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66f800f950 con 0x7f66fc158a20 2023-11-08T19:25:47.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.034+0000 7f67037a6700 1 -- 172.21.15.143:0/3643787177 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66fc159940 con 0x7f66fc158a20 2023-11-08T19:25:47.050 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.034+0000 7f66f1ffb700 1 -- 172.21.15.143:0/3643787177 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f800fd00 con 0x7f66fc158a20 2023-11-08T19:25:47.050 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.034+0000 7f67037a6700 1 -- 172.21.15.143:0/3643787177 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66fc0c8210 con 0x7f66fc158a20 2023-11-08T19:25:47.050 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.035+0000 7f67037a6700 1 -- 172.21.15.143:0/3643787177 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66fc0ca580 con 0x7f66fc158a20 2023-11-08T19:25:47.050 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.035+0000 7f66f1ffb700 1 -- 172.21.15.143:0/3643787177 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f66f801d460 con 0x7f66fc158a20 2023-11-08T19:25:47.050 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.036+0000 7f66f1ffb700 1 --2- 172.21.15.143:0/3643787177 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f66e006af00 0x7f66e006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:47.051 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.036+0000 7f66f1ffb700 1 -- 172.21.15.143:0/3643787177 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f66f80892b0 con 0x7f66fc158a20 2023-11-08T19:25:47.051 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.036+0000 7f6700d41700 1 --2- 172.21.15.143:0/3643787177 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f66e006af00 0x7f66e006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:47.051 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.036+0000 7f6700d41700 1 --2- 172.21.15.143:0/3643787177 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f66e006af00 0x7f66e006d3c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f66fc0cdd50 tx=0x7f66ec009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:47.051 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.041+0000 7f66f1ffb700 1 -- 172.21.15.143:0/3643787177 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f66f8054740 con 0x7f66fc158a20 2023-11-08T19:25:47.229 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.227+0000 7f67037a6700 1 -- 172.21.15.143:0/3643787177 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f66fc04eee0 con 0x7f66fc158a20 2023-11-08T19:25:47.230 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.227+0000 7f66f1ffb700 1 -- 172.21.15.143:0/3643787177 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) v1 ==== 93+0+10 (secure 0 0 0) 0x7f66f8057d60 con 0x7f66fc158a20 2023-11-08T19:25:47.230 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2023-11-08T19:25:47.232 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.229+0000 7f67037a6700 1 -- 172.21.15.143:0/3643787177 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f66e006af00 msgr2=0x7f66e006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:47.232 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.229+0000 7f67037a6700 1 --2- 172.21.15.143:0/3643787177 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f66e006af00 0x7f66e006d3c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f66fc0cdd50 tx=0x7f66ec009040 comp rx=0 tx=0).stop 2023-11-08T19:25:47.232 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.229+0000 7f67037a6700 1 -- 172.21.15.143:0/3643787177 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f66fc158a20 msgr2=0x7f66fc159e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:47.232 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.229+0000 7f67037a6700 1 --2- 172.21.15.143:0/3643787177 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f66fc158a20 0x7f66fc159e60 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f66f8007290 tx=0x7f66f8007650 comp rx=0 tx=0).stop 2023-11-08T19:25:47.233 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.229+0000 7f67037a6700 1 -- 172.21.15.143:0/3643787177 shutdown_connections 2023-11-08T19:25:47.233 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.229+0000 7f67037a6700 1 --2- 172.21.15.143:0/3643787177 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f66e006af00 0x7f66e006d3c0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:47.233 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.229+0000 7f67037a6700 1 --2- 172.21.15.143:0/3643787177 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f66fc158e00 0x7f66fc0b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:47.233 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.229+0000 7f67037a6700 1 --2- 172.21.15.143:0/3643787177 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f66fc158a20 0x7f66fc159e60 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:47.233 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.229+0000 7f67037a6700 1 --2- 172.21.15.143:0/3643787177 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f66fc15a3a0 0x7f66fc0b46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:47.233 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.229+0000 7f67037a6700 1 -- 172.21.15.143:0/3643787177 >> 172.21.15.143:0/3643787177 conn(0x7f66fc066d00 msgr2=0x7f66fc0d1fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:47.234 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.229+0000 7f67037a6700 1 -- 172.21.15.143:0/3643787177 shutdown_connections 2023-11-08T19:25:47.234 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.229+0000 7f67037a6700 1 -- 172.21.15.143:0/3643787177 wait complete. 2023-11-08T19:25:47.244 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-08T19:25:47.363 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.361+0000 7f996c8ba700 1 Processor -- start 2023-11-08T19:25:47.363 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.361+0000 7f996c8ba700 1 -- start start 2023-11-08T19:25:47.369 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.361+0000 7f996c8ba700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f996806c030 0x7f99680cd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:47.369 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.361+0000 7f996c8ba700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99680cd480 con 0x7f996815de80 2023-11-08T19:25:47.370 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.361+0000 7f996c8ba700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99680cc610 con 0x7f996806c730 2023-11-08T19:25:47.370 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.361+0000 7f996c8ba700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99680cc870 con 0x7f996806c030 2023-11-08T19:25:47.370 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.361+0000 7f9966d9d700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f996815de80 0x7f99680cbf00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:46672/0 (socket says 172.21.15.143:46672) 2023-11-08T19:25:47.370 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.362+0000 7f9966d9d700 1 -- 172.21.15.143:0/3028624841 learned_addr learned my addr 172.21.15.143:0/3028624841 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:25:47.370 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.362+0000 7f9965d9b700 1 --2- 172.21.15.143:0/3028624841 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f996806c030 0x7f99680cd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:47.371 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.362+0000 7f996559a700 1 -- 172.21.15.143:0/3028624841 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 986855686 0 0) 0x7f99680cd480 con 0x7f996815de80 2023-11-08T19:25:47.371 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.362+0000 7f996559a700 1 -- 172.21.15.143:0/3028624841 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f99500035b0 con 0x7f996815de80 2023-11-08T19:25:47.371 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.362+0000 7f996559a700 1 -- 172.21.15.143:0/3028624841 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3972158371 0 0) 0x7f99500035b0 con 0x7f996815de80 2023-11-08T19:25:47.371 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.362+0000 7f996559a700 1 -- 172.21.15.143:0/3028624841 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f996806c030 msgr2=0x7f99680cd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:47.371 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.362+0000 7f996559a700 1 --2- 172.21.15.143:0/3028624841 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f996806c030 0x7f99680cd910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:47.372 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.362+0000 7f996559a700 1 -- 172.21.15.143:0/3028624841 >> v1:172.21.15.174:6789/0 conn(0x7f996806c730 legacy=0x7f996806cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:47.372 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.362+0000 7f996559a700 1 -- 172.21.15.143:0/3028624841 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99680ccb50 con 0x7f996815de80 2023-11-08T19:25:47.372 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.362+0000 7f996559a700 1 -- 172.21.15.143:0/3028624841 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4147756303 0 0) 0x7f99600019f0 con 0x7f996815de80 2023-11-08T19:25:47.372 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.362+0000 7f996559a700 1 -- 172.21.15.143:0/3028624841 >> v1:172.21.15.143:6789/0 conn(0x7f996815de80 legacy=0x7f99680cbf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:47.372 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.362+0000 7f996559a700 1 --2- 172.21.15.143:0/3028624841 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9950003a30 0x7f9950003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:47.373 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.362+0000 7f996559a700 1 --2- 172.21.15.143:0/3028624841 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f996806c030 0x7f9950004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:47.373 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.362+0000 7f996559a700 1 --2- 172.21.15.143:0/3028624841 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9950004d90 0x7f9950005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:47.373 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.362+0000 7f996559a700 1 -- 172.21.15.143:0/3028624841 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99680cc870 con 0x7f9950003a30 2023-11-08T19:25:47.373 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.362+0000 7f996559a700 1 -- 172.21.15.143:0/3028624841 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f995000ba10 con 0x7f9950004d90 2023-11-08T19:25:47.373 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.362+0000 7f996559a700 1 -- 172.21.15.143:0/3028624841 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f995000bbd0 con 0x7f996806c030 2023-11-08T19:25:47.374 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.362+0000 7f996659c700 1 --2- 172.21.15.143:0/3028624841 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9950003a30 0x7f9950003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:47.374 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.362+0000 7f9966d9d700 1 --2- 172.21.15.143:0/3028624841 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9950004d90 0x7f9950005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:47.374 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.362+0000 7f9965d9b700 1 --2- 172.21.15.143:0/3028624841 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f996806c030 0x7f9950004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:47.374 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.362+0000 7f996659c700 1 -- 172.21.15.143:0/3028624841 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f996806c030 msgr2=0x7f9950004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:47.375 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.362+0000 7f996659c700 1 --2- 172.21.15.143:0/3028624841 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f996806c030 0x7f9950004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:47.375 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.362+0000 7f996659c700 1 -- 172.21.15.143:0/3028624841 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9950004d90 msgr2=0x7f9950005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:47.375 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.362+0000 7f996659c700 1 --2- 172.21.15.143:0/3028624841 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9950004d90 0x7f9950005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:47.375 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.362+0000 7f996659c700 1 -- 172.21.15.143:0/3028624841 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99500035b0 con 0x7f9950003a30 2023-11-08T19:25:47.375 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.362+0000 7f9966d9d700 1 --2- 172.21.15.143:0/3028624841 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9950004d90 0x7f9950005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:25:47.375 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.362+0000 7f9965d9b700 1 --2- 172.21.15.143:0/3028624841 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f996806c030 0x7f9950004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:25:47.376 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.363+0000 7f996659c700 1 --2- 172.21.15.143:0/3028624841 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9950003a30 0x7f9950003e10 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f995c0018d0 tx=0x7f9950005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a057a6d5b3b82be0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:47.376 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.363+0000 7f996559a700 1 -- 172.21.15.143:0/3028624841 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f995c012620 con 0x7f9950003a30 2023-11-08T19:25:47.376 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.363+0000 7f996559a700 1 -- 172.21.15.143:0/3028624841 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f995c0127a0 con 0x7f9950003a30 2023-11-08T19:25:47.376 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.363+0000 7f996559a700 1 -- 172.21.15.143:0/3028624841 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f995c012b50 con 0x7f9950003a30 2023-11-08T19:25:47.376 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.363+0000 7f996c8ba700 1 -- 172.21.15.143:0/3028624841 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9950003a30 msgr2=0x7f9950003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:47.377 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.363+0000 7f996c8ba700 1 --2- 172.21.15.143:0/3028624841 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9950003a30 0x7f9950003e10 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f995c0018d0 tx=0x7f9950005880 comp rx=0 tx=0).stop 2023-11-08T19:25:47.377 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.363+0000 7f9966d9d700 1 -- 172.21.15.143:0/3028624841 reap_dead start 2023-11-08T19:25:47.377 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.363+0000 7f996c8ba700 1 -- 172.21.15.143:0/3028624841 shutdown_connections 2023-11-08T19:25:47.377 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.363+0000 7f996c8ba700 1 -- 172.21.15.143:0/3028624841 >> 172.21.15.143:0/3028624841 conn(0x7f996808b000 msgr2=0x7f996809ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:47.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.363+0000 7f996c8ba700 1 -- 172.21.15.143:0/3028624841 shutdown_connections 2023-11-08T19:25:47.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.363+0000 7f996c8ba700 1 -- 172.21.15.143:0/3028624841 wait complete. 2023-11-08T19:25:47.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.365+0000 7f996c8ba700 1 Processor -- start 2023-11-08T19:25:47.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.365+0000 7f996c8ba700 1 -- start start 2023-11-08T19:25:47.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.365+0000 7f996c8ba700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9950003a30 0x7f99680b44d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:47.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.365+0000 7f996c8ba700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f99680b6ed0 0x7f9968152c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:47.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.366+0000 7f996c8ba700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f99680b4ee0 0x7f99680b5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:47.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.366+0000 7f996c8ba700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99680b58d0 con 0x7f99680b4ee0 2023-11-08T19:25:47.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.366+0000 7f996c8ba700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99681f3e00 con 0x7f99680b6ed0 2023-11-08T19:25:47.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.366+0000 7f996c8ba700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99681f3270 con 0x7f9950003a30 2023-11-08T19:25:47.380 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.366+0000 7f9966d9d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f99680b4ee0 0x7f99680b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:47.380 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.366+0000 7f9966d9d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f99680b4ee0 0x7f99680b5390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50530/0 (socket says 172.21.15.143:50530) 2023-11-08T19:25:47.380 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.366+0000 7f9966d9d700 1 -- 172.21.15.143:0/1079001064 learned_addr learned my addr 172.21.15.143:0/1079001064 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:47.380 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.366+0000 7f996659c700 1 --2- 172.21.15.143:0/1079001064 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9950003a30 0x7f99680b44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:47.380 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.366+0000 7f9966d9d700 1 -- 172.21.15.143:0/1079001064 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9950003a30 msgr2=0x7f99680b44d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:47.381 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.366+0000 7f9966d9d700 1 --2- 172.21.15.143:0/1079001064 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9950003a30 0x7f99680b44d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:47.381 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.366+0000 7f9965d9b700 1 --2- 172.21.15.143:0/1079001064 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f99680b6ed0 0x7f9968152c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:47.381 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.366+0000 7f9966d9d700 1 -- 172.21.15.143:0/1079001064 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f99680b6ed0 msgr2=0x7f9968152c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:47.381 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.366+0000 7f9966d9d700 1 --2- 172.21.15.143:0/1079001064 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f99680b6ed0 0x7f9968152c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:47.381 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.366+0000 7f9966d9d700 1 -- 172.21.15.143:0/1079001064 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99681f34d0 con 0x7f99680b4ee0 2023-11-08T19:25:47.382 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.366+0000 7f996659c700 1 --2- 172.21.15.143:0/1079001064 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9950003a30 0x7f99680b44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:25:47.382 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.366+0000 7f9965d9b700 1 --2- 172.21.15.143:0/1079001064 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f99680b6ed0 0x7f9968152c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:25:47.382 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.366+0000 7f9966d9d700 1 --2- 172.21.15.143:0/1079001064 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f99680b4ee0 0x7f99680b5390 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f99680c8930 tx=0x7f9960007c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:47.382 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.366+0000 7f9956ffd700 1 -- 172.21.15.143:0/1079001064 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f996000ed10 con 0x7f99680b4ee0 2023-11-08T19:25:47.382 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.366+0000 7f9956ffd700 1 -- 172.21.15.143:0/1079001064 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f996000ee70 con 0x7f99680b4ee0 2023-11-08T19:25:47.383 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.366+0000 7f996c8ba700 1 -- 172.21.15.143:0/1079001064 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99681f37b0 con 0x7f99680b4ee0 2023-11-08T19:25:47.383 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.366+0000 7f9956ffd700 1 -- 172.21.15.143:0/1079001064 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9960013660 con 0x7f99680b4ee0 2023-11-08T19:25:47.383 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.366+0000 7f996c8ba700 1 -- 172.21.15.143:0/1079001064 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99681531a0 con 0x7f99680b4ee0 2023-11-08T19:25:47.384 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.367+0000 7f996c8ba700 1 -- 172.21.15.143:0/1079001064 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99681f3f70 con 0x7f99680b4ee0 2023-11-08T19:25:47.384 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.368+0000 7f9956ffd700 1 -- 172.21.15.143:0/1079001064 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f99600137c0 con 0x7f99680b4ee0 2023-11-08T19:25:47.385 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.368+0000 7f9956ffd700 1 --2- 172.21.15.143:0/1079001064 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f994806af00 0x7f994806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:47.385 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.368+0000 7f9956ffd700 1 -- 172.21.15.143:0/1079001064 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f99600898e0 con 0x7f99680b4ee0 2023-11-08T19:25:47.385 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.369+0000 7f996659c700 1 --2- 172.21.15.143:0/1079001064 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f994806af00 0x7f994806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:47.385 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.369+0000 7f996659c700 1 --2- 172.21.15.143:0/1079001064 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f994806af00 0x7f994806d3c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f99680d2450 tx=0x7f995c005040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:47.385 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.373+0000 7f9956ffd700 1 -- 172.21.15.143:0/1079001064 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9960054ce0 con 0x7f99680b4ee0 2023-11-08T19:25:47.567 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.565+0000 7f996c8ba700 1 -- 172.21.15.143:0/1079001064 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f9968153330 con 0x7f99680b4ee0 2023-11-08T19:25:47.569 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.567+0000 7f9956ffd700 1 -- 172.21.15.143:0/1079001064 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v29) v1 ==== 84+0+65 (secure 0 0 0) 0x7f9968153330 con 0x7f99680b4ee0 2023-11-08T19:25:47.569 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:25:47.570 INFO:teuthology.orchestra.run.smithi143.stdout:[ 2023-11-08T19:25:47.570 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-08T19:25:47.570 INFO:teuthology.orchestra.run.smithi143.stdout: "poolnum": 1, 2023-11-08T19:25:47.570 INFO:teuthology.orchestra.run.smithi143.stdout: "poolname": ".mgr" 2023-11-08T19:25:47.571 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-08T19:25:47.571 INFO:teuthology.orchestra.run.smithi143.stdout:] 2023-11-08T19:25:47.572 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.569+0000 7f996c8ba700 1 -- 172.21.15.143:0/1079001064 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f994806af00 msgr2=0x7f994806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:47.572 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.569+0000 7f996c8ba700 1 --2- 172.21.15.143:0/1079001064 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f994806af00 0x7f994806d3c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f99680d2450 tx=0x7f995c005040 comp rx=0 tx=0).stop 2023-11-08T19:25:47.572 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.569+0000 7f996c8ba700 1 -- 172.21.15.143:0/1079001064 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f99680b4ee0 msgr2=0x7f99680b5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:47.572 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.569+0000 7f996c8ba700 1 --2- 172.21.15.143:0/1079001064 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f99680b4ee0 0x7f99680b5390 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f99680c8930 tx=0x7f9960007c90 comp rx=0 tx=0).stop 2023-11-08T19:25:47.573 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.569+0000 7f996c8ba700 1 -- 172.21.15.143:0/1079001064 shutdown_connections 2023-11-08T19:25:47.573 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.569+0000 7f996c8ba700 1 --2- 172.21.15.143:0/1079001064 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f994806af00 0x7f994806d3c0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:47.573 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.569+0000 7f996c8ba700 1 --2- 172.21.15.143:0/1079001064 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9950003a30 0x7f99680b44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:47.573 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.569+0000 7f996c8ba700 1 --2- 172.21.15.143:0/1079001064 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f99680b4ee0 0x7f99680b5390 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:47.573 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.569+0000 7f996c8ba700 1 --2- 172.21.15.143:0/1079001064 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f99680b6ed0 0x7f9968152c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:47.574 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.569+0000 7f996c8ba700 1 -- 172.21.15.143:0/1079001064 >> 172.21.15.143:0/1079001064 conn(0x7f996808b000 msgr2=0x7f996806f310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:47.574 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.569+0000 7f996c8ba700 1 -- 172.21.15.143:0/1079001064 shutdown_connections 2023-11-08T19:25:47.574 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.569+0000 7f996c8ba700 1 -- 172.21.15.143:0/1079001064 wait complete. 2023-11-08T19:25:47.583 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-08T19:25:47.584 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-08T19:25:47.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.706+0000 7fea959a5700 1 Processor -- start 2023-11-08T19:25:47.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.706+0000 7fea959a5700 1 -- start start 2023-11-08T19:25:47.710 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.706+0000 7fea959a5700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fea900b5630 0x7fea900b5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:47.710 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.706+0000 7fea959a5700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fea900d1bd0 0x7fea900d1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:47.710 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.706+0000 7fea959a5700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea900b9b00 con 0x7fea9015dca0 2023-11-08T19:25:47.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.706+0000 7fea959a5700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea900b6020 con 0x7fea900b5630 2023-11-08T19:25:47.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.706+0000 7fea959a5700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea900b3360 con 0x7fea900d1bd0 2023-11-08T19:25:47.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.706+0000 7fea8e7fc700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fea900b5630 0x7fea900b5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:47.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.706+0000 7fea8e7fc700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fea900b5630 0x7fea900b5ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50536/0 (socket says 172.21.15.143:50536) 2023-11-08T19:25:47.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.706+0000 7fea8e7fc700 1 -- 172.21.15.143:0/4029133664 learned_addr learned my addr 172.21.15.143:0/4029133664 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:47.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.706+0000 7fea8f7fe700 1 --2- 172.21.15.143:0/4029133664 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fea900d1bd0 0x7fea900d1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:47.712 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.706+0000 7fea8e7fc700 1 -- 172.21.15.143:0/4029133664 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fea900d1bd0 msgr2=0x7fea900d1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:47.712 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.706+0000 7fea8e7fc700 1 --2- 172.21.15.143:0/4029133664 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fea900d1bd0 0x7fea900d1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:47.712 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.706+0000 7fea8e7fc700 1 -- 172.21.15.143:0/4029133664 >> v1:172.21.15.174:6789/0 conn(0x7fea9015dca0 legacy=0x7fea900b93f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:47.712 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.706+0000 7fea8e7fc700 1 -- 172.21.15.143:0/4029133664 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea900b8f50 con 0x7fea900b5630 2023-11-08T19:25:47.713 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.706+0000 7fea8e7fc700 1 --2- 172.21.15.143:0/4029133664 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fea900b5630 0x7fea900b5ae0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fea80009920 tx=0x7fea80009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1c80f7c63c4451a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:47.713 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.706+0000 7fea8dffb700 1 -- 172.21.15.143:0/4029133664 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea8000ba20 con 0x7fea900b5630 2023-11-08T19:25:47.713 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.706+0000 7fea8dffb700 1 -- 172.21.15.143:0/4029133664 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea8000bb80 con 0x7fea900b5630 2023-11-08T19:25:47.713 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.706+0000 7fea8dffb700 1 -- 172.21.15.143:0/4029133664 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea80009e70 con 0x7fea900b5630 2023-11-08T19:25:47.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.708+0000 7fea959a5700 1 -- 172.21.15.143:0/4029133664 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fea900b5630 msgr2=0x7fea900b5ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:47.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.708+0000 7fea959a5700 1 --2- 172.21.15.143:0/4029133664 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fea900b5630 0x7fea900b5ae0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fea80009920 tx=0x7fea80009c30 comp rx=0 tx=0).stop 2023-11-08T19:25:47.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.708+0000 7fea959a5700 1 -- 172.21.15.143:0/4029133664 shutdown_connections 2023-11-08T19:25:47.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.708+0000 7fea959a5700 1 --2- 172.21.15.143:0/4029133664 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fea900d1bd0 0x7fea900d1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:47.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.708+0000 7fea959a5700 1 --2- 172.21.15.143:0/4029133664 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fea900b5630 0x7fea900b5ae0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:47.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.708+0000 7fea959a5700 1 -- 172.21.15.143:0/4029133664 >> 172.21.15.143:0/4029133664 conn(0x7fea90066d00 msgr2=0x7fea90067110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:47.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.708+0000 7fea959a5700 1 -- 172.21.15.143:0/4029133664 shutdown_connections 2023-11-08T19:25:47.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.708+0000 7fea959a5700 1 -- 172.21.15.143:0/4029133664 wait complete. 2023-11-08T19:25:47.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.709+0000 7fea959a5700 1 Processor -- start 2023-11-08T19:25:47.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.709+0000 7fea959a5700 1 -- start start 2023-11-08T19:25:47.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.710+0000 7fea959a5700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fea900b5630 0x7fea9015b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:47.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.710+0000 7fea959a5700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fea900d1bd0 0x7fea900c9290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:47.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.710+0000 7fea959a5700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fea9015dca0 0x7fea900c7530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:47.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.710+0000 7fea959a5700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea9015bb50 con 0x7fea9015dca0 2023-11-08T19:25:47.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.710+0000 7fea959a5700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea9015e080 con 0x7fea900d1bd0 2023-11-08T19:25:47.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.710+0000 7fea959a5700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea901f35f0 con 0x7fea900b5630 2023-11-08T19:25:47.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.710+0000 7fea8f7fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fea9015dca0 0x7fea900c7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:47.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.710+0000 7fea8f7fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fea9015dca0 0x7fea900c7530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50544/0 (socket says 172.21.15.143:50544) 2023-11-08T19:25:47.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.710+0000 7fea8f7fe700 1 -- 172.21.15.143:0/3811998745 learned_addr learned my addr 172.21.15.143:0/3811998745 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:47.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.710+0000 7fea8e7fc700 1 --2- 172.21.15.143:0/3811998745 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fea900d1bd0 0x7fea900c9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:47.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.710+0000 7fea8effd700 1 --2- 172.21.15.143:0/3811998745 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fea900b5630 0x7fea9015b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:47.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.710+0000 7fea8f7fe700 1 -- 172.21.15.143:0/3811998745 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fea900b5630 msgr2=0x7fea9015b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:47.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.710+0000 7fea8f7fe700 1 --2- 172.21.15.143:0/3811998745 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fea900b5630 0x7fea9015b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:47.721 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.710+0000 7fea8f7fe700 1 -- 172.21.15.143:0/3811998745 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fea900d1bd0 msgr2=0x7fea900c9290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:47.721 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.710+0000 7fea8f7fe700 1 --2- 172.21.15.143:0/3811998745 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fea900d1bd0 0x7fea900c9290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:47.721 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.710+0000 7fea8f7fe700 1 -- 172.21.15.143:0/3811998745 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea80009550 con 0x7fea9015dca0 2023-11-08T19:25:47.722 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.710+0000 7fea8f7fe700 1 --2- 172.21.15.143:0/3811998745 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fea9015dca0 0x7fea900c7530 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fea88007e40 tx=0x7fea8800e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:47.722 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.710+0000 7fea77fff700 1 -- 172.21.15.143:0/3811998745 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea8800eea0 con 0x7fea9015dca0 2023-11-08T19:25:47.722 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.710+0000 7fea77fff700 1 -- 172.21.15.143:0/3811998745 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea88014420 con 0x7fea9015dca0 2023-11-08T19:25:47.722 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.710+0000 7fea959a5700 1 -- 172.21.15.143:0/3811998745 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea901f38d0 con 0x7fea9015dca0 2023-11-08T19:25:47.722 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.710+0000 7fea77fff700 1 -- 172.21.15.143:0/3811998745 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea880147b0 con 0x7fea9015dca0 2023-11-08T19:25:47.723 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.710+0000 7fea959a5700 1 -- 172.21.15.143:0/3811998745 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea901f3e90 con 0x7fea9015dca0 2023-11-08T19:25:47.723 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.712+0000 7fea959a5700 1 -- 172.21.15.143:0/3811998745 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea9015e2a0 con 0x7fea9015dca0 2023-11-08T19:25:47.723 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.712+0000 7fea77fff700 1 -- 172.21.15.143:0/3811998745 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fea88014910 con 0x7fea9015dca0 2023-11-08T19:25:47.723 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.713+0000 7fea77fff700 1 --2- 172.21.15.143:0/3811998745 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fea7c06af00 0x7fea7c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:47.723 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.713+0000 7fea77fff700 1 -- 172.21.15.143:0/3811998745 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fea88089ae0 con 0x7fea9015dca0 2023-11-08T19:25:47.724 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.713+0000 7fea8effd700 1 --2- 172.21.15.143:0/3811998745 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fea7c06af00 0x7fea7c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:47.724 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.713+0000 7fea8effd700 1 --2- 172.21.15.143:0/3811998745 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fea7c06af00 0x7fea7c06d3c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fea900d43d0 tx=0x7fea84009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:47.724 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.717+0000 7fea77fff700 1 -- 172.21.15.143:0/3811998745 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fea9015e2a0 con 0x7fea9015dca0 2023-11-08T19:25:47.908 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:47.907+0000 7fea959a5700 1 -- 172.21.15.143:0/3811998745 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fea9015e2a0 con 0x7fea9015dca0 2023-11-08T19:25:48.854 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:48.853+0000 7fea77fff700 1 -- 172.21.15.143:0/3811998745 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v30) v1 ==== 138+0+0 (secure 0 0 0) 0x7fea9015e2a0 con 0x7fea9015dca0 2023-11-08T19:25:48.861 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:48.856+0000 7fea959a5700 1 -- 172.21.15.143:0/3811998745 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fea7c06af00 msgr2=0x7fea7c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:48.861 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:48.856+0000 7fea959a5700 1 --2- 172.21.15.143:0/3811998745 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fea7c06af00 0x7fea7c06d3c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fea900d43d0 tx=0x7fea84009040 comp rx=0 tx=0).stop 2023-11-08T19:25:48.862 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:48.856+0000 7fea959a5700 1 -- 172.21.15.143:0/3811998745 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fea9015dca0 msgr2=0x7fea900c7530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:48.862 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:48.856+0000 7fea959a5700 1 --2- 172.21.15.143:0/3811998745 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fea9015dca0 0x7fea900c7530 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fea88007e40 tx=0x7fea8800e3c0 comp rx=0 tx=0).stop 2023-11-08T19:25:48.862 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:48.856+0000 7fea959a5700 1 -- 172.21.15.143:0/3811998745 shutdown_connections 2023-11-08T19:25:48.862 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:48.856+0000 7fea959a5700 1 --2- 172.21.15.143:0/3811998745 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fea7c06af00 0x7fea7c06d3c0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:48.862 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:48.856+0000 7fea959a5700 1 --2- 172.21.15.143:0/3811998745 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fea900b5630 0x7fea9015b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:48.863 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:48.856+0000 7fea959a5700 1 --2- 172.21.15.143:0/3811998745 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fea9015dca0 0x7fea900c7530 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:48.863 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:48.856+0000 7fea959a5700 1 --2- 172.21.15.143:0/3811998745 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fea900d1bd0 0x7fea900c9290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:48.863 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:48.856+0000 7fea959a5700 1 -- 172.21.15.143:0/3811998745 >> 172.21.15.143:0/3811998745 conn(0x7fea90066d00 msgr2=0x7fea900b34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:48.863 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:48.857+0000 7fea959a5700 1 -- 172.21.15.143:0/3811998745 shutdown_connections 2023-11-08T19:25:48.863 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:48.857+0000 7fea959a5700 1 -- 172.21.15.143:0/3811998745 wait complete. 2023-11-08T19:25:48.864 INFO:teuthology.orchestra.run.smithi143.stderr:pool 'cephfs_metadata' created 2023-11-08T19:25:48.875 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-08T19:25:49.008 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:49.007+0000 7fc9d9a15700 1 Processor -- start 2023-11-08T19:25:49.009 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:49.007+0000 7fc9d9a15700 1 -- start start 2023-11-08T19:25:49.011 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:49.008+0000 7fc9d9a15700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc9d415dca0 0x7fc9d40b93f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:49.011 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:49.008+0000 7fc9d9a15700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fc9d40b5630 0x7fc9d40b5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:49.011 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:49.008+0000 7fc9d9a15700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc9d40b6b70 0x7fc9d40b6f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:49.012 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:49.008+0000 7fc9d9a15700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9d40b6020 con 0x7fc9d415dca0 2023-11-08T19:25:49.012 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:49.008+0000 7fc9d9a15700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9d40b3360 con 0x7fc9d40b5630 2023-11-08T19:25:49.012 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:49.008+0000 7fc9d9a15700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9d40b8f50 con 0x7fc9d40b6b70 2023-11-08T19:25:49.012 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:49.008+0000 7fc9d2ffd700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc9d415dca0 0x7fc9d40b93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:49.012 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:49.008+0000 7fc9d2ffd700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc9d415dca0 0x7fc9d40b93f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50548/0 (socket says 172.21.15.143:50548) 2023-11-08T19:25:49.013 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:49.008+0000 7fc9d2ffd700 1 -- 172.21.15.143:0/1464868030 learned_addr learned my addr 172.21.15.143:0/1464868030 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:49.013 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:49.008+0000 7fc9d37fe700 1 --2- 172.21.15.143:0/1464868030 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc9d40b6b70 0x7fc9d40b6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:49.013 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:49.008+0000 7fc9d2ffd700 1 -- 172.21.15.143:0/1464868030 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc9d40b6b70 msgr2=0x7fc9d40b6f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:49.013 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:49.008+0000 7fc9d27fc700 1 --2- 172.21.15.143:0/1464868030 >> v2:172.21.15.174:3300/0 conn(0x7fc9d40b5630 0x7fc9d40b5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:49.013 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:49.008+0000 7fc9d2ffd700 1 --2- 172.21.15.143:0/1464868030 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc9d40b6b70 0x7fc9d40b6f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:49.014 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:49.008+0000 7fc9d2ffd700 1 -- 172.21.15.143:0/1464868030 >> v2:172.21.15.174:3300/0 conn(0x7fc9d40b5630 msgr2=0x7fc9d40b5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:49.014 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:49.008+0000 7fc9d2ffd700 1 --2- 172.21.15.143:0/1464868030 >> v2:172.21.15.174:3300/0 conn(0x7fc9d40b5630 0x7fc9d40b5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:49.014 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:49.008+0000 7fc9d2ffd700 1 -- 172.21.15.143:0/1464868030 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9d40b7eb0 con 0x7fc9d415dca0 2023-11-08T19:25:49.015 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:49.008+0000 7fc9d37fe700 1 --2- 172.21.15.143:0/1464868030 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc9d40b6b70 0x7fc9d40b6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:25:49.015 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:49.008+0000 7fc9d27fc700 1 --2- 172.21.15.143:0/1464868030 >> v2:172.21.15.174:3300/0 conn(0x7fc9d40b5630 0x7fc9d40b5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:25:49.015 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:49.008+0000 7fc9d2ffd700 1 --2- 172.21.15.143:0/1464868030 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc9d415dca0 0x7fc9d40b93f0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fc9d40cf5f0 tx=0x7fc9c4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db9560dee9c38f30 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:49.015 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:49.008+0000 7fc9d1ffb700 1 -- 172.21.15.143:0/1464868030 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c400b870 con 0x7fc9d415dca0 2023-11-08T19:25:49.015 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:49.008+0000 7fc9d1ffb700 1 -- 172.21.15.143:0/1464868030 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9c400b9d0 con 0x7fc9d415dca0 2023-11-08T19:25:49.016 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:49.008+0000 7fc9d1ffb700 1 -- 172.21.15.143:0/1464868030 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c400bd60 con 0x7fc9d415dca0 2023-11-08T19:25:49.016 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:49.009+0000 7fc9d9a15700 1 -- 172.21.15.143:0/1464868030 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc9d415dca0 msgr2=0x7fc9d40b93f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:49.016 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:49.009+0000 7fc9d9a15700 1 --2- 172.21.15.143:0/1464868030 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc9d415dca0 0x7fc9d40b93f0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fc9d40cf5f0 tx=0x7fc9c4009bd0 comp rx=0 tx=0).stop 2023-11-08T19:25:49.016 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:49.009+0000 7fc9d9a15700 1 -- 172.21.15.143:0/1464868030 shutdown_connections 2023-11-08T19:25:49.016 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:49.009+0000 7fc9d9a15700 1 --2- 172.21.15.143:0/1464868030 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc9d40b6b70 0x7fc9d40b6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:49.017 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:49.009+0000 7fc9d9a15700 1 --2- 172.21.15.143:0/1464868030 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc9d415dca0 0x7fc9d40b93f0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:49.017 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:49.009+0000 7fc9d9a15700 1 --2- 172.21.15.143:0/1464868030 >> v2:172.21.15.174:3300/0 conn(0x7fc9d40b5630 0x7fc9d40b5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:49.017 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:49.009+0000 7fc9d9a15700 1 -- 172.21.15.143:0/1464868030 >> 172.21.15.143:0/1464868030 conn(0x7fc9d4066d00 msgr2=0x7fc9d4067110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:49.017 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:49.009+0000 7fc9d9a15700 1 -- 172.21.15.143:0/1464868030 shutdown_connections 2023-11-08T19:25:49.017 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:49.009+0000 7fc9d9a15700 1 -- 172.21.15.143:0/1464868030 wait complete. 2023-11-08T19:25:49.020 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:49.010+0000 7fc9d9a15700 1 Processor -- start 2023-11-08T19:25:49.020 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:49.010+0000 7fc9d9a15700 1 -- start start 2023-11-08T19:25:49.020 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:49.011+0000 7fc9d9a15700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc9d40b5630 0x7fc9d40cd470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:49.021 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:49.011+0000 7fc9d9a15700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc9d40b6b70 0x7fc9d4144f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:49.021 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:49.011+0000 7fc9d9a15700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc9d415dca0 0x7fc9d41454d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:49.021 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:49.011+0000 7fc9d9a15700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9d40c9440 con 0x7fc9d40b5630 2023-11-08T19:25:49.021 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:49.011+0000 7fc9d9a15700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9d40c9720 con 0x7fc9d40b6b70 2023-11-08T19:25:49.021 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:49.011+0000 7fc9d9a15700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9d40c9a00 con 0x7fc9d415dca0 2023-11-08T19:25:49.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:49.011+0000 7fc9d2ffd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc9d40b5630 0x7fc9d40cd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:49.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:49.011+0000 7fc9d2ffd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc9d40b5630 0x7fc9d40cd470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50554/0 (socket says 172.21.15.143:50554) 2023-11-08T19:25:49.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:49.011+0000 7fc9d2ffd700 1 -- 172.21.15.143:0/2834470545 learned_addr learned my addr 172.21.15.143:0/2834470545 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:49.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:49.011+0000 7fc9d27fc700 1 --2- 172.21.15.143:0/2834470545 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc9d40b6b70 0x7fc9d4144f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:49.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:49.011+0000 7fc9d37fe700 1 --2- 172.21.15.143:0/2834470545 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc9d415dca0 0x7fc9d41454d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:49.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:49.012+0000 7fc9d2ffd700 1 -- 172.21.15.143:0/2834470545 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc9d415dca0 msgr2=0x7fc9d41454d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:49.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:49.012+0000 7fc9d2ffd700 1 --2- 172.21.15.143:0/2834470545 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc9d415dca0 0x7fc9d41454d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:49.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:49.012+0000 7fc9d2ffd700 1 -- 172.21.15.143:0/2834470545 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc9d40b6b70 msgr2=0x7fc9d4144f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:49.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:49.012+0000 7fc9d2ffd700 1 --2- 172.21.15.143:0/2834470545 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc9d40b6b70 0x7fc9d4144f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:49.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:49.012+0000 7fc9d2ffd700 1 -- 172.21.15.143:0/2834470545 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9c4009550 con 0x7fc9d40b5630 2023-11-08T19:25:49.024 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:49.012+0000 7fc9d2ffd700 1 --2- 172.21.15.143:0/2834470545 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc9d40b5630 0x7fc9d40cd470 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fc9c4014040 tx=0x7fc9c4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:49.024 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:49.012+0000 7fc9bbfff700 1 -- 172.21.15.143:0/2834470545 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c4004250 con 0x7fc9d40b5630 2023-11-08T19:25:49.024 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:49.012+0000 7fc9bbfff700 1 -- 172.21.15.143:0/2834470545 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9c401e070 con 0x7fc9d40b5630 2023-11-08T19:25:49.024 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:49.012+0000 7fc9d9a15700 1 -- 172.21.15.143:0/2834470545 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9d40c9ce0 con 0x7fc9d40b5630 2023-11-08T19:25:49.024 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:49.012+0000 7fc9bbfff700 1 -- 172.21.15.143:0/2834470545 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c40036a0 con 0x7fc9d40b5630 2023-11-08T19:25:49.024 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:49.012+0000 7fc9d9a15700 1 -- 172.21.15.143:0/2834470545 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9d40cdb90 con 0x7fc9d40b5630 2023-11-08T19:25:49.025 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:49.013+0000 7fc9d9a15700 1 -- 172.21.15.143:0/2834470545 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9d404eee0 con 0x7fc9d40b5630 2023-11-08T19:25:49.025 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:49.013+0000 7fc9bbfff700 1 -- 172.21.15.143:0/2834470545 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc9c4020030 con 0x7fc9d40b5630 2023-11-08T19:25:49.025 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:49.014+0000 7fc9bbfff700 1 --2- 172.21.15.143:0/2834470545 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc9c006af00 0x7fc9c006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:49.025 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:49.014+0000 7fc9bbfff700 1 -- 172.21.15.143:0/2834470545 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5227+0+0 (secure 0 0 0) 0x7fc9c408a4a0 con 0x7fc9d40b5630 2023-11-08T19:25:49.025 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:49.014+0000 7fc9d27fc700 1 --2- 172.21.15.143:0/2834470545 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc9c006af00 0x7fc9c006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:49.026 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:49.014+0000 7fc9d27fc700 1 --2- 172.21.15.143:0/2834470545 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc9c006af00 0x7fc9c006d3c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc9c8006ba0 tx=0x7fc9c8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:49.026 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:49.018+0000 7fc9bbfff700 1 -- 172.21.15.143:0/2834470545 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc9c4055730 con 0x7fc9d40b5630 2023-11-08T19:25:49.206 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:49.205+0000 7fc9d9a15700 1 -- 172.21.15.143:0/2834470545 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fc9d40b2d00 con 0x7fc9d40b5630 2023-11-08T19:25:49.862 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:49.861+0000 7fc9bbfff700 1 -- 172.21.15.143:0/2834470545 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v31) v1 ==== 170+0+0 (secure 0 0 0) 0x7fc9c4058d50 con 0x7fc9d40b5630 2023-11-08T19:25:49.865 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:49.863+0000 7fc9d9a15700 1 -- 172.21.15.143:0/2834470545 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc9c006af00 msgr2=0x7fc9c006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:49.865 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:49.863+0000 7fc9d9a15700 1 --2- 172.21.15.143:0/2834470545 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc9c006af00 0x7fc9c006d3c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc9c8006ba0 tx=0x7fc9c8008040 comp rx=0 tx=0).stop 2023-11-08T19:25:49.866 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:49.863+0000 7fc9d9a15700 1 -- 172.21.15.143:0/2834470545 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc9d40b5630 msgr2=0x7fc9d40cd470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:49.866 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:49.863+0000 7fc9d9a15700 1 --2- 172.21.15.143:0/2834470545 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc9d40b5630 0x7fc9d40cd470 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fc9c4014040 tx=0x7fc9c4004750 comp rx=0 tx=0).stop 2023-11-08T19:25:49.867 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:49.863+0000 7fc9d9a15700 1 -- 172.21.15.143:0/2834470545 shutdown_connections 2023-11-08T19:25:49.867 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:49.863+0000 7fc9d9a15700 1 --2- 172.21.15.143:0/2834470545 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc9c006af00 0x7fc9c006d3c0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:49.867 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:49.863+0000 7fc9d9a15700 1 --2- 172.21.15.143:0/2834470545 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc9d415dca0 0x7fc9d41454d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:49.868 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:49.863+0000 7fc9d9a15700 1 --2- 172.21.15.143:0/2834470545 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc9d40b5630 0x7fc9d40cd470 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:49.869 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:49.863+0000 7fc9d9a15700 1 --2- 172.21.15.143:0/2834470545 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc9d40b6b70 0x7fc9d4144f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:49.869 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:49.863+0000 7fc9d9a15700 1 -- 172.21.15.143:0/2834470545 >> 172.21.15.143:0/2834470545 conn(0x7fc9d4066d00 msgr2=0x7fc9d40b7420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:49.869 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:49.864+0000 7fc9d9a15700 1 -- 172.21.15.143:0/2834470545 shutdown_connections 2023-11-08T19:25:49.869 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:49.864+0000 7fc9d9a15700 1 -- 172.21.15.143:0/2834470545 wait complete. 2023-11-08T19:25:49.870 INFO:teuthology.orchestra.run.smithi143.stderr:pool 'cephfs_data' created 2023-11-08T19:25:49.888 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-08T19:25:50.015 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:50.013+0000 7f8b782eb700 1 Processor -- start 2023-11-08T19:25:50.015 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:50.013+0000 7f8b782eb700 1 -- start start 2023-11-08T19:25:50.017 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:50.014+0000 7f8b782eb700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f8b7015a950 0x7f8b700d01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:50.017 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:50.014+0000 7f8b782eb700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8b7006bd20 0x7f8b70072370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:50.018 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:50.014+0000 7f8b782eb700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b700cc2a0 con 0x7f8b700728b0 2023-11-08T19:25:50.018 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:50.014+0000 7f8b782eb700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b700cc130 con 0x7f8b7015a950 2023-11-08T19:25:50.018 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:50.014+0000 7f8b782eb700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b7006c1b0 con 0x7f8b7006bd20 2023-11-08T19:25:50.018 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:50.014+0000 7f8b76087700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f8b7015a950 0x7f8b700d01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:50.018 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:50.014+0000 7f8b76087700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f8b7015a950 0x7f8b700d01b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50562/0 (socket says 172.21.15.143:50562) 2023-11-08T19:25:50.019 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:50.014+0000 7f8b76087700 1 -- 172.21.15.143:0/3717018556 learned_addr learned my addr 172.21.15.143:0/3717018556 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:50.019 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:50.015+0000 7f8b75886700 1 --2- 172.21.15.143:0/3717018556 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8b7006bd20 0x7f8b70072370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:50.019 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:50.015+0000 7f8b76087700 1 -- 172.21.15.143:0/3717018556 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8b7006bd20 msgr2=0x7f8b70072370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:50.019 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:50.015+0000 7f8b76087700 1 --2- 172.21.15.143:0/3717018556 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8b7006bd20 0x7f8b70072370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:50.019 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:50.015+0000 7f8b76087700 1 -- 172.21.15.143:0/3717018556 >> v1:172.21.15.174:6789/0 conn(0x7f8b700728b0 legacy=0x7f8b700cc730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:50.019 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:50.015+0000 7f8b76087700 1 -- 172.21.15.143:0/3717018556 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b7006c490 con 0x7f8b7015a950 2023-11-08T19:25:50.020 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:50.015+0000 7f8b76087700 1 --2- 172.21.15.143:0/3717018556 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b7015a950 0x7f8b700d01b0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f8b6c0098f0 tx=0x7f8b6c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fc4549d060f2b65 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:50.020 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:50.015+0000 7f8b75085700 1 -- 172.21.15.143:0/3717018556 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b6c011830 con 0x7f8b7015a950 2023-11-08T19:25:50.020 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:50.015+0000 7f8b75085700 1 -- 172.21.15.143:0/3717018556 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b6c011990 con 0x7f8b7015a950 2023-11-08T19:25:50.020 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:50.015+0000 7f8b75085700 1 -- 172.21.15.143:0/3717018556 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b6c011d20 con 0x7f8b7015a950 2023-11-08T19:25:50.020 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:50.016+0000 7f8b782eb700 1 -- 172.21.15.143:0/3717018556 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b7015a950 msgr2=0x7f8b700d01b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:50.021 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:50.016+0000 7f8b782eb700 1 --2- 172.21.15.143:0/3717018556 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b7015a950 0x7f8b700d01b0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f8b6c0098f0 tx=0x7f8b6c009c00 comp rx=0 tx=0).stop 2023-11-08T19:25:50.021 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:50.016+0000 7f8b782eb700 1 -- 172.21.15.143:0/3717018556 shutdown_connections 2023-11-08T19:25:50.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:50.016+0000 7f8b782eb700 1 --2- 172.21.15.143:0/3717018556 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b7015a950 0x7f8b700d01b0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:50.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:50.016+0000 7f8b782eb700 1 --2- 172.21.15.143:0/3717018556 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8b7006bd20 0x7f8b70072370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:50.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:50.016+0000 7f8b782eb700 1 -- 172.21.15.143:0/3717018556 >> 172.21.15.143:0/3717018556 conn(0x7f8b700cd2e0 msgr2=0x7f8b700cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:50.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:50.016+0000 7f8b782eb700 1 -- 172.21.15.143:0/3717018556 shutdown_connections 2023-11-08T19:25:50.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:50.016+0000 7f8b782eb700 1 -- 172.21.15.143:0/3717018556 wait complete. 2023-11-08T19:25:50.025 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:50.017+0000 7f8b782eb700 1 Processor -- start 2023-11-08T19:25:50.025 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:50.017+0000 7f8b782eb700 1 -- start start 2023-11-08T19:25:50.025 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:50.018+0000 7f8b782eb700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8b700728b0 0x7f8b700b5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:50.026 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:50.018+0000 7f8b782eb700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b7015a950 0x7f8b70155820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:50.026 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:50.018+0000 7f8b782eb700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8b701501d0 0x7f8b7015c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:50.026 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:50.018+0000 7f8b782eb700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b700b4be0 con 0x7f8b7015a950 2023-11-08T19:25:50.027 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:50.018+0000 7f8b782eb700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b700ca420 con 0x7f8b701501d0 2023-11-08T19:25:50.027 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:50.018+0000 7f8b782eb700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b700c9e60 con 0x7f8b700728b0 2023-11-08T19:25:50.027 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:50.018+0000 7f8b75886700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b7015a950 0x7f8b70155820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:50.027 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:50.018+0000 7f8b75886700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b7015a950 0x7f8b70155820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50570/0 (socket says 172.21.15.143:50570) 2023-11-08T19:25:50.027 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:50.018+0000 7f8b75886700 1 -- 172.21.15.143:0/2887251384 learned_addr learned my addr 172.21.15.143:0/2887251384 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:50.027 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:50.018+0000 7f8b76087700 1 --2- 172.21.15.143:0/2887251384 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8b700728b0 0x7f8b700b5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:50.028 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:50.018+0000 7f8b76888700 1 --2- 172.21.15.143:0/2887251384 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8b701501d0 0x7f8b7015c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:50.028 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:50.018+0000 7f8b75886700 1 -- 172.21.15.143:0/2887251384 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8b700728b0 msgr2=0x7f8b700b5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:50.028 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:50.018+0000 7f8b75886700 1 --2- 172.21.15.143:0/2887251384 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8b700728b0 0x7f8b700b5070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:50.028 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:50.018+0000 7f8b75886700 1 -- 172.21.15.143:0/2887251384 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8b701501d0 msgr2=0x7f8b7015c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:50.028 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:50.018+0000 7f8b75886700 1 --2- 172.21.15.143:0/2887251384 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8b701501d0 0x7f8b7015c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:50.029 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:50.018+0000 7f8b75886700 1 -- 172.21.15.143:0/2887251384 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b6c009550 con 0x7f8b7015a950 2023-11-08T19:25:50.029 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:50.018+0000 7f8b75886700 1 --2- 172.21.15.143:0/2887251384 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b7015a950 0x7f8b70155820 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f8b6000e9b0 tx=0x7f8b6000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:50.029 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:50.018+0000 7f8b66ffd700 1 -- 172.21.15.143:0/2887251384 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b60009710 con 0x7f8b7015a950 2023-11-08T19:25:50.029 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:50.018+0000 7f8b66ffd700 1 -- 172.21.15.143:0/2887251384 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b6000c850 con 0x7f8b7015a950 2023-11-08T19:25:50.029 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:50.018+0000 7f8b782eb700 1 -- 172.21.15.143:0/2887251384 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b700ca140 con 0x7f8b7015a950 2023-11-08T19:25:50.030 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:50.018+0000 7f8b66ffd700 1 -- 172.21.15.143:0/2887251384 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b6000cbc0 con 0x7f8b7015a950 2023-11-08T19:25:50.030 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:50.018+0000 7f8b782eb700 1 -- 172.21.15.143:0/2887251384 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b70155d60 con 0x7f8b7015a950 2023-11-08T19:25:50.030 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:50.019+0000 7f8b782eb700 1 -- 172.21.15.143:0/2887251384 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b700b46d0 con 0x7f8b7015a950 2023-11-08T19:25:50.030 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:50.020+0000 7f8b66ffd700 1 -- 172.21.15.143:0/2887251384 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8b6000cd20 con 0x7f8b7015a950 2023-11-08T19:25:50.030 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:50.020+0000 7f8b66ffd700 1 --2- 172.21.15.143:0/2887251384 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f8b5406af00 0x7f8b5406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:50.031 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:50.020+0000 7f8b66ffd700 1 -- 172.21.15.143:0/2887251384 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f8b60089650 con 0x7f8b7015a950 2023-11-08T19:25:50.031 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:50.020+0000 7f8b76087700 1 --2- 172.21.15.143:0/2887251384 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f8b5406af00 0x7f8b5406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:50.031 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:50.021+0000 7f8b76087700 1 --2- 172.21.15.143:0/2887251384 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f8b5406af00 0x7f8b5406d3c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f8b6c014040 tx=0x7f8b6c008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:50.032 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:50.025+0000 7f8b66ffd700 1 -- 172.21.15.143:0/2887251384 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8b700b46d0 con 0x7f8b7015a950 2023-11-08T19:25:50.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:50.255+0000 7f8b782eb700 1 -- 172.21.15.143:0/2887251384 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f8b700b46d0 con 0x7f8b7015a950 2023-11-08T19:25:50.881 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:50.878+0000 7f8b66ffd700 1 -- 172.21.15.143:0/2887251384 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v12) v1 ==== 172+0+0 (secure 0 0 0) 0x7f8b700b46d0 con 0x7f8b7015a950 2023-11-08T19:25:50.886 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:50.884+0000 7f8b782eb700 1 -- 172.21.15.143:0/2887251384 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f8b5406af00 msgr2=0x7f8b5406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:50.887 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:50.884+0000 7f8b782eb700 1 --2- 172.21.15.143:0/2887251384 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f8b5406af00 0x7f8b5406d3c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f8b6c014040 tx=0x7f8b6c008030 comp rx=0 tx=0).stop 2023-11-08T19:25:50.887 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:50.884+0000 7f8b782eb700 1 -- 172.21.15.143:0/2887251384 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b7015a950 msgr2=0x7f8b70155820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:50.887 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:50.884+0000 7f8b782eb700 1 --2- 172.21.15.143:0/2887251384 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b7015a950 0x7f8b70155820 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f8b6000e9b0 tx=0x7f8b6000ed70 comp rx=0 tx=0).stop 2023-11-08T19:25:50.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:50.889+0000 7f8b782eb700 1 -- 172.21.15.143:0/2887251384 shutdown_connections 2023-11-08T19:25:50.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:50.889+0000 7f8b782eb700 1 --2- 172.21.15.143:0/2887251384 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f8b5406af00 0x7f8b5406d3c0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:50.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:50.889+0000 7f8b782eb700 1 --2- 172.21.15.143:0/2887251384 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8b700728b0 0x7f8b700b5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:50.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:50.889+0000 7f8b782eb700 1 --2- 172.21.15.143:0/2887251384 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8b701501d0 0x7f8b7015c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:50.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:50.889+0000 7f8b782eb700 1 --2- 172.21.15.143:0/2887251384 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b7015a950 0x7f8b70155820 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:50.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:50.889+0000 7f8b782eb700 1 -- 172.21.15.143:0/2887251384 >> 172.21.15.143:0/2887251384 conn(0x7f8b700cd2e0 msgr2=0x7f8b700cc730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:50.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:50.891+0000 7f8b782eb700 1 -- 172.21.15.143:0/2887251384 shutdown_connections 2023-11-08T19:25:50.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:50.892+0000 7f8b782eb700 1 -- 172.21.15.143:0/2887251384 wait complete. 2023-11-08T19:25:50.895 INFO:teuthology.orchestra.run.smithi143.stderr:new fs with metadata pool 4 and data pool 5 2023-11-08T19:25:50.909 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:25:51.032 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.030+0000 7fd684d4b700 1 Processor -- start 2023-11-08T19:25:51.032 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.031+0000 7fd684d4b700 1 -- start start 2023-11-08T19:25:51.034 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.031+0000 7fd684d4b700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd68006ed60 0x7fd68006b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:51.035 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.031+0000 7fd684d4b700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fd6800cc410 0x7fd6800cc820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:51.035 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.031+0000 7fd684d4b700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6800b2a60 con 0x7fd680159870 2023-11-08T19:25:51.035 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.032+0000 7fd684d4b700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd68006b090 con 0x7fd68006ed60 2023-11-08T19:25:51.035 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.032+0000 7fd684d4b700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6800b2d40 con 0x7fd6800cc410 2023-11-08T19:25:51.036 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.032+0000 7fd67dd9b700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd68006ed60 0x7fd68006b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:51.036 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.032+0000 7fd67dd9b700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd68006ed60 0x7fd68006b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50592/0 (socket says 172.21.15.143:50592) 2023-11-08T19:25:51.036 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.032+0000 7fd67dd9b700 1 -- 172.21.15.143:0/3378900061 learned_addr learned my addr 172.21.15.143:0/3378900061 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:51.036 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.032+0000 7fd67ed9d700 1 --2- 172.21.15.143:0/3378900061 >> v2:172.21.15.174:3300/0 conn(0x7fd6800cc410 0x7fd6800cc820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:51.036 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.032+0000 7fd67dd9b700 1 -- 172.21.15.143:0/3378900061 >> v2:172.21.15.174:3300/0 conn(0x7fd6800cc410 msgr2=0x7fd6800cc820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:51.037 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.032+0000 7fd67dd9b700 1 --2- 172.21.15.143:0/3378900061 >> v2:172.21.15.174:3300/0 conn(0x7fd6800cc410 0x7fd6800cc820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:51.037 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.032+0000 7fd67dd9b700 1 -- 172.21.15.143:0/3378900061 >> v1:172.21.15.174:6789/0 conn(0x7fd680159870 legacy=0x7fd6800b2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:51.037 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.032+0000 7fd67dd9b700 1 -- 172.21.15.143:0/3378900061 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6800b3020 con 0x7fd68006ed60 2023-11-08T19:25:51.037 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.032+0000 7fd67dd9b700 1 --2- 172.21.15.143:0/3378900061 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd68006ed60 0x7fd68006b4b0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fd6800c41c0 tx=0x7fd670009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81a477a522c65538 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:51.037 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.032+0000 7fd67d59a700 1 -- 172.21.15.143:0/3378900061 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd670011830 con 0x7fd68006ed60 2023-11-08T19:25:51.037 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.032+0000 7fd67d59a700 1 -- 172.21.15.143:0/3378900061 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd670011990 con 0x7fd68006ed60 2023-11-08T19:25:51.038 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.032+0000 7fd67d59a700 1 -- 172.21.15.143:0/3378900061 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd670011d20 con 0x7fd68006ed60 2023-11-08T19:25:51.038 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.033+0000 7fd684d4b700 1 -- 172.21.15.143:0/3378900061 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd68006ed60 msgr2=0x7fd68006b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:51.038 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.033+0000 7fd684d4b700 1 --2- 172.21.15.143:0/3378900061 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd68006ed60 0x7fd68006b4b0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fd6800c41c0 tx=0x7fd670009bd0 comp rx=0 tx=0).stop 2023-11-08T19:25:51.038 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.033+0000 7fd684d4b700 1 -- 172.21.15.143:0/3378900061 shutdown_connections 2023-11-08T19:25:51.039 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.033+0000 7fd684d4b700 1 --2- 172.21.15.143:0/3378900061 >> v2:172.21.15.174:3300/0 conn(0x7fd6800cc410 0x7fd6800cc820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:51.039 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.033+0000 7fd684d4b700 1 --2- 172.21.15.143:0/3378900061 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd68006ed60 0x7fd68006b4b0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:51.039 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.033+0000 7fd684d4b700 1 -- 172.21.15.143:0/3378900061 >> 172.21.15.143:0/3378900061 conn(0x7fd680099ab0 msgr2=0x7fd6800d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:51.039 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.033+0000 7fd684d4b700 1 -- 172.21.15.143:0/3378900061 shutdown_connections 2023-11-08T19:25:51.039 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.033+0000 7fd684d4b700 1 -- 172.21.15.143:0/3378900061 wait complete. 2023-11-08T19:25:51.043 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.035+0000 7fd684d4b700 1 Processor -- start 2023-11-08T19:25:51.043 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.035+0000 7fd684d4b700 1 -- start start 2023-11-08T19:25:51.043 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.036+0000 7fd684d4b700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd68006ed60 0x7fd6800b0340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:51.043 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.036+0000 7fd684d4b700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd6800cc410 0x7fd6800b3d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:51.043 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.036+0000 7fd684d4b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd680159870 0x7fd6801568f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:51.044 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.036+0000 7fd684d4b700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6800b6680 con 0x7fd68006ed60 2023-11-08T19:25:51.044 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.036+0000 7fd684d4b700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6800b3660 con 0x7fd680159870 2023-11-08T19:25:51.044 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.036+0000 7fd684d4b700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6800b5de0 con 0x7fd6800cc410 2023-11-08T19:25:51.045 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.036+0000 7fd67e59c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd68006ed60 0x7fd6800b0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:51.045 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.036+0000 7fd67e59c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd68006ed60 0x7fd6800b0340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50596/0 (socket says 172.21.15.143:50596) 2023-11-08T19:25:51.045 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.036+0000 7fd67e59c700 1 -- 172.21.15.143:0/4128843530 learned_addr learned my addr 172.21.15.143:0/4128843530 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:51.045 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.036+0000 7fd67ed9d700 1 --2- 172.21.15.143:0/4128843530 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd680159870 0x7fd6801568f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:51.045 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.036+0000 7fd67dd9b700 1 --2- 172.21.15.143:0/4128843530 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd6800cc410 0x7fd6800b3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:51.046 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.036+0000 7fd67e59c700 1 -- 172.21.15.143:0/4128843530 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd6800cc410 msgr2=0x7fd6800b3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:51.046 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.036+0000 7fd67e59c700 1 --2- 172.21.15.143:0/4128843530 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd6800cc410 0x7fd6800b3d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:51.046 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.036+0000 7fd67e59c700 1 -- 172.21.15.143:0/4128843530 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd680159870 msgr2=0x7fd6801568f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:51.046 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.036+0000 7fd67e59c700 1 --2- 172.21.15.143:0/4128843530 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd680159870 0x7fd6801568f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:51.046 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.036+0000 7fd67e59c700 1 -- 172.21.15.143:0/4128843530 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd670009550 con 0x7fd68006ed60 2023-11-08T19:25:51.048 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.036+0000 7fd67e59c700 1 --2- 172.21.15.143:0/4128843530 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd68006ed60 0x7fd6800b0340 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fd67400c4d0 tx=0x7fd67400c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:51.048 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.036+0000 7fd66effd700 1 -- 172.21.15.143:0/4128843530 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6740052d0 con 0x7fd68006ed60 2023-11-08T19:25:51.048 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.036+0000 7fd66effd700 1 -- 172.21.15.143:0/4128843530 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd674005430 con 0x7fd68006ed60 2023-11-08T19:25:51.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.036+0000 7fd684d4b700 1 -- 172.21.15.143:0/4128843530 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6800b60c0 con 0x7fd68006ed60 2023-11-08T19:25:51.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.036+0000 7fd66effd700 1 -- 172.21.15.143:0/4128843530 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6740038f0 con 0x7fd68006ed60 2023-11-08T19:25:51.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.036+0000 7fd684d4b700 1 -- 172.21.15.143:0/4128843530 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6800b0880 con 0x7fd68006ed60 2023-11-08T19:25:51.050 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.037+0000 7fd684d4b700 1 -- 172.21.15.143:0/4128843530 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6800c8de0 con 0x7fd68006ed60 2023-11-08T19:25:51.050 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.038+0000 7fd66effd700 1 -- 172.21.15.143:0/4128843530 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd674003a50 con 0x7fd68006ed60 2023-11-08T19:25:51.050 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.038+0000 7fd66effd700 1 --2- 172.21.15.143:0/4128843530 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd66006af00 0x7fd66006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:51.050 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.038+0000 7fd66effd700 1 -- 172.21.15.143:0/4128843530 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd6740888c0 con 0x7fd68006ed60 2023-11-08T19:25:51.051 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.038+0000 7fd67dd9b700 1 --2- 172.21.15.143:0/4128843530 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd66006af00 0x7fd66006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:51.051 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.039+0000 7fd67dd9b700 1 --2- 172.21.15.143:0/4128843530 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd66006af00 0x7fd66006d3c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd670014040 tx=0x7fd670008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:51.052 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.043+0000 7fd66effd700 1 -- 172.21.15.143:0/4128843530 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd6800c8de0 con 0x7fd68006ed60 2023-11-08T19:25:51.236 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.233+0000 7fd684d4b700 1 -- 172.21.15.143:0/4128843530 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd6800c8de0 con 0x7fd68006ed60 2023-11-08T19:25:51.237 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.234+0000 7fd66effd700 1 -- 172.21.15.143:0/4128843530 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+16630 (secure 0 0 0) 0x7fd6800c8de0 con 0x7fd68006ed60 2023-11-08T19:25:51.238 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:25:51.238 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":32,"fsid":"968483af-77fc-4068-ad7c-5c2a1a4613d7","created":"2023-11-08T19:24:32.826484+0000","modified":"2023-11-08T19:25:50.256969+0000","last_up_change":"2023-11-08T19:24:43.573880+0000","last_in_change":"2023-11-08T19:24:35.314525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:44.551530+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-08T19:25:47.925761+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-08T19:25:49.224734+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bae98ec1-d94e-403d-a220-a522a53ea6f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6809","nonce":1487830566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6811","nonce":1487830566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6815","nonce":1487830566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6813","nonce":1487830566}]},"public_addr":"172.21.15.143:6809/1487830566","cluster_addr":"172.21.15.143:6811/1487830566","heartbeat_back_addr":"172.21.15.143:6815/1487830566","heartbeat_front_addr":"172.21.15.143:6813/1487830566","state":["exists","up"]},{"osd":1,"uuid":"5cb6538a-1ccb-4322-8633-09f66af0b7e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6801","nonce":787341397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6803","nonce":787341397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6807","nonce":787341397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6805","nonce":787341397}]},"public_addr":"172.21.15.143:6801/787341397","cluster_addr":"172.21.15.143:6803/787341397","heartbeat_back_addr":"172.21.15.143:6807/787341397","heartbeat_front_addr":"172.21.15.143:6805/787341397","state":["exists","up"]},{"osd":2,"uuid":"cdbf522c-60ef-4d60-9653-8dcb529c101a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6822","nonce":2301325597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6826","nonce":2301325597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6831","nonce":2301325597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6829","nonce":2301325597}]},"public_addr":"172.21.15.143:6822/2301325597","cluster_addr":"172.21.15.143:6826/2301325597","heartbeat_back_addr":"172.21.15.143:6831/2301325597","heartbeat_front_addr":"172.21.15.143:6829/2301325597","state":["exists","up"]},{"osd":3,"uuid":"4e2783cf-ebb9-4dd8-8245-bc1d17ce4325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6817","nonce":2021504428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6819","nonce":2021504428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6827","nonce":2021504428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6823","nonce":2021504428}]},"public_addr":"172.21.15.143:6817/2021504428","cluster_addr":"172.21.15.143:6819/2021504428","heartbeat_back_addr":"172.21.15.143:6827/2021504428","heartbeat_front_addr":"172.21.15.143:6823/2021504428","state":["exists","up"]},{"osd":4,"uuid":"691256fa-9b77-49b9-a346-7ae1adf0ca80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6801","nonce":1993380595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6803","nonce":1993380595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6807","nonce":1993380595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6805","nonce":1993380595}]},"public_addr":"172.21.15.174:6801/1993380595","cluster_addr":"172.21.15.174:6803/1993380595","heartbeat_back_addr":"172.21.15.174:6807/1993380595","heartbeat_front_addr":"172.21.15.174:6805/1993380595","state":["exists","up"]},{"osd":5,"uuid":"756fc352-48c9-46ce-b52e-0541fe6f8525","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6809","nonce":1777923303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6811","nonce":1777923303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6815","nonce":1777923303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6813","nonce":1777923303}]},"public_addr":"172.21.15.174:6809/1777923303","cluster_addr":"172.21.15.174:6811/1777923303","heartbeat_back_addr":"172.21.15.174:6815/1777923303","heartbeat_front_addr":"172.21.15.174:6813/1777923303","state":["exists","up"]},{"osd":6,"uuid":"fab2ac2b-ed97-4cde-9825-b480eabf8b8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6825","nonce":2755130316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6827","nonce":2755130316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6831","nonce":2755130316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6829","nonce":2755130316}]},"public_addr":"172.21.15.174:6825/2755130316","cluster_addr":"172.21.15.174:6827/2755130316","heartbeat_back_addr":"172.21.15.174:6831/2755130316","heartbeat_front_addr":"172.21.15.174:6829/2755130316","state":["exists","up"]},{"osd":7,"uuid":"3de9b797-2ab6-4d41-a863-7e7d638fd703","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6817","nonce":2847141411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6819","nonce":2847141411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6823","nonce":2847141411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6821","nonce":2847141411}]},"public_addr":"172.21.15.174:6817/2847141411","cluster_addr":"172.21.15.174:6819/2847141411","heartbeat_back_addr":"172.21.15.174:6823/2847141411","heartbeat_front_addr":"172.21.15.174:6821/2847141411","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.477076+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.496890+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.539305+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.454673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.538881+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.521294+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.520712+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.540404+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:25:51.240 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.236+0000 7fd684d4b700 1 -- 172.21.15.143:0/4128843530 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd66006af00 msgr2=0x7fd66006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:51.240 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.236+0000 7fd684d4b700 1 --2- 172.21.15.143:0/4128843530 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd66006af00 0x7fd66006d3c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd670014040 tx=0x7fd670008030 comp rx=0 tx=0).stop 2023-11-08T19:25:51.240 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.236+0000 7fd684d4b700 1 -- 172.21.15.143:0/4128843530 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd68006ed60 msgr2=0x7fd6800b0340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:51.241 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.236+0000 7fd684d4b700 1 --2- 172.21.15.143:0/4128843530 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd68006ed60 0x7fd6800b0340 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fd67400c4d0 tx=0x7fd67400c7e0 comp rx=0 tx=0).stop 2023-11-08T19:25:51.241 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.237+0000 7fd684d4b700 1 -- 172.21.15.143:0/4128843530 shutdown_connections 2023-11-08T19:25:51.242 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.237+0000 7fd684d4b700 1 --2- 172.21.15.143:0/4128843530 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd66006af00 0x7fd66006d3c0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:51.242 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.237+0000 7fd684d4b700 1 --2- 172.21.15.143:0/4128843530 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd680159870 0x7fd6801568f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:51.242 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.237+0000 7fd684d4b700 1 --2- 172.21.15.143:0/4128843530 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd68006ed60 0x7fd6800b0340 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:51.243 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.237+0000 7fd684d4b700 1 --2- 172.21.15.143:0/4128843530 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd6800cc410 0x7fd6800b3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:51.243 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.237+0000 7fd684d4b700 1 -- 172.21.15.143:0/4128843530 >> 172.21.15.143:0/4128843530 conn(0x7fd680099ab0 msgr2=0x7fd6800add80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:51.243 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.237+0000 7fd684d4b700 1 -- 172.21.15.143:0/4128843530 shutdown_connections 2023-11-08T19:25:51.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.237+0000 7fd684d4b700 1 -- 172.21.15.143:0/4128843530 wait complete. 2023-11-08T19:25:51.250 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:25:51.371 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.369+0000 7f9f2740c700 1 Processor -- start 2023-11-08T19:25:51.371 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.370+0000 7f9f2740c700 1 -- start start 2023-11-08T19:25:51.375 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.370+0000 7f9f2740c700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9f2006b320 0x7f9f20168c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:51.375 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.370+0000 7f9f2740c700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f20161df0 con 0x7f9f201618f0 2023-11-08T19:25:51.375 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.370+0000 7f9f2740c700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f201620d0 con 0x7f9f2015dd00 2023-11-08T19:25:51.376 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.370+0000 7f9f2740c700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f200cad50 con 0x7f9f2006b320 2023-11-08T19:25:51.376 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.370+0000 7f9f249a7700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f9f201618f0 0x7f9f2006ac10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:46760/0 (socket says 172.21.15.143:46760) 2023-11-08T19:25:51.376 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.370+0000 7f9f249a7700 1 -- 172.21.15.143:0/986385524 learned_addr learned my addr 172.21.15.143:0/986385524 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:25:51.376 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.371+0000 7f9f13fff700 1 -- 172.21.15.143:0/986385524 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 995771385 0 0) 0x7f9f20161df0 con 0x7f9f201618f0 2023-11-08T19:25:51.377 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.371+0000 7f9f259a9700 1 --2- 172.21.15.143:0/986385524 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9f2006b320 0x7f9f20168c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:51.377 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.371+0000 7f9f13fff700 1 -- 172.21.15.143:0/986385524 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f0c0035b0 con 0x7f9f201618f0 2023-11-08T19:25:51.377 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.371+0000 7f9f13fff700 1 -- 172.21.15.143:0/986385524 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 58474126 0 0) 0x7f9f0c0035b0 con 0x7f9f201618f0 2023-11-08T19:25:51.377 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.371+0000 7f9f13fff700 1 -- 172.21.15.143:0/986385524 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9f2006b320 msgr2=0x7f9f20168c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:51.377 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.371+0000 7f9f13fff700 1 --2- 172.21.15.143:0/986385524 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9f2006b320 0x7f9f20168c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:51.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.371+0000 7f9f13fff700 1 -- 172.21.15.143:0/986385524 >> v1:172.21.15.174:6789/0 conn(0x7f9f2015dd00 legacy=0x7f9f200cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:51.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.371+0000 7f9f13fff700 1 -- 172.21.15.143:0/986385524 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f201623b0 con 0x7f9f201618f0 2023-11-08T19:25:51.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.371+0000 7f9f13fff700 1 -- 172.21.15.143:0/986385524 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4147756303 0 0) 0x7f9f14001670 con 0x7f9f201618f0 2023-11-08T19:25:51.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.371+0000 7f9f13fff700 1 -- 172.21.15.143:0/986385524 >> v1:172.21.15.143:6789/0 conn(0x7f9f201618f0 legacy=0x7f9f2006ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:51.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.371+0000 7f9f13fff700 1 --2- 172.21.15.143:0/986385524 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9f0c003a30 0x7f9f0c003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:51.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.371+0000 7f9f13fff700 1 --2- 172.21.15.143:0/986385524 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9f2006b320 0x7f9f0c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:51.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.371+0000 7f9f13fff700 1 --2- 172.21.15.143:0/986385524 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9f0c004d90 0x7f9f0c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:51.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.371+0000 7f9f13fff700 1 -- 172.21.15.143:0/986385524 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f0c00bac0 con 0x7f9f0c004d90 2023-11-08T19:25:51.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.371+0000 7f9f13fff700 1 -- 172.21.15.143:0/986385524 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f0c00bc30 con 0x7f9f0c003a30 2023-11-08T19:25:51.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.371+0000 7f9f13fff700 1 -- 172.21.15.143:0/986385524 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f0c00bdf0 con 0x7f9f2006b320 2023-11-08T19:25:51.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.371+0000 7f9f259a9700 1 --2- 172.21.15.143:0/986385524 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9f0c004d90 0x7f9f0c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:51.380 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.371+0000 7f9f251a8700 1 --2- 172.21.15.143:0/986385524 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9f0c003a30 0x7f9f0c003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:51.380 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.371+0000 7f9f249a7700 1 --2- 172.21.15.143:0/986385524 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9f2006b320 0x7f9f0c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:51.380 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.371+0000 7f9f259a9700 1 -- 172.21.15.143:0/986385524 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9f2006b320 msgr2=0x7f9f0c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:51.380 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.371+0000 7f9f259a9700 1 --2- 172.21.15.143:0/986385524 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9f2006b320 0x7f9f0c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:51.380 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.371+0000 7f9f259a9700 1 -- 172.21.15.143:0/986385524 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9f0c003a30 msgr2=0x7f9f0c003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:51.381 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.371+0000 7f9f259a9700 1 --2- 172.21.15.143:0/986385524 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9f0c003a30 0x7f9f0c003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:51.381 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.371+0000 7f9f259a9700 1 -- 172.21.15.143:0/986385524 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f0c0035b0 con 0x7f9f0c004d90 2023-11-08T19:25:51.381 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.372+0000 7f9f259a9700 1 --2- 172.21.15.143:0/986385524 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9f0c004d90 0x7f9f0c005210 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f9f18009f90 tx=0x7f9f0c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e03001465e8ae854 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:51.381 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.372+0000 7f9f13fff700 1 -- 172.21.15.143:0/986385524 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f1800ec30 con 0x7f9f0c004d90 2023-11-08T19:25:51.381 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.372+0000 7f9f13fff700 1 -- 172.21.15.143:0/986385524 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f1800f040 con 0x7f9f0c004d90 2023-11-08T19:25:51.382 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.372+0000 7f9f13fff700 1 -- 172.21.15.143:0/986385524 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f18014480 con 0x7f9f0c004d90 2023-11-08T19:25:51.382 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.372+0000 7f9f2740c700 1 -- 172.21.15.143:0/986385524 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9f0c004d90 msgr2=0x7f9f0c005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:51.382 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.372+0000 7f9f2740c700 1 --2- 172.21.15.143:0/986385524 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9f0c004d90 0x7f9f0c005210 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f9f18009f90 tx=0x7f9f0c005880 comp rx=0 tx=0).stop 2023-11-08T19:25:51.382 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.372+0000 7f9f259a9700 1 -- 172.21.15.143:0/986385524 reap_dead start 2023-11-08T19:25:51.382 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.372+0000 7f9f2740c700 1 -- 172.21.15.143:0/986385524 shutdown_connections 2023-11-08T19:25:51.383 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.372+0000 7f9f2740c700 1 -- 172.21.15.143:0/986385524 >> 172.21.15.143:0/986385524 conn(0x7f9f20099ab0 msgr2=0x7f9f200d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:51.383 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.372+0000 7f9f2740c700 1 -- 172.21.15.143:0/986385524 shutdown_connections 2023-11-08T19:25:51.383 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.372+0000 7f9f2740c700 1 -- 172.21.15.143:0/986385524 wait complete. 2023-11-08T19:25:51.383 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.374+0000 7f9f2740c700 1 Processor -- start 2023-11-08T19:25:51.384 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.374+0000 7f9f2740c700 1 -- start start 2023-11-08T19:25:51.387 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.374+0000 7f9f2740c700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9f0c004d90 0x7f9f20149f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:51.387 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.374+0000 7f9f2740c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9f2015fe10 0x7f9f20160290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:51.387 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.374+0000 7f9f2740c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9f200c5b10 0x7f9f200c5f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:51.387 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.374+0000 7f9f2740c700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f2015ad40 con 0x7f9f200c5b10 2023-11-08T19:25:51.387 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.374+0000 7f9f2740c700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f201607d0 con 0x7f9f2015fe10 2023-11-08T19:25:51.388 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.374+0000 7f9f2740c700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f200c64d0 con 0x7f9f0c004d90 2023-11-08T19:25:51.388 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.374+0000 7f9f259a9700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9f200c5b10 0x7f9f200c5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:51.388 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.374+0000 7f9f259a9700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9f200c5b10 0x7f9f200c5f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50618/0 (socket says 172.21.15.143:50618) 2023-11-08T19:25:51.388 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.374+0000 7f9f259a9700 1 -- 172.21.15.143:0/962569857 learned_addr learned my addr 172.21.15.143:0/962569857 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:51.388 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.375+0000 7f9f249a7700 1 --2- 172.21.15.143:0/962569857 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9f2015fe10 0x7f9f20160290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:51.389 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.375+0000 7f9f251a8700 1 --2- 172.21.15.143:0/962569857 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9f0c004d90 0x7f9f20149f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:51.389 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.375+0000 7f9f259a9700 1 -- 172.21.15.143:0/962569857 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9f0c004d90 msgr2=0x7f9f20149f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:51.389 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.375+0000 7f9f259a9700 1 --2- 172.21.15.143:0/962569857 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9f0c004d90 0x7f9f20149f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:51.389 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.375+0000 7f9f259a9700 1 -- 172.21.15.143:0/962569857 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9f2015fe10 msgr2=0x7f9f20160290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:51.390 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.375+0000 7f9f259a9700 1 --2- 172.21.15.143:0/962569857 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9f2015fe10 0x7f9f20160290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:51.390 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.375+0000 7f9f259a9700 1 -- 172.21.15.143:0/962569857 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f2015e260 con 0x7f9f200c5b10 2023-11-08T19:25:51.390 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.375+0000 7f9f251a8700 1 --2- 172.21.15.143:0/962569857 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9f0c004d90 0x7f9f20149f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:25:51.390 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.375+0000 7f9f259a9700 1 --2- 172.21.15.143:0/962569857 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9f200c5b10 0x7f9f200c5f90 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f9f0c00bc00 tx=0x7f9f18009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:51.390 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.375+0000 7f9f11ffb700 1 -- 172.21.15.143:0/962569857 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f18009410 con 0x7f9f200c5b10 2023-11-08T19:25:51.391 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.375+0000 7f9f11ffb700 1 -- 172.21.15.143:0/962569857 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f18021070 con 0x7f9f200c5b10 2023-11-08T19:25:51.391 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.375+0000 7f9f2740c700 1 -- 172.21.15.143:0/962569857 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f200c6720 con 0x7f9f200c5b10 2023-11-08T19:25:51.391 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.375+0000 7f9f11ffb700 1 -- 172.21.15.143:0/962569857 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f18014d70 con 0x7f9f200c5b10 2023-11-08T19:25:51.391 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.375+0000 7f9f2740c700 1 -- 172.21.15.143:0/962569857 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f200c6ce0 con 0x7f9f200c5b10 2023-11-08T19:25:51.392 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.376+0000 7f9f2740c700 1 -- 172.21.15.143:0/962569857 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f2004eee0 con 0x7f9f200c5b10 2023-11-08T19:25:51.392 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.377+0000 7f9f11ffb700 1 -- 172.21.15.143:0/962569857 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9f1801d420 con 0x7f9f200c5b10 2023-11-08T19:25:51.392 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.377+0000 7f9f11ffb700 1 --2- 172.21.15.143:0/962569857 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f9f0406af00 0x7f9f0406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:51.392 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.377+0000 7f9f11ffb700 1 -- 172.21.15.143:0/962569857 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9f1808a9b0 con 0x7f9f200c5b10 2023-11-08T19:25:51.392 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.377+0000 7f9f251a8700 1 --2- 172.21.15.143:0/962569857 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f9f0406af00 0x7f9f0406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:51.393 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.378+0000 7f9f251a8700 1 --2- 172.21.15.143:0/962569857 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f9f0406af00 0x7f9f0406d3c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c0056a0 tx=0x7f9f1c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:51.393 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.381+0000 7f9f11ffb700 1 -- 172.21.15.143:0/962569857 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9f18055c10 con 0x7f9f200c5b10 2023-11-08T19:25:51.574 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.572+0000 7f9f2740c700 1 -- 172.21.15.143:0/962569857 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9f200b3130 con 0x7f9f200c5b10 2023-11-08T19:25:51.575 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.574+0000 7f9f11ffb700 1 -- 172.21.15.143:0/962569857 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+16630 (secure 0 0 0) 0x7f9f1802b090 con 0x7f9f200c5b10 2023-11-08T19:25:51.575 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:25:51.576 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":32,"fsid":"968483af-77fc-4068-ad7c-5c2a1a4613d7","created":"2023-11-08T19:24:32.826484+0000","modified":"2023-11-08T19:25:50.256969+0000","last_up_change":"2023-11-08T19:24:43.573880+0000","last_in_change":"2023-11-08T19:24:35.314525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:44.551530+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-08T19:25:47.925761+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-08T19:25:49.224734+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bae98ec1-d94e-403d-a220-a522a53ea6f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6809","nonce":1487830566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6811","nonce":1487830566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6815","nonce":1487830566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6813","nonce":1487830566}]},"public_addr":"172.21.15.143:6809/1487830566","cluster_addr":"172.21.15.143:6811/1487830566","heartbeat_back_addr":"172.21.15.143:6815/1487830566","heartbeat_front_addr":"172.21.15.143:6813/1487830566","state":["exists","up"]},{"osd":1,"uuid":"5cb6538a-1ccb-4322-8633-09f66af0b7e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6801","nonce":787341397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6803","nonce":787341397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6807","nonce":787341397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6805","nonce":787341397}]},"public_addr":"172.21.15.143:6801/787341397","cluster_addr":"172.21.15.143:6803/787341397","heartbeat_back_addr":"172.21.15.143:6807/787341397","heartbeat_front_addr":"172.21.15.143:6805/787341397","state":["exists","up"]},{"osd":2,"uuid":"cdbf522c-60ef-4d60-9653-8dcb529c101a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6822","nonce":2301325597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6826","nonce":2301325597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6831","nonce":2301325597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6829","nonce":2301325597}]},"public_addr":"172.21.15.143:6822/2301325597","cluster_addr":"172.21.15.143:6826/2301325597","heartbeat_back_addr":"172.21.15.143:6831/2301325597","heartbeat_front_addr":"172.21.15.143:6829/2301325597","state":["exists","up"]},{"osd":3,"uuid":"4e2783cf-ebb9-4dd8-8245-bc1d17ce4325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6817","nonce":2021504428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6819","nonce":2021504428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6827","nonce":2021504428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6823","nonce":2021504428}]},"public_addr":"172.21.15.143:6817/2021504428","cluster_addr":"172.21.15.143:6819/2021504428","heartbeat_back_addr":"172.21.15.143:6827/2021504428","heartbeat_front_addr":"172.21.15.143:6823/2021504428","state":["exists","up"]},{"osd":4,"uuid":"691256fa-9b77-49b9-a346-7ae1adf0ca80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6801","nonce":1993380595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6803","nonce":1993380595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6807","nonce":1993380595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6805","nonce":1993380595}]},"public_addr":"172.21.15.174:6801/1993380595","cluster_addr":"172.21.15.174:6803/1993380595","heartbeat_back_addr":"172.21.15.174:6807/1993380595","heartbeat_front_addr":"172.21.15.174:6805/1993380595","state":["exists","up"]},{"osd":5,"uuid":"756fc352-48c9-46ce-b52e-0541fe6f8525","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6809","nonce":1777923303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6811","nonce":1777923303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6815","nonce":1777923303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6813","nonce":1777923303}]},"public_addr":"172.21.15.174:6809/1777923303","cluster_addr":"172.21.15.174:6811/1777923303","heartbeat_back_addr":"172.21.15.174:6815/1777923303","heartbeat_front_addr":"172.21.15.174:6813/1777923303","state":["exists","up"]},{"osd":6,"uuid":"fab2ac2b-ed97-4cde-9825-b480eabf8b8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6825","nonce":2755130316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6827","nonce":2755130316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6831","nonce":2755130316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6829","nonce":2755130316}]},"public_addr":"172.21.15.174:6825/2755130316","cluster_addr":"172.21.15.174:6827/2755130316","heartbeat_back_addr":"172.21.15.174:6831/2755130316","heartbeat_front_addr":"172.21.15.174:6829/2755130316","state":["exists","up"]},{"osd":7,"uuid":"3de9b797-2ab6-4d41-a863-7e7d638fd703","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6817","nonce":2847141411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6819","nonce":2847141411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6823","nonce":2847141411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6821","nonce":2847141411}]},"public_addr":"172.21.15.174:6817/2847141411","cluster_addr":"172.21.15.174:6819/2847141411","heartbeat_back_addr":"172.21.15.174:6823/2847141411","heartbeat_front_addr":"172.21.15.174:6821/2847141411","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.477076+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.496890+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.539305+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.454673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.538881+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.521294+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.520712+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.540404+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:25:51.577 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.576+0000 7f9f2740c700 1 -- 172.21.15.143:0/962569857 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f9f0406af00 msgr2=0x7f9f0406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:51.578 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.576+0000 7f9f2740c700 1 --2- 172.21.15.143:0/962569857 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f9f0406af00 0x7f9f0406d3c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c0056a0 tx=0x7f9f1c009040 comp rx=0 tx=0).stop 2023-11-08T19:25:51.578 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.576+0000 7f9f2740c700 1 -- 172.21.15.143:0/962569857 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9f200c5b10 msgr2=0x7f9f200c5f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:51.578 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.576+0000 7f9f2740c700 1 --2- 172.21.15.143:0/962569857 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9f200c5b10 0x7f9f200c5f90 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f9f0c00bc00 tx=0x7f9f18009750 comp rx=0 tx=0).stop 2023-11-08T19:25:51.578 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.576+0000 7f9f2740c700 1 -- 172.21.15.143:0/962569857 shutdown_connections 2023-11-08T19:25:51.579 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.576+0000 7f9f2740c700 1 --2- 172.21.15.143:0/962569857 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f9f0406af00 0x7f9f0406d3c0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:51.579 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.576+0000 7f9f2740c700 1 --2- 172.21.15.143:0/962569857 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9f0c004d90 0x7f9f20149f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:51.579 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.576+0000 7f9f2740c700 1 --2- 172.21.15.143:0/962569857 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9f200c5b10 0x7f9f200c5f90 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:51.579 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.576+0000 7f9f2740c700 1 --2- 172.21.15.143:0/962569857 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9f2015fe10 0x7f9f20160290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:51.579 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.576+0000 7f9f2740c700 1 -- 172.21.15.143:0/962569857 >> 172.21.15.143:0/962569857 conn(0x7f9f20099ab0 msgr2=0x7f9f200ad460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:51.580 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.576+0000 7f9f2740c700 1 -- 172.21.15.143:0/962569857 shutdown_connections 2023-11-08T19:25:51.580 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.576+0000 7f9f2740c700 1 -- 172.21.15.143:0/962569857 wait complete. 2023-11-08T19:25:51.590 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-08T19:25:51.713 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.711+0000 7f3093cd2700 1 Processor -- start 2023-11-08T19:25:51.713 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.712+0000 7f3093cd2700 1 -- start start 2023-11-08T19:25:51.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.712+0000 7f3093cd2700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f308c158610 0x7f308c15b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:51.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.712+0000 7f3093cd2700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f308c072010 0x7f308c06aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:51.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.712+0000 7f3093cd2700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f308c072c40 con 0x7f308c0cc350 2023-11-08T19:25:51.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.712+0000 7f3093cd2700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f308c0cf150 con 0x7f308c158610 2023-11-08T19:25:51.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.712+0000 7f3093cd2700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f308c15be90 con 0x7f308c072010 2023-11-08T19:25:51.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.713+0000 7f3091a6e700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f308c158610 0x7f308c15b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:51.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.713+0000 7f309126d700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f308c0cc350 0x7f308c071900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:46778/0 (socket says 172.21.15.143:46778) 2023-11-08T19:25:51.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.713+0000 7f309126d700 1 -- 172.21.15.143:0/1711414009 learned_addr learned my addr 172.21.15.143:0/1711414009 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:25:51.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.713+0000 7f309226f700 1 --2- 172.21.15.143:0/1711414009 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f308c072010 0x7f308c06aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:51.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.713+0000 7f3091a6e700 1 -- 172.21.15.143:0/1711414009 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f308c072010 msgr2=0x7f308c06aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:51.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.713+0000 7f3091a6e700 1 --2- 172.21.15.143:0/1711414009 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f308c072010 0x7f308c06aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:51.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.713+0000 7f3091a6e700 1 -- 172.21.15.143:0/1711414009 >> v1:172.21.15.143:6789/0 conn(0x7f308c0cc350 legacy=0x7f308c071900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:51.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.713+0000 7f3090a6c700 1 -- 172.21.15.143:0/1711414009 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1387269672 0 0) 0x7f308c072c40 con 0x7f308c0cc350 2023-11-08T19:25:51.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.713+0000 7f3091a6e700 1 -- 172.21.15.143:0/1711414009 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f308c0cf360 con 0x7f308c158610 2023-11-08T19:25:51.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.713+0000 7f3091a6e700 1 --2- 172.21.15.143:0/1711414009 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f308c158610 0x7f308c15b950 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f307c009840 tx=0x7f307c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ff617068aec53ff server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:51.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.713+0000 7f3090a6c700 1 -- 172.21.15.143:0/1711414009 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f307c011830 con 0x7f308c158610 2023-11-08T19:25:51.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.713+0000 7f3090a6c700 1 -- 172.21.15.143:0/1711414009 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f307c011990 con 0x7f308c158610 2023-11-08T19:25:51.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.713+0000 7f3090a6c700 1 -- 172.21.15.143:0/1711414009 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f307c011d20 con 0x7f308c158610 2023-11-08T19:25:51.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.714+0000 7f3093cd2700 1 -- 172.21.15.143:0/1711414009 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f308c158610 msgr2=0x7f308c15b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:51.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.714+0000 7f3093cd2700 1 --2- 172.21.15.143:0/1711414009 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f308c158610 0x7f308c15b950 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f307c009840 tx=0x7f307c009b50 comp rx=0 tx=0).stop 2023-11-08T19:25:51.721 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.714+0000 7f3093cd2700 1 -- 172.21.15.143:0/1711414009 shutdown_connections 2023-11-08T19:25:51.721 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.714+0000 7f3093cd2700 1 --2- 172.21.15.143:0/1711414009 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f308c072010 0x7f308c06aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:51.721 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.714+0000 7f3093cd2700 1 --2- 172.21.15.143:0/1711414009 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f308c158610 0x7f308c15b950 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:51.721 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.714+0000 7f3093cd2700 1 -- 172.21.15.143:0/1711414009 >> 172.21.15.143:0/1711414009 conn(0x7f308c0b1c70 msgr2=0x7f308c0b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:51.721 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.714+0000 7f3093cd2700 1 -- 172.21.15.143:0/1711414009 shutdown_connections 2023-11-08T19:25:51.722 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.714+0000 7f3093cd2700 1 -- 172.21.15.143:0/1711414009 wait complete. 2023-11-08T19:25:51.724 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.715+0000 7f3093cd2700 1 Processor -- start 2023-11-08T19:25:51.725 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.715+0000 7f3093cd2700 1 -- start start 2023-11-08T19:25:51.725 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.716+0000 7f3093cd2700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f308c072010 0x7f308c1e5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:51.725 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.716+0000 7f3093cd2700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f308c0cc350 0x7f308c1e5690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:51.725 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.716+0000 7f3093cd2700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f308c158610 0x7f308c14b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:51.725 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.716+0000 7f3093cd2700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f308c1f6f90 con 0x7f308c158610 2023-11-08T19:25:51.726 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.716+0000 7f3093cd2700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f308c0c9db0 con 0x7f308c0cc350 2023-11-08T19:25:51.726 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.716+0000 7f3093cd2700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f308c1f71a0 con 0x7f308c072010 2023-11-08T19:25:51.726 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.716+0000 7f309226f700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f308c158610 0x7f308c14b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:51.726 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.716+0000 7f309226f700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f308c158610 0x7f308c14b230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50634/0 (socket says 172.21.15.143:50634) 2023-11-08T19:25:51.727 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.716+0000 7f309226f700 1 -- 172.21.15.143:0/3969645374 learned_addr learned my addr 172.21.15.143:0/3969645374 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:51.727 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.716+0000 7f309126d700 1 --2- 172.21.15.143:0/3969645374 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f308c0cc350 0x7f308c1e5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:51.727 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.716+0000 7f3091a6e700 1 --2- 172.21.15.143:0/3969645374 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f308c072010 0x7f308c1e5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:51.727 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.716+0000 7f309226f700 1 -- 172.21.15.143:0/3969645374 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f308c072010 msgr2=0x7f308c1e5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:51.727 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.716+0000 7f309226f700 1 --2- 172.21.15.143:0/3969645374 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f308c072010 0x7f308c1e5150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:51.728 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.716+0000 7f309226f700 1 -- 172.21.15.143:0/3969645374 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f308c0cc350 msgr2=0x7f308c1e5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:51.728 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.716+0000 7f309226f700 1 --2- 172.21.15.143:0/3969645374 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f308c0cc350 0x7f308c1e5690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:51.728 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.716+0000 7f309226f700 1 -- 172.21.15.143:0/3969645374 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f307c0094a0 con 0x7f308c158610 2023-11-08T19:25:51.728 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.716+0000 7f309126d700 1 --2- 172.21.15.143:0/3969645374 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f308c0cc350 0x7f308c1e5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:25:51.728 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.716+0000 7f309226f700 1 --2- 172.21.15.143:0/3969645374 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f308c158610 0x7f308c14b230 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f3084007d80 tx=0x7f308400e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:51.729 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.716+0000 7f30827fc700 1 -- 172.21.15.143:0/3969645374 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f308400ede0 con 0x7f308c158610 2023-11-08T19:25:51.729 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.716+0000 7f30827fc700 1 -- 172.21.15.143:0/3969645374 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3084014420 con 0x7f308c158610 2023-11-08T19:25:51.729 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.716+0000 7f3093cd2700 1 -- 172.21.15.143:0/3969645374 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f308c1f7480 con 0x7f308c158610 2023-11-08T19:25:51.729 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.716+0000 7f30827fc700 1 -- 172.21.15.143:0/3969645374 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30840147d0 con 0x7f308c158610 2023-11-08T19:25:51.729 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.717+0000 7f3093cd2700 1 -- 172.21.15.143:0/3969645374 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f308c1f7c80 con 0x7f308c158610 2023-11-08T19:25:51.730 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.717+0000 7f3093cd2700 1 -- 172.21.15.143:0/3969645374 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f308c1f7e40 con 0x7f308c158610 2023-11-08T19:25:51.730 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.718+0000 7f30827fc700 1 -- 172.21.15.143:0/3969645374 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3084014930 con 0x7f308c158610 2023-11-08T19:25:51.730 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.718+0000 7f30827fc700 1 --2- 172.21.15.143:0/3969645374 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f307006af00 0x7f307006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:51.730 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.718+0000 7f30827fc700 1 -- 172.21.15.143:0/3969645374 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3084089d80 con 0x7f308c158610 2023-11-08T19:25:51.730 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.718+0000 7f3091a6e700 1 --2- 172.21.15.143:0/3969645374 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f307006af00 0x7f307006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:51.731 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.719+0000 7f3091a6e700 1 --2- 172.21.15.143:0/3969645374 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f307006af00 0x7f307006d3c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f308c15abd0 tx=0x7f307c008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:51.731 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.723+0000 7f30827fc700 1 -- 172.21.15.143:0/3969645374 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3084054e30 con 0x7f308c158610 2023-11-08T19:25:51.959 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:51.958+0000 7f3093cd2700 1 -- 172.21.15.143:0/3969645374 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f308c04eee0 con 0x7f308c158610 2023-11-08T19:25:52.846 INFO:tasks.ceph.mds.c.smithi143.stderr:2023-11-08T19:25:52.845+0000 7f1a2ad00700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-08T19:25:52.888 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:52.887+0000 7f30827fc700 1 -- 172.21.15.143:0/3969645374 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v15) v1 ==== 118+0+0 (secure 0 0 0) 0x7f3084058450 con 0x7f308c158610 2023-11-08T19:25:52.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:52.890+0000 7f3093cd2700 1 -- 172.21.15.143:0/3969645374 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f307006af00 msgr2=0x7f307006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:52.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:52.890+0000 7f3093cd2700 1 --2- 172.21.15.143:0/3969645374 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f307006af00 0x7f307006d3c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f308c15abd0 tx=0x7f307c008030 comp rx=0 tx=0).stop 2023-11-08T19:25:52.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:52.890+0000 7f3093cd2700 1 -- 172.21.15.143:0/3969645374 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f308c158610 msgr2=0x7f308c14b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:52.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:52.890+0000 7f3093cd2700 1 --2- 172.21.15.143:0/3969645374 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f308c158610 0x7f308c14b230 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f3084007d80 tx=0x7f308400e3c0 comp rx=0 tx=0).stop 2023-11-08T19:25:52.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:52.890+0000 7f3093cd2700 1 -- 172.21.15.143:0/3969645374 shutdown_connections 2023-11-08T19:25:52.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:52.890+0000 7f3093cd2700 1 --2- 172.21.15.143:0/3969645374 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f307006af00 0x7f307006d3c0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:52.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:52.890+0000 7f3093cd2700 1 --2- 172.21.15.143:0/3969645374 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f308c072010 0x7f308c1e5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:52.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:52.890+0000 7f3093cd2700 1 --2- 172.21.15.143:0/3969645374 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f308c158610 0x7f308c14b230 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:52.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:52.890+0000 7f3093cd2700 1 --2- 172.21.15.143:0/3969645374 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f308c0cc350 0x7f308c1e5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:52.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:52.890+0000 7f3093cd2700 1 -- 172.21.15.143:0/3969645374 >> 172.21.15.143:0/3969645374 conn(0x7f308c0b1c70 msgr2=0x7f308c0caab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:52.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:52.890+0000 7f3093cd2700 1 -- 172.21.15.143:0/3969645374 shutdown_connections 2023-11-08T19:25:52.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:52.890+0000 7f3093cd2700 1 -- 172.21.15.143:0/3969645374 wait complete. 2023-11-08T19:25:52.906 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-08T19:25:52.907 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-08T19:25:53.037 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:53.035+0000 7f61d0620700 1 Processor -- start 2023-11-08T19:25:53.037 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:53.035+0000 7f61d0620700 1 -- start start 2023-11-08T19:25:53.039 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:53.036+0000 7f61d0620700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f61c815bb20 0x7f61c80b2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:53.040 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:53.036+0000 7f61d0620700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f61c80b1730 0x7f61c80d1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:53.040 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:53.036+0000 7f61d0620700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61c80d0420 con 0x7f61c80d1bb0 2023-11-08T19:25:53.040 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:53.036+0000 7f61d0620700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61c80cf600 con 0x7f61c80b1730 2023-11-08T19:25:53.040 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:53.036+0000 7f61d0620700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61c80d2040 con 0x7f61c815bb20 2023-11-08T19:25:53.041 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:53.036+0000 7f61cdbbb700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f61c80b1730 0x7f61c80d1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:53.041 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:53.036+0000 7f61cdbbb700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f61c80b1730 0x7f61c80d1670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50638/0 (socket says 172.21.15.143:50638) 2023-11-08T19:25:53.041 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:53.036+0000 7f61cdbbb700 1 -- 172.21.15.143:0/1741949034 learned_addr learned my addr 172.21.15.143:0/1741949034 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:53.041 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:53.036+0000 7f61cebbd700 1 --1- 172.21.15.143:0/1741949034 >> v1:172.21.15.143:6789/0 conn(0x7f61c80d1bb0 0x7f61c80ce420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:46796/0 (socket says 172.21.15.143:46796) 2023-11-08T19:25:53.041 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:53.037+0000 7f61ce3bc700 1 --2- 172.21.15.143:0/1741949034 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f61c815bb20 0x7f61c80b2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:53.042 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:53.037+0000 7f61cdbbb700 1 -- 172.21.15.143:0/1741949034 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f61c815bb20 msgr2=0x7f61c80b2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:53.042 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:53.037+0000 7f61cdbbb700 1 --2- 172.21.15.143:0/1741949034 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f61c815bb20 0x7f61c80b2670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:53.042 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:53.037+0000 7f61cdbbb700 1 -- 172.21.15.143:0/1741949034 >> v1:172.21.15.143:6789/0 conn(0x7f61c80d1bb0 legacy=0x7f61c80ce420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:53.042 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:53.037+0000 7f61cdbbb700 1 -- 172.21.15.143:0/1741949034 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61c80cf810 con 0x7f61c80b1730 2023-11-08T19:25:53.042 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:53.037+0000 7f61cd3ba700 1 -- 172.21.15.143:0/1741949034 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3253805276 0 0) 0x7f61c80d0420 con 0x7f61c80d1bb0 2023-11-08T19:25:53.043 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:53.037+0000 7f61cdbbb700 1 --2- 172.21.15.143:0/1741949034 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61c80b1730 0x7f61c80d1670 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f61c40098f0 tx=0x7f61c4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80e358f1183d586a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:53.043 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:53.037+0000 7f61cd3ba700 1 -- 172.21.15.143:0/1741949034 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c400b8a0 con 0x7f61c80b1730 2023-11-08T19:25:53.043 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:53.037+0000 7f61cd3ba700 1 -- 172.21.15.143:0/1741949034 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61c400ba00 con 0x7f61c80b1730 2023-11-08T19:25:53.043 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:53.037+0000 7f61cd3ba700 1 -- 172.21.15.143:0/1741949034 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c400bd90 con 0x7f61c80b1730 2023-11-08T19:25:53.043 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:53.037+0000 7f61d0620700 1 -- 172.21.15.143:0/1741949034 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61c80b1730 msgr2=0x7f61c80d1670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:53.044 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:53.037+0000 7f61d0620700 1 --2- 172.21.15.143:0/1741949034 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61c80b1730 0x7f61c80d1670 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f61c40098f0 tx=0x7f61c4009c00 comp rx=0 tx=0).stop 2023-11-08T19:25:53.044 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:53.038+0000 7f61d0620700 1 -- 172.21.15.143:0/1741949034 shutdown_connections 2023-11-08T19:25:53.044 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:53.038+0000 7f61d0620700 1 --2- 172.21.15.143:0/1741949034 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f61c815bb20 0x7f61c80b2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:53.044 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:53.038+0000 7f61d0620700 1 --2- 172.21.15.143:0/1741949034 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61c80b1730 0x7f61c80d1670 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:53.044 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:53.038+0000 7f61d0620700 1 -- 172.21.15.143:0/1741949034 >> 172.21.15.143:0/1741949034 conn(0x7f61c808b000 msgr2=0x7f61c809ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:53.045 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:53.038+0000 7f61d0620700 1 -- 172.21.15.143:0/1741949034 shutdown_connections 2023-11-08T19:25:53.045 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:53.038+0000 7f61d0620700 1 -- 172.21.15.143:0/1741949034 wait complete. 2023-11-08T19:25:53.047 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:53.039+0000 7f61d0620700 1 Processor -- start 2023-11-08T19:25:53.048 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:53.039+0000 7f61d0620700 1 -- start start 2023-11-08T19:25:53.048 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:53.040+0000 7f61d0620700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61c80b1730 0x7f61c80b6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:53.048 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:53.040+0000 7f61d0620700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f61c80d1bb0 0x7f61c8153190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:53.048 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:53.040+0000 7f61d0620700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f61c815bb20 0x7f61c80c58b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:53.048 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:53.040+0000 7f61d0620700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61c80ce4e0 con 0x7f61c80b1730 2023-11-08T19:25:53.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:53.040+0000 7f61d0620700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61c80b5060 con 0x7f61c80d1bb0 2023-11-08T19:25:53.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:53.040+0000 7f61d0620700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61c80b5270 con 0x7f61c815bb20 2023-11-08T19:25:53.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:53.040+0000 7f61ce3bc700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61c80b1730 0x7f61c80b6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:53.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:53.040+0000 7f61ce3bc700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61c80b1730 0x7f61c80b6310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50644/0 (socket says 172.21.15.143:50644) 2023-11-08T19:25:53.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:53.040+0000 7f61ce3bc700 1 -- 172.21.15.143:0/3327021174 learned_addr learned my addr 172.21.15.143:0/3327021174 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:53.050 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:53.040+0000 7f61cdbbb700 1 --2- 172.21.15.143:0/3327021174 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f61c80d1bb0 0x7f61c8153190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:53.050 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:53.040+0000 7f61ce3bc700 1 -- 172.21.15.143:0/3327021174 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f61c815bb20 msgr2=0x7f61c80c58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:53.050 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:53.040+0000 7f61ce3bc700 1 --2- 172.21.15.143:0/3327021174 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f61c815bb20 0x7f61c80c58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:53.050 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:53.040+0000 7f61ce3bc700 1 -- 172.21.15.143:0/3327021174 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f61c80d1bb0 msgr2=0x7f61c8153190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:53.050 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:53.040+0000 7f61ce3bc700 1 --2- 172.21.15.143:0/3327021174 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f61c80d1bb0 0x7f61c8153190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:53.051 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:53.040+0000 7f61ce3bc700 1 -- 172.21.15.143:0/3327021174 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61c4009550 con 0x7f61c80b1730 2023-11-08T19:25:53.051 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:53.041+0000 7f61ce3bc700 1 --2- 172.21.15.143:0/3327021174 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61c80b1730 0x7f61c80b6310 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f61b800c7f0 tx=0x7f61b800cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:53.051 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:53.041+0000 7f61beffd700 1 -- 172.21.15.143:0/3327021174 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b8011800 con 0x7f61c80b1730 2023-11-08T19:25:53.051 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:53.041+0000 7f61beffd700 1 -- 172.21.15.143:0/3327021174 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61b8011960 con 0x7f61c80b1730 2023-11-08T19:25:53.051 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:53.041+0000 7f61beffd700 1 -- 172.21.15.143:0/3327021174 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b8011cf0 con 0x7f61c80b1730 2023-11-08T19:25:53.052 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:53.041+0000 7f61d0620700 1 -- 172.21.15.143:0/3327021174 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61c80b5550 con 0x7f61c80b1730 2023-11-08T19:25:53.052 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:53.041+0000 7f61d0620700 1 -- 172.21.15.143:0/3327021174 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61c80b5b10 con 0x7f61c80b1730 2023-11-08T19:25:53.052 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:53.042+0000 7f61d0620700 1 -- 172.21.15.143:0/3327021174 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61c804eee0 con 0x7f61c80b1730 2023-11-08T19:25:53.052 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:53.042+0000 7f61beffd700 1 -- 172.21.15.143:0/3327021174 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f61b8018360 con 0x7f61c80b1730 2023-11-08T19:25:53.052 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:53.043+0000 7f61beffd700 1 --2- 172.21.15.143:0/3327021174 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f61ac06af00 0x7f61ac06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:53.053 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:53.043+0000 7f61beffd700 1 -- 172.21.15.143:0/3327021174 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f61b8089560 con 0x7f61c80b1730 2023-11-08T19:25:53.053 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:53.043+0000 7f61cdbbb700 1 --2- 172.21.15.143:0/3327021174 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f61ac06af00 0x7f61ac06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:53.053 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:53.043+0000 7f61cdbbb700 1 --2- 172.21.15.143:0/3327021174 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f61ac06af00 0x7f61ac06d3c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f61c80c56a0 tx=0x7f61c4008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:53.053 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:53.047+0000 7f61beffd700 1 -- 172.21.15.143:0/3327021174 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f61b8054610 con 0x7f61c80b1730 2023-11-08T19:25:53.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:53.277+0000 7f61d0620700 1 -- 172.21.15.143:0/3327021174 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f61c80b6850 con 0x7f61c80b1730 2023-11-08T19:25:53.889 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:53.888+0000 7f61beffd700 1 -- 172.21.15.143:0/3327021174 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7f61b8057c30 con 0x7f61c80b1730 2023-11-08T19:25:53.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:53.890+0000 7f61d0620700 1 -- 172.21.15.143:0/3327021174 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f61ac06af00 msgr2=0x7f61ac06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:53.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:53.890+0000 7f61d0620700 1 --2- 172.21.15.143:0/3327021174 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f61ac06af00 0x7f61ac06d3c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f61c80c56a0 tx=0x7f61c4008030 comp rx=0 tx=0).stop 2023-11-08T19:25:53.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:53.890+0000 7f61d0620700 1 -- 172.21.15.143:0/3327021174 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61c80b1730 msgr2=0x7f61c80b6310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:53.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:53.890+0000 7f61d0620700 1 --2- 172.21.15.143:0/3327021174 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61c80b1730 0x7f61c80b6310 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f61b800c7f0 tx=0x7f61b800cbb0 comp rx=0 tx=0).stop 2023-11-08T19:25:53.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:53.890+0000 7f61d0620700 1 -- 172.21.15.143:0/3327021174 shutdown_connections 2023-11-08T19:25:53.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:53.890+0000 7f61d0620700 1 --2- 172.21.15.143:0/3327021174 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f61ac06af00 0x7f61ac06d3c0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:53.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:53.890+0000 7f61d0620700 1 --2- 172.21.15.143:0/3327021174 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f61c815bb20 0x7f61c80c58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:53.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:53.890+0000 7f61d0620700 1 --2- 172.21.15.143:0/3327021174 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61c80b1730 0x7f61c80b6310 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:53.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:53.890+0000 7f61d0620700 1 --2- 172.21.15.143:0/3327021174 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f61c80d1bb0 0x7f61c8153190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:53.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:53.890+0000 7f61d0620700 1 -- 172.21.15.143:0/3327021174 >> 172.21.15.143:0/3327021174 conn(0x7f61c808b000 msgr2=0x7f61c80ae5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:53.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:53.890+0000 7f61d0620700 1 -- 172.21.15.143:0/3327021174 shutdown_connections 2023-11-08T19:25:53.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:53.890+0000 7f61d0620700 1 -- 172.21.15.143:0/3327021174 wait complete. 2023-11-08T19:25:53.904 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-08T19:25:53.905 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:25:54.026 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.025+0000 7fd32ffa3700 1 Processor -- start 2023-11-08T19:25:54.027 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.025+0000 7fd32ffa3700 1 -- start start 2023-11-08T19:25:54.030 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.026+0000 7fd32ffa3700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd328159870 0x7fd3280b2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:54.030 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.026+0000 7fd32ffa3700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3280b2a60 con 0x7fd3280cc410 2023-11-08T19:25:54.030 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.026+0000 7fd32ffa3700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3280b2d40 con 0x7fd32806ed60 2023-11-08T19:25:54.030 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.026+0000 7fd32ffa3700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd32806b090 con 0x7fd328159870 2023-11-08T19:25:54.031 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.026+0000 7fd32e540700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fd3280cc410 0x7fd3280cc820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:46810/0 (socket says 172.21.15.143:46810) 2023-11-08T19:25:54.031 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.026+0000 7fd32e540700 1 -- 172.21.15.143:0/2192636170 learned_addr learned my addr 172.21.15.143:0/2192636170 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:25:54.031 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.026+0000 7fd32dd3f700 1 --2- 172.21.15.143:0/2192636170 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd328159870 0x7fd3280b2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:54.031 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.026+0000 7fd32cd3d700 1 -- 172.21.15.143:0/2192636170 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3426684888 0 0) 0x7fd3280b2a60 con 0x7fd3280cc410 2023-11-08T19:25:54.032 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.026+0000 7fd32cd3d700 1 -- 172.21.15.143:0/2192636170 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd3140035b0 con 0x7fd3280cc410 2023-11-08T19:25:54.034 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.026+0000 7fd32cd3d700 1 -- 172.21.15.143:0/2192636170 <== mon.1 v1:172.21.15.174:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2709811667 0 0) 0x7fd3280b2d40 con 0x7fd32806ed60 2023-11-08T19:25:54.034 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.026+0000 7fd32cd3d700 1 -- 172.21.15.143:0/2192636170 --> v1:172.21.15.174:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd3280b2a60 con 0x7fd32806ed60 2023-11-08T19:25:54.034 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.026+0000 7fd32cd3d700 1 -- 172.21.15.143:0/2192636170 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3356844516 0 0) 0x7fd3140035b0 con 0x7fd3280cc410 2023-11-08T19:25:54.034 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.026+0000 7fd32dd3f700 1 -- 172.21.15.143:0/2192636170 >> v1:172.21.15.174:6789/0 conn(0x7fd32806ed60 legacy=0x7fd32806b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:54.035 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.026+0000 7fd32dd3f700 1 -- 172.21.15.143:0/2192636170 >> v1:172.21.15.143:6789/0 conn(0x7fd3280cc410 legacy=0x7fd3280cc820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:54.035 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.026+0000 7fd32dd3f700 1 -- 172.21.15.143:0/2192636170 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3280b3020 con 0x7fd328159870 2023-11-08T19:25:54.035 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.027+0000 7fd32dd3f700 1 --2- 172.21.15.143:0/2192636170 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd328159870 0x7fd3280b2210 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd324009850 tx=0x7fd324009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc19ab9da24095df server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:54.035 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.027+0000 7fd32cd3d700 1 -- 172.21.15.143:0/2192636170 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3200019f0 con 0x7fd328159870 2023-11-08T19:25:54.036 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.027+0000 7fd32cd3d700 1 -- 172.21.15.143:0/2192636170 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd324005c30 con 0x7fd328159870 2023-11-08T19:25:54.036 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.027+0000 7fd32cd3d700 1 -- 172.21.15.143:0/2192636170 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd324004030 con 0x7fd328159870 2023-11-08T19:25:54.036 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.027+0000 7fd32ffa3700 1 -- 172.21.15.143:0/2192636170 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd328159870 msgr2=0x7fd3280b2210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:54.036 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.027+0000 7fd32ffa3700 1 --2- 172.21.15.143:0/2192636170 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd328159870 0x7fd3280b2210 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd324009850 tx=0x7fd324009b60 comp rx=0 tx=0).stop 2023-11-08T19:25:54.037 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.027+0000 7fd32ffa3700 1 -- 172.21.15.143:0/2192636170 shutdown_connections 2023-11-08T19:25:54.037 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.027+0000 7fd32ffa3700 1 --2- 172.21.15.143:0/2192636170 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd328159870 0x7fd3280b2210 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:54.037 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.027+0000 7fd32ffa3700 1 -- 172.21.15.143:0/2192636170 >> 172.21.15.143:0/2192636170 conn(0x7fd328099ab0 msgr2=0x7fd3280d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:54.037 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.027+0000 7fd32ffa3700 1 -- 172.21.15.143:0/2192636170 shutdown_connections 2023-11-08T19:25:54.037 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.028+0000 7fd32ffa3700 1 -- 172.21.15.143:0/2192636170 wait complete. 2023-11-08T19:25:54.038 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.029+0000 7fd32ffa3700 1 Processor -- start 2023-11-08T19:25:54.038 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.029+0000 7fd32ffa3700 1 -- start start 2023-11-08T19:25:54.040 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.029+0000 7fd32ffa3700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd32806ed60 0x7fd3281589f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:54.041 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.029+0000 7fd32ffa3700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd3280cc410 0x7fd3280cf660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:54.041 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.029+0000 7fd32ffa3700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd328159870 0x7fd3280ae480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:54.041 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.029+0000 7fd32ffa3700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3280d0320 con 0x7fd328159870 2023-11-08T19:25:54.041 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.029+0000 7fd32ffa3700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3280cfba0 con 0x7fd3280cc410 2023-11-08T19:25:54.041 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.029+0000 7fd32ffa3700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3281f32a0 con 0x7fd32806ed60 2023-11-08T19:25:54.042 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.029+0000 7fd32e540700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd328159870 0x7fd3280ae480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:54.042 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.029+0000 7fd32e540700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd328159870 0x7fd3280ae480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50662/0 (socket says 172.21.15.143:50662) 2023-11-08T19:25:54.042 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.029+0000 7fd32e540700 1 -- 172.21.15.143:0/811051113 learned_addr learned my addr 172.21.15.143:0/811051113 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:54.042 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.030+0000 7fd32dd3f700 1 --2- 172.21.15.143:0/811051113 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd32806ed60 0x7fd3281589f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:54.043 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.030+0000 7fd32d53e700 1 --2- 172.21.15.143:0/811051113 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd3280cc410 0x7fd3280cf660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:54.043 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.030+0000 7fd32e540700 1 -- 172.21.15.143:0/811051113 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd32806ed60 msgr2=0x7fd3281589f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:54.043 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.030+0000 7fd32e540700 1 --2- 172.21.15.143:0/811051113 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd32806ed60 0x7fd3281589f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:54.043 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.030+0000 7fd32e540700 1 -- 172.21.15.143:0/811051113 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd3280cc410 msgr2=0x7fd3280cf660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:54.045 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.030+0000 7fd32e540700 1 --2- 172.21.15.143:0/811051113 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd3280cc410 0x7fd3280cf660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:54.045 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.030+0000 7fd32e540700 1 -- 172.21.15.143:0/811051113 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd324009430 con 0x7fd328159870 2023-11-08T19:25:54.045 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.030+0000 7fd32d53e700 1 --2- 172.21.15.143:0/811051113 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd3280cc410 0x7fd3280cf660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:25:54.046 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.030+0000 7fd32dd3f700 1 --2- 172.21.15.143:0/811051113 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd32806ed60 0x7fd3281589f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:25:54.046 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.030+0000 7fd32e540700 1 --2- 172.21.15.143:0/811051113 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd328159870 0x7fd3280ae480 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fd32000cca0 tx=0x7fd32000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:54.046 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.030+0000 7fd31e7fc700 1 -- 172.21.15.143:0/811051113 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd32000cea0 con 0x7fd328159870 2023-11-08T19:25:54.046 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.030+0000 7fd31e7fc700 1 -- 172.21.15.143:0/811051113 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd32000ada0 con 0x7fd328159870 2023-11-08T19:25:54.047 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.030+0000 7fd32ffa3700 1 -- 172.21.15.143:0/811051113 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3281f3550 con 0x7fd328159870 2023-11-08T19:25:54.047 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.030+0000 7fd31e7fc700 1 -- 172.21.15.143:0/811051113 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd320007310 con 0x7fd328159870 2023-11-08T19:25:54.047 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.030+0000 7fd32ffa3700 1 -- 172.21.15.143:0/811051113 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3281f3b10 con 0x7fd328159870 2023-11-08T19:25:54.047 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.031+0000 7fd32ffa3700 1 -- 172.21.15.143:0/811051113 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd32804eee0 con 0x7fd328159870 2023-11-08T19:25:54.047 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.031+0000 7fd31e7fc700 1 -- 172.21.15.143:0/811051113 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd320007470 con 0x7fd328159870 2023-11-08T19:25:54.048 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.032+0000 7fd31e7fc700 1 --2- 172.21.15.143:0/811051113 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd30c06af00 0x7fd30c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:54.048 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.032+0000 7fd31e7fc700 1 -- 172.21.15.143:0/811051113 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd320089500 con 0x7fd328159870 2023-11-08T19:25:54.048 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.032+0000 7fd32dd3f700 1 --2- 172.21.15.143:0/811051113 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd30c06af00 0x7fd30c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:54.048 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.033+0000 7fd32dd3f700 1 --2- 172.21.15.143:0/811051113 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd30c06af00 0x7fd30c06d3c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd3240097a0 tx=0x7fd324008070 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:54.048 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.036+0000 7fd31e7fc700 1 -- 172.21.15.143:0/811051113 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd3200545b0 con 0x7fd328159870 2023-11-08T19:25:54.265 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.264+0000 7fd32ffa3700 1 -- 172.21.15.143:0/811051113 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd3280cfdd0 con 0x7fd328159870 2023-11-08T19:25:54.266 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.265+0000 7fd31e7fc700 1 -- 172.21.15.143:0/811051113 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4925 (secure 0 0 0) 0x7fd320057bd0 con 0x7fd328159870 2023-11-08T19:25:54.267 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:25:54.267 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4918,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6836/3265911434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3265911434},{"type":"v1","addr":"172.21.15.174:6836","nonce":3265911434}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4921,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/2092876318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6835","nonce":2092876318},{"type":"v1","addr":"172.21.15.174:6837","nonce":2092876318}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5106,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6834/4077367240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":4077367240},{"type":"v1","addr":"172.21.15.143:6834","nonce":4077367240}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T19:25:50.257292+0000","modified":"2023-11-08T19:25:53.278532+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5109},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5109":{"gid":5109,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.143:6835/1574070101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":1574070101},{"type":"v1","addr":"172.21.15.143:6835","nonce":1574070101}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-08T19:25:54.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.267+0000 7fd32ffa3700 1 -- 172.21.15.143:0/811051113 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd30c06af00 msgr2=0x7fd30c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:54.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.267+0000 7fd32ffa3700 1 --2- 172.21.15.143:0/811051113 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd30c06af00 0x7fd30c06d3c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd3240097a0 tx=0x7fd324008070 comp rx=0 tx=0).stop 2023-11-08T19:25:54.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.267+0000 7fd32ffa3700 1 -- 172.21.15.143:0/811051113 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd328159870 msgr2=0x7fd3280ae480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:54.270 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.267+0000 7fd32ffa3700 1 --2- 172.21.15.143:0/811051113 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd328159870 0x7fd3280ae480 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fd32000cca0 tx=0x7fd32000a340 comp rx=0 tx=0).stop 2023-11-08T19:25:54.270 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.267+0000 7fd32ffa3700 1 -- 172.21.15.143:0/811051113 shutdown_connections 2023-11-08T19:25:54.270 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.267+0000 7fd32ffa3700 1 --2- 172.21.15.143:0/811051113 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd30c06af00 0x7fd30c06d3c0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:54.270 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.267+0000 7fd32ffa3700 1 --2- 172.21.15.143:0/811051113 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd32806ed60 0x7fd3281589f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:54.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.267+0000 7fd32ffa3700 1 --2- 172.21.15.143:0/811051113 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd328159870 0x7fd3280ae480 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:54.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.267+0000 7fd32ffa3700 1 --2- 172.21.15.143:0/811051113 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd3280cc410 0x7fd3280cf660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:54.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.267+0000 7fd32ffa3700 1 -- 172.21.15.143:0/811051113 >> 172.21.15.143:0/811051113 conn(0x7fd328099ab0 msgr2=0x7fd3280d06f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:54.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.267+0000 7fd32ffa3700 1 -- 172.21.15.143:0/811051113 shutdown_connections 2023-11-08T19:25:54.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.267+0000 7fd32ffa3700 1 -- 172.21.15.143:0/811051113 wait complete. 2023-11-08T19:25:54.272 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 16 2023-11-08T19:25:54.281 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:25:54.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.401+0000 7f536d074700 1 Processor -- start 2023-11-08T19:25:54.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.401+0000 7f536d074700 1 -- start start 2023-11-08T19:25:54.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.401+0000 7f536d074700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f53681618f0 0x7f536806ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:54.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.401+0000 7f536d074700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f536806b150 0x7f53680cb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:54.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.401+0000 7f536d074700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5368161df0 con 0x7f536815dd00 2023-11-08T19:25:54.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.401+0000 7f536d074700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f536806b600 con 0x7f53681618f0 2023-11-08T19:25:54.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.401+0000 7f536d074700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53680cad50 con 0x7f536806b150 2023-11-08T19:25:54.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.402+0000 7f536759e700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f536806b150 0x7f53680cb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:54.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.402+0000 7f536659c700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f53681618f0 0x7f536806ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:54.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.402+0000 7f536759e700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f536806b150 0x7f53680cb1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.143:38308/0 (socket says 172.21.15.143:38308) 2023-11-08T19:25:54.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.402+0000 7f536659c700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f53681618f0 0x7f536806ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.143:34166/0 (socket says 172.21.15.143:34166) 2023-11-08T19:25:54.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.402+0000 7f536759e700 1 -- 172.21.15.143:0/1617750479 learned_addr learned my addr 172.21.15.143:0/1617750479 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:54.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.402+0000 7f5366d9d700 1 --1- >> v1:172.21.15.174:6789/0 conn(0x7f536815dd00 0x7f53680cca70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.174:6789/0 says I am v1:172.21.15.143:57754/0 (socket says 172.21.15.143:57754) 2023-11-08T19:25:54.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.402+0000 7f536759e700 1 -- 172.21.15.143:0/1617750479 >> v2:172.21.15.174:3300/0 conn(0x7f53681618f0 msgr2=0x7f536806ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:54.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.402+0000 7f536759e700 1 --2- 172.21.15.143:0/1617750479 >> v2:172.21.15.174:3300/0 conn(0x7f53681618f0 0x7f536806ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:54.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.402+0000 7f536759e700 1 -- 172.21.15.143:0/1617750479 >> v1:172.21.15.174:6789/0 conn(0x7f536815dd00 legacy=0x7f53680cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:54.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.402+0000 7f536759e700 1 -- 172.21.15.143:0/1617750479 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53681620d0 con 0x7f536806b150 2023-11-08T19:25:54.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.402+0000 7f536759e700 1 --2- 172.21.15.143:0/1617750479 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f536806b150 0x7f53680cb1a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5360009e50 tx=0x7f536000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dbef6ac016d8cf3c server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:54.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.402+0000 7f5365d9b700 1 -- 172.21.15.143:0/1617750479 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f536000e070 con 0x7f536806b150 2023-11-08T19:25:54.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.402+0000 7f5365d9b700 1 -- 172.21.15.143:0/1617750479 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f536000b910 con 0x7f536806b150 2023-11-08T19:25:54.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.402+0000 7f5365d9b700 1 -- 172.21.15.143:0/1617750479 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f536000bcc0 con 0x7f536806b150 2023-11-08T19:25:54.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.403+0000 7f536d074700 1 -- 172.21.15.143:0/1617750479 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f536806b150 msgr2=0x7f53680cb1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:54.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.403+0000 7f536d074700 1 --2- 172.21.15.143:0/1617750479 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f536806b150 0x7f53680cb1a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5360009e50 tx=0x7f536000b330 comp rx=0 tx=0).stop 2023-11-08T19:25:54.409 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.403+0000 7f536d074700 1 -- 172.21.15.143:0/1617750479 shutdown_connections 2023-11-08T19:25:54.409 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.403+0000 7f536d074700 1 --2- 172.21.15.143:0/1617750479 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f536806b150 0x7f53680cb1a0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:54.409 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.403+0000 7f536d074700 1 --2- 172.21.15.143:0/1617750479 >> v2:172.21.15.174:3300/0 conn(0x7f53681618f0 0x7f536806ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:54.409 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.403+0000 7f536d074700 1 -- 172.21.15.143:0/1617750479 >> 172.21.15.143:0/1617750479 conn(0x7f5368099ab0 msgr2=0x7f53680d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:54.409 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.403+0000 7f536d074700 1 -- 172.21.15.143:0/1617750479 shutdown_connections 2023-11-08T19:25:54.410 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.403+0000 7f536d074700 1 -- 172.21.15.143:0/1617750479 wait complete. 2023-11-08T19:25:54.412 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.404+0000 7f536d074700 1 Processor -- start 2023-11-08T19:25:54.412 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.405+0000 7f536d074700 1 -- start start 2023-11-08T19:25:54.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.405+0000 7f536d074700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f536806b150 0x7f53680b33b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:54.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.405+0000 7f536d074700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f536815dd00 0x7f53681f2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:54.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.405+0000 7f536d074700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f53681618f0 0x7f53680c4230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:54.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.405+0000 7f536d074700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53681f3510 con 0x7f536815dd00 2023-11-08T19:25:54.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.405+0000 7f536d074700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53680c5240 con 0x7f536806b150 2023-11-08T19:25:54.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.405+0000 7f536d074700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53680c4c80 con 0x7f53681618f0 2023-11-08T19:25:54.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.405+0000 7f536659c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f536815dd00 0x7f53681f2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:54.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.405+0000 7f536659c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f536815dd00 0x7f53681f2fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50674/0 (socket says 172.21.15.143:50674) 2023-11-08T19:25:54.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.405+0000 7f536659c700 1 -- 172.21.15.143:0/2920386882 learned_addr learned my addr 172.21.15.143:0/2920386882 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:54.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.406+0000 7f536759e700 1 --2- 172.21.15.143:0/2920386882 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f53681618f0 0x7f53680c4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:54.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.406+0000 7f5366d9d700 1 --2- 172.21.15.143:0/2920386882 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f536806b150 0x7f53680b33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:54.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.406+0000 7f536659c700 1 -- 172.21.15.143:0/2920386882 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f53681618f0 msgr2=0x7f53680c4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:54.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.406+0000 7f536659c700 1 --2- 172.21.15.143:0/2920386882 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f53681618f0 0x7f53680c4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:54.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.406+0000 7f536659c700 1 -- 172.21.15.143:0/2920386882 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f536806b150 msgr2=0x7f53680b33b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:54.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.406+0000 7f536659c700 1 --2- 172.21.15.143:0/2920386882 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f536806b150 0x7f53680b33b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:54.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.406+0000 7f536659c700 1 -- 172.21.15.143:0/2920386882 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5360009b30 con 0x7f536815dd00 2023-11-08T19:25:54.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.406+0000 7f536659c700 1 --2- 172.21.15.143:0/2920386882 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f536815dd00 0x7f53681f2fd0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f535c00e870 tx=0x7f535c00eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:54.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.406+0000 7f534f7fe700 1 -- 172.21.15.143:0/2920386882 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f535c009590 con 0x7f536815dd00 2023-11-08T19:25:54.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.406+0000 7f534f7fe700 1 -- 172.21.15.143:0/2920386882 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f535c013680 con 0x7f536815dd00 2023-11-08T19:25:54.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.406+0000 7f536d074700 1 -- 172.21.15.143:0/2920386882 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53680c4f60 con 0x7f536815dd00 2023-11-08T19:25:54.417 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.406+0000 7f534f7fe700 1 -- 172.21.15.143:0/2920386882 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f535c0139f0 con 0x7f536815dd00 2023-11-08T19:25:54.417 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.406+0000 7f536d074700 1 -- 172.21.15.143:0/2920386882 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53680c5ee0 con 0x7f536815dd00 2023-11-08T19:25:54.417 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.407+0000 7f536d074700 1 -- 172.21.15.143:0/2920386882 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53680aff60 con 0x7f536815dd00 2023-11-08T19:25:54.417 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.408+0000 7f534f7fe700 1 -- 172.21.15.143:0/2920386882 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f535c013b50 con 0x7f536815dd00 2023-11-08T19:25:54.417 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.408+0000 7f534f7fe700 1 --2- 172.21.15.143:0/2920386882 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f535406af00 0x7f535406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:54.418 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.408+0000 7f534f7fe700 1 -- 172.21.15.143:0/2920386882 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f535c089b80 con 0x7f536815dd00 2023-11-08T19:25:54.418 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.408+0000 7f5366d9d700 1 --2- 172.21.15.143:0/2920386882 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f535406af00 0x7f535406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:54.418 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.409+0000 7f5366d9d700 1 --2- 172.21.15.143:0/2920386882 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f535406af00 0x7f535406d3c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f5358000fb0 tx=0x7f5358007ec0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:54.418 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.412+0000 7f534f7fe700 1 -- 172.21.15.143:0/2920386882 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f535c054d10 con 0x7f536815dd00 2023-11-08T19:25:54.605 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.604+0000 7f536d074700 1 -- 172.21.15.143:0/2920386882 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f536804eee0 con 0x7f536815dd00 2023-11-08T19:25:54.606 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.605+0000 7f534f7fe700 1 -- 172.21.15.143:0/2920386882 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16617 (secure 0 0 0) 0x7f535c017090 con 0x7f536815dd00 2023-11-08T19:25:54.607 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:25:54.607 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":33,"fsid":"968483af-77fc-4068-ad7c-5c2a1a4613d7","created":"2023-11-08T19:24:32.826484+0000","modified":"2023-11-08T19:25:51.873738+0000","last_up_change":"2023-11-08T19:24:43.573880+0000","last_in_change":"2023-11-08T19:24:35.314525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:44.551530+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-08T19:25:47.925761+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-08T19:25:49.224734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bae98ec1-d94e-403d-a220-a522a53ea6f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6809","nonce":1487830566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6811","nonce":1487830566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6815","nonce":1487830566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6813","nonce":1487830566}]},"public_addr":"172.21.15.143:6809/1487830566","cluster_addr":"172.21.15.143:6811/1487830566","heartbeat_back_addr":"172.21.15.143:6815/1487830566","heartbeat_front_addr":"172.21.15.143:6813/1487830566","state":["exists","up"]},{"osd":1,"uuid":"5cb6538a-1ccb-4322-8633-09f66af0b7e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6801","nonce":787341397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6803","nonce":787341397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6807","nonce":787341397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6805","nonce":787341397}]},"public_addr":"172.21.15.143:6801/787341397","cluster_addr":"172.21.15.143:6803/787341397","heartbeat_back_addr":"172.21.15.143:6807/787341397","heartbeat_front_addr":"172.21.15.143:6805/787341397","state":["exists","up"]},{"osd":2,"uuid":"cdbf522c-60ef-4d60-9653-8dcb529c101a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6822","nonce":2301325597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6826","nonce":2301325597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6831","nonce":2301325597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6829","nonce":2301325597}]},"public_addr":"172.21.15.143:6822/2301325597","cluster_addr":"172.21.15.143:6826/2301325597","heartbeat_back_addr":"172.21.15.143:6831/2301325597","heartbeat_front_addr":"172.21.15.143:6829/2301325597","state":["exists","up"]},{"osd":3,"uuid":"4e2783cf-ebb9-4dd8-8245-bc1d17ce4325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6817","nonce":2021504428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6819","nonce":2021504428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6827","nonce":2021504428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6823","nonce":2021504428}]},"public_addr":"172.21.15.143:6817/2021504428","cluster_addr":"172.21.15.143:6819/2021504428","heartbeat_back_addr":"172.21.15.143:6827/2021504428","heartbeat_front_addr":"172.21.15.143:6823/2021504428","state":["exists","up"]},{"osd":4,"uuid":"691256fa-9b77-49b9-a346-7ae1adf0ca80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6801","nonce":1993380595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6803","nonce":1993380595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6807","nonce":1993380595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6805","nonce":1993380595}]},"public_addr":"172.21.15.174:6801/1993380595","cluster_addr":"172.21.15.174:6803/1993380595","heartbeat_back_addr":"172.21.15.174:6807/1993380595","heartbeat_front_addr":"172.21.15.174:6805/1993380595","state":["exists","up"]},{"osd":5,"uuid":"756fc352-48c9-46ce-b52e-0541fe6f8525","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6809","nonce":1777923303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6811","nonce":1777923303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6815","nonce":1777923303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6813","nonce":1777923303}]},"public_addr":"172.21.15.174:6809/1777923303","cluster_addr":"172.21.15.174:6811/1777923303","heartbeat_back_addr":"172.21.15.174:6815/1777923303","heartbeat_front_addr":"172.21.15.174:6813/1777923303","state":["exists","up"]},{"osd":6,"uuid":"fab2ac2b-ed97-4cde-9825-b480eabf8b8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6825","nonce":2755130316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6827","nonce":2755130316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6831","nonce":2755130316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6829","nonce":2755130316}]},"public_addr":"172.21.15.174:6825/2755130316","cluster_addr":"172.21.15.174:6827/2755130316","heartbeat_back_addr":"172.21.15.174:6831/2755130316","heartbeat_front_addr":"172.21.15.174:6829/2755130316","state":["exists","up"]},{"osd":7,"uuid":"3de9b797-2ab6-4d41-a863-7e7d638fd703","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6817","nonce":2847141411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6819","nonce":2847141411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6823","nonce":2847141411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6821","nonce":2847141411}]},"public_addr":"172.21.15.174:6817/2847141411","cluster_addr":"172.21.15.174:6819/2847141411","heartbeat_back_addr":"172.21.15.174:6823/2847141411","heartbeat_front_addr":"172.21.15.174:6821/2847141411","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.477076+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.496890+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.539305+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.454673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.538881+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.521294+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.520712+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.540404+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:25:54.609 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.607+0000 7f536d074700 1 -- 172.21.15.143:0/2920386882 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f535406af00 msgr2=0x7f535406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:54.609 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.607+0000 7f536d074700 1 --2- 172.21.15.143:0/2920386882 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f535406af00 0x7f535406d3c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f5358000fb0 tx=0x7f5358007ec0 comp rx=0 tx=0).stop 2023-11-08T19:25:54.610 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.607+0000 7f536d074700 1 -- 172.21.15.143:0/2920386882 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f536815dd00 msgr2=0x7f53681f2fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:54.610 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.607+0000 7f536d074700 1 --2- 172.21.15.143:0/2920386882 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f536815dd00 0x7f53681f2fd0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f535c00e870 tx=0x7f535c00eb80 comp rx=0 tx=0).stop 2023-11-08T19:25:54.610 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.607+0000 7f536d074700 1 -- 172.21.15.143:0/2920386882 shutdown_connections 2023-11-08T19:25:54.610 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.607+0000 7f536d074700 1 --2- 172.21.15.143:0/2920386882 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f535406af00 0x7f535406d3c0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:54.610 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.608+0000 7f536d074700 1 --2- 172.21.15.143:0/2920386882 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f53681618f0 0x7f53680c4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:54.611 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.608+0000 7f536d074700 1 --2- 172.21.15.143:0/2920386882 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f536806b150 0x7f53680b33b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:54.611 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.608+0000 7f536d074700 1 --2- 172.21.15.143:0/2920386882 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f536815dd00 0x7f53681f2fd0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:54.611 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.608+0000 7f536d074700 1 -- 172.21.15.143:0/2920386882 >> 172.21.15.143:0/2920386882 conn(0x7f5368099ab0 msgr2=0x7f53680d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:54.611 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.608+0000 7f536d074700 1 -- 172.21.15.143:0/2920386882 shutdown_connections 2023-11-08T19:25:54.611 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.608+0000 7f536d074700 1 -- 172.21.15.143:0/2920386882 wait complete. 2023-11-08T19:25:54.622 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-08T19:25:54.622 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-08T19:25:54.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.741+0000 7f8246ca8700 1 Processor -- start 2023-11-08T19:25:54.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.741+0000 7f8246ca8700 1 -- start start 2023-11-08T19:25:54.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.741+0000 7f8246ca8700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f824015dd00 0x7f82400cca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:54.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.741+0000 7f8246ca8700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f824006b320 0x7f82400cb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:54.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.741+0000 7f8246ca8700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82400ccfb0 con 0x7f82401618f0 2023-11-08T19:25:54.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.742+0000 7f8246ca8700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82400cad50 con 0x7f824006b320 2023-11-08T19:25:54.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.742+0000 7f8246ca8700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8240161df0 con 0x7f824015dd00 2023-11-08T19:25:54.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.742+0000 7f823ffff700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f82401618f0 0x7f824006ac10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:46834/0 (socket says 172.21.15.143:46834) 2023-11-08T19:25:54.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.742+0000 7f8245245700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f824006b320 0x7f82400cb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:54.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.742+0000 7f823ffff700 1 -- 172.21.15.143:0/3517293404 learned_addr learned my addr 172.21.15.143:0/3517293404 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:25:54.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.742+0000 7f8244a44700 1 --2- 172.21.15.143:0/3517293404 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f824015dd00 0x7f82400cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:54.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.742+0000 7f8245245700 1 -- 172.21.15.143:0/3517293404 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f824015dd00 msgr2=0x7f82400cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:54.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.742+0000 7f8245245700 1 --2- 172.21.15.143:0/3517293404 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f824015dd00 0x7f82400cca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:54.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.742+0000 7f8245245700 1 -- 172.21.15.143:0/3517293404 >> v1:172.21.15.143:6789/0 conn(0x7f82401618f0 legacy=0x7f824006ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:54.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.742+0000 7f8245245700 1 -- 172.21.15.143:0/3517293404 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82401620d0 con 0x7f824006b320 2023-11-08T19:25:54.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.742+0000 7f823f7fe700 1 -- 172.21.15.143:0/3517293404 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 347928542 0 0) 0x7f82400ccfb0 con 0x7f82401618f0 2023-11-08T19:25:54.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.742+0000 7f8244a44700 1 --2- 172.21.15.143:0/3517293404 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f824015dd00 0x7f82400cca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:25:54.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.742+0000 7f8245245700 1 --2- 172.21.15.143:0/3517293404 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f824006b320 0x7f82400cb1a0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f8238009d90 tx=0x7f823800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1035b110a920f28c server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:54.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.742+0000 7f823f7fe700 1 -- 172.21.15.143:0/3517293404 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f823800e070 con 0x7f824006b320 2023-11-08T19:25:54.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.742+0000 7f823f7fe700 1 -- 172.21.15.143:0/3517293404 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f823800bbd0 con 0x7f824006b320 2023-11-08T19:25:54.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.742+0000 7f823f7fe700 1 -- 172.21.15.143:0/3517293404 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8238012430 con 0x7f824006b320 2023-11-08T19:25:54.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.743+0000 7f8246ca8700 1 -- 172.21.15.143:0/3517293404 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f824006b320 msgr2=0x7f82400cb1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:54.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.743+0000 7f8246ca8700 1 --2- 172.21.15.143:0/3517293404 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f824006b320 0x7f82400cb1a0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f8238009d90 tx=0x7f823800b330 comp rx=0 tx=0).stop 2023-11-08T19:25:54.750 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.743+0000 7f8246ca8700 1 -- 172.21.15.143:0/3517293404 shutdown_connections 2023-11-08T19:25:54.750 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.743+0000 7f8246ca8700 1 --2- 172.21.15.143:0/3517293404 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f824006b320 0x7f82400cb1a0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:54.750 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.743+0000 7f8246ca8700 1 --2- 172.21.15.143:0/3517293404 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f824015dd00 0x7f82400cca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:54.750 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.743+0000 7f8246ca8700 1 -- 172.21.15.143:0/3517293404 >> 172.21.15.143:0/3517293404 conn(0x7f8240099ab0 msgr2=0x7f82400d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:54.750 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.743+0000 7f8246ca8700 1 -- 172.21.15.143:0/3517293404 shutdown_connections 2023-11-08T19:25:54.751 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.743+0000 7f8246ca8700 1 -- 172.21.15.143:0/3517293404 wait complete. 2023-11-08T19:25:54.751 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.744+0000 7f8246ca8700 1 Processor -- start 2023-11-08T19:25:54.751 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.744+0000 7f8246ca8700 1 -- start start 2023-11-08T19:25:54.753 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.745+0000 7f8246ca8700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f824006b320 0x7f82400af280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:54.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.745+0000 7f8246ca8700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f824015dd00 0x7f8240157650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:54.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.745+0000 7f8246ca8700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f82401618f0 0x7f8240157b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:54.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.745+0000 7f8246ca8700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82400c6240 con 0x7f824006b320 2023-11-08T19:25:54.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.745+0000 7f8246ca8700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82400c55c0 con 0x7f824015dd00 2023-11-08T19:25:54.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.745+0000 7f8246ca8700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82400c58a0 con 0x7f82401618f0 2023-11-08T19:25:54.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.745+0000 7f8244a44700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f824006b320 0x7f82400af280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:54.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.745+0000 7f8244a44700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f824006b320 0x7f82400af280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50686/0 (socket says 172.21.15.143:50686) 2023-11-08T19:25:54.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.745+0000 7f8244a44700 1 -- 172.21.15.143:0/3257881325 learned_addr learned my addr 172.21.15.143:0/3257881325 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:54.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.745+0000 7f8245245700 1 --2- 172.21.15.143:0/3257881325 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f82401618f0 0x7f8240157b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:54.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.745+0000 7f823ffff700 1 --2- 172.21.15.143:0/3257881325 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f824015dd00 0x7f8240157650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:54.756 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.745+0000 7f8244a44700 1 -- 172.21.15.143:0/3257881325 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f82401618f0 msgr2=0x7f8240157b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:54.756 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.745+0000 7f8244a44700 1 --2- 172.21.15.143:0/3257881325 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f82401618f0 0x7f8240157b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:54.756 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.745+0000 7f8244a44700 1 -- 172.21.15.143:0/3257881325 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f824015dd00 msgr2=0x7f8240157650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:54.756 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.745+0000 7f8244a44700 1 --2- 172.21.15.143:0/3257881325 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f824015dd00 0x7f8240157650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:54.757 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.745+0000 7f8244a44700 1 -- 172.21.15.143:0/3257881325 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8238009a70 con 0x7f824006b320 2023-11-08T19:25:54.757 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.745+0000 7f8244a44700 1 --2- 172.21.15.143:0/3257881325 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f824006b320 0x7f82400af280 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f823400c7f0 tx=0x7f823400cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:54.757 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.746+0000 7f823d7fa700 1 -- 172.21.15.143:0/3257881325 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8234005c20 con 0x7f824006b320 2023-11-08T19:25:54.757 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.746+0000 7f823d7fa700 1 -- 172.21.15.143:0/3257881325 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8234005d80 con 0x7f824006b320 2023-11-08T19:25:54.757 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.746+0000 7f8246ca8700 1 -- 172.21.15.143:0/3257881325 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82400c5b80 con 0x7f824006b320 2023-11-08T19:25:54.758 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.746+0000 7f823d7fa700 1 -- 172.21.15.143:0/3257881325 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8234011820 con 0x7f824006b320 2023-11-08T19:25:54.758 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.746+0000 7f8246ca8700 1 -- 172.21.15.143:0/3257881325 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82400af7c0 con 0x7f824006b320 2023-11-08T19:25:54.758 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.747+0000 7f8246ca8700 1 -- 172.21.15.143:0/3257881325 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82400aff60 con 0x7f824006b320 2023-11-08T19:25:54.758 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.747+0000 7f823d7fa700 1 -- 172.21.15.143:0/3257881325 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8234011980 con 0x7f824006b320 2023-11-08T19:25:54.758 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.747+0000 7f823d7fa700 1 --2- 172.21.15.143:0/3257881325 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f822406af00 0x7f822406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:54.759 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.748+0000 7f823d7fa700 1 -- 172.21.15.143:0/3257881325 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f82340896a0 con 0x7f824006b320 2023-11-08T19:25:54.759 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.748+0000 7f823ffff700 1 --2- 172.21.15.143:0/3257881325 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f822406af00 0x7f822406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:54.759 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.748+0000 7f823ffff700 1 --2- 172.21.15.143:0/3257881325 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f822406af00 0x7f822406d3c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f824015c870 tx=0x7f822c007e70 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:54.759 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.752+0000 7f823d7fa700 1 -- 172.21.15.143:0/3257881325 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f82340547e0 con 0x7f824006b320 2023-11-08T19:25:54.941 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.939+0000 7f8246ca8700 1 -- 172.21.15.143:0/3257881325 --> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f82400d16e0 con 0x7f822406af00 2023-11-08T19:25:54.950 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.949+0000 7f823d7fa700 1 -- 172.21.15.143:0/3257881325 <== mgr.4099 v2:172.21.15.174:6832/4037280178 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286848 (secure 0 0 0) 0x7f82400d16e0 con 0x7f822406af00 2023-11-08T19:25:54.953 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:25:54.956 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.952+0000 7f8246ca8700 1 -- 172.21.15.143:0/3257881325 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f822406af00 msgr2=0x7f822406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:54.956 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.952+0000 7f8246ca8700 1 --2- 172.21.15.143:0/3257881325 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f822406af00 0x7f822406d3c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f824015c870 tx=0x7f822c007e70 comp rx=0 tx=0).stop 2023-11-08T19:25:54.956 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.952+0000 7f8246ca8700 1 -- 172.21.15.143:0/3257881325 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f824006b320 msgr2=0x7f82400af280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:54.956 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.952+0000 7f8246ca8700 1 --2- 172.21.15.143:0/3257881325 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f824006b320 0x7f82400af280 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f823400c7f0 tx=0x7f823400cb00 comp rx=0 tx=0).stop 2023-11-08T19:25:54.957 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.952+0000 7f8246ca8700 1 -- 172.21.15.143:0/3257881325 shutdown_connections 2023-11-08T19:25:54.957 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.952+0000 7f8246ca8700 1 --2- 172.21.15.143:0/3257881325 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f822406af00 0x7f822406d3c0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:54.957 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.952+0000 7f8246ca8700 1 --2- 172.21.15.143:0/3257881325 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f82401618f0 0x7f8240157b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:54.957 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.952+0000 7f8246ca8700 1 --2- 172.21.15.143:0/3257881325 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f824006b320 0x7f82400af280 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:54.957 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.952+0000 7f8246ca8700 1 --2- 172.21.15.143:0/3257881325 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f824015dd00 0x7f8240157650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:54.958 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.952+0000 7f8246ca8700 1 -- 172.21.15.143:0/3257881325 >> 172.21.15.143:0/3257881325 conn(0x7f8240099ab0 msgr2=0x7f82400d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:54.958 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.952+0000 7f8246ca8700 1 -- 172.21.15.143:0/3257881325 shutdown_connections 2023-11-08T19:25:54.958 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:54.952+0000 7f8246ca8700 1 -- 172.21.15.143:0/3257881325 wait complete. 2023-11-08T19:25:54.958 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2023-11-08T19:25:54.966 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":61,"stamp":"2023-11-08T19:25:54.525094+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592196,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":200,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":152,"ondisk_log_size":152,"up":162,"acting":162,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":117,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220240,"kb_used_data":4052,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754480,"statfs":{"total":773094113280,"available":772868587520,"internally_reserved":0,"allocated":4149248,"data_stored":2313094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1828,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.669558"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:25:49.854746+0000","last_change":"2023-11-08T19:25:49.854746+0000","last_active":"2023-11-08T19:25:49.854746+0000","last_peered":"2023-11-08T19:25:49.854746+0000","last_clean":"2023-11-08T19:25:49.854746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:25:49.854746+0000","last_undegraded":"2023-11-08T19:25:49.854746+0000","last_fullsized":"2023-11-08T19:25:49.854746+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204169+0000","last_change":"2023-11-08T19:25:50.890421+0000","last_active":"2023-11-08T19:25:52.204169+0000","last_peered":"2023-11-08T19:25:52.204169+0000","last_clean":"2023-11-08T19:25:52.204169+0000","last_became_active":"2023-11-08T19:25:50.890142+0000","last_became_peered":"2023-11-08T19:25:50.890142+0000","last_unstale":"2023-11-08T19:25:52.204169+0000","last_undegraded":"2023-11-08T19:25:52.204169+0000","last_fullsized":"2023-11-08T19:25:52.204169+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:01:43.267493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.205154+0000","last_change":"2023-11-08T19:25:50.875008+0000","last_active":"2023-11-08T19:25:52.205154+0000","last_peered":"2023-11-08T19:25:52.205154+0000","last_clean":"2023-11-08T19:25:52.205154+0000","last_became_active":"2023-11-08T19:25:50.874499+0000","last_became_peered":"2023-11-08T19:25:50.874499+0000","last_unstale":"2023-11-08T19:25:52.205154+0000","last_undegraded":"2023-11-08T19:25:52.205154+0000","last_fullsized":"2023-11-08T19:25:52.205154+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:27:11.209258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.991817+0000","last_change":"2023-11-08T19:25:50.883677+0000","last_active":"2023-11-08T19:25:51.991817+0000","last_peered":"2023-11-08T19:25:51.991817+0000","last_clean":"2023-11-08T19:25:51.991817+0000","last_became_active":"2023-11-08T19:25:50.883141+0000","last_became_peered":"2023-11-08T19:25:50.883141+0000","last_unstale":"2023-11-08T19:25:51.991817+0000","last_undegraded":"2023-11-08T19:25:51.991817+0000","last_fullsized":"2023-11-08T19:25:51.991817+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:56:01.687509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.876619+0000","last_change":"2023-11-08T19:25:50.877229+0000","last_active":"2023-11-08T19:25:51.876619+0000","last_peered":"2023-11-08T19:25:51.876619+0000","last_clean":"2023-11-08T19:25:51.876619+0000","last_became_active":"2023-11-08T19:25:50.876959+0000","last_became_peered":"2023-11-08T19:25:50.876959+0000","last_unstale":"2023-11-08T19:25:51.876619+0000","last_undegraded":"2023-11-08T19:25:51.876619+0000","last_fullsized":"2023-11-08T19:25:51.876619+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:27:09.954811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204135+0000","last_change":"2023-11-08T19:25:50.883530+0000","last_active":"2023-11-08T19:25:52.204135+0000","last_peered":"2023-11-08T19:25:52.204135+0000","last_clean":"2023-11-08T19:25:52.204135+0000","last_became_active":"2023-11-08T19:25:50.883198+0000","last_became_peered":"2023-11-08T19:25:50.883198+0000","last_unstale":"2023-11-08T19:25:52.204135+0000","last_undegraded":"2023-11-08T19:25:52.204135+0000","last_fullsized":"2023-11-08T19:25:52.204135+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:27:11.209258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:25:49.854746+0000","last_change":"2023-11-08T19:25:49.854746+0000","last_active":"2023-11-08T19:25:49.854746+0000","last_peered":"2023-11-08T19:25:49.854746+0000","last_clean":"2023-11-08T19:25:49.854746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:25:49.854746+0000","last_undegraded":"2023-11-08T19:25:49.854746+0000","last_fullsized":"2023-11-08T19:25:49.854746+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:25:49.854746+0000","last_change":"2023-11-08T19:25:49.854746+0000","last_active":"2023-11-08T19:25:49.854746+0000","last_peered":"2023-11-08T19:25:49.854746+0000","last_clean":"2023-11-08T19:25:49.854746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:25:49.854746+0000","last_undegraded":"2023-11-08T19:25:49.854746+0000","last_fullsized":"2023-11-08T19:25:49.854746+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.205585+0000","last_change":"2023-11-08T19:25:50.888816+0000","last_active":"2023-11-08T19:25:52.205585+0000","last_peered":"2023-11-08T19:25:52.205585+0000","last_clean":"2023-11-08T19:25:52.205585+0000","last_became_active":"2023-11-08T19:25:50.888536+0000","last_became_peered":"2023-11-08T19:25:50.888536+0000","last_unstale":"2023-11-08T19:25:52.205585+0000","last_undegraded":"2023-11-08T19:25:52.205585+0000","last_fullsized":"2023-11-08T19:25:52.205585+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:56:01.687509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.991994+0000","last_change":"2023-11-08T19:25:50.878973+0000","last_active":"2023-11-08T19:25:51.991994+0000","last_peered":"2023-11-08T19:25:51.991994+0000","last_clean":"2023-11-08T19:25:51.991994+0000","last_became_active":"2023-11-08T19:25:50.878706+0000","last_became_peered":"2023-11-08T19:25:50.878706+0000","last_unstale":"2023-11-08T19:25:51.991994+0000","last_undegraded":"2023-11-08T19:25:51.991994+0000","last_fullsized":"2023-11-08T19:25:51.991994+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:49:41.475597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:25:49.854746+0000","last_change":"2023-11-08T19:25:49.854746+0000","last_active":"2023-11-08T19:25:49.854746+0000","last_peered":"2023-11-08T19:25:49.854746+0000","last_clean":"2023-11-08T19:25:49.854746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:25:49.854746+0000","last_undegraded":"2023-11-08T19:25:49.854746+0000","last_fullsized":"2023-11-08T19:25:49.854746+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204768+0000","last_change":"2023-11-08T19:25:50.877227+0000","last_active":"2023-11-08T19:25:52.204768+0000","last_peered":"2023-11-08T19:25:52.204768+0000","last_clean":"2023-11-08T19:25:52.204768+0000","last_became_active":"2023-11-08T19:25:50.876876+0000","last_became_peered":"2023-11-08T19:25:50.876876+0000","last_unstale":"2023-11-08T19:25:52.204768+0000","last_undegraded":"2023-11-08T19:25:52.204768+0000","last_fullsized":"2023-11-08T19:25:52.204768+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:05:33.227810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.876804+0000","last_change":"2023-11-08T19:25:50.890044+0000","last_active":"2023-11-08T19:25:51.876804+0000","last_peered":"2023-11-08T19:25:51.876804+0000","last_clean":"2023-11-08T19:25:51.876804+0000","last_became_active":"2023-11-08T19:25:50.889741+0000","last_became_peered":"2023-11-08T19:25:50.889741+0000","last_unstale":"2023-11-08T19:25:51.876804+0000","last_undegraded":"2023-11-08T19:25:51.876804+0000","last_fullsized":"2023-11-08T19:25:51.876804+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:54:55.202302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204255+0000","last_change":"2023-11-08T19:25:50.876264+0000","last_active":"2023-11-08T19:25:52.204255+0000","last_peered":"2023-11-08T19:25:52.204255+0000","last_clean":"2023-11-08T19:25:52.204255+0000","last_became_active":"2023-11-08T19:25:50.875971+0000","last_became_peered":"2023-11-08T19:25:50.875971+0000","last_unstale":"2023-11-08T19:25:52.204255+0000","last_undegraded":"2023-11-08T19:25:52.204255+0000","last_fullsized":"2023-11-08T19:25:52.204255+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:50:30.103716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204868+0000","last_change":"2023-11-08T19:25:50.877018+0000","last_active":"2023-11-08T19:25:52.204868+0000","last_peered":"2023-11-08T19:25:52.204868+0000","last_clean":"2023-11-08T19:25:52.204868+0000","last_became_active":"2023-11-08T19:25:50.876682+0000","last_became_peered":"2023-11-08T19:25:50.876682+0000","last_unstale":"2023-11-08T19:25:52.204868+0000","last_undegraded":"2023-11-08T19:25:52.204868+0000","last_fullsized":"2023-11-08T19:25:52.204868+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:52:20.334760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:25:49.854746+0000","last_change":"2023-11-08T19:25:49.854746+0000","last_active":"2023-11-08T19:25:49.854746+0000","last_peered":"2023-11-08T19:25:49.854746+0000","last_clean":"2023-11-08T19:25:49.854746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:25:49.854746+0000","last_undegraded":"2023-11-08T19:25:49.854746+0000","last_fullsized":"2023-11-08T19:25:49.854746+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:25:49.854746+0000","last_change":"2023-11-08T19:25:49.854746+0000","last_active":"2023-11-08T19:25:49.854746+0000","last_peered":"2023-11-08T19:25:49.854746+0000","last_clean":"2023-11-08T19:25:49.854746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:25:49.854746+0000","last_undegraded":"2023-11-08T19:25:49.854746+0000","last_fullsized":"2023-11-08T19:25:49.854746+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204251+0000","last_change":"2023-11-08T19:25:50.890954+0000","last_active":"2023-11-08T19:25:52.204251+0000","last_peered":"2023-11-08T19:25:52.204251+0000","last_clean":"2023-11-08T19:25:52.204251+0000","last_became_active":"2023-11-08T19:25:50.890695+0000","last_became_peered":"2023-11-08T19:25:50.890695+0000","last_unstale":"2023-11-08T19:25:52.204251+0000","last_undegraded":"2023-11-08T19:25:52.204251+0000","last_fullsized":"2023-11-08T19:25:52.204251+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:44:02.415941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204041+0000","last_change":"2023-11-08T19:25:50.892390+0000","last_active":"2023-11-08T19:25:52.204041+0000","last_peered":"2023-11-08T19:25:52.204041+0000","last_clean":"2023-11-08T19:25:52.204041+0000","last_became_active":"2023-11-08T19:25:50.892131+0000","last_became_peered":"2023-11-08T19:25:50.892131+0000","last_unstale":"2023-11-08T19:25:52.204041+0000","last_undegraded":"2023-11-08T19:25:52.204041+0000","last_fullsized":"2023-11-08T19:25:52.204041+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:25:07.133280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.876861+0000","last_change":"2023-11-08T19:25:50.874502+0000","last_active":"2023-11-08T19:25:51.876861+0000","last_peered":"2023-11-08T19:25:51.876861+0000","last_clean":"2023-11-08T19:25:51.876861+0000","last_became_active":"2023-11-08T19:25:50.874216+0000","last_became_peered":"2023-11-08T19:25:50.874216+0000","last_unstale":"2023-11-08T19:25:51.876861+0000","last_undegraded":"2023-11-08T19:25:51.876861+0000","last_fullsized":"2023-11-08T19:25:51.876861+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:39:23.438674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204719+0000","last_change":"2023-11-08T19:25:50.886322+0000","last_active":"2023-11-08T19:25:52.204719+0000","last_peered":"2023-11-08T19:25:52.204719+0000","last_clean":"2023-11-08T19:25:52.204719+0000","last_became_active":"2023-11-08T19:25:50.886011+0000","last_became_peered":"2023-11-08T19:25:50.886011+0000","last_unstale":"2023-11-08T19:25:52.204719+0000","last_undegraded":"2023-11-08T19:25:52.204719+0000","last_fullsized":"2023-11-08T19:25:52.204719+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:09:05.379033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204755+0000","last_change":"2023-11-08T19:25:50.881922+0000","last_active":"2023-11-08T19:25:52.204755+0000","last_peered":"2023-11-08T19:25:52.204755+0000","last_clean":"2023-11-08T19:25:52.204755+0000","last_became_active":"2023-11-08T19:25:50.880836+0000","last_became_peered":"2023-11-08T19:25:50.880836+0000","last_unstale":"2023-11-08T19:25:52.204755+0000","last_undegraded":"2023-11-08T19:25:52.204755+0000","last_fullsized":"2023-11-08T19:25:52.204755+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:49:45.059924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.989623+0000","last_change":"2023-11-08T19:25:50.886648+0000","last_active":"2023-11-08T19:25:51.989623+0000","last_peered":"2023-11-08T19:25:51.989623+0000","last_clean":"2023-11-08T19:25:51.989623+0000","last_became_active":"2023-11-08T19:25:50.886376+0000","last_became_peered":"2023-11-08T19:25:50.886376+0000","last_unstale":"2023-11-08T19:25:51.989623+0000","last_undegraded":"2023-11-08T19:25:51.989623+0000","last_fullsized":"2023-11-08T19:25:51.989623+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:49:45.059924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204354+0000","last_change":"2023-11-08T19:25:50.875989+0000","last_active":"2023-11-08T19:25:52.204354+0000","last_peered":"2023-11-08T19:25:52.204354+0000","last_clean":"2023-11-08T19:25:52.204354+0000","last_became_active":"2023-11-08T19:25:50.874428+0000","last_became_peered":"2023-11-08T19:25:50.874428+0000","last_unstale":"2023-11-08T19:25:52.204354+0000","last_undegraded":"2023-11-08T19:25:52.204354+0000","last_fullsized":"2023-11-08T19:25:52.204354+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:04:01.492989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.991856+0000","last_change":"2023-11-08T19:25:50.885263+0000","last_active":"2023-11-08T19:25:51.991856+0000","last_peered":"2023-11-08T19:25:51.991856+0000","last_clean":"2023-11-08T19:25:51.991856+0000","last_became_active":"2023-11-08T19:25:50.884856+0000","last_became_peered":"2023-11-08T19:25:50.884856+0000","last_unstale":"2023-11-08T19:25:51.991856+0000","last_undegraded":"2023-11-08T19:25:51.991856+0000","last_fullsized":"2023-11-08T19:25:51.991856+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:17:24.728184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204268+0000","last_change":"2023-11-08T19:25:50.884128+0000","last_active":"2023-11-08T19:25:52.204268+0000","last_peered":"2023-11-08T19:25:52.204268+0000","last_clean":"2023-11-08T19:25:52.204268+0000","last_became_active":"2023-11-08T19:25:50.883848+0000","last_became_peered":"2023-11-08T19:25:50.883848+0000","last_unstale":"2023-11-08T19:25:52.204268+0000","last_undegraded":"2023-11-08T19:25:52.204268+0000","last_fullsized":"2023-11-08T19:25:52.204268+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:32:01.237850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.992064+0000","last_change":"2023-11-08T19:25:50.885482+0000","last_active":"2023-11-08T19:25:51.992064+0000","last_peered":"2023-11-08T19:25:51.992064+0000","last_clean":"2023-11-08T19:25:51.992064+0000","last_became_active":"2023-11-08T19:25:50.884816+0000","last_became_peered":"2023-11-08T19:25:50.884816+0000","last_unstale":"2023-11-08T19:25:51.992064+0000","last_undegraded":"2023-11-08T19:25:51.992064+0000","last_fullsized":"2023-11-08T19:25:51.992064+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:05:33.227810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204941+0000","last_change":"2023-11-08T19:25:50.871479+0000","last_active":"2023-11-08T19:25:52.204941+0000","last_peered":"2023-11-08T19:25:52.204941+0000","last_clean":"2023-11-08T19:25:52.204941+0000","last_became_active":"2023-11-08T19:25:50.871211+0000","last_became_peered":"2023-11-08T19:25:50.871211+0000","last_unstale":"2023-11-08T19:25:52.204941+0000","last_undegraded":"2023-11-08T19:25:52.204941+0000","last_fullsized":"2023-11-08T19:25:52.204941+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:25:16.223226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204982+0000","last_change":"2023-11-08T19:25:50.881545+0000","last_active":"2023-11-08T19:25:52.204982+0000","last_peered":"2023-11-08T19:25:52.204982+0000","last_clean":"2023-11-08T19:25:52.204982+0000","last_became_active":"2023-11-08T19:25:50.881277+0000","last_became_peered":"2023-11-08T19:25:50.881277+0000","last_unstale":"2023-11-08T19:25:52.204982+0000","last_undegraded":"2023-11-08T19:25:52.204982+0000","last_fullsized":"2023-11-08T19:25:52.204982+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:18:57.865706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:25:49.854746+0000","last_change":"2023-11-08T19:25:49.854746+0000","last_active":"2023-11-08T19:25:49.854746+0000","last_peered":"2023-11-08T19:25:49.854746+0000","last_clean":"2023-11-08T19:25:49.854746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:25:49.854746+0000","last_undegraded":"2023-11-08T19:25:49.854746+0000","last_fullsized":"2023-11-08T19:25:49.854746+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:25:49.854746+0000","last_change":"2023-11-08T19:25:49.854746+0000","last_active":"2023-11-08T19:25:49.854746+0000","last_peered":"2023-11-08T19:25:49.854746+0000","last_clean":"2023-11-08T19:25:49.854746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:25:49.854746+0000","last_undegraded":"2023-11-08T19:25:49.854746+0000","last_fullsized":"2023-11-08T19:25:49.854746+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:25:49.854746+0000","last_change":"2023-11-08T19:25:49.854746+0000","last_active":"2023-11-08T19:25:49.854746+0000","last_peered":"2023-11-08T19:25:49.854746+0000","last_clean":"2023-11-08T19:25:49.854746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:25:49.854746+0000","last_undegraded":"2023-11-08T19:25:49.854746+0000","last_fullsized":"2023-11-08T19:25:49.854746+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.989618+0000","last_change":"2023-11-08T19:25:50.889063+0000","last_active":"2023-11-08T19:25:51.989618+0000","last_peered":"2023-11-08T19:25:51.989618+0000","last_clean":"2023-11-08T19:25:51.989618+0000","last_became_active":"2023-11-08T19:25:50.888760+0000","last_became_peered":"2023-11-08T19:25:50.888760+0000","last_unstale":"2023-11-08T19:25:51.989618+0000","last_undegraded":"2023-11-08T19:25:51.989618+0000","last_fullsized":"2023-11-08T19:25:51.989618+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:37:10.264673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-11-08T19:25:48.857285+0000","last_change":"2023-11-08T19:25:48.857285+0000","last_active":"2023-11-08T19:25:48.844481+0000","last_peered":"2023-11-08T19:25:48.844481+0000","last_clean":"2023-11-08T19:25:48.844481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:25:48.857285+0000","last_undegraded":"2023-11-08T19:25:48.857285+0000","last_fullsized":"2023-11-08T19:25:48.857285+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:25:49.854746+0000","last_change":"2023-11-08T19:25:49.854746+0000","last_active":"2023-11-08T19:25:49.854746+0000","last_peered":"2023-11-08T19:25:49.854746+0000","last_clean":"2023-11-08T19:25:49.854746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:25:49.854746+0000","last_undegraded":"2023-11-08T19:25:49.854746+0000","last_fullsized":"2023-11-08T19:25:49.854746+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1e","version":"32'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.989742+0000","last_change":"2023-11-08T19:25:49.879837+0000","last_active":"2023-11-08T19:25:51.989742+0000","last_peered":"2023-11-08T19:25:51.989742+0000","last_clean":"2023-11-08T19:25:51.989742+0000","last_became_active":"2023-11-08T19:25:49.879570+0000","last_became_peered":"2023-11-08T19:25:49.879570+0000","last_unstale":"2023-11-08T19:25:51.989742+0000","last_undegraded":"2023-11-08T19:25:51.989742+0000","last_fullsized":"2023-11-08T19:25:51.989742+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:00:52.169251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.205335+0000","last_change":"2023-11-08T19:25:50.881012+0000","last_active":"2023-11-08T19:25:52.205335+0000","last_peered":"2023-11-08T19:25:52.205335+0000","last_clean":"2023-11-08T19:25:52.205335+0000","last_became_active":"2023-11-08T19:25:50.880752+0000","last_became_peered":"2023-11-08T19:25:50.880752+0000","last_unstale":"2023-11-08T19:25:52.205335+0000","last_undegraded":"2023-11-08T19:25:52.205335+0000","last_fullsized":"2023-11-08T19:25:52.205335+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:33:12.567239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"32'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.203995+0000","last_change":"2023-11-08T19:25:49.886087+0000","last_active":"2023-11-08T19:25:52.203995+0000","last_peered":"2023-11-08T19:25:52.203995+0000","last_clean":"2023-11-08T19:25:52.203995+0000","last_became_active":"2023-11-08T19:25:49.885801+0000","last_became_peered":"2023-11-08T19:25:49.885801+0000","last_unstale":"2023-11-08T19:25:52.203995+0000","last_undegraded":"2023-11-08T19:25:52.203995+0000","last_fullsized":"2023-11-08T19:25:52.203995+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:04:00.482724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.992009+0000","last_change":"2023-11-08T19:25:50.885648+0000","last_active":"2023-11-08T19:25:51.992009+0000","last_peered":"2023-11-08T19:25:51.992009+0000","last_clean":"2023-11-08T19:25:51.992009+0000","last_became_active":"2023-11-08T19:25:50.885179+0000","last_became_peered":"2023-11-08T19:25:50.885179+0000","last_unstale":"2023-11-08T19:25:51.992009+0000","last_undegraded":"2023-11-08T19:25:51.992009+0000","last_fullsized":"2023-11-08T19:25:51.992009+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:52:20.334760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"32'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.876725+0000","last_change":"2023-11-08T19:25:49.883073+0000","last_active":"2023-11-08T19:25:51.876725+0000","last_peered":"2023-11-08T19:25:51.876725+0000","last_clean":"2023-11-08T19:25:51.876725+0000","last_became_active":"2023-11-08T19:25:49.882522+0000","last_became_peered":"2023-11-08T19:25:49.882522+0000","last_unstale":"2023-11-08T19:25:51.876725+0000","last_undegraded":"2023-11-08T19:25:51.876725+0000","last_fullsized":"2023-11-08T19:25:51.876725+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:27:43.830493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:25:49.854746+0000","last_change":"2023-11-08T19:25:49.854746+0000","last_active":"2023-11-08T19:25:49.854746+0000","last_peered":"2023-11-08T19:25:49.854746+0000","last_clean":"2023-11-08T19:25:49.854746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:25:49.854746+0000","last_undegraded":"2023-11-08T19:25:49.854746+0000","last_fullsized":"2023-11-08T19:25:49.854746+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1b","version":"32'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.205020+0000","last_change":"2023-11-08T19:25:49.884628+0000","last_active":"2023-11-08T19:25:52.205020+0000","last_peered":"2023-11-08T19:25:52.205020+0000","last_clean":"2023-11-08T19:25:52.205020+0000","last_became_active":"2023-11-08T19:25:49.884384+0000","last_became_peered":"2023-11-08T19:25:49.884384+0000","last_unstale":"2023-11-08T19:25:52.205020+0000","last_undegraded":"2023-11-08T19:25:52.205020+0000","last_fullsized":"2023-11-08T19:25:52.205020+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:17:52.641253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204834+0000","last_change":"2023-11-08T19:25:50.874965+0000","last_active":"2023-11-08T19:25:52.204834+0000","last_peered":"2023-11-08T19:25:52.204834+0000","last_clean":"2023-11-08T19:25:52.204834+0000","last_became_active":"2023-11-08T19:25:50.874696+0000","last_became_peered":"2023-11-08T19:25:50.874696+0000","last_unstale":"2023-11-08T19:25:52.204834+0000","last_undegraded":"2023-11-08T19:25:52.204834+0000","last_fullsized":"2023-11-08T19:25:52.204834+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:49:41.475597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"32'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204932+0000","last_change":"2023-11-08T19:25:49.884607+0000","last_active":"2023-11-08T19:25:52.204932+0000","last_peered":"2023-11-08T19:25:52.204932+0000","last_clean":"2023-11-08T19:25:52.204932+0000","last_became_active":"2023-11-08T19:25:49.884392+0000","last_became_peered":"2023-11-08T19:25:49.884392+0000","last_unstale":"2023-11-08T19:25:52.204932+0000","last_undegraded":"2023-11-08T19:25:52.204932+0000","last_fullsized":"2023-11-08T19:25:52.204932+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:54:54.192037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204370+0000","last_change":"2023-11-08T19:25:50.881747+0000","last_active":"2023-11-08T19:25:52.204370+0000","last_peered":"2023-11-08T19:25:52.204370+0000","last_clean":"2023-11-08T19:25:52.204370+0000","last_became_active":"2023-11-08T19:25:50.881225+0000","last_became_peered":"2023-11-08T19:25:50.881225+0000","last_unstale":"2023-11-08T19:25:52.204370+0000","last_undegraded":"2023-11-08T19:25:52.204370+0000","last_fullsized":"2023-11-08T19:25:52.204370+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:37:10.264673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.203749+0000","last_change":"2023-11-08T19:25:49.882231+0000","last_active":"2023-11-08T19:25:52.203749+0000","last_peered":"2023-11-08T19:25:52.203749+0000","last_clean":"2023-11-08T19:25:52.203749+0000","last_became_active":"2023-11-08T19:25:49.881992+0000","last_became_peered":"2023-11-08T19:25:49.881992+0000","last_unstale":"2023-11-08T19:25:52.203749+0000","last_undegraded":"2023-11-08T19:25:52.203749+0000","last_fullsized":"2023-11-08T19:25:52.203749+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:00:52.169251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.991649+0000","last_change":"2023-11-08T19:25:50.883482+0000","last_active":"2023-11-08T19:25:51.991649+0000","last_peered":"2023-11-08T19:25:51.991649+0000","last_clean":"2023-11-08T19:25:51.991649+0000","last_became_active":"2023-11-08T19:25:50.883187+0000","last_became_peered":"2023-11-08T19:25:50.883187+0000","last_unstale":"2023-11-08T19:25:51.991649+0000","last_undegraded":"2023-11-08T19:25:51.991649+0000","last_fullsized":"2023-11-08T19:25:51.991649+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:33:12.567239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204890+0000","last_change":"2023-11-08T19:25:49.878388+0000","last_active":"2023-11-08T19:25:52.204890+0000","last_peered":"2023-11-08T19:25:52.204890+0000","last_clean":"2023-11-08T19:25:52.204890+0000","last_became_active":"2023-11-08T19:25:49.878111+0000","last_became_peered":"2023-11-08T19:25:49.878111+0000","last_unstale":"2023-11-08T19:25:52.204890+0000","last_undegraded":"2023-11-08T19:25:52.204890+0000","last_fullsized":"2023-11-08T19:25:52.204890+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:39:22.428409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204894+0000","last_change":"2023-11-08T19:25:50.877613+0000","last_active":"2023-11-08T19:25:52.204894+0000","last_peered":"2023-11-08T19:25:52.204894+0000","last_clean":"2023-11-08T19:25:52.204894+0000","last_became_active":"2023-11-08T19:25:50.877311+0000","last_became_peered":"2023-11-08T19:25:50.877311+0000","last_unstale":"2023-11-08T19:25:52.204894+0000","last_undegraded":"2023-11-08T19:25:52.204894+0000","last_fullsized":"2023-11-08T19:25:52.204894+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:50:52.902620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204190+0000","last_change":"2023-11-08T19:25:49.884355+0000","last_active":"2023-11-08T19:25:52.204190+0000","last_peered":"2023-11-08T19:25:52.204190+0000","last_clean":"2023-11-08T19:25:52.204190+0000","last_became_active":"2023-11-08T19:25:49.884065+0000","last_became_peered":"2023-11-08T19:25:49.884065+0000","last_unstale":"2023-11-08T19:25:52.204190+0000","last_undegraded":"2023-11-08T19:25:52.204190+0000","last_fullsized":"2023-11-08T19:25:52.204190+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:05:32.217545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204103+0000","last_change":"2023-11-08T19:25:50.888556+0000","last_active":"2023-11-08T19:25:52.204103+0000","last_peered":"2023-11-08T19:25:52.204103+0000","last_clean":"2023-11-08T19:25:52.204103+0000","last_became_active":"2023-11-08T19:25:50.887909+0000","last_became_peered":"2023-11-08T19:25:50.887909+0000","last_unstale":"2023-11-08T19:25:52.204103+0000","last_undegraded":"2023-11-08T19:25:52.204103+0000","last_fullsized":"2023-11-08T19:25:52.204103+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:38:37.341013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.989663+0000","last_change":"2023-11-08T19:25:49.884919+0000","last_active":"2023-11-08T19:25:51.989663+0000","last_peered":"2023-11-08T19:25:51.989663+0000","last_clean":"2023-11-08T19:25:51.989663+0000","last_became_active":"2023-11-08T19:25:49.884679+0000","last_became_peered":"2023-11-08T19:25:49.884679+0000","last_unstale":"2023-11-08T19:25:51.989663+0000","last_undegraded":"2023-11-08T19:25:51.989663+0000","last_fullsized":"2023-11-08T19:25:51.989663+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:39:22.428409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204236+0000","last_change":"2023-11-08T19:25:50.882028+0000","last_active":"2023-11-08T19:25:52.204236+0000","last_peered":"2023-11-08T19:25:52.204236+0000","last_clean":"2023-11-08T19:25:52.204236+0000","last_became_active":"2023-11-08T19:25:50.881746+0000","last_became_peered":"2023-11-08T19:25:50.881746+0000","last_unstale":"2023-11-08T19:25:52.204236+0000","last_undegraded":"2023-11-08T19:25:52.204236+0000","last_fullsized":"2023-11-08T19:25:52.204236+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:39:57.441098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"32'3","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204967+0000","last_change":"2023-11-08T19:25:49.878409+0000","last_active":"2023-11-08T19:25:52.204967+0000","last_peered":"2023-11-08T19:25:52.204967+0000","last_clean":"2023-11-08T19:25:52.204967+0000","last_became_active":"2023-11-08T19:25:49.878105+0000","last_became_peered":"2023-11-08T19:25:49.878105+0000","last_unstale":"2023-11-08T19:25:52.204967+0000","last_undegraded":"2023-11-08T19:25:52.204967+0000","last_fullsized":"2023-11-08T19:25:52.204967+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:55:30.168605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204866+0000","last_change":"2023-11-08T19:25:50.887174+0000","last_active":"2023-11-08T19:25:52.204866+0000","last_peered":"2023-11-08T19:25:52.204866+0000","last_clean":"2023-11-08T19:25:52.204866+0000","last_became_active":"2023-11-08T19:25:50.886902+0000","last_became_peered":"2023-11-08T19:25:50.886902+0000","last_unstale":"2023-11-08T19:25:52.204866+0000","last_undegraded":"2023-11-08T19:25:52.204866+0000","last_fullsized":"2023-11-08T19:25:52.204866+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:19:42.562982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"32'6","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204091+0000","last_change":"2023-11-08T19:25:49.886499+0000","last_active":"2023-11-08T19:25:52.204091+0000","last_peered":"2023-11-08T19:25:52.204091+0000","last_clean":"2023-11-08T19:25:52.204091+0000","last_became_active":"2023-11-08T19:25:49.886169+0000","last_became_peered":"2023-11-08T19:25:49.886169+0000","last_unstale":"2023-11-08T19:25:52.204091+0000","last_undegraded":"2023-11-08T19:25:52.204091+0000","last_fullsized":"2023-11-08T19:25:52.204091+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:19:41.552717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.991670+0000","last_change":"2023-11-08T19:25:50.885893+0000","last_active":"2023-11-08T19:25:51.991670+0000","last_peered":"2023-11-08T19:25:51.991670+0000","last_clean":"2023-11-08T19:25:51.991670+0000","last_became_active":"2023-11-08T19:25:50.880205+0000","last_became_peered":"2023-11-08T19:25:50.880205+0000","last_unstale":"2023-11-08T19:25:51.991670+0000","last_undegraded":"2023-11-08T19:25:51.991670+0000","last_fullsized":"2023-11-08T19:25:51.991670+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:55:31.178870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"32'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.205247+0000","last_change":"2023-11-08T19:25:49.879256+0000","last_active":"2023-11-08T19:25:52.205247+0000","last_peered":"2023-11-08T19:25:52.205247+0000","last_clean":"2023-11-08T19:25:52.205247+0000","last_became_active":"2023-11-08T19:25:49.878963+0000","last_became_peered":"2023-11-08T19:25:49.878963+0000","last_unstale":"2023-11-08T19:25:52.205247+0000","last_undegraded":"2023-11-08T19:25:52.205247+0000","last_fullsized":"2023-11-08T19:25:52.205247+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:12:39.742779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.203953+0000","last_change":"2023-11-08T19:25:50.891183+0000","last_active":"2023-11-08T19:25:52.203953+0000","last_peered":"2023-11-08T19:25:52.203953+0000","last_clean":"2023-11-08T19:25:52.203953+0000","last_became_active":"2023-11-08T19:25:50.890889+0000","last_became_peered":"2023-11-08T19:25:50.890889+0000","last_unstale":"2023-11-08T19:25:52.203953+0000","last_undegraded":"2023-11-08T19:25:52.203953+0000","last_fullsized":"2023-11-08T19:25:52.203953+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:25:16.223226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.991756+0000","last_change":"2023-11-08T19:25:49.880102+0000","last_active":"2023-11-08T19:25:51.991756+0000","last_peered":"2023-11-08T19:25:51.991756+0000","last_clean":"2023-11-08T19:25:51.991756+0000","last_became_active":"2023-11-08T19:25:49.879816+0000","last_became_peered":"2023-11-08T19:25:49.879816+0000","last_unstale":"2023-11-08T19:25:51.991756+0000","last_undegraded":"2023-11-08T19:25:51.991756+0000","last_fullsized":"2023-11-08T19:25:51.991756+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:37:09.254408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:25:49.854746+0000","last_change":"2023-11-08T19:25:49.854746+0000","last_active":"2023-11-08T19:25:49.854746+0000","last_peered":"2023-11-08T19:25:49.854746+0000","last_clean":"2023-11-08T19:25:49.854746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:25:49.854746+0000","last_undegraded":"2023-11-08T19:25:49.854746+0000","last_fullsized":"2023-11-08T19:25:49.854746+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.203841+0000","last_change":"2023-11-08T19:25:49.884359+0000","last_active":"2023-11-08T19:25:52.203841+0000","last_peered":"2023-11-08T19:25:52.203841+0000","last_clean":"2023-11-08T19:25:52.203841+0000","last_became_active":"2023-11-08T19:25:49.884119+0000","last_became_peered":"2023-11-08T19:25:49.884119+0000","last_unstale":"2023-11-08T19:25:52.203841+0000","last_undegraded":"2023-11-08T19:25:52.203841+0000","last_fullsized":"2023-11-08T19:25:52.203841+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:55:30.168605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:25:49.854746+0000","last_change":"2023-11-08T19:25:49.854746+0000","last_active":"2023-11-08T19:25:49.854746+0000","last_peered":"2023-11-08T19:25:49.854746+0000","last_clean":"2023-11-08T19:25:49.854746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:25:49.854746+0000","last_undegraded":"2023-11-08T19:25:49.854746+0000","last_fullsized":"2023-11-08T19:25:49.854746+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.10","version":"32'8","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.203707+0000","last_change":"2023-11-08T19:25:49.886483+0000","last_active":"2023-11-08T19:25:52.203707+0000","last_peered":"2023-11-08T19:25:52.203707+0000","last_clean":"2023-11-08T19:25:52.203707+0000","last_became_active":"2023-11-08T19:25:49.886209+0000","last_became_peered":"2023-11-08T19:25:49.886209+0000","last_unstale":"2023-11-08T19:25:52.203707+0000","last_undegraded":"2023-11-08T19:25:52.203707+0000","last_fullsized":"2023-11-08T19:25:52.203707+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:49:40.465332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.205492+0000","last_change":"2023-11-08T19:25:50.884739+0000","last_active":"2023-11-08T19:25:52.205492+0000","last_peered":"2023-11-08T19:25:52.205492+0000","last_clean":"2023-11-08T19:25:52.205492+0000","last_became_active":"2023-11-08T19:25:50.884476+0000","last_became_peered":"2023-11-08T19:25:50.884476+0000","last_unstale":"2023-11-08T19:25:52.205492+0000","last_undegraded":"2023-11-08T19:25:52.205492+0000","last_fullsized":"2023-11-08T19:25:52.205492+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:55:31.178870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"32'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.992046+0000","last_change":"2023-11-08T19:25:49.884770+0000","last_active":"2023-11-08T19:25:51.992046+0000","last_peered":"2023-11-08T19:25:51.992046+0000","last_clean":"2023-11-08T19:25:51.992046+0000","last_became_active":"2023-11-08T19:25:49.884476+0000","last_became_peered":"2023-11-08T19:25:49.884476+0000","last_unstale":"2023-11-08T19:25:51.992046+0000","last_undegraded":"2023-11-08T19:25:51.992046+0000","last_fullsized":"2023-11-08T19:25:51.992046+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:18:56.855441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204945+0000","last_change":"2023-11-08T19:25:50.877298+0000","last_active":"2023-11-08T19:25:52.204945+0000","last_peered":"2023-11-08T19:25:52.204945+0000","last_clean":"2023-11-08T19:25:52.204945+0000","last_became_active":"2023-11-08T19:25:50.877018+0000","last_became_peered":"2023-11-08T19:25:50.877018+0000","last_unstale":"2023-11-08T19:25:52.204945+0000","last_undegraded":"2023-11-08T19:25:52.204945+0000","last_fullsized":"2023-11-08T19:25:52.204945+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:25:07.133280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"32'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.205477+0000","last_change":"2023-11-08T19:25:49.878520+0000","last_active":"2023-11-08T19:25:52.205477+0000","last_peered":"2023-11-08T19:25:52.205477+0000","last_clean":"2023-11-08T19:25:52.205477+0000","last_became_active":"2023-11-08T19:25:49.878244+0000","last_became_peered":"2023-11-08T19:25:49.878244+0000","last_unstale":"2023-11-08T19:25:52.205477+0000","last_undegraded":"2023-11-08T19:25:52.205477+0000","last_fullsized":"2023-11-08T19:25:52.205477+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:23:30.383150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.991899+0000","last_change":"2023-11-08T19:25:50.878056+0000","last_active":"2023-11-08T19:25:51.991899+0000","last_peered":"2023-11-08T19:25:51.991899+0000","last_clean":"2023-11-08T19:25:51.991899+0000","last_became_active":"2023-11-08T19:25:50.877738+0000","last_became_peered":"2023-11-08T19:25:50.877738+0000","last_unstale":"2023-11-08T19:25:51.991899+0000","last_undegraded":"2023-11-08T19:25:51.991899+0000","last_fullsized":"2023-11-08T19:25:51.991899+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:04:01.492989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"32'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.205456+0000","last_change":"2023-11-08T19:25:49.881744+0000","last_active":"2023-11-08T19:25:52.205456+0000","last_peered":"2023-11-08T19:25:52.205456+0000","last_clean":"2023-11-08T19:25:52.205456+0000","last_became_active":"2023-11-08T19:25:49.881436+0000","last_became_peered":"2023-11-08T19:25:49.881436+0000","last_unstale":"2023-11-08T19:25:52.205456+0000","last_undegraded":"2023-11-08T19:25:52.205456+0000","last_fullsized":"2023-11-08T19:25:52.205456+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:08:07.992929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.876822+0000","last_change":"2023-11-08T19:25:50.881348+0000","last_active":"2023-11-08T19:25:51.876822+0000","last_peered":"2023-11-08T19:25:51.876822+0000","last_clean":"2023-11-08T19:25:51.876822+0000","last_became_active":"2023-11-08T19:25:50.881064+0000","last_became_peered":"2023-11-08T19:25:50.881064+0000","last_unstale":"2023-11-08T19:25:51.876822+0000","last_undegraded":"2023-11-08T19:25:51.876822+0000","last_fullsized":"2023-11-08T19:25:51.876822+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:23:31.393415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.205574+0000","last_change":"2023-11-08T19:25:49.884654+0000","last_active":"2023-11-08T19:25:52.205574+0000","last_peered":"2023-11-08T19:25:52.205574+0000","last_clean":"2023-11-08T19:25:52.205574+0000","last_became_active":"2023-11-08T19:25:49.884322+0000","last_became_peered":"2023-11-08T19:25:49.884322+0000","last_unstale":"2023-11-08T19:25:52.205574+0000","last_undegraded":"2023-11-08T19:25:52.205574+0000","last_fullsized":"2023-11-08T19:25:52.205574+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:49:07.738959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.876689+0000","last_change":"2023-11-08T19:25:50.878414+0000","last_active":"2023-11-08T19:25:51.876689+0000","last_peered":"2023-11-08T19:25:51.876689+0000","last_clean":"2023-11-08T19:25:51.876689+0000","last_became_active":"2023-11-08T19:25:50.878145+0000","last_became_peered":"2023-11-08T19:25:50.878145+0000","last_unstale":"2023-11-08T19:25:51.876689+0000","last_undegraded":"2023-11-08T19:25:51.876689+0000","last_fullsized":"2023-11-08T19:25:51.876689+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:49:08.749224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"32'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.989615+0000","last_change":"2023-11-08T19:25:49.878923+0000","last_active":"2023-11-08T19:25:51.989615+0000","last_peered":"2023-11-08T19:25:51.989615+0000","last_clean":"2023-11-08T19:25:51.989615+0000","last_became_active":"2023-11-08T19:25:49.878167+0000","last_became_peered":"2023-11-08T19:25:49.878167+0000","last_unstale":"2023-11-08T19:25:51.989615+0000","last_undegraded":"2023-11-08T19:25:51.989615+0000","last_fullsized":"2023-11-08T19:25:51.989615+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:51:26.807463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.876711+0000","last_change":"2023-11-08T19:25:50.887699+0000","last_active":"2023-11-08T19:25:51.876711+0000","last_peered":"2023-11-08T19:25:51.876711+0000","last_clean":"2023-11-08T19:25:51.876711+0000","last_became_active":"2023-11-08T19:25:50.887423+0000","last_became_peered":"2023-11-08T19:25:50.887423+0000","last_unstale":"2023-11-08T19:25:51.876711+0000","last_undegraded":"2023-11-08T19:25:51.876711+0000","last_fullsized":"2023-11-08T19:25:51.876711+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:08:09.003194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-11-08T19:25:48.857323+0000","last_change":"2023-11-08T19:25:48.857323+0000","last_active":"2023-11-08T19:25:48.844481+0000","last_peered":"2023-11-08T19:25:48.844481+0000","last_clean":"2023-11-08T19:25:48.844481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:25:48.857323+0000","last_undegraded":"2023-11-08T19:25:48.857323+0000","last_fullsized":"2023-11-08T19:25:48.857323+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.876619+0000","last_change":"2023-11-08T19:25:50.888597+0000","last_active":"2023-11-08T19:25:51.876619+0000","last_peered":"2023-11-08T19:25:51.876619+0000","last_clean":"2023-11-08T19:25:51.876619+0000","last_became_active":"2023-11-08T19:25:50.887065+0000","last_became_peered":"2023-11-08T19:25:50.887065+0000","last_unstale":"2023-11-08T19:25:51.876619+0000","last_undegraded":"2023-11-08T19:25:51.876619+0000","last_fullsized":"2023-11-08T19:25:51.876619+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:02:25.988939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.205088+0000","last_change":"2023-11-08T19:25:49.884053+0000","last_active":"2023-11-08T19:25:52.205088+0000","last_peered":"2023-11-08T19:25:52.205088+0000","last_clean":"2023-11-08T19:25:52.205088+0000","last_became_active":"2023-11-08T19:25:49.883822+0000","last_became_peered":"2023-11-08T19:25:49.883822+0000","last_unstale":"2023-11-08T19:25:52.205088+0000","last_undegraded":"2023-11-08T19:25:52.205088+0000","last_fullsized":"2023-11-08T19:25:52.205088+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:12:47.116061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:25:49.854746+0000","last_change":"2023-11-08T19:25:49.854746+0000","last_active":"2023-11-08T19:25:49.854746+0000","last_peered":"2023-11-08T19:25:49.854746+0000","last_clean":"2023-11-08T19:25:49.854746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:25:49.854746+0000","last_undegraded":"2023-11-08T19:25:49.854746+0000","last_fullsized":"2023-11-08T19:25:49.854746+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204303+0000","last_change":"2023-11-08T19:25:49.879327+0000","last_active":"2023-11-08T19:25:52.204303+0000","last_peered":"2023-11-08T19:25:52.204303+0000","last_clean":"2023-11-08T19:25:52.204303+0000","last_became_active":"2023-11-08T19:25:49.879054+0000","last_became_peered":"2023-11-08T19:25:49.879054+0000","last_unstale":"2023-11-08T19:25:52.204303+0000","last_undegraded":"2023-11-08T19:25:52.204303+0000","last_fullsized":"2023-11-08T19:25:52.204303+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:09:04.368768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.876752+0000","last_change":"2023-11-08T19:25:50.874836+0000","last_active":"2023-11-08T19:25:51.876752+0000","last_peered":"2023-11-08T19:25:51.876752+0000","last_clean":"2023-11-08T19:25:51.876752+0000","last_became_active":"2023-11-08T19:25:50.874272+0000","last_became_peered":"2023-11-08T19:25:50.874272+0000","last_unstale":"2023-11-08T19:25:51.876752+0000","last_undegraded":"2023-11-08T19:25:51.876752+0000","last_fullsized":"2023-11-08T19:25:51.876752+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:17:53.651518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"32'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.991956+0000","last_change":"2023-11-08T19:25:49.880740+0000","last_active":"2023-11-08T19:25:51.991956+0000","last_peered":"2023-11-08T19:25:51.991956+0000","last_clean":"2023-11-08T19:25:51.991956+0000","last_became_active":"2023-11-08T19:25:49.880458+0000","last_became_peered":"2023-11-08T19:25:49.880458+0000","last_unstale":"2023-11-08T19:25:51.991956+0000","last_undegraded":"2023-11-08T19:25:51.991956+0000","last_fullsized":"2023-11-08T19:25:51.991956+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:20:42.164809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.205146+0000","last_change":"2023-11-08T19:25:50.883014+0000","last_active":"2023-11-08T19:25:52.205146+0000","last_peered":"2023-11-08T19:25:52.205146+0000","last_clean":"2023-11-08T19:25:52.205146+0000","last_became_active":"2023-11-08T19:25:50.882737+0000","last_became_peered":"2023-11-08T19:25:50.882737+0000","last_unstale":"2023-11-08T19:25:52.205146+0000","last_undegraded":"2023-11-08T19:25:52.205146+0000","last_fullsized":"2023-11-08T19:25:52.205146+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:38:37.341013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"33'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.991098+0000","last_change":"2023-11-08T19:25:49.880299+0000","last_active":"2023-11-08T19:25:51.991098+0000","last_peered":"2023-11-08T19:25:51.991098+0000","last_clean":"2023-11-08T19:25:51.991098+0000","last_became_active":"2023-11-08T19:25:49.878038+0000","last_became_peered":"2023-11-08T19:25:49.878038+0000","last_unstale":"2023-11-08T19:25:51.991098+0000","last_undegraded":"2023-11-08T19:25:51.991098+0000","last_fullsized":"2023-11-08T19:25:51.991098+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:12:39.742779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.989626+0000","last_change":"2023-11-08T19:25:50.888755+0000","last_active":"2023-11-08T19:25:51.989626+0000","last_peered":"2023-11-08T19:25:51.989626+0000","last_clean":"2023-11-08T19:25:51.989626+0000","last_became_active":"2023-11-08T19:25:50.888494+0000","last_became_peered":"2023-11-08T19:25:50.888494+0000","last_unstale":"2023-11-08T19:25:51.989626+0000","last_undegraded":"2023-11-08T19:25:51.989626+0000","last_fullsized":"2023-11-08T19:25:51.989626+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:09:05.379033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.991905+0000","last_change":"2023-11-08T19:25:49.881699+0000","last_active":"2023-11-08T19:25:51.991905+0000","last_peered":"2023-11-08T19:25:51.991905+0000","last_clean":"2023-11-08T19:25:51.991905+0000","last_became_active":"2023-11-08T19:25:49.881062+0000","last_became_peered":"2023-11-08T19:25:49.881062+0000","last_unstale":"2023-11-08T19:25:51.991905+0000","last_undegraded":"2023-11-08T19:25:51.991905+0000","last_fullsized":"2023-11-08T19:25:51.991905+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:49:44.049659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:25:49.854746+0000","last_change":"2023-11-08T19:25:49.854746+0000","last_active":"2023-11-08T19:25:49.854746+0000","last_peered":"2023-11-08T19:25:49.854746+0000","last_clean":"2023-11-08T19:25:49.854746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:25:49.854746+0000","last_undegraded":"2023-11-08T19:25:49.854746+0000","last_fullsized":"2023-11-08T19:25:49.854746+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.989575+0000","last_change":"2023-11-08T19:25:49.879113+0000","last_active":"2023-11-08T19:25:51.989575+0000","last_peered":"2023-11-08T19:25:51.989575+0000","last_clean":"2023-11-08T19:25:51.989575+0000","last_became_active":"2023-11-08T19:25:49.878579+0000","last_became_peered":"2023-11-08T19:25:49.878579+0000","last_unstale":"2023-11-08T19:25:51.989575+0000","last_undegraded":"2023-11-08T19:25:51.989575+0000","last_fullsized":"2023-11-08T19:25:51.989575+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:12:47.116061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.989707+0000","last_change":"2023-11-08T19:25:50.880600+0000","last_active":"2023-11-08T19:25:51.989707+0000","last_peered":"2023-11-08T19:25:51.989707+0000","last_clean":"2023-11-08T19:25:51.989707+0000","last_became_active":"2023-11-08T19:25:50.880384+0000","last_became_peered":"2023-11-08T19:25:50.880384+0000","last_unstale":"2023-11-08T19:25:51.989707+0000","last_undegraded":"2023-11-08T19:25:51.989707+0000","last_fullsized":"2023-11-08T19:25:51.989707+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:20:43.175074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.991693+0000","last_change":"2023-11-08T19:25:49.877196+0000","last_active":"2023-11-08T19:25:51.991693+0000","last_peered":"2023-11-08T19:25:51.991693+0000","last_clean":"2023-11-08T19:25:51.991693+0000","last_became_active":"2023-11-08T19:25:49.876517+0000","last_became_peered":"2023-11-08T19:25:49.876517+0000","last_unstale":"2023-11-08T19:25:51.991693+0000","last_undegraded":"2023-11-08T19:25:51.991693+0000","last_fullsized":"2023-11-08T19:25:51.991693+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:00:52.169251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.203760+0000","last_change":"2023-11-08T19:25:50.893157+0000","last_active":"2023-11-08T19:25:52.203760+0000","last_peered":"2023-11-08T19:25:52.203760+0000","last_clean":"2023-11-08T19:25:52.203760+0000","last_became_active":"2023-11-08T19:25:50.892637+0000","last_became_peered":"2023-11-08T19:25:50.892637+0000","last_unstale":"2023-11-08T19:25:52.203760+0000","last_undegraded":"2023-11-08T19:25:52.203760+0000","last_fullsized":"2023-11-08T19:25:52.203760+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:50:52.902620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.205077+0000","last_change":"2023-11-08T19:25:49.881923+0000","last_active":"2023-11-08T19:25:52.205077+0000","last_peered":"2023-11-08T19:25:52.205077+0000","last_clean":"2023-11-08T19:25:52.205077+0000","last_became_active":"2023-11-08T19:25:49.881670+0000","last_became_peered":"2023-11-08T19:25:49.881670+0000","last_unstale":"2023-11-08T19:25:52.205077+0000","last_undegraded":"2023-11-08T19:25:52.205077+0000","last_fullsized":"2023-11-08T19:25:52.205077+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:51:26.807463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204772+0000","last_change":"2023-11-08T19:25:50.883910+0000","last_active":"2023-11-08T19:25:52.204772+0000","last_peered":"2023-11-08T19:25:52.204772+0000","last_clean":"2023-11-08T19:25:52.204772+0000","last_became_active":"2023-11-08T19:25:50.883615+0000","last_became_peered":"2023-11-08T19:25:50.883615+0000","last_unstale":"2023-11-08T19:25:52.204772+0000","last_undegraded":"2023-11-08T19:25:52.204772+0000","last_fullsized":"2023-11-08T19:25:52.204772+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:20:43.175074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.203857+0000","last_change":"2023-11-08T19:25:49.884441+0000","last_active":"2023-11-08T19:25:52.203857+0000","last_peered":"2023-11-08T19:25:52.203857+0000","last_clean":"2023-11-08T19:25:52.203857+0000","last_became_active":"2023-11-08T19:25:49.884125+0000","last_became_peered":"2023-11-08T19:25:49.884125+0000","last_unstale":"2023-11-08T19:25:52.203857+0000","last_undegraded":"2023-11-08T19:25:52.203857+0000","last_fullsized":"2023-11-08T19:25:52.203857+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:52:19.324495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:25:49.854746+0000","last_change":"2023-11-08T19:25:49.854746+0000","last_active":"2023-11-08T19:25:49.854746+0000","last_peered":"2023-11-08T19:25:49.854746+0000","last_clean":"2023-11-08T19:25:49.854746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:25:49.854746+0000","last_undegraded":"2023-11-08T19:25:49.854746+0000","last_fullsized":"2023-11-08T19:25:49.854746+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-11-08T19:25:48.857272+0000","last_change":"2023-11-08T19:25:48.857272+0000","last_active":"2023-11-08T19:25:48.844481+0000","last_peered":"2023-11-08T19:25:48.844481+0000","last_clean":"2023-11-08T19:25:48.844481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:25:48.857272+0000","last_undegraded":"2023-11-08T19:25:48.857272+0000","last_fullsized":"2023-11-08T19:25:48.857272+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":175,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-11-08T19:25:43.589068+0000","last_change":"2023-11-08T19:24:45.888122+0000","last_active":"2023-11-08T19:25:43.589068+0000","last_peered":"2023-11-08T19:25:43.589068+0000","last_clean":"2023-11-08T19:25:43.589068+0000","last_became_active":"2023-11-08T19:24:45.887048+0000","last_became_peered":"2023-11-08T19:24:45.887048+0000","last_unstale":"2023-11-08T19:25:43.589068+0000","last_undegraded":"2023-11-08T19:25:43.589068+0000","last_fullsized":"2023-11-08T19:25:43.589068+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:44.575133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:44.575133+0000","last_clean_scrub_stamp":"2023-11-08T19:24:44.575133+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:19:20.202062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":96,"acting":96,"num_store_stats":6},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":1828,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":4114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":35,"ondisk_log_size":35,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574865,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27928,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343912,"statfs":{"total":96636764160,"available":96608165888,"internally_reserved":0,"allocated":929792,"data_stored":716892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.60999999999999999}]},{"osd":1,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.76100000000000001}]},{"osd":2,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.63200000000000001}]},{"osd":3,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.746},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.67800000000000005}]},{"osd":4,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.58299999999999996}]},{"osd":5,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.71999999999999997}]},{"osd":6,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.64400000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":335872,"data_stored":126524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":21.359999999999999,"5min":21.359999999999999,"15min":21.359999999999999},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":10.329000000000001,"5min":10.329000000000001,"15min":10.329000000000001},"last":0.46700000000000003}]},{"osd":1,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":3.4239999999999999,"5min":3.4239999999999999,"15min":3.4239999999999999},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":3.4809999999999999,"5min":3.4809999999999999,"15min":3.4809999999999999},"last":0.66500000000000004}]},{"osd":2,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":10.398,"5min":10.398,"15min":10.398},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":5.0170000000000003,"5min":5.0170000000000003,"15min":5.0170000000000003},"last":0.60999999999999999}]},{"osd":3,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":3.5710000000000002,"5min":3.5710000000000002,"15min":3.5710000000000002},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":10.368,"5min":10.368,"15min":10.368},"last":0.70299999999999996}]},{"osd":4,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.78500000000000003}]},{"osd":5,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.753},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.82099999999999995}]},{"osd":7,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.80700000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574866,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27404,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344436,"statfs":{"total":96636764160,"available":96608702464,"internally_reserved":0,"allocated":389120,"data_stored":147500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":1.389,"5min":1.389,"15min":1.389},"last":0.499},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":9.1370000000000005,"5min":9.1370000000000005,"15min":9.1370000000000005},"last":0.39400000000000002}]},{"osd":2,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.433,"5min":0.433,"15min":0.433},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":1.425,"5min":1.425,"15min":1.425},"last":0.42699999999999999}]},{"osd":3,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":11.957000000000001,"5min":11.957000000000001,"15min":11.957000000000001},"last":0.47699999999999998},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":18.039000000000001,"5min":18.039000000000001,"15min":18.039000000000001},"last":0.46300000000000002}]},{"osd":4,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":6.181,"5min":6.181,"15min":6.181},"last":1.4370000000000001},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.488,"5min":1.488,"15min":1.488},"last":1.411}]},{"osd":5,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":9.2420000000000009,"5min":9.2420000000000009,"15min":9.2420000000000009},"last":1.421},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":6.2110000000000003,"5min":6.2110000000000003,"15min":6.2110000000000003},"last":1.2969999999999999}]},{"osd":6,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":9.1880000000000006,"5min":9.1880000000000006,"15min":9.1880000000000006},"last":1.4530000000000001},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.462,"5min":1.462,"15min":1.462},"last":1.272}]},{"osd":7,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":6.1479999999999997,"5min":6.1479999999999997,"15min":6.1479999999999997},"last":1.3979999999999999},{"interface":"front","average":{"1min":1.028,"5min":1.028,"15min":1.028},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":9.2170000000000005,"5min":9.2170000000000005,"15min":9.2170000000000005},"last":1.353}]}]},{"osd":0,"up_from":13,"seq":55834574866,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27972,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343868,"statfs":{"total":96636764160,"available":96608120832,"internally_reserved":0,"allocated":966656,"data_stored":736234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":4.1200000000000001,"5min":4.1200000000000001,"15min":4.1200000000000001},"last":0.32200000000000001},{"interface":"front","average":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":16.940999999999999,"5min":16.940999999999999,"15min":16.940999999999999},"last":0.44500000000000001}]},{"osd":2,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.37},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":7.0209999999999999,"5min":7.0209999999999999,"15min":7.0209999999999999},"last":0.41399999999999998}]},{"osd":3,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":4.0940000000000003,"5min":4.0940000000000003,"15min":4.0940000000000003},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.432}]},{"osd":4,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":4.0049999999999999,"5min":4.0049999999999999,"15min":4.0049999999999999},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.79200000000000004}]},{"osd":5,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":4.3090000000000002,"5min":4.3090000000000002,"15min":4.3090000000000002},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":4.0709999999999997,"5min":4.0709999999999997,"15min":4.0709999999999997},"last":0.67300000000000004}]},{"osd":6,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":4.2560000000000002,"5min":4.2560000000000002,"15min":4.2560000000000002},"last":0.75900000000000001}]},{"osd":7,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":4.2850000000000001,"5min":4.2850000000000001,"15min":4.2850000000000001},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":4.0460000000000003,"5min":4.0460000000000003,"15min":4.0460000000000003},"last":0.80600000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574866,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27392,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":376832,"data_stored":145936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"min":{"1min":0.20100000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":6.0999999999999996,"5min":6.0999999999999996,"15min":6.0999999999999996},"last":0.249}]},{"osd":1,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"last":0.28799999999999998},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.624,"5min":0.624,"15min":0.624},"last":0.313}]},{"osd":3,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.38300000000000001}]},{"osd":4,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.55500000000000005}]},{"osd":5,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.57599999999999996}]},{"osd":6,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.65100000000000002}]},{"osd":7,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.628},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.80000000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574867,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27404,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344436,"statfs":{"total":96636764160,"available":96608702464,"internally_reserved":0,"allocated":389120,"data_stored":147500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:25:46 2023","interfaces":[{"interface":"back","average":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.626,"5min":0.626,"15min":0.626},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.33100000000000002}]},{"osd":1,"last update":"Wed Nov 8 19:25:46 2023","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.34999999999999998}]},{"osd":2,"last update":"Wed Nov 8 19:25:46 2023","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.37,"15min":0.37},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"last":0.40000000000000002}]},{"osd":4,"last update":"Wed Nov 8 19:25:46 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.70599999999999996}]},{"osd":5,"last update":"Wed Nov 8 19:25:46 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.56699999999999995}]},{"osd":6,"last update":"Wed Nov 8 19:25:46 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.74199999999999999}]},{"osd":7,"last update":"Wed Nov 8 19:25:46 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.66400000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574866,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":380928,"data_stored":145970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:25:47 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":6.5970000000000004,"5min":6.5970000000000004,"15min":6.5970000000000004},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":2.5030000000000001,"5min":2.5030000000000001,"15min":2.5030000000000001},"last":0.75}]},{"osd":1,"last update":"Wed Nov 8 19:25:47 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.72599999999999998}]},{"osd":2,"last update":"Wed Nov 8 19:25:47 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":6.6920000000000002,"5min":6.6920000000000002,"15min":6.6920000000000002},"last":0.871}]},{"osd":3,"last update":"Wed Nov 8 19:25:47 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.91700000000000004}]},{"osd":5,"last update":"Wed Nov 8 19:25:47 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.65900000000000003}]},{"osd":6,"last update":"Wed Nov 8 19:25:47 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.78100000000000003}]},{"osd":7,"last update":"Wed Nov 8 19:25:47 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.85299999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574866,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":380928,"data_stored":146538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.627},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.438}]},{"osd":1,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.50600000000000001}]},{"osd":2,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.46300000000000002}]},{"osd":3,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.58299999999999996}]},{"osd":4,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.59699999999999998}]},{"osd":6,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.67700000000000005}]},{"osd":7,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.61499999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-08T19:25:54.973 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-08T19:25:55.126 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.124+0000 7f87888de700 1 Processor -- start 2023-11-08T19:25:55.126 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.124+0000 7f87888de700 1 -- start start 2023-11-08T19:25:55.129 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.125+0000 7f87888de700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f878415ba30 0x7f87840b14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:55.129 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.125+0000 7f87888de700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f87840d03a0 0x7f87840d1560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:55.130 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.125+0000 7f87888de700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87840b1bc0 con 0x7f87840d1aa0 2023-11-08T19:25:55.130 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.125+0000 7f87888de700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87840b1ea0 con 0x7f87840d03a0 2023-11-08T19:25:55.130 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.125+0000 7f87888de700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87840b2180 con 0x7f878415ba30 2023-11-08T19:25:55.130 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.125+0000 7f8781d9b700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f87840d03a0 0x7f87840d1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:55.130 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.125+0000 7f8782d9d700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f87840d1aa0 0x7f87840ce2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:46852/0 (socket says 172.21.15.143:46852) 2023-11-08T19:25:55.131 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.125+0000 7f8782d9d700 1 -- 172.21.15.143:0/1036001374 learned_addr learned my addr 172.21.15.143:0/1036001374 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:25:55.131 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.126+0000 7f878259c700 1 --2- 172.21.15.143:0/1036001374 >> v2:172.21.15.174:3300/0 conn(0x7f878415ba30 0x7f87840b14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:55.131 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.126+0000 7f878159a700 1 -- 172.21.15.143:0/1036001374 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3128124276 0 0) 0x7f87840b1bc0 con 0x7f87840d1aa0 2023-11-08T19:25:55.131 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.126+0000 7f878159a700 1 -- 172.21.15.143:0/1036001374 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87700035b0 con 0x7f87840d1aa0 2023-11-08T19:25:55.132 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.126+0000 7f8781d9b700 1 -- 172.21.15.143:0/1036001374 >> v2:172.21.15.174:3300/0 conn(0x7f878415ba30 msgr2=0x7f87840b14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:55.132 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.126+0000 7f8781d9b700 1 --2- 172.21.15.143:0/1036001374 >> v2:172.21.15.174:3300/0 conn(0x7f878415ba30 0x7f87840b14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:55.132 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.126+0000 7f8781d9b700 1 -- 172.21.15.143:0/1036001374 >> v1:172.21.15.143:6789/0 conn(0x7f87840d1aa0 legacy=0x7f87840ce2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:55.132 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.126+0000 7f8781d9b700 1 -- 172.21.15.143:0/1036001374 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87840b2460 con 0x7f87840d03a0 2023-11-08T19:25:55.132 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.126+0000 7f8781d9b700 1 --2- 172.21.15.143:0/1036001374 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f87840d03a0 0x7f87840d1560 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f87840cf3d0 tx=0x7f876c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e854b2247fc297f server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:55.133 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.126+0000 7f878159a700 1 -- 172.21.15.143:0/1036001374 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f876c011830 con 0x7f87840d03a0 2023-11-08T19:25:55.133 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.126+0000 7f878159a700 1 -- 172.21.15.143:0/1036001374 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f876c011990 con 0x7f87840d03a0 2023-11-08T19:25:55.133 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.126+0000 7f878159a700 1 -- 172.21.15.143:0/1036001374 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f876c011d00 con 0x7f87840d03a0 2023-11-08T19:25:55.133 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.126+0000 7f87888de700 1 -- 172.21.15.143:0/1036001374 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f87840d03a0 msgr2=0x7f87840d1560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:55.133 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.126+0000 7f87888de700 1 --2- 172.21.15.143:0/1036001374 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f87840d03a0 0x7f87840d1560 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f87840cf3d0 tx=0x7f876c009b20 comp rx=0 tx=0).stop 2023-11-08T19:25:55.134 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.127+0000 7f87888de700 1 -- 172.21.15.143:0/1036001374 shutdown_connections 2023-11-08T19:25:55.134 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.127+0000 7f87888de700 1 --2- 172.21.15.143:0/1036001374 >> v2:172.21.15.174:3300/0 conn(0x7f878415ba30 0x7f87840b14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:55.134 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.127+0000 7f87888de700 1 --2- 172.21.15.143:0/1036001374 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f87840d03a0 0x7f87840d1560 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:55.134 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.127+0000 7f87888de700 1 -- 172.21.15.143:0/1036001374 >> 172.21.15.143:0/1036001374 conn(0x7f878408b000 msgr2=0x7f878409ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:55.134 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.127+0000 7f87888de700 1 -- 172.21.15.143:0/1036001374 shutdown_connections 2023-11-08T19:25:55.135 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.127+0000 7f87888de700 1 -- 172.21.15.143:0/1036001374 wait complete. 2023-11-08T19:25:55.138 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.128+0000 7f87888de700 1 Processor -- start 2023-11-08T19:25:55.138 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.128+0000 7f87888de700 1 -- start start 2023-11-08T19:25:55.139 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.129+0000 7f87888de700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f87840d03a0 0x7f8784159b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:55.139 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.129+0000 7f87888de700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f87840d1aa0 0x7f87840c72b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:55.139 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.129+0000 7f87888de700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f878415ba30 0x7f878415de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:55.139 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.129+0000 7f87888de700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87841f5550 con 0x7f878415ba30 2023-11-08T19:25:55.140 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.129+0000 7f87888de700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87841f48d0 con 0x7f87840d1aa0 2023-11-08T19:25:55.140 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.129+0000 7f87888de700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87841f4bb0 con 0x7f87840d03a0 2023-11-08T19:25:55.140 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.129+0000 7f8782d9d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f878415ba30 0x7f878415de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:55.140 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.129+0000 7f8782d9d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f878415ba30 0x7f878415de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50704/0 (socket says 172.21.15.143:50704) 2023-11-08T19:25:55.141 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.129+0000 7f8782d9d700 1 -- 172.21.15.143:0/1040577120 learned_addr learned my addr 172.21.15.143:0/1040577120 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:55.141 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.130+0000 7f878259c700 1 --2- 172.21.15.143:0/1040577120 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f87840d03a0 0x7f8784159b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:55.141 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.130+0000 7f8781d9b700 1 --2- 172.21.15.143:0/1040577120 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f87840d1aa0 0x7f87840c72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:55.141 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.130+0000 7f8782d9d700 1 -- 172.21.15.143:0/1040577120 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f87840d03a0 msgr2=0x7f8784159b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:55.142 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.130+0000 7f8782d9d700 1 --2- 172.21.15.143:0/1040577120 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f87840d03a0 0x7f8784159b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:55.142 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.130+0000 7f8782d9d700 1 -- 172.21.15.143:0/1040577120 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f87840d1aa0 msgr2=0x7f87840c72b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:55.142 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.130+0000 7f8782d9d700 1 --2- 172.21.15.143:0/1040577120 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f87840d1aa0 0x7f87840c72b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:55.142 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.130+0000 7f8782d9d700 1 -- 172.21.15.143:0/1040577120 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f876c0094a0 con 0x7f878415ba30 2023-11-08T19:25:55.143 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.130+0000 7f878259c700 1 --2- 172.21.15.143:0/1040577120 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f87840d03a0 0x7f8784159b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:25:55.143 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.130+0000 7f8781d9b700 1 --2- 172.21.15.143:0/1040577120 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f87840d1aa0 0x7f87840c72b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:25:55.143 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.130+0000 7f8782d9d700 1 --2- 172.21.15.143:0/1040577120 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f878415ba30 0x7f878415de30 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f877400c4d0 tx=0x7f877400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:55.144 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.130+0000 7f877affd700 1 -- 172.21.15.143:0/1040577120 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8774005d70 con 0x7f878415ba30 2023-11-08T19:25:55.144 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.130+0000 7f877affd700 1 -- 172.21.15.143:0/1040577120 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87740070d0 con 0x7f878415ba30 2023-11-08T19:25:55.144 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.130+0000 7f87888de700 1 -- 172.21.15.143:0/1040577120 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87841f4e90 con 0x7f878415ba30 2023-11-08T19:25:55.144 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.130+0000 7f877affd700 1 -- 172.21.15.143:0/1040577120 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8774007440 con 0x7f878415ba30 2023-11-08T19:25:55.145 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.130+0000 7f87888de700 1 -- 172.21.15.143:0/1040577120 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f878415a0c0 con 0x7f878415ba30 2023-11-08T19:25:55.145 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.131+0000 7f87888de700 1 -- 172.21.15.143:0/1040577120 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87841f5690 con 0x7f878415ba30 2023-11-08T19:25:55.145 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.131+0000 7f877affd700 1 -- 172.21.15.143:0/1040577120 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f87740075a0 con 0x7f878415ba30 2023-11-08T19:25:55.145 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.132+0000 7f877affd700 1 --2- 172.21.15.143:0/1040577120 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f876406af00 0x7f876406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:55.145 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.132+0000 7f877affd700 1 -- 172.21.15.143:0/1040577120 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f87740889a0 con 0x7f878415ba30 2023-11-08T19:25:55.146 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.132+0000 7f878259c700 1 --2- 172.21.15.143:0/1040577120 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f876406af00 0x7f876406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:55.146 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.133+0000 7f878259c700 1 --2- 172.21.15.143:0/1040577120 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f876406af00 0x7f876406d3c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f877c009fd0 tx=0x7f877c00b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:55.146 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.136+0000 7f877affd700 1 -- 172.21.15.143:0/1040577120 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8774053b90 con 0x7f878415ba30 2023-11-08T19:25:55.323 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.322+0000 7f87888de700 1 -- 172.21.15.143:0/1040577120 --> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f87840b0430 con 0x7f876406af00 2023-11-08T19:25:55.332 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.330+0000 7f877affd700 1 -- 172.21.15.143:0/1040577120 <== mgr.4099 v2:172.21.15.174:6832/4037280178 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286848 (secure 0 0 0) 0x7f87840b0430 con 0x7f876406af00 2023-11-08T19:25:55.336 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:25:55.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.333+0000 7f87888de700 1 -- 172.21.15.143:0/1040577120 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f876406af00 msgr2=0x7f876406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:55.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.333+0000 7f87888de700 1 --2- 172.21.15.143:0/1040577120 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f876406af00 0x7f876406d3c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f877c009fd0 tx=0x7f877c00b040 comp rx=0 tx=0).stop 2023-11-08T19:25:55.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.333+0000 7f87888de700 1 -- 172.21.15.143:0/1040577120 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f878415ba30 msgr2=0x7f878415de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:55.341 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.333+0000 7f87888de700 1 --2- 172.21.15.143:0/1040577120 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f878415ba30 0x7f878415de30 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f877400c4d0 tx=0x7f877400c890 comp rx=0 tx=0).stop 2023-11-08T19:25:55.341 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.334+0000 7f87888de700 1 -- 172.21.15.143:0/1040577120 shutdown_connections 2023-11-08T19:25:55.341 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.334+0000 7f87888de700 1 --2- 172.21.15.143:0/1040577120 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f876406af00 0x7f876406d3c0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:55.342 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.334+0000 7f87888de700 1 --2- 172.21.15.143:0/1040577120 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f87840d03a0 0x7f8784159b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:55.342 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.334+0000 7f87888de700 1 --2- 172.21.15.143:0/1040577120 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f878415ba30 0x7f878415de30 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:55.342 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.334+0000 7f87888de700 1 --2- 172.21.15.143:0/1040577120 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f87840d1aa0 0x7f87840c72b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:55.342 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.334+0000 7f87888de700 1 -- 172.21.15.143:0/1040577120 >> 172.21.15.143:0/1040577120 conn(0x7f878408b000 msgr2=0x7f87840aed10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:55.343 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.334+0000 7f87888de700 1 -- 172.21.15.143:0/1040577120 shutdown_connections 2023-11-08T19:25:55.343 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.334+0000 7f87888de700 1 -- 172.21.15.143:0/1040577120 wait complete. 2023-11-08T19:25:55.343 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2023-11-08T19:25:55.347 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":61,"stamp":"2023-11-08T19:25:54.525094+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592196,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":200,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":152,"ondisk_log_size":152,"up":162,"acting":162,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":117,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220240,"kb_used_data":4052,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754480,"statfs":{"total":773094113280,"available":772868587520,"internally_reserved":0,"allocated":4149248,"data_stored":2313094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1828,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.669558"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:25:49.854746+0000","last_change":"2023-11-08T19:25:49.854746+0000","last_active":"2023-11-08T19:25:49.854746+0000","last_peered":"2023-11-08T19:25:49.854746+0000","last_clean":"2023-11-08T19:25:49.854746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:25:49.854746+0000","last_undegraded":"2023-11-08T19:25:49.854746+0000","last_fullsized":"2023-11-08T19:25:49.854746+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204169+0000","last_change":"2023-11-08T19:25:50.890421+0000","last_active":"2023-11-08T19:25:52.204169+0000","last_peered":"2023-11-08T19:25:52.204169+0000","last_clean":"2023-11-08T19:25:52.204169+0000","last_became_active":"2023-11-08T19:25:50.890142+0000","last_became_peered":"2023-11-08T19:25:50.890142+0000","last_unstale":"2023-11-08T19:25:52.204169+0000","last_undegraded":"2023-11-08T19:25:52.204169+0000","last_fullsized":"2023-11-08T19:25:52.204169+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:01:43.267493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.205154+0000","last_change":"2023-11-08T19:25:50.875008+0000","last_active":"2023-11-08T19:25:52.205154+0000","last_peered":"2023-11-08T19:25:52.205154+0000","last_clean":"2023-11-08T19:25:52.205154+0000","last_became_active":"2023-11-08T19:25:50.874499+0000","last_became_peered":"2023-11-08T19:25:50.874499+0000","last_unstale":"2023-11-08T19:25:52.205154+0000","last_undegraded":"2023-11-08T19:25:52.205154+0000","last_fullsized":"2023-11-08T19:25:52.205154+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:27:11.209258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.991817+0000","last_change":"2023-11-08T19:25:50.883677+0000","last_active":"2023-11-08T19:25:51.991817+0000","last_peered":"2023-11-08T19:25:51.991817+0000","last_clean":"2023-11-08T19:25:51.991817+0000","last_became_active":"2023-11-08T19:25:50.883141+0000","last_became_peered":"2023-11-08T19:25:50.883141+0000","last_unstale":"2023-11-08T19:25:51.991817+0000","last_undegraded":"2023-11-08T19:25:51.991817+0000","last_fullsized":"2023-11-08T19:25:51.991817+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:56:01.687509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.876619+0000","last_change":"2023-11-08T19:25:50.877229+0000","last_active":"2023-11-08T19:25:51.876619+0000","last_peered":"2023-11-08T19:25:51.876619+0000","last_clean":"2023-11-08T19:25:51.876619+0000","last_became_active":"2023-11-08T19:25:50.876959+0000","last_became_peered":"2023-11-08T19:25:50.876959+0000","last_unstale":"2023-11-08T19:25:51.876619+0000","last_undegraded":"2023-11-08T19:25:51.876619+0000","last_fullsized":"2023-11-08T19:25:51.876619+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:27:09.954811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204135+0000","last_change":"2023-11-08T19:25:50.883530+0000","last_active":"2023-11-08T19:25:52.204135+0000","last_peered":"2023-11-08T19:25:52.204135+0000","last_clean":"2023-11-08T19:25:52.204135+0000","last_became_active":"2023-11-08T19:25:50.883198+0000","last_became_peered":"2023-11-08T19:25:50.883198+0000","last_unstale":"2023-11-08T19:25:52.204135+0000","last_undegraded":"2023-11-08T19:25:52.204135+0000","last_fullsized":"2023-11-08T19:25:52.204135+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:27:11.209258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:25:49.854746+0000","last_change":"2023-11-08T19:25:49.854746+0000","last_active":"2023-11-08T19:25:49.854746+0000","last_peered":"2023-11-08T19:25:49.854746+0000","last_clean":"2023-11-08T19:25:49.854746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:25:49.854746+0000","last_undegraded":"2023-11-08T19:25:49.854746+0000","last_fullsized":"2023-11-08T19:25:49.854746+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:25:49.854746+0000","last_change":"2023-11-08T19:25:49.854746+0000","last_active":"2023-11-08T19:25:49.854746+0000","last_peered":"2023-11-08T19:25:49.854746+0000","last_clean":"2023-11-08T19:25:49.854746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:25:49.854746+0000","last_undegraded":"2023-11-08T19:25:49.854746+0000","last_fullsized":"2023-11-08T19:25:49.854746+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.205585+0000","last_change":"2023-11-08T19:25:50.888816+0000","last_active":"2023-11-08T19:25:52.205585+0000","last_peered":"2023-11-08T19:25:52.205585+0000","last_clean":"2023-11-08T19:25:52.205585+0000","last_became_active":"2023-11-08T19:25:50.888536+0000","last_became_peered":"2023-11-08T19:25:50.888536+0000","last_unstale":"2023-11-08T19:25:52.205585+0000","last_undegraded":"2023-11-08T19:25:52.205585+0000","last_fullsized":"2023-11-08T19:25:52.205585+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:56:01.687509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.991994+0000","last_change":"2023-11-08T19:25:50.878973+0000","last_active":"2023-11-08T19:25:51.991994+0000","last_peered":"2023-11-08T19:25:51.991994+0000","last_clean":"2023-11-08T19:25:51.991994+0000","last_became_active":"2023-11-08T19:25:50.878706+0000","last_became_peered":"2023-11-08T19:25:50.878706+0000","last_unstale":"2023-11-08T19:25:51.991994+0000","last_undegraded":"2023-11-08T19:25:51.991994+0000","last_fullsized":"2023-11-08T19:25:51.991994+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:49:41.475597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:25:49.854746+0000","last_change":"2023-11-08T19:25:49.854746+0000","last_active":"2023-11-08T19:25:49.854746+0000","last_peered":"2023-11-08T19:25:49.854746+0000","last_clean":"2023-11-08T19:25:49.854746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:25:49.854746+0000","last_undegraded":"2023-11-08T19:25:49.854746+0000","last_fullsized":"2023-11-08T19:25:49.854746+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204768+0000","last_change":"2023-11-08T19:25:50.877227+0000","last_active":"2023-11-08T19:25:52.204768+0000","last_peered":"2023-11-08T19:25:52.204768+0000","last_clean":"2023-11-08T19:25:52.204768+0000","last_became_active":"2023-11-08T19:25:50.876876+0000","last_became_peered":"2023-11-08T19:25:50.876876+0000","last_unstale":"2023-11-08T19:25:52.204768+0000","last_undegraded":"2023-11-08T19:25:52.204768+0000","last_fullsized":"2023-11-08T19:25:52.204768+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:05:33.227810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.876804+0000","last_change":"2023-11-08T19:25:50.890044+0000","last_active":"2023-11-08T19:25:51.876804+0000","last_peered":"2023-11-08T19:25:51.876804+0000","last_clean":"2023-11-08T19:25:51.876804+0000","last_became_active":"2023-11-08T19:25:50.889741+0000","last_became_peered":"2023-11-08T19:25:50.889741+0000","last_unstale":"2023-11-08T19:25:51.876804+0000","last_undegraded":"2023-11-08T19:25:51.876804+0000","last_fullsized":"2023-11-08T19:25:51.876804+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:54:55.202302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204255+0000","last_change":"2023-11-08T19:25:50.876264+0000","last_active":"2023-11-08T19:25:52.204255+0000","last_peered":"2023-11-08T19:25:52.204255+0000","last_clean":"2023-11-08T19:25:52.204255+0000","last_became_active":"2023-11-08T19:25:50.875971+0000","last_became_peered":"2023-11-08T19:25:50.875971+0000","last_unstale":"2023-11-08T19:25:52.204255+0000","last_undegraded":"2023-11-08T19:25:52.204255+0000","last_fullsized":"2023-11-08T19:25:52.204255+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:50:30.103716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204868+0000","last_change":"2023-11-08T19:25:50.877018+0000","last_active":"2023-11-08T19:25:52.204868+0000","last_peered":"2023-11-08T19:25:52.204868+0000","last_clean":"2023-11-08T19:25:52.204868+0000","last_became_active":"2023-11-08T19:25:50.876682+0000","last_became_peered":"2023-11-08T19:25:50.876682+0000","last_unstale":"2023-11-08T19:25:52.204868+0000","last_undegraded":"2023-11-08T19:25:52.204868+0000","last_fullsized":"2023-11-08T19:25:52.204868+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:52:20.334760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:25:49.854746+0000","last_change":"2023-11-08T19:25:49.854746+0000","last_active":"2023-11-08T19:25:49.854746+0000","last_peered":"2023-11-08T19:25:49.854746+0000","last_clean":"2023-11-08T19:25:49.854746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:25:49.854746+0000","last_undegraded":"2023-11-08T19:25:49.854746+0000","last_fullsized":"2023-11-08T19:25:49.854746+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:25:49.854746+0000","last_change":"2023-11-08T19:25:49.854746+0000","last_active":"2023-11-08T19:25:49.854746+0000","last_peered":"2023-11-08T19:25:49.854746+0000","last_clean":"2023-11-08T19:25:49.854746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:25:49.854746+0000","last_undegraded":"2023-11-08T19:25:49.854746+0000","last_fullsized":"2023-11-08T19:25:49.854746+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204251+0000","last_change":"2023-11-08T19:25:50.890954+0000","last_active":"2023-11-08T19:25:52.204251+0000","last_peered":"2023-11-08T19:25:52.204251+0000","last_clean":"2023-11-08T19:25:52.204251+0000","last_became_active":"2023-11-08T19:25:50.890695+0000","last_became_peered":"2023-11-08T19:25:50.890695+0000","last_unstale":"2023-11-08T19:25:52.204251+0000","last_undegraded":"2023-11-08T19:25:52.204251+0000","last_fullsized":"2023-11-08T19:25:52.204251+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:44:02.415941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204041+0000","last_change":"2023-11-08T19:25:50.892390+0000","last_active":"2023-11-08T19:25:52.204041+0000","last_peered":"2023-11-08T19:25:52.204041+0000","last_clean":"2023-11-08T19:25:52.204041+0000","last_became_active":"2023-11-08T19:25:50.892131+0000","last_became_peered":"2023-11-08T19:25:50.892131+0000","last_unstale":"2023-11-08T19:25:52.204041+0000","last_undegraded":"2023-11-08T19:25:52.204041+0000","last_fullsized":"2023-11-08T19:25:52.204041+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:25:07.133280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.876861+0000","last_change":"2023-11-08T19:25:50.874502+0000","last_active":"2023-11-08T19:25:51.876861+0000","last_peered":"2023-11-08T19:25:51.876861+0000","last_clean":"2023-11-08T19:25:51.876861+0000","last_became_active":"2023-11-08T19:25:50.874216+0000","last_became_peered":"2023-11-08T19:25:50.874216+0000","last_unstale":"2023-11-08T19:25:51.876861+0000","last_undegraded":"2023-11-08T19:25:51.876861+0000","last_fullsized":"2023-11-08T19:25:51.876861+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:39:23.438674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204719+0000","last_change":"2023-11-08T19:25:50.886322+0000","last_active":"2023-11-08T19:25:52.204719+0000","last_peered":"2023-11-08T19:25:52.204719+0000","last_clean":"2023-11-08T19:25:52.204719+0000","last_became_active":"2023-11-08T19:25:50.886011+0000","last_became_peered":"2023-11-08T19:25:50.886011+0000","last_unstale":"2023-11-08T19:25:52.204719+0000","last_undegraded":"2023-11-08T19:25:52.204719+0000","last_fullsized":"2023-11-08T19:25:52.204719+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:09:05.379033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204755+0000","last_change":"2023-11-08T19:25:50.881922+0000","last_active":"2023-11-08T19:25:52.204755+0000","last_peered":"2023-11-08T19:25:52.204755+0000","last_clean":"2023-11-08T19:25:52.204755+0000","last_became_active":"2023-11-08T19:25:50.880836+0000","last_became_peered":"2023-11-08T19:25:50.880836+0000","last_unstale":"2023-11-08T19:25:52.204755+0000","last_undegraded":"2023-11-08T19:25:52.204755+0000","last_fullsized":"2023-11-08T19:25:52.204755+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:49:45.059924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.989623+0000","last_change":"2023-11-08T19:25:50.886648+0000","last_active":"2023-11-08T19:25:51.989623+0000","last_peered":"2023-11-08T19:25:51.989623+0000","last_clean":"2023-11-08T19:25:51.989623+0000","last_became_active":"2023-11-08T19:25:50.886376+0000","last_became_peered":"2023-11-08T19:25:50.886376+0000","last_unstale":"2023-11-08T19:25:51.989623+0000","last_undegraded":"2023-11-08T19:25:51.989623+0000","last_fullsized":"2023-11-08T19:25:51.989623+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:49:45.059924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204354+0000","last_change":"2023-11-08T19:25:50.875989+0000","last_active":"2023-11-08T19:25:52.204354+0000","last_peered":"2023-11-08T19:25:52.204354+0000","last_clean":"2023-11-08T19:25:52.204354+0000","last_became_active":"2023-11-08T19:25:50.874428+0000","last_became_peered":"2023-11-08T19:25:50.874428+0000","last_unstale":"2023-11-08T19:25:52.204354+0000","last_undegraded":"2023-11-08T19:25:52.204354+0000","last_fullsized":"2023-11-08T19:25:52.204354+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:04:01.492989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.991856+0000","last_change":"2023-11-08T19:25:50.885263+0000","last_active":"2023-11-08T19:25:51.991856+0000","last_peered":"2023-11-08T19:25:51.991856+0000","last_clean":"2023-11-08T19:25:51.991856+0000","last_became_active":"2023-11-08T19:25:50.884856+0000","last_became_peered":"2023-11-08T19:25:50.884856+0000","last_unstale":"2023-11-08T19:25:51.991856+0000","last_undegraded":"2023-11-08T19:25:51.991856+0000","last_fullsized":"2023-11-08T19:25:51.991856+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:17:24.728184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204268+0000","last_change":"2023-11-08T19:25:50.884128+0000","last_active":"2023-11-08T19:25:52.204268+0000","last_peered":"2023-11-08T19:25:52.204268+0000","last_clean":"2023-11-08T19:25:52.204268+0000","last_became_active":"2023-11-08T19:25:50.883848+0000","last_became_peered":"2023-11-08T19:25:50.883848+0000","last_unstale":"2023-11-08T19:25:52.204268+0000","last_undegraded":"2023-11-08T19:25:52.204268+0000","last_fullsized":"2023-11-08T19:25:52.204268+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:32:01.237850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.992064+0000","last_change":"2023-11-08T19:25:50.885482+0000","last_active":"2023-11-08T19:25:51.992064+0000","last_peered":"2023-11-08T19:25:51.992064+0000","last_clean":"2023-11-08T19:25:51.992064+0000","last_became_active":"2023-11-08T19:25:50.884816+0000","last_became_peered":"2023-11-08T19:25:50.884816+0000","last_unstale":"2023-11-08T19:25:51.992064+0000","last_undegraded":"2023-11-08T19:25:51.992064+0000","last_fullsized":"2023-11-08T19:25:51.992064+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:05:33.227810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204941+0000","last_change":"2023-11-08T19:25:50.871479+0000","last_active":"2023-11-08T19:25:52.204941+0000","last_peered":"2023-11-08T19:25:52.204941+0000","last_clean":"2023-11-08T19:25:52.204941+0000","last_became_active":"2023-11-08T19:25:50.871211+0000","last_became_peered":"2023-11-08T19:25:50.871211+0000","last_unstale":"2023-11-08T19:25:52.204941+0000","last_undegraded":"2023-11-08T19:25:52.204941+0000","last_fullsized":"2023-11-08T19:25:52.204941+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:25:16.223226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204982+0000","last_change":"2023-11-08T19:25:50.881545+0000","last_active":"2023-11-08T19:25:52.204982+0000","last_peered":"2023-11-08T19:25:52.204982+0000","last_clean":"2023-11-08T19:25:52.204982+0000","last_became_active":"2023-11-08T19:25:50.881277+0000","last_became_peered":"2023-11-08T19:25:50.881277+0000","last_unstale":"2023-11-08T19:25:52.204982+0000","last_undegraded":"2023-11-08T19:25:52.204982+0000","last_fullsized":"2023-11-08T19:25:52.204982+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:18:57.865706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:25:49.854746+0000","last_change":"2023-11-08T19:25:49.854746+0000","last_active":"2023-11-08T19:25:49.854746+0000","last_peered":"2023-11-08T19:25:49.854746+0000","last_clean":"2023-11-08T19:25:49.854746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:25:49.854746+0000","last_undegraded":"2023-11-08T19:25:49.854746+0000","last_fullsized":"2023-11-08T19:25:49.854746+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:25:49.854746+0000","last_change":"2023-11-08T19:25:49.854746+0000","last_active":"2023-11-08T19:25:49.854746+0000","last_peered":"2023-11-08T19:25:49.854746+0000","last_clean":"2023-11-08T19:25:49.854746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:25:49.854746+0000","last_undegraded":"2023-11-08T19:25:49.854746+0000","last_fullsized":"2023-11-08T19:25:49.854746+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:25:49.854746+0000","last_change":"2023-11-08T19:25:49.854746+0000","last_active":"2023-11-08T19:25:49.854746+0000","last_peered":"2023-11-08T19:25:49.854746+0000","last_clean":"2023-11-08T19:25:49.854746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:25:49.854746+0000","last_undegraded":"2023-11-08T19:25:49.854746+0000","last_fullsized":"2023-11-08T19:25:49.854746+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.989618+0000","last_change":"2023-11-08T19:25:50.889063+0000","last_active":"2023-11-08T19:25:51.989618+0000","last_peered":"2023-11-08T19:25:51.989618+0000","last_clean":"2023-11-08T19:25:51.989618+0000","last_became_active":"2023-11-08T19:25:50.888760+0000","last_became_peered":"2023-11-08T19:25:50.888760+0000","last_unstale":"2023-11-08T19:25:51.989618+0000","last_undegraded":"2023-11-08T19:25:51.989618+0000","last_fullsized":"2023-11-08T19:25:51.989618+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:37:10.264673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-11-08T19:25:48.857285+0000","last_change":"2023-11-08T19:25:48.857285+0000","last_active":"2023-11-08T19:25:48.844481+0000","last_peered":"2023-11-08T19:25:48.844481+0000","last_clean":"2023-11-08T19:25:48.844481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:25:48.857285+0000","last_undegraded":"2023-11-08T19:25:48.857285+0000","last_fullsized":"2023-11-08T19:25:48.857285+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:25:49.854746+0000","last_change":"2023-11-08T19:25:49.854746+0000","last_active":"2023-11-08T19:25:49.854746+0000","last_peered":"2023-11-08T19:25:49.854746+0000","last_clean":"2023-11-08T19:25:49.854746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:25:49.854746+0000","last_undegraded":"2023-11-08T19:25:49.854746+0000","last_fullsized":"2023-11-08T19:25:49.854746+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1e","version":"32'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.989742+0000","last_change":"2023-11-08T19:25:49.879837+0000","last_active":"2023-11-08T19:25:51.989742+0000","last_peered":"2023-11-08T19:25:51.989742+0000","last_clean":"2023-11-08T19:25:51.989742+0000","last_became_active":"2023-11-08T19:25:49.879570+0000","last_became_peered":"2023-11-08T19:25:49.879570+0000","last_unstale":"2023-11-08T19:25:51.989742+0000","last_undegraded":"2023-11-08T19:25:51.989742+0000","last_fullsized":"2023-11-08T19:25:51.989742+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:00:52.169251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.205335+0000","last_change":"2023-11-08T19:25:50.881012+0000","last_active":"2023-11-08T19:25:52.205335+0000","last_peered":"2023-11-08T19:25:52.205335+0000","last_clean":"2023-11-08T19:25:52.205335+0000","last_became_active":"2023-11-08T19:25:50.880752+0000","last_became_peered":"2023-11-08T19:25:50.880752+0000","last_unstale":"2023-11-08T19:25:52.205335+0000","last_undegraded":"2023-11-08T19:25:52.205335+0000","last_fullsized":"2023-11-08T19:25:52.205335+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:33:12.567239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"32'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.203995+0000","last_change":"2023-11-08T19:25:49.886087+0000","last_active":"2023-11-08T19:25:52.203995+0000","last_peered":"2023-11-08T19:25:52.203995+0000","last_clean":"2023-11-08T19:25:52.203995+0000","last_became_active":"2023-11-08T19:25:49.885801+0000","last_became_peered":"2023-11-08T19:25:49.885801+0000","last_unstale":"2023-11-08T19:25:52.203995+0000","last_undegraded":"2023-11-08T19:25:52.203995+0000","last_fullsized":"2023-11-08T19:25:52.203995+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:04:00.482724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.992009+0000","last_change":"2023-11-08T19:25:50.885648+0000","last_active":"2023-11-08T19:25:51.992009+0000","last_peered":"2023-11-08T19:25:51.992009+0000","last_clean":"2023-11-08T19:25:51.992009+0000","last_became_active":"2023-11-08T19:25:50.885179+0000","last_became_peered":"2023-11-08T19:25:50.885179+0000","last_unstale":"2023-11-08T19:25:51.992009+0000","last_undegraded":"2023-11-08T19:25:51.992009+0000","last_fullsized":"2023-11-08T19:25:51.992009+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:52:20.334760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"32'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.876725+0000","last_change":"2023-11-08T19:25:49.883073+0000","last_active":"2023-11-08T19:25:51.876725+0000","last_peered":"2023-11-08T19:25:51.876725+0000","last_clean":"2023-11-08T19:25:51.876725+0000","last_became_active":"2023-11-08T19:25:49.882522+0000","last_became_peered":"2023-11-08T19:25:49.882522+0000","last_unstale":"2023-11-08T19:25:51.876725+0000","last_undegraded":"2023-11-08T19:25:51.876725+0000","last_fullsized":"2023-11-08T19:25:51.876725+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:27:43.830493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:25:49.854746+0000","last_change":"2023-11-08T19:25:49.854746+0000","last_active":"2023-11-08T19:25:49.854746+0000","last_peered":"2023-11-08T19:25:49.854746+0000","last_clean":"2023-11-08T19:25:49.854746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:25:49.854746+0000","last_undegraded":"2023-11-08T19:25:49.854746+0000","last_fullsized":"2023-11-08T19:25:49.854746+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1b","version":"32'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.205020+0000","last_change":"2023-11-08T19:25:49.884628+0000","last_active":"2023-11-08T19:25:52.205020+0000","last_peered":"2023-11-08T19:25:52.205020+0000","last_clean":"2023-11-08T19:25:52.205020+0000","last_became_active":"2023-11-08T19:25:49.884384+0000","last_became_peered":"2023-11-08T19:25:49.884384+0000","last_unstale":"2023-11-08T19:25:52.205020+0000","last_undegraded":"2023-11-08T19:25:52.205020+0000","last_fullsized":"2023-11-08T19:25:52.205020+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:17:52.641253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204834+0000","last_change":"2023-11-08T19:25:50.874965+0000","last_active":"2023-11-08T19:25:52.204834+0000","last_peered":"2023-11-08T19:25:52.204834+0000","last_clean":"2023-11-08T19:25:52.204834+0000","last_became_active":"2023-11-08T19:25:50.874696+0000","last_became_peered":"2023-11-08T19:25:50.874696+0000","last_unstale":"2023-11-08T19:25:52.204834+0000","last_undegraded":"2023-11-08T19:25:52.204834+0000","last_fullsized":"2023-11-08T19:25:52.204834+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:49:41.475597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"32'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204932+0000","last_change":"2023-11-08T19:25:49.884607+0000","last_active":"2023-11-08T19:25:52.204932+0000","last_peered":"2023-11-08T19:25:52.204932+0000","last_clean":"2023-11-08T19:25:52.204932+0000","last_became_active":"2023-11-08T19:25:49.884392+0000","last_became_peered":"2023-11-08T19:25:49.884392+0000","last_unstale":"2023-11-08T19:25:52.204932+0000","last_undegraded":"2023-11-08T19:25:52.204932+0000","last_fullsized":"2023-11-08T19:25:52.204932+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:54:54.192037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204370+0000","last_change":"2023-11-08T19:25:50.881747+0000","last_active":"2023-11-08T19:25:52.204370+0000","last_peered":"2023-11-08T19:25:52.204370+0000","last_clean":"2023-11-08T19:25:52.204370+0000","last_became_active":"2023-11-08T19:25:50.881225+0000","last_became_peered":"2023-11-08T19:25:50.881225+0000","last_unstale":"2023-11-08T19:25:52.204370+0000","last_undegraded":"2023-11-08T19:25:52.204370+0000","last_fullsized":"2023-11-08T19:25:52.204370+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:37:10.264673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.203749+0000","last_change":"2023-11-08T19:25:49.882231+0000","last_active":"2023-11-08T19:25:52.203749+0000","last_peered":"2023-11-08T19:25:52.203749+0000","last_clean":"2023-11-08T19:25:52.203749+0000","last_became_active":"2023-11-08T19:25:49.881992+0000","last_became_peered":"2023-11-08T19:25:49.881992+0000","last_unstale":"2023-11-08T19:25:52.203749+0000","last_undegraded":"2023-11-08T19:25:52.203749+0000","last_fullsized":"2023-11-08T19:25:52.203749+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:00:52.169251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.991649+0000","last_change":"2023-11-08T19:25:50.883482+0000","last_active":"2023-11-08T19:25:51.991649+0000","last_peered":"2023-11-08T19:25:51.991649+0000","last_clean":"2023-11-08T19:25:51.991649+0000","last_became_active":"2023-11-08T19:25:50.883187+0000","last_became_peered":"2023-11-08T19:25:50.883187+0000","last_unstale":"2023-11-08T19:25:51.991649+0000","last_undegraded":"2023-11-08T19:25:51.991649+0000","last_fullsized":"2023-11-08T19:25:51.991649+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:33:12.567239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204890+0000","last_change":"2023-11-08T19:25:49.878388+0000","last_active":"2023-11-08T19:25:52.204890+0000","last_peered":"2023-11-08T19:25:52.204890+0000","last_clean":"2023-11-08T19:25:52.204890+0000","last_became_active":"2023-11-08T19:25:49.878111+0000","last_became_peered":"2023-11-08T19:25:49.878111+0000","last_unstale":"2023-11-08T19:25:52.204890+0000","last_undegraded":"2023-11-08T19:25:52.204890+0000","last_fullsized":"2023-11-08T19:25:52.204890+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:39:22.428409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204894+0000","last_change":"2023-11-08T19:25:50.877613+0000","last_active":"2023-11-08T19:25:52.204894+0000","last_peered":"2023-11-08T19:25:52.204894+0000","last_clean":"2023-11-08T19:25:52.204894+0000","last_became_active":"2023-11-08T19:25:50.877311+0000","last_became_peered":"2023-11-08T19:25:50.877311+0000","last_unstale":"2023-11-08T19:25:52.204894+0000","last_undegraded":"2023-11-08T19:25:52.204894+0000","last_fullsized":"2023-11-08T19:25:52.204894+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:50:52.902620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204190+0000","last_change":"2023-11-08T19:25:49.884355+0000","last_active":"2023-11-08T19:25:52.204190+0000","last_peered":"2023-11-08T19:25:52.204190+0000","last_clean":"2023-11-08T19:25:52.204190+0000","last_became_active":"2023-11-08T19:25:49.884065+0000","last_became_peered":"2023-11-08T19:25:49.884065+0000","last_unstale":"2023-11-08T19:25:52.204190+0000","last_undegraded":"2023-11-08T19:25:52.204190+0000","last_fullsized":"2023-11-08T19:25:52.204190+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:05:32.217545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204103+0000","last_change":"2023-11-08T19:25:50.888556+0000","last_active":"2023-11-08T19:25:52.204103+0000","last_peered":"2023-11-08T19:25:52.204103+0000","last_clean":"2023-11-08T19:25:52.204103+0000","last_became_active":"2023-11-08T19:25:50.887909+0000","last_became_peered":"2023-11-08T19:25:50.887909+0000","last_unstale":"2023-11-08T19:25:52.204103+0000","last_undegraded":"2023-11-08T19:25:52.204103+0000","last_fullsized":"2023-11-08T19:25:52.204103+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:38:37.341013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.989663+0000","last_change":"2023-11-08T19:25:49.884919+0000","last_active":"2023-11-08T19:25:51.989663+0000","last_peered":"2023-11-08T19:25:51.989663+0000","last_clean":"2023-11-08T19:25:51.989663+0000","last_became_active":"2023-11-08T19:25:49.884679+0000","last_became_peered":"2023-11-08T19:25:49.884679+0000","last_unstale":"2023-11-08T19:25:51.989663+0000","last_undegraded":"2023-11-08T19:25:51.989663+0000","last_fullsized":"2023-11-08T19:25:51.989663+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:39:22.428409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204236+0000","last_change":"2023-11-08T19:25:50.882028+0000","last_active":"2023-11-08T19:25:52.204236+0000","last_peered":"2023-11-08T19:25:52.204236+0000","last_clean":"2023-11-08T19:25:52.204236+0000","last_became_active":"2023-11-08T19:25:50.881746+0000","last_became_peered":"2023-11-08T19:25:50.881746+0000","last_unstale":"2023-11-08T19:25:52.204236+0000","last_undegraded":"2023-11-08T19:25:52.204236+0000","last_fullsized":"2023-11-08T19:25:52.204236+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:39:57.441098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"32'3","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204967+0000","last_change":"2023-11-08T19:25:49.878409+0000","last_active":"2023-11-08T19:25:52.204967+0000","last_peered":"2023-11-08T19:25:52.204967+0000","last_clean":"2023-11-08T19:25:52.204967+0000","last_became_active":"2023-11-08T19:25:49.878105+0000","last_became_peered":"2023-11-08T19:25:49.878105+0000","last_unstale":"2023-11-08T19:25:52.204967+0000","last_undegraded":"2023-11-08T19:25:52.204967+0000","last_fullsized":"2023-11-08T19:25:52.204967+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:55:30.168605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204866+0000","last_change":"2023-11-08T19:25:50.887174+0000","last_active":"2023-11-08T19:25:52.204866+0000","last_peered":"2023-11-08T19:25:52.204866+0000","last_clean":"2023-11-08T19:25:52.204866+0000","last_became_active":"2023-11-08T19:25:50.886902+0000","last_became_peered":"2023-11-08T19:25:50.886902+0000","last_unstale":"2023-11-08T19:25:52.204866+0000","last_undegraded":"2023-11-08T19:25:52.204866+0000","last_fullsized":"2023-11-08T19:25:52.204866+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:19:42.562982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"32'6","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204091+0000","last_change":"2023-11-08T19:25:49.886499+0000","last_active":"2023-11-08T19:25:52.204091+0000","last_peered":"2023-11-08T19:25:52.204091+0000","last_clean":"2023-11-08T19:25:52.204091+0000","last_became_active":"2023-11-08T19:25:49.886169+0000","last_became_peered":"2023-11-08T19:25:49.886169+0000","last_unstale":"2023-11-08T19:25:52.204091+0000","last_undegraded":"2023-11-08T19:25:52.204091+0000","last_fullsized":"2023-11-08T19:25:52.204091+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:19:41.552717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.991670+0000","last_change":"2023-11-08T19:25:50.885893+0000","last_active":"2023-11-08T19:25:51.991670+0000","last_peered":"2023-11-08T19:25:51.991670+0000","last_clean":"2023-11-08T19:25:51.991670+0000","last_became_active":"2023-11-08T19:25:50.880205+0000","last_became_peered":"2023-11-08T19:25:50.880205+0000","last_unstale":"2023-11-08T19:25:51.991670+0000","last_undegraded":"2023-11-08T19:25:51.991670+0000","last_fullsized":"2023-11-08T19:25:51.991670+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:55:31.178870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"32'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.205247+0000","last_change":"2023-11-08T19:25:49.879256+0000","last_active":"2023-11-08T19:25:52.205247+0000","last_peered":"2023-11-08T19:25:52.205247+0000","last_clean":"2023-11-08T19:25:52.205247+0000","last_became_active":"2023-11-08T19:25:49.878963+0000","last_became_peered":"2023-11-08T19:25:49.878963+0000","last_unstale":"2023-11-08T19:25:52.205247+0000","last_undegraded":"2023-11-08T19:25:52.205247+0000","last_fullsized":"2023-11-08T19:25:52.205247+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:12:39.742779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.203953+0000","last_change":"2023-11-08T19:25:50.891183+0000","last_active":"2023-11-08T19:25:52.203953+0000","last_peered":"2023-11-08T19:25:52.203953+0000","last_clean":"2023-11-08T19:25:52.203953+0000","last_became_active":"2023-11-08T19:25:50.890889+0000","last_became_peered":"2023-11-08T19:25:50.890889+0000","last_unstale":"2023-11-08T19:25:52.203953+0000","last_undegraded":"2023-11-08T19:25:52.203953+0000","last_fullsized":"2023-11-08T19:25:52.203953+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:25:16.223226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.991756+0000","last_change":"2023-11-08T19:25:49.880102+0000","last_active":"2023-11-08T19:25:51.991756+0000","last_peered":"2023-11-08T19:25:51.991756+0000","last_clean":"2023-11-08T19:25:51.991756+0000","last_became_active":"2023-11-08T19:25:49.879816+0000","last_became_peered":"2023-11-08T19:25:49.879816+0000","last_unstale":"2023-11-08T19:25:51.991756+0000","last_undegraded":"2023-11-08T19:25:51.991756+0000","last_fullsized":"2023-11-08T19:25:51.991756+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:37:09.254408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:25:49.854746+0000","last_change":"2023-11-08T19:25:49.854746+0000","last_active":"2023-11-08T19:25:49.854746+0000","last_peered":"2023-11-08T19:25:49.854746+0000","last_clean":"2023-11-08T19:25:49.854746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:25:49.854746+0000","last_undegraded":"2023-11-08T19:25:49.854746+0000","last_fullsized":"2023-11-08T19:25:49.854746+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.203841+0000","last_change":"2023-11-08T19:25:49.884359+0000","last_active":"2023-11-08T19:25:52.203841+0000","last_peered":"2023-11-08T19:25:52.203841+0000","last_clean":"2023-11-08T19:25:52.203841+0000","last_became_active":"2023-11-08T19:25:49.884119+0000","last_became_peered":"2023-11-08T19:25:49.884119+0000","last_unstale":"2023-11-08T19:25:52.203841+0000","last_undegraded":"2023-11-08T19:25:52.203841+0000","last_fullsized":"2023-11-08T19:25:52.203841+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:55:30.168605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:25:49.854746+0000","last_change":"2023-11-08T19:25:49.854746+0000","last_active":"2023-11-08T19:25:49.854746+0000","last_peered":"2023-11-08T19:25:49.854746+0000","last_clean":"2023-11-08T19:25:49.854746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:25:49.854746+0000","last_undegraded":"2023-11-08T19:25:49.854746+0000","last_fullsized":"2023-11-08T19:25:49.854746+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.10","version":"32'8","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.203707+0000","last_change":"2023-11-08T19:25:49.886483+0000","last_active":"2023-11-08T19:25:52.203707+0000","last_peered":"2023-11-08T19:25:52.203707+0000","last_clean":"2023-11-08T19:25:52.203707+0000","last_became_active":"2023-11-08T19:25:49.886209+0000","last_became_peered":"2023-11-08T19:25:49.886209+0000","last_unstale":"2023-11-08T19:25:52.203707+0000","last_undegraded":"2023-11-08T19:25:52.203707+0000","last_fullsized":"2023-11-08T19:25:52.203707+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:49:40.465332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.205492+0000","last_change":"2023-11-08T19:25:50.884739+0000","last_active":"2023-11-08T19:25:52.205492+0000","last_peered":"2023-11-08T19:25:52.205492+0000","last_clean":"2023-11-08T19:25:52.205492+0000","last_became_active":"2023-11-08T19:25:50.884476+0000","last_became_peered":"2023-11-08T19:25:50.884476+0000","last_unstale":"2023-11-08T19:25:52.205492+0000","last_undegraded":"2023-11-08T19:25:52.205492+0000","last_fullsized":"2023-11-08T19:25:52.205492+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:55:31.178870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"32'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.992046+0000","last_change":"2023-11-08T19:25:49.884770+0000","last_active":"2023-11-08T19:25:51.992046+0000","last_peered":"2023-11-08T19:25:51.992046+0000","last_clean":"2023-11-08T19:25:51.992046+0000","last_became_active":"2023-11-08T19:25:49.884476+0000","last_became_peered":"2023-11-08T19:25:49.884476+0000","last_unstale":"2023-11-08T19:25:51.992046+0000","last_undegraded":"2023-11-08T19:25:51.992046+0000","last_fullsized":"2023-11-08T19:25:51.992046+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:18:56.855441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204945+0000","last_change":"2023-11-08T19:25:50.877298+0000","last_active":"2023-11-08T19:25:52.204945+0000","last_peered":"2023-11-08T19:25:52.204945+0000","last_clean":"2023-11-08T19:25:52.204945+0000","last_became_active":"2023-11-08T19:25:50.877018+0000","last_became_peered":"2023-11-08T19:25:50.877018+0000","last_unstale":"2023-11-08T19:25:52.204945+0000","last_undegraded":"2023-11-08T19:25:52.204945+0000","last_fullsized":"2023-11-08T19:25:52.204945+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:25:07.133280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"32'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.205477+0000","last_change":"2023-11-08T19:25:49.878520+0000","last_active":"2023-11-08T19:25:52.205477+0000","last_peered":"2023-11-08T19:25:52.205477+0000","last_clean":"2023-11-08T19:25:52.205477+0000","last_became_active":"2023-11-08T19:25:49.878244+0000","last_became_peered":"2023-11-08T19:25:49.878244+0000","last_unstale":"2023-11-08T19:25:52.205477+0000","last_undegraded":"2023-11-08T19:25:52.205477+0000","last_fullsized":"2023-11-08T19:25:52.205477+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:23:30.383150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.991899+0000","last_change":"2023-11-08T19:25:50.878056+0000","last_active":"2023-11-08T19:25:51.991899+0000","last_peered":"2023-11-08T19:25:51.991899+0000","last_clean":"2023-11-08T19:25:51.991899+0000","last_became_active":"2023-11-08T19:25:50.877738+0000","last_became_peered":"2023-11-08T19:25:50.877738+0000","last_unstale":"2023-11-08T19:25:51.991899+0000","last_undegraded":"2023-11-08T19:25:51.991899+0000","last_fullsized":"2023-11-08T19:25:51.991899+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:04:01.492989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"32'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.205456+0000","last_change":"2023-11-08T19:25:49.881744+0000","last_active":"2023-11-08T19:25:52.205456+0000","last_peered":"2023-11-08T19:25:52.205456+0000","last_clean":"2023-11-08T19:25:52.205456+0000","last_became_active":"2023-11-08T19:25:49.881436+0000","last_became_peered":"2023-11-08T19:25:49.881436+0000","last_unstale":"2023-11-08T19:25:52.205456+0000","last_undegraded":"2023-11-08T19:25:52.205456+0000","last_fullsized":"2023-11-08T19:25:52.205456+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:08:07.992929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.876822+0000","last_change":"2023-11-08T19:25:50.881348+0000","last_active":"2023-11-08T19:25:51.876822+0000","last_peered":"2023-11-08T19:25:51.876822+0000","last_clean":"2023-11-08T19:25:51.876822+0000","last_became_active":"2023-11-08T19:25:50.881064+0000","last_became_peered":"2023-11-08T19:25:50.881064+0000","last_unstale":"2023-11-08T19:25:51.876822+0000","last_undegraded":"2023-11-08T19:25:51.876822+0000","last_fullsized":"2023-11-08T19:25:51.876822+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:23:31.393415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.205574+0000","last_change":"2023-11-08T19:25:49.884654+0000","last_active":"2023-11-08T19:25:52.205574+0000","last_peered":"2023-11-08T19:25:52.205574+0000","last_clean":"2023-11-08T19:25:52.205574+0000","last_became_active":"2023-11-08T19:25:49.884322+0000","last_became_peered":"2023-11-08T19:25:49.884322+0000","last_unstale":"2023-11-08T19:25:52.205574+0000","last_undegraded":"2023-11-08T19:25:52.205574+0000","last_fullsized":"2023-11-08T19:25:52.205574+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:49:07.738959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.876689+0000","last_change":"2023-11-08T19:25:50.878414+0000","last_active":"2023-11-08T19:25:51.876689+0000","last_peered":"2023-11-08T19:25:51.876689+0000","last_clean":"2023-11-08T19:25:51.876689+0000","last_became_active":"2023-11-08T19:25:50.878145+0000","last_became_peered":"2023-11-08T19:25:50.878145+0000","last_unstale":"2023-11-08T19:25:51.876689+0000","last_undegraded":"2023-11-08T19:25:51.876689+0000","last_fullsized":"2023-11-08T19:25:51.876689+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:49:08.749224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"32'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.989615+0000","last_change":"2023-11-08T19:25:49.878923+0000","last_active":"2023-11-08T19:25:51.989615+0000","last_peered":"2023-11-08T19:25:51.989615+0000","last_clean":"2023-11-08T19:25:51.989615+0000","last_became_active":"2023-11-08T19:25:49.878167+0000","last_became_peered":"2023-11-08T19:25:49.878167+0000","last_unstale":"2023-11-08T19:25:51.989615+0000","last_undegraded":"2023-11-08T19:25:51.989615+0000","last_fullsized":"2023-11-08T19:25:51.989615+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:51:26.807463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.876711+0000","last_change":"2023-11-08T19:25:50.887699+0000","last_active":"2023-11-08T19:25:51.876711+0000","last_peered":"2023-11-08T19:25:51.876711+0000","last_clean":"2023-11-08T19:25:51.876711+0000","last_became_active":"2023-11-08T19:25:50.887423+0000","last_became_peered":"2023-11-08T19:25:50.887423+0000","last_unstale":"2023-11-08T19:25:51.876711+0000","last_undegraded":"2023-11-08T19:25:51.876711+0000","last_fullsized":"2023-11-08T19:25:51.876711+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:08:09.003194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-11-08T19:25:48.857323+0000","last_change":"2023-11-08T19:25:48.857323+0000","last_active":"2023-11-08T19:25:48.844481+0000","last_peered":"2023-11-08T19:25:48.844481+0000","last_clean":"2023-11-08T19:25:48.844481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:25:48.857323+0000","last_undegraded":"2023-11-08T19:25:48.857323+0000","last_fullsized":"2023-11-08T19:25:48.857323+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.876619+0000","last_change":"2023-11-08T19:25:50.888597+0000","last_active":"2023-11-08T19:25:51.876619+0000","last_peered":"2023-11-08T19:25:51.876619+0000","last_clean":"2023-11-08T19:25:51.876619+0000","last_became_active":"2023-11-08T19:25:50.887065+0000","last_became_peered":"2023-11-08T19:25:50.887065+0000","last_unstale":"2023-11-08T19:25:51.876619+0000","last_undegraded":"2023-11-08T19:25:51.876619+0000","last_fullsized":"2023-11-08T19:25:51.876619+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:02:25.988939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.205088+0000","last_change":"2023-11-08T19:25:49.884053+0000","last_active":"2023-11-08T19:25:52.205088+0000","last_peered":"2023-11-08T19:25:52.205088+0000","last_clean":"2023-11-08T19:25:52.205088+0000","last_became_active":"2023-11-08T19:25:49.883822+0000","last_became_peered":"2023-11-08T19:25:49.883822+0000","last_unstale":"2023-11-08T19:25:52.205088+0000","last_undegraded":"2023-11-08T19:25:52.205088+0000","last_fullsized":"2023-11-08T19:25:52.205088+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:12:47.116061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:25:49.854746+0000","last_change":"2023-11-08T19:25:49.854746+0000","last_active":"2023-11-08T19:25:49.854746+0000","last_peered":"2023-11-08T19:25:49.854746+0000","last_clean":"2023-11-08T19:25:49.854746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:25:49.854746+0000","last_undegraded":"2023-11-08T19:25:49.854746+0000","last_fullsized":"2023-11-08T19:25:49.854746+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204303+0000","last_change":"2023-11-08T19:25:49.879327+0000","last_active":"2023-11-08T19:25:52.204303+0000","last_peered":"2023-11-08T19:25:52.204303+0000","last_clean":"2023-11-08T19:25:52.204303+0000","last_became_active":"2023-11-08T19:25:49.879054+0000","last_became_peered":"2023-11-08T19:25:49.879054+0000","last_unstale":"2023-11-08T19:25:52.204303+0000","last_undegraded":"2023-11-08T19:25:52.204303+0000","last_fullsized":"2023-11-08T19:25:52.204303+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:09:04.368768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.876752+0000","last_change":"2023-11-08T19:25:50.874836+0000","last_active":"2023-11-08T19:25:51.876752+0000","last_peered":"2023-11-08T19:25:51.876752+0000","last_clean":"2023-11-08T19:25:51.876752+0000","last_became_active":"2023-11-08T19:25:50.874272+0000","last_became_peered":"2023-11-08T19:25:50.874272+0000","last_unstale":"2023-11-08T19:25:51.876752+0000","last_undegraded":"2023-11-08T19:25:51.876752+0000","last_fullsized":"2023-11-08T19:25:51.876752+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:17:53.651518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"32'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.991956+0000","last_change":"2023-11-08T19:25:49.880740+0000","last_active":"2023-11-08T19:25:51.991956+0000","last_peered":"2023-11-08T19:25:51.991956+0000","last_clean":"2023-11-08T19:25:51.991956+0000","last_became_active":"2023-11-08T19:25:49.880458+0000","last_became_peered":"2023-11-08T19:25:49.880458+0000","last_unstale":"2023-11-08T19:25:51.991956+0000","last_undegraded":"2023-11-08T19:25:51.991956+0000","last_fullsized":"2023-11-08T19:25:51.991956+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:20:42.164809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.205146+0000","last_change":"2023-11-08T19:25:50.883014+0000","last_active":"2023-11-08T19:25:52.205146+0000","last_peered":"2023-11-08T19:25:52.205146+0000","last_clean":"2023-11-08T19:25:52.205146+0000","last_became_active":"2023-11-08T19:25:50.882737+0000","last_became_peered":"2023-11-08T19:25:50.882737+0000","last_unstale":"2023-11-08T19:25:52.205146+0000","last_undegraded":"2023-11-08T19:25:52.205146+0000","last_fullsized":"2023-11-08T19:25:52.205146+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:38:37.341013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"33'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.991098+0000","last_change":"2023-11-08T19:25:49.880299+0000","last_active":"2023-11-08T19:25:51.991098+0000","last_peered":"2023-11-08T19:25:51.991098+0000","last_clean":"2023-11-08T19:25:51.991098+0000","last_became_active":"2023-11-08T19:25:49.878038+0000","last_became_peered":"2023-11-08T19:25:49.878038+0000","last_unstale":"2023-11-08T19:25:51.991098+0000","last_undegraded":"2023-11-08T19:25:51.991098+0000","last_fullsized":"2023-11-08T19:25:51.991098+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:12:39.742779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.989626+0000","last_change":"2023-11-08T19:25:50.888755+0000","last_active":"2023-11-08T19:25:51.989626+0000","last_peered":"2023-11-08T19:25:51.989626+0000","last_clean":"2023-11-08T19:25:51.989626+0000","last_became_active":"2023-11-08T19:25:50.888494+0000","last_became_peered":"2023-11-08T19:25:50.888494+0000","last_unstale":"2023-11-08T19:25:51.989626+0000","last_undegraded":"2023-11-08T19:25:51.989626+0000","last_fullsized":"2023-11-08T19:25:51.989626+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:09:05.379033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.991905+0000","last_change":"2023-11-08T19:25:49.881699+0000","last_active":"2023-11-08T19:25:51.991905+0000","last_peered":"2023-11-08T19:25:51.991905+0000","last_clean":"2023-11-08T19:25:51.991905+0000","last_became_active":"2023-11-08T19:25:49.881062+0000","last_became_peered":"2023-11-08T19:25:49.881062+0000","last_unstale":"2023-11-08T19:25:51.991905+0000","last_undegraded":"2023-11-08T19:25:51.991905+0000","last_fullsized":"2023-11-08T19:25:51.991905+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:49:44.049659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:25:49.854746+0000","last_change":"2023-11-08T19:25:49.854746+0000","last_active":"2023-11-08T19:25:49.854746+0000","last_peered":"2023-11-08T19:25:49.854746+0000","last_clean":"2023-11-08T19:25:49.854746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:25:49.854746+0000","last_undegraded":"2023-11-08T19:25:49.854746+0000","last_fullsized":"2023-11-08T19:25:49.854746+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.989575+0000","last_change":"2023-11-08T19:25:49.879113+0000","last_active":"2023-11-08T19:25:51.989575+0000","last_peered":"2023-11-08T19:25:51.989575+0000","last_clean":"2023-11-08T19:25:51.989575+0000","last_became_active":"2023-11-08T19:25:49.878579+0000","last_became_peered":"2023-11-08T19:25:49.878579+0000","last_unstale":"2023-11-08T19:25:51.989575+0000","last_undegraded":"2023-11-08T19:25:51.989575+0000","last_fullsized":"2023-11-08T19:25:51.989575+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:12:47.116061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.989707+0000","last_change":"2023-11-08T19:25:50.880600+0000","last_active":"2023-11-08T19:25:51.989707+0000","last_peered":"2023-11-08T19:25:51.989707+0000","last_clean":"2023-11-08T19:25:51.989707+0000","last_became_active":"2023-11-08T19:25:50.880384+0000","last_became_peered":"2023-11-08T19:25:50.880384+0000","last_unstale":"2023-11-08T19:25:51.989707+0000","last_undegraded":"2023-11-08T19:25:51.989707+0000","last_fullsized":"2023-11-08T19:25:51.989707+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:20:43.175074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.991693+0000","last_change":"2023-11-08T19:25:49.877196+0000","last_active":"2023-11-08T19:25:51.991693+0000","last_peered":"2023-11-08T19:25:51.991693+0000","last_clean":"2023-11-08T19:25:51.991693+0000","last_became_active":"2023-11-08T19:25:49.876517+0000","last_became_peered":"2023-11-08T19:25:49.876517+0000","last_unstale":"2023-11-08T19:25:51.991693+0000","last_undegraded":"2023-11-08T19:25:51.991693+0000","last_fullsized":"2023-11-08T19:25:51.991693+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:00:52.169251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.203760+0000","last_change":"2023-11-08T19:25:50.893157+0000","last_active":"2023-11-08T19:25:52.203760+0000","last_peered":"2023-11-08T19:25:52.203760+0000","last_clean":"2023-11-08T19:25:52.203760+0000","last_became_active":"2023-11-08T19:25:50.892637+0000","last_became_peered":"2023-11-08T19:25:50.892637+0000","last_unstale":"2023-11-08T19:25:52.203760+0000","last_undegraded":"2023-11-08T19:25:52.203760+0000","last_fullsized":"2023-11-08T19:25:52.203760+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:50:52.902620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.205077+0000","last_change":"2023-11-08T19:25:49.881923+0000","last_active":"2023-11-08T19:25:52.205077+0000","last_peered":"2023-11-08T19:25:52.205077+0000","last_clean":"2023-11-08T19:25:52.205077+0000","last_became_active":"2023-11-08T19:25:49.881670+0000","last_became_peered":"2023-11-08T19:25:49.881670+0000","last_unstale":"2023-11-08T19:25:52.205077+0000","last_undegraded":"2023-11-08T19:25:52.205077+0000","last_fullsized":"2023-11-08T19:25:52.205077+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:51:26.807463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204772+0000","last_change":"2023-11-08T19:25:50.883910+0000","last_active":"2023-11-08T19:25:52.204772+0000","last_peered":"2023-11-08T19:25:52.204772+0000","last_clean":"2023-11-08T19:25:52.204772+0000","last_became_active":"2023-11-08T19:25:50.883615+0000","last_became_peered":"2023-11-08T19:25:50.883615+0000","last_unstale":"2023-11-08T19:25:52.204772+0000","last_undegraded":"2023-11-08T19:25:52.204772+0000","last_fullsized":"2023-11-08T19:25:52.204772+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:20:43.175074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.203857+0000","last_change":"2023-11-08T19:25:49.884441+0000","last_active":"2023-11-08T19:25:52.203857+0000","last_peered":"2023-11-08T19:25:52.203857+0000","last_clean":"2023-11-08T19:25:52.203857+0000","last_became_active":"2023-11-08T19:25:49.884125+0000","last_became_peered":"2023-11-08T19:25:49.884125+0000","last_unstale":"2023-11-08T19:25:52.203857+0000","last_undegraded":"2023-11-08T19:25:52.203857+0000","last_fullsized":"2023-11-08T19:25:52.203857+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:52:19.324495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:25:49.854746+0000","last_change":"2023-11-08T19:25:49.854746+0000","last_active":"2023-11-08T19:25:49.854746+0000","last_peered":"2023-11-08T19:25:49.854746+0000","last_clean":"2023-11-08T19:25:49.854746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:25:49.854746+0000","last_undegraded":"2023-11-08T19:25:49.854746+0000","last_fullsized":"2023-11-08T19:25:49.854746+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-11-08T19:25:48.857272+0000","last_change":"2023-11-08T19:25:48.857272+0000","last_active":"2023-11-08T19:25:48.844481+0000","last_peered":"2023-11-08T19:25:48.844481+0000","last_clean":"2023-11-08T19:25:48.844481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:25:48.857272+0000","last_undegraded":"2023-11-08T19:25:48.857272+0000","last_fullsized":"2023-11-08T19:25:48.857272+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":175,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-11-08T19:25:43.589068+0000","last_change":"2023-11-08T19:24:45.888122+0000","last_active":"2023-11-08T19:25:43.589068+0000","last_peered":"2023-11-08T19:25:43.589068+0000","last_clean":"2023-11-08T19:25:43.589068+0000","last_became_active":"2023-11-08T19:24:45.887048+0000","last_became_peered":"2023-11-08T19:24:45.887048+0000","last_unstale":"2023-11-08T19:25:43.589068+0000","last_undegraded":"2023-11-08T19:25:43.589068+0000","last_fullsized":"2023-11-08T19:25:43.589068+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:44.575133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:44.575133+0000","last_clean_scrub_stamp":"2023-11-08T19:24:44.575133+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:19:20.202062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":96,"acting":96,"num_store_stats":6},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":1828,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":4114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":35,"ondisk_log_size":35,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574865,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27928,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343912,"statfs":{"total":96636764160,"available":96608165888,"internally_reserved":0,"allocated":929792,"data_stored":716892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.60999999999999999}]},{"osd":1,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.76100000000000001}]},{"osd":2,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.63200000000000001}]},{"osd":3,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.746},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.67800000000000005}]},{"osd":4,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.58299999999999996}]},{"osd":5,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.71999999999999997}]},{"osd":6,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.64400000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":335872,"data_stored":126524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":21.359999999999999,"5min":21.359999999999999,"15min":21.359999999999999},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":10.329000000000001,"5min":10.329000000000001,"15min":10.329000000000001},"last":0.46700000000000003}]},{"osd":1,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":3.4239999999999999,"5min":3.4239999999999999,"15min":3.4239999999999999},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":3.4809999999999999,"5min":3.4809999999999999,"15min":3.4809999999999999},"last":0.66500000000000004}]},{"osd":2,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":10.398,"5min":10.398,"15min":10.398},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":5.0170000000000003,"5min":5.0170000000000003,"15min":5.0170000000000003},"last":0.60999999999999999}]},{"osd":3,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":3.5710000000000002,"5min":3.5710000000000002,"15min":3.5710000000000002},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":10.368,"5min":10.368,"15min":10.368},"last":0.70299999999999996}]},{"osd":4,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.78500000000000003}]},{"osd":5,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.753},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.82099999999999995}]},{"osd":7,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.80700000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574866,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27404,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344436,"statfs":{"total":96636764160,"available":96608702464,"internally_reserved":0,"allocated":389120,"data_stored":147500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":1.389,"5min":1.389,"15min":1.389},"last":0.499},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":9.1370000000000005,"5min":9.1370000000000005,"15min":9.1370000000000005},"last":0.39400000000000002}]},{"osd":2,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.433,"5min":0.433,"15min":0.433},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":1.425,"5min":1.425,"15min":1.425},"last":0.42699999999999999}]},{"osd":3,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":11.957000000000001,"5min":11.957000000000001,"15min":11.957000000000001},"last":0.47699999999999998},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":18.039000000000001,"5min":18.039000000000001,"15min":18.039000000000001},"last":0.46300000000000002}]},{"osd":4,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":6.181,"5min":6.181,"15min":6.181},"last":1.4370000000000001},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.488,"5min":1.488,"15min":1.488},"last":1.411}]},{"osd":5,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":9.2420000000000009,"5min":9.2420000000000009,"15min":9.2420000000000009},"last":1.421},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":6.2110000000000003,"5min":6.2110000000000003,"15min":6.2110000000000003},"last":1.2969999999999999}]},{"osd":6,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":9.1880000000000006,"5min":9.1880000000000006,"15min":9.1880000000000006},"last":1.4530000000000001},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.462,"5min":1.462,"15min":1.462},"last":1.272}]},{"osd":7,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":6.1479999999999997,"5min":6.1479999999999997,"15min":6.1479999999999997},"last":1.3979999999999999},{"interface":"front","average":{"1min":1.028,"5min":1.028,"15min":1.028},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":9.2170000000000005,"5min":9.2170000000000005,"15min":9.2170000000000005},"last":1.353}]}]},{"osd":0,"up_from":13,"seq":55834574866,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27972,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343868,"statfs":{"total":96636764160,"available":96608120832,"internally_reserved":0,"allocated":966656,"data_stored":736234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":4.1200000000000001,"5min":4.1200000000000001,"15min":4.1200000000000001},"last":0.32200000000000001},{"interface":"front","average":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":16.940999999999999,"5min":16.940999999999999,"15min":16.940999999999999},"last":0.44500000000000001}]},{"osd":2,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.37},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":7.0209999999999999,"5min":7.0209999999999999,"15min":7.0209999999999999},"last":0.41399999999999998}]},{"osd":3,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":4.0940000000000003,"5min":4.0940000000000003,"15min":4.0940000000000003},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.432}]},{"osd":4,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":4.0049999999999999,"5min":4.0049999999999999,"15min":4.0049999999999999},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.79200000000000004}]},{"osd":5,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":4.3090000000000002,"5min":4.3090000000000002,"15min":4.3090000000000002},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":4.0709999999999997,"5min":4.0709999999999997,"15min":4.0709999999999997},"last":0.67300000000000004}]},{"osd":6,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":4.2560000000000002,"5min":4.2560000000000002,"15min":4.2560000000000002},"last":0.75900000000000001}]},{"osd":7,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":4.2850000000000001,"5min":4.2850000000000001,"15min":4.2850000000000001},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":4.0460000000000003,"5min":4.0460000000000003,"15min":4.0460000000000003},"last":0.80600000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574866,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27392,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":376832,"data_stored":145936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"min":{"1min":0.20100000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":6.0999999999999996,"5min":6.0999999999999996,"15min":6.0999999999999996},"last":0.249}]},{"osd":1,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"last":0.28799999999999998},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.624,"5min":0.624,"15min":0.624},"last":0.313}]},{"osd":3,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.38300000000000001}]},{"osd":4,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.55500000000000005}]},{"osd":5,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.57599999999999996}]},{"osd":6,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.65100000000000002}]},{"osd":7,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.628},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.80000000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574867,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27404,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344436,"statfs":{"total":96636764160,"available":96608702464,"internally_reserved":0,"allocated":389120,"data_stored":147500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:25:46 2023","interfaces":[{"interface":"back","average":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.626,"5min":0.626,"15min":0.626},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.33100000000000002}]},{"osd":1,"last update":"Wed Nov 8 19:25:46 2023","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.34999999999999998}]},{"osd":2,"last update":"Wed Nov 8 19:25:46 2023","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.37,"15min":0.37},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"last":0.40000000000000002}]},{"osd":4,"last update":"Wed Nov 8 19:25:46 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.70599999999999996}]},{"osd":5,"last update":"Wed Nov 8 19:25:46 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.56699999999999995}]},{"osd":6,"last update":"Wed Nov 8 19:25:46 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.74199999999999999}]},{"osd":7,"last update":"Wed Nov 8 19:25:46 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.66400000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574866,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":380928,"data_stored":145970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:25:47 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":6.5970000000000004,"5min":6.5970000000000004,"15min":6.5970000000000004},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":2.5030000000000001,"5min":2.5030000000000001,"15min":2.5030000000000001},"last":0.75}]},{"osd":1,"last update":"Wed Nov 8 19:25:47 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.72599999999999998}]},{"osd":2,"last update":"Wed Nov 8 19:25:47 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":6.6920000000000002,"5min":6.6920000000000002,"15min":6.6920000000000002},"last":0.871}]},{"osd":3,"last update":"Wed Nov 8 19:25:47 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.91700000000000004}]},{"osd":5,"last update":"Wed Nov 8 19:25:47 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.65900000000000003}]},{"osd":6,"last update":"Wed Nov 8 19:25:47 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.78100000000000003}]},{"osd":7,"last update":"Wed Nov 8 19:25:47 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.85299999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574866,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":380928,"data_stored":146538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.627},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.438}]},{"osd":1,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.50600000000000001}]},{"osd":2,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.46300000000000002}]},{"osd":3,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.58299999999999996}]},{"osd":4,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.59699999999999998}]},{"osd":6,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.67700000000000005}]},{"osd":7,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.61499999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-08T19:25:55.356 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-08T19:25:55.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.505+0000 7fa12f195700 1 Processor -- start 2023-11-08T19:25:55.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.505+0000 7fa12f195700 1 -- start start 2023-11-08T19:25:55.509 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.506+0000 7fa12f195700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa12815dd00 0x7fa1280cca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:55.509 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.506+0000 7fa12f195700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fa1281618f0 0x7fa12806ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:55.510 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.506+0000 7fa12f195700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1280ccfb0 con 0x7fa12806b150 2023-11-08T19:25:55.510 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.506+0000 7fa12f195700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa12806b600 con 0x7fa12815dd00 2023-11-08T19:25:55.510 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.506+0000 7fa12f195700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1280cad50 con 0x7fa1281618f0 2023-11-08T19:25:55.510 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.506+0000 7fa12cf31700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa12815dd00 0x7fa1280cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:55.510 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.506+0000 7fa12d732700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fa12806b150 0x7fa1280b2210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:46864/0 (socket says 172.21.15.143:46864) 2023-11-08T19:25:55.511 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.506+0000 7fa12d732700 1 -- 172.21.15.143:0/4178044919 learned_addr learned my addr 172.21.15.143:0/4178044919 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:25:55.511 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.506+0000 7fa127fff700 1 --2- 172.21.15.143:0/4178044919 >> v2:172.21.15.174:3300/0 conn(0x7fa1281618f0 0x7fa12806ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:55.511 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.506+0000 7fa12cf31700 1 -- 172.21.15.143:0/4178044919 >> v2:172.21.15.174:3300/0 conn(0x7fa1281618f0 msgr2=0x7fa12806ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:55.511 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.506+0000 7fa12cf31700 1 --2- 172.21.15.143:0/4178044919 >> v2:172.21.15.174:3300/0 conn(0x7fa1281618f0 0x7fa12806ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:55.511 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.506+0000 7fa12cf31700 1 -- 172.21.15.143:0/4178044919 >> v1:172.21.15.143:6789/0 conn(0x7fa12806b150 legacy=0x7fa1280b2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:55.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.506+0000 7fa1277fe700 1 -- 172.21.15.143:0/4178044919 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1586419027 0 0) 0x7fa1280ccfb0 con 0x7fa12806b150 2023-11-08T19:25:55.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.506+0000 7fa12cf31700 1 -- 172.21.15.143:0/4178044919 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa128161df0 con 0x7fa12815dd00 2023-11-08T19:25:55.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.507+0000 7fa12cf31700 1 --2- 172.21.15.143:0/4178044919 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa12815dd00 0x7fa1280cca70 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fa128161850 tx=0x7fa11c009b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d09c16b5acc0d76f server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:55.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.507+0000 7fa1277fe700 1 -- 172.21.15.143:0/4178044919 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa11c005e30 con 0x7fa12815dd00 2023-11-08T19:25:55.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.507+0000 7fa1277fe700 1 -- 172.21.15.143:0/4178044919 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa11c006030 con 0x7fa12815dd00 2023-11-08T19:25:55.513 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.507+0000 7fa1277fe700 1 -- 172.21.15.143:0/4178044919 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa11c0063c0 con 0x7fa12815dd00 2023-11-08T19:25:55.513 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.507+0000 7fa12f195700 1 -- 172.21.15.143:0/4178044919 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa12815dd00 msgr2=0x7fa1280cca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:55.513 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.507+0000 7fa12f195700 1 --2- 172.21.15.143:0/4178044919 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa12815dd00 0x7fa1280cca70 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fa128161850 tx=0x7fa11c009b00 comp rx=0 tx=0).stop 2023-11-08T19:25:55.513 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.507+0000 7fa12f195700 1 -- 172.21.15.143:0/4178044919 shutdown_connections 2023-11-08T19:25:55.513 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.507+0000 7fa12f195700 1 --2- 172.21.15.143:0/4178044919 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa12815dd00 0x7fa1280cca70 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:55.514 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.507+0000 7fa12f195700 1 --2- 172.21.15.143:0/4178044919 >> v2:172.21.15.174:3300/0 conn(0x7fa1281618f0 0x7fa12806ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:55.514 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.507+0000 7fa12f195700 1 -- 172.21.15.143:0/4178044919 >> 172.21.15.143:0/4178044919 conn(0x7fa128099ab0 msgr2=0x7fa1280d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:55.514 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.507+0000 7fa12f195700 1 -- 172.21.15.143:0/4178044919 shutdown_connections 2023-11-08T19:25:55.514 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.507+0000 7fa12f195700 1 -- 172.21.15.143:0/4178044919 wait complete. 2023-11-08T19:25:55.517 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.508+0000 7fa12f195700 1 Processor -- start 2023-11-08T19:25:55.517 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.509+0000 7fa12f195700 1 -- start start 2023-11-08T19:25:55.518 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.509+0000 7fa12f195700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa12806b150 0x7fa128155020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:55.518 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.509+0000 7fa12f195700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa12815dd00 0x7fa1280c77d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:55.518 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.509+0000 7fa12f195700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa1281618f0 0x7fa12815a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:55.518 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.509+0000 7fa12f195700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1280c4b90 con 0x7fa1281618f0 2023-11-08T19:25:55.518 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.509+0000 7fa12f195700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1280c8290 con 0x7fa12806b150 2023-11-08T19:25:55.519 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.509+0000 7fa12f195700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1280c8570 con 0x7fa12815dd00 2023-11-08T19:25:55.519 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.510+0000 7fa12d732700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa1281618f0 0x7fa12815a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:55.519 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.510+0000 7fa12d732700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa1281618f0 0x7fa12815a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50716/0 (socket says 172.21.15.143:50716) 2023-11-08T19:25:55.519 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.510+0000 7fa12d732700 1 -- 172.21.15.143:0/3329049907 learned_addr learned my addr 172.21.15.143:0/3329049907 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:55.519 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.510+0000 7fa12cf31700 1 --2- 172.21.15.143:0/3329049907 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa12806b150 0x7fa128155020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:55.520 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.510+0000 7fa127fff700 1 --2- 172.21.15.143:0/3329049907 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa12815dd00 0x7fa1280c77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:55.520 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.510+0000 7fa12d732700 1 -- 172.21.15.143:0/3329049907 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa12815dd00 msgr2=0x7fa1280c77d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:55.520 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.510+0000 7fa12d732700 1 --2- 172.21.15.143:0/3329049907 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa12815dd00 0x7fa1280c77d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:55.520 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.510+0000 7fa12d732700 1 -- 172.21.15.143:0/3329049907 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa12806b150 msgr2=0x7fa128155020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:55.520 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.510+0000 7fa12d732700 1 --2- 172.21.15.143:0/3329049907 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa12806b150 0x7fa128155020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:55.521 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.510+0000 7fa12d732700 1 -- 172.21.15.143:0/3329049907 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa11c009430 con 0x7fa1281618f0 2023-11-08T19:25:55.521 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.510+0000 7fa12d732700 1 --2- 172.21.15.143:0/3329049907 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa1281618f0 0x7fa12815a1f0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fa12000cba0 tx=0x7fa12000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:55.521 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.510+0000 7fa1257fa700 1 -- 172.21.15.143:0/3329049907 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa12000ab80 con 0x7fa1281618f0 2023-11-08T19:25:55.521 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.510+0000 7fa1257fa700 1 -- 172.21.15.143:0/3329049907 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa12000ad00 con 0x7fa1281618f0 2023-11-08T19:25:55.521 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.510+0000 7fa12f195700 1 -- 172.21.15.143:0/3329049907 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1280c8850 con 0x7fa1281618f0 2023-11-08T19:25:55.521 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.510+0000 7fa1257fa700 1 -- 172.21.15.143:0/3329049907 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa120007310 con 0x7fa1281618f0 2023-11-08T19:25:55.522 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.510+0000 7fa12f195700 1 -- 172.21.15.143:0/3329049907 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1281600a0 con 0x7fa1281618f0 2023-11-08T19:25:55.522 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.511+0000 7fa12f195700 1 -- 172.21.15.143:0/3329049907 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1280aff60 con 0x7fa1281618f0 2023-11-08T19:25:55.522 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.512+0000 7fa1257fa700 1 -- 172.21.15.143:0/3329049907 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa120007470 con 0x7fa1281618f0 2023-11-08T19:25:55.522 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.512+0000 7fa1257fa700 1 --2- 172.21.15.143:0/3329049907 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa10c06af00 0x7fa10c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:55.522 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.512+0000 7fa1257fa700 1 -- 172.21.15.143:0/3329049907 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa120089470 con 0x7fa1281618f0 2023-11-08T19:25:55.523 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.512+0000 7fa12cf31700 1 --2- 172.21.15.143:0/3329049907 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa10c06af00 0x7fa10c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:55.523 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.513+0000 7fa12cf31700 1 --2- 172.21.15.143:0/3329049907 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa10c06af00 0x7fa10c06d3c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa11c000c00 tx=0x7fa11c008070 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:55.523 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.516+0000 7fa1257fa700 1 -- 172.21.15.143:0/3329049907 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa120054520 con 0x7fa1281618f0 2023-11-08T19:25:55.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.702+0000 7fa12f195700 1 -- 172.21.15.143:0/3329049907 --> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa1280c86e0 con 0x7fa10c06af00 2023-11-08T19:25:55.712 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.711+0000 7fa1257fa700 1 -- 172.21.15.143:0/3329049907 <== mgr.4099 v2:172.21.15.174:6832/4037280178 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286848 (secure 0 0 0) 0x7fa1280c86e0 con 0x7fa10c06af00 2023-11-08T19:25:55.715 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:25:55.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.714+0000 7fa12f195700 1 -- 172.21.15.143:0/3329049907 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa10c06af00 msgr2=0x7fa10c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:55.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.714+0000 7fa12f195700 1 --2- 172.21.15.143:0/3329049907 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa10c06af00 0x7fa10c06d3c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa11c000c00 tx=0x7fa11c008070 comp rx=0 tx=0).stop 2023-11-08T19:25:55.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.714+0000 7fa12f195700 1 -- 172.21.15.143:0/3329049907 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa1281618f0 msgr2=0x7fa12815a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:55.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.714+0000 7fa12f195700 1 --2- 172.21.15.143:0/3329049907 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa1281618f0 0x7fa12815a1f0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fa12000cba0 tx=0x7fa12000cf60 comp rx=0 tx=0).stop 2023-11-08T19:25:55.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.714+0000 7fa12f195700 1 -- 172.21.15.143:0/3329049907 shutdown_connections 2023-11-08T19:25:55.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.714+0000 7fa12f195700 1 --2- 172.21.15.143:0/3329049907 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa10c06af00 0x7fa10c06d3c0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:55.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.714+0000 7fa12f195700 1 --2- 172.21.15.143:0/3329049907 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa1281618f0 0x7fa12815a1f0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:55.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.714+0000 7fa12f195700 1 --2- 172.21.15.143:0/3329049907 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa12806b150 0x7fa128155020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:55.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.714+0000 7fa12f195700 1 --2- 172.21.15.143:0/3329049907 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa12815dd00 0x7fa1280c77d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:55.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.714+0000 7fa12f195700 1 -- 172.21.15.143:0/3329049907 >> 172.21.15.143:0/3329049907 conn(0x7fa128099ab0 msgr2=0x7fa1280af6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:55.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.714+0000 7fa12f195700 1 -- 172.21.15.143:0/3329049907 shutdown_connections 2023-11-08T19:25:55.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.714+0000 7fa12f195700 1 -- 172.21.15.143:0/3329049907 wait complete. 2023-11-08T19:25:55.720 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2023-11-08T19:25:55.728 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":61,"stamp":"2023-11-08T19:25:54.525094+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592196,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":200,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":152,"ondisk_log_size":152,"up":162,"acting":162,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":117,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220240,"kb_used_data":4052,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754480,"statfs":{"total":773094113280,"available":772868587520,"internally_reserved":0,"allocated":4149248,"data_stored":2313094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1828,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.669558"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:25:49.854746+0000","last_change":"2023-11-08T19:25:49.854746+0000","last_active":"2023-11-08T19:25:49.854746+0000","last_peered":"2023-11-08T19:25:49.854746+0000","last_clean":"2023-11-08T19:25:49.854746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:25:49.854746+0000","last_undegraded":"2023-11-08T19:25:49.854746+0000","last_fullsized":"2023-11-08T19:25:49.854746+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204169+0000","last_change":"2023-11-08T19:25:50.890421+0000","last_active":"2023-11-08T19:25:52.204169+0000","last_peered":"2023-11-08T19:25:52.204169+0000","last_clean":"2023-11-08T19:25:52.204169+0000","last_became_active":"2023-11-08T19:25:50.890142+0000","last_became_peered":"2023-11-08T19:25:50.890142+0000","last_unstale":"2023-11-08T19:25:52.204169+0000","last_undegraded":"2023-11-08T19:25:52.204169+0000","last_fullsized":"2023-11-08T19:25:52.204169+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:01:43.267493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.205154+0000","last_change":"2023-11-08T19:25:50.875008+0000","last_active":"2023-11-08T19:25:52.205154+0000","last_peered":"2023-11-08T19:25:52.205154+0000","last_clean":"2023-11-08T19:25:52.205154+0000","last_became_active":"2023-11-08T19:25:50.874499+0000","last_became_peered":"2023-11-08T19:25:50.874499+0000","last_unstale":"2023-11-08T19:25:52.205154+0000","last_undegraded":"2023-11-08T19:25:52.205154+0000","last_fullsized":"2023-11-08T19:25:52.205154+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:27:11.209258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.991817+0000","last_change":"2023-11-08T19:25:50.883677+0000","last_active":"2023-11-08T19:25:51.991817+0000","last_peered":"2023-11-08T19:25:51.991817+0000","last_clean":"2023-11-08T19:25:51.991817+0000","last_became_active":"2023-11-08T19:25:50.883141+0000","last_became_peered":"2023-11-08T19:25:50.883141+0000","last_unstale":"2023-11-08T19:25:51.991817+0000","last_undegraded":"2023-11-08T19:25:51.991817+0000","last_fullsized":"2023-11-08T19:25:51.991817+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:56:01.687509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.876619+0000","last_change":"2023-11-08T19:25:50.877229+0000","last_active":"2023-11-08T19:25:51.876619+0000","last_peered":"2023-11-08T19:25:51.876619+0000","last_clean":"2023-11-08T19:25:51.876619+0000","last_became_active":"2023-11-08T19:25:50.876959+0000","last_became_peered":"2023-11-08T19:25:50.876959+0000","last_unstale":"2023-11-08T19:25:51.876619+0000","last_undegraded":"2023-11-08T19:25:51.876619+0000","last_fullsized":"2023-11-08T19:25:51.876619+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:27:09.954811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204135+0000","last_change":"2023-11-08T19:25:50.883530+0000","last_active":"2023-11-08T19:25:52.204135+0000","last_peered":"2023-11-08T19:25:52.204135+0000","last_clean":"2023-11-08T19:25:52.204135+0000","last_became_active":"2023-11-08T19:25:50.883198+0000","last_became_peered":"2023-11-08T19:25:50.883198+0000","last_unstale":"2023-11-08T19:25:52.204135+0000","last_undegraded":"2023-11-08T19:25:52.204135+0000","last_fullsized":"2023-11-08T19:25:52.204135+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:27:11.209258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:25:49.854746+0000","last_change":"2023-11-08T19:25:49.854746+0000","last_active":"2023-11-08T19:25:49.854746+0000","last_peered":"2023-11-08T19:25:49.854746+0000","last_clean":"2023-11-08T19:25:49.854746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:25:49.854746+0000","last_undegraded":"2023-11-08T19:25:49.854746+0000","last_fullsized":"2023-11-08T19:25:49.854746+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:25:49.854746+0000","last_change":"2023-11-08T19:25:49.854746+0000","last_active":"2023-11-08T19:25:49.854746+0000","last_peered":"2023-11-08T19:25:49.854746+0000","last_clean":"2023-11-08T19:25:49.854746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:25:49.854746+0000","last_undegraded":"2023-11-08T19:25:49.854746+0000","last_fullsized":"2023-11-08T19:25:49.854746+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.205585+0000","last_change":"2023-11-08T19:25:50.888816+0000","last_active":"2023-11-08T19:25:52.205585+0000","last_peered":"2023-11-08T19:25:52.205585+0000","last_clean":"2023-11-08T19:25:52.205585+0000","last_became_active":"2023-11-08T19:25:50.888536+0000","last_became_peered":"2023-11-08T19:25:50.888536+0000","last_unstale":"2023-11-08T19:25:52.205585+0000","last_undegraded":"2023-11-08T19:25:52.205585+0000","last_fullsized":"2023-11-08T19:25:52.205585+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:56:01.687509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.991994+0000","last_change":"2023-11-08T19:25:50.878973+0000","last_active":"2023-11-08T19:25:51.991994+0000","last_peered":"2023-11-08T19:25:51.991994+0000","last_clean":"2023-11-08T19:25:51.991994+0000","last_became_active":"2023-11-08T19:25:50.878706+0000","last_became_peered":"2023-11-08T19:25:50.878706+0000","last_unstale":"2023-11-08T19:25:51.991994+0000","last_undegraded":"2023-11-08T19:25:51.991994+0000","last_fullsized":"2023-11-08T19:25:51.991994+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:49:41.475597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:25:49.854746+0000","last_change":"2023-11-08T19:25:49.854746+0000","last_active":"2023-11-08T19:25:49.854746+0000","last_peered":"2023-11-08T19:25:49.854746+0000","last_clean":"2023-11-08T19:25:49.854746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:25:49.854746+0000","last_undegraded":"2023-11-08T19:25:49.854746+0000","last_fullsized":"2023-11-08T19:25:49.854746+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204768+0000","last_change":"2023-11-08T19:25:50.877227+0000","last_active":"2023-11-08T19:25:52.204768+0000","last_peered":"2023-11-08T19:25:52.204768+0000","last_clean":"2023-11-08T19:25:52.204768+0000","last_became_active":"2023-11-08T19:25:50.876876+0000","last_became_peered":"2023-11-08T19:25:50.876876+0000","last_unstale":"2023-11-08T19:25:52.204768+0000","last_undegraded":"2023-11-08T19:25:52.204768+0000","last_fullsized":"2023-11-08T19:25:52.204768+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:05:33.227810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.876804+0000","last_change":"2023-11-08T19:25:50.890044+0000","last_active":"2023-11-08T19:25:51.876804+0000","last_peered":"2023-11-08T19:25:51.876804+0000","last_clean":"2023-11-08T19:25:51.876804+0000","last_became_active":"2023-11-08T19:25:50.889741+0000","last_became_peered":"2023-11-08T19:25:50.889741+0000","last_unstale":"2023-11-08T19:25:51.876804+0000","last_undegraded":"2023-11-08T19:25:51.876804+0000","last_fullsized":"2023-11-08T19:25:51.876804+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:54:55.202302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204255+0000","last_change":"2023-11-08T19:25:50.876264+0000","last_active":"2023-11-08T19:25:52.204255+0000","last_peered":"2023-11-08T19:25:52.204255+0000","last_clean":"2023-11-08T19:25:52.204255+0000","last_became_active":"2023-11-08T19:25:50.875971+0000","last_became_peered":"2023-11-08T19:25:50.875971+0000","last_unstale":"2023-11-08T19:25:52.204255+0000","last_undegraded":"2023-11-08T19:25:52.204255+0000","last_fullsized":"2023-11-08T19:25:52.204255+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:50:30.103716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204868+0000","last_change":"2023-11-08T19:25:50.877018+0000","last_active":"2023-11-08T19:25:52.204868+0000","last_peered":"2023-11-08T19:25:52.204868+0000","last_clean":"2023-11-08T19:25:52.204868+0000","last_became_active":"2023-11-08T19:25:50.876682+0000","last_became_peered":"2023-11-08T19:25:50.876682+0000","last_unstale":"2023-11-08T19:25:52.204868+0000","last_undegraded":"2023-11-08T19:25:52.204868+0000","last_fullsized":"2023-11-08T19:25:52.204868+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:52:20.334760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:25:49.854746+0000","last_change":"2023-11-08T19:25:49.854746+0000","last_active":"2023-11-08T19:25:49.854746+0000","last_peered":"2023-11-08T19:25:49.854746+0000","last_clean":"2023-11-08T19:25:49.854746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:25:49.854746+0000","last_undegraded":"2023-11-08T19:25:49.854746+0000","last_fullsized":"2023-11-08T19:25:49.854746+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:25:49.854746+0000","last_change":"2023-11-08T19:25:49.854746+0000","last_active":"2023-11-08T19:25:49.854746+0000","last_peered":"2023-11-08T19:25:49.854746+0000","last_clean":"2023-11-08T19:25:49.854746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:25:49.854746+0000","last_undegraded":"2023-11-08T19:25:49.854746+0000","last_fullsized":"2023-11-08T19:25:49.854746+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204251+0000","last_change":"2023-11-08T19:25:50.890954+0000","last_active":"2023-11-08T19:25:52.204251+0000","last_peered":"2023-11-08T19:25:52.204251+0000","last_clean":"2023-11-08T19:25:52.204251+0000","last_became_active":"2023-11-08T19:25:50.890695+0000","last_became_peered":"2023-11-08T19:25:50.890695+0000","last_unstale":"2023-11-08T19:25:52.204251+0000","last_undegraded":"2023-11-08T19:25:52.204251+0000","last_fullsized":"2023-11-08T19:25:52.204251+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:44:02.415941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204041+0000","last_change":"2023-11-08T19:25:50.892390+0000","last_active":"2023-11-08T19:25:52.204041+0000","last_peered":"2023-11-08T19:25:52.204041+0000","last_clean":"2023-11-08T19:25:52.204041+0000","last_became_active":"2023-11-08T19:25:50.892131+0000","last_became_peered":"2023-11-08T19:25:50.892131+0000","last_unstale":"2023-11-08T19:25:52.204041+0000","last_undegraded":"2023-11-08T19:25:52.204041+0000","last_fullsized":"2023-11-08T19:25:52.204041+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:25:07.133280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.876861+0000","last_change":"2023-11-08T19:25:50.874502+0000","last_active":"2023-11-08T19:25:51.876861+0000","last_peered":"2023-11-08T19:25:51.876861+0000","last_clean":"2023-11-08T19:25:51.876861+0000","last_became_active":"2023-11-08T19:25:50.874216+0000","last_became_peered":"2023-11-08T19:25:50.874216+0000","last_unstale":"2023-11-08T19:25:51.876861+0000","last_undegraded":"2023-11-08T19:25:51.876861+0000","last_fullsized":"2023-11-08T19:25:51.876861+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:39:23.438674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204719+0000","last_change":"2023-11-08T19:25:50.886322+0000","last_active":"2023-11-08T19:25:52.204719+0000","last_peered":"2023-11-08T19:25:52.204719+0000","last_clean":"2023-11-08T19:25:52.204719+0000","last_became_active":"2023-11-08T19:25:50.886011+0000","last_became_peered":"2023-11-08T19:25:50.886011+0000","last_unstale":"2023-11-08T19:25:52.204719+0000","last_undegraded":"2023-11-08T19:25:52.204719+0000","last_fullsized":"2023-11-08T19:25:52.204719+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:09:05.379033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204755+0000","last_change":"2023-11-08T19:25:50.881922+0000","last_active":"2023-11-08T19:25:52.204755+0000","last_peered":"2023-11-08T19:25:52.204755+0000","last_clean":"2023-11-08T19:25:52.204755+0000","last_became_active":"2023-11-08T19:25:50.880836+0000","last_became_peered":"2023-11-08T19:25:50.880836+0000","last_unstale":"2023-11-08T19:25:52.204755+0000","last_undegraded":"2023-11-08T19:25:52.204755+0000","last_fullsized":"2023-11-08T19:25:52.204755+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:49:45.059924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.989623+0000","last_change":"2023-11-08T19:25:50.886648+0000","last_active":"2023-11-08T19:25:51.989623+0000","last_peered":"2023-11-08T19:25:51.989623+0000","last_clean":"2023-11-08T19:25:51.989623+0000","last_became_active":"2023-11-08T19:25:50.886376+0000","last_became_peered":"2023-11-08T19:25:50.886376+0000","last_unstale":"2023-11-08T19:25:51.989623+0000","last_undegraded":"2023-11-08T19:25:51.989623+0000","last_fullsized":"2023-11-08T19:25:51.989623+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:49:45.059924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204354+0000","last_change":"2023-11-08T19:25:50.875989+0000","last_active":"2023-11-08T19:25:52.204354+0000","last_peered":"2023-11-08T19:25:52.204354+0000","last_clean":"2023-11-08T19:25:52.204354+0000","last_became_active":"2023-11-08T19:25:50.874428+0000","last_became_peered":"2023-11-08T19:25:50.874428+0000","last_unstale":"2023-11-08T19:25:52.204354+0000","last_undegraded":"2023-11-08T19:25:52.204354+0000","last_fullsized":"2023-11-08T19:25:52.204354+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:04:01.492989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.991856+0000","last_change":"2023-11-08T19:25:50.885263+0000","last_active":"2023-11-08T19:25:51.991856+0000","last_peered":"2023-11-08T19:25:51.991856+0000","last_clean":"2023-11-08T19:25:51.991856+0000","last_became_active":"2023-11-08T19:25:50.884856+0000","last_became_peered":"2023-11-08T19:25:50.884856+0000","last_unstale":"2023-11-08T19:25:51.991856+0000","last_undegraded":"2023-11-08T19:25:51.991856+0000","last_fullsized":"2023-11-08T19:25:51.991856+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:17:24.728184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204268+0000","last_change":"2023-11-08T19:25:50.884128+0000","last_active":"2023-11-08T19:25:52.204268+0000","last_peered":"2023-11-08T19:25:52.204268+0000","last_clean":"2023-11-08T19:25:52.204268+0000","last_became_active":"2023-11-08T19:25:50.883848+0000","last_became_peered":"2023-11-08T19:25:50.883848+0000","last_unstale":"2023-11-08T19:25:52.204268+0000","last_undegraded":"2023-11-08T19:25:52.204268+0000","last_fullsized":"2023-11-08T19:25:52.204268+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:32:01.237850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.992064+0000","last_change":"2023-11-08T19:25:50.885482+0000","last_active":"2023-11-08T19:25:51.992064+0000","last_peered":"2023-11-08T19:25:51.992064+0000","last_clean":"2023-11-08T19:25:51.992064+0000","last_became_active":"2023-11-08T19:25:50.884816+0000","last_became_peered":"2023-11-08T19:25:50.884816+0000","last_unstale":"2023-11-08T19:25:51.992064+0000","last_undegraded":"2023-11-08T19:25:51.992064+0000","last_fullsized":"2023-11-08T19:25:51.992064+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:05:33.227810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204941+0000","last_change":"2023-11-08T19:25:50.871479+0000","last_active":"2023-11-08T19:25:52.204941+0000","last_peered":"2023-11-08T19:25:52.204941+0000","last_clean":"2023-11-08T19:25:52.204941+0000","last_became_active":"2023-11-08T19:25:50.871211+0000","last_became_peered":"2023-11-08T19:25:50.871211+0000","last_unstale":"2023-11-08T19:25:52.204941+0000","last_undegraded":"2023-11-08T19:25:52.204941+0000","last_fullsized":"2023-11-08T19:25:52.204941+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:25:16.223226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204982+0000","last_change":"2023-11-08T19:25:50.881545+0000","last_active":"2023-11-08T19:25:52.204982+0000","last_peered":"2023-11-08T19:25:52.204982+0000","last_clean":"2023-11-08T19:25:52.204982+0000","last_became_active":"2023-11-08T19:25:50.881277+0000","last_became_peered":"2023-11-08T19:25:50.881277+0000","last_unstale":"2023-11-08T19:25:52.204982+0000","last_undegraded":"2023-11-08T19:25:52.204982+0000","last_fullsized":"2023-11-08T19:25:52.204982+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:18:57.865706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:25:49.854746+0000","last_change":"2023-11-08T19:25:49.854746+0000","last_active":"2023-11-08T19:25:49.854746+0000","last_peered":"2023-11-08T19:25:49.854746+0000","last_clean":"2023-11-08T19:25:49.854746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:25:49.854746+0000","last_undegraded":"2023-11-08T19:25:49.854746+0000","last_fullsized":"2023-11-08T19:25:49.854746+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:25:49.854746+0000","last_change":"2023-11-08T19:25:49.854746+0000","last_active":"2023-11-08T19:25:49.854746+0000","last_peered":"2023-11-08T19:25:49.854746+0000","last_clean":"2023-11-08T19:25:49.854746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:25:49.854746+0000","last_undegraded":"2023-11-08T19:25:49.854746+0000","last_fullsized":"2023-11-08T19:25:49.854746+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:25:49.854746+0000","last_change":"2023-11-08T19:25:49.854746+0000","last_active":"2023-11-08T19:25:49.854746+0000","last_peered":"2023-11-08T19:25:49.854746+0000","last_clean":"2023-11-08T19:25:49.854746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:25:49.854746+0000","last_undegraded":"2023-11-08T19:25:49.854746+0000","last_fullsized":"2023-11-08T19:25:49.854746+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.989618+0000","last_change":"2023-11-08T19:25:50.889063+0000","last_active":"2023-11-08T19:25:51.989618+0000","last_peered":"2023-11-08T19:25:51.989618+0000","last_clean":"2023-11-08T19:25:51.989618+0000","last_became_active":"2023-11-08T19:25:50.888760+0000","last_became_peered":"2023-11-08T19:25:50.888760+0000","last_unstale":"2023-11-08T19:25:51.989618+0000","last_undegraded":"2023-11-08T19:25:51.989618+0000","last_fullsized":"2023-11-08T19:25:51.989618+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:37:10.264673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-11-08T19:25:48.857285+0000","last_change":"2023-11-08T19:25:48.857285+0000","last_active":"2023-11-08T19:25:48.844481+0000","last_peered":"2023-11-08T19:25:48.844481+0000","last_clean":"2023-11-08T19:25:48.844481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:25:48.857285+0000","last_undegraded":"2023-11-08T19:25:48.857285+0000","last_fullsized":"2023-11-08T19:25:48.857285+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:25:49.854746+0000","last_change":"2023-11-08T19:25:49.854746+0000","last_active":"2023-11-08T19:25:49.854746+0000","last_peered":"2023-11-08T19:25:49.854746+0000","last_clean":"2023-11-08T19:25:49.854746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:25:49.854746+0000","last_undegraded":"2023-11-08T19:25:49.854746+0000","last_fullsized":"2023-11-08T19:25:49.854746+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1e","version":"32'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.989742+0000","last_change":"2023-11-08T19:25:49.879837+0000","last_active":"2023-11-08T19:25:51.989742+0000","last_peered":"2023-11-08T19:25:51.989742+0000","last_clean":"2023-11-08T19:25:51.989742+0000","last_became_active":"2023-11-08T19:25:49.879570+0000","last_became_peered":"2023-11-08T19:25:49.879570+0000","last_unstale":"2023-11-08T19:25:51.989742+0000","last_undegraded":"2023-11-08T19:25:51.989742+0000","last_fullsized":"2023-11-08T19:25:51.989742+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:00:52.169251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.205335+0000","last_change":"2023-11-08T19:25:50.881012+0000","last_active":"2023-11-08T19:25:52.205335+0000","last_peered":"2023-11-08T19:25:52.205335+0000","last_clean":"2023-11-08T19:25:52.205335+0000","last_became_active":"2023-11-08T19:25:50.880752+0000","last_became_peered":"2023-11-08T19:25:50.880752+0000","last_unstale":"2023-11-08T19:25:52.205335+0000","last_undegraded":"2023-11-08T19:25:52.205335+0000","last_fullsized":"2023-11-08T19:25:52.205335+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:33:12.567239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"32'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.203995+0000","last_change":"2023-11-08T19:25:49.886087+0000","last_active":"2023-11-08T19:25:52.203995+0000","last_peered":"2023-11-08T19:25:52.203995+0000","last_clean":"2023-11-08T19:25:52.203995+0000","last_became_active":"2023-11-08T19:25:49.885801+0000","last_became_peered":"2023-11-08T19:25:49.885801+0000","last_unstale":"2023-11-08T19:25:52.203995+0000","last_undegraded":"2023-11-08T19:25:52.203995+0000","last_fullsized":"2023-11-08T19:25:52.203995+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:04:00.482724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.992009+0000","last_change":"2023-11-08T19:25:50.885648+0000","last_active":"2023-11-08T19:25:51.992009+0000","last_peered":"2023-11-08T19:25:51.992009+0000","last_clean":"2023-11-08T19:25:51.992009+0000","last_became_active":"2023-11-08T19:25:50.885179+0000","last_became_peered":"2023-11-08T19:25:50.885179+0000","last_unstale":"2023-11-08T19:25:51.992009+0000","last_undegraded":"2023-11-08T19:25:51.992009+0000","last_fullsized":"2023-11-08T19:25:51.992009+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:52:20.334760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"32'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.876725+0000","last_change":"2023-11-08T19:25:49.883073+0000","last_active":"2023-11-08T19:25:51.876725+0000","last_peered":"2023-11-08T19:25:51.876725+0000","last_clean":"2023-11-08T19:25:51.876725+0000","last_became_active":"2023-11-08T19:25:49.882522+0000","last_became_peered":"2023-11-08T19:25:49.882522+0000","last_unstale":"2023-11-08T19:25:51.876725+0000","last_undegraded":"2023-11-08T19:25:51.876725+0000","last_fullsized":"2023-11-08T19:25:51.876725+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:27:43.830493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:25:49.854746+0000","last_change":"2023-11-08T19:25:49.854746+0000","last_active":"2023-11-08T19:25:49.854746+0000","last_peered":"2023-11-08T19:25:49.854746+0000","last_clean":"2023-11-08T19:25:49.854746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:25:49.854746+0000","last_undegraded":"2023-11-08T19:25:49.854746+0000","last_fullsized":"2023-11-08T19:25:49.854746+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1b","version":"32'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.205020+0000","last_change":"2023-11-08T19:25:49.884628+0000","last_active":"2023-11-08T19:25:52.205020+0000","last_peered":"2023-11-08T19:25:52.205020+0000","last_clean":"2023-11-08T19:25:52.205020+0000","last_became_active":"2023-11-08T19:25:49.884384+0000","last_became_peered":"2023-11-08T19:25:49.884384+0000","last_unstale":"2023-11-08T19:25:52.205020+0000","last_undegraded":"2023-11-08T19:25:52.205020+0000","last_fullsized":"2023-11-08T19:25:52.205020+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:17:52.641253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204834+0000","last_change":"2023-11-08T19:25:50.874965+0000","last_active":"2023-11-08T19:25:52.204834+0000","last_peered":"2023-11-08T19:25:52.204834+0000","last_clean":"2023-11-08T19:25:52.204834+0000","last_became_active":"2023-11-08T19:25:50.874696+0000","last_became_peered":"2023-11-08T19:25:50.874696+0000","last_unstale":"2023-11-08T19:25:52.204834+0000","last_undegraded":"2023-11-08T19:25:52.204834+0000","last_fullsized":"2023-11-08T19:25:52.204834+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:49:41.475597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"32'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204932+0000","last_change":"2023-11-08T19:25:49.884607+0000","last_active":"2023-11-08T19:25:52.204932+0000","last_peered":"2023-11-08T19:25:52.204932+0000","last_clean":"2023-11-08T19:25:52.204932+0000","last_became_active":"2023-11-08T19:25:49.884392+0000","last_became_peered":"2023-11-08T19:25:49.884392+0000","last_unstale":"2023-11-08T19:25:52.204932+0000","last_undegraded":"2023-11-08T19:25:52.204932+0000","last_fullsized":"2023-11-08T19:25:52.204932+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:54:54.192037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204370+0000","last_change":"2023-11-08T19:25:50.881747+0000","last_active":"2023-11-08T19:25:52.204370+0000","last_peered":"2023-11-08T19:25:52.204370+0000","last_clean":"2023-11-08T19:25:52.204370+0000","last_became_active":"2023-11-08T19:25:50.881225+0000","last_became_peered":"2023-11-08T19:25:50.881225+0000","last_unstale":"2023-11-08T19:25:52.204370+0000","last_undegraded":"2023-11-08T19:25:52.204370+0000","last_fullsized":"2023-11-08T19:25:52.204370+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:37:10.264673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.203749+0000","last_change":"2023-11-08T19:25:49.882231+0000","last_active":"2023-11-08T19:25:52.203749+0000","last_peered":"2023-11-08T19:25:52.203749+0000","last_clean":"2023-11-08T19:25:52.203749+0000","last_became_active":"2023-11-08T19:25:49.881992+0000","last_became_peered":"2023-11-08T19:25:49.881992+0000","last_unstale":"2023-11-08T19:25:52.203749+0000","last_undegraded":"2023-11-08T19:25:52.203749+0000","last_fullsized":"2023-11-08T19:25:52.203749+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:00:52.169251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.991649+0000","last_change":"2023-11-08T19:25:50.883482+0000","last_active":"2023-11-08T19:25:51.991649+0000","last_peered":"2023-11-08T19:25:51.991649+0000","last_clean":"2023-11-08T19:25:51.991649+0000","last_became_active":"2023-11-08T19:25:50.883187+0000","last_became_peered":"2023-11-08T19:25:50.883187+0000","last_unstale":"2023-11-08T19:25:51.991649+0000","last_undegraded":"2023-11-08T19:25:51.991649+0000","last_fullsized":"2023-11-08T19:25:51.991649+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:33:12.567239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204890+0000","last_change":"2023-11-08T19:25:49.878388+0000","last_active":"2023-11-08T19:25:52.204890+0000","last_peered":"2023-11-08T19:25:52.204890+0000","last_clean":"2023-11-08T19:25:52.204890+0000","last_became_active":"2023-11-08T19:25:49.878111+0000","last_became_peered":"2023-11-08T19:25:49.878111+0000","last_unstale":"2023-11-08T19:25:52.204890+0000","last_undegraded":"2023-11-08T19:25:52.204890+0000","last_fullsized":"2023-11-08T19:25:52.204890+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:39:22.428409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204894+0000","last_change":"2023-11-08T19:25:50.877613+0000","last_active":"2023-11-08T19:25:52.204894+0000","last_peered":"2023-11-08T19:25:52.204894+0000","last_clean":"2023-11-08T19:25:52.204894+0000","last_became_active":"2023-11-08T19:25:50.877311+0000","last_became_peered":"2023-11-08T19:25:50.877311+0000","last_unstale":"2023-11-08T19:25:52.204894+0000","last_undegraded":"2023-11-08T19:25:52.204894+0000","last_fullsized":"2023-11-08T19:25:52.204894+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:50:52.902620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204190+0000","last_change":"2023-11-08T19:25:49.884355+0000","last_active":"2023-11-08T19:25:52.204190+0000","last_peered":"2023-11-08T19:25:52.204190+0000","last_clean":"2023-11-08T19:25:52.204190+0000","last_became_active":"2023-11-08T19:25:49.884065+0000","last_became_peered":"2023-11-08T19:25:49.884065+0000","last_unstale":"2023-11-08T19:25:52.204190+0000","last_undegraded":"2023-11-08T19:25:52.204190+0000","last_fullsized":"2023-11-08T19:25:52.204190+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:05:32.217545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204103+0000","last_change":"2023-11-08T19:25:50.888556+0000","last_active":"2023-11-08T19:25:52.204103+0000","last_peered":"2023-11-08T19:25:52.204103+0000","last_clean":"2023-11-08T19:25:52.204103+0000","last_became_active":"2023-11-08T19:25:50.887909+0000","last_became_peered":"2023-11-08T19:25:50.887909+0000","last_unstale":"2023-11-08T19:25:52.204103+0000","last_undegraded":"2023-11-08T19:25:52.204103+0000","last_fullsized":"2023-11-08T19:25:52.204103+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:38:37.341013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.989663+0000","last_change":"2023-11-08T19:25:49.884919+0000","last_active":"2023-11-08T19:25:51.989663+0000","last_peered":"2023-11-08T19:25:51.989663+0000","last_clean":"2023-11-08T19:25:51.989663+0000","last_became_active":"2023-11-08T19:25:49.884679+0000","last_became_peered":"2023-11-08T19:25:49.884679+0000","last_unstale":"2023-11-08T19:25:51.989663+0000","last_undegraded":"2023-11-08T19:25:51.989663+0000","last_fullsized":"2023-11-08T19:25:51.989663+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:39:22.428409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204236+0000","last_change":"2023-11-08T19:25:50.882028+0000","last_active":"2023-11-08T19:25:52.204236+0000","last_peered":"2023-11-08T19:25:52.204236+0000","last_clean":"2023-11-08T19:25:52.204236+0000","last_became_active":"2023-11-08T19:25:50.881746+0000","last_became_peered":"2023-11-08T19:25:50.881746+0000","last_unstale":"2023-11-08T19:25:52.204236+0000","last_undegraded":"2023-11-08T19:25:52.204236+0000","last_fullsized":"2023-11-08T19:25:52.204236+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:39:57.441098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"32'3","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204967+0000","last_change":"2023-11-08T19:25:49.878409+0000","last_active":"2023-11-08T19:25:52.204967+0000","last_peered":"2023-11-08T19:25:52.204967+0000","last_clean":"2023-11-08T19:25:52.204967+0000","last_became_active":"2023-11-08T19:25:49.878105+0000","last_became_peered":"2023-11-08T19:25:49.878105+0000","last_unstale":"2023-11-08T19:25:52.204967+0000","last_undegraded":"2023-11-08T19:25:52.204967+0000","last_fullsized":"2023-11-08T19:25:52.204967+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:55:30.168605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204866+0000","last_change":"2023-11-08T19:25:50.887174+0000","last_active":"2023-11-08T19:25:52.204866+0000","last_peered":"2023-11-08T19:25:52.204866+0000","last_clean":"2023-11-08T19:25:52.204866+0000","last_became_active":"2023-11-08T19:25:50.886902+0000","last_became_peered":"2023-11-08T19:25:50.886902+0000","last_unstale":"2023-11-08T19:25:52.204866+0000","last_undegraded":"2023-11-08T19:25:52.204866+0000","last_fullsized":"2023-11-08T19:25:52.204866+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:19:42.562982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"32'6","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204091+0000","last_change":"2023-11-08T19:25:49.886499+0000","last_active":"2023-11-08T19:25:52.204091+0000","last_peered":"2023-11-08T19:25:52.204091+0000","last_clean":"2023-11-08T19:25:52.204091+0000","last_became_active":"2023-11-08T19:25:49.886169+0000","last_became_peered":"2023-11-08T19:25:49.886169+0000","last_unstale":"2023-11-08T19:25:52.204091+0000","last_undegraded":"2023-11-08T19:25:52.204091+0000","last_fullsized":"2023-11-08T19:25:52.204091+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:19:41.552717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.991670+0000","last_change":"2023-11-08T19:25:50.885893+0000","last_active":"2023-11-08T19:25:51.991670+0000","last_peered":"2023-11-08T19:25:51.991670+0000","last_clean":"2023-11-08T19:25:51.991670+0000","last_became_active":"2023-11-08T19:25:50.880205+0000","last_became_peered":"2023-11-08T19:25:50.880205+0000","last_unstale":"2023-11-08T19:25:51.991670+0000","last_undegraded":"2023-11-08T19:25:51.991670+0000","last_fullsized":"2023-11-08T19:25:51.991670+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:55:31.178870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"32'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.205247+0000","last_change":"2023-11-08T19:25:49.879256+0000","last_active":"2023-11-08T19:25:52.205247+0000","last_peered":"2023-11-08T19:25:52.205247+0000","last_clean":"2023-11-08T19:25:52.205247+0000","last_became_active":"2023-11-08T19:25:49.878963+0000","last_became_peered":"2023-11-08T19:25:49.878963+0000","last_unstale":"2023-11-08T19:25:52.205247+0000","last_undegraded":"2023-11-08T19:25:52.205247+0000","last_fullsized":"2023-11-08T19:25:52.205247+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:12:39.742779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.203953+0000","last_change":"2023-11-08T19:25:50.891183+0000","last_active":"2023-11-08T19:25:52.203953+0000","last_peered":"2023-11-08T19:25:52.203953+0000","last_clean":"2023-11-08T19:25:52.203953+0000","last_became_active":"2023-11-08T19:25:50.890889+0000","last_became_peered":"2023-11-08T19:25:50.890889+0000","last_unstale":"2023-11-08T19:25:52.203953+0000","last_undegraded":"2023-11-08T19:25:52.203953+0000","last_fullsized":"2023-11-08T19:25:52.203953+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:25:16.223226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.991756+0000","last_change":"2023-11-08T19:25:49.880102+0000","last_active":"2023-11-08T19:25:51.991756+0000","last_peered":"2023-11-08T19:25:51.991756+0000","last_clean":"2023-11-08T19:25:51.991756+0000","last_became_active":"2023-11-08T19:25:49.879816+0000","last_became_peered":"2023-11-08T19:25:49.879816+0000","last_unstale":"2023-11-08T19:25:51.991756+0000","last_undegraded":"2023-11-08T19:25:51.991756+0000","last_fullsized":"2023-11-08T19:25:51.991756+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:37:09.254408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:25:49.854746+0000","last_change":"2023-11-08T19:25:49.854746+0000","last_active":"2023-11-08T19:25:49.854746+0000","last_peered":"2023-11-08T19:25:49.854746+0000","last_clean":"2023-11-08T19:25:49.854746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:25:49.854746+0000","last_undegraded":"2023-11-08T19:25:49.854746+0000","last_fullsized":"2023-11-08T19:25:49.854746+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.203841+0000","last_change":"2023-11-08T19:25:49.884359+0000","last_active":"2023-11-08T19:25:52.203841+0000","last_peered":"2023-11-08T19:25:52.203841+0000","last_clean":"2023-11-08T19:25:52.203841+0000","last_became_active":"2023-11-08T19:25:49.884119+0000","last_became_peered":"2023-11-08T19:25:49.884119+0000","last_unstale":"2023-11-08T19:25:52.203841+0000","last_undegraded":"2023-11-08T19:25:52.203841+0000","last_fullsized":"2023-11-08T19:25:52.203841+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:55:30.168605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:25:49.854746+0000","last_change":"2023-11-08T19:25:49.854746+0000","last_active":"2023-11-08T19:25:49.854746+0000","last_peered":"2023-11-08T19:25:49.854746+0000","last_clean":"2023-11-08T19:25:49.854746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:25:49.854746+0000","last_undegraded":"2023-11-08T19:25:49.854746+0000","last_fullsized":"2023-11-08T19:25:49.854746+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.10","version":"32'8","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.203707+0000","last_change":"2023-11-08T19:25:49.886483+0000","last_active":"2023-11-08T19:25:52.203707+0000","last_peered":"2023-11-08T19:25:52.203707+0000","last_clean":"2023-11-08T19:25:52.203707+0000","last_became_active":"2023-11-08T19:25:49.886209+0000","last_became_peered":"2023-11-08T19:25:49.886209+0000","last_unstale":"2023-11-08T19:25:52.203707+0000","last_undegraded":"2023-11-08T19:25:52.203707+0000","last_fullsized":"2023-11-08T19:25:52.203707+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:49:40.465332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.205492+0000","last_change":"2023-11-08T19:25:50.884739+0000","last_active":"2023-11-08T19:25:52.205492+0000","last_peered":"2023-11-08T19:25:52.205492+0000","last_clean":"2023-11-08T19:25:52.205492+0000","last_became_active":"2023-11-08T19:25:50.884476+0000","last_became_peered":"2023-11-08T19:25:50.884476+0000","last_unstale":"2023-11-08T19:25:52.205492+0000","last_undegraded":"2023-11-08T19:25:52.205492+0000","last_fullsized":"2023-11-08T19:25:52.205492+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:55:31.178870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"32'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.992046+0000","last_change":"2023-11-08T19:25:49.884770+0000","last_active":"2023-11-08T19:25:51.992046+0000","last_peered":"2023-11-08T19:25:51.992046+0000","last_clean":"2023-11-08T19:25:51.992046+0000","last_became_active":"2023-11-08T19:25:49.884476+0000","last_became_peered":"2023-11-08T19:25:49.884476+0000","last_unstale":"2023-11-08T19:25:51.992046+0000","last_undegraded":"2023-11-08T19:25:51.992046+0000","last_fullsized":"2023-11-08T19:25:51.992046+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:18:56.855441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204945+0000","last_change":"2023-11-08T19:25:50.877298+0000","last_active":"2023-11-08T19:25:52.204945+0000","last_peered":"2023-11-08T19:25:52.204945+0000","last_clean":"2023-11-08T19:25:52.204945+0000","last_became_active":"2023-11-08T19:25:50.877018+0000","last_became_peered":"2023-11-08T19:25:50.877018+0000","last_unstale":"2023-11-08T19:25:52.204945+0000","last_undegraded":"2023-11-08T19:25:52.204945+0000","last_fullsized":"2023-11-08T19:25:52.204945+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:25:07.133280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"32'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.205477+0000","last_change":"2023-11-08T19:25:49.878520+0000","last_active":"2023-11-08T19:25:52.205477+0000","last_peered":"2023-11-08T19:25:52.205477+0000","last_clean":"2023-11-08T19:25:52.205477+0000","last_became_active":"2023-11-08T19:25:49.878244+0000","last_became_peered":"2023-11-08T19:25:49.878244+0000","last_unstale":"2023-11-08T19:25:52.205477+0000","last_undegraded":"2023-11-08T19:25:52.205477+0000","last_fullsized":"2023-11-08T19:25:52.205477+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:23:30.383150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.991899+0000","last_change":"2023-11-08T19:25:50.878056+0000","last_active":"2023-11-08T19:25:51.991899+0000","last_peered":"2023-11-08T19:25:51.991899+0000","last_clean":"2023-11-08T19:25:51.991899+0000","last_became_active":"2023-11-08T19:25:50.877738+0000","last_became_peered":"2023-11-08T19:25:50.877738+0000","last_unstale":"2023-11-08T19:25:51.991899+0000","last_undegraded":"2023-11-08T19:25:51.991899+0000","last_fullsized":"2023-11-08T19:25:51.991899+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:04:01.492989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"32'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.205456+0000","last_change":"2023-11-08T19:25:49.881744+0000","last_active":"2023-11-08T19:25:52.205456+0000","last_peered":"2023-11-08T19:25:52.205456+0000","last_clean":"2023-11-08T19:25:52.205456+0000","last_became_active":"2023-11-08T19:25:49.881436+0000","last_became_peered":"2023-11-08T19:25:49.881436+0000","last_unstale":"2023-11-08T19:25:52.205456+0000","last_undegraded":"2023-11-08T19:25:52.205456+0000","last_fullsized":"2023-11-08T19:25:52.205456+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:08:07.992929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.876822+0000","last_change":"2023-11-08T19:25:50.881348+0000","last_active":"2023-11-08T19:25:51.876822+0000","last_peered":"2023-11-08T19:25:51.876822+0000","last_clean":"2023-11-08T19:25:51.876822+0000","last_became_active":"2023-11-08T19:25:50.881064+0000","last_became_peered":"2023-11-08T19:25:50.881064+0000","last_unstale":"2023-11-08T19:25:51.876822+0000","last_undegraded":"2023-11-08T19:25:51.876822+0000","last_fullsized":"2023-11-08T19:25:51.876822+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:23:31.393415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.205574+0000","last_change":"2023-11-08T19:25:49.884654+0000","last_active":"2023-11-08T19:25:52.205574+0000","last_peered":"2023-11-08T19:25:52.205574+0000","last_clean":"2023-11-08T19:25:52.205574+0000","last_became_active":"2023-11-08T19:25:49.884322+0000","last_became_peered":"2023-11-08T19:25:49.884322+0000","last_unstale":"2023-11-08T19:25:52.205574+0000","last_undegraded":"2023-11-08T19:25:52.205574+0000","last_fullsized":"2023-11-08T19:25:52.205574+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:49:07.738959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.876689+0000","last_change":"2023-11-08T19:25:50.878414+0000","last_active":"2023-11-08T19:25:51.876689+0000","last_peered":"2023-11-08T19:25:51.876689+0000","last_clean":"2023-11-08T19:25:51.876689+0000","last_became_active":"2023-11-08T19:25:50.878145+0000","last_became_peered":"2023-11-08T19:25:50.878145+0000","last_unstale":"2023-11-08T19:25:51.876689+0000","last_undegraded":"2023-11-08T19:25:51.876689+0000","last_fullsized":"2023-11-08T19:25:51.876689+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:49:08.749224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"32'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.989615+0000","last_change":"2023-11-08T19:25:49.878923+0000","last_active":"2023-11-08T19:25:51.989615+0000","last_peered":"2023-11-08T19:25:51.989615+0000","last_clean":"2023-11-08T19:25:51.989615+0000","last_became_active":"2023-11-08T19:25:49.878167+0000","last_became_peered":"2023-11-08T19:25:49.878167+0000","last_unstale":"2023-11-08T19:25:51.989615+0000","last_undegraded":"2023-11-08T19:25:51.989615+0000","last_fullsized":"2023-11-08T19:25:51.989615+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:51:26.807463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.876711+0000","last_change":"2023-11-08T19:25:50.887699+0000","last_active":"2023-11-08T19:25:51.876711+0000","last_peered":"2023-11-08T19:25:51.876711+0000","last_clean":"2023-11-08T19:25:51.876711+0000","last_became_active":"2023-11-08T19:25:50.887423+0000","last_became_peered":"2023-11-08T19:25:50.887423+0000","last_unstale":"2023-11-08T19:25:51.876711+0000","last_undegraded":"2023-11-08T19:25:51.876711+0000","last_fullsized":"2023-11-08T19:25:51.876711+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:08:09.003194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-11-08T19:25:48.857323+0000","last_change":"2023-11-08T19:25:48.857323+0000","last_active":"2023-11-08T19:25:48.844481+0000","last_peered":"2023-11-08T19:25:48.844481+0000","last_clean":"2023-11-08T19:25:48.844481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:25:48.857323+0000","last_undegraded":"2023-11-08T19:25:48.857323+0000","last_fullsized":"2023-11-08T19:25:48.857323+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.876619+0000","last_change":"2023-11-08T19:25:50.888597+0000","last_active":"2023-11-08T19:25:51.876619+0000","last_peered":"2023-11-08T19:25:51.876619+0000","last_clean":"2023-11-08T19:25:51.876619+0000","last_became_active":"2023-11-08T19:25:50.887065+0000","last_became_peered":"2023-11-08T19:25:50.887065+0000","last_unstale":"2023-11-08T19:25:51.876619+0000","last_undegraded":"2023-11-08T19:25:51.876619+0000","last_fullsized":"2023-11-08T19:25:51.876619+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:02:25.988939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.205088+0000","last_change":"2023-11-08T19:25:49.884053+0000","last_active":"2023-11-08T19:25:52.205088+0000","last_peered":"2023-11-08T19:25:52.205088+0000","last_clean":"2023-11-08T19:25:52.205088+0000","last_became_active":"2023-11-08T19:25:49.883822+0000","last_became_peered":"2023-11-08T19:25:49.883822+0000","last_unstale":"2023-11-08T19:25:52.205088+0000","last_undegraded":"2023-11-08T19:25:52.205088+0000","last_fullsized":"2023-11-08T19:25:52.205088+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:12:47.116061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:25:49.854746+0000","last_change":"2023-11-08T19:25:49.854746+0000","last_active":"2023-11-08T19:25:49.854746+0000","last_peered":"2023-11-08T19:25:49.854746+0000","last_clean":"2023-11-08T19:25:49.854746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:25:49.854746+0000","last_undegraded":"2023-11-08T19:25:49.854746+0000","last_fullsized":"2023-11-08T19:25:49.854746+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204303+0000","last_change":"2023-11-08T19:25:49.879327+0000","last_active":"2023-11-08T19:25:52.204303+0000","last_peered":"2023-11-08T19:25:52.204303+0000","last_clean":"2023-11-08T19:25:52.204303+0000","last_became_active":"2023-11-08T19:25:49.879054+0000","last_became_peered":"2023-11-08T19:25:49.879054+0000","last_unstale":"2023-11-08T19:25:52.204303+0000","last_undegraded":"2023-11-08T19:25:52.204303+0000","last_fullsized":"2023-11-08T19:25:52.204303+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:09:04.368768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.876752+0000","last_change":"2023-11-08T19:25:50.874836+0000","last_active":"2023-11-08T19:25:51.876752+0000","last_peered":"2023-11-08T19:25:51.876752+0000","last_clean":"2023-11-08T19:25:51.876752+0000","last_became_active":"2023-11-08T19:25:50.874272+0000","last_became_peered":"2023-11-08T19:25:50.874272+0000","last_unstale":"2023-11-08T19:25:51.876752+0000","last_undegraded":"2023-11-08T19:25:51.876752+0000","last_fullsized":"2023-11-08T19:25:51.876752+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:17:53.651518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"32'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.991956+0000","last_change":"2023-11-08T19:25:49.880740+0000","last_active":"2023-11-08T19:25:51.991956+0000","last_peered":"2023-11-08T19:25:51.991956+0000","last_clean":"2023-11-08T19:25:51.991956+0000","last_became_active":"2023-11-08T19:25:49.880458+0000","last_became_peered":"2023-11-08T19:25:49.880458+0000","last_unstale":"2023-11-08T19:25:51.991956+0000","last_undegraded":"2023-11-08T19:25:51.991956+0000","last_fullsized":"2023-11-08T19:25:51.991956+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:20:42.164809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.205146+0000","last_change":"2023-11-08T19:25:50.883014+0000","last_active":"2023-11-08T19:25:52.205146+0000","last_peered":"2023-11-08T19:25:52.205146+0000","last_clean":"2023-11-08T19:25:52.205146+0000","last_became_active":"2023-11-08T19:25:50.882737+0000","last_became_peered":"2023-11-08T19:25:50.882737+0000","last_unstale":"2023-11-08T19:25:52.205146+0000","last_undegraded":"2023-11-08T19:25:52.205146+0000","last_fullsized":"2023-11-08T19:25:52.205146+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:38:37.341013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"33'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.991098+0000","last_change":"2023-11-08T19:25:49.880299+0000","last_active":"2023-11-08T19:25:51.991098+0000","last_peered":"2023-11-08T19:25:51.991098+0000","last_clean":"2023-11-08T19:25:51.991098+0000","last_became_active":"2023-11-08T19:25:49.878038+0000","last_became_peered":"2023-11-08T19:25:49.878038+0000","last_unstale":"2023-11-08T19:25:51.991098+0000","last_undegraded":"2023-11-08T19:25:51.991098+0000","last_fullsized":"2023-11-08T19:25:51.991098+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:12:39.742779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.989626+0000","last_change":"2023-11-08T19:25:50.888755+0000","last_active":"2023-11-08T19:25:51.989626+0000","last_peered":"2023-11-08T19:25:51.989626+0000","last_clean":"2023-11-08T19:25:51.989626+0000","last_became_active":"2023-11-08T19:25:50.888494+0000","last_became_peered":"2023-11-08T19:25:50.888494+0000","last_unstale":"2023-11-08T19:25:51.989626+0000","last_undegraded":"2023-11-08T19:25:51.989626+0000","last_fullsized":"2023-11-08T19:25:51.989626+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:09:05.379033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.991905+0000","last_change":"2023-11-08T19:25:49.881699+0000","last_active":"2023-11-08T19:25:51.991905+0000","last_peered":"2023-11-08T19:25:51.991905+0000","last_clean":"2023-11-08T19:25:51.991905+0000","last_became_active":"2023-11-08T19:25:49.881062+0000","last_became_peered":"2023-11-08T19:25:49.881062+0000","last_unstale":"2023-11-08T19:25:51.991905+0000","last_undegraded":"2023-11-08T19:25:51.991905+0000","last_fullsized":"2023-11-08T19:25:51.991905+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:49:44.049659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:25:49.854746+0000","last_change":"2023-11-08T19:25:49.854746+0000","last_active":"2023-11-08T19:25:49.854746+0000","last_peered":"2023-11-08T19:25:49.854746+0000","last_clean":"2023-11-08T19:25:49.854746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:25:49.854746+0000","last_undegraded":"2023-11-08T19:25:49.854746+0000","last_fullsized":"2023-11-08T19:25:49.854746+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.989575+0000","last_change":"2023-11-08T19:25:49.879113+0000","last_active":"2023-11-08T19:25:51.989575+0000","last_peered":"2023-11-08T19:25:51.989575+0000","last_clean":"2023-11-08T19:25:51.989575+0000","last_became_active":"2023-11-08T19:25:49.878579+0000","last_became_peered":"2023-11-08T19:25:49.878579+0000","last_unstale":"2023-11-08T19:25:51.989575+0000","last_undegraded":"2023-11-08T19:25:51.989575+0000","last_fullsized":"2023-11-08T19:25:51.989575+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:12:47.116061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.989707+0000","last_change":"2023-11-08T19:25:50.880600+0000","last_active":"2023-11-08T19:25:51.989707+0000","last_peered":"2023-11-08T19:25:51.989707+0000","last_clean":"2023-11-08T19:25:51.989707+0000","last_became_active":"2023-11-08T19:25:50.880384+0000","last_became_peered":"2023-11-08T19:25:50.880384+0000","last_unstale":"2023-11-08T19:25:51.989707+0000","last_undegraded":"2023-11-08T19:25:51.989707+0000","last_fullsized":"2023-11-08T19:25:51.989707+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:20:43.175074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.991693+0000","last_change":"2023-11-08T19:25:49.877196+0000","last_active":"2023-11-08T19:25:51.991693+0000","last_peered":"2023-11-08T19:25:51.991693+0000","last_clean":"2023-11-08T19:25:51.991693+0000","last_became_active":"2023-11-08T19:25:49.876517+0000","last_became_peered":"2023-11-08T19:25:49.876517+0000","last_unstale":"2023-11-08T19:25:51.991693+0000","last_undegraded":"2023-11-08T19:25:51.991693+0000","last_fullsized":"2023-11-08T19:25:51.991693+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:00:52.169251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.203760+0000","last_change":"2023-11-08T19:25:50.893157+0000","last_active":"2023-11-08T19:25:52.203760+0000","last_peered":"2023-11-08T19:25:52.203760+0000","last_clean":"2023-11-08T19:25:52.203760+0000","last_became_active":"2023-11-08T19:25:50.892637+0000","last_became_peered":"2023-11-08T19:25:50.892637+0000","last_unstale":"2023-11-08T19:25:52.203760+0000","last_undegraded":"2023-11-08T19:25:52.203760+0000","last_fullsized":"2023-11-08T19:25:52.203760+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:50:52.902620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.205077+0000","last_change":"2023-11-08T19:25:49.881923+0000","last_active":"2023-11-08T19:25:52.205077+0000","last_peered":"2023-11-08T19:25:52.205077+0000","last_clean":"2023-11-08T19:25:52.205077+0000","last_became_active":"2023-11-08T19:25:49.881670+0000","last_became_peered":"2023-11-08T19:25:49.881670+0000","last_unstale":"2023-11-08T19:25:52.205077+0000","last_undegraded":"2023-11-08T19:25:52.205077+0000","last_fullsized":"2023-11-08T19:25:52.205077+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:51:26.807463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204772+0000","last_change":"2023-11-08T19:25:50.883910+0000","last_active":"2023-11-08T19:25:52.204772+0000","last_peered":"2023-11-08T19:25:52.204772+0000","last_clean":"2023-11-08T19:25:52.204772+0000","last_became_active":"2023-11-08T19:25:50.883615+0000","last_became_peered":"2023-11-08T19:25:50.883615+0000","last_unstale":"2023-11-08T19:25:52.204772+0000","last_undegraded":"2023-11-08T19:25:52.204772+0000","last_fullsized":"2023-11-08T19:25:52.204772+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:20:43.175074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.203857+0000","last_change":"2023-11-08T19:25:49.884441+0000","last_active":"2023-11-08T19:25:52.203857+0000","last_peered":"2023-11-08T19:25:52.203857+0000","last_clean":"2023-11-08T19:25:52.203857+0000","last_became_active":"2023-11-08T19:25:49.884125+0000","last_became_peered":"2023-11-08T19:25:49.884125+0000","last_unstale":"2023-11-08T19:25:52.203857+0000","last_undegraded":"2023-11-08T19:25:52.203857+0000","last_fullsized":"2023-11-08T19:25:52.203857+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:52:19.324495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:25:49.854746+0000","last_change":"2023-11-08T19:25:49.854746+0000","last_active":"2023-11-08T19:25:49.854746+0000","last_peered":"2023-11-08T19:25:49.854746+0000","last_clean":"2023-11-08T19:25:49.854746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:25:49.854746+0000","last_undegraded":"2023-11-08T19:25:49.854746+0000","last_fullsized":"2023-11-08T19:25:49.854746+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-11-08T19:25:48.857272+0000","last_change":"2023-11-08T19:25:48.857272+0000","last_active":"2023-11-08T19:25:48.844481+0000","last_peered":"2023-11-08T19:25:48.844481+0000","last_clean":"2023-11-08T19:25:48.844481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:25:48.857272+0000","last_undegraded":"2023-11-08T19:25:48.857272+0000","last_fullsized":"2023-11-08T19:25:48.857272+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":175,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-11-08T19:25:43.589068+0000","last_change":"2023-11-08T19:24:45.888122+0000","last_active":"2023-11-08T19:25:43.589068+0000","last_peered":"2023-11-08T19:25:43.589068+0000","last_clean":"2023-11-08T19:25:43.589068+0000","last_became_active":"2023-11-08T19:24:45.887048+0000","last_became_peered":"2023-11-08T19:24:45.887048+0000","last_unstale":"2023-11-08T19:25:43.589068+0000","last_undegraded":"2023-11-08T19:25:43.589068+0000","last_fullsized":"2023-11-08T19:25:43.589068+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:44.575133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:44.575133+0000","last_clean_scrub_stamp":"2023-11-08T19:24:44.575133+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:19:20.202062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":96,"acting":96,"num_store_stats":6},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":1828,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":4114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":35,"ondisk_log_size":35,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574865,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27928,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343912,"statfs":{"total":96636764160,"available":96608165888,"internally_reserved":0,"allocated":929792,"data_stored":716892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.60999999999999999}]},{"osd":1,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.76100000000000001}]},{"osd":2,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.63200000000000001}]},{"osd":3,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.746},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.67800000000000005}]},{"osd":4,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.58299999999999996}]},{"osd":5,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.71999999999999997}]},{"osd":6,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.64400000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":335872,"data_stored":126524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":21.359999999999999,"5min":21.359999999999999,"15min":21.359999999999999},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":10.329000000000001,"5min":10.329000000000001,"15min":10.329000000000001},"last":0.46700000000000003}]},{"osd":1,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":3.4239999999999999,"5min":3.4239999999999999,"15min":3.4239999999999999},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":3.4809999999999999,"5min":3.4809999999999999,"15min":3.4809999999999999},"last":0.66500000000000004}]},{"osd":2,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":10.398,"5min":10.398,"15min":10.398},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":5.0170000000000003,"5min":5.0170000000000003,"15min":5.0170000000000003},"last":0.60999999999999999}]},{"osd":3,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":3.5710000000000002,"5min":3.5710000000000002,"15min":3.5710000000000002},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":10.368,"5min":10.368,"15min":10.368},"last":0.70299999999999996}]},{"osd":4,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.78500000000000003}]},{"osd":5,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.753},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.82099999999999995}]},{"osd":7,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.80700000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574866,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27404,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344436,"statfs":{"total":96636764160,"available":96608702464,"internally_reserved":0,"allocated":389120,"data_stored":147500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":1.389,"5min":1.389,"15min":1.389},"last":0.499},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":9.1370000000000005,"5min":9.1370000000000005,"15min":9.1370000000000005},"last":0.39400000000000002}]},{"osd":2,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.433,"5min":0.433,"15min":0.433},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":1.425,"5min":1.425,"15min":1.425},"last":0.42699999999999999}]},{"osd":3,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":11.957000000000001,"5min":11.957000000000001,"15min":11.957000000000001},"last":0.47699999999999998},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":18.039000000000001,"5min":18.039000000000001,"15min":18.039000000000001},"last":0.46300000000000002}]},{"osd":4,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":6.181,"5min":6.181,"15min":6.181},"last":1.4370000000000001},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.488,"5min":1.488,"15min":1.488},"last":1.411}]},{"osd":5,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":9.2420000000000009,"5min":9.2420000000000009,"15min":9.2420000000000009},"last":1.421},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":6.2110000000000003,"5min":6.2110000000000003,"15min":6.2110000000000003},"last":1.2969999999999999}]},{"osd":6,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":9.1880000000000006,"5min":9.1880000000000006,"15min":9.1880000000000006},"last":1.4530000000000001},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.462,"5min":1.462,"15min":1.462},"last":1.272}]},{"osd":7,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":6.1479999999999997,"5min":6.1479999999999997,"15min":6.1479999999999997},"last":1.3979999999999999},{"interface":"front","average":{"1min":1.028,"5min":1.028,"15min":1.028},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":9.2170000000000005,"5min":9.2170000000000005,"15min":9.2170000000000005},"last":1.353}]}]},{"osd":0,"up_from":13,"seq":55834574866,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27972,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343868,"statfs":{"total":96636764160,"available":96608120832,"internally_reserved":0,"allocated":966656,"data_stored":736234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":4.1200000000000001,"5min":4.1200000000000001,"15min":4.1200000000000001},"last":0.32200000000000001},{"interface":"front","average":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":16.940999999999999,"5min":16.940999999999999,"15min":16.940999999999999},"last":0.44500000000000001}]},{"osd":2,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.37},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":7.0209999999999999,"5min":7.0209999999999999,"15min":7.0209999999999999},"last":0.41399999999999998}]},{"osd":3,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":4.0940000000000003,"5min":4.0940000000000003,"15min":4.0940000000000003},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.432}]},{"osd":4,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":4.0049999999999999,"5min":4.0049999999999999,"15min":4.0049999999999999},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.79200000000000004}]},{"osd":5,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":4.3090000000000002,"5min":4.3090000000000002,"15min":4.3090000000000002},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":4.0709999999999997,"5min":4.0709999999999997,"15min":4.0709999999999997},"last":0.67300000000000004}]},{"osd":6,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":4.2560000000000002,"5min":4.2560000000000002,"15min":4.2560000000000002},"last":0.75900000000000001}]},{"osd":7,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":4.2850000000000001,"5min":4.2850000000000001,"15min":4.2850000000000001},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":4.0460000000000003,"5min":4.0460000000000003,"15min":4.0460000000000003},"last":0.80600000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574866,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27392,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":376832,"data_stored":145936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"min":{"1min":0.20100000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":6.0999999999999996,"5min":6.0999999999999996,"15min":6.0999999999999996},"last":0.249}]},{"osd":1,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"last":0.28799999999999998},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.624,"5min":0.624,"15min":0.624},"last":0.313}]},{"osd":3,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.38300000000000001}]},{"osd":4,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.55500000000000005}]},{"osd":5,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.57599999999999996}]},{"osd":6,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.65100000000000002}]},{"osd":7,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.628},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.80000000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574867,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27404,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344436,"statfs":{"total":96636764160,"available":96608702464,"internally_reserved":0,"allocated":389120,"data_stored":147500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:25:46 2023","interfaces":[{"interface":"back","average":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.626,"5min":0.626,"15min":0.626},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.33100000000000002}]},{"osd":1,"last update":"Wed Nov 8 19:25:46 2023","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.34999999999999998}]},{"osd":2,"last update":"Wed Nov 8 19:25:46 2023","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.37,"15min":0.37},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"last":0.40000000000000002}]},{"osd":4,"last update":"Wed Nov 8 19:25:46 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.70599999999999996}]},{"osd":5,"last update":"Wed Nov 8 19:25:46 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.56699999999999995}]},{"osd":6,"last update":"Wed Nov 8 19:25:46 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.74199999999999999}]},{"osd":7,"last update":"Wed Nov 8 19:25:46 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.66400000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574866,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":380928,"data_stored":145970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:25:47 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":6.5970000000000004,"5min":6.5970000000000004,"15min":6.5970000000000004},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":2.5030000000000001,"5min":2.5030000000000001,"15min":2.5030000000000001},"last":0.75}]},{"osd":1,"last update":"Wed Nov 8 19:25:47 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.72599999999999998}]},{"osd":2,"last update":"Wed Nov 8 19:25:47 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":6.6920000000000002,"5min":6.6920000000000002,"15min":6.6920000000000002},"last":0.871}]},{"osd":3,"last update":"Wed Nov 8 19:25:47 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.91700000000000004}]},{"osd":5,"last update":"Wed Nov 8 19:25:47 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.65900000000000003}]},{"osd":6,"last update":"Wed Nov 8 19:25:47 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.78100000000000003}]},{"osd":7,"last update":"Wed Nov 8 19:25:47 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.85299999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574866,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":380928,"data_stored":146538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.627},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.438}]},{"osd":1,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.50600000000000001}]},{"osd":2,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.46300000000000002}]},{"osd":3,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.58299999999999996}]},{"osd":4,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.59699999999999998}]},{"osd":6,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.67700000000000005}]},{"osd":7,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.61499999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-08T19:25:55.734 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3f is not active+clean 2023-11-08T19:25:55.734 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-08T19:25:49.854746+0000', 'last_change': '2023-11-08T19:25:49.854746+0000', 'last_active': '2023-11-08T19:25:49.854746+0000', 'last_peered': '2023-11-08T19:25:49.854746+0000', 'last_clean': '2023-11-08T19:25:49.854746+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:25:49.854746+0000', 'last_undegraded': '2023-11-08T19:25:49.854746+0000', 'last_fullsized': '2023-11-08T19:25:49.854746+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:25:49.854746+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:25:49.854746+0000', 'last_clean_scrub_stamp': '2023-11-08T19:25:49.854746+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-08T19:25:55.735 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.39 is not active+clean 2023-11-08T19:25:55.736 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-08T19:25:49.854746+0000', 'last_change': '2023-11-08T19:25:49.854746+0000', 'last_active': '2023-11-08T19:25:49.854746+0000', 'last_peered': '2023-11-08T19:25:49.854746+0000', 'last_clean': '2023-11-08T19:25:49.854746+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:25:49.854746+0000', 'last_undegraded': '2023-11-08T19:25:49.854746+0000', 'last_fullsized': '2023-11-08T19:25:49.854746+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:25:49.854746+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:25:49.854746+0000', 'last_clean_scrub_stamp': '2023-11-08T19:25:49.854746+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-08T19:25:55.736 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.38 is not active+clean 2023-11-08T19:25:55.737 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-08T19:25:49.854746+0000', 'last_change': '2023-11-08T19:25:49.854746+0000', 'last_active': '2023-11-08T19:25:49.854746+0000', 'last_peered': '2023-11-08T19:25:49.854746+0000', 'last_clean': '2023-11-08T19:25:49.854746+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:25:49.854746+0000', 'last_undegraded': '2023-11-08T19:25:49.854746+0000', 'last_fullsized': '2023-11-08T19:25:49.854746+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:25:49.854746+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:25:49.854746+0000', 'last_clean_scrub_stamp': '2023-11-08T19:25:49.854746+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-08T19:25:55.737 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.35 is not active+clean 2023-11-08T19:25:55.738 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-08T19:25:49.854746+0000', 'last_change': '2023-11-08T19:25:49.854746+0000', 'last_active': '2023-11-08T19:25:49.854746+0000', 'last_peered': '2023-11-08T19:25:49.854746+0000', 'last_clean': '2023-11-08T19:25:49.854746+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:25:49.854746+0000', 'last_undegraded': '2023-11-08T19:25:49.854746+0000', 'last_fullsized': '2023-11-08T19:25:49.854746+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:25:49.854746+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:25:49.854746+0000', 'last_clean_scrub_stamp': '2023-11-08T19:25:49.854746+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-08T19:25:55.739 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.30 is not active+clean 2023-11-08T19:25:55.739 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-08T19:25:49.854746+0000', 'last_change': '2023-11-08T19:25:49.854746+0000', 'last_active': '2023-11-08T19:25:49.854746+0000', 'last_peered': '2023-11-08T19:25:49.854746+0000', 'last_clean': '2023-11-08T19:25:49.854746+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:25:49.854746+0000', 'last_undegraded': '2023-11-08T19:25:49.854746+0000', 'last_fullsized': '2023-11-08T19:25:49.854746+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:25:49.854746+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:25:49.854746+0000', 'last_clean_scrub_stamp': '2023-11-08T19:25:49.854746+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-08T19:25:55.739 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2f is not active+clean 2023-11-08T19:25:55.740 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-08T19:25:49.854746+0000', 'last_change': '2023-11-08T19:25:49.854746+0000', 'last_active': '2023-11-08T19:25:49.854746+0000', 'last_peered': '2023-11-08T19:25:49.854746+0000', 'last_clean': '2023-11-08T19:25:49.854746+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:25:49.854746+0000', 'last_undegraded': '2023-11-08T19:25:49.854746+0000', 'last_fullsized': '2023-11-08T19:25:49.854746+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:25:49.854746+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:25:49.854746+0000', 'last_clean_scrub_stamp': '2023-11-08T19:25:49.854746+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-08T19:25:55.740 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.22 is not active+clean 2023-11-08T19:25:55.741 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-08T19:25:49.854746+0000', 'last_change': '2023-11-08T19:25:49.854746+0000', 'last_active': '2023-11-08T19:25:49.854746+0000', 'last_peered': '2023-11-08T19:25:49.854746+0000', 'last_clean': '2023-11-08T19:25:49.854746+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:25:49.854746+0000', 'last_undegraded': '2023-11-08T19:25:49.854746+0000', 'last_fullsized': '2023-11-08T19:25:49.854746+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:25:49.854746+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:25:49.854746+0000', 'last_clean_scrub_stamp': '2023-11-08T19:25:49.854746+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-08T19:25:55.741 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.21 is not active+clean 2023-11-08T19:25:55.742 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-08T19:25:49.854746+0000', 'last_change': '2023-11-08T19:25:49.854746+0000', 'last_active': '2023-11-08T19:25:49.854746+0000', 'last_peered': '2023-11-08T19:25:49.854746+0000', 'last_clean': '2023-11-08T19:25:49.854746+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:25:49.854746+0000', 'last_undegraded': '2023-11-08T19:25:49.854746+0000', 'last_fullsized': '2023-11-08T19:25:49.854746+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:25:49.854746+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:25:49.854746+0000', 'last_clean_scrub_stamp': '2023-11-08T19:25:49.854746+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-08T19:25:55.742 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.20 is not active+clean 2023-11-08T19:25:55.742 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-08T19:25:49.854746+0000', 'last_change': '2023-11-08T19:25:49.854746+0000', 'last_active': '2023-11-08T19:25:49.854746+0000', 'last_peered': '2023-11-08T19:25:49.854746+0000', 'last_clean': '2023-11-08T19:25:49.854746+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:25:49.854746+0000', 'last_undegraded': '2023-11-08T19:25:49.854746+0000', 'last_fullsized': '2023-11-08T19:25:49.854746+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:25:49.854746+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:25:49.854746+0000', 'last_clean_scrub_stamp': '2023-11-08T19:25:49.854746+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-08T19:25:55.743 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1f is not active+clean 2023-11-08T19:25:55.743 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2023-11-08T19:25:48.857285+0000', 'last_change': '2023-11-08T19:25:48.857285+0000', 'last_active': '2023-11-08T19:25:48.844481+0000', 'last_peered': '2023-11-08T19:25:48.844481+0000', 'last_clean': '2023-11-08T19:25:48.844481+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:25:48.857285+0000', 'last_undegraded': '2023-11-08T19:25:48.857285+0000', 'last_fullsized': '2023-11-08T19:25:48.857285+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:25:48.844481+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:25:48.844481+0000', 'last_clean_scrub_stamp': '2023-11-08T19:25:48.844481+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-08T19:25:55.744 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1f is not active+clean 2023-11-08T19:25:55.744 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-08T19:25:49.854746+0000', 'last_change': '2023-11-08T19:25:49.854746+0000', 'last_active': '2023-11-08T19:25:49.854746+0000', 'last_peered': '2023-11-08T19:25:49.854746+0000', 'last_clean': '2023-11-08T19:25:49.854746+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:25:49.854746+0000', 'last_undegraded': '2023-11-08T19:25:49.854746+0000', 'last_fullsized': '2023-11-08T19:25:49.854746+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:25:49.854746+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:25:49.854746+0000', 'last_clean_scrub_stamp': '2023-11-08T19:25:49.854746+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-08T19:25:55.745 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1a is not active+clean 2023-11-08T19:25:55.745 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-08T19:25:49.854746+0000', 'last_change': '2023-11-08T19:25:49.854746+0000', 'last_active': '2023-11-08T19:25:49.854746+0000', 'last_peered': '2023-11-08T19:25:49.854746+0000', 'last_clean': '2023-11-08T19:25:49.854746+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:25:49.854746+0000', 'last_undegraded': '2023-11-08T19:25:49.854746+0000', 'last_fullsized': '2023-11-08T19:25:49.854746+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:25:49.854746+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:25:49.854746+0000', 'last_clean_scrub_stamp': '2023-11-08T19:25:49.854746+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-08T19:25:55.746 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.10 is not active+clean 2023-11-08T19:25:55.746 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-08T19:25:49.854746+0000', 'last_change': '2023-11-08T19:25:49.854746+0000', 'last_active': '2023-11-08T19:25:49.854746+0000', 'last_peered': '2023-11-08T19:25:49.854746+0000', 'last_clean': '2023-11-08T19:25:49.854746+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:25:49.854746+0000', 'last_undegraded': '2023-11-08T19:25:49.854746+0000', 'last_fullsized': '2023-11-08T19:25:49.854746+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:25:49.854746+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:25:49.854746+0000', 'last_clean_scrub_stamp': '2023-11-08T19:25:49.854746+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-08T19:25:55.747 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.11 is not active+clean 2023-11-08T19:25:55.747 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-08T19:25:49.854746+0000', 'last_change': '2023-11-08T19:25:49.854746+0000', 'last_active': '2023-11-08T19:25:49.854746+0000', 'last_peered': '2023-11-08T19:25:49.854746+0000', 'last_clean': '2023-11-08T19:25:49.854746+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:25:49.854746+0000', 'last_undegraded': '2023-11-08T19:25:49.854746+0000', 'last_fullsized': '2023-11-08T19:25:49.854746+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:25:49.854746+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:25:49.854746+0000', 'last_clean_scrub_stamp': '2023-11-08T19:25:49.854746+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-08T19:25:55.748 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.a is not active+clean 2023-11-08T19:25:55.748 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2023-11-08T19:25:48.857323+0000', 'last_change': '2023-11-08T19:25:48.857323+0000', 'last_active': '2023-11-08T19:25:48.844481+0000', 'last_peered': '2023-11-08T19:25:48.844481+0000', 'last_clean': '2023-11-08T19:25:48.844481+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:25:48.857323+0000', 'last_undegraded': '2023-11-08T19:25:48.857323+0000', 'last_fullsized': '2023-11-08T19:25:48.857323+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:25:48.844481+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:25:48.844481+0000', 'last_clean_scrub_stamp': '2023-11-08T19:25:48.844481+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-08T19:25:55.748 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.9 is not active+clean 2023-11-08T19:25:55.749 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-08T19:25:49.854746+0000', 'last_change': '2023-11-08T19:25:49.854746+0000', 'last_active': '2023-11-08T19:25:49.854746+0000', 'last_peered': '2023-11-08T19:25:49.854746+0000', 'last_clean': '2023-11-08T19:25:49.854746+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:25:49.854746+0000', 'last_undegraded': '2023-11-08T19:25:49.854746+0000', 'last_fullsized': '2023-11-08T19:25:49.854746+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:25:49.854746+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:25:49.854746+0000', 'last_clean_scrub_stamp': '2023-11-08T19:25:49.854746+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-08T19:25:55.749 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2 is not active+clean 2023-11-08T19:25:55.750 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-08T19:25:49.854746+0000', 'last_change': '2023-11-08T19:25:49.854746+0000', 'last_active': '2023-11-08T19:25:49.854746+0000', 'last_peered': '2023-11-08T19:25:49.854746+0000', 'last_clean': '2023-11-08T19:25:49.854746+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:25:49.854746+0000', 'last_undegraded': '2023-11-08T19:25:49.854746+0000', 'last_fullsized': '2023-11-08T19:25:49.854746+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:25:49.854746+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:25:49.854746+0000', 'last_clean_scrub_stamp': '2023-11-08T19:25:49.854746+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-08T19:25:55.750 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.4 is not active+clean 2023-11-08T19:25:55.750 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-08T19:25:49.854746+0000', 'last_change': '2023-11-08T19:25:49.854746+0000', 'last_active': '2023-11-08T19:25:49.854746+0000', 'last_peered': '2023-11-08T19:25:49.854746+0000', 'last_clean': '2023-11-08T19:25:49.854746+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:25:49.854746+0000', 'last_undegraded': '2023-11-08T19:25:49.854746+0000', 'last_fullsized': '2023-11-08T19:25:49.854746+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:25:49.854746+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:25:49.854746+0000', 'last_clean_scrub_stamp': '2023-11-08T19:25:49.854746+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-08T19:25:55.751 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.5 is not active+clean 2023-11-08T19:25:55.751 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 30, 'state': 'creating+peering', 'last_fresh': '2023-11-08T19:25:48.857272+0000', 'last_change': '2023-11-08T19:25:48.857272+0000', 'last_active': '2023-11-08T19:25:48.844481+0000', 'last_peered': '2023-11-08T19:25:48.844481+0000', 'last_clean': '2023-11-08T19:25:48.844481+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:25:48.857272+0000', 'last_undegraded': '2023-11-08T19:25:48.857272+0000', 'last_fullsized': '2023-11-08T19:25:48.857272+0000', 'mapping_epoch': 30, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 30, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:25:48.844481+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:25:48.844481+0000', 'last_clean_scrub_stamp': '2023-11-08T19:25:48.844481+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-08T19:25:55.752 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-11-08T19:25:55.887 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.885+0000 7fdb30b4e700 1 Processor -- start 2023-11-08T19:25:55.888 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.885+0000 7fdb30b4e700 1 -- start start 2023-11-08T19:25:55.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.886+0000 7fdb30b4e700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fdb2c15de50 0x7fdb2c0cad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:55.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.886+0000 7fdb30b4e700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdb2c0d1170 0x7fdb2c06ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:55.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.886+0000 7fdb30b4e700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fdb2c0ce300 0x7fdb2c06b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:55.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.886+0000 7fdb30b4e700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb2c06e250 con 0x7fdb2c0ce300 2023-11-08T19:25:55.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.886+0000 7fdb30b4e700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb2c15e230 con 0x7fdb2c15de50 2023-11-08T19:25:55.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.886+0000 7fdb30b4e700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb2c06f750 con 0x7fdb2c0d1170 2023-11-08T19:25:55.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.886+0000 7fdb2ad9d700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fdb2c0ce300 0x7fdb2c06b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:55.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.886+0000 7fdb2ad9d700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fdb2c0ce300 0x7fdb2c06b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50724/0 (socket says 172.21.15.143:50724) 2023-11-08T19:25:55.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.887+0000 7fdb2ad9d700 1 -- 172.21.15.143:0/2788880051 learned_addr learned my addr 172.21.15.143:0/2788880051 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:55.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.887+0000 7fdb2a59c700 1 --2- 172.21.15.143:0/2788880051 >> v2:172.21.15.174:3300/0 conn(0x7fdb2c15de50 0x7fdb2c0cad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:55.895 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.887+0000 7fdb29d9b700 1 --2- 172.21.15.143:0/2788880051 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdb2c0d1170 0x7fdb2c06ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:55.895 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.887+0000 7fdb2ad9d700 1 -- 172.21.15.143:0/2788880051 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdb2c0d1170 msgr2=0x7fdb2c06ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:55.895 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.887+0000 7fdb2ad9d700 1 --2- 172.21.15.143:0/2788880051 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdb2c0d1170 0x7fdb2c06ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:55.895 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.887+0000 7fdb2ad9d700 1 -- 172.21.15.143:0/2788880051 >> v2:172.21.15.174:3300/0 conn(0x7fdb2c15de50 msgr2=0x7fdb2c0cad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:55.895 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.887+0000 7fdb2ad9d700 1 --2- 172.21.15.143:0/2788880051 >> v2:172.21.15.174:3300/0 conn(0x7fdb2c15de50 0x7fdb2c0cad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:55.896 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.887+0000 7fdb2ad9d700 1 -- 172.21.15.143:0/2788880051 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb2c06fa30 con 0x7fdb2c0ce300 2023-11-08T19:25:55.896 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.887+0000 7fdb2ad9d700 1 --2- 172.21.15.143:0/2788880051 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb2c0ce300 0x7fdb2c06b150 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c0cd210 tx=0x7fdb2400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15d11024c9b1f1ba server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:55.896 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.887+0000 7fdb2959a700 1 -- 172.21.15.143:0/2788880051 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb2400e070 con 0x7fdb2c0ce300 2023-11-08T19:25:55.896 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.887+0000 7fdb2959a700 1 -- 172.21.15.143:0/2788880051 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb2400b910 con 0x7fdb2c0ce300 2023-11-08T19:25:55.896 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.887+0000 7fdb2959a700 1 -- 172.21.15.143:0/2788880051 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb2400bcc0 con 0x7fdb2c0ce300 2023-11-08T19:25:55.897 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.888+0000 7fdb30b4e700 1 -- 172.21.15.143:0/2788880051 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb2c0ce300 msgr2=0x7fdb2c06b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:55.897 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.888+0000 7fdb30b4e700 1 --2- 172.21.15.143:0/2788880051 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb2c0ce300 0x7fdb2c06b150 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c0cd210 tx=0x7fdb2400b330 comp rx=0 tx=0).stop 2023-11-08T19:25:55.897 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.888+0000 7fdb30b4e700 1 -- 172.21.15.143:0/2788880051 shutdown_connections 2023-11-08T19:25:55.897 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.888+0000 7fdb30b4e700 1 --2- 172.21.15.143:0/2788880051 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb2c0ce300 0x7fdb2c06b150 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:55.898 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.888+0000 7fdb30b4e700 1 --2- 172.21.15.143:0/2788880051 >> v2:172.21.15.174:3300/0 conn(0x7fdb2c15de50 0x7fdb2c0cad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:55.898 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.888+0000 7fdb30b4e700 1 --2- 172.21.15.143:0/2788880051 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdb2c0d1170 0x7fdb2c06ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:55.898 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.888+0000 7fdb30b4e700 1 -- 172.21.15.143:0/2788880051 >> 172.21.15.143:0/2788880051 conn(0x7fdb2c099a70 msgr2=0x7fdb2c0d07c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:55.898 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.888+0000 7fdb30b4e700 1 -- 172.21.15.143:0/2788880051 shutdown_connections 2023-11-08T19:25:55.898 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.888+0000 7fdb30b4e700 1 -- 172.21.15.143:0/2788880051 wait complete. 2023-11-08T19:25:55.899 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.889+0000 7fdb30b4e700 1 Processor -- start 2023-11-08T19:25:55.899 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.889+0000 7fdb30b4e700 1 -- start start 2023-11-08T19:25:55.899 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.890+0000 7fdb30b4e700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdb2c0ce300 0x7fdb2c159bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:55.899 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.890+0000 7fdb30b4e700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdb2c0d1170 0x7fdb2c15b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:55.899 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.890+0000 7fdb30b4e700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb2c15de50 0x7fdb2c0ae670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:55.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.890+0000 7fdb30b4e700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb2c0c4a60 con 0x7fdb2c15de50 2023-11-08T19:25:55.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.890+0000 7fdb30b4e700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb2c0c4d00 con 0x7fdb2c0ce300 2023-11-08T19:25:55.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.890+0000 7fdb30b4e700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb2c0c4fe0 con 0x7fdb2c0d1170 2023-11-08T19:25:55.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.890+0000 7fdb2ad9d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb2c15de50 0x7fdb2c0ae670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:55.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.890+0000 7fdb2ad9d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb2c15de50 0x7fdb2c0ae670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50730/0 (socket says 172.21.15.143:50730) 2023-11-08T19:25:55.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.890+0000 7fdb2ad9d700 1 -- 172.21.15.143:0/2550130722 learned_addr learned my addr 172.21.15.143:0/2550130722 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:55.901 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.890+0000 7fdb2a59c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdb2c0ce300 0x7fdb2c159bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:55.901 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.890+0000 7fdb29d9b700 1 --2- 172.21.15.143:0/2550130722 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdb2c0d1170 0x7fdb2c15b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:55.901 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.890+0000 7fdb2ad9d700 1 -- 172.21.15.143:0/2550130722 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdb2c0d1170 msgr2=0x7fdb2c15b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:55.901 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.890+0000 7fdb2ad9d700 1 --2- 172.21.15.143:0/2550130722 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdb2c0d1170 0x7fdb2c15b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:55.901 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.890+0000 7fdb2ad9d700 1 -- 172.21.15.143:0/2550130722 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdb2c0ce300 msgr2=0x7fdb2c159bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:55.902 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.890+0000 7fdb2ad9d700 1 --2- 172.21.15.143:0/2550130722 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdb2c0ce300 0x7fdb2c159bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:55.902 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.890+0000 7fdb2ad9d700 1 -- 172.21.15.143:0/2550130722 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb24009b30 con 0x7fdb2c15de50 2023-11-08T19:25:55.902 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.890+0000 7fdb2ad9d700 1 --2- 172.21.15.143:0/2550130722 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb2c15de50 0x7fdb2c0ae670 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fdb24012000 tx=0x7fdb24004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:55.902 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.890+0000 7fdb1affd700 1 -- 172.21.15.143:0/2550130722 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb240099b0 con 0x7fdb2c15de50 2023-11-08T19:25:55.902 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.891+0000 7fdb1affd700 1 -- 172.21.15.143:0/2550130722 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb2400ce80 con 0x7fdb2c15de50 2023-11-08T19:25:55.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.891+0000 7fdb30b4e700 1 -- 172.21.15.143:0/2550130722 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb2c0c52c0 con 0x7fdb2c15de50 2023-11-08T19:25:55.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.891+0000 7fdb1affd700 1 -- 172.21.15.143:0/2550130722 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb24004490 con 0x7fdb2c15de50 2023-11-08T19:25:55.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.891+0000 7fdb30b4e700 1 -- 172.21.15.143:0/2550130722 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb2c15a100 con 0x7fdb2c15de50 2023-11-08T19:25:55.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.891+0000 7fdb30b4e700 1 -- 172.21.15.143:0/2550130722 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb2c0c57e0 con 0x7fdb2c15de50 2023-11-08T19:25:55.904 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.892+0000 7fdb1affd700 1 -- 172.21.15.143:0/2550130722 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fdb240045f0 con 0x7fdb2c15de50 2023-11-08T19:25:55.905 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.892+0000 7fdb1affd700 1 --2- 172.21.15.143:0/2550130722 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fdb0c06af00 0x7fdb0c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:55.905 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.892+0000 7fdb1affd700 1 -- 172.21.15.143:0/2550130722 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdb2408a4b0 con 0x7fdb2c15de50 2023-11-08T19:25:55.905 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.892+0000 7fdb2a59c700 1 --2- 172.21.15.143:0/2550130722 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fdb0c06af00 0x7fdb0c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:55.905 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.893+0000 7fdb2a59c700 1 --2- 172.21.15.143:0/2550130722 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fdb0c06af00 0x7fdb0c06d3c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fdb20009b20 tx=0x7fdb200091c0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:55.905 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:55.896+0000 7fdb1affd700 1 -- 172.21.15.143:0/2550130722 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdb2c0c57e0 con 0x7fdb2c15de50 2023-11-08T19:25:56.168 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:56.167+0000 7fdb30b4e700 1 -- 172.21.15.143:0/2550130722 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fdb2c0c57e0 con 0x7fdb2c15de50 2023-11-08T19:25:56.169 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:56.167+0000 7fdb1affd700 1 -- 172.21.15.143:0/2550130722 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1721 (secure 0 0 0) 0x7fdb2401f6d0 con 0x7fdb2c15de50 2023-11-08T19:25:56.169 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:25:56.170 INFO:teuthology.orchestra.run.smithi143.stdout:{"fsid":"968483af-77fc-4068-ad7c-5c2a1a4613d7","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":83,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":33,"num_osds":8,"num_up_osds":8,"osd_up_since":1699471483,"num_in_osds":8,"osd_in_since":1699471475,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":78},{"state_name":"unknown","count":16},{"state_name":"creating+peering","count":3}],"num_pgs":97,"num_pools":3,"num_objects":22,"data_bytes":592196,"bytes_used":225525760,"bytes_avail":772868587520,"bytes_total":773094113280,"unknown_pgs_ratio":0.16494844853878021,"inactive_pgs_ratio":0.030927835032343864,"write_bytes_sec":2347,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":16,"id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"c","status":"up:active","gid":5109}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":4,"modified":"2023-11-08T19:25:46.523402+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-11-08T19:25:56.171 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:56.169+0000 7fdb30b4e700 1 -- 172.21.15.143:0/2550130722 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fdb0c06af00 msgr2=0x7fdb0c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:56.172 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:56.169+0000 7fdb30b4e700 1 --2- 172.21.15.143:0/2550130722 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fdb0c06af00 0x7fdb0c06d3c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fdb20009b20 tx=0x7fdb200091c0 comp rx=0 tx=0).stop 2023-11-08T19:25:56.172 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:56.169+0000 7fdb30b4e700 1 -- 172.21.15.143:0/2550130722 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb2c15de50 msgr2=0x7fdb2c0ae670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:56.172 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:56.169+0000 7fdb30b4e700 1 --2- 172.21.15.143:0/2550130722 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb2c15de50 0x7fdb2c0ae670 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fdb24012000 tx=0x7fdb24004ad0 comp rx=0 tx=0).stop 2023-11-08T19:25:56.172 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:56.169+0000 7fdb30b4e700 1 -- 172.21.15.143:0/2550130722 shutdown_connections 2023-11-08T19:25:56.173 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:56.169+0000 7fdb30b4e700 1 --2- 172.21.15.143:0/2550130722 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fdb0c06af00 0x7fdb0c06d3c0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:56.173 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:56.169+0000 7fdb30b4e700 1 --2- 172.21.15.143:0/2550130722 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb2c15de50 0x7fdb2c0ae670 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:56.174 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:56.169+0000 7fdb30b4e700 1 --2- 172.21.15.143:0/2550130722 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdb2c0ce300 0x7fdb2c159bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:56.174 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:56.169+0000 7fdb30b4e700 1 --2- 172.21.15.143:0/2550130722 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdb2c0d1170 0x7fdb2c15b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:56.175 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:56.169+0000 7fdb30b4e700 1 -- 172.21.15.143:0/2550130722 >> 172.21.15.143:0/2550130722 conn(0x7fdb2c099a70 msgr2=0x7fdb2c1e90f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:56.175 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:56.170+0000 7fdb30b4e700 1 -- 172.21.15.143:0/2550130722 shutdown_connections 2023-11-08T19:25:56.176 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:56.170+0000 7fdb30b4e700 1 -- 172.21.15.143:0/2550130722 wait complete. 2023-11-08T19:25:56.183 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-11-08T19:25:56.184 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-08T19:25:56.305 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:56.303+0000 7f52420af700 1 Processor -- start 2023-11-08T19:25:56.305 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:56.303+0000 7f52420af700 1 -- start start 2023-11-08T19:25:56.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:56.304+0000 7f52420af700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f523c0b2920 0x7f523c0cf510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:56.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:56.304+0000 7f52420af700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f523c0cf080 con 0x7f523c15aa30 2023-11-08T19:25:56.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:56.304+0000 7f52420af700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f523c0ce470 con 0x7f523c0cbe20 2023-11-08T19:25:56.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:56.304+0000 7f52420af700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f523c0ce750 con 0x7f523c0b2920 2023-11-08T19:25:56.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:56.304+0000 7f523b7fe700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f523c15aa30 0x7f523c0b2210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:46888/0 (socket says 172.21.15.143:46888) 2023-11-08T19:25:56.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:56.304+0000 7f523b7fe700 1 -- 172.21.15.143:0/931295375 learned_addr learned my addr 172.21.15.143:0/931295375 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:25:56.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:56.305+0000 7f523affd700 1 --2- 172.21.15.143:0/931295375 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f523c0b2920 0x7f523c0cf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:56.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:56.305+0000 7f523a7fc700 1 -- 172.21.15.143:0/931295375 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 490324016 0 0) 0x7f523c0cf080 con 0x7f523c15aa30 2023-11-08T19:25:56.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:56.305+0000 7f523a7fc700 1 -- 172.21.15.143:0/931295375 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52280057c0 con 0x7f523c15aa30 2023-11-08T19:25:56.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:56.305+0000 7f523a7fc700 1 -- 172.21.15.143:0/931295375 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1315952618 0 0) 0x7f52280057c0 con 0x7f523c15aa30 2023-11-08T19:25:56.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:56.305+0000 7f523a7fc700 1 -- 172.21.15.143:0/931295375 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f523c0b2920 msgr2=0x7f523c0cf510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:56.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:56.305+0000 7f523a7fc700 1 --2- 172.21.15.143:0/931295375 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f523c0b2920 0x7f523c0cf510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:56.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:56.305+0000 7f523a7fc700 1 -- 172.21.15.143:0/931295375 >> v1:172.21.15.174:6789/0 conn(0x7f523c0cbe20 legacy=0x7f523c0cc250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:56.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:56.305+0000 7f523a7fc700 1 -- 172.21.15.143:0/931295375 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f523c0cea30 con 0x7f523c15aa30 2023-11-08T19:25:56.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:56.305+0000 7f523a7fc700 1 -- 172.21.15.143:0/931295375 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4147756303 0 0) 0x7f52240017f0 con 0x7f523c15aa30 2023-11-08T19:25:56.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:56.305+0000 7f523a7fc700 1 -- 172.21.15.143:0/931295375 >> v1:172.21.15.143:6789/0 conn(0x7f523c15aa30 legacy=0x7f523c0b2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:56.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:56.305+0000 7f523a7fc700 1 --2- 172.21.15.143:0/931295375 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5228005c40 0x7f5228006020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:56.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:56.305+0000 7f523a7fc700 1 --2- 172.21.15.143:0/931295375 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f52280065f0 0x7f5228006ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:56.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:56.305+0000 7f523a7fc700 1 --2- 172.21.15.143:0/931295375 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f523c0b2920 0x7f5228007320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:56.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:56.305+0000 7f523a7fc700 1 -- 172.21.15.143:0/931295375 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f523c0ce750 con 0x7f5228005c40 2023-11-08T19:25:56.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:56.305+0000 7f523a7fc700 1 -- 172.21.15.143:0/931295375 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f522800dc20 con 0x7f52280065f0 2023-11-08T19:25:56.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:56.305+0000 7f523a7fc700 1 -- 172.21.15.143:0/931295375 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f522800dde0 con 0x7f523c0b2920 2023-11-08T19:25:56.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:56.305+0000 7f523b7fe700 1 --2- 172.21.15.143:0/931295375 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5228005c40 0x7f5228006020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:56.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:56.305+0000 7f523bfff700 1 --2- 172.21.15.143:0/931295375 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f523c0b2920 0x7f5228007320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:56.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:56.305+0000 7f523affd700 1 --2- 172.21.15.143:0/931295375 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f52280065f0 0x7f5228006ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:56.315 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:56.305+0000 7f523b7fe700 1 -- 172.21.15.143:0/931295375 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f523c0b2920 msgr2=0x7f5228007320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:56.315 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:56.305+0000 7f523b7fe700 1 --2- 172.21.15.143:0/931295375 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f523c0b2920 0x7f5228007320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:56.315 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:56.305+0000 7f523b7fe700 1 -- 172.21.15.143:0/931295375 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f52280065f0 msgr2=0x7f5228006ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:56.315 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:56.305+0000 7f523b7fe700 1 --2- 172.21.15.143:0/931295375 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f52280065f0 0x7f5228006ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:56.315 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:56.305+0000 7f523b7fe700 1 -- 172.21.15.143:0/931295375 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52280057c0 con 0x7f5228005c40 2023-11-08T19:25:56.316 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:56.305+0000 7f523affd700 1 --2- 172.21.15.143:0/931295375 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f52280065f0 0x7f5228006ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:25:56.316 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:56.306+0000 7f523b7fe700 1 --2- 172.21.15.143:0/931295375 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5228005c40 0x7f5228006020 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f522400c240 tx=0x7f5228007a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4658145b67e34c0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:56.316 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:56.306+0000 7f523a7fc700 1 -- 172.21.15.143:0/931295375 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f522400cde0 con 0x7f5228005c40 2023-11-08T19:25:56.316 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:56.306+0000 7f523a7fc700 1 -- 172.21.15.143:0/931295375 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5224007ba0 con 0x7f5228005c40 2023-11-08T19:25:56.316 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:56.306+0000 7f523a7fc700 1 -- 172.21.15.143:0/931295375 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52240072f0 con 0x7f5228005c40 2023-11-08T19:25:56.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:56.306+0000 7f52420af700 1 -- 172.21.15.143:0/931295375 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5228005c40 msgr2=0x7f5228006020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:56.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:56.306+0000 7f52420af700 1 --2- 172.21.15.143:0/931295375 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5228005c40 0x7f5228006020 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f522400c240 tx=0x7f5228007a90 comp rx=0 tx=0).stop 2023-11-08T19:25:56.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:56.306+0000 7f523bfff700 1 -- 172.21.15.143:0/931295375 reap_dead start 2023-11-08T19:25:56.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:56.306+0000 7f52420af700 1 -- 172.21.15.143:0/931295375 shutdown_connections 2023-11-08T19:25:56.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:56.306+0000 7f52420af700 1 -- 172.21.15.143:0/931295375 >> 172.21.15.143:0/931295375 conn(0x7f523c099ab0 msgr2=0x7f523c0d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:56.318 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:56.306+0000 7f52420af700 1 -- 172.21.15.143:0/931295375 shutdown_connections 2023-11-08T19:25:56.318 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:56.306+0000 7f52420af700 1 -- 172.21.15.143:0/931295375 wait complete. 2023-11-08T19:25:56.318 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:56.308+0000 7f52420af700 1 Processor -- start 2023-11-08T19:25:56.318 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:56.308+0000 7f52420af700 1 -- start start 2023-11-08T19:25:56.321 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:56.309+0000 7f52420af700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5228005c40 0x7f523c1f35f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:56.322 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:56.309+0000 7f52420af700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f523c0b33b0 0x7f523c0c6a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:56.322 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:56.309+0000 7f52420af700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f523c0b5880 0x7f523c0b5d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:56.322 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:56.309+0000 7f52420af700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f523c0b4710 con 0x7f523c0b5880 2023-11-08T19:25:56.322 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:56.309+0000 7f52420af700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f523c0b6240 con 0x7f523c0b33b0 2023-11-08T19:25:56.322 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:56.309+0000 7f52420af700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f523c0b3830 con 0x7f5228005c40 2023-11-08T19:25:56.323 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:56.309+0000 7f523bfff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f523c0b5880 0x7f523c0b5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:56.323 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:56.309+0000 7f523bfff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f523c0b5880 0x7f523c0b5d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50750/0 (socket says 172.21.15.143:50750) 2023-11-08T19:25:56.323 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:56.309+0000 7f523bfff700 1 -- 172.21.15.143:0/3476353676 learned_addr learned my addr 172.21.15.143:0/3476353676 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:56.323 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:56.309+0000 7f523b7fe700 1 --2- 172.21.15.143:0/3476353676 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5228005c40 0x7f523c1f35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:56.323 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:56.309+0000 7f523affd700 1 --2- 172.21.15.143:0/3476353676 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f523c0b33b0 0x7f523c0c6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:56.324 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:56.309+0000 7f523bfff700 1 -- 172.21.15.143:0/3476353676 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5228005c40 msgr2=0x7f523c1f35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:56.324 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:56.309+0000 7f523bfff700 1 --2- 172.21.15.143:0/3476353676 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5228005c40 0x7f523c1f35f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:56.324 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:56.309+0000 7f523bfff700 1 -- 172.21.15.143:0/3476353676 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f523c0b33b0 msgr2=0x7f523c0c6a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:56.324 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:56.309+0000 7f523bfff700 1 --2- 172.21.15.143:0/3476353676 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f523c0b33b0 0x7f523c0c6a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:56.324 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:56.309+0000 7f523bfff700 1 -- 172.21.15.143:0/3476353676 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f523c0ca700 con 0x7f523c0b5880 2023-11-08T19:25:56.325 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:56.310+0000 7f523bfff700 1 --2- 172.21.15.143:0/3476353676 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f523c0b5880 0x7f523c0b5d00 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f523c072e70 tx=0x7f523400d660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:56.325 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:56.310+0000 7f5233fff700 1 -- 172.21.15.143:0/3476353676 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f523400dab0 con 0x7f523c0b5880 2023-11-08T19:25:56.325 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:56.310+0000 7f5233fff700 1 -- 172.21.15.143:0/3476353676 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f523400dc10 con 0x7f523c0b5880 2023-11-08T19:25:56.325 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:56.310+0000 7f52420af700 1 -- 172.21.15.143:0/3476353676 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f523c0b3a90 con 0x7f523c0b5880 2023-11-08T19:25:56.325 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:56.310+0000 7f5233fff700 1 -- 172.21.15.143:0/3476353676 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5234003a00 con 0x7f523c0b5880 2023-11-08T19:25:56.326 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:56.310+0000 7f52420af700 1 -- 172.21.15.143:0/3476353676 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f523c0b4050 con 0x7f523c0b5880 2023-11-08T19:25:56.326 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:56.311+0000 7f52420af700 1 -- 172.21.15.143:0/3476353676 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f523c04eee0 con 0x7f523c0b5880 2023-11-08T19:25:56.326 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:56.311+0000 7f5233fff700 1 -- 172.21.15.143:0/3476353676 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5234003b60 con 0x7f523c0b5880 2023-11-08T19:25:56.326 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:56.311+0000 7f5233fff700 1 --2- 172.21.15.143:0/3476353676 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f521c06af00 0x7f521c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:56.326 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:56.311+0000 7f5233fff700 1 -- 172.21.15.143:0/3476353676 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5234089760 con 0x7f523c0b5880 2023-11-08T19:25:56.327 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:56.312+0000 7f523b7fe700 1 --2- 172.21.15.143:0/3476353676 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f521c06af00 0x7f521c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:56.327 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:56.312+0000 7f523b7fe700 1 --2- 172.21.15.143:0/3476353676 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f521c06af00 0x7f521c06d3c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5224007d50 tx=0x7f5224007000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:56.327 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:56.316+0000 7f5233fff700 1 -- 172.21.15.143:0/3476353676 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f52340548a0 con 0x7f523c0b5880 2023-11-08T19:25:56.501 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:56.500+0000 7f52420af700 1 -- 172.21.15.143:0/3476353676 --> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f523c15ae10 con 0x7f521c06af00 2023-11-08T19:25:56.509 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:56.508+0000 7f5233fff700 1 -- 172.21.15.143:0/3476353676 <== mgr.4099 v2:172.21.15.174:6832/4037280178 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286848 (secure 0 0 0) 0x7f5224022070 con 0x7f521c06af00 2023-11-08T19:25:56.513 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:25:56.516 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:56.511+0000 7f52420af700 1 -- 172.21.15.143:0/3476353676 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f521c06af00 msgr2=0x7f521c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:56.516 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:56.511+0000 7f52420af700 1 --2- 172.21.15.143:0/3476353676 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f521c06af00 0x7f521c06d3c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5224007d50 tx=0x7f5224007000 comp rx=0 tx=0).stop 2023-11-08T19:25:56.517 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:56.511+0000 7f52420af700 1 -- 172.21.15.143:0/3476353676 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f523c0b5880 msgr2=0x7f523c0b5d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:56.517 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:56.511+0000 7f52420af700 1 --2- 172.21.15.143:0/3476353676 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f523c0b5880 0x7f523c0b5d00 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f523c072e70 tx=0x7f523400d660 comp rx=0 tx=0).stop 2023-11-08T19:25:56.517 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:56.511+0000 7f52420af700 1 -- 172.21.15.143:0/3476353676 shutdown_connections 2023-11-08T19:25:56.517 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:56.511+0000 7f52420af700 1 --2- 172.21.15.143:0/3476353676 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f521c06af00 0x7f521c06d3c0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:56.517 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:56.511+0000 7f52420af700 1 --2- 172.21.15.143:0/3476353676 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5228005c40 0x7f523c1f35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:56.518 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:56.511+0000 7f52420af700 1 --2- 172.21.15.143:0/3476353676 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f523c0b5880 0x7f523c0b5d00 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:56.518 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:56.511+0000 7f52420af700 1 --2- 172.21.15.143:0/3476353676 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f523c0b33b0 0x7f523c0c6a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:56.518 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:56.511+0000 7f52420af700 1 -- 172.21.15.143:0/3476353676 >> 172.21.15.143:0/3476353676 conn(0x7f523c099ab0 msgr2=0x7f523c0af390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:56.518 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:56.511+0000 7f52420af700 1 -- 172.21.15.143:0/3476353676 shutdown_connections 2023-11-08T19:25:56.519 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:56.511+0000 7f52420af700 1 -- 172.21.15.143:0/3476353676 wait complete. 2023-11-08T19:25:56.519 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2023-11-08T19:25:56.525 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":61,"stamp":"2023-11-08T19:25:54.525094+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592196,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":200,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":152,"ondisk_log_size":152,"up":162,"acting":162,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":117,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220240,"kb_used_data":4052,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754480,"statfs":{"total":773094113280,"available":772868587520,"internally_reserved":0,"allocated":4149248,"data_stored":2313094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1828,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.669558"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:25:49.854746+0000","last_change":"2023-11-08T19:25:49.854746+0000","last_active":"2023-11-08T19:25:49.854746+0000","last_peered":"2023-11-08T19:25:49.854746+0000","last_clean":"2023-11-08T19:25:49.854746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:25:49.854746+0000","last_undegraded":"2023-11-08T19:25:49.854746+0000","last_fullsized":"2023-11-08T19:25:49.854746+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204169+0000","last_change":"2023-11-08T19:25:50.890421+0000","last_active":"2023-11-08T19:25:52.204169+0000","last_peered":"2023-11-08T19:25:52.204169+0000","last_clean":"2023-11-08T19:25:52.204169+0000","last_became_active":"2023-11-08T19:25:50.890142+0000","last_became_peered":"2023-11-08T19:25:50.890142+0000","last_unstale":"2023-11-08T19:25:52.204169+0000","last_undegraded":"2023-11-08T19:25:52.204169+0000","last_fullsized":"2023-11-08T19:25:52.204169+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:01:43.267493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.205154+0000","last_change":"2023-11-08T19:25:50.875008+0000","last_active":"2023-11-08T19:25:52.205154+0000","last_peered":"2023-11-08T19:25:52.205154+0000","last_clean":"2023-11-08T19:25:52.205154+0000","last_became_active":"2023-11-08T19:25:50.874499+0000","last_became_peered":"2023-11-08T19:25:50.874499+0000","last_unstale":"2023-11-08T19:25:52.205154+0000","last_undegraded":"2023-11-08T19:25:52.205154+0000","last_fullsized":"2023-11-08T19:25:52.205154+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:27:11.209258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.991817+0000","last_change":"2023-11-08T19:25:50.883677+0000","last_active":"2023-11-08T19:25:51.991817+0000","last_peered":"2023-11-08T19:25:51.991817+0000","last_clean":"2023-11-08T19:25:51.991817+0000","last_became_active":"2023-11-08T19:25:50.883141+0000","last_became_peered":"2023-11-08T19:25:50.883141+0000","last_unstale":"2023-11-08T19:25:51.991817+0000","last_undegraded":"2023-11-08T19:25:51.991817+0000","last_fullsized":"2023-11-08T19:25:51.991817+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:56:01.687509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.876619+0000","last_change":"2023-11-08T19:25:50.877229+0000","last_active":"2023-11-08T19:25:51.876619+0000","last_peered":"2023-11-08T19:25:51.876619+0000","last_clean":"2023-11-08T19:25:51.876619+0000","last_became_active":"2023-11-08T19:25:50.876959+0000","last_became_peered":"2023-11-08T19:25:50.876959+0000","last_unstale":"2023-11-08T19:25:51.876619+0000","last_undegraded":"2023-11-08T19:25:51.876619+0000","last_fullsized":"2023-11-08T19:25:51.876619+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:27:09.954811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204135+0000","last_change":"2023-11-08T19:25:50.883530+0000","last_active":"2023-11-08T19:25:52.204135+0000","last_peered":"2023-11-08T19:25:52.204135+0000","last_clean":"2023-11-08T19:25:52.204135+0000","last_became_active":"2023-11-08T19:25:50.883198+0000","last_became_peered":"2023-11-08T19:25:50.883198+0000","last_unstale":"2023-11-08T19:25:52.204135+0000","last_undegraded":"2023-11-08T19:25:52.204135+0000","last_fullsized":"2023-11-08T19:25:52.204135+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:27:11.209258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:25:49.854746+0000","last_change":"2023-11-08T19:25:49.854746+0000","last_active":"2023-11-08T19:25:49.854746+0000","last_peered":"2023-11-08T19:25:49.854746+0000","last_clean":"2023-11-08T19:25:49.854746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:25:49.854746+0000","last_undegraded":"2023-11-08T19:25:49.854746+0000","last_fullsized":"2023-11-08T19:25:49.854746+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:25:49.854746+0000","last_change":"2023-11-08T19:25:49.854746+0000","last_active":"2023-11-08T19:25:49.854746+0000","last_peered":"2023-11-08T19:25:49.854746+0000","last_clean":"2023-11-08T19:25:49.854746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:25:49.854746+0000","last_undegraded":"2023-11-08T19:25:49.854746+0000","last_fullsized":"2023-11-08T19:25:49.854746+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.205585+0000","last_change":"2023-11-08T19:25:50.888816+0000","last_active":"2023-11-08T19:25:52.205585+0000","last_peered":"2023-11-08T19:25:52.205585+0000","last_clean":"2023-11-08T19:25:52.205585+0000","last_became_active":"2023-11-08T19:25:50.888536+0000","last_became_peered":"2023-11-08T19:25:50.888536+0000","last_unstale":"2023-11-08T19:25:52.205585+0000","last_undegraded":"2023-11-08T19:25:52.205585+0000","last_fullsized":"2023-11-08T19:25:52.205585+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:56:01.687509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.991994+0000","last_change":"2023-11-08T19:25:50.878973+0000","last_active":"2023-11-08T19:25:51.991994+0000","last_peered":"2023-11-08T19:25:51.991994+0000","last_clean":"2023-11-08T19:25:51.991994+0000","last_became_active":"2023-11-08T19:25:50.878706+0000","last_became_peered":"2023-11-08T19:25:50.878706+0000","last_unstale":"2023-11-08T19:25:51.991994+0000","last_undegraded":"2023-11-08T19:25:51.991994+0000","last_fullsized":"2023-11-08T19:25:51.991994+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:49:41.475597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:25:49.854746+0000","last_change":"2023-11-08T19:25:49.854746+0000","last_active":"2023-11-08T19:25:49.854746+0000","last_peered":"2023-11-08T19:25:49.854746+0000","last_clean":"2023-11-08T19:25:49.854746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:25:49.854746+0000","last_undegraded":"2023-11-08T19:25:49.854746+0000","last_fullsized":"2023-11-08T19:25:49.854746+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204768+0000","last_change":"2023-11-08T19:25:50.877227+0000","last_active":"2023-11-08T19:25:52.204768+0000","last_peered":"2023-11-08T19:25:52.204768+0000","last_clean":"2023-11-08T19:25:52.204768+0000","last_became_active":"2023-11-08T19:25:50.876876+0000","last_became_peered":"2023-11-08T19:25:50.876876+0000","last_unstale":"2023-11-08T19:25:52.204768+0000","last_undegraded":"2023-11-08T19:25:52.204768+0000","last_fullsized":"2023-11-08T19:25:52.204768+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:05:33.227810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.876804+0000","last_change":"2023-11-08T19:25:50.890044+0000","last_active":"2023-11-08T19:25:51.876804+0000","last_peered":"2023-11-08T19:25:51.876804+0000","last_clean":"2023-11-08T19:25:51.876804+0000","last_became_active":"2023-11-08T19:25:50.889741+0000","last_became_peered":"2023-11-08T19:25:50.889741+0000","last_unstale":"2023-11-08T19:25:51.876804+0000","last_undegraded":"2023-11-08T19:25:51.876804+0000","last_fullsized":"2023-11-08T19:25:51.876804+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:54:55.202302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204255+0000","last_change":"2023-11-08T19:25:50.876264+0000","last_active":"2023-11-08T19:25:52.204255+0000","last_peered":"2023-11-08T19:25:52.204255+0000","last_clean":"2023-11-08T19:25:52.204255+0000","last_became_active":"2023-11-08T19:25:50.875971+0000","last_became_peered":"2023-11-08T19:25:50.875971+0000","last_unstale":"2023-11-08T19:25:52.204255+0000","last_undegraded":"2023-11-08T19:25:52.204255+0000","last_fullsized":"2023-11-08T19:25:52.204255+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:50:30.103716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204868+0000","last_change":"2023-11-08T19:25:50.877018+0000","last_active":"2023-11-08T19:25:52.204868+0000","last_peered":"2023-11-08T19:25:52.204868+0000","last_clean":"2023-11-08T19:25:52.204868+0000","last_became_active":"2023-11-08T19:25:50.876682+0000","last_became_peered":"2023-11-08T19:25:50.876682+0000","last_unstale":"2023-11-08T19:25:52.204868+0000","last_undegraded":"2023-11-08T19:25:52.204868+0000","last_fullsized":"2023-11-08T19:25:52.204868+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:52:20.334760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:25:49.854746+0000","last_change":"2023-11-08T19:25:49.854746+0000","last_active":"2023-11-08T19:25:49.854746+0000","last_peered":"2023-11-08T19:25:49.854746+0000","last_clean":"2023-11-08T19:25:49.854746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:25:49.854746+0000","last_undegraded":"2023-11-08T19:25:49.854746+0000","last_fullsized":"2023-11-08T19:25:49.854746+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:25:49.854746+0000","last_change":"2023-11-08T19:25:49.854746+0000","last_active":"2023-11-08T19:25:49.854746+0000","last_peered":"2023-11-08T19:25:49.854746+0000","last_clean":"2023-11-08T19:25:49.854746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:25:49.854746+0000","last_undegraded":"2023-11-08T19:25:49.854746+0000","last_fullsized":"2023-11-08T19:25:49.854746+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204251+0000","last_change":"2023-11-08T19:25:50.890954+0000","last_active":"2023-11-08T19:25:52.204251+0000","last_peered":"2023-11-08T19:25:52.204251+0000","last_clean":"2023-11-08T19:25:52.204251+0000","last_became_active":"2023-11-08T19:25:50.890695+0000","last_became_peered":"2023-11-08T19:25:50.890695+0000","last_unstale":"2023-11-08T19:25:52.204251+0000","last_undegraded":"2023-11-08T19:25:52.204251+0000","last_fullsized":"2023-11-08T19:25:52.204251+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:44:02.415941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204041+0000","last_change":"2023-11-08T19:25:50.892390+0000","last_active":"2023-11-08T19:25:52.204041+0000","last_peered":"2023-11-08T19:25:52.204041+0000","last_clean":"2023-11-08T19:25:52.204041+0000","last_became_active":"2023-11-08T19:25:50.892131+0000","last_became_peered":"2023-11-08T19:25:50.892131+0000","last_unstale":"2023-11-08T19:25:52.204041+0000","last_undegraded":"2023-11-08T19:25:52.204041+0000","last_fullsized":"2023-11-08T19:25:52.204041+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:25:07.133280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.876861+0000","last_change":"2023-11-08T19:25:50.874502+0000","last_active":"2023-11-08T19:25:51.876861+0000","last_peered":"2023-11-08T19:25:51.876861+0000","last_clean":"2023-11-08T19:25:51.876861+0000","last_became_active":"2023-11-08T19:25:50.874216+0000","last_became_peered":"2023-11-08T19:25:50.874216+0000","last_unstale":"2023-11-08T19:25:51.876861+0000","last_undegraded":"2023-11-08T19:25:51.876861+0000","last_fullsized":"2023-11-08T19:25:51.876861+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:39:23.438674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204719+0000","last_change":"2023-11-08T19:25:50.886322+0000","last_active":"2023-11-08T19:25:52.204719+0000","last_peered":"2023-11-08T19:25:52.204719+0000","last_clean":"2023-11-08T19:25:52.204719+0000","last_became_active":"2023-11-08T19:25:50.886011+0000","last_became_peered":"2023-11-08T19:25:50.886011+0000","last_unstale":"2023-11-08T19:25:52.204719+0000","last_undegraded":"2023-11-08T19:25:52.204719+0000","last_fullsized":"2023-11-08T19:25:52.204719+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:09:05.379033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204755+0000","last_change":"2023-11-08T19:25:50.881922+0000","last_active":"2023-11-08T19:25:52.204755+0000","last_peered":"2023-11-08T19:25:52.204755+0000","last_clean":"2023-11-08T19:25:52.204755+0000","last_became_active":"2023-11-08T19:25:50.880836+0000","last_became_peered":"2023-11-08T19:25:50.880836+0000","last_unstale":"2023-11-08T19:25:52.204755+0000","last_undegraded":"2023-11-08T19:25:52.204755+0000","last_fullsized":"2023-11-08T19:25:52.204755+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:49:45.059924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.989623+0000","last_change":"2023-11-08T19:25:50.886648+0000","last_active":"2023-11-08T19:25:51.989623+0000","last_peered":"2023-11-08T19:25:51.989623+0000","last_clean":"2023-11-08T19:25:51.989623+0000","last_became_active":"2023-11-08T19:25:50.886376+0000","last_became_peered":"2023-11-08T19:25:50.886376+0000","last_unstale":"2023-11-08T19:25:51.989623+0000","last_undegraded":"2023-11-08T19:25:51.989623+0000","last_fullsized":"2023-11-08T19:25:51.989623+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:49:45.059924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204354+0000","last_change":"2023-11-08T19:25:50.875989+0000","last_active":"2023-11-08T19:25:52.204354+0000","last_peered":"2023-11-08T19:25:52.204354+0000","last_clean":"2023-11-08T19:25:52.204354+0000","last_became_active":"2023-11-08T19:25:50.874428+0000","last_became_peered":"2023-11-08T19:25:50.874428+0000","last_unstale":"2023-11-08T19:25:52.204354+0000","last_undegraded":"2023-11-08T19:25:52.204354+0000","last_fullsized":"2023-11-08T19:25:52.204354+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:04:01.492989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.991856+0000","last_change":"2023-11-08T19:25:50.885263+0000","last_active":"2023-11-08T19:25:51.991856+0000","last_peered":"2023-11-08T19:25:51.991856+0000","last_clean":"2023-11-08T19:25:51.991856+0000","last_became_active":"2023-11-08T19:25:50.884856+0000","last_became_peered":"2023-11-08T19:25:50.884856+0000","last_unstale":"2023-11-08T19:25:51.991856+0000","last_undegraded":"2023-11-08T19:25:51.991856+0000","last_fullsized":"2023-11-08T19:25:51.991856+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:17:24.728184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204268+0000","last_change":"2023-11-08T19:25:50.884128+0000","last_active":"2023-11-08T19:25:52.204268+0000","last_peered":"2023-11-08T19:25:52.204268+0000","last_clean":"2023-11-08T19:25:52.204268+0000","last_became_active":"2023-11-08T19:25:50.883848+0000","last_became_peered":"2023-11-08T19:25:50.883848+0000","last_unstale":"2023-11-08T19:25:52.204268+0000","last_undegraded":"2023-11-08T19:25:52.204268+0000","last_fullsized":"2023-11-08T19:25:52.204268+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:32:01.237850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.992064+0000","last_change":"2023-11-08T19:25:50.885482+0000","last_active":"2023-11-08T19:25:51.992064+0000","last_peered":"2023-11-08T19:25:51.992064+0000","last_clean":"2023-11-08T19:25:51.992064+0000","last_became_active":"2023-11-08T19:25:50.884816+0000","last_became_peered":"2023-11-08T19:25:50.884816+0000","last_unstale":"2023-11-08T19:25:51.992064+0000","last_undegraded":"2023-11-08T19:25:51.992064+0000","last_fullsized":"2023-11-08T19:25:51.992064+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:05:33.227810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204941+0000","last_change":"2023-11-08T19:25:50.871479+0000","last_active":"2023-11-08T19:25:52.204941+0000","last_peered":"2023-11-08T19:25:52.204941+0000","last_clean":"2023-11-08T19:25:52.204941+0000","last_became_active":"2023-11-08T19:25:50.871211+0000","last_became_peered":"2023-11-08T19:25:50.871211+0000","last_unstale":"2023-11-08T19:25:52.204941+0000","last_undegraded":"2023-11-08T19:25:52.204941+0000","last_fullsized":"2023-11-08T19:25:52.204941+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:25:16.223226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204982+0000","last_change":"2023-11-08T19:25:50.881545+0000","last_active":"2023-11-08T19:25:52.204982+0000","last_peered":"2023-11-08T19:25:52.204982+0000","last_clean":"2023-11-08T19:25:52.204982+0000","last_became_active":"2023-11-08T19:25:50.881277+0000","last_became_peered":"2023-11-08T19:25:50.881277+0000","last_unstale":"2023-11-08T19:25:52.204982+0000","last_undegraded":"2023-11-08T19:25:52.204982+0000","last_fullsized":"2023-11-08T19:25:52.204982+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:18:57.865706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:25:49.854746+0000","last_change":"2023-11-08T19:25:49.854746+0000","last_active":"2023-11-08T19:25:49.854746+0000","last_peered":"2023-11-08T19:25:49.854746+0000","last_clean":"2023-11-08T19:25:49.854746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:25:49.854746+0000","last_undegraded":"2023-11-08T19:25:49.854746+0000","last_fullsized":"2023-11-08T19:25:49.854746+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:25:49.854746+0000","last_change":"2023-11-08T19:25:49.854746+0000","last_active":"2023-11-08T19:25:49.854746+0000","last_peered":"2023-11-08T19:25:49.854746+0000","last_clean":"2023-11-08T19:25:49.854746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:25:49.854746+0000","last_undegraded":"2023-11-08T19:25:49.854746+0000","last_fullsized":"2023-11-08T19:25:49.854746+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:25:49.854746+0000","last_change":"2023-11-08T19:25:49.854746+0000","last_active":"2023-11-08T19:25:49.854746+0000","last_peered":"2023-11-08T19:25:49.854746+0000","last_clean":"2023-11-08T19:25:49.854746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:25:49.854746+0000","last_undegraded":"2023-11-08T19:25:49.854746+0000","last_fullsized":"2023-11-08T19:25:49.854746+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.989618+0000","last_change":"2023-11-08T19:25:50.889063+0000","last_active":"2023-11-08T19:25:51.989618+0000","last_peered":"2023-11-08T19:25:51.989618+0000","last_clean":"2023-11-08T19:25:51.989618+0000","last_became_active":"2023-11-08T19:25:50.888760+0000","last_became_peered":"2023-11-08T19:25:50.888760+0000","last_unstale":"2023-11-08T19:25:51.989618+0000","last_undegraded":"2023-11-08T19:25:51.989618+0000","last_fullsized":"2023-11-08T19:25:51.989618+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:37:10.264673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-11-08T19:25:48.857285+0000","last_change":"2023-11-08T19:25:48.857285+0000","last_active":"2023-11-08T19:25:48.844481+0000","last_peered":"2023-11-08T19:25:48.844481+0000","last_clean":"2023-11-08T19:25:48.844481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:25:48.857285+0000","last_undegraded":"2023-11-08T19:25:48.857285+0000","last_fullsized":"2023-11-08T19:25:48.857285+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:25:49.854746+0000","last_change":"2023-11-08T19:25:49.854746+0000","last_active":"2023-11-08T19:25:49.854746+0000","last_peered":"2023-11-08T19:25:49.854746+0000","last_clean":"2023-11-08T19:25:49.854746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:25:49.854746+0000","last_undegraded":"2023-11-08T19:25:49.854746+0000","last_fullsized":"2023-11-08T19:25:49.854746+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1e","version":"32'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.989742+0000","last_change":"2023-11-08T19:25:49.879837+0000","last_active":"2023-11-08T19:25:51.989742+0000","last_peered":"2023-11-08T19:25:51.989742+0000","last_clean":"2023-11-08T19:25:51.989742+0000","last_became_active":"2023-11-08T19:25:49.879570+0000","last_became_peered":"2023-11-08T19:25:49.879570+0000","last_unstale":"2023-11-08T19:25:51.989742+0000","last_undegraded":"2023-11-08T19:25:51.989742+0000","last_fullsized":"2023-11-08T19:25:51.989742+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:00:52.169251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.205335+0000","last_change":"2023-11-08T19:25:50.881012+0000","last_active":"2023-11-08T19:25:52.205335+0000","last_peered":"2023-11-08T19:25:52.205335+0000","last_clean":"2023-11-08T19:25:52.205335+0000","last_became_active":"2023-11-08T19:25:50.880752+0000","last_became_peered":"2023-11-08T19:25:50.880752+0000","last_unstale":"2023-11-08T19:25:52.205335+0000","last_undegraded":"2023-11-08T19:25:52.205335+0000","last_fullsized":"2023-11-08T19:25:52.205335+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:33:12.567239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"32'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.203995+0000","last_change":"2023-11-08T19:25:49.886087+0000","last_active":"2023-11-08T19:25:52.203995+0000","last_peered":"2023-11-08T19:25:52.203995+0000","last_clean":"2023-11-08T19:25:52.203995+0000","last_became_active":"2023-11-08T19:25:49.885801+0000","last_became_peered":"2023-11-08T19:25:49.885801+0000","last_unstale":"2023-11-08T19:25:52.203995+0000","last_undegraded":"2023-11-08T19:25:52.203995+0000","last_fullsized":"2023-11-08T19:25:52.203995+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:04:00.482724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.992009+0000","last_change":"2023-11-08T19:25:50.885648+0000","last_active":"2023-11-08T19:25:51.992009+0000","last_peered":"2023-11-08T19:25:51.992009+0000","last_clean":"2023-11-08T19:25:51.992009+0000","last_became_active":"2023-11-08T19:25:50.885179+0000","last_became_peered":"2023-11-08T19:25:50.885179+0000","last_unstale":"2023-11-08T19:25:51.992009+0000","last_undegraded":"2023-11-08T19:25:51.992009+0000","last_fullsized":"2023-11-08T19:25:51.992009+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:52:20.334760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"32'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.876725+0000","last_change":"2023-11-08T19:25:49.883073+0000","last_active":"2023-11-08T19:25:51.876725+0000","last_peered":"2023-11-08T19:25:51.876725+0000","last_clean":"2023-11-08T19:25:51.876725+0000","last_became_active":"2023-11-08T19:25:49.882522+0000","last_became_peered":"2023-11-08T19:25:49.882522+0000","last_unstale":"2023-11-08T19:25:51.876725+0000","last_undegraded":"2023-11-08T19:25:51.876725+0000","last_fullsized":"2023-11-08T19:25:51.876725+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:27:43.830493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:25:49.854746+0000","last_change":"2023-11-08T19:25:49.854746+0000","last_active":"2023-11-08T19:25:49.854746+0000","last_peered":"2023-11-08T19:25:49.854746+0000","last_clean":"2023-11-08T19:25:49.854746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:25:49.854746+0000","last_undegraded":"2023-11-08T19:25:49.854746+0000","last_fullsized":"2023-11-08T19:25:49.854746+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1b","version":"32'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.205020+0000","last_change":"2023-11-08T19:25:49.884628+0000","last_active":"2023-11-08T19:25:52.205020+0000","last_peered":"2023-11-08T19:25:52.205020+0000","last_clean":"2023-11-08T19:25:52.205020+0000","last_became_active":"2023-11-08T19:25:49.884384+0000","last_became_peered":"2023-11-08T19:25:49.884384+0000","last_unstale":"2023-11-08T19:25:52.205020+0000","last_undegraded":"2023-11-08T19:25:52.205020+0000","last_fullsized":"2023-11-08T19:25:52.205020+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:17:52.641253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204834+0000","last_change":"2023-11-08T19:25:50.874965+0000","last_active":"2023-11-08T19:25:52.204834+0000","last_peered":"2023-11-08T19:25:52.204834+0000","last_clean":"2023-11-08T19:25:52.204834+0000","last_became_active":"2023-11-08T19:25:50.874696+0000","last_became_peered":"2023-11-08T19:25:50.874696+0000","last_unstale":"2023-11-08T19:25:52.204834+0000","last_undegraded":"2023-11-08T19:25:52.204834+0000","last_fullsized":"2023-11-08T19:25:52.204834+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:49:41.475597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"32'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204932+0000","last_change":"2023-11-08T19:25:49.884607+0000","last_active":"2023-11-08T19:25:52.204932+0000","last_peered":"2023-11-08T19:25:52.204932+0000","last_clean":"2023-11-08T19:25:52.204932+0000","last_became_active":"2023-11-08T19:25:49.884392+0000","last_became_peered":"2023-11-08T19:25:49.884392+0000","last_unstale":"2023-11-08T19:25:52.204932+0000","last_undegraded":"2023-11-08T19:25:52.204932+0000","last_fullsized":"2023-11-08T19:25:52.204932+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:54:54.192037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204370+0000","last_change":"2023-11-08T19:25:50.881747+0000","last_active":"2023-11-08T19:25:52.204370+0000","last_peered":"2023-11-08T19:25:52.204370+0000","last_clean":"2023-11-08T19:25:52.204370+0000","last_became_active":"2023-11-08T19:25:50.881225+0000","last_became_peered":"2023-11-08T19:25:50.881225+0000","last_unstale":"2023-11-08T19:25:52.204370+0000","last_undegraded":"2023-11-08T19:25:52.204370+0000","last_fullsized":"2023-11-08T19:25:52.204370+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:37:10.264673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.203749+0000","last_change":"2023-11-08T19:25:49.882231+0000","last_active":"2023-11-08T19:25:52.203749+0000","last_peered":"2023-11-08T19:25:52.203749+0000","last_clean":"2023-11-08T19:25:52.203749+0000","last_became_active":"2023-11-08T19:25:49.881992+0000","last_became_peered":"2023-11-08T19:25:49.881992+0000","last_unstale":"2023-11-08T19:25:52.203749+0000","last_undegraded":"2023-11-08T19:25:52.203749+0000","last_fullsized":"2023-11-08T19:25:52.203749+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:00:52.169251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.991649+0000","last_change":"2023-11-08T19:25:50.883482+0000","last_active":"2023-11-08T19:25:51.991649+0000","last_peered":"2023-11-08T19:25:51.991649+0000","last_clean":"2023-11-08T19:25:51.991649+0000","last_became_active":"2023-11-08T19:25:50.883187+0000","last_became_peered":"2023-11-08T19:25:50.883187+0000","last_unstale":"2023-11-08T19:25:51.991649+0000","last_undegraded":"2023-11-08T19:25:51.991649+0000","last_fullsized":"2023-11-08T19:25:51.991649+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:33:12.567239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204890+0000","last_change":"2023-11-08T19:25:49.878388+0000","last_active":"2023-11-08T19:25:52.204890+0000","last_peered":"2023-11-08T19:25:52.204890+0000","last_clean":"2023-11-08T19:25:52.204890+0000","last_became_active":"2023-11-08T19:25:49.878111+0000","last_became_peered":"2023-11-08T19:25:49.878111+0000","last_unstale":"2023-11-08T19:25:52.204890+0000","last_undegraded":"2023-11-08T19:25:52.204890+0000","last_fullsized":"2023-11-08T19:25:52.204890+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:39:22.428409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204894+0000","last_change":"2023-11-08T19:25:50.877613+0000","last_active":"2023-11-08T19:25:52.204894+0000","last_peered":"2023-11-08T19:25:52.204894+0000","last_clean":"2023-11-08T19:25:52.204894+0000","last_became_active":"2023-11-08T19:25:50.877311+0000","last_became_peered":"2023-11-08T19:25:50.877311+0000","last_unstale":"2023-11-08T19:25:52.204894+0000","last_undegraded":"2023-11-08T19:25:52.204894+0000","last_fullsized":"2023-11-08T19:25:52.204894+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:50:52.902620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204190+0000","last_change":"2023-11-08T19:25:49.884355+0000","last_active":"2023-11-08T19:25:52.204190+0000","last_peered":"2023-11-08T19:25:52.204190+0000","last_clean":"2023-11-08T19:25:52.204190+0000","last_became_active":"2023-11-08T19:25:49.884065+0000","last_became_peered":"2023-11-08T19:25:49.884065+0000","last_unstale":"2023-11-08T19:25:52.204190+0000","last_undegraded":"2023-11-08T19:25:52.204190+0000","last_fullsized":"2023-11-08T19:25:52.204190+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:05:32.217545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204103+0000","last_change":"2023-11-08T19:25:50.888556+0000","last_active":"2023-11-08T19:25:52.204103+0000","last_peered":"2023-11-08T19:25:52.204103+0000","last_clean":"2023-11-08T19:25:52.204103+0000","last_became_active":"2023-11-08T19:25:50.887909+0000","last_became_peered":"2023-11-08T19:25:50.887909+0000","last_unstale":"2023-11-08T19:25:52.204103+0000","last_undegraded":"2023-11-08T19:25:52.204103+0000","last_fullsized":"2023-11-08T19:25:52.204103+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:38:37.341013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.989663+0000","last_change":"2023-11-08T19:25:49.884919+0000","last_active":"2023-11-08T19:25:51.989663+0000","last_peered":"2023-11-08T19:25:51.989663+0000","last_clean":"2023-11-08T19:25:51.989663+0000","last_became_active":"2023-11-08T19:25:49.884679+0000","last_became_peered":"2023-11-08T19:25:49.884679+0000","last_unstale":"2023-11-08T19:25:51.989663+0000","last_undegraded":"2023-11-08T19:25:51.989663+0000","last_fullsized":"2023-11-08T19:25:51.989663+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:39:22.428409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204236+0000","last_change":"2023-11-08T19:25:50.882028+0000","last_active":"2023-11-08T19:25:52.204236+0000","last_peered":"2023-11-08T19:25:52.204236+0000","last_clean":"2023-11-08T19:25:52.204236+0000","last_became_active":"2023-11-08T19:25:50.881746+0000","last_became_peered":"2023-11-08T19:25:50.881746+0000","last_unstale":"2023-11-08T19:25:52.204236+0000","last_undegraded":"2023-11-08T19:25:52.204236+0000","last_fullsized":"2023-11-08T19:25:52.204236+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:39:57.441098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"32'3","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204967+0000","last_change":"2023-11-08T19:25:49.878409+0000","last_active":"2023-11-08T19:25:52.204967+0000","last_peered":"2023-11-08T19:25:52.204967+0000","last_clean":"2023-11-08T19:25:52.204967+0000","last_became_active":"2023-11-08T19:25:49.878105+0000","last_became_peered":"2023-11-08T19:25:49.878105+0000","last_unstale":"2023-11-08T19:25:52.204967+0000","last_undegraded":"2023-11-08T19:25:52.204967+0000","last_fullsized":"2023-11-08T19:25:52.204967+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:55:30.168605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204866+0000","last_change":"2023-11-08T19:25:50.887174+0000","last_active":"2023-11-08T19:25:52.204866+0000","last_peered":"2023-11-08T19:25:52.204866+0000","last_clean":"2023-11-08T19:25:52.204866+0000","last_became_active":"2023-11-08T19:25:50.886902+0000","last_became_peered":"2023-11-08T19:25:50.886902+0000","last_unstale":"2023-11-08T19:25:52.204866+0000","last_undegraded":"2023-11-08T19:25:52.204866+0000","last_fullsized":"2023-11-08T19:25:52.204866+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:19:42.562982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"32'6","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204091+0000","last_change":"2023-11-08T19:25:49.886499+0000","last_active":"2023-11-08T19:25:52.204091+0000","last_peered":"2023-11-08T19:25:52.204091+0000","last_clean":"2023-11-08T19:25:52.204091+0000","last_became_active":"2023-11-08T19:25:49.886169+0000","last_became_peered":"2023-11-08T19:25:49.886169+0000","last_unstale":"2023-11-08T19:25:52.204091+0000","last_undegraded":"2023-11-08T19:25:52.204091+0000","last_fullsized":"2023-11-08T19:25:52.204091+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:19:41.552717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.991670+0000","last_change":"2023-11-08T19:25:50.885893+0000","last_active":"2023-11-08T19:25:51.991670+0000","last_peered":"2023-11-08T19:25:51.991670+0000","last_clean":"2023-11-08T19:25:51.991670+0000","last_became_active":"2023-11-08T19:25:50.880205+0000","last_became_peered":"2023-11-08T19:25:50.880205+0000","last_unstale":"2023-11-08T19:25:51.991670+0000","last_undegraded":"2023-11-08T19:25:51.991670+0000","last_fullsized":"2023-11-08T19:25:51.991670+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:55:31.178870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"32'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.205247+0000","last_change":"2023-11-08T19:25:49.879256+0000","last_active":"2023-11-08T19:25:52.205247+0000","last_peered":"2023-11-08T19:25:52.205247+0000","last_clean":"2023-11-08T19:25:52.205247+0000","last_became_active":"2023-11-08T19:25:49.878963+0000","last_became_peered":"2023-11-08T19:25:49.878963+0000","last_unstale":"2023-11-08T19:25:52.205247+0000","last_undegraded":"2023-11-08T19:25:52.205247+0000","last_fullsized":"2023-11-08T19:25:52.205247+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:12:39.742779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.203953+0000","last_change":"2023-11-08T19:25:50.891183+0000","last_active":"2023-11-08T19:25:52.203953+0000","last_peered":"2023-11-08T19:25:52.203953+0000","last_clean":"2023-11-08T19:25:52.203953+0000","last_became_active":"2023-11-08T19:25:50.890889+0000","last_became_peered":"2023-11-08T19:25:50.890889+0000","last_unstale":"2023-11-08T19:25:52.203953+0000","last_undegraded":"2023-11-08T19:25:52.203953+0000","last_fullsized":"2023-11-08T19:25:52.203953+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:25:16.223226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.991756+0000","last_change":"2023-11-08T19:25:49.880102+0000","last_active":"2023-11-08T19:25:51.991756+0000","last_peered":"2023-11-08T19:25:51.991756+0000","last_clean":"2023-11-08T19:25:51.991756+0000","last_became_active":"2023-11-08T19:25:49.879816+0000","last_became_peered":"2023-11-08T19:25:49.879816+0000","last_unstale":"2023-11-08T19:25:51.991756+0000","last_undegraded":"2023-11-08T19:25:51.991756+0000","last_fullsized":"2023-11-08T19:25:51.991756+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:37:09.254408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:25:49.854746+0000","last_change":"2023-11-08T19:25:49.854746+0000","last_active":"2023-11-08T19:25:49.854746+0000","last_peered":"2023-11-08T19:25:49.854746+0000","last_clean":"2023-11-08T19:25:49.854746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:25:49.854746+0000","last_undegraded":"2023-11-08T19:25:49.854746+0000","last_fullsized":"2023-11-08T19:25:49.854746+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.203841+0000","last_change":"2023-11-08T19:25:49.884359+0000","last_active":"2023-11-08T19:25:52.203841+0000","last_peered":"2023-11-08T19:25:52.203841+0000","last_clean":"2023-11-08T19:25:52.203841+0000","last_became_active":"2023-11-08T19:25:49.884119+0000","last_became_peered":"2023-11-08T19:25:49.884119+0000","last_unstale":"2023-11-08T19:25:52.203841+0000","last_undegraded":"2023-11-08T19:25:52.203841+0000","last_fullsized":"2023-11-08T19:25:52.203841+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:55:30.168605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:25:49.854746+0000","last_change":"2023-11-08T19:25:49.854746+0000","last_active":"2023-11-08T19:25:49.854746+0000","last_peered":"2023-11-08T19:25:49.854746+0000","last_clean":"2023-11-08T19:25:49.854746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:25:49.854746+0000","last_undegraded":"2023-11-08T19:25:49.854746+0000","last_fullsized":"2023-11-08T19:25:49.854746+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.10","version":"32'8","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.203707+0000","last_change":"2023-11-08T19:25:49.886483+0000","last_active":"2023-11-08T19:25:52.203707+0000","last_peered":"2023-11-08T19:25:52.203707+0000","last_clean":"2023-11-08T19:25:52.203707+0000","last_became_active":"2023-11-08T19:25:49.886209+0000","last_became_peered":"2023-11-08T19:25:49.886209+0000","last_unstale":"2023-11-08T19:25:52.203707+0000","last_undegraded":"2023-11-08T19:25:52.203707+0000","last_fullsized":"2023-11-08T19:25:52.203707+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:49:40.465332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.205492+0000","last_change":"2023-11-08T19:25:50.884739+0000","last_active":"2023-11-08T19:25:52.205492+0000","last_peered":"2023-11-08T19:25:52.205492+0000","last_clean":"2023-11-08T19:25:52.205492+0000","last_became_active":"2023-11-08T19:25:50.884476+0000","last_became_peered":"2023-11-08T19:25:50.884476+0000","last_unstale":"2023-11-08T19:25:52.205492+0000","last_undegraded":"2023-11-08T19:25:52.205492+0000","last_fullsized":"2023-11-08T19:25:52.205492+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:55:31.178870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"32'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.992046+0000","last_change":"2023-11-08T19:25:49.884770+0000","last_active":"2023-11-08T19:25:51.992046+0000","last_peered":"2023-11-08T19:25:51.992046+0000","last_clean":"2023-11-08T19:25:51.992046+0000","last_became_active":"2023-11-08T19:25:49.884476+0000","last_became_peered":"2023-11-08T19:25:49.884476+0000","last_unstale":"2023-11-08T19:25:51.992046+0000","last_undegraded":"2023-11-08T19:25:51.992046+0000","last_fullsized":"2023-11-08T19:25:51.992046+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:18:56.855441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204945+0000","last_change":"2023-11-08T19:25:50.877298+0000","last_active":"2023-11-08T19:25:52.204945+0000","last_peered":"2023-11-08T19:25:52.204945+0000","last_clean":"2023-11-08T19:25:52.204945+0000","last_became_active":"2023-11-08T19:25:50.877018+0000","last_became_peered":"2023-11-08T19:25:50.877018+0000","last_unstale":"2023-11-08T19:25:52.204945+0000","last_undegraded":"2023-11-08T19:25:52.204945+0000","last_fullsized":"2023-11-08T19:25:52.204945+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:25:07.133280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"32'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.205477+0000","last_change":"2023-11-08T19:25:49.878520+0000","last_active":"2023-11-08T19:25:52.205477+0000","last_peered":"2023-11-08T19:25:52.205477+0000","last_clean":"2023-11-08T19:25:52.205477+0000","last_became_active":"2023-11-08T19:25:49.878244+0000","last_became_peered":"2023-11-08T19:25:49.878244+0000","last_unstale":"2023-11-08T19:25:52.205477+0000","last_undegraded":"2023-11-08T19:25:52.205477+0000","last_fullsized":"2023-11-08T19:25:52.205477+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:23:30.383150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.991899+0000","last_change":"2023-11-08T19:25:50.878056+0000","last_active":"2023-11-08T19:25:51.991899+0000","last_peered":"2023-11-08T19:25:51.991899+0000","last_clean":"2023-11-08T19:25:51.991899+0000","last_became_active":"2023-11-08T19:25:50.877738+0000","last_became_peered":"2023-11-08T19:25:50.877738+0000","last_unstale":"2023-11-08T19:25:51.991899+0000","last_undegraded":"2023-11-08T19:25:51.991899+0000","last_fullsized":"2023-11-08T19:25:51.991899+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:04:01.492989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"32'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.205456+0000","last_change":"2023-11-08T19:25:49.881744+0000","last_active":"2023-11-08T19:25:52.205456+0000","last_peered":"2023-11-08T19:25:52.205456+0000","last_clean":"2023-11-08T19:25:52.205456+0000","last_became_active":"2023-11-08T19:25:49.881436+0000","last_became_peered":"2023-11-08T19:25:49.881436+0000","last_unstale":"2023-11-08T19:25:52.205456+0000","last_undegraded":"2023-11-08T19:25:52.205456+0000","last_fullsized":"2023-11-08T19:25:52.205456+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:08:07.992929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.876822+0000","last_change":"2023-11-08T19:25:50.881348+0000","last_active":"2023-11-08T19:25:51.876822+0000","last_peered":"2023-11-08T19:25:51.876822+0000","last_clean":"2023-11-08T19:25:51.876822+0000","last_became_active":"2023-11-08T19:25:50.881064+0000","last_became_peered":"2023-11-08T19:25:50.881064+0000","last_unstale":"2023-11-08T19:25:51.876822+0000","last_undegraded":"2023-11-08T19:25:51.876822+0000","last_fullsized":"2023-11-08T19:25:51.876822+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:23:31.393415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.205574+0000","last_change":"2023-11-08T19:25:49.884654+0000","last_active":"2023-11-08T19:25:52.205574+0000","last_peered":"2023-11-08T19:25:52.205574+0000","last_clean":"2023-11-08T19:25:52.205574+0000","last_became_active":"2023-11-08T19:25:49.884322+0000","last_became_peered":"2023-11-08T19:25:49.884322+0000","last_unstale":"2023-11-08T19:25:52.205574+0000","last_undegraded":"2023-11-08T19:25:52.205574+0000","last_fullsized":"2023-11-08T19:25:52.205574+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:49:07.738959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.876689+0000","last_change":"2023-11-08T19:25:50.878414+0000","last_active":"2023-11-08T19:25:51.876689+0000","last_peered":"2023-11-08T19:25:51.876689+0000","last_clean":"2023-11-08T19:25:51.876689+0000","last_became_active":"2023-11-08T19:25:50.878145+0000","last_became_peered":"2023-11-08T19:25:50.878145+0000","last_unstale":"2023-11-08T19:25:51.876689+0000","last_undegraded":"2023-11-08T19:25:51.876689+0000","last_fullsized":"2023-11-08T19:25:51.876689+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:49:08.749224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"32'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.989615+0000","last_change":"2023-11-08T19:25:49.878923+0000","last_active":"2023-11-08T19:25:51.989615+0000","last_peered":"2023-11-08T19:25:51.989615+0000","last_clean":"2023-11-08T19:25:51.989615+0000","last_became_active":"2023-11-08T19:25:49.878167+0000","last_became_peered":"2023-11-08T19:25:49.878167+0000","last_unstale":"2023-11-08T19:25:51.989615+0000","last_undegraded":"2023-11-08T19:25:51.989615+0000","last_fullsized":"2023-11-08T19:25:51.989615+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:51:26.807463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.876711+0000","last_change":"2023-11-08T19:25:50.887699+0000","last_active":"2023-11-08T19:25:51.876711+0000","last_peered":"2023-11-08T19:25:51.876711+0000","last_clean":"2023-11-08T19:25:51.876711+0000","last_became_active":"2023-11-08T19:25:50.887423+0000","last_became_peered":"2023-11-08T19:25:50.887423+0000","last_unstale":"2023-11-08T19:25:51.876711+0000","last_undegraded":"2023-11-08T19:25:51.876711+0000","last_fullsized":"2023-11-08T19:25:51.876711+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:08:09.003194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-11-08T19:25:48.857323+0000","last_change":"2023-11-08T19:25:48.857323+0000","last_active":"2023-11-08T19:25:48.844481+0000","last_peered":"2023-11-08T19:25:48.844481+0000","last_clean":"2023-11-08T19:25:48.844481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:25:48.857323+0000","last_undegraded":"2023-11-08T19:25:48.857323+0000","last_fullsized":"2023-11-08T19:25:48.857323+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.876619+0000","last_change":"2023-11-08T19:25:50.888597+0000","last_active":"2023-11-08T19:25:51.876619+0000","last_peered":"2023-11-08T19:25:51.876619+0000","last_clean":"2023-11-08T19:25:51.876619+0000","last_became_active":"2023-11-08T19:25:50.887065+0000","last_became_peered":"2023-11-08T19:25:50.887065+0000","last_unstale":"2023-11-08T19:25:51.876619+0000","last_undegraded":"2023-11-08T19:25:51.876619+0000","last_fullsized":"2023-11-08T19:25:51.876619+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:02:25.988939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.205088+0000","last_change":"2023-11-08T19:25:49.884053+0000","last_active":"2023-11-08T19:25:52.205088+0000","last_peered":"2023-11-08T19:25:52.205088+0000","last_clean":"2023-11-08T19:25:52.205088+0000","last_became_active":"2023-11-08T19:25:49.883822+0000","last_became_peered":"2023-11-08T19:25:49.883822+0000","last_unstale":"2023-11-08T19:25:52.205088+0000","last_undegraded":"2023-11-08T19:25:52.205088+0000","last_fullsized":"2023-11-08T19:25:52.205088+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:12:47.116061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:25:49.854746+0000","last_change":"2023-11-08T19:25:49.854746+0000","last_active":"2023-11-08T19:25:49.854746+0000","last_peered":"2023-11-08T19:25:49.854746+0000","last_clean":"2023-11-08T19:25:49.854746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:25:49.854746+0000","last_undegraded":"2023-11-08T19:25:49.854746+0000","last_fullsized":"2023-11-08T19:25:49.854746+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204303+0000","last_change":"2023-11-08T19:25:49.879327+0000","last_active":"2023-11-08T19:25:52.204303+0000","last_peered":"2023-11-08T19:25:52.204303+0000","last_clean":"2023-11-08T19:25:52.204303+0000","last_became_active":"2023-11-08T19:25:49.879054+0000","last_became_peered":"2023-11-08T19:25:49.879054+0000","last_unstale":"2023-11-08T19:25:52.204303+0000","last_undegraded":"2023-11-08T19:25:52.204303+0000","last_fullsized":"2023-11-08T19:25:52.204303+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:09:04.368768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.876752+0000","last_change":"2023-11-08T19:25:50.874836+0000","last_active":"2023-11-08T19:25:51.876752+0000","last_peered":"2023-11-08T19:25:51.876752+0000","last_clean":"2023-11-08T19:25:51.876752+0000","last_became_active":"2023-11-08T19:25:50.874272+0000","last_became_peered":"2023-11-08T19:25:50.874272+0000","last_unstale":"2023-11-08T19:25:51.876752+0000","last_undegraded":"2023-11-08T19:25:51.876752+0000","last_fullsized":"2023-11-08T19:25:51.876752+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:17:53.651518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"32'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.991956+0000","last_change":"2023-11-08T19:25:49.880740+0000","last_active":"2023-11-08T19:25:51.991956+0000","last_peered":"2023-11-08T19:25:51.991956+0000","last_clean":"2023-11-08T19:25:51.991956+0000","last_became_active":"2023-11-08T19:25:49.880458+0000","last_became_peered":"2023-11-08T19:25:49.880458+0000","last_unstale":"2023-11-08T19:25:51.991956+0000","last_undegraded":"2023-11-08T19:25:51.991956+0000","last_fullsized":"2023-11-08T19:25:51.991956+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:20:42.164809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.205146+0000","last_change":"2023-11-08T19:25:50.883014+0000","last_active":"2023-11-08T19:25:52.205146+0000","last_peered":"2023-11-08T19:25:52.205146+0000","last_clean":"2023-11-08T19:25:52.205146+0000","last_became_active":"2023-11-08T19:25:50.882737+0000","last_became_peered":"2023-11-08T19:25:50.882737+0000","last_unstale":"2023-11-08T19:25:52.205146+0000","last_undegraded":"2023-11-08T19:25:52.205146+0000","last_fullsized":"2023-11-08T19:25:52.205146+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:38:37.341013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"33'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.991098+0000","last_change":"2023-11-08T19:25:49.880299+0000","last_active":"2023-11-08T19:25:51.991098+0000","last_peered":"2023-11-08T19:25:51.991098+0000","last_clean":"2023-11-08T19:25:51.991098+0000","last_became_active":"2023-11-08T19:25:49.878038+0000","last_became_peered":"2023-11-08T19:25:49.878038+0000","last_unstale":"2023-11-08T19:25:51.991098+0000","last_undegraded":"2023-11-08T19:25:51.991098+0000","last_fullsized":"2023-11-08T19:25:51.991098+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:12:39.742779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.989626+0000","last_change":"2023-11-08T19:25:50.888755+0000","last_active":"2023-11-08T19:25:51.989626+0000","last_peered":"2023-11-08T19:25:51.989626+0000","last_clean":"2023-11-08T19:25:51.989626+0000","last_became_active":"2023-11-08T19:25:50.888494+0000","last_became_peered":"2023-11-08T19:25:50.888494+0000","last_unstale":"2023-11-08T19:25:51.989626+0000","last_undegraded":"2023-11-08T19:25:51.989626+0000","last_fullsized":"2023-11-08T19:25:51.989626+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:09:05.379033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.991905+0000","last_change":"2023-11-08T19:25:49.881699+0000","last_active":"2023-11-08T19:25:51.991905+0000","last_peered":"2023-11-08T19:25:51.991905+0000","last_clean":"2023-11-08T19:25:51.991905+0000","last_became_active":"2023-11-08T19:25:49.881062+0000","last_became_peered":"2023-11-08T19:25:49.881062+0000","last_unstale":"2023-11-08T19:25:51.991905+0000","last_undegraded":"2023-11-08T19:25:51.991905+0000","last_fullsized":"2023-11-08T19:25:51.991905+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:49:44.049659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:25:49.854746+0000","last_change":"2023-11-08T19:25:49.854746+0000","last_active":"2023-11-08T19:25:49.854746+0000","last_peered":"2023-11-08T19:25:49.854746+0000","last_clean":"2023-11-08T19:25:49.854746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:25:49.854746+0000","last_undegraded":"2023-11-08T19:25:49.854746+0000","last_fullsized":"2023-11-08T19:25:49.854746+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.989575+0000","last_change":"2023-11-08T19:25:49.879113+0000","last_active":"2023-11-08T19:25:51.989575+0000","last_peered":"2023-11-08T19:25:51.989575+0000","last_clean":"2023-11-08T19:25:51.989575+0000","last_became_active":"2023-11-08T19:25:49.878579+0000","last_became_peered":"2023-11-08T19:25:49.878579+0000","last_unstale":"2023-11-08T19:25:51.989575+0000","last_undegraded":"2023-11-08T19:25:51.989575+0000","last_fullsized":"2023-11-08T19:25:51.989575+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:12:47.116061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.989707+0000","last_change":"2023-11-08T19:25:50.880600+0000","last_active":"2023-11-08T19:25:51.989707+0000","last_peered":"2023-11-08T19:25:51.989707+0000","last_clean":"2023-11-08T19:25:51.989707+0000","last_became_active":"2023-11-08T19:25:50.880384+0000","last_became_peered":"2023-11-08T19:25:50.880384+0000","last_unstale":"2023-11-08T19:25:51.989707+0000","last_undegraded":"2023-11-08T19:25:51.989707+0000","last_fullsized":"2023-11-08T19:25:51.989707+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:20:43.175074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.991693+0000","last_change":"2023-11-08T19:25:49.877196+0000","last_active":"2023-11-08T19:25:51.991693+0000","last_peered":"2023-11-08T19:25:51.991693+0000","last_clean":"2023-11-08T19:25:51.991693+0000","last_became_active":"2023-11-08T19:25:49.876517+0000","last_became_peered":"2023-11-08T19:25:49.876517+0000","last_unstale":"2023-11-08T19:25:51.991693+0000","last_undegraded":"2023-11-08T19:25:51.991693+0000","last_fullsized":"2023-11-08T19:25:51.991693+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:00:52.169251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.203760+0000","last_change":"2023-11-08T19:25:50.893157+0000","last_active":"2023-11-08T19:25:52.203760+0000","last_peered":"2023-11-08T19:25:52.203760+0000","last_clean":"2023-11-08T19:25:52.203760+0000","last_became_active":"2023-11-08T19:25:50.892637+0000","last_became_peered":"2023-11-08T19:25:50.892637+0000","last_unstale":"2023-11-08T19:25:52.203760+0000","last_undegraded":"2023-11-08T19:25:52.203760+0000","last_fullsized":"2023-11-08T19:25:52.203760+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:50:52.902620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.205077+0000","last_change":"2023-11-08T19:25:49.881923+0000","last_active":"2023-11-08T19:25:52.205077+0000","last_peered":"2023-11-08T19:25:52.205077+0000","last_clean":"2023-11-08T19:25:52.205077+0000","last_became_active":"2023-11-08T19:25:49.881670+0000","last_became_peered":"2023-11-08T19:25:49.881670+0000","last_unstale":"2023-11-08T19:25:52.205077+0000","last_undegraded":"2023-11-08T19:25:52.205077+0000","last_fullsized":"2023-11-08T19:25:52.205077+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:51:26.807463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204772+0000","last_change":"2023-11-08T19:25:50.883910+0000","last_active":"2023-11-08T19:25:52.204772+0000","last_peered":"2023-11-08T19:25:52.204772+0000","last_clean":"2023-11-08T19:25:52.204772+0000","last_became_active":"2023-11-08T19:25:50.883615+0000","last_became_peered":"2023-11-08T19:25:50.883615+0000","last_unstale":"2023-11-08T19:25:52.204772+0000","last_undegraded":"2023-11-08T19:25:52.204772+0000","last_fullsized":"2023-11-08T19:25:52.204772+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:20:43.175074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.203857+0000","last_change":"2023-11-08T19:25:49.884441+0000","last_active":"2023-11-08T19:25:52.203857+0000","last_peered":"2023-11-08T19:25:52.203857+0000","last_clean":"2023-11-08T19:25:52.203857+0000","last_became_active":"2023-11-08T19:25:49.884125+0000","last_became_peered":"2023-11-08T19:25:49.884125+0000","last_unstale":"2023-11-08T19:25:52.203857+0000","last_undegraded":"2023-11-08T19:25:52.203857+0000","last_fullsized":"2023-11-08T19:25:52.203857+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:52:19.324495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:25:49.854746+0000","last_change":"2023-11-08T19:25:49.854746+0000","last_active":"2023-11-08T19:25:49.854746+0000","last_peered":"2023-11-08T19:25:49.854746+0000","last_clean":"2023-11-08T19:25:49.854746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:25:49.854746+0000","last_undegraded":"2023-11-08T19:25:49.854746+0000","last_fullsized":"2023-11-08T19:25:49.854746+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-11-08T19:25:48.857272+0000","last_change":"2023-11-08T19:25:48.857272+0000","last_active":"2023-11-08T19:25:48.844481+0000","last_peered":"2023-11-08T19:25:48.844481+0000","last_clean":"2023-11-08T19:25:48.844481+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:25:48.857272+0000","last_undegraded":"2023-11-08T19:25:48.857272+0000","last_fullsized":"2023-11-08T19:25:48.857272+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":175,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-11-08T19:25:43.589068+0000","last_change":"2023-11-08T19:24:45.888122+0000","last_active":"2023-11-08T19:25:43.589068+0000","last_peered":"2023-11-08T19:25:43.589068+0000","last_clean":"2023-11-08T19:25:43.589068+0000","last_became_active":"2023-11-08T19:24:45.887048+0000","last_became_peered":"2023-11-08T19:24:45.887048+0000","last_unstale":"2023-11-08T19:25:43.589068+0000","last_undegraded":"2023-11-08T19:25:43.589068+0000","last_fullsized":"2023-11-08T19:25:43.589068+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:44.575133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:44.575133+0000","last_clean_scrub_stamp":"2023-11-08T19:24:44.575133+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:19:20.202062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":96,"acting":96,"num_store_stats":6},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":1828,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":4114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":35,"ondisk_log_size":35,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574865,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27928,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343912,"statfs":{"total":96636764160,"available":96608165888,"internally_reserved":0,"allocated":929792,"data_stored":716892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.60999999999999999}]},{"osd":1,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.76100000000000001}]},{"osd":2,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.63200000000000001}]},{"osd":3,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.746},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.67800000000000005}]},{"osd":4,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.58299999999999996}]},{"osd":5,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.71999999999999997}]},{"osd":6,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.64400000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":335872,"data_stored":126524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":21.359999999999999,"5min":21.359999999999999,"15min":21.359999999999999},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":10.329000000000001,"5min":10.329000000000001,"15min":10.329000000000001},"last":0.46700000000000003}]},{"osd":1,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":3.4239999999999999,"5min":3.4239999999999999,"15min":3.4239999999999999},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":3.4809999999999999,"5min":3.4809999999999999,"15min":3.4809999999999999},"last":0.66500000000000004}]},{"osd":2,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":10.398,"5min":10.398,"15min":10.398},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":5.0170000000000003,"5min":5.0170000000000003,"15min":5.0170000000000003},"last":0.60999999999999999}]},{"osd":3,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":3.5710000000000002,"5min":3.5710000000000002,"15min":3.5710000000000002},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":10.368,"5min":10.368,"15min":10.368},"last":0.70299999999999996}]},{"osd":4,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.78500000000000003}]},{"osd":5,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.753},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.82099999999999995}]},{"osd":7,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.80700000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574866,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27404,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344436,"statfs":{"total":96636764160,"available":96608702464,"internally_reserved":0,"allocated":389120,"data_stored":147500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":1.389,"5min":1.389,"15min":1.389},"last":0.499},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":9.1370000000000005,"5min":9.1370000000000005,"15min":9.1370000000000005},"last":0.39400000000000002}]},{"osd":2,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.433,"5min":0.433,"15min":0.433},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":1.425,"5min":1.425,"15min":1.425},"last":0.42699999999999999}]},{"osd":3,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":11.957000000000001,"5min":11.957000000000001,"15min":11.957000000000001},"last":0.47699999999999998},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":18.039000000000001,"5min":18.039000000000001,"15min":18.039000000000001},"last":0.46300000000000002}]},{"osd":4,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":6.181,"5min":6.181,"15min":6.181},"last":1.4370000000000001},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.488,"5min":1.488,"15min":1.488},"last":1.411}]},{"osd":5,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":9.2420000000000009,"5min":9.2420000000000009,"15min":9.2420000000000009},"last":1.421},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":6.2110000000000003,"5min":6.2110000000000003,"15min":6.2110000000000003},"last":1.2969999999999999}]},{"osd":6,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":9.1880000000000006,"5min":9.1880000000000006,"15min":9.1880000000000006},"last":1.4530000000000001},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.462,"5min":1.462,"15min":1.462},"last":1.272}]},{"osd":7,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":6.1479999999999997,"5min":6.1479999999999997,"15min":6.1479999999999997},"last":1.3979999999999999},{"interface":"front","average":{"1min":1.028,"5min":1.028,"15min":1.028},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":9.2170000000000005,"5min":9.2170000000000005,"15min":9.2170000000000005},"last":1.353}]}]},{"osd":0,"up_from":13,"seq":55834574866,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27972,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343868,"statfs":{"total":96636764160,"available":96608120832,"internally_reserved":0,"allocated":966656,"data_stored":736234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":4.1200000000000001,"5min":4.1200000000000001,"15min":4.1200000000000001},"last":0.32200000000000001},{"interface":"front","average":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":16.940999999999999,"5min":16.940999999999999,"15min":16.940999999999999},"last":0.44500000000000001}]},{"osd":2,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.37},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":7.0209999999999999,"5min":7.0209999999999999,"15min":7.0209999999999999},"last":0.41399999999999998}]},{"osd":3,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":4.0940000000000003,"5min":4.0940000000000003,"15min":4.0940000000000003},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.432}]},{"osd":4,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":4.0049999999999999,"5min":4.0049999999999999,"15min":4.0049999999999999},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.79200000000000004}]},{"osd":5,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":4.3090000000000002,"5min":4.3090000000000002,"15min":4.3090000000000002},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":4.0709999999999997,"5min":4.0709999999999997,"15min":4.0709999999999997},"last":0.67300000000000004}]},{"osd":6,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":4.2560000000000002,"5min":4.2560000000000002,"15min":4.2560000000000002},"last":0.75900000000000001}]},{"osd":7,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":4.2850000000000001,"5min":4.2850000000000001,"15min":4.2850000000000001},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":4.0460000000000003,"5min":4.0460000000000003,"15min":4.0460000000000003},"last":0.80600000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574866,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27392,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":376832,"data_stored":145936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"min":{"1min":0.20100000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":6.0999999999999996,"5min":6.0999999999999996,"15min":6.0999999999999996},"last":0.249}]},{"osd":1,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"last":0.28799999999999998},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.624,"5min":0.624,"15min":0.624},"last":0.313}]},{"osd":3,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.38300000000000001}]},{"osd":4,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.55500000000000005}]},{"osd":5,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.57599999999999996}]},{"osd":6,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.65100000000000002}]},{"osd":7,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.628},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.80000000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574867,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27404,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344436,"statfs":{"total":96636764160,"available":96608702464,"internally_reserved":0,"allocated":389120,"data_stored":147500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:25:46 2023","interfaces":[{"interface":"back","average":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.626,"5min":0.626,"15min":0.626},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.33100000000000002}]},{"osd":1,"last update":"Wed Nov 8 19:25:46 2023","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.34999999999999998}]},{"osd":2,"last update":"Wed Nov 8 19:25:46 2023","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.37,"15min":0.37},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"last":0.40000000000000002}]},{"osd":4,"last update":"Wed Nov 8 19:25:46 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.70599999999999996}]},{"osd":5,"last update":"Wed Nov 8 19:25:46 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.56699999999999995}]},{"osd":6,"last update":"Wed Nov 8 19:25:46 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.74199999999999999}]},{"osd":7,"last update":"Wed Nov 8 19:25:46 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.66400000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574866,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":380928,"data_stored":145970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:25:47 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":6.5970000000000004,"5min":6.5970000000000004,"15min":6.5970000000000004},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":2.5030000000000001,"5min":2.5030000000000001,"15min":2.5030000000000001},"last":0.75}]},{"osd":1,"last update":"Wed Nov 8 19:25:47 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.72599999999999998}]},{"osd":2,"last update":"Wed Nov 8 19:25:47 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":6.6920000000000002,"5min":6.6920000000000002,"15min":6.6920000000000002},"last":0.871}]},{"osd":3,"last update":"Wed Nov 8 19:25:47 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.91700000000000004}]},{"osd":5,"last update":"Wed Nov 8 19:25:47 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.65900000000000003}]},{"osd":6,"last update":"Wed Nov 8 19:25:47 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.78100000000000003}]},{"osd":7,"last update":"Wed Nov 8 19:25:47 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.85299999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574866,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":380928,"data_stored":146538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.627},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.438}]},{"osd":1,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.50600000000000001}]},{"osd":2,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.46300000000000002}]},{"osd":3,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.58299999999999996}]},{"osd":4,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.59699999999999998}]},{"osd":6,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.67700000000000005}]},{"osd":7,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.61499999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-08T19:25:56.750 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T19:25:56.753 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T19:25:59.533 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-08T19:25:59.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:59.666+0000 7ff1391e4700 1 Processor -- start 2023-11-08T19:25:59.668 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:59.666+0000 7ff1391e4700 1 -- start start 2023-11-08T19:25:59.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:59.667+0000 7ff1391e4700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff13415dd00 0x7ff1340cca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:59.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:59.667+0000 7ff1391e4700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff13406b320 0x7ff1340cb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:59.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:59.667+0000 7ff1391e4700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1340ccfb0 con 0x7ff1341618f0 2023-11-08T19:25:59.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:59.667+0000 7ff1391e4700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1340cad50 con 0x7ff13406b320 2023-11-08T19:25:59.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:59.667+0000 7ff1391e4700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff134161df0 con 0x7ff13415dd00 2023-11-08T19:25:59.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:59.667+0000 7ff13359e700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff13406b320 0x7ff1340cb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:59.672 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:59.667+0000 7ff13359e700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff13406b320 0x7ff1340cb1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50756/0 (socket says 172.21.15.143:50756) 2023-11-08T19:25:59.672 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:59.667+0000 7ff13359e700 1 -- 172.21.15.143:0/1832105547 learned_addr learned my addr 172.21.15.143:0/1832105547 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:59.672 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:59.667+0000 7ff132d9d700 1 --2- 172.21.15.143:0/1832105547 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff13415dd00 0x7ff1340cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:59.672 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:59.667+0000 7ff13359e700 1 -- 172.21.15.143:0/1832105547 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff13415dd00 msgr2=0x7ff1340cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:59.672 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:59.667+0000 7ff13359e700 1 --2- 172.21.15.143:0/1832105547 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff13415dd00 0x7ff1340cca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:59.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:59.667+0000 7ff13359e700 1 -- 172.21.15.143:0/1832105547 >> v1:172.21.15.174:6789/0 conn(0x7ff1341618f0 legacy=0x7ff13406ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:59.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:59.667+0000 7ff13359e700 1 -- 172.21.15.143:0/1832105547 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1341620d0 con 0x7ff13406b320 2023-11-08T19:25:59.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:59.667+0000 7ff13359e700 1 --2- 172.21.15.143:0/1832105547 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff13406b320 0x7ff1340cb1a0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7ff12c009e50 tx=0x7ff12c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9b20c96c9ebb451 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:59.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:59.667+0000 7ff131d9b700 1 -- 172.21.15.143:0/1832105547 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff12c00e070 con 0x7ff13406b320 2023-11-08T19:25:59.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:59.668+0000 7ff131d9b700 1 -- 172.21.15.143:0/1832105547 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff12c00b9c0 con 0x7ff13406b320 2023-11-08T19:25:59.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:59.668+0000 7ff131d9b700 1 -- 172.21.15.143:0/1832105547 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff12c00bd70 con 0x7ff13406b320 2023-11-08T19:25:59.674 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:59.668+0000 7ff1391e4700 1 -- 172.21.15.143:0/1832105547 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff13406b320 msgr2=0x7ff1340cb1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:59.674 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:59.668+0000 7ff1391e4700 1 --2- 172.21.15.143:0/1832105547 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff13406b320 0x7ff1340cb1a0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7ff12c009e50 tx=0x7ff12c00b330 comp rx=0 tx=0).stop 2023-11-08T19:25:59.674 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:59.668+0000 7ff1391e4700 1 -- 172.21.15.143:0/1832105547 shutdown_connections 2023-11-08T19:25:59.674 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:59.668+0000 7ff1391e4700 1 --2- 172.21.15.143:0/1832105547 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff13406b320 0x7ff1340cb1a0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:59.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:59.668+0000 7ff1391e4700 1 --2- 172.21.15.143:0/1832105547 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff13415dd00 0x7ff1340cca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:59.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:59.668+0000 7ff1391e4700 1 -- 172.21.15.143:0/1832105547 >> 172.21.15.143:0/1832105547 conn(0x7ff134099ab0 msgr2=0x7ff1340d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:59.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:59.668+0000 7ff1391e4700 1 -- 172.21.15.143:0/1832105547 shutdown_connections 2023-11-08T19:25:59.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:59.668+0000 7ff1391e4700 1 -- 172.21.15.143:0/1832105547 wait complete. 2023-11-08T19:25:59.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:59.670+0000 7ff1391e4700 1 Processor -- start 2023-11-08T19:25:59.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:59.670+0000 7ff1391e4700 1 -- start start 2023-11-08T19:25:59.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:59.670+0000 7ff1391e4700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff13406b320 0x7ff1340b5630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:59.679 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:59.670+0000 7ff1391e4700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff13415dd00 0x7ff1340b00f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:59.679 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:59.670+0000 7ff1391e4700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff1341618f0 0x7ff1340b0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:59.679 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:59.670+0000 7ff1391e4700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff13415ab20 con 0x7ff13415dd00 2023-11-08T19:25:59.679 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:59.670+0000 7ff1391e4700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1340b0d00 con 0x7ff1341618f0 2023-11-08T19:25:59.679 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:59.670+0000 7ff1391e4700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1340b0fe0 con 0x7ff13406b320 2023-11-08T19:25:59.680 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:59.670+0000 7ff13259c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff13415dd00 0x7ff1340b00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:59.680 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:59.671+0000 7ff13259c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff13415dd00 0x7ff1340b00f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50760/0 (socket says 172.21.15.143:50760) 2023-11-08T19:25:59.680 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:59.671+0000 7ff13259c700 1 -- 172.21.15.143:0/3393430840 learned_addr learned my addr 172.21.15.143:0/3393430840 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:25:59.680 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:59.671+0000 7ff132d9d700 1 --2- 172.21.15.143:0/3393430840 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff13406b320 0x7ff1340b5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:59.680 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:59.671+0000 7ff13359e700 1 --2- 172.21.15.143:0/3393430840 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff1341618f0 0x7ff1340b0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:59.681 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:59.671+0000 7ff13259c700 1 -- 172.21.15.143:0/3393430840 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff13406b320 msgr2=0x7ff1340b5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:59.681 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:59.671+0000 7ff13259c700 1 --2- 172.21.15.143:0/3393430840 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff13406b320 0x7ff1340b5630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:59.681 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:59.671+0000 7ff13259c700 1 -- 172.21.15.143:0/3393430840 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff1341618f0 msgr2=0x7ff1340b0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:59.681 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:59.671+0000 7ff13259c700 1 --2- 172.21.15.143:0/3393430840 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff1341618f0 0x7ff1340b0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:59.681 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:59.671+0000 7ff13259c700 1 -- 172.21.15.143:0/3393430840 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff12c009b30 con 0x7ff13415dd00 2023-11-08T19:25:59.682 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:59.671+0000 7ff13259c700 1 --2- 172.21.15.143:0/3393430840 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff13415dd00 0x7ff1340b00f0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7ff12800c450 tx=0x7ff12800c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:59.682 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:59.671+0000 7ff11b7fe700 1 -- 172.21.15.143:0/3393430840 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1280052d0 con 0x7ff13415dd00 2023-11-08T19:25:59.682 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:59.671+0000 7ff11b7fe700 1 -- 172.21.15.143:0/3393430840 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff128005430 con 0x7ff13415dd00 2023-11-08T19:25:59.682 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:59.671+0000 7ff1391e4700 1 -- 172.21.15.143:0/3393430840 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1340b12c0 con 0x7ff13415dd00 2023-11-08T19:25:59.682 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:59.671+0000 7ff11b7fe700 1 -- 172.21.15.143:0/3393430840 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1280038c0 con 0x7ff13415dd00 2023-11-08T19:25:59.683 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:59.671+0000 7ff1391e4700 1 -- 172.21.15.143:0/3393430840 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1340b1880 con 0x7ff13415dd00 2023-11-08T19:25:59.683 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:59.672+0000 7ff1391e4700 1 -- 172.21.15.143:0/3393430840 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1340aff60 con 0x7ff13415dd00 2023-11-08T19:25:59.683 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:59.672+0000 7ff11b7fe700 1 -- 172.21.15.143:0/3393430840 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff128003a20 con 0x7ff13415dd00 2023-11-08T19:25:59.683 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:59.673+0000 7ff11b7fe700 1 --2- 172.21.15.143:0/3393430840 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7ff12006af00 0x7ff12006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:59.683 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:59.673+0000 7ff11b7fe700 1 -- 172.21.15.143:0/3393430840 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff128088930 con 0x7ff13415dd00 2023-11-08T19:25:59.684 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:59.673+0000 7ff132d9d700 1 --2- 172.21.15.143:0/3393430840 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7ff12006af00 0x7ff12006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:59.684 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:59.674+0000 7ff132d9d700 1 --2- 172.21.15.143:0/3393430840 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7ff12006af00 0x7ff12006d3c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff124006fd0 tx=0x7ff124008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:59.684 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:59.678+0000 7ff11b7fe700 1 -- 172.21.15.143:0/3393430840 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff1280539e0 con 0x7ff13415dd00 2023-11-08T19:25:59.862 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:59.861+0000 7ff1391e4700 1 -- 172.21.15.143:0/3393430840 --> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff134067c00 con 0x7ff12006af00 2023-11-08T19:25:59.871 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:59.869+0000 7ff11b7fe700 1 -- 172.21.15.143:0/3393430840 <== mgr.4099 v2:172.21.15.174:6832/4037280178 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289592 (secure 0 0 0) 0x7ff134067c00 con 0x7ff12006af00 2023-11-08T19:25:59.876 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:25:59.879 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:59.872+0000 7ff1391e4700 1 -- 172.21.15.143:0/3393430840 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7ff12006af00 msgr2=0x7ff12006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:59.879 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:59.872+0000 7ff1391e4700 1 --2- 172.21.15.143:0/3393430840 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7ff12006af00 0x7ff12006d3c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff124006fd0 tx=0x7ff124008040 comp rx=0 tx=0).stop 2023-11-08T19:25:59.879 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:59.872+0000 7ff1391e4700 1 -- 172.21.15.143:0/3393430840 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff13415dd00 msgr2=0x7ff1340b00f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:59.880 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:59.872+0000 7ff1391e4700 1 --2- 172.21.15.143:0/3393430840 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff13415dd00 0x7ff1340b00f0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7ff12800c450 tx=0x7ff12800c810 comp rx=0 tx=0).stop 2023-11-08T19:25:59.880 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:59.872+0000 7ff1391e4700 1 -- 172.21.15.143:0/3393430840 shutdown_connections 2023-11-08T19:25:59.880 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:59.872+0000 7ff1391e4700 1 --2- 172.21.15.143:0/3393430840 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7ff12006af00 0x7ff12006d3c0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:59.880 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:59.872+0000 7ff1391e4700 1 --2- 172.21.15.143:0/3393430840 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff13406b320 0x7ff1340b5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:59.881 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:59.872+0000 7ff1391e4700 1 --2- 172.21.15.143:0/3393430840 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff1341618f0 0x7ff1340b0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:59.881 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:59.872+0000 7ff1391e4700 1 --2- 172.21.15.143:0/3393430840 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff13415dd00 0x7ff1340b00f0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:59.881 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:59.872+0000 7ff1391e4700 1 -- 172.21.15.143:0/3393430840 >> 172.21.15.143:0/3393430840 conn(0x7ff134099ab0 msgr2=0x7ff13406f4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:59.881 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:59.873+0000 7ff1391e4700 1 -- 172.21.15.143:0/3393430840 shutdown_connections 2023-11-08T19:25:59.881 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:25:59.873+0000 7ff1391e4700 1 -- 172.21.15.143:0/3393430840 wait complete. 2023-11-08T19:25:59.882 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2023-11-08T19:25:59.886 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":63,"stamp":"2023-11-08T19:25:58.526344+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":152,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220336,"kb_used_data":4136,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754384,"statfs":{"total":773094113280,"available":772868489216,"internally_reserved":0,"allocated":4235264,"data_stored":2352484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002221"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.879024+0000","last_change":"2023-11-08T19:25:50.873993+0000","last_active":"2023-11-08T19:25:51.879024+0000","last_peered":"2023-11-08T19:25:51.879024+0000","last_clean":"2023-11-08T19:25:51.879024+0000","last_became_active":"2023-11-08T19:25:50.873677+0000","last_became_peered":"2023-11-08T19:25:50.873677+0000","last_unstale":"2023-11-08T19:25:51.879024+0000","last_undegraded":"2023-11-08T19:25:51.879024+0000","last_fullsized":"2023-11-08T19:25:51.879024+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:09:05.379033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204169+0000","last_change":"2023-11-08T19:25:50.890421+0000","last_active":"2023-11-08T19:25:52.204169+0000","last_peered":"2023-11-08T19:25:52.204169+0000","last_clean":"2023-11-08T19:25:52.204169+0000","last_became_active":"2023-11-08T19:25:50.890142+0000","last_became_peered":"2023-11-08T19:25:50.890142+0000","last_unstale":"2023-11-08T19:25:52.204169+0000","last_undegraded":"2023-11-08T19:25:52.204169+0000","last_fullsized":"2023-11-08T19:25:52.204169+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:01:43.267493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.205154+0000","last_change":"2023-11-08T19:25:50.875008+0000","last_active":"2023-11-08T19:25:52.205154+0000","last_peered":"2023-11-08T19:25:52.205154+0000","last_clean":"2023-11-08T19:25:52.205154+0000","last_became_active":"2023-11-08T19:25:50.874499+0000","last_became_peered":"2023-11-08T19:25:50.874499+0000","last_unstale":"2023-11-08T19:25:52.205154+0000","last_undegraded":"2023-11-08T19:25:52.205154+0000","last_fullsized":"2023-11-08T19:25:52.205154+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:27:11.209258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.991817+0000","last_change":"2023-11-08T19:25:50.883677+0000","last_active":"2023-11-08T19:25:51.991817+0000","last_peered":"2023-11-08T19:25:51.991817+0000","last_clean":"2023-11-08T19:25:51.991817+0000","last_became_active":"2023-11-08T19:25:50.883141+0000","last_became_peered":"2023-11-08T19:25:50.883141+0000","last_unstale":"2023-11-08T19:25:51.991817+0000","last_undegraded":"2023-11-08T19:25:51.991817+0000","last_fullsized":"2023-11-08T19:25:51.991817+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:56:01.687509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.876619+0000","last_change":"2023-11-08T19:25:50.877229+0000","last_active":"2023-11-08T19:25:51.876619+0000","last_peered":"2023-11-08T19:25:51.876619+0000","last_clean":"2023-11-08T19:25:51.876619+0000","last_became_active":"2023-11-08T19:25:50.876959+0000","last_became_peered":"2023-11-08T19:25:50.876959+0000","last_unstale":"2023-11-08T19:25:51.876619+0000","last_undegraded":"2023-11-08T19:25:51.876619+0000","last_fullsized":"2023-11-08T19:25:51.876619+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:27:09.954811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204135+0000","last_change":"2023-11-08T19:25:50.883530+0000","last_active":"2023-11-08T19:25:52.204135+0000","last_peered":"2023-11-08T19:25:52.204135+0000","last_clean":"2023-11-08T19:25:52.204135+0000","last_became_active":"2023-11-08T19:25:50.883198+0000","last_became_peered":"2023-11-08T19:25:50.883198+0000","last_unstale":"2023-11-08T19:25:52.204135+0000","last_undegraded":"2023-11-08T19:25:52.204135+0000","last_fullsized":"2023-11-08T19:25:52.204135+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:27:11.209258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.878653+0000","last_change":"2023-11-08T19:25:50.882248+0000","last_active":"2023-11-08T19:25:51.878653+0000","last_peered":"2023-11-08T19:25:51.878653+0000","last_clean":"2023-11-08T19:25:51.878653+0000","last_became_active":"2023-11-08T19:25:50.880074+0000","last_became_peered":"2023-11-08T19:25:50.880074+0000","last_unstale":"2023-11-08T19:25:51.878653+0000","last_undegraded":"2023-11-08T19:25:51.878653+0000","last_fullsized":"2023-11-08T19:25:51.878653+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:52:20.334760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.878362+0000","last_change":"2023-11-08T19:25:50.881827+0000","last_active":"2023-11-08T19:25:51.878362+0000","last_peered":"2023-11-08T19:25:51.878362+0000","last_clean":"2023-11-08T19:25:51.878362+0000","last_became_active":"2023-11-08T19:25:50.881556+0000","last_became_peered":"2023-11-08T19:25:50.881556+0000","last_unstale":"2023-11-08T19:25:51.878362+0000","last_undegraded":"2023-11-08T19:25:51.878362+0000","last_fullsized":"2023-11-08T19:25:51.878362+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:04:01.492989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.205585+0000","last_change":"2023-11-08T19:25:50.888816+0000","last_active":"2023-11-08T19:25:52.205585+0000","last_peered":"2023-11-08T19:25:52.205585+0000","last_clean":"2023-11-08T19:25:52.205585+0000","last_became_active":"2023-11-08T19:25:50.888536+0000","last_became_peered":"2023-11-08T19:25:50.888536+0000","last_unstale":"2023-11-08T19:25:52.205585+0000","last_undegraded":"2023-11-08T19:25:52.205585+0000","last_fullsized":"2023-11-08T19:25:52.205585+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:56:01.687509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.991994+0000","last_change":"2023-11-08T19:25:50.878973+0000","last_active":"2023-11-08T19:25:51.991994+0000","last_peered":"2023-11-08T19:25:51.991994+0000","last_clean":"2023-11-08T19:25:51.991994+0000","last_became_active":"2023-11-08T19:25:50.878706+0000","last_became_peered":"2023-11-08T19:25:50.878706+0000","last_unstale":"2023-11-08T19:25:51.991994+0000","last_undegraded":"2023-11-08T19:25:51.991994+0000","last_fullsized":"2023-11-08T19:25:51.991994+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:49:41.475597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.879592+0000","last_change":"2023-11-08T19:25:50.888879+0000","last_active":"2023-11-08T19:25:51.879592+0000","last_peered":"2023-11-08T19:25:51.879592+0000","last_clean":"2023-11-08T19:25:51.879592+0000","last_became_active":"2023-11-08T19:25:50.888643+0000","last_became_peered":"2023-11-08T19:25:50.888643+0000","last_unstale":"2023-11-08T19:25:51.879592+0000","last_undegraded":"2023-11-08T19:25:51.879592+0000","last_fullsized":"2023-11-08T19:25:51.879592+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:01:43.267493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204768+0000","last_change":"2023-11-08T19:25:50.877227+0000","last_active":"2023-11-08T19:25:52.204768+0000","last_peered":"2023-11-08T19:25:52.204768+0000","last_clean":"2023-11-08T19:25:52.204768+0000","last_became_active":"2023-11-08T19:25:50.876876+0000","last_became_peered":"2023-11-08T19:25:50.876876+0000","last_unstale":"2023-11-08T19:25:52.204768+0000","last_undegraded":"2023-11-08T19:25:52.204768+0000","last_fullsized":"2023-11-08T19:25:52.204768+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:05:33.227810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.876804+0000","last_change":"2023-11-08T19:25:50.890044+0000","last_active":"2023-11-08T19:25:51.876804+0000","last_peered":"2023-11-08T19:25:51.876804+0000","last_clean":"2023-11-08T19:25:51.876804+0000","last_became_active":"2023-11-08T19:25:50.889741+0000","last_became_peered":"2023-11-08T19:25:50.889741+0000","last_unstale":"2023-11-08T19:25:51.876804+0000","last_undegraded":"2023-11-08T19:25:51.876804+0000","last_fullsized":"2023-11-08T19:25:51.876804+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:54:55.202302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204255+0000","last_change":"2023-11-08T19:25:50.876264+0000","last_active":"2023-11-08T19:25:52.204255+0000","last_peered":"2023-11-08T19:25:52.204255+0000","last_clean":"2023-11-08T19:25:52.204255+0000","last_became_active":"2023-11-08T19:25:50.875971+0000","last_became_peered":"2023-11-08T19:25:50.875971+0000","last_unstale":"2023-11-08T19:25:52.204255+0000","last_undegraded":"2023-11-08T19:25:52.204255+0000","last_fullsized":"2023-11-08T19:25:52.204255+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:50:30.103716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204868+0000","last_change":"2023-11-08T19:25:50.877018+0000","last_active":"2023-11-08T19:25:52.204868+0000","last_peered":"2023-11-08T19:25:52.204868+0000","last_clean":"2023-11-08T19:25:52.204868+0000","last_became_active":"2023-11-08T19:25:50.876682+0000","last_became_peered":"2023-11-08T19:25:50.876682+0000","last_unstale":"2023-11-08T19:25:52.204868+0000","last_undegraded":"2023-11-08T19:25:52.204868+0000","last_fullsized":"2023-11-08T19:25:52.204868+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:52:20.334760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.878515+0000","last_change":"2023-11-08T19:25:50.881104+0000","last_active":"2023-11-08T19:25:51.878515+0000","last_peered":"2023-11-08T19:25:51.878515+0000","last_clean":"2023-11-08T19:25:51.878515+0000","last_became_active":"2023-11-08T19:25:50.880840+0000","last_became_peered":"2023-11-08T19:25:50.880840+0000","last_unstale":"2023-11-08T19:25:51.878515+0000","last_undegraded":"2023-11-08T19:25:51.878515+0000","last_fullsized":"2023-11-08T19:25:51.878515+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:49:41.475597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.879244+0000","last_change":"2023-11-08T19:25:50.880286+0000","last_active":"2023-11-08T19:25:51.879244+0000","last_peered":"2023-11-08T19:25:51.879244+0000","last_clean":"2023-11-08T19:25:51.879244+0000","last_became_active":"2023-11-08T19:25:50.879915+0000","last_became_peered":"2023-11-08T19:25:50.879915+0000","last_unstale":"2023-11-08T19:25:51.879244+0000","last_undegraded":"2023-11-08T19:25:51.879244+0000","last_fullsized":"2023-11-08T19:25:51.879244+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:38:37.341013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204251+0000","last_change":"2023-11-08T19:25:50.890954+0000","last_active":"2023-11-08T19:25:52.204251+0000","last_peered":"2023-11-08T19:25:52.204251+0000","last_clean":"2023-11-08T19:25:52.204251+0000","last_became_active":"2023-11-08T19:25:50.890695+0000","last_became_peered":"2023-11-08T19:25:50.890695+0000","last_unstale":"2023-11-08T19:25:52.204251+0000","last_undegraded":"2023-11-08T19:25:52.204251+0000","last_fullsized":"2023-11-08T19:25:52.204251+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:44:02.415941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204041+0000","last_change":"2023-11-08T19:25:50.892390+0000","last_active":"2023-11-08T19:25:52.204041+0000","last_peered":"2023-11-08T19:25:52.204041+0000","last_clean":"2023-11-08T19:25:52.204041+0000","last_became_active":"2023-11-08T19:25:50.892131+0000","last_became_peered":"2023-11-08T19:25:50.892131+0000","last_unstale":"2023-11-08T19:25:52.204041+0000","last_undegraded":"2023-11-08T19:25:52.204041+0000","last_fullsized":"2023-11-08T19:25:52.204041+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:25:07.133280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.876861+0000","last_change":"2023-11-08T19:25:50.874502+0000","last_active":"2023-11-08T19:25:51.876861+0000","last_peered":"2023-11-08T19:25:51.876861+0000","last_clean":"2023-11-08T19:25:51.876861+0000","last_became_active":"2023-11-08T19:25:50.874216+0000","last_became_peered":"2023-11-08T19:25:50.874216+0000","last_unstale":"2023-11-08T19:25:51.876861+0000","last_undegraded":"2023-11-08T19:25:51.876861+0000","last_fullsized":"2023-11-08T19:25:51.876861+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:39:23.438674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204719+0000","last_change":"2023-11-08T19:25:50.886322+0000","last_active":"2023-11-08T19:25:52.204719+0000","last_peered":"2023-11-08T19:25:52.204719+0000","last_clean":"2023-11-08T19:25:52.204719+0000","last_became_active":"2023-11-08T19:25:50.886011+0000","last_became_peered":"2023-11-08T19:25:50.886011+0000","last_unstale":"2023-11-08T19:25:52.204719+0000","last_undegraded":"2023-11-08T19:25:52.204719+0000","last_fullsized":"2023-11-08T19:25:52.204719+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:09:05.379033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204755+0000","last_change":"2023-11-08T19:25:50.881922+0000","last_active":"2023-11-08T19:25:52.204755+0000","last_peered":"2023-11-08T19:25:52.204755+0000","last_clean":"2023-11-08T19:25:52.204755+0000","last_became_active":"2023-11-08T19:25:50.880836+0000","last_became_peered":"2023-11-08T19:25:50.880836+0000","last_unstale":"2023-11-08T19:25:52.204755+0000","last_undegraded":"2023-11-08T19:25:52.204755+0000","last_fullsized":"2023-11-08T19:25:52.204755+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:49:45.059924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.989623+0000","last_change":"2023-11-08T19:25:50.886648+0000","last_active":"2023-11-08T19:25:51.989623+0000","last_peered":"2023-11-08T19:25:51.989623+0000","last_clean":"2023-11-08T19:25:51.989623+0000","last_became_active":"2023-11-08T19:25:50.886376+0000","last_became_peered":"2023-11-08T19:25:50.886376+0000","last_unstale":"2023-11-08T19:25:51.989623+0000","last_undegraded":"2023-11-08T19:25:51.989623+0000","last_fullsized":"2023-11-08T19:25:51.989623+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:49:45.059924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204354+0000","last_change":"2023-11-08T19:25:50.875989+0000","last_active":"2023-11-08T19:25:52.204354+0000","last_peered":"2023-11-08T19:25:52.204354+0000","last_clean":"2023-11-08T19:25:52.204354+0000","last_became_active":"2023-11-08T19:25:50.874428+0000","last_became_peered":"2023-11-08T19:25:50.874428+0000","last_unstale":"2023-11-08T19:25:52.204354+0000","last_undegraded":"2023-11-08T19:25:52.204354+0000","last_fullsized":"2023-11-08T19:25:52.204354+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:04:01.492989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.991856+0000","last_change":"2023-11-08T19:25:50.885263+0000","last_active":"2023-11-08T19:25:51.991856+0000","last_peered":"2023-11-08T19:25:51.991856+0000","last_clean":"2023-11-08T19:25:51.991856+0000","last_became_active":"2023-11-08T19:25:50.884856+0000","last_became_peered":"2023-11-08T19:25:50.884856+0000","last_unstale":"2023-11-08T19:25:51.991856+0000","last_undegraded":"2023-11-08T19:25:51.991856+0000","last_fullsized":"2023-11-08T19:25:51.991856+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:17:24.728184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204268+0000","last_change":"2023-11-08T19:25:50.884128+0000","last_active":"2023-11-08T19:25:52.204268+0000","last_peered":"2023-11-08T19:25:52.204268+0000","last_clean":"2023-11-08T19:25:52.204268+0000","last_became_active":"2023-11-08T19:25:50.883848+0000","last_became_peered":"2023-11-08T19:25:50.883848+0000","last_unstale":"2023-11-08T19:25:52.204268+0000","last_undegraded":"2023-11-08T19:25:52.204268+0000","last_fullsized":"2023-11-08T19:25:52.204268+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:32:01.237850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.992064+0000","last_change":"2023-11-08T19:25:50.885482+0000","last_active":"2023-11-08T19:25:51.992064+0000","last_peered":"2023-11-08T19:25:51.992064+0000","last_clean":"2023-11-08T19:25:51.992064+0000","last_became_active":"2023-11-08T19:25:50.884816+0000","last_became_peered":"2023-11-08T19:25:50.884816+0000","last_unstale":"2023-11-08T19:25:51.992064+0000","last_undegraded":"2023-11-08T19:25:51.992064+0000","last_fullsized":"2023-11-08T19:25:51.992064+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:05:33.227810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204941+0000","last_change":"2023-11-08T19:25:50.871479+0000","last_active":"2023-11-08T19:25:52.204941+0000","last_peered":"2023-11-08T19:25:52.204941+0000","last_clean":"2023-11-08T19:25:52.204941+0000","last_became_active":"2023-11-08T19:25:50.871211+0000","last_became_peered":"2023-11-08T19:25:50.871211+0000","last_unstale":"2023-11-08T19:25:52.204941+0000","last_undegraded":"2023-11-08T19:25:52.204941+0000","last_fullsized":"2023-11-08T19:25:52.204941+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:25:16.223226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204982+0000","last_change":"2023-11-08T19:25:50.881545+0000","last_active":"2023-11-08T19:25:52.204982+0000","last_peered":"2023-11-08T19:25:52.204982+0000","last_clean":"2023-11-08T19:25:52.204982+0000","last_became_active":"2023-11-08T19:25:50.881277+0000","last_became_peered":"2023-11-08T19:25:50.881277+0000","last_unstale":"2023-11-08T19:25:52.204982+0000","last_undegraded":"2023-11-08T19:25:52.204982+0000","last_fullsized":"2023-11-08T19:25:52.204982+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:18:57.865706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.879237+0000","last_change":"2023-11-08T19:25:50.879468+0000","last_active":"2023-11-08T19:25:51.879237+0000","last_peered":"2023-11-08T19:25:51.879237+0000","last_clean":"2023-11-08T19:25:51.879237+0000","last_became_active":"2023-11-08T19:25:50.879185+0000","last_became_peered":"2023-11-08T19:25:50.879185+0000","last_unstale":"2023-11-08T19:25:51.879237+0000","last_undegraded":"2023-11-08T19:25:51.879237+0000","last_fullsized":"2023-11-08T19:25:51.879237+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:25:16.223226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.878926+0000","last_change":"2023-11-08T19:25:50.873969+0000","last_active":"2023-11-08T19:25:51.878926+0000","last_peered":"2023-11-08T19:25:51.878926+0000","last_clean":"2023-11-08T19:25:51.878926+0000","last_became_active":"2023-11-08T19:25:50.873089+0000","last_became_peered":"2023-11-08T19:25:50.873089+0000","last_unstale":"2023-11-08T19:25:51.878926+0000","last_undegraded":"2023-11-08T19:25:51.878926+0000","last_fullsized":"2023-11-08T19:25:51.878926+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:37:10.264673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.878879+0000","last_change":"2023-11-08T19:25:50.886543+0000","last_active":"2023-11-08T19:25:51.878879+0000","last_peered":"2023-11-08T19:25:51.878879+0000","last_clean":"2023-11-08T19:25:51.878879+0000","last_became_active":"2023-11-08T19:25:50.886236+0000","last_became_peered":"2023-11-08T19:25:50.886236+0000","last_unstale":"2023-11-08T19:25:51.878879+0000","last_undegraded":"2023-11-08T19:25:51.878879+0000","last_fullsized":"2023-11-08T19:25:51.878879+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:50:52.902620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.989618+0000","last_change":"2023-11-08T19:25:50.889063+0000","last_active":"2023-11-08T19:25:51.989618+0000","last_peered":"2023-11-08T19:25:51.989618+0000","last_clean":"2023-11-08T19:25:51.989618+0000","last_became_active":"2023-11-08T19:25:50.888760+0000","last_became_peered":"2023-11-08T19:25:50.888760+0000","last_unstale":"2023-11-08T19:25:51.989618+0000","last_undegraded":"2023-11-08T19:25:51.989618+0000","last_fullsized":"2023-11-08T19:25:51.989618+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:37:10.264673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"32'4","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-08T19:25:50.891804+0000","last_change":"2023-11-08T19:25:49.880831+0000","last_active":"2023-11-08T19:25:50.891804+0000","last_peered":"2023-11-08T19:25:50.891804+0000","last_clean":"2023-11-08T19:25:50.891804+0000","last_became_active":"2023-11-08T19:25:49.880560+0000","last_became_peered":"2023-11-08T19:25:49.880560+0000","last_unstale":"2023-11-08T19:25:50.891804+0000","last_undegraded":"2023-11-08T19:25:50.891804+0000","last_fullsized":"2023-11-08T19:25:50.891804+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:12:39.742779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.879175+0000","last_change":"2023-11-08T19:25:50.880315+0000","last_active":"2023-11-08T19:25:51.879175+0000","last_peered":"2023-11-08T19:25:51.879175+0000","last_clean":"2023-11-08T19:25:51.879175+0000","last_became_active":"2023-11-08T19:25:50.880025+0000","last_became_peered":"2023-11-08T19:25:50.880025+0000","last_unstale":"2023-11-08T19:25:51.879175+0000","last_undegraded":"2023-11-08T19:25:51.879175+0000","last_fullsized":"2023-11-08T19:25:51.879175+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:49:45.059924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"32'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.989742+0000","last_change":"2023-11-08T19:25:49.879837+0000","last_active":"2023-11-08T19:25:51.989742+0000","last_peered":"2023-11-08T19:25:51.989742+0000","last_clean":"2023-11-08T19:25:51.989742+0000","last_became_active":"2023-11-08T19:25:49.879570+0000","last_became_peered":"2023-11-08T19:25:49.879570+0000","last_unstale":"2023-11-08T19:25:51.989742+0000","last_undegraded":"2023-11-08T19:25:51.989742+0000","last_fullsized":"2023-11-08T19:25:51.989742+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:00:52.169251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.205335+0000","last_change":"2023-11-08T19:25:50.881012+0000","last_active":"2023-11-08T19:25:52.205335+0000","last_peered":"2023-11-08T19:25:52.205335+0000","last_clean":"2023-11-08T19:25:52.205335+0000","last_became_active":"2023-11-08T19:25:50.880752+0000","last_became_peered":"2023-11-08T19:25:50.880752+0000","last_unstale":"2023-11-08T19:25:52.205335+0000","last_undegraded":"2023-11-08T19:25:52.205335+0000","last_fullsized":"2023-11-08T19:25:52.205335+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:33:12.567239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"32'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.203995+0000","last_change":"2023-11-08T19:25:49.886087+0000","last_active":"2023-11-08T19:25:52.203995+0000","last_peered":"2023-11-08T19:25:52.203995+0000","last_clean":"2023-11-08T19:25:52.203995+0000","last_became_active":"2023-11-08T19:25:49.885801+0000","last_became_peered":"2023-11-08T19:25:49.885801+0000","last_unstale":"2023-11-08T19:25:52.203995+0000","last_undegraded":"2023-11-08T19:25:52.203995+0000","last_fullsized":"2023-11-08T19:25:52.203995+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:04:00.482724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.992009+0000","last_change":"2023-11-08T19:25:50.885648+0000","last_active":"2023-11-08T19:25:51.992009+0000","last_peered":"2023-11-08T19:25:51.992009+0000","last_clean":"2023-11-08T19:25:51.992009+0000","last_became_active":"2023-11-08T19:25:50.885179+0000","last_became_peered":"2023-11-08T19:25:50.885179+0000","last_unstale":"2023-11-08T19:25:51.992009+0000","last_undegraded":"2023-11-08T19:25:51.992009+0000","last_fullsized":"2023-11-08T19:25:51.992009+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:52:20.334760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"32'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.876725+0000","last_change":"2023-11-08T19:25:49.883073+0000","last_active":"2023-11-08T19:25:51.876725+0000","last_peered":"2023-11-08T19:25:51.876725+0000","last_clean":"2023-11-08T19:25:51.876725+0000","last_became_active":"2023-11-08T19:25:49.882522+0000","last_became_peered":"2023-11-08T19:25:49.882522+0000","last_unstale":"2023-11-08T19:25:51.876725+0000","last_undegraded":"2023-11-08T19:25:51.876725+0000","last_fullsized":"2023-11-08T19:25:51.876725+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:27:43.830493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.879054+0000","last_change":"2023-11-08T19:25:50.888132+0000","last_active":"2023-11-08T19:25:51.879054+0000","last_peered":"2023-11-08T19:25:51.879054+0000","last_clean":"2023-11-08T19:25:51.879054+0000","last_became_active":"2023-11-08T19:25:50.886029+0000","last_became_peered":"2023-11-08T19:25:50.886029+0000","last_unstale":"2023-11-08T19:25:51.879054+0000","last_undegraded":"2023-11-08T19:25:51.879054+0000","last_fullsized":"2023-11-08T19:25:51.879054+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:25:07.133280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"32'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.205020+0000","last_change":"2023-11-08T19:25:49.884628+0000","last_active":"2023-11-08T19:25:52.205020+0000","last_peered":"2023-11-08T19:25:52.205020+0000","last_clean":"2023-11-08T19:25:52.205020+0000","last_became_active":"2023-11-08T19:25:49.884384+0000","last_became_peered":"2023-11-08T19:25:49.884384+0000","last_unstale":"2023-11-08T19:25:52.205020+0000","last_undegraded":"2023-11-08T19:25:52.205020+0000","last_fullsized":"2023-11-08T19:25:52.205020+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:17:52.641253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204834+0000","last_change":"2023-11-08T19:25:50.874965+0000","last_active":"2023-11-08T19:25:52.204834+0000","last_peered":"2023-11-08T19:25:52.204834+0000","last_clean":"2023-11-08T19:25:52.204834+0000","last_became_active":"2023-11-08T19:25:50.874696+0000","last_became_peered":"2023-11-08T19:25:50.874696+0000","last_unstale":"2023-11-08T19:25:52.204834+0000","last_undegraded":"2023-11-08T19:25:52.204834+0000","last_fullsized":"2023-11-08T19:25:52.204834+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:49:41.475597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"32'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204932+0000","last_change":"2023-11-08T19:25:49.884607+0000","last_active":"2023-11-08T19:25:52.204932+0000","last_peered":"2023-11-08T19:25:52.204932+0000","last_clean":"2023-11-08T19:25:52.204932+0000","last_became_active":"2023-11-08T19:25:49.884392+0000","last_became_peered":"2023-11-08T19:25:49.884392+0000","last_unstale":"2023-11-08T19:25:52.204932+0000","last_undegraded":"2023-11-08T19:25:52.204932+0000","last_fullsized":"2023-11-08T19:25:52.204932+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:54:54.192037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204370+0000","last_change":"2023-11-08T19:25:50.881747+0000","last_active":"2023-11-08T19:25:52.204370+0000","last_peered":"2023-11-08T19:25:52.204370+0000","last_clean":"2023-11-08T19:25:52.204370+0000","last_became_active":"2023-11-08T19:25:50.881225+0000","last_became_peered":"2023-11-08T19:25:50.881225+0000","last_unstale":"2023-11-08T19:25:52.204370+0000","last_undegraded":"2023-11-08T19:25:52.204370+0000","last_fullsized":"2023-11-08T19:25:52.204370+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:37:10.264673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.203749+0000","last_change":"2023-11-08T19:25:49.882231+0000","last_active":"2023-11-08T19:25:52.203749+0000","last_peered":"2023-11-08T19:25:52.203749+0000","last_clean":"2023-11-08T19:25:52.203749+0000","last_became_active":"2023-11-08T19:25:49.881992+0000","last_became_peered":"2023-11-08T19:25:49.881992+0000","last_unstale":"2023-11-08T19:25:52.203749+0000","last_undegraded":"2023-11-08T19:25:52.203749+0000","last_fullsized":"2023-11-08T19:25:52.203749+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:00:52.169251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.991649+0000","last_change":"2023-11-08T19:25:50.883482+0000","last_active":"2023-11-08T19:25:51.991649+0000","last_peered":"2023-11-08T19:25:51.991649+0000","last_clean":"2023-11-08T19:25:51.991649+0000","last_became_active":"2023-11-08T19:25:50.883187+0000","last_became_peered":"2023-11-08T19:25:50.883187+0000","last_unstale":"2023-11-08T19:25:51.991649+0000","last_undegraded":"2023-11-08T19:25:51.991649+0000","last_fullsized":"2023-11-08T19:25:51.991649+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:33:12.567239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204890+0000","last_change":"2023-11-08T19:25:49.878388+0000","last_active":"2023-11-08T19:25:52.204890+0000","last_peered":"2023-11-08T19:25:52.204890+0000","last_clean":"2023-11-08T19:25:52.204890+0000","last_became_active":"2023-11-08T19:25:49.878111+0000","last_became_peered":"2023-11-08T19:25:49.878111+0000","last_unstale":"2023-11-08T19:25:52.204890+0000","last_undegraded":"2023-11-08T19:25:52.204890+0000","last_fullsized":"2023-11-08T19:25:52.204890+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:39:22.428409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204894+0000","last_change":"2023-11-08T19:25:50.877613+0000","last_active":"2023-11-08T19:25:52.204894+0000","last_peered":"2023-11-08T19:25:52.204894+0000","last_clean":"2023-11-08T19:25:52.204894+0000","last_became_active":"2023-11-08T19:25:50.877311+0000","last_became_peered":"2023-11-08T19:25:50.877311+0000","last_unstale":"2023-11-08T19:25:52.204894+0000","last_undegraded":"2023-11-08T19:25:52.204894+0000","last_fullsized":"2023-11-08T19:25:52.204894+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:50:52.902620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204190+0000","last_change":"2023-11-08T19:25:49.884355+0000","last_active":"2023-11-08T19:25:52.204190+0000","last_peered":"2023-11-08T19:25:52.204190+0000","last_clean":"2023-11-08T19:25:52.204190+0000","last_became_active":"2023-11-08T19:25:49.884065+0000","last_became_peered":"2023-11-08T19:25:49.884065+0000","last_unstale":"2023-11-08T19:25:52.204190+0000","last_undegraded":"2023-11-08T19:25:52.204190+0000","last_fullsized":"2023-11-08T19:25:52.204190+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:05:32.217545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204103+0000","last_change":"2023-11-08T19:25:50.888556+0000","last_active":"2023-11-08T19:25:52.204103+0000","last_peered":"2023-11-08T19:25:52.204103+0000","last_clean":"2023-11-08T19:25:52.204103+0000","last_became_active":"2023-11-08T19:25:50.887909+0000","last_became_peered":"2023-11-08T19:25:50.887909+0000","last_unstale":"2023-11-08T19:25:52.204103+0000","last_undegraded":"2023-11-08T19:25:52.204103+0000","last_fullsized":"2023-11-08T19:25:52.204103+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:38:37.341013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.989663+0000","last_change":"2023-11-08T19:25:49.884919+0000","last_active":"2023-11-08T19:25:51.989663+0000","last_peered":"2023-11-08T19:25:51.989663+0000","last_clean":"2023-11-08T19:25:51.989663+0000","last_became_active":"2023-11-08T19:25:49.884679+0000","last_became_peered":"2023-11-08T19:25:49.884679+0000","last_unstale":"2023-11-08T19:25:51.989663+0000","last_undegraded":"2023-11-08T19:25:51.989663+0000","last_fullsized":"2023-11-08T19:25:51.989663+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:39:22.428409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204236+0000","last_change":"2023-11-08T19:25:50.882028+0000","last_active":"2023-11-08T19:25:52.204236+0000","last_peered":"2023-11-08T19:25:52.204236+0000","last_clean":"2023-11-08T19:25:52.204236+0000","last_became_active":"2023-11-08T19:25:50.881746+0000","last_became_peered":"2023-11-08T19:25:50.881746+0000","last_unstale":"2023-11-08T19:25:52.204236+0000","last_undegraded":"2023-11-08T19:25:52.204236+0000","last_fullsized":"2023-11-08T19:25:52.204236+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:39:57.441098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"32'3","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204967+0000","last_change":"2023-11-08T19:25:49.878409+0000","last_active":"2023-11-08T19:25:52.204967+0000","last_peered":"2023-11-08T19:25:52.204967+0000","last_clean":"2023-11-08T19:25:52.204967+0000","last_became_active":"2023-11-08T19:25:49.878105+0000","last_became_peered":"2023-11-08T19:25:49.878105+0000","last_unstale":"2023-11-08T19:25:52.204967+0000","last_undegraded":"2023-11-08T19:25:52.204967+0000","last_fullsized":"2023-11-08T19:25:52.204967+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:55:30.168605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204866+0000","last_change":"2023-11-08T19:25:50.887174+0000","last_active":"2023-11-08T19:25:52.204866+0000","last_peered":"2023-11-08T19:25:52.204866+0000","last_clean":"2023-11-08T19:25:52.204866+0000","last_became_active":"2023-11-08T19:25:50.886902+0000","last_became_peered":"2023-11-08T19:25:50.886902+0000","last_unstale":"2023-11-08T19:25:52.204866+0000","last_undegraded":"2023-11-08T19:25:52.204866+0000","last_fullsized":"2023-11-08T19:25:52.204866+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:19:42.562982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"32'6","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204091+0000","last_change":"2023-11-08T19:25:49.886499+0000","last_active":"2023-11-08T19:25:52.204091+0000","last_peered":"2023-11-08T19:25:52.204091+0000","last_clean":"2023-11-08T19:25:52.204091+0000","last_became_active":"2023-11-08T19:25:49.886169+0000","last_became_peered":"2023-11-08T19:25:49.886169+0000","last_unstale":"2023-11-08T19:25:52.204091+0000","last_undegraded":"2023-11-08T19:25:52.204091+0000","last_fullsized":"2023-11-08T19:25:52.204091+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:19:41.552717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.991670+0000","last_change":"2023-11-08T19:25:50.885893+0000","last_active":"2023-11-08T19:25:51.991670+0000","last_peered":"2023-11-08T19:25:51.991670+0000","last_clean":"2023-11-08T19:25:51.991670+0000","last_became_active":"2023-11-08T19:25:50.880205+0000","last_became_peered":"2023-11-08T19:25:50.880205+0000","last_unstale":"2023-11-08T19:25:51.991670+0000","last_undegraded":"2023-11-08T19:25:51.991670+0000","last_fullsized":"2023-11-08T19:25:51.991670+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:55:31.178870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"32'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.205247+0000","last_change":"2023-11-08T19:25:49.879256+0000","last_active":"2023-11-08T19:25:52.205247+0000","last_peered":"2023-11-08T19:25:52.205247+0000","last_clean":"2023-11-08T19:25:52.205247+0000","last_became_active":"2023-11-08T19:25:49.878963+0000","last_became_peered":"2023-11-08T19:25:49.878963+0000","last_unstale":"2023-11-08T19:25:52.205247+0000","last_undegraded":"2023-11-08T19:25:52.205247+0000","last_fullsized":"2023-11-08T19:25:52.205247+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:12:39.742779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.203953+0000","last_change":"2023-11-08T19:25:50.891183+0000","last_active":"2023-11-08T19:25:52.203953+0000","last_peered":"2023-11-08T19:25:52.203953+0000","last_clean":"2023-11-08T19:25:52.203953+0000","last_became_active":"2023-11-08T19:25:50.890889+0000","last_became_peered":"2023-11-08T19:25:50.890889+0000","last_unstale":"2023-11-08T19:25:52.203953+0000","last_undegraded":"2023-11-08T19:25:52.203953+0000","last_fullsized":"2023-11-08T19:25:52.203953+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:25:16.223226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.991756+0000","last_change":"2023-11-08T19:25:49.880102+0000","last_active":"2023-11-08T19:25:51.991756+0000","last_peered":"2023-11-08T19:25:51.991756+0000","last_clean":"2023-11-08T19:25:51.991756+0000","last_became_active":"2023-11-08T19:25:49.879816+0000","last_became_peered":"2023-11-08T19:25:49.879816+0000","last_unstale":"2023-11-08T19:25:51.991756+0000","last_undegraded":"2023-11-08T19:25:51.991756+0000","last_fullsized":"2023-11-08T19:25:51.991756+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:37:09.254408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.878734+0000","last_change":"2023-11-08T19:25:50.886385+0000","last_active":"2023-11-08T19:25:51.878734+0000","last_peered":"2023-11-08T19:25:51.878734+0000","last_clean":"2023-11-08T19:25:51.878734+0000","last_became_active":"2023-11-08T19:25:50.886070+0000","last_became_peered":"2023-11-08T19:25:50.886070+0000","last_unstale":"2023-11-08T19:25:51.878734+0000","last_undegraded":"2023-11-08T19:25:51.878734+0000","last_fullsized":"2023-11-08T19:25:51.878734+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:05:33.227810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.203841+0000","last_change":"2023-11-08T19:25:49.884359+0000","last_active":"2023-11-08T19:25:52.203841+0000","last_peered":"2023-11-08T19:25:52.203841+0000","last_clean":"2023-11-08T19:25:52.203841+0000","last_became_active":"2023-11-08T19:25:49.884119+0000","last_became_peered":"2023-11-08T19:25:49.884119+0000","last_unstale":"2023-11-08T19:25:52.203841+0000","last_undegraded":"2023-11-08T19:25:52.203841+0000","last_fullsized":"2023-11-08T19:25:52.203841+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:55:30.168605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.878777+0000","last_change":"2023-11-08T19:25:50.879603+0000","last_active":"2023-11-08T19:25:51.878777+0000","last_peered":"2023-11-08T19:25:51.878777+0000","last_clean":"2023-11-08T19:25:51.878777+0000","last_became_active":"2023-11-08T19:25:50.879267+0000","last_became_peered":"2023-11-08T19:25:50.879267+0000","last_unstale":"2023-11-08T19:25:51.878777+0000","last_undegraded":"2023-11-08T19:25:51.878777+0000","last_fullsized":"2023-11-08T19:25:51.878777+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:51:27.817728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"32'8","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.203707+0000","last_change":"2023-11-08T19:25:49.886483+0000","last_active":"2023-11-08T19:25:52.203707+0000","last_peered":"2023-11-08T19:25:52.203707+0000","last_clean":"2023-11-08T19:25:52.203707+0000","last_became_active":"2023-11-08T19:25:49.886209+0000","last_became_peered":"2023-11-08T19:25:49.886209+0000","last_unstale":"2023-11-08T19:25:52.203707+0000","last_undegraded":"2023-11-08T19:25:52.203707+0000","last_fullsized":"2023-11-08T19:25:52.203707+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:49:40.465332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.205492+0000","last_change":"2023-11-08T19:25:50.884739+0000","last_active":"2023-11-08T19:25:52.205492+0000","last_peered":"2023-11-08T19:25:52.205492+0000","last_clean":"2023-11-08T19:25:52.205492+0000","last_became_active":"2023-11-08T19:25:50.884476+0000","last_became_peered":"2023-11-08T19:25:50.884476+0000","last_unstale":"2023-11-08T19:25:52.205492+0000","last_undegraded":"2023-11-08T19:25:52.205492+0000","last_fullsized":"2023-11-08T19:25:52.205492+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:55:31.178870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"32'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.992046+0000","last_change":"2023-11-08T19:25:49.884770+0000","last_active":"2023-11-08T19:25:51.992046+0000","last_peered":"2023-11-08T19:25:51.992046+0000","last_clean":"2023-11-08T19:25:51.992046+0000","last_became_active":"2023-11-08T19:25:49.884476+0000","last_became_peered":"2023-11-08T19:25:49.884476+0000","last_unstale":"2023-11-08T19:25:51.992046+0000","last_undegraded":"2023-11-08T19:25:51.992046+0000","last_fullsized":"2023-11-08T19:25:51.992046+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:18:56.855441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204945+0000","last_change":"2023-11-08T19:25:50.877298+0000","last_active":"2023-11-08T19:25:52.204945+0000","last_peered":"2023-11-08T19:25:52.204945+0000","last_clean":"2023-11-08T19:25:52.204945+0000","last_became_active":"2023-11-08T19:25:50.877018+0000","last_became_peered":"2023-11-08T19:25:50.877018+0000","last_unstale":"2023-11-08T19:25:52.204945+0000","last_undegraded":"2023-11-08T19:25:52.204945+0000","last_fullsized":"2023-11-08T19:25:52.204945+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:25:07.133280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"32'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.205477+0000","last_change":"2023-11-08T19:25:49.878520+0000","last_active":"2023-11-08T19:25:52.205477+0000","last_peered":"2023-11-08T19:25:52.205477+0000","last_clean":"2023-11-08T19:25:52.205477+0000","last_became_active":"2023-11-08T19:25:49.878244+0000","last_became_peered":"2023-11-08T19:25:49.878244+0000","last_unstale":"2023-11-08T19:25:52.205477+0000","last_undegraded":"2023-11-08T19:25:52.205477+0000","last_fullsized":"2023-11-08T19:25:52.205477+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:23:30.383150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.991899+0000","last_change":"2023-11-08T19:25:50.878056+0000","last_active":"2023-11-08T19:25:51.991899+0000","last_peered":"2023-11-08T19:25:51.991899+0000","last_clean":"2023-11-08T19:25:51.991899+0000","last_became_active":"2023-11-08T19:25:50.877738+0000","last_became_peered":"2023-11-08T19:25:50.877738+0000","last_unstale":"2023-11-08T19:25:51.991899+0000","last_undegraded":"2023-11-08T19:25:51.991899+0000","last_fullsized":"2023-11-08T19:25:51.991899+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:04:01.492989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"32'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.205456+0000","last_change":"2023-11-08T19:25:49.881744+0000","last_active":"2023-11-08T19:25:52.205456+0000","last_peered":"2023-11-08T19:25:52.205456+0000","last_clean":"2023-11-08T19:25:52.205456+0000","last_became_active":"2023-11-08T19:25:49.881436+0000","last_became_peered":"2023-11-08T19:25:49.881436+0000","last_unstale":"2023-11-08T19:25:52.205456+0000","last_undegraded":"2023-11-08T19:25:52.205456+0000","last_fullsized":"2023-11-08T19:25:52.205456+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:08:07.992929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.876822+0000","last_change":"2023-11-08T19:25:50.881348+0000","last_active":"2023-11-08T19:25:51.876822+0000","last_peered":"2023-11-08T19:25:51.876822+0000","last_clean":"2023-11-08T19:25:51.876822+0000","last_became_active":"2023-11-08T19:25:50.881064+0000","last_became_peered":"2023-11-08T19:25:50.881064+0000","last_unstale":"2023-11-08T19:25:51.876822+0000","last_undegraded":"2023-11-08T19:25:51.876822+0000","last_fullsized":"2023-11-08T19:25:51.876822+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:23:31.393415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.205574+0000","last_change":"2023-11-08T19:25:49.884654+0000","last_active":"2023-11-08T19:25:52.205574+0000","last_peered":"2023-11-08T19:25:52.205574+0000","last_clean":"2023-11-08T19:25:52.205574+0000","last_became_active":"2023-11-08T19:25:49.884322+0000","last_became_peered":"2023-11-08T19:25:49.884322+0000","last_unstale":"2023-11-08T19:25:52.205574+0000","last_undegraded":"2023-11-08T19:25:52.205574+0000","last_fullsized":"2023-11-08T19:25:52.205574+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:49:07.738959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.876689+0000","last_change":"2023-11-08T19:25:50.878414+0000","last_active":"2023-11-08T19:25:51.876689+0000","last_peered":"2023-11-08T19:25:51.876689+0000","last_clean":"2023-11-08T19:25:51.876689+0000","last_became_active":"2023-11-08T19:25:50.878145+0000","last_became_peered":"2023-11-08T19:25:50.878145+0000","last_unstale":"2023-11-08T19:25:51.876689+0000","last_undegraded":"2023-11-08T19:25:51.876689+0000","last_fullsized":"2023-11-08T19:25:51.876689+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:49:08.749224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"32'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.989615+0000","last_change":"2023-11-08T19:25:49.878923+0000","last_active":"2023-11-08T19:25:51.989615+0000","last_peered":"2023-11-08T19:25:51.989615+0000","last_clean":"2023-11-08T19:25:51.989615+0000","last_became_active":"2023-11-08T19:25:49.878167+0000","last_became_peered":"2023-11-08T19:25:49.878167+0000","last_unstale":"2023-11-08T19:25:51.989615+0000","last_undegraded":"2023-11-08T19:25:51.989615+0000","last_fullsized":"2023-11-08T19:25:51.989615+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:51:26.807463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.876711+0000","last_change":"2023-11-08T19:25:50.887699+0000","last_active":"2023-11-08T19:25:51.876711+0000","last_peered":"2023-11-08T19:25:51.876711+0000","last_clean":"2023-11-08T19:25:51.876711+0000","last_became_active":"2023-11-08T19:25:50.887423+0000","last_became_peered":"2023-11-08T19:25:50.887423+0000","last_unstale":"2023-11-08T19:25:51.876711+0000","last_undegraded":"2023-11-08T19:25:51.876711+0000","last_fullsized":"2023-11-08T19:25:51.876711+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:08:09.003194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-08T19:25:50.869233+0000","last_change":"2023-11-08T19:25:49.875215+0000","last_active":"2023-11-08T19:25:50.869233+0000","last_peered":"2023-11-08T19:25:50.869233+0000","last_clean":"2023-11-08T19:25:50.869233+0000","last_became_active":"2023-11-08T19:25:49.874913+0000","last_became_peered":"2023-11-08T19:25:49.874913+0000","last_unstale":"2023-11-08T19:25:50.869233+0000","last_undegraded":"2023-11-08T19:25:50.869233+0000","last_fullsized":"2023-11-08T19:25:50.869233+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:12:47.116061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.876619+0000","last_change":"2023-11-08T19:25:50.888597+0000","last_active":"2023-11-08T19:25:51.876619+0000","last_peered":"2023-11-08T19:25:51.876619+0000","last_clean":"2023-11-08T19:25:51.876619+0000","last_became_active":"2023-11-08T19:25:50.887065+0000","last_became_peered":"2023-11-08T19:25:50.887065+0000","last_unstale":"2023-11-08T19:25:51.876619+0000","last_undegraded":"2023-11-08T19:25:51.876619+0000","last_fullsized":"2023-11-08T19:25:51.876619+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:02:25.988939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.205088+0000","last_change":"2023-11-08T19:25:49.884053+0000","last_active":"2023-11-08T19:25:52.205088+0000","last_peered":"2023-11-08T19:25:52.205088+0000","last_clean":"2023-11-08T19:25:52.205088+0000","last_became_active":"2023-11-08T19:25:49.883822+0000","last_became_peered":"2023-11-08T19:25:49.883822+0000","last_unstale":"2023-11-08T19:25:52.205088+0000","last_undegraded":"2023-11-08T19:25:52.205088+0000","last_fullsized":"2023-11-08T19:25:52.205088+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:12:47.116061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.878803+0000","last_change":"2023-11-08T19:25:50.882444+0000","last_active":"2023-11-08T19:25:51.878803+0000","last_peered":"2023-11-08T19:25:51.878803+0000","last_clean":"2023-11-08T19:25:51.878803+0000","last_became_active":"2023-11-08T19:25:50.881119+0000","last_became_peered":"2023-11-08T19:25:50.881119+0000","last_unstale":"2023-11-08T19:25:51.878803+0000","last_undegraded":"2023-11-08T19:25:51.878803+0000","last_fullsized":"2023-11-08T19:25:51.878803+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:19:42.562982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204303+0000","last_change":"2023-11-08T19:25:49.879327+0000","last_active":"2023-11-08T19:25:52.204303+0000","last_peered":"2023-11-08T19:25:52.204303+0000","last_clean":"2023-11-08T19:25:52.204303+0000","last_became_active":"2023-11-08T19:25:49.879054+0000","last_became_peered":"2023-11-08T19:25:49.879054+0000","last_unstale":"2023-11-08T19:25:52.204303+0000","last_undegraded":"2023-11-08T19:25:52.204303+0000","last_fullsized":"2023-11-08T19:25:52.204303+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:09:04.368768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.876752+0000","last_change":"2023-11-08T19:25:50.874836+0000","last_active":"2023-11-08T19:25:51.876752+0000","last_peered":"2023-11-08T19:25:51.876752+0000","last_clean":"2023-11-08T19:25:51.876752+0000","last_became_active":"2023-11-08T19:25:50.874272+0000","last_became_peered":"2023-11-08T19:25:50.874272+0000","last_unstale":"2023-11-08T19:25:51.876752+0000","last_undegraded":"2023-11-08T19:25:51.876752+0000","last_fullsized":"2023-11-08T19:25:51.876752+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:17:53.651518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"32'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.991956+0000","last_change":"2023-11-08T19:25:49.880740+0000","last_active":"2023-11-08T19:25:51.991956+0000","last_peered":"2023-11-08T19:25:51.991956+0000","last_clean":"2023-11-08T19:25:51.991956+0000","last_became_active":"2023-11-08T19:25:49.880458+0000","last_became_peered":"2023-11-08T19:25:49.880458+0000","last_unstale":"2023-11-08T19:25:51.991956+0000","last_undegraded":"2023-11-08T19:25:51.991956+0000","last_fullsized":"2023-11-08T19:25:51.991956+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:20:42.164809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.205146+0000","last_change":"2023-11-08T19:25:50.883014+0000","last_active":"2023-11-08T19:25:52.205146+0000","last_peered":"2023-11-08T19:25:52.205146+0000","last_clean":"2023-11-08T19:25:52.205146+0000","last_became_active":"2023-11-08T19:25:50.882737+0000","last_became_peered":"2023-11-08T19:25:50.882737+0000","last_unstale":"2023-11-08T19:25:52.205146+0000","last_undegraded":"2023-11-08T19:25:52.205146+0000","last_fullsized":"2023-11-08T19:25:52.205146+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:38:37.341013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"33'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.991098+0000","last_change":"2023-11-08T19:25:49.880299+0000","last_active":"2023-11-08T19:25:51.991098+0000","last_peered":"2023-11-08T19:25:51.991098+0000","last_clean":"2023-11-08T19:25:51.991098+0000","last_became_active":"2023-11-08T19:25:49.878038+0000","last_became_peered":"2023-11-08T19:25:49.878038+0000","last_unstale":"2023-11-08T19:25:51.991098+0000","last_undegraded":"2023-11-08T19:25:51.991098+0000","last_fullsized":"2023-11-08T19:25:51.991098+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:12:39.742779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.989626+0000","last_change":"2023-11-08T19:25:50.888755+0000","last_active":"2023-11-08T19:25:51.989626+0000","last_peered":"2023-11-08T19:25:51.989626+0000","last_clean":"2023-11-08T19:25:51.989626+0000","last_became_active":"2023-11-08T19:25:50.888494+0000","last_became_peered":"2023-11-08T19:25:50.888494+0000","last_unstale":"2023-11-08T19:25:51.989626+0000","last_undegraded":"2023-11-08T19:25:51.989626+0000","last_fullsized":"2023-11-08T19:25:51.989626+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:09:05.379033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.991905+0000","last_change":"2023-11-08T19:25:49.881699+0000","last_active":"2023-11-08T19:25:51.991905+0000","last_peered":"2023-11-08T19:25:51.991905+0000","last_clean":"2023-11-08T19:25:51.991905+0000","last_became_active":"2023-11-08T19:25:49.881062+0000","last_became_peered":"2023-11-08T19:25:49.881062+0000","last_unstale":"2023-11-08T19:25:51.991905+0000","last_undegraded":"2023-11-08T19:25:51.991905+0000","last_fullsized":"2023-11-08T19:25:51.991905+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:49:44.049659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.878830+0000","last_change":"2023-11-08T19:25:50.882630+0000","last_active":"2023-11-08T19:25:51.878830+0000","last_peered":"2023-11-08T19:25:51.878830+0000","last_clean":"2023-11-08T19:25:51.878830+0000","last_became_active":"2023-11-08T19:25:50.882359+0000","last_became_peered":"2023-11-08T19:25:50.882359+0000","last_unstale":"2023-11-08T19:25:51.878830+0000","last_undegraded":"2023-11-08T19:25:51.878830+0000","last_fullsized":"2023-11-08T19:25:51.878830+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:00:53.179516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.989575+0000","last_change":"2023-11-08T19:25:49.879113+0000","last_active":"2023-11-08T19:25:51.989575+0000","last_peered":"2023-11-08T19:25:51.989575+0000","last_clean":"2023-11-08T19:25:51.989575+0000","last_became_active":"2023-11-08T19:25:49.878579+0000","last_became_peered":"2023-11-08T19:25:49.878579+0000","last_unstale":"2023-11-08T19:25:51.989575+0000","last_undegraded":"2023-11-08T19:25:51.989575+0000","last_fullsized":"2023-11-08T19:25:51.989575+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:12:47.116061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.989707+0000","last_change":"2023-11-08T19:25:50.880600+0000","last_active":"2023-11-08T19:25:51.989707+0000","last_peered":"2023-11-08T19:25:51.989707+0000","last_clean":"2023-11-08T19:25:51.989707+0000","last_became_active":"2023-11-08T19:25:50.880384+0000","last_became_peered":"2023-11-08T19:25:50.880384+0000","last_unstale":"2023-11-08T19:25:51.989707+0000","last_undegraded":"2023-11-08T19:25:51.989707+0000","last_fullsized":"2023-11-08T19:25:51.989707+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:20:43.175074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.991693+0000","last_change":"2023-11-08T19:25:49.877196+0000","last_active":"2023-11-08T19:25:51.991693+0000","last_peered":"2023-11-08T19:25:51.991693+0000","last_clean":"2023-11-08T19:25:51.991693+0000","last_became_active":"2023-11-08T19:25:49.876517+0000","last_became_peered":"2023-11-08T19:25:49.876517+0000","last_unstale":"2023-11-08T19:25:51.991693+0000","last_undegraded":"2023-11-08T19:25:51.991693+0000","last_fullsized":"2023-11-08T19:25:51.991693+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:00:52.169251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.203760+0000","last_change":"2023-11-08T19:25:50.893157+0000","last_active":"2023-11-08T19:25:52.203760+0000","last_peered":"2023-11-08T19:25:52.203760+0000","last_clean":"2023-11-08T19:25:52.203760+0000","last_became_active":"2023-11-08T19:25:50.892637+0000","last_became_peered":"2023-11-08T19:25:50.892637+0000","last_unstale":"2023-11-08T19:25:52.203760+0000","last_undegraded":"2023-11-08T19:25:52.203760+0000","last_fullsized":"2023-11-08T19:25:52.203760+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:50:52.902620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.205077+0000","last_change":"2023-11-08T19:25:49.881923+0000","last_active":"2023-11-08T19:25:52.205077+0000","last_peered":"2023-11-08T19:25:52.205077+0000","last_clean":"2023-11-08T19:25:52.205077+0000","last_became_active":"2023-11-08T19:25:49.881670+0000","last_became_peered":"2023-11-08T19:25:49.881670+0000","last_unstale":"2023-11-08T19:25:52.205077+0000","last_undegraded":"2023-11-08T19:25:52.205077+0000","last_fullsized":"2023-11-08T19:25:52.205077+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:51:26.807463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.204772+0000","last_change":"2023-11-08T19:25:50.883910+0000","last_active":"2023-11-08T19:25:52.204772+0000","last_peered":"2023-11-08T19:25:52.204772+0000","last_clean":"2023-11-08T19:25:52.204772+0000","last_became_active":"2023-11-08T19:25:50.883615+0000","last_became_peered":"2023-11-08T19:25:50.883615+0000","last_unstale":"2023-11-08T19:25:52.204772+0000","last_undegraded":"2023-11-08T19:25:52.204772+0000","last_fullsized":"2023-11-08T19:25:52.204772+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:20:43.175074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:52.203857+0000","last_change":"2023-11-08T19:25:49.884441+0000","last_active":"2023-11-08T19:25:52.203857+0000","last_peered":"2023-11-08T19:25:52.203857+0000","last_clean":"2023-11-08T19:25:52.203857+0000","last_became_active":"2023-11-08T19:25:49.884125+0000","last_became_peered":"2023-11-08T19:25:49.884125+0000","last_unstale":"2023-11-08T19:25:52.203857+0000","last_undegraded":"2023-11-08T19:25:52.203857+0000","last_fullsized":"2023-11-08T19:25:52.203857+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:52:19.324495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T19:25:51.879275+0000","last_change":"2023-11-08T19:25:50.879136+0000","last_active":"2023-11-08T19:25:51.879275+0000","last_peered":"2023-11-08T19:25:51.879275+0000","last_clean":"2023-11-08T19:25:51.879275+0000","last_became_active":"2023-11-08T19:25:50.878874+0000","last_became_peered":"2023-11-08T19:25:50.878874+0000","last_unstale":"2023-11-08T19:25:51.879275+0000","last_undegraded":"2023-11-08T19:25:51.879275+0000","last_fullsized":"2023-11-08T19:25:51.879275+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:49.854746+0000","last_clean_scrub_stamp":"2023-11-08T19:25:49.854746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:27:11.209258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-08T19:25:50.868878+0000","last_change":"2023-11-08T19:25:49.879080+0000","last_active":"2023-11-08T19:25:50.868878+0000","last_peered":"2023-11-08T19:25:50.868878+0000","last_clean":"2023-11-08T19:25:50.868878+0000","last_became_active":"2023-11-08T19:25:49.878779+0000","last_became_peered":"2023-11-08T19:25:49.878779+0000","last_unstale":"2023-11-08T19:25:50.868878+0000","last_undegraded":"2023-11-08T19:25:50.868878+0000","last_fullsized":"2023-11-08T19:25:50.868878+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:48.844481+0000","last_clean_scrub_stamp":"2023-11-08T19:25:48.844481+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:17:52.641253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":178,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-08T19:25:50.870867+0000","last_change":"2023-11-08T19:24:45.888122+0000","last_active":"2023-11-08T19:25:50.870867+0000","last_peered":"2023-11-08T19:25:50.870867+0000","last_clean":"2023-11-08T19:25:50.870867+0000","last_became_active":"2023-11-08T19:24:45.887048+0000","last_became_peered":"2023-11-08T19:24:45.887048+0000","last_unstale":"2023-11-08T19:25:50.870867+0000","last_undegraded":"2023-11-08T19:25:50.870867+0000","last_fullsized":"2023-11-08T19:25:50.870867+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:44.575133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:44.575133+0000","last_clean_scrub_stamp":"2023-11-08T19:24:44.575133+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:19:20.202062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574866,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27976,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343864,"statfs":{"total":96636764160,"available":96608116736,"internally_reserved":0,"allocated":970752,"data_stored":736324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.432}]},{"osd":1,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.55300000000000005}]},{"osd":2,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.51000000000000001}]},{"osd":3,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.629}]},{"osd":4,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.52800000000000002}]},{"osd":5,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.61899999999999999}]},{"osd":6,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.67400000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574866,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":380928,"data_stored":146482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":21.359999999999999,"5min":21.359999999999999,"15min":21.359999999999999},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":10.329000000000001,"5min":10.329000000000001,"15min":10.329000000000001},"last":0.90300000000000002}]},{"osd":1,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":3.4239999999999999,"5min":3.4239999999999999,"15min":3.4239999999999999},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":3.4809999999999999,"5min":3.4809999999999999,"15min":3.4809999999999999},"last":0.94999999999999996}]},{"osd":2,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":10.398,"5min":10.398,"15min":10.398},"last":0.96399999999999997},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":5.0170000000000003,"5min":5.0170000000000003,"15min":5.0170000000000003},"last":0.89100000000000001}]},{"osd":3,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":3.5710000000000002,"5min":3.5710000000000002,"15min":3.5710000000000002},"last":0.877},{"interface":"front","average":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":10.368,"5min":10.368,"15min":10.368},"last":0.93500000000000005}]},{"osd":4,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.753}]},{"osd":5,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.82199999999999995}]},{"osd":7,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.72799999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574866,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27404,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344436,"statfs":{"total":96636764160,"available":96608702464,"internally_reserved":0,"allocated":389120,"data_stored":147500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":1.389,"5min":1.389,"15min":1.389},"last":0.499},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":9.1370000000000005,"5min":9.1370000000000005,"15min":9.1370000000000005},"last":0.39400000000000002}]},{"osd":2,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.433,"5min":0.433,"15min":0.433},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":1.425,"5min":1.425,"15min":1.425},"last":0.42699999999999999}]},{"osd":3,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":11.957000000000001,"5min":11.957000000000001,"15min":11.957000000000001},"last":0.47699999999999998},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":18.039000000000001,"5min":18.039000000000001,"15min":18.039000000000001},"last":0.46300000000000002}]},{"osd":4,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":6.181,"5min":6.181,"15min":6.181},"last":1.4370000000000001},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.488,"5min":1.488,"15min":1.488},"last":1.411}]},{"osd":5,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":9.2420000000000009,"5min":9.2420000000000009,"15min":9.2420000000000009},"last":1.421},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":6.2110000000000003,"5min":6.2110000000000003,"15min":6.2110000000000003},"last":1.2969999999999999}]},{"osd":6,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":9.1880000000000006,"5min":9.1880000000000006,"15min":9.1880000000000006},"last":1.4530000000000001},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.462,"5min":1.462,"15min":1.462},"last":1.272}]},{"osd":7,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":6.1479999999999997,"5min":6.1479999999999997,"15min":6.1479999999999997},"last":1.3979999999999999},{"interface":"front","average":{"1min":1.028,"5min":1.028,"15min":1.028},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":9.2170000000000005,"5min":9.2170000000000005,"15min":9.2170000000000005},"last":1.353}]}]},{"osd":0,"up_from":13,"seq":55834574866,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27972,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343868,"statfs":{"total":96636764160,"available":96608120832,"internally_reserved":0,"allocated":966656,"data_stored":736234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":4.1200000000000001,"5min":4.1200000000000001,"15min":4.1200000000000001},"last":0.32200000000000001},{"interface":"front","average":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":16.940999999999999,"5min":16.940999999999999,"15min":16.940999999999999},"last":0.44500000000000001}]},{"osd":2,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.37},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":7.0209999999999999,"5min":7.0209999999999999,"15min":7.0209999999999999},"last":0.41399999999999998}]},{"osd":3,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":4.0940000000000003,"5min":4.0940000000000003,"15min":4.0940000000000003},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.432}]},{"osd":4,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":4.0049999999999999,"5min":4.0049999999999999,"15min":4.0049999999999999},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.79200000000000004}]},{"osd":5,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":4.3090000000000002,"5min":4.3090000000000002,"15min":4.3090000000000002},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":4.0709999999999997,"5min":4.0709999999999997,"15min":4.0709999999999997},"last":0.67300000000000004}]},{"osd":6,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":4.2560000000000002,"5min":4.2560000000000002,"15min":4.2560000000000002},"last":0.75900000000000001}]},{"osd":7,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":4.2850000000000001,"5min":4.2850000000000001,"15min":4.2850000000000001},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":4.0460000000000003,"5min":4.0460000000000003,"15min":4.0460000000000003},"last":0.80600000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574866,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27392,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":376832,"data_stored":145936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"min":{"1min":0.20100000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":6.0999999999999996,"5min":6.0999999999999996,"15min":6.0999999999999996},"last":0.249}]},{"osd":1,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"last":0.28799999999999998},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.624,"5min":0.624,"15min":0.624},"last":0.313}]},{"osd":3,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.38300000000000001}]},{"osd":4,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.55500000000000005}]},{"osd":5,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.57599999999999996}]},{"osd":6,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.65100000000000002}]},{"osd":7,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.628},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.80000000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574867,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27404,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344436,"statfs":{"total":96636764160,"available":96608702464,"internally_reserved":0,"allocated":389120,"data_stored":147500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:25:46 2023","interfaces":[{"interface":"back","average":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.626,"5min":0.626,"15min":0.626},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.33100000000000002}]},{"osd":1,"last update":"Wed Nov 8 19:25:46 2023","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.34999999999999998}]},{"osd":2,"last update":"Wed Nov 8 19:25:46 2023","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.37,"15min":0.37},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"last":0.40000000000000002}]},{"osd":4,"last update":"Wed Nov 8 19:25:46 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.70599999999999996}]},{"osd":5,"last update":"Wed Nov 8 19:25:46 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.56699999999999995}]},{"osd":6,"last update":"Wed Nov 8 19:25:46 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.74199999999999999}]},{"osd":7,"last update":"Wed Nov 8 19:25:46 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.66400000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574866,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":380928,"data_stored":145970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:25:47 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":6.5970000000000004,"5min":6.5970000000000004,"15min":6.5970000000000004},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":2.5030000000000001,"5min":2.5030000000000001,"15min":2.5030000000000001},"last":0.75}]},{"osd":1,"last update":"Wed Nov 8 19:25:47 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.72599999999999998}]},{"osd":2,"last update":"Wed Nov 8 19:25:47 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":6.6920000000000002,"5min":6.6920000000000002,"15min":6.6920000000000002},"last":0.871}]},{"osd":3,"last update":"Wed Nov 8 19:25:47 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.91700000000000004}]},{"osd":5,"last update":"Wed Nov 8 19:25:47 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.65900000000000003}]},{"osd":6,"last update":"Wed Nov 8 19:25:47 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.78100000000000003}]},{"osd":7,"last update":"Wed Nov 8 19:25:47 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.85299999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574866,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":380928,"data_stored":146538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.627},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.438}]},{"osd":1,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.50600000000000001}]},{"osd":2,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.46300000000000002}]},{"osd":3,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.58299999999999996}]},{"osd":4,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.59699999999999998}]},{"osd":6,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.67700000000000005}]},{"osd":7,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.61499999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-08T19:25:59.894 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-08T19:25:59.895 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-08T19:26:00.054 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.053+0000 7f68f8118700 1 Processor -- start 2023-11-08T19:26:00.054 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.053+0000 7f68f8118700 1 -- start start 2023-11-08T19:26:00.064 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.054+0000 7f68f8118700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f68f015bb20 0x7f68f00b2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:00.064 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.054+0000 7f68f8118700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68f00d0420 con 0x7f68f00d1d80 2023-11-08T19:26:00.065 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.054+0000 7f68f8118700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68f00cf810 con 0x7f68f00b1730 2023-11-08T19:26:00.065 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.054+0000 7f68f8118700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68f00cf600 con 0x7f68f015bb20 2023-11-08T19:26:00.065 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.054+0000 7f68f66b5700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f68f00d1d80 0x7f68f00cc4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:46922/0 (socket says 172.21.15.143:46922) 2023-11-08T19:26:00.066 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.054+0000 7f68f66b5700 1 -- 172.21.15.143:0/3699657321 learned_addr learned my addr 172.21.15.143:0/3699657321 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:26:00.066 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.054+0000 7f68f5eb4700 1 --2- 172.21.15.143:0/3699657321 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f68f015bb20 0x7f68f00b2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:00.067 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.054+0000 7f68f4eb2700 1 -- 172.21.15.143:0/3699657321 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1611582032 0 0) 0x7f68f00d0420 con 0x7f68f00d1d80 2023-11-08T19:26:00.067 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.054+0000 7f68f4eb2700 1 -- 172.21.15.143:0/3699657321 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68dc0035b0 con 0x7f68f00d1d80 2023-11-08T19:26:00.067 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.054+0000 7f68f4eb2700 1 -- 172.21.15.143:0/3699657321 <== mon.1 v1:172.21.15.174:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 901172420 0 0) 0x7f68f00cf810 con 0x7f68f00b1730 2023-11-08T19:26:00.068 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.054+0000 7f68f4eb2700 1 -- 172.21.15.143:0/3699657321 --> v1:172.21.15.174:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68f00d0420 con 0x7f68f00b1730 2023-11-08T19:26:00.068 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.054+0000 7f68f5eb4700 1 -- 172.21.15.143:0/3699657321 >> v1:172.21.15.174:6789/0 conn(0x7f68f00b1730 legacy=0x7f68f00d1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:00.069 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.054+0000 7f68f5eb4700 1 -- 172.21.15.143:0/3699657321 >> v1:172.21.15.143:6789/0 conn(0x7f68f00d1d80 legacy=0x7f68f00cc4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:00.069 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.054+0000 7f68f4eb2700 1 -- 172.21.15.143:0/3699657321 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 25419274 0 0) 0x7f68dc0035b0 con 0x7f68f00d1d80 2023-11-08T19:26:00.069 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.054+0000 7f68f5eb4700 1 -- 172.21.15.143:0/3699657321 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68f00cfaf0 con 0x7f68f015bb20 2023-11-08T19:26:00.070 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.055+0000 7f68f5eb4700 1 --2- 172.21.15.143:0/3699657321 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f68f015bb20 0x7f68f00b2670 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f68ec009870 tx=0x7f68ec009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dafd352ddf6b5a6a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:00.070 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.055+0000 7f68f4eb2700 1 -- 172.21.15.143:0/3699657321 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68e80019f0 con 0x7f68f015bb20 2023-11-08T19:26:00.070 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.055+0000 7f68f4eb2700 1 -- 172.21.15.143:0/3699657321 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68ec004030 con 0x7f68f015bb20 2023-11-08T19:26:00.071 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.055+0000 7f68f4eb2700 1 -- 172.21.15.143:0/3699657321 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68ec0043c0 con 0x7f68f015bb20 2023-11-08T19:26:00.071 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.055+0000 7f68f8118700 1 -- 172.21.15.143:0/3699657321 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f68f015bb20 msgr2=0x7f68f00b2670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:00.071 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.055+0000 7f68f8118700 1 --2- 172.21.15.143:0/3699657321 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f68f015bb20 0x7f68f00b2670 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f68ec009870 tx=0x7f68ec009b80 comp rx=0 tx=0).stop 2023-11-08T19:26:00.071 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.055+0000 7f68f8118700 1 -- 172.21.15.143:0/3699657321 shutdown_connections 2023-11-08T19:26:00.072 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.055+0000 7f68f8118700 1 --2- 172.21.15.143:0/3699657321 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f68f015bb20 0x7f68f00b2670 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:00.072 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.055+0000 7f68f8118700 1 -- 172.21.15.143:0/3699657321 >> 172.21.15.143:0/3699657321 conn(0x7f68f008b000 msgr2=0x7f68f009ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:00.072 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.055+0000 7f68f8118700 1 -- 172.21.15.143:0/3699657321 shutdown_connections 2023-11-08T19:26:00.072 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.055+0000 7f68f8118700 1 -- 172.21.15.143:0/3699657321 wait complete. 2023-11-08T19:26:00.073 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.056+0000 7f68f8118700 1 Processor -- start 2023-11-08T19:26:00.073 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.056+0000 7f68f8118700 1 -- start start 2023-11-08T19:26:00.073 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.057+0000 7f68f8118700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f68f00b1730 0x7f68f00c4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:00.073 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.057+0000 7f68f8118700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f68f00d1d80 0x7f68f00ceeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:00.073 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.057+0000 7f68f8118700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f68f015bb20 0x7f68f0154080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:00.074 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.057+0000 7f68f8118700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68f01e5190 con 0x7f68f015bb20 2023-11-08T19:26:00.074 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.057+0000 7f68f8118700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68f00cca90 con 0x7f68f00b1730 2023-11-08T19:26:00.074 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.057+0000 7f68f8118700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68f00cc4d0 con 0x7f68f00d1d80 2023-11-08T19:26:00.074 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.057+0000 7f68f66b5700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f68f015bb20 0x7f68f0154080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:00.075 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.057+0000 7f68f66b5700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f68f015bb20 0x7f68f0154080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50774/0 (socket says 172.21.15.143:50774) 2023-11-08T19:26:00.075 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.057+0000 7f68f66b5700 1 -- 172.21.15.143:0/2196869725 learned_addr learned my addr 172.21.15.143:0/2196869725 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:00.075 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.058+0000 7f68f56b3700 1 --2- 172.21.15.143:0/2196869725 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f68f00d1d80 0x7f68f00ceeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:00.075 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.058+0000 7f68f5eb4700 1 --2- 172.21.15.143:0/2196869725 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f68f00b1730 0x7f68f00c4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:00.076 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.058+0000 7f68f66b5700 1 -- 172.21.15.143:0/2196869725 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f68f00d1d80 msgr2=0x7f68f00ceeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:00.076 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.058+0000 7f68f66b5700 1 --2- 172.21.15.143:0/2196869725 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f68f00d1d80 0x7f68f00ceeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:00.076 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.058+0000 7f68f66b5700 1 -- 172.21.15.143:0/2196869725 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f68f00b1730 msgr2=0x7f68f00c4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:00.076 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.058+0000 7f68f66b5700 1 --2- 172.21.15.143:0/2196869725 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f68f00b1730 0x7f68f00c4420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:00.077 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.058+0000 7f68f66b5700 1 -- 172.21.15.143:0/2196869725 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68ec0094a0 con 0x7f68f015bb20 2023-11-08T19:26:00.077 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.058+0000 7f68f66b5700 1 --2- 172.21.15.143:0/2196869725 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f68f015bb20 0x7f68f0154080 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f68e8001560 tx=0x7f68e800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:00.077 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.058+0000 7f68e67fc700 1 -- 172.21.15.143:0/2196869725 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68e800ade0 con 0x7f68f015bb20 2023-11-08T19:26:00.078 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.058+0000 7f68e67fc700 1 -- 172.21.15.143:0/2196869725 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68e80019c0 con 0x7f68f015bb20 2023-11-08T19:26:00.078 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.058+0000 7f68f8118700 1 -- 172.21.15.143:0/2196869725 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68f00cc7b0 con 0x7f68f015bb20 2023-11-08T19:26:00.078 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.058+0000 7f68e67fc700 1 -- 172.21.15.143:0/2196869725 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68e80072f0 con 0x7f68f015bb20 2023-11-08T19:26:00.078 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.058+0000 7f68f8118700 1 -- 172.21.15.143:0/2196869725 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68f006d100 con 0x7f68f015bb20 2023-11-08T19:26:00.079 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.059+0000 7f68f8118700 1 -- 172.21.15.143:0/2196869725 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68f01e52d0 con 0x7f68f015bb20 2023-11-08T19:26:00.079 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.059+0000 7f68e67fc700 1 -- 172.21.15.143:0/2196869725 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f68e8007450 con 0x7f68f015bb20 2023-11-08T19:26:00.079 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.060+0000 7f68e67fc700 1 --2- 172.21.15.143:0/2196869725 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f68d406af00 0x7f68d406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:00.079 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.060+0000 7f68e67fc700 1 -- 172.21.15.143:0/2196869725 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f68e80894f0 con 0x7f68f015bb20 2023-11-08T19:26:00.080 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.060+0000 7f68f5eb4700 1 --2- 172.21.15.143:0/2196869725 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f68d406af00 0x7f68d406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:00.080 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.060+0000 7f68f5eb4700 1 --2- 172.21.15.143:0/2196869725 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f68d406af00 0x7f68d406d3c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f68ec000c00 tx=0x7f68ec013040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:00.080 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.064+0000 7f68e67fc700 1 -- 172.21.15.143:0/2196869725 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f68f01e52d0 con 0x7f68f015bb20 2023-11-08T19:26:00.332 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.331+0000 7f68f8118700 1 -- 172.21.15.143:0/2196869725 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f68f01e52d0 con 0x7f68f015bb20 2023-11-08T19:26:00.334 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.333+0000 7f68e67fc700 1 -- 172.21.15.143:0/2196869725 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v9) v1 ==== 190+0+147 (secure 0 0 0) 0x7f68e80180b0 con 0x7f68f015bb20 2023-11-08T19:26:00.334 INFO:teuthology.orchestra.run.smithi143.stdout:[client.0] 2023-11-08T19:26:00.335 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQBP4Etl0GdSFxAAVkuHL0EnjrtTUMkj/afPkg== 2023-11-08T19:26:00.335 INFO:teuthology.orchestra.run.smithi143.stdout: caps mds = "allow" 2023-11-08T19:26:00.335 INFO:teuthology.orchestra.run.smithi143.stdout: caps mon = "allow r" 2023-11-08T19:26:00.335 INFO:teuthology.orchestra.run.smithi143.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-11-08T19:26:00.337 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.336+0000 7f68f8118700 1 -- 172.21.15.143:0/2196869725 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f68d406af00 msgr2=0x7f68d406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:00.338 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.336+0000 7f68f8118700 1 --2- 172.21.15.143:0/2196869725 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f68d406af00 0x7f68d406d3c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f68ec000c00 tx=0x7f68ec013040 comp rx=0 tx=0).stop 2023-11-08T19:26:00.338 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.336+0000 7f68f8118700 1 -- 172.21.15.143:0/2196869725 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f68f015bb20 msgr2=0x7f68f0154080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:00.338 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.336+0000 7f68f8118700 1 --2- 172.21.15.143:0/2196869725 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f68f015bb20 0x7f68f0154080 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f68e8001560 tx=0x7f68e800a340 comp rx=0 tx=0).stop 2023-11-08T19:26:00.338 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.336+0000 7f68f8118700 1 -- 172.21.15.143:0/2196869725 shutdown_connections 2023-11-08T19:26:00.338 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.336+0000 7f68f8118700 1 --2- 172.21.15.143:0/2196869725 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f68d406af00 0x7f68d406d3c0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:00.339 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.336+0000 7f68f8118700 1 --2- 172.21.15.143:0/2196869725 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f68f015bb20 0x7f68f0154080 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:00.339 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.336+0000 7f68f8118700 1 --2- 172.21.15.143:0/2196869725 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f68f00b1730 0x7f68f00c4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:00.339 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.336+0000 7f68f8118700 1 --2- 172.21.15.143:0/2196869725 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f68f00d1d80 0x7f68f00ceeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:00.339 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.336+0000 7f68f8118700 1 -- 172.21.15.143:0/2196869725 >> 172.21.15.143:0/2196869725 conn(0x7f68f008b000 msgr2=0x7f68f00d1580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:00.339 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.336+0000 7f68f8118700 1 -- 172.21.15.143:0/2196869725 shutdown_connections 2023-11-08T19:26:00.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.336+0000 7f68f8118700 1 -- 172.21.15.143:0/2196869725 wait complete. 2023-11-08T19:26:00.340 INFO:teuthology.orchestra.run.smithi143.stderr:updated caps for client.0 2023-11-08T19:26:00.349 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:26:00.469 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.468+0000 7fcd371dc700 1 Processor -- start 2023-11-08T19:26:00.469 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.468+0000 7fcd371dc700 1 -- start start 2023-11-08T19:26:00.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.468+0000 7fcd371dc700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fcd3015dd00 0x7fcd300cca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:00.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.468+0000 7fcd371dc700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fcd301618f0 0x7fcd3006ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:00.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.468+0000 7fcd371dc700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd300ccfb0 con 0x7fcd3006b150 2023-11-08T19:26:00.473 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.469+0000 7fcd371dc700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd3006b600 con 0x7fcd301618f0 2023-11-08T19:26:00.473 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.469+0000 7fcd371dc700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd300cad50 con 0x7fcd3015dd00 2023-11-08T19:26:00.473 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.469+0000 7fcd2ffff700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fcd301618f0 0x7fcd3006ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:00.473 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.469+0000 7fcd35779700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fcd3006b150 0x7fcd300b2210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:46936/0 (socket says 172.21.15.143:46936) 2023-11-08T19:26:00.473 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.469+0000 7fcd35779700 1 -- 172.21.15.143:0/658019239 learned_addr learned my addr 172.21.15.143:0/658019239 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:26:00.474 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.469+0000 7fcd2ffff700 1 -- 172.21.15.143:0/658019239 >> v2:172.21.15.174:3300/0 conn(0x7fcd3015dd00 msgr2=0x7fcd300cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:00.474 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.469+0000 7fcd34f78700 1 --2- 172.21.15.143:0/658019239 >> v2:172.21.15.174:3300/0 conn(0x7fcd3015dd00 0x7fcd300cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:00.474 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.469+0000 7fcd2ffff700 1 --2- 172.21.15.143:0/658019239 >> v2:172.21.15.174:3300/0 conn(0x7fcd3015dd00 0x7fcd300cca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:00.474 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.469+0000 7fcd2ffff700 1 -- 172.21.15.143:0/658019239 >> v1:172.21.15.143:6789/0 conn(0x7fcd3006b150 legacy=0x7fcd300b2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:00.474 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.469+0000 7fcd2ffff700 1 -- 172.21.15.143:0/658019239 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd30161df0 con 0x7fcd301618f0 2023-11-08T19:26:00.475 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.469+0000 7fcd2f7fe700 1 -- 172.21.15.143:0/658019239 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2833844135 0 0) 0x7fcd300ccfb0 con 0x7fcd3006b150 2023-11-08T19:26:00.475 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.469+0000 7fcd2ffff700 1 --2- 172.21.15.143:0/658019239 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcd301618f0 0x7fcd3006ac10 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fcd30070a60 tx=0x7fcd1c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4986c9b33a2e38c2 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:00.475 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.469+0000 7fcd2f7fe700 1 -- 172.21.15.143:0/658019239 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd1c011830 con 0x7fcd301618f0 2023-11-08T19:26:00.475 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.469+0000 7fcd2f7fe700 1 -- 172.21.15.143:0/658019239 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd1c011990 con 0x7fcd301618f0 2023-11-08T19:26:00.475 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.469+0000 7fcd2f7fe700 1 -- 172.21.15.143:0/658019239 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd1c011d20 con 0x7fcd301618f0 2023-11-08T19:26:00.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.470+0000 7fcd371dc700 1 -- 172.21.15.143:0/658019239 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcd301618f0 msgr2=0x7fcd3006ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:00.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.470+0000 7fcd371dc700 1 --2- 172.21.15.143:0/658019239 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcd301618f0 0x7fcd3006ac10 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fcd30070a60 tx=0x7fcd1c009b20 comp rx=0 tx=0).stop 2023-11-08T19:26:00.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.470+0000 7fcd371dc700 1 -- 172.21.15.143:0/658019239 shutdown_connections 2023-11-08T19:26:00.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.470+0000 7fcd371dc700 1 --2- 172.21.15.143:0/658019239 >> v2:172.21.15.174:3300/0 conn(0x7fcd3015dd00 0x7fcd300cca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:00.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.470+0000 7fcd371dc700 1 --2- 172.21.15.143:0/658019239 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcd301618f0 0x7fcd3006ac10 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:00.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.470+0000 7fcd371dc700 1 -- 172.21.15.143:0/658019239 >> 172.21.15.143:0/658019239 conn(0x7fcd30099ab0 msgr2=0x7fcd300d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:00.477 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.470+0000 7fcd371dc700 1 -- 172.21.15.143:0/658019239 shutdown_connections 2023-11-08T19:26:00.477 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.470+0000 7fcd371dc700 1 -- 172.21.15.143:0/658019239 wait complete. 2023-11-08T19:26:00.477 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.471+0000 7fcd371dc700 1 Processor -- start 2023-11-08T19:26:00.477 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.471+0000 7fcd371dc700 1 -- start start 2023-11-08T19:26:00.480 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.472+0000 7fcd371dc700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcd3006b150 0x7fcd300b00f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:00.481 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.472+0000 7fcd371dc700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcd3015dd00 0x7fcd30156d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:00.481 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.472+0000 7fcd371dc700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcd301618f0 0x7fcd300b4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:00.481 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.472+0000 7fcd371dc700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd300c4cc0 con 0x7fcd3015dd00 2023-11-08T19:26:00.481 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.472+0000 7fcd371dc700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd300c43b0 con 0x7fcd3006b150 2023-11-08T19:26:00.482 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.472+0000 7fcd371dc700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd300b0630 con 0x7fcd301618f0 2023-11-08T19:26:00.482 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.472+0000 7fcd2ffff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcd3015dd00 0x7fcd30156d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:00.482 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.472+0000 7fcd2ffff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcd3015dd00 0x7fcd30156d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50786/0 (socket says 172.21.15.143:50786) 2023-11-08T19:26:00.482 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.472+0000 7fcd2ffff700 1 -- 172.21.15.143:0/3401431421 learned_addr learned my addr 172.21.15.143:0/3401431421 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:00.482 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.472+0000 7fcd34f78700 1 --2- 172.21.15.143:0/3401431421 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcd3006b150 0x7fcd300b00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:00.483 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.472+0000 7fcd35779700 1 --2- 172.21.15.143:0/3401431421 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcd301618f0 0x7fcd300b4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:00.483 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.472+0000 7fcd2ffff700 1 -- 172.21.15.143:0/3401431421 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcd301618f0 msgr2=0x7fcd300b4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:00.483 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.472+0000 7fcd2ffff700 1 --2- 172.21.15.143:0/3401431421 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcd301618f0 0x7fcd300b4900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:00.483 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.472+0000 7fcd2ffff700 1 -- 172.21.15.143:0/3401431421 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcd3006b150 msgr2=0x7fcd300b00f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:00.483 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.472+0000 7fcd2ffff700 1 --2- 172.21.15.143:0/3401431421 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcd3006b150 0x7fcd300b00f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:00.484 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.472+0000 7fcd2ffff700 1 -- 172.21.15.143:0/3401431421 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd1c0094a0 con 0x7fcd3015dd00 2023-11-08T19:26:00.484 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.472+0000 7fcd2ffff700 1 --2- 172.21.15.143:0/3401431421 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcd3015dd00 0x7fcd30156d20 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c000c00 tx=0x7fcd1c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:00.484 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.473+0000 7fcd2d7fa700 1 -- 172.21.15.143:0/3401431421 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd1c0038a0 con 0x7fcd3015dd00 2023-11-08T19:26:00.484 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.473+0000 7fcd2d7fa700 1 -- 172.21.15.143:0/3401431421 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd1c01f070 con 0x7fcd3015dd00 2023-11-08T19:26:00.484 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.473+0000 7fcd371dc700 1 -- 172.21.15.143:0/3401431421 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd300c4670 con 0x7fcd3015dd00 2023-11-08T19:26:00.484 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.473+0000 7fcd2d7fa700 1 -- 172.21.15.143:0/3401431421 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd1c003c40 con 0x7fcd3015dd00 2023-11-08T19:26:00.485 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.473+0000 7fcd371dc700 1 -- 172.21.15.143:0/3401431421 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd30157260 con 0x7fcd3015dd00 2023-11-08T19:26:00.485 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.474+0000 7fcd371dc700 1 -- 172.21.15.143:0/3401431421 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd300aff60 con 0x7fcd3015dd00 2023-11-08T19:26:00.485 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.474+0000 7fcd2d7fa700 1 -- 172.21.15.143:0/3401431421 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fcd1c026480 con 0x7fcd3015dd00 2023-11-08T19:26:00.485 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.474+0000 7fcd2d7fa700 1 --2- 172.21.15.143:0/3401431421 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fcd1406af30 0x7fcd1406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:00.485 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.474+0000 7fcd2d7fa700 1 -- 172.21.15.143:0/3401431421 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcd1c021070 con 0x7fcd3015dd00 2023-11-08T19:26:00.486 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.475+0000 7fcd34f78700 1 --2- 172.21.15.143:0/3401431421 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fcd1406af30 0x7fcd1406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:00.486 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.475+0000 7fcd34f78700 1 --2- 172.21.15.143:0/3401431421 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fcd1406af30 0x7fcd1406d3f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fcd24006fd0 tx=0x7fcd24008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:00.486 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.479+0000 7fcd2d7fa700 1 -- 172.21.15.143:0/3401431421 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcd1c0560f0 con 0x7fcd3015dd00 2023-11-08T19:26:00.713 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.711+0000 7fcd371dc700 1 -- 172.21.15.143:0/3401431421 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcd301620a0 con 0x7fcd3015dd00 2023-11-08T19:26:00.713 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.712+0000 7fcd2d7fa700 1 -- 172.21.15.143:0/3401431421 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4925 (secure 0 0 0) 0x7fcd1c015020 con 0x7fcd3015dd00 2023-11-08T19:26:00.714 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:26:00.714 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4918,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6836/3265911434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3265911434},{"type":"v1","addr":"172.21.15.174:6836","nonce":3265911434}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4921,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/2092876318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6835","nonce":2092876318},{"type":"v1","addr":"172.21.15.174:6837","nonce":2092876318}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5106,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6834/4077367240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":4077367240},{"type":"v1","addr":"172.21.15.143:6834","nonce":4077367240}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T19:25:50.257292+0000","modified":"2023-11-08T19:25:53.278532+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5109},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5109":{"gid":5109,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.143:6835/1574070101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":1574070101},{"type":"v1","addr":"172.21.15.143:6835","nonce":1574070101}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-08T19:26:00.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.714+0000 7fcd371dc700 1 -- 172.21.15.143:0/3401431421 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fcd1406af30 msgr2=0x7fcd1406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:00.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.714+0000 7fcd371dc700 1 --2- 172.21.15.143:0/3401431421 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fcd1406af30 0x7fcd1406d3f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fcd24006fd0 tx=0x7fcd24008040 comp rx=0 tx=0).stop 2023-11-08T19:26:00.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.714+0000 7fcd371dc700 1 -- 172.21.15.143:0/3401431421 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcd3015dd00 msgr2=0x7fcd30156d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:00.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.714+0000 7fcd371dc700 1 --2- 172.21.15.143:0/3401431421 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcd3015dd00 0x7fcd30156d20 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c000c00 tx=0x7fcd1c004750 comp rx=0 tx=0).stop 2023-11-08T19:26:00.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.714+0000 7fcd371dc700 1 -- 172.21.15.143:0/3401431421 shutdown_connections 2023-11-08T19:26:00.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.714+0000 7fcd371dc700 1 --2- 172.21.15.143:0/3401431421 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fcd1406af30 0x7fcd1406d3f0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:00.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.714+0000 7fcd371dc700 1 --2- 172.21.15.143:0/3401431421 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcd301618f0 0x7fcd300b4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:00.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.714+0000 7fcd371dc700 1 --2- 172.21.15.143:0/3401431421 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcd3006b150 0x7fcd300b00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:00.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.714+0000 7fcd371dc700 1 --2- 172.21.15.143:0/3401431421 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcd3015dd00 0x7fcd30156d20 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:00.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.714+0000 7fcd371dc700 1 -- 172.21.15.143:0/3401431421 >> 172.21.15.143:0/3401431421 conn(0x7fcd30099ab0 msgr2=0x7fcd3015b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:00.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.714+0000 7fcd371dc700 1 -- 172.21.15.143:0/3401431421 shutdown_connections 2023-11-08T19:26:00.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:00.714+0000 7fcd371dc700 1 -- 172.21.15.143:0/3401431421 wait complete. 2023-11-08T19:26:00.718 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 16 2023-11-08T19:26:01.130 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-08T19:25:50.257292+0000', 'modified': '2023-11-08T19:25:53.278532+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5109}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5109': {'gid': 5109, 'name': 'c', 'rank': 0, 'incarnation': 12, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.143:6835/1574070101', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.143:6833', 'nonce': 1574070101}, {'type': 'v1', 'addr': '172.21.15.143:6835', 'nonce': 1574070101}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-08T19:26:01.131 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-08T19:26:01.131 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-11-08T19:26:01.253 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.252+0000 7fc95cb0b700 1 Processor -- start 2023-11-08T19:26:01.254 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.252+0000 7fc95cb0b700 1 -- start start 2023-11-08T19:26:01.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.253+0000 7fc95cb0b700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc95815dd00 0x7fc9580cca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:01.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.253+0000 7fc95cb0b700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc9581618f0 0x7fc95806ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:01.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.253+0000 7fc95cb0b700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fc95806b150 0x7fc9580b2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:01.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.253+0000 7fc95cb0b700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc95806b600 con 0x7fc9581618f0 2023-11-08T19:26:01.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.253+0000 7fc95cb0b700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9580cad50 con 0x7fc95806b150 2023-11-08T19:26:01.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.253+0000 7fc95cb0b700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9580ccfb0 con 0x7fc95815dd00 2023-11-08T19:26:01.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.253+0000 7fc955d9b700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc9581618f0 0x7fc95806ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:01.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.253+0000 7fc955d9b700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc9581618f0 0x7fc95806ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50794/0 (socket says 172.21.15.143:50794) 2023-11-08T19:26:01.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.253+0000 7fc955d9b700 1 -- 172.21.15.143:0/2910909080 learned_addr learned my addr 172.21.15.143:0/2910909080 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:01.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.253+0000 7fc956d9d700 1 --2- 172.21.15.143:0/2910909080 >> v2:172.21.15.174:3300/0 conn(0x7fc95806b150 0x7fc9580b2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:01.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.254+0000 7fc95659c700 1 --2- 172.21.15.143:0/2910909080 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc95815dd00 0x7fc9580cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:01.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.254+0000 7fc955d9b700 1 -- 172.21.15.143:0/2910909080 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc95815dd00 msgr2=0x7fc9580cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:01.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.254+0000 7fc955d9b700 1 --2- 172.21.15.143:0/2910909080 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc95815dd00 0x7fc9580cca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:01.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.254+0000 7fc955d9b700 1 -- 172.21.15.143:0/2910909080 >> v2:172.21.15.174:3300/0 conn(0x7fc95806b150 msgr2=0x7fc9580b2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:01.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.254+0000 7fc955d9b700 1 --2- 172.21.15.143:0/2910909080 >> v2:172.21.15.174:3300/0 conn(0x7fc95806b150 0x7fc9580b2210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:01.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.254+0000 7fc955d9b700 1 -- 172.21.15.143:0/2910909080 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc958161df0 con 0x7fc9581618f0 2023-11-08T19:26:01.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.254+0000 7fc955d9b700 1 --2- 172.21.15.143:0/2910909080 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc9581618f0 0x7fc95806ac10 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fc9580708b0 tx=0x7fc948009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7522bf2c23e0871d server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:01.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.254+0000 7fc95559a700 1 -- 172.21.15.143:0/2910909080 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc94800b870 con 0x7fc9581618f0 2023-11-08T19:26:01.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.254+0000 7fc95559a700 1 -- 172.21.15.143:0/2910909080 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc94800b9d0 con 0x7fc9581618f0 2023-11-08T19:26:01.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.254+0000 7fc95559a700 1 -- 172.21.15.143:0/2910909080 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc94800bd60 con 0x7fc9581618f0 2023-11-08T19:26:01.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.254+0000 7fc95cb0b700 1 -- 172.21.15.143:0/2910909080 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc9581618f0 msgr2=0x7fc95806ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:01.261 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.254+0000 7fc95cb0b700 1 --2- 172.21.15.143:0/2910909080 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc9581618f0 0x7fc95806ac10 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fc9580708b0 tx=0x7fc948009bd0 comp rx=0 tx=0).stop 2023-11-08T19:26:01.261 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.255+0000 7fc95cb0b700 1 -- 172.21.15.143:0/2910909080 shutdown_connections 2023-11-08T19:26:01.261 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.255+0000 7fc95cb0b700 1 --2- 172.21.15.143:0/2910909080 >> v2:172.21.15.174:3300/0 conn(0x7fc95806b150 0x7fc9580b2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:01.261 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.255+0000 7fc95cb0b700 1 --2- 172.21.15.143:0/2910909080 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc95815dd00 0x7fc9580cca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:01.261 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.255+0000 7fc95cb0b700 1 --2- 172.21.15.143:0/2910909080 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc9581618f0 0x7fc95806ac10 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:01.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.255+0000 7fc95cb0b700 1 -- 172.21.15.143:0/2910909080 >> 172.21.15.143:0/2910909080 conn(0x7fc958099ab0 msgr2=0x7fc9580d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:01.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.255+0000 7fc95cb0b700 1 -- 172.21.15.143:0/2910909080 shutdown_connections 2023-11-08T19:26:01.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.255+0000 7fc95cb0b700 1 -- 172.21.15.143:0/2910909080 wait complete. 2023-11-08T19:26:01.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.256+0000 7fc95cb0b700 1 Processor -- start 2023-11-08T19:26:01.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.256+0000 7fc95cb0b700 1 -- start start 2023-11-08T19:26:01.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.257+0000 7fc95cb0b700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc95806b150 0x7fc9580c5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:01.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.257+0000 7fc95cb0b700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc95815dd00 0x7fc9580c77d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:01.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.257+0000 7fc95cb0b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc9581618f0 0x7fc9580b26b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:01.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.257+0000 7fc95cb0b700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9580b2290 con 0x7fc95806b150 2023-11-08T19:26:01.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.257+0000 7fc95cb0b700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9580c5740 con 0x7fc9581618f0 2023-11-08T19:26:01.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.257+0000 7fc95cb0b700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9580b1bd0 con 0x7fc95815dd00 2023-11-08T19:26:01.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.257+0000 7fc95659c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc95806b150 0x7fc9580c5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:01.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.257+0000 7fc95659c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc95806b150 0x7fc9580c5ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50798/0 (socket says 172.21.15.143:50798) 2023-11-08T19:26:01.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.257+0000 7fc95659c700 1 -- 172.21.15.143:0/4213769252 learned_addr learned my addr 172.21.15.143:0/4213769252 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:01.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.257+0000 7fc955d9b700 1 --2- 172.21.15.143:0/4213769252 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc95815dd00 0x7fc9580c77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:01.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.257+0000 7fc956d9d700 1 --2- 172.21.15.143:0/4213769252 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc9581618f0 0x7fc9580b26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:01.274 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.257+0000 7fc95659c700 1 -- 172.21.15.143:0/4213769252 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc95815dd00 msgr2=0x7fc9580c77d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:01.274 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.257+0000 7fc95659c700 1 --2- 172.21.15.143:0/4213769252 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc95815dd00 0x7fc9580c77d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:01.274 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.257+0000 7fc95659c700 1 -- 172.21.15.143:0/4213769252 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc9581618f0 msgr2=0x7fc9580b26b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:01.274 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.257+0000 7fc95659c700 1 --2- 172.21.15.143:0/4213769252 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc9581618f0 0x7fc9580b26b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:01.274 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.257+0000 7fc95659c700 1 -- 172.21.15.143:0/4213769252 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc948009550 con 0x7fc95806b150 2023-11-08T19:26:01.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.257+0000 7fc956d9d700 1 --2- 172.21.15.143:0/4213769252 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc9581618f0 0x7fc9580b26b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:26:01.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.257+0000 7fc95659c700 1 --2- 172.21.15.143:0/4213769252 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc95806b150 0x7fc9580c5ea0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fc94c00e730 tx=0x7fc94c00eaf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:01.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.257+0000 7fc946ffd700 1 -- 172.21.15.143:0/4213769252 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc94c013610 con 0x7fc95806b150 2023-11-08T19:26:01.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.257+0000 7fc946ffd700 1 -- 172.21.15.143:0/4213769252 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc94c013770 con 0x7fc95806b150 2023-11-08T19:26:01.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.257+0000 7fc946ffd700 1 -- 172.21.15.143:0/4213769252 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc94c013b20 con 0x7fc95806b150 2023-11-08T19:26:01.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.257+0000 7fc95cb0b700 1 -- 172.21.15.143:0/4213769252 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9580b1eb0 con 0x7fc95806b150 2023-11-08T19:26:01.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.258+0000 7fc95cb0b700 1 -- 172.21.15.143:0/4213769252 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc958151eb0 con 0x7fc95806b150 2023-11-08T19:26:01.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.259+0000 7fc946ffd700 1 -- 172.21.15.143:0/4213769252 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc94c013c80 con 0x7fc95806b150 2023-11-08T19:26:01.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.259+0000 7fc946ffd700 1 --2- 172.21.15.143:0/4213769252 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc93806af00 0x7fc93806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:01.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.260+0000 7fc946ffd700 1 -- 172.21.15.143:0/4213769252 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc94c088fe0 con 0x7fc95806b150 2023-11-08T19:26:01.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.260+0000 7fc955d9b700 1 --2- 172.21.15.143:0/4213769252 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc93806af00 0x7fc93806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:01.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.260+0000 7fc955d9b700 1 --2- 172.21.15.143:0/4213769252 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc93806af00 0x7fc93806d3c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc948000c00 tx=0x7fc948008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:01.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.261+0000 7fc968d04b80 1 Processor -- start 2023-11-08T19:26:01.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.262+0000 7fc968d04b80 1 -- start start 2023-11-08T19:26:01.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.262+0000 7fc968d04b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f39abf46f0 0x55f39ac24450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:01.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.262+0000 7fc968d04b80 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55f39abf6c30 0x55f39ac24990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:01.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.262+0000 7fc968d04b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f39abf4ad0 0x55f39ac0ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:01.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.262+0000 7fc968d04b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f39ab606e0 con 0x55f39abf46f0 2023-11-08T19:26:01.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.262+0000 7fc968d04b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f39ab543b0 con 0x55f39abf4ad0 2023-11-08T19:26:01.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.262+0000 7fc968d04b80 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f39ab51cd0 con 0x55f39abf6c30 2023-11-08T19:26:01.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.263+0000 7fc95659c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f39abf46f0 0x55f39ac24450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:01.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.263+0000 7fc95659c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f39abf46f0 0x55f39ac24450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50806/0 (socket says 172.21.15.143:50806) 2023-11-08T19:26:01.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.263+0000 7fc95659c700 1 -- 172.21.15.143:0/382707408 learned_addr learned my addr 172.21.15.143:0/382707408 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:01.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.263+0000 7fc955d9b700 1 --2- 172.21.15.143:0/382707408 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55f39abf6c30 0x55f39ac24990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:01.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.263+0000 7fc956d9d700 1 --2- 172.21.15.143:0/382707408 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f39abf4ad0 0x55f39ac0ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:01.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.263+0000 7fc95659c700 1 -- 172.21.15.143:0/382707408 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55f39abf6c30 msgr2=0x55f39ac24990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:01.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.263+0000 7fc95659c700 1 --2- 172.21.15.143:0/382707408 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55f39abf6c30 0x55f39ac24990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:01.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.263+0000 7fc95659c700 1 -- 172.21.15.143:0/382707408 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f39abf4ad0 msgr2=0x55f39ac0ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:01.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.263+0000 7fc95659c700 1 --2- 172.21.15.143:0/382707408 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f39abf4ad0 0x55f39ac0ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:01.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.263+0000 7fc95659c700 1 -- 172.21.15.143:0/382707408 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f39a9ca5e0 con 0x55f39abf46f0 2023-11-08T19:26:01.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.263+0000 7fc955d9b700 1 --2- 172.21.15.143:0/382707408 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55f39abf6c30 0x55f39ac24990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:26:01.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.263+0000 7fc95659c700 1 --2- 172.21.15.143:0/382707408 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f39abf46f0 0x55f39ac24450 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fc94c05fac0 tx=0x7fc94c0584c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fe4b74ba1e9f6df server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:01.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.263+0000 7fc937fff700 1 -- 172.21.15.143:0/382707408 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc94c05fff0 con 0x55f39abf46f0 2023-11-08T19:26:01.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.263+0000 7fc937fff700 1 -- 172.21.15.143:0/382707408 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc94c056090 con 0x55f39abf46f0 2023-11-08T19:26:01.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.263+0000 7fc937fff700 1 -- 172.21.15.143:0/382707408 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc94c0573e0 con 0x55f39abf46f0 2023-11-08T19:26:01.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.264+0000 7fc968d04b80 1 -- 172.21.15.143:0/382707408 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f39abf46f0 msgr2=0x55f39ac24450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:01.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.264+0000 7fc968d04b80 1 --2- 172.21.15.143:0/382707408 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f39abf46f0 0x55f39ac24450 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fc94c05fac0 tx=0x7fc94c0584c0 comp rx=0 tx=0).stop 2023-11-08T19:26:01.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.264+0000 7fc968d04b80 1 -- 172.21.15.143:0/382707408 shutdown_connections 2023-11-08T19:26:01.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.264+0000 7fc968d04b80 1 --2- 172.21.15.143:0/382707408 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f39abf4ad0 0x55f39ac0ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:01.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.264+0000 7fc968d04b80 1 --2- 172.21.15.143:0/382707408 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f39abf46f0 0x55f39ac24450 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:01.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.264+0000 7fc968d04b80 1 --2- 172.21.15.143:0/382707408 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55f39abf6c30 0x55f39ac24990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:01.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.264+0000 7fc968d04b80 1 -- 172.21.15.143:0/382707408 >> 172.21.15.143:0/382707408 conn(0x55f39ab6f660 msgr2=0x55f39abfa3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:01.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.264+0000 7fc968d04b80 1 -- 172.21.15.143:0/382707408 shutdown_connections 2023-11-08T19:26:01.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.264+0000 7fc968d04b80 1 -- 172.21.15.143:0/382707408 wait complete. 2023-11-08T19:26:01.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.264+0000 7fc968d04b80 1 -- start start 2023-11-08T19:26:01.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.264+0000 7fc968d04b80 1 Processor -- start 2023-11-08T19:26:01.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.265+0000 7fc968d04b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f39abf46f0 0x55f39ac0fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:01.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.265+0000 7fc968d04b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f39abf4ad0 0x55f39ac39f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:01.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.265+0000 7fc968d04b80 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55f39ac3a4a0 0x55f39ac37a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:01.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.265+0000 7fc968d04b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f39ab4d5c0 con 0x55f39abf46f0 2023-11-08T19:26:01.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.265+0000 7fc968d04b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f39aae90d0 con 0x55f39abf4ad0 2023-11-08T19:26:01.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.265+0000 7fc968d04b80 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f39aae8d70 con 0x55f39ac3a4a0 2023-11-08T19:26:01.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.265+0000 7fc968d04b80 10 client.0 _ll_register_callbacks cb 0x55f39a9457b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T19:26:01.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.265+0000 7fc95659c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f39abf46f0 0x55f39ac0fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:01.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.266+0000 7fc95659c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f39abf46f0 0x55f39ac0fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50812/0 (socket says 172.21.15.143:50812) 2023-11-08T19:26:01.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.266+0000 7fc95659c700 1 -- 172.21.15.143:0/3971842004 learned_addr learned my addr 172.21.15.143:0/3971842004 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:01.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.266+0000 7fc955d9b700 1 --2- 172.21.15.143:0/3971842004 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f39abf4ad0 0x55f39ac39f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:01.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.266+0000 7fc956d9d700 1 --2- 172.21.15.143:0/3971842004 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55f39ac3a4a0 0x55f39ac37a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:01.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.266+0000 7fc95659c700 1 -- 172.21.15.143:0/3971842004 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55f39ac3a4a0 msgr2=0x55f39ac37a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:01.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.266+0000 7fc95659c700 1 --2- 172.21.15.143:0/3971842004 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55f39ac3a4a0 0x55f39ac37a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:01.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.266+0000 7fc95659c700 1 -- 172.21.15.143:0/3971842004 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f39abf4ad0 msgr2=0x55f39ac39f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:01.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.266+0000 7fc95659c700 1 --2- 172.21.15.143:0/3971842004 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f39abf4ad0 0x55f39ac39f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:01.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.266+0000 7fc95659c700 1 -- 172.21.15.143:0/3971842004 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc958151eb0 con 0x55f39abf46f0 2023-11-08T19:26:01.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.266+0000 7fc955d9b700 1 --2- 172.21.15.143:0/3971842004 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f39abf4ad0 0x55f39ac39f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:26:01.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.266+0000 7fc95659c700 1 --2- 172.21.15.143:0/3971842004 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f39abf46f0 0x55f39ac0fd40 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fc94c058350 tx=0x7fc94c01d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c61682e5c780a284 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:01.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.266+0000 7fc9357fa700 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2023-11-08T19:26:01.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.266+0000 7fc9357fa700 1 -- 172.21.15.143:0/3971842004 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc94c054030 con 0x55f39abf46f0 2023-11-08T19:26:01.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.266+0000 7fc9357fa700 1 -- 172.21.15.143:0/3971842004 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc94c05fff0 con 0x55f39abf46f0 2023-11-08T19:26:01.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.266+0000 7fc968d04b80 1 -- 172.21.15.143:0/3971842004 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f39abec880 con 0x55f39abf46f0 2023-11-08T19:26:01.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.266+0000 7fc9357fa700 1 -- 172.21.15.143:0/3971842004 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc94c05fd70 con 0x55f39abf46f0 2023-11-08T19:26:01.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.266+0000 7fc968d04b80 10 client.5328 fetch_fsmap learned FSMap version 16 2023-11-08T19:26:01.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.266+0000 7fc968d04b80 1 -- 172.21.15.143:0/3971842004 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x55f39a976d90 con 0x55f39abf46f0 2023-11-08T19:26:01.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.266+0000 7fc9357fa700 1 -- 172.21.15.143:0/3971842004 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f39abec880 con 0x55f39abf46f0 2023-11-08T19:26:01.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.266+0000 7fc9357fa700 1 -- 172.21.15.143:0/3971842004 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 16) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc94c05cb80 con 0x55f39abf46f0 2023-11-08T19:26:01.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.266+0000 7fc968d04b80 10 client.5328 fetch_fsmap finished waiting for FSMap version 16 2023-11-08T19:26:01.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.266+0000 7fc968d04b80 10 client.5328 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-08T19:26:01.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.266+0000 7fc968d04b80 20 client.5328 populate_metadata read hostname 'smithi143' 2023-11-08T19:26:01.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.266+0000 7fc968d04b80 1 --2- 172.21.15.143:0/3971842004 >> [v2:172.21.15.143:6833/1574070101,v1:172.21.15.143:6835/1574070101] conn(0x55f39ac52710 0x55f39ac52af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:01.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.266+0000 7fc968d04b80 4 client.5328 mds_command: new command op to 5109 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-08T19:26:01.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.266+0000 7fc968d04b80 1 -- 172.21.15.143:0/3971842004 --> [v2:172.21.15.143:6833/1574070101,v1:172.21.15.143:6835/1574070101] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f39abc8a30 con 0x55f39ac52710 2023-11-08T19:26:01.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.267+0000 7fc955d9b700 1 --2- 172.21.15.143:0/3971842004 >> [v2:172.21.15.143:6833/1574070101,v1:172.21.15.143:6835/1574070101] conn(0x55f39ac52710 0x55f39ac52af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:01.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.267+0000 7fc9357fa700 10 client.5328 ms_handle_connect on v2:172.21.15.143:6833/1574070101 2023-11-08T19:26:01.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.267+0000 7fc955d9b700 1 --2- 172.21.15.143:0/3971842004 >> [v2:172.21.15.143:6833/1574070101,v1:172.21.15.143:6835/1574070101] conn(0x55f39ac52710 0x55f39ac52af0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5a6f1a1afeb2a79 server_cookie=d417ec758cede482 in_seq=0 out_seq=0 2023-11-08T19:26:01.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.267+0000 7fc9357fa700 1 -- 172.21.15.143:0/3971842004 <== mds.0 v2:172.21.15.143:6833/1574070101 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f39abc8a30 con 0x55f39ac52710 2023-11-08T19:26:01.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.267+0000 7fc9357fa700 10 client.5328 handle_command_reply: tid=0 2023-11-08T19:26:01.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.267+0000 7fc968d04b80 1 client.5328 shutdown 2023-11-08T19:26:01.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.268+0000 7fc968d04b80 1 -- 172.21.15.143:0/3971842004 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f39abf46f0 msgr2=0x55f39ac0fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:01.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.268+0000 7fc968d04b80 1 --2- 172.21.15.143:0/3971842004 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f39abf46f0 0x55f39ac0fd40 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fc94c058350 tx=0x7fc94c01d000 comp rx=0 tx=0).stop 2023-11-08T19:26:01.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.268+0000 7fc968d04b80 1 -- 172.21.15.143:0/3971842004 shutdown_connections 2023-11-08T19:26:01.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.268+0000 7fc968d04b80 1 --2- 172.21.15.143:0/3971842004 >> [v2:172.21.15.143:6833/1574070101,v1:172.21.15.143:6835/1574070101] conn(0x55f39ac52710 0x55f39ac52af0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:01.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.268+0000 7fc968d04b80 1 --2- 172.21.15.143:0/3971842004 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55f39ac3a4a0 0x55f39ac37a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:01.293 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.268+0000 7fc968d04b80 1 --2- 172.21.15.143:0/3971842004 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f39abf46f0 0x55f39ac0fd40 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:01.293 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.268+0000 7fc968d04b80 1 --2- 172.21.15.143:0/3971842004 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f39abf4ad0 0x55f39ac39f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:01.293 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.268+0000 7fc9357fa700 0 client.5328 ms_handle_reset on v2:172.21.15.143:6833/1574070101 2023-11-08T19:26:01.293 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.268+0000 7fc968d04b80 1 -- 172.21.15.143:0/3971842004 >> 172.21.15.143:0/3971842004 conn(0x55f39ab6f660 msgr2=0x55f39ac248e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:01.293 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.268+0000 7fc968d04b80 1 -- 172.21.15.143:0/3971842004 shutdown_connections 2023-11-08T19:26:01.294 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.268+0000 7fc968d04b80 1 -- 172.21.15.143:0/3971842004 wait complete. 2023-11-08T19:26:01.294 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.268+0000 7fc968d04b80 20 client.5328 trim_cache size 0 max 16384 2023-11-08T19:26:01.300 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.286+0000 7fc968d04b80 1 Processor -- start 2023-11-08T19:26:01.300 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.286+0000 7fc968d04b80 1 -- start start 2023-11-08T19:26:01.300 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.286+0000 7fc968d04b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f39abf46f0 0x55f39ac21c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:01.300 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.286+0000 7fc968d04b80 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55f39abf4ad0 0x55f39ac39a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:01.301 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.286+0000 7fc968d04b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f39ac3a4a0 0x55f39abf3500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:01.301 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.286+0000 7fc968d04b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f39a9cc8d0 con 0x55f39ac3a4a0 2023-11-08T19:26:01.301 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.286+0000 7fc968d04b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f39ac521c0 con 0x55f39abf46f0 2023-11-08T19:26:01.301 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.286+0000 7fc968d04b80 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f39ac43950 con 0x55f39abf4ad0 2023-11-08T19:26:01.301 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.286+0000 7fc956d9d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f39ac3a4a0 0x55f39abf3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:01.302 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.286+0000 7fc956d9d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f39ac3a4a0 0x55f39abf3500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50822/0 (socket says 172.21.15.143:50822) 2023-11-08T19:26:01.302 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.287+0000 7fc956d9d700 1 -- 172.21.15.143:0/666562313 learned_addr learned my addr 172.21.15.143:0/666562313 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:01.302 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.287+0000 7fc955d9b700 1 --2- 172.21.15.143:0/666562313 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55f39abf4ad0 0x55f39ac39a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:01.302 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.287+0000 7fc95659c700 1 --2- 172.21.15.143:0/666562313 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f39abf46f0 0x55f39ac21c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:01.302 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.287+0000 7fc956d9d700 1 -- 172.21.15.143:0/666562313 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55f39abf4ad0 msgr2=0x55f39ac39a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:01.303 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.287+0000 7fc956d9d700 1 --2- 172.21.15.143:0/666562313 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55f39abf4ad0 0x55f39ac39a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:01.303 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.287+0000 7fc956d9d700 1 -- 172.21.15.143:0/666562313 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f39abf46f0 msgr2=0x55f39ac21c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:01.303 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.287+0000 7fc956d9d700 1 --2- 172.21.15.143:0/666562313 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f39abf46f0 0x55f39ac21c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:01.303 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.287+0000 7fc956d9d700 1 -- 172.21.15.143:0/666562313 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f39a95fc10 con 0x55f39ac3a4a0 2023-11-08T19:26:01.303 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.287+0000 7fc95659c700 1 --2- 172.21.15.143:0/666562313 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f39abf46f0 0x55f39ac21c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:26:01.304 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.287+0000 7fc956d9d700 1 --2- 172.21.15.143:0/666562313 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f39ac3a4a0 0x55f39abf3500 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fc950009d90 tx=0x7fc950013530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=156f32b319169861 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:01.304 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.287+0000 7fc944ff9700 1 -- 172.21.15.143:0/666562313 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc950013cf0 con 0x55f39ac3a4a0 2023-11-08T19:26:01.304 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.287+0000 7fc944ff9700 1 -- 172.21.15.143:0/666562313 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc95000f040 con 0x55f39ac3a4a0 2023-11-08T19:26:01.304 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.287+0000 7fc944ff9700 1 -- 172.21.15.143:0/666562313 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc950003c60 con 0x55f39ac3a4a0 2023-11-08T19:26:01.304 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.288+0000 7fc968d04b80 1 -- 172.21.15.143:0/666562313 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f39ac3a4a0 msgr2=0x55f39abf3500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:01.305 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.288+0000 7fc968d04b80 1 --2- 172.21.15.143:0/666562313 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f39ac3a4a0 0x55f39abf3500 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fc950009d90 tx=0x7fc950013530 comp rx=0 tx=0).stop 2023-11-08T19:26:01.305 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.288+0000 7fc968d04b80 1 -- 172.21.15.143:0/666562313 shutdown_connections 2023-11-08T19:26:01.305 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.288+0000 7fc968d04b80 1 --2- 172.21.15.143:0/666562313 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f39ac3a4a0 0x55f39abf3500 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:01.305 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.288+0000 7fc968d04b80 1 --2- 172.21.15.143:0/666562313 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f39abf46f0 0x55f39ac21c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:01.305 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.288+0000 7fc968d04b80 1 --2- 172.21.15.143:0/666562313 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55f39abf4ad0 0x55f39ac39a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:01.305 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.288+0000 7fc968d04b80 1 -- 172.21.15.143:0/666562313 >> 172.21.15.143:0/666562313 conn(0x55f39ab6f660 msgr2=0x55f39abf07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:01.306 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.288+0000 7fc968d04b80 1 -- 172.21.15.143:0/666562313 shutdown_connections 2023-11-08T19:26:01.306 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.288+0000 7fc968d04b80 1 -- 172.21.15.143:0/666562313 wait complete. 2023-11-08T19:26:01.306 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.288+0000 7fc968d04b80 1 -- start start 2023-11-08T19:26:01.306 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.288+0000 7fc968d04b80 1 Processor -- start 2023-11-08T19:26:01.307 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.290+0000 7fc968d04b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f39abf46f0 0x55f39ac38940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:01.307 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.290+0000 7fc968d04b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f39abf4ad0 0x55f39abf57a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:01.307 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.290+0000 7fc968d04b80 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55f39ac3a4a0 0x55f39ac4db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:01.307 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.290+0000 7fc968d04b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f39ac39350 con 0x55f39abf4ad0 2023-11-08T19:26:01.307 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.290+0000 7fc968d04b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f39ac51d60 con 0x55f39abf46f0 2023-11-08T19:26:01.307 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.290+0000 7fc968d04b80 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f39aca95d0 con 0x55f39ac3a4a0 2023-11-08T19:26:01.308 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.290+0000 7fc968d04b80 10 client.0 _ll_register_callbacks cb 0x7fc94c05b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T19:26:01.308 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.290+0000 7fc955d9b700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f39abf4ad0 0x55f39abf57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:01.308 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.290+0000 7fc955d9b700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f39abf4ad0 0x55f39abf57a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50828/0 (socket says 172.21.15.143:50828) 2023-11-08T19:26:01.308 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.290+0000 7fc955d9b700 1 -- 172.21.15.143:0/3123648971 learned_addr learned my addr 172.21.15.143:0/3123648971 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:01.308 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.290+0000 7fc956d9d700 1 --2- 172.21.15.143:0/3123648971 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55f39ac3a4a0 0x55f39ac4db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:01.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.290+0000 7fc95659c700 1 --2- 172.21.15.143:0/3123648971 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f39abf46f0 0x55f39ac38940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:01.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.290+0000 7fc955d9b700 1 -- 172.21.15.143:0/3123648971 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55f39ac3a4a0 msgr2=0x55f39ac4db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:01.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.290+0000 7fc955d9b700 1 --2- 172.21.15.143:0/3123648971 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55f39ac3a4a0 0x55f39ac4db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:01.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.290+0000 7fc955d9b700 1 -- 172.21.15.143:0/3123648971 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f39abf46f0 msgr2=0x55f39ac38940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:01.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.290+0000 7fc955d9b700 1 --2- 172.21.15.143:0/3123648971 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f39abf46f0 0x55f39ac38940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:01.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.290+0000 7fc955d9b700 1 -- 172.21.15.143:0/3123648971 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9500099c0 con 0x55f39abf4ad0 2023-11-08T19:26:01.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.290+0000 7fc955d9b700 1 --2- 172.21.15.143:0/3123648971 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f39abf4ad0 0x55f39abf57a0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fc948013000 tx=0x7fc9480047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0e6f650d2d16910 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:01.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.290+0000 7fc9357fa700 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2023-11-08T19:26:01.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.290+0000 7fc9357fa700 1 -- 172.21.15.143:0/3123648971 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9480327c0 con 0x55f39abf4ad0 2023-11-08T19:26:01.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.290+0000 7fc9357fa700 1 -- 172.21.15.143:0/3123648971 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc9480036a0 con 0x55f39abf4ad0 2023-11-08T19:26:01.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.290+0000 7fc968d04b80 1 -- 172.21.15.143:0/3123648971 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc94c056090 con 0x55f39abf4ad0 2023-11-08T19:26:01.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.290+0000 7fc9357fa700 1 -- 172.21.15.143:0/3123648971 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc948032920 con 0x55f39abf4ad0 2023-11-08T19:26:01.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.290+0000 7fc968d04b80 10 client.5334 fetch_fsmap learned FSMap version 16 2023-11-08T19:26:01.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.291+0000 7fc968d04b80 1 -- 172.21.15.143:0/3123648971 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x55f39a99d200 con 0x55f39abf4ad0 2023-11-08T19:26:01.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.291+0000 7fc9357fa700 1 -- 172.21.15.143:0/3123648971 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc94c056090 con 0x55f39abf4ad0 2023-11-08T19:26:01.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.291+0000 7fc9357fa700 1 -- 172.21.15.143:0/3123648971 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 16) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc948003860 con 0x55f39abf4ad0 2023-11-08T19:26:01.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.291+0000 7fc968d04b80 10 client.5334 fetch_fsmap finished waiting for FSMap version 16 2023-11-08T19:26:01.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.291+0000 7fc968d04b80 10 client.5334 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-08T19:26:01.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.291+0000 7fc968d04b80 20 client.5334 populate_metadata read hostname 'smithi143' 2023-11-08T19:26:01.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.291+0000 7fc968d04b80 1 --2- 172.21.15.143:0/3123648971 >> [v2:172.21.15.143:6833/1574070101,v1:172.21.15.143:6835/1574070101] conn(0x55f39ac4e0b0 0x55f39ac34de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:01.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.291+0000 7fc968d04b80 4 client.5334 mds_command: new command op to 5109 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-08T19:26:01.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.291+0000 7fc968d04b80 1 -- 172.21.15.143:0/3123648971 --> [v2:172.21.15.143:6833/1574070101,v1:172.21.15.143:6835/1574070101] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55f39ab465a0 con 0x55f39ac4e0b0 2023-11-08T19:26:01.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.291+0000 7fc95659c700 1 --2- 172.21.15.143:0/3123648971 >> [v2:172.21.15.143:6833/1574070101,v1:172.21.15.143:6835/1574070101] conn(0x55f39ac4e0b0 0x55f39ac34de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:01.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.291+0000 7fc9357fa700 10 client.5334 ms_handle_connect on v2:172.21.15.143:6833/1574070101 2023-11-08T19:26:01.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.291+0000 7fc95659c700 1 --2- 172.21.15.143:0/3123648971 >> [v2:172.21.15.143:6833/1574070101,v1:172.21.15.143:6835/1574070101] conn(0x55f39ac4e0b0 0x55f39ac34de0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cff080732d436006 server_cookie=36fd8bce23d58f40 in_seq=0 out_seq=0 2023-11-08T19:26:01.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.291+0000 7fc9357fa700 1 -- 172.21.15.143:0/3123648971 <== mds.0 v2:172.21.15.143:6833/1574070101 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+319 (crc 0 0 0) 0x55f39ab465a0 con 0x55f39ac4e0b0 2023-11-08T19:26:01.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.291+0000 7fc9357fa700 10 client.5334 handle_command_reply: tid=0 2023-11-08T19:26:01.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.291+0000 7fc968d04b80 1 client.5334 shutdown 2023-11-08T19:26:01.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.291+0000 7fc968d04b80 1 -- 172.21.15.143:0/3123648971 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f39abf4ad0 msgr2=0x55f39abf57a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:01.315 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.291+0000 7fc968d04b80 1 --2- 172.21.15.143:0/3123648971 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f39abf4ad0 0x55f39abf57a0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fc948013000 tx=0x7fc9480047c0 comp rx=0 tx=0).stop 2023-11-08T19:26:01.315 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.292+0000 7fc968d04b80 1 -- 172.21.15.143:0/3123648971 shutdown_connections 2023-11-08T19:26:01.315 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.292+0000 7fc968d04b80 1 --2- 172.21.15.143:0/3123648971 >> [v2:172.21.15.143:6833/1574070101,v1:172.21.15.143:6835/1574070101] conn(0x55f39ac4e0b0 0x55f39ac34de0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:01.315 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.292+0000 7fc968d04b80 1 --2- 172.21.15.143:0/3123648971 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55f39ac3a4a0 0x55f39ac4db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:01.315 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.292+0000 7fc9357fa700 0 client.5334 ms_handle_reset on v2:172.21.15.143:6833/1574070101 2023-11-08T19:26:01.316 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.292+0000 7fc968d04b80 1 --2- 172.21.15.143:0/3123648971 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f39abf46f0 0x55f39ac38940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:01.316 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.292+0000 7fc968d04b80 1 --2- 172.21.15.143:0/3123648971 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f39abf4ad0 0x55f39abf57a0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:01.316 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.292+0000 7fc968d04b80 1 -- 172.21.15.143:0/3123648971 >> 172.21.15.143:0/3123648971 conn(0x55f39ab6f660 msgr2=0x55f39abf88f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:01.316 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.292+0000 7fc968d04b80 1 -- 172.21.15.143:0/3123648971 shutdown_connections 2023-11-08T19:26:01.316 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.292+0000 7fc968d04b80 1 -- 172.21.15.143:0/3123648971 wait complete. 2023-11-08T19:26:01.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.292+0000 7fc968d04b80 20 client.5334 trim_cache size 0 max 16384 2023-11-08T19:26:01.317 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2023-11-08T19:26:01.317 INFO:teuthology.orchestra.run.smithi143.stdout: "cluster_fsid": "968483af-77fc-4068-ad7c-5c2a1a4613d7", 2023-11-08T19:26:01.318 INFO:teuthology.orchestra.run.smithi143.stdout: "whoami": 0, 2023-11-08T19:26:01.318 INFO:teuthology.orchestra.run.smithi143.stdout: "id": 5109, 2023-11-08T19:26:01.318 INFO:teuthology.orchestra.run.smithi143.stdout: "want_state": "up:active", 2023-11-08T19:26:01.318 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "up:active", 2023-11-08T19:26:01.318 INFO:teuthology.orchestra.run.smithi143.stdout: "fs_name": "cephfs", 2023-11-08T19:26:01.319 INFO:teuthology.orchestra.run.smithi143.stdout: "rank_uptime": 10.417876174, 2023-11-08T19:26:01.319 INFO:teuthology.orchestra.run.smithi143.stdout: "mdsmap_epoch": 16, 2023-11-08T19:26:01.319 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_epoch": 33, 2023-11-08T19:26:01.319 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_epoch_barrier": 33, 2023-11-08T19:26:01.319 INFO:teuthology.orchestra.run.smithi143.stdout: "uptime": 19.307251468 2023-11-08T19:26:01.320 INFO:teuthology.orchestra.run.smithi143.stdout:} 2023-11-08T19:26:01.321 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.292+0000 7fc95cb0b700 1 -- 172.21.15.143:0/4213769252 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc93806af00 msgr2=0x7fc93806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:01.321 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.292+0000 7fc95cb0b700 1 --2- 172.21.15.143:0/4213769252 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc93806af00 0x7fc93806d3c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc948000c00 tx=0x7fc948008030 comp rx=0 tx=0).stop 2023-11-08T19:26:01.322 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.292+0000 7fc95cb0b700 1 -- 172.21.15.143:0/4213769252 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc95806b150 msgr2=0x7fc9580c5ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:01.322 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.292+0000 7fc95cb0b700 1 --2- 172.21.15.143:0/4213769252 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc95806b150 0x7fc9580c5ea0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fc94c00e730 tx=0x7fc94c00eaf0 comp rx=0 tx=0).stop 2023-11-08T19:26:01.322 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.292+0000 7fc95cb0b700 1 -- 172.21.15.143:0/4213769252 shutdown_connections 2023-11-08T19:26:01.322 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.292+0000 7fc95cb0b700 1 --2- 172.21.15.143:0/4213769252 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc93806af00 0x7fc93806d3c0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:01.322 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.292+0000 7fc95cb0b700 1 --2- 172.21.15.143:0/4213769252 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc9581618f0 0x7fc9580b26b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:01.323 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.292+0000 7fc95cb0b700 1 --2- 172.21.15.143:0/4213769252 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc95806b150 0x7fc9580c5ea0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:01.323 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.292+0000 7fc95cb0b700 1 --2- 172.21.15.143:0/4213769252 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc95815dd00 0x7fc9580c77d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:01.323 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.292+0000 7fc95cb0b700 1 -- 172.21.15.143:0/4213769252 >> 172.21.15.143:0/4213769252 conn(0x7fc958099ab0 msgr2=0x7fc9580cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:01.323 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.292+0000 7fc95cb0b700 1 -- 172.21.15.143:0/4213769252 shutdown_connections 2023-11-08T19:26:01.323 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.292+0000 7fc95cb0b700 1 -- 172.21.15.143:0/4213769252 wait complete. 2023-11-08T19:26:01.324 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-08T19:26:01.445 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.443+0000 7f39fec58700 1 Processor -- start 2023-11-08T19:26:01.445 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.443+0000 7f39fec58700 1 -- start start 2023-11-08T19:26:01.447 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.444+0000 7f39fec58700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f39f80cf090 0x7f39f80d1f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:01.447 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.444+0000 7f39fec58700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f39f80d3690 0x7f39f80b9690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:01.448 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.444+0000 7f39fec58700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39f80d3200 con 0x7f39f815dc80 2023-11-08T19:26:01.448 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.444+0000 7f39fec58700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39f80b9bd0 con 0x7f39f80d3690 2023-11-08T19:26:01.448 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.444+0000 7f39fec58700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39f80b7cf0 con 0x7f39f80cf090 2023-11-08T19:26:01.448 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.444+0000 7f39fd1f5700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f39f80d3690 0x7f39f80b9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:01.448 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.444+0000 7f39fd1f5700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f39f80d3690 0x7f39f80b9690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50838/0 (socket says 172.21.15.143:50838) 2023-11-08T19:26:01.449 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.444+0000 7f39fd1f5700 1 -- 172.21.15.143:0/107390118 learned_addr learned my addr 172.21.15.143:0/107390118 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:01.449 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.444+0000 7f39f7fff700 1 --2- 172.21.15.143:0/107390118 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f39f80cf090 0x7f39f80d1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:01.449 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.445+0000 7f39fd1f5700 1 -- 172.21.15.143:0/107390118 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f39f80cf090 msgr2=0x7f39f80d1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:01.449 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.445+0000 7f39fd1f5700 1 --2- 172.21.15.143:0/107390118 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f39f80cf090 0x7f39f80d1f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:01.449 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.445+0000 7f39fd1f5700 1 -- 172.21.15.143:0/107390118 >> v1:172.21.15.174:6789/0 conn(0x7f39f815dc80 legacy=0x7f39f80ce980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:01.450 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.445+0000 7f39fd1f5700 1 -- 172.21.15.143:0/107390118 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39f815e060 con 0x7f39f80d3690 2023-11-08T19:26:01.450 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.445+0000 7f39fd1f5700 1 --2- 172.21.15.143:0/107390118 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f39f80d3690 0x7f39f80b9690 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f39f0009e50 tx=0x7f39f000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ec24f1412a423b2 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:01.450 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.445+0000 7f39f77fe700 1 -- 172.21.15.143:0/107390118 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f000e070 con 0x7f39f80d3690 2023-11-08T19:26:01.450 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.445+0000 7f39f77fe700 1 -- 172.21.15.143:0/107390118 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39f000b910 con 0x7f39f80d3690 2023-11-08T19:26:01.450 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.445+0000 7f39f77fe700 1 -- 172.21.15.143:0/107390118 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f000bcc0 con 0x7f39f80d3690 2023-11-08T19:26:01.451 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.445+0000 7f39fec58700 1 -- 172.21.15.143:0/107390118 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f39f80d3690 msgr2=0x7f39f80b9690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:01.451 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.445+0000 7f39fec58700 1 --2- 172.21.15.143:0/107390118 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f39f80d3690 0x7f39f80b9690 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f39f0009e50 tx=0x7f39f000b330 comp rx=0 tx=0).stop 2023-11-08T19:26:01.451 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.445+0000 7f39fec58700 1 -- 172.21.15.143:0/107390118 shutdown_connections 2023-11-08T19:26:01.451 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.445+0000 7f39fec58700 1 --2- 172.21.15.143:0/107390118 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f39f80d3690 0x7f39f80b9690 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:01.451 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.445+0000 7f39fec58700 1 --2- 172.21.15.143:0/107390118 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f39f80cf090 0x7f39f80d1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:01.452 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.445+0000 7f39fec58700 1 -- 172.21.15.143:0/107390118 >> 172.21.15.143:0/107390118 conn(0x7f39f80668e0 msgr2=0x7f39f8066cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:01.452 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.446+0000 7f39fec58700 1 -- 172.21.15.143:0/107390118 shutdown_connections 2023-11-08T19:26:01.452 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.446+0000 7f39fec58700 1 -- 172.21.15.143:0/107390118 wait complete. 2023-11-08T19:26:01.455 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.447+0000 7f39fec58700 1 Processor -- start 2023-11-08T19:26:01.455 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.447+0000 7f39fec58700 1 -- start start 2023-11-08T19:26:01.455 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.448+0000 7f39fec58700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f39f80cf090 0x7f39f80c9090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:01.455 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.448+0000 7f39fec58700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f39f80d3690 0x7f39f81f2a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:01.455 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.448+0000 7f39fec58700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f39f815dc80 0x7f39f80c76a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:01.456 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.448+0000 7f39fec58700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39f81f2fc0 con 0x7f39f815dc80 2023-11-08T19:26:01.456 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.448+0000 7f39fec58700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39f80cb290 con 0x7f39f80cf090 2023-11-08T19:26:01.456 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.448+0000 7f39fec58700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39f80cb570 con 0x7f39f80d3690 2023-11-08T19:26:01.456 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.448+0000 7f39fd1f5700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f39f815dc80 0x7f39f80c76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:01.456 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.448+0000 7f39fd1f5700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f39f815dc80 0x7f39f80c76a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50844/0 (socket says 172.21.15.143:50844) 2023-11-08T19:26:01.457 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.448+0000 7f39fd1f5700 1 -- 172.21.15.143:0/431090149 learned_addr learned my addr 172.21.15.143:0/431090149 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:01.457 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.448+0000 7f39fc9f4700 1 --2- 172.21.15.143:0/431090149 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f39f80cf090 0x7f39f80c9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:01.457 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.448+0000 7f39f7fff700 1 --2- 172.21.15.143:0/431090149 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f39f80d3690 0x7f39f81f2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:01.457 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.449+0000 7f39fd1f5700 1 -- 172.21.15.143:0/431090149 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f39f80d3690 msgr2=0x7f39f81f2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:01.457 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.449+0000 7f39fd1f5700 1 --2- 172.21.15.143:0/431090149 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f39f80d3690 0x7f39f81f2a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:01.458 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.449+0000 7f39fd1f5700 1 -- 172.21.15.143:0/431090149 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f39f80cf090 msgr2=0x7f39f80c9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:01.458 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.449+0000 7f39fd1f5700 1 --2- 172.21.15.143:0/431090149 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f39f80cf090 0x7f39f80c9090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:01.458 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.449+0000 7f39fd1f5700 1 -- 172.21.15.143:0/431090149 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39f0009b30 con 0x7f39f815dc80 2023-11-08T19:26:01.458 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.449+0000 7f39fd1f5700 1 --2- 172.21.15.143:0/431090149 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f39f815dc80 0x7f39f80c76a0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f39f0012000 tx=0x7f39f0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:01.458 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.449+0000 7f39f57fa700 1 -- 172.21.15.143:0/431090149 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f0004800 con 0x7f39f815dc80 2023-11-08T19:26:01.459 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.449+0000 7f39f57fa700 1 -- 172.21.15.143:0/431090149 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39f0004960 con 0x7f39f815dc80 2023-11-08T19:26:01.459 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.449+0000 7f39fec58700 1 -- 172.21.15.143:0/431090149 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39f80cb850 con 0x7f39f815dc80 2023-11-08T19:26:01.459 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.449+0000 7f39f57fa700 1 -- 172.21.15.143:0/431090149 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f001c680 con 0x7f39f815dc80 2023-11-08T19:26:01.459 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.449+0000 7f39fec58700 1 -- 172.21.15.143:0/431090149 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39f80bda00 con 0x7f39f815dc80 2023-11-08T19:26:01.459 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.450+0000 7f39fec58700 1 -- 172.21.15.143:0/431090149 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39f80cb3d0 con 0x7f39f815dc80 2023-11-08T19:26:01.460 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.450+0000 7f39f57fa700 1 -- 172.21.15.143:0/431090149 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f39f001c7e0 con 0x7f39f815dc80 2023-11-08T19:26:01.460 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.450+0000 7f39f57fa700 1 --2- 172.21.15.143:0/431090149 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f39dc06af30 0x7f39dc06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:01.460 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.451+0000 7f39f57fa700 1 -- 172.21.15.143:0/431090149 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f39f008ad20 con 0x7f39f815dc80 2023-11-08T19:26:01.460 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.451+0000 7f39fc9f4700 1 --2- 172.21.15.143:0/431090149 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f39dc06af30 0x7f39dc06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:01.460 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.451+0000 7f39fc9f4700 1 --2- 172.21.15.143:0/431090149 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f39dc06af30 0x7f39dc06d3f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f39ec007fd0 tx=0x7f39ec009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:01.461 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.455+0000 7f39f57fa700 1 -- 172.21.15.143:0/431090149 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f39f80cb3d0 con 0x7f39f815dc80 2023-11-08T19:26:01.691 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.689+0000 7f39fec58700 1 -- 172.21.15.143:0/431090149 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f39f80cbd70 con 0x7f39f815dc80 2023-11-08T19:26:01.691 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.689+0000 7f39f57fa700 1 -- 172.21.15.143:0/431090149 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) v1 ==== 53+0+73 (secure 0 0 0) 0x7f39f80cbd70 con 0x7f39f815dc80 2023-11-08T19:26:01.691 INFO:teuthology.orchestra.run.smithi143.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-08T19:26:01.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.691+0000 7f39fec58700 1 -- 172.21.15.143:0/431090149 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f39dc06af30 msgr2=0x7f39dc06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:01.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.691+0000 7f39fec58700 1 --2- 172.21.15.143:0/431090149 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f39dc06af30 0x7f39dc06d3f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f39ec007fd0 tx=0x7f39ec009040 comp rx=0 tx=0).stop 2023-11-08T19:26:01.694 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.691+0000 7f39fec58700 1 -- 172.21.15.143:0/431090149 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f39f815dc80 msgr2=0x7f39f80c76a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:01.694 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.691+0000 7f39fec58700 1 --2- 172.21.15.143:0/431090149 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f39f815dc80 0x7f39f80c76a0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f39f0012000 tx=0x7f39f0004ad0 comp rx=0 tx=0).stop 2023-11-08T19:26:01.694 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.692+0000 7f39fec58700 1 -- 172.21.15.143:0/431090149 shutdown_connections 2023-11-08T19:26:01.694 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.692+0000 7f39fec58700 1 --2- 172.21.15.143:0/431090149 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f39dc06af30 0x7f39dc06d3f0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:01.694 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.692+0000 7f39fec58700 1 --2- 172.21.15.143:0/431090149 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f39f815dc80 0x7f39f80c76a0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:01.695 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.692+0000 7f39fec58700 1 --2- 172.21.15.143:0/431090149 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f39f80cf090 0x7f39f80c9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:01.695 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.692+0000 7f39fec58700 1 --2- 172.21.15.143:0/431090149 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f39f80d3690 0x7f39f81f2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:01.695 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.692+0000 7f39fec58700 1 -- 172.21.15.143:0/431090149 >> 172.21.15.143:0/431090149 conn(0x7f39f80668e0 msgr2=0x7f39f815b900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:01.695 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.692+0000 7f39fec58700 1 -- 172.21.15.143:0/431090149 shutdown_connections 2023-11-08T19:26:01.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:01.692+0000 7f39fec58700 1 -- 172.21.15.143:0/431090149 wait complete. 2023-11-08T19:26:01.705 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-08T19:26:01.705 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-08T19:26:01.706 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi143.front.sepia.ceph.com 2023-11-08T19:26:01.706 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-08T19:26:01.706 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-08T19:26:01.707 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-08T19:26:01.707 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-08T19:26:01.707 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-08T19:26:01.708 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-08T19:26:01.708 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:26:01.708 DEBUG:teuthology.orchestra.run.smithi143:> ip addr 2023-11-08T19:26:01.726 INFO:teuthology.orchestra.run.smithi143.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-08T19:26:01.726 INFO:teuthology.orchestra.run.smithi143.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-08T19:26:01.726 INFO:teuthology.orchestra.run.smithi143.stdout: inet 127.0.0.1/8 scope host lo 2023-11-08T19:26:01.726 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:26:01.727 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 ::1/128 scope host 2023-11-08T19:26:01.727 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:26:01.727 INFO:teuthology.orchestra.run.smithi143.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-08T19:26:01.727 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:d9:3b:94 brd ff:ff:ff:ff:ff:ff 2023-11-08T19:26:01.727 INFO:teuthology.orchestra.run.smithi143.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-08T19:26:01.728 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:d9:3b:95 brd ff:ff:ff:ff:ff:ff 2023-11-08T19:26:01.728 INFO:teuthology.orchestra.run.smithi143.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-08T19:26:01.728 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:88:77:82 brd ff:ff:ff:ff:ff:ff 2023-11-08T19:26:01.728 INFO:teuthology.orchestra.run.smithi143.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-08T19:26:01.728 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:88:77:83 brd ff:ff:ff:ff:ff:ff 2023-11-08T19:26:01.729 INFO:teuthology.orchestra.run.smithi143.stdout: inet 172.21.15.143/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-08T19:26:01.729 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft 42305sec preferred_lft 42305sec 2023-11-08T19:26:01.729 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 fe80::e628:d259:2f76:52a6/64 scope link noprefixroute 2023-11-08T19:26:01.729 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:26:01.729 INFO:teuthology.orchestra.run.smithi143.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-08T19:26:01.729 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 46:e3:3e:66:77:ea brd ff:ff:ff:ff:ff:ff 2023-11-08T19:26:01.730 INFO:teuthology.orchestra.run.smithi143.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-08T19:26:01.730 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:26:01.730 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 fe80::749a:8aff:fe59:325c/64 scope link 2023-11-08T19:26:01.730 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:26:01.731 INFO:teuthology.orchestra.run.smithi143.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-08T19:26:01.731 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 46:e3:3e:66:77:ea brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-08T19:26:01.732 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 fe80::44e3:3eff:fe66:77ea/64 scope link 2023-11-08T19:26:01.732 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:26:01.732 INFO:teuthology.orchestra.run.smithi143.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-08T19:26:01.732 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether ba:92:23:3f:5e:c8 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-08T19:26:01.732 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 fe80::b892:23ff:fe3f:5ec8/64 scope link 2023-11-08T19:26:01.733 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:26:01.733 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:26:01.733 DEBUG:teuthology.orchestra.run.smithi143:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-08T19:26:01.791 INFO:teuthology.orchestra.run.smithi143.stdout:1 2023-11-08T19:26:01.793 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:26:01.793 DEBUG:teuthology.orchestra.run.smithi143:> route 2023-11-08T19:26:01.852 INFO:teuthology.orchestra.run.smithi143.stdout:Kernel IP routing table 2023-11-08T19:26:01.852 INFO:teuthology.orchestra.run.smithi143.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-08T19:26:01.852 INFO:teuthology.orchestra.run.smithi143.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-08T19:26:01.853 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-08T19:26:01.853 INFO:teuthology.orchestra.run.smithi143.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-08T19:26:01.853 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:26:01.854 DEBUG:teuthology.orchestra.run.smithi143:> (cd / && exec bash -c ' 2023-11-08T19:26:01.854 DEBUG:teuthology.orchestra.run.smithi143:> set -e 2023-11-08T19:26:01.854 DEBUG:teuthology.orchestra.run.smithi143:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-08T19:26:01.855 DEBUG:teuthology.orchestra.run.smithi143:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-08T19:26:01.855 DEBUG:teuthology.orchestra.run.smithi143:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-08T19:26:01.855 DEBUG:teuthology.orchestra.run.smithi143:> ') 2023-11-08T19:26:01.956 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:26:01.957 DEBUG:teuthology.orchestra.run.smithi143:> ip netns list 2023-11-08T19:26:01.974 INFO:teuthology.orchestra.run.smithi143.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-08T19:26:01.974 INFO:teuthology.orchestra.run.smithi143.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-08T19:26:01.975 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-08T19:26:01.975 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:26:01.975 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip link set brx.0 up 2023-11-08T19:26:02.041 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-08T19:26:02.042 DEBUG:teuthology.orchestra.run.smithi143:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-08T19:26:02.098 INFO:teuthology.orchestra.run.smithi143.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-08T19:26:02.099 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-08T19:26:02.099 DEBUG:teuthology.orchestra.run.smithi143:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-08T19:26:02.154 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-11-08T19:26:02.155 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:26:02.155 DEBUG:teuthology.orchestra.run.smithi143:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-08T19:26:02.239 INFO:teuthology.orchestra.run.smithi143.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-08T19:26:02.239 INFO:teuthology.orchestra.run.smithi143.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-11-08T19:26:02.240 INFO:teuthology.orchestra.run.smithi143.stdout:invalid new device string format 2023-11-08T19:26:02.248 INFO:teuthology.orchestra.run.smithi143.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-08T19:26:02.248 INFO:teuthology.orchestra.run.smithi143.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-11-08T19:26:02.249 INFO:teuthology.orchestra.run.smithi143.stdout:invalid new device string format 2023-11-08T19:26:02.249 INFO:teuthology.orchestra.run.smithi143.stdout:mount.ceph: resolved to: "172.21.15.143:3300,172.21.15.174:3300,172.21.15.174:3301" 2023-11-08T19:26:02.249 INFO:teuthology.orchestra.run.smithi143.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-08T19:26:02.323 INFO:teuthology.orchestra.run.smithi143.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-08T19:26:02.323 INFO:teuthology.orchestra.run.smithi143.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-11-08T19:26:02.323 INFO:teuthology.orchestra.run.smithi143.stdout:invalid new device string format 2023-11-08T19:26:02.323 INFO:teuthology.orchestra.run.smithi143.stdout:mount.ceph: resolved to: "172.21.15.143:3300,172.21.15.174:3300,172.21.15.174:3301" 2023-11-08T19:26:02.324 INFO:teuthology.orchestra.run.smithi143.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-08T19:26:02.324 INFO:teuthology.orchestra.run.smithi143.stdout:mount.ceph: resolved to: "172.21.15.143:6789,172.21.15.174:6789,172.21.15.174:6790" 2023-11-08T19:26:02.324 INFO:teuthology.orchestra.run.smithi143.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.143:6789,172.21.15.174:6789,172.21.15.174:6790:/ 2023-11-08T19:26:02.324 INFO:teuthology.orchestra.run.smithi143.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=968483af-77fc-4068-ad7c-5c2a1a4613d7" will pass to kernel 2023-11-08T19:26:02.325 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-08T19:26:02.326 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:26:02.326 DEBUG:teuthology.orchestra.run.smithi143:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-08T19:26:02.353 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:26:02.354 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-08T19:26:02.436 INFO:teuthology.orchestra.run.smithi143.stdout:client5059 2023-11-08T19:26:02.436 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-08T19:26:02.437 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:26:02.494 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create snap-test-pool 2023-11-08T19:26:02.657 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:02.655+0000 7f1edc2f5700 1 Processor -- start 2023-11-08T19:26:02.657 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:02.655+0000 7f1edc2f5700 1 -- start start 2023-11-08T19:26:02.660 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:02.656+0000 7f1edc2f5700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f1ed415a950 0x7f1ed40d01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:02.660 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:02.656+0000 7f1edc2f5700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f1ed406bd20 0x7f1ed4072370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:02.660 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:02.656+0000 7f1edc2f5700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1ed40728b0 0x7f1ed40cc730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:02.661 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:02.656+0000 7f1edc2f5700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ed40cc130 con 0x7f1ed406bd20 2023-11-08T19:26:02.661 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:02.656+0000 7f1edc2f5700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ed40cc2a0 con 0x7f1ed415a950 2023-11-08T19:26:02.661 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:02.656+0000 7f1edc2f5700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed406c1b0 con 0x7f1ed40728b0 2023-11-08T19:26:02.661 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:02.656+0000 7f1ed9890700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f1ed406bd20 0x7f1ed4072370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:02.661 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:02.656+0000 7f1ed9890700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f1ed406bd20 0x7f1ed4072370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50852/0 (socket says 172.21.15.143:50852) 2023-11-08T19:26:02.662 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:02.656+0000 7f1ed9890700 1 -- 172.21.15.143:0/777603420 learned_addr learned my addr 172.21.15.143:0/777603420 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:02.662 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:02.657+0000 7f1eda892700 1 --2- 172.21.15.143:0/777603420 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1ed40728b0 0x7f1ed40cc730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:02.662 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:02.657+0000 7f1eda091700 1 --2- 172.21.15.143:0/777603420 >> v2:172.21.15.174:3300/0 conn(0x7f1ed415a950 0x7f1ed40d01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:02.662 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:02.657+0000 7f1ed9890700 1 -- 172.21.15.143:0/777603420 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1ed40728b0 msgr2=0x7f1ed40cc730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:02.662 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:02.657+0000 7f1ed9890700 1 --2- 172.21.15.143:0/777603420 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1ed40728b0 0x7f1ed40cc730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:02.663 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:02.657+0000 7f1ed9890700 1 -- 172.21.15.143:0/777603420 >> v2:172.21.15.174:3300/0 conn(0x7f1ed415a950 msgr2=0x7f1ed40d01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:02.663 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:02.657+0000 7f1ed9890700 1 --2- 172.21.15.143:0/777603420 >> v2:172.21.15.174:3300/0 conn(0x7f1ed415a950 0x7f1ed40d01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:02.663 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:02.657+0000 7f1ed9890700 1 -- 172.21.15.143:0/777603420 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ed406c490 con 0x7f1ed406bd20 2023-11-08T19:26:02.663 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:02.657+0000 7f1eda892700 1 --2- 172.21.15.143:0/777603420 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1ed40728b0 0x7f1ed40cc730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:26:02.663 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:02.657+0000 7f1eda091700 1 --2- 172.21.15.143:0/777603420 >> v2:172.21.15.174:3300/0 conn(0x7f1ed415a950 0x7f1ed40d01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:26:02.664 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:02.657+0000 7f1ed9890700 1 --2- 172.21.15.143:0/777603420 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1ed406bd20 0x7f1ed4072370 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f1ed40cdf90 tx=0x7f1ed0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65a3d01bb4d9eaf3 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:02.664 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:02.657+0000 7f1ed908f700 1 -- 172.21.15.143:0/777603420 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ed000b870 con 0x7f1ed406bd20 2023-11-08T19:26:02.664 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:02.657+0000 7f1ed908f700 1 -- 172.21.15.143:0/777603420 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ed000b9d0 con 0x7f1ed406bd20 2023-11-08T19:26:02.665 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:02.657+0000 7f1ed908f700 1 -- 172.21.15.143:0/777603420 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ed000bd60 con 0x7f1ed406bd20 2023-11-08T19:26:02.665 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:02.658+0000 7f1edc2f5700 1 -- 172.21.15.143:0/777603420 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1ed406bd20 msgr2=0x7f1ed4072370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:02.666 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:02.658+0000 7f1edc2f5700 1 --2- 172.21.15.143:0/777603420 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1ed406bd20 0x7f1ed4072370 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f1ed40cdf90 tx=0x7f1ed0009bd0 comp rx=0 tx=0).stop 2023-11-08T19:26:02.666 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:02.658+0000 7f1edc2f5700 1 -- 172.21.15.143:0/777603420 shutdown_connections 2023-11-08T19:26:02.666 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:02.658+0000 7f1edc2f5700 1 --2- 172.21.15.143:0/777603420 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1ed40728b0 0x7f1ed40cc730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:02.666 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:02.658+0000 7f1edc2f5700 1 --2- 172.21.15.143:0/777603420 >> v2:172.21.15.174:3300/0 conn(0x7f1ed415a950 0x7f1ed40d01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:02.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:02.658+0000 7f1edc2f5700 1 --2- 172.21.15.143:0/777603420 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1ed406bd20 0x7f1ed4072370 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:02.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:02.658+0000 7f1edc2f5700 1 -- 172.21.15.143:0/777603420 >> 172.21.15.143:0/777603420 conn(0x7f1ed40cd2e0 msgr2=0x7f1ed40cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:02.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:02.658+0000 7f1edc2f5700 1 -- 172.21.15.143:0/777603420 shutdown_connections 2023-11-08T19:26:02.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:02.658+0000 7f1edc2f5700 1 -- 172.21.15.143:0/777603420 wait complete. 2023-11-08T19:26:02.668 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:02.659+0000 7f1edc2f5700 1 Processor -- start 2023-11-08T19:26:02.668 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:02.659+0000 7f1edc2f5700 1 -- start start 2023-11-08T19:26:02.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:02.660+0000 7f1edc2f5700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1ed40728b0 0x7f1ed415c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:02.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:02.660+0000 7f1edc2f5700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1ed415a950 0x7f1ed40c8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:02.672 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:02.660+0000 7f1edc2f5700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1ed4155b30 0x7f1ed4155390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:02.672 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:02.660+0000 7f1edc2f5700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed40c96c0 con 0x7f1ed415a950 2023-11-08T19:26:02.672 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:02.660+0000 7f1edc2f5700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed40c9420 con 0x7f1ed40728b0 2023-11-08T19:26:02.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:02.660+0000 7f1edc2f5700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed415bae0 con 0x7f1ed4155b30 2023-11-08T19:26:02.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:02.660+0000 7f1ed9890700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1ed415a950 0x7f1ed40c8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:02.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:02.660+0000 7f1ed9890700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1ed415a950 0x7f1ed40c8ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50860/0 (socket says 172.21.15.143:50860) 2023-11-08T19:26:02.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:02.660+0000 7f1ed9890700 1 -- 172.21.15.143:0/4146862084 learned_addr learned my addr 172.21.15.143:0/4146862084 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:02.674 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:02.660+0000 7f1eda892700 1 --2- 172.21.15.143:0/4146862084 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1ed4155b30 0x7f1ed4155390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:02.674 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:02.660+0000 7f1eda091700 1 --2- 172.21.15.143:0/4146862084 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1ed40728b0 0x7f1ed415c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:02.674 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:02.660+0000 7f1ed9890700 1 -- 172.21.15.143:0/4146862084 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1ed4155b30 msgr2=0x7f1ed4155390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:02.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:02.660+0000 7f1ed9890700 1 --2- 172.21.15.143:0/4146862084 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1ed4155b30 0x7f1ed4155390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:02.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:02.660+0000 7f1ed9890700 1 -- 172.21.15.143:0/4146862084 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1ed40728b0 msgr2=0x7f1ed415c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:02.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:02.660+0000 7f1ed9890700 1 --2- 172.21.15.143:0/4146862084 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1ed40728b0 0x7f1ed415c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:02.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:02.660+0000 7f1ed9890700 1 -- 172.21.15.143:0/4146862084 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ed0009550 con 0x7f1ed415a950 2023-11-08T19:26:02.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:02.660+0000 7f1ed9890700 1 --2- 172.21.15.143:0/4146862084 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1ed415a950 0x7f1ed40c8ee0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f1ed0000c00 tx=0x7f1ed0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:02.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:02.660+0000 7f1ecaffd700 1 -- 172.21.15.143:0/4146862084 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ed0004480 con 0x7f1ed415a950 2023-11-08T19:26:02.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:02.660+0000 7f1ecaffd700 1 -- 172.21.15.143:0/4146862084 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ed00045e0 con 0x7f1ed415a950 2023-11-08T19:26:02.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:02.660+0000 7f1ecaffd700 1 -- 172.21.15.143:0/4146862084 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ed00038e0 con 0x7f1ed415a950 2023-11-08T19:26:02.677 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:02.660+0000 7f1edc2f5700 1 -- 172.21.15.143:0/4146862084 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ed415bdc0 con 0x7f1ed415a950 2023-11-08T19:26:02.677 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:02.660+0000 7f1edc2f5700 1 -- 172.21.15.143:0/4146862084 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ed415c380 con 0x7f1ed415a950 2023-11-08T19:26:02.677 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:02.661+0000 7f1edc2f5700 1 -- 172.21.15.143:0/4146862084 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ed404eee0 con 0x7f1ed415a950 2023-11-08T19:26:02.677 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:02.662+0000 7f1ecaffd700 1 -- 172.21.15.143:0/4146862084 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1ed0021070 con 0x7f1ed415a950 2023-11-08T19:26:02.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:02.662+0000 7f1ecaffd700 1 --2- 172.21.15.143:0/4146862084 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f1eb806af30 0x7f1eb806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:02.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:02.662+0000 7f1ecaffd700 1 -- 172.21.15.143:0/4146862084 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1ed008aa80 con 0x7f1ed415a950 2023-11-08T19:26:02.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:02.662+0000 7f1eda091700 1 --2- 172.21.15.143:0/4146862084 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f1eb806af30 0x7f1eb806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:02.679 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:02.663+0000 7f1eda091700 1 --2- 172.21.15.143:0/4146862084 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f1eb806af30 0x7f1eb806d3f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1ec400c940 tx=0x7f1ec4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:02.679 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:02.666+0000 7f1ecaffd700 1 -- 172.21.15.143:0/4146862084 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1ed0055bc0 con 0x7f1ed415a950 2023-11-08T19:26:02.857 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:02.855+0000 7f1edc2f5700 1 -- 172.21.15.143:0/4146862084 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "snap-test-pool"} v 0) v1 -- 0x7f1ed4072150 con 0x7f1ed415a950 2023-11-08T19:26:03.586 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:03.584+0000 7f1ecaffd700 1 -- 172.21.15.143:0/4146862084 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "snap-test-pool"}]=0 pool 'snap-test-pool' created v34) v1 ==== 118+0+0 (secure 0 0 0) 0x7f1ed00290a0 con 0x7f1ed415a950 2023-11-08T19:26:03.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:03.588+0000 7f1edc2f5700 1 -- 172.21.15.143:0/4146862084 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f1eb806af30 msgr2=0x7f1eb806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:03.590 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:03.588+0000 7f1edc2f5700 1 --2- 172.21.15.143:0/4146862084 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f1eb806af30 0x7f1eb806d3f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1ec400c940 tx=0x7f1ec4008040 comp rx=0 tx=0).stop 2023-11-08T19:26:03.590 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:03.588+0000 7f1edc2f5700 1 -- 172.21.15.143:0/4146862084 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1ed415a950 msgr2=0x7f1ed40c8ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:03.590 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:03.588+0000 7f1edc2f5700 1 --2- 172.21.15.143:0/4146862084 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1ed415a950 0x7f1ed40c8ee0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f1ed0000c00 tx=0x7f1ed0004750 comp rx=0 tx=0).stop 2023-11-08T19:26:03.591 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:03.589+0000 7f1edc2f5700 1 -- 172.21.15.143:0/4146862084 shutdown_connections 2023-11-08T19:26:03.591 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:03.589+0000 7f1edc2f5700 1 --2- 172.21.15.143:0/4146862084 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f1eb806af30 0x7f1eb806d3f0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:03.592 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:03.589+0000 7f1edc2f5700 1 --2- 172.21.15.143:0/4146862084 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1ed4155b30 0x7f1ed4155390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:03.592 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:03.589+0000 7f1edc2f5700 1 --2- 172.21.15.143:0/4146862084 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1ed40728b0 0x7f1ed415c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:03.592 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:03.589+0000 7f1edc2f5700 1 --2- 172.21.15.143:0/4146862084 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1ed415a950 0x7f1ed40c8ee0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:03.592 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:03.589+0000 7f1edc2f5700 1 -- 172.21.15.143:0/4146862084 >> 172.21.15.143:0/4146862084 conn(0x7f1ed40cd2e0 msgr2=0x7f1ed40c7930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:03.593 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:03.589+0000 7f1edc2f5700 1 -- 172.21.15.143:0/4146862084 shutdown_connections 2023-11-08T19:26:03.593 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:03.589+0000 7f1edc2f5700 1 -- 172.21.15.143:0/4146862084 wait complete. 2023-11-08T19:26:03.596 INFO:teuthology.orchestra.run.smithi143.stderr:pool 'snap-test-pool' created 2023-11-08T19:26:03.609 DEBUG:teuthology.orchestra.run.smithi143:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool snap-test-pool mksnap snap3 2023-11-08T19:26:03.656 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:03.654+0000 7f991d23c080 1 Processor -- start 2023-11-08T19:26:03.656 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:03.654+0000 7f991d23c080 1 -- start start 2023-11-08T19:26:03.662 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:03.655+0000 7f991d23c080 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5585001063b0 0x5585001da060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:03.662 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:03.655+0000 7f991d23c080 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x558500136000 0x55850012ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:03.662 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:03.655+0000 7f991d23c080 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5585001db460 0x558500130440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:03.662 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:03.655+0000 7f991d23c080 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585000af520 con 0x5585001db460 2023-11-08T19:26:03.663 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:03.655+0000 7f991d23c080 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x558500049740 con 0x5585001063b0 2023-11-08T19:26:03.663 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:03.655+0000 7f991d23c080 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x5585000493e0 con 0x558500136000 2023-11-08T19:26:03.663 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:03.655+0000 7f9913ad2700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5585001db460 0x558500130440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:03.663 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:03.655+0000 7f9913ad2700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5585001db460 0x558500130440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50874/0 (socket says 172.21.15.143:50874) 2023-11-08T19:26:03.663 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:03.655+0000 7f9912ad0700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x558500136000 0x55850012ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:03.664 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:03.655+0000 7f9913ad2700 1 -- 172.21.15.143:0/4140668571 learned_addr learned my addr 172.21.15.143:0/4140668571 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:03.664 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:03.655+0000 7f99132d1700 1 --2- 172.21.15.143:0/4140668571 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5585001063b0 0x5585001da060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:03.664 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:03.655+0000 7f9913ad2700 1 -- 172.21.15.143:0/4140668571 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x558500136000 msgr2=0x55850012ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:03.664 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:03.655+0000 7f9913ad2700 1 --2- 172.21.15.143:0/4140668571 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x558500136000 0x55850012ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:03.664 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:03.655+0000 7f9913ad2700 1 -- 172.21.15.143:0/4140668571 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5585001063b0 msgr2=0x5585001da060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:03.665 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:03.655+0000 7f9913ad2700 1 --2- 172.21.15.143:0/4140668571 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5585001063b0 0x5585001da060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:03.665 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:03.655+0000 7f9913ad2700 1 -- 172.21.15.143:0/4140668571 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9904005de0 con 0x5585001db460 2023-11-08T19:26:03.665 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:03.656+0000 7f9913ad2700 1 --2- 172.21.15.143:0/4140668571 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5585001db460 0x558500130440 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f990c00a7c0 tx=0x7f990c00f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4403a654872c9e4a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:03.665 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:03.656+0000 7f9911ace700 1 -- 172.21.15.143:0/4140668571 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f990c00f790 con 0x5585001db460 2023-11-08T19:26:03.665 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:03.656+0000 7f9911ace700 1 -- 172.21.15.143:0/4140668571 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f990c00ae80 con 0x5585001db460 2023-11-08T19:26:03.666 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:03.656+0000 7f9911ace700 1 -- 172.21.15.143:0/4140668571 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f990c00fc60 con 0x5585001db460 2023-11-08T19:26:03.666 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:03.656+0000 7f991d23c080 1 -- 172.21.15.143:0/4140668571 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5585001db460 msgr2=0x558500130440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:03.666 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:03.656+0000 7f991d23c080 1 --2- 172.21.15.143:0/4140668571 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5585001db460 0x558500130440 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f990c00a7c0 tx=0x7f990c00f000 comp rx=0 tx=0).stop 2023-11-08T19:26:03.666 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:03.656+0000 7f991d23c080 1 -- 172.21.15.143:0/4140668571 shutdown_connections 2023-11-08T19:26:03.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:03.656+0000 7f991d23c080 1 --2- 172.21.15.143:0/4140668571 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5585001db460 0x558500130440 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:03.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:03.656+0000 7f991d23c080 1 --2- 172.21.15.143:0/4140668571 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5585001063b0 0x5585001da060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:03.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:03.656+0000 7f991d23c080 1 --2- 172.21.15.143:0/4140668571 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x558500136000 0x55850012ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:03.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:03.656+0000 7f991d23c080 1 -- 172.21.15.143:0/4140668571 >> 172.21.15.143:0/4140668571 conn(0x5585001503c0 msgr2=0x558500131470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:03.668 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:03.656+0000 7f991d23c080 1 -- 172.21.15.143:0/4140668571 shutdown_connections 2023-11-08T19:26:03.668 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:03.656+0000 7f991d23c080 1 -- 172.21.15.143:0/4140668571 wait complete. 2023-11-08T19:26:03.668 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:03.657+0000 7f991d23c080 1 Processor -- start 2023-11-08T19:26:03.668 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:03.657+0000 7f991d23c080 1 -- start start 2023-11-08T19:26:03.668 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:03.658+0000 7f991d23c080 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5585001063b0 0x5585001cc930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:03.669 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:03.658+0000 7f991d23c080 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x558500136000 0x558500131260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:03.669 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:03.658+0000 7f991d23c080 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5585001db460 0x5585001317a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:03.669 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:03.658+0000 7f991d23c080 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x55850003d680 con 0x5585001063b0 2023-11-08T19:26:03.669 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:03.658+0000 7f991d23c080 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x558500127450 con 0x5585001db460 2023-11-08T19:26:03.669 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:03.658+0000 7f991d23c080 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x558500127170 con 0x558500136000 2023-11-08T19:26:03.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:03.658+0000 7f99132d1700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5585001063b0 0x5585001cc930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:03.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:03.658+0000 7f99132d1700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5585001063b0 0x5585001cc930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50876/0 (socket says 172.21.15.143:50876) 2023-11-08T19:26:03.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:03.658+0000 7f99132d1700 1 -- 172.21.15.143:0/1893423037 learned_addr learned my addr 172.21.15.143:0/1893423037 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:03.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:03.658+0000 7f9912ad0700 1 --2- 172.21.15.143:0/1893423037 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x558500136000 0x558500131260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:03.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:03.658+0000 7f9913ad2700 1 --2- 172.21.15.143:0/1893423037 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5585001db460 0x5585001317a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:03.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:03.658+0000 7f99132d1700 1 -- 172.21.15.143:0/1893423037 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x558500136000 msgr2=0x558500131260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:03.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:03.658+0000 7f99132d1700 1 --2- 172.21.15.143:0/1893423037 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x558500136000 0x558500131260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:03.672 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:03.658+0000 7f99132d1700 1 -- 172.21.15.143:0/1893423037 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5585001db460 msgr2=0x5585001317a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:03.672 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:03.658+0000 7f99132d1700 1 --2- 172.21.15.143:0/1893423037 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5585001db460 0x5585001317a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:03.672 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:03.658+0000 7f99132d1700 1 -- 172.21.15.143:0/1893423037 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55850006f7c0 con 0x5585001063b0 2023-11-08T19:26:03.672 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:03.658+0000 7f99132d1700 1 --2- 172.21.15.143:0/1893423037 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5585001063b0 0x5585001cc930 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f990800efb0 tx=0x7f990800c250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:03.672 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:03.658+0000 7f9910acc700 1 -- 172.21.15.143:0/1893423037 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9908002bb0 con 0x5585001063b0 2023-11-08T19:26:03.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:03.658+0000 7f9910acc700 1 -- 172.21.15.143:0/1893423037 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9908008030 con 0x5585001063b0 2023-11-08T19:26:03.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:03.658+0000 7f991d23c080 1 -- 172.21.15.143:0/1893423037 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f990c00a100 con 0x5585001063b0 2023-11-08T19:26:03.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:03.659+0000 7f9910acc700 1 -- 172.21.15.143:0/1893423037 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f990800cd00 con 0x5585001063b0 2023-11-08T19:26:03.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:03.659+0000 7f991d23c080 1 -- 172.21.15.143:0/1893423037 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5585000557d0 con 0x5585001063b0 2023-11-08T19:26:03.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:03.660+0000 7f9910acc700 1 -- 172.21.15.143:0/1893423037 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9908011450 con 0x5585001063b0 2023-11-08T19:26:03.674 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:03.660+0000 7f9910acc700 1 --2- 172.21.15.143:0/1893423037 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f9900069820 0x7f990006bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:03.674 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:03.660+0000 7f9910acc700 1 -- 172.21.15.143:0/1893423037 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f99080882e0 con 0x5585001063b0 2023-11-08T19:26:03.674 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:03.660+0000 7f9912ad0700 1 --2- 172.21.15.143:0/1893423037 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f9900069820 0x7f990006bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:03.674 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:03.661+0000 7f991d23c080 1 -- 172.21.15.143:0/1893423037 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- pool_op(create snap pool 6 tid 1 name snap3 v0) v4 -- 0x5585000b94a0 con 0x5585001063b0 2023-11-08T19:26:03.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:03.661+0000 7f9912ad0700 1 --2- 172.21.15.143:0/1893423037 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f9900069820 0x7f990006bce0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x55850014b490 tx=0x7f990400c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:04.591 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:04.589+0000 7f9910acc700 1 -- 172.21.15.143:0/1893423037 <== mon.0 v2:172.21.15.143:3300/0 6 ==== pool_op_reply(tid 1 (0) Success v35) v1 ==== 43+0+0 (secure 0 0 0) 0x7f99080532c0 con 0x5585001063b0 2023-11-08T19:26:04.591 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:04.590+0000 7f9910acc700 1 -- 172.21.15.143:0/1893423037 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=35}) v3 -- 0x7f9900072000 con 0x5585001063b0 2023-11-08T19:26:04.593 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:04.592+0000 7f9910acc700 1 -- 172.21.15.143:0/1893423037 <== mon.0 v2:172.21.15.143:3300/0 7 ==== osd_map(35..35 src has 1..35) v4 ==== 710+0+0 (secure 0 0 0) 0x7f9908021300 con 0x5585001063b0 2023-11-08T19:26:04.594 INFO:teuthology.orchestra.run.smithi143.stdout:created pool snap-test-pool snap snap3 2023-11-08T19:26:04.594 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:04.593+0000 7f991d23c080 1 -- 172.21.15.143:0/1893423037 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f9900069820 msgr2=0x7f990006bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:04.595 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:04.593+0000 7f991d23c080 1 --2- 172.21.15.143:0/1893423037 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f9900069820 0x7f990006bce0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x55850014b490 tx=0x7f990400c040 comp rx=0 tx=0).stop 2023-11-08T19:26:04.595 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:04.593+0000 7f991d23c080 1 -- 172.21.15.143:0/1893423037 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5585001063b0 msgr2=0x5585001cc930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:04.595 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:04.593+0000 7f991d23c080 1 --2- 172.21.15.143:0/1893423037 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5585001063b0 0x5585001cc930 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f990800efb0 tx=0x7f990800c250 comp rx=0 tx=0).stop 2023-11-08T19:26:04.596 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:04.594+0000 7f991d23c080 1 -- 172.21.15.143:0/1893423037 shutdown_connections 2023-11-08T19:26:04.596 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:04.594+0000 7f991d23c080 1 --2- 172.21.15.143:0/1893423037 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f9900069820 0x7f990006bce0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:04.596 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:04.594+0000 7f991d23c080 1 --2- 172.21.15.143:0/1893423037 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5585001db460 0x5585001317a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:04.596 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:04.594+0000 7f991d23c080 1 --2- 172.21.15.143:0/1893423037 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5585001063b0 0x5585001cc930 secure :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f990800efb0 tx=0x7f990800c250 comp rx=0 tx=0).stop 2023-11-08T19:26:04.596 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:04.594+0000 7f991d23c080 1 --2- 172.21.15.143:0/1893423037 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x558500136000 0x558500131260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:04.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:04.594+0000 7f991d23c080 1 -- 172.21.15.143:0/1893423037 >> 172.21.15.143:0/1893423037 conn(0x5585001503c0 msgr2=0x5585001480a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:04.598 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:04.597+0000 7f991d23c080 1 -- 172.21.15.143:0/1893423037 shutdown_connections 2023-11-08T19:26:04.598 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:04.597+0000 7f991d23c080 1 -- 172.21.15.143:0/1893423037 wait complete. 2023-11-08T19:26:04.604 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs snap-test-pool 2023-11-08T19:26:04.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:04.733+0000 7f0c7648b700 1 Processor -- start 2023-11-08T19:26:04.735 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:04.733+0000 7f0c7648b700 1 -- start start 2023-11-08T19:26:04.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:04.734+0000 7f0c7648b700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0c7006bd20 0x7f0c700d01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:04.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:04.734+0000 7f0c7648b700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f0c700b61e0 0x7f0c700cc730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:04.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:04.734+0000 7f0c7648b700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c700cc2a0 con 0x7f0c7015a950 2023-11-08T19:26:04.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:04.734+0000 7f0c7648b700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c700cc130 con 0x7f0c700b61e0 2023-11-08T19:26:04.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:04.734+0000 7f0c7648b700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c7006c1b0 con 0x7f0c7006bd20 2023-11-08T19:26:04.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:04.734+0000 7f0c6ffff700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f0c7015a950 0x7f0c70072370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:47038/0 (socket says 172.21.15.143:47038) 2023-11-08T19:26:04.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:04.734+0000 7f0c6ffff700 1 -- 172.21.15.143:0/3673438948 learned_addr learned my addr 172.21.15.143:0/3673438948 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:26:04.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:04.734+0000 7f0c6f7fe700 1 --2- 172.21.15.143:0/3673438948 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0c7006bd20 0x7f0c700d01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:04.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:04.734+0000 7f0c74a28700 1 --2- 172.21.15.143:0/3673438948 >> v2:172.21.15.174:3300/0 conn(0x7f0c700b61e0 0x7f0c700cc730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:04.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:04.734+0000 7f0c6effd700 1 -- 172.21.15.143:0/3673438948 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1218439042 0 0) 0x7f0c700cc2a0 con 0x7f0c7015a950 2023-11-08T19:26:04.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:04.734+0000 7f0c6effd700 1 -- 172.21.15.143:0/3673438948 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c580035b0 con 0x7f0c7015a950 2023-11-08T19:26:04.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:04.735+0000 7f0c6effd700 1 -- 172.21.15.143:0/3673438948 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3480463654 0 0) 0x7f0c580035b0 con 0x7f0c7015a950 2023-11-08T19:26:04.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:04.735+0000 7f0c6effd700 1 -- 172.21.15.143:0/3673438948 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0c7006bd20 msgr2=0x7f0c700d01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:04.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:04.735+0000 7f0c6effd700 1 --2- 172.21.15.143:0/3673438948 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0c7006bd20 0x7f0c700d01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:04.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:04.735+0000 7f0c6effd700 1 -- 172.21.15.143:0/3673438948 >> v2:172.21.15.174:3300/0 conn(0x7f0c700b61e0 msgr2=0x7f0c700cc730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:04.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:04.735+0000 7f0c6effd700 1 --2- 172.21.15.143:0/3673438948 >> v2:172.21.15.174:3300/0 conn(0x7f0c700b61e0 0x7f0c700cc730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:04.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:04.735+0000 7f0c6effd700 1 -- 172.21.15.143:0/3673438948 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c7006c490 con 0x7f0c7015a950 2023-11-08T19:26:04.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:04.735+0000 7f0c6effd700 1 -- 172.21.15.143:0/3673438948 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4147756303 0 0) 0x7f0c60001670 con 0x7f0c7015a950 2023-11-08T19:26:04.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:04.735+0000 7f0c6effd700 1 -- 172.21.15.143:0/3673438948 >> v1:172.21.15.143:6789/0 conn(0x7f0c7015a950 legacy=0x7f0c70072370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:04.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:04.735+0000 7f0c6effd700 1 --2- 172.21.15.143:0/3673438948 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0c58003a00 0x7f0c58003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:04.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:04.735+0000 7f0c6effd700 1 --2- 172.21.15.143:0/3673438948 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0c58004360 0x7f0c58004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:04.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:04.735+0000 7f0c6effd700 1 --2- 172.21.15.143:0/3673438948 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0c58004ea0 0x7f0c58005320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:04.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:04.735+0000 7f0c6effd700 1 -- 172.21.15.143:0/3673438948 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c7006c1b0 con 0x7f0c58004360 2023-11-08T19:26:04.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:04.735+0000 7f0c6effd700 1 -- 172.21.15.143:0/3673438948 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c5800bdb0 con 0x7f0c58004ea0 2023-11-08T19:26:04.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:04.735+0000 7f0c6effd700 1 -- 172.21.15.143:0/3673438948 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c5800bf70 con 0x7f0c58003a00 2023-11-08T19:26:04.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:04.735+0000 7f0c6f7fe700 1 --2- 172.21.15.143:0/3673438948 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0c58004360 0x7f0c58004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:04.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:04.735+0000 7f0c6ffff700 1 --2- 172.21.15.143:0/3673438948 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0c58003a00 0x7f0c58003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:04.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:04.735+0000 7f0c74a28700 1 --2- 172.21.15.143:0/3673438948 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0c58004ea0 0x7f0c58005320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:04.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:04.735+0000 7f0c6f7fe700 1 -- 172.21.15.143:0/3673438948 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0c58003a00 msgr2=0x7f0c58003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:04.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:04.735+0000 7f0c6f7fe700 1 --2- 172.21.15.143:0/3673438948 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0c58003a00 0x7f0c58003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:04.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:04.735+0000 7f0c6f7fe700 1 -- 172.21.15.143:0/3673438948 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0c58004ea0 msgr2=0x7f0c58005320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:04.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:04.735+0000 7f0c6f7fe700 1 --2- 172.21.15.143:0/3673438948 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0c58004ea0 0x7f0c58005320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:04.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:04.735+0000 7f0c6f7fe700 1 -- 172.21.15.143:0/3673438948 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c580035b0 con 0x7f0c58004360 2023-11-08T19:26:04.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:04.735+0000 7f0c6f7fe700 1 --2- 172.21.15.143:0/3673438948 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0c58004360 0x7f0c58004850 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f0c64009b60 tx=0x7f0c58005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5ab0a3e09b17775 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:04.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:04.735+0000 7f0c6effd700 1 -- 172.21.15.143:0/3673438948 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c6400e6c0 con 0x7f0c58004360 2023-11-08T19:26:04.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:04.735+0000 7f0c6effd700 1 -- 172.21.15.143:0/3673438948 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c6400e820 con 0x7f0c58004360 2023-11-08T19:26:04.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:04.735+0000 7f0c6effd700 1 -- 172.21.15.143:0/3673438948 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c6400eb90 con 0x7f0c58004360 2023-11-08T19:26:04.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:04.736+0000 7f0c7648b700 1 -- 172.21.15.143:0/3673438948 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0c58004360 msgr2=0x7f0c58004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:04.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:04.736+0000 7f0c7648b700 1 --2- 172.21.15.143:0/3673438948 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0c58004360 0x7f0c58004850 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f0c64009b60 tx=0x7f0c58005880 comp rx=0 tx=0).stop 2023-11-08T19:26:04.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:04.736+0000 7f0c74a28700 1 -- 172.21.15.143:0/3673438948 reap_dead start 2023-11-08T19:26:04.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:04.736+0000 7f0c7648b700 1 -- 172.21.15.143:0/3673438948 shutdown_connections 2023-11-08T19:26:04.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:04.736+0000 7f0c7648b700 1 -- 172.21.15.143:0/3673438948 >> 172.21.15.143:0/3673438948 conn(0x7f0c700cd2e0 msgr2=0x7f0c700cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:04.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:04.736+0000 7f0c7648b700 1 -- 172.21.15.143:0/3673438948 shutdown_connections 2023-11-08T19:26:04.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:04.736+0000 7f0c7648b700 1 -- 172.21.15.143:0/3673438948 wait complete. 2023-11-08T19:26:04.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:04.737+0000 7f0c7648b700 1 Processor -- start 2023-11-08T19:26:04.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:04.737+0000 7f0c7648b700 1 -- start start 2023-11-08T19:26:04.752 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:04.738+0000 7f0c7648b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0c58004360 0x7f0c700af4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:04.752 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:04.738+0000 7f0c7648b700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0c700c8620 0x7f0c7015e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:04.752 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:04.738+0000 7f0c7648b700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0c7015e7d0 0x7f0c701f3480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:04.753 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:04.738+0000 7f0c7648b700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c700c7930 con 0x7f0c7015e7d0 2023-11-08T19:26:04.753 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:04.738+0000 7f0c7648b700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c7015ec50 con 0x7f0c58004360 2023-11-08T19:26:04.753 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:04.738+0000 7f0c7648b700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c701f41e0 con 0x7f0c700c8620 2023-11-08T19:26:04.753 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:04.738+0000 7f0c74a28700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0c7015e7d0 0x7f0c701f3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:04.753 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:04.738+0000 7f0c74a28700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0c7015e7d0 0x7f0c701f3480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50900/0 (socket says 172.21.15.143:50900) 2023-11-08T19:26:04.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:04.738+0000 7f0c74a28700 1 -- 172.21.15.143:0/3444454994 learned_addr learned my addr 172.21.15.143:0/3444454994 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:04.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:04.738+0000 7f0c6f7fe700 1 --2- 172.21.15.143:0/3444454994 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0c700c8620 0x7f0c7015e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:04.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:04.738+0000 7f0c6ffff700 1 --2- 172.21.15.143:0/3444454994 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0c58004360 0x7f0c700af4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:04.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:04.739+0000 7f0c74a28700 1 -- 172.21.15.143:0/3444454994 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0c700c8620 msgr2=0x7f0c7015e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:04.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:04.739+0000 7f0c74a28700 1 --2- 172.21.15.143:0/3444454994 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0c700c8620 0x7f0c7015e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:04.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:04.739+0000 7f0c74a28700 1 -- 172.21.15.143:0/3444454994 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0c58004360 msgr2=0x7f0c700af4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:04.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:04.739+0000 7f0c74a28700 1 --2- 172.21.15.143:0/3444454994 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0c58004360 0x7f0c700af4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:04.756 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:04.739+0000 7f0c74a28700 1 -- 172.21.15.143:0/3444454994 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c700c7b40 con 0x7f0c7015e7d0 2023-11-08T19:26:04.756 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:04.739+0000 7f0c74a28700 1 --2- 172.21.15.143:0/3444454994 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0c7015e7d0 0x7f0c701f3480 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f0c700cbbc0 tx=0x7f0c68011ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:04.756 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:04.739+0000 7f0c6cff9700 1 -- 172.21.15.143:0/3444454994 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c6800e040 con 0x7f0c7015e7d0 2023-11-08T19:26:04.756 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:04.739+0000 7f0c6cff9700 1 -- 172.21.15.143:0/3444454994 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c6800f040 con 0x7f0c7015e7d0 2023-11-08T19:26:04.756 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:04.739+0000 7f0c7648b700 1 -- 172.21.15.143:0/3444454994 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c700c7e20 con 0x7f0c7015e7d0 2023-11-08T19:26:04.757 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:04.739+0000 7f0c6cff9700 1 -- 172.21.15.143:0/3444454994 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c6800fc40 con 0x7f0c7015e7d0 2023-11-08T19:26:04.757 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:04.739+0000 7f0c7648b700 1 -- 172.21.15.143:0/3444454994 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c700afa30 con 0x7f0c7015e7d0 2023-11-08T19:26:04.757 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:04.740+0000 7f0c7648b700 1 -- 172.21.15.143:0/3444454994 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c7004eee0 con 0x7f0c7015e7d0 2023-11-08T19:26:04.757 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:04.740+0000 7f0c6cff9700 1 -- 172.21.15.143:0/3444454994 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0c68022460 con 0x7f0c7015e7d0 2023-11-08T19:26:04.757 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:04.741+0000 7f0c6cff9700 1 --2- 172.21.15.143:0/3444454994 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f0c5c06af00 0x7f0c5c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:04.758 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:04.741+0000 7f0c6cff9700 1 -- 172.21.15.143:0/3444454994 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0c68089c40 con 0x7f0c7015e7d0 2023-11-08T19:26:04.758 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:04.741+0000 7f0c6ffff700 1 --2- 172.21.15.143:0/3444454994 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f0c5c06af00 0x7f0c5c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:04.758 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:04.741+0000 7f0c6ffff700 1 --2- 172.21.15.143:0/3444454994 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f0c5c06af00 0x7f0c5c06d3c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0c60000c00 tx=0x7f0c60007980 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:04.758 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:04.745+0000 7f0c6cff9700 1 -- 172.21.15.143:0/3444454994 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0c68054b60 con 0x7f0c7015e7d0 2023-11-08T19:26:04.980 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:04.979+0000 7f0c7648b700 1 -- 172.21.15.143:0/3444454994 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "snap-test-pool"} v 0) v1 -- 0x7f0c700b5360 con 0x7f0c7015e7d0 2023-11-08T19:26:04.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:04.980+0000 7f0c6cff9700 1 -- 172.21.15.143:0/3444454994 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "snap-test-pool"}]=-95 pool(6) already has mon-managed snaps; can't attach pool to fs v16) v1 ==== 173+0+0 (secure 0 0 0) 0x7f0c68058180 con 0x7f0c7015e7d0 2023-11-08T19:26:04.982 INFO:teuthology.orchestra.run.smithi143.stderr:Error ENOTSUP: pool(6) already has mon-managed snaps; can't attach pool to fs 2023-11-08T19:26:04.986 DEBUG:teuthology.orchestra.run:got remote process result: 95 2023-11-08T19:26:04.987 DEBUG:teuthology.orchestra.run.smithi143:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool snap-test-pool rmsnap snap3 2023-11-08T19:26:05.030 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.029+0000 7faab88c1080 1 Processor -- start 2023-11-08T19:26:05.030 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.029+0000 7faab88c1080 1 -- start start 2023-11-08T19:26:05.035 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.029+0000 7faab88c1080 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55d075b007b0 0x55d075bcb4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:05.035 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.029+0000 7faab88c1080 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d075b00b90 0x55d075b2b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:05.035 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.029+0000 7faab88c1080 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d075b471a0 0x55d075b2ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:05.035 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.029+0000 7faab88c1080 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d075a43740 con 0x55d075b00b90 2023-11-08T19:26:05.036 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.029+0000 7faab88c1080 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d075a433e0 con 0x55d075b471a0 2023-11-08T19:26:05.036 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.029+0000 7faab88c1080 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d075a37680 con 0x55d075b007b0 2023-11-08T19:26:05.036 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.029+0000 7faaae155700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d075b00b90 0x55d075b2b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:05.036 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.030+0000 7faaae155700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d075b00b90 0x55d075b2b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50912/0 (socket says 172.21.15.143:50912) 2023-11-08T19:26:05.036 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.030+0000 7faaae155700 1 -- 172.21.15.143:0/1563743886 learned_addr learned my addr 172.21.15.143:0/1563743886 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:05.037 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.030+0000 7faaae956700 1 --2- 172.21.15.143:0/1563743886 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55d075b007b0 0x55d075bcb4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:05.037 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.030+0000 7faaaf157700 1 --2- 172.21.15.143:0/1563743886 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d075b471a0 0x55d075b2ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:05.037 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.030+0000 7faaae155700 1 -- 172.21.15.143:0/1563743886 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55d075b007b0 msgr2=0x55d075bcb4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:05.037 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.030+0000 7faaae155700 1 --2- 172.21.15.143:0/1563743886 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55d075b007b0 0x55d075bcb4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:05.037 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.030+0000 7faaae155700 1 -- 172.21.15.143:0/1563743886 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d075b471a0 msgr2=0x55d075b2ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:05.038 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.030+0000 7faaae155700 1 --2- 172.21.15.143:0/1563743886 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d075b471a0 0x55d075b2ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:05.038 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.030+0000 7faaae155700 1 -- 172.21.15.143:0/1563743886 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaa0005de0 con 0x55d075b00b90 2023-11-08T19:26:05.038 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.030+0000 7faaaf157700 1 --2- 172.21.15.143:0/1563743886 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d075b471a0 0x55d075b2ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:26:05.038 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.030+0000 7faaae155700 1 --2- 172.21.15.143:0/1563743886 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d075b00b90 0x55d075b2b4e0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7faaa400a770 tx=0x7faaa400aa80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8dfb4f40a468911 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:05.038 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.030+0000 7faaad153700 1 -- 172.21.15.143:0/1563743886 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaa400f070 con 0x55d075b00b90 2023-11-08T19:26:05.039 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.030+0000 7faaad153700 1 -- 172.21.15.143:0/1563743886 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faaa4007600 con 0x55d075b00b90 2023-11-08T19:26:05.039 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.030+0000 7faaad153700 1 -- 172.21.15.143:0/1563743886 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaa4007970 con 0x55d075b00b90 2023-11-08T19:26:05.039 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.030+0000 7faab88c1080 1 -- 172.21.15.143:0/1563743886 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d075b00b90 msgr2=0x55d075b2b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:05.039 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.030+0000 7faab88c1080 1 --2- 172.21.15.143:0/1563743886 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d075b00b90 0x55d075b2b4e0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7faaa400a770 tx=0x7faaa400aa80 comp rx=0 tx=0).stop 2023-11-08T19:26:05.039 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.030+0000 7faab88c1080 1 -- 172.21.15.143:0/1563743886 shutdown_connections 2023-11-08T19:26:05.040 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.030+0000 7faab88c1080 1 --2- 172.21.15.143:0/1563743886 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55d075b007b0 0x55d075bcb4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:05.040 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.031+0000 7faab88c1080 1 --2- 172.21.15.143:0/1563743886 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d075b471a0 0x55d075b2ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:05.040 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.031+0000 7faab88c1080 1 --2- 172.21.15.143:0/1563743886 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d075b00b90 0x55d075b2b4e0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:05.040 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.031+0000 7faab88c1080 1 -- 172.21.15.143:0/1563743886 >> 172.21.15.143:0/1563743886 conn(0x55d075b4ac40 msgr2=0x55d075bd43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:05.040 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.031+0000 7faab88c1080 1 -- 172.21.15.143:0/1563743886 shutdown_connections 2023-11-08T19:26:05.041 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.031+0000 7faab88c1080 1 -- 172.21.15.143:0/1563743886 wait complete. 2023-11-08T19:26:05.041 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.031+0000 7faab88c1080 1 Processor -- start 2023-11-08T19:26:05.041 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.031+0000 7faab88c1080 1 -- start start 2023-11-08T19:26:05.041 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.032+0000 7faab88c1080 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d075b007b0 0x55d075bca200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:05.041 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.032+0000 7faab88c1080 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d075b00b90 0x55d075bd6030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:05.042 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.032+0000 7faab88c1080 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55d075b471a0 0x55d075bd6570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:05.042 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.032+0000 7faab88c1080 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d075b400e0 con 0x55d075b00b90 2023-11-08T19:26:05.042 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.032+0000 7faab88c1080 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d075b3f4d0 con 0x55d075b007b0 2023-11-08T19:26:05.042 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.032+0000 7faab88c1080 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d075b3f7b0 con 0x55d075b471a0 2023-11-08T19:26:05.042 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.032+0000 7faaae155700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d075b00b90 0x55d075bd6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:05.043 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.032+0000 7faaae155700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d075b00b90 0x55d075bd6030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50916/0 (socket says 172.21.15.143:50916) 2023-11-08T19:26:05.043 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.032+0000 7faaae155700 1 -- 172.21.15.143:0/2792443135 learned_addr learned my addr 172.21.15.143:0/2792443135 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:05.043 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.032+0000 7faaaf157700 1 --2- 172.21.15.143:0/2792443135 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55d075b471a0 0x55d075bd6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:05.043 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.032+0000 7faaae956700 1 --2- 172.21.15.143:0/2792443135 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d075b007b0 0x55d075bca200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:05.043 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.032+0000 7faaae155700 1 -- 172.21.15.143:0/2792443135 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55d075b471a0 msgr2=0x55d075bd6570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:05.044 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.032+0000 7faaae155700 1 --2- 172.21.15.143:0/2792443135 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55d075b471a0 0x55d075bd6570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:05.044 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.032+0000 7faaae155700 1 -- 172.21.15.143:0/2792443135 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d075b007b0 msgr2=0x55d075bca200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:05.044 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.032+0000 7faaae155700 1 --2- 172.21.15.143:0/2792443135 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d075b007b0 0x55d075bca200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:05.044 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.032+0000 7faaae155700 1 -- 172.21.15.143:0/2792443135 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaa400a420 con 0x55d075b00b90 2023-11-08T19:26:05.044 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.033+0000 7faaae155700 1 --2- 172.21.15.143:0/2792443135 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d075b00b90 0x55d075bd6030 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7faaa4017040 tx=0x7faaa4003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:05.045 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.033+0000 7faa97fff700 1 -- 172.21.15.143:0/2792443135 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaa4007ea0 con 0x55d075b00b90 2023-11-08T19:26:05.045 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.033+0000 7faa97fff700 1 -- 172.21.15.143:0/2792443135 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faaa40148a0 con 0x55d075b00b90 2023-11-08T19:26:05.045 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.033+0000 7faa97fff700 1 -- 172.21.15.143:0/2792443135 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaa4014c50 con 0x55d075b00b90 2023-11-08T19:26:05.045 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.033+0000 7faab88c1080 1 -- 172.21.15.143:0/2792443135 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55d075a697c0 con 0x55d075b00b90 2023-11-08T19:26:05.045 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.033+0000 7faab88c1080 1 -- 172.21.15.143:0/2792443135 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d075a4f7d0 con 0x55d075b00b90 2023-11-08T19:26:05.046 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.034+0000 7faa97fff700 1 -- 172.21.15.143:0/2792443135 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7faaa401b040 con 0x55d075b00b90 2023-11-08T19:26:05.047 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.034+0000 7faa97fff700 1 --2- 172.21.15.143:0/2792443135 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7faa9c069850 0x7faa9c06bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:05.047 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.034+0000 7faa97fff700 1 -- 172.21.15.143:0/2792443135 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6101+0+0 (secure 0 0 0) 0x7faaa40896b0 con 0x55d075b00b90 2023-11-08T19:26:05.047 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.035+0000 7faab88c1080 1 -- 172.21.15.143:0/2792443135 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- pool_op(delete snap pool 6 tid 1 name snap3 v0) v4 -- 0x55d075ab34a0 con 0x55d075b00b90 2023-11-08T19:26:05.047 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.035+0000 7faaae956700 1 --2- 172.21.15.143:0/2792443135 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7faa9c069850 0x7faa9c06bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:05.048 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.035+0000 7faaae956700 1 --2- 172.21.15.143:0/2792443135 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7faa9c069850 0x7faa9c06bd10 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7faaa000a8c0 tx=0x7faaa000f000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:05.606 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.605+0000 7faa97fff700 1 -- 172.21.15.143:0/2792443135 <== mon.0 v2:172.21.15.143:3300/0 6 ==== pool_op_reply(tid 1 (0) Success v36) v1 ==== 43+0+0 (secure 0 0 0) 0x7faaa4054660 con 0x55d075b00b90 2023-11-08T19:26:05.606 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.605+0000 7faa97fff700 1 -- 172.21.15.143:0/2792443135 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=36}) v3 -- 0x7faa9c072090 con 0x55d075b00b90 2023-11-08T19:26:05.607 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.605+0000 7faa97fff700 1 -- 172.21.15.143:0/2792443135 <== mon.0 v2:172.21.15.143:3300/0 7 ==== osd_map(36..36 src has 1..36) v4 ==== 643+0+0 (secure 0 0 0) 0x7faaa4058a20 con 0x55d075b00b90 2023-11-08T19:26:05.607 INFO:teuthology.orchestra.run.smithi143.stdout:removed pool snap-test-pool snap snap3 2023-11-08T19:26:05.608 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.605+0000 7faab88c1080 1 -- 172.21.15.143:0/2792443135 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7faa9c069850 msgr2=0x7faa9c06bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:05.608 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.605+0000 7faab88c1080 1 --2- 172.21.15.143:0/2792443135 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7faa9c069850 0x7faa9c06bd10 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7faaa000a8c0 tx=0x7faaa000f000 comp rx=0 tx=0).stop 2023-11-08T19:26:05.609 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.605+0000 7faab88c1080 1 -- 172.21.15.143:0/2792443135 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d075b00b90 msgr2=0x55d075bd6030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:05.609 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.605+0000 7faab88c1080 1 --2- 172.21.15.143:0/2792443135 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d075b00b90 0x55d075bd6030 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7faaa4017040 tx=0x7faaa4003020 comp rx=0 tx=0).stop 2023-11-08T19:26:05.609 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.605+0000 7faab88c1080 1 -- 172.21.15.143:0/2792443135 shutdown_connections 2023-11-08T19:26:05.609 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.606+0000 7faab88c1080 1 --2- 172.21.15.143:0/2792443135 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7faa9c069850 0x7faa9c06bd10 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:05.609 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.606+0000 7faab88c1080 1 --2- 172.21.15.143:0/2792443135 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55d075b471a0 0x55d075bd6570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:05.610 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.606+0000 7faab88c1080 1 --2- 172.21.15.143:0/2792443135 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d075b007b0 0x55d075bca200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:05.610 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.606+0000 7faab88c1080 1 --2- 172.21.15.143:0/2792443135 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d075b00b90 0x55d075bd6030 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:05.610 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.606+0000 7faab88c1080 1 -- 172.21.15.143:0/2792443135 >> 172.21.15.143:0/2792443135 conn(0x55d075b4ac40 msgr2=0x55d075bd6eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:05.610 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.606+0000 7faab88c1080 1 -- 172.21.15.143:0/2792443135 shutdown_connections 2023-11-08T19:26:05.610 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.606+0000 7faab88c1080 1 -- 172.21.15.143:0/2792443135 wait complete. 2023-11-08T19:26:05.611 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete snap-test-pool 2023-11-08T19:26:05.738 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.737+0000 7f095884e700 1 Processor -- start 2023-11-08T19:26:05.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.737+0000 7f095884e700 1 -- start start 2023-11-08T19:26:05.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.738+0000 7f095884e700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f095415dba0 0x7f095406c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:05.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.738+0000 7f095884e700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f09540b22e0 0x7f09540b2770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:05.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.738+0000 7f095884e700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0954161890 con 0x7f095406b240 2023-11-08T19:26:05.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.738+0000 7f095884e700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09540b2cb0 con 0x7f09540b22e0 2023-11-08T19:26:05.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.738+0000 7f095884e700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f095406b740 con 0x7f095415dba0 2023-11-08T19:26:05.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.738+0000 7f0952d9d700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f09540b22e0 0x7f09540b2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:05.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.738+0000 7f0952d9d700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f09540b22e0 0x7f09540b2770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50928/0 (socket says 172.21.15.143:50928) 2023-11-08T19:26:05.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.738+0000 7f0952d9d700 1 -- 172.21.15.143:0/2540639361 learned_addr learned my addr 172.21.15.143:0/2540639361 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:05.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.738+0000 7f095259c700 1 --2- 172.21.15.143:0/2540639361 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f095415dba0 0x7f095406c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:05.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.738+0000 7f0952d9d700 1 -- 172.21.15.143:0/2540639361 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f095415dba0 msgr2=0x7f095406c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:05.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.738+0000 7f0952d9d700 1 --2- 172.21.15.143:0/2540639361 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f095415dba0 0x7f095406c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:05.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.738+0000 7f0952d9d700 1 -- 172.21.15.143:0/2540639361 >> v1:172.21.15.174:6789/0 conn(0x7f095406b240 legacy=0x7f09540caff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:05.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.738+0000 7f0952d9d700 1 -- 172.21.15.143:0/2540639361 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f095406ba20 con 0x7f09540b22e0 2023-11-08T19:26:05.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.738+0000 7f095259c700 1 --2- 172.21.15.143:0/2540639361 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f095415dba0 0x7f095406c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:26:05.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.738+0000 7f0952d9d700 1 --2- 172.21.15.143:0/2540639361 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f09540b22e0 0x7f09540b2770 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f094c009e50 tx=0x7f094c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cd19b81372aa78f server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:05.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.738+0000 7f095159a700 1 -- 172.21.15.143:0/2540639361 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f094c00e070 con 0x7f09540b22e0 2023-11-08T19:26:05.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.739+0000 7f095159a700 1 -- 172.21.15.143:0/2540639361 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f094c003a20 con 0x7f09540b22e0 2023-11-08T19:26:05.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.739+0000 7f095159a700 1 -- 172.21.15.143:0/2540639361 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f094c003dd0 con 0x7f09540b22e0 2023-11-08T19:26:05.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.739+0000 7f095884e700 1 -- 172.21.15.143:0/2540639361 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f09540b22e0 msgr2=0x7f09540b2770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:05.750 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.739+0000 7f095884e700 1 --2- 172.21.15.143:0/2540639361 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f09540b22e0 0x7f09540b2770 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f094c009e50 tx=0x7f094c00b330 comp rx=0 tx=0).stop 2023-11-08T19:26:05.750 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.739+0000 7f095884e700 1 -- 172.21.15.143:0/2540639361 shutdown_connections 2023-11-08T19:26:05.750 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.739+0000 7f095884e700 1 --2- 172.21.15.143:0/2540639361 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f09540b22e0 0x7f09540b2770 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:05.750 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.739+0000 7f095884e700 1 --2- 172.21.15.143:0/2540639361 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f095415dba0 0x7f095406c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:05.750 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.739+0000 7f095884e700 1 -- 172.21.15.143:0/2540639361 >> 172.21.15.143:0/2540639361 conn(0x7f09540cd2e0 msgr2=0x7f09540cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:05.751 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.739+0000 7f095884e700 1 -- 172.21.15.143:0/2540639361 shutdown_connections 2023-11-08T19:26:05.751 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.739+0000 7f095884e700 1 -- 172.21.15.143:0/2540639361 wait complete. 2023-11-08T19:26:05.751 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.740+0000 7f095884e700 1 Processor -- start 2023-11-08T19:26:05.751 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.740+0000 7f095884e700 1 -- start start 2023-11-08T19:26:05.751 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.741+0000 7f095884e700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f095406b240 0x7f09540c9f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:05.752 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.741+0000 7f095884e700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f09540b22e0 0x7f09540c6d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:05.752 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.741+0000 7f095884e700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f095415dba0 0x7f0954148390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:05.752 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.741+0000 7f095884e700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09540b64b0 con 0x7f09540b22e0 2023-11-08T19:26:05.752 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.741+0000 7f095884e700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09540c8e60 con 0x7f095415dba0 2023-11-08T19:26:05.752 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.741+0000 7f095884e700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09540c9140 con 0x7f095406b240 2023-11-08T19:26:05.753 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.741+0000 7f0951d9b700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f09540b22e0 0x7f09540c6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:05.753 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.741+0000 7f0951d9b700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f09540b22e0 0x7f09540c6d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50930/0 (socket says 172.21.15.143:50930) 2023-11-08T19:26:05.753 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.741+0000 7f0951d9b700 1 -- 172.21.15.143:0/3614155653 learned_addr learned my addr 172.21.15.143:0/3614155653 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:05.753 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.742+0000 7f095259c700 1 --2- 172.21.15.143:0/3614155653 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f095406b240 0x7f09540c9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:05.753 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.742+0000 7f0952d9d700 1 --2- 172.21.15.143:0/3614155653 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f095415dba0 0x7f0954148390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:05.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.742+0000 7f0951d9b700 1 -- 172.21.15.143:0/3614155653 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f095406b240 msgr2=0x7f09540c9f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:05.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.742+0000 7f0951d9b700 1 --2- 172.21.15.143:0/3614155653 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f095406b240 0x7f09540c9f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:05.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.742+0000 7f0951d9b700 1 -- 172.21.15.143:0/3614155653 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f095415dba0 msgr2=0x7f0954148390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:05.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.742+0000 7f0951d9b700 1 --2- 172.21.15.143:0/3614155653 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f095415dba0 0x7f0954148390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:05.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.742+0000 7f0951d9b700 1 -- 172.21.15.143:0/3614155653 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f094c009b30 con 0x7f09540b22e0 2023-11-08T19:26:05.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.742+0000 7f0952d9d700 1 --2- 172.21.15.143:0/3614155653 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f095415dba0 0x7f0954148390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:26:05.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.742+0000 7f0951d9b700 1 --2- 172.21.15.143:0/3614155653 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f09540b22e0 0x7f09540c6d60 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f094800c4d0 tx=0x7f094800c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:05.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.742+0000 7f0942ffd700 1 -- 172.21.15.143:0/3614155653 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0948011790 con 0x7f09540b22e0 2023-11-08T19:26:05.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.742+0000 7f0942ffd700 1 -- 172.21.15.143:0/3614155653 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09480118f0 con 0x7f09540b22e0 2023-11-08T19:26:05.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.742+0000 7f0942ffd700 1 -- 172.21.15.143:0/3614155653 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0948011ca0 con 0x7f09540b22e0 2023-11-08T19:26:05.756 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.742+0000 7f095884e700 1 -- 172.21.15.143:0/3614155653 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09540c9420 con 0x7f09540b22e0 2023-11-08T19:26:05.756 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.742+0000 7f095884e700 1 -- 172.21.15.143:0/3614155653 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09540c99e0 con 0x7f09540b22e0 2023-11-08T19:26:05.757 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.743+0000 7f095884e700 1 -- 172.21.15.143:0/3614155653 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f095404eee0 con 0x7f09540b22e0 2023-11-08T19:26:05.758 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.743+0000 7f0942ffd700 1 -- 172.21.15.143:0/3614155653 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f094801c460 con 0x7f09540b22e0 2023-11-08T19:26:05.758 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.744+0000 7f0942ffd700 1 --2- 172.21.15.143:0/3614155653 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f093406af00 0x7f093406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:05.758 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.744+0000 7f0942ffd700 1 -- 172.21.15.143:0/3614155653 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6118+0+0 (secure 0 0 0) 0x7f09480894c0 con 0x7f09540b22e0 2023-11-08T19:26:05.758 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.744+0000 7f095259c700 1 --2- 172.21.15.143:0/3614155653 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f093406af00 0x7f093406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:05.758 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.744+0000 7f095259c700 1 --2- 172.21.15.143:0/3614155653 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f093406af00 0x7f093406d3c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f0944006fd0 tx=0x7f0944008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:05.759 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.748+0000 7f0942ffd700 1 -- 172.21.15.143:0/3614155653 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0948054460 con 0x7f09540b22e0 2023-11-08T19:26:05.940 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.939+0000 7f095884e700 1 -- 172.21.15.143:0/3614155653 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "snap-test-pool"} v 0) v1 -- 0x7f09540b67e0 con 0x7f09540b22e0 2023-11-08T19:26:05.941 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:05.939+0000 7f0942ffd700 1 -- 172.21.15.143:0/3614155653 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "snap-test-pool"}]=-1 WARNING: this will *PERMANENTLY DESTROY* all data stored in pool snap-test-pool. If you are *ABSOLUTELY CERTAIN* that is what you want, pass the pool name *twice*, followed by --yes-i-really-really-mean-it. v36) v1 ==== 296+0+0 (secure 0 0 0) 0x7f0948057a80 con 0x7f09540b22e0 2023-11-08T19:26:05.942 INFO:teuthology.orchestra.run.smithi143.stderr:Error EPERM: WARNING: this will *PERMANENTLY DESTROY* all data stored in pool snap-test-pool. If you are *ABSOLUTELY CERTAIN* that is what you want, pass the pool name *twice*, followed by --yes-i-really-really-mean-it. 2023-11-08T19:26:05.946 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-08T19:26:05.947 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-08T19:26:05.947 DEBUG:teuthology.orchestra.run.smithi143:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-08T19:26:05.992 DEBUG:teuthology.orchestra.run.smithi174:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-08T19:26:06.042 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-08T19:26:06.043 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-08T19:26:06.043 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:26:06.060 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-08T19:26:06.061 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:26:06.117 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:26:06.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.275+0000 7f75da0d3700 1 Processor -- start 2023-11-08T19:26:06.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.275+0000 7f75da0d3700 1 -- start start 2023-11-08T19:26:06.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.276+0000 7f75da0d3700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f75d415dd00 0x7f75d40cca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:06.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.276+0000 7f75da0d3700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f75d41618f0 0x7f75d406ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:06.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.276+0000 7f75da0d3700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75d40ccfb0 con 0x7f75d406b150 2023-11-08T19:26:06.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.276+0000 7f75da0d3700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75d406b600 con 0x7f75d41618f0 2023-11-08T19:26:06.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.276+0000 7f75da0d3700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75d40cad50 con 0x7f75d415dd00 2023-11-08T19:26:06.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.276+0000 7f75d2ffd700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f75d41618f0 0x7f75d406ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:06.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.276+0000 7f75d2ffd700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f75d41618f0 0x7f75d406ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50940/0 (socket says 172.21.15.143:50940) 2023-11-08T19:26:06.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.276+0000 7f75d2ffd700 1 -- 172.21.15.143:0/3534285686 learned_addr learned my addr 172.21.15.143:0/3534285686 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:06.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.277+0000 7f75d2ffd700 1 -- 172.21.15.143:0/3534285686 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f75d415dd00 msgr2=0x7f75d40cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:06.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.277+0000 7f75d37fe700 1 --2- 172.21.15.143:0/3534285686 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f75d415dd00 0x7f75d40cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:06.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.277+0000 7f75d2ffd700 1 --2- 172.21.15.143:0/3534285686 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f75d415dd00 0x7f75d40cca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:06.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.277+0000 7f75d2ffd700 1 -- 172.21.15.143:0/3534285686 >> v1:172.21.15.174:6789/0 conn(0x7f75d406b150 legacy=0x7f75d40b2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:06.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.277+0000 7f75d2ffd700 1 -- 172.21.15.143:0/3534285686 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75d4161df0 con 0x7f75d41618f0 2023-11-08T19:26:06.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.277+0000 7f75d2ffd700 1 --2- 172.21.15.143:0/3534285686 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f75d41618f0 0x7f75d406ac10 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f75bc0098f0 tx=0x7f75bc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a2a046850688543 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:06.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.277+0000 7f75d27fc700 1 -- 172.21.15.143:0/3534285686 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75bc004230 con 0x7f75d41618f0 2023-11-08T19:26:06.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.277+0000 7f75d27fc700 1 -- 172.21.15.143:0/3534285686 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75bc004390 con 0x7f75d41618f0 2023-11-08T19:26:06.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.277+0000 7f75d27fc700 1 -- 172.21.15.143:0/3534285686 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75bc0036a0 con 0x7f75d41618f0 2023-11-08T19:26:06.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.277+0000 7f75da0d3700 1 -- 172.21.15.143:0/3534285686 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f75d41618f0 msgr2=0x7f75d406ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:06.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.277+0000 7f75da0d3700 1 --2- 172.21.15.143:0/3534285686 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f75d41618f0 0x7f75d406ac10 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f75bc0098f0 tx=0x7f75bc009c00 comp rx=0 tx=0).stop 2023-11-08T19:26:06.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.277+0000 7f75da0d3700 1 -- 172.21.15.143:0/3534285686 shutdown_connections 2023-11-08T19:26:06.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.277+0000 7f75da0d3700 1 --2- 172.21.15.143:0/3534285686 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f75d415dd00 0x7f75d40cca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:06.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.277+0000 7f75da0d3700 1 --2- 172.21.15.143:0/3534285686 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f75d41618f0 0x7f75d406ac10 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:06.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.278+0000 7f75da0d3700 1 -- 172.21.15.143:0/3534285686 >> 172.21.15.143:0/3534285686 conn(0x7f75d4099ab0 msgr2=0x7f75d40d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:06.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.278+0000 7f75da0d3700 1 -- 172.21.15.143:0/3534285686 shutdown_connections 2023-11-08T19:26:06.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.278+0000 7f75da0d3700 1 -- 172.21.15.143:0/3534285686 wait complete. 2023-11-08T19:26:06.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.279+0000 7f75da0d3700 1 Processor -- start 2023-11-08T19:26:06.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.279+0000 7f75da0d3700 1 -- start start 2023-11-08T19:26:06.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.280+0000 7f75da0d3700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f75d406b150 0x7f75d40c6b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:06.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.280+0000 7f75da0d3700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f75d415dd00 0x7f75d4154660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:06.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.280+0000 7f75da0d3700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f75d41618f0 0x7f75d41f6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:06.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.280+0000 7f75da0d3700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75d4154240 con 0x7f75d406b150 2023-11-08T19:26:06.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.280+0000 7f75da0d3700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75d41f6990 con 0x7f75d41618f0 2023-11-08T19:26:06.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.280+0000 7f75da0d3700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75d40c70b0 con 0x7f75d415dd00 2023-11-08T19:26:06.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.280+0000 7f75d37fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f75d406b150 0x7f75d40c6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:06.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.280+0000 7f75d37fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f75d406b150 0x7f75d40c6b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50944/0 (socket says 172.21.15.143:50944) 2023-11-08T19:26:06.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.280+0000 7f75d37fe700 1 -- 172.21.15.143:0/2646790937 learned_addr learned my addr 172.21.15.143:0/2646790937 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:06.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.281+0000 7f75d2ffd700 1 --2- 172.21.15.143:0/2646790937 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f75d415dd00 0x7f75d4154660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:06.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.281+0000 7f75d37fe700 1 -- 172.21.15.143:0/2646790937 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f75d415dd00 msgr2=0x7f75d4154660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:06.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.281+0000 7f75d37fe700 1 --2- 172.21.15.143:0/2646790937 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f75d415dd00 0x7f75d4154660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:06.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.281+0000 7f75d37fe700 1 -- 172.21.15.143:0/2646790937 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f75d41618f0 msgr2=0x7f75d41f6ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:06.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.281+0000 7f75d37fe700 1 --2- 172.21.15.143:0/2646790937 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f75d41618f0 0x7f75d41f6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:06.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.281+0000 7f75d37fe700 1 -- 172.21.15.143:0/2646790937 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75bc009550 con 0x7f75d406b150 2023-11-08T19:26:06.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.281+0000 7f75d37fe700 1 --2- 172.21.15.143:0/2646790937 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f75d406b150 0x7f75d40c6b70 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f75c400c820 tx=0x7f75c400cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:06.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.281+0000 7f75cbfff700 1 -- 172.21.15.143:0/2646790937 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75c4011800 con 0x7f75d406b150 2023-11-08T19:26:06.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.281+0000 7f75cbfff700 1 -- 172.21.15.143:0/2646790937 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75c4011960 con 0x7f75d406b150 2023-11-08T19:26:06.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.281+0000 7f75da0d3700 1 -- 172.21.15.143:0/2646790937 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75d41535c0 con 0x7f75d406b150 2023-11-08T19:26:06.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.281+0000 7f75cbfff700 1 -- 172.21.15.143:0/2646790937 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75c4011cf0 con 0x7f75d406b150 2023-11-08T19:26:06.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.281+0000 7f75da0d3700 1 -- 172.21.15.143:0/2646790937 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75d4153b80 con 0x7f75d406b150 2023-11-08T19:26:06.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.282+0000 7f75da0d3700 1 -- 172.21.15.143:0/2646790937 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75d40aff60 con 0x7f75d406b150 2023-11-08T19:26:06.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.282+0000 7f75cbfff700 1 -- 172.21.15.143:0/2646790937 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f75c4018360 con 0x7f75d406b150 2023-11-08T19:26:06.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.283+0000 7f75cbfff700 1 --2- 172.21.15.143:0/2646790937 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f75b406af00 0x7f75b406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:06.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.283+0000 7f75cbfff700 1 -- 172.21.15.143:0/2646790937 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6118+0+0 (secure 0 0 0) 0x7f75c4089670 con 0x7f75d406b150 2023-11-08T19:26:06.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.283+0000 7f75d2ffd700 1 --2- 172.21.15.143:0/2646790937 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f75b406af00 0x7f75b406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:06.293 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.284+0000 7f75d2ffd700 1 --2- 172.21.15.143:0/2646790937 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f75b406af00 0x7f75b406d3c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f75d4155130 tx=0x7f75bc008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:06.293 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.287+0000 7f75cbfff700 1 -- 172.21.15.143:0/2646790937 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f75c4054610 con 0x7f75d406b150 2023-11-08T19:26:06.484 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.483+0000 7f75da0d3700 1 -- 172.21.15.143:0/2646790937 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f75d404eee0 con 0x7f75d406b150 2023-11-08T19:26:06.485 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.484+0000 7f75cbfff700 1 -- 172.21.15.143:0/2646790937 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+18456 (secure 0 0 0) 0x7f75c4057c30 con 0x7f75d406b150 2023-11-08T19:26:06.486 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:26:06.486 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":36,"fsid":"968483af-77fc-4068-ad7c-5c2a1a4613d7","created":"2023-11-08T19:24:32.826484+0000","modified":"2023-11-08T19:26:05.598707+0000","last_up_change":"2023-11-08T19:24:43.573880+0000","last_in_change":"2023-11-08T19:24:35.314525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:44.551530+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-08T19:25:47.925761+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-08T19:25:49.224734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"snap-test-pool","create_time":"2023-11-08T19:26:02.874384+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":36,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bae98ec1-d94e-403d-a220-a522a53ea6f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6809","nonce":1487830566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6811","nonce":1487830566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6815","nonce":1487830566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6813","nonce":1487830566}]},"public_addr":"172.21.15.143:6809/1487830566","cluster_addr":"172.21.15.143:6811/1487830566","heartbeat_back_addr":"172.21.15.143:6815/1487830566","heartbeat_front_addr":"172.21.15.143:6813/1487830566","state":["exists","up"]},{"osd":1,"uuid":"5cb6538a-1ccb-4322-8633-09f66af0b7e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6801","nonce":787341397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6803","nonce":787341397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6807","nonce":787341397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6805","nonce":787341397}]},"public_addr":"172.21.15.143:6801/787341397","cluster_addr":"172.21.15.143:6803/787341397","heartbeat_back_addr":"172.21.15.143:6807/787341397","heartbeat_front_addr":"172.21.15.143:6805/787341397","state":["exists","up"]},{"osd":2,"uuid":"cdbf522c-60ef-4d60-9653-8dcb529c101a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6822","nonce":2301325597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6826","nonce":2301325597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6831","nonce":2301325597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6829","nonce":2301325597}]},"public_addr":"172.21.15.143:6822/2301325597","cluster_addr":"172.21.15.143:6826/2301325597","heartbeat_back_addr":"172.21.15.143:6831/2301325597","heartbeat_front_addr":"172.21.15.143:6829/2301325597","state":["exists","up"]},{"osd":3,"uuid":"4e2783cf-ebb9-4dd8-8245-bc1d17ce4325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6817","nonce":2021504428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6819","nonce":2021504428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6827","nonce":2021504428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6823","nonce":2021504428}]},"public_addr":"172.21.15.143:6817/2021504428","cluster_addr":"172.21.15.143:6819/2021504428","heartbeat_back_addr":"172.21.15.143:6827/2021504428","heartbeat_front_addr":"172.21.15.143:6823/2021504428","state":["exists","up"]},{"osd":4,"uuid":"691256fa-9b77-49b9-a346-7ae1adf0ca80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6801","nonce":1993380595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6803","nonce":1993380595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6807","nonce":1993380595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6805","nonce":1993380595}]},"public_addr":"172.21.15.174:6801/1993380595","cluster_addr":"172.21.15.174:6803/1993380595","heartbeat_back_addr":"172.21.15.174:6807/1993380595","heartbeat_front_addr":"172.21.15.174:6805/1993380595","state":["exists","up"]},{"osd":5,"uuid":"756fc352-48c9-46ce-b52e-0541fe6f8525","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6809","nonce":1777923303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6811","nonce":1777923303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6815","nonce":1777923303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6813","nonce":1777923303}]},"public_addr":"172.21.15.174:6809/1777923303","cluster_addr":"172.21.15.174:6811/1777923303","heartbeat_back_addr":"172.21.15.174:6815/1777923303","heartbeat_front_addr":"172.21.15.174:6813/1777923303","state":["exists","up"]},{"osd":6,"uuid":"fab2ac2b-ed97-4cde-9825-b480eabf8b8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6825","nonce":2755130316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6827","nonce":2755130316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6831","nonce":2755130316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6829","nonce":2755130316}]},"public_addr":"172.21.15.174:6825/2755130316","cluster_addr":"172.21.15.174:6827/2755130316","heartbeat_back_addr":"172.21.15.174:6831/2755130316","heartbeat_front_addr":"172.21.15.174:6829/2755130316","state":["exists","up"]},{"osd":7,"uuid":"3de9b797-2ab6-4d41-a863-7e7d638fd703","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6817","nonce":2847141411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6819","nonce":2847141411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6823","nonce":2847141411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6821","nonce":2847141411}]},"public_addr":"172.21.15.174:6817/2847141411","cluster_addr":"172.21.15.174:6819/2847141411","heartbeat_back_addr":"172.21.15.174:6823/2847141411","heartbeat_front_addr":"172.21.15.174:6821/2847141411","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.477076+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.496890+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.539305+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.454673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.538881+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.521294+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.520712+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.540404+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":1,"length":1}]}],"new_removed_snaps":[{"pool":6,"snaps":[{"begin":1,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:26:06.488 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.486+0000 7f75da0d3700 1 -- 172.21.15.143:0/2646790937 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f75b406af00 msgr2=0x7f75b406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:06.488 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.486+0000 7f75da0d3700 1 --2- 172.21.15.143:0/2646790937 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f75b406af00 0x7f75b406d3c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f75d4155130 tx=0x7f75bc008030 comp rx=0 tx=0).stop 2023-11-08T19:26:06.488 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.486+0000 7f75da0d3700 1 -- 172.21.15.143:0/2646790937 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f75d406b150 msgr2=0x7f75d40c6b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:06.488 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.486+0000 7f75da0d3700 1 --2- 172.21.15.143:0/2646790937 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f75d406b150 0x7f75d40c6b70 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f75c400c820 tx=0x7f75c400cbe0 comp rx=0 tx=0).stop 2023-11-08T19:26:06.489 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.486+0000 7f75da0d3700 1 -- 172.21.15.143:0/2646790937 shutdown_connections 2023-11-08T19:26:06.489 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.486+0000 7f75da0d3700 1 --2- 172.21.15.143:0/2646790937 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f75b406af00 0x7f75b406d3c0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:06.489 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.486+0000 7f75da0d3700 1 --2- 172.21.15.143:0/2646790937 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f75d41618f0 0x7f75d41f6ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:06.489 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.486+0000 7f75da0d3700 1 --2- 172.21.15.143:0/2646790937 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f75d406b150 0x7f75d40c6b70 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:06.489 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.486+0000 7f75da0d3700 1 --2- 172.21.15.143:0/2646790937 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f75d415dd00 0x7f75d4154660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:06.490 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.486+0000 7f75da0d3700 1 -- 172.21.15.143:0/2646790937 >> 172.21.15.143:0/2646790937 conn(0x7f75d4099ab0 msgr2=0x7f75d40cb780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:06.490 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.486+0000 7f75da0d3700 1 -- 172.21.15.143:0/2646790937 shutdown_connections 2023-11-08T19:26:06.490 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.486+0000 7f75da0d3700 1 -- 172.21.15.143:0/2646790937 wait complete. 2023-11-08T19:26:06.500 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T19:24:44.551530+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-08T19:25:47.925761+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2023-11-08T19:25:49.224734+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'snap-test-pool', 'create_time': '2023-11-08T19:26:02.874384+0000', 'flags': 16385, 'flags_names': 'hashpspool,pool_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '36', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'pool', 'snap_seq': 2, 'snap_epoch': 36, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-08T19:26:06.501 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T19:26:06.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.624+0000 7fb780e27700 1 Processor -- start 2023-11-08T19:26:06.626 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.624+0000 7fb780e27700 1 -- start start 2023-11-08T19:26:06.628 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.625+0000 7fb780e27700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb77c15a950 0x7fb77c0d01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:06.629 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.625+0000 7fb780e27700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fb77c06bd20 0x7fb77c072370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:06.629 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.625+0000 7fb780e27700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb77c0cc2a0 con 0x7fb77c0728b0 2023-11-08T19:26:06.629 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.625+0000 7fb780e27700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb77c0cc130 con 0x7fb77c15a950 2023-11-08T19:26:06.629 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.625+0000 7fb780e27700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb77c06c1b0 con 0x7fb77c06bd20 2023-11-08T19:26:06.630 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.625+0000 7fb77a59c700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb77c15a950 0x7fb77c0d01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:06.630 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.625+0000 7fb77ad9d700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fb77c0728b0 0x7fb77c0cc730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:47106/0 (socket says 172.21.15.143:47106) 2023-11-08T19:26:06.630 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.625+0000 7fb77ad9d700 1 -- 172.21.15.143:0/2145581275 learned_addr learned my addr 172.21.15.143:0/2145581275 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:26:06.630 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.625+0000 7fb77a59c700 1 -- 172.21.15.143:0/2145581275 >> v2:172.21.15.174:3300/0 conn(0x7fb77c06bd20 msgr2=0x7fb77c072370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:06.630 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.625+0000 7fb779d9b700 1 --2- 172.21.15.143:0/2145581275 >> v2:172.21.15.174:3300/0 conn(0x7fb77c06bd20 0x7fb77c072370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:06.631 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.625+0000 7fb77a59c700 1 --2- 172.21.15.143:0/2145581275 >> v2:172.21.15.174:3300/0 conn(0x7fb77c06bd20 0x7fb77c072370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:06.631 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.625+0000 7fb77a59c700 1 -- 172.21.15.143:0/2145581275 >> v1:172.21.15.143:6789/0 conn(0x7fb77c0728b0 legacy=0x7fb77c0cc730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:06.631 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.625+0000 7fb77959a700 1 -- 172.21.15.143:0/2145581275 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3588294774 0 0) 0x7fb77c0cc2a0 con 0x7fb77c0728b0 2023-11-08T19:26:06.631 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.625+0000 7fb77a59c700 1 -- 172.21.15.143:0/2145581275 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb77c06c490 con 0x7fb77c15a950 2023-11-08T19:26:06.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.625+0000 7fb779d9b700 1 --2- 172.21.15.143:0/2145581275 >> v2:172.21.15.174:3300/0 conn(0x7fb77c06bd20 0x7fb77c072370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:26:06.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.625+0000 7fb77a59c700 1 --2- 172.21.15.143:0/2145581275 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb77c15a950 0x7fb77c0d01b0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fb77c0cd240 tx=0x7fb774009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7dcc51b7eb1cc6b server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:06.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.626+0000 7fb77959a700 1 -- 172.21.15.143:0/2145581275 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb774011830 con 0x7fb77c15a950 2023-11-08T19:26:06.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.626+0000 7fb77959a700 1 -- 172.21.15.143:0/2145581275 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb774011990 con 0x7fb77c15a950 2023-11-08T19:26:06.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.626+0000 7fb77959a700 1 -- 172.21.15.143:0/2145581275 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb774011d20 con 0x7fb77c15a950 2023-11-08T19:26:06.633 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.626+0000 7fb780e27700 1 -- 172.21.15.143:0/2145581275 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb77c15a950 msgr2=0x7fb77c0d01b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:06.633 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.626+0000 7fb780e27700 1 --2- 172.21.15.143:0/2145581275 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb77c15a950 0x7fb77c0d01b0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fb77c0cd240 tx=0x7fb774009b20 comp rx=0 tx=0).stop 2023-11-08T19:26:06.633 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.626+0000 7fb780e27700 1 -- 172.21.15.143:0/2145581275 shutdown_connections 2023-11-08T19:26:06.633 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.626+0000 7fb780e27700 1 --2- 172.21.15.143:0/2145581275 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb77c15a950 0x7fb77c0d01b0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:06.634 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.626+0000 7fb780e27700 1 --2- 172.21.15.143:0/2145581275 >> v2:172.21.15.174:3300/0 conn(0x7fb77c06bd20 0x7fb77c072370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:06.634 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.626+0000 7fb780e27700 1 -- 172.21.15.143:0/2145581275 >> 172.21.15.143:0/2145581275 conn(0x7fb77c0cd2e0 msgr2=0x7fb77c0cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:06.634 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.626+0000 7fb780e27700 1 -- 172.21.15.143:0/2145581275 shutdown_connections 2023-11-08T19:26:06.634 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.626+0000 7fb780e27700 1 -- 172.21.15.143:0/2145581275 wait complete. 2023-11-08T19:26:06.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.628+0000 7fb780e27700 1 Processor -- start 2023-11-08T19:26:06.638 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.628+0000 7fb780e27700 1 -- start start 2023-11-08T19:26:06.638 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.628+0000 7fb780e27700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb77c0728b0 0x7fb77c147c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:06.638 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.628+0000 7fb780e27700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb77c15a950 0x7fb77c06ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:06.639 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.628+0000 7fb780e27700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb77c0b4da0 0x7fb77c06b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:06.639 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.628+0000 7fb780e27700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb77c1481b0 con 0x7fb77c0728b0 2023-11-08T19:26:06.639 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.628+0000 7fb780e27700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb77c0c8360 con 0x7fb77c15a950 2023-11-08T19:26:06.640 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.628+0000 7fb780e27700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb77c0b52a0 con 0x7fb77c0b4da0 2023-11-08T19:26:06.640 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.628+0000 7fb77a59c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb77c0728b0 0x7fb77c147c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:06.640 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.628+0000 7fb77a59c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb77c0728b0 0x7fb77c147c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50956/0 (socket says 172.21.15.143:50956) 2023-11-08T19:26:06.640 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.628+0000 7fb77a59c700 1 -- 172.21.15.143:0/705290137 learned_addr learned my addr 172.21.15.143:0/705290137 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:06.641 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.629+0000 7fb77ad9d700 1 --2- 172.21.15.143:0/705290137 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb77c0b4da0 0x7fb77c06b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:06.641 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.629+0000 7fb779d9b700 1 --2- 172.21.15.143:0/705290137 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb77c15a950 0x7fb77c06ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:06.641 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.629+0000 7fb77a59c700 1 -- 172.21.15.143:0/705290137 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb77c0b4da0 msgr2=0x7fb77c06b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:06.641 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.629+0000 7fb77a59c700 1 --2- 172.21.15.143:0/705290137 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb77c0b4da0 0x7fb77c06b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:06.642 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.629+0000 7fb77a59c700 1 -- 172.21.15.143:0/705290137 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb77c15a950 msgr2=0x7fb77c06ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:06.642 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.629+0000 7fb77a59c700 1 --2- 172.21.15.143:0/705290137 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb77c15a950 0x7fb77c06ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:06.642 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.629+0000 7fb77a59c700 1 -- 172.21.15.143:0/705290137 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7740094a0 con 0x7fb77c0728b0 2023-11-08T19:26:06.642 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.629+0000 7fb77a59c700 1 --2- 172.21.15.143:0/705290137 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb77c0728b0 0x7fb77c147c70 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fb774014040 tx=0x7fb774004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:06.642 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.629+0000 7fb772ffd700 1 -- 172.21.15.143:0/705290137 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7740038a0 con 0x7fb77c0728b0 2023-11-08T19:26:06.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.629+0000 7fb772ffd700 1 -- 172.21.15.143:0/705290137 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb77401e070 con 0x7fb77c0728b0 2023-11-08T19:26:06.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.629+0000 7fb780e27700 1 -- 172.21.15.143:0/705290137 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb77c0c7d10 con 0x7fb77c0728b0 2023-11-08T19:26:06.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.629+0000 7fb772ffd700 1 -- 172.21.15.143:0/705290137 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb774003c40 con 0x7fb77c0728b0 2023-11-08T19:26:06.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.629+0000 7fb780e27700 1 -- 172.21.15.143:0/705290137 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb77c1f3ca0 con 0x7fb77c0728b0 2023-11-08T19:26:06.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.630+0000 7fb780e27700 1 -- 172.21.15.143:0/705290137 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb77c067c00 con 0x7fb77c0728b0 2023-11-08T19:26:06.644 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.630+0000 7fb772ffd700 1 -- 172.21.15.143:0/705290137 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb77402d450 con 0x7fb77c0728b0 2023-11-08T19:26:06.644 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.631+0000 7fb772ffd700 1 --2- 172.21.15.143:0/705290137 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fb75c06af00 0x7fb75c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:06.644 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.631+0000 7fb772ffd700 1 -- 172.21.15.143:0/705290137 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6118+0+0 (secure 0 0 0) 0x7fb77408a870 con 0x7fb77c0728b0 2023-11-08T19:26:06.644 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.631+0000 7fb779d9b700 1 --2- 172.21.15.143:0/705290137 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fb75c06af00 0x7fb75c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:06.644 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.631+0000 7fb779d9b700 1 --2- 172.21.15.143:0/705290137 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fb75c06af00 0x7fb75c06d3c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb77c152450 tx=0x7fb764008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:06.645 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.635+0000 7fb772ffd700 1 -- 172.21.15.143:0/705290137 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb774055810 con 0x7fb77c0728b0 2023-11-08T19:26:06.824 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.823+0000 7fb780e27700 1 -- 172.21.15.143:0/705290137 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb77c15cf30 con 0x7fb77c0728b0 2023-11-08T19:26:06.824 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.823+0000 7fb772ffd700 1 -- 172.21.15.143:0/705290137 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v36) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb7740150a0 con 0x7fb77c0728b0 2023-11-08T19:26:06.825 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2023-11-08T19:26:06.827 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.825+0000 7fb780e27700 1 -- 172.21.15.143:0/705290137 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fb75c06af00 msgr2=0x7fb75c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:06.827 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.825+0000 7fb780e27700 1 --2- 172.21.15.143:0/705290137 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fb75c06af00 0x7fb75c06d3c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb77c152450 tx=0x7fb764008040 comp rx=0 tx=0).stop 2023-11-08T19:26:06.827 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.825+0000 7fb780e27700 1 -- 172.21.15.143:0/705290137 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb77c0728b0 msgr2=0x7fb77c147c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:06.828 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.825+0000 7fb780e27700 1 --2- 172.21.15.143:0/705290137 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb77c0728b0 0x7fb77c147c70 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fb774014040 tx=0x7fb774004750 comp rx=0 tx=0).stop 2023-11-08T19:26:06.828 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.825+0000 7fb780e27700 1 -- 172.21.15.143:0/705290137 shutdown_connections 2023-11-08T19:26:06.828 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.825+0000 7fb780e27700 1 --2- 172.21.15.143:0/705290137 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fb75c06af00 0x7fb75c06d3c0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:06.828 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.825+0000 7fb780e27700 1 --2- 172.21.15.143:0/705290137 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb77c0b4da0 0x7fb77c06b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:06.828 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.825+0000 7fb780e27700 1 --2- 172.21.15.143:0/705290137 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb77c0728b0 0x7fb77c147c70 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:06.828 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.825+0000 7fb780e27700 1 --2- 172.21.15.143:0/705290137 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb77c15a950 0x7fb77c06ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:06.829 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.825+0000 7fb780e27700 1 -- 172.21.15.143:0/705290137 >> 172.21.15.143:0/705290137 conn(0x7fb77c0cd2e0 msgr2=0x7fb77c15d630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:06.829 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.825+0000 7fb780e27700 1 -- 172.21.15.143:0/705290137 shutdown_connections 2023-11-08T19:26:06.829 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.825+0000 7fb780e27700 1 -- 172.21.15.143:0/705290137 wait complete. 2023-11-08T19:26:06.838 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-08T19:26:06.964 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.962+0000 7f550f8d7700 1 Processor -- start 2023-11-08T19:26:06.964 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.963+0000 7f550f8d7700 1 -- start start 2023-11-08T19:26:06.968 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.964+0000 7f550f8d7700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f550806b240 0x7f55080caff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:06.969 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.964+0000 7f550f8d7700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5508161890 con 0x7f55080cb530 2023-11-08T19:26:06.969 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.964+0000 7f550f8d7700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f550806b740 con 0x7f550815dba0 2023-11-08T19:26:06.969 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.964+0000 7f550f8d7700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f550806cd20 con 0x7f550806b240 2023-11-08T19:26:06.969 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.964+0000 7f550de74700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f55080cb530 0x7f55080b22e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:47122/0 (socket says 172.21.15.143:47122) 2023-11-08T19:26:06.970 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.964+0000 7f550de74700 1 -- 172.21.15.143:0/2720741721 learned_addr learned my addr 172.21.15.143:0/2720741721 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:26:06.970 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.964+0000 7f550ce72700 1 --2- 172.21.15.143:0/2720741721 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f550806b240 0x7f55080caff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:06.970 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.964+0000 7f54fffff700 1 -- 172.21.15.143:0/2720741721 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 760031157 0 0) 0x7f5508161890 con 0x7f55080cb530 2023-11-08T19:26:06.970 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.964+0000 7f54fffff700 1 -- 172.21.15.143:0/2720741721 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f54f40035b0 con 0x7f55080cb530 2023-11-08T19:26:06.973 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.964+0000 7f54fffff700 1 -- 172.21.15.143:0/2720741721 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 645780860 0 0) 0x7f54f40035b0 con 0x7f55080cb530 2023-11-08T19:26:06.973 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.964+0000 7f54fffff700 1 -- 172.21.15.143:0/2720741721 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f550806b240 msgr2=0x7f55080caff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:06.974 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.964+0000 7f54fffff700 1 --2- 172.21.15.143:0/2720741721 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f550806b240 0x7f55080caff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:06.974 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.964+0000 7f54fffff700 1 -- 172.21.15.143:0/2720741721 >> v1:172.21.15.174:6789/0 conn(0x7f550815dba0 legacy=0x7f550806c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:06.974 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.964+0000 7f54fffff700 1 -- 172.21.15.143:0/2720741721 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f550806ba20 con 0x7f55080cb530 2023-11-08T19:26:06.974 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.964+0000 7f54fffff700 1 -- 172.21.15.143:0/2720741721 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4147756303 0 0) 0x7f55000019f0 con 0x7f55080cb530 2023-11-08T19:26:06.974 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.964+0000 7f54fffff700 1 -- 172.21.15.143:0/2720741721 >> v1:172.21.15.143:6789/0 conn(0x7f55080cb530 legacy=0x7f55080b22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:06.975 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.965+0000 7f54fffff700 1 --2- 172.21.15.143:0/2720741721 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f54f4003a30 0x7f54f4003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:06.975 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.965+0000 7f54fffff700 1 --2- 172.21.15.143:0/2720741721 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f54f4004850 0x7f54f4004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:06.975 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.965+0000 7f54fffff700 1 --2- 172.21.15.143:0/2720741721 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f550806b240 0x7f54f4005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:06.975 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.965+0000 7f54fffff700 1 -- 172.21.15.143:0/2720741721 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f550806cd20 con 0x7f54f4003a30 2023-11-08T19:26:06.975 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.965+0000 7f54fffff700 1 -- 172.21.15.143:0/2720741721 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54f400bac0 con 0x7f54f4004850 2023-11-08T19:26:06.976 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.965+0000 7f54fffff700 1 -- 172.21.15.143:0/2720741721 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54f400bc80 con 0x7f550806b240 2023-11-08T19:26:06.976 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.965+0000 7f550d673700 1 --2- 172.21.15.143:0/2720741721 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f54f4003a30 0x7f54f4003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:06.976 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.965+0000 7f550ce72700 1 --2- 172.21.15.143:0/2720741721 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f54f4004850 0x7f54f4004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:06.976 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.965+0000 7f550de74700 1 --2- 172.21.15.143:0/2720741721 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f550806b240 0x7f54f4005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:06.976 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.965+0000 7f550d673700 1 -- 172.21.15.143:0/2720741721 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f550806b240 msgr2=0x7f54f4005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:06.977 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.965+0000 7f550d673700 1 --2- 172.21.15.143:0/2720741721 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f550806b240 0x7f54f4005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:06.977 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.965+0000 7f550d673700 1 -- 172.21.15.143:0/2720741721 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f54f4004850 msgr2=0x7f54f4004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:06.977 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.965+0000 7f550d673700 1 --2- 172.21.15.143:0/2720741721 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f54f4004850 0x7f54f4004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:06.977 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.965+0000 7f550d673700 1 -- 172.21.15.143:0/2720741721 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54f40035b0 con 0x7f54f4003a30 2023-11-08T19:26:06.977 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.965+0000 7f550de74700 1 --2- 172.21.15.143:0/2720741721 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f550806b240 0x7f54f4005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:26:06.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.965+0000 7f550d673700 1 --2- 172.21.15.143:0/2720741721 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f54f4003a30 0x7f54f4003e50 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f54f8007fd0 tx=0x7f54f4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16844d3935c317df server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:06.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.965+0000 7f54fffff700 1 -- 172.21.15.143:0/2720741721 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54f80052d0 con 0x7f54f4003a30 2023-11-08T19:26:06.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.965+0000 7f54fffff700 1 -- 172.21.15.143:0/2720741721 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54f8005470 con 0x7f54f4003a30 2023-11-08T19:26:06.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.965+0000 7f54fffff700 1 -- 172.21.15.143:0/2720741721 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54f80038c0 con 0x7f54f4003a30 2023-11-08T19:26:06.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.966+0000 7f550f8d7700 1 -- 172.21.15.143:0/2720741721 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f54f4003a30 msgr2=0x7f54f4003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:06.979 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.966+0000 7f550f8d7700 1 --2- 172.21.15.143:0/2720741721 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f54f4003a30 0x7f54f4003e50 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f54f8007fd0 tx=0x7f54f4005880 comp rx=0 tx=0).stop 2023-11-08T19:26:06.979 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.966+0000 7f550de74700 1 -- 172.21.15.143:0/2720741721 reap_dead start 2023-11-08T19:26:06.979 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.966+0000 7f550f8d7700 1 -- 172.21.15.143:0/2720741721 shutdown_connections 2023-11-08T19:26:06.979 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.966+0000 7f550f8d7700 1 -- 172.21.15.143:0/2720741721 >> 172.21.15.143:0/2720741721 conn(0x7f55080cd2e0 msgr2=0x7f55080cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:06.979 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.966+0000 7f550f8d7700 1 -- 172.21.15.143:0/2720741721 shutdown_connections 2023-11-08T19:26:06.980 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.966+0000 7f550f8d7700 1 -- 172.21.15.143:0/2720741721 wait complete. 2023-11-08T19:26:06.980 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.967+0000 7f550f8d7700 1 Processor -- start 2023-11-08T19:26:06.980 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.967+0000 7f550f8d7700 1 -- start start 2023-11-08T19:26:06.983 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.968+0000 7f550f8d7700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f54f4003a30 0x7f550815f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:06.983 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.968+0000 7f550f8d7700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f55080c6d40 0x7f55081f6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:06.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.968+0000 7f550f8d7700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f55081f73f0 0x7f55080c9130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:06.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.968+0000 7f550f8d7700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f550815f0c0 con 0x7f55080c6d40 2023-11-08T19:26:06.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.968+0000 7f550f8d7700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f550815ead0 con 0x7f54f4003a30 2023-11-08T19:26:06.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.968+0000 7f550f8d7700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f550815fa20 con 0x7f55081f73f0 2023-11-08T19:26:06.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.968+0000 7f550ce72700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f55080c6d40 0x7f55081f6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:06.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.968+0000 7f550ce72700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f55080c6d40 0x7f55081f6eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50978/0 (socket says 172.21.15.143:50978) 2023-11-08T19:26:06.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.968+0000 7f550ce72700 1 -- 172.21.15.143:0/3041775186 learned_addr learned my addr 172.21.15.143:0/3041775186 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:06.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.968+0000 7f550d673700 1 --2- 172.21.15.143:0/3041775186 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f54f4003a30 0x7f550815f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:06.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.968+0000 7f550de74700 1 --2- 172.21.15.143:0/3041775186 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f55081f73f0 0x7f55080c9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:06.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.968+0000 7f550ce72700 1 -- 172.21.15.143:0/3041775186 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f55081f73f0 msgr2=0x7f55080c9130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:06.986 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.968+0000 7f550ce72700 1 --2- 172.21.15.143:0/3041775186 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f55081f73f0 0x7f55080c9130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:06.986 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.968+0000 7f550ce72700 1 -- 172.21.15.143:0/3041775186 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f54f4003a30 msgr2=0x7f550815f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:06.986 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.968+0000 7f550ce72700 1 --2- 172.21.15.143:0/3041775186 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f54f4003a30 0x7f550815f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:06.986 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.968+0000 7f550ce72700 1 -- 172.21.15.143:0/3041775186 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55081f78e0 con 0x7f55080c6d40 2023-11-08T19:26:06.986 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.968+0000 7f550de74700 1 --2- 172.21.15.143:0/3041775186 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f55081f73f0 0x7f55080c9130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:26:06.987 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.968+0000 7f550ce72700 1 --2- 172.21.15.143:0/3041775186 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f55080c6d40 0x7f55081f6eb0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f5504011aa0 tx=0x7f5504011db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:06.987 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.968+0000 7f54fdffb700 1 -- 172.21.15.143:0/3041775186 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f550400f940 con 0x7f55080c6d40 2023-11-08T19:26:06.987 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.968+0000 7f54fdffb700 1 -- 172.21.15.143:0/3041775186 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f550400faa0 con 0x7f55080c6d40 2023-11-08T19:26:06.987 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.968+0000 7f550f8d7700 1 -- 172.21.15.143:0/3041775186 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55081f7bc0 con 0x7f55080c6d40 2023-11-08T19:26:06.987 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.968+0000 7f54fdffb700 1 -- 172.21.15.143:0/3041775186 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f550400fe50 con 0x7f55080c6d40 2023-11-08T19:26:06.988 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.969+0000 7f550f8d7700 1 -- 172.21.15.143:0/3041775186 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55081f3af0 con 0x7f55080c6d40 2023-11-08T19:26:06.988 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.970+0000 7f550f8d7700 1 -- 172.21.15.143:0/3041775186 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f550815f200 con 0x7f55080c6d40 2023-11-08T19:26:06.988 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.970+0000 7f54fdffb700 1 -- 172.21.15.143:0/3041775186 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f550401d400 con 0x7f55080c6d40 2023-11-08T19:26:06.988 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.970+0000 7f54fdffb700 1 --2- 172.21.15.143:0/3041775186 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f54ec06af00 0x7f54ec06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:06.988 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.970+0000 7f54fdffb700 1 -- 172.21.15.143:0/3041775186 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6118+0+0 (secure 0 0 0) 0x7f5504089d60 con 0x7f55080c6d40 2023-11-08T19:26:06.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.971+0000 7f550d673700 1 --2- 172.21.15.143:0/3041775186 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f54ec06af00 0x7f54ec06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:06.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.971+0000 7f550d673700 1 --2- 172.21.15.143:0/3041775186 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f54ec06af00 0x7f54ec06d3c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f54f8007fd0 tx=0x7f54f8009030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:06.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:06.975+0000 7f54fdffb700 1 -- 172.21.15.143:0/3041775186 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5504054d30 con 0x7f55080c6d40 2023-11-08T19:26:07.166 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.164+0000 7f550f8d7700 1 -- 172.21.15.143:0/3041775186 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f550804eee0 con 0x7f55080c6d40 2023-11-08T19:26:07.166 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.164+0000 7f54fdffb700 1 -- 172.21.15.143:0/3041775186 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v36) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5504058350 con 0x7f55080c6d40 2023-11-08T19:26:07.166 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2023-11-08T19:26:07.168 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.166+0000 7f550f8d7700 1 -- 172.21.15.143:0/3041775186 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f54ec06af00 msgr2=0x7f54ec06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:07.168 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.166+0000 7f550f8d7700 1 --2- 172.21.15.143:0/3041775186 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f54ec06af00 0x7f54ec06d3c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f54f8007fd0 tx=0x7f54f8009030 comp rx=0 tx=0).stop 2023-11-08T19:26:07.168 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.166+0000 7f550f8d7700 1 -- 172.21.15.143:0/3041775186 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f55080c6d40 msgr2=0x7f55081f6eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:07.169 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.166+0000 7f550f8d7700 1 --2- 172.21.15.143:0/3041775186 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f55080c6d40 0x7f55081f6eb0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f5504011aa0 tx=0x7f5504011db0 comp rx=0 tx=0).stop 2023-11-08T19:26:07.169 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.166+0000 7f550f8d7700 1 -- 172.21.15.143:0/3041775186 shutdown_connections 2023-11-08T19:26:07.169 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.166+0000 7f550f8d7700 1 --2- 172.21.15.143:0/3041775186 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f54ec06af00 0x7f54ec06d3c0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:07.169 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.166+0000 7f550f8d7700 1 --2- 172.21.15.143:0/3041775186 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f55081f73f0 0x7f55080c9130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:07.170 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.166+0000 7f550f8d7700 1 --2- 172.21.15.143:0/3041775186 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f54f4003a30 0x7f550815f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:07.170 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.166+0000 7f550f8d7700 1 --2- 172.21.15.143:0/3041775186 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f55080c6d40 0x7f55081f6eb0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:07.170 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.166+0000 7f550f8d7700 1 -- 172.21.15.143:0/3041775186 >> 172.21.15.143:0/3041775186 conn(0x7f55080cd2e0 msgr2=0x7f550815cd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:07.171 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.166+0000 7f550f8d7700 1 -- 172.21.15.143:0/3041775186 shutdown_connections 2023-11-08T19:26:07.171 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.166+0000 7f550f8d7700 1 -- 172.21.15.143:0/3041775186 wait complete. 2023-11-08T19:26:07.180 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-08T19:26:07.303 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.301+0000 7fc39327a700 1 Processor -- start 2023-11-08T19:26:07.303 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.301+0000 7fc39327a700 1 -- start start 2023-11-08T19:26:07.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.302+0000 7fc39327a700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fc38c15bb20 0x7fc38c0b2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:07.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.302+0000 7fc39327a700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc38c0b1730 0x7fc38c0d1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:07.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.302+0000 7fc39327a700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc38c0d1bb0 0x7fc38c0ce420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:07.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.302+0000 7fc39327a700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc38c0cf600 con 0x7fc38c0b1730 2023-11-08T19:26:07.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.302+0000 7fc39327a700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc38c0d0420 con 0x7fc38c15bb20 2023-11-08T19:26:07.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.302+0000 7fc39327a700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc38c0d2020 con 0x7fc38c0d1bb0 2023-11-08T19:26:07.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.302+0000 7fc390815700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc38c0b1730 0x7fc38c0d1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:07.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.302+0000 7fc390815700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc38c0b1730 0x7fc38c0d1670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50984/0 (socket says 172.21.15.143:50984) 2023-11-08T19:26:07.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.302+0000 7fc390815700 1 -- 172.21.15.143:0/3487339633 learned_addr learned my addr 172.21.15.143:0/3487339633 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:07.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.303+0000 7fc391817700 1 --2- 172.21.15.143:0/3487339633 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc38c0d1bb0 0x7fc38c0ce420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:07.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.303+0000 7fc391016700 1 --2- 172.21.15.143:0/3487339633 >> v2:172.21.15.174:3300/0 conn(0x7fc38c15bb20 0x7fc38c0b2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:07.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.303+0000 7fc390815700 1 -- 172.21.15.143:0/3487339633 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc38c0d1bb0 msgr2=0x7fc38c0ce420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:07.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.303+0000 7fc390815700 1 --2- 172.21.15.143:0/3487339633 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc38c0d1bb0 0x7fc38c0ce420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:07.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.303+0000 7fc390815700 1 -- 172.21.15.143:0/3487339633 >> v2:172.21.15.174:3300/0 conn(0x7fc38c15bb20 msgr2=0x7fc38c0b2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:07.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.303+0000 7fc390815700 1 --2- 172.21.15.143:0/3487339633 >> v2:172.21.15.174:3300/0 conn(0x7fc38c15bb20 0x7fc38c0b2670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:07.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.303+0000 7fc390815700 1 -- 172.21.15.143:0/3487339633 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc38c0cf810 con 0x7fc38c0b1730 2023-11-08T19:26:07.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.303+0000 7fc391016700 1 --2- 172.21.15.143:0/3487339633 >> v2:172.21.15.174:3300/0 conn(0x7fc38c15bb20 0x7fc38c0b2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:26:07.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.303+0000 7fc390815700 1 --2- 172.21.15.143:0/3487339633 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc38c0b1730 0x7fc38c0d1670 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fc38c08b850 tx=0x7fc388009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f12cb35b96bd18c server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:07.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.303+0000 7fc383fff700 1 -- 172.21.15.143:0/3487339633 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc38800b870 con 0x7fc38c0b1730 2023-11-08T19:26:07.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.303+0000 7fc383fff700 1 -- 172.21.15.143:0/3487339633 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc38800b9d0 con 0x7fc38c0b1730 2023-11-08T19:26:07.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.303+0000 7fc383fff700 1 -- 172.21.15.143:0/3487339633 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc38800bd60 con 0x7fc38c0b1730 2023-11-08T19:26:07.315 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.303+0000 7fc39327a700 1 -- 172.21.15.143:0/3487339633 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc38c0b1730 msgr2=0x7fc38c0d1670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:07.315 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.303+0000 7fc39327a700 1 --2- 172.21.15.143:0/3487339633 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc38c0b1730 0x7fc38c0d1670 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fc38c08b850 tx=0x7fc388009bd0 comp rx=0 tx=0).stop 2023-11-08T19:26:07.315 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.304+0000 7fc39327a700 1 -- 172.21.15.143:0/3487339633 shutdown_connections 2023-11-08T19:26:07.315 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.304+0000 7fc39327a700 1 --2- 172.21.15.143:0/3487339633 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc38c0d1bb0 0x7fc38c0ce420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:07.315 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.304+0000 7fc39327a700 1 --2- 172.21.15.143:0/3487339633 >> v2:172.21.15.174:3300/0 conn(0x7fc38c15bb20 0x7fc38c0b2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:07.316 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.304+0000 7fc39327a700 1 --2- 172.21.15.143:0/3487339633 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc38c0b1730 0x7fc38c0d1670 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:07.316 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.304+0000 7fc39327a700 1 -- 172.21.15.143:0/3487339633 >> 172.21.15.143:0/3487339633 conn(0x7fc38c08b000 msgr2=0x7fc38c09ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:07.316 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.304+0000 7fc39327a700 1 -- 172.21.15.143:0/3487339633 shutdown_connections 2023-11-08T19:26:07.316 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.304+0000 7fc39327a700 1 -- 172.21.15.143:0/3487339633 wait complete. 2023-11-08T19:26:07.316 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.305+0000 7fc39327a700 1 Processor -- start 2023-11-08T19:26:07.316 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.305+0000 7fc39327a700 1 -- start start 2023-11-08T19:26:07.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.305+0000 7fc39327a700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc38c0b1730 0x7fc38c1f5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:07.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.305+0000 7fc39327a700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc38c0d1bb0 0x7fc38c070210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:07.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.306+0000 7fc39327a700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc38c15bb20 0x7fc38c1f0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:07.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.306+0000 7fc39327a700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc38c0ca900 con 0x7fc38c0d1bb0 2023-11-08T19:26:07.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.306+0000 7fc39327a700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc38c1f56b0 con 0x7fc38c15bb20 2023-11-08T19:26:07.318 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.306+0000 7fc39327a700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc38c0c9cf0 con 0x7fc38c0b1730 2023-11-08T19:26:07.318 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.306+0000 7fc390815700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc38c0d1bb0 0x7fc38c070210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:07.318 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.306+0000 7fc390815700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc38c0d1bb0 0x7fc38c070210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50990/0 (socket says 172.21.15.143:50990) 2023-11-08T19:26:07.318 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.306+0000 7fc390815700 1 -- 172.21.15.143:0/4002277533 learned_addr learned my addr 172.21.15.143:0/4002277533 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:07.318 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.306+0000 7fc391817700 1 --2- 172.21.15.143:0/4002277533 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc38c15bb20 0x7fc38c1f0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:07.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.306+0000 7fc391016700 1 --2- 172.21.15.143:0/4002277533 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc38c0b1730 0x7fc38c1f5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:07.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.306+0000 7fc390815700 1 -- 172.21.15.143:0/4002277533 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc38c0b1730 msgr2=0x7fc38c1f5170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:07.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.306+0000 7fc390815700 1 --2- 172.21.15.143:0/4002277533 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc38c0b1730 0x7fc38c1f5170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:07.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.306+0000 7fc390815700 1 -- 172.21.15.143:0/4002277533 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc38c15bb20 msgr2=0x7fc38c1f0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:07.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.306+0000 7fc390815700 1 --2- 172.21.15.143:0/4002277533 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc38c15bb20 0x7fc38c1f0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:07.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.306+0000 7fc390815700 1 -- 172.21.15.143:0/4002277533 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc388009550 con 0x7fc38c0d1bb0 2023-11-08T19:26:07.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.306+0000 7fc390815700 1 --2- 172.21.15.143:0/4002277533 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc38c0d1bb0 0x7fc38c070210 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fc388000c00 tx=0x7fc388004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:07.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.306+0000 7fc381ffb700 1 -- 172.21.15.143:0/4002277533 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc388004520 con 0x7fc38c0d1bb0 2023-11-08T19:26:07.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.306+0000 7fc381ffb700 1 -- 172.21.15.143:0/4002277533 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc38801e070 con 0x7fc38c0d1bb0 2023-11-08T19:26:07.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.306+0000 7fc39327a700 1 -- 172.21.15.143:0/4002277533 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc38c0c9fd0 con 0x7fc38c0d1bb0 2023-11-08T19:26:07.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.306+0000 7fc381ffb700 1 -- 172.21.15.143:0/4002277533 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3880038c0 con 0x7fc38c0d1bb0 2023-11-08T19:26:07.321 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.306+0000 7fc39327a700 1 -- 172.21.15.143:0/4002277533 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc38c0ca590 con 0x7fc38c0d1bb0 2023-11-08T19:26:07.321 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.307+0000 7fc39327a700 1 -- 172.21.15.143:0/4002277533 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc38c0ca720 con 0x7fc38c0d1bb0 2023-11-08T19:26:07.321 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.308+0000 7fc381ffb700 1 -- 172.21.15.143:0/4002277533 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc388003a20 con 0x7fc38c0d1bb0 2023-11-08T19:26:07.321 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.308+0000 7fc381ffb700 1 --2- 172.21.15.143:0/4002277533 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc37006af00 0x7fc37006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:07.321 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.308+0000 7fc381ffb700 1 -- 172.21.15.143:0/4002277533 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6118+0+0 (secure 0 0 0) 0x7fc388020070 con 0x7fc38c0d1bb0 2023-11-08T19:26:07.322 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.309+0000 7fc391016700 1 --2- 172.21.15.143:0/4002277533 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc37006af00 0x7fc37006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:07.322 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.309+0000 7fc391016700 1 --2- 172.21.15.143:0/4002277533 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc37006af00 0x7fc37006d3c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fc37c006fd0 tx=0x7fc37c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:07.322 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.313+0000 7fc381ffb700 1 -- 172.21.15.143:0/4002277533 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc388055bf0 con 0x7fc38c0d1bb0 2023-11-08T19:26:07.504 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.503+0000 7fc39327a700 1 -- 172.21.15.143:0/4002277533 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc38c04eee0 con 0x7fc38c0d1bb0 2023-11-08T19:26:07.504 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.503+0000 7fc381ffb700 1 -- 172.21.15.143:0/4002277533 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v36) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc388015020 con 0x7fc38c0d1bb0 2023-11-08T19:26:07.505 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 64 2023-11-08T19:26:07.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.505+0000 7fc39327a700 1 -- 172.21.15.143:0/4002277533 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc37006af00 msgr2=0x7fc37006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:07.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.505+0000 7fc39327a700 1 --2- 172.21.15.143:0/4002277533 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc37006af00 0x7fc37006d3c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fc37c006fd0 tx=0x7fc37c008040 comp rx=0 tx=0).stop 2023-11-08T19:26:07.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.505+0000 7fc39327a700 1 -- 172.21.15.143:0/4002277533 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc38c0d1bb0 msgr2=0x7fc38c070210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:07.508 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.505+0000 7fc39327a700 1 --2- 172.21.15.143:0/4002277533 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc38c0d1bb0 0x7fc38c070210 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fc388000c00 tx=0x7fc388004750 comp rx=0 tx=0).stop 2023-11-08T19:26:07.508 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.505+0000 7fc39327a700 1 -- 172.21.15.143:0/4002277533 shutdown_connections 2023-11-08T19:26:07.508 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.505+0000 7fc39327a700 1 --2- 172.21.15.143:0/4002277533 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc37006af00 0x7fc37006d3c0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:07.508 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.505+0000 7fc39327a700 1 --2- 172.21.15.143:0/4002277533 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc38c0b1730 0x7fc38c1f5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:07.508 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.505+0000 7fc39327a700 1 --2- 172.21.15.143:0/4002277533 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc38c15bb20 0x7fc38c1f0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:07.509 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.505+0000 7fc39327a700 1 --2- 172.21.15.143:0/4002277533 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc38c0d1bb0 0x7fc38c070210 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:07.509 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.505+0000 7fc39327a700 1 -- 172.21.15.143:0/4002277533 >> 172.21.15.143:0/4002277533 conn(0x7fc38c08b000 msgr2=0x7fc38c0cd780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:07.509 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.505+0000 7fc39327a700 1 -- 172.21.15.143:0/4002277533 shutdown_connections 2023-11-08T19:26:07.509 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.505+0000 7fc39327a700 1 -- 172.21.15.143:0/4002277533 wait complete. 2023-11-08T19:26:07.519 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get snap-test-pool pg_num 2023-11-08T19:26:07.645 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.643+0000 7fcb2ce64700 1 Processor -- start 2023-11-08T19:26:07.645 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.643+0000 7fcb2ce64700 1 -- start start 2023-11-08T19:26:07.649 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.644+0000 7fcb2ce64700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcb2815a950 0x7fcb280d01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:07.649 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.644+0000 7fcb2ce64700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fcb2806bd20 0x7fcb28072370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:07.649 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.644+0000 7fcb2ce64700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb280cc2a0 con 0x7fcb280728b0 2023-11-08T19:26:07.649 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.644+0000 7fcb2ce64700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb280cc130 con 0x7fcb2806bd20 2023-11-08T19:26:07.650 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.644+0000 7fcb2ce64700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb2806c1b0 con 0x7fcb2815a950 2023-11-08T19:26:07.650 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.645+0000 7fcb26d9d700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fcb280728b0 0x7fcb280cc730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:47154/0 (socket says 172.21.15.143:47154) 2023-11-08T19:26:07.650 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.645+0000 7fcb26d9d700 1 -- 172.21.15.143:0/2052859197 learned_addr learned my addr 172.21.15.143:0/2052859197 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:26:07.650 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.645+0000 7fcb2659c700 1 --2- 172.21.15.143:0/2052859197 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcb2815a950 0x7fcb280d01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:07.651 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.645+0000 7fcb25d9b700 1 --2- 172.21.15.143:0/2052859197 >> v2:172.21.15.174:3300/0 conn(0x7fcb2806bd20 0x7fcb28072370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:07.651 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.645+0000 7fcb2559a700 1 -- 172.21.15.143:0/2052859197 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 377282995 0 0) 0x7fcb280cc2a0 con 0x7fcb280728b0 2023-11-08T19:26:07.651 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.645+0000 7fcb2559a700 1 -- 172.21.15.143:0/2052859197 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb100035b0 con 0x7fcb280728b0 2023-11-08T19:26:07.651 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.645+0000 7fcb2559a700 1 -- 172.21.15.143:0/2052859197 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3801682971 0 0) 0x7fcb100035b0 con 0x7fcb280728b0 2023-11-08T19:26:07.651 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.645+0000 7fcb2559a700 1 -- 172.21.15.143:0/2052859197 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcb2815a950 msgr2=0x7fcb280d01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:07.652 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.645+0000 7fcb2559a700 1 --2- 172.21.15.143:0/2052859197 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcb2815a950 0x7fcb280d01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:07.652 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.645+0000 7fcb2559a700 1 -- 172.21.15.143:0/2052859197 >> v2:172.21.15.174:3300/0 conn(0x7fcb2806bd20 msgr2=0x7fcb28072370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:07.652 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.645+0000 7fcb2559a700 1 --2- 172.21.15.143:0/2052859197 >> v2:172.21.15.174:3300/0 conn(0x7fcb2806bd20 0x7fcb28072370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:07.652 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.645+0000 7fcb2559a700 1 -- 172.21.15.143:0/2052859197 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb2806c490 con 0x7fcb280728b0 2023-11-08T19:26:07.652 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.645+0000 7fcb2559a700 1 -- 172.21.15.143:0/2052859197 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4147756303 0 0) 0x7fcb200019f0 con 0x7fcb280728b0 2023-11-08T19:26:07.653 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.645+0000 7fcb25d9b700 1 --2- 172.21.15.143:0/2052859197 >> v2:172.21.15.174:3300/0 conn(0x7fcb2806bd20 0x7fcb28072370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T19:26:07.653 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.645+0000 7fcb2559a700 1 -- 172.21.15.143:0/2052859197 >> v1:172.21.15.143:6789/0 conn(0x7fcb280728b0 legacy=0x7fcb280cc730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:07.653 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.645+0000 7fcb2559a700 1 --2- 172.21.15.143:0/2052859197 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcb10003a00 0x7fcb10003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:07.653 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.645+0000 7fcb2559a700 1 --2- 172.21.15.143:0/2052859197 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcb10004850 0x7fcb10004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:07.653 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.645+0000 7fcb2559a700 1 --2- 172.21.15.143:0/2052859197 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcb2815a950 0x7fcb10005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:07.654 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.645+0000 7fcb2559a700 1 -- 172.21.15.143:0/2052859197 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb2806c1b0 con 0x7fcb10003a00 2023-11-08T19:26:07.654 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.645+0000 7fcb2559a700 1 -- 172.21.15.143:0/2052859197 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb1000ba10 con 0x7fcb10004850 2023-11-08T19:26:07.654 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.645+0000 7fcb2559a700 1 -- 172.21.15.143:0/2052859197 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb1000bb50 con 0x7fcb2815a950 2023-11-08T19:26:07.655 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.645+0000 7fcb2659c700 1 --2- 172.21.15.143:0/2052859197 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcb10003a00 0x7fcb10003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:07.655 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.645+0000 7fcb26d9d700 1 --2- 172.21.15.143:0/2052859197 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcb2815a950 0x7fcb10005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:07.655 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.645+0000 7fcb25d9b700 1 --2- 172.21.15.143:0/2052859197 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcb10004850 0x7fcb10004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:07.656 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.646+0000 7fcb2659c700 1 -- 172.21.15.143:0/2052859197 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcb2815a950 msgr2=0x7fcb10005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:07.656 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.646+0000 7fcb2659c700 1 --2- 172.21.15.143:0/2052859197 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcb2815a950 0x7fcb10005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:07.656 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.646+0000 7fcb2659c700 1 -- 172.21.15.143:0/2052859197 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcb10004850 msgr2=0x7fcb10004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:07.656 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.646+0000 7fcb2659c700 1 --2- 172.21.15.143:0/2052859197 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcb10004850 0x7fcb10004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:07.656 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.646+0000 7fcb2659c700 1 -- 172.21.15.143:0/2052859197 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb100035b0 con 0x7fcb10003a00 2023-11-08T19:26:07.657 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.646+0000 7fcb25d9b700 1 --2- 172.21.15.143:0/2052859197 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcb10004850 0x7fcb10004cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:26:07.657 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.646+0000 7fcb2659c700 1 --2- 172.21.15.143:0/2052859197 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcb10003a00 0x7fcb10003e20 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fcb1c009b90 tx=0x7fcb10005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d0474ce4b0fdf05 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:07.657 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.646+0000 7fcb2559a700 1 -- 172.21.15.143:0/2052859197 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb1c00e6c0 con 0x7fcb10003a00 2023-11-08T19:26:07.657 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.646+0000 7fcb2559a700 1 -- 172.21.15.143:0/2052859197 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb1c00e820 con 0x7fcb10003a00 2023-11-08T19:26:07.657 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.646+0000 7fcb2559a700 1 -- 172.21.15.143:0/2052859197 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb1c00eb90 con 0x7fcb10003a00 2023-11-08T19:26:07.658 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.646+0000 7fcb2ce64700 1 -- 172.21.15.143:0/2052859197 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcb10003a00 msgr2=0x7fcb10003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:07.658 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.646+0000 7fcb2ce64700 1 --2- 172.21.15.143:0/2052859197 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcb10003a00 0x7fcb10003e20 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fcb1c009b90 tx=0x7fcb10005880 comp rx=0 tx=0).stop 2023-11-08T19:26:07.658 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.646+0000 7fcb26d9d700 1 -- 172.21.15.143:0/2052859197 reap_dead start 2023-11-08T19:26:07.658 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.646+0000 7fcb2ce64700 1 -- 172.21.15.143:0/2052859197 shutdown_connections 2023-11-08T19:26:07.658 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.646+0000 7fcb2ce64700 1 -- 172.21.15.143:0/2052859197 >> 172.21.15.143:0/2052859197 conn(0x7fcb280cd2e0 msgr2=0x7fcb280cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:07.659 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.647+0000 7fcb2ce64700 1 -- 172.21.15.143:0/2052859197 shutdown_connections 2023-11-08T19:26:07.659 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.647+0000 7fcb2ce64700 1 -- 172.21.15.143:0/2052859197 wait complete. 2023-11-08T19:26:07.662 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.648+0000 7fcb2ce64700 1 Processor -- start 2023-11-08T19:26:07.662 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.648+0000 7fcb2ce64700 1 -- start start 2023-11-08T19:26:07.662 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.649+0000 7fcb2ce64700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcb10003a00 0x7fcb2806ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:07.662 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.649+0000 7fcb2ce64700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcb280b5960 0x7fcb280a84e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:07.663 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.649+0000 7fcb2ce64700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcb280b5de0 0x7fcb280a8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:07.663 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.649+0000 7fcb2ce64700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb280b54d0 con 0x7fcb280b5960 2023-11-08T19:26:07.663 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.649+0000 7fcb2ce64700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb280b4150 con 0x7fcb280b5de0 2023-11-08T19:26:07.663 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.649+0000 7fcb2ce64700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb280b4430 con 0x7fcb10003a00 2023-11-08T19:26:07.663 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.649+0000 7fcb25d9b700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcb280b5960 0x7fcb280a84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:07.664 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.649+0000 7fcb25d9b700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcb280b5960 0x7fcb280a84e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51008/0 (socket says 172.21.15.143:51008) 2023-11-08T19:26:07.664 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.649+0000 7fcb25d9b700 1 -- 172.21.15.143:0/3513588960 learned_addr learned my addr 172.21.15.143:0/3513588960 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:07.664 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.650+0000 7fcb2659c700 1 --2- 172.21.15.143:0/3513588960 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcb10003a00 0x7fcb2806ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:07.664 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.650+0000 7fcb26d9d700 1 --2- 172.21.15.143:0/3513588960 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcb280b5de0 0x7fcb280a8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:07.664 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.650+0000 7fcb25d9b700 1 -- 172.21.15.143:0/3513588960 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcb10003a00 msgr2=0x7fcb2806ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:07.665 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.650+0000 7fcb25d9b700 1 --2- 172.21.15.143:0/3513588960 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcb10003a00 0x7fcb2806ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:07.665 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.650+0000 7fcb25d9b700 1 -- 172.21.15.143:0/3513588960 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcb280b5de0 msgr2=0x7fcb280a8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:07.665 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.650+0000 7fcb25d9b700 1 --2- 172.21.15.143:0/3513588960 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcb280b5de0 0x7fcb280a8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:07.665 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.650+0000 7fcb25d9b700 1 -- 172.21.15.143:0/3513588960 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb280b4e80 con 0x7fcb280b5960 2023-11-08T19:26:07.665 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.650+0000 7fcb2659c700 1 --2- 172.21.15.143:0/3513588960 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcb10003a00 0x7fcb2806ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:26:07.666 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.650+0000 7fcb26d9d700 1 --2- 172.21.15.143:0/3513588960 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcb280b5de0 0x7fcb280a8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:26:07.666 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.650+0000 7fcb25d9b700 1 --2- 172.21.15.143:0/3513588960 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcb280b5960 0x7fcb280a84e0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fcb18010c30 tx=0x7fcb18009eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:07.666 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.650+0000 7fcb16ffd700 1 -- 172.21.15.143:0/3513588960 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb1800e9c0 con 0x7fcb280b5960 2023-11-08T19:26:07.666 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.650+0000 7fcb16ffd700 1 -- 172.21.15.143:0/3513588960 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb1800eb20 con 0x7fcb280b5960 2023-11-08T19:26:07.666 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.650+0000 7fcb2ce64700 1 -- 172.21.15.143:0/3513588960 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb280b5160 con 0x7fcb280b5960 2023-11-08T19:26:07.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.650+0000 7fcb16ffd700 1 -- 172.21.15.143:0/3513588960 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb1800ee90 con 0x7fcb280b5960 2023-11-08T19:26:07.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.650+0000 7fcb2ce64700 1 -- 172.21.15.143:0/3513588960 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb2806b5a0 con 0x7fcb280b5960 2023-11-08T19:26:07.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.651+0000 7fcb2ce64700 1 -- 172.21.15.143:0/3513588960 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb280b5640 con 0x7fcb280b5960 2023-11-08T19:26:07.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.651+0000 7fcb16ffd700 1 -- 172.21.15.143:0/3513588960 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fcb18016460 con 0x7fcb280b5960 2023-11-08T19:26:07.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.652+0000 7fcb16ffd700 1 --2- 172.21.15.143:0/3513588960 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fcb0806af00 0x7fcb0806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:07.668 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.652+0000 7fcb16ffd700 1 -- 172.21.15.143:0/3513588960 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6118+0+0 (secure 0 0 0) 0x7fcb18089af0 con 0x7fcb280b5960 2023-11-08T19:26:07.668 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.652+0000 7fcb2659c700 1 --2- 172.21.15.143:0/3513588960 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fcb0806af00 0x7fcb0806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:07.668 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.652+0000 7fcb2659c700 1 --2- 172.21.15.143:0/3513588960 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fcb0806af00 0x7fcb0806d3c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fcb1c009b30 tx=0x7fcb1c008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:07.668 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.656+0000 7fcb16ffd700 1 -- 172.21.15.143:0/3513588960 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcb18054a00 con 0x7fcb280b5960 2023-11-08T19:26:07.846 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.844+0000 7fcb2ce64700 1 -- 172.21.15.143:0/3513588960 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"} v 0) v1 -- 0x7fcb2806b5a0 con 0x7fcb280b5960 2023-11-08T19:26:07.846 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.844+0000 7fcb16ffd700 1 -- 172.21.15.143:0/3513588960 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"}]=0 v36) v1 ==== 103+0+11 (secure 0 0 0) 0x7fcb2806b5a0 con 0x7fcb280b5960 2023-11-08T19:26:07.847 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2023-11-08T19:26:07.848 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.847+0000 7fcb2ce64700 1 -- 172.21.15.143:0/3513588960 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fcb0806af00 msgr2=0x7fcb0806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:07.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.847+0000 7fcb2ce64700 1 --2- 172.21.15.143:0/3513588960 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fcb0806af00 0x7fcb0806d3c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fcb1c009b30 tx=0x7fcb1c008030 comp rx=0 tx=0).stop 2023-11-08T19:26:07.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.847+0000 7fcb2ce64700 1 -- 172.21.15.143:0/3513588960 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcb280b5960 msgr2=0x7fcb280a84e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:07.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.847+0000 7fcb2ce64700 1 --2- 172.21.15.143:0/3513588960 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcb280b5960 0x7fcb280a84e0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fcb18010c30 tx=0x7fcb18009eb0 comp rx=0 tx=0).stop 2023-11-08T19:26:07.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.847+0000 7fcb2ce64700 1 -- 172.21.15.143:0/3513588960 shutdown_connections 2023-11-08T19:26:07.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.847+0000 7fcb2ce64700 1 --2- 172.21.15.143:0/3513588960 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fcb0806af00 0x7fcb0806d3c0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:07.850 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.847+0000 7fcb2ce64700 1 --2- 172.21.15.143:0/3513588960 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcb10003a00 0x7fcb2806ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:07.850 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.847+0000 7fcb2ce64700 1 --2- 172.21.15.143:0/3513588960 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcb280b5de0 0x7fcb280a8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:07.850 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.847+0000 7fcb2ce64700 1 --2- 172.21.15.143:0/3513588960 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcb280b5960 0x7fcb280a84e0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:07.850 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.847+0000 7fcb2ce64700 1 -- 172.21.15.143:0/3513588960 >> 172.21.15.143:0/3513588960 conn(0x7fcb280cd2e0 msgr2=0x7fcb2815d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:07.850 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.847+0000 7fcb2ce64700 1 -- 172.21.15.143:0/3513588960 shutdown_connections 2023-11-08T19:26:07.851 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.847+0000 7fcb2ce64700 1 -- 172.21.15.143:0/3513588960 wait complete. 2023-11-08T19:26:07.860 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-08T19:26:07.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.980+0000 7f116f95f700 1 Processor -- start 2023-11-08T19:26:07.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.980+0000 7f116f95f700 1 -- start start 2023-11-08T19:26:07.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.981+0000 7f116f95f700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1168159680 0x7f11680cc7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:07.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.981+0000 7f116f95f700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f11680b7200 0x7f11680cf3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:07.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.981+0000 7f116f95f700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f11680b3260 0x7f11680b3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:07.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.981+0000 7f116f95f700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11680b3c50 con 0x7f11680b3260 2023-11-08T19:26:07.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.981+0000 7f116f95f700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f116815ce10 con 0x7f11680b7200 2023-11-08T19:26:07.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.981+0000 7f116f95f700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f116815c290 con 0x7f1168159680 2023-11-08T19:26:07.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.981+0000 7f116defc700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f11680b3260 0x7f11680b3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:07.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.981+0000 7f116defc700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f11680b3260 0x7f11680b3710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51018/0 (socket says 172.21.15.143:51018) 2023-11-08T19:26:07.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.981+0000 7f116defc700 1 -- 172.21.15.143:0/2166553980 learned_addr learned my addr 172.21.15.143:0/2166553980 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:07.986 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.981+0000 7f116d6fb700 1 --2- 172.21.15.143:0/2166553980 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1168159680 0x7f11680cc7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:07.986 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.981+0000 7f116cefa700 1 --2- 172.21.15.143:0/2166553980 >> v2:172.21.15.174:3300/0 conn(0x7f11680b7200 0x7f11680cf3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:07.986 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.981+0000 7f116defc700 1 -- 172.21.15.143:0/2166553980 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1168159680 msgr2=0x7f11680cc7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:07.986 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.981+0000 7f116defc700 1 --2- 172.21.15.143:0/2166553980 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1168159680 0x7f11680cc7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:07.986 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.981+0000 7f116defc700 1 -- 172.21.15.143:0/2166553980 >> v2:172.21.15.174:3300/0 conn(0x7f11680b7200 msgr2=0x7f11680cf3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:07.987 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.981+0000 7f116defc700 1 --2- 172.21.15.143:0/2166553980 >> v2:172.21.15.174:3300/0 conn(0x7f11680b7200 0x7f11680cf3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:07.987 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.981+0000 7f116defc700 1 -- 172.21.15.143:0/2166553980 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f116815c570 con 0x7f11680b3260 2023-11-08T19:26:07.987 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.981+0000 7f116defc700 1 --2- 172.21.15.143:0/2166553980 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f11680b3260 0x7f11680b3710 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f116806ebb0 tx=0x7f116000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e6e099a1abb4437 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:07.987 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.981+0000 7f115ffff700 1 -- 172.21.15.143:0/2166553980 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f116000e070 con 0x7f11680b3260 2023-11-08T19:26:07.987 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.982+0000 7f115ffff700 1 -- 172.21.15.143:0/2166553980 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f116000b910 con 0x7f11680b3260 2023-11-08T19:26:07.988 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.982+0000 7f115ffff700 1 -- 172.21.15.143:0/2166553980 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f116000bcc0 con 0x7f11680b3260 2023-11-08T19:26:07.988 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.982+0000 7f116f95f700 1 -- 172.21.15.143:0/2166553980 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f11680b3260 msgr2=0x7f11680b3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:07.988 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.982+0000 7f116f95f700 1 --2- 172.21.15.143:0/2166553980 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f11680b3260 0x7f11680b3710 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f116806ebb0 tx=0x7f116000b330 comp rx=0 tx=0).stop 2023-11-08T19:26:07.988 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.982+0000 7f116f95f700 1 -- 172.21.15.143:0/2166553980 shutdown_connections 2023-11-08T19:26:07.988 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.982+0000 7f116f95f700 1 --2- 172.21.15.143:0/2166553980 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f11680b3260 0x7f11680b3710 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:07.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.982+0000 7f116f95f700 1 --2- 172.21.15.143:0/2166553980 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1168159680 0x7f11680cc7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:07.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.982+0000 7f116f95f700 1 --2- 172.21.15.143:0/2166553980 >> v2:172.21.15.174:3300/0 conn(0x7f11680b7200 0x7f11680cf3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:07.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.982+0000 7f116f95f700 1 -- 172.21.15.143:0/2166553980 >> 172.21.15.143:0/2166553980 conn(0x7f11680ce260 msgr2=0x7f11680ce670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:07.990 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.982+0000 7f116f95f700 1 -- 172.21.15.143:0/2166553980 shutdown_connections 2023-11-08T19:26:07.990 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.982+0000 7f116f95f700 1 -- 172.21.15.143:0/2166553980 wait complete. 2023-11-08T19:26:07.993 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.983+0000 7f116f95f700 1 Processor -- start 2023-11-08T19:26:07.993 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.983+0000 7f116f95f700 1 -- start start 2023-11-08T19:26:07.993 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.984+0000 7f116f95f700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f11680b3260 0x7f1168157970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:07.994 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.984+0000 7f116f95f700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f11680b7200 0x7f1168159a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:07.994 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.984+0000 7f116f95f700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1168159680 0x7f116806cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:07.994 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.984+0000 7f116f95f700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f116806c930 con 0x7f11680b3260 2023-11-08T19:26:07.994 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.984+0000 7f116f95f700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f116806bd20 con 0x7f1168159680 2023-11-08T19:26:07.994 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.984+0000 7f116f95f700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f116806c000 con 0x7f11680b7200 2023-11-08T19:26:07.995 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.984+0000 7f116d6fb700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f11680b3260 0x7f1168157970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:07.995 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.984+0000 7f116d6fb700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f11680b3260 0x7f1168157970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51020/0 (socket says 172.21.15.143:51020) 2023-11-08T19:26:07.995 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.984+0000 7f116d6fb700 1 -- 172.21.15.143:0/365499149 learned_addr learned my addr 172.21.15.143:0/365499149 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:07.995 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.984+0000 7f116cefa700 1 --2- 172.21.15.143:0/365499149 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f11680b7200 0x7f1168159a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:07.995 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.984+0000 7f116defc700 1 --2- 172.21.15.143:0/365499149 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1168159680 0x7f116806cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:07.996 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.984+0000 7f116d6fb700 1 -- 172.21.15.143:0/365499149 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f11680b7200 msgr2=0x7f1168159a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:07.996 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.984+0000 7f116d6fb700 1 --2- 172.21.15.143:0/365499149 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f11680b7200 0x7f1168159a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:07.996 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.984+0000 7f116d6fb700 1 -- 172.21.15.143:0/365499149 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1168159680 msgr2=0x7f116806cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:07.996 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.984+0000 7f116d6fb700 1 --2- 172.21.15.143:0/365499149 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1168159680 0x7f116806cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:07.996 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.984+0000 7f116d6fb700 1 -- 172.21.15.143:0/365499149 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1160009b30 con 0x7f11680b3260 2023-11-08T19:26:07.997 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.984+0000 7f116defc700 1 --2- 172.21.15.143:0/365499149 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1168159680 0x7f116806cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:26:07.997 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.985+0000 7f116d6fb700 1 --2- 172.21.15.143:0/365499149 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f11680b3260 0x7f1168157970 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f116400e870 tx=0x7f116400ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:07.997 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.985+0000 7f115dffb700 1 -- 172.21.15.143:0/365499149 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1164009590 con 0x7f11680b3260 2023-11-08T19:26:07.997 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.985+0000 7f115dffb700 1 -- 172.21.15.143:0/365499149 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f116400ee70 con 0x7f11680b3260 2023-11-08T19:26:07.998 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.985+0000 7f116f95f700 1 -- 172.21.15.143:0/365499149 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f116806c2e0 con 0x7f11680b3260 2023-11-08T19:26:07.998 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.985+0000 7f115dffb700 1 -- 172.21.15.143:0/365499149 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f116400ca50 con 0x7f11680b3260 2023-11-08T19:26:07.998 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.985+0000 7f116f95f700 1 -- 172.21.15.143:0/365499149 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1168157eb0 con 0x7f11680b3260 2023-11-08T19:26:07.998 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.986+0000 7f116f95f700 1 -- 172.21.15.143:0/365499149 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f116806cc10 con 0x7f11680b3260 2023-11-08T19:26:07.998 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.987+0000 7f115dffb700 1 -- 172.21.15.143:0/365499149 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f116400cbb0 con 0x7f11680b3260 2023-11-08T19:26:07.999 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.987+0000 7f115dffb700 1 --2- 172.21.15.143:0/365499149 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f114c06af30 0x7f114c06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:07.999 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.987+0000 7f115dffb700 1 -- 172.21.15.143:0/365499149 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6118+0+0 (secure 0 0 0) 0x7f1164089e70 con 0x7f11680b3260 2023-11-08T19:26:07.999 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.988+0000 7f116cefa700 1 --2- 172.21.15.143:0/365499149 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f114c06af30 0x7f114c06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:07.999 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.988+0000 7f116cefa700 1 --2- 172.21.15.143:0/365499149 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f114c06af30 0x7f114c06d3f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f11680c82f0 tx=0x7f1158008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:08.001 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:07.991+0000 7f115dffb700 1 -- 172.21.15.143:0/365499149 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1164054dd0 con 0x7f11680b3260 2023-11-08T19:26:08.196 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.195+0000 7f116f95f700 1 -- 172.21.15.143:0/365499149 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f116815cb00 con 0x7f11680b3260 2023-11-08T19:26:08.196 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.195+0000 7f115dffb700 1 -- 172.21.15.143:0/365499149 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v36) v1 ==== 80+0+0 (secure 0 0 0) 0x7f1164021080 con 0x7f11680b3260 2023-11-08T19:26:08.199 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.197+0000 7f116f95f700 1 -- 172.21.15.143:0/365499149 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f114c06af30 msgr2=0x7f114c06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:08.199 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.197+0000 7f116f95f700 1 --2- 172.21.15.143:0/365499149 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f114c06af30 0x7f114c06d3f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f11680c82f0 tx=0x7f1158008040 comp rx=0 tx=0).stop 2023-11-08T19:26:08.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.197+0000 7f116f95f700 1 -- 172.21.15.143:0/365499149 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f11680b3260 msgr2=0x7f1168157970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:08.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.197+0000 7f116f95f700 1 --2- 172.21.15.143:0/365499149 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f11680b3260 0x7f1168157970 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f116400e870 tx=0x7f116400ec30 comp rx=0 tx=0).stop 2023-11-08T19:26:08.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.197+0000 7f116f95f700 1 -- 172.21.15.143:0/365499149 shutdown_connections 2023-11-08T19:26:08.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.197+0000 7f116f95f700 1 --2- 172.21.15.143:0/365499149 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f114c06af30 0x7f114c06d3f0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:08.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.197+0000 7f116f95f700 1 --2- 172.21.15.143:0/365499149 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1168159680 0x7f116806cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:08.201 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.197+0000 7f116f95f700 1 --2- 172.21.15.143:0/365499149 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f11680b3260 0x7f1168157970 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:08.201 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.197+0000 7f116f95f700 1 --2- 172.21.15.143:0/365499149 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f11680b7200 0x7f1168159a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:08.201 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.197+0000 7f116f95f700 1 -- 172.21.15.143:0/365499149 >> 172.21.15.143:0/365499149 conn(0x7f11680ce260 msgr2=0x7f116815e390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:08.201 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.197+0000 7f116f95f700 1 -- 172.21.15.143:0/365499149 shutdown_connections 2023-11-08T19:26:08.201 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.198+0000 7f116f95f700 1 -- 172.21.15.143:0/365499149 wait complete. 2023-11-08T19:26:08.202 INFO:teuthology.orchestra.run.smithi143.stderr:listed 0 entries 2023-11-08T19:26:08.211 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-08T19:26:08.212 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:26:08.212 DEBUG:teuthology.orchestra.run.smithi143:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-11-08T19:26:08.255 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi143.front.sepia.ceph.com 2023-11-08T19:26:08.256 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:26:08.256 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-08T19:26:08.313 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2023-11-08T19:26:08.314 DEBUG:teuthology.orchestra.run.smithi174:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:26:08.335 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284224k,nr_inodes=4071056,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=31,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19549 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260544k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-08T19:26:08.336 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-08T19:26:08.336 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:26:08.371 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284264k,nr_inodes=4071066,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=44,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20643 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-08T19:26:08.372 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2023-11-08T19:26:08.372 DEBUG:teuthology.orchestra.run.smithi174:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:26:08.395 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284224k,nr_inodes=4071056,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=31,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19549 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260544k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-08T19:26:08.396 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:26:08.530 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.529+0000 7f833a151700 1 Processor -- start 2023-11-08T19:26:08.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.529+0000 7f833a151700 1 -- start start 2023-11-08T19:26:08.533 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.530+0000 7f833a151700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f833415dd00 0x7f83340cca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:08.533 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.530+0000 7f833a151700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f83341618f0 0x7f833406ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:08.533 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.530+0000 7f833a151700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83340ccfb0 con 0x7f833406b150 2023-11-08T19:26:08.534 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.530+0000 7f833a151700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f833406b600 con 0x7f833415dd00 2023-11-08T19:26:08.534 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.530+0000 7f833a151700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83340cad50 con 0x7f83341618f0 2023-11-08T19:26:08.534 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.530+0000 7f83337fe700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f833415dd00 0x7f83340cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:08.534 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.530+0000 7f83337fe700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f833415dd00 0x7f83340cca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51028/0 (socket says 172.21.15.143:51028) 2023-11-08T19:26:08.534 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.530+0000 7f83337fe700 1 -- 172.21.15.143:0/1407864696 learned_addr learned my addr 172.21.15.143:0/1407864696 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:08.535 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.530+0000 7f8332ffd700 1 --2- 172.21.15.143:0/1407864696 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f83341618f0 0x7f833406ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:08.535 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.530+0000 7f83337fe700 1 -- 172.21.15.143:0/1407864696 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f83341618f0 msgr2=0x7f833406ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:08.535 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.530+0000 7f83337fe700 1 --2- 172.21.15.143:0/1407864696 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f83341618f0 0x7f833406ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:08.535 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.530+0000 7f83337fe700 1 -- 172.21.15.143:0/1407864696 >> v1:172.21.15.174:6789/0 conn(0x7f833406b150 legacy=0x7f83340b2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:08.535 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.530+0000 7f83337fe700 1 -- 172.21.15.143:0/1407864696 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8334161df0 con 0x7f833415dd00 2023-11-08T19:26:08.536 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.530+0000 7f8332ffd700 1 --2- 172.21.15.143:0/1407864696 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f83341618f0 0x7f833406ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:26:08.536 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.531+0000 7f83337fe700 1 --2- 172.21.15.143:0/1407864696 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f833415dd00 0x7f83340cca70 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f8324006fd0 tx=0x7f8324009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9efbfdcee7bab975 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:08.536 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.531+0000 7f83327fc700 1 -- 172.21.15.143:0/1407864696 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8324005e30 con 0x7f833415dd00 2023-11-08T19:26:08.536 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.531+0000 7f83327fc700 1 -- 172.21.15.143:0/1407864696 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8324004030 con 0x7f833415dd00 2023-11-08T19:26:08.536 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.531+0000 7f83327fc700 1 -- 172.21.15.143:0/1407864696 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83240043e0 con 0x7f833415dd00 2023-11-08T19:26:08.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.531+0000 7f833a151700 1 -- 172.21.15.143:0/1407864696 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f833415dd00 msgr2=0x7f83340cca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:08.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.531+0000 7f833a151700 1 --2- 172.21.15.143:0/1407864696 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f833415dd00 0x7f83340cca70 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f8324006fd0 tx=0x7f8324009bd0 comp rx=0 tx=0).stop 2023-11-08T19:26:08.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.531+0000 7f833a151700 1 -- 172.21.15.143:0/1407864696 shutdown_connections 2023-11-08T19:26:08.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.531+0000 7f833a151700 1 --2- 172.21.15.143:0/1407864696 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f833415dd00 0x7f83340cca70 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:08.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.531+0000 7f833a151700 1 --2- 172.21.15.143:0/1407864696 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f83341618f0 0x7f833406ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:08.538 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.531+0000 7f833a151700 1 -- 172.21.15.143:0/1407864696 >> 172.21.15.143:0/1407864696 conn(0x7f8334099ab0 msgr2=0x7f83340d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:08.538 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.531+0000 7f833a151700 1 -- 172.21.15.143:0/1407864696 shutdown_connections 2023-11-08T19:26:08.538 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.531+0000 7f833a151700 1 -- 172.21.15.143:0/1407864696 wait complete. 2023-11-08T19:26:08.541 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.532+0000 7f833a151700 1 Processor -- start 2023-11-08T19:26:08.541 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.533+0000 7f833a151700 1 -- start start 2023-11-08T19:26:08.541 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.533+0000 7f833a151700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f833406b150 0x7f83340b00f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:08.541 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.533+0000 7f833a151700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f833415dd00 0x7f83340b0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:08.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.533+0000 7f833a151700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f83341618f0 0x7f83340b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:08.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.533+0000 7f833a151700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83340c6c80 con 0x7f83341618f0 2023-11-08T19:26:08.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.533+0000 7f833a151700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f833415ebb0 con 0x7f833415dd00 2023-11-08T19:26:08.543 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.533+0000 7f833a151700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f833415e280 con 0x7f833406b150 2023-11-08T19:26:08.543 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.533+0000 7f8333fff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f83341618f0 0x7f83340b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:08.543 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.534+0000 7f8333fff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f83341618f0 0x7f83340b2000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51036/0 (socket says 172.21.15.143:51036) 2023-11-08T19:26:08.543 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.534+0000 7f8333fff700 1 -- 172.21.15.143:0/2064325665 learned_addr learned my addr 172.21.15.143:0/2064325665 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:08.543 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.534+0000 7f83337fe700 1 --2- 172.21.15.143:0/2064325665 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f833406b150 0x7f83340b00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:08.544 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.534+0000 7f8332ffd700 1 --2- 172.21.15.143:0/2064325665 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f833415dd00 0x7f83340b0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:08.544 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.534+0000 7f8333fff700 1 -- 172.21.15.143:0/2064325665 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f833406b150 msgr2=0x7f83340b00f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:08.544 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.534+0000 7f8333fff700 1 --2- 172.21.15.143:0/2064325665 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f833406b150 0x7f83340b00f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:08.544 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.534+0000 7f8333fff700 1 -- 172.21.15.143:0/2064325665 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f833415dd00 msgr2=0x7f83340b0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:08.544 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.534+0000 7f8333fff700 1 --2- 172.21.15.143:0/2064325665 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f833415dd00 0x7f83340b0630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:08.545 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.534+0000 7f8333fff700 1 -- 172.21.15.143:0/2064325665 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8324009500 con 0x7f83341618f0 2023-11-08T19:26:08.545 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.534+0000 7f8333fff700 1 --2- 172.21.15.143:0/2064325665 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f83341618f0 0x7f83340b2000 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f832c00cc50 tx=0x7f832c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:08.545 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.534+0000 7f831bfff700 1 -- 172.21.15.143:0/2064325665 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f832c00ad80 con 0x7f83341618f0 2023-11-08T19:26:08.545 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.534+0000 7f831bfff700 1 -- 172.21.15.143:0/2064325665 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f832c00ce50 con 0x7f83341618f0 2023-11-08T19:26:08.545 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.534+0000 7f833a151700 1 -- 172.21.15.143:0/2064325665 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f833415e560 con 0x7f83341618f0 2023-11-08T19:26:08.546 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.534+0000 7f831bfff700 1 -- 172.21.15.143:0/2064325665 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f832c003c40 con 0x7f83341618f0 2023-11-08T19:26:08.546 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.534+0000 7f833a151700 1 -- 172.21.15.143:0/2064325665 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83340b2540 con 0x7f83341618f0 2023-11-08T19:26:08.546 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.535+0000 7f833a151700 1 -- 172.21.15.143:0/2064325665 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83340aff60 con 0x7f83341618f0 2023-11-08T19:26:08.547 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.535+0000 7f831bfff700 1 -- 172.21.15.143:0/2064325665 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f832c0050d0 con 0x7f83341618f0 2023-11-08T19:26:08.547 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.536+0000 7f831bfff700 1 --2- 172.21.15.143:0/2064325665 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f832006af00 0x7f832006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:08.547 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.536+0000 7f831bfff700 1 -- 172.21.15.143:0/2064325665 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6118+0+0 (secure 0 0 0) 0x7f832c0898a0 con 0x7f83341618f0 2023-11-08T19:26:08.547 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.536+0000 7f83337fe700 1 --2- 172.21.15.143:0/2064325665 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f832006af00 0x7f832006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:08.547 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.536+0000 7f83337fe700 1 --2- 172.21.15.143:0/2064325665 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f832006af00 0x7f832006d3c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f8324014040 tx=0x7f8324008070 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:08.548 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.540+0000 7f831bfff700 1 -- 172.21.15.143:0/2064325665 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f832c0547b0 con 0x7f83341618f0 2023-11-08T19:26:08.772 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.771+0000 7f833a151700 1 -- 172.21.15.143:0/2064325665 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8334161670 con 0x7f83341618f0 2023-11-08T19:26:08.773 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.771+0000 7f831bfff700 1 -- 172.21.15.143:0/2064325665 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4925 (secure 0 0 0) 0x7f832c057dd0 con 0x7f83341618f0 2023-11-08T19:26:08.773 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:26:08.774 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4918,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6836/3265911434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3265911434},{"type":"v1","addr":"172.21.15.174:6836","nonce":3265911434}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4921,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/2092876318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6835","nonce":2092876318},{"type":"v1","addr":"172.21.15.174:6837","nonce":2092876318}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5106,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6834/4077367240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":4077367240},{"type":"v1","addr":"172.21.15.143:6834","nonce":4077367240}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T19:25:50.257292+0000","modified":"2023-11-08T19:25:53.278532+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5109},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5109":{"gid":5109,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.143:6835/1574070101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":1574070101},{"type":"v1","addr":"172.21.15.143:6835","nonce":1574070101}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-08T19:26:08.775 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.774+0000 7f833a151700 1 -- 172.21.15.143:0/2064325665 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f832006af00 msgr2=0x7f832006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:08.775 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.774+0000 7f833a151700 1 --2- 172.21.15.143:0/2064325665 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f832006af00 0x7f832006d3c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f8324014040 tx=0x7f8324008070 comp rx=0 tx=0).stop 2023-11-08T19:26:08.776 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.774+0000 7f833a151700 1 -- 172.21.15.143:0/2064325665 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f83341618f0 msgr2=0x7f83340b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:08.776 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.774+0000 7f833a151700 1 --2- 172.21.15.143:0/2064325665 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f83341618f0 0x7f83340b2000 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f832c00cc50 tx=0x7f832c00a340 comp rx=0 tx=0).stop 2023-11-08T19:26:08.776 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.774+0000 7f833a151700 1 -- 172.21.15.143:0/2064325665 shutdown_connections 2023-11-08T19:26:08.776 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.774+0000 7f833a151700 1 --2- 172.21.15.143:0/2064325665 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f832006af00 0x7f832006d3c0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:08.776 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.774+0000 7f833a151700 1 --2- 172.21.15.143:0/2064325665 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f833406b150 0x7f83340b00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:08.777 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.774+0000 7f833a151700 1 --2- 172.21.15.143:0/2064325665 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f83341618f0 0x7f83340b2000 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:08.777 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.774+0000 7f833a151700 1 --2- 172.21.15.143:0/2064325665 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f833415dd00 0x7f83340b0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:08.777 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.774+0000 7f833a151700 1 -- 172.21.15.143:0/2064325665 >> 172.21.15.143:0/2064325665 conn(0x7f8334099ab0 msgr2=0x7f83340d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:08.777 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.774+0000 7f833a151700 1 -- 172.21.15.143:0/2064325665 shutdown_connections 2023-11-08T19:26:08.777 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.774+0000 7f833a151700 1 -- 172.21.15.143:0/2064325665 wait complete. 2023-11-08T19:26:08.778 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 16 2023-11-08T19:26:08.787 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:26:08.908 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.907+0000 7fed6ac49700 1 Processor -- start 2023-11-08T19:26:08.909 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.907+0000 7fed6ac49700 1 -- start start 2023-11-08T19:26:08.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.908+0000 7fed6ac49700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fed641618f0 0x7fed6406ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:08.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.908+0000 7fed6ac49700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fed6406b150 0x7fed640cb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:08.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.908+0000 7fed6ac49700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed64161df0 con 0x7fed6415dd00 2023-11-08T19:26:08.912 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.908+0000 7fed6ac49700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed6406b600 con 0x7fed641618f0 2023-11-08T19:26:08.912 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.908+0000 7fed6ac49700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed640cad50 con 0x7fed6406b150 2023-11-08T19:26:08.912 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.908+0000 7fed63fff700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fed641618f0 0x7fed6406ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:08.912 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.908+0000 7fed63fff700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fed641618f0 0x7fed6406ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51042/0 (socket says 172.21.15.143:51042) 2023-11-08T19:26:08.912 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.908+0000 7fed63fff700 1 -- 172.21.15.143:0/1878190348 learned_addr learned my addr 172.21.15.143:0/1878190348 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:08.913 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.908+0000 7fed691e6700 1 --2- 172.21.15.143:0/1878190348 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fed6406b150 0x7fed640cb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:08.913 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.908+0000 7fed63fff700 1 -- 172.21.15.143:0/1878190348 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fed6406b150 msgr2=0x7fed640cb1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:08.913 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.908+0000 7fed63fff700 1 --2- 172.21.15.143:0/1878190348 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fed6406b150 0x7fed640cb1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:08.913 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.908+0000 7fed63fff700 1 -- 172.21.15.143:0/1878190348 >> v1:172.21.15.174:6789/0 conn(0x7fed6415dd00 legacy=0x7fed640cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:08.913 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.908+0000 7fed63fff700 1 -- 172.21.15.143:0/1878190348 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed641620d0 con 0x7fed641618f0 2023-11-08T19:26:08.913 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.908+0000 7fed63fff700 1 --2- 172.21.15.143:0/1878190348 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fed641618f0 0x7fed6406ac10 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fed500098f0 tx=0x7fed50009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57a9693987d237a3 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:08.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.909+0000 7fed637fe700 1 -- 172.21.15.143:0/1878190348 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed50004230 con 0x7fed641618f0 2023-11-08T19:26:08.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.909+0000 7fed637fe700 1 -- 172.21.15.143:0/1878190348 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed50004390 con 0x7fed641618f0 2023-11-08T19:26:08.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.909+0000 7fed637fe700 1 -- 172.21.15.143:0/1878190348 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed500036a0 con 0x7fed641618f0 2023-11-08T19:26:08.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.909+0000 7fed6ac49700 1 -- 172.21.15.143:0/1878190348 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fed641618f0 msgr2=0x7fed6406ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:08.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.909+0000 7fed6ac49700 1 --2- 172.21.15.143:0/1878190348 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fed641618f0 0x7fed6406ac10 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fed500098f0 tx=0x7fed50009c00 comp rx=0 tx=0).stop 2023-11-08T19:26:08.915 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.909+0000 7fed6ac49700 1 -- 172.21.15.143:0/1878190348 shutdown_connections 2023-11-08T19:26:08.915 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.909+0000 7fed6ac49700 1 --2- 172.21.15.143:0/1878190348 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fed6406b150 0x7fed640cb1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:08.915 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.909+0000 7fed6ac49700 1 --2- 172.21.15.143:0/1878190348 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fed641618f0 0x7fed6406ac10 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:08.915 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.909+0000 7fed6ac49700 1 -- 172.21.15.143:0/1878190348 >> 172.21.15.143:0/1878190348 conn(0x7fed64099ab0 msgr2=0x7fed640d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:08.915 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.909+0000 7fed6ac49700 1 -- 172.21.15.143:0/1878190348 shutdown_connections 2023-11-08T19:26:08.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.909+0000 7fed6ac49700 1 -- 172.21.15.143:0/1878190348 wait complete. 2023-11-08T19:26:08.918 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.911+0000 7fed6ac49700 1 Processor -- start 2023-11-08T19:26:08.919 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.911+0000 7fed6ac49700 1 -- start start 2023-11-08T19:26:08.919 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.912+0000 7fed6ac49700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fed6406b150 0x7fed640b5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:08.919 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.912+0000 7fed6ac49700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fed6415dd00 0x7fed64154a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:08.919 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.912+0000 7fed6ac49700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fed641618f0 0x7fed640c77d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:08.920 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.912+0000 7fed6ac49700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed640c6610 con 0x7fed6406b150 2023-11-08T19:26:08.920 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.912+0000 7fed6ac49700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed64154f60 con 0x7fed6415dd00 2023-11-08T19:26:08.920 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.912+0000 7fed6ac49700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed640b6220 con 0x7fed641618f0 2023-11-08T19:26:08.920 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.912+0000 7fed689e5700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fed6406b150 0x7fed640b5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:08.920 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.912+0000 7fed689e5700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fed6406b150 0x7fed640b5bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51048/0 (socket says 172.21.15.143:51048) 2023-11-08T19:26:08.921 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.912+0000 7fed689e5700 1 -- 172.21.15.143:0/3672329642 learned_addr learned my addr 172.21.15.143:0/3672329642 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:08.921 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.912+0000 7fed691e6700 1 --2- 172.21.15.143:0/3672329642 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fed641618f0 0x7fed640c77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:08.921 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.912+0000 7fed63fff700 1 --2- 172.21.15.143:0/3672329642 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fed6415dd00 0x7fed64154a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:08.921 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.912+0000 7fed689e5700 1 -- 172.21.15.143:0/3672329642 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fed641618f0 msgr2=0x7fed640c77d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:08.921 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.912+0000 7fed689e5700 1 --2- 172.21.15.143:0/3672329642 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fed641618f0 0x7fed640c77d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:08.921 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.912+0000 7fed689e5700 1 -- 172.21.15.143:0/3672329642 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fed6415dd00 msgr2=0x7fed64154a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:08.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.912+0000 7fed689e5700 1 --2- 172.21.15.143:0/3672329642 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fed6415dd00 0x7fed64154a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:08.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.912+0000 7fed689e5700 1 -- 172.21.15.143:0/3672329642 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed50009550 con 0x7fed6406b150 2023-11-08T19:26:08.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.912+0000 7fed689e5700 1 --2- 172.21.15.143:0/3672329642 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fed6406b150 0x7fed640b5bd0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fed5800c4b0 tx=0x7fed5800c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:08.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.913+0000 7fed617fa700 1 -- 172.21.15.143:0/3672329642 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed58011790 con 0x7fed6406b150 2023-11-08T19:26:08.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.913+0000 7fed617fa700 1 -- 172.21.15.143:0/3672329642 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed580118f0 con 0x7fed6406b150 2023-11-08T19:26:08.923 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.913+0000 7fed617fa700 1 -- 172.21.15.143:0/3672329642 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed58011ca0 con 0x7fed6406b150 2023-11-08T19:26:08.923 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.913+0000 7fed6ac49700 1 -- 172.21.15.143:0/3672329642 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed640c5ce0 con 0x7fed6406b150 2023-11-08T19:26:08.923 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.913+0000 7fed6ac49700 1 -- 172.21.15.143:0/3672329642 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed640c62a0 con 0x7fed6406b150 2023-11-08T19:26:08.923 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.914+0000 7fed6ac49700 1 -- 172.21.15.143:0/3672329642 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed640aff60 con 0x7fed6406b150 2023-11-08T19:26:08.923 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.914+0000 7fed617fa700 1 -- 172.21.15.143:0/3672329642 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fed58012420 con 0x7fed6406b150 2023-11-08T19:26:08.924 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.914+0000 7fed617fa700 1 --2- 172.21.15.143:0/3672329642 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fed4806af00 0x7fed4806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:08.924 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.914+0000 7fed617fa700 1 -- 172.21.15.143:0/3672329642 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6118+0+0 (secure 0 0 0) 0x7fed580891f0 con 0x7fed6406b150 2023-11-08T19:26:08.924 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.914+0000 7fed63fff700 1 --2- 172.21.15.143:0/3672329642 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fed4806af00 0x7fed4806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:08.924 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.915+0000 7fed63fff700 1 --2- 172.21.15.143:0/3672329642 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fed4806af00 0x7fed4806d3c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fed50009130 tx=0x7fed50008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:08.925 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:08.919+0000 7fed617fa700 1 -- 172.21.15.143:0/3672329642 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fed58054100 con 0x7fed6406b150 2023-11-08T19:26:09.110 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.109+0000 7fed6ac49700 1 -- 172.21.15.143:0/3672329642 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fed64067c00 con 0x7fed6406b150 2023-11-08T19:26:09.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.110+0000 7fed617fa700 1 -- 172.21.15.143:0/3672329642 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+18456 (secure 0 0 0) 0x7fed58057720 con 0x7fed6406b150 2023-11-08T19:26:09.112 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:26:09.113 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":36,"fsid":"968483af-77fc-4068-ad7c-5c2a1a4613d7","created":"2023-11-08T19:24:32.826484+0000","modified":"2023-11-08T19:26:05.598707+0000","last_up_change":"2023-11-08T19:24:43.573880+0000","last_in_change":"2023-11-08T19:24:35.314525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:44.551530+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-08T19:25:47.925761+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-08T19:25:49.224734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"snap-test-pool","create_time":"2023-11-08T19:26:02.874384+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":36,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bae98ec1-d94e-403d-a220-a522a53ea6f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6809","nonce":1487830566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6811","nonce":1487830566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6815","nonce":1487830566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6813","nonce":1487830566}]},"public_addr":"172.21.15.143:6809/1487830566","cluster_addr":"172.21.15.143:6811/1487830566","heartbeat_back_addr":"172.21.15.143:6815/1487830566","heartbeat_front_addr":"172.21.15.143:6813/1487830566","state":["exists","up"]},{"osd":1,"uuid":"5cb6538a-1ccb-4322-8633-09f66af0b7e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6801","nonce":787341397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6803","nonce":787341397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6807","nonce":787341397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6805","nonce":787341397}]},"public_addr":"172.21.15.143:6801/787341397","cluster_addr":"172.21.15.143:6803/787341397","heartbeat_back_addr":"172.21.15.143:6807/787341397","heartbeat_front_addr":"172.21.15.143:6805/787341397","state":["exists","up"]},{"osd":2,"uuid":"cdbf522c-60ef-4d60-9653-8dcb529c101a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6822","nonce":2301325597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6826","nonce":2301325597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6831","nonce":2301325597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6829","nonce":2301325597}]},"public_addr":"172.21.15.143:6822/2301325597","cluster_addr":"172.21.15.143:6826/2301325597","heartbeat_back_addr":"172.21.15.143:6831/2301325597","heartbeat_front_addr":"172.21.15.143:6829/2301325597","state":["exists","up"]},{"osd":3,"uuid":"4e2783cf-ebb9-4dd8-8245-bc1d17ce4325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6817","nonce":2021504428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6819","nonce":2021504428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6827","nonce":2021504428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6823","nonce":2021504428}]},"public_addr":"172.21.15.143:6817/2021504428","cluster_addr":"172.21.15.143:6819/2021504428","heartbeat_back_addr":"172.21.15.143:6827/2021504428","heartbeat_front_addr":"172.21.15.143:6823/2021504428","state":["exists","up"]},{"osd":4,"uuid":"691256fa-9b77-49b9-a346-7ae1adf0ca80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6801","nonce":1993380595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6803","nonce":1993380595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6807","nonce":1993380595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6805","nonce":1993380595}]},"public_addr":"172.21.15.174:6801/1993380595","cluster_addr":"172.21.15.174:6803/1993380595","heartbeat_back_addr":"172.21.15.174:6807/1993380595","heartbeat_front_addr":"172.21.15.174:6805/1993380595","state":["exists","up"]},{"osd":5,"uuid":"756fc352-48c9-46ce-b52e-0541fe6f8525","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6809","nonce":1777923303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6811","nonce":1777923303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6815","nonce":1777923303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6813","nonce":1777923303}]},"public_addr":"172.21.15.174:6809/1777923303","cluster_addr":"172.21.15.174:6811/1777923303","heartbeat_back_addr":"172.21.15.174:6815/1777923303","heartbeat_front_addr":"172.21.15.174:6813/1777923303","state":["exists","up"]},{"osd":6,"uuid":"fab2ac2b-ed97-4cde-9825-b480eabf8b8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6825","nonce":2755130316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6827","nonce":2755130316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6831","nonce":2755130316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6829","nonce":2755130316}]},"public_addr":"172.21.15.174:6825/2755130316","cluster_addr":"172.21.15.174:6827/2755130316","heartbeat_back_addr":"172.21.15.174:6831/2755130316","heartbeat_front_addr":"172.21.15.174:6829/2755130316","state":["exists","up"]},{"osd":7,"uuid":"3de9b797-2ab6-4d41-a863-7e7d638fd703","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6817","nonce":2847141411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6819","nonce":2847141411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6823","nonce":2847141411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6821","nonce":2847141411}]},"public_addr":"172.21.15.174:6817/2847141411","cluster_addr":"172.21.15.174:6819/2847141411","heartbeat_back_addr":"172.21.15.174:6823/2847141411","heartbeat_front_addr":"172.21.15.174:6821/2847141411","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.477076+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.496890+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.539305+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.454673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.538881+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.521294+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.520712+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.540404+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":1,"length":1}]}],"new_removed_snaps":[{"pool":6,"snaps":[{"begin":1,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:26:09.114 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.113+0000 7fed6ac49700 1 -- 172.21.15.143:0/3672329642 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fed4806af00 msgr2=0x7fed4806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:09.115 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.113+0000 7fed6ac49700 1 --2- 172.21.15.143:0/3672329642 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fed4806af00 0x7fed4806d3c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fed50009130 tx=0x7fed50008030 comp rx=0 tx=0).stop 2023-11-08T19:26:09.115 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.113+0000 7fed6ac49700 1 -- 172.21.15.143:0/3672329642 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fed6406b150 msgr2=0x7fed640b5bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:09.115 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.113+0000 7fed6ac49700 1 --2- 172.21.15.143:0/3672329642 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fed6406b150 0x7fed640b5bd0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fed5800c4b0 tx=0x7fed5800c7c0 comp rx=0 tx=0).stop 2023-11-08T19:26:09.115 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.113+0000 7fed6ac49700 1 -- 172.21.15.143:0/3672329642 shutdown_connections 2023-11-08T19:26:09.115 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.113+0000 7fed6ac49700 1 --2- 172.21.15.143:0/3672329642 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fed4806af00 0x7fed4806d3c0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:09.116 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.113+0000 7fed6ac49700 1 --2- 172.21.15.143:0/3672329642 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fed641618f0 0x7fed640c77d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:09.116 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.113+0000 7fed6ac49700 1 --2- 172.21.15.143:0/3672329642 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fed6406b150 0x7fed640b5bd0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:09.116 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.113+0000 7fed6ac49700 1 --2- 172.21.15.143:0/3672329642 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fed6415dd00 0x7fed64154a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:09.116 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.113+0000 7fed6ac49700 1 -- 172.21.15.143:0/3672329642 >> 172.21.15.143:0/3672329642 conn(0x7fed64099ab0 msgr2=0x7fed640ad380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:09.116 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.113+0000 7fed6ac49700 1 -- 172.21.15.143:0/3672329642 shutdown_connections 2023-11-08T19:26:09.117 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.113+0000 7fed6ac49700 1 -- 172.21.15.143:0/3672329642 wait complete. 2023-11-08T19:26:09.127 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T19:24:44.551530+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-08T19:25:47.925761+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2023-11-08T19:25:49.224734+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'snap-test-pool', 'create_time': '2023-11-08T19:26:02.874384+0000', 'flags': 16385, 'flags_names': 'hashpspool,pool_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '36', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'pool', 'snap_seq': 2, 'snap_epoch': 36, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-08T19:26:09.127 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T19:26:09.252 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.250+0000 7f24b53f1700 1 Processor -- start 2023-11-08T19:26:09.252 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.250+0000 7f24b53f1700 1 -- start start 2023-11-08T19:26:09.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.251+0000 7f24b53f1700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f24b015dc20 0x7f24b00c4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:09.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.251+0000 7f24b53f1700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f24b00cc9e0 0x7f24b00cce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:09.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.251+0000 7f24b53f1700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24b006ad90 con 0x7f24b015e000 2023-11-08T19:26:09.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.251+0000 7f24b53f1700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24b00cd3d0 con 0x7f24b015dc20 2023-11-08T19:26:09.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.251+0000 7f24b53f1700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24b006bb50 con 0x7f24b00cc9e0 2023-11-08T19:26:09.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.251+0000 7f24ae7fc700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f24b015e000 0x7f24b00c5850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:47208/0 (socket says 172.21.15.143:47208) 2023-11-08T19:26:09.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.251+0000 7f24ae7fc700 1 -- 172.21.15.143:0/154474994 learned_addr learned my addr 172.21.15.143:0/154474994 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:26:09.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.252+0000 7f24af7fe700 1 --2- 172.21.15.143:0/154474994 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f24b00cc9e0 0x7f24b00cce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:09.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.252+0000 7f24aeffd700 1 --2- 172.21.15.143:0/154474994 >> v2:172.21.15.174:3300/0 conn(0x7f24b015dc20 0x7f24b00c4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:09.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.252+0000 7f24adffb700 1 -- 172.21.15.143:0/154474994 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2778522463 0 0) 0x7f24b006ad90 con 0x7f24b015e000 2023-11-08T19:26:09.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.252+0000 7f24adffb700 1 -- 172.21.15.143:0/154474994 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f24980035b0 con 0x7f24b015e000 2023-11-08T19:26:09.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.252+0000 7f24af7fe700 1 -- 172.21.15.143:0/154474994 >> v2:172.21.15.174:3300/0 conn(0x7f24b015dc20 msgr2=0x7f24b00c4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:09.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.252+0000 7f24af7fe700 1 --2- 172.21.15.143:0/154474994 >> v2:172.21.15.174:3300/0 conn(0x7f24b015dc20 0x7f24b00c4ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:09.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.252+0000 7f24af7fe700 1 -- 172.21.15.143:0/154474994 >> v1:172.21.15.143:6789/0 conn(0x7f24b015e000 legacy=0x7f24b00c5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:09.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.252+0000 7f24adffb700 1 -- 172.21.15.143:0/154474994 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3327718241 0 0) 0x7f24980035b0 con 0x7f24b015e000 2023-11-08T19:26:09.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.252+0000 7f24af7fe700 1 -- 172.21.15.143:0/154474994 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24b006b070 con 0x7f24b00cc9e0 2023-11-08T19:26:09.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.252+0000 7f24af7fe700 1 --2- 172.21.15.143:0/154474994 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f24b00cc9e0 0x7f24b00cce90 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f24a8009d90 tx=0x7f24a800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=143834a49b2fbb71 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:09.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.252+0000 7f24adffb700 1 -- 172.21.15.143:0/154474994 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a800e070 con 0x7f24b00cc9e0 2023-11-08T19:26:09.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.252+0000 7f24adffb700 1 -- 172.21.15.143:0/154474994 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24a8003a20 con 0x7f24b00cc9e0 2023-11-08T19:26:09.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.252+0000 7f24adffb700 1 -- 172.21.15.143:0/154474994 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a0001670 con 0x7f24b00cc9e0 2023-11-08T19:26:09.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.253+0000 7f24b53f1700 1 -- 172.21.15.143:0/154474994 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f24b00cc9e0 msgr2=0x7f24b00cce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:09.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.253+0000 7f24b53f1700 1 --2- 172.21.15.143:0/154474994 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f24b00cc9e0 0x7f24b00cce90 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f24a8009d90 tx=0x7f24a800b330 comp rx=0 tx=0).stop 2023-11-08T19:26:09.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.253+0000 7f24b53f1700 1 -- 172.21.15.143:0/154474994 shutdown_connections 2023-11-08T19:26:09.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.253+0000 7f24b53f1700 1 --2- 172.21.15.143:0/154474994 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f24b00cc9e0 0x7f24b00cce90 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:09.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.253+0000 7f24b53f1700 1 --2- 172.21.15.143:0/154474994 >> v2:172.21.15.174:3300/0 conn(0x7f24b015dc20 0x7f24b00c4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:09.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.253+0000 7f24b53f1700 1 -- 172.21.15.143:0/154474994 >> 172.21.15.143:0/154474994 conn(0x7f24b00b1c70 msgr2=0x7f24b00b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:09.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.253+0000 7f24b53f1700 1 -- 172.21.15.143:0/154474994 shutdown_connections 2023-11-08T19:26:09.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.253+0000 7f24b53f1700 1 -- 172.21.15.143:0/154474994 wait complete. 2023-11-08T19:26:09.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.254+0000 7f24b53f1700 1 Processor -- start 2023-11-08T19:26:09.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.254+0000 7f24b53f1700 1 -- start start 2023-11-08T19:26:09.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.255+0000 7f24b53f1700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f24b00cc9e0 0x7f24b014ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:09.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.255+0000 7f24b53f1700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f24b015dc20 0x7f24b00c78c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:09.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.255+0000 7f24b53f1700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f24b015e000 0x7f24b00cbb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:09.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.255+0000 7f24b53f1700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24b014f900 con 0x7f24b00cc9e0 2023-11-08T19:26:09.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.255+0000 7f24b53f1700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24b014fbe0 con 0x7f24b015dc20 2023-11-08T19:26:09.264 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.255+0000 7f24b53f1700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24b014fec0 con 0x7f24b015e000 2023-11-08T19:26:09.264 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.255+0000 7f24aeffd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f24b00cc9e0 0x7f24b014ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:09.264 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.255+0000 7f24aeffd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f24b00cc9e0 0x7f24b014ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51058/0 (socket says 172.21.15.143:51058) 2023-11-08T19:26:09.264 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.255+0000 7f24aeffd700 1 -- 172.21.15.143:0/2268585842 learned_addr learned my addr 172.21.15.143:0/2268585842 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:09.264 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.255+0000 7f24ae7fc700 1 --2- 172.21.15.143:0/2268585842 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f24b015dc20 0x7f24b00c78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:09.265 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.255+0000 7f24af7fe700 1 --2- 172.21.15.143:0/2268585842 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f24b015e000 0x7f24b00cbb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:09.265 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.255+0000 7f24aeffd700 1 -- 172.21.15.143:0/2268585842 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f24b015e000 msgr2=0x7f24b00cbb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:09.265 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.255+0000 7f24aeffd700 1 --2- 172.21.15.143:0/2268585842 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f24b015e000 0x7f24b00cbb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:09.265 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.255+0000 7f24aeffd700 1 -- 172.21.15.143:0/2268585842 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f24b015dc20 msgr2=0x7f24b00c78c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:09.265 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.255+0000 7f24aeffd700 1 --2- 172.21.15.143:0/2268585842 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f24b015dc20 0x7f24b00c78c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:09.266 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.255+0000 7f24aeffd700 1 -- 172.21.15.143:0/2268585842 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24a8009a70 con 0x7f24b00cc9e0 2023-11-08T19:26:09.266 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.256+0000 7f24aeffd700 1 --2- 172.21.15.143:0/2268585842 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f24b00cc9e0 0x7f24b014ef50 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f24a400e870 tx=0x7f24a400eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:09.266 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.256+0000 7f24977fe700 1 -- 172.21.15.143:0/2268585842 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a4009590 con 0x7f24b00cc9e0 2023-11-08T19:26:09.266 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.256+0000 7f24b53f1700 1 -- 172.21.15.143:0/2268585842 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24b01501a0 con 0x7f24b00cc9e0 2023-11-08T19:26:09.266 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.256+0000 7f24977fe700 1 -- 172.21.15.143:0/2268585842 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24a4013680 con 0x7f24b00cc9e0 2023-11-08T19:26:09.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.256+0000 7f24977fe700 1 -- 172.21.15.143:0/2268585842 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a40139f0 con 0x7f24b00cc9e0 2023-11-08T19:26:09.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.256+0000 7f24b53f1700 1 -- 172.21.15.143:0/2268585842 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24b014f520 con 0x7f24b00cc9e0 2023-11-08T19:26:09.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.257+0000 7f24b53f1700 1 -- 172.21.15.143:0/2268585842 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24b01507f0 con 0x7f24b00cc9e0 2023-11-08T19:26:09.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.257+0000 7f24977fe700 1 -- 172.21.15.143:0/2268585842 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f24a4013b50 con 0x7f24b00cc9e0 2023-11-08T19:26:09.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.257+0000 7f24977fe700 1 --2- 172.21.15.143:0/2268585842 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f249c06af00 0x7f249c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:09.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.258+0000 7f24977fe700 1 -- 172.21.15.143:0/2268585842 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6118+0+0 (secure 0 0 0) 0x7f24a40895d0 con 0x7f24b00cc9e0 2023-11-08T19:26:09.268 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.258+0000 7f24ae7fc700 1 --2- 172.21.15.143:0/2268585842 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f249c06af00 0x7f249c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:09.268 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.258+0000 7f24ae7fc700 1 --2- 172.21.15.143:0/2268585842 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f249c06af00 0x7f249c06d3c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f24b00c5370 tx=0x7f24a0001230 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:09.268 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.262+0000 7f24977fe700 1 -- 172.21.15.143:0/2268585842 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f24a4054460 con 0x7f24b00cc9e0 2023-11-08T19:26:09.452 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.451+0000 7f24b53f1700 1 -- 172.21.15.143:0/2268585842 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f24b004eee0 con 0x7f24b00cc9e0 2023-11-08T19:26:09.453 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.451+0000 7f24977fe700 1 -- 172.21.15.143:0/2268585842 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v36) v1 ==== 93+0+10 (secure 0 0 0) 0x7f24a4057a80 con 0x7f24b00cc9e0 2023-11-08T19:26:09.453 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2023-11-08T19:26:09.455 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.454+0000 7f24b53f1700 1 -- 172.21.15.143:0/2268585842 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f249c06af00 msgr2=0x7f249c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:09.455 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.454+0000 7f24b53f1700 1 --2- 172.21.15.143:0/2268585842 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f249c06af00 0x7f249c06d3c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f24b00c5370 tx=0x7f24a0001230 comp rx=0 tx=0).stop 2023-11-08T19:26:09.456 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.454+0000 7f24b53f1700 1 -- 172.21.15.143:0/2268585842 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f24b00cc9e0 msgr2=0x7f24b014ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:09.456 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.454+0000 7f24b53f1700 1 --2- 172.21.15.143:0/2268585842 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f24b00cc9e0 0x7f24b014ef50 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f24a400e870 tx=0x7f24a400eb80 comp rx=0 tx=0).stop 2023-11-08T19:26:09.456 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.454+0000 7f24b53f1700 1 -- 172.21.15.143:0/2268585842 shutdown_connections 2023-11-08T19:26:09.456 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.454+0000 7f24b53f1700 1 --2- 172.21.15.143:0/2268585842 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f249c06af00 0x7f249c06d3c0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:09.456 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.454+0000 7f24b53f1700 1 --2- 172.21.15.143:0/2268585842 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f24b015e000 0x7f24b00cbb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:09.457 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.454+0000 7f24b53f1700 1 --2- 172.21.15.143:0/2268585842 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f24b00cc9e0 0x7f24b014ef50 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:09.457 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.454+0000 7f24b53f1700 1 --2- 172.21.15.143:0/2268585842 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f24b015dc20 0x7f24b00c78c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:09.457 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.454+0000 7f24b53f1700 1 -- 172.21.15.143:0/2268585842 >> 172.21.15.143:0/2268585842 conn(0x7f24b00b1c70 msgr2=0x7f24b015a070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:09.457 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.454+0000 7f24b53f1700 1 -- 172.21.15.143:0/2268585842 shutdown_connections 2023-11-08T19:26:09.457 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.454+0000 7f24b53f1700 1 -- 172.21.15.143:0/2268585842 wait complete. 2023-11-08T19:26:09.467 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-08T19:26:09.591 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.589+0000 7f404039b700 1 Processor -- start 2023-11-08T19:26:09.591 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.589+0000 7f404039b700 1 -- start start 2023-11-08T19:26:09.595 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.590+0000 7f404039b700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f403815bd10 0x7f40380d0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:09.595 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.590+0000 7f404039b700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f40380cc900 0x7f40380b0c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:09.595 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.590+0000 7f404039b700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40380d0580 con 0x7f40380b0800 2023-11-08T19:26:09.595 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.590+0000 7f404039b700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40380cceb0 con 0x7f403815bd10 2023-11-08T19:26:09.595 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.590+0000 7f404039b700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40380b11c0 con 0x7f40380cc900 2023-11-08T19:26:09.596 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.590+0000 7f403d936700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f40380b0800 0x7f40380cc1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:47222/0 (socket says 172.21.15.143:47222) 2023-11-08T19:26:09.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.590+0000 7f403d936700 1 -- 172.21.15.143:0/3745900339 learned_addr learned my addr 172.21.15.143:0/3745900339 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:26:09.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.591+0000 7f403e137700 1 --2- 172.21.15.143:0/3745900339 >> v2:172.21.15.174:3300/0 conn(0x7f403815bd10 0x7f40380d0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:09.598 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.591+0000 7f403d135700 1 -- 172.21.15.143:0/3745900339 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2216477447 0 0) 0x7f40380d0580 con 0x7f40380b0800 2023-11-08T19:26:09.598 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.591+0000 7f403e938700 1 --2- 172.21.15.143:0/3745900339 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f40380cc900 0x7f40380b0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:09.598 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.591+0000 7f403d135700 1 -- 172.21.15.143:0/3745900339 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f40240035b0 con 0x7f40380b0800 2023-11-08T19:26:09.598 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.591+0000 7f403d135700 1 -- 172.21.15.143:0/3745900339 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 637696634 0 0) 0x7f40240035b0 con 0x7f40380b0800 2023-11-08T19:26:09.598 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.591+0000 7f403d135700 1 -- 172.21.15.143:0/3745900339 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f40380cc900 msgr2=0x7f40380b0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:09.599 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.591+0000 7f403d135700 1 --2- 172.21.15.143:0/3745900339 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f40380cc900 0x7f40380b0c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:09.599 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.591+0000 7f403d135700 1 -- 172.21.15.143:0/3745900339 >> v2:172.21.15.174:3300/0 conn(0x7f403815bd10 msgr2=0x7f40380d0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:09.599 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.591+0000 7f403d135700 1 --2- 172.21.15.143:0/3745900339 >> v2:172.21.15.174:3300/0 conn(0x7f403815bd10 0x7f40380d0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:09.599 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.591+0000 7f403d135700 1 -- 172.21.15.143:0/3745900339 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40380cfc50 con 0x7f40380b0800 2023-11-08T19:26:09.599 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.591+0000 7f403d135700 1 -- 172.21.15.143:0/3745900339 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4147756303 0 0) 0x7f4034001670 con 0x7f40380b0800 2023-11-08T19:26:09.600 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.591+0000 7f403d135700 1 -- 172.21.15.143:0/3745900339 >> v1:172.21.15.143:6789/0 conn(0x7f40380b0800 legacy=0x7f40380cc1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:09.600 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.591+0000 7f403d135700 1 --2- 172.21.15.143:0/3745900339 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4024003a00 0x7f4024003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:09.600 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.591+0000 7f403d135700 1 --2- 172.21.15.143:0/3745900339 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f40380cc900 0x7f4024004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:09.600 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.591+0000 7f403d135700 1 --2- 172.21.15.143:0/3745900339 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4024004d90 0x7f4024005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:09.600 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.591+0000 7f403d135700 1 -- 172.21.15.143:0/3745900339 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40380cceb0 con 0x7f4024004d90 2023-11-08T19:26:09.600 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.591+0000 7f403d135700 1 -- 172.21.15.143:0/3745900339 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40380b11c0 con 0x7f4024003a00 2023-11-08T19:26:09.601 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.591+0000 7f403d135700 1 -- 172.21.15.143:0/3745900339 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f402400ba10 con 0x7f40380cc900 2023-11-08T19:26:09.601 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.591+0000 7f403e938700 1 --2- 172.21.15.143:0/3745900339 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4024004d90 0x7f4024005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:09.601 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.591+0000 7f403d936700 1 --2- 172.21.15.143:0/3745900339 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f40380cc900 0x7f4024004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:09.601 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.591+0000 7f403e137700 1 --2- 172.21.15.143:0/3745900339 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4024003a00 0x7f4024003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:09.601 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.591+0000 7f403e938700 1 -- 172.21.15.143:0/3745900339 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f40380cc900 msgr2=0x7f4024004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:09.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.591+0000 7f403e938700 1 --2- 172.21.15.143:0/3745900339 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f40380cc900 0x7f4024004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:09.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.591+0000 7f403e938700 1 -- 172.21.15.143:0/3745900339 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4024003a00 msgr2=0x7f4024003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:09.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.591+0000 7f403e938700 1 --2- 172.21.15.143:0/3745900339 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4024003a00 0x7f4024003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:09.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.591+0000 7f403e938700 1 -- 172.21.15.143:0/3745900339 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40240035b0 con 0x7f4024004d90 2023-11-08T19:26:09.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.592+0000 7f403d936700 1 --2- 172.21.15.143:0/3745900339 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f40380cc900 0x7f4024004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:26:09.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.592+0000 7f403e938700 1 --2- 172.21.15.143:0/3745900339 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4024004d90 0x7f4024005210 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f4030009fc0 tx=0x7f4024005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fefbf4b6a335dec server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:09.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.592+0000 7f403d135700 1 -- 172.21.15.143:0/3745900339 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f403000ec30 con 0x7f4024004d90 2023-11-08T19:26:09.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.592+0000 7f403d135700 1 -- 172.21.15.143:0/3745900339 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f403000f040 con 0x7f4024004d90 2023-11-08T19:26:09.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.592+0000 7f403d135700 1 -- 172.21.15.143:0/3745900339 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4030014480 con 0x7f4024004d90 2023-11-08T19:26:09.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.592+0000 7f404039b700 1 -- 172.21.15.143:0/3745900339 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4024004d90 msgr2=0x7f4024005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:09.604 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.592+0000 7f404039b700 1 --2- 172.21.15.143:0/3745900339 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4024004d90 0x7f4024005210 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f4030009fc0 tx=0x7f4024005880 comp rx=0 tx=0).stop 2023-11-08T19:26:09.604 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.592+0000 7f403e938700 1 -- 172.21.15.143:0/3745900339 reap_dead start 2023-11-08T19:26:09.604 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.592+0000 7f404039b700 1 -- 172.21.15.143:0/3745900339 shutdown_connections 2023-11-08T19:26:09.605 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.592+0000 7f404039b700 1 -- 172.21.15.143:0/3745900339 >> 172.21.15.143:0/3745900339 conn(0x7f403806f2e0 msgr2=0x7f403806f6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:09.605 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.592+0000 7f404039b700 1 -- 172.21.15.143:0/3745900339 shutdown_connections 2023-11-08T19:26:09.606 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.592+0000 7f404039b700 1 -- 172.21.15.143:0/3745900339 wait complete. 2023-11-08T19:26:09.609 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.594+0000 7f404039b700 1 Processor -- start 2023-11-08T19:26:09.609 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.594+0000 7f404039b700 1 -- start start 2023-11-08T19:26:09.609 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.595+0000 7f404039b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4024004d90 0x7f40380c7e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:09.609 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.595+0000 7f404039b700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f403815d330 0x7f403815eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:09.609 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.595+0000 7f404039b700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f40380ca050 0x7f40380ca4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:09.610 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.595+0000 7f404039b700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40380caa10 con 0x7f40380ca050 2023-11-08T19:26:09.610 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.595+0000 7f404039b700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40380c6a10 con 0x7f4024004d90 2023-11-08T19:26:09.610 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.595+0000 7f404039b700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f403815daa0 con 0x7f403815d330 2023-11-08T19:26:09.610 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.595+0000 7f403e938700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f40380ca050 0x7f40380ca4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:09.611 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.595+0000 7f403e938700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f40380ca050 0x7f40380ca4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51080/0 (socket says 172.21.15.143:51080) 2023-11-08T19:26:09.611 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.595+0000 7f403e938700 1 -- 172.21.15.143:0/630851732 learned_addr learned my addr 172.21.15.143:0/630851732 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:09.611 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.595+0000 7f403e137700 1 --2- 172.21.15.143:0/630851732 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4024004d90 0x7f40380c7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:09.611 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.595+0000 7f403d936700 1 --2- 172.21.15.143:0/630851732 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f403815d330 0x7f403815eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:09.611 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.595+0000 7f403e938700 1 -- 172.21.15.143:0/630851732 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f403815d330 msgr2=0x7f403815eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:09.611 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.595+0000 7f403e938700 1 --2- 172.21.15.143:0/630851732 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f403815d330 0x7f403815eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:09.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.595+0000 7f403e938700 1 -- 172.21.15.143:0/630851732 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4024004d90 msgr2=0x7f40380c7e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:09.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.595+0000 7f403e938700 1 --2- 172.21.15.143:0/630851732 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4024004d90 0x7f40380c7e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:09.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.595+0000 7f403e938700 1 -- 172.21.15.143:0/630851732 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f403815dd80 con 0x7f40380ca050 2023-11-08T19:26:09.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.595+0000 7f403d936700 1 --2- 172.21.15.143:0/630851732 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f403815d330 0x7f403815eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:26:09.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.596+0000 7f403e938700 1 --2- 172.21.15.143:0/630851732 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f40380ca050 0x7f40380ca4d0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f4030009f60 tx=0x7f4030009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:09.613 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.596+0000 7f402effd700 1 -- 172.21.15.143:0/630851732 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4030007000 con 0x7f40380ca050 2023-11-08T19:26:09.613 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.596+0000 7f402effd700 1 -- 172.21.15.143:0/630851732 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4030014cf0 con 0x7f40380ca050 2023-11-08T19:26:09.613 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.596+0000 7f404039b700 1 -- 172.21.15.143:0/630851732 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f403815e060 con 0x7f40380ca050 2023-11-08T19:26:09.613 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.596+0000 7f402effd700 1 -- 172.21.15.143:0/630851732 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4030014ea0 con 0x7f40380ca050 2023-11-08T19:26:09.613 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.596+0000 7f404039b700 1 -- 172.21.15.143:0/630851732 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f403815e620 con 0x7f40380ca050 2023-11-08T19:26:09.614 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.597+0000 7f404039b700 1 -- 172.21.15.143:0/630851732 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f403804eee0 con 0x7f40380ca050 2023-11-08T19:26:09.614 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.597+0000 7f402effd700 1 -- 172.21.15.143:0/630851732 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4030024020 con 0x7f40380ca050 2023-11-08T19:26:09.614 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.598+0000 7f402effd700 1 --2- 172.21.15.143:0/630851732 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f401c06af00 0x7f401c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:09.614 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.598+0000 7f402effd700 1 -- 172.21.15.143:0/630851732 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6118+0+0 (secure 0 0 0) 0x7f403008af60 con 0x7f40380ca050 2023-11-08T19:26:09.614 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.598+0000 7f403e137700 1 --2- 172.21.15.143:0/630851732 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f401c06af00 0x7f401c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:09.615 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.598+0000 7f403e137700 1 --2- 172.21.15.143:0/630851732 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f401c06af00 0x7f401c06d3c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4028009c60 tx=0x7f4028009210 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:09.615 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.602+0000 7f402effd700 1 -- 172.21.15.143:0/630851732 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4030055f80 con 0x7f40380ca050 2023-11-08T19:26:09.795 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.794+0000 7f404039b700 1 -- 172.21.15.143:0/630851732 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f40380b0f80 con 0x7f40380ca050 2023-11-08T19:26:09.795 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.794+0000 7f402effd700 1 -- 172.21.15.143:0/630851732 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v36) v1 ==== 104+0+11 (secure 0 0 0) 0x7f40300180a0 con 0x7f40380ca050 2023-11-08T19:26:09.796 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2023-11-08T19:26:09.798 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.796+0000 7f404039b700 1 -- 172.21.15.143:0/630851732 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f401c06af00 msgr2=0x7f401c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:09.798 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.796+0000 7f404039b700 1 --2- 172.21.15.143:0/630851732 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f401c06af00 0x7f401c06d3c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4028009c60 tx=0x7f4028009210 comp rx=0 tx=0).stop 2023-11-08T19:26:09.799 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.796+0000 7f404039b700 1 -- 172.21.15.143:0/630851732 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f40380ca050 msgr2=0x7f40380ca4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:09.799 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.796+0000 7f404039b700 1 --2- 172.21.15.143:0/630851732 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f40380ca050 0x7f40380ca4d0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f4030009f60 tx=0x7f4030009750 comp rx=0 tx=0).stop 2023-11-08T19:26:09.799 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.796+0000 7f404039b700 1 -- 172.21.15.143:0/630851732 shutdown_connections 2023-11-08T19:26:09.799 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.796+0000 7f404039b700 1 --2- 172.21.15.143:0/630851732 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f401c06af00 0x7f401c06d3c0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:09.800 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.796+0000 7f404039b700 1 --2- 172.21.15.143:0/630851732 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f40380ca050 0x7f40380ca4d0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:09.800 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.796+0000 7f404039b700 1 --2- 172.21.15.143:0/630851732 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4024004d90 0x7f40380c7e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:09.800 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.796+0000 7f404039b700 1 --2- 172.21.15.143:0/630851732 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f403815d330 0x7f403815eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:09.800 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.796+0000 7f404039b700 1 -- 172.21.15.143:0/630851732 >> 172.21.15.143:0/630851732 conn(0x7f403806f2e0 msgr2=0x7f40380b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:09.800 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.796+0000 7f404039b700 1 -- 172.21.15.143:0/630851732 shutdown_connections 2023-11-08T19:26:09.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.796+0000 7f404039b700 1 -- 172.21.15.143:0/630851732 wait complete. 2023-11-08T19:26:09.810 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-08T19:26:09.934 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.932+0000 7fb38315c700 1 Processor -- start 2023-11-08T19:26:09.934 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.932+0000 7fb38315c700 1 -- start start 2023-11-08T19:26:09.936 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.933+0000 7fb38315c700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb37c06bd20 0x7fb37c0d19c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:09.936 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.933+0000 7fb38315c700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fb37c0d1f00 0x7fb37c0cfd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:09.937 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.933+0000 7fb38315c700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb37c06c2c0 con 0x7fb37c15df20 2023-11-08T19:26:09.937 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.933+0000 7fb38315c700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb37c0d2390 con 0x7fb37c0d1f00 2023-11-08T19:26:09.937 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.933+0000 7fb38315c700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb37c06c5a0 con 0x7fb37c06bd20 2023-11-08T19:26:09.937 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.933+0000 7fb37bfff700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb37c06bd20 0x7fb37c0d19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:09.937 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.933+0000 7fb37bfff700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb37c06bd20 0x7fb37c0d19c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.143:38726/0 (socket says 172.21.15.143:38726) 2023-11-08T19:26:09.938 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.933+0000 7fb37bfff700 1 -- 172.21.15.143:0/2248210620 learned_addr learned my addr 172.21.15.143:0/2248210620 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:09.938 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.933+0000 7fb3816f9700 1 --2- 172.21.15.143:0/2248210620 >> v2:172.21.15.174:3300/0 conn(0x7fb37c0d1f00 0x7fb37c0cfd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:09.938 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.934+0000 7fb37bfff700 1 -- 172.21.15.143:0/2248210620 >> v2:172.21.15.174:3300/0 conn(0x7fb37c0d1f00 msgr2=0x7fb37c0cfd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:09.938 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.934+0000 7fb37bfff700 1 --2- 172.21.15.143:0/2248210620 >> v2:172.21.15.174:3300/0 conn(0x7fb37c0d1f00 0x7fb37c0cfd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:09.938 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.934+0000 7fb37bfff700 1 -- 172.21.15.143:0/2248210620 >> v1:172.21.15.174:6789/0 conn(0x7fb37c15df20 legacy=0x7fb37c0b25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:09.940 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.934+0000 7fb37bfff700 1 -- 172.21.15.143:0/2248210620 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb37c06c880 con 0x7fb37c06bd20 2023-11-08T19:26:09.940 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.934+0000 7fb37bfff700 1 --2- 172.21.15.143:0/2248210620 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb37c06bd20 0x7fb37c0d19c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb368009920 tx=0x7fb368009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7fef42beea52d3af server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:09.940 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.934+0000 7fb37b7fe700 1 -- 172.21.15.143:0/2248210620 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb368011830 con 0x7fb37c06bd20 2023-11-08T19:26:09.940 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.934+0000 7fb37b7fe700 1 -- 172.21.15.143:0/2248210620 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb368011990 con 0x7fb37c06bd20 2023-11-08T19:26:09.941 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.934+0000 7fb37b7fe700 1 -- 172.21.15.143:0/2248210620 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb368011d20 con 0x7fb37c06bd20 2023-11-08T19:26:09.941 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.935+0000 7fb38315c700 1 -- 172.21.15.143:0/2248210620 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb37c06bd20 msgr2=0x7fb37c0d19c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:09.941 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.935+0000 7fb38315c700 1 --2- 172.21.15.143:0/2248210620 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb37c06bd20 0x7fb37c0d19c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb368009920 tx=0x7fb368009c30 comp rx=0 tx=0).stop 2023-11-08T19:26:09.941 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.935+0000 7fb38315c700 1 -- 172.21.15.143:0/2248210620 shutdown_connections 2023-11-08T19:26:09.941 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.935+0000 7fb38315c700 1 --2- 172.21.15.143:0/2248210620 >> v2:172.21.15.174:3300/0 conn(0x7fb37c0d1f00 0x7fb37c0cfd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:09.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.935+0000 7fb38315c700 1 --2- 172.21.15.143:0/2248210620 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb37c06bd20 0x7fb37c0d19c0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:09.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.935+0000 7fb38315c700 1 -- 172.21.15.143:0/2248210620 >> 172.21.15.143:0/2248210620 conn(0x7fb37c08b000 msgr2=0x7fb37c09ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:09.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.935+0000 7fb38315c700 1 -- 172.21.15.143:0/2248210620 shutdown_connections 2023-11-08T19:26:09.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.935+0000 7fb38315c700 1 -- 172.21.15.143:0/2248210620 wait complete. 2023-11-08T19:26:09.945 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.936+0000 7fb38315c700 1 Processor -- start 2023-11-08T19:26:09.945 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.936+0000 7fb38315c700 1 -- start start 2023-11-08T19:26:09.946 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.937+0000 7fb38315c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb37c06bd20 0x7fb37c0c9950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:09.946 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.937+0000 7fb38315c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb37c0d1f00 0x7fb37c15c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:09.946 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.937+0000 7fb38315c700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb37c15df20 0x7fb37c0b5840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:09.946 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.937+0000 7fb38315c700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb37c15c860 con 0x7fb37c06bd20 2023-11-08T19:26:09.946 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.937+0000 7fb38315c700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb37c0ca030 con 0x7fb37c0d1f00 2023-11-08T19:26:09.947 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.937+0000 7fb38315c700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb37c15e300 con 0x7fb37c15df20 2023-11-08T19:26:09.947 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.937+0000 7fb380ef8700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb37c06bd20 0x7fb37c0c9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:09.947 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.937+0000 7fb380ef8700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb37c06bd20 0x7fb37c0c9950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51092/0 (socket says 172.21.15.143:51092) 2023-11-08T19:26:09.947 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.937+0000 7fb380ef8700 1 -- 172.21.15.143:0/759701294 learned_addr learned my addr 172.21.15.143:0/759701294 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:09.947 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.937+0000 7fb3816f9700 1 --2- 172.21.15.143:0/759701294 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb37c15df20 0x7fb37c0b5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:09.948 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.937+0000 7fb37bfff700 1 --2- 172.21.15.143:0/759701294 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb37c0d1f00 0x7fb37c15c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:09.948 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.937+0000 7fb380ef8700 1 -- 172.21.15.143:0/759701294 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb37c15df20 msgr2=0x7fb37c0b5840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:09.948 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.937+0000 7fb380ef8700 1 --2- 172.21.15.143:0/759701294 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb37c15df20 0x7fb37c0b5840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:09.948 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.937+0000 7fb380ef8700 1 -- 172.21.15.143:0/759701294 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb37c0d1f00 msgr2=0x7fb37c15c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:09.948 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.937+0000 7fb380ef8700 1 --2- 172.21.15.143:0/759701294 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb37c0d1f00 0x7fb37c15c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:09.949 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.937+0000 7fb380ef8700 1 -- 172.21.15.143:0/759701294 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb368009550 con 0x7fb37c06bd20 2023-11-08T19:26:09.949 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.937+0000 7fb3816f9700 1 --2- 172.21.15.143:0/759701294 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb37c15df20 0x7fb37c0b5840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:26:09.949 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.937+0000 7fb380ef8700 1 --2- 172.21.15.143:0/759701294 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb37c06bd20 0x7fb37c0c9950 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fb37000c430 tx=0x7fb37000c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:09.949 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.937+0000 7fb3797fa700 1 -- 172.21.15.143:0/759701294 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3700052d0 con 0x7fb37c06bd20 2023-11-08T19:26:09.949 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.937+0000 7fb3797fa700 1 -- 172.21.15.143:0/759701294 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb370005450 con 0x7fb37c06bd20 2023-11-08T19:26:09.950 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.937+0000 7fb38315c700 1 -- 172.21.15.143:0/759701294 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb37c15e5e0 con 0x7fb37c06bd20 2023-11-08T19:26:09.950 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.937+0000 7fb3797fa700 1 -- 172.21.15.143:0/759701294 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb370006aa0 con 0x7fb37c06bd20 2023-11-08T19:26:09.950 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.938+0000 7fb38315c700 1 -- 172.21.15.143:0/759701294 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb37c15eba0 con 0x7fb37c06bd20 2023-11-08T19:26:09.950 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.938+0000 7fb38315c700 1 -- 172.21.15.143:0/759701294 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb37c04eee0 con 0x7fb37c06bd20 2023-11-08T19:26:09.950 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.939+0000 7fb3797fa700 1 -- 172.21.15.143:0/759701294 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb370003680 con 0x7fb37c06bd20 2023-11-08T19:26:09.951 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.939+0000 7fb3797fa700 1 --2- 172.21.15.143:0/759701294 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fb36006af00 0x7fb36006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:09.951 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.939+0000 7fb3797fa700 1 -- 172.21.15.143:0/759701294 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6118+0+0 (secure 0 0 0) 0x7fb370088a40 con 0x7fb37c06bd20 2023-11-08T19:26:09.951 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.939+0000 7fb37bfff700 1 --2- 172.21.15.143:0/759701294 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fb36006af00 0x7fb36006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:09.951 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.940+0000 7fb37bfff700 1 --2- 172.21.15.143:0/759701294 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fb36006af00 0x7fb36006d3c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb37c0c8360 tx=0x7fb368008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:09.951 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:09.943+0000 7fb3797fa700 1 -- 172.21.15.143:0/759701294 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb37008ae60 con 0x7fb37c06bd20 2023-11-08T19:26:10.134 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.133+0000 7fb38315c700 1 -- 172.21.15.143:0/759701294 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb37c0d0240 con 0x7fb37c06bd20 2023-11-08T19:26:10.135 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.133+0000 7fb3797fa700 1 -- 172.21.15.143:0/759701294 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v36) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb3700538d0 con 0x7fb37c06bd20 2023-11-08T19:26:10.135 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 64 2023-11-08T19:26:10.137 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.135+0000 7fb38315c700 1 -- 172.21.15.143:0/759701294 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fb36006af00 msgr2=0x7fb36006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:10.137 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.135+0000 7fb38315c700 1 --2- 172.21.15.143:0/759701294 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fb36006af00 0x7fb36006d3c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb37c0c8360 tx=0x7fb368008030 comp rx=0 tx=0).stop 2023-11-08T19:26:10.137 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.135+0000 7fb38315c700 1 -- 172.21.15.143:0/759701294 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb37c06bd20 msgr2=0x7fb37c0c9950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:10.137 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.135+0000 7fb38315c700 1 --2- 172.21.15.143:0/759701294 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb37c06bd20 0x7fb37c0c9950 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fb37000c430 tx=0x7fb37000c7f0 comp rx=0 tx=0).stop 2023-11-08T19:26:10.138 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.135+0000 7fb38315c700 1 -- 172.21.15.143:0/759701294 shutdown_connections 2023-11-08T19:26:10.138 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.135+0000 7fb38315c700 1 --2- 172.21.15.143:0/759701294 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fb36006af00 0x7fb36006d3c0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:10.138 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.135+0000 7fb38315c700 1 --2- 172.21.15.143:0/759701294 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb37c15df20 0x7fb37c0b5840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:10.138 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.135+0000 7fb38315c700 1 --2- 172.21.15.143:0/759701294 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb37c06bd20 0x7fb37c0c9950 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:10.139 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.135+0000 7fb38315c700 1 --2- 172.21.15.143:0/759701294 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb37c0d1f00 0x7fb37c15c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:10.139 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.135+0000 7fb38315c700 1 -- 172.21.15.143:0/759701294 >> 172.21.15.143:0/759701294 conn(0x7fb37c08b000 msgr2=0x7fb37c0b25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:10.139 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.135+0000 7fb38315c700 1 -- 172.21.15.143:0/759701294 shutdown_connections 2023-11-08T19:26:10.139 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.135+0000 7fb38315c700 1 -- 172.21.15.143:0/759701294 wait complete. 2023-11-08T19:26:10.149 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get snap-test-pool pg_num 2023-11-08T19:26:10.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.271+0000 7f8197243700 1 Processor -- start 2023-11-08T19:26:10.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.271+0000 7f8197243700 1 -- start start 2023-11-08T19:26:10.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.272+0000 7f8197243700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f819015dba0 0x7f819006c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:10.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.272+0000 7f8197243700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f819006b240 0x7f81900caff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:10.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.272+0000 7f8197243700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8190161890 con 0x7f81900cb530 2023-11-08T19:26:10.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.272+0000 7f8197243700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f819006b740 con 0x7f819006b240 2023-11-08T19:26:10.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.272+0000 7f8197243700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f819006ba20 con 0x7f819015dba0 2023-11-08T19:26:10.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.272+0000 7f81957e0700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f81900cb530 0x7f81900b22e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:47258/0 (socket says 172.21.15.143:47258) 2023-11-08T19:26:10.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.272+0000 7f81957e0700 1 -- 172.21.15.143:0/841322407 learned_addr learned my addr 172.21.15.143:0/841322407 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:26:10.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.272+0000 7f8194fdf700 1 --2- 172.21.15.143:0/841322407 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f819015dba0 0x7f819006c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:10.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.272+0000 7f818ffff700 1 --2- 172.21.15.143:0/841322407 >> v2:172.21.15.174:3300/0 conn(0x7f819006b240 0x7f81900caff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:10.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.272+0000 7f818f7fe700 1 -- 172.21.15.143:0/841322407 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 652518262 0 0) 0x7f8190161890 con 0x7f81900cb530 2023-11-08T19:26:10.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.272+0000 7f818f7fe700 1 -- 172.21.15.143:0/841322407 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81800035b0 con 0x7f81900cb530 2023-11-08T19:26:10.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.272+0000 7f818ffff700 1 -- 172.21.15.143:0/841322407 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f819015dba0 msgr2=0x7f819006c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:10.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.272+0000 7f818ffff700 1 --2- 172.21.15.143:0/841322407 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f819015dba0 0x7f819006c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:10.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.272+0000 7f818ffff700 1 -- 172.21.15.143:0/841322407 >> v1:172.21.15.143:6789/0 conn(0x7f81900cb530 legacy=0x7f81900b22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:10.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.272+0000 7f818ffff700 1 -- 172.21.15.143:0/841322407 --> v2:172.21.15.174:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f819006bd00 con 0x7f819006b240 2023-11-08T19:26:10.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.272+0000 7f8194fdf700 1 --2- 172.21.15.143:0/841322407 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f819015dba0 0x7f819006c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T19:26:10.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.273+0000 7f818ffff700 1 --2- 172.21.15.143:0/841322407 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f819006b240 0x7f81900caff0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f817c009840 tx=0x7f817c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e01d4412424e9b39 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:10.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.273+0000 7f818f7fe700 1 -- 172.21.15.143:0/841322407 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f817c011830 con 0x7f819006b240 2023-11-08T19:26:10.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.273+0000 7f818f7fe700 1 -- 172.21.15.143:0/841322407 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f817c011990 con 0x7f819006b240 2023-11-08T19:26:10.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.273+0000 7f818f7fe700 1 -- 172.21.15.143:0/841322407 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f817c011d20 con 0x7f819006b240 2023-11-08T19:26:10.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.273+0000 7f8197243700 1 -- 172.21.15.143:0/841322407 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f819006b240 msgr2=0x7f81900caff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:10.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.273+0000 7f8197243700 1 --2- 172.21.15.143:0/841322407 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f819006b240 0x7f81900caff0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f817c009840 tx=0x7f817c009b50 comp rx=0 tx=0).stop 2023-11-08T19:26:10.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.273+0000 7f8197243700 1 -- 172.21.15.143:0/841322407 shutdown_connections 2023-11-08T19:26:10.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.273+0000 7f8197243700 1 --2- 172.21.15.143:0/841322407 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f819015dba0 0x7f819006c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:10.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.273+0000 7f8197243700 1 --2- 172.21.15.143:0/841322407 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f819006b240 0x7f81900caff0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:10.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.273+0000 7f8197243700 1 -- 172.21.15.143:0/841322407 >> 172.21.15.143:0/841322407 conn(0x7f81900cd2e0 msgr2=0x7f81900cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:10.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.273+0000 7f8197243700 1 -- 172.21.15.143:0/841322407 shutdown_connections 2023-11-08T19:26:10.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.273+0000 7f8197243700 1 -- 172.21.15.143:0/841322407 wait complete. 2023-11-08T19:26:10.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.275+0000 7f8197243700 1 Processor -- start 2023-11-08T19:26:10.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.275+0000 7f8197243700 1 -- start start 2023-11-08T19:26:10.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.275+0000 7f8197243700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f819006b240 0x7f819015ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:10.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.276+0000 7f8197243700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f81900cb530 0x7f81900c4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:10.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.276+0000 7f8197243700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f819015dba0 0x7f81900c8340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:10.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.276+0000 7f8197243700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f819015b350 con 0x7f81900cb530 2023-11-08T19:26:10.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.276+0000 7f8197243700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81900c8d90 con 0x7f819006b240 2023-11-08T19:26:10.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.276+0000 7f8197243700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81900c9070 con 0x7f819015dba0 2023-11-08T19:26:10.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.276+0000 7f818ffff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f81900cb530 0x7f81900c4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:10.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.276+0000 7f818ffff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f81900cb530 0x7f81900c4390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51108/0 (socket says 172.21.15.143:51108) 2023-11-08T19:26:10.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.276+0000 7f818ffff700 1 -- 172.21.15.143:0/37751524 learned_addr learned my addr 172.21.15.143:0/37751524 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:10.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.276+0000 7f81957e0700 1 --2- 172.21.15.143:0/37751524 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f819015dba0 0x7f81900c8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:10.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.276+0000 7f818ffff700 1 -- 172.21.15.143:0/37751524 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f819015dba0 msgr2=0x7f81900c8340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:10.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.276+0000 7f8194fdf700 1 --2- 172.21.15.143:0/37751524 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f819006b240 0x7f819015ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:10.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.276+0000 7f818ffff700 1 --2- 172.21.15.143:0/37751524 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f819015dba0 0x7f81900c8340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:10.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.276+0000 7f818ffff700 1 -- 172.21.15.143:0/37751524 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f819006b240 msgr2=0x7f819015ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:10.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.276+0000 7f818ffff700 1 --2- 172.21.15.143:0/37751524 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f819006b240 0x7f819015ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:10.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.276+0000 7f818ffff700 1 -- 172.21.15.143:0/37751524 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81840094a0 con 0x7f81900cb530 2023-11-08T19:26:10.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.276+0000 7f81957e0700 1 --2- 172.21.15.143:0/37751524 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f819015dba0 0x7f81900c8340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:26:10.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.276+0000 7f818ffff700 1 --2- 172.21.15.143:0/37751524 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f81900cb530 0x7f81900c4390 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f817c0090b0 tx=0x7f817c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:10.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.276+0000 7f818d7fa700 1 -- 172.21.15.143:0/37751524 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f817c0036a0 con 0x7f81900cb530 2023-11-08T19:26:10.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.276+0000 7f818d7fa700 1 -- 172.21.15.143:0/37751524 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f817c01f070 con 0x7f81900cb530 2023-11-08T19:26:10.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.276+0000 7f8197243700 1 -- 172.21.15.143:0/37751524 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f817c0094a0 con 0x7f81900cb530 2023-11-08T19:26:10.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.276+0000 7f818d7fa700 1 -- 172.21.15.143:0/37751524 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f817c003a40 con 0x7f81900cb530 2023-11-08T19:26:10.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.276+0000 7f8197243700 1 -- 172.21.15.143:0/37751524 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81900c9630 con 0x7f81900cb530 2023-11-08T19:26:10.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.277+0000 7f8197243700 1 -- 172.21.15.143:0/37751524 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f819004eee0 con 0x7f81900cb530 2023-11-08T19:26:10.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.278+0000 7f818d7fa700 1 -- 172.21.15.143:0/37751524 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f817c003ba0 con 0x7f81900cb530 2023-11-08T19:26:10.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.278+0000 7f818d7fa700 1 --2- 172.21.15.143:0/37751524 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f817406af00 0x7f817406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:10.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.278+0000 7f818d7fa700 1 -- 172.21.15.143:0/37751524 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6118+0+0 (secure 0 0 0) 0x7f817c021070 con 0x7f81900cb530 2023-11-08T19:26:10.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.278+0000 7f8194fdf700 1 --2- 172.21.15.143:0/37751524 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f817406af00 0x7f817406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:10.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.279+0000 7f8194fdf700 1 --2- 172.21.15.143:0/37751524 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f817406af00 0x7f817406d3c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f8184009b80 tx=0x7f8184008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:10.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.282+0000 7f818d7fa700 1 -- 172.21.15.143:0/37751524 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f817c055f60 con 0x7f81900cb530 2023-11-08T19:26:10.471 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.470+0000 7f8197243700 1 -- 172.21.15.143:0/37751524 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"} v 0) v1 -- 0x7f81900b2aa0 con 0x7f81900cb530 2023-11-08T19:26:10.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.470+0000 7f818d7fa700 1 -- 172.21.15.143:0/37751524 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"}]=0 v36) v1 ==== 103+0+11 (secure 0 0 0) 0x7f817c015020 con 0x7f81900cb530 2023-11-08T19:26:10.472 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2023-11-08T19:26:10.474 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.472+0000 7f8197243700 1 -- 172.21.15.143:0/37751524 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f817406af00 msgr2=0x7f817406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:10.474 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.472+0000 7f8197243700 1 --2- 172.21.15.143:0/37751524 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f817406af00 0x7f817406d3c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f8184009b80 tx=0x7f8184008040 comp rx=0 tx=0).stop 2023-11-08T19:26:10.474 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.472+0000 7f8197243700 1 -- 172.21.15.143:0/37751524 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f81900cb530 msgr2=0x7f81900c4390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:10.474 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.472+0000 7f8197243700 1 --2- 172.21.15.143:0/37751524 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f81900cb530 0x7f81900c4390 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f817c0090b0 tx=0x7f817c004750 comp rx=0 tx=0).stop 2023-11-08T19:26:10.475 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.472+0000 7f8197243700 1 -- 172.21.15.143:0/37751524 shutdown_connections 2023-11-08T19:26:10.475 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.472+0000 7f8197243700 1 --2- 172.21.15.143:0/37751524 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f817406af00 0x7f817406d3c0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:10.475 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.472+0000 7f8197243700 1 --2- 172.21.15.143:0/37751524 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f819015dba0 0x7f81900c8340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:10.475 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.472+0000 7f8197243700 1 --2- 172.21.15.143:0/37751524 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f819006b240 0x7f819015ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:10.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.472+0000 7f8197243700 1 --2- 172.21.15.143:0/37751524 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f81900cb530 0x7f81900c4390 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:10.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.472+0000 7f8197243700 1 -- 172.21.15.143:0/37751524 >> 172.21.15.143:0/37751524 conn(0x7f81900cd2e0 msgr2=0x7f8190072030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:10.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.472+0000 7f8197243700 1 -- 172.21.15.143:0/37751524 shutdown_connections 2023-11-08T19:26:10.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.472+0000 7f8197243700 1 -- 172.21.15.143:0/37751524 wait complete. 2023-11-08T19:26:10.486 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:26:10.607 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.606+0000 7f45e33da700 1 Processor -- start 2023-11-08T19:26:10.608 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.606+0000 7f45e33da700 1 -- start start 2023-11-08T19:26:10.610 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.607+0000 7f45e33da700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f45dc15aa30 0x7f45dc0cf510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:10.610 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.607+0000 7f45e33da700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f45dc0b2210 0x7f45dc0b2690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:10.611 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.607+0000 7f45e33da700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45dc0cf080 con 0x7f45dc0cbe20 2023-11-08T19:26:10.611 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.607+0000 7f45e33da700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45dc0b2bd0 con 0x7f45dc0b2210 2023-11-08T19:26:10.611 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.607+0000 7f45e33da700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45dc0ce470 con 0x7f45dc15aa30 2023-11-08T19:26:10.611 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.607+0000 7f45e0975700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f45dc0b2210 0x7f45dc0b2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:10.611 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.607+0000 7f45e0975700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f45dc0b2210 0x7f45dc0b2690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51114/0 (socket says 172.21.15.143:51114) 2023-11-08T19:26:10.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.607+0000 7f45e0975700 1 -- 172.21.15.143:0/3768870980 learned_addr learned my addr 172.21.15.143:0/3768870980 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:10.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.607+0000 7f45e1176700 1 --2- 172.21.15.143:0/3768870980 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f45dc15aa30 0x7f45dc0cf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:10.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.607+0000 7f45e0975700 1 -- 172.21.15.143:0/3768870980 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f45dc15aa30 msgr2=0x7f45dc0cf510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:10.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.607+0000 7f45e0975700 1 --2- 172.21.15.143:0/3768870980 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f45dc15aa30 0x7f45dc0cf510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:10.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.607+0000 7f45e0975700 1 -- 172.21.15.143:0/3768870980 >> v1:172.21.15.174:6789/0 conn(0x7f45dc0cbe20 legacy=0x7f45dc0cc270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:10.613 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.607+0000 7f45e0975700 1 -- 172.21.15.143:0/3768870980 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45dc0ce750 con 0x7f45dc0b2210 2023-11-08T19:26:10.613 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.607+0000 7f45e1176700 1 --2- 172.21.15.143:0/3768870980 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f45dc15aa30 0x7f45dc0cf510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:26:10.613 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.607+0000 7f45e0975700 1 --2- 172.21.15.143:0/3768870980 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f45dc0b2210 0x7f45dc0b2690 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f45d80098f0 tx=0x7f45d8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67423334b958e03 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:10.613 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.607+0000 7f45d3fff700 1 -- 172.21.15.143:0/3768870980 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45d800b8a0 con 0x7f45dc0b2210 2023-11-08T19:26:10.613 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.607+0000 7f45d3fff700 1 -- 172.21.15.143:0/3768870980 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45d800ba00 con 0x7f45dc0b2210 2023-11-08T19:26:10.614 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.607+0000 7f45d3fff700 1 -- 172.21.15.143:0/3768870980 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45d800bd90 con 0x7f45dc0b2210 2023-11-08T19:26:10.614 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.608+0000 7f45e33da700 1 -- 172.21.15.143:0/3768870980 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f45dc0b2210 msgr2=0x7f45dc0b2690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:10.614 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.608+0000 7f45e33da700 1 --2- 172.21.15.143:0/3768870980 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f45dc0b2210 0x7f45dc0b2690 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f45d80098f0 tx=0x7f45d8009c00 comp rx=0 tx=0).stop 2023-11-08T19:26:10.614 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.608+0000 7f45e33da700 1 -- 172.21.15.143:0/3768870980 shutdown_connections 2023-11-08T19:26:10.614 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.608+0000 7f45e33da700 1 --2- 172.21.15.143:0/3768870980 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f45dc15aa30 0x7f45dc0cf510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:10.615 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.608+0000 7f45e33da700 1 --2- 172.21.15.143:0/3768870980 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f45dc0b2210 0x7f45dc0b2690 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:10.615 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.608+0000 7f45e33da700 1 -- 172.21.15.143:0/3768870980 >> 172.21.15.143:0/3768870980 conn(0x7f45dc099ab0 msgr2=0x7f45dc0d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:10.615 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.608+0000 7f45e33da700 1 -- 172.21.15.143:0/3768870980 shutdown_connections 2023-11-08T19:26:10.615 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.608+0000 7f45e33da700 1 -- 172.21.15.143:0/3768870980 wait complete. 2023-11-08T19:26:10.615 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.609+0000 7f45e33da700 1 Processor -- start 2023-11-08T19:26:10.615 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.609+0000 7f45e33da700 1 -- start start 2023-11-08T19:26:10.618 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.610+0000 7f45e33da700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f45dc0b2210 0x7f45dc158980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:10.618 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.610+0000 7f45e33da700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f45dc0cbe20 0x7f45dc15bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:10.618 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.610+0000 7f45e33da700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f45dc15aa30 0x7f45dc15d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:10.619 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.610+0000 7f45e33da700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45dc15b1a0 con 0x7f45dc0b2210 2023-11-08T19:26:10.619 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.610+0000 7f45e33da700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45dc15b440 con 0x7f45dc0cbe20 2023-11-08T19:26:10.619 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.610+0000 7f45e33da700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45dc15b720 con 0x7f45dc15aa30 2023-11-08T19:26:10.619 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.610+0000 7f45e1176700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f45dc0b2210 0x7f45dc158980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:10.619 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.610+0000 7f45e1176700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f45dc0b2210 0x7f45dc158980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51120/0 (socket says 172.21.15.143:51120) 2023-11-08T19:26:10.620 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.610+0000 7f45e1176700 1 -- 172.21.15.143:0/3550260944 learned_addr learned my addr 172.21.15.143:0/3550260944 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:10.620 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.610+0000 7f45e0975700 1 --2- 172.21.15.143:0/3550260944 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f45dc0cbe20 0x7f45dc15bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:10.620 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.610+0000 7f45e1977700 1 --2- 172.21.15.143:0/3550260944 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f45dc15aa30 0x7f45dc15d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:10.620 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.610+0000 7f45e1176700 1 -- 172.21.15.143:0/3550260944 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f45dc15aa30 msgr2=0x7f45dc15d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:10.621 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.610+0000 7f45e1176700 1 --2- 172.21.15.143:0/3550260944 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f45dc15aa30 0x7f45dc15d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:10.621 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.610+0000 7f45e1176700 1 -- 172.21.15.143:0/3550260944 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f45dc0cbe20 msgr2=0x7f45dc15bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:10.621 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.610+0000 7f45e1176700 1 --2- 172.21.15.143:0/3550260944 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f45dc0cbe20 0x7f45dc15bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:10.621 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.610+0000 7f45e1176700 1 -- 172.21.15.143:0/3550260944 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45d8009550 con 0x7f45dc0b2210 2023-11-08T19:26:10.621 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.610+0000 7f45e1176700 1 --2- 172.21.15.143:0/3550260944 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f45dc0b2210 0x7f45dc158980 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f45cc00e810 tx=0x7f45cc00eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:10.622 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.611+0000 7f45d1ffb700 1 -- 172.21.15.143:0/3550260944 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45cc005c20 con 0x7f45dc0b2210 2023-11-08T19:26:10.622 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.611+0000 7f45d1ffb700 1 -- 172.21.15.143:0/3550260944 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45cc005d80 con 0x7f45dc0b2210 2023-11-08T19:26:10.622 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.611+0000 7f45e33da700 1 -- 172.21.15.143:0/3550260944 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45dc15ba00 con 0x7f45dc0b2210 2023-11-08T19:26:10.622 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.611+0000 7f45d1ffb700 1 -- 172.21.15.143:0/3550260944 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45cc004260 con 0x7f45dc0b2210 2023-11-08T19:26:10.622 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.611+0000 7f45e33da700 1 -- 172.21.15.143:0/3550260944 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45dc15dfd0 con 0x7f45dc0b2210 2023-11-08T19:26:10.623 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.612+0000 7f45e33da700 1 -- 172.21.15.143:0/3550260944 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45dc15bd70 con 0x7f45dc0b2210 2023-11-08T19:26:10.623 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.612+0000 7f45d1ffb700 1 -- 172.21.15.143:0/3550260944 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f45cc0043c0 con 0x7f45dc0b2210 2023-11-08T19:26:10.623 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.612+0000 7f45d1ffb700 1 --2- 172.21.15.143:0/3550260944 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f45c006af00 0x7f45c006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:10.623 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.613+0000 7f45d1ffb700 1 -- 172.21.15.143:0/3550260944 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6118+0+0 (secure 0 0 0) 0x7f45cc0893c0 con 0x7f45dc0b2210 2023-11-08T19:26:10.623 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.613+0000 7f45e0975700 1 --2- 172.21.15.143:0/3550260944 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f45c006af00 0x7f45c006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:10.623 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.613+0000 7f45e0975700 1 --2- 172.21.15.143:0/3550260944 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f45c006af00 0x7f45c006d3c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f45d8009c40 tx=0x7f45d8008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:10.624 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.617+0000 7f45d1ffb700 1 -- 172.21.15.143:0/3550260944 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f45cc0542d0 con 0x7f45dc0b2210 2023-11-08T19:26:10.850 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.848+0000 7f45e33da700 1 -- 172.21.15.143:0/3550260944 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f45dc04eee0 con 0x7f45dc0b2210 2023-11-08T19:26:10.850 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.849+0000 7f45d1ffb700 1 -- 172.21.15.143:0/3550260944 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4925 (secure 0 0 0) 0x7f45cc0578f0 con 0x7f45dc0b2210 2023-11-08T19:26:10.850 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:26:10.851 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4918,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6836/3265911434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3265911434},{"type":"v1","addr":"172.21.15.174:6836","nonce":3265911434}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4921,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/2092876318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6835","nonce":2092876318},{"type":"v1","addr":"172.21.15.174:6837","nonce":2092876318}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5106,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6834/4077367240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":4077367240},{"type":"v1","addr":"172.21.15.143:6834","nonce":4077367240}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T19:25:50.257292+0000","modified":"2023-11-08T19:25:53.278532+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5109},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5109":{"gid":5109,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.143:6835/1574070101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":1574070101},{"type":"v1","addr":"172.21.15.143:6835","nonce":1574070101}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-08T19:26:10.852 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.851+0000 7f45e33da700 1 -- 172.21.15.143:0/3550260944 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f45c006af00 msgr2=0x7f45c006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:10.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.851+0000 7f45e33da700 1 --2- 172.21.15.143:0/3550260944 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f45c006af00 0x7f45c006d3c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f45d8009c40 tx=0x7f45d8008030 comp rx=0 tx=0).stop 2023-11-08T19:26:10.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.851+0000 7f45e33da700 1 -- 172.21.15.143:0/3550260944 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f45dc0b2210 msgr2=0x7f45dc158980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:10.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.851+0000 7f45e33da700 1 --2- 172.21.15.143:0/3550260944 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f45dc0b2210 0x7f45dc158980 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f45cc00e810 tx=0x7f45cc00eb20 comp rx=0 tx=0).stop 2023-11-08T19:26:10.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.851+0000 7f45e33da700 1 -- 172.21.15.143:0/3550260944 shutdown_connections 2023-11-08T19:26:10.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.851+0000 7f45e33da700 1 --2- 172.21.15.143:0/3550260944 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f45c006af00 0x7f45c006d3c0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:10.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.851+0000 7f45e33da700 1 --2- 172.21.15.143:0/3550260944 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f45dc15aa30 0x7f45dc15d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:10.854 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.851+0000 7f45e33da700 1 --2- 172.21.15.143:0/3550260944 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f45dc0b2210 0x7f45dc158980 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:10.854 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.851+0000 7f45e33da700 1 --2- 172.21.15.143:0/3550260944 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f45dc0cbe20 0x7f45dc15bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:10.854 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.851+0000 7f45e33da700 1 -- 172.21.15.143:0/3550260944 >> 172.21.15.143:0/3550260944 conn(0x7f45dc099ab0 msgr2=0x7f45dc06eb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:10.854 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.851+0000 7f45e33da700 1 -- 172.21.15.143:0/3550260944 shutdown_connections 2023-11-08T19:26:10.855 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.851+0000 7f45e33da700 1 -- 172.21.15.143:0/3550260944 wait complete. 2023-11-08T19:26:10.855 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 16 2023-11-08T19:26:10.864 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:26:10.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.983+0000 7f8f6a2bf700 1 Processor -- start 2023-11-08T19:26:10.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.983+0000 7f8f6a2bf700 1 -- start start 2023-11-08T19:26:10.988 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.984+0000 7f8f6a2bf700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f8f6406b320 0x7f8f64168c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:10.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.984+0000 7f8f6a2bf700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f64161df0 con 0x7f8f641618f0 2023-11-08T19:26:10.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.984+0000 7f8f6a2bf700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f641620d0 con 0x7f8f6415dd00 2023-11-08T19:26:10.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.984+0000 7f8f6a2bf700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f640cad50 con 0x7f8f6406b320 2023-11-08T19:26:10.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.984+0000 7f8f637fe700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f8f641618f0 0x7f8f6406ac10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:47280/0 (socket says 172.21.15.143:47280) 2023-11-08T19:26:10.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.984+0000 7f8f637fe700 1 -- 172.21.15.143:0/1996930983 learned_addr learned my addr 172.21.15.143:0/1996930983 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:26:10.990 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.984+0000 7f8f62ffd700 1 -- 172.21.15.143:0/1996930983 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2084266426 0 0) 0x7f8f64161df0 con 0x7f8f641618f0 2023-11-08T19:26:10.990 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.984+0000 7f8f62ffd700 1 -- 172.21.15.143:0/1996930983 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f4c0035b0 con 0x7f8f641618f0 2023-11-08T19:26:10.990 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.984+0000 7f8f6885c700 1 --2- 172.21.15.143:0/1996930983 >> v2:172.21.15.174:3300/0 conn(0x7f8f6406b320 0x7f8f64168c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:10.990 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.985+0000 7f8f62ffd700 1 -- 172.21.15.143:0/1996930983 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 607351017 0 0) 0x7f8f4c0035b0 con 0x7f8f641618f0 2023-11-08T19:26:10.990 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.985+0000 7f8f62ffd700 1 -- 172.21.15.143:0/1996930983 >> v2:172.21.15.174:3300/0 conn(0x7f8f6406b320 msgr2=0x7f8f64168c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:10.991 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.985+0000 7f8f62ffd700 1 --2- 172.21.15.143:0/1996930983 >> v2:172.21.15.174:3300/0 conn(0x7f8f6406b320 0x7f8f64168c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:10.991 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.985+0000 7f8f62ffd700 1 -- 172.21.15.143:0/1996930983 >> v1:172.21.15.174:6789/0 conn(0x7f8f6415dd00 legacy=0x7f8f640cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:10.991 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.985+0000 7f8f62ffd700 1 -- 172.21.15.143:0/1996930983 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f641623b0 con 0x7f8f641618f0 2023-11-08T19:26:10.991 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.985+0000 7f8f62ffd700 1 -- 172.21.15.143:0/1996930983 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4147756303 0 0) 0x7f8f54001670 con 0x7f8f641618f0 2023-11-08T19:26:10.992 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.985+0000 7f8f62ffd700 1 -- 172.21.15.143:0/1996930983 >> v1:172.21.15.143:6789/0 conn(0x7f8f641618f0 legacy=0x7f8f6406ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:10.992 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.985+0000 7f8f62ffd700 1 --2- 172.21.15.143:0/1996930983 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f4c003a00 0x7f8f4c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:10.992 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.985+0000 7f8f62ffd700 1 --2- 172.21.15.143:0/1996930983 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8f4c004850 0x7f8f4c004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:10.992 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.985+0000 7f8f62ffd700 1 --2- 172.21.15.143:0/1996930983 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8f4c005210 0x7f8f4c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:10.992 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.985+0000 7f8f62ffd700 1 -- 172.21.15.143:0/1996930983 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4c00c080 con 0x7f8f4c003a00 2023-11-08T19:26:10.992 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.985+0000 7f8f62ffd700 1 -- 172.21.15.143:0/1996930983 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4c00c1f0 con 0x7f8f4c005210 2023-11-08T19:26:10.993 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.985+0000 7f8f62ffd700 1 -- 172.21.15.143:0/1996930983 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4c00c3b0 con 0x7f8f4c004850 2023-11-08T19:26:10.994 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.985+0000 7f8f63fff700 1 --2- 172.21.15.143:0/1996930983 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f4c003a00 0x7f8f4c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:10.994 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.985+0000 7f8f6885c700 1 --2- 172.21.15.143:0/1996930983 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8f4c005210 0x7f8f4c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:10.994 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.985+0000 7f8f637fe700 1 --2- 172.21.15.143:0/1996930983 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8f4c004850 0x7f8f4c004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:10.995 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.985+0000 7f8f63fff700 1 -- 172.21.15.143:0/1996930983 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8f4c004850 msgr2=0x7f8f4c004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:10.995 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.985+0000 7f8f63fff700 1 --2- 172.21.15.143:0/1996930983 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8f4c004850 0x7f8f4c004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:10.995 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.985+0000 7f8f63fff700 1 -- 172.21.15.143:0/1996930983 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8f4c005210 msgr2=0x7f8f4c00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:10.995 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.985+0000 7f8f63fff700 1 --2- 172.21.15.143:0/1996930983 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8f4c005210 0x7f8f4c00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:10.995 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.985+0000 7f8f63fff700 1 -- 172.21.15.143:0/1996930983 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f4c0035b0 con 0x7f8f4c003a00 2023-11-08T19:26:10.995 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.985+0000 7f8f6885c700 1 --2- 172.21.15.143:0/1996930983 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8f4c005210 0x7f8f4c00ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:26:10.996 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.985+0000 7f8f6885c700 1 -- 172.21.15.143:0/1996930983 reap_dead start 2023-11-08T19:26:10.996 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.985+0000 7f8f63fff700 1 --2- 172.21.15.143:0/1996930983 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f4c003a00 0x7f8f4c003e20 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f8f58001820 tx=0x7f8f4c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99a6735d17285908 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:10.996 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.985+0000 7f8f62ffd700 1 -- 172.21.15.143:0/1996930983 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f580012e0 con 0x7f8f4c003a00 2023-11-08T19:26:10.996 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.985+0000 7f8f62ffd700 1 -- 172.21.15.143:0/1996930983 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f580052d0 con 0x7f8f4c003a00 2023-11-08T19:26:10.996 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.985+0000 7f8f62ffd700 1 -- 172.21.15.143:0/1996930983 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f580036d0 con 0x7f8f4c003a00 2023-11-08T19:26:10.997 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.986+0000 7f8f6a2bf700 1 -- 172.21.15.143:0/1996930983 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f4c003a00 msgr2=0x7f8f4c003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:10.997 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.986+0000 7f8f6a2bf700 1 --2- 172.21.15.143:0/1996930983 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f4c003a00 0x7f8f4c003e20 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f8f58001820 tx=0x7f8f4c005880 comp rx=0 tx=0).stop 2023-11-08T19:26:10.997 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.986+0000 7f8f6a2bf700 1 -- 172.21.15.143:0/1996930983 shutdown_connections 2023-11-08T19:26:10.997 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.986+0000 7f8f6a2bf700 1 --2- 172.21.15.143:0/1996930983 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f4c003a00 0x7f8f4c003e20 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:10.997 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.986+0000 7f8f6a2bf700 1 -- 172.21.15.143:0/1996930983 >> 172.21.15.143:0/1996930983 conn(0x7f8f64099ab0 msgr2=0x7f8f640d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:10.998 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.986+0000 7f8f6a2bf700 1 -- 172.21.15.143:0/1996930983 shutdown_connections 2023-11-08T19:26:10.998 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.986+0000 7f8f6a2bf700 1 -- 172.21.15.143:0/1996930983 wait complete. 2023-11-08T19:26:10.998 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.987+0000 7f8f6a2bf700 1 Processor -- start 2023-11-08T19:26:10.998 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.987+0000 7f8f6a2bf700 1 -- start start 2023-11-08T19:26:11.001 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.988+0000 7f8f6a2bf700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f4c003a00 0x7f8f640b50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:11.001 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.988+0000 7f8f6a2bf700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8f640b0070 0x7f8f640b6080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:11.001 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.988+0000 7f8f6a2bf700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8f641f2f80 0x7f8f640b2150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:11.001 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.988+0000 7f8f6a2bf700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f640b59c0 con 0x7f8f4c003a00 2023-11-08T19:26:11.002 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.988+0000 7f8f6a2bf700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f640b56e0 con 0x7f8f641f2f80 2023-11-08T19:26:11.002 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.988+0000 7f8f6a2bf700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f640c42d0 con 0x7f8f640b0070 2023-11-08T19:26:11.002 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.988+0000 7f8f63fff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f4c003a00 0x7f8f640b50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:11.002 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.988+0000 7f8f63fff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f4c003a00 0x7f8f640b50d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51138/0 (socket says 172.21.15.143:51138) 2023-11-08T19:26:11.002 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.988+0000 7f8f63fff700 1 -- 172.21.15.143:0/2791189247 learned_addr learned my addr 172.21.15.143:0/2791189247 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:11.003 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.988+0000 7f8f6885c700 1 --2- 172.21.15.143:0/2791189247 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8f641f2f80 0x7f8f640b2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:11.003 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.988+0000 7f8f637fe700 1 --2- 172.21.15.143:0/2791189247 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8f640b0070 0x7f8f640b6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:11.003 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.988+0000 7f8f63fff700 1 -- 172.21.15.143:0/2791189247 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8f640b0070 msgr2=0x7f8f640b6080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:11.003 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.988+0000 7f8f63fff700 1 --2- 172.21.15.143:0/2791189247 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8f640b0070 0x7f8f640b6080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:11.003 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.988+0000 7f8f63fff700 1 -- 172.21.15.143:0/2791189247 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8f641f2f80 msgr2=0x7f8f640b2150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:11.004 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.988+0000 7f8f63fff700 1 --2- 172.21.15.143:0/2791189247 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8f641f2f80 0x7f8f640b2150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:11.004 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.988+0000 7f8f63fff700 1 -- 172.21.15.143:0/2791189247 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f6415e260 con 0x7f8f4c003a00 2023-11-08T19:26:11.004 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.989+0000 7f8f63fff700 1 --2- 172.21.15.143:0/2791189247 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f4c003a00 0x7f8f640b50d0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c00c1c0 tx=0x7f8f58001170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:11.004 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.989+0000 7f8f60ff9700 1 -- 172.21.15.143:0/2791189247 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f58000f20 con 0x7f8f4c003a00 2023-11-08T19:26:11.005 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.989+0000 7f8f60ff9700 1 -- 172.21.15.143:0/2791189247 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f58003e40 con 0x7f8f4c003a00 2023-11-08T19:26:11.005 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.989+0000 7f8f60ff9700 1 -- 172.21.15.143:0/2791189247 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f5801d420 con 0x7f8f4c003a00 2023-11-08T19:26:11.005 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.989+0000 7f8f6a2bf700 1 -- 172.21.15.143:0/2791189247 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f640c45b0 con 0x7f8f4c003a00 2023-11-08T19:26:11.005 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.989+0000 7f8f6a2bf700 1 -- 172.21.15.143:0/2791189247 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f640b2f40 con 0x7f8f4c003a00 2023-11-08T19:26:11.005 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.990+0000 7f8f6a2bf700 1 -- 172.21.15.143:0/2791189247 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f640c4c00 con 0x7f8f4c003a00 2023-11-08T19:26:11.006 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.990+0000 7f8f60ff9700 1 -- 172.21.15.143:0/2791189247 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8f5801d580 con 0x7f8f4c003a00 2023-11-08T19:26:11.006 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.991+0000 7f8f60ff9700 1 --2- 172.21.15.143:0/2791189247 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f8f5006af00 0x7f8f5006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:11.006 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.991+0000 7f8f60ff9700 1 -- 172.21.15.143:0/2791189247 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6118+0+0 (secure 0 0 0) 0x7f8f5808a060 con 0x7f8f4c003a00 2023-11-08T19:26:11.006 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.991+0000 7f8f637fe700 1 --2- 172.21.15.143:0/2791189247 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f8f5006af00 0x7f8f5006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:11.006 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.991+0000 7f8f637fe700 1 --2- 172.21.15.143:0/2791189247 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f8f5006af00 0x7f8f5006d3c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f8f6415b210 tx=0x7f8f54009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:11.007 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:10.995+0000 7f8f60ff9700 1 -- 172.21.15.143:0/2791189247 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8f58055120 con 0x7f8f4c003a00 2023-11-08T19:26:11.187 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.186+0000 7f8f6a2bf700 1 -- 172.21.15.143:0/2791189247 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8f6404eee0 con 0x7f8f4c003a00 2023-11-08T19:26:11.188 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.187+0000 7f8f60ff9700 1 -- 172.21.15.143:0/2791189247 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+18456 (secure 0 0 0) 0x7f8f58058740 con 0x7f8f4c003a00 2023-11-08T19:26:11.189 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:26:11.189 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":36,"fsid":"968483af-77fc-4068-ad7c-5c2a1a4613d7","created":"2023-11-08T19:24:32.826484+0000","modified":"2023-11-08T19:26:05.598707+0000","last_up_change":"2023-11-08T19:24:43.573880+0000","last_in_change":"2023-11-08T19:24:35.314525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:44.551530+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-08T19:25:47.925761+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-08T19:25:49.224734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"snap-test-pool","create_time":"2023-11-08T19:26:02.874384+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":36,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bae98ec1-d94e-403d-a220-a522a53ea6f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6809","nonce":1487830566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6811","nonce":1487830566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6815","nonce":1487830566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6813","nonce":1487830566}]},"public_addr":"172.21.15.143:6809/1487830566","cluster_addr":"172.21.15.143:6811/1487830566","heartbeat_back_addr":"172.21.15.143:6815/1487830566","heartbeat_front_addr":"172.21.15.143:6813/1487830566","state":["exists","up"]},{"osd":1,"uuid":"5cb6538a-1ccb-4322-8633-09f66af0b7e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6801","nonce":787341397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6803","nonce":787341397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6807","nonce":787341397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6805","nonce":787341397}]},"public_addr":"172.21.15.143:6801/787341397","cluster_addr":"172.21.15.143:6803/787341397","heartbeat_back_addr":"172.21.15.143:6807/787341397","heartbeat_front_addr":"172.21.15.143:6805/787341397","state":["exists","up"]},{"osd":2,"uuid":"cdbf522c-60ef-4d60-9653-8dcb529c101a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6822","nonce":2301325597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6826","nonce":2301325597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6831","nonce":2301325597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6829","nonce":2301325597}]},"public_addr":"172.21.15.143:6822/2301325597","cluster_addr":"172.21.15.143:6826/2301325597","heartbeat_back_addr":"172.21.15.143:6831/2301325597","heartbeat_front_addr":"172.21.15.143:6829/2301325597","state":["exists","up"]},{"osd":3,"uuid":"4e2783cf-ebb9-4dd8-8245-bc1d17ce4325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6817","nonce":2021504428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6819","nonce":2021504428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6827","nonce":2021504428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6823","nonce":2021504428}]},"public_addr":"172.21.15.143:6817/2021504428","cluster_addr":"172.21.15.143:6819/2021504428","heartbeat_back_addr":"172.21.15.143:6827/2021504428","heartbeat_front_addr":"172.21.15.143:6823/2021504428","state":["exists","up"]},{"osd":4,"uuid":"691256fa-9b77-49b9-a346-7ae1adf0ca80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6801","nonce":1993380595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6803","nonce":1993380595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6807","nonce":1993380595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6805","nonce":1993380595}]},"public_addr":"172.21.15.174:6801/1993380595","cluster_addr":"172.21.15.174:6803/1993380595","heartbeat_back_addr":"172.21.15.174:6807/1993380595","heartbeat_front_addr":"172.21.15.174:6805/1993380595","state":["exists","up"]},{"osd":5,"uuid":"756fc352-48c9-46ce-b52e-0541fe6f8525","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6809","nonce":1777923303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6811","nonce":1777923303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6815","nonce":1777923303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6813","nonce":1777923303}]},"public_addr":"172.21.15.174:6809/1777923303","cluster_addr":"172.21.15.174:6811/1777923303","heartbeat_back_addr":"172.21.15.174:6815/1777923303","heartbeat_front_addr":"172.21.15.174:6813/1777923303","state":["exists","up"]},{"osd":6,"uuid":"fab2ac2b-ed97-4cde-9825-b480eabf8b8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6825","nonce":2755130316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6827","nonce":2755130316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6831","nonce":2755130316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6829","nonce":2755130316}]},"public_addr":"172.21.15.174:6825/2755130316","cluster_addr":"172.21.15.174:6827/2755130316","heartbeat_back_addr":"172.21.15.174:6831/2755130316","heartbeat_front_addr":"172.21.15.174:6829/2755130316","state":["exists","up"]},{"osd":7,"uuid":"3de9b797-2ab6-4d41-a863-7e7d638fd703","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6817","nonce":2847141411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6819","nonce":2847141411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6823","nonce":2847141411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6821","nonce":2847141411}]},"public_addr":"172.21.15.174:6817/2847141411","cluster_addr":"172.21.15.174:6819/2847141411","heartbeat_back_addr":"172.21.15.174:6823/2847141411","heartbeat_front_addr":"172.21.15.174:6821/2847141411","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.477076+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.496890+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.539305+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.454673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.538881+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.521294+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.520712+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.540404+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":1,"length":1}]}],"new_removed_snaps":[{"pool":6,"snaps":[{"begin":1,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:26:11.191 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.189+0000 7f8f6a2bf700 1 -- 172.21.15.143:0/2791189247 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f8f5006af00 msgr2=0x7f8f5006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:11.192 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.189+0000 7f8f6a2bf700 1 --2- 172.21.15.143:0/2791189247 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f8f5006af00 0x7f8f5006d3c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f8f6415b210 tx=0x7f8f54009040 comp rx=0 tx=0).stop 2023-11-08T19:26:11.192 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.189+0000 7f8f6a2bf700 1 -- 172.21.15.143:0/2791189247 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f4c003a00 msgr2=0x7f8f640b50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:11.192 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.189+0000 7f8f6a2bf700 1 --2- 172.21.15.143:0/2791189247 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f4c003a00 0x7f8f640b50d0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c00c1c0 tx=0x7f8f58001170 comp rx=0 tx=0).stop 2023-11-08T19:26:11.192 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.189+0000 7f8f6a2bf700 1 -- 172.21.15.143:0/2791189247 shutdown_connections 2023-11-08T19:26:11.192 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.189+0000 7f8f6a2bf700 1 --2- 172.21.15.143:0/2791189247 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f8f5006af00 0x7f8f5006d3c0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:11.193 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.189+0000 7f8f6a2bf700 1 --2- 172.21.15.143:0/2791189247 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8f641f2f80 0x7f8f640b2150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:11.193 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.189+0000 7f8f6a2bf700 1 --2- 172.21.15.143:0/2791189247 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f4c003a00 0x7f8f640b50d0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:11.193 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.189+0000 7f8f6a2bf700 1 --2- 172.21.15.143:0/2791189247 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8f640b0070 0x7f8f640b6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:11.193 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.189+0000 7f8f6a2bf700 1 -- 172.21.15.143:0/2791189247 >> 172.21.15.143:0/2791189247 conn(0x7f8f64099ab0 msgr2=0x7f8f640ad3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:11.193 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.189+0000 7f8f6a2bf700 1 -- 172.21.15.143:0/2791189247 shutdown_connections 2023-11-08T19:26:11.193 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.190+0000 7f8f6a2bf700 1 -- 172.21.15.143:0/2791189247 wait complete. 2023-11-08T19:26:11.204 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-08T19:26:11.204 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:26:11.325 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.324+0000 7fc020f1e700 1 Processor -- start 2023-11-08T19:26:11.326 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.324+0000 7fc020f1e700 1 -- start start 2023-11-08T19:26:11.329 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.325+0000 7fc020f1e700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fc01c15ba30 0x7fc01c0b14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:11.330 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.325+0000 7fc020f1e700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc01c0d03a0 0x7fc01c0d1560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:11.330 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.325+0000 7fc020f1e700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc01c0d1f10 con 0x7fc01c0d1aa0 2023-11-08T19:26:11.330 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.325+0000 7fc020f1e700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc01c0b1bc0 con 0x7fc01c15ba30 2023-11-08T19:26:11.331 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.325+0000 7fc020f1e700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc01c0b1ea0 con 0x7fc01c0d03a0 2023-11-08T19:26:11.331 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.325+0000 7fc019d9b700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc01c0d03a0 0x7fc01c0d1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:11.331 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.325+0000 7fc01a59c700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fc01c15ba30 0x7fc01c0b14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:11.332 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.325+0000 7fc019d9b700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc01c0d03a0 0x7fc01c0d1560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.143:38784/0 (socket says 172.21.15.143:38784) 2023-11-08T19:26:11.332 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.325+0000 7fc019d9b700 1 -- 172.21.15.143:0/3957063042 learned_addr learned my addr 172.21.15.143:0/3957063042 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:11.332 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.325+0000 7fc019d9b700 1 -- 172.21.15.143:0/3957063042 >> v2:172.21.15.174:3300/0 conn(0x7fc01c15ba30 msgr2=0x7fc01c0b14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:11.332 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.325+0000 7fc019d9b700 1 --2- 172.21.15.143:0/3957063042 >> v2:172.21.15.174:3300/0 conn(0x7fc01c15ba30 0x7fc01c0b14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:11.332 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.325+0000 7fc019d9b700 1 -- 172.21.15.143:0/3957063042 >> v1:172.21.15.174:6789/0 conn(0x7fc01c0d1aa0 legacy=0x7fc01c0ce2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:11.333 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.326+0000 7fc019d9b700 1 -- 172.21.15.143:0/3957063042 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc01c0b2180 con 0x7fc01c0d03a0 2023-11-08T19:26:11.333 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.326+0000 7fc019d9b700 1 --2- 172.21.15.143:0/3957063042 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc01c0d03a0 0x7fc01c0d1560 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc0040098f0 tx=0x7fc004009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82c7ed501c3cdad1 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:11.333 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.326+0000 7fc01959a700 1 -- 172.21.15.143:0/3957063042 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc004011830 con 0x7fc01c0d03a0 2023-11-08T19:26:11.334 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.326+0000 7fc01959a700 1 -- 172.21.15.143:0/3957063042 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc004011990 con 0x7fc01c0d03a0 2023-11-08T19:26:11.334 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.326+0000 7fc01959a700 1 -- 172.21.15.143:0/3957063042 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc004011d20 con 0x7fc01c0d03a0 2023-11-08T19:26:11.334 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.326+0000 7fc020f1e700 1 -- 172.21.15.143:0/3957063042 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc01c0d03a0 msgr2=0x7fc01c0d1560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:11.335 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.326+0000 7fc020f1e700 1 --2- 172.21.15.143:0/3957063042 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc01c0d03a0 0x7fc01c0d1560 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc0040098f0 tx=0x7fc004009c00 comp rx=0 tx=0).stop 2023-11-08T19:26:11.336 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.327+0000 7fc020f1e700 1 -- 172.21.15.143:0/3957063042 shutdown_connections 2023-11-08T19:26:11.336 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.327+0000 7fc020f1e700 1 --2- 172.21.15.143:0/3957063042 >> v2:172.21.15.174:3300/0 conn(0x7fc01c15ba30 0x7fc01c0b14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:11.337 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.327+0000 7fc020f1e700 1 --2- 172.21.15.143:0/3957063042 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc01c0d03a0 0x7fc01c0d1560 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:11.337 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.327+0000 7fc020f1e700 1 -- 172.21.15.143:0/3957063042 >> 172.21.15.143:0/3957063042 conn(0x7fc01c08b000 msgr2=0x7fc01c09ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:11.337 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.327+0000 7fc020f1e700 1 -- 172.21.15.143:0/3957063042 shutdown_connections 2023-11-08T19:26:11.337 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.327+0000 7fc020f1e700 1 -- 172.21.15.143:0/3957063042 wait complete. 2023-11-08T19:26:11.337 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.328+0000 7fc020f1e700 1 Processor -- start 2023-11-08T19:26:11.338 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.328+0000 7fc020f1e700 1 -- start start 2023-11-08T19:26:11.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.329+0000 7fc020f1e700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc01c0d03a0 0x7fc01c159980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:11.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.329+0000 7fc020f1e700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc01c0d1aa0 0x7fc01c0b5c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:11.341 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.329+0000 7fc020f1e700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc01c15ba30 0x7fc01c0cab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:11.341 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.329+0000 7fc020f1e700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc01c1f45b0 con 0x7fc01c0d1aa0 2023-11-08T19:26:11.341 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.329+0000 7fc020f1e700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc01c1f42d0 con 0x7fc01c15ba30 2023-11-08T19:26:11.341 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.329+0000 7fc020f1e700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc01c0c5510 con 0x7fc01c0d03a0 2023-11-08T19:26:11.341 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.329+0000 7fc019d9b700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc01c0d1aa0 0x7fc01c0b5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:11.342 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.329+0000 7fc019d9b700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc01c0d1aa0 0x7fc01c0b5c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51152/0 (socket says 172.21.15.143:51152) 2023-11-08T19:26:11.342 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.329+0000 7fc019d9b700 1 -- 172.21.15.143:0/3713438253 learned_addr learned my addr 172.21.15.143:0/3713438253 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:11.342 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.329+0000 7fc01a59c700 1 --2- 172.21.15.143:0/3713438253 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc01c0d03a0 0x7fc01c159980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:11.342 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.329+0000 7fc01ad9d700 1 --2- 172.21.15.143:0/3713438253 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc01c15ba30 0x7fc01c0cab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:11.342 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.329+0000 7fc019d9b700 1 -- 172.21.15.143:0/3713438253 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc01c0d03a0 msgr2=0x7fc01c159980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:11.343 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.329+0000 7fc019d9b700 1 --2- 172.21.15.143:0/3713438253 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc01c0d03a0 0x7fc01c159980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:11.343 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.329+0000 7fc019d9b700 1 -- 172.21.15.143:0/3713438253 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc01c15ba30 msgr2=0x7fc01c0cab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:11.343 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.329+0000 7fc019d9b700 1 --2- 172.21.15.143:0/3713438253 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc01c15ba30 0x7fc01c0cab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:11.343 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.329+0000 7fc019d9b700 1 -- 172.21.15.143:0/3713438253 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc004009550 con 0x7fc01c0d1aa0 2023-11-08T19:26:11.343 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.329+0000 7fc019d9b700 1 --2- 172.21.15.143:0/3713438253 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc01c0d1aa0 0x7fc01c0b5c90 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fc004000c00 tx=0x7fc004004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:11.344 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.329+0000 7fc012ffd700 1 -- 172.21.15.143:0/3713438253 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc004004400 con 0x7fc01c0d1aa0 2023-11-08T19:26:11.344 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.330+0000 7fc012ffd700 1 -- 172.21.15.143:0/3713438253 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc00401f070 con 0x7fc01c0d1aa0 2023-11-08T19:26:11.344 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.330+0000 7fc020f1e700 1 -- 172.21.15.143:0/3713438253 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc01c0c57f0 con 0x7fc01c0d1aa0 2023-11-08T19:26:11.344 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.330+0000 7fc012ffd700 1 -- 172.21.15.143:0/3713438253 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0040038e0 con 0x7fc01c0d1aa0 2023-11-08T19:26:11.344 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.330+0000 7fc020f1e700 1 -- 172.21.15.143:0/3713438253 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc01c0b61d0 con 0x7fc01c0d1aa0 2023-11-08T19:26:11.345 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.330+0000 7fc020f1e700 1 -- 172.21.15.143:0/3713438253 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc01c0c5d10 con 0x7fc01c0d1aa0 2023-11-08T19:26:11.345 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.331+0000 7fc012ffd700 1 -- 172.21.15.143:0/3713438253 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc004021030 con 0x7fc01c0d1aa0 2023-11-08T19:26:11.345 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.331+0000 7fc012ffd700 1 --2- 172.21.15.143:0/3713438253 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fbffc06af00 0x7fbffc06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:11.345 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.331+0000 7fc012ffd700 1 -- 172.21.15.143:0/3713438253 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6118+0+0 (secure 0 0 0) 0x7fc00408aae0 con 0x7fc01c0d1aa0 2023-11-08T19:26:11.345 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.332+0000 7fc01a59c700 1 --2- 172.21.15.143:0/3713438253 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fbffc06af00 0x7fbffc06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:11.346 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.332+0000 7fc01a59c700 1 --2- 172.21.15.143:0/3713438253 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fbffc06af00 0x7fbffc06d3c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc00c009b40 tx=0x7fc00c009210 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:11.346 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.336+0000 7fc012ffd700 1 -- 172.21.15.143:0/3713438253 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc01c0c5d10 con 0x7fc01c0d1aa0 2023-11-08T19:26:11.567 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.565+0000 7fc020f1e700 1 -- 172.21.15.143:0/3713438253 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc01c0c5d10 con 0x7fc01c0d1aa0 2023-11-08T19:26:11.568 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.566+0000 7fc012ffd700 1 -- 172.21.15.143:0/3713438253 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4925 (secure 0 0 0) 0x7fc01c0c5d10 con 0x7fc01c0d1aa0 2023-11-08T19:26:11.568 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:26:11.568 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4918,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6836/3265911434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3265911434},{"type":"v1","addr":"172.21.15.174:6836","nonce":3265911434}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4921,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/2092876318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6835","nonce":2092876318},{"type":"v1","addr":"172.21.15.174:6837","nonce":2092876318}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5106,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6834/4077367240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":4077367240},{"type":"v1","addr":"172.21.15.143:6834","nonce":4077367240}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T19:25:50.257292+0000","modified":"2023-11-08T19:25:53.278532+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5109},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5109":{"gid":5109,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.143:6835/1574070101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":1574070101},{"type":"v1","addr":"172.21.15.143:6835","nonce":1574070101}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-08T19:26:11.570 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.568+0000 7fc020f1e700 1 -- 172.21.15.143:0/3713438253 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fbffc06af00 msgr2=0x7fbffc06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:11.570 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.568+0000 7fc020f1e700 1 --2- 172.21.15.143:0/3713438253 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fbffc06af00 0x7fbffc06d3c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc00c009b40 tx=0x7fc00c009210 comp rx=0 tx=0).stop 2023-11-08T19:26:11.570 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.568+0000 7fc020f1e700 1 -- 172.21.15.143:0/3713438253 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc01c0d1aa0 msgr2=0x7fc01c0b5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:11.571 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.568+0000 7fc020f1e700 1 --2- 172.21.15.143:0/3713438253 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc01c0d1aa0 0x7fc01c0b5c90 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fc004000c00 tx=0x7fc004004750 comp rx=0 tx=0).stop 2023-11-08T19:26:11.571 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.568+0000 7fc020f1e700 1 -- 172.21.15.143:0/3713438253 shutdown_connections 2023-11-08T19:26:11.571 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.568+0000 7fc020f1e700 1 --2- 172.21.15.143:0/3713438253 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fbffc06af00 0x7fbffc06d3c0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:11.571 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.568+0000 7fc020f1e700 1 --2- 172.21.15.143:0/3713438253 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc01c0d03a0 0x7fc01c159980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:11.571 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.568+0000 7fc020f1e700 1 --2- 172.21.15.143:0/3713438253 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc01c15ba30 0x7fc01c0cab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:11.572 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.568+0000 7fc020f1e700 1 --2- 172.21.15.143:0/3713438253 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc01c0d1aa0 0x7fc01c0b5c90 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:11.572 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.568+0000 7fc020f1e700 1 -- 172.21.15.143:0/3713438253 >> 172.21.15.143:0/3713438253 conn(0x7fc01c08b000 msgr2=0x7fc01c15ad80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:11.572 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.568+0000 7fc020f1e700 1 -- 172.21.15.143:0/3713438253 shutdown_connections 2023-11-08T19:26:11.572 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.568+0000 7fc020f1e700 1 -- 172.21.15.143:0/3713438253 wait complete. 2023-11-08T19:26:11.573 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 16 2023-11-08T19:26:11.582 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:26:11.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.700+0000 7f7f74d6c700 1 Processor -- start 2023-11-08T19:26:11.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.700+0000 7f7f74d6c700 1 -- start start 2023-11-08T19:26:11.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.701+0000 7f7f74d6c700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f7f7015ba30 0x7f7f700b14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:11.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.701+0000 7f7f74d6c700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f7f700d03a0 0x7f7f700d1560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:11.705 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.701+0000 7f7f74d6c700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f700b1bc0 con 0x7f7f700d1aa0 2023-11-08T19:26:11.705 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.701+0000 7f7f74d6c700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f700b1ea0 con 0x7f7f7015ba30 2023-11-08T19:26:11.705 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.701+0000 7f7f74d6c700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f700b2180 con 0x7f7f700d03a0 2023-11-08T19:26:11.705 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.701+0000 7f7f6e59c700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f7f7015ba30 0x7f7f700b14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:11.705 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.701+0000 7f7f6ed9d700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f7f700d1aa0 0x7f7f700ce2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:47316/0 (socket says 172.21.15.143:47316) 2023-11-08T19:26:11.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.701+0000 7f7f6ed9d700 1 -- 172.21.15.143:0/1892317499 learned_addr learned my addr 172.21.15.143:0/1892317499 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:26:11.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.702+0000 7f7f6dd9b700 1 --2- 172.21.15.143:0/1892317499 >> v2:172.21.15.174:3300/0 conn(0x7f7f700d03a0 0x7f7f700d1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:11.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.702+0000 7f7f6e59c700 1 -- 172.21.15.143:0/1892317499 >> v2:172.21.15.174:3300/0 conn(0x7f7f700d03a0 msgr2=0x7f7f700d1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:11.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.702+0000 7f7f6e59c700 1 --2- 172.21.15.143:0/1892317499 >> v2:172.21.15.174:3300/0 conn(0x7f7f700d03a0 0x7f7f700d1560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:11.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.702+0000 7f7f6e59c700 1 -- 172.21.15.143:0/1892317499 >> v1:172.21.15.143:6789/0 conn(0x7f7f700d1aa0 legacy=0x7f7f700ce2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:11.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.702+0000 7f7f6e59c700 1 -- 172.21.15.143:0/1892317499 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f700b2460 con 0x7f7f7015ba30 2023-11-08T19:26:11.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.702+0000 7f7f6d59a700 1 -- 172.21.15.143:0/1892317499 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4078300371 0 0) 0x7f7f700b1bc0 con 0x7f7f700d1aa0 2023-11-08T19:26:11.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.702+0000 7f7f6e59c700 1 --2- 172.21.15.143:0/1892317499 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f7015ba30 0x7f7f700b14c0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f7f700cbe60 tx=0x7f7f60009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbc898e66e3cf3f0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:11.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.702+0000 7f7f6d59a700 1 -- 172.21.15.143:0/1892317499 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f60011830 con 0x7f7f7015ba30 2023-11-08T19:26:11.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.702+0000 7f7f6d59a700 1 -- 172.21.15.143:0/1892317499 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f60011990 con 0x7f7f7015ba30 2023-11-08T19:26:11.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.702+0000 7f7f6d59a700 1 -- 172.21.15.143:0/1892317499 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f60011d20 con 0x7f7f7015ba30 2023-11-08T19:26:11.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.703+0000 7f7f74d6c700 1 -- 172.21.15.143:0/1892317499 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f7015ba30 msgr2=0x7f7f700b14c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:11.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.703+0000 7f7f74d6c700 1 --2- 172.21.15.143:0/1892317499 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f7015ba30 0x7f7f700b14c0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f7f700cbe60 tx=0x7f7f60009b20 comp rx=0 tx=0).stop 2023-11-08T19:26:11.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.703+0000 7f7f74d6c700 1 -- 172.21.15.143:0/1892317499 shutdown_connections 2023-11-08T19:26:11.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.703+0000 7f7f74d6c700 1 --2- 172.21.15.143:0/1892317499 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f7015ba30 0x7f7f700b14c0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:11.709 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.703+0000 7f7f74d6c700 1 --2- 172.21.15.143:0/1892317499 >> v2:172.21.15.174:3300/0 conn(0x7f7f700d03a0 0x7f7f700d1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:11.709 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.703+0000 7f7f74d6c700 1 -- 172.21.15.143:0/1892317499 >> 172.21.15.143:0/1892317499 conn(0x7f7f7008b000 msgr2=0x7f7f7009ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:11.709 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.703+0000 7f7f74d6c700 1 -- 172.21.15.143:0/1892317499 shutdown_connections 2023-11-08T19:26:11.709 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.703+0000 7f7f74d6c700 1 -- 172.21.15.143:0/1892317499 wait complete. 2023-11-08T19:26:11.712 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.704+0000 7f7f74d6c700 1 Processor -- start 2023-11-08T19:26:11.712 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.704+0000 7f7f74d6c700 1 -- start start 2023-11-08T19:26:11.712 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.705+0000 7f7f74d6c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f700d03a0 0x7f7f70156d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:11.712 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.705+0000 7f7f74d6c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7f700d1aa0 0x7f7f7015dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:11.713 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.705+0000 7f7f74d6c700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7f7015ba30 0x7f7f7015e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:11.713 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.705+0000 7f7f74d6c700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f700c7430 con 0x7f7f700d03a0 2023-11-08T19:26:11.713 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.705+0000 7f7f74d6c700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f700c8eb0 con 0x7f7f700d1aa0 2023-11-08T19:26:11.713 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.705+0000 7f7f74d6c700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f700b5af0 con 0x7f7f7015ba30 2023-11-08T19:26:11.713 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.705+0000 7f7f6e59c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f700d03a0 0x7f7f70156d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:11.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.705+0000 7f7f6e59c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f700d03a0 0x7f7f70156d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51164/0 (socket says 172.21.15.143:51164) 2023-11-08T19:26:11.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.705+0000 7f7f6e59c700 1 -- 172.21.15.143:0/559158574 learned_addr learned my addr 172.21.15.143:0/559158574 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:11.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.705+0000 7f7f6ed9d700 1 --2- 172.21.15.143:0/559158574 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7f7015ba30 0x7f7f7015e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:11.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.705+0000 7f7f6dd9b700 1 --2- 172.21.15.143:0/559158574 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7f700d1aa0 0x7f7f7015dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:11.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.705+0000 7f7f6e59c700 1 -- 172.21.15.143:0/559158574 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7f7015ba30 msgr2=0x7f7f7015e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:11.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.705+0000 7f7f6e59c700 1 --2- 172.21.15.143:0/559158574 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7f7015ba30 0x7f7f7015e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:11.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.705+0000 7f7f6e59c700 1 -- 172.21.15.143:0/559158574 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7f700d1aa0 msgr2=0x7f7f7015dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:11.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.705+0000 7f7f6e59c700 1 --2- 172.21.15.143:0/559158574 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7f700d1aa0 0x7f7f7015dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:11.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.705+0000 7f7f6e59c700 1 -- 172.21.15.143:0/559158574 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f600094a0 con 0x7f7f700d03a0 2023-11-08T19:26:11.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.705+0000 7f7f6ed9d700 1 --2- 172.21.15.143:0/559158574 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7f7015ba30 0x7f7f7015e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:26:11.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.705+0000 7f7f6dd9b700 1 --2- 172.21.15.143:0/559158574 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7f700d1aa0 0x7f7f7015dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:26:11.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.705+0000 7f7f6e59c700 1 --2- 172.21.15.143:0/559158574 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f700d03a0 0x7f7f70156d10 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f7f600090b0 tx=0x7f7f60004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:11.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.705+0000 7f7f66ffd700 1 -- 172.21.15.143:0/559158574 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f600038a0 con 0x7f7f700d03a0 2023-11-08T19:26:11.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.706+0000 7f7f66ffd700 1 -- 172.21.15.143:0/559158574 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f6001e070 con 0x7f7f700d03a0 2023-11-08T19:26:11.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.706+0000 7f7f74d6c700 1 -- 172.21.15.143:0/559158574 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f700b5dd0 con 0x7f7f700d03a0 2023-11-08T19:26:11.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.706+0000 7f7f66ffd700 1 -- 172.21.15.143:0/559158574 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f60003c20 con 0x7f7f700d03a0 2023-11-08T19:26:11.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.706+0000 7f7f74d6c700 1 -- 172.21.15.143:0/559158574 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f700b6390 con 0x7f7f700d03a0 2023-11-08T19:26:11.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.706+0000 7f7f74d6c700 1 -- 172.21.15.143:0/559158574 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f7004eee0 con 0x7f7f700d03a0 2023-11-08T19:26:11.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.708+0000 7f7f66ffd700 1 -- 172.21.15.143:0/559158574 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7f6001a420 con 0x7f7f700d03a0 2023-11-08T19:26:11.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.708+0000 7f7f66ffd700 1 --2- 172.21.15.143:0/559158574 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f7f5006af00 0x7f7f5006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:11.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.708+0000 7f7f66ffd700 1 -- 172.21.15.143:0/559158574 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6118+0+0 (secure 0 0 0) 0x7f7f6008b0e0 con 0x7f7f700d03a0 2023-11-08T19:26:11.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.708+0000 7f7f6dd9b700 1 --2- 172.21.15.143:0/559158574 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f7f5006af00 0x7f7f5006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:11.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.709+0000 7f7f6dd9b700 1 --2- 172.21.15.143:0/559158574 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f7f5006af00 0x7f7f5006d3c0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f7f700cce10 tx=0x7f7f58008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:11.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.711+0000 7f7f66ffd700 1 -- 172.21.15.143:0/559158574 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7f60056060 con 0x7f7f700d03a0 2023-11-08T19:26:11.944 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.942+0000 7f7f74d6c700 1 -- 172.21.15.143:0/559158574 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7f700741d0 con 0x7f7f700d03a0 2023-11-08T19:26:11.945 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.943+0000 7f7f66ffd700 1 -- 172.21.15.143:0/559158574 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4925 (secure 0 0 0) 0x7f7f60020090 con 0x7f7f700d03a0 2023-11-08T19:26:11.945 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:26:11.945 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4918,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6836/3265911434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3265911434},{"type":"v1","addr":"172.21.15.174:6836","nonce":3265911434}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4921,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/2092876318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6835","nonce":2092876318},{"type":"v1","addr":"172.21.15.174:6837","nonce":2092876318}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5106,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6834/4077367240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":4077367240},{"type":"v1","addr":"172.21.15.143:6834","nonce":4077367240}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T19:25:50.257292+0000","modified":"2023-11-08T19:25:53.278532+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5109},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5109":{"gid":5109,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.143:6835/1574070101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":1574070101},{"type":"v1","addr":"172.21.15.143:6835","nonce":1574070101}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-08T19:26:11.947 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.945+0000 7f7f74d6c700 1 -- 172.21.15.143:0/559158574 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f7f5006af00 msgr2=0x7f7f5006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:11.947 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.945+0000 7f7f74d6c700 1 --2- 172.21.15.143:0/559158574 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f7f5006af00 0x7f7f5006d3c0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f7f700cce10 tx=0x7f7f58008040 comp rx=0 tx=0).stop 2023-11-08T19:26:11.947 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.945+0000 7f7f74d6c700 1 -- 172.21.15.143:0/559158574 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f700d03a0 msgr2=0x7f7f70156d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:11.948 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.945+0000 7f7f74d6c700 1 --2- 172.21.15.143:0/559158574 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f700d03a0 0x7f7f70156d10 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f7f600090b0 tx=0x7f7f60004750 comp rx=0 tx=0).stop 2023-11-08T19:26:11.948 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.945+0000 7f7f74d6c700 1 -- 172.21.15.143:0/559158574 shutdown_connections 2023-11-08T19:26:11.948 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.945+0000 7f7f74d6c700 1 --2- 172.21.15.143:0/559158574 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f7f5006af00 0x7f7f5006d3c0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:11.948 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.945+0000 7f7f74d6c700 1 --2- 172.21.15.143:0/559158574 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7f7015ba30 0x7f7f7015e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:11.948 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.945+0000 7f7f74d6c700 1 --2- 172.21.15.143:0/559158574 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f700d03a0 0x7f7f70156d10 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:11.949 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.945+0000 7f7f74d6c700 1 --2- 172.21.15.143:0/559158574 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7f700d1aa0 0x7f7f7015dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:11.949 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.945+0000 7f7f74d6c700 1 -- 172.21.15.143:0/559158574 >> 172.21.15.143:0/559158574 conn(0x7f7f7008b000 msgr2=0x7f7f700b13f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:11.949 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.945+0000 7f7f74d6c700 1 -- 172.21.15.143:0/559158574 shutdown_connections 2023-11-08T19:26:11.949 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:11.945+0000 7f7f74d6c700 1 -- 172.21.15.143:0/559158574 wait complete. 2023-11-08T19:26:11.950 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 16 2023-11-08T19:26:11.959 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:26:12.079 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.077+0000 7faca66ae700 1 Processor -- start 2023-11-08T19:26:12.079 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.077+0000 7faca66ae700 1 -- start start 2023-11-08T19:26:12.082 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.078+0000 7faca66ae700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7faca006ed60 0x7faca006b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:12.082 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.078+0000 7faca66ae700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7faca00cc410 0x7faca00cc820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:12.082 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.078+0000 7faca66ae700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faca00b2a60 con 0x7faca0159870 2023-11-08T19:26:12.082 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.078+0000 7faca66ae700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faca006b090 con 0x7faca00cc410 2023-11-08T19:26:12.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.078+0000 7faca66ae700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faca00b2d40 con 0x7faca006ed60 2023-11-08T19:26:12.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.078+0000 7faca4c4b700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7faca00cc410 0x7faca00cc820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:12.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.078+0000 7faca4c4b700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7faca00cc410 0x7faca00cc820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51176/0 (socket says 172.21.15.143:51176) 2023-11-08T19:26:12.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.079+0000 7faca4c4b700 1 -- 172.21.15.143:0/897461528 learned_addr learned my addr 172.21.15.143:0/897461528 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:12.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.078+0000 7fac9ffff700 1 --1- 172.21.15.143:0/897461528 >> v1:172.21.15.143:6789/0 conn(0x7faca0159870 0x7faca00b2210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:47326/0 (socket says 172.21.15.143:47326) 2023-11-08T19:26:12.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.079+0000 7fac9f7fe700 1 --2- 172.21.15.143:0/897461528 >> v2:172.21.15.174:3300/0 conn(0x7faca006ed60 0x7faca006b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:12.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.079+0000 7faca4c4b700 1 -- 172.21.15.143:0/897461528 >> v2:172.21.15.174:3300/0 conn(0x7faca006ed60 msgr2=0x7faca006b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:12.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.079+0000 7faca4c4b700 1 --2- 172.21.15.143:0/897461528 >> v2:172.21.15.174:3300/0 conn(0x7faca006ed60 0x7faca006b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:12.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.079+0000 7faca4c4b700 1 -- 172.21.15.143:0/897461528 >> v1:172.21.15.143:6789/0 conn(0x7faca0159870 legacy=0x7faca00b2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:12.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.079+0000 7faca4c4b700 1 -- 172.21.15.143:0/897461528 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faca00b3020 con 0x7faca00cc410 2023-11-08T19:26:12.085 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.079+0000 7fac9f7fe700 1 --2- 172.21.15.143:0/897461528 >> v2:172.21.15.174:3300/0 conn(0x7faca006ed60 0x7faca006b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:26:12.085 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.079+0000 7fac9effd700 1 -- 172.21.15.143:0/897461528 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 642210798 0 0) 0x7faca00b2a60 con 0x7faca0159870 2023-11-08T19:26:12.085 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.079+0000 7faca4c4b700 1 --2- 172.21.15.143:0/897461528 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faca00cc410 0x7faca00cc820 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7faca0073350 tx=0x7fac9800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29d872c4db4caa2b server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:12.085 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.079+0000 7fac9effd700 1 -- 172.21.15.143:0/897461528 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac9800e070 con 0x7faca00cc410 2023-11-08T19:26:12.085 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.079+0000 7fac9effd700 1 -- 172.21.15.143:0/897461528 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac9800b9c0 con 0x7faca00cc410 2023-11-08T19:26:12.086 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.079+0000 7fac9effd700 1 -- 172.21.15.143:0/897461528 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac9800bd70 con 0x7faca00cc410 2023-11-08T19:26:12.086 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.080+0000 7faca66ae700 1 -- 172.21.15.143:0/897461528 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faca00cc410 msgr2=0x7faca00cc820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:12.086 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.080+0000 7faca66ae700 1 --2- 172.21.15.143:0/897461528 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faca00cc410 0x7faca00cc820 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7faca0073350 tx=0x7fac9800b330 comp rx=0 tx=0).stop 2023-11-08T19:26:12.086 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.080+0000 7faca66ae700 1 -- 172.21.15.143:0/897461528 shutdown_connections 2023-11-08T19:26:12.086 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.080+0000 7faca66ae700 1 --2- 172.21.15.143:0/897461528 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faca00cc410 0x7faca00cc820 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:12.087 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.080+0000 7faca66ae700 1 --2- 172.21.15.143:0/897461528 >> v2:172.21.15.174:3300/0 conn(0x7faca006ed60 0x7faca006b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:12.087 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.080+0000 7faca66ae700 1 -- 172.21.15.143:0/897461528 >> 172.21.15.143:0/897461528 conn(0x7faca0099ab0 msgr2=0x7faca00d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:12.087 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.080+0000 7faca66ae700 1 -- 172.21.15.143:0/897461528 shutdown_connections 2023-11-08T19:26:12.087 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.080+0000 7faca66ae700 1 -- 172.21.15.143:0/897461528 wait complete. 2023-11-08T19:26:12.087 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.081+0000 7faca66ae700 1 Processor -- start 2023-11-08T19:26:12.090 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.081+0000 7faca66ae700 1 -- start start 2023-11-08T19:26:12.090 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.082+0000 7faca66ae700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faca006ed60 0x7faca0157e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:12.090 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.082+0000 7faca66ae700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7faca00cc410 0x7faca00ce9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:12.091 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.082+0000 7faca66ae700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faca0159870 0x7faca00ceee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:12.091 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.082+0000 7faca66ae700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faca00c91f0 con 0x7faca006ed60 2023-11-08T19:26:12.091 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.082+0000 7faca66ae700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faca015cd60 con 0x7faca0159870 2023-11-08T19:26:12.091 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.082+0000 7faca66ae700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faca00c88c0 con 0x7faca00cc410 2023-11-08T19:26:12.091 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.082+0000 7fac9ffff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faca006ed60 0x7faca0157e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:12.092 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.082+0000 7fac9ffff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faca006ed60 0x7faca0157e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51178/0 (socket says 172.21.15.143:51178) 2023-11-08T19:26:12.092 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.082+0000 7fac9ffff700 1 -- 172.21.15.143:0/1621931838 learned_addr learned my addr 172.21.15.143:0/1621931838 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:12.092 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.082+0000 7fac9ffff700 1 -- 172.21.15.143:0/1621931838 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7faca00cc410 msgr2=0x7faca00ce9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:12.092 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.082+0000 7faca4c4b700 1 --2- 172.21.15.143:0/1621931838 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faca0159870 0x7faca00ceee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:12.092 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.082+0000 7fac9ffff700 1 --2- 172.21.15.143:0/1621931838 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7faca00cc410 0x7faca00ce9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:12.093 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.082+0000 7fac9ffff700 1 -- 172.21.15.143:0/1621931838 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faca0159870 msgr2=0x7faca00ceee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:12.093 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.082+0000 7fac9ffff700 1 --2- 172.21.15.143:0/1621931838 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faca0159870 0x7faca00ceee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:12.093 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.082+0000 7fac9ffff700 1 -- 172.21.15.143:0/1621931838 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac98009b30 con 0x7faca006ed60 2023-11-08T19:26:12.093 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.082+0000 7fac9ffff700 1 --2- 172.21.15.143:0/1621931838 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faca006ed60 0x7faca0157e30 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fac9000c570 tx=0x7fac9000c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:12.093 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.082+0000 7fac9cff9700 1 -- 172.21.15.143:0/1621931838 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac900074e0 con 0x7faca006ed60 2023-11-08T19:26:12.094 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.082+0000 7fac9cff9700 1 -- 172.21.15.143:0/1621931838 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac90007660 con 0x7faca006ed60 2023-11-08T19:26:12.094 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.082+0000 7faca66ae700 1 -- 172.21.15.143:0/1621931838 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faca00c8ba0 con 0x7faca006ed60 2023-11-08T19:26:12.094 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.082+0000 7fac9cff9700 1 -- 172.21.15.143:0/1621931838 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac90005a70 con 0x7faca006ed60 2023-11-08T19:26:12.094 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.082+0000 7faca66ae700 1 -- 172.21.15.143:0/1621931838 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faca0158370 con 0x7faca006ed60 2023-11-08T19:26:12.094 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.083+0000 7faca66ae700 1 -- 172.21.15.143:0/1621931838 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faca004eee0 con 0x7faca006ed60 2023-11-08T19:26:12.095 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.084+0000 7fac9cff9700 1 -- 172.21.15.143:0/1621931838 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fac90005bd0 con 0x7faca006ed60 2023-11-08T19:26:12.095 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.084+0000 7fac9cff9700 1 --2- 172.21.15.143:0/1621931838 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fac8006af00 0x7fac8006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:12.095 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.084+0000 7fac9cff9700 1 -- 172.21.15.143:0/1621931838 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6118+0+0 (secure 0 0 0) 0x7fac90088df0 con 0x7faca006ed60 2023-11-08T19:26:12.095 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.084+0000 7fac9f7fe700 1 --2- 172.21.15.143:0/1621931838 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fac8006af00 0x7fac8006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:12.095 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.085+0000 7fac9f7fe700 1 --2- 172.21.15.143:0/1621931838 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fac8006af00 0x7fac8006d3c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fac88006fd0 tx=0x7fac88008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:12.096 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.089+0000 7fac9cff9700 1 -- 172.21.15.143:0/1621931838 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fac90053c80 con 0x7faca006ed60 2023-11-08T19:26:12.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.276+0000 7faca66ae700 1 -- 172.21.15.143:0/1621931838 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faca00ca800 con 0x7faca006ed60 2023-11-08T19:26:12.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.277+0000 7fac9cff9700 1 -- 172.21.15.143:0/1621931838 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+18456 (secure 0 0 0) 0x7fac900572a0 con 0x7faca006ed60 2023-11-08T19:26:12.279 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:26:12.279 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":36,"fsid":"968483af-77fc-4068-ad7c-5c2a1a4613d7","created":"2023-11-08T19:24:32.826484+0000","modified":"2023-11-08T19:26:05.598707+0000","last_up_change":"2023-11-08T19:24:43.573880+0000","last_in_change":"2023-11-08T19:24:35.314525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:44.551530+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-08T19:25:47.925761+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-08T19:25:49.224734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"snap-test-pool","create_time":"2023-11-08T19:26:02.874384+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":36,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bae98ec1-d94e-403d-a220-a522a53ea6f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6809","nonce":1487830566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6811","nonce":1487830566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6815","nonce":1487830566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6813","nonce":1487830566}]},"public_addr":"172.21.15.143:6809/1487830566","cluster_addr":"172.21.15.143:6811/1487830566","heartbeat_back_addr":"172.21.15.143:6815/1487830566","heartbeat_front_addr":"172.21.15.143:6813/1487830566","state":["exists","up"]},{"osd":1,"uuid":"5cb6538a-1ccb-4322-8633-09f66af0b7e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6801","nonce":787341397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6803","nonce":787341397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6807","nonce":787341397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6805","nonce":787341397}]},"public_addr":"172.21.15.143:6801/787341397","cluster_addr":"172.21.15.143:6803/787341397","heartbeat_back_addr":"172.21.15.143:6807/787341397","heartbeat_front_addr":"172.21.15.143:6805/787341397","state":["exists","up"]},{"osd":2,"uuid":"cdbf522c-60ef-4d60-9653-8dcb529c101a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6822","nonce":2301325597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6826","nonce":2301325597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6831","nonce":2301325597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6829","nonce":2301325597}]},"public_addr":"172.21.15.143:6822/2301325597","cluster_addr":"172.21.15.143:6826/2301325597","heartbeat_back_addr":"172.21.15.143:6831/2301325597","heartbeat_front_addr":"172.21.15.143:6829/2301325597","state":["exists","up"]},{"osd":3,"uuid":"4e2783cf-ebb9-4dd8-8245-bc1d17ce4325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6817","nonce":2021504428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6819","nonce":2021504428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6827","nonce":2021504428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6823","nonce":2021504428}]},"public_addr":"172.21.15.143:6817/2021504428","cluster_addr":"172.21.15.143:6819/2021504428","heartbeat_back_addr":"172.21.15.143:6827/2021504428","heartbeat_front_addr":"172.21.15.143:6823/2021504428","state":["exists","up"]},{"osd":4,"uuid":"691256fa-9b77-49b9-a346-7ae1adf0ca80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6801","nonce":1993380595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6803","nonce":1993380595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6807","nonce":1993380595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6805","nonce":1993380595}]},"public_addr":"172.21.15.174:6801/1993380595","cluster_addr":"172.21.15.174:6803/1993380595","heartbeat_back_addr":"172.21.15.174:6807/1993380595","heartbeat_front_addr":"172.21.15.174:6805/1993380595","state":["exists","up"]},{"osd":5,"uuid":"756fc352-48c9-46ce-b52e-0541fe6f8525","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6809","nonce":1777923303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6811","nonce":1777923303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6815","nonce":1777923303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6813","nonce":1777923303}]},"public_addr":"172.21.15.174:6809/1777923303","cluster_addr":"172.21.15.174:6811/1777923303","heartbeat_back_addr":"172.21.15.174:6815/1777923303","heartbeat_front_addr":"172.21.15.174:6813/1777923303","state":["exists","up"]},{"osd":6,"uuid":"fab2ac2b-ed97-4cde-9825-b480eabf8b8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6825","nonce":2755130316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6827","nonce":2755130316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6831","nonce":2755130316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6829","nonce":2755130316}]},"public_addr":"172.21.15.174:6825/2755130316","cluster_addr":"172.21.15.174:6827/2755130316","heartbeat_back_addr":"172.21.15.174:6831/2755130316","heartbeat_front_addr":"172.21.15.174:6829/2755130316","state":["exists","up"]},{"osd":7,"uuid":"3de9b797-2ab6-4d41-a863-7e7d638fd703","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6817","nonce":2847141411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6819","nonce":2847141411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6823","nonce":2847141411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6821","nonce":2847141411}]},"public_addr":"172.21.15.174:6817/2847141411","cluster_addr":"172.21.15.174:6819/2847141411","heartbeat_back_addr":"172.21.15.174:6823/2847141411","heartbeat_front_addr":"172.21.15.174:6821/2847141411","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.477076+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.496890+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.539305+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.454673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.538881+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.521294+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.520712+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.540404+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":1,"length":1}]}],"new_removed_snaps":[{"pool":6,"snaps":[{"begin":1,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:26:12.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.280+0000 7faca66ae700 1 -- 172.21.15.143:0/1621931838 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fac8006af00 msgr2=0x7fac8006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:12.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.280+0000 7faca66ae700 1 --2- 172.21.15.143:0/1621931838 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fac8006af00 0x7fac8006d3c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fac88006fd0 tx=0x7fac88008040 comp rx=0 tx=0).stop 2023-11-08T19:26:12.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.280+0000 7faca66ae700 1 -- 172.21.15.143:0/1621931838 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faca006ed60 msgr2=0x7faca0157e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:12.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.280+0000 7faca66ae700 1 --2- 172.21.15.143:0/1621931838 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faca006ed60 0x7faca0157e30 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fac9000c570 tx=0x7fac9000c930 comp rx=0 tx=0).stop 2023-11-08T19:26:12.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.280+0000 7faca66ae700 1 -- 172.21.15.143:0/1621931838 shutdown_connections 2023-11-08T19:26:12.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.280+0000 7faca66ae700 1 --2- 172.21.15.143:0/1621931838 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fac8006af00 0x7fac8006d3c0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:12.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.280+0000 7faca66ae700 1 --2- 172.21.15.143:0/1621931838 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faca0159870 0x7faca00ceee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:12.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.280+0000 7faca66ae700 1 --2- 172.21.15.143:0/1621931838 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faca006ed60 0x7faca0157e30 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:12.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.280+0000 7faca66ae700 1 --2- 172.21.15.143:0/1621931838 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7faca00cc410 0x7faca00ce9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:12.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.280+0000 7faca66ae700 1 -- 172.21.15.143:0/1621931838 >> 172.21.15.143:0/1621931838 conn(0x7faca0099ab0 msgr2=0x7faca00af440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:12.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.280+0000 7faca66ae700 1 -- 172.21.15.143:0/1621931838 shutdown_connections 2023-11-08T19:26:12.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.280+0000 7faca66ae700 1 -- 172.21.15.143:0/1621931838 wait complete. 2023-11-08T19:26:12.293 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-08T19:26:12.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.413+0000 7fc102170700 1 Processor -- start 2023-11-08T19:26:12.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.413+0000 7fc102170700 1 -- start start 2023-11-08T19:26:12.417 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.414+0000 7fc102170700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc0fc06c030 0x7fc0fc0cd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:12.417 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.414+0000 7fc102170700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc0fc15de80 0x7fc0fc0cbf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:12.417 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.414+0000 7fc102170700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0fc0cd480 con 0x7fc0fc06c730 2023-11-08T19:26:12.417 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.414+0000 7fc102170700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0fc0cc870 con 0x7fc0fc06c030 2023-11-08T19:26:12.418 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.414+0000 7fc102170700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0fc0ccb50 con 0x7fc0fc15de80 2023-11-08T19:26:12.418 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.414+0000 7fc0faffd700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc0fc06c030 0x7fc0fc0cd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:12.418 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.414+0000 7fc0faffd700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc0fc06c030 0x7fc0fc0cd910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51188/0 (socket says 172.21.15.143:51188) 2023-11-08T19:26:12.418 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.414+0000 7fc0faffd700 1 -- 172.21.15.143:0/3818283754 learned_addr learned my addr 172.21.15.143:0/3818283754 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:12.418 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.414+0000 7fc0fbfff700 1 --2- 172.21.15.143:0/3818283754 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc0fc15de80 0x7fc0fc0cbf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:12.419 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.414+0000 7fc0faffd700 1 -- 172.21.15.143:0/3818283754 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc0fc15de80 msgr2=0x7fc0fc0cbf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:12.419 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.414+0000 7fc0faffd700 1 --2- 172.21.15.143:0/3818283754 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc0fc15de80 0x7fc0fc0cbf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:12.419 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.414+0000 7fc0faffd700 1 -- 172.21.15.143:0/3818283754 >> v1:172.21.15.174:6789/0 conn(0x7fc0fc06c730 legacy=0x7fc0fc06cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:12.419 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.414+0000 7fc0faffd700 1 -- 172.21.15.143:0/3818283754 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0fc0cce30 con 0x7fc0fc06c030 2023-11-08T19:26:12.419 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.414+0000 7fc0faffd700 1 --2- 172.21.15.143:0/3818283754 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc0fc06c030 0x7fc0fc0cd910 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fc0ec0098f0 tx=0x7fc0ec009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6b163d3755fd468 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:12.419 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.415+0000 7fc0fa7fc700 1 -- 172.21.15.143:0/3818283754 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0ec004230 con 0x7fc0fc06c030 2023-11-08T19:26:12.420 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.415+0000 7fc0fa7fc700 1 -- 172.21.15.143:0/3818283754 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0ec004390 con 0x7fc0fc06c030 2023-11-08T19:26:12.420 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.415+0000 7fc0fa7fc700 1 -- 172.21.15.143:0/3818283754 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0ec0036a0 con 0x7fc0fc06c030 2023-11-08T19:26:12.420 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.415+0000 7fc102170700 1 -- 172.21.15.143:0/3818283754 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc0fc06c030 msgr2=0x7fc0fc0cd910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:12.420 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.415+0000 7fc102170700 1 --2- 172.21.15.143:0/3818283754 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc0fc06c030 0x7fc0fc0cd910 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fc0ec0098f0 tx=0x7fc0ec009c00 comp rx=0 tx=0).stop 2023-11-08T19:26:12.420 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.415+0000 7fc102170700 1 -- 172.21.15.143:0/3818283754 shutdown_connections 2023-11-08T19:26:12.421 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.415+0000 7fc102170700 1 --2- 172.21.15.143:0/3818283754 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc0fc15de80 0x7fc0fc0cbf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:12.421 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.415+0000 7fc102170700 1 --2- 172.21.15.143:0/3818283754 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc0fc06c030 0x7fc0fc0cd910 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:12.421 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.415+0000 7fc102170700 1 -- 172.21.15.143:0/3818283754 >> 172.21.15.143:0/3818283754 conn(0x7fc0fc08b000 msgr2=0x7fc0fc09ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:12.421 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.415+0000 7fc102170700 1 -- 172.21.15.143:0/3818283754 shutdown_connections 2023-11-08T19:26:12.421 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.415+0000 7fc102170700 1 -- 172.21.15.143:0/3818283754 wait complete. 2023-11-08T19:26:12.424 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.417+0000 7fc102170700 1 Processor -- start 2023-11-08T19:26:12.424 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.417+0000 7fc102170700 1 -- start start 2023-11-08T19:26:12.424 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.418+0000 7fc102170700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc0fc06c030 0x7fc0fc15ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:12.425 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.418+0000 7fc102170700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc0fc06c730 0x7fc0fc0d19a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:12.425 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.418+0000 7fc102170700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc0fc15de80 0x7fc0fc0d1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:12.425 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.418+0000 7fc102170700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0fc0c8a20 con 0x7fc0fc06c730 2023-11-08T19:26:12.425 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.418+0000 7fc102170700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0fc0c8d00 con 0x7fc0fc06c030 2023-11-08T19:26:12.425 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.418+0000 7fc102170700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0fc0c8fe0 con 0x7fc0fc15de80 2023-11-08T19:26:12.425 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.418+0000 7fc0faffd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc0fc06c730 0x7fc0fc0d19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:12.426 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.418+0000 7fc0faffd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc0fc06c730 0x7fc0fc0d19a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51196/0 (socket says 172.21.15.143:51196) 2023-11-08T19:26:12.426 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.418+0000 7fc0fb7fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc0fc06c030 0x7fc0fc15ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:12.426 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.418+0000 7fc0faffd700 1 -- 172.21.15.143:0/26218289 learned_addr learned my addr 172.21.15.143:0/26218289 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:12.426 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.418+0000 7fc0fbfff700 1 --2- 172.21.15.143:0/26218289 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc0fc15de80 0x7fc0fc0d1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:12.426 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.418+0000 7fc0faffd700 1 -- 172.21.15.143:0/26218289 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc0fc15de80 msgr2=0x7fc0fc0d1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:12.427 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.418+0000 7fc0faffd700 1 --2- 172.21.15.143:0/26218289 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc0fc15de80 0x7fc0fc0d1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:12.427 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.418+0000 7fc0faffd700 1 -- 172.21.15.143:0/26218289 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc0fc06c030 msgr2=0x7fc0fc15ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:12.427 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.418+0000 7fc0faffd700 1 --2- 172.21.15.143:0/26218289 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc0fc06c030 0x7fc0fc15ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:12.427 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.418+0000 7fc0faffd700 1 -- 172.21.15.143:0/26218289 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0ec009550 con 0x7fc0fc06c730 2023-11-08T19:26:12.427 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.419+0000 7fc0faffd700 1 --2- 172.21.15.143:0/26218289 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc0fc06c730 0x7fc0fc0d19a0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fc0ec009130 tx=0x7fc0ec004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:12.428 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.419+0000 7fc0e3fff700 1 -- 172.21.15.143:0/26218289 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0ec003aa0 con 0x7fc0fc06c730 2023-11-08T19:26:12.428 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.419+0000 7fc0e3fff700 1 -- 172.21.15.143:0/26218289 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0ec01f070 con 0x7fc0fc06c730 2023-11-08T19:26:12.428 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.419+0000 7fc102170700 1 -- 172.21.15.143:0/26218289 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0fc0c92c0 con 0x7fc0fc06c730 2023-11-08T19:26:12.428 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.419+0000 7fc0e3fff700 1 -- 172.21.15.143:0/26218289 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0ec003e40 con 0x7fc0fc06c730 2023-11-08T19:26:12.428 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.419+0000 7fc102170700 1 -- 172.21.15.143:0/26218289 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0fc15f8b0 con 0x7fc0fc06c730 2023-11-08T19:26:12.428 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.420+0000 7fc102170700 1 -- 172.21.15.143:0/26218289 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0fc15fa40 con 0x7fc0fc06c730 2023-11-08T19:26:12.429 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.420+0000 7fc0e3fff700 1 -- 172.21.15.143:0/26218289 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc0ec021030 con 0x7fc0fc06c730 2023-11-08T19:26:12.429 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.420+0000 7fc0e3fff700 1 --2- 172.21.15.143:0/26218289 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc0e806af00 0x7fc0e806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:12.429 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.421+0000 7fc0e3fff700 1 -- 172.21.15.143:0/26218289 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6118+0+0 (secure 0 0 0) 0x7fc0ec08acc0 con 0x7fc0fc06c730 2023-11-08T19:26:12.429 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.421+0000 7fc0fb7fe700 1 --2- 172.21.15.143:0/26218289 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc0e806af00 0x7fc0e806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:12.429 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.421+0000 7fc0fb7fe700 1 --2- 172.21.15.143:0/26218289 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc0e806af00 0x7fc0e806d3c0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc0f0007fd0 tx=0x7fc0f0007ec0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:12.430 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.425+0000 7fc0e3fff700 1 -- 172.21.15.143:0/26218289 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc0ec055bc0 con 0x7fc0fc06c730 2023-11-08T19:26:12.668 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.667+0000 7fc102170700 1 -- 172.21.15.143:0/26218289 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fc0fc0c8b60 con 0x7fc0fc06c730 2023-11-08T19:26:12.668 INFO:tasks.ceph.mon.a.smithi143.stderr:2023-11-08T19:26:12.667+0000 7f65b584e700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-08T19:26:12.669 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.668+0000 7fc0e3fff700 1 -- 172.21.15.143:0/26218289 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v17) v1 ==== 161+0+0 (secure 0 0 0) 0x7fc0fc0c8b60 con 0x7fc0fc06c730 2023-11-08T19:26:12.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.671+0000 7fc102170700 1 -- 172.21.15.143:0/26218289 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc0e806af00 msgr2=0x7fc0e806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:12.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.671+0000 7fc102170700 1 --2- 172.21.15.143:0/26218289 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc0e806af00 0x7fc0e806d3c0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc0f0007fd0 tx=0x7fc0f0007ec0 comp rx=0 tx=0).stop 2023-11-08T19:26:12.674 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.671+0000 7fc102170700 1 -- 172.21.15.143:0/26218289 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc0fc06c730 msgr2=0x7fc0fc0d19a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:12.674 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.671+0000 7fc102170700 1 --2- 172.21.15.143:0/26218289 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc0fc06c730 0x7fc0fc0d19a0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fc0ec009130 tx=0x7fc0ec004750 comp rx=0 tx=0).stop 2023-11-08T19:26:12.674 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.672+0000 7fc102170700 1 -- 172.21.15.143:0/26218289 shutdown_connections 2023-11-08T19:26:12.674 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.672+0000 7fc102170700 1 --2- 172.21.15.143:0/26218289 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc0e806af00 0x7fc0e806d3c0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:12.674 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.672+0000 7fc102170700 1 --2- 172.21.15.143:0/26218289 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc0fc15de80 0x7fc0fc0d1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:12.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.672+0000 7fc102170700 1 --2- 172.21.15.143:0/26218289 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc0fc06c030 0x7fc0fc15ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:12.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.672+0000 7fc102170700 1 --2- 172.21.15.143:0/26218289 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc0fc06c730 0x7fc0fc0d19a0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:12.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.672+0000 7fc102170700 1 -- 172.21.15.143:0/26218289 >> 172.21.15.143:0/26218289 conn(0x7fc0fc08b000 msgr2=0x7fc0fc0d0340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:12.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.672+0000 7fc102170700 1 -- 172.21.15.143:0/26218289 shutdown_connections 2023-11-08T19:26:12.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.672+0000 7fc102170700 1 -- 172.21.15.143:0/26218289 wait complete. 2023-11-08T19:26:12.676 INFO:teuthology.orchestra.run.smithi143.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-08T19:26:12.676 INFO:tasks.ceph.mds.c.smithi143.stderr: -3131> 2023-11-08T19:25:52.845+0000 7f1a2ad00700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-08T19:26:12.688 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-08T19:26:12.706 INFO:tasks.ceph.mds.c.smithi143.stdout:starting mds.c at 2023-11-08T19:26:12.810 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.809+0000 7f9c055c7700 1 Processor -- start 2023-11-08T19:26:12.811 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.809+0000 7f9c055c7700 1 -- start start 2023-11-08T19:26:12.813 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.810+0000 7f9c055c7700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9c000b5ae0 0x7f9c000d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:12.813 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.810+0000 7f9c055c7700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c000b9b00 con 0x7f9c0015dca0 2023-11-08T19:26:12.814 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.810+0000 7f9c055c7700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c000b8f50 con 0x7f9c000b5630 2023-11-08T19:26:12.814 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.810+0000 7f9c055c7700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c000b3360 con 0x7f9c000b5ae0 2023-11-08T19:26:12.814 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.810+0000 7f9bfeffd700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f9c0015dca0 0x7f9c000b93f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:47364/0 (socket says 172.21.15.143:47364) 2023-11-08T19:26:12.814 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.810+0000 7f9bfeffd700 1 -- 172.21.15.143:0/3771331187 learned_addr learned my addr 172.21.15.143:0/3771331187 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:26:12.814 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.810+0000 7f9bff7fe700 1 --2- 172.21.15.143:0/3771331187 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9c000b5ae0 0x7f9c000d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:12.815 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.810+0000 7f9bfdffb700 1 -- 172.21.15.143:0/3771331187 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1762797453 0 0) 0x7f9c000b9b00 con 0x7f9c0015dca0 2023-11-08T19:26:12.815 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.810+0000 7f9bfdffb700 1 -- 172.21.15.143:0/3771331187 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9be80035b0 con 0x7f9c0015dca0 2023-11-08T19:26:12.815 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.810+0000 7f9bfdffb700 1 -- 172.21.15.143:0/3771331187 <== mon.1 v1:172.21.15.174:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3559887433 0 0) 0x7f9c000b8f50 con 0x7f9c000b5630 2023-11-08T19:26:12.815 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.810+0000 7f9bfdffb700 1 -- 172.21.15.143:0/3771331187 --> v1:172.21.15.174:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9c000b9b00 con 0x7f9c000b5630 2023-11-08T19:26:12.815 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.810+0000 7f9bff7fe700 1 -- 172.21.15.143:0/3771331187 >> v1:172.21.15.174:6789/0 conn(0x7f9c000b5630 legacy=0x7f9c000d1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:12.816 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.810+0000 7f9bff7fe700 1 -- 172.21.15.143:0/3771331187 >> v1:172.21.15.143:6789/0 conn(0x7f9c0015dca0 legacy=0x7f9c000b93f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:12.816 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.810+0000 7f9bff7fe700 1 -- 172.21.15.143:0/3771331187 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c000b7eb0 con 0x7f9c000b5ae0 2023-11-08T19:26:12.816 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.810+0000 7f9bff7fe700 1 --2- 172.21.15.143:0/3771331187 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9c000b5ae0 0x7f9c000d22e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9bf8009dc0 tx=0x7f9bf800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3dec7c4d69f859b8 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:12.816 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.811+0000 7f9bfdffb700 1 -- 172.21.15.143:0/3771331187 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf800e070 con 0x7f9c000b5ae0 2023-11-08T19:26:12.816 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.811+0000 7f9bfdffb700 1 -- 172.21.15.143:0/3771331187 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bf800bb20 con 0x7f9c000b5ae0 2023-11-08T19:26:12.817 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.811+0000 7f9bfdffb700 1 -- 172.21.15.143:0/3771331187 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf8012430 con 0x7f9c000b5ae0 2023-11-08T19:26:12.817 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.811+0000 7f9c055c7700 1 -- 172.21.15.143:0/3771331187 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9c000b5ae0 msgr2=0x7f9c000d22e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:12.817 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.811+0000 7f9c055c7700 1 --2- 172.21.15.143:0/3771331187 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9c000b5ae0 0x7f9c000d22e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9bf8009dc0 tx=0x7f9bf800b330 comp rx=0 tx=0).stop 2023-11-08T19:26:12.817 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.811+0000 7f9c055c7700 1 -- 172.21.15.143:0/3771331187 shutdown_connections 2023-11-08T19:26:12.817 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.811+0000 7f9c055c7700 1 --2- 172.21.15.143:0/3771331187 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9c000b5ae0 0x7f9c000d22e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:12.818 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.811+0000 7f9c055c7700 1 -- 172.21.15.143:0/3771331187 >> 172.21.15.143:0/3771331187 conn(0x7f9c00066d00 msgr2=0x7f9c00067110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:12.818 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.811+0000 7f9c055c7700 1 -- 172.21.15.143:0/3771331187 shutdown_connections 2023-11-08T19:26:12.818 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.811+0000 7f9c055c7700 1 -- 172.21.15.143:0/3771331187 wait complete. 2023-11-08T19:26:12.821 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.813+0000 7f9c055c7700 1 Processor -- start 2023-11-08T19:26:12.821 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.813+0000 7f9c055c7700 1 -- start start 2023-11-08T19:26:12.821 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.813+0000 7f9c055c7700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9c000b5630 0x7f9c000c98b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:12.822 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.813+0000 7f9c055c7700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9c000b5ae0 0x7f9c000c7530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:12.822 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.813+0000 7f9c055c7700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9c0015dca0 0x7f9c0015b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:12.822 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.813+0000 7f9c055c7700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c000c9df0 con 0x7f9c0015dca0 2023-11-08T19:26:12.822 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.813+0000 7f9c055c7700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c000c7d60 con 0x7f9c000b5ae0 2023-11-08T19:26:12.822 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.813+0000 7f9c055c7700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c000c8040 con 0x7f9c000b5630 2023-11-08T19:26:12.823 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.814+0000 7f9bff7fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9c0015dca0 0x7f9c0015b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:12.823 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.814+0000 7f9bff7fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9c0015dca0 0x7f9c0015b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51220/0 (socket says 172.21.15.143:51220) 2023-11-08T19:26:12.823 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.814+0000 7f9bff7fe700 1 -- 172.21.15.143:0/790434736 learned_addr learned my addr 172.21.15.143:0/790434736 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:12.823 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.814+0000 7f9bfe7fc700 1 --2- 172.21.15.143:0/790434736 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9c000b5ae0 0x7f9c000c7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:12.823 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.814+0000 7f9bfeffd700 1 --2- 172.21.15.143:0/790434736 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9c000b5630 0x7f9c000c98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:12.824 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.814+0000 7f9bff7fe700 1 -- 172.21.15.143:0/790434736 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9c000b5630 msgr2=0x7f9c000c98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:12.824 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.814+0000 7f9bff7fe700 1 --2- 172.21.15.143:0/790434736 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9c000b5630 0x7f9c000c98b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:12.824 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.814+0000 7f9bff7fe700 1 -- 172.21.15.143:0/790434736 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9c000b5ae0 msgr2=0x7f9c000c7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:12.824 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.814+0000 7f9bff7fe700 1 --2- 172.21.15.143:0/790434736 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9c000b5ae0 0x7f9c000c7530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:12.824 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.814+0000 7f9bff7fe700 1 -- 172.21.15.143:0/790434736 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bf8009a70 con 0x7f9c0015dca0 2023-11-08T19:26:12.825 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.814+0000 7f9bfe7fc700 1 --2- 172.21.15.143:0/790434736 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9c000b5ae0 0x7f9c000c7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:26:12.825 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.814+0000 7f9bff7fe700 1 --2- 172.21.15.143:0/790434736 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9c0015dca0 0x7f9c0015b1a0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f9bf800b370 tx=0x7f9bf8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:12.825 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.814+0000 7f9be77fe700 1 -- 172.21.15.143:0/790434736 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf801b810 con 0x7f9c0015dca0 2023-11-08T19:26:12.825 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.814+0000 7f9be77fe700 1 -- 172.21.15.143:0/790434736 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bf8020070 con 0x7f9c0015dca0 2023-11-08T19:26:12.825 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.814+0000 7f9be77fe700 1 -- 172.21.15.143:0/790434736 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf801bbf0 con 0x7f9c0015dca0 2023-11-08T19:26:12.826 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.814+0000 7f9c055c7700 1 -- 172.21.15.143:0/790434736 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c000c8320 con 0x7f9c0015dca0 2023-11-08T19:26:12.826 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.814+0000 7f9c055c7700 1 -- 172.21.15.143:0/790434736 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c000c7a70 con 0x7f9c0015dca0 2023-11-08T19:26:12.826 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.815+0000 7f9c055c7700 1 -- 172.21.15.143:0/790434736 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c0015ec00 con 0x7f9c0015dca0 2023-11-08T19:26:12.826 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.816+0000 7f9be77fe700 1 -- 172.21.15.143:0/790434736 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9bf801bd50 con 0x7f9c0015dca0 2023-11-08T19:26:12.826 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.816+0000 7f9be77fe700 1 --2- 172.21.15.143:0/790434736 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f9bec06af00 0x7f9bec06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:12.826 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.816+0000 7f9be77fe700 1 -- 172.21.15.143:0/790434736 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f9bf8022070 con 0x7f9c0015dca0 2023-11-08T19:26:12.827 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.816+0000 7f9bfeffd700 1 --2- 172.21.15.143:0/790434736 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f9bec06af00 0x7f9bec06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:12.827 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.817+0000 7f9bfeffd700 1 --2- 172.21.15.143:0/790434736 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f9bec06af00 0x7f9bec06d3c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f9bf40011e0 tx=0x7f9bf4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:12.827 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:12.820+0000 7f9be77fe700 1 -- 172.21.15.143:0/790434736 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9c0015ec00 con 0x7f9c0015dca0 2023-11-08T19:26:13.053 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:13.052+0000 7f9c055c7700 1 -- 172.21.15.143:0/790434736 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f9c0015efe0 con 0x7f9c0015dca0 2023-11-08T19:26:13.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:13.670+0000 7f9be77fe700 1 -- 172.21.15.143:0/790434736 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v18) v1 ==== 104+0+0 (secure 0 0 0) 0x7f9c0015efe0 con 0x7f9c0015dca0 2023-11-08T19:26:13.674 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:13.673+0000 7f9c055c7700 1 -- 172.21.15.143:0/790434736 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f9bec06af00 msgr2=0x7f9bec06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:13.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:13.673+0000 7f9c055c7700 1 --2- 172.21.15.143:0/790434736 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f9bec06af00 0x7f9bec06d3c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f9bf40011e0 tx=0x7f9bf4009040 comp rx=0 tx=0).stop 2023-11-08T19:26:13.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:13.673+0000 7f9c055c7700 1 -- 172.21.15.143:0/790434736 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9c0015dca0 msgr2=0x7f9c0015b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:13.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:13.673+0000 7f9c055c7700 1 --2- 172.21.15.143:0/790434736 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9c0015dca0 0x7f9c0015b1a0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f9bf800b370 tx=0x7f9bf8004ad0 comp rx=0 tx=0).stop 2023-11-08T19:26:13.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:13.673+0000 7f9c055c7700 1 -- 172.21.15.143:0/790434736 shutdown_connections 2023-11-08T19:26:13.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:13.673+0000 7f9c055c7700 1 --2- 172.21.15.143:0/790434736 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f9bec06af00 0x7f9bec06d3c0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:13.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:13.673+0000 7f9c055c7700 1 --2- 172.21.15.143:0/790434736 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9c000b5630 0x7f9c000c98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:13.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:13.673+0000 7f9c055c7700 1 --2- 172.21.15.143:0/790434736 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9c0015dca0 0x7f9c0015b1a0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:13.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:13.673+0000 7f9c055c7700 1 --2- 172.21.15.143:0/790434736 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9c000b5ae0 0x7f9c000c7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:13.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:13.673+0000 7f9c055c7700 1 -- 172.21.15.143:0/790434736 >> 172.21.15.143:0/790434736 conn(0x7f9c00066d00 msgr2=0x7f9c000d1a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:13.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:13.673+0000 7f9c055c7700 1 -- 172.21.15.143:0/790434736 shutdown_connections 2023-11-08T19:26:13.677 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:13.673+0000 7f9c055c7700 1 -- 172.21.15.143:0/790434736 wait complete. 2023-11-08T19:26:13.688 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-08T19:26:13.859 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:13.858+0000 7f5ac272c700 1 Processor -- start 2023-11-08T19:26:13.860 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:13.858+0000 7f5ac272c700 1 -- start start 2023-11-08T19:26:13.863 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:13.859+0000 7f5ac272c700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f5abc072010 0x7f5abc15b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:13.864 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:13.859+0000 7f5ac272c700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5abc072c40 con 0x7f5abc0cc350 2023-11-08T19:26:13.864 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:13.859+0000 7f5ac272c700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5abc15be90 con 0x7f5abc158610 2023-11-08T19:26:13.864 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:13.859+0000 7f5ac272c700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5abc0cf150 con 0x7f5abc072010 2023-11-08T19:26:13.864 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:13.859+0000 7f5ac0cc9700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f5abc0cc350 0x7f5abc06aff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:47384/0 (socket says 172.21.15.143:47384) 2023-11-08T19:26:13.864 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:13.859+0000 7f5ac0cc9700 1 -- 172.21.15.143:0/2187432563 learned_addr learned my addr 172.21.15.143:0/2187432563 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:26:13.865 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:13.859+0000 7f5abb7fe700 1 --2- 172.21.15.143:0/2187432563 >> v2:172.21.15.174:3300/0 conn(0x7f5abc072010 0x7f5abc15b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:13.865 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:13.859+0000 7f5abaffd700 1 -- 172.21.15.143:0/2187432563 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1651537233 0 0) 0x7f5abc072c40 con 0x7f5abc0cc350 2023-11-08T19:26:13.865 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:13.859+0000 7f5abaffd700 1 -- 172.21.15.143:0/2187432563 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5aa40035b0 con 0x7f5abc0cc350 2023-11-08T19:26:13.865 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:13.859+0000 7f5abaffd700 1 -- 172.21.15.143:0/2187432563 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3599999861 0 0) 0x7f5aa40035b0 con 0x7f5abc0cc350 2023-11-08T19:26:13.865 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:13.859+0000 7f5abaffd700 1 -- 172.21.15.143:0/2187432563 >> v2:172.21.15.174:3300/0 conn(0x7f5abc072010 msgr2=0x7f5abc15b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:13.866 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:13.859+0000 7f5abaffd700 1 --2- 172.21.15.143:0/2187432563 >> v2:172.21.15.174:3300/0 conn(0x7f5abc072010 0x7f5abc15b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:13.866 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:13.859+0000 7f5abaffd700 1 -- 172.21.15.143:0/2187432563 >> v1:172.21.15.174:6789/0 conn(0x7f5abc158610 legacy=0x7f5abc071900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:13.866 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:13.859+0000 7f5abaffd700 1 -- 172.21.15.143:0/2187432563 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5abc0cf360 con 0x7f5abc0cc350 2023-11-08T19:26:13.866 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:13.859+0000 7f5abaffd700 1 -- 172.21.15.143:0/2187432563 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4147756303 0 0) 0x7f5ab40019f0 con 0x7f5abc0cc350 2023-11-08T19:26:13.866 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:13.860+0000 7f5abaffd700 1 -- 172.21.15.143:0/2187432563 >> v1:172.21.15.143:6789/0 conn(0x7f5abc0cc350 legacy=0x7f5abc06aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:13.867 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:13.860+0000 7f5abaffd700 1 --2- 172.21.15.143:0/2187432563 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5aa4003a00 0x7f5aa4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:13.867 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:13.860+0000 7f5abaffd700 1 --2- 172.21.15.143:0/2187432563 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5aa4004400 0x7f5aa40048d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:13.867 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:13.860+0000 7f5abaffd700 1 --2- 172.21.15.143:0/2187432563 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5aa40050d0 0x7f5aa400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:13.867 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:13.860+0000 7f5abaffd700 1 -- 172.21.15.143:0/2187432563 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5abc0cf150 con 0x7f5aa4003a00 2023-11-08T19:26:13.867 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:13.860+0000 7f5abaffd700 1 -- 172.21.15.143:0/2187432563 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa400c030 con 0x7f5aa4004400 2023-11-08T19:26:13.868 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:13.860+0000 7f5abaffd700 1 -- 172.21.15.143:0/2187432563 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa400c1f0 con 0x7f5aa40050d0 2023-11-08T19:26:13.868 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:13.860+0000 7f5abbfff700 1 --2- 172.21.15.143:0/2187432563 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5aa4003a00 0x7f5aa4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:13.868 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:13.860+0000 7f5abb7fe700 1 --2- 172.21.15.143:0/2187432563 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5aa4004400 0x7f5aa40048d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:13.868 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:13.860+0000 7f5ac0cc9700 1 --2- 172.21.15.143:0/2187432563 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5aa40050d0 0x7f5aa400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:13.868 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:13.860+0000 7f5abbfff700 1 -- 172.21.15.143:0/2187432563 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5aa40050d0 msgr2=0x7f5aa400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:13.868 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:13.860+0000 7f5abbfff700 1 --2- 172.21.15.143:0/2187432563 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5aa40050d0 0x7f5aa400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:13.869 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:13.860+0000 7f5abbfff700 1 -- 172.21.15.143:0/2187432563 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5aa4004400 msgr2=0x7f5aa40048d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:13.869 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:13.860+0000 7f5abbfff700 1 --2- 172.21.15.143:0/2187432563 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5aa4004400 0x7f5aa40048d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:13.869 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:13.860+0000 7f5ac0cc9700 1 -- 172.21.15.143:0/2187432563 reap_dead start 2023-11-08T19:26:13.869 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:13.860+0000 7f5abbfff700 1 -- 172.21.15.143:0/2187432563 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5aa40035b0 con 0x7f5aa4003a00 2023-11-08T19:26:13.869 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:13.860+0000 7f5abbfff700 1 --2- 172.21.15.143:0/2187432563 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5aa4003a00 0x7f5aa4003de0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f5aac0018c0 tx=0x7f5aa4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34f207ef280651b1 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:13.870 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:13.860+0000 7f5abaffd700 1 -- 172.21.15.143:0/2187432563 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aac00c660 con 0x7f5aa4003a00 2023-11-08T19:26:13.870 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:13.860+0000 7f5abaffd700 1 -- 172.21.15.143:0/2187432563 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5aac00c7c0 con 0x7f5aa4003a00 2023-11-08T19:26:13.870 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:13.860+0000 7f5abaffd700 1 -- 172.21.15.143:0/2187432563 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aac00cb70 con 0x7f5aa4003a00 2023-11-08T19:26:13.870 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:13.861+0000 7f5ac272c700 1 -- 172.21.15.143:0/2187432563 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5aa4003a00 msgr2=0x7f5aa4003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:13.870 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:13.861+0000 7f5ac272c700 1 --2- 172.21.15.143:0/2187432563 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5aa4003a00 0x7f5aa4003de0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f5aac0018c0 tx=0x7f5aa4005880 comp rx=0 tx=0).stop 2023-11-08T19:26:13.871 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:13.861+0000 7f5ac272c700 1 -- 172.21.15.143:0/2187432563 shutdown_connections 2023-11-08T19:26:13.871 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:13.861+0000 7f5ac272c700 1 --2- 172.21.15.143:0/2187432563 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5aa4003a00 0x7f5aa4003de0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:13.871 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:13.861+0000 7f5ac272c700 1 -- 172.21.15.143:0/2187432563 >> 172.21.15.143:0/2187432563 conn(0x7f5abc0b1c70 msgr2=0x7f5abc0b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:13.871 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:13.861+0000 7f5ac272c700 1 -- 172.21.15.143:0/2187432563 shutdown_connections 2023-11-08T19:26:13.871 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:13.861+0000 7f5ac272c700 1 -- 172.21.15.143:0/2187432563 wait complete. 2023-11-08T19:26:13.872 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:13.862+0000 7f5ac272c700 1 Processor -- start 2023-11-08T19:26:13.872 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:13.862+0000 7f5ac272c700 1 -- start start 2023-11-08T19:26:13.874 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:13.863+0000 7f5ac272c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5aa4003a00 0x7f5abc1f5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:13.875 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:13.863+0000 7f5ac272c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5abc1f6e00 0x7f5abc1f49c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:13.875 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:13.863+0000 7f5ac272c700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5abc0b3550 0x7f5abc0ccac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:13.875 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:13.863+0000 7f5ac272c700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5abc1f7280 con 0x7f5aa4003a00 2023-11-08T19:26:13.875 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:13.863+0000 7f5ac272c700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5abc0c8710 con 0x7f5abc1f6e00 2023-11-08T19:26:13.875 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:13.863+0000 7f5ac272c700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5abc0b39d0 con 0x7f5abc0b3550 2023-11-08T19:26:13.876 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:13.863+0000 7f5abbfff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5aa4003a00 0x7f5abc1f5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:13.876 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:13.863+0000 7f5abbfff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5aa4003a00 0x7f5abc1f5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51238/0 (socket says 172.21.15.143:51238) 2023-11-08T19:26:13.876 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:13.863+0000 7f5abbfff700 1 -- 172.21.15.143:0/2649702293 learned_addr learned my addr 172.21.15.143:0/2649702293 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:13.876 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:13.863+0000 7f5abb7fe700 1 --2- 172.21.15.143:0/2649702293 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5abc1f6e00 0x7f5abc1f49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:13.876 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:13.863+0000 7f5ac0cc9700 1 --2- 172.21.15.143:0/2649702293 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5abc0b3550 0x7f5abc0ccac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:13.877 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:13.863+0000 7f5abbfff700 1 -- 172.21.15.143:0/2649702293 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5abc0b3550 msgr2=0x7f5abc0ccac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:13.877 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:13.863+0000 7f5abbfff700 1 --2- 172.21.15.143:0/2649702293 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5abc0b3550 0x7f5abc0ccac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:13.877 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:13.863+0000 7f5abbfff700 1 -- 172.21.15.143:0/2649702293 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5abc1f6e00 msgr2=0x7f5abc1f49c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:13.877 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:13.863+0000 7f5abbfff700 1 --2- 172.21.15.143:0/2649702293 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5abc1f6e00 0x7f5abc1f49c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:13.878 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:13.863+0000 7f5abbfff700 1 -- 172.21.15.143:0/2649702293 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5abc0b3c80 con 0x7f5aa4003a00 2023-11-08T19:26:13.878 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:13.863+0000 7f5abb7fe700 1 --2- 172.21.15.143:0/2649702293 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5abc1f6e00 0x7f5abc1f49c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:26:13.878 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:13.863+0000 7f5ac0cc9700 1 --2- 172.21.15.143:0/2649702293 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5abc0b3550 0x7f5abc0ccac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:26:13.878 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:13.863+0000 7f5abbfff700 1 --2- 172.21.15.143:0/2649702293 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5aa4003a00 0x7f5abc1f5050 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f5aa400c000 tx=0x7f5aac0013c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:13.878 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:13.864+0000 7f5ab8ff9700 1 -- 172.21.15.143:0/2649702293 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aac0054e0 con 0x7f5aa4003a00 2023-11-08T19:26:13.879 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:13.864+0000 7f5ab8ff9700 1 -- 172.21.15.143:0/2649702293 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5aac00cec0 con 0x7f5aa4003a00 2023-11-08T19:26:13.879 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:13.864+0000 7f5ac272c700 1 -- 172.21.15.143:0/2649702293 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5abc0b3f60 con 0x7f5aa4003a00 2023-11-08T19:26:13.879 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:13.864+0000 7f5ab8ff9700 1 -- 172.21.15.143:0/2649702293 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aac006a80 con 0x7f5aa4003a00 2023-11-08T19:26:13.879 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:13.864+0000 7f5ac272c700 1 -- 172.21.15.143:0/2649702293 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5abc0b4520 con 0x7f5aa4003a00 2023-11-08T19:26:13.879 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:13.864+0000 7f5ac272c700 1 -- 172.21.15.143:0/2649702293 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5abc04eee0 con 0x7f5aa4003a00 2023-11-08T19:26:13.879 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:13.865+0000 7f5ab8ff9700 1 -- 172.21.15.143:0/2649702293 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5aac022070 con 0x7f5aa4003a00 2023-11-08T19:26:13.880 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:13.865+0000 7f5ab8ff9700 1 --2- 172.21.15.143:0/2649702293 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f5aa806af00 0x7f5aa806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:13.880 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:13.865+0000 7f5ab8ff9700 1 -- 172.21.15.143:0/2649702293 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f5aac08a310 con 0x7f5aa4003a00 2023-11-08T19:26:13.880 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:13.866+0000 7f5abb7fe700 1 --2- 172.21.15.143:0/2649702293 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f5aa806af00 0x7f5aa806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:13.880 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:13.866+0000 7f5abb7fe700 1 --2- 172.21.15.143:0/2649702293 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f5aa806af00 0x7f5aa806d3c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f5abc06bfd0 tx=0x7f5ab0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:13.880 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:13.870+0000 7f5ab8ff9700 1 -- 172.21.15.143:0/2649702293 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5aac0553b0 con 0x7f5aa4003a00 2023-11-08T19:26:14.061 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:14.060+0000 7f5ac272c700 1 -- 172.21.15.143:0/2649702293 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f5abc15be90 con 0x7f5aa4003a00 2023-11-08T19:26:14.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:14.674+0000 7f5ab8ff9700 1 -- 172.21.15.143:0/2649702293 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v39) v1 ==== 181+0+0 (secure 0 0 0) 0x7f5abc15be90 con 0x7f5aa4003a00 2023-11-08T19:26:14.679 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:14.678+0000 7f5ac272c700 1 -- 172.21.15.143:0/2649702293 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f5aa806af00 msgr2=0x7f5aa806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:14.679 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:14.678+0000 7f5ac272c700 1 --2- 172.21.15.143:0/2649702293 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f5aa806af00 0x7f5aa806d3c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f5abc06bfd0 tx=0x7f5ab0008040 comp rx=0 tx=0).stop 2023-11-08T19:26:14.680 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:14.678+0000 7f5ac272c700 1 -- 172.21.15.143:0/2649702293 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5aa4003a00 msgr2=0x7f5abc1f5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:14.680 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:14.678+0000 7f5ac272c700 1 --2- 172.21.15.143:0/2649702293 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5aa4003a00 0x7f5abc1f5050 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f5aa400c000 tx=0x7f5aac0013c0 comp rx=0 tx=0).stop 2023-11-08T19:26:14.681 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:14.678+0000 7f5ac272c700 1 -- 172.21.15.143:0/2649702293 shutdown_connections 2023-11-08T19:26:14.681 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:14.678+0000 7f5ac272c700 1 --2- 172.21.15.143:0/2649702293 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f5aa806af00 0x7f5aa806d3c0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:14.682 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:14.678+0000 7f5ac272c700 1 --2- 172.21.15.143:0/2649702293 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5abc0b3550 0x7f5abc0ccac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:14.682 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:14.678+0000 7f5ac272c700 1 --2- 172.21.15.143:0/2649702293 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5aa4003a00 0x7f5abc1f5050 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:14.682 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:14.678+0000 7f5ac272c700 1 --2- 172.21.15.143:0/2649702293 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5abc1f6e00 0x7f5abc1f49c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:14.682 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:14.678+0000 7f5ac272c700 1 -- 172.21.15.143:0/2649702293 >> 172.21.15.143:0/2649702293 conn(0x7f5abc0b1c70 msgr2=0x7f5abc0afbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:14.683 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:14.679+0000 7f5ac272c700 1 -- 172.21.15.143:0/2649702293 shutdown_connections 2023-11-08T19:26:14.683 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:14.679+0000 7f5ac272c700 1 -- 172.21.15.143:0/2649702293 wait complete. 2023-11-08T19:26:14.684 INFO:teuthology.orchestra.run.smithi143.stderr:pool 'cephfs_metadata' removed 2023-11-08T19:26:14.696 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-08T19:26:14.826 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:14.825+0000 7f9f3d343700 1 Processor -- start 2023-11-08T19:26:14.827 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:14.825+0000 7f9f3d343700 1 -- start start 2023-11-08T19:26:14.829 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:14.826+0000 7f9f3d343700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f9f381610b0 0x7f9f380cf560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:14.830 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:14.826+0000 7f9f3d343700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f9f380ceab0 0x7f9f381622a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:14.830 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:14.826+0000 7f9f3d343700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f381627e0 con 0x7f9f380cc120 2023-11-08T19:26:14.830 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:14.826+0000 7f9f3d343700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f380d0350 con 0x7f9f381610b0 2023-11-08T19:26:14.830 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:14.826+0000 7f9f3d343700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f380cfb40 con 0x7f9f380ceab0 2023-11-08T19:26:14.830 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:14.826+0000 7f9f36ffd700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f9f381610b0 0x7f9f380cf560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:14.831 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:14.826+0000 7f9f367fc700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f9f380cc120 0x7f9f380ce3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:47400/0 (socket says 172.21.15.143:47400) 2023-11-08T19:26:14.831 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:14.826+0000 7f9f36ffd700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f9f381610b0 0x7f9f380cf560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51246/0 (socket says 172.21.15.143:51246) 2023-11-08T19:26:14.831 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:14.826+0000 7f9f367fc700 1 -- 172.21.15.143:0/1709270612 learned_addr learned my addr 172.21.15.143:0/1709270612 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:26:14.831 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:14.826+0000 7f9f377fe700 1 --2- 172.21.15.143:0/1709270612 >> v2:172.21.15.174:3300/0 conn(0x7f9f380ceab0 0x7f9f381622a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:14.831 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:14.826+0000 7f9f36ffd700 1 -- 172.21.15.143:0/1709270612 >> v2:172.21.15.174:3300/0 conn(0x7f9f380ceab0 msgr2=0x7f9f381622a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:14.832 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:14.826+0000 7f9f36ffd700 1 --2- 172.21.15.143:0/1709270612 >> v2:172.21.15.174:3300/0 conn(0x7f9f380ceab0 0x7f9f381622a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:14.832 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:14.826+0000 7f9f36ffd700 1 -- 172.21.15.143:0/1709270612 >> v1:172.21.15.143:6789/0 conn(0x7f9f380cc120 legacy=0x7f9f380ce3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:14.832 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:14.826+0000 7f9f35ffb700 1 -- 172.21.15.143:0/1709270612 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2917098767 0 0) 0x7f9f28001110 con 0x7f9f380cc120 2023-11-08T19:26:14.832 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:14.826+0000 7f9f36ffd700 1 -- 172.21.15.143:0/1709270612 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f3806c3c0 con 0x7f9f381610b0 2023-11-08T19:26:14.832 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:14.826+0000 7f9f36ffd700 1 --2- 172.21.15.143:0/1709270612 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9f381610b0 0x7f9f380cf560 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f9f380cf470 tx=0x7f9f2c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=173b4d468fafc35c server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:14.833 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:14.827+0000 7f9f35ffb700 1 -- 172.21.15.143:0/1709270612 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f2c011830 con 0x7f9f381610b0 2023-11-08T19:26:14.833 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:14.827+0000 7f9f35ffb700 1 -- 172.21.15.143:0/1709270612 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f2c011990 con 0x7f9f381610b0 2023-11-08T19:26:14.833 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:14.827+0000 7f9f35ffb700 1 -- 172.21.15.143:0/1709270612 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f2c011d20 con 0x7f9f381610b0 2023-11-08T19:26:14.833 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:14.827+0000 7f9f3d343700 1 -- 172.21.15.143:0/1709270612 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9f381610b0 msgr2=0x7f9f380cf560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:14.833 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:14.827+0000 7f9f3d343700 1 --2- 172.21.15.143:0/1709270612 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9f381610b0 0x7f9f380cf560 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f9f380cf470 tx=0x7f9f2c009b20 comp rx=0 tx=0).stop 2023-11-08T19:26:14.834 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:14.827+0000 7f9f3d343700 1 -- 172.21.15.143:0/1709270612 shutdown_connections 2023-11-08T19:26:14.834 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:14.827+0000 7f9f3d343700 1 --2- 172.21.15.143:0/1709270612 >> v2:172.21.15.174:3300/0 conn(0x7f9f380ceab0 0x7f9f381622a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:14.834 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:14.827+0000 7f9f3d343700 1 --2- 172.21.15.143:0/1709270612 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9f381610b0 0x7f9f380cf560 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:14.834 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:14.827+0000 7f9f3d343700 1 -- 172.21.15.143:0/1709270612 >> 172.21.15.143:0/1709270612 conn(0x7f9f3806f2e0 msgr2=0x7f9f3806f6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:14.835 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:14.827+0000 7f9f3d343700 1 -- 172.21.15.143:0/1709270612 shutdown_connections 2023-11-08T19:26:14.835 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:14.827+0000 7f9f3d343700 1 -- 172.21.15.143:0/1709270612 wait complete. 2023-11-08T19:26:14.838 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:14.829+0000 7f9f3d343700 1 Processor -- start 2023-11-08T19:26:14.838 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:14.829+0000 7f9f3d343700 1 -- start start 2023-11-08T19:26:14.838 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:14.829+0000 7f9f3d343700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9f380cc120 0x7f9f380b30b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:14.838 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:14.829+0000 7f9f3d343700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9f380ceab0 0x7f9f38157f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:14.839 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:14.829+0000 7f9f3d343700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9f381610b0 0x7f9f380b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:14.839 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:14.829+0000 7f9f3d343700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f381f6e30 con 0x7f9f380cc120 2023-11-08T19:26:14.839 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:14.829+0000 7f9f3d343700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f380c5960 con 0x7f9f381610b0 2023-11-08T19:26:14.839 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:14.829+0000 7f9f3d343700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f380c5c40 con 0x7f9f380ceab0 2023-11-08T19:26:14.839 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:14.829+0000 7f9f36ffd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9f380cc120 0x7f9f380b30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:14.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:14.829+0000 7f9f36ffd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9f380cc120 0x7f9f380b30b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51250/0 (socket says 172.21.15.143:51250) 2023-11-08T19:26:14.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:14.829+0000 7f9f36ffd700 1 -- 172.21.15.143:0/2072600250 learned_addr learned my addr 172.21.15.143:0/2072600250 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:14.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:14.830+0000 7f9f377fe700 1 --2- 172.21.15.143:0/2072600250 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9f381610b0 0x7f9f380b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:14.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:14.830+0000 7f9f367fc700 1 --2- 172.21.15.143:0/2072600250 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9f380ceab0 0x7f9f38157f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:14.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:14.830+0000 7f9f36ffd700 1 -- 172.21.15.143:0/2072600250 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9f380ceab0 msgr2=0x7f9f38157f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:14.841 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:14.830+0000 7f9f36ffd700 1 --2- 172.21.15.143:0/2072600250 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9f380ceab0 0x7f9f38157f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:14.841 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:14.830+0000 7f9f36ffd700 1 -- 172.21.15.143:0/2072600250 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9f381610b0 msgr2=0x7f9f380b4de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:14.841 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:14.830+0000 7f9f36ffd700 1 --2- 172.21.15.143:0/2072600250 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9f381610b0 0x7f9f380b4de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:14.841 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:14.830+0000 7f9f36ffd700 1 -- 172.21.15.143:0/2072600250 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f2c0094a0 con 0x7f9f380cc120 2023-11-08T19:26:14.841 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:14.830+0000 7f9f367fc700 1 --2- 172.21.15.143:0/2072600250 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9f380ceab0 0x7f9f38157f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:26:14.842 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:14.830+0000 7f9f36ffd700 1 --2- 172.21.15.143:0/2072600250 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9f380cc120 0x7f9f380b30b0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c0090b0 tx=0x7f9f2c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:14.842 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:14.830+0000 7f9f1f7fe700 1 -- 172.21.15.143:0/2072600250 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f2c0038a0 con 0x7f9f380cc120 2023-11-08T19:26:14.842 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:14.830+0000 7f9f1f7fe700 1 -- 172.21.15.143:0/2072600250 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f2c01f070 con 0x7f9f380cc120 2023-11-08T19:26:14.842 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:14.830+0000 7f9f3d343700 1 -- 172.21.15.143:0/2072600250 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f380c5f20 con 0x7f9f380cc120 2023-11-08T19:26:14.842 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:14.830+0000 7f9f1f7fe700 1 -- 172.21.15.143:0/2072600250 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f2c003c20 con 0x7f9f380cc120 2023-11-08T19:26:14.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:14.830+0000 7f9f3d343700 1 -- 172.21.15.143:0/2072600250 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f380b35f0 con 0x7f9f380cc120 2023-11-08T19:26:14.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:14.831+0000 7f9f3d343700 1 -- 172.21.15.143:0/2072600250 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f380b3780 con 0x7f9f380cc120 2023-11-08T19:26:14.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:14.831+0000 7f9f1f7fe700 1 -- 172.21.15.143:0/2072600250 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9f2c01a430 con 0x7f9f380cc120 2023-11-08T19:26:14.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:14.832+0000 7f9f1f7fe700 1 --2- 172.21.15.143:0/2072600250 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f9f2406af00 0x7f9f2406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:14.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:14.832+0000 7f9f1f7fe700 1 -- 172.21.15.143:0/2072600250 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5710+0+0 (secure 0 0 0) 0x7f9f2c021070 con 0x7f9f380cc120 2023-11-08T19:26:14.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:14.832+0000 7f9f367fc700 1 --2- 172.21.15.143:0/2072600250 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f9f2406af00 0x7f9f2406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:14.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:14.833+0000 7f9f367fc700 1 --2- 172.21.15.143:0/2072600250 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f9f2406af00 0x7f9f2406d3c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f9f28007fd0 tx=0x7f9f28009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:14.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:14.836+0000 7f9f1f7fe700 1 -- 172.21.15.143:0/2072600250 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9f380b3780 con 0x7f9f380cc120 2023-11-08T19:26:15.030 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:15.028+0000 7f9f3d343700 1 -- 172.21.15.143:0/2072600250 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f9f380b3780 con 0x7f9f380cc120 2023-11-08T19:26:15.681 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:15.680+0000 7f9f1f7fe700 1 -- 172.21.15.143:0/2072600250 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v40) v1 ==== 169+0+0 (secure 0 0 0) 0x7f9f380b3780 con 0x7f9f380cc120 2023-11-08T19:26:15.686 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:15.684+0000 7f9f3d343700 1 -- 172.21.15.143:0/2072600250 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f9f2406af00 msgr2=0x7f9f2406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:15.686 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:15.684+0000 7f9f3d343700 1 --2- 172.21.15.143:0/2072600250 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f9f2406af00 0x7f9f2406d3c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f9f28007fd0 tx=0x7f9f28009040 comp rx=0 tx=0).stop 2023-11-08T19:26:15.686 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:15.684+0000 7f9f3d343700 1 -- 172.21.15.143:0/2072600250 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9f380cc120 msgr2=0x7f9f380b30b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:15.686 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:15.684+0000 7f9f3d343700 1 --2- 172.21.15.143:0/2072600250 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9f380cc120 0x7f9f380b30b0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c0090b0 tx=0x7f9f2c004750 comp rx=0 tx=0).stop 2023-11-08T19:26:15.686 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:15.684+0000 7f9f3d343700 1 -- 172.21.15.143:0/2072600250 shutdown_connections 2023-11-08T19:26:15.687 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:15.684+0000 7f9f3d343700 1 --2- 172.21.15.143:0/2072600250 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f9f2406af00 0x7f9f2406d3c0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:15.687 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:15.684+0000 7f9f3d343700 1 --2- 172.21.15.143:0/2072600250 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9f381610b0 0x7f9f380b4de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:15.687 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:15.684+0000 7f9f3d343700 1 --2- 172.21.15.143:0/2072600250 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9f380cc120 0x7f9f380b30b0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:15.687 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:15.684+0000 7f9f3d343700 1 --2- 172.21.15.143:0/2072600250 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9f380ceab0 0x7f9f38157f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:15.687 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:15.684+0000 7f9f3d343700 1 -- 172.21.15.143:0/2072600250 >> 172.21.15.143:0/2072600250 conn(0x7f9f3806f2e0 msgr2=0x7f9f380ce3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:15.688 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:15.684+0000 7f9f3d343700 1 -- 172.21.15.143:0/2072600250 shutdown_connections 2023-11-08T19:26:15.688 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:15.684+0000 7f9f3d343700 1 -- 172.21.15.143:0/2072600250 wait complete. 2023-11-08T19:26:15.688 INFO:teuthology.orchestra.run.smithi143.stderr:pool 'cephfs_data' removed 2023-11-08T19:26:15.702 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-08T19:26:15.835 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:15.834+0000 7f926615e700 1 Processor -- start 2023-11-08T19:26:15.836 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:15.834+0000 7f926615e700 1 -- start start 2023-11-08T19:26:15.839 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:15.835+0000 7f926615e700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f926015a950 0x7f92600d01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:15.839 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:15.835+0000 7f926615e700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f92600b61e0 0x7f92600cc730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:15.839 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:15.835+0000 7f926615e700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92600cc2a0 con 0x7f926006bd20 2023-11-08T19:26:15.839 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:15.835+0000 7f926615e700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92600cc130 con 0x7f92600b61e0 2023-11-08T19:26:15.839 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:15.835+0000 7f926615e700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f926006c1b0 con 0x7f926015a950 2023-11-08T19:26:15.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:15.835+0000 7f925ffff700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f92600b61e0 0x7f92600cc730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:15.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:15.835+0000 7f925ffff700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f92600b61e0 0x7f92600cc730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51262/0 (socket says 172.21.15.143:51262) 2023-11-08T19:26:15.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:15.835+0000 7f925ffff700 1 -- 172.21.15.143:0/4048562610 learned_addr learned my addr 172.21.15.143:0/4048562610 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:15.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:15.835+0000 7f925f7fe700 1 --2- 172.21.15.143:0/4048562610 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f926015a950 0x7f92600d01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:15.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:15.835+0000 7f925ffff700 1 -- 172.21.15.143:0/4048562610 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f926015a950 msgr2=0x7f92600d01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:15.841 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:15.835+0000 7f925ffff700 1 --2- 172.21.15.143:0/4048562610 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f926015a950 0x7f92600d01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:15.841 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:15.835+0000 7f925ffff700 1 -- 172.21.15.143:0/4048562610 >> v1:172.21.15.174:6789/0 conn(0x7f926006bd20 legacy=0x7f9260072370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:15.841 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:15.835+0000 7f925ffff700 1 -- 172.21.15.143:0/4048562610 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f926006c490 con 0x7f92600b61e0 2023-11-08T19:26:15.841 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:15.835+0000 7f925ffff700 1 --2- 172.21.15.143:0/4048562610 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f92600b61e0 0x7f92600cc730 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f925800d150 tx=0x7f925800d460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d13bf0baa7866c85 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:15.841 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:15.836+0000 7f925e7fc700 1 -- 172.21.15.143:0/4048562610 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9258010070 con 0x7f92600b61e0 2023-11-08T19:26:15.842 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:15.836+0000 7f925e7fc700 1 -- 172.21.15.143:0/4048562610 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f925800ddc0 con 0x7f92600b61e0 2023-11-08T19:26:15.842 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:15.836+0000 7f925e7fc700 1 -- 172.21.15.143:0/4048562610 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f925800b310 con 0x7f92600b61e0 2023-11-08T19:26:15.842 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:15.836+0000 7f926615e700 1 -- 172.21.15.143:0/4048562610 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f92600b61e0 msgr2=0x7f92600cc730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:15.842 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:15.836+0000 7f926615e700 1 --2- 172.21.15.143:0/4048562610 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f92600b61e0 0x7f92600cc730 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f925800d150 tx=0x7f925800d460 comp rx=0 tx=0).stop 2023-11-08T19:26:15.842 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:15.836+0000 7f926615e700 1 -- 172.21.15.143:0/4048562610 shutdown_connections 2023-11-08T19:26:15.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:15.836+0000 7f926615e700 1 --2- 172.21.15.143:0/4048562610 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f92600b61e0 0x7f92600cc730 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:15.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:15.836+0000 7f926615e700 1 --2- 172.21.15.143:0/4048562610 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f926015a950 0x7f92600d01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:15.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:15.836+0000 7f926615e700 1 -- 172.21.15.143:0/4048562610 >> 172.21.15.143:0/4048562610 conn(0x7f92600cd2e0 msgr2=0x7f92600cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:15.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:15.836+0000 7f926615e700 1 -- 172.21.15.143:0/4048562610 shutdown_connections 2023-11-08T19:26:15.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:15.836+0000 7f926615e700 1 -- 172.21.15.143:0/4048562610 wait complete. 2023-11-08T19:26:15.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:15.837+0000 7f926615e700 1 Processor -- start 2023-11-08T19:26:15.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:15.838+0000 7f926615e700 1 -- start start 2023-11-08T19:26:15.846 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:15.838+0000 7f926615e700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f92600b61e0 0x7f9260158000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:15.847 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:15.838+0000 7f926615e700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f926015a950 0x7f92600b0990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:15.847 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:15.838+0000 7f926615e700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f92600c6f00 0x7f92600b1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:15.847 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:15.838+0000 7f926615e700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92600b3720 con 0x7f92600b61e0 2023-11-08T19:26:15.847 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:15.838+0000 7f926615e700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92600b3dd0 con 0x7f926015a950 2023-11-08T19:26:15.847 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:15.838+0000 7f926615e700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92600b40b0 con 0x7f92600c6f00 2023-11-08T19:26:15.848 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:15.838+0000 7f925f7fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f92600b61e0 0x7f9260158000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:15.848 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:15.838+0000 7f925f7fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f92600b61e0 0x7f9260158000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51264/0 (socket says 172.21.15.143:51264) 2023-11-08T19:26:15.848 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:15.838+0000 7f925f7fe700 1 -- 172.21.15.143:0/3961256947 learned_addr learned my addr 172.21.15.143:0/3961256947 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:15.848 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:15.839+0000 7f925ffff700 1 --2- 172.21.15.143:0/3961256947 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f92600c6f00 0x7f92600b1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:15.848 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:15.839+0000 7f925effd700 1 --2- 172.21.15.143:0/3961256947 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f926015a950 0x7f92600b0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:15.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:15.839+0000 7f925f7fe700 1 -- 172.21.15.143:0/3961256947 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f92600c6f00 msgr2=0x7f92600b1bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:15.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:15.839+0000 7f925f7fe700 1 --2- 172.21.15.143:0/3961256947 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f92600c6f00 0x7f92600b1bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:15.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:15.839+0000 7f925f7fe700 1 -- 172.21.15.143:0/3961256947 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f926015a950 msgr2=0x7f92600b0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:15.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:15.839+0000 7f925f7fe700 1 --2- 172.21.15.143:0/3961256947 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f926015a950 0x7f92600b0990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:15.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:15.839+0000 7f925f7fe700 1 -- 172.21.15.143:0/3961256947 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f925800bd40 con 0x7f92600b61e0 2023-11-08T19:26:15.850 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:15.839+0000 7f925f7fe700 1 --2- 172.21.15.143:0/3961256947 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f92600b61e0 0x7f9260158000 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f925400d800 tx=0x7f925400dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:15.850 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:15.839+0000 7f9247fff700 1 -- 172.21.15.143:0/3961256947 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92540125f0 con 0x7f92600b61e0 2023-11-08T19:26:15.850 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:15.839+0000 7f9247fff700 1 -- 172.21.15.143:0/3961256947 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9254012750 con 0x7f92600b61e0 2023-11-08T19:26:15.850 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:15.839+0000 7f926615e700 1 -- 172.21.15.143:0/3961256947 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92600b4390 con 0x7f92600b61e0 2023-11-08T19:26:15.850 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:15.839+0000 7f9247fff700 1 -- 172.21.15.143:0/3961256947 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9254012b00 con 0x7f92600b61e0 2023-11-08T19:26:15.851 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:15.839+0000 7f926615e700 1 -- 172.21.15.143:0/3961256947 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92600b4950 con 0x7f92600b61e0 2023-11-08T19:26:15.851 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:15.840+0000 7f926615e700 1 -- 172.21.15.143:0/3961256947 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92600b4ae0 con 0x7f92600b61e0 2023-11-08T19:26:15.851 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:15.840+0000 7f9247fff700 1 -- 172.21.15.143:0/3961256947 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9254012c60 con 0x7f92600b61e0 2023-11-08T19:26:15.851 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:15.841+0000 7f9247fff700 1 --2- 172.21.15.143:0/3961256947 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f924c06af00 0x7f924c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:15.851 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:15.841+0000 7f9247fff700 1 -- 172.21.15.143:0/3961256947 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f92540897e0 con 0x7f92600b61e0 2023-11-08T19:26:15.852 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:15.841+0000 7f925effd700 1 --2- 172.21.15.143:0/3961256947 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f924c06af00 0x7f924c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:15.852 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:15.841+0000 7f925effd700 1 --2- 172.21.15.143:0/3961256947 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f924c06af00 0x7f924c06d3c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f9260159b10 tx=0x7f9250009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:15.852 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:15.845+0000 7f9247fff700 1 -- 172.21.15.143:0/3961256947 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9254054a30 con 0x7f92600b61e0 2023-11-08T19:26:16.037 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.035+0000 7f926615e700 1 -- 172.21.15.143:0/3961256947 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f92600b4cc0 con 0x7f92600b61e0 2023-11-08T19:26:16.039 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.038+0000 7f9247fff700 1 -- 172.21.15.143:0/3961256947 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v12) v1 ==== 104+0+0 (secure 0 0 0) 0x7f9254058050 con 0x7f92600b61e0 2023-11-08T19:26:16.042 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.040+0000 7f926615e700 1 -- 172.21.15.143:0/3961256947 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f924c06af00 msgr2=0x7f924c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:16.042 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.040+0000 7f926615e700 1 --2- 172.21.15.143:0/3961256947 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f924c06af00 0x7f924c06d3c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f9260159b10 tx=0x7f9250009040 comp rx=0 tx=0).stop 2023-11-08T19:26:16.042 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.040+0000 7f926615e700 1 -- 172.21.15.143:0/3961256947 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f92600b61e0 msgr2=0x7f9260158000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:16.043 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.040+0000 7f926615e700 1 --2- 172.21.15.143:0/3961256947 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f92600b61e0 0x7f9260158000 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f925400d800 tx=0x7f925400dbc0 comp rx=0 tx=0).stop 2023-11-08T19:26:16.043 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.040+0000 7f926615e700 1 -- 172.21.15.143:0/3961256947 shutdown_connections 2023-11-08T19:26:16.043 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.040+0000 7f926615e700 1 --2- 172.21.15.143:0/3961256947 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f924c06af00 0x7f924c06d3c0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:16.043 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.040+0000 7f926615e700 1 --2- 172.21.15.143:0/3961256947 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f92600c6f00 0x7f92600b1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:16.043 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.040+0000 7f926615e700 1 --2- 172.21.15.143:0/3961256947 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f92600b61e0 0x7f9260158000 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:16.044 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.040+0000 7f926615e700 1 --2- 172.21.15.143:0/3961256947 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f926015a950 0x7f92600b0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:16.044 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.040+0000 7f926615e700 1 -- 172.21.15.143:0/3961256947 >> 172.21.15.143:0/3961256947 conn(0x7f92600cd2e0 msgr2=0x7f926015af70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:16.044 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.041+0000 7f926615e700 1 -- 172.21.15.143:0/3961256947 shutdown_connections 2023-11-08T19:26:16.044 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.041+0000 7f926615e700 1 -- 172.21.15.143:0/3961256947 wait complete. 2023-11-08T19:26:16.054 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snapshots.TestMonSnapsAndFsPools.test_attaching_pools_with_snaps_to_fs_fails' 2023-11-08T19:26:16.175 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.174+0000 7f9c5d4ce700 1 Processor -- start 2023-11-08T19:26:16.175 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.174+0000 7f9c5d4ce700 1 -- start start 2023-11-08T19:26:16.178 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.174+0000 7f9c5d4ce700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9c5815dc80 0x7f9c580d1f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:16.178 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.174+0000 7f9c5d4ce700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f9c580cf090 0x7f9c580b9690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:16.178 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.174+0000 7f9c5d4ce700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c580d3200 con 0x7f9c580d3690 2023-11-08T19:26:16.178 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.174+0000 7f9c5d4ce700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c580b9bd0 con 0x7f9c580cf090 2023-11-08T19:26:16.179 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.174+0000 7f9c5d4ce700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c580b7cf0 con 0x7f9c5815dc80 2023-11-08T19:26:16.179 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.175+0000 7f9c577fe700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f9c580cf090 0x7f9c580b9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:16.179 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.175+0000 7f9c577fe700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f9c580cf090 0x7f9c580b9690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51276/0 (socket says 172.21.15.143:51276) 2023-11-08T19:26:16.179 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.175+0000 7f9c577fe700 1 -- 172.21.15.143:0/1782453755 learned_addr learned my addr 172.21.15.143:0/1782453755 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:16.180 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.175+0000 7f9c56ffd700 1 --2- 172.21.15.143:0/1782453755 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9c5815dc80 0x7f9c580d1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:16.180 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.175+0000 7f9c577fe700 1 -- 172.21.15.143:0/1782453755 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9c5815dc80 msgr2=0x7f9c580d1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:16.180 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.175+0000 7f9c577fe700 1 --2- 172.21.15.143:0/1782453755 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9c5815dc80 0x7f9c580d1f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:16.180 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.175+0000 7f9c577fe700 1 -- 172.21.15.143:0/1782453755 >> v1:172.21.15.174:6789/0 conn(0x7f9c580d3690 legacy=0x7f9c580ce980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:16.180 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.175+0000 7f9c577fe700 1 -- 172.21.15.143:0/1782453755 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c5815e060 con 0x7f9c580cf090 2023-11-08T19:26:16.181 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.175+0000 7f9c56ffd700 1 --2- 172.21.15.143:0/1782453755 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9c5815dc80 0x7f9c580d1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:26:16.181 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.175+0000 7f9c577fe700 1 --2- 172.21.15.143:0/1782453755 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9c580cf090 0x7f9c580b9690 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f9c50009e50 tx=0x7f9c5000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7fd4f846eed276e server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:16.181 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.175+0000 7f9c55ffb700 1 -- 172.21.15.143:0/1782453755 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c5000e070 con 0x7f9c580cf090 2023-11-08T19:26:16.181 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.175+0000 7f9c55ffb700 1 -- 172.21.15.143:0/1782453755 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c50003a20 con 0x7f9c580cf090 2023-11-08T19:26:16.181 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.175+0000 7f9c55ffb700 1 -- 172.21.15.143:0/1782453755 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c50003dd0 con 0x7f9c580cf090 2023-11-08T19:26:16.182 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.176+0000 7f9c5d4ce700 1 -- 172.21.15.143:0/1782453755 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9c580cf090 msgr2=0x7f9c580b9690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:16.182 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.176+0000 7f9c5d4ce700 1 --2- 172.21.15.143:0/1782453755 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9c580cf090 0x7f9c580b9690 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f9c50009e50 tx=0x7f9c5000b330 comp rx=0 tx=0).stop 2023-11-08T19:26:16.182 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.176+0000 7f9c5d4ce700 1 -- 172.21.15.143:0/1782453755 shutdown_connections 2023-11-08T19:26:16.182 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.176+0000 7f9c5d4ce700 1 --2- 172.21.15.143:0/1782453755 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9c580cf090 0x7f9c580b9690 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:16.182 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.176+0000 7f9c5d4ce700 1 --2- 172.21.15.143:0/1782453755 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9c5815dc80 0x7f9c580d1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:16.183 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.176+0000 7f9c5d4ce700 1 -- 172.21.15.143:0/1782453755 >> 172.21.15.143:0/1782453755 conn(0x7f9c580668e0 msgr2=0x7f9c58066cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:16.183 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.176+0000 7f9c5d4ce700 1 -- 172.21.15.143:0/1782453755 shutdown_connections 2023-11-08T19:26:16.183 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.176+0000 7f9c5d4ce700 1 -- 172.21.15.143:0/1782453755 wait complete. 2023-11-08T19:26:16.185 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.177+0000 7f9c5d4ce700 1 Processor -- start 2023-11-08T19:26:16.186 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.177+0000 7f9c5d4ce700 1 -- start start 2023-11-08T19:26:16.186 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.178+0000 7f9c5d4ce700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9c580cf090 0x7f9c5814fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:16.186 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.178+0000 7f9c5d4ce700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9c580d3690 0x7f9c581605c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:16.186 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.178+0000 7f9c5d4ce700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9c5815dc80 0x7f9c58160b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:16.187 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.178+0000 7f9c5d4ce700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c581524d0 con 0x7f9c580cf090 2023-11-08T19:26:16.187 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.178+0000 7f9c5d4ce700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c58151430 con 0x7f9c580d3690 2023-11-08T19:26:16.187 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.178+0000 7f9c5d4ce700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c58151710 con 0x7f9c5815dc80 2023-11-08T19:26:16.187 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.178+0000 7f9c56ffd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9c580cf090 0x7f9c5814fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:16.187 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.178+0000 7f9c56ffd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9c580cf090 0x7f9c5814fe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51282/0 (socket says 172.21.15.143:51282) 2023-11-08T19:26:16.188 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.178+0000 7f9c56ffd700 1 -- 172.21.15.143:0/1144315178 learned_addr learned my addr 172.21.15.143:0/1144315178 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:16.188 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.178+0000 7f9c567fc700 1 --2- 172.21.15.143:0/1144315178 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9c580d3690 0x7f9c581605c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:16.188 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.178+0000 7f9c577fe700 1 --2- 172.21.15.143:0/1144315178 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9c5815dc80 0x7f9c58160b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:16.188 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.178+0000 7f9c56ffd700 1 -- 172.21.15.143:0/1144315178 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9c5815dc80 msgr2=0x7f9c58160b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:16.188 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.178+0000 7f9c56ffd700 1 --2- 172.21.15.143:0/1144315178 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9c5815dc80 0x7f9c58160b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:16.189 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.178+0000 7f9c56ffd700 1 -- 172.21.15.143:0/1144315178 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9c580d3690 msgr2=0x7f9c581605c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:16.189 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.178+0000 7f9c56ffd700 1 --2- 172.21.15.143:0/1144315178 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9c580d3690 0x7f9c581605c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:16.189 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.178+0000 7f9c56ffd700 1 -- 172.21.15.143:0/1144315178 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c50009b30 con 0x7f9c580cf090 2023-11-08T19:26:16.189 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.178+0000 7f9c56ffd700 1 --2- 172.21.15.143:0/1144315178 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9c580cf090 0x7f9c5814fe50 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f9c4000c7f0 tx=0x7f9c4000cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:16.189 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.178+0000 7f9c4f7fe700 1 -- 172.21.15.143:0/1144315178 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c40011800 con 0x7f9c580cf090 2023-11-08T19:26:16.190 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.179+0000 7f9c4f7fe700 1 -- 172.21.15.143:0/1144315178 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c40011960 con 0x7f9c580cf090 2023-11-08T19:26:16.190 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.179+0000 7f9c5d4ce700 1 -- 172.21.15.143:0/1144315178 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c581519f0 con 0x7f9c580cf090 2023-11-08T19:26:16.190 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.179+0000 7f9c4f7fe700 1 -- 172.21.15.143:0/1144315178 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c40011cf0 con 0x7f9c580cf090 2023-11-08T19:26:16.190 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.179+0000 7f9c5d4ce700 1 -- 172.21.15.143:0/1144315178 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c58151fb0 con 0x7f9c580cf090 2023-11-08T19:26:16.190 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.180+0000 7f9c5d4ce700 1 -- 172.21.15.143:0/1144315178 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c5804eee0 con 0x7f9c580cf090 2023-11-08T19:26:16.191 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.180+0000 7f9c4f7fe700 1 -- 172.21.15.143:0/1144315178 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9c40018360 con 0x7f9c580cf090 2023-11-08T19:26:16.191 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.180+0000 7f9c4f7fe700 1 --2- 172.21.15.143:0/1144315178 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f9c3806af00 0x7f9c3806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:16.191 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.181+0000 7f9c4f7fe700 1 -- 172.21.15.143:0/1144315178 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f9c40089290 con 0x7f9c580cf090 2023-11-08T19:26:16.191 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.181+0000 7f9c567fc700 1 --2- 172.21.15.143:0/1144315178 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f9c3806af00 0x7f9c3806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:16.191 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.181+0000 7f9c567fc700 1 --2- 172.21.15.143:0/1144315178 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f9c3806af00 0x7f9c3806d3c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f9c48007fd0 tx=0x7f9c48007ec0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:16.192 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.185+0000 7f9c4f7fe700 1 -- 172.21.15.143:0/1144315178 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9c400544e0 con 0x7f9c580cf090 2023-11-08T19:26:16.458 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.457+0000 7f9c5d4ce700 1 -- 172.21.15.143:0/1144315178 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snapshots.TestMonSnapsAndFsPools.test_attaching_pools_with_snaps_to_fs_fails"]} v 0) v1 -- 0x7f9c58152320 con 0x7f9c580cf090 2023-11-08T19:26:16.682 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.681+0000 7f9c4f7fe700 1 -- 172.21.15.143:0/1144315178 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snapshots.TestMonSnapsAndFsPools.test_attaching_pools_with_snaps_to_fs_fails"]}]=0 v99) v1 ==== 173+0+0 (secure 0 0 0) 0x7f9c40057b00 con 0x7f9c580cf090 2023-11-08T19:26:16.685 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.684+0000 7f9c5d4ce700 1 -- 172.21.15.143:0/1144315178 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f9c3806af00 msgr2=0x7f9c3806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:16.685 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.684+0000 7f9c5d4ce700 1 --2- 172.21.15.143:0/1144315178 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f9c3806af00 0x7f9c3806d3c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f9c48007fd0 tx=0x7f9c48007ec0 comp rx=0 tx=0).stop 2023-11-08T19:26:16.686 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.684+0000 7f9c5d4ce700 1 -- 172.21.15.143:0/1144315178 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9c580cf090 msgr2=0x7f9c5814fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:16.686 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.684+0000 7f9c5d4ce700 1 --2- 172.21.15.143:0/1144315178 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9c580cf090 0x7f9c5814fe50 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f9c4000c7f0 tx=0x7f9c4000cb00 comp rx=0 tx=0).stop 2023-11-08T19:26:16.686 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.684+0000 7f9c5d4ce700 1 -- 172.21.15.143:0/1144315178 shutdown_connections 2023-11-08T19:26:16.686 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.684+0000 7f9c5d4ce700 1 --2- 172.21.15.143:0/1144315178 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f9c3806af00 0x7f9c3806d3c0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:16.687 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.684+0000 7f9c5d4ce700 1 --2- 172.21.15.143:0/1144315178 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9c5815dc80 0x7f9c58160b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:16.687 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.684+0000 7f9c5d4ce700 1 --2- 172.21.15.143:0/1144315178 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9c580cf090 0x7f9c5814fe50 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:16.687 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.684+0000 7f9c5d4ce700 1 --2- 172.21.15.143:0/1144315178 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9c580d3690 0x7f9c581605c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:16.687 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.684+0000 7f9c5d4ce700 1 -- 172.21.15.143:0/1144315178 >> 172.21.15.143:0/1144315178 conn(0x7f9c580668e0 msgr2=0x7f9c580d3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:16.687 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.684+0000 7f9c5d4ce700 1 -- 172.21.15.143:0/1144315178 shutdown_connections 2023-11-08T19:26:16.688 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.684+0000 7f9c5d4ce700 1 -- 172.21.15.143:0/1144315178 wait complete. 2023-11-08T19:26:16.698 INFO:tasks.cephfs_test_runner:Test that attempt to attach pool with snapshots to an fs fails ... ok 2023-11-08T19:26:16.698 INFO:tasks.cephfs_test_runner:Starting test: test_disallow_monitor_managed_snaps_for_fs_pools (tasks.cephfs.test_snapshots.TestMonSnapsAndFsPools) Test that creation of monitor managed snaps fails for pools attached 2023-11-08T19:26:16.699 INFO:tasks.cephfs_test_runner:test_disallow_monitor_managed_snaps_for_fs_pools (tasks.cephfs.test_snapshots.TestMonSnapsAndFsPools) 2023-11-08T19:26:16.699 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:26:16.820 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.819+0000 7f7a684c2700 1 Processor -- start 2023-11-08T19:26:16.820 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.819+0000 7f7a684c2700 1 -- start start 2023-11-08T19:26:16.823 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.820+0000 7f7a684c2700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f7a6015dd00 0x7f7a600cca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:16.823 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.820+0000 7f7a684c2700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7a601618f0 0x7f7a6006ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:16.825 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.820+0000 7f7a684c2700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f7a6006b150 0x7f7a600b2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:16.826 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.820+0000 7f7a684c2700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a6006b600 con 0x7f7a6015dd00 2023-11-08T19:26:16.826 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.820+0000 7f7a684c2700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a600cad50 con 0x7f7a6006b150 2023-11-08T19:26:16.826 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.820+0000 7f7a684c2700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a600ccfb0 con 0x7f7a601618f0 2023-11-08T19:26:16.826 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.820+0000 7f7a6625e700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f7a6015dd00 0x7f7a600cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:16.826 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.820+0000 7f7a6625e700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f7a6015dd00 0x7f7a600cca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51286/0 (socket says 172.21.15.143:51286) 2023-11-08T19:26:16.827 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.820+0000 7f7a6625e700 1 -- 172.21.15.143:0/1431931585 learned_addr learned my addr 172.21.15.143:0/1431931585 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:16.827 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.820+0000 7f7a66a5f700 1 --2- 172.21.15.143:0/1431931585 >> v2:172.21.15.174:3300/0 conn(0x7f7a6006b150 0x7f7a600b2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:16.827 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.820+0000 7f7a65a5d700 1 --2- 172.21.15.143:0/1431931585 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7a601618f0 0x7f7a6006ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:16.827 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.820+0000 7f7a6625e700 1 -- 172.21.15.143:0/1431931585 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7a601618f0 msgr2=0x7f7a6006ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:16.827 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.820+0000 7f7a6625e700 1 --2- 172.21.15.143:0/1431931585 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7a601618f0 0x7f7a6006ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:16.828 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.820+0000 7f7a6625e700 1 -- 172.21.15.143:0/1431931585 >> v2:172.21.15.174:3300/0 conn(0x7f7a6006b150 msgr2=0x7f7a600b2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:16.828 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.820+0000 7f7a6625e700 1 --2- 172.21.15.143:0/1431931585 >> v2:172.21.15.174:3300/0 conn(0x7f7a6006b150 0x7f7a600b2210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:16.828 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.820+0000 7f7a6625e700 1 -- 172.21.15.143:0/1431931585 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a60161df0 con 0x7f7a6015dd00 2023-11-08T19:26:16.828 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.820+0000 7f7a6625e700 1 --2- 172.21.15.143:0/1431931585 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a6015dd00 0x7f7a600cca70 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f7a600708b0 tx=0x7f7a5c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=448ff823fd997355 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:16.829 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.821+0000 7f7a6525c700 1 -- 172.21.15.143:0/1431931585 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a5c005e30 con 0x7f7a6015dd00 2023-11-08T19:26:16.829 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.821+0000 7f7a6525c700 1 -- 172.21.15.143:0/1431931585 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a5c004030 con 0x7f7a6015dd00 2023-11-08T19:26:16.829 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.821+0000 7f7a6525c700 1 -- 172.21.15.143:0/1431931585 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a5c0043e0 con 0x7f7a6015dd00 2023-11-08T19:26:16.829 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.821+0000 7f7a684c2700 1 -- 172.21.15.143:0/1431931585 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a6015dd00 msgr2=0x7f7a600cca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:16.829 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.821+0000 7f7a684c2700 1 --2- 172.21.15.143:0/1431931585 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a6015dd00 0x7f7a600cca70 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f7a600708b0 tx=0x7f7a5c009bd0 comp rx=0 tx=0).stop 2023-11-08T19:26:16.830 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.821+0000 7f7a684c2700 1 -- 172.21.15.143:0/1431931585 shutdown_connections 2023-11-08T19:26:16.830 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.821+0000 7f7a684c2700 1 --2- 172.21.15.143:0/1431931585 >> v2:172.21.15.174:3300/0 conn(0x7f7a6006b150 0x7f7a600b2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:16.830 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.821+0000 7f7a684c2700 1 --2- 172.21.15.143:0/1431931585 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a6015dd00 0x7f7a600cca70 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:16.830 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.821+0000 7f7a684c2700 1 --2- 172.21.15.143:0/1431931585 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7a601618f0 0x7f7a6006ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:16.830 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.821+0000 7f7a684c2700 1 -- 172.21.15.143:0/1431931585 >> 172.21.15.143:0/1431931585 conn(0x7f7a60099ab0 msgr2=0x7f7a600d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:16.831 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.821+0000 7f7a684c2700 1 -- 172.21.15.143:0/1431931585 shutdown_connections 2023-11-08T19:26:16.831 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.821+0000 7f7a684c2700 1 -- 172.21.15.143:0/1431931585 wait complete. 2023-11-08T19:26:16.834 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.822+0000 7f7a684c2700 1 Processor -- start 2023-11-08T19:26:16.834 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.822+0000 7f7a684c2700 1 -- start start 2023-11-08T19:26:16.834 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.823+0000 7f7a684c2700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a6006b150 0x7f7a60154940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:16.835 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.823+0000 7f7a684c2700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7a6015dd00 0x7f7a600b1df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:16.835 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.823+0000 7f7a684c2700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7a601618f0 0x7f7a600b2330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:16.835 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.823+0000 7f7a684c2700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a600c4f50 con 0x7f7a6006b150 2023-11-08T19:26:16.835 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.823+0000 7f7a684c2700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a600c42d0 con 0x7f7a601618f0 2023-11-08T19:26:16.835 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.823+0000 7f7a684c2700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a600c45b0 con 0x7f7a6015dd00 2023-11-08T19:26:16.836 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.823+0000 7f7a6625e700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a6006b150 0x7f7a60154940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:16.836 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.823+0000 7f7a6625e700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a6006b150 0x7f7a60154940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51292/0 (socket says 172.21.15.143:51292) 2023-11-08T19:26:16.836 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.823+0000 7f7a6625e700 1 -- 172.21.15.143:0/1862175420 learned_addr learned my addr 172.21.15.143:0/1862175420 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:16.836 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.823+0000 7f7a65a5d700 1 --2- 172.21.15.143:0/1862175420 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7a6015dd00 0x7f7a600b1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:16.836 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.823+0000 7f7a66a5f700 1 --2- 172.21.15.143:0/1862175420 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7a601618f0 0x7f7a600b2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:16.837 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.823+0000 7f7a6625e700 1 -- 172.21.15.143:0/1862175420 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7a6015dd00 msgr2=0x7f7a600b1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:16.837 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.823+0000 7f7a6625e700 1 --2- 172.21.15.143:0/1862175420 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7a6015dd00 0x7f7a600b1df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:16.837 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.823+0000 7f7a6625e700 1 -- 172.21.15.143:0/1862175420 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7a601618f0 msgr2=0x7f7a600b2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:16.837 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.823+0000 7f7a6625e700 1 --2- 172.21.15.143:0/1862175420 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7a601618f0 0x7f7a600b2330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:16.837 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.823+0000 7f7a6625e700 1 -- 172.21.15.143:0/1862175420 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a5c009500 con 0x7f7a6006b150 2023-11-08T19:26:16.838 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.823+0000 7f7a66a5f700 1 --2- 172.21.15.143:0/1862175420 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7a601618f0 0x7f7a600b2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:26:16.838 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.824+0000 7f7a6625e700 1 --2- 172.21.15.143:0/1862175420 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a6006b150 0x7f7a60154940 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c009130 tx=0x7f7a5c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:16.838 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.824+0000 7f7a56ffd700 1 -- 172.21.15.143:0/1862175420 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a5c003cf0 con 0x7f7a6006b150 2023-11-08T19:26:16.838 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.824+0000 7f7a56ffd700 1 -- 172.21.15.143:0/1862175420 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a5c003e50 con 0x7f7a6006b150 2023-11-08T19:26:16.838 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.824+0000 7f7a684c2700 1 -- 172.21.15.143:0/1862175420 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a600c4890 con 0x7f7a6006b150 2023-11-08T19:26:16.839 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.824+0000 7f7a56ffd700 1 -- 172.21.15.143:0/1862175420 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a5c01d670 con 0x7f7a6006b150 2023-11-08T19:26:16.839 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.824+0000 7f7a684c2700 1 -- 172.21.15.143:0/1862175420 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a60160e70 con 0x7f7a6006b150 2023-11-08T19:26:16.839 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.825+0000 7f7a684c2700 1 -- 172.21.15.143:0/1862175420 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a600aff60 con 0x7f7a6006b150 2023-11-08T19:26:16.839 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.825+0000 7f7a56ffd700 1 -- 172.21.15.143:0/1862175420 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7a5c01d7d0 con 0x7f7a6006b150 2023-11-08T19:26:16.839 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.826+0000 7f7a56ffd700 1 --2- 172.21.15.143:0/1862175420 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f7a4406af00 0x7f7a4406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:16.839 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.826+0000 7f7a56ffd700 1 -- 172.21.15.143:0/1862175420 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f7a5c08a640 con 0x7f7a6006b150 2023-11-08T19:26:16.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.826+0000 7f7a65a5d700 1 --2- 172.21.15.143:0/1862175420 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f7a4406af00 0x7f7a4406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:16.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.826+0000 7f7a65a5d700 1 --2- 172.21.15.143:0/1862175420 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f7a4406af00 0x7f7a4406d3c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f7a50009be0 tx=0x7f7a50009210 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:16.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:16.830+0000 7f7a56ffd700 1 -- 172.21.15.143:0/1862175420 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7a5c055920 con 0x7f7a6006b150 2023-11-08T19:26:17.021 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.020+0000 7f7a684c2700 1 -- 172.21.15.143:0/1862175420 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7a600c5090 con 0x7f7a6006b150 2023-11-08T19:26:17.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.021+0000 7f7a56ffd700 1 -- 172.21.15.143:0/1862175420 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+14789 (secure 0 0 0) 0x7f7a5c058f40 con 0x7f7a6006b150 2023-11-08T19:26:17.022 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:26:17.023 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":40,"fsid":"968483af-77fc-4068-ad7c-5c2a1a4613d7","created":"2023-11-08T19:24:32.826484+0000","modified":"2023-11-08T19:26:15.675175+0000","last_up_change":"2023-11-08T19:24:43.573880+0000","last_in_change":"2023-11-08T19:24:35.314525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:44.551530+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"snap-test-pool","create_time":"2023-11-08T19:26:02.874384+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":36,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bae98ec1-d94e-403d-a220-a522a53ea6f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6809","nonce":1487830566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6811","nonce":1487830566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6815","nonce":1487830566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6813","nonce":1487830566}]},"public_addr":"172.21.15.143:6809/1487830566","cluster_addr":"172.21.15.143:6811/1487830566","heartbeat_back_addr":"172.21.15.143:6815/1487830566","heartbeat_front_addr":"172.21.15.143:6813/1487830566","state":["exists","up"]},{"osd":1,"uuid":"5cb6538a-1ccb-4322-8633-09f66af0b7e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6801","nonce":787341397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6803","nonce":787341397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6807","nonce":787341397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6805","nonce":787341397}]},"public_addr":"172.21.15.143:6801/787341397","cluster_addr":"172.21.15.143:6803/787341397","heartbeat_back_addr":"172.21.15.143:6807/787341397","heartbeat_front_addr":"172.21.15.143:6805/787341397","state":["exists","up"]},{"osd":2,"uuid":"cdbf522c-60ef-4d60-9653-8dcb529c101a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6822","nonce":2301325597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6826","nonce":2301325597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6831","nonce":2301325597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6829","nonce":2301325597}]},"public_addr":"172.21.15.143:6822/2301325597","cluster_addr":"172.21.15.143:6826/2301325597","heartbeat_back_addr":"172.21.15.143:6831/2301325597","heartbeat_front_addr":"172.21.15.143:6829/2301325597","state":["exists","up"]},{"osd":3,"uuid":"4e2783cf-ebb9-4dd8-8245-bc1d17ce4325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6817","nonce":2021504428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6819","nonce":2021504428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6827","nonce":2021504428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6823","nonce":2021504428}]},"public_addr":"172.21.15.143:6817/2021504428","cluster_addr":"172.21.15.143:6819/2021504428","heartbeat_back_addr":"172.21.15.143:6827/2021504428","heartbeat_front_addr":"172.21.15.143:6823/2021504428","state":["exists","up"]},{"osd":4,"uuid":"691256fa-9b77-49b9-a346-7ae1adf0ca80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6801","nonce":1993380595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6803","nonce":1993380595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6807","nonce":1993380595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6805","nonce":1993380595}]},"public_addr":"172.21.15.174:6801/1993380595","cluster_addr":"172.21.15.174:6803/1993380595","heartbeat_back_addr":"172.21.15.174:6807/1993380595","heartbeat_front_addr":"172.21.15.174:6805/1993380595","state":["exists","up"]},{"osd":5,"uuid":"756fc352-48c9-46ce-b52e-0541fe6f8525","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6809","nonce":1777923303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6811","nonce":1777923303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6815","nonce":1777923303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6813","nonce":1777923303}]},"public_addr":"172.21.15.174:6809/1777923303","cluster_addr":"172.21.15.174:6811/1777923303","heartbeat_back_addr":"172.21.15.174:6815/1777923303","heartbeat_front_addr":"172.21.15.174:6813/1777923303","state":["exists","up"]},{"osd":6,"uuid":"fab2ac2b-ed97-4cde-9825-b480eabf8b8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6825","nonce":2755130316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6827","nonce":2755130316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6831","nonce":2755130316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6829","nonce":2755130316}]},"public_addr":"172.21.15.174:6825/2755130316","cluster_addr":"172.21.15.174:6827/2755130316","heartbeat_back_addr":"172.21.15.174:6831/2755130316","heartbeat_front_addr":"172.21.15.174:6829/2755130316","state":["exists","up"]},{"osd":7,"uuid":"3de9b797-2ab6-4d41-a863-7e7d638fd703","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6817","nonce":2847141411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6819","nonce":2847141411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6823","nonce":2847141411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6821","nonce":2847141411}]},"public_addr":"172.21.15.174:6817/2847141411","cluster_addr":"172.21.15.174:6819/2847141411","heartbeat_back_addr":"172.21.15.174:6823/2847141411","heartbeat_front_addr":"172.21.15.174:6821/2847141411","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.477076+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.496890+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.539305+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.454673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.538881+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.521294+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.520712+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.540404+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.143:6835/1574070101":"2023-11-09T19:26:12.668326+0000","172.21.15.143:6833/1574070101":"2023-11-09T19:26:12.668326+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:26:17.024 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.023+0000 7f7a684c2700 1 -- 172.21.15.143:0/1862175420 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f7a4406af00 msgr2=0x7f7a4406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:17.025 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.023+0000 7f7a684c2700 1 --2- 172.21.15.143:0/1862175420 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f7a4406af00 0x7f7a4406d3c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f7a50009be0 tx=0x7f7a50009210 comp rx=0 tx=0).stop 2023-11-08T19:26:17.025 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.023+0000 7f7a684c2700 1 -- 172.21.15.143:0/1862175420 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a6006b150 msgr2=0x7f7a60154940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:17.025 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.023+0000 7f7a684c2700 1 --2- 172.21.15.143:0/1862175420 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a6006b150 0x7f7a60154940 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c009130 tx=0x7f7a5c004750 comp rx=0 tx=0).stop 2023-11-08T19:26:17.025 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.023+0000 7f7a684c2700 1 -- 172.21.15.143:0/1862175420 shutdown_connections 2023-11-08T19:26:17.026 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.023+0000 7f7a684c2700 1 --2- 172.21.15.143:0/1862175420 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f7a4406af00 0x7f7a4406d3c0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:17.026 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.023+0000 7f7a684c2700 1 --2- 172.21.15.143:0/1862175420 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7a601618f0 0x7f7a600b2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:17.026 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.023+0000 7f7a684c2700 1 --2- 172.21.15.143:0/1862175420 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a6006b150 0x7f7a60154940 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:17.030 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.023+0000 7f7a684c2700 1 --2- 172.21.15.143:0/1862175420 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7a6015dd00 0x7f7a600b1df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:17.030 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.023+0000 7f7a684c2700 1 -- 172.21.15.143:0/1862175420 >> 172.21.15.143:0/1862175420 conn(0x7f7a60099ab0 msgr2=0x7f7a6006eaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:17.030 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.023+0000 7f7a684c2700 1 -- 172.21.15.143:0/1862175420 shutdown_connections 2023-11-08T19:26:17.030 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.023+0000 7f7a684c2700 1 -- 172.21.15.143:0/1862175420 wait complete. 2023-11-08T19:26:17.037 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T19:24:44.551530+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'snap-test-pool', 'create_time': '2023-11-08T19:26:02.874384+0000', 'flags': 16385, 'flags_names': 'hashpspool,pool_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '36', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'pool', 'snap_seq': 2, 'snap_epoch': 36, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-08T19:26:17.038 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T19:26:17.161 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.159+0000 7faaea8ef700 1 Processor -- start 2023-11-08T19:26:17.161 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.159+0000 7faaea8ef700 1 -- start start 2023-11-08T19:26:17.164 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.160+0000 7faaea8ef700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7faae415dc20 0x7faae40c4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:17.164 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.160+0000 7faaea8ef700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7faae40cc9e0 0x7faae40cce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:17.164 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.160+0000 7faaea8ef700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faae406ad90 con 0x7faae415e000 2023-11-08T19:26:17.164 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.160+0000 7faaea8ef700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faae406bb50 con 0x7faae415dc20 2023-11-08T19:26:17.164 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.160+0000 7faaea8ef700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faae40cd390 con 0x7faae40cc9e0 2023-11-08T19:26:17.165 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.160+0000 7faae3fff700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7faae415dc20 0x7faae40c4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:17.165 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.161+0000 7faae3fff700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7faae415dc20 0x7faae40c4ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51300/0 (socket says 172.21.15.143:51300) 2023-11-08T19:26:17.165 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.161+0000 7faae3fff700 1 -- 172.21.15.143:0/3509881465 learned_addr learned my addr 172.21.15.143:0/3509881465 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:17.165 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.161+0000 7faae3fff700 1 -- 172.21.15.143:0/3509881465 >> v2:172.21.15.174:3300/0 conn(0x7faae40cc9e0 msgr2=0x7faae40cce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:17.165 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.161+0000 7faae3fff700 1 --2- 172.21.15.143:0/3509881465 >> v2:172.21.15.174:3300/0 conn(0x7faae40cc9e0 0x7faae40cce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:17.166 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.161+0000 7faae3fff700 1 -- 172.21.15.143:0/3509881465 >> v1:172.21.15.174:6789/0 conn(0x7faae415e000 legacy=0x7faae40c5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:17.166 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.161+0000 7faae3fff700 1 -- 172.21.15.143:0/3509881465 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faae406b070 con 0x7faae415dc20 2023-11-08T19:26:17.166 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.161+0000 7faae3fff700 1 --2- 172.21.15.143:0/3509881465 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faae415dc20 0x7faae40c4ee0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7faae40cdd30 tx=0x7faad8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=169a30acef7af1d8 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:17.166 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.161+0000 7faae2ffd700 1 -- 172.21.15.143:0/3509881465 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faad8004230 con 0x7faae415dc20 2023-11-08T19:26:17.167 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.161+0000 7faae2ffd700 1 -- 172.21.15.143:0/3509881465 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faad8004390 con 0x7faae415dc20 2023-11-08T19:26:17.167 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.161+0000 7faae2ffd700 1 -- 172.21.15.143:0/3509881465 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faad80036a0 con 0x7faae415dc20 2023-11-08T19:26:17.167 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.161+0000 7faaea8ef700 1 -- 172.21.15.143:0/3509881465 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faae415dc20 msgr2=0x7faae40c4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:17.167 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.161+0000 7faaea8ef700 1 --2- 172.21.15.143:0/3509881465 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faae415dc20 0x7faae40c4ee0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7faae40cdd30 tx=0x7faad8009bd0 comp rx=0 tx=0).stop 2023-11-08T19:26:17.167 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.162+0000 7faaea8ef700 1 -- 172.21.15.143:0/3509881465 shutdown_connections 2023-11-08T19:26:17.168 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.162+0000 7faaea8ef700 1 --2- 172.21.15.143:0/3509881465 >> v2:172.21.15.174:3300/0 conn(0x7faae40cc9e0 0x7faae40cce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:17.168 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.162+0000 7faaea8ef700 1 --2- 172.21.15.143:0/3509881465 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faae415dc20 0x7faae40c4ee0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:17.168 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.162+0000 7faaea8ef700 1 -- 172.21.15.143:0/3509881465 >> 172.21.15.143:0/3509881465 conn(0x7faae40b1c70 msgr2=0x7faae40b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:17.168 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.162+0000 7faaea8ef700 1 -- 172.21.15.143:0/3509881465 shutdown_connections 2023-11-08T19:26:17.168 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.162+0000 7faaea8ef700 1 -- 172.21.15.143:0/3509881465 wait complete. 2023-11-08T19:26:17.169 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.163+0000 7faaea8ef700 1 Processor -- start 2023-11-08T19:26:17.171 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.163+0000 7faaea8ef700 1 -- start start 2023-11-08T19:26:17.171 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.164+0000 7faaea8ef700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faae40cc9e0 0x7faae4153520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:17.172 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.164+0000 7faaea8ef700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faae415dc20 0x7faae41e3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:17.172 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.164+0000 7faaea8ef700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7faae415e000 0x7faae415c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:17.172 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.164+0000 7faaea8ef700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faae41ea230 con 0x7faae415dc20 2023-11-08T19:26:17.172 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.164+0000 7faaea8ef700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faae40c9af0 con 0x7faae40cc9e0 2023-11-08T19:26:17.172 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.164+0000 7faaea8ef700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faae4153a60 con 0x7faae415e000 2023-11-08T19:26:17.173 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.164+0000 7faae37fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faae415dc20 0x7faae41e3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:17.173 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.164+0000 7faae37fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faae415dc20 0x7faae41e3830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51308/0 (socket says 172.21.15.143:51308) 2023-11-08T19:26:17.173 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.164+0000 7faae37fe700 1 -- 172.21.15.143:0/4066518860 learned_addr learned my addr 172.21.15.143:0/4066518860 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:17.173 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.164+0000 7faae8e8c700 1 --2- 172.21.15.143:0/4066518860 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7faae415e000 0x7faae415c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:17.173 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.164+0000 7faae3fff700 1 --2- 172.21.15.143:0/4066518860 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faae40cc9e0 0x7faae4153520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:17.174 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.164+0000 7faae37fe700 1 -- 172.21.15.143:0/4066518860 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7faae415e000 msgr2=0x7faae415c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:17.174 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.164+0000 7faae37fe700 1 --2- 172.21.15.143:0/4066518860 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7faae415e000 0x7faae415c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:17.174 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.164+0000 7faae37fe700 1 -- 172.21.15.143:0/4066518860 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faae40cc9e0 msgr2=0x7faae4153520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:17.174 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.164+0000 7faae37fe700 1 --2- 172.21.15.143:0/4066518860 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faae40cc9e0 0x7faae4153520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:17.174 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.164+0000 7faae37fe700 1 -- 172.21.15.143:0/4066518860 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faad8009550 con 0x7faae415dc20 2023-11-08T19:26:17.175 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.164+0000 7faae8e8c700 1 --2- 172.21.15.143:0/4066518860 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7faae415e000 0x7faae415c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:26:17.175 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.164+0000 7faae3fff700 1 --2- 172.21.15.143:0/4066518860 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faae40cc9e0 0x7faae4153520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:26:17.175 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.164+0000 7faae37fe700 1 --2- 172.21.15.143:0/4066518860 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faae415dc20 0x7faae41e3830 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7faad400c4d0 tx=0x7faad400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:17.175 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.164+0000 7faae0ff9700 1 -- 172.21.15.143:0/4066518860 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faad40054e0 con 0x7faae415dc20 2023-11-08T19:26:17.175 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.164+0000 7faae0ff9700 1 -- 172.21.15.143:0/4066518860 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faad4003680 con 0x7faae415dc20 2023-11-08T19:26:17.176 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.164+0000 7faaea8ef700 1 -- 172.21.15.143:0/4066518860 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faae40c8ee0 con 0x7faae415dc20 2023-11-08T19:26:17.176 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.164+0000 7faae0ff9700 1 -- 172.21.15.143:0/4066518860 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faad4003a10 con 0x7faae415dc20 2023-11-08T19:26:17.176 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.164+0000 7faaea8ef700 1 -- 172.21.15.143:0/4066518860 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faae40c94a0 con 0x7faae415dc20 2023-11-08T19:26:17.176 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.165+0000 7faaea8ef700 1 -- 172.21.15.143:0/4066518860 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faae404eee0 con 0x7faae415dc20 2023-11-08T19:26:17.176 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.166+0000 7faae0ff9700 1 -- 172.21.15.143:0/4066518860 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7faad4003b70 con 0x7faae415dc20 2023-11-08T19:26:17.177 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.166+0000 7faae0ff9700 1 --2- 172.21.15.143:0/4066518860 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7faad006af00 0x7faad006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:17.177 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.166+0000 7faae0ff9700 1 -- 172.21.15.143:0/4066518860 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5296+0+0 (secure 0 0 0) 0x7faad4088820 con 0x7faae415dc20 2023-11-08T19:26:17.177 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.166+0000 7faae3fff700 1 --2- 172.21.15.143:0/4066518860 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7faad006af00 0x7faad006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:17.177 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.167+0000 7faae3fff700 1 --2- 172.21.15.143:0/4066518860 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7faad006af00 0x7faad006d3c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7faad8009c10 tx=0x7faad8008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:17.178 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.170+0000 7faae0ff9700 1 -- 172.21.15.143:0/4066518860 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7faad4053b20 con 0x7faae415dc20 2023-11-08T19:26:17.359 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.358+0000 7faaea8ef700 1 -- 172.21.15.143:0/4066518860 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7faae40669b0 con 0x7faae415dc20 2023-11-08T19:26:17.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.358+0000 7faae0ff9700 1 -- 172.21.15.143:0/4066518860 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v40) v1 ==== 93+0+10 (secure 0 0 0) 0x7faad40170a0 con 0x7faae415dc20 2023-11-08T19:26:17.360 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2023-11-08T19:26:17.363 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.361+0000 7faaea8ef700 1 -- 172.21.15.143:0/4066518860 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7faad006af00 msgr2=0x7faad006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:17.363 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.361+0000 7faaea8ef700 1 --2- 172.21.15.143:0/4066518860 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7faad006af00 0x7faad006d3c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7faad8009c10 tx=0x7faad8008030 comp rx=0 tx=0).stop 2023-11-08T19:26:17.363 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.361+0000 7faaea8ef700 1 -- 172.21.15.143:0/4066518860 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faae415dc20 msgr2=0x7faae41e3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:17.363 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.361+0000 7faaea8ef700 1 --2- 172.21.15.143:0/4066518860 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faae415dc20 0x7faae41e3830 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7faad400c4d0 tx=0x7faad400c890 comp rx=0 tx=0).stop 2023-11-08T19:26:17.363 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.361+0000 7faaea8ef700 1 -- 172.21.15.143:0/4066518860 shutdown_connections 2023-11-08T19:26:17.364 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.361+0000 7faaea8ef700 1 --2- 172.21.15.143:0/4066518860 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7faad006af00 0x7faad006d3c0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:17.364 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.361+0000 7faaea8ef700 1 --2- 172.21.15.143:0/4066518860 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7faae415e000 0x7faae415c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:17.364 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.361+0000 7faaea8ef700 1 --2- 172.21.15.143:0/4066518860 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faae40cc9e0 0x7faae4153520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:17.364 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.361+0000 7faaea8ef700 1 --2- 172.21.15.143:0/4066518860 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faae415dc20 0x7faae41e3830 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:17.364 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.361+0000 7faaea8ef700 1 -- 172.21.15.143:0/4066518860 >> 172.21.15.143:0/4066518860 conn(0x7faae40b1c70 msgr2=0x7faae406bf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:17.365 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.361+0000 7faaea8ef700 1 -- 172.21.15.143:0/4066518860 shutdown_connections 2023-11-08T19:26:17.365 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.361+0000 7faaea8ef700 1 -- 172.21.15.143:0/4066518860 wait complete. 2023-11-08T19:26:17.374 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get snap-test-pool pg_num 2023-11-08T19:26:17.498 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.496+0000 7fb8aa616700 1 Processor -- start 2023-11-08T19:26:17.498 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.496+0000 7fb8aa616700 1 -- start start 2023-11-08T19:26:17.501 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.497+0000 7fb8aa616700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fb8a415aac0 0x7fb8a415f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:17.501 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.497+0000 7fb8aa616700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb8a40cd840 0x7fb8a406ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:17.501 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.497+0000 7fb8aa616700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8a406dbd0 con 0x7fb8a40cb470 2023-11-08T19:26:17.502 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.497+0000 7fb8aa616700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8a415f5b0 con 0x7fb8a40cd840 2023-11-08T19:26:17.502 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.497+0000 7fb8aa616700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8a40cc860 con 0x7fb8a415aac0 2023-11-08T19:26:17.502 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.497+0000 7fb8a8bb3700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb8a40cd840 0x7fb8a406ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:17.502 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.497+0000 7fb8a8bb3700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb8a40cd840 0x7fb8a406ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51318/0 (socket says 172.21.15.143:51318) 2023-11-08T19:26:17.503 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.497+0000 7fb8a8bb3700 1 -- 172.21.15.143:0/522840376 learned_addr learned my addr 172.21.15.143:0/522840376 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:17.503 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.497+0000 7fb8a3fff700 1 --2- 172.21.15.143:0/522840376 >> v2:172.21.15.174:3300/0 conn(0x7fb8a415aac0 0x7fb8a415f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:17.503 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.498+0000 7fb8a8bb3700 1 -- 172.21.15.143:0/522840376 >> v2:172.21.15.174:3300/0 conn(0x7fb8a415aac0 msgr2=0x7fb8a415f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:17.503 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.498+0000 7fb8a8bb3700 1 --2- 172.21.15.143:0/522840376 >> v2:172.21.15.174:3300/0 conn(0x7fb8a415aac0 0x7fb8a415f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:17.503 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.498+0000 7fb8a8bb3700 1 -- 172.21.15.143:0/522840376 >> v1:172.21.15.174:6789/0 conn(0x7fb8a40cb470 legacy=0x7fb8a40cd130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:17.504 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.498+0000 7fb8a8bb3700 1 -- 172.21.15.143:0/522840376 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8a415beb0 con 0x7fb8a40cd840 2023-11-08T19:26:17.504 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.498+0000 7fb8a8bb3700 1 --2- 172.21.15.143:0/522840376 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb8a40cd840 0x7fb8a406ed70 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fb8a4070a00 tx=0x7fb89c00d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e4b9d23f3463fba server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:17.504 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.498+0000 7fb8a2ffd700 1 -- 172.21.15.143:0/522840376 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb89c010070 con 0x7fb8a40cd840 2023-11-08T19:26:17.504 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.498+0000 7fb8a2ffd700 1 -- 172.21.15.143:0/522840376 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb89c00dd90 con 0x7fb8a40cd840 2023-11-08T19:26:17.505 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.498+0000 7fb8a2ffd700 1 -- 172.21.15.143:0/522840376 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb89c00b310 con 0x7fb8a40cd840 2023-11-08T19:26:17.505 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.498+0000 7fb8aa616700 1 -- 172.21.15.143:0/522840376 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb8a40cd840 msgr2=0x7fb8a406ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:17.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.498+0000 7fb8aa616700 1 --2- 172.21.15.143:0/522840376 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb8a40cd840 0x7fb8a406ed70 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fb8a4070a00 tx=0x7fb89c00d430 comp rx=0 tx=0).stop 2023-11-08T19:26:17.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.498+0000 7fb8aa616700 1 -- 172.21.15.143:0/522840376 shutdown_connections 2023-11-08T19:26:17.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.498+0000 7fb8aa616700 1 --2- 172.21.15.143:0/522840376 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb8a40cd840 0x7fb8a406ed70 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:17.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.498+0000 7fb8aa616700 1 --2- 172.21.15.143:0/522840376 >> v2:172.21.15.174:3300/0 conn(0x7fb8a415aac0 0x7fb8a415f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:17.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.498+0000 7fb8aa616700 1 -- 172.21.15.143:0/522840376 >> 172.21.15.143:0/522840376 conn(0x7fb8a4099b20 msgr2=0x7fb8a40d07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:17.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.499+0000 7fb8aa616700 1 -- 172.21.15.143:0/522840376 shutdown_connections 2023-11-08T19:26:17.508 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.499+0000 7fb8aa616700 1 -- 172.21.15.143:0/522840376 wait complete. 2023-11-08T19:26:17.508 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.500+0000 7fb8aa616700 1 Processor -- start 2023-11-08T19:26:17.511 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.500+0000 7fb8aa616700 1 -- start start 2023-11-08T19:26:17.511 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.501+0000 7fb8aa616700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb8a40cb470 0x7fb8a414cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:17.511 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.501+0000 7fb8aa616700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb8a40cd840 0x7fb8a415cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:17.511 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.501+0000 7fb8aa616700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb8a415aac0 0x7fb8a415aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:17.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.501+0000 7fb8aa616700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8a41554f0 con 0x7fb8a40cd840 2023-11-08T19:26:17.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.501+0000 7fb8aa616700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8a41557b0 con 0x7fb8a40cb470 2023-11-08T19:26:17.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.501+0000 7fb8aa616700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8a4155a90 con 0x7fb8a415aac0 2023-11-08T19:26:17.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.501+0000 7fb8a37fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb8a40cd840 0x7fb8a415cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:17.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.501+0000 7fb8a37fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb8a40cd840 0x7fb8a415cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51322/0 (socket says 172.21.15.143:51322) 2023-11-08T19:26:17.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.501+0000 7fb8a37fe700 1 -- 172.21.15.143:0/2642307661 learned_addr learned my addr 172.21.15.143:0/2642307661 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:17.513 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.501+0000 7fb8a3fff700 1 --2- 172.21.15.143:0/2642307661 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb8a40cb470 0x7fb8a414cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:17.513 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.501+0000 7fb8a8bb3700 1 --2- 172.21.15.143:0/2642307661 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb8a415aac0 0x7fb8a415aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:17.513 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.502+0000 7fb8a37fe700 1 -- 172.21.15.143:0/2642307661 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb8a415aac0 msgr2=0x7fb8a415aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:17.513 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.502+0000 7fb8a37fe700 1 --2- 172.21.15.143:0/2642307661 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb8a415aac0 0x7fb8a415aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:17.513 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.502+0000 7fb8a37fe700 1 -- 172.21.15.143:0/2642307661 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb8a40cb470 msgr2=0x7fb8a414cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:17.514 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.502+0000 7fb8a37fe700 1 --2- 172.21.15.143:0/2642307661 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb8a40cb470 0x7fb8a414cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:17.514 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.502+0000 7fb8a37fe700 1 -- 172.21.15.143:0/2642307661 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb89c00bd40 con 0x7fb8a40cd840 2023-11-08T19:26:17.514 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.502+0000 7fb8a8bb3700 1 --2- 172.21.15.143:0/2642307661 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb8a415aac0 0x7fb8a415aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:26:17.514 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.502+0000 7fb8a3fff700 1 --2- 172.21.15.143:0/2642307661 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb8a40cb470 0x7fb8a414cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:26:17.514 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.502+0000 7fb8a37fe700 1 --2- 172.21.15.143:0/2642307661 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb8a40cd840 0x7fb8a415cab0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fb89400c4e0 tx=0x7fb89400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:17.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.502+0000 7fb8a0ff9700 1 -- 172.21.15.143:0/2642307661 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8940052d0 con 0x7fb8a40cd840 2023-11-08T19:26:17.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.502+0000 7fb8a0ff9700 1 -- 172.21.15.143:0/2642307661 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb894005430 con 0x7fb8a40cd840 2023-11-08T19:26:17.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.502+0000 7fb8aa616700 1 -- 172.21.15.143:0/2642307661 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8a4155d70 con 0x7fb8a40cd840 2023-11-08T19:26:17.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.502+0000 7fb8a0ff9700 1 -- 172.21.15.143:0/2642307661 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8940038c0 con 0x7fb8a40cd840 2023-11-08T19:26:17.516 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.502+0000 7fb8aa616700 1 -- 172.21.15.143:0/2642307661 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8a414dce0 con 0x7fb8a40cd840 2023-11-08T19:26:17.516 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.503+0000 7fb8aa616700 1 -- 172.21.15.143:0/2642307661 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8a4156230 con 0x7fb8a40cd840 2023-11-08T19:26:17.517 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.503+0000 7fb8a0ff9700 1 -- 172.21.15.143:0/2642307661 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb894003a20 con 0x7fb8a40cd840 2023-11-08T19:26:17.517 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.504+0000 7fb8a0ff9700 1 --2- 172.21.15.143:0/2642307661 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fb89006af00 0x7fb89006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:17.517 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.504+0000 7fb8a0ff9700 1 -- 172.21.15.143:0/2642307661 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fb894088880 con 0x7fb8a40cd840 2023-11-08T19:26:17.517 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.504+0000 7fb8a3fff700 1 --2- 172.21.15.143:0/2642307661 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fb89006af00 0x7fb89006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:17.518 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.504+0000 7fb8a3fff700 1 --2- 172.21.15.143:0/2642307661 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fb89006af00 0x7fb89006d3c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb898006fd0 tx=0x7fb898008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:17.518 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.508+0000 7fb8a0ff9700 1 -- 172.21.15.143:0/2642307661 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb894053b60 con 0x7fb8a40cd840 2023-11-08T19:26:17.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.699+0000 7fb8aa616700 1 -- 172.21.15.143:0/2642307661 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"} v 0) v1 -- 0x7fb8a404eee0 con 0x7fb8a40cd840 2023-11-08T19:26:17.701 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.699+0000 7fb8a0ff9700 1 -- 172.21.15.143:0/2642307661 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"}]=0 v40) v1 ==== 103+0+11 (secure 0 0 0) 0x7fb8940170a0 con 0x7fb8a40cd840 2023-11-08T19:26:17.701 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2023-11-08T19:26:17.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.701+0000 7fb8aa616700 1 -- 172.21.15.143:0/2642307661 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fb89006af00 msgr2=0x7fb89006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:17.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.701+0000 7fb8aa616700 1 --2- 172.21.15.143:0/2642307661 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fb89006af00 0x7fb89006d3c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb898006fd0 tx=0x7fb898008040 comp rx=0 tx=0).stop 2023-11-08T19:26:17.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.701+0000 7fb8aa616700 1 -- 172.21.15.143:0/2642307661 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb8a40cd840 msgr2=0x7fb8a415cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:17.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.701+0000 7fb8aa616700 1 --2- 172.21.15.143:0/2642307661 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb8a40cd840 0x7fb8a415cab0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fb89400c4e0 tx=0x7fb89400c8a0 comp rx=0 tx=0).stop 2023-11-08T19:26:17.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.702+0000 7fb8aa616700 1 -- 172.21.15.143:0/2642307661 shutdown_connections 2023-11-08T19:26:17.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.702+0000 7fb8aa616700 1 --2- 172.21.15.143:0/2642307661 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fb89006af00 0x7fb89006d3c0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:17.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.702+0000 7fb8aa616700 1 --2- 172.21.15.143:0/2642307661 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb8a415aac0 0x7fb8a415aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:17.705 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.702+0000 7fb8aa616700 1 --2- 172.21.15.143:0/2642307661 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb8a40cb470 0x7fb8a414cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:17.705 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.702+0000 7fb8aa616700 1 --2- 172.21.15.143:0/2642307661 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb8a40cd840 0x7fb8a415cab0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:17.705 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.702+0000 7fb8aa616700 1 -- 172.21.15.143:0/2642307661 >> 172.21.15.143:0/2642307661 conn(0x7fb8a4099b20 msgr2=0x7fb8a4159de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:17.705 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.702+0000 7fb8aa616700 1 -- 172.21.15.143:0/2642307661 shutdown_connections 2023-11-08T19:26:17.705 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.702+0000 7fb8aa616700 1 -- 172.21.15.143:0/2642307661 wait complete. 2023-11-08T19:26:17.715 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snapshots.TestMonSnapsAndFsPools.test_disallow_monitor_managed_snaps_for_fs_pools' 2023-11-08T19:26:17.835 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.834+0000 7fd6876c8700 1 Processor -- start 2023-11-08T19:26:17.836 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.834+0000 7fd6876c8700 1 -- start start 2023-11-08T19:26:17.838 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.835+0000 7fd6876c8700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd68015dd00 0x7fd6800cca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:17.839 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.835+0000 7fd6876c8700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd6801618f0 0x7fd68006ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:17.839 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.835+0000 7fd6876c8700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fd68006b150 0x7fd6800b2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:17.839 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.835+0000 7fd6876c8700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd68006b600 con 0x7fd68015dd00 2023-11-08T19:26:17.839 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.835+0000 7fd6876c8700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6800cad50 con 0x7fd68006b150 2023-11-08T19:26:17.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.835+0000 7fd6876c8700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6800ccfb0 con 0x7fd6801618f0 2023-11-08T19:26:17.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.835+0000 7fd685464700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd68015dd00 0x7fd6800cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:17.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.835+0000 7fd685464700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd68015dd00 0x7fd6800cca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51328/0 (socket says 172.21.15.143:51328) 2023-11-08T19:26:17.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.835+0000 7fd685464700 1 -- 172.21.15.143:0/3208200571 learned_addr learned my addr 172.21.15.143:0/3208200571 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:17.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.835+0000 7fd684c63700 1 --2- 172.21.15.143:0/3208200571 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd6801618f0 0x7fd68006ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:17.841 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.835+0000 7fd685c65700 1 --2- 172.21.15.143:0/3208200571 >> v2:172.21.15.174:3300/0 conn(0x7fd68006b150 0x7fd6800b2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:17.841 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.835+0000 7fd685464700 1 -- 172.21.15.143:0/3208200571 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd6801618f0 msgr2=0x7fd68006ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:17.841 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.835+0000 7fd685464700 1 --2- 172.21.15.143:0/3208200571 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd6801618f0 0x7fd68006ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:17.841 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.835+0000 7fd685464700 1 -- 172.21.15.143:0/3208200571 >> v2:172.21.15.174:3300/0 conn(0x7fd68006b150 msgr2=0x7fd6800b2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:17.841 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.835+0000 7fd685464700 1 --2- 172.21.15.143:0/3208200571 >> v2:172.21.15.174:3300/0 conn(0x7fd68006b150 0x7fd6800b2210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:17.842 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.835+0000 7fd685464700 1 -- 172.21.15.143:0/3208200571 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd680161df0 con 0x7fd68015dd00 2023-11-08T19:26:17.842 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.835+0000 7fd684c63700 1 --2- 172.21.15.143:0/3208200571 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd6801618f0 0x7fd68006ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:26:17.842 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.836+0000 7fd685464700 1 --2- 172.21.15.143:0/3208200571 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd68015dd00 0x7fd6800cca70 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fd6800708b0 tx=0x7fd670009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dec29454f629d48e server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:17.842 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.836+0000 7fd677fff700 1 -- 172.21.15.143:0/3208200571 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd670005e30 con 0x7fd68015dd00 2023-11-08T19:26:17.842 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.836+0000 7fd677fff700 1 -- 172.21.15.143:0/3208200571 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd670004030 con 0x7fd68015dd00 2023-11-08T19:26:17.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.836+0000 7fd677fff700 1 -- 172.21.15.143:0/3208200571 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6700043e0 con 0x7fd68015dd00 2023-11-08T19:26:17.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.836+0000 7fd6876c8700 1 -- 172.21.15.143:0/3208200571 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd68015dd00 msgr2=0x7fd6800cca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:17.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.836+0000 7fd6876c8700 1 --2- 172.21.15.143:0/3208200571 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd68015dd00 0x7fd6800cca70 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fd6800708b0 tx=0x7fd670009bd0 comp rx=0 tx=0).stop 2023-11-08T19:26:17.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.836+0000 7fd6876c8700 1 -- 172.21.15.143:0/3208200571 shutdown_connections 2023-11-08T19:26:17.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.836+0000 7fd6876c8700 1 --2- 172.21.15.143:0/3208200571 >> v2:172.21.15.174:3300/0 conn(0x7fd68006b150 0x7fd6800b2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:17.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.836+0000 7fd6876c8700 1 --2- 172.21.15.143:0/3208200571 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd68015dd00 0x7fd6800cca70 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:17.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.836+0000 7fd6876c8700 1 --2- 172.21.15.143:0/3208200571 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd6801618f0 0x7fd68006ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:17.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.836+0000 7fd6876c8700 1 -- 172.21.15.143:0/3208200571 >> 172.21.15.143:0/3208200571 conn(0x7fd680099ab0 msgr2=0x7fd6800d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:17.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.836+0000 7fd6876c8700 1 -- 172.21.15.143:0/3208200571 shutdown_connections 2023-11-08T19:26:17.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.836+0000 7fd6876c8700 1 -- 172.21.15.143:0/3208200571 wait complete. 2023-11-08T19:26:17.851 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.838+0000 7fd6876c8700 1 Processor -- start 2023-11-08T19:26:17.852 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.838+0000 7fd6876c8700 1 -- start start 2023-11-08T19:26:17.852 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.839+0000 7fd6876c8700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd68006b150 0x7fd6801f4a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:17.852 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.839+0000 7fd6876c8700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd68015dd00 0x7fd6800cb410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:17.852 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.839+0000 7fd6876c8700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd6801618f0 0x7fd6800c53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:17.852 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.839+0000 7fd6876c8700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6800c6630 con 0x7fd68015dd00 2023-11-08T19:26:17.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.839+0000 7fd6876c8700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6800c8860 con 0x7fd68006b150 2023-11-08T19:26:17.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.839+0000 7fd6876c8700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6800c8ae0 con 0x7fd6801618f0 2023-11-08T19:26:17.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.839+0000 7fd684c63700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd68015dd00 0x7fd6800cb410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:17.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.839+0000 7fd684c63700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd68015dd00 0x7fd6800cb410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51336/0 (socket says 172.21.15.143:51336) 2023-11-08T19:26:17.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.839+0000 7fd684c63700 1 -- 172.21.15.143:0/1777058434 learned_addr learned my addr 172.21.15.143:0/1777058434 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:17.854 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.839+0000 7fd685c65700 1 --2- 172.21.15.143:0/1777058434 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd6801618f0 0x7fd6800c53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:17.854 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.839+0000 7fd685464700 1 --2- 172.21.15.143:0/1777058434 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd68006b150 0x7fd6801f4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:17.854 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.839+0000 7fd684c63700 1 -- 172.21.15.143:0/1777058434 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd6801618f0 msgr2=0x7fd6800c53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:17.854 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.839+0000 7fd684c63700 1 --2- 172.21.15.143:0/1777058434 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd6801618f0 0x7fd6800c53b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:17.854 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.839+0000 7fd684c63700 1 -- 172.21.15.143:0/1777058434 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd68006b150 msgr2=0x7fd6801f4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:17.855 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.839+0000 7fd684c63700 1 --2- 172.21.15.143:0/1777058434 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd68006b150 0x7fd6801f4a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:17.855 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.839+0000 7fd684c63700 1 -- 172.21.15.143:0/1777058434 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd670009500 con 0x7fd68015dd00 2023-11-08T19:26:17.855 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.839+0000 7fd685c65700 1 --2- 172.21.15.143:0/1777058434 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd6801618f0 0x7fd6800c53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:26:17.855 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.839+0000 7fd685464700 1 --2- 172.21.15.143:0/1777058434 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd68006b150 0x7fd6801f4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:26:17.856 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.840+0000 7fd684c63700 1 --2- 172.21.15.143:0/1777058434 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd68015dd00 0x7fd6800cb410 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fd67c00e810 tx=0x7fd67c00ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:17.856 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.840+0000 7fd675ffb700 1 -- 172.21.15.143:0/1777058434 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd67c004230 con 0x7fd68015dd00 2023-11-08T19:26:17.856 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.840+0000 7fd675ffb700 1 -- 172.21.15.143:0/1777058434 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd67c004390 con 0x7fd68015dd00 2023-11-08T19:26:17.856 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.840+0000 7fd675ffb700 1 -- 172.21.15.143:0/1777058434 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd67c0036a0 con 0x7fd68015dd00 2023-11-08T19:26:17.856 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.840+0000 7fd6876c8700 1 -- 172.21.15.143:0/1777058434 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6800c5d90 con 0x7fd68015dd00 2023-11-08T19:26:17.857 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.840+0000 7fd6876c8700 1 -- 172.21.15.143:0/1777058434 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6800c6350 con 0x7fd68015dd00 2023-11-08T19:26:17.857 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.841+0000 7fd6876c8700 1 -- 172.21.15.143:0/1777058434 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6800aff60 con 0x7fd68015dd00 2023-11-08T19:26:17.857 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.841+0000 7fd675ffb700 1 -- 172.21.15.143:0/1777058434 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd67c003800 con 0x7fd68015dd00 2023-11-08T19:26:17.857 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.842+0000 7fd675ffb700 1 --2- 172.21.15.143:0/1777058434 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd66406af00 0x7fd66406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:17.857 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.842+0000 7fd675ffb700 1 -- 172.21.15.143:0/1777058434 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fd67c088ea0 con 0x7fd68015dd00 2023-11-08T19:26:17.858 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.842+0000 7fd685464700 1 --2- 172.21.15.143:0/1777058434 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd66406af00 0x7fd66406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:17.858 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.842+0000 7fd685464700 1 --2- 172.21.15.143:0/1777058434 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd66406af00 0x7fd66406d3c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd6800708b0 tx=0x7fd670008070 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:17.858 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:17.846+0000 7fd675ffb700 1 -- 172.21.15.143:0/1777058434 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd67c054180 con 0x7fd68015dd00 2023-11-08T19:26:18.115 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:18.113+0000 7fd6876c8700 1 -- 172.21.15.143:0/1777058434 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snapshots.TestMonSnapsAndFsPools.test_disallow_monitor_managed_snaps_for_fs_pools"]} v 0) v1 -- 0x7fd6800c8ca0 con 0x7fd68015dd00 2023-11-08T19:26:18.691 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:18.690+0000 7fd675ffb700 1 -- 172.21.15.143:0/1777058434 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snapshots.TestMonSnapsAndFsPools.test_disallow_monitor_managed_snaps_for_fs_pools"]}]=0 v101) v1 ==== 181+0+0 (secure 0 0 0) 0x7fd67c0577a0 con 0x7fd68015dd00 2023-11-08T19:26:18.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:18.692+0000 7fd6876c8700 1 -- 172.21.15.143:0/1777058434 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd66406af00 msgr2=0x7fd66406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:18.694 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:18.692+0000 7fd6876c8700 1 --2- 172.21.15.143:0/1777058434 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd66406af00 0x7fd66406d3c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd6800708b0 tx=0x7fd670008070 comp rx=0 tx=0).stop 2023-11-08T19:26:18.694 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:18.692+0000 7fd6876c8700 1 -- 172.21.15.143:0/1777058434 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd68015dd00 msgr2=0x7fd6800cb410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:18.694 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:18.692+0000 7fd6876c8700 1 --2- 172.21.15.143:0/1777058434 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd68015dd00 0x7fd6800cb410 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fd67c00e810 tx=0x7fd67c00ebd0 comp rx=0 tx=0).stop 2023-11-08T19:26:18.694 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:18.692+0000 7fd6876c8700 1 -- 172.21.15.143:0/1777058434 shutdown_connections 2023-11-08T19:26:18.695 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:18.692+0000 7fd6876c8700 1 --2- 172.21.15.143:0/1777058434 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd66406af00 0x7fd66406d3c0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:18.695 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:18.692+0000 7fd6876c8700 1 --2- 172.21.15.143:0/1777058434 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd6801618f0 0x7fd6800c53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:18.695 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:18.692+0000 7fd6876c8700 1 --2- 172.21.15.143:0/1777058434 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd68006b150 0x7fd6801f4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:18.695 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:18.692+0000 7fd6876c8700 1 --2- 172.21.15.143:0/1777058434 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd68015dd00 0x7fd6800cb410 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:18.695 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:18.692+0000 7fd6876c8700 1 -- 172.21.15.143:0/1777058434 >> 172.21.15.143:0/1777058434 conn(0x7fd680099ab0 msgr2=0x7fd68015c330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:18.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:18.692+0000 7fd6876c8700 1 -- 172.21.15.143:0/1777058434 shutdown_connections 2023-11-08T19:26:18.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:18.692+0000 7fd6876c8700 1 -- 172.21.15.143:0/1777058434 wait complete. 2023-11-08T19:26:18.707 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-08T19:26:18.833 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:18.831+0000 7f4bda17f700 1 Processor -- start 2023-11-08T19:26:18.833 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:18.832+0000 7f4bda17f700 1 -- start start 2023-11-08T19:26:18.836 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:18.832+0000 7f4bda17f700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f4bd415dba0 0x7f4bd406c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:18.836 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:18.832+0000 7f4bda17f700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4bd406b240 0x7f4bd40caff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:18.836 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:18.832+0000 7f4bda17f700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bd4161890 con 0x7f4bd40cb530 2023-11-08T19:26:18.836 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:18.832+0000 7f4bda17f700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bd406b740 con 0x7f4bd415dba0 2023-11-08T19:26:18.837 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:18.832+0000 7f4bda17f700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd406ba20 con 0x7f4bd406b240 2023-11-08T19:26:18.837 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:18.832+0000 7f4bd37fe700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f4bd415dba0 0x7f4bd406c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:18.837 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:18.832+0000 7f4bd37fe700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f4bd415dba0 0x7f4bd406c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51342/0 (socket says 172.21.15.143:51342) 2023-11-08T19:26:18.837 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:18.832+0000 7f4bd37fe700 1 -- 172.21.15.143:0/1312745135 learned_addr learned my addr 172.21.15.143:0/1312745135 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:18.837 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:18.833+0000 7f4bd2ffd700 1 --2- 172.21.15.143:0/1312745135 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4bd406b240 0x7f4bd40caff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:18.838 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:18.833+0000 7f4bd37fe700 1 -- 172.21.15.143:0/1312745135 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4bd406b240 msgr2=0x7f4bd40caff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:18.838 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:18.833+0000 7f4bd37fe700 1 --2- 172.21.15.143:0/1312745135 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4bd406b240 0x7f4bd40caff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:18.838 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:18.833+0000 7f4bd37fe700 1 -- 172.21.15.143:0/1312745135 >> v1:172.21.15.174:6789/0 conn(0x7f4bd40cb530 legacy=0x7f4bd40b22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:18.838 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:18.833+0000 7f4bd37fe700 1 -- 172.21.15.143:0/1312745135 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bd406bd00 con 0x7f4bd415dba0 2023-11-08T19:26:18.838 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:18.833+0000 7f4bd37fe700 1 --2- 172.21.15.143:0/1312745135 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4bd415dba0 0x7f4bd406c610 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f4bc80098f0 tx=0x7f4bc8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd071631046af05f server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:18.839 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:18.833+0000 7f4bd27fc700 1 -- 172.21.15.143:0/1312745135 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bc800b8a0 con 0x7f4bd415dba0 2023-11-08T19:26:18.839 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:18.833+0000 7f4bd27fc700 1 -- 172.21.15.143:0/1312745135 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bc800ba00 con 0x7f4bd415dba0 2023-11-08T19:26:18.839 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:18.833+0000 7f4bd27fc700 1 -- 172.21.15.143:0/1312745135 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bc800bd90 con 0x7f4bd415dba0 2023-11-08T19:26:18.839 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:18.834+0000 7f4bda17f700 1 -- 172.21.15.143:0/1312745135 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4bd415dba0 msgr2=0x7f4bd406c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:18.839 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:18.834+0000 7f4bda17f700 1 --2- 172.21.15.143:0/1312745135 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4bd415dba0 0x7f4bd406c610 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f4bc80098f0 tx=0x7f4bc8009c00 comp rx=0 tx=0).stop 2023-11-08T19:26:18.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:18.834+0000 7f4bda17f700 1 -- 172.21.15.143:0/1312745135 shutdown_connections 2023-11-08T19:26:18.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:18.834+0000 7f4bda17f700 1 --2- 172.21.15.143:0/1312745135 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4bd415dba0 0x7f4bd406c610 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:18.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:18.834+0000 7f4bda17f700 1 --2- 172.21.15.143:0/1312745135 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4bd406b240 0x7f4bd40caff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:18.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:18.834+0000 7f4bda17f700 1 -- 172.21.15.143:0/1312745135 >> 172.21.15.143:0/1312745135 conn(0x7f4bd40cd2e0 msgr2=0x7f4bd40cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:18.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:18.834+0000 7f4bda17f700 1 -- 172.21.15.143:0/1312745135 shutdown_connections 2023-11-08T19:26:18.841 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:18.834+0000 7f4bda17f700 1 -- 172.21.15.143:0/1312745135 wait complete. 2023-11-08T19:26:18.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:18.835+0000 7f4bda17f700 1 Processor -- start 2023-11-08T19:26:18.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:18.835+0000 7f4bda17f700 1 -- start start 2023-11-08T19:26:18.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:18.836+0000 7f4bda17f700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4bd406b240 0x7f4bd4151c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:18.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:18.836+0000 7f4bda17f700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4bd40cb530 0x7f4bd40c7930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:18.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:18.836+0000 7f4bda17f700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4bd415dba0 0x7f4bd40c8b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:18.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:18.836+0000 7f4bda17f700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd40c81a0 con 0x7f4bd415dba0 2023-11-08T19:26:18.845 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:18.836+0000 7f4bda17f700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd40c8480 con 0x7f4bd406b240 2023-11-08T19:26:18.845 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:18.836+0000 7f4bda17f700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd40c8760 con 0x7f4bd40cb530 2023-11-08T19:26:18.845 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:18.836+0000 7f4bd3fff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4bd415dba0 0x7f4bd40c8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:18.845 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:18.836+0000 7f4bd3fff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4bd415dba0 0x7f4bd40c8b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51352/0 (socket says 172.21.15.143:51352) 2023-11-08T19:26:18.845 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:18.836+0000 7f4bd3fff700 1 -- 172.21.15.143:0/18697229 learned_addr learned my addr 172.21.15.143:0/18697229 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:18.846 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:18.836+0000 7f4bd37fe700 1 --2- 172.21.15.143:0/18697229 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4bd406b240 0x7f4bd4151c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:18.846 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:18.836+0000 7f4bd2ffd700 1 --2- 172.21.15.143:0/18697229 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4bd40cb530 0x7f4bd40c7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:18.846 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:18.836+0000 7f4bd3fff700 1 -- 172.21.15.143:0/18697229 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4bd40cb530 msgr2=0x7f4bd40c7930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:18.846 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:18.836+0000 7f4bd3fff700 1 --2- 172.21.15.143:0/18697229 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4bd40cb530 0x7f4bd40c7930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:18.846 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:18.836+0000 7f4bd3fff700 1 -- 172.21.15.143:0/18697229 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4bd406b240 msgr2=0x7f4bd4151c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:18.847 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:18.836+0000 7f4bd3fff700 1 --2- 172.21.15.143:0/18697229 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4bd406b240 0x7f4bd4151c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:18.847 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:18.836+0000 7f4bd3fff700 1 -- 172.21.15.143:0/18697229 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bc8009550 con 0x7f4bd415dba0 2023-11-08T19:26:18.847 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:18.836+0000 7f4bd37fe700 1 --2- 172.21.15.143:0/18697229 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4bd406b240 0x7f4bd4151c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:26:18.847 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:18.836+0000 7f4bd3fff700 1 --2- 172.21.15.143:0/18697229 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4bd415dba0 0x7f4bd40c8b10 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc00cbc0 tx=0x7f4bcc00cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:18.847 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:18.837+0000 7f4bbbfff700 1 -- 172.21.15.143:0/18697229 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bcc003c00 con 0x7f4bd415dba0 2023-11-08T19:26:18.848 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:18.837+0000 7f4bbbfff700 1 -- 172.21.15.143:0/18697229 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bcc003d60 con 0x7f4bd415dba0 2023-11-08T19:26:18.848 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:18.837+0000 7f4bbbfff700 1 -- 172.21.15.143:0/18697229 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bcc00abf0 con 0x7f4bd415dba0 2023-11-08T19:26:18.848 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:18.837+0000 7f4bda17f700 1 -- 172.21.15.143:0/18697229 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bd40acc90 con 0x7f4bd415dba0 2023-11-08T19:26:18.848 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:18.837+0000 7f4bda17f700 1 -- 172.21.15.143:0/18697229 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bd40c7e70 con 0x7f4bd415dba0 2023-11-08T19:26:18.848 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:18.837+0000 7f4bda17f700 1 -- 172.21.15.143:0/18697229 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bd40ad1b0 con 0x7f4bd415dba0 2023-11-08T19:26:18.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:18.838+0000 7f4bbbfff700 1 -- 172.21.15.143:0/18697229 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4bcc00ad50 con 0x7f4bd415dba0 2023-11-08T19:26:18.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:18.838+0000 7f4bbbfff700 1 --2- 172.21.15.143:0/18697229 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f4bc006af00 0x7f4bc006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:18.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:18.838+0000 7f4bbbfff700 1 -- 172.21.15.143:0/18697229 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f4bcc0894c0 con 0x7f4bd415dba0 2023-11-08T19:26:18.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:18.839+0000 7f4bd37fe700 1 --2- 172.21.15.143:0/18697229 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f4bc006af00 0x7f4bc006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:18.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:18.839+0000 7f4bd37fe700 1 --2- 172.21.15.143:0/18697229 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f4bc006af00 0x7f4bc006d3c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f4bc8014040 tx=0x7f4bc8008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:18.850 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:18.843+0000 7f4bbbfff700 1 -- 172.21.15.143:0/18697229 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4bcc0547a0 con 0x7f4bd415dba0 2023-11-08T19:26:19.036 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.035+0000 7f4bda17f700 1 -- 172.21.15.143:0/18697229 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f4bd404eee0 con 0x7f4bd415dba0 2023-11-08T19:26:19.041 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.040+0000 7f4bbbfff700 1 -- 172.21.15.143:0/18697229 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v13)=0 v13) v1 ==== 122+0+0 (secure 0 0 0) 0x7f4bcc057dc0 con 0x7f4bd415dba0 2023-11-08T19:26:19.044 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.042+0000 7f4bda17f700 1 -- 172.21.15.143:0/18697229 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f4bc006af00 msgr2=0x7f4bc006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:19.044 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.042+0000 7f4bda17f700 1 --2- 172.21.15.143:0/18697229 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f4bc006af00 0x7f4bc006d3c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f4bc8014040 tx=0x7f4bc8008030 comp rx=0 tx=0).stop 2023-11-08T19:26:19.044 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.042+0000 7f4bda17f700 1 -- 172.21.15.143:0/18697229 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4bd415dba0 msgr2=0x7f4bd40c8b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:19.045 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.042+0000 7f4bda17f700 1 --2- 172.21.15.143:0/18697229 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4bd415dba0 0x7f4bd40c8b10 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc00cbc0 tx=0x7f4bcc00cf80 comp rx=0 tx=0).stop 2023-11-08T19:26:19.045 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.042+0000 7f4bda17f700 1 -- 172.21.15.143:0/18697229 shutdown_connections 2023-11-08T19:26:19.045 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.042+0000 7f4bda17f700 1 --2- 172.21.15.143:0/18697229 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f4bc006af00 0x7f4bc006d3c0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:19.045 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.042+0000 7f4bda17f700 1 --2- 172.21.15.143:0/18697229 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4bd415dba0 0x7f4bd40c8b10 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:19.045 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.042+0000 7f4bda17f700 1 --2- 172.21.15.143:0/18697229 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4bd406b240 0x7f4bd4151c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:19.046 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.042+0000 7f4bda17f700 1 --2- 172.21.15.143:0/18697229 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4bd40cb530 0x7f4bd40c7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:19.046 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.042+0000 7f4bda17f700 1 -- 172.21.15.143:0/18697229 >> 172.21.15.143:0/18697229 conn(0x7f4bd40cd2e0 msgr2=0x7f4bd415ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:19.046 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.043+0000 7f4bda17f700 1 -- 172.21.15.143:0/18697229 shutdown_connections 2023-11-08T19:26:19.046 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.043+0000 7f4bda17f700 1 -- 172.21.15.143:0/18697229 wait complete. 2023-11-08T19:26:19.056 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-08T19:26:19.057 DEBUG:teuthology.orchestra.run.smithi143:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-08T19:26:19.100 DEBUG:teuthology.orchestra.run.smithi174:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-08T19:26:19.150 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-08T19:26:19.151 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-08T19:26:19.151 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:26:19.168 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284264k,nr_inodes=4071066,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=44,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20643 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-08T19:26:19.169 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2023-11-08T19:26:19.169 DEBUG:teuthology.orchestra.run.smithi174:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:26:19.189 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284224k,nr_inodes=4071056,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=31,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19549 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260544k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-08T19:26:19.190 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-08T19:26:19.190 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:26:19.227 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284264k,nr_inodes=4071066,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=44,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20643 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-08T19:26:19.228 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2023-11-08T19:26:19.229 DEBUG:teuthology.orchestra.run.smithi174:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:26:19.250 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284224k,nr_inodes=4071056,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=31,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19549 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260544k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-08T19:26:19.250 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-08T19:26:19.252 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:26:19.389 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.387+0000 7f119a775700 1 Processor -- start 2023-11-08T19:26:19.389 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.387+0000 7f119a775700 1 -- start start 2023-11-08T19:26:19.392 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.388+0000 7f119a775700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1194159870 0x7f11940b2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:19.392 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.388+0000 7f119a775700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11940b2a60 con 0x7f119406ed60 2023-11-08T19:26:19.392 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.388+0000 7f119a775700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11940b2d40 con 0x7f11940cc410 2023-11-08T19:26:19.393 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.388+0000 7f119a775700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f119406b090 con 0x7f1194159870 2023-11-08T19:26:19.393 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.388+0000 7f11937fe700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f119406ed60 0x7f119406b4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:47510/0 (socket says 172.21.15.143:47510) 2023-11-08T19:26:19.393 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.388+0000 7f11937fe700 1 -- 172.21.15.143:0/3816814801 learned_addr learned my addr 172.21.15.143:0/3816814801 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:26:19.393 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.388+0000 7f1193fff700 1 --2- 172.21.15.143:0/3816814801 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1194159870 0x7f11940b2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:19.394 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.388+0000 7f1192ffd700 1 -- 172.21.15.143:0/3816814801 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3291516485 0 0) 0x7f11940b2a60 con 0x7f119406ed60 2023-11-08T19:26:19.394 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.388+0000 7f1192ffd700 1 -- 172.21.15.143:0/3816814801 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f117c0035b0 con 0x7f119406ed60 2023-11-08T19:26:19.394 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.389+0000 7f1192ffd700 1 -- 172.21.15.143:0/3816814801 <== mon.1 v1:172.21.15.174:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4064814899 0 0) 0x7f11940b2d40 con 0x7f11940cc410 2023-11-08T19:26:19.395 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.389+0000 7f1192ffd700 1 -- 172.21.15.143:0/3816814801 --> v1:172.21.15.174:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f11940b2a60 con 0x7f11940cc410 2023-11-08T19:26:19.395 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.389+0000 7f1193fff700 1 -- 172.21.15.143:0/3816814801 >> v1:172.21.15.174:6789/0 conn(0x7f11940cc410 legacy=0x7f11940cc820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:19.395 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.389+0000 7f1193fff700 1 -- 172.21.15.143:0/3816814801 >> v1:172.21.15.143:6789/0 conn(0x7f119406ed60 legacy=0x7f119406b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:19.395 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.389+0000 7f1192ffd700 1 -- 172.21.15.143:0/3816814801 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 377548996 0 0) 0x7f117c0035b0 con 0x7f119406ed60 2023-11-08T19:26:19.396 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.389+0000 7f1193fff700 1 -- 172.21.15.143:0/3816814801 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11940b3020 con 0x7f1194159870 2023-11-08T19:26:19.396 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.389+0000 7f1193fff700 1 --2- 172.21.15.143:0/3816814801 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1194159870 0x7f11940b2210 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1188009850 tx=0x7f1188009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2bba07fad616e79 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:19.396 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.389+0000 7f1192ffd700 1 -- 172.21.15.143:0/3816814801 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1184001670 con 0x7f1194159870 2023-11-08T19:26:19.396 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.389+0000 7f1192ffd700 1 -- 172.21.15.143:0/3816814801 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1188005c30 con 0x7f1194159870 2023-11-08T19:26:19.396 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.389+0000 7f1192ffd700 1 -- 172.21.15.143:0/3816814801 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1188004030 con 0x7f1194159870 2023-11-08T19:26:19.397 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.390+0000 7f119a775700 1 -- 172.21.15.143:0/3816814801 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1194159870 msgr2=0x7f11940b2210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:19.397 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.390+0000 7f119a775700 1 --2- 172.21.15.143:0/3816814801 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1194159870 0x7f11940b2210 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1188009850 tx=0x7f1188009b60 comp rx=0 tx=0).stop 2023-11-08T19:26:19.397 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.390+0000 7f119a775700 1 -- 172.21.15.143:0/3816814801 shutdown_connections 2023-11-08T19:26:19.397 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.390+0000 7f119a775700 1 --2- 172.21.15.143:0/3816814801 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1194159870 0x7f11940b2210 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:19.397 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.390+0000 7f119a775700 1 -- 172.21.15.143:0/3816814801 >> 172.21.15.143:0/3816814801 conn(0x7f1194099ab0 msgr2=0x7f11940d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:19.398 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.390+0000 7f119a775700 1 -- 172.21.15.143:0/3816814801 shutdown_connections 2023-11-08T19:26:19.398 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.390+0000 7f119a775700 1 -- 172.21.15.143:0/3816814801 wait complete. 2023-11-08T19:26:19.398 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.391+0000 7f119a775700 1 Processor -- start 2023-11-08T19:26:19.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.391+0000 7f119a775700 1 -- start start 2023-11-08T19:26:19.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.392+0000 7f119a775700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f119406ed60 0x7f11940c4c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:19.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.392+0000 7f119a775700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f11940cc410 0x7f11940c5af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:19.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.392+0000 7f119a775700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1194159870 0x7f11941f05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:19.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.392+0000 7f119a775700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11940b3a60 con 0x7f1194159870 2023-11-08T19:26:19.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.392+0000 7f119a775700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11940b5330 con 0x7f11940cc410 2023-11-08T19:26:19.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.392+0000 7f119a775700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11940b3450 con 0x7f119406ed60 2023-11-08T19:26:19.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.392+0000 7f1198d12700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1194159870 0x7f11941f05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:19.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.392+0000 7f1198d12700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1194159870 0x7f11941f05c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51362/0 (socket says 172.21.15.143:51362) 2023-11-08T19:26:19.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.392+0000 7f1198d12700 1 -- 172.21.15.143:0/3008866756 learned_addr learned my addr 172.21.15.143:0/3008866756 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:19.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.392+0000 7f11937fe700 1 --2- 172.21.15.143:0/3008866756 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f11940cc410 0x7f11940c5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:19.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.392+0000 7f1193fff700 1 --2- 172.21.15.143:0/3008866756 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f119406ed60 0x7f11940c4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:19.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.392+0000 7f1198d12700 1 -- 172.21.15.143:0/3008866756 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f119406ed60 msgr2=0x7f11940c4c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:19.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.392+0000 7f1198d12700 1 --2- 172.21.15.143:0/3008866756 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f119406ed60 0x7f11940c4c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:19.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.392+0000 7f1198d12700 1 -- 172.21.15.143:0/3008866756 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f11940cc410 msgr2=0x7f11940c5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:19.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.392+0000 7f1198d12700 1 --2- 172.21.15.143:0/3008866756 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f11940cc410 0x7f11940c5af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:19.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.392+0000 7f1198d12700 1 -- 172.21.15.143:0/3008866756 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1188009430 con 0x7f1194159870 2023-11-08T19:26:19.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.392+0000 7f11937fe700 1 --2- 172.21.15.143:0/3008866756 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f11940cc410 0x7f11940c5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:26:19.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.392+0000 7f1198d12700 1 --2- 172.21.15.143:0/3008866756 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1194159870 0x7f11941f05c0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f118c00cc20 tx=0x7f118c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:19.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.392+0000 7f1190ff9700 1 -- 172.21.15.143:0/3008866756 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f118c00ac20 con 0x7f1194159870 2023-11-08T19:26:19.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.392+0000 7f1190ff9700 1 -- 172.21.15.143:0/3008866756 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f118c00ad80 con 0x7f1194159870 2023-11-08T19:26:19.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.392+0000 7f119a775700 1 -- 172.21.15.143:0/3008866756 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11940c5320 con 0x7f1194159870 2023-11-08T19:26:19.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.392+0000 7f1190ff9700 1 -- 172.21.15.143:0/3008866756 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f118c003c20 con 0x7f1194159870 2023-11-08T19:26:19.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.392+0000 7f119a775700 1 -- 172.21.15.143:0/3008866756 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11940c5600 con 0x7f1194159870 2023-11-08T19:26:19.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.393+0000 7f119a775700 1 -- 172.21.15.143:0/3008866756 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11940b3c10 con 0x7f1194159870 2023-11-08T19:26:19.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.394+0000 7f1190ff9700 1 -- 172.21.15.143:0/3008866756 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f118c0050d0 con 0x7f1194159870 2023-11-08T19:26:19.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.394+0000 7f1190ff9700 1 --2- 172.21.15.143:0/3008866756 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f118006af00 0x7f118006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:19.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.394+0000 7f1190ff9700 1 -- 172.21.15.143:0/3008866756 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f118c0895f0 con 0x7f1194159870 2023-11-08T19:26:19.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.394+0000 7f1193fff700 1 --2- 172.21.15.143:0/3008866756 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f118006af00 0x7f118006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:19.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.395+0000 7f1193fff700 1 --2- 172.21.15.143:0/3008866756 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f118006af00 0x7f118006d3c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f11880097a0 tx=0x7f1188008070 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:19.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.399+0000 7f1190ff9700 1 -- 172.21.15.143:0/3008866756 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f11940b3c10 con 0x7f1194159870 2023-11-08T19:26:19.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.631+0000 7f119a775700 1 -- 172.21.15.143:0/3008866756 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f119404eee0 con 0x7f1194159870 2023-11-08T19:26:19.633 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.632+0000 7f1190ff9700 1 -- 172.21.15.143:0/3008866756 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+3708 (secure 0 0 0) 0x7f118c054840 con 0x7f1194159870 2023-11-08T19:26:19.634 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:26:19.634 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":18,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4918,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6836/3265911434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3265911434},{"type":"v1","addr":"172.21.15.174:6836","nonce":3265911434}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4921,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/2092876318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6835","nonce":2092876318},{"type":"v1","addr":"172.21.15.174:6837","nonce":2092876318}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5106,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6834/4077367240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":4077367240},{"type":"v1","addr":"172.21.15.143:6834","nonce":4077367240}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5490,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/286710645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":286710645},{"type":"v1","addr":"172.21.15.143:6835","nonce":286710645}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18}],"filesystems":[]} 2023-11-08T19:26:19.636 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.634+0000 7f119a775700 1 -- 172.21.15.143:0/3008866756 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f118006af00 msgr2=0x7f118006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:19.636 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.634+0000 7f119a775700 1 --2- 172.21.15.143:0/3008866756 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f118006af00 0x7f118006d3c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f11880097a0 tx=0x7f1188008070 comp rx=0 tx=0).stop 2023-11-08T19:26:19.636 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.634+0000 7f119a775700 1 -- 172.21.15.143:0/3008866756 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1194159870 msgr2=0x7f11941f05c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:19.636 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.634+0000 7f119a775700 1 --2- 172.21.15.143:0/3008866756 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1194159870 0x7f11941f05c0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f118c00cc20 tx=0x7f118c00a060 comp rx=0 tx=0).stop 2023-11-08T19:26:19.636 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.634+0000 7f119a775700 1 -- 172.21.15.143:0/3008866756 shutdown_connections 2023-11-08T19:26:19.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.634+0000 7f119a775700 1 --2- 172.21.15.143:0/3008866756 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f118006af00 0x7f118006d3c0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:19.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.634+0000 7f119a775700 1 --2- 172.21.15.143:0/3008866756 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f119406ed60 0x7f11940c4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:19.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.634+0000 7f119a775700 1 --2- 172.21.15.143:0/3008866756 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1194159870 0x7f11941f05c0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:19.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.634+0000 7f119a775700 1 --2- 172.21.15.143:0/3008866756 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f11940cc410 0x7f11940c5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:19.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.634+0000 7f119a775700 1 -- 172.21.15.143:0/3008866756 >> 172.21.15.143:0/3008866756 conn(0x7f1194099ab0 msgr2=0x7f11940d0720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:19.638 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.634+0000 7f119a775700 1 -- 172.21.15.143:0/3008866756 shutdown_connections 2023-11-08T19:26:19.638 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.634+0000 7f119a775700 1 -- 172.21.15.143:0/3008866756 wait complete. 2023-11-08T19:26:19.638 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 18 2023-11-08T19:26:19.648 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-08T19:26:19.649 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-08T19:26:19.649 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-08T19:26:19.649 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T19:26:19.650 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-08T19:26:19.650 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-08T19:26:19.650 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-08T19:26:19.651 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T19:26:19.651 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-08T19:26:19.651 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-08T19:26:19.652 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-08T19:26:19.652 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T19:26:19.652 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-08T19:26:19.652 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-08T19:26:19.653 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-08T19:26:19.653 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T19:26:19.658 INFO:tasks.ceph.mds.a:Stopped 2023-11-08T19:26:19.659 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-08T19:26:19.660 INFO:tasks.ceph.mds.c:Stopped 2023-11-08T19:26:19.661 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-08T19:26:19.663 INFO:tasks.ceph.mds.b:Stopped 2023-11-08T19:26:19.664 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-08T19:26:19.665 INFO:tasks.ceph.mds.d:Stopped 2023-11-08T19:26:19.665 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-08T19:26:19.702 INFO:tasks.ceph.mds.a:Started 2023-11-08T19:26:19.704 INFO:tasks.ceph.mds.c:Started 2023-11-08T19:26:19.704 DEBUG:teuthology.parallel:result is None 2023-11-08T19:26:19.705 DEBUG:teuthology.parallel:result is None 2023-11-08T19:26:19.707 INFO:tasks.ceph.mds.b:Started 2023-11-08T19:26:19.708 DEBUG:teuthology.parallel:result is None 2023-11-08T19:26:19.710 INFO:tasks.ceph.mds.d:Started 2023-11-08T19:26:19.710 DEBUG:teuthology.parallel:result is None 2023-11-08T19:26:19.711 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-08T19:26:19.780 INFO:tasks.ceph.mds.c.smithi143.stdout:starting mds.c at 2023-11-08T19:26:19.781 INFO:tasks.ceph.mds.a.smithi143.stdout:starting mds.a at 2023-11-08T19:26:19.795 INFO:tasks.ceph.mds.d.smithi174.stdout:starting mds.d at 2023-11-08T19:26:19.795 INFO:tasks.ceph.mds.b.smithi174.stdout:starting mds.b at 2023-11-08T19:26:19.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.839+0000 7f6a22549700 1 Processor -- start 2023-11-08T19:26:19.841 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.839+0000 7f6a22549700 1 -- start start 2023-11-08T19:26:19.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.840+0000 7f6a22549700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f6a1c15aa30 0x7f6a1c0cf510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:19.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.840+0000 7f6a22549700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f6a1c0b2210 0x7f6a1c0b26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:19.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.840+0000 7f6a22549700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a1c0cf080 con 0x7f6a1c0cbe20 2023-11-08T19:26:19.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.840+0000 7f6a22549700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a1c0b2be0 con 0x7f6a1c15aa30 2023-11-08T19:26:19.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.840+0000 7f6a22549700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a1c0ce470 con 0x7f6a1c0b2210 2023-11-08T19:26:19.845 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.840+0000 7f6a1bfff700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f6a1c15aa30 0x7f6a1c0cf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:19.845 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.840+0000 7f6a1bfff700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f6a1c15aa30 0x7f6a1c0cf510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51396/0 (socket says 172.21.15.143:51396) 2023-11-08T19:26:19.845 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.840+0000 7f6a1bfff700 1 -- 172.21.15.143:0/335701996 learned_addr learned my addr 172.21.15.143:0/335701996 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:19.845 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.840+0000 7f6a1b7fe700 1 --2- 172.21.15.143:0/335701996 >> v2:172.21.15.174:3300/0 conn(0x7f6a1c0b2210 0x7f6a1c0b26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:19.845 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.840+0000 7f6a1bfff700 1 -- 172.21.15.143:0/335701996 >> v2:172.21.15.174:3300/0 conn(0x7f6a1c0b2210 msgr2=0x7f6a1c0b26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:19.846 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.840+0000 7f6a1bfff700 1 --2- 172.21.15.143:0/335701996 >> v2:172.21.15.174:3300/0 conn(0x7f6a1c0b2210 0x7f6a1c0b26a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:19.846 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.840+0000 7f6a1bfff700 1 -- 172.21.15.143:0/335701996 >> v1:172.21.15.174:6789/0 conn(0x7f6a1c0cbe20 legacy=0x7f6a1c0cc220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:19.846 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.840+0000 7f6a1bfff700 1 -- 172.21.15.143:0/335701996 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a1c0ce750 con 0x7f6a1c15aa30 2023-11-08T19:26:19.846 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.840+0000 7f6a1b7fe700 1 --2- 172.21.15.143:0/335701996 >> v2:172.21.15.174:3300/0 conn(0x7f6a1c0b2210 0x7f6a1c0b26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:26:19.847 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.840+0000 7f6a1bfff700 1 --2- 172.21.15.143:0/335701996 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a1c15aa30 0x7f6a1c0cf510 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c0ce260 tx=0x7f6a0c00be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90bbe1e57aea1cd8 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:19.847 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.841+0000 7f6a1affd700 1 -- 172.21.15.143:0/335701996 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a0c005e30 con 0x7f6a1c15aa30 2023-11-08T19:26:19.847 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.841+0000 7f6a1affd700 1 -- 172.21.15.143:0/335701996 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a0c00d8b0 con 0x7f6a1c15aa30 2023-11-08T19:26:19.847 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.841+0000 7f6a1affd700 1 -- 172.21.15.143:0/335701996 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a0c00dc40 con 0x7f6a1c15aa30 2023-11-08T19:26:19.847 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.841+0000 7f6a22549700 1 -- 172.21.15.143:0/335701996 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a1c15aa30 msgr2=0x7f6a1c0cf510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:19.848 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.841+0000 7f6a22549700 1 --2- 172.21.15.143:0/335701996 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a1c15aa30 0x7f6a1c0cf510 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c0ce260 tx=0x7f6a0c00be70 comp rx=0 tx=0).stop 2023-11-08T19:26:19.848 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.841+0000 7f6a22549700 1 -- 172.21.15.143:0/335701996 shutdown_connections 2023-11-08T19:26:19.848 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.841+0000 7f6a22549700 1 --2- 172.21.15.143:0/335701996 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a1c15aa30 0x7f6a1c0cf510 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:19.848 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.841+0000 7f6a22549700 1 --2- 172.21.15.143:0/335701996 >> v2:172.21.15.174:3300/0 conn(0x7f6a1c0b2210 0x7f6a1c0b26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:19.848 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.841+0000 7f6a22549700 1 -- 172.21.15.143:0/335701996 >> 172.21.15.143:0/335701996 conn(0x7f6a1c099ab0 msgr2=0x7f6a1c0d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:19.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.841+0000 7f6a22549700 1 -- 172.21.15.143:0/335701996 shutdown_connections 2023-11-08T19:26:19.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.841+0000 7f6a22549700 1 -- 172.21.15.143:0/335701996 wait complete. 2023-11-08T19:26:19.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.842+0000 7f6a22549700 1 Processor -- start 2023-11-08T19:26:19.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.843+0000 7f6a22549700 1 -- start start 2023-11-08T19:26:19.852 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.843+0000 7f6a22549700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6a1c0b2210 0x7f6a1c0c3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:19.852 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.843+0000 7f6a22549700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6a1c0cbe20 0x7f6a1c0cc200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:19.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.843+0000 7f6a22549700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a1c15aa30 0x7f6a1c0cc740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:19.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.843+0000 7f6a22549700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a1c0c3ab0 con 0x7f6a1c15aa30 2023-11-08T19:26:19.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.843+0000 7f6a22549700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a1c15dc80 con 0x7f6a1c0b2210 2023-11-08T19:26:19.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.843+0000 7f6a22549700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a1c15d350 con 0x7f6a1c0cbe20 2023-11-08T19:26:19.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.843+0000 7f6a20ae6700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a1c15aa30 0x7f6a1c0cc740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:19.854 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.843+0000 7f6a20ae6700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a1c15aa30 0x7f6a1c0cc740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51404/0 (socket says 172.21.15.143:51404) 2023-11-08T19:26:19.854 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.843+0000 7f6a20ae6700 1 -- 172.21.15.143:0/2824937057 learned_addr learned my addr 172.21.15.143:0/2824937057 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:19.854 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.844+0000 7f6a1bfff700 1 --2- 172.21.15.143:0/2824937057 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6a1c0b2210 0x7f6a1c0c3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:19.854 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.844+0000 7f6a1b7fe700 1 --2- 172.21.15.143:0/2824937057 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6a1c0cbe20 0x7f6a1c0cc200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:19.855 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.844+0000 7f6a20ae6700 1 -- 172.21.15.143:0/2824937057 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6a1c0cbe20 msgr2=0x7f6a1c0cc200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:19.855 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.844+0000 7f6a20ae6700 1 --2- 172.21.15.143:0/2824937057 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6a1c0cbe20 0x7f6a1c0cc200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:19.855 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.844+0000 7f6a20ae6700 1 -- 172.21.15.143:0/2824937057 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6a1c0b2210 msgr2=0x7f6a1c0c3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:19.855 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.844+0000 7f6a20ae6700 1 --2- 172.21.15.143:0/2824937057 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6a1c0b2210 0x7f6a1c0c3570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:19.855 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.844+0000 7f6a20ae6700 1 -- 172.21.15.143:0/2824937057 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a0c00b7a0 con 0x7f6a1c15aa30 2023-11-08T19:26:19.856 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.844+0000 7f6a1bfff700 1 --2- 172.21.15.143:0/2824937057 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6a1c0b2210 0x7f6a1c0c3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:26:19.856 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.844+0000 7f6a1b7fe700 1 --2- 172.21.15.143:0/2824937057 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6a1c0cbe20 0x7f6a1c0cc200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:26:19.856 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.844+0000 7f6a20ae6700 1 --2- 172.21.15.143:0/2824937057 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a1c15aa30 0x7f6a1c0cc740 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f6a1400cc40 tx=0x7f6a1400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:19.856 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.844+0000 7f6a18ff9700 1 -- 172.21.15.143:0/2824937057 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a1400aba0 con 0x7f6a1c15aa30 2023-11-08T19:26:19.856 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.844+0000 7f6a18ff9700 1 -- 172.21.15.143:0/2824937057 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a1400ad00 con 0x7f6a1c15aa30 2023-11-08T19:26:19.856 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.844+0000 7f6a22549700 1 -- 172.21.15.143:0/2824937057 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a1c15d630 con 0x7f6a1c15aa30 2023-11-08T19:26:19.857 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.844+0000 7f6a18ff9700 1 -- 172.21.15.143:0/2824937057 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a14012660 con 0x7f6a1c15aa30 2023-11-08T19:26:19.857 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.844+0000 7f6a22549700 1 -- 172.21.15.143:0/2824937057 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a1c15d060 con 0x7f6a1c15aa30 2023-11-08T19:26:19.857 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.845+0000 7f6a22549700 1 -- 172.21.15.143:0/2824937057 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a1c15de10 con 0x7f6a1c15aa30 2023-11-08T19:26:19.857 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.846+0000 7f6a18ff9700 1 -- 172.21.15.143:0/2824937057 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6a140127c0 con 0x7f6a1c15aa30 2023-11-08T19:26:19.857 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.846+0000 7f6a18ff9700 1 --2- 172.21.15.143:0/2824937057 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f6a0806af00 0x7f6a0806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:19.858 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.846+0000 7f6a18ff9700 1 -- 172.21.15.143:0/2824937057 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f6a14089ba0 con 0x7f6a1c15aa30 2023-11-08T19:26:19.858 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.846+0000 7f6a1bfff700 1 --2- 172.21.15.143:0/2824937057 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f6a0806af00 0x7f6a0806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:19.858 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.846+0000 7f6a1bfff700 1 --2- 172.21.15.143:0/2824937057 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f6a0806af00 0x7f6a0806d3c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c00beb0 tx=0x7f6a0c012000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:19.858 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:19.850+0000 7f6a18ff9700 1 -- 172.21.15.143:0/2824937057 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6a14054df0 con 0x7f6a1c15aa30 2023-11-08T19:26:20.054 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:20.053+0000 7f6a22549700 1 -- 172.21.15.143:0/2824937057 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f6a1c04eee0 con 0x7f6a1c15aa30 2023-11-08T19:26:20.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:20.707+0000 7f6a18ff9700 1 -- 172.21.15.143:0/2824937057 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v42) v1 ==== 97+0+0 (secure 0 0 0) 0x7f6a14016070 con 0x7f6a1c15aa30 2023-11-08T19:26:20.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:20.709+0000 7f6a22549700 1 -- 172.21.15.143:0/2824937057 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f6a0806af00 msgr2=0x7f6a0806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:20.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:20.709+0000 7f6a22549700 1 --2- 172.21.15.143:0/2824937057 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f6a0806af00 0x7f6a0806d3c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c00beb0 tx=0x7f6a0c012000 comp rx=0 tx=0).stop 2023-11-08T19:26:20.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:20.709+0000 7f6a22549700 1 -- 172.21.15.143:0/2824937057 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a1c15aa30 msgr2=0x7f6a1c0cc740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:20.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:20.709+0000 7f6a22549700 1 --2- 172.21.15.143:0/2824937057 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a1c15aa30 0x7f6a1c0cc740 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f6a1400cc40 tx=0x7f6a1400a060 comp rx=0 tx=0).stop 2023-11-08T19:26:20.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:20.709+0000 7f6a22549700 1 -- 172.21.15.143:0/2824937057 shutdown_connections 2023-11-08T19:26:20.712 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:20.709+0000 7f6a22549700 1 --2- 172.21.15.143:0/2824937057 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f6a0806af00 0x7f6a0806d3c0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:20.713 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:20.709+0000 7f6a22549700 1 --2- 172.21.15.143:0/2824937057 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a1c15aa30 0x7f6a1c0cc740 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:20.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:20.709+0000 7f6a22549700 1 --2- 172.21.15.143:0/2824937057 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6a1c0b2210 0x7f6a1c0c3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:20.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:20.709+0000 7f6a22549700 1 --2- 172.21.15.143:0/2824937057 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6a1c0cbe20 0x7f6a1c0cc200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:20.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:20.709+0000 7f6a22549700 1 -- 172.21.15.143:0/2824937057 >> 172.21.15.143:0/2824937057 conn(0x7f6a1c099ab0 msgr2=0x7f6a1c06f2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:20.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:20.709+0000 7f6a22549700 1 -- 172.21.15.143:0/2824937057 shutdown_connections 2023-11-08T19:26:20.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:20.709+0000 7f6a22549700 1 -- 172.21.15.143:0/2824937057 wait complete. 2023-11-08T19:26:20.716 INFO:teuthology.orchestra.run.smithi143.stderr: removed all blocklist entries 2023-11-08T19:26:20.726 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-08T19:26:20.850 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:20.849+0000 7f76f567d700 1 Processor -- start 2023-11-08T19:26:20.851 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:20.849+0000 7f76f567d700 1 -- start start 2023-11-08T19:26:20.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:20.850+0000 7f76f567d700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f76f015dc80 0x7f76f00d1f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:20.854 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:20.850+0000 7f76f567d700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f76f00d3690 0x7f76f00b9690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:20.854 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:20.850+0000 7f76f567d700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f76f00ce980 0x7f76f00cee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:20.854 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:20.850+0000 7f76f567d700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76f00cf350 con 0x7f76f015dc80 2023-11-08T19:26:20.854 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:20.850+0000 7f76f567d700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76f00b9bd0 con 0x7f76f00d3690 2023-11-08T19:26:20.855 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:20.850+0000 7f76f567d700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76f00d3200 con 0x7f76f00ce980 2023-11-08T19:26:20.855 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:20.850+0000 7f76eeffd700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f76f015dc80 0x7f76f00d1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:20.855 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:20.850+0000 7f76eeffd700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f76f015dc80 0x7f76f00d1f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51412/0 (socket says 172.21.15.143:51412) 2023-11-08T19:26:20.855 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:20.850+0000 7f76eeffd700 1 -- 172.21.15.143:0/601955249 learned_addr learned my addr 172.21.15.143:0/601955249 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:20.856 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:20.850+0000 7f76ef7fe700 1 --2- 172.21.15.143:0/601955249 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f76f00ce980 0x7f76f00cee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:20.856 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:20.850+0000 7f76ee7fc700 1 --2- 172.21.15.143:0/601955249 >> v2:172.21.15.174:3300/0 conn(0x7f76f00d3690 0x7f76f00b9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:20.856 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:20.850+0000 7f76eeffd700 1 -- 172.21.15.143:0/601955249 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f76f00ce980 msgr2=0x7f76f00cee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:20.856 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:20.850+0000 7f76eeffd700 1 --2- 172.21.15.143:0/601955249 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f76f00ce980 0x7f76f00cee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:20.857 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:20.850+0000 7f76eeffd700 1 -- 172.21.15.143:0/601955249 >> v2:172.21.15.174:3300/0 conn(0x7f76f00d3690 msgr2=0x7f76f00b9690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:20.857 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:20.850+0000 7f76eeffd700 1 --2- 172.21.15.143:0/601955249 >> v2:172.21.15.174:3300/0 conn(0x7f76f00d3690 0x7f76f00b9690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:20.857 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:20.850+0000 7f76eeffd700 1 -- 172.21.15.143:0/601955249 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76f015e060 con 0x7f76f015dc80 2023-11-08T19:26:20.857 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:20.850+0000 7f76ef7fe700 1 --2- 172.21.15.143:0/601955249 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f76f00ce980 0x7f76f00cee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:26:20.857 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:20.850+0000 7f76ee7fc700 1 --2- 172.21.15.143:0/601955249 >> v2:172.21.15.174:3300/0 conn(0x7f76f00d3690 0x7f76f00b9690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:26:20.858 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:20.850+0000 7f76eeffd700 1 --2- 172.21.15.143:0/601955249 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f76f015dc80 0x7f76f00d1f80 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f76f0161760 tx=0x7f76e0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4646b396fe577d55 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:20.858 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:20.850+0000 7f76edffb700 1 -- 172.21.15.143:0/601955249 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76e000b860 con 0x7f76f015dc80 2023-11-08T19:26:20.858 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:20.850+0000 7f76edffb700 1 -- 172.21.15.143:0/601955249 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76e000b9c0 con 0x7f76f015dc80 2023-11-08T19:26:20.858 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:20.850+0000 7f76edffb700 1 -- 172.21.15.143:0/601955249 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76e000bd50 con 0x7f76f015dc80 2023-11-08T19:26:20.859 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:20.851+0000 7f76f567d700 1 -- 172.21.15.143:0/601955249 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f76f015dc80 msgr2=0x7f76f00d1f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:20.859 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:20.851+0000 7f76f567d700 1 --2- 172.21.15.143:0/601955249 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f76f015dc80 0x7f76f00d1f80 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f76f0161760 tx=0x7f76e0009c00 comp rx=0 tx=0).stop 2023-11-08T19:26:20.859 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:20.851+0000 7f76f567d700 1 -- 172.21.15.143:0/601955249 shutdown_connections 2023-11-08T19:26:20.859 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:20.851+0000 7f76f567d700 1 --2- 172.21.15.143:0/601955249 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f76f00ce980 0x7f76f00cee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:20.859 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:20.851+0000 7f76f567d700 1 --2- 172.21.15.143:0/601955249 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f76f015dc80 0x7f76f00d1f80 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:20.860 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:20.851+0000 7f76f567d700 1 --2- 172.21.15.143:0/601955249 >> v2:172.21.15.174:3300/0 conn(0x7f76f00d3690 0x7f76f00b9690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:20.860 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:20.851+0000 7f76f567d700 1 -- 172.21.15.143:0/601955249 >> 172.21.15.143:0/601955249 conn(0x7f76f00668e0 msgr2=0x7f76f0066cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:20.860 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:20.851+0000 7f76f567d700 1 -- 172.21.15.143:0/601955249 shutdown_connections 2023-11-08T19:26:20.861 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:20.851+0000 7f76f567d700 1 -- 172.21.15.143:0/601955249 wait complete. 2023-11-08T19:26:20.861 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:20.852+0000 7f76f567d700 1 Processor -- start 2023-11-08T19:26:20.861 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:20.852+0000 7f76f567d700 1 -- start start 2023-11-08T19:26:20.864 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:20.853+0000 7f76f567d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f76f00ce980 0x7f76f0150280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:20.864 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:20.853+0000 7f76f567d700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f76f00d3690 0x7f76f00c8810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:20.864 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:20.853+0000 7f76f567d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f76f015dc80 0x7f76f015fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:20.865 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:20.853+0000 7f76f567d700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76f00c9180 con 0x7f76f015dc80 2023-11-08T19:26:20.865 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:20.853+0000 7f76f567d700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76f00c9460 con 0x7f76f00ce980 2023-11-08T19:26:20.865 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:20.853+0000 7f76f567d700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76f00c9740 con 0x7f76f00d3690 2023-11-08T19:26:20.865 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:20.853+0000 7f76ef7fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f76f015dc80 0x7f76f015fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:20.865 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:20.853+0000 7f76ef7fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f76f015dc80 0x7f76f015fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51420/0 (socket says 172.21.15.143:51420) 2023-11-08T19:26:20.866 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:20.853+0000 7f76ef7fe700 1 -- 172.21.15.143:0/2912902253 learned_addr learned my addr 172.21.15.143:0/2912902253 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:20.866 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:20.853+0000 7f76eeffd700 1 --2- 172.21.15.143:0/2912902253 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f76f00ce980 0x7f76f0150280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:20.866 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:20.853+0000 7f76ee7fc700 1 --2- 172.21.15.143:0/2912902253 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f76f00d3690 0x7f76f00c8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:20.866 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:20.853+0000 7f76ef7fe700 1 -- 172.21.15.143:0/2912902253 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f76f00d3690 msgr2=0x7f76f00c8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:20.867 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:20.853+0000 7f76ef7fe700 1 --2- 172.21.15.143:0/2912902253 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f76f00d3690 0x7f76f00c8810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:20.867 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:20.853+0000 7f76ef7fe700 1 -- 172.21.15.143:0/2912902253 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f76f00ce980 msgr2=0x7f76f0150280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:20.867 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:20.853+0000 7f76ef7fe700 1 --2- 172.21.15.143:0/2912902253 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f76f00ce980 0x7f76f0150280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:20.867 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:20.853+0000 7f76ef7fe700 1 -- 172.21.15.143:0/2912902253 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76e0009550 con 0x7f76f015dc80 2023-11-08T19:26:20.867 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:20.854+0000 7f76ef7fe700 1 --2- 172.21.15.143:0/2912902253 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f76f015dc80 0x7f76f015fd30 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f76e800cd10 tx=0x7f76e800a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:20.868 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:20.854+0000 7f76d77fe700 1 -- 172.21.15.143:0/2912902253 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76e800ad10 con 0x7f76f015dc80 2023-11-08T19:26:20.868 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:20.854+0000 7f76d77fe700 1 -- 172.21.15.143:0/2912902253 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76e800ae90 con 0x7f76f015dc80 2023-11-08T19:26:20.868 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:20.854+0000 7f76f567d700 1 -- 172.21.15.143:0/2912902253 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76f00c9a20 con 0x7f76f015dc80 2023-11-08T19:26:20.868 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:20.854+0000 7f76d77fe700 1 -- 172.21.15.143:0/2912902253 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76e800f6c0 con 0x7f76f015dc80 2023-11-08T19:26:20.868 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:20.854+0000 7f76f567d700 1 -- 172.21.15.143:0/2912902253 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76f01507c0 con 0x7f76f015dc80 2023-11-08T19:26:20.869 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:20.855+0000 7f76f567d700 1 -- 172.21.15.143:0/2912902253 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76f004eee0 con 0x7f76f015dc80 2023-11-08T19:26:20.869 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:20.855+0000 7f76d77fe700 1 -- 172.21.15.143:0/2912902253 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f76e800b040 con 0x7f76f015dc80 2023-11-08T19:26:20.869 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:20.856+0000 7f76d77fe700 1 --2- 172.21.15.143:0/2912902253 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f76dc06af00 0x7f76dc06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:20.869 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:20.856+0000 7f76d77fe700 1 -- 172.21.15.143:0/2912902253 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f76e80899e0 con 0x7f76f015dc80 2023-11-08T19:26:20.869 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:20.856+0000 7f76eeffd700 1 --2- 172.21.15.143:0/2912902253 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f76dc06af00 0x7f76dc06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:20.870 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:20.856+0000 7f76eeffd700 1 --2- 172.21.15.143:0/2912902253 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f76dc06af00 0x7f76dc06d3c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f76e0000c00 tx=0x7f76e0008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:20.870 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:20.860+0000 7f76d77fe700 1 -- 172.21.15.143:0/2912902253 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f76e8054be0 con 0x7f76f015dc80 2023-11-08T19:26:21.126 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.124+0000 7f76f567d700 1 -- 172.21.15.143:0/2912902253 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f76f0074130 con 0x7f76f015dc80 2023-11-08T19:26:21.126 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.125+0000 7f76d77fe700 1 -- 172.21.15.143:0/2912902253 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v9) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f76e8058200 con 0x7f76f015dc80 2023-11-08T19:26:21.127 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:26:21.127 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2023-11-08T19:26:21.127 INFO:teuthology.orchestra.run.smithi143.stdout: "auth_dump": [ 2023-11-08T19:26:21.127 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-08T19:26:21.128 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "mds.a", 2023-11-08T19:26:21.128 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQBO4EtlqQPkOhAACwt3IOZLFtEQle9/FFliYA==", 2023-11-08T19:26:21.128 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-08T19:26:21.128 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2023-11-08T19:26:21.128 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow *", 2023-11-08T19:26:21.129 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow *", 2023-11-08T19:26:21.129 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-08T19:26:21.129 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-08T19:26:21.129 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-08T19:26:21.129 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-08T19:26:21.130 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "mds.b", 2023-11-08T19:26:21.130 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQBP4EtlHJqqCxAAxgT7Ce6QICWZ0qebHWZhMQ==", 2023-11-08T19:26:21.130 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-08T19:26:21.130 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2023-11-08T19:26:21.130 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow *", 2023-11-08T19:26:21.131 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow *", 2023-11-08T19:26:21.131 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-08T19:26:21.131 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-08T19:26:21.131 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-08T19:26:21.131 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-08T19:26:21.132 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "mds.c", 2023-11-08T19:26:21.132 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQBP4EtlGEeDBhAAihr82JiBJZR0rRkQj2n8wA==", 2023-11-08T19:26:21.132 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-08T19:26:21.132 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2023-11-08T19:26:21.133 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow *", 2023-11-08T19:26:21.133 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow *", 2023-11-08T19:26:21.133 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-08T19:26:21.133 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-08T19:26:21.133 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-08T19:26:21.134 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-08T19:26:21.134 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "mds.d", 2023-11-08T19:26:21.134 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQBP4EtlW5//EhAArVOkzNpCYd6Skl6eh5EYUQ==", 2023-11-08T19:26:21.134 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-08T19:26:21.134 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2023-11-08T19:26:21.135 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow *", 2023-11-08T19:26:21.135 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow *", 2023-11-08T19:26:21.135 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-08T19:26:21.135 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-08T19:26:21.135 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-08T19:26:21.136 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-08T19:26:21.136 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.0", 2023-11-08T19:26:21.136 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQBS4EtloVd4KRAAi5nj/vB9UK/mWZ9lkaFAxg==", 2023-11-08T19:26:21.136 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-08T19:26:21.136 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2023-11-08T19:26:21.137 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2023-11-08T19:26:21.137 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-08T19:26:21.137 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-08T19:26:21.137 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-08T19:26:21.137 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-08T19:26:21.138 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.1", 2023-11-08T19:26:21.138 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQBV4EtlWlOaKRAAooz5P+AoTgyuL/QAv4REKA==", 2023-11-08T19:26:21.138 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-08T19:26:21.138 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2023-11-08T19:26:21.138 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2023-11-08T19:26:21.139 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-08T19:26:21.139 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-08T19:26:21.139 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-08T19:26:21.139 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-08T19:26:21.139 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.2", 2023-11-08T19:26:21.140 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQBY4EtlBpaWKhAAmob9TfEKw5x6dEIu8ALVGA==", 2023-11-08T19:26:21.140 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-08T19:26:21.140 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2023-11-08T19:26:21.140 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2023-11-08T19:26:21.140 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-08T19:26:21.141 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-08T19:26:21.141 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-08T19:26:21.141 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-08T19:26:21.141 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.3", 2023-11-08T19:26:21.141 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQBb4Etl5pkrKhAA38ATMC3NkZ4plRRA69i0vw==", 2023-11-08T19:26:21.142 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-08T19:26:21.142 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2023-11-08T19:26:21.142 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2023-11-08T19:26:21.142 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-08T19:26:21.142 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-08T19:26:21.143 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-08T19:26:21.143 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-08T19:26:21.143 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.4", 2023-11-08T19:26:21.143 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQBh4EtlqLnUMBAABIrSskahtrZR0ClfczX5Ag==", 2023-11-08T19:26:21.144 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-08T19:26:21.144 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2023-11-08T19:26:21.144 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2023-11-08T19:26:21.144 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-08T19:26:21.144 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-08T19:26:21.145 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-08T19:26:21.145 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-08T19:26:21.145 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.5", 2023-11-08T19:26:21.145 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQBk4Etlt9PTMBAAz55IHg6Jrin3BUAvhOM/Fg==", 2023-11-08T19:26:21.145 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-08T19:26:21.146 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2023-11-08T19:26:21.146 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2023-11-08T19:26:21.146 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-08T19:26:21.146 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-08T19:26:21.146 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-08T19:26:21.147 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-08T19:26:21.147 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.6", 2023-11-08T19:26:21.147 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQBn4EtlcTkyMBAA5fD3h4s+99rgd6fqHHCFMw==", 2023-11-08T19:26:21.147 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-08T19:26:21.147 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2023-11-08T19:26:21.148 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2023-11-08T19:26:21.148 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-08T19:26:21.148 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-08T19:26:21.148 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-08T19:26:21.148 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-08T19:26:21.149 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.7", 2023-11-08T19:26:21.149 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQBq4EtldEviMBAAyGYrHVGwxHdmtDOwWCSHDQ==", 2023-11-08T19:26:21.149 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-08T19:26:21.149 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2023-11-08T19:26:21.150 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2023-11-08T19:26:21.150 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-08T19:26:21.150 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-08T19:26:21.150 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-08T19:26:21.150 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-08T19:26:21.151 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "client.0", 2023-11-08T19:26:21.151 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQBP4Etl0GdSFxAAVkuHL0EnjrtTUMkj/afPkg==", 2023-11-08T19:26:21.151 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-08T19:26:21.151 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2023-11-08T19:26:21.151 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow r", 2023-11-08T19:26:21.152 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-11-08T19:26:21.152 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-08T19:26:21.152 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-08T19:26:21.152 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-08T19:26:21.152 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "client.1", 2023-11-08T19:26:21.153 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQBP4EtlPQ5yIBAAiyQaqCJph2KOPBZPSIh4ZQ==", 2023-11-08T19:26:21.153 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-08T19:26:21.153 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2023-11-08T19:26:21.153 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow r", 2023-11-08T19:26:21.153 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow rw", 2023-11-08T19:26:21.153 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow rwx" 2023-11-08T19:26:21.154 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-08T19:26:21.154 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-08T19:26:21.154 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-08T19:26:21.154 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "client.2", 2023-11-08T19:26:21.155 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQBP4Etlm+8SHRAAAwxcy2EzUvtFw9/mTzyk7w==", 2023-11-08T19:26:21.155 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-08T19:26:21.155 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2023-11-08T19:26:21.155 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow r", 2023-11-08T19:26:21.155 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow rw", 2023-11-08T19:26:21.155 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow rwx" 2023-11-08T19:26:21.156 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-08T19:26:21.156 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-08T19:26:21.156 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-08T19:26:21.156 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "client.3", 2023-11-08T19:26:21.156 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQBP4EtlNDC8IxAACBkCM+3TXhJv5eE9xLvwKA==", 2023-11-08T19:26:21.157 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-08T19:26:21.157 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2023-11-08T19:26:21.157 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow r", 2023-11-08T19:26:21.157 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow rw", 2023-11-08T19:26:21.157 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow rwx" 2023-11-08T19:26:21.158 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-08T19:26:21.158 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-08T19:26:21.158 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-08T19:26:21.158 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "client.admin", 2023-11-08T19:26:21.158 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQBO4Etlr5yuHRAAblpqD18z9bT3Wmgn74dehg==", 2023-11-08T19:26:21.159 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-08T19:26:21.160 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow *", 2023-11-08T19:26:21.161 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow *", 2023-11-08T19:26:21.161 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow *", 2023-11-08T19:26:21.161 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-08T19:26:21.161 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-08T19:26:21.161 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-08T19:26:21.162 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-08T19:26:21.162 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "mgr.x", 2023-11-08T19:26:21.162 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQBO4EtlzrqBNxAA5rtcgkzroSTKuBBmeQRiig==", 2023-11-08T19:26:21.162 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-08T19:26:21.162 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow *", 2023-11-08T19:26:21.163 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile mgr", 2023-11-08T19:26:21.163 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-08T19:26:21.163 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-08T19:26:21.163 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-08T19:26:21.163 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-08T19:26:21.164 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "mgr.y", 2023-11-08T19:26:21.164 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQBO4Etl4Gl4MxAArxdUnjLDhTLO3feXHV4HyQ==", 2023-11-08T19:26:21.164 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-08T19:26:21.164 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow *", 2023-11-08T19:26:21.164 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile mgr", 2023-11-08T19:26:21.165 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-08T19:26:21.165 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-08T19:26:21.165 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-08T19:26:21.165 INFO:teuthology.orchestra.run.smithi143.stdout: ] 2023-11-08T19:26:21.165 INFO:teuthology.orchestra.run.smithi143.stdout:} 2023-11-08T19:26:21.167 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.127+0000 7f76f567d700 1 -- 172.21.15.143:0/2912902253 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f76dc06af00 msgr2=0x7f76dc06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:21.167 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.127+0000 7f76f567d700 1 --2- 172.21.15.143:0/2912902253 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f76dc06af00 0x7f76dc06d3c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f76e0000c00 tx=0x7f76e0008030 comp rx=0 tx=0).stop 2023-11-08T19:26:21.167 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.127+0000 7f76f567d700 1 -- 172.21.15.143:0/2912902253 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f76f015dc80 msgr2=0x7f76f015fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:21.168 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.127+0000 7f76f567d700 1 --2- 172.21.15.143:0/2912902253 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f76f015dc80 0x7f76f015fd30 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f76e800cd10 tx=0x7f76e800a560 comp rx=0 tx=0).stop 2023-11-08T19:26:21.168 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.127+0000 7f76f567d700 1 -- 172.21.15.143:0/2912902253 shutdown_connections 2023-11-08T19:26:21.168 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.127+0000 7f76f567d700 1 --2- 172.21.15.143:0/2912902253 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f76dc06af00 0x7f76dc06d3c0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:21.168 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.127+0000 7f76f567d700 1 --2- 172.21.15.143:0/2912902253 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f76f015dc80 0x7f76f015fd30 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:21.168 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.127+0000 7f76f567d700 1 --2- 172.21.15.143:0/2912902253 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f76f00ce980 0x7f76f0150280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:21.169 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.127+0000 7f76f567d700 1 --2- 172.21.15.143:0/2912902253 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f76f00d3690 0x7f76f00c8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:21.169 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.127+0000 7f76f567d700 1 -- 172.21.15.143:0/2912902253 >> 172.21.15.143:0/2912902253 conn(0x7f76f00668e0 msgr2=0x7f76f00d0600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:21.169 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.127+0000 7f76f567d700 1 -- 172.21.15.143:0/2912902253 shutdown_connections 2023-11-08T19:26:21.169 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.127+0000 7f76f567d700 1 -- 172.21.15.143:0/2912902253 wait complete. 2023-11-08T19:26:21.170 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:26:21.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.289+0000 7f94e318d700 1 Processor -- start 2023-11-08T19:26:21.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.289+0000 7f94e318d700 1 -- start start 2023-11-08T19:26:21.295 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.290+0000 7f94e318d700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f94dc0cc530 0x7f94dc0cf510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:21.295 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.290+0000 7f94e318d700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94dc0cf080 con 0x7f94dc15aa30 2023-11-08T19:26:21.295 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.290+0000 7f94e318d700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94dc0ce470 con 0x7f94dc0b2920 2023-11-08T19:26:21.295 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.290+0000 7f94e318d700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94dc0ce750 con 0x7f94dc0cc530 2023-11-08T19:26:21.295 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.290+0000 7f94e0f29700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f94dc15aa30 0x7f94dc0b2210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:47578/0 (socket says 172.21.15.143:47578) 2023-11-08T19:26:21.296 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.290+0000 7f94e0f29700 1 -- 172.21.15.143:0/3710081356 learned_addr learned my addr 172.21.15.143:0/3710081356 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:26:21.296 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.290+0000 7f94db7fe700 1 -- 172.21.15.143:0/3710081356 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2759775493 0 0) 0x7f94dc0cf080 con 0x7f94dc15aa30 2023-11-08T19:26:21.296 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.290+0000 7f94db7fe700 1 -- 172.21.15.143:0/3710081356 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94cc0035b0 con 0x7f94dc15aa30 2023-11-08T19:26:21.296 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.290+0000 7f94e172a700 1 --2- 172.21.15.143:0/3710081356 >> v2:172.21.15.174:3300/0 conn(0x7f94dc0cc530 0x7f94dc0cf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:21.296 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.290+0000 7f94db7fe700 1 -- 172.21.15.143:0/3710081356 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2319526069 0 0) 0x7f94cc0035b0 con 0x7f94dc15aa30 2023-11-08T19:26:21.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.290+0000 7f94db7fe700 1 -- 172.21.15.143:0/3710081356 >> v2:172.21.15.174:3300/0 conn(0x7f94dc0cc530 msgr2=0x7f94dc0cf510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:21.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.290+0000 7f94db7fe700 1 --2- 172.21.15.143:0/3710081356 >> v2:172.21.15.174:3300/0 conn(0x7f94dc0cc530 0x7f94dc0cf510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:21.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.290+0000 7f94db7fe700 1 -- 172.21.15.143:0/3710081356 >> v1:172.21.15.174:6789/0 conn(0x7f94dc0b2920 legacy=0x7f94dc0cbe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:21.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.290+0000 7f94db7fe700 1 -- 172.21.15.143:0/3710081356 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94dc0cea30 con 0x7f94dc15aa30 2023-11-08T19:26:21.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.290+0000 7f94db7fe700 1 -- 172.21.15.143:0/3710081356 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4147756303 0 0) 0x7f94d00017f0 con 0x7f94dc15aa30 2023-11-08T19:26:21.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.290+0000 7f94db7fe700 1 -- 172.21.15.143:0/3710081356 >> v1:172.21.15.143:6789/0 conn(0x7f94dc15aa30 legacy=0x7f94dc0b2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:21.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.291+0000 7f94db7fe700 1 --2- 172.21.15.143:0/3710081356 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f94cc003a00 0x7f94cc003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:21.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.291+0000 7f94db7fe700 1 --2- 172.21.15.143:0/3710081356 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94cc004850 0x7f94cc004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:21.299 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.291+0000 7f94db7fe700 1 --2- 172.21.15.143:0/3710081356 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f94cc005210 0x7f94cc00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:21.299 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.291+0000 7f94db7fe700 1 -- 172.21.15.143:0/3710081356 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94dc0ce750 con 0x7f94cc004850 2023-11-08T19:26:21.299 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.291+0000 7f94db7fe700 1 -- 172.21.15.143:0/3710081356 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94cc00c1b0 con 0x7f94cc003a00 2023-11-08T19:26:21.299 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.291+0000 7f94db7fe700 1 -- 172.21.15.143:0/3710081356 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94cc00c370 con 0x7f94cc005210 2023-11-08T19:26:21.300 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.291+0000 7f94dbfff700 1 --2- 172.21.15.143:0/3710081356 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94cc004850 0x7f94cc004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:21.300 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.291+0000 7f94e0f29700 1 --2- 172.21.15.143:0/3710081356 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f94cc003a00 0x7f94cc003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:21.300 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.291+0000 7f94e172a700 1 --2- 172.21.15.143:0/3710081356 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f94cc005210 0x7f94cc00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:21.300 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.291+0000 7f94dbfff700 1 -- 172.21.15.143:0/3710081356 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f94cc005210 msgr2=0x7f94cc00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:21.301 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.291+0000 7f94dbfff700 1 --2- 172.21.15.143:0/3710081356 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f94cc005210 0x7f94cc00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:21.301 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.291+0000 7f94dbfff700 1 -- 172.21.15.143:0/3710081356 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f94cc003a00 msgr2=0x7f94cc003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:21.301 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.291+0000 7f94dbfff700 1 --2- 172.21.15.143:0/3710081356 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f94cc003a00 0x7f94cc003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:21.301 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.291+0000 7f94dbfff700 1 -- 172.21.15.143:0/3710081356 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94cc0035b0 con 0x7f94cc004850 2023-11-08T19:26:21.301 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.291+0000 7f94e172a700 1 -- 172.21.15.143:0/3710081356 reap_dead start 2023-11-08T19:26:21.301 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.291+0000 7f94e0f29700 1 --2- 172.21.15.143:0/3710081356 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f94cc003a00 0x7f94cc003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:26:21.302 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.291+0000 7f94dbfff700 1 --2- 172.21.15.143:0/3710081356 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94cc004850 0x7f94cc004cd0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f94c8001800 tx=0x7f94cc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e01bbe01fa4368f2 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:21.302 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.291+0000 7f94db7fe700 1 -- 172.21.15.143:0/3710081356 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94c80012f0 con 0x7f94cc004850 2023-11-08T19:26:21.302 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.291+0000 7f94db7fe700 1 -- 172.21.15.143:0/3710081356 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94c8005310 con 0x7f94cc004850 2023-11-08T19:26:21.302 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.291+0000 7f94db7fe700 1 -- 172.21.15.143:0/3710081356 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94c8006860 con 0x7f94cc004850 2023-11-08T19:26:21.303 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.292+0000 7f94e318d700 1 -- 172.21.15.143:0/3710081356 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94cc004850 msgr2=0x7f94cc004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:21.303 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.292+0000 7f94e318d700 1 --2- 172.21.15.143:0/3710081356 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94cc004850 0x7f94cc004cd0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f94c8001800 tx=0x7f94cc005880 comp rx=0 tx=0).stop 2023-11-08T19:26:21.303 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.292+0000 7f94e318d700 1 -- 172.21.15.143:0/3710081356 shutdown_connections 2023-11-08T19:26:21.303 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.292+0000 7f94e318d700 1 --2- 172.21.15.143:0/3710081356 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94cc004850 0x7f94cc004cd0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:21.303 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.292+0000 7f94e318d700 1 -- 172.21.15.143:0/3710081356 >> 172.21.15.143:0/3710081356 conn(0x7f94dc099ab0 msgr2=0x7f94dc0d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:21.304 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.292+0000 7f94e318d700 1 -- 172.21.15.143:0/3710081356 shutdown_connections 2023-11-08T19:26:21.304 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.292+0000 7f94e318d700 1 -- 172.21.15.143:0/3710081356 wait complete. 2023-11-08T19:26:21.304 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.293+0000 7f94e318d700 1 Processor -- start 2023-11-08T19:26:21.304 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.293+0000 7f94e318d700 1 -- start start 2023-11-08T19:26:21.307 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.294+0000 7f94e318d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f94cc004850 0x7f94dc15c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:21.307 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.294+0000 7f94e318d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94dc0b1220 0x7f94dc14f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:21.307 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.294+0000 7f94e318d700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f94dc1eeb40 0x7f94dc0ccf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:21.307 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.294+0000 7f94e318d700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94dc15af90 con 0x7f94dc0b1220 2023-11-08T19:26:21.308 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.294+0000 7f94e318d700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94dc15c9b0 con 0x7f94cc004850 2023-11-08T19:26:21.308 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.294+0000 7f94e318d700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94dc14fb90 con 0x7f94dc1eeb40 2023-11-08T19:26:21.308 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.295+0000 7f94dbfff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94dc0b1220 0x7f94dc14f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:21.308 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.295+0000 7f94dbfff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94dc0b1220 0x7f94dc14f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51436/0 (socket says 172.21.15.143:51436) 2023-11-08T19:26:21.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.295+0000 7f94dbfff700 1 -- 172.21.15.143:0/698626249 learned_addr learned my addr 172.21.15.143:0/698626249 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:21.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.295+0000 7f94e0f29700 1 --2- 172.21.15.143:0/698626249 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f94cc004850 0x7f94dc15c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:21.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.295+0000 7f94e172a700 1 --2- 172.21.15.143:0/698626249 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f94dc1eeb40 0x7f94dc0ccf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:21.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.295+0000 7f94dbfff700 1 -- 172.21.15.143:0/698626249 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f94dc1eeb40 msgr2=0x7f94dc0ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:21.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.295+0000 7f94dbfff700 1 --2- 172.21.15.143:0/698626249 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f94dc1eeb40 0x7f94dc0ccf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:21.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.295+0000 7f94dbfff700 1 -- 172.21.15.143:0/698626249 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f94cc004850 msgr2=0x7f94dc15c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:21.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.295+0000 7f94dbfff700 1 --2- 172.21.15.143:0/698626249 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f94cc004850 0x7f94dc15c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:21.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.295+0000 7f94dbfff700 1 -- 172.21.15.143:0/698626249 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94dc15cc50 con 0x7f94dc0b1220 2023-11-08T19:26:21.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.295+0000 7f94e0f29700 1 --2- 172.21.15.143:0/698626249 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f94cc004850 0x7f94dc15c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:26:21.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.295+0000 7f94dbfff700 1 --2- 172.21.15.143:0/698626249 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94dc0b1220 0x7f94dc14f650 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f94c8000fb0 tx=0x7f94c8001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:21.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.295+0000 7f94d97fa700 1 -- 172.21.15.143:0/698626249 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94c8006e70 con 0x7f94dc0b1220 2023-11-08T19:26:21.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.295+0000 7f94d97fa700 1 -- 172.21.15.143:0/698626249 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94c8001290 con 0x7f94dc0b1220 2023-11-08T19:26:21.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.295+0000 7f94e318d700 1 -- 172.21.15.143:0/698626249 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94dc15cf30 con 0x7f94dc0b1220 2023-11-08T19:26:21.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.295+0000 7f94d97fa700 1 -- 172.21.15.143:0/698626249 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94c8007250 con 0x7f94dc0b1220 2023-11-08T19:26:21.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.295+0000 7f94e318d700 1 -- 172.21.15.143:0/698626249 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94dc15b1a0 con 0x7f94dc0b1220 2023-11-08T19:26:21.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.296+0000 7f94e318d700 1 -- 172.21.15.143:0/698626249 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94dc15b330 con 0x7f94dc0b1220 2023-11-08T19:26:21.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.297+0000 7f94d97fa700 1 -- 172.21.15.143:0/698626249 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f94c80073d0 con 0x7f94dc0b1220 2023-11-08T19:26:21.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.297+0000 7f94d97fa700 1 --2- 172.21.15.143:0/698626249 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f94c006af00 0x7f94c006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:21.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.297+0000 7f94d97fa700 1 -- 172.21.15.143:0/698626249 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f94c808a510 con 0x7f94dc0b1220 2023-11-08T19:26:21.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.297+0000 7f94e0f29700 1 --2- 172.21.15.143:0/698626249 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f94c006af00 0x7f94c006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:21.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.298+0000 7f94e0f29700 1 --2- 172.21.15.143:0/698626249 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f94c006af00 0x7f94c006d3c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f94d0000ef0 tx=0x7f94d0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:21.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.301+0000 7f94d97fa700 1 -- 172.21.15.143:0/698626249 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f94c8055840 con 0x7f94dc0b1220 2023-11-08T19:26:21.495 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.494+0000 7f94e318d700 1 -- 172.21.15.143:0/698626249 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f94dc04eee0 con 0x7f94dc0b1220 2023-11-08T19:26:21.496 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.495+0000 7f94d97fa700 1 -- 172.21.15.143:0/698626249 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) v1 ==== 74+0+14658 (secure 0 0 0) 0x7f94c8058e60 con 0x7f94dc0b1220 2023-11-08T19:26:21.496 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:26:21.497 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":44,"fsid":"968483af-77fc-4068-ad7c-5c2a1a4613d7","created":"2023-11-08T19:24:32.826484+0000","modified":"2023-11-08T19:26:20.719942+0000","last_up_change":"2023-11-08T19:24:43.573880+0000","last_in_change":"2023-11-08T19:24:35.314525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:44.551530+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"snap-test-pool","create_time":"2023-11-08T19:26:02.874384+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":36,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bae98ec1-d94e-403d-a220-a522a53ea6f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6809","nonce":1487830566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6811","nonce":1487830566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6815","nonce":1487830566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6813","nonce":1487830566}]},"public_addr":"172.21.15.143:6809/1487830566","cluster_addr":"172.21.15.143:6811/1487830566","heartbeat_back_addr":"172.21.15.143:6815/1487830566","heartbeat_front_addr":"172.21.15.143:6813/1487830566","state":["exists","up"]},{"osd":1,"uuid":"5cb6538a-1ccb-4322-8633-09f66af0b7e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6801","nonce":787341397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6803","nonce":787341397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6807","nonce":787341397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6805","nonce":787341397}]},"public_addr":"172.21.15.143:6801/787341397","cluster_addr":"172.21.15.143:6803/787341397","heartbeat_back_addr":"172.21.15.143:6807/787341397","heartbeat_front_addr":"172.21.15.143:6805/787341397","state":["exists","up"]},{"osd":2,"uuid":"cdbf522c-60ef-4d60-9653-8dcb529c101a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6822","nonce":2301325597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6826","nonce":2301325597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6831","nonce":2301325597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6829","nonce":2301325597}]},"public_addr":"172.21.15.143:6822/2301325597","cluster_addr":"172.21.15.143:6826/2301325597","heartbeat_back_addr":"172.21.15.143:6831/2301325597","heartbeat_front_addr":"172.21.15.143:6829/2301325597","state":["exists","up"]},{"osd":3,"uuid":"4e2783cf-ebb9-4dd8-8245-bc1d17ce4325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6817","nonce":2021504428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6819","nonce":2021504428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6827","nonce":2021504428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6823","nonce":2021504428}]},"public_addr":"172.21.15.143:6817/2021504428","cluster_addr":"172.21.15.143:6819/2021504428","heartbeat_back_addr":"172.21.15.143:6827/2021504428","heartbeat_front_addr":"172.21.15.143:6823/2021504428","state":["exists","up"]},{"osd":4,"uuid":"691256fa-9b77-49b9-a346-7ae1adf0ca80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6801","nonce":1993380595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6803","nonce":1993380595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6807","nonce":1993380595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6805","nonce":1993380595}]},"public_addr":"172.21.15.174:6801/1993380595","cluster_addr":"172.21.15.174:6803/1993380595","heartbeat_back_addr":"172.21.15.174:6807/1993380595","heartbeat_front_addr":"172.21.15.174:6805/1993380595","state":["exists","up"]},{"osd":5,"uuid":"756fc352-48c9-46ce-b52e-0541fe6f8525","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6809","nonce":1777923303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6811","nonce":1777923303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6815","nonce":1777923303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6813","nonce":1777923303}]},"public_addr":"172.21.15.174:6809/1777923303","cluster_addr":"172.21.15.174:6811/1777923303","heartbeat_back_addr":"172.21.15.174:6815/1777923303","heartbeat_front_addr":"172.21.15.174:6813/1777923303","state":["exists","up"]},{"osd":6,"uuid":"fab2ac2b-ed97-4cde-9825-b480eabf8b8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6825","nonce":2755130316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6827","nonce":2755130316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6831","nonce":2755130316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6829","nonce":2755130316}]},"public_addr":"172.21.15.174:6825/2755130316","cluster_addr":"172.21.15.174:6827/2755130316","heartbeat_back_addr":"172.21.15.174:6831/2755130316","heartbeat_front_addr":"172.21.15.174:6829/2755130316","state":["exists","up"]},{"osd":7,"uuid":"3de9b797-2ab6-4d41-a863-7e7d638fd703","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6817","nonce":2847141411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6819","nonce":2847141411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6823","nonce":2847141411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6821","nonce":2847141411}]},"public_addr":"172.21.15.174:6817/2847141411","cluster_addr":"172.21.15.174:6819/2847141411","heartbeat_back_addr":"172.21.15.174:6823/2847141411","heartbeat_front_addr":"172.21.15.174:6821/2847141411","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.477076+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.496890+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.539305+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.454673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.538881+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.521294+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.520712+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.540404+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:26:21.499 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.497+0000 7f94e318d700 1 -- 172.21.15.143:0/698626249 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f94c006af00 msgr2=0x7f94c006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:21.499 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.497+0000 7f94e318d700 1 --2- 172.21.15.143:0/698626249 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f94c006af00 0x7f94c006d3c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f94d0000ef0 tx=0x7f94d0009040 comp rx=0 tx=0).stop 2023-11-08T19:26:21.499 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.497+0000 7f94e318d700 1 -- 172.21.15.143:0/698626249 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94dc0b1220 msgr2=0x7f94dc14f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:21.499 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.497+0000 7f94e318d700 1 --2- 172.21.15.143:0/698626249 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94dc0b1220 0x7f94dc14f650 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f94c8000fb0 tx=0x7f94c8001120 comp rx=0 tx=0).stop 2023-11-08T19:26:21.499 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.497+0000 7f94e318d700 1 -- 172.21.15.143:0/698626249 shutdown_connections 2023-11-08T19:26:21.500 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.497+0000 7f94e318d700 1 --2- 172.21.15.143:0/698626249 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f94c006af00 0x7f94c006d3c0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:21.500 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.497+0000 7f94e318d700 1 --2- 172.21.15.143:0/698626249 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f94dc1eeb40 0x7f94dc0ccf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:21.500 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.497+0000 7f94e318d700 1 --2- 172.21.15.143:0/698626249 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f94cc004850 0x7f94dc15c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:21.500 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.497+0000 7f94e318d700 1 --2- 172.21.15.143:0/698626249 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94dc0b1220 0x7f94dc14f650 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:21.501 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.497+0000 7f94e318d700 1 -- 172.21.15.143:0/698626249 >> 172.21.15.143:0/698626249 conn(0x7f94dc099ab0 msgr2=0x7f94dc1589f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:21.501 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.497+0000 7f94e318d700 1 -- 172.21.15.143:0/698626249 shutdown_connections 2023-11-08T19:26:21.501 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.497+0000 7f94e318d700 1 -- 172.21.15.143:0/698626249 wait complete. 2023-11-08T19:26:21.511 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T19:24:44.551530+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'snap-test-pool', 'create_time': '2023-11-08T19:26:02.874384+0000', 'flags': 16385, 'flags_names': 'hashpspool,pool_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '36', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'pool', 'snap_seq': 2, 'snap_epoch': 36, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-08T19:26:21.511 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T19:26:21.635 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.633+0000 7ff2e2705700 1 Processor -- start 2023-11-08T19:26:21.773 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.634+0000 7ff2e2705700 1 -- start start 2023-11-08T19:26:21.777 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.634+0000 7ff2e2705700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff2dc15a950 0x7ff2dc0d01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:21.778 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.634+0000 7ff2e2705700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2dc0cc2a0 con 0x7ff2dc06bd20 2023-11-08T19:26:21.778 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.634+0000 7ff2e2705700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2dc06c1b0 con 0x7ff2dc0b61e0 2023-11-08T19:26:21.778 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.634+0000 7ff2e2705700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2dc0cc130 con 0x7ff2dc15a950 2023-11-08T19:26:21.778 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.634+0000 7ff2dbfff700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff2dc15a950 0x7ff2dc0d01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:21.779 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.635+0000 7ff2db7fe700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7ff2dc06bd20 0x7ff2dc072370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:47598/0 (socket says 172.21.15.143:47598) 2023-11-08T19:26:21.779 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.635+0000 7ff2db7fe700 1 -- 172.21.15.143:0/396376536 learned_addr learned my addr 172.21.15.143:0/396376536 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:26:21.779 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.635+0000 7ff2dbfff700 1 -- 172.21.15.143:0/396376536 >> v1:172.21.15.174:6789/0 conn(0x7ff2dc0b61e0 legacy=0x7ff2dc0cc730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:21.779 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.635+0000 7ff2dbfff700 1 -- 172.21.15.143:0/396376536 >> v1:172.21.15.143:6789/0 conn(0x7ff2dc06bd20 legacy=0x7ff2dc072370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:21.779 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.635+0000 7ff2dbfff700 1 -- 172.21.15.143:0/396376536 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2dc06c490 con 0x7ff2dc15a950 2023-11-08T19:26:21.779 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.635+0000 7ff2daffd700 1 -- 172.21.15.143:0/396376536 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1842165394 0 0) 0x7ff2dc0cc2a0 con 0x7ff2dc06bd20 2023-11-08T19:26:21.780 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.635+0000 7ff2dbfff700 1 --2- 172.21.15.143:0/396376536 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff2dc15a950 0x7ff2dc0d01b0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc009840 tx=0x7ff2cc009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bde9c5df85dab69 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:21.780 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.635+0000 7ff2daffd700 1 -- 172.21.15.143:0/396376536 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2cc004230 con 0x7ff2dc15a950 2023-11-08T19:26:21.780 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.635+0000 7ff2daffd700 1 -- 172.21.15.143:0/396376536 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2cc004390 con 0x7ff2dc15a950 2023-11-08T19:26:21.780 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.635+0000 7ff2daffd700 1 -- 172.21.15.143:0/396376536 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2cc0036a0 con 0x7ff2dc15a950 2023-11-08T19:26:21.781 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.636+0000 7ff2e2705700 1 -- 172.21.15.143:0/396376536 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff2dc15a950 msgr2=0x7ff2dc0d01b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:21.781 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.636+0000 7ff2e2705700 1 --2- 172.21.15.143:0/396376536 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff2dc15a950 0x7ff2dc0d01b0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc009840 tx=0x7ff2cc009b50 comp rx=0 tx=0).stop 2023-11-08T19:26:21.781 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.636+0000 7ff2e2705700 1 -- 172.21.15.143:0/396376536 shutdown_connections 2023-11-08T19:26:21.781 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.636+0000 7ff2e2705700 1 --2- 172.21.15.143:0/396376536 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff2dc15a950 0x7ff2dc0d01b0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:21.781 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.636+0000 7ff2e2705700 1 -- 172.21.15.143:0/396376536 >> 172.21.15.143:0/396376536 conn(0x7ff2dc0cd2e0 msgr2=0x7ff2dc0cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:21.782 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.636+0000 7ff2e2705700 1 -- 172.21.15.143:0/396376536 shutdown_connections 2023-11-08T19:26:21.782 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.636+0000 7ff2e2705700 1 -- 172.21.15.143:0/396376536 wait complete. 2023-11-08T19:26:21.782 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.637+0000 7ff2e2705700 1 Processor -- start 2023-11-08T19:26:21.782 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.637+0000 7ff2e2705700 1 -- start start 2023-11-08T19:26:21.783 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.638+0000 7ff2e2705700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff2dc0b61e0 0x7ff2dc153b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:21.783 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.638+0000 7ff2e2705700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff2dc15a950 0x7ff2dc154570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:21.783 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.638+0000 7ff2e2705700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff2dc154ab0 0x7ff2dc0d0790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:21.783 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.638+0000 7ff2e2705700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2dc154040 con 0x7ff2dc154ab0 2023-11-08T19:26:21.783 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.638+0000 7ff2e2705700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2dc154ff0 con 0x7ff2dc15a950 2023-11-08T19:26:21.784 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.638+0000 7ff2e2705700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2dc1552d0 con 0x7ff2dc0b61e0 2023-11-08T19:26:21.784 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.638+0000 7ff2e0ca2700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff2dc154ab0 0x7ff2dc0d0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:21.784 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.638+0000 7ff2e0ca2700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff2dc154ab0 0x7ff2dc0d0790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51454/0 (socket says 172.21.15.143:51454) 2023-11-08T19:26:21.784 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.638+0000 7ff2e0ca2700 1 -- 172.21.15.143:0/760043466 learned_addr learned my addr 172.21.15.143:0/760043466 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:21.784 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.638+0000 7ff2db7fe700 1 --2- 172.21.15.143:0/760043466 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff2dc15a950 0x7ff2dc154570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:21.785 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.638+0000 7ff2dbfff700 1 --2- 172.21.15.143:0/760043466 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff2dc0b61e0 0x7ff2dc153b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:21.785 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.638+0000 7ff2e0ca2700 1 -- 172.21.15.143:0/760043466 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff2dc0b61e0 msgr2=0x7ff2dc153b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:21.785 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.638+0000 7ff2e0ca2700 1 --2- 172.21.15.143:0/760043466 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff2dc0b61e0 0x7ff2dc153b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:21.785 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.638+0000 7ff2e0ca2700 1 -- 172.21.15.143:0/760043466 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff2dc15a950 msgr2=0x7ff2dc154570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:21.785 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.638+0000 7ff2e0ca2700 1 --2- 172.21.15.143:0/760043466 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff2dc15a950 0x7ff2dc154570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:21.786 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.638+0000 7ff2e0ca2700 1 -- 172.21.15.143:0/760043466 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2cc0094a0 con 0x7ff2dc154ab0 2023-11-08T19:26:21.786 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.638+0000 7ff2db7fe700 1 --2- 172.21.15.143:0/760043466 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff2dc15a950 0x7ff2dc154570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:26:21.786 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.638+0000 7ff2e0ca2700 1 --2- 172.21.15.143:0/760043466 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff2dc154ab0 0x7ff2dc0d0790 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7ff2d400cc30 tx=0x7ff2d400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:21.786 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.638+0000 7ff2d8ff9700 1 -- 172.21.15.143:0/760043466 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2d400ac20 con 0x7ff2dc154ab0 2023-11-08T19:26:21.786 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.638+0000 7ff2d8ff9700 1 -- 172.21.15.143:0/760043466 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2d400ad80 con 0x7ff2dc154ab0 2023-11-08T19:26:21.787 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.638+0000 7ff2e2705700 1 -- 172.21.15.143:0/760043466 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2dc1555b0 con 0x7ff2dc154ab0 2023-11-08T19:26:21.787 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.638+0000 7ff2d8ff9700 1 -- 172.21.15.143:0/760043466 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2d4012660 con 0x7ff2dc154ab0 2023-11-08T19:26:21.787 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.638+0000 7ff2e2705700 1 -- 172.21.15.143:0/760043466 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2dc155b70 con 0x7ff2dc154ab0 2023-11-08T19:26:21.787 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.639+0000 7ff2e2705700 1 -- 172.21.15.143:0/760043466 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2dc04eee0 con 0x7ff2dc154ab0 2023-11-08T19:26:21.787 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.640+0000 7ff2d8ff9700 1 -- 172.21.15.143:0/760043466 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff2d40127c0 con 0x7ff2dc154ab0 2023-11-08T19:26:21.788 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.640+0000 7ff2d8ff9700 1 --2- 172.21.15.143:0/760043466 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7ff2c806af00 0x7ff2c806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:21.788 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.640+0000 7ff2d8ff9700 1 -- 172.21.15.143:0/760043466 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5210+0+0 (secure 0 0 0) 0x7ff2d40895f0 con 0x7ff2dc154ab0 2023-11-08T19:26:21.788 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.640+0000 7ff2dbfff700 1 --2- 172.21.15.143:0/760043466 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7ff2c806af00 0x7ff2c806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:21.788 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.641+0000 7ff2dbfff700 1 --2- 172.21.15.143:0/760043466 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7ff2c806af00 0x7ff2c806d3c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc0090b0 tx=0x7ff2cc008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:21.788 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.645+0000 7ff2d8ff9700 1 -- 172.21.15.143:0/760043466 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff2d4054890 con 0x7ff2dc154ab0 2023-11-08T19:26:21.835 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.834+0000 7ff2e2705700 1 -- 172.21.15.143:0/760043466 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff2dc072150 con 0x7ff2dc154ab0 2023-11-08T19:26:21.836 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.834+0000 7ff2d8ff9700 1 -- 172.21.15.143:0/760043466 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v44) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff2d4016020 con 0x7ff2dc154ab0 2023-11-08T19:26:21.836 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2023-11-08T19:26:21.838 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.836+0000 7ff2e2705700 1 -- 172.21.15.143:0/760043466 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7ff2c806af00 msgr2=0x7ff2c806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:21.838 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.836+0000 7ff2e2705700 1 --2- 172.21.15.143:0/760043466 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7ff2c806af00 0x7ff2c806d3c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc0090b0 tx=0x7ff2cc008030 comp rx=0 tx=0).stop 2023-11-08T19:26:21.838 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.836+0000 7ff2e2705700 1 -- 172.21.15.143:0/760043466 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff2dc154ab0 msgr2=0x7ff2dc0d0790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:21.839 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.836+0000 7ff2e2705700 1 --2- 172.21.15.143:0/760043466 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff2dc154ab0 0x7ff2dc0d0790 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7ff2d400cc30 tx=0x7ff2d400a060 comp rx=0 tx=0).stop 2023-11-08T19:26:21.839 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.836+0000 7ff2e2705700 1 -- 172.21.15.143:0/760043466 shutdown_connections 2023-11-08T19:26:21.839 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.836+0000 7ff2e2705700 1 --2- 172.21.15.143:0/760043466 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7ff2c806af00 0x7ff2c806d3c0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:21.839 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.836+0000 7ff2e2705700 1 --2- 172.21.15.143:0/760043466 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff2dc0b61e0 0x7ff2dc153b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:21.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.836+0000 7ff2e2705700 1 --2- 172.21.15.143:0/760043466 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff2dc154ab0 0x7ff2dc0d0790 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:21.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.836+0000 7ff2e2705700 1 --2- 172.21.15.143:0/760043466 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff2dc15a950 0x7ff2dc154570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:21.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.836+0000 7ff2e2705700 1 -- 172.21.15.143:0/760043466 >> 172.21.15.143:0/760043466 conn(0x7ff2dc0cd2e0 msgr2=0x7ff2dc15c470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:21.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.836+0000 7ff2e2705700 1 -- 172.21.15.143:0/760043466 shutdown_connections 2023-11-08T19:26:21.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.836+0000 7ff2e2705700 1 -- 172.21.15.143:0/760043466 wait complete. 2023-11-08T19:26:21.851 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get snap-test-pool pg_num 2023-11-08T19:26:21.975 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.973+0000 7f17ff437700 1 Processor -- start 2023-11-08T19:26:21.976 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.974+0000 7f17ff437700 1 -- start start 2023-11-08T19:26:21.982 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.974+0000 7f17ff437700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f17f81610b0 0x7f17f80cf560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:21.982 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.974+0000 7f17ff437700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f17f80ceab0 0x7f17f81622a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:21.982 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.974+0000 7f17ff437700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17f81627e0 con 0x7f17f80cc120 2023-11-08T19:26:21.983 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.974+0000 7f17ff437700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17f80d0350 con 0x7f17f81610b0 2023-11-08T19:26:21.983 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.974+0000 7f17ff437700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17f80cfb40 con 0x7f17f80ceab0 2023-11-08T19:26:21.983 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.975+0000 7f17fc9d2700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f17f80cc120 0x7f17f80ce3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:47614/0 (socket says 172.21.15.143:47614) 2023-11-08T19:26:21.983 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.975+0000 7f17fc9d2700 1 -- 172.21.15.143:0/1808521838 learned_addr learned my addr 172.21.15.143:0/1808521838 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:26:21.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.975+0000 7f17fd9d4700 1 --2- 172.21.15.143:0/1808521838 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f17f80ceab0 0x7f17f81622a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:21.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.975+0000 7f17fd1d3700 1 --2- 172.21.15.143:0/1808521838 >> v2:172.21.15.174:3300/0 conn(0x7f17f81610b0 0x7f17f80cf560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:21.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.975+0000 7f17effff700 1 -- 172.21.15.143:0/1808521838 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 571126644 0 0) 0x7f17f4001110 con 0x7f17f80cc120 2023-11-08T19:26:21.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.975+0000 7f17effff700 1 -- 172.21.15.143:0/1808521838 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f17e40035b0 con 0x7f17f80cc120 2023-11-08T19:26:21.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.975+0000 7f17effff700 1 -- 172.21.15.143:0/1808521838 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4032889141 0 0) 0x7f17e40035b0 con 0x7f17f80cc120 2023-11-08T19:26:21.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.975+0000 7f17effff700 1 -- 172.21.15.143:0/1808521838 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f17f80ceab0 msgr2=0x7f17f81622a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:21.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.975+0000 7f17effff700 1 --2- 172.21.15.143:0/1808521838 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f17f80ceab0 0x7f17f81622a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:21.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.975+0000 7f17effff700 1 -- 172.21.15.143:0/1808521838 >> v2:172.21.15.174:3300/0 conn(0x7f17f81610b0 msgr2=0x7f17f80cf560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:21.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.975+0000 7f17effff700 1 --2- 172.21.15.143:0/1808521838 >> v2:172.21.15.174:3300/0 conn(0x7f17f81610b0 0x7f17f80cf560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:21.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.975+0000 7f17effff700 1 -- 172.21.15.143:0/1808521838 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17f806c3c0 con 0x7f17f80cc120 2023-11-08T19:26:21.986 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.975+0000 7f17effff700 1 -- 172.21.15.143:0/1808521838 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4147756303 0 0) 0x7f17f4001800 con 0x7f17f80cc120 2023-11-08T19:26:21.986 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.975+0000 7f17effff700 1 -- 172.21.15.143:0/1808521838 >> v1:172.21.15.143:6789/0 conn(0x7f17f80cc120 legacy=0x7f17f80ce3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:21.986 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.975+0000 7f17effff700 1 --2- 172.21.15.143:0/1808521838 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f17e4003a00 0x7f17e4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:21.986 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.975+0000 7f17effff700 1 --2- 172.21.15.143:0/1808521838 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f17e4004850 0x7f17e4004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:21.987 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.975+0000 7f17effff700 1 --2- 172.21.15.143:0/1808521838 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f17f80ceab0 0x7f17e4005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:21.987 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.975+0000 7f17effff700 1 -- 172.21.15.143:0/1808521838 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17f80d0350 con 0x7f17e4003a00 2023-11-08T19:26:21.987 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.975+0000 7f17effff700 1 -- 172.21.15.143:0/1808521838 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17f80cfb40 con 0x7f17e4004850 2023-11-08T19:26:21.987 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.975+0000 7f17effff700 1 -- 172.21.15.143:0/1808521838 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17e400ba10 con 0x7f17f80ceab0 2023-11-08T19:26:21.987 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.975+0000 7f17fd1d3700 1 --2- 172.21.15.143:0/1808521838 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f17e4003a00 0x7f17e4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:21.988 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.975+0000 7f17fd9d4700 1 --2- 172.21.15.143:0/1808521838 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f17f80ceab0 0x7f17e4005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:21.988 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.975+0000 7f17fc9d2700 1 --2- 172.21.15.143:0/1808521838 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f17e4004850 0x7f17e4004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:21.988 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.976+0000 7f17fd1d3700 1 -- 172.21.15.143:0/1808521838 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f17f80ceab0 msgr2=0x7f17e4005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:21.988 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.976+0000 7f17fd1d3700 1 --2- 172.21.15.143:0/1808521838 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f17f80ceab0 0x7f17e4005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:21.988 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.976+0000 7f17fd1d3700 1 -- 172.21.15.143:0/1808521838 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f17e4004850 msgr2=0x7f17e4004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:21.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.976+0000 7f17fd1d3700 1 --2- 172.21.15.143:0/1808521838 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f17e4004850 0x7f17e4004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:21.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.976+0000 7f17fd1d3700 1 -- 172.21.15.143:0/1808521838 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17e40035b0 con 0x7f17e4003a00 2023-11-08T19:26:21.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.976+0000 7f17fd1d3700 1 --2- 172.21.15.143:0/1808521838 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f17e4003a00 0x7f17e4003e20 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f17e8009b60 tx=0x7f17e4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb6ae1035e0da98a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:21.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.976+0000 7f17effff700 1 -- 172.21.15.143:0/1808521838 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17e800e6c0 con 0x7f17e4003a00 2023-11-08T19:26:21.990 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.976+0000 7f17effff700 1 -- 172.21.15.143:0/1808521838 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17e800e820 con 0x7f17e4003a00 2023-11-08T19:26:21.990 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.976+0000 7f17effff700 1 -- 172.21.15.143:0/1808521838 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17e800eb90 con 0x7f17e4003a00 2023-11-08T19:26:21.990 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.976+0000 7f17ff437700 1 -- 172.21.15.143:0/1808521838 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f17e4003a00 msgr2=0x7f17e4003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:21.990 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.976+0000 7f17ff437700 1 --2- 172.21.15.143:0/1808521838 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f17e4003a00 0x7f17e4003e20 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f17e8009b60 tx=0x7f17e4005880 comp rx=0 tx=0).stop 2023-11-08T19:26:21.990 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.976+0000 7f17fd9d4700 1 -- 172.21.15.143:0/1808521838 reap_dead start 2023-11-08T19:26:21.991 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.976+0000 7f17ff437700 1 -- 172.21.15.143:0/1808521838 shutdown_connections 2023-11-08T19:26:21.991 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.976+0000 7f17ff437700 1 -- 172.21.15.143:0/1808521838 >> 172.21.15.143:0/1808521838 conn(0x7f17f806f2e0 msgr2=0x7f17f806f6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:21.991 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.977+0000 7f17ff437700 1 -- 172.21.15.143:0/1808521838 shutdown_connections 2023-11-08T19:26:21.991 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.977+0000 7f17ff437700 1 -- 172.21.15.143:0/1808521838 wait complete. 2023-11-08T19:26:21.992 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.978+0000 7f17ff437700 1 Processor -- start 2023-11-08T19:26:21.992 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.978+0000 7f17ff437700 1 -- start start 2023-11-08T19:26:21.992 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.979+0000 7f17ff437700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f17e4003a00 0x7f17f80b2c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:21.992 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.979+0000 7f17ff437700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f17f8159640 0x7f17f8145530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:21.993 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.979+0000 7f17ff437700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f17f8144bf0 0x7f17f815edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:21.993 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.979+0000 7f17ff437700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17f8145c10 con 0x7f17f8159640 2023-11-08T19:26:21.993 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.979+0000 7f17ff437700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17f81475c0 con 0x7f17e4003a00 2023-11-08T19:26:21.993 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.979+0000 7f17ff437700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17f80b32b0 con 0x7f17f8144bf0 2023-11-08T19:26:21.993 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.979+0000 7f17fc9d2700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f17f8159640 0x7f17f8145530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:21.994 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.979+0000 7f17fc9d2700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f17f8159640 0x7f17f8145530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51474/0 (socket says 172.21.15.143:51474) 2023-11-08T19:26:21.994 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.979+0000 7f17fc9d2700 1 -- 172.21.15.143:0/3312902548 learned_addr learned my addr 172.21.15.143:0/3312902548 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:21.994 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.979+0000 7f17fd9d4700 1 --2- 172.21.15.143:0/3312902548 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f17f8144bf0 0x7f17f815edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:21.994 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.979+0000 7f17fd1d3700 1 --2- 172.21.15.143:0/3312902548 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f17e4003a00 0x7f17f80b2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:21.995 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.979+0000 7f17fc9d2700 1 -- 172.21.15.143:0/3312902548 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f17f8144bf0 msgr2=0x7f17f815edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:21.995 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.979+0000 7f17fc9d2700 1 --2- 172.21.15.143:0/3312902548 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f17f8144bf0 0x7f17f815edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:21.995 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.979+0000 7f17fc9d2700 1 -- 172.21.15.143:0/3312902548 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f17e4003a00 msgr2=0x7f17f80b2c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:21.995 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.979+0000 7f17fc9d2700 1 --2- 172.21.15.143:0/3312902548 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f17e4003a00 0x7f17f80b2c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:21.996 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.979+0000 7f17fc9d2700 1 -- 172.21.15.143:0/3312902548 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17f80b3590 con 0x7f17f8159640 2023-11-08T19:26:21.996 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.980+0000 7f17fc9d2700 1 --2- 172.21.15.143:0/3312902548 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f17f8159640 0x7f17f8145530 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f17f4001c30 tx=0x7f17f40075f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:21.996 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.980+0000 7f17edffb700 1 -- 172.21.15.143:0/3312902548 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17f400f6c0 con 0x7f17f8159640 2023-11-08T19:26:21.996 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.980+0000 7f17edffb700 1 -- 172.21.15.143:0/3312902548 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17f400f860 con 0x7f17f8159640 2023-11-08T19:26:21.997 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.980+0000 7f17ff437700 1 -- 172.21.15.143:0/3312902548 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17f80b3870 con 0x7f17f8159640 2023-11-08T19:26:21.997 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.980+0000 7f17edffb700 1 -- 172.21.15.143:0/3312902548 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17f400fc10 con 0x7f17f8159640 2023-11-08T19:26:21.997 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.980+0000 7f17ff437700 1 -- 172.21.15.143:0/3312902548 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17f8159db0 con 0x7f17f8159640 2023-11-08T19:26:21.999 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.981+0000 7f17ff437700 1 -- 172.21.15.143:0/3312902548 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17f81477e0 con 0x7f17f8159640 2023-11-08T19:26:21.999 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.981+0000 7f17edffb700 1 -- 172.21.15.143:0/3312902548 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f17f400fd70 con 0x7f17f8159640 2023-11-08T19:26:21.999 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.981+0000 7f17edffb700 1 --2- 172.21.15.143:0/3312902548 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f17dc06af00 0x7f17dc06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:21.999 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.982+0000 7f17edffb700 1 -- 172.21.15.143:0/3312902548 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f17f40892b0 con 0x7f17f8159640 2023-11-08T19:26:22.000 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.982+0000 7f17fd1d3700 1 --2- 172.21.15.143:0/3312902548 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f17dc06af00 0x7f17dc06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:22.000 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.982+0000 7f17fd1d3700 1 --2- 172.21.15.143:0/3312902548 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f17dc06af00 0x7f17dc06d3c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f17e800a010 tx=0x7f17e8008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:22.000 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:21.986+0000 7f17edffb700 1 -- 172.21.15.143:0/3312902548 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f17f81477e0 con 0x7f17f8159640 2023-11-08T19:26:22.176 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.175+0000 7f17ff437700 1 -- 172.21.15.143:0/3312902548 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"} v 0) v1 -- 0x7f17f81627e0 con 0x7f17f8159640 2023-11-08T19:26:22.177 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.175+0000 7f17edffb700 1 -- 172.21.15.143:0/3312902548 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"}]=0 v44) v1 ==== 103+0+11 (secure 0 0 0) 0x7f17f81627e0 con 0x7f17f8159640 2023-11-08T19:26:22.177 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2023-11-08T19:26:22.179 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.177+0000 7f17ff437700 1 -- 172.21.15.143:0/3312902548 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f17dc06af00 msgr2=0x7f17dc06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:22.179 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.177+0000 7f17ff437700 1 --2- 172.21.15.143:0/3312902548 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f17dc06af00 0x7f17dc06d3c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f17e800a010 tx=0x7f17e8008030 comp rx=0 tx=0).stop 2023-11-08T19:26:22.180 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.177+0000 7f17ff437700 1 -- 172.21.15.143:0/3312902548 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f17f8159640 msgr2=0x7f17f8145530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:22.180 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.177+0000 7f17ff437700 1 --2- 172.21.15.143:0/3312902548 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f17f8159640 0x7f17f8145530 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f17f4001c30 tx=0x7f17f40075f0 comp rx=0 tx=0).stop 2023-11-08T19:26:22.180 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.177+0000 7f17ff437700 1 -- 172.21.15.143:0/3312902548 shutdown_connections 2023-11-08T19:26:22.180 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.177+0000 7f17ff437700 1 --2- 172.21.15.143:0/3312902548 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f17dc06af00 0x7f17dc06d3c0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:22.180 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.177+0000 7f17ff437700 1 --2- 172.21.15.143:0/3312902548 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f17f8144bf0 0x7f17f815edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:22.181 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.178+0000 7f17ff437700 1 --2- 172.21.15.143:0/3312902548 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f17e4003a00 0x7f17f80b2c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:22.181 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.178+0000 7f17ff437700 1 --2- 172.21.15.143:0/3312902548 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f17f8159640 0x7f17f8145530 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:22.181 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.178+0000 7f17ff437700 1 -- 172.21.15.143:0/3312902548 >> 172.21.15.143:0/3312902548 conn(0x7f17f806f2e0 msgr2=0x7f17f8160150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:22.181 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.178+0000 7f17ff437700 1 -- 172.21.15.143:0/3312902548 shutdown_connections 2023-11-08T19:26:22.181 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.178+0000 7f17ff437700 1 -- 172.21.15.143:0/3312902548 wait complete. 2023-11-08T19:26:22.191 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-08T19:26:22.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.310+0000 7f8ed5677700 1 Processor -- start 2023-11-08T19:26:22.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.310+0000 7f8ed5677700 1 -- start start 2023-11-08T19:26:22.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.310+0000 7f8ed5677700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f8ed01585e0 0x7f8ed006ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:22.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.310+0000 7f8ed5677700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8ed00b2510 0x7f8ed00cec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:22.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.310+0000 7f8ed5677700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f8ed015e530 0x7f8ed00cf150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:22.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.310+0000 7f8ed5677700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ed015bf30 con 0x7f8ed01585e0 2023-11-08T19:26:22.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.310+0000 7f8ed5677700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ed01589c0 con 0x7f8ed015e530 2023-11-08T19:26:22.315 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.310+0000 7f8ed5677700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed00cf9f0 con 0x7f8ed00b2510 2023-11-08T19:26:22.315 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.310+0000 7f8eceffd700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f8ed01585e0 0x7f8ed006ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:22.315 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.311+0000 7f8eceffd700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f8ed01585e0 0x7f8ed006ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51480/0 (socket says 172.21.15.143:51480) 2023-11-08T19:26:22.315 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.311+0000 7f8eceffd700 1 -- 172.21.15.143:0/4250772641 learned_addr learned my addr 172.21.15.143:0/4250772641 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:22.315 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.311+0000 7f8ecf7fe700 1 --2- 172.21.15.143:0/4250772641 >> v2:172.21.15.174:3300/0 conn(0x7f8ed015e530 0x7f8ed00cf150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:22.316 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.311+0000 7f8ece7fc700 1 --2- 172.21.15.143:0/4250772641 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8ed00b2510 0x7f8ed00cec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:22.316 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.311+0000 7f8eceffd700 1 -- 172.21.15.143:0/4250772641 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8ed00b2510 msgr2=0x7f8ed00cec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:22.316 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.311+0000 7f8eceffd700 1 --2- 172.21.15.143:0/4250772641 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8ed00b2510 0x7f8ed00cec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:22.316 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.311+0000 7f8eceffd700 1 -- 172.21.15.143:0/4250772641 >> v2:172.21.15.174:3300/0 conn(0x7f8ed015e530 msgr2=0x7f8ed00cf150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:22.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.311+0000 7f8eceffd700 1 --2- 172.21.15.143:0/4250772641 >> v2:172.21.15.174:3300/0 conn(0x7f8ed015e530 0x7f8ed00cf150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:22.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.311+0000 7f8eceffd700 1 -- 172.21.15.143:0/4250772641 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ed015b320 con 0x7f8ed01585e0 2023-11-08T19:26:22.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.311+0000 7f8eceffd700 1 --2- 172.21.15.143:0/4250772641 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8ed01585e0 0x7f8ed006ac10 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f8ec40098f0 tx=0x7f8ec4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18cd3bc9b0ec017c server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:22.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.311+0000 7f8ecdffb700 1 -- 172.21.15.143:0/4250772641 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ec400b860 con 0x7f8ed01585e0 2023-11-08T19:26:22.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.311+0000 7f8ecdffb700 1 -- 172.21.15.143:0/4250772641 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ec400b9c0 con 0x7f8ed01585e0 2023-11-08T19:26:22.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.311+0000 7f8ecdffb700 1 -- 172.21.15.143:0/4250772641 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ec400bd50 con 0x7f8ed01585e0 2023-11-08T19:26:22.318 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.312+0000 7f8ed5677700 1 -- 172.21.15.143:0/4250772641 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8ed01585e0 msgr2=0x7f8ed006ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:22.318 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.312+0000 7f8ed5677700 1 --2- 172.21.15.143:0/4250772641 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8ed01585e0 0x7f8ed006ac10 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f8ec40098f0 tx=0x7f8ec4009c00 comp rx=0 tx=0).stop 2023-11-08T19:26:22.318 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.312+0000 7f8ed5677700 1 -- 172.21.15.143:0/4250772641 shutdown_connections 2023-11-08T19:26:22.318 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.312+0000 7f8ed5677700 1 --2- 172.21.15.143:0/4250772641 >> v2:172.21.15.174:3300/0 conn(0x7f8ed015e530 0x7f8ed00cf150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:22.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.312+0000 7f8ed5677700 1 --2- 172.21.15.143:0/4250772641 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8ed01585e0 0x7f8ed006ac10 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:22.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.312+0000 7f8ed5677700 1 --2- 172.21.15.143:0/4250772641 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8ed00b2510 0x7f8ed00cec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:22.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.312+0000 7f8ed5677700 1 -- 172.21.15.143:0/4250772641 >> 172.21.15.143:0/4250772641 conn(0x7f8ed00b1a20 msgr2=0x7f8ed00b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:22.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.312+0000 7f8ed5677700 1 -- 172.21.15.143:0/4250772641 shutdown_connections 2023-11-08T19:26:22.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.312+0000 7f8ed5677700 1 -- 172.21.15.143:0/4250772641 wait complete. 2023-11-08T19:26:22.323 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.313+0000 7f8ed5677700 1 Processor -- start 2023-11-08T19:26:22.323 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.313+0000 7f8ed5677700 1 -- start start 2023-11-08T19:26:22.323 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.314+0000 7f8ed5677700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8ed00b2510 0x7f8ed015a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:22.323 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.314+0000 7f8ed5677700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8ed01585e0 0x7f8ed00c5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:22.324 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.314+0000 7f8ed5677700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8ed015e530 0x7f8ed00b5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:22.324 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.314+0000 7f8ed5677700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed00c6700 con 0x7f8ed015e530 2023-11-08T19:26:22.324 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.314+0000 7f8ed5677700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed00b4540 con 0x7f8ed00b2510 2023-11-08T19:26:22.324 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.314+0000 7f8ed5677700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed00b4820 con 0x7f8ed01585e0 2023-11-08T19:26:22.325 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.314+0000 7f8ecf7fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8ed015e530 0x7f8ed00b5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:22.325 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.314+0000 7f8ecf7fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8ed015e530 0x7f8ed00b5020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51488/0 (socket says 172.21.15.143:51488) 2023-11-08T19:26:22.325 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.314+0000 7f8ecf7fe700 1 -- 172.21.15.143:0/3680403312 learned_addr learned my addr 172.21.15.143:0/3680403312 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:22.325 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.314+0000 7f8eceffd700 1 --2- 172.21.15.143:0/3680403312 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8ed00b2510 0x7f8ed015a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:22.326 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.314+0000 7f8ece7fc700 1 --2- 172.21.15.143:0/3680403312 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8ed01585e0 0x7f8ed00c5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:22.326 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.314+0000 7f8ecf7fe700 1 -- 172.21.15.143:0/3680403312 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8ed01585e0 msgr2=0x7f8ed00c5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:22.326 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.314+0000 7f8ecf7fe700 1 --2- 172.21.15.143:0/3680403312 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8ed01585e0 0x7f8ed00c5c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:22.326 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.314+0000 7f8ecf7fe700 1 -- 172.21.15.143:0/3680403312 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8ed00b2510 msgr2=0x7f8ed015a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:22.326 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.314+0000 7f8ecf7fe700 1 --2- 172.21.15.143:0/3680403312 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8ed00b2510 0x7f8ed015a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:22.327 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.314+0000 7f8ecf7fe700 1 -- 172.21.15.143:0/3680403312 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ec4009550 con 0x7f8ed015e530 2023-11-08T19:26:22.327 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.314+0000 7f8ecf7fe700 1 --2- 172.21.15.143:0/3680403312 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8ed015e530 0x7f8ed00b5020 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f8ec8009fd0 tx=0x7f8ec800e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:22.327 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.315+0000 7f8eb77fe700 1 -- 172.21.15.143:0/3680403312 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ec8007ea0 con 0x7f8ed015e530 2023-11-08T19:26:22.327 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.315+0000 7f8eb77fe700 1 -- 172.21.15.143:0/3680403312 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ec800ed30 con 0x7f8ed015e530 2023-11-08T19:26:22.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.315+0000 7f8ed5677700 1 -- 172.21.15.143:0/3680403312 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ed00b4b00 con 0x7f8ed015e530 2023-11-08T19:26:22.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.315+0000 7f8eb77fe700 1 -- 172.21.15.143:0/3680403312 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ec800eea0 con 0x7f8ed015e530 2023-11-08T19:26:22.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.315+0000 7f8ed5677700 1 -- 172.21.15.143:0/3680403312 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ed01f4a40 con 0x7f8ed015e530 2023-11-08T19:26:22.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.315+0000 7f8ed5677700 1 -- 172.21.15.143:0/3680403312 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ed004eee0 con 0x7f8ed015e530 2023-11-08T19:26:22.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.316+0000 7f8eb77fe700 1 -- 172.21.15.143:0/3680403312 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8ec8014660 con 0x7f8ed015e530 2023-11-08T19:26:22.329 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.316+0000 7f8eb77fe700 1 --2- 172.21.15.143:0/3680403312 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f8ebc06af00 0x7f8ebc06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:22.329 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.316+0000 7f8eb77fe700 1 -- 172.21.15.143:0/3680403312 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f8ec80899f0 con 0x7f8ed015e530 2023-11-08T19:26:22.329 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.317+0000 7f8eceffd700 1 --2- 172.21.15.143:0/3680403312 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f8ebc06af00 0x7f8ebc06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:22.329 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.317+0000 7f8eceffd700 1 --2- 172.21.15.143:0/3680403312 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f8ebc06af00 0x7f8ebc06d3c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f8ec4009c40 tx=0x7f8ec4008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:22.329 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.321+0000 7f8eb77fe700 1 -- 172.21.15.143:0/3680403312 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8ec8054ca0 con 0x7f8ed015e530 2023-11-08T19:26:22.511 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.510+0000 7f8ed5677700 1 -- 172.21.15.143:0/3680403312 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f8ed006d430 con 0x7f8ed015e530 2023-11-08T19:26:22.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.510+0000 7f8eb77fe700 1 -- 172.21.15.143:0/3680403312 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v44) v1 ==== 84+0+137 (secure 0 0 0) 0x7f8ec80582c0 con 0x7f8ed015e530 2023-11-08T19:26:22.512 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:26:22.512 INFO:teuthology.orchestra.run.smithi143.stdout:[ 2023-11-08T19:26:22.512 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-08T19:26:22.513 INFO:teuthology.orchestra.run.smithi143.stdout: "poolnum": 1, 2023-11-08T19:26:22.513 INFO:teuthology.orchestra.run.smithi143.stdout: "poolname": ".mgr" 2023-11-08T19:26:22.513 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-08T19:26:22.513 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-08T19:26:22.513 INFO:teuthology.orchestra.run.smithi143.stdout: "poolnum": 6, 2023-11-08T19:26:22.514 INFO:teuthology.orchestra.run.smithi143.stdout: "poolname": "snap-test-pool" 2023-11-08T19:26:22.514 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-08T19:26:22.514 INFO:teuthology.orchestra.run.smithi143.stdout:] 2023-11-08T19:26:22.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.512+0000 7f8ed5677700 1 -- 172.21.15.143:0/3680403312 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f8ebc06af00 msgr2=0x7f8ebc06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:22.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.512+0000 7f8ed5677700 1 --2- 172.21.15.143:0/3680403312 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f8ebc06af00 0x7f8ebc06d3c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f8ec4009c40 tx=0x7f8ec4008030 comp rx=0 tx=0).stop 2023-11-08T19:26:22.516 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.512+0000 7f8ed5677700 1 -- 172.21.15.143:0/3680403312 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8ed015e530 msgr2=0x7f8ed00b5020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:22.516 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.512+0000 7f8ed5677700 1 --2- 172.21.15.143:0/3680403312 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8ed015e530 0x7f8ed00b5020 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f8ec8009fd0 tx=0x7f8ec800e3c0 comp rx=0 tx=0).stop 2023-11-08T19:26:22.516 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.512+0000 7f8ed5677700 1 -- 172.21.15.143:0/3680403312 shutdown_connections 2023-11-08T19:26:22.516 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.512+0000 7f8ed5677700 1 --2- 172.21.15.143:0/3680403312 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f8ebc06af00 0x7f8ebc06d3c0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:22.516 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.512+0000 7f8ed5677700 1 --2- 172.21.15.143:0/3680403312 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8ed015e530 0x7f8ed00b5020 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:22.517 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.512+0000 7f8ed5677700 1 --2- 172.21.15.143:0/3680403312 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8ed00b2510 0x7f8ed015a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:22.517 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.512+0000 7f8ed5677700 1 --2- 172.21.15.143:0/3680403312 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8ed01585e0 0x7f8ed00c5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:22.517 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.512+0000 7f8ed5677700 1 -- 172.21.15.143:0/3680403312 >> 172.21.15.143:0/3680403312 conn(0x7f8ed00b1a20 msgr2=0x7f8ed00afe00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:22.517 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.512+0000 7f8ed5677700 1 -- 172.21.15.143:0/3680403312 shutdown_connections 2023-11-08T19:26:22.517 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.512+0000 7f8ed5677700 1 -- 172.21.15.143:0/3680403312 wait complete. 2023-11-08T19:26:22.526 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-08T19:26:22.526 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-08T19:26:22.651 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.649+0000 7fda4ecb3700 1 Processor -- start 2023-11-08T19:26:22.651 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.649+0000 7fda4ecb3700 1 -- start start 2023-11-08T19:26:22.654 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.650+0000 7fda4ecb3700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fda4815dba0 0x7fda4806c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:22.654 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.650+0000 7fda4ecb3700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fda4806b240 0x7fda480caff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:22.654 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.650+0000 7fda4ecb3700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fda480cb530 0x7fda480b22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:22.655 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.650+0000 7fda4ecb3700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda48161890 con 0x7fda4806b240 2023-11-08T19:26:22.655 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.650+0000 7fda4ecb3700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda4806b740 con 0x7fda480cb530 2023-11-08T19:26:22.655 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.650+0000 7fda4ecb3700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda4806ba20 con 0x7fda4815dba0 2023-11-08T19:26:22.655 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.650+0000 7fda47fff700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fda4806b240 0x7fda480caff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:22.655 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.650+0000 7fda47fff700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fda4806b240 0x7fda480caff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51492/0 (socket says 172.21.15.143:51492) 2023-11-08T19:26:22.656 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.650+0000 7fda47fff700 1 -- 172.21.15.143:0/3190666805 learned_addr learned my addr 172.21.15.143:0/3190666805 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:22.656 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.650+0000 7fda47fff700 1 -- 172.21.15.143:0/3190666805 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fda4815dba0 msgr2=0x7fda4806c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:22.656 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.650+0000 7fda4d250700 1 --2- 172.21.15.143:0/3190666805 >> v2:172.21.15.174:3300/0 conn(0x7fda480cb530 0x7fda480b22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:22.656 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.650+0000 7fda4ca4f700 1 --2- 172.21.15.143:0/3190666805 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fda4815dba0 0x7fda4806c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:22.656 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.651+0000 7fda47fff700 1 --2- 172.21.15.143:0/3190666805 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fda4815dba0 0x7fda4806c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:22.657 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.651+0000 7fda47fff700 1 -- 172.21.15.143:0/3190666805 >> v2:172.21.15.174:3300/0 conn(0x7fda480cb530 msgr2=0x7fda480b22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:22.657 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.651+0000 7fda47fff700 1 --2- 172.21.15.143:0/3190666805 >> v2:172.21.15.174:3300/0 conn(0x7fda480cb530 0x7fda480b22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:22.657 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.651+0000 7fda47fff700 1 -- 172.21.15.143:0/3190666805 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda4806bd00 con 0x7fda4806b240 2023-11-08T19:26:22.657 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.651+0000 7fda4d250700 1 --2- 172.21.15.143:0/3190666805 >> v2:172.21.15.174:3300/0 conn(0x7fda480cb530 0x7fda480b22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:26:22.658 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.651+0000 7fda47fff700 1 --2- 172.21.15.143:0/3190666805 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fda4806b240 0x7fda480caff0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fda4806f460 tx=0x7fda34009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6581014a5b1db3e server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:22.658 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.651+0000 7fda477fe700 1 -- 172.21.15.143:0/3190666805 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda3400b870 con 0x7fda4806b240 2023-11-08T19:26:22.658 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.651+0000 7fda477fe700 1 -- 172.21.15.143:0/3190666805 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda3400b9d0 con 0x7fda4806b240 2023-11-08T19:26:22.658 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.651+0000 7fda477fe700 1 -- 172.21.15.143:0/3190666805 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda3400bd60 con 0x7fda4806b240 2023-11-08T19:26:22.659 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.651+0000 7fda4ecb3700 1 -- 172.21.15.143:0/3190666805 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fda4806b240 msgr2=0x7fda480caff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:22.659 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.651+0000 7fda4ecb3700 1 --2- 172.21.15.143:0/3190666805 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fda4806b240 0x7fda480caff0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fda4806f460 tx=0x7fda34009bd0 comp rx=0 tx=0).stop 2023-11-08T19:26:22.659 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.651+0000 7fda4ecb3700 1 -- 172.21.15.143:0/3190666805 shutdown_connections 2023-11-08T19:26:22.659 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.651+0000 7fda4ecb3700 1 --2- 172.21.15.143:0/3190666805 >> v2:172.21.15.174:3300/0 conn(0x7fda480cb530 0x7fda480b22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:22.660 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.651+0000 7fda4ecb3700 1 --2- 172.21.15.143:0/3190666805 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fda4815dba0 0x7fda4806c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:22.660 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.651+0000 7fda4ecb3700 1 --2- 172.21.15.143:0/3190666805 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fda4806b240 0x7fda480caff0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:22.660 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.651+0000 7fda4ecb3700 1 -- 172.21.15.143:0/3190666805 >> 172.21.15.143:0/3190666805 conn(0x7fda480cd2e0 msgr2=0x7fda480cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:22.660 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.651+0000 7fda4ecb3700 1 -- 172.21.15.143:0/3190666805 shutdown_connections 2023-11-08T19:26:22.660 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.652+0000 7fda4ecb3700 1 -- 172.21.15.143:0/3190666805 wait complete. 2023-11-08T19:26:22.661 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.653+0000 7fda4ecb3700 1 Processor -- start 2023-11-08T19:26:22.661 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.653+0000 7fda4ecb3700 1 -- start start 2023-11-08T19:26:22.664 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.654+0000 7fda4ecb3700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fda4806b240 0x7fda480ca880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:22.665 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.654+0000 7fda4ecb3700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fda480cb530 0x7fda480c82c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:22.665 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.654+0000 7fda4ecb3700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fda4815dba0 0x7fda481e7180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:22.665 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.654+0000 7fda4ecb3700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda480cadc0 con 0x7fda4806b240 2023-11-08T19:26:22.665 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.654+0000 7fda4ecb3700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda4814c9c0 con 0x7fda4815dba0 2023-11-08T19:26:22.665 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.654+0000 7fda4ecb3700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda4814d7e0 con 0x7fda480cb530 2023-11-08T19:26:22.666 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.654+0000 7fda4ca4f700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fda4806b240 0x7fda480ca880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:22.666 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.654+0000 7fda4ca4f700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fda4806b240 0x7fda480ca880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51498/0 (socket says 172.21.15.143:51498) 2023-11-08T19:26:22.666 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.654+0000 7fda4ca4f700 1 -- 172.21.15.143:0/2403758129 learned_addr learned my addr 172.21.15.143:0/2403758129 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:22.666 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.654+0000 7fda4d250700 1 --2- 172.21.15.143:0/2403758129 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fda4815dba0 0x7fda481e7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:22.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.654+0000 7fda47fff700 1 --2- 172.21.15.143:0/2403758129 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fda480cb530 0x7fda480c82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:22.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.654+0000 7fda4ca4f700 1 -- 172.21.15.143:0/2403758129 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fda480cb530 msgr2=0x7fda480c82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:22.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.654+0000 7fda4ca4f700 1 --2- 172.21.15.143:0/2403758129 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fda480cb530 0x7fda480c82c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:22.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.654+0000 7fda4ca4f700 1 -- 172.21.15.143:0/2403758129 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fda4815dba0 msgr2=0x7fda481e7180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:22.668 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.654+0000 7fda4ca4f700 1 --2- 172.21.15.143:0/2403758129 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fda4815dba0 0x7fda481e7180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:22.668 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.654+0000 7fda4ca4f700 1 -- 172.21.15.143:0/2403758129 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda34009550 con 0x7fda4806b240 2023-11-08T19:26:22.668 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.654+0000 7fda4ca4f700 1 --2- 172.21.15.143:0/2403758129 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fda4806b240 0x7fda480ca880 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fda3c00c7f0 tx=0x7fda3c00cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:22.668 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.654+0000 7fda457fa700 1 -- 172.21.15.143:0/2403758129 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda3c011800 con 0x7fda4806b240 2023-11-08T19:26:22.668 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.654+0000 7fda457fa700 1 -- 172.21.15.143:0/2403758129 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda3c011960 con 0x7fda4806b240 2023-11-08T19:26:22.669 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.654+0000 7fda4ecb3700 1 -- 172.21.15.143:0/2403758129 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda480c7c50 con 0x7fda4806b240 2023-11-08T19:26:22.669 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.654+0000 7fda457fa700 1 -- 172.21.15.143:0/2403758129 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda3c011cf0 con 0x7fda4806b240 2023-11-08T19:26:22.669 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.654+0000 7fda4ecb3700 1 -- 172.21.15.143:0/2403758129 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda4814ceb0 con 0x7fda4806b240 2023-11-08T19:26:22.669 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.655+0000 7fda4ecb3700 1 -- 172.21.15.143:0/2403758129 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda4804eee0 con 0x7fda4806b240 2023-11-08T19:26:22.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.656+0000 7fda457fa700 1 -- 172.21.15.143:0/2403758129 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fda3c018360 con 0x7fda4806b240 2023-11-08T19:26:22.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.656+0000 7fda457fa700 1 --2- 172.21.15.143:0/2403758129 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fda2c06af00 0x7fda2c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:22.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.656+0000 7fda457fa700 1 -- 172.21.15.143:0/2403758129 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fda3c089240 con 0x7fda4806b240 2023-11-08T19:26:22.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.656+0000 7fda47fff700 1 --2- 172.21.15.143:0/2403758129 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fda2c06af00 0x7fda2c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:22.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.657+0000 7fda47fff700 1 --2- 172.21.15.143:0/2403758129 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fda2c06af00 0x7fda2c06d3c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fda34000c00 tx=0x7fda34008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:22.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.661+0000 7fda457fa700 1 -- 172.21.15.143:0/2403758129 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fda3c054570 con 0x7fda4806b240 2023-11-08T19:26:22.852 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:22.851+0000 7fda4ecb3700 1 -- 172.21.15.143:0/2403758129 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fda480ceea0 con 0x7fda4806b240 2023-11-08T19:26:23.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:23.715+0000 7fda457fa700 1 -- 172.21.15.143:0/2403758129 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v45) v1 ==== 138+0+0 (secure 0 0 0) 0x7fda3c057b90 con 0x7fda4806b240 2023-11-08T19:26:23.722 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:23.719+0000 7fda4ecb3700 1 -- 172.21.15.143:0/2403758129 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fda2c06af00 msgr2=0x7fda2c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:23.722 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:23.719+0000 7fda4ecb3700 1 --2- 172.21.15.143:0/2403758129 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fda2c06af00 0x7fda2c06d3c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fda34000c00 tx=0x7fda34008030 comp rx=0 tx=0).stop 2023-11-08T19:26:23.722 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:23.720+0000 7fda4ecb3700 1 -- 172.21.15.143:0/2403758129 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fda4806b240 msgr2=0x7fda480ca880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:23.722 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:23.720+0000 7fda4ecb3700 1 --2- 172.21.15.143:0/2403758129 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fda4806b240 0x7fda480ca880 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fda3c00c7f0 tx=0x7fda3c00cbb0 comp rx=0 tx=0).stop 2023-11-08T19:26:23.723 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:23.720+0000 7fda4ecb3700 1 -- 172.21.15.143:0/2403758129 shutdown_connections 2023-11-08T19:26:23.723 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:23.720+0000 7fda4ecb3700 1 --2- 172.21.15.143:0/2403758129 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fda2c06af00 0x7fda2c06d3c0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:23.723 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:23.720+0000 7fda4ecb3700 1 --2- 172.21.15.143:0/2403758129 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fda4815dba0 0x7fda481e7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:23.724 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:23.720+0000 7fda4ecb3700 1 --2- 172.21.15.143:0/2403758129 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fda4806b240 0x7fda480ca880 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:23.724 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:23.720+0000 7fda4ecb3700 1 --2- 172.21.15.143:0/2403758129 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fda480cb530 0x7fda480c82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:23.724 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:23.720+0000 7fda4ecb3700 1 -- 172.21.15.143:0/2403758129 >> 172.21.15.143:0/2403758129 conn(0x7fda480cd2e0 msgr2=0x7fda480cd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:23.724 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:23.720+0000 7fda4ecb3700 1 -- 172.21.15.143:0/2403758129 shutdown_connections 2023-11-08T19:26:23.725 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:23.720+0000 7fda4ecb3700 1 -- 172.21.15.143:0/2403758129 wait complete. 2023-11-08T19:26:23.725 INFO:teuthology.orchestra.run.smithi143.stderr:pool 'cephfs_metadata' created 2023-11-08T19:26:23.741 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-08T19:26:23.874 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:23.872+0000 7fb2add0d700 1 Processor -- start 2023-11-08T19:26:23.875 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:23.872+0000 7fb2add0d700 1 -- start start 2023-11-08T19:26:23.877 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:23.873+0000 7fb2add0d700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb2a806c0c0 0x7fb2a80ccfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:23.877 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:23.873+0000 7fb2add0d700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb2a8072010 0x7fb2a81522a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:23.878 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:23.873+0000 7fb2add0d700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2a80ced30 con 0x7fb2a815c9b0 2023-11-08T19:26:23.878 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:23.873+0000 7fb2add0d700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2a80cd4f0 con 0x7fb2a8072010 2023-11-08T19:26:23.878 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:23.873+0000 7fb2add0d700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2a80cdf60 con 0x7fb2a806c0c0 2023-11-08T19:26:23.879 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:23.873+0000 7fb2a7fff700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb2a8072010 0x7fb2a81522a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:23.879 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:23.873+0000 7fb2a7fff700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb2a8072010 0x7fb2a81522a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51510/0 (socket says 172.21.15.143:51510) 2023-11-08T19:26:23.879 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:23.873+0000 7fb2a7fff700 1 -- 172.21.15.143:0/427292051 learned_addr learned my addr 172.21.15.143:0/427292051 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:23.879 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:23.873+0000 7fb2a77fe700 1 --2- 172.21.15.143:0/427292051 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb2a806c0c0 0x7fb2a80ccfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:23.879 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:23.873+0000 7fb2a7fff700 1 -- 172.21.15.143:0/427292051 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb2a806c0c0 msgr2=0x7fb2a80ccfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:23.880 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:23.873+0000 7fb2a7fff700 1 --2- 172.21.15.143:0/427292051 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb2a806c0c0 0x7fb2a80ccfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:23.880 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:23.873+0000 7fb2a7fff700 1 -- 172.21.15.143:0/427292051 >> v1:172.21.15.174:6789/0 conn(0x7fb2a815c9b0 legacy=0x7fb2a8071900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:23.880 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:23.873+0000 7fb2a7fff700 1 -- 172.21.15.143:0/427292051 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2a80cea50 con 0x7fb2a8072010 2023-11-08T19:26:23.880 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:23.873+0000 7fb2a77fe700 1 --2- 172.21.15.143:0/427292051 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb2a806c0c0 0x7fb2a80ccfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:26:23.880 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:23.874+0000 7fb2a7fff700 1 --2- 172.21.15.143:0/427292051 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb2a8072010 0x7fb2a81522a0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fb2a0009e50 tx=0x7fb2a000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdcb8fd98776b13e server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:23.881 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:23.874+0000 7fb2a67fc700 1 -- 172.21.15.143:0/427292051 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2a000e070 con 0x7fb2a8072010 2023-11-08T19:26:23.881 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:23.874+0000 7fb2a67fc700 1 -- 172.21.15.143:0/427292051 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2a0003a20 con 0x7fb2a8072010 2023-11-08T19:26:23.881 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:23.874+0000 7fb2a67fc700 1 -- 172.21.15.143:0/427292051 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2a0003dd0 con 0x7fb2a8072010 2023-11-08T19:26:23.881 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:23.874+0000 7fb2add0d700 1 -- 172.21.15.143:0/427292051 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb2a8072010 msgr2=0x7fb2a81522a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:23.881 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:23.874+0000 7fb2add0d700 1 --2- 172.21.15.143:0/427292051 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb2a8072010 0x7fb2a81522a0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fb2a0009e50 tx=0x7fb2a000b330 comp rx=0 tx=0).stop 2023-11-08T19:26:23.881 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:23.874+0000 7fb2add0d700 1 -- 172.21.15.143:0/427292051 shutdown_connections 2023-11-08T19:26:23.882 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:23.874+0000 7fb2add0d700 1 --2- 172.21.15.143:0/427292051 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb2a8072010 0x7fb2a81522a0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:23.882 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:23.874+0000 7fb2add0d700 1 --2- 172.21.15.143:0/427292051 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb2a806c0c0 0x7fb2a80ccfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:23.882 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:23.874+0000 7fb2add0d700 1 -- 172.21.15.143:0/427292051 >> 172.21.15.143:0/427292051 conn(0x7fb2a80b1c70 msgr2=0x7fb2a80b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:23.882 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:23.874+0000 7fb2add0d700 1 -- 172.21.15.143:0/427292051 shutdown_connections 2023-11-08T19:26:23.882 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:23.874+0000 7fb2add0d700 1 -- 172.21.15.143:0/427292051 wait complete. 2023-11-08T19:26:23.885 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:23.875+0000 7fb2add0d700 1 Processor -- start 2023-11-08T19:26:23.885 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:23.876+0000 7fb2add0d700 1 -- start start 2023-11-08T19:26:23.885 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:23.876+0000 7fb2add0d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb2a806c0c0 0x7fb2a80c9b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:23.886 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:23.876+0000 7fb2add0d700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb2a8072010 0x7fb2a80b5720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:23.886 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:23.876+0000 7fb2add0d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb2a815c9b0 0x7fb2a80b3290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:23.886 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:23.876+0000 7fb2add0d700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2a80c6330 con 0x7fb2a806c0c0 2023-11-08T19:26:23.887 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:23.876+0000 7fb2add0d700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2a80aef40 con 0x7fb2a815c9b0 2023-11-08T19:26:23.887 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:23.876+0000 7fb2add0d700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2a80c5a00 con 0x7fb2a8072010 2023-11-08T19:26:23.887 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:23.877+0000 7fb2a77fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb2a806c0c0 0x7fb2a80c9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:23.888 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:23.877+0000 7fb2a77fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb2a806c0c0 0x7fb2a80c9b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51514/0 (socket says 172.21.15.143:51514) 2023-11-08T19:26:23.888 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:23.877+0000 7fb2a77fe700 1 -- 172.21.15.143:0/3436866339 learned_addr learned my addr 172.21.15.143:0/3436866339 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:23.888 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:23.877+0000 7fb2a7fff700 1 --2- 172.21.15.143:0/3436866339 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb2a815c9b0 0x7fb2a80b3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:23.889 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:23.877+0000 7fb2a6ffd700 1 --2- 172.21.15.143:0/3436866339 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb2a8072010 0x7fb2a80b5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:23.889 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:23.877+0000 7fb2a77fe700 1 -- 172.21.15.143:0/3436866339 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb2a8072010 msgr2=0x7fb2a80b5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:23.889 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:23.877+0000 7fb2a77fe700 1 --2- 172.21.15.143:0/3436866339 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb2a8072010 0x7fb2a80b5720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:23.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:23.877+0000 7fb2a77fe700 1 -- 172.21.15.143:0/3436866339 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb2a815c9b0 msgr2=0x7fb2a80b3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:23.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:23.877+0000 7fb2a77fe700 1 --2- 172.21.15.143:0/3436866339 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb2a815c9b0 0x7fb2a80b3290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:23.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:23.877+0000 7fb2a77fe700 1 -- 172.21.15.143:0/3436866339 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2a0009b30 con 0x7fb2a806c0c0 2023-11-08T19:26:23.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:23.877+0000 7fb2a6ffd700 1 --2- 172.21.15.143:0/3436866339 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb2a8072010 0x7fb2a80b5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:26:23.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:23.877+0000 7fb2a77fe700 1 --2- 172.21.15.143:0/3436866339 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb2a806c0c0 0x7fb2a80c9b00 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fb29c00c750 tx=0x7fb29c00cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:23.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:23.877+0000 7fb28ffff700 1 -- 172.21.15.143:0/3436866339 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb29c011800 con 0x7fb2a806c0c0 2023-11-08T19:26:23.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:23.877+0000 7fb28ffff700 1 -- 172.21.15.143:0/3436866339 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb29c011960 con 0x7fb2a806c0c0 2023-11-08T19:26:23.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:23.877+0000 7fb2add0d700 1 -- 172.21.15.143:0/3436866339 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2a80c5ce0 con 0x7fb2a806c0c0 2023-11-08T19:26:23.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:23.877+0000 7fb28ffff700 1 -- 172.21.15.143:0/3436866339 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb29c011cd0 con 0x7fb2a806c0c0 2023-11-08T19:26:23.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:23.877+0000 7fb2add0d700 1 -- 172.21.15.143:0/3436866339 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2a80ca040 con 0x7fb2a806c0c0 2023-11-08T19:26:23.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:23.878+0000 7fb2add0d700 1 -- 172.21.15.143:0/3436866339 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2a804eee0 con 0x7fb2a806c0c0 2023-11-08T19:26:23.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:23.879+0000 7fb28ffff700 1 -- 172.21.15.143:0/3436866339 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb29c021350 con 0x7fb2a806c0c0 2023-11-08T19:26:23.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:23.879+0000 7fb28ffff700 1 --2- 172.21.15.143:0/3436866339 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fb29406af00 0x7fb29406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:23.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:23.879+0000 7fb28ffff700 1 -- 172.21.15.143:0/3436866339 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5580+0+0 (secure 0 0 0) 0x7fb29c0894e0 con 0x7fb2a806c0c0 2023-11-08T19:26:23.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:23.879+0000 7fb2a6ffd700 1 --2- 172.21.15.143:0/3436866339 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fb29406af00 0x7fb29406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:23.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:23.880+0000 7fb2a6ffd700 1 --2- 172.21.15.143:0/3436866339 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fb29406af00 0x7fb29406d3c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fb2a8159020 tx=0x7fb298008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:23.895 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:23.884+0000 7fb28ffff700 1 -- 172.21.15.143:0/3436866339 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb29c054610 con 0x7fb2a806c0c0 2023-11-08T19:26:24.075 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:24.073+0000 7fb2add0d700 1 -- 172.21.15.143:0/3436866339 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fb2a8072740 con 0x7fb2a806c0c0 2023-11-08T19:26:24.723 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:24.721+0000 7fb28ffff700 1 -- 172.21.15.143:0/3436866339 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v46) v1 ==== 170+0+0 (secure 0 0 0) 0x7fb29c057c30 con 0x7fb2a806c0c0 2023-11-08T19:26:24.726 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:24.724+0000 7fb2add0d700 1 -- 172.21.15.143:0/3436866339 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fb29406af00 msgr2=0x7fb29406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:24.726 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:24.724+0000 7fb2add0d700 1 --2- 172.21.15.143:0/3436866339 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fb29406af00 0x7fb29406d3c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fb2a8159020 tx=0x7fb298008040 comp rx=0 tx=0).stop 2023-11-08T19:26:24.726 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:24.724+0000 7fb2add0d700 1 -- 172.21.15.143:0/3436866339 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb2a806c0c0 msgr2=0x7fb2a80c9b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:24.726 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:24.724+0000 7fb2add0d700 1 --2- 172.21.15.143:0/3436866339 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb2a806c0c0 0x7fb2a80c9b00 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fb29c00c750 tx=0x7fb29c00cb10 comp rx=0 tx=0).stop 2023-11-08T19:26:24.727 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:24.725+0000 7fb2add0d700 1 -- 172.21.15.143:0/3436866339 shutdown_connections 2023-11-08T19:26:24.727 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:24.725+0000 7fb2add0d700 1 --2- 172.21.15.143:0/3436866339 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fb29406af00 0x7fb29406d3c0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:24.727 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:24.725+0000 7fb2add0d700 1 --2- 172.21.15.143:0/3436866339 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb2a815c9b0 0x7fb2a80b3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:24.728 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:24.725+0000 7fb2add0d700 1 --2- 172.21.15.143:0/3436866339 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb2a806c0c0 0x7fb2a80c9b00 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:24.728 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:24.725+0000 7fb2add0d700 1 --2- 172.21.15.143:0/3436866339 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb2a8072010 0x7fb2a80b5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:24.728 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:24.725+0000 7fb2add0d700 1 -- 172.21.15.143:0/3436866339 >> 172.21.15.143:0/3436866339 conn(0x7fb2a80b1c70 msgr2=0x7fb2a80cb3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:24.729 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:24.726+0000 7fb2add0d700 1 -- 172.21.15.143:0/3436866339 shutdown_connections 2023-11-08T19:26:24.738 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:24.736+0000 7fb2add0d700 1 -- 172.21.15.143:0/3436866339 wait complete. 2023-11-08T19:26:24.744 INFO:teuthology.orchestra.run.smithi143.stderr:pool 'cephfs_data' created 2023-11-08T19:26:24.759 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-08T19:26:24.886 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:24.884+0000 7fa07955e700 1 Processor -- start 2023-11-08T19:26:24.887 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:24.884+0000 7fa07955e700 1 -- start start 2023-11-08T19:26:24.889 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:24.885+0000 7fa07955e700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa07415dca0 0x7fa0740b93f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:24.889 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:24.885+0000 7fa07955e700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa0740b5ab0 0x7fa0740b7280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:24.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:24.885+0000 7fa07955e700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0740b8f50 con 0x7fa0740b5630 2023-11-08T19:26:24.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:24.885+0000 7fa07955e700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0740b3360 con 0x7fa0740b5ab0 2023-11-08T19:26:24.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:24.885+0000 7fa07955e700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0740b7eb0 con 0x7fa07415dca0 2023-11-08T19:26:24.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:24.885+0000 7fa0737fe700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa0740b5ab0 0x7fa0740b7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:24.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:24.885+0000 7fa0737fe700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa0740b5ab0 0x7fa0740b7280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51524/0 (socket says 172.21.15.143:51524) 2023-11-08T19:26:24.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:24.885+0000 7fa0737fe700 1 -- 172.21.15.143:0/2699273799 learned_addr learned my addr 172.21.15.143:0/2699273799 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:24.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:24.885+0000 7fa072ffd700 1 --2- 172.21.15.143:0/2699273799 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa07415dca0 0x7fa0740b93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:24.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:24.885+0000 7fa0737fe700 1 -- 172.21.15.143:0/2699273799 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa07415dca0 msgr2=0x7fa0740b93f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:24.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:24.885+0000 7fa0737fe700 1 --2- 172.21.15.143:0/2699273799 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa07415dca0 0x7fa0740b93f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:24.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:24.885+0000 7fa0737fe700 1 -- 172.21.15.143:0/2699273799 >> v1:172.21.15.174:6789/0 conn(0x7fa0740b5630 legacy=0x7fa0740b6b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:24.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:24.885+0000 7fa0737fe700 1 -- 172.21.15.143:0/2699273799 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0740b8190 con 0x7fa0740b5ab0 2023-11-08T19:26:24.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:24.885+0000 7fa072ffd700 1 --2- 172.21.15.143:0/2699273799 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa07415dca0 0x7fa0740b93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:26:24.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:24.885+0000 7fa0737fe700 1 --2- 172.21.15.143:0/2699273799 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa0740b5ab0 0x7fa0740b7280 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fa06c009e50 tx=0x7fa06c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92f69734aaa9f4f7 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:24.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:24.885+0000 7fa071ffb700 1 -- 172.21.15.143:0/2699273799 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa06c00e070 con 0x7fa0740b5ab0 2023-11-08T19:26:24.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:24.886+0000 7fa071ffb700 1 -- 172.21.15.143:0/2699273799 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa06c00b9c0 con 0x7fa0740b5ab0 2023-11-08T19:26:24.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:24.886+0000 7fa071ffb700 1 -- 172.21.15.143:0/2699273799 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa06c00bd70 con 0x7fa0740b5ab0 2023-11-08T19:26:24.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:24.886+0000 7fa07955e700 1 -- 172.21.15.143:0/2699273799 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa0740b5ab0 msgr2=0x7fa0740b7280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:24.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:24.886+0000 7fa07955e700 1 --2- 172.21.15.143:0/2699273799 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa0740b5ab0 0x7fa0740b7280 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fa06c009e50 tx=0x7fa06c00b330 comp rx=0 tx=0).stop 2023-11-08T19:26:24.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:24.886+0000 7fa07955e700 1 -- 172.21.15.143:0/2699273799 shutdown_connections 2023-11-08T19:26:24.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:24.886+0000 7fa07955e700 1 --2- 172.21.15.143:0/2699273799 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa0740b5ab0 0x7fa0740b7280 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:24.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:24.886+0000 7fa07955e700 1 --2- 172.21.15.143:0/2699273799 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa07415dca0 0x7fa0740b93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:24.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:24.886+0000 7fa07955e700 1 -- 172.21.15.143:0/2699273799 >> 172.21.15.143:0/2699273799 conn(0x7fa074066d00 msgr2=0x7fa074067110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:24.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:24.886+0000 7fa07955e700 1 -- 172.21.15.143:0/2699273799 shutdown_connections 2023-11-08T19:26:24.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:24.886+0000 7fa07955e700 1 -- 172.21.15.143:0/2699273799 wait complete. 2023-11-08T19:26:24.897 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:24.887+0000 7fa07955e700 1 Processor -- start 2023-11-08T19:26:24.897 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:24.887+0000 7fa07955e700 1 -- start start 2023-11-08T19:26:24.898 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:24.888+0000 7fa07955e700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa0740b5630 0x7fa0741e5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:24.898 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:24.888+0000 7fa07955e700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa0740b5ab0 0x7fa0740c8f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:24.898 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:24.888+0000 7fa07955e700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa07415dca0 0x7fa0741e3de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:24.898 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:24.888+0000 7fa07955e700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0741e5fa0 con 0x7fa0740b5ab0 2023-11-08T19:26:24.898 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:24.888+0000 7fa07955e700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa07415cb80 con 0x7fa0740b5630 2023-11-08T19:26:24.899 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:24.888+0000 7fa07955e700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0741e49c0 con 0x7fa07415dca0 2023-11-08T19:26:24.899 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:24.888+0000 7fa0727fc700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa0740b5ab0 0x7fa0740c8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:24.899 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:24.888+0000 7fa0727fc700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa0740b5ab0 0x7fa0740c8f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51526/0 (socket says 172.21.15.143:51526) 2023-11-08T19:26:24.899 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:24.888+0000 7fa0727fc700 1 -- 172.21.15.143:0/541008094 learned_addr learned my addr 172.21.15.143:0/541008094 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:24.899 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:24.888+0000 7fa072ffd700 1 --2- 172.21.15.143:0/541008094 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa0740b5630 0x7fa0741e5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:24.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:24.888+0000 7fa0737fe700 1 --2- 172.21.15.143:0/541008094 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa07415dca0 0x7fa0741e3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:24.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:24.889+0000 7fa0727fc700 1 -- 172.21.15.143:0/541008094 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa07415dca0 msgr2=0x7fa0741e3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:24.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:24.889+0000 7fa0727fc700 1 --2- 172.21.15.143:0/541008094 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa07415dca0 0x7fa0741e3de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:24.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:24.889+0000 7fa0727fc700 1 -- 172.21.15.143:0/541008094 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa0740b5630 msgr2=0x7fa0741e5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:24.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:24.889+0000 7fa0727fc700 1 --2- 172.21.15.143:0/541008094 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa0740b5630 0x7fa0741e5a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:24.901 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:24.889+0000 7fa0727fc700 1 -- 172.21.15.143:0/541008094 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa06c009b30 con 0x7fa0740b5ab0 2023-11-08T19:26:24.901 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:24.889+0000 7fa072ffd700 1 --2- 172.21.15.143:0/541008094 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa0740b5630 0x7fa0741e5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:26:24.901 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:24.889+0000 7fa0727fc700 1 --2- 172.21.15.143:0/541008094 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa0740b5ab0 0x7fa0740c8f40 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fa06400c4e0 tx=0x7fa06400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:24.901 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:24.889+0000 7fa05b7fe700 1 -- 172.21.15.143:0/541008094 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0640052d0 con 0x7fa0740b5ab0 2023-11-08T19:26:24.901 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:24.889+0000 7fa05b7fe700 1 -- 172.21.15.143:0/541008094 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa064005430 con 0x7fa0740b5ab0 2023-11-08T19:26:24.902 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:24.889+0000 7fa05b7fe700 1 -- 172.21.15.143:0/541008094 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0640038c0 con 0x7fa0740b5ab0 2023-11-08T19:26:24.902 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:24.889+0000 7fa07955e700 1 -- 172.21.15.143:0/541008094 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0741e4ca0 con 0x7fa0740b5ab0 2023-11-08T19:26:24.902 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:24.889+0000 7fa07955e700 1 -- 172.21.15.143:0/541008094 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0741e5260 con 0x7fa0740b5ab0 2023-11-08T19:26:24.902 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:24.890+0000 7fa07955e700 1 -- 172.21.15.143:0/541008094 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa07404eee0 con 0x7fa0740b5ab0 2023-11-08T19:26:24.902 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:24.890+0000 7fa05b7fe700 1 -- 172.21.15.143:0/541008094 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa064003a20 con 0x7fa0740b5ab0 2023-11-08T19:26:24.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:24.891+0000 7fa05b7fe700 1 --2- 172.21.15.143:0/541008094 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa06006af00 0x7fa06006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:24.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:24.891+0000 7fa05b7fe700 1 -- 172.21.15.143:0/541008094 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5962+0+0 (secure 0 0 0) 0x7fa064088c60 con 0x7fa0740b5ab0 2023-11-08T19:26:24.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:24.891+0000 7fa072ffd700 1 --2- 172.21.15.143:0/541008094 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa06006af00 0x7fa06006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:24.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:24.891+0000 7fa072ffd700 1 --2- 172.21.15.143:0/541008094 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa06006af00 0x7fa06006d3c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fa06800aa20 tx=0x7fa068008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:24.904 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:24.895+0000 7fa05b7fe700 1 -- 172.21.15.143:0/541008094 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa064053b70 con 0x7fa0740b5ab0 2023-11-08T19:26:25.128 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:25.126+0000 7fa07955e700 1 -- 172.21.15.143:0/541008094 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fa0740d1180 con 0x7fa0740b5ab0 2023-11-08T19:26:25.726 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:25.723+0000 7fa05b7fe700 1 -- 172.21.15.143:0/541008094 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 7 and data pool 8 v21) v1 ==== 172+0+0 (secure 0 0 0) 0x7fa0640170c0 con 0x7fa0740b5ab0 2023-11-08T19:26:25.729 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:25.727+0000 7fa07955e700 1 -- 172.21.15.143:0/541008094 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa06006af00 msgr2=0x7fa06006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:25.729 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:25.727+0000 7fa07955e700 1 --2- 172.21.15.143:0/541008094 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa06006af00 0x7fa06006d3c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fa06800aa20 tx=0x7fa068008040 comp rx=0 tx=0).stop 2023-11-08T19:26:25.730 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:25.727+0000 7fa07955e700 1 -- 172.21.15.143:0/541008094 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa0740b5ab0 msgr2=0x7fa0740c8f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:25.730 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:25.727+0000 7fa07955e700 1 --2- 172.21.15.143:0/541008094 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa0740b5ab0 0x7fa0740c8f40 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fa06400c4e0 tx=0x7fa06400c8a0 comp rx=0 tx=0).stop 2023-11-08T19:26:25.730 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:25.728+0000 7fa07955e700 1 -- 172.21.15.143:0/541008094 shutdown_connections 2023-11-08T19:26:25.731 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:25.728+0000 7fa07955e700 1 --2- 172.21.15.143:0/541008094 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa06006af00 0x7fa06006d3c0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:25.731 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:25.728+0000 7fa07955e700 1 --2- 172.21.15.143:0/541008094 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa07415dca0 0x7fa0741e3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:25.731 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:25.728+0000 7fa07955e700 1 --2- 172.21.15.143:0/541008094 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa0740b5630 0x7fa0741e5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:25.731 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:25.728+0000 7fa07955e700 1 --2- 172.21.15.143:0/541008094 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa0740b5ab0 0x7fa0740c8f40 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:25.731 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:25.728+0000 7fa07955e700 1 -- 172.21.15.143:0/541008094 >> 172.21.15.143:0/541008094 conn(0x7fa074066d00 msgr2=0x7fa0740b6b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:25.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:25.732+0000 7fa07955e700 1 -- 172.21.15.143:0/541008094 shutdown_connections 2023-11-08T19:26:25.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:25.739+0000 7fa07955e700 1 -- 172.21.15.143:0/541008094 wait complete. 2023-11-08T19:26:25.753 INFO:teuthology.orchestra.run.smithi143.stderr:new fs with metadata pool 7 and data pool 8 2023-11-08T19:26:25.766 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:26:25.887 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:25.886+0000 7f0bf89ad700 1 Processor -- start 2023-11-08T19:26:25.887 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:25.886+0000 7f0bf89ad700 1 -- start start 2023-11-08T19:26:25.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:25.887+0000 7f0bf89ad700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f0bf415dd00 0x7f0bf40cca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:25.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:25.887+0000 7f0bf89ad700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0bf406b320 0x7f0bf40cb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:25.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:25.887+0000 7f0bf89ad700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bf40ccfb0 con 0x7f0bf41618f0 2023-11-08T19:26:25.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:25.887+0000 7f0bf89ad700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bf40cad50 con 0x7f0bf415dd00 2023-11-08T19:26:25.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:25.887+0000 7f0bf89ad700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf4161df0 con 0x7f0bf406b320 2023-11-08T19:26:25.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:25.887+0000 7f0bf259c700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f0bf415dd00 0x7f0bf40cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:25.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:25.887+0000 7f0bf1d9b700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f0bf41618f0 0x7f0bf406ac10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:47704/0 (socket says 172.21.15.143:47704) 2023-11-08T19:26:25.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:25.887+0000 7f0bf1d9b700 1 -- 172.21.15.143:0/2323590374 learned_addr learned my addr 172.21.15.143:0/2323590374 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:26:25.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:25.887+0000 7f0bf259c700 1 -- 172.21.15.143:0/2323590374 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0bf406b320 msgr2=0x7f0bf40cb1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:25.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:25.887+0000 7f0bf259c700 1 --2- 172.21.15.143:0/2323590374 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0bf406b320 0x7f0bf40cb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:25.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:25.887+0000 7f0bf259c700 1 -- 172.21.15.143:0/2323590374 >> v1:172.21.15.143:6789/0 conn(0x7f0bf41618f0 legacy=0x7f0bf406ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:25.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:25.887+0000 7f0bf259c700 1 -- 172.21.15.143:0/2323590374 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bf41620d0 con 0x7f0bf415dd00 2023-11-08T19:26:25.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:25.887+0000 7f0bf159a700 1 -- 172.21.15.143:0/2323590374 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1120785702 0 0) 0x7f0bf40ccfb0 con 0x7f0bf41618f0 2023-11-08T19:26:25.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:25.887+0000 7f0bf259c700 1 --2- 172.21.15.143:0/2323590374 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0bf415dd00 0x7f0bf40cca70 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f0be4009820 tx=0x7f0be4009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1604399b06b3500a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:25.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:25.887+0000 7f0bf159a700 1 -- 172.21.15.143:0/2323590374 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0be4005e30 con 0x7f0bf415dd00 2023-11-08T19:26:25.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:25.887+0000 7f0bf159a700 1 -- 172.21.15.143:0/2323590374 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0be4006030 con 0x7f0bf415dd00 2023-11-08T19:26:25.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:25.887+0000 7f0bf159a700 1 -- 172.21.15.143:0/2323590374 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0be40063e0 con 0x7f0bf415dd00 2023-11-08T19:26:25.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:25.888+0000 7f0bf89ad700 1 -- 172.21.15.143:0/2323590374 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0bf415dd00 msgr2=0x7f0bf40cca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:25.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:25.888+0000 7f0bf89ad700 1 --2- 172.21.15.143:0/2323590374 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0bf415dd00 0x7f0bf40cca70 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f0be4009820 tx=0x7f0be4009b30 comp rx=0 tx=0).stop 2023-11-08T19:26:25.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:25.888+0000 7f0bf89ad700 1 -- 172.21.15.143:0/2323590374 shutdown_connections 2023-11-08T19:26:25.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:25.888+0000 7f0bf89ad700 1 --2- 172.21.15.143:0/2323590374 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0bf406b320 0x7f0bf40cb1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:25.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:25.888+0000 7f0bf89ad700 1 --2- 172.21.15.143:0/2323590374 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0bf415dd00 0x7f0bf40cca70 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:25.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:25.888+0000 7f0bf89ad700 1 -- 172.21.15.143:0/2323590374 >> 172.21.15.143:0/2323590374 conn(0x7f0bf4099ab0 msgr2=0x7f0bf40d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:25.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:25.888+0000 7f0bf89ad700 1 -- 172.21.15.143:0/2323590374 shutdown_connections 2023-11-08T19:26:25.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:25.888+0000 7f0bf89ad700 1 -- 172.21.15.143:0/2323590374 wait complete. 2023-11-08T19:26:25.897 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:25.889+0000 7f0bf89ad700 1 Processor -- start 2023-11-08T19:26:25.897 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:25.889+0000 7f0bf89ad700 1 -- start start 2023-11-08T19:26:25.897 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:25.890+0000 7f0bf89ad700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0bf406b320 0x7f0bf40b4400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:25.898 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:25.890+0000 7f0bf89ad700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0bf415dd00 0x7f0bf41f3120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:25.898 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:25.890+0000 7f0bf89ad700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0bf41618f0 0x7f0bf40b59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:25.898 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:25.890+0000 7f0bf89ad700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf40b5360 con 0x7f0bf406b320 2023-11-08T19:26:25.898 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:25.890+0000 7f0bf89ad700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf40b5f00 con 0x7f0bf41618f0 2023-11-08T19:26:25.898 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:25.890+0000 7f0bf89ad700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf40b0ef0 con 0x7f0bf415dd00 2023-11-08T19:26:25.899 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:25.890+0000 7f0bf259c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0bf406b320 0x7f0bf40b4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:25.899 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:25.890+0000 7f0bf259c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0bf406b320 0x7f0bf40b4400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51554/0 (socket says 172.21.15.143:51554) 2023-11-08T19:26:25.899 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:25.890+0000 7f0bf259c700 1 -- 172.21.15.143:0/3659566428 learned_addr learned my addr 172.21.15.143:0/3659566428 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:25.899 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:25.890+0000 7f0bf259c700 1 -- 172.21.15.143:0/3659566428 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0bf415dd00 msgr2=0x7f0bf41f3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:25.899 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:25.890+0000 7f0bf259c700 1 --2- 172.21.15.143:0/3659566428 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0bf415dd00 0x7f0bf41f3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:25.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:25.891+0000 7f0bf259c700 1 -- 172.21.15.143:0/3659566428 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0bf41618f0 msgr2=0x7f0bf40b59c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:25.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:25.891+0000 7f0bf259c700 1 --2- 172.21.15.143:0/3659566428 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0bf41618f0 0x7f0bf40b59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:25.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:25.891+0000 7f0bf259c700 1 -- 172.21.15.143:0/3659566428 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0be4009430 con 0x7f0bf406b320 2023-11-08T19:26:25.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:25.891+0000 7f0bf259c700 1 --2- 172.21.15.143:0/3659566428 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0bf406b320 0x7f0bf40b4400 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f0be4009b70 tx=0x7f0be4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:25.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:25.891+0000 7f0be2ffd700 1 -- 172.21.15.143:0/3659566428 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0be4004040 con 0x7f0bf406b320 2023-11-08T19:26:25.901 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:25.891+0000 7f0be2ffd700 1 -- 172.21.15.143:0/3659566428 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0be4020070 con 0x7f0bf406b320 2023-11-08T19:26:25.901 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:25.891+0000 7f0bf89ad700 1 -- 172.21.15.143:0/3659566428 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bf40b0bc0 con 0x7f0bf406b320 2023-11-08T19:26:25.901 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:25.891+0000 7f0be2ffd700 1 -- 172.21.15.143:0/3659566428 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0be4004400 con 0x7f0bf406b320 2023-11-08T19:26:25.901 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:25.891+0000 7f0bf89ad700 1 -- 172.21.15.143:0/3659566428 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bf40b4da0 con 0x7f0bf406b320 2023-11-08T19:26:25.901 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:25.892+0000 7f0bf89ad700 1 -- 172.21.15.143:0/3659566428 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bf404eee0 con 0x7f0bf406b320 2023-11-08T19:26:25.902 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:25.892+0000 7f0be2ffd700 1 -- 172.21.15.143:0/3659566428 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0be4022030 con 0x7f0bf406b320 2023-11-08T19:26:25.902 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:25.893+0000 7f0be2ffd700 1 --2- 172.21.15.143:0/3659566428 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f0bd406af00 0x7f0bd406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:25.902 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:25.893+0000 7f0be2ffd700 1 -- 172.21.15.143:0/3659566428 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0be408aed0 con 0x7f0bf406b320 2023-11-08T19:26:25.902 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:25.893+0000 7f0bf1d9b700 1 --2- 172.21.15.143:0/3659566428 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f0bd406af00 0x7f0bd406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:25.902 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:25.893+0000 7f0bf1d9b700 1 --2- 172.21.15.143:0/3659566428 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f0bd406af00 0x7f0bd406d3c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f0bf40b54c0 tx=0x7f0be80079f0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:25.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:25.897+0000 7f0be2ffd700 1 -- 172.21.15.143:0/3659566428 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0be4055e10 con 0x7f0bf406b320 2023-11-08T19:26:26.086 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.085+0000 7f0bf89ad700 1 -- 172.21.15.143:0/3659566428 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0bf406b920 con 0x7f0bf406b320 2023-11-08T19:26:26.087 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.086+0000 7f0be2ffd700 1 -- 172.21.15.143:0/3659566428 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) v1 ==== 74+0+18383 (secure 0 0 0) 0x7f0be4059430 con 0x7f0bf406b320 2023-11-08T19:26:26.088 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:26:26.088 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":47,"fsid":"968483af-77fc-4068-ad7c-5c2a1a4613d7","created":"2023-11-08T19:24:32.826484+0000","modified":"2023-11-08T19:26:25.128323+0000","last_up_change":"2023-11-08T19:24:43.573880+0000","last_in_change":"2023-11-08T19:24:35.314525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:44.551530+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"snap-test-pool","create_time":"2023-11-08T19:26:02.874384+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":36,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-11-08T19:26:22.869780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-11-08T19:26:24.093179+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bae98ec1-d94e-403d-a220-a522a53ea6f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6809","nonce":1487830566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6811","nonce":1487830566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6815","nonce":1487830566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6813","nonce":1487830566}]},"public_addr":"172.21.15.143:6809/1487830566","cluster_addr":"172.21.15.143:6811/1487830566","heartbeat_back_addr":"172.21.15.143:6815/1487830566","heartbeat_front_addr":"172.21.15.143:6813/1487830566","state":["exists","up"]},{"osd":1,"uuid":"5cb6538a-1ccb-4322-8633-09f66af0b7e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6801","nonce":787341397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6803","nonce":787341397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6807","nonce":787341397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6805","nonce":787341397}]},"public_addr":"172.21.15.143:6801/787341397","cluster_addr":"172.21.15.143:6803/787341397","heartbeat_back_addr":"172.21.15.143:6807/787341397","heartbeat_front_addr":"172.21.15.143:6805/787341397","state":["exists","up"]},{"osd":2,"uuid":"cdbf522c-60ef-4d60-9653-8dcb529c101a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6822","nonce":2301325597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6826","nonce":2301325597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6831","nonce":2301325597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6829","nonce":2301325597}]},"public_addr":"172.21.15.143:6822/2301325597","cluster_addr":"172.21.15.143:6826/2301325597","heartbeat_back_addr":"172.21.15.143:6831/2301325597","heartbeat_front_addr":"172.21.15.143:6829/2301325597","state":["exists","up"]},{"osd":3,"uuid":"4e2783cf-ebb9-4dd8-8245-bc1d17ce4325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6817","nonce":2021504428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6819","nonce":2021504428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6827","nonce":2021504428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6823","nonce":2021504428}]},"public_addr":"172.21.15.143:6817/2021504428","cluster_addr":"172.21.15.143:6819/2021504428","heartbeat_back_addr":"172.21.15.143:6827/2021504428","heartbeat_front_addr":"172.21.15.143:6823/2021504428","state":["exists","up"]},{"osd":4,"uuid":"691256fa-9b77-49b9-a346-7ae1adf0ca80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6801","nonce":1993380595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6803","nonce":1993380595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6807","nonce":1993380595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6805","nonce":1993380595}]},"public_addr":"172.21.15.174:6801/1993380595","cluster_addr":"172.21.15.174:6803/1993380595","heartbeat_back_addr":"172.21.15.174:6807/1993380595","heartbeat_front_addr":"172.21.15.174:6805/1993380595","state":["exists","up"]},{"osd":5,"uuid":"756fc352-48c9-46ce-b52e-0541fe6f8525","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6809","nonce":1777923303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6811","nonce":1777923303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6815","nonce":1777923303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6813","nonce":1777923303}]},"public_addr":"172.21.15.174:6809/1777923303","cluster_addr":"172.21.15.174:6811/1777923303","heartbeat_back_addr":"172.21.15.174:6815/1777923303","heartbeat_front_addr":"172.21.15.174:6813/1777923303","state":["exists","up"]},{"osd":6,"uuid":"fab2ac2b-ed97-4cde-9825-b480eabf8b8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6825","nonce":2755130316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6827","nonce":2755130316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6831","nonce":2755130316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6829","nonce":2755130316}]},"public_addr":"172.21.15.174:6825/2755130316","cluster_addr":"172.21.15.174:6827/2755130316","heartbeat_back_addr":"172.21.15.174:6831/2755130316","heartbeat_front_addr":"172.21.15.174:6829/2755130316","state":["exists","up"]},{"osd":7,"uuid":"3de9b797-2ab6-4d41-a863-7e7d638fd703","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6817","nonce":2847141411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6819","nonce":2847141411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6823","nonce":2847141411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6821","nonce":2847141411}]},"public_addr":"172.21.15.174:6817/2847141411","cluster_addr":"172.21.15.174:6819/2847141411","heartbeat_back_addr":"172.21.15.174:6823/2847141411","heartbeat_front_addr":"172.21.15.174:6821/2847141411","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.477076+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.496890+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.539305+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.454673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.538881+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.521294+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.520712+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.540404+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:26:26.090 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.088+0000 7f0bf89ad700 1 -- 172.21.15.143:0/3659566428 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f0bd406af00 msgr2=0x7f0bd406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:26.090 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.088+0000 7f0bf89ad700 1 --2- 172.21.15.143:0/3659566428 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f0bd406af00 0x7f0bd406d3c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f0bf40b54c0 tx=0x7f0be80079f0 comp rx=0 tx=0).stop 2023-11-08T19:26:26.091 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.088+0000 7f0bf89ad700 1 -- 172.21.15.143:0/3659566428 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0bf406b320 msgr2=0x7f0bf40b4400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:26.091 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.088+0000 7f0bf89ad700 1 --2- 172.21.15.143:0/3659566428 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0bf406b320 0x7f0bf40b4400 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f0be4009b70 tx=0x7f0be4004750 comp rx=0 tx=0).stop 2023-11-08T19:26:26.091 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.088+0000 7f0bf89ad700 1 -- 172.21.15.143:0/3659566428 shutdown_connections 2023-11-08T19:26:26.091 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.088+0000 7f0bf89ad700 1 --2- 172.21.15.143:0/3659566428 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f0bd406af00 0x7f0bd406d3c0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:26.091 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.088+0000 7f0bf89ad700 1 --2- 172.21.15.143:0/3659566428 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0bf41618f0 0x7f0bf40b59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:26.092 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.088+0000 7f0bf89ad700 1 --2- 172.21.15.143:0/3659566428 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0bf406b320 0x7f0bf40b4400 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:26.092 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.088+0000 7f0bf89ad700 1 --2- 172.21.15.143:0/3659566428 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0bf415dd00 0x7f0bf41f3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:26.092 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.088+0000 7f0bf89ad700 1 -- 172.21.15.143:0/3659566428 >> 172.21.15.143:0/3659566428 conn(0x7f0bf4099ab0 msgr2=0x7f0bf406ef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:26.092 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.088+0000 7f0bf89ad700 1 -- 172.21.15.143:0/3659566428 shutdown_connections 2023-11-08T19:26:26.092 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.088+0000 7f0bf89ad700 1 -- 172.21.15.143:0/3659566428 wait complete. 2023-11-08T19:26:26.103 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:26:26.223 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.222+0000 7f30609a3700 1 Processor -- start 2023-11-08T19:26:26.223 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.222+0000 7f30609a3700 1 -- start start 2023-11-08T19:26:26.226 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.223+0000 7f30609a3700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f305c0b14c0 0x7f305c0d03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:26.226 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.223+0000 7f30609a3700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f305c0d08e0 con 0x7f305c15ba30 2023-11-08T19:26:26.226 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.223+0000 7f30609a3700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f305c0b1910 con 0x7f305c0d1c70 2023-11-08T19:26:26.227 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.223+0000 7f30609a3700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f305c0b1bc0 con 0x7f305c0b14c0 2023-11-08T19:26:26.227 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.223+0000 7f305ad9d700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f305c0b14c0 0x7f305c0d03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:26.227 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.223+0000 7f305a59c700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f305c15ba30 0x7f305c0d1560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:47716/0 (socket says 172.21.15.143:47716) 2023-11-08T19:26:26.227 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.223+0000 7f305a59c700 1 -- 172.21.15.143:0/920985470 learned_addr learned my addr 172.21.15.143:0/920985470 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:26:26.227 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.223+0000 7f305ad9d700 1 -- 172.21.15.143:0/920985470 >> v1:172.21.15.174:6789/0 conn(0x7f305c0d1c70 legacy=0x7f305c0cc5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:26.228 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.223+0000 7f305ad9d700 1 -- 172.21.15.143:0/920985470 >> v1:172.21.15.143:6789/0 conn(0x7f305c15ba30 legacy=0x7f305c0d1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:26.228 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.223+0000 7f305ad9d700 1 -- 172.21.15.143:0/920985470 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f305c0b1ea0 con 0x7f305c0b14c0 2023-11-08T19:26:26.228 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.223+0000 7f305959a700 1 -- 172.21.15.143:0/920985470 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3410618279 0 0) 0x7f304c001110 con 0x7f305c15ba30 2023-11-08T19:26:26.228 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.223+0000 7f305ad9d700 1 --2- 172.21.15.143:0/920985470 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f305c0b14c0 0x7f305c0d03a0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f3054009d90 tx=0x7f305400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17f23fc7e2c69a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:26.229 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.223+0000 7f305959a700 1 -- 172.21.15.143:0/920985470 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f305400e070 con 0x7f305c0b14c0 2023-11-08T19:26:26.229 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.223+0000 7f305959a700 1 -- 172.21.15.143:0/920985470 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f305400bbd0 con 0x7f305c0b14c0 2023-11-08T19:26:26.229 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.223+0000 7f305959a700 1 -- 172.21.15.143:0/920985470 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3054003a20 con 0x7f305c0b14c0 2023-11-08T19:26:26.229 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.224+0000 7f30609a3700 1 -- 172.21.15.143:0/920985470 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f305c0b14c0 msgr2=0x7f305c0d03a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:26.230 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.224+0000 7f30609a3700 1 --2- 172.21.15.143:0/920985470 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f305c0b14c0 0x7f305c0d03a0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f3054009d90 tx=0x7f305400b330 comp rx=0 tx=0).stop 2023-11-08T19:26:26.230 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.224+0000 7f30609a3700 1 -- 172.21.15.143:0/920985470 shutdown_connections 2023-11-08T19:26:26.230 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.224+0000 7f30609a3700 1 --2- 172.21.15.143:0/920985470 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f305c0b14c0 0x7f305c0d03a0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:26.230 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.224+0000 7f30609a3700 1 -- 172.21.15.143:0/920985470 >> 172.21.15.143:0/920985470 conn(0x7f305c08b000 msgr2=0x7f305c09ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:26.231 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.224+0000 7f30609a3700 1 -- 172.21.15.143:0/920985470 shutdown_connections 2023-11-08T19:26:26.231 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.224+0000 7f30609a3700 1 -- 172.21.15.143:0/920985470 wait complete. 2023-11-08T19:26:26.234 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.225+0000 7f30609a3700 1 Processor -- start 2023-11-08T19:26:26.234 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.225+0000 7f30609a3700 1 -- start start 2023-11-08T19:26:26.235 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.226+0000 7f30609a3700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f305c0b14c0 0x7f305c152070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:26.235 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.226+0000 7f30609a3700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f305c0d1c70 0x7f305c151420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:26.235 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.226+0000 7f30609a3700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f305c15ba30 0x7f305c0c6e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:26.235 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.226+0000 7f30609a3700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f305c0c9970 con 0x7f305c15ba30 2023-11-08T19:26:26.236 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.226+0000 7f30609a3700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f305c0c9660 con 0x7f305c0d1c70 2023-11-08T19:26:26.236 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.226+0000 7f30609a3700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f305c151960 con 0x7f305c0b14c0 2023-11-08T19:26:26.236 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.226+0000 7f305ad9d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f305c15ba30 0x7f305c0c6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:26.236 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.226+0000 7f305ad9d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f305c15ba30 0x7f305c0c6e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51572/0 (socket says 172.21.15.143:51572) 2023-11-08T19:26:26.236 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.226+0000 7f305ad9d700 1 -- 172.21.15.143:0/4286047098 learned_addr learned my addr 172.21.15.143:0/4286047098 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:26.237 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.226+0000 7f305a59c700 1 --2- 172.21.15.143:0/4286047098 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f305c0b14c0 0x7f305c152070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:26.237 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.226+0000 7f3059d9b700 1 --2- 172.21.15.143:0/4286047098 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f305c0d1c70 0x7f305c151420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:26.237 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.226+0000 7f305ad9d700 1 -- 172.21.15.143:0/4286047098 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f305c0b14c0 msgr2=0x7f305c152070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:26.237 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.226+0000 7f305ad9d700 1 --2- 172.21.15.143:0/4286047098 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f305c0b14c0 0x7f305c152070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:26.237 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.226+0000 7f305ad9d700 1 -- 172.21.15.143:0/4286047098 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f305c0d1c70 msgr2=0x7f305c151420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:26.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.226+0000 7f305ad9d700 1 --2- 172.21.15.143:0/4286047098 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f305c0d1c70 0x7f305c151420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:26.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.226+0000 7f305ad9d700 1 -- 172.21.15.143:0/4286047098 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3054009a70 con 0x7f305c15ba30 2023-11-08T19:26:26.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.227+0000 7f305ad9d700 1 --2- 172.21.15.143:0/4286047098 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f305c15ba30 0x7f305c0c6e40 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f305c08c000 tx=0x7f3054004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:26.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.227+0000 7f304affd700 1 -- 172.21.15.143:0/4286047098 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3054012630 con 0x7f305c15ba30 2023-11-08T19:26:26.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.227+0000 7f304affd700 1 -- 172.21.15.143:0/4286047098 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30540127b0 con 0x7f305c15ba30 2023-11-08T19:26:26.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.227+0000 7f30609a3700 1 -- 172.21.15.143:0/4286047098 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f305c0c7380 con 0x7f305c15ba30 2023-11-08T19:26:26.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.227+0000 7f304affd700 1 -- 172.21.15.143:0/4286047098 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3054012b60 con 0x7f305c15ba30 2023-11-08T19:26:26.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.227+0000 7f30609a3700 1 -- 172.21.15.143:0/4286047098 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f305c0c7900 con 0x7f305c15ba30 2023-11-08T19:26:26.240 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.228+0000 7f30609a3700 1 -- 172.21.15.143:0/4286047098 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f305c0c9ab0 con 0x7f305c15ba30 2023-11-08T19:26:26.240 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.228+0000 7f304affd700 1 -- 172.21.15.143:0/4286047098 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3054012cc0 con 0x7f305c15ba30 2023-11-08T19:26:26.240 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.229+0000 7f304affd700 1 --2- 172.21.15.143:0/4286047098 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f303c06af00 0x7f303c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:26.240 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.229+0000 7f304affd700 1 -- 172.21.15.143:0/4286047098 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f305408bd50 con 0x7f305c15ba30 2023-11-08T19:26:26.241 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.229+0000 7f305a59c700 1 --2- 172.21.15.143:0/4286047098 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f303c06af00 0x7f303c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:26.241 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.229+0000 7f305a59c700 1 --2- 172.21.15.143:0/4286047098 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f303c06af00 0x7f303c06d3c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f304c000f60 tx=0x7f304c007c90 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:26.241 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.233+0000 7f304affd700 1 -- 172.21.15.143:0/4286047098 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f305c0c9ab0 con 0x7f305c15ba30 2023-11-08T19:26:26.424 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.422+0000 7f30609a3700 1 -- 172.21.15.143:0/4286047098 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f305c0c9ab0 con 0x7f305c15ba30 2023-11-08T19:26:26.425 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.424+0000 7f304affd700 1 -- 172.21.15.143:0/4286047098 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) v1 ==== 74+0+18383 (secure 0 0 0) 0x7f305c0c9ab0 con 0x7f305c15ba30 2023-11-08T19:26:26.425 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:26:26.426 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":47,"fsid":"968483af-77fc-4068-ad7c-5c2a1a4613d7","created":"2023-11-08T19:24:32.826484+0000","modified":"2023-11-08T19:26:25.128323+0000","last_up_change":"2023-11-08T19:24:43.573880+0000","last_in_change":"2023-11-08T19:24:35.314525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:44.551530+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"snap-test-pool","create_time":"2023-11-08T19:26:02.874384+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":36,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-11-08T19:26:22.869780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-11-08T19:26:24.093179+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bae98ec1-d94e-403d-a220-a522a53ea6f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6809","nonce":1487830566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6811","nonce":1487830566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6815","nonce":1487830566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6813","nonce":1487830566}]},"public_addr":"172.21.15.143:6809/1487830566","cluster_addr":"172.21.15.143:6811/1487830566","heartbeat_back_addr":"172.21.15.143:6815/1487830566","heartbeat_front_addr":"172.21.15.143:6813/1487830566","state":["exists","up"]},{"osd":1,"uuid":"5cb6538a-1ccb-4322-8633-09f66af0b7e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6801","nonce":787341397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6803","nonce":787341397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6807","nonce":787341397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6805","nonce":787341397}]},"public_addr":"172.21.15.143:6801/787341397","cluster_addr":"172.21.15.143:6803/787341397","heartbeat_back_addr":"172.21.15.143:6807/787341397","heartbeat_front_addr":"172.21.15.143:6805/787341397","state":["exists","up"]},{"osd":2,"uuid":"cdbf522c-60ef-4d60-9653-8dcb529c101a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6822","nonce":2301325597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6826","nonce":2301325597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6831","nonce":2301325597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6829","nonce":2301325597}]},"public_addr":"172.21.15.143:6822/2301325597","cluster_addr":"172.21.15.143:6826/2301325597","heartbeat_back_addr":"172.21.15.143:6831/2301325597","heartbeat_front_addr":"172.21.15.143:6829/2301325597","state":["exists","up"]},{"osd":3,"uuid":"4e2783cf-ebb9-4dd8-8245-bc1d17ce4325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6817","nonce":2021504428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6819","nonce":2021504428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6827","nonce":2021504428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6823","nonce":2021504428}]},"public_addr":"172.21.15.143:6817/2021504428","cluster_addr":"172.21.15.143:6819/2021504428","heartbeat_back_addr":"172.21.15.143:6827/2021504428","heartbeat_front_addr":"172.21.15.143:6823/2021504428","state":["exists","up"]},{"osd":4,"uuid":"691256fa-9b77-49b9-a346-7ae1adf0ca80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6801","nonce":1993380595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6803","nonce":1993380595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6807","nonce":1993380595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6805","nonce":1993380595}]},"public_addr":"172.21.15.174:6801/1993380595","cluster_addr":"172.21.15.174:6803/1993380595","heartbeat_back_addr":"172.21.15.174:6807/1993380595","heartbeat_front_addr":"172.21.15.174:6805/1993380595","state":["exists","up"]},{"osd":5,"uuid":"756fc352-48c9-46ce-b52e-0541fe6f8525","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6809","nonce":1777923303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6811","nonce":1777923303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6815","nonce":1777923303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6813","nonce":1777923303}]},"public_addr":"172.21.15.174:6809/1777923303","cluster_addr":"172.21.15.174:6811/1777923303","heartbeat_back_addr":"172.21.15.174:6815/1777923303","heartbeat_front_addr":"172.21.15.174:6813/1777923303","state":["exists","up"]},{"osd":6,"uuid":"fab2ac2b-ed97-4cde-9825-b480eabf8b8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6825","nonce":2755130316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6827","nonce":2755130316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6831","nonce":2755130316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6829","nonce":2755130316}]},"public_addr":"172.21.15.174:6825/2755130316","cluster_addr":"172.21.15.174:6827/2755130316","heartbeat_back_addr":"172.21.15.174:6831/2755130316","heartbeat_front_addr":"172.21.15.174:6829/2755130316","state":["exists","up"]},{"osd":7,"uuid":"3de9b797-2ab6-4d41-a863-7e7d638fd703","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6817","nonce":2847141411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6819","nonce":2847141411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6823","nonce":2847141411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6821","nonce":2847141411}]},"public_addr":"172.21.15.174:6817/2847141411","cluster_addr":"172.21.15.174:6819/2847141411","heartbeat_back_addr":"172.21.15.174:6823/2847141411","heartbeat_front_addr":"172.21.15.174:6821/2847141411","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.477076+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.496890+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.539305+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.454673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.538881+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.521294+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.520712+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.540404+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:26:26.428 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.426+0000 7f30609a3700 1 -- 172.21.15.143:0/4286047098 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f303c06af00 msgr2=0x7f303c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:26.428 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.426+0000 7f30609a3700 1 --2- 172.21.15.143:0/4286047098 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f303c06af00 0x7f303c06d3c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f304c000f60 tx=0x7f304c007c90 comp rx=0 tx=0).stop 2023-11-08T19:26:26.428 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.426+0000 7f30609a3700 1 -- 172.21.15.143:0/4286047098 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f305c15ba30 msgr2=0x7f305c0c6e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:26.428 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.426+0000 7f30609a3700 1 --2- 172.21.15.143:0/4286047098 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f305c15ba30 0x7f305c0c6e40 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f305c08c000 tx=0x7f3054004ad0 comp rx=0 tx=0).stop 2023-11-08T19:26:26.428 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.426+0000 7f30609a3700 1 -- 172.21.15.143:0/4286047098 shutdown_connections 2023-11-08T19:26:26.429 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.426+0000 7f30609a3700 1 --2- 172.21.15.143:0/4286047098 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f303c06af00 0x7f303c06d3c0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:26.429 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.426+0000 7f30609a3700 1 --2- 172.21.15.143:0/4286047098 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f305c0b14c0 0x7f305c152070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:26.429 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.426+0000 7f30609a3700 1 --2- 172.21.15.143:0/4286047098 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f305c15ba30 0x7f305c0c6e40 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:26.429 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.426+0000 7f30609a3700 1 --2- 172.21.15.143:0/4286047098 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f305c0d1c70 0x7f305c151420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:26.429 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.426+0000 7f30609a3700 1 -- 172.21.15.143:0/4286047098 >> 172.21.15.143:0/4286047098 conn(0x7f305c08b000 msgr2=0x7f305c0af7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:26.430 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.426+0000 7f30609a3700 1 -- 172.21.15.143:0/4286047098 shutdown_connections 2023-11-08T19:26:26.430 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.426+0000 7f30609a3700 1 -- 172.21.15.143:0/4286047098 wait complete. 2023-11-08T19:26:26.440 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-08T19:26:26.563 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.561+0000 7f7403974700 1 Processor -- start 2023-11-08T19:26:26.563 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.562+0000 7f7403974700 1 -- start start 2023-11-08T19:26:26.566 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.563+0000 7f7403974700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f73fc15dba0 0x7f73fc06c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:26.566 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.563+0000 7f7403974700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f73fc0b22e0 0x7f73fc0b2750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:26.566 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.563+0000 7f7403974700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73fc161890 con 0x7f73fc06b240 2023-11-08T19:26:26.567 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.563+0000 7f7403974700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73fc0b2c90 con 0x7f73fc15dba0 2023-11-08T19:26:26.567 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.563+0000 7f7403974700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73fc06b740 con 0x7f73fc0b22e0 2023-11-08T19:26:26.567 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.563+0000 7f7401710700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f73fc15dba0 0x7f73fc06c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:26.567 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.563+0000 7f7400f0f700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f73fc06b240 0x7f73fc0caff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:47730/0 (socket says 172.21.15.143:47730) 2023-11-08T19:26:26.568 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.563+0000 7f7401710700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f73fc15dba0 0x7f73fc06c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51576/0 (socket says 172.21.15.143:51576) 2023-11-08T19:26:26.568 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.563+0000 7f7400f0f700 1 -- 172.21.15.143:0/2841770419 learned_addr learned my addr 172.21.15.143:0/2841770419 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:26:26.568 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.563+0000 7f7401f11700 1 --2- 172.21.15.143:0/2841770419 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f73fc0b22e0 0x7f73fc0b2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:26.568 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.563+0000 7f7401710700 1 -- 172.21.15.143:0/2841770419 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f73fc0b22e0 msgr2=0x7f73fc0b2750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:26.568 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.563+0000 7f7401710700 1 --2- 172.21.15.143:0/2841770419 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f73fc0b22e0 0x7f73fc0b2750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:26.569 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.563+0000 7f7401710700 1 -- 172.21.15.143:0/2841770419 >> v1:172.21.15.143:6789/0 conn(0x7f73fc06b240 legacy=0x7f73fc0caff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:26.569 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.563+0000 7f7401710700 1 -- 172.21.15.143:0/2841770419 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73fc06ba20 con 0x7f73fc15dba0 2023-11-08T19:26:26.569 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.563+0000 7f7401710700 1 --2- 172.21.15.143:0/2841770419 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f73fc15dba0 0x7f73fc06c610 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f73ec009840 tx=0x7f73ec009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe27287301086113 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:26.570 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.563+0000 7f73f3fff700 1 -- 172.21.15.143:0/2841770419 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73ec004230 con 0x7f73fc15dba0 2023-11-08T19:26:26.570 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.563+0000 7f73f3fff700 1 -- 172.21.15.143:0/2841770419 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73ec004390 con 0x7f73fc15dba0 2023-11-08T19:26:26.570 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.563+0000 7f73f3fff700 1 -- 172.21.15.143:0/2841770419 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73ec0036a0 con 0x7f73fc15dba0 2023-11-08T19:26:26.570 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.564+0000 7f7403974700 1 -- 172.21.15.143:0/2841770419 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f73fc15dba0 msgr2=0x7f73fc06c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:26.571 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.564+0000 7f7403974700 1 --2- 172.21.15.143:0/2841770419 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f73fc15dba0 0x7f73fc06c610 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f73ec009840 tx=0x7f73ec009b50 comp rx=0 tx=0).stop 2023-11-08T19:26:26.571 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.564+0000 7f7403974700 1 -- 172.21.15.143:0/2841770419 shutdown_connections 2023-11-08T19:26:26.571 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.564+0000 7f7403974700 1 --2- 172.21.15.143:0/2841770419 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f73fc0b22e0 0x7f73fc0b2750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:26.571 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.564+0000 7f7403974700 1 --2- 172.21.15.143:0/2841770419 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f73fc15dba0 0x7f73fc06c610 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:26.571 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.564+0000 7f7403974700 1 -- 172.21.15.143:0/2841770419 >> 172.21.15.143:0/2841770419 conn(0x7f73fc0cd2e0 msgr2=0x7f73fc0cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:26.572 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.564+0000 7f7403974700 1 -- 172.21.15.143:0/2841770419 shutdown_connections 2023-11-08T19:26:26.572 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.564+0000 7f7403974700 1 -- 172.21.15.143:0/2841770419 wait complete. 2023-11-08T19:26:26.576 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.565+0000 7f7403974700 1 Processor -- start 2023-11-08T19:26:26.577 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.565+0000 7f7403974700 1 -- start start 2023-11-08T19:26:26.577 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.566+0000 7f7403974700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f73fc06b240 0x7f73fc15abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:26.578 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.566+0000 7f7403974700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f73fc0b22e0 0x7f73fc15b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:26.578 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.566+0000 7f7403974700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f73fc15dba0 0x7f73fc1f3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:26.579 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.566+0000 7f7403974700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73fc0c7930 con 0x7f73fc0b22e0 2023-11-08T19:26:26.579 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.566+0000 7f7403974700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73fc0af740 con 0x7f73fc06b240 2023-11-08T19:26:26.579 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.566+0000 7f7403974700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73fc15eee0 con 0x7f73fc15dba0 2023-11-08T19:26:26.580 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.566+0000 7f7400f0f700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f73fc0b22e0 0x7f73fc15b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:26.580 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.566+0000 7f7400f0f700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f73fc0b22e0 0x7f73fc15b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51584/0 (socket says 172.21.15.143:51584) 2023-11-08T19:26:26.580 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.566+0000 7f7400f0f700 1 -- 172.21.15.143:0/3739093375 learned_addr learned my addr 172.21.15.143:0/3739093375 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:26.581 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.566+0000 7f7401710700 1 --2- 172.21.15.143:0/3739093375 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f73fc06b240 0x7f73fc15abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:26.581 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.566+0000 7f7401f11700 1 --2- 172.21.15.143:0/3739093375 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f73fc15dba0 0x7f73fc1f3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:26.581 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.566+0000 7f7400f0f700 1 -- 172.21.15.143:0/3739093375 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f73fc15dba0 msgr2=0x7f73fc1f3ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:26.581 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.566+0000 7f7400f0f700 1 --2- 172.21.15.143:0/3739093375 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f73fc15dba0 0x7f73fc1f3ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:26.582 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.566+0000 7f7400f0f700 1 -- 172.21.15.143:0/3739093375 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f73fc06b240 msgr2=0x7f73fc15abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:26.582 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.566+0000 7f7400f0f700 1 --2- 172.21.15.143:0/3739093375 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f73fc06b240 0x7f73fc15abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:26.582 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.566+0000 7f7400f0f700 1 -- 172.21.15.143:0/3739093375 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73ec0094a0 con 0x7f73fc0b22e0 2023-11-08T19:26:26.583 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.567+0000 7f7400f0f700 1 --2- 172.21.15.143:0/3739093375 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f73fc0b22e0 0x7f73fc15b130 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f73f800c7b0 tx=0x7f73f800cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:26.583 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.567+0000 7f73f1ffb700 1 -- 172.21.15.143:0/3739093375 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73f8004c80 con 0x7f73fc0b22e0 2023-11-08T19:26:26.583 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.567+0000 7f73f1ffb700 1 -- 172.21.15.143:0/3739093375 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73f8004de0 con 0x7f73fc0b22e0 2023-11-08T19:26:26.584 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.567+0000 7f7403974700 1 -- 172.21.15.143:0/3739093375 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73fc15e2d0 con 0x7f73fc0b22e0 2023-11-08T19:26:26.584 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.567+0000 7f73f1ffb700 1 -- 172.21.15.143:0/3739093375 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73f8007310 con 0x7f73fc0b22e0 2023-11-08T19:26:26.584 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.567+0000 7f7403974700 1 -- 172.21.15.143:0/3739093375 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73fc15e890 con 0x7f73fc0b22e0 2023-11-08T19:26:26.584 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.568+0000 7f7403974700 1 -- 172.21.15.143:0/3739093375 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73fc04eee0 con 0x7f73fc0b22e0 2023-11-08T19:26:26.585 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.568+0000 7f73f1ffb700 1 -- 172.21.15.143:0/3739093375 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f73f8007470 con 0x7f73fc0b22e0 2023-11-08T19:26:26.585 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.569+0000 7f73f1ffb700 1 --2- 172.21.15.143:0/3739093375 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f73e006af00 0x7f73e006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:26.585 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.569+0000 7f73f1ffb700 1 -- 172.21.15.143:0/3739093375 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f73f80894e0 con 0x7f73fc0b22e0 2023-11-08T19:26:26.585 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.569+0000 7f7401710700 1 --2- 172.21.15.143:0/3739093375 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f73e006af00 0x7f73e006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:26.585 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.569+0000 7f7401710700 1 --2- 172.21.15.143:0/3739093375 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f73e006af00 0x7f73e006d3c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f73ec000c00 tx=0x7f73ec008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:26.586 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.573+0000 7f73f1ffb700 1 -- 172.21.15.143:0/3739093375 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f73f8054430 con 0x7f73fc0b22e0 2023-11-08T19:26:26.814 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:26.812+0000 7f7403974700 1 -- 172.21.15.143:0/3739093375 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f73fc0b67e0 con 0x7f73fc0b22e0 2023-11-08T19:26:26.826 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T19:26:26.830 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T19:26:27.761 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:27.759+0000 7f73f1ffb700 1 -- 172.21.15.143:0/3739093375 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v24) v1 ==== 118+0+0 (secure 0 0 0) 0x7f73f8007720 con 0x7f73fc0b22e0 2023-11-08T19:26:27.764 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:27.762+0000 7f7403974700 1 -- 172.21.15.143:0/3739093375 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f73e006af00 msgr2=0x7f73e006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:27.764 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:27.762+0000 7f7403974700 1 --2- 172.21.15.143:0/3739093375 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f73e006af00 0x7f73e006d3c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f73ec000c00 tx=0x7f73ec008030 comp rx=0 tx=0).stop 2023-11-08T19:26:27.765 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:27.763+0000 7f7403974700 1 -- 172.21.15.143:0/3739093375 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f73fc0b22e0 msgr2=0x7f73fc15b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:27.765 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:27.763+0000 7f7403974700 1 --2- 172.21.15.143:0/3739093375 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f73fc0b22e0 0x7f73fc15b130 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f73f800c7b0 tx=0x7f73f800cac0 comp rx=0 tx=0).stop 2023-11-08T19:26:27.765 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:27.763+0000 7f7403974700 1 -- 172.21.15.143:0/3739093375 shutdown_connections 2023-11-08T19:26:27.765 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:27.763+0000 7f7403974700 1 --2- 172.21.15.143:0/3739093375 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f73e006af00 0x7f73e006d3c0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:27.765 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:27.763+0000 7f7403974700 1 --2- 172.21.15.143:0/3739093375 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f73fc15dba0 0x7f73fc1f3ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:27.766 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:27.763+0000 7f7403974700 1 --2- 172.21.15.143:0/3739093375 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f73fc06b240 0x7f73fc15abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:27.766 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:27.763+0000 7f7403974700 1 --2- 172.21.15.143:0/3739093375 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f73fc0b22e0 0x7f73fc15b130 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:27.766 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:27.763+0000 7f7403974700 1 -- 172.21.15.143:0/3739093375 >> 172.21.15.143:0/3739093375 conn(0x7f73fc0cd2e0 msgr2=0x7f73fc0b0a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:27.766 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:27.763+0000 7f7403974700 1 -- 172.21.15.143:0/3739093375 shutdown_connections 2023-11-08T19:26:27.766 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:27.763+0000 7f7403974700 1 -- 172.21.15.143:0/3739093375 wait complete. 2023-11-08T19:26:27.780 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-08T19:26:27.781 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-08T19:26:27.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:27.909+0000 7f82d214f700 1 Processor -- start 2023-11-08T19:26:27.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:27.909+0000 7f82d214f700 1 -- start start 2023-11-08T19:26:27.913 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:27.910+0000 7f82d214f700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f82cc15dba0 0x7f82cc06c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:27.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:27.910+0000 7f82d214f700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f82cc06b240 0x7f82cc0caff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:27.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:27.910+0000 7f82d214f700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82cc161890 con 0x7f82cc0cb530 2023-11-08T19:26:27.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:27.910+0000 7f82d214f700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82cc06b740 con 0x7f82cc06b240 2023-11-08T19:26:27.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:27.910+0000 7f82d214f700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82cc06ba20 con 0x7f82cc15dba0 2023-11-08T19:26:27.915 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:27.910+0000 7f82caffd700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f82cc06b240 0x7f82cc0caff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:27.915 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:27.910+0000 7f82caffd700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f82cc06b240 0x7f82cc0caff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51590/0 (socket says 172.21.15.143:51590) 2023-11-08T19:26:27.915 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:27.910+0000 7f82caffd700 1 -- 172.21.15.143:0/690326721 learned_addr learned my addr 172.21.15.143:0/690326721 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:27.915 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:27.911+0000 7f82caffd700 1 -- 172.21.15.143:0/690326721 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f82cc15dba0 msgr2=0x7f82cc06c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:27.915 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:27.911+0000 7f82caffd700 1 --2- 172.21.15.143:0/690326721 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f82cc15dba0 0x7f82cc06c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:27.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:27.911+0000 7f82caffd700 1 -- 172.21.15.143:0/690326721 >> v1:172.21.15.174:6789/0 conn(0x7f82cc0cb530 legacy=0x7f82cc0b22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:27.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:27.911+0000 7f82caffd700 1 -- 172.21.15.143:0/690326721 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82cc06bd00 con 0x7f82cc06b240 2023-11-08T19:26:27.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:27.911+0000 7f82caffd700 1 --2- 172.21.15.143:0/690326721 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82cc06b240 0x7f82cc0caff0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f82bc0098f0 tx=0x7f82bc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9aef802557d0229 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:27.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:27.911+0000 7f82ca7fc700 1 -- 172.21.15.143:0/690326721 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82bc004230 con 0x7f82cc06b240 2023-11-08T19:26:27.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:27.911+0000 7f82ca7fc700 1 -- 172.21.15.143:0/690326721 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82bc004390 con 0x7f82cc06b240 2023-11-08T19:26:27.917 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:27.911+0000 7f82ca7fc700 1 -- 172.21.15.143:0/690326721 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82bc0036a0 con 0x7f82cc06b240 2023-11-08T19:26:27.917 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:27.912+0000 7f82d214f700 1 -- 172.21.15.143:0/690326721 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82cc06b240 msgr2=0x7f82cc0caff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:27.917 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:27.912+0000 7f82d214f700 1 --2- 172.21.15.143:0/690326721 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82cc06b240 0x7f82cc0caff0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f82bc0098f0 tx=0x7f82bc009c00 comp rx=0 tx=0).stop 2023-11-08T19:26:27.917 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:27.912+0000 7f82d214f700 1 -- 172.21.15.143:0/690326721 shutdown_connections 2023-11-08T19:26:27.918 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:27.912+0000 7f82d214f700 1 --2- 172.21.15.143:0/690326721 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f82cc15dba0 0x7f82cc06c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:27.918 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:27.912+0000 7f82d214f700 1 --2- 172.21.15.143:0/690326721 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82cc06b240 0x7f82cc0caff0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:27.918 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:27.912+0000 7f82d214f700 1 -- 172.21.15.143:0/690326721 >> 172.21.15.143:0/690326721 conn(0x7f82cc0cd2e0 msgr2=0x7f82cc0cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:27.918 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:27.912+0000 7f82d214f700 1 -- 172.21.15.143:0/690326721 shutdown_connections 2023-11-08T19:26:27.918 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:27.912+0000 7f82d214f700 1 -- 172.21.15.143:0/690326721 wait complete. 2023-11-08T19:26:27.921 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:27.913+0000 7f82d214f700 1 Processor -- start 2023-11-08T19:26:27.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:27.913+0000 7f82d214f700 1 -- start start 2023-11-08T19:26:27.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:27.914+0000 7f82d214f700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f82cc06b240 0x7f82cc1f6920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:27.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:27.914+0000 7f82d214f700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82cc0cb530 0x7f82cc15ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:27.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:27.914+0000 7f82d214f700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f82cc15dba0 0x7f82cc14ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:27.923 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:27.914+0000 7f82d214f700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82cc0c9e10 con 0x7f82cc0cb530 2023-11-08T19:26:27.923 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:27.914+0000 7f82d214f700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82cc15f270 con 0x7f82cc06b240 2023-11-08T19:26:27.923 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:27.914+0000 7f82d214f700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82cc1f78c0 con 0x7f82cc15dba0 2023-11-08T19:26:27.923 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:27.914+0000 7f82caffd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82cc0cb530 0x7f82cc15ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:27.923 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:27.914+0000 7f82caffd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82cc0cb530 0x7f82cc15ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51598/0 (socket says 172.21.15.143:51598) 2023-11-08T19:26:27.924 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:27.914+0000 7f82caffd700 1 -- 172.21.15.143:0/4096140120 learned_addr learned my addr 172.21.15.143:0/4096140120 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:27.924 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:27.915+0000 7f82caffd700 1 -- 172.21.15.143:0/4096140120 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f82cc15dba0 msgr2=0x7f82cc14ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:27.924 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:27.915+0000 7f82caffd700 1 --2- 172.21.15.143:0/4096140120 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f82cc15dba0 0x7f82cc14ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:27.924 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:27.915+0000 7f82caffd700 1 -- 172.21.15.143:0/4096140120 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f82cc06b240 msgr2=0x7f82cc1f6920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:27.925 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:27.915+0000 7f82cb7fe700 1 --2- 172.21.15.143:0/4096140120 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f82cc06b240 0x7f82cc1f6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:27.925 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:27.915+0000 7f82caffd700 1 --2- 172.21.15.143:0/4096140120 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f82cc06b240 0x7f82cc1f6920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:27.925 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:27.915+0000 7f82caffd700 1 -- 172.21.15.143:0/4096140120 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82bc009550 con 0x7f82cc0cb530 2023-11-08T19:26:27.925 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:27.915+0000 7f82caffd700 1 --2- 172.21.15.143:0/4096140120 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82cc0cb530 0x7f82cc15ed30 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f82cc067530 tx=0x7f82bc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:27.925 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:27.915+0000 7f82b3fff700 1 -- 172.21.15.143:0/4096140120 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82bc003e70 con 0x7f82cc0cb530 2023-11-08T19:26:27.926 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:27.915+0000 7f82b3fff700 1 -- 172.21.15.143:0/4096140120 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82bc01f070 con 0x7f82cc0cb530 2023-11-08T19:26:27.926 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:27.915+0000 7f82d214f700 1 -- 172.21.15.143:0/4096140120 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82cc14f220 con 0x7f82cc0cb530 2023-11-08T19:26:27.926 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:27.915+0000 7f82b3fff700 1 -- 172.21.15.143:0/4096140120 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82bc00b8a0 con 0x7f82cc0cb530 2023-11-08T19:26:27.926 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:27.915+0000 7f82d214f700 1 -- 172.21.15.143:0/4096140120 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82cc1f7020 con 0x7f82cc0cb530 2023-11-08T19:26:27.926 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:27.916+0000 7f82d214f700 1 -- 172.21.15.143:0/4096140120 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82cc0c9f50 con 0x7f82cc0cb530 2023-11-08T19:26:27.927 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:27.916+0000 7f82b3fff700 1 -- 172.21.15.143:0/4096140120 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f82bc021030 con 0x7f82cc0cb530 2023-11-08T19:26:27.927 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:27.917+0000 7f82b3fff700 1 --2- 172.21.15.143:0/4096140120 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f82b806af00 0x7f82b806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:27.927 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:27.917+0000 7f82b3fff700 1 -- 172.21.15.143:0/4096140120 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f82bc08aff0 con 0x7f82cc0cb530 2023-11-08T19:26:27.927 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:27.917+0000 7f82cb7fe700 1 --2- 172.21.15.143:0/4096140120 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f82b806af00 0x7f82b806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:27.927 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:27.917+0000 7f82cb7fe700 1 --2- 172.21.15.143:0/4096140120 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f82b806af00 0x7f82b806d3c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f82c000a810 tx=0x7f82c0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:27.928 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:27.921+0000 7f82b3fff700 1 -- 172.21.15.143:0/4096140120 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f82cc0c9f50 con 0x7f82cc0cb530 2023-11-08T19:26:28.156 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:28.155+0000 7f82d214f700 1 -- 172.21.15.143:0/4096140120 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f82cc0c9f50 con 0x7f82cc0cb530 2023-11-08T19:26:28.768 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:28.766+0000 7f82b3fff700 1 -- 172.21.15.143:0/4096140120 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v25) v1 ==== 122+0+0 (secure 0 0 0) 0x7f82cc0c9f50 con 0x7f82cc0cb530 2023-11-08T19:26:28.771 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:28.769+0000 7f82d214f700 1 -- 172.21.15.143:0/4096140120 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f82b806af00 msgr2=0x7f82b806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:28.771 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:28.769+0000 7f82d214f700 1 --2- 172.21.15.143:0/4096140120 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f82b806af00 0x7f82b806d3c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f82c000a810 tx=0x7f82c0008040 comp rx=0 tx=0).stop 2023-11-08T19:26:28.771 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:28.769+0000 7f82d214f700 1 -- 172.21.15.143:0/4096140120 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82cc0cb530 msgr2=0x7f82cc15ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:28.771 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:28.769+0000 7f82d214f700 1 --2- 172.21.15.143:0/4096140120 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82cc0cb530 0x7f82cc15ed30 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f82cc067530 tx=0x7f82bc004750 comp rx=0 tx=0).stop 2023-11-08T19:26:28.772 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:28.769+0000 7f82d214f700 1 -- 172.21.15.143:0/4096140120 shutdown_connections 2023-11-08T19:26:28.772 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:28.769+0000 7f82d214f700 1 --2- 172.21.15.143:0/4096140120 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f82b806af00 0x7f82b806d3c0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:28.772 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:28.769+0000 7f82d214f700 1 --2- 172.21.15.143:0/4096140120 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f82cc15dba0 0x7f82cc14ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:28.772 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:28.769+0000 7f82d214f700 1 --2- 172.21.15.143:0/4096140120 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f82cc06b240 0x7f82cc1f6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:28.772 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:28.769+0000 7f82d214f700 1 --2- 172.21.15.143:0/4096140120 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82cc0cb530 0x7f82cc15ed30 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:28.773 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:28.769+0000 7f82d214f700 1 -- 172.21.15.143:0/4096140120 >> 172.21.15.143:0/4096140120 conn(0x7f82cc0cd2e0 msgr2=0x7f82cc15cce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:28.773 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:28.769+0000 7f82d214f700 1 -- 172.21.15.143:0/4096140120 shutdown_connections 2023-11-08T19:26:28.773 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:28.769+0000 7f82d214f700 1 -- 172.21.15.143:0/4096140120 wait complete. 2023-11-08T19:26:28.784 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-08T19:26:28.784 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:26:28.907 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:28.905+0000 7fd0d4ea9700 1 Processor -- start 2023-11-08T19:26:28.907 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:28.905+0000 7fd0d4ea9700 1 -- start start 2023-11-08T19:26:28.909 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:28.906+0000 7fd0d4ea9700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd0d01618f0 0x7fd0d006ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:28.910 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:28.906+0000 7fd0d4ea9700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0d0161df0 con 0x7fd0d015dd00 2023-11-08T19:26:28.910 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:28.906+0000 7fd0d4ea9700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0d01620d0 con 0x7fd0d006b150 2023-11-08T19:26:28.910 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:28.906+0000 7fd0d4ea9700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0d006b600 con 0x7fd0d01618f0 2023-11-08T19:26:28.910 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:28.906+0000 7fd0cdd9b700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd0d01618f0 0x7fd0d006ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:28.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:28.906+0000 7fd0ce59c700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fd0d015dd00 0x7fd0d00cca70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:47758/0 (socket says 172.21.15.143:47758) 2023-11-08T19:26:28.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:28.906+0000 7fd0ce59c700 1 -- 172.21.15.143:0/1619539625 learned_addr learned my addr 172.21.15.143:0/1619539625 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:26:28.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:28.907+0000 7fd0cdd9b700 1 -- 172.21.15.143:0/1619539625 >> v1:172.21.15.174:6789/0 conn(0x7fd0d006b150 legacy=0x7fd0d00cb1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:28.912 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:28.907+0000 7fd0cdd9b700 1 -- 172.21.15.143:0/1619539625 >> v1:172.21.15.143:6789/0 conn(0x7fd0d015dd00 legacy=0x7fd0d00cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:28.912 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:28.907+0000 7fd0cdd9b700 1 -- 172.21.15.143:0/1619539625 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0d01623b0 con 0x7fd0d01618f0 2023-11-08T19:26:28.912 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:28.907+0000 7fd0cd59a700 1 -- 172.21.15.143:0/1619539625 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1775533712 0 0) 0x7fd0d0161df0 con 0x7fd0d015dd00 2023-11-08T19:26:28.913 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:28.907+0000 7fd0cdd9b700 1 --2- 172.21.15.143:0/1619539625 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd0d01618f0 0x7fd0d006ac10 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fd0c0009840 tx=0x7fd0c0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e327c0ba01209ee8 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:28.913 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:28.907+0000 7fd0cd59a700 1 -- 172.21.15.143:0/1619539625 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0c0004230 con 0x7fd0d01618f0 2023-11-08T19:26:28.913 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:28.907+0000 7fd0cd59a700 1 -- 172.21.15.143:0/1619539625 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0c0004390 con 0x7fd0d01618f0 2023-11-08T19:26:28.913 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:28.907+0000 7fd0cd59a700 1 -- 172.21.15.143:0/1619539625 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0c00036a0 con 0x7fd0d01618f0 2023-11-08T19:26:28.913 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:28.907+0000 7fd0d4ea9700 1 -- 172.21.15.143:0/1619539625 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd0d01618f0 msgr2=0x7fd0d006ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:28.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:28.907+0000 7fd0d4ea9700 1 --2- 172.21.15.143:0/1619539625 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd0d01618f0 0x7fd0d006ac10 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fd0c0009840 tx=0x7fd0c0009b50 comp rx=0 tx=0).stop 2023-11-08T19:26:28.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:28.908+0000 7fd0d4ea9700 1 -- 172.21.15.143:0/1619539625 shutdown_connections 2023-11-08T19:26:28.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:28.908+0000 7fd0d4ea9700 1 --2- 172.21.15.143:0/1619539625 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd0d01618f0 0x7fd0d006ac10 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:28.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:28.908+0000 7fd0d4ea9700 1 -- 172.21.15.143:0/1619539625 >> 172.21.15.143:0/1619539625 conn(0x7fd0d0099ab0 msgr2=0x7fd0d00d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:28.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:28.908+0000 7fd0d4ea9700 1 -- 172.21.15.143:0/1619539625 shutdown_connections 2023-11-08T19:26:28.915 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:28.908+0000 7fd0d4ea9700 1 -- 172.21.15.143:0/1619539625 wait complete. 2023-11-08T19:26:28.917 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:28.909+0000 7fd0d4ea9700 1 Processor -- start 2023-11-08T19:26:28.917 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:28.909+0000 7fd0d4ea9700 1 -- start start 2023-11-08T19:26:28.918 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:28.910+0000 7fd0d4ea9700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd0d006b150 0x7fd0d01f6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:28.918 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:28.910+0000 7fd0d4ea9700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd0d015dd00 0x7fd0d0160830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:28.918 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:28.910+0000 7fd0d4ea9700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd0d01618f0 0x7fd0d01f2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:28.918 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:28.910+0000 7fd0d4ea9700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0d0155520 con 0x7fd0d01618f0 2023-11-08T19:26:28.918 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:28.910+0000 7fd0d4ea9700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0d01f7bd0 con 0x7fd0d006b150 2023-11-08T19:26:28.919 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:28.910+0000 7fd0d4ea9700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0d01f7610 con 0x7fd0d015dd00 2023-11-08T19:26:28.919 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:28.910+0000 7fd0ced9d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd0d01618f0 0x7fd0d01f2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:28.919 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:28.910+0000 7fd0ced9d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd0d01618f0 0x7fd0d01f2b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51614/0 (socket says 172.21.15.143:51614) 2023-11-08T19:26:28.919 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:28.910+0000 7fd0ced9d700 1 -- 172.21.15.143:0/1764519576 learned_addr learned my addr 172.21.15.143:0/1764519576 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:28.919 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:28.910+0000 7fd0cdd9b700 1 --2- 172.21.15.143:0/1764519576 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd0d015dd00 0x7fd0d0160830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:28.920 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:28.910+0000 7fd0ce59c700 1 --2- 172.21.15.143:0/1764519576 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd0d006b150 0x7fd0d01f6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:28.920 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:28.910+0000 7fd0ced9d700 1 -- 172.21.15.143:0/1764519576 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd0d015dd00 msgr2=0x7fd0d0160830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:28.920 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:28.910+0000 7fd0ced9d700 1 --2- 172.21.15.143:0/1764519576 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd0d015dd00 0x7fd0d0160830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:28.920 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:28.910+0000 7fd0ced9d700 1 -- 172.21.15.143:0/1764519576 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd0d006b150 msgr2=0x7fd0d01f6ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:28.921 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:28.910+0000 7fd0ced9d700 1 --2- 172.21.15.143:0/1764519576 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd0d006b150 0x7fd0d01f6ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:28.921 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:28.910+0000 7fd0ced9d700 1 -- 172.21.15.143:0/1764519576 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0c00094a0 con 0x7fd0d01618f0 2023-11-08T19:26:28.921 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:28.910+0000 7fd0cdd9b700 1 --2- 172.21.15.143:0/1764519576 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd0d015dd00 0x7fd0d0160830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:26:28.921 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:28.910+0000 7fd0ced9d700 1 --2- 172.21.15.143:0/1764519576 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd0d01618f0 0x7fd0d01f2b80 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fd0c800cc10 tx=0x7fd0c800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:28.921 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:28.910+0000 7fd0beffd700 1 -- 172.21.15.143:0/1764519576 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0c800abb0 con 0x7fd0d01618f0 2023-11-08T19:26:28.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:28.910+0000 7fd0beffd700 1 -- 172.21.15.143:0/1764519576 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0c800ad10 con 0x7fd0d01618f0 2023-11-08T19:26:28.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:28.910+0000 7fd0d4ea9700 1 -- 172.21.15.143:0/1764519576 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0d015e260 con 0x7fd0d01618f0 2023-11-08T19:26:28.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:28.910+0000 7fd0beffd700 1 -- 172.21.15.143:0/1764519576 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0c8012660 con 0x7fd0d01618f0 2023-11-08T19:26:28.923 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:28.911+0000 7fd0d4ea9700 1 -- 172.21.15.143:0/1764519576 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0d00b43b0 con 0x7fd0d01618f0 2023-11-08T19:26:28.924 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:28.911+0000 7fd0d4ea9700 1 -- 172.21.15.143:0/1764519576 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0d00b45a0 con 0x7fd0d01618f0 2023-11-08T19:26:28.924 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:28.912+0000 7fd0beffd700 1 -- 172.21.15.143:0/1764519576 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd0c80127c0 con 0x7fd0d01618f0 2023-11-08T19:26:28.924 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:28.912+0000 7fd0beffd700 1 --2- 172.21.15.143:0/1764519576 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd0b006af00 0x7fd0b006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:28.924 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:28.912+0000 7fd0beffd700 1 -- 172.21.15.143:0/1764519576 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd0c8089ea0 con 0x7fd0d01618f0 2023-11-08T19:26:28.924 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:28.912+0000 7fd0ce59c700 1 --2- 172.21.15.143:0/1764519576 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd0b006af00 0x7fd0b006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:28.925 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:28.913+0000 7fd0ce59c700 1 --2- 172.21.15.143:0/1764519576 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd0b006af00 0x7fd0b006d3c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fd0c4007fd0 tx=0x7fd0c4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:28.925 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:28.916+0000 7fd0beffd700 1 -- 172.21.15.143:0/1764519576 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd0c8054e80 con 0x7fd0d01618f0 2023-11-08T19:26:29.149 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.147+0000 7fd0d4ea9700 1 -- 172.21.15.143:0/1764519576 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd0d004eee0 con 0x7fd0d01618f0 2023-11-08T19:26:29.149 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.148+0000 7fd0beffd700 1 -- 172.21.15.143:0/1764519576 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4925 (secure 0 0 0) 0x7fd0c8016070 con 0x7fd0d01618f0 2023-11-08T19:26:29.150 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:26:29.150 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4988,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3140991763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":3140991763},{"type":"v1","addr":"172.21.15.143:6835","nonce":3140991763}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":4997,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6836/2388011942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2388011942},{"type":"v1","addr":"172.21.15.174:6836","nonce":2388011942}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5000,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/1339111936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6835","nonce":1339111936},{"type":"v1","addr":"172.21.15.174:6837","nonce":1339111936}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T19:26:25.128617+0000","modified":"2023-11-08T19:26:28.156717+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5242},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5242":{"gid":5242,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.143:6834/3591097261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":3591097261},{"type":"v1","addr":"172.21.15.143:6834","nonce":3591097261}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-08T19:26:29.152 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.150+0000 7fd0d4ea9700 1 -- 172.21.15.143:0/1764519576 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd0b006af00 msgr2=0x7fd0b006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:29.152 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.150+0000 7fd0d4ea9700 1 --2- 172.21.15.143:0/1764519576 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd0b006af00 0x7fd0b006d3c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fd0c4007fd0 tx=0x7fd0c4009040 comp rx=0 tx=0).stop 2023-11-08T19:26:29.152 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.150+0000 7fd0d4ea9700 1 -- 172.21.15.143:0/1764519576 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd0d01618f0 msgr2=0x7fd0d01f2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:29.152 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.150+0000 7fd0d4ea9700 1 --2- 172.21.15.143:0/1764519576 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd0d01618f0 0x7fd0d01f2b80 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fd0c800cc10 tx=0x7fd0c800a060 comp rx=0 tx=0).stop 2023-11-08T19:26:29.153 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.150+0000 7fd0d4ea9700 1 -- 172.21.15.143:0/1764519576 shutdown_connections 2023-11-08T19:26:29.153 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.150+0000 7fd0d4ea9700 1 --2- 172.21.15.143:0/1764519576 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd0b006af00 0x7fd0b006d3c0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:29.153 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.150+0000 7fd0d4ea9700 1 --2- 172.21.15.143:0/1764519576 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd0d01618f0 0x7fd0d01f2b80 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:29.153 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.150+0000 7fd0d4ea9700 1 --2- 172.21.15.143:0/1764519576 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd0d006b150 0x7fd0d01f6ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:29.154 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.150+0000 7fd0d4ea9700 1 --2- 172.21.15.143:0/1764519576 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd0d015dd00 0x7fd0d0160830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:29.154 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.150+0000 7fd0d4ea9700 1 -- 172.21.15.143:0/1764519576 >> 172.21.15.143:0/1764519576 conn(0x7fd0d0099ab0 msgr2=0x7fd0d00ad380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:29.154 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.150+0000 7fd0d4ea9700 1 -- 172.21.15.143:0/1764519576 shutdown_connections 2023-11-08T19:26:29.155 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.150+0000 7fd0d4ea9700 1 -- 172.21.15.143:0/1764519576 wait complete. 2023-11-08T19:26:29.155 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 25 2023-11-08T19:26:29.163 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:26:29.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.282+0000 7fce8e69f700 1 Processor -- start 2023-11-08T19:26:29.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.282+0000 7fce8e69f700 1 -- start start 2023-11-08T19:26:29.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.282+0000 7fce8e69f700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fce8806c730 0x7fce880cd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:29.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.282+0000 7fce8e69f700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fce8815de80 0x7fce880cbf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:29.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.282+0000 7fce8e69f700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce880cd480 con 0x7fce8806c030 2023-11-08T19:26:29.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.282+0000 7fce8e69f700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce880cc870 con 0x7fce8815de80 2023-11-08T19:26:29.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.282+0000 7fce8e69f700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce880ccb50 con 0x7fce8806c730 2023-11-08T19:26:29.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.283+0000 7fce8cc3c700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fce8815de80 0x7fce880cbf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:29.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.283+0000 7fce8cc3c700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fce8815de80 0x7fce880cbf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51622/0 (socket says 172.21.15.143:51622) 2023-11-08T19:26:29.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.283+0000 7fce8cc3c700 1 -- 172.21.15.143:0/3907850653 learned_addr learned my addr 172.21.15.143:0/3907850653 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:29.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.283+0000 7fce87fff700 1 --2- 172.21.15.143:0/3907850653 >> v2:172.21.15.174:3300/0 conn(0x7fce8806c730 0x7fce880cd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:29.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.283+0000 7fce8cc3c700 1 -- 172.21.15.143:0/3907850653 >> v2:172.21.15.174:3300/0 conn(0x7fce8806c730 msgr2=0x7fce880cd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:29.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.283+0000 7fce8cc3c700 1 --2- 172.21.15.143:0/3907850653 >> v2:172.21.15.174:3300/0 conn(0x7fce8806c730 0x7fce880cd910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:29.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.283+0000 7fce8cc3c700 1 -- 172.21.15.143:0/3907850653 >> v1:172.21.15.174:6789/0 conn(0x7fce8806c030 legacy=0x7fce8806cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:29.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.283+0000 7fce8cc3c700 1 -- 172.21.15.143:0/3907850653 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce880cce30 con 0x7fce8815de80 2023-11-08T19:26:29.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.283+0000 7fce8cc3c700 1 --2- 172.21.15.143:0/3907850653 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fce8815de80 0x7fce880cbf00 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fce8808bd60 tx=0x7fce8000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc8ce402d96ae496 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:29.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.283+0000 7fce86ffd700 1 -- 172.21.15.143:0/3907850653 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce8000e070 con 0x7fce8815de80 2023-11-08T19:26:29.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.283+0000 7fce86ffd700 1 -- 172.21.15.143:0/3907850653 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce80003a20 con 0x7fce8815de80 2023-11-08T19:26:29.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.283+0000 7fce86ffd700 1 -- 172.21.15.143:0/3907850653 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce80003dd0 con 0x7fce8815de80 2023-11-08T19:26:29.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.284+0000 7fce8e69f700 1 -- 172.21.15.143:0/3907850653 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fce8815de80 msgr2=0x7fce880cbf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:29.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.284+0000 7fce8e69f700 1 --2- 172.21.15.143:0/3907850653 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fce8815de80 0x7fce880cbf00 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fce8808bd60 tx=0x7fce8000b330 comp rx=0 tx=0).stop 2023-11-08T19:26:29.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.284+0000 7fce8e69f700 1 -- 172.21.15.143:0/3907850653 shutdown_connections 2023-11-08T19:26:29.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.284+0000 7fce8e69f700 1 --2- 172.21.15.143:0/3907850653 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fce8815de80 0x7fce880cbf00 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:29.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.284+0000 7fce8e69f700 1 --2- 172.21.15.143:0/3907850653 >> v2:172.21.15.174:3300/0 conn(0x7fce8806c730 0x7fce880cd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:29.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.284+0000 7fce8e69f700 1 -- 172.21.15.143:0/3907850653 >> 172.21.15.143:0/3907850653 conn(0x7fce8808b000 msgr2=0x7fce8809ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:29.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.284+0000 7fce8e69f700 1 -- 172.21.15.143:0/3907850653 shutdown_connections 2023-11-08T19:26:29.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.284+0000 7fce8e69f700 1 -- 172.21.15.143:0/3907850653 wait complete. 2023-11-08T19:26:29.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.285+0000 7fce8e69f700 1 Processor -- start 2023-11-08T19:26:29.294 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.285+0000 7fce8e69f700 1 -- start start 2023-11-08T19:26:29.294 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.286+0000 7fce8e69f700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fce8806c030 0x7fce881f6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:29.295 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.286+0000 7fce8e69f700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fce8806c730 0x7fce881f7300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:29.295 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.286+0000 7fce8e69f700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fce8815de80 0x7fce881602b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:29.295 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.286+0000 7fce8e69f700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce880b5a50 con 0x7fce8806c030 2023-11-08T19:26:29.295 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.286+0000 7fce8e69f700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce880b5d30 con 0x7fce8815de80 2023-11-08T19:26:29.295 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.286+0000 7fce8e69f700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce881f4990 con 0x7fce8806c730 2023-11-08T19:26:29.296 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.286+0000 7fce87fff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fce8806c030 0x7fce881f6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:29.296 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.286+0000 7fce87fff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fce8806c030 0x7fce881f6dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51624/0 (socket says 172.21.15.143:51624) 2023-11-08T19:26:29.296 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.286+0000 7fce87fff700 1 -- 172.21.15.143:0/989772505 learned_addr learned my addr 172.21.15.143:0/989772505 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:29.296 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.286+0000 7fce877fe700 1 --2- 172.21.15.143:0/989772505 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fce8806c730 0x7fce881f7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:29.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.286+0000 7fce8cc3c700 1 --2- 172.21.15.143:0/989772505 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fce8815de80 0x7fce881602b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:29.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.286+0000 7fce87fff700 1 -- 172.21.15.143:0/989772505 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fce8806c730 msgr2=0x7fce881f7300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:29.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.286+0000 7fce87fff700 1 --2- 172.21.15.143:0/989772505 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fce8806c730 0x7fce881f7300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:29.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.286+0000 7fce87fff700 1 -- 172.21.15.143:0/989772505 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fce8815de80 msgr2=0x7fce881602b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:29.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.286+0000 7fce87fff700 1 --2- 172.21.15.143:0/989772505 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fce8815de80 0x7fce881602b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:29.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.286+0000 7fce87fff700 1 -- 172.21.15.143:0/989772505 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce80009b30 con 0x7fce8806c030 2023-11-08T19:26:29.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.286+0000 7fce8cc3c700 1 --2- 172.21.15.143:0/989772505 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fce8815de80 0x7fce881602b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:26:29.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.286+0000 7fce877fe700 1 --2- 172.21.15.143:0/989772505 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fce8806c730 0x7fce881f7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:26:29.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.287+0000 7fce87fff700 1 --2- 172.21.15.143:0/989772505 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fce8806c030 0x7fce881f6dc0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fce7800e810 tx=0x7fce7800ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:29.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.287+0000 7fce84ff9700 1 -- 172.21.15.143:0/989772505 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce78013610 con 0x7fce8806c030 2023-11-08T19:26:29.299 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.287+0000 7fce84ff9700 1 -- 172.21.15.143:0/989772505 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce78013770 con 0x7fce8806c030 2023-11-08T19:26:29.299 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.287+0000 7fce8e69f700 1 -- 172.21.15.143:0/989772505 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce881f4c70 con 0x7fce8806c030 2023-11-08T19:26:29.299 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.287+0000 7fce84ff9700 1 -- 172.21.15.143:0/989772505 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce78013b20 con 0x7fce8806c030 2023-11-08T19:26:29.299 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.287+0000 7fce8e69f700 1 -- 172.21.15.143:0/989772505 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce880b6c40 con 0x7fce8806c030 2023-11-08T19:26:29.299 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.288+0000 7fce8e69f700 1 -- 172.21.15.143:0/989772505 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce8804eee0 con 0x7fce8806c030 2023-11-08T19:26:29.300 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.288+0000 7fce84ff9700 1 -- 172.21.15.143:0/989772505 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fce78013c80 con 0x7fce8806c030 2023-11-08T19:26:29.300 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.288+0000 7fce84ff9700 1 --2- 172.21.15.143:0/989772505 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fce7406af00 0x7fce7406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:29.300 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.289+0000 7fce84ff9700 1 -- 172.21.15.143:0/989772505 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fce78089480 con 0x7fce8806c030 2023-11-08T19:26:29.300 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.289+0000 7fce877fe700 1 --2- 172.21.15.143:0/989772505 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fce7406af00 0x7fce7406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:29.300 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.289+0000 7fce877fe700 1 --2- 172.21.15.143:0/989772505 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fce7406af00 0x7fce7406d3c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fce8806c410 tx=0x7fce7c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:29.301 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.293+0000 7fce84ff9700 1 -- 172.21.15.143:0/989772505 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fce780543e0 con 0x7fce8806c030 2023-11-08T19:26:29.484 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.483+0000 7fce8e69f700 1 -- 172.21.15.143:0/989772505 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fce880c8710 con 0x7fce8806c030 2023-11-08T19:26:29.486 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.484+0000 7fce84ff9700 1 -- 172.21.15.143:0/989772505 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) v1 ==== 74+0+18370 (secure 0 0 0) 0x7fce78057a00 con 0x7fce8806c030 2023-11-08T19:26:29.486 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:26:29.486 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":48,"fsid":"968483af-77fc-4068-ad7c-5c2a1a4613d7","created":"2023-11-08T19:24:32.826484+0000","modified":"2023-11-08T19:26:26.724579+0000","last_up_change":"2023-11-08T19:24:43.573880+0000","last_in_change":"2023-11-08T19:24:35.314525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:44.551530+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"snap-test-pool","create_time":"2023-11-08T19:26:02.874384+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":36,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-11-08T19:26:22.869780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-11-08T19:26:24.093179+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bae98ec1-d94e-403d-a220-a522a53ea6f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6809","nonce":1487830566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6811","nonce":1487830566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6815","nonce":1487830566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6813","nonce":1487830566}]},"public_addr":"172.21.15.143:6809/1487830566","cluster_addr":"172.21.15.143:6811/1487830566","heartbeat_back_addr":"172.21.15.143:6815/1487830566","heartbeat_front_addr":"172.21.15.143:6813/1487830566","state":["exists","up"]},{"osd":1,"uuid":"5cb6538a-1ccb-4322-8633-09f66af0b7e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6801","nonce":787341397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6803","nonce":787341397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6807","nonce":787341397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6805","nonce":787341397}]},"public_addr":"172.21.15.143:6801/787341397","cluster_addr":"172.21.15.143:6803/787341397","heartbeat_back_addr":"172.21.15.143:6807/787341397","heartbeat_front_addr":"172.21.15.143:6805/787341397","state":["exists","up"]},{"osd":2,"uuid":"cdbf522c-60ef-4d60-9653-8dcb529c101a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6822","nonce":2301325597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6826","nonce":2301325597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6831","nonce":2301325597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6829","nonce":2301325597}]},"public_addr":"172.21.15.143:6822/2301325597","cluster_addr":"172.21.15.143:6826/2301325597","heartbeat_back_addr":"172.21.15.143:6831/2301325597","heartbeat_front_addr":"172.21.15.143:6829/2301325597","state":["exists","up"]},{"osd":3,"uuid":"4e2783cf-ebb9-4dd8-8245-bc1d17ce4325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6817","nonce":2021504428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6819","nonce":2021504428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6827","nonce":2021504428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6823","nonce":2021504428}]},"public_addr":"172.21.15.143:6817/2021504428","cluster_addr":"172.21.15.143:6819/2021504428","heartbeat_back_addr":"172.21.15.143:6827/2021504428","heartbeat_front_addr":"172.21.15.143:6823/2021504428","state":["exists","up"]},{"osd":4,"uuid":"691256fa-9b77-49b9-a346-7ae1adf0ca80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6801","nonce":1993380595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6803","nonce":1993380595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6807","nonce":1993380595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6805","nonce":1993380595}]},"public_addr":"172.21.15.174:6801/1993380595","cluster_addr":"172.21.15.174:6803/1993380595","heartbeat_back_addr":"172.21.15.174:6807/1993380595","heartbeat_front_addr":"172.21.15.174:6805/1993380595","state":["exists","up"]},{"osd":5,"uuid":"756fc352-48c9-46ce-b52e-0541fe6f8525","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6809","nonce":1777923303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6811","nonce":1777923303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6815","nonce":1777923303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6813","nonce":1777923303}]},"public_addr":"172.21.15.174:6809/1777923303","cluster_addr":"172.21.15.174:6811/1777923303","heartbeat_back_addr":"172.21.15.174:6815/1777923303","heartbeat_front_addr":"172.21.15.174:6813/1777923303","state":["exists","up"]},{"osd":6,"uuid":"fab2ac2b-ed97-4cde-9825-b480eabf8b8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6825","nonce":2755130316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6827","nonce":2755130316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6831","nonce":2755130316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6829","nonce":2755130316}]},"public_addr":"172.21.15.174:6825/2755130316","cluster_addr":"172.21.15.174:6827/2755130316","heartbeat_back_addr":"172.21.15.174:6831/2755130316","heartbeat_front_addr":"172.21.15.174:6829/2755130316","state":["exists","up"]},{"osd":7,"uuid":"3de9b797-2ab6-4d41-a863-7e7d638fd703","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6817","nonce":2847141411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6819","nonce":2847141411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6823","nonce":2847141411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6821","nonce":2847141411}]},"public_addr":"172.21.15.174:6817/2847141411","cluster_addr":"172.21.15.174:6819/2847141411","heartbeat_back_addr":"172.21.15.174:6823/2847141411","heartbeat_front_addr":"172.21.15.174:6821/2847141411","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.477076+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.496890+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.539305+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.454673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.538881+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.521294+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.520712+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.540404+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:26:29.488 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.487+0000 7fce8e69f700 1 -- 172.21.15.143:0/989772505 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fce7406af00 msgr2=0x7fce7406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:29.488 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.487+0000 7fce8e69f700 1 --2- 172.21.15.143:0/989772505 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fce7406af00 0x7fce7406d3c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fce8806c410 tx=0x7fce7c009040 comp rx=0 tx=0).stop 2023-11-08T19:26:29.489 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.487+0000 7fce8e69f700 1 -- 172.21.15.143:0/989772505 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fce8806c030 msgr2=0x7fce881f6dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:29.489 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.487+0000 7fce8e69f700 1 --2- 172.21.15.143:0/989772505 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fce8806c030 0x7fce881f6dc0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fce7800e810 tx=0x7fce7800ebd0 comp rx=0 tx=0).stop 2023-11-08T19:26:29.489 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.487+0000 7fce8e69f700 1 -- 172.21.15.143:0/989772505 shutdown_connections 2023-11-08T19:26:29.489 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.487+0000 7fce8e69f700 1 --2- 172.21.15.143:0/989772505 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fce7406af00 0x7fce7406d3c0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:29.490 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.487+0000 7fce8e69f700 1 --2- 172.21.15.143:0/989772505 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fce8815de80 0x7fce881602b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:29.490 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.487+0000 7fce8e69f700 1 --2- 172.21.15.143:0/989772505 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fce8806c030 0x7fce881f6dc0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:29.490 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.487+0000 7fce8e69f700 1 --2- 172.21.15.143:0/989772505 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fce8806c730 0x7fce881f7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:29.490 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.487+0000 7fce8e69f700 1 -- 172.21.15.143:0/989772505 >> 172.21.15.143:0/989772505 conn(0x7fce8808b000 msgr2=0x7fce88070bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:29.490 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.487+0000 7fce8e69f700 1 -- 172.21.15.143:0/989772505 shutdown_connections 2023-11-08T19:26:29.491 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.487+0000 7fce8e69f700 1 -- 172.21.15.143:0/989772505 wait complete. 2023-11-08T19:26:29.501 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-08T19:26:29.501 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-08T19:26:29.621 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.619+0000 7f3d5abdd700 1 Processor -- start 2023-11-08T19:26:29.621 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.619+0000 7f3d5abdd700 1 -- start start 2023-11-08T19:26:29.623 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.620+0000 7f3d5abdd700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f3d5415aa30 0x7f3d540cf510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:29.623 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.620+0000 7f3d5abdd700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f3d540cc530 0x7f3d540b26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:29.624 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.620+0000 7f3d5abdd700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d540cf080 con 0x7f3d540b2210 2023-11-08T19:26:29.624 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.620+0000 7f3d5abdd700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d540b2be0 con 0x7f3d5415aa30 2023-11-08T19:26:29.624 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.620+0000 7f3d5abdd700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d540ce470 con 0x7f3d540cc530 2023-11-08T19:26:29.624 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.621+0000 7f3d58979700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f3d5415aa30 0x7f3d540cf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:29.624 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.621+0000 7f3d53fff700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f3d540b2210 0x7f3d540cbe20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:47786/0 (socket says 172.21.15.143:47786) 2023-11-08T19:26:29.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.621+0000 7f3d53fff700 1 -- 172.21.15.143:0/3827879521 learned_addr learned my addr 172.21.15.143:0/3827879521 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:26:29.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.621+0000 7f3d5917a700 1 --2- 172.21.15.143:0/3827879521 >> v2:172.21.15.174:3300/0 conn(0x7f3d540cc530 0x7f3d540b26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:29.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.621+0000 7f3d58979700 1 -- 172.21.15.143:0/3827879521 >> v2:172.21.15.174:3300/0 conn(0x7f3d540cc530 msgr2=0x7f3d540b26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:29.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.621+0000 7f3d58979700 1 --2- 172.21.15.143:0/3827879521 >> v2:172.21.15.174:3300/0 conn(0x7f3d540cc530 0x7f3d540b26a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:29.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.621+0000 7f3d58979700 1 -- 172.21.15.143:0/3827879521 >> v1:172.21.15.143:6789/0 conn(0x7f3d540b2210 legacy=0x7f3d540cbe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:29.626 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.621+0000 7f3d58979700 1 -- 172.21.15.143:0/3827879521 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d540ce750 con 0x7f3d5415aa30 2023-11-08T19:26:29.626 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.621+0000 7f3d58979700 1 --2- 172.21.15.143:0/3827879521 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3d5415aa30 0x7f3d540cf510 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f3d540709e0 tx=0x7f3d48009b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bc4f03c2ddd79a9 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:29.626 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.621+0000 7f3d537fe700 1 -- 172.21.15.143:0/3827879521 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d48005e30 con 0x7f3d5415aa30 2023-11-08T19:26:29.626 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.621+0000 7f3d537fe700 1 -- 172.21.15.143:0/3827879521 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d48006030 con 0x7f3d5415aa30 2023-11-08T19:26:29.626 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.621+0000 7f3d537fe700 1 -- 172.21.15.143:0/3827879521 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d480063c0 con 0x7f3d5415aa30 2023-11-08T19:26:29.627 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.622+0000 7f3d5abdd700 1 -- 172.21.15.143:0/3827879521 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3d5415aa30 msgr2=0x7f3d540cf510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:29.627 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.622+0000 7f3d5abdd700 1 --2- 172.21.15.143:0/3827879521 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3d5415aa30 0x7f3d540cf510 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f3d540709e0 tx=0x7f3d48009b00 comp rx=0 tx=0).stop 2023-11-08T19:26:29.627 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.622+0000 7f3d5abdd700 1 -- 172.21.15.143:0/3827879521 shutdown_connections 2023-11-08T19:26:29.627 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.622+0000 7f3d5abdd700 1 --2- 172.21.15.143:0/3827879521 >> v2:172.21.15.174:3300/0 conn(0x7f3d540cc530 0x7f3d540b26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:29.627 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.622+0000 7f3d5abdd700 1 --2- 172.21.15.143:0/3827879521 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3d5415aa30 0x7f3d540cf510 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:29.628 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.622+0000 7f3d5abdd700 1 -- 172.21.15.143:0/3827879521 >> 172.21.15.143:0/3827879521 conn(0x7f3d54099ab0 msgr2=0x7f3d540d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:29.628 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.622+0000 7f3d5abdd700 1 -- 172.21.15.143:0/3827879521 shutdown_connections 2023-11-08T19:26:29.628 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.622+0000 7f3d5abdd700 1 -- 172.21.15.143:0/3827879521 wait complete. 2023-11-08T19:26:29.630 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.623+0000 7f3d5abdd700 1 Processor -- start 2023-11-08T19:26:29.631 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.623+0000 7f3d5abdd700 1 -- start start 2023-11-08T19:26:29.631 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.624+0000 7f3d5abdd700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3d540b2210 0x7f3d5415d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:29.631 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.624+0000 7f3d5abdd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3d540cc530 0x7f3d5415b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:29.631 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.624+0000 7f3d5abdd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3d5415aa30 0x7f3d5415b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:29.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.624+0000 7f3d5abdd700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d540c8f10 con 0x7f3d5415aa30 2023-11-08T19:26:29.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.624+0000 7f3d5abdd700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d5415dd00 con 0x7f3d540cc530 2023-11-08T19:26:29.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.624+0000 7f3d5abdd700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d5415dfe0 con 0x7f3d540b2210 2023-11-08T19:26:29.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.624+0000 7f3d5917a700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3d5415aa30 0x7f3d5415b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:29.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.625+0000 7f3d5917a700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3d5415aa30 0x7f3d5415b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51642/0 (socket says 172.21.15.143:51642) 2023-11-08T19:26:29.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.625+0000 7f3d5917a700 1 -- 172.21.15.143:0/3220661129 learned_addr learned my addr 172.21.15.143:0/3220661129 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:29.633 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.625+0000 7f3d53fff700 1 --2- 172.21.15.143:0/3220661129 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3d540cc530 0x7f3d5415b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:29.633 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.625+0000 7f3d5917a700 1 -- 172.21.15.143:0/3220661129 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3d540b2210 msgr2=0x7f3d5415d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:29.633 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.625+0000 7f3d5917a700 1 --2- 172.21.15.143:0/3220661129 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3d540b2210 0x7f3d5415d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:29.633 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.625+0000 7f3d5917a700 1 -- 172.21.15.143:0/3220661129 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3d540cc530 msgr2=0x7f3d5415b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:29.633 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.625+0000 7f3d5917a700 1 --2- 172.21.15.143:0/3220661129 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3d540cc530 0x7f3d5415b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:29.634 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.625+0000 7f3d5917a700 1 -- 172.21.15.143:0/3220661129 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d48009430 con 0x7f3d5415aa30 2023-11-08T19:26:29.634 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.625+0000 7f3d5917a700 1 --2- 172.21.15.143:0/3220661129 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3d5415aa30 0x7f3d5415b7b0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c007e40 tx=0x7f3d4c00e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:29.634 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.625+0000 7f3d517fa700 1 -- 172.21.15.143:0/3220661129 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d4c00eea0 con 0x7f3d5415aa30 2023-11-08T19:26:29.634 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.625+0000 7f3d517fa700 1 -- 172.21.15.143:0/3220661129 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d4c014420 con 0x7f3d5415aa30 2023-11-08T19:26:29.634 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.625+0000 7f3d5abdd700 1 -- 172.21.15.143:0/3220661129 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d540c88c0 con 0x7f3d5415aa30 2023-11-08T19:26:29.635 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.625+0000 7f3d517fa700 1 -- 172.21.15.143:0/3220661129 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d4c0147b0 con 0x7f3d5415aa30 2023-11-08T19:26:29.635 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.625+0000 7f3d5abdd700 1 -- 172.21.15.143:0/3220661129 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d5415d810 con 0x7f3d5415aa30 2023-11-08T19:26:29.635 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.626+0000 7f3d5abdd700 1 -- 172.21.15.143:0/3220661129 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d5404eee0 con 0x7f3d5415aa30 2023-11-08T19:26:29.635 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.627+0000 7f3d517fa700 1 -- 172.21.15.143:0/3220661129 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3d4c014910 con 0x7f3d5415aa30 2023-11-08T19:26:29.635 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.627+0000 7f3d517fa700 1 --2- 172.21.15.143:0/3220661129 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f3d3806af00 0x7f3d3806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:29.635 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.627+0000 7f3d517fa700 1 -- 172.21.15.143:0/3220661129 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3d4c089f20 con 0x7f3d5415aa30 2023-11-08T19:26:29.636 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.627+0000 7f3d58979700 1 --2- 172.21.15.143:0/3220661129 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f3d3806af00 0x7f3d3806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:29.636 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.628+0000 7f3d58979700 1 --2- 172.21.15.143:0/3220661129 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f3d3806af00 0x7f3d3806d3c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f3d48013040 tx=0x7f3d48010000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:29.636 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.631+0000 7f3d517fa700 1 -- 172.21.15.143:0/3220661129 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3d4c054e70 con 0x7f3d5415aa30 2023-11-08T19:26:29.818 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.817+0000 7f3d5abdd700 1 -- 172.21.15.143:0/3220661129 --> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3d540ced10 con 0x7f3d3806af00 2023-11-08T19:26:29.828 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.827+0000 7f3d517fa700 1 -- 172.21.15.143:0/3220661129 <== mgr.4099 v2:172.21.15.174:6832/4037280178 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+356754 (secure 0 0 0) 0x7f3d48020070 con 0x7f3d3806af00 2023-11-08T19:26:29.833 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:26:29.836 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.830+0000 7f3d5abdd700 1 -- 172.21.15.143:0/3220661129 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f3d3806af00 msgr2=0x7f3d3806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:29.836 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.830+0000 7f3d5abdd700 1 --2- 172.21.15.143:0/3220661129 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f3d3806af00 0x7f3d3806d3c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f3d48013040 tx=0x7f3d48010000 comp rx=0 tx=0).stop 2023-11-08T19:26:29.836 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.830+0000 7f3d5abdd700 1 -- 172.21.15.143:0/3220661129 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3d5415aa30 msgr2=0x7f3d5415b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:29.837 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.830+0000 7f3d5abdd700 1 --2- 172.21.15.143:0/3220661129 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3d5415aa30 0x7f3d5415b7b0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c007e40 tx=0x7f3d4c00e3c0 comp rx=0 tx=0).stop 2023-11-08T19:26:29.837 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.831+0000 7f3d5abdd700 1 -- 172.21.15.143:0/3220661129 shutdown_connections 2023-11-08T19:26:29.837 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.831+0000 7f3d5abdd700 1 --2- 172.21.15.143:0/3220661129 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f3d3806af00 0x7f3d3806d3c0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:29.837 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.831+0000 7f3d5abdd700 1 --2- 172.21.15.143:0/3220661129 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3d540b2210 0x7f3d5415d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:29.837 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.831+0000 7f3d5abdd700 1 --2- 172.21.15.143:0/3220661129 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3d5415aa30 0x7f3d5415b7b0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:29.838 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.831+0000 7f3d5abdd700 1 --2- 172.21.15.143:0/3220661129 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3d540cc530 0x7f3d5415b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:29.838 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.831+0000 7f3d5abdd700 1 -- 172.21.15.143:0/3220661129 >> 172.21.15.143:0/3220661129 conn(0x7f3d54099ab0 msgr2=0x7f3d540af850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:29.838 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.831+0000 7f3d5abdd700 1 -- 172.21.15.143:0/3220661129 shutdown_connections 2023-11-08T19:26:29.838 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:29.831+0000 7f3d5abdd700 1 -- 172.21.15.143:0/3220661129 wait complete. 2023-11-08T19:26:29.838 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2023-11-08T19:26:29.844 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":93,"stamp":"2023-11-08T19:26:28.533968+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":86,"acting":86,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":66,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":221384,"kb_used_data":5216,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753336,"statfs":{"total":773094113280,"available":772867416064,"internally_reserved":0,"allocated":5341184,"data_stored":2894828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.810858"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:23.712793+0000","last_change":"2023-11-08T19:26:23.712793+0000","last_active":"2023-11-08T19:26:23.712793+0000","last_peered":"2023-11-08T19:26:23.712793+0000","last_clean":"2023-11-08T19:26:23.712793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:23.712793+0000","last_undegraded":"2023-11-08T19:26:23.712793+0000","last_fullsized":"2023-11-08T19:26:23.712793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":22,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-11-08T19:26:20.697680+0000","last_change":"2023-11-08T19:26:06.297255+0000","last_active":"2023-11-08T19:26:20.697680+0000","last_peered":"2023-11-08T19:26:20.697680+0000","last_clean":"2023-11-08T19:26:20.697680+0000","last_became_active":"2023-11-08T19:26:04.603979+0000","last_became_peered":"2023-11-08T19:26:04.603979+0000","last_unstale":"2023-11-08T19:26:20.697680+0000","last_undegraded":"2023-11-08T19:26:20.697680+0000","last_fullsized":"2023-11-08T19:26:20.697680+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:40:11.164951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079032099999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:23.712793+0000","last_change":"2023-11-08T19:26:23.712793+0000","last_active":"2023-11-08T19:26:23.712793+0000","last_peered":"2023-11-08T19:26:23.712793+0000","last_clean":"2023-11-08T19:26:23.712793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:23.712793+0000","last_undegraded":"2023-11-08T19:26:23.712793+0000","last_fullsized":"2023-11-08T19:26:23.712793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":25,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-08T19:26:20.730391+0000","last_change":"2023-11-08T19:26:05.609625+0000","last_active":"2023-11-08T19:26:20.730391+0000","last_peered":"2023-11-08T19:26:20.730391+0000","last_clean":"2023-11-08T19:26:20.730391+0000","last_became_active":"2023-11-08T19:26:04.606312+0000","last_became_peered":"2023-11-08T19:26:04.606312+0000","last_unstale":"2023-11-08T19:26:20.730391+0000","last_undegraded":"2023-11-08T19:26:20.730391+0000","last_fullsized":"2023-11-08T19:26:20.730391+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:54:42.032901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000505935,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:23.712793+0000","last_change":"2023-11-08T19:26:23.712793+0000","last_active":"2023-11-08T19:26:23.712793+0000","last_peered":"2023-11-08T19:26:23.712793+0000","last_clean":"2023-11-08T19:26:23.712793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:23.712793+0000","last_undegraded":"2023-11-08T19:26:23.712793+0000","last_fullsized":"2023-11-08T19:26:23.712793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":22,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-11-08T19:26:20.798379+0000","last_change":"2023-11-08T19:26:06.425133+0000","last_active":"2023-11-08T19:26:20.798379+0000","last_peered":"2023-11-08T19:26:20.798379+0000","last_clean":"2023-11-08T19:26:20.798379+0000","last_became_active":"2023-11-08T19:26:04.601103+0000","last_became_peered":"2023-11-08T19:26:04.601103+0000","last_unstale":"2023-11-08T19:26:20.798379+0000","last_undegraded":"2023-11-08T19:26:20.798379+0000","last_fullsized":"2023-11-08T19:26:20.798379+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:31:14.042561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000229244,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-11-08T19:26:23.728996+0000","last_change":"2023-11-08T19:26:23.728996+0000","last_active":"2023-11-08T19:26:23.712793+0000","last_peered":"2023-11-08T19:26:23.712793+0000","last_clean":"2023-11-08T19:26:23.712793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:23.728996+0000","last_undegraded":"2023-11-08T19:26:23.728996+0000","last_fullsized":"2023-11-08T19:26:23.728996+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":22,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-11-08T19:26:20.798246+0000","last_change":"2023-11-08T19:26:06.265690+0000","last_active":"2023-11-08T19:26:20.798246+0000","last_peered":"2023-11-08T19:26:20.798246+0000","last_clean":"2023-11-08T19:26:20.798246+0000","last_became_active":"2023-11-08T19:26:04.603397+0000","last_became_peered":"2023-11-08T19:26:04.603397+0000","last_unstale":"2023-11-08T19:26:20.798246+0000","last_undegraded":"2023-11-08T19:26:20.798246+0000","last_fullsized":"2023-11-08T19:26:20.798246+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:50:43.827570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053361499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:23.712793+0000","last_change":"2023-11-08T19:26:23.712793+0000","last_active":"2023-11-08T19:26:23.712793+0000","last_peered":"2023-11-08T19:26:23.712793+0000","last_clean":"2023-11-08T19:26:23.712793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:23.712793+0000","last_undegraded":"2023-11-08T19:26:23.712793+0000","last_fullsized":"2023-11-08T19:26:23.712793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":22,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-08T19:26:20.712654+0000","last_change":"2023-11-08T19:26:06.541338+0000","last_active":"2023-11-08T19:26:20.712654+0000","last_peered":"2023-11-08T19:26:20.712654+0000","last_clean":"2023-11-08T19:26:20.712654+0000","last_became_active":"2023-11-08T19:26:04.602183+0000","last_became_peered":"2023-11-08T19:26:04.602183+0000","last_unstale":"2023-11-08T19:26:20.712654+0000","last_undegraded":"2023-11-08T19:26:20.712654+0000","last_fullsized":"2023-11-08T19:26:20.712654+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:06:47.446686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052029099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-11-08T19:26:23.728537+0000","last_change":"2023-11-08T19:26:23.728537+0000","last_active":"2023-11-08T19:26:23.712793+0000","last_peered":"2023-11-08T19:26:23.712793+0000","last_clean":"2023-11-08T19:26:23.712793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:23.728537+0000","last_undegraded":"2023-11-08T19:26:23.728537+0000","last_fullsized":"2023-11-08T19:26:23.728537+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":22,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T19:26:20.722344+0000","last_change":"2023-11-08T19:26:06.394081+0000","last_active":"2023-11-08T19:26:20.722344+0000","last_peered":"2023-11-08T19:26:20.722344+0000","last_clean":"2023-11-08T19:26:20.722344+0000","last_became_active":"2023-11-08T19:26:04.602137+0000","last_became_peered":"2023-11-08T19:26:04.602137+0000","last_unstale":"2023-11-08T19:26:20.722344+0000","last_undegraded":"2023-11-08T19:26:20.722344+0000","last_fullsized":"2023-11-08T19:26:20.722344+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:05:46.951663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056998200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:23.712793+0000","last_change":"2023-11-08T19:26:23.712793+0000","last_active":"2023-11-08T19:26:23.712793+0000","last_peered":"2023-11-08T19:26:23.712793+0000","last_clean":"2023-11-08T19:26:23.712793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:23.712793+0000","last_undegraded":"2023-11-08T19:26:23.712793+0000","last_fullsized":"2023-11-08T19:26:23.712793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":22,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-11-08T19:26:20.798119+0000","last_change":"2023-11-08T19:26:06.423863+0000","last_active":"2023-11-08T19:26:20.798119+0000","last_peered":"2023-11-08T19:26:20.798119+0000","last_clean":"2023-11-08T19:26:20.798119+0000","last_became_active":"2023-11-08T19:26:04.608639+0000","last_became_peered":"2023-11-08T19:26:04.608639+0000","last_unstale":"2023-11-08T19:26:20.798119+0000","last_undegraded":"2023-11-08T19:26:20.798119+0000","last_fullsized":"2023-11-08T19:26:20.798119+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:04:43.631102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044384999999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:23.712793+0000","last_change":"2023-11-08T19:26:23.712793+0000","last_active":"2023-11-08T19:26:23.712793+0000","last_peered":"2023-11-08T19:26:23.712793+0000","last_clean":"2023-11-08T19:26:23.712793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:23.712793+0000","last_undegraded":"2023-11-08T19:26:23.712793+0000","last_fullsized":"2023-11-08T19:26:23.712793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":22,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-11-08T19:26:20.798167+0000","last_change":"2023-11-08T19:26:06.265318+0000","last_active":"2023-11-08T19:26:20.798167+0000","last_peered":"2023-11-08T19:26:20.798167+0000","last_clean":"2023-11-08T19:26:20.798167+0000","last_became_active":"2023-11-08T19:26:04.603409+0000","last_became_peered":"2023-11-08T19:26:04.603409+0000","last_unstale":"2023-11-08T19:26:20.798167+0000","last_undegraded":"2023-11-08T19:26:20.798167+0000","last_fullsized":"2023-11-08T19:26:20.798167+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:27:24.933111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065684300000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:23.712793+0000","last_change":"2023-11-08T19:26:23.712793+0000","last_active":"2023-11-08T19:26:23.712793+0000","last_peered":"2023-11-08T19:26:23.712793+0000","last_clean":"2023-11-08T19:26:23.712793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:23.712793+0000","last_undegraded":"2023-11-08T19:26:23.712793+0000","last_fullsized":"2023-11-08T19:26:23.712793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":25,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-08T19:26:20.730441+0000","last_change":"2023-11-08T19:26:05.610012+0000","last_active":"2023-11-08T19:26:20.730441+0000","last_peered":"2023-11-08T19:26:20.730441+0000","last_clean":"2023-11-08T19:26:20.730441+0000","last_became_active":"2023-11-08T19:26:04.603209+0000","last_became_peered":"2023-11-08T19:26:04.603209+0000","last_unstale":"2023-11-08T19:26:20.730441+0000","last_undegraded":"2023-11-08T19:26:20.730441+0000","last_fullsized":"2023-11-08T19:26:20.730441+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:35:02.306997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053335100000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-11-08T19:26:23.729497+0000","last_change":"2023-11-08T19:26:23.729497+0000","last_active":"2023-11-08T19:26:23.712793+0000","last_peered":"2023-11-08T19:26:23.712793+0000","last_clean":"2023-11-08T19:26:23.712793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:23.729497+0000","last_undegraded":"2023-11-08T19:26:23.729497+0000","last_fullsized":"2023-11-08T19:26:23.729497+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":22,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-11-08T19:26:20.707190+0000","last_change":"2023-11-08T19:26:05.606946+0000","last_active":"2023-11-08T19:26:20.707190+0000","last_peered":"2023-11-08T19:26:20.707190+0000","last_clean":"2023-11-08T19:26:20.707190+0000","last_became_active":"2023-11-08T19:26:04.598775+0000","last_became_peered":"2023-11-08T19:26:04.598775+0000","last_unstale":"2023-11-08T19:26:20.707190+0000","last_undegraded":"2023-11-08T19:26:20.707190+0000","last_fullsized":"2023-11-08T19:26:20.707190+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:20:56.898927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049870399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:23.712793+0000","last_change":"2023-11-08T19:26:23.712793+0000","last_active":"2023-11-08T19:26:23.712793+0000","last_peered":"2023-11-08T19:26:23.712793+0000","last_clean":"2023-11-08T19:26:23.712793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:23.712793+0000","last_undegraded":"2023-11-08T19:26:23.712793+0000","last_fullsized":"2023-11-08T19:26:23.712793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":22,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-11-08T19:26:20.798225+0000","last_change":"2023-11-08T19:26:06.265907+0000","last_active":"2023-11-08T19:26:20.798225+0000","last_peered":"2023-11-08T19:26:20.798225+0000","last_clean":"2023-11-08T19:26:20.798225+0000","last_became_active":"2023-11-08T19:26:04.604772+0000","last_became_peered":"2023-11-08T19:26:04.604772+0000","last_unstale":"2023-11-08T19:26:20.798225+0000","last_undegraded":"2023-11-08T19:26:20.798225+0000","last_fullsized":"2023-11-08T19:26:20.798225+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:44:16.139794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066875300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-11-08T19:26:23.729045+0000","last_change":"2023-11-08T19:26:23.729045+0000","last_active":"2023-11-08T19:26:23.712793+0000","last_peered":"2023-11-08T19:26:23.712793+0000","last_clean":"2023-11-08T19:26:23.712793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:23.729045+0000","last_undegraded":"2023-11-08T19:26:23.729045+0000","last_fullsized":"2023-11-08T19:26:23.729045+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":22,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T19:26:20.722456+0000","last_change":"2023-11-08T19:26:06.394488+0000","last_active":"2023-11-08T19:26:20.722456+0000","last_peered":"2023-11-08T19:26:20.722456+0000","last_clean":"2023-11-08T19:26:20.722456+0000","last_became_active":"2023-11-08T19:26:04.606805+0000","last_became_peered":"2023-11-08T19:26:04.606805+0000","last_unstale":"2023-11-08T19:26:20.722456+0000","last_undegraded":"2023-11-08T19:26:20.722456+0000","last_fullsized":"2023-11-08T19:26:20.722456+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:25:20.857133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045317000000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:23.712793+0000","last_change":"2023-11-08T19:26:23.712793+0000","last_active":"2023-11-08T19:26:23.712793+0000","last_peered":"2023-11-08T19:26:23.712793+0000","last_clean":"2023-11-08T19:26:23.712793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:23.712793+0000","last_undegraded":"2023-11-08T19:26:23.712793+0000","last_fullsized":"2023-11-08T19:26:23.712793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":22,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-11-08T19:26:20.798078+0000","last_change":"2023-11-08T19:26:06.423754+0000","last_active":"2023-11-08T19:26:20.798078+0000","last_peered":"2023-11-08T19:26:20.798078+0000","last_clean":"2023-11-08T19:26:20.798078+0000","last_became_active":"2023-11-08T19:26:04.599322+0000","last_became_peered":"2023-11-08T19:26:04.599322+0000","last_unstale":"2023-11-08T19:26:20.798078+0000","last_undegraded":"2023-11-08T19:26:20.798078+0000","last_fullsized":"2023-11-08T19:26:20.798078+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:13:08.689128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042670000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-11-08T19:26:23.729048+0000","last_change":"2023-11-08T19:26:23.729048+0000","last_active":"2023-11-08T19:26:23.712793+0000","last_peered":"2023-11-08T19:26:23.712793+0000","last_clean":"2023-11-08T19:26:23.712793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:23.729048+0000","last_undegraded":"2023-11-08T19:26:23.729048+0000","last_fullsized":"2023-11-08T19:26:23.729048+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":22,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T19:26:20.722488+0000","last_change":"2023-11-08T19:26:06.394320+0000","last_active":"2023-11-08T19:26:20.722488+0000","last_peered":"2023-11-08T19:26:20.722488+0000","last_clean":"2023-11-08T19:26:20.722488+0000","last_became_active":"2023-11-08T19:26:04.603212+0000","last_became_peered":"2023-11-08T19:26:04.603212+0000","last_unstale":"2023-11-08T19:26:20.722488+0000","last_undegraded":"2023-11-08T19:26:20.722488+0000","last_fullsized":"2023-11-08T19:26:20.722488+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:25:29.947079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054695399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-11-08T19:26:23.729027+0000","last_change":"2023-11-08T19:26:23.729027+0000","last_active":"2023-11-08T19:26:23.712793+0000","last_peered":"2023-11-08T19:26:23.712793+0000","last_clean":"2023-11-08T19:26:23.712793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:23.729027+0000","last_undegraded":"2023-11-08T19:26:23.729027+0000","last_fullsized":"2023-11-08T19:26:23.729027+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":22,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-11-08T19:26:20.798197+0000","last_change":"2023-11-08T19:26:06.424073+0000","last_active":"2023-11-08T19:26:20.798197+0000","last_peered":"2023-11-08T19:26:20.798197+0000","last_clean":"2023-11-08T19:26:20.798197+0000","last_became_active":"2023-11-08T19:26:04.599046+0000","last_became_peered":"2023-11-08T19:26:04.599046+0000","last_unstale":"2023-11-08T19:26:20.798197+0000","last_undegraded":"2023-11-08T19:26:20.798197+0000","last_fullsized":"2023-11-08T19:26:20.798197+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:04:43.562566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035092199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:23.712793+0000","last_change":"2023-11-08T19:26:23.712793+0000","last_active":"2023-11-08T19:26:23.712793+0000","last_peered":"2023-11-08T19:26:23.712793+0000","last_clean":"2023-11-08T19:26:23.712793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:23.712793+0000","last_undegraded":"2023-11-08T19:26:23.712793+0000","last_fullsized":"2023-11-08T19:26:23.712793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":25,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-08T19:26:20.730344+0000","last_change":"2023-11-08T19:26:05.608980+0000","last_active":"2023-11-08T19:26:20.730344+0000","last_peered":"2023-11-08T19:26:20.730344+0000","last_clean":"2023-11-08T19:26:20.730344+0000","last_became_active":"2023-11-08T19:26:04.606386+0000","last_became_peered":"2023-11-08T19:26:04.606386+0000","last_unstale":"2023-11-08T19:26:20.730344+0000","last_undegraded":"2023-11-08T19:26:20.730344+0000","last_fullsized":"2023-11-08T19:26:20.730344+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:41:50.771457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037950599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-11-08T19:26:23.727431+0000","last_change":"2023-11-08T19:26:23.727431+0000","last_active":"2023-11-08T19:26:23.712793+0000","last_peered":"2023-11-08T19:26:23.712793+0000","last_clean":"2023-11-08T19:26:23.712793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:23.727431+0000","last_undegraded":"2023-11-08T19:26:23.727431+0000","last_fullsized":"2023-11-08T19:26:23.727431+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":22,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-08T19:26:20.713091+0000","last_change":"2023-11-08T19:26:06.541623+0000","last_active":"2023-11-08T19:26:20.713091+0000","last_peered":"2023-11-08T19:26:20.713091+0000","last_clean":"2023-11-08T19:26:20.713091+0000","last_became_active":"2023-11-08T19:26:04.601961+0000","last_became_peered":"2023-11-08T19:26:04.601961+0000","last_unstale":"2023-11-08T19:26:20.713091+0000","last_undegraded":"2023-11-08T19:26:20.713091+0000","last_fullsized":"2023-11-08T19:26:20.713091+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:11:26.508634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000466386,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-11-08T19:26:23.726886+0000","last_change":"2023-11-08T19:26:23.726886+0000","last_active":"2023-11-08T19:26:23.712793+0000","last_peered":"2023-11-08T19:26:23.712793+0000","last_clean":"2023-11-08T19:26:23.712793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:23.726886+0000","last_undegraded":"2023-11-08T19:26:23.726886+0000","last_fullsized":"2023-11-08T19:26:23.726886+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":22,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-11-08T19:26:20.798250+0000","last_change":"2023-11-08T19:26:06.424464+0000","last_active":"2023-11-08T19:26:20.798250+0000","last_peered":"2023-11-08T19:26:20.798250+0000","last_clean":"2023-11-08T19:26:20.798250+0000","last_became_active":"2023-11-08T19:26:04.603144+0000","last_became_peered":"2023-11-08T19:26:04.603144+0000","last_unstale":"2023-11-08T19:26:20.798250+0000","last_undegraded":"2023-11-08T19:26:20.798250+0000","last_fullsized":"2023-11-08T19:26:20.798250+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:42:55.269365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054090799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:23.712793+0000","last_change":"2023-11-08T19:26:23.712793+0000","last_active":"2023-11-08T19:26:23.712793+0000","last_peered":"2023-11-08T19:26:23.712793+0000","last_clean":"2023-11-08T19:26:23.712793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:23.712793+0000","last_undegraded":"2023-11-08T19:26:23.712793+0000","last_fullsized":"2023-11-08T19:26:23.712793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":22,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-11-08T19:26:20.707210+0000","last_change":"2023-11-08T19:26:05.606898+0000","last_active":"2023-11-08T19:26:20.707210+0000","last_peered":"2023-11-08T19:26:20.707210+0000","last_clean":"2023-11-08T19:26:20.707210+0000","last_became_active":"2023-11-08T19:26:04.603955+0000","last_became_peered":"2023-11-08T19:26:04.603955+0000","last_unstale":"2023-11-08T19:26:20.707210+0000","last_undegraded":"2023-11-08T19:26:20.707210+0000","last_fullsized":"2023-11-08T19:26:20.707210+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:19:11.589559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036473499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:23.712793+0000","last_change":"2023-11-08T19:26:23.712793+0000","last_active":"2023-11-08T19:26:23.712793+0000","last_peered":"2023-11-08T19:26:23.712793+0000","last_clean":"2023-11-08T19:26:23.712793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:23.712793+0000","last_undegraded":"2023-11-08T19:26:23.712793+0000","last_fullsized":"2023-11-08T19:26:23.712793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":22,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-11-08T19:26:20.798219+0000","last_change":"2023-11-08T19:26:06.265686+0000","last_active":"2023-11-08T19:26:20.798219+0000","last_peered":"2023-11-08T19:26:20.798219+0000","last_clean":"2023-11-08T19:26:20.798219+0000","last_became_active":"2023-11-08T19:26:04.602111+0000","last_became_peered":"2023-11-08T19:26:04.602111+0000","last_unstale":"2023-11-08T19:26:20.798219+0000","last_undegraded":"2023-11-08T19:26:20.798219+0000","last_fullsized":"2023-11-08T19:26:20.798219+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:40:11.164951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056197899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:23.712793+0000","last_change":"2023-11-08T19:26:23.712793+0000","last_active":"2023-11-08T19:26:23.712793+0000","last_peered":"2023-11-08T19:26:23.712793+0000","last_clean":"2023-11-08T19:26:23.712793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:23.712793+0000","last_undegraded":"2023-11-08T19:26:23.712793+0000","last_fullsized":"2023-11-08T19:26:23.712793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":22,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T19:26:20.722400+0000","last_change":"2023-11-08T19:26:06.394075+0000","last_active":"2023-11-08T19:26:20.722400+0000","last_peered":"2023-11-08T19:26:20.722400+0000","last_clean":"2023-11-08T19:26:20.722400+0000","last_became_active":"2023-11-08T19:26:04.606967+0000","last_became_peered":"2023-11-08T19:26:04.606967+0000","last_unstale":"2023-11-08T19:26:20.722400+0000","last_undegraded":"2023-11-08T19:26:20.722400+0000","last_fullsized":"2023-11-08T19:26:20.722400+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:19:56.286835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040971999999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:23.712793+0000","last_change":"2023-11-08T19:26:23.712793+0000","last_active":"2023-11-08T19:26:23.712793+0000","last_peered":"2023-11-08T19:26:23.712793+0000","last_clean":"2023-11-08T19:26:23.712793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:23.712793+0000","last_undegraded":"2023-11-08T19:26:23.712793+0000","last_fullsized":"2023-11-08T19:26:23.712793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":25,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-08T19:26:20.730298+0000","last_change":"2023-11-08T19:26:05.608892+0000","last_active":"2023-11-08T19:26:20.730298+0000","last_peered":"2023-11-08T19:26:20.730298+0000","last_clean":"2023-11-08T19:26:20.730298+0000","last_became_active":"2023-11-08T19:26:04.604519+0000","last_became_peered":"2023-11-08T19:26:04.604519+0000","last_unstale":"2023-11-08T19:26:20.730298+0000","last_undegraded":"2023-11-08T19:26:20.730298+0000","last_fullsized":"2023-11-08T19:26:20.730298+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:11:26.508634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051248099999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:23.712793+0000","last_change":"2023-11-08T19:26:23.712793+0000","last_active":"2023-11-08T19:26:23.712793+0000","last_peered":"2023-11-08T19:26:23.712793+0000","last_clean":"2023-11-08T19:26:23.712793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:23.712793+0000","last_undegraded":"2023-11-08T19:26:23.712793+0000","last_fullsized":"2023-11-08T19:26:23.712793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":22,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-11-08T19:26:20.798170+0000","last_change":"2023-11-08T19:26:06.424236+0000","last_active":"2023-11-08T19:26:20.798170+0000","last_peered":"2023-11-08T19:26:20.798170+0000","last_clean":"2023-11-08T19:26:20.798170+0000","last_became_active":"2023-11-08T19:26:04.600115+0000","last_became_peered":"2023-11-08T19:26:04.600115+0000","last_unstale":"2023-11-08T19:26:20.798170+0000","last_undegraded":"2023-11-08T19:26:20.798170+0000","last_fullsized":"2023-11-08T19:26:20.798170+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:18:01.876605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065803900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:23.712793+0000","last_change":"2023-11-08T19:26:23.712793+0000","last_active":"2023-11-08T19:26:23.712793+0000","last_peered":"2023-11-08T19:26:23.712793+0000","last_clean":"2023-11-08T19:26:23.712793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:23.712793+0000","last_undegraded":"2023-11-08T19:26:23.712793+0000","last_fullsized":"2023-11-08T19:26:23.712793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":22,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-08T19:26:20.712723+0000","last_change":"2023-11-08T19:26:06.541360+0000","last_active":"2023-11-08T19:26:20.712723+0000","last_peered":"2023-11-08T19:26:20.712723+0000","last_clean":"2023-11-08T19:26:20.712723+0000","last_became_active":"2023-11-08T19:26:04.603725+0000","last_became_peered":"2023-11-08T19:26:04.603725+0000","last_unstale":"2023-11-08T19:26:20.712723+0000","last_undegraded":"2023-11-08T19:26:20.712723+0000","last_fullsized":"2023-11-08T19:26:20.712723+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:28:48.576742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053365400000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:23.712793+0000","last_change":"2023-11-08T19:26:23.712793+0000","last_active":"2023-11-08T19:26:23.712793+0000","last_peered":"2023-11-08T19:26:23.712793+0000","last_clean":"2023-11-08T19:26:23.712793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:23.712793+0000","last_undegraded":"2023-11-08T19:26:23.712793+0000","last_fullsized":"2023-11-08T19:26:23.712793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":25,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-08T19:26:20.730515+0000","last_change":"2023-11-08T19:26:05.609824+0000","last_active":"2023-11-08T19:26:20.730515+0000","last_peered":"2023-11-08T19:26:20.730515+0000","last_clean":"2023-11-08T19:26:20.730515+0000","last_became_active":"2023-11-08T19:26:04.603320+0000","last_became_peered":"2023-11-08T19:26:04.603320+0000","last_unstale":"2023-11-08T19:26:20.730515+0000","last_undegraded":"2023-11-08T19:26:20.730515+0000","last_fullsized":"2023-11-08T19:26:20.730515+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:05:45.976615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063225,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-11-08T19:26:23.729027+0000","last_change":"2023-11-08T19:26:23.729027+0000","last_active":"2023-11-08T19:26:23.712793+0000","last_peered":"2023-11-08T19:26:23.712793+0000","last_clean":"2023-11-08T19:26:23.712793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:23.729027+0000","last_undegraded":"2023-11-08T19:26:23.729027+0000","last_fullsized":"2023-11-08T19:26:23.729027+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":22,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-11-08T19:26:20.697607+0000","last_change":"2023-11-08T19:26:06.296937+0000","last_active":"2023-11-08T19:26:20.697607+0000","last_peered":"2023-11-08T19:26:20.697607+0000","last_clean":"2023-11-08T19:26:20.697607+0000","last_became_active":"2023-11-08T19:26:04.599999+0000","last_became_peered":"2023-11-08T19:26:04.599999+0000","last_unstale":"2023-11-08T19:26:20.697607+0000","last_undegraded":"2023-11-08T19:26:20.697607+0000","last_fullsized":"2023-11-08T19:26:20.697607+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:50:43.827570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025605700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-11-08T19:26:23.728527+0000","last_change":"2023-11-08T19:26:23.728527+0000","last_active":"2023-11-08T19:26:23.712793+0000","last_peered":"2023-11-08T19:26:23.712793+0000","last_clean":"2023-11-08T19:26:23.712793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:23.728527+0000","last_undegraded":"2023-11-08T19:26:23.728527+0000","last_fullsized":"2023-11-08T19:26:23.728527+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":22,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-11-08T19:26:20.798161+0000","last_change":"2023-11-08T19:26:06.424511+0000","last_active":"2023-11-08T19:26:20.798161+0000","last_peered":"2023-11-08T19:26:20.798161+0000","last_clean":"2023-11-08T19:26:20.798161+0000","last_became_active":"2023-11-08T19:26:04.598973+0000","last_became_peered":"2023-11-08T19:26:04.598973+0000","last_unstale":"2023-11-08T19:26:20.798161+0000","last_undegraded":"2023-11-08T19:26:20.798161+0000","last_fullsized":"2023-11-08T19:26:20.798161+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:41:53.497456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031755900000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:23.712793+0000","last_change":"2023-11-08T19:26:23.712793+0000","last_active":"2023-11-08T19:26:23.712793+0000","last_peered":"2023-11-08T19:26:23.712793+0000","last_clean":"2023-11-08T19:26:23.712793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:23.712793+0000","last_undegraded":"2023-11-08T19:26:23.712793+0000","last_fullsized":"2023-11-08T19:26:23.712793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":22,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-11-08T19:26:20.697609+0000","last_change":"2023-11-08T19:26:06.296634+0000","last_active":"2023-11-08T19:26:20.697609+0000","last_peered":"2023-11-08T19:26:20.697609+0000","last_clean":"2023-11-08T19:26:20.697609+0000","last_became_active":"2023-11-08T19:26:04.603088+0000","last_became_peered":"2023-11-08T19:26:04.603088+0000","last_unstale":"2023-11-08T19:26:20.697609+0000","last_undegraded":"2023-11-08T19:26:20.697609+0000","last_fullsized":"2023-11-08T19:26:20.697609+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:01:56.991346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038180899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:23.712793+0000","last_change":"2023-11-08T19:26:23.712793+0000","last_active":"2023-11-08T19:26:23.712793+0000","last_peered":"2023-11-08T19:26:23.712793+0000","last_clean":"2023-11-08T19:26:23.712793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:23.712793+0000","last_undegraded":"2023-11-08T19:26:23.712793+0000","last_fullsized":"2023-11-08T19:26:23.712793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":22,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T19:26:20.722382+0000","last_change":"2023-11-08T19:26:06.394300+0000","last_active":"2023-11-08T19:26:20.722382+0000","last_peered":"2023-11-08T19:26:20.722382+0000","last_clean":"2023-11-08T19:26:20.722382+0000","last_became_active":"2023-11-08T19:26:04.601685+0000","last_became_peered":"2023-11-08T19:26:04.601685+0000","last_unstale":"2023-11-08T19:26:20.722382+0000","last_undegraded":"2023-11-08T19:26:20.722382+0000","last_fullsized":"2023-11-08T19:26:20.722382+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:51:06.626474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061044099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:23.712793+0000","last_change":"2023-11-08T19:26:23.712793+0000","last_active":"2023-11-08T19:26:23.712793+0000","last_peered":"2023-11-08T19:26:23.712793+0000","last_clean":"2023-11-08T19:26:23.712793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:23.712793+0000","last_undegraded":"2023-11-08T19:26:23.712793+0000","last_fullsized":"2023-11-08T19:26:23.712793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":25,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-08T19:26:20.730700+0000","last_change":"2023-11-08T19:26:05.609570+0000","last_active":"2023-11-08T19:26:20.730700+0000","last_peered":"2023-11-08T19:26:20.730700+0000","last_clean":"2023-11-08T19:26:20.730700+0000","last_became_active":"2023-11-08T19:26:04.608379+0000","last_became_peered":"2023-11-08T19:26:04.608379+0000","last_unstale":"2023-11-08T19:26:20.730700+0000","last_undegraded":"2023-11-08T19:26:20.730700+0000","last_fullsized":"2023-11-08T19:26:20.730700+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:40:08.986649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000529657,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:23.712793+0000","last_change":"2023-11-08T19:26:23.712793+0000","last_active":"2023-11-08T19:26:23.712793+0000","last_peered":"2023-11-08T19:26:23.712793+0000","last_clean":"2023-11-08T19:26:23.712793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:23.712793+0000","last_undegraded":"2023-11-08T19:26:23.712793+0000","last_fullsized":"2023-11-08T19:26:23.712793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":22,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-11-08T19:26:20.798171+0000","last_change":"2023-11-08T19:26:06.265419+0000","last_active":"2023-11-08T19:26:20.798171+0000","last_peered":"2023-11-08T19:26:20.798171+0000","last_clean":"2023-11-08T19:26:20.798171+0000","last_became_active":"2023-11-08T19:26:04.605333+0000","last_became_peered":"2023-11-08T19:26:04.605333+0000","last_unstale":"2023-11-08T19:26:20.798171+0000","last_undegraded":"2023-11-08T19:26:20.798171+0000","last_fullsized":"2023-11-08T19:26:20.798171+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:01:56.991346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064606400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:23.712793+0000","last_change":"2023-11-08T19:26:23.712793+0000","last_active":"2023-11-08T19:26:23.712793+0000","last_peered":"2023-11-08T19:26:23.712793+0000","last_clean":"2023-11-08T19:26:23.712793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:23.712793+0000","last_undegraded":"2023-11-08T19:26:23.712793+0000","last_fullsized":"2023-11-08T19:26:23.712793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":22,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-08T19:26:20.713122+0000","last_change":"2023-11-08T19:26:06.542044+0000","last_active":"2023-11-08T19:26:20.713122+0000","last_peered":"2023-11-08T19:26:20.713122+0000","last_clean":"2023-11-08T19:26:20.713122+0000","last_became_active":"2023-11-08T19:26:04.603056+0000","last_became_peered":"2023-11-08T19:26:04.603056+0000","last_unstale":"2023-11-08T19:26:20.713122+0000","last_undegraded":"2023-11-08T19:26:20.713122+0000","last_fullsized":"2023-11-08T19:26:20.713122+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:41:50.771457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000517414,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":190,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-08T19:26:20.730354+0000","last_change":"2023-11-08T19:24:45.888122+0000","last_active":"2023-11-08T19:26:20.730354+0000","last_peered":"2023-11-08T19:26:20.730354+0000","last_clean":"2023-11-08T19:26:20.730354+0000","last_became_active":"2023-11-08T19:24:45.887048+0000","last_became_peered":"2023-11-08T19:24:45.887048+0000","last_unstale":"2023-11-08T19:26:20.730354+0000","last_undegraded":"2023-11-08T19:26:20.730354+0000","last_fullsized":"2023-11-08T19:26:20.730354+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:44.575133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:44.575133+0000","last_clean_scrub_stamp":"2023-11-08T19:24:44.575133+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:19:20.202062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":20,"acting":20,"num_store_stats":2},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574872,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28116,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343724,"statfs":{"total":96636764160,"available":96607973376,"internally_reserved":0,"allocated":1122304,"data_stored":809257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.41499999999999998}]},{"osd":1,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.47499999999999998}]},{"osd":2,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.628}]},{"osd":3,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.442},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.65000000000000002}]},{"osd":4,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.502}]},{"osd":5,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.54800000000000004}]},{"osd":6,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.57799999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574872,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27536,"kb_used_data":516,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344304,"statfs":{"total":96636764160,"available":96608567296,"internally_reserved":0,"allocated":528384,"data_stored":218889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":21.359999999999999,"5min":21.359999999999999,"15min":21.359999999999999},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":10.329000000000001,"5min":10.329000000000001,"15min":10.329000000000001},"last":0.45500000000000002}]},{"osd":1,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":3.4239999999999999,"5min":3.4239999999999999,"15min":3.4239999999999999},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":3.4809999999999999,"5min":3.4809999999999999,"15min":3.4809999999999999},"last":0.40999999999999998}]},{"osd":2,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":10.398,"5min":10.398,"15min":10.398},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":5.0170000000000003,"5min":5.0170000000000003,"15min":5.0170000000000003},"last":0.42799999999999999}]},{"osd":3,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":3.5710000000000002,"5min":3.5710000000000002,"15min":3.5710000000000002},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":10.368,"5min":10.368,"15min":10.368},"last":0.55200000000000005}]},{"osd":4,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.50600000000000001}]},{"osd":5,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.51600000000000001}]},{"osd":7,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.54000000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574872,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27524,"kb_used_data":504,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344316,"statfs":{"total":96636764160,"available":96608579584,"internally_reserved":0,"allocated":516096,"data_stored":212719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":1.389,"5min":1.389,"15min":1.389},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":9.1370000000000005,"5min":9.1370000000000005,"15min":9.1370000000000005},"last":0.30299999999999999}]},{"osd":2,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.25600000000000001},{"interface":"front","average":{"1min":0.433,"5min":0.433,"15min":0.433},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":1.425,"5min":1.425,"15min":1.425},"last":0.35499999999999998}]},{"osd":3,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":11.957000000000001,"5min":11.957000000000001,"15min":11.957000000000001},"last":0.28000000000000003},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":18.039000000000001,"5min":18.039000000000001,"15min":18.039000000000001},"last":0.36899999999999999}]},{"osd":4,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":6.181,"5min":6.181,"15min":6.181},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.488,"5min":1.488,"15min":1.488},"last":0.66000000000000003}]},{"osd":5,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":9.2420000000000009,"5min":9.2420000000000009,"15min":9.2420000000000009},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":6.2110000000000003,"5min":6.2110000000000003,"15min":6.2110000000000003},"last":0.59899999999999998}]},{"osd":6,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":9.1880000000000006,"5min":9.1880000000000006,"15min":9.1880000000000006},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.462,"5min":1.462,"15min":1.462},"last":0.64100000000000001}]},{"osd":7,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":6.1479999999999997,"5min":6.1479999999999997,"15min":6.1479999999999997},"last":0.66900000000000004},{"interface":"front","average":{"1min":1.028,"5min":1.028,"15min":1.028},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":9.2170000000000005,"5min":9.2170000000000005,"15min":9.2170000000000005},"last":0.67900000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574872,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28108,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343732,"statfs":{"total":96636764160,"available":96607981568,"internally_reserved":0,"allocated":1110016,"data_stored":803087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":4.1200000000000001,"5min":4.1200000000000001,"15min":4.1200000000000001},"last":0.433},{"interface":"front","average":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":16.940999999999999,"5min":16.940999999999999,"15min":16.940999999999999},"last":0.504}]},{"osd":2,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":7.0209999999999999,"5min":7.0209999999999999,"15min":7.0209999999999999},"last":0.47099999999999997}]},{"osd":3,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":4.0940000000000003,"5min":4.0940000000000003,"15min":4.0940000000000003},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.48999999999999999}]},{"osd":4,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":4.0049999999999999,"5min":4.0049999999999999,"15min":4.0049999999999999},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.64400000000000002}]},{"osd":5,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":4.3090000000000002,"5min":4.3090000000000002,"15min":4.3090000000000002},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":4.0709999999999997,"5min":4.0709999999999997,"15min":4.0709999999999997},"last":0.67900000000000005}]},{"osd":6,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":4.2560000000000002,"5min":4.2560000000000002,"15min":4.2560000000000002},"last":0.68799999999999994}]},{"osd":7,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":4.2850000000000001,"5min":4.2850000000000001,"15min":4.2850000000000001},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":4.0460000000000003,"5min":4.0460000000000003,"15min":4.0460000000000003},"last":0.623}]}]},{"osd":2,"up_from":13,"seq":55834574872,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27524,"kb_used_data":504,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344316,"statfs":{"total":96636764160,"available":96608579584,"internally_reserved":0,"allocated":516096,"data_stored":212719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"min":{"1min":0.20100000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":6.0999999999999996,"5min":6.0999999999999996,"15min":6.0999999999999996},"last":0.23400000000000001}]},{"osd":1,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"last":0.312},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.624,"5min":0.624,"15min":0.624},"last":0.42399999999999999}]},{"osd":3,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"last":0.27800000000000002},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.32500000000000001}]},{"osd":4,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.55400000000000005}]},{"osd":5,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.497}]},{"osd":6,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.57199999999999995}]},{"osd":7,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.61399999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574873,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27528,"kb_used_data":504,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344312,"statfs":{"total":96636764160,"available":96608575488,"internally_reserved":0,"allocated":516096,"data_stored":212719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:25:46 2023","interfaces":[{"interface":"back","average":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.626,"5min":0.626,"15min":0.626},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.40799999999999997}]},{"osd":1,"last update":"Wed Nov 8 19:25:46 2023","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.31900000000000001}]},{"osd":2,"last update":"Wed Nov 8 19:25:46 2023","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.37,"15min":0.37},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"last":0.36899999999999999}]},{"osd":4,"last update":"Wed Nov 8 19:25:46 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.71299999999999997}]},{"osd":5,"last update":"Wed Nov 8 19:25:46 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.80800000000000005}]},{"osd":6,"last update":"Wed Nov 8 19:25:46 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.79500000000000004}]},{"osd":7,"last update":"Wed Nov 8 19:25:46 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.755},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.68000000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574872,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27524,"kb_used_data":504,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344316,"statfs":{"total":96636764160,"available":96608579584,"internally_reserved":0,"allocated":516096,"data_stored":212719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:25:47 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":6.5970000000000004,"5min":6.5970000000000004,"15min":6.5970000000000004},"last":0.501},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":2.5030000000000001,"5min":2.5030000000000001,"15min":2.5030000000000001},"last":0.436}]},{"osd":1,"last update":"Wed Nov 8 19:25:47 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.57599999999999996}]},{"osd":2,"last update":"Wed Nov 8 19:25:47 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":6.6920000000000002,"5min":6.6920000000000002,"15min":6.6920000000000002},"last":0.623}]},{"osd":3,"last update":"Wed Nov 8 19:25:47 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.46800000000000003}]},{"osd":5,"last update":"Wed Nov 8 19:25:47 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.60199999999999998}]},{"osd":6,"last update":"Wed Nov 8 19:25:47 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.66000000000000003}]},{"osd":7,"last update":"Wed Nov 8 19:25:47 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.59199999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574872,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27524,"kb_used_data":504,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344316,"statfs":{"total":96636764160,"available":96608579584,"internally_reserved":0,"allocated":516096,"data_stored":212719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.47399999999999998}]},{"osd":1,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.38900000000000001}]},{"osd":2,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.35199999999999998}]},{"osd":3,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.40799999999999997}]},{"osd":4,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.54000000000000004}]},{"osd":6,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.52800000000000002}]},{"osd":7,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.371},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.48199999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-08T19:26:29.852 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-08T19:26:30.004 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.002+0000 7f511013f700 1 Processor -- start 2023-11-08T19:26:30.004 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.003+0000 7f511013f700 1 -- start start 2023-11-08T19:26:30.008 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.003+0000 7f511013f700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f510815dd00 0x7f51080cca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:30.008 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.003+0000 7f511013f700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51080ccfb0 con 0x7f51081618f0 2023-11-08T19:26:30.008 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.003+0000 7f511013f700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5108161df0 con 0x7f510806b320 2023-11-08T19:26:30.009 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.003+0000 7f511013f700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51080cad50 con 0x7f510815dd00 2023-11-08T19:26:30.009 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.004+0000 7f510d6da700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f51081618f0 0x7f510806ac10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:47802/0 (socket says 172.21.15.143:47802) 2023-11-08T19:26:30.009 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.004+0000 7f510d6da700 1 -- 172.21.15.143:0/2099413418 learned_addr learned my addr 172.21.15.143:0/2099413418 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:26:30.009 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.004+0000 7f510dedb700 1 --2- 172.21.15.143:0/2099413418 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f510815dd00 0x7f51080cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:30.009 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.004+0000 7f510ced9700 1 -- 172.21.15.143:0/2099413418 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1958262782 0 0) 0x7f51080ccfb0 con 0x7f51081618f0 2023-11-08T19:26:30.010 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.004+0000 7f510ced9700 1 -- 172.21.15.143:0/2099413418 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f50f40035b0 con 0x7f51081618f0 2023-11-08T19:26:30.010 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.004+0000 7f510ced9700 1 -- 172.21.15.143:0/2099413418 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1088675799 0 0) 0x7f50f40035b0 con 0x7f51081618f0 2023-11-08T19:26:30.010 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.004+0000 7f510ced9700 1 -- 172.21.15.143:0/2099413418 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f510815dd00 msgr2=0x7f51080cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:30.010 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.004+0000 7f510ced9700 1 --2- 172.21.15.143:0/2099413418 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f510815dd00 0x7f51080cca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:30.010 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.004+0000 7f510ced9700 1 -- 172.21.15.143:0/2099413418 >> v1:172.21.15.174:6789/0 conn(0x7f510806b320 legacy=0x7f51080cb1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:30.011 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.004+0000 7f510ced9700 1 -- 172.21.15.143:0/2099413418 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51081620d0 con 0x7f51081618f0 2023-11-08T19:26:30.011 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.004+0000 7f510ced9700 1 -- 172.21.15.143:0/2099413418 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4147756303 0 0) 0x7f5104001670 con 0x7f51081618f0 2023-11-08T19:26:30.011 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.004+0000 7f510ced9700 1 -- 172.21.15.143:0/2099413418 >> v1:172.21.15.143:6789/0 conn(0x7f51081618f0 legacy=0x7f510806ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:30.011 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.004+0000 7f510ced9700 1 --2- 172.21.15.143:0/2099413418 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f50f4003a30 0x7f50f4003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:30.011 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.004+0000 7f510ced9700 1 --2- 172.21.15.143:0/2099413418 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f50f4004850 0x7f50f4004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:30.012 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.004+0000 7f510ced9700 1 --2- 172.21.15.143:0/2099413418 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f510815dd00 0x7f50f4005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:30.012 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.004+0000 7f510ced9700 1 -- 172.21.15.143:0/2099413418 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50f400bac0 con 0x7f50f4003a30 2023-11-08T19:26:30.012 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.004+0000 7f510ced9700 1 -- 172.21.15.143:0/2099413418 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50f400bc00 con 0x7f50f4004850 2023-11-08T19:26:30.012 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.004+0000 7f510ced9700 1 -- 172.21.15.143:0/2099413418 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50f400bdc0 con 0x7f510815dd00 2023-11-08T19:26:30.012 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.004+0000 7f510dedb700 1 --2- 172.21.15.143:0/2099413418 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f50f4003a30 0x7f50f4003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:30.012 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.004+0000 7f510e6dc700 1 --2- 172.21.15.143:0/2099413418 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f510815dd00 0x7f50f4005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:30.013 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.004+0000 7f510d6da700 1 --2- 172.21.15.143:0/2099413418 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f50f4004850 0x7f50f4004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:30.013 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.004+0000 7f510dedb700 1 -- 172.21.15.143:0/2099413418 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f510815dd00 msgr2=0x7f50f4005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:30.013 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.004+0000 7f510dedb700 1 --2- 172.21.15.143:0/2099413418 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f510815dd00 0x7f50f4005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:30.013 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.004+0000 7f510dedb700 1 -- 172.21.15.143:0/2099413418 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f50f4004850 msgr2=0x7f50f4004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:30.014 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.004+0000 7f510dedb700 1 --2- 172.21.15.143:0/2099413418 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f50f4004850 0x7f50f4004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:30.014 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.004+0000 7f510dedb700 1 -- 172.21.15.143:0/2099413418 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50f40035b0 con 0x7f50f4003a30 2023-11-08T19:26:30.014 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.005+0000 7f510e6dc700 1 --2- 172.21.15.143:0/2099413418 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f510815dd00 0x7f50f4005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:26:30.014 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.005+0000 7f510dedb700 1 --2- 172.21.15.143:0/2099413418 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f50f4003a30 0x7f50f4003e50 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f50f8009b10 tx=0x7f50f4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4789acc02167cfd server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:30.014 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.005+0000 7f510ced9700 1 -- 172.21.15.143:0/2099413418 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50f8005a20 con 0x7f50f4003a30 2023-11-08T19:26:30.015 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.005+0000 7f510ced9700 1 -- 172.21.15.143:0/2099413418 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50f8005b80 con 0x7f50f4003a30 2023-11-08T19:26:30.015 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.005+0000 7f510ced9700 1 -- 172.21.15.143:0/2099413418 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50f800e6c0 con 0x7f50f4003a30 2023-11-08T19:26:30.015 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.005+0000 7f511013f700 1 -- 172.21.15.143:0/2099413418 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f50f4003a30 msgr2=0x7f50f4003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:30.015 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.005+0000 7f511013f700 1 --2- 172.21.15.143:0/2099413418 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f50f4003a30 0x7f50f4003e50 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f50f8009b10 tx=0x7f50f4005880 comp rx=0 tx=0).stop 2023-11-08T19:26:30.015 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.005+0000 7f510e6dc700 1 -- 172.21.15.143:0/2099413418 reap_dead start 2023-11-08T19:26:30.016 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.005+0000 7f511013f700 1 -- 172.21.15.143:0/2099413418 shutdown_connections 2023-11-08T19:26:30.016 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.005+0000 7f511013f700 1 -- 172.21.15.143:0/2099413418 >> 172.21.15.143:0/2099413418 conn(0x7f5108099ab0 msgr2=0x7f51080d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:30.016 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.005+0000 7f511013f700 1 -- 172.21.15.143:0/2099413418 shutdown_connections 2023-11-08T19:26:30.016 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.005+0000 7f511013f700 1 -- 172.21.15.143:0/2099413418 wait complete. 2023-11-08T19:26:30.016 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.007+0000 7f511013f700 1 Processor -- start 2023-11-08T19:26:30.017 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.007+0000 7f511013f700 1 -- start start 2023-11-08T19:26:30.020 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.007+0000 7f511013f700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f50f4003a30 0x7f5108157e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:30.020 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.007+0000 7f511013f700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f51081583a0 0x7f51080c6b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:30.020 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.007+0000 7f511013f700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f510815be50 0x7f510815c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:30.020 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.007+0000 7f511013f700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f510815c840 con 0x7f50f4003a30 2023-11-08T19:26:30.020 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.007+0000 7f511013f700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51080c4230 con 0x7f51081583a0 2023-11-08T19:26:30.021 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.007+0000 7f511013f700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51080b5140 con 0x7f510815be50 2023-11-08T19:26:30.021 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.008+0000 7f510dedb700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f50f4003a30 0x7f5108157e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:30.021 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.008+0000 7f510dedb700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f50f4003a30 0x7f5108157e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51658/0 (socket says 172.21.15.143:51658) 2023-11-08T19:26:30.021 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.008+0000 7f510dedb700 1 -- 172.21.15.143:0/1181097678 learned_addr learned my addr 172.21.15.143:0/1181097678 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:30.021 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.008+0000 7f510e6dc700 1 --2- 172.21.15.143:0/1181097678 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f510815be50 0x7f510815c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:30.021 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.008+0000 7f510d6da700 1 --2- 172.21.15.143:0/1181097678 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f51081583a0 0x7f51080c6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:30.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.008+0000 7f510dedb700 1 -- 172.21.15.143:0/1181097678 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f510815be50 msgr2=0x7f510815c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:30.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.008+0000 7f510dedb700 1 --2- 172.21.15.143:0/1181097678 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f510815be50 0x7f510815c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:30.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.008+0000 7f510dedb700 1 -- 172.21.15.143:0/1181097678 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f51081583a0 msgr2=0x7f51080c6b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:30.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.008+0000 7f510dedb700 1 --2- 172.21.15.143:0/1181097678 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f51081583a0 0x7f51080c6b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:30.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.008+0000 7f510dedb700 1 -- 172.21.15.143:0/1181097678 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51080c7080 con 0x7f50f4003a30 2023-11-08T19:26:30.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.008+0000 7f510dedb700 1 --2- 172.21.15.143:0/1181097678 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f50f4003a30 0x7f5108157e30 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f50f800ff40 tx=0x7f50f80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:30.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.008+0000 7f50fe7fc700 1 -- 172.21.15.143:0/1181097678 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50f8020070 con 0x7f50f4003a30 2023-11-08T19:26:30.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.008+0000 7f50fe7fc700 1 -- 172.21.15.143:0/1181097678 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50f8004050 con 0x7f50f4003a30 2023-11-08T19:26:30.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.008+0000 7f511013f700 1 -- 172.21.15.143:0/1181097678 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51080b4530 con 0x7f50f4003a30 2023-11-08T19:26:30.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.008+0000 7f50fe7fc700 1 -- 172.21.15.143:0/1181097678 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50f80041e0 con 0x7f50f4003a30 2023-11-08T19:26:30.024 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.008+0000 7f511013f700 1 -- 172.21.15.143:0/1181097678 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51080b4af0 con 0x7f50f4003a30 2023-11-08T19:26:30.024 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.009+0000 7f511013f700 1 -- 172.21.15.143:0/1181097678 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f510804eee0 con 0x7f50f4003a30 2023-11-08T19:26:30.024 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.009+0000 7f50fe7fc700 1 -- 172.21.15.143:0/1181097678 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f50f8004340 con 0x7f50f4003a30 2023-11-08T19:26:30.024 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.010+0000 7f50fe7fc700 1 --2- 172.21.15.143:0/1181097678 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f50ec06af00 0x7f50ec06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:30.024 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.010+0000 7f50fe7fc700 1 -- 172.21.15.143:0/1181097678 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f50f808a5e0 con 0x7f50f4003a30 2023-11-08T19:26:30.025 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.010+0000 7f510d6da700 1 --2- 172.21.15.143:0/1181097678 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f50ec06af00 0x7f50ec06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:30.025 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.010+0000 7f510d6da700 1 --2- 172.21.15.143:0/1181097678 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f50ec06af00 0x7f50ec06d3c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f5104001c90 tx=0x7f5104009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:30.025 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.014+0000 7f50fe7fc700 1 -- 172.21.15.143:0/1181097678 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f50f80555c0 con 0x7f50f4003a30 2023-11-08T19:26:30.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.199+0000 7f511013f700 1 -- 172.21.15.143:0/1181097678 --> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5108162690 con 0x7f50ec06af00 2023-11-08T19:26:30.210 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.209+0000 7f50fe7fc700 1 -- 172.21.15.143:0/1181097678 <== mgr.4099 v2:172.21.15.174:6832/4037280178 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+356754 (secure 0 0 0) 0x7f5104004870 con 0x7f50ec06af00 2023-11-08T19:26:30.214 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:26:30.217 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.212+0000 7f511013f700 1 -- 172.21.15.143:0/1181097678 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f50ec06af00 msgr2=0x7f50ec06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:30.218 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.212+0000 7f511013f700 1 --2- 172.21.15.143:0/1181097678 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f50ec06af00 0x7f50ec06d3c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f5104001c90 tx=0x7f5104009040 comp rx=0 tx=0).stop 2023-11-08T19:26:30.218 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.212+0000 7f511013f700 1 -- 172.21.15.143:0/1181097678 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f50f4003a30 msgr2=0x7f5108157e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:30.218 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.212+0000 7f511013f700 1 --2- 172.21.15.143:0/1181097678 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f50f4003a30 0x7f5108157e30 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f50f800ff40 tx=0x7f50f80047c0 comp rx=0 tx=0).stop 2023-11-08T19:26:30.218 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.212+0000 7f511013f700 1 -- 172.21.15.143:0/1181097678 shutdown_connections 2023-11-08T19:26:30.219 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.212+0000 7f511013f700 1 --2- 172.21.15.143:0/1181097678 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f50ec06af00 0x7f50ec06d3c0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:30.219 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.212+0000 7f511013f700 1 --2- 172.21.15.143:0/1181097678 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f510815be50 0x7f510815c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:30.219 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.212+0000 7f511013f700 1 --2- 172.21.15.143:0/1181097678 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f50f4003a30 0x7f5108157e30 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:30.219 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.212+0000 7f511013f700 1 --2- 172.21.15.143:0/1181097678 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f51081583a0 0x7f51080c6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:30.219 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.212+0000 7f511013f700 1 -- 172.21.15.143:0/1181097678 >> 172.21.15.143:0/1181097678 conn(0x7f5108099ab0 msgr2=0x7f51080ad420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:30.220 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.212+0000 7f511013f700 1 -- 172.21.15.143:0/1181097678 shutdown_connections 2023-11-08T19:26:30.220 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.212+0000 7f511013f700 1 -- 172.21.15.143:0/1181097678 wait complete. 2023-11-08T19:26:30.220 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2023-11-08T19:26:30.226 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":93,"stamp":"2023-11-08T19:26:28.533968+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":86,"acting":86,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":66,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":221384,"kb_used_data":5216,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753336,"statfs":{"total":773094113280,"available":772867416064,"internally_reserved":0,"allocated":5341184,"data_stored":2894828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.810858"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:23.712793+0000","last_change":"2023-11-08T19:26:23.712793+0000","last_active":"2023-11-08T19:26:23.712793+0000","last_peered":"2023-11-08T19:26:23.712793+0000","last_clean":"2023-11-08T19:26:23.712793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:23.712793+0000","last_undegraded":"2023-11-08T19:26:23.712793+0000","last_fullsized":"2023-11-08T19:26:23.712793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":22,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-11-08T19:26:20.697680+0000","last_change":"2023-11-08T19:26:06.297255+0000","last_active":"2023-11-08T19:26:20.697680+0000","last_peered":"2023-11-08T19:26:20.697680+0000","last_clean":"2023-11-08T19:26:20.697680+0000","last_became_active":"2023-11-08T19:26:04.603979+0000","last_became_peered":"2023-11-08T19:26:04.603979+0000","last_unstale":"2023-11-08T19:26:20.697680+0000","last_undegraded":"2023-11-08T19:26:20.697680+0000","last_fullsized":"2023-11-08T19:26:20.697680+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:40:11.164951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079032099999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:23.712793+0000","last_change":"2023-11-08T19:26:23.712793+0000","last_active":"2023-11-08T19:26:23.712793+0000","last_peered":"2023-11-08T19:26:23.712793+0000","last_clean":"2023-11-08T19:26:23.712793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:23.712793+0000","last_undegraded":"2023-11-08T19:26:23.712793+0000","last_fullsized":"2023-11-08T19:26:23.712793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":25,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-08T19:26:20.730391+0000","last_change":"2023-11-08T19:26:05.609625+0000","last_active":"2023-11-08T19:26:20.730391+0000","last_peered":"2023-11-08T19:26:20.730391+0000","last_clean":"2023-11-08T19:26:20.730391+0000","last_became_active":"2023-11-08T19:26:04.606312+0000","last_became_peered":"2023-11-08T19:26:04.606312+0000","last_unstale":"2023-11-08T19:26:20.730391+0000","last_undegraded":"2023-11-08T19:26:20.730391+0000","last_fullsized":"2023-11-08T19:26:20.730391+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:54:42.032901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000505935,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:23.712793+0000","last_change":"2023-11-08T19:26:23.712793+0000","last_active":"2023-11-08T19:26:23.712793+0000","last_peered":"2023-11-08T19:26:23.712793+0000","last_clean":"2023-11-08T19:26:23.712793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:23.712793+0000","last_undegraded":"2023-11-08T19:26:23.712793+0000","last_fullsized":"2023-11-08T19:26:23.712793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":22,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-11-08T19:26:20.798379+0000","last_change":"2023-11-08T19:26:06.425133+0000","last_active":"2023-11-08T19:26:20.798379+0000","last_peered":"2023-11-08T19:26:20.798379+0000","last_clean":"2023-11-08T19:26:20.798379+0000","last_became_active":"2023-11-08T19:26:04.601103+0000","last_became_peered":"2023-11-08T19:26:04.601103+0000","last_unstale":"2023-11-08T19:26:20.798379+0000","last_undegraded":"2023-11-08T19:26:20.798379+0000","last_fullsized":"2023-11-08T19:26:20.798379+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:31:14.042561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000229244,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-11-08T19:26:23.728996+0000","last_change":"2023-11-08T19:26:23.728996+0000","last_active":"2023-11-08T19:26:23.712793+0000","last_peered":"2023-11-08T19:26:23.712793+0000","last_clean":"2023-11-08T19:26:23.712793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:23.728996+0000","last_undegraded":"2023-11-08T19:26:23.728996+0000","last_fullsized":"2023-11-08T19:26:23.728996+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":22,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-11-08T19:26:20.798246+0000","last_change":"2023-11-08T19:26:06.265690+0000","last_active":"2023-11-08T19:26:20.798246+0000","last_peered":"2023-11-08T19:26:20.798246+0000","last_clean":"2023-11-08T19:26:20.798246+0000","last_became_active":"2023-11-08T19:26:04.603397+0000","last_became_peered":"2023-11-08T19:26:04.603397+0000","last_unstale":"2023-11-08T19:26:20.798246+0000","last_undegraded":"2023-11-08T19:26:20.798246+0000","last_fullsized":"2023-11-08T19:26:20.798246+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:50:43.827570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053361499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:23.712793+0000","last_change":"2023-11-08T19:26:23.712793+0000","last_active":"2023-11-08T19:26:23.712793+0000","last_peered":"2023-11-08T19:26:23.712793+0000","last_clean":"2023-11-08T19:26:23.712793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:23.712793+0000","last_undegraded":"2023-11-08T19:26:23.712793+0000","last_fullsized":"2023-11-08T19:26:23.712793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":22,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-08T19:26:20.712654+0000","last_change":"2023-11-08T19:26:06.541338+0000","last_active":"2023-11-08T19:26:20.712654+0000","last_peered":"2023-11-08T19:26:20.712654+0000","last_clean":"2023-11-08T19:26:20.712654+0000","last_became_active":"2023-11-08T19:26:04.602183+0000","last_became_peered":"2023-11-08T19:26:04.602183+0000","last_unstale":"2023-11-08T19:26:20.712654+0000","last_undegraded":"2023-11-08T19:26:20.712654+0000","last_fullsized":"2023-11-08T19:26:20.712654+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:06:47.446686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052029099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-11-08T19:26:23.728537+0000","last_change":"2023-11-08T19:26:23.728537+0000","last_active":"2023-11-08T19:26:23.712793+0000","last_peered":"2023-11-08T19:26:23.712793+0000","last_clean":"2023-11-08T19:26:23.712793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:23.728537+0000","last_undegraded":"2023-11-08T19:26:23.728537+0000","last_fullsized":"2023-11-08T19:26:23.728537+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":22,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T19:26:20.722344+0000","last_change":"2023-11-08T19:26:06.394081+0000","last_active":"2023-11-08T19:26:20.722344+0000","last_peered":"2023-11-08T19:26:20.722344+0000","last_clean":"2023-11-08T19:26:20.722344+0000","last_became_active":"2023-11-08T19:26:04.602137+0000","last_became_peered":"2023-11-08T19:26:04.602137+0000","last_unstale":"2023-11-08T19:26:20.722344+0000","last_undegraded":"2023-11-08T19:26:20.722344+0000","last_fullsized":"2023-11-08T19:26:20.722344+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:05:46.951663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056998200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:23.712793+0000","last_change":"2023-11-08T19:26:23.712793+0000","last_active":"2023-11-08T19:26:23.712793+0000","last_peered":"2023-11-08T19:26:23.712793+0000","last_clean":"2023-11-08T19:26:23.712793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:23.712793+0000","last_undegraded":"2023-11-08T19:26:23.712793+0000","last_fullsized":"2023-11-08T19:26:23.712793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":22,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-11-08T19:26:20.798119+0000","last_change":"2023-11-08T19:26:06.423863+0000","last_active":"2023-11-08T19:26:20.798119+0000","last_peered":"2023-11-08T19:26:20.798119+0000","last_clean":"2023-11-08T19:26:20.798119+0000","last_became_active":"2023-11-08T19:26:04.608639+0000","last_became_peered":"2023-11-08T19:26:04.608639+0000","last_unstale":"2023-11-08T19:26:20.798119+0000","last_undegraded":"2023-11-08T19:26:20.798119+0000","last_fullsized":"2023-11-08T19:26:20.798119+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:04:43.631102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044384999999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:23.712793+0000","last_change":"2023-11-08T19:26:23.712793+0000","last_active":"2023-11-08T19:26:23.712793+0000","last_peered":"2023-11-08T19:26:23.712793+0000","last_clean":"2023-11-08T19:26:23.712793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:23.712793+0000","last_undegraded":"2023-11-08T19:26:23.712793+0000","last_fullsized":"2023-11-08T19:26:23.712793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":22,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-11-08T19:26:20.798167+0000","last_change":"2023-11-08T19:26:06.265318+0000","last_active":"2023-11-08T19:26:20.798167+0000","last_peered":"2023-11-08T19:26:20.798167+0000","last_clean":"2023-11-08T19:26:20.798167+0000","last_became_active":"2023-11-08T19:26:04.603409+0000","last_became_peered":"2023-11-08T19:26:04.603409+0000","last_unstale":"2023-11-08T19:26:20.798167+0000","last_undegraded":"2023-11-08T19:26:20.798167+0000","last_fullsized":"2023-11-08T19:26:20.798167+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:27:24.933111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065684300000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:23.712793+0000","last_change":"2023-11-08T19:26:23.712793+0000","last_active":"2023-11-08T19:26:23.712793+0000","last_peered":"2023-11-08T19:26:23.712793+0000","last_clean":"2023-11-08T19:26:23.712793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:23.712793+0000","last_undegraded":"2023-11-08T19:26:23.712793+0000","last_fullsized":"2023-11-08T19:26:23.712793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":25,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-08T19:26:20.730441+0000","last_change":"2023-11-08T19:26:05.610012+0000","last_active":"2023-11-08T19:26:20.730441+0000","last_peered":"2023-11-08T19:26:20.730441+0000","last_clean":"2023-11-08T19:26:20.730441+0000","last_became_active":"2023-11-08T19:26:04.603209+0000","last_became_peered":"2023-11-08T19:26:04.603209+0000","last_unstale":"2023-11-08T19:26:20.730441+0000","last_undegraded":"2023-11-08T19:26:20.730441+0000","last_fullsized":"2023-11-08T19:26:20.730441+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:35:02.306997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053335100000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-11-08T19:26:23.729497+0000","last_change":"2023-11-08T19:26:23.729497+0000","last_active":"2023-11-08T19:26:23.712793+0000","last_peered":"2023-11-08T19:26:23.712793+0000","last_clean":"2023-11-08T19:26:23.712793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:23.729497+0000","last_undegraded":"2023-11-08T19:26:23.729497+0000","last_fullsized":"2023-11-08T19:26:23.729497+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":22,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-11-08T19:26:20.707190+0000","last_change":"2023-11-08T19:26:05.606946+0000","last_active":"2023-11-08T19:26:20.707190+0000","last_peered":"2023-11-08T19:26:20.707190+0000","last_clean":"2023-11-08T19:26:20.707190+0000","last_became_active":"2023-11-08T19:26:04.598775+0000","last_became_peered":"2023-11-08T19:26:04.598775+0000","last_unstale":"2023-11-08T19:26:20.707190+0000","last_undegraded":"2023-11-08T19:26:20.707190+0000","last_fullsized":"2023-11-08T19:26:20.707190+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:20:56.898927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049870399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:23.712793+0000","last_change":"2023-11-08T19:26:23.712793+0000","last_active":"2023-11-08T19:26:23.712793+0000","last_peered":"2023-11-08T19:26:23.712793+0000","last_clean":"2023-11-08T19:26:23.712793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:23.712793+0000","last_undegraded":"2023-11-08T19:26:23.712793+0000","last_fullsized":"2023-11-08T19:26:23.712793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":22,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-11-08T19:26:20.798225+0000","last_change":"2023-11-08T19:26:06.265907+0000","last_active":"2023-11-08T19:26:20.798225+0000","last_peered":"2023-11-08T19:26:20.798225+0000","last_clean":"2023-11-08T19:26:20.798225+0000","last_became_active":"2023-11-08T19:26:04.604772+0000","last_became_peered":"2023-11-08T19:26:04.604772+0000","last_unstale":"2023-11-08T19:26:20.798225+0000","last_undegraded":"2023-11-08T19:26:20.798225+0000","last_fullsized":"2023-11-08T19:26:20.798225+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:44:16.139794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066875300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-11-08T19:26:23.729045+0000","last_change":"2023-11-08T19:26:23.729045+0000","last_active":"2023-11-08T19:26:23.712793+0000","last_peered":"2023-11-08T19:26:23.712793+0000","last_clean":"2023-11-08T19:26:23.712793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:23.729045+0000","last_undegraded":"2023-11-08T19:26:23.729045+0000","last_fullsized":"2023-11-08T19:26:23.729045+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":22,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T19:26:20.722456+0000","last_change":"2023-11-08T19:26:06.394488+0000","last_active":"2023-11-08T19:26:20.722456+0000","last_peered":"2023-11-08T19:26:20.722456+0000","last_clean":"2023-11-08T19:26:20.722456+0000","last_became_active":"2023-11-08T19:26:04.606805+0000","last_became_peered":"2023-11-08T19:26:04.606805+0000","last_unstale":"2023-11-08T19:26:20.722456+0000","last_undegraded":"2023-11-08T19:26:20.722456+0000","last_fullsized":"2023-11-08T19:26:20.722456+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:25:20.857133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045317000000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:23.712793+0000","last_change":"2023-11-08T19:26:23.712793+0000","last_active":"2023-11-08T19:26:23.712793+0000","last_peered":"2023-11-08T19:26:23.712793+0000","last_clean":"2023-11-08T19:26:23.712793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:23.712793+0000","last_undegraded":"2023-11-08T19:26:23.712793+0000","last_fullsized":"2023-11-08T19:26:23.712793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":22,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-11-08T19:26:20.798078+0000","last_change":"2023-11-08T19:26:06.423754+0000","last_active":"2023-11-08T19:26:20.798078+0000","last_peered":"2023-11-08T19:26:20.798078+0000","last_clean":"2023-11-08T19:26:20.798078+0000","last_became_active":"2023-11-08T19:26:04.599322+0000","last_became_peered":"2023-11-08T19:26:04.599322+0000","last_unstale":"2023-11-08T19:26:20.798078+0000","last_undegraded":"2023-11-08T19:26:20.798078+0000","last_fullsized":"2023-11-08T19:26:20.798078+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:13:08.689128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042670000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-11-08T19:26:23.729048+0000","last_change":"2023-11-08T19:26:23.729048+0000","last_active":"2023-11-08T19:26:23.712793+0000","last_peered":"2023-11-08T19:26:23.712793+0000","last_clean":"2023-11-08T19:26:23.712793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:23.729048+0000","last_undegraded":"2023-11-08T19:26:23.729048+0000","last_fullsized":"2023-11-08T19:26:23.729048+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":22,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T19:26:20.722488+0000","last_change":"2023-11-08T19:26:06.394320+0000","last_active":"2023-11-08T19:26:20.722488+0000","last_peered":"2023-11-08T19:26:20.722488+0000","last_clean":"2023-11-08T19:26:20.722488+0000","last_became_active":"2023-11-08T19:26:04.603212+0000","last_became_peered":"2023-11-08T19:26:04.603212+0000","last_unstale":"2023-11-08T19:26:20.722488+0000","last_undegraded":"2023-11-08T19:26:20.722488+0000","last_fullsized":"2023-11-08T19:26:20.722488+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:25:29.947079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054695399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-11-08T19:26:23.729027+0000","last_change":"2023-11-08T19:26:23.729027+0000","last_active":"2023-11-08T19:26:23.712793+0000","last_peered":"2023-11-08T19:26:23.712793+0000","last_clean":"2023-11-08T19:26:23.712793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:23.729027+0000","last_undegraded":"2023-11-08T19:26:23.729027+0000","last_fullsized":"2023-11-08T19:26:23.729027+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":22,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-11-08T19:26:20.798197+0000","last_change":"2023-11-08T19:26:06.424073+0000","last_active":"2023-11-08T19:26:20.798197+0000","last_peered":"2023-11-08T19:26:20.798197+0000","last_clean":"2023-11-08T19:26:20.798197+0000","last_became_active":"2023-11-08T19:26:04.599046+0000","last_became_peered":"2023-11-08T19:26:04.599046+0000","last_unstale":"2023-11-08T19:26:20.798197+0000","last_undegraded":"2023-11-08T19:26:20.798197+0000","last_fullsized":"2023-11-08T19:26:20.798197+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:04:43.562566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035092199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:23.712793+0000","last_change":"2023-11-08T19:26:23.712793+0000","last_active":"2023-11-08T19:26:23.712793+0000","last_peered":"2023-11-08T19:26:23.712793+0000","last_clean":"2023-11-08T19:26:23.712793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:23.712793+0000","last_undegraded":"2023-11-08T19:26:23.712793+0000","last_fullsized":"2023-11-08T19:26:23.712793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":25,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-08T19:26:20.730344+0000","last_change":"2023-11-08T19:26:05.608980+0000","last_active":"2023-11-08T19:26:20.730344+0000","last_peered":"2023-11-08T19:26:20.730344+0000","last_clean":"2023-11-08T19:26:20.730344+0000","last_became_active":"2023-11-08T19:26:04.606386+0000","last_became_peered":"2023-11-08T19:26:04.606386+0000","last_unstale":"2023-11-08T19:26:20.730344+0000","last_undegraded":"2023-11-08T19:26:20.730344+0000","last_fullsized":"2023-11-08T19:26:20.730344+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:41:50.771457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037950599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-11-08T19:26:23.727431+0000","last_change":"2023-11-08T19:26:23.727431+0000","last_active":"2023-11-08T19:26:23.712793+0000","last_peered":"2023-11-08T19:26:23.712793+0000","last_clean":"2023-11-08T19:26:23.712793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:23.727431+0000","last_undegraded":"2023-11-08T19:26:23.727431+0000","last_fullsized":"2023-11-08T19:26:23.727431+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":22,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-08T19:26:20.713091+0000","last_change":"2023-11-08T19:26:06.541623+0000","last_active":"2023-11-08T19:26:20.713091+0000","last_peered":"2023-11-08T19:26:20.713091+0000","last_clean":"2023-11-08T19:26:20.713091+0000","last_became_active":"2023-11-08T19:26:04.601961+0000","last_became_peered":"2023-11-08T19:26:04.601961+0000","last_unstale":"2023-11-08T19:26:20.713091+0000","last_undegraded":"2023-11-08T19:26:20.713091+0000","last_fullsized":"2023-11-08T19:26:20.713091+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:11:26.508634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000466386,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-11-08T19:26:23.726886+0000","last_change":"2023-11-08T19:26:23.726886+0000","last_active":"2023-11-08T19:26:23.712793+0000","last_peered":"2023-11-08T19:26:23.712793+0000","last_clean":"2023-11-08T19:26:23.712793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:23.726886+0000","last_undegraded":"2023-11-08T19:26:23.726886+0000","last_fullsized":"2023-11-08T19:26:23.726886+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":22,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-11-08T19:26:20.798250+0000","last_change":"2023-11-08T19:26:06.424464+0000","last_active":"2023-11-08T19:26:20.798250+0000","last_peered":"2023-11-08T19:26:20.798250+0000","last_clean":"2023-11-08T19:26:20.798250+0000","last_became_active":"2023-11-08T19:26:04.603144+0000","last_became_peered":"2023-11-08T19:26:04.603144+0000","last_unstale":"2023-11-08T19:26:20.798250+0000","last_undegraded":"2023-11-08T19:26:20.798250+0000","last_fullsized":"2023-11-08T19:26:20.798250+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:42:55.269365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054090799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:23.712793+0000","last_change":"2023-11-08T19:26:23.712793+0000","last_active":"2023-11-08T19:26:23.712793+0000","last_peered":"2023-11-08T19:26:23.712793+0000","last_clean":"2023-11-08T19:26:23.712793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:23.712793+0000","last_undegraded":"2023-11-08T19:26:23.712793+0000","last_fullsized":"2023-11-08T19:26:23.712793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":22,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-11-08T19:26:20.707210+0000","last_change":"2023-11-08T19:26:05.606898+0000","last_active":"2023-11-08T19:26:20.707210+0000","last_peered":"2023-11-08T19:26:20.707210+0000","last_clean":"2023-11-08T19:26:20.707210+0000","last_became_active":"2023-11-08T19:26:04.603955+0000","last_became_peered":"2023-11-08T19:26:04.603955+0000","last_unstale":"2023-11-08T19:26:20.707210+0000","last_undegraded":"2023-11-08T19:26:20.707210+0000","last_fullsized":"2023-11-08T19:26:20.707210+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:19:11.589559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036473499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:23.712793+0000","last_change":"2023-11-08T19:26:23.712793+0000","last_active":"2023-11-08T19:26:23.712793+0000","last_peered":"2023-11-08T19:26:23.712793+0000","last_clean":"2023-11-08T19:26:23.712793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:23.712793+0000","last_undegraded":"2023-11-08T19:26:23.712793+0000","last_fullsized":"2023-11-08T19:26:23.712793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":22,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-11-08T19:26:20.798219+0000","last_change":"2023-11-08T19:26:06.265686+0000","last_active":"2023-11-08T19:26:20.798219+0000","last_peered":"2023-11-08T19:26:20.798219+0000","last_clean":"2023-11-08T19:26:20.798219+0000","last_became_active":"2023-11-08T19:26:04.602111+0000","last_became_peered":"2023-11-08T19:26:04.602111+0000","last_unstale":"2023-11-08T19:26:20.798219+0000","last_undegraded":"2023-11-08T19:26:20.798219+0000","last_fullsized":"2023-11-08T19:26:20.798219+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:40:11.164951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056197899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:23.712793+0000","last_change":"2023-11-08T19:26:23.712793+0000","last_active":"2023-11-08T19:26:23.712793+0000","last_peered":"2023-11-08T19:26:23.712793+0000","last_clean":"2023-11-08T19:26:23.712793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:23.712793+0000","last_undegraded":"2023-11-08T19:26:23.712793+0000","last_fullsized":"2023-11-08T19:26:23.712793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":22,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T19:26:20.722400+0000","last_change":"2023-11-08T19:26:06.394075+0000","last_active":"2023-11-08T19:26:20.722400+0000","last_peered":"2023-11-08T19:26:20.722400+0000","last_clean":"2023-11-08T19:26:20.722400+0000","last_became_active":"2023-11-08T19:26:04.606967+0000","last_became_peered":"2023-11-08T19:26:04.606967+0000","last_unstale":"2023-11-08T19:26:20.722400+0000","last_undegraded":"2023-11-08T19:26:20.722400+0000","last_fullsized":"2023-11-08T19:26:20.722400+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:19:56.286835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040971999999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:23.712793+0000","last_change":"2023-11-08T19:26:23.712793+0000","last_active":"2023-11-08T19:26:23.712793+0000","last_peered":"2023-11-08T19:26:23.712793+0000","last_clean":"2023-11-08T19:26:23.712793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:23.712793+0000","last_undegraded":"2023-11-08T19:26:23.712793+0000","last_fullsized":"2023-11-08T19:26:23.712793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":25,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-08T19:26:20.730298+0000","last_change":"2023-11-08T19:26:05.608892+0000","last_active":"2023-11-08T19:26:20.730298+0000","last_peered":"2023-11-08T19:26:20.730298+0000","last_clean":"2023-11-08T19:26:20.730298+0000","last_became_active":"2023-11-08T19:26:04.604519+0000","last_became_peered":"2023-11-08T19:26:04.604519+0000","last_unstale":"2023-11-08T19:26:20.730298+0000","last_undegraded":"2023-11-08T19:26:20.730298+0000","last_fullsized":"2023-11-08T19:26:20.730298+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:11:26.508634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051248099999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:23.712793+0000","last_change":"2023-11-08T19:26:23.712793+0000","last_active":"2023-11-08T19:26:23.712793+0000","last_peered":"2023-11-08T19:26:23.712793+0000","last_clean":"2023-11-08T19:26:23.712793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:23.712793+0000","last_undegraded":"2023-11-08T19:26:23.712793+0000","last_fullsized":"2023-11-08T19:26:23.712793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":22,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-11-08T19:26:20.798170+0000","last_change":"2023-11-08T19:26:06.424236+0000","last_active":"2023-11-08T19:26:20.798170+0000","last_peered":"2023-11-08T19:26:20.798170+0000","last_clean":"2023-11-08T19:26:20.798170+0000","last_became_active":"2023-11-08T19:26:04.600115+0000","last_became_peered":"2023-11-08T19:26:04.600115+0000","last_unstale":"2023-11-08T19:26:20.798170+0000","last_undegraded":"2023-11-08T19:26:20.798170+0000","last_fullsized":"2023-11-08T19:26:20.798170+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:18:01.876605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065803900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:23.712793+0000","last_change":"2023-11-08T19:26:23.712793+0000","last_active":"2023-11-08T19:26:23.712793+0000","last_peered":"2023-11-08T19:26:23.712793+0000","last_clean":"2023-11-08T19:26:23.712793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:23.712793+0000","last_undegraded":"2023-11-08T19:26:23.712793+0000","last_fullsized":"2023-11-08T19:26:23.712793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":22,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-08T19:26:20.712723+0000","last_change":"2023-11-08T19:26:06.541360+0000","last_active":"2023-11-08T19:26:20.712723+0000","last_peered":"2023-11-08T19:26:20.712723+0000","last_clean":"2023-11-08T19:26:20.712723+0000","last_became_active":"2023-11-08T19:26:04.603725+0000","last_became_peered":"2023-11-08T19:26:04.603725+0000","last_unstale":"2023-11-08T19:26:20.712723+0000","last_undegraded":"2023-11-08T19:26:20.712723+0000","last_fullsized":"2023-11-08T19:26:20.712723+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:28:48.576742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053365400000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:23.712793+0000","last_change":"2023-11-08T19:26:23.712793+0000","last_active":"2023-11-08T19:26:23.712793+0000","last_peered":"2023-11-08T19:26:23.712793+0000","last_clean":"2023-11-08T19:26:23.712793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:23.712793+0000","last_undegraded":"2023-11-08T19:26:23.712793+0000","last_fullsized":"2023-11-08T19:26:23.712793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":25,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-08T19:26:20.730515+0000","last_change":"2023-11-08T19:26:05.609824+0000","last_active":"2023-11-08T19:26:20.730515+0000","last_peered":"2023-11-08T19:26:20.730515+0000","last_clean":"2023-11-08T19:26:20.730515+0000","last_became_active":"2023-11-08T19:26:04.603320+0000","last_became_peered":"2023-11-08T19:26:04.603320+0000","last_unstale":"2023-11-08T19:26:20.730515+0000","last_undegraded":"2023-11-08T19:26:20.730515+0000","last_fullsized":"2023-11-08T19:26:20.730515+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:05:45.976615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063225,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-11-08T19:26:23.729027+0000","last_change":"2023-11-08T19:26:23.729027+0000","last_active":"2023-11-08T19:26:23.712793+0000","last_peered":"2023-11-08T19:26:23.712793+0000","last_clean":"2023-11-08T19:26:23.712793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:23.729027+0000","last_undegraded":"2023-11-08T19:26:23.729027+0000","last_fullsized":"2023-11-08T19:26:23.729027+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":22,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-11-08T19:26:20.697607+0000","last_change":"2023-11-08T19:26:06.296937+0000","last_active":"2023-11-08T19:26:20.697607+0000","last_peered":"2023-11-08T19:26:20.697607+0000","last_clean":"2023-11-08T19:26:20.697607+0000","last_became_active":"2023-11-08T19:26:04.599999+0000","last_became_peered":"2023-11-08T19:26:04.599999+0000","last_unstale":"2023-11-08T19:26:20.697607+0000","last_undegraded":"2023-11-08T19:26:20.697607+0000","last_fullsized":"2023-11-08T19:26:20.697607+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:50:43.827570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025605700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-11-08T19:26:23.728527+0000","last_change":"2023-11-08T19:26:23.728527+0000","last_active":"2023-11-08T19:26:23.712793+0000","last_peered":"2023-11-08T19:26:23.712793+0000","last_clean":"2023-11-08T19:26:23.712793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:23.728527+0000","last_undegraded":"2023-11-08T19:26:23.728527+0000","last_fullsized":"2023-11-08T19:26:23.728527+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":22,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-11-08T19:26:20.798161+0000","last_change":"2023-11-08T19:26:06.424511+0000","last_active":"2023-11-08T19:26:20.798161+0000","last_peered":"2023-11-08T19:26:20.798161+0000","last_clean":"2023-11-08T19:26:20.798161+0000","last_became_active":"2023-11-08T19:26:04.598973+0000","last_became_peered":"2023-11-08T19:26:04.598973+0000","last_unstale":"2023-11-08T19:26:20.798161+0000","last_undegraded":"2023-11-08T19:26:20.798161+0000","last_fullsized":"2023-11-08T19:26:20.798161+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:41:53.497456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031755900000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:23.712793+0000","last_change":"2023-11-08T19:26:23.712793+0000","last_active":"2023-11-08T19:26:23.712793+0000","last_peered":"2023-11-08T19:26:23.712793+0000","last_clean":"2023-11-08T19:26:23.712793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:23.712793+0000","last_undegraded":"2023-11-08T19:26:23.712793+0000","last_fullsized":"2023-11-08T19:26:23.712793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":22,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-11-08T19:26:20.697609+0000","last_change":"2023-11-08T19:26:06.296634+0000","last_active":"2023-11-08T19:26:20.697609+0000","last_peered":"2023-11-08T19:26:20.697609+0000","last_clean":"2023-11-08T19:26:20.697609+0000","last_became_active":"2023-11-08T19:26:04.603088+0000","last_became_peered":"2023-11-08T19:26:04.603088+0000","last_unstale":"2023-11-08T19:26:20.697609+0000","last_undegraded":"2023-11-08T19:26:20.697609+0000","last_fullsized":"2023-11-08T19:26:20.697609+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:01:56.991346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038180899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:23.712793+0000","last_change":"2023-11-08T19:26:23.712793+0000","last_active":"2023-11-08T19:26:23.712793+0000","last_peered":"2023-11-08T19:26:23.712793+0000","last_clean":"2023-11-08T19:26:23.712793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:23.712793+0000","last_undegraded":"2023-11-08T19:26:23.712793+0000","last_fullsized":"2023-11-08T19:26:23.712793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":22,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T19:26:20.722382+0000","last_change":"2023-11-08T19:26:06.394300+0000","last_active":"2023-11-08T19:26:20.722382+0000","last_peered":"2023-11-08T19:26:20.722382+0000","last_clean":"2023-11-08T19:26:20.722382+0000","last_became_active":"2023-11-08T19:26:04.601685+0000","last_became_peered":"2023-11-08T19:26:04.601685+0000","last_unstale":"2023-11-08T19:26:20.722382+0000","last_undegraded":"2023-11-08T19:26:20.722382+0000","last_fullsized":"2023-11-08T19:26:20.722382+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:51:06.626474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061044099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:23.712793+0000","last_change":"2023-11-08T19:26:23.712793+0000","last_active":"2023-11-08T19:26:23.712793+0000","last_peered":"2023-11-08T19:26:23.712793+0000","last_clean":"2023-11-08T19:26:23.712793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:23.712793+0000","last_undegraded":"2023-11-08T19:26:23.712793+0000","last_fullsized":"2023-11-08T19:26:23.712793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":25,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-08T19:26:20.730700+0000","last_change":"2023-11-08T19:26:05.609570+0000","last_active":"2023-11-08T19:26:20.730700+0000","last_peered":"2023-11-08T19:26:20.730700+0000","last_clean":"2023-11-08T19:26:20.730700+0000","last_became_active":"2023-11-08T19:26:04.608379+0000","last_became_peered":"2023-11-08T19:26:04.608379+0000","last_unstale":"2023-11-08T19:26:20.730700+0000","last_undegraded":"2023-11-08T19:26:20.730700+0000","last_fullsized":"2023-11-08T19:26:20.730700+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:40:08.986649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000529657,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:23.712793+0000","last_change":"2023-11-08T19:26:23.712793+0000","last_active":"2023-11-08T19:26:23.712793+0000","last_peered":"2023-11-08T19:26:23.712793+0000","last_clean":"2023-11-08T19:26:23.712793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:23.712793+0000","last_undegraded":"2023-11-08T19:26:23.712793+0000","last_fullsized":"2023-11-08T19:26:23.712793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":22,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-11-08T19:26:20.798171+0000","last_change":"2023-11-08T19:26:06.265419+0000","last_active":"2023-11-08T19:26:20.798171+0000","last_peered":"2023-11-08T19:26:20.798171+0000","last_clean":"2023-11-08T19:26:20.798171+0000","last_became_active":"2023-11-08T19:26:04.605333+0000","last_became_peered":"2023-11-08T19:26:04.605333+0000","last_unstale":"2023-11-08T19:26:20.798171+0000","last_undegraded":"2023-11-08T19:26:20.798171+0000","last_fullsized":"2023-11-08T19:26:20.798171+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:01:56.991346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064606400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:24.718298+0000","last_change":"2023-11-08T19:26:24.718298+0000","last_active":"2023-11-08T19:26:24.718298+0000","last_peered":"2023-11-08T19:26:24.718298+0000","last_clean":"2023-11-08T19:26:24.718298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:24.718298+0000","last_undegraded":"2023-11-08T19:26:24.718298+0000","last_fullsized":"2023-11-08T19:26:24.718298+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:26:23.712793+0000","last_change":"2023-11-08T19:26:23.712793+0000","last_active":"2023-11-08T19:26:23.712793+0000","last_peered":"2023-11-08T19:26:23.712793+0000","last_clean":"2023-11-08T19:26:23.712793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:26:23.712793+0000","last_undegraded":"2023-11-08T19:26:23.712793+0000","last_fullsized":"2023-11-08T19:26:23.712793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":22,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-08T19:26:20.713122+0000","last_change":"2023-11-08T19:26:06.542044+0000","last_active":"2023-11-08T19:26:20.713122+0000","last_peered":"2023-11-08T19:26:20.713122+0000","last_clean":"2023-11-08T19:26:20.713122+0000","last_became_active":"2023-11-08T19:26:04.603056+0000","last_became_peered":"2023-11-08T19:26:04.603056+0000","last_unstale":"2023-11-08T19:26:20.713122+0000","last_undegraded":"2023-11-08T19:26:20.713122+0000","last_fullsized":"2023-11-08T19:26:20.713122+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:41:50.771457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000517414,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":190,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-08T19:26:20.730354+0000","last_change":"2023-11-08T19:24:45.888122+0000","last_active":"2023-11-08T19:26:20.730354+0000","last_peered":"2023-11-08T19:26:20.730354+0000","last_clean":"2023-11-08T19:26:20.730354+0000","last_became_active":"2023-11-08T19:24:45.887048+0000","last_became_peered":"2023-11-08T19:24:45.887048+0000","last_unstale":"2023-11-08T19:26:20.730354+0000","last_undegraded":"2023-11-08T19:26:20.730354+0000","last_fullsized":"2023-11-08T19:26:20.730354+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:44.575133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:44.575133+0000","last_clean_scrub_stamp":"2023-11-08T19:24:44.575133+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:19:20.202062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":20,"acting":20,"num_store_stats":2},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574872,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28116,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343724,"statfs":{"total":96636764160,"available":96607973376,"internally_reserved":0,"allocated":1122304,"data_stored":809257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.41499999999999998}]},{"osd":1,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.47499999999999998}]},{"osd":2,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.628}]},{"osd":3,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.442},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.65000000000000002}]},{"osd":4,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.502}]},{"osd":5,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.54800000000000004}]},{"osd":6,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.57799999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574872,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27536,"kb_used_data":516,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344304,"statfs":{"total":96636764160,"available":96608567296,"internally_reserved":0,"allocated":528384,"data_stored":218889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":21.359999999999999,"5min":21.359999999999999,"15min":21.359999999999999},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":10.329000000000001,"5min":10.329000000000001,"15min":10.329000000000001},"last":0.45500000000000002}]},{"osd":1,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":3.4239999999999999,"5min":3.4239999999999999,"15min":3.4239999999999999},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":3.4809999999999999,"5min":3.4809999999999999,"15min":3.4809999999999999},"last":0.40999999999999998}]},{"osd":2,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":10.398,"5min":10.398,"15min":10.398},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":5.0170000000000003,"5min":5.0170000000000003,"15min":5.0170000000000003},"last":0.42799999999999999}]},{"osd":3,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":3.5710000000000002,"5min":3.5710000000000002,"15min":3.5710000000000002},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":10.368,"5min":10.368,"15min":10.368},"last":0.55200000000000005}]},{"osd":4,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.50600000000000001}]},{"osd":5,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.51600000000000001}]},{"osd":7,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.54000000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574872,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27524,"kb_used_data":504,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344316,"statfs":{"total":96636764160,"available":96608579584,"internally_reserved":0,"allocated":516096,"data_stored":212719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":1.389,"5min":1.389,"15min":1.389},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":9.1370000000000005,"5min":9.1370000000000005,"15min":9.1370000000000005},"last":0.30299999999999999}]},{"osd":2,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.25600000000000001},{"interface":"front","average":{"1min":0.433,"5min":0.433,"15min":0.433},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":1.425,"5min":1.425,"15min":1.425},"last":0.35499999999999998}]},{"osd":3,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":11.957000000000001,"5min":11.957000000000001,"15min":11.957000000000001},"last":0.28000000000000003},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":18.039000000000001,"5min":18.039000000000001,"15min":18.039000000000001},"last":0.36899999999999999}]},{"osd":4,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":6.181,"5min":6.181,"15min":6.181},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.488,"5min":1.488,"15min":1.488},"last":0.66000000000000003}]},{"osd":5,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":9.2420000000000009,"5min":9.2420000000000009,"15min":9.2420000000000009},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":6.2110000000000003,"5min":6.2110000000000003,"15min":6.2110000000000003},"last":0.59899999999999998}]},{"osd":6,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":9.1880000000000006,"5min":9.1880000000000006,"15min":9.1880000000000006},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.462,"5min":1.462,"15min":1.462},"last":0.64100000000000001}]},{"osd":7,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":6.1479999999999997,"5min":6.1479999999999997,"15min":6.1479999999999997},"last":0.66900000000000004},{"interface":"front","average":{"1min":1.028,"5min":1.028,"15min":1.028},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":9.2170000000000005,"5min":9.2170000000000005,"15min":9.2170000000000005},"last":0.67900000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574872,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28108,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343732,"statfs":{"total":96636764160,"available":96607981568,"internally_reserved":0,"allocated":1110016,"data_stored":803087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":4.1200000000000001,"5min":4.1200000000000001,"15min":4.1200000000000001},"last":0.433},{"interface":"front","average":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":16.940999999999999,"5min":16.940999999999999,"15min":16.940999999999999},"last":0.504}]},{"osd":2,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":7.0209999999999999,"5min":7.0209999999999999,"15min":7.0209999999999999},"last":0.47099999999999997}]},{"osd":3,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":4.0940000000000003,"5min":4.0940000000000003,"15min":4.0940000000000003},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.48999999999999999}]},{"osd":4,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":4.0049999999999999,"5min":4.0049999999999999,"15min":4.0049999999999999},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.64400000000000002}]},{"osd":5,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":4.3090000000000002,"5min":4.3090000000000002,"15min":4.3090000000000002},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":4.0709999999999997,"5min":4.0709999999999997,"15min":4.0709999999999997},"last":0.67900000000000005}]},{"osd":6,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":4.2560000000000002,"5min":4.2560000000000002,"15min":4.2560000000000002},"last":0.68799999999999994}]},{"osd":7,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":4.2850000000000001,"5min":4.2850000000000001,"15min":4.2850000000000001},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":4.0460000000000003,"5min":4.0460000000000003,"15min":4.0460000000000003},"last":0.623}]}]},{"osd":2,"up_from":13,"seq":55834574872,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27524,"kb_used_data":504,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344316,"statfs":{"total":96636764160,"available":96608579584,"internally_reserved":0,"allocated":516096,"data_stored":212719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"min":{"1min":0.20100000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":6.0999999999999996,"5min":6.0999999999999996,"15min":6.0999999999999996},"last":0.23400000000000001}]},{"osd":1,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"last":0.312},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.624,"5min":0.624,"15min":0.624},"last":0.42399999999999999}]},{"osd":3,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"last":0.27800000000000002},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.32500000000000001}]},{"osd":4,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.55400000000000005}]},{"osd":5,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.497}]},{"osd":6,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.57199999999999995}]},{"osd":7,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.61399999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574873,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27528,"kb_used_data":504,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344312,"statfs":{"total":96636764160,"available":96608575488,"internally_reserved":0,"allocated":516096,"data_stored":212719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:25:46 2023","interfaces":[{"interface":"back","average":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.626,"5min":0.626,"15min":0.626},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.40799999999999997}]},{"osd":1,"last update":"Wed Nov 8 19:25:46 2023","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.31900000000000001}]},{"osd":2,"last update":"Wed Nov 8 19:25:46 2023","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.37,"15min":0.37},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"last":0.36899999999999999}]},{"osd":4,"last update":"Wed Nov 8 19:25:46 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.71299999999999997}]},{"osd":5,"last update":"Wed Nov 8 19:25:46 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.80800000000000005}]},{"osd":6,"last update":"Wed Nov 8 19:25:46 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.79500000000000004}]},{"osd":7,"last update":"Wed Nov 8 19:25:46 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.755},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.68000000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574872,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27524,"kb_used_data":504,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344316,"statfs":{"total":96636764160,"available":96608579584,"internally_reserved":0,"allocated":516096,"data_stored":212719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:25:47 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":6.5970000000000004,"5min":6.5970000000000004,"15min":6.5970000000000004},"last":0.501},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":2.5030000000000001,"5min":2.5030000000000001,"15min":2.5030000000000001},"last":0.436}]},{"osd":1,"last update":"Wed Nov 8 19:25:47 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.57599999999999996}]},{"osd":2,"last update":"Wed Nov 8 19:25:47 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":6.6920000000000002,"5min":6.6920000000000002,"15min":6.6920000000000002},"last":0.623}]},{"osd":3,"last update":"Wed Nov 8 19:25:47 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.46800000000000003}]},{"osd":5,"last update":"Wed Nov 8 19:25:47 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.60199999999999998}]},{"osd":6,"last update":"Wed Nov 8 19:25:47 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.66000000000000003}]},{"osd":7,"last update":"Wed Nov 8 19:25:47 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.59199999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574872,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27524,"kb_used_data":504,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344316,"statfs":{"total":96636764160,"available":96608579584,"internally_reserved":0,"allocated":516096,"data_stored":212719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.47399999999999998}]},{"osd":1,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.38900000000000001}]},{"osd":2,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.35199999999999998}]},{"osd":3,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.40799999999999997}]},{"osd":4,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.54000000000000004}]},{"osd":6,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.52800000000000002}]},{"osd":7,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.371},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.48199999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-08T19:26:30.234 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-08T19:26:30.387 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.385+0000 7f47cf5b7700 1 Processor -- start 2023-11-08T19:26:30.387 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.386+0000 7f47cf5b7700 1 -- start start 2023-11-08T19:26:30.391 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.386+0000 7f47cf5b7700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f47c815dd00 0x7f47c80cca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:30.391 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.386+0000 7f47cf5b7700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47c80ccfb0 con 0x7f47c806b320 2023-11-08T19:26:30.391 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.386+0000 7f47cf5b7700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47c8161df0 con 0x7f47c81618f0 2023-11-08T19:26:30.392 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.386+0000 7f47cf5b7700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47c80cad50 con 0x7f47c815dd00 2023-11-08T19:26:30.392 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.386+0000 7f47cdb54700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f47c806b320 0x7f47c80cb1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:47822/0 (socket says 172.21.15.143:47822) 2023-11-08T19:26:30.392 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.387+0000 7f47cdb54700 1 -- 172.21.15.143:0/3409865310 learned_addr learned my addr 172.21.15.143:0/3409865310 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:26:30.392 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.387+0000 7f47cd353700 1 --2- 172.21.15.143:0/3409865310 >> v2:172.21.15.174:3300/0 conn(0x7f47c815dd00 0x7f47c80cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:30.392 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.387+0000 7f47bffff700 1 -- 172.21.15.143:0/3409865310 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1399761631 0 0) 0x7f47c80ccfb0 con 0x7f47c806b320 2023-11-08T19:26:30.393 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.387+0000 7f47bffff700 1 -- 172.21.15.143:0/3409865310 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47b40035b0 con 0x7f47c806b320 2023-11-08T19:26:30.393 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.387+0000 7f47bffff700 1 -- 172.21.15.143:0/3409865310 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2414398693 0 0) 0x7f47b40035b0 con 0x7f47c806b320 2023-11-08T19:26:30.393 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.387+0000 7f47bffff700 1 -- 172.21.15.143:0/3409865310 >> v2:172.21.15.174:3300/0 conn(0x7f47c815dd00 msgr2=0x7f47c80cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:30.393 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.387+0000 7f47bffff700 1 --2- 172.21.15.143:0/3409865310 >> v2:172.21.15.174:3300/0 conn(0x7f47c815dd00 0x7f47c80cca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:30.393 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.387+0000 7f47bffff700 1 -- 172.21.15.143:0/3409865310 >> v1:172.21.15.174:6789/0 conn(0x7f47c81618f0 legacy=0x7f47c806ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:30.394 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.387+0000 7f47bffff700 1 -- 172.21.15.143:0/3409865310 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47c81620d0 con 0x7f47c806b320 2023-11-08T19:26:30.394 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.387+0000 7f47bffff700 1 -- 172.21.15.143:0/3409865310 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4147756303 0 0) 0x7f47c00019f0 con 0x7f47c806b320 2023-11-08T19:26:30.394 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.387+0000 7f47bffff700 1 -- 172.21.15.143:0/3409865310 >> v1:172.21.15.143:6789/0 conn(0x7f47c806b320 legacy=0x7f47c80cb1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:30.394 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.387+0000 7f47bffff700 1 --2- 172.21.15.143:0/3409865310 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f47b4003a00 0x7f47b4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:30.394 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.387+0000 7f47bffff700 1 --2- 172.21.15.143:0/3409865310 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f47b4004850 0x7f47b4004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:30.395 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.387+0000 7f47bffff700 1 --2- 172.21.15.143:0/3409865310 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f47b4005210 0x7f47b400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:30.395 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.387+0000 7f47bffff700 1 -- 172.21.15.143:0/3409865310 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47b400c080 con 0x7f47b4005210 2023-11-08T19:26:30.395 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.387+0000 7f47bffff700 1 -- 172.21.15.143:0/3409865310 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47b400c1f0 con 0x7f47b4004850 2023-11-08T19:26:30.395 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.387+0000 7f47bffff700 1 -- 172.21.15.143:0/3409865310 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47b400c3b0 con 0x7f47b4003a00 2023-11-08T19:26:30.396 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.387+0000 7f47cdb54700 1 --2- 172.21.15.143:0/3409865310 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f47b4005210 0x7f47b400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:30.396 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.387+0000 7f47cd353700 1 --2- 172.21.15.143:0/3409865310 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f47b4003a00 0x7f47b4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:30.396 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.387+0000 7f47ccb52700 1 --2- 172.21.15.143:0/3409865310 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f47b4004850 0x7f47b4004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:30.396 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.387+0000 7f47cdb54700 1 -- 172.21.15.143:0/3409865310 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f47b4003a00 msgr2=0x7f47b4003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:30.397 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.387+0000 7f47cdb54700 1 --2- 172.21.15.143:0/3409865310 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f47b4003a00 0x7f47b4003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:30.397 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.387+0000 7f47cdb54700 1 -- 172.21.15.143:0/3409865310 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f47b4004850 msgr2=0x7f47b4004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:30.397 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.387+0000 7f47cdb54700 1 --2- 172.21.15.143:0/3409865310 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f47b4004850 0x7f47b4004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:30.397 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.388+0000 7f47cdb54700 1 -- 172.21.15.143:0/3409865310 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47b40035b0 con 0x7f47b4005210 2023-11-08T19:26:30.397 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.388+0000 7f47cdb54700 1 -- 172.21.15.143:0/3409865310 reap_dead start 2023-11-08T19:26:30.398 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.388+0000 7f47cdb54700 1 --2- 172.21.15.143:0/3409865310 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f47b4005210 0x7f47b400ba10 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f47c80708d0 tx=0x7f47b4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c357b99c5b7502b server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:30.398 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.388+0000 7f47bffff700 1 -- 172.21.15.143:0/3409865310 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c0012600 con 0x7f47b4005210 2023-11-08T19:26:30.398 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.388+0000 7f47bffff700 1 -- 172.21.15.143:0/3409865310 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47c0012780 con 0x7f47b4005210 2023-11-08T19:26:30.398 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.388+0000 7f47bffff700 1 -- 172.21.15.143:0/3409865310 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c0012b30 con 0x7f47b4005210 2023-11-08T19:26:30.398 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.388+0000 7f47cf5b7700 1 -- 172.21.15.143:0/3409865310 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f47b4005210 msgr2=0x7f47b400ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:30.399 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.388+0000 7f47cf5b7700 1 --2- 172.21.15.143:0/3409865310 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f47b4005210 0x7f47b400ba10 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f47c80708d0 tx=0x7f47b4005880 comp rx=0 tx=0).stop 2023-11-08T19:26:30.399 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.388+0000 7f47cf5b7700 1 -- 172.21.15.143:0/3409865310 shutdown_connections 2023-11-08T19:26:30.399 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.388+0000 7f47cf5b7700 1 --2- 172.21.15.143:0/3409865310 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f47b4005210 0x7f47b400ba10 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:30.399 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.388+0000 7f47cf5b7700 1 -- 172.21.15.143:0/3409865310 >> 172.21.15.143:0/3409865310 conn(0x7f47c8099ab0 msgr2=0x7f47c80d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:30.399 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.388+0000 7f47cf5b7700 1 -- 172.21.15.143:0/3409865310 shutdown_connections 2023-11-08T19:26:30.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.388+0000 7f47cf5b7700 1 -- 172.21.15.143:0/3409865310 wait complete. 2023-11-08T19:26:30.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.390+0000 7f47cf5b7700 1 Processor -- start 2023-11-08T19:26:30.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.390+0000 7f47cf5b7700 1 -- start start 2023-11-08T19:26:30.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.391+0000 7f47cf5b7700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f47b4005210 0x7f47c80ca010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:30.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.391+0000 7f47cf5b7700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f47c81519c0 0x7f47c8151160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:30.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.391+0000 7f47cf5b7700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f47c80b12d0 0x7f47c80b1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:30.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.391+0000 7f47cf5b7700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47c80c44d0 con 0x7f47c80b12d0 2023-11-08T19:26:30.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.391+0000 7f47cf5b7700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47c8152210 con 0x7f47b4005210 2023-11-08T19:26:30.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.391+0000 7f47cf5b7700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47c80ca550 con 0x7f47c81519c0 2023-11-08T19:26:30.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.391+0000 7f47cdb54700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f47c80b12d0 0x7f47c80b1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:30.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.391+0000 7f47cdb54700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f47c80b12d0 0x7f47c80b1760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51680/0 (socket says 172.21.15.143:51680) 2023-11-08T19:26:30.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.391+0000 7f47cdb54700 1 -- 172.21.15.143:0/4064238029 learned_addr learned my addr 172.21.15.143:0/4064238029 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:30.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.391+0000 7f47cd353700 1 --2- 172.21.15.143:0/4064238029 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f47b4005210 0x7f47c80ca010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:30.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.391+0000 7f47ccb52700 1 --2- 172.21.15.143:0/4064238029 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f47c81519c0 0x7f47c8151160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:30.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.391+0000 7f47cdb54700 1 -- 172.21.15.143:0/4064238029 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f47c81519c0 msgr2=0x7f47c8151160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:30.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.391+0000 7f47cdb54700 1 --2- 172.21.15.143:0/4064238029 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f47c81519c0 0x7f47c8151160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:30.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.391+0000 7f47cdb54700 1 -- 172.21.15.143:0/4064238029 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f47b4005210 msgr2=0x7f47c80ca010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:30.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.391+0000 7f47cdb54700 1 --2- 172.21.15.143:0/4064238029 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f47b4005210 0x7f47c80ca010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:30.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.391+0000 7f47cdb54700 1 -- 172.21.15.143:0/4064238029 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47c80b2100 con 0x7f47c80b12d0 2023-11-08T19:26:30.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.391+0000 7f47cdb54700 1 --2- 172.21.15.143:0/4064238029 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f47c80b12d0 0x7f47c80b1760 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f47c80ce8a0 tx=0x7f47c0001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:30.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.392+0000 7f47bdffb700 1 -- 172.21.15.143:0/4064238029 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c001e630 con 0x7f47c80b12d0 2023-11-08T19:26:30.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.392+0000 7f47bdffb700 1 -- 172.21.15.143:0/4064238029 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47c001e7b0 con 0x7f47c80b12d0 2023-11-08T19:26:30.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.392+0000 7f47bdffb700 1 -- 172.21.15.143:0/4064238029 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c001eb60 con 0x7f47c80b12d0 2023-11-08T19:26:30.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.392+0000 7f47cf5b7700 1 -- 172.21.15.143:0/4064238029 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47c80b23e0 con 0x7f47c80b12d0 2023-11-08T19:26:30.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.392+0000 7f47cf5b7700 1 -- 172.21.15.143:0/4064238029 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47c80b1ca0 con 0x7f47c80b12d0 2023-11-08T19:26:30.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.393+0000 7f47cf5b7700 1 -- 172.21.15.143:0/4064238029 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47c80b1ec0 con 0x7f47c80b12d0 2023-11-08T19:26:30.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.393+0000 7f47bdffb700 1 -- 172.21.15.143:0/4064238029 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f47c001ecc0 con 0x7f47c80b12d0 2023-11-08T19:26:30.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.393+0000 7f47bdffb700 1 --2- 172.21.15.143:0/4064238029 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f47ac06af00 0x7f47ac06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:30.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.394+0000 7f47bdffb700 1 -- 172.21.15.143:0/4064238029 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f47c008ba80 con 0x7f47c80b12d0 2023-11-08T19:26:30.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.394+0000 7f47cd353700 1 --2- 172.21.15.143:0/4064238029 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f47ac06af00 0x7f47ac06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:30.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.394+0000 7f47cd353700 1 --2- 172.21.15.143:0/4064238029 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f47ac06af00 0x7f47ac06d3c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f47c806f820 tx=0x7f47b80091c0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:30.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.398+0000 7f47bdffb700 1 -- 172.21.15.143:0/4064238029 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f47c80b1ec0 con 0x7f47c80b12d0 2023-11-08T19:26:30.584 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.583+0000 7f47cf5b7700 1 -- 172.21.15.143:0/4064238029 --> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f47c8162690 con 0x7f47ac06af00 2023-11-08T19:26:30.594 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.593+0000 7f47bdffb700 1 -- 172.21.15.143:0/4064238029 <== mgr.4099 v2:172.21.15.174:6832/4037280178 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+370727 (secure 0 0 0) 0x7f47b8004030 con 0x7f47ac06af00 2023-11-08T19:26:30.599 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:26:30.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.597+0000 7f47cf5b7700 1 -- 172.21.15.143:0/4064238029 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f47ac06af00 msgr2=0x7f47ac06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:30.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.597+0000 7f47cf5b7700 1 --2- 172.21.15.143:0/4064238029 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f47ac06af00 0x7f47ac06d3c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f47c806f820 tx=0x7f47b80091c0 comp rx=0 tx=0).stop 2023-11-08T19:26:30.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.597+0000 7f47cf5b7700 1 -- 172.21.15.143:0/4064238029 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f47c80b12d0 msgr2=0x7f47c80b1760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:30.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.597+0000 7f47cf5b7700 1 --2- 172.21.15.143:0/4064238029 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f47c80b12d0 0x7f47c80b1760 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f47c80ce8a0 tx=0x7f47c0001d80 comp rx=0 tx=0).stop 2023-11-08T19:26:30.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.597+0000 7f47cf5b7700 1 -- 172.21.15.143:0/4064238029 shutdown_connections 2023-11-08T19:26:30.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.597+0000 7f47cf5b7700 1 --2- 172.21.15.143:0/4064238029 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f47ac06af00 0x7f47ac06d3c0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:30.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.597+0000 7f47cf5b7700 1 --2- 172.21.15.143:0/4064238029 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f47c80b12d0 0x7f47c80b1760 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:30.604 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.597+0000 7f47cf5b7700 1 --2- 172.21.15.143:0/4064238029 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f47b4005210 0x7f47c80ca010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:30.604 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.597+0000 7f47cf5b7700 1 --2- 172.21.15.143:0/4064238029 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f47c81519c0 0x7f47c8151160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:30.604 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.597+0000 7f47cf5b7700 1 -- 172.21.15.143:0/4064238029 >> 172.21.15.143:0/4064238029 conn(0x7f47c8099ab0 msgr2=0x7f47c80ad3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:30.604 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.597+0000 7f47cf5b7700 1 -- 172.21.15.143:0/4064238029 shutdown_connections 2023-11-08T19:26:30.605 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.597+0000 7f47cf5b7700 1 -- 172.21.15.143:0/4064238029 wait complete. 2023-11-08T19:26:30.605 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2023-11-08T19:26:30.611 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":94,"stamp":"2023-11-08T19:26:30.534447+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":258,"acting":258,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":227,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":221848,"kb_used_data":5640,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754752872,"statfs":{"total":773094113280,"available":772866940928,"internally_reserved":0,"allocated":5775360,"data_stored":3099940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001350"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.731024+0000","last_change":"2023-11-08T19:26:25.752333+0000","last_active":"2023-11-08T19:26:26.731024+0000","last_peered":"2023-11-08T19:26:26.731024+0000","last_clean":"2023-11-08T19:26:26.731024+0000","last_became_active":"2023-11-08T19:26:25.752055+0000","last_became_peered":"2023-11-08T19:26:25.752055+0000","last_unstale":"2023-11-08T19:26:26.731024+0000","last_undegraded":"2023-11-08T19:26:26.731024+0000","last_fullsized":"2023-11-08T19:26:26.731024+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:29:44.685878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.730541+0000","last_change":"2023-11-08T19:26:25.746072+0000","last_active":"2023-11-08T19:26:26.730541+0000","last_peered":"2023-11-08T19:26:26.730541+0000","last_clean":"2023-11-08T19:26:26.730541+0000","last_became_active":"2023-11-08T19:26:25.745753+0000","last_became_peered":"2023-11-08T19:26:25.745753+0000","last_unstale":"2023-11-08T19:26:26.730541+0000","last_undegraded":"2023-11-08T19:26:26.730541+0000","last_fullsized":"2023-11-08T19:26:26.730541+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:02:15.103886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.762914+0000","last_change":"2023-11-08T19:26:25.746891+0000","last_active":"2023-11-08T19:26:26.762914+0000","last_peered":"2023-11-08T19:26:26.762914+0000","last_clean":"2023-11-08T19:26:26.762914+0000","last_became_active":"2023-11-08T19:26:25.746609+0000","last_became_peered":"2023-11-08T19:26:25.746609+0000","last_unstale":"2023-11-08T19:26:26.762914+0000","last_undegraded":"2023-11-08T19:26:26.762914+0000","last_fullsized":"2023-11-08T19:26:26.762914+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:02:15.103886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.730147+0000","last_change":"2023-11-08T19:26:25.739800+0000","last_active":"2023-11-08T19:26:26.730147+0000","last_peered":"2023-11-08T19:26:26.730147+0000","last_clean":"2023-11-08T19:26:26.730147+0000","last_became_active":"2023-11-08T19:26:25.739529+0000","last_became_peered":"2023-11-08T19:26:25.739529+0000","last_unstale":"2023-11-08T19:26:26.730147+0000","last_undegraded":"2023-11-08T19:26:26.730147+0000","last_fullsized":"2023-11-08T19:26:26.730147+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:48:09.695836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.762381+0000","last_change":"2023-11-08T19:26:25.741966+0000","last_active":"2023-11-08T19:26:26.762381+0000","last_peered":"2023-11-08T19:26:26.762381+0000","last_clean":"2023-11-08T19:26:26.762381+0000","last_became_active":"2023-11-08T19:26:25.741388+0000","last_became_peered":"2023-11-08T19:26:25.741388+0000","last_unstale":"2023-11-08T19:26:26.762381+0000","last_undegraded":"2023-11-08T19:26:26.762381+0000","last_fullsized":"2023-11-08T19:26:26.762381+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:31:35.182260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.730525+0000","last_change":"2023-11-08T19:26:25.757491+0000","last_active":"2023-11-08T19:26:26.730525+0000","last_peered":"2023-11-08T19:26:26.730525+0000","last_clean":"2023-11-08T19:26:26.730525+0000","last_became_active":"2023-11-08T19:26:25.757106+0000","last_became_peered":"2023-11-08T19:26:25.757106+0000","last_unstale":"2023-11-08T19:26:26.730525+0000","last_undegraded":"2023-11-08T19:26:26.730525+0000","last_fullsized":"2023-11-08T19:26:26.730525+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:09:25.716422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.761990+0000","last_change":"2023-11-08T19:26:25.742389+0000","last_active":"2023-11-08T19:26:26.761990+0000","last_peered":"2023-11-08T19:26:26.761990+0000","last_clean":"2023-11-08T19:26:26.761990+0000","last_became_active":"2023-11-08T19:26:25.742128+0000","last_became_peered":"2023-11-08T19:26:25.742128+0000","last_unstale":"2023-11-08T19:26:26.761990+0000","last_undegraded":"2023-11-08T19:26:26.761990+0000","last_fullsized":"2023-11-08T19:26:26.761990+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:42:13.350558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.761951+0000","last_change":"2023-11-08T19:26:25.748084+0000","last_active":"2023-11-08T19:26:26.761951+0000","last_peered":"2023-11-08T19:26:26.761951+0000","last_clean":"2023-11-08T19:26:26.761951+0000","last_became_active":"2023-11-08T19:26:25.747737+0000","last_became_peered":"2023-11-08T19:26:25.747737+0000","last_unstale":"2023-11-08T19:26:26.761951+0000","last_undegraded":"2023-11-08T19:26:26.761951+0000","last_fullsized":"2023-11-08T19:26:26.761951+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:33:05.719333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.762276+0000","last_change":"2023-11-08T19:26:25.747778+0000","last_active":"2023-11-08T19:26:26.762276+0000","last_peered":"2023-11-08T19:26:26.762276+0000","last_clean":"2023-11-08T19:26:26.762276+0000","last_became_active":"2023-11-08T19:26:25.747302+0000","last_became_peered":"2023-11-08T19:26:25.747302+0000","last_unstale":"2023-11-08T19:26:26.762276+0000","last_undegraded":"2023-11-08T19:26:26.762276+0000","last_fullsized":"2023-11-08T19:26:26.762276+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:59:22.071613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.730998+0000","last_change":"2023-11-08T19:26:25.745318+0000","last_active":"2023-11-08T19:26:26.730998+0000","last_peered":"2023-11-08T19:26:26.730998+0000","last_clean":"2023-11-08T19:26:26.730998+0000","last_became_active":"2023-11-08T19:26:25.745040+0000","last_became_peered":"2023-11-08T19:26:25.745040+0000","last_unstale":"2023-11-08T19:26:26.730998+0000","last_undegraded":"2023-11-08T19:26:26.730998+0000","last_fullsized":"2023-11-08T19:26:26.730998+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:29:30.597869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.762525+0000","last_change":"2023-11-08T19:26:25.741866+0000","last_active":"2023-11-08T19:26:26.762525+0000","last_peered":"2023-11-08T19:26:26.762525+0000","last_clean":"2023-11-08T19:26:26.762525+0000","last_became_active":"2023-11-08T19:26:25.741549+0000","last_became_peered":"2023-11-08T19:26:25.741549+0000","last_unstale":"2023-11-08T19:26:26.762525+0000","last_undegraded":"2023-11-08T19:26:26.762525+0000","last_fullsized":"2023-11-08T19:26:26.762525+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:27:46.072811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.762113+0000","last_change":"2023-11-08T19:26:25.738224+0000","last_active":"2023-11-08T19:26:26.762113+0000","last_peered":"2023-11-08T19:26:26.762113+0000","last_clean":"2023-11-08T19:26:26.762113+0000","last_became_active":"2023-11-08T19:26:25.737961+0000","last_became_peered":"2023-11-08T19:26:25.737961+0000","last_unstale":"2023-11-08T19:26:26.762113+0000","last_undegraded":"2023-11-08T19:26:26.762113+0000","last_fullsized":"2023-11-08T19:26:26.762113+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:42:48.346407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.730528+0000","last_change":"2023-11-08T19:26:25.746303+0000","last_active":"2023-11-08T19:26:26.730528+0000","last_peered":"2023-11-08T19:26:26.730528+0000","last_clean":"2023-11-08T19:26:26.730528+0000","last_became_active":"2023-11-08T19:26:25.745987+0000","last_became_peered":"2023-11-08T19:26:25.745987+0000","last_unstale":"2023-11-08T19:26:26.730528+0000","last_undegraded":"2023-11-08T19:26:26.730528+0000","last_fullsized":"2023-11-08T19:26:26.730528+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:35:24.623946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.762451+0000","last_change":"2023-11-08T19:26:25.744431+0000","last_active":"2023-11-08T19:26:26.762451+0000","last_peered":"2023-11-08T19:26:26.762451+0000","last_clean":"2023-11-08T19:26:26.762451+0000","last_became_active":"2023-11-08T19:26:25.742917+0000","last_became_peered":"2023-11-08T19:26:25.742917+0000","last_unstale":"2023-11-08T19:26:26.762451+0000","last_undegraded":"2023-11-08T19:26:26.762451+0000","last_fullsized":"2023-11-08T19:26:26.762451+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:22:57.218646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.730191+0000","last_change":"2023-11-08T19:26:25.737530+0000","last_active":"2023-11-08T19:26:26.730191+0000","last_peered":"2023-11-08T19:26:26.730191+0000","last_clean":"2023-11-08T19:26:26.730191+0000","last_became_active":"2023-11-08T19:26:25.737241+0000","last_became_peered":"2023-11-08T19:26:25.737241+0000","last_unstale":"2023-11-08T19:26:26.730191+0000","last_undegraded":"2023-11-08T19:26:26.730191+0000","last_fullsized":"2023-11-08T19:26:26.730191+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:34:52.582926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.730618+0000","last_change":"2023-11-08T19:26:25.745857+0000","last_active":"2023-11-08T19:26:26.730618+0000","last_peered":"2023-11-08T19:26:26.730618+0000","last_clean":"2023-11-08T19:26:26.730618+0000","last_became_active":"2023-11-08T19:26:25.745594+0000","last_became_peered":"2023-11-08T19:26:25.745594+0000","last_unstale":"2023-11-08T19:26:26.730618+0000","last_undegraded":"2023-11-08T19:26:26.730618+0000","last_fullsized":"2023-11-08T19:26:26.730618+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:56:38.230155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.763193+0000","last_change":"2023-11-08T19:26:25.755356+0000","last_active":"2023-11-08T19:26:26.763193+0000","last_peered":"2023-11-08T19:26:26.763193+0000","last_clean":"2023-11-08T19:26:26.763193+0000","last_became_active":"2023-11-08T19:26:25.755155+0000","last_became_peered":"2023-11-08T19:26:25.755155+0000","last_unstale":"2023-11-08T19:26:26.763193+0000","last_undegraded":"2023-11-08T19:26:26.763193+0000","last_fullsized":"2023-11-08T19:26:26.763193+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:05:31.920008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.730577+0000","last_change":"2023-11-08T19:26:25.743838+0000","last_active":"2023-11-08T19:26:26.730577+0000","last_peered":"2023-11-08T19:26:26.730577+0000","last_clean":"2023-11-08T19:26:26.730577+0000","last_became_active":"2023-11-08T19:26:25.743563+0000","last_became_peered":"2023-11-08T19:26:25.743563+0000","last_unstale":"2023-11-08T19:26:26.730577+0000","last_undegraded":"2023-11-08T19:26:26.730577+0000","last_fullsized":"2023-11-08T19:26:26.730577+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:03:31.037228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.763160+0000","last_change":"2023-11-08T19:26:25.745921+0000","last_active":"2023-11-08T19:26:26.763160+0000","last_peered":"2023-11-08T19:26:26.763160+0000","last_clean":"2023-11-08T19:26:26.763160+0000","last_became_active":"2023-11-08T19:26:25.745437+0000","last_became_peered":"2023-11-08T19:26:25.745437+0000","last_unstale":"2023-11-08T19:26:26.763160+0000","last_undegraded":"2023-11-08T19:26:26.763160+0000","last_fullsized":"2023-11-08T19:26:26.763160+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:25:17.502470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.730619+0000","last_change":"2023-11-08T19:26:25.738981+0000","last_active":"2023-11-08T19:26:26.730619+0000","last_peered":"2023-11-08T19:26:26.730619+0000","last_clean":"2023-11-08T19:26:26.730619+0000","last_became_active":"2023-11-08T19:26:25.738626+0000","last_became_peered":"2023-11-08T19:26:25.738626+0000","last_unstale":"2023-11-08T19:26:26.730619+0000","last_undegraded":"2023-11-08T19:26:26.730619+0000","last_fullsized":"2023-11-08T19:26:26.730619+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:55:03.172600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.730693+0000","last_change":"2023-11-08T19:26:25.737510+0000","last_active":"2023-11-08T19:26:26.730693+0000","last_peered":"2023-11-08T19:26:26.730693+0000","last_clean":"2023-11-08T19:26:26.730693+0000","last_became_active":"2023-11-08T19:26:25.737239+0000","last_became_peered":"2023-11-08T19:26:25.737239+0000","last_unstale":"2023-11-08T19:26:26.730693+0000","last_undegraded":"2023-11-08T19:26:26.730693+0000","last_fullsized":"2023-11-08T19:26:26.730693+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:06:07.116314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.761602+0000","last_change":"2023-11-08T19:26:25.747608+0000","last_active":"2023-11-08T19:26:26.761602+0000","last_peered":"2023-11-08T19:26:26.761602+0000","last_clean":"2023-11-08T19:26:26.761602+0000","last_became_active":"2023-11-08T19:26:25.747337+0000","last_became_peered":"2023-11-08T19:26:25.747337+0000","last_unstale":"2023-11-08T19:26:26.761602+0000","last_undegraded":"2023-11-08T19:26:26.761602+0000","last_fullsized":"2023-11-08T19:26:26.761602+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:44:11.903976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.761525+0000","last_change":"2023-11-08T19:26:25.743572+0000","last_active":"2023-11-08T19:26:26.761525+0000","last_peered":"2023-11-08T19:26:26.761525+0000","last_clean":"2023-11-08T19:26:26.761525+0000","last_became_active":"2023-11-08T19:26:25.743102+0000","last_became_peered":"2023-11-08T19:26:25.743102+0000","last_unstale":"2023-11-08T19:26:26.761525+0000","last_undegraded":"2023-11-08T19:26:26.761525+0000","last_fullsized":"2023-11-08T19:26:26.761525+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:39:12.204565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.761855+0000","last_change":"2023-11-08T19:26:25.745635+0000","last_active":"2023-11-08T19:26:26.761855+0000","last_peered":"2023-11-08T19:26:26.761855+0000","last_clean":"2023-11-08T19:26:26.761855+0000","last_became_active":"2023-11-08T19:26:25.742827+0000","last_became_peered":"2023-11-08T19:26:25.742827+0000","last_unstale":"2023-11-08T19:26:26.761855+0000","last_undegraded":"2023-11-08T19:26:26.761855+0000","last_fullsized":"2023-11-08T19:26:26.761855+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:29:30.597869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.763073+0000","last_change":"2023-11-08T19:26:25.749212+0000","last_active":"2023-11-08T19:26:26.763073+0000","last_peered":"2023-11-08T19:26:26.763073+0000","last_clean":"2023-11-08T19:26:26.763073+0000","last_became_active":"2023-11-08T19:26:25.748823+0000","last_became_peered":"2023-11-08T19:26:25.748823+0000","last_unstale":"2023-11-08T19:26:26.763073+0000","last_undegraded":"2023-11-08T19:26:26.763073+0000","last_fullsized":"2023-11-08T19:26:26.763073+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:18:26.809499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.762362+0000","last_change":"2023-11-08T19:26:25.741532+0000","last_active":"2023-11-08T19:26:26.762362+0000","last_peered":"2023-11-08T19:26:26.762362+0000","last_clean":"2023-11-08T19:26:26.762362+0000","last_became_active":"2023-11-08T19:26:25.741251+0000","last_became_peered":"2023-11-08T19:26:25.741251+0000","last_unstale":"2023-11-08T19:26:26.762362+0000","last_undegraded":"2023-11-08T19:26:26.762362+0000","last_fullsized":"2023-11-08T19:26:26.762362+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:51:48.252035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.730688+0000","last_change":"2023-11-08T19:26:25.758350+0000","last_active":"2023-11-08T19:26:26.730688+0000","last_peered":"2023-11-08T19:26:26.730688+0000","last_clean":"2023-11-08T19:26:26.730688+0000","last_became_active":"2023-11-08T19:26:25.757848+0000","last_became_peered":"2023-11-08T19:26:25.757848+0000","last_unstale":"2023-11-08T19:26:26.730688+0000","last_undegraded":"2023-11-08T19:26:26.730688+0000","last_fullsized":"2023-11-08T19:26:26.730688+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:23:33.302754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.730778+0000","last_change":"2023-11-08T19:26:25.742808+0000","last_active":"2023-11-08T19:26:26.730778+0000","last_peered":"2023-11-08T19:26:26.730778+0000","last_clean":"2023-11-08T19:26:26.730778+0000","last_became_active":"2023-11-08T19:26:25.742514+0000","last_became_peered":"2023-11-08T19:26:25.742514+0000","last_unstale":"2023-11-08T19:26:26.730778+0000","last_undegraded":"2023-11-08T19:26:26.730778+0000","last_fullsized":"2023-11-08T19:26:26.730778+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:48:11.183540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.762397+0000","last_change":"2023-11-08T19:26:25.744287+0000","last_active":"2023-11-08T19:26:26.762397+0000","last_peered":"2023-11-08T19:26:26.762397+0000","last_clean":"2023-11-08T19:26:26.762397+0000","last_became_active":"2023-11-08T19:26:25.742841+0000","last_became_peered":"2023-11-08T19:26:25.742841+0000","last_unstale":"2023-11-08T19:26:26.762397+0000","last_undegraded":"2023-11-08T19:26:26.762397+0000","last_fullsized":"2023-11-08T19:26:26.762397+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:45:24.697693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.729997+0000","last_change":"2023-11-08T19:26:25.752986+0000","last_active":"2023-11-08T19:26:26.729997+0000","last_peered":"2023-11-08T19:26:26.729997+0000","last_clean":"2023-11-08T19:26:26.729997+0000","last_became_active":"2023-11-08T19:26:25.752647+0000","last_became_peered":"2023-11-08T19:26:25.752647+0000","last_unstale":"2023-11-08T19:26:26.729997+0000","last_undegraded":"2023-11-08T19:26:26.729997+0000","last_fullsized":"2023-11-08T19:26:26.729997+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:22:57.218646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.762957+0000","last_change":"2023-11-08T19:26:25.747032+0000","last_active":"2023-11-08T19:26:26.762957+0000","last_peered":"2023-11-08T19:26:26.762957+0000","last_clean":"2023-11-08T19:26:26.762957+0000","last_became_active":"2023-11-08T19:26:25.746745+0000","last_became_peered":"2023-11-08T19:26:25.746745+0000","last_unstale":"2023-11-08T19:26:26.762957+0000","last_undegraded":"2023-11-08T19:26:26.762957+0000","last_fullsized":"2023-11-08T19:26:26.762957+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:29:44.685878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.761757+0000","last_change":"2023-11-08T19:26:25.753806+0000","last_active":"2023-11-08T19:26:26.761757+0000","last_peered":"2023-11-08T19:26:26.761757+0000","last_clean":"2023-11-08T19:26:26.761757+0000","last_became_active":"2023-11-08T19:26:25.753437+0000","last_became_peered":"2023-11-08T19:26:25.753437+0000","last_unstale":"2023-11-08T19:26:26.761757+0000","last_undegraded":"2023-11-08T19:26:26.761757+0000","last_fullsized":"2023-11-08T19:26:26.761757+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:23:47.505101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.762407+0000","last_change":"2023-11-08T19:26:25.741528+0000","last_active":"2023-11-08T19:26:26.762407+0000","last_peered":"2023-11-08T19:26:26.762407+0000","last_clean":"2023-11-08T19:26:26.762407+0000","last_became_active":"2023-11-08T19:26:25.740687+0000","last_became_peered":"2023-11-08T19:26:25.740687+0000","last_unstale":"2023-11-08T19:26:26.762407+0000","last_undegraded":"2023-11-08T19:26:26.762407+0000","last_fullsized":"2023-11-08T19:26:26.762407+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:02:36.877898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"47'2","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.762067+0000","last_change":"2023-11-08T19:26:24.743581+0000","last_active":"2023-11-08T19:26:26.762067+0000","last_peered":"2023-11-08T19:26:26.762067+0000","last_clean":"2023-11-08T19:26:26.762067+0000","last_became_active":"2023-11-08T19:26:24.743284+0000","last_became_peered":"2023-11-08T19:26:24.743284+0000","last_unstale":"2023-11-08T19:26:26.762067+0000","last_undegraded":"2023-11-08T19:26:26.762067+0000","last_fullsized":"2023-11-08T19:26:26.762067+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:35:23.618441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.739324+0000","last_change":"2023-11-08T19:26:06.297255+0000","last_active":"2023-11-08T19:26:25.739324+0000","last_peered":"2023-11-08T19:26:25.739324+0000","last_clean":"2023-11-08T19:26:25.739324+0000","last_became_active":"2023-11-08T19:26:04.603979+0000","last_became_peered":"2023-11-08T19:26:04.603979+0000","last_unstale":"2023-11-08T19:26:25.739324+0000","last_undegraded":"2023-11-08T19:26:25.739324+0000","last_fullsized":"2023-11-08T19:26:25.739324+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:40:11.164951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079032099999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.730750+0000","last_change":"2023-11-08T19:26:25.740830+0000","last_active":"2023-11-08T19:26:26.730750+0000","last_peered":"2023-11-08T19:26:26.730750+0000","last_clean":"2023-11-08T19:26:26.730750+0000","last_became_active":"2023-11-08T19:26:25.740548+0000","last_became_peered":"2023-11-08T19:26:25.740548+0000","last_unstale":"2023-11-08T19:26:26.730750+0000","last_undegraded":"2023-11-08T19:26:26.730750+0000","last_fullsized":"2023-11-08T19:26:26.730750+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:35:23.446696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"47'2","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.730598+0000","last_change":"2023-11-08T19:26:24.736817+0000","last_active":"2023-11-08T19:26:26.730598+0000","last_peered":"2023-11-08T19:26:26.730598+0000","last_clean":"2023-11-08T19:26:26.730598+0000","last_became_active":"2023-11-08T19:26:24.736538+0000","last_became_peered":"2023-11-08T19:26:24.736538+0000","last_unstale":"2023-11-08T19:26:26.730598+0000","last_undegraded":"2023-11-08T19:26:26.730598+0000","last_fullsized":"2023-11-08T19:26:26.730598+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:22:58.776603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":28,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.729426+0000","last_change":"2023-11-08T19:26:05.609625+0000","last_active":"2023-11-08T19:26:25.729426+0000","last_peered":"2023-11-08T19:26:25.729426+0000","last_clean":"2023-11-08T19:26:25.729426+0000","last_became_active":"2023-11-08T19:26:04.606312+0000","last_became_peered":"2023-11-08T19:26:04.606312+0000","last_unstale":"2023-11-08T19:26:25.729426+0000","last_undegraded":"2023-11-08T19:26:25.729426+0000","last_fullsized":"2023-11-08T19:26:25.729426+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:54:42.032901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000505935,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.762035+0000","last_change":"2023-11-08T19:26:25.745326+0000","last_active":"2023-11-08T19:26:26.762035+0000","last_peered":"2023-11-08T19:26:26.762035+0000","last_clean":"2023-11-08T19:26:26.762035+0000","last_became_active":"2023-11-08T19:26:25.744490+0000","last_became_peered":"2023-11-08T19:26:25.744490+0000","last_unstale":"2023-11-08T19:26:26.762035+0000","last_undegraded":"2023-11-08T19:26:26.762035+0000","last_fullsized":"2023-11-08T19:26:26.762035+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:48:27.754051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"47'1","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.730896+0000","last_change":"2023-11-08T19:26:24.744835+0000","last_active":"2023-11-08T19:26:26.730896+0000","last_peered":"2023-11-08T19:26:26.730896+0000","last_clean":"2023-11-08T19:26:26.730896+0000","last_became_active":"2023-11-08T19:26:24.744487+0000","last_became_peered":"2023-11-08T19:26:24.744487+0000","last_unstale":"2023-11-08T19:26:26.730896+0000","last_undegraded":"2023-11-08T19:26:26.730896+0000","last_fullsized":"2023-11-08T19:26:26.730896+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:13:28.823322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.727850+0000","last_change":"2023-11-08T19:26:06.425133+0000","last_active":"2023-11-08T19:26:25.727850+0000","last_peered":"2023-11-08T19:26:25.727850+0000","last_clean":"2023-11-08T19:26:25.727850+0000","last_became_active":"2023-11-08T19:26:04.601103+0000","last_became_peered":"2023-11-08T19:26:04.601103+0000","last_unstale":"2023-11-08T19:26:25.727850+0000","last_undegraded":"2023-11-08T19:26:25.727850+0000","last_fullsized":"2023-11-08T19:26:25.727850+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:31:14.042561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000229244,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.729685+0000","last_change":"2023-11-08T19:26:25.748005+0000","last_active":"2023-11-08T19:26:26.729685+0000","last_peered":"2023-11-08T19:26:26.729685+0000","last_clean":"2023-11-08T19:26:26.729685+0000","last_became_active":"2023-11-08T19:26:25.747426+0000","last_became_peered":"2023-11-08T19:26:25.747426+0000","last_unstale":"2023-11-08T19:26:26.729685+0000","last_undegraded":"2023-11-08T19:26:26.729685+0000","last_fullsized":"2023-11-08T19:26:26.729685+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:49:29.652891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"47'4","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.748120+0000","last_change":"2023-11-08T19:26:24.738249+0000","last_active":"2023-11-08T19:26:25.748120+0000","last_peered":"2023-11-08T19:26:25.748120+0000","last_clean":"2023-11-08T19:26:25.748120+0000","last_became_active":"2023-11-08T19:26:24.737997+0000","last_became_peered":"2023-11-08T19:26:24.737997+0000","last_unstale":"2023-11-08T19:26:25.748120+0000","last_undegraded":"2023-11-08T19:26:25.748120+0000","last_fullsized":"2023-11-08T19:26:25.748120+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:40:31.299145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.732153+0000","last_change":"2023-11-08T19:26:06.265690+0000","last_active":"2023-11-08T19:26:25.732153+0000","last_peered":"2023-11-08T19:26:25.732153+0000","last_clean":"2023-11-08T19:26:25.732153+0000","last_became_active":"2023-11-08T19:26:04.603397+0000","last_became_peered":"2023-11-08T19:26:04.603397+0000","last_unstale":"2023-11-08T19:26:25.732153+0000","last_undegraded":"2023-11-08T19:26:25.732153+0000","last_fullsized":"2023-11-08T19:26:25.732153+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:50:43.827570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053361499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.730396+0000","last_change":"2023-11-08T19:26:25.743979+0000","last_active":"2023-11-08T19:26:26.730396+0000","last_peered":"2023-11-08T19:26:26.730396+0000","last_clean":"2023-11-08T19:26:26.730396+0000","last_became_active":"2023-11-08T19:26:25.743680+0000","last_became_peered":"2023-11-08T19:26:25.743680+0000","last_unstale":"2023-11-08T19:26:26.730396+0000","last_undegraded":"2023-11-08T19:26:26.730396+0000","last_fullsized":"2023-11-08T19:26:26.730396+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:38:44.411483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.761728+0000","last_change":"2023-11-08T19:26:24.739326+0000","last_active":"2023-11-08T19:26:26.761728+0000","last_peered":"2023-11-08T19:26:26.761728+0000","last_clean":"2023-11-08T19:26:26.761728+0000","last_became_active":"2023-11-08T19:26:24.738986+0000","last_became_peered":"2023-11-08T19:26:24.738986+0000","last_unstale":"2023-11-08T19:26:26.761728+0000","last_undegraded":"2023-11-08T19:26:26.761728+0000","last_fullsized":"2023-11-08T19:26:26.761728+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:23:32.297248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.738268+0000","last_change":"2023-11-08T19:26:06.541338+0000","last_active":"2023-11-08T19:26:25.738268+0000","last_peered":"2023-11-08T19:26:25.738268+0000","last_clean":"2023-11-08T19:26:25.738268+0000","last_became_active":"2023-11-08T19:26:04.602183+0000","last_became_peered":"2023-11-08T19:26:04.602183+0000","last_unstale":"2023-11-08T19:26:25.738268+0000","last_undegraded":"2023-11-08T19:26:25.738268+0000","last_fullsized":"2023-11-08T19:26:25.738268+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:06:47.446686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052029099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.762021+0000","last_change":"2023-11-08T19:26:25.741349+0000","last_active":"2023-11-08T19:26:26.762021+0000","last_peered":"2023-11-08T19:26:26.762021+0000","last_clean":"2023-11-08T19:26:26.762021+0000","last_became_active":"2023-11-08T19:26:25.741052+0000","last_became_peered":"2023-11-08T19:26:25.741052+0000","last_unstale":"2023-11-08T19:26:26.762021+0000","last_undegraded":"2023-11-08T19:26:26.762021+0000","last_fullsized":"2023-11-08T19:26:26.762021+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:49:22.367937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.730162+0000","last_change":"2023-11-08T19:26:24.740255+0000","last_active":"2023-11-08T19:26:25.730162+0000","last_peered":"2023-11-08T19:26:25.730162+0000","last_clean":"2023-11-08T19:26:25.730162+0000","last_became_active":"2023-11-08T19:26:24.739986+0000","last_became_peered":"2023-11-08T19:26:24.739986+0000","last_unstale":"2023-11-08T19:26:25.730162+0000","last_undegraded":"2023-11-08T19:26:25.730162+0000","last_fullsized":"2023-11-08T19:26:25.730162+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:11:46.642828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.734103+0000","last_change":"2023-11-08T19:26:06.394081+0000","last_active":"2023-11-08T19:26:25.734103+0000","last_peered":"2023-11-08T19:26:25.734103+0000","last_clean":"2023-11-08T19:26:25.734103+0000","last_became_active":"2023-11-08T19:26:04.602137+0000","last_became_peered":"2023-11-08T19:26:04.602137+0000","last_unstale":"2023-11-08T19:26:25.734103+0000","last_undegraded":"2023-11-08T19:26:25.734103+0000","last_fullsized":"2023-11-08T19:26:25.734103+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:05:46.951663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056998200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.730862+0000","last_change":"2023-11-08T19:26:25.758675+0000","last_active":"2023-11-08T19:26:26.730862+0000","last_peered":"2023-11-08T19:26:26.730862+0000","last_clean":"2023-11-08T19:26:26.730862+0000","last_became_active":"2023-11-08T19:26:25.758495+0000","last_became_peered":"2023-11-08T19:26:25.758495+0000","last_unstale":"2023-11-08T19:26:26.730862+0000","last_undegraded":"2023-11-08T19:26:26.730862+0000","last_fullsized":"2023-11-08T19:26:26.730862+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:18:26.809499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"47'1","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.762131+0000","last_change":"2023-11-08T19:26:24.738043+0000","last_active":"2023-11-08T19:26:26.762131+0000","last_peered":"2023-11-08T19:26:26.762131+0000","last_clean":"2023-11-08T19:26:26.762131+0000","last_became_active":"2023-11-08T19:26:24.737782+0000","last_became_peered":"2023-11-08T19:26:24.737782+0000","last_unstale":"2023-11-08T19:26:26.762131+0000","last_undegraded":"2023-11-08T19:26:26.762131+0000","last_fullsized":"2023-11-08T19:26:26.762131+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:18:22.010798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.728229+0000","last_change":"2023-11-08T19:26:06.423863+0000","last_active":"2023-11-08T19:26:25.728229+0000","last_peered":"2023-11-08T19:26:25.728229+0000","last_clean":"2023-11-08T19:26:25.728229+0000","last_became_active":"2023-11-08T19:26:04.608639+0000","last_became_peered":"2023-11-08T19:26:04.608639+0000","last_unstale":"2023-11-08T19:26:25.728229+0000","last_undegraded":"2023-11-08T19:26:25.728229+0000","last_fullsized":"2023-11-08T19:26:25.728229+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:04:43.631102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044384999999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.762598+0000","last_change":"2023-11-08T19:26:25.746985+0000","last_active":"2023-11-08T19:26:26.762598+0000","last_peered":"2023-11-08T19:26:26.762598+0000","last_clean":"2023-11-08T19:26:26.762598+0000","last_became_active":"2023-11-08T19:26:25.746718+0000","last_became_peered":"2023-11-08T19:26:25.746718+0000","last_unstale":"2023-11-08T19:26:26.762598+0000","last_undegraded":"2023-11-08T19:26:26.762598+0000","last_fullsized":"2023-11-08T19:26:26.762598+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:36:07.345392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"47'1","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.762456+0000","last_change":"2023-11-08T19:26:24.742192+0000","last_active":"2023-11-08T19:26:26.762456+0000","last_peered":"2023-11-08T19:26:26.762456+0000","last_clean":"2023-11-08T19:26:26.762456+0000","last_became_active":"2023-11-08T19:26:24.741925+0000","last_became_peered":"2023-11-08T19:26:24.741925+0000","last_unstale":"2023-11-08T19:26:26.762456+0000","last_undegraded":"2023-11-08T19:26:26.762456+0000","last_fullsized":"2023-11-08T19:26:26.762456+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:05:03.696760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.733044+0000","last_change":"2023-11-08T19:26:06.265318+0000","last_active":"2023-11-08T19:26:25.733044+0000","last_peered":"2023-11-08T19:26:25.733044+0000","last_clean":"2023-11-08T19:26:25.733044+0000","last_became_active":"2023-11-08T19:26:04.603409+0000","last_became_peered":"2023-11-08T19:26:04.603409+0000","last_unstale":"2023-11-08T19:26:25.733044+0000","last_undegraded":"2023-11-08T19:26:25.733044+0000","last_fullsized":"2023-11-08T19:26:25.733044+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:27:24.933111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065684300000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.762588+0000","last_change":"2023-11-08T19:26:25.750379+0000","last_active":"2023-11-08T19:26:26.762588+0000","last_peered":"2023-11-08T19:26:26.762588+0000","last_clean":"2023-11-08T19:26:26.762588+0000","last_became_active":"2023-11-08T19:26:25.750060+0000","last_became_peered":"2023-11-08T19:26:25.750060+0000","last_unstale":"2023-11-08T19:26:26.762588+0000","last_undegraded":"2023-11-08T19:26:26.762588+0000","last_fullsized":"2023-11-08T19:26:26.762588+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:22:59.782108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"47'6","reported_seq":18,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.761687+0000","last_change":"2023-11-08T19:26:24.742213+0000","last_active":"2023-11-08T19:26:26.761687+0000","last_peered":"2023-11-08T19:26:26.761687+0000","last_clean":"2023-11-08T19:26:26.761687+0000","last_became_active":"2023-11-08T19:26:24.741916+0000","last_became_peered":"2023-11-08T19:26:24.741916+0000","last_unstale":"2023-11-08T19:26:26.761687+0000","last_undegraded":"2023-11-08T19:26:26.761687+0000","last_fullsized":"2023-11-08T19:26:26.761687+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:42:12.345053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":28,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.728613+0000","last_change":"2023-11-08T19:26:05.610012+0000","last_active":"2023-11-08T19:26:25.728613+0000","last_peered":"2023-11-08T19:26:25.728613+0000","last_clean":"2023-11-08T19:26:25.728613+0000","last_became_active":"2023-11-08T19:26:04.603209+0000","last_became_peered":"2023-11-08T19:26:04.603209+0000","last_unstale":"2023-11-08T19:26:25.728613+0000","last_undegraded":"2023-11-08T19:26:25.728613+0000","last_fullsized":"2023-11-08T19:26:25.728613+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:35:02.306997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053335100000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.762447+0000","last_change":"2023-11-08T19:26:25.749064+0000","last_active":"2023-11-08T19:26:26.762447+0000","last_peered":"2023-11-08T19:26:26.762447+0000","last_clean":"2023-11-08T19:26:26.762447+0000","last_became_active":"2023-11-08T19:26:25.748500+0000","last_became_peered":"2023-11-08T19:26:25.748500+0000","last_unstale":"2023-11-08T19:26:26.762447+0000","last_undegraded":"2023-11-08T19:26:26.762447+0000","last_fullsized":"2023-11-08T19:26:26.762447+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:49:29.652891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"47'3","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.760398+0000","last_change":"2023-11-08T19:26:24.744478+0000","last_active":"2023-11-08T19:26:25.760398+0000","last_peered":"2023-11-08T19:26:25.760398+0000","last_clean":"2023-11-08T19:26:25.760398+0000","last_became_active":"2023-11-08T19:26:24.744253+0000","last_became_peered":"2023-11-08T19:26:24.744253+0000","last_unstale":"2023-11-08T19:26:25.760398+0000","last_undegraded":"2023-11-08T19:26:25.760398+0000","last_fullsized":"2023-11-08T19:26:25.760398+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:23:46.499595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.737675+0000","last_change":"2023-11-08T19:26:05.606946+0000","last_active":"2023-11-08T19:26:25.737675+0000","last_peered":"2023-11-08T19:26:25.737675+0000","last_clean":"2023-11-08T19:26:25.737675+0000","last_became_active":"2023-11-08T19:26:04.598775+0000","last_became_peered":"2023-11-08T19:26:04.598775+0000","last_unstale":"2023-11-08T19:26:25.737675+0000","last_undegraded":"2023-11-08T19:26:25.737675+0000","last_fullsized":"2023-11-08T19:26:25.737675+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:20:56.898927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049870399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.762480+0000","last_change":"2023-11-08T19:26:25.746669+0000","last_active":"2023-11-08T19:26:26.762480+0000","last_peered":"2023-11-08T19:26:26.762480+0000","last_clean":"2023-11-08T19:26:26.762480+0000","last_became_active":"2023-11-08T19:26:25.746375+0000","last_became_peered":"2023-11-08T19:26:25.746375+0000","last_unstale":"2023-11-08T19:26:26.762480+0000","last_undegraded":"2023-11-08T19:26:26.762480+0000","last_fullsized":"2023-11-08T19:26:26.762480+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:42:13.350558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.762625+0000","last_change":"2023-11-08T19:26:24.745917+0000","last_active":"2023-11-08T19:26:26.762625+0000","last_peered":"2023-11-08T19:26:26.762625+0000","last_clean":"2023-11-08T19:26:26.762625+0000","last_became_active":"2023-11-08T19:26:24.745606+0000","last_became_peered":"2023-11-08T19:26:24.745606+0000","last_unstale":"2023-11-08T19:26:26.762625+0000","last_undegraded":"2023-11-08T19:26:26.762625+0000","last_fullsized":"2023-11-08T19:26:26.762625+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:48:10.178035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.731955+0000","last_change":"2023-11-08T19:26:06.265907+0000","last_active":"2023-11-08T19:26:25.731955+0000","last_peered":"2023-11-08T19:26:25.731955+0000","last_clean":"2023-11-08T19:26:25.731955+0000","last_became_active":"2023-11-08T19:26:04.604772+0000","last_became_peered":"2023-11-08T19:26:04.604772+0000","last_unstale":"2023-11-08T19:26:25.731955+0000","last_undegraded":"2023-11-08T19:26:25.731955+0000","last_fullsized":"2023-11-08T19:26:25.731955+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:44:16.139794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066875300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.761632+0000","last_change":"2023-11-08T19:26:25.751452+0000","last_active":"2023-11-08T19:26:26.761632+0000","last_peered":"2023-11-08T19:26:26.761632+0000","last_clean":"2023-11-08T19:26:26.761632+0000","last_became_active":"2023-11-08T19:26:25.751140+0000","last_became_peered":"2023-11-08T19:26:25.751140+0000","last_unstale":"2023-11-08T19:26:26.761632+0000","last_undegraded":"2023-11-08T19:26:26.761632+0000","last_fullsized":"2023-11-08T19:26:26.761632+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:43:16.409064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.731355+0000","last_change":"2023-11-08T19:26:24.741119+0000","last_active":"2023-11-08T19:26:25.731355+0000","last_peered":"2023-11-08T19:26:25.731355+0000","last_clean":"2023-11-08T19:26:25.731355+0000","last_became_active":"2023-11-08T19:26:24.740821+0000","last_became_peered":"2023-11-08T19:26:24.740821+0000","last_unstale":"2023-11-08T19:26:25.731355+0000","last_undegraded":"2023-11-08T19:26:25.731355+0000","last_fullsized":"2023-11-08T19:26:25.731355+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:45:23.692188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.734895+0000","last_change":"2023-11-08T19:26:06.394488+0000","last_active":"2023-11-08T19:26:25.734895+0000","last_peered":"2023-11-08T19:26:25.734895+0000","last_clean":"2023-11-08T19:26:25.734895+0000","last_became_active":"2023-11-08T19:26:04.606805+0000","last_became_peered":"2023-11-08T19:26:04.606805+0000","last_unstale":"2023-11-08T19:26:25.734895+0000","last_undegraded":"2023-11-08T19:26:25.734895+0000","last_fullsized":"2023-11-08T19:26:25.734895+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:25:20.857133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045317000000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.763082+0000","last_change":"2023-11-08T19:26:25.755102+0000","last_active":"2023-11-08T19:26:26.763082+0000","last_peered":"2023-11-08T19:26:26.763082+0000","last_clean":"2023-11-08T19:26:26.763082+0000","last_became_active":"2023-11-08T19:26:25.750220+0000","last_became_peered":"2023-11-08T19:26:25.750220+0000","last_unstale":"2023-11-08T19:26:26.763082+0000","last_undegraded":"2023-11-08T19:26:26.763082+0000","last_fullsized":"2023-11-08T19:26:26.763082+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:11:13.832264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"47'8","reported_seq":20,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.762303+0000","last_change":"2023-11-08T19:26:24.743410+0000","last_active":"2023-11-08T19:26:26.762303+0000","last_peered":"2023-11-08T19:26:26.762303+0000","last_clean":"2023-11-08T19:26:26.762303+0000","last_became_active":"2023-11-08T19:26:24.743162+0000","last_became_peered":"2023-11-08T19:26:24.743162+0000","last_unstale":"2023-11-08T19:26:26.762303+0000","last_undegraded":"2023-11-08T19:26:26.762303+0000","last_fullsized":"2023-11-08T19:26:26.762303+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:22:56.213140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.727378+0000","last_change":"2023-11-08T19:26:06.423754+0000","last_active":"2023-11-08T19:26:25.727378+0000","last_peered":"2023-11-08T19:26:25.727378+0000","last_clean":"2023-11-08T19:26:25.727378+0000","last_became_active":"2023-11-08T19:26:04.599322+0000","last_became_peered":"2023-11-08T19:26:04.599322+0000","last_unstale":"2023-11-08T19:26:25.727378+0000","last_undegraded":"2023-11-08T19:26:25.727378+0000","last_fullsized":"2023-11-08T19:26:25.727378+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:13:08.689128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042670000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.761894+0000","last_change":"2023-11-08T19:26:25.736435+0000","last_active":"2023-11-08T19:26:26.761894+0000","last_peered":"2023-11-08T19:26:26.761894+0000","last_clean":"2023-11-08T19:26:26.761894+0000","last_became_active":"2023-11-08T19:26:25.736134+0000","last_became_peered":"2023-11-08T19:26:25.736134+0000","last_unstale":"2023-11-08T19:26:26.761894+0000","last_undegraded":"2023-11-08T19:26:26.761894+0000","last_fullsized":"2023-11-08T19:26:26.761894+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:06:45.575533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.729611+0000","last_change":"2023-11-08T19:26:24.743478+0000","last_active":"2023-11-08T19:26:25.729611+0000","last_peered":"2023-11-08T19:26:25.729611+0000","last_clean":"2023-11-08T19:26:25.729611+0000","last_became_active":"2023-11-08T19:26:24.743177+0000","last_became_peered":"2023-11-08T19:26:24.743177+0000","last_unstale":"2023-11-08T19:26:25.729611+0000","last_undegraded":"2023-11-08T19:26:25.729611+0000","last_fullsized":"2023-11-08T19:26:25.729611+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:29:43.680373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.733903+0000","last_change":"2023-11-08T19:26:06.394320+0000","last_active":"2023-11-08T19:26:25.733903+0000","last_peered":"2023-11-08T19:26:25.733903+0000","last_clean":"2023-11-08T19:26:25.733903+0000","last_became_active":"2023-11-08T19:26:04.603212+0000","last_became_peered":"2023-11-08T19:26:04.603212+0000","last_unstale":"2023-11-08T19:26:25.733903+0000","last_undegraded":"2023-11-08T19:26:25.733903+0000","last_fullsized":"2023-11-08T19:26:25.733903+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:25:29.947079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054695399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.763031+0000","last_change":"2023-11-08T19:26:25.758900+0000","last_active":"2023-11-08T19:26:26.763031+0000","last_peered":"2023-11-08T19:26:26.763031+0000","last_clean":"2023-11-08T19:26:26.763031+0000","last_became_active":"2023-11-08T19:26:25.758612+0000","last_became_peered":"2023-11-08T19:26:25.758612+0000","last_unstale":"2023-11-08T19:26:26.763031+0000","last_undegraded":"2023-11-08T19:26:26.763031+0000","last_fullsized":"2023-11-08T19:26:26.763031+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:03:31.037228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.728395+0000","last_change":"2023-11-08T19:26:24.739300+0000","last_active":"2023-11-08T19:26:25.728395+0000","last_peered":"2023-11-08T19:26:25.728395+0000","last_clean":"2023-11-08T19:26:25.728395+0000","last_became_active":"2023-11-08T19:26:24.738988+0000","last_became_peered":"2023-11-08T19:26:24.738988+0000","last_unstale":"2023-11-08T19:26:25.728395+0000","last_undegraded":"2023-11-08T19:26:25.728395+0000","last_fullsized":"2023-11-08T19:26:25.728395+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:44:36.273988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.728522+0000","last_change":"2023-11-08T19:26:06.424073+0000","last_active":"2023-11-08T19:26:25.728522+0000","last_peered":"2023-11-08T19:26:25.728522+0000","last_clean":"2023-11-08T19:26:25.728522+0000","last_became_active":"2023-11-08T19:26:04.599046+0000","last_became_peered":"2023-11-08T19:26:04.599046+0000","last_unstale":"2023-11-08T19:26:25.728522+0000","last_undegraded":"2023-11-08T19:26:25.728522+0000","last_fullsized":"2023-11-08T19:26:25.728522+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:04:43.562566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035092199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.762035+0000","last_change":"2023-11-08T19:26:25.744142+0000","last_active":"2023-11-08T19:26:26.762035+0000","last_peered":"2023-11-08T19:26:26.762035+0000","last_clean":"2023-11-08T19:26:26.762035+0000","last_became_active":"2023-11-08T19:26:25.743686+0000","last_became_peered":"2023-11-08T19:26:25.743686+0000","last_unstale":"2023-11-08T19:26:26.762035+0000","last_undegraded":"2023-11-08T19:26:26.762035+0000","last_fullsized":"2023-11-08T19:26:26.762035+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:29:30.597869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"47'2","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.762518+0000","last_change":"2023-11-08T19:26:24.740785+0000","last_active":"2023-11-08T19:26:26.762518+0000","last_peered":"2023-11-08T19:26:26.762518+0000","last_clean":"2023-11-08T19:26:26.762518+0000","last_became_active":"2023-11-08T19:26:24.740482+0000","last_became_peered":"2023-11-08T19:26:24.740482+0000","last_unstale":"2023-11-08T19:26:26.762518+0000","last_undegraded":"2023-11-08T19:26:26.762518+0000","last_fullsized":"2023-11-08T19:26:26.762518+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:22:25.145083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":28,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.728645+0000","last_change":"2023-11-08T19:26:05.608980+0000","last_active":"2023-11-08T19:26:25.728645+0000","last_peered":"2023-11-08T19:26:25.728645+0000","last_clean":"2023-11-08T19:26:25.728645+0000","last_became_active":"2023-11-08T19:26:04.606386+0000","last_became_peered":"2023-11-08T19:26:04.606386+0000","last_unstale":"2023-11-08T19:26:25.728645+0000","last_undegraded":"2023-11-08T19:26:25.728645+0000","last_fullsized":"2023-11-08T19:26:25.728645+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:41:50.771457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037950599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.730654+0000","last_change":"2023-11-08T19:26:25.757783+0000","last_active":"2023-11-08T19:26:26.730654+0000","last_peered":"2023-11-08T19:26:26.730654+0000","last_clean":"2023-11-08T19:26:26.730654+0000","last_became_active":"2023-11-08T19:26:25.757482+0000","last_became_peered":"2023-11-08T19:26:25.757482+0000","last_unstale":"2023-11-08T19:26:26.730654+0000","last_undegraded":"2023-11-08T19:26:26.730654+0000","last_fullsized":"2023-11-08T19:26:26.730654+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:11:18.036694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.728806+0000","last_change":"2023-11-08T19:26:24.743633+0000","last_active":"2023-11-08T19:26:25.728806+0000","last_peered":"2023-11-08T19:26:25.728806+0000","last_clean":"2023-11-08T19:26:25.728806+0000","last_became_active":"2023-11-08T19:26:24.743288+0000","last_became_peered":"2023-11-08T19:26:24.743288+0000","last_unstale":"2023-11-08T19:26:25.728806+0000","last_undegraded":"2023-11-08T19:26:25.728806+0000","last_fullsized":"2023-11-08T19:26:25.728806+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:20:09.309300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.738107+0000","last_change":"2023-11-08T19:26:06.541623+0000","last_active":"2023-11-08T19:26:25.738107+0000","last_peered":"2023-11-08T19:26:25.738107+0000","last_clean":"2023-11-08T19:26:25.738107+0000","last_became_active":"2023-11-08T19:26:04.601961+0000","last_became_peered":"2023-11-08T19:26:04.601961+0000","last_unstale":"2023-11-08T19:26:25.738107+0000","last_undegraded":"2023-11-08T19:26:25.738107+0000","last_fullsized":"2023-11-08T19:26:25.738107+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:11:26.508634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000466386,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.761661+0000","last_change":"2023-11-08T19:26:25.747292+0000","last_active":"2023-11-08T19:26:26.761661+0000","last_peered":"2023-11-08T19:26:26.761661+0000","last_clean":"2023-11-08T19:26:26.761661+0000","last_became_active":"2023-11-08T19:26:25.747014+0000","last_became_peered":"2023-11-08T19:26:25.747014+0000","last_unstale":"2023-11-08T19:26:26.761661+0000","last_undegraded":"2023-11-08T19:26:26.761661+0000","last_fullsized":"2023-11-08T19:26:26.761661+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:42:14.637155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"47'1","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.759266+0000","last_change":"2023-11-08T19:26:24.733981+0000","last_active":"2023-11-08T19:26:25.759266+0000","last_peered":"2023-11-08T19:26:25.759266+0000","last_clean":"2023-11-08T19:26:25.759266+0000","last_became_active":"2023-11-08T19:26:24.733628+0000","last_became_peered":"2023-11-08T19:26:24.733628+0000","last_unstale":"2023-11-08T19:26:25.759266+0000","last_undegraded":"2023-11-08T19:26:25.759266+0000","last_fullsized":"2023-11-08T19:26:25.759266+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:07:07.580880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.727605+0000","last_change":"2023-11-08T19:26:06.424464+0000","last_active":"2023-11-08T19:26:25.727605+0000","last_peered":"2023-11-08T19:26:25.727605+0000","last_clean":"2023-11-08T19:26:25.727605+0000","last_became_active":"2023-11-08T19:26:04.603144+0000","last_became_peered":"2023-11-08T19:26:04.603144+0000","last_unstale":"2023-11-08T19:26:25.727605+0000","last_undegraded":"2023-11-08T19:26:25.727605+0000","last_fullsized":"2023-11-08T19:26:25.727605+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:42:55.269365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054090799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.730354+0000","last_change":"2023-11-08T19:26:25.745301+0000","last_active":"2023-11-08T19:26:26.730354+0000","last_peered":"2023-11-08T19:26:26.730354+0000","last_clean":"2023-11-08T19:26:26.730354+0000","last_became_active":"2023-11-08T19:26:25.745014+0000","last_became_peered":"2023-11-08T19:26:25.745014+0000","last_unstale":"2023-11-08T19:26:26.730354+0000","last_undegraded":"2023-11-08T19:26:26.730354+0000","last_fullsized":"2023-11-08T19:26:26.730354+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:22:26.150588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"47'2","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.762839+0000","last_change":"2023-11-08T19:26:24.739629+0000","last_active":"2023-11-08T19:26:26.762839+0000","last_peered":"2023-11-08T19:26:26.762839+0000","last_clean":"2023-11-08T19:26:26.762839+0000","last_became_active":"2023-11-08T19:26:24.739346+0000","last_became_peered":"2023-11-08T19:26:24.739346+0000","last_unstale":"2023-11-08T19:26:26.762839+0000","last_undegraded":"2023-11-08T19:26:26.762839+0000","last_fullsized":"2023-11-08T19:26:26.762839+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:25:50.081273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.738253+0000","last_change":"2023-11-08T19:26:05.606898+0000","last_active":"2023-11-08T19:26:25.738253+0000","last_peered":"2023-11-08T19:26:25.738253+0000","last_clean":"2023-11-08T19:26:25.738253+0000","last_became_active":"2023-11-08T19:26:04.603955+0000","last_became_peered":"2023-11-08T19:26:04.603955+0000","last_unstale":"2023-11-08T19:26:25.738253+0000","last_undegraded":"2023-11-08T19:26:25.738253+0000","last_fullsized":"2023-11-08T19:26:25.738253+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:19:11.589559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036473499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.762090+0000","last_change":"2023-11-08T19:26:25.749337+0000","last_active":"2023-11-08T19:26:26.762090+0000","last_peered":"2023-11-08T19:26:26.762090+0000","last_clean":"2023-11-08T19:26:26.762090+0000","last_became_active":"2023-11-08T19:26:25.749069+0000","last_became_peered":"2023-11-08T19:26:25.749069+0000","last_unstale":"2023-11-08T19:26:26.762090+0000","last_undegraded":"2023-11-08T19:26:26.762090+0000","last_fullsized":"2023-11-08T19:26:26.762090+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:03:20.639343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"47'1","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.762883+0000","last_change":"2023-11-08T19:26:24.742330+0000","last_active":"2023-11-08T19:26:26.762883+0000","last_peered":"2023-11-08T19:26:26.762883+0000","last_clean":"2023-11-08T19:26:26.762883+0000","last_became_active":"2023-11-08T19:26:24.742055+0000","last_became_peered":"2023-11-08T19:26:24.742055+0000","last_unstale":"2023-11-08T19:26:26.762883+0000","last_undegraded":"2023-11-08T19:26:26.762883+0000","last_fullsized":"2023-11-08T19:26:26.762883+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:48:26.748546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.732066+0000","last_change":"2023-11-08T19:26:06.265686+0000","last_active":"2023-11-08T19:26:25.732066+0000","last_peered":"2023-11-08T19:26:25.732066+0000","last_clean":"2023-11-08T19:26:25.732066+0000","last_became_active":"2023-11-08T19:26:04.602111+0000","last_became_peered":"2023-11-08T19:26:04.602111+0000","last_unstale":"2023-11-08T19:26:25.732066+0000","last_undegraded":"2023-11-08T19:26:25.732066+0000","last_fullsized":"2023-11-08T19:26:25.732066+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:40:11.164951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056197899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.730988+0000","last_change":"2023-11-08T19:26:25.746346+0000","last_active":"2023-11-08T19:26:26.730988+0000","last_peered":"2023-11-08T19:26:26.730988+0000","last_clean":"2023-11-08T19:26:26.730988+0000","last_became_active":"2023-11-08T19:26:25.746052+0000","last_became_peered":"2023-11-08T19:26:25.746052+0000","last_unstale":"2023-11-08T19:26:26.730988+0000","last_undegraded":"2023-11-08T19:26:26.730988+0000","last_fullsized":"2023-11-08T19:26:26.730988+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:48:27.754051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.761287+0000","last_change":"2023-11-08T19:26:24.740736+0000","last_active":"2023-11-08T19:26:26.761287+0000","last_peered":"2023-11-08T19:26:26.761287+0000","last_clean":"2023-11-08T19:26:26.761287+0000","last_became_active":"2023-11-08T19:26:24.740479+0000","last_became_peered":"2023-11-08T19:26:24.740479+0000","last_unstale":"2023-11-08T19:26:26.761287+0000","last_undegraded":"2023-11-08T19:26:26.761287+0000","last_fullsized":"2023-11-08T19:26:26.761287+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:20:16.421029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.734247+0000","last_change":"2023-11-08T19:26:06.394075+0000","last_active":"2023-11-08T19:26:25.734247+0000","last_peered":"2023-11-08T19:26:25.734247+0000","last_clean":"2023-11-08T19:26:25.734247+0000","last_became_active":"2023-11-08T19:26:04.606967+0000","last_became_peered":"2023-11-08T19:26:04.606967+0000","last_unstale":"2023-11-08T19:26:25.734247+0000","last_undegraded":"2023-11-08T19:26:25.734247+0000","last_fullsized":"2023-11-08T19:26:25.734247+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:19:56.286835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040971999999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.762648+0000","last_change":"2023-11-08T19:26:25.744126+0000","last_active":"2023-11-08T19:26:26.762648+0000","last_peered":"2023-11-08T19:26:26.762648+0000","last_clean":"2023-11-08T19:26:26.762648+0000","last_became_active":"2023-11-08T19:26:25.743853+0000","last_became_peered":"2023-11-08T19:26:25.743853+0000","last_unstale":"2023-11-08T19:26:26.762648+0000","last_undegraded":"2023-11-08T19:26:26.762648+0000","last_fullsized":"2023-11-08T19:26:26.762648+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:34:52.582926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.762073+0000","last_change":"2023-11-08T19:26:24.746451+0000","last_active":"2023-11-08T19:26:26.762073+0000","last_peered":"2023-11-08T19:26:26.762073+0000","last_clean":"2023-11-08T19:26:26.762073+0000","last_became_active":"2023-11-08T19:26:24.746152+0000","last_became_peered":"2023-11-08T19:26:24.746152+0000","last_unstale":"2023-11-08T19:26:26.762073+0000","last_undegraded":"2023-11-08T19:26:26.762073+0000","last_fullsized":"2023-11-08T19:26:26.762073+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:56:37.224649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":28,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.731324+0000","last_change":"2023-11-08T19:26:05.608892+0000","last_active":"2023-11-08T19:26:25.731324+0000","last_peered":"2023-11-08T19:26:25.731324+0000","last_clean":"2023-11-08T19:26:25.731324+0000","last_became_active":"2023-11-08T19:26:04.604519+0000","last_became_peered":"2023-11-08T19:26:04.604519+0000","last_unstale":"2023-11-08T19:26:25.731324+0000","last_undegraded":"2023-11-08T19:26:25.731324+0000","last_fullsized":"2023-11-08T19:26:25.731324+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:11:26.508634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051248099999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.730957+0000","last_change":"2023-11-08T19:26:25.734469+0000","last_active":"2023-11-08T19:26:26.730957+0000","last_peered":"2023-11-08T19:26:26.730957+0000","last_clean":"2023-11-08T19:26:26.730957+0000","last_became_active":"2023-11-08T19:26:25.734211+0000","last_became_peered":"2023-11-08T19:26:25.734211+0000","last_unstale":"2023-11-08T19:26:26.730957+0000","last_undegraded":"2023-11-08T19:26:26.730957+0000","last_fullsized":"2023-11-08T19:26:26.730957+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:13:29.828827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.761686+0000","last_change":"2023-11-08T19:26:24.742825+0000","last_active":"2023-11-08T19:26:26.761686+0000","last_peered":"2023-11-08T19:26:26.761686+0000","last_clean":"2023-11-08T19:26:26.761686+0000","last_became_active":"2023-11-08T19:26:24.742467+0000","last_became_peered":"2023-11-08T19:26:24.742467+0000","last_unstale":"2023-11-08T19:26:26.761686+0000","last_undegraded":"2023-11-08T19:26:26.761686+0000","last_fullsized":"2023-11-08T19:26:26.761686+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:49:52.879282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.728303+0000","last_change":"2023-11-08T19:26:06.424236+0000","last_active":"2023-11-08T19:26:25.728303+0000","last_peered":"2023-11-08T19:26:25.728303+0000","last_clean":"2023-11-08T19:26:25.728303+0000","last_became_active":"2023-11-08T19:26:04.600115+0000","last_became_peered":"2023-11-08T19:26:04.600115+0000","last_unstale":"2023-11-08T19:26:25.728303+0000","last_undegraded":"2023-11-08T19:26:25.728303+0000","last_fullsized":"2023-11-08T19:26:25.728303+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:18:01.876605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065803900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.762236+0000","last_change":"2023-11-08T19:26:25.743964+0000","last_active":"2023-11-08T19:26:26.762236+0000","last_peered":"2023-11-08T19:26:26.762236+0000","last_clean":"2023-11-08T19:26:26.762236+0000","last_became_active":"2023-11-08T19:26:25.743403+0000","last_became_peered":"2023-11-08T19:26:25.743403+0000","last_unstale":"2023-11-08T19:26:26.762236+0000","last_undegraded":"2023-11-08T19:26:26.762236+0000","last_fullsized":"2023-11-08T19:26:26.762236+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:29:44.685878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"47'2","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.730526+0000","last_change":"2023-11-08T19:26:24.735862+0000","last_active":"2023-11-08T19:26:26.730526+0000","last_peered":"2023-11-08T19:26:26.730526+0000","last_clean":"2023-11-08T19:26:26.730526+0000","last_became_active":"2023-11-08T19:26:24.735588+0000","last_became_peered":"2023-11-08T19:26:24.735588+0000","last_unstale":"2023-11-08T19:26:26.730526+0000","last_undegraded":"2023-11-08T19:26:26.730526+0000","last_fullsized":"2023-11-08T19:26:26.730526+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:05:03.696760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.737942+0000","last_change":"2023-11-08T19:26:06.541360+0000","last_active":"2023-11-08T19:26:25.737942+0000","last_peered":"2023-11-08T19:26:25.737942+0000","last_clean":"2023-11-08T19:26:25.737942+0000","last_became_active":"2023-11-08T19:26:04.603725+0000","last_became_peered":"2023-11-08T19:26:04.603725+0000","last_unstale":"2023-11-08T19:26:25.737942+0000","last_undegraded":"2023-11-08T19:26:25.737942+0000","last_fullsized":"2023-11-08T19:26:25.737942+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:28:48.576742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053365400000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.762526+0000","last_change":"2023-11-08T19:26:25.736017+0000","last_active":"2023-11-08T19:26:26.762526+0000","last_peered":"2023-11-08T19:26:26.762526+0000","last_clean":"2023-11-08T19:26:26.762526+0000","last_became_active":"2023-11-08T19:26:25.735715+0000","last_became_peered":"2023-11-08T19:26:25.735715+0000","last_unstale":"2023-11-08T19:26:26.762526+0000","last_undegraded":"2023-11-08T19:26:26.762526+0000","last_fullsized":"2023-11-08T19:26:26.762526+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:56:44.858396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.762649+0000","last_change":"2023-11-08T19:26:24.741527+0000","last_active":"2023-11-08T19:26:26.762649+0000","last_peered":"2023-11-08T19:26:26.762649+0000","last_clean":"2023-11-08T19:26:26.762649+0000","last_became_active":"2023-11-08T19:26:24.741178+0000","last_became_peered":"2023-11-08T19:26:24.741178+0000","last_unstale":"2023-11-08T19:26:26.762649+0000","last_undegraded":"2023-11-08T19:26:26.762649+0000","last_fullsized":"2023-11-08T19:26:26.762649+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:06:06.110808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":28,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.728501+0000","last_change":"2023-11-08T19:26:05.609824+0000","last_active":"2023-11-08T19:26:25.728501+0000","last_peered":"2023-11-08T19:26:25.728501+0000","last_clean":"2023-11-08T19:26:25.728501+0000","last_became_active":"2023-11-08T19:26:04.603320+0000","last_became_peered":"2023-11-08T19:26:04.603320+0000","last_unstale":"2023-11-08T19:26:25.728501+0000","last_undegraded":"2023-11-08T19:26:25.728501+0000","last_fullsized":"2023-11-08T19:26:25.728501+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:05:45.976615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063225,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.730644+0000","last_change":"2023-11-08T19:26:25.746887+0000","last_active":"2023-11-08T19:26:26.730644+0000","last_peered":"2023-11-08T19:26:26.730644+0000","last_clean":"2023-11-08T19:26:26.730644+0000","last_became_active":"2023-11-08T19:26:25.746588+0000","last_became_peered":"2023-11-08T19:26:25.746588+0000","last_unstale":"2023-11-08T19:26:26.730644+0000","last_undegraded":"2023-11-08T19:26:26.730644+0000","last_fullsized":"2023-11-08T19:26:26.730644+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:23:47.505101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.728774+0000","last_change":"2023-11-08T19:26:24.742178+0000","last_active":"2023-11-08T19:26:25.728774+0000","last_peered":"2023-11-08T19:26:25.728774+0000","last_clean":"2023-11-08T19:26:25.728774+0000","last_became_active":"2023-11-08T19:26:24.741868+0000","last_became_peered":"2023-11-08T19:26:24.741868+0000","last_unstale":"2023-11-08T19:26:25.728774+0000","last_undegraded":"2023-11-08T19:26:25.728774+0000","last_fullsized":"2023-11-08T19:26:25.728774+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:35:23.618441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.732053+0000","last_change":"2023-11-08T19:26:06.296937+0000","last_active":"2023-11-08T19:26:25.732053+0000","last_peered":"2023-11-08T19:26:25.732053+0000","last_clean":"2023-11-08T19:26:25.732053+0000","last_became_active":"2023-11-08T19:26:04.599999+0000","last_became_peered":"2023-11-08T19:26:04.599999+0000","last_unstale":"2023-11-08T19:26:25.732053+0000","last_undegraded":"2023-11-08T19:26:25.732053+0000","last_fullsized":"2023-11-08T19:26:25.732053+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:50:43.827570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025605700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.761976+0000","last_change":"2023-11-08T19:26:25.753296+0000","last_active":"2023-11-08T19:26:26.761976+0000","last_peered":"2023-11-08T19:26:26.761976+0000","last_clean":"2023-11-08T19:26:26.761976+0000","last_became_active":"2023-11-08T19:26:25.752989+0000","last_became_peered":"2023-11-08T19:26:25.752989+0000","last_unstale":"2023-11-08T19:26:26.761976+0000","last_undegraded":"2023-11-08T19:26:26.761976+0000","last_fullsized":"2023-11-08T19:26:26.761976+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:23:33.302754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"47'2","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.747075+0000","last_change":"2023-11-08T19:26:24.741588+0000","last_active":"2023-11-08T19:26:25.747075+0000","last_peered":"2023-11-08T19:26:25.747075+0000","last_clean":"2023-11-08T19:26:25.747075+0000","last_became_active":"2023-11-08T19:26:24.741371+0000","last_became_peered":"2023-11-08T19:26:24.741371+0000","last_unstale":"2023-11-08T19:26:25.747075+0000","last_undegraded":"2023-11-08T19:26:25.747075+0000","last_fullsized":"2023-11-08T19:26:25.747075+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:51:03.961763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.728463+0000","last_change":"2023-11-08T19:26:06.424511+0000","last_active":"2023-11-08T19:26:25.728463+0000","last_peered":"2023-11-08T19:26:25.728463+0000","last_clean":"2023-11-08T19:26:25.728463+0000","last_became_active":"2023-11-08T19:26:04.598973+0000","last_became_peered":"2023-11-08T19:26:04.598973+0000","last_unstale":"2023-11-08T19:26:25.728463+0000","last_undegraded":"2023-11-08T19:26:25.728463+0000","last_fullsized":"2023-11-08T19:26:25.728463+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:41:53.497456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031755900000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.762212+0000","last_change":"2023-11-08T19:26:25.742775+0000","last_active":"2023-11-08T19:26:26.762212+0000","last_peered":"2023-11-08T19:26:26.762212+0000","last_clean":"2023-11-08T19:26:26.762212+0000","last_became_active":"2023-11-08T19:26:25.740742+0000","last_became_peered":"2023-11-08T19:26:25.740742+0000","last_unstale":"2023-11-08T19:26:26.762212+0000","last_undegraded":"2023-11-08T19:26:26.762212+0000","last_fullsized":"2023-11-08T19:26:26.762212+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:19:35.879794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.761883+0000","last_change":"2023-11-08T19:26:24.736978+0000","last_active":"2023-11-08T19:26:26.761883+0000","last_peered":"2023-11-08T19:26:26.761883+0000","last_clean":"2023-11-08T19:26:26.761883+0000","last_became_active":"2023-11-08T19:26:24.736651+0000","last_became_peered":"2023-11-08T19:26:24.736651+0000","last_unstale":"2023-11-08T19:26:26.761883+0000","last_undegraded":"2023-11-08T19:26:26.761883+0000","last_fullsized":"2023-11-08T19:26:26.761883+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:22:25.145083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.731536+0000","last_change":"2023-11-08T19:26:06.296634+0000","last_active":"2023-11-08T19:26:25.731536+0000","last_peered":"2023-11-08T19:26:25.731536+0000","last_clean":"2023-11-08T19:26:25.731536+0000","last_became_active":"2023-11-08T19:26:04.603088+0000","last_became_peered":"2023-11-08T19:26:04.603088+0000","last_unstale":"2023-11-08T19:26:25.731536+0000","last_undegraded":"2023-11-08T19:26:25.731536+0000","last_fullsized":"2023-11-08T19:26:25.731536+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:01:56.991346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038180899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.762568+0000","last_change":"2023-11-08T19:26:25.749860+0000","last_active":"2023-11-08T19:26:26.762568+0000","last_peered":"2023-11-08T19:26:26.762568+0000","last_clean":"2023-11-08T19:26:26.762568+0000","last_became_active":"2023-11-08T19:26:25.749612+0000","last_became_peered":"2023-11-08T19:26:25.749612+0000","last_unstale":"2023-11-08T19:26:26.762568+0000","last_undegraded":"2023-11-08T19:26:26.762568+0000","last_fullsized":"2023-11-08T19:26:26.762568+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:48:09.695836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.761578+0000","last_change":"2023-11-08T19:26:24.742441+0000","last_active":"2023-11-08T19:26:26.761578+0000","last_peered":"2023-11-08T19:26:26.761578+0000","last_clean":"2023-11-08T19:26:26.761578+0000","last_became_active":"2023-11-08T19:26:24.742069+0000","last_became_peered":"2023-11-08T19:26:24.742069+0000","last_unstale":"2023-11-08T19:26:26.761578+0000","last_undegraded":"2023-11-08T19:26:26.761578+0000","last_fullsized":"2023-11-08T19:26:26.761578+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:35:22.441191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.734004+0000","last_change":"2023-11-08T19:26:06.394300+0000","last_active":"2023-11-08T19:26:25.734004+0000","last_peered":"2023-11-08T19:26:25.734004+0000","last_clean":"2023-11-08T19:26:25.734004+0000","last_became_active":"2023-11-08T19:26:04.601685+0000","last_became_peered":"2023-11-08T19:26:04.601685+0000","last_unstale":"2023-11-08T19:26:25.734004+0000","last_undegraded":"2023-11-08T19:26:25.734004+0000","last_fullsized":"2023-11-08T19:26:25.734004+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:51:06.626474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061044099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.761844+0000","last_change":"2023-11-08T19:26:25.744017+0000","last_active":"2023-11-08T19:26:26.761844+0000","last_peered":"2023-11-08T19:26:26.761844+0000","last_clean":"2023-11-08T19:26:26.761844+0000","last_became_active":"2023-11-08T19:26:25.743747+0000","last_became_peered":"2023-11-08T19:26:25.743747+0000","last_unstale":"2023-11-08T19:26:26.761844+0000","last_undegraded":"2023-11-08T19:26:26.761844+0000","last_fullsized":"2023-11-08T19:26:26.761844+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:11:18.036694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.761455+0000","last_change":"2023-11-08T19:26:24.743608+0000","last_active":"2023-11-08T19:26:26.761455+0000","last_peered":"2023-11-08T19:26:26.761455+0000","last_clean":"2023-11-08T19:26:26.761455+0000","last_became_active":"2023-11-08T19:26:24.742838+0000","last_became_peered":"2023-11-08T19:26:24.742838+0000","last_unstale":"2023-11-08T19:26:26.761455+0000","last_undegraded":"2023-11-08T19:26:26.761455+0000","last_fullsized":"2023-11-08T19:26:26.761455+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:25:16.496965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":28,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.728055+0000","last_change":"2023-11-08T19:26:05.609570+0000","last_active":"2023-11-08T19:26:25.728055+0000","last_peered":"2023-11-08T19:26:25.728055+0000","last_clean":"2023-11-08T19:26:25.728055+0000","last_became_active":"2023-11-08T19:26:04.608379+0000","last_became_peered":"2023-11-08T19:26:04.608379+0000","last_unstale":"2023-11-08T19:26:25.728055+0000","last_undegraded":"2023-11-08T19:26:25.728055+0000","last_fullsized":"2023-11-08T19:26:25.728055+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:40:08.986649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000529657,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.762778+0000","last_change":"2023-11-08T19:26:25.740002+0000","last_active":"2023-11-08T19:26:26.762778+0000","last_peered":"2023-11-08T19:26:26.762778+0000","last_clean":"2023-11-08T19:26:26.762778+0000","last_became_active":"2023-11-08T19:26:25.739710+0000","last_became_peered":"2023-11-08T19:26:25.739710+0000","last_unstale":"2023-11-08T19:26:26.762778+0000","last_undegraded":"2023-11-08T19:26:26.762778+0000","last_fullsized":"2023-11-08T19:26:26.762778+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:38:44.411483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.762711+0000","last_change":"2023-11-08T19:26:24.742104+0000","last_active":"2023-11-08T19:26:26.762711+0000","last_peered":"2023-11-08T19:26:26.762711+0000","last_clean":"2023-11-08T19:26:26.762711+0000","last_became_active":"2023-11-08T19:26:24.741846+0000","last_became_peered":"2023-11-08T19:26:24.741846+0000","last_unstale":"2023-11-08T19:26:26.762711+0000","last_undegraded":"2023-11-08T19:26:26.762711+0000","last_fullsized":"2023-11-08T19:26:26.762711+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:18:25.803994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.733125+0000","last_change":"2023-11-08T19:26:06.265419+0000","last_active":"2023-11-08T19:26:25.733125+0000","last_peered":"2023-11-08T19:26:25.733125+0000","last_clean":"2023-11-08T19:26:25.733125+0000","last_became_active":"2023-11-08T19:26:04.605333+0000","last_became_peered":"2023-11-08T19:26:04.605333+0000","last_unstale":"2023-11-08T19:26:25.733125+0000","last_undegraded":"2023-11-08T19:26:25.733125+0000","last_fullsized":"2023-11-08T19:26:25.733125+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:01:56.991346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064606400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.729672+0000","last_change":"2023-11-08T19:26:25.745842+0000","last_active":"2023-11-08T19:26:26.729672+0000","last_peered":"2023-11-08T19:26:26.729672+0000","last_clean":"2023-11-08T19:26:26.729672+0000","last_became_active":"2023-11-08T19:26:25.745304+0000","last_became_peered":"2023-11-08T19:26:25.745304+0000","last_unstale":"2023-11-08T19:26:26.729672+0000","last_undegraded":"2023-11-08T19:26:26.729672+0000","last_fullsized":"2023-11-08T19:26:26.729672+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:40:30.126348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.762726+0000","last_change":"2023-11-08T19:26:24.740671+0000","last_active":"2023-11-08T19:26:26.762726+0000","last_peered":"2023-11-08T19:26:26.762726+0000","last_clean":"2023-11-08T19:26:26.762726+0000","last_became_active":"2023-11-08T19:26:24.740395+0000","last_became_peered":"2023-11-08T19:26:24.740395+0000","last_unstale":"2023-11-08T19:26:26.762726+0000","last_undegraded":"2023-11-08T19:26:26.762726+0000","last_fullsized":"2023-11-08T19:26:26.762726+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:09:24.710916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.739334+0000","last_change":"2023-11-08T19:26:06.542044+0000","last_active":"2023-11-08T19:26:25.739334+0000","last_peered":"2023-11-08T19:26:25.739334+0000","last_clean":"2023-11-08T19:26:25.739334+0000","last_became_active":"2023-11-08T19:26:04.603056+0000","last_became_peered":"2023-11-08T19:26:04.603056+0000","last_unstale":"2023-11-08T19:26:25.739334+0000","last_undegraded":"2023-11-08T19:26:25.739334+0000","last_fullsized":"2023-11-08T19:26:25.739334+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:41:50.771457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000517414,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":193,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.731732+0000","last_change":"2023-11-08T19:24:45.888122+0000","last_active":"2023-11-08T19:26:25.731732+0000","last_peered":"2023-11-08T19:26:25.731732+0000","last_clean":"2023-11-08T19:26:25.731732+0000","last_became_active":"2023-11-08T19:24:45.887048+0000","last_became_peered":"2023-11-08T19:24:45.887048+0000","last_unstale":"2023-11-08T19:26:25.731732+0000","last_undegraded":"2023-11-08T19:26:25.731732+0000","last_fullsized":"2023-11-08T19:26:25.731732+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:44.575133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:44.575133+0000","last_clean_scrub_stamp":"2023-11-08T19:24:44.575133+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:19:20.202062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574873,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28160,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343680,"statfs":{"total":96636764160,"available":96607928320,"internally_reserved":0,"allocated":1167360,"data_stored":829790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.61699999999999999}]},{"osd":1,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.63},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.64500000000000002}]},{"osd":2,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.68300000000000005}]},{"osd":3,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.502},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.73499999999999999}]},{"osd":4,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.44}]},{"osd":5,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.67000000000000004}]},{"osd":6,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.59899999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574873,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27588,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344252,"statfs":{"total":96636764160,"available":96608514048,"internally_reserved":0,"allocated":573440,"data_stored":239950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":21.359999999999999,"5min":21.359999999999999,"15min":21.359999999999999},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":10.329000000000001,"5min":10.329000000000001,"15min":10.329000000000001},"last":0.42799999999999999}]},{"osd":1,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":3.4239999999999999,"5min":3.4239999999999999,"15min":3.4239999999999999},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":3.4809999999999999,"5min":3.4809999999999999,"15min":3.4809999999999999},"last":0.45200000000000001}]},{"osd":2,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":10.398,"5min":10.398,"15min":10.398},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":5.0170000000000003,"5min":5.0170000000000003,"15min":5.0170000000000003},"last":0.374}]},{"osd":3,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":3.5710000000000002,"5min":3.5710000000000002,"15min":3.5710000000000002},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":10.368,"5min":10.368,"15min":10.368},"last":0.57099999999999995}]},{"osd":4,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.39200000000000002}]},{"osd":5,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.441},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.59099999999999997}]},{"osd":7,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.498}]}]},{"osd":1,"up_from":13,"seq":55834574873,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27588,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344252,"statfs":{"total":96636764160,"available":96608514048,"internally_reserved":0,"allocated":573440,"data_stored":239458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":1.389,"5min":1.389,"15min":1.389},"last":0.26800000000000002},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":9.1370000000000005,"5min":9.1370000000000005,"15min":9.1370000000000005},"last":0.32900000000000001}]},{"osd":2,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.28499999999999998},{"interface":"front","average":{"1min":0.433,"5min":0.433,"15min":0.433},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":1.425,"5min":1.425,"15min":1.425},"last":0.38700000000000001}]},{"osd":3,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":11.957000000000001,"5min":11.957000000000001,"15min":11.957000000000001},"last":0.30299999999999999},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":18.039000000000001,"5min":18.039000000000001,"15min":18.039000000000001},"last":0.36699999999999999}]},{"osd":4,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":6.181,"5min":6.181,"15min":6.181},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.488,"5min":1.488,"15min":1.488},"last":0.61299999999999999}]},{"osd":5,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":9.2420000000000009,"5min":9.2420000000000009,"15min":9.2420000000000009},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":6.2110000000000003,"5min":6.2110000000000003,"15min":6.2110000000000003},"last":0.63400000000000001}]},{"osd":6,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":9.1880000000000006,"5min":9.1880000000000006,"15min":9.1880000000000006},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.462,"5min":1.462,"15min":1.462},"last":0.65000000000000002}]},{"osd":7,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":6.1479999999999997,"5min":6.1479999999999997,"15min":6.1479999999999997},"last":0.54600000000000004},{"interface":"front","average":{"1min":1.028,"5min":1.028,"15min":1.028},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":9.2170000000000005,"5min":9.2170000000000005,"15min":9.2170000000000005},"last":0.58299999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574873,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28160,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343680,"statfs":{"total":96636764160,"available":96607928320,"internally_reserved":0,"allocated":1167360,"data_stored":829722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":4.1200000000000001,"5min":4.1200000000000001,"15min":4.1200000000000001},"last":0.39800000000000002},{"interface":"front","average":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":16.940999999999999,"5min":16.940999999999999,"15min":16.940999999999999},"last":0.439}]},{"osd":2,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":7.0209999999999999,"5min":7.0209999999999999,"15min":7.0209999999999999},"last":0.41099999999999998}]},{"osd":3,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":4.0940000000000003,"5min":4.0940000000000003,"15min":4.0940000000000003},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.379}]},{"osd":4,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":4.0049999999999999,"5min":4.0049999999999999,"15min":4.0049999999999999},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.49199999999999999}]},{"osd":5,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":4.3090000000000002,"5min":4.3090000000000002,"15min":4.3090000000000002},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":4.0709999999999997,"5min":4.0709999999999997,"15min":4.0709999999999997},"last":0.505}]},{"osd":6,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":4.2560000000000002,"5min":4.2560000000000002,"15min":4.2560000000000002},"last":0.60499999999999998}]},{"osd":7,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":4.2850000000000001,"5min":4.2850000000000001,"15min":4.2850000000000001},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":4.0460000000000003,"5min":4.0460000000000003,"15min":4.0460000000000003},"last":0.47199999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574873,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27592,"kb_used_data":564,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344248,"statfs":{"total":96636764160,"available":96608509952,"internally_reserved":0,"allocated":577536,"data_stored":240910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"min":{"1min":0.20100000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"last":0.27700000000000002},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":6.0999999999999996,"5min":6.0999999999999996,"15min":6.0999999999999996},"last":0.29299999999999998}]},{"osd":1,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"last":0.23999999999999999},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.624,"5min":0.624,"15min":0.624},"last":0.34000000000000002}]},{"osd":3,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"last":0.31},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.32100000000000001}]},{"osd":4,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.40999999999999998}]},{"osd":5,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.51800000000000002}]},{"osd":6,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.40300000000000002}]},{"osd":7,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.38400000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574874,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27596,"kb_used_data":568,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344244,"statfs":{"total":96636764160,"available":96608505856,"internally_reserved":0,"allocated":581632,"data_stored":240932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:25:46 2023","interfaces":[{"interface":"back","average":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.626,"5min":0.626,"15min":0.626},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.36399999999999999}]},{"osd":1,"last update":"Wed Nov 8 19:25:46 2023","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.39400000000000002}]},{"osd":2,"last update":"Wed Nov 8 19:25:46 2023","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.37,"15min":0.37},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"last":0.50700000000000001}]},{"osd":4,"last update":"Wed Nov 8 19:25:46 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.79000000000000004}]},{"osd":5,"last update":"Wed Nov 8 19:25:46 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.81999999999999995}]},{"osd":6,"last update":"Wed Nov 8 19:25:46 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.66600000000000004}]},{"osd":7,"last update":"Wed Nov 8 19:25:46 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.65100000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574873,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27584,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344256,"statfs":{"total":96636764160,"available":96608518144,"internally_reserved":0,"allocated":569344,"data_stored":239880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:25:47 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":6.5970000000000004,"5min":6.5970000000000004,"15min":6.5970000000000004},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":2.5030000000000001,"5min":2.5030000000000001,"15min":2.5030000000000001},"last":0.56000000000000005}]},{"osd":1,"last update":"Wed Nov 8 19:25:47 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.66300000000000003}]},{"osd":2,"last update":"Wed Nov 8 19:25:47 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":6.6920000000000002,"5min":6.6920000000000002,"15min":6.6920000000000002},"last":0.53900000000000003}]},{"osd":3,"last update":"Wed Nov 8 19:25:47 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.68200000000000005}]},{"osd":5,"last update":"Wed Nov 8 19:25:47 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.46400000000000002}]},{"osd":6,"last update":"Wed Nov 8 19:25:47 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.63}]},{"osd":7,"last update":"Wed Nov 8 19:25:47 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.48999999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574873,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27580,"kb_used_data":552,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344260,"statfs":{"total":96636764160,"available":96608522240,"internally_reserved":0,"allocated":565248,"data_stored":239298,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.56599999999999995}]},{"osd":1,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.57999999999999996}]},{"osd":2,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.53600000000000003}]},{"osd":3,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.59499999999999997}]},{"osd":4,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.61199999999999999}]},{"osd":6,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.50700000000000001}]},{"osd":7,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.374},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.48199999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-08T19:26:30.620 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-11-08T19:26:30.770 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.769+0000 7f3fc4290700 1 Processor -- start 2023-11-08T19:26:30.771 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.769+0000 7f3fc4290700 1 -- start start 2023-11-08T19:26:30.773 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.770+0000 7f3fc4290700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f3fbc15dd90 0x7f3fbc0d0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:30.773 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.770+0000 7f3fc4290700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fbc0b4710 con 0x7f3fbc06c760 2023-11-08T19:26:30.773 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.770+0000 7f3fc4290700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fbc0cbf10 con 0x7f3fbc0d0c40 2023-11-08T19:26:30.773 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.770+0000 7f3fc4290700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fbc0cc1f0 con 0x7f3fbc15dd90 2023-11-08T19:26:30.774 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.770+0000 7f3fc202c700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f3fbc15dd90 0x7f3fbc0d0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:30.774 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.770+0000 7f3fc282d700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f3fbc06c760 0x7f3fbc0cdfc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:47840/0 (socket says 172.21.15.143:47840) 2023-11-08T19:26:30.774 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.770+0000 7f3fc282d700 1 -- 172.21.15.143:0/4271864114 learned_addr learned my addr 172.21.15.143:0/4271864114 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:26:30.774 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.770+0000 7f3fc202c700 1 -- 172.21.15.143:0/4271864114 >> v1:172.21.15.174:6789/0 conn(0x7f3fbc0d0c40 legacy=0x7f3fbc06c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:30.774 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.770+0000 7f3fc202c700 1 -- 172.21.15.143:0/4271864114 >> v1:172.21.15.143:6789/0 conn(0x7f3fbc06c760 legacy=0x7f3fbc0cdfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:30.775 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.770+0000 7f3fc202c700 1 -- 172.21.15.143:0/4271864114 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fbc0cc4d0 con 0x7f3fbc15dd90 2023-11-08T19:26:30.775 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.770+0000 7f3fc202c700 1 --2- 172.21.15.143:0/4271864114 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3fbc15dd90 0x7f3fbc0d0700 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f3fb8009840 tx=0x7f3fb8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18409a5a11a9068b server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:30.775 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.771+0000 7f3fc102a700 1 -- 172.21.15.143:0/4271864114 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fb8011830 con 0x7f3fbc15dd90 2023-11-08T19:26:30.775 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.771+0000 7f3fc102a700 1 -- 172.21.15.143:0/4271864114 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fb8011990 con 0x7f3fbc15dd90 2023-11-08T19:26:30.775 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.771+0000 7f3fc102a700 1 -- 172.21.15.143:0/4271864114 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fb8011d20 con 0x7f3fbc15dd90 2023-11-08T19:26:30.776 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.771+0000 7f3fc4290700 1 -- 172.21.15.143:0/4271864114 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3fbc15dd90 msgr2=0x7f3fbc0d0700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:30.776 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.771+0000 7f3fc4290700 1 --2- 172.21.15.143:0/4271864114 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3fbc15dd90 0x7f3fbc0d0700 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f3fb8009840 tx=0x7f3fb8009b50 comp rx=0 tx=0).stop 2023-11-08T19:26:30.776 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.771+0000 7f3fc4290700 1 -- 172.21.15.143:0/4271864114 shutdown_connections 2023-11-08T19:26:30.776 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.771+0000 7f3fc4290700 1 --2- 172.21.15.143:0/4271864114 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3fbc15dd90 0x7f3fbc0d0700 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:30.776 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.771+0000 7f3fc4290700 1 -- 172.21.15.143:0/4271864114 >> 172.21.15.143:0/4271864114 conn(0x7f3fbc0d18c0 msgr2=0x7f3fbc0d1cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:30.776 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.771+0000 7f3fc4290700 1 -- 172.21.15.143:0/4271864114 shutdown_connections 2023-11-08T19:26:30.777 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.771+0000 7f3fc4290700 1 -- 172.21.15.143:0/4271864114 wait complete. 2023-11-08T19:26:30.779 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.773+0000 7f3fc4290700 1 Processor -- start 2023-11-08T19:26:30.779 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.773+0000 7f3fc4290700 1 -- start start 2023-11-08T19:26:30.780 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.773+0000 7f3fc4290700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3fbc06c760 0x7f3fbc150c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:30.780 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.773+0000 7f3fc4290700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3fbc0d0c40 0x7f3fbc15e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:30.780 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.773+0000 7f3fc4290700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3fbc15dd90 0x7f3fbc15fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:30.780 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.773+0000 7f3fc4290700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fbc0b4c50 con 0x7f3fbc15dd90 2023-11-08T19:26:30.780 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.773+0000 7f3fc4290700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fbc0b4ef0 con 0x7f3fbc06c760 2023-11-08T19:26:30.781 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.773+0000 7f3fc4290700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fbc0b51d0 con 0x7f3fbc0d0c40 2023-11-08T19:26:30.781 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.773+0000 7f3fc282d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3fbc15dd90 0x7f3fbc15fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:30.781 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.774+0000 7f3fc282d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3fbc15dd90 0x7f3fbc15fe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51692/0 (socket says 172.21.15.143:51692) 2023-11-08T19:26:30.781 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.774+0000 7f3fc282d700 1 -- 172.21.15.143:0/306413703 learned_addr learned my addr 172.21.15.143:0/306413703 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:30.781 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.774+0000 7f3fc202c700 1 --2- 172.21.15.143:0/306413703 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3fbc06c760 0x7f3fbc150c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:30.782 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.774+0000 7f3fc182b700 1 --2- 172.21.15.143:0/306413703 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3fbc0d0c40 0x7f3fbc15e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:30.782 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.774+0000 7f3fc282d700 1 -- 172.21.15.143:0/306413703 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3fbc0d0c40 msgr2=0x7f3fbc15e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:30.782 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.774+0000 7f3fc282d700 1 --2- 172.21.15.143:0/306413703 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3fbc0d0c40 0x7f3fbc15e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:30.782 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.774+0000 7f3fc282d700 1 -- 172.21.15.143:0/306413703 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3fbc06c760 msgr2=0x7f3fbc150c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:30.782 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.774+0000 7f3fc282d700 1 --2- 172.21.15.143:0/306413703 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3fbc06c760 0x7f3fbc150c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:30.783 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.774+0000 7f3fc282d700 1 -- 172.21.15.143:0/306413703 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fb80094a0 con 0x7f3fbc15dd90 2023-11-08T19:26:30.783 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.774+0000 7f3fc282d700 1 --2- 172.21.15.143:0/306413703 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3fbc15dd90 0x7f3fbc15fe50 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f3fb400cbc0 tx=0x7f3fb400ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:30.783 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.775+0000 7f3fb2ffd700 1 -- 172.21.15.143:0/306413703 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fb400a9e0 con 0x7f3fbc15dd90 2023-11-08T19:26:30.783 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.775+0000 7f3fb2ffd700 1 -- 172.21.15.143:0/306413703 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fb400ab40 con 0x7f3fbc15dd90 2023-11-08T19:26:30.783 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.775+0000 7f3fc4290700 1 -- 172.21.15.143:0/306413703 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fbc0b54b0 con 0x7f3fbc15dd90 2023-11-08T19:26:30.784 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.775+0000 7f3fb2ffd700 1 -- 172.21.15.143:0/306413703 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fb4012400 con 0x7f3fbc15dd90 2023-11-08T19:26:30.784 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.775+0000 7f3fc4290700 1 -- 172.21.15.143:0/306413703 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fbc0b5a70 con 0x7f3fbc15dd90 2023-11-08T19:26:30.784 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.776+0000 7f3fc4290700 1 -- 172.21.15.143:0/306413703 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fbc0b5c90 con 0x7f3fbc15dd90 2023-11-08T19:26:30.784 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.776+0000 7f3fb2ffd700 1 -- 172.21.15.143:0/306413703 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3fb401b3b0 con 0x7f3fbc15dd90 2023-11-08T19:26:30.784 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.777+0000 7f3fb2ffd700 1 --2- 172.21.15.143:0/306413703 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f3fa006af00 0x7f3fa006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:30.784 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.777+0000 7f3fb2ffd700 1 -- 172.21.15.143:0/306413703 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3fb4089220 con 0x7f3fbc15dd90 2023-11-08T19:26:30.785 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.777+0000 7f3fc202c700 1 --2- 172.21.15.143:0/306413703 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f3fa006af00 0x7f3fa006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:30.785 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.777+0000 7f3fc202c700 1 --2- 172.21.15.143:0/306413703 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f3fa006af00 0x7f3fa006d3c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f3fb80090b0 tx=0x7f3fb801c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:30.785 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:30.781+0000 7f3fb2ffd700 1 -- 172.21.15.143:0/306413703 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3fb4054180 con 0x7f3fbc15dd90 2023-11-08T19:26:31.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:31.047+0000 7f3fc4290700 1 -- 172.21.15.143:0/306413703 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f3fbc04eee0 con 0x7f3fbc15dd90 2023-11-08T19:26:31.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:31.048+0000 7f3fb2ffd700 1 -- 172.21.15.143:0/306413703 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1706 (secure 0 0 0) 0x7f3fb40577a0 con 0x7f3fbc15dd90 2023-11-08T19:26:31.049 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:26:31.050 INFO:teuthology.orchestra.run.smithi143.stdout:{"fsid":"968483af-77fc-4068-ad7c-5c2a1a4613d7","health":{"status":"HEALTH_WARN","checks":{"POOL_APP_NOT_ENABLED":{"severity":"HEALTH_WARN","summary":{"message":"1 pool(s) do not have an application enabled","count":1},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":118,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":48,"num_osds":8,"num_up_osds":8,"osd_up_since":1699471483,"num_in_osds":8,"osd_in_since":1699471475,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":129}],"num_pgs":129,"num_pools":4,"num_objects":23,"data_bytes":592778,"bytes_used":227172352,"bytes_avail":772866940928,"bytes_total":773094113280,"write_bytes_sec":2388,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":25,"id":3,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":3,"rank":0,"name":"c","status":"up:active","gid":5242}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":4,"modified":"2023-11-08T19:25:46.523402+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-11-08T19:26:31.051 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:31.050+0000 7f3fc4290700 1 -- 172.21.15.143:0/306413703 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f3fa006af00 msgr2=0x7f3fa006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:31.051 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:31.050+0000 7f3fc4290700 1 --2- 172.21.15.143:0/306413703 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f3fa006af00 0x7f3fa006d3c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f3fb80090b0 tx=0x7f3fb801c040 comp rx=0 tx=0).stop 2023-11-08T19:26:31.052 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:31.050+0000 7f3fc4290700 1 -- 172.21.15.143:0/306413703 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3fbc15dd90 msgr2=0x7f3fbc15fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:31.052 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:31.050+0000 7f3fc4290700 1 --2- 172.21.15.143:0/306413703 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3fbc15dd90 0x7f3fbc15fe50 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f3fb400cbc0 tx=0x7f3fb400ced0 comp rx=0 tx=0).stop 2023-11-08T19:26:31.052 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:31.050+0000 7f3fc4290700 1 -- 172.21.15.143:0/306413703 shutdown_connections 2023-11-08T19:26:31.052 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:31.050+0000 7f3fc4290700 1 --2- 172.21.15.143:0/306413703 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f3fa006af00 0x7f3fa006d3c0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:31.052 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:31.050+0000 7f3fc4290700 1 --2- 172.21.15.143:0/306413703 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3fbc15dd90 0x7f3fbc15fe50 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:31.053 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:31.050+0000 7f3fc4290700 1 --2- 172.21.15.143:0/306413703 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3fbc06c760 0x7f3fbc150c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:31.053 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:31.050+0000 7f3fc4290700 1 --2- 172.21.15.143:0/306413703 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3fbc0d0c40 0x7f3fbc15e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:31.053 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:31.050+0000 7f3fc4290700 1 -- 172.21.15.143:0/306413703 >> 172.21.15.143:0/306413703 conn(0x7f3fbc0d18c0 msgr2=0x7f3fbc152eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:31.053 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:31.050+0000 7f3fc4290700 1 -- 172.21.15.143:0/306413703 shutdown_connections 2023-11-08T19:26:31.053 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:31.050+0000 7f3fc4290700 1 -- 172.21.15.143:0/306413703 wait complete. 2023-11-08T19:26:31.063 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-11-08T19:26:31.064 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-08T19:26:31.183 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:31.182+0000 7f0326998700 1 Processor -- start 2023-11-08T19:26:31.184 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:31.182+0000 7f0326998700 1 -- start start 2023-11-08T19:26:31.187 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:31.183+0000 7f0326998700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f03200b2920 0x7f03200cf510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:31.187 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:31.183+0000 7f0326998700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f03200cbe20 0x7f03200cc250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:31.188 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:31.183+0000 7f0326998700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03200cf080 con 0x7f032015aa30 2023-11-08T19:26:31.188 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:31.183+0000 7f0326998700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03200ce470 con 0x7f03200cbe20 2023-11-08T19:26:31.188 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:31.183+0000 7f0326998700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03200ce750 con 0x7f03200b2920 2023-11-08T19:26:31.188 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:31.183+0000 7f031ffff700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f032015aa30 0x7f03200b2210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:47850/0 (socket says 172.21.15.143:47850) 2023-11-08T19:26:31.189 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:31.183+0000 7f031ffff700 1 -- 172.21.15.143:0/490641212 learned_addr learned my addr 172.21.15.143:0/490641212 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:26:31.189 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:31.183+0000 7f031effd700 1 -- 172.21.15.143:0/490641212 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2311762743 0 0) 0x7f03200cf080 con 0x7f032015aa30 2023-11-08T19:26:31.189 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:31.183+0000 7f031f7fe700 1 --2- 172.21.15.143:0/490641212 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f03200b2920 0x7f03200cf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:31.189 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:31.183+0000 7f031effd700 1 -- 172.21.15.143:0/490641212 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f03080035b0 con 0x7f032015aa30 2023-11-08T19:26:31.189 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:31.183+0000 7f0324f35700 1 --2- 172.21.15.143:0/490641212 >> v2:172.21.15.174:3300/0 conn(0x7f03200cbe20 0x7f03200cc250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:31.190 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:31.183+0000 7f031effd700 1 -- 172.21.15.143:0/490641212 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2307005335 0 0) 0x7f03080035b0 con 0x7f032015aa30 2023-11-08T19:26:31.190 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:31.183+0000 7f031effd700 1 -- 172.21.15.143:0/490641212 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f03200b2920 msgr2=0x7f03200cf510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:31.190 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:31.183+0000 7f031effd700 1 --2- 172.21.15.143:0/490641212 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f03200b2920 0x7f03200cf510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:31.190 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:31.183+0000 7f031effd700 1 -- 172.21.15.143:0/490641212 >> v2:172.21.15.174:3300/0 conn(0x7f03200cbe20 msgr2=0x7f03200cc250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:31.190 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:31.183+0000 7f031effd700 1 --2- 172.21.15.143:0/490641212 >> v2:172.21.15.174:3300/0 conn(0x7f03200cbe20 0x7f03200cc250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:31.191 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:31.183+0000 7f031effd700 1 -- 172.21.15.143:0/490641212 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03200cea30 con 0x7f032015aa30 2023-11-08T19:26:31.191 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:31.183+0000 7f031effd700 1 -- 172.21.15.143:0/490641212 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4147756303 0 0) 0x7f03100017f0 con 0x7f032015aa30 2023-11-08T19:26:31.192 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:31.183+0000 7f0324f35700 1 --2- 172.21.15.143:0/490641212 >> v2:172.21.15.174:3300/0 conn(0x7f03200cbe20 0x7f03200cc250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:26:31.192 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:31.183+0000 7f031effd700 1 -- 172.21.15.143:0/490641212 >> v1:172.21.15.143:6789/0 conn(0x7f032015aa30 legacy=0x7f03200b2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:31.192 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:31.183+0000 7f031effd700 1 --2- 172.21.15.143:0/490641212 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f03200b2920 0x7f0308003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:31.192 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:31.183+0000 7f031effd700 1 --2- 172.21.15.143:0/490641212 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0308003f80 0x7f0308004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:31.193 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:31.183+0000 7f031effd700 1 --2- 172.21.15.143:0/490641212 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0308004d90 0x7f0308005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:31.193 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:31.183+0000 7f031effd700 1 -- 172.21.15.143:0/490641212 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03200ce470 con 0x7f0308004d90 2023-11-08T19:26:31.193 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:31.183+0000 7f031effd700 1 -- 172.21.15.143:0/490641212 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03200ce750 con 0x7f0308003f80 2023-11-08T19:26:31.193 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:31.183+0000 7f031effd700 1 -- 172.21.15.143:0/490641212 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f030800ba60 con 0x7f03200b2920 2023-11-08T19:26:31.193 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:31.183+0000 7f0324f35700 1 --2- 172.21.15.143:0/490641212 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0308004d90 0x7f0308005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:31.193 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:31.183+0000 7f031ffff700 1 --2- 172.21.15.143:0/490641212 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f03200b2920 0x7f0308003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:31.194 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:31.183+0000 7f031f7fe700 1 --2- 172.21.15.143:0/490641212 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0308003f80 0x7f0308004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:31.194 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:31.184+0000 7f0324f35700 1 -- 172.21.15.143:0/490641212 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f03200b2920 msgr2=0x7f0308003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:31.194 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:31.184+0000 7f0324f35700 1 --2- 172.21.15.143:0/490641212 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f03200b2920 0x7f0308003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:31.194 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:31.184+0000 7f0324f35700 1 -- 172.21.15.143:0/490641212 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0308003f80 msgr2=0x7f0308004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:31.194 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:31.184+0000 7f0324f35700 1 --2- 172.21.15.143:0/490641212 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0308003f80 0x7f0308004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:31.195 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:31.184+0000 7f0324f35700 1 -- 172.21.15.143:0/490641212 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03080035b0 con 0x7f0308004d90 2023-11-08T19:26:31.195 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:31.184+0000 7f0324f35700 1 --2- 172.21.15.143:0/490641212 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0308004d90 0x7f0308005210 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f03180073e0 tx=0x7f0308005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2900a9d7311aca9 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:31.195 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:31.184+0000 7f031effd700 1 -- 172.21.15.143:0/490641212 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0318014400 con 0x7f0308004d90 2023-11-08T19:26:31.195 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:31.184+0000 7f031effd700 1 -- 172.21.15.143:0/490641212 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0318014560 con 0x7f0308004d90 2023-11-08T19:26:31.195 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:31.184+0000 7f031effd700 1 -- 172.21.15.143:0/490641212 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03180148d0 con 0x7f0308004d90 2023-11-08T19:26:31.196 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:31.184+0000 7f0326998700 1 -- 172.21.15.143:0/490641212 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0308004d90 msgr2=0x7f0308005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:31.196 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:31.184+0000 7f0326998700 1 --2- 172.21.15.143:0/490641212 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0308004d90 0x7f0308005210 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f03180073e0 tx=0x7f0308005880 comp rx=0 tx=0).stop 2023-11-08T19:26:31.196 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:31.184+0000 7f0324f35700 1 -- 172.21.15.143:0/490641212 reap_dead start 2023-11-08T19:26:31.196 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:31.184+0000 7f0326998700 1 -- 172.21.15.143:0/490641212 shutdown_connections 2023-11-08T19:26:31.196 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:31.184+0000 7f0326998700 1 -- 172.21.15.143:0/490641212 >> 172.21.15.143:0/490641212 conn(0x7f0320099ab0 msgr2=0x7f03200d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:31.197 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:31.185+0000 7f0326998700 1 -- 172.21.15.143:0/490641212 shutdown_connections 2023-11-08T19:26:31.197 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:31.185+0000 7f0326998700 1 -- 172.21.15.143:0/490641212 wait complete. 2023-11-08T19:26:31.197 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:31.186+0000 7f0326998700 1 Processor -- start 2023-11-08T19:26:31.197 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:31.186+0000 7f0326998700 1 -- start start 2023-11-08T19:26:31.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:31.187+0000 7f0326998700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f03200c6b20 0x7f03201f42a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:31.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:31.187+0000 7f0326998700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f03200b5360 0x7f03200b33b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:31.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:31.187+0000 7f0326998700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f03200b38f0 0x7f03201467f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:31.201 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:31.187+0000 7f0326998700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03200b3d70 con 0x7f03200b38f0 2023-11-08T19:26:31.201 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:31.187+0000 7f0326998700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03200b64c0 con 0x7f03200b5360 2023-11-08T19:26:31.201 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:31.187+0000 7f0326998700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03200b58b0 con 0x7f03200c6b20 2023-11-08T19:26:31.201 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:31.187+0000 7f0324f35700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f03200b38f0 0x7f03201467f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:31.201 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:31.187+0000 7f0324f35700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f03200b38f0 0x7f03201467f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51712/0 (socket says 172.21.15.143:51712) 2023-11-08T19:26:31.202 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:31.187+0000 7f0324f35700 1 -- 172.21.15.143:0/1115335107 learned_addr learned my addr 172.21.15.143:0/1115335107 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:31.202 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:31.187+0000 7f031f7fe700 1 --2- 172.21.15.143:0/1115335107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f03200b5360 0x7f03200b33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:31.202 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:31.187+0000 7f031ffff700 1 --2- 172.21.15.143:0/1115335107 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f03200c6b20 0x7f03201f42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:31.202 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:31.187+0000 7f0324f35700 1 -- 172.21.15.143:0/1115335107 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f03200c6b20 msgr2=0x7f03201f42a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:31.202 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:31.187+0000 7f0324f35700 1 --2- 172.21.15.143:0/1115335107 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f03200c6b20 0x7f03201f42a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:31.203 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:31.187+0000 7f0324f35700 1 -- 172.21.15.143:0/1115335107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f03200b5360 msgr2=0x7f03200b33b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:31.203 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:31.187+0000 7f0324f35700 1 --2- 172.21.15.143:0/1115335107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f03200b5360 0x7f03200b33b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:31.203 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:31.187+0000 7f0324f35700 1 -- 172.21.15.143:0/1115335107 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03200b5b90 con 0x7f03200b38f0 2023-11-08T19:26:31.203 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:31.187+0000 7f031ffff700 1 --2- 172.21.15.143:0/1115335107 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f03200c6b20 0x7f03201f42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:26:31.203 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:31.187+0000 7f0324f35700 1 --2- 172.21.15.143:0/1115335107 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f03200b38f0 0x7f03201467f0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f03080047a0 tx=0x7f0318004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:31.204 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:31.187+0000 7f031cff9700 1 -- 172.21.15.143:0/1115335107 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0318014400 con 0x7f03200b38f0 2023-11-08T19:26:31.204 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:31.187+0000 7f031cff9700 1 -- 172.21.15.143:0/1115335107 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0318003e10 con 0x7f03200b38f0 2023-11-08T19:26:31.204 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:31.187+0000 7f0326998700 1 -- 172.21.15.143:0/1115335107 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03200b5e70 con 0x7f03200b38f0 2023-11-08T19:26:31.204 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:31.187+0000 7f031cff9700 1 -- 172.21.15.143:0/1115335107 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0318007000 con 0x7f03200b38f0 2023-11-08T19:26:31.204 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:31.187+0000 7f0326998700 1 -- 172.21.15.143:0/1115335107 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03201f47e0 con 0x7f03200b38f0 2023-11-08T19:26:31.205 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:31.188+0000 7f0326998700 1 -- 172.21.15.143:0/1115335107 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0320067c00 con 0x7f03200b38f0 2023-11-08T19:26:31.205 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:31.189+0000 7f031cff9700 1 -- 172.21.15.143:0/1115335107 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f031801b050 con 0x7f03200b38f0 2023-11-08T19:26:31.205 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:31.189+0000 7f031cff9700 1 --2- 172.21.15.143:0/1115335107 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f030c06af00 0x7f030c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:31.205 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:31.189+0000 7f031cff9700 1 -- 172.21.15.143:0/1115335107 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f031808a7b0 con 0x7f03200b38f0 2023-11-08T19:26:31.205 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:31.190+0000 7f031ffff700 1 --2- 172.21.15.143:0/1115335107 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f030c06af00 0x7f030c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:31.206 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:31.190+0000 7f031ffff700 1 --2- 172.21.15.143:0/1115335107 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f030c06af00 0x7f030c06d3c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f0310007fd0 tx=0x7f0310009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:31.206 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:31.194+0000 7f031cff9700 1 -- 172.21.15.143:0/1115335107 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0318055850 con 0x7f03200b38f0 2023-11-08T19:26:31.381 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:31.379+0000 7f0326998700 1 -- 172.21.15.143:0/1115335107 --> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f03200cc790 con 0x7f030c06af00 2023-11-08T19:26:31.391 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:31.390+0000 7f031cff9700 1 -- 172.21.15.143:0/1115335107 <== mgr.4099 v2:172.21.15.174:6832/4037280178 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+370727 (secure 0 0 0) 0x7f0310007050 con 0x7f030c06af00 2023-11-08T19:26:31.396 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:26:31.399 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:31.394+0000 7f0326998700 1 -- 172.21.15.143:0/1115335107 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f030c06af00 msgr2=0x7f030c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:31.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:31.394+0000 7f0326998700 1 --2- 172.21.15.143:0/1115335107 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f030c06af00 0x7f030c06d3c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f0310007fd0 tx=0x7f0310009040 comp rx=0 tx=0).stop 2023-11-08T19:26:31.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:31.394+0000 7f0326998700 1 -- 172.21.15.143:0/1115335107 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f03200b38f0 msgr2=0x7f03201467f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:31.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:31.394+0000 7f0326998700 1 --2- 172.21.15.143:0/1115335107 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f03200b38f0 0x7f03201467f0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f03080047a0 tx=0x7f0318004b40 comp rx=0 tx=0).stop 2023-11-08T19:26:31.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:31.394+0000 7f0326998700 1 -- 172.21.15.143:0/1115335107 shutdown_connections 2023-11-08T19:26:31.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:31.394+0000 7f0326998700 1 --2- 172.21.15.143:0/1115335107 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f030c06af00 0x7f030c06d3c0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:31.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:31.394+0000 7f0326998700 1 --2- 172.21.15.143:0/1115335107 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f03200c6b20 0x7f03201f42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:31.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:31.394+0000 7f0326998700 1 --2- 172.21.15.143:0/1115335107 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f03200b38f0 0x7f03201467f0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:31.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:31.394+0000 7f0326998700 1 --2- 172.21.15.143:0/1115335107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f03200b5360 0x7f03200b33b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:31.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:31.394+0000 7f0326998700 1 -- 172.21.15.143:0/1115335107 >> 172.21.15.143:0/1115335107 conn(0x7f0320099ab0 msgr2=0x7f03200af5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:31.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:31.394+0000 7f0326998700 1 -- 172.21.15.143:0/1115335107 shutdown_connections 2023-11-08T19:26:31.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:31.394+0000 7f0326998700 1 -- 172.21.15.143:0/1115335107 wait complete. 2023-11-08T19:26:31.402 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2023-11-08T19:26:31.407 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":94,"stamp":"2023-11-08T19:26:30.534447+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":258,"acting":258,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":227,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":221848,"kb_used_data":5640,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754752872,"statfs":{"total":773094113280,"available":772866940928,"internally_reserved":0,"allocated":5775360,"data_stored":3099940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001350"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.731024+0000","last_change":"2023-11-08T19:26:25.752333+0000","last_active":"2023-11-08T19:26:26.731024+0000","last_peered":"2023-11-08T19:26:26.731024+0000","last_clean":"2023-11-08T19:26:26.731024+0000","last_became_active":"2023-11-08T19:26:25.752055+0000","last_became_peered":"2023-11-08T19:26:25.752055+0000","last_unstale":"2023-11-08T19:26:26.731024+0000","last_undegraded":"2023-11-08T19:26:26.731024+0000","last_fullsized":"2023-11-08T19:26:26.731024+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:29:44.685878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.730541+0000","last_change":"2023-11-08T19:26:25.746072+0000","last_active":"2023-11-08T19:26:26.730541+0000","last_peered":"2023-11-08T19:26:26.730541+0000","last_clean":"2023-11-08T19:26:26.730541+0000","last_became_active":"2023-11-08T19:26:25.745753+0000","last_became_peered":"2023-11-08T19:26:25.745753+0000","last_unstale":"2023-11-08T19:26:26.730541+0000","last_undegraded":"2023-11-08T19:26:26.730541+0000","last_fullsized":"2023-11-08T19:26:26.730541+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:02:15.103886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.762914+0000","last_change":"2023-11-08T19:26:25.746891+0000","last_active":"2023-11-08T19:26:26.762914+0000","last_peered":"2023-11-08T19:26:26.762914+0000","last_clean":"2023-11-08T19:26:26.762914+0000","last_became_active":"2023-11-08T19:26:25.746609+0000","last_became_peered":"2023-11-08T19:26:25.746609+0000","last_unstale":"2023-11-08T19:26:26.762914+0000","last_undegraded":"2023-11-08T19:26:26.762914+0000","last_fullsized":"2023-11-08T19:26:26.762914+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:02:15.103886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.730147+0000","last_change":"2023-11-08T19:26:25.739800+0000","last_active":"2023-11-08T19:26:26.730147+0000","last_peered":"2023-11-08T19:26:26.730147+0000","last_clean":"2023-11-08T19:26:26.730147+0000","last_became_active":"2023-11-08T19:26:25.739529+0000","last_became_peered":"2023-11-08T19:26:25.739529+0000","last_unstale":"2023-11-08T19:26:26.730147+0000","last_undegraded":"2023-11-08T19:26:26.730147+0000","last_fullsized":"2023-11-08T19:26:26.730147+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:48:09.695836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.762381+0000","last_change":"2023-11-08T19:26:25.741966+0000","last_active":"2023-11-08T19:26:26.762381+0000","last_peered":"2023-11-08T19:26:26.762381+0000","last_clean":"2023-11-08T19:26:26.762381+0000","last_became_active":"2023-11-08T19:26:25.741388+0000","last_became_peered":"2023-11-08T19:26:25.741388+0000","last_unstale":"2023-11-08T19:26:26.762381+0000","last_undegraded":"2023-11-08T19:26:26.762381+0000","last_fullsized":"2023-11-08T19:26:26.762381+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:31:35.182260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.730525+0000","last_change":"2023-11-08T19:26:25.757491+0000","last_active":"2023-11-08T19:26:26.730525+0000","last_peered":"2023-11-08T19:26:26.730525+0000","last_clean":"2023-11-08T19:26:26.730525+0000","last_became_active":"2023-11-08T19:26:25.757106+0000","last_became_peered":"2023-11-08T19:26:25.757106+0000","last_unstale":"2023-11-08T19:26:26.730525+0000","last_undegraded":"2023-11-08T19:26:26.730525+0000","last_fullsized":"2023-11-08T19:26:26.730525+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:09:25.716422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.761990+0000","last_change":"2023-11-08T19:26:25.742389+0000","last_active":"2023-11-08T19:26:26.761990+0000","last_peered":"2023-11-08T19:26:26.761990+0000","last_clean":"2023-11-08T19:26:26.761990+0000","last_became_active":"2023-11-08T19:26:25.742128+0000","last_became_peered":"2023-11-08T19:26:25.742128+0000","last_unstale":"2023-11-08T19:26:26.761990+0000","last_undegraded":"2023-11-08T19:26:26.761990+0000","last_fullsized":"2023-11-08T19:26:26.761990+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:42:13.350558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.761951+0000","last_change":"2023-11-08T19:26:25.748084+0000","last_active":"2023-11-08T19:26:26.761951+0000","last_peered":"2023-11-08T19:26:26.761951+0000","last_clean":"2023-11-08T19:26:26.761951+0000","last_became_active":"2023-11-08T19:26:25.747737+0000","last_became_peered":"2023-11-08T19:26:25.747737+0000","last_unstale":"2023-11-08T19:26:26.761951+0000","last_undegraded":"2023-11-08T19:26:26.761951+0000","last_fullsized":"2023-11-08T19:26:26.761951+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:33:05.719333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.762276+0000","last_change":"2023-11-08T19:26:25.747778+0000","last_active":"2023-11-08T19:26:26.762276+0000","last_peered":"2023-11-08T19:26:26.762276+0000","last_clean":"2023-11-08T19:26:26.762276+0000","last_became_active":"2023-11-08T19:26:25.747302+0000","last_became_peered":"2023-11-08T19:26:25.747302+0000","last_unstale":"2023-11-08T19:26:26.762276+0000","last_undegraded":"2023-11-08T19:26:26.762276+0000","last_fullsized":"2023-11-08T19:26:26.762276+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:59:22.071613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.730998+0000","last_change":"2023-11-08T19:26:25.745318+0000","last_active":"2023-11-08T19:26:26.730998+0000","last_peered":"2023-11-08T19:26:26.730998+0000","last_clean":"2023-11-08T19:26:26.730998+0000","last_became_active":"2023-11-08T19:26:25.745040+0000","last_became_peered":"2023-11-08T19:26:25.745040+0000","last_unstale":"2023-11-08T19:26:26.730998+0000","last_undegraded":"2023-11-08T19:26:26.730998+0000","last_fullsized":"2023-11-08T19:26:26.730998+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:29:30.597869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.762525+0000","last_change":"2023-11-08T19:26:25.741866+0000","last_active":"2023-11-08T19:26:26.762525+0000","last_peered":"2023-11-08T19:26:26.762525+0000","last_clean":"2023-11-08T19:26:26.762525+0000","last_became_active":"2023-11-08T19:26:25.741549+0000","last_became_peered":"2023-11-08T19:26:25.741549+0000","last_unstale":"2023-11-08T19:26:26.762525+0000","last_undegraded":"2023-11-08T19:26:26.762525+0000","last_fullsized":"2023-11-08T19:26:26.762525+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:27:46.072811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.762113+0000","last_change":"2023-11-08T19:26:25.738224+0000","last_active":"2023-11-08T19:26:26.762113+0000","last_peered":"2023-11-08T19:26:26.762113+0000","last_clean":"2023-11-08T19:26:26.762113+0000","last_became_active":"2023-11-08T19:26:25.737961+0000","last_became_peered":"2023-11-08T19:26:25.737961+0000","last_unstale":"2023-11-08T19:26:26.762113+0000","last_undegraded":"2023-11-08T19:26:26.762113+0000","last_fullsized":"2023-11-08T19:26:26.762113+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:42:48.346407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.730528+0000","last_change":"2023-11-08T19:26:25.746303+0000","last_active":"2023-11-08T19:26:26.730528+0000","last_peered":"2023-11-08T19:26:26.730528+0000","last_clean":"2023-11-08T19:26:26.730528+0000","last_became_active":"2023-11-08T19:26:25.745987+0000","last_became_peered":"2023-11-08T19:26:25.745987+0000","last_unstale":"2023-11-08T19:26:26.730528+0000","last_undegraded":"2023-11-08T19:26:26.730528+0000","last_fullsized":"2023-11-08T19:26:26.730528+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:35:24.623946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.762451+0000","last_change":"2023-11-08T19:26:25.744431+0000","last_active":"2023-11-08T19:26:26.762451+0000","last_peered":"2023-11-08T19:26:26.762451+0000","last_clean":"2023-11-08T19:26:26.762451+0000","last_became_active":"2023-11-08T19:26:25.742917+0000","last_became_peered":"2023-11-08T19:26:25.742917+0000","last_unstale":"2023-11-08T19:26:26.762451+0000","last_undegraded":"2023-11-08T19:26:26.762451+0000","last_fullsized":"2023-11-08T19:26:26.762451+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:22:57.218646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.730191+0000","last_change":"2023-11-08T19:26:25.737530+0000","last_active":"2023-11-08T19:26:26.730191+0000","last_peered":"2023-11-08T19:26:26.730191+0000","last_clean":"2023-11-08T19:26:26.730191+0000","last_became_active":"2023-11-08T19:26:25.737241+0000","last_became_peered":"2023-11-08T19:26:25.737241+0000","last_unstale":"2023-11-08T19:26:26.730191+0000","last_undegraded":"2023-11-08T19:26:26.730191+0000","last_fullsized":"2023-11-08T19:26:26.730191+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:34:52.582926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.730618+0000","last_change":"2023-11-08T19:26:25.745857+0000","last_active":"2023-11-08T19:26:26.730618+0000","last_peered":"2023-11-08T19:26:26.730618+0000","last_clean":"2023-11-08T19:26:26.730618+0000","last_became_active":"2023-11-08T19:26:25.745594+0000","last_became_peered":"2023-11-08T19:26:25.745594+0000","last_unstale":"2023-11-08T19:26:26.730618+0000","last_undegraded":"2023-11-08T19:26:26.730618+0000","last_fullsized":"2023-11-08T19:26:26.730618+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:56:38.230155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.763193+0000","last_change":"2023-11-08T19:26:25.755356+0000","last_active":"2023-11-08T19:26:26.763193+0000","last_peered":"2023-11-08T19:26:26.763193+0000","last_clean":"2023-11-08T19:26:26.763193+0000","last_became_active":"2023-11-08T19:26:25.755155+0000","last_became_peered":"2023-11-08T19:26:25.755155+0000","last_unstale":"2023-11-08T19:26:26.763193+0000","last_undegraded":"2023-11-08T19:26:26.763193+0000","last_fullsized":"2023-11-08T19:26:26.763193+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:05:31.920008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.730577+0000","last_change":"2023-11-08T19:26:25.743838+0000","last_active":"2023-11-08T19:26:26.730577+0000","last_peered":"2023-11-08T19:26:26.730577+0000","last_clean":"2023-11-08T19:26:26.730577+0000","last_became_active":"2023-11-08T19:26:25.743563+0000","last_became_peered":"2023-11-08T19:26:25.743563+0000","last_unstale":"2023-11-08T19:26:26.730577+0000","last_undegraded":"2023-11-08T19:26:26.730577+0000","last_fullsized":"2023-11-08T19:26:26.730577+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:03:31.037228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.763160+0000","last_change":"2023-11-08T19:26:25.745921+0000","last_active":"2023-11-08T19:26:26.763160+0000","last_peered":"2023-11-08T19:26:26.763160+0000","last_clean":"2023-11-08T19:26:26.763160+0000","last_became_active":"2023-11-08T19:26:25.745437+0000","last_became_peered":"2023-11-08T19:26:25.745437+0000","last_unstale":"2023-11-08T19:26:26.763160+0000","last_undegraded":"2023-11-08T19:26:26.763160+0000","last_fullsized":"2023-11-08T19:26:26.763160+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:25:17.502470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.730619+0000","last_change":"2023-11-08T19:26:25.738981+0000","last_active":"2023-11-08T19:26:26.730619+0000","last_peered":"2023-11-08T19:26:26.730619+0000","last_clean":"2023-11-08T19:26:26.730619+0000","last_became_active":"2023-11-08T19:26:25.738626+0000","last_became_peered":"2023-11-08T19:26:25.738626+0000","last_unstale":"2023-11-08T19:26:26.730619+0000","last_undegraded":"2023-11-08T19:26:26.730619+0000","last_fullsized":"2023-11-08T19:26:26.730619+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:55:03.172600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.730693+0000","last_change":"2023-11-08T19:26:25.737510+0000","last_active":"2023-11-08T19:26:26.730693+0000","last_peered":"2023-11-08T19:26:26.730693+0000","last_clean":"2023-11-08T19:26:26.730693+0000","last_became_active":"2023-11-08T19:26:25.737239+0000","last_became_peered":"2023-11-08T19:26:25.737239+0000","last_unstale":"2023-11-08T19:26:26.730693+0000","last_undegraded":"2023-11-08T19:26:26.730693+0000","last_fullsized":"2023-11-08T19:26:26.730693+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:06:07.116314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.761602+0000","last_change":"2023-11-08T19:26:25.747608+0000","last_active":"2023-11-08T19:26:26.761602+0000","last_peered":"2023-11-08T19:26:26.761602+0000","last_clean":"2023-11-08T19:26:26.761602+0000","last_became_active":"2023-11-08T19:26:25.747337+0000","last_became_peered":"2023-11-08T19:26:25.747337+0000","last_unstale":"2023-11-08T19:26:26.761602+0000","last_undegraded":"2023-11-08T19:26:26.761602+0000","last_fullsized":"2023-11-08T19:26:26.761602+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:44:11.903976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.761525+0000","last_change":"2023-11-08T19:26:25.743572+0000","last_active":"2023-11-08T19:26:26.761525+0000","last_peered":"2023-11-08T19:26:26.761525+0000","last_clean":"2023-11-08T19:26:26.761525+0000","last_became_active":"2023-11-08T19:26:25.743102+0000","last_became_peered":"2023-11-08T19:26:25.743102+0000","last_unstale":"2023-11-08T19:26:26.761525+0000","last_undegraded":"2023-11-08T19:26:26.761525+0000","last_fullsized":"2023-11-08T19:26:26.761525+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:39:12.204565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.761855+0000","last_change":"2023-11-08T19:26:25.745635+0000","last_active":"2023-11-08T19:26:26.761855+0000","last_peered":"2023-11-08T19:26:26.761855+0000","last_clean":"2023-11-08T19:26:26.761855+0000","last_became_active":"2023-11-08T19:26:25.742827+0000","last_became_peered":"2023-11-08T19:26:25.742827+0000","last_unstale":"2023-11-08T19:26:26.761855+0000","last_undegraded":"2023-11-08T19:26:26.761855+0000","last_fullsized":"2023-11-08T19:26:26.761855+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:29:30.597869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.763073+0000","last_change":"2023-11-08T19:26:25.749212+0000","last_active":"2023-11-08T19:26:26.763073+0000","last_peered":"2023-11-08T19:26:26.763073+0000","last_clean":"2023-11-08T19:26:26.763073+0000","last_became_active":"2023-11-08T19:26:25.748823+0000","last_became_peered":"2023-11-08T19:26:25.748823+0000","last_unstale":"2023-11-08T19:26:26.763073+0000","last_undegraded":"2023-11-08T19:26:26.763073+0000","last_fullsized":"2023-11-08T19:26:26.763073+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:18:26.809499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.762362+0000","last_change":"2023-11-08T19:26:25.741532+0000","last_active":"2023-11-08T19:26:26.762362+0000","last_peered":"2023-11-08T19:26:26.762362+0000","last_clean":"2023-11-08T19:26:26.762362+0000","last_became_active":"2023-11-08T19:26:25.741251+0000","last_became_peered":"2023-11-08T19:26:25.741251+0000","last_unstale":"2023-11-08T19:26:26.762362+0000","last_undegraded":"2023-11-08T19:26:26.762362+0000","last_fullsized":"2023-11-08T19:26:26.762362+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:51:48.252035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.730688+0000","last_change":"2023-11-08T19:26:25.758350+0000","last_active":"2023-11-08T19:26:26.730688+0000","last_peered":"2023-11-08T19:26:26.730688+0000","last_clean":"2023-11-08T19:26:26.730688+0000","last_became_active":"2023-11-08T19:26:25.757848+0000","last_became_peered":"2023-11-08T19:26:25.757848+0000","last_unstale":"2023-11-08T19:26:26.730688+0000","last_undegraded":"2023-11-08T19:26:26.730688+0000","last_fullsized":"2023-11-08T19:26:26.730688+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:23:33.302754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.730778+0000","last_change":"2023-11-08T19:26:25.742808+0000","last_active":"2023-11-08T19:26:26.730778+0000","last_peered":"2023-11-08T19:26:26.730778+0000","last_clean":"2023-11-08T19:26:26.730778+0000","last_became_active":"2023-11-08T19:26:25.742514+0000","last_became_peered":"2023-11-08T19:26:25.742514+0000","last_unstale":"2023-11-08T19:26:26.730778+0000","last_undegraded":"2023-11-08T19:26:26.730778+0000","last_fullsized":"2023-11-08T19:26:26.730778+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:48:11.183540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.762397+0000","last_change":"2023-11-08T19:26:25.744287+0000","last_active":"2023-11-08T19:26:26.762397+0000","last_peered":"2023-11-08T19:26:26.762397+0000","last_clean":"2023-11-08T19:26:26.762397+0000","last_became_active":"2023-11-08T19:26:25.742841+0000","last_became_peered":"2023-11-08T19:26:25.742841+0000","last_unstale":"2023-11-08T19:26:26.762397+0000","last_undegraded":"2023-11-08T19:26:26.762397+0000","last_fullsized":"2023-11-08T19:26:26.762397+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:45:24.697693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.729997+0000","last_change":"2023-11-08T19:26:25.752986+0000","last_active":"2023-11-08T19:26:26.729997+0000","last_peered":"2023-11-08T19:26:26.729997+0000","last_clean":"2023-11-08T19:26:26.729997+0000","last_became_active":"2023-11-08T19:26:25.752647+0000","last_became_peered":"2023-11-08T19:26:25.752647+0000","last_unstale":"2023-11-08T19:26:26.729997+0000","last_undegraded":"2023-11-08T19:26:26.729997+0000","last_fullsized":"2023-11-08T19:26:26.729997+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:22:57.218646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.762957+0000","last_change":"2023-11-08T19:26:25.747032+0000","last_active":"2023-11-08T19:26:26.762957+0000","last_peered":"2023-11-08T19:26:26.762957+0000","last_clean":"2023-11-08T19:26:26.762957+0000","last_became_active":"2023-11-08T19:26:25.746745+0000","last_became_peered":"2023-11-08T19:26:25.746745+0000","last_unstale":"2023-11-08T19:26:26.762957+0000","last_undegraded":"2023-11-08T19:26:26.762957+0000","last_fullsized":"2023-11-08T19:26:26.762957+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:29:44.685878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.761757+0000","last_change":"2023-11-08T19:26:25.753806+0000","last_active":"2023-11-08T19:26:26.761757+0000","last_peered":"2023-11-08T19:26:26.761757+0000","last_clean":"2023-11-08T19:26:26.761757+0000","last_became_active":"2023-11-08T19:26:25.753437+0000","last_became_peered":"2023-11-08T19:26:25.753437+0000","last_unstale":"2023-11-08T19:26:26.761757+0000","last_undegraded":"2023-11-08T19:26:26.761757+0000","last_fullsized":"2023-11-08T19:26:26.761757+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:23:47.505101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.762407+0000","last_change":"2023-11-08T19:26:25.741528+0000","last_active":"2023-11-08T19:26:26.762407+0000","last_peered":"2023-11-08T19:26:26.762407+0000","last_clean":"2023-11-08T19:26:26.762407+0000","last_became_active":"2023-11-08T19:26:25.740687+0000","last_became_peered":"2023-11-08T19:26:25.740687+0000","last_unstale":"2023-11-08T19:26:26.762407+0000","last_undegraded":"2023-11-08T19:26:26.762407+0000","last_fullsized":"2023-11-08T19:26:26.762407+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:02:36.877898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"47'2","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.762067+0000","last_change":"2023-11-08T19:26:24.743581+0000","last_active":"2023-11-08T19:26:26.762067+0000","last_peered":"2023-11-08T19:26:26.762067+0000","last_clean":"2023-11-08T19:26:26.762067+0000","last_became_active":"2023-11-08T19:26:24.743284+0000","last_became_peered":"2023-11-08T19:26:24.743284+0000","last_unstale":"2023-11-08T19:26:26.762067+0000","last_undegraded":"2023-11-08T19:26:26.762067+0000","last_fullsized":"2023-11-08T19:26:26.762067+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:35:23.618441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.739324+0000","last_change":"2023-11-08T19:26:06.297255+0000","last_active":"2023-11-08T19:26:25.739324+0000","last_peered":"2023-11-08T19:26:25.739324+0000","last_clean":"2023-11-08T19:26:25.739324+0000","last_became_active":"2023-11-08T19:26:04.603979+0000","last_became_peered":"2023-11-08T19:26:04.603979+0000","last_unstale":"2023-11-08T19:26:25.739324+0000","last_undegraded":"2023-11-08T19:26:25.739324+0000","last_fullsized":"2023-11-08T19:26:25.739324+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:40:11.164951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079032099999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.730750+0000","last_change":"2023-11-08T19:26:25.740830+0000","last_active":"2023-11-08T19:26:26.730750+0000","last_peered":"2023-11-08T19:26:26.730750+0000","last_clean":"2023-11-08T19:26:26.730750+0000","last_became_active":"2023-11-08T19:26:25.740548+0000","last_became_peered":"2023-11-08T19:26:25.740548+0000","last_unstale":"2023-11-08T19:26:26.730750+0000","last_undegraded":"2023-11-08T19:26:26.730750+0000","last_fullsized":"2023-11-08T19:26:26.730750+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:35:23.446696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"47'2","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.730598+0000","last_change":"2023-11-08T19:26:24.736817+0000","last_active":"2023-11-08T19:26:26.730598+0000","last_peered":"2023-11-08T19:26:26.730598+0000","last_clean":"2023-11-08T19:26:26.730598+0000","last_became_active":"2023-11-08T19:26:24.736538+0000","last_became_peered":"2023-11-08T19:26:24.736538+0000","last_unstale":"2023-11-08T19:26:26.730598+0000","last_undegraded":"2023-11-08T19:26:26.730598+0000","last_fullsized":"2023-11-08T19:26:26.730598+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:22:58.776603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":28,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.729426+0000","last_change":"2023-11-08T19:26:05.609625+0000","last_active":"2023-11-08T19:26:25.729426+0000","last_peered":"2023-11-08T19:26:25.729426+0000","last_clean":"2023-11-08T19:26:25.729426+0000","last_became_active":"2023-11-08T19:26:04.606312+0000","last_became_peered":"2023-11-08T19:26:04.606312+0000","last_unstale":"2023-11-08T19:26:25.729426+0000","last_undegraded":"2023-11-08T19:26:25.729426+0000","last_fullsized":"2023-11-08T19:26:25.729426+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:54:42.032901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000505935,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.762035+0000","last_change":"2023-11-08T19:26:25.745326+0000","last_active":"2023-11-08T19:26:26.762035+0000","last_peered":"2023-11-08T19:26:26.762035+0000","last_clean":"2023-11-08T19:26:26.762035+0000","last_became_active":"2023-11-08T19:26:25.744490+0000","last_became_peered":"2023-11-08T19:26:25.744490+0000","last_unstale":"2023-11-08T19:26:26.762035+0000","last_undegraded":"2023-11-08T19:26:26.762035+0000","last_fullsized":"2023-11-08T19:26:26.762035+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:48:27.754051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"47'1","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.730896+0000","last_change":"2023-11-08T19:26:24.744835+0000","last_active":"2023-11-08T19:26:26.730896+0000","last_peered":"2023-11-08T19:26:26.730896+0000","last_clean":"2023-11-08T19:26:26.730896+0000","last_became_active":"2023-11-08T19:26:24.744487+0000","last_became_peered":"2023-11-08T19:26:24.744487+0000","last_unstale":"2023-11-08T19:26:26.730896+0000","last_undegraded":"2023-11-08T19:26:26.730896+0000","last_fullsized":"2023-11-08T19:26:26.730896+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:13:28.823322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.727850+0000","last_change":"2023-11-08T19:26:06.425133+0000","last_active":"2023-11-08T19:26:25.727850+0000","last_peered":"2023-11-08T19:26:25.727850+0000","last_clean":"2023-11-08T19:26:25.727850+0000","last_became_active":"2023-11-08T19:26:04.601103+0000","last_became_peered":"2023-11-08T19:26:04.601103+0000","last_unstale":"2023-11-08T19:26:25.727850+0000","last_undegraded":"2023-11-08T19:26:25.727850+0000","last_fullsized":"2023-11-08T19:26:25.727850+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:31:14.042561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000229244,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.729685+0000","last_change":"2023-11-08T19:26:25.748005+0000","last_active":"2023-11-08T19:26:26.729685+0000","last_peered":"2023-11-08T19:26:26.729685+0000","last_clean":"2023-11-08T19:26:26.729685+0000","last_became_active":"2023-11-08T19:26:25.747426+0000","last_became_peered":"2023-11-08T19:26:25.747426+0000","last_unstale":"2023-11-08T19:26:26.729685+0000","last_undegraded":"2023-11-08T19:26:26.729685+0000","last_fullsized":"2023-11-08T19:26:26.729685+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:49:29.652891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"47'4","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.748120+0000","last_change":"2023-11-08T19:26:24.738249+0000","last_active":"2023-11-08T19:26:25.748120+0000","last_peered":"2023-11-08T19:26:25.748120+0000","last_clean":"2023-11-08T19:26:25.748120+0000","last_became_active":"2023-11-08T19:26:24.737997+0000","last_became_peered":"2023-11-08T19:26:24.737997+0000","last_unstale":"2023-11-08T19:26:25.748120+0000","last_undegraded":"2023-11-08T19:26:25.748120+0000","last_fullsized":"2023-11-08T19:26:25.748120+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:40:31.299145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.732153+0000","last_change":"2023-11-08T19:26:06.265690+0000","last_active":"2023-11-08T19:26:25.732153+0000","last_peered":"2023-11-08T19:26:25.732153+0000","last_clean":"2023-11-08T19:26:25.732153+0000","last_became_active":"2023-11-08T19:26:04.603397+0000","last_became_peered":"2023-11-08T19:26:04.603397+0000","last_unstale":"2023-11-08T19:26:25.732153+0000","last_undegraded":"2023-11-08T19:26:25.732153+0000","last_fullsized":"2023-11-08T19:26:25.732153+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:50:43.827570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053361499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.730396+0000","last_change":"2023-11-08T19:26:25.743979+0000","last_active":"2023-11-08T19:26:26.730396+0000","last_peered":"2023-11-08T19:26:26.730396+0000","last_clean":"2023-11-08T19:26:26.730396+0000","last_became_active":"2023-11-08T19:26:25.743680+0000","last_became_peered":"2023-11-08T19:26:25.743680+0000","last_unstale":"2023-11-08T19:26:26.730396+0000","last_undegraded":"2023-11-08T19:26:26.730396+0000","last_fullsized":"2023-11-08T19:26:26.730396+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:38:44.411483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.761728+0000","last_change":"2023-11-08T19:26:24.739326+0000","last_active":"2023-11-08T19:26:26.761728+0000","last_peered":"2023-11-08T19:26:26.761728+0000","last_clean":"2023-11-08T19:26:26.761728+0000","last_became_active":"2023-11-08T19:26:24.738986+0000","last_became_peered":"2023-11-08T19:26:24.738986+0000","last_unstale":"2023-11-08T19:26:26.761728+0000","last_undegraded":"2023-11-08T19:26:26.761728+0000","last_fullsized":"2023-11-08T19:26:26.761728+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:23:32.297248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.738268+0000","last_change":"2023-11-08T19:26:06.541338+0000","last_active":"2023-11-08T19:26:25.738268+0000","last_peered":"2023-11-08T19:26:25.738268+0000","last_clean":"2023-11-08T19:26:25.738268+0000","last_became_active":"2023-11-08T19:26:04.602183+0000","last_became_peered":"2023-11-08T19:26:04.602183+0000","last_unstale":"2023-11-08T19:26:25.738268+0000","last_undegraded":"2023-11-08T19:26:25.738268+0000","last_fullsized":"2023-11-08T19:26:25.738268+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:06:47.446686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052029099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.762021+0000","last_change":"2023-11-08T19:26:25.741349+0000","last_active":"2023-11-08T19:26:26.762021+0000","last_peered":"2023-11-08T19:26:26.762021+0000","last_clean":"2023-11-08T19:26:26.762021+0000","last_became_active":"2023-11-08T19:26:25.741052+0000","last_became_peered":"2023-11-08T19:26:25.741052+0000","last_unstale":"2023-11-08T19:26:26.762021+0000","last_undegraded":"2023-11-08T19:26:26.762021+0000","last_fullsized":"2023-11-08T19:26:26.762021+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:49:22.367937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.730162+0000","last_change":"2023-11-08T19:26:24.740255+0000","last_active":"2023-11-08T19:26:25.730162+0000","last_peered":"2023-11-08T19:26:25.730162+0000","last_clean":"2023-11-08T19:26:25.730162+0000","last_became_active":"2023-11-08T19:26:24.739986+0000","last_became_peered":"2023-11-08T19:26:24.739986+0000","last_unstale":"2023-11-08T19:26:25.730162+0000","last_undegraded":"2023-11-08T19:26:25.730162+0000","last_fullsized":"2023-11-08T19:26:25.730162+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:11:46.642828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.734103+0000","last_change":"2023-11-08T19:26:06.394081+0000","last_active":"2023-11-08T19:26:25.734103+0000","last_peered":"2023-11-08T19:26:25.734103+0000","last_clean":"2023-11-08T19:26:25.734103+0000","last_became_active":"2023-11-08T19:26:04.602137+0000","last_became_peered":"2023-11-08T19:26:04.602137+0000","last_unstale":"2023-11-08T19:26:25.734103+0000","last_undegraded":"2023-11-08T19:26:25.734103+0000","last_fullsized":"2023-11-08T19:26:25.734103+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:05:46.951663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056998200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.730862+0000","last_change":"2023-11-08T19:26:25.758675+0000","last_active":"2023-11-08T19:26:26.730862+0000","last_peered":"2023-11-08T19:26:26.730862+0000","last_clean":"2023-11-08T19:26:26.730862+0000","last_became_active":"2023-11-08T19:26:25.758495+0000","last_became_peered":"2023-11-08T19:26:25.758495+0000","last_unstale":"2023-11-08T19:26:26.730862+0000","last_undegraded":"2023-11-08T19:26:26.730862+0000","last_fullsized":"2023-11-08T19:26:26.730862+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:18:26.809499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"47'1","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.762131+0000","last_change":"2023-11-08T19:26:24.738043+0000","last_active":"2023-11-08T19:26:26.762131+0000","last_peered":"2023-11-08T19:26:26.762131+0000","last_clean":"2023-11-08T19:26:26.762131+0000","last_became_active":"2023-11-08T19:26:24.737782+0000","last_became_peered":"2023-11-08T19:26:24.737782+0000","last_unstale":"2023-11-08T19:26:26.762131+0000","last_undegraded":"2023-11-08T19:26:26.762131+0000","last_fullsized":"2023-11-08T19:26:26.762131+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:18:22.010798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.728229+0000","last_change":"2023-11-08T19:26:06.423863+0000","last_active":"2023-11-08T19:26:25.728229+0000","last_peered":"2023-11-08T19:26:25.728229+0000","last_clean":"2023-11-08T19:26:25.728229+0000","last_became_active":"2023-11-08T19:26:04.608639+0000","last_became_peered":"2023-11-08T19:26:04.608639+0000","last_unstale":"2023-11-08T19:26:25.728229+0000","last_undegraded":"2023-11-08T19:26:25.728229+0000","last_fullsized":"2023-11-08T19:26:25.728229+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:04:43.631102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044384999999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.762598+0000","last_change":"2023-11-08T19:26:25.746985+0000","last_active":"2023-11-08T19:26:26.762598+0000","last_peered":"2023-11-08T19:26:26.762598+0000","last_clean":"2023-11-08T19:26:26.762598+0000","last_became_active":"2023-11-08T19:26:25.746718+0000","last_became_peered":"2023-11-08T19:26:25.746718+0000","last_unstale":"2023-11-08T19:26:26.762598+0000","last_undegraded":"2023-11-08T19:26:26.762598+0000","last_fullsized":"2023-11-08T19:26:26.762598+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:36:07.345392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"47'1","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.762456+0000","last_change":"2023-11-08T19:26:24.742192+0000","last_active":"2023-11-08T19:26:26.762456+0000","last_peered":"2023-11-08T19:26:26.762456+0000","last_clean":"2023-11-08T19:26:26.762456+0000","last_became_active":"2023-11-08T19:26:24.741925+0000","last_became_peered":"2023-11-08T19:26:24.741925+0000","last_unstale":"2023-11-08T19:26:26.762456+0000","last_undegraded":"2023-11-08T19:26:26.762456+0000","last_fullsized":"2023-11-08T19:26:26.762456+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:05:03.696760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.733044+0000","last_change":"2023-11-08T19:26:06.265318+0000","last_active":"2023-11-08T19:26:25.733044+0000","last_peered":"2023-11-08T19:26:25.733044+0000","last_clean":"2023-11-08T19:26:25.733044+0000","last_became_active":"2023-11-08T19:26:04.603409+0000","last_became_peered":"2023-11-08T19:26:04.603409+0000","last_unstale":"2023-11-08T19:26:25.733044+0000","last_undegraded":"2023-11-08T19:26:25.733044+0000","last_fullsized":"2023-11-08T19:26:25.733044+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:27:24.933111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065684300000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.762588+0000","last_change":"2023-11-08T19:26:25.750379+0000","last_active":"2023-11-08T19:26:26.762588+0000","last_peered":"2023-11-08T19:26:26.762588+0000","last_clean":"2023-11-08T19:26:26.762588+0000","last_became_active":"2023-11-08T19:26:25.750060+0000","last_became_peered":"2023-11-08T19:26:25.750060+0000","last_unstale":"2023-11-08T19:26:26.762588+0000","last_undegraded":"2023-11-08T19:26:26.762588+0000","last_fullsized":"2023-11-08T19:26:26.762588+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:22:59.782108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"47'6","reported_seq":18,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.761687+0000","last_change":"2023-11-08T19:26:24.742213+0000","last_active":"2023-11-08T19:26:26.761687+0000","last_peered":"2023-11-08T19:26:26.761687+0000","last_clean":"2023-11-08T19:26:26.761687+0000","last_became_active":"2023-11-08T19:26:24.741916+0000","last_became_peered":"2023-11-08T19:26:24.741916+0000","last_unstale":"2023-11-08T19:26:26.761687+0000","last_undegraded":"2023-11-08T19:26:26.761687+0000","last_fullsized":"2023-11-08T19:26:26.761687+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:42:12.345053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":28,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.728613+0000","last_change":"2023-11-08T19:26:05.610012+0000","last_active":"2023-11-08T19:26:25.728613+0000","last_peered":"2023-11-08T19:26:25.728613+0000","last_clean":"2023-11-08T19:26:25.728613+0000","last_became_active":"2023-11-08T19:26:04.603209+0000","last_became_peered":"2023-11-08T19:26:04.603209+0000","last_unstale":"2023-11-08T19:26:25.728613+0000","last_undegraded":"2023-11-08T19:26:25.728613+0000","last_fullsized":"2023-11-08T19:26:25.728613+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:35:02.306997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053335100000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.762447+0000","last_change":"2023-11-08T19:26:25.749064+0000","last_active":"2023-11-08T19:26:26.762447+0000","last_peered":"2023-11-08T19:26:26.762447+0000","last_clean":"2023-11-08T19:26:26.762447+0000","last_became_active":"2023-11-08T19:26:25.748500+0000","last_became_peered":"2023-11-08T19:26:25.748500+0000","last_unstale":"2023-11-08T19:26:26.762447+0000","last_undegraded":"2023-11-08T19:26:26.762447+0000","last_fullsized":"2023-11-08T19:26:26.762447+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:49:29.652891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"47'3","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.760398+0000","last_change":"2023-11-08T19:26:24.744478+0000","last_active":"2023-11-08T19:26:25.760398+0000","last_peered":"2023-11-08T19:26:25.760398+0000","last_clean":"2023-11-08T19:26:25.760398+0000","last_became_active":"2023-11-08T19:26:24.744253+0000","last_became_peered":"2023-11-08T19:26:24.744253+0000","last_unstale":"2023-11-08T19:26:25.760398+0000","last_undegraded":"2023-11-08T19:26:25.760398+0000","last_fullsized":"2023-11-08T19:26:25.760398+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:23:46.499595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.737675+0000","last_change":"2023-11-08T19:26:05.606946+0000","last_active":"2023-11-08T19:26:25.737675+0000","last_peered":"2023-11-08T19:26:25.737675+0000","last_clean":"2023-11-08T19:26:25.737675+0000","last_became_active":"2023-11-08T19:26:04.598775+0000","last_became_peered":"2023-11-08T19:26:04.598775+0000","last_unstale":"2023-11-08T19:26:25.737675+0000","last_undegraded":"2023-11-08T19:26:25.737675+0000","last_fullsized":"2023-11-08T19:26:25.737675+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:20:56.898927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049870399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.762480+0000","last_change":"2023-11-08T19:26:25.746669+0000","last_active":"2023-11-08T19:26:26.762480+0000","last_peered":"2023-11-08T19:26:26.762480+0000","last_clean":"2023-11-08T19:26:26.762480+0000","last_became_active":"2023-11-08T19:26:25.746375+0000","last_became_peered":"2023-11-08T19:26:25.746375+0000","last_unstale":"2023-11-08T19:26:26.762480+0000","last_undegraded":"2023-11-08T19:26:26.762480+0000","last_fullsized":"2023-11-08T19:26:26.762480+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:42:13.350558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.762625+0000","last_change":"2023-11-08T19:26:24.745917+0000","last_active":"2023-11-08T19:26:26.762625+0000","last_peered":"2023-11-08T19:26:26.762625+0000","last_clean":"2023-11-08T19:26:26.762625+0000","last_became_active":"2023-11-08T19:26:24.745606+0000","last_became_peered":"2023-11-08T19:26:24.745606+0000","last_unstale":"2023-11-08T19:26:26.762625+0000","last_undegraded":"2023-11-08T19:26:26.762625+0000","last_fullsized":"2023-11-08T19:26:26.762625+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:48:10.178035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.731955+0000","last_change":"2023-11-08T19:26:06.265907+0000","last_active":"2023-11-08T19:26:25.731955+0000","last_peered":"2023-11-08T19:26:25.731955+0000","last_clean":"2023-11-08T19:26:25.731955+0000","last_became_active":"2023-11-08T19:26:04.604772+0000","last_became_peered":"2023-11-08T19:26:04.604772+0000","last_unstale":"2023-11-08T19:26:25.731955+0000","last_undegraded":"2023-11-08T19:26:25.731955+0000","last_fullsized":"2023-11-08T19:26:25.731955+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:44:16.139794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066875300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.761632+0000","last_change":"2023-11-08T19:26:25.751452+0000","last_active":"2023-11-08T19:26:26.761632+0000","last_peered":"2023-11-08T19:26:26.761632+0000","last_clean":"2023-11-08T19:26:26.761632+0000","last_became_active":"2023-11-08T19:26:25.751140+0000","last_became_peered":"2023-11-08T19:26:25.751140+0000","last_unstale":"2023-11-08T19:26:26.761632+0000","last_undegraded":"2023-11-08T19:26:26.761632+0000","last_fullsized":"2023-11-08T19:26:26.761632+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:43:16.409064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.731355+0000","last_change":"2023-11-08T19:26:24.741119+0000","last_active":"2023-11-08T19:26:25.731355+0000","last_peered":"2023-11-08T19:26:25.731355+0000","last_clean":"2023-11-08T19:26:25.731355+0000","last_became_active":"2023-11-08T19:26:24.740821+0000","last_became_peered":"2023-11-08T19:26:24.740821+0000","last_unstale":"2023-11-08T19:26:25.731355+0000","last_undegraded":"2023-11-08T19:26:25.731355+0000","last_fullsized":"2023-11-08T19:26:25.731355+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:45:23.692188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.734895+0000","last_change":"2023-11-08T19:26:06.394488+0000","last_active":"2023-11-08T19:26:25.734895+0000","last_peered":"2023-11-08T19:26:25.734895+0000","last_clean":"2023-11-08T19:26:25.734895+0000","last_became_active":"2023-11-08T19:26:04.606805+0000","last_became_peered":"2023-11-08T19:26:04.606805+0000","last_unstale":"2023-11-08T19:26:25.734895+0000","last_undegraded":"2023-11-08T19:26:25.734895+0000","last_fullsized":"2023-11-08T19:26:25.734895+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:25:20.857133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045317000000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.763082+0000","last_change":"2023-11-08T19:26:25.755102+0000","last_active":"2023-11-08T19:26:26.763082+0000","last_peered":"2023-11-08T19:26:26.763082+0000","last_clean":"2023-11-08T19:26:26.763082+0000","last_became_active":"2023-11-08T19:26:25.750220+0000","last_became_peered":"2023-11-08T19:26:25.750220+0000","last_unstale":"2023-11-08T19:26:26.763082+0000","last_undegraded":"2023-11-08T19:26:26.763082+0000","last_fullsized":"2023-11-08T19:26:26.763082+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:11:13.832264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"47'8","reported_seq":20,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.762303+0000","last_change":"2023-11-08T19:26:24.743410+0000","last_active":"2023-11-08T19:26:26.762303+0000","last_peered":"2023-11-08T19:26:26.762303+0000","last_clean":"2023-11-08T19:26:26.762303+0000","last_became_active":"2023-11-08T19:26:24.743162+0000","last_became_peered":"2023-11-08T19:26:24.743162+0000","last_unstale":"2023-11-08T19:26:26.762303+0000","last_undegraded":"2023-11-08T19:26:26.762303+0000","last_fullsized":"2023-11-08T19:26:26.762303+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:22:56.213140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.727378+0000","last_change":"2023-11-08T19:26:06.423754+0000","last_active":"2023-11-08T19:26:25.727378+0000","last_peered":"2023-11-08T19:26:25.727378+0000","last_clean":"2023-11-08T19:26:25.727378+0000","last_became_active":"2023-11-08T19:26:04.599322+0000","last_became_peered":"2023-11-08T19:26:04.599322+0000","last_unstale":"2023-11-08T19:26:25.727378+0000","last_undegraded":"2023-11-08T19:26:25.727378+0000","last_fullsized":"2023-11-08T19:26:25.727378+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:13:08.689128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042670000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.761894+0000","last_change":"2023-11-08T19:26:25.736435+0000","last_active":"2023-11-08T19:26:26.761894+0000","last_peered":"2023-11-08T19:26:26.761894+0000","last_clean":"2023-11-08T19:26:26.761894+0000","last_became_active":"2023-11-08T19:26:25.736134+0000","last_became_peered":"2023-11-08T19:26:25.736134+0000","last_unstale":"2023-11-08T19:26:26.761894+0000","last_undegraded":"2023-11-08T19:26:26.761894+0000","last_fullsized":"2023-11-08T19:26:26.761894+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:06:45.575533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.729611+0000","last_change":"2023-11-08T19:26:24.743478+0000","last_active":"2023-11-08T19:26:25.729611+0000","last_peered":"2023-11-08T19:26:25.729611+0000","last_clean":"2023-11-08T19:26:25.729611+0000","last_became_active":"2023-11-08T19:26:24.743177+0000","last_became_peered":"2023-11-08T19:26:24.743177+0000","last_unstale":"2023-11-08T19:26:25.729611+0000","last_undegraded":"2023-11-08T19:26:25.729611+0000","last_fullsized":"2023-11-08T19:26:25.729611+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:29:43.680373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.733903+0000","last_change":"2023-11-08T19:26:06.394320+0000","last_active":"2023-11-08T19:26:25.733903+0000","last_peered":"2023-11-08T19:26:25.733903+0000","last_clean":"2023-11-08T19:26:25.733903+0000","last_became_active":"2023-11-08T19:26:04.603212+0000","last_became_peered":"2023-11-08T19:26:04.603212+0000","last_unstale":"2023-11-08T19:26:25.733903+0000","last_undegraded":"2023-11-08T19:26:25.733903+0000","last_fullsized":"2023-11-08T19:26:25.733903+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:25:29.947079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054695399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.763031+0000","last_change":"2023-11-08T19:26:25.758900+0000","last_active":"2023-11-08T19:26:26.763031+0000","last_peered":"2023-11-08T19:26:26.763031+0000","last_clean":"2023-11-08T19:26:26.763031+0000","last_became_active":"2023-11-08T19:26:25.758612+0000","last_became_peered":"2023-11-08T19:26:25.758612+0000","last_unstale":"2023-11-08T19:26:26.763031+0000","last_undegraded":"2023-11-08T19:26:26.763031+0000","last_fullsized":"2023-11-08T19:26:26.763031+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:03:31.037228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.728395+0000","last_change":"2023-11-08T19:26:24.739300+0000","last_active":"2023-11-08T19:26:25.728395+0000","last_peered":"2023-11-08T19:26:25.728395+0000","last_clean":"2023-11-08T19:26:25.728395+0000","last_became_active":"2023-11-08T19:26:24.738988+0000","last_became_peered":"2023-11-08T19:26:24.738988+0000","last_unstale":"2023-11-08T19:26:25.728395+0000","last_undegraded":"2023-11-08T19:26:25.728395+0000","last_fullsized":"2023-11-08T19:26:25.728395+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:44:36.273988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.728522+0000","last_change":"2023-11-08T19:26:06.424073+0000","last_active":"2023-11-08T19:26:25.728522+0000","last_peered":"2023-11-08T19:26:25.728522+0000","last_clean":"2023-11-08T19:26:25.728522+0000","last_became_active":"2023-11-08T19:26:04.599046+0000","last_became_peered":"2023-11-08T19:26:04.599046+0000","last_unstale":"2023-11-08T19:26:25.728522+0000","last_undegraded":"2023-11-08T19:26:25.728522+0000","last_fullsized":"2023-11-08T19:26:25.728522+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:04:43.562566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035092199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.762035+0000","last_change":"2023-11-08T19:26:25.744142+0000","last_active":"2023-11-08T19:26:26.762035+0000","last_peered":"2023-11-08T19:26:26.762035+0000","last_clean":"2023-11-08T19:26:26.762035+0000","last_became_active":"2023-11-08T19:26:25.743686+0000","last_became_peered":"2023-11-08T19:26:25.743686+0000","last_unstale":"2023-11-08T19:26:26.762035+0000","last_undegraded":"2023-11-08T19:26:26.762035+0000","last_fullsized":"2023-11-08T19:26:26.762035+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:29:30.597869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"47'2","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.762518+0000","last_change":"2023-11-08T19:26:24.740785+0000","last_active":"2023-11-08T19:26:26.762518+0000","last_peered":"2023-11-08T19:26:26.762518+0000","last_clean":"2023-11-08T19:26:26.762518+0000","last_became_active":"2023-11-08T19:26:24.740482+0000","last_became_peered":"2023-11-08T19:26:24.740482+0000","last_unstale":"2023-11-08T19:26:26.762518+0000","last_undegraded":"2023-11-08T19:26:26.762518+0000","last_fullsized":"2023-11-08T19:26:26.762518+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:22:25.145083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":28,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.728645+0000","last_change":"2023-11-08T19:26:05.608980+0000","last_active":"2023-11-08T19:26:25.728645+0000","last_peered":"2023-11-08T19:26:25.728645+0000","last_clean":"2023-11-08T19:26:25.728645+0000","last_became_active":"2023-11-08T19:26:04.606386+0000","last_became_peered":"2023-11-08T19:26:04.606386+0000","last_unstale":"2023-11-08T19:26:25.728645+0000","last_undegraded":"2023-11-08T19:26:25.728645+0000","last_fullsized":"2023-11-08T19:26:25.728645+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:41:50.771457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037950599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.730654+0000","last_change":"2023-11-08T19:26:25.757783+0000","last_active":"2023-11-08T19:26:26.730654+0000","last_peered":"2023-11-08T19:26:26.730654+0000","last_clean":"2023-11-08T19:26:26.730654+0000","last_became_active":"2023-11-08T19:26:25.757482+0000","last_became_peered":"2023-11-08T19:26:25.757482+0000","last_unstale":"2023-11-08T19:26:26.730654+0000","last_undegraded":"2023-11-08T19:26:26.730654+0000","last_fullsized":"2023-11-08T19:26:26.730654+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:11:18.036694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.728806+0000","last_change":"2023-11-08T19:26:24.743633+0000","last_active":"2023-11-08T19:26:25.728806+0000","last_peered":"2023-11-08T19:26:25.728806+0000","last_clean":"2023-11-08T19:26:25.728806+0000","last_became_active":"2023-11-08T19:26:24.743288+0000","last_became_peered":"2023-11-08T19:26:24.743288+0000","last_unstale":"2023-11-08T19:26:25.728806+0000","last_undegraded":"2023-11-08T19:26:25.728806+0000","last_fullsized":"2023-11-08T19:26:25.728806+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:20:09.309300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.738107+0000","last_change":"2023-11-08T19:26:06.541623+0000","last_active":"2023-11-08T19:26:25.738107+0000","last_peered":"2023-11-08T19:26:25.738107+0000","last_clean":"2023-11-08T19:26:25.738107+0000","last_became_active":"2023-11-08T19:26:04.601961+0000","last_became_peered":"2023-11-08T19:26:04.601961+0000","last_unstale":"2023-11-08T19:26:25.738107+0000","last_undegraded":"2023-11-08T19:26:25.738107+0000","last_fullsized":"2023-11-08T19:26:25.738107+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:11:26.508634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000466386,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.761661+0000","last_change":"2023-11-08T19:26:25.747292+0000","last_active":"2023-11-08T19:26:26.761661+0000","last_peered":"2023-11-08T19:26:26.761661+0000","last_clean":"2023-11-08T19:26:26.761661+0000","last_became_active":"2023-11-08T19:26:25.747014+0000","last_became_peered":"2023-11-08T19:26:25.747014+0000","last_unstale":"2023-11-08T19:26:26.761661+0000","last_undegraded":"2023-11-08T19:26:26.761661+0000","last_fullsized":"2023-11-08T19:26:26.761661+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:42:14.637155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"47'1","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.759266+0000","last_change":"2023-11-08T19:26:24.733981+0000","last_active":"2023-11-08T19:26:25.759266+0000","last_peered":"2023-11-08T19:26:25.759266+0000","last_clean":"2023-11-08T19:26:25.759266+0000","last_became_active":"2023-11-08T19:26:24.733628+0000","last_became_peered":"2023-11-08T19:26:24.733628+0000","last_unstale":"2023-11-08T19:26:25.759266+0000","last_undegraded":"2023-11-08T19:26:25.759266+0000","last_fullsized":"2023-11-08T19:26:25.759266+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:07:07.580880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.727605+0000","last_change":"2023-11-08T19:26:06.424464+0000","last_active":"2023-11-08T19:26:25.727605+0000","last_peered":"2023-11-08T19:26:25.727605+0000","last_clean":"2023-11-08T19:26:25.727605+0000","last_became_active":"2023-11-08T19:26:04.603144+0000","last_became_peered":"2023-11-08T19:26:04.603144+0000","last_unstale":"2023-11-08T19:26:25.727605+0000","last_undegraded":"2023-11-08T19:26:25.727605+0000","last_fullsized":"2023-11-08T19:26:25.727605+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:42:55.269365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054090799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.730354+0000","last_change":"2023-11-08T19:26:25.745301+0000","last_active":"2023-11-08T19:26:26.730354+0000","last_peered":"2023-11-08T19:26:26.730354+0000","last_clean":"2023-11-08T19:26:26.730354+0000","last_became_active":"2023-11-08T19:26:25.745014+0000","last_became_peered":"2023-11-08T19:26:25.745014+0000","last_unstale":"2023-11-08T19:26:26.730354+0000","last_undegraded":"2023-11-08T19:26:26.730354+0000","last_fullsized":"2023-11-08T19:26:26.730354+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:22:26.150588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"47'2","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.762839+0000","last_change":"2023-11-08T19:26:24.739629+0000","last_active":"2023-11-08T19:26:26.762839+0000","last_peered":"2023-11-08T19:26:26.762839+0000","last_clean":"2023-11-08T19:26:26.762839+0000","last_became_active":"2023-11-08T19:26:24.739346+0000","last_became_peered":"2023-11-08T19:26:24.739346+0000","last_unstale":"2023-11-08T19:26:26.762839+0000","last_undegraded":"2023-11-08T19:26:26.762839+0000","last_fullsized":"2023-11-08T19:26:26.762839+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:25:50.081273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.738253+0000","last_change":"2023-11-08T19:26:05.606898+0000","last_active":"2023-11-08T19:26:25.738253+0000","last_peered":"2023-11-08T19:26:25.738253+0000","last_clean":"2023-11-08T19:26:25.738253+0000","last_became_active":"2023-11-08T19:26:04.603955+0000","last_became_peered":"2023-11-08T19:26:04.603955+0000","last_unstale":"2023-11-08T19:26:25.738253+0000","last_undegraded":"2023-11-08T19:26:25.738253+0000","last_fullsized":"2023-11-08T19:26:25.738253+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:19:11.589559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036473499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.762090+0000","last_change":"2023-11-08T19:26:25.749337+0000","last_active":"2023-11-08T19:26:26.762090+0000","last_peered":"2023-11-08T19:26:26.762090+0000","last_clean":"2023-11-08T19:26:26.762090+0000","last_became_active":"2023-11-08T19:26:25.749069+0000","last_became_peered":"2023-11-08T19:26:25.749069+0000","last_unstale":"2023-11-08T19:26:26.762090+0000","last_undegraded":"2023-11-08T19:26:26.762090+0000","last_fullsized":"2023-11-08T19:26:26.762090+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:03:20.639343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"47'1","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.762883+0000","last_change":"2023-11-08T19:26:24.742330+0000","last_active":"2023-11-08T19:26:26.762883+0000","last_peered":"2023-11-08T19:26:26.762883+0000","last_clean":"2023-11-08T19:26:26.762883+0000","last_became_active":"2023-11-08T19:26:24.742055+0000","last_became_peered":"2023-11-08T19:26:24.742055+0000","last_unstale":"2023-11-08T19:26:26.762883+0000","last_undegraded":"2023-11-08T19:26:26.762883+0000","last_fullsized":"2023-11-08T19:26:26.762883+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:48:26.748546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.732066+0000","last_change":"2023-11-08T19:26:06.265686+0000","last_active":"2023-11-08T19:26:25.732066+0000","last_peered":"2023-11-08T19:26:25.732066+0000","last_clean":"2023-11-08T19:26:25.732066+0000","last_became_active":"2023-11-08T19:26:04.602111+0000","last_became_peered":"2023-11-08T19:26:04.602111+0000","last_unstale":"2023-11-08T19:26:25.732066+0000","last_undegraded":"2023-11-08T19:26:25.732066+0000","last_fullsized":"2023-11-08T19:26:25.732066+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:40:11.164951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056197899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.730988+0000","last_change":"2023-11-08T19:26:25.746346+0000","last_active":"2023-11-08T19:26:26.730988+0000","last_peered":"2023-11-08T19:26:26.730988+0000","last_clean":"2023-11-08T19:26:26.730988+0000","last_became_active":"2023-11-08T19:26:25.746052+0000","last_became_peered":"2023-11-08T19:26:25.746052+0000","last_unstale":"2023-11-08T19:26:26.730988+0000","last_undegraded":"2023-11-08T19:26:26.730988+0000","last_fullsized":"2023-11-08T19:26:26.730988+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:48:27.754051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.761287+0000","last_change":"2023-11-08T19:26:24.740736+0000","last_active":"2023-11-08T19:26:26.761287+0000","last_peered":"2023-11-08T19:26:26.761287+0000","last_clean":"2023-11-08T19:26:26.761287+0000","last_became_active":"2023-11-08T19:26:24.740479+0000","last_became_peered":"2023-11-08T19:26:24.740479+0000","last_unstale":"2023-11-08T19:26:26.761287+0000","last_undegraded":"2023-11-08T19:26:26.761287+0000","last_fullsized":"2023-11-08T19:26:26.761287+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:20:16.421029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.734247+0000","last_change":"2023-11-08T19:26:06.394075+0000","last_active":"2023-11-08T19:26:25.734247+0000","last_peered":"2023-11-08T19:26:25.734247+0000","last_clean":"2023-11-08T19:26:25.734247+0000","last_became_active":"2023-11-08T19:26:04.606967+0000","last_became_peered":"2023-11-08T19:26:04.606967+0000","last_unstale":"2023-11-08T19:26:25.734247+0000","last_undegraded":"2023-11-08T19:26:25.734247+0000","last_fullsized":"2023-11-08T19:26:25.734247+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:19:56.286835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040971999999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.762648+0000","last_change":"2023-11-08T19:26:25.744126+0000","last_active":"2023-11-08T19:26:26.762648+0000","last_peered":"2023-11-08T19:26:26.762648+0000","last_clean":"2023-11-08T19:26:26.762648+0000","last_became_active":"2023-11-08T19:26:25.743853+0000","last_became_peered":"2023-11-08T19:26:25.743853+0000","last_unstale":"2023-11-08T19:26:26.762648+0000","last_undegraded":"2023-11-08T19:26:26.762648+0000","last_fullsized":"2023-11-08T19:26:26.762648+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:34:52.582926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.762073+0000","last_change":"2023-11-08T19:26:24.746451+0000","last_active":"2023-11-08T19:26:26.762073+0000","last_peered":"2023-11-08T19:26:26.762073+0000","last_clean":"2023-11-08T19:26:26.762073+0000","last_became_active":"2023-11-08T19:26:24.746152+0000","last_became_peered":"2023-11-08T19:26:24.746152+0000","last_unstale":"2023-11-08T19:26:26.762073+0000","last_undegraded":"2023-11-08T19:26:26.762073+0000","last_fullsized":"2023-11-08T19:26:26.762073+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:56:37.224649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":28,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.731324+0000","last_change":"2023-11-08T19:26:05.608892+0000","last_active":"2023-11-08T19:26:25.731324+0000","last_peered":"2023-11-08T19:26:25.731324+0000","last_clean":"2023-11-08T19:26:25.731324+0000","last_became_active":"2023-11-08T19:26:04.604519+0000","last_became_peered":"2023-11-08T19:26:04.604519+0000","last_unstale":"2023-11-08T19:26:25.731324+0000","last_undegraded":"2023-11-08T19:26:25.731324+0000","last_fullsized":"2023-11-08T19:26:25.731324+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:11:26.508634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051248099999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.730957+0000","last_change":"2023-11-08T19:26:25.734469+0000","last_active":"2023-11-08T19:26:26.730957+0000","last_peered":"2023-11-08T19:26:26.730957+0000","last_clean":"2023-11-08T19:26:26.730957+0000","last_became_active":"2023-11-08T19:26:25.734211+0000","last_became_peered":"2023-11-08T19:26:25.734211+0000","last_unstale":"2023-11-08T19:26:26.730957+0000","last_undegraded":"2023-11-08T19:26:26.730957+0000","last_fullsized":"2023-11-08T19:26:26.730957+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:13:29.828827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.761686+0000","last_change":"2023-11-08T19:26:24.742825+0000","last_active":"2023-11-08T19:26:26.761686+0000","last_peered":"2023-11-08T19:26:26.761686+0000","last_clean":"2023-11-08T19:26:26.761686+0000","last_became_active":"2023-11-08T19:26:24.742467+0000","last_became_peered":"2023-11-08T19:26:24.742467+0000","last_unstale":"2023-11-08T19:26:26.761686+0000","last_undegraded":"2023-11-08T19:26:26.761686+0000","last_fullsized":"2023-11-08T19:26:26.761686+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:49:52.879282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.728303+0000","last_change":"2023-11-08T19:26:06.424236+0000","last_active":"2023-11-08T19:26:25.728303+0000","last_peered":"2023-11-08T19:26:25.728303+0000","last_clean":"2023-11-08T19:26:25.728303+0000","last_became_active":"2023-11-08T19:26:04.600115+0000","last_became_peered":"2023-11-08T19:26:04.600115+0000","last_unstale":"2023-11-08T19:26:25.728303+0000","last_undegraded":"2023-11-08T19:26:25.728303+0000","last_fullsized":"2023-11-08T19:26:25.728303+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:18:01.876605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065803900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.762236+0000","last_change":"2023-11-08T19:26:25.743964+0000","last_active":"2023-11-08T19:26:26.762236+0000","last_peered":"2023-11-08T19:26:26.762236+0000","last_clean":"2023-11-08T19:26:26.762236+0000","last_became_active":"2023-11-08T19:26:25.743403+0000","last_became_peered":"2023-11-08T19:26:25.743403+0000","last_unstale":"2023-11-08T19:26:26.762236+0000","last_undegraded":"2023-11-08T19:26:26.762236+0000","last_fullsized":"2023-11-08T19:26:26.762236+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:29:44.685878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"47'2","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.730526+0000","last_change":"2023-11-08T19:26:24.735862+0000","last_active":"2023-11-08T19:26:26.730526+0000","last_peered":"2023-11-08T19:26:26.730526+0000","last_clean":"2023-11-08T19:26:26.730526+0000","last_became_active":"2023-11-08T19:26:24.735588+0000","last_became_peered":"2023-11-08T19:26:24.735588+0000","last_unstale":"2023-11-08T19:26:26.730526+0000","last_undegraded":"2023-11-08T19:26:26.730526+0000","last_fullsized":"2023-11-08T19:26:26.730526+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:05:03.696760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.737942+0000","last_change":"2023-11-08T19:26:06.541360+0000","last_active":"2023-11-08T19:26:25.737942+0000","last_peered":"2023-11-08T19:26:25.737942+0000","last_clean":"2023-11-08T19:26:25.737942+0000","last_became_active":"2023-11-08T19:26:04.603725+0000","last_became_peered":"2023-11-08T19:26:04.603725+0000","last_unstale":"2023-11-08T19:26:25.737942+0000","last_undegraded":"2023-11-08T19:26:25.737942+0000","last_fullsized":"2023-11-08T19:26:25.737942+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:28:48.576742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053365400000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.762526+0000","last_change":"2023-11-08T19:26:25.736017+0000","last_active":"2023-11-08T19:26:26.762526+0000","last_peered":"2023-11-08T19:26:26.762526+0000","last_clean":"2023-11-08T19:26:26.762526+0000","last_became_active":"2023-11-08T19:26:25.735715+0000","last_became_peered":"2023-11-08T19:26:25.735715+0000","last_unstale":"2023-11-08T19:26:26.762526+0000","last_undegraded":"2023-11-08T19:26:26.762526+0000","last_fullsized":"2023-11-08T19:26:26.762526+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:56:44.858396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.762649+0000","last_change":"2023-11-08T19:26:24.741527+0000","last_active":"2023-11-08T19:26:26.762649+0000","last_peered":"2023-11-08T19:26:26.762649+0000","last_clean":"2023-11-08T19:26:26.762649+0000","last_became_active":"2023-11-08T19:26:24.741178+0000","last_became_peered":"2023-11-08T19:26:24.741178+0000","last_unstale":"2023-11-08T19:26:26.762649+0000","last_undegraded":"2023-11-08T19:26:26.762649+0000","last_fullsized":"2023-11-08T19:26:26.762649+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:06:06.110808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":28,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.728501+0000","last_change":"2023-11-08T19:26:05.609824+0000","last_active":"2023-11-08T19:26:25.728501+0000","last_peered":"2023-11-08T19:26:25.728501+0000","last_clean":"2023-11-08T19:26:25.728501+0000","last_became_active":"2023-11-08T19:26:04.603320+0000","last_became_peered":"2023-11-08T19:26:04.603320+0000","last_unstale":"2023-11-08T19:26:25.728501+0000","last_undegraded":"2023-11-08T19:26:25.728501+0000","last_fullsized":"2023-11-08T19:26:25.728501+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:05:45.976615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063225,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.730644+0000","last_change":"2023-11-08T19:26:25.746887+0000","last_active":"2023-11-08T19:26:26.730644+0000","last_peered":"2023-11-08T19:26:26.730644+0000","last_clean":"2023-11-08T19:26:26.730644+0000","last_became_active":"2023-11-08T19:26:25.746588+0000","last_became_peered":"2023-11-08T19:26:25.746588+0000","last_unstale":"2023-11-08T19:26:26.730644+0000","last_undegraded":"2023-11-08T19:26:26.730644+0000","last_fullsized":"2023-11-08T19:26:26.730644+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:23:47.505101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.728774+0000","last_change":"2023-11-08T19:26:24.742178+0000","last_active":"2023-11-08T19:26:25.728774+0000","last_peered":"2023-11-08T19:26:25.728774+0000","last_clean":"2023-11-08T19:26:25.728774+0000","last_became_active":"2023-11-08T19:26:24.741868+0000","last_became_peered":"2023-11-08T19:26:24.741868+0000","last_unstale":"2023-11-08T19:26:25.728774+0000","last_undegraded":"2023-11-08T19:26:25.728774+0000","last_fullsized":"2023-11-08T19:26:25.728774+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:35:23.618441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.732053+0000","last_change":"2023-11-08T19:26:06.296937+0000","last_active":"2023-11-08T19:26:25.732053+0000","last_peered":"2023-11-08T19:26:25.732053+0000","last_clean":"2023-11-08T19:26:25.732053+0000","last_became_active":"2023-11-08T19:26:04.599999+0000","last_became_peered":"2023-11-08T19:26:04.599999+0000","last_unstale":"2023-11-08T19:26:25.732053+0000","last_undegraded":"2023-11-08T19:26:25.732053+0000","last_fullsized":"2023-11-08T19:26:25.732053+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:50:43.827570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025605700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.761976+0000","last_change":"2023-11-08T19:26:25.753296+0000","last_active":"2023-11-08T19:26:26.761976+0000","last_peered":"2023-11-08T19:26:26.761976+0000","last_clean":"2023-11-08T19:26:26.761976+0000","last_became_active":"2023-11-08T19:26:25.752989+0000","last_became_peered":"2023-11-08T19:26:25.752989+0000","last_unstale":"2023-11-08T19:26:26.761976+0000","last_undegraded":"2023-11-08T19:26:26.761976+0000","last_fullsized":"2023-11-08T19:26:26.761976+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:23:33.302754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"47'2","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.747075+0000","last_change":"2023-11-08T19:26:24.741588+0000","last_active":"2023-11-08T19:26:25.747075+0000","last_peered":"2023-11-08T19:26:25.747075+0000","last_clean":"2023-11-08T19:26:25.747075+0000","last_became_active":"2023-11-08T19:26:24.741371+0000","last_became_peered":"2023-11-08T19:26:24.741371+0000","last_unstale":"2023-11-08T19:26:25.747075+0000","last_undegraded":"2023-11-08T19:26:25.747075+0000","last_fullsized":"2023-11-08T19:26:25.747075+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:51:03.961763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.728463+0000","last_change":"2023-11-08T19:26:06.424511+0000","last_active":"2023-11-08T19:26:25.728463+0000","last_peered":"2023-11-08T19:26:25.728463+0000","last_clean":"2023-11-08T19:26:25.728463+0000","last_became_active":"2023-11-08T19:26:04.598973+0000","last_became_peered":"2023-11-08T19:26:04.598973+0000","last_unstale":"2023-11-08T19:26:25.728463+0000","last_undegraded":"2023-11-08T19:26:25.728463+0000","last_fullsized":"2023-11-08T19:26:25.728463+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:41:53.497456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031755900000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.762212+0000","last_change":"2023-11-08T19:26:25.742775+0000","last_active":"2023-11-08T19:26:26.762212+0000","last_peered":"2023-11-08T19:26:26.762212+0000","last_clean":"2023-11-08T19:26:26.762212+0000","last_became_active":"2023-11-08T19:26:25.740742+0000","last_became_peered":"2023-11-08T19:26:25.740742+0000","last_unstale":"2023-11-08T19:26:26.762212+0000","last_undegraded":"2023-11-08T19:26:26.762212+0000","last_fullsized":"2023-11-08T19:26:26.762212+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:19:35.879794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.761883+0000","last_change":"2023-11-08T19:26:24.736978+0000","last_active":"2023-11-08T19:26:26.761883+0000","last_peered":"2023-11-08T19:26:26.761883+0000","last_clean":"2023-11-08T19:26:26.761883+0000","last_became_active":"2023-11-08T19:26:24.736651+0000","last_became_peered":"2023-11-08T19:26:24.736651+0000","last_unstale":"2023-11-08T19:26:26.761883+0000","last_undegraded":"2023-11-08T19:26:26.761883+0000","last_fullsized":"2023-11-08T19:26:26.761883+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:22:25.145083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.731536+0000","last_change":"2023-11-08T19:26:06.296634+0000","last_active":"2023-11-08T19:26:25.731536+0000","last_peered":"2023-11-08T19:26:25.731536+0000","last_clean":"2023-11-08T19:26:25.731536+0000","last_became_active":"2023-11-08T19:26:04.603088+0000","last_became_peered":"2023-11-08T19:26:04.603088+0000","last_unstale":"2023-11-08T19:26:25.731536+0000","last_undegraded":"2023-11-08T19:26:25.731536+0000","last_fullsized":"2023-11-08T19:26:25.731536+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:01:56.991346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038180899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.762568+0000","last_change":"2023-11-08T19:26:25.749860+0000","last_active":"2023-11-08T19:26:26.762568+0000","last_peered":"2023-11-08T19:26:26.762568+0000","last_clean":"2023-11-08T19:26:26.762568+0000","last_became_active":"2023-11-08T19:26:25.749612+0000","last_became_peered":"2023-11-08T19:26:25.749612+0000","last_unstale":"2023-11-08T19:26:26.762568+0000","last_undegraded":"2023-11-08T19:26:26.762568+0000","last_fullsized":"2023-11-08T19:26:26.762568+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:48:09.695836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.761578+0000","last_change":"2023-11-08T19:26:24.742441+0000","last_active":"2023-11-08T19:26:26.761578+0000","last_peered":"2023-11-08T19:26:26.761578+0000","last_clean":"2023-11-08T19:26:26.761578+0000","last_became_active":"2023-11-08T19:26:24.742069+0000","last_became_peered":"2023-11-08T19:26:24.742069+0000","last_unstale":"2023-11-08T19:26:26.761578+0000","last_undegraded":"2023-11-08T19:26:26.761578+0000","last_fullsized":"2023-11-08T19:26:26.761578+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:35:22.441191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.734004+0000","last_change":"2023-11-08T19:26:06.394300+0000","last_active":"2023-11-08T19:26:25.734004+0000","last_peered":"2023-11-08T19:26:25.734004+0000","last_clean":"2023-11-08T19:26:25.734004+0000","last_became_active":"2023-11-08T19:26:04.601685+0000","last_became_peered":"2023-11-08T19:26:04.601685+0000","last_unstale":"2023-11-08T19:26:25.734004+0000","last_undegraded":"2023-11-08T19:26:25.734004+0000","last_fullsized":"2023-11-08T19:26:25.734004+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:51:06.626474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061044099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.761844+0000","last_change":"2023-11-08T19:26:25.744017+0000","last_active":"2023-11-08T19:26:26.761844+0000","last_peered":"2023-11-08T19:26:26.761844+0000","last_clean":"2023-11-08T19:26:26.761844+0000","last_became_active":"2023-11-08T19:26:25.743747+0000","last_became_peered":"2023-11-08T19:26:25.743747+0000","last_unstale":"2023-11-08T19:26:26.761844+0000","last_undegraded":"2023-11-08T19:26:26.761844+0000","last_fullsized":"2023-11-08T19:26:26.761844+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:11:18.036694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.761455+0000","last_change":"2023-11-08T19:26:24.743608+0000","last_active":"2023-11-08T19:26:26.761455+0000","last_peered":"2023-11-08T19:26:26.761455+0000","last_clean":"2023-11-08T19:26:26.761455+0000","last_became_active":"2023-11-08T19:26:24.742838+0000","last_became_peered":"2023-11-08T19:26:24.742838+0000","last_unstale":"2023-11-08T19:26:26.761455+0000","last_undegraded":"2023-11-08T19:26:26.761455+0000","last_fullsized":"2023-11-08T19:26:26.761455+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:25:16.496965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":28,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.728055+0000","last_change":"2023-11-08T19:26:05.609570+0000","last_active":"2023-11-08T19:26:25.728055+0000","last_peered":"2023-11-08T19:26:25.728055+0000","last_clean":"2023-11-08T19:26:25.728055+0000","last_became_active":"2023-11-08T19:26:04.608379+0000","last_became_peered":"2023-11-08T19:26:04.608379+0000","last_unstale":"2023-11-08T19:26:25.728055+0000","last_undegraded":"2023-11-08T19:26:25.728055+0000","last_fullsized":"2023-11-08T19:26:25.728055+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:40:08.986649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000529657,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.762778+0000","last_change":"2023-11-08T19:26:25.740002+0000","last_active":"2023-11-08T19:26:26.762778+0000","last_peered":"2023-11-08T19:26:26.762778+0000","last_clean":"2023-11-08T19:26:26.762778+0000","last_became_active":"2023-11-08T19:26:25.739710+0000","last_became_peered":"2023-11-08T19:26:25.739710+0000","last_unstale":"2023-11-08T19:26:26.762778+0000","last_undegraded":"2023-11-08T19:26:26.762778+0000","last_fullsized":"2023-11-08T19:26:26.762778+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:38:44.411483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.762711+0000","last_change":"2023-11-08T19:26:24.742104+0000","last_active":"2023-11-08T19:26:26.762711+0000","last_peered":"2023-11-08T19:26:26.762711+0000","last_clean":"2023-11-08T19:26:26.762711+0000","last_became_active":"2023-11-08T19:26:24.741846+0000","last_became_peered":"2023-11-08T19:26:24.741846+0000","last_unstale":"2023-11-08T19:26:26.762711+0000","last_undegraded":"2023-11-08T19:26:26.762711+0000","last_fullsized":"2023-11-08T19:26:26.762711+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:18:25.803994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.733125+0000","last_change":"2023-11-08T19:26:06.265419+0000","last_active":"2023-11-08T19:26:25.733125+0000","last_peered":"2023-11-08T19:26:25.733125+0000","last_clean":"2023-11-08T19:26:25.733125+0000","last_became_active":"2023-11-08T19:26:04.605333+0000","last_became_peered":"2023-11-08T19:26:04.605333+0000","last_unstale":"2023-11-08T19:26:25.733125+0000","last_undegraded":"2023-11-08T19:26:25.733125+0000","last_fullsized":"2023-11-08T19:26:25.733125+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:01:56.991346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064606400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.729672+0000","last_change":"2023-11-08T19:26:25.745842+0000","last_active":"2023-11-08T19:26:26.729672+0000","last_peered":"2023-11-08T19:26:26.729672+0000","last_clean":"2023-11-08T19:26:26.729672+0000","last_became_active":"2023-11-08T19:26:25.745304+0000","last_became_peered":"2023-11-08T19:26:25.745304+0000","last_unstale":"2023-11-08T19:26:26.729672+0000","last_undegraded":"2023-11-08T19:26:26.729672+0000","last_fullsized":"2023-11-08T19:26:26.729672+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:40:30.126348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.762726+0000","last_change":"2023-11-08T19:26:24.740671+0000","last_active":"2023-11-08T19:26:26.762726+0000","last_peered":"2023-11-08T19:26:26.762726+0000","last_clean":"2023-11-08T19:26:26.762726+0000","last_became_active":"2023-11-08T19:26:24.740395+0000","last_became_peered":"2023-11-08T19:26:24.740395+0000","last_unstale":"2023-11-08T19:26:26.762726+0000","last_undegraded":"2023-11-08T19:26:26.762726+0000","last_fullsized":"2023-11-08T19:26:26.762726+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:09:24.710916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.739334+0000","last_change":"2023-11-08T19:26:06.542044+0000","last_active":"2023-11-08T19:26:25.739334+0000","last_peered":"2023-11-08T19:26:25.739334+0000","last_clean":"2023-11-08T19:26:25.739334+0000","last_became_active":"2023-11-08T19:26:04.603056+0000","last_became_peered":"2023-11-08T19:26:04.603056+0000","last_unstale":"2023-11-08T19:26:25.739334+0000","last_undegraded":"2023-11-08T19:26:25.739334+0000","last_fullsized":"2023-11-08T19:26:25.739334+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:41:50.771457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000517414,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":193,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.731732+0000","last_change":"2023-11-08T19:24:45.888122+0000","last_active":"2023-11-08T19:26:25.731732+0000","last_peered":"2023-11-08T19:26:25.731732+0000","last_clean":"2023-11-08T19:26:25.731732+0000","last_became_active":"2023-11-08T19:24:45.887048+0000","last_became_peered":"2023-11-08T19:24:45.887048+0000","last_unstale":"2023-11-08T19:26:25.731732+0000","last_undegraded":"2023-11-08T19:26:25.731732+0000","last_fullsized":"2023-11-08T19:26:25.731732+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:44.575133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:44.575133+0000","last_clean_scrub_stamp":"2023-11-08T19:24:44.575133+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:19:20.202062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574873,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28160,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343680,"statfs":{"total":96636764160,"available":96607928320,"internally_reserved":0,"allocated":1167360,"data_stored":829790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.61699999999999999}]},{"osd":1,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.63},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.64500000000000002}]},{"osd":2,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.68300000000000005}]},{"osd":3,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.502},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.73499999999999999}]},{"osd":4,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.44}]},{"osd":5,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.67000000000000004}]},{"osd":6,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.59899999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574873,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27588,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344252,"statfs":{"total":96636764160,"available":96608514048,"internally_reserved":0,"allocated":573440,"data_stored":239950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":21.359999999999999,"5min":21.359999999999999,"15min":21.359999999999999},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":10.329000000000001,"5min":10.329000000000001,"15min":10.329000000000001},"last":0.42799999999999999}]},{"osd":1,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":3.4239999999999999,"5min":3.4239999999999999,"15min":3.4239999999999999},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":3.4809999999999999,"5min":3.4809999999999999,"15min":3.4809999999999999},"last":0.45200000000000001}]},{"osd":2,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":10.398,"5min":10.398,"15min":10.398},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":5.0170000000000003,"5min":5.0170000000000003,"15min":5.0170000000000003},"last":0.374}]},{"osd":3,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":3.5710000000000002,"5min":3.5710000000000002,"15min":3.5710000000000002},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":10.368,"5min":10.368,"15min":10.368},"last":0.57099999999999995}]},{"osd":4,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.39200000000000002}]},{"osd":5,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.441},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.59099999999999997}]},{"osd":7,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.498}]}]},{"osd":1,"up_from":13,"seq":55834574873,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27588,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344252,"statfs":{"total":96636764160,"available":96608514048,"internally_reserved":0,"allocated":573440,"data_stored":239458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":1.389,"5min":1.389,"15min":1.389},"last":0.26800000000000002},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":9.1370000000000005,"5min":9.1370000000000005,"15min":9.1370000000000005},"last":0.32900000000000001}]},{"osd":2,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.28499999999999998},{"interface":"front","average":{"1min":0.433,"5min":0.433,"15min":0.433},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":1.425,"5min":1.425,"15min":1.425},"last":0.38700000000000001}]},{"osd":3,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":11.957000000000001,"5min":11.957000000000001,"15min":11.957000000000001},"last":0.30299999999999999},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":18.039000000000001,"5min":18.039000000000001,"15min":18.039000000000001},"last":0.36699999999999999}]},{"osd":4,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":6.181,"5min":6.181,"15min":6.181},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.488,"5min":1.488,"15min":1.488},"last":0.61299999999999999}]},{"osd":5,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":9.2420000000000009,"5min":9.2420000000000009,"15min":9.2420000000000009},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":6.2110000000000003,"5min":6.2110000000000003,"15min":6.2110000000000003},"last":0.63400000000000001}]},{"osd":6,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":9.1880000000000006,"5min":9.1880000000000006,"15min":9.1880000000000006},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.462,"5min":1.462,"15min":1.462},"last":0.65000000000000002}]},{"osd":7,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":6.1479999999999997,"5min":6.1479999999999997,"15min":6.1479999999999997},"last":0.54600000000000004},{"interface":"front","average":{"1min":1.028,"5min":1.028,"15min":1.028},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":9.2170000000000005,"5min":9.2170000000000005,"15min":9.2170000000000005},"last":0.58299999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574873,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28160,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343680,"statfs":{"total":96636764160,"available":96607928320,"internally_reserved":0,"allocated":1167360,"data_stored":829722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":4.1200000000000001,"5min":4.1200000000000001,"15min":4.1200000000000001},"last":0.39800000000000002},{"interface":"front","average":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":16.940999999999999,"5min":16.940999999999999,"15min":16.940999999999999},"last":0.439}]},{"osd":2,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":7.0209999999999999,"5min":7.0209999999999999,"15min":7.0209999999999999},"last":0.41099999999999998}]},{"osd":3,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":4.0940000000000003,"5min":4.0940000000000003,"15min":4.0940000000000003},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.379}]},{"osd":4,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":4.0049999999999999,"5min":4.0049999999999999,"15min":4.0049999999999999},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.49199999999999999}]},{"osd":5,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":4.3090000000000002,"5min":4.3090000000000002,"15min":4.3090000000000002},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":4.0709999999999997,"5min":4.0709999999999997,"15min":4.0709999999999997},"last":0.505}]},{"osd":6,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":4.2560000000000002,"5min":4.2560000000000002,"15min":4.2560000000000002},"last":0.60499999999999998}]},{"osd":7,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":4.2850000000000001,"5min":4.2850000000000001,"15min":4.2850000000000001},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":4.0460000000000003,"5min":4.0460000000000003,"15min":4.0460000000000003},"last":0.47199999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574873,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27592,"kb_used_data":564,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344248,"statfs":{"total":96636764160,"available":96608509952,"internally_reserved":0,"allocated":577536,"data_stored":240910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"min":{"1min":0.20100000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"last":0.27700000000000002},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":6.0999999999999996,"5min":6.0999999999999996,"15min":6.0999999999999996},"last":0.29299999999999998}]},{"osd":1,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"last":0.23999999999999999},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.624,"5min":0.624,"15min":0.624},"last":0.34000000000000002}]},{"osd":3,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"last":0.31},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.32100000000000001}]},{"osd":4,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.40999999999999998}]},{"osd":5,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.51800000000000002}]},{"osd":6,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.40300000000000002}]},{"osd":7,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.38400000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574874,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27596,"kb_used_data":568,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344244,"statfs":{"total":96636764160,"available":96608505856,"internally_reserved":0,"allocated":581632,"data_stored":240932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:25:46 2023","interfaces":[{"interface":"back","average":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.626,"5min":0.626,"15min":0.626},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.36399999999999999}]},{"osd":1,"last update":"Wed Nov 8 19:25:46 2023","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.39400000000000002}]},{"osd":2,"last update":"Wed Nov 8 19:25:46 2023","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.37,"15min":0.37},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"last":0.50700000000000001}]},{"osd":4,"last update":"Wed Nov 8 19:25:46 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.79000000000000004}]},{"osd":5,"last update":"Wed Nov 8 19:25:46 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.81999999999999995}]},{"osd":6,"last update":"Wed Nov 8 19:25:46 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.66600000000000004}]},{"osd":7,"last update":"Wed Nov 8 19:25:46 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.65100000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574873,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27584,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344256,"statfs":{"total":96636764160,"available":96608518144,"internally_reserved":0,"allocated":569344,"data_stored":239880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:25:47 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":6.5970000000000004,"5min":6.5970000000000004,"15min":6.5970000000000004},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":2.5030000000000001,"5min":2.5030000000000001,"15min":2.5030000000000001},"last":0.56000000000000005}]},{"osd":1,"last update":"Wed Nov 8 19:25:47 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.66300000000000003}]},{"osd":2,"last update":"Wed Nov 8 19:25:47 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":6.6920000000000002,"5min":6.6920000000000002,"15min":6.6920000000000002},"last":0.53900000000000003}]},{"osd":3,"last update":"Wed Nov 8 19:25:47 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.68200000000000005}]},{"osd":5,"last update":"Wed Nov 8 19:25:47 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.46400000000000002}]},{"osd":6,"last update":"Wed Nov 8 19:25:47 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.63}]},{"osd":7,"last update":"Wed Nov 8 19:25:47 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.48999999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574873,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27580,"kb_used_data":552,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344260,"statfs":{"total":96636764160,"available":96608522240,"internally_reserved":0,"allocated":565248,"data_stored":239298,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.56599999999999995}]},{"osd":1,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.57999999999999996}]},{"osd":2,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.53600000000000003}]},{"osd":3,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.59499999999999997}]},{"osd":4,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.61199999999999999}]},{"osd":6,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.50700000000000001}]},{"osd":7,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.374},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.48199999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-08T19:26:31.709 INFO:tasks.ceph.mds.c.smithi143.stderr:2023-11-08T19:26:31.708+0000 7f38b102d700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-08T19:26:34.416 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-08T19:26:34.550 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.549+0000 7f1b8f297700 1 Processor -- start 2023-11-08T19:26:34.550 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.549+0000 7f1b8f297700 1 -- start start 2023-11-08T19:26:34.553 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.550+0000 7f1b8f297700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1b8815dd00 0x7f1b880cca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:34.553 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.550+0000 7f1b8f297700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f1b881618f0 0x7f1b8806ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:34.553 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.550+0000 7f1b8f297700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b880ccfb0 con 0x7f1b8806b150 2023-11-08T19:26:34.554 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.550+0000 7f1b8f297700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b8806b600 con 0x7f1b881618f0 2023-11-08T19:26:34.554 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.550+0000 7f1b8f297700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b880cad50 con 0x7f1b8815dd00 2023-11-08T19:26:34.554 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.550+0000 7f1b8c832700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f1b881618f0 0x7f1b8806ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:34.554 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.550+0000 7f1b8d033700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1b8815dd00 0x7f1b880cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:34.554 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.550+0000 7f1b8d834700 1 --1- >> v1:172.21.15.174:6789/0 conn(0x7f1b8806b150 0x7f1b880b2210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.174:6789/0 says I am v1:172.21.15.143:58808/0 (socket says 172.21.15.143:58808) 2023-11-08T19:26:34.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.550+0000 7f1b8c832700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f1b881618f0 0x7f1b8806ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.143:35214/0 (socket says 172.21.15.143:35214) 2023-11-08T19:26:34.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.550+0000 7f1b8d834700 1 -- 172.21.15.143:0/731261284 learned_addr learned my addr 172.21.15.143:0/731261284 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:26:34.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.550+0000 7f1b8d033700 1 -- 172.21.15.143:0/731261284 >> v2:172.21.15.174:3300/0 conn(0x7f1b881618f0 msgr2=0x7f1b8806ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:34.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.550+0000 7f1b8d033700 1 --2- 172.21.15.143:0/731261284 >> v2:172.21.15.174:3300/0 conn(0x7f1b881618f0 0x7f1b8806ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:34.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.550+0000 7f1b8d033700 1 -- 172.21.15.143:0/731261284 >> v1:172.21.15.174:6789/0 conn(0x7f1b8806b150 legacy=0x7f1b880b2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:34.556 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.550+0000 7f1b8d033700 1 -- 172.21.15.143:0/731261284 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b88161df0 con 0x7f1b8815dd00 2023-11-08T19:26:34.556 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.550+0000 7f1b8c832700 1 --2- 172.21.15.143:0/731261284 >> v2:172.21.15.174:3300/0 conn(0x7f1b881618f0 0x7f1b8806ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T19:26:34.556 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.551+0000 7f1b8d033700 1 --2- 172.21.15.143:0/731261284 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1b8815dd00 0x7f1b880cca70 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1b84009820 tx=0x7f1b84009b30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d3b597711f49df server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:34.556 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.551+0000 7f1b7ffff700 1 -- 172.21.15.143:0/731261284 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b84005e30 con 0x7f1b8815dd00 2023-11-08T19:26:34.556 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.551+0000 7f1b7ffff700 1 -- 172.21.15.143:0/731261284 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b84006030 con 0x7f1b8815dd00 2023-11-08T19:26:34.557 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.551+0000 7f1b7ffff700 1 -- 172.21.15.143:0/731261284 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b840063c0 con 0x7f1b8815dd00 2023-11-08T19:26:34.557 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.551+0000 7f1b8f297700 1 -- 172.21.15.143:0/731261284 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1b8815dd00 msgr2=0x7f1b880cca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:34.557 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.551+0000 7f1b8f297700 1 --2- 172.21.15.143:0/731261284 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1b8815dd00 0x7f1b880cca70 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1b84009820 tx=0x7f1b84009b30 comp rx=0 tx=0).stop 2023-11-08T19:26:34.557 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.551+0000 7f1b8f297700 1 -- 172.21.15.143:0/731261284 shutdown_connections 2023-11-08T19:26:34.557 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.551+0000 7f1b8f297700 1 --2- 172.21.15.143:0/731261284 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1b8815dd00 0x7f1b880cca70 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:34.558 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.551+0000 7f1b8f297700 1 --2- 172.21.15.143:0/731261284 >> v2:172.21.15.174:3300/0 conn(0x7f1b881618f0 0x7f1b8806ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:34.558 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.551+0000 7f1b8f297700 1 -- 172.21.15.143:0/731261284 >> 172.21.15.143:0/731261284 conn(0x7f1b88099ab0 msgr2=0x7f1b880d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:34.558 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.552+0000 7f1b8f297700 1 -- 172.21.15.143:0/731261284 shutdown_connections 2023-11-08T19:26:34.558 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.552+0000 7f1b8f297700 1 -- 172.21.15.143:0/731261284 wait complete. 2023-11-08T19:26:34.561 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.553+0000 7f1b8f297700 1 Processor -- start 2023-11-08T19:26:34.561 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.553+0000 7f1b8f297700 1 -- start start 2023-11-08T19:26:34.562 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.553+0000 7f1b8f297700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1b8806b150 0x7f1b881f7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:34.562 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.553+0000 7f1b8f297700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1b8815dd00 0x7f1b880b4400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:34.562 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.553+0000 7f1b8f297700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1b881618f0 0x7f1b88152cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:34.562 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.553+0000 7f1b8f297700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b880b4940 con 0x7f1b881618f0 2023-11-08T19:26:34.562 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.553+0000 7f1b8f297700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b880b4b50 con 0x7f1b8815dd00 2023-11-08T19:26:34.563 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.553+0000 7f1b8f297700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b880b4e30 con 0x7f1b8806b150 2023-11-08T19:26:34.563 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.554+0000 7f1b8d834700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1b881618f0 0x7f1b88152cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:34.563 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.554+0000 7f1b8d834700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1b881618f0 0x7f1b88152cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51726/0 (socket says 172.21.15.143:51726) 2023-11-08T19:26:34.563 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.554+0000 7f1b8d834700 1 -- 172.21.15.143:0/3852403459 learned_addr learned my addr 172.21.15.143:0/3852403459 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:34.563 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.554+0000 7f1b8c832700 1 --2- 172.21.15.143:0/3852403459 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1b8815dd00 0x7f1b880b4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:34.564 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.554+0000 7f1b8d033700 1 --2- 172.21.15.143:0/3852403459 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1b8806b150 0x7f1b881f7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:34.564 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.554+0000 7f1b8d834700 1 -- 172.21.15.143:0/3852403459 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1b8806b150 msgr2=0x7f1b881f7870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:34.564 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.554+0000 7f1b8d834700 1 --2- 172.21.15.143:0/3852403459 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1b8806b150 0x7f1b881f7870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:34.564 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.554+0000 7f1b8d834700 1 -- 172.21.15.143:0/3852403459 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1b8815dd00 msgr2=0x7f1b880b4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:34.564 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.554+0000 7f1b8d834700 1 --2- 172.21.15.143:0/3852403459 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1b8815dd00 0x7f1b880b4400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:34.564 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.554+0000 7f1b8d834700 1 -- 172.21.15.143:0/3852403459 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b780094a0 con 0x7f1b881618f0 2023-11-08T19:26:34.565 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.554+0000 7f1b8c832700 1 --2- 172.21.15.143:0/3852403459 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1b8815dd00 0x7f1b880b4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:26:34.565 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.554+0000 7f1b8d834700 1 --2- 172.21.15.143:0/3852403459 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1b881618f0 0x7f1b88152cc0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f1b8000cbc0 tx=0x7f1b8000ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:34.565 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.554+0000 7f1b7dffb700 1 -- 172.21.15.143:0/3852403459 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b8000aaa0 con 0x7f1b881618f0 2023-11-08T19:26:34.565 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.554+0000 7f1b7dffb700 1 -- 172.21.15.143:0/3852403459 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b8000ac00 con 0x7f1b881618f0 2023-11-08T19:26:34.565 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.554+0000 7f1b8f297700 1 -- 172.21.15.143:0/3852403459 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b84009430 con 0x7f1b881618f0 2023-11-08T19:26:34.566 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.554+0000 7f1b7dffb700 1 -- 172.21.15.143:0/3852403459 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b80012400 con 0x7f1b881618f0 2023-11-08T19:26:34.566 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.555+0000 7f1b8f297700 1 -- 172.21.15.143:0/3852403459 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b880b53f0 con 0x7f1b881618f0 2023-11-08T19:26:34.566 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.555+0000 7f1b8f297700 1 -- 172.21.15.143:0/3852403459 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b880aff60 con 0x7f1b881618f0 2023-11-08T19:26:34.566 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.556+0000 7f1b7dffb700 1 -- 172.21.15.143:0/3852403459 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1b8001b3b0 con 0x7f1b881618f0 2023-11-08T19:26:34.566 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.556+0000 7f1b7dffb700 1 --2- 172.21.15.143:0/3852403459 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f1b6c06af00 0x7f1b6c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:34.567 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.556+0000 7f1b7dffb700 1 -- 172.21.15.143:0/3852403459 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1b800892d0 con 0x7f1b881618f0 2023-11-08T19:26:34.567 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.556+0000 7f1b8d033700 1 --2- 172.21.15.143:0/3852403459 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f1b6c06af00 0x7f1b6c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:34.567 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.557+0000 7f1b8d033700 1 --2- 172.21.15.143:0/3852403459 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f1b6c06af00 0x7f1b6c06d3c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f1b84000c00 tx=0x7f1b84008070 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:34.567 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.561+0000 7f1b7dffb700 1 -- 172.21.15.143:0/3852403459 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1b800542e0 con 0x7f1b881618f0 2023-11-08T19:26:34.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.747+0000 7f1b8f297700 1 -- 172.21.15.143:0/3852403459 --> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1b881623b0 con 0x7f1b6c06af00 2023-11-08T19:26:34.759 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.758+0000 7f1b7dffb700 1 -- 172.21.15.143:0/3852403459 <== mgr.4099 v2:172.21.15.174:6832/4037280178 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+370612 (secure 0 0 0) 0x7f1b84004140 con 0x7f1b6c06af00 2023-11-08T19:26:34.764 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:26:34.767 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.761+0000 7f1b8f297700 1 -- 172.21.15.143:0/3852403459 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f1b6c06af00 msgr2=0x7f1b6c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:34.768 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.761+0000 7f1b8f297700 1 --2- 172.21.15.143:0/3852403459 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f1b6c06af00 0x7f1b6c06d3c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f1b84000c00 tx=0x7f1b84008070 comp rx=0 tx=0).stop 2023-11-08T19:26:34.768 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.761+0000 7f1b8f297700 1 -- 172.21.15.143:0/3852403459 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1b881618f0 msgr2=0x7f1b88152cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:34.768 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.761+0000 7f1b8f297700 1 --2- 172.21.15.143:0/3852403459 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1b881618f0 0x7f1b88152cc0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f1b8000cbc0 tx=0x7f1b8000ced0 comp rx=0 tx=0).stop 2023-11-08T19:26:34.768 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.762+0000 7f1b8f297700 1 -- 172.21.15.143:0/3852403459 shutdown_connections 2023-11-08T19:26:34.769 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.762+0000 7f1b8f297700 1 --2- 172.21.15.143:0/3852403459 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f1b6c06af00 0x7f1b6c06d3c0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:34.769 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.762+0000 7f1b8f297700 1 --2- 172.21.15.143:0/3852403459 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1b8806b150 0x7f1b881f7870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:34.769 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.762+0000 7f1b8f297700 1 --2- 172.21.15.143:0/3852403459 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1b881618f0 0x7f1b88152cc0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:34.769 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.762+0000 7f1b8f297700 1 --2- 172.21.15.143:0/3852403459 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1b8815dd00 0x7f1b880b4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:34.769 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.762+0000 7f1b8f297700 1 -- 172.21.15.143:0/3852403459 >> 172.21.15.143:0/3852403459 conn(0x7f1b88099ab0 msgr2=0x7f1b8806e700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:34.770 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.762+0000 7f1b8f297700 1 -- 172.21.15.143:0/3852403459 shutdown_connections 2023-11-08T19:26:34.770 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.762+0000 7f1b8f297700 1 -- 172.21.15.143:0/3852403459 wait complete. 2023-11-08T19:26:34.770 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2023-11-08T19:26:34.775 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":96,"stamp":"2023-11-08T19:26:34.535713+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":258,"acting":258,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":258,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":221848,"kb_used_data":5640,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754752872,"statfs":{"total":773094113280,"available":772866940928,"internally_reserved":0,"allocated":5775360,"data_stored":3099940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.809757"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.731024+0000","last_change":"2023-11-08T19:26:25.752333+0000","last_active":"2023-11-08T19:26:26.731024+0000","last_peered":"2023-11-08T19:26:26.731024+0000","last_clean":"2023-11-08T19:26:26.731024+0000","last_became_active":"2023-11-08T19:26:25.752055+0000","last_became_peered":"2023-11-08T19:26:25.752055+0000","last_unstale":"2023-11-08T19:26:26.731024+0000","last_undegraded":"2023-11-08T19:26:26.731024+0000","last_fullsized":"2023-11-08T19:26:26.731024+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:29:44.685878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.730541+0000","last_change":"2023-11-08T19:26:25.746072+0000","last_active":"2023-11-08T19:26:26.730541+0000","last_peered":"2023-11-08T19:26:26.730541+0000","last_clean":"2023-11-08T19:26:26.730541+0000","last_became_active":"2023-11-08T19:26:25.745753+0000","last_became_peered":"2023-11-08T19:26:25.745753+0000","last_unstale":"2023-11-08T19:26:26.730541+0000","last_undegraded":"2023-11-08T19:26:26.730541+0000","last_fullsized":"2023-11-08T19:26:26.730541+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:02:15.103886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.762914+0000","last_change":"2023-11-08T19:26:25.746891+0000","last_active":"2023-11-08T19:26:26.762914+0000","last_peered":"2023-11-08T19:26:26.762914+0000","last_clean":"2023-11-08T19:26:26.762914+0000","last_became_active":"2023-11-08T19:26:25.746609+0000","last_became_peered":"2023-11-08T19:26:25.746609+0000","last_unstale":"2023-11-08T19:26:26.762914+0000","last_undegraded":"2023-11-08T19:26:26.762914+0000","last_fullsized":"2023-11-08T19:26:26.762914+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:02:15.103886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.730147+0000","last_change":"2023-11-08T19:26:25.739800+0000","last_active":"2023-11-08T19:26:26.730147+0000","last_peered":"2023-11-08T19:26:26.730147+0000","last_clean":"2023-11-08T19:26:26.730147+0000","last_became_active":"2023-11-08T19:26:25.739529+0000","last_became_peered":"2023-11-08T19:26:25.739529+0000","last_unstale":"2023-11-08T19:26:26.730147+0000","last_undegraded":"2023-11-08T19:26:26.730147+0000","last_fullsized":"2023-11-08T19:26:26.730147+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:48:09.695836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.762381+0000","last_change":"2023-11-08T19:26:25.741966+0000","last_active":"2023-11-08T19:26:26.762381+0000","last_peered":"2023-11-08T19:26:26.762381+0000","last_clean":"2023-11-08T19:26:26.762381+0000","last_became_active":"2023-11-08T19:26:25.741388+0000","last_became_peered":"2023-11-08T19:26:25.741388+0000","last_unstale":"2023-11-08T19:26:26.762381+0000","last_undegraded":"2023-11-08T19:26:26.762381+0000","last_fullsized":"2023-11-08T19:26:26.762381+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:31:35.182260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.730525+0000","last_change":"2023-11-08T19:26:25.757491+0000","last_active":"2023-11-08T19:26:26.730525+0000","last_peered":"2023-11-08T19:26:26.730525+0000","last_clean":"2023-11-08T19:26:26.730525+0000","last_became_active":"2023-11-08T19:26:25.757106+0000","last_became_peered":"2023-11-08T19:26:25.757106+0000","last_unstale":"2023-11-08T19:26:26.730525+0000","last_undegraded":"2023-11-08T19:26:26.730525+0000","last_fullsized":"2023-11-08T19:26:26.730525+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:09:25.716422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.761990+0000","last_change":"2023-11-08T19:26:25.742389+0000","last_active":"2023-11-08T19:26:26.761990+0000","last_peered":"2023-11-08T19:26:26.761990+0000","last_clean":"2023-11-08T19:26:26.761990+0000","last_became_active":"2023-11-08T19:26:25.742128+0000","last_became_peered":"2023-11-08T19:26:25.742128+0000","last_unstale":"2023-11-08T19:26:26.761990+0000","last_undegraded":"2023-11-08T19:26:26.761990+0000","last_fullsized":"2023-11-08T19:26:26.761990+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:42:13.350558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.761951+0000","last_change":"2023-11-08T19:26:25.748084+0000","last_active":"2023-11-08T19:26:26.761951+0000","last_peered":"2023-11-08T19:26:26.761951+0000","last_clean":"2023-11-08T19:26:26.761951+0000","last_became_active":"2023-11-08T19:26:25.747737+0000","last_became_peered":"2023-11-08T19:26:25.747737+0000","last_unstale":"2023-11-08T19:26:26.761951+0000","last_undegraded":"2023-11-08T19:26:26.761951+0000","last_fullsized":"2023-11-08T19:26:26.761951+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:33:05.719333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.762276+0000","last_change":"2023-11-08T19:26:25.747778+0000","last_active":"2023-11-08T19:26:26.762276+0000","last_peered":"2023-11-08T19:26:26.762276+0000","last_clean":"2023-11-08T19:26:26.762276+0000","last_became_active":"2023-11-08T19:26:25.747302+0000","last_became_peered":"2023-11-08T19:26:25.747302+0000","last_unstale":"2023-11-08T19:26:26.762276+0000","last_undegraded":"2023-11-08T19:26:26.762276+0000","last_fullsized":"2023-11-08T19:26:26.762276+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:59:22.071613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.730998+0000","last_change":"2023-11-08T19:26:25.745318+0000","last_active":"2023-11-08T19:26:26.730998+0000","last_peered":"2023-11-08T19:26:26.730998+0000","last_clean":"2023-11-08T19:26:26.730998+0000","last_became_active":"2023-11-08T19:26:25.745040+0000","last_became_peered":"2023-11-08T19:26:25.745040+0000","last_unstale":"2023-11-08T19:26:26.730998+0000","last_undegraded":"2023-11-08T19:26:26.730998+0000","last_fullsized":"2023-11-08T19:26:26.730998+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:29:30.597869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.762525+0000","last_change":"2023-11-08T19:26:25.741866+0000","last_active":"2023-11-08T19:26:26.762525+0000","last_peered":"2023-11-08T19:26:26.762525+0000","last_clean":"2023-11-08T19:26:26.762525+0000","last_became_active":"2023-11-08T19:26:25.741549+0000","last_became_peered":"2023-11-08T19:26:25.741549+0000","last_unstale":"2023-11-08T19:26:26.762525+0000","last_undegraded":"2023-11-08T19:26:26.762525+0000","last_fullsized":"2023-11-08T19:26:26.762525+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:27:46.072811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.762113+0000","last_change":"2023-11-08T19:26:25.738224+0000","last_active":"2023-11-08T19:26:26.762113+0000","last_peered":"2023-11-08T19:26:26.762113+0000","last_clean":"2023-11-08T19:26:26.762113+0000","last_became_active":"2023-11-08T19:26:25.737961+0000","last_became_peered":"2023-11-08T19:26:25.737961+0000","last_unstale":"2023-11-08T19:26:26.762113+0000","last_undegraded":"2023-11-08T19:26:26.762113+0000","last_fullsized":"2023-11-08T19:26:26.762113+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:42:48.346407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.730528+0000","last_change":"2023-11-08T19:26:25.746303+0000","last_active":"2023-11-08T19:26:26.730528+0000","last_peered":"2023-11-08T19:26:26.730528+0000","last_clean":"2023-11-08T19:26:26.730528+0000","last_became_active":"2023-11-08T19:26:25.745987+0000","last_became_peered":"2023-11-08T19:26:25.745987+0000","last_unstale":"2023-11-08T19:26:26.730528+0000","last_undegraded":"2023-11-08T19:26:26.730528+0000","last_fullsized":"2023-11-08T19:26:26.730528+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:35:24.623946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.762451+0000","last_change":"2023-11-08T19:26:25.744431+0000","last_active":"2023-11-08T19:26:26.762451+0000","last_peered":"2023-11-08T19:26:26.762451+0000","last_clean":"2023-11-08T19:26:26.762451+0000","last_became_active":"2023-11-08T19:26:25.742917+0000","last_became_peered":"2023-11-08T19:26:25.742917+0000","last_unstale":"2023-11-08T19:26:26.762451+0000","last_undegraded":"2023-11-08T19:26:26.762451+0000","last_fullsized":"2023-11-08T19:26:26.762451+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:22:57.218646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.730191+0000","last_change":"2023-11-08T19:26:25.737530+0000","last_active":"2023-11-08T19:26:26.730191+0000","last_peered":"2023-11-08T19:26:26.730191+0000","last_clean":"2023-11-08T19:26:26.730191+0000","last_became_active":"2023-11-08T19:26:25.737241+0000","last_became_peered":"2023-11-08T19:26:25.737241+0000","last_unstale":"2023-11-08T19:26:26.730191+0000","last_undegraded":"2023-11-08T19:26:26.730191+0000","last_fullsized":"2023-11-08T19:26:26.730191+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:34:52.582926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.730618+0000","last_change":"2023-11-08T19:26:25.745857+0000","last_active":"2023-11-08T19:26:26.730618+0000","last_peered":"2023-11-08T19:26:26.730618+0000","last_clean":"2023-11-08T19:26:26.730618+0000","last_became_active":"2023-11-08T19:26:25.745594+0000","last_became_peered":"2023-11-08T19:26:25.745594+0000","last_unstale":"2023-11-08T19:26:26.730618+0000","last_undegraded":"2023-11-08T19:26:26.730618+0000","last_fullsized":"2023-11-08T19:26:26.730618+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:56:38.230155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.763193+0000","last_change":"2023-11-08T19:26:25.755356+0000","last_active":"2023-11-08T19:26:26.763193+0000","last_peered":"2023-11-08T19:26:26.763193+0000","last_clean":"2023-11-08T19:26:26.763193+0000","last_became_active":"2023-11-08T19:26:25.755155+0000","last_became_peered":"2023-11-08T19:26:25.755155+0000","last_unstale":"2023-11-08T19:26:26.763193+0000","last_undegraded":"2023-11-08T19:26:26.763193+0000","last_fullsized":"2023-11-08T19:26:26.763193+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:05:31.920008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.730577+0000","last_change":"2023-11-08T19:26:25.743838+0000","last_active":"2023-11-08T19:26:26.730577+0000","last_peered":"2023-11-08T19:26:26.730577+0000","last_clean":"2023-11-08T19:26:26.730577+0000","last_became_active":"2023-11-08T19:26:25.743563+0000","last_became_peered":"2023-11-08T19:26:25.743563+0000","last_unstale":"2023-11-08T19:26:26.730577+0000","last_undegraded":"2023-11-08T19:26:26.730577+0000","last_fullsized":"2023-11-08T19:26:26.730577+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:03:31.037228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.763160+0000","last_change":"2023-11-08T19:26:25.745921+0000","last_active":"2023-11-08T19:26:26.763160+0000","last_peered":"2023-11-08T19:26:26.763160+0000","last_clean":"2023-11-08T19:26:26.763160+0000","last_became_active":"2023-11-08T19:26:25.745437+0000","last_became_peered":"2023-11-08T19:26:25.745437+0000","last_unstale":"2023-11-08T19:26:26.763160+0000","last_undegraded":"2023-11-08T19:26:26.763160+0000","last_fullsized":"2023-11-08T19:26:26.763160+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:25:17.502470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.730619+0000","last_change":"2023-11-08T19:26:25.738981+0000","last_active":"2023-11-08T19:26:26.730619+0000","last_peered":"2023-11-08T19:26:26.730619+0000","last_clean":"2023-11-08T19:26:26.730619+0000","last_became_active":"2023-11-08T19:26:25.738626+0000","last_became_peered":"2023-11-08T19:26:25.738626+0000","last_unstale":"2023-11-08T19:26:26.730619+0000","last_undegraded":"2023-11-08T19:26:26.730619+0000","last_fullsized":"2023-11-08T19:26:26.730619+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:55:03.172600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.730693+0000","last_change":"2023-11-08T19:26:25.737510+0000","last_active":"2023-11-08T19:26:26.730693+0000","last_peered":"2023-11-08T19:26:26.730693+0000","last_clean":"2023-11-08T19:26:26.730693+0000","last_became_active":"2023-11-08T19:26:25.737239+0000","last_became_peered":"2023-11-08T19:26:25.737239+0000","last_unstale":"2023-11-08T19:26:26.730693+0000","last_undegraded":"2023-11-08T19:26:26.730693+0000","last_fullsized":"2023-11-08T19:26:26.730693+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:06:07.116314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.761602+0000","last_change":"2023-11-08T19:26:25.747608+0000","last_active":"2023-11-08T19:26:26.761602+0000","last_peered":"2023-11-08T19:26:26.761602+0000","last_clean":"2023-11-08T19:26:26.761602+0000","last_became_active":"2023-11-08T19:26:25.747337+0000","last_became_peered":"2023-11-08T19:26:25.747337+0000","last_unstale":"2023-11-08T19:26:26.761602+0000","last_undegraded":"2023-11-08T19:26:26.761602+0000","last_fullsized":"2023-11-08T19:26:26.761602+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:44:11.903976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.761525+0000","last_change":"2023-11-08T19:26:25.743572+0000","last_active":"2023-11-08T19:26:26.761525+0000","last_peered":"2023-11-08T19:26:26.761525+0000","last_clean":"2023-11-08T19:26:26.761525+0000","last_became_active":"2023-11-08T19:26:25.743102+0000","last_became_peered":"2023-11-08T19:26:25.743102+0000","last_unstale":"2023-11-08T19:26:26.761525+0000","last_undegraded":"2023-11-08T19:26:26.761525+0000","last_fullsized":"2023-11-08T19:26:26.761525+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:39:12.204565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.761855+0000","last_change":"2023-11-08T19:26:25.745635+0000","last_active":"2023-11-08T19:26:26.761855+0000","last_peered":"2023-11-08T19:26:26.761855+0000","last_clean":"2023-11-08T19:26:26.761855+0000","last_became_active":"2023-11-08T19:26:25.742827+0000","last_became_peered":"2023-11-08T19:26:25.742827+0000","last_unstale":"2023-11-08T19:26:26.761855+0000","last_undegraded":"2023-11-08T19:26:26.761855+0000","last_fullsized":"2023-11-08T19:26:26.761855+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:29:30.597869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.763073+0000","last_change":"2023-11-08T19:26:25.749212+0000","last_active":"2023-11-08T19:26:26.763073+0000","last_peered":"2023-11-08T19:26:26.763073+0000","last_clean":"2023-11-08T19:26:26.763073+0000","last_became_active":"2023-11-08T19:26:25.748823+0000","last_became_peered":"2023-11-08T19:26:25.748823+0000","last_unstale":"2023-11-08T19:26:26.763073+0000","last_undegraded":"2023-11-08T19:26:26.763073+0000","last_fullsized":"2023-11-08T19:26:26.763073+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:18:26.809499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.762362+0000","last_change":"2023-11-08T19:26:25.741532+0000","last_active":"2023-11-08T19:26:26.762362+0000","last_peered":"2023-11-08T19:26:26.762362+0000","last_clean":"2023-11-08T19:26:26.762362+0000","last_became_active":"2023-11-08T19:26:25.741251+0000","last_became_peered":"2023-11-08T19:26:25.741251+0000","last_unstale":"2023-11-08T19:26:26.762362+0000","last_undegraded":"2023-11-08T19:26:26.762362+0000","last_fullsized":"2023-11-08T19:26:26.762362+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:51:48.252035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.730688+0000","last_change":"2023-11-08T19:26:25.758350+0000","last_active":"2023-11-08T19:26:26.730688+0000","last_peered":"2023-11-08T19:26:26.730688+0000","last_clean":"2023-11-08T19:26:26.730688+0000","last_became_active":"2023-11-08T19:26:25.757848+0000","last_became_peered":"2023-11-08T19:26:25.757848+0000","last_unstale":"2023-11-08T19:26:26.730688+0000","last_undegraded":"2023-11-08T19:26:26.730688+0000","last_fullsized":"2023-11-08T19:26:26.730688+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:23:33.302754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.730778+0000","last_change":"2023-11-08T19:26:25.742808+0000","last_active":"2023-11-08T19:26:26.730778+0000","last_peered":"2023-11-08T19:26:26.730778+0000","last_clean":"2023-11-08T19:26:26.730778+0000","last_became_active":"2023-11-08T19:26:25.742514+0000","last_became_peered":"2023-11-08T19:26:25.742514+0000","last_unstale":"2023-11-08T19:26:26.730778+0000","last_undegraded":"2023-11-08T19:26:26.730778+0000","last_fullsized":"2023-11-08T19:26:26.730778+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:48:11.183540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.762397+0000","last_change":"2023-11-08T19:26:25.744287+0000","last_active":"2023-11-08T19:26:26.762397+0000","last_peered":"2023-11-08T19:26:26.762397+0000","last_clean":"2023-11-08T19:26:26.762397+0000","last_became_active":"2023-11-08T19:26:25.742841+0000","last_became_peered":"2023-11-08T19:26:25.742841+0000","last_unstale":"2023-11-08T19:26:26.762397+0000","last_undegraded":"2023-11-08T19:26:26.762397+0000","last_fullsized":"2023-11-08T19:26:26.762397+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:45:24.697693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.729997+0000","last_change":"2023-11-08T19:26:25.752986+0000","last_active":"2023-11-08T19:26:26.729997+0000","last_peered":"2023-11-08T19:26:26.729997+0000","last_clean":"2023-11-08T19:26:26.729997+0000","last_became_active":"2023-11-08T19:26:25.752647+0000","last_became_peered":"2023-11-08T19:26:25.752647+0000","last_unstale":"2023-11-08T19:26:26.729997+0000","last_undegraded":"2023-11-08T19:26:26.729997+0000","last_fullsized":"2023-11-08T19:26:26.729997+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:22:57.218646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.762957+0000","last_change":"2023-11-08T19:26:25.747032+0000","last_active":"2023-11-08T19:26:26.762957+0000","last_peered":"2023-11-08T19:26:26.762957+0000","last_clean":"2023-11-08T19:26:26.762957+0000","last_became_active":"2023-11-08T19:26:25.746745+0000","last_became_peered":"2023-11-08T19:26:25.746745+0000","last_unstale":"2023-11-08T19:26:26.762957+0000","last_undegraded":"2023-11-08T19:26:26.762957+0000","last_fullsized":"2023-11-08T19:26:26.762957+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:29:44.685878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.761757+0000","last_change":"2023-11-08T19:26:25.753806+0000","last_active":"2023-11-08T19:26:26.761757+0000","last_peered":"2023-11-08T19:26:26.761757+0000","last_clean":"2023-11-08T19:26:26.761757+0000","last_became_active":"2023-11-08T19:26:25.753437+0000","last_became_peered":"2023-11-08T19:26:25.753437+0000","last_unstale":"2023-11-08T19:26:26.761757+0000","last_undegraded":"2023-11-08T19:26:26.761757+0000","last_fullsized":"2023-11-08T19:26:26.761757+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:23:47.505101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.762407+0000","last_change":"2023-11-08T19:26:25.741528+0000","last_active":"2023-11-08T19:26:26.762407+0000","last_peered":"2023-11-08T19:26:26.762407+0000","last_clean":"2023-11-08T19:26:26.762407+0000","last_became_active":"2023-11-08T19:26:25.740687+0000","last_became_peered":"2023-11-08T19:26:25.740687+0000","last_unstale":"2023-11-08T19:26:26.762407+0000","last_undegraded":"2023-11-08T19:26:26.762407+0000","last_fullsized":"2023-11-08T19:26:26.762407+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:02:36.877898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"47'2","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.762067+0000","last_change":"2023-11-08T19:26:24.743581+0000","last_active":"2023-11-08T19:26:26.762067+0000","last_peered":"2023-11-08T19:26:26.762067+0000","last_clean":"2023-11-08T19:26:26.762067+0000","last_became_active":"2023-11-08T19:26:24.743284+0000","last_became_peered":"2023-11-08T19:26:24.743284+0000","last_unstale":"2023-11-08T19:26:26.762067+0000","last_undegraded":"2023-11-08T19:26:26.762067+0000","last_fullsized":"2023-11-08T19:26:26.762067+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:35:23.618441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.739324+0000","last_change":"2023-11-08T19:26:06.297255+0000","last_active":"2023-11-08T19:26:25.739324+0000","last_peered":"2023-11-08T19:26:25.739324+0000","last_clean":"2023-11-08T19:26:25.739324+0000","last_became_active":"2023-11-08T19:26:04.603979+0000","last_became_peered":"2023-11-08T19:26:04.603979+0000","last_unstale":"2023-11-08T19:26:25.739324+0000","last_undegraded":"2023-11-08T19:26:25.739324+0000","last_fullsized":"2023-11-08T19:26:25.739324+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:40:11.164951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079032099999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.730750+0000","last_change":"2023-11-08T19:26:25.740830+0000","last_active":"2023-11-08T19:26:26.730750+0000","last_peered":"2023-11-08T19:26:26.730750+0000","last_clean":"2023-11-08T19:26:26.730750+0000","last_became_active":"2023-11-08T19:26:25.740548+0000","last_became_peered":"2023-11-08T19:26:25.740548+0000","last_unstale":"2023-11-08T19:26:26.730750+0000","last_undegraded":"2023-11-08T19:26:26.730750+0000","last_fullsized":"2023-11-08T19:26:26.730750+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:35:23.446696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"47'2","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.730598+0000","last_change":"2023-11-08T19:26:24.736817+0000","last_active":"2023-11-08T19:26:26.730598+0000","last_peered":"2023-11-08T19:26:26.730598+0000","last_clean":"2023-11-08T19:26:26.730598+0000","last_became_active":"2023-11-08T19:26:24.736538+0000","last_became_peered":"2023-11-08T19:26:24.736538+0000","last_unstale":"2023-11-08T19:26:26.730598+0000","last_undegraded":"2023-11-08T19:26:26.730598+0000","last_fullsized":"2023-11-08T19:26:26.730598+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:22:58.776603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":28,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.729426+0000","last_change":"2023-11-08T19:26:05.609625+0000","last_active":"2023-11-08T19:26:25.729426+0000","last_peered":"2023-11-08T19:26:25.729426+0000","last_clean":"2023-11-08T19:26:25.729426+0000","last_became_active":"2023-11-08T19:26:04.606312+0000","last_became_peered":"2023-11-08T19:26:04.606312+0000","last_unstale":"2023-11-08T19:26:25.729426+0000","last_undegraded":"2023-11-08T19:26:25.729426+0000","last_fullsized":"2023-11-08T19:26:25.729426+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:54:42.032901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000505935,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.762035+0000","last_change":"2023-11-08T19:26:25.745326+0000","last_active":"2023-11-08T19:26:26.762035+0000","last_peered":"2023-11-08T19:26:26.762035+0000","last_clean":"2023-11-08T19:26:26.762035+0000","last_became_active":"2023-11-08T19:26:25.744490+0000","last_became_peered":"2023-11-08T19:26:25.744490+0000","last_unstale":"2023-11-08T19:26:26.762035+0000","last_undegraded":"2023-11-08T19:26:26.762035+0000","last_fullsized":"2023-11-08T19:26:26.762035+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:48:27.754051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"47'1","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.730896+0000","last_change":"2023-11-08T19:26:24.744835+0000","last_active":"2023-11-08T19:26:26.730896+0000","last_peered":"2023-11-08T19:26:26.730896+0000","last_clean":"2023-11-08T19:26:26.730896+0000","last_became_active":"2023-11-08T19:26:24.744487+0000","last_became_peered":"2023-11-08T19:26:24.744487+0000","last_unstale":"2023-11-08T19:26:26.730896+0000","last_undegraded":"2023-11-08T19:26:26.730896+0000","last_fullsized":"2023-11-08T19:26:26.730896+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:13:28.823322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.727850+0000","last_change":"2023-11-08T19:26:06.425133+0000","last_active":"2023-11-08T19:26:25.727850+0000","last_peered":"2023-11-08T19:26:25.727850+0000","last_clean":"2023-11-08T19:26:25.727850+0000","last_became_active":"2023-11-08T19:26:04.601103+0000","last_became_peered":"2023-11-08T19:26:04.601103+0000","last_unstale":"2023-11-08T19:26:25.727850+0000","last_undegraded":"2023-11-08T19:26:25.727850+0000","last_fullsized":"2023-11-08T19:26:25.727850+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:31:14.042561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000229244,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.729685+0000","last_change":"2023-11-08T19:26:25.748005+0000","last_active":"2023-11-08T19:26:26.729685+0000","last_peered":"2023-11-08T19:26:26.729685+0000","last_clean":"2023-11-08T19:26:26.729685+0000","last_became_active":"2023-11-08T19:26:25.747426+0000","last_became_peered":"2023-11-08T19:26:25.747426+0000","last_unstale":"2023-11-08T19:26:26.729685+0000","last_undegraded":"2023-11-08T19:26:26.729685+0000","last_fullsized":"2023-11-08T19:26:26.729685+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:49:29.652891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"47'4","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.748120+0000","last_change":"2023-11-08T19:26:24.738249+0000","last_active":"2023-11-08T19:26:25.748120+0000","last_peered":"2023-11-08T19:26:25.748120+0000","last_clean":"2023-11-08T19:26:25.748120+0000","last_became_active":"2023-11-08T19:26:24.737997+0000","last_became_peered":"2023-11-08T19:26:24.737997+0000","last_unstale":"2023-11-08T19:26:25.748120+0000","last_undegraded":"2023-11-08T19:26:25.748120+0000","last_fullsized":"2023-11-08T19:26:25.748120+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:40:31.299145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.732153+0000","last_change":"2023-11-08T19:26:06.265690+0000","last_active":"2023-11-08T19:26:25.732153+0000","last_peered":"2023-11-08T19:26:25.732153+0000","last_clean":"2023-11-08T19:26:25.732153+0000","last_became_active":"2023-11-08T19:26:04.603397+0000","last_became_peered":"2023-11-08T19:26:04.603397+0000","last_unstale":"2023-11-08T19:26:25.732153+0000","last_undegraded":"2023-11-08T19:26:25.732153+0000","last_fullsized":"2023-11-08T19:26:25.732153+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:50:43.827570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053361499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.730396+0000","last_change":"2023-11-08T19:26:25.743979+0000","last_active":"2023-11-08T19:26:26.730396+0000","last_peered":"2023-11-08T19:26:26.730396+0000","last_clean":"2023-11-08T19:26:26.730396+0000","last_became_active":"2023-11-08T19:26:25.743680+0000","last_became_peered":"2023-11-08T19:26:25.743680+0000","last_unstale":"2023-11-08T19:26:26.730396+0000","last_undegraded":"2023-11-08T19:26:26.730396+0000","last_fullsized":"2023-11-08T19:26:26.730396+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:38:44.411483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.761728+0000","last_change":"2023-11-08T19:26:24.739326+0000","last_active":"2023-11-08T19:26:26.761728+0000","last_peered":"2023-11-08T19:26:26.761728+0000","last_clean":"2023-11-08T19:26:26.761728+0000","last_became_active":"2023-11-08T19:26:24.738986+0000","last_became_peered":"2023-11-08T19:26:24.738986+0000","last_unstale":"2023-11-08T19:26:26.761728+0000","last_undegraded":"2023-11-08T19:26:26.761728+0000","last_fullsized":"2023-11-08T19:26:26.761728+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:23:32.297248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.738268+0000","last_change":"2023-11-08T19:26:06.541338+0000","last_active":"2023-11-08T19:26:25.738268+0000","last_peered":"2023-11-08T19:26:25.738268+0000","last_clean":"2023-11-08T19:26:25.738268+0000","last_became_active":"2023-11-08T19:26:04.602183+0000","last_became_peered":"2023-11-08T19:26:04.602183+0000","last_unstale":"2023-11-08T19:26:25.738268+0000","last_undegraded":"2023-11-08T19:26:25.738268+0000","last_fullsized":"2023-11-08T19:26:25.738268+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:06:47.446686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052029099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.762021+0000","last_change":"2023-11-08T19:26:25.741349+0000","last_active":"2023-11-08T19:26:26.762021+0000","last_peered":"2023-11-08T19:26:26.762021+0000","last_clean":"2023-11-08T19:26:26.762021+0000","last_became_active":"2023-11-08T19:26:25.741052+0000","last_became_peered":"2023-11-08T19:26:25.741052+0000","last_unstale":"2023-11-08T19:26:26.762021+0000","last_undegraded":"2023-11-08T19:26:26.762021+0000","last_fullsized":"2023-11-08T19:26:26.762021+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:49:22.367937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.730162+0000","last_change":"2023-11-08T19:26:24.740255+0000","last_active":"2023-11-08T19:26:25.730162+0000","last_peered":"2023-11-08T19:26:25.730162+0000","last_clean":"2023-11-08T19:26:25.730162+0000","last_became_active":"2023-11-08T19:26:24.739986+0000","last_became_peered":"2023-11-08T19:26:24.739986+0000","last_unstale":"2023-11-08T19:26:25.730162+0000","last_undegraded":"2023-11-08T19:26:25.730162+0000","last_fullsized":"2023-11-08T19:26:25.730162+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:11:46.642828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.734103+0000","last_change":"2023-11-08T19:26:06.394081+0000","last_active":"2023-11-08T19:26:25.734103+0000","last_peered":"2023-11-08T19:26:25.734103+0000","last_clean":"2023-11-08T19:26:25.734103+0000","last_became_active":"2023-11-08T19:26:04.602137+0000","last_became_peered":"2023-11-08T19:26:04.602137+0000","last_unstale":"2023-11-08T19:26:25.734103+0000","last_undegraded":"2023-11-08T19:26:25.734103+0000","last_fullsized":"2023-11-08T19:26:25.734103+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:05:46.951663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056998200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.730862+0000","last_change":"2023-11-08T19:26:25.758675+0000","last_active":"2023-11-08T19:26:26.730862+0000","last_peered":"2023-11-08T19:26:26.730862+0000","last_clean":"2023-11-08T19:26:26.730862+0000","last_became_active":"2023-11-08T19:26:25.758495+0000","last_became_peered":"2023-11-08T19:26:25.758495+0000","last_unstale":"2023-11-08T19:26:26.730862+0000","last_undegraded":"2023-11-08T19:26:26.730862+0000","last_fullsized":"2023-11-08T19:26:26.730862+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:18:26.809499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"47'1","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.762131+0000","last_change":"2023-11-08T19:26:24.738043+0000","last_active":"2023-11-08T19:26:26.762131+0000","last_peered":"2023-11-08T19:26:26.762131+0000","last_clean":"2023-11-08T19:26:26.762131+0000","last_became_active":"2023-11-08T19:26:24.737782+0000","last_became_peered":"2023-11-08T19:26:24.737782+0000","last_unstale":"2023-11-08T19:26:26.762131+0000","last_undegraded":"2023-11-08T19:26:26.762131+0000","last_fullsized":"2023-11-08T19:26:26.762131+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:18:22.010798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.728229+0000","last_change":"2023-11-08T19:26:06.423863+0000","last_active":"2023-11-08T19:26:25.728229+0000","last_peered":"2023-11-08T19:26:25.728229+0000","last_clean":"2023-11-08T19:26:25.728229+0000","last_became_active":"2023-11-08T19:26:04.608639+0000","last_became_peered":"2023-11-08T19:26:04.608639+0000","last_unstale":"2023-11-08T19:26:25.728229+0000","last_undegraded":"2023-11-08T19:26:25.728229+0000","last_fullsized":"2023-11-08T19:26:25.728229+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:04:43.631102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044384999999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.762598+0000","last_change":"2023-11-08T19:26:25.746985+0000","last_active":"2023-11-08T19:26:26.762598+0000","last_peered":"2023-11-08T19:26:26.762598+0000","last_clean":"2023-11-08T19:26:26.762598+0000","last_became_active":"2023-11-08T19:26:25.746718+0000","last_became_peered":"2023-11-08T19:26:25.746718+0000","last_unstale":"2023-11-08T19:26:26.762598+0000","last_undegraded":"2023-11-08T19:26:26.762598+0000","last_fullsized":"2023-11-08T19:26:26.762598+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:36:07.345392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"47'1","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.762456+0000","last_change":"2023-11-08T19:26:24.742192+0000","last_active":"2023-11-08T19:26:26.762456+0000","last_peered":"2023-11-08T19:26:26.762456+0000","last_clean":"2023-11-08T19:26:26.762456+0000","last_became_active":"2023-11-08T19:26:24.741925+0000","last_became_peered":"2023-11-08T19:26:24.741925+0000","last_unstale":"2023-11-08T19:26:26.762456+0000","last_undegraded":"2023-11-08T19:26:26.762456+0000","last_fullsized":"2023-11-08T19:26:26.762456+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:05:03.696760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.733044+0000","last_change":"2023-11-08T19:26:06.265318+0000","last_active":"2023-11-08T19:26:25.733044+0000","last_peered":"2023-11-08T19:26:25.733044+0000","last_clean":"2023-11-08T19:26:25.733044+0000","last_became_active":"2023-11-08T19:26:04.603409+0000","last_became_peered":"2023-11-08T19:26:04.603409+0000","last_unstale":"2023-11-08T19:26:25.733044+0000","last_undegraded":"2023-11-08T19:26:25.733044+0000","last_fullsized":"2023-11-08T19:26:25.733044+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:27:24.933111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065684300000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.762588+0000","last_change":"2023-11-08T19:26:25.750379+0000","last_active":"2023-11-08T19:26:26.762588+0000","last_peered":"2023-11-08T19:26:26.762588+0000","last_clean":"2023-11-08T19:26:26.762588+0000","last_became_active":"2023-11-08T19:26:25.750060+0000","last_became_peered":"2023-11-08T19:26:25.750060+0000","last_unstale":"2023-11-08T19:26:26.762588+0000","last_undegraded":"2023-11-08T19:26:26.762588+0000","last_fullsized":"2023-11-08T19:26:26.762588+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:22:59.782108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"47'6","reported_seq":18,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.761687+0000","last_change":"2023-11-08T19:26:24.742213+0000","last_active":"2023-11-08T19:26:26.761687+0000","last_peered":"2023-11-08T19:26:26.761687+0000","last_clean":"2023-11-08T19:26:26.761687+0000","last_became_active":"2023-11-08T19:26:24.741916+0000","last_became_peered":"2023-11-08T19:26:24.741916+0000","last_unstale":"2023-11-08T19:26:26.761687+0000","last_undegraded":"2023-11-08T19:26:26.761687+0000","last_fullsized":"2023-11-08T19:26:26.761687+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:42:12.345053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":28,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.728613+0000","last_change":"2023-11-08T19:26:05.610012+0000","last_active":"2023-11-08T19:26:25.728613+0000","last_peered":"2023-11-08T19:26:25.728613+0000","last_clean":"2023-11-08T19:26:25.728613+0000","last_became_active":"2023-11-08T19:26:04.603209+0000","last_became_peered":"2023-11-08T19:26:04.603209+0000","last_unstale":"2023-11-08T19:26:25.728613+0000","last_undegraded":"2023-11-08T19:26:25.728613+0000","last_fullsized":"2023-11-08T19:26:25.728613+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:35:02.306997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053335100000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.762447+0000","last_change":"2023-11-08T19:26:25.749064+0000","last_active":"2023-11-08T19:26:26.762447+0000","last_peered":"2023-11-08T19:26:26.762447+0000","last_clean":"2023-11-08T19:26:26.762447+0000","last_became_active":"2023-11-08T19:26:25.748500+0000","last_became_peered":"2023-11-08T19:26:25.748500+0000","last_unstale":"2023-11-08T19:26:26.762447+0000","last_undegraded":"2023-11-08T19:26:26.762447+0000","last_fullsized":"2023-11-08T19:26:26.762447+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:49:29.652891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"47'3","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.760398+0000","last_change":"2023-11-08T19:26:24.744478+0000","last_active":"2023-11-08T19:26:25.760398+0000","last_peered":"2023-11-08T19:26:25.760398+0000","last_clean":"2023-11-08T19:26:25.760398+0000","last_became_active":"2023-11-08T19:26:24.744253+0000","last_became_peered":"2023-11-08T19:26:24.744253+0000","last_unstale":"2023-11-08T19:26:25.760398+0000","last_undegraded":"2023-11-08T19:26:25.760398+0000","last_fullsized":"2023-11-08T19:26:25.760398+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:23:46.499595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.737675+0000","last_change":"2023-11-08T19:26:05.606946+0000","last_active":"2023-11-08T19:26:25.737675+0000","last_peered":"2023-11-08T19:26:25.737675+0000","last_clean":"2023-11-08T19:26:25.737675+0000","last_became_active":"2023-11-08T19:26:04.598775+0000","last_became_peered":"2023-11-08T19:26:04.598775+0000","last_unstale":"2023-11-08T19:26:25.737675+0000","last_undegraded":"2023-11-08T19:26:25.737675+0000","last_fullsized":"2023-11-08T19:26:25.737675+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:20:56.898927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049870399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.762480+0000","last_change":"2023-11-08T19:26:25.746669+0000","last_active":"2023-11-08T19:26:26.762480+0000","last_peered":"2023-11-08T19:26:26.762480+0000","last_clean":"2023-11-08T19:26:26.762480+0000","last_became_active":"2023-11-08T19:26:25.746375+0000","last_became_peered":"2023-11-08T19:26:25.746375+0000","last_unstale":"2023-11-08T19:26:26.762480+0000","last_undegraded":"2023-11-08T19:26:26.762480+0000","last_fullsized":"2023-11-08T19:26:26.762480+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:42:13.350558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.762625+0000","last_change":"2023-11-08T19:26:24.745917+0000","last_active":"2023-11-08T19:26:26.762625+0000","last_peered":"2023-11-08T19:26:26.762625+0000","last_clean":"2023-11-08T19:26:26.762625+0000","last_became_active":"2023-11-08T19:26:24.745606+0000","last_became_peered":"2023-11-08T19:26:24.745606+0000","last_unstale":"2023-11-08T19:26:26.762625+0000","last_undegraded":"2023-11-08T19:26:26.762625+0000","last_fullsized":"2023-11-08T19:26:26.762625+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:48:10.178035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.731955+0000","last_change":"2023-11-08T19:26:06.265907+0000","last_active":"2023-11-08T19:26:25.731955+0000","last_peered":"2023-11-08T19:26:25.731955+0000","last_clean":"2023-11-08T19:26:25.731955+0000","last_became_active":"2023-11-08T19:26:04.604772+0000","last_became_peered":"2023-11-08T19:26:04.604772+0000","last_unstale":"2023-11-08T19:26:25.731955+0000","last_undegraded":"2023-11-08T19:26:25.731955+0000","last_fullsized":"2023-11-08T19:26:25.731955+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:44:16.139794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066875300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.761632+0000","last_change":"2023-11-08T19:26:25.751452+0000","last_active":"2023-11-08T19:26:26.761632+0000","last_peered":"2023-11-08T19:26:26.761632+0000","last_clean":"2023-11-08T19:26:26.761632+0000","last_became_active":"2023-11-08T19:26:25.751140+0000","last_became_peered":"2023-11-08T19:26:25.751140+0000","last_unstale":"2023-11-08T19:26:26.761632+0000","last_undegraded":"2023-11-08T19:26:26.761632+0000","last_fullsized":"2023-11-08T19:26:26.761632+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:43:16.409064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.731355+0000","last_change":"2023-11-08T19:26:24.741119+0000","last_active":"2023-11-08T19:26:25.731355+0000","last_peered":"2023-11-08T19:26:25.731355+0000","last_clean":"2023-11-08T19:26:25.731355+0000","last_became_active":"2023-11-08T19:26:24.740821+0000","last_became_peered":"2023-11-08T19:26:24.740821+0000","last_unstale":"2023-11-08T19:26:25.731355+0000","last_undegraded":"2023-11-08T19:26:25.731355+0000","last_fullsized":"2023-11-08T19:26:25.731355+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:45:23.692188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.734895+0000","last_change":"2023-11-08T19:26:06.394488+0000","last_active":"2023-11-08T19:26:25.734895+0000","last_peered":"2023-11-08T19:26:25.734895+0000","last_clean":"2023-11-08T19:26:25.734895+0000","last_became_active":"2023-11-08T19:26:04.606805+0000","last_became_peered":"2023-11-08T19:26:04.606805+0000","last_unstale":"2023-11-08T19:26:25.734895+0000","last_undegraded":"2023-11-08T19:26:25.734895+0000","last_fullsized":"2023-11-08T19:26:25.734895+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:25:20.857133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045317000000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.763082+0000","last_change":"2023-11-08T19:26:25.755102+0000","last_active":"2023-11-08T19:26:26.763082+0000","last_peered":"2023-11-08T19:26:26.763082+0000","last_clean":"2023-11-08T19:26:26.763082+0000","last_became_active":"2023-11-08T19:26:25.750220+0000","last_became_peered":"2023-11-08T19:26:25.750220+0000","last_unstale":"2023-11-08T19:26:26.763082+0000","last_undegraded":"2023-11-08T19:26:26.763082+0000","last_fullsized":"2023-11-08T19:26:26.763082+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:11:13.832264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"47'8","reported_seq":20,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.762303+0000","last_change":"2023-11-08T19:26:24.743410+0000","last_active":"2023-11-08T19:26:26.762303+0000","last_peered":"2023-11-08T19:26:26.762303+0000","last_clean":"2023-11-08T19:26:26.762303+0000","last_became_active":"2023-11-08T19:26:24.743162+0000","last_became_peered":"2023-11-08T19:26:24.743162+0000","last_unstale":"2023-11-08T19:26:26.762303+0000","last_undegraded":"2023-11-08T19:26:26.762303+0000","last_fullsized":"2023-11-08T19:26:26.762303+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:22:56.213140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.727378+0000","last_change":"2023-11-08T19:26:06.423754+0000","last_active":"2023-11-08T19:26:25.727378+0000","last_peered":"2023-11-08T19:26:25.727378+0000","last_clean":"2023-11-08T19:26:25.727378+0000","last_became_active":"2023-11-08T19:26:04.599322+0000","last_became_peered":"2023-11-08T19:26:04.599322+0000","last_unstale":"2023-11-08T19:26:25.727378+0000","last_undegraded":"2023-11-08T19:26:25.727378+0000","last_fullsized":"2023-11-08T19:26:25.727378+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:13:08.689128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042670000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.761894+0000","last_change":"2023-11-08T19:26:25.736435+0000","last_active":"2023-11-08T19:26:26.761894+0000","last_peered":"2023-11-08T19:26:26.761894+0000","last_clean":"2023-11-08T19:26:26.761894+0000","last_became_active":"2023-11-08T19:26:25.736134+0000","last_became_peered":"2023-11-08T19:26:25.736134+0000","last_unstale":"2023-11-08T19:26:26.761894+0000","last_undegraded":"2023-11-08T19:26:26.761894+0000","last_fullsized":"2023-11-08T19:26:26.761894+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:06:45.575533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.729611+0000","last_change":"2023-11-08T19:26:24.743478+0000","last_active":"2023-11-08T19:26:25.729611+0000","last_peered":"2023-11-08T19:26:25.729611+0000","last_clean":"2023-11-08T19:26:25.729611+0000","last_became_active":"2023-11-08T19:26:24.743177+0000","last_became_peered":"2023-11-08T19:26:24.743177+0000","last_unstale":"2023-11-08T19:26:25.729611+0000","last_undegraded":"2023-11-08T19:26:25.729611+0000","last_fullsized":"2023-11-08T19:26:25.729611+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:29:43.680373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.733903+0000","last_change":"2023-11-08T19:26:06.394320+0000","last_active":"2023-11-08T19:26:25.733903+0000","last_peered":"2023-11-08T19:26:25.733903+0000","last_clean":"2023-11-08T19:26:25.733903+0000","last_became_active":"2023-11-08T19:26:04.603212+0000","last_became_peered":"2023-11-08T19:26:04.603212+0000","last_unstale":"2023-11-08T19:26:25.733903+0000","last_undegraded":"2023-11-08T19:26:25.733903+0000","last_fullsized":"2023-11-08T19:26:25.733903+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:25:29.947079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054695399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.763031+0000","last_change":"2023-11-08T19:26:25.758900+0000","last_active":"2023-11-08T19:26:26.763031+0000","last_peered":"2023-11-08T19:26:26.763031+0000","last_clean":"2023-11-08T19:26:26.763031+0000","last_became_active":"2023-11-08T19:26:25.758612+0000","last_became_peered":"2023-11-08T19:26:25.758612+0000","last_unstale":"2023-11-08T19:26:26.763031+0000","last_undegraded":"2023-11-08T19:26:26.763031+0000","last_fullsized":"2023-11-08T19:26:26.763031+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:03:31.037228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.728395+0000","last_change":"2023-11-08T19:26:24.739300+0000","last_active":"2023-11-08T19:26:25.728395+0000","last_peered":"2023-11-08T19:26:25.728395+0000","last_clean":"2023-11-08T19:26:25.728395+0000","last_became_active":"2023-11-08T19:26:24.738988+0000","last_became_peered":"2023-11-08T19:26:24.738988+0000","last_unstale":"2023-11-08T19:26:25.728395+0000","last_undegraded":"2023-11-08T19:26:25.728395+0000","last_fullsized":"2023-11-08T19:26:25.728395+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:44:36.273988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.728522+0000","last_change":"2023-11-08T19:26:06.424073+0000","last_active":"2023-11-08T19:26:25.728522+0000","last_peered":"2023-11-08T19:26:25.728522+0000","last_clean":"2023-11-08T19:26:25.728522+0000","last_became_active":"2023-11-08T19:26:04.599046+0000","last_became_peered":"2023-11-08T19:26:04.599046+0000","last_unstale":"2023-11-08T19:26:25.728522+0000","last_undegraded":"2023-11-08T19:26:25.728522+0000","last_fullsized":"2023-11-08T19:26:25.728522+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:04:43.562566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035092199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.762035+0000","last_change":"2023-11-08T19:26:25.744142+0000","last_active":"2023-11-08T19:26:26.762035+0000","last_peered":"2023-11-08T19:26:26.762035+0000","last_clean":"2023-11-08T19:26:26.762035+0000","last_became_active":"2023-11-08T19:26:25.743686+0000","last_became_peered":"2023-11-08T19:26:25.743686+0000","last_unstale":"2023-11-08T19:26:26.762035+0000","last_undegraded":"2023-11-08T19:26:26.762035+0000","last_fullsized":"2023-11-08T19:26:26.762035+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:29:30.597869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"47'2","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.762518+0000","last_change":"2023-11-08T19:26:24.740785+0000","last_active":"2023-11-08T19:26:26.762518+0000","last_peered":"2023-11-08T19:26:26.762518+0000","last_clean":"2023-11-08T19:26:26.762518+0000","last_became_active":"2023-11-08T19:26:24.740482+0000","last_became_peered":"2023-11-08T19:26:24.740482+0000","last_unstale":"2023-11-08T19:26:26.762518+0000","last_undegraded":"2023-11-08T19:26:26.762518+0000","last_fullsized":"2023-11-08T19:26:26.762518+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:22:25.145083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":28,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.728645+0000","last_change":"2023-11-08T19:26:05.608980+0000","last_active":"2023-11-08T19:26:25.728645+0000","last_peered":"2023-11-08T19:26:25.728645+0000","last_clean":"2023-11-08T19:26:25.728645+0000","last_became_active":"2023-11-08T19:26:04.606386+0000","last_became_peered":"2023-11-08T19:26:04.606386+0000","last_unstale":"2023-11-08T19:26:25.728645+0000","last_undegraded":"2023-11-08T19:26:25.728645+0000","last_fullsized":"2023-11-08T19:26:25.728645+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:41:50.771457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037950599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.730654+0000","last_change":"2023-11-08T19:26:25.757783+0000","last_active":"2023-11-08T19:26:26.730654+0000","last_peered":"2023-11-08T19:26:26.730654+0000","last_clean":"2023-11-08T19:26:26.730654+0000","last_became_active":"2023-11-08T19:26:25.757482+0000","last_became_peered":"2023-11-08T19:26:25.757482+0000","last_unstale":"2023-11-08T19:26:26.730654+0000","last_undegraded":"2023-11-08T19:26:26.730654+0000","last_fullsized":"2023-11-08T19:26:26.730654+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:11:18.036694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.728806+0000","last_change":"2023-11-08T19:26:24.743633+0000","last_active":"2023-11-08T19:26:25.728806+0000","last_peered":"2023-11-08T19:26:25.728806+0000","last_clean":"2023-11-08T19:26:25.728806+0000","last_became_active":"2023-11-08T19:26:24.743288+0000","last_became_peered":"2023-11-08T19:26:24.743288+0000","last_unstale":"2023-11-08T19:26:25.728806+0000","last_undegraded":"2023-11-08T19:26:25.728806+0000","last_fullsized":"2023-11-08T19:26:25.728806+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:20:09.309300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.738107+0000","last_change":"2023-11-08T19:26:06.541623+0000","last_active":"2023-11-08T19:26:25.738107+0000","last_peered":"2023-11-08T19:26:25.738107+0000","last_clean":"2023-11-08T19:26:25.738107+0000","last_became_active":"2023-11-08T19:26:04.601961+0000","last_became_peered":"2023-11-08T19:26:04.601961+0000","last_unstale":"2023-11-08T19:26:25.738107+0000","last_undegraded":"2023-11-08T19:26:25.738107+0000","last_fullsized":"2023-11-08T19:26:25.738107+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:11:26.508634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000466386,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.761661+0000","last_change":"2023-11-08T19:26:25.747292+0000","last_active":"2023-11-08T19:26:26.761661+0000","last_peered":"2023-11-08T19:26:26.761661+0000","last_clean":"2023-11-08T19:26:26.761661+0000","last_became_active":"2023-11-08T19:26:25.747014+0000","last_became_peered":"2023-11-08T19:26:25.747014+0000","last_unstale":"2023-11-08T19:26:26.761661+0000","last_undegraded":"2023-11-08T19:26:26.761661+0000","last_fullsized":"2023-11-08T19:26:26.761661+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:42:14.637155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"47'1","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.759266+0000","last_change":"2023-11-08T19:26:24.733981+0000","last_active":"2023-11-08T19:26:25.759266+0000","last_peered":"2023-11-08T19:26:25.759266+0000","last_clean":"2023-11-08T19:26:25.759266+0000","last_became_active":"2023-11-08T19:26:24.733628+0000","last_became_peered":"2023-11-08T19:26:24.733628+0000","last_unstale":"2023-11-08T19:26:25.759266+0000","last_undegraded":"2023-11-08T19:26:25.759266+0000","last_fullsized":"2023-11-08T19:26:25.759266+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:07:07.580880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.727605+0000","last_change":"2023-11-08T19:26:06.424464+0000","last_active":"2023-11-08T19:26:25.727605+0000","last_peered":"2023-11-08T19:26:25.727605+0000","last_clean":"2023-11-08T19:26:25.727605+0000","last_became_active":"2023-11-08T19:26:04.603144+0000","last_became_peered":"2023-11-08T19:26:04.603144+0000","last_unstale":"2023-11-08T19:26:25.727605+0000","last_undegraded":"2023-11-08T19:26:25.727605+0000","last_fullsized":"2023-11-08T19:26:25.727605+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:42:55.269365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054090799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.730354+0000","last_change":"2023-11-08T19:26:25.745301+0000","last_active":"2023-11-08T19:26:26.730354+0000","last_peered":"2023-11-08T19:26:26.730354+0000","last_clean":"2023-11-08T19:26:26.730354+0000","last_became_active":"2023-11-08T19:26:25.745014+0000","last_became_peered":"2023-11-08T19:26:25.745014+0000","last_unstale":"2023-11-08T19:26:26.730354+0000","last_undegraded":"2023-11-08T19:26:26.730354+0000","last_fullsized":"2023-11-08T19:26:26.730354+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:22:26.150588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"47'2","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.762839+0000","last_change":"2023-11-08T19:26:24.739629+0000","last_active":"2023-11-08T19:26:26.762839+0000","last_peered":"2023-11-08T19:26:26.762839+0000","last_clean":"2023-11-08T19:26:26.762839+0000","last_became_active":"2023-11-08T19:26:24.739346+0000","last_became_peered":"2023-11-08T19:26:24.739346+0000","last_unstale":"2023-11-08T19:26:26.762839+0000","last_undegraded":"2023-11-08T19:26:26.762839+0000","last_fullsized":"2023-11-08T19:26:26.762839+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:25:50.081273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.738253+0000","last_change":"2023-11-08T19:26:05.606898+0000","last_active":"2023-11-08T19:26:25.738253+0000","last_peered":"2023-11-08T19:26:25.738253+0000","last_clean":"2023-11-08T19:26:25.738253+0000","last_became_active":"2023-11-08T19:26:04.603955+0000","last_became_peered":"2023-11-08T19:26:04.603955+0000","last_unstale":"2023-11-08T19:26:25.738253+0000","last_undegraded":"2023-11-08T19:26:25.738253+0000","last_fullsized":"2023-11-08T19:26:25.738253+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:19:11.589559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036473499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.762090+0000","last_change":"2023-11-08T19:26:25.749337+0000","last_active":"2023-11-08T19:26:26.762090+0000","last_peered":"2023-11-08T19:26:26.762090+0000","last_clean":"2023-11-08T19:26:26.762090+0000","last_became_active":"2023-11-08T19:26:25.749069+0000","last_became_peered":"2023-11-08T19:26:25.749069+0000","last_unstale":"2023-11-08T19:26:26.762090+0000","last_undegraded":"2023-11-08T19:26:26.762090+0000","last_fullsized":"2023-11-08T19:26:26.762090+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:03:20.639343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"47'1","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.762883+0000","last_change":"2023-11-08T19:26:24.742330+0000","last_active":"2023-11-08T19:26:26.762883+0000","last_peered":"2023-11-08T19:26:26.762883+0000","last_clean":"2023-11-08T19:26:26.762883+0000","last_became_active":"2023-11-08T19:26:24.742055+0000","last_became_peered":"2023-11-08T19:26:24.742055+0000","last_unstale":"2023-11-08T19:26:26.762883+0000","last_undegraded":"2023-11-08T19:26:26.762883+0000","last_fullsized":"2023-11-08T19:26:26.762883+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:48:26.748546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.732066+0000","last_change":"2023-11-08T19:26:06.265686+0000","last_active":"2023-11-08T19:26:25.732066+0000","last_peered":"2023-11-08T19:26:25.732066+0000","last_clean":"2023-11-08T19:26:25.732066+0000","last_became_active":"2023-11-08T19:26:04.602111+0000","last_became_peered":"2023-11-08T19:26:04.602111+0000","last_unstale":"2023-11-08T19:26:25.732066+0000","last_undegraded":"2023-11-08T19:26:25.732066+0000","last_fullsized":"2023-11-08T19:26:25.732066+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:40:11.164951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056197899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.730988+0000","last_change":"2023-11-08T19:26:25.746346+0000","last_active":"2023-11-08T19:26:26.730988+0000","last_peered":"2023-11-08T19:26:26.730988+0000","last_clean":"2023-11-08T19:26:26.730988+0000","last_became_active":"2023-11-08T19:26:25.746052+0000","last_became_peered":"2023-11-08T19:26:25.746052+0000","last_unstale":"2023-11-08T19:26:26.730988+0000","last_undegraded":"2023-11-08T19:26:26.730988+0000","last_fullsized":"2023-11-08T19:26:26.730988+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:48:27.754051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.761287+0000","last_change":"2023-11-08T19:26:24.740736+0000","last_active":"2023-11-08T19:26:26.761287+0000","last_peered":"2023-11-08T19:26:26.761287+0000","last_clean":"2023-11-08T19:26:26.761287+0000","last_became_active":"2023-11-08T19:26:24.740479+0000","last_became_peered":"2023-11-08T19:26:24.740479+0000","last_unstale":"2023-11-08T19:26:26.761287+0000","last_undegraded":"2023-11-08T19:26:26.761287+0000","last_fullsized":"2023-11-08T19:26:26.761287+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:20:16.421029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.734247+0000","last_change":"2023-11-08T19:26:06.394075+0000","last_active":"2023-11-08T19:26:25.734247+0000","last_peered":"2023-11-08T19:26:25.734247+0000","last_clean":"2023-11-08T19:26:25.734247+0000","last_became_active":"2023-11-08T19:26:04.606967+0000","last_became_peered":"2023-11-08T19:26:04.606967+0000","last_unstale":"2023-11-08T19:26:25.734247+0000","last_undegraded":"2023-11-08T19:26:25.734247+0000","last_fullsized":"2023-11-08T19:26:25.734247+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:19:56.286835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040971999999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.762648+0000","last_change":"2023-11-08T19:26:25.744126+0000","last_active":"2023-11-08T19:26:26.762648+0000","last_peered":"2023-11-08T19:26:26.762648+0000","last_clean":"2023-11-08T19:26:26.762648+0000","last_became_active":"2023-11-08T19:26:25.743853+0000","last_became_peered":"2023-11-08T19:26:25.743853+0000","last_unstale":"2023-11-08T19:26:26.762648+0000","last_undegraded":"2023-11-08T19:26:26.762648+0000","last_fullsized":"2023-11-08T19:26:26.762648+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:34:52.582926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.762073+0000","last_change":"2023-11-08T19:26:24.746451+0000","last_active":"2023-11-08T19:26:26.762073+0000","last_peered":"2023-11-08T19:26:26.762073+0000","last_clean":"2023-11-08T19:26:26.762073+0000","last_became_active":"2023-11-08T19:26:24.746152+0000","last_became_peered":"2023-11-08T19:26:24.746152+0000","last_unstale":"2023-11-08T19:26:26.762073+0000","last_undegraded":"2023-11-08T19:26:26.762073+0000","last_fullsized":"2023-11-08T19:26:26.762073+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:56:37.224649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":28,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.731324+0000","last_change":"2023-11-08T19:26:05.608892+0000","last_active":"2023-11-08T19:26:25.731324+0000","last_peered":"2023-11-08T19:26:25.731324+0000","last_clean":"2023-11-08T19:26:25.731324+0000","last_became_active":"2023-11-08T19:26:04.604519+0000","last_became_peered":"2023-11-08T19:26:04.604519+0000","last_unstale":"2023-11-08T19:26:25.731324+0000","last_undegraded":"2023-11-08T19:26:25.731324+0000","last_fullsized":"2023-11-08T19:26:25.731324+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:11:26.508634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051248099999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.730957+0000","last_change":"2023-11-08T19:26:25.734469+0000","last_active":"2023-11-08T19:26:26.730957+0000","last_peered":"2023-11-08T19:26:26.730957+0000","last_clean":"2023-11-08T19:26:26.730957+0000","last_became_active":"2023-11-08T19:26:25.734211+0000","last_became_peered":"2023-11-08T19:26:25.734211+0000","last_unstale":"2023-11-08T19:26:26.730957+0000","last_undegraded":"2023-11-08T19:26:26.730957+0000","last_fullsized":"2023-11-08T19:26:26.730957+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:13:29.828827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.761686+0000","last_change":"2023-11-08T19:26:24.742825+0000","last_active":"2023-11-08T19:26:26.761686+0000","last_peered":"2023-11-08T19:26:26.761686+0000","last_clean":"2023-11-08T19:26:26.761686+0000","last_became_active":"2023-11-08T19:26:24.742467+0000","last_became_peered":"2023-11-08T19:26:24.742467+0000","last_unstale":"2023-11-08T19:26:26.761686+0000","last_undegraded":"2023-11-08T19:26:26.761686+0000","last_fullsized":"2023-11-08T19:26:26.761686+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:49:52.879282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.728303+0000","last_change":"2023-11-08T19:26:06.424236+0000","last_active":"2023-11-08T19:26:25.728303+0000","last_peered":"2023-11-08T19:26:25.728303+0000","last_clean":"2023-11-08T19:26:25.728303+0000","last_became_active":"2023-11-08T19:26:04.600115+0000","last_became_peered":"2023-11-08T19:26:04.600115+0000","last_unstale":"2023-11-08T19:26:25.728303+0000","last_undegraded":"2023-11-08T19:26:25.728303+0000","last_fullsized":"2023-11-08T19:26:25.728303+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:18:01.876605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065803900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.762236+0000","last_change":"2023-11-08T19:26:25.743964+0000","last_active":"2023-11-08T19:26:26.762236+0000","last_peered":"2023-11-08T19:26:26.762236+0000","last_clean":"2023-11-08T19:26:26.762236+0000","last_became_active":"2023-11-08T19:26:25.743403+0000","last_became_peered":"2023-11-08T19:26:25.743403+0000","last_unstale":"2023-11-08T19:26:26.762236+0000","last_undegraded":"2023-11-08T19:26:26.762236+0000","last_fullsized":"2023-11-08T19:26:26.762236+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:29:44.685878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"47'2","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.730526+0000","last_change":"2023-11-08T19:26:24.735862+0000","last_active":"2023-11-08T19:26:26.730526+0000","last_peered":"2023-11-08T19:26:26.730526+0000","last_clean":"2023-11-08T19:26:26.730526+0000","last_became_active":"2023-11-08T19:26:24.735588+0000","last_became_peered":"2023-11-08T19:26:24.735588+0000","last_unstale":"2023-11-08T19:26:26.730526+0000","last_undegraded":"2023-11-08T19:26:26.730526+0000","last_fullsized":"2023-11-08T19:26:26.730526+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:05:03.696760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.737942+0000","last_change":"2023-11-08T19:26:06.541360+0000","last_active":"2023-11-08T19:26:25.737942+0000","last_peered":"2023-11-08T19:26:25.737942+0000","last_clean":"2023-11-08T19:26:25.737942+0000","last_became_active":"2023-11-08T19:26:04.603725+0000","last_became_peered":"2023-11-08T19:26:04.603725+0000","last_unstale":"2023-11-08T19:26:25.737942+0000","last_undegraded":"2023-11-08T19:26:25.737942+0000","last_fullsized":"2023-11-08T19:26:25.737942+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:28:48.576742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053365400000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.762526+0000","last_change":"2023-11-08T19:26:25.736017+0000","last_active":"2023-11-08T19:26:26.762526+0000","last_peered":"2023-11-08T19:26:26.762526+0000","last_clean":"2023-11-08T19:26:26.762526+0000","last_became_active":"2023-11-08T19:26:25.735715+0000","last_became_peered":"2023-11-08T19:26:25.735715+0000","last_unstale":"2023-11-08T19:26:26.762526+0000","last_undegraded":"2023-11-08T19:26:26.762526+0000","last_fullsized":"2023-11-08T19:26:26.762526+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:56:44.858396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.762649+0000","last_change":"2023-11-08T19:26:24.741527+0000","last_active":"2023-11-08T19:26:26.762649+0000","last_peered":"2023-11-08T19:26:26.762649+0000","last_clean":"2023-11-08T19:26:26.762649+0000","last_became_active":"2023-11-08T19:26:24.741178+0000","last_became_peered":"2023-11-08T19:26:24.741178+0000","last_unstale":"2023-11-08T19:26:26.762649+0000","last_undegraded":"2023-11-08T19:26:26.762649+0000","last_fullsized":"2023-11-08T19:26:26.762649+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:06:06.110808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":28,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.728501+0000","last_change":"2023-11-08T19:26:05.609824+0000","last_active":"2023-11-08T19:26:25.728501+0000","last_peered":"2023-11-08T19:26:25.728501+0000","last_clean":"2023-11-08T19:26:25.728501+0000","last_became_active":"2023-11-08T19:26:04.603320+0000","last_became_peered":"2023-11-08T19:26:04.603320+0000","last_unstale":"2023-11-08T19:26:25.728501+0000","last_undegraded":"2023-11-08T19:26:25.728501+0000","last_fullsized":"2023-11-08T19:26:25.728501+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:05:45.976615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063225,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.730644+0000","last_change":"2023-11-08T19:26:25.746887+0000","last_active":"2023-11-08T19:26:26.730644+0000","last_peered":"2023-11-08T19:26:26.730644+0000","last_clean":"2023-11-08T19:26:26.730644+0000","last_became_active":"2023-11-08T19:26:25.746588+0000","last_became_peered":"2023-11-08T19:26:25.746588+0000","last_unstale":"2023-11-08T19:26:26.730644+0000","last_undegraded":"2023-11-08T19:26:26.730644+0000","last_fullsized":"2023-11-08T19:26:26.730644+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:23:47.505101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.728774+0000","last_change":"2023-11-08T19:26:24.742178+0000","last_active":"2023-11-08T19:26:25.728774+0000","last_peered":"2023-11-08T19:26:25.728774+0000","last_clean":"2023-11-08T19:26:25.728774+0000","last_became_active":"2023-11-08T19:26:24.741868+0000","last_became_peered":"2023-11-08T19:26:24.741868+0000","last_unstale":"2023-11-08T19:26:25.728774+0000","last_undegraded":"2023-11-08T19:26:25.728774+0000","last_fullsized":"2023-11-08T19:26:25.728774+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:35:23.618441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.732053+0000","last_change":"2023-11-08T19:26:06.296937+0000","last_active":"2023-11-08T19:26:25.732053+0000","last_peered":"2023-11-08T19:26:25.732053+0000","last_clean":"2023-11-08T19:26:25.732053+0000","last_became_active":"2023-11-08T19:26:04.599999+0000","last_became_peered":"2023-11-08T19:26:04.599999+0000","last_unstale":"2023-11-08T19:26:25.732053+0000","last_undegraded":"2023-11-08T19:26:25.732053+0000","last_fullsized":"2023-11-08T19:26:25.732053+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:50:43.827570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025605700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.761976+0000","last_change":"2023-11-08T19:26:25.753296+0000","last_active":"2023-11-08T19:26:26.761976+0000","last_peered":"2023-11-08T19:26:26.761976+0000","last_clean":"2023-11-08T19:26:26.761976+0000","last_became_active":"2023-11-08T19:26:25.752989+0000","last_became_peered":"2023-11-08T19:26:25.752989+0000","last_unstale":"2023-11-08T19:26:26.761976+0000","last_undegraded":"2023-11-08T19:26:26.761976+0000","last_fullsized":"2023-11-08T19:26:26.761976+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:23:33.302754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"47'2","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.747075+0000","last_change":"2023-11-08T19:26:24.741588+0000","last_active":"2023-11-08T19:26:25.747075+0000","last_peered":"2023-11-08T19:26:25.747075+0000","last_clean":"2023-11-08T19:26:25.747075+0000","last_became_active":"2023-11-08T19:26:24.741371+0000","last_became_peered":"2023-11-08T19:26:24.741371+0000","last_unstale":"2023-11-08T19:26:25.747075+0000","last_undegraded":"2023-11-08T19:26:25.747075+0000","last_fullsized":"2023-11-08T19:26:25.747075+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:51:03.961763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.728463+0000","last_change":"2023-11-08T19:26:06.424511+0000","last_active":"2023-11-08T19:26:25.728463+0000","last_peered":"2023-11-08T19:26:25.728463+0000","last_clean":"2023-11-08T19:26:25.728463+0000","last_became_active":"2023-11-08T19:26:04.598973+0000","last_became_peered":"2023-11-08T19:26:04.598973+0000","last_unstale":"2023-11-08T19:26:25.728463+0000","last_undegraded":"2023-11-08T19:26:25.728463+0000","last_fullsized":"2023-11-08T19:26:25.728463+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:41:53.497456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031755900000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.762212+0000","last_change":"2023-11-08T19:26:25.742775+0000","last_active":"2023-11-08T19:26:26.762212+0000","last_peered":"2023-11-08T19:26:26.762212+0000","last_clean":"2023-11-08T19:26:26.762212+0000","last_became_active":"2023-11-08T19:26:25.740742+0000","last_became_peered":"2023-11-08T19:26:25.740742+0000","last_unstale":"2023-11-08T19:26:26.762212+0000","last_undegraded":"2023-11-08T19:26:26.762212+0000","last_fullsized":"2023-11-08T19:26:26.762212+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:19:35.879794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.761883+0000","last_change":"2023-11-08T19:26:24.736978+0000","last_active":"2023-11-08T19:26:26.761883+0000","last_peered":"2023-11-08T19:26:26.761883+0000","last_clean":"2023-11-08T19:26:26.761883+0000","last_became_active":"2023-11-08T19:26:24.736651+0000","last_became_peered":"2023-11-08T19:26:24.736651+0000","last_unstale":"2023-11-08T19:26:26.761883+0000","last_undegraded":"2023-11-08T19:26:26.761883+0000","last_fullsized":"2023-11-08T19:26:26.761883+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:22:25.145083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.731536+0000","last_change":"2023-11-08T19:26:06.296634+0000","last_active":"2023-11-08T19:26:25.731536+0000","last_peered":"2023-11-08T19:26:25.731536+0000","last_clean":"2023-11-08T19:26:25.731536+0000","last_became_active":"2023-11-08T19:26:04.603088+0000","last_became_peered":"2023-11-08T19:26:04.603088+0000","last_unstale":"2023-11-08T19:26:25.731536+0000","last_undegraded":"2023-11-08T19:26:25.731536+0000","last_fullsized":"2023-11-08T19:26:25.731536+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:01:56.991346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038180899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.762568+0000","last_change":"2023-11-08T19:26:25.749860+0000","last_active":"2023-11-08T19:26:26.762568+0000","last_peered":"2023-11-08T19:26:26.762568+0000","last_clean":"2023-11-08T19:26:26.762568+0000","last_became_active":"2023-11-08T19:26:25.749612+0000","last_became_peered":"2023-11-08T19:26:25.749612+0000","last_unstale":"2023-11-08T19:26:26.762568+0000","last_undegraded":"2023-11-08T19:26:26.762568+0000","last_fullsized":"2023-11-08T19:26:26.762568+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:48:09.695836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.761578+0000","last_change":"2023-11-08T19:26:24.742441+0000","last_active":"2023-11-08T19:26:26.761578+0000","last_peered":"2023-11-08T19:26:26.761578+0000","last_clean":"2023-11-08T19:26:26.761578+0000","last_became_active":"2023-11-08T19:26:24.742069+0000","last_became_peered":"2023-11-08T19:26:24.742069+0000","last_unstale":"2023-11-08T19:26:26.761578+0000","last_undegraded":"2023-11-08T19:26:26.761578+0000","last_fullsized":"2023-11-08T19:26:26.761578+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:35:22.441191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.734004+0000","last_change":"2023-11-08T19:26:06.394300+0000","last_active":"2023-11-08T19:26:25.734004+0000","last_peered":"2023-11-08T19:26:25.734004+0000","last_clean":"2023-11-08T19:26:25.734004+0000","last_became_active":"2023-11-08T19:26:04.601685+0000","last_became_peered":"2023-11-08T19:26:04.601685+0000","last_unstale":"2023-11-08T19:26:25.734004+0000","last_undegraded":"2023-11-08T19:26:25.734004+0000","last_fullsized":"2023-11-08T19:26:25.734004+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:51:06.626474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061044099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.761844+0000","last_change":"2023-11-08T19:26:25.744017+0000","last_active":"2023-11-08T19:26:26.761844+0000","last_peered":"2023-11-08T19:26:26.761844+0000","last_clean":"2023-11-08T19:26:26.761844+0000","last_became_active":"2023-11-08T19:26:25.743747+0000","last_became_peered":"2023-11-08T19:26:25.743747+0000","last_unstale":"2023-11-08T19:26:26.761844+0000","last_undegraded":"2023-11-08T19:26:26.761844+0000","last_fullsized":"2023-11-08T19:26:26.761844+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:11:18.036694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.761455+0000","last_change":"2023-11-08T19:26:24.743608+0000","last_active":"2023-11-08T19:26:26.761455+0000","last_peered":"2023-11-08T19:26:26.761455+0000","last_clean":"2023-11-08T19:26:26.761455+0000","last_became_active":"2023-11-08T19:26:24.742838+0000","last_became_peered":"2023-11-08T19:26:24.742838+0000","last_unstale":"2023-11-08T19:26:26.761455+0000","last_undegraded":"2023-11-08T19:26:26.761455+0000","last_fullsized":"2023-11-08T19:26:26.761455+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:25:16.496965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":28,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.728055+0000","last_change":"2023-11-08T19:26:05.609570+0000","last_active":"2023-11-08T19:26:25.728055+0000","last_peered":"2023-11-08T19:26:25.728055+0000","last_clean":"2023-11-08T19:26:25.728055+0000","last_became_active":"2023-11-08T19:26:04.608379+0000","last_became_peered":"2023-11-08T19:26:04.608379+0000","last_unstale":"2023-11-08T19:26:25.728055+0000","last_undegraded":"2023-11-08T19:26:25.728055+0000","last_fullsized":"2023-11-08T19:26:25.728055+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:40:08.986649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000529657,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.762778+0000","last_change":"2023-11-08T19:26:25.740002+0000","last_active":"2023-11-08T19:26:26.762778+0000","last_peered":"2023-11-08T19:26:26.762778+0000","last_clean":"2023-11-08T19:26:26.762778+0000","last_became_active":"2023-11-08T19:26:25.739710+0000","last_became_peered":"2023-11-08T19:26:25.739710+0000","last_unstale":"2023-11-08T19:26:26.762778+0000","last_undegraded":"2023-11-08T19:26:26.762778+0000","last_fullsized":"2023-11-08T19:26:26.762778+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:38:44.411483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.762711+0000","last_change":"2023-11-08T19:26:24.742104+0000","last_active":"2023-11-08T19:26:26.762711+0000","last_peered":"2023-11-08T19:26:26.762711+0000","last_clean":"2023-11-08T19:26:26.762711+0000","last_became_active":"2023-11-08T19:26:24.741846+0000","last_became_peered":"2023-11-08T19:26:24.741846+0000","last_unstale":"2023-11-08T19:26:26.762711+0000","last_undegraded":"2023-11-08T19:26:26.762711+0000","last_fullsized":"2023-11-08T19:26:26.762711+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:18:25.803994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.733125+0000","last_change":"2023-11-08T19:26:06.265419+0000","last_active":"2023-11-08T19:26:25.733125+0000","last_peered":"2023-11-08T19:26:25.733125+0000","last_clean":"2023-11-08T19:26:25.733125+0000","last_became_active":"2023-11-08T19:26:04.605333+0000","last_became_peered":"2023-11-08T19:26:04.605333+0000","last_unstale":"2023-11-08T19:26:25.733125+0000","last_undegraded":"2023-11-08T19:26:25.733125+0000","last_fullsized":"2023-11-08T19:26:25.733125+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:01:56.991346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064606400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:26.729672+0000","last_change":"2023-11-08T19:26:25.745842+0000","last_active":"2023-11-08T19:26:26.729672+0000","last_peered":"2023-11-08T19:26:26.729672+0000","last_clean":"2023-11-08T19:26:26.729672+0000","last_became_active":"2023-11-08T19:26:25.745304+0000","last_became_peered":"2023-11-08T19:26:25.745304+0000","last_unstale":"2023-11-08T19:26:26.729672+0000","last_undegraded":"2023-11-08T19:26:26.729672+0000","last_fullsized":"2023-11-08T19:26:26.729672+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:24.718298+0000","last_clean_scrub_stamp":"2023-11-08T19:26:24.718298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:40:30.126348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"48'1","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-08T19:26:28.600231+0000","last_change":"2023-11-08T19:26:24.740671+0000","last_active":"2023-11-08T19:26:28.600231+0000","last_peered":"2023-11-08T19:26:28.600231+0000","last_clean":"2023-11-08T19:26:28.600231+0000","last_became_active":"2023-11-08T19:26:24.740395+0000","last_became_peered":"2023-11-08T19:26:24.740395+0000","last_unstale":"2023-11-08T19:26:28.600231+0000","last_undegraded":"2023-11-08T19:26:28.600231+0000","last_fullsized":"2023-11-08T19:26:28.600231+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:23.712793+0000","last_clean_scrub_stamp":"2023-11-08T19:26:23.712793+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:09:24.710916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":26,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.739334+0000","last_change":"2023-11-08T19:26:06.542044+0000","last_active":"2023-11-08T19:26:25.739334+0000","last_peered":"2023-11-08T19:26:25.739334+0000","last_clean":"2023-11-08T19:26:25.739334+0000","last_became_active":"2023-11-08T19:26:04.603056+0000","last_became_peered":"2023-11-08T19:26:04.603056+0000","last_unstale":"2023-11-08T19:26:25.739334+0000","last_undegraded":"2023-11-08T19:26:25.739334+0000","last_fullsized":"2023-11-08T19:26:25.739334+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:41:50.771457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000517414,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":193,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-08T19:26:25.731732+0000","last_change":"2023-11-08T19:24:45.888122+0000","last_active":"2023-11-08T19:26:25.731732+0000","last_peered":"2023-11-08T19:26:25.731732+0000","last_clean":"2023-11-08T19:26:25.731732+0000","last_became_active":"2023-11-08T19:24:45.887048+0000","last_became_peered":"2023-11-08T19:24:45.887048+0000","last_unstale":"2023-11-08T19:26:25.731732+0000","last_undegraded":"2023-11-08T19:26:25.731732+0000","last_fullsized":"2023-11-08T19:26:25.731732+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:44.575133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:44.575133+0000","last_clean_scrub_stamp":"2023-11-08T19:24:44.575133+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:19:20.202062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574873,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28160,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343680,"statfs":{"total":96636764160,"available":96607928320,"internally_reserved":0,"allocated":1167360,"data_stored":829790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.61699999999999999}]},{"osd":1,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.63},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.64500000000000002}]},{"osd":2,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.68300000000000005}]},{"osd":3,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.502},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.73499999999999999}]},{"osd":4,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.44}]},{"osd":5,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.67000000000000004}]},{"osd":6,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.59899999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574873,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27588,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344252,"statfs":{"total":96636764160,"available":96608514048,"internally_reserved":0,"allocated":573440,"data_stored":239950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":21.359999999999999,"5min":21.359999999999999,"15min":21.359999999999999},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":10.329000000000001,"5min":10.329000000000001,"15min":10.329000000000001},"last":0.42799999999999999}]},{"osd":1,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":3.4239999999999999,"5min":3.4239999999999999,"15min":3.4239999999999999},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":3.4809999999999999,"5min":3.4809999999999999,"15min":3.4809999999999999},"last":0.45200000000000001}]},{"osd":2,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":10.398,"5min":10.398,"15min":10.398},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":5.0170000000000003,"5min":5.0170000000000003,"15min":5.0170000000000003},"last":0.374}]},{"osd":3,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":3.5710000000000002,"5min":3.5710000000000002,"15min":3.5710000000000002},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":10.368,"5min":10.368,"15min":10.368},"last":0.57099999999999995}]},{"osd":4,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.39200000000000002}]},{"osd":5,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.441},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.59099999999999997}]},{"osd":7,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.498}]}]},{"osd":1,"up_from":13,"seq":55834574874,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27588,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344252,"statfs":{"total":96636764160,"available":96608514048,"internally_reserved":0,"allocated":573440,"data_stored":239458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":1.389,"5min":1.389,"15min":1.389},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":9.1370000000000005,"5min":9.1370000000000005,"15min":9.1370000000000005},"last":0.48099999999999998}]},{"osd":2,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.433,"5min":0.433,"15min":0.433},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":1.425,"5min":1.425,"15min":1.425},"last":0.56399999999999995}]},{"osd":3,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":11.957000000000001,"5min":11.957000000000001,"15min":11.957000000000001},"last":0.53100000000000003},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":18.039000000000001,"5min":18.039000000000001,"15min":18.039000000000001},"last":0.54400000000000004}]},{"osd":4,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":6.181,"5min":6.181,"15min":6.181},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.488,"5min":1.488,"15min":1.488},"last":0.65400000000000003}]},{"osd":5,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":9.2420000000000009,"5min":9.2420000000000009,"15min":9.2420000000000009},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":6.2110000000000003,"5min":6.2110000000000003,"15min":6.2110000000000003},"last":0.84399999999999997}]},{"osd":6,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":9.1880000000000006,"5min":9.1880000000000006,"15min":9.1880000000000006},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.462,"5min":1.462,"15min":1.462},"last":0.82799999999999996}]},{"osd":7,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":6.1479999999999997,"5min":6.1479999999999997,"15min":6.1479999999999997},"last":0.76400000000000001},{"interface":"front","average":{"1min":1.028,"5min":1.028,"15min":1.028},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":9.2170000000000005,"5min":9.2170000000000005,"15min":9.2170000000000005},"last":0.66600000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574874,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28160,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343680,"statfs":{"total":96636764160,"available":96607928320,"internally_reserved":0,"allocated":1167360,"data_stored":829722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":4.1200000000000001,"5min":4.1200000000000001,"15min":4.1200000000000001},"last":0.378},{"interface":"front","average":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":16.940999999999999,"5min":16.940999999999999,"15min":16.940999999999999},"last":0.26800000000000002}]},{"osd":2,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":7.0209999999999999,"5min":7.0209999999999999,"15min":7.0209999999999999},"last":0.32500000000000001}]},{"osd":3,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":4.0940000000000003,"5min":4.0940000000000003,"15min":4.0940000000000003},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.42199999999999999}]},{"osd":4,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":4.0049999999999999,"5min":4.0049999999999999,"15min":4.0049999999999999},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.74199999999999999}]},{"osd":5,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":4.3090000000000002,"5min":4.3090000000000002,"15min":4.3090000000000002},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":4.0709999999999997,"5min":4.0709999999999997,"15min":4.0709999999999997},"last":0.58499999999999996}]},{"osd":6,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":4.2560000000000002,"5min":4.2560000000000002,"15min":4.2560000000000002},"last":0.76000000000000001}]},{"osd":7,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":4.2850000000000001,"5min":4.2850000000000001,"15min":4.2850000000000001},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":4.0460000000000003,"5min":4.0460000000000003,"15min":4.0460000000000003},"last":0.79000000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574874,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27592,"kb_used_data":564,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344248,"statfs":{"total":96636764160,"available":96608509952,"internally_reserved":0,"allocated":577536,"data_stored":240910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"min":{"1min":0.20100000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":6.0999999999999996,"5min":6.0999999999999996,"15min":6.0999999999999996},"last":0.29499999999999998}]},{"osd":1,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.624,"5min":0.624,"15min":0.624},"last":0.34399999999999997}]},{"osd":3,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.315}]},{"osd":4,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.73799999999999999}]},{"osd":5,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.70199999999999996}]},{"osd":6,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.56399999999999995}]},{"osd":7,"last update":"Wed Nov 8 19:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.754},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.59599999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574875,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27596,"kb_used_data":568,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344244,"statfs":{"total":96636764160,"available":96608505856,"internally_reserved":0,"allocated":581632,"data_stored":240932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:25:46 2023","interfaces":[{"interface":"back","average":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.626,"5min":0.626,"15min":0.626},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.315}]},{"osd":1,"last update":"Wed Nov 8 19:25:46 2023","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.47799999999999998}]},{"osd":2,"last update":"Wed Nov 8 19:25:46 2023","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.37,"15min":0.37},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"last":0.44500000000000001}]},{"osd":4,"last update":"Wed Nov 8 19:25:46 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.503}]},{"osd":5,"last update":"Wed Nov 8 19:25:46 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.51900000000000002}]},{"osd":6,"last update":"Wed Nov 8 19:25:46 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.66200000000000003}]},{"osd":7,"last update":"Wed Nov 8 19:25:46 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.432}]}]},{"osd":4,"up_from":13,"seq":55834574874,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27584,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344256,"statfs":{"total":96636764160,"available":96608518144,"internally_reserved":0,"allocated":569344,"data_stored":239880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:25:47 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":6.5970000000000004,"5min":6.5970000000000004,"15min":6.5970000000000004},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":2.5030000000000001,"5min":2.5030000000000001,"15min":2.5030000000000001},"last":0.65400000000000003}]},{"osd":1,"last update":"Wed Nov 8 19:25:47 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.54300000000000004}]},{"osd":2,"last update":"Wed Nov 8 19:25:47 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":6.6920000000000002,"5min":6.6920000000000002,"15min":6.6920000000000002},"last":0.58899999999999997}]},{"osd":3,"last update":"Wed Nov 8 19:25:47 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.51800000000000002}]},{"osd":5,"last update":"Wed Nov 8 19:25:47 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.44500000000000001}]},{"osd":6,"last update":"Wed Nov 8 19:25:47 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.497}]},{"osd":7,"last update":"Wed Nov 8 19:25:47 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":0.373},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.41799999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574874,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27580,"kb_used_data":552,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344260,"statfs":{"total":96636764160,"available":96608522240,"internally_reserved":0,"allocated":565248,"data_stored":239298,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.753},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.51300000000000001}]},{"osd":1,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.70699999999999996}]},{"osd":2,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.69599999999999995}]},{"osd":3,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.55700000000000005}]},{"osd":4,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.61699999999999999}]},{"osd":6,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.67000000000000004}]},{"osd":7,"last update":"Wed Nov 8 19:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.73699999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-08T19:26:34.783 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-08T19:26:34.783 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-08T19:26:34.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.940+0000 7fd5d113b700 1 Processor -- start 2023-11-08T19:26:34.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.940+0000 7fd5d113b700 1 -- start start 2023-11-08T19:26:34.944 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.941+0000 7fd5d113b700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd5cc06bd20 0x7fd5cc0d19c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:34.944 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.941+0000 7fd5d113b700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd5cc0d1f00 0x7fd5cc0cfd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:34.945 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.941+0000 7fd5d113b700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5cc06c2c0 con 0x7fd5cc15df20 2023-11-08T19:26:34.945 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.941+0000 7fd5d113b700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5cc0d2390 con 0x7fd5cc0d1f00 2023-11-08T19:26:34.945 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.941+0000 7fd5d113b700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5cc06c5a0 con 0x7fd5cc06bd20 2023-11-08T19:26:34.945 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.941+0000 7fd5cb59e700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd5cc0d1f00 0x7fd5cc0cfd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:34.945 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.941+0000 7fd5cb59e700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd5cc0d1f00 0x7fd5cc0cfd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51734/0 (socket says 172.21.15.143:51734) 2023-11-08T19:26:34.945 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.941+0000 7fd5cb59e700 1 -- 172.21.15.143:0/675634456 learned_addr learned my addr 172.21.15.143:0/675634456 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:34.946 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.942+0000 7fd5ca59c700 1 --2- 172.21.15.143:0/675634456 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd5cc06bd20 0x7fd5cc0d19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:34.946 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.942+0000 7fd5cb59e700 1 -- 172.21.15.143:0/675634456 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd5cc06bd20 msgr2=0x7fd5cc0d19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:34.946 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.942+0000 7fd5cb59e700 1 --2- 172.21.15.143:0/675634456 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd5cc06bd20 0x7fd5cc0d19c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:34.946 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.942+0000 7fd5cb59e700 1 -- 172.21.15.143:0/675634456 >> v1:172.21.15.174:6789/0 conn(0x7fd5cc15df20 legacy=0x7fd5cc0b25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:34.947 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.942+0000 7fd5cb59e700 1 -- 172.21.15.143:0/675634456 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5cc06c880 con 0x7fd5cc0d1f00 2023-11-08T19:26:34.947 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.942+0000 7fd5cb59e700 1 --2- 172.21.15.143:0/675634456 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd5cc0d1f00 0x7fd5cc0cfd30 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fd5c4009e80 tx=0x7fd5c400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6e005d21fbf8130 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:34.947 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.942+0000 7fd5c9d9b700 1 -- 172.21.15.143:0/675634456 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5c400e070 con 0x7fd5cc0d1f00 2023-11-08T19:26:34.947 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.942+0000 7fd5c9d9b700 1 -- 172.21.15.143:0/675634456 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5c400b980 con 0x7fd5cc0d1f00 2023-11-08T19:26:34.948 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.942+0000 7fd5c9d9b700 1 -- 172.21.15.143:0/675634456 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5c400bd30 con 0x7fd5cc0d1f00 2023-11-08T19:26:34.948 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.942+0000 7fd5d113b700 1 -- 172.21.15.143:0/675634456 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd5cc0d1f00 msgr2=0x7fd5cc0cfd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:34.948 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.942+0000 7fd5d113b700 1 --2- 172.21.15.143:0/675634456 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd5cc0d1f00 0x7fd5cc0cfd30 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fd5c4009e80 tx=0x7fd5c400b330 comp rx=0 tx=0).stop 2023-11-08T19:26:34.948 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.942+0000 7fd5d113b700 1 -- 172.21.15.143:0/675634456 shutdown_connections 2023-11-08T19:26:34.949 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.942+0000 7fd5d113b700 1 --2- 172.21.15.143:0/675634456 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd5cc0d1f00 0x7fd5cc0cfd30 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:34.949 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.942+0000 7fd5d113b700 1 --2- 172.21.15.143:0/675634456 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd5cc06bd20 0x7fd5cc0d19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:34.949 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.943+0000 7fd5d113b700 1 -- 172.21.15.143:0/675634456 >> 172.21.15.143:0/675634456 conn(0x7fd5cc08b000 msgr2=0x7fd5cc09ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:34.949 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.943+0000 7fd5d113b700 1 -- 172.21.15.143:0/675634456 shutdown_connections 2023-11-08T19:26:34.950 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.943+0000 7fd5d113b700 1 -- 172.21.15.143:0/675634456 wait complete. 2023-11-08T19:26:34.953 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.944+0000 7fd5d113b700 1 Processor -- start 2023-11-08T19:26:34.953 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.944+0000 7fd5d113b700 1 -- start start 2023-11-08T19:26:34.953 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.945+0000 7fd5d113b700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd5cc06bd20 0x7fd5cc160950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:34.953 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.945+0000 7fd5d113b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd5cc0d1f00 0x7fd5cc0c9510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:34.954 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.945+0000 7fd5d113b700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd5cc15df20 0x7fd5cc0d05d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:34.954 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.945+0000 7fd5d113b700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5cc0cb180 con 0x7fd5cc15df20 2023-11-08T19:26:34.954 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.945+0000 7fd5d113b700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5cc0cb400 con 0x7fd5cc0d1f00 2023-11-08T19:26:34.954 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.945+0000 7fd5d113b700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5cc0cb6e0 con 0x7fd5cc06bd20 2023-11-08T19:26:34.954 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.945+0000 7fd5cb59e700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd5cc15df20 0x7fd5cc0d05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:34.955 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.946+0000 7fd5cb59e700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd5cc15df20 0x7fd5cc0d05d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51740/0 (socket says 172.21.15.143:51740) 2023-11-08T19:26:34.955 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.946+0000 7fd5cb59e700 1 -- 172.21.15.143:0/1779595353 learned_addr learned my addr 172.21.15.143:0/1779595353 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:34.955 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.946+0000 7fd5cad9d700 1 --2- 172.21.15.143:0/1779595353 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd5cc06bd20 0x7fd5cc160950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:34.955 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.946+0000 7fd5ca59c700 1 --2- 172.21.15.143:0/1779595353 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd5cc0d1f00 0x7fd5cc0c9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:34.955 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.946+0000 7fd5cb59e700 1 -- 172.21.15.143:0/1779595353 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd5cc06bd20 msgr2=0x7fd5cc160950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:34.956 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.946+0000 7fd5cb59e700 1 --2- 172.21.15.143:0/1779595353 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd5cc06bd20 0x7fd5cc160950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:34.956 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.946+0000 7fd5cb59e700 1 -- 172.21.15.143:0/1779595353 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd5cc0d1f00 msgr2=0x7fd5cc0c9510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:34.956 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.946+0000 7fd5cb59e700 1 --2- 172.21.15.143:0/1779595353 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd5cc0d1f00 0x7fd5cc0c9510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:34.956 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.946+0000 7fd5cb59e700 1 -- 172.21.15.143:0/1779595353 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5c4009b30 con 0x7fd5cc15df20 2023-11-08T19:26:34.956 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.946+0000 7fd5ca59c700 1 --2- 172.21.15.143:0/1779595353 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd5cc0d1f00 0x7fd5cc0c9510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:26:34.957 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.946+0000 7fd5cb59e700 1 --2- 172.21.15.143:0/1779595353 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd5cc15df20 0x7fd5cc0d05d0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fd5cc06f1a0 tx=0x7fd5c4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:34.958 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.946+0000 7fd5b37fe700 1 -- 172.21.15.143:0/1779595353 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5c4004960 con 0x7fd5cc15df20 2023-11-08T19:26:34.958 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.946+0000 7fd5b37fe700 1 -- 172.21.15.143:0/1779595353 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5c40099b0 con 0x7fd5cc15df20 2023-11-08T19:26:34.958 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.946+0000 7fd5d113b700 1 -- 172.21.15.143:0/1779595353 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5cc0cb9c0 con 0x7fd5cc15df20 2023-11-08T19:26:34.959 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.946+0000 7fd5b37fe700 1 -- 172.21.15.143:0/1779595353 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5c401c680 con 0x7fd5cc15df20 2023-11-08T19:26:34.959 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.946+0000 7fd5d113b700 1 -- 172.21.15.143:0/1779595353 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5cc1f49e0 con 0x7fd5cc15df20 2023-11-08T19:26:34.959 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.947+0000 7fd5d113b700 1 -- 172.21.15.143:0/1779595353 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5cc1f4bd0 con 0x7fd5cc15df20 2023-11-08T19:26:34.959 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.947+0000 7fd5b37fe700 1 -- 172.21.15.143:0/1779595353 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd5c401c7e0 con 0x7fd5cc15df20 2023-11-08T19:26:34.960 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.948+0000 7fd5b37fe700 1 --2- 172.21.15.143:0/1779595353 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd5b806af00 0x7fd5b806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:34.960 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.948+0000 7fd5b37fe700 1 -- 172.21.15.143:0/1779595353 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd5c408b040 con 0x7fd5cc15df20 2023-11-08T19:26:34.960 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.948+0000 7fd5cad9d700 1 --2- 172.21.15.143:0/1779595353 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd5b806af00 0x7fd5b806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:34.960 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.948+0000 7fd5cad9d700 1 --2- 172.21.15.143:0/1779595353 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd5b806af00 0x7fd5b806d3c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fd5c0000f60 tx=0x7fd5c0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:34.961 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:34.952+0000 7fd5b37fe700 1 -- 172.21.15.143:0/1779595353 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd5c4056020 con 0x7fd5cc15df20 2023-11-08T19:26:35.222 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:35.221+0000 7fd5d113b700 1 -- 172.21.15.143:0/1779595353 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fd5cc04eee0 con 0x7fd5cc15df20 2023-11-08T19:26:35.224 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:35.223+0000 7fd5b37fe700 1 -- 172.21.15.143:0/1779595353 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v10) v1 ==== 190+0+147 (secure 0 0 0) 0x7fd5c4059640 con 0x7fd5cc15df20 2023-11-08T19:26:35.225 INFO:teuthology.orchestra.run.smithi143.stdout:[client.0] 2023-11-08T19:26:35.225 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQBP4Etl0GdSFxAAVkuHL0EnjrtTUMkj/afPkg== 2023-11-08T19:26:35.225 INFO:teuthology.orchestra.run.smithi143.stdout: caps mds = "allow" 2023-11-08T19:26:35.226 INFO:teuthology.orchestra.run.smithi143.stdout: caps mon = "allow r" 2023-11-08T19:26:35.226 INFO:teuthology.orchestra.run.smithi143.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-11-08T19:26:35.228 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:35.226+0000 7fd5d113b700 1 -- 172.21.15.143:0/1779595353 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd5b806af00 msgr2=0x7fd5b806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:35.228 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:35.226+0000 7fd5d113b700 1 --2- 172.21.15.143:0/1779595353 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd5b806af00 0x7fd5b806d3c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fd5c0000f60 tx=0x7fd5c0009040 comp rx=0 tx=0).stop 2023-11-08T19:26:35.228 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:35.226+0000 7fd5d113b700 1 -- 172.21.15.143:0/1779595353 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd5cc15df20 msgr2=0x7fd5cc0d05d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:35.228 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:35.226+0000 7fd5d113b700 1 --2- 172.21.15.143:0/1779595353 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd5cc15df20 0x7fd5cc0d05d0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fd5cc06f1a0 tx=0x7fd5c4004ad0 comp rx=0 tx=0).stop 2023-11-08T19:26:35.229 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:35.226+0000 7fd5d113b700 1 -- 172.21.15.143:0/1779595353 shutdown_connections 2023-11-08T19:26:35.229 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:35.226+0000 7fd5d113b700 1 --2- 172.21.15.143:0/1779595353 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd5b806af00 0x7fd5b806d3c0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:35.229 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:35.226+0000 7fd5d113b700 1 --2- 172.21.15.143:0/1779595353 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd5cc06bd20 0x7fd5cc160950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:35.229 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:35.226+0000 7fd5d113b700 1 --2- 172.21.15.143:0/1779595353 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd5cc15df20 0x7fd5cc0d05d0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:35.230 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:35.226+0000 7fd5d113b700 1 --2- 172.21.15.143:0/1779595353 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd5cc0d1f00 0x7fd5cc0c9510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:35.230 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:35.226+0000 7fd5d113b700 1 -- 172.21.15.143:0/1779595353 >> 172.21.15.143:0/1779595353 conn(0x7fd5cc08b000 msgr2=0x7fd5cc15cfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:35.230 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:35.226+0000 7fd5d113b700 1 -- 172.21.15.143:0/1779595353 shutdown_connections 2023-11-08T19:26:35.230 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:35.226+0000 7fd5d113b700 1 -- 172.21.15.143:0/1779595353 wait complete. 2023-11-08T19:26:35.231 INFO:teuthology.orchestra.run.smithi143.stderr:updated caps for client.0 2023-11-08T19:26:35.239 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:26:35.359 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:35.358+0000 7fb0ef5f6700 1 Processor -- start 2023-11-08T19:26:35.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:35.358+0000 7fb0ef5f6700 1 -- start start 2023-11-08T19:26:35.363 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:35.359+0000 7fb0ef5f6700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb0e806b320 0x7fb0e8168c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:35.364 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:35.359+0000 7fb0ef5f6700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0e8161df0 con 0x7fb0e815dd00 2023-11-08T19:26:35.364 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:35.359+0000 7fb0ef5f6700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0e81620d0 con 0x7fb0e81618f0 2023-11-08T19:26:35.364 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:35.359+0000 7fb0ef5f6700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e80cad50 con 0x7fb0e806b320 2023-11-08T19:26:35.364 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:35.359+0000 7fb0ed392700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fb0e815dd00 0x7fb0e80cca70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:47898/0 (socket says 172.21.15.143:47898) 2023-11-08T19:26:35.365 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:35.359+0000 7fb0ed392700 1 -- 172.21.15.143:0/3788108018 learned_addr learned my addr 172.21.15.143:0/3788108018 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:26:35.365 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:35.359+0000 7fb0edb93700 1 --2- 172.21.15.143:0/3788108018 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb0e806b320 0x7fb0e8168c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:35.365 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:35.359+0000 7fb0dffff700 1 -- 172.21.15.143:0/3788108018 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2520864434 0 0) 0x7fb0e8161df0 con 0x7fb0e815dd00 2023-11-08T19:26:35.365 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:35.359+0000 7fb0dffff700 1 -- 172.21.15.143:0/3788108018 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb0d40035b0 con 0x7fb0e815dd00 2023-11-08T19:26:35.365 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:35.360+0000 7fb0dffff700 1 -- 172.21.15.143:0/3788108018 <== mon.1 v1:172.21.15.174:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3306196167 0 0) 0x7fb0e81620d0 con 0x7fb0e81618f0 2023-11-08T19:26:35.366 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:35.360+0000 7fb0dffff700 1 -- 172.21.15.143:0/3788108018 --> v1:172.21.15.174:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb0e8161df0 con 0x7fb0e81618f0 2023-11-08T19:26:35.366 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:35.360+0000 7fb0dffff700 1 -- 172.21.15.143:0/3788108018 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 862388749 0 0) 0x7fb0d40035b0 con 0x7fb0e815dd00 2023-11-08T19:26:35.366 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:35.360+0000 7fb0dffff700 1 -- 172.21.15.143:0/3788108018 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb0e806b320 msgr2=0x7fb0e8168c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:35.366 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:35.360+0000 7fb0dffff700 1 --2- 172.21.15.143:0/3788108018 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb0e806b320 0x7fb0e8168c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:35.366 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:35.360+0000 7fb0dffff700 1 -- 172.21.15.143:0/3788108018 >> v1:172.21.15.174:6789/0 conn(0x7fb0e81618f0 legacy=0x7fb0e806ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:35.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:35.360+0000 7fb0dffff700 1 -- 172.21.15.143:0/3788108018 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0e81623b0 con 0x7fb0e815dd00 2023-11-08T19:26:35.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:35.360+0000 7fb0dffff700 1 -- 172.21.15.143:0/3788108018 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4147756303 0 0) 0x7fb0d80017f0 con 0x7fb0e815dd00 2023-11-08T19:26:35.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:35.360+0000 7fb0dffff700 1 -- 172.21.15.143:0/3788108018 >> v1:172.21.15.143:6789/0 conn(0x7fb0e815dd00 legacy=0x7fb0e80cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:35.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:35.360+0000 7fb0dffff700 1 --2- 172.21.15.143:0/3788108018 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb0d4003bc0 0x7fb0d4003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:35.369 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:35.360+0000 7fb0dffff700 1 --2- 172.21.15.143:0/3788108018 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb0d4004850 0x7fb0d4004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:35.369 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:35.360+0000 7fb0dffff700 1 --2- 172.21.15.143:0/3788108018 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0d4005210 0x7fb0d400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:35.369 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:35.360+0000 7fb0dffff700 1 -- 172.21.15.143:0/3788108018 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0d400c000 con 0x7fb0d4003bc0 2023-11-08T19:26:35.369 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:35.360+0000 7fb0dffff700 1 -- 172.21.15.143:0/3788108018 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0d400c140 con 0x7fb0d4005210 2023-11-08T19:26:35.370 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:35.360+0000 7fb0dffff700 1 -- 172.21.15.143:0/3788108018 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0d400c300 con 0x7fb0d4004850 2023-11-08T19:26:35.370 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:35.360+0000 7fb0edb93700 1 --2- 172.21.15.143:0/3788108018 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb0e806b320 0x7fb0e8168c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T19:26:35.370 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:35.360+0000 7fb0ed392700 1 --2- 172.21.15.143:0/3788108018 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb0d4003bc0 0x7fb0d4003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:35.370 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:35.360+0000 7fb0ecb91700 1 --2- 172.21.15.143:0/3788108018 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb0d4004850 0x7fb0d4004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:35.370 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:35.360+0000 7fb0edb93700 1 --2- 172.21.15.143:0/3788108018 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0d4005210 0x7fb0d400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:35.371 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:35.360+0000 7fb0ed392700 1 -- 172.21.15.143:0/3788108018 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb0d4004850 msgr2=0x7fb0d4004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:35.371 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:35.360+0000 7fb0ed392700 1 --2- 172.21.15.143:0/3788108018 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb0d4004850 0x7fb0d4004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:35.371 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:35.360+0000 7fb0ed392700 1 -- 172.21.15.143:0/3788108018 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0d4005210 msgr2=0x7fb0d400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:35.371 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:35.360+0000 7fb0ed392700 1 --2- 172.21.15.143:0/3788108018 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0d4005210 0x7fb0d400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:35.371 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:35.360+0000 7fb0ed392700 1 -- 172.21.15.143:0/3788108018 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0d40035b0 con 0x7fb0d4003bc0 2023-11-08T19:26:35.372 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:35.360+0000 7fb0edb93700 1 --2- 172.21.15.143:0/3788108018 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0d4005210 0x7fb0d400ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:26:35.372 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:35.360+0000 7fb0ed392700 1 --2- 172.21.15.143:0/3788108018 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb0d4003bc0 0x7fb0d4003fc0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fb0d800c240 tx=0x7fb0d4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fa63a4ad5e1eedb server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:35.372 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:35.360+0000 7fb0dffff700 1 -- 172.21.15.143:0/3788108018 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d800cd80 con 0x7fb0d4003bc0 2023-11-08T19:26:35.372 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:35.361+0000 7fb0dffff700 1 -- 172.21.15.143:0/3788108018 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0d8007b70 con 0x7fb0d4003bc0 2023-11-08T19:26:35.372 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:35.361+0000 7fb0dffff700 1 -- 172.21.15.143:0/3788108018 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d8007310 con 0x7fb0d4003bc0 2023-11-08T19:26:35.373 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:35.361+0000 7fb0ef5f6700 1 -- 172.21.15.143:0/3788108018 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb0d4003bc0 msgr2=0x7fb0d4003fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:35.373 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:35.361+0000 7fb0ef5f6700 1 --2- 172.21.15.143:0/3788108018 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb0d4003bc0 0x7fb0d4003fc0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fb0d800c240 tx=0x7fb0d4005880 comp rx=0 tx=0).stop 2023-11-08T19:26:35.373 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:35.361+0000 7fb0edb93700 1 -- 172.21.15.143:0/3788108018 reap_dead start 2023-11-08T19:26:35.373 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:35.361+0000 7fb0ef5f6700 1 -- 172.21.15.143:0/3788108018 shutdown_connections 2023-11-08T19:26:35.373 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:35.361+0000 7fb0ef5f6700 1 -- 172.21.15.143:0/3788108018 >> 172.21.15.143:0/3788108018 conn(0x7fb0e8099ab0 msgr2=0x7fb0e80d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:35.374 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:35.361+0000 7fb0ef5f6700 1 -- 172.21.15.143:0/3788108018 shutdown_connections 2023-11-08T19:26:35.374 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:35.361+0000 7fb0ef5f6700 1 -- 172.21.15.143:0/3788108018 wait complete. 2023-11-08T19:26:35.374 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:35.362+0000 7fb0ef5f6700 1 Processor -- start 2023-11-08T19:26:35.374 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:35.362+0000 7fb0ef5f6700 1 -- start start 2023-11-08T19:26:35.377 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:35.363+0000 7fb0ef5f6700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb0d4003bc0 0x7fb0e8157e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:35.377 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:35.363+0000 7fb0ef5f6700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0e80b0ed0 0x7fb0e80b2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:35.377 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:35.363+0000 7fb0ef5f6700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb0e80c5fc0 0x7fb0e80c6470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:35.377 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:35.363+0000 7fb0ef5f6700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e80c69b0 con 0x7fb0d4003bc0 2023-11-08T19:26:35.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:35.363+0000 7fb0ef5f6700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e8158370 con 0x7fb0e80b0ed0 2023-11-08T19:26:35.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:35.363+0000 7fb0ef5f6700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e80b30d0 con 0x7fb0e80c5fc0 2023-11-08T19:26:35.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:35.363+0000 7fb0ed392700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb0d4003bc0 0x7fb0e8157e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:35.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:35.363+0000 7fb0ed392700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb0d4003bc0 0x7fb0e8157e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51756/0 (socket says 172.21.15.143:51756) 2023-11-08T19:26:35.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:35.363+0000 7fb0ed392700 1 -- 172.21.15.143:0/1669108552 learned_addr learned my addr 172.21.15.143:0/1669108552 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:35.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:35.363+0000 7fb0ed392700 1 -- 172.21.15.143:0/1669108552 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb0e80c5fc0 msgr2=0x7fb0e80c6470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:35.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:35.363+0000 7fb0edb93700 1 --2- 172.21.15.143:0/1669108552 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb0e80c5fc0 0x7fb0e80c6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:35.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:35.363+0000 7fb0ed392700 1 --2- 172.21.15.143:0/1669108552 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb0e80c5fc0 0x7fb0e80c6470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:35.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:35.363+0000 7fb0ed392700 1 -- 172.21.15.143:0/1669108552 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0e80b0ed0 msgr2=0x7fb0e80b2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:35.380 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:35.363+0000 7fb0ed392700 1 --2- 172.21.15.143:0/1669108552 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0e80b0ed0 0x7fb0e80b2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:35.380 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:35.363+0000 7fb0ed392700 1 -- 172.21.15.143:0/1669108552 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0e815e260 con 0x7fb0d4003bc0 2023-11-08T19:26:35.380 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:35.364+0000 7fb0ed392700 1 --2- 172.21.15.143:0/1669108552 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb0d4003bc0 0x7fb0e8157e30 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fb0d8015040 tx=0x7fb0d8001b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:35.380 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:35.364+0000 7fb0ddffb700 1 -- 172.21.15.143:0/1669108552 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d8005d00 con 0x7fb0d4003bc0 2023-11-08T19:26:35.380 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:35.364+0000 7fb0ddffb700 1 -- 172.21.15.143:0/1669108552 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0d8005e60 con 0x7fb0d4003bc0 2023-11-08T19:26:35.381 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:35.364+0000 7fb0ef5f6700 1 -- 172.21.15.143:0/1669108552 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0e80b1660 con 0x7fb0d4003bc0 2023-11-08T19:26:35.381 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:35.364+0000 7fb0ddffb700 1 -- 172.21.15.143:0/1669108552 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d8007310 con 0x7fb0d4003bc0 2023-11-08T19:26:35.381 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:35.364+0000 7fb0ef5f6700 1 -- 172.21.15.143:0/1669108552 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0e80b1c20 con 0x7fb0d4003bc0 2023-11-08T19:26:35.381 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:35.365+0000 7fb0ef5f6700 1 -- 172.21.15.143:0/1669108552 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0e815d860 con 0x7fb0d4003bc0 2023-11-08T19:26:35.381 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:35.366+0000 7fb0ddffb700 1 -- 172.21.15.143:0/1669108552 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb0d8019070 con 0x7fb0d4003bc0 2023-11-08T19:26:35.382 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:35.366+0000 7fb0ddffb700 1 --2- 172.21.15.143:0/1669108552 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fb0cc06af00 0x7fb0cc06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:35.382 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:35.366+0000 7fb0ddffb700 1 -- 172.21.15.143:0/1669108552 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb0d808aa90 con 0x7fb0d4003bc0 2023-11-08T19:26:35.382 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:35.366+0000 7fb0ecb91700 1 --2- 172.21.15.143:0/1669108552 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fb0cc06af00 0x7fb0cc06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:35.382 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:35.367+0000 7fb0ecb91700 1 --2- 172.21.15.143:0/1669108552 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fb0cc06af00 0x7fb0cc06d3c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fb0e80ceb10 tx=0x7fb0e4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:35.382 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:35.370+0000 7fb0ddffb700 1 -- 172.21.15.143:0/1669108552 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb0d8055a70 con 0x7fb0d4003bc0 2023-11-08T19:26:35.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:35.602+0000 7fb0ef5f6700 1 -- 172.21.15.143:0/1669108552 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb0e804eee0 con 0x7fb0d4003bc0 2023-11-08T19:26:35.604 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:35.602+0000 7fb0ddffb700 1 -- 172.21.15.143:0/1669108552 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4925 (secure 0 0 0) 0x7fb0d8017090 con 0x7fb0d4003bc0 2023-11-08T19:26:35.604 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:26:35.604 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4988,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3140991763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":3140991763},{"type":"v1","addr":"172.21.15.143:6835","nonce":3140991763}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":4997,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6836/2388011942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2388011942},{"type":"v1","addr":"172.21.15.174:6836","nonce":2388011942}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5000,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/1339111936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6835","nonce":1339111936},{"type":"v1","addr":"172.21.15.174:6837","nonce":1339111936}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T19:26:25.128617+0000","modified":"2023-11-08T19:26:28.156717+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5242},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5242":{"gid":5242,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.143:6834/3591097261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":3591097261},{"type":"v1","addr":"172.21.15.143:6834","nonce":3591097261}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-08T19:26:35.607 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:35.604+0000 7fb0ef5f6700 1 -- 172.21.15.143:0/1669108552 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fb0cc06af00 msgr2=0x7fb0cc06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:35.607 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:35.604+0000 7fb0ef5f6700 1 --2- 172.21.15.143:0/1669108552 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fb0cc06af00 0x7fb0cc06d3c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fb0e80ceb10 tx=0x7fb0e4009040 comp rx=0 tx=0).stop 2023-11-08T19:26:35.607 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:35.604+0000 7fb0ef5f6700 1 -- 172.21.15.143:0/1669108552 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb0d4003bc0 msgr2=0x7fb0e8157e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:35.607 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:35.604+0000 7fb0ef5f6700 1 --2- 172.21.15.143:0/1669108552 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb0d4003bc0 0x7fb0e8157e30 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fb0d8015040 tx=0x7fb0d8001b00 comp rx=0 tx=0).stop 2023-11-08T19:26:35.608 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:35.604+0000 7fb0ef5f6700 1 -- 172.21.15.143:0/1669108552 shutdown_connections 2023-11-08T19:26:35.608 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:35.604+0000 7fb0ef5f6700 1 --2- 172.21.15.143:0/1669108552 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fb0cc06af00 0x7fb0cc06d3c0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:35.608 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:35.604+0000 7fb0ef5f6700 1 --2- 172.21.15.143:0/1669108552 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb0e80c5fc0 0x7fb0e80c6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:35.608 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:35.604+0000 7fb0ef5f6700 1 --2- 172.21.15.143:0/1669108552 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb0d4003bc0 0x7fb0e8157e30 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:35.609 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:35.604+0000 7fb0ef5f6700 1 --2- 172.21.15.143:0/1669108552 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0e80b0ed0 0x7fb0e80b2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:35.609 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:35.604+0000 7fb0ef5f6700 1 -- 172.21.15.143:0/1669108552 >> 172.21.15.143:0/1669108552 conn(0x7fb0e8099ab0 msgr2=0x7fb0e80ad450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:35.609 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:35.604+0000 7fb0ef5f6700 1 -- 172.21.15.143:0/1669108552 shutdown_connections 2023-11-08T19:26:35.609 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:35.604+0000 7fb0ef5f6700 1 -- 172.21.15.143:0/1669108552 wait complete. 2023-11-08T19:26:35.609 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 25 2023-11-08T19:26:36.020 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-08T19:26:25.128617+0000', 'modified': '2023-11-08T19:26:28.156717+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5242}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5242': {'gid': 5242, 'name': 'c', 'rank': 0, 'incarnation': 21, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.143:6834/3591097261', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.143:6832', 'nonce': 3591097261}, {'type': 'v1', 'addr': '172.21.15.143:6834', 'nonce': 3591097261}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-08T19:26:36.021 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-08T19:26:36.022 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-11-08T19:26:36.143 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.141+0000 7f4891dd1700 1 Processor -- start 2023-11-08T19:26:36.144 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.142+0000 7f4891dd1700 1 -- start start 2023-11-08T19:26:36.149 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.142+0000 7f4891dd1700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f488c15dd00 0x7f488c0cca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:36.149 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.142+0000 7f4891dd1700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f488c1618f0 0x7f488c06ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:36.149 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.142+0000 7f4891dd1700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f488c06b150 0x7f488c0b2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:36.149 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.142+0000 7f4891dd1700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f488c06b600 con 0x7f488c06b150 2023-11-08T19:26:36.150 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.142+0000 7f4891dd1700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f488c0cad50 con 0x7f488c15dd00 2023-11-08T19:26:36.150 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.142+0000 7f4891dd1700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f488c0ccfb0 con 0x7f488c1618f0 2023-11-08T19:26:36.150 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.142+0000 7f488bfff700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f488c06b150 0x7f488c0b2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:36.150 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.142+0000 7f488bfff700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f488c06b150 0x7f488c0b2210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51766/0 (socket says 172.21.15.143:51766) 2023-11-08T19:26:36.150 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.142+0000 7f488bfff700 1 -- 172.21.15.143:0/3438927189 learned_addr learned my addr 172.21.15.143:0/3438927189 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:36.151 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.143+0000 7f488b7fe700 1 --2- 172.21.15.143:0/3438927189 >> v2:172.21.15.174:3300/0 conn(0x7f488c15dd00 0x7f488c0cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:36.151 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.143+0000 7f488affd700 1 --2- 172.21.15.143:0/3438927189 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f488c1618f0 0x7f488c06ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:36.151 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.143+0000 7f488bfff700 1 -- 172.21.15.143:0/3438927189 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f488c1618f0 msgr2=0x7f488c06ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:36.151 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.143+0000 7f488bfff700 1 --2- 172.21.15.143:0/3438927189 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f488c1618f0 0x7f488c06ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:36.152 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.143+0000 7f488bfff700 1 -- 172.21.15.143:0/3438927189 >> v2:172.21.15.174:3300/0 conn(0x7f488c15dd00 msgr2=0x7f488c0cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:36.152 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.143+0000 7f488bfff700 1 --2- 172.21.15.143:0/3438927189 >> v2:172.21.15.174:3300/0 conn(0x7f488c15dd00 0x7f488c0cca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:36.152 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.143+0000 7f488bfff700 1 -- 172.21.15.143:0/3438927189 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f488c161df0 con 0x7f488c06b150 2023-11-08T19:26:36.152 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.143+0000 7f488bfff700 1 --2- 172.21.15.143:0/3438927189 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f488c06b150 0x7f488c0b2210 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f488c070a60 tx=0x7f488400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e63083ffdd95a49b server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:36.153 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.143+0000 7f488a7fc700 1 -- 172.21.15.143:0/3438927189 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f488400e070 con 0x7f488c06b150 2023-11-08T19:26:36.153 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.143+0000 7f488a7fc700 1 -- 172.21.15.143:0/3438927189 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f488400b9c0 con 0x7f488c06b150 2023-11-08T19:26:36.153 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.143+0000 7f488a7fc700 1 -- 172.21.15.143:0/3438927189 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f488400bd70 con 0x7f488c06b150 2023-11-08T19:26:36.153 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.144+0000 7f4891dd1700 1 -- 172.21.15.143:0/3438927189 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f488c06b150 msgr2=0x7f488c0b2210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:36.154 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.144+0000 7f4891dd1700 1 --2- 172.21.15.143:0/3438927189 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f488c06b150 0x7f488c0b2210 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f488c070a60 tx=0x7f488400b330 comp rx=0 tx=0).stop 2023-11-08T19:26:36.154 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.144+0000 7f4891dd1700 1 -- 172.21.15.143:0/3438927189 shutdown_connections 2023-11-08T19:26:36.154 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.144+0000 7f4891dd1700 1 --2- 172.21.15.143:0/3438927189 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f488c06b150 0x7f488c0b2210 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:36.154 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.144+0000 7f4891dd1700 1 --2- 172.21.15.143:0/3438927189 >> v2:172.21.15.174:3300/0 conn(0x7f488c15dd00 0x7f488c0cca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:36.154 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.144+0000 7f4891dd1700 1 --2- 172.21.15.143:0/3438927189 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f488c1618f0 0x7f488c06ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:36.155 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.144+0000 7f4891dd1700 1 -- 172.21.15.143:0/3438927189 >> 172.21.15.143:0/3438927189 conn(0x7f488c099ab0 msgr2=0x7f488c0d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:36.155 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.144+0000 7f4891dd1700 1 -- 172.21.15.143:0/3438927189 shutdown_connections 2023-11-08T19:26:36.155 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.144+0000 7f4891dd1700 1 -- 172.21.15.143:0/3438927189 wait complete. 2023-11-08T19:26:36.155 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.145+0000 7f4891dd1700 1 Processor -- start 2023-11-08T19:26:36.155 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.145+0000 7f4891dd1700 1 -- start start 2023-11-08T19:26:36.156 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.146+0000 7f4891dd1700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f488c06b150 0x7f488c1f7590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:36.156 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.146+0000 7f4891dd1700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f488c15dd00 0x7f488c1f7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:36.156 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.146+0000 7f4891dd1700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f488c1618f0 0x7f488c0c50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:36.156 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.146+0000 7f4891dd1700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f488c0b62f0 con 0x7f488c06b150 2023-11-08T19:26:36.157 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.146+0000 7f4891dd1700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f488c0c5d10 con 0x7f488c1618f0 2023-11-08T19:26:36.157 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.146+0000 7f4891dd1700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f488c0c5ff0 con 0x7f488c15dd00 2023-11-08T19:26:36.157 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.146+0000 7f488b7fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f488c06b150 0x7f488c1f7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:36.157 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.146+0000 7f488b7fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f488c06b150 0x7f488c1f7590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51768/0 (socket says 172.21.15.143:51768) 2023-11-08T19:26:36.157 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.146+0000 7f488b7fe700 1 -- 172.21.15.143:0/249050111 learned_addr learned my addr 172.21.15.143:0/249050111 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:36.158 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.146+0000 7f488affd700 1 --2- 172.21.15.143:0/249050111 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f488c15dd00 0x7f488c1f7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:36.158 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.146+0000 7f488bfff700 1 --2- 172.21.15.143:0/249050111 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f488c1618f0 0x7f488c0c50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:36.158 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.146+0000 7f488b7fe700 1 -- 172.21.15.143:0/249050111 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f488c15dd00 msgr2=0x7f488c1f7ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:36.158 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.146+0000 7f488b7fe700 1 --2- 172.21.15.143:0/249050111 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f488c15dd00 0x7f488c1f7ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:36.159 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.146+0000 7f488b7fe700 1 -- 172.21.15.143:0/249050111 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f488c1618f0 msgr2=0x7f488c0c50d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:36.159 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.146+0000 7f488b7fe700 1 --2- 172.21.15.143:0/249050111 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f488c1618f0 0x7f488c0c50d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:36.159 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.146+0000 7f488b7fe700 1 -- 172.21.15.143:0/249050111 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4884009b30 con 0x7f488c06b150 2023-11-08T19:26:36.160 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.146+0000 7f488b7fe700 1 --2- 172.21.15.143:0/249050111 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f488c06b150 0x7f488c1f7590 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f488000e7e0 tx=0x7f488000eba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:36.160 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.146+0000 7f4873fff700 1 -- 172.21.15.143:0/249050111 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4880013610 con 0x7f488c06b150 2023-11-08T19:26:36.160 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.146+0000 7f4873fff700 1 -- 172.21.15.143:0/249050111 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4880013770 con 0x7f488c06b150 2023-11-08T19:26:36.161 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.146+0000 7f4891dd1700 1 -- 172.21.15.143:0/249050111 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f488c0b0390 con 0x7f488c06b150 2023-11-08T19:26:36.161 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.146+0000 7f4873fff700 1 -- 172.21.15.143:0/249050111 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4880013b00 con 0x7f488c06b150 2023-11-08T19:26:36.161 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.146+0000 7f4891dd1700 1 -- 172.21.15.143:0/249050111 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f488c0b0950 con 0x7f488c06b150 2023-11-08T19:26:36.170 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.148+0000 7f4873fff700 1 -- 172.21.15.143:0/249050111 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4880013c60 con 0x7f488c06b150 2023-11-08T19:26:36.170 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.148+0000 7f4873fff700 1 --2- 172.21.15.143:0/249050111 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f487806af00 0x7f487806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:36.170 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.148+0000 7f4873fff700 1 -- 172.21.15.143:0/249050111 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f48800896f0 con 0x7f488c06b150 2023-11-08T19:26:36.170 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.148+0000 7f488affd700 1 --2- 172.21.15.143:0/249050111 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f487806af00 0x7f487806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:36.171 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.149+0000 7f488affd700 1 --2- 172.21.15.143:0/249050111 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f487806af00 0x7f487806d3c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f487c009930 tx=0x7f487c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:36.171 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.150+0000 7f489dfcab80 1 Processor -- start 2023-11-08T19:26:36.171 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.150+0000 7f489dfcab80 1 -- start start 2023-11-08T19:26:36.171 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.151+0000 7f489dfcab80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c07ae906f0 0x55c07aec0450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:36.172 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.151+0000 7f489dfcab80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c07ae92c30 0x55c07aec0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:36.172 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.151+0000 7f489dfcab80 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55c07ae90ad0 0x55c07aeaaf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:36.172 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.151+0000 7f489dfcab80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c07adfc6e0 con 0x55c07ae906f0 2023-11-08T19:26:36.172 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.151+0000 7f489dfcab80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c07adf03b0 con 0x55c07ae92c30 2023-11-08T19:26:36.172 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.151+0000 7f489dfcab80 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c07adedcd0 con 0x55c07ae90ad0 2023-11-08T19:26:36.173 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.151+0000 7f488b7fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c07ae906f0 0x55c07aec0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:36.173 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.151+0000 7f488b7fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c07ae906f0 0x55c07aec0450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51778/0 (socket says 172.21.15.143:51778) 2023-11-08T19:26:36.174 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.151+0000 7f488b7fe700 1 -- 172.21.15.143:0/944941043 learned_addr learned my addr 172.21.15.143:0/944941043 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:36.174 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.151+0000 7f488affd700 1 --2- 172.21.15.143:0/944941043 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c07ae92c30 0x55c07aec0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:36.174 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.151+0000 7f488bfff700 1 --2- 172.21.15.143:0/944941043 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55c07ae90ad0 0x55c07aeaaf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:36.174 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.151+0000 7f488b7fe700 1 -- 172.21.15.143:0/944941043 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55c07ae90ad0 msgr2=0x55c07aeaaf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:36.175 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.151+0000 7f488b7fe700 1 --2- 172.21.15.143:0/944941043 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55c07ae90ad0 0x55c07aeaaf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:36.175 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.151+0000 7f488b7fe700 1 -- 172.21.15.143:0/944941043 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c07ae92c30 msgr2=0x55c07aec0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:36.175 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.151+0000 7f488b7fe700 1 --2- 172.21.15.143:0/944941043 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c07ae92c30 0x55c07aec0990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:36.175 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.152+0000 7f488b7fe700 1 -- 172.21.15.143:0/944941043 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c07ac665e0 con 0x55c07ae906f0 2023-11-08T19:26:36.176 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.152+0000 7f488b7fe700 1 --2- 172.21.15.143:0/944941043 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c07ae906f0 0x55c07aec0450 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f4880006ac0 tx=0x7f4880058a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5efdd079d70e194d server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:36.176 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.152+0000 7f48717fa700 1 -- 172.21.15.143:0/944941043 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f488005c450 con 0x55c07ae906f0 2023-11-08T19:26:36.176 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.152+0000 7f48717fa700 1 -- 172.21.15.143:0/944941043 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4880060600 con 0x55c07ae906f0 2023-11-08T19:26:36.176 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.152+0000 7f48717fa700 1 -- 172.21.15.143:0/944941043 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f488005c0d0 con 0x55c07ae906f0 2023-11-08T19:26:36.177 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.152+0000 7f489dfcab80 1 -- 172.21.15.143:0/944941043 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c07ae906f0 msgr2=0x55c07aec0450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:36.177 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.152+0000 7f489dfcab80 1 --2- 172.21.15.143:0/944941043 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c07ae906f0 0x55c07aec0450 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f4880006ac0 tx=0x7f4880058a80 comp rx=0 tx=0).stop 2023-11-08T19:26:36.177 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.153+0000 7f489dfcab80 1 -- 172.21.15.143:0/944941043 shutdown_connections 2023-11-08T19:26:36.177 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.153+0000 7f489dfcab80 1 --2- 172.21.15.143:0/944941043 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55c07ae90ad0 0x55c07aeaaf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:36.178 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.153+0000 7f489dfcab80 1 --2- 172.21.15.143:0/944941043 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c07ae906f0 0x55c07aec0450 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:36.178 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.153+0000 7f489dfcab80 1 --2- 172.21.15.143:0/944941043 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c07ae92c30 0x55c07aec0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:36.178 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.153+0000 7f489dfcab80 1 -- 172.21.15.143:0/944941043 >> 172.21.15.143:0/944941043 conn(0x55c07ae0b660 msgr2=0x55c07ae963c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:36.178 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.153+0000 7f489dfcab80 1 -- 172.21.15.143:0/944941043 shutdown_connections 2023-11-08T19:26:36.179 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.153+0000 7f489dfcab80 1 -- 172.21.15.143:0/944941043 wait complete. 2023-11-08T19:26:36.179 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.153+0000 7f489dfcab80 1 -- start start 2023-11-08T19:26:36.179 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.153+0000 7f489dfcab80 1 Processor -- start 2023-11-08T19:26:36.179 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.154+0000 7f489dfcab80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c07ae906f0 0x55c07aeed820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:36.180 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.154+0000 7f489dfcab80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c07ae90ad0 0x55c07aed35e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:36.180 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.154+0000 7f489dfcab80 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55c07aed3b20 0x55c07aed5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:36.180 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.154+0000 7f489dfcab80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c07ade95c0 con 0x55c07ae90ad0 2023-11-08T19:26:36.180 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.154+0000 7f489dfcab80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c07ad850d0 con 0x55c07ae906f0 2023-11-08T19:26:36.180 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.154+0000 7f489dfcab80 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c07ad84d70 con 0x55c07aed3b20 2023-11-08T19:26:36.181 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.154+0000 7f489dfcab80 10 client.0 _ll_register_callbacks cb 0x55c07abe17b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T19:26:36.181 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.154+0000 7f488affd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c07ae90ad0 0x55c07aed35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:36.181 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.154+0000 7f488affd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c07ae90ad0 0x55c07aed35e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51784/0 (socket says 172.21.15.143:51784) 2023-11-08T19:26:36.181 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.154+0000 7f488affd700 1 -- 172.21.15.143:0/3640570032 learned_addr learned my addr 172.21.15.143:0/3640570032 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:36.181 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.154+0000 7f488b7fe700 1 --2- 172.21.15.143:0/3640570032 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c07ae906f0 0x55c07aeed820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:36.182 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.154+0000 7f488bfff700 1 --2- 172.21.15.143:0/3640570032 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55c07aed3b20 0x55c07aed5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:36.182 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.154+0000 7f488affd700 1 -- 172.21.15.143:0/3640570032 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55c07aed3b20 msgr2=0x55c07aed5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:36.182 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.154+0000 7f488affd700 1 --2- 172.21.15.143:0/3640570032 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55c07aed3b20 0x55c07aed5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:36.182 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.154+0000 7f488affd700 1 -- 172.21.15.143:0/3640570032 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c07ae906f0 msgr2=0x55c07aeed820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:36.182 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.154+0000 7f488affd700 1 --2- 172.21.15.143:0/3640570032 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c07ae906f0 0x55c07aeed820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:36.183 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.154+0000 7f488affd700 1 -- 172.21.15.143:0/3640570032 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f488c0b0950 con 0x55c07ae90ad0 2023-11-08T19:26:36.183 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.154+0000 7f488b7fe700 1 --2- 172.21.15.143:0/3640570032 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c07ae906f0 0x55c07aeed820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:26:36.183 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.155+0000 7f488affd700 1 --2- 172.21.15.143:0/3640570032 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c07ae90ad0 0x55c07aed35e0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f487c015780 tx=0x7f487c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=906540a60d5c01a4 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:36.183 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.155+0000 7f486a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2023-11-08T19:26:36.183 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.155+0000 7f486a7fc700 1 -- 172.21.15.143:0/3640570032 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f487c004220 con 0x55c07ae90ad0 2023-11-08T19:26:36.184 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.155+0000 7f486a7fc700 1 -- 172.21.15.143:0/3640570032 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(48..48 src has 1..48) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f487c004380 con 0x55c07ae90ad0 2023-11-08T19:26:36.184 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.155+0000 7f489dfcab80 1 -- 172.21.15.143:0/3640570032 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c07ae88880 con 0x55c07ae90ad0 2023-11-08T19:26:36.184 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.155+0000 7f486a7fc700 1 -- 172.21.15.143:0/3640570032 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f487c004540 con 0x55c07ae90ad0 2023-11-08T19:26:36.184 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.155+0000 7f489dfcab80 10 client.5745 fetch_fsmap learned FSMap version 25 2023-11-08T19:26:36.185 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.155+0000 7f489dfcab80 1 -- 172.21.15.143:0/3640570032 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x55c07ac12d90 con 0x55c07ae90ad0 2023-11-08T19:26:36.185 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.155+0000 7f486a7fc700 1 -- 172.21.15.143:0/3640570032 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c07ae88880 con 0x55c07ae90ad0 2023-11-08T19:26:36.185 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.155+0000 7f486a7fc700 1 -- 172.21.15.143:0/3640570032 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f487c0036a0 con 0x55c07ae90ad0 2023-11-08T19:26:36.185 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.155+0000 7f489dfcab80 10 client.5745 fetch_fsmap finished waiting for FSMap version 25 2023-11-08T19:26:36.185 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.155+0000 7f489dfcab80 10 client.5745 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-08T19:26:36.185 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.155+0000 7f489dfcab80 20 client.5745 populate_metadata read hostname 'smithi143' 2023-11-08T19:26:36.186 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.155+0000 7f489dfcab80 1 --2- 172.21.15.143:0/3640570032 >> [v2:172.21.15.143:6832/3591097261,v1:172.21.15.143:6834/3591097261] conn(0x55c07aeeeb80 0x55c07aeebf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:36.186 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.155+0000 7f489dfcab80 4 client.5745 mds_command: new command op to 5242 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-08T19:26:36.186 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.155+0000 7f489dfcab80 1 -- 172.21.15.143:0/3640570032 --> [v2:172.21.15.143:6832/3591097261,v1:172.21.15.143:6834/3591097261] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c07ae64a30 con 0x55c07aeeeb80 2023-11-08T19:26:36.186 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.155+0000 7f488b7fe700 1 --2- 172.21.15.143:0/3640570032 >> [v2:172.21.15.143:6832/3591097261,v1:172.21.15.143:6834/3591097261] conn(0x55c07aeeeb80 0x55c07aeebf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:36.186 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.156+0000 7f486a7fc700 10 client.5745 ms_handle_connect on v2:172.21.15.143:6832/3591097261 2023-11-08T19:26:36.187 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.156+0000 7f488b7fe700 1 --2- 172.21.15.143:0/3640570032 >> [v2:172.21.15.143:6832/3591097261,v1:172.21.15.143:6834/3591097261] conn(0x55c07aeeeb80 0x55c07aeebf50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1da3b49ed6bff304 server_cookie=a67816aa12524316 in_seq=0 out_seq=0 2023-11-08T19:26:36.187 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.156+0000 7f486a7fc700 1 -- 172.21.15.143:0/3640570032 <== mds.0 v2:172.21.15.143:6832/3591097261 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c07ae64a30 con 0x55c07aeeeb80 2023-11-08T19:26:36.187 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.156+0000 7f486a7fc700 10 client.5745 handle_command_reply: tid=0 2023-11-08T19:26:36.187 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.156+0000 7f489dfcab80 1 client.5745 shutdown 2023-11-08T19:26:36.187 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.156+0000 7f489dfcab80 1 -- 172.21.15.143:0/3640570032 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c07ae90ad0 msgr2=0x55c07aed35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:36.188 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.156+0000 7f489dfcab80 1 --2- 172.21.15.143:0/3640570032 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c07ae90ad0 0x55c07aed35e0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f487c015780 tx=0x7f487c004750 comp rx=0 tx=0).stop 2023-11-08T19:26:36.188 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.156+0000 7f489dfcab80 1 -- 172.21.15.143:0/3640570032 shutdown_connections 2023-11-08T19:26:36.188 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.156+0000 7f489dfcab80 1 --2- 172.21.15.143:0/3640570032 >> [v2:172.21.15.143:6832/3591097261,v1:172.21.15.143:6834/3591097261] conn(0x55c07aeeeb80 0x55c07aeebf50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:36.188 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.156+0000 7f489dfcab80 1 --2- 172.21.15.143:0/3640570032 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55c07aed3b20 0x55c07aed5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:36.188 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.156+0000 7f489dfcab80 1 --2- 172.21.15.143:0/3640570032 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c07ae906f0 0x55c07aeed820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:36.189 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.156+0000 7f489dfcab80 1 --2- 172.21.15.143:0/3640570032 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c07ae90ad0 0x55c07aed35e0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:36.189 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.156+0000 7f486a7fc700 0 client.5745 ms_handle_reset on v2:172.21.15.143:6832/3591097261 2023-11-08T19:26:36.189 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.156+0000 7f489dfcab80 1 -- 172.21.15.143:0/3640570032 >> 172.21.15.143:0/3640570032 conn(0x55c07ae0b660 msgr2=0x55c07ae91020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:36.189 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.156+0000 7f489dfcab80 1 -- 172.21.15.143:0/3640570032 shutdown_connections 2023-11-08T19:26:36.189 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.156+0000 7f489dfcab80 1 -- 172.21.15.143:0/3640570032 wait complete. 2023-11-08T19:26:36.190 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.156+0000 7f489dfcab80 20 client.5745 trim_cache size 0 max 16384 2023-11-08T19:26:36.196 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.174+0000 7f489dfcab80 1 Processor -- start 2023-11-08T19:26:36.197 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.174+0000 7f489dfcab80 1 -- start start 2023-11-08T19:26:36.197 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.175+0000 7f489dfcab80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c07ae906f0 0x55c07aed3310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:36.197 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.175+0000 7f489dfcab80 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55c07ae90ad0 0x55c07aec3d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:36.197 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.175+0000 7f489dfcab80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c07aed3b20 0x55c07aec42d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:36.197 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.175+0000 7f489dfcab80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c07ac688d0 con 0x55c07ae906f0 2023-11-08T19:26:36.198 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.175+0000 7f489dfcab80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c07aeac170 con 0x55c07aed3b20 2023-11-08T19:26:36.198 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.175+0000 7f489dfcab80 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c07aedf560 con 0x55c07ae90ad0 2023-11-08T19:26:36.198 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.175+0000 7f488b7fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c07ae906f0 0x55c07aed3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:36.198 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.175+0000 7f488b7fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c07ae906f0 0x55c07aed3310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51790/0 (socket says 172.21.15.143:51790) 2023-11-08T19:26:36.199 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.175+0000 7f488b7fe700 1 -- 172.21.15.143:0/3680153842 learned_addr learned my addr 172.21.15.143:0/3680153842 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:36.199 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.176+0000 7f488b7fe700 1 -- 172.21.15.143:0/3680153842 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55c07ae90ad0 msgr2=0x55c07aec3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:36.199 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.176+0000 7f488affd700 1 --2- 172.21.15.143:0/3680153842 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55c07ae90ad0 0x55c07aec3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:36.199 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.176+0000 7f488bfff700 1 --2- 172.21.15.143:0/3680153842 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c07aed3b20 0x55c07aec42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:36.199 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.176+0000 7f488b7fe700 1 --2- 172.21.15.143:0/3680153842 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55c07ae90ad0 0x55c07aec3d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:36.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.176+0000 7f488b7fe700 1 -- 172.21.15.143:0/3680153842 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c07aed3b20 msgr2=0x55c07aec42d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:36.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.176+0000 7f488b7fe700 1 --2- 172.21.15.143:0/3680153842 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c07aed3b20 0x55c07aec42d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:36.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.176+0000 7f488b7fe700 1 -- 172.21.15.143:0/3680153842 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c07abfbc10 con 0x55c07ae906f0 2023-11-08T19:26:36.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.176+0000 7f488b7fe700 1 --2- 172.21.15.143:0/3680153842 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c07ae906f0 0x55c07aed3310 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f48800584c0 tx=0x7f4880058b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6593584ac2c75578 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:36.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.176+0000 7f4871ffb700 1 -- 172.21.15.143:0/3680153842 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f488007e070 con 0x55c07ae906f0 2023-11-08T19:26:36.201 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.176+0000 7f4871ffb700 1 -- 172.21.15.143:0/3680153842 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4880058a00 con 0x55c07ae906f0 2023-11-08T19:26:36.201 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.176+0000 7f4871ffb700 1 -- 172.21.15.143:0/3680153842 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4880075040 con 0x55c07ae906f0 2023-11-08T19:26:36.201 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.177+0000 7f489dfcab80 1 -- 172.21.15.143:0/3680153842 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c07ae906f0 msgr2=0x55c07aed3310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:36.201 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.177+0000 7f489dfcab80 1 --2- 172.21.15.143:0/3680153842 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c07ae906f0 0x55c07aed3310 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f48800584c0 tx=0x7f4880058b70 comp rx=0 tx=0).stop 2023-11-08T19:26:36.201 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.177+0000 7f489dfcab80 1 -- 172.21.15.143:0/3680153842 shutdown_connections 2023-11-08T19:26:36.202 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.177+0000 7f489dfcab80 1 --2- 172.21.15.143:0/3680153842 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c07aed3b20 0x55c07aec42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:36.202 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.177+0000 7f489dfcab80 1 --2- 172.21.15.143:0/3680153842 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c07ae906f0 0x55c07aed3310 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:36.202 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.177+0000 7f489dfcab80 1 --2- 172.21.15.143:0/3680153842 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55c07ae90ad0 0x55c07aec3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:36.202 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.177+0000 7f489dfcab80 1 -- 172.21.15.143:0/3680153842 >> 172.21.15.143:0/3680153842 conn(0x55c07ae0b660 msgr2=0x55c07ae8c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:36.202 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.177+0000 7f489dfcab80 1 -- 172.21.15.143:0/3680153842 shutdown_connections 2023-11-08T19:26:36.203 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.177+0000 7f489dfcab80 1 -- 172.21.15.143:0/3680153842 wait complete. 2023-11-08T19:26:36.203 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.177+0000 7f489dfcab80 1 -- start start 2023-11-08T19:26:36.203 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.177+0000 7f489dfcab80 1 Processor -- start 2023-11-08T19:26:36.203 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.178+0000 7f489dfcab80 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55c07ae906f0 0x55c07af43ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:36.203 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.178+0000 7f489dfcab80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c07ae90ad0 0x55c07aec3d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:36.204 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.178+0000 7f489dfcab80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c07aed3b20 0x55c07aec42d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:36.204 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.178+0000 7f489dfcab80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c07aed2c70 con 0x55c07aed3b20 2023-11-08T19:26:36.204 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.178+0000 7f489dfcab80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c07aed0de0 con 0x55c07ae90ad0 2023-11-08T19:26:36.204 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.178+0000 7f489dfcab80 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c07ae93a70 con 0x55c07ae906f0 2023-11-08T19:26:36.204 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.178+0000 7f489dfcab80 10 client.0 _ll_register_callbacks cb 0x7f487c010e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T19:26:36.205 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.178+0000 7f488bfff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c07aed3b20 0x55c07aec42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:36.205 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.178+0000 7f488bfff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c07aed3b20 0x55c07aec42d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51800/0 (socket says 172.21.15.143:51800) 2023-11-08T19:26:36.205 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.178+0000 7f488bfff700 1 -- 172.21.15.143:0/954418061 learned_addr learned my addr 172.21.15.143:0/954418061 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:36.205 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.178+0000 7f488b7fe700 1 --2- 172.21.15.143:0/954418061 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55c07ae906f0 0x55c07af43ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:36.205 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.178+0000 7f488affd700 1 --2- 172.21.15.143:0/954418061 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c07ae90ad0 0x55c07aec3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:36.206 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.178+0000 7f488bfff700 1 -- 172.21.15.143:0/954418061 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55c07ae906f0 msgr2=0x55c07af43ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:36.206 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.178+0000 7f488bfff700 1 --2- 172.21.15.143:0/954418061 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55c07ae906f0 0x55c07af43ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:36.206 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.178+0000 7f488bfff700 1 -- 172.21.15.143:0/954418061 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c07ae90ad0 msgr2=0x55c07aec3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:36.206 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.178+0000 7f488bfff700 1 --2- 172.21.15.143:0/954418061 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c07ae90ad0 0x55c07aec3d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:36.206 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.178+0000 7f488bfff700 1 -- 172.21.15.143:0/954418061 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c07ac665e0 con 0x55c07aed3b20 2023-11-08T19:26:36.207 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.178+0000 7f488b7fe700 1 --2- 172.21.15.143:0/954418061 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55c07ae906f0 0x55c07af43ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:26:36.207 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.179+0000 7f488bfff700 1 --2- 172.21.15.143:0/954418061 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c07aed3b20 0x55c07aec42d0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f488c070a90 tx=0x7f4884004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12e91aab4007cd20 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:36.207 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.179+0000 7f486a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2023-11-08T19:26:36.207 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.179+0000 7f486a7fc700 1 -- 172.21.15.143:0/954418061 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4884022070 con 0x55c07aed3b20 2023-11-08T19:26:36.207 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.179+0000 7f486a7fc700 1 -- 172.21.15.143:0/954418061 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(48..48 src has 1..48) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4884003e10 con 0x55c07aed3b20 2023-11-08T19:26:36.208 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.179+0000 7f489dfcab80 1 -- 172.21.15.143:0/954418061 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c07aeee200 con 0x55c07aed3b20 2023-11-08T19:26:36.208 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.179+0000 7f486a7fc700 1 -- 172.21.15.143:0/954418061 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4884004250 con 0x55c07aed3b20 2023-11-08T19:26:36.208 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.179+0000 7f489dfcab80 10 client.5751 fetch_fsmap learned FSMap version 25 2023-11-08T19:26:36.208 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.179+0000 7f489dfcab80 1 -- 172.21.15.143:0/954418061 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x55c07ac39200 con 0x55c07aed3b20 2023-11-08T19:26:36.208 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.179+0000 7f486a7fc700 1 -- 172.21.15.143:0/954418061 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c07aeee200 con 0x55c07aed3b20 2023-11-08T19:26:36.209 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.179+0000 7f486a7fc700 1 -- 172.21.15.143:0/954418061 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f488400b050 con 0x55c07aed3b20 2023-11-08T19:26:36.209 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.179+0000 7f489dfcab80 10 client.5751 fetch_fsmap finished waiting for FSMap version 25 2023-11-08T19:26:36.209 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.179+0000 7f489dfcab80 10 client.5751 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-08T19:26:36.209 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.179+0000 7f489dfcab80 20 client.5751 populate_metadata read hostname 'smithi143' 2023-11-08T19:26:36.209 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.179+0000 7f489dfcab80 1 --2- 172.21.15.143:0/954418061 >> [v2:172.21.15.143:6832/3591097261,v1:172.21.15.143:6834/3591097261] conn(0x55c07aebcd10 0x55c07aebd0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:36.210 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.179+0000 7f489dfcab80 4 client.5751 mds_command: new command op to 5242 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-08T19:26:36.210 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.179+0000 7f489dfcab80 1 -- 172.21.15.143:0/954418061 --> [v2:172.21.15.143:6832/3591097261,v1:172.21.15.143:6834/3591097261] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7f4880060600 con 0x55c07aebcd10 2023-11-08T19:26:36.210 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.179+0000 7f488b7fe700 1 --2- 172.21.15.143:0/954418061 >> [v2:172.21.15.143:6832/3591097261,v1:172.21.15.143:6834/3591097261] conn(0x55c07aebcd10 0x55c07aebd0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:36.210 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.180+0000 7f486a7fc700 10 client.5751 ms_handle_connect on v2:172.21.15.143:6832/3591097261 2023-11-08T19:26:36.210 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.180+0000 7f488b7fe700 1 --2- 172.21.15.143:0/954418061 >> [v2:172.21.15.143:6832/3591097261,v1:172.21.15.143:6834/3591097261] conn(0x55c07aebcd10 0x55c07aebd0f0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e504d94aae9bac16 server_cookie=7f3312f9e77b34fc in_seq=0 out_seq=0 2023-11-08T19:26:36.211 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.180+0000 7f486a7fc700 1 -- 172.21.15.143:0/954418061 <== mds.0 v2:172.21.15.143:6832/3591097261 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+319 (crc 0 0 0) 0x7f4880060600 con 0x55c07aebcd10 2023-11-08T19:26:36.211 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.180+0000 7f486a7fc700 10 client.5751 handle_command_reply: tid=0 2023-11-08T19:26:36.211 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.180+0000 7f489dfcab80 1 client.5751 shutdown 2023-11-08T19:26:36.211 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.180+0000 7f489dfcab80 1 -- 172.21.15.143:0/954418061 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c07aed3b20 msgr2=0x55c07aec42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:36.211 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.180+0000 7f489dfcab80 1 --2- 172.21.15.143:0/954418061 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c07aed3b20 0x55c07aec42d0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f488c070a90 tx=0x7f4884004b40 comp rx=0 tx=0).stop 2023-11-08T19:26:36.212 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.180+0000 7f489dfcab80 1 -- 172.21.15.143:0/954418061 shutdown_connections 2023-11-08T19:26:36.212 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.180+0000 7f489dfcab80 1 --2- 172.21.15.143:0/954418061 >> [v2:172.21.15.143:6832/3591097261,v1:172.21.15.143:6834/3591097261] conn(0x55c07aebcd10 0x55c07aebd0f0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:36.212 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.180+0000 7f489dfcab80 1 --2- 172.21.15.143:0/954418061 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55c07ae906f0 0x55c07af43ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:36.212 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.180+0000 7f486a7fc700 0 client.5751 ms_handle_reset on v2:172.21.15.143:6832/3591097261 2023-11-08T19:26:36.212 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.180+0000 7f489dfcab80 1 --2- 172.21.15.143:0/954418061 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c07aed3b20 0x55c07aec42d0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:36.213 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.180+0000 7f489dfcab80 1 --2- 172.21.15.143:0/954418061 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c07ae90ad0 0x55c07aec3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:36.213 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.180+0000 7f489dfcab80 1 -- 172.21.15.143:0/954418061 >> 172.21.15.143:0/954418061 conn(0x55c07ae0b660 msgr2=0x55c07aed3310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:36.213 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.180+0000 7f489dfcab80 1 -- 172.21.15.143:0/954418061 shutdown_connections 2023-11-08T19:26:36.213 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.180+0000 7f489dfcab80 1 -- 172.21.15.143:0/954418061 wait complete. 2023-11-08T19:26:36.213 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.180+0000 7f489dfcab80 20 client.5751 trim_cache size 0 max 16384 2023-11-08T19:26:36.214 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2023-11-08T19:26:36.214 INFO:teuthology.orchestra.run.smithi143.stdout: "cluster_fsid": "968483af-77fc-4068-ad7c-5c2a1a4613d7", 2023-11-08T19:26:36.214 INFO:teuthology.orchestra.run.smithi143.stdout: "whoami": 0, 2023-11-08T19:26:36.215 INFO:teuthology.orchestra.run.smithi143.stdout: "id": 5242, 2023-11-08T19:26:36.215 INFO:teuthology.orchestra.run.smithi143.stdout: "want_state": "up:active", 2023-11-08T19:26:36.215 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "up:active", 2023-11-08T19:26:36.215 INFO:teuthology.orchestra.run.smithi143.stdout: "fs_name": "cephfs", 2023-11-08T19:26:36.215 INFO:teuthology.orchestra.run.smithi143.stdout: "rank_uptime": 10.455347686, 2023-11-08T19:26:36.215 INFO:teuthology.orchestra.run.smithi143.stdout: "mdsmap_epoch": 25, 2023-11-08T19:26:36.216 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_epoch": 48, 2023-11-08T19:26:36.216 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_epoch_barrier": 48, 2023-11-08T19:26:36.216 INFO:teuthology.orchestra.run.smithi143.stdout: "uptime": 16.399249889 2023-11-08T19:26:36.216 INFO:teuthology.orchestra.run.smithi143.stdout:} 2023-11-08T19:26:36.218 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.180+0000 7f4891dd1700 1 -- 172.21.15.143:0/249050111 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f487806af00 msgr2=0x7f487806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:36.218 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.180+0000 7f4891dd1700 1 --2- 172.21.15.143:0/249050111 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f487806af00 0x7f487806d3c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f487c009930 tx=0x7f487c008040 comp rx=0 tx=0).stop 2023-11-08T19:26:36.218 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.180+0000 7f4891dd1700 1 -- 172.21.15.143:0/249050111 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f488c06b150 msgr2=0x7f488c1f7590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:36.218 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.180+0000 7f4891dd1700 1 --2- 172.21.15.143:0/249050111 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f488c06b150 0x7f488c1f7590 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f488000e7e0 tx=0x7f488000eba0 comp rx=0 tx=0).stop 2023-11-08T19:26:36.219 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.181+0000 7f4891dd1700 1 -- 172.21.15.143:0/249050111 shutdown_connections 2023-11-08T19:26:36.219 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.181+0000 7f4891dd1700 1 --2- 172.21.15.143:0/249050111 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f487806af00 0x7f487806d3c0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:36.219 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.181+0000 7f4891dd1700 1 --2- 172.21.15.143:0/249050111 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f488c1618f0 0x7f488c0c50d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:36.219 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.181+0000 7f4891dd1700 1 --2- 172.21.15.143:0/249050111 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f488c06b150 0x7f488c1f7590 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:36.219 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.181+0000 7f4891dd1700 1 --2- 172.21.15.143:0/249050111 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f488c15dd00 0x7f488c1f7ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:36.220 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.181+0000 7f4891dd1700 1 -- 172.21.15.143:0/249050111 >> 172.21.15.143:0/249050111 conn(0x7f488c099ab0 msgr2=0x7f488c15d020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:36.220 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.181+0000 7f4891dd1700 1 -- 172.21.15.143:0/249050111 shutdown_connections 2023-11-08T19:26:36.220 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.181+0000 7f4891dd1700 1 -- 172.21.15.143:0/249050111 wait complete. 2023-11-08T19:26:36.221 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-08T19:26:36.341 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.339+0000 7fd19cd07700 1 Processor -- start 2023-11-08T19:26:36.341 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.339+0000 7fd19cd07700 1 -- start start 2023-11-08T19:26:36.344 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.340+0000 7fd19cd07700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd19806dbb0 0x7fd19815fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:36.344 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.340+0000 7fd19cd07700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fd19806e9a0 0x7fd1980b3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:36.344 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.340+0000 7fd19cd07700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd19806bd50 con 0x7fd198072860 2023-11-08T19:26:36.344 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.340+0000 7fd19cd07700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1980cb1c0 con 0x7fd19806dbb0 2023-11-08T19:26:36.344 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.340+0000 7fd19cd07700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd19806bff0 con 0x7fd19806e9a0 2023-11-08T19:26:36.345 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.341+0000 7fd19659c700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd19806dbb0 0x7fd19815fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:36.345 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.341+0000 7fd195d9b700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fd198072860 0x7fd19815ffa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:47958/0 (socket says 172.21.15.143:47958) 2023-11-08T19:26:36.345 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.341+0000 7fd195d9b700 1 -- 172.21.15.143:0/3501085897 learned_addr learned my addr 172.21.15.143:0/3501085897 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:26:36.345 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.341+0000 7fd196d9d700 1 --2- 172.21.15.143:0/3501085897 >> v2:172.21.15.174:3300/0 conn(0x7fd19806e9a0 0x7fd1980b3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:36.345 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.341+0000 7fd19659c700 1 -- 172.21.15.143:0/3501085897 >> v2:172.21.15.174:3300/0 conn(0x7fd19806e9a0 msgr2=0x7fd1980b3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:36.346 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.341+0000 7fd19659c700 1 --2- 172.21.15.143:0/3501085897 >> v2:172.21.15.174:3300/0 conn(0x7fd19806e9a0 0x7fd1980b3240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:36.346 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.341+0000 7fd19659c700 1 -- 172.21.15.143:0/3501085897 >> v1:172.21.15.143:6789/0 conn(0x7fd198072860 legacy=0x7fd19815ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:36.346 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.341+0000 7fd19659c700 1 -- 172.21.15.143:0/3501085897 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd19806c2d0 con 0x7fd19806dbb0 2023-11-08T19:26:36.346 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.341+0000 7fd19659c700 1 --2- 172.21.15.143:0/3501085897 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd19806dbb0 0x7fd19815fa60 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fd1980c38b0 tx=0x7fd180009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cf75cbc53f7a021 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:36.346 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.341+0000 7fd19559a700 1 -- 172.21.15.143:0/3501085897 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd180011830 con 0x7fd19806dbb0 2023-11-08T19:26:36.347 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.341+0000 7fd19559a700 1 -- 172.21.15.143:0/3501085897 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd180011990 con 0x7fd19806dbb0 2023-11-08T19:26:36.347 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.341+0000 7fd19559a700 1 -- 172.21.15.143:0/3501085897 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd180011d20 con 0x7fd19806dbb0 2023-11-08T19:26:36.347 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.342+0000 7fd19cd07700 1 -- 172.21.15.143:0/3501085897 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd19806dbb0 msgr2=0x7fd19815fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:36.347 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.342+0000 7fd19cd07700 1 --2- 172.21.15.143:0/3501085897 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd19806dbb0 0x7fd19815fa60 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fd1980c38b0 tx=0x7fd180009b20 comp rx=0 tx=0).stop 2023-11-08T19:26:36.347 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.342+0000 7fd19cd07700 1 -- 172.21.15.143:0/3501085897 shutdown_connections 2023-11-08T19:26:36.348 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.342+0000 7fd19cd07700 1 --2- 172.21.15.143:0/3501085897 >> v2:172.21.15.174:3300/0 conn(0x7fd19806e9a0 0x7fd1980b3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:36.348 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.342+0000 7fd19cd07700 1 --2- 172.21.15.143:0/3501085897 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd19806dbb0 0x7fd19815fa60 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:36.348 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.342+0000 7fd19cd07700 1 -- 172.21.15.143:0/3501085897 >> 172.21.15.143:0/3501085897 conn(0x7fd1980b1a20 msgr2=0x7fd1980b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:36.348 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.342+0000 7fd19cd07700 1 -- 172.21.15.143:0/3501085897 shutdown_connections 2023-11-08T19:26:36.348 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.342+0000 7fd19cd07700 1 -- 172.21.15.143:0/3501085897 wait complete. 2023-11-08T19:26:36.351 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.343+0000 7fd19cd07700 1 Processor -- start 2023-11-08T19:26:36.352 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.343+0000 7fd19cd07700 1 -- start start 2023-11-08T19:26:36.352 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.344+0000 7fd19cd07700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd19806dbb0 0x7fd19815d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:36.352 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.344+0000 7fd19cd07700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd19806e9a0 0x7fd1981537a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:36.352 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.344+0000 7fd19cd07700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd198072860 0x7fd198150c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:36.352 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.344+0000 7fd19cd07700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1981e5310 con 0x7fd19806e9a0 2023-11-08T19:26:36.353 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.344+0000 7fd19cd07700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1981e4700 con 0x7fd198072860 2023-11-08T19:26:36.353 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.344+0000 7fd19cd07700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1981e49e0 con 0x7fd19806dbb0 2023-11-08T19:26:36.353 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.344+0000 7fd195d9b700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd19806e9a0 0x7fd1981537a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:36.353 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.344+0000 7fd195d9b700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd19806e9a0 0x7fd1981537a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51812/0 (socket says 172.21.15.143:51812) 2023-11-08T19:26:36.353 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.344+0000 7fd195d9b700 1 -- 172.21.15.143:0/4227012215 learned_addr learned my addr 172.21.15.143:0/4227012215 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:36.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.344+0000 7fd19659c700 1 --2- 172.21.15.143:0/4227012215 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd19806dbb0 0x7fd19815d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:36.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.344+0000 7fd196d9d700 1 --2- 172.21.15.143:0/4227012215 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd198072860 0x7fd198150c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:36.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.344+0000 7fd195d9b700 1 -- 172.21.15.143:0/4227012215 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd19806dbb0 msgr2=0x7fd19815d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:36.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.344+0000 7fd195d9b700 1 --2- 172.21.15.143:0/4227012215 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd19806dbb0 0x7fd19815d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:36.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.344+0000 7fd195d9b700 1 -- 172.21.15.143:0/4227012215 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd198072860 msgr2=0x7fd198150c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:36.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.344+0000 7fd195d9b700 1 --2- 172.21.15.143:0/4227012215 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd198072860 0x7fd198150c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:36.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.344+0000 7fd195d9b700 1 -- 172.21.15.143:0/4227012215 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1800094a0 con 0x7fd19806e9a0 2023-11-08T19:26:36.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.344+0000 7fd195d9b700 1 --2- 172.21.15.143:0/4227012215 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd19806e9a0 0x7fd1981537a0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fd18800c480 tx=0x7fd18800c790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:36.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.345+0000 7fd18effd700 1 -- 172.21.15.143:0/4227012215 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd188005d70 con 0x7fd19806e9a0 2023-11-08T19:26:36.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.345+0000 7fd18effd700 1 -- 172.21.15.143:0/4227012215 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd18800ce90 con 0x7fd19806e9a0 2023-11-08T19:26:36.356 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.345+0000 7fd19cd07700 1 -- 172.21.15.143:0/4227012215 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1981e4cc0 con 0x7fd19806e9a0 2023-11-08T19:26:36.356 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.345+0000 7fd18effd700 1 -- 172.21.15.143:0/4227012215 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1880072d0 con 0x7fd19806e9a0 2023-11-08T19:26:36.356 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.345+0000 7fd19cd07700 1 -- 172.21.15.143:0/4227012215 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd19815dc80 con 0x7fd19806e9a0 2023-11-08T19:26:36.356 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.346+0000 7fd19cd07700 1 -- 172.21.15.143:0/4227012215 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd19815de70 con 0x7fd19806e9a0 2023-11-08T19:26:36.356 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.346+0000 7fd18effd700 1 -- 172.21.15.143:0/4227012215 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd188007430 con 0x7fd19806e9a0 2023-11-08T19:26:36.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.347+0000 7fd18effd700 1 --2- 172.21.15.143:0/4227012215 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd17806af00 0x7fd17806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:36.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.347+0000 7fd18effd700 1 -- 172.21.15.143:0/4227012215 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd188088900 con 0x7fd19806e9a0 2023-11-08T19:26:36.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.347+0000 7fd19659c700 1 --2- 172.21.15.143:0/4227012215 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd17806af00 0x7fd17806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:36.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.347+0000 7fd19659c700 1 --2- 172.21.15.143:0/4227012215 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd17806af00 0x7fd17806d3c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fd1800090b0 tx=0x7fd180008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:36.358 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.351+0000 7fd18effd700 1 -- 172.21.15.143:0/4227012215 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd188053910 con 0x7fd19806e9a0 2023-11-08T19:26:36.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.586+0000 7fd19cd07700 1 -- 172.21.15.143:0/4227012215 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fd19804eee0 con 0x7fd19806e9a0 2023-11-08T19:26:36.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.586+0000 7fd18effd700 1 -- 172.21.15.143:0/4227012215 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v25) v1 ==== 53+0+73 (secure 0 0 0) 0x7fd188056f30 con 0x7fd19806e9a0 2023-11-08T19:26:36.588 INFO:teuthology.orchestra.run.smithi143.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-08T19:26:36.590 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.588+0000 7fd19cd07700 1 -- 172.21.15.143:0/4227012215 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd17806af00 msgr2=0x7fd17806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:36.590 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.588+0000 7fd19cd07700 1 --2- 172.21.15.143:0/4227012215 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd17806af00 0x7fd17806d3c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fd1800090b0 tx=0x7fd180008030 comp rx=0 tx=0).stop 2023-11-08T19:26:36.591 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.588+0000 7fd19cd07700 1 -- 172.21.15.143:0/4227012215 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd19806e9a0 msgr2=0x7fd1981537a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:36.591 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.588+0000 7fd19cd07700 1 --2- 172.21.15.143:0/4227012215 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd19806e9a0 0x7fd1981537a0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fd18800c480 tx=0x7fd18800c790 comp rx=0 tx=0).stop 2023-11-08T19:26:36.591 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.589+0000 7fd19cd07700 1 -- 172.21.15.143:0/4227012215 shutdown_connections 2023-11-08T19:26:36.591 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.589+0000 7fd19cd07700 1 --2- 172.21.15.143:0/4227012215 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd17806af00 0x7fd17806d3c0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:36.591 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.589+0000 7fd19cd07700 1 --2- 172.21.15.143:0/4227012215 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd19806dbb0 0x7fd19815d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:36.592 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.589+0000 7fd19cd07700 1 --2- 172.21.15.143:0/4227012215 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd198072860 0x7fd198150c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:36.592 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.589+0000 7fd19cd07700 1 --2- 172.21.15.143:0/4227012215 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd19806e9a0 0x7fd1981537a0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:36.592 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.589+0000 7fd19cd07700 1 -- 172.21.15.143:0/4227012215 >> 172.21.15.143:0/4227012215 conn(0x7fd1980b1a20 msgr2=0x7fd19815b240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:36.592 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.589+0000 7fd19cd07700 1 -- 172.21.15.143:0/4227012215 shutdown_connections 2023-11-08T19:26:36.593 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:36.589+0000 7fd19cd07700 1 -- 172.21.15.143:0/4227012215 wait complete. 2023-11-08T19:26:36.602 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-08T19:26:36.602 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-08T19:26:36.603 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi143.front.sepia.ceph.com 2023-11-08T19:26:36.603 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-08T19:26:36.604 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-08T19:26:36.604 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-08T19:26:36.604 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-08T19:26:36.605 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-08T19:26:36.605 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-08T19:26:36.606 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:26:36.606 DEBUG:teuthology.orchestra.run.smithi143:> ip addr 2023-11-08T19:26:36.623 INFO:teuthology.orchestra.run.smithi143.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-08T19:26:36.623 INFO:teuthology.orchestra.run.smithi143.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-08T19:26:36.624 INFO:teuthology.orchestra.run.smithi143.stdout: inet 127.0.0.1/8 scope host lo 2023-11-08T19:26:36.624 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:26:36.624 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 ::1/128 scope host 2023-11-08T19:26:36.624 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:26:36.624 INFO:teuthology.orchestra.run.smithi143.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-08T19:26:36.625 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:d9:3b:94 brd ff:ff:ff:ff:ff:ff 2023-11-08T19:26:36.625 INFO:teuthology.orchestra.run.smithi143.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-08T19:26:36.625 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:d9:3b:95 brd ff:ff:ff:ff:ff:ff 2023-11-08T19:26:36.625 INFO:teuthology.orchestra.run.smithi143.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-08T19:26:36.625 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:88:77:82 brd ff:ff:ff:ff:ff:ff 2023-11-08T19:26:36.626 INFO:teuthology.orchestra.run.smithi143.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-08T19:26:36.626 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:88:77:83 brd ff:ff:ff:ff:ff:ff 2023-11-08T19:26:36.626 INFO:teuthology.orchestra.run.smithi143.stdout: inet 172.21.15.143/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-08T19:26:36.626 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft 42271sec preferred_lft 42271sec 2023-11-08T19:26:36.626 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 fe80::e628:d259:2f76:52a6/64 scope link noprefixroute 2023-11-08T19:26:36.627 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:26:36.627 INFO:teuthology.orchestra.run.smithi143.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-08T19:26:36.627 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 46:e3:3e:66:77:ea brd ff:ff:ff:ff:ff:ff 2023-11-08T19:26:36.627 INFO:teuthology.orchestra.run.smithi143.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-08T19:26:36.627 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:26:36.628 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 fe80::749a:8aff:fe59:325c/64 scope link 2023-11-08T19:26:36.628 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:26:36.628 INFO:teuthology.orchestra.run.smithi143.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-08T19:26:36.628 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 46:e3:3e:66:77:ea brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-08T19:26:36.628 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 fe80::44e3:3eff:fe66:77ea/64 scope link 2023-11-08T19:26:36.629 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:26:36.629 INFO:teuthology.orchestra.run.smithi143.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-08T19:26:36.629 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether ba:92:23:3f:5e:c8 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-08T19:26:36.629 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 fe80::b892:23ff:fe3f:5ec8/64 scope link 2023-11-08T19:26:36.629 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:26:36.630 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:26:36.630 DEBUG:teuthology.orchestra.run.smithi143:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-08T19:26:36.690 INFO:teuthology.orchestra.run.smithi143.stdout:1 2023-11-08T19:26:36.690 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:26:36.691 DEBUG:teuthology.orchestra.run.smithi143:> route 2023-11-08T19:26:36.749 INFO:teuthology.orchestra.run.smithi143.stdout:Kernel IP routing table 2023-11-08T19:26:36.749 INFO:teuthology.orchestra.run.smithi143.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-08T19:26:36.750 INFO:teuthology.orchestra.run.smithi143.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-08T19:26:36.750 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-08T19:26:36.750 INFO:teuthology.orchestra.run.smithi143.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-08T19:26:36.751 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:26:36.751 DEBUG:teuthology.orchestra.run.smithi143:> (cd / && exec bash -c ' 2023-11-08T19:26:36.751 DEBUG:teuthology.orchestra.run.smithi143:> set -e 2023-11-08T19:26:36.752 DEBUG:teuthology.orchestra.run.smithi143:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-08T19:26:36.752 DEBUG:teuthology.orchestra.run.smithi143:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-08T19:26:36.753 DEBUG:teuthology.orchestra.run.smithi143:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-08T19:26:36.753 DEBUG:teuthology.orchestra.run.smithi143:> ') 2023-11-08T19:26:36.853 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:26:36.854 DEBUG:teuthology.orchestra.run.smithi143:> ip netns list 2023-11-08T19:26:36.870 INFO:teuthology.orchestra.run.smithi143.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-08T19:26:36.871 INFO:teuthology.orchestra.run.smithi143.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-08T19:26:36.871 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-08T19:26:36.871 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:26:36.872 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip link set brx.0 up 2023-11-08T19:26:36.938 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-08T19:26:36.938 DEBUG:teuthology.orchestra.run.smithi143:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-08T19:26:36.999 INFO:teuthology.orchestra.run.smithi143.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-08T19:26:36.999 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-08T19:26:37.000 DEBUG:teuthology.orchestra.run.smithi143:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-08T19:26:37.068 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-11-08T19:26:37.068 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:26:37.069 DEBUG:teuthology.orchestra.run.smithi143:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-08T19:26:37.169 INFO:teuthology.orchestra.run.smithi143.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-08T19:26:37.169 INFO:teuthology.orchestra.run.smithi143.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-11-08T19:26:37.170 INFO:teuthology.orchestra.run.smithi143.stdout:invalid new device string format 2023-11-08T19:26:37.178 INFO:teuthology.orchestra.run.smithi143.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-08T19:26:37.178 INFO:teuthology.orchestra.run.smithi143.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-11-08T19:26:37.178 INFO:teuthology.orchestra.run.smithi143.stdout:invalid new device string format 2023-11-08T19:26:37.179 INFO:teuthology.orchestra.run.smithi143.stdout:mount.ceph: resolved to: "172.21.15.143:3300,172.21.15.174:3300,172.21.15.174:3301" 2023-11-08T19:26:37.179 INFO:teuthology.orchestra.run.smithi143.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-08T19:26:37.252 INFO:teuthology.orchestra.run.smithi143.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-08T19:26:37.253 INFO:teuthology.orchestra.run.smithi143.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-11-08T19:26:37.253 INFO:teuthology.orchestra.run.smithi143.stdout:invalid new device string format 2023-11-08T19:26:37.253 INFO:teuthology.orchestra.run.smithi143.stdout:mount.ceph: resolved to: "172.21.15.143:3300,172.21.15.174:3300,172.21.15.174:3301" 2023-11-08T19:26:37.253 INFO:teuthology.orchestra.run.smithi143.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-08T19:26:37.253 INFO:teuthology.orchestra.run.smithi143.stdout:mount.ceph: resolved to: "172.21.15.143:6789,172.21.15.174:6789,172.21.15.174:6790" 2023-11-08T19:26:37.254 INFO:teuthology.orchestra.run.smithi143.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.143:6789,172.21.15.174:6789,172.21.15.174:6790:/ 2023-11-08T19:26:37.254 INFO:teuthology.orchestra.run.smithi143.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=968483af-77fc-4068-ad7c-5c2a1a4613d7" will pass to kernel 2023-11-08T19:26:37.255 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-08T19:26:37.255 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:26:37.256 DEBUG:teuthology.orchestra.run.smithi143:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-08T19:26:37.325 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:26:37.325 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-08T19:26:37.412 INFO:teuthology.orchestra.run.smithi143.stdout:client5371 2023-11-08T19:26:37.413 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-08T19:26:37.413 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:26:37.475 DEBUG:teuthology.orchestra.run.smithi143:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data mksnap snap1 2023-11-08T19:26:37.560 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.559+0000 7fb266ee1080 1 Processor -- start 2023-11-08T19:26:37.560 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.559+0000 7fb266ee1080 1 -- start start 2023-11-08T19:26:37.565 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.559+0000 7fb266ee1080 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb24c002440 0x55b5822af8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:37.565 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.559+0000 7fb266ee1080 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55b582357720 0x55b5822b3c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:37.565 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.559+0000 7fb266ee1080 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b5822cf890 0x55b5822cfd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:37.565 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.559+0000 7fb266ee1080 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b582233c30 con 0x55b5822cf890 2023-11-08T19:26:37.565 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.559+0000 7fb266ee1080 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b58222f520 con 0x7fb24c002440 2023-11-08T19:26:37.566 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.559+0000 7fb266ee1080 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b5821c9740 con 0x55b582357720 2023-11-08T19:26:37.566 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.559+0000 7fb25d777700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b5822cf890 0x55b5822cfd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:37.566 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.559+0000 7fb25d777700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b5822cf890 0x55b5822cfd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51838/0 (socket says 172.21.15.143:51838) 2023-11-08T19:26:37.566 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.559+0000 7fb25d777700 1 -- 172.21.15.143:0/1754397558 learned_addr learned my addr 172.21.15.143:0/1754397558 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:37.567 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.559+0000 7fb25cf76700 1 --2- 172.21.15.143:0/1754397558 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb24c002440 0x55b5822af8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:37.567 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.560+0000 7fb257fff700 1 --2- 172.21.15.143:0/1754397558 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55b582357720 0x55b5822b3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:37.567 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.560+0000 7fb25d777700 1 -- 172.21.15.143:0/1754397558 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55b582357720 msgr2=0x55b5822b3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:37.567 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.560+0000 7fb25d777700 1 --2- 172.21.15.143:0/1754397558 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55b582357720 0x55b5822b3c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:37.567 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.560+0000 7fb25d777700 1 -- 172.21.15.143:0/1754397558 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb24c002440 msgr2=0x55b5822af8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:37.568 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.560+0000 7fb25d777700 1 --2- 172.21.15.143:0/1754397558 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb24c002440 0x55b5822af8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:37.568 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.560+0000 7fb25d777700 1 -- 172.21.15.143:0/1754397558 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b5821c8b70 con 0x55b5822cf890 2023-11-08T19:26:37.568 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.560+0000 7fb25d777700 1 --2- 172.21.15.143:0/1754397558 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b5822cf890 0x55b5822cfd10 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x55b582285900 tx=0x7fb258002210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd666893e7216a5f server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:37.568 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.560+0000 7fb256ffd700 1 -- 172.21.15.143:0/1754397558 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb25800f450 con 0x55b5822cf890 2023-11-08T19:26:37.568 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.560+0000 7fb256ffd700 1 -- 172.21.15.143:0/1754397558 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb25800f5b0 con 0x55b5822cf890 2023-11-08T19:26:37.569 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.560+0000 7fb256ffd700 1 -- 172.21.15.143:0/1754397558 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb25800f960 con 0x55b5822cf890 2023-11-08T19:26:37.569 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.560+0000 7fb266ee1080 1 -- 172.21.15.143:0/1754397558 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b5822cf890 msgr2=0x55b5822cfd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:37.569 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.560+0000 7fb266ee1080 1 --2- 172.21.15.143:0/1754397558 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b5822cf890 0x55b5822cfd10 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x55b582285900 tx=0x7fb258002210 comp rx=0 tx=0).stop 2023-11-08T19:26:37.569 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.560+0000 7fb266ee1080 1 -- 172.21.15.143:0/1754397558 shutdown_connections 2023-11-08T19:26:37.569 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.560+0000 7fb266ee1080 1 --2- 172.21.15.143:0/1754397558 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b5822cf890 0x55b5822cfd10 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:37.569 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.560+0000 7fb266ee1080 1 --2- 172.21.15.143:0/1754397558 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb24c002440 0x55b5822af8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:37.570 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.560+0000 7fb266ee1080 1 --2- 172.21.15.143:0/1754397558 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55b582357720 0x55b5822b3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:37.570 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.560+0000 7fb266ee1080 1 -- 172.21.15.143:0/1754397558 >> 172.21.15.143:0/1754397558 conn(0x55b5822d03c0 msgr2=0x55b5822b5940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:37.570 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.560+0000 7fb266ee1080 1 -- 172.21.15.143:0/1754397558 shutdown_connections 2023-11-08T19:26:37.570 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.561+0000 7fb266ee1080 1 -- 172.21.15.143:0/1754397558 wait complete. 2023-11-08T19:26:37.570 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.561+0000 7fb266ee1080 1 Processor -- start 2023-11-08T19:26:37.571 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.561+0000 7fb266ee1080 1 -- start start 2023-11-08T19:26:37.571 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.562+0000 7fb266ee1080 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55b5822cf890 0x55b5822c8a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:37.571 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.562+0000 7fb266ee1080 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b582357720 0x55b5823509e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:37.571 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.562+0000 7fb266ee1080 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb24c002440 0x55b5822b48a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:37.571 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.562+0000 7fb266ee1080 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5821c93e0 con 0x55b582357720 2023-11-08T19:26:37.572 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.562+0000 7fb266ee1080 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5821bd680 con 0x7fb24c002440 2023-11-08T19:26:37.572 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.562+0000 7fb266ee1080 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b5823522e0 con 0x55b5822cf890 2023-11-08T19:26:37.572 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.562+0000 7fb257fff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b582357720 0x55b5823509e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:37.572 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.562+0000 7fb257fff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b582357720 0x55b5823509e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51844/0 (socket says 172.21.15.143:51844) 2023-11-08T19:26:37.572 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.562+0000 7fb257fff700 1 -- 172.21.15.143:0/1201643484 learned_addr learned my addr 172.21.15.143:0/1201643484 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:37.573 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.562+0000 7fb25cf76700 1 --2- 172.21.15.143:0/1201643484 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55b5822cf890 0x55b5822c8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:37.573 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.562+0000 7fb25d777700 1 --2- 172.21.15.143:0/1201643484 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb24c002440 0x55b5822b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:37.573 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.562+0000 7fb257fff700 1 -- 172.21.15.143:0/1201643484 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55b5822cf890 msgr2=0x55b5822c8a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:37.573 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.562+0000 7fb257fff700 1 --2- 172.21.15.143:0/1201643484 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55b5822cf890 0x55b5822c8a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:37.573 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.562+0000 7fb257fff700 1 -- 172.21.15.143:0/1201643484 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb24c002440 msgr2=0x55b5822b48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:37.574 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.562+0000 7fb257fff700 1 --2- 172.21.15.143:0/1201643484 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb24c002440 0x55b5822b48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:37.574 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.562+0000 7fb257fff700 1 -- 172.21.15.143:0/1201643484 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb25800e040 con 0x55b582357720 2023-11-08T19:26:37.574 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.562+0000 7fb257fff700 1 --2- 172.21.15.143:0/1201643484 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b582357720 0x55b5823509e0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fb250003db0 tx=0x7fb25000e5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:37.574 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.562+0000 7fb255ffb700 1 -- 172.21.15.143:0/1201643484 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb25000ecc0 con 0x55b582357720 2023-11-08T19:26:37.574 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.563+0000 7fb255ffb700 1 -- 172.21.15.143:0/1201643484 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb250008040 con 0x55b582357720 2023-11-08T19:26:37.575 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.563+0000 7fb266ee1080 1 -- 172.21.15.143:0/1201643484 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55b5821c85d0 con 0x55b582357720 2023-11-08T19:26:37.575 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.563+0000 7fb255ffb700 1 -- 172.21.15.143:0/1201643484 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb25000ee40 con 0x55b582357720 2023-11-08T19:26:37.575 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.563+0000 7fb266ee1080 1 -- 172.21.15.143:0/1201643484 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b5823528a0 con 0x55b582357720 2023-11-08T19:26:37.577 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-08T19:26:37.578 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.564+0000 7fb255ffb700 1 -- 172.21.15.143:0/1201643484 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb250003110 con 0x55b582357720 2023-11-08T19:26:37.578 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.564+0000 7fb255ffb700 1 --2- 172.21.15.143:0/1201643484 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fb240069820 0x7fb24006bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:37.578 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.564+0000 7fb255ffb700 1 -- 172.21.15.143:0/1201643484 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb250088770 con 0x55b582357720 2023-11-08T19:26:37.579 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.564+0000 7fb25cf76700 1 --2- 172.21.15.143:0/1201643484 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fb240069820 0x7fb24006bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:37.579 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.564+0000 7fb266ee1080 1 -- 172.21.15.143:0/1201643484 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- pool_op(create snap pool 8 tid 1 name snap1 v0) v4 -- 0x55b5822394a0 con 0x55b582357720 2023-11-08T19:26:37.579 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.565+0000 7fb255ffb700 1 -- 172.21.15.143:0/1201643484 <== mon.0 v2:172.21.15.143:3300/0 6 ==== pool_op_reply(tid 1 (95) Operation not supported v48) v1 ==== 43+0+0 (secure 0 0 0) 0x7fb250053760 con 0x55b582357720 2023-11-08T19:26:37.579 INFO:teuthology.orchestra.run.smithi143.stderr:error creating pool cephfs_data snapshot snap1: (95) Operation not supported 2023-11-08T19:26:37.579 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.565+0000 7fb266ee1080 1 -- 172.21.15.143:0/1201643484 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fb240069820 msgr2=0x7fb24006bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:37.580 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.565+0000 7fb266ee1080 1 --2- 172.21.15.143:0/1201643484 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fb240069820 0x7fb24006bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:37.580 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.565+0000 7fb266ee1080 1 -- 172.21.15.143:0/1201643484 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b582357720 msgr2=0x55b5823509e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:37.580 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.565+0000 7fb266ee1080 1 --2- 172.21.15.143:0/1201643484 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b582357720 0x55b5823509e0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fb250003db0 tx=0x7fb25000e5b0 comp rx=0 tx=0).stop 2023-11-08T19:26:37.580 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.565+0000 7fb266ee1080 1 -- 172.21.15.143:0/1201643484 shutdown_connections 2023-11-08T19:26:37.580 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.565+0000 7fb266ee1080 1 --2- 172.21.15.143:0/1201643484 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fb240069820 0x7fb24006bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:37.581 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.565+0000 7fb266ee1080 1 --2- 172.21.15.143:0/1201643484 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55b5822cf890 0x55b5822c8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:37.581 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.565+0000 7fb266ee1080 1 --2- 172.21.15.143:0/1201643484 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb24c002440 0x55b5822b48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:37.581 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.565+0000 7fb266ee1080 1 --2- 172.21.15.143:0/1201643484 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b582357720 0x55b5823509e0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:37.581 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.565+0000 7fb266ee1080 1 -- 172.21.15.143:0/1201643484 >> 172.21.15.143:0/1201643484 conn(0x55b5822d03c0 msgr2=0x55b5822cfc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:37.581 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.565+0000 7fb266ee1080 1 -- 172.21.15.143:0/1201643484 shutdown_connections 2023-11-08T19:26:37.582 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.565+0000 7fb266ee1080 1 -- 172.21.15.143:0/1201643484 wait complete. 2023-11-08T19:26:37.582 DEBUG:teuthology.orchestra.run.smithi143:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata mksnap snap2 2023-11-08T19:26:37.662 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.660+0000 7f789520e080 1 Processor -- start 2023-11-08T19:26:37.662 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.660+0000 7f789520e080 1 -- start start 2023-11-08T19:26:37.666 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.661+0000 7f789520e080 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x5597e181d7d0 0x5597e18f2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:37.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.661+0000 7f789520e080 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5597e18f6770 0x5597e185fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:37.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.661+0000 7f789520e080 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597e18f6b50 0x5597e18f0b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:37.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.661+0000 7f789520e080 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597e17ccc30 con 0x5597e18f6770 2023-11-08T19:26:37.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.661+0000 7f789520e080 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597e17c8520 con 0x5597e18f6b50 2023-11-08T19:26:37.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.661+0000 7f789520e080 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x5597e1762740 con 0x5597e181d7d0 2023-11-08T19:26:37.668 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.661+0000 7f788aaa2700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5597e18f6770 0x5597e185fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:37.668 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.661+0000 7f788aaa2700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5597e18f6770 0x5597e185fe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51856/0 (socket says 172.21.15.143:51856) 2023-11-08T19:26:37.668 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.661+0000 7f788aaa2700 1 -- 172.21.15.143:0/2554851938 learned_addr learned my addr 172.21.15.143:0/2554851938 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:37.668 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.661+0000 7f788aaa2700 1 -- 172.21.15.143:0/2554851938 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x5597e181d7d0 msgr2=0x5597e18f2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:37.668 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.661+0000 7f788aaa2700 1 --2- 172.21.15.143:0/2554851938 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x5597e181d7d0 0x5597e18f2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:37.669 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.661+0000 7f788aaa2700 1 -- 172.21.15.143:0/2554851938 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597e18f6b50 msgr2=0x5597e18f0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:37.669 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.661+0000 7f788aaa2700 1 --2- 172.21.15.143:0/2554851938 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597e18f6b50 0x5597e18f0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:37.669 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.661+0000 7f788aaa2700 1 -- 172.21.15.143:0/2554851938 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f788400a100 con 0x5597e18f6770 2023-11-08T19:26:37.669 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.662+0000 7f788aaa2700 1 --2- 172.21.15.143:0/2554851938 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5597e18f6770 0x5597e185fe90 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f787c001ea0 tx=0x7f787c00b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4400a855fe927c4 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:37.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.662+0000 7f7889aa0700 1 -- 172.21.15.143:0/2554851938 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f787c014070 con 0x5597e18f6770 2023-11-08T19:26:37.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.662+0000 7f7889aa0700 1 -- 172.21.15.143:0/2554851938 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f787c00f3f0 con 0x5597e18f6770 2023-11-08T19:26:37.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.662+0000 7f7889aa0700 1 -- 172.21.15.143:0/2554851938 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f787c00f780 con 0x5597e18f6770 2023-11-08T19:26:37.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.662+0000 7f789520e080 1 -- 172.21.15.143:0/2554851938 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5597e18f6770 msgr2=0x5597e185fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:37.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.662+0000 7f789520e080 1 --2- 172.21.15.143:0/2554851938 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5597e18f6770 0x5597e185fe90 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f787c001ea0 tx=0x7f787c00b4f0 comp rx=0 tx=0).stop 2023-11-08T19:26:37.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.662+0000 7f789520e080 1 -- 172.21.15.143:0/2554851938 shutdown_connections 2023-11-08T19:26:37.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.662+0000 7f789520e080 1 --2- 172.21.15.143:0/2554851938 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x5597e181d7d0 0x5597e18f2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:37.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.662+0000 7f789520e080 1 --2- 172.21.15.143:0/2554851938 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597e18f6b50 0x5597e18f0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:37.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.662+0000 7f789520e080 1 --2- 172.21.15.143:0/2554851938 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5597e18f6770 0x5597e185fe90 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:37.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.662+0000 7f789520e080 1 -- 172.21.15.143:0/2554851938 >> 172.21.15.143:0/2554851938 conn(0x5597e1869c10 msgr2=0x5597e1849f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:37.672 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.662+0000 7f789520e080 1 -- 172.21.15.143:0/2554851938 shutdown_connections 2023-11-08T19:26:37.672 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.662+0000 7f789520e080 1 -- 172.21.15.143:0/2554851938 wait complete. 2023-11-08T19:26:37.672 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.663+0000 7f789520e080 1 Processor -- start 2023-11-08T19:26:37.672 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.663+0000 7f789520e080 1 -- start start 2023-11-08T19:26:37.672 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.664+0000 7f789520e080 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597e181d7d0 0x5597e198d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:37.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.664+0000 7f789520e080 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5597e18f6770 0x5597e18eea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:37.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.664+0000 7f789520e080 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x5597e18f6b50 0x5597e1867e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:37.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.664+0000 7f789520e080 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597e17623e0 con 0x5597e18f6770 2023-11-08T19:26:37.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.664+0000 7f789520e080 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597e1756680 con 0x5597e181d7d0 2023-11-08T19:26:37.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.664+0000 7f789520e080 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x5597e184c860 con 0x5597e18f6b50 2023-11-08T19:26:37.674 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.664+0000 7f788aaa2700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5597e18f6770 0x5597e18eea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:37.674 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.664+0000 7f788aaa2700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5597e18f6770 0x5597e18eea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51862/0 (socket says 172.21.15.143:51862) 2023-11-08T19:26:37.674 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.664+0000 7f788aaa2700 1 -- 172.21.15.143:0/3021691875 learned_addr learned my addr 172.21.15.143:0/3021691875 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:37.674 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.664+0000 7f788b2a3700 1 --2- 172.21.15.143:0/3021691875 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597e181d7d0 0x5597e198d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:37.674 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.664+0000 7f788baa4700 1 --2- 172.21.15.143:0/3021691875 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x5597e18f6b50 0x5597e1867e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:37.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.664+0000 7f788aaa2700 1 -- 172.21.15.143:0/3021691875 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x5597e18f6b50 msgr2=0x5597e1867e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:37.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.664+0000 7f788aaa2700 1 --2- 172.21.15.143:0/3021691875 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x5597e18f6b50 0x5597e1867e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:37.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.664+0000 7f788aaa2700 1 -- 172.21.15.143:0/3021691875 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597e181d7d0 msgr2=0x5597e198d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:37.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.664+0000 7f788aaa2700 1 --2- 172.21.15.143:0/3021691875 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597e181d7d0 0x5597e198d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:37.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.664+0000 7f788aaa2700 1 -- 172.21.15.143:0/3021691875 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f787c001ad0 con 0x5597e18f6770 2023-11-08T19:26:37.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.664+0000 7f788aaa2700 1 --2- 172.21.15.143:0/3021691875 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5597e18f6770 0x5597e18eea40 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f787c000f60 tx=0x7f787c000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:37.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.664+0000 7f7888a9e700 1 -- 172.21.15.143:0/3021691875 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f787c014070 con 0x5597e18f6770 2023-11-08T19:26:37.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.664+0000 7f7888a9e700 1 -- 172.21.15.143:0/3021691875 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f787c0011e0 con 0x5597e18f6770 2023-11-08T19:26:37.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.664+0000 7f789520e080 1 -- 172.21.15.143:0/3021691875 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5597e1761b70 con 0x5597e18f6770 2023-11-08T19:26:37.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.664+0000 7f7888a9e700 1 -- 172.21.15.143:0/3021691875 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f787c01c650 con 0x5597e18f6770 2023-11-08T19:26:37.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.664+0000 7f789520e080 1 -- 172.21.15.143:0/3021691875 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5597e17615d0 con 0x5597e18f6770 2023-11-08T19:26:37.679 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-08T19:26:37.679 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.666+0000 7f7888a9e700 1 -- 172.21.15.143:0/3021691875 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f787c00d040 con 0x5597e18f6770 2023-11-08T19:26:37.679 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.666+0000 7f7888a9e700 1 --2- 172.21.15.143:0/3021691875 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f7878069820 0x7f787806bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:37.680 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.666+0000 7f7888a9e700 1 -- 172.21.15.143:0/3021691875 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f787c00a040 con 0x5597e18f6770 2023-11-08T19:26:37.680 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.666+0000 7f789520e080 1 -- 172.21.15.143:0/3021691875 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- pool_op(create snap pool 7 tid 1 name snap2 v0) v4 -- 0x5597e17d24a0 con 0x5597e18f6770 2023-11-08T19:26:37.680 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.666+0000 7f788b2a3700 1 --2- 172.21.15.143:0/3021691875 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f7878069820 0x7f787806bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:37.680 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.666+0000 7f7888a9e700 1 -- 172.21.15.143:0/3021691875 <== mon.0 v2:172.21.15.143:3300/0 6 ==== pool_op_reply(tid 1 (95) Operation not supported v48) v1 ==== 43+0+0 (secure 0 0 0) 0x7f787c020080 con 0x5597e18f6770 2023-11-08T19:26:37.680 INFO:teuthology.orchestra.run.smithi143.stderr:error creating pool cephfs_metadata snapshot snap2: (95) Operation not supported 2023-11-08T19:26:37.681 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.667+0000 7f789520e080 1 -- 172.21.15.143:0/3021691875 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f7878069820 msgr2=0x7f787806bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:37.681 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.667+0000 7f789520e080 1 --2- 172.21.15.143:0/3021691875 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f7878069820 0x7f787806bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:37.681 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.667+0000 7f789520e080 1 -- 172.21.15.143:0/3021691875 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5597e18f6770 msgr2=0x5597e18eea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:37.681 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.667+0000 7f789520e080 1 --2- 172.21.15.143:0/3021691875 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5597e18f6770 0x5597e18eea40 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f787c000f60 tx=0x7f787c000ed0 comp rx=0 tx=0).stop 2023-11-08T19:26:37.681 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.667+0000 7f789520e080 1 -- 172.21.15.143:0/3021691875 shutdown_connections 2023-11-08T19:26:37.682 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.667+0000 7f789520e080 1 --2- 172.21.15.143:0/3021691875 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f7878069820 0x7f787806bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:37.682 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.667+0000 7f789520e080 1 --2- 172.21.15.143:0/3021691875 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x5597e18f6b50 0x5597e1867e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:37.682 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.667+0000 7f789520e080 1 --2- 172.21.15.143:0/3021691875 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597e181d7d0 0x5597e198d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:37.682 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.667+0000 7f789520e080 1 --2- 172.21.15.143:0/3021691875 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5597e18f6770 0x5597e18eea40 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:37.682 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.667+0000 7f789520e080 1 -- 172.21.15.143:0/3021691875 >> 172.21.15.143:0/3021691875 conn(0x5597e1869c10 msgr2=0x5597e18f01c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:37.683 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.667+0000 7f789520e080 1 -- 172.21.15.143:0/3021691875 shutdown_connections 2023-11-08T19:26:37.683 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.667+0000 7f789520e080 1 -- 172.21.15.143:0/3021691875 wait complete. 2023-11-08T19:26:37.683 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool mksnap cephfs_data snap3 2023-11-08T19:26:37.845 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.844+0000 7f20425aa700 1 Processor -- start 2023-11-08T19:26:37.846 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.844+0000 7f20425aa700 1 -- start start 2023-11-08T19:26:37.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.845+0000 7f20425aa700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f203c0b22e0 0x7f203c0b2750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:37.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.845+0000 7f20425aa700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f203c161890 con 0x7f203c15dba0 2023-11-08T19:26:37.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.845+0000 7f20425aa700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f203c06b740 con 0x7f203c06b240 2023-11-08T19:26:37.850 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.845+0000 7f20425aa700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f203c06cd20 con 0x7f203c0b22e0 2023-11-08T19:26:37.850 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.845+0000 7f203bfff700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f203c15dba0 0x7f203c06c610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:48022/0 (socket says 172.21.15.143:48022) 2023-11-08T19:26:37.850 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.845+0000 7f203bfff700 1 -- 172.21.15.143:0/3559794785 learned_addr learned my addr 172.21.15.143:0/3559794785 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:26:37.850 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.845+0000 7f2040b47700 1 --2- 172.21.15.143:0/3559794785 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f203c0b22e0 0x7f203c0b2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:37.851 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.845+0000 7f203affd700 1 -- 172.21.15.143:0/3559794785 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1423688906 0 0) 0x7f203c161890 con 0x7f203c15dba0 2023-11-08T19:26:37.851 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.845+0000 7f203affd700 1 -- 172.21.15.143:0/3559794785 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20240035b0 con 0x7f203c15dba0 2023-11-08T19:26:37.851 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.845+0000 7f203affd700 1 -- 172.21.15.143:0/3559794785 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3723005555 0 0) 0x7f20240035b0 con 0x7f203c15dba0 2023-11-08T19:26:37.851 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.845+0000 7f203affd700 1 -- 172.21.15.143:0/3559794785 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f203c0b22e0 msgr2=0x7f203c0b2750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:37.851 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.845+0000 7f203affd700 1 --2- 172.21.15.143:0/3559794785 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f203c0b22e0 0x7f203c0b2750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:37.852 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.845+0000 7f203affd700 1 -- 172.21.15.143:0/3559794785 >> v1:172.21.15.174:6789/0 conn(0x7f203c06b240 legacy=0x7f203c0caff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:37.852 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.845+0000 7f203affd700 1 -- 172.21.15.143:0/3559794785 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f203c06ba20 con 0x7f203c15dba0 2023-11-08T19:26:37.852 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.845+0000 7f203affd700 1 -- 172.21.15.143:0/3559794785 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4147756303 0 0) 0x7f202c001670 con 0x7f203c15dba0 2023-11-08T19:26:37.852 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.845+0000 7f203affd700 1 -- 172.21.15.143:0/3559794785 >> v1:172.21.15.143:6789/0 conn(0x7f203c15dba0 legacy=0x7f203c06c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:37.852 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.845+0000 7f203affd700 1 --2- 172.21.15.143:0/3559794785 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2024003a30 0x7f2024003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:37.852 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.845+0000 7f203affd700 1 --2- 172.21.15.143:0/3559794785 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f203c0b22e0 0x7f2024004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:37.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.845+0000 7f203affd700 1 --2- 172.21.15.143:0/3559794785 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2024004d90 0x7f2024005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:37.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.845+0000 7f203affd700 1 -- 172.21.15.143:0/3559794785 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f203c06cd20 con 0x7f2024003a30 2023-11-08T19:26:37.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.845+0000 7f203affd700 1 -- 172.21.15.143:0/3559794785 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f202400ba10 con 0x7f2024004d90 2023-11-08T19:26:37.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.845+0000 7f203affd700 1 -- 172.21.15.143:0/3559794785 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f202400bbd0 con 0x7f203c0b22e0 2023-11-08T19:26:37.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.845+0000 7f203bfff700 1 --2- 172.21.15.143:0/3559794785 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2024003a30 0x7f2024003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:37.854 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.846+0000 7f2040b47700 1 --2- 172.21.15.143:0/3559794785 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2024004d90 0x7f2024005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:37.854 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.846+0000 7f203b7fe700 1 --2- 172.21.15.143:0/3559794785 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f203c0b22e0 0x7f2024004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:37.854 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.846+0000 7f203bfff700 1 -- 172.21.15.143:0/3559794785 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f203c0b22e0 msgr2=0x7f2024004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:37.854 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.846+0000 7f203bfff700 1 --2- 172.21.15.143:0/3559794785 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f203c0b22e0 0x7f2024004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:37.854 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.846+0000 7f203bfff700 1 -- 172.21.15.143:0/3559794785 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2024004d90 msgr2=0x7f2024005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:37.855 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.846+0000 7f203bfff700 1 --2- 172.21.15.143:0/3559794785 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2024004d90 0x7f2024005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:37.855 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.846+0000 7f203bfff700 1 -- 172.21.15.143:0/3559794785 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20240035b0 con 0x7f2024003a30 2023-11-08T19:26:37.855 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.846+0000 7f203b7fe700 1 --2- 172.21.15.143:0/3559794785 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f203c0b22e0 0x7f2024004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:26:37.855 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.846+0000 7f203bfff700 1 --2- 172.21.15.143:0/3559794785 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2024003a30 0x7f2024003e10 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f202c00c240 tx=0x7f2024005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=895c1388529e95c0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:37.855 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.846+0000 7f203affd700 1 -- 172.21.15.143:0/3559794785 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f202c00cd20 con 0x7f2024003a30 2023-11-08T19:26:37.856 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.846+0000 7f203affd700 1 -- 172.21.15.143:0/3559794785 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f202c00ce80 con 0x7f2024003a30 2023-11-08T19:26:37.856 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.846+0000 7f203affd700 1 -- 172.21.15.143:0/3559794785 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f202c005b80 con 0x7f2024003a30 2023-11-08T19:26:37.856 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.846+0000 7f20425aa700 1 -- 172.21.15.143:0/3559794785 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2024003a30 msgr2=0x7f2024003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:37.856 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.846+0000 7f20425aa700 1 --2- 172.21.15.143:0/3559794785 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2024003a30 0x7f2024003e10 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f202c00c240 tx=0x7f2024005880 comp rx=0 tx=0).stop 2023-11-08T19:26:37.856 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.846+0000 7f2040b47700 1 -- 172.21.15.143:0/3559794785 reap_dead start 2023-11-08T19:26:37.857 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.847+0000 7f20425aa700 1 -- 172.21.15.143:0/3559794785 shutdown_connections 2023-11-08T19:26:37.857 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.847+0000 7f20425aa700 1 -- 172.21.15.143:0/3559794785 >> 172.21.15.143:0/3559794785 conn(0x7f203c0cd2e0 msgr2=0x7f203c0cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:37.857 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.847+0000 7f20425aa700 1 -- 172.21.15.143:0/3559794785 shutdown_connections 2023-11-08T19:26:37.857 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.847+0000 7f20425aa700 1 -- 172.21.15.143:0/3559794785 wait complete. 2023-11-08T19:26:37.857 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.848+0000 7f20425aa700 1 Processor -- start 2023-11-08T19:26:37.860 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.848+0000 7f20425aa700 1 -- start start 2023-11-08T19:26:37.860 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.849+0000 7f20425aa700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2024003a30 0x7f203c15b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:37.861 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.849+0000 7f20425aa700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f203c15bff0 0x7f203c15e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:37.861 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.849+0000 7f20425aa700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f203c0ad690 0x7f203c0ca2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:37.861 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.849+0000 7f20425aa700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f203c0b44b0 con 0x7f203c0ad690 2023-11-08T19:26:37.861 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.849+0000 7f20425aa700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f203c0b4700 con 0x7f2024003a30 2023-11-08T19:26:37.861 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.849+0000 7f20425aa700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f203c0b49e0 con 0x7f203c15bff0 2023-11-08T19:26:37.862 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.849+0000 7f2040b47700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f203c0ad690 0x7f203c0ca2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:37.862 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.849+0000 7f2040b47700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f203c0ad690 0x7f203c0ca2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51884/0 (socket says 172.21.15.143:51884) 2023-11-08T19:26:37.862 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.849+0000 7f2040b47700 1 -- 172.21.15.143:0/2006806512 learned_addr learned my addr 172.21.15.143:0/2006806512 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:37.862 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.849+0000 7f203bfff700 1 --2- 172.21.15.143:0/2006806512 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2024003a30 0x7f203c15b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:37.862 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.850+0000 7f203b7fe700 1 --2- 172.21.15.143:0/2006806512 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f203c15bff0 0x7f203c15e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:37.863 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.850+0000 7f2040b47700 1 -- 172.21.15.143:0/2006806512 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f203c15bff0 msgr2=0x7f203c15e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:37.863 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.850+0000 7f2040b47700 1 --2- 172.21.15.143:0/2006806512 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f203c15bff0 0x7f203c15e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:37.863 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.850+0000 7f2040b47700 1 -- 172.21.15.143:0/2006806512 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2024003a30 msgr2=0x7f203c15b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:37.863 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.850+0000 7f2040b47700 1 --2- 172.21.15.143:0/2006806512 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2024003a30 0x7f203c15b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:37.864 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.850+0000 7f2040b47700 1 -- 172.21.15.143:0/2006806512 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f203c0b4cc0 con 0x7f203c0ad690 2023-11-08T19:26:37.864 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.850+0000 7f203bfff700 1 --2- 172.21.15.143:0/2006806512 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2024003a30 0x7f203c15b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:26:37.864 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.850+0000 7f2040b47700 1 --2- 172.21.15.143:0/2006806512 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f203c0ad690 0x7f203c0ca2e0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f203c0cd1c0 tx=0x7f2034011f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:37.865 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.850+0000 7f2038ff9700 1 -- 172.21.15.143:0/2006806512 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f203400e040 con 0x7f203c0ad690 2023-11-08T19:26:37.865 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.850+0000 7f2038ff9700 1 -- 172.21.15.143:0/2006806512 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f203400f040 con 0x7f203c0ad690 2023-11-08T19:26:37.865 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.850+0000 7f20425aa700 1 -- 172.21.15.143:0/2006806512 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f203c0b4fa0 con 0x7f203c0ad690 2023-11-08T19:26:37.865 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.850+0000 7f2038ff9700 1 -- 172.21.15.143:0/2006806512 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f203400fdb0 con 0x7f203c0ad690 2023-11-08T19:26:37.865 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.850+0000 7f20425aa700 1 -- 172.21.15.143:0/2006806512 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f203c0adb20 con 0x7f203c0ad690 2023-11-08T19:26:37.866 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.851+0000 7f20425aa700 1 -- 172.21.15.143:0/2006806512 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f203c04eee0 con 0x7f203c0ad690 2023-11-08T19:26:37.866 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.851+0000 7f2038ff9700 1 -- 172.21.15.143:0/2006806512 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2034022460 con 0x7f203c0ad690 2023-11-08T19:26:37.866 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.852+0000 7f2038ff9700 1 --2- 172.21.15.143:0/2006806512 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f202806af00 0x7f202806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:37.866 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.852+0000 7f2038ff9700 1 -- 172.21.15.143:0/2006806512 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2034089c60 con 0x7f203c0ad690 2023-11-08T19:26:37.866 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.852+0000 7f203bfff700 1 --2- 172.21.15.143:0/2006806512 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f202806af00 0x7f202806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:37.867 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.852+0000 7f203bfff700 1 --2- 172.21.15.143:0/2006806512 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f202806af00 0x7f202806d3c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f202c015040 tx=0x7f202c007000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:37.867 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:37.856+0000 7f2038ff9700 1 -- 172.21.15.143:0/2006806512 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2034054bb0 con 0x7f203c0ad690 2023-11-08T19:26:38.045 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.044+0000 7f20425aa700 1 -- 172.21.15.143:0/2006806512 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool mksnap", "pool": "cephfs_data", "snap": "snap3"} v 0) v1 -- 0x7f203c0b67e0 con 0x7f203c0ad690 2023-11-08T19:26:38.045 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.044+0000 7f2038ff9700 1 -- 172.21.15.143:0/2006806512 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool mksnap", "pool": "cephfs_data", "snap": "snap3"}]=-95 (95) Operation not supported v48) v1 ==== 131+0+0 (secure 0 0 0) 0x7f20340581d0 con 0x7f203c0ad690 2023-11-08T19:26:38.047 INFO:teuthology.orchestra.run.smithi143.stderr:Error ENOTSUP: (95) Operation not supported 2023-11-08T19:26:38.050 DEBUG:teuthology.orchestra.run:got remote process result: 95 2023-11-08T19:26:38.051 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool mksnap cephfs_metadata snap4 2023-11-08T19:26:38.174 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.173+0000 7fc7d6b92700 1 Processor -- start 2023-11-08T19:26:38.174 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.173+0000 7fc7d6b92700 1 -- start start 2023-11-08T19:26:38.177 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.173+0000 7fc7d6b92700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc7d015dd60 0x7fc7d01623f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:38.177 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.173+0000 7fc7d6b92700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc7d00ce660 0x7fc7d00b2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:38.177 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.173+0000 7fc7d6b92700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fc7d00b2840 0x7fc7d00721c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:38.177 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.173+0000 7fc7d6b92700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7d00b2ca0 con 0x7fc7d015dd60 2023-11-08T19:26:38.178 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.174+0000 7fc7d6b92700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7d0162930 con 0x7fc7d00b2840 2023-11-08T19:26:38.178 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.174+0000 7fc7d6b92700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d006b2b0 con 0x7fc7d00ce660 2023-11-08T19:26:38.178 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.174+0000 7fc7d492e700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc7d015dd60 0x7fc7d01623f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:38.178 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.174+0000 7fc7d492e700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc7d015dd60 0x7fc7d01623f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51888/0 (socket says 172.21.15.143:51888) 2023-11-08T19:26:38.179 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.174+0000 7fc7d492e700 1 -- 172.21.15.143:0/754139533 learned_addr learned my addr 172.21.15.143:0/754139533 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:38.179 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.174+0000 7fc7d512f700 1 --2- 172.21.15.143:0/754139533 >> v2:172.21.15.174:3300/0 conn(0x7fc7d00b2840 0x7fc7d00721c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:38.179 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.174+0000 7fc7cffff700 1 --2- 172.21.15.143:0/754139533 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc7d00ce660 0x7fc7d00b2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:38.179 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.174+0000 7fc7d492e700 1 -- 172.21.15.143:0/754139533 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc7d00ce660 msgr2=0x7fc7d00b2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:38.179 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.174+0000 7fc7d492e700 1 --2- 172.21.15.143:0/754139533 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc7d00ce660 0x7fc7d00b2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:38.180 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.174+0000 7fc7d492e700 1 -- 172.21.15.143:0/754139533 >> v2:172.21.15.174:3300/0 conn(0x7fc7d00b2840 msgr2=0x7fc7d00721c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:38.180 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.174+0000 7fc7d492e700 1 --2- 172.21.15.143:0/754139533 >> v2:172.21.15.174:3300/0 conn(0x7fc7d00b2840 0x7fc7d00721c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:38.180 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.174+0000 7fc7d492e700 1 -- 172.21.15.143:0/754139533 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7d00cf000 con 0x7fc7d015dd60 2023-11-08T19:26:38.180 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.174+0000 7fc7d492e700 1 --2- 172.21.15.143:0/754139533 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc7d015dd60 0x7fc7d01623f0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fc7d00ce060 tx=0x7fc7c4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3c8ae6c69475a97 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:38.181 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.174+0000 7fc7cf7fe700 1 -- 172.21.15.143:0/754139533 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c400b870 con 0x7fc7d015dd60 2023-11-08T19:26:38.181 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.174+0000 7fc7cf7fe700 1 -- 172.21.15.143:0/754139533 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7c400b9d0 con 0x7fc7d015dd60 2023-11-08T19:26:38.181 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.174+0000 7fc7cf7fe700 1 -- 172.21.15.143:0/754139533 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c400bd60 con 0x7fc7d015dd60 2023-11-08T19:26:38.181 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.175+0000 7fc7d6b92700 1 -- 172.21.15.143:0/754139533 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc7d015dd60 msgr2=0x7fc7d01623f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:38.182 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.175+0000 7fc7d6b92700 1 --2- 172.21.15.143:0/754139533 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc7d015dd60 0x7fc7d01623f0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fc7d00ce060 tx=0x7fc7c4009bd0 comp rx=0 tx=0).stop 2023-11-08T19:26:38.182 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.175+0000 7fc7d6b92700 1 -- 172.21.15.143:0/754139533 shutdown_connections 2023-11-08T19:26:38.182 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.175+0000 7fc7d6b92700 1 --2- 172.21.15.143:0/754139533 >> v2:172.21.15.174:3300/0 conn(0x7fc7d00b2840 0x7fc7d00721c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:38.183 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.175+0000 7fc7d6b92700 1 --2- 172.21.15.143:0/754139533 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc7d015dd60 0x7fc7d01623f0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:38.183 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.175+0000 7fc7d6b92700 1 --2- 172.21.15.143:0/754139533 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc7d00ce660 0x7fc7d00b2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:38.183 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.175+0000 7fc7d6b92700 1 -- 172.21.15.143:0/754139533 >> 172.21.15.143:0/754139533 conn(0x7fc7d00cd300 msgr2=0x7fc7d00cd710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:38.183 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.175+0000 7fc7d6b92700 1 -- 172.21.15.143:0/754139533 shutdown_connections 2023-11-08T19:26:38.183 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.175+0000 7fc7d6b92700 1 -- 172.21.15.143:0/754139533 wait complete. 2023-11-08T19:26:38.186 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.176+0000 7fc7d6b92700 1 Processor -- start 2023-11-08T19:26:38.187 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.176+0000 7fc7d6b92700 1 -- start start 2023-11-08T19:26:38.187 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.177+0000 7fc7d6b92700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc7d00b2840 0x7fc7d01f0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:38.187 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.177+0000 7fc7d6b92700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc7d00ce660 0x7fc7d01f4860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:38.187 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.178+0000 7fc7d6b92700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc7d015dd60 0x7fc7d00c9c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:38.188 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.178+0000 7fc7d6b92700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d00c55d0 con 0x7fc7d00ce660 2023-11-08T19:26:38.188 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.178+0000 7fc7d6b92700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d00c58b0 con 0x7fc7d00b2840 2023-11-08T19:26:38.188 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.178+0000 7fc7d6b92700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d00c5b90 con 0x7fc7d015dd60 2023-11-08T19:26:38.188 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.178+0000 7fc7cffff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc7d00ce660 0x7fc7d01f4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:38.188 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.178+0000 7fc7cffff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc7d00ce660 0x7fc7d01f4860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51896/0 (socket says 172.21.15.143:51896) 2023-11-08T19:26:38.189 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.178+0000 7fc7cffff700 1 -- 172.21.15.143:0/627153383 learned_addr learned my addr 172.21.15.143:0/627153383 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:38.189 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.178+0000 7fc7d492e700 1 --2- 172.21.15.143:0/627153383 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc7d00b2840 0x7fc7d01f0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:38.189 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.178+0000 7fc7cffff700 1 -- 172.21.15.143:0/627153383 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc7d015dd60 msgr2=0x7fc7d00c9c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:38.189 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.178+0000 7fc7cffff700 1 --2- 172.21.15.143:0/627153383 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc7d015dd60 0x7fc7d00c9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:38.189 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.178+0000 7fc7cffff700 1 -- 172.21.15.143:0/627153383 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc7d00b2840 msgr2=0x7fc7d01f0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:38.190 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.178+0000 7fc7cffff700 1 --2- 172.21.15.143:0/627153383 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc7d00b2840 0x7fc7d01f0cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:38.190 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.178+0000 7fc7cffff700 1 -- 172.21.15.143:0/627153383 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7c4009550 con 0x7fc7d00ce660 2023-11-08T19:26:38.190 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.178+0000 7fc7cffff700 1 --2- 172.21.15.143:0/627153383 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc7d00ce660 0x7fc7d01f4860 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fc7bc00d800 tx=0x7fc7bc00dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:38.190 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.178+0000 7fc7cd7fa700 1 -- 172.21.15.143:0/627153383 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7bc0125f0 con 0x7fc7d00ce660 2023-11-08T19:26:38.191 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.178+0000 7fc7cd7fa700 1 -- 172.21.15.143:0/627153383 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7bc012750 con 0x7fc7d00ce660 2023-11-08T19:26:38.191 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.178+0000 7fc7d6b92700 1 -- 172.21.15.143:0/627153383 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7d00c5e70 con 0x7fc7d00ce660 2023-11-08T19:26:38.191 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.178+0000 7fc7cd7fa700 1 -- 172.21.15.143:0/627153383 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7bc012b00 con 0x7fc7d00ce660 2023-11-08T19:26:38.191 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.178+0000 7fc7d6b92700 1 -- 172.21.15.143:0/627153383 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7d01f4da0 con 0x7fc7d00ce660 2023-11-08T19:26:38.192 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.179+0000 7fc7d6b92700 1 -- 172.21.15.143:0/627153383 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7d004eee0 con 0x7fc7d00ce660 2023-11-08T19:26:38.192 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.180+0000 7fc7cd7fa700 1 -- 172.21.15.143:0/627153383 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc7bc012c60 con 0x7fc7d00ce660 2023-11-08T19:26:38.192 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.180+0000 7fc7cd7fa700 1 --2- 172.21.15.143:0/627153383 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc7b406af00 0x7fc7b406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:38.192 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.180+0000 7fc7cd7fa700 1 -- 172.21.15.143:0/627153383 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc7bc089b70 con 0x7fc7d00ce660 2023-11-08T19:26:38.192 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.180+0000 7fc7d492e700 1 --2- 172.21.15.143:0/627153383 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc7b406af00 0x7fc7b406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:38.193 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.181+0000 7fc7d492e700 1 --2- 172.21.15.143:0/627153383 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc7b406af00 0x7fc7b406d3c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fc7c4009c10 tx=0x7fc7c4008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:38.193 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.184+0000 7fc7cd7fa700 1 -- 172.21.15.143:0/627153383 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc7bc054a40 con 0x7fc7d00ce660 2023-11-08T19:26:38.375 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.373+0000 7fc7d6b92700 1 -- 172.21.15.143:0/627153383 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool mksnap", "pool": "cephfs_metadata", "snap": "snap4"} v 0) v1 -- 0x7fc7d00c5710 con 0x7fc7d00ce660 2023-11-08T19:26:38.375 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.374+0000 7fc7cd7fa700 1 -- 172.21.15.143:0/627153383 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool mksnap", "pool": "cephfs_metadata", "snap": "snap4"}]=-95 (95) Operation not supported v48) v1 ==== 135+0+0 (secure 0 0 0) 0x7fc7d00c5710 con 0x7fc7d00ce660 2023-11-08T19:26:38.377 INFO:teuthology.orchestra.run.smithi143.stderr:Error ENOTSUP: (95) Operation not supported 2023-11-08T19:26:38.380 DEBUG:teuthology.orchestra.run:got remote process result: 95 2023-11-08T19:26:38.381 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-08T19:26:38.381 DEBUG:teuthology.orchestra.run.smithi143:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-08T19:26:38.424 DEBUG:teuthology.orchestra.run.smithi174:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-08T19:26:38.474 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-08T19:26:38.474 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-08T19:26:38.475 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:26:38.493 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-08T19:26:38.493 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:26:38.551 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:26:38.710 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.709+0000 7f65980dc700 1 Processor -- start 2023-11-08T19:26:38.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.709+0000 7f65980dc700 1 -- start start 2023-11-08T19:26:38.713 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.710+0000 7f65980dc700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f65901618f0 0x7f659006ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:38.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.710+0000 7f65980dc700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f659006b150 0x7f65900cb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:38.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.710+0000 7f65980dc700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6590161df0 con 0x7f659015dd00 2023-11-08T19:26:38.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.710+0000 7f65980dc700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f659006b600 con 0x7f65901618f0 2023-11-08T19:26:38.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.710+0000 7f65980dc700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65900cad50 con 0x7f659006b150 2023-11-08T19:26:38.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.710+0000 7f6596679700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f659006b150 0x7f65900cb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:38.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.710+0000 7f6595677700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f65901618f0 0x7f659006ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:38.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.710+0000 7f6595e78700 1 --1- >> v1:172.21.15.174:6789/0 conn(0x7f659015dd00 0x7f65900cca70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.174:6789/0 says I am v1:172.21.15.143:58990/0 (socket says 172.21.15.143:58990) 2023-11-08T19:26:38.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.710+0000 7f6595e78700 1 -- 172.21.15.143:0/3046873438 learned_addr learned my addr 172.21.15.143:0/3046873438 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:26:38.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.711+0000 7f6595677700 1 -- 172.21.15.143:0/3046873438 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f659006b150 msgr2=0x7f65900cb1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:38.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.711+0000 7f6595677700 1 --2- 172.21.15.143:0/3046873438 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f659006b150 0x7f65900cb1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:38.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.711+0000 7f6595677700 1 -- 172.21.15.143:0/3046873438 >> v1:172.21.15.174:6789/0 conn(0x7f659015dd00 legacy=0x7f65900cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:38.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.711+0000 7f6595677700 1 -- 172.21.15.143:0/3046873438 --> v2:172.21.15.174:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65901620d0 con 0x7f65901618f0 2023-11-08T19:26:38.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.711+0000 7f6596679700 1 --2- 172.21.15.143:0/3046873438 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f659006b150 0x7f65900cb1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T19:26:38.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.711+0000 7f6594e76700 1 -- 172.21.15.143:0/3046873438 <== mon.1 v1:172.21.15.174:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2822190421 0 0) 0x7f6590161df0 con 0x7f659015dd00 2023-11-08T19:26:38.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.711+0000 7f6595677700 1 --2- 172.21.15.143:0/3046873438 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f65901618f0 0x7f659006ac10 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f658c009840 tx=0x7f658c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b63522285bd9614c server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:38.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.711+0000 7f6594e76700 1 -- 172.21.15.143:0/3046873438 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f658c011830 con 0x7f65901618f0 2023-11-08T19:26:38.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.711+0000 7f6594e76700 1 -- 172.21.15.143:0/3046873438 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f658c011990 con 0x7f65901618f0 2023-11-08T19:26:38.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.711+0000 7f6594e76700 1 -- 172.21.15.143:0/3046873438 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f658c011d20 con 0x7f65901618f0 2023-11-08T19:26:38.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.712+0000 7f65980dc700 1 -- 172.21.15.143:0/3046873438 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f65901618f0 msgr2=0x7f659006ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:38.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.712+0000 7f65980dc700 1 --2- 172.21.15.143:0/3046873438 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f65901618f0 0x7f659006ac10 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f658c009840 tx=0x7f658c009b50 comp rx=0 tx=0).stop 2023-11-08T19:26:38.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.712+0000 7f65980dc700 1 -- 172.21.15.143:0/3046873438 shutdown_connections 2023-11-08T19:26:38.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.712+0000 7f65980dc700 1 --2- 172.21.15.143:0/3046873438 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f659006b150 0x7f65900cb1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:38.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.712+0000 7f65980dc700 1 --2- 172.21.15.143:0/3046873438 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f65901618f0 0x7f659006ac10 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:38.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.712+0000 7f65980dc700 1 -- 172.21.15.143:0/3046873438 >> 172.21.15.143:0/3046873438 conn(0x7f6590099ab0 msgr2=0x7f65900d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:38.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.712+0000 7f65980dc700 1 -- 172.21.15.143:0/3046873438 shutdown_connections 2023-11-08T19:26:38.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.712+0000 7f65980dc700 1 -- 172.21.15.143:0/3046873438 wait complete. 2023-11-08T19:26:38.724 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.713+0000 7f65980dc700 1 Processor -- start 2023-11-08T19:26:38.724 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.713+0000 7f65980dc700 1 -- start start 2023-11-08T19:26:38.724 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.714+0000 7f65980dc700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f659006b150 0x7f6590145290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:38.724 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.714+0000 7f65980dc700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f659015dd00 0x7f65900c77d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:38.725 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.714+0000 7f65980dc700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f65901618f0 0x7f659006e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:38.725 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.714+0000 7f65980dc700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65901457d0 con 0x7f65901618f0 2023-11-08T19:26:38.725 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.714+0000 7f65980dc700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65900ca1e0 con 0x7f659006b150 2023-11-08T19:26:38.726 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.714+0000 7f65980dc700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65900c95d0 con 0x7f659015dd00 2023-11-08T19:26:38.726 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.714+0000 7f6596679700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f65901618f0 0x7f659006e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:38.727 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.714+0000 7f6596679700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f65901618f0 0x7f659006e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51910/0 (socket says 172.21.15.143:51910) 2023-11-08T19:26:38.727 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.714+0000 7f6596679700 1 -- 172.21.15.143:0/159720660 learned_addr learned my addr 172.21.15.143:0/159720660 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:38.727 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.714+0000 7f6595677700 1 --2- 172.21.15.143:0/159720660 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f659015dd00 0x7f65900c77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:38.727 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.714+0000 7f6595e78700 1 --2- 172.21.15.143:0/159720660 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f659006b150 0x7f6590145290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:38.728 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.714+0000 7f6596679700 1 -- 172.21.15.143:0/159720660 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f659015dd00 msgr2=0x7f65900c77d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:38.728 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.714+0000 7f6596679700 1 --2- 172.21.15.143:0/159720660 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f659015dd00 0x7f65900c77d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:38.728 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.714+0000 7f6596679700 1 -- 172.21.15.143:0/159720660 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f659006b150 msgr2=0x7f6590145290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:38.728 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.714+0000 7f6596679700 1 --2- 172.21.15.143:0/159720660 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f659006b150 0x7f6590145290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:38.728 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.714+0000 7f6596679700 1 -- 172.21.15.143:0/159720660 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6588009a70 con 0x7f65901618f0 2023-11-08T19:26:38.728 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.714+0000 7f6596679700 1 --2- 172.21.15.143:0/159720660 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f65901618f0 0x7f659006e800 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f658800efc0 tx=0x7f658800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:38.729 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.715+0000 7f65867fc700 1 -- 172.21.15.143:0/159720660 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f658800ca90 con 0x7f65901618f0 2023-11-08T19:26:38.729 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.715+0000 7f65867fc700 1 -- 172.21.15.143:0/159720660 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f658800f040 con 0x7f65901618f0 2023-11-08T19:26:38.729 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.715+0000 7f65980dc700 1 -- 172.21.15.143:0/159720660 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f658c0094a0 con 0x7f65901618f0 2023-11-08T19:26:38.729 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.715+0000 7f65867fc700 1 -- 172.21.15.143:0/159720660 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f658800ce10 con 0x7f65901618f0 2023-11-08T19:26:38.729 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.715+0000 7f65980dc700 1 -- 172.21.15.143:0/159720660 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65900c9b90 con 0x7f65901618f0 2023-11-08T19:26:38.730 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.716+0000 7f65980dc700 1 -- 172.21.15.143:0/159720660 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65900aff60 con 0x7f65901618f0 2023-11-08T19:26:38.730 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.716+0000 7f65867fc700 1 -- 172.21.15.143:0/159720660 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6588014480 con 0x7f65901618f0 2023-11-08T19:26:38.730 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.716+0000 7f65867fc700 1 --2- 172.21.15.143:0/159720660 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f657406af00 0x7f657406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:38.730 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.717+0000 7f65867fc700 1 -- 172.21.15.143:0/159720660 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6588089db0 con 0x7f65901618f0 2023-11-08T19:26:38.730 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.717+0000 7f6595e78700 1 --2- 172.21.15.143:0/159720660 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f657406af00 0x7f657406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:38.731 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.717+0000 7f6595e78700 1 --2- 172.21.15.143:0/159720660 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f657406af00 0x7f657406d3c0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f6580000fb0 tx=0x7f6580007db0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:38.731 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.721+0000 7f65867fc700 1 -- 172.21.15.143:0/159720660 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6588054d00 con 0x7f65901618f0 2023-11-08T19:26:38.919 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.918+0000 7f65980dc700 1 -- 172.21.15.143:0/159720660 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f65900ca350 con 0x7f65901618f0 2023-11-08T19:26:38.921 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.919+0000 7f65867fc700 1 -- 172.21.15.143:0/159720660 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) v1 ==== 74+0+18466 (secure 0 0 0) 0x7f6588009d70 con 0x7f65901618f0 2023-11-08T19:26:38.921 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:26:38.922 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":49,"fsid":"968483af-77fc-4068-ad7c-5c2a1a4613d7","created":"2023-11-08T19:24:32.826484+0000","modified":"2023-11-08T19:26:38.790282+0000","last_up_change":"2023-11-08T19:24:43.573880+0000","last_in_change":"2023-11-08T19:24:35.314525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:44.551530+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"snap-test-pool","create_time":"2023-11-08T19:26:02.874384+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":36,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-11-08T19:26:22.869780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-11-08T19:26:24.093179+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bae98ec1-d94e-403d-a220-a522a53ea6f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6809","nonce":1487830566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6811","nonce":1487830566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6815","nonce":1487830566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6813","nonce":1487830566}]},"public_addr":"172.21.15.143:6809/1487830566","cluster_addr":"172.21.15.143:6811/1487830566","heartbeat_back_addr":"172.21.15.143:6815/1487830566","heartbeat_front_addr":"172.21.15.143:6813/1487830566","state":["exists","up"]},{"osd":1,"uuid":"5cb6538a-1ccb-4322-8633-09f66af0b7e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6801","nonce":787341397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6803","nonce":787341397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6807","nonce":787341397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6805","nonce":787341397}]},"public_addr":"172.21.15.143:6801/787341397","cluster_addr":"172.21.15.143:6803/787341397","heartbeat_back_addr":"172.21.15.143:6807/787341397","heartbeat_front_addr":"172.21.15.143:6805/787341397","state":["exists","up"]},{"osd":2,"uuid":"cdbf522c-60ef-4d60-9653-8dcb529c101a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6822","nonce":2301325597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6826","nonce":2301325597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6831","nonce":2301325597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6829","nonce":2301325597}]},"public_addr":"172.21.15.143:6822/2301325597","cluster_addr":"172.21.15.143:6826/2301325597","heartbeat_back_addr":"172.21.15.143:6831/2301325597","heartbeat_front_addr":"172.21.15.143:6829/2301325597","state":["exists","up"]},{"osd":3,"uuid":"4e2783cf-ebb9-4dd8-8245-bc1d17ce4325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6817","nonce":2021504428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6819","nonce":2021504428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6827","nonce":2021504428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6823","nonce":2021504428}]},"public_addr":"172.21.15.143:6817/2021504428","cluster_addr":"172.21.15.143:6819/2021504428","heartbeat_back_addr":"172.21.15.143:6827/2021504428","heartbeat_front_addr":"172.21.15.143:6823/2021504428","state":["exists","up"]},{"osd":4,"uuid":"691256fa-9b77-49b9-a346-7ae1adf0ca80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6801","nonce":1993380595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6803","nonce":1993380595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6807","nonce":1993380595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6805","nonce":1993380595}]},"public_addr":"172.21.15.174:6801/1993380595","cluster_addr":"172.21.15.174:6803/1993380595","heartbeat_back_addr":"172.21.15.174:6807/1993380595","heartbeat_front_addr":"172.21.15.174:6805/1993380595","state":["exists","up"]},{"osd":5,"uuid":"756fc352-48c9-46ce-b52e-0541fe6f8525","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6809","nonce":1777923303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6811","nonce":1777923303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6815","nonce":1777923303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6813","nonce":1777923303}]},"public_addr":"172.21.15.174:6809/1777923303","cluster_addr":"172.21.15.174:6811/1777923303","heartbeat_back_addr":"172.21.15.174:6815/1777923303","heartbeat_front_addr":"172.21.15.174:6813/1777923303","state":["exists","up"]},{"osd":6,"uuid":"fab2ac2b-ed97-4cde-9825-b480eabf8b8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6825","nonce":2755130316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6827","nonce":2755130316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6831","nonce":2755130316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6829","nonce":2755130316}]},"public_addr":"172.21.15.174:6825/2755130316","cluster_addr":"172.21.15.174:6827/2755130316","heartbeat_back_addr":"172.21.15.174:6831/2755130316","heartbeat_front_addr":"172.21.15.174:6829/2755130316","state":["exists","up"]},{"osd":7,"uuid":"3de9b797-2ab6-4d41-a863-7e7d638fd703","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6817","nonce":2847141411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6819","nonce":2847141411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6823","nonce":2847141411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6821","nonce":2847141411}]},"public_addr":"172.21.15.174:6817/2847141411","cluster_addr":"172.21.15.174:6819/2847141411","heartbeat_back_addr":"172.21.15.174:6823/2847141411","heartbeat_front_addr":"172.21.15.174:6821/2847141411","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.477076+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.496890+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.539305+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.454673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.538881+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.521294+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.520712+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.540404+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.1a","mappings":[{"from":0,"to":1}]},{"pgid":"8.28","mappings":[{"from":0,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:26:38.923 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.921+0000 7f65980dc700 1 -- 172.21.15.143:0/159720660 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f657406af00 msgr2=0x7f657406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:38.923 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.921+0000 7f65980dc700 1 --2- 172.21.15.143:0/159720660 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f657406af00 0x7f657406d3c0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f6580000fb0 tx=0x7f6580007db0 comp rx=0 tx=0).stop 2023-11-08T19:26:38.923 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.921+0000 7f65980dc700 1 -- 172.21.15.143:0/159720660 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f65901618f0 msgr2=0x7f659006e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:38.924 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.921+0000 7f65980dc700 1 --2- 172.21.15.143:0/159720660 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f65901618f0 0x7f659006e800 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f658800efc0 tx=0x7f658800c3a0 comp rx=0 tx=0).stop 2023-11-08T19:26:38.924 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.921+0000 7f65980dc700 1 -- 172.21.15.143:0/159720660 shutdown_connections 2023-11-08T19:26:38.924 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.921+0000 7f65980dc700 1 --2- 172.21.15.143:0/159720660 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f657406af00 0x7f657406d3c0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:38.924 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.921+0000 7f65980dc700 1 --2- 172.21.15.143:0/159720660 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f65901618f0 0x7f659006e800 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:38.925 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.921+0000 7f65980dc700 1 --2- 172.21.15.143:0/159720660 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f659006b150 0x7f6590145290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:38.925 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.921+0000 7f65980dc700 1 --2- 172.21.15.143:0/159720660 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f659015dd00 0x7f65900c77d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:38.925 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.921+0000 7f65980dc700 1 -- 172.21.15.143:0/159720660 >> 172.21.15.143:0/159720660 conn(0x7f6590099ab0 msgr2=0x7f65900cc700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:38.925 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.921+0000 7f65980dc700 1 -- 172.21.15.143:0/159720660 shutdown_connections 2023-11-08T19:26:38.925 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:38.921+0000 7f65980dc700 1 -- 172.21.15.143:0/159720660 wait complete. 2023-11-08T19:26:38.935 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T19:24:44.551530+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'snap-test-pool', 'create_time': '2023-11-08T19:26:02.874384+0000', 'flags': 16385, 'flags_names': 'hashpspool,pool_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '36', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'pool', 'snap_seq': 2, 'snap_epoch': 36, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-08T19:26:22.869780+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '49', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2023-11-08T19:26:24.093179+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '49', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-08T19:26:38.936 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T19:26:39.059 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.058+0000 7f0dc790b700 1 Processor -- start 2023-11-08T19:26:39.060 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.058+0000 7f0dc790b700 1 -- start start 2023-11-08T19:26:39.062 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.059+0000 7f0dc790b700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f0dc015dd60 0x7f0dc0071b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:39.063 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.059+0000 7f0dc790b700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f0dc00721e0 0x7f0dc0161840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:39.063 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.059+0000 7f0dc790b700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0dc006ac10 con 0x7f0dc006d500 2023-11-08T19:26:39.063 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.059+0000 7f0dc790b700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0dc015e140 con 0x7f0dc015dd60 2023-11-08T19:26:39.063 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.059+0000 7f0dc790b700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0dc006ae60 con 0x7f0dc00721e0 2023-11-08T19:26:39.063 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.059+0000 7f0dc56a7700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f0dc015dd60 0x7f0dc0071b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:39.064 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.059+0000 7f0dc56a7700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f0dc015dd60 0x7f0dc0071b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51916/0 (socket says 172.21.15.143:51916) 2023-11-08T19:26:39.064 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.059+0000 7f0dc56a7700 1 -- 172.21.15.143:0/3585554523 learned_addr learned my addr 172.21.15.143:0/3585554523 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:39.064 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.059+0000 7f0dc56a7700 1 -- 172.21.15.143:0/3585554523 >> v2:172.21.15.174:3300/0 conn(0x7f0dc00721e0 msgr2=0x7f0dc0161840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:39.065 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.059+0000 7f0dc4ea6700 1 --2- 172.21.15.143:0/3585554523 >> v2:172.21.15.174:3300/0 conn(0x7f0dc00721e0 0x7f0dc0161840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:39.065 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.059+0000 7f0dc56a7700 1 --2- 172.21.15.143:0/3585554523 >> v2:172.21.15.174:3300/0 conn(0x7f0dc00721e0 0x7f0dc0161840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:39.065 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.059+0000 7f0dc56a7700 1 -- 172.21.15.143:0/3585554523 >> v1:172.21.15.174:6789/0 conn(0x7f0dc006d500 legacy=0x7f0dc00ceef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:39.065 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.059+0000 7f0dc56a7700 1 -- 172.21.15.143:0/3585554523 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dc006b140 con 0x7f0dc015dd60 2023-11-08T19:26:39.066 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.059+0000 7f0dc56a7700 1 --2- 172.21.15.143:0/3585554523 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0dc015dd60 0x7f0dc0071b90 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f0dc00b2840 tx=0x7f0dbc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4453d50ea6434e60 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:39.066 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.059+0000 7f0db7fff700 1 -- 172.21.15.143:0/3585554523 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dbc004230 con 0x7f0dc015dd60 2023-11-08T19:26:39.066 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.059+0000 7f0db7fff700 1 -- 172.21.15.143:0/3585554523 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dbc004390 con 0x7f0dc015dd60 2023-11-08T19:26:39.067 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.060+0000 7f0db7fff700 1 -- 172.21.15.143:0/3585554523 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dbc0036a0 con 0x7f0dc015dd60 2023-11-08T19:26:39.067 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.060+0000 7f0dc790b700 1 -- 172.21.15.143:0/3585554523 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0dc015dd60 msgr2=0x7f0dc0071b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:39.067 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.060+0000 7f0dc790b700 1 --2- 172.21.15.143:0/3585554523 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0dc015dd60 0x7f0dc0071b90 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f0dc00b2840 tx=0x7f0dbc009bd0 comp rx=0 tx=0).stop 2023-11-08T19:26:39.067 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.060+0000 7f0dc790b700 1 -- 172.21.15.143:0/3585554523 shutdown_connections 2023-11-08T19:26:39.067 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.060+0000 7f0dc790b700 1 --2- 172.21.15.143:0/3585554523 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0dc015dd60 0x7f0dc0071b90 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:39.068 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.060+0000 7f0dc790b700 1 --2- 172.21.15.143:0/3585554523 >> v2:172.21.15.174:3300/0 conn(0x7f0dc00721e0 0x7f0dc0161840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:39.068 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.060+0000 7f0dc790b700 1 -- 172.21.15.143:0/3585554523 >> 172.21.15.143:0/3585554523 conn(0x7f0dc00b1dd0 msgr2=0x7f0dc00cfb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:39.068 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.060+0000 7f0dc790b700 1 -- 172.21.15.143:0/3585554523 shutdown_connections 2023-11-08T19:26:39.068 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.060+0000 7f0dc790b700 1 -- 172.21.15.143:0/3585554523 wait complete. 2023-11-08T19:26:39.069 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.061+0000 7f0dc790b700 1 Processor -- start 2023-11-08T19:26:39.069 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.061+0000 7f0dc790b700 1 -- start start 2023-11-08T19:26:39.072 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.062+0000 7f0dc790b700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0dc006d500 0x7f0dc0147f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:39.073 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.062+0000 7f0dc790b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0dc00721e0 0x7f0dc00b4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:39.073 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.062+0000 7f0dc790b700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0dc015dd60 0x7f0dc015a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:39.073 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.062+0000 7f0dc790b700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dc0149930 con 0x7f0dc015dd60 2023-11-08T19:26:39.073 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.062+0000 7f0dc790b700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dc0148cb0 con 0x7f0dc00721e0 2023-11-08T19:26:39.073 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.062+0000 7f0dc790b700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dc0148f90 con 0x7f0dc006d500 2023-11-08T19:26:39.074 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.062+0000 7f0dc5ea8700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0dc015dd60 0x7f0dc015a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:39.074 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.062+0000 7f0dc5ea8700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0dc015dd60 0x7f0dc015a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51926/0 (socket says 172.21.15.143:51926) 2023-11-08T19:26:39.074 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.062+0000 7f0dc5ea8700 1 -- 172.21.15.143:0/189113930 learned_addr learned my addr 172.21.15.143:0/189113930 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:39.074 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.062+0000 7f0dc4ea6700 1 --2- 172.21.15.143:0/189113930 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0dc00721e0 0x7f0dc00b4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:39.075 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.062+0000 7f0dc56a7700 1 --2- 172.21.15.143:0/189113930 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0dc006d500 0x7f0dc0147f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:39.075 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.062+0000 7f0dc5ea8700 1 -- 172.21.15.143:0/189113930 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0dc006d500 msgr2=0x7f0dc0147f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:39.075 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.062+0000 7f0dc5ea8700 1 --2- 172.21.15.143:0/189113930 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0dc006d500 0x7f0dc0147f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:39.075 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.062+0000 7f0dc5ea8700 1 -- 172.21.15.143:0/189113930 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0dc00721e0 msgr2=0x7f0dc00b4aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:39.076 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.062+0000 7f0dc5ea8700 1 --2- 172.21.15.143:0/189113930 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0dc00721e0 0x7f0dc00b4aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:39.076 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.062+0000 7f0dc5ea8700 1 -- 172.21.15.143:0/189113930 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dbc009550 con 0x7f0dc015dd60 2023-11-08T19:26:39.076 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.062+0000 7f0dc4ea6700 1 --2- 172.21.15.143:0/189113930 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0dc00721e0 0x7f0dc00b4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:26:39.076 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.063+0000 7f0dc5ea8700 1 --2- 172.21.15.143:0/189113930 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0dc015dd60 0x7f0dc015a240 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f0db800b990 tx=0x7f0db800bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:39.076 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.063+0000 7f0db5ffb700 1 -- 172.21.15.143:0/189113930 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0db800c9d0 con 0x7f0dc015dd60 2023-11-08T19:26:39.077 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.063+0000 7f0db5ffb700 1 -- 172.21.15.143:0/189113930 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0db800cb30 con 0x7f0dc015dd60 2023-11-08T19:26:39.077 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.063+0000 7f0dc790b700 1 -- 172.21.15.143:0/189113930 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dc0149270 con 0x7f0dc015dd60 2023-11-08T19:26:39.077 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.063+0000 7f0db5ffb700 1 -- 172.21.15.143:0/189113930 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0db8014440 con 0x7f0dc015dd60 2023-11-08T19:26:39.077 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.063+0000 7f0dc790b700 1 -- 172.21.15.143:0/189113930 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dc0148480 con 0x7f0dc015dd60 2023-11-08T19:26:39.078 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.064+0000 7f0dc790b700 1 -- 172.21.15.143:0/189113930 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0dc0149a70 con 0x7f0dc015dd60 2023-11-08T19:26:39.078 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.064+0000 7f0db5ffb700 1 -- 172.21.15.143:0/189113930 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0db8006d50 con 0x7f0dc015dd60 2023-11-08T19:26:39.078 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.065+0000 7f0db5ffb700 1 --2- 172.21.15.143:0/189113930 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f0da406af00 0x7f0da406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:39.078 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.065+0000 7f0db5ffb700 1 -- 172.21.15.143:0/189113930 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f0db80891e0 con 0x7f0dc015dd60 2023-11-08T19:26:39.078 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.065+0000 7f0dc56a7700 1 --2- 172.21.15.143:0/189113930 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f0da406af00 0x7f0da406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:39.079 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.065+0000 7f0dc56a7700 1 --2- 172.21.15.143:0/189113930 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f0da406af00 0x7f0da406d3c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc000c00 tx=0x7f0dbc008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:39.079 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.069+0000 7f0db5ffb700 1 -- 172.21.15.143:0/189113930 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0dc0149a70 con 0x7f0dc015dd60 2023-11-08T19:26:39.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.255+0000 7f0dc790b700 1 -- 172.21.15.143:0/189113930 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0dc0149a70 con 0x7f0dc015dd60 2023-11-08T19:26:39.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.256+0000 7f0db5ffb700 1 -- 172.21.15.143:0/189113930 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v49) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0dc0149a70 con 0x7f0dc015dd60 2023-11-08T19:26:39.258 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2023-11-08T19:26:39.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.258+0000 7f0dc790b700 1 -- 172.21.15.143:0/189113930 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f0da406af00 msgr2=0x7f0da406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:39.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.258+0000 7f0dc790b700 1 --2- 172.21.15.143:0/189113930 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f0da406af00 0x7f0da406d3c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc000c00 tx=0x7f0dbc008030 comp rx=0 tx=0).stop 2023-11-08T19:26:39.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.258+0000 7f0dc790b700 1 -- 172.21.15.143:0/189113930 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0dc015dd60 msgr2=0x7f0dc015a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:39.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.258+0000 7f0dc790b700 1 --2- 172.21.15.143:0/189113930 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0dc015dd60 0x7f0dc015a240 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f0db800b990 tx=0x7f0db800bd50 comp rx=0 tx=0).stop 2023-11-08T19:26:39.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.258+0000 7f0dc790b700 1 -- 172.21.15.143:0/189113930 shutdown_connections 2023-11-08T19:26:39.261 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.258+0000 7f0dc790b700 1 --2- 172.21.15.143:0/189113930 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f0da406af00 0x7f0da406d3c0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:39.261 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.258+0000 7f0dc790b700 1 --2- 172.21.15.143:0/189113930 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0dc006d500 0x7f0dc0147f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:39.261 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.258+0000 7f0dc790b700 1 --2- 172.21.15.143:0/189113930 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0dc015dd60 0x7f0dc015a240 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:39.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.258+0000 7f0dc790b700 1 --2- 172.21.15.143:0/189113930 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0dc00721e0 0x7f0dc00b4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:39.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.258+0000 7f0dc790b700 1 -- 172.21.15.143:0/189113930 >> 172.21.15.143:0/189113930 conn(0x7f0dc00b1dd0 msgr2=0x7f0dc0160440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:39.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.258+0000 7f0dc790b700 1 -- 172.21.15.143:0/189113930 shutdown_connections 2023-11-08T19:26:39.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.258+0000 7f0dc790b700 1 -- 172.21.15.143:0/189113930 wait complete. 2023-11-08T19:26:39.271 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get snap-test-pool pg_num 2023-11-08T19:26:39.394 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.393+0000 7f8618995700 1 Processor -- start 2023-11-08T19:26:39.394 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.393+0000 7f8618995700 1 -- start start 2023-11-08T19:26:39.397 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.394+0000 7f8618995700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f861415dba0 0x7f861406c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:39.397 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.394+0000 7f8618995700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8614161890 con 0x7f861406b240 2023-11-08T19:26:39.397 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.394+0000 7f8618995700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f861406b740 con 0x7f86140b22e0 2023-11-08T19:26:39.397 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.394+0000 7f8618995700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f861406ba20 con 0x7f861415dba0 2023-11-08T19:26:39.397 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.394+0000 7f8611d9b700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f861406b240 0x7f86140caff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:48086/0 (socket says 172.21.15.143:48086) 2023-11-08T19:26:39.398 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.394+0000 7f861259c700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f861415dba0 0x7f861406c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:39.398 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.394+0000 7f8611d9b700 1 -- 172.21.15.143:0/1354306864 learned_addr learned my addr 172.21.15.143:0/1354306864 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:26:39.398 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.394+0000 7f861159a700 1 -- 172.21.15.143:0/1354306864 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3059868762 0 0) 0x7f8614161890 con 0x7f861406b240 2023-11-08T19:26:39.398 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.394+0000 7f861159a700 1 -- 172.21.15.143:0/1354306864 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f85fc0035b0 con 0x7f861406b240 2023-11-08T19:26:39.398 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.394+0000 7f861259c700 1 -- 172.21.15.143:0/1354306864 >> v1:172.21.15.174:6789/0 conn(0x7f86140b22e0 legacy=0x7f86140b2700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:39.399 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.394+0000 7f861259c700 1 -- 172.21.15.143:0/1354306864 >> v1:172.21.15.143:6789/0 conn(0x7f861406b240 legacy=0x7f86140caff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:39.399 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.394+0000 7f861259c700 1 -- 172.21.15.143:0/1354306864 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f861406bd00 con 0x7f861415dba0 2023-11-08T19:26:39.399 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.394+0000 7f861259c700 1 --2- 172.21.15.143:0/1354306864 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f861415dba0 0x7f861406c610 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f8604009840 tx=0x7f8604009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1096bc436de948ae server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:39.399 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.394+0000 7f861159a700 1 -- 172.21.15.143:0/1354306864 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8604011830 con 0x7f861415dba0 2023-11-08T19:26:39.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.394+0000 7f861159a700 1 -- 172.21.15.143:0/1354306864 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8604011990 con 0x7f861415dba0 2023-11-08T19:26:39.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.394+0000 7f861159a700 1 -- 172.21.15.143:0/1354306864 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8604011d20 con 0x7f861415dba0 2023-11-08T19:26:39.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.395+0000 7f8618995700 1 -- 172.21.15.143:0/1354306864 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f861415dba0 msgr2=0x7f861406c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:39.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.395+0000 7f8618995700 1 --2- 172.21.15.143:0/1354306864 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f861415dba0 0x7f861406c610 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f8604009840 tx=0x7f8604009b50 comp rx=0 tx=0).stop 2023-11-08T19:26:39.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.395+0000 7f8618995700 1 -- 172.21.15.143:0/1354306864 shutdown_connections 2023-11-08T19:26:39.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.395+0000 7f8618995700 1 --2- 172.21.15.143:0/1354306864 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f861415dba0 0x7f861406c610 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:39.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.395+0000 7f8618995700 1 -- 172.21.15.143:0/1354306864 >> 172.21.15.143:0/1354306864 conn(0x7f86140cd2e0 msgr2=0x7f86140cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:39.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.395+0000 7f8618995700 1 -- 172.21.15.143:0/1354306864 shutdown_connections 2023-11-08T19:26:39.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.395+0000 7f8618995700 1 -- 172.21.15.143:0/1354306864 wait complete. 2023-11-08T19:26:39.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.396+0000 7f8618995700 1 Processor -- start 2023-11-08T19:26:39.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.396+0000 7f8618995700 1 -- start start 2023-11-08T19:26:39.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.397+0000 7f8618995700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f861406b240 0x7f861415b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:39.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.397+0000 7f8618995700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86140b22e0 0x7f86140af260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:39.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.397+0000 7f8618995700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f861415dba0 0x7f86140ad8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:39.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.397+0000 7f8618995700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86140af7a0 con 0x7f86140b22e0 2023-11-08T19:26:39.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.397+0000 7f8618995700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86140ae1c0 con 0x7f861406b240 2023-11-08T19:26:39.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.397+0000 7f8618995700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86140ae4a0 con 0x7f861415dba0 2023-11-08T19:26:39.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.397+0000 7f8611d9b700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86140b22e0 0x7f86140af260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:39.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.397+0000 7f8611d9b700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86140b22e0 0x7f86140af260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51938/0 (socket says 172.21.15.143:51938) 2023-11-08T19:26:39.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.397+0000 7f8611d9b700 1 -- 172.21.15.143:0/1600284495 learned_addr learned my addr 172.21.15.143:0/1600284495 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:39.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.398+0000 7f861259c700 1 --2- 172.21.15.143:0/1600284495 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f861406b240 0x7f861415b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:39.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.398+0000 7f8612d9d700 1 --2- 172.21.15.143:0/1600284495 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f861415dba0 0x7f86140ad8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:39.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.398+0000 7f8611d9b700 1 -- 172.21.15.143:0/1600284495 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f861415dba0 msgr2=0x7f86140ad8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:39.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.398+0000 7f8611d9b700 1 --2- 172.21.15.143:0/1600284495 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f861415dba0 0x7f86140ad8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:39.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.398+0000 7f8611d9b700 1 -- 172.21.15.143:0/1600284495 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f861406b240 msgr2=0x7f861415b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:39.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.398+0000 7f8611d9b700 1 --2- 172.21.15.143:0/1600284495 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f861406b240 0x7f861415b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:39.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.398+0000 7f8611d9b700 1 -- 172.21.15.143:0/1600284495 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86040094a0 con 0x7f86140b22e0 2023-11-08T19:26:39.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.398+0000 7f8611d9b700 1 --2- 172.21.15.143:0/1600284495 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86140b22e0 0x7f86140af260 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f860800c700 tx=0x7f860800ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:39.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.398+0000 7f8602ffd700 1 -- 172.21.15.143:0/1600284495 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8608005d70 con 0x7f86140b22e0 2023-11-08T19:26:39.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.398+0000 7f8602ffd700 1 -- 172.21.15.143:0/1600284495 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86080070d0 con 0x7f86140b22e0 2023-11-08T19:26:39.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.398+0000 7f8618995700 1 -- 172.21.15.143:0/1600284495 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86140ae780 con 0x7f86140b22e0 2023-11-08T19:26:39.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.398+0000 7f8602ffd700 1 -- 172.21.15.143:0/1600284495 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8608007460 con 0x7f86140b22e0 2023-11-08T19:26:39.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.398+0000 7f8618995700 1 -- 172.21.15.143:0/1600284495 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86140aed40 con 0x7f86140b22e0 2023-11-08T19:26:39.409 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.399+0000 7f8618995700 1 -- 172.21.15.143:0/1600284495 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8614066c50 con 0x7f86140b22e0 2023-11-08T19:26:39.409 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.400+0000 7f8602ffd700 1 -- 172.21.15.143:0/1600284495 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f86080075c0 con 0x7f86140b22e0 2023-11-08T19:26:39.409 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.400+0000 7f8602ffd700 1 --2- 172.21.15.143:0/1600284495 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f85f406af00 0x7f85f406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:39.409 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.400+0000 7f8602ffd700 1 -- 172.21.15.143:0/1600284495 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f8608088bb0 con 0x7f86140b22e0 2023-11-08T19:26:39.409 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.400+0000 7f861259c700 1 --2- 172.21.15.143:0/1600284495 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f85f406af00 0x7f85f406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:39.410 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.401+0000 7f861259c700 1 --2- 172.21.15.143:0/1600284495 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f85f406af00 0x7f85f406d3c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f86040090b0 tx=0x7f860401c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:39.410 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.404+0000 7f8602ffd700 1 -- 172.21.15.143:0/1600284495 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8608053ac0 con 0x7f86140b22e0 2023-11-08T19:26:39.598 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.596+0000 7f8618995700 1 -- 172.21.15.143:0/1600284495 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"} v 0) v1 -- 0x7f86140aef30 con 0x7f86140b22e0 2023-11-08T19:26:39.598 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.597+0000 7f8602ffd700 1 -- 172.21.15.143:0/1600284495 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"}]=0 v49) v1 ==== 103+0+11 (secure 0 0 0) 0x7f8608017020 con 0x7f86140b22e0 2023-11-08T19:26:39.599 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2023-11-08T19:26:39.600 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.599+0000 7f8618995700 1 -- 172.21.15.143:0/1600284495 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f85f406af00 msgr2=0x7f85f406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:39.601 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.599+0000 7f8618995700 1 --2- 172.21.15.143:0/1600284495 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f85f406af00 0x7f85f406d3c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f86040090b0 tx=0x7f860401c040 comp rx=0 tx=0).stop 2023-11-08T19:26:39.601 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.599+0000 7f8618995700 1 -- 172.21.15.143:0/1600284495 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86140b22e0 msgr2=0x7f86140af260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:39.601 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.599+0000 7f8618995700 1 --2- 172.21.15.143:0/1600284495 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86140b22e0 0x7f86140af260 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f860800c700 tx=0x7f860800ca10 comp rx=0 tx=0).stop 2023-11-08T19:26:39.601 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.599+0000 7f8618995700 1 -- 172.21.15.143:0/1600284495 shutdown_connections 2023-11-08T19:26:39.601 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.599+0000 7f8618995700 1 --2- 172.21.15.143:0/1600284495 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f85f406af00 0x7f85f406d3c0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:39.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.599+0000 7f8618995700 1 --2- 172.21.15.143:0/1600284495 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f861415dba0 0x7f86140ad8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:39.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.599+0000 7f8618995700 1 --2- 172.21.15.143:0/1600284495 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f861406b240 0x7f861415b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:39.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.599+0000 7f8618995700 1 --2- 172.21.15.143:0/1600284495 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86140b22e0 0x7f86140af260 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:39.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.599+0000 7f8618995700 1 -- 172.21.15.143:0/1600284495 >> 172.21.15.143:0/1600284495 conn(0x7f86140cd2e0 msgr2=0x7f861415cd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:39.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.599+0000 7f8618995700 1 -- 172.21.15.143:0/1600284495 shutdown_connections 2023-11-08T19:26:39.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.599+0000 7f8618995700 1 -- 172.21.15.143:0/1600284495 wait complete. 2023-11-08T19:26:39.612 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-08T19:26:39.736 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.734+0000 7f8a3b994700 1 Processor -- start 2023-11-08T19:26:39.736 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.734+0000 7f8a3b994700 1 -- start start 2023-11-08T19:26:39.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.735+0000 7f8a3b994700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f8a3415a950 0x7f8a340d01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:39.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.735+0000 7f8a3b994700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8a3406bd20 0x7f8a34072370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:39.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.735+0000 7f8a3b994700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a340cc2a0 con 0x7f8a340728b0 2023-11-08T19:26:39.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.735+0000 7f8a3b994700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a340cc130 con 0x7f8a3415a950 2023-11-08T19:26:39.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.735+0000 7f8a3b994700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a3406c1b0 con 0x7f8a3406bd20 2023-11-08T19:26:39.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.735+0000 7f8a39f31700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f8a340728b0 0x7f8a340cc730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:48100/0 (socket says 172.21.15.143:48100) 2023-11-08T19:26:39.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.735+0000 7f8a39f31700 1 -- 172.21.15.143:0/940339907 learned_addr learned my addr 172.21.15.143:0/940339907 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:26:39.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.736+0000 7f8a38f2f700 1 --2- 172.21.15.143:0/940339907 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8a3406bd20 0x7f8a34072370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:39.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.736+0000 7f8a39730700 1 --2- 172.21.15.143:0/940339907 >> v2:172.21.15.174:3300/0 conn(0x7f8a3415a950 0x7f8a340d01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:39.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.736+0000 7f8a2bfff700 1 -- 172.21.15.143:0/940339907 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 646616624 0 0) 0x7f8a340cc2a0 con 0x7f8a340728b0 2023-11-08T19:26:39.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.736+0000 7f8a2bfff700 1 -- 172.21.15.143:0/940339907 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8a200035b0 con 0x7f8a340728b0 2023-11-08T19:26:39.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.736+0000 7f8a2bfff700 1 -- 172.21.15.143:0/940339907 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3118460500 0 0) 0x7f8a200035b0 con 0x7f8a340728b0 2023-11-08T19:26:39.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.736+0000 7f8a2bfff700 1 -- 172.21.15.143:0/940339907 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8a3406bd20 msgr2=0x7f8a34072370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:39.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.736+0000 7f8a2bfff700 1 --2- 172.21.15.143:0/940339907 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8a3406bd20 0x7f8a34072370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:39.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.736+0000 7f8a2bfff700 1 -- 172.21.15.143:0/940339907 >> v2:172.21.15.174:3300/0 conn(0x7f8a3415a950 msgr2=0x7f8a340d01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:39.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.736+0000 7f8a2bfff700 1 --2- 172.21.15.143:0/940339907 >> v2:172.21.15.174:3300/0 conn(0x7f8a3415a950 0x7f8a340d01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:39.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.736+0000 7f8a2bfff700 1 -- 172.21.15.143:0/940339907 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a3406c490 con 0x7f8a340728b0 2023-11-08T19:26:39.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.736+0000 7f8a2bfff700 1 -- 172.21.15.143:0/940339907 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4147756303 0 0) 0x7f8a2c0019f0 con 0x7f8a340728b0 2023-11-08T19:26:39.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.736+0000 7f8a2bfff700 1 -- 172.21.15.143:0/940339907 >> v1:172.21.15.143:6789/0 conn(0x7f8a340728b0 legacy=0x7f8a340cc730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:39.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.736+0000 7f8a2bfff700 1 --2- 172.21.15.143:0/940339907 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8a20003a00 0x7f8a20003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:39.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.736+0000 7f8a2bfff700 1 --2- 172.21.15.143:0/940339907 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8a20004850 0x7f8a20004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:39.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.736+0000 7f8a2bfff700 1 --2- 172.21.15.143:0/940339907 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a200051d0 0x7f8a2000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:39.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.736+0000 7f8a2bfff700 1 -- 172.21.15.143:0/940339907 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a3406c1b0 con 0x7f8a200051d0 2023-11-08T19:26:39.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.736+0000 7f8a2bfff700 1 -- 172.21.15.143:0/940339907 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a20004600 con 0x7f8a20003a00 2023-11-08T19:26:39.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.736+0000 7f8a2bfff700 1 -- 172.21.15.143:0/940339907 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a2000bf50 con 0x7f8a20004850 2023-11-08T19:26:39.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.736+0000 7f8a39f31700 1 --2- 172.21.15.143:0/940339907 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a200051d0 0x7f8a2000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:39.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.736+0000 7f8a39730700 1 --2- 172.21.15.143:0/940339907 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8a20003a00 0x7f8a20003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:39.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.736+0000 7f8a38f2f700 1 --2- 172.21.15.143:0/940339907 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8a20004850 0x7f8a20004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:39.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.736+0000 7f8a39f31700 1 -- 172.21.15.143:0/940339907 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8a20004850 msgr2=0x7f8a20004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:39.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.736+0000 7f8a39f31700 1 --2- 172.21.15.143:0/940339907 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8a20004850 0x7f8a20004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:39.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.736+0000 7f8a39f31700 1 -- 172.21.15.143:0/940339907 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8a20003a00 msgr2=0x7f8a20003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:39.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.736+0000 7f8a39f31700 1 --2- 172.21.15.143:0/940339907 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8a20003a00 0x7f8a20003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:39.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.736+0000 7f8a39f31700 1 -- 172.21.15.143:0/940339907 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a200035b0 con 0x7f8a200051d0 2023-11-08T19:26:39.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.736+0000 7f8a39f31700 1 --2- 172.21.15.143:0/940339907 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a200051d0 0x7f8a2000ba10 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c00a890 tx=0x7f8a20005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d705ba4616207ad server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:39.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.737+0000 7f8a2bfff700 1 -- 172.21.15.143:0/940339907 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a2c012600 con 0x7f8a200051d0 2023-11-08T19:26:39.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.737+0000 7f8a2bfff700 1 -- 172.21.15.143:0/940339907 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a2c012780 con 0x7f8a200051d0 2023-11-08T19:26:39.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.737+0000 7f8a2bfff700 1 -- 172.21.15.143:0/940339907 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a2c012b30 con 0x7f8a200051d0 2023-11-08T19:26:39.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.737+0000 7f8a3b994700 1 -- 172.21.15.143:0/940339907 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a200051d0 msgr2=0x7f8a2000ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:39.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.737+0000 7f8a3b994700 1 --2- 172.21.15.143:0/940339907 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a200051d0 0x7f8a2000ba10 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c00a890 tx=0x7f8a20005880 comp rx=0 tx=0).stop 2023-11-08T19:26:39.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.737+0000 7f8a39f31700 1 -- 172.21.15.143:0/940339907 reap_dead start 2023-11-08T19:26:39.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.737+0000 7f8a3b994700 1 -- 172.21.15.143:0/940339907 shutdown_connections 2023-11-08T19:26:39.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.737+0000 7f8a3b994700 1 -- 172.21.15.143:0/940339907 >> 172.21.15.143:0/940339907 conn(0x7f8a340cd2e0 msgr2=0x7f8a340cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:39.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.737+0000 7f8a3b994700 1 -- 172.21.15.143:0/940339907 shutdown_connections 2023-11-08T19:26:39.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.737+0000 7f8a3b994700 1 -- 172.21.15.143:0/940339907 wait complete. 2023-11-08T19:26:39.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.738+0000 7f8a3b994700 1 Processor -- start 2023-11-08T19:26:39.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.739+0000 7f8a3b994700 1 -- start start 2023-11-08T19:26:39.753 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.739+0000 7f8a3b994700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8a3406b560 0x7f8a340c3920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:39.753 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.739+0000 7f8a3b994700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a3406b940 0x7f8a340c3e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:39.753 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.739+0000 7f8a3b994700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8a340c43a0 0x7f8a34145ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:39.753 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.739+0000 7f8a3b994700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a340c9a30 con 0x7f8a3406b940 2023-11-08T19:26:39.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.739+0000 7f8a3b994700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a340c5c90 con 0x7f8a3406b560 2023-11-08T19:26:39.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.739+0000 7f8a3b994700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a340c60c0 con 0x7f8a340c43a0 2023-11-08T19:26:39.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.739+0000 7f8a38f2f700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a3406b940 0x7f8a340c3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:39.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.739+0000 7f8a38f2f700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a3406b940 0x7f8a340c3e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51956/0 (socket says 172.21.15.143:51956) 2023-11-08T19:26:39.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.739+0000 7f8a38f2f700 1 -- 172.21.15.143:0/1242777549 learned_addr learned my addr 172.21.15.143:0/1242777549 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:39.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.739+0000 7f8a39730700 1 --2- 172.21.15.143:0/1242777549 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8a3406b560 0x7f8a340c3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:39.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.739+0000 7f8a39f31700 1 --2- 172.21.15.143:0/1242777549 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8a340c43a0 0x7f8a34145ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:39.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.740+0000 7f8a38f2f700 1 -- 172.21.15.143:0/1242777549 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8a340c43a0 msgr2=0x7f8a34145ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:39.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.740+0000 7f8a38f2f700 1 --2- 172.21.15.143:0/1242777549 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8a340c43a0 0x7f8a34145ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:39.756 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.740+0000 7f8a38f2f700 1 -- 172.21.15.143:0/1242777549 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8a3406b560 msgr2=0x7f8a340c3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:39.756 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.740+0000 7f8a38f2f700 1 --2- 172.21.15.143:0/1242777549 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8a3406b560 0x7f8a340c3920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:39.756 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.740+0000 7f8a38f2f700 1 -- 172.21.15.143:0/1242777549 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a340c59b0 con 0x7f8a3406b940 2023-11-08T19:26:39.756 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.740+0000 7f8a39730700 1 --2- 172.21.15.143:0/1242777549 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8a3406b560 0x7f8a340c3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:26:39.756 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.740+0000 7f8a38f2f700 1 --2- 172.21.15.143:0/1242777549 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a3406b940 0x7f8a340c3e60 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f8a30011aa0 tx=0x7f8a30011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:39.756 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.740+0000 7f8a29ffb700 1 -- 172.21.15.143:0/1242777549 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a3000f980 con 0x7f8a3406b940 2023-11-08T19:26:39.757 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.740+0000 7f8a29ffb700 1 -- 172.21.15.143:0/1242777549 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a3000fae0 con 0x7f8a3406b940 2023-11-08T19:26:39.757 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.740+0000 7f8a3b994700 1 -- 172.21.15.143:0/1242777549 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a341454a0 con 0x7f8a3406b940 2023-11-08T19:26:39.757 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.740+0000 7f8a29ffb700 1 -- 172.21.15.143:0/1242777549 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a3000fe90 con 0x7f8a3406b940 2023-11-08T19:26:39.758 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.740+0000 7f8a3b994700 1 -- 172.21.15.143:0/1242777549 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a340c63c0 con 0x7f8a3406b940 2023-11-08T19:26:39.758 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.741+0000 7f8a3b994700 1 -- 172.21.15.143:0/1242777549 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a340c65e0 con 0x7f8a3406b940 2023-11-08T19:26:39.758 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.741+0000 7f8a29ffb700 1 -- 172.21.15.143:0/1242777549 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8a30014440 con 0x7f8a3406b940 2023-11-08T19:26:39.759 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.742+0000 7f8a29ffb700 1 --2- 172.21.15.143:0/1242777549 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f8a1806af00 0x7f8a1806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:39.759 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.742+0000 7f8a29ffb700 1 -- 172.21.15.143:0/1242777549 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f8a30089f80 con 0x7f8a3406b940 2023-11-08T19:26:39.759 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.742+0000 7f8a39730700 1 --2- 172.21.15.143:0/1242777549 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f8a1806af00 0x7f8a1806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:39.759 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.742+0000 7f8a39730700 1 --2- 172.21.15.143:0/1242777549 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f8a1806af00 0x7f8a1806d3c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f8a24009c60 tx=0x7f8a24008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:39.759 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.746+0000 7f8a29ffb700 1 -- 172.21.15.143:0/1242777549 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8a340c65e0 con 0x7f8a3406b940 2023-11-08T19:26:39.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.940+0000 7f8a3b994700 1 -- 172.21.15.143:0/1242777549 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8a340c65e0 con 0x7f8a3406b940 2023-11-08T19:26:39.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.941+0000 7f8a29ffb700 1 -- 172.21.15.143:0/1242777549 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v50) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8a340c65e0 con 0x7f8a3406b940 2023-11-08T19:26:39.942 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2023-11-08T19:26:39.944 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.943+0000 7f8a3b994700 1 -- 172.21.15.143:0/1242777549 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f8a1806af00 msgr2=0x7f8a1806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:39.944 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.943+0000 7f8a3b994700 1 --2- 172.21.15.143:0/1242777549 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f8a1806af00 0x7f8a1806d3c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f8a24009c60 tx=0x7f8a24008040 comp rx=0 tx=0).stop 2023-11-08T19:26:39.945 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.943+0000 7f8a3b994700 1 -- 172.21.15.143:0/1242777549 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a3406b940 msgr2=0x7f8a340c3e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:39.945 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.943+0000 7f8a3b994700 1 --2- 172.21.15.143:0/1242777549 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a3406b940 0x7f8a340c3e60 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f8a30011aa0 tx=0x7f8a30011e60 comp rx=0 tx=0).stop 2023-11-08T19:26:39.945 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.943+0000 7f8a3b994700 1 -- 172.21.15.143:0/1242777549 shutdown_connections 2023-11-08T19:26:39.945 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.943+0000 7f8a3b994700 1 --2- 172.21.15.143:0/1242777549 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f8a1806af00 0x7f8a1806d3c0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:39.945 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.943+0000 7f8a3b994700 1 --2- 172.21.15.143:0/1242777549 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8a340c43a0 0x7f8a34145ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:39.946 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.943+0000 7f8a3b994700 1 --2- 172.21.15.143:0/1242777549 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8a3406b560 0x7f8a340c3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:39.946 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.943+0000 7f8a3b994700 1 --2- 172.21.15.143:0/1242777549 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a3406b940 0x7f8a340c3e60 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:39.946 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.943+0000 7f8a3b994700 1 -- 172.21.15.143:0/1242777549 >> 172.21.15.143:0/1242777549 conn(0x7f8a340cd2e0 msgr2=0x7f8a340cec80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:39.946 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.943+0000 7f8a3b994700 1 -- 172.21.15.143:0/1242777549 shutdown_connections 2023-11-08T19:26:39.947 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:39.943+0000 7f8a3b994700 1 -- 172.21.15.143:0/1242777549 wait complete. 2023-11-08T19:26:39.956 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-08T19:26:40.080 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.078+0000 7f78e74fb700 1 Processor -- start 2023-11-08T19:26:40.080 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.079+0000 7f78e74fb700 1 -- start start 2023-11-08T19:26:40.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.079+0000 7f78e74fb700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f78e006bd20 0x7f78e00d01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:40.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.079+0000 7f78e74fb700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78e00cc2a0 con 0x7f78e00b61e0 2023-11-08T19:26:40.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.079+0000 7f78e74fb700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78e006c1b0 con 0x7f78e015a950 2023-11-08T19:26:40.085 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.079+0000 7f78e74fb700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78e00cc130 con 0x7f78e006bd20 2023-11-08T19:26:40.085 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.079+0000 7f78e5a98700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f78e00b61e0 0x7f78e00cc730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:48120/0 (socket says 172.21.15.143:48120) 2023-11-08T19:26:40.085 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.080+0000 7f78e5a98700 1 -- 172.21.15.143:0/3637588886 learned_addr learned my addr 172.21.15.143:0/3637588886 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:26:40.085 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.080+0000 7f78d7fff700 1 -- 172.21.15.143:0/3637588886 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4184051348 0 0) 0x7f78e00cc2a0 con 0x7f78e00b61e0 2023-11-08T19:26:40.086 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.080+0000 7f78d7fff700 1 -- 172.21.15.143:0/3637588886 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f78cc0035b0 con 0x7f78e00b61e0 2023-11-08T19:26:40.086 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.080+0000 7f78e4a96700 1 --2- 172.21.15.143:0/3637588886 >> v2:172.21.15.174:3300/0 conn(0x7f78e006bd20 0x7f78e00d01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:40.086 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.080+0000 7f78d7fff700 1 -- 172.21.15.143:0/3637588886 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3403554121 0 0) 0x7f78cc0035b0 con 0x7f78e00b61e0 2023-11-08T19:26:40.086 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.080+0000 7f78d7fff700 1 -- 172.21.15.143:0/3637588886 >> v2:172.21.15.174:3300/0 conn(0x7f78e006bd20 msgr2=0x7f78e00d01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:40.086 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.080+0000 7f78d7fff700 1 --2- 172.21.15.143:0/3637588886 >> v2:172.21.15.174:3300/0 conn(0x7f78e006bd20 0x7f78e00d01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:40.087 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.080+0000 7f78d7fff700 1 -- 172.21.15.143:0/3637588886 >> v1:172.21.15.174:6789/0 conn(0x7f78e015a950 legacy=0x7f78e0072370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:40.087 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.080+0000 7f78d7fff700 1 -- 172.21.15.143:0/3637588886 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78e006c490 con 0x7f78e00b61e0 2023-11-08T19:26:40.087 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.080+0000 7f78d7fff700 1 -- 172.21.15.143:0/3637588886 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4147756303 0 0) 0x7f78d80019f0 con 0x7f78e00b61e0 2023-11-08T19:26:40.087 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.080+0000 7f78e4a96700 1 --2- 172.21.15.143:0/3637588886 >> v2:172.21.15.174:3300/0 conn(0x7f78e006bd20 0x7f78e00d01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:26:40.087 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.080+0000 7f78d7fff700 1 -- 172.21.15.143:0/3637588886 >> v1:172.21.15.143:6789/0 conn(0x7f78e00b61e0 legacy=0x7f78e00cc730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:40.088 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.080+0000 7f78d7fff700 1 --2- 172.21.15.143:0/3637588886 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f78cc003a00 0x7f78cc003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:40.088 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.080+0000 7f78d7fff700 1 --2- 172.21.15.143:0/3637588886 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f78cc004850 0x7f78cc004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:40.088 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.080+0000 7f78d7fff700 1 --2- 172.21.15.143:0/3637588886 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f78cc005210 0x7f78cc00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:40.088 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.080+0000 7f78d7fff700 1 -- 172.21.15.143:0/3637588886 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78cc00c130 con 0x7f78cc004850 2023-11-08T19:26:40.088 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.080+0000 7f78d7fff700 1 -- 172.21.15.143:0/3637588886 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78cc00c2a0 con 0x7f78cc003a00 2023-11-08T19:26:40.089 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.080+0000 7f78d7fff700 1 -- 172.21.15.143:0/3637588886 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78cc00c460 con 0x7f78cc005210 2023-11-08T19:26:40.089 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.080+0000 7f78e4a96700 1 --2- 172.21.15.143:0/3637588886 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f78cc004850 0x7f78cc004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:40.089 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.080+0000 7f78e5297700 1 --2- 172.21.15.143:0/3637588886 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f78cc003a00 0x7f78cc003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:40.089 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.080+0000 7f78e5a98700 1 --2- 172.21.15.143:0/3637588886 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f78cc005210 0x7f78cc00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:40.089 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.080+0000 7f78e4a96700 1 -- 172.21.15.143:0/3637588886 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f78cc005210 msgr2=0x7f78cc00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:40.090 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.080+0000 7f78e4a96700 1 --2- 172.21.15.143:0/3637588886 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f78cc005210 0x7f78cc00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:40.090 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.080+0000 7f78e4a96700 1 -- 172.21.15.143:0/3637588886 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f78cc003a00 msgr2=0x7f78cc003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:40.090 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.080+0000 7f78e4a96700 1 --2- 172.21.15.143:0/3637588886 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f78cc003a00 0x7f78cc003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:40.090 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.080+0000 7f78e4a96700 1 -- 172.21.15.143:0/3637588886 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78cc0035b0 con 0x7f78cc004850 2023-11-08T19:26:40.091 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.080+0000 7f78e5297700 1 --2- 172.21.15.143:0/3637588886 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f78cc003a00 0x7f78cc003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:26:40.091 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.080+0000 7f78e5a98700 1 --2- 172.21.15.143:0/3637588886 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f78cc005210 0x7f78cc00ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:26:40.091 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.081+0000 7f78e5a98700 1 -- 172.21.15.143:0/3637588886 reap_dead start 2023-11-08T19:26:40.091 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.081+0000 7f78e4a96700 1 --2- 172.21.15.143:0/3637588886 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f78cc004850 0x7f78cc004cd0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f78dc009950 tx=0x7f78cc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdadd8a8ca21eb9b server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:40.091 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.081+0000 7f78d7fff700 1 -- 172.21.15.143:0/3637588886 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78dc00c070 con 0x7f78cc004850 2023-11-08T19:26:40.092 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.081+0000 7f78d7fff700 1 -- 172.21.15.143:0/3637588886 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78dc013610 con 0x7f78cc004850 2023-11-08T19:26:40.092 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.081+0000 7f78d7fff700 1 -- 172.21.15.143:0/3637588886 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78dc009560 con 0x7f78cc004850 2023-11-08T19:26:40.092 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.081+0000 7f78e74fb700 1 -- 172.21.15.143:0/3637588886 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f78cc004850 msgr2=0x7f78cc004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:40.092 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.081+0000 7f78e74fb700 1 --2- 172.21.15.143:0/3637588886 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f78cc004850 0x7f78cc004cd0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f78dc009950 tx=0x7f78cc005880 comp rx=0 tx=0).stop 2023-11-08T19:26:40.092 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.081+0000 7f78e74fb700 1 -- 172.21.15.143:0/3637588886 shutdown_connections 2023-11-08T19:26:40.093 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.081+0000 7f78e74fb700 1 --2- 172.21.15.143:0/3637588886 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f78cc004850 0x7f78cc004cd0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:40.093 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.081+0000 7f78e74fb700 1 -- 172.21.15.143:0/3637588886 >> 172.21.15.143:0/3637588886 conn(0x7f78e00cd2e0 msgr2=0x7f78e00cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:40.093 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.081+0000 7f78e74fb700 1 -- 172.21.15.143:0/3637588886 shutdown_connections 2023-11-08T19:26:40.093 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.081+0000 7f78e74fb700 1 -- 172.21.15.143:0/3637588886 wait complete. 2023-11-08T19:26:40.093 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.083+0000 7f78e74fb700 1 Processor -- start 2023-11-08T19:26:40.094 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.083+0000 7f78e74fb700 1 -- start start 2023-11-08T19:26:40.096 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.084+0000 7f78e74fb700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f78cc004850 0x7f78e0151640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:40.096 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.084+0000 7f78e74fb700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f78e0147e70 0x7f78e00b5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:40.097 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.084+0000 7f78e74fb700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f78e01589a0 0x7f78e0158e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:40.097 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.084+0000 7f78e74fb700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78e01f3890 con 0x7f78cc004850 2023-11-08T19:26:40.097 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.084+0000 7f78e74fb700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78e0159370 con 0x7f78e0147e70 2023-11-08T19:26:40.098 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.084+0000 7f78e74fb700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78e00c6b70 con 0x7f78e01589a0 2023-11-08T19:26:40.098 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.084+0000 7f78e5297700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f78cc004850 0x7f78e0151640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:40.098 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.084+0000 7f78e5297700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f78cc004850 0x7f78e0151640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51974/0 (socket says 172.21.15.143:51974) 2023-11-08T19:26:40.098 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.084+0000 7f78e5297700 1 -- 172.21.15.143:0/3811763175 learned_addr learned my addr 172.21.15.143:0/3811763175 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:40.098 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.084+0000 7f78e4a96700 1 --2- 172.21.15.143:0/3811763175 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f78e0147e70 0x7f78e00b5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:40.099 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.084+0000 7f78e5a98700 1 --2- 172.21.15.143:0/3811763175 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f78e01589a0 0x7f78e0158e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:40.099 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.084+0000 7f78e5297700 1 -- 172.21.15.143:0/3811763175 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f78e01589a0 msgr2=0x7f78e0158e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:40.099 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.084+0000 7f78e5297700 1 --2- 172.21.15.143:0/3811763175 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f78e01589a0 0x7f78e0158e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:40.099 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.084+0000 7f78e5297700 1 -- 172.21.15.143:0/3811763175 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f78e0147e70 msgr2=0x7f78e00b5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:40.099 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.084+0000 7f78e5297700 1 --2- 172.21.15.143:0/3811763175 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f78e0147e70 0x7f78e00b5030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:40.100 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.084+0000 7f78e5297700 1 -- 172.21.15.143:0/3811763175 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78e0151cb0 con 0x7f78cc004850 2023-11-08T19:26:40.100 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.084+0000 7f78e5297700 1 --2- 172.21.15.143:0/3811763175 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f78cc004850 0x7f78e0151640 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f78d000e550 tx=0x7f78d000e860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:40.100 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.084+0000 7f78d5ffb700 1 -- 172.21.15.143:0/3811763175 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78d00052e0 con 0x7f78cc004850 2023-11-08T19:26:40.100 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.084+0000 7f78d5ffb700 1 -- 172.21.15.143:0/3811763175 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78d0005440 con 0x7f78cc004850 2023-11-08T19:26:40.100 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.085+0000 7f78e74fb700 1 -- 172.21.15.143:0/3811763175 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78e0151f90 con 0x7f78cc004850 2023-11-08T19:26:40.101 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.085+0000 7f78d5ffb700 1 -- 172.21.15.143:0/3811763175 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78d00038c0 con 0x7f78cc004850 2023-11-08T19:26:40.101 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.085+0000 7f78e74fb700 1 -- 172.21.15.143:0/3811763175 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78e0152550 con 0x7f78cc004850 2023-11-08T19:26:40.101 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.086+0000 7f78e74fb700 1 -- 172.21.15.143:0/3811763175 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78e00c8980 con 0x7f78cc004850 2023-11-08T19:26:40.101 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.086+0000 7f78d5ffb700 1 -- 172.21.15.143:0/3811763175 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f78d0003a20 con 0x7f78cc004850 2023-11-08T19:26:40.101 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.086+0000 7f78d5ffb700 1 --2- 172.21.15.143:0/3811763175 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f78c406af00 0x7f78c406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:40.102 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.086+0000 7f78d5ffb700 1 -- 172.21.15.143:0/3811763175 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f78d0088c40 con 0x7f78cc004850 2023-11-08T19:26:40.102 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.087+0000 7f78e4a96700 1 --2- 172.21.15.143:0/3811763175 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f78c406af00 0x7f78c406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:40.102 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.087+0000 7f78e4a96700 1 --2- 172.21.15.143:0/3811763175 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f78c406af00 0x7f78c406d3c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f78dc000c00 tx=0x7f78dc01f040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:40.102 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.091+0000 7f78d5ffb700 1 -- 172.21.15.143:0/3811763175 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f78d0053c90 con 0x7f78cc004850 2023-11-08T19:26:40.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.279+0000 7f78e74fb700 1 -- 172.21.15.143:0/3811763175 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f78e004eee0 con 0x7f78cc004850 2023-11-08T19:26:40.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.279+0000 7f78d5ffb700 1 -- 172.21.15.143:0/3811763175 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v50) v1 ==== 100+0+11 (secure 0 0 0) 0x7f78d0019020 con 0x7f78cc004850 2023-11-08T19:26:40.281 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 64 2023-11-08T19:26:40.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.281+0000 7f78e74fb700 1 -- 172.21.15.143:0/3811763175 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f78c406af00 msgr2=0x7f78c406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:40.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.281+0000 7f78e74fb700 1 --2- 172.21.15.143:0/3811763175 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f78c406af00 0x7f78c406d3c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f78dc000c00 tx=0x7f78dc01f040 comp rx=0 tx=0).stop 2023-11-08T19:26:40.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.281+0000 7f78e74fb700 1 -- 172.21.15.143:0/3811763175 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f78cc004850 msgr2=0x7f78e0151640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:40.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.281+0000 7f78e74fb700 1 --2- 172.21.15.143:0/3811763175 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f78cc004850 0x7f78e0151640 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f78d000e550 tx=0x7f78d000e860 comp rx=0 tx=0).stop 2023-11-08T19:26:40.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.281+0000 7f78e74fb700 1 -- 172.21.15.143:0/3811763175 shutdown_connections 2023-11-08T19:26:40.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.281+0000 7f78e74fb700 1 --2- 172.21.15.143:0/3811763175 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f78c406af00 0x7f78c406d3c0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:40.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.281+0000 7f78e74fb700 1 --2- 172.21.15.143:0/3811763175 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f78e01589a0 0x7f78e0158e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:40.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.281+0000 7f78e74fb700 1 --2- 172.21.15.143:0/3811763175 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f78cc004850 0x7f78e0151640 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:40.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.281+0000 7f78e74fb700 1 --2- 172.21.15.143:0/3811763175 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f78e0147e70 0x7f78e00b5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:40.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.281+0000 7f78e74fb700 1 -- 172.21.15.143:0/3811763175 >> 172.21.15.143:0/3811763175 conn(0x7f78e00cd2e0 msgr2=0x7f78e00cf380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:40.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.281+0000 7f78e74fb700 1 -- 172.21.15.143:0/3811763175 shutdown_connections 2023-11-08T19:26:40.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.281+0000 7f78e74fb700 1 -- 172.21.15.143:0/3811763175 wait complete. 2023-11-08T19:26:40.296 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-08T19:26:40.418 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.417+0000 7f6dfff4c700 1 Processor -- start 2023-11-08T19:26:40.419 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.417+0000 7f6dfff4c700 1 -- start start 2023-11-08T19:26:40.427 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.418+0000 7f6dfff4c700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6df815aa30 0x7f6df80cf510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:40.427 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.418+0000 7f6dfff4c700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f6df80b2210 0x7f6df80b2690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:40.428 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.418+0000 7f6dfff4c700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6df80cf080 con 0x7f6df80cbe20 2023-11-08T19:26:40.428 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.418+0000 7f6dfff4c700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6df80b2bd0 con 0x7f6df80b2210 2023-11-08T19:26:40.428 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.418+0000 7f6dfff4c700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6df80ce470 con 0x7f6df815aa30 2023-11-08T19:26:40.429 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.418+0000 7f6dfd4e7700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f6df80b2210 0x7f6df80b2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:40.429 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.418+0000 7f6dfd4e7700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f6df80b2210 0x7f6df80b2690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51984/0 (socket says 172.21.15.143:51984) 2023-11-08T19:26:40.430 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.418+0000 7f6dfd4e7700 1 -- 172.21.15.143:0/1022139919 learned_addr learned my addr 172.21.15.143:0/1022139919 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:40.430 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.418+0000 7f6dfdce8700 1 --2- 172.21.15.143:0/1022139919 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6df815aa30 0x7f6df80cf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:40.430 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.418+0000 7f6dfd4e7700 1 -- 172.21.15.143:0/1022139919 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6df815aa30 msgr2=0x7f6df80cf510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:40.431 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.418+0000 7f6dfd4e7700 1 --2- 172.21.15.143:0/1022139919 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6df815aa30 0x7f6df80cf510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:40.431 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.418+0000 7f6dfd4e7700 1 -- 172.21.15.143:0/1022139919 >> v1:172.21.15.174:6789/0 conn(0x7f6df80cbe20 legacy=0x7f6df80cc270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:40.431 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.418+0000 7f6dfd4e7700 1 -- 172.21.15.143:0/1022139919 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6df80ce750 con 0x7f6df80b2210 2023-11-08T19:26:40.432 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.419+0000 7f6dfd4e7700 1 --2- 172.21.15.143:0/1022139919 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6df80b2210 0x7f6df80b2690 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f6df40098f0 tx=0x7f6df4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c23655e3f1b0755 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:40.432 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.419+0000 7f6dfcce6700 1 -- 172.21.15.143:0/1022139919 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6df4011830 con 0x7f6df80b2210 2023-11-08T19:26:40.433 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.419+0000 7f6dfcce6700 1 -- 172.21.15.143:0/1022139919 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6df4011990 con 0x7f6df80b2210 2023-11-08T19:26:40.433 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.419+0000 7f6dfcce6700 1 -- 172.21.15.143:0/1022139919 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6df4011d20 con 0x7f6df80b2210 2023-11-08T19:26:40.433 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.419+0000 7f6dfff4c700 1 -- 172.21.15.143:0/1022139919 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6df80b2210 msgr2=0x7f6df80b2690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:40.434 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.419+0000 7f6dfff4c700 1 --2- 172.21.15.143:0/1022139919 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6df80b2210 0x7f6df80b2690 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f6df40098f0 tx=0x7f6df4009c00 comp rx=0 tx=0).stop 2023-11-08T19:26:40.434 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.419+0000 7f6dfff4c700 1 -- 172.21.15.143:0/1022139919 shutdown_connections 2023-11-08T19:26:40.434 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.419+0000 7f6dfff4c700 1 --2- 172.21.15.143:0/1022139919 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6df815aa30 0x7f6df80cf510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:40.435 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.419+0000 7f6dfff4c700 1 --2- 172.21.15.143:0/1022139919 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6df80b2210 0x7f6df80b2690 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:40.435 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.419+0000 7f6dfff4c700 1 -- 172.21.15.143:0/1022139919 >> 172.21.15.143:0/1022139919 conn(0x7f6df8099ab0 msgr2=0x7f6df80d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:40.435 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.419+0000 7f6dfff4c700 1 -- 172.21.15.143:0/1022139919 shutdown_connections 2023-11-08T19:26:40.436 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.420+0000 7f6dfff4c700 1 -- 172.21.15.143:0/1022139919 wait complete. 2023-11-08T19:26:40.436 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.421+0000 7f6dfff4c700 1 Processor -- start 2023-11-08T19:26:40.436 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.421+0000 7f6dfff4c700 1 -- start start 2023-11-08T19:26:40.436 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.421+0000 7f6dfff4c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6df80b2210 0x7f6df8164960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:40.437 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.421+0000 7f6dfff4c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6df80cbe20 0x7f6df8166ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:40.437 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.421+0000 7f6dfff4c700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6df815aa30 0x7f6df8150f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:40.437 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.421+0000 7f6dfff4c700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6df8164540 con 0x7f6df80cbe20 2023-11-08T19:26:40.437 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.421+0000 7f6dfff4c700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6df8166a90 con 0x7f6df80b2210 2023-11-08T19:26:40.437 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.421+0000 7f6dfff4c700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6df81667b0 con 0x7f6df815aa30 2023-11-08T19:26:40.438 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.421+0000 7f6dfd4e7700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6df80cbe20 0x7f6df8166ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:40.438 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.421+0000 7f6dfd4e7700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6df80cbe20 0x7f6df8166ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51988/0 (socket says 172.21.15.143:51988) 2023-11-08T19:26:40.438 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.421+0000 7f6dfd4e7700 1 -- 172.21.15.143:0/2675604978 learned_addr learned my addr 172.21.15.143:0/2675604978 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:40.438 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.422+0000 7f6dfd4e7700 1 -- 172.21.15.143:0/2675604978 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6df815aa30 msgr2=0x7f6df8150f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:40.438 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.422+0000 7f6dfd4e7700 1 --2- 172.21.15.143:0/2675604978 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6df815aa30 0x7f6df8150f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:40.439 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.422+0000 7f6dfd4e7700 1 -- 172.21.15.143:0/2675604978 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6df80b2210 msgr2=0x7f6df8164960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:40.439 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.422+0000 7f6dfd4e7700 1 --2- 172.21.15.143:0/2675604978 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6df80b2210 0x7f6df8164960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:40.439 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.422+0000 7f6dfd4e7700 1 -- 172.21.15.143:0/2675604978 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6df4009550 con 0x7f6df80cbe20 2023-11-08T19:26:40.439 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.422+0000 7f6dfd4e7700 1 --2- 172.21.15.143:0/2675604978 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6df80cbe20 0x7f6df8166ef0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f6df4009c40 tx=0x7f6df4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:40.439 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.422+0000 7f6dee7fc700 1 -- 172.21.15.143:0/2675604978 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6df40036a0 con 0x7f6df80cbe20 2023-11-08T19:26:40.440 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.422+0000 7f6dee7fc700 1 -- 172.21.15.143:0/2675604978 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6df401f070 con 0x7f6df80cbe20 2023-11-08T19:26:40.440 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.422+0000 7f6dfff4c700 1 -- 172.21.15.143:0/2675604978 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6df8163e80 con 0x7f6df80cbe20 2023-11-08T19:26:40.440 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.422+0000 7f6dee7fc700 1 -- 172.21.15.143:0/2675604978 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6df4003a40 con 0x7f6df80cbe20 2023-11-08T19:26:40.440 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.422+0000 7f6dfff4c700 1 -- 172.21.15.143:0/2675604978 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6df8164ea0 con 0x7f6df80cbe20 2023-11-08T19:26:40.440 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.423+0000 7f6dfff4c700 1 -- 172.21.15.143:0/2675604978 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6df8165030 con 0x7f6df80cbe20 2023-11-08T19:26:40.441 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.423+0000 7f6dee7fc700 1 -- 172.21.15.143:0/2675604978 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6df4003ba0 con 0x7f6df80cbe20 2023-11-08T19:26:40.441 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.424+0000 7f6dee7fc700 1 --2- 172.21.15.143:0/2675604978 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f6ddc06af00 0x7f6ddc06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:40.441 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.424+0000 7f6dee7fc700 1 -- 172.21.15.143:0/2675604978 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f6df4021070 con 0x7f6df80cbe20 2023-11-08T19:26:40.441 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.424+0000 7f6dfdce8700 1 --2- 172.21.15.143:0/2675604978 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f6ddc06af00 0x7f6ddc06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:40.441 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.424+0000 7f6dfdce8700 1 --2- 172.21.15.143:0/2675604978 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f6ddc06af00 0x7f6ddc06d3c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f6de8006ac0 tx=0x7f6de8006a50 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:40.442 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.428+0000 7f6dee7fc700 1 -- 172.21.15.143:0/2675604978 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6df40560b0 con 0x7f6df80cbe20 2023-11-08T19:26:40.636 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.634+0000 7f6dfff4c700 1 -- 172.21.15.143:0/2675604978 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f6df8164680 con 0x7f6df80cbe20 2023-11-08T19:26:40.636 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.635+0000 7f6dee7fc700 1 -- 172.21.15.143:0/2675604978 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v50) v1 ==== 80+0+0 (secure 0 0 0) 0x7f6df8164680 con 0x7f6df80cbe20 2023-11-08T19:26:40.638 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.637+0000 7f6dfff4c700 1 -- 172.21.15.143:0/2675604978 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f6ddc06af00 msgr2=0x7f6ddc06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:40.639 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.637+0000 7f6dfff4c700 1 --2- 172.21.15.143:0/2675604978 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f6ddc06af00 0x7f6ddc06d3c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f6de8006ac0 tx=0x7f6de8006a50 comp rx=0 tx=0).stop 2023-11-08T19:26:40.639 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.637+0000 7f6dfff4c700 1 -- 172.21.15.143:0/2675604978 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6df80cbe20 msgr2=0x7f6df8166ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:40.639 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.637+0000 7f6dfff4c700 1 --2- 172.21.15.143:0/2675604978 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6df80cbe20 0x7f6df8166ef0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f6df4009c40 tx=0x7f6df4004750 comp rx=0 tx=0).stop 2023-11-08T19:26:40.639 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.637+0000 7f6dfff4c700 1 -- 172.21.15.143:0/2675604978 shutdown_connections 2023-11-08T19:26:40.640 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.637+0000 7f6dfff4c700 1 --2- 172.21.15.143:0/2675604978 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f6ddc06af00 0x7f6ddc06d3c0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:40.640 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.637+0000 7f6dfff4c700 1 --2- 172.21.15.143:0/2675604978 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6df815aa30 0x7f6df8150f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:40.640 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.637+0000 7f6dfff4c700 1 --2- 172.21.15.143:0/2675604978 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6df80b2210 0x7f6df8164960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:40.640 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.637+0000 7f6dfff4c700 1 --2- 172.21.15.143:0/2675604978 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6df80cbe20 0x7f6df8166ef0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:40.640 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.637+0000 7f6dfff4c700 1 -- 172.21.15.143:0/2675604978 >> 172.21.15.143:0/2675604978 conn(0x7f6df8099ab0 msgr2=0x7f6df80b0280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:40.641 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.637+0000 7f6dfff4c700 1 -- 172.21.15.143:0/2675604978 shutdown_connections 2023-11-08T19:26:40.641 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.637+0000 7f6dfff4c700 1 -- 172.21.15.143:0/2675604978 wait complete. 2023-11-08T19:26:40.641 INFO:teuthology.orchestra.run.smithi143.stderr:listed 0 entries 2023-11-08T19:26:40.650 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-08T19:26:40.650 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:26:40.651 DEBUG:teuthology.orchestra.run.smithi143:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-11-08T19:26:40.700 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi143.front.sepia.ceph.com 2023-11-08T19:26:40.700 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:26:40.701 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-08T19:26:40.718 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2023-11-08T19:26:40.718 DEBUG:teuthology.orchestra.run.smithi174:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:26:40.739 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284224k,nr_inodes=4071056,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=31,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19549 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260544k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-08T19:26:40.740 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-08T19:26:40.740 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:26:40.775 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284264k,nr_inodes=4071066,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=44,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20643 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-08T19:26:40.776 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2023-11-08T19:26:40.776 DEBUG:teuthology.orchestra.run.smithi174:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:26:40.798 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284224k,nr_inodes=4071056,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=31,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19549 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260544k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-08T19:26:40.799 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:26:40.936 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.935+0000 7f5af4d9a700 1 Processor -- start 2023-11-08T19:26:40.937 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.935+0000 7f5af4d9a700 1 -- start start 2023-11-08T19:26:40.939 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.936+0000 7f5af4d9a700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f5af0159870 0x7f5af00b2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:40.940 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.936+0000 7f5af4d9a700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5af00cc410 0x7f5af00cc860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:40.940 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.936+0000 7f5af4d9a700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5af00b2a60 con 0x7f5af006ed60 2023-11-08T19:26:40.940 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.936+0000 7f5af4d9a700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5af006b090 con 0x7f5af0159870 2023-11-08T19:26:40.940 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.936+0000 7f5af4d9a700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5af006bbc0 con 0x7f5af00cc410 2023-11-08T19:26:40.940 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.936+0000 7f5aee59c700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f5af0159870 0x7f5af00b2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:40.941 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.936+0000 7f5aedd9b700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f5af006ed60 0x7f5af006b4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:48154/0 (socket says 172.21.15.143:48154) 2023-11-08T19:26:40.941 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.936+0000 7f5aedd9b700 1 -- 172.21.15.143:0/3538800610 learned_addr learned my addr 172.21.15.143:0/3538800610 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:26:40.941 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.936+0000 7f5aeed9d700 1 --2- 172.21.15.143:0/3538800610 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5af00cc410 0x7f5af00cc860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:40.941 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.936+0000 7f5aee59c700 1 -- 172.21.15.143:0/3538800610 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5af00cc410 msgr2=0x7f5af00cc860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:40.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.936+0000 7f5aee59c700 1 --2- 172.21.15.143:0/3538800610 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5af00cc410 0x7f5af00cc860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:40.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.936+0000 7f5aee59c700 1 -- 172.21.15.143:0/3538800610 >> v1:172.21.15.143:6789/0 conn(0x7f5af006ed60 legacy=0x7f5af006b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:40.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.936+0000 7f5aed59a700 1 -- 172.21.15.143:0/3538800610 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1853118292 0 0) 0x7f5af00b2a60 con 0x7f5af006ed60 2023-11-08T19:26:40.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.936+0000 7f5aee59c700 1 -- 172.21.15.143:0/3538800610 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5af00b2d40 con 0x7f5af0159870 2023-11-08T19:26:40.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.937+0000 7f5aee59c700 1 --2- 172.21.15.143:0/3538800610 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5af0159870 0x7f5af00b2210 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f5ae4009820 tx=0x7f5ae4009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b85a89529e73b3a3 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:40.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.937+0000 7f5aed59a700 1 -- 172.21.15.143:0/3538800610 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae4005e30 con 0x7f5af0159870 2023-11-08T19:26:40.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.937+0000 7f5aed59a700 1 -- 172.21.15.143:0/3538800610 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ae4006030 con 0x7f5af0159870 2023-11-08T19:26:40.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.937+0000 7f5aed59a700 1 -- 172.21.15.143:0/3538800610 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae40063e0 con 0x7f5af0159870 2023-11-08T19:26:40.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.937+0000 7f5af4d9a700 1 -- 172.21.15.143:0/3538800610 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5af0159870 msgr2=0x7f5af00b2210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:40.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.937+0000 7f5af4d9a700 1 --2- 172.21.15.143:0/3538800610 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5af0159870 0x7f5af00b2210 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f5ae4009820 tx=0x7f5ae4009b30 comp rx=0 tx=0).stop 2023-11-08T19:26:40.944 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.937+0000 7f5af4d9a700 1 -- 172.21.15.143:0/3538800610 shutdown_connections 2023-11-08T19:26:40.944 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.937+0000 7f5af4d9a700 1 --2- 172.21.15.143:0/3538800610 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5af00cc410 0x7f5af00cc860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:40.944 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.937+0000 7f5af4d9a700 1 --2- 172.21.15.143:0/3538800610 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5af0159870 0x7f5af00b2210 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:40.944 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.937+0000 7f5af4d9a700 1 -- 172.21.15.143:0/3538800610 >> 172.21.15.143:0/3538800610 conn(0x7f5af0099ab0 msgr2=0x7f5af00d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:40.944 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.937+0000 7f5af4d9a700 1 -- 172.21.15.143:0/3538800610 shutdown_connections 2023-11-08T19:26:40.945 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.937+0000 7f5af4d9a700 1 -- 172.21.15.143:0/3538800610 wait complete. 2023-11-08T19:26:40.947 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.939+0000 7f5af4d9a700 1 Processor -- start 2023-11-08T19:26:40.947 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.939+0000 7f5af4d9a700 1 -- start start 2023-11-08T19:26:40.948 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.939+0000 7f5af4d9a700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5af006ed60 0x7f5af015bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:40.948 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.939+0000 7f5af4d9a700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5af00cc410 0x7f5af00af040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:40.948 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.939+0000 7f5af4d9a700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5af0159870 0x7f5af01f5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:40.948 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.939+0000 7f5af4d9a700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5af00af6e0 con 0x7f5af006ed60 2023-11-08T19:26:40.948 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.939+0000 7f5af4d9a700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5af015ae00 con 0x7f5af00cc410 2023-11-08T19:26:40.949 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.939+0000 7f5af4d9a700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5af015b010 con 0x7f5af0159870 2023-11-08T19:26:40.949 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.939+0000 7f5aee59c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5af006ed60 0x7f5af015bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:40.949 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.940+0000 7f5aee59c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5af006ed60 0x7f5af015bdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52002/0 (socket says 172.21.15.143:52002) 2023-11-08T19:26:40.949 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.940+0000 7f5aee59c700 1 -- 172.21.15.143:0/2939606469 learned_addr learned my addr 172.21.15.143:0/2939606469 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:40.949 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.940+0000 7f5aedd9b700 1 --2- 172.21.15.143:0/2939606469 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5af00cc410 0x7f5af00af040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:40.950 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.940+0000 7f5aeed9d700 1 --2- 172.21.15.143:0/2939606469 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5af0159870 0x7f5af01f5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:40.950 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.940+0000 7f5aee59c700 1 -- 172.21.15.143:0/2939606469 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5af0159870 msgr2=0x7f5af01f5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:40.950 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.940+0000 7f5aee59c700 1 --2- 172.21.15.143:0/2939606469 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5af0159870 0x7f5af01f5390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:40.950 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.940+0000 7f5aee59c700 1 -- 172.21.15.143:0/2939606469 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5af00cc410 msgr2=0x7f5af00af040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:40.951 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.940+0000 7f5aee59c700 1 --2- 172.21.15.143:0/2939606469 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5af00cc410 0x7f5af00af040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:40.951 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.940+0000 7f5aee59c700 1 -- 172.21.15.143:0/2939606469 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ae4009430 con 0x7f5af006ed60 2023-11-08T19:26:40.951 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.940+0000 7f5aee59c700 1 --2- 172.21.15.143:0/2939606469 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5af006ed60 0x7f5af015bdd0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f5ae4013040 tx=0x7f5ae4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:40.951 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.940+0000 7f5adeffd700 1 -- 172.21.15.143:0/2939606469 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae4025430 con 0x7f5af006ed60 2023-11-08T19:26:40.952 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.940+0000 7f5adeffd700 1 -- 172.21.15.143:0/2939606469 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ae4020070 con 0x7f5af006ed60 2023-11-08T19:26:40.952 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.940+0000 7f5adeffd700 1 -- 172.21.15.143:0/2939606469 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae4025810 con 0x7f5af006ed60 2023-11-08T19:26:40.952 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.940+0000 7f5af4d9a700 1 -- 172.21.15.143:0/2939606469 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5af015b2f0 con 0x7f5af006ed60 2023-11-08T19:26:40.952 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.940+0000 7f5af4d9a700 1 -- 172.21.15.143:0/2939606469 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5af015b8b0 con 0x7f5af006ed60 2023-11-08T19:26:40.952 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.941+0000 7f5af4d9a700 1 -- 172.21.15.143:0/2939606469 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5af004eee0 con 0x7f5af006ed60 2023-11-08T19:26:40.953 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.942+0000 7f5adeffd700 1 -- 172.21.15.143:0/2939606469 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5ae4022030 con 0x7f5af006ed60 2023-11-08T19:26:40.953 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.942+0000 7f5adeffd700 1 --2- 172.21.15.143:0/2939606469 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f5ad006af00 0x7f5ad006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:40.953 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.942+0000 7f5adeffd700 1 -- 172.21.15.143:0/2939606469 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f5ae408ad80 con 0x7f5af006ed60 2023-11-08T19:26:40.953 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.942+0000 7f5aedd9b700 1 --2- 172.21.15.143:0/2939606469 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f5ad006af00 0x7f5ad006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:40.953 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.943+0000 7f5aedd9b700 1 --2- 172.21.15.143:0/2939606469 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f5ad006af00 0x7f5ad006d3c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f5af00b63b0 tx=0x7f5ae0007c90 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:40.954 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:40.947+0000 7f5adeffd700 1 -- 172.21.15.143:0/2939606469 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5ae4055d20 con 0x7f5af006ed60 2023-11-08T19:26:41.179 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.178+0000 7f5af4d9a700 1 -- 172.21.15.143:0/2939606469 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5af00ccec0 con 0x7f5af006ed60 2023-11-08T19:26:41.180 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.178+0000 7f5adeffd700 1 -- 172.21.15.143:0/2939606469 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4925 (secure 0 0 0) 0x7f5ae4059340 con 0x7f5af006ed60 2023-11-08T19:26:41.180 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:26:41.180 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4988,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3140991763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":3140991763},{"type":"v1","addr":"172.21.15.143:6835","nonce":3140991763}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":4997,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6836/2388011942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2388011942},{"type":"v1","addr":"172.21.15.174:6836","nonce":2388011942}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5000,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/1339111936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6835","nonce":1339111936},{"type":"v1","addr":"172.21.15.174:6837","nonce":1339111936}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T19:26:25.128617+0000","modified":"2023-11-08T19:26:28.156717+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5242},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5242":{"gid":5242,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.143:6834/3591097261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":3591097261},{"type":"v1","addr":"172.21.15.143:6834","nonce":3591097261}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-08T19:26:41.182 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.180+0000 7f5af4d9a700 1 -- 172.21.15.143:0/2939606469 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f5ad006af00 msgr2=0x7f5ad006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:41.183 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.180+0000 7f5af4d9a700 1 --2- 172.21.15.143:0/2939606469 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f5ad006af00 0x7f5ad006d3c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f5af00b63b0 tx=0x7f5ae0007c90 comp rx=0 tx=0).stop 2023-11-08T19:26:41.183 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.180+0000 7f5af4d9a700 1 -- 172.21.15.143:0/2939606469 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5af006ed60 msgr2=0x7f5af015bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:41.183 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.180+0000 7f5af4d9a700 1 --2- 172.21.15.143:0/2939606469 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5af006ed60 0x7f5af015bdd0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f5ae4013040 tx=0x7f5ae4004750 comp rx=0 tx=0).stop 2023-11-08T19:26:41.183 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.181+0000 7f5af4d9a700 1 -- 172.21.15.143:0/2939606469 shutdown_connections 2023-11-08T19:26:41.183 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.181+0000 7f5af4d9a700 1 --2- 172.21.15.143:0/2939606469 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f5ad006af00 0x7f5ad006d3c0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:41.184 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.181+0000 7f5af4d9a700 1 --2- 172.21.15.143:0/2939606469 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5af0159870 0x7f5af01f5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:41.184 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.181+0000 7f5af4d9a700 1 --2- 172.21.15.143:0/2939606469 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5af006ed60 0x7f5af015bdd0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:41.184 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.181+0000 7f5af4d9a700 1 --2- 172.21.15.143:0/2939606469 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5af00cc410 0x7f5af00af040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:41.184 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.181+0000 7f5af4d9a700 1 -- 172.21.15.143:0/2939606469 >> 172.21.15.143:0/2939606469 conn(0x7f5af0099ab0 msgr2=0x7f5af01579e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:41.184 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.181+0000 7f5af4d9a700 1 -- 172.21.15.143:0/2939606469 shutdown_connections 2023-11-08T19:26:41.185 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.181+0000 7f5af4d9a700 1 -- 172.21.15.143:0/2939606469 wait complete. 2023-11-08T19:26:41.185 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 25 2023-11-08T19:26:41.194 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:26:41.315 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.314+0000 7feef6340700 1 Processor -- start 2023-11-08T19:26:41.316 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.314+0000 7feef6340700 1 -- start start 2023-11-08T19:26:41.318 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.315+0000 7feef6340700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7feef015dd00 0x7feef00cca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:41.318 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.315+0000 7feef6340700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7feef006b320 0x7feef00cb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:41.318 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.315+0000 7feef6340700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feef00ccfb0 con 0x7feef01618f0 2023-11-08T19:26:41.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.315+0000 7feef6340700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feef00cad50 con 0x7feef015dd00 2023-11-08T19:26:41.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.315+0000 7feef6340700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feef0161df0 con 0x7feef006b320 2023-11-08T19:26:41.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.315+0000 7feeeffff700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7feef015dd00 0x7feef00cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:41.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.315+0000 7feeef7fe700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7feef01618f0 0x7feef006ac10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:48164/0 (socket says 172.21.15.143:48164) 2023-11-08T19:26:41.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.315+0000 7feeeffff700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7feef015dd00 0x7feef00cca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52010/0 (socket says 172.21.15.143:52010) 2023-11-08T19:26:41.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.315+0000 7feeef7fe700 1 -- 172.21.15.143:0/3480788509 learned_addr learned my addr 172.21.15.143:0/3480788509 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:26:41.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.315+0000 7feef48dd700 1 --2- 172.21.15.143:0/3480788509 >> v2:172.21.15.174:3300/0 conn(0x7feef006b320 0x7feef00cb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:41.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.315+0000 7feeeffff700 1 -- 172.21.15.143:0/3480788509 >> v2:172.21.15.174:3300/0 conn(0x7feef006b320 msgr2=0x7feef00cb1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:41.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.315+0000 7feeeffff700 1 --2- 172.21.15.143:0/3480788509 >> v2:172.21.15.174:3300/0 conn(0x7feef006b320 0x7feef00cb1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:41.321 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.315+0000 7feeeffff700 1 -- 172.21.15.143:0/3480788509 >> v1:172.21.15.143:6789/0 conn(0x7feef01618f0 legacy=0x7feef006ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:41.321 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.315+0000 7feeeffff700 1 -- 172.21.15.143:0/3480788509 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feef01620d0 con 0x7feef015dd00 2023-11-08T19:26:41.321 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.315+0000 7feeeeffd700 1 -- 172.21.15.143:0/3480788509 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1273200089 0 0) 0x7feef00ccfb0 con 0x7feef01618f0 2023-11-08T19:26:41.321 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.315+0000 7feef48dd700 1 --2- 172.21.15.143:0/3480788509 >> v2:172.21.15.174:3300/0 conn(0x7feef006b320 0x7feef00cb1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:26:41.321 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.315+0000 7feeeffff700 1 --2- 172.21.15.143:0/3480788509 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feef015dd00 0x7feef00cca70 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7feef00ce8d0 tx=0x7feee4009b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6424bdfebfd9d608 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:41.322 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.315+0000 7feeeeffd700 1 -- 172.21.15.143:0/3480788509 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee4005e30 con 0x7feef015dd00 2023-11-08T19:26:41.322 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.316+0000 7feeeeffd700 1 -- 172.21.15.143:0/3480788509 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feee4006030 con 0x7feef015dd00 2023-11-08T19:26:41.322 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.316+0000 7feeeeffd700 1 -- 172.21.15.143:0/3480788509 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee40063c0 con 0x7feef015dd00 2023-11-08T19:26:41.322 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.316+0000 7feef6340700 1 -- 172.21.15.143:0/3480788509 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feef015dd00 msgr2=0x7feef00cca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:41.322 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.316+0000 7feef6340700 1 --2- 172.21.15.143:0/3480788509 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feef015dd00 0x7feef00cca70 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7feef00ce8d0 tx=0x7feee4009b00 comp rx=0 tx=0).stop 2023-11-08T19:26:41.323 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.316+0000 7feef6340700 1 -- 172.21.15.143:0/3480788509 shutdown_connections 2023-11-08T19:26:41.323 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.316+0000 7feef6340700 1 --2- 172.21.15.143:0/3480788509 >> v2:172.21.15.174:3300/0 conn(0x7feef006b320 0x7feef00cb1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:41.323 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.316+0000 7feef6340700 1 --2- 172.21.15.143:0/3480788509 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feef015dd00 0x7feef00cca70 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:41.323 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.316+0000 7feef6340700 1 -- 172.21.15.143:0/3480788509 >> 172.21.15.143:0/3480788509 conn(0x7feef0099ab0 msgr2=0x7feef00d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:41.323 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.316+0000 7feef6340700 1 -- 172.21.15.143:0/3480788509 shutdown_connections 2023-11-08T19:26:41.324 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.316+0000 7feef6340700 1 -- 172.21.15.143:0/3480788509 wait complete. 2023-11-08T19:26:41.326 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.317+0000 7feef6340700 1 Processor -- start 2023-11-08T19:26:41.327 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.318+0000 7feef6340700 1 -- start start 2023-11-08T19:26:41.327 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.318+0000 7feef6340700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7feef006b320 0x7feef00b5b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:41.327 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.318+0000 7feef6340700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feef015dd00 0x7feef01545f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:41.327 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.318+0000 7feef6340700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feef01618f0 0x7feef00c8f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:41.327 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.318+0000 7feef6340700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feef00c67b0 con 0x7feef01618f0 2023-11-08T19:26:41.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.318+0000 7feef6340700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feef0155220 con 0x7feef015dd00 2023-11-08T19:26:41.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.318+0000 7feef6340700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feef00ca4b0 con 0x7feef006b320 2023-11-08T19:26:41.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.318+0000 7feef48dd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feef01618f0 0x7feef00c8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:41.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.318+0000 7feef48dd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feef01618f0 0x7feef00c8f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52020/0 (socket says 172.21.15.143:52020) 2023-11-08T19:26:41.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.318+0000 7feef48dd700 1 -- 172.21.15.143:0/1746533478 learned_addr learned my addr 172.21.15.143:0/1746533478 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:41.329 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.318+0000 7feeeffff700 1 --2- 172.21.15.143:0/1746533478 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7feef006b320 0x7feef00b5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:41.329 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.318+0000 7feeef7fe700 1 --2- 172.21.15.143:0/1746533478 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feef015dd00 0x7feef01545f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:41.329 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.319+0000 7feef48dd700 1 -- 172.21.15.143:0/1746533478 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7feef006b320 msgr2=0x7feef00b5b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:41.329 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.319+0000 7feef48dd700 1 --2- 172.21.15.143:0/1746533478 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7feef006b320 0x7feef00b5b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:41.329 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.319+0000 7feef48dd700 1 -- 172.21.15.143:0/1746533478 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feef015dd00 msgr2=0x7feef01545f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:41.330 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.319+0000 7feef48dd700 1 --2- 172.21.15.143:0/1746533478 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feef015dd00 0x7feef01545f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:41.330 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.319+0000 7feef48dd700 1 -- 172.21.15.143:0/1746533478 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feee4009430 con 0x7feef01618f0 2023-11-08T19:26:41.330 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.319+0000 7feeef7fe700 1 --2- 172.21.15.143:0/1746533478 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feef015dd00 0x7feef01545f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:26:41.330 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.319+0000 7feeeffff700 1 --2- 172.21.15.143:0/1746533478 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7feef006b320 0x7feef00b5b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:26:41.330 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.319+0000 7feef48dd700 1 --2- 172.21.15.143:0/1746533478 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feef01618f0 0x7feef00c8f00 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7feee8007e10 tx=0x7feee800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:41.331 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.319+0000 7feeecff9700 1 -- 172.21.15.143:0/1746533478 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee800de30 con 0x7feef01618f0 2023-11-08T19:26:41.331 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.319+0000 7feeecff9700 1 -- 172.21.15.143:0/1746533478 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feee800f040 con 0x7feef01618f0 2023-11-08T19:26:41.331 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.319+0000 7feeecff9700 1 -- 172.21.15.143:0/1746533478 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee8015640 con 0x7feef01618f0 2023-11-08T19:26:41.331 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.319+0000 7feef6340700 1 -- 172.21.15.143:0/1746533478 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feef00c61f0 con 0x7feef01618f0 2023-11-08T19:26:41.331 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.319+0000 7feef6340700 1 -- 172.21.15.143:0/1746533478 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feef00c7970 con 0x7feef01618f0 2023-11-08T19:26:41.332 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.320+0000 7feef6340700 1 -- 172.21.15.143:0/1746533478 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feef00aff60 con 0x7feef01618f0 2023-11-08T19:26:41.332 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.320+0000 7feeecff9700 1 -- 172.21.15.143:0/1746533478 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7feee801e3b0 con 0x7feef01618f0 2023-11-08T19:26:41.332 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.321+0000 7feeecff9700 1 --2- 172.21.15.143:0/1746533478 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7feedc06af00 0x7feedc06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:41.332 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.321+0000 7feeecff9700 1 -- 172.21.15.143:0/1746533478 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6120+0+0 (secure 0 0 0) 0x7feee8089810 con 0x7feef01618f0 2023-11-08T19:26:41.332 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.321+0000 7feeeffff700 1 --2- 172.21.15.143:0/1746533478 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7feedc06af00 0x7feedc06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:41.333 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.321+0000 7feeeffff700 1 --2- 172.21.15.143:0/1746533478 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7feedc06af00 0x7feedc06d3c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7feee4009b40 tx=0x7feee4008070 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:41.333 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.325+0000 7feeecff9700 1 -- 172.21.15.143:0/1746533478 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7feee80547b0 con 0x7feef01618f0 2023-11-08T19:26:41.516 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.515+0000 7feef6340700 1 -- 172.21.15.143:0/1746533478 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7feef00c77d0 con 0x7feef01618f0 2023-11-08T19:26:41.517 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.516+0000 7feeecff9700 1 -- 172.21.15.143:0/1746533478 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) v1 ==== 74+0+18466 (secure 0 0 0) 0x7feef00c77d0 con 0x7feef01618f0 2023-11-08T19:26:41.518 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:26:41.518 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":50,"fsid":"968483af-77fc-4068-ad7c-5c2a1a4613d7","created":"2023-11-08T19:24:32.826484+0000","modified":"2023-11-08T19:26:39.794453+0000","last_up_change":"2023-11-08T19:24:43.573880+0000","last_in_change":"2023-11-08T19:24:35.314525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:44.551530+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"snap-test-pool","create_time":"2023-11-08T19:26:02.874384+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":36,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-11-08T19:26:22.869780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-11-08T19:26:24.093179+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bae98ec1-d94e-403d-a220-a522a53ea6f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6809","nonce":1487830566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6811","nonce":1487830566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6815","nonce":1487830566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6813","nonce":1487830566}]},"public_addr":"172.21.15.143:6809/1487830566","cluster_addr":"172.21.15.143:6811/1487830566","heartbeat_back_addr":"172.21.15.143:6815/1487830566","heartbeat_front_addr":"172.21.15.143:6813/1487830566","state":["exists","up"]},{"osd":1,"uuid":"5cb6538a-1ccb-4322-8633-09f66af0b7e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6801","nonce":787341397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6803","nonce":787341397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6807","nonce":787341397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6805","nonce":787341397}]},"public_addr":"172.21.15.143:6801/787341397","cluster_addr":"172.21.15.143:6803/787341397","heartbeat_back_addr":"172.21.15.143:6807/787341397","heartbeat_front_addr":"172.21.15.143:6805/787341397","state":["exists","up"]},{"osd":2,"uuid":"cdbf522c-60ef-4d60-9653-8dcb529c101a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6822","nonce":2301325597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6826","nonce":2301325597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6831","nonce":2301325597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6829","nonce":2301325597}]},"public_addr":"172.21.15.143:6822/2301325597","cluster_addr":"172.21.15.143:6826/2301325597","heartbeat_back_addr":"172.21.15.143:6831/2301325597","heartbeat_front_addr":"172.21.15.143:6829/2301325597","state":["exists","up"]},{"osd":3,"uuid":"4e2783cf-ebb9-4dd8-8245-bc1d17ce4325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6817","nonce":2021504428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6819","nonce":2021504428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6827","nonce":2021504428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6823","nonce":2021504428}]},"public_addr":"172.21.15.143:6817/2021504428","cluster_addr":"172.21.15.143:6819/2021504428","heartbeat_back_addr":"172.21.15.143:6827/2021504428","heartbeat_front_addr":"172.21.15.143:6823/2021504428","state":["exists","up"]},{"osd":4,"uuid":"691256fa-9b77-49b9-a346-7ae1adf0ca80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6801","nonce":1993380595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6803","nonce":1993380595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6807","nonce":1993380595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6805","nonce":1993380595}]},"public_addr":"172.21.15.174:6801/1993380595","cluster_addr":"172.21.15.174:6803/1993380595","heartbeat_back_addr":"172.21.15.174:6807/1993380595","heartbeat_front_addr":"172.21.15.174:6805/1993380595","state":["exists","up"]},{"osd":5,"uuid":"756fc352-48c9-46ce-b52e-0541fe6f8525","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6809","nonce":1777923303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6811","nonce":1777923303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6815","nonce":1777923303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6813","nonce":1777923303}]},"public_addr":"172.21.15.174:6809/1777923303","cluster_addr":"172.21.15.174:6811/1777923303","heartbeat_back_addr":"172.21.15.174:6815/1777923303","heartbeat_front_addr":"172.21.15.174:6813/1777923303","state":["exists","up"]},{"osd":6,"uuid":"fab2ac2b-ed97-4cde-9825-b480eabf8b8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6825","nonce":2755130316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6827","nonce":2755130316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6831","nonce":2755130316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6829","nonce":2755130316}]},"public_addr":"172.21.15.174:6825/2755130316","cluster_addr":"172.21.15.174:6827/2755130316","heartbeat_back_addr":"172.21.15.174:6831/2755130316","heartbeat_front_addr":"172.21.15.174:6829/2755130316","state":["exists","up"]},{"osd":7,"uuid":"3de9b797-2ab6-4d41-a863-7e7d638fd703","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6817","nonce":2847141411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6819","nonce":2847141411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6823","nonce":2847141411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6821","nonce":2847141411}]},"public_addr":"172.21.15.174:6817/2847141411","cluster_addr":"172.21.15.174:6819/2847141411","heartbeat_back_addr":"172.21.15.174:6823/2847141411","heartbeat_front_addr":"172.21.15.174:6821/2847141411","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.477076+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.496890+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.539305+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.454673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.538881+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.521294+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.520712+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.540404+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.1a","mappings":[{"from":0,"to":1}]},{"pgid":"8.28","mappings":[{"from":0,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:26:41.520 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.518+0000 7feef6340700 1 -- 172.21.15.143:0/1746533478 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7feedc06af00 msgr2=0x7feedc06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:41.520 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.518+0000 7feef6340700 1 --2- 172.21.15.143:0/1746533478 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7feedc06af00 0x7feedc06d3c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7feee4009b40 tx=0x7feee4008070 comp rx=0 tx=0).stop 2023-11-08T19:26:41.520 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.518+0000 7feef6340700 1 -- 172.21.15.143:0/1746533478 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feef01618f0 msgr2=0x7feef00c8f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:41.520 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.518+0000 7feef6340700 1 --2- 172.21.15.143:0/1746533478 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feef01618f0 0x7feef00c8f00 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7feee8007e10 tx=0x7feee800d3b0 comp rx=0 tx=0).stop 2023-11-08T19:26:41.520 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.518+0000 7feef6340700 1 -- 172.21.15.143:0/1746533478 shutdown_connections 2023-11-08T19:26:41.521 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.518+0000 7feef6340700 1 --2- 172.21.15.143:0/1746533478 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7feedc06af00 0x7feedc06d3c0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:41.521 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.518+0000 7feef6340700 1 --2- 172.21.15.143:0/1746533478 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7feef006b320 0x7feef00b5b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:41.521 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.518+0000 7feef6340700 1 --2- 172.21.15.143:0/1746533478 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feef01618f0 0x7feef00c8f00 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:41.521 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.518+0000 7feef6340700 1 --2- 172.21.15.143:0/1746533478 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feef015dd00 0x7feef01545f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:41.521 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.518+0000 7feef6340700 1 -- 172.21.15.143:0/1746533478 >> 172.21.15.143:0/1746533478 conn(0x7feef0099ab0 msgr2=0x7feef006f330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:41.522 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.518+0000 7feef6340700 1 -- 172.21.15.143:0/1746533478 shutdown_connections 2023-11-08T19:26:41.522 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.518+0000 7feef6340700 1 -- 172.21.15.143:0/1746533478 wait complete. 2023-11-08T19:26:41.532 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T19:24:44.551530+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'snap-test-pool', 'create_time': '2023-11-08T19:26:02.874384+0000', 'flags': 16385, 'flags_names': 'hashpspool,pool_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '36', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'pool', 'snap_seq': 2, 'snap_epoch': 36, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-08T19:26:22.869780+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '49', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2023-11-08T19:26:24.093179+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '49', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-08T19:26:41.533 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T19:26:41.656 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.654+0000 7f77dbe18700 1 Processor -- start 2023-11-08T19:26:41.656 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.654+0000 7f77dbe18700 1 -- start start 2023-11-08T19:26:41.663 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.655+0000 7f77dbe18700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f77d406bd20 0x7f77d40d01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:41.664 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.656+0000 7f77dbe18700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f77d40b61e0 0x7f77d40cc730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:41.664 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.656+0000 7f77dbe18700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77d40cc2a0 con 0x7f77d415a950 2023-11-08T19:26:41.664 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.656+0000 7f77dbe18700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77d40cc130 con 0x7f77d40b61e0 2023-11-08T19:26:41.664 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.656+0000 7f77dbe18700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77d406c1b0 con 0x7f77d406bd20 2023-11-08T19:26:41.665 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.656+0000 7f77d9bb4700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f77d415a950 0x7f77d4072370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:48178/0 (socket says 172.21.15.143:48178) 2023-11-08T19:26:41.665 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.656+0000 7f77d9bb4700 1 -- 172.21.15.143:0/1218383506 learned_addr learned my addr 172.21.15.143:0/1218383506 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:26:41.665 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.656+0000 7f77d8bb2700 1 -- 172.21.15.143:0/1218383506 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4047518454 0 0) 0x7f77d40cc2a0 con 0x7f77d415a950 2023-11-08T19:26:41.665 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.656+0000 7f77da3b5700 1 --2- 172.21.15.143:0/1218383506 >> v2:172.21.15.174:3300/0 conn(0x7f77d40b61e0 0x7f77d40cc730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:41.666 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.656+0000 7f77d93b3700 1 --2- 172.21.15.143:0/1218383506 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f77d406bd20 0x7f77d40d01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:41.666 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.656+0000 7f77d8bb2700 1 -- 172.21.15.143:0/1218383506 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f77c00035b0 con 0x7f77d415a950 2023-11-08T19:26:41.666 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.656+0000 7f77d8bb2700 1 -- 172.21.15.143:0/1218383506 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2478610366 0 0) 0x7f77c00035b0 con 0x7f77d415a950 2023-11-08T19:26:41.666 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.656+0000 7f77d8bb2700 1 -- 172.21.15.143:0/1218383506 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f77d406bd20 msgr2=0x7f77d40d01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:41.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.656+0000 7f77d8bb2700 1 --2- 172.21.15.143:0/1218383506 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f77d406bd20 0x7f77d40d01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:41.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.656+0000 7f77d8bb2700 1 -- 172.21.15.143:0/1218383506 >> v2:172.21.15.174:3300/0 conn(0x7f77d40b61e0 msgr2=0x7f77d40cc730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:41.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.656+0000 7f77d8bb2700 1 --2- 172.21.15.143:0/1218383506 >> v2:172.21.15.174:3300/0 conn(0x7f77d40b61e0 0x7f77d40cc730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:41.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.656+0000 7f77d8bb2700 1 -- 172.21.15.143:0/1218383506 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77d406c490 con 0x7f77d415a950 2023-11-08T19:26:41.668 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.656+0000 7f77d8bb2700 1 -- 172.21.15.143:0/1218383506 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4147756303 0 0) 0x7f77c4001670 con 0x7f77d415a950 2023-11-08T19:26:41.668 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.656+0000 7f77d8bb2700 1 -- 172.21.15.143:0/1218383506 >> v1:172.21.15.143:6789/0 conn(0x7f77d415a950 legacy=0x7f77d4072370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:41.668 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.656+0000 7f77d8bb2700 1 --2- 172.21.15.143:0/1218383506 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f77c0003a00 0x7f77c0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:41.668 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.656+0000 7f77d8bb2700 1 --2- 172.21.15.143:0/1218383506 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f77c0004360 0x7f77c0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:41.668 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.656+0000 7f77d8bb2700 1 --2- 172.21.15.143:0/1218383506 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f77c0004ea0 0x7f77c0005320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:41.669 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.656+0000 7f77d8bb2700 1 -- 172.21.15.143:0/1218383506 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77d406c1b0 con 0x7f77c0004ea0 2023-11-08T19:26:41.669 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.656+0000 7f77d8bb2700 1 -- 172.21.15.143:0/1218383506 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77c000bdb0 con 0x7f77c0004360 2023-11-08T19:26:41.669 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.656+0000 7f77d8bb2700 1 -- 172.21.15.143:0/1218383506 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77c000bf70 con 0x7f77c0003a00 2023-11-08T19:26:41.669 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.656+0000 7f77da3b5700 1 --2- 172.21.15.143:0/1218383506 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f77c0004ea0 0x7f77c0005320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:41.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.656+0000 7f77d9bb4700 1 --2- 172.21.15.143:0/1218383506 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f77c0003a00 0x7f77c0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:41.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.656+0000 7f77d93b3700 1 --2- 172.21.15.143:0/1218383506 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f77c0004360 0x7f77c0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:41.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.657+0000 7f77da3b5700 1 -- 172.21.15.143:0/1218383506 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f77c0003a00 msgr2=0x7f77c0003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:41.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.657+0000 7f77da3b5700 1 --2- 172.21.15.143:0/1218383506 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f77c0003a00 0x7f77c0003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:41.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.657+0000 7f77da3b5700 1 -- 172.21.15.143:0/1218383506 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f77c0004360 msgr2=0x7f77c0004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:41.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.657+0000 7f77da3b5700 1 --2- 172.21.15.143:0/1218383506 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f77c0004360 0x7f77c0004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:41.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.657+0000 7f77da3b5700 1 -- 172.21.15.143:0/1218383506 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77c00035b0 con 0x7f77c0004ea0 2023-11-08T19:26:41.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.657+0000 7f77da3b5700 1 --2- 172.21.15.143:0/1218383506 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f77c0004ea0 0x7f77c0005320 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f77cc009fc0 tx=0x7f77c0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbd51bfc58ef1a59 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:41.672 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.657+0000 7f77d8bb2700 1 -- 172.21.15.143:0/1218383506 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77cc00ec30 con 0x7f77c0004ea0 2023-11-08T19:26:41.672 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.657+0000 7f77d8bb2700 1 -- 172.21.15.143:0/1218383506 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77cc00f040 con 0x7f77c0004ea0 2023-11-08T19:26:41.672 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.657+0000 7f77d8bb2700 1 -- 172.21.15.143:0/1218383506 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77cc014480 con 0x7f77c0004ea0 2023-11-08T19:26:41.672 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.657+0000 7f77dbe18700 1 -- 172.21.15.143:0/1218383506 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f77c0004ea0 msgr2=0x7f77c0005320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:41.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.657+0000 7f77dbe18700 1 --2- 172.21.15.143:0/1218383506 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f77c0004ea0 0x7f77c0005320 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f77cc009fc0 tx=0x7f77c0005880 comp rx=0 tx=0).stop 2023-11-08T19:26:41.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.657+0000 7f77da3b5700 1 -- 172.21.15.143:0/1218383506 reap_dead start 2023-11-08T19:26:41.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.657+0000 7f77dbe18700 1 -- 172.21.15.143:0/1218383506 shutdown_connections 2023-11-08T19:26:41.674 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.657+0000 7f77dbe18700 1 -- 172.21.15.143:0/1218383506 >> 172.21.15.143:0/1218383506 conn(0x7f77d40cd2e0 msgr2=0x7f77d40cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:41.674 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.658+0000 7f77dbe18700 1 -- 172.21.15.143:0/1218383506 shutdown_connections 2023-11-08T19:26:41.674 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.658+0000 7f77dbe18700 1 -- 172.21.15.143:0/1218383506 wait complete. 2023-11-08T19:26:41.674 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.659+0000 7f77dbe18700 1 Processor -- start 2023-11-08T19:26:41.674 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.659+0000 7f77dbe18700 1 -- start start 2023-11-08T19:26:41.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.659+0000 7f77dbe18700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f77d41f19c0 0x7f77d415c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:41.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.659+0000 7f77dbe18700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f77d41f1da0 0x7f77d41f2260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:41.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.659+0000 7f77dbe18700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f77d415c9f0 0x7f77d41f27a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:41.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.660+0000 7f77dbe18700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77d415db80 con 0x7f77d415c9f0 2023-11-08T19:26:41.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.660+0000 7f77dbe18700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77d415cf70 con 0x7f77d41f19c0 2023-11-08T19:26:41.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.660+0000 7f77dbe18700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77d415d250 con 0x7f77d41f1da0 2023-11-08T19:26:41.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.660+0000 7f77da3b5700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f77d415c9f0 0x7f77d41f27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:41.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.660+0000 7f77da3b5700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f77d415c9f0 0x7f77d41f27a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52038/0 (socket says 172.21.15.143:52038) 2023-11-08T19:26:41.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.660+0000 7f77da3b5700 1 -- 172.21.15.143:0/733211552 learned_addr learned my addr 172.21.15.143:0/733211552 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:41.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.660+0000 7f77d9bb4700 1 --2- 172.21.15.143:0/733211552 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f77d41f19c0 0x7f77d415c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:41.677 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.660+0000 7f77d93b3700 1 --2- 172.21.15.143:0/733211552 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f77d41f1da0 0x7f77d41f2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:41.677 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.660+0000 7f77da3b5700 1 -- 172.21.15.143:0/733211552 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f77d41f1da0 msgr2=0x7f77d41f2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:41.677 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.660+0000 7f77da3b5700 1 --2- 172.21.15.143:0/733211552 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f77d41f1da0 0x7f77d41f2260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:41.677 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.660+0000 7f77da3b5700 1 -- 172.21.15.143:0/733211552 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f77d41f19c0 msgr2=0x7f77d415c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:41.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.660+0000 7f77da3b5700 1 --2- 172.21.15.143:0/733211552 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f77d41f19c0 0x7f77d415c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:41.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.660+0000 7f77da3b5700 1 -- 172.21.15.143:0/733211552 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77d415d530 con 0x7f77d415c9f0 2023-11-08T19:26:41.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.660+0000 7f77d9bb4700 1 --2- 172.21.15.143:0/733211552 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f77d41f19c0 0x7f77d415c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:26:41.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.660+0000 7f77da3b5700 1 --2- 172.21.15.143:0/733211552 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f77d415c9f0 0x7f77d41f27a0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f77cc016000 tx=0x7f77cc009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:41.679 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.660+0000 7f77ca7fc700 1 -- 172.21.15.143:0/733211552 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77cc007000 con 0x7f77d415c9f0 2023-11-08T19:26:41.679 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.660+0000 7f77ca7fc700 1 -- 172.21.15.143:0/733211552 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77cc015b30 con 0x7f77d415c9f0 2023-11-08T19:26:41.679 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.660+0000 7f77ca7fc700 1 -- 172.21.15.143:0/733211552 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77cc01d420 con 0x7f77d415c9f0 2023-11-08T19:26:41.679 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.660+0000 7f77dbe18700 1 -- 172.21.15.143:0/733211552 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77d415d810 con 0x7f77d415c9f0 2023-11-08T19:26:41.680 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.660+0000 7f77dbe18700 1 -- 172.21.15.143:0/733211552 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77d41eec40 con 0x7f77d415c9f0 2023-11-08T19:26:41.680 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.661+0000 7f77dbe18700 1 -- 172.21.15.143:0/733211552 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77d415d9d0 con 0x7f77d415c9f0 2023-11-08T19:26:41.680 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.662+0000 7f77ca7fc700 1 -- 172.21.15.143:0/733211552 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f77cc01d580 con 0x7f77d415c9f0 2023-11-08T19:26:41.680 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.662+0000 7f77ca7fc700 1 --2- 172.21.15.143:0/733211552 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f77b806aef0 0x7f77b806d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:41.681 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.662+0000 7f77ca7fc700 1 -- 172.21.15.143:0/733211552 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f77cc08b060 con 0x7f77d415c9f0 2023-11-08T19:26:41.681 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.662+0000 7f77d9bb4700 1 --2- 172.21.15.143:0/733211552 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f77b806aef0 0x7f77b806d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:41.682 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.663+0000 7f77d9bb4700 1 --2- 172.21.15.143:0/733211552 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f77b806aef0 0x7f77b806d3b0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f77c40048d0 tx=0x7f77c4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:41.682 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.666+0000 7f77ca7fc700 1 -- 172.21.15.143:0/733211552 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f77cc056120 con 0x7f77d415c9f0 2023-11-08T19:26:41.861 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.860+0000 7f77dbe18700 1 -- 172.21.15.143:0/733211552 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f77d404eee0 con 0x7f77d415c9f0 2023-11-08T19:26:41.862 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.860+0000 7f77ca7fc700 1 -- 172.21.15.143:0/733211552 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v50) v1 ==== 93+0+10 (secure 0 0 0) 0x7f77cc059740 con 0x7f77d415c9f0 2023-11-08T19:26:41.862 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2023-11-08T19:26:41.864 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.862+0000 7f77dbe18700 1 -- 172.21.15.143:0/733211552 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f77b806aef0 msgr2=0x7f77b806d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:41.864 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.862+0000 7f77dbe18700 1 --2- 172.21.15.143:0/733211552 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f77b806aef0 0x7f77b806d3b0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f77c40048d0 tx=0x7f77c4009040 comp rx=0 tx=0).stop 2023-11-08T19:26:41.864 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.862+0000 7f77dbe18700 1 -- 172.21.15.143:0/733211552 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f77d415c9f0 msgr2=0x7f77d41f27a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:41.865 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.862+0000 7f77dbe18700 1 --2- 172.21.15.143:0/733211552 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f77d415c9f0 0x7f77d41f27a0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f77cc016000 tx=0x7f77cc009750 comp rx=0 tx=0).stop 2023-11-08T19:26:41.865 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.862+0000 7f77dbe18700 1 -- 172.21.15.143:0/733211552 shutdown_connections 2023-11-08T19:26:41.865 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.862+0000 7f77dbe18700 1 --2- 172.21.15.143:0/733211552 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f77b806aef0 0x7f77b806d3b0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:41.865 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.862+0000 7f77dbe18700 1 --2- 172.21.15.143:0/733211552 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f77d415c9f0 0x7f77d41f27a0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:41.865 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.862+0000 7f77dbe18700 1 --2- 172.21.15.143:0/733211552 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f77d41f19c0 0x7f77d415c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:41.865 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.862+0000 7f77dbe18700 1 --2- 172.21.15.143:0/733211552 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f77d41f1da0 0x7f77d41f2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:41.866 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.862+0000 7f77dbe18700 1 -- 172.21.15.143:0/733211552 >> 172.21.15.143:0/733211552 conn(0x7f77d40cd2e0 msgr2=0x7f77d40cc850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:41.866 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.862+0000 7f77dbe18700 1 -- 172.21.15.143:0/733211552 shutdown_connections 2023-11-08T19:26:41.866 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.863+0000 7f77dbe18700 1 -- 172.21.15.143:0/733211552 wait complete. 2023-11-08T19:26:41.876 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get snap-test-pool pg_num 2023-11-08T19:26:41.999 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.998+0000 7f19e342c700 1 Processor -- start 2023-11-08T19:26:42.000 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.998+0000 7f19e342c700 1 -- start start 2023-11-08T19:26:42.002 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.999+0000 7f19e342c700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f19dc0b22e0 0x7f19dc0b2700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:42.002 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.999+0000 7f19e342c700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19dc161890 con 0x7f19dc15dba0 2023-11-08T19:26:42.003 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.999+0000 7f19e342c700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19dc0b2c40 con 0x7f19dc06b240 2023-11-08T19:26:42.003 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.999+0000 7f19e342c700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19dc06cd20 con 0x7f19dc0b22e0 2023-11-08T19:26:42.003 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.999+0000 7f19e11c8700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f19dc15dba0 0x7f19dc06c610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:48196/0 (socket says 172.21.15.143:48196) 2023-11-08T19:26:42.003 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.999+0000 7f19e11c8700 1 -- 172.21.15.143:0/1446917831 learned_addr learned my addr 172.21.15.143:0/1446917831 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:26:42.004 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.999+0000 7f19e19c9700 1 --2- 172.21.15.143:0/1446917831 >> v2:172.21.15.143:3300/0 conn(0x7f19dc0b22e0 0x7f19dc0b2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:42.004 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:41.999+0000 7f19d3fff700 1 -- 172.21.15.143:0/1446917831 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 364549870 0 0) 0x7f19dc161890 con 0x7f19dc15dba0 2023-11-08T19:26:42.004 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.000+0000 7f19d3fff700 1 -- 172.21.15.143:0/1446917831 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f19c80035b0 con 0x7f19dc15dba0 2023-11-08T19:26:42.004 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.000+0000 7f19e19c9700 1 -- 172.21.15.143:0/1446917831 >> v1:172.21.15.174:6789/0 conn(0x7f19dc06b240 legacy=0x7f19dc0caff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:42.004 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.000+0000 7f19e19c9700 1 -- 172.21.15.143:0/1446917831 >> v1:172.21.15.143:6789/0 conn(0x7f19dc15dba0 legacy=0x7f19dc06c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:42.005 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.000+0000 7f19e19c9700 1 -- 172.21.15.143:0/1446917831 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19dc06b740 con 0x7f19dc0b22e0 2023-11-08T19:26:42.005 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.000+0000 7f19e19c9700 1 --2- 172.21.15.143:0/1446917831 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f19dc0b22e0 0x7f19dc0b2700 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f19d4009d90 tx=0x7f19d400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1df782ded18451da server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:42.005 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.000+0000 7f19d3fff700 1 -- 172.21.15.143:0/1446917831 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19d400e070 con 0x7f19dc0b22e0 2023-11-08T19:26:42.005 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.000+0000 7f19d3fff700 1 -- 172.21.15.143:0/1446917831 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19d400bb20 con 0x7f19dc0b22e0 2023-11-08T19:26:42.006 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.000+0000 7f19d3fff700 1 -- 172.21.15.143:0/1446917831 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19d4012430 con 0x7f19dc0b22e0 2023-11-08T19:26:42.006 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.000+0000 7f19e342c700 1 -- 172.21.15.143:0/1446917831 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f19dc0b22e0 msgr2=0x7f19dc0b2700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:42.006 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.000+0000 7f19e342c700 1 --2- 172.21.15.143:0/1446917831 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f19dc0b22e0 0x7f19dc0b2700 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f19d4009d90 tx=0x7f19d400b330 comp rx=0 tx=0).stop 2023-11-08T19:26:42.006 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.000+0000 7f19e342c700 1 -- 172.21.15.143:0/1446917831 shutdown_connections 2023-11-08T19:26:42.006 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.000+0000 7f19e342c700 1 --2- 172.21.15.143:0/1446917831 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f19dc0b22e0 0x7f19dc0b2700 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:42.007 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.000+0000 7f19e342c700 1 -- 172.21.15.143:0/1446917831 >> 172.21.15.143:0/1446917831 conn(0x7f19dc0cd2e0 msgr2=0x7f19dc0cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:42.007 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.001+0000 7f19e342c700 1 -- 172.21.15.143:0/1446917831 shutdown_connections 2023-11-08T19:26:42.007 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.001+0000 7f19e342c700 1 -- 172.21.15.143:0/1446917831 wait complete. 2023-11-08T19:26:42.010 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.002+0000 7f19e342c700 1 Processor -- start 2023-11-08T19:26:42.010 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.002+0000 7f19e342c700 1 -- start start 2023-11-08T19:26:42.010 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.002+0000 7f19e342c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f19dc06b240 0x7f19dc1e7a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:42.010 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.002+0000 7f19e342c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f19dc0b22e0 0x7f19dc0c4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:42.011 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.002+0000 7f19e342c700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f19dc15dba0 0x7f19dc0b3480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:42.011 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.003+0000 7f19e342c700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19dc0add60 con 0x7f19dc0b22e0 2023-11-08T19:26:42.011 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.003+0000 7f19e342c700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19dc0c48d0 con 0x7f19dc06b240 2023-11-08T19:26:42.011 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.003+0000 7f19e342c700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19dc0ada30 con 0x7f19dc15dba0 2023-11-08T19:26:42.011 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.003+0000 7f19e09c7700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f19dc0b22e0 0x7f19dc0c4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:42.011 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.003+0000 7f19e09c7700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f19dc0b22e0 0x7f19dc0c4390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52048/0 (socket says 172.21.15.143:52048) 2023-11-08T19:26:42.012 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.003+0000 7f19e09c7700 1 -- 172.21.15.143:0/3096052276 learned_addr learned my addr 172.21.15.143:0/3096052276 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:42.012 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.003+0000 7f19e09c7700 1 -- 172.21.15.143:0/3096052276 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f19dc15dba0 msgr2=0x7f19dc0b3480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:42.012 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.003+0000 7f19e09c7700 1 --2- 172.21.15.143:0/3096052276 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f19dc15dba0 0x7f19dc0b3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:42.012 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.003+0000 7f19e09c7700 1 -- 172.21.15.143:0/3096052276 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f19dc06b240 msgr2=0x7f19dc1e7a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:42.013 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.003+0000 7f19e09c7700 1 --2- 172.21.15.143:0/3096052276 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f19dc06b240 0x7f19dc1e7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:42.013 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.003+0000 7f19e09c7700 1 -- 172.21.15.143:0/3096052276 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19d4009a70 con 0x7f19dc0b22e0 2023-11-08T19:26:42.013 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.003+0000 7f19e09c7700 1 --2- 172.21.15.143:0/3096052276 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f19dc0b22e0 0x7f19dc0c4390 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f19d800c4e0 tx=0x7f19d800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:42.013 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.003+0000 7f19d1ffb700 1 -- 172.21.15.143:0/3096052276 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19d80052d0 con 0x7f19dc0b22e0 2023-11-08T19:26:42.013 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.003+0000 7f19d1ffb700 1 -- 172.21.15.143:0/3096052276 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19d8005430 con 0x7f19dc0b22e0 2023-11-08T19:26:42.013 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.003+0000 7f19e342c700 1 -- 172.21.15.143:0/3096052276 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19dc0c4b30 con 0x7f19dc0b22e0 2023-11-08T19:26:42.014 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.003+0000 7f19d1ffb700 1 -- 172.21.15.143:0/3096052276 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19d80038c0 con 0x7f19dc0b22e0 2023-11-08T19:26:42.014 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.003+0000 7f19e342c700 1 -- 172.21.15.143:0/3096052276 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19dc0b39c0 con 0x7f19dc0b22e0 2023-11-08T19:26:42.014 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.004+0000 7f19e342c700 1 -- 172.21.15.143:0/3096052276 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19dc04eee0 con 0x7f19dc0b22e0 2023-11-08T19:26:42.014 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.005+0000 7f19d1ffb700 1 -- 172.21.15.143:0/3096052276 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f19d8003a20 con 0x7f19dc0b22e0 2023-11-08T19:26:42.017 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.005+0000 7f19d1ffb700 1 --2- 172.21.15.143:0/3096052276 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f19c006af00 0x7f19c006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:42.017 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.005+0000 7f19d1ffb700 1 -- 172.21.15.143:0/3096052276 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f19d8088c00 con 0x7f19dc0b22e0 2023-11-08T19:26:42.018 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.005+0000 7f19e11c8700 1 --2- 172.21.15.143:0/3096052276 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f19c006af00 0x7f19c006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:42.018 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.006+0000 7f19e11c8700 1 --2- 172.21.15.143:0/3096052276 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f19c006af00 0x7f19c006d3c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f19cc001610 tx=0x7f19cc0079c0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:42.019 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.009+0000 7f19d1ffb700 1 -- 172.21.15.143:0/3096052276 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f19d808f080 con 0x7f19dc0b22e0 2023-11-08T19:26:42.198 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.196+0000 7f19e342c700 1 -- 172.21.15.143:0/3096052276 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"} v 0) v1 -- 0x7f19dc0adea0 con 0x7f19dc0b22e0 2023-11-08T19:26:42.198 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.197+0000 7f19d1ffb700 1 -- 172.21.15.143:0/3096052276 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"}]=0 v50) v1 ==== 103+0+11 (secure 0 0 0) 0x7f19dc0adea0 con 0x7f19dc0b22e0 2023-11-08T19:26:42.199 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2023-11-08T19:26:42.201 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.199+0000 7f19e342c700 1 -- 172.21.15.143:0/3096052276 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f19c006af00 msgr2=0x7f19c006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:42.201 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.199+0000 7f19e342c700 1 --2- 172.21.15.143:0/3096052276 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f19c006af00 0x7f19c006d3c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f19cc001610 tx=0x7f19cc0079c0 comp rx=0 tx=0).stop 2023-11-08T19:26:42.201 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.199+0000 7f19e342c700 1 -- 172.21.15.143:0/3096052276 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f19dc0b22e0 msgr2=0x7f19dc0c4390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:42.201 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.199+0000 7f19e342c700 1 --2- 172.21.15.143:0/3096052276 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f19dc0b22e0 0x7f19dc0c4390 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f19d800c4e0 tx=0x7f19d800c8a0 comp rx=0 tx=0).stop 2023-11-08T19:26:42.202 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.199+0000 7f19e342c700 1 -- 172.21.15.143:0/3096052276 shutdown_connections 2023-11-08T19:26:42.202 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.199+0000 7f19e342c700 1 --2- 172.21.15.143:0/3096052276 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f19c006af00 0x7f19c006d3c0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:42.202 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.199+0000 7f19e342c700 1 --2- 172.21.15.143:0/3096052276 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f19dc15dba0 0x7f19dc0b3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:42.202 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.199+0000 7f19e342c700 1 --2- 172.21.15.143:0/3096052276 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f19dc06b240 0x7f19dc1e7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:42.202 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.199+0000 7f19e342c700 1 --2- 172.21.15.143:0/3096052276 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f19dc0b22e0 0x7f19dc0c4390 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:42.203 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.199+0000 7f19e342c700 1 -- 172.21.15.143:0/3096052276 >> 172.21.15.143:0/3096052276 conn(0x7f19dc0cd2e0 msgr2=0x7f19dc15ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:42.203 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.199+0000 7f19e342c700 1 -- 172.21.15.143:0/3096052276 shutdown_connections 2023-11-08T19:26:42.203 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.199+0000 7f19e342c700 1 -- 172.21.15.143:0/3096052276 wait complete. 2023-11-08T19:26:42.212 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-08T19:26:42.336 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.334+0000 7fe8d3ac7700 1 Processor -- start 2023-11-08T19:26:42.336 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.335+0000 7fe8d3ac7700 1 -- start start 2023-11-08T19:26:42.339 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.335+0000 7fe8d3ac7700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe8cc158610 0x7fe8cc15b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:42.339 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.335+0000 7fe8d3ac7700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe8cc072010 0x7fe8cc06aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:42.339 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.335+0000 7fe8d3ac7700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8cc072c40 con 0x7fe8cc0cc350 2023-11-08T19:26:42.339 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.335+0000 7fe8d3ac7700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8cc0cf150 con 0x7fe8cc072010 2023-11-08T19:26:42.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.335+0000 7fe8d3ac7700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8cc15be90 con 0x7fe8cc158610 2023-11-08T19:26:42.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.336+0000 7fe8d2064700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe8cc072010 0x7fe8cc06aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:42.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.336+0000 7fe8d1062700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fe8cc0cc350 0x7fe8cc071900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:48210/0 (socket says 172.21.15.143:48210) 2023-11-08T19:26:42.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.336+0000 7fe8d1062700 1 -- 172.21.15.143:0/1145825004 learned_addr learned my addr 172.21.15.143:0/1145825004 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:26:42.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.336+0000 7fe8d1863700 1 --2- 172.21.15.143:0/1145825004 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe8cc158610 0x7fe8cc15b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:42.341 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.336+0000 7fe8d2064700 1 -- 172.21.15.143:0/1145825004 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe8cc158610 msgr2=0x7fe8cc15b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:42.341 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.336+0000 7fe8d2064700 1 --2- 172.21.15.143:0/1145825004 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe8cc158610 0x7fe8cc15b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:42.341 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.336+0000 7fe8d2064700 1 -- 172.21.15.143:0/1145825004 >> v1:172.21.15.143:6789/0 conn(0x7fe8cc0cc350 legacy=0x7fe8cc071900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:42.341 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.336+0000 7fe8d2064700 1 -- 172.21.15.143:0/1145825004 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8cc0cf360 con 0x7fe8cc072010 2023-11-08T19:26:42.341 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.336+0000 7fe8d0861700 1 -- 172.21.15.143:0/1145825004 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3046469393 0 0) 0x7fe8cc072c40 con 0x7fe8cc0cc350 2023-11-08T19:26:42.342 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.336+0000 7fe8d2064700 1 --2- 172.21.15.143:0/1145825004 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe8cc072010 0x7fe8cc06aff0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fe8c4009d90 tx=0x7fe8c400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9adec0a2c126630 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:42.342 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.336+0000 7fe8d0861700 1 -- 172.21.15.143:0/1145825004 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c400e070 con 0x7fe8cc072010 2023-11-08T19:26:42.343 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.336+0000 7fe8d0861700 1 -- 172.21.15.143:0/1145825004 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8c400bbd0 con 0x7fe8cc072010 2023-11-08T19:26:42.343 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.336+0000 7fe8d0861700 1 -- 172.21.15.143:0/1145825004 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c4012430 con 0x7fe8cc072010 2023-11-08T19:26:42.343 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.337+0000 7fe8d3ac7700 1 -- 172.21.15.143:0/1145825004 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe8cc072010 msgr2=0x7fe8cc06aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:42.343 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.337+0000 7fe8d3ac7700 1 --2- 172.21.15.143:0/1145825004 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe8cc072010 0x7fe8cc06aff0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fe8c4009d90 tx=0x7fe8c400b330 comp rx=0 tx=0).stop 2023-11-08T19:26:42.343 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.337+0000 7fe8d3ac7700 1 -- 172.21.15.143:0/1145825004 shutdown_connections 2023-11-08T19:26:42.344 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.337+0000 7fe8d3ac7700 1 --2- 172.21.15.143:0/1145825004 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe8cc072010 0x7fe8cc06aff0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:42.344 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.337+0000 7fe8d3ac7700 1 --2- 172.21.15.143:0/1145825004 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe8cc158610 0x7fe8cc15b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:42.344 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.337+0000 7fe8d3ac7700 1 -- 172.21.15.143:0/1145825004 >> 172.21.15.143:0/1145825004 conn(0x7fe8cc0b1c70 msgr2=0x7fe8cc0b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:42.344 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.337+0000 7fe8d3ac7700 1 -- 172.21.15.143:0/1145825004 shutdown_connections 2023-11-08T19:26:42.344 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.337+0000 7fe8d3ac7700 1 -- 172.21.15.143:0/1145825004 wait complete. 2023-11-08T19:26:42.347 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.338+0000 7fe8d3ac7700 1 Processor -- start 2023-11-08T19:26:42.348 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.338+0000 7fe8d3ac7700 1 -- start start 2023-11-08T19:26:42.348 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.339+0000 7fe8d3ac7700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe8cc072010 0x7fe8cc0c4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:42.348 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.339+0000 7fe8d3ac7700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe8cc0cc350 0x7fe8cc0c99c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:42.348 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.339+0000 7fe8d3ac7700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe8cc158610 0x7fe8cc0b5410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:42.348 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.339+0000 7fe8d3ac7700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8cc0c54c0 con 0x7fe8cc0cc350 2023-11-08T19:26:42.349 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.339+0000 7fe8d3ac7700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8cc156160 con 0x7fe8cc072010 2023-11-08T19:26:42.349 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.339+0000 7fe8d3ac7700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8cc156440 con 0x7fe8cc158610 2023-11-08T19:26:42.349 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.339+0000 7fe8d1062700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe8cc0cc350 0x7fe8cc0c99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:42.349 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.339+0000 7fe8d1062700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe8cc0cc350 0x7fe8cc0c99c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52062/0 (socket says 172.21.15.143:52062) 2023-11-08T19:26:42.349 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.339+0000 7fe8d1062700 1 -- 172.21.15.143:0/3753953301 learned_addr learned my addr 172.21.15.143:0/3753953301 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:42.350 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.339+0000 7fe8d2064700 1 --2- 172.21.15.143:0/3753953301 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe8cc158610 0x7fe8cc0b5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:42.350 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.339+0000 7fe8d1863700 1 --2- 172.21.15.143:0/3753953301 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe8cc072010 0x7fe8cc0c4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:42.350 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.339+0000 7fe8d1062700 1 -- 172.21.15.143:0/3753953301 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe8cc158610 msgr2=0x7fe8cc0b5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:42.350 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.339+0000 7fe8d1062700 1 --2- 172.21.15.143:0/3753953301 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe8cc158610 0x7fe8cc0b5410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:42.350 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.339+0000 7fe8d1062700 1 -- 172.21.15.143:0/3753953301 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe8cc072010 msgr2=0x7fe8cc0c4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:42.351 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.339+0000 7fe8d1062700 1 --2- 172.21.15.143:0/3753953301 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe8cc072010 0x7fe8cc0c4ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:42.351 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.339+0000 7fe8d1062700 1 -- 172.21.15.143:0/3753953301 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8c4009a70 con 0x7fe8cc0cc350 2023-11-08T19:26:42.351 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.339+0000 7fe8d2064700 1 --2- 172.21.15.143:0/3753953301 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe8cc158610 0x7fe8cc0b5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:26:42.351 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.339+0000 7fe8d1863700 1 --2- 172.21.15.143:0/3753953301 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe8cc072010 0x7fe8cc0c4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:26:42.351 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.339+0000 7fe8d1062700 1 --2- 172.21.15.143:0/3753953301 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe8cc0cc350 0x7fe8cc0c99c0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fe8c800c520 tx=0x7fe8c800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:42.352 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.339+0000 7fe8c27fc700 1 -- 172.21.15.143:0/3753953301 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c8004c80 con 0x7fe8cc0cc350 2023-11-08T19:26:42.352 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.339+0000 7fe8c27fc700 1 -- 172.21.15.143:0/3753953301 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8c8004de0 con 0x7fe8cc0cc350 2023-11-08T19:26:42.352 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.339+0000 7fe8d3ac7700 1 -- 172.21.15.143:0/3753953301 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8cc156720 con 0x7fe8cc0cc350 2023-11-08T19:26:42.352 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.339+0000 7fe8c27fc700 1 -- 172.21.15.143:0/3753953301 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c80072d0 con 0x7fe8cc0cc350 2023-11-08T19:26:42.352 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.339+0000 7fe8d3ac7700 1 -- 172.21.15.143:0/3753953301 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8cc0b5950 con 0x7fe8cc0cc350 2023-11-08T19:26:42.353 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.340+0000 7fe8d3ac7700 1 -- 172.21.15.143:0/3753953301 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8cc155fc0 con 0x7fe8cc0cc350 2023-11-08T19:26:42.353 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.341+0000 7fe8c27fc700 1 -- 172.21.15.143:0/3753953301 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe8c8007430 con 0x7fe8cc0cc350 2023-11-08T19:26:42.353 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.341+0000 7fe8c27fc700 1 --2- 172.21.15.143:0/3753953301 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fe8b006af00 0x7fe8b006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:42.353 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.341+0000 7fe8c27fc700 1 -- 172.21.15.143:0/3753953301 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fe8c8089490 con 0x7fe8cc0cc350 2023-11-08T19:26:42.353 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.341+0000 7fe8d1863700 1 --2- 172.21.15.143:0/3753953301 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fe8b006af00 0x7fe8b006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:42.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.342+0000 7fe8d1863700 1 --2- 172.21.15.143:0/3753953301 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fe8b006af00 0x7fe8b006d3c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc0099f0 tx=0x7fe8bc008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:42.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.346+0000 7fe8c27fc700 1 -- 172.21.15.143:0/3753953301 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe8cc155fc0 con 0x7fe8cc0cc350 2023-11-08T19:26:42.538 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.537+0000 7fe8d3ac7700 1 -- 172.21.15.143:0/3753953301 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe8cc155fc0 con 0x7fe8cc0cc350 2023-11-08T19:26:42.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.537+0000 7fe8c27fc700 1 -- 172.21.15.143:0/3753953301 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v50) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe8cc155fc0 con 0x7fe8cc0cc350 2023-11-08T19:26:42.539 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2023-11-08T19:26:42.541 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.539+0000 7fe8d3ac7700 1 -- 172.21.15.143:0/3753953301 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fe8b006af00 msgr2=0x7fe8b006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:42.541 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.539+0000 7fe8d3ac7700 1 --2- 172.21.15.143:0/3753953301 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fe8b006af00 0x7fe8b006d3c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc0099f0 tx=0x7fe8bc008040 comp rx=0 tx=0).stop 2023-11-08T19:26:42.541 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.539+0000 7fe8d3ac7700 1 -- 172.21.15.143:0/3753953301 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe8cc0cc350 msgr2=0x7fe8cc0c99c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:42.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.539+0000 7fe8d3ac7700 1 --2- 172.21.15.143:0/3753953301 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe8cc0cc350 0x7fe8cc0c99c0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fe8c800c520 tx=0x7fe8c800c8e0 comp rx=0 tx=0).stop 2023-11-08T19:26:42.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.539+0000 7fe8d3ac7700 1 -- 172.21.15.143:0/3753953301 shutdown_connections 2023-11-08T19:26:42.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.539+0000 7fe8d3ac7700 1 --2- 172.21.15.143:0/3753953301 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fe8b006af00 0x7fe8b006d3c0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:42.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.539+0000 7fe8d3ac7700 1 --2- 172.21.15.143:0/3753953301 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe8cc158610 0x7fe8cc0b5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:42.543 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.539+0000 7fe8d3ac7700 1 --2- 172.21.15.143:0/3753953301 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe8cc072010 0x7fe8cc0c4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:42.543 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.539+0000 7fe8d3ac7700 1 --2- 172.21.15.143:0/3753953301 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe8cc0cc350 0x7fe8cc0c99c0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:42.543 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.539+0000 7fe8d3ac7700 1 -- 172.21.15.143:0/3753953301 >> 172.21.15.143:0/3753953301 conn(0x7fe8cc0b1c70 msgr2=0x7fe8cc0afbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:42.543 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.539+0000 7fe8d3ac7700 1 -- 172.21.15.143:0/3753953301 shutdown_connections 2023-11-08T19:26:42.543 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.539+0000 7fe8d3ac7700 1 -- 172.21.15.143:0/3753953301 wait complete. 2023-11-08T19:26:42.553 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-08T19:26:42.677 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.676+0000 7f0865946700 1 Processor -- start 2023-11-08T19:26:42.677 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.676+0000 7f0865946700 1 -- start start 2023-11-08T19:26:42.679 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.676+0000 7f0865946700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0860158140 0x7f086015eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:42.680 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.676+0000 7f0865946700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f086006b620 0x7f08600b1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:42.680 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.676+0000 7f0865946700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f08600b1ff0 0x7f08601616e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:42.680 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.676+0000 7f0865946700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08600b2440 con 0x7f08600b1ff0 2023-11-08T19:26:42.680 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.676+0000 7f0865946700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f086015e200 con 0x7f086006b620 2023-11-08T19:26:42.681 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.676+0000 7f0865946700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08600ce740 con 0x7f0860158140 2023-11-08T19:26:42.681 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.677+0000 7f085f7fe700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f08600b1ff0 0x7f08601616e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:42.681 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.677+0000 7f085f7fe700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f08600b1ff0 0x7f08601616e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52072/0 (socket says 172.21.15.143:52072) 2023-11-08T19:26:42.681 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.677+0000 7f085f7fe700 1 -- 172.21.15.143:0/1690612395 learned_addr learned my addr 172.21.15.143:0/1690612395 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:42.681 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.677+0000 7f085e7fc700 1 --2- 172.21.15.143:0/1690612395 >> v2:172.21.15.174:3300/0 conn(0x7f086006b620 0x7f08600b1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:42.682 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.677+0000 7f085effd700 1 --2- 172.21.15.143:0/1690612395 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0860158140 0x7f086015eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:42.682 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.677+0000 7f085f7fe700 1 -- 172.21.15.143:0/1690612395 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0860158140 msgr2=0x7f086015eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:42.682 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.677+0000 7f085f7fe700 1 --2- 172.21.15.143:0/1690612395 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0860158140 0x7f086015eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:42.682 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.677+0000 7f085f7fe700 1 -- 172.21.15.143:0/1690612395 >> v2:172.21.15.174:3300/0 conn(0x7f086006b620 msgr2=0x7f08600b1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:42.682 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.677+0000 7f085f7fe700 1 --2- 172.21.15.143:0/1690612395 >> v2:172.21.15.174:3300/0 conn(0x7f086006b620 0x7f08600b1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:42.683 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.677+0000 7f085f7fe700 1 -- 172.21.15.143:0/1690612395 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f086015e4a0 con 0x7f08600b1ff0 2023-11-08T19:26:42.683 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.677+0000 7f085f7fe700 1 --2- 172.21.15.143:0/1690612395 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f08600b1ff0 0x7f08601616e0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f0860072a30 tx=0x7f085800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd653a9a0c42cfa0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:42.683 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.677+0000 7f085dffb700 1 -- 172.21.15.143:0/1690612395 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f085800e070 con 0x7f08600b1ff0 2023-11-08T19:26:42.683 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.677+0000 7f085dffb700 1 -- 172.21.15.143:0/1690612395 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f085800b910 con 0x7f08600b1ff0 2023-11-08T19:26:42.683 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.677+0000 7f085dffb700 1 -- 172.21.15.143:0/1690612395 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f085800bcc0 con 0x7f08600b1ff0 2023-11-08T19:26:42.684 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.678+0000 7f0865946700 1 -- 172.21.15.143:0/1690612395 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f08600b1ff0 msgr2=0x7f08601616e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:42.684 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.678+0000 7f0865946700 1 --2- 172.21.15.143:0/1690612395 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f08600b1ff0 0x7f08601616e0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f0860072a30 tx=0x7f085800b330 comp rx=0 tx=0).stop 2023-11-08T19:26:42.684 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.678+0000 7f0865946700 1 -- 172.21.15.143:0/1690612395 shutdown_connections 2023-11-08T19:26:42.684 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.678+0000 7f0865946700 1 --2- 172.21.15.143:0/1690612395 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f08600b1ff0 0x7f08601616e0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:42.684 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.678+0000 7f0865946700 1 --2- 172.21.15.143:0/1690612395 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0860158140 0x7f086015eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:42.685 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.678+0000 7f0865946700 1 --2- 172.21.15.143:0/1690612395 >> v2:172.21.15.174:3300/0 conn(0x7f086006b620 0x7f08600b1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:42.685 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.678+0000 7f0865946700 1 -- 172.21.15.143:0/1690612395 >> 172.21.15.143:0/1690612395 conn(0x7f08600d0630 msgr2=0x7f08600cc9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:42.685 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.678+0000 7f0865946700 1 -- 172.21.15.143:0/1690612395 shutdown_connections 2023-11-08T19:26:42.685 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.678+0000 7f0865946700 1 -- 172.21.15.143:0/1690612395 wait complete. 2023-11-08T19:26:42.688 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.679+0000 7f0865946700 1 Processor -- start 2023-11-08T19:26:42.688 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.679+0000 7f0865946700 1 -- start start 2023-11-08T19:26:42.689 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.680+0000 7f0865946700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f086006b620 0x7f08601f2c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:42.689 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.680+0000 7f0865946700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f08600b1ff0 0x7f08600c91a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:42.689 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.680+0000 7f0865946700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0860158140 0x7f08600ac8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:42.689 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.680+0000 7f0865946700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08600c9f40 con 0x7f086006b620 2023-11-08T19:26:42.690 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.680+0000 7f0865946700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08600c5ac0 con 0x7f0860158140 2023-11-08T19:26:42.690 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.680+0000 7f0865946700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08600c5210 con 0x7f08600b1ff0 2023-11-08T19:26:42.690 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.680+0000 7f085effd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f086006b620 0x7f08601f2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:42.690 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.680+0000 7f085effd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f086006b620 0x7f08601f2c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52074/0 (socket says 172.21.15.143:52074) 2023-11-08T19:26:42.690 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.680+0000 7f085effd700 1 -- 172.21.15.143:0/304200784 learned_addr learned my addr 172.21.15.143:0/304200784 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:42.691 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.680+0000 7f085e7fc700 1 --2- 172.21.15.143:0/304200784 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f08600b1ff0 0x7f08600c91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:42.691 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.680+0000 7f085f7fe700 1 --2- 172.21.15.143:0/304200784 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0860158140 0x7f08600ac8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:42.691 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.680+0000 7f085effd700 1 -- 172.21.15.143:0/304200784 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f08600b1ff0 msgr2=0x7f08600c91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:42.691 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.680+0000 7f085effd700 1 --2- 172.21.15.143:0/304200784 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f08600b1ff0 0x7f08600c91a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:42.691 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.680+0000 7f085effd700 1 -- 172.21.15.143:0/304200784 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0860158140 msgr2=0x7f08600ac8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:42.692 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.680+0000 7f085effd700 1 --2- 172.21.15.143:0/304200784 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0860158140 0x7f08600ac8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:42.692 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.680+0000 7f085effd700 1 -- 172.21.15.143:0/304200784 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0858009b30 con 0x7f086006b620 2023-11-08T19:26:42.692 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.680+0000 7f085f7fe700 1 --2- 172.21.15.143:0/304200784 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0860158140 0x7f08600ac8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:26:42.692 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.680+0000 7f085effd700 1 --2- 172.21.15.143:0/304200784 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f086006b620 0x7f08601f2c20 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f085400e870 tx=0x7f085400ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:42.692 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.681+0000 7f0847fff700 1 -- 172.21.15.143:0/304200784 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0854009590 con 0x7f086006b620 2023-11-08T19:26:42.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.681+0000 7f0847fff700 1 -- 172.21.15.143:0/304200784 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f085400ee70 con 0x7f086006b620 2023-11-08T19:26:42.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.681+0000 7f0865946700 1 -- 172.21.15.143:0/304200784 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08600c54b0 con 0x7f086006b620 2023-11-08T19:26:42.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.681+0000 7f0847fff700 1 -- 172.21.15.143:0/304200784 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f085400ca50 con 0x7f086006b620 2023-11-08T19:26:42.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.681+0000 7f0865946700 1 -- 172.21.15.143:0/304200784 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08600c5750 con 0x7f086006b620 2023-11-08T19:26:42.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.682+0000 7f0865946700 1 -- 172.21.15.143:0/304200784 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08600c5da0 con 0x7f086006b620 2023-11-08T19:26:42.694 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.682+0000 7f0847fff700 1 -- 172.21.15.143:0/304200784 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f085400cbb0 con 0x7f086006b620 2023-11-08T19:26:42.694 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.682+0000 7f0847fff700 1 --2- 172.21.15.143:0/304200784 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f084c06af00 0x7f084c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:42.694 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.682+0000 7f0847fff700 1 -- 172.21.15.143:0/304200784 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f08540899d0 con 0x7f086006b620 2023-11-08T19:26:42.694 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.683+0000 7f085e7fc700 1 --2- 172.21.15.143:0/304200784 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f084c06af00 0x7f084c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:42.694 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.683+0000 7f085e7fc700 1 --2- 172.21.15.143:0/304200784 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f084c06af00 0x7f084c06d3c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f08600b1690 tx=0x7f0850008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:42.695 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.687+0000 7f0847fff700 1 -- 172.21.15.143:0/304200784 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0854054970 con 0x7f086006b620 2023-11-08T19:26:42.880 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.879+0000 7f0865946700 1 -- 172.21.15.143:0/304200784 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f086004eee0 con 0x7f086006b620 2023-11-08T19:26:42.881 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.879+0000 7f0847fff700 1 -- 172.21.15.143:0/304200784 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v50) v1 ==== 100+0+11 (secure 0 0 0) 0x7f08540170a0 con 0x7f086006b620 2023-11-08T19:26:42.881 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 64 2023-11-08T19:26:42.883 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.882+0000 7f0865946700 1 -- 172.21.15.143:0/304200784 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f084c06af00 msgr2=0x7f084c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:42.883 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.882+0000 7f0865946700 1 --2- 172.21.15.143:0/304200784 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f084c06af00 0x7f084c06d3c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f08600b1690 tx=0x7f0850008040 comp rx=0 tx=0).stop 2023-11-08T19:26:42.884 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.882+0000 7f0865946700 1 -- 172.21.15.143:0/304200784 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f086006b620 msgr2=0x7f08601f2c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:42.884 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.882+0000 7f0865946700 1 --2- 172.21.15.143:0/304200784 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f086006b620 0x7f08601f2c20 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f085400e870 tx=0x7f085400ec30 comp rx=0 tx=0).stop 2023-11-08T19:26:42.884 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.882+0000 7f0865946700 1 -- 172.21.15.143:0/304200784 shutdown_connections 2023-11-08T19:26:42.884 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.882+0000 7f0865946700 1 --2- 172.21.15.143:0/304200784 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f084c06af00 0x7f084c06d3c0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:42.884 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.882+0000 7f0865946700 1 --2- 172.21.15.143:0/304200784 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0860158140 0x7f08600ac8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:42.885 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.882+0000 7f0865946700 1 --2- 172.21.15.143:0/304200784 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f086006b620 0x7f08601f2c20 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:42.885 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.882+0000 7f0865946700 1 --2- 172.21.15.143:0/304200784 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f08600b1ff0 0x7f08600c91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:42.885 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.882+0000 7f0865946700 1 -- 172.21.15.143:0/304200784 >> 172.21.15.143:0/304200784 conn(0x7f08600d0630 msgr2=0x7f08600b1820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:42.885 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.882+0000 7f0865946700 1 -- 172.21.15.143:0/304200784 shutdown_connections 2023-11-08T19:26:42.885 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:42.882+0000 7f0865946700 1 -- 172.21.15.143:0/304200784 wait complete. 2023-11-08T19:26:42.896 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:26:43.016 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.014+0000 7f5dcbb39700 1 Processor -- start 2023-11-08T19:26:43.016 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.015+0000 7f5dcbb39700 1 -- start start 2023-11-08T19:26:43.020 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.016+0000 7f5dcbb39700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f5dc4159870 0x7f5dc40b2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:43.020 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.016+0000 7f5dcbb39700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5dc40cc410 0x7f5dc40cc860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:43.020 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.016+0000 7f5dcbb39700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5dc40b2a60 con 0x7f5dc406ed60 2023-11-08T19:26:43.020 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.016+0000 7f5dcbb39700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5dc406b090 con 0x7f5dc4159870 2023-11-08T19:26:43.021 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.016+0000 7f5dcbb39700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc406bbc0 con 0x7f5dc40cc410 2023-11-08T19:26:43.021 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.016+0000 7f5dc90d4700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f5dc406ed60 0x7f5dc406b4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:48238/0 (socket says 172.21.15.143:48238) 2023-11-08T19:26:43.021 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.016+0000 7f5dc90d4700 1 -- 172.21.15.143:0/1506227547 learned_addr learned my addr 172.21.15.143:0/1506227547 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:26:43.021 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.016+0000 7f5dc88d3700 1 -- 172.21.15.143:0/1506227547 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 225284145 0 0) 0x7f5dc40b2a60 con 0x7f5dc406ed60 2023-11-08T19:26:43.021 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.016+0000 7f5dc88d3700 1 -- 172.21.15.143:0/1506227547 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5db00035b0 con 0x7f5dc406ed60 2023-11-08T19:26:43.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.016+0000 7f5dca0d6700 1 --2- 172.21.15.143:0/1506227547 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5dc40cc410 0x7f5dc40cc860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:43.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.016+0000 7f5dc98d5700 1 --2- 172.21.15.143:0/1506227547 >> v2:172.21.15.174:3300/0 conn(0x7f5dc4159870 0x7f5dc40b2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:43.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.016+0000 7f5dc88d3700 1 -- 172.21.15.143:0/1506227547 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3569752312 0 0) 0x7f5db00035b0 con 0x7f5dc406ed60 2023-11-08T19:26:43.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.016+0000 7f5dc88d3700 1 -- 172.21.15.143:0/1506227547 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5dc40cc410 msgr2=0x7f5dc40cc860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:43.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.016+0000 7f5dc88d3700 1 --2- 172.21.15.143:0/1506227547 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5dc40cc410 0x7f5dc40cc860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:43.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.016+0000 7f5dc88d3700 1 -- 172.21.15.143:0/1506227547 >> v2:172.21.15.174:3300/0 conn(0x7f5dc4159870 msgr2=0x7f5dc40b2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:43.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.016+0000 7f5dc88d3700 1 --2- 172.21.15.143:0/1506227547 >> v2:172.21.15.174:3300/0 conn(0x7f5dc4159870 0x7f5dc40b2210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:43.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.016+0000 7f5dc88d3700 1 -- 172.21.15.143:0/1506227547 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dc40b2d40 con 0x7f5dc406ed60 2023-11-08T19:26:43.024 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.016+0000 7f5dc88d3700 1 -- 172.21.15.143:0/1506227547 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4147756303 0 0) 0x7f5dc0001670 con 0x7f5dc406ed60 2023-11-08T19:26:43.024 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.016+0000 7f5dc88d3700 1 -- 172.21.15.143:0/1506227547 >> v1:172.21.15.143:6789/0 conn(0x7f5dc406ed60 legacy=0x7f5dc406b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:43.024 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.016+0000 7f5dc88d3700 1 --2- 172.21.15.143:0/1506227547 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5dc40cc410 0x7f5db0003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:43.024 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.016+0000 7f5dc88d3700 1 --2- 172.21.15.143:0/1506227547 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5db0003f80 0x7f5db0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:43.024 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.016+0000 7f5dc88d3700 1 --2- 172.21.15.143:0/1506227547 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5db0004d90 0x7f5db0005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:43.025 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.016+0000 7f5dc88d3700 1 -- 172.21.15.143:0/1506227547 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc406b090 con 0x7f5db0003f80 2023-11-08T19:26:43.025 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.016+0000 7f5dc88d3700 1 -- 172.21.15.143:0/1506227547 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc406bbc0 con 0x7f5db0004d90 2023-11-08T19:26:43.025 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.016+0000 7f5dc88d3700 1 -- 172.21.15.143:0/1506227547 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5db000baf0 con 0x7f5dc40cc410 2023-11-08T19:26:43.025 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.017+0000 7f5dc90d4700 1 --2- 172.21.15.143:0/1506227547 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5db0003f80 0x7f5db0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:43.025 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.017+0000 7f5dc98d5700 1 --2- 172.21.15.143:0/1506227547 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5dc40cc410 0x7f5db0003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:43.026 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.017+0000 7f5dca0d6700 1 --2- 172.21.15.143:0/1506227547 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5db0004d90 0x7f5db0005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:43.026 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.017+0000 7f5dc90d4700 1 -- 172.21.15.143:0/1506227547 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5dc40cc410 msgr2=0x7f5db0003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:43.026 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.017+0000 7f5dc90d4700 1 --2- 172.21.15.143:0/1506227547 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5dc40cc410 0x7f5db0003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:43.026 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.017+0000 7f5dc90d4700 1 -- 172.21.15.143:0/1506227547 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5db0004d90 msgr2=0x7f5db0005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:43.027 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.017+0000 7f5dc90d4700 1 --2- 172.21.15.143:0/1506227547 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5db0004d90 0x7f5db0005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:43.027 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.017+0000 7f5dc90d4700 1 -- 172.21.15.143:0/1506227547 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5db00035b0 con 0x7f5db0003f80 2023-11-08T19:26:43.027 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.017+0000 7f5dc90d4700 1 --2- 172.21.15.143:0/1506227547 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5db0003f80 0x7f5db0004850 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f5dc000c370 tx=0x7f5db0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0553838d46ad1aa server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:43.027 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.017+0000 7f5dc88d3700 1 -- 172.21.15.143:0/1506227547 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dc0007c60 con 0x7f5db0003f80 2023-11-08T19:26:43.027 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.017+0000 7f5dc88d3700 1 -- 172.21.15.143:0/1506227547 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dc0005960 con 0x7f5db0003f80 2023-11-08T19:26:43.028 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.017+0000 7f5dc88d3700 1 -- 172.21.15.143:0/1506227547 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dc0005d10 con 0x7f5db0003f80 2023-11-08T19:26:43.028 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.018+0000 7f5dcbb39700 1 -- 172.21.15.143:0/1506227547 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5db0003f80 msgr2=0x7f5db0004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:43.028 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.018+0000 7f5dcbb39700 1 --2- 172.21.15.143:0/1506227547 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5db0003f80 0x7f5db0004850 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f5dc000c370 tx=0x7f5db0005880 comp rx=0 tx=0).stop 2023-11-08T19:26:43.028 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.018+0000 7f5dca0d6700 1 -- 172.21.15.143:0/1506227547 reap_dead start 2023-11-08T19:26:43.028 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.018+0000 7f5dcbb39700 1 -- 172.21.15.143:0/1506227547 shutdown_connections 2023-11-08T19:26:43.029 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.018+0000 7f5dcbb39700 1 -- 172.21.15.143:0/1506227547 >> 172.21.15.143:0/1506227547 conn(0x7f5dc4099ab0 msgr2=0x7f5dc40d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:43.029 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.018+0000 7f5dcbb39700 1 -- 172.21.15.143:0/1506227547 shutdown_connections 2023-11-08T19:26:43.029 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.018+0000 7f5dcbb39700 1 -- 172.21.15.143:0/1506227547 wait complete. 2023-11-08T19:26:43.032 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.019+0000 7f5dcbb39700 1 Processor -- start 2023-11-08T19:26:43.032 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.019+0000 7f5dcbb39700 1 -- start start 2023-11-08T19:26:43.032 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.020+0000 7f5dcbb39700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5db0003f80 0x7f5dc414ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:43.032 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.020+0000 7f5dcbb39700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5dc40c5aa0 0x7f5dc40c5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:43.032 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.020+0000 7f5dcbb39700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5dc4159e80 0x7f5dc415a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:43.033 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.020+0000 7f5dcbb39700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc40c6460 con 0x7f5dc40c5aa0 2023-11-08T19:26:43.033 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.020+0000 7f5dcbb39700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc40b3e90 con 0x7f5db0003f80 2023-11-08T19:26:43.033 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.020+0000 7f5dcbb39700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc414cf70 con 0x7f5dc4159e80 2023-11-08T19:26:43.033 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.020+0000 7f5dc90d4700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5dc40c5aa0 0x7f5dc40c5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:43.033 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.020+0000 7f5dc90d4700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5dc40c5aa0 0x7f5dc40c5f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52096/0 (socket says 172.21.15.143:52096) 2023-11-08T19:26:43.034 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.020+0000 7f5dc90d4700 1 -- 172.21.15.143:0/845928893 learned_addr learned my addr 172.21.15.143:0/845928893 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:43.034 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.020+0000 7f5dc98d5700 1 --2- 172.21.15.143:0/845928893 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5db0003f80 0x7f5dc414ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:43.034 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.020+0000 7f5dca0d6700 1 --2- 172.21.15.143:0/845928893 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5dc4159e80 0x7f5dc415a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:43.034 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.020+0000 7f5dc90d4700 1 -- 172.21.15.143:0/845928893 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5dc4159e80 msgr2=0x7f5dc415a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:43.035 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.020+0000 7f5dc90d4700 1 --2- 172.21.15.143:0/845928893 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5dc4159e80 0x7f5dc415a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:43.035 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.020+0000 7f5dc90d4700 1 -- 172.21.15.143:0/845928893 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5db0003f80 msgr2=0x7f5dc414ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:43.035 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.020+0000 7f5dc90d4700 1 --2- 172.21.15.143:0/845928893 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5db0003f80 0x7f5dc414ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:43.035 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.020+0000 7f5dc90d4700 1 -- 172.21.15.143:0/845928893 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dc415abb0 con 0x7f5dc40c5aa0 2023-11-08T19:26:43.035 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.020+0000 7f5dc90d4700 1 --2- 172.21.15.143:0/845928893 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5dc40c5aa0 0x7f5dc40c5f20 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f5db000ba40 tx=0x7f5dc0000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:43.036 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.020+0000 7f5dba7fc700 1 -- 172.21.15.143:0/845928893 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dc0004870 con 0x7f5dc40c5aa0 2023-11-08T19:26:43.036 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.020+0000 7f5dba7fc700 1 -- 172.21.15.143:0/845928893 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dc00049f0 con 0x7f5dc40c5aa0 2023-11-08T19:26:43.036 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.020+0000 7f5dcbb39700 1 -- 172.21.15.143:0/845928893 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dc415ae90 con 0x7f5dc40c5aa0 2023-11-08T19:26:43.036 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.020+0000 7f5dba7fc700 1 -- 172.21.15.143:0/845928893 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dc0004da0 con 0x7f5dc40c5aa0 2023-11-08T19:26:43.036 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.020+0000 7f5dcbb39700 1 -- 172.21.15.143:0/845928893 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dc415b450 con 0x7f5dc40c5aa0 2023-11-08T19:26:43.037 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.021+0000 7f5dcbb39700 1 -- 172.21.15.143:0/845928893 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dc40b4040 con 0x7f5dc40c5aa0 2023-11-08T19:26:43.037 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.022+0000 7f5dba7fc700 1 -- 172.21.15.143:0/845928893 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5dc00070d0 con 0x7f5dc40c5aa0 2023-11-08T19:26:43.037 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.022+0000 7f5dba7fc700 1 --2- 172.21.15.143:0/845928893 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f5da806af00 0x7f5da806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:43.037 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.022+0000 7f5dba7fc700 1 -- 172.21.15.143:0/845928893 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f5dc008a830 con 0x7f5dc40c5aa0 2023-11-08T19:26:43.037 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.022+0000 7f5dc98d5700 1 --2- 172.21.15.143:0/845928893 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f5da806af00 0x7f5da806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:43.038 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.023+0000 7f5dc98d5700 1 --2- 172.21.15.143:0/845928893 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f5da806af00 0x7f5da806d3c0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f5dc4158420 tx=0x7f5db40091c0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:43.038 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.026+0000 7f5dba7fc700 1 -- 172.21.15.143:0/845928893 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5dc40b4040 con 0x7f5dc40c5aa0 2023-11-08T19:26:43.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.256+0000 7f5dcbb39700 1 -- 172.21.15.143:0/845928893 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5dc40b4000 con 0x7f5dc40c5aa0 2023-11-08T19:26:43.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.257+0000 7f5dba7fc700 1 -- 172.21.15.143:0/845928893 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4925 (secure 0 0 0) 0x7f5dc00557d0 con 0x7f5dc40c5aa0 2023-11-08T19:26:43.258 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:26:43.259 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4988,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3140991763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":3140991763},{"type":"v1","addr":"172.21.15.143:6835","nonce":3140991763}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":4997,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6836/2388011942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2388011942},{"type":"v1","addr":"172.21.15.174:6836","nonce":2388011942}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5000,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/1339111936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6835","nonce":1339111936},{"type":"v1","addr":"172.21.15.174:6837","nonce":1339111936}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T19:26:25.128617+0000","modified":"2023-11-08T19:26:28.156717+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5242},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5242":{"gid":5242,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.143:6834/3591097261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":3591097261},{"type":"v1","addr":"172.21.15.143:6834","nonce":3591097261}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-08T19:26:43.261 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.259+0000 7f5dcbb39700 1 -- 172.21.15.143:0/845928893 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f5da806af00 msgr2=0x7f5da806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:43.261 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.259+0000 7f5dcbb39700 1 --2- 172.21.15.143:0/845928893 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f5da806af00 0x7f5da806d3c0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f5dc4158420 tx=0x7f5db40091c0 comp rx=0 tx=0).stop 2023-11-08T19:26:43.261 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.259+0000 7f5dcbb39700 1 -- 172.21.15.143:0/845928893 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5dc40c5aa0 msgr2=0x7f5dc40c5f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:43.261 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.259+0000 7f5dcbb39700 1 --2- 172.21.15.143:0/845928893 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5dc40c5aa0 0x7f5dc40c5f20 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f5db000ba40 tx=0x7f5dc0000fe0 comp rx=0 tx=0).stop 2023-11-08T19:26:43.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.259+0000 7f5dcbb39700 1 -- 172.21.15.143:0/845928893 shutdown_connections 2023-11-08T19:26:43.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.259+0000 7f5dcbb39700 1 --2- 172.21.15.143:0/845928893 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f5da806af00 0x7f5da806d3c0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:43.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.259+0000 7f5dcbb39700 1 --2- 172.21.15.143:0/845928893 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5dc4159e80 0x7f5dc415a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:43.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.259+0000 7f5dcbb39700 1 --2- 172.21.15.143:0/845928893 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5db0003f80 0x7f5dc414ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:43.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.259+0000 7f5dcbb39700 1 --2- 172.21.15.143:0/845928893 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5dc40c5aa0 0x7f5dc40c5f20 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:43.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.259+0000 7f5dcbb39700 1 -- 172.21.15.143:0/845928893 >> 172.21.15.143:0/845928893 conn(0x7f5dc4099ab0 msgr2=0x7f5dc40af390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:43.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.259+0000 7f5dcbb39700 1 -- 172.21.15.143:0/845928893 shutdown_connections 2023-11-08T19:26:43.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.259+0000 7f5dcbb39700 1 -- 172.21.15.143:0/845928893 wait complete. 2023-11-08T19:26:43.263 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 25 2023-11-08T19:26:43.273 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:26:43.393 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.391+0000 7f9487cdf700 1 Processor -- start 2023-11-08T19:26:43.393 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.391+0000 7f9487cdf700 1 -- start start 2023-11-08T19:26:43.395 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.392+0000 7f9487cdf700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f94800b2920 0x7f94800cf510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:43.396 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.392+0000 7f9487cdf700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94800cf080 con 0x7f94800cbe20 2023-11-08T19:26:43.396 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.392+0000 7f9487cdf700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94800ce470 con 0x7f948015aa30 2023-11-08T19:26:43.396 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.392+0000 7f9487cdf700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94800ce750 con 0x7f94800b2920 2023-11-08T19:26:43.396 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.392+0000 7f948627c700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f94800cbe20 0x7f94800cc290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:48260/0 (socket says 172.21.15.143:48260) 2023-11-08T19:26:43.397 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.392+0000 7f948627c700 1 -- 172.21.15.143:0/1819056096 learned_addr learned my addr 172.21.15.143:0/1819056096 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:26:43.397 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.393+0000 7f948527a700 1 --2- 172.21.15.143:0/1819056096 >> v2:172.21.15.174:3300/0 conn(0x7f94800b2920 0x7f94800cf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:43.397 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.393+0000 7f9484a79700 1 -- 172.21.15.143:0/1819056096 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 505582510 0 0) 0x7f94800cf080 con 0x7f94800cbe20 2023-11-08T19:26:43.397 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.393+0000 7f9484a79700 1 -- 172.21.15.143:0/1819056096 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f946c0035b0 con 0x7f94800cbe20 2023-11-08T19:26:43.397 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.393+0000 7f9484a79700 1 -- 172.21.15.143:0/1819056096 <== mon.1 v1:172.21.15.174:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2900400413 0 0) 0x7f94800ce470 con 0x7f948015aa30 2023-11-08T19:26:43.398 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.393+0000 7f9484a79700 1 -- 172.21.15.143:0/1819056096 --> v1:172.21.15.174:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94800cf080 con 0x7f948015aa30 2023-11-08T19:26:43.398 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.393+0000 7f9484a79700 1 -- 172.21.15.143:0/1819056096 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2073171064 0 0) 0x7f946c0035b0 con 0x7f94800cbe20 2023-11-08T19:26:43.398 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.393+0000 7f948527a700 1 -- 172.21.15.143:0/1819056096 >> v1:172.21.15.174:6789/0 conn(0x7f948015aa30 legacy=0x7f94800b2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:43.398 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.393+0000 7f948527a700 1 -- 172.21.15.143:0/1819056096 >> v1:172.21.15.143:6789/0 conn(0x7f94800cbe20 legacy=0x7f94800cc290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:43.399 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.393+0000 7f948527a700 1 -- 172.21.15.143:0/1819056096 --> v2:172.21.15.174:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94800cea30 con 0x7f94800b2920 2023-11-08T19:26:43.399 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.393+0000 7f948527a700 1 --2- 172.21.15.143:0/1819056096 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f94800b2920 0x7f94800cf510 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f947c009840 tx=0x7f947c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d321bea94c6b920 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:43.399 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.393+0000 7f9484a79700 1 -- 172.21.15.143:0/1819056096 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94780019f0 con 0x7f94800b2920 2023-11-08T19:26:43.399 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.393+0000 7f9484a79700 1 -- 172.21.15.143:0/1819056096 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f947c004030 con 0x7f94800b2920 2023-11-08T19:26:43.399 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.393+0000 7f9484a79700 1 -- 172.21.15.143:0/1819056096 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f947c0043c0 con 0x7f94800b2920 2023-11-08T19:26:43.399 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.394+0000 7f9487cdf700 1 -- 172.21.15.143:0/1819056096 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f94800b2920 msgr2=0x7f94800cf510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:43.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.394+0000 7f9487cdf700 1 --2- 172.21.15.143:0/1819056096 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f94800b2920 0x7f94800cf510 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f947c009840 tx=0x7f947c009b50 comp rx=0 tx=0).stop 2023-11-08T19:26:43.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.394+0000 7f9487cdf700 1 -- 172.21.15.143:0/1819056096 shutdown_connections 2023-11-08T19:26:43.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.394+0000 7f9487cdf700 1 --2- 172.21.15.143:0/1819056096 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f94800b2920 0x7f94800cf510 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:43.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.394+0000 7f9487cdf700 1 -- 172.21.15.143:0/1819056096 >> 172.21.15.143:0/1819056096 conn(0x7f9480099ab0 msgr2=0x7f94800d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:43.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.394+0000 7f9487cdf700 1 -- 172.21.15.143:0/1819056096 shutdown_connections 2023-11-08T19:26:43.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.394+0000 7f9487cdf700 1 -- 172.21.15.143:0/1819056096 wait complete. 2023-11-08T19:26:43.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.395+0000 7f9487cdf700 1 Processor -- start 2023-11-08T19:26:43.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.395+0000 7f9487cdf700 1 -- start start 2023-11-08T19:26:43.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.396+0000 7f9487cdf700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f94800b2920 0x7f948015d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:43.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.396+0000 7f9487cdf700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94800cbe20 0x7f94800b5d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:43.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.396+0000 7f9487cdf700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f948015aa30 0x7f9480152d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:43.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.396+0000 7f9487cdf700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9480157430 con 0x7f94800cbe20 2023-11-08T19:26:43.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.396+0000 7f9487cdf700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94801550f0 con 0x7f948015aa30 2023-11-08T19:26:43.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.396+0000 7f9487cdf700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9480156860 con 0x7f94800b2920 2023-11-08T19:26:43.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.396+0000 7f948527a700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94800cbe20 0x7f94800b5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:43.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.396+0000 7f948527a700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94800cbe20 0x7f94800b5d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52110/0 (socket says 172.21.15.143:52110) 2023-11-08T19:26:43.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.396+0000 7f948527a700 1 -- 172.21.15.143:0/2142563017 learned_addr learned my addr 172.21.15.143:0/2142563017 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:43.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.396+0000 7f9485a7b700 1 --2- 172.21.15.143:0/2142563017 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f94800b2920 0x7f948015d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:43.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.396+0000 7f948627c700 1 --2- 172.21.15.143:0/2142563017 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f948015aa30 0x7f9480152d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:43.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.396+0000 7f948527a700 1 -- 172.21.15.143:0/2142563017 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f94800b2920 msgr2=0x7f948015d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:43.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.396+0000 7f948527a700 1 --2- 172.21.15.143:0/2142563017 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f94800b2920 0x7f948015d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:43.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.396+0000 7f948527a700 1 -- 172.21.15.143:0/2142563017 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f948015aa30 msgr2=0x7f9480152d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:43.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.396+0000 7f948527a700 1 --2- 172.21.15.143:0/2142563017 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f948015aa30 0x7f9480152d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:43.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.396+0000 7f948527a700 1 -- 172.21.15.143:0/2142563017 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f947c0094a0 con 0x7f94800cbe20 2023-11-08T19:26:43.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.396+0000 7f948527a700 1 --2- 172.21.15.143:0/2142563017 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94800cbe20 0x7f94800b5d40 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f947c0090b0 tx=0x7f947c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:43.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.396+0000 7f94767fc700 1 -- 172.21.15.143:0/2142563017 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f947c004570 con 0x7f94800cbe20 2023-11-08T19:26:43.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.396+0000 7f94767fc700 1 -- 172.21.15.143:0/2142563017 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f947c01f070 con 0x7f94800cbe20 2023-11-08T19:26:43.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.397+0000 7f9487cdf700 1 -- 172.21.15.143:0/2142563017 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9480156b00 con 0x7f94800cbe20 2023-11-08T19:26:43.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.397+0000 7f94767fc700 1 -- 172.21.15.143:0/2142563017 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f947c011870 con 0x7f94800cbe20 2023-11-08T19:26:43.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.397+0000 7f9487cdf700 1 -- 172.21.15.143:0/2142563017 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94801570c0 con 0x7f94800cbe20 2023-11-08T19:26:43.409 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.397+0000 7f9487cdf700 1 -- 172.21.15.143:0/2142563017 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94801575a0 con 0x7f94800cbe20 2023-11-08T19:26:43.409 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.398+0000 7f94767fc700 1 -- 172.21.15.143:0/2142563017 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f947c0119d0 con 0x7f94800cbe20 2023-11-08T19:26:43.409 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.398+0000 7f94767fc700 1 --2- 172.21.15.143:0/2142563017 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f946406af00 0x7f946406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:43.409 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.399+0000 7f94767fc700 1 -- 172.21.15.143:0/2142563017 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f947c021070 con 0x7f94800cbe20 2023-11-08T19:26:43.409 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.399+0000 7f9485a7b700 1 --2- 172.21.15.143:0/2142563017 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f946406af00 0x7f946406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:43.410 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.399+0000 7f9485a7b700 1 --2- 172.21.15.143:0/2142563017 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f946406af00 0x7f946406d3c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f9470007fd0 tx=0x7f9470009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:43.410 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.403+0000 7f94767fc700 1 -- 172.21.15.143:0/2142563017 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f947c055f30 con 0x7f94800cbe20 2023-11-08T19:26:43.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.595+0000 7f9487cdf700 1 -- 172.21.15.143:0/2142563017 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f94800b2ff0 con 0x7f94800cbe20 2023-11-08T19:26:43.598 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.597+0000 7f94767fc700 1 -- 172.21.15.143:0/2142563017 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) v1 ==== 74+0+18466 (secure 0 0 0) 0x7f947c02b020 con 0x7f94800cbe20 2023-11-08T19:26:43.598 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:26:43.599 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":50,"fsid":"968483af-77fc-4068-ad7c-5c2a1a4613d7","created":"2023-11-08T19:24:32.826484+0000","modified":"2023-11-08T19:26:39.794453+0000","last_up_change":"2023-11-08T19:24:43.573880+0000","last_in_change":"2023-11-08T19:24:35.314525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:44.551530+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"snap-test-pool","create_time":"2023-11-08T19:26:02.874384+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":36,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-11-08T19:26:22.869780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-11-08T19:26:24.093179+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bae98ec1-d94e-403d-a220-a522a53ea6f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6809","nonce":1487830566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6811","nonce":1487830566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6815","nonce":1487830566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6813","nonce":1487830566}]},"public_addr":"172.21.15.143:6809/1487830566","cluster_addr":"172.21.15.143:6811/1487830566","heartbeat_back_addr":"172.21.15.143:6815/1487830566","heartbeat_front_addr":"172.21.15.143:6813/1487830566","state":["exists","up"]},{"osd":1,"uuid":"5cb6538a-1ccb-4322-8633-09f66af0b7e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6801","nonce":787341397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6803","nonce":787341397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6807","nonce":787341397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6805","nonce":787341397}]},"public_addr":"172.21.15.143:6801/787341397","cluster_addr":"172.21.15.143:6803/787341397","heartbeat_back_addr":"172.21.15.143:6807/787341397","heartbeat_front_addr":"172.21.15.143:6805/787341397","state":["exists","up"]},{"osd":2,"uuid":"cdbf522c-60ef-4d60-9653-8dcb529c101a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6822","nonce":2301325597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6826","nonce":2301325597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6831","nonce":2301325597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6829","nonce":2301325597}]},"public_addr":"172.21.15.143:6822/2301325597","cluster_addr":"172.21.15.143:6826/2301325597","heartbeat_back_addr":"172.21.15.143:6831/2301325597","heartbeat_front_addr":"172.21.15.143:6829/2301325597","state":["exists","up"]},{"osd":3,"uuid":"4e2783cf-ebb9-4dd8-8245-bc1d17ce4325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6817","nonce":2021504428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6819","nonce":2021504428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6827","nonce":2021504428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6823","nonce":2021504428}]},"public_addr":"172.21.15.143:6817/2021504428","cluster_addr":"172.21.15.143:6819/2021504428","heartbeat_back_addr":"172.21.15.143:6827/2021504428","heartbeat_front_addr":"172.21.15.143:6823/2021504428","state":["exists","up"]},{"osd":4,"uuid":"691256fa-9b77-49b9-a346-7ae1adf0ca80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6801","nonce":1993380595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6803","nonce":1993380595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6807","nonce":1993380595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6805","nonce":1993380595}]},"public_addr":"172.21.15.174:6801/1993380595","cluster_addr":"172.21.15.174:6803/1993380595","heartbeat_back_addr":"172.21.15.174:6807/1993380595","heartbeat_front_addr":"172.21.15.174:6805/1993380595","state":["exists","up"]},{"osd":5,"uuid":"756fc352-48c9-46ce-b52e-0541fe6f8525","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6809","nonce":1777923303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6811","nonce":1777923303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6815","nonce":1777923303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6813","nonce":1777923303}]},"public_addr":"172.21.15.174:6809/1777923303","cluster_addr":"172.21.15.174:6811/1777923303","heartbeat_back_addr":"172.21.15.174:6815/1777923303","heartbeat_front_addr":"172.21.15.174:6813/1777923303","state":["exists","up"]},{"osd":6,"uuid":"fab2ac2b-ed97-4cde-9825-b480eabf8b8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6825","nonce":2755130316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6827","nonce":2755130316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6831","nonce":2755130316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6829","nonce":2755130316}]},"public_addr":"172.21.15.174:6825/2755130316","cluster_addr":"172.21.15.174:6827/2755130316","heartbeat_back_addr":"172.21.15.174:6831/2755130316","heartbeat_front_addr":"172.21.15.174:6829/2755130316","state":["exists","up"]},{"osd":7,"uuid":"3de9b797-2ab6-4d41-a863-7e7d638fd703","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6817","nonce":2847141411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6819","nonce":2847141411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6823","nonce":2847141411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6821","nonce":2847141411}]},"public_addr":"172.21.15.174:6817/2847141411","cluster_addr":"172.21.15.174:6819/2847141411","heartbeat_back_addr":"172.21.15.174:6823/2847141411","heartbeat_front_addr":"172.21.15.174:6821/2847141411","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.477076+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.496890+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.539305+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.454673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.538881+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.521294+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.520712+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.540404+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.1a","mappings":[{"from":0,"to":1}]},{"pgid":"8.28","mappings":[{"from":0,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:26:43.601 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.599+0000 7f9487cdf700 1 -- 172.21.15.143:0/2142563017 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f946406af00 msgr2=0x7f946406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:43.601 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.599+0000 7f9487cdf700 1 --2- 172.21.15.143:0/2142563017 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f946406af00 0x7f946406d3c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f9470007fd0 tx=0x7f9470009040 comp rx=0 tx=0).stop 2023-11-08T19:26:43.601 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.599+0000 7f9487cdf700 1 -- 172.21.15.143:0/2142563017 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94800cbe20 msgr2=0x7f94800b5d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:43.601 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.599+0000 7f9487cdf700 1 --2- 172.21.15.143:0/2142563017 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94800cbe20 0x7f94800b5d40 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f947c0090b0 tx=0x7f947c004750 comp rx=0 tx=0).stop 2023-11-08T19:26:43.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.599+0000 7f9487cdf700 1 -- 172.21.15.143:0/2142563017 shutdown_connections 2023-11-08T19:26:43.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.599+0000 7f9487cdf700 1 --2- 172.21.15.143:0/2142563017 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f946406af00 0x7f946406d3c0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:43.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.599+0000 7f9487cdf700 1 --2- 172.21.15.143:0/2142563017 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f94800b2920 0x7f948015d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:43.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.599+0000 7f9487cdf700 1 --2- 172.21.15.143:0/2142563017 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f948015aa30 0x7f9480152d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:43.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.599+0000 7f9487cdf700 1 --2- 172.21.15.143:0/2142563017 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94800cbe20 0x7f94800b5d40 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:43.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.599+0000 7f9487cdf700 1 -- 172.21.15.143:0/2142563017 >> 172.21.15.143:0/2142563017 conn(0x7f9480099ab0 msgr2=0x7f9480155590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:43.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.599+0000 7f9487cdf700 1 -- 172.21.15.143:0/2142563017 shutdown_connections 2023-11-08T19:26:43.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.599+0000 7f9487cdf700 1 -- 172.21.15.143:0/2142563017 wait complete. 2023-11-08T19:26:43.613 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-08T19:26:43.614 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:26:43.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.733+0000 7f59908b8700 1 Processor -- start 2023-11-08T19:26:43.735 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.733+0000 7f59908b8700 1 -- start start 2023-11-08T19:26:43.737 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.734+0000 7f59908b8700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f598c15aa30 0x7f598c0cf510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:43.738 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.734+0000 7f59908b8700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f598c0b2210 0x7f598c0b2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:43.738 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.734+0000 7f59908b8700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f598c0cbe20 0x7f598c0cc270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:43.738 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.734+0000 7f59908b8700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f598c0b2bb0 con 0x7f598c15aa30 2023-11-08T19:26:43.738 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.734+0000 7f59908b8700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f598c0cf080 con 0x7f598c0cbe20 2023-11-08T19:26:43.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.734+0000 7f59908b8700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f598c0ce470 con 0x7f598c0b2210 2023-11-08T19:26:43.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.734+0000 7f598a59c700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f598c15aa30 0x7f598c0cf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:43.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.734+0000 7f598a59c700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f598c15aa30 0x7f598c0cf510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52116/0 (socket says 172.21.15.143:52116) 2023-11-08T19:26:43.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.734+0000 7f598a59c700 1 -- 172.21.15.143:0/221753172 learned_addr learned my addr 172.21.15.143:0/221753172 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:43.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.734+0000 7f598ad9d700 1 --2- 172.21.15.143:0/221753172 >> v2:172.21.15.174:3300/0 conn(0x7f598c0cbe20 0x7f598c0cc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:43.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.734+0000 7f5989d9b700 1 --2- 172.21.15.143:0/221753172 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f598c0b2210 0x7f598c0b2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:43.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.734+0000 7f598a59c700 1 -- 172.21.15.143:0/221753172 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f598c0b2210 msgr2=0x7f598c0b2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:43.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.734+0000 7f598a59c700 1 --2- 172.21.15.143:0/221753172 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f598c0b2210 0x7f598c0b2670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:43.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.734+0000 7f598a59c700 1 -- 172.21.15.143:0/221753172 >> v2:172.21.15.174:3300/0 conn(0x7f598c0cbe20 msgr2=0x7f598c0cc270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:43.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.734+0000 7f598a59c700 1 --2- 172.21.15.143:0/221753172 >> v2:172.21.15.174:3300/0 conn(0x7f598c0cbe20 0x7f598c0cc270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:43.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.734+0000 7f598a59c700 1 -- 172.21.15.143:0/221753172 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f598c0ce750 con 0x7f598c15aa30 2023-11-08T19:26:43.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.734+0000 7f598a59c700 1 --2- 172.21.15.143:0/221753172 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f598c15aa30 0x7f598c0cf510 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f598c0ccf00 tx=0x7f597c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85bc1b2d6a4a7860 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:43.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.735+0000 7f598959a700 1 -- 172.21.15.143:0/221753172 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f597c005e30 con 0x7f598c15aa30 2023-11-08T19:26:43.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.735+0000 7f598959a700 1 -- 172.21.15.143:0/221753172 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f597c004030 con 0x7f598c15aa30 2023-11-08T19:26:43.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.735+0000 7f598959a700 1 -- 172.21.15.143:0/221753172 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f597c0043e0 con 0x7f598c15aa30 2023-11-08T19:26:43.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.735+0000 7f59908b8700 1 -- 172.21.15.143:0/221753172 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f598c15aa30 msgr2=0x7f598c0cf510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:43.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.735+0000 7f59908b8700 1 --2- 172.21.15.143:0/221753172 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f598c15aa30 0x7f598c0cf510 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f598c0ccf00 tx=0x7f597c009bd0 comp rx=0 tx=0).stop 2023-11-08T19:26:43.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.735+0000 7f59908b8700 1 -- 172.21.15.143:0/221753172 shutdown_connections 2023-11-08T19:26:43.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.735+0000 7f59908b8700 1 --2- 172.21.15.143:0/221753172 >> v2:172.21.15.174:3300/0 conn(0x7f598c0cbe20 0x7f598c0cc270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:43.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.735+0000 7f59908b8700 1 --2- 172.21.15.143:0/221753172 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f598c15aa30 0x7f598c0cf510 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:43.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.735+0000 7f59908b8700 1 --2- 172.21.15.143:0/221753172 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f598c0b2210 0x7f598c0b2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:43.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.735+0000 7f59908b8700 1 -- 172.21.15.143:0/221753172 >> 172.21.15.143:0/221753172 conn(0x7f598c099ab0 msgr2=0x7f598c0d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:43.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.735+0000 7f59908b8700 1 -- 172.21.15.143:0/221753172 shutdown_connections 2023-11-08T19:26:43.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.735+0000 7f59908b8700 1 -- 172.21.15.143:0/221753172 wait complete. 2023-11-08T19:26:43.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.737+0000 7f59908b8700 1 Processor -- start 2023-11-08T19:26:43.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.737+0000 7f59908b8700 1 -- start start 2023-11-08T19:26:43.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.737+0000 7f59908b8700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f598c0b2210 0x7f598c15ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:43.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.737+0000 7f59908b8700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f598c0cbe20 0x7f598c0c58d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:43.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.737+0000 7f59908b8700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f598c15aa30 0x7f598c1ef7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:43.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.737+0000 7f59908b8700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f598c15b350 con 0x7f598c0b2210 2023-11-08T19:26:43.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.737+0000 7f59908b8700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f598c1f3930 con 0x7f598c15aa30 2023-11-08T19:26:43.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.737+0000 7f59908b8700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f598c14ece0 con 0x7f598c0cbe20 2023-11-08T19:26:43.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.737+0000 7f598a59c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f598c0b2210 0x7f598c15ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:43.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.738+0000 7f598a59c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f598c0b2210 0x7f598c15ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52122/0 (socket says 172.21.15.143:52122) 2023-11-08T19:26:43.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.738+0000 7f598a59c700 1 -- 172.21.15.143:0/903612598 learned_addr learned my addr 172.21.15.143:0/903612598 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:43.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.738+0000 7f5989d9b700 1 --2- 172.21.15.143:0/903612598 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f598c0cbe20 0x7f598c0c58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:43.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.738+0000 7f598ad9d700 1 --2- 172.21.15.143:0/903612598 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f598c15aa30 0x7f598c1ef7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:43.750 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.738+0000 7f598a59c700 1 -- 172.21.15.143:0/903612598 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f598c0cbe20 msgr2=0x7f598c0c58d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:43.750 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.738+0000 7f598a59c700 1 --2- 172.21.15.143:0/903612598 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f598c0cbe20 0x7f598c0c58d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:43.750 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.738+0000 7f598a59c700 1 -- 172.21.15.143:0/903612598 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f598c15aa30 msgr2=0x7f598c1ef7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:43.751 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.738+0000 7f598a59c700 1 --2- 172.21.15.143:0/903612598 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f598c15aa30 0x7f598c1ef7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:43.751 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.738+0000 7f598a59c700 1 -- 172.21.15.143:0/903612598 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f597c009500 con 0x7f598c0b2210 2023-11-08T19:26:43.751 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.738+0000 7f598a59c700 1 --2- 172.21.15.143:0/903612598 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f598c0b2210 0x7f598c15ae10 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f597c009c10 tx=0x7f597c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:43.751 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.738+0000 7f597affd700 1 -- 172.21.15.143:0/903612598 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f597c003cb0 con 0x7f598c0b2210 2023-11-08T19:26:43.751 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.738+0000 7f597affd700 1 -- 172.21.15.143:0/903612598 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f597c003e10 con 0x7f598c0b2210 2023-11-08T19:26:43.752 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.738+0000 7f59908b8700 1 -- 172.21.15.143:0/903612598 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f598c1f3b80 con 0x7f598c0b2210 2023-11-08T19:26:43.752 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.738+0000 7f597affd700 1 -- 172.21.15.143:0/903612598 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f597c01d630 con 0x7f598c0b2210 2023-11-08T19:26:43.752 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.738+0000 7f59908b8700 1 -- 172.21.15.143:0/903612598 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f598c1f4140 con 0x7f598c0b2210 2023-11-08T19:26:43.752 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.739+0000 7f59908b8700 1 -- 172.21.15.143:0/903612598 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f598c04eee0 con 0x7f598c0b2210 2023-11-08T19:26:43.752 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.739+0000 7f597affd700 1 -- 172.21.15.143:0/903612598 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f597c01d790 con 0x7f598c0b2210 2023-11-08T19:26:43.753 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.740+0000 7f597affd700 1 --2- 172.21.15.143:0/903612598 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f596c06af00 0x7f596c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:43.753 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.740+0000 7f597affd700 1 -- 172.21.15.143:0/903612598 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f597c08ab00 con 0x7f598c0b2210 2023-11-08T19:26:43.753 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.740+0000 7f5989d9b700 1 --2- 172.21.15.143:0/903612598 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f596c06af00 0x7f596c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:43.753 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.740+0000 7f5989d9b700 1 --2- 172.21.15.143:0/903612598 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f596c06af00 0x7f596c06d3c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f598c0b6790 tx=0x7f5980008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:43.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.744+0000 7f597affd700 1 -- 172.21.15.143:0/903612598 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f597c055aa0 con 0x7f598c0b2210 2023-11-08T19:26:43.979 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.978+0000 7f59908b8700 1 -- 172.21.15.143:0/903612598 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f598c0b2ff0 con 0x7f598c0b2210 2023-11-08T19:26:43.980 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.978+0000 7f597affd700 1 -- 172.21.15.143:0/903612598 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4925 (secure 0 0 0) 0x7f597c0590c0 con 0x7f598c0b2210 2023-11-08T19:26:43.980 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:26:43.980 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4988,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3140991763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":3140991763},{"type":"v1","addr":"172.21.15.143:6835","nonce":3140991763}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":4997,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6836/2388011942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2388011942},{"type":"v1","addr":"172.21.15.174:6836","nonce":2388011942}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5000,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/1339111936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6835","nonce":1339111936},{"type":"v1","addr":"172.21.15.174:6837","nonce":1339111936}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T19:26:25.128617+0000","modified":"2023-11-08T19:26:28.156717+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5242},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5242":{"gid":5242,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.143:6834/3591097261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":3591097261},{"type":"v1","addr":"172.21.15.143:6834","nonce":3591097261}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-08T19:26:43.982 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.981+0000 7f59908b8700 1 -- 172.21.15.143:0/903612598 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f596c06af00 msgr2=0x7f596c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:43.983 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.981+0000 7f59908b8700 1 --2- 172.21.15.143:0/903612598 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f596c06af00 0x7f596c06d3c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f598c0b6790 tx=0x7f5980008040 comp rx=0 tx=0).stop 2023-11-08T19:26:43.983 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.981+0000 7f59908b8700 1 -- 172.21.15.143:0/903612598 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f598c0b2210 msgr2=0x7f598c15ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:43.983 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.981+0000 7f59908b8700 1 --2- 172.21.15.143:0/903612598 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f598c0b2210 0x7f598c15ae10 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f597c009c10 tx=0x7f597c004750 comp rx=0 tx=0).stop 2023-11-08T19:26:43.983 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.981+0000 7f59908b8700 1 -- 172.21.15.143:0/903612598 shutdown_connections 2023-11-08T19:26:43.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.981+0000 7f59908b8700 1 --2- 172.21.15.143:0/903612598 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f596c06af00 0x7f596c06d3c0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:43.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.981+0000 7f59908b8700 1 --2- 172.21.15.143:0/903612598 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f598c15aa30 0x7f598c1ef7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:43.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.981+0000 7f59908b8700 1 --2- 172.21.15.143:0/903612598 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f598c0b2210 0x7f598c15ae10 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:43.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.981+0000 7f59908b8700 1 --2- 172.21.15.143:0/903612598 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f598c0cbe20 0x7f598c0c58d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:43.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.981+0000 7f59908b8700 1 -- 172.21.15.143:0/903612598 >> 172.21.15.143:0/903612598 conn(0x7f598c099ab0 msgr2=0x7f598c06db40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:43.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.981+0000 7f59908b8700 1 -- 172.21.15.143:0/903612598 shutdown_connections 2023-11-08T19:26:43.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:43.981+0000 7f59908b8700 1 -- 172.21.15.143:0/903612598 wait complete. 2023-11-08T19:26:43.985 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 25 2023-11-08T19:26:43.995 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:26:44.115 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.114+0000 7efc18b31700 1 Processor -- start 2023-11-08T19:26:44.115 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.114+0000 7efc18b31700 1 -- start start 2023-11-08T19:26:44.118 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.115+0000 7efc18b31700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efc14159870 0x7efc140b2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:44.118 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.115+0000 7efc18b31700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7efc140cc410 0x7efc140cc820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:44.119 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.115+0000 7efc18b31700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc140b2a60 con 0x7efc1406ed60 2023-11-08T19:26:44.119 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.115+0000 7efc18b31700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc1406b090 con 0x7efc140cc410 2023-11-08T19:26:44.119 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.115+0000 7efc18b31700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc1406bbc0 con 0x7efc14159870 2023-11-08T19:26:44.119 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.115+0000 7efc12d9d700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7efc140cc410 0x7efc140cc820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:44.120 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.115+0000 7efc11d9b700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7efc1406ed60 0x7efc1406b4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:48284/0 (socket says 172.21.15.143:48284) 2023-11-08T19:26:44.120 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.115+0000 7efc11d9b700 1 -- 172.21.15.143:0/2014858815 learned_addr learned my addr 172.21.15.143:0/2014858815 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:26:44.120 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.115+0000 7efc1259c700 1 --2- 172.21.15.143:0/2014858815 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efc14159870 0x7efc140b2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:44.120 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.115+0000 7efc12d9d700 1 -- 172.21.15.143:0/2014858815 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efc14159870 msgr2=0x7efc140b2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:44.121 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.115+0000 7efc12d9d700 1 --2- 172.21.15.143:0/2014858815 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efc14159870 0x7efc140b2210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:44.121 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.115+0000 7efc12d9d700 1 -- 172.21.15.143:0/2014858815 >> v1:172.21.15.143:6789/0 conn(0x7efc1406ed60 legacy=0x7efc1406b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:44.121 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.115+0000 7efc12d9d700 1 -- 172.21.15.143:0/2014858815 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc140b2d40 con 0x7efc140cc410 2023-11-08T19:26:44.121 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.115+0000 7efc1159a700 1 -- 172.21.15.143:0/2014858815 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3936081192 0 0) 0x7efc140b2a60 con 0x7efc1406ed60 2023-11-08T19:26:44.122 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.115+0000 7efc12d9d700 1 --2- 172.21.15.143:0/2014858815 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efc140cc410 0x7efc140cc820 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7efc0c009d90 tx=0x7efc0c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c193f574ab7dd55 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:44.122 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.115+0000 7efc1159a700 1 -- 172.21.15.143:0/2014858815 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc0c00e070 con 0x7efc140cc410 2023-11-08T19:26:44.122 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.116+0000 7efc1159a700 1 -- 172.21.15.143:0/2014858815 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc0c00bbd0 con 0x7efc140cc410 2023-11-08T19:26:44.122 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.116+0000 7efc1159a700 1 -- 172.21.15.143:0/2014858815 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc0c012430 con 0x7efc140cc410 2023-11-08T19:26:44.123 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.116+0000 7efc18b31700 1 -- 172.21.15.143:0/2014858815 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efc140cc410 msgr2=0x7efc140cc820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:44.123 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.116+0000 7efc18b31700 1 --2- 172.21.15.143:0/2014858815 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efc140cc410 0x7efc140cc820 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7efc0c009d90 tx=0x7efc0c00b330 comp rx=0 tx=0).stop 2023-11-08T19:26:44.123 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.116+0000 7efc18b31700 1 -- 172.21.15.143:0/2014858815 shutdown_connections 2023-11-08T19:26:44.123 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.116+0000 7efc18b31700 1 --2- 172.21.15.143:0/2014858815 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efc140cc410 0x7efc140cc820 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:44.124 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.116+0000 7efc18b31700 1 --2- 172.21.15.143:0/2014858815 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efc14159870 0x7efc140b2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:44.124 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.116+0000 7efc18b31700 1 -- 172.21.15.143:0/2014858815 >> 172.21.15.143:0/2014858815 conn(0x7efc14099ab0 msgr2=0x7efc140d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:44.125 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.116+0000 7efc18b31700 1 -- 172.21.15.143:0/2014858815 shutdown_connections 2023-11-08T19:26:44.125 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.116+0000 7efc18b31700 1 -- 172.21.15.143:0/2014858815 wait complete. 2023-11-08T19:26:44.128 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.118+0000 7efc18b31700 1 Processor -- start 2023-11-08T19:26:44.129 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.118+0000 7efc18b31700 1 -- start start 2023-11-08T19:26:44.129 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.119+0000 7efc18b31700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efc1406ed60 0x7efc14157f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:44.129 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.119+0000 7efc18b31700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efc140cc410 0x7efc140c8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:44.130 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.119+0000 7efc18b31700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efc14159870 0x7efc140ad380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:44.130 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.119+0000 7efc18b31700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc141f11f0 con 0x7efc14159870 2023-11-08T19:26:44.130 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.119+0000 7efc18b31700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc140c77d0 con 0x7efc1406ed60 2023-11-08T19:26:44.130 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.119+0000 7efc18b31700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc141f1490 con 0x7efc140cc410 2023-11-08T19:26:44.131 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.119+0000 7efc12d9d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efc14159870 0x7efc140ad380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:44.131 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.119+0000 7efc12d9d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efc14159870 0x7efc140ad380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52138/0 (socket says 172.21.15.143:52138) 2023-11-08T19:26:44.131 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.119+0000 7efc12d9d700 1 -- 172.21.15.143:0/1137145476 learned_addr learned my addr 172.21.15.143:0/1137145476 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:44.131 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.119+0000 7efc1259c700 1 --2- 172.21.15.143:0/1137145476 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efc1406ed60 0x7efc14157f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:44.132 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.119+0000 7efc11d9b700 1 --2- 172.21.15.143:0/1137145476 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efc140cc410 0x7efc140c8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:44.132 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.119+0000 7efc12d9d700 1 -- 172.21.15.143:0/1137145476 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efc140cc410 msgr2=0x7efc140c8d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:44.132 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.119+0000 7efc12d9d700 1 --2- 172.21.15.143:0/1137145476 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efc140cc410 0x7efc140c8d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:44.132 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.119+0000 7efc12d9d700 1 -- 172.21.15.143:0/1137145476 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efc1406ed60 msgr2=0x7efc14157f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:44.133 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.119+0000 7efc12d9d700 1 --2- 172.21.15.143:0/1137145476 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efc1406ed60 0x7efc14157f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:44.133 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.119+0000 7efc12d9d700 1 -- 172.21.15.143:0/1137145476 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc0c009a70 con 0x7efc14159870 2023-11-08T19:26:44.133 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.119+0000 7efc1259c700 1 --2- 172.21.15.143:0/1137145476 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efc1406ed60 0x7efc14157f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:26:44.133 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.119+0000 7efc12d9d700 1 --2- 172.21.15.143:0/1137145476 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efc14159870 0x7efc140ad380 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7efc1406f210 tx=0x7efc0c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:44.134 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.119+0000 7efc02ffd700 1 -- 172.21.15.143:0/1137145476 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc0c012430 con 0x7efc14159870 2023-11-08T19:26:44.134 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.119+0000 7efc02ffd700 1 -- 172.21.15.143:0/1137145476 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc0c020070 con 0x7efc14159870 2023-11-08T19:26:44.134 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.119+0000 7efc18b31700 1 -- 172.21.15.143:0/1137145476 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc141f1770 con 0x7efc14159870 2023-11-08T19:26:44.134 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.120+0000 7efc02ffd700 1 -- 172.21.15.143:0/1137145476 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc0c004480 con 0x7efc14159870 2023-11-08T19:26:44.134 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.120+0000 7efc18b31700 1 -- 172.21.15.143:0/1137145476 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc141f1d30 con 0x7efc14159870 2023-11-08T19:26:44.135 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.120+0000 7efc18b31700 1 -- 172.21.15.143:0/1137145476 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc1404eee0 con 0x7efc14159870 2023-11-08T19:26:44.135 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.121+0000 7efc02ffd700 1 -- 172.21.15.143:0/1137145476 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7efc0c0045e0 con 0x7efc14159870 2023-11-08T19:26:44.135 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.121+0000 7efc02ffd700 1 --2- 172.21.15.143:0/1137145476 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7efbf406af00 0x7efbf406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:44.135 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.121+0000 7efc02ffd700 1 -- 172.21.15.143:0/1137145476 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6120+0+0 (secure 0 0 0) 0x7efc0c022070 con 0x7efc14159870 2023-11-08T19:26:44.136 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.121+0000 7efc1259c700 1 --2- 172.21.15.143:0/1137145476 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7efbf406af00 0x7efbf406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:44.136 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.122+0000 7efc1259c700 1 --2- 172.21.15.143:0/1137145476 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7efbf406af00 0x7efbf406d3c0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7efc04006fd0 tx=0x7efc04008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:44.136 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.126+0000 7efc02ffd700 1 -- 172.21.15.143:0/1137145476 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7efc0c056d10 con 0x7efc14159870 2023-11-08T19:26:44.358 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.357+0000 7efc18b31700 1 -- 172.21.15.143:0/1137145476 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efc1415e500 con 0x7efc14159870 2023-11-08T19:26:44.359 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.358+0000 7efc02ffd700 1 -- 172.21.15.143:0/1137145476 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4925 (secure 0 0 0) 0x7efc0c05a330 con 0x7efc14159870 2023-11-08T19:26:44.359 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:26:44.360 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4988,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3140991763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":3140991763},{"type":"v1","addr":"172.21.15.143:6835","nonce":3140991763}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":4997,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6836/2388011942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2388011942},{"type":"v1","addr":"172.21.15.174:6836","nonce":2388011942}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5000,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/1339111936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6835","nonce":1339111936},{"type":"v1","addr":"172.21.15.174:6837","nonce":1339111936}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T19:26:25.128617+0000","modified":"2023-11-08T19:26:28.156717+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5242},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5242":{"gid":5242,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.143:6834/3591097261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":3591097261},{"type":"v1","addr":"172.21.15.143:6834","nonce":3591097261}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-08T19:26:44.361 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.360+0000 7efc18b31700 1 -- 172.21.15.143:0/1137145476 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7efbf406af00 msgr2=0x7efbf406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:44.362 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.360+0000 7efc18b31700 1 --2- 172.21.15.143:0/1137145476 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7efbf406af00 0x7efbf406d3c0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7efc04006fd0 tx=0x7efc04008040 comp rx=0 tx=0).stop 2023-11-08T19:26:44.362 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.360+0000 7efc18b31700 1 -- 172.21.15.143:0/1137145476 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efc14159870 msgr2=0x7efc140ad380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:44.362 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.360+0000 7efc18b31700 1 --2- 172.21.15.143:0/1137145476 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efc14159870 0x7efc140ad380 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7efc1406f210 tx=0x7efc0c004ad0 comp rx=0 tx=0).stop 2023-11-08T19:26:44.362 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.360+0000 7efc18b31700 1 -- 172.21.15.143:0/1137145476 shutdown_connections 2023-11-08T19:26:44.362 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.360+0000 7efc18b31700 1 --2- 172.21.15.143:0/1137145476 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7efbf406af00 0x7efbf406d3c0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:44.363 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.360+0000 7efc18b31700 1 --2- 172.21.15.143:0/1137145476 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efc14159870 0x7efc140ad380 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:44.363 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.360+0000 7efc18b31700 1 --2- 172.21.15.143:0/1137145476 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efc1406ed60 0x7efc14157f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:44.363 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.360+0000 7efc18b31700 1 --2- 172.21.15.143:0/1137145476 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efc140cc410 0x7efc140c8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:44.363 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.360+0000 7efc18b31700 1 -- 172.21.15.143:0/1137145476 >> 172.21.15.143:0/1137145476 conn(0x7efc14099ab0 msgr2=0x7efc140b01d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:44.363 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.360+0000 7efc18b31700 1 -- 172.21.15.143:0/1137145476 shutdown_connections 2023-11-08T19:26:44.364 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.360+0000 7efc18b31700 1 -- 172.21.15.143:0/1137145476 wait complete. 2023-11-08T19:26:44.364 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 25 2023-11-08T19:26:44.373 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:26:44.493 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.492+0000 7f40da3ec700 1 Processor -- start 2023-11-08T19:26:44.494 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.492+0000 7f40da3ec700 1 -- start start 2023-11-08T19:26:44.496 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.493+0000 7f40da3ec700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f40d415dd00 0x7f40d40cca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:44.496 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.493+0000 7f40da3ec700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f40d41618f0 0x7f40d406ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:44.496 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.493+0000 7f40da3ec700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40d40ccfb0 con 0x7f40d406b150 2023-11-08T19:26:44.497 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.493+0000 7f40da3ec700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40d406b600 con 0x7f40d41618f0 2023-11-08T19:26:44.497 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.493+0000 7f40da3ec700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40d40cad50 con 0x7f40d415dd00 2023-11-08T19:26:44.497 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.493+0000 7f40d37fe700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f40d41618f0 0x7f40d406ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:44.497 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.493+0000 7f40d8989700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f40d406b150 0x7f40d40b2210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:48298/0 (socket says 172.21.15.143:48298) 2023-11-08T19:26:44.497 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.493+0000 7f40d8989700 1 -- 172.21.15.143:0/648728900 learned_addr learned my addr 172.21.15.143:0/648728900 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:26:44.498 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.493+0000 7f40d3fff700 1 --2- 172.21.15.143:0/648728900 >> v2:172.21.15.174:3300/0 conn(0x7f40d415dd00 0x7f40d40cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:44.498 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.493+0000 7f40d37fe700 1 -- 172.21.15.143:0/648728900 >> v2:172.21.15.174:3300/0 conn(0x7f40d415dd00 msgr2=0x7f40d40cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:44.498 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.493+0000 7f40d37fe700 1 --2- 172.21.15.143:0/648728900 >> v2:172.21.15.174:3300/0 conn(0x7f40d415dd00 0x7f40d40cca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:44.498 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.493+0000 7f40d37fe700 1 -- 172.21.15.143:0/648728900 >> v1:172.21.15.143:6789/0 conn(0x7f40d406b150 legacy=0x7f40d40b2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:44.498 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.493+0000 7f40d2ffd700 1 -- 172.21.15.143:0/648728900 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2251745415 0 0) 0x7f40d40ccfb0 con 0x7f40d406b150 2023-11-08T19:26:44.499 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.493+0000 7f40d37fe700 1 -- 172.21.15.143:0/648728900 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40d4161df0 con 0x7f40d41618f0 2023-11-08T19:26:44.499 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.493+0000 7f40d37fe700 1 --2- 172.21.15.143:0/648728900 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f40d41618f0 0x7f40d406ac10 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f40d4070a60 tx=0x7f40c4009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f21fd003c4a7f7a7 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:44.499 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.493+0000 7f40d2ffd700 1 -- 172.21.15.143:0/648728900 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40c4011830 con 0x7f40d41618f0 2023-11-08T19:26:44.499 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.494+0000 7f40d2ffd700 1 -- 172.21.15.143:0/648728900 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40c4011990 con 0x7f40d41618f0 2023-11-08T19:26:44.499 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.494+0000 7f40d2ffd700 1 -- 172.21.15.143:0/648728900 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40c4011d20 con 0x7f40d41618f0 2023-11-08T19:26:44.500 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.494+0000 7f40da3ec700 1 -- 172.21.15.143:0/648728900 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f40d41618f0 msgr2=0x7f40d406ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:44.500 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.494+0000 7f40da3ec700 1 --2- 172.21.15.143:0/648728900 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f40d41618f0 0x7f40d406ac10 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f40d4070a60 tx=0x7f40c4009b20 comp rx=0 tx=0).stop 2023-11-08T19:26:44.500 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.494+0000 7f40da3ec700 1 -- 172.21.15.143:0/648728900 shutdown_connections 2023-11-08T19:26:44.500 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.494+0000 7f40da3ec700 1 --2- 172.21.15.143:0/648728900 >> v2:172.21.15.174:3300/0 conn(0x7f40d415dd00 0x7f40d40cca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:44.500 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.494+0000 7f40da3ec700 1 --2- 172.21.15.143:0/648728900 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f40d41618f0 0x7f40d406ac10 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:44.501 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.494+0000 7f40da3ec700 1 -- 172.21.15.143:0/648728900 >> 172.21.15.143:0/648728900 conn(0x7f40d4099ab0 msgr2=0x7f40d40d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:44.501 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.494+0000 7f40da3ec700 1 -- 172.21.15.143:0/648728900 shutdown_connections 2023-11-08T19:26:44.501 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.494+0000 7f40da3ec700 1 -- 172.21.15.143:0/648728900 wait complete. 2023-11-08T19:26:44.504 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.495+0000 7f40da3ec700 1 Processor -- start 2023-11-08T19:26:44.504 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.496+0000 7f40da3ec700 1 -- start start 2023-11-08T19:26:44.504 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.496+0000 7f40da3ec700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f40d406b150 0x7f40d40c64b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:44.504 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.496+0000 7f40da3ec700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f40d415dd00 0x7f40d415a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:44.505 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.496+0000 7f40da3ec700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f40d41618f0 0x7f40d415a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:44.505 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.496+0000 7f40da3ec700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40d41f44c0 con 0x7f40d415dd00 2023-11-08T19:26:44.505 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.496+0000 7f40da3ec700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40d415ae60 con 0x7f40d406b150 2023-11-08T19:26:44.505 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.496+0000 7f40da3ec700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40d41f3840 con 0x7f40d41618f0 2023-11-08T19:26:44.505 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.496+0000 7f40d37fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f40d415dd00 0x7f40d415a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:44.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.496+0000 7f40d37fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f40d415dd00 0x7f40d415a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52146/0 (socket says 172.21.15.143:52146) 2023-11-08T19:26:44.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.496+0000 7f40d37fe700 1 -- 172.21.15.143:0/3901764997 learned_addr learned my addr 172.21.15.143:0/3901764997 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:44.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.497+0000 7f40d37fe700 1 -- 172.21.15.143:0/3901764997 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f40d41618f0 msgr2=0x7f40d415a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:44.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.497+0000 7f40d8989700 1 --2- 172.21.15.143:0/3901764997 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f40d41618f0 0x7f40d415a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:44.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.497+0000 7f40d3fff700 1 --2- 172.21.15.143:0/3901764997 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f40d406b150 0x7f40d40c64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:44.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.497+0000 7f40d37fe700 1 --2- 172.21.15.143:0/3901764997 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f40d41618f0 0x7f40d415a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:44.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.497+0000 7f40d37fe700 1 -- 172.21.15.143:0/3901764997 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f40d406b150 msgr2=0x7f40d40c64b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:44.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.497+0000 7f40d37fe700 1 --2- 172.21.15.143:0/3901764997 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f40d406b150 0x7f40d40c64b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:44.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.497+0000 7f40d37fe700 1 -- 172.21.15.143:0/3901764997 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40c40094a0 con 0x7f40d415dd00 2023-11-08T19:26:44.508 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.497+0000 7f40d3fff700 1 --2- 172.21.15.143:0/3901764997 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f40d406b150 0x7f40d40c64b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:26:44.508 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.497+0000 7f40d37fe700 1 --2- 172.21.15.143:0/3901764997 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f40d415dd00 0x7f40d415a3e0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f40c40090b0 tx=0x7f40c4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:44.508 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.497+0000 7f40d0ff9700 1 -- 172.21.15.143:0/3901764997 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40c40038a0 con 0x7f40d415dd00 2023-11-08T19:26:44.508 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.497+0000 7f40d0ff9700 1 -- 172.21.15.143:0/3901764997 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40c401f070 con 0x7f40d415dd00 2023-11-08T19:26:44.508 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.497+0000 7f40da3ec700 1 -- 172.21.15.143:0/3901764997 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40d41f3b20 con 0x7f40d415dd00 2023-11-08T19:26:44.509 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.497+0000 7f40d0ff9700 1 -- 172.21.15.143:0/3901764997 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40c4003c20 con 0x7f40d415dd00 2023-11-08T19:26:44.510 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.497+0000 7f40da3ec700 1 -- 172.21.15.143:0/3901764997 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40d41f40e0 con 0x7f40d415dd00 2023-11-08T19:26:44.510 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.498+0000 7f40da3ec700 1 -- 172.21.15.143:0/3901764997 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40d41f4600 con 0x7f40d415dd00 2023-11-08T19:26:44.510 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.498+0000 7f40d0ff9700 1 -- 172.21.15.143:0/3901764997 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f40c401a430 con 0x7f40d415dd00 2023-11-08T19:26:44.510 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.499+0000 7f40d0ff9700 1 --2- 172.21.15.143:0/3901764997 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f40b406af00 0x7f40b406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:44.511 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.499+0000 7f40d0ff9700 1 -- 172.21.15.143:0/3901764997 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f40c4021070 con 0x7f40d415dd00 2023-11-08T19:26:44.511 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.499+0000 7f40d3fff700 1 --2- 172.21.15.143:0/3901764997 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f40b406af00 0x7f40b406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:44.511 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.499+0000 7f40d3fff700 1 --2- 172.21.15.143:0/3901764997 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f40b406af00 0x7f40b406d3c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f40bc006fd0 tx=0x7f40bc008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:44.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.503+0000 7f40d0ff9700 1 -- 172.21.15.143:0/3901764997 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f40c40bdca0 con 0x7f40d415dd00 2023-11-08T19:26:44.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.694+0000 7f40da3ec700 1 -- 172.21.15.143:0/3901764997 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f40d41620a0 con 0x7f40d415dd00 2023-11-08T19:26:44.697 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.696+0000 7f40d0ff9700 1 -- 172.21.15.143:0/3901764997 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) v1 ==== 74+0+18466 (secure 0 0 0) 0x7f40d41f4600 con 0x7f40d415dd00 2023-11-08T19:26:44.697 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:26:44.698 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":50,"fsid":"968483af-77fc-4068-ad7c-5c2a1a4613d7","created":"2023-11-08T19:24:32.826484+0000","modified":"2023-11-08T19:26:39.794453+0000","last_up_change":"2023-11-08T19:24:43.573880+0000","last_in_change":"2023-11-08T19:24:35.314525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:44.551530+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"snap-test-pool","create_time":"2023-11-08T19:26:02.874384+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":36,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-11-08T19:26:22.869780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-11-08T19:26:24.093179+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bae98ec1-d94e-403d-a220-a522a53ea6f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6809","nonce":1487830566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6811","nonce":1487830566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6815","nonce":1487830566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6813","nonce":1487830566}]},"public_addr":"172.21.15.143:6809/1487830566","cluster_addr":"172.21.15.143:6811/1487830566","heartbeat_back_addr":"172.21.15.143:6815/1487830566","heartbeat_front_addr":"172.21.15.143:6813/1487830566","state":["exists","up"]},{"osd":1,"uuid":"5cb6538a-1ccb-4322-8633-09f66af0b7e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6801","nonce":787341397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6803","nonce":787341397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6807","nonce":787341397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6805","nonce":787341397}]},"public_addr":"172.21.15.143:6801/787341397","cluster_addr":"172.21.15.143:6803/787341397","heartbeat_back_addr":"172.21.15.143:6807/787341397","heartbeat_front_addr":"172.21.15.143:6805/787341397","state":["exists","up"]},{"osd":2,"uuid":"cdbf522c-60ef-4d60-9653-8dcb529c101a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6822","nonce":2301325597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6826","nonce":2301325597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6831","nonce":2301325597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6829","nonce":2301325597}]},"public_addr":"172.21.15.143:6822/2301325597","cluster_addr":"172.21.15.143:6826/2301325597","heartbeat_back_addr":"172.21.15.143:6831/2301325597","heartbeat_front_addr":"172.21.15.143:6829/2301325597","state":["exists","up"]},{"osd":3,"uuid":"4e2783cf-ebb9-4dd8-8245-bc1d17ce4325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6817","nonce":2021504428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6819","nonce":2021504428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6827","nonce":2021504428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6823","nonce":2021504428}]},"public_addr":"172.21.15.143:6817/2021504428","cluster_addr":"172.21.15.143:6819/2021504428","heartbeat_back_addr":"172.21.15.143:6827/2021504428","heartbeat_front_addr":"172.21.15.143:6823/2021504428","state":["exists","up"]},{"osd":4,"uuid":"691256fa-9b77-49b9-a346-7ae1adf0ca80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6801","nonce":1993380595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6803","nonce":1993380595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6807","nonce":1993380595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6805","nonce":1993380595}]},"public_addr":"172.21.15.174:6801/1993380595","cluster_addr":"172.21.15.174:6803/1993380595","heartbeat_back_addr":"172.21.15.174:6807/1993380595","heartbeat_front_addr":"172.21.15.174:6805/1993380595","state":["exists","up"]},{"osd":5,"uuid":"756fc352-48c9-46ce-b52e-0541fe6f8525","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6809","nonce":1777923303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6811","nonce":1777923303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6815","nonce":1777923303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6813","nonce":1777923303}]},"public_addr":"172.21.15.174:6809/1777923303","cluster_addr":"172.21.15.174:6811/1777923303","heartbeat_back_addr":"172.21.15.174:6815/1777923303","heartbeat_front_addr":"172.21.15.174:6813/1777923303","state":["exists","up"]},{"osd":6,"uuid":"fab2ac2b-ed97-4cde-9825-b480eabf8b8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6825","nonce":2755130316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6827","nonce":2755130316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6831","nonce":2755130316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6829","nonce":2755130316}]},"public_addr":"172.21.15.174:6825/2755130316","cluster_addr":"172.21.15.174:6827/2755130316","heartbeat_back_addr":"172.21.15.174:6831/2755130316","heartbeat_front_addr":"172.21.15.174:6829/2755130316","state":["exists","up"]},{"osd":7,"uuid":"3de9b797-2ab6-4d41-a863-7e7d638fd703","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6817","nonce":2847141411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6819","nonce":2847141411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6823","nonce":2847141411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6821","nonce":2847141411}]},"public_addr":"172.21.15.174:6817/2847141411","cluster_addr":"172.21.15.174:6819/2847141411","heartbeat_back_addr":"172.21.15.174:6823/2847141411","heartbeat_front_addr":"172.21.15.174:6821/2847141411","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.477076+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.496890+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.539305+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.454673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.538881+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.521294+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.520712+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.540404+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.1a","mappings":[{"from":0,"to":1}]},{"pgid":"8.28","mappings":[{"from":0,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:26:44.699 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.698+0000 7f40da3ec700 1 -- 172.21.15.143:0/3901764997 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f40b406af00 msgr2=0x7f40b406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:44.699 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.698+0000 7f40da3ec700 1 --2- 172.21.15.143:0/3901764997 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f40b406af00 0x7f40b406d3c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f40bc006fd0 tx=0x7f40bc008040 comp rx=0 tx=0).stop 2023-11-08T19:26:44.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.698+0000 7f40da3ec700 1 -- 172.21.15.143:0/3901764997 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f40d415dd00 msgr2=0x7f40d415a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:44.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.698+0000 7f40da3ec700 1 --2- 172.21.15.143:0/3901764997 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f40d415dd00 0x7f40d415a3e0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f40c40090b0 tx=0x7f40c4004750 comp rx=0 tx=0).stop 2023-11-08T19:26:44.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.698+0000 7f40da3ec700 1 -- 172.21.15.143:0/3901764997 shutdown_connections 2023-11-08T19:26:44.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.698+0000 7f40da3ec700 1 --2- 172.21.15.143:0/3901764997 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f40b406af00 0x7f40b406d3c0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:44.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.698+0000 7f40da3ec700 1 --2- 172.21.15.143:0/3901764997 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f40d41618f0 0x7f40d415a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:44.701 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.698+0000 7f40da3ec700 1 --2- 172.21.15.143:0/3901764997 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f40d406b150 0x7f40d40c64b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:44.701 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.698+0000 7f40da3ec700 1 --2- 172.21.15.143:0/3901764997 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f40d415dd00 0x7f40d415a3e0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:44.701 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.698+0000 7f40da3ec700 1 -- 172.21.15.143:0/3901764997 >> 172.21.15.143:0/3901764997 conn(0x7f40d4099ab0 msgr2=0x7f40d40cc700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:44.701 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.698+0000 7f40da3ec700 1 -- 172.21.15.143:0/3901764997 shutdown_connections 2023-11-08T19:26:44.701 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.698+0000 7f40da3ec700 1 -- 172.21.15.143:0/3901764997 wait complete. 2023-11-08T19:26:44.712 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-08T19:26:44.833 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.832+0000 7f55e2aa2700 1 Processor -- start 2023-11-08T19:26:44.834 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.832+0000 7f55e2aa2700 1 -- start start 2023-11-08T19:26:44.837 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.833+0000 7f55e2aa2700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f55dc1618f0 0x7f55dc06ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:44.837 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.833+0000 7f55e2aa2700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f55dc06b150 0x7f55dc0cb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:44.838 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.833+0000 7f55e2aa2700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55dc161df0 con 0x7f55dc15dd00 2023-11-08T19:26:44.838 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.833+0000 7f55e2aa2700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55dc06b600 con 0x7f55dc1618f0 2023-11-08T19:26:44.838 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.833+0000 7f55e2aa2700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55dc0cad50 con 0x7f55dc06b150 2023-11-08T19:26:44.838 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.833+0000 7f55e083e700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f55dc15dd00 0x7f55dc0cca70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:48308/0 (socket says 172.21.15.143:48308) 2023-11-08T19:26:44.839 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.833+0000 7f55e083e700 1 -- 172.21.15.143:0/1399881996 learned_addr learned my addr 172.21.15.143:0/1399881996 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:26:44.839 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.833+0000 7f55e103f700 1 --2- 172.21.15.143:0/1399881996 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f55dc06b150 0x7f55dc0cb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:44.839 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.833+0000 7f55db7fe700 1 -- 172.21.15.143:0/1399881996 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1206862475 0 0) 0x7f55dc161df0 con 0x7f55dc15dd00 2023-11-08T19:26:44.839 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.833+0000 7f55dbfff700 1 --2- 172.21.15.143:0/1399881996 >> v2:172.21.15.174:3300/0 conn(0x7f55dc1618f0 0x7f55dc06ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:44.839 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.833+0000 7f55db7fe700 1 -- 172.21.15.143:0/1399881996 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f55cc0035b0 con 0x7f55dc15dd00 2023-11-08T19:26:44.839 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.834+0000 7f55db7fe700 1 -- 172.21.15.143:0/1399881996 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2240660333 0 0) 0x7f55cc0035b0 con 0x7f55dc15dd00 2023-11-08T19:26:44.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.834+0000 7f55db7fe700 1 -- 172.21.15.143:0/1399881996 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f55dc06b150 msgr2=0x7f55dc0cb1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:44.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.834+0000 7f55db7fe700 1 --2- 172.21.15.143:0/1399881996 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f55dc06b150 0x7f55dc0cb1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:44.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.834+0000 7f55db7fe700 1 -- 172.21.15.143:0/1399881996 >> v2:172.21.15.174:3300/0 conn(0x7f55dc1618f0 msgr2=0x7f55dc06ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:44.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.834+0000 7f55db7fe700 1 --2- 172.21.15.143:0/1399881996 >> v2:172.21.15.174:3300/0 conn(0x7f55dc1618f0 0x7f55dc06ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:44.841 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.834+0000 7f55db7fe700 1 -- 172.21.15.143:0/1399881996 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55dc1620d0 con 0x7f55dc15dd00 2023-11-08T19:26:44.841 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.834+0000 7f55db7fe700 1 -- 172.21.15.143:0/1399881996 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4147756303 0 0) 0x7f55d00017f0 con 0x7f55dc15dd00 2023-11-08T19:26:44.841 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.834+0000 7f55db7fe700 1 -- 172.21.15.143:0/1399881996 >> v1:172.21.15.143:6789/0 conn(0x7f55dc15dd00 legacy=0x7f55dc0cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:44.841 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.834+0000 7f55db7fe700 1 --2- 172.21.15.143:0/1399881996 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f55dc06b150 0x7f55cc003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:44.842 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.834+0000 7f55db7fe700 1 --2- 172.21.15.143:0/1399881996 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f55cc003f80 0x7f55cc004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:44.842 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.834+0000 7f55db7fe700 1 --2- 172.21.15.143:0/1399881996 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f55cc004d90 0x7f55cc005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:44.842 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.834+0000 7f55db7fe700 1 -- 172.21.15.143:0/1399881996 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55dc06b600 con 0x7f55cc003f80 2023-11-08T19:26:44.842 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.834+0000 7f55db7fe700 1 -- 172.21.15.143:0/1399881996 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55cc00ba10 con 0x7f55cc004d90 2023-11-08T19:26:44.842 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.834+0000 7f55db7fe700 1 -- 172.21.15.143:0/1399881996 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55cc00bb50 con 0x7f55dc06b150 2023-11-08T19:26:44.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.834+0000 7f55dbfff700 1 --2- 172.21.15.143:0/1399881996 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f55cc003f80 0x7f55cc004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:44.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.834+0000 7f55e083e700 1 --2- 172.21.15.143:0/1399881996 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f55dc06b150 0x7f55cc003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:44.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.834+0000 7f55e103f700 1 --2- 172.21.15.143:0/1399881996 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f55cc004d90 0x7f55cc005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:44.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.834+0000 7f55dbfff700 1 -- 172.21.15.143:0/1399881996 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f55dc06b150 msgr2=0x7f55cc003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:44.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.834+0000 7f55dbfff700 1 --2- 172.21.15.143:0/1399881996 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f55dc06b150 0x7f55cc003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:44.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.834+0000 7f55dbfff700 1 -- 172.21.15.143:0/1399881996 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f55cc004d90 msgr2=0x7f55cc005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:44.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.834+0000 7f55dbfff700 1 --2- 172.21.15.143:0/1399881996 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f55cc004d90 0x7f55cc005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:44.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.834+0000 7f55dbfff700 1 -- 172.21.15.143:0/1399881996 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55cc0035b0 con 0x7f55cc003f80 2023-11-08T19:26:44.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.834+0000 7f55e103f700 1 --2- 172.21.15.143:0/1399881996 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f55cc004d90 0x7f55cc005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:26:44.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.834+0000 7f55dbfff700 1 --2- 172.21.15.143:0/1399881996 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f55cc003f80 0x7f55cc004850 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f55c8009b60 tx=0x7f55cc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82eb6b3acfee403e server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:44.845 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.834+0000 7f55db7fe700 1 -- 172.21.15.143:0/1399881996 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55c800e660 con 0x7f55cc003f80 2023-11-08T19:26:44.845 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.834+0000 7f55db7fe700 1 -- 172.21.15.143:0/1399881996 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55c800e7c0 con 0x7f55cc003f80 2023-11-08T19:26:44.845 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.834+0000 7f55db7fe700 1 -- 172.21.15.143:0/1399881996 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55c800eb30 con 0x7f55cc003f80 2023-11-08T19:26:44.845 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.835+0000 7f55e2aa2700 1 -- 172.21.15.143:0/1399881996 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f55cc003f80 msgr2=0x7f55cc004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:44.845 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.835+0000 7f55e2aa2700 1 --2- 172.21.15.143:0/1399881996 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f55cc003f80 0x7f55cc004850 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f55c8009b60 tx=0x7f55cc005880 comp rx=0 tx=0).stop 2023-11-08T19:26:44.846 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.835+0000 7f55e103f700 1 -- 172.21.15.143:0/1399881996 reap_dead start 2023-11-08T19:26:44.846 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.835+0000 7f55e2aa2700 1 -- 172.21.15.143:0/1399881996 shutdown_connections 2023-11-08T19:26:44.846 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.835+0000 7f55e2aa2700 1 -- 172.21.15.143:0/1399881996 >> 172.21.15.143:0/1399881996 conn(0x7f55dc099ab0 msgr2=0x7f55dc0d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:44.846 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.835+0000 7f55e2aa2700 1 -- 172.21.15.143:0/1399881996 shutdown_connections 2023-11-08T19:26:44.846 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.835+0000 7f55e2aa2700 1 -- 172.21.15.143:0/1399881996 wait complete. 2023-11-08T19:26:44.847 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.836+0000 7f55e2aa2700 1 Processor -- start 2023-11-08T19:26:44.847 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.836+0000 7f55e2aa2700 1 -- start start 2023-11-08T19:26:44.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.837+0000 7f55e2aa2700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f55cc003f80 0x7f55dc0c6440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:44.850 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.837+0000 7f55e2aa2700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f55dc0c5500 0x7f55dc0b51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:44.850 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.837+0000 7f55e2aa2700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f55dc0c5980 0x7f55dc0b5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:44.850 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.837+0000 7f55e2aa2700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55dc0c5100 con 0x7f55dc0c5980 2023-11-08T19:26:44.850 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.837+0000 7f55e2aa2700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55dc0c4230 con 0x7f55dc0c5500 2023-11-08T19:26:44.850 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.837+0000 7f55e2aa2700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55dc0c6980 con 0x7f55cc003f80 2023-11-08T19:26:44.851 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.837+0000 7f55e103f700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f55dc0c5980 0x7f55dc0b5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:44.851 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.837+0000 7f55e103f700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f55dc0c5980 0x7f55dc0b5710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52170/0 (socket says 172.21.15.143:52170) 2023-11-08T19:26:44.851 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.837+0000 7f55e103f700 1 -- 172.21.15.143:0/1717422198 learned_addr learned my addr 172.21.15.143:0/1717422198 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:44.851 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.837+0000 7f55dbfff700 1 --2- 172.21.15.143:0/1717422198 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f55dc0c5500 0x7f55dc0b51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:44.851 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.837+0000 7f55e083e700 1 --2- 172.21.15.143:0/1717422198 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f55cc003f80 0x7f55dc0c6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:44.852 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.837+0000 7f55e103f700 1 -- 172.21.15.143:0/1717422198 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f55cc003f80 msgr2=0x7f55dc0c6440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:44.852 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.837+0000 7f55e103f700 1 --2- 172.21.15.143:0/1717422198 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f55cc003f80 0x7f55dc0c6440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:44.852 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.837+0000 7f55e103f700 1 -- 172.21.15.143:0/1717422198 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f55dc0c5500 msgr2=0x7f55dc0b51d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:44.852 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.837+0000 7f55e103f700 1 --2- 172.21.15.143:0/1717422198 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f55dc0c5500 0x7f55dc0b51d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:44.852 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.837+0000 7f55e103f700 1 -- 172.21.15.143:0/1717422198 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55dc0c4480 con 0x7f55dc0c5980 2023-11-08T19:26:44.852 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.837+0000 7f55e083e700 1 --2- 172.21.15.143:0/1717422198 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f55cc003f80 0x7f55dc0c6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:26:44.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.837+0000 7f55e103f700 1 --2- 172.21.15.143:0/1717422198 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f55dc0c5980 0x7f55dc0b5710 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f55dc072540 tx=0x7f55d40104a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:44.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.838+0000 7f55d97fa700 1 -- 172.21.15.143:0/1717422198 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55d400e050 con 0x7f55dc0c5980 2023-11-08T19:26:44.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.838+0000 7f55d97fa700 1 -- 172.21.15.143:0/1717422198 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55d400f040 con 0x7f55dc0c5980 2023-11-08T19:26:44.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.838+0000 7f55e2aa2700 1 -- 172.21.15.143:0/1717422198 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55dc0c4760 con 0x7f55dc0c5980 2023-11-08T19:26:44.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.838+0000 7f55d97fa700 1 -- 172.21.15.143:0/1717422198 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55d4007ea0 con 0x7f55dc0c5980 2023-11-08T19:26:44.854 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.838+0000 7f55e2aa2700 1 -- 172.21.15.143:0/1717422198 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55dc0c4d20 con 0x7f55dc0c5980 2023-11-08T19:26:44.854 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.839+0000 7f55e2aa2700 1 -- 172.21.15.143:0/1717422198 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55dc04eee0 con 0x7f55dc0c5980 2023-11-08T19:26:44.854 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.839+0000 7f55d97fa700 1 -- 172.21.15.143:0/1717422198 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f55d4015440 con 0x7f55dc0c5980 2023-11-08T19:26:44.854 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.839+0000 7f55d97fa700 1 --2- 172.21.15.143:0/1717422198 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f55c006af00 0x7f55c006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:44.854 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.839+0000 7f55d97fa700 1 -- 172.21.15.143:0/1717422198 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f55d4089ab0 con 0x7f55dc0c5980 2023-11-08T19:26:44.855 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.840+0000 7f55e083e700 1 --2- 172.21.15.143:0/1717422198 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f55c006af00 0x7f55c006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:44.855 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.840+0000 7f55e083e700 1 --2- 172.21.15.143:0/1717422198 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f55c006af00 0x7f55c006d3c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f55d00011d0 tx=0x7f55d0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:44.855 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:44.844+0000 7f55d97fa700 1 -- 172.21.15.143:0/1717422198 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f55d4054a50 con 0x7f55dc0c5980 2023-11-08T19:26:45.076 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:45.074+0000 7f55e2aa2700 1 -- 172.21.15.143:0/1717422198 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f55dc06b920 con 0x7f55dc0c5980 2023-11-08T19:26:45.076 INFO:tasks.ceph.mon.a.smithi143.stderr:2023-11-08T19:26:45.075+0000 7f65b584e700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-08T19:26:45.080 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:45.079+0000 7f55d97fa700 1 -- 172.21.15.143:0/1717422198 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v26) v1 ==== 161+0+0 (secure 0 0 0) 0x7f55d4058070 con 0x7f55dc0c5980 2023-11-08T19:26:45.082 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:45.081+0000 7f55e2aa2700 1 -- 172.21.15.143:0/1717422198 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f55c006af00 msgr2=0x7f55c006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:45.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:45.081+0000 7f55e2aa2700 1 --2- 172.21.15.143:0/1717422198 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f55c006af00 0x7f55c006d3c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f55d00011d0 tx=0x7f55d0009040 comp rx=0 tx=0).stop 2023-11-08T19:26:45.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:45.081+0000 7f55e2aa2700 1 -- 172.21.15.143:0/1717422198 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f55dc0c5980 msgr2=0x7f55dc0b5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:45.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:45.081+0000 7f55e2aa2700 1 --2- 172.21.15.143:0/1717422198 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f55dc0c5980 0x7f55dc0b5710 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f55dc072540 tx=0x7f55d40104a0 comp rx=0 tx=0).stop 2023-11-08T19:26:45.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:45.081+0000 7f55e2aa2700 1 -- 172.21.15.143:0/1717422198 shutdown_connections 2023-11-08T19:26:45.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:45.081+0000 7f55e2aa2700 1 --2- 172.21.15.143:0/1717422198 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f55c006af00 0x7f55c006d3c0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:45.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:45.081+0000 7f55e2aa2700 1 --2- 172.21.15.143:0/1717422198 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f55cc003f80 0x7f55dc0c6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:45.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:45.081+0000 7f55e2aa2700 1 --2- 172.21.15.143:0/1717422198 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f55dc0c5980 0x7f55dc0b5710 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:45.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:45.081+0000 7f55e2aa2700 1 --2- 172.21.15.143:0/1717422198 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f55dc0c5500 0x7f55dc0b51d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:45.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:45.081+0000 7f55e2aa2700 1 -- 172.21.15.143:0/1717422198 >> 172.21.15.143:0/1717422198 conn(0x7f55dc099ab0 msgr2=0x7f55dc0ae790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:45.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:45.081+0000 7f55e2aa2700 1 -- 172.21.15.143:0/1717422198 shutdown_connections 2023-11-08T19:26:45.085 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:45.081+0000 7f55e2aa2700 1 -- 172.21.15.143:0/1717422198 wait complete. 2023-11-08T19:26:45.085 INFO:teuthology.orchestra.run.smithi143.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-08T19:26:45.089 INFO:tasks.ceph.mds.c.smithi143.stderr: -2919> 2023-11-08T19:26:31.708+0000 7f38b102d700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-08T19:26:45.099 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-08T19:26:45.120 INFO:tasks.ceph.mds.c.smithi143.stdout:starting mds.c at 2023-11-08T19:26:45.222 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:45.220+0000 7f81a3471700 1 Processor -- start 2023-11-08T19:26:45.222 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:45.220+0000 7f81a3471700 1 -- start start 2023-11-08T19:26:45.225 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:45.221+0000 7f81a3471700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f819c15dba0 0x7f819c06c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:45.225 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:45.221+0000 7f81a3471700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f819c06b240 0x7f819c0caff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:45.225 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:45.221+0000 7f81a3471700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f819c161890 con 0x7f819c0cb530 2023-11-08T19:26:45.225 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:45.221+0000 7f81a3471700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f819c06b740 con 0x7f819c15dba0 2023-11-08T19:26:45.226 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:45.221+0000 7f81a3471700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f819c06ba20 con 0x7f819c06b240 2023-11-08T19:26:45.226 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:45.221+0000 7f81a120d700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f819c15dba0 0x7f819c06c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:45.226 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:45.221+0000 7f81a120d700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f819c15dba0 0x7f819c06c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52192/0 (socket says 172.21.15.143:52192) 2023-11-08T19:26:45.226 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:45.221+0000 7f81a120d700 1 -- 172.21.15.143:0/369365817 learned_addr learned my addr 172.21.15.143:0/369365817 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:45.227 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:45.222+0000 7f81a0a0c700 1 --2- 172.21.15.143:0/369365817 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f819c06b240 0x7f819c0caff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:45.227 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:45.222+0000 7f81a120d700 1 -- 172.21.15.143:0/369365817 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f819c06b240 msgr2=0x7f819c0caff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:45.227 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:45.222+0000 7f81a120d700 1 --2- 172.21.15.143:0/369365817 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f819c06b240 0x7f819c0caff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:45.227 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:45.222+0000 7f81a120d700 1 -- 172.21.15.143:0/369365817 >> v1:172.21.15.174:6789/0 conn(0x7f819c0cb530 legacy=0x7f819c0b22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:45.228 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:45.222+0000 7f81a120d700 1 -- 172.21.15.143:0/369365817 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f819c06bd00 con 0x7f819c15dba0 2023-11-08T19:26:45.228 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:45.222+0000 7f81a120d700 1 --2- 172.21.15.143:0/369365817 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f819c15dba0 0x7f819c06c610 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f818c0098f0 tx=0x7f818c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f00d5558368a7bce server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:45.228 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:45.222+0000 7f8193fff700 1 -- 172.21.15.143:0/369365817 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f818c011830 con 0x7f819c15dba0 2023-11-08T19:26:45.228 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:45.222+0000 7f8193fff700 1 -- 172.21.15.143:0/369365817 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f818c011990 con 0x7f819c15dba0 2023-11-08T19:26:45.229 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:45.222+0000 7f8193fff700 1 -- 172.21.15.143:0/369365817 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f818c011d20 con 0x7f819c15dba0 2023-11-08T19:26:45.229 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:45.222+0000 7f81a3471700 1 -- 172.21.15.143:0/369365817 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f819c15dba0 msgr2=0x7f819c06c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:45.229 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:45.222+0000 7f81a3471700 1 --2- 172.21.15.143:0/369365817 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f819c15dba0 0x7f819c06c610 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f818c0098f0 tx=0x7f818c009c00 comp rx=0 tx=0).stop 2023-11-08T19:26:45.229 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:45.223+0000 7f81a3471700 1 -- 172.21.15.143:0/369365817 shutdown_connections 2023-11-08T19:26:45.230 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:45.223+0000 7f81a3471700 1 --2- 172.21.15.143:0/369365817 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f819c15dba0 0x7f819c06c610 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:45.230 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:45.223+0000 7f81a3471700 1 --2- 172.21.15.143:0/369365817 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f819c06b240 0x7f819c0caff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:45.230 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:45.223+0000 7f81a3471700 1 -- 172.21.15.143:0/369365817 >> 172.21.15.143:0/369365817 conn(0x7f819c0cd2e0 msgr2=0x7f819c0cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:45.230 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:45.223+0000 7f81a3471700 1 -- 172.21.15.143:0/369365817 shutdown_connections 2023-11-08T19:26:45.230 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:45.223+0000 7f81a3471700 1 -- 172.21.15.143:0/369365817 wait complete. 2023-11-08T19:26:45.231 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:45.224+0000 7f81a3471700 1 Processor -- start 2023-11-08T19:26:45.231 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:45.224+0000 7f81a3471700 1 -- start start 2023-11-08T19:26:45.234 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:45.224+0000 7f81a3471700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f819c06b240 0x7f819c151c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:45.235 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:45.225+0000 7f81a3471700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f819c0cb530 0x7f819c0c7930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:45.235 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:45.225+0000 7f81a3471700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f819c15dba0 0x7f819c0c8b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:45.235 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:45.225+0000 7f81a3471700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f819c0c81a0 con 0x7f819c15dba0 2023-11-08T19:26:45.235 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:45.225+0000 7f81a3471700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f819c0c8480 con 0x7f819c0cb530 2023-11-08T19:26:45.236 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:45.225+0000 7f81a3471700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f819c0c8760 con 0x7f819c06b240 2023-11-08T19:26:45.236 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:45.225+0000 7f81a1a0e700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f819c15dba0 0x7f819c0c8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:45.236 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:45.225+0000 7f81a1a0e700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f819c15dba0 0x7f819c0c8b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52202/0 (socket says 172.21.15.143:52202) 2023-11-08T19:26:45.236 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:45.225+0000 7f81a1a0e700 1 -- 172.21.15.143:0/826376654 learned_addr learned my addr 172.21.15.143:0/826376654 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:45.236 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:45.225+0000 7f81a120d700 1 --2- 172.21.15.143:0/826376654 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f819c06b240 0x7f819c151c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:45.237 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:45.225+0000 7f81a0a0c700 1 --2- 172.21.15.143:0/826376654 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f819c0cb530 0x7f819c0c7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:45.237 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:45.225+0000 7f81a1a0e700 1 -- 172.21.15.143:0/826376654 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f819c06b240 msgr2=0x7f819c151c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:45.237 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:45.225+0000 7f81a1a0e700 1 --2- 172.21.15.143:0/826376654 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f819c06b240 0x7f819c151c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:45.237 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:45.225+0000 7f81a1a0e700 1 -- 172.21.15.143:0/826376654 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f819c0cb530 msgr2=0x7f819c0c7930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:45.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:45.225+0000 7f81a1a0e700 1 --2- 172.21.15.143:0/826376654 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f819c0cb530 0x7f819c0c7930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:45.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:45.225+0000 7f81a1a0e700 1 -- 172.21.15.143:0/826376654 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f818c009550 con 0x7f819c15dba0 2023-11-08T19:26:45.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:45.225+0000 7f81a1a0e700 1 --2- 172.21.15.143:0/826376654 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f819c15dba0 0x7f819c0c8b10 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f819400cbc0 tx=0x7f819400ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:45.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:45.225+0000 7f8191ffb700 1 -- 172.21.15.143:0/826376654 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8194003c00 con 0x7f819c15dba0 2023-11-08T19:26:45.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:45.225+0000 7f8191ffb700 1 -- 172.21.15.143:0/826376654 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8194003d60 con 0x7f819c15dba0 2023-11-08T19:26:45.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:45.225+0000 7f8191ffb700 1 -- 172.21.15.143:0/826376654 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f819400aae0 con 0x7f819c15dba0 2023-11-08T19:26:45.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:45.225+0000 7f81a3471700 1 -- 172.21.15.143:0/826376654 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f819c0acc90 con 0x7f819c15dba0 2023-11-08T19:26:45.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:45.226+0000 7f81a3471700 1 -- 172.21.15.143:0/826376654 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f819c0c7e70 con 0x7f819c15dba0 2023-11-08T19:26:45.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:45.226+0000 7f81a3471700 1 -- 172.21.15.143:0/826376654 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f819c0ad1b0 con 0x7f819c15dba0 2023-11-08T19:26:45.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:45.227+0000 7f8191ffb700 1 -- 172.21.15.143:0/826376654 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f819400ac40 con 0x7f819c15dba0 2023-11-08T19:26:45.240 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:45.227+0000 7f8191ffb700 1 --2- 172.21.15.143:0/826376654 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f818006af00 0x7f818006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:45.240 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:45.227+0000 7f8191ffb700 1 -- 172.21.15.143:0/826376654 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6206+0+0 (secure 0 0 0) 0x7f81940896f0 con 0x7f819c15dba0 2023-11-08T19:26:45.240 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:45.227+0000 7f81a120d700 1 --2- 172.21.15.143:0/826376654 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f818006af00 0x7f818006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:45.240 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:45.228+0000 7f81a120d700 1 --2- 172.21.15.143:0/826376654 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f818006af00 0x7f818006d3c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f818c014040 tx=0x7f818c008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:45.241 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:45.232+0000 7f8191ffb700 1 -- 172.21.15.143:0/826376654 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8194054640 con 0x7f819c15dba0 2023-11-08T19:26:45.465 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:45.463+0000 7f81a3471700 1 -- 172.21.15.143:0/826376654 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f819c04eee0 con 0x7f819c15dba0 2023-11-08T19:26:46.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:46.083+0000 7f8191ffb700 1 -- 172.21.15.143:0/826376654 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v27) v1 ==== 104+0+0 (secure 0 0 0) 0x7f8194057c60 con 0x7f819c15dba0 2023-11-08T19:26:46.088 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:46.085+0000 7f81a3471700 1 -- 172.21.15.143:0/826376654 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f818006af00 msgr2=0x7f818006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:46.088 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:46.085+0000 7f81a3471700 1 --2- 172.21.15.143:0/826376654 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f818006af00 0x7f818006d3c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f818c014040 tx=0x7f818c008030 comp rx=0 tx=0).stop 2023-11-08T19:26:46.088 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:46.085+0000 7f81a3471700 1 -- 172.21.15.143:0/826376654 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f819c15dba0 msgr2=0x7f819c0c8b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:46.089 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:46.085+0000 7f81a3471700 1 --2- 172.21.15.143:0/826376654 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f819c15dba0 0x7f819c0c8b10 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f819400cbc0 tx=0x7f819400ced0 comp rx=0 tx=0).stop 2023-11-08T19:26:46.089 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:46.085+0000 7f81a3471700 1 -- 172.21.15.143:0/826376654 shutdown_connections 2023-11-08T19:26:46.089 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:46.085+0000 7f81a3471700 1 --2- 172.21.15.143:0/826376654 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f818006af00 0x7f818006d3c0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:46.089 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:46.085+0000 7f81a3471700 1 --2- 172.21.15.143:0/826376654 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f819c06b240 0x7f819c151c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:46.089 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:46.085+0000 7f81a3471700 1 --2- 172.21.15.143:0/826376654 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f819c15dba0 0x7f819c0c8b10 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:46.090 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:46.085+0000 7f81a3471700 1 --2- 172.21.15.143:0/826376654 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f819c0cb530 0x7f819c0c7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:46.090 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:46.085+0000 7f81a3471700 1 -- 172.21.15.143:0/826376654 >> 172.21.15.143:0/826376654 conn(0x7f819c0cd2e0 msgr2=0x7f819c15ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:46.090 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:46.086+0000 7f81a3471700 1 -- 172.21.15.143:0/826376654 shutdown_connections 2023-11-08T19:26:46.090 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:46.086+0000 7f81a3471700 1 -- 172.21.15.143:0/826376654 wait complete. 2023-11-08T19:26:46.101 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-08T19:26:46.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:46.267+0000 7ff037cf0700 1 Processor -- start 2023-11-08T19:26:46.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:46.268+0000 7ff037cf0700 1 -- start start 2023-11-08T19:26:46.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:46.268+0000 7ff037cf0700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff03015a9e0 0x7ff0300b2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:46.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:46.268+0000 7ff037cf0700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff0300b2840 0x7ff0300af760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:46.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:46.268+0000 7ff037cf0700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7ff0300afca0 0x7ff0300cc1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:46.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:46.268+0000 7ff037cf0700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff030073020 con 0x7ff0300b2840 2023-11-08T19:26:46.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:46.268+0000 7ff037cf0700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0300b0110 con 0x7ff0300afca0 2023-11-08T19:26:46.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:46.269+0000 7ff037cf0700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0300b2dc0 con 0x7ff03015a9e0 2023-11-08T19:26:46.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:46.269+0000 7ff03528b700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff0300b2840 0x7ff0300af760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:46.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:46.269+0000 7ff03528b700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff0300b2840 0x7ff0300af760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52210/0 (socket says 172.21.15.143:52210) 2023-11-08T19:26:46.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:46.269+0000 7ff03528b700 1 -- 172.21.15.143:0/1974545743 learned_addr learned my addr 172.21.15.143:0/1974545743 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:46.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:46.269+0000 7ff03628d700 1 --2- 172.21.15.143:0/1974545743 >> v2:172.21.15.174:3300/0 conn(0x7ff0300afca0 0x7ff0300cc1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:46.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:46.269+0000 7ff035a8c700 1 --2- 172.21.15.143:0/1974545743 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff03015a9e0 0x7ff0300b2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:46.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:46.269+0000 7ff03528b700 1 -- 172.21.15.143:0/1974545743 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff03015a9e0 msgr2=0x7ff0300b2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:46.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:46.269+0000 7ff03528b700 1 --2- 172.21.15.143:0/1974545743 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff03015a9e0 0x7ff0300b2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:46.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:46.269+0000 7ff03528b700 1 -- 172.21.15.143:0/1974545743 >> v2:172.21.15.174:3300/0 conn(0x7ff0300afca0 msgr2=0x7ff0300cc1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:46.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:46.269+0000 7ff03528b700 1 --2- 172.21.15.143:0/1974545743 >> v2:172.21.15.174:3300/0 conn(0x7ff0300afca0 0x7ff0300cc1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:46.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:46.269+0000 7ff03528b700 1 -- 172.21.15.143:0/1974545743 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff030072260 con 0x7ff0300b2840 2023-11-08T19:26:46.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:46.269+0000 7ff03628d700 1 --2- 172.21.15.143:0/1974545743 >> v2:172.21.15.174:3300/0 conn(0x7ff0300afca0 0x7ff0300cc1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:26:46.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:46.269+0000 7ff035a8c700 1 --2- 172.21.15.143:0/1974545743 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff03015a9e0 0x7ff0300b2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:26:46.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:46.269+0000 7ff03528b700 1 --2- 172.21.15.143:0/1974545743 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff0300b2840 0x7ff0300af760 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7ff03015e530 tx=0x7ff02c00be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df691d9a5e8c13d0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:46.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:46.269+0000 7ff034a8a700 1 -- 172.21.15.143:0/1974545743 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff02c00dd30 con 0x7ff0300b2840 2023-11-08T19:26:46.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:46.269+0000 7ff034a8a700 1 -- 172.21.15.143:0/1974545743 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff02c00de90 con 0x7ff0300b2840 2023-11-08T19:26:46.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:46.269+0000 7ff034a8a700 1 -- 172.21.15.143:0/1974545743 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff02c010690 con 0x7ff0300b2840 2023-11-08T19:26:46.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:46.270+0000 7ff037cf0700 1 -- 172.21.15.143:0/1974545743 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff0300b2840 msgr2=0x7ff0300af760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:46.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:46.270+0000 7ff037cf0700 1 --2- 172.21.15.143:0/1974545743 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff0300b2840 0x7ff0300af760 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7ff03015e530 tx=0x7ff02c00be70 comp rx=0 tx=0).stop 2023-11-08T19:26:46.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:46.270+0000 7ff037cf0700 1 -- 172.21.15.143:0/1974545743 shutdown_connections 2023-11-08T19:26:46.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:46.270+0000 7ff037cf0700 1 --2- 172.21.15.143:0/1974545743 >> v2:172.21.15.174:3300/0 conn(0x7ff0300afca0 0x7ff0300cc1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:46.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:46.270+0000 7ff037cf0700 1 --2- 172.21.15.143:0/1974545743 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff03015a9e0 0x7ff0300b2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:46.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:46.270+0000 7ff037cf0700 1 --2- 172.21.15.143:0/1974545743 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff0300b2840 0x7ff0300af760 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:46.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:46.270+0000 7ff037cf0700 1 -- 172.21.15.143:0/1974545743 >> 172.21.15.143:0/1974545743 conn(0x7ff0300cd300 msgr2=0x7ff0300cd710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:46.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:46.270+0000 7ff037cf0700 1 -- 172.21.15.143:0/1974545743 shutdown_connections 2023-11-08T19:26:46.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:46.270+0000 7ff037cf0700 1 -- 172.21.15.143:0/1974545743 wait complete. 2023-11-08T19:26:46.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:46.271+0000 7ff037cf0700 1 Processor -- start 2023-11-08T19:26:46.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:46.271+0000 7ff037cf0700 1 -- start start 2023-11-08T19:26:46.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:46.272+0000 7ff037cf0700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff0300afca0 0x7ff030154f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:46.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:46.272+0000 7ff037cf0700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff0300b2840 0x7ff0300b34a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:46.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:46.272+0000 7ff037cf0700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff03015a9e0 0x7ff0300b39e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:46.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:46.272+0000 7ff037cf0700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0300c5910 con 0x7ff03015a9e0 2023-11-08T19:26:46.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:46.272+0000 7ff037cf0700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0301f2d20 con 0x7ff0300b2840 2023-11-08T19:26:46.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:46.272+0000 7ff037cf0700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0300b6780 con 0x7ff0300afca0 2023-11-08T19:26:46.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:46.272+0000 7ff03628d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff03015a9e0 0x7ff0300b39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:46.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:46.272+0000 7ff03628d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff03015a9e0 0x7ff0300b39e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52218/0 (socket says 172.21.15.143:52218) 2023-11-08T19:26:46.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:46.272+0000 7ff03628d700 1 -- 172.21.15.143:0/1181802489 learned_addr learned my addr 172.21.15.143:0/1181802489 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:46.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:46.272+0000 7ff035a8c700 1 --2- 172.21.15.143:0/1181802489 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff0300afca0 0x7ff030154f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:46.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:46.272+0000 7ff03528b700 1 --2- 172.21.15.143:0/1181802489 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff0300b2840 0x7ff0300b34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:46.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:46.272+0000 7ff03628d700 1 -- 172.21.15.143:0/1181802489 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff0300afca0 msgr2=0x7ff030154f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:46.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:46.272+0000 7ff03628d700 1 --2- 172.21.15.143:0/1181802489 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff0300afca0 0x7ff030154f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:46.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:46.272+0000 7ff03628d700 1 -- 172.21.15.143:0/1181802489 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff0300b2840 msgr2=0x7ff0300b34a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:46.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:46.272+0000 7ff03628d700 1 --2- 172.21.15.143:0/1181802489 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff0300b2840 0x7ff0300b34a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:46.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:46.272+0000 7ff03628d700 1 -- 172.21.15.143:0/1181802489 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff02c00b7f0 con 0x7ff03015a9e0 2023-11-08T19:26:46.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:46.273+0000 7ff03628d700 1 --2- 172.21.15.143:0/1181802489 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff03015a9e0 0x7ff0300b39e0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7ff028007e10 tx=0x7ff02800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:46.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:46.273+0000 7ff0267fc700 1 -- 172.21.15.143:0/1181802489 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff02800dd20 con 0x7ff03015a9e0 2023-11-08T19:26:46.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:46.273+0000 7ff0267fc700 1 -- 172.21.15.143:0/1181802489 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff02800f040 con 0x7ff03015a9e0 2023-11-08T19:26:46.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:46.273+0000 7ff037cf0700 1 -- 172.21.15.143:0/1181802489 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0300b61c0 con 0x7ff03015a9e0 2023-11-08T19:26:46.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:46.273+0000 7ff0267fc700 1 -- 172.21.15.143:0/1181802489 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff028003c60 con 0x7ff03015a9e0 2023-11-08T19:26:46.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:46.273+0000 7ff037cf0700 1 -- 172.21.15.143:0/1181802489 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0300c5530 con 0x7ff03015a9e0 2023-11-08T19:26:46.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:46.274+0000 7ff037cf0700 1 -- 172.21.15.143:0/1181802489 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0300c5a50 con 0x7ff03015a9e0 2023-11-08T19:26:46.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:46.274+0000 7ff0267fc700 1 -- 172.21.15.143:0/1181802489 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff028004030 con 0x7ff03015a9e0 2023-11-08T19:26:46.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:46.275+0000 7ff0267fc700 1 --2- 172.21.15.143:0/1181802489 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7ff01406af00 0x7ff01406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:46.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:46.275+0000 7ff0267fc700 1 -- 172.21.15.143:0/1181802489 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6206+0+0 (secure 0 0 0) 0x7ff028089700 con 0x7ff03015a9e0 2023-11-08T19:26:46.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:46.275+0000 7ff035a8c700 1 --2- 172.21.15.143:0/1181802489 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7ff01406af00 0x7ff01406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:46.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:46.275+0000 7ff035a8c700 1 --2- 172.21.15.143:0/1181802489 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7ff01406af00 0x7ff01406d3c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ff0300c9d40 tx=0x7ff020008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:46.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:46.279+0000 7ff0267fc700 1 -- 172.21.15.143:0/1181802489 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff0300c5a50 con 0x7ff03015a9e0 2023-11-08T19:26:46.469 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:46.468+0000 7ff037cf0700 1 -- 172.21.15.143:0/1181802489 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff0300c5a50 con 0x7ff03015a9e0 2023-11-08T19:26:47.092 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:47.091+0000 7ff0267fc700 1 -- 172.21.15.143:0/1181802489 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v52) v1 ==== 181+0+0 (secure 0 0 0) 0x7ff0300c5a50 con 0x7ff03015a9e0 2023-11-08T19:26:47.095 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:47.093+0000 7ff037cf0700 1 -- 172.21.15.143:0/1181802489 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7ff01406af00 msgr2=0x7ff01406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:47.095 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:47.093+0000 7ff037cf0700 1 --2- 172.21.15.143:0/1181802489 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7ff01406af00 0x7ff01406d3c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ff0300c9d40 tx=0x7ff020008040 comp rx=0 tx=0).stop 2023-11-08T19:26:47.095 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:47.093+0000 7ff037cf0700 1 -- 172.21.15.143:0/1181802489 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff03015a9e0 msgr2=0x7ff0300b39e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:47.096 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:47.093+0000 7ff037cf0700 1 --2- 172.21.15.143:0/1181802489 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff03015a9e0 0x7ff0300b39e0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7ff028007e10 tx=0x7ff02800d3b0 comp rx=0 tx=0).stop 2023-11-08T19:26:47.096 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:47.093+0000 7ff037cf0700 1 -- 172.21.15.143:0/1181802489 shutdown_connections 2023-11-08T19:26:47.096 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:47.093+0000 7ff037cf0700 1 --2- 172.21.15.143:0/1181802489 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7ff01406af00 0x7ff01406d3c0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:47.096 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:47.093+0000 7ff037cf0700 1 --2- 172.21.15.143:0/1181802489 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff0300afca0 0x7ff030154f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:47.096 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:47.093+0000 7ff037cf0700 1 --2- 172.21.15.143:0/1181802489 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff03015a9e0 0x7ff0300b39e0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:47.097 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:47.093+0000 7ff037cf0700 1 --2- 172.21.15.143:0/1181802489 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff0300b2840 0x7ff0300b34a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:47.097 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:47.093+0000 7ff037cf0700 1 -- 172.21.15.143:0/1181802489 >> 172.21.15.143:0/1181802489 conn(0x7ff0300cd300 msgr2=0x7ff0300cc1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:47.097 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:47.094+0000 7ff037cf0700 1 -- 172.21.15.143:0/1181802489 shutdown_connections 2023-11-08T19:26:47.097 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:47.094+0000 7ff037cf0700 1 -- 172.21.15.143:0/1181802489 wait complete. 2023-11-08T19:26:47.098 INFO:teuthology.orchestra.run.smithi143.stderr:pool 'cephfs_metadata' removed 2023-11-08T19:26:47.111 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-08T19:26:47.242 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:47.240+0000 7fbf6d2b3700 1 Processor -- start 2023-11-08T19:26:47.242 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:47.240+0000 7fbf6d2b3700 1 -- start start 2023-11-08T19:26:47.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:47.241+0000 7fbf6d2b3700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fbf6806bd20 0x7fbf680d01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:47.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:47.241+0000 7fbf6d2b3700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fbf680b61e0 0x7fbf680cc730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:47.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:47.241+0000 7fbf6d2b3700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf680cc2a0 con 0x7fbf6815a950 2023-11-08T19:26:47.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:47.241+0000 7fbf6d2b3700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf680cc130 con 0x7fbf6806bd20 2023-11-08T19:26:47.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:47.241+0000 7fbf6d2b3700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf6806c1b0 con 0x7fbf680b61e0 2023-11-08T19:26:47.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:47.241+0000 7fbf667fc700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fbf6806bd20 0x7fbf680d01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:47.246 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:47.241+0000 7fbf66ffd700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fbf6815a950 0x7fbf68072370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:48376/0 (socket says 172.21.15.143:48376) 2023-11-08T19:26:47.246 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:47.241+0000 7fbf66ffd700 1 -- 172.21.15.143:0/543779502 learned_addr learned my addr 172.21.15.143:0/543779502 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:26:47.246 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:47.242+0000 7fbf677fe700 1 --2- 172.21.15.143:0/543779502 >> v2:172.21.15.174:3300/0 conn(0x7fbf680b61e0 0x7fbf680cc730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:47.246 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:47.242+0000 7fbf667fc700 1 -- 172.21.15.143:0/543779502 >> v2:172.21.15.174:3300/0 conn(0x7fbf680b61e0 msgr2=0x7fbf680cc730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:47.246 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:47.242+0000 7fbf667fc700 1 --2- 172.21.15.143:0/543779502 >> v2:172.21.15.174:3300/0 conn(0x7fbf680b61e0 0x7fbf680cc730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:47.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:47.242+0000 7fbf667fc700 1 -- 172.21.15.143:0/543779502 >> v1:172.21.15.143:6789/0 conn(0x7fbf6815a950 legacy=0x7fbf68072370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:47.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:47.242+0000 7fbf667fc700 1 -- 172.21.15.143:0/543779502 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf6806c490 con 0x7fbf6806bd20 2023-11-08T19:26:47.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:47.242+0000 7fbf65ffb700 1 -- 172.21.15.143:0/543779502 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1215151670 0 0) 0x7fbf680cc2a0 con 0x7fbf6815a950 2023-11-08T19:26:47.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:47.242+0000 7fbf667fc700 1 --2- 172.21.15.143:0/543779502 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf6806bd20 0x7fbf680d01b0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fbf680c7890 tx=0x7fbf58009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67d789094f71394 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:47.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:47.242+0000 7fbf65ffb700 1 -- 172.21.15.143:0/543779502 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf58011830 con 0x7fbf6806bd20 2023-11-08T19:26:47.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:47.242+0000 7fbf65ffb700 1 -- 172.21.15.143:0/543779502 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf58011990 con 0x7fbf6806bd20 2023-11-08T19:26:47.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:47.242+0000 7fbf65ffb700 1 -- 172.21.15.143:0/543779502 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf58011d20 con 0x7fbf6806bd20 2023-11-08T19:26:47.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:47.243+0000 7fbf6d2b3700 1 -- 172.21.15.143:0/543779502 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf6806bd20 msgr2=0x7fbf680d01b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:47.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:47.243+0000 7fbf6d2b3700 1 --2- 172.21.15.143:0/543779502 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf6806bd20 0x7fbf680d01b0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fbf680c7890 tx=0x7fbf58009b20 comp rx=0 tx=0).stop 2023-11-08T19:26:47.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:47.243+0000 7fbf6d2b3700 1 -- 172.21.15.143:0/543779502 shutdown_connections 2023-11-08T19:26:47.249 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:47.243+0000 7fbf6d2b3700 1 --2- 172.21.15.143:0/543779502 >> v2:172.21.15.174:3300/0 conn(0x7fbf680b61e0 0x7fbf680cc730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:47.249 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:47.243+0000 7fbf6d2b3700 1 --2- 172.21.15.143:0/543779502 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf6806bd20 0x7fbf680d01b0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:47.249 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:47.243+0000 7fbf6d2b3700 1 -- 172.21.15.143:0/543779502 >> 172.21.15.143:0/543779502 conn(0x7fbf680cd2e0 msgr2=0x7fbf680cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:47.249 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:47.243+0000 7fbf6d2b3700 1 -- 172.21.15.143:0/543779502 shutdown_connections 2023-11-08T19:26:47.249 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:47.243+0000 7fbf6d2b3700 1 -- 172.21.15.143:0/543779502 wait complete. 2023-11-08T19:26:47.252 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:47.244+0000 7fbf6d2b3700 1 Processor -- start 2023-11-08T19:26:47.252 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:47.244+0000 7fbf6d2b3700 1 -- start start 2023-11-08T19:26:47.252 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:47.245+0000 7fbf6d2b3700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf680b61e0 0x7fbf68152050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:47.253 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:47.245+0000 7fbf6d2b3700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbf6815a950 0x7fbf680b4b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:47.253 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:47.245+0000 7fbf6d2b3700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbf680c4330 0x7fbf680c8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:47.253 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:47.245+0000 7fbf6d2b3700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf680b5090 con 0x7fbf680b61e0 2023-11-08T19:26:47.253 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:47.245+0000 7fbf6d2b3700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf680c47c0 con 0x7fbf6815a950 2023-11-08T19:26:47.253 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:47.245+0000 7fbf6d2b3700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf680c4aa0 con 0x7fbf680c4330 2023-11-08T19:26:47.254 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:47.245+0000 7fbf66ffd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf680b61e0 0x7fbf68152050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:47.254 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:47.245+0000 7fbf66ffd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf680b61e0 0x7fbf68152050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52228/0 (socket says 172.21.15.143:52228) 2023-11-08T19:26:47.254 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:47.245+0000 7fbf66ffd700 1 -- 172.21.15.143:0/120601391 learned_addr learned my addr 172.21.15.143:0/120601391 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:47.254 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:47.245+0000 7fbf667fc700 1 --2- 172.21.15.143:0/120601391 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbf6815a950 0x7fbf680b4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:47.254 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:47.245+0000 7fbf677fe700 1 --2- 172.21.15.143:0/120601391 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbf680c4330 0x7fbf680c8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:47.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:47.245+0000 7fbf66ffd700 1 -- 172.21.15.143:0/120601391 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbf680c4330 msgr2=0x7fbf680c8ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:47.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:47.245+0000 7fbf66ffd700 1 --2- 172.21.15.143:0/120601391 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbf680c4330 0x7fbf680c8ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:47.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:47.245+0000 7fbf66ffd700 1 -- 172.21.15.143:0/120601391 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbf6815a950 msgr2=0x7fbf680b4b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:47.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:47.245+0000 7fbf66ffd700 1 --2- 172.21.15.143:0/120601391 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbf6815a950 0x7fbf680b4b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:47.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:47.245+0000 7fbf66ffd700 1 -- 172.21.15.143:0/120601391 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf580094a0 con 0x7fbf680b61e0 2023-11-08T19:26:47.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:47.245+0000 7fbf677fe700 1 --2- 172.21.15.143:0/120601391 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbf680c4330 0x7fbf680c8ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:26:47.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:47.245+0000 7fbf66ffd700 1 --2- 172.21.15.143:0/120601391 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf680b61e0 0x7fbf68152050 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c00c550 tx=0x7fbf5c00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:47.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:47.245+0000 7fbf4f7fe700 1 -- 172.21.15.143:0/120601391 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf5c004c80 con 0x7fbf680b61e0 2023-11-08T19:26:47.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:47.245+0000 7fbf4f7fe700 1 -- 172.21.15.143:0/120601391 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf5c004e00 con 0x7fbf680b61e0 2023-11-08T19:26:47.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:47.245+0000 7fbf6d2b3700 1 -- 172.21.15.143:0/120601391 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf680c4d80 con 0x7fbf680b61e0 2023-11-08T19:26:47.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:47.246+0000 7fbf4f7fe700 1 -- 172.21.15.143:0/120601391 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf5c007310 con 0x7fbf680b61e0 2023-11-08T19:26:47.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:47.246+0000 7fbf6d2b3700 1 -- 172.21.15.143:0/120601391 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf68152890 con 0x7fbf680b61e0 2023-11-08T19:26:47.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:47.247+0000 7fbf6d2b3700 1 -- 172.21.15.143:0/120601391 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf6804eee0 con 0x7fbf680b61e0 2023-11-08T19:26:47.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:47.247+0000 7fbf4f7fe700 1 -- 172.21.15.143:0/120601391 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbf5c007470 con 0x7fbf680b61e0 2023-11-08T19:26:47.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:47.247+0000 7fbf4f7fe700 1 --2- 172.21.15.143:0/120601391 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fbf5406af00 0x7fbf5406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:47.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:47.248+0000 7fbf4f7fe700 1 -- 172.21.15.143:0/120601391 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5768+0+0 (secure 0 0 0) 0x7fbf5c0891d0 con 0x7fbf680b61e0 2023-11-08T19:26:47.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:47.248+0000 7fbf667fc700 1 --2- 172.21.15.143:0/120601391 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fbf5406af00 0x7fbf5406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:47.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:47.248+0000 7fbf667fc700 1 --2- 172.21.15.143:0/120601391 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fbf5406af00 0x7fbf5406d3c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fbf580090b0 tx=0x7fbf58008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:47.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:47.252+0000 7fbf4f7fe700 1 -- 172.21.15.143:0/120601391 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbf5c0542d0 con 0x7fbf680b61e0 2023-11-08T19:26:47.444 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:47.442+0000 7fbf6d2b3700 1 -- 172.21.15.143:0/120601391 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fbf68072150 con 0x7fbf680b61e0 2023-11-08T19:26:48.095 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.094+0000 7fbf4f7fe700 1 -- 172.21.15.143:0/120601391 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v53) v1 ==== 169+0+0 (secure 0 0 0) 0x7fbf5c0578f0 con 0x7fbf680b61e0 2023-11-08T19:26:48.098 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.097+0000 7fbf6d2b3700 1 -- 172.21.15.143:0/120601391 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fbf5406af00 msgr2=0x7fbf5406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:48.098 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.097+0000 7fbf6d2b3700 1 --2- 172.21.15.143:0/120601391 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fbf5406af00 0x7fbf5406d3c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fbf580090b0 tx=0x7fbf58008030 comp rx=0 tx=0).stop 2023-11-08T19:26:48.099 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.097+0000 7fbf6d2b3700 1 -- 172.21.15.143:0/120601391 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf680b61e0 msgr2=0x7fbf68152050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:48.099 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.097+0000 7fbf6d2b3700 1 --2- 172.21.15.143:0/120601391 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf680b61e0 0x7fbf68152050 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c00c550 tx=0x7fbf5c00c860 comp rx=0 tx=0).stop 2023-11-08T19:26:48.099 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.097+0000 7fbf6d2b3700 1 -- 172.21.15.143:0/120601391 shutdown_connections 2023-11-08T19:26:48.099 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.097+0000 7fbf6d2b3700 1 --2- 172.21.15.143:0/120601391 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fbf5406af00 0x7fbf5406d3c0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:48.100 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.097+0000 7fbf6d2b3700 1 --2- 172.21.15.143:0/120601391 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbf680c4330 0x7fbf680c8ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:48.100 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.097+0000 7fbf6d2b3700 1 --2- 172.21.15.143:0/120601391 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf680b61e0 0x7fbf68152050 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:48.100 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.097+0000 7fbf6d2b3700 1 --2- 172.21.15.143:0/120601391 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbf6815a950 0x7fbf680b4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:48.100 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.097+0000 7fbf6d2b3700 1 -- 172.21.15.143:0/120601391 >> 172.21.15.143:0/120601391 conn(0x7fbf680cd2e0 msgr2=0x7fbf6815c060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:48.100 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.097+0000 7fbf6d2b3700 1 -- 172.21.15.143:0/120601391 shutdown_connections 2023-11-08T19:26:48.101 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.097+0000 7fbf6d2b3700 1 -- 172.21.15.143:0/120601391 wait complete. 2023-11-08T19:26:48.101 INFO:teuthology.orchestra.run.smithi143.stderr:pool 'cephfs_data' removed 2023-11-08T19:26:48.116 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-08T19:26:48.241 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.240+0000 7f4cf670e700 1 Processor -- start 2023-11-08T19:26:48.242 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.240+0000 7f4cf670e700 1 -- start start 2023-11-08T19:26:48.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.241+0000 7f4cf670e700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f4cf006b240 0x7f4cf00caff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:48.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.241+0000 7f4cf670e700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f4cf00cb530 0x7f4cf00b22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:48.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.241+0000 7f4cf670e700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4cf0161890 con 0x7f4cf015dba0 2023-11-08T19:26:48.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.241+0000 7f4cf670e700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cf006cd20 con 0x7f4cf00cb530 2023-11-08T19:26:48.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.241+0000 7f4cf670e700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cf006b740 con 0x7f4cf006b240 2023-11-08T19:26:48.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.241+0000 7f4cf4cab700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f4cf00cb530 0x7f4cf00b22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:48.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.241+0000 7f4cf4cab700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f4cf00cb530 0x7f4cf00b22e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52240/0 (socket says 172.21.15.143:52240) 2023-11-08T19:26:48.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.241+0000 7f4cf4cab700 1 -- 172.21.15.143:0/886245607 learned_addr learned my addr 172.21.15.143:0/886245607 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:48.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.241+0000 7f4cef7fe700 1 --2- 172.21.15.143:0/886245607 >> v2:172.21.15.174:3300/0 conn(0x7f4cf006b240 0x7f4cf00caff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:48.246 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.241+0000 7f4cf4cab700 1 -- 172.21.15.143:0/886245607 >> v2:172.21.15.174:3300/0 conn(0x7f4cf006b240 msgr2=0x7f4cf00caff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:48.246 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.241+0000 7f4cf4cab700 1 --2- 172.21.15.143:0/886245607 >> v2:172.21.15.174:3300/0 conn(0x7f4cf006b240 0x7f4cf00caff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:48.246 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.241+0000 7f4cf4cab700 1 -- 172.21.15.143:0/886245607 >> v1:172.21.15.174:6789/0 conn(0x7f4cf015dba0 legacy=0x7f4cf006c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:48.246 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.241+0000 7f4cf4cab700 1 -- 172.21.15.143:0/886245607 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cf006ba20 con 0x7f4cf00cb530 2023-11-08T19:26:48.246 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.242+0000 7f4cf4cab700 1 --2- 172.21.15.143:0/886245607 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4cf00cb530 0x7f4cf00b22e0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f4cf00d0810 tx=0x7f4ce800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e86ec5c1df57245 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:48.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.242+0000 7f4ceeffd700 1 -- 172.21.15.143:0/886245607 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce800e070 con 0x7f4cf00cb530 2023-11-08T19:26:48.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.242+0000 7f4ceeffd700 1 -- 172.21.15.143:0/886245607 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ce800b910 con 0x7f4cf00cb530 2023-11-08T19:26:48.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.242+0000 7f4ceeffd700 1 -- 172.21.15.143:0/886245607 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce800bcc0 con 0x7f4cf00cb530 2023-11-08T19:26:48.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.242+0000 7f4cf670e700 1 -- 172.21.15.143:0/886245607 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4cf00cb530 msgr2=0x7f4cf00b22e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:48.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.242+0000 7f4cf670e700 1 --2- 172.21.15.143:0/886245607 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4cf00cb530 0x7f4cf00b22e0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f4cf00d0810 tx=0x7f4ce800b330 comp rx=0 tx=0).stop 2023-11-08T19:26:48.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.242+0000 7f4cf670e700 1 -- 172.21.15.143:0/886245607 shutdown_connections 2023-11-08T19:26:48.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.242+0000 7f4cf670e700 1 --2- 172.21.15.143:0/886245607 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4cf00cb530 0x7f4cf00b22e0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:48.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.242+0000 7f4cf670e700 1 --2- 172.21.15.143:0/886245607 >> v2:172.21.15.174:3300/0 conn(0x7f4cf006b240 0x7f4cf00caff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:48.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.242+0000 7f4cf670e700 1 -- 172.21.15.143:0/886245607 >> 172.21.15.143:0/886245607 conn(0x7f4cf00cd2e0 msgr2=0x7f4cf00cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:48.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.242+0000 7f4cf670e700 1 -- 172.21.15.143:0/886245607 shutdown_connections 2023-11-08T19:26:48.249 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.242+0000 7f4cf670e700 1 -- 172.21.15.143:0/886245607 wait complete. 2023-11-08T19:26:48.251 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.244+0000 7f4cf670e700 1 Processor -- start 2023-11-08T19:26:48.251 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.244+0000 7f4cf670e700 1 -- start start 2023-11-08T19:26:48.252 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.244+0000 7f4cf670e700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4cf006b240 0x7f4cf015ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:48.252 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.245+0000 7f4cf670e700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4cf00cb530 0x7f4cf014ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:48.252 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.245+0000 7f4cf670e700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4cf015dba0 0x7f4cf00c7930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:48.252 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.245+0000 7f4cf670e700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf00c8d60 con 0x7f4cf015dba0 2023-11-08T19:26:48.253 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.245+0000 7f4cf670e700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf00c8150 con 0x7f4cf006b240 2023-11-08T19:26:48.253 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.245+0000 7f4cf670e700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf00c8430 con 0x7f4cf00cb530 2023-11-08T19:26:48.253 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.245+0000 7f4cf4cab700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4cf015dba0 0x7f4cf00c7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:48.253 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.245+0000 7f4cf4cab700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4cf015dba0 0x7f4cf00c7930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52246/0 (socket says 172.21.15.143:52246) 2023-11-08T19:26:48.253 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.245+0000 7f4cf4cab700 1 -- 172.21.15.143:0/2918430433 learned_addr learned my addr 172.21.15.143:0/2918430433 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:48.254 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.245+0000 7f4ceffff700 1 --2- 172.21.15.143:0/2918430433 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4cf006b240 0x7f4cf015ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:48.254 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.245+0000 7f4cef7fe700 1 --2- 172.21.15.143:0/2918430433 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4cf00cb530 0x7f4cf014ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:48.254 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.245+0000 7f4cf4cab700 1 -- 172.21.15.143:0/2918430433 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4cf00cb530 msgr2=0x7f4cf014ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:48.254 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.245+0000 7f4cf4cab700 1 --2- 172.21.15.143:0/2918430433 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4cf00cb530 0x7f4cf014ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:48.254 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.245+0000 7f4cf4cab700 1 -- 172.21.15.143:0/2918430433 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4cf006b240 msgr2=0x7f4cf015ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:48.254 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.245+0000 7f4cf4cab700 1 --2- 172.21.15.143:0/2918430433 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4cf006b240 0x7f4cf015ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:48.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.245+0000 7f4cf4cab700 1 -- 172.21.15.143:0/2918430433 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ce8009b30 con 0x7f4cf015dba0 2023-11-08T19:26:48.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.245+0000 7f4ceffff700 1 --2- 172.21.15.143:0/2918430433 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4cf006b240 0x7f4cf015ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:26:48.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.245+0000 7f4cf4cab700 1 --2- 172.21.15.143:0/2918430433 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4cf015dba0 0x7f4cf00c7930 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f4cf0067530 tx=0x7f4ce8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:48.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.245+0000 7f4cecff9700 1 -- 172.21.15.143:0/2918430433 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce80099b0 con 0x7f4cf015dba0 2023-11-08T19:26:48.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.245+0000 7f4cecff9700 1 -- 172.21.15.143:0/2918430433 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ce80092e0 con 0x7f4cf015dba0 2023-11-08T19:26:48.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.245+0000 7f4cf670e700 1 -- 172.21.15.143:0/2918430433 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cf00c8710 con 0x7f4cf015dba0 2023-11-08T19:26:48.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.245+0000 7f4cecff9700 1 -- 172.21.15.143:0/2918430433 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce801c640 con 0x7f4cf015dba0 2023-11-08T19:26:48.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.245+0000 7f4cf670e700 1 -- 172.21.15.143:0/2918430433 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cf00c91f0 con 0x7f4cf015dba0 2023-11-08T19:26:48.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.246+0000 7f4cf670e700 1 -- 172.21.15.143:0/2918430433 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cf004eee0 con 0x7f4cf015dba0 2023-11-08T19:26:48.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.248+0000 7f4cecff9700 1 -- 172.21.15.143:0/2918430433 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4ce801c7a0 con 0x7f4cf015dba0 2023-11-08T19:26:48.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.248+0000 7f4cecff9700 1 --2- 172.21.15.143:0/2918430433 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f4cdc06af00 0x7f4cdc06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:48.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.248+0000 7f4cecff9700 1 -- 172.21.15.143:0/2918430433 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f4ce808b140 con 0x7f4cf015dba0 2023-11-08T19:26:48.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.248+0000 7f4ceffff700 1 --2- 172.21.15.143:0/2918430433 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f4cdc06af00 0x7f4cdc06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:48.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.249+0000 7f4ceffff700 1 --2- 172.21.15.143:0/2918430433 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f4cdc06af00 0x7f4cdc06d3c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f4cf0073040 tx=0x7f4ce4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:48.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.252+0000 7f4cecff9700 1 -- 172.21.15.143:0/2918430433 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4ce8056520 con 0x7f4cf015dba0 2023-11-08T19:26:48.453 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.452+0000 7f4cf670e700 1 -- 172.21.15.143:0/2918430433 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f4cf00b67e0 con 0x7f4cf015dba0 2023-11-08T19:26:48.456 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.455+0000 7f4cecff9700 1 -- 172.21.15.143:0/2918430433 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v14) v1 ==== 104+0+0 (secure 0 0 0) 0x7f4ce8059b40 con 0x7f4cf015dba0 2023-11-08T19:26:48.459 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.457+0000 7f4cf670e700 1 -- 172.21.15.143:0/2918430433 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f4cdc06af00 msgr2=0x7f4cdc06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:48.459 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.457+0000 7f4cf670e700 1 --2- 172.21.15.143:0/2918430433 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f4cdc06af00 0x7f4cdc06d3c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f4cf0073040 tx=0x7f4ce4009040 comp rx=0 tx=0).stop 2023-11-08T19:26:48.459 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.457+0000 7f4cf670e700 1 -- 172.21.15.143:0/2918430433 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4cf015dba0 msgr2=0x7f4cf00c7930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:48.459 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.457+0000 7f4cf670e700 1 --2- 172.21.15.143:0/2918430433 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4cf015dba0 0x7f4cf00c7930 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f4cf0067530 tx=0x7f4ce8004ad0 comp rx=0 tx=0).stop 2023-11-08T19:26:48.460 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.457+0000 7f4cf670e700 1 -- 172.21.15.143:0/2918430433 shutdown_connections 2023-11-08T19:26:48.460 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.457+0000 7f4cf670e700 1 --2- 172.21.15.143:0/2918430433 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f4cdc06af00 0x7f4cdc06d3c0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:48.460 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.457+0000 7f4cf670e700 1 --2- 172.21.15.143:0/2918430433 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4cf015dba0 0x7f4cf00c7930 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:48.460 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.457+0000 7f4cf670e700 1 --2- 172.21.15.143:0/2918430433 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4cf006b240 0x7f4cf015ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:48.460 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.457+0000 7f4cf670e700 1 --2- 172.21.15.143:0/2918430433 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4cf00cb530 0x7f4cf014ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:48.461 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.457+0000 7f4cf670e700 1 -- 172.21.15.143:0/2918430433 >> 172.21.15.143:0/2918430433 conn(0x7f4cf00cd2e0 msgr2=0x7f4cf00b10f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:48.461 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.458+0000 7f4cf670e700 1 -- 172.21.15.143:0/2918430433 shutdown_connections 2023-11-08T19:26:48.461 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.458+0000 7f4cf670e700 1 -- 172.21.15.143:0/2918430433 wait complete. 2023-11-08T19:26:48.471 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snapshots.TestMonSnapsAndFsPools.test_disallow_monitor_managed_snaps_for_fs_pools' 2023-11-08T19:26:48.591 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.590+0000 7f25a96f7700 1 Processor -- start 2023-11-08T19:26:48.592 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.590+0000 7f25a96f7700 1 -- start start 2023-11-08T19:26:48.594 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.591+0000 7f25a96f7700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f25a40cf090 0x7f25a40d1f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:48.594 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.591+0000 7f25a96f7700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f25a40d3690 0x7f25a40b9690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:48.594 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.591+0000 7f25a96f7700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25a40d3200 con 0x7f25a415dc80 2023-11-08T19:26:48.594 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.591+0000 7f25a96f7700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25a40b9bd0 con 0x7f25a40d3690 2023-11-08T19:26:48.595 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.591+0000 7f25a96f7700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25a40b7cf0 con 0x7f25a40cf090 2023-11-08T19:26:48.595 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.591+0000 7f25a37fe700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f25a40d3690 0x7f25a40b9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:48.595 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.591+0000 7f25a37fe700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f25a40d3690 0x7f25a40b9690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52254/0 (socket says 172.21.15.143:52254) 2023-11-08T19:26:48.595 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.591+0000 7f25a37fe700 1 -- 172.21.15.143:0/1329461227 learned_addr learned my addr 172.21.15.143:0/1329461227 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:48.595 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.591+0000 7f25a27fc700 1 --2- 172.21.15.143:0/1329461227 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f25a40cf090 0x7f25a40d1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:48.596 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.591+0000 7f25a37fe700 1 -- 172.21.15.143:0/1329461227 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f25a40cf090 msgr2=0x7f25a40d1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:48.596 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.591+0000 7f25a37fe700 1 --2- 172.21.15.143:0/1329461227 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f25a40cf090 0x7f25a40d1f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:48.596 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.591+0000 7f25a37fe700 1 -- 172.21.15.143:0/1329461227 >> v1:172.21.15.174:6789/0 conn(0x7f25a415dc80 legacy=0x7f25a40ce980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:48.596 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.591+0000 7f25a37fe700 1 -- 172.21.15.143:0/1329461227 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25a415e060 con 0x7f25a40d3690 2023-11-08T19:26:48.596 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.591+0000 7f25a37fe700 1 --2- 172.21.15.143:0/1329461227 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f25a40d3690 0x7f25a40b9690 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f259c009e50 tx=0x7f259c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc661bd7f5c890f server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:48.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.592+0000 7f25a1ffb700 1 -- 172.21.15.143:0/1329461227 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f259c00e070 con 0x7f25a40d3690 2023-11-08T19:26:48.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.592+0000 7f25a1ffb700 1 -- 172.21.15.143:0/1329461227 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f259c00b9c0 con 0x7f25a40d3690 2023-11-08T19:26:48.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.592+0000 7f25a1ffb700 1 -- 172.21.15.143:0/1329461227 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f259c00bd70 con 0x7f25a40d3690 2023-11-08T19:26:48.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.592+0000 7f25a96f7700 1 -- 172.21.15.143:0/1329461227 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f25a40d3690 msgr2=0x7f25a40b9690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:48.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.592+0000 7f25a96f7700 1 --2- 172.21.15.143:0/1329461227 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f25a40d3690 0x7f25a40b9690 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f259c009e50 tx=0x7f259c00b330 comp rx=0 tx=0).stop 2023-11-08T19:26:48.598 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.592+0000 7f25a96f7700 1 -- 172.21.15.143:0/1329461227 shutdown_connections 2023-11-08T19:26:48.598 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.592+0000 7f25a96f7700 1 --2- 172.21.15.143:0/1329461227 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f25a40d3690 0x7f25a40b9690 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:48.598 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.592+0000 7f25a96f7700 1 --2- 172.21.15.143:0/1329461227 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f25a40cf090 0x7f25a40d1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:48.598 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.592+0000 7f25a96f7700 1 -- 172.21.15.143:0/1329461227 >> 172.21.15.143:0/1329461227 conn(0x7f25a40668e0 msgr2=0x7f25a4066cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:48.599 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.592+0000 7f25a96f7700 1 -- 172.21.15.143:0/1329461227 shutdown_connections 2023-11-08T19:26:48.599 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.592+0000 7f25a96f7700 1 -- 172.21.15.143:0/1329461227 wait complete. 2023-11-08T19:26:48.601 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.593+0000 7f25a96f7700 1 Processor -- start 2023-11-08T19:26:48.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.594+0000 7f25a96f7700 1 -- start start 2023-11-08T19:26:48.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.594+0000 7f25a96f7700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f25a40cf090 0x7f25a40c9090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:48.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.594+0000 7f25a96f7700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f25a40d3690 0x7f25a41f2a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:48.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.594+0000 7f25a96f7700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f25a415dc80 0x7f25a40c76a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:48.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.594+0000 7f25a96f7700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25a41f2fc0 con 0x7f25a415dc80 2023-11-08T19:26:48.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.594+0000 7f25a96f7700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25a40cb290 con 0x7f25a40cf090 2023-11-08T19:26:48.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.594+0000 7f25a96f7700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25a40cb570 con 0x7f25a40d3690 2023-11-08T19:26:48.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.594+0000 7f25a37fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f25a415dc80 0x7f25a40c76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:48.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.594+0000 7f25a37fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f25a415dc80 0x7f25a40c76a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52260/0 (socket says 172.21.15.143:52260) 2023-11-08T19:26:48.604 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.594+0000 7f25a37fe700 1 -- 172.21.15.143:0/418039258 learned_addr learned my addr 172.21.15.143:0/418039258 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:48.604 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.595+0000 7f25a2ffd700 1 --2- 172.21.15.143:0/418039258 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f25a40cf090 0x7f25a40c9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:48.604 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.595+0000 7f25a27fc700 1 --2- 172.21.15.143:0/418039258 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f25a40d3690 0x7f25a41f2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:48.604 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.595+0000 7f25a37fe700 1 -- 172.21.15.143:0/418039258 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f25a40d3690 msgr2=0x7f25a41f2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:48.604 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.595+0000 7f25a37fe700 1 --2- 172.21.15.143:0/418039258 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f25a40d3690 0x7f25a41f2a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:48.605 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.595+0000 7f25a37fe700 1 -- 172.21.15.143:0/418039258 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f25a40cf090 msgr2=0x7f25a40c9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:48.605 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.595+0000 7f25a37fe700 1 --2- 172.21.15.143:0/418039258 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f25a40cf090 0x7f25a40c9090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:48.605 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.595+0000 7f25a37fe700 1 -- 172.21.15.143:0/418039258 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f259c009b30 con 0x7f25a415dc80 2023-11-08T19:26:48.605 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.595+0000 7f25a37fe700 1 --2- 172.21.15.143:0/418039258 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f25a415dc80 0x7f25a40c76a0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f259c012000 tx=0x7f259c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:48.605 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.595+0000 7f258b7fe700 1 -- 172.21.15.143:0/418039258 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f259c004960 con 0x7f25a415dc80 2023-11-08T19:26:48.605 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.595+0000 7f258b7fe700 1 -- 172.21.15.143:0/418039258 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f259c0092e0 con 0x7f25a415dc80 2023-11-08T19:26:48.606 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.595+0000 7f25a96f7700 1 -- 172.21.15.143:0/418039258 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25a40cb850 con 0x7f25a415dc80 2023-11-08T19:26:48.606 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.595+0000 7f258b7fe700 1 -- 172.21.15.143:0/418039258 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f259c01c660 con 0x7f25a415dc80 2023-11-08T19:26:48.606 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.595+0000 7f25a96f7700 1 -- 172.21.15.143:0/418039258 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25a40bda00 con 0x7f25a415dc80 2023-11-08T19:26:48.606 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.596+0000 7f25a96f7700 1 -- 172.21.15.143:0/418039258 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25a40cb3d0 con 0x7f25a415dc80 2023-11-08T19:26:48.606 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.596+0000 7f258b7fe700 1 -- 172.21.15.143:0/418039258 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f259c01c7c0 con 0x7f25a415dc80 2023-11-08T19:26:48.607 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.597+0000 7f258b7fe700 1 --2- 172.21.15.143:0/418039258 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f259006af30 0x7f259006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:48.607 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.597+0000 7f258b7fe700 1 -- 172.21.15.143:0/418039258 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f259c08ad10 con 0x7f25a415dc80 2023-11-08T19:26:48.607 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.597+0000 7f25a2ffd700 1 --2- 172.21.15.143:0/418039258 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f259006af30 0x7f259006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:48.607 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.597+0000 7f25a2ffd700 1 --2- 172.21.15.143:0/418039258 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f259006af30 0x7f259006d3f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f2598000f60 tx=0x7f2598009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:48.608 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.601+0000 7f258b7fe700 1 -- 172.21.15.143:0/418039258 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f25a40cb3d0 con 0x7f25a415dc80 2023-11-08T19:26:48.871 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:48.870+0000 7f25a96f7700 1 -- 172.21.15.143:0/418039258 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snapshots.TestMonSnapsAndFsPools.test_disallow_monitor_managed_snaps_for_fs_pools"]} v 0) v1 -- 0x7f25a40cbd70 con 0x7f25a415dc80 2023-11-08T19:26:49.096 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.095+0000 7f258b7fe700 1 -- 172.21.15.143:0/418039258 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snapshots.TestMonSnapsAndFsPools.test_disallow_monitor_managed_snaps_for_fs_pools"]}]=0 v130) v1 ==== 178+0+0 (secure 0 0 0) 0x7f25a40cbd70 con 0x7f25a415dc80 2023-11-08T19:26:49.099 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.097+0000 7f25a96f7700 1 -- 172.21.15.143:0/418039258 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f259006af30 msgr2=0x7f259006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:49.100 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.097+0000 7f25a96f7700 1 --2- 172.21.15.143:0/418039258 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f259006af30 0x7f259006d3f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f2598000f60 tx=0x7f2598009040 comp rx=0 tx=0).stop 2023-11-08T19:26:49.100 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.098+0000 7f25a96f7700 1 -- 172.21.15.143:0/418039258 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f25a415dc80 msgr2=0x7f25a40c76a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:49.100 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.098+0000 7f25a96f7700 1 --2- 172.21.15.143:0/418039258 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f25a415dc80 0x7f25a40c76a0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f259c012000 tx=0x7f259c004ad0 comp rx=0 tx=0).stop 2023-11-08T19:26:49.100 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.098+0000 7f25a96f7700 1 -- 172.21.15.143:0/418039258 shutdown_connections 2023-11-08T19:26:49.100 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.098+0000 7f25a96f7700 1 --2- 172.21.15.143:0/418039258 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f259006af30 0x7f259006d3f0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:49.101 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.098+0000 7f25a96f7700 1 --2- 172.21.15.143:0/418039258 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f25a415dc80 0x7f25a40c76a0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:49.101 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.098+0000 7f25a96f7700 1 --2- 172.21.15.143:0/418039258 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f25a40cf090 0x7f25a40c9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:49.101 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.098+0000 7f25a96f7700 1 --2- 172.21.15.143:0/418039258 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f25a40d3690 0x7f25a41f2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:49.101 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.098+0000 7f25a96f7700 1 -- 172.21.15.143:0/418039258 >> 172.21.15.143:0/418039258 conn(0x7f25a40668e0 msgr2=0x7f25a415b900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:49.102 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.098+0000 7f25a96f7700 1 -- 172.21.15.143:0/418039258 shutdown_connections 2023-11-08T19:26:49.102 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.098+0000 7f25a96f7700 1 -- 172.21.15.143:0/418039258 wait complete. 2023-11-08T19:26:49.112 INFO:tasks.cephfs_test_runner:Test that creation of monitor managed snaps fails for pools attached ... ok 2023-11-08T19:26:49.112 INFO:tasks.cephfs_test_runner:Starting test: test_using_pool_with_snap_fails_fs_creation (tasks.cephfs.test_snapshots.TestMonSnapsAndFsPools) Test that using a pool with snaps for fs creation fails 2023-11-08T19:26:49.112 INFO:tasks.cephfs_test_runner:test_using_pool_with_snap_fails_fs_creation (tasks.cephfs.test_snapshots.TestMonSnapsAndFsPools) 2023-11-08T19:26:49.113 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:26:49.234 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.232+0000 7f243f06d700 1 Processor -- start 2023-11-08T19:26:49.234 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.232+0000 7f243f06d700 1 -- start start 2023-11-08T19:26:49.237 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.233+0000 7f243f06d700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f24381585e0 0x7f243806ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:49.237 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.233+0000 7f243f06d700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f24380b2510 0x7f24380cec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:49.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.233+0000 7f243f06d700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24381589c0 con 0x7f243815e530 2023-11-08T19:26:49.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.233+0000 7f243f06d700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f243815bf30 con 0x7f24380b2510 2023-11-08T19:26:49.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.233+0000 7f243f06d700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24380cf9f0 con 0x7f24381585e0 2023-11-08T19:26:49.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.234+0000 7f243ce09700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f24381585e0 0x7f243806ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:49.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.234+0000 7f2437fff700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f24380b2510 0x7f24380cec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:49.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.234+0000 7f243d60a700 1 --1- >> v1:172.21.15.174:6789/0 conn(0x7f243815e530 0x7f24380cb900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.174:6789/0 says I am v1:172.21.15.143:59354/0 (socket says 172.21.15.143:59354) 2023-11-08T19:26:49.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.234+0000 7f243d60a700 1 -- 172.21.15.143:0/1172151927 learned_addr learned my addr 172.21.15.143:0/1172151927 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:26:49.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.234+0000 7f2437fff700 1 -- 172.21.15.143:0/1172151927 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f24381585e0 msgr2=0x7f243806ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:49.240 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.234+0000 7f2437fff700 1 --2- 172.21.15.143:0/1172151927 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f24381585e0 0x7f243806ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:49.240 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.234+0000 7f2437fff700 1 -- 172.21.15.143:0/1172151927 >> v1:172.21.15.174:6789/0 conn(0x7f243815e530 legacy=0x7f24380cb900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:49.240 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.234+0000 7f2437fff700 1 -- 172.21.15.143:0/1172151927 --> v2:172.21.15.174:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f243815b320 con 0x7f24380b2510 2023-11-08T19:26:49.240 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.234+0000 7f24377fe700 1 -- 172.21.15.143:0/1172151927 <== mon.1 v1:172.21.15.174:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2422575828 0 0) 0x7f2430001490 con 0x7f243815e530 2023-11-08T19:26:49.241 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.234+0000 7f243ce09700 1 --2- 172.21.15.143:0/1172151927 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f24381585e0 0x7f243806ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T19:26:49.241 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.234+0000 7f2437fff700 1 --2- 172.21.15.143:0/1172151927 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f24380b2510 0x7f24380cec10 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f2424009870 tx=0x7f2424009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b36f41b14cc9970e server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:49.241 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.234+0000 7f24377fe700 1 -- 172.21.15.143:0/1172151927 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2424011830 con 0x7f24380b2510 2023-11-08T19:26:49.241 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.234+0000 7f24377fe700 1 -- 172.21.15.143:0/1172151927 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2424011990 con 0x7f24380b2510 2023-11-08T19:26:49.242 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.234+0000 7f24377fe700 1 -- 172.21.15.143:0/1172151927 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2424011d20 con 0x7f24380b2510 2023-11-08T19:26:49.242 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.235+0000 7f243f06d700 1 -- 172.21.15.143:0/1172151927 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f24380b2510 msgr2=0x7f24380cec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:49.242 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.235+0000 7f243f06d700 1 --2- 172.21.15.143:0/1172151927 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f24380b2510 0x7f24380cec10 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f2424009870 tx=0x7f2424009b80 comp rx=0 tx=0).stop 2023-11-08T19:26:49.242 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.235+0000 7f243f06d700 1 -- 172.21.15.143:0/1172151927 shutdown_connections 2023-11-08T19:26:49.243 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.235+0000 7f243f06d700 1 --2- 172.21.15.143:0/1172151927 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f24381585e0 0x7f243806ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:49.243 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.235+0000 7f243f06d700 1 --2- 172.21.15.143:0/1172151927 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f24380b2510 0x7f24380cec10 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:49.243 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.235+0000 7f243f06d700 1 -- 172.21.15.143:0/1172151927 >> 172.21.15.143:0/1172151927 conn(0x7f24380b1a20 msgr2=0x7f24380b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:49.243 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.235+0000 7f243f06d700 1 -- 172.21.15.143:0/1172151927 shutdown_connections 2023-11-08T19:26:49.243 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.235+0000 7f243f06d700 1 -- 172.21.15.143:0/1172151927 wait complete. 2023-11-08T19:26:49.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.236+0000 7f243f06d700 1 Processor -- start 2023-11-08T19:26:49.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.236+0000 7f243f06d700 1 -- start start 2023-11-08T19:26:49.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.237+0000 7f243f06d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f24380b2510 0x7f24381f0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:49.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.237+0000 7f243f06d700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f24381585e0 0x7f2438154ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:49.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.237+0000 7f243f06d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f243815e530 0x7f24380c3950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:49.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.237+0000 7f243f06d700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24380c33f0 con 0x7f24380b2510 2023-11-08T19:26:49.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.237+0000 7f243f06d700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2438156890 con 0x7f243815e530 2023-11-08T19:26:49.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.237+0000 7f243f06d700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24381f1170 con 0x7f24381585e0 2023-11-08T19:26:49.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.237+0000 7f243ce09700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f24380b2510 0x7f24381f0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:49.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.237+0000 7f243ce09700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f24380b2510 0x7f24381f0c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52270/0 (socket says 172.21.15.143:52270) 2023-11-08T19:26:49.249 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.237+0000 7f243ce09700 1 -- 172.21.15.143:0/2112333822 learned_addr learned my addr 172.21.15.143:0/2112333822 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:49.249 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.237+0000 7f243d60a700 1 --2- 172.21.15.143:0/2112333822 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f243815e530 0x7f24380c3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:49.249 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.237+0000 7f2437fff700 1 --2- 172.21.15.143:0/2112333822 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f24381585e0 0x7f2438154ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:49.249 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.237+0000 7f243ce09700 1 -- 172.21.15.143:0/2112333822 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f24381585e0 msgr2=0x7f2438154ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:49.250 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.237+0000 7f243ce09700 1 --2- 172.21.15.143:0/2112333822 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f24381585e0 0x7f2438154ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:49.250 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.237+0000 7f243ce09700 1 -- 172.21.15.143:0/2112333822 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f243815e530 msgr2=0x7f24380c3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:49.250 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.237+0000 7f243ce09700 1 --2- 172.21.15.143:0/2112333822 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f243815e530 0x7f24380c3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:49.251 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.237+0000 7f243ce09700 1 -- 172.21.15.143:0/2112333822 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f242c0094a0 con 0x7f24380b2510 2023-11-08T19:26:49.251 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.237+0000 7f2437fff700 1 --2- 172.21.15.143:0/2112333822 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f24381585e0 0x7f2438154ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:26:49.251 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.238+0000 7f243ce09700 1 --2- 172.21.15.143:0/2112333822 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f24380b2510 0x7f24381f0c30 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f242c00ebe0 tx=0x7f242c009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:49.251 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.238+0000 7f24357fa700 1 -- 172.21.15.143:0/2112333822 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f242c00c910 con 0x7f24380b2510 2023-11-08T19:26:49.252 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.238+0000 7f24357fa700 1 -- 172.21.15.143:0/2112333822 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f242c00ca70 con 0x7f24380b2510 2023-11-08T19:26:49.252 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.238+0000 7f243f06d700 1 -- 172.21.15.143:0/2112333822 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24240094a0 con 0x7f24380b2510 2023-11-08T19:26:49.252 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.238+0000 7f24357fa700 1 -- 172.21.15.143:0/2112333822 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f242c00cde0 con 0x7f24380b2510 2023-11-08T19:26:49.252 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.238+0000 7f243f06d700 1 -- 172.21.15.143:0/2112333822 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2438155ad0 con 0x7f24380b2510 2023-11-08T19:26:49.253 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.239+0000 7f243f06d700 1 -- 172.21.15.143:0/2112333822 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f243804eee0 con 0x7f24380b2510 2023-11-08T19:26:49.253 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.239+0000 7f24357fa700 1 -- 172.21.15.143:0/2112333822 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f242c013470 con 0x7f24380b2510 2023-11-08T19:26:49.253 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.240+0000 7f24357fa700 1 --2- 172.21.15.143:0/2112333822 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f241c06af00 0x7f241c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:49.253 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.240+0000 7f24357fa700 1 -- 172.21.15.143:0/2112333822 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f242c0895e0 con 0x7f24380b2510 2023-11-08T19:26:49.254 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.240+0000 7f2437fff700 1 --2- 172.21.15.143:0/2112333822 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f241c06af00 0x7f241c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:49.254 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.240+0000 7f2437fff700 1 --2- 172.21.15.143:0/2112333822 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f241c06af00 0x7f241c06d3c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f24240090b0 tx=0x7f242401c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:49.254 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.244+0000 7f24357fa700 1 -- 172.21.15.143:0/2112333822 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f242c054830 con 0x7f24380b2510 2023-11-08T19:26:49.436 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.434+0000 7f243f06d700 1 -- 172.21.15.143:0/2112333822 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2438072910 con 0x7f24380b2510 2023-11-08T19:26:49.436 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.435+0000 7f24357fa700 1 -- 172.21.15.143:0/2112333822 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) v1 ==== 74+0+14789 (secure 0 0 0) 0x7f242c057e50 con 0x7f24380b2510 2023-11-08T19:26:49.437 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:26:49.437 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":53,"fsid":"968483af-77fc-4068-ad7c-5c2a1a4613d7","created":"2023-11-08T19:24:32.826484+0000","modified":"2023-11-08T19:26:48.091855+0000","last_up_change":"2023-11-08T19:24:43.573880+0000","last_in_change":"2023-11-08T19:24:35.314525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:44.551530+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"snap-test-pool","create_time":"2023-11-08T19:26:02.874384+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":36,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bae98ec1-d94e-403d-a220-a522a53ea6f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6809","nonce":1487830566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6811","nonce":1487830566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6815","nonce":1487830566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6813","nonce":1487830566}]},"public_addr":"172.21.15.143:6809/1487830566","cluster_addr":"172.21.15.143:6811/1487830566","heartbeat_back_addr":"172.21.15.143:6815/1487830566","heartbeat_front_addr":"172.21.15.143:6813/1487830566","state":["exists","up"]},{"osd":1,"uuid":"5cb6538a-1ccb-4322-8633-09f66af0b7e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6801","nonce":787341397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6803","nonce":787341397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6807","nonce":787341397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6805","nonce":787341397}]},"public_addr":"172.21.15.143:6801/787341397","cluster_addr":"172.21.15.143:6803/787341397","heartbeat_back_addr":"172.21.15.143:6807/787341397","heartbeat_front_addr":"172.21.15.143:6805/787341397","state":["exists","up"]},{"osd":2,"uuid":"cdbf522c-60ef-4d60-9653-8dcb529c101a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6822","nonce":2301325597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6826","nonce":2301325597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6831","nonce":2301325597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6829","nonce":2301325597}]},"public_addr":"172.21.15.143:6822/2301325597","cluster_addr":"172.21.15.143:6826/2301325597","heartbeat_back_addr":"172.21.15.143:6831/2301325597","heartbeat_front_addr":"172.21.15.143:6829/2301325597","state":["exists","up"]},{"osd":3,"uuid":"4e2783cf-ebb9-4dd8-8245-bc1d17ce4325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6817","nonce":2021504428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6819","nonce":2021504428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6827","nonce":2021504428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6823","nonce":2021504428}]},"public_addr":"172.21.15.143:6817/2021504428","cluster_addr":"172.21.15.143:6819/2021504428","heartbeat_back_addr":"172.21.15.143:6827/2021504428","heartbeat_front_addr":"172.21.15.143:6823/2021504428","state":["exists","up"]},{"osd":4,"uuid":"691256fa-9b77-49b9-a346-7ae1adf0ca80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6801","nonce":1993380595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6803","nonce":1993380595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6807","nonce":1993380595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6805","nonce":1993380595}]},"public_addr":"172.21.15.174:6801/1993380595","cluster_addr":"172.21.15.174:6803/1993380595","heartbeat_back_addr":"172.21.15.174:6807/1993380595","heartbeat_front_addr":"172.21.15.174:6805/1993380595","state":["exists","up"]},{"osd":5,"uuid":"756fc352-48c9-46ce-b52e-0541fe6f8525","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6809","nonce":1777923303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6811","nonce":1777923303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6815","nonce":1777923303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6813","nonce":1777923303}]},"public_addr":"172.21.15.174:6809/1777923303","cluster_addr":"172.21.15.174:6811/1777923303","heartbeat_back_addr":"172.21.15.174:6815/1777923303","heartbeat_front_addr":"172.21.15.174:6813/1777923303","state":["exists","up"]},{"osd":6,"uuid":"fab2ac2b-ed97-4cde-9825-b480eabf8b8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6825","nonce":2755130316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6827","nonce":2755130316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6831","nonce":2755130316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6829","nonce":2755130316}]},"public_addr":"172.21.15.174:6825/2755130316","cluster_addr":"172.21.15.174:6827/2755130316","heartbeat_back_addr":"172.21.15.174:6831/2755130316","heartbeat_front_addr":"172.21.15.174:6829/2755130316","state":["exists","up"]},{"osd":7,"uuid":"3de9b797-2ab6-4d41-a863-7e7d638fd703","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6817","nonce":2847141411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6819","nonce":2847141411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6823","nonce":2847141411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6821","nonce":2847141411}]},"public_addr":"172.21.15.174:6817/2847141411","cluster_addr":"172.21.15.174:6819/2847141411","heartbeat_back_addr":"172.21.15.174:6823/2847141411","heartbeat_front_addr":"172.21.15.174:6821/2847141411","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.477076+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.496890+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.539305+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.454673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.538881+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.521294+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.520712+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.540404+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.143:6834/3591097261":"2023-11-09T19:26:45.076217+0000","172.21.15.143:6832/3591097261":"2023-11-09T19:26:45.076217+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:26:49.439 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.437+0000 7f243f06d700 1 -- 172.21.15.143:0/2112333822 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f241c06af00 msgr2=0x7f241c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:49.439 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.437+0000 7f243f06d700 1 --2- 172.21.15.143:0/2112333822 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f241c06af00 0x7f241c06d3c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f24240090b0 tx=0x7f242401c040 comp rx=0 tx=0).stop 2023-11-08T19:26:49.440 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.437+0000 7f243f06d700 1 -- 172.21.15.143:0/2112333822 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f24380b2510 msgr2=0x7f24381f0c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:49.440 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.437+0000 7f243f06d700 1 --2- 172.21.15.143:0/2112333822 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f24380b2510 0x7f24381f0c30 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f242c00ebe0 tx=0x7f242c009ec0 comp rx=0 tx=0).stop 2023-11-08T19:26:49.440 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.438+0000 7f243f06d700 1 -- 172.21.15.143:0/2112333822 shutdown_connections 2023-11-08T19:26:49.441 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.438+0000 7f243f06d700 1 --2- 172.21.15.143:0/2112333822 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f241c06af00 0x7f241c06d3c0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:49.441 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.438+0000 7f243f06d700 1 --2- 172.21.15.143:0/2112333822 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f243815e530 0x7f24380c3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:49.441 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.438+0000 7f243f06d700 1 --2- 172.21.15.143:0/2112333822 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f24380b2510 0x7f24381f0c30 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:49.441 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.438+0000 7f243f06d700 1 --2- 172.21.15.143:0/2112333822 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f24381585e0 0x7f2438154ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:49.442 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.438+0000 7f243f06d700 1 -- 172.21.15.143:0/2112333822 >> 172.21.15.143:0/2112333822 conn(0x7f24380b1a20 msgr2=0x7f243806e330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:49.442 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.438+0000 7f243f06d700 1 -- 172.21.15.143:0/2112333822 shutdown_connections 2023-11-08T19:26:49.442 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.438+0000 7f243f06d700 1 -- 172.21.15.143:0/2112333822 wait complete. 2023-11-08T19:26:49.451 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T19:24:44.551530+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'snap-test-pool', 'create_time': '2023-11-08T19:26:02.874384+0000', 'flags': 16385, 'flags_names': 'hashpspool,pool_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '36', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'pool', 'snap_seq': 2, 'snap_epoch': 36, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-08T19:26:49.452 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T19:26:49.576 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.575+0000 7ff771df4700 1 Processor -- start 2023-11-08T19:26:49.576 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.575+0000 7ff771df4700 1 -- start start 2023-11-08T19:26:49.579 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.575+0000 7ff771df4700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff76c0b61e0 0x7ff76c0d01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:49.579 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.575+0000 7ff771df4700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff76c0cc2a0 con 0x7ff76c06bd20 2023-11-08T19:26:49.579 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.575+0000 7ff771df4700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff76c06c1b0 con 0x7ff76c15a950 2023-11-08T19:26:49.579 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.575+0000 7ff771df4700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff76c0cc130 con 0x7ff76c0b61e0 2023-11-08T19:26:49.580 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.575+0000 7ff76bfff700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff76c0b61e0 0x7ff76c0d01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:49.580 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.575+0000 7ff76bfff700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff76c0b61e0 0x7ff76c0d01b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52282/0 (socket says 172.21.15.143:52282) 2023-11-08T19:26:49.580 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.575+0000 7ff76bfff700 1 -- 172.21.15.143:0/1078254560 learned_addr learned my addr 172.21.15.143:0/1078254560 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:49.580 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.575+0000 7ff76affd700 1 --1- 172.21.15.143:0/1078254560 >> v1:172.21.15.143:6789/0 conn(0x7ff76c06bd20 0x7ff76c0cc730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:48434/0 (socket says 172.21.15.143:48434) 2023-11-08T19:26:49.581 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.576+0000 7ff76bfff700 1 -- 172.21.15.143:0/1078254560 >> v1:172.21.15.174:6789/0 conn(0x7ff76c15a950 legacy=0x7ff76c072370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:49.581 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.576+0000 7ff76bfff700 1 -- 172.21.15.143:0/1078254560 >> v1:172.21.15.143:6789/0 conn(0x7ff76c06bd20 legacy=0x7ff76c0cc730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:49.581 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.576+0000 7ff76bfff700 1 -- 172.21.15.143:0/1078254560 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff76c06c490 con 0x7ff76c0b61e0 2023-11-08T19:26:49.581 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.576+0000 7ff76a7fc700 1 -- 172.21.15.143:0/1078254560 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2261885202 0 0) 0x7ff76c0cc2a0 con 0x7ff76c06bd20 2023-11-08T19:26:49.581 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.576+0000 7ff76bfff700 1 --2- 172.21.15.143:0/1078254560 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff76c0b61e0 0x7ff76c0d01b0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7ff76400d150 tx=0x7ff76400d460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=944c6cfcf9df5bcb server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:49.582 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.576+0000 7ff76a7fc700 1 -- 172.21.15.143:0/1078254560 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff764010070 con 0x7ff76c0b61e0 2023-11-08T19:26:49.582 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.576+0000 7ff76a7fc700 1 -- 172.21.15.143:0/1078254560 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff76400dd80 con 0x7ff76c0b61e0 2023-11-08T19:26:49.582 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.576+0000 7ff76a7fc700 1 -- 172.21.15.143:0/1078254560 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff76400b310 con 0x7ff76c0b61e0 2023-11-08T19:26:49.583 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.577+0000 7ff771df4700 1 -- 172.21.15.143:0/1078254560 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff76c0b61e0 msgr2=0x7ff76c0d01b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:49.583 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.577+0000 7ff771df4700 1 --2- 172.21.15.143:0/1078254560 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff76c0b61e0 0x7ff76c0d01b0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7ff76400d150 tx=0x7ff76400d460 comp rx=0 tx=0).stop 2023-11-08T19:26:49.583 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.577+0000 7ff771df4700 1 -- 172.21.15.143:0/1078254560 shutdown_connections 2023-11-08T19:26:49.583 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.577+0000 7ff771df4700 1 --2- 172.21.15.143:0/1078254560 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff76c0b61e0 0x7ff76c0d01b0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:49.584 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.577+0000 7ff771df4700 1 -- 172.21.15.143:0/1078254560 >> 172.21.15.143:0/1078254560 conn(0x7ff76c0cd2e0 msgr2=0x7ff76c0cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:49.584 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.577+0000 7ff771df4700 1 -- 172.21.15.143:0/1078254560 shutdown_connections 2023-11-08T19:26:49.584 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.577+0000 7ff771df4700 1 -- 172.21.15.143:0/1078254560 wait complete. 2023-11-08T19:26:49.584 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.578+0000 7ff771df4700 1 Processor -- start 2023-11-08T19:26:49.587 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.578+0000 7ff771df4700 1 -- start start 2023-11-08T19:26:49.587 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.579+0000 7ff771df4700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff76c0b61e0 0x7ff76c156430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:49.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.579+0000 7ff771df4700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff76c15a950 0x7ff76c0ca080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:49.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.579+0000 7ff771df4700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff76c156ad0 0x7ff76c0c5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:49.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.579+0000 7ff771df4700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff76c0c5410 con 0x7ff76c15a950 2023-11-08T19:26:49.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.579+0000 7ff771df4700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff76c156970 con 0x7ff76c0b61e0 2023-11-08T19:26:49.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.579+0000 7ff771df4700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff76c0c4820 con 0x7ff76c156ad0 2023-11-08T19:26:49.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.579+0000 7ff76affd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff76c15a950 0x7ff76c0ca080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:49.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.579+0000 7ff76affd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff76c15a950 0x7ff76c0ca080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52286/0 (socket says 172.21.15.143:52286) 2023-11-08T19:26:49.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.579+0000 7ff76affd700 1 -- 172.21.15.143:0/1515808083 learned_addr learned my addr 172.21.15.143:0/1515808083 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:49.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.579+0000 7ff76b7fe700 1 --2- 172.21.15.143:0/1515808083 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff76c0b61e0 0x7ff76c156430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:49.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.579+0000 7ff76bfff700 1 --2- 172.21.15.143:0/1515808083 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff76c156ad0 0x7ff76c0c5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:49.590 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.579+0000 7ff76affd700 1 -- 172.21.15.143:0/1515808083 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff76c156ad0 msgr2=0x7ff76c0c5830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:49.590 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.579+0000 7ff76affd700 1 --2- 172.21.15.143:0/1515808083 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff76c156ad0 0x7ff76c0c5830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:49.590 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.579+0000 7ff76affd700 1 -- 172.21.15.143:0/1515808083 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff76c0b61e0 msgr2=0x7ff76c156430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:49.590 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.579+0000 7ff76affd700 1 --2- 172.21.15.143:0/1515808083 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff76c0b61e0 0x7ff76c156430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:49.591 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.579+0000 7ff76affd700 1 -- 172.21.15.143:0/1515808083 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff76400bd40 con 0x7ff76c15a950 2023-11-08T19:26:49.591 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.579+0000 7ff76affd700 1 --2- 172.21.15.143:0/1515808083 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff76c15a950 0x7ff76c0ca080 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7ff75400c750 tx=0x7ff75400ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:49.591 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.579+0000 7ff763fff700 1 -- 172.21.15.143:0/1515808083 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff754004c80 con 0x7ff76c15a950 2023-11-08T19:26:49.591 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.580+0000 7ff763fff700 1 -- 172.21.15.143:0/1515808083 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff754004e00 con 0x7ff76c15a950 2023-11-08T19:26:49.592 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.580+0000 7ff763fff700 1 -- 172.21.15.143:0/1515808083 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff754007310 con 0x7ff76c15a950 2023-11-08T19:26:49.593 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.580+0000 7ff771df4700 1 -- 172.21.15.143:0/1515808083 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff76c0c4a70 con 0x7ff76c15a950 2023-11-08T19:26:49.593 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.580+0000 7ff771df4700 1 -- 172.21.15.143:0/1515808083 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff76c0c5030 con 0x7ff76c15a950 2023-11-08T19:26:49.593 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.581+0000 7ff771df4700 1 -- 172.21.15.143:0/1515808083 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff76c0c5550 con 0x7ff76c15a950 2023-11-08T19:26:49.594 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.581+0000 7ff763fff700 1 -- 172.21.15.143:0/1515808083 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff754007470 con 0x7ff76c15a950 2023-11-08T19:26:49.594 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.581+0000 7ff763fff700 1 --2- 172.21.15.143:0/1515808083 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7ff74c06af00 0x7ff74c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:49.594 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.582+0000 7ff763fff700 1 -- 172.21.15.143:0/1515808083 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5296+0+0 (secure 0 0 0) 0x7ff754089020 con 0x7ff76c15a950 2023-11-08T19:26:49.594 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.582+0000 7ff76b7fe700 1 --2- 172.21.15.143:0/1515808083 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7ff74c06af00 0x7ff74c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:49.594 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.583+0000 7ff76b7fe700 1 --2- 172.21.15.143:0/1515808083 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7ff74c06af00 0x7ff74c06d3c0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ff75c009960 tx=0x7ff75c0090e0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:49.595 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.586+0000 7ff763fff700 1 -- 172.21.15.143:0/1515808083 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff76c0c5550 con 0x7ff76c15a950 2023-11-08T19:26:49.777 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.776+0000 7ff771df4700 1 -- 172.21.15.143:0/1515808083 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff76c0c5550 con 0x7ff76c15a950 2023-11-08T19:26:49.778 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.777+0000 7ff763fff700 1 -- 172.21.15.143:0/1515808083 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v53) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff76c0c5550 con 0x7ff76c15a950 2023-11-08T19:26:49.779 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2023-11-08T19:26:49.780 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.779+0000 7ff771df4700 1 -- 172.21.15.143:0/1515808083 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7ff74c06af00 msgr2=0x7ff74c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:49.781 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.779+0000 7ff771df4700 1 --2- 172.21.15.143:0/1515808083 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7ff74c06af00 0x7ff74c06d3c0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ff75c009960 tx=0x7ff75c0090e0 comp rx=0 tx=0).stop 2023-11-08T19:26:49.781 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.779+0000 7ff771df4700 1 -- 172.21.15.143:0/1515808083 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff76c15a950 msgr2=0x7ff76c0ca080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:49.781 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.779+0000 7ff771df4700 1 --2- 172.21.15.143:0/1515808083 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff76c15a950 0x7ff76c0ca080 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7ff75400c750 tx=0x7ff75400ca60 comp rx=0 tx=0).stop 2023-11-08T19:26:49.781 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.779+0000 7ff771df4700 1 -- 172.21.15.143:0/1515808083 shutdown_connections 2023-11-08T19:26:49.781 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.779+0000 7ff771df4700 1 --2- 172.21.15.143:0/1515808083 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7ff74c06af00 0x7ff74c06d3c0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:49.782 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.779+0000 7ff771df4700 1 --2- 172.21.15.143:0/1515808083 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff76c156ad0 0x7ff76c0c5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:49.782 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.779+0000 7ff771df4700 1 --2- 172.21.15.143:0/1515808083 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff76c0b61e0 0x7ff76c156430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:49.782 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.779+0000 7ff771df4700 1 --2- 172.21.15.143:0/1515808083 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff76c15a950 0x7ff76c0ca080 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:49.782 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.779+0000 7ff771df4700 1 -- 172.21.15.143:0/1515808083 >> 172.21.15.143:0/1515808083 conn(0x7ff76c0cd2e0 msgr2=0x7ff76c0ceec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:49.783 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.779+0000 7ff771df4700 1 -- 172.21.15.143:0/1515808083 shutdown_connections 2023-11-08T19:26:49.783 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.779+0000 7ff771df4700 1 -- 172.21.15.143:0/1515808083 wait complete. 2023-11-08T19:26:49.792 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get snap-test-pool pg_num 2023-11-08T19:26:49.915 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.913+0000 7fcdb4fdb700 1 Processor -- start 2023-11-08T19:26:49.915 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.913+0000 7fcdb4fdb700 1 -- start start 2023-11-08T19:26:49.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.914+0000 7fcdb4fdb700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fcdb006bd20 0x7fcdb00d01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:49.923 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.914+0000 7fcdb4fdb700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fcdb00b61e0 0x7fcdb00cc730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:49.923 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.914+0000 7fcdb4fdb700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdb00cc2a0 con 0x7fcdb015a950 2023-11-08T19:26:49.923 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.914+0000 7fcdb4fdb700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdb00cc130 con 0x7fcdb00b61e0 2023-11-08T19:26:49.923 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.914+0000 7fcdb4fdb700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdb006c1b0 con 0x7fcdb006bd20 2023-11-08T19:26:49.924 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.914+0000 7fcdaed9d700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fcdb00b61e0 0x7fcdb00cc730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:49.924 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.914+0000 7fcdae59c700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fcdb015a950 0x7fcdb0072370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:48446/0 (socket says 172.21.15.143:48446) 2023-11-08T19:26:49.924 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.915+0000 7fcdaed9d700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fcdb00b61e0 0x7fcdb00cc730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52296/0 (socket says 172.21.15.143:52296) 2023-11-08T19:26:49.924 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.915+0000 7fcdae59c700 1 -- 172.21.15.143:0/3728417855 learned_addr learned my addr 172.21.15.143:0/3728417855 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:26:49.924 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.915+0000 7fcdadd9b700 1 --2- 172.21.15.143:0/3728417855 >> v2:172.21.15.174:3300/0 conn(0x7fcdb006bd20 0x7fcdb00d01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:49.924 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.915+0000 7fcdaed9d700 1 -- 172.21.15.143:0/3728417855 >> v2:172.21.15.174:3300/0 conn(0x7fcdb006bd20 msgr2=0x7fcdb00d01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:49.925 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.915+0000 7fcdaed9d700 1 --2- 172.21.15.143:0/3728417855 >> v2:172.21.15.174:3300/0 conn(0x7fcdb006bd20 0x7fcdb00d01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:49.925 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.915+0000 7fcdaed9d700 1 -- 172.21.15.143:0/3728417855 >> v1:172.21.15.143:6789/0 conn(0x7fcdb015a950 legacy=0x7fcdb0072370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:49.925 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.915+0000 7fcdaed9d700 1 -- 172.21.15.143:0/3728417855 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdb006c490 con 0x7fcdb00b61e0 2023-11-08T19:26:49.925 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.915+0000 7fcdaed9d700 1 --2- 172.21.15.143:0/3728417855 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcdb00b61e0 0x7fcdb00cc730 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fcdb00cd240 tx=0x7fcda800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b41238e5607e8f4 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:49.925 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.915+0000 7fcdad59a700 1 -- 172.21.15.143:0/3728417855 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcda800e070 con 0x7fcdb00b61e0 2023-11-08T19:26:49.926 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.915+0000 7fcdad59a700 1 -- 172.21.15.143:0/3728417855 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcda800bbd0 con 0x7fcdb00b61e0 2023-11-08T19:26:49.926 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.915+0000 7fcdad59a700 1 -- 172.21.15.143:0/3728417855 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcda8012430 con 0x7fcdb00b61e0 2023-11-08T19:26:49.926 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.916+0000 7fcdb4fdb700 1 -- 172.21.15.143:0/3728417855 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcdb00b61e0 msgr2=0x7fcdb00cc730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:49.926 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.916+0000 7fcdb4fdb700 1 --2- 172.21.15.143:0/3728417855 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcdb00b61e0 0x7fcdb00cc730 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fcdb00cd240 tx=0x7fcda800b330 comp rx=0 tx=0).stop 2023-11-08T19:26:49.926 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.916+0000 7fcdb4fdb700 1 -- 172.21.15.143:0/3728417855 shutdown_connections 2023-11-08T19:26:49.927 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.916+0000 7fcdb4fdb700 1 --2- 172.21.15.143:0/3728417855 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcdb00b61e0 0x7fcdb00cc730 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:49.927 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.916+0000 7fcdb4fdb700 1 --2- 172.21.15.143:0/3728417855 >> v2:172.21.15.174:3300/0 conn(0x7fcdb006bd20 0x7fcdb00d01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:49.927 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.916+0000 7fcdb4fdb700 1 -- 172.21.15.143:0/3728417855 >> 172.21.15.143:0/3728417855 conn(0x7fcdb00cd2e0 msgr2=0x7fcdb00cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:49.927 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.916+0000 7fcdb4fdb700 1 -- 172.21.15.143:0/3728417855 shutdown_connections 2023-11-08T19:26:49.927 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.916+0000 7fcdb4fdb700 1 -- 172.21.15.143:0/3728417855 wait complete. 2023-11-08T19:26:49.928 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.917+0000 7fcdb4fdb700 1 Processor -- start 2023-11-08T19:26:49.928 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.917+0000 7fcdb4fdb700 1 -- start start 2023-11-08T19:26:49.929 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.918+0000 7fcdb4fdb700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcdb00b61e0 0x7fcdb00c9600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:49.929 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.918+0000 7fcdb4fdb700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcdb015a950 0x7fcdb00ce800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:49.929 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.918+0000 7fcdb4fdb700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcdb00b2360 0x7fcdb00b2810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:49.929 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.918+0000 7fcdb4fdb700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdb01f0050 con 0x7fcdb015a950 2023-11-08T19:26:49.929 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.918+0000 7fcdb4fdb700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdb00b2d50 con 0x7fcdb00b61e0 2023-11-08T19:26:49.930 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.918+0000 7fcdb4fdb700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdb01efd70 con 0x7fcdb00b2360 2023-11-08T19:26:49.930 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.918+0000 7fcdadd9b700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcdb015a950 0x7fcdb00ce800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:49.930 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.918+0000 7fcdadd9b700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcdb015a950 0x7fcdb00ce800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52298/0 (socket says 172.21.15.143:52298) 2023-11-08T19:26:49.931 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.918+0000 7fcdadd9b700 1 -- 172.21.15.143:0/726222522 learned_addr learned my addr 172.21.15.143:0/726222522 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:49.931 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.918+0000 7fcdae59c700 1 --2- 172.21.15.143:0/726222522 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcdb00b61e0 0x7fcdb00c9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:49.931 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.918+0000 7fcdaed9d700 1 --2- 172.21.15.143:0/726222522 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcdb00b2360 0x7fcdb00b2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:49.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.918+0000 7fcdadd9b700 1 -- 172.21.15.143:0/726222522 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcdb00b2360 msgr2=0x7fcdb00b2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:49.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.918+0000 7fcdadd9b700 1 --2- 172.21.15.143:0/726222522 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcdb00b2360 0x7fcdb00b2810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:49.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.918+0000 7fcdadd9b700 1 -- 172.21.15.143:0/726222522 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcdb00b61e0 msgr2=0x7fcdb00c9600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:49.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.918+0000 7fcdadd9b700 1 --2- 172.21.15.143:0/726222522 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcdb00b61e0 0x7fcdb00c9600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:49.933 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.918+0000 7fcdadd9b700 1 -- 172.21.15.143:0/726222522 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcda8009a70 con 0x7fcdb015a950 2023-11-08T19:26:49.933 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.918+0000 7fcdadd9b700 1 --2- 172.21.15.143:0/726222522 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcdb015a950 0x7fcdb00ce800 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fcda000e810 tx=0x7fcda000ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:49.933 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.918+0000 7fcd9effd700 1 -- 172.21.15.143:0/726222522 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcda0013610 con 0x7fcdb015a950 2023-11-08T19:26:49.933 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.918+0000 7fcd9effd700 1 -- 172.21.15.143:0/726222522 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcda0013770 con 0x7fcdb015a950 2023-11-08T19:26:49.933 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.918+0000 7fcd9effd700 1 -- 172.21.15.143:0/726222522 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcda0013b20 con 0x7fcdb015a950 2023-11-08T19:26:49.934 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.919+0000 7fcdb4fdb700 1 -- 172.21.15.143:0/726222522 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdb00c8b20 con 0x7fcdb015a950 2023-11-08T19:26:49.934 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.919+0000 7fcdb4fdb700 1 -- 172.21.15.143:0/726222522 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdb00c90e0 con 0x7fcdb015a950 2023-11-08T19:26:49.934 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.920+0000 7fcdb4fdb700 1 -- 172.21.15.143:0/726222522 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdb00c8980 con 0x7fcdb015a950 2023-11-08T19:26:49.935 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.920+0000 7fcd9effd700 1 -- 172.21.15.143:0/726222522 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fcda0013c80 con 0x7fcdb015a950 2023-11-08T19:26:49.935 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.920+0000 7fcd9effd700 1 --2- 172.21.15.143:0/726222522 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fcd9006af00 0x7fcd9006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:49.935 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.920+0000 7fcd9effd700 1 -- 172.21.15.143:0/726222522 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fcda00890e0 con 0x7fcdb015a950 2023-11-08T19:26:49.935 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.921+0000 7fcdae59c700 1 --2- 172.21.15.143:0/726222522 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fcd9006af00 0x7fcd9006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:49.935 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.921+0000 7fcdae59c700 1 --2- 172.21.15.143:0/726222522 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fcd9006af00 0x7fcd9006d3c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fcda4000f60 tx=0x7fcda4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:49.936 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:49.925+0000 7fcd9effd700 1 -- 172.21.15.143:0/726222522 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcdb00c8980 con 0x7fcdb015a950 2023-11-08T19:26:50.115 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:50.114+0000 7fcdb4fdb700 1 -- 172.21.15.143:0/726222522 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"} v 0) v1 -- 0x7fcdb00c8980 con 0x7fcdb015a950 2023-11-08T19:26:50.116 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:50.114+0000 7fcd9effd700 1 -- 172.21.15.143:0/726222522 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"}]=0 v53) v1 ==== 103+0+11 (secure 0 0 0) 0x7fcdb00c8980 con 0x7fcdb015a950 2023-11-08T19:26:50.116 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2023-11-08T19:26:50.118 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:50.117+0000 7fcdb4fdb700 1 -- 172.21.15.143:0/726222522 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fcd9006af00 msgr2=0x7fcd9006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:50.119 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:50.117+0000 7fcdb4fdb700 1 --2- 172.21.15.143:0/726222522 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fcd9006af00 0x7fcd9006d3c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fcda4000f60 tx=0x7fcda4009040 comp rx=0 tx=0).stop 2023-11-08T19:26:50.119 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:50.117+0000 7fcdb4fdb700 1 -- 172.21.15.143:0/726222522 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcdb015a950 msgr2=0x7fcdb00ce800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:50.119 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:50.117+0000 7fcdb4fdb700 1 --2- 172.21.15.143:0/726222522 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcdb015a950 0x7fcdb00ce800 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fcda000e810 tx=0x7fcda000ebd0 comp rx=0 tx=0).stop 2023-11-08T19:26:50.119 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:50.117+0000 7fcdb4fdb700 1 -- 172.21.15.143:0/726222522 shutdown_connections 2023-11-08T19:26:50.120 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:50.117+0000 7fcdb4fdb700 1 --2- 172.21.15.143:0/726222522 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fcd9006af00 0x7fcd9006d3c0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:50.120 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:50.117+0000 7fcdb4fdb700 1 --2- 172.21.15.143:0/726222522 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcdb00b2360 0x7fcdb00b2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:50.120 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:50.117+0000 7fcdb4fdb700 1 --2- 172.21.15.143:0/726222522 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcdb00b61e0 0x7fcdb00c9600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:50.120 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:50.117+0000 7fcdb4fdb700 1 --2- 172.21.15.143:0/726222522 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcdb015a950 0x7fcdb00ce800 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:50.120 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:50.117+0000 7fcdb4fdb700 1 -- 172.21.15.143:0/726222522 >> 172.21.15.143:0/726222522 conn(0x7fcdb00cd2e0 msgr2=0x7fcdb015af70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:50.121 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:50.117+0000 7fcdb4fdb700 1 -- 172.21.15.143:0/726222522 shutdown_connections 2023-11-08T19:26:50.121 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:50.117+0000 7fcdb4fdb700 1 -- 172.21.15.143:0/726222522 wait complete. 2023-11-08T19:26:50.130 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snapshots.TestMonSnapsAndFsPools.test_using_pool_with_snap_fails_fs_creation' 2023-11-08T19:26:50.251 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:50.249+0000 7efcc7223700 1 Processor -- start 2023-11-08T19:26:50.251 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:50.249+0000 7efcc7223700 1 -- start start 2023-11-08T19:26:50.254 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:50.250+0000 7efcc7223700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7efcc015dd00 0x7efcc00cca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:50.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:50.250+0000 7efcc7223700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7efcc01618f0 0x7efcc006ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:50.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:50.250+0000 7efcc7223700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcc00ccfb0 con 0x7efcc006b150 2023-11-08T19:26:50.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:50.250+0000 7efcc7223700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcc006b600 con 0x7efcc015dd00 2023-11-08T19:26:50.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:50.250+0000 7efcc7223700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcc00cad50 con 0x7efcc01618f0 2023-11-08T19:26:50.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:50.250+0000 7efcc4fbf700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7efcc015dd00 0x7efcc00cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:50.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:50.250+0000 7efcc4fbf700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7efcc015dd00 0x7efcc00cca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52306/0 (socket says 172.21.15.143:52306) 2023-11-08T19:26:50.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:50.250+0000 7efcc4fbf700 1 -- 172.21.15.143:0/3722358451 learned_addr learned my addr 172.21.15.143:0/3722358451 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:50.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:50.251+0000 7efcc4fbf700 1 -- 172.21.15.143:0/3722358451 >> v2:172.21.15.174:3300/0 conn(0x7efcc01618f0 msgr2=0x7efcc006ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:50.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:50.251+0000 7efcc4fbf700 1 --2- 172.21.15.143:0/3722358451 >> v2:172.21.15.174:3300/0 conn(0x7efcc01618f0 0x7efcc006ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:50.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:50.251+0000 7efcc4fbf700 1 -- 172.21.15.143:0/3722358451 >> v1:172.21.15.174:6789/0 conn(0x7efcc006b150 legacy=0x7efcc00b2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:50.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:50.251+0000 7efcc4fbf700 1 -- 172.21.15.143:0/3722358451 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcc0161df0 con 0x7efcc015dd00 2023-11-08T19:26:50.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:50.251+0000 7efcc4fbf700 1 --2- 172.21.15.143:0/3722358451 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efcc015dd00 0x7efcc00cca70 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7efcc0161850 tx=0x7efcb4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5df84028b298edd7 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:50.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:50.251+0000 7efcbf7fe700 1 -- 172.21.15.143:0/3722358451 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcb4005e30 con 0x7efcc015dd00 2023-11-08T19:26:50.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:50.251+0000 7efcbf7fe700 1 -- 172.21.15.143:0/3722358451 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcb4006030 con 0x7efcc015dd00 2023-11-08T19:26:50.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:50.251+0000 7efcbf7fe700 1 -- 172.21.15.143:0/3722358451 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcb40063e0 con 0x7efcc015dd00 2023-11-08T19:26:50.261 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:50.251+0000 7efcc7223700 1 -- 172.21.15.143:0/3722358451 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efcc015dd00 msgr2=0x7efcc00cca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:50.261 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:50.251+0000 7efcc7223700 1 --2- 172.21.15.143:0/3722358451 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efcc015dd00 0x7efcc00cca70 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7efcc0161850 tx=0x7efcb4009bd0 comp rx=0 tx=0).stop 2023-11-08T19:26:50.261 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:50.252+0000 7efcc7223700 1 -- 172.21.15.143:0/3722358451 shutdown_connections 2023-11-08T19:26:50.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:50.252+0000 7efcc7223700 1 --2- 172.21.15.143:0/3722358451 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efcc015dd00 0x7efcc00cca70 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:50.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:50.252+0000 7efcc7223700 1 --2- 172.21.15.143:0/3722358451 >> v2:172.21.15.174:3300/0 conn(0x7efcc01618f0 0x7efcc006ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:50.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:50.252+0000 7efcc7223700 1 -- 172.21.15.143:0/3722358451 >> 172.21.15.143:0/3722358451 conn(0x7efcc0099ab0 msgr2=0x7efcc00d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:50.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:50.252+0000 7efcc7223700 1 -- 172.21.15.143:0/3722358451 shutdown_connections 2023-11-08T19:26:50.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:50.252+0000 7efcc7223700 1 -- 172.21.15.143:0/3722358451 wait complete. 2023-11-08T19:26:50.264 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:50.253+0000 7efcc7223700 1 Processor -- start 2023-11-08T19:26:50.264 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:50.253+0000 7efcc7223700 1 -- start start 2023-11-08T19:26:50.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:50.254+0000 7efcc7223700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efcc006b150 0x7efcc0152de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:50.268 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:50.254+0000 7efcc7223700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efcc015dd00 0x7efcc01e8190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:50.268 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:50.254+0000 7efcc7223700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efcc01618f0 0x7efcc00b24d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:50.268 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:50.254+0000 7efcc7223700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcc015e180 con 0x7efcc01618f0 2023-11-08T19:26:50.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:50.254+0000 7efcc7223700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcc00b05d0 con 0x7efcc015dd00 2023-11-08T19:26:50.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:50.254+0000 7efcc7223700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcc00b08b0 con 0x7efcc006b150 2023-11-08T19:26:50.270 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:50.254+0000 7efcc57c0700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efcc01618f0 0x7efcc00b24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:50.270 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:50.254+0000 7efcc57c0700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efcc01618f0 0x7efcc00b24d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52316/0 (socket says 172.21.15.143:52316) 2023-11-08T19:26:50.270 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:50.254+0000 7efcc57c0700 1 -- 172.21.15.143:0/2442132544 learned_addr learned my addr 172.21.15.143:0/2442132544 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:50.270 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:50.254+0000 7efcc4fbf700 1 --2- 172.21.15.143:0/2442132544 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efcc006b150 0x7efcc0152de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:50.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:50.254+0000 7efcbffff700 1 --2- 172.21.15.143:0/2442132544 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efcc015dd00 0x7efcc01e8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:50.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:50.254+0000 7efcc57c0700 1 -- 172.21.15.143:0/2442132544 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efcc006b150 msgr2=0x7efcc0152de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:50.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:50.254+0000 7efcc57c0700 1 --2- 172.21.15.143:0/2442132544 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efcc006b150 0x7efcc0152de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:50.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:50.254+0000 7efcc57c0700 1 -- 172.21.15.143:0/2442132544 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efcc015dd00 msgr2=0x7efcc01e8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:50.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:50.254+0000 7efcc57c0700 1 --2- 172.21.15.143:0/2442132544 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efcc015dd00 0x7efcc01e8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:50.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:50.254+0000 7efcc57c0700 1 -- 172.21.15.143:0/2442132544 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcb4009500 con 0x7efcc01618f0 2023-11-08T19:26:50.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:50.254+0000 7efcc57c0700 1 --2- 172.21.15.143:0/2442132544 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efcc01618f0 0x7efcc00b24d0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7efcb800b990 tx=0x7efcb800bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:50.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:50.254+0000 7efcbd7fa700 1 -- 172.21.15.143:0/2442132544 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcb800c9d0 con 0x7efcc01618f0 2023-11-08T19:26:50.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:50.254+0000 7efcbd7fa700 1 -- 172.21.15.143:0/2442132544 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcb800cb30 con 0x7efcc01618f0 2023-11-08T19:26:50.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:50.254+0000 7efcc7223700 1 -- 172.21.15.143:0/2442132544 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcc00b0b90 con 0x7efcc01618f0 2023-11-08T19:26:50.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:50.254+0000 7efcbd7fa700 1 -- 172.21.15.143:0/2442132544 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcb8014440 con 0x7efcc01618f0 2023-11-08T19:26:50.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:50.254+0000 7efcc7223700 1 -- 172.21.15.143:0/2442132544 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcc0153320 con 0x7efcc01618f0 2023-11-08T19:26:50.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:50.255+0000 7efcc7223700 1 -- 172.21.15.143:0/2442132544 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcc004eee0 con 0x7efcc01618f0 2023-11-08T19:26:50.274 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:50.256+0000 7efcbd7fa700 1 -- 172.21.15.143:0/2442132544 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7efcb8006d50 con 0x7efcc01618f0 2023-11-08T19:26:50.274 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:50.256+0000 7efcbd7fa700 1 --2- 172.21.15.143:0/2442132544 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7efca406af00 0x7efca406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:50.274 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:50.256+0000 7efcbd7fa700 1 -- 172.21.15.143:0/2442132544 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5296+0+0 (secure 0 0 0) 0x7efcb8088fd0 con 0x7efcc01618f0 2023-11-08T19:26:50.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:50.256+0000 7efcc4fbf700 1 --2- 172.21.15.143:0/2442132544 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7efca406af00 0x7efca406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:50.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:50.257+0000 7efcc4fbf700 1 --2- 172.21.15.143:0/2442132544 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7efca406af00 0x7efca406d3c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7efcc0161850 tx=0x7efcb4008070 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:50.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:50.261+0000 7efcbd7fa700 1 -- 172.21.15.143:0/2442132544 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7efcb80541a0 con 0x7efcc01618f0 2023-11-08T19:26:50.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:50.530+0000 7efcc7223700 1 -- 172.21.15.143:0/2442132544 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snapshots.TestMonSnapsAndFsPools.test_using_pool_with_snap_fails_fs_creation"]} v 0) v1 -- 0x7efcc006b920 con 0x7efcc01618f0 2023-11-08T19:26:51.103 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.102+0000 7efcbd7fa700 1 -- 172.21.15.143:0/2442132544 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snapshots.TestMonSnapsAndFsPools.test_using_pool_with_snap_fails_fs_creation"]}]=0 v132) v1 ==== 176+0+0 (secure 0 0 0) 0x7efcb80577c0 con 0x7efcc01618f0 2023-11-08T19:26:51.106 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.104+0000 7efcc7223700 1 -- 172.21.15.143:0/2442132544 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7efca406af00 msgr2=0x7efca406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:51.106 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.104+0000 7efcc7223700 1 --2- 172.21.15.143:0/2442132544 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7efca406af00 0x7efca406d3c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7efcc0161850 tx=0x7efcb4008070 comp rx=0 tx=0).stop 2023-11-08T19:26:51.107 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.104+0000 7efcc7223700 1 -- 172.21.15.143:0/2442132544 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efcc01618f0 msgr2=0x7efcc00b24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:51.107 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.104+0000 7efcc7223700 1 --2- 172.21.15.143:0/2442132544 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efcc01618f0 0x7efcc00b24d0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7efcb800b990 tx=0x7efcb800bd50 comp rx=0 tx=0).stop 2023-11-08T19:26:51.107 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.104+0000 7efcc7223700 1 -- 172.21.15.143:0/2442132544 shutdown_connections 2023-11-08T19:26:51.107 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.104+0000 7efcc7223700 1 --2- 172.21.15.143:0/2442132544 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7efca406af00 0x7efca406d3c0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:51.108 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.104+0000 7efcc7223700 1 --2- 172.21.15.143:0/2442132544 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efcc006b150 0x7efcc0152de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:51.108 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.104+0000 7efcc7223700 1 --2- 172.21.15.143:0/2442132544 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efcc01618f0 0x7efcc00b24d0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:51.108 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.104+0000 7efcc7223700 1 --2- 172.21.15.143:0/2442132544 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efcc015dd00 0x7efcc01e8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:51.108 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.104+0000 7efcc7223700 1 -- 172.21.15.143:0/2442132544 >> 172.21.15.143:0/2442132544 conn(0x7efcc0099ab0 msgr2=0x7efcc00ad380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:51.109 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.105+0000 7efcc7223700 1 -- 172.21.15.143:0/2442132544 shutdown_connections 2023-11-08T19:26:51.109 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.105+0000 7efcc7223700 1 -- 172.21.15.143:0/2442132544 wait complete. 2023-11-08T19:26:51.120 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-08T19:26:51.252 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.247+0000 7f309606a700 1 Processor -- start 2023-11-08T19:26:51.252 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.247+0000 7f309606a700 1 -- start start 2023-11-08T19:26:51.252 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.248+0000 7f309606a700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f309006b0b0 0x7f309006c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:51.253 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.248+0000 7f309606a700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f30900b22e0 0x7f30900b2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:51.253 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.248+0000 7f309606a700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30900b2c50 con 0x7f309015bc80 2023-11-08T19:26:51.253 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.248+0000 7f309606a700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30900ce290 con 0x7f309006b0b0 2023-11-08T19:26:51.254 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.248+0000 7f309606a700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f309015c060 con 0x7f30900b22e0 2023-11-08T19:26:51.254 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.248+0000 7f308effd700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f309006b0b0 0x7f309006c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:51.254 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.248+0000 7f308f7fe700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f309015bc80 0x7f30900b6280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:48474/0 (socket says 172.21.15.143:48474) 2023-11-08T19:26:51.254 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.248+0000 7f308f7fe700 1 -- 172.21.15.143:0/299160624 learned_addr learned my addr 172.21.15.143:0/299160624 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:26:51.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.248+0000 7f308ffff700 1 --2- 172.21.15.143:0/299160624 >> v2:172.21.15.174:3300/0 conn(0x7f30900b22e0 0x7f30900b2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:51.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.248+0000 7f308effd700 1 -- 172.21.15.143:0/299160624 >> v2:172.21.15.174:3300/0 conn(0x7f30900b22e0 msgr2=0x7f30900b2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:51.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.248+0000 7f308effd700 1 --2- 172.21.15.143:0/299160624 >> v2:172.21.15.174:3300/0 conn(0x7f30900b22e0 0x7f30900b2710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:51.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.248+0000 7f308effd700 1 -- 172.21.15.143:0/299160624 >> v1:172.21.15.143:6789/0 conn(0x7f309015bc80 legacy=0x7f30900b6280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:51.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.248+0000 7f308effd700 1 -- 172.21.15.143:0/299160624 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f309006cc50 con 0x7f309006b0b0 2023-11-08T19:26:51.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.248+0000 7f308e7fc700 1 -- 172.21.15.143:0/299160624 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1500589404 0 0) 0x7f30900b2c50 con 0x7f309015bc80 2023-11-08T19:26:51.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.248+0000 7f308ffff700 1 --2- 172.21.15.143:0/299160624 >> v2:172.21.15.174:3300/0 conn(0x7f30900b22e0 0x7f30900b2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:26:51.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.248+0000 7f308effd700 1 --2- 172.21.15.143:0/299160624 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f309006b0b0 0x7f309006c710 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f30900720c0 tx=0x7f3084009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e7a069a00a779e6 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:51.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.248+0000 7f308e7fc700 1 -- 172.21.15.143:0/299160624 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3084011830 con 0x7f309006b0b0 2023-11-08T19:26:51.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.248+0000 7f308e7fc700 1 -- 172.21.15.143:0/299160624 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3084011990 con 0x7f309006b0b0 2023-11-08T19:26:51.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.248+0000 7f308e7fc700 1 -- 172.21.15.143:0/299160624 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3084011d20 con 0x7f309006b0b0 2023-11-08T19:26:51.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.249+0000 7f309606a700 1 -- 172.21.15.143:0/299160624 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f309006b0b0 msgr2=0x7f309006c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:51.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.249+0000 7f309606a700 1 --2- 172.21.15.143:0/299160624 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f309006b0b0 0x7f309006c710 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f30900720c0 tx=0x7f3084009b20 comp rx=0 tx=0).stop 2023-11-08T19:26:51.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.249+0000 7f309606a700 1 -- 172.21.15.143:0/299160624 shutdown_connections 2023-11-08T19:26:51.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.249+0000 7f309606a700 1 --2- 172.21.15.143:0/299160624 >> v2:172.21.15.174:3300/0 conn(0x7f30900b22e0 0x7f30900b2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:51.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.249+0000 7f309606a700 1 --2- 172.21.15.143:0/299160624 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f309006b0b0 0x7f309006c710 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:51.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.249+0000 7f309606a700 1 -- 172.21.15.143:0/299160624 >> 172.21.15.143:0/299160624 conn(0x7f30900cd2e0 msgr2=0x7f30900cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:51.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.249+0000 7f309606a700 1 -- 172.21.15.143:0/299160624 shutdown_connections 2023-11-08T19:26:51.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.249+0000 7f309606a700 1 -- 172.21.15.143:0/299160624 wait complete. 2023-11-08T19:26:51.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.250+0000 7f309606a700 1 Processor -- start 2023-11-08T19:26:51.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.250+0000 7f309606a700 1 -- start start 2023-11-08T19:26:51.261 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.251+0000 7f309606a700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f309006b0b0 0x7f30900b1130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:51.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.251+0000 7f309606a700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f30900b22e0 0x7f30900b1670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:51.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.251+0000 7f309606a700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f309015bc80 0x7f30900c6570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:51.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.251+0000 7f309606a700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f309015c240 con 0x7f309006b0b0 2023-11-08T19:26:51.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.251+0000 7f309606a700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f309015c4e0 con 0x7f30900b22e0 2023-11-08T19:26:51.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.251+0000 7f309606a700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f309015c7c0 con 0x7f309015bc80 2023-11-08T19:26:51.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.251+0000 7f308f7fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f309006b0b0 0x7f30900b1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:51.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.251+0000 7f308f7fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f309006b0b0 0x7f30900b1130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52326/0 (socket says 172.21.15.143:52326) 2023-11-08T19:26:51.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.251+0000 7f308f7fe700 1 -- 172.21.15.143:0/4096455152 learned_addr learned my addr 172.21.15.143:0/4096455152 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:51.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.251+0000 7f308ffff700 1 --2- 172.21.15.143:0/4096455152 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f309015bc80 0x7f30900c6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:51.265 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.251+0000 7f308effd700 1 --2- 172.21.15.143:0/4096455152 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f30900b22e0 0x7f30900b1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:51.265 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.251+0000 7f308f7fe700 1 -- 172.21.15.143:0/4096455152 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f309015bc80 msgr2=0x7f30900c6570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:51.265 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.252+0000 7f308f7fe700 1 --2- 172.21.15.143:0/4096455152 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f309015bc80 0x7f30900c6570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:51.266 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.252+0000 7f308f7fe700 1 -- 172.21.15.143:0/4096455152 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f30900b22e0 msgr2=0x7f30900b1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:51.266 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.252+0000 7f308f7fe700 1 --2- 172.21.15.143:0/4096455152 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f30900b22e0 0x7f30900b1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:51.266 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.252+0000 7f308f7fe700 1 -- 172.21.15.143:0/4096455152 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30840094a0 con 0x7f309006b0b0 2023-11-08T19:26:51.266 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.252+0000 7f308effd700 1 --2- 172.21.15.143:0/4096455152 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f30900b22e0 0x7f30900b1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:26:51.266 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.252+0000 7f308f7fe700 1 --2- 172.21.15.143:0/4096455152 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f309006b0b0 0x7f30900b1130 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f308000c550 tx=0x7f308000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:51.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.252+0000 7f3077fff700 1 -- 172.21.15.143:0/4096455152 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3080004c80 con 0x7f309006b0b0 2023-11-08T19:26:51.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.252+0000 7f3077fff700 1 -- 172.21.15.143:0/4096455152 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3080004e00 con 0x7f309006b0b0 2023-11-08T19:26:51.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.252+0000 7f309606a700 1 -- 172.21.15.143:0/4096455152 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f309015caa0 con 0x7f309006b0b0 2023-11-08T19:26:51.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.252+0000 7f3077fff700 1 -- 172.21.15.143:0/4096455152 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3080007310 con 0x7f309006b0b0 2023-11-08T19:26:51.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.252+0000 7f309606a700 1 -- 172.21.15.143:0/4096455152 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f309015d2a0 con 0x7f309006b0b0 2023-11-08T19:26:51.268 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.253+0000 7f309606a700 1 -- 172.21.15.143:0/4096455152 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f309015d4f0 con 0x7f309006b0b0 2023-11-08T19:26:51.268 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.253+0000 7f3077fff700 1 -- 172.21.15.143:0/4096455152 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3080007470 con 0x7f309006b0b0 2023-11-08T19:26:51.268 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.254+0000 7f3077fff700 1 --2- 172.21.15.143:0/4096455152 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f307c06af00 0x7f307c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:51.268 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.254+0000 7f3077fff700 1 -- 172.21.15.143:0/4096455152 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f3080089410 con 0x7f309006b0b0 2023-11-08T19:26:51.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.254+0000 7f308effd700 1 --2- 172.21.15.143:0/4096455152 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f307c06af00 0x7f307c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:51.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.255+0000 7f308effd700 1 --2- 172.21.15.143:0/4096455152 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f307c06af00 0x7f307c06d3c0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f30840090b0 tx=0x7f3084008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:51.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.257+0000 7f3077fff700 1 -- 172.21.15.143:0/4096455152 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3080054660 con 0x7f309006b0b0 2023-11-08T19:26:51.447 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.446+0000 7f309606a700 1 -- 172.21.15.143:0/4096455152 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f309015d4f0 con 0x7f309006b0b0 2023-11-08T19:26:51.450 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.449+0000 7f3077fff700 1 -- 172.21.15.143:0/4096455152 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v15)=0 v15) v1 ==== 122+0+0 (secure 0 0 0) 0x7f309015d4f0 con 0x7f309006b0b0 2023-11-08T19:26:51.454 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.451+0000 7f309606a700 1 -- 172.21.15.143:0/4096455152 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f307c06af00 msgr2=0x7f307c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:51.454 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.451+0000 7f309606a700 1 --2- 172.21.15.143:0/4096455152 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f307c06af00 0x7f307c06d3c0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f30840090b0 tx=0x7f3084008030 comp rx=0 tx=0).stop 2023-11-08T19:26:51.455 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.451+0000 7f309606a700 1 -- 172.21.15.143:0/4096455152 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f309006b0b0 msgr2=0x7f30900b1130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:51.455 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.451+0000 7f309606a700 1 --2- 172.21.15.143:0/4096455152 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f309006b0b0 0x7f30900b1130 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f308000c550 tx=0x7f308000c910 comp rx=0 tx=0).stop 2023-11-08T19:26:51.455 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.452+0000 7f309606a700 1 -- 172.21.15.143:0/4096455152 shutdown_connections 2023-11-08T19:26:51.455 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.452+0000 7f309606a700 1 --2- 172.21.15.143:0/4096455152 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f307c06af00 0x7f307c06d3c0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:51.455 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.452+0000 7f309606a700 1 --2- 172.21.15.143:0/4096455152 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f309015bc80 0x7f30900c6570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:51.455 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.452+0000 7f309606a700 1 --2- 172.21.15.143:0/4096455152 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f309006b0b0 0x7f30900b1130 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:51.456 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.452+0000 7f309606a700 1 --2- 172.21.15.143:0/4096455152 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f30900b22e0 0x7f30900b1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:51.456 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.452+0000 7f309606a700 1 -- 172.21.15.143:0/4096455152 >> 172.21.15.143:0/4096455152 conn(0x7f30900cd2e0 msgr2=0x7f30900b6010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:51.456 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.452+0000 7f309606a700 1 -- 172.21.15.143:0/4096455152 shutdown_connections 2023-11-08T19:26:51.456 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.452+0000 7f309606a700 1 -- 172.21.15.143:0/4096455152 wait complete. 2023-11-08T19:26:51.465 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-08T19:26:51.466 DEBUG:teuthology.orchestra.run.smithi143:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-08T19:26:51.505 DEBUG:teuthology.orchestra.run.smithi174:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-08T19:26:51.554 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-08T19:26:51.555 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-08T19:26:51.555 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:26:51.573 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284264k,nr_inodes=4071066,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=44,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20643 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-08T19:26:51.574 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2023-11-08T19:26:51.574 DEBUG:teuthology.orchestra.run.smithi174:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:26:51.594 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284224k,nr_inodes=4071056,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=31,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19549 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260544k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-08T19:26:51.595 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-08T19:26:51.595 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:26:51.631 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284264k,nr_inodes=4071066,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=44,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20643 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-08T19:26:51.631 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2023-11-08T19:26:51.632 DEBUG:teuthology.orchestra.run.smithi174:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:26:51.653 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284224k,nr_inodes=4071056,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=31,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19549 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260544k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-08T19:26:51.653 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-08T19:26:51.654 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:26:51.791 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.790+0000 7ffa4146b700 1 Processor -- start 2023-11-08T19:26:51.792 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.790+0000 7ffa4146b700 1 -- start start 2023-11-08T19:26:51.794 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.791+0000 7ffa4146b700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ffa3c0cf090 0x7ffa3c0d1f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:51.794 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.791+0000 7ffa4146b700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7ffa3c0d3690 0x7ffa3c0b9690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:51.795 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.791+0000 7ffa4146b700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa3c0d3200 con 0x7ffa3c15dc80 2023-11-08T19:26:51.795 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.791+0000 7ffa4146b700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa3c0b9bd0 con 0x7ffa3c0d3690 2023-11-08T19:26:51.795 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.791+0000 7ffa4146b700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa3c0b7cf0 con 0x7ffa3c0cf090 2023-11-08T19:26:51.795 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.791+0000 7ffa3affd700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7ffa3c15dc80 0x7ffa3c0ce980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:48488/0 (socket says 172.21.15.143:48488) 2023-11-08T19:26:51.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.791+0000 7ffa3affd700 1 -- 172.21.15.143:0/4196236543 learned_addr learned my addr 172.21.15.143:0/4196236543 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:26:51.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.791+0000 7ffa3b7fe700 1 --2- 172.21.15.143:0/4196236543 >> v2:172.21.15.174:3300/0 conn(0x7ffa3c0d3690 0x7ffa3c0b9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:51.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.791+0000 7ffa3a7fc700 1 --2- 172.21.15.143:0/4196236543 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ffa3c0cf090 0x7ffa3c0d1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:51.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.791+0000 7ffa39ffb700 1 -- 172.21.15.143:0/4196236543 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4271436913 0 0) 0x7ffa3c0d3200 con 0x7ffa3c15dc80 2023-11-08T19:26:51.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.791+0000 7ffa39ffb700 1 -- 172.21.15.143:0/4196236543 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa240035b0 con 0x7ffa3c15dc80 2023-11-08T19:26:51.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.791+0000 7ffa39ffb700 1 -- 172.21.15.143:0/4196236543 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2712889268 0 0) 0x7ffa240035b0 con 0x7ffa3c15dc80 2023-11-08T19:26:51.797 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.791+0000 7ffa3a7fc700 1 -- 172.21.15.143:0/4196236543 >> v2:172.21.15.174:3300/0 conn(0x7ffa3c0d3690 msgr2=0x7ffa3c0b9690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:51.797 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.791+0000 7ffa3a7fc700 1 --2- 172.21.15.143:0/4196236543 >> v2:172.21.15.174:3300/0 conn(0x7ffa3c0d3690 0x7ffa3c0b9690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:51.797 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.791+0000 7ffa3a7fc700 1 -- 172.21.15.143:0/4196236543 >> v1:172.21.15.143:6789/0 conn(0x7ffa3c15dc80 legacy=0x7ffa3c0ce980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:51.797 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.791+0000 7ffa3a7fc700 1 -- 172.21.15.143:0/4196236543 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa3c15e060 con 0x7ffa3c0cf090 2023-11-08T19:26:51.798 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.791+0000 7ffa3b7fe700 1 --2- 172.21.15.143:0/4196236543 >> v2:172.21.15.174:3300/0 conn(0x7ffa3c0d3690 0x7ffa3c0b9690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T19:26:51.798 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.792+0000 7ffa3a7fc700 1 --2- 172.21.15.143:0/4196236543 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ffa3c0cf090 0x7ffa3c0d1f80 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c009840 tx=0x7ffa2c009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81d2cac2e4b2746e server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:51.798 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.792+0000 7ffa39ffb700 1 -- 172.21.15.143:0/4196236543 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa30001670 con 0x7ffa3c0cf090 2023-11-08T19:26:51.798 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.792+0000 7ffa39ffb700 1 -- 172.21.15.143:0/4196236543 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa2c004030 con 0x7ffa3c0cf090 2023-11-08T19:26:51.798 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.792+0000 7ffa39ffb700 1 -- 172.21.15.143:0/4196236543 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa2c0043a0 con 0x7ffa3c0cf090 2023-11-08T19:26:51.799 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.792+0000 7ffa4146b700 1 -- 172.21.15.143:0/4196236543 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ffa3c0cf090 msgr2=0x7ffa3c0d1f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:51.799 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.792+0000 7ffa4146b700 1 --2- 172.21.15.143:0/4196236543 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ffa3c0cf090 0x7ffa3c0d1f80 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c009840 tx=0x7ffa2c009b50 comp rx=0 tx=0).stop 2023-11-08T19:26:51.799 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.792+0000 7ffa4146b700 1 -- 172.21.15.143:0/4196236543 shutdown_connections 2023-11-08T19:26:51.799 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.792+0000 7ffa4146b700 1 --2- 172.21.15.143:0/4196236543 >> v2:172.21.15.174:3300/0 conn(0x7ffa3c0d3690 0x7ffa3c0b9690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:51.799 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.792+0000 7ffa4146b700 1 --2- 172.21.15.143:0/4196236543 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ffa3c0cf090 0x7ffa3c0d1f80 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:51.799 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.792+0000 7ffa4146b700 1 -- 172.21.15.143:0/4196236543 >> 172.21.15.143:0/4196236543 conn(0x7ffa3c0668e0 msgr2=0x7ffa3c066cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:51.800 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.792+0000 7ffa4146b700 1 -- 172.21.15.143:0/4196236543 shutdown_connections 2023-11-08T19:26:51.800 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.792+0000 7ffa4146b700 1 -- 172.21.15.143:0/4196236543 wait complete. 2023-11-08T19:26:51.802 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.794+0000 7ffa4146b700 1 Processor -- start 2023-11-08T19:26:51.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.794+0000 7ffa4146b700 1 -- start start 2023-11-08T19:26:51.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.794+0000 7ffa4146b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffa3c0cf090 0x7ffa3c1504d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:51.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.794+0000 7ffa4146b700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ffa3c0d3690 0x7ffa3c151350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:51.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.794+0000 7ffa4146b700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffa3c15dc80 0x7ffa3c1548d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:51.804 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.794+0000 7ffa4146b700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa3c151c80 con 0x7ffa3c15dc80 2023-11-08T19:26:51.804 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.794+0000 7ffa4146b700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa3c151f60 con 0x7ffa3c0cf090 2023-11-08T19:26:51.804 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.794+0000 7ffa4146b700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa3c152240 con 0x7ffa3c0d3690 2023-11-08T19:26:51.804 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.794+0000 7ffa3b7fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffa3c15dc80 0x7ffa3c1548d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:51.804 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.794+0000 7ffa3b7fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffa3c15dc80 0x7ffa3c1548d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52344/0 (socket says 172.21.15.143:52344) 2023-11-08T19:26:51.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.794+0000 7ffa3b7fe700 1 -- 172.21.15.143:0/75687696 learned_addr learned my addr 172.21.15.143:0/75687696 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:51.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.795+0000 7ffa3a7fc700 1 --2- 172.21.15.143:0/75687696 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ffa3c0d3690 0x7ffa3c151350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:51.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.795+0000 7ffa3affd700 1 --2- 172.21.15.143:0/75687696 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffa3c0cf090 0x7ffa3c1504d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:51.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.795+0000 7ffa3b7fe700 1 -- 172.21.15.143:0/75687696 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ffa3c0d3690 msgr2=0x7ffa3c151350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:51.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.795+0000 7ffa3b7fe700 1 --2- 172.21.15.143:0/75687696 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ffa3c0d3690 0x7ffa3c151350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:51.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.795+0000 7ffa3b7fe700 1 -- 172.21.15.143:0/75687696 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffa3c0cf090 msgr2=0x7ffa3c1504d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:51.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.795+0000 7ffa3b7fe700 1 --2- 172.21.15.143:0/75687696 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffa3c0cf090 0x7ffa3c1504d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:51.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.795+0000 7ffa3b7fe700 1 -- 172.21.15.143:0/75687696 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa34009a70 con 0x7ffa3c15dc80 2023-11-08T19:26:51.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.795+0000 7ffa3b7fe700 1 --2- 172.21.15.143:0/75687696 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffa3c15dc80 0x7ffa3c1548d0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7ffa3400efc0 tx=0x7ffa3400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:51.807 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.795+0000 7ffa237fe700 1 -- 172.21.15.143:0/75687696 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa3400cb40 con 0x7ffa3c15dc80 2023-11-08T19:26:51.807 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.795+0000 7ffa237fe700 1 -- 172.21.15.143:0/75687696 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa3400f040 con 0x7ffa3c15dc80 2023-11-08T19:26:51.807 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.795+0000 7ffa4146b700 1 -- 172.21.15.143:0/75687696 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa2c0094a0 con 0x7ffa3c15dc80 2023-11-08T19:26:51.807 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.795+0000 7ffa237fe700 1 -- 172.21.15.143:0/75687696 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa34003a20 con 0x7ffa3c15dc80 2023-11-08T19:26:51.807 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.795+0000 7ffa4146b700 1 -- 172.21.15.143:0/75687696 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa3c152800 con 0x7ffa3c15dc80 2023-11-08T19:26:51.808 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.796+0000 7ffa4146b700 1 -- 172.21.15.143:0/75687696 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa3c1529f0 con 0x7ffa3c15dc80 2023-11-08T19:26:51.808 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.796+0000 7ffa237fe700 1 -- 172.21.15.143:0/75687696 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ffa34014480 con 0x7ffa3c15dc80 2023-11-08T19:26:51.808 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.797+0000 7ffa237fe700 1 --2- 172.21.15.143:0/75687696 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7ffa2806af00 0x7ffa2806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:51.808 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.797+0000 7ffa237fe700 1 -- 172.21.15.143:0/75687696 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5296+0+0 (secure 0 0 0) 0x7ffa34089e30 con 0x7ffa3c15dc80 2023-11-08T19:26:51.809 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.797+0000 7ffa3affd700 1 --2- 172.21.15.143:0/75687696 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7ffa2806af00 0x7ffa2806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:51.809 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.797+0000 7ffa3affd700 1 --2- 172.21.15.143:0/75687696 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7ffa2806af00 0x7ffa2806d3c0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7ffa30001eb0 tx=0x7ffa30001c60 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:51.809 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:51.801+0000 7ffa237fe700 1 -- 172.21.15.143:0/75687696 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ffa34055000 con 0x7ffa3c15dc80 2023-11-08T19:26:52.032 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:52.031+0000 7ffa4146b700 1 -- 172.21.15.143:0/75687696 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffa3c04eee0 con 0x7ffa3c15dc80 2023-11-08T19:26:52.033 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:52.031+0000 7ffa237fe700 1 -- 172.21.15.143:0/75687696 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+3711 (secure 0 0 0) 0x7ffa34018e90 con 0x7ffa3c15dc80 2023-11-08T19:26:52.033 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:26:52.034 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":27,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4988,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3140991763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":3140991763},{"type":"v1","addr":"172.21.15.143:6835","nonce":3140991763}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":4997,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6836/2388011942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2388011942},{"type":"v1","addr":"172.21.15.174:6836","nonce":2388011942}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5000,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/1339111936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6835","nonce":1339111936},{"type":"v1","addr":"172.21.15.174:6837","nonce":1339111936}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5925,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6834/1634451108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":1634451108},{"type":"v1","addr":"172.21.15.143:6834","nonce":1634451108}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[]} 2023-11-08T19:26:52.035 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:52.033+0000 7ffa4146b700 1 -- 172.21.15.143:0/75687696 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7ffa2806af00 msgr2=0x7ffa2806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:52.035 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:52.033+0000 7ffa4146b700 1 --2- 172.21.15.143:0/75687696 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7ffa2806af00 0x7ffa2806d3c0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7ffa30001eb0 tx=0x7ffa30001c60 comp rx=0 tx=0).stop 2023-11-08T19:26:52.036 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:52.033+0000 7ffa4146b700 1 -- 172.21.15.143:0/75687696 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffa3c15dc80 msgr2=0x7ffa3c1548d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:52.036 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:52.033+0000 7ffa4146b700 1 --2- 172.21.15.143:0/75687696 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffa3c15dc80 0x7ffa3c1548d0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7ffa3400efc0 tx=0x7ffa3400c3a0 comp rx=0 tx=0).stop 2023-11-08T19:26:52.036 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:52.033+0000 7ffa4146b700 1 -- 172.21.15.143:0/75687696 shutdown_connections 2023-11-08T19:26:52.036 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:52.033+0000 7ffa4146b700 1 --2- 172.21.15.143:0/75687696 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7ffa2806af00 0x7ffa2806d3c0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:52.037 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:52.033+0000 7ffa4146b700 1 --2- 172.21.15.143:0/75687696 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffa3c15dc80 0x7ffa3c1548d0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:52.037 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:52.033+0000 7ffa4146b700 1 --2- 172.21.15.143:0/75687696 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffa3c0cf090 0x7ffa3c1504d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:52.037 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:52.033+0000 7ffa4146b700 1 --2- 172.21.15.143:0/75687696 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ffa3c0d3690 0x7ffa3c151350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:52.037 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:52.033+0000 7ffa4146b700 1 -- 172.21.15.143:0/75687696 >> 172.21.15.143:0/75687696 conn(0x7ffa3c0668e0 msgr2=0x7ffa3c0d3b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:52.038 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:52.033+0000 7ffa4146b700 1 -- 172.21.15.143:0/75687696 shutdown_connections 2023-11-08T19:26:52.038 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:52.033+0000 7ffa4146b700 1 -- 172.21.15.143:0/75687696 wait complete. 2023-11-08T19:26:52.038 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 27 2023-11-08T19:26:52.047 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-08T19:26:52.047 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-08T19:26:52.048 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-08T19:26:52.048 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T19:26:52.048 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-08T19:26:52.049 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-08T19:26:52.049 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-08T19:26:52.049 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T19:26:52.049 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-08T19:26:52.050 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-08T19:26:52.050 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-08T19:26:52.050 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T19:26:52.051 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-08T19:26:52.051 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-08T19:26:52.051 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-08T19:26:52.051 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T19:26:52.057 INFO:tasks.ceph.mds.a:Stopped 2023-11-08T19:26:52.057 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-08T19:26:52.059 INFO:tasks.ceph.mds.c:Stopped 2023-11-08T19:26:52.059 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-08T19:26:52.062 INFO:tasks.ceph.mds.b:Stopped 2023-11-08T19:26:52.062 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-08T19:26:52.063 INFO:tasks.ceph.mds.d:Stopped 2023-11-08T19:26:52.063 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-08T19:26:52.101 INFO:tasks.ceph.mds.a:Started 2023-11-08T19:26:52.102 DEBUG:teuthology.parallel:result is None 2023-11-08T19:26:52.103 INFO:tasks.ceph.mds.c:Started 2023-11-08T19:26:52.103 DEBUG:teuthology.parallel:result is None 2023-11-08T19:26:52.110 INFO:tasks.ceph.mds.b:Started 2023-11-08T19:26:52.111 INFO:tasks.ceph.mds.d:Started 2023-11-08T19:26:52.111 DEBUG:teuthology.parallel:result is None 2023-11-08T19:26:52.112 DEBUG:teuthology.parallel:result is None 2023-11-08T19:26:52.112 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-08T19:26:52.181 INFO:tasks.ceph.mds.a.smithi143.stdout:starting mds.a at 2023-11-08T19:26:52.181 INFO:tasks.ceph.mds.c.smithi143.stdout:starting mds.c at 2023-11-08T19:26:52.194 INFO:tasks.ceph.mds.b.smithi174.stdout:starting mds.b at 2023-11-08T19:26:52.195 INFO:tasks.ceph.mds.d.smithi174.stdout:starting mds.d at 2023-11-08T19:26:52.240 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:52.239+0000 7fce5e79b700 1 Processor -- start 2023-11-08T19:26:52.241 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:52.239+0000 7fce5e79b700 1 -- start start 2023-11-08T19:26:52.243 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:52.240+0000 7fce5e79b700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fce5806c730 0x7fce580cd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:52.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:52.240+0000 7fce5e79b700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fce5815de80 0x7fce580cbf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:52.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:52.240+0000 7fce5e79b700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce580cd480 con 0x7fce5806c030 2023-11-08T19:26:52.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:52.240+0000 7fce5e79b700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce580cc870 con 0x7fce5806c730 2023-11-08T19:26:52.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:52.240+0000 7fce5e79b700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce580ccb50 con 0x7fce5815de80 2023-11-08T19:26:52.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:52.240+0000 7fce577fe700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fce5806c030 0x7fce5806cb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:48526/0 (socket says 172.21.15.143:48526) 2023-11-08T19:26:52.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:52.240+0000 7fce577fe700 1 -- 172.21.15.143:0/3756542030 learned_addr learned my addr 172.21.15.143:0/3756542030 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:26:52.246 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:52.240+0000 7fce5cd38700 1 --2- 172.21.15.143:0/3756542030 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fce5815de80 0x7fce580cbf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:52.246 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:52.240+0000 7fce57fff700 1 --2- 172.21.15.143:0/3756542030 >> v2:172.21.15.174:3300/0 conn(0x7fce5806c730 0x7fce580cd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:52.246 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:52.240+0000 7fce56ffd700 1 -- 172.21.15.143:0/3756542030 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1155985153 0 0) 0x7fce580cd480 con 0x7fce5806c030 2023-11-08T19:26:52.246 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:52.240+0000 7fce56ffd700 1 -- 172.21.15.143:0/3756542030 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce440035b0 con 0x7fce5806c030 2023-11-08T19:26:52.246 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:52.240+0000 7fce5cd38700 1 -- 172.21.15.143:0/3756542030 >> v2:172.21.15.174:3300/0 conn(0x7fce5806c730 msgr2=0x7fce580cd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:52.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:52.240+0000 7fce5cd38700 1 --2- 172.21.15.143:0/3756542030 >> v2:172.21.15.174:3300/0 conn(0x7fce5806c730 0x7fce580cd910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:52.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:52.240+0000 7fce5cd38700 1 -- 172.21.15.143:0/3756542030 >> v1:172.21.15.143:6789/0 conn(0x7fce5806c030 legacy=0x7fce5806cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:52.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:52.240+0000 7fce5cd38700 1 -- 172.21.15.143:0/3756542030 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce580cce30 con 0x7fce5815de80 2023-11-08T19:26:52.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:52.240+0000 7fce5cd38700 1 --2- 172.21.15.143:0/3756542030 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fce5815de80 0x7fce580cbf00 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fce50009d90 tx=0x7fce5000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a441273ad207731 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:52.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:52.241+0000 7fce56ffd700 1 -- 172.21.15.143:0/3756542030 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce5000e070 con 0x7fce5815de80 2023-11-08T19:26:52.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:52.241+0000 7fce56ffd700 1 -- 172.21.15.143:0/3756542030 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce5000bb20 con 0x7fce5815de80 2023-11-08T19:26:52.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:52.241+0000 7fce56ffd700 1 -- 172.21.15.143:0/3756542030 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce50012430 con 0x7fce5815de80 2023-11-08T19:26:52.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:52.241+0000 7fce5e79b700 1 -- 172.21.15.143:0/3756542030 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fce5815de80 msgr2=0x7fce580cbf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:52.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:52.241+0000 7fce5e79b700 1 --2- 172.21.15.143:0/3756542030 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fce5815de80 0x7fce580cbf00 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fce50009d90 tx=0x7fce5000b330 comp rx=0 tx=0).stop 2023-11-08T19:26:52.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:52.241+0000 7fce5e79b700 1 -- 172.21.15.143:0/3756542030 shutdown_connections 2023-11-08T19:26:52.249 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:52.241+0000 7fce5e79b700 1 --2- 172.21.15.143:0/3756542030 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fce5815de80 0x7fce580cbf00 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:52.249 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:52.241+0000 7fce5e79b700 1 --2- 172.21.15.143:0/3756542030 >> v2:172.21.15.174:3300/0 conn(0x7fce5806c730 0x7fce580cd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:52.249 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:52.241+0000 7fce5e79b700 1 -- 172.21.15.143:0/3756542030 >> 172.21.15.143:0/3756542030 conn(0x7fce5808b000 msgr2=0x7fce5809ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:52.249 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:52.241+0000 7fce5e79b700 1 -- 172.21.15.143:0/3756542030 shutdown_connections 2023-11-08T19:26:52.249 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:52.241+0000 7fce5e79b700 1 -- 172.21.15.143:0/3756542030 wait complete. 2023-11-08T19:26:52.250 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:52.242+0000 7fce5e79b700 1 Processor -- start 2023-11-08T19:26:52.250 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:52.243+0000 7fce5e79b700 1 -- start start 2023-11-08T19:26:52.252 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:52.243+0000 7fce5e79b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fce5806c030 0x7fce5814fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:52.253 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:52.243+0000 7fce5e79b700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fce5806c730 0x7fce581f3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:52.253 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:52.243+0000 7fce5e79b700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fce5815de80 0x7fce580c9f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:52.253 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:52.243+0000 7fce5e79b700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce58150370 con 0x7fce5815de80 2023-11-08T19:26:52.253 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:52.243+0000 7fce5e79b700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce5815e2b0 con 0x7fce5806c030 2023-11-08T19:26:52.254 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:52.243+0000 7fce5e79b700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce5815f110 con 0x7fce5806c730 2023-11-08T19:26:52.254 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:52.243+0000 7fce5cd38700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fce5815de80 0x7fce580c9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:52.254 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:52.243+0000 7fce5cd38700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fce5815de80 0x7fce580c9f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52382/0 (socket says 172.21.15.143:52382) 2023-11-08T19:26:52.254 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:52.243+0000 7fce5cd38700 1 -- 172.21.15.143:0/3544460870 learned_addr learned my addr 172.21.15.143:0/3544460870 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:52.254 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:52.244+0000 7fce577fe700 1 --2- 172.21.15.143:0/3544460870 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fce5806c730 0x7fce581f3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:52.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:52.244+0000 7fce57fff700 1 --2- 172.21.15.143:0/3544460870 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fce5806c030 0x7fce5814fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:52.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:52.244+0000 7fce5cd38700 1 -- 172.21.15.143:0/3544460870 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fce5806c730 msgr2=0x7fce581f3eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:52.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:52.244+0000 7fce5cd38700 1 --2- 172.21.15.143:0/3544460870 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fce5806c730 0x7fce581f3eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:52.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:52.244+0000 7fce5cd38700 1 -- 172.21.15.143:0/3544460870 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fce5806c030 msgr2=0x7fce5814fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:52.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:52.244+0000 7fce5cd38700 1 --2- 172.21.15.143:0/3544460870 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fce5806c030 0x7fce5814fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:52.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:52.244+0000 7fce5cd38700 1 -- 172.21.15.143:0/3544460870 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce50009a70 con 0x7fce5815de80 2023-11-08T19:26:52.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:52.244+0000 7fce577fe700 1 --2- 172.21.15.143:0/3544460870 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fce5806c730 0x7fce581f3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:26:52.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:52.244+0000 7fce57fff700 1 --2- 172.21.15.143:0/3544460870 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fce5806c030 0x7fce5814fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:26:52.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:52.244+0000 7fce5cd38700 1 --2- 172.21.15.143:0/3544460870 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fce5815de80 0x7fce580c9f70 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fce50014040 tx=0x7fce50004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:52.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:52.244+0000 7fce54ff9700 1 -- 172.21.15.143:0/3544460870 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce500129f0 con 0x7fce5815de80 2023-11-08T19:26:52.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:52.244+0000 7fce54ff9700 1 -- 172.21.15.143:0/3544460870 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce50020070 con 0x7fce5815de80 2023-11-08T19:26:52.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:52.244+0000 7fce5e79b700 1 -- 172.21.15.143:0/3544460870 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce581f43f0 con 0x7fce5815de80 2023-11-08T19:26:52.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:52.244+0000 7fce54ff9700 1 -- 172.21.15.143:0/3544460870 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce50012db0 con 0x7fce5815de80 2023-11-08T19:26:52.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:52.244+0000 7fce5e79b700 1 -- 172.21.15.143:0/3544460870 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce5815e7e0 con 0x7fce5815de80 2023-11-08T19:26:52.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:52.245+0000 7fce5e79b700 1 -- 172.21.15.143:0/3544460870 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce5815f2c0 con 0x7fce5815de80 2023-11-08T19:26:52.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:52.245+0000 7fce54ff9700 1 -- 172.21.15.143:0/3544460870 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fce50004040 con 0x7fce5815de80 2023-11-08T19:26:52.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:52.246+0000 7fce54ff9700 1 --2- 172.21.15.143:0/3544460870 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fce3806af00 0x7fce3806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:52.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:52.246+0000 7fce54ff9700 1 -- 172.21.15.143:0/3544460870 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fce50022070 con 0x7fce5815de80 2023-11-08T19:26:52.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:52.246+0000 7fce57fff700 1 --2- 172.21.15.143:0/3544460870 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fce3806af00 0x7fce3806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:52.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:52.246+0000 7fce57fff700 1 --2- 172.21.15.143:0/3544460870 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fce3806af00 0x7fce3806d3c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fce40009b60 tx=0x7fce40008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:52.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:52.250+0000 7fce54ff9700 1 -- 172.21.15.143:0/3544460870 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fce50059fd0 con 0x7fce5815de80 2023-11-08T19:26:52.457 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:52.455+0000 7fce5e79b700 1 -- 172.21.15.143:0/3544460870 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fce5804eee0 con 0x7fce5815de80 2023-11-08T19:26:53.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:53.595+0000 7fce54ff9700 1 -- 172.21.15.143:0/3544460870 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v58) v1 ==== 97+0+0 (secure 0 0 0) 0x7fce50059b60 con 0x7fce5815de80 2023-11-08T19:26:53.600 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:53.599+0000 7fce5e79b700 1 -- 172.21.15.143:0/3544460870 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fce3806af00 msgr2=0x7fce3806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:53.600 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:53.599+0000 7fce5e79b700 1 --2- 172.21.15.143:0/3544460870 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fce3806af00 0x7fce3806d3c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fce40009b60 tx=0x7fce40008040 comp rx=0 tx=0).stop 2023-11-08T19:26:53.601 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:53.599+0000 7fce5e79b700 1 -- 172.21.15.143:0/3544460870 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fce5815de80 msgr2=0x7fce580c9f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:53.601 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:53.599+0000 7fce5e79b700 1 --2- 172.21.15.143:0/3544460870 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fce5815de80 0x7fce580c9f70 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fce50014040 tx=0x7fce50004ad0 comp rx=0 tx=0).stop 2023-11-08T19:26:53.601 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:53.599+0000 7fce5e79b700 1 -- 172.21.15.143:0/3544460870 shutdown_connections 2023-11-08T19:26:53.601 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:53.599+0000 7fce5e79b700 1 --2- 172.21.15.143:0/3544460870 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fce3806af00 0x7fce3806d3c0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:53.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:53.599+0000 7fce5e79b700 1 --2- 172.21.15.143:0/3544460870 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fce5815de80 0x7fce580c9f70 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:53.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:53.599+0000 7fce5e79b700 1 --2- 172.21.15.143:0/3544460870 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fce5806c030 0x7fce5814fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:53.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:53.599+0000 7fce5e79b700 1 --2- 172.21.15.143:0/3544460870 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fce5806c730 0x7fce581f3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:53.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:53.599+0000 7fce5e79b700 1 -- 172.21.15.143:0/3544460870 >> 172.21.15.143:0/3544460870 conn(0x7fce5808b000 msgr2=0x7fce580cdee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:53.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:53.599+0000 7fce5e79b700 1 -- 172.21.15.143:0/3544460870 shutdown_connections 2023-11-08T19:26:53.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:53.599+0000 7fce5e79b700 1 -- 172.21.15.143:0/3544460870 wait complete. 2023-11-08T19:26:53.603 INFO:teuthology.orchestra.run.smithi143.stderr: removed all blocklist entries 2023-11-08T19:26:53.615 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-08T19:26:53.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:53.739+0000 7f90c2e24700 1 Processor -- start 2023-11-08T19:26:53.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:53.739+0000 7f90c2e24700 1 -- start start 2023-11-08T19:26:53.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:53.740+0000 7f90c2e24700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f90bc15ba30 0x7f90bc0b14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:53.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:53.740+0000 7f90c2e24700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f90bc0d1c70 0x7f90bc0cc5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:53.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:53.740+0000 7f90c2e24700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90bc0b1bc0 con 0x7f90bc0d03a0 2023-11-08T19:26:53.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:53.740+0000 7f90c2e24700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90bc0b1ea0 con 0x7f90bc0d1c70 2023-11-08T19:26:53.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:53.740+0000 7f90c2e24700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90bc0b2180 con 0x7f90bc15ba30 2023-11-08T19:26:53.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:53.741+0000 7f90c0bc0700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f90bc15ba30 0x7f90bc0b14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:53.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:53.741+0000 7f90c0bc0700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f90bc15ba30 0x7f90bc0b14c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.143:40028/0 (socket says 172.21.15.143:40028) 2023-11-08T19:26:53.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:53.741+0000 7f90c0bc0700 1 -- 172.21.15.143:0/1075007332 learned_addr learned my addr 172.21.15.143:0/1075007332 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:53.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:53.741+0000 7f90c13c1700 1 --2- 172.21.15.143:0/1075007332 >> v2:172.21.15.174:3300/0 conn(0x7f90bc0d1c70 0x7f90bc0cc5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:53.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:53.741+0000 7f90c0bc0700 1 -- 172.21.15.143:0/1075007332 >> v2:172.21.15.174:3300/0 conn(0x7f90bc0d1c70 msgr2=0x7f90bc0cc5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:53.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:53.741+0000 7f90c0bc0700 1 --2- 172.21.15.143:0/1075007332 >> v2:172.21.15.174:3300/0 conn(0x7f90bc0d1c70 0x7f90bc0cc5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:53.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:53.741+0000 7f90c0bc0700 1 -- 172.21.15.143:0/1075007332 >> v1:172.21.15.174:6789/0 conn(0x7f90bc0d03a0 legacy=0x7f90bc0d1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:53.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:53.741+0000 7f90c0bc0700 1 -- 172.21.15.143:0/1075007332 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90bc0b2460 con 0x7f90bc15ba30 2023-11-08T19:26:53.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:53.741+0000 7f90c0bc0700 1 --2- 172.21.15.143:0/1075007332 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f90bc15ba30 0x7f90bc0b14c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f90b00098f0 tx=0x7f90b0009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=870f46bbdb9f3e2 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:53.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:53.741+0000 7f90bb7fe700 1 -- 172.21.15.143:0/1075007332 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b0011830 con 0x7f90bc15ba30 2023-11-08T19:26:53.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:53.741+0000 7f90bb7fe700 1 -- 172.21.15.143:0/1075007332 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90b0011990 con 0x7f90bc15ba30 2023-11-08T19:26:53.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:53.741+0000 7f90bb7fe700 1 -- 172.21.15.143:0/1075007332 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b0011d20 con 0x7f90bc15ba30 2023-11-08T19:26:53.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:53.742+0000 7f90c2e24700 1 -- 172.21.15.143:0/1075007332 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f90bc15ba30 msgr2=0x7f90bc0b14c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:53.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:53.742+0000 7f90c2e24700 1 --2- 172.21.15.143:0/1075007332 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f90bc15ba30 0x7f90bc0b14c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f90b00098f0 tx=0x7f90b0009c00 comp rx=0 tx=0).stop 2023-11-08T19:26:53.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:53.742+0000 7f90c2e24700 1 -- 172.21.15.143:0/1075007332 shutdown_connections 2023-11-08T19:26:53.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:53.742+0000 7f90c2e24700 1 --2- 172.21.15.143:0/1075007332 >> v2:172.21.15.174:3300/0 conn(0x7f90bc0d1c70 0x7f90bc0cc5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:53.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:53.742+0000 7f90c2e24700 1 --2- 172.21.15.143:0/1075007332 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f90bc15ba30 0x7f90bc0b14c0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:53.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:53.742+0000 7f90c2e24700 1 -- 172.21.15.143:0/1075007332 >> 172.21.15.143:0/1075007332 conn(0x7f90bc08b000 msgr2=0x7f90bc09ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:53.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:53.742+0000 7f90c2e24700 1 -- 172.21.15.143:0/1075007332 shutdown_connections 2023-11-08T19:26:53.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:53.742+0000 7f90c2e24700 1 -- 172.21.15.143:0/1075007332 wait complete. 2023-11-08T19:26:53.751 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:53.743+0000 7f90c2e24700 1 Processor -- start 2023-11-08T19:26:53.751 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:53.743+0000 7f90c2e24700 1 -- start start 2023-11-08T19:26:53.752 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:53.744+0000 7f90c2e24700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f90bc0d03a0 0x7f90bc157150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:53.752 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:53.744+0000 7f90c2e24700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f90bc0d1c70 0x7f90bc0b53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:53.752 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:53.744+0000 7f90c2e24700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f90bc15ba30 0x7f90bc0b6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:53.752 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:53.744+0000 7f90c2e24700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90bc0b6190 con 0x7f90bc0d1c70 2023-11-08T19:26:53.752 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:53.744+0000 7f90c2e24700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90bc0c8e70 con 0x7f90bc15ba30 2023-11-08T19:26:53.753 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:53.744+0000 7f90c2e24700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90bc0b5930 con 0x7f90bc0d03a0 2023-11-08T19:26:53.753 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:53.744+0000 7f90bbfff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f90bc0d1c70 0x7f90bc0b53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:53.753 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:53.744+0000 7f90bbfff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f90bc0d1c70 0x7f90bc0b53f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52398/0 (socket says 172.21.15.143:52398) 2023-11-08T19:26:53.753 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:53.744+0000 7f90bbfff700 1 -- 172.21.15.143:0/2032963532 learned_addr learned my addr 172.21.15.143:0/2032963532 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:53.753 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:53.744+0000 7f90c0bc0700 1 --2- 172.21.15.143:0/2032963532 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f90bc0d03a0 0x7f90bc157150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:53.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:53.744+0000 7f90c13c1700 1 --2- 172.21.15.143:0/2032963532 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f90bc15ba30 0x7f90bc0b6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:53.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:53.744+0000 7f90bbfff700 1 -- 172.21.15.143:0/2032963532 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f90bc0d03a0 msgr2=0x7f90bc157150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:53.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:53.744+0000 7f90bbfff700 1 --2- 172.21.15.143:0/2032963532 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f90bc0d03a0 0x7f90bc157150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:53.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:53.744+0000 7f90bbfff700 1 -- 172.21.15.143:0/2032963532 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f90bc15ba30 msgr2=0x7f90bc0b6330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:53.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:53.744+0000 7f90bbfff700 1 --2- 172.21.15.143:0/2032963532 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f90bc15ba30 0x7f90bc0b6330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:53.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:53.744+0000 7f90bbfff700 1 -- 172.21.15.143:0/2032963532 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90b0009550 con 0x7f90bc0d1c70 2023-11-08T19:26:53.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:53.745+0000 7f90bbfff700 1 --2- 172.21.15.143:0/2032963532 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f90bc0d1c70 0x7f90bc0b53f0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f90a800c4e0 tx=0x7f90a800c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:53.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:53.745+0000 7f90b97fa700 1 -- 172.21.15.143:0/2032963532 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90a80052d0 con 0x7f90bc0d1c70 2023-11-08T19:26:53.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:53.745+0000 7f90b97fa700 1 -- 172.21.15.143:0/2032963532 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90a8005430 con 0x7f90bc0d1c70 2023-11-08T19:26:53.756 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:53.745+0000 7f90c2e24700 1 -- 172.21.15.143:0/2032963532 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90bc0b5bd0 con 0x7f90bc0d1c70 2023-11-08T19:26:53.756 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:53.745+0000 7f90b97fa700 1 -- 172.21.15.143:0/2032963532 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90a80038c0 con 0x7f90bc0d1c70 2023-11-08T19:26:53.756 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:53.745+0000 7f90c2e24700 1 -- 172.21.15.143:0/2032963532 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90bc0c8b00 con 0x7f90bc0d1c70 2023-11-08T19:26:53.756 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:53.746+0000 7f90c2e24700 1 -- 172.21.15.143:0/2032963532 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90bc0c8cc0 con 0x7f90bc0d1c70 2023-11-08T19:26:53.756 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:53.746+0000 7f90b97fa700 1 -- 172.21.15.143:0/2032963532 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f90a8003a20 con 0x7f90bc0d1c70 2023-11-08T19:26:53.757 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:53.746+0000 7f90b97fa700 1 --2- 172.21.15.143:0/2032963532 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f90a006af00 0x7f90a006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:53.757 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:53.747+0000 7f90b97fa700 1 -- 172.21.15.143:0/2032963532 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f90a80886e0 con 0x7f90bc0d1c70 2023-11-08T19:26:53.757 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:53.747+0000 7f90c0bc0700 1 --2- 172.21.15.143:0/2032963532 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f90a006af00 0x7f90a006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:53.757 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:53.747+0000 7f90c0bc0700 1 --2- 172.21.15.143:0/2032963532 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f90a006af00 0x7f90a006d3c0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f90b0014040 tx=0x7f90b0008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:53.758 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:53.751+0000 7f90b97fa700 1 -- 172.21.15.143:0/2032963532 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f90a8053a10 con 0x7f90bc0d1c70 2023-11-08T19:26:54.015 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.014+0000 7f90c2e24700 1 -- 172.21.15.143:0/2032963532 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f90bc0c8fe0 con 0x7f90bc0d1c70 2023-11-08T19:26:54.016 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.014+0000 7f90b97fa700 1 -- 172.21.15.143:0/2032963532 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v10) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f90a8057030 con 0x7f90bc0d1c70 2023-11-08T19:26:54.016 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:26:54.016 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2023-11-08T19:26:54.016 INFO:teuthology.orchestra.run.smithi143.stdout: "auth_dump": [ 2023-11-08T19:26:54.017 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-08T19:26:54.017 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "mds.a", 2023-11-08T19:26:54.017 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQBO4EtlqQPkOhAACwt3IOZLFtEQle9/FFliYA==", 2023-11-08T19:26:54.017 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-08T19:26:54.017 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2023-11-08T19:26:54.018 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow *", 2023-11-08T19:26:54.018 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow *", 2023-11-08T19:26:54.018 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-08T19:26:54.018 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-08T19:26:54.018 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-08T19:26:54.019 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-08T19:26:54.019 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "mds.b", 2023-11-08T19:26:54.019 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQBP4EtlHJqqCxAAxgT7Ce6QICWZ0qebHWZhMQ==", 2023-11-08T19:26:54.019 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-08T19:26:54.019 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2023-11-08T19:26:54.020 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow *", 2023-11-08T19:26:54.020 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow *", 2023-11-08T19:26:54.020 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-08T19:26:54.020 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-08T19:26:54.020 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-08T19:26:54.021 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-08T19:26:54.021 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "mds.c", 2023-11-08T19:26:54.021 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQBP4EtlGEeDBhAAihr82JiBJZR0rRkQj2n8wA==", 2023-11-08T19:26:54.021 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-08T19:26:54.021 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2023-11-08T19:26:54.022 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow *", 2023-11-08T19:26:54.022 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow *", 2023-11-08T19:26:54.022 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-08T19:26:54.022 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-08T19:26:54.022 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-08T19:26:54.023 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-08T19:26:54.023 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "mds.d", 2023-11-08T19:26:54.023 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQBP4EtlW5//EhAArVOkzNpCYd6Skl6eh5EYUQ==", 2023-11-08T19:26:54.023 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-08T19:26:54.023 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2023-11-08T19:26:54.024 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow *", 2023-11-08T19:26:54.024 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow *", 2023-11-08T19:26:54.024 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-08T19:26:54.024 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-08T19:26:54.024 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-08T19:26:54.025 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-08T19:26:54.025 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.0", 2023-11-08T19:26:54.025 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQBS4EtloVd4KRAAi5nj/vB9UK/mWZ9lkaFAxg==", 2023-11-08T19:26:54.025 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-08T19:26:54.025 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2023-11-08T19:26:54.026 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2023-11-08T19:26:54.026 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-08T19:26:54.026 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-08T19:26:54.026 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-08T19:26:54.026 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-08T19:26:54.027 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.1", 2023-11-08T19:26:54.027 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQBV4EtlWlOaKRAAooz5P+AoTgyuL/QAv4REKA==", 2023-11-08T19:26:54.027 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-08T19:26:54.027 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2023-11-08T19:26:54.027 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2023-11-08T19:26:54.028 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-08T19:26:54.028 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-08T19:26:54.028 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-08T19:26:54.028 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-08T19:26:54.028 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.2", 2023-11-08T19:26:54.029 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQBY4EtlBpaWKhAAmob9TfEKw5x6dEIu8ALVGA==", 2023-11-08T19:26:54.029 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-08T19:26:54.029 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2023-11-08T19:26:54.029 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2023-11-08T19:26:54.029 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-08T19:26:54.030 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-08T19:26:54.030 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-08T19:26:54.030 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-08T19:26:54.030 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.3", 2023-11-08T19:26:54.030 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQBb4Etl5pkrKhAA38ATMC3NkZ4plRRA69i0vw==", 2023-11-08T19:26:54.031 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-08T19:26:54.031 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2023-11-08T19:26:54.031 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2023-11-08T19:26:54.031 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-08T19:26:54.031 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-08T19:26:54.032 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-08T19:26:54.032 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-08T19:26:54.032 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.4", 2023-11-08T19:26:54.032 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQBh4EtlqLnUMBAABIrSskahtrZR0ClfczX5Ag==", 2023-11-08T19:26:54.032 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-08T19:26:54.033 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2023-11-08T19:26:54.033 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2023-11-08T19:26:54.033 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-08T19:26:54.033 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-08T19:26:54.033 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-08T19:26:54.034 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-08T19:26:54.034 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.5", 2023-11-08T19:26:54.034 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQBk4Etlt9PTMBAAz55IHg6Jrin3BUAvhOM/Fg==", 2023-11-08T19:26:54.034 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-08T19:26:54.034 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2023-11-08T19:26:54.035 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2023-11-08T19:26:54.035 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-08T19:26:54.035 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-08T19:26:54.035 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-08T19:26:54.035 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-08T19:26:54.036 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.6", 2023-11-08T19:26:54.036 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQBn4EtlcTkyMBAA5fD3h4s+99rgd6fqHHCFMw==", 2023-11-08T19:26:54.036 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-08T19:26:54.036 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2023-11-08T19:26:54.036 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2023-11-08T19:26:54.037 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-08T19:26:54.037 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-08T19:26:54.037 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-08T19:26:54.037 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-08T19:26:54.037 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.7", 2023-11-08T19:26:54.038 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQBq4EtldEviMBAAyGYrHVGwxHdmtDOwWCSHDQ==", 2023-11-08T19:26:54.038 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-08T19:26:54.038 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2023-11-08T19:26:54.038 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2023-11-08T19:26:54.038 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-08T19:26:54.039 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-08T19:26:54.039 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-08T19:26:54.039 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-08T19:26:54.039 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "client.0", 2023-11-08T19:26:54.039 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQBP4Etl0GdSFxAAVkuHL0EnjrtTUMkj/afPkg==", 2023-11-08T19:26:54.040 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-08T19:26:54.040 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2023-11-08T19:26:54.040 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow r", 2023-11-08T19:26:54.040 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-11-08T19:26:54.040 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-08T19:26:54.041 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-08T19:26:54.041 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-08T19:26:54.041 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "client.1", 2023-11-08T19:26:54.041 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQBP4EtlPQ5yIBAAiyQaqCJph2KOPBZPSIh4ZQ==", 2023-11-08T19:26:54.041 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-08T19:26:54.042 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2023-11-08T19:26:54.042 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow r", 2023-11-08T19:26:54.042 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow rw", 2023-11-08T19:26:54.042 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow rwx" 2023-11-08T19:26:54.042 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-08T19:26:54.043 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-08T19:26:54.043 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-08T19:26:54.043 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "client.2", 2023-11-08T19:26:54.043 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQBP4Etlm+8SHRAAAwxcy2EzUvtFw9/mTzyk7w==", 2023-11-08T19:26:54.043 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-08T19:26:54.044 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2023-11-08T19:26:54.044 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow r", 2023-11-08T19:26:54.044 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow rw", 2023-11-08T19:26:54.044 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow rwx" 2023-11-08T19:26:54.044 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-08T19:26:54.044 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-08T19:26:54.045 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-08T19:26:54.045 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "client.3", 2023-11-08T19:26:54.045 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQBP4EtlNDC8IxAACBkCM+3TXhJv5eE9xLvwKA==", 2023-11-08T19:26:54.045 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-08T19:26:54.046 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2023-11-08T19:26:54.046 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow r", 2023-11-08T19:26:54.046 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow rw", 2023-11-08T19:26:54.046 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow rwx" 2023-11-08T19:26:54.046 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-08T19:26:54.047 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-08T19:26:54.047 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-08T19:26:54.047 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "client.admin", 2023-11-08T19:26:54.047 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQBO4Etlr5yuHRAAblpqD18z9bT3Wmgn74dehg==", 2023-11-08T19:26:54.047 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-08T19:26:54.048 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow *", 2023-11-08T19:26:54.048 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow *", 2023-11-08T19:26:54.048 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow *", 2023-11-08T19:26:54.048 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-08T19:26:54.048 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-08T19:26:54.049 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-08T19:26:54.049 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-08T19:26:54.049 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "mgr.x", 2023-11-08T19:26:54.049 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQBO4EtlzrqBNxAA5rtcgkzroSTKuBBmeQRiig==", 2023-11-08T19:26:54.049 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-08T19:26:54.050 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow *", 2023-11-08T19:26:54.050 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile mgr", 2023-11-08T19:26:54.050 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-08T19:26:54.050 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-08T19:26:54.050 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-08T19:26:54.051 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-08T19:26:54.051 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "mgr.y", 2023-11-08T19:26:54.051 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQBO4Etl4Gl4MxAArxdUnjLDhTLO3feXHV4HyQ==", 2023-11-08T19:26:54.051 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-08T19:26:54.051 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow *", 2023-11-08T19:26:54.052 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile mgr", 2023-11-08T19:26:54.052 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-08T19:26:54.052 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-08T19:26:54.052 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-08T19:26:54.052 INFO:teuthology.orchestra.run.smithi143.stdout: ] 2023-11-08T19:26:54.052 INFO:teuthology.orchestra.run.smithi143.stdout:} 2023-11-08T19:26:54.054 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.016+0000 7f90c2e24700 1 -- 172.21.15.143:0/2032963532 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f90a006af00 msgr2=0x7f90a006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:54.054 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.016+0000 7f90c2e24700 1 --2- 172.21.15.143:0/2032963532 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f90a006af00 0x7f90a006d3c0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f90b0014040 tx=0x7f90b0008030 comp rx=0 tx=0).stop 2023-11-08T19:26:54.055 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.016+0000 7f90c2e24700 1 -- 172.21.15.143:0/2032963532 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f90bc0d1c70 msgr2=0x7f90bc0b53f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:54.055 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.016+0000 7f90c2e24700 1 --2- 172.21.15.143:0/2032963532 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f90bc0d1c70 0x7f90bc0b53f0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f90a800c4e0 tx=0x7f90a800c7f0 comp rx=0 tx=0).stop 2023-11-08T19:26:54.055 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.016+0000 7f90c2e24700 1 -- 172.21.15.143:0/2032963532 shutdown_connections 2023-11-08T19:26:54.055 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.016+0000 7f90c2e24700 1 --2- 172.21.15.143:0/2032963532 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f90a006af00 0x7f90a006d3c0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:54.055 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.016+0000 7f90c2e24700 1 --2- 172.21.15.143:0/2032963532 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f90bc0d03a0 0x7f90bc157150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:54.056 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.016+0000 7f90c2e24700 1 --2- 172.21.15.143:0/2032963532 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f90bc15ba30 0x7f90bc0b6330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:54.056 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.016+0000 7f90c2e24700 1 --2- 172.21.15.143:0/2032963532 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f90bc0d1c70 0x7f90bc0b53f0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:54.056 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.016+0000 7f90c2e24700 1 -- 172.21.15.143:0/2032963532 >> 172.21.15.143:0/2032963532 conn(0x7f90bc08b000 msgr2=0x7f90bc0afed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:54.056 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.017+0000 7f90c2e24700 1 -- 172.21.15.143:0/2032963532 shutdown_connections 2023-11-08T19:26:54.056 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.017+0000 7f90c2e24700 1 -- 172.21.15.143:0/2032963532 wait complete. 2023-11-08T19:26:54.057 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:26:54.178 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.177+0000 7f9472ff5700 1 Processor -- start 2023-11-08T19:26:54.179 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.177+0000 7f9472ff5700 1 -- start start 2023-11-08T19:26:54.181 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.178+0000 7f9472ff5700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f946c1618f0 0x7f946c06ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:54.181 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.178+0000 7f9472ff5700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f946c06b150 0x7f946c0cb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:54.182 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.178+0000 7f9472ff5700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f946c161df0 con 0x7f946c15dd00 2023-11-08T19:26:54.182 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.178+0000 7f9472ff5700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f946c06b600 con 0x7f946c1618f0 2023-11-08T19:26:54.182 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.178+0000 7f9472ff5700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f946c0cad50 con 0x7f946c06b150 2023-11-08T19:26:54.182 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.178+0000 7f946bfff700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f946c1618f0 0x7f946c06ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:54.182 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.178+0000 7f946bfff700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f946c1618f0 0x7f946c06ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52406/0 (socket says 172.21.15.143:52406) 2023-11-08T19:26:54.183 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.178+0000 7f946bfff700 1 -- 172.21.15.143:0/1173977124 learned_addr learned my addr 172.21.15.143:0/1173977124 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:54.183 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.178+0000 7f9471592700 1 --2- 172.21.15.143:0/1173977124 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f946c06b150 0x7f946c0cb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:54.183 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.178+0000 7f946bfff700 1 -- 172.21.15.143:0/1173977124 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f946c06b150 msgr2=0x7f946c0cb1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:54.183 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.178+0000 7f946bfff700 1 --2- 172.21.15.143:0/1173977124 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f946c06b150 0x7f946c0cb1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:54.184 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.178+0000 7f946bfff700 1 -- 172.21.15.143:0/1173977124 >> v1:172.21.15.174:6789/0 conn(0x7f946c15dd00 legacy=0x7f946c0cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:54.184 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.178+0000 7f946bfff700 1 -- 172.21.15.143:0/1173977124 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f946c1620d0 con 0x7f946c1618f0 2023-11-08T19:26:54.184 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.178+0000 7f946bfff700 1 --2- 172.21.15.143:0/1173977124 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f946c1618f0 0x7f946c06ac10 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f94580098f0 tx=0x7f9458009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=256e3233b58b8714 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:54.184 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.178+0000 7f946b7fe700 1 -- 172.21.15.143:0/1173977124 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f945800b8a0 con 0x7f946c1618f0 2023-11-08T19:26:54.184 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.179+0000 7f946b7fe700 1 -- 172.21.15.143:0/1173977124 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f945800ba00 con 0x7f946c1618f0 2023-11-08T19:26:54.185 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.179+0000 7f946b7fe700 1 -- 172.21.15.143:0/1173977124 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f945800bd90 con 0x7f946c1618f0 2023-11-08T19:26:54.185 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.179+0000 7f9472ff5700 1 -- 172.21.15.143:0/1173977124 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f946c1618f0 msgr2=0x7f946c06ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:54.185 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.179+0000 7f9472ff5700 1 --2- 172.21.15.143:0/1173977124 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f946c1618f0 0x7f946c06ac10 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f94580098f0 tx=0x7f9458009c00 comp rx=0 tx=0).stop 2023-11-08T19:26:54.185 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.179+0000 7f9472ff5700 1 -- 172.21.15.143:0/1173977124 shutdown_connections 2023-11-08T19:26:54.185 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.179+0000 7f9472ff5700 1 --2- 172.21.15.143:0/1173977124 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f946c06b150 0x7f946c0cb1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:54.186 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.179+0000 7f9472ff5700 1 --2- 172.21.15.143:0/1173977124 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f946c1618f0 0x7f946c06ac10 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:54.186 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.179+0000 7f9472ff5700 1 -- 172.21.15.143:0/1173977124 >> 172.21.15.143:0/1173977124 conn(0x7f946c099ab0 msgr2=0x7f946c0d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:54.186 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.179+0000 7f9472ff5700 1 -- 172.21.15.143:0/1173977124 shutdown_connections 2023-11-08T19:26:54.186 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.179+0000 7f9472ff5700 1 -- 172.21.15.143:0/1173977124 wait complete. 2023-11-08T19:26:54.189 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.181+0000 7f9472ff5700 1 Processor -- start 2023-11-08T19:26:54.189 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.181+0000 7f9472ff5700 1 -- start start 2023-11-08T19:26:54.190 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.182+0000 7f9472ff5700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f946c06b150 0x7f946c0b25c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:54.190 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.182+0000 7f9472ff5700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f946c15dd00 0x7f946c157e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:54.190 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.182+0000 7f9472ff5700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f946c1618f0 0x7f946c0c62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:54.190 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.182+0000 7f9472ff5700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f946c0b2b00 con 0x7f946c1618f0 2023-11-08T19:26:54.190 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.182+0000 7f9472ff5700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f946c0b1870 con 0x7f946c06b150 2023-11-08T19:26:54.191 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.182+0000 7f9472ff5700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f946c0b1ae0 con 0x7f946c15dd00 2023-11-08T19:26:54.191 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.182+0000 7f9471592700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f946c1618f0 0x7f946c0c62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:54.191 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.182+0000 7f9471592700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f946c1618f0 0x7f946c0c62e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52414/0 (socket says 172.21.15.143:52414) 2023-11-08T19:26:54.191 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.182+0000 7f9471592700 1 -- 172.21.15.143:0/1065122998 learned_addr learned my addr 172.21.15.143:0/1065122998 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:54.191 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.182+0000 7f946bfff700 1 --2- 172.21.15.143:0/1065122998 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f946c15dd00 0x7f946c157e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:54.192 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.182+0000 7f9470d91700 1 --2- 172.21.15.143:0/1065122998 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f946c06b150 0x7f946c0b25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:54.192 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.182+0000 7f9471592700 1 -- 172.21.15.143:0/1065122998 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f946c15dd00 msgr2=0x7f946c157e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:54.192 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.182+0000 7f9471592700 1 --2- 172.21.15.143:0/1065122998 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f946c15dd00 0x7f946c157e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:54.192 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.182+0000 7f9471592700 1 -- 172.21.15.143:0/1065122998 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f946c06b150 msgr2=0x7f946c0b25c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:54.193 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.182+0000 7f9471592700 1 --2- 172.21.15.143:0/1065122998 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f946c06b150 0x7f946c0b25c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:54.193 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.182+0000 7f9471592700 1 -- 172.21.15.143:0/1065122998 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9458009550 con 0x7f946c1618f0 2023-11-08T19:26:54.193 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.182+0000 7f9470d91700 1 --2- 172.21.15.143:0/1065122998 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f946c06b150 0x7f946c0b25c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:26:54.193 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.182+0000 7f9471592700 1 --2- 172.21.15.143:0/1065122998 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f946c1618f0 0x7f946c0c62e0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f9464007e40 tx=0x7f946400e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:54.194 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.182+0000 7f94697fa700 1 -- 172.21.15.143:0/1065122998 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f946400ede0 con 0x7f946c1618f0 2023-11-08T19:26:54.194 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.182+0000 7f94697fa700 1 -- 172.21.15.143:0/1065122998 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9464014420 con 0x7f946c1618f0 2023-11-08T19:26:54.194 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.182+0000 7f94697fa700 1 -- 172.21.15.143:0/1065122998 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94640147d0 con 0x7f946c1618f0 2023-11-08T19:26:54.194 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.182+0000 7f9472ff5700 1 -- 172.21.15.143:0/1065122998 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f946c0b1dc0 con 0x7f946c1618f0 2023-11-08T19:26:54.194 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.182+0000 7f9472ff5700 1 -- 172.21.15.143:0/1065122998 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f946c1f8960 con 0x7f946c1618f0 2023-11-08T19:26:54.195 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.183+0000 7f9472ff5700 1 -- 172.21.15.143:0/1065122998 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f946c0aff60 con 0x7f946c1618f0 2023-11-08T19:26:54.195 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.184+0000 7f94697fa700 1 -- 172.21.15.143:0/1065122998 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9464014930 con 0x7f946c1618f0 2023-11-08T19:26:54.195 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.184+0000 7f94697fa700 1 --2- 172.21.15.143:0/1065122998 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f945006af00 0x7f945006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:54.195 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.184+0000 7f94697fa700 1 -- 172.21.15.143:0/1065122998 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f9464089b20 con 0x7f946c1618f0 2023-11-08T19:26:54.195 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.184+0000 7f9470d91700 1 --2- 172.21.15.143:0/1065122998 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f945006af00 0x7f945006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:54.196 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.185+0000 7f9470d91700 1 --2- 172.21.15.143:0/1065122998 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f945006af00 0x7f945006d3c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f9460000fb0 tx=0x7f9460009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:54.196 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.189+0000 7f94697fa700 1 -- 172.21.15.143:0/1065122998 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9464054dc0 con 0x7f946c1618f0 2023-11-08T19:26:54.382 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.381+0000 7f9472ff5700 1 -- 172.21.15.143:0/1065122998 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f946c0b22e0 con 0x7f946c1618f0 2023-11-08T19:26:54.383 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.382+0000 7f94697fa700 1 -- 172.21.15.143:0/1065122998 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) v1 ==== 74+0+14658 (secure 0 0 0) 0x7f94640583e0 con 0x7f946c1618f0 2023-11-08T19:26:54.383 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:26:54.383 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":58,"fsid":"968483af-77fc-4068-ad7c-5c2a1a4613d7","created":"2023-11-08T19:24:32.826484+0000","modified":"2023-11-08T19:26:53.594824+0000","last_up_change":"2023-11-08T19:24:43.573880+0000","last_in_change":"2023-11-08T19:24:35.314525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:44.551530+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"snap-test-pool","create_time":"2023-11-08T19:26:02.874384+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":36,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bae98ec1-d94e-403d-a220-a522a53ea6f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6809","nonce":1487830566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6811","nonce":1487830566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6815","nonce":1487830566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6813","nonce":1487830566}]},"public_addr":"172.21.15.143:6809/1487830566","cluster_addr":"172.21.15.143:6811/1487830566","heartbeat_back_addr":"172.21.15.143:6815/1487830566","heartbeat_front_addr":"172.21.15.143:6813/1487830566","state":["exists","up"]},{"osd":1,"uuid":"5cb6538a-1ccb-4322-8633-09f66af0b7e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6801","nonce":787341397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6803","nonce":787341397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6807","nonce":787341397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6805","nonce":787341397}]},"public_addr":"172.21.15.143:6801/787341397","cluster_addr":"172.21.15.143:6803/787341397","heartbeat_back_addr":"172.21.15.143:6807/787341397","heartbeat_front_addr":"172.21.15.143:6805/787341397","state":["exists","up"]},{"osd":2,"uuid":"cdbf522c-60ef-4d60-9653-8dcb529c101a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6822","nonce":2301325597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6826","nonce":2301325597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6831","nonce":2301325597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6829","nonce":2301325597}]},"public_addr":"172.21.15.143:6822/2301325597","cluster_addr":"172.21.15.143:6826/2301325597","heartbeat_back_addr":"172.21.15.143:6831/2301325597","heartbeat_front_addr":"172.21.15.143:6829/2301325597","state":["exists","up"]},{"osd":3,"uuid":"4e2783cf-ebb9-4dd8-8245-bc1d17ce4325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6817","nonce":2021504428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6819","nonce":2021504428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6827","nonce":2021504428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6823","nonce":2021504428}]},"public_addr":"172.21.15.143:6817/2021504428","cluster_addr":"172.21.15.143:6819/2021504428","heartbeat_back_addr":"172.21.15.143:6827/2021504428","heartbeat_front_addr":"172.21.15.143:6823/2021504428","state":["exists","up"]},{"osd":4,"uuid":"691256fa-9b77-49b9-a346-7ae1adf0ca80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6801","nonce":1993380595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6803","nonce":1993380595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6807","nonce":1993380595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6805","nonce":1993380595}]},"public_addr":"172.21.15.174:6801/1993380595","cluster_addr":"172.21.15.174:6803/1993380595","heartbeat_back_addr":"172.21.15.174:6807/1993380595","heartbeat_front_addr":"172.21.15.174:6805/1993380595","state":["exists","up"]},{"osd":5,"uuid":"756fc352-48c9-46ce-b52e-0541fe6f8525","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6809","nonce":1777923303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6811","nonce":1777923303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6815","nonce":1777923303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6813","nonce":1777923303}]},"public_addr":"172.21.15.174:6809/1777923303","cluster_addr":"172.21.15.174:6811/1777923303","heartbeat_back_addr":"172.21.15.174:6815/1777923303","heartbeat_front_addr":"172.21.15.174:6813/1777923303","state":["exists","up"]},{"osd":6,"uuid":"fab2ac2b-ed97-4cde-9825-b480eabf8b8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6825","nonce":2755130316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6827","nonce":2755130316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6831","nonce":2755130316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6829","nonce":2755130316}]},"public_addr":"172.21.15.174:6825/2755130316","cluster_addr":"172.21.15.174:6827/2755130316","heartbeat_back_addr":"172.21.15.174:6831/2755130316","heartbeat_front_addr":"172.21.15.174:6829/2755130316","state":["exists","up"]},{"osd":7,"uuid":"3de9b797-2ab6-4d41-a863-7e7d638fd703","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6817","nonce":2847141411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6819","nonce":2847141411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6823","nonce":2847141411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6821","nonce":2847141411}]},"public_addr":"172.21.15.174:6817/2847141411","cluster_addr":"172.21.15.174:6819/2847141411","heartbeat_back_addr":"172.21.15.174:6823/2847141411","heartbeat_front_addr":"172.21.15.174:6821/2847141411","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.477076+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.496890+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.539305+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.454673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.538881+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.521294+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.520712+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.540404+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:26:54.386 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.384+0000 7f9472ff5700 1 -- 172.21.15.143:0/1065122998 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f945006af00 msgr2=0x7f945006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:54.386 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.384+0000 7f9472ff5700 1 --2- 172.21.15.143:0/1065122998 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f945006af00 0x7f945006d3c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f9460000fb0 tx=0x7f9460009040 comp rx=0 tx=0).stop 2023-11-08T19:26:54.386 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.384+0000 7f9472ff5700 1 -- 172.21.15.143:0/1065122998 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f946c1618f0 msgr2=0x7f946c0c62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:54.386 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.384+0000 7f9472ff5700 1 --2- 172.21.15.143:0/1065122998 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f946c1618f0 0x7f946c0c62e0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f9464007e40 tx=0x7f946400e3c0 comp rx=0 tx=0).stop 2023-11-08T19:26:54.387 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.384+0000 7f9472ff5700 1 -- 172.21.15.143:0/1065122998 shutdown_connections 2023-11-08T19:26:54.387 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.384+0000 7f9472ff5700 1 --2- 172.21.15.143:0/1065122998 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f945006af00 0x7f945006d3c0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:54.387 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.384+0000 7f9472ff5700 1 --2- 172.21.15.143:0/1065122998 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f946c1618f0 0x7f946c0c62e0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:54.387 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.384+0000 7f9472ff5700 1 --2- 172.21.15.143:0/1065122998 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f946c06b150 0x7f946c0b25c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:54.387 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.384+0000 7f9472ff5700 1 --2- 172.21.15.143:0/1065122998 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f946c15dd00 0x7f946c157e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:54.388 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.384+0000 7f9472ff5700 1 -- 172.21.15.143:0/1065122998 >> 172.21.15.143:0/1065122998 conn(0x7f946c099ab0 msgr2=0x7f946c0ad560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:54.388 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.384+0000 7f9472ff5700 1 -- 172.21.15.143:0/1065122998 shutdown_connections 2023-11-08T19:26:54.388 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.384+0000 7f9472ff5700 1 -- 172.21.15.143:0/1065122998 wait complete. 2023-11-08T19:26:54.398 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T19:24:44.551530+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'snap-test-pool', 'create_time': '2023-11-08T19:26:02.874384+0000', 'flags': 16385, 'flags_names': 'hashpspool,pool_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '36', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'pool', 'snap_seq': 2, 'snap_epoch': 36, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-08T19:26:54.399 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T19:26:54.522 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.520+0000 7fa3536d2700 1 Processor -- start 2023-11-08T19:26:54.522 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.521+0000 7fa3536d2700 1 -- start start 2023-11-08T19:26:54.530 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.521+0000 7fa3536d2700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa34c15dba0 0x7fa34c06c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:54.530 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.521+0000 7fa3536d2700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa34c0b22e0 0x7fa34c0b2770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:54.530 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.521+0000 7fa3536d2700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa34c161890 con 0x7fa34c06b240 2023-11-08T19:26:54.530 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.521+0000 7fa3536d2700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa34c0b2cb0 con 0x7fa34c0b22e0 2023-11-08T19:26:54.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.521+0000 7fa3536d2700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa34c06b740 con 0x7fa34c15dba0 2023-11-08T19:26:54.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.522+0000 7fa351c6f700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa34c0b22e0 0x7fa34c0b2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:54.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.522+0000 7fa351c6f700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa34c0b22e0 0x7fa34c0b2770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52420/0 (socket says 172.21.15.143:52420) 2023-11-08T19:26:54.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.522+0000 7fa351c6f700 1 -- 172.21.15.143:0/4259154043 learned_addr learned my addr 172.21.15.143:0/4259154043 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:54.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.522+0000 7fa35146e700 1 --2- 172.21.15.143:0/4259154043 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa34c15dba0 0x7fa34c06c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:54.532 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.522+0000 7fa351c6f700 1 -- 172.21.15.143:0/4259154043 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa34c15dba0 msgr2=0x7fa34c06c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:54.532 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.522+0000 7fa351c6f700 1 --2- 172.21.15.143:0/4259154043 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa34c15dba0 0x7fa34c06c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:54.532 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.522+0000 7fa351c6f700 1 -- 172.21.15.143:0/4259154043 >> v1:172.21.15.174:6789/0 conn(0x7fa34c06b240 legacy=0x7fa34c0caff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:54.532 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.522+0000 7fa351c6f700 1 -- 172.21.15.143:0/4259154043 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa34c06ba20 con 0x7fa34c0b22e0 2023-11-08T19:26:54.532 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.522+0000 7fa351c6f700 1 --2- 172.21.15.143:0/4259154043 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa34c0b22e0 0x7fa34c0b2770 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fa344009e50 tx=0x7fa34400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3254cdb2f1c6c5ab server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:54.533 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.522+0000 7fa343fff700 1 -- 172.21.15.143:0/4259154043 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa34400e070 con 0x7fa34c0b22e0 2023-11-08T19:26:54.533 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.522+0000 7fa343fff700 1 -- 172.21.15.143:0/4259154043 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa34400b9c0 con 0x7fa34c0b22e0 2023-11-08T19:26:54.533 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.522+0000 7fa343fff700 1 -- 172.21.15.143:0/4259154043 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa34400bd70 con 0x7fa34c0b22e0 2023-11-08T19:26:54.533 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.523+0000 7fa3536d2700 1 -- 172.21.15.143:0/4259154043 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa34c0b22e0 msgr2=0x7fa34c0b2770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:54.533 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.523+0000 7fa3536d2700 1 --2- 172.21.15.143:0/4259154043 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa34c0b22e0 0x7fa34c0b2770 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fa344009e50 tx=0x7fa34400b330 comp rx=0 tx=0).stop 2023-11-08T19:26:54.534 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.523+0000 7fa3536d2700 1 -- 172.21.15.143:0/4259154043 shutdown_connections 2023-11-08T19:26:54.534 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.523+0000 7fa3536d2700 1 --2- 172.21.15.143:0/4259154043 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa34c0b22e0 0x7fa34c0b2770 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:54.534 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.523+0000 7fa3536d2700 1 --2- 172.21.15.143:0/4259154043 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa34c15dba0 0x7fa34c06c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:54.534 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.523+0000 7fa3536d2700 1 -- 172.21.15.143:0/4259154043 >> 172.21.15.143:0/4259154043 conn(0x7fa34c0cd2e0 msgr2=0x7fa34c0cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:54.535 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.523+0000 7fa3536d2700 1 -- 172.21.15.143:0/4259154043 shutdown_connections 2023-11-08T19:26:54.535 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.523+0000 7fa3536d2700 1 -- 172.21.15.143:0/4259154043 wait complete. 2023-11-08T19:26:54.535 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.524+0000 7fa3536d2700 1 Processor -- start 2023-11-08T19:26:54.535 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.524+0000 7fa3536d2700 1 -- start start 2023-11-08T19:26:54.535 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.525+0000 7fa3536d2700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa34c06b240 0x7fa34c0c4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:54.536 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.525+0000 7fa3536d2700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa34c0b22e0 0x7fa34c15e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:54.536 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.525+0000 7fa3536d2700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa34c15dba0 0x7fa34c15aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:54.536 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.525+0000 7fa3536d2700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa34c0c5580 con 0x7fa34c0b22e0 2023-11-08T19:26:54.536 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.525+0000 7fa3536d2700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa34c15eed0 con 0x7fa34c15dba0 2023-11-08T19:26:54.536 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.525+0000 7fa3536d2700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa34c0c4a00 con 0x7fa34c06b240 2023-11-08T19:26:54.536 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.525+0000 7fa350c6d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa34c0b22e0 0x7fa34c15e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:54.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.525+0000 7fa350c6d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa34c0b22e0 0x7fa34c15e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52424/0 (socket says 172.21.15.143:52424) 2023-11-08T19:26:54.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.525+0000 7fa350c6d700 1 -- 172.21.15.143:0/3495319249 learned_addr learned my addr 172.21.15.143:0/3495319249 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:54.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.525+0000 7fa35146e700 1 --2- 172.21.15.143:0/3495319249 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa34c06b240 0x7fa34c0c4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:54.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.525+0000 7fa351c6f700 1 --2- 172.21.15.143:0/3495319249 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa34c15dba0 0x7fa34c15aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:54.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.525+0000 7fa350c6d700 1 -- 172.21.15.143:0/3495319249 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa34c06b240 msgr2=0x7fa34c0c4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:54.538 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.525+0000 7fa350c6d700 1 --2- 172.21.15.143:0/3495319249 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa34c06b240 0x7fa34c0c4390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:54.538 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.525+0000 7fa350c6d700 1 -- 172.21.15.143:0/3495319249 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa34c15dba0 msgr2=0x7fa34c15aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:54.538 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.525+0000 7fa350c6d700 1 --2- 172.21.15.143:0/3495319249 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa34c15dba0 0x7fa34c15aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:54.538 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.525+0000 7fa350c6d700 1 -- 172.21.15.143:0/3495319249 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa344009b30 con 0x7fa34c0b22e0 2023-11-08T19:26:54.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.525+0000 7fa350c6d700 1 --2- 172.21.15.143:0/3495319249 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa34c0b22e0 0x7fa34c15e990 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fa33c00c4e0 tx=0x7fa33c00c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:54.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.525+0000 7fa341ffb700 1 -- 172.21.15.143:0/3495319249 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa33c0052d0 con 0x7fa34c0b22e0 2023-11-08T19:26:54.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.525+0000 7fa341ffb700 1 -- 172.21.15.143:0/3495319249 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa33c005430 con 0x7fa34c0b22e0 2023-11-08T19:26:54.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.525+0000 7fa3536d2700 1 -- 172.21.15.143:0/3495319249 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa34c0c4ce0 con 0x7fa34c0b22e0 2023-11-08T19:26:54.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.525+0000 7fa341ffb700 1 -- 172.21.15.143:0/3495319249 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa33c0038c0 con 0x7fa34c0b22e0 2023-11-08T19:26:54.540 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.526+0000 7fa3536d2700 1 -- 172.21.15.143:0/3495319249 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa34c0c52a0 con 0x7fa34c0b22e0 2023-11-08T19:26:54.540 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.526+0000 7fa3536d2700 1 -- 172.21.15.143:0/3495319249 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa34c04eee0 con 0x7fa34c0b22e0 2023-11-08T19:26:54.540 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.527+0000 7fa341ffb700 1 -- 172.21.15.143:0/3495319249 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa33c003a20 con 0x7fa34c0b22e0 2023-11-08T19:26:54.540 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.528+0000 7fa341ffb700 1 --2- 172.21.15.143:0/3495319249 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa33006af00 0x7fa33006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:54.541 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.528+0000 7fa341ffb700 1 -- 172.21.15.143:0/3495319249 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fa33c088a80 con 0x7fa34c0b22e0 2023-11-08T19:26:54.541 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.528+0000 7fa35146e700 1 --2- 172.21.15.143:0/3495319249 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa33006af00 0x7fa33006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:54.541 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.528+0000 7fa35146e700 1 --2- 172.21.15.143:0/3495319249 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa33006af00 0x7fa33006d3c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fa348007a30 tx=0x7fa348008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:54.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.531+0000 7fa341ffb700 1 -- 172.21.15.143:0/3495319249 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa33c053e90 con 0x7fa34c0b22e0 2023-11-08T19:26:54.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.718+0000 7fa3536d2700 1 -- 172.21.15.143:0/3495319249 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa34c0b1f80 con 0x7fa34c0b22e0 2023-11-08T19:26:54.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.719+0000 7fa341ffb700 1 -- 172.21.15.143:0/3495319249 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v58) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa33c0170a0 con 0x7fa34c0b22e0 2023-11-08T19:26:54.721 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2023-11-08T19:26:54.723 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.721+0000 7fa3536d2700 1 -- 172.21.15.143:0/3495319249 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa33006af00 msgr2=0x7fa33006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:54.723 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.721+0000 7fa3536d2700 1 --2- 172.21.15.143:0/3495319249 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa33006af00 0x7fa33006d3c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fa348007a30 tx=0x7fa348008040 comp rx=0 tx=0).stop 2023-11-08T19:26:54.723 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.721+0000 7fa3536d2700 1 -- 172.21.15.143:0/3495319249 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa34c0b22e0 msgr2=0x7fa34c15e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:54.723 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.721+0000 7fa3536d2700 1 --2- 172.21.15.143:0/3495319249 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa34c0b22e0 0x7fa34c15e990 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fa33c00c4e0 tx=0x7fa33c00c7f0 comp rx=0 tx=0).stop 2023-11-08T19:26:54.723 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.721+0000 7fa3536d2700 1 -- 172.21.15.143:0/3495319249 shutdown_connections 2023-11-08T19:26:54.724 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.721+0000 7fa3536d2700 1 --2- 172.21.15.143:0/3495319249 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa33006af00 0x7fa33006d3c0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:54.726 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.721+0000 7fa3536d2700 1 --2- 172.21.15.143:0/3495319249 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa34c06b240 0x7fa34c0c4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:54.726 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.721+0000 7fa3536d2700 1 --2- 172.21.15.143:0/3495319249 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa34c15dba0 0x7fa34c15aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:54.726 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.721+0000 7fa3536d2700 1 --2- 172.21.15.143:0/3495319249 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa34c0b22e0 0x7fa34c15e990 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:54.727 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.721+0000 7fa3536d2700 1 -- 172.21.15.143:0/3495319249 >> 172.21.15.143:0/3495319249 conn(0x7fa34c0cd2e0 msgr2=0x7fa34c15cd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:54.727 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.721+0000 7fa3536d2700 1 -- 172.21.15.143:0/3495319249 shutdown_connections 2023-11-08T19:26:54.727 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.721+0000 7fa3536d2700 1 -- 172.21.15.143:0/3495319249 wait complete. 2023-11-08T19:26:54.734 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get snap-test-pool pg_num 2023-11-08T19:26:54.858 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.857+0000 7f080a247700 1 Processor -- start 2023-11-08T19:26:54.859 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.857+0000 7f080a247700 1 -- start start 2023-11-08T19:26:54.861 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.858+0000 7f080a247700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f080415dba0 0x7f080406c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:54.861 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.858+0000 7f080a247700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f08040b22e0 0x7f08040b2770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:54.861 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.858+0000 7f080a247700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0804161890 con 0x7f080406b240 2023-11-08T19:26:54.862 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.858+0000 7f080a247700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08040b2cb0 con 0x7f08040b22e0 2023-11-08T19:26:54.862 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.858+0000 7f080a247700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f080406b740 con 0x7f080415dba0 2023-11-08T19:26:54.862 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.858+0000 7f08037fe700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f080415dba0 0x7f080406c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:54.862 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.858+0000 7f0803fff700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f08040b22e0 0x7f08040b2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:54.863 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.858+0000 7f0803fff700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f08040b22e0 0x7f08040b2770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.143:35932/0 (socket says 172.21.15.143:35932) 2023-11-08T19:26:54.863 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.858+0000 7f08037fe700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f080415dba0 0x7f080406c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.143:40068/0 (socket says 172.21.15.143:40068) 2023-11-08T19:26:54.863 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.858+0000 7f0803fff700 1 -- 172.21.15.143:0/3156531167 learned_addr learned my addr 172.21.15.143:0/3156531167 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:54.864 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.858+0000 7f0803fff700 1 -- 172.21.15.143:0/3156531167 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f080415dba0 msgr2=0x7f080406c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:54.864 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.858+0000 7f0803fff700 1 --2- 172.21.15.143:0/3156531167 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f080415dba0 0x7f080406c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:54.864 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.858+0000 7f0803fff700 1 -- 172.21.15.143:0/3156531167 >> v1:172.21.15.174:6789/0 conn(0x7f080406b240 legacy=0x7f08040caff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:54.864 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.858+0000 7f0803fff700 1 -- 172.21.15.143:0/3156531167 --> v2:172.21.15.174:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f080406ba20 con 0x7f08040b22e0 2023-11-08T19:26:54.865 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.859+0000 7f0803fff700 1 --2- 172.21.15.143:0/3156531167 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f08040b22e0 0x7f08040b2770 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f07fc009e50 tx=0x7f07fc00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c990f1896f828d7e server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:54.865 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.859+0000 7f08027fc700 1 -- 172.21.15.143:0/3156531167 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07fc00e070 con 0x7f08040b22e0 2023-11-08T19:26:54.865 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.859+0000 7f08027fc700 1 -- 172.21.15.143:0/3156531167 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07fc00b910 con 0x7f08040b22e0 2023-11-08T19:26:54.865 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.859+0000 7f08027fc700 1 -- 172.21.15.143:0/3156531167 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07fc00bcc0 con 0x7f08040b22e0 2023-11-08T19:26:54.866 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.859+0000 7f080a247700 1 -- 172.21.15.143:0/3156531167 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f08040b22e0 msgr2=0x7f08040b2770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:54.866 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.859+0000 7f080a247700 1 --2- 172.21.15.143:0/3156531167 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f08040b22e0 0x7f08040b2770 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f07fc009e50 tx=0x7f07fc00b330 comp rx=0 tx=0).stop 2023-11-08T19:26:54.866 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.859+0000 7f080a247700 1 -- 172.21.15.143:0/3156531167 shutdown_connections 2023-11-08T19:26:54.867 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.859+0000 7f080a247700 1 --2- 172.21.15.143:0/3156531167 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f08040b22e0 0x7f08040b2770 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:54.867 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.859+0000 7f080a247700 1 --2- 172.21.15.143:0/3156531167 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f080415dba0 0x7f080406c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:54.867 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.859+0000 7f080a247700 1 -- 172.21.15.143:0/3156531167 >> 172.21.15.143:0/3156531167 conn(0x7f08040cd2e0 msgr2=0x7f08040cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:54.867 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.859+0000 7f080a247700 1 -- 172.21.15.143:0/3156531167 shutdown_connections 2023-11-08T19:26:54.867 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.859+0000 7f080a247700 1 -- 172.21.15.143:0/3156531167 wait complete. 2023-11-08T19:26:54.870 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.861+0000 7f080a247700 1 Processor -- start 2023-11-08T19:26:54.871 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.861+0000 7f080a247700 1 -- start start 2023-11-08T19:26:54.871 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.861+0000 7f080a247700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f080406b240 0x7f080415b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:54.871 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.861+0000 7f080a247700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f08040b22e0 0x7f08041ef060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:54.871 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.861+0000 7f080a247700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f080415dba0 0x7f080415c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:54.871 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.861+0000 7f080a247700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f080415f6f0 con 0x7f080415dba0 2023-11-08T19:26:54.872 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.861+0000 7f080a247700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08041f01a0 con 0x7f08040b22e0 2023-11-08T19:26:54.872 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.861+0000 7f080a247700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08041ef900 con 0x7f080406b240 2023-11-08T19:26:54.872 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.861+0000 7f0803fff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f080415dba0 0x7f080415c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:54.872 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.862+0000 7f0803fff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f080415dba0 0x7f080415c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52440/0 (socket says 172.21.15.143:52440) 2023-11-08T19:26:54.873 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.862+0000 7f0803fff700 1 -- 172.21.15.143:0/687899575 learned_addr learned my addr 172.21.15.143:0/687899575 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:54.873 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.862+0000 7f08037fe700 1 --2- 172.21.15.143:0/687899575 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f080406b240 0x7f080415b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:54.873 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.862+0000 7f0802ffd700 1 --2- 172.21.15.143:0/687899575 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f08040b22e0 0x7f08041ef060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:54.874 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.862+0000 7f0803fff700 1 -- 172.21.15.143:0/687899575 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f080406b240 msgr2=0x7f080415b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:54.874 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.862+0000 7f0803fff700 1 --2- 172.21.15.143:0/687899575 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f080406b240 0x7f080415b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:54.874 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.862+0000 7f0803fff700 1 -- 172.21.15.143:0/687899575 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f08040b22e0 msgr2=0x7f08041ef060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:54.874 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.862+0000 7f0803fff700 1 --2- 172.21.15.143:0/687899575 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f08040b22e0 0x7f08041ef060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:54.875 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.862+0000 7f0803fff700 1 -- 172.21.15.143:0/687899575 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07fc009b30 con 0x7f080415dba0 2023-11-08T19:26:54.875 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.862+0000 7f0802ffd700 1 --2- 172.21.15.143:0/687899575 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f08040b22e0 0x7f08041ef060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:26:54.875 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.862+0000 7f0803fff700 1 --2- 172.21.15.143:0/687899575 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f080415dba0 0x7f080415c960 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f0804067530 tx=0x7f07fc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:54.875 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.862+0000 7f07ebfff700 1 -- 172.21.15.143:0/687899575 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07fc0048b0 con 0x7f080415dba0 2023-11-08T19:26:54.875 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.862+0000 7f07ebfff700 1 -- 172.21.15.143:0/687899575 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07fc0099b0 con 0x7f080415dba0 2023-11-08T19:26:54.876 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.862+0000 7f080a247700 1 -- 172.21.15.143:0/687899575 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08041efbe0 con 0x7f080415dba0 2023-11-08T19:26:54.876 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.862+0000 7f07ebfff700 1 -- 172.21.15.143:0/687899575 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07fc01c680 con 0x7f080415dba0 2023-11-08T19:26:54.876 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.862+0000 7f080a247700 1 -- 172.21.15.143:0/687899575 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08040b3640 con 0x7f080415dba0 2023-11-08T19:26:54.876 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.863+0000 7f080a247700 1 -- 172.21.15.143:0/687899575 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f080404eee0 con 0x7f080415dba0 2023-11-08T19:26:54.876 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.864+0000 7f07ebfff700 1 -- 172.21.15.143:0/687899575 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f07fc01c7e0 con 0x7f080415dba0 2023-11-08T19:26:54.877 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.864+0000 7f07ebfff700 1 --2- 172.21.15.143:0/687899575 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f07f006af00 0x7f07f006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:54.877 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.864+0000 7f07ebfff700 1 -- 172.21.15.143:0/687899575 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f07fc08b090 con 0x7f080415dba0 2023-11-08T19:26:54.877 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.864+0000 7f08037fe700 1 --2- 172.21.15.143:0/687899575 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f07f006af00 0x7f07f006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:54.877 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.865+0000 7f08037fe700 1 --2- 172.21.15.143:0/687899575 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f07f006af00 0x7f07f006d3c0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f0804161710 tx=0x7f07f4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:54.877 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:54.868+0000 7f07ebfff700 1 -- 172.21.15.143:0/687899575 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f07fc056340 con 0x7f080415dba0 2023-11-08T19:26:55.057 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.055+0000 7f080a247700 1 -- 172.21.15.143:0/687899575 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"} v 0) v1 -- 0x7f08040af9a0 con 0x7f080415dba0 2023-11-08T19:26:55.057 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.056+0000 7f07ebfff700 1 -- 172.21.15.143:0/687899575 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"}]=0 v58) v1 ==== 103+0+11 (secure 0 0 0) 0x7f07fc059960 con 0x7f080415dba0 2023-11-08T19:26:55.058 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2023-11-08T19:26:55.060 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.058+0000 7f080a247700 1 -- 172.21.15.143:0/687899575 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f07f006af00 msgr2=0x7f07f006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:55.060 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.058+0000 7f080a247700 1 --2- 172.21.15.143:0/687899575 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f07f006af00 0x7f07f006d3c0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f0804161710 tx=0x7f07f4008040 comp rx=0 tx=0).stop 2023-11-08T19:26:55.060 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.058+0000 7f080a247700 1 -- 172.21.15.143:0/687899575 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f080415dba0 msgr2=0x7f080415c960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:55.061 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.058+0000 7f080a247700 1 --2- 172.21.15.143:0/687899575 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f080415dba0 0x7f080415c960 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f0804067530 tx=0x7f07fc004ad0 comp rx=0 tx=0).stop 2023-11-08T19:26:55.061 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.058+0000 7f080a247700 1 -- 172.21.15.143:0/687899575 shutdown_connections 2023-11-08T19:26:55.061 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.058+0000 7f080a247700 1 --2- 172.21.15.143:0/687899575 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f07f006af00 0x7f07f006d3c0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:55.061 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.058+0000 7f080a247700 1 --2- 172.21.15.143:0/687899575 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f080406b240 0x7f080415b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:55.061 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.058+0000 7f080a247700 1 --2- 172.21.15.143:0/687899575 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f080415dba0 0x7f080415c960 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:55.062 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.058+0000 7f080a247700 1 --2- 172.21.15.143:0/687899575 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f08040b22e0 0x7f08041ef060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:55.062 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.058+0000 7f080a247700 1 -- 172.21.15.143:0/687899575 >> 172.21.15.143:0/687899575 conn(0x7f08040cd2e0 msgr2=0x7f08040b1af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:55.062 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.058+0000 7f080a247700 1 -- 172.21.15.143:0/687899575 shutdown_connections 2023-11-08T19:26:55.062 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.058+0000 7f080a247700 1 -- 172.21.15.143:0/687899575 wait complete. 2023-11-08T19:26:55.072 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-08T19:26:55.192 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.190+0000 7fc975df9700 1 Processor -- start 2023-11-08T19:26:55.192 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.191+0000 7fc975df9700 1 -- start start 2023-11-08T19:26:55.195 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.191+0000 7fc975df9700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc9701618f0 0x7fc97006ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:55.195 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.191+0000 7fc975df9700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc97006b150 0x7fc9700cb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:55.195 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.191+0000 7fc975df9700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc970161df0 con 0x7fc97015dd00 2023-11-08T19:26:55.195 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.191+0000 7fc975df9700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc97006b600 con 0x7fc97006b150 2023-11-08T19:26:55.195 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.191+0000 7fc975df9700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9700cad50 con 0x7fc9701618f0 2023-11-08T19:26:55.196 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.192+0000 7fc96ffff700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc97006b150 0x7fc9700cb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:55.196 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.192+0000 7fc96ffff700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc97006b150 0x7fc9700cb1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52448/0 (socket says 172.21.15.143:52448) 2023-11-08T19:26:55.196 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.192+0000 7fc96ffff700 1 -- 172.21.15.143:0/1772759257 learned_addr learned my addr 172.21.15.143:0/1772759257 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:55.196 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.192+0000 7fc96effd700 1 --2- 172.21.15.143:0/1772759257 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc9701618f0 0x7fc97006ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:55.196 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.192+0000 7fc96ffff700 1 -- 172.21.15.143:0/1772759257 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc9701618f0 msgr2=0x7fc97006ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:55.197 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.192+0000 7fc96ffff700 1 --2- 172.21.15.143:0/1772759257 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc9701618f0 0x7fc97006ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:55.197 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.192+0000 7fc96ffff700 1 -- 172.21.15.143:0/1772759257 >> v1:172.21.15.174:6789/0 conn(0x7fc97015dd00 legacy=0x7fc9700cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:55.197 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.192+0000 7fc96ffff700 1 -- 172.21.15.143:0/1772759257 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9701620d0 con 0x7fc97006b150 2023-11-08T19:26:55.197 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.192+0000 7fc96ffff700 1 --2- 172.21.15.143:0/1772759257 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc97006b150 0x7fc9700cb1a0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fc968009e50 tx=0x7fc96800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b1055c98ea2cce server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:55.197 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.192+0000 7fc96e7fc700 1 -- 172.21.15.143:0/1772759257 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc96800e070 con 0x7fc97006b150 2023-11-08T19:26:55.198 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.192+0000 7fc96e7fc700 1 -- 172.21.15.143:0/1772759257 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc968003a20 con 0x7fc97006b150 2023-11-08T19:26:55.198 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.192+0000 7fc96e7fc700 1 -- 172.21.15.143:0/1772759257 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc968003dd0 con 0x7fc97006b150 2023-11-08T19:26:55.198 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.193+0000 7fc975df9700 1 -- 172.21.15.143:0/1772759257 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc97006b150 msgr2=0x7fc9700cb1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:55.198 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.193+0000 7fc975df9700 1 --2- 172.21.15.143:0/1772759257 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc97006b150 0x7fc9700cb1a0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fc968009e50 tx=0x7fc96800b330 comp rx=0 tx=0).stop 2023-11-08T19:26:55.199 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.193+0000 7fc975df9700 1 -- 172.21.15.143:0/1772759257 shutdown_connections 2023-11-08T19:26:55.199 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.193+0000 7fc975df9700 1 --2- 172.21.15.143:0/1772759257 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc97006b150 0x7fc9700cb1a0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:55.199 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.193+0000 7fc975df9700 1 --2- 172.21.15.143:0/1772759257 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc9701618f0 0x7fc97006ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:55.199 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.193+0000 7fc975df9700 1 -- 172.21.15.143:0/1772759257 >> 172.21.15.143:0/1772759257 conn(0x7fc970099ab0 msgr2=0x7fc9700d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:55.199 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.193+0000 7fc975df9700 1 -- 172.21.15.143:0/1772759257 shutdown_connections 2023-11-08T19:26:55.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.193+0000 7fc975df9700 1 -- 172.21.15.143:0/1772759257 wait complete. 2023-11-08T19:26:55.203 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.194+0000 7fc975df9700 1 Processor -- start 2023-11-08T19:26:55.203 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.194+0000 7fc975df9700 1 -- start start 2023-11-08T19:26:55.204 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.195+0000 7fc975df9700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc97006b150 0x7fc9700b33b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:55.204 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.195+0000 7fc975df9700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc97015dd00 0x7fc9700b5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:55.204 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.195+0000 7fc975df9700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc9701618f0 0x7fc9700b5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:55.204 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.195+0000 7fc975df9700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9700c4b40 con 0x7fc97015dd00 2023-11-08T19:26:55.204 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.195+0000 7fc975df9700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9700b5cc0 con 0x7fc9701618f0 2023-11-08T19:26:55.205 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.195+0000 7fc975df9700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9701532d0 con 0x7fc97006b150 2023-11-08T19:26:55.205 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.195+0000 7fc96effd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc97015dd00 0x7fc9700b5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:55.205 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.195+0000 7fc96effd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc97015dd00 0x7fc9700b5240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52452/0 (socket says 172.21.15.143:52452) 2023-11-08T19:26:55.205 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.195+0000 7fc96effd700 1 -- 172.21.15.143:0/2728325273 learned_addr learned my addr 172.21.15.143:0/2728325273 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:55.205 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.195+0000 7fc96f7fe700 1 --2- 172.21.15.143:0/2728325273 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc97006b150 0x7fc9700b33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:55.206 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.195+0000 7fc96ffff700 1 --2- 172.21.15.143:0/2728325273 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc9701618f0 0x7fc9700b5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:55.206 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.195+0000 7fc96effd700 1 -- 172.21.15.143:0/2728325273 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc97006b150 msgr2=0x7fc9700b33b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:55.206 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.195+0000 7fc96effd700 1 --2- 172.21.15.143:0/2728325273 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc97006b150 0x7fc9700b33b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:55.207 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.195+0000 7fc96effd700 1 -- 172.21.15.143:0/2728325273 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc9701618f0 msgr2=0x7fc9700b5780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:55.207 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.195+0000 7fc96effd700 1 --2- 172.21.15.143:0/2728325273 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc9701618f0 0x7fc9700b5780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:55.207 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.195+0000 7fc96effd700 1 -- 172.21.15.143:0/2728325273 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc968009b30 con 0x7fc97015dd00 2023-11-08T19:26:55.207 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.195+0000 7fc96effd700 1 --2- 172.21.15.143:0/2728325273 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc97015dd00 0x7fc9700b5240 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fc95800c7f0 tx=0x7fc95800cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:55.208 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.195+0000 7fc967fff700 1 -- 172.21.15.143:0/2728325273 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc958011800 con 0x7fc97015dd00 2023-11-08T19:26:55.208 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.195+0000 7fc967fff700 1 -- 172.21.15.143:0/2728325273 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc958011960 con 0x7fc97015dd00 2023-11-08T19:26:55.208 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.195+0000 7fc967fff700 1 -- 172.21.15.143:0/2728325273 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc958011cd0 con 0x7fc97015dd00 2023-11-08T19:26:55.209 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.196+0000 7fc975df9700 1 -- 172.21.15.143:0/2728325273 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9700b5ee0 con 0x7fc97015dd00 2023-11-08T19:26:55.209 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.196+0000 7fc975df9700 1 -- 172.21.15.143:0/2728325273 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9700c4760 con 0x7fc97015dd00 2023-11-08T19:26:55.209 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.197+0000 7fc975df9700 1 -- 172.21.15.143:0/2728325273 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9700aff60 con 0x7fc97015dd00 2023-11-08T19:26:55.210 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.197+0000 7fc967fff700 1 -- 172.21.15.143:0/2728325273 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc958010350 con 0x7fc97015dd00 2023-11-08T19:26:55.210 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.197+0000 7fc967fff700 1 --2- 172.21.15.143:0/2728325273 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc95006af00 0x7fc95006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:55.210 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.197+0000 7fc967fff700 1 -- 172.21.15.143:0/2728325273 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fc958088e80 con 0x7fc97015dd00 2023-11-08T19:26:55.211 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.198+0000 7fc96f7fe700 1 --2- 172.21.15.143:0/2728325273 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc95006af00 0x7fc95006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:55.211 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.198+0000 7fc96f7fe700 1 --2- 172.21.15.143:0/2728325273 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc95006af00 0x7fc95006d3c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fc9600050c0 tx=0x7fc960007f10 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:55.211 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.202+0000 7fc967fff700 1 -- 172.21.15.143:0/2728325273 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc958054110 con 0x7fc97015dd00 2023-11-08T19:26:55.395 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.394+0000 7fc975df9700 1 -- 172.21.15.143:0/2728325273 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fc97004eee0 con 0x7fc97015dd00 2023-11-08T19:26:55.396 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.394+0000 7fc967fff700 1 -- 172.21.15.143:0/2728325273 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v58) v1 ==== 84+0+137 (secure 0 0 0) 0x7fc95801f0a0 con 0x7fc97015dd00 2023-11-08T19:26:55.396 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:26:55.396 INFO:teuthology.orchestra.run.smithi143.stdout:[ 2023-11-08T19:26:55.396 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-08T19:26:55.397 INFO:teuthology.orchestra.run.smithi143.stdout: "poolnum": 1, 2023-11-08T19:26:55.397 INFO:teuthology.orchestra.run.smithi143.stdout: "poolname": ".mgr" 2023-11-08T19:26:55.397 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-08T19:26:55.397 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-08T19:26:55.397 INFO:teuthology.orchestra.run.smithi143.stdout: "poolnum": 6, 2023-11-08T19:26:55.398 INFO:teuthology.orchestra.run.smithi143.stdout: "poolname": "snap-test-pool" 2023-11-08T19:26:55.398 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-08T19:26:55.398 INFO:teuthology.orchestra.run.smithi143.stdout:] 2023-11-08T19:26:55.399 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.396+0000 7fc975df9700 1 -- 172.21.15.143:0/2728325273 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc95006af00 msgr2=0x7fc95006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:55.399 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.396+0000 7fc975df9700 1 --2- 172.21.15.143:0/2728325273 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc95006af00 0x7fc95006d3c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fc9600050c0 tx=0x7fc960007f10 comp rx=0 tx=0).stop 2023-11-08T19:26:55.399 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.396+0000 7fc975df9700 1 -- 172.21.15.143:0/2728325273 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc97015dd00 msgr2=0x7fc9700b5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:55.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.396+0000 7fc975df9700 1 --2- 172.21.15.143:0/2728325273 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc97015dd00 0x7fc9700b5240 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fc95800c7f0 tx=0x7fc95800cb00 comp rx=0 tx=0).stop 2023-11-08T19:26:55.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.396+0000 7fc975df9700 1 -- 172.21.15.143:0/2728325273 shutdown_connections 2023-11-08T19:26:55.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.396+0000 7fc975df9700 1 --2- 172.21.15.143:0/2728325273 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc95006af00 0x7fc95006d3c0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:55.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.396+0000 7fc975df9700 1 --2- 172.21.15.143:0/2728325273 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc97006b150 0x7fc9700b33b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:55.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.396+0000 7fc975df9700 1 --2- 172.21.15.143:0/2728325273 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc9701618f0 0x7fc9700b5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:55.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.396+0000 7fc975df9700 1 --2- 172.21.15.143:0/2728325273 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc97015dd00 0x7fc9700b5240 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:55.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.396+0000 7fc975df9700 1 -- 172.21.15.143:0/2728325273 >> 172.21.15.143:0/2728325273 conn(0x7fc970099ab0 msgr2=0x7fc97006f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:55.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.396+0000 7fc975df9700 1 -- 172.21.15.143:0/2728325273 shutdown_connections 2023-11-08T19:26:55.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.396+0000 7fc975df9700 1 -- 172.21.15.143:0/2728325273 wait complete. 2023-11-08T19:26:55.410 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-08T19:26:55.410 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-08T19:26:55.535 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.534+0000 7fcc3216c700 1 Processor -- start 2023-11-08T19:26:55.536 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.534+0000 7fcc3216c700 1 -- start start 2023-11-08T19:26:55.549 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.534+0000 7fcc3216c700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fcc2c15dba0 0x7fcc2c06c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:55.549 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.534+0000 7fcc3216c700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcc2c0b22e0 0x7fcc2c0b2750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:55.550 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.534+0000 7fcc3216c700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc2c161890 con 0x7fcc2c06b240 2023-11-08T19:26:55.550 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.534+0000 7fcc3216c700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc2c0b2c90 con 0x7fcc2c15dba0 2023-11-08T19:26:55.551 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.534+0000 7fcc3216c700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc2c06b740 con 0x7fcc2c0b22e0 2023-11-08T19:26:55.551 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.535+0000 7fcc2b7fe700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fcc2c15dba0 0x7fcc2c06c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:55.551 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.535+0000 7fcc2b7fe700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fcc2c15dba0 0x7fcc2c06c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52458/0 (socket says 172.21.15.143:52458) 2023-11-08T19:26:55.552 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.535+0000 7fcc2b7fe700 1 -- 172.21.15.143:0/2957210578 learned_addr learned my addr 172.21.15.143:0/2957210578 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:55.552 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.535+0000 7fcc2bfff700 1 --2- 172.21.15.143:0/2957210578 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcc2c0b22e0 0x7fcc2c0b2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:55.553 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.535+0000 7fcc2b7fe700 1 -- 172.21.15.143:0/2957210578 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcc2c0b22e0 msgr2=0x7fcc2c0b2750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:55.553 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.535+0000 7fcc2b7fe700 1 --2- 172.21.15.143:0/2957210578 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcc2c0b22e0 0x7fcc2c0b2750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:55.553 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.535+0000 7fcc2b7fe700 1 -- 172.21.15.143:0/2957210578 >> v1:172.21.15.174:6789/0 conn(0x7fcc2c06b240 legacy=0x7fcc2c0caff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:55.554 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.535+0000 7fcc2b7fe700 1 -- 172.21.15.143:0/2957210578 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc2c06ba20 con 0x7fcc2c15dba0 2023-11-08T19:26:55.554 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.535+0000 7fcc2bfff700 1 --2- 172.21.15.143:0/2957210578 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcc2c0b22e0 0x7fcc2c0b2750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:26:55.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.535+0000 7fcc2b7fe700 1 --2- 172.21.15.143:0/2957210578 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcc2c15dba0 0x7fcc2c06c610 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c0098f0 tx=0x7fcc1c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=134605846710bb39 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:55.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.535+0000 7fcc2a7fc700 1 -- 172.21.15.143:0/2957210578 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc1c00b8a0 con 0x7fcc2c15dba0 2023-11-08T19:26:55.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.535+0000 7fcc2a7fc700 1 -- 172.21.15.143:0/2957210578 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc1c00ba00 con 0x7fcc2c15dba0 2023-11-08T19:26:55.556 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.535+0000 7fcc2a7fc700 1 -- 172.21.15.143:0/2957210578 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc1c00bd90 con 0x7fcc2c15dba0 2023-11-08T19:26:55.556 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.536+0000 7fcc3216c700 1 -- 172.21.15.143:0/2957210578 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcc2c15dba0 msgr2=0x7fcc2c06c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:55.557 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.536+0000 7fcc3216c700 1 --2- 172.21.15.143:0/2957210578 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcc2c15dba0 0x7fcc2c06c610 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c0098f0 tx=0x7fcc1c009c00 comp rx=0 tx=0).stop 2023-11-08T19:26:55.557 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.536+0000 7fcc3216c700 1 -- 172.21.15.143:0/2957210578 shutdown_connections 2023-11-08T19:26:55.558 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.536+0000 7fcc3216c700 1 --2- 172.21.15.143:0/2957210578 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcc2c0b22e0 0x7fcc2c0b2750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:55.558 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.536+0000 7fcc3216c700 1 --2- 172.21.15.143:0/2957210578 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcc2c15dba0 0x7fcc2c06c610 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:55.558 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.536+0000 7fcc3216c700 1 -- 172.21.15.143:0/2957210578 >> 172.21.15.143:0/2957210578 conn(0x7fcc2c0cd2e0 msgr2=0x7fcc2c0cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:55.559 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.536+0000 7fcc3216c700 1 -- 172.21.15.143:0/2957210578 shutdown_connections 2023-11-08T19:26:55.559 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.536+0000 7fcc3216c700 1 -- 172.21.15.143:0/2957210578 wait complete. 2023-11-08T19:26:55.560 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.537+0000 7fcc3216c700 1 Processor -- start 2023-11-08T19:26:55.560 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.538+0000 7fcc3216c700 1 -- start start 2023-11-08T19:26:55.560 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.538+0000 7fcc3216c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcc2c06b240 0x7fcc2c15c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:55.561 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.538+0000 7fcc3216c700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcc2c0b22e0 0x7fcc2c0b5800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:55.561 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.538+0000 7fcc3216c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcc2c15dba0 0x7fcc2c1e7a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:55.561 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.538+0000 7fcc3216c700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc2c0c5190 con 0x7fcc2c15dba0 2023-11-08T19:26:55.562 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.538+0000 7fcc3216c700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc2c1f46c0 con 0x7fcc2c06b240 2023-11-08T19:26:55.562 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.538+0000 7fcc3216c700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc2c0c4510 con 0x7fcc2c0b22e0 2023-11-08T19:26:55.563 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.539+0000 7fcc2bfff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcc2c15dba0 0x7fcc2c1e7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:55.563 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.539+0000 7fcc2bfff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcc2c15dba0 0x7fcc2c1e7a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52468/0 (socket says 172.21.15.143:52468) 2023-11-08T19:26:55.563 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.539+0000 7fcc2bfff700 1 -- 172.21.15.143:0/1049695276 learned_addr learned my addr 172.21.15.143:0/1049695276 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:55.564 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.539+0000 7fcc2affd700 1 --2- 172.21.15.143:0/1049695276 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcc2c0b22e0 0x7fcc2c0b5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:55.564 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.539+0000 7fcc2b7fe700 1 --2- 172.21.15.143:0/1049695276 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcc2c06b240 0x7fcc2c15c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:55.564 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.539+0000 7fcc2bfff700 1 -- 172.21.15.143:0/1049695276 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcc2c0b22e0 msgr2=0x7fcc2c0b5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:55.565 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.539+0000 7fcc2bfff700 1 --2- 172.21.15.143:0/1049695276 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcc2c0b22e0 0x7fcc2c0b5800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:55.565 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.539+0000 7fcc2bfff700 1 -- 172.21.15.143:0/1049695276 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcc2c06b240 msgr2=0x7fcc2c15c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:55.566 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.539+0000 7fcc2bfff700 1 --2- 172.21.15.143:0/1049695276 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcc2c06b240 0x7fcc2c15c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:55.566 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.539+0000 7fcc2bfff700 1 -- 172.21.15.143:0/1049695276 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc1c009550 con 0x7fcc2c15dba0 2023-11-08T19:26:55.566 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.539+0000 7fcc2b7fe700 1 --2- 172.21.15.143:0/1049695276 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcc2c06b240 0x7fcc2c15c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:26:55.567 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.539+0000 7fcc2bfff700 1 --2- 172.21.15.143:0/1049695276 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcc2c15dba0 0x7fcc2c1e7a80 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fcc24007ef0 tx=0x7fcc2400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:55.567 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.539+0000 7fcc13fff700 1 -- 172.21.15.143:0/1049695276 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc2400de30 con 0x7fcc2c15dba0 2023-11-08T19:26:55.568 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.539+0000 7fcc13fff700 1 -- 172.21.15.143:0/1049695276 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc2400f040 con 0x7fcc2c15dba0 2023-11-08T19:26:55.568 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.539+0000 7fcc3216c700 1 -- 172.21.15.143:0/1049695276 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc2c0c47f0 con 0x7fcc2c15dba0 2023-11-08T19:26:55.568 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.539+0000 7fcc13fff700 1 -- 172.21.15.143:0/1049695276 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc24003c60 con 0x7fcc2c15dba0 2023-11-08T19:26:55.569 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.539+0000 7fcc3216c700 1 -- 172.21.15.143:0/1049695276 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc2c0c4db0 con 0x7fcc2c15dba0 2023-11-08T19:26:55.569 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.540+0000 7fcc3216c700 1 -- 172.21.15.143:0/1049695276 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc2c0c52d0 con 0x7fcc2c15dba0 2023-11-08T19:26:55.569 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.541+0000 7fcc13fff700 1 -- 172.21.15.143:0/1049695276 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fcc24004030 con 0x7fcc2c15dba0 2023-11-08T19:26:55.570 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.541+0000 7fcc13fff700 1 --2- 172.21.15.143:0/1049695276 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fcc1806af00 0x7fcc1806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:55.571 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.541+0000 7fcc13fff700 1 -- 172.21.15.143:0/1049695276 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fcc24089500 con 0x7fcc2c15dba0 2023-11-08T19:26:55.571 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.541+0000 7fcc2b7fe700 1 --2- 172.21.15.143:0/1049695276 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fcc1806af00 0x7fcc1806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:55.571 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.542+0000 7fcc2b7fe700 1 --2- 172.21.15.143:0/1049695276 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fcc1806af00 0x7fcc1806d3c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c014040 tx=0x7fcc1c008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:55.572 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.545+0000 7fcc13fff700 1 -- 172.21.15.143:0/1049695276 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcc2c0c52d0 con 0x7fcc2c15dba0 2023-11-08T19:26:55.736 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:55.735+0000 7fcc3216c700 1 -- 172.21.15.143:0/1049695276 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fcc2c0c52d0 con 0x7fcc2c15dba0 2023-11-08T19:26:56.122 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:56.120+0000 7fcc13fff700 1 -- 172.21.15.143:0/1049695276 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v59) v1 ==== 138+0+0 (secure 0 0 0) 0x7fcc2c0c52d0 con 0x7fcc2c15dba0 2023-11-08T19:26:56.125 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:56.123+0000 7fcc3216c700 1 -- 172.21.15.143:0/1049695276 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fcc1806af00 msgr2=0x7fcc1806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:56.125 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:56.123+0000 7fcc3216c700 1 --2- 172.21.15.143:0/1049695276 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fcc1806af00 0x7fcc1806d3c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c014040 tx=0x7fcc1c008030 comp rx=0 tx=0).stop 2023-11-08T19:26:56.125 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:56.123+0000 7fcc3216c700 1 -- 172.21.15.143:0/1049695276 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcc2c15dba0 msgr2=0x7fcc2c1e7a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:56.125 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:56.123+0000 7fcc3216c700 1 --2- 172.21.15.143:0/1049695276 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcc2c15dba0 0x7fcc2c1e7a80 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fcc24007ef0 tx=0x7fcc2400d3b0 comp rx=0 tx=0).stop 2023-11-08T19:26:56.125 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:56.123+0000 7fcc3216c700 1 -- 172.21.15.143:0/1049695276 shutdown_connections 2023-11-08T19:26:56.126 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:56.123+0000 7fcc3216c700 1 --2- 172.21.15.143:0/1049695276 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fcc1806af00 0x7fcc1806d3c0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:56.126 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:56.123+0000 7fcc3216c700 1 --2- 172.21.15.143:0/1049695276 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcc2c15dba0 0x7fcc2c1e7a80 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:56.126 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:56.123+0000 7fcc3216c700 1 --2- 172.21.15.143:0/1049695276 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcc2c06b240 0x7fcc2c15c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:56.126 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:56.123+0000 7fcc3216c700 1 --2- 172.21.15.143:0/1049695276 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcc2c0b22e0 0x7fcc2c0b5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:56.127 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:56.123+0000 7fcc3216c700 1 -- 172.21.15.143:0/1049695276 >> 172.21.15.143:0/1049695276 conn(0x7fcc2c0cd2e0 msgr2=0x7fcc2c0cc9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:56.127 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:56.123+0000 7fcc3216c700 1 -- 172.21.15.143:0/1049695276 shutdown_connections 2023-11-08T19:26:56.127 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:56.123+0000 7fcc3216c700 1 -- 172.21.15.143:0/1049695276 wait complete. 2023-11-08T19:26:56.127 INFO:teuthology.orchestra.run.smithi143.stderr:pool 'cephfs_metadata' created 2023-11-08T19:26:56.140 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-08T19:26:56.268 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:56.266+0000 7f4bf0c27700 1 Processor -- start 2023-11-08T19:26:56.268 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:56.266+0000 7f4bf0c27700 1 -- start start 2023-11-08T19:26:56.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:56.267+0000 7f4bf0c27700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f4bec15dba0 0x7f4bec06c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:56.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:56.267+0000 7f4bf0c27700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f4bec06b240 0x7f4bec0caff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:56.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:56.267+0000 7f4bf0c27700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4bec0cb530 0x7f4bec0b22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:56.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:56.267+0000 7f4bf0c27700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bec161890 con 0x7f4bec06b240 2023-11-08T19:26:56.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:56.267+0000 7f4bf0c27700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bec06b740 con 0x7f4bec15dba0 2023-11-08T19:26:56.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:56.267+0000 7f4bf0c27700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bec06ba20 con 0x7f4bec0cb530 2023-11-08T19:26:56.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:56.267+0000 7f4be9d9b700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f4bec06b240 0x7f4bec0caff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:56.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:56.267+0000 7f4be9d9b700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f4bec06b240 0x7f4bec0caff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52474/0 (socket says 172.21.15.143:52474) 2023-11-08T19:26:56.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:56.267+0000 7f4be9d9b700 1 -- 172.21.15.143:0/975209438 learned_addr learned my addr 172.21.15.143:0/975209438 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:56.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:56.267+0000 7f4bead9d700 1 --2- 172.21.15.143:0/975209438 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4bec0cb530 0x7f4bec0b22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:56.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:56.267+0000 7f4bea59c700 1 --2- 172.21.15.143:0/975209438 >> v2:172.21.15.174:3300/0 conn(0x7f4bec15dba0 0x7f4bec06c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:56.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:56.268+0000 7f4be9d9b700 1 -- 172.21.15.143:0/975209438 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4bec0cb530 msgr2=0x7f4bec0b22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:56.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:56.268+0000 7f4be9d9b700 1 --2- 172.21.15.143:0/975209438 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4bec0cb530 0x7f4bec0b22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:56.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:56.268+0000 7f4be9d9b700 1 -- 172.21.15.143:0/975209438 >> v2:172.21.15.174:3300/0 conn(0x7f4bec15dba0 msgr2=0x7f4bec06c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:56.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:56.268+0000 7f4be9d9b700 1 --2- 172.21.15.143:0/975209438 >> v2:172.21.15.174:3300/0 conn(0x7f4bec15dba0 0x7f4bec06c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:56.274 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:56.268+0000 7f4be9d9b700 1 -- 172.21.15.143:0/975209438 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bec06bd00 con 0x7f4bec06b240 2023-11-08T19:26:56.274 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:56.268+0000 7f4bea59c700 1 --2- 172.21.15.143:0/975209438 >> v2:172.21.15.174:3300/0 conn(0x7f4bec15dba0 0x7f4bec06c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:26:56.274 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:56.268+0000 7f4be9d9b700 1 --2- 172.21.15.143:0/975209438 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4bec06b240 0x7f4bec0caff0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f4bec067460 tx=0x7f4bdc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b20eaed6fbc311a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:56.274 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:56.268+0000 7f4be959a700 1 -- 172.21.15.143:0/975209438 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bdc011830 con 0x7f4bec06b240 2023-11-08T19:26:56.274 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:56.268+0000 7f4be959a700 1 -- 172.21.15.143:0/975209438 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bdc011990 con 0x7f4bec06b240 2023-11-08T19:26:56.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:56.268+0000 7f4be959a700 1 -- 172.21.15.143:0/975209438 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bdc011d20 con 0x7f4bec06b240 2023-11-08T19:26:56.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:56.268+0000 7f4bf0c27700 1 -- 172.21.15.143:0/975209438 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4bec06b240 msgr2=0x7f4bec0caff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:56.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:56.269+0000 7f4bf0c27700 1 --2- 172.21.15.143:0/975209438 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4bec06b240 0x7f4bec0caff0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f4bec067460 tx=0x7f4bdc009bd0 comp rx=0 tx=0).stop 2023-11-08T19:26:56.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:56.269+0000 7f4bf0c27700 1 -- 172.21.15.143:0/975209438 shutdown_connections 2023-11-08T19:26:56.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:56.269+0000 7f4bf0c27700 1 --2- 172.21.15.143:0/975209438 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4bec0cb530 0x7f4bec0b22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:56.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:56.269+0000 7f4bf0c27700 1 --2- 172.21.15.143:0/975209438 >> v2:172.21.15.174:3300/0 conn(0x7f4bec15dba0 0x7f4bec06c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:56.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:56.269+0000 7f4bf0c27700 1 --2- 172.21.15.143:0/975209438 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4bec06b240 0x7f4bec0caff0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:56.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:56.269+0000 7f4bf0c27700 1 -- 172.21.15.143:0/975209438 >> 172.21.15.143:0/975209438 conn(0x7f4bec0cd2e0 msgr2=0x7f4bec0cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:56.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:56.269+0000 7f4bf0c27700 1 -- 172.21.15.143:0/975209438 shutdown_connections 2023-11-08T19:26:56.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:56.269+0000 7f4bf0c27700 1 -- 172.21.15.143:0/975209438 wait complete. 2023-11-08T19:26:56.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:56.270+0000 7f4bf0c27700 1 Processor -- start 2023-11-08T19:26:56.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:56.270+0000 7f4bf0c27700 1 -- start start 2023-11-08T19:26:56.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:56.271+0000 7f4bf0c27700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4bec06b240 0x7f4bec15ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:56.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:56.271+0000 7f4bf0c27700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4bec0cb530 0x7f4bec0ac8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:56.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:56.271+0000 7f4bf0c27700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4bec15dba0 0x7f4bec1e6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:56.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:56.271+0000 7f4bf0c27700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bec15b250 con 0x7f4bec0cb530 2023-11-08T19:26:56.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:56.271+0000 7f4bf0c27700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bec0c79d0 con 0x7f4bec06b240 2023-11-08T19:26:56.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:56.271+0000 7f4bf0c27700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bec0c7cb0 con 0x7f4bec15dba0 2023-11-08T19:26:56.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:56.271+0000 7f4be9d9b700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4bec0cb530 0x7f4bec0ac8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:56.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:56.271+0000 7f4be9d9b700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4bec0cb530 0x7f4bec0ac8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52480/0 (socket says 172.21.15.143:52480) 2023-11-08T19:26:56.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:56.271+0000 7f4be9d9b700 1 -- 172.21.15.143:0/635588033 learned_addr learned my addr 172.21.15.143:0/635588033 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:56.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:56.271+0000 7f4bead9d700 1 --2- 172.21.15.143:0/635588033 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4bec15dba0 0x7f4bec1e6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:56.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:56.271+0000 7f4bea59c700 1 --2- 172.21.15.143:0/635588033 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4bec06b240 0x7f4bec15ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:56.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:56.271+0000 7f4be9d9b700 1 -- 172.21.15.143:0/635588033 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4bec15dba0 msgr2=0x7f4bec1e6cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:56.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:56.271+0000 7f4be9d9b700 1 --2- 172.21.15.143:0/635588033 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4bec15dba0 0x7f4bec1e6cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:56.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:56.271+0000 7f4be9d9b700 1 -- 172.21.15.143:0/635588033 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4bec06b240 msgr2=0x7f4bec15ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:56.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:56.271+0000 7f4be9d9b700 1 --2- 172.21.15.143:0/635588033 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4bec06b240 0x7f4bec15ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:56.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:56.271+0000 7f4be9d9b700 1 -- 172.21.15.143:0/635588033 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bdc009550 con 0x7f4bec0cb530 2023-11-08T19:26:56.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:56.271+0000 7f4bea59c700 1 --2- 172.21.15.143:0/635588033 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4bec06b240 0x7f4bec15ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:26:56.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:56.271+0000 7f4be9d9b700 1 --2- 172.21.15.143:0/635588033 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4bec0cb530 0x7f4bec0ac8f0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f4bdc000c00 tx=0x7f4bdc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:56.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:56.271+0000 7f4bdaffd700 1 -- 172.21.15.143:0/635588033 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bdc004300 con 0x7f4bec0cb530 2023-11-08T19:26:56.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:56.271+0000 7f4bdaffd700 1 -- 172.21.15.143:0/635588033 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bdc01f070 con 0x7f4bec0cb530 2023-11-08T19:26:56.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:56.271+0000 7f4bf0c27700 1 -- 172.21.15.143:0/635588033 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bec0c7f90 con 0x7f4bec0cb530 2023-11-08T19:26:56.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:56.271+0000 7f4bdaffd700 1 -- 172.21.15.143:0/635588033 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bdc0036a0 con 0x7f4bec0cb530 2023-11-08T19:26:56.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:56.272+0000 7f4bf0c27700 1 -- 172.21.15.143:0/635588033 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bec0ad3f0 con 0x7f4bec0cb530 2023-11-08T19:26:56.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:56.272+0000 7f4bf0c27700 1 -- 172.21.15.143:0/635588033 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bec04eee0 con 0x7f4bec0cb530 2023-11-08T19:26:56.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:56.273+0000 7f4bdaffd700 1 -- 172.21.15.143:0/635588033 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4bdc021030 con 0x7f4bec0cb530 2023-11-08T19:26:56.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:56.274+0000 7f4bdaffd700 1 --2- 172.21.15.143:0/635588033 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f4bcc06af00 0x7f4bcc06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:56.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:56.274+0000 7f4bdaffd700 1 -- 172.21.15.143:0/635588033 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5580+0+0 (secure 0 0 0) 0x7f4bdc08ab10 con 0x7f4bec0cb530 2023-11-08T19:26:56.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:56.274+0000 7f4bea59c700 1 --2- 172.21.15.143:0/635588033 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f4bcc06af00 0x7f4bcc06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:56.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:56.274+0000 7f4bea59c700 1 --2- 172.21.15.143:0/635588033 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f4bcc06af00 0x7f4bcc06d3c0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f4bec0b2e70 tx=0x7f4be0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:56.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:56.278+0000 7f4bdaffd700 1 -- 172.21.15.143:0/635588033 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4bdc055db0 con 0x7f4bec0cb530 2023-11-08T19:26:56.468 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:56.467+0000 7f4bf0c27700 1 -- 172.21.15.143:0/635588033 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f4bec0cad00 con 0x7f4bec0cb530 2023-11-08T19:26:56.889 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T19:26:56.892 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T19:26:57.128 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:57.126+0000 7f4bdaffd700 1 -- 172.21.15.143:0/635588033 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v60) v1 ==== 170+0+0 (secure 0 0 0) 0x7f4bdc0593d0 con 0x7f4bec0cb530 2023-11-08T19:26:57.131 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:57.129+0000 7f4bf0c27700 1 -- 172.21.15.143:0/635588033 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f4bcc06af00 msgr2=0x7f4bcc06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:57.131 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:57.129+0000 7f4bf0c27700 1 --2- 172.21.15.143:0/635588033 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f4bcc06af00 0x7f4bcc06d3c0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f4bec0b2e70 tx=0x7f4be0008040 comp rx=0 tx=0).stop 2023-11-08T19:26:57.131 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:57.129+0000 7f4bf0c27700 1 -- 172.21.15.143:0/635588033 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4bec0cb530 msgr2=0x7f4bec0ac8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:57.131 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:57.129+0000 7f4bf0c27700 1 --2- 172.21.15.143:0/635588033 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4bec0cb530 0x7f4bec0ac8f0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f4bdc000c00 tx=0x7f4bdc004750 comp rx=0 tx=0).stop 2023-11-08T19:26:57.132 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:57.129+0000 7f4bf0c27700 1 -- 172.21.15.143:0/635588033 shutdown_connections 2023-11-08T19:26:57.132 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:57.129+0000 7f4bf0c27700 1 --2- 172.21.15.143:0/635588033 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f4bcc06af00 0x7f4bcc06d3c0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:57.132 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:57.129+0000 7f4bf0c27700 1 --2- 172.21.15.143:0/635588033 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4bec15dba0 0x7f4bec1e6cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:57.132 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:57.129+0000 7f4bf0c27700 1 --2- 172.21.15.143:0/635588033 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4bec06b240 0x7f4bec15ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:57.133 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:57.129+0000 7f4bf0c27700 1 --2- 172.21.15.143:0/635588033 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4bec0cb530 0x7f4bec0ac8f0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:57.133 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:57.129+0000 7f4bf0c27700 1 -- 172.21.15.143:0/635588033 >> 172.21.15.143:0/635588033 conn(0x7f4bec0cd2e0 msgr2=0x7f4bec15cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:57.133 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:57.129+0000 7f4bf0c27700 1 -- 172.21.15.143:0/635588033 shutdown_connections 2023-11-08T19:26:57.133 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:57.129+0000 7f4bf0c27700 1 -- 172.21.15.143:0/635588033 wait complete. 2023-11-08T19:26:57.134 INFO:teuthology.orchestra.run.smithi143.stderr:pool 'cephfs_data' created 2023-11-08T19:26:57.153 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-08T19:26:57.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:57.277+0000 7fd9e2bbf700 1 Processor -- start 2023-11-08T19:26:57.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:57.277+0000 7fd9e2bbf700 1 -- start start 2023-11-08T19:26:57.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:57.278+0000 7fd9e2bbf700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd9dc15bb20 0x7fd9dc0b2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:57.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:57.278+0000 7fd9e2bbf700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9dc0d0420 con 0x7fd9dc0d1d80 2023-11-08T19:26:57.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:57.278+0000 7fd9e2bbf700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9dc0cf810 con 0x7fd9dc0b1730 2023-11-08T19:26:57.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:57.278+0000 7fd9e2bbf700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9dc0cf600 con 0x7fd9dc15bb20 2023-11-08T19:26:57.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:57.278+0000 7fd9e095b700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd9dc15bb20 0x7fd9dc0b2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:57.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:57.278+0000 7fd9e115c700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fd9dc0d1d80 0x7fd9dc0cc4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:48644/0 (socket says 172.21.15.143:48644) 2023-11-08T19:26:57.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:57.278+0000 7fd9e115c700 1 -- 172.21.15.143:0/957658921 learned_addr learned my addr 172.21.15.143:0/957658921 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:26:57.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:57.278+0000 7fd9e095b700 1 -- 172.21.15.143:0/957658921 >> v1:172.21.15.174:6789/0 conn(0x7fd9dc0b1730 legacy=0x7fd9dc0d1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:57.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:57.278+0000 7fd9e095b700 1 -- 172.21.15.143:0/957658921 >> v1:172.21.15.143:6789/0 conn(0x7fd9dc0d1d80 legacy=0x7fd9dc0cc4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:57.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:57.278+0000 7fd9e095b700 1 -- 172.21.15.143:0/957658921 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9dc0cfaf0 con 0x7fd9dc15bb20 2023-11-08T19:26:57.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:57.278+0000 7fd9e095b700 1 --2- 172.21.15.143:0/957658921 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd9dc15bb20 0x7fd9dc0b2670 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fd9d0009840 tx=0x7fd9d0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa0c83d98f1924fd server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:57.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:57.278+0000 7fd9db7fe700 1 -- 172.21.15.143:0/957658921 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d0004230 con 0x7fd9dc15bb20 2023-11-08T19:26:57.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:57.279+0000 7fd9db7fe700 1 -- 172.21.15.143:0/957658921 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9d0004390 con 0x7fd9dc15bb20 2023-11-08T19:26:57.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:57.279+0000 7fd9db7fe700 1 -- 172.21.15.143:0/957658921 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d00036a0 con 0x7fd9dc15bb20 2023-11-08T19:26:57.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:57.279+0000 7fd9e2bbf700 1 -- 172.21.15.143:0/957658921 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd9dc15bb20 msgr2=0x7fd9dc0b2670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:57.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:57.279+0000 7fd9e2bbf700 1 --2- 172.21.15.143:0/957658921 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd9dc15bb20 0x7fd9dc0b2670 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fd9d0009840 tx=0x7fd9d0009b50 comp rx=0 tx=0).stop 2023-11-08T19:26:57.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:57.279+0000 7fd9e2bbf700 1 -- 172.21.15.143:0/957658921 shutdown_connections 2023-11-08T19:26:57.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:57.279+0000 7fd9e2bbf700 1 --2- 172.21.15.143:0/957658921 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd9dc15bb20 0x7fd9dc0b2670 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:57.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:57.279+0000 7fd9e2bbf700 1 -- 172.21.15.143:0/957658921 >> 172.21.15.143:0/957658921 conn(0x7fd9dc08b000 msgr2=0x7fd9dc09ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:57.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:57.279+0000 7fd9e2bbf700 1 -- 172.21.15.143:0/957658921 shutdown_connections 2023-11-08T19:26:57.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:57.279+0000 7fd9e2bbf700 1 -- 172.21.15.143:0/957658921 wait complete. 2023-11-08T19:26:57.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:57.280+0000 7fd9e2bbf700 1 Processor -- start 2023-11-08T19:26:57.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:57.280+0000 7fd9e2bbf700 1 -- start start 2023-11-08T19:26:57.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:57.281+0000 7fd9e2bbf700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd9dc0b1730 0x7fd9dc153c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:57.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:57.281+0000 7fd9e2bbf700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd9dc0d1d80 0x7fd9dc06d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:57.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:57.281+0000 7fd9e2bbf700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd9dc15bb20 0x7fd9dc0cac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:57.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:57.281+0000 7fd9e2bbf700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9dc06bd20 con 0x7fd9dc15bb20 2023-11-08T19:26:57.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:57.281+0000 7fd9e2bbf700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9dc154660 con 0x7fd9dc0b1730 2023-11-08T19:26:57.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:57.281+0000 7fd9e2bbf700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9dc154170 con 0x7fd9dc0d1d80 2023-11-08T19:26:57.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:57.281+0000 7fd9e115c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd9dc15bb20 0x7fd9dc0cac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:57.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:57.281+0000 7fd9e115c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd9dc15bb20 0x7fd9dc0cac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52496/0 (socket says 172.21.15.143:52496) 2023-11-08T19:26:57.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:57.281+0000 7fd9e115c700 1 -- 172.21.15.143:0/2193699526 learned_addr learned my addr 172.21.15.143:0/2193699526 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:57.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:57.281+0000 7fd9e095b700 1 --2- 172.21.15.143:0/2193699526 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd9dc0b1730 0x7fd9dc153c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:57.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:57.281+0000 7fd9dbfff700 1 --2- 172.21.15.143:0/2193699526 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd9dc0d1d80 0x7fd9dc06d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:57.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:57.282+0000 7fd9e115c700 1 -- 172.21.15.143:0/2193699526 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd9dc0d1d80 msgr2=0x7fd9dc06d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:57.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:57.282+0000 7fd9e115c700 1 --2- 172.21.15.143:0/2193699526 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd9dc0d1d80 0x7fd9dc06d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:57.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:57.282+0000 7fd9e115c700 1 -- 172.21.15.143:0/2193699526 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd9dc0b1730 msgr2=0x7fd9dc153c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:57.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:57.282+0000 7fd9e115c700 1 --2- 172.21.15.143:0/2193699526 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd9dc0b1730 0x7fd9dc153c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:57.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:57.282+0000 7fd9e115c700 1 -- 172.21.15.143:0/2193699526 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9d00094a0 con 0x7fd9dc15bb20 2023-11-08T19:26:57.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:57.282+0000 7fd9e095b700 1 --2- 172.21.15.143:0/2193699526 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd9dc0b1730 0x7fd9dc153c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:26:57.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:57.282+0000 7fd9e115c700 1 --2- 172.21.15.143:0/2193699526 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd9dc15bb20 0x7fd9dc0cac40 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fd9d4002030 tx=0x7fd9d400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:57.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:57.282+0000 7fd9d97fa700 1 -- 172.21.15.143:0/2193699526 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d400ad00 con 0x7fd9dc15bb20 2023-11-08T19:26:57.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:57.282+0000 7fd9d97fa700 1 -- 172.21.15.143:0/2193699526 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9d400ae60 con 0x7fd9dc15bb20 2023-11-08T19:26:57.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:57.282+0000 7fd9e2bbf700 1 -- 172.21.15.143:0/2193699526 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9dc06bf70 con 0x7fd9dc15bb20 2023-11-08T19:26:57.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:57.282+0000 7fd9d97fa700 1 -- 172.21.15.143:0/2193699526 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d4007310 con 0x7fd9dc15bb20 2023-11-08T19:26:57.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:57.282+0000 7fd9e2bbf700 1 -- 172.21.15.143:0/2193699526 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9dc06c530 con 0x7fd9dc15bb20 2023-11-08T19:26:57.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:57.283+0000 7fd9e2bbf700 1 -- 172.21.15.143:0/2193699526 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9dc04eee0 con 0x7fd9dc15bb20 2023-11-08T19:26:57.293 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:57.283+0000 7fd9d97fa700 1 -- 172.21.15.143:0/2193699526 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd9d4007470 con 0x7fd9dc15bb20 2023-11-08T19:26:57.293 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:57.284+0000 7fd9d97fa700 1 --2- 172.21.15.143:0/2193699526 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd9c006af00 0x7fd9c006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:57.293 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:57.284+0000 7fd9d97fa700 1 -- 172.21.15.143:0/2193699526 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5962+0+0 (secure 0 0 0) 0x7fd9d40895d0 con 0x7fd9dc15bb20 2023-11-08T19:26:57.293 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:57.284+0000 7fd9e095b700 1 --2- 172.21.15.143:0/2193699526 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd9c006af00 0x7fd9c006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:57.293 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:57.285+0000 7fd9e095b700 1 --2- 172.21.15.143:0/2193699526 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd9c006af00 0x7fd9c006d3c0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fd9d00090b0 tx=0x7fd9d0008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:57.294 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:57.288+0000 7fd9d97fa700 1 -- 172.21.15.143:0/2193699526 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd9d4054610 con 0x7fd9dc15bb20 2023-11-08T19:26:57.522 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:57.521+0000 7fd9e2bbf700 1 -- 172.21.15.143:0/2193699526 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fd9dc0cfda0 con 0x7fd9dc15bb20 2023-11-08T19:26:58.129 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.128+0000 7fd9d97fa700 1 -- 172.21.15.143:0/2193699526 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 9 and data pool 10 v29) v1 ==== 173+0+0 (secure 0 0 0) 0x7fd9d4057c30 con 0x7fd9dc15bb20 2023-11-08T19:26:58.132 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.131+0000 7fd9e2bbf700 1 -- 172.21.15.143:0/2193699526 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd9c006af00 msgr2=0x7fd9c006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:58.133 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.131+0000 7fd9e2bbf700 1 --2- 172.21.15.143:0/2193699526 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd9c006af00 0x7fd9c006d3c0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fd9d00090b0 tx=0x7fd9d0008030 comp rx=0 tx=0).stop 2023-11-08T19:26:58.133 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.131+0000 7fd9e2bbf700 1 -- 172.21.15.143:0/2193699526 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd9dc15bb20 msgr2=0x7fd9dc0cac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:58.133 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.131+0000 7fd9e2bbf700 1 --2- 172.21.15.143:0/2193699526 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd9dc15bb20 0x7fd9dc0cac40 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fd9d4002030 tx=0x7fd9d400a340 comp rx=0 tx=0).stop 2023-11-08T19:26:58.133 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.131+0000 7fd9e2bbf700 1 -- 172.21.15.143:0/2193699526 shutdown_connections 2023-11-08T19:26:58.133 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.131+0000 7fd9e2bbf700 1 --2- 172.21.15.143:0/2193699526 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd9c006af00 0x7fd9c006d3c0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:58.134 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.131+0000 7fd9e2bbf700 1 --2- 172.21.15.143:0/2193699526 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd9dc15bb20 0x7fd9dc0cac40 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:58.134 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.131+0000 7fd9e2bbf700 1 --2- 172.21.15.143:0/2193699526 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd9dc0b1730 0x7fd9dc153c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:58.134 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.131+0000 7fd9e2bbf700 1 --2- 172.21.15.143:0/2193699526 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd9dc0d1d80 0x7fd9dc06d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:58.134 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.131+0000 7fd9e2bbf700 1 -- 172.21.15.143:0/2193699526 >> 172.21.15.143:0/2193699526 conn(0x7fd9dc08b000 msgr2=0x7fd9dc0ce2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:58.135 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.131+0000 7fd9e2bbf700 1 -- 172.21.15.143:0/2193699526 shutdown_connections 2023-11-08T19:26:58.135 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.131+0000 7fd9e2bbf700 1 -- 172.21.15.143:0/2193699526 wait complete. 2023-11-08T19:26:58.137 INFO:teuthology.orchestra.run.smithi143.stderr:new fs with metadata pool 9 and data pool 10 2023-11-08T19:26:58.151 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:26:58.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.272+0000 7f477d1b8700 1 Processor -- start 2023-11-08T19:26:58.274 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.272+0000 7f477d1b8700 1 -- start start 2023-11-08T19:26:58.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.273+0000 7f477d1b8700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f477806b320 0x7f4778168c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:58.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.273+0000 7f477d1b8700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4778161df0 con 0x7f47781618f0 2023-11-08T19:26:58.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.273+0000 7f477d1b8700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47781620d0 con 0x7f477815dd00 2023-11-08T19:26:58.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.273+0000 7f477d1b8700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47780cad50 con 0x7f477806b320 2023-11-08T19:26:58.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.273+0000 7f477659c700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f47781618f0 0x7f477806ac10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:48666/0 (socket says 172.21.15.143:48666) 2023-11-08T19:26:58.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.273+0000 7f477659c700 1 -- 172.21.15.143:0/2469298080 learned_addr learned my addr 172.21.15.143:0/2469298080 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:26:58.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.273+0000 7f477759e700 1 --2- 172.21.15.143:0/2469298080 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f477806b320 0x7f4778168c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:58.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.273+0000 7f4775d9b700 1 -- 172.21.15.143:0/2469298080 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1936473709 0 0) 0x7f4778161df0 con 0x7f47781618f0 2023-11-08T19:26:58.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.273+0000 7f4775d9b700 1 -- 172.21.15.143:0/2469298080 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47600035b0 con 0x7f47781618f0 2023-11-08T19:26:58.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.273+0000 7f4775d9b700 1 -- 172.21.15.143:0/2469298080 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1561049255 0 0) 0x7f47600035b0 con 0x7f47781618f0 2023-11-08T19:26:58.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.273+0000 7f4775d9b700 1 -- 172.21.15.143:0/2469298080 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f477806b320 msgr2=0x7f4778168c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:58.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.273+0000 7f4775d9b700 1 --2- 172.21.15.143:0/2469298080 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f477806b320 0x7f4778168c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:58.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.273+0000 7f4775d9b700 1 -- 172.21.15.143:0/2469298080 >> v1:172.21.15.174:6789/0 conn(0x7f477815dd00 legacy=0x7f47780cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:58.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.273+0000 7f4775d9b700 1 -- 172.21.15.143:0/2469298080 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47781623b0 con 0x7f47781618f0 2023-11-08T19:26:58.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.273+0000 7f4775d9b700 1 -- 172.21.15.143:0/2469298080 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4147756303 0 0) 0x7f4768001670 con 0x7f47781618f0 2023-11-08T19:26:58.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.274+0000 7f4775d9b700 1 -- 172.21.15.143:0/2469298080 >> v1:172.21.15.143:6789/0 conn(0x7f47781618f0 legacy=0x7f477806ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:58.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.274+0000 7f4775d9b700 1 --2- 172.21.15.143:0/2469298080 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4760003a30 0x7f4760003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:58.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.274+0000 7f4775d9b700 1 --2- 172.21.15.143:0/2469298080 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f47600043e0 0x7f47600048b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:58.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.274+0000 7f4775d9b700 1 --2- 172.21.15.143:0/2469298080 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f477806b320 0x7f4760005110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:58.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.274+0000 7f4775d9b700 1 -- 172.21.15.143:0/2469298080 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f476000ba10 con 0x7f4760003a30 2023-11-08T19:26:58.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.274+0000 7f4775d9b700 1 -- 172.21.15.143:0/2469298080 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f476000bb80 con 0x7f47600043e0 2023-11-08T19:26:58.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.274+0000 7f4775d9b700 1 -- 172.21.15.143:0/2469298080 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f476000bd40 con 0x7f477806b320 2023-11-08T19:26:58.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.274+0000 7f4776d9d700 1 --2- 172.21.15.143:0/2469298080 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4760003a30 0x7f4760003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:58.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.274+0000 7f477659c700 1 --2- 172.21.15.143:0/2469298080 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f47600043e0 0x7f47600048b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:58.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.274+0000 7f477759e700 1 --2- 172.21.15.143:0/2469298080 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f477806b320 0x7f4760005110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:58.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.274+0000 7f4776d9d700 1 -- 172.21.15.143:0/2469298080 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f477806b320 msgr2=0x7f4760005110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:58.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.274+0000 7f4776d9d700 1 --2- 172.21.15.143:0/2469298080 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f477806b320 0x7f4760005110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:58.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.274+0000 7f4776d9d700 1 -- 172.21.15.143:0/2469298080 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f47600043e0 msgr2=0x7f47600048b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:58.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.274+0000 7f4776d9d700 1 --2- 172.21.15.143:0/2469298080 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f47600043e0 0x7f47600048b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:58.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.274+0000 7f4776d9d700 1 -- 172.21.15.143:0/2469298080 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47600035b0 con 0x7f4760003a30 2023-11-08T19:26:58.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.274+0000 7f477659c700 1 --2- 172.21.15.143:0/2469298080 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f47600043e0 0x7f47600048b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:26:58.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.274+0000 7f4776d9d700 1 --2- 172.21.15.143:0/2469298080 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4760003a30 0x7f4760003e10 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f476c0018e0 tx=0x7f4760005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=502f44a25dca5752 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:58.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.274+0000 7f4775d9b700 1 -- 172.21.15.143:0/2469298080 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f476c00c660 con 0x7f4760003a30 2023-11-08T19:26:58.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.274+0000 7f4775d9b700 1 -- 172.21.15.143:0/2469298080 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f476c00c7e0 con 0x7f4760003a30 2023-11-08T19:26:58.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.274+0000 7f4775d9b700 1 -- 172.21.15.143:0/2469298080 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f476c00cb90 con 0x7f4760003a30 2023-11-08T19:26:58.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.275+0000 7f477d1b8700 1 -- 172.21.15.143:0/2469298080 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4760003a30 msgr2=0x7f4760003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:58.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.275+0000 7f477d1b8700 1 --2- 172.21.15.143:0/2469298080 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4760003a30 0x7f4760003e10 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f476c0018e0 tx=0x7f4760005880 comp rx=0 tx=0).stop 2023-11-08T19:26:58.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.275+0000 7f477759e700 1 -- 172.21.15.143:0/2469298080 reap_dead start 2023-11-08T19:26:58.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.275+0000 7f477d1b8700 1 -- 172.21.15.143:0/2469298080 shutdown_connections 2023-11-08T19:26:58.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.275+0000 7f477d1b8700 1 -- 172.21.15.143:0/2469298080 >> 172.21.15.143:0/2469298080 conn(0x7f4778099ab0 msgr2=0x7f47780d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:58.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.275+0000 7f477d1b8700 1 -- 172.21.15.143:0/2469298080 shutdown_connections 2023-11-08T19:26:58.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.275+0000 7f477d1b8700 1 -- 172.21.15.143:0/2469298080 wait complete. 2023-11-08T19:26:58.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.276+0000 7f477d1b8700 1 Processor -- start 2023-11-08T19:26:58.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.276+0000 7f477d1b8700 1 -- start start 2023-11-08T19:26:58.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.277+0000 7f477d1b8700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4760003a30 0x7f477815aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:58.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.277+0000 7f477d1b8700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f47780c8fb0 0x7f47780c81e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:58.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.277+0000 7f477d1b8700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f47780aefc0 0x7f47780af450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:58.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.277+0000 7f477d1b8700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47780af990 con 0x7f47780aefc0 2023-11-08T19:26:58.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.277+0000 7f477d1b8700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47780ca2f0 con 0x7f47780c8fb0 2023-11-08T19:26:58.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.277+0000 7f477d1b8700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47780c96e0 con 0x7f4760003a30 2023-11-08T19:26:58.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.277+0000 7f477759e700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f47780aefc0 0x7f47780af450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:58.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.277+0000 7f477759e700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f47780aefc0 0x7f47780af450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52524/0 (socket says 172.21.15.143:52524) 2023-11-08T19:26:58.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.277+0000 7f477759e700 1 -- 172.21.15.143:0/3914360804 learned_addr learned my addr 172.21.15.143:0/3914360804 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:58.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.277+0000 7f477659c700 1 --2- 172.21.15.143:0/3914360804 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f47780c8fb0 0x7f47780c81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:58.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.277+0000 7f4776d9d700 1 --2- 172.21.15.143:0/3914360804 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4760003a30 0x7f477815aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:58.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.277+0000 7f477759e700 1 -- 172.21.15.143:0/3914360804 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4760003a30 msgr2=0x7f477815aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:58.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.277+0000 7f477759e700 1 --2- 172.21.15.143:0/3914360804 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4760003a30 0x7f477815aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:58.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.277+0000 7f477759e700 1 -- 172.21.15.143:0/3914360804 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f47780c8fb0 msgr2=0x7f47780c81e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:58.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.277+0000 7f477759e700 1 --2- 172.21.15.143:0/3914360804 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f47780c8fb0 0x7f47780c81e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:58.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.277+0000 7f477759e700 1 -- 172.21.15.143:0/3914360804 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f477815e260 con 0x7f47780aefc0 2023-11-08T19:26:58.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.278+0000 7f477759e700 1 --2- 172.21.15.143:0/3914360804 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f47780aefc0 0x7f47780af450 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f4778073130 tx=0x7f47700114b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:58.293 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.278+0000 7f475f7fe700 1 -- 172.21.15.143:0/3914360804 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f477000e070 con 0x7f47780aefc0 2023-11-08T19:26:58.293 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.278+0000 7f475f7fe700 1 -- 172.21.15.143:0/3914360804 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4770003cf0 con 0x7f47780aefc0 2023-11-08T19:26:58.293 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.278+0000 7f477d1b8700 1 -- 172.21.15.143:0/3914360804 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47780c99c0 con 0x7f47780aefc0 2023-11-08T19:26:58.294 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.278+0000 7f475f7fe700 1 -- 172.21.15.143:0/3914360804 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4770003ea0 con 0x7f47780aefc0 2023-11-08T19:26:58.294 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.278+0000 7f477d1b8700 1 -- 172.21.15.143:0/3914360804 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47780c9f80 con 0x7f47780aefc0 2023-11-08T19:26:58.294 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.279+0000 7f477d1b8700 1 -- 172.21.15.143:0/3914360804 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47780b3110 con 0x7f47780aefc0 2023-11-08T19:26:58.294 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.279+0000 7f475f7fe700 1 -- 172.21.15.143:0/3914360804 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4770011960 con 0x7f47780aefc0 2023-11-08T19:26:58.296 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.280+0000 7f475f7fe700 1 --2- 172.21.15.143:0/3914360804 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f476406af00 0x7f476406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:58.296 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.280+0000 7f475f7fe700 1 -- 172.21.15.143:0/3914360804 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4770089ac0 con 0x7f47780aefc0 2023-11-08T19:26:58.296 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.280+0000 7f4776d9d700 1 --2- 172.21.15.143:0/3914360804 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f476406af00 0x7f476406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:58.296 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.280+0000 7f4776d9d700 1 --2- 172.21.15.143:0/3914360804 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f476406af00 0x7f476406d3c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f476c0018e0 tx=0x7f476c009030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:58.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.284+0000 7f475f7fe700 1 -- 172.21.15.143:0/3914360804 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4770054aa0 con 0x7f47780aefc0 2023-11-08T19:26:58.475 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.474+0000 7f477d1b8700 1 -- 172.21.15.143:0/3914360804 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f477804eee0 con 0x7f47780aefc0 2023-11-08T19:26:58.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.475+0000 7f475f7fe700 1 -- 172.21.15.143:0/3914360804 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) v1 ==== 74+0+18325 (secure 0 0 0) 0x7f47700580c0 con 0x7f47780aefc0 2023-11-08T19:26:58.477 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:26:58.477 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":61,"fsid":"968483af-77fc-4068-ad7c-5c2a1a4613d7","created":"2023-11-08T19:24:32.826484+0000","modified":"2023-11-08T19:26:57.522463+0000","last_up_change":"2023-11-08T19:24:43.573880+0000","last_in_change":"2023-11-08T19:24:35.314525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:44.551530+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"snap-test-pool","create_time":"2023-11-08T19:26:02.874384+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":36,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-11-08T19:26:55.754227+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-11-08T19:26:56.486110+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bae98ec1-d94e-403d-a220-a522a53ea6f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6809","nonce":1487830566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6811","nonce":1487830566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6815","nonce":1487830566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6813","nonce":1487830566}]},"public_addr":"172.21.15.143:6809/1487830566","cluster_addr":"172.21.15.143:6811/1487830566","heartbeat_back_addr":"172.21.15.143:6815/1487830566","heartbeat_front_addr":"172.21.15.143:6813/1487830566","state":["exists","up"]},{"osd":1,"uuid":"5cb6538a-1ccb-4322-8633-09f66af0b7e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6801","nonce":787341397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6803","nonce":787341397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6807","nonce":787341397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6805","nonce":787341397}]},"public_addr":"172.21.15.143:6801/787341397","cluster_addr":"172.21.15.143:6803/787341397","heartbeat_back_addr":"172.21.15.143:6807/787341397","heartbeat_front_addr":"172.21.15.143:6805/787341397","state":["exists","up"]},{"osd":2,"uuid":"cdbf522c-60ef-4d60-9653-8dcb529c101a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6822","nonce":2301325597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6826","nonce":2301325597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6831","nonce":2301325597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6829","nonce":2301325597}]},"public_addr":"172.21.15.143:6822/2301325597","cluster_addr":"172.21.15.143:6826/2301325597","heartbeat_back_addr":"172.21.15.143:6831/2301325597","heartbeat_front_addr":"172.21.15.143:6829/2301325597","state":["exists","up"]},{"osd":3,"uuid":"4e2783cf-ebb9-4dd8-8245-bc1d17ce4325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6817","nonce":2021504428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6819","nonce":2021504428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6827","nonce":2021504428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6823","nonce":2021504428}]},"public_addr":"172.21.15.143:6817/2021504428","cluster_addr":"172.21.15.143:6819/2021504428","heartbeat_back_addr":"172.21.15.143:6827/2021504428","heartbeat_front_addr":"172.21.15.143:6823/2021504428","state":["exists","up"]},{"osd":4,"uuid":"691256fa-9b77-49b9-a346-7ae1adf0ca80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6801","nonce":1993380595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6803","nonce":1993380595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6807","nonce":1993380595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6805","nonce":1993380595}]},"public_addr":"172.21.15.174:6801/1993380595","cluster_addr":"172.21.15.174:6803/1993380595","heartbeat_back_addr":"172.21.15.174:6807/1993380595","heartbeat_front_addr":"172.21.15.174:6805/1993380595","state":["exists","up"]},{"osd":5,"uuid":"756fc352-48c9-46ce-b52e-0541fe6f8525","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6809","nonce":1777923303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6811","nonce":1777923303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6815","nonce":1777923303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6813","nonce":1777923303}]},"public_addr":"172.21.15.174:6809/1777923303","cluster_addr":"172.21.15.174:6811/1777923303","heartbeat_back_addr":"172.21.15.174:6815/1777923303","heartbeat_front_addr":"172.21.15.174:6813/1777923303","state":["exists","up"]},{"osd":6,"uuid":"fab2ac2b-ed97-4cde-9825-b480eabf8b8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6825","nonce":2755130316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6827","nonce":2755130316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6831","nonce":2755130316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6829","nonce":2755130316}]},"public_addr":"172.21.15.174:6825/2755130316","cluster_addr":"172.21.15.174:6827/2755130316","heartbeat_back_addr":"172.21.15.174:6831/2755130316","heartbeat_front_addr":"172.21.15.174:6829/2755130316","state":["exists","up"]},{"osd":7,"uuid":"3de9b797-2ab6-4d41-a863-7e7d638fd703","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6817","nonce":2847141411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6819","nonce":2847141411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6823","nonce":2847141411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6821","nonce":2847141411}]},"public_addr":"172.21.15.174:6817/2847141411","cluster_addr":"172.21.15.174:6819/2847141411","heartbeat_back_addr":"172.21.15.174:6823/2847141411","heartbeat_front_addr":"172.21.15.174:6821/2847141411","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.477076+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.496890+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.539305+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.454673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.538881+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.521294+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.520712+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.540404+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:26:58.479 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.477+0000 7f477d1b8700 1 -- 172.21.15.143:0/3914360804 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f476406af00 msgr2=0x7f476406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:58.479 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.477+0000 7f477d1b8700 1 --2- 172.21.15.143:0/3914360804 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f476406af00 0x7f476406d3c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f476c0018e0 tx=0x7f476c009030 comp rx=0 tx=0).stop 2023-11-08T19:26:58.479 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.477+0000 7f477d1b8700 1 -- 172.21.15.143:0/3914360804 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f47780aefc0 msgr2=0x7f47780af450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:58.479 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.477+0000 7f477d1b8700 1 --2- 172.21.15.143:0/3914360804 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f47780aefc0 0x7f47780af450 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f4778073130 tx=0x7f47700114b0 comp rx=0 tx=0).stop 2023-11-08T19:26:58.480 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.477+0000 7f477d1b8700 1 -- 172.21.15.143:0/3914360804 shutdown_connections 2023-11-08T19:26:58.480 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.477+0000 7f477d1b8700 1 --2- 172.21.15.143:0/3914360804 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f476406af00 0x7f476406d3c0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:58.480 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.477+0000 7f477d1b8700 1 --2- 172.21.15.143:0/3914360804 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4760003a30 0x7f477815aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:58.480 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.477+0000 7f477d1b8700 1 --2- 172.21.15.143:0/3914360804 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f47780aefc0 0x7f47780af450 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:58.480 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.477+0000 7f477d1b8700 1 --2- 172.21.15.143:0/3914360804 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f47780c8fb0 0x7f47780c81e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:58.481 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.477+0000 7f477d1b8700 1 -- 172.21.15.143:0/3914360804 >> 172.21.15.143:0/3914360804 conn(0x7f4778099ab0 msgr2=0x7f477806db40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:58.481 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.477+0000 7f477d1b8700 1 -- 172.21.15.143:0/3914360804 shutdown_connections 2023-11-08T19:26:58.481 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.477+0000 7f477d1b8700 1 -- 172.21.15.143:0/3914360804 wait complete. 2023-11-08T19:26:58.491 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:26:58.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.611+0000 7fc4dc6d3700 1 Processor -- start 2023-11-08T19:26:58.613 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.611+0000 7fc4dc6d3700 1 -- start start 2023-11-08T19:26:58.615 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.612+0000 7fc4dc6d3700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fc4d40b2920 0x7fc4d40cf510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:58.615 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.612+0000 7fc4dc6d3700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc4d40cbe20 0x7fc4d40cc290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:58.616 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.612+0000 7fc4dc6d3700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4d40cf080 con 0x7fc4d415aa30 2023-11-08T19:26:58.616 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.612+0000 7fc4dc6d3700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4d40cc7d0 con 0x7fc4d40cbe20 2023-11-08T19:26:58.616 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.612+0000 7fc4dc6d3700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4d40ce470 con 0x7fc4d40b2920 2023-11-08T19:26:58.616 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.612+0000 7fc4da46f700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fc4d415aa30 0x7fc4d40b2210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:48682/0 (socket says 172.21.15.143:48682) 2023-11-08T19:26:58.616 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.612+0000 7fc4da46f700 1 -- 172.21.15.143:0/170683007 learned_addr learned my addr 172.21.15.143:0/170683007 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:26:58.617 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.612+0000 7fc4dac70700 1 --2- 172.21.15.143:0/170683007 >> v2:172.21.15.143:3300/0 conn(0x7fc4d40cbe20 0x7fc4d40cc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:58.617 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.612+0000 7fc4d946d700 1 -- 172.21.15.143:0/170683007 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3000901538 0 0) 0x7fc4d40cf080 con 0x7fc4d415aa30 2023-11-08T19:26:58.617 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.612+0000 7fc4d946d700 1 -- 172.21.15.143:0/170683007 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc4c00035b0 con 0x7fc4d415aa30 2023-11-08T19:26:58.617 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.612+0000 7fc4d9c6e700 1 --2- 172.21.15.143:0/170683007 >> v2:172.21.15.174:3300/0 conn(0x7fc4d40b2920 0x7fc4d40cf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:58.617 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.612+0000 7fc4dac70700 1 -- 172.21.15.143:0/170683007 >> v2:172.21.15.174:3300/0 conn(0x7fc4d40b2920 msgr2=0x7fc4d40cf510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:58.618 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.613+0000 7fc4dac70700 1 --2- 172.21.15.143:0/170683007 >> v2:172.21.15.174:3300/0 conn(0x7fc4d40b2920 0x7fc4d40cf510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:58.618 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.613+0000 7fc4dac70700 1 -- 172.21.15.143:0/170683007 >> v1:172.21.15.143:6789/0 conn(0x7fc4d415aa30 legacy=0x7fc4d40b2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:58.618 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.613+0000 7fc4dac70700 1 -- 172.21.15.143:0/170683007 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4d40ce750 con 0x7fc4d40cbe20 2023-11-08T19:26:58.618 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.613+0000 7fc4d9c6e700 1 --2- 172.21.15.143:0/170683007 >> v2:172.21.15.174:3300/0 conn(0x7fc4d40b2920 0x7fc4d40cf510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:26:58.619 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.613+0000 7fc4dac70700 1 --2- 172.21.15.143:0/170683007 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc4d40cbe20 0x7fc4d40cc290 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fc4d40ce260 tx=0x7fc4cc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b34c2aabcddd1ed3 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:58.619 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.613+0000 7fc4d946d700 1 -- 172.21.15.143:0/170683007 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4cc00e070 con 0x7fc4d40cbe20 2023-11-08T19:26:58.620 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.613+0000 7fc4d946d700 1 -- 172.21.15.143:0/170683007 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4cc00bb20 con 0x7fc4d40cbe20 2023-11-08T19:26:58.620 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.613+0000 7fc4d946d700 1 -- 172.21.15.143:0/170683007 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4cc00be90 con 0x7fc4d40cbe20 2023-11-08T19:26:58.620 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.613+0000 7fc4dc6d3700 1 -- 172.21.15.143:0/170683007 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc4d40cbe20 msgr2=0x7fc4d40cc290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:58.620 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.613+0000 7fc4dc6d3700 1 --2- 172.21.15.143:0/170683007 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc4d40cbe20 0x7fc4d40cc290 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fc4d40ce260 tx=0x7fc4cc00b330 comp rx=0 tx=0).stop 2023-11-08T19:26:58.620 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.613+0000 7fc4dc6d3700 1 -- 172.21.15.143:0/170683007 shutdown_connections 2023-11-08T19:26:58.621 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.613+0000 7fc4dc6d3700 1 --2- 172.21.15.143:0/170683007 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc4d40cbe20 0x7fc4d40cc290 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:58.621 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.613+0000 7fc4dc6d3700 1 --2- 172.21.15.143:0/170683007 >> v2:172.21.15.174:3300/0 conn(0x7fc4d40b2920 0x7fc4d40cf510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:58.621 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.613+0000 7fc4dc6d3700 1 -- 172.21.15.143:0/170683007 >> 172.21.15.143:0/170683007 conn(0x7fc4d4099ab0 msgr2=0x7fc4d40d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:58.621 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.613+0000 7fc4dc6d3700 1 -- 172.21.15.143:0/170683007 shutdown_connections 2023-11-08T19:26:58.621 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.613+0000 7fc4dc6d3700 1 -- 172.21.15.143:0/170683007 wait complete. 2023-11-08T19:26:58.624 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.615+0000 7fc4dc6d3700 1 Processor -- start 2023-11-08T19:26:58.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.615+0000 7fc4dc6d3700 1 -- start start 2023-11-08T19:26:58.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.615+0000 7fc4dc6d3700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc4d40b2920 0x7fc4d40b01f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:58.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.615+0000 7fc4dc6d3700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc4d40cbe20 0x7fc4d40b1c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:58.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.615+0000 7fc4dc6d3700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc4d415aa30 0x7fc4d40c63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:58.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.615+0000 7fc4dc6d3700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4d40b07d0 con 0x7fc4d40b2920 2023-11-08T19:26:58.626 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.615+0000 7fc4dc6d3700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4d40b0ab0 con 0x7fc4d40cbe20 2023-11-08T19:26:58.626 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.615+0000 7fc4dc6d3700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4d40b0d90 con 0x7fc4d415aa30 2023-11-08T19:26:58.626 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.615+0000 7fc4da46f700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc4d40b2920 0x7fc4d40b01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:58.626 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.615+0000 7fc4da46f700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc4d40b2920 0x7fc4d40b01f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52534/0 (socket says 172.21.15.143:52534) 2023-11-08T19:26:58.626 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.615+0000 7fc4da46f700 1 -- 172.21.15.143:0/3644288802 learned_addr learned my addr 172.21.15.143:0/3644288802 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:58.627 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.616+0000 7fc4dac70700 1 --2- 172.21.15.143:0/3644288802 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc4d415aa30 0x7fc4d40c63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:58.627 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.616+0000 7fc4da46f700 1 -- 172.21.15.143:0/3644288802 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc4d415aa30 msgr2=0x7fc4d40c63b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:58.627 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.616+0000 7fc4d9c6e700 1 --2- 172.21.15.143:0/3644288802 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc4d40cbe20 0x7fc4d40b1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:58.627 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.616+0000 7fc4da46f700 1 --2- 172.21.15.143:0/3644288802 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc4d415aa30 0x7fc4d40c63b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:58.627 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.616+0000 7fc4da46f700 1 -- 172.21.15.143:0/3644288802 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc4d40cbe20 msgr2=0x7fc4d40b1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:58.628 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.616+0000 7fc4da46f700 1 --2- 172.21.15.143:0/3644288802 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc4d40cbe20 0x7fc4d40b1c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:58.628 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.616+0000 7fc4da46f700 1 -- 172.21.15.143:0/3644288802 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4cc009a70 con 0x7fc4d40b2920 2023-11-08T19:26:58.628 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.616+0000 7fc4dac70700 1 --2- 172.21.15.143:0/3644288802 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc4d415aa30 0x7fc4d40c63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:26:58.628 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.616+0000 7fc4d9c6e700 1 --2- 172.21.15.143:0/3644288802 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc4d40cbe20 0x7fc4d40b1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:26:58.628 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.616+0000 7fc4da46f700 1 --2- 172.21.15.143:0/3644288802 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc4d40b2920 0x7fc4d40b01f0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fc4c400c4d0 tx=0x7fc4c400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:58.629 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.616+0000 7fc4caffd700 1 -- 172.21.15.143:0/3644288802 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4c40074e0 con 0x7fc4d40b2920 2023-11-08T19:26:58.629 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.616+0000 7fc4caffd700 1 -- 172.21.15.143:0/3644288802 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4c4007640 con 0x7fc4d40b2920 2023-11-08T19:26:58.629 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.616+0000 7fc4caffd700 1 -- 172.21.15.143:0/3644288802 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4c4005a70 con 0x7fc4d40b2920 2023-11-08T19:26:58.630 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.616+0000 7fc4dc6d3700 1 -- 172.21.15.143:0/3644288802 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4d40b1070 con 0x7fc4d40b2920 2023-11-08T19:26:58.630 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.616+0000 7fc4dc6d3700 1 -- 172.21.15.143:0/3644288802 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4d40c68f0 con 0x7fc4d40b2920 2023-11-08T19:26:58.630 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.617+0000 7fc4dc6d3700 1 -- 172.21.15.143:0/3644288802 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4d404eee0 con 0x7fc4d40b2920 2023-11-08T19:26:58.630 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.617+0000 7fc4caffd700 1 -- 172.21.15.143:0/3644288802 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc4c4005bd0 con 0x7fc4d40b2920 2023-11-08T19:26:58.630 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.618+0000 7fc4caffd700 1 --2- 172.21.15.143:0/3644288802 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc4b806af00 0x7fc4b806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:58.631 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.618+0000 7fc4caffd700 1 -- 172.21.15.143:0/3644288802 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc4c4088c60 con 0x7fc4d40b2920 2023-11-08T19:26:58.631 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.618+0000 7fc4d9c6e700 1 --2- 172.21.15.143:0/3644288802 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc4b806af00 0x7fc4b806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:58.631 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.618+0000 7fc4d9c6e700 1 --2- 172.21.15.143:0/3644288802 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc4b806af00 0x7fc4b806d3c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fc4d40b2270 tx=0x7fc4d0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:58.631 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.622+0000 7fc4caffd700 1 -- 172.21.15.143:0/3644288802 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc4c4053bb0 con 0x7fc4d40b2920 2023-11-08T19:26:58.813 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.812+0000 7fc4dc6d3700 1 -- 172.21.15.143:0/3644288802 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc4d40b2ff0 con 0x7fc4d40b2920 2023-11-08T19:26:58.814 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.813+0000 7fc4caffd700 1 -- 172.21.15.143:0/3644288802 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v62) v1 ==== 74+0+18312 (secure 0 0 0) 0x7fc4c4017020 con 0x7fc4d40b2920 2023-11-08T19:26:58.815 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:26:58.815 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":62,"fsid":"968483af-77fc-4068-ad7c-5c2a1a4613d7","created":"2023-11-08T19:24:32.826484+0000","modified":"2023-11-08T19:26:58.579564+0000","last_up_change":"2023-11-08T19:24:43.573880+0000","last_in_change":"2023-11-08T19:24:35.314525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:44.551530+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"snap-test-pool","create_time":"2023-11-08T19:26:02.874384+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":36,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-11-08T19:26:55.754227+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-11-08T19:26:56.486110+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bae98ec1-d94e-403d-a220-a522a53ea6f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6809","nonce":1487830566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6811","nonce":1487830566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6815","nonce":1487830566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6813","nonce":1487830566}]},"public_addr":"172.21.15.143:6809/1487830566","cluster_addr":"172.21.15.143:6811/1487830566","heartbeat_back_addr":"172.21.15.143:6815/1487830566","heartbeat_front_addr":"172.21.15.143:6813/1487830566","state":["exists","up"]},{"osd":1,"uuid":"5cb6538a-1ccb-4322-8633-09f66af0b7e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6801","nonce":787341397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6803","nonce":787341397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6807","nonce":787341397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6805","nonce":787341397}]},"public_addr":"172.21.15.143:6801/787341397","cluster_addr":"172.21.15.143:6803/787341397","heartbeat_back_addr":"172.21.15.143:6807/787341397","heartbeat_front_addr":"172.21.15.143:6805/787341397","state":["exists","up"]},{"osd":2,"uuid":"cdbf522c-60ef-4d60-9653-8dcb529c101a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6822","nonce":2301325597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6826","nonce":2301325597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6831","nonce":2301325597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6829","nonce":2301325597}]},"public_addr":"172.21.15.143:6822/2301325597","cluster_addr":"172.21.15.143:6826/2301325597","heartbeat_back_addr":"172.21.15.143:6831/2301325597","heartbeat_front_addr":"172.21.15.143:6829/2301325597","state":["exists","up"]},{"osd":3,"uuid":"4e2783cf-ebb9-4dd8-8245-bc1d17ce4325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6817","nonce":2021504428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6819","nonce":2021504428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6827","nonce":2021504428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6823","nonce":2021504428}]},"public_addr":"172.21.15.143:6817/2021504428","cluster_addr":"172.21.15.143:6819/2021504428","heartbeat_back_addr":"172.21.15.143:6827/2021504428","heartbeat_front_addr":"172.21.15.143:6823/2021504428","state":["exists","up"]},{"osd":4,"uuid":"691256fa-9b77-49b9-a346-7ae1adf0ca80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6801","nonce":1993380595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6803","nonce":1993380595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6807","nonce":1993380595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6805","nonce":1993380595}]},"public_addr":"172.21.15.174:6801/1993380595","cluster_addr":"172.21.15.174:6803/1993380595","heartbeat_back_addr":"172.21.15.174:6807/1993380595","heartbeat_front_addr":"172.21.15.174:6805/1993380595","state":["exists","up"]},{"osd":5,"uuid":"756fc352-48c9-46ce-b52e-0541fe6f8525","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6809","nonce":1777923303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6811","nonce":1777923303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6815","nonce":1777923303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6813","nonce":1777923303}]},"public_addr":"172.21.15.174:6809/1777923303","cluster_addr":"172.21.15.174:6811/1777923303","heartbeat_back_addr":"172.21.15.174:6815/1777923303","heartbeat_front_addr":"172.21.15.174:6813/1777923303","state":["exists","up"]},{"osd":6,"uuid":"fab2ac2b-ed97-4cde-9825-b480eabf8b8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6825","nonce":2755130316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6827","nonce":2755130316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6831","nonce":2755130316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6829","nonce":2755130316}]},"public_addr":"172.21.15.174:6825/2755130316","cluster_addr":"172.21.15.174:6827/2755130316","heartbeat_back_addr":"172.21.15.174:6831/2755130316","heartbeat_front_addr":"172.21.15.174:6829/2755130316","state":["exists","up"]},{"osd":7,"uuid":"3de9b797-2ab6-4d41-a863-7e7d638fd703","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6817","nonce":2847141411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6819","nonce":2847141411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6823","nonce":2847141411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6821","nonce":2847141411}]},"public_addr":"172.21.15.174:6817/2847141411","cluster_addr":"172.21.15.174:6819/2847141411","heartbeat_back_addr":"172.21.15.174:6823/2847141411","heartbeat_front_addr":"172.21.15.174:6821/2847141411","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.477076+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.496890+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.539305+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.454673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.538881+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.521294+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.520712+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.540404+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:26:58.817 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.815+0000 7fc4dc6d3700 1 -- 172.21.15.143:0/3644288802 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc4b806af00 msgr2=0x7fc4b806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:58.817 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.815+0000 7fc4dc6d3700 1 --2- 172.21.15.143:0/3644288802 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc4b806af00 0x7fc4b806d3c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fc4d40b2270 tx=0x7fc4d0008040 comp rx=0 tx=0).stop 2023-11-08T19:26:58.817 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.815+0000 7fc4dc6d3700 1 -- 172.21.15.143:0/3644288802 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc4d40b2920 msgr2=0x7fc4d40b01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:58.818 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.815+0000 7fc4dc6d3700 1 --2- 172.21.15.143:0/3644288802 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc4d40b2920 0x7fc4d40b01f0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fc4c400c4d0 tx=0x7fc4c400c890 comp rx=0 tx=0).stop 2023-11-08T19:26:58.818 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.815+0000 7fc4dc6d3700 1 -- 172.21.15.143:0/3644288802 shutdown_connections 2023-11-08T19:26:58.818 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.815+0000 7fc4dc6d3700 1 --2- 172.21.15.143:0/3644288802 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc4b806af00 0x7fc4b806d3c0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:58.818 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.815+0000 7fc4dc6d3700 1 --2- 172.21.15.143:0/3644288802 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc4d415aa30 0x7fc4d40c63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:58.818 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.815+0000 7fc4dc6d3700 1 --2- 172.21.15.143:0/3644288802 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc4d40b2920 0x7fc4d40b01f0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:58.819 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.815+0000 7fc4dc6d3700 1 --2- 172.21.15.143:0/3644288802 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc4d40cbe20 0x7fc4d40b1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:58.819 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.815+0000 7fc4dc6d3700 1 -- 172.21.15.143:0/3644288802 >> 172.21.15.143:0/3644288802 conn(0x7fc4d4099ab0 msgr2=0x7fc4d4158050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:58.819 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.815+0000 7fc4dc6d3700 1 -- 172.21.15.143:0/3644288802 shutdown_connections 2023-11-08T19:26:58.819 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.815+0000 7fc4dc6d3700 1 -- 172.21.15.143:0/3644288802 wait complete. 2023-11-08T19:26:58.830 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-08T19:26:58.953 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.952+0000 7f6b1e86b700 1 Processor -- start 2023-11-08T19:26:58.953 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.952+0000 7f6b1e86b700 1 -- start start 2023-11-08T19:26:58.956 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.952+0000 7f6b1e86b700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f6b1815dba0 0x7f6b1806c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:58.956 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.952+0000 7f6b1e86b700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f6b180b22e0 0x7f6b180b2700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:58.956 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.952+0000 7f6b1e86b700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b18161890 con 0x7f6b1806b240 2023-11-08T19:26:58.956 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.952+0000 7f6b1e86b700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b180b2c40 con 0x7f6b1815dba0 2023-11-08T19:26:58.957 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.952+0000 7f6b1e86b700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b1806b740 con 0x7f6b180b22e0 2023-11-08T19:26:58.957 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.953+0000 7f6b17fff700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f6b1815dba0 0x7f6b1806c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:58.957 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.953+0000 7f6b177fe700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f6b1806b240 0x7f6b180caff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:48698/0 (socket says 172.21.15.143:48698) 2023-11-08T19:26:58.957 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.953+0000 7f6b177fe700 1 -- 172.21.15.143:0/3836479991 learned_addr learned my addr 172.21.15.143:0/3836479991 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:26:58.958 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.953+0000 7f6b1ce08700 1 --2- 172.21.15.143:0/3836479991 >> v2:172.21.15.174:3300/0 conn(0x7f6b180b22e0 0x7f6b180b2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:58.958 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.953+0000 7f6b17fff700 1 -- 172.21.15.143:0/3836479991 >> v2:172.21.15.174:3300/0 conn(0x7f6b180b22e0 msgr2=0x7f6b180b2700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:58.958 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.953+0000 7f6b17fff700 1 --2- 172.21.15.143:0/3836479991 >> v2:172.21.15.174:3300/0 conn(0x7f6b180b22e0 0x7f6b180b2700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:58.958 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.953+0000 7f6b17fff700 1 -- 172.21.15.143:0/3836479991 >> v1:172.21.15.143:6789/0 conn(0x7f6b1806b240 legacy=0x7f6b180caff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:58.959 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.953+0000 7f6b17fff700 1 -- 172.21.15.143:0/3836479991 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b1806ba20 con 0x7f6b1815dba0 2023-11-08T19:26:58.959 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.953+0000 7f6b16ffd700 1 -- 172.21.15.143:0/3836479991 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3454291079 0 0) 0x7f6b18161890 con 0x7f6b1806b240 2023-11-08T19:26:58.959 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.953+0000 7f6b17fff700 1 --2- 172.21.15.143:0/3836479991 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6b1815dba0 0x7f6b1806c610 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f6b1806c2c0 tx=0x7f6b08009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d5eef65f1f4c20a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:58.959 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.953+0000 7f6b16ffd700 1 -- 172.21.15.143:0/3836479991 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b08011830 con 0x7f6b1815dba0 2023-11-08T19:26:58.960 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.953+0000 7f6b16ffd700 1 -- 172.21.15.143:0/3836479991 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b08011990 con 0x7f6b1815dba0 2023-11-08T19:26:58.960 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.953+0000 7f6b16ffd700 1 -- 172.21.15.143:0/3836479991 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b08011d20 con 0x7f6b1815dba0 2023-11-08T19:26:58.960 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.954+0000 7f6b1e86b700 1 -- 172.21.15.143:0/3836479991 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6b1815dba0 msgr2=0x7f6b1806c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:58.960 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.954+0000 7f6b1e86b700 1 --2- 172.21.15.143:0/3836479991 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6b1815dba0 0x7f6b1806c610 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f6b1806c2c0 tx=0x7f6b08009b20 comp rx=0 tx=0).stop 2023-11-08T19:26:58.961 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.954+0000 7f6b1e86b700 1 -- 172.21.15.143:0/3836479991 shutdown_connections 2023-11-08T19:26:58.961 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.954+0000 7f6b1e86b700 1 --2- 172.21.15.143:0/3836479991 >> v2:172.21.15.174:3300/0 conn(0x7f6b180b22e0 0x7f6b180b2700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:58.961 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.954+0000 7f6b1e86b700 1 --2- 172.21.15.143:0/3836479991 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6b1815dba0 0x7f6b1806c610 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:58.961 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.954+0000 7f6b1e86b700 1 -- 172.21.15.143:0/3836479991 >> 172.21.15.143:0/3836479991 conn(0x7f6b180cd2e0 msgr2=0x7f6b180cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:58.961 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.954+0000 7f6b1e86b700 1 -- 172.21.15.143:0/3836479991 shutdown_connections 2023-11-08T19:26:58.962 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.954+0000 7f6b1e86b700 1 -- 172.21.15.143:0/3836479991 wait complete. 2023-11-08T19:26:58.964 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.955+0000 7f6b1e86b700 1 Processor -- start 2023-11-08T19:26:58.965 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.956+0000 7f6b1e86b700 1 -- start start 2023-11-08T19:26:58.965 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.956+0000 7f6b1e86b700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6b1806b240 0x7f6b180c7930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:58.965 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.956+0000 7f6b1e86b700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6b180b22e0 0x7f6b181f2e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:58.965 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.956+0000 7f6b1e86b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6b1815dba0 0x7f6b180b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:58.965 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.956+0000 7f6b1e86b700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b180c8d20 con 0x7f6b180b22e0 2023-11-08T19:26:58.966 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.956+0000 7f6b1e86b700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b180c7e70 con 0x7f6b1815dba0 2023-11-08T19:26:58.966 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.956+0000 7f6b1e86b700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b180c8110 con 0x7f6b1806b240 2023-11-08T19:26:58.966 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.957+0000 7f6b177fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6b180b22e0 0x7f6b181f2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:58.966 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.957+0000 7f6b177fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6b180b22e0 0x7f6b181f2e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52550/0 (socket says 172.21.15.143:52550) 2023-11-08T19:26:58.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.957+0000 7f6b177fe700 1 -- 172.21.15.143:0/1394813403 learned_addr learned my addr 172.21.15.143:0/1394813403 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:26:58.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.957+0000 7f6b17fff700 1 --2- 172.21.15.143:0/1394813403 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6b1806b240 0x7f6b180c7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:58.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.957+0000 7f6b177fe700 1 -- 172.21.15.143:0/1394813403 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6b1806b240 msgr2=0x7f6b180c7930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:58.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.957+0000 7f6b177fe700 1 --2- 172.21.15.143:0/1394813403 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6b1806b240 0x7f6b180c7930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:58.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.957+0000 7f6b177fe700 1 -- 172.21.15.143:0/1394813403 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6b1815dba0 msgr2=0x7f6b180b4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:58.968 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.957+0000 7f6b177fe700 1 --2- 172.21.15.143:0/1394813403 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6b1815dba0 0x7f6b180b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:58.968 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.957+0000 7f6b177fe700 1 -- 172.21.15.143:0/1394813403 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b080094a0 con 0x7f6b180b22e0 2023-11-08T19:26:58.968 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.957+0000 7f6b177fe700 1 --2- 172.21.15.143:0/1394813403 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6b180b22e0 0x7f6b181f2e60 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c0011e0 tx=0x7f6b0c00cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:58.968 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.957+0000 7f6b14ff9700 1 -- 172.21.15.143:0/1394813403 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b0c004c80 con 0x7f6b180b22e0 2023-11-08T19:26:58.968 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.957+0000 7f6b14ff9700 1 -- 172.21.15.143:0/1394813403 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b0c004e00 con 0x7f6b180b22e0 2023-11-08T19:26:58.968 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.957+0000 7f6b1e86b700 1 -- 172.21.15.143:0/1394813403 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b180c83f0 con 0x7f6b180b22e0 2023-11-08T19:26:58.969 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.957+0000 7f6b14ff9700 1 -- 172.21.15.143:0/1394813403 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b0c007310 con 0x7f6b180b22e0 2023-11-08T19:26:58.969 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.957+0000 7f6b1e86b700 1 -- 172.21.15.143:0/1394813403 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b180c89b0 con 0x7f6b180b22e0 2023-11-08T19:26:58.969 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.958+0000 7f6b1e86b700 1 -- 172.21.15.143:0/1394813403 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b180c8e90 con 0x7f6b180b22e0 2023-11-08T19:26:58.969 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.959+0000 7f6b14ff9700 1 -- 172.21.15.143:0/1394813403 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6b0c007470 con 0x7f6b180b22e0 2023-11-08T19:26:58.969 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.959+0000 7f6b14ff9700 1 --2- 172.21.15.143:0/1394813403 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f6b0406af00 0x7f6b0406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:58.970 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.959+0000 7f6b14ff9700 1 -- 172.21.15.143:0/1394813403 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6b0c089570 con 0x7f6b180b22e0 2023-11-08T19:26:58.970 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.959+0000 7f6b17fff700 1 --2- 172.21.15.143:0/1394813403 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f6b0406af00 0x7f6b0406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:58.970 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.960+0000 7f6b17fff700 1 --2- 172.21.15.143:0/1394813403 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f6b0406af00 0x7f6b0406d3c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f6b08000c00 tx=0x7f6b08008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:58.970 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:58.963+0000 7f6b14ff9700 1 -- 172.21.15.143:0/1394813403 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6b0c054430 con 0x7f6b180b22e0 2023-11-08T19:26:59.202 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:26:59.200+0000 7f6b1e86b700 1 -- 172.21.15.143:0/1394813403 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f6b1804eee0 con 0x7f6b180b22e0 2023-11-08T19:27:00.144 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:00.142+0000 7f6b14ff9700 1 -- 172.21.15.143:0/1394813403 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v32) v1 ==== 118+0+0 (secure 0 0 0) 0x7f6b0c057a50 con 0x7f6b180b22e0 2023-11-08T19:27:00.147 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:00.145+0000 7f6b1e86b700 1 -- 172.21.15.143:0/1394813403 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f6b0406af00 msgr2=0x7f6b0406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:00.147 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:00.145+0000 7f6b1e86b700 1 --2- 172.21.15.143:0/1394813403 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f6b0406af00 0x7f6b0406d3c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f6b08000c00 tx=0x7f6b08008030 comp rx=0 tx=0).stop 2023-11-08T19:27:00.148 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:00.145+0000 7f6b1e86b700 1 -- 172.21.15.143:0/1394813403 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6b180b22e0 msgr2=0x7f6b181f2e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:00.148 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:00.145+0000 7f6b1e86b700 1 --2- 172.21.15.143:0/1394813403 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6b180b22e0 0x7f6b181f2e60 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c0011e0 tx=0x7f6b0c00cae0 comp rx=0 tx=0).stop 2023-11-08T19:27:00.148 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:00.145+0000 7f6b1e86b700 1 -- 172.21.15.143:0/1394813403 shutdown_connections 2023-11-08T19:27:00.148 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:00.145+0000 7f6b1e86b700 1 --2- 172.21.15.143:0/1394813403 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f6b0406af00 0x7f6b0406d3c0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:00.149 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:00.145+0000 7f6b1e86b700 1 --2- 172.21.15.143:0/1394813403 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6b1806b240 0x7f6b180c7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:00.149 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:00.145+0000 7f6b1e86b700 1 --2- 172.21.15.143:0/1394813403 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6b1815dba0 0x7f6b180b4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:00.149 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:00.145+0000 7f6b1e86b700 1 --2- 172.21.15.143:0/1394813403 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6b180b22e0 0x7f6b181f2e60 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:00.149 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:00.145+0000 7f6b1e86b700 1 -- 172.21.15.143:0/1394813403 >> 172.21.15.143:0/1394813403 conn(0x7f6b180cd2e0 msgr2=0x7f6b1806c2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:00.149 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:00.146+0000 7f6b1e86b700 1 -- 172.21.15.143:0/1394813403 shutdown_connections 2023-11-08T19:27:00.150 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:00.146+0000 7f6b1e86b700 1 -- 172.21.15.143:0/1394813403 wait complete. 2023-11-08T19:27:00.162 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-08T19:27:00.163 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-08T19:27:00.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:00.290+0000 7f88473c7700 1 Processor -- start 2023-11-08T19:27:00.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:00.290+0000 7f88473c7700 1 -- start start 2023-11-08T19:27:00.294 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:00.291+0000 7f88473c7700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f88400d1d80 0x7f88400b2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:00.294 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:00.291+0000 7f88473c7700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88400d0420 con 0x7f884015bb20 2023-11-08T19:27:00.295 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:00.291+0000 7f88473c7700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88400cf810 con 0x7f88400b1730 2023-11-08T19:27:00.295 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:00.291+0000 7f88473c7700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88400cf600 con 0x7f88400d1d80 2023-11-08T19:27:00.295 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:00.291+0000 7f8844962700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f88400d1d80 0x7f88400b2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:00.295 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:00.291+0000 7f8845163700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f884015bb20 0x7f88400d1670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:48710/0 (socket says 172.21.15.143:48710) 2023-11-08T19:27:00.296 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:00.291+0000 7f8845163700 1 -- 172.21.15.143:0/1584869506 learned_addr learned my addr 172.21.15.143:0/1584869506 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:27:00.296 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:00.291+0000 7f8844962700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f88400d1d80 0x7f88400b2670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52558/0 (socket says 172.21.15.143:52558) 2023-11-08T19:27:00.296 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:00.291+0000 7f8844962700 1 -- 172.21.15.143:0/1584869506 >> v1:172.21.15.174:6789/0 conn(0x7f88400b1730 legacy=0x7f88400cc4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:00.296 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:00.292+0000 7f8844962700 1 -- 172.21.15.143:0/1584869506 >> v1:172.21.15.143:6789/0 conn(0x7f884015bb20 legacy=0x7f88400d1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:00.296 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:00.292+0000 7f8844962700 1 -- 172.21.15.143:0/1584869506 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88400cfaf0 con 0x7f88400d1d80 2023-11-08T19:27:00.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:00.292+0000 7f8837fff700 1 -- 172.21.15.143:0/1584869506 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3978838435 0 0) 0x7f88400d0420 con 0x7f884015bb20 2023-11-08T19:27:00.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:00.292+0000 7f8844962700 1 --2- 172.21.15.143:0/1584869506 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f88400d1d80 0x7f88400b2670 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f883c009840 tx=0x7f883c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=285088dd5d515388 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:00.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:00.292+0000 7f8837fff700 1 -- 172.21.15.143:0/1584869506 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f883c011830 con 0x7f88400d1d80 2023-11-08T19:27:00.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:00.292+0000 7f8837fff700 1 -- 172.21.15.143:0/1584869506 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f883c011990 con 0x7f88400d1d80 2023-11-08T19:27:00.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:00.292+0000 7f8837fff700 1 -- 172.21.15.143:0/1584869506 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f883c011d20 con 0x7f88400d1d80 2023-11-08T19:27:00.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:00.292+0000 7f88473c7700 1 -- 172.21.15.143:0/1584869506 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f88400d1d80 msgr2=0x7f88400b2670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:00.299 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:00.292+0000 7f88473c7700 1 --2- 172.21.15.143:0/1584869506 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f88400d1d80 0x7f88400b2670 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f883c009840 tx=0x7f883c009b50 comp rx=0 tx=0).stop 2023-11-08T19:27:00.300 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:00.292+0000 7f88473c7700 1 -- 172.21.15.143:0/1584869506 shutdown_connections 2023-11-08T19:27:00.300 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:00.292+0000 7f88473c7700 1 --2- 172.21.15.143:0/1584869506 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f88400d1d80 0x7f88400b2670 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:00.300 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:00.292+0000 7f88473c7700 1 -- 172.21.15.143:0/1584869506 >> 172.21.15.143:0/1584869506 conn(0x7f884008b000 msgr2=0x7f884009ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:00.300 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:00.292+0000 7f88473c7700 1 -- 172.21.15.143:0/1584869506 shutdown_connections 2023-11-08T19:27:00.301 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:00.293+0000 7f88473c7700 1 -- 172.21.15.143:0/1584869506 wait complete. 2023-11-08T19:27:00.304 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:00.294+0000 7f88473c7700 1 Processor -- start 2023-11-08T19:27:00.304 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:00.294+0000 7f88473c7700 1 -- start start 2023-11-08T19:27:00.304 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:00.294+0000 7f88473c7700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f88400b1730 0x7f88401570e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:00.304 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:00.294+0000 7f88473c7700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f88400d1d80 0x7f88400b6d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:00.305 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:00.294+0000 7f88473c7700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f884015bb20 0x7f88401f51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:00.305 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:00.294+0000 7f88473c7700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8840157d10 con 0x7f88400b1730 2023-11-08T19:27:00.305 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:00.294+0000 7f88473c7700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88400b72b0 con 0x7f884015bb20 2023-11-08T19:27:00.305 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:00.294+0000 7f88473c7700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88401f4d60 con 0x7f88400d1d80 2023-11-08T19:27:00.305 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:00.294+0000 7f8845163700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f88400b1730 0x7f88401570e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:00.306 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:00.294+0000 7f8845163700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f88400b1730 0x7f88401570e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52562/0 (socket says 172.21.15.143:52562) 2023-11-08T19:27:00.306 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:00.294+0000 7f8845163700 1 -- 172.21.15.143:0/2543868795 learned_addr learned my addr 172.21.15.143:0/2543868795 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:00.306 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:00.295+0000 7f8844962700 1 --2- 172.21.15.143:0/2543868795 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f88400d1d80 0x7f88400b6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:00.306 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:00.295+0000 7f8845964700 1 --2- 172.21.15.143:0/2543868795 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f884015bb20 0x7f88401f51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:00.306 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:00.295+0000 7f8845163700 1 -- 172.21.15.143:0/2543868795 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f88400d1d80 msgr2=0x7f88400b6d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:00.307 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:00.295+0000 7f8845163700 1 --2- 172.21.15.143:0/2543868795 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f88400d1d80 0x7f88400b6d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:00.307 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:00.295+0000 7f8845163700 1 -- 172.21.15.143:0/2543868795 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f884015bb20 msgr2=0x7f88401f51f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:00.307 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:00.295+0000 7f8845163700 1 --2- 172.21.15.143:0/2543868795 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f884015bb20 0x7f88401f51f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:00.307 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:00.295+0000 7f8845163700 1 -- 172.21.15.143:0/2543868795 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f883c0094a0 con 0x7f88400b1730 2023-11-08T19:27:00.307 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:00.295+0000 7f8844962700 1 --2- 172.21.15.143:0/2543868795 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f88400d1d80 0x7f88400b6d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:27:00.308 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:00.295+0000 7f8845163700 1 --2- 172.21.15.143:0/2543868795 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f88400b1730 0x7f88401570e0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f883000c550 tx=0x7f883000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:00.308 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:00.295+0000 7f8835ffb700 1 -- 172.21.15.143:0/2543868795 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8830004c80 con 0x7f88400b1730 2023-11-08T19:27:00.308 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:00.295+0000 7f8835ffb700 1 -- 172.21.15.143:0/2543868795 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8830004de0 con 0x7f88400b1730 2023-11-08T19:27:00.308 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:00.295+0000 7f88473c7700 1 -- 172.21.15.143:0/2543868795 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88401f5730 con 0x7f88400b1730 2023-11-08T19:27:00.308 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:00.295+0000 7f8835ffb700 1 -- 172.21.15.143:0/2543868795 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8830007310 con 0x7f88400b1730 2023-11-08T19:27:00.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:00.295+0000 7f88473c7700 1 -- 172.21.15.143:0/2543868795 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8840157a30 con 0x7f88400b1730 2023-11-08T19:27:00.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:00.296+0000 7f88473c7700 1 -- 172.21.15.143:0/2543868795 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88401f5040 con 0x7f88400b1730 2023-11-08T19:27:00.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:00.297+0000 7f8835ffb700 1 -- 172.21.15.143:0/2543868795 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8830007470 con 0x7f88400b1730 2023-11-08T19:27:00.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:00.297+0000 7f8835ffb700 1 --2- 172.21.15.143:0/2543868795 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f882406af00 0x7f882406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:00.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:00.297+0000 7f8835ffb700 1 -- 172.21.15.143:0/2543868795 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8830089380 con 0x7f88400b1730 2023-11-08T19:27:00.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:00.297+0000 7f8844962700 1 --2- 172.21.15.143:0/2543868795 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f882406af00 0x7f882406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:00.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:00.298+0000 7f8844962700 1 --2- 172.21.15.143:0/2543868795 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f882406af00 0x7f882406d3c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f88400cdc50 tx=0x7f883c01c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:00.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:00.301+0000 7f8835ffb700 1 -- 172.21.15.143:0/2543868795 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8830054360 con 0x7f88400b1730 2023-11-08T19:27:00.536 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:00.535+0000 7f88473c7700 1 -- 172.21.15.143:0/2543868795 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f884004eee0 con 0x7f88400b1730 2023-11-08T19:27:01.145 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.144+0000 7f8835ffb700 1 -- 172.21.15.143:0/2543868795 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v33) v1 ==== 122+0+0 (secure 0 0 0) 0x7f8830057980 con 0x7f88400b1730 2023-11-08T19:27:01.148 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.147+0000 7f88473c7700 1 -- 172.21.15.143:0/2543868795 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f882406af00 msgr2=0x7f882406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:01.149 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.147+0000 7f88473c7700 1 --2- 172.21.15.143:0/2543868795 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f882406af00 0x7f882406d3c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f88400cdc50 tx=0x7f883c01c040 comp rx=0 tx=0).stop 2023-11-08T19:27:01.149 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.147+0000 7f88473c7700 1 -- 172.21.15.143:0/2543868795 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f88400b1730 msgr2=0x7f88401570e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:01.149 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.147+0000 7f88473c7700 1 --2- 172.21.15.143:0/2543868795 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f88400b1730 0x7f88401570e0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f883000c550 tx=0x7f883000c910 comp rx=0 tx=0).stop 2023-11-08T19:27:01.149 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.147+0000 7f88473c7700 1 -- 172.21.15.143:0/2543868795 shutdown_connections 2023-11-08T19:27:01.149 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.147+0000 7f88473c7700 1 --2- 172.21.15.143:0/2543868795 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f882406af00 0x7f882406d3c0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:01.150 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.147+0000 7f88473c7700 1 --2- 172.21.15.143:0/2543868795 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f884015bb20 0x7f88401f51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:01.150 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.147+0000 7f88473c7700 1 --2- 172.21.15.143:0/2543868795 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f88400b1730 0x7f88401570e0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:01.150 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.147+0000 7f88473c7700 1 --2- 172.21.15.143:0/2543868795 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f88400d1d80 0x7f88400b6d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:01.150 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.147+0000 7f88473c7700 1 -- 172.21.15.143:0/2543868795 >> 172.21.15.143:0/2543868795 conn(0x7f884008b000 msgr2=0x7f8840070870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:01.151 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.147+0000 7f88473c7700 1 -- 172.21.15.143:0/2543868795 shutdown_connections 2023-11-08T19:27:01.151 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.147+0000 7f88473c7700 1 -- 172.21.15.143:0/2543868795 wait complete. 2023-11-08T19:27:01.162 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-08T19:27:01.163 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:27:01.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.283+0000 7f38100c7700 1 Processor -- start 2023-11-08T19:27:01.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.283+0000 7f38100c7700 1 -- start start 2023-11-08T19:27:01.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.284+0000 7f38100c7700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f380806b320 0x7f3808168c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:01.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.284+0000 7f38100c7700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3808161df0 con 0x7f380815dd00 2023-11-08T19:27:01.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.284+0000 7f38100c7700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38081620d0 con 0x7f38081618f0 2023-11-08T19:27:01.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.284+0000 7f38100c7700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38080cad50 con 0x7f380806b320 2023-11-08T19:27:01.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.284+0000 7f380de63700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f380815dd00 0x7f38080cca70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:48724/0 (socket says 172.21.15.143:48724) 2023-11-08T19:27:01.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.284+0000 7f380de63700 1 -- 172.21.15.143:0/4140910933 learned_addr learned my addr 172.21.15.143:0/4140910933 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:27:01.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.284+0000 7f380ce61700 1 -- 172.21.15.143:0/4140910933 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3245724380 0 0) 0x7f3808161df0 con 0x7f380815dd00 2023-11-08T19:27:01.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.284+0000 7f380ce61700 1 -- 172.21.15.143:0/4140910933 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f37f40035b0 con 0x7f380815dd00 2023-11-08T19:27:01.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.284+0000 7f380e664700 1 --2- 172.21.15.143:0/4140910933 >> v2:172.21.15.174:3300/0 conn(0x7f380806b320 0x7f3808168c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:01.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.285+0000 7f380ce61700 1 -- 172.21.15.143:0/4140910933 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 717466261 0 0) 0x7f37f40035b0 con 0x7f380815dd00 2023-11-08T19:27:01.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.285+0000 7f380ce61700 1 -- 172.21.15.143:0/4140910933 >> v2:172.21.15.174:3300/0 conn(0x7f380806b320 msgr2=0x7f3808168c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:01.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.285+0000 7f380ce61700 1 --2- 172.21.15.143:0/4140910933 >> v2:172.21.15.174:3300/0 conn(0x7f380806b320 0x7f3808168c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:01.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.285+0000 7f380ce61700 1 -- 172.21.15.143:0/4140910933 >> v1:172.21.15.174:6789/0 conn(0x7f38081618f0 legacy=0x7f380806ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:01.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.285+0000 7f380ce61700 1 -- 172.21.15.143:0/4140910933 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38081623b0 con 0x7f380815dd00 2023-11-08T19:27:01.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.285+0000 7f380ce61700 1 -- 172.21.15.143:0/4140910933 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4147756303 0 0) 0x7f38040017f0 con 0x7f380815dd00 2023-11-08T19:27:01.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.285+0000 7f380ce61700 1 -- 172.21.15.143:0/4140910933 >> v1:172.21.15.143:6789/0 conn(0x7f380815dd00 legacy=0x7f38080cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:01.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.285+0000 7f380ce61700 1 --2- 172.21.15.143:0/4140910933 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f37f4003a00 0x7f37f4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:01.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.285+0000 7f380ce61700 1 --2- 172.21.15.143:0/4140910933 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f37f4004850 0x7f37f4004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:01.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.285+0000 7f380ce61700 1 --2- 172.21.15.143:0/4140910933 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f37f4005210 0x7f37f400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:01.293 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.285+0000 7f380ce61700 1 -- 172.21.15.143:0/4140910933 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37f400c080 con 0x7f37f4004850 2023-11-08T19:27:01.293 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.285+0000 7f380ce61700 1 -- 172.21.15.143:0/4140910933 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37f400c1f0 con 0x7f37f4005210 2023-11-08T19:27:01.293 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.285+0000 7f380ce61700 1 -- 172.21.15.143:0/4140910933 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37f400c3b0 con 0x7f37f4003a00 2023-11-08T19:27:01.293 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.285+0000 7f380d662700 1 --2- 172.21.15.143:0/4140910933 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f37f4004850 0x7f37f4004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:01.293 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.285+0000 7f380de63700 1 --2- 172.21.15.143:0/4140910933 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f37f4003a00 0x7f37f4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:01.294 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.285+0000 7f380e664700 1 --2- 172.21.15.143:0/4140910933 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f37f4005210 0x7f37f400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:01.294 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.285+0000 7f380d662700 1 -- 172.21.15.143:0/4140910933 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f37f4003a00 msgr2=0x7f37f4003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:01.294 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.285+0000 7f380d662700 1 --2- 172.21.15.143:0/4140910933 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f37f4003a00 0x7f37f4003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:01.294 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.285+0000 7f380d662700 1 -- 172.21.15.143:0/4140910933 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f37f4005210 msgr2=0x7f37f400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:01.294 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.285+0000 7f380d662700 1 --2- 172.21.15.143:0/4140910933 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f37f4005210 0x7f37f400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:01.295 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.285+0000 7f380d662700 1 -- 172.21.15.143:0/4140910933 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37f40035b0 con 0x7f37f4004850 2023-11-08T19:27:01.295 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.285+0000 7f380de63700 1 --2- 172.21.15.143:0/4140910933 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f37f4003a00 0x7f37f4003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:27:01.295 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.285+0000 7f380e664700 1 --2- 172.21.15.143:0/4140910933 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f37f4005210 0x7f37f400ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:27:01.295 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.285+0000 7f380e664700 1 -- 172.21.15.143:0/4140910933 reap_dead start 2023-11-08T19:27:01.295 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.285+0000 7f380d662700 1 --2- 172.21.15.143:0/4140910933 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f37f4004850 0x7f37f4004cd0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f37f80017c0 tx=0x7f37f4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f99a0471f6e01c5 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:01.296 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.285+0000 7f380ce61700 1 -- 172.21.15.143:0/4140910933 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f80052d0 con 0x7f37f4004850 2023-11-08T19:27:01.296 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.285+0000 7f380ce61700 1 -- 172.21.15.143:0/4140910933 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37f8005430 con 0x7f37f4004850 2023-11-08T19:27:01.296 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.285+0000 7f380ce61700 1 -- 172.21.15.143:0/4140910933 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f80038c0 con 0x7f37f4004850 2023-11-08T19:27:01.296 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.286+0000 7f38100c7700 1 -- 172.21.15.143:0/4140910933 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f37f4004850 msgr2=0x7f37f4004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:01.296 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.286+0000 7f38100c7700 1 --2- 172.21.15.143:0/4140910933 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f37f4004850 0x7f37f4004cd0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f37f80017c0 tx=0x7f37f4005880 comp rx=0 tx=0).stop 2023-11-08T19:27:01.296 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.286+0000 7f38100c7700 1 -- 172.21.15.143:0/4140910933 shutdown_connections 2023-11-08T19:27:01.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.286+0000 7f38100c7700 1 --2- 172.21.15.143:0/4140910933 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f37f4004850 0x7f37f4004cd0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:01.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.286+0000 7f38100c7700 1 -- 172.21.15.143:0/4140910933 >> 172.21.15.143:0/4140910933 conn(0x7f3808099ab0 msgr2=0x7f38080d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:01.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.286+0000 7f38100c7700 1 -- 172.21.15.143:0/4140910933 shutdown_connections 2023-11-08T19:27:01.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.286+0000 7f38100c7700 1 -- 172.21.15.143:0/4140910933 wait complete. 2023-11-08T19:27:01.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.287+0000 7f38100c7700 1 Processor -- start 2023-11-08T19:27:01.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.287+0000 7f38100c7700 1 -- start start 2023-11-08T19:27:01.300 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.288+0000 7f38100c7700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f37f4004850 0x7f38080b0a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:01.300 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.288+0000 7f38100c7700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f38081f6c70 0x7f38081474a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:01.301 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.288+0000 7f38100c7700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f38081479e0 0x7f38080b53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:01.301 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.288+0000 7f38100c7700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38081455d0 con 0x7f37f4004850 2023-11-08T19:27:01.301 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.288+0000 7f38100c7700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3808147e90 con 0x7f38081479e0 2023-11-08T19:27:01.301 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.288+0000 7f38100c7700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38080b1110 con 0x7f38081f6c70 2023-11-08T19:27:01.301 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.288+0000 7f380de63700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f37f4004850 0x7f38080b0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:01.302 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.288+0000 7f380de63700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f37f4004850 0x7f38080b0a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52580/0 (socket says 172.21.15.143:52580) 2023-11-08T19:27:01.302 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.288+0000 7f380de63700 1 -- 172.21.15.143:0/3760354051 learned_addr learned my addr 172.21.15.143:0/3760354051 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:01.302 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.288+0000 7f380d662700 1 --2- 172.21.15.143:0/3760354051 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f38081f6c70 0x7f38081474a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:01.302 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.288+0000 7f380e664700 1 --2- 172.21.15.143:0/3760354051 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f38081479e0 0x7f38080b53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:01.302 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.288+0000 7f380de63700 1 -- 172.21.15.143:0/3760354051 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f38081f6c70 msgr2=0x7f38081474a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:01.303 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.288+0000 7f380de63700 1 --2- 172.21.15.143:0/3760354051 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f38081f6c70 0x7f38081474a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:01.303 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.288+0000 7f380de63700 1 -- 172.21.15.143:0/3760354051 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f38081479e0 msgr2=0x7f38080b53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:01.303 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.288+0000 7f380de63700 1 --2- 172.21.15.143:0/3760354051 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f38081479e0 0x7f38080b53d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:01.303 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.288+0000 7f380de63700 1 -- 172.21.15.143:0/3760354051 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f380815e260 con 0x7f37f4004850 2023-11-08T19:27:01.303 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.288+0000 7f380d662700 1 --2- 172.21.15.143:0/3760354051 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f38081f6c70 0x7f38081474a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:27:01.304 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.288+0000 7f380e664700 1 --2- 172.21.15.143:0/3760354051 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f38081479e0 0x7f38080b53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:27:01.304 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.289+0000 7f380de63700 1 --2- 172.21.15.143:0/3760354051 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f37f4004850 0x7f38080b0a80 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f3804001e10 tx=0x7f380400eb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:01.304 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.289+0000 7f37fe7fc700 1 -- 172.21.15.143:0/3760354051 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38040072e0 con 0x7f37f4004850 2023-11-08T19:27:01.304 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.289+0000 7f37fe7fc700 1 -- 172.21.15.143:0/3760354051 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3804007440 con 0x7f37f4004850 2023-11-08T19:27:01.304 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.289+0000 7f37fe7fc700 1 -- 172.21.15.143:0/3760354051 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3804005830 con 0x7f37f4004850 2023-11-08T19:27:01.305 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.289+0000 7f38100c7700 1 -- 172.21.15.143:0/3760354051 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38080b13f0 con 0x7f37f4004850 2023-11-08T19:27:01.305 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.289+0000 7f38100c7700 1 -- 172.21.15.143:0/3760354051 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38080b19b0 con 0x7f37f4004850 2023-11-08T19:27:01.305 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.290+0000 7f38100c7700 1 -- 172.21.15.143:0/3760354051 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f380804eee0 con 0x7f37f4004850 2023-11-08T19:27:01.305 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.290+0000 7f37fe7fc700 1 -- 172.21.15.143:0/3760354051 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3804005990 con 0x7f37f4004850 2023-11-08T19:27:01.305 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.290+0000 7f37fe7fc700 1 --2- 172.21.15.143:0/3760354051 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f37ec06af00 0x7f37ec06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:01.305 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.291+0000 7f37fe7fc700 1 -- 172.21.15.143:0/3760354051 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3804089590 con 0x7f37f4004850 2023-11-08T19:27:01.306 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.291+0000 7f380d662700 1 --2- 172.21.15.143:0/3760354051 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f37ec06af00 0x7f37ec06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:01.306 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.291+0000 7f380d662700 1 --2- 172.21.15.143:0/3760354051 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f37ec06af00 0x7f37ec06d3c0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f37f8015040 tx=0x7f37f8009030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:01.306 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.294+0000 7f37fe7fc700 1 -- 172.21.15.143:0/3760354051 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f38040544f0 con 0x7f37f4004850 2023-11-08T19:27:01.527 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.525+0000 7f38100c7700 1 -- 172.21.15.143:0/3760354051 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f38080b30a0 con 0x7f37f4004850 2023-11-08T19:27:01.528 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.526+0000 7f37fe7fc700 1 -- 172.21.15.143:0/3760354051 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+4923 (secure 0 0 0) 0x7f3804057b10 con 0x7f37f4004850 2023-11-08T19:27:01.529 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:27:01.529 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":33,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5231,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6836/4083537882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":4083537882},{"type":"v1","addr":"172.21.15.174:6836","nonce":4083537882}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":5234,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/494923804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6835","nonce":494923804},{"type":"v1","addr":"172.21.15.174:6837","nonce":494923804}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":6009,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6834/3383638390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":3383638390},{"type":"v1","addr":"172.21.15.143:6834","nonce":3383638390}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T19:26:57.522785+0000","modified":"2023-11-08T19:27:00.536968+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6012},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6012":{"gid":6012,"name":"c","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.143:6835/2613282947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":2613282947},{"type":"v1","addr":"172.21.15.143:6835","nonce":2613282947}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-08T19:27:01.530 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.528+0000 7f38100c7700 1 -- 172.21.15.143:0/3760354051 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f37ec06af00 msgr2=0x7f37ec06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:01.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.528+0000 7f38100c7700 1 --2- 172.21.15.143:0/3760354051 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f37ec06af00 0x7f37ec06d3c0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f37f8015040 tx=0x7f37f8009030 comp rx=0 tx=0).stop 2023-11-08T19:27:01.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.528+0000 7f38100c7700 1 -- 172.21.15.143:0/3760354051 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f37f4004850 msgr2=0x7f38080b0a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:01.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.528+0000 7f38100c7700 1 --2- 172.21.15.143:0/3760354051 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f37f4004850 0x7f38080b0a80 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f3804001e10 tx=0x7f380400eb10 comp rx=0 tx=0).stop 2023-11-08T19:27:01.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.528+0000 7f38100c7700 1 -- 172.21.15.143:0/3760354051 shutdown_connections 2023-11-08T19:27:01.532 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.528+0000 7f38100c7700 1 --2- 172.21.15.143:0/3760354051 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f37ec06af00 0x7f37ec06d3c0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:01.532 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.528+0000 7f38100c7700 1 --2- 172.21.15.143:0/3760354051 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f38081479e0 0x7f38080b53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:01.532 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.528+0000 7f38100c7700 1 --2- 172.21.15.143:0/3760354051 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f37f4004850 0x7f38080b0a80 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:01.532 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.528+0000 7f38100c7700 1 --2- 172.21.15.143:0/3760354051 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f38081f6c70 0x7f38081474a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:01.533 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.528+0000 7f38100c7700 1 -- 172.21.15.143:0/3760354051 >> 172.21.15.143:0/3760354051 conn(0x7f3808099ab0 msgr2=0x7f38080ad410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:01.533 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.528+0000 7f38100c7700 1 -- 172.21.15.143:0/3760354051 shutdown_connections 2023-11-08T19:27:01.533 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.528+0000 7f38100c7700 1 -- 172.21.15.143:0/3760354051 wait complete. 2023-11-08T19:27:01.533 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 33 2023-11-08T19:27:01.542 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:27:01.663 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.661+0000 7f1c9d33e700 1 Processor -- start 2023-11-08T19:27:01.663 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.661+0000 7f1c9d33e700 1 -- start start 2023-11-08T19:27:01.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.661+0000 7f1c9d33e700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f1c980cf090 0x7f1c980d1f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:01.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.661+0000 7f1c9d33e700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c980d3200 con 0x7f1c980d3690 2023-11-08T19:27:01.668 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.661+0000 7f1c9d33e700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c9815e060 con 0x7f1c9815dc80 2023-11-08T19:27:01.668 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.661+0000 7f1c9d33e700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c980d25f0 con 0x7f1c980cf090 2023-11-08T19:27:01.668 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.661+0000 7f1c977fe700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f1c980d3690 0x7f1c980b6ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:48746/0 (socket says 172.21.15.143:48746) 2023-11-08T19:27:01.668 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.661+0000 7f1c977fe700 1 -- 172.21.15.143:0/2045423162 learned_addr learned my addr 172.21.15.143:0/2045423162 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:27:01.668 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.662+0000 7f1c967fc700 1 --2- 172.21.15.143:0/2045423162 >> v2:172.21.15.174:3300/0 conn(0x7f1c980cf090 0x7f1c980d1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:01.669 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.662+0000 7f1c95ffb700 1 -- 172.21.15.143:0/2045423162 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2182099923 0 0) 0x7f1c980d3200 con 0x7f1c980d3690 2023-11-08T19:27:01.669 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.662+0000 7f1c95ffb700 1 -- 172.21.15.143:0/2045423162 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1c800035b0 con 0x7f1c980d3690 2023-11-08T19:27:01.669 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.662+0000 7f1c95ffb700 1 -- 172.21.15.143:0/2045423162 <== mon.1 v1:172.21.15.174:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3446383274 0 0) 0x7f1c9815e060 con 0x7f1c9815dc80 2023-11-08T19:27:01.669 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.662+0000 7f1c95ffb700 1 -- 172.21.15.143:0/2045423162 --> v1:172.21.15.174:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1c980d3200 con 0x7f1c9815dc80 2023-11-08T19:27:01.669 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.662+0000 7f1c95ffb700 1 -- 172.21.15.143:0/2045423162 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1346138258 0 0) 0x7f1c800035b0 con 0x7f1c980d3690 2023-11-08T19:27:01.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.662+0000 7f1c95ffb700 1 -- 172.21.15.143:0/2045423162 >> v2:172.21.15.174:3300/0 conn(0x7f1c980cf090 msgr2=0x7f1c980d1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:01.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.662+0000 7f1c95ffb700 1 --2- 172.21.15.143:0/2045423162 >> v2:172.21.15.174:3300/0 conn(0x7f1c980cf090 0x7f1c980d1f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:01.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.662+0000 7f1c95ffb700 1 -- 172.21.15.143:0/2045423162 >> v1:172.21.15.174:6789/0 conn(0x7f1c9815dc80 legacy=0x7f1c980ce980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:01.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.662+0000 7f1c95ffb700 1 -- 172.21.15.143:0/2045423162 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c980d28d0 con 0x7f1c980d3690 2023-11-08T19:27:01.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.662+0000 7f1c95ffb700 1 -- 172.21.15.143:0/2045423162 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4147756303 0 0) 0x7f1c900019f0 con 0x7f1c980d3690 2023-11-08T19:27:01.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.662+0000 7f1c95ffb700 1 -- 172.21.15.143:0/2045423162 >> v1:172.21.15.143:6789/0 conn(0x7f1c980d3690 legacy=0x7f1c980b6ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:01.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.662+0000 7f1c95ffb700 1 --2- 172.21.15.143:0/2045423162 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1c80003be0 0x7f1c80003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:01.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.662+0000 7f1c95ffb700 1 --2- 172.21.15.143:0/2045423162 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1c80004850 0x7f1c80004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:01.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.662+0000 7f1c95ffb700 1 --2- 172.21.15.143:0/2045423162 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1c80005260 0x7f1c8000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:01.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.662+0000 7f1c95ffb700 1 -- 172.21.15.143:0/2045423162 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c980d25f0 con 0x7f1c80004850 2023-11-08T19:27:01.672 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.662+0000 7f1c95ffb700 1 -- 172.21.15.143:0/2045423162 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c8000c190 con 0x7f1c80005260 2023-11-08T19:27:01.672 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.662+0000 7f1c95ffb700 1 -- 172.21.15.143:0/2045423162 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c8000c350 con 0x7f1c80003be0 2023-11-08T19:27:01.672 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.662+0000 7f1c967fc700 1 --2- 172.21.15.143:0/2045423162 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1c80004850 0x7f1c80004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:01.672 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.662+0000 7f1c977fe700 1 --2- 172.21.15.143:0/2045423162 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1c80005260 0x7f1c8000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:01.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.662+0000 7f1c96ffd700 1 --2- 172.21.15.143:0/2045423162 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1c80003be0 0x7f1c80003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:01.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.662+0000 7f1c967fc700 1 -- 172.21.15.143:0/2045423162 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1c80003be0 msgr2=0x7f1c80003fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:01.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.662+0000 7f1c967fc700 1 --2- 172.21.15.143:0/2045423162 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1c80003be0 0x7f1c80003fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:01.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.662+0000 7f1c967fc700 1 -- 172.21.15.143:0/2045423162 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1c80005260 msgr2=0x7f1c8000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:01.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.662+0000 7f1c967fc700 1 --2- 172.21.15.143:0/2045423162 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1c80005260 0x7f1c8000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:01.674 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.662+0000 7f1c977fe700 1 -- 172.21.15.143:0/2045423162 reap_dead start 2023-11-08T19:27:01.674 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.662+0000 7f1c967fc700 1 -- 172.21.15.143:0/2045423162 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c800035b0 con 0x7f1c80004850 2023-11-08T19:27:01.674 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.662+0000 7f1c96ffd700 1 --2- 172.21.15.143:0/2045423162 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1c80003be0 0x7f1c80003fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:27:01.674 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.663+0000 7f1c967fc700 1 --2- 172.21.15.143:0/2045423162 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1c80004850 0x7f1c80004cd0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f1c88009b60 tx=0x7f1c80005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc42fbcf523b5a7e server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:01.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.663+0000 7f1c95ffb700 1 -- 172.21.15.143:0/2045423162 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c8800e6c0 con 0x7f1c80004850 2023-11-08T19:27:01.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.663+0000 7f1c95ffb700 1 -- 172.21.15.143:0/2045423162 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c8800e820 con 0x7f1c80004850 2023-11-08T19:27:01.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.663+0000 7f1c95ffb700 1 -- 172.21.15.143:0/2045423162 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c8800eb90 con 0x7f1c80004850 2023-11-08T19:27:01.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.663+0000 7f1c9d33e700 1 -- 172.21.15.143:0/2045423162 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1c80004850 msgr2=0x7f1c80004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:01.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.663+0000 7f1c9d33e700 1 --2- 172.21.15.143:0/2045423162 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1c80004850 0x7f1c80004cd0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f1c88009b60 tx=0x7f1c80005880 comp rx=0 tx=0).stop 2023-11-08T19:27:01.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.663+0000 7f1c9d33e700 1 -- 172.21.15.143:0/2045423162 shutdown_connections 2023-11-08T19:27:01.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.663+0000 7f1c9d33e700 1 --2- 172.21.15.143:0/2045423162 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1c80004850 0x7f1c80004cd0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:01.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.663+0000 7f1c9d33e700 1 -- 172.21.15.143:0/2045423162 >> 172.21.15.143:0/2045423162 conn(0x7f1c980668e0 msgr2=0x7f1c98066cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:01.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.663+0000 7f1c9d33e700 1 -- 172.21.15.143:0/2045423162 shutdown_connections 2023-11-08T19:27:01.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.663+0000 7f1c9d33e700 1 -- 172.21.15.143:0/2045423162 wait complete. 2023-11-08T19:27:01.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.665+0000 7f1c9d33e700 1 Processor -- start 2023-11-08T19:27:01.677 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.665+0000 7f1c9d33e700 1 -- start start 2023-11-08T19:27:01.679 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.666+0000 7f1c9d33e700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1c80004850 0x7f1c98150080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:01.679 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.666+0000 7f1c9d33e700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1c981f35e0 0x7f1c980b4490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:01.680 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.666+0000 7f1c9d33e700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1c981f3a60 0x7f1c980c8610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:01.680 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.666+0000 7f1c9d33e700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c9815f810 con 0x7f1c981f3a60 2023-11-08T19:27:01.680 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.666+0000 7f1c9d33e700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c9815faf0 con 0x7f1c80004850 2023-11-08T19:27:01.680 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.666+0000 7f1c9d33e700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c9815fdd0 con 0x7f1c981f35e0 2023-11-08T19:27:01.681 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.666+0000 7f1c977fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1c981f3a60 0x7f1c980c8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:01.681 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.666+0000 7f1c977fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1c981f3a60 0x7f1c980c8610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52604/0 (socket says 172.21.15.143:52604) 2023-11-08T19:27:01.681 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.666+0000 7f1c977fe700 1 -- 172.21.15.143:0/1652959263 learned_addr learned my addr 172.21.15.143:0/1652959263 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:01.681 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.666+0000 7f1c967fc700 1 --2- 172.21.15.143:0/1652959263 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1c981f35e0 0x7f1c980b4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:01.681 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.666+0000 7f1c96ffd700 1 --2- 172.21.15.143:0/1652959263 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1c80004850 0x7f1c98150080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:01.682 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.666+0000 7f1c977fe700 1 -- 172.21.15.143:0/1652959263 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1c981f35e0 msgr2=0x7f1c980b4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:01.682 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.666+0000 7f1c977fe700 1 --2- 172.21.15.143:0/1652959263 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1c981f35e0 0x7f1c980b4490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:01.682 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.666+0000 7f1c977fe700 1 -- 172.21.15.143:0/1652959263 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1c80004850 msgr2=0x7f1c98150080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:01.682 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.666+0000 7f1c977fe700 1 --2- 172.21.15.143:0/1652959263 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1c80004850 0x7f1c98150080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:01.682 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.666+0000 7f1c977fe700 1 -- 172.21.15.143:0/1652959263 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c981600b0 con 0x7f1c981f3a60 2023-11-08T19:27:01.683 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.667+0000 7f1c977fe700 1 --2- 172.21.15.143:0/1652959263 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1c981f3a60 0x7f1c980c8610 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f1c90002230 tx=0x7f1c9000a510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:01.683 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.667+0000 7f1c7f7fe700 1 -- 172.21.15.143:0/1652959263 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c9000aba0 con 0x7f1c981f3a60 2023-11-08T19:27:01.683 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.667+0000 7f1c7f7fe700 1 -- 172.21.15.143:0/1652959263 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c9000ad00 con 0x7f1c981f3a60 2023-11-08T19:27:01.683 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.667+0000 7f1c9d33e700 1 -- 172.21.15.143:0/1652959263 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c981505c0 con 0x7f1c981f3a60 2023-11-08T19:27:01.683 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.667+0000 7f1c7f7fe700 1 -- 172.21.15.143:0/1652959263 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c90013630 con 0x7f1c981f3a60 2023-11-08T19:27:01.684 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.667+0000 7f1c9d33e700 1 -- 172.21.15.143:0/1652959263 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c980b4f50 con 0x7f1c981f3a60 2023-11-08T19:27:01.684 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.668+0000 7f1c9d33e700 1 -- 172.21.15.143:0/1652959263 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c9804eee0 con 0x7f1c981f3a60 2023-11-08T19:27:01.684 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.668+0000 7f1c7f7fe700 1 -- 172.21.15.143:0/1652959263 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1c90013790 con 0x7f1c981f3a60 2023-11-08T19:27:01.684 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.668+0000 7f1c7f7fe700 1 --2- 172.21.15.143:0/1652959263 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f1c8406af00 0x7f1c8406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:01.684 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.669+0000 7f1c7f7fe700 1 -- 172.21.15.143:0/1652959263 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1c9008a200 con 0x7f1c981f3a60 2023-11-08T19:27:01.685 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.669+0000 7f1c96ffd700 1 --2- 172.21.15.143:0/1652959263 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f1c8406af00 0x7f1c8406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:01.685 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.669+0000 7f1c96ffd700 1 --2- 172.21.15.143:0/1652959263 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f1c8406af00 0x7f1c8406d3c0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f1c980692c0 tx=0x7f1c8c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:01.685 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.673+0000 7f1c7f7fe700 1 -- 172.21.15.143:0/1652959263 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1c900550b0 con 0x7f1c981f3a60 2023-11-08T19:27:01.867 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.865+0000 7f1c9d33e700 1 -- 172.21.15.143:0/1652959263 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1c9815f950 con 0x7f1c981f3a60 2023-11-08T19:27:01.868 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.866+0000 7f1c7f7fe700 1 -- 172.21.15.143:0/1652959263 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v62) v1 ==== 74+0+18312 (secure 0 0 0) 0x7f1c9815f950 con 0x7f1c981f3a60 2023-11-08T19:27:01.869 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:27:01.869 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":62,"fsid":"968483af-77fc-4068-ad7c-5c2a1a4613d7","created":"2023-11-08T19:24:32.826484+0000","modified":"2023-11-08T19:26:58.579564+0000","last_up_change":"2023-11-08T19:24:43.573880+0000","last_in_change":"2023-11-08T19:24:35.314525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:44.551530+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"snap-test-pool","create_time":"2023-11-08T19:26:02.874384+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":36,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-11-08T19:26:55.754227+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-11-08T19:26:56.486110+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bae98ec1-d94e-403d-a220-a522a53ea6f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6809","nonce":1487830566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6811","nonce":1487830566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6815","nonce":1487830566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6813","nonce":1487830566}]},"public_addr":"172.21.15.143:6809/1487830566","cluster_addr":"172.21.15.143:6811/1487830566","heartbeat_back_addr":"172.21.15.143:6815/1487830566","heartbeat_front_addr":"172.21.15.143:6813/1487830566","state":["exists","up"]},{"osd":1,"uuid":"5cb6538a-1ccb-4322-8633-09f66af0b7e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6801","nonce":787341397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6803","nonce":787341397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6807","nonce":787341397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6805","nonce":787341397}]},"public_addr":"172.21.15.143:6801/787341397","cluster_addr":"172.21.15.143:6803/787341397","heartbeat_back_addr":"172.21.15.143:6807/787341397","heartbeat_front_addr":"172.21.15.143:6805/787341397","state":["exists","up"]},{"osd":2,"uuid":"cdbf522c-60ef-4d60-9653-8dcb529c101a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6822","nonce":2301325597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6826","nonce":2301325597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6831","nonce":2301325597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6829","nonce":2301325597}]},"public_addr":"172.21.15.143:6822/2301325597","cluster_addr":"172.21.15.143:6826/2301325597","heartbeat_back_addr":"172.21.15.143:6831/2301325597","heartbeat_front_addr":"172.21.15.143:6829/2301325597","state":["exists","up"]},{"osd":3,"uuid":"4e2783cf-ebb9-4dd8-8245-bc1d17ce4325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6817","nonce":2021504428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6819","nonce":2021504428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6827","nonce":2021504428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6823","nonce":2021504428}]},"public_addr":"172.21.15.143:6817/2021504428","cluster_addr":"172.21.15.143:6819/2021504428","heartbeat_back_addr":"172.21.15.143:6827/2021504428","heartbeat_front_addr":"172.21.15.143:6823/2021504428","state":["exists","up"]},{"osd":4,"uuid":"691256fa-9b77-49b9-a346-7ae1adf0ca80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6801","nonce":1993380595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6803","nonce":1993380595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6807","nonce":1993380595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6805","nonce":1993380595}]},"public_addr":"172.21.15.174:6801/1993380595","cluster_addr":"172.21.15.174:6803/1993380595","heartbeat_back_addr":"172.21.15.174:6807/1993380595","heartbeat_front_addr":"172.21.15.174:6805/1993380595","state":["exists","up"]},{"osd":5,"uuid":"756fc352-48c9-46ce-b52e-0541fe6f8525","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6809","nonce":1777923303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6811","nonce":1777923303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6815","nonce":1777923303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6813","nonce":1777923303}]},"public_addr":"172.21.15.174:6809/1777923303","cluster_addr":"172.21.15.174:6811/1777923303","heartbeat_back_addr":"172.21.15.174:6815/1777923303","heartbeat_front_addr":"172.21.15.174:6813/1777923303","state":["exists","up"]},{"osd":6,"uuid":"fab2ac2b-ed97-4cde-9825-b480eabf8b8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6825","nonce":2755130316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6827","nonce":2755130316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6831","nonce":2755130316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6829","nonce":2755130316}]},"public_addr":"172.21.15.174:6825/2755130316","cluster_addr":"172.21.15.174:6827/2755130316","heartbeat_back_addr":"172.21.15.174:6831/2755130316","heartbeat_front_addr":"172.21.15.174:6829/2755130316","state":["exists","up"]},{"osd":7,"uuid":"3de9b797-2ab6-4d41-a863-7e7d638fd703","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6817","nonce":2847141411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6819","nonce":2847141411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6823","nonce":2847141411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6821","nonce":2847141411}]},"public_addr":"172.21.15.174:6817/2847141411","cluster_addr":"172.21.15.174:6819/2847141411","heartbeat_back_addr":"172.21.15.174:6823/2847141411","heartbeat_front_addr":"172.21.15.174:6821/2847141411","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.477076+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.496890+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.539305+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.454673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.538881+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.521294+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.520712+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.540404+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:27:01.870 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.868+0000 7f1c9d33e700 1 -- 172.21.15.143:0/1652959263 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f1c8406af00 msgr2=0x7f1c8406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:01.871 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.868+0000 7f1c9d33e700 1 --2- 172.21.15.143:0/1652959263 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f1c8406af00 0x7f1c8406d3c0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f1c980692c0 tx=0x7f1c8c009040 comp rx=0 tx=0).stop 2023-11-08T19:27:01.871 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.868+0000 7f1c9d33e700 1 -- 172.21.15.143:0/1652959263 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1c981f3a60 msgr2=0x7f1c980c8610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:01.871 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.868+0000 7f1c9d33e700 1 --2- 172.21.15.143:0/1652959263 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1c981f3a60 0x7f1c980c8610 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f1c90002230 tx=0x7f1c9000a510 comp rx=0 tx=0).stop 2023-11-08T19:27:01.871 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.868+0000 7f1c9d33e700 1 -- 172.21.15.143:0/1652959263 shutdown_connections 2023-11-08T19:27:01.872 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.868+0000 7f1c9d33e700 1 --2- 172.21.15.143:0/1652959263 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f1c8406af00 0x7f1c8406d3c0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:01.872 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.868+0000 7f1c9d33e700 1 --2- 172.21.15.143:0/1652959263 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1c981f3a60 0x7f1c980c8610 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:01.872 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.868+0000 7f1c9d33e700 1 --2- 172.21.15.143:0/1652959263 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1c80004850 0x7f1c98150080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:01.872 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.868+0000 7f1c9d33e700 1 --2- 172.21.15.143:0/1652959263 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1c981f35e0 0x7f1c980b4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:01.872 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.868+0000 7f1c9d33e700 1 -- 172.21.15.143:0/1652959263 >> 172.21.15.143:0/1652959263 conn(0x7f1c980668e0 msgr2=0x7f1c980d0480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:01.873 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.868+0000 7f1c9d33e700 1 -- 172.21.15.143:0/1652959263 shutdown_connections 2023-11-08T19:27:01.873 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:01.868+0000 7f1c9d33e700 1 -- 172.21.15.143:0/1652959263 wait complete. 2023-11-08T19:27:01.883 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-08T19:27:01.884 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-08T19:27:02.004 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.001+0000 7f601e55f700 1 Processor -- start 2023-11-08T19:27:02.004 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.001+0000 7f601e55f700 1 -- start start 2023-11-08T19:27:02.006 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.002+0000 7f601e55f700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f601815aa30 0x7f60180cf510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:02.007 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.002+0000 7f601e55f700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f60180b2210 0x7f60180b2690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:02.007 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.002+0000 7f601e55f700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f60180cbe20 0x7f60180cc270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:02.007 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.002+0000 7f601e55f700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60180b2bd0 con 0x7f60180b2210 2023-11-08T19:27:02.007 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.002+0000 7f601e55f700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60180cf080 con 0x7f60180cbe20 2023-11-08T19:27:02.007 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.002+0000 7f601e55f700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60180ce470 con 0x7f601815aa30 2023-11-08T19:27:02.008 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.002+0000 7f60177fe700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f60180b2210 0x7f60180b2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:02.008 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.002+0000 7f60177fe700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f60180b2210 0x7f60180b2690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52610/0 (socket says 172.21.15.143:52610) 2023-11-08T19:27:02.008 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.002+0000 7f60177fe700 1 -- 172.21.15.143:0/2508027322 learned_addr learned my addr 172.21.15.143:0/2508027322 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:02.008 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.003+0000 7f6017fff700 1 --2- 172.21.15.143:0/2508027322 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f601815aa30 0x7f60180cf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:02.008 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.003+0000 7f601cafc700 1 --2- 172.21.15.143:0/2508027322 >> v2:172.21.15.174:3300/0 conn(0x7f60180cbe20 0x7f60180cc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:02.009 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.003+0000 7f60177fe700 1 -- 172.21.15.143:0/2508027322 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f601815aa30 msgr2=0x7f60180cf510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:02.009 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.003+0000 7f60177fe700 1 --2- 172.21.15.143:0/2508027322 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f601815aa30 0x7f60180cf510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:02.009 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.003+0000 7f60177fe700 1 -- 172.21.15.143:0/2508027322 >> v2:172.21.15.174:3300/0 conn(0x7f60180cbe20 msgr2=0x7f60180cc270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:02.009 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.003+0000 7f60177fe700 1 --2- 172.21.15.143:0/2508027322 >> v2:172.21.15.174:3300/0 conn(0x7f60180cbe20 0x7f60180cc270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:02.010 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.003+0000 7f60177fe700 1 -- 172.21.15.143:0/2508027322 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60180ce750 con 0x7f60180b2210 2023-11-08T19:27:02.010 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.003+0000 7f601cafc700 1 --2- 172.21.15.143:0/2508027322 >> v2:172.21.15.174:3300/0 conn(0x7f60180cbe20 0x7f60180cc270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:27:02.010 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.003+0000 7f6017fff700 1 --2- 172.21.15.143:0/2508027322 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f601815aa30 0x7f60180cf510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:27:02.010 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.003+0000 7f60177fe700 1 --2- 172.21.15.143:0/2508027322 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f60180b2210 0x7f60180b2690 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f60180ccf00 tx=0x7f6000009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=906692ddbad02c30 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:02.010 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.003+0000 7f6016ffd700 1 -- 172.21.15.143:0/2508027322 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f600000b870 con 0x7f60180b2210 2023-11-08T19:27:02.011 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.003+0000 7f6016ffd700 1 -- 172.21.15.143:0/2508027322 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f600000b9d0 con 0x7f60180b2210 2023-11-08T19:27:02.011 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.003+0000 7f6016ffd700 1 -- 172.21.15.143:0/2508027322 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f600000bd60 con 0x7f60180b2210 2023-11-08T19:27:02.011 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.003+0000 7f601e55f700 1 -- 172.21.15.143:0/2508027322 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f60180b2210 msgr2=0x7f60180b2690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:02.011 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.003+0000 7f601e55f700 1 --2- 172.21.15.143:0/2508027322 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f60180b2210 0x7f60180b2690 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f60180ccf00 tx=0x7f6000009bd0 comp rx=0 tx=0).stop 2023-11-08T19:27:02.011 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.004+0000 7f601e55f700 1 -- 172.21.15.143:0/2508027322 shutdown_connections 2023-11-08T19:27:02.012 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.004+0000 7f601e55f700 1 --2- 172.21.15.143:0/2508027322 >> v2:172.21.15.174:3300/0 conn(0x7f60180cbe20 0x7f60180cc270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:02.012 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.004+0000 7f601e55f700 1 --2- 172.21.15.143:0/2508027322 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f601815aa30 0x7f60180cf510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:02.012 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.004+0000 7f601e55f700 1 --2- 172.21.15.143:0/2508027322 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f60180b2210 0x7f60180b2690 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:02.012 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.004+0000 7f601e55f700 1 -- 172.21.15.143:0/2508027322 >> 172.21.15.143:0/2508027322 conn(0x7f6018099ab0 msgr2=0x7f60180d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:02.012 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.004+0000 7f601e55f700 1 -- 172.21.15.143:0/2508027322 shutdown_connections 2023-11-08T19:27:02.013 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.004+0000 7f601e55f700 1 -- 172.21.15.143:0/2508027322 wait complete. 2023-11-08T19:27:02.015 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.005+0000 7f601e55f700 1 Processor -- start 2023-11-08T19:27:02.016 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.005+0000 7f601e55f700 1 -- start start 2023-11-08T19:27:02.016 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.006+0000 7f601e55f700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f60180b2210 0x7f601814e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:02.016 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.006+0000 7f601e55f700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f60180cbe20 0x7f601814ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:02.016 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.006+0000 7f601e55f700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f601815aa30 0x7f601815b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:02.017 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.006+0000 7f601e55f700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f601814f180 con 0x7f60180cbe20 2023-11-08T19:27:02.017 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.006+0000 7f601e55f700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60180ae9b0 con 0x7f60180b2210 2023-11-08T19:27:02.017 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.006+0000 7f601e55f700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f601815c610 con 0x7f601815aa30 2023-11-08T19:27:02.017 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.006+0000 7f60177fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f60180cbe20 0x7f601814ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:02.017 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.006+0000 7f60177fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f60180cbe20 0x7f601814ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52616/0 (socket says 172.21.15.143:52616) 2023-11-08T19:27:02.018 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.006+0000 7f60177fe700 1 -- 172.21.15.143:0/2486148535 learned_addr learned my addr 172.21.15.143:0/2486148535 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:02.018 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.006+0000 7f60177fe700 1 -- 172.21.15.143:0/2486148535 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f601815aa30 msgr2=0x7f601815b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:02.018 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.006+0000 7f601cafc700 1 --2- 172.21.15.143:0/2486148535 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f601815aa30 0x7f601815b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:02.018 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.006+0000 7f6017fff700 1 --2- 172.21.15.143:0/2486148535 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f60180b2210 0x7f601814e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:02.018 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.006+0000 7f60177fe700 1 --2- 172.21.15.143:0/2486148535 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f601815aa30 0x7f601815b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:02.019 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.006+0000 7f60177fe700 1 -- 172.21.15.143:0/2486148535 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f60180b2210 msgr2=0x7f601814e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:02.019 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.006+0000 7f60177fe700 1 --2- 172.21.15.143:0/2486148535 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f60180b2210 0x7f601814e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:02.019 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.006+0000 7f60177fe700 1 -- 172.21.15.143:0/2486148535 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6000009550 con 0x7f60180cbe20 2023-11-08T19:27:02.019 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.006+0000 7f60177fe700 1 --2- 172.21.15.143:0/2486148535 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f60180cbe20 0x7f601814ec40 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f6000000c00 tx=0x7f6000004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:02.019 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.006+0000 7f6014ff9700 1 -- 172.21.15.143:0/2486148535 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6000004540 con 0x7f60180cbe20 2023-11-08T19:27:02.020 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.006+0000 7f6014ff9700 1 -- 172.21.15.143:0/2486148535 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60000036a0 con 0x7f60180cbe20 2023-11-08T19:27:02.020 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.006+0000 7f601e55f700 1 -- 172.21.15.143:0/2486148535 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60180aec00 con 0x7f60180cbe20 2023-11-08T19:27:02.020 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.006+0000 7f6014ff9700 1 -- 172.21.15.143:0/2486148535 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6000003a50 con 0x7f60180cbe20 2023-11-08T19:27:02.020 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.006+0000 7f601e55f700 1 -- 172.21.15.143:0/2486148535 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60180af1c0 con 0x7f60180cbe20 2023-11-08T19:27:02.020 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.007+0000 7f601e55f700 1 -- 172.21.15.143:0/2486148535 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f601804eee0 con 0x7f60180cbe20 2023-11-08T19:27:02.021 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.008+0000 7f6014ff9700 1 -- 172.21.15.143:0/2486148535 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6000021070 con 0x7f60180cbe20 2023-11-08T19:27:02.021 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.008+0000 7f6014ff9700 1 --2- 172.21.15.143:0/2486148535 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f5ff806af00 0x7f5ff806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:02.021 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.008+0000 7f6014ff9700 1 -- 172.21.15.143:0/2486148535 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f600008adc0 con 0x7f60180cbe20 2023-11-08T19:27:02.021 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.008+0000 7f6017fff700 1 --2- 172.21.15.143:0/2486148535 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f5ff806af00 0x7f5ff806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:02.021 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.009+0000 7f6017fff700 1 --2- 172.21.15.143:0/2486148535 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f5ff806af00 0x7f5ff806d3c0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f6018167580 tx=0x7f6008008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:02.021 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.012+0000 7f6014ff9700 1 -- 172.21.15.143:0/2486148535 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6000055da0 con 0x7f60180cbe20 2023-11-08T19:27:02.202 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.200+0000 7f601e55f700 1 -- 172.21.15.143:0/2486148535 --> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f60180ced10 con 0x7f5ff806af00 2023-11-08T19:27:02.213 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.211+0000 7f6014ff9700 1 -- 172.21.15.143:0/2486148535 <== mgr.4099 v2:172.21.15.174:6832/4037280178 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+370633 (secure 0 0 0) 0x7f6008014460 con 0x7f5ff806af00 2023-11-08T19:27:02.217 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:27:02.220 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.215+0000 7f601e55f700 1 -- 172.21.15.143:0/2486148535 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f5ff806af00 msgr2=0x7f5ff806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:02.221 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.215+0000 7f601e55f700 1 --2- 172.21.15.143:0/2486148535 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f5ff806af00 0x7f5ff806d3c0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f6018167580 tx=0x7f6008008040 comp rx=0 tx=0).stop 2023-11-08T19:27:02.221 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.215+0000 7f601e55f700 1 -- 172.21.15.143:0/2486148535 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f60180cbe20 msgr2=0x7f601814ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:02.221 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.215+0000 7f601e55f700 1 --2- 172.21.15.143:0/2486148535 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f60180cbe20 0x7f601814ec40 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f6000000c00 tx=0x7f6000004750 comp rx=0 tx=0).stop 2023-11-08T19:27:02.221 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.215+0000 7f601e55f700 1 -- 172.21.15.143:0/2486148535 shutdown_connections 2023-11-08T19:27:02.221 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.215+0000 7f601e55f700 1 --2- 172.21.15.143:0/2486148535 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f5ff806af00 0x7f5ff806d3c0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:02.222 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.215+0000 7f601e55f700 1 --2- 172.21.15.143:0/2486148535 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f601815aa30 0x7f601815b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:02.222 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.215+0000 7f601e55f700 1 --2- 172.21.15.143:0/2486148535 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f60180b2210 0x7f601814e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:02.222 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.215+0000 7f601e55f700 1 --2- 172.21.15.143:0/2486148535 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f60180cbe20 0x7f601814ec40 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:02.222 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.215+0000 7f601e55f700 1 -- 172.21.15.143:0/2486148535 >> 172.21.15.143:0/2486148535 conn(0x7f6018099ab0 msgr2=0x7f6018151810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:02.222 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.215+0000 7f601e55f700 1 -- 172.21.15.143:0/2486148535 shutdown_connections 2023-11-08T19:27:02.223 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.215+0000 7f601e55f700 1 -- 172.21.15.143:0/2486148535 wait complete. 2023-11-08T19:27:02.223 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2023-11-08T19:27:02.229 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":123,"stamp":"2023-11-08T19:27:00.542859+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":258,"acting":258,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":205,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":223184,"kb_used_data":6976,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754751536,"statfs":{"total":773094113280,"available":772865572864,"internally_reserved":0,"allocated":7143424,"data_stored":3751294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.420184"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.608559+0000","last_change":"2023-11-08T19:26:57.607231+0000","last_active":"2023-11-08T19:26:57.608559+0000","last_peered":"2023-11-08T19:26:57.608559+0000","last_clean":"2023-11-08T19:26:57.608559+0000","last_became_active":"2023-11-08T19:26:57.606996+0000","last_became_peered":"2023-11-08T19:26:57.606996+0000","last_unstale":"2023-11-08T19:26:57.608559+0000","last_undegraded":"2023-11-08T19:26:57.608559+0000","last_fullsized":"2023-11-08T19:26:57.608559+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:26:42.768409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.595787+0000","last_change":"2023-11-08T19:26:57.592516+0000","last_active":"2023-11-08T19:26:57.595787+0000","last_peered":"2023-11-08T19:26:57.595787+0000","last_clean":"2023-11-08T19:26:57.595787+0000","last_became_active":"2023-11-08T19:26:57.592236+0000","last_became_peered":"2023-11-08T19:26:57.592236+0000","last_unstale":"2023-11-08T19:26:57.595787+0000","last_undegraded":"2023-11-08T19:26:57.595787+0000","last_fullsized":"2023-11-08T19:26:57.595787+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:21:04.810797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-08T19:26:58.584656+0000","last_change":"2023-11-08T19:26:57.591582+0000","last_active":"2023-11-08T19:26:58.584656+0000","last_peered":"2023-11-08T19:26:58.584656+0000","last_clean":"2023-11-08T19:26:58.584656+0000","last_became_active":"2023-11-08T19:26:57.591303+0000","last_became_peered":"2023-11-08T19:26:57.591303+0000","last_unstale":"2023-11-08T19:26:58.584656+0000","last_undegraded":"2023-11-08T19:26:58.584656+0000","last_fullsized":"2023-11-08T19:26:58.584656+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:50:28.981159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.608194+0000","last_change":"2023-11-08T19:26:57.603108+0000","last_active":"2023-11-08T19:26:57.608194+0000","last_peered":"2023-11-08T19:26:57.608194+0000","last_clean":"2023-11-08T19:26:57.608194+0000","last_became_active":"2023-11-08T19:26:57.602693+0000","last_became_peered":"2023-11-08T19:26:57.602693+0000","last_unstale":"2023-11-08T19:26:57.608194+0000","last_undegraded":"2023-11-08T19:26:57.608194+0000","last_fullsized":"2023-11-08T19:26:57.608194+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:43:20.750289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-08T19:26:58.585310+0000","last_change":"2023-11-08T19:26:57.601753+0000","last_active":"2023-11-08T19:26:58.585310+0000","last_peered":"2023-11-08T19:26:58.585310+0000","last_clean":"2023-11-08T19:26:58.585310+0000","last_became_active":"2023-11-08T19:26:57.601442+0000","last_became_peered":"2023-11-08T19:26:57.601442+0000","last_unstale":"2023-11-08T19:26:58.585310+0000","last_undegraded":"2023-11-08T19:26:58.585310+0000","last_fullsized":"2023-11-08T19:26:58.585310+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:57:17.262278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.598707+0000","last_change":"2023-11-08T19:26:57.595873+0000","last_active":"2023-11-08T19:26:57.598707+0000","last_peered":"2023-11-08T19:26:57.598707+0000","last_clean":"2023-11-08T19:26:57.598707+0000","last_became_active":"2023-11-08T19:26:57.595600+0000","last_became_peered":"2023-11-08T19:26:57.595600+0000","last_unstale":"2023-11-08T19:26:57.598707+0000","last_undegraded":"2023-11-08T19:26:57.598707+0000","last_fullsized":"2023-11-08T19:26:57.598707+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:22:30.255080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.597912+0000","last_change":"2023-11-08T19:26:57.594848+0000","last_active":"2023-11-08T19:26:57.597912+0000","last_peered":"2023-11-08T19:26:57.597912+0000","last_clean":"2023-11-08T19:26:57.597912+0000","last_became_active":"2023-11-08T19:26:57.594595+0000","last_became_peered":"2023-11-08T19:26:57.594595+0000","last_unstale":"2023-11-08T19:26:57.597912+0000","last_undegraded":"2023-11-08T19:26:57.597912+0000","last_fullsized":"2023-11-08T19:26:57.597912+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:41:02.530230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-08T19:26:58.584211+0000","last_change":"2023-11-08T19:26:57.606413+0000","last_active":"2023-11-08T19:26:58.584211+0000","last_peered":"2023-11-08T19:26:58.584211+0000","last_clean":"2023-11-08T19:26:58.584211+0000","last_became_active":"2023-11-08T19:26:57.605975+0000","last_became_peered":"2023-11-08T19:26:57.605975+0000","last_unstale":"2023-11-08T19:26:58.584211+0000","last_undegraded":"2023-11-08T19:26:58.584211+0000","last_fullsized":"2023-11-08T19:26:58.584211+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:12:16.951947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-08T19:26:58.585424+0000","last_change":"2023-11-08T19:26:57.602119+0000","last_active":"2023-11-08T19:26:58.585424+0000","last_peered":"2023-11-08T19:26:58.585424+0000","last_clean":"2023-11-08T19:26:58.585424+0000","last_became_active":"2023-11-08T19:26:57.601800+0000","last_became_peered":"2023-11-08T19:26:57.601800+0000","last_unstale":"2023-11-08T19:26:58.585424+0000","last_undegraded":"2023-11-08T19:26:58.585424+0000","last_fullsized":"2023-11-08T19:26:58.585424+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:00:45.833292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-08T19:26:58.584872+0000","last_change":"2023-11-08T19:26:57.594212+0000","last_active":"2023-11-08T19:26:58.584872+0000","last_peered":"2023-11-08T19:26:58.584872+0000","last_clean":"2023-11-08T19:26:58.584872+0000","last_became_active":"2023-11-08T19:26:57.593917+0000","last_became_peered":"2023-11-08T19:26:57.593917+0000","last_unstale":"2023-11-08T19:26:58.584872+0000","last_undegraded":"2023-11-08T19:26:58.584872+0000","last_fullsized":"2023-11-08T19:26:58.584872+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:25:18.573108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.605497+0000","last_change":"2023-11-08T19:26:57.601350+0000","last_active":"2023-11-08T19:26:57.605497+0000","last_peered":"2023-11-08T19:26:57.605497+0000","last_clean":"2023-11-08T19:26:57.605497+0000","last_became_active":"2023-11-08T19:26:57.600360+0000","last_became_peered":"2023-11-08T19:26:57.600360+0000","last_unstale":"2023-11-08T19:26:57.605497+0000","last_undegraded":"2023-11-08T19:26:57.605497+0000","last_fullsized":"2023-11-08T19:26:57.605497+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:44:44.307857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-08T19:26:58.584601+0000","last_change":"2023-11-08T19:26:57.606323+0000","last_active":"2023-11-08T19:26:58.584601+0000","last_peered":"2023-11-08T19:26:58.584601+0000","last_clean":"2023-11-08T19:26:58.584601+0000","last_became_active":"2023-11-08T19:26:57.605882+0000","last_became_peered":"2023-11-08T19:26:57.605882+0000","last_unstale":"2023-11-08T19:26:58.584601+0000","last_undegraded":"2023-11-08T19:26:58.584601+0000","last_fullsized":"2023-11-08T19:26:58.584601+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:11:09.736119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-08T19:26:58.584436+0000","last_change":"2023-11-08T19:26:57.607261+0000","last_active":"2023-11-08T19:26:58.584436+0000","last_peered":"2023-11-08T19:26:58.584436+0000","last_clean":"2023-11-08T19:26:58.584436+0000","last_became_active":"2023-11-08T19:26:57.606604+0000","last_became_peered":"2023-11-08T19:26:57.606604+0000","last_unstale":"2023-11-08T19:26:58.584436+0000","last_undegraded":"2023-11-08T19:26:58.584436+0000","last_fullsized":"2023-11-08T19:26:58.584436+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:41:44.310351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-08T19:26:58.585652+0000","last_change":"2023-11-08T19:26:57.591002+0000","last_active":"2023-11-08T19:26:58.585652+0000","last_peered":"2023-11-08T19:26:58.585652+0000","last_clean":"2023-11-08T19:26:58.585652+0000","last_became_active":"2023-11-08T19:26:57.590729+0000","last_became_peered":"2023-11-08T19:26:57.590729+0000","last_unstale":"2023-11-08T19:26:58.585652+0000","last_undegraded":"2023-11-08T19:26:58.585652+0000","last_fullsized":"2023-11-08T19:26:58.585652+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:18:10.924385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-08T19:26:58.585887+0000","last_change":"2023-11-08T19:26:57.592868+0000","last_active":"2023-11-08T19:26:58.585887+0000","last_peered":"2023-11-08T19:26:58.585887+0000","last_clean":"2023-11-08T19:26:58.585887+0000","last_became_active":"2023-11-08T19:26:57.592584+0000","last_became_peered":"2023-11-08T19:26:57.592584+0000","last_unstale":"2023-11-08T19:26:58.585887+0000","last_undegraded":"2023-11-08T19:26:58.585887+0000","last_fullsized":"2023-11-08T19:26:58.585887+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:35:20.028780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.607796+0000","last_change":"2023-11-08T19:26:57.606188+0000","last_active":"2023-11-08T19:26:57.607796+0000","last_peered":"2023-11-08T19:26:57.607796+0000","last_clean":"2023-11-08T19:26:57.607796+0000","last_became_active":"2023-11-08T19:26:57.605938+0000","last_became_peered":"2023-11-08T19:26:57.605938+0000","last_unstale":"2023-11-08T19:26:57.607796+0000","last_undegraded":"2023-11-08T19:26:57.607796+0000","last_fullsized":"2023-11-08T19:26:57.607796+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:35:55.850578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.602400+0000","last_change":"2023-11-08T19:26:57.601197+0000","last_active":"2023-11-08T19:26:57.602400+0000","last_peered":"2023-11-08T19:26:57.602400+0000","last_clean":"2023-11-08T19:26:57.602400+0000","last_became_active":"2023-11-08T19:26:57.600828+0000","last_became_peered":"2023-11-08T19:26:57.600828+0000","last_unstale":"2023-11-08T19:26:57.602400+0000","last_undegraded":"2023-11-08T19:26:57.602400+0000","last_fullsized":"2023-11-08T19:26:57.602400+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:40:19.853844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.608091+0000","last_change":"2023-11-08T19:26:57.600944+0000","last_active":"2023-11-08T19:26:57.608091+0000","last_peered":"2023-11-08T19:26:57.608091+0000","last_clean":"2023-11-08T19:26:57.608091+0000","last_became_active":"2023-11-08T19:26:57.600673+0000","last_became_peered":"2023-11-08T19:26:57.600673+0000","last_unstale":"2023-11-08T19:26:57.608091+0000","last_undegraded":"2023-11-08T19:26:57.608091+0000","last_fullsized":"2023-11-08T19:26:57.608091+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:03:53.043225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-08T19:26:58.584996+0000","last_change":"2023-11-08T19:26:57.595187+0000","last_active":"2023-11-08T19:26:58.584996+0000","last_peered":"2023-11-08T19:26:58.584996+0000","last_clean":"2023-11-08T19:26:58.584996+0000","last_became_active":"2023-11-08T19:26:57.594245+0000","last_became_peered":"2023-11-08T19:26:57.594245+0000","last_unstale":"2023-11-08T19:26:58.584996+0000","last_undegraded":"2023-11-08T19:26:58.584996+0000","last_fullsized":"2023-11-08T19:26:58.584996+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:20:08.283676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.597754+0000","last_change":"2023-11-08T19:26:57.596204+0000","last_active":"2023-11-08T19:26:57.597754+0000","last_peered":"2023-11-08T19:26:57.597754+0000","last_clean":"2023-11-08T19:26:57.597754+0000","last_became_active":"2023-11-08T19:26:57.595948+0000","last_became_peered":"2023-11-08T19:26:57.595948+0000","last_unstale":"2023-11-08T19:26:57.597754+0000","last_undegraded":"2023-11-08T19:26:57.597754+0000","last_fullsized":"2023-11-08T19:26:57.597754+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:00:45.833292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-08T19:26:58.584380+0000","last_change":"2023-11-08T19:26:57.607025+0000","last_active":"2023-11-08T19:26:58.584380+0000","last_peered":"2023-11-08T19:26:58.584380+0000","last_clean":"2023-11-08T19:26:58.584380+0000","last_became_active":"2023-11-08T19:26:57.606722+0000","last_became_peered":"2023-11-08T19:26:57.606722+0000","last_unstale":"2023-11-08T19:26:58.584380+0000","last_undegraded":"2023-11-08T19:26:58.584380+0000","last_fullsized":"2023-11-08T19:26:58.584380+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:36:03.215026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.596164+0000","last_change":"2023-11-08T19:26:57.594306+0000","last_active":"2023-11-08T19:26:57.596164+0000","last_peered":"2023-11-08T19:26:57.596164+0000","last_clean":"2023-11-08T19:26:57.596164+0000","last_became_active":"2023-11-08T19:26:57.594040+0000","last_became_peered":"2023-11-08T19:26:57.594040+0000","last_unstale":"2023-11-08T19:26:57.596164+0000","last_undegraded":"2023-11-08T19:26:57.596164+0000","last_fullsized":"2023-11-08T19:26:57.596164+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:58:39.317164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.596291+0000","last_change":"2023-11-08T19:26:57.594290+0000","last_active":"2023-11-08T19:26:57.596291+0000","last_peered":"2023-11-08T19:26:57.596291+0000","last_clean":"2023-11-08T19:26:57.596291+0000","last_became_active":"2023-11-08T19:26:57.594014+0000","last_became_peered":"2023-11-08T19:26:57.594014+0000","last_unstale":"2023-11-08T19:26:57.596291+0000","last_undegraded":"2023-11-08T19:26:57.596291+0000","last_fullsized":"2023-11-08T19:26:57.596291+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:46:31.276014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-08T19:26:58.585761+0000","last_change":"2023-11-08T19:26:57.591067+0000","last_active":"2023-11-08T19:26:58.585761+0000","last_peered":"2023-11-08T19:26:58.585761+0000","last_clean":"2023-11-08T19:26:58.585761+0000","last_became_active":"2023-11-08T19:26:57.590802+0000","last_became_peered":"2023-11-08T19:26:57.590802+0000","last_unstale":"2023-11-08T19:26:58.585761+0000","last_undegraded":"2023-11-08T19:26:58.585761+0000","last_fullsized":"2023-11-08T19:26:58.585761+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:14:08.564956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-08T19:26:58.584702+0000","last_change":"2023-11-08T19:26:57.606394+0000","last_active":"2023-11-08T19:26:58.584702+0000","last_peered":"2023-11-08T19:26:58.584702+0000","last_clean":"2023-11-08T19:26:58.584702+0000","last_became_active":"2023-11-08T19:26:57.606021+0000","last_became_peered":"2023-11-08T19:26:57.606021+0000","last_unstale":"2023-11-08T19:26:58.584702+0000","last_undegraded":"2023-11-08T19:26:58.584702+0000","last_fullsized":"2023-11-08T19:26:58.584702+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:05:13.476820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-08T19:26:58.584853+0000","last_change":"2023-11-08T19:26:57.607036+0000","last_active":"2023-11-08T19:26:58.584853+0000","last_peered":"2023-11-08T19:26:58.584853+0000","last_clean":"2023-11-08T19:26:58.584853+0000","last_became_active":"2023-11-08T19:26:57.606757+0000","last_became_peered":"2023-11-08T19:26:57.606757+0000","last_unstale":"2023-11-08T19:26:58.584853+0000","last_undegraded":"2023-11-08T19:26:58.584853+0000","last_fullsized":"2023-11-08T19:26:58.584853+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:02:00.818297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.605540+0000","last_change":"2023-11-08T19:26:57.601522+0000","last_active":"2023-11-08T19:26:57.605540+0000","last_peered":"2023-11-08T19:26:57.605540+0000","last_clean":"2023-11-08T19:26:57.605540+0000","last_became_active":"2023-11-08T19:26:57.600770+0000","last_became_peered":"2023-11-08T19:26:57.600770+0000","last_unstale":"2023-11-08T19:26:57.605540+0000","last_undegraded":"2023-11-08T19:26:57.605540+0000","last_fullsized":"2023-11-08T19:26:57.605540+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:33:38.123215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.598998+0000","last_change":"2023-11-08T19:26:57.597582+0000","last_active":"2023-11-08T19:26:57.598998+0000","last_peered":"2023-11-08T19:26:57.598998+0000","last_clean":"2023-11-08T19:26:57.598998+0000","last_became_active":"2023-11-08T19:26:57.597249+0000","last_became_peered":"2023-11-08T19:26:57.597249+0000","last_unstale":"2023-11-08T19:26:57.598998+0000","last_undegraded":"2023-11-08T19:26:57.598998+0000","last_fullsized":"2023-11-08T19:26:57.598998+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:19:20.298031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-08T19:26:58.584541+0000","last_change":"2023-11-08T19:26:57.601661+0000","last_active":"2023-11-08T19:26:58.584541+0000","last_peered":"2023-11-08T19:26:58.584541+0000","last_clean":"2023-11-08T19:26:58.584541+0000","last_became_active":"2023-11-08T19:26:57.601350+0000","last_became_peered":"2023-11-08T19:26:57.601350+0000","last_unstale":"2023-11-08T19:26:58.584541+0000","last_undegraded":"2023-11-08T19:26:58.584541+0000","last_fullsized":"2023-11-08T19:26:58.584541+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:55:27.608415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-08T19:26:58.585868+0000","last_change":"2023-11-08T19:26:57.607208+0000","last_active":"2023-11-08T19:26:58.585868+0000","last_peered":"2023-11-08T19:26:58.585868+0000","last_clean":"2023-11-08T19:26:58.585868+0000","last_became_active":"2023-11-08T19:26:57.606672+0000","last_became_peered":"2023-11-08T19:26:57.606672+0000","last_unstale":"2023-11-08T19:26:58.585868+0000","last_undegraded":"2023-11-08T19:26:58.585868+0000","last_fullsized":"2023-11-08T19:26:58.585868+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:08:41.819313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.602731+0000","last_change":"2023-11-08T19:26:57.597658+0000","last_active":"2023-11-08T19:26:57.602731+0000","last_peered":"2023-11-08T19:26:57.602731+0000","last_clean":"2023-11-08T19:26:57.602731+0000","last_became_active":"2023-11-08T19:26:57.597396+0000","last_became_peered":"2023-11-08T19:26:57.597396+0000","last_unstale":"2023-11-08T19:26:57.602731+0000","last_undegraded":"2023-11-08T19:26:57.602731+0000","last_fullsized":"2023-11-08T19:26:57.602731+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:06:04.323890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.597110+0000","last_change":"2023-11-08T19:26:57.593868+0000","last_active":"2023-11-08T19:26:57.597110+0000","last_peered":"2023-11-08T19:26:57.597110+0000","last_clean":"2023-11-08T19:26:57.597110+0000","last_became_active":"2023-11-08T19:26:57.593635+0000","last_became_peered":"2023-11-08T19:26:57.593635+0000","last_unstale":"2023-11-08T19:26:57.597110+0000","last_undegraded":"2023-11-08T19:26:57.597110+0000","last_fullsized":"2023-11-08T19:26:57.597110+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:08:41.819313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-08T19:26:58.584691+0000","last_change":"2023-11-08T19:26:57.606432+0000","last_active":"2023-11-08T19:26:58.584691+0000","last_peered":"2023-11-08T19:26:58.584691+0000","last_clean":"2023-11-08T19:26:58.584691+0000","last_became_active":"2023-11-08T19:26:57.606146+0000","last_became_peered":"2023-11-08T19:26:57.606146+0000","last_unstale":"2023-11-08T19:26:58.584691+0000","last_undegraded":"2023-11-08T19:26:58.584691+0000","last_fullsized":"2023-11-08T19:26:58.584691+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:15:10.416716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-08T19:26:58.584127+0000","last_change":"2023-11-08T19:26:57.148136+0000","last_active":"2023-11-08T19:26:58.584127+0000","last_peered":"2023-11-08T19:26:58.584127+0000","last_clean":"2023-11-08T19:26:58.584127+0000","last_became_active":"2023-11-08T19:26:57.147859+0000","last_became_peered":"2023-11-08T19:26:57.147859+0000","last_unstale":"2023-11-08T19:26:58.584127+0000","last_undegraded":"2023-11-08T19:26:58.584127+0000","last_fullsized":"2023-11-08T19:26:58.584127+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_clean_scrub_stamp":"2023-11-08T19:26:56.117060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:31:40.425519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":36,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.149273+0000","last_change":"2023-11-08T19:26:06.297255+0000","last_active":"2023-11-08T19:26:57.149273+0000","last_peered":"2023-11-08T19:26:57.149273+0000","last_clean":"2023-11-08T19:26:57.149273+0000","last_became_active":"2023-11-08T19:26:04.603979+0000","last_became_peered":"2023-11-08T19:26:04.603979+0000","last_unstale":"2023-11-08T19:26:57.149273+0000","last_undegraded":"2023-11-08T19:26:57.149273+0000","last_fullsized":"2023-11-08T19:26:57.149273+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:40:11.164951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079032099999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.605481+0000","last_change":"2023-11-08T19:26:57.600053+0000","last_active":"2023-11-08T19:26:57.605481+0000","last_peered":"2023-11-08T19:26:57.605481+0000","last_clean":"2023-11-08T19:26:57.605481+0000","last_became_active":"2023-11-08T19:26:57.599773+0000","last_became_peered":"2023-11-08T19:26:57.599773+0000","last_unstale":"2023-11-08T19:26:57.605481+0000","last_undegraded":"2023-11-08T19:26:57.605481+0000","last_fullsized":"2023-11-08T19:26:57.605481+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:50:26.288669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"61'2","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-08T19:26:58.584309+0000","last_change":"2023-11-08T19:26:57.144661+0000","last_active":"2023-11-08T19:26:58.584309+0000","last_peered":"2023-11-08T19:26:58.584309+0000","last_clean":"2023-11-08T19:26:58.584309+0000","last_became_active":"2023-11-08T19:26:57.144233+0000","last_became_peered":"2023-11-08T19:26:57.144233+0000","last_unstale":"2023-11-08T19:26:58.584309+0000","last_undegraded":"2023-11-08T19:26:58.584309+0000","last_fullsized":"2023-11-08T19:26:58.584309+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_clean_scrub_stamp":"2023-11-08T19:26:56.117060+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:11:08.730999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":42,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.584567+0000","last_change":"2023-11-08T19:26:05.609625+0000","last_active":"2023-11-08T19:26:57.584567+0000","last_peered":"2023-11-08T19:26:57.584567+0000","last_clean":"2023-11-08T19:26:57.584567+0000","last_became_active":"2023-11-08T19:26:04.606312+0000","last_became_peered":"2023-11-08T19:26:04.606312+0000","last_unstale":"2023-11-08T19:26:57.584567+0000","last_undegraded":"2023-11-08T19:26:57.584567+0000","last_fullsized":"2023-11-08T19:26:57.584567+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:54:42.032901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000505935,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.598184+0000","last_change":"2023-11-08T19:26:57.595669+0000","last_active":"2023-11-08T19:26:57.598184+0000","last_peered":"2023-11-08T19:26:57.598184+0000","last_clean":"2023-11-08T19:26:57.598184+0000","last_became_active":"2023-11-08T19:26:57.595394+0000","last_became_peered":"2023-11-08T19:26:57.595394+0000","last_unstale":"2023-11-08T19:26:57.598184+0000","last_undegraded":"2023-11-08T19:26:57.598184+0000","last_fullsized":"2023-11-08T19:26:57.598184+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:59:19.833259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"61'8","reported_seq":19,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:58.145042+0000","last_change":"2023-11-08T19:26:57.144627+0000","last_active":"2023-11-08T19:26:58.145042+0000","last_peered":"2023-11-08T19:26:58.145042+0000","last_clean":"2023-11-08T19:26:58.145042+0000","last_became_active":"2023-11-08T19:26:57.144310+0000","last_became_peered":"2023-11-08T19:26:57.144310+0000","last_unstale":"2023-11-08T19:26:58.145042+0000","last_undegraded":"2023-11-08T19:26:58.145042+0000","last_fullsized":"2023-11-08T19:26:58.145042+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_clean_scrub_stamp":"2023-11-08T19:26:56.117060+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:49:53.766699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":37,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.139034+0000","last_change":"2023-11-08T19:26:06.425133+0000","last_active":"2023-11-08T19:26:57.139034+0000","last_peered":"2023-11-08T19:26:57.139034+0000","last_clean":"2023-11-08T19:26:57.139034+0000","last_became_active":"2023-11-08T19:26:04.601103+0000","last_became_peered":"2023-11-08T19:26:04.601103+0000","last_unstale":"2023-11-08T19:26:57.139034+0000","last_undegraded":"2023-11-08T19:26:57.139034+0000","last_fullsized":"2023-11-08T19:26:57.139034+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:31:14.042561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000229244,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-08T19:26:58.585203+0000","last_change":"2023-11-08T19:26:57.595589+0000","last_active":"2023-11-08T19:26:58.585203+0000","last_peered":"2023-11-08T19:26:58.585203+0000","last_clean":"2023-11-08T19:26:58.585203+0000","last_became_active":"2023-11-08T19:26:57.594674+0000","last_became_peered":"2023-11-08T19:26:57.594674+0000","last_unstale":"2023-11-08T19:26:58.585203+0000","last_undegraded":"2023-11-08T19:26:58.585203+0000","last_fullsized":"2023-11-08T19:26:58.585203+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:03:09.281780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-08T19:26:58.585716+0000","last_change":"2023-11-08T19:26:57.142517+0000","last_active":"2023-11-08T19:26:58.585716+0000","last_peered":"2023-11-08T19:26:58.585716+0000","last_clean":"2023-11-08T19:26:58.585716+0000","last_became_active":"2023-11-08T19:26:57.142257+0000","last_became_peered":"2023-11-08T19:26:57.142257+0000","last_unstale":"2023-11-08T19:26:58.585716+0000","last_undegraded":"2023-11-08T19:26:58.585716+0000","last_fullsized":"2023-11-08T19:26:58.585716+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_clean_scrub_stamp":"2023-11-08T19:26:56.117060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:56:54.191245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":35,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.140494+0000","last_change":"2023-11-08T19:26:06.265690+0000","last_active":"2023-11-08T19:26:57.140494+0000","last_peered":"2023-11-08T19:26:57.140494+0000","last_clean":"2023-11-08T19:26:57.140494+0000","last_became_active":"2023-11-08T19:26:04.603397+0000","last_became_peered":"2023-11-08T19:26:04.603397+0000","last_unstale":"2023-11-08T19:26:57.140494+0000","last_undegraded":"2023-11-08T19:26:57.140494+0000","last_fullsized":"2023-11-08T19:26:57.140494+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:50:43.827570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053361499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.607522+0000","last_change":"2023-11-08T19:26:57.606678+0000","last_active":"2023-11-08T19:26:57.607522+0000","last_peered":"2023-11-08T19:26:57.607522+0000","last_clean":"2023-11-08T19:26:57.607522+0000","last_became_active":"2023-11-08T19:26:57.606388+0000","last_became_peered":"2023-11-08T19:26:57.606388+0000","last_unstale":"2023-11-08T19:26:57.607522+0000","last_undegraded":"2023-11-08T19:26:57.607522+0000","last_fullsized":"2023-11-08T19:26:57.607522+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:14:08.564956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.599700+0000","last_change":"2023-11-08T19:26:57.154506+0000","last_active":"2023-11-08T19:26:57.599700+0000","last_peered":"2023-11-08T19:26:57.599700+0000","last_clean":"2023-11-08T19:26:57.599700+0000","last_became_active":"2023-11-08T19:26:57.154236+0000","last_became_peered":"2023-11-08T19:26:57.154236+0000","last_unstale":"2023-11-08T19:26:57.599700+0000","last_undegraded":"2023-11-08T19:26:57.599700+0000","last_fullsized":"2023-11-08T19:26:57.599700+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_clean_scrub_stamp":"2023-11-08T19:26:56.117060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:59:53.470375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":38,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.598883+0000","last_change":"2023-11-08T19:26:06.541338+0000","last_active":"2023-11-08T19:26:57.598883+0000","last_peered":"2023-11-08T19:26:57.598883+0000","last_clean":"2023-11-08T19:26:57.598883+0000","last_became_active":"2023-11-08T19:26:04.602183+0000","last_became_peered":"2023-11-08T19:26:04.602183+0000","last_unstale":"2023-11-08T19:26:57.598883+0000","last_undegraded":"2023-11-08T19:26:57.598883+0000","last_fullsized":"2023-11-08T19:26:57.598883+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:06:47.446686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052029099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.597659+0000","last_change":"2023-11-08T19:26:57.594369+0000","last_active":"2023-11-08T19:26:57.597659+0000","last_peered":"2023-11-08T19:26:57.597659+0000","last_clean":"2023-11-08T19:26:57.597659+0000","last_became_active":"2023-11-08T19:26:57.593927+0000","last_became_peered":"2023-11-08T19:26:57.593927+0000","last_unstale":"2023-11-08T19:26:57.597659+0000","last_undegraded":"2023-11-08T19:26:57.597659+0000","last_fullsized":"2023-11-08T19:26:57.597659+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:31:41.430639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-08T19:26:58.585847+0000","last_change":"2023-11-08T19:26:57.154295+0000","last_active":"2023-11-08T19:26:58.585847+0000","last_peered":"2023-11-08T19:26:58.585847+0000","last_clean":"2023-11-08T19:26:58.585847+0000","last_became_active":"2023-11-08T19:26:57.153971+0000","last_became_peered":"2023-11-08T19:26:57.153971+0000","last_unstale":"2023-11-08T19:26:58.585847+0000","last_undegraded":"2023-11-08T19:26:58.585847+0000","last_fullsized":"2023-11-08T19:26:58.585847+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_clean_scrub_stamp":"2023-11-08T19:26:56.117060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:49:05.408409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":35,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.138718+0000","last_change":"2023-11-08T19:26:06.394081+0000","last_active":"2023-11-08T19:26:57.138718+0000","last_peered":"2023-11-08T19:26:57.138718+0000","last_clean":"2023-11-08T19:26:57.138718+0000","last_became_active":"2023-11-08T19:26:04.602137+0000","last_became_peered":"2023-11-08T19:26:04.602137+0000","last_unstale":"2023-11-08T19:26:57.138718+0000","last_undegraded":"2023-11-08T19:26:57.138718+0000","last_fullsized":"2023-11-08T19:26:57.138718+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:05:46.951663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056998200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.605523+0000","last_change":"2023-11-08T19:26:57.601714+0000","last_active":"2023-11-08T19:26:57.605523+0000","last_peered":"2023-11-08T19:26:57.605523+0000","last_clean":"2023-11-08T19:26:57.605523+0000","last_became_active":"2023-11-08T19:26:57.599850+0000","last_became_peered":"2023-11-08T19:26:57.599850+0000","last_unstale":"2023-11-08T19:26:57.605523+0000","last_undegraded":"2023-11-08T19:26:57.605523+0000","last_fullsized":"2023-11-08T19:26:57.605523+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:17:48.204358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"61'6","reported_seq":17,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:58.147383+0000","last_change":"2023-11-08T19:26:57.150200+0000","last_active":"2023-11-08T19:26:58.147383+0000","last_peered":"2023-11-08T19:26:58.147383+0000","last_clean":"2023-11-08T19:26:58.147383+0000","last_became_active":"2023-11-08T19:26:57.149918+0000","last_became_peered":"2023-11-08T19:26:57.149918+0000","last_unstale":"2023-11-08T19:26:58.147383+0000","last_undegraded":"2023-11-08T19:26:58.147383+0000","last_fullsized":"2023-11-08T19:26:58.147383+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_clean_scrub_stamp":"2023-11-08T19:26:56.117060+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:35:09.627917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":37,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.138819+0000","last_change":"2023-11-08T19:26:06.423863+0000","last_active":"2023-11-08T19:26:57.138819+0000","last_peered":"2023-11-08T19:26:57.138819+0000","last_clean":"2023-11-08T19:26:57.138819+0000","last_became_active":"2023-11-08T19:26:04.608639+0000","last_became_peered":"2023-11-08T19:26:04.608639+0000","last_unstale":"2023-11-08T19:26:57.138819+0000","last_undegraded":"2023-11-08T19:26:57.138819+0000","last_fullsized":"2023-11-08T19:26:57.138819+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:04:43.631102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044384999999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-08T19:26:58.585525+0000","last_change":"2023-11-08T19:26:57.608608+0000","last_active":"2023-11-08T19:26:58.585525+0000","last_peered":"2023-11-08T19:26:58.585525+0000","last_clean":"2023-11-08T19:26:58.585525+0000","last_became_active":"2023-11-08T19:26:57.608344+0000","last_became_peered":"2023-11-08T19:26:57.608344+0000","last_unstale":"2023-11-08T19:26:58.585525+0000","last_undegraded":"2023-11-08T19:26:58.585525+0000","last_fullsized":"2023-11-08T19:26:58.585525+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:19:20.298031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"61'3","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-08T19:26:58.584509+0000","last_change":"2023-11-08T19:26:57.154362+0000","last_active":"2023-11-08T19:26:58.584509+0000","last_peered":"2023-11-08T19:26:58.584509+0000","last_clean":"2023-11-08T19:26:58.584509+0000","last_became_active":"2023-11-08T19:26:57.154116+0000","last_became_peered":"2023-11-08T19:26:57.154116+0000","last_unstale":"2023-11-08T19:26:58.584509+0000","last_undegraded":"2023-11-08T19:26:58.584509+0000","last_fullsized":"2023-11-08T19:26:58.584509+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_clean_scrub_stamp":"2023-11-08T19:26:56.117060+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:41:43.305231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":35,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.135363+0000","last_change":"2023-11-08T19:26:06.265318+0000","last_active":"2023-11-08T19:26:57.135363+0000","last_peered":"2023-11-08T19:26:57.135363+0000","last_clean":"2023-11-08T19:26:57.135363+0000","last_became_active":"2023-11-08T19:26:04.603409+0000","last_became_peered":"2023-11-08T19:26:04.603409+0000","last_unstale":"2023-11-08T19:26:57.135363+0000","last_undegraded":"2023-11-08T19:26:57.135363+0000","last_fullsized":"2023-11-08T19:26:57.135363+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:27:24.933111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065684300000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.606871+0000","last_change":"2023-11-08T19:26:57.606073+0000","last_active":"2023-11-08T19:26:57.606871+0000","last_peered":"2023-11-08T19:26:57.606871+0000","last_clean":"2023-11-08T19:26:57.606871+0000","last_became_active":"2023-11-08T19:26:57.605705+0000","last_became_peered":"2023-11-08T19:26:57.605705+0000","last_unstale":"2023-11-08T19:26:57.606871+0000","last_undegraded":"2023-11-08T19:26:57.606871+0000","last_fullsized":"2023-11-08T19:26:57.606871+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:03:37.833816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"61'1","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:58.136433+0000","last_change":"2023-11-08T19:26:57.146499+0000","last_active":"2023-11-08T19:26:58.136433+0000","last_peered":"2023-11-08T19:26:58.136433+0000","last_clean":"2023-11-08T19:26:58.136433+0000","last_became_active":"2023-11-08T19:26:57.146173+0000","last_became_peered":"2023-11-08T19:26:57.146173+0000","last_unstale":"2023-11-08T19:26:58.136433+0000","last_undegraded":"2023-11-08T19:26:58.136433+0000","last_fullsized":"2023-11-08T19:26:58.136433+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_clean_scrub_stamp":"2023-11-08T19:26:56.117060+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:18:09.919265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":42,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.585155+0000","last_change":"2023-11-08T19:26:05.610012+0000","last_active":"2023-11-08T19:26:57.585155+0000","last_peered":"2023-11-08T19:26:57.585155+0000","last_clean":"2023-11-08T19:26:57.585155+0000","last_became_active":"2023-11-08T19:26:04.603209+0000","last_became_peered":"2023-11-08T19:26:04.603209+0000","last_unstale":"2023-11-08T19:26:57.585155+0000","last_undegraded":"2023-11-08T19:26:57.585155+0000","last_fullsized":"2023-11-08T19:26:57.585155+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:35:02.306997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053335100000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.594769+0000","last_change":"2023-11-08T19:26:57.593598+0000","last_active":"2023-11-08T19:26:57.594769+0000","last_peered":"2023-11-08T19:26:57.594769+0000","last_clean":"2023-11-08T19:26:57.594769+0000","last_became_active":"2023-11-08T19:26:57.593055+0000","last_became_peered":"2023-11-08T19:26:57.593055+0000","last_unstale":"2023-11-08T19:26:57.594769+0000","last_undegraded":"2023-11-08T19:26:57.594769+0000","last_fullsized":"2023-11-08T19:26:57.594769+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:55:35.576482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"61'1","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:58.145688+0000","last_change":"2023-11-08T19:26:57.150582+0000","last_active":"2023-11-08T19:26:58.145688+0000","last_peered":"2023-11-08T19:26:58.145688+0000","last_clean":"2023-11-08T19:26:58.145688+0000","last_became_active":"2023-11-08T19:26:57.150316+0000","last_became_peered":"2023-11-08T19:26:57.150316+0000","last_unstale":"2023-11-08T19:26:58.145688+0000","last_undegraded":"2023-11-08T19:26:58.145688+0000","last_fullsized":"2023-11-08T19:26:58.145688+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_clean_scrub_stamp":"2023-11-08T19:26:56.117060+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:40:37.959491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":36,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.141723+0000","last_change":"2023-11-08T19:26:05.606946+0000","last_active":"2023-11-08T19:26:57.141723+0000","last_peered":"2023-11-08T19:26:57.141723+0000","last_clean":"2023-11-08T19:26:57.141723+0000","last_became_active":"2023-11-08T19:26:04.598775+0000","last_became_peered":"2023-11-08T19:26:04.598775+0000","last_unstale":"2023-11-08T19:26:57.141723+0000","last_undegraded":"2023-11-08T19:26:57.141723+0000","last_fullsized":"2023-11-08T19:26:57.141723+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:20:56.898927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049870399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.601528+0000","last_change":"2023-11-08T19:26:57.601148+0000","last_active":"2023-11-08T19:26:57.601528+0000","last_peered":"2023-11-08T19:26:57.601528+0000","last_clean":"2023-11-08T19:26:57.601528+0000","last_became_active":"2023-11-08T19:26:57.600907+0000","last_became_peered":"2023-11-08T19:26:57.600907+0000","last_unstale":"2023-11-08T19:26:57.601528+0000","last_undegraded":"2023-11-08T19:26:57.601528+0000","last_fullsized":"2023-11-08T19:26:57.601528+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:25:47.748513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-08T19:26:58.584514+0000","last_change":"2023-11-08T19:26:57.143692+0000","last_active":"2023-11-08T19:26:58.584514+0000","last_peered":"2023-11-08T19:26:58.584514+0000","last_clean":"2023-11-08T19:26:58.584514+0000","last_became_active":"2023-11-08T19:26:57.142738+0000","last_became_peered":"2023-11-08T19:26:57.142738+0000","last_unstale":"2023-11-08T19:26:58.584514+0000","last_undegraded":"2023-11-08T19:26:58.584514+0000","last_fullsized":"2023-11-08T19:26:58.584514+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_clean_scrub_stamp":"2023-11-08T19:26:56.117060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:44:43.302738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":35,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.139559+0000","last_change":"2023-11-08T19:26:06.265907+0000","last_active":"2023-11-08T19:26:57.139559+0000","last_peered":"2023-11-08T19:26:57.139559+0000","last_clean":"2023-11-08T19:26:57.139559+0000","last_became_active":"2023-11-08T19:26:04.604772+0000","last_became_peered":"2023-11-08T19:26:04.604772+0000","last_unstale":"2023-11-08T19:26:57.139559+0000","last_undegraded":"2023-11-08T19:26:57.139559+0000","last_fullsized":"2023-11-08T19:26:57.139559+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:44:16.139794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066875300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-08T19:26:58.585041+0000","last_change":"2023-11-08T19:26:57.590852+0000","last_active":"2023-11-08T19:26:58.585041+0000","last_peered":"2023-11-08T19:26:58.585041+0000","last_clean":"2023-11-08T19:26:58.585041+0000","last_became_active":"2023-11-08T19:26:57.590503+0000","last_became_peered":"2023-11-08T19:26:57.590503+0000","last_unstale":"2023-11-08T19:26:58.585041+0000","last_undegraded":"2023-11-08T19:26:58.585041+0000","last_fullsized":"2023-11-08T19:26:58.585041+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:59:54.475495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.588456+0000","last_change":"2023-11-08T19:26:57.143857+0000","last_active":"2023-11-08T19:26:57.588456+0000","last_peered":"2023-11-08T19:26:57.588456+0000","last_clean":"2023-11-08T19:26:57.588456+0000","last_became_active":"2023-11-08T19:26:57.143525+0000","last_became_peered":"2023-11-08T19:26:57.143525+0000","last_unstale":"2023-11-08T19:26:57.588456+0000","last_undegraded":"2023-11-08T19:26:57.588456+0000","last_fullsized":"2023-11-08T19:26:57.588456+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_clean_scrub_stamp":"2023-11-08T19:26:56.117060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:31:06.035836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":35,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.139273+0000","last_change":"2023-11-08T19:26:06.394488+0000","last_active":"2023-11-08T19:26:57.139273+0000","last_peered":"2023-11-08T19:26:57.139273+0000","last_clean":"2023-11-08T19:26:57.139273+0000","last_became_active":"2023-11-08T19:26:04.606805+0000","last_became_peered":"2023-11-08T19:26:04.606805+0000","last_unstale":"2023-11-08T19:26:57.139273+0000","last_undegraded":"2023-11-08T19:26:57.139273+0000","last_fullsized":"2023-11-08T19:26:57.139273+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:25:20.857133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045317000000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.608238+0000","last_change":"2023-11-08T19:26:57.603508+0000","last_active":"2023-11-08T19:26:57.608238+0000","last_peered":"2023-11-08T19:26:57.608238+0000","last_clean":"2023-11-08T19:26:57.608238+0000","last_became_active":"2023-11-08T19:26:57.603223+0000","last_became_peered":"2023-11-08T19:26:57.603223+0000","last_unstale":"2023-11-08T19:26:57.608238+0000","last_undegraded":"2023-11-08T19:26:57.608238+0000","last_fullsized":"2023-11-08T19:26:57.608238+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:07:17.979415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"61'1","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:58.145500+0000","last_change":"2023-11-08T19:26:57.150422+0000","last_active":"2023-11-08T19:26:58.145500+0000","last_peered":"2023-11-08T19:26:58.145500+0000","last_clean":"2023-11-08T19:26:58.145500+0000","last_became_active":"2023-11-08T19:26:57.150147+0000","last_became_peered":"2023-11-08T19:26:57.150147+0000","last_unstale":"2023-11-08T19:26:58.145500+0000","last_undegraded":"2023-11-08T19:26:58.145500+0000","last_fullsized":"2023-11-08T19:26:58.145500+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_clean_scrub_stamp":"2023-11-08T19:26:56.117060+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:00:44.828172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":37,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.138770+0000","last_change":"2023-11-08T19:26:06.423754+0000","last_active":"2023-11-08T19:26:57.138770+0000","last_peered":"2023-11-08T19:26:57.138770+0000","last_clean":"2023-11-08T19:26:57.138770+0000","last_became_active":"2023-11-08T19:26:04.599322+0000","last_became_peered":"2023-11-08T19:26:04.599322+0000","last_unstale":"2023-11-08T19:26:57.138770+0000","last_undegraded":"2023-11-08T19:26:57.138770+0000","last_fullsized":"2023-11-08T19:26:57.138770+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:13:08.689128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042670000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-08T19:26:58.584626+0000","last_change":"2023-11-08T19:26:57.595871+0000","last_active":"2023-11-08T19:26:58.584626+0000","last_peered":"2023-11-08T19:26:58.584626+0000","last_clean":"2023-11-08T19:26:58.584626+0000","last_became_active":"2023-11-08T19:26:57.595610+0000","last_became_peered":"2023-11-08T19:26:57.595610+0000","last_unstale":"2023-11-08T19:26:58.584626+0000","last_undegraded":"2023-11-08T19:26:58.584626+0000","last_fullsized":"2023-11-08T19:26:58.584626+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:43:20.750289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"61'4","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:58.142409+0000","last_change":"2023-11-08T19:26:57.151102+0000","last_active":"2023-11-08T19:26:58.142409+0000","last_peered":"2023-11-08T19:26:58.142409+0000","last_clean":"2023-11-08T19:26:58.142409+0000","last_became_active":"2023-11-08T19:26:57.150727+0000","last_became_peered":"2023-11-08T19:26:57.150727+0000","last_unstale":"2023-11-08T19:26:58.142409+0000","last_undegraded":"2023-11-08T19:26:58.142409+0000","last_fullsized":"2023-11-08T19:26:58.142409+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_clean_scrub_stamp":"2023-11-08T19:26:56.117060+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:56:54.191245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":35,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.138522+0000","last_change":"2023-11-08T19:26:06.394320+0000","last_active":"2023-11-08T19:26:57.138522+0000","last_peered":"2023-11-08T19:26:57.138522+0000","last_clean":"2023-11-08T19:26:57.138522+0000","last_became_active":"2023-11-08T19:26:04.603212+0000","last_became_peered":"2023-11-08T19:26:04.603212+0000","last_unstale":"2023-11-08T19:26:57.138522+0000","last_undegraded":"2023-11-08T19:26:57.138522+0000","last_fullsized":"2023-11-08T19:26:57.138522+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:25:29.947079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054695399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-08T19:26:58.585584+0000","last_change":"2023-11-08T19:26:57.592878+0000","last_active":"2023-11-08T19:26:58.585584+0000","last_peered":"2023-11-08T19:26:58.585584+0000","last_clean":"2023-11-08T19:26:58.585584+0000","last_became_active":"2023-11-08T19:26:57.592598+0000","last_became_peered":"2023-11-08T19:26:57.592598+0000","last_unstale":"2023-11-08T19:26:58.585584+0000","last_undegraded":"2023-11-08T19:26:58.585584+0000","last_fullsized":"2023-11-08T19:26:58.585584+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:58:39.317164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"61'2","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:58.145630+0000","last_change":"2023-11-08T19:26:57.154544+0000","last_active":"2023-11-08T19:26:58.145630+0000","last_peered":"2023-11-08T19:26:58.145630+0000","last_clean":"2023-11-08T19:26:58.145630+0000","last_became_active":"2023-11-08T19:26:57.154258+0000","last_became_peered":"2023-11-08T19:26:57.154258+0000","last_unstale":"2023-11-08T19:26:58.145630+0000","last_undegraded":"2023-11-08T19:26:58.145630+0000","last_fullsized":"2023-11-08T19:26:58.145630+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_clean_scrub_stamp":"2023-11-08T19:26:56.117060+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:25:46.743393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":37,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.138987+0000","last_change":"2023-11-08T19:26:06.424073+0000","last_active":"2023-11-08T19:26:57.138987+0000","last_peered":"2023-11-08T19:26:57.138987+0000","last_clean":"2023-11-08T19:26:57.138987+0000","last_became_active":"2023-11-08T19:26:04.599046+0000","last_became_peered":"2023-11-08T19:26:04.599046+0000","last_unstale":"2023-11-08T19:26:57.138987+0000","last_undegraded":"2023-11-08T19:26:57.138987+0000","last_fullsized":"2023-11-08T19:26:57.138987+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:04:43.562566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035092199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.597168+0000","last_change":"2023-11-08T19:26:57.594451+0000","last_active":"2023-11-08T19:26:57.597168+0000","last_peered":"2023-11-08T19:26:57.597168+0000","last_clean":"2023-11-08T19:26:57.597168+0000","last_became_active":"2023-11-08T19:26:57.594233+0000","last_became_peered":"2023-11-08T19:26:57.594233+0000","last_unstale":"2023-11-08T19:26:57.597168+0000","last_undegraded":"2023-11-08T19:26:57.597168+0000","last_fullsized":"2023-11-08T19:26:57.597168+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:50:28.981159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"61'2","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:58.145610+0000","last_change":"2023-11-08T19:26:57.146870+0000","last_active":"2023-11-08T19:26:58.145610+0000","last_peered":"2023-11-08T19:26:58.145610+0000","last_clean":"2023-11-08T19:26:58.145610+0000","last_became_active":"2023-11-08T19:26:57.146618+0000","last_became_peered":"2023-11-08T19:26:57.146618+0000","last_unstale":"2023-11-08T19:26:58.145610+0000","last_undegraded":"2023-11-08T19:26:58.145610+0000","last_fullsized":"2023-11-08T19:26:58.145610+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_clean_scrub_stamp":"2023-11-08T19:26:56.117060+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:55:54.246933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":42,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.584338+0000","last_change":"2023-11-08T19:26:05.608980+0000","last_active":"2023-11-08T19:26:57.584338+0000","last_peered":"2023-11-08T19:26:57.584338+0000","last_clean":"2023-11-08T19:26:57.584338+0000","last_became_active":"2023-11-08T19:26:04.606386+0000","last_became_peered":"2023-11-08T19:26:04.606386+0000","last_unstale":"2023-11-08T19:26:57.584338+0000","last_undegraded":"2023-11-08T19:26:57.584338+0000","last_fullsized":"2023-11-08T19:26:57.584338+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:41:50.771457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037950599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.605420+0000","last_change":"2023-11-08T19:26:57.600033+0000","last_active":"2023-11-08T19:26:57.605420+0000","last_peered":"2023-11-08T19:26:57.605420+0000","last_clean":"2023-11-08T19:26:57.605420+0000","last_became_active":"2023-11-08T19:26:57.599766+0000","last_became_peered":"2023-11-08T19:26:57.599766+0000","last_unstale":"2023-11-08T19:26:57.605420+0000","last_undegraded":"2023-11-08T19:26:57.605420+0000","last_fullsized":"2023-11-08T19:26:57.605420+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:25:49.906352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.587451+0000","last_change":"2023-11-08T19:26:57.148628+0000","last_active":"2023-11-08T19:26:57.587451+0000","last_peered":"2023-11-08T19:26:57.587451+0000","last_clean":"2023-11-08T19:26:57.587451+0000","last_became_active":"2023-11-08T19:26:57.148355+0000","last_became_peered":"2023-11-08T19:26:57.148355+0000","last_unstale":"2023-11-08T19:26:57.587451+0000","last_undegraded":"2023-11-08T19:26:57.587451+0000","last_fullsized":"2023-11-08T19:26:57.587451+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_clean_scrub_stamp":"2023-11-08T19:26:56.117060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:35:19.023660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":38,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.598895+0000","last_change":"2023-11-08T19:26:06.541623+0000","last_active":"2023-11-08T19:26:57.598895+0000","last_peered":"2023-11-08T19:26:57.598895+0000","last_clean":"2023-11-08T19:26:57.598895+0000","last_became_active":"2023-11-08T19:26:04.601961+0000","last_became_peered":"2023-11-08T19:26:04.601961+0000","last_unstale":"2023-11-08T19:26:57.598895+0000","last_undegraded":"2023-11-08T19:26:57.598895+0000","last_fullsized":"2023-11-08T19:26:57.598895+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:11:26.508634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000466386,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-08T19:26:58.584423+0000","last_change":"2023-11-08T19:26:57.595480+0000","last_active":"2023-11-08T19:26:58.584423+0000","last_peered":"2023-11-08T19:26:58.584423+0000","last_clean":"2023-11-08T19:26:58.584423+0000","last_became_active":"2023-11-08T19:26:57.595097+0000","last_became_peered":"2023-11-08T19:26:57.595097+0000","last_unstale":"2023-11-08T19:26:58.584423+0000","last_undegraded":"2023-11-08T19:26:58.584423+0000","last_fullsized":"2023-11-08T19:26:58.584423+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:03:53.043225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-08T19:26:58.584678+0000","last_change":"2023-11-08T19:26:57.142090+0000","last_active":"2023-11-08T19:26:58.584678+0000","last_peered":"2023-11-08T19:26:58.584678+0000","last_clean":"2023-11-08T19:26:58.584678+0000","last_became_active":"2023-11-08T19:26:57.141726+0000","last_became_peered":"2023-11-08T19:26:57.141726+0000","last_unstale":"2023-11-08T19:26:58.584678+0000","last_undegraded":"2023-11-08T19:26:58.584678+0000","last_fullsized":"2023-11-08T19:26:58.584678+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_clean_scrub_stamp":"2023-11-08T19:26:56.117060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:45:33.328935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":37,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.139040+0000","last_change":"2023-11-08T19:26:06.424464+0000","last_active":"2023-11-08T19:26:57.139040+0000","last_peered":"2023-11-08T19:26:57.139040+0000","last_clean":"2023-11-08T19:26:57.139040+0000","last_became_active":"2023-11-08T19:26:04.603144+0000","last_became_peered":"2023-11-08T19:26:04.603144+0000","last_unstale":"2023-11-08T19:26:57.139040+0000","last_undegraded":"2023-11-08T19:26:57.139040+0000","last_fullsized":"2023-11-08T19:26:57.139040+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:42:55.269365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054090799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.607827+0000","last_change":"2023-11-08T19:26:57.606630+0000","last_active":"2023-11-08T19:26:57.607827+0000","last_peered":"2023-11-08T19:26:57.607827+0000","last_clean":"2023-11-08T19:26:57.607827+0000","last_became_active":"2023-11-08T19:26:57.606394+0000","last_became_peered":"2023-11-08T19:26:57.606394+0000","last_unstale":"2023-11-08T19:26:57.607827+0000","last_undegraded":"2023-11-08T19:26:57.607827+0000","last_fullsized":"2023-11-08T19:26:57.607827+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:06:39.520196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.587796+0000","last_change":"2023-11-08T19:26:57.148121+0000","last_active":"2023-11-08T19:26:57.587796+0000","last_peered":"2023-11-08T19:26:57.587796+0000","last_clean":"2023-11-08T19:26:57.587796+0000","last_became_active":"2023-11-08T19:26:57.147858+0000","last_became_peered":"2023-11-08T19:26:57.147858+0000","last_unstale":"2023-11-08T19:26:57.587796+0000","last_undegraded":"2023-11-08T19:26:57.587796+0000","last_fullsized":"2023-11-08T19:26:57.587796+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_clean_scrub_stamp":"2023-11-08T19:26:56.117060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:35:07.925172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":36,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.141849+0000","last_change":"2023-11-08T19:26:05.606898+0000","last_active":"2023-11-08T19:26:57.141849+0000","last_peered":"2023-11-08T19:26:57.141849+0000","last_clean":"2023-11-08T19:26:57.141849+0000","last_became_active":"2023-11-08T19:26:04.603955+0000","last_became_peered":"2023-11-08T19:26:04.603955+0000","last_unstale":"2023-11-08T19:26:57.141849+0000","last_undegraded":"2023-11-08T19:26:57.141849+0000","last_fullsized":"2023-11-08T19:26:57.141849+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:19:11.589559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036473499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.600408+0000","last_change":"2023-11-08T19:26:57.600008+0000","last_active":"2023-11-08T19:26:57.600408+0000","last_peered":"2023-11-08T19:26:57.600408+0000","last_clean":"2023-11-08T19:26:57.600408+0000","last_became_active":"2023-11-08T19:26:57.599778+0000","last_became_peered":"2023-11-08T19:26:57.599778+0000","last_unstale":"2023-11-08T19:26:57.600408+0000","last_undegraded":"2023-11-08T19:26:57.600408+0000","last_fullsized":"2023-11-08T19:26:57.600408+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:00:17.866236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.588330+0000","last_change":"2023-11-08T19:26:57.143777+0000","last_active":"2023-11-08T19:26:57.588330+0000","last_peered":"2023-11-08T19:26:57.588330+0000","last_clean":"2023-11-08T19:26:57.588330+0000","last_became_active":"2023-11-08T19:26:57.143173+0000","last_became_peered":"2023-11-08T19:26:57.143173+0000","last_unstale":"2023-11-08T19:26:57.588330+0000","last_undegraded":"2023-11-08T19:26:57.588330+0000","last_fullsized":"2023-11-08T19:26:57.588330+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_clean_scrub_stamp":"2023-11-08T19:26:56.117060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:35:04.669889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":35,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.140339+0000","last_change":"2023-11-08T19:26:06.265686+0000","last_active":"2023-11-08T19:26:57.140339+0000","last_peered":"2023-11-08T19:26:57.140339+0000","last_clean":"2023-11-08T19:26:57.140339+0000","last_became_active":"2023-11-08T19:26:04.602111+0000","last_became_peered":"2023-11-08T19:26:04.602111+0000","last_unstale":"2023-11-08T19:26:57.140339+0000","last_undegraded":"2023-11-08T19:26:57.140339+0000","last_fullsized":"2023-11-08T19:26:57.140339+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:40:11.164951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056197899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.594800+0000","last_change":"2023-11-08T19:26:57.593896+0000","last_active":"2023-11-08T19:26:57.594800+0000","last_peered":"2023-11-08T19:26:57.594800+0000","last_clean":"2023-11-08T19:26:57.594800+0000","last_became_active":"2023-11-08T19:26:57.593676+0000","last_became_peered":"2023-11-08T19:26:57.593676+0000","last_unstale":"2023-11-08T19:26:57.594800+0000","last_undegraded":"2023-11-08T19:26:57.594800+0000","last_fullsized":"2023-11-08T19:26:57.594800+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:48:43.587422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-08T19:26:58.585176+0000","last_change":"2023-11-08T19:26:57.146397+0000","last_active":"2023-11-08T19:26:58.585176+0000","last_peered":"2023-11-08T19:26:58.585176+0000","last_clean":"2023-11-08T19:26:58.585176+0000","last_became_active":"2023-11-08T19:26:57.146037+0000","last_became_peered":"2023-11-08T19:26:57.146037+0000","last_unstale":"2023-11-08T19:26:58.585176+0000","last_undegraded":"2023-11-08T19:26:58.585176+0000","last_fullsized":"2023-11-08T19:26:58.585176+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_clean_scrub_stamp":"2023-11-08T19:26:56.117060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:44:00.118685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":35,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.138609+0000","last_change":"2023-11-08T19:26:06.394075+0000","last_active":"2023-11-08T19:26:57.138609+0000","last_peered":"2023-11-08T19:26:57.138609+0000","last_clean":"2023-11-08T19:26:57.138609+0000","last_became_active":"2023-11-08T19:26:04.606967+0000","last_became_peered":"2023-11-08T19:26:04.606967+0000","last_unstale":"2023-11-08T19:26:57.138609+0000","last_undegraded":"2023-11-08T19:26:57.138609+0000","last_fullsized":"2023-11-08T19:26:57.138609+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:19:56.286835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040971999999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.602438+0000","last_change":"2023-11-08T19:26:57.601327+0000","last_active":"2023-11-08T19:26:57.602438+0000","last_peered":"2023-11-08T19:26:57.602438+0000","last_clean":"2023-11-08T19:26:57.602438+0000","last_became_active":"2023-11-08T19:26:57.600975+0000","last_became_peered":"2023-11-08T19:26:57.600975+0000","last_unstale":"2023-11-08T19:26:57.602438+0000","last_undegraded":"2023-11-08T19:26:57.602438+0000","last_fullsized":"2023-11-08T19:26:57.602438+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:55:27.608415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"61'2","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-08T19:26:58.584883+0000","last_change":"2023-11-08T19:26:57.143247+0000","last_active":"2023-11-08T19:26:58.584883+0000","last_peered":"2023-11-08T19:26:58.584883+0000","last_clean":"2023-11-08T19:26:58.584883+0000","last_became_active":"2023-11-08T19:26:57.142961+0000","last_became_peered":"2023-11-08T19:26:57.142961+0000","last_unstale":"2023-11-08T19:26:58.584883+0000","last_undegraded":"2023-11-08T19:26:58.584883+0000","last_fullsized":"2023-11-08T19:26:58.584883+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_clean_scrub_stamp":"2023-11-08T19:26:56.117060+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:21:03.805677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":42,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.584093+0000","last_change":"2023-11-08T19:26:05.608892+0000","last_active":"2023-11-08T19:26:57.584093+0000","last_peered":"2023-11-08T19:26:57.584093+0000","last_clean":"2023-11-08T19:26:57.584093+0000","last_became_active":"2023-11-08T19:26:04.604519+0000","last_became_peered":"2023-11-08T19:26:04.604519+0000","last_unstale":"2023-11-08T19:26:57.584093+0000","last_undegraded":"2023-11-08T19:26:57.584093+0000","last_fullsized":"2023-11-08T19:26:57.584093+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:11:26.508634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051248099999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-08T19:26:58.585731+0000","last_change":"2023-11-08T19:26:57.592692+0000","last_active":"2023-11-08T19:26:58.585731+0000","last_peered":"2023-11-08T19:26:58.585731+0000","last_clean":"2023-11-08T19:26:58.585731+0000","last_became_active":"2023-11-08T19:26:57.592401+0000","last_became_peered":"2023-11-08T19:26:57.592401+0000","last_unstale":"2023-11-08T19:26:58.585731+0000","last_undegraded":"2023-11-08T19:26:58.585731+0000","last_fullsized":"2023-11-08T19:26:58.585731+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:03:37.833816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.589194+0000","last_change":"2023-11-08T19:26:57.150117+0000","last_active":"2023-11-08T19:26:57.589194+0000","last_peered":"2023-11-08T19:26:57.589194+0000","last_clean":"2023-11-08T19:26:57.589194+0000","last_became_active":"2023-11-08T19:26:57.149674+0000","last_became_peered":"2023-11-08T19:26:57.149674+0000","last_unstale":"2023-11-08T19:26:57.589194+0000","last_undegraded":"2023-11-08T19:26:57.589194+0000","last_fullsized":"2023-11-08T19:26:57.589194+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_clean_scrub_stamp":"2023-11-08T19:26:56.117060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:44:00.118685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":37,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.138897+0000","last_change":"2023-11-08T19:26:06.424236+0000","last_active":"2023-11-08T19:26:57.138897+0000","last_peered":"2023-11-08T19:26:57.138897+0000","last_clean":"2023-11-08T19:26:57.138897+0000","last_became_active":"2023-11-08T19:26:04.600115+0000","last_became_peered":"2023-11-08T19:26:04.600115+0000","last_unstale":"2023-11-08T19:26:57.138897+0000","last_undegraded":"2023-11-08T19:26:57.138897+0000","last_fullsized":"2023-11-08T19:26:57.138897+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:18:01.876605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065803900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-08T19:26:58.585003+0000","last_change":"2023-11-08T19:26:57.607268+0000","last_active":"2023-11-08T19:26:58.585003+0000","last_peered":"2023-11-08T19:26:58.585003+0000","last_clean":"2023-11-08T19:26:58.585003+0000","last_became_active":"2023-11-08T19:26:57.606979+0000","last_became_peered":"2023-11-08T19:26:57.606979+0000","last_unstale":"2023-11-08T19:26:58.585003+0000","last_undegraded":"2023-11-08T19:26:58.585003+0000","last_fullsized":"2023-11-08T19:26:58.585003+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:32:57.602413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.584015+0000","last_change":"2023-11-08T19:26:57.146108+0000","last_active":"2023-11-08T19:26:57.584015+0000","last_peered":"2023-11-08T19:26:57.584015+0000","last_clean":"2023-11-08T19:26:57.584015+0000","last_became_active":"2023-11-08T19:26:57.145813+0000","last_became_peered":"2023-11-08T19:26:57.145813+0000","last_unstale":"2023-11-08T19:26:57.584015+0000","last_undegraded":"2023-11-08T19:26:57.584015+0000","last_fullsized":"2023-11-08T19:26:57.584015+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_clean_scrub_stamp":"2023-11-08T19:26:56.117060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:25:04.442302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":38,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.599951+0000","last_change":"2023-11-08T19:26:06.541360+0000","last_active":"2023-11-08T19:26:57.599951+0000","last_peered":"2023-11-08T19:26:57.599951+0000","last_clean":"2023-11-08T19:26:57.599951+0000","last_became_active":"2023-11-08T19:26:04.603725+0000","last_became_peered":"2023-11-08T19:26:04.603725+0000","last_unstale":"2023-11-08T19:26:57.599951+0000","last_undegraded":"2023-11-08T19:26:57.599951+0000","last_fullsized":"2023-11-08T19:26:57.599951+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:28:48.576742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053365400000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-08T19:26:58.585692+0000","last_change":"2023-11-08T19:26:57.604025+0000","last_active":"2023-11-08T19:26:58.585692+0000","last_peered":"2023-11-08T19:26:58.585692+0000","last_clean":"2023-11-08T19:26:58.585692+0000","last_became_active":"2023-11-08T19:26:57.603781+0000","last_became_peered":"2023-11-08T19:26:57.603781+0000","last_unstale":"2023-11-08T19:26:58.585692+0000","last_undegraded":"2023-11-08T19:26:58.585692+0000","last_fullsized":"2023-11-08T19:26:58.585692+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:45:34.334055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-08T19:26:58.585726+0000","last_change":"2023-11-08T19:26:57.148107+0000","last_active":"2023-11-08T19:26:58.585726+0000","last_peered":"2023-11-08T19:26:58.585726+0000","last_clean":"2023-11-08T19:26:58.585726+0000","last_became_active":"2023-11-08T19:26:57.147847+0000","last_became_peered":"2023-11-08T19:26:57.147847+0000","last_unstale":"2023-11-08T19:26:58.585726+0000","last_undegraded":"2023-11-08T19:26:58.585726+0000","last_fullsized":"2023-11-08T19:26:58.585726+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_clean_scrub_stamp":"2023-11-08T19:26:56.117060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:56:54.191245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":42,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.585071+0000","last_change":"2023-11-08T19:26:05.609824+0000","last_active":"2023-11-08T19:26:57.585071+0000","last_peered":"2023-11-08T19:26:57.585071+0000","last_clean":"2023-11-08T19:26:57.585071+0000","last_became_active":"2023-11-08T19:26:04.603320+0000","last_became_peered":"2023-11-08T19:26:04.603320+0000","last_unstale":"2023-11-08T19:26:57.585071+0000","last_undegraded":"2023-11-08T19:26:57.585071+0000","last_fullsized":"2023-11-08T19:26:57.585071+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:05:45.976615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063225,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.598146+0000","last_change":"2023-11-08T19:26:57.595984+0000","last_active":"2023-11-08T19:26:57.598146+0000","last_peered":"2023-11-08T19:26:57.598146+0000","last_clean":"2023-11-08T19:26:57.598146+0000","last_became_active":"2023-11-08T19:26:57.595187+0000","last_became_peered":"2023-11-08T19:26:57.595187+0000","last_unstale":"2023-11-08T19:26:57.598146+0000","last_undegraded":"2023-11-08T19:26:57.598146+0000","last_fullsized":"2023-11-08T19:26:57.598146+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:25:18.573108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"61'2","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:58.145607+0000","last_change":"2023-11-08T19:26:57.144514+0000","last_active":"2023-11-08T19:26:58.145607+0000","last_peered":"2023-11-08T19:26:58.145607+0000","last_clean":"2023-11-08T19:26:58.145607+0000","last_became_active":"2023-11-08T19:26:57.144231+0000","last_became_peered":"2023-11-08T19:26:57.144231+0000","last_unstale":"2023-11-08T19:26:58.145607+0000","last_undegraded":"2023-11-08T19:26:58.145607+0000","last_fullsized":"2023-11-08T19:26:58.145607+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_clean_scrub_stamp":"2023-11-08T19:26:56.117060+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:49:05.408409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":36,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.147690+0000","last_change":"2023-11-08T19:26:06.296937+0000","last_active":"2023-11-08T19:26:57.147690+0000","last_peered":"2023-11-08T19:26:57.147690+0000","last_clean":"2023-11-08T19:26:57.147690+0000","last_became_active":"2023-11-08T19:26:04.599999+0000","last_became_peered":"2023-11-08T19:26:04.599999+0000","last_unstale":"2023-11-08T19:26:57.147690+0000","last_undegraded":"2023-11-08T19:26:57.147690+0000","last_fullsized":"2023-11-08T19:26:57.147690+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:50:43.827570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025605700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-08T19:26:58.585354+0000","last_change":"2023-11-08T19:26:57.587741+0000","last_active":"2023-11-08T19:26:58.585354+0000","last_peered":"2023-11-08T19:26:58.585354+0000","last_clean":"2023-11-08T19:26:58.585354+0000","last_became_active":"2023-11-08T19:26:57.587468+0000","last_became_peered":"2023-11-08T19:26:57.587468+0000","last_unstale":"2023-11-08T19:26:58.585354+0000","last_undegraded":"2023-11-08T19:26:58.585354+0000","last_fullsized":"2023-11-08T19:26:58.585354+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:37:13.974293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.583790+0000","last_change":"2023-11-08T19:26:57.150200+0000","last_active":"2023-11-08T19:26:57.583790+0000","last_peered":"2023-11-08T19:26:57.583790+0000","last_clean":"2023-11-08T19:26:57.583790+0000","last_became_active":"2023-11-08T19:26:57.149872+0000","last_became_peered":"2023-11-08T19:26:57.149872+0000","last_unstale":"2023-11-08T19:26:57.583790+0000","last_undegraded":"2023-11-08T19:26:57.583790+0000","last_fullsized":"2023-11-08T19:26:57.583790+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_clean_scrub_stamp":"2023-11-08T19:26:56.117060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:45:33.328935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":37,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.138914+0000","last_change":"2023-11-08T19:26:06.424511+0000","last_active":"2023-11-08T19:26:57.138914+0000","last_peered":"2023-11-08T19:26:57.138914+0000","last_clean":"2023-11-08T19:26:57.138914+0000","last_became_active":"2023-11-08T19:26:04.598973+0000","last_became_peered":"2023-11-08T19:26:04.598973+0000","last_unstale":"2023-11-08T19:26:57.138914+0000","last_undegraded":"2023-11-08T19:26:57.138914+0000","last_fullsized":"2023-11-08T19:26:57.138914+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:41:53.497456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031755900000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-08T19:26:58.585138+0000","last_change":"2023-11-08T19:26:57.603772+0000","last_active":"2023-11-08T19:26:58.585138+0000","last_peered":"2023-11-08T19:26:58.585138+0000","last_clean":"2023-11-08T19:26:58.585138+0000","last_became_active":"2023-11-08T19:26:57.603487+0000","last_became_peered":"2023-11-08T19:26:57.603487+0000","last_unstale":"2023-11-08T19:26:58.585138+0000","last_undegraded":"2023-11-08T19:26:58.585138+0000","last_fullsized":"2023-11-08T19:26:58.585138+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:52:20.655917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"61'2","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-08T19:26:58.585095+0000","last_change":"2023-11-08T19:26:57.148644+0000","last_active":"2023-11-08T19:26:58.585095+0000","last_peered":"2023-11-08T19:26:58.585095+0000","last_clean":"2023-11-08T19:26:58.585095+0000","last_became_active":"2023-11-08T19:26:57.148334+0000","last_became_peered":"2023-11-08T19:26:57.148334+0000","last_unstale":"2023-11-08T19:26:58.585095+0000","last_undegraded":"2023-11-08T19:26:58.585095+0000","last_fullsized":"2023-11-08T19:26:58.585095+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_clean_scrub_stamp":"2023-11-08T19:26:56.117060+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:32:56.597293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":36,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.146253+0000","last_change":"2023-11-08T19:26:06.296634+0000","last_active":"2023-11-08T19:26:57.146253+0000","last_peered":"2023-11-08T19:26:57.146253+0000","last_clean":"2023-11-08T19:26:57.146253+0000","last_became_active":"2023-11-08T19:26:04.603088+0000","last_became_peered":"2023-11-08T19:26:04.603088+0000","last_unstale":"2023-11-08T19:26:57.146253+0000","last_undegraded":"2023-11-08T19:26:57.146253+0000","last_fullsized":"2023-11-08T19:26:57.146253+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:01:56.991346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038180899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-08T19:26:58.585061+0000","last_change":"2023-11-08T19:26:57.602122+0000","last_active":"2023-11-08T19:26:58.585061+0000","last_peered":"2023-11-08T19:26:58.585061+0000","last_clean":"2023-11-08T19:26:58.585061+0000","last_became_active":"2023-11-08T19:26:57.601572+0000","last_became_peered":"2023-11-08T19:26:57.601572+0000","last_unstale":"2023-11-08T19:26:58.585061+0000","last_undegraded":"2023-11-08T19:26:58.585061+0000","last_fullsized":"2023-11-08T19:26:58.585061+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:25:47.748513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"61'1","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-08T19:26:58.584790+0000","last_change":"2023-11-08T19:26:57.148794+0000","last_active":"2023-11-08T19:26:58.584790+0000","last_peered":"2023-11-08T19:26:58.584790+0000","last_clean":"2023-11-08T19:26:58.584790+0000","last_became_active":"2023-11-08T19:26:57.148474+0000","last_became_peered":"2023-11-08T19:26:57.148474+0000","last_unstale":"2023-11-08T19:26:58.584790+0000","last_undegraded":"2023-11-08T19:26:58.584790+0000","last_fullsized":"2023-11-08T19:26:58.584790+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_clean_scrub_stamp":"2023-11-08T19:26:56.117060+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:26:41.763289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":35,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.138624+0000","last_change":"2023-11-08T19:26:06.394300+0000","last_active":"2023-11-08T19:26:57.138624+0000","last_peered":"2023-11-08T19:26:57.138624+0000","last_clean":"2023-11-08T19:26:57.138624+0000","last_became_active":"2023-11-08T19:26:04.601685+0000","last_became_peered":"2023-11-08T19:26:04.601685+0000","last_unstale":"2023-11-08T19:26:57.138624+0000","last_undegraded":"2023-11-08T19:26:57.138624+0000","last_fullsized":"2023-11-08T19:26:57.138624+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:51:06.626474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061044099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-08T19:26:58.585687+0000","last_change":"2023-11-08T19:26:57.593050+0000","last_active":"2023-11-08T19:26:58.585687+0000","last_peered":"2023-11-08T19:26:58.585687+0000","last_clean":"2023-11-08T19:26:58.585687+0000","last_became_active":"2023-11-08T19:26:57.592759+0000","last_became_peered":"2023-11-08T19:26:57.592759+0000","last_unstale":"2023-11-08T19:26:58.585687+0000","last_undegraded":"2023-11-08T19:26:58.585687+0000","last_fullsized":"2023-11-08T19:26:58.585687+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:40:19.853844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.587822+0000","last_change":"2023-11-08T19:26:57.148765+0000","last_active":"2023-11-08T19:26:57.587822+0000","last_peered":"2023-11-08T19:26:57.587822+0000","last_clean":"2023-11-08T19:26:57.587822+0000","last_became_active":"2023-11-08T19:26:57.148468+0000","last_became_peered":"2023-11-08T19:26:57.148468+0000","last_unstale":"2023-11-08T19:26:57.587822+0000","last_undegraded":"2023-11-08T19:26:57.587822+0000","last_fullsized":"2023-11-08T19:26:57.587822+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_clean_scrub_stamp":"2023-11-08T19:26:56.117060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:26:21.727377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":42,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.584432+0000","last_change":"2023-11-08T19:26:05.609570+0000","last_active":"2023-11-08T19:26:57.584432+0000","last_peered":"2023-11-08T19:26:57.584432+0000","last_clean":"2023-11-08T19:26:57.584432+0000","last_became_active":"2023-11-08T19:26:04.608379+0000","last_became_peered":"2023-11-08T19:26:04.608379+0000","last_unstale":"2023-11-08T19:26:57.584432+0000","last_undegraded":"2023-11-08T19:26:57.584432+0000","last_fullsized":"2023-11-08T19:26:57.584432+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:40:08.986649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000529657,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.608296+0000","last_change":"2023-11-08T19:26:57.604293+0000","last_active":"2023-11-08T19:26:57.608296+0000","last_peered":"2023-11-08T19:26:57.608296+0000","last_clean":"2023-11-08T19:26:57.608296+0000","last_became_active":"2023-11-08T19:26:57.603462+0000","last_became_peered":"2023-11-08T19:26:57.603462+0000","last_unstale":"2023-11-08T19:26:57.608296+0000","last_undegraded":"2023-11-08T19:26:57.608296+0000","last_fullsized":"2023-11-08T19:26:57.608296+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:20:08.283676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"61'1","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-08T19:26:58.584751+0000","last_change":"2023-11-08T19:26:57.154388+0000","last_active":"2023-11-08T19:26:58.584751+0000","last_peered":"2023-11-08T19:26:58.584751+0000","last_clean":"2023-11-08T19:26:58.584751+0000","last_became_active":"2023-11-08T19:26:57.154161+0000","last_became_peered":"2023-11-08T19:26:57.154161+0000","last_unstale":"2023-11-08T19:26:58.584751+0000","last_undegraded":"2023-11-08T19:26:58.584751+0000","last_fullsized":"2023-11-08T19:26:58.584751+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_clean_scrub_stamp":"2023-11-08T19:26:56.117060+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:15:09.411596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":35,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.135452+0000","last_change":"2023-11-08T19:26:06.265419+0000","last_active":"2023-11-08T19:26:57.135452+0000","last_peered":"2023-11-08T19:26:57.135452+0000","last_clean":"2023-11-08T19:26:57.135452+0000","last_became_active":"2023-11-08T19:26:04.605333+0000","last_became_peered":"2023-11-08T19:26:04.605333+0000","last_unstale":"2023-11-08T19:26:57.135452+0000","last_undegraded":"2023-11-08T19:26:57.135452+0000","last_fullsized":"2023-11-08T19:26:57.135452+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:01:56.991346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064606400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-08T19:26:58.585469+0000","last_change":"2023-11-08T19:26:57.594992+0000","last_active":"2023-11-08T19:26:58.585469+0000","last_peered":"2023-11-08T19:26:58.585469+0000","last_clean":"2023-11-08T19:26:58.585469+0000","last_became_active":"2023-11-08T19:26:57.594693+0000","last_became_peered":"2023-11-08T19:26:57.594693+0000","last_unstale":"2023-11-08T19:26:58.585469+0000","last_undegraded":"2023-11-08T19:26:58.585469+0000","last_fullsized":"2023-11-08T19:26:58.585469+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:00:17.866236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.586797+0000","last_change":"2023-11-08T19:26:57.150678+0000","last_active":"2023-11-08T19:26:57.586797+0000","last_peered":"2023-11-08T19:26:57.586797+0000","last_clean":"2023-11-08T19:26:57.586797+0000","last_became_active":"2023-11-08T19:26:57.150388+0000","last_became_peered":"2023-11-08T19:26:57.150388+0000","last_unstale":"2023-11-08T19:26:57.586797+0000","last_undegraded":"2023-11-08T19:26:57.586797+0000","last_fullsized":"2023-11-08T19:26:57.586797+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_clean_scrub_stamp":"2023-11-08T19:26:56.117060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:25:17.567988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":38,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.599249+0000","last_change":"2023-11-08T19:26:06.542044+0000","last_active":"2023-11-08T19:26:57.599249+0000","last_peered":"2023-11-08T19:26:57.599249+0000","last_clean":"2023-11-08T19:26:57.599249+0000","last_became_active":"2023-11-08T19:26:04.603056+0000","last_became_peered":"2023-11-08T19:26:04.603056+0000","last_unstale":"2023-11-08T19:26:57.599249+0000","last_undegraded":"2023-11-08T19:26:57.599249+0000","last_fullsized":"2023-11-08T19:26:57.599249+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:41:50.771457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000517414,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":208,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.584161+0000","last_change":"2023-11-08T19:24:45.888122+0000","last_active":"2023-11-08T19:26:57.584161+0000","last_peered":"2023-11-08T19:26:57.584161+0000","last_clean":"2023-11-08T19:26:57.584161+0000","last_became_active":"2023-11-08T19:24:45.887048+0000","last_became_peered":"2023-11-08T19:24:45.887048+0000","last_unstale":"2023-11-08T19:26:57.584161+0000","last_undegraded":"2023-11-08T19:26:57.584161+0000","last_fullsized":"2023-11-08T19:26:57.584161+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:44.575133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:44.575133+0000","last_clean_scrub_stamp":"2023-11-08T19:24:44.575133+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:19:20.202062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574879,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28356,"kb_used_data":1324,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343484,"statfs":{"total":96636764160,"available":96607727616,"internally_reserved":0,"allocated":1355776,"data_stored":916851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:26:48 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.504,"15min":0.498},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.78600000000000003,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.51700000000000002,"15min":0.51500000000000001},"min":{"1min":0.35099999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.65400000000000003,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.69099999999999995}]},{"osd":1,"last update":"Wed Nov 8 19:26:48 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.56999999999999995,"15min":0.56599999999999995},"min":{"1min":0.42099999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.87},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.56100000000000005,"15min":0.55500000000000005},"min":{"1min":0.432,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.76100000000000001,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.89500000000000002}]},{"osd":2,"last update":"Wed Nov 8 19:26:48 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.501,"15min":0.49099999999999999},"min":{"1min":0.39500000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.55400000000000005,"15min":0.54900000000000004},"min":{"1min":0.42999999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.80300000000000005}]},{"osd":3,"last update":"Wed Nov 8 19:26:48 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56000000000000005,"15min":0.55900000000000005},"min":{"1min":0.442,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.58899999999999997,"15min":0.57699999999999996},"min":{"1min":0.47899999999999998,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.77000000000000002}]},{"osd":4,"last update":"Wed Nov 8 19:26:48 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.51600000000000001,"15min":0.51000000000000001},"min":{"1min":0.38300000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.73299999999999998,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.54500000000000004,"15min":0.54100000000000004},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.753}]},{"osd":5,"last update":"Wed Nov 8 19:26:48 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.58699999999999997,"15min":0.58099999999999996},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.64700000000000002,"15min":0.64300000000000002},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.78300000000000003,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.84499999999999997}]},{"osd":6,"last update":"Wed Nov 8 19:26:48 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.60999999999999999,"15min":0.60699999999999998},"min":{"1min":0.47599999999999998,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.81100000000000005,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.61699999999999999,"15min":0.61199999999999999},"min":{"1min":0.47699999999999998,"5min":0.443,"15min":0.443},"max":{"1min":0.78800000000000003,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.82999999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574879,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27760,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344080,"statfs":{"total":96636764160,"available":96608337920,"internally_reserved":0,"allocated":749568,"data_stored":325741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:26:47 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":1.26,"15min":1.3799999999999999},"min":{"1min":0.33900000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.91700000000000004,"5min":21.359999999999999,"15min":21.359999999999999},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.86099999999999999,"15min":0.91300000000000003},"min":{"1min":0.38100000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.90300000000000002,"5min":10.329000000000001,"15min":10.329000000000001},"last":0.628}]},{"osd":1,"last update":"Wed Nov 8 19:26:47 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.60299999999999998,"15min":0.61699999999999999},"min":{"1min":0.35999999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.86299999999999999,"5min":3.4239999999999999,"15min":3.4239999999999999},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.63100000000000001,"15min":0.64300000000000002},"min":{"1min":0.40999999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.94999999999999996,"5min":3.4809999999999999,"15min":3.4809999999999999},"last":0.56100000000000005}]},{"osd":2,"last update":"Wed Nov 8 19:26:47 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.879,"15min":0.93100000000000005},"min":{"1min":0.38500000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.96399999999999997,"5min":10.398,"15min":10.398},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.67800000000000005,"15min":0.69799999999999995},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.89100000000000001,"5min":5.0170000000000003,"15min":5.0170000000000003},"last":0.40799999999999997}]},{"osd":3,"last update":"Wed Nov 8 19:26:47 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.65600000000000003,"15min":0.66900000000000004},"min":{"1min":0.34599999999999997,"5min":0.317,"15min":0.317},"max":{"1min":0.877,"5min":3.5710000000000002,"15min":3.5710000000000002},"last":0.496},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.91000000000000003,"15min":0.96399999999999997},"min":{"1min":0.41099999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.93500000000000005,"5min":10.368,"15min":10.368},"last":0.54000000000000004}]},{"osd":4,"last update":"Wed Nov 8 19:26:47 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.52500000000000002,"15min":0.51300000000000001},"min":{"1min":0.36899999999999999,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.56499999999999995,"15min":0.56000000000000005},"min":{"1min":0.39200000000000002,"5min":0.373,"15min":0.373},"max":{"1min":0.753,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.47899999999999998}]},{"osd":5,"last update":"Wed Nov 8 19:26:47 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.53900000000000003,"15min":0.52900000000000003},"min":{"1min":0.41599999999999998,"5min":0.307,"15min":0.307},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.58999999999999997,"15min":0.58099999999999996},"min":{"1min":0.51600000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.64000000000000001}]},{"osd":7,"last update":"Wed Nov 8 19:26:47 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.58499999999999996,"15min":0.57499999999999996},"min":{"1min":0.48599999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.84199999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.59799999999999998,"15min":0.58999999999999997},"min":{"1min":0.498,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.80500000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.57799999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574879,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27768,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344072,"statfs":{"total":96636764160,"available":96608329728,"internally_reserved":0,"allocated":761856,"data_stored":321430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:26:47 2023","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.377,"15min":0.38100000000000001},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.499,"5min":1.389,"15min":1.389},"last":0.28899999999999998},{"interface":"front","average":{"1min":0.34899999999999998,"5min":0.66500000000000004,"15min":0.71699999999999997},"min":{"1min":0.23499999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.55200000000000005,"5min":9.1370000000000005,"15min":9.1370000000000005},"last":0.25900000000000001}]},{"osd":2,"last update":"Wed Nov 8 19:26:47 2023","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.35999999999999999,"15min":0.36499999999999999},"min":{"1min":0.25600000000000001,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.47799999999999998,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.42299999999999999,"15min":0.42899999999999999},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.56399999999999995,"5min":1.425,"15min":1.425},"last":0.36099999999999999}]},{"osd":3,"last update":"Wed Nov 8 19:26:47 2023","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.80700000000000005,"15min":0.877},"min":{"1min":0.28000000000000003,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.54300000000000004,"5min":11.957000000000001,"15min":11.957000000000001},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.379,"5min":1.0209999999999999,"15min":1.1279999999999999},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.54400000000000004,"5min":18.039000000000001,"15min":18.039000000000001},"last":0.312}]},{"osd":4,"last update":"Wed Nov 8 19:26:47 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.78300000000000003,"15min":0.81399999999999995},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.4370000000000001,"5min":6.181,"15min":6.181},"last":0.442},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.64300000000000002,"15min":0.64100000000000001},"min":{"1min":0.45200000000000001,"5min":0.377,"15min":0.377},"max":{"1min":1.411,"5min":1.488,"15min":1.488},"last":0.498}]},{"osd":5,"last update":"Wed Nov 8 19:26:47 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.92000000000000004,"15min":0.96399999999999997},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.421,"5min":9.2420000000000009,"15min":9.2420000000000009},"last":0.437},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.83299999999999996,"15min":0.86499999999999999},"min":{"1min":0.439,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.2969999999999999,"5min":6.2110000000000003,"15min":6.2110000000000003},"last":0.51200000000000001}]},{"osd":6,"last update":"Wed Nov 8 19:26:47 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.92800000000000005,"15min":0.96699999999999997},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.4530000000000001,"5min":9.1880000000000006,"15min":9.1880000000000006},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.272,"5min":1.462,"15min":1.462},"last":0.52900000000000003}]},{"osd":7,"last update":"Wed Nov 8 19:26:47 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.82699999999999996,"15min":0.85499999999999998},"min":{"1min":0.46899999999999997,"5min":0.435,"15min":0.435},"max":{"1min":1.3979999999999999,"5min":6.1479999999999997,"15min":6.1479999999999997},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.95699999999999996,"15min":1.004},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.353,"5min":9.2170000000000005,"15min":9.2170000000000005},"last":0.46100000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574879,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28324,"kb_used_data":1296,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343516,"statfs":{"total":96636764160,"available":96607760384,"internally_reserved":0,"allocated":1327104,"data_stored":909422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Nov 8 19:26:46 2023","interfaces":[{"interface":"back","average":{"1min":0.31900000000000001,"5min":0.48199999999999998,"15min":0.50900000000000001},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.45200000000000001,"5min":4.1200000000000001,"15min":4.1200000000000001},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.36199999999999999,"5min":1.026,"15min":1.137},"min":{"1min":0.26200000000000001,"5min":0.253,"15min":0.253},"max":{"1min":0.52200000000000002,"5min":16.940999999999999,"15min":16.940999999999999},"last":0.317}]},{"osd":2,"last update":"Wed Nov 8 19:26:46 2023","interfaces":[{"interface":"back","average":{"1min":0.34200000000000003,"5min":0.39100000000000001,"15min":0.39900000000000002},"min":{"1min":0.25700000000000001,"5min":0.246,"15min":0.246},"max":{"1min":0.54500000000000004,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.22},{"interface":"front","average":{"1min":0.378,"5min":0.64500000000000002,"15min":0.68899999999999995},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.56899999999999995,"5min":7.0209999999999999,"15min":7.0209999999999999},"last":0.27300000000000002}]},{"osd":3,"last update":"Wed Nov 8 19:26:46 2023","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.53700000000000003,"15min":0.56299999999999994},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.51500000000000001,"5min":4.0940000000000003,"15min":4.0940000000000003},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.439,"15min":0.44700000000000001},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.62,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.29199999999999998}]},{"osd":4,"last update":"Wed Nov 8 19:26:46 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.73599999999999999,"15min":0.76600000000000001},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.70799999999999996,"5min":4.0049999999999999,"15min":4.0049999999999999},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.63400000000000001,"15min":0.64000000000000001},"min":{"1min":0.45700000000000002,"5min":0.436,"15min":0.436},"max":{"1min":0.79200000000000004,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.51300000000000001}]},{"osd":5,"last update":"Wed Nov 8 19:26:46 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.753,"15min":0.78000000000000003},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.81699999999999995,"5min":4.3090000000000002,"15min":4.3090000000000002},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.745,"15min":0.76900000000000002},"min":{"1min":0.496,"5min":0.437,"15min":0.437},"max":{"1min":0.77200000000000002,"5min":4.0709999999999997,"15min":4.0709999999999997},"last":0.69999999999999996}]},{"osd":6,"last update":"Wed Nov 8 19:26:46 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.66500000000000004,"15min":0.67300000000000004},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.82299999999999995,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.80900000000000005,"15min":0.83799999999999997},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.76000000000000001,"5min":4.2560000000000002,"15min":4.2560000000000002},"last":0.62}]},{"osd":7,"last update":"Wed Nov 8 19:26:46 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.76600000000000001,"15min":0.79000000000000004},"min":{"1min":0.49299999999999999,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.81299999999999994,"5min":4.2850000000000001,"15min":4.2850000000000001},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.79200000000000004,"15min":0.81799999999999995},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.85699999999999998,"5min":4.0460000000000003,"15min":4.0460000000000003},"last":0.71699999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574879,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27740,"kb_used_data":716,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344100,"statfs":{"total":96636764160,"available":96608358400,"internally_reserved":0,"allocated":733184,"data_stored":319054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:26:46 2023","interfaces":[{"interface":"back","average":{"1min":0.34300000000000003,"5min":0.33400000000000002,"15min":0.33200000000000002},"min":{"1min":0.254,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.496,"5min":0.496,"15min":0.496},"last":0.21099999999999999},{"interface":"front","average":{"1min":0.309,"5min":0.58899999999999997,"15min":0.63600000000000001},"min":{"1min":0.23400000000000001,"5min":0.218,"15min":0.218},"max":{"1min":0.45600000000000002,"5min":6.0999999999999996,"15min":6.0999999999999996},"last":0.23000000000000001}]},{"osd":1,"last update":"Wed Nov 8 19:26:46 2023","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.44,"5min":0.48299999999999998,"15min":0.48299999999999998},"last":0.27100000000000002},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.38600000000000001,"15min":0.38900000000000001},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.48899999999999999,"5min":0.624,"15min":0.624},"last":0.28699999999999998}]},{"osd":3,"last update":"Wed Nov 8 19:26:46 2023","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.38100000000000001,"15min":0.38400000000000001},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.46000000000000002,"5min":0.53700000000000003,"15min":0.53700000000000003},"last":0.317},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.39000000000000001,"15min":0.39400000000000002},"min":{"1min":0.29399999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.439,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.30099999999999999}]},{"osd":4,"last update":"Wed Nov 8 19:26:46 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.57899999999999996,"15min":0.58299999999999996},"min":{"1min":0.46000000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.71999999999999997,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.57999999999999996,"15min":0.58299999999999996},"min":{"1min":0.40999999999999998,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.74199999999999999,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.498}]},{"osd":5,"last update":"Wed Nov 8 19:26:46 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.58599999999999997,"15min":0.58299999999999996},"min":{"1min":0.50700000000000001,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.76400000000000001,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.60899999999999999,"15min":0.61199999999999999},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.76100000000000001,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.41999999999999998}]},{"osd":6,"last update":"Wed Nov 8 19:26:46 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.59799999999999998,"15min":0.59999999999999998},"min":{"1min":0.47699999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.77300000000000002,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.40300000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.70699999999999996,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.55000000000000004}]},{"osd":7,"last update":"Wed Nov 8 19:26:46 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.58199999999999996,"15min":0.57999999999999996},"min":{"1min":0.497,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.81499999999999995,"5min":0.878,"15min":0.878},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.60099999999999998,"15min":0.60299999999999998},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.80000000000000004,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.53100000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574880,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27740,"kb_used_data":716,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344100,"statfs":{"total":96636764160,"available":96608358400,"internally_reserved":0,"allocated":733184,"data_stored":319054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:26:49 2023","interfaces":[{"interface":"back","average":{"1min":0.32100000000000001,"5min":0.33800000000000002,"15min":0.34100000000000003},"min":{"1min":0.20300000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.42299999999999999,"5min":0.626,"15min":0.626},"last":0.30399999999999999},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.34999999999999998,"15min":0.34899999999999998},"min":{"1min":0.28100000000000003,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.46899999999999997,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.33100000000000002}]},{"osd":1,"last update":"Wed Nov 8 19:26:49 2023","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.35099999999999998,"15min":0.35299999999999998},"min":{"1min":0.24099999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.42699999999999999,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"min":{"1min":0.27900000000000003,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.49199999999999999,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.36399999999999999}]},{"osd":2,"last update":"Wed Nov 8 19:26:49 2023","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.374,"15min":0.371},"min":{"1min":0.30199999999999999,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.48199999999999998,"5min":0.58499999999999996,"15min":0.58499999999999996},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.40500000000000003,"15min":0.40400000000000003},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"last":0.40100000000000002}]},{"osd":4,"last update":"Wed Nov 8 19:26:49 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.60399999999999998,"15min":0.59999999999999998},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.80200000000000005,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.628},{"interface":"front","average":{"1min":0.624,"5min":0.60799999999999998,"15min":0.60599999999999998},"min":{"1min":0.45000000000000001,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.79000000000000004,"5min":0.871,"15min":0.871},"last":0.67800000000000005}]},{"osd":5,"last update":"Wed Nov 8 19:26:49 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.58299999999999996,"15min":0.57699999999999996},"min":{"1min":0.39800000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.625,"5min":0.58599999999999997,"15min":0.57999999999999996},"min":{"1min":0.499,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.81999999999999995,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.59099999999999997}]},{"osd":6,"last update":"Wed Nov 8 19:26:49 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.58399999999999996,"15min":0.58099999999999996},"min":{"1min":0.42999999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.61299999999999999,"15min":0.60699999999999998},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.84099999999999997,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.52900000000000003}]},{"osd":7,"last update":"Wed Nov 8 19:26:49 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.59599999999999997,"15min":0.58999999999999997},"min":{"1min":0.46400000000000002,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.88100000000000001,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.59799999999999998,"15min":0.59599999999999997},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.76600000000000001,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.57299999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574879,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27752,"kb_used_data":728,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344088,"statfs":{"total":96636764160,"available":96608346112,"internally_reserved":0,"allocated":745472,"data_stored":320666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:26:50 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.73099999999999998,"15min":0.76000000000000001},"min":{"1min":0.40999999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.78100000000000003,"5min":6.5970000000000004,"15min":6.5970000000000004},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.56599999999999995,"15min":0.57399999999999995},"min":{"1min":0.378,"5min":0.317,"15min":0.317},"max":{"1min":0.75900000000000001,"5min":2.5030000000000001,"15min":2.5030000000000001},"last":0.436}]},{"osd":1,"last update":"Wed Nov 8 19:26:50 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.50700000000000001,"15min":0.499},"min":{"1min":0.372,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.51600000000000001,"15min":0.51000000000000001},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.46400000000000002}]},{"osd":2,"last update":"Wed Nov 8 19:26:50 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.55500000000000005,"15min":0.55200000000000005},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.623,"5min":0.80100000000000005,"15min":0.83099999999999996},"min":{"1min":0.45900000000000002,"5min":0.435,"15min":0.435},"max":{"1min":0.871,"5min":6.6920000000000002,"15min":6.6920000000000002},"last":0.38600000000000001}]},{"osd":3,"last update":"Wed Nov 8 19:26:50 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58799999999999997,"15min":0.58899999999999997},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.71099999999999997,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.627,"5min":0.61399999999999999,"15min":0.61199999999999999},"min":{"1min":0.46800000000000003,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.48799999999999999}]},{"osd":5,"last update":"Wed Nov 8 19:26:50 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.34300000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.59099999999999997,"15min":0.59499999999999997},"min":{"1min":0.42999999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.876,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.505}]},{"osd":6,"last update":"Wed Nov 8 19:26:50 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.61799999999999999,"15min":0.61499999999999999},"min":{"1min":0.38900000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.78100000000000003,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.56200000000000006}]},{"osd":7,"last update":"Wed Nov 8 19:26:50 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.625,"15min":0.63200000000000001},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.82699999999999996,"5min":1.141,"15min":1.141},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.63900000000000001,"15min":0.64500000000000002},"min":{"1min":0.38700000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.85299999999999998,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.58899999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574879,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27744,"kb_used_data":720,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344096,"statfs":{"total":96636764160,"available":96608354304,"internally_reserved":0,"allocated":737280,"data_stored":319076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:26:44 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.51700000000000002,"15min":0.52100000000000002},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.502},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.51100000000000001,"15min":0.50900000000000001},"min":{"1min":0.373,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.65100000000000002,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.58499999999999996}]},{"osd":1,"last update":"Wed Nov 8 19:26:44 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.52200000000000002,"15min":0.52300000000000002},"min":{"1min":0.42099999999999999,"5min":0.371,"15min":0.371},"max":{"1min":0.65300000000000002,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53500000000000003,"15min":0.53600000000000003},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.44800000000000001}]},{"osd":2,"last update":"Wed Nov 8 19:26:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.53800000000000003,"15min":0.53600000000000003},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.56599999999999995,"15min":0.57299999999999995},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.69599999999999995,"5min":0.747,"15min":0.747},"last":0.627}]},{"osd":3,"last update":"Wed Nov 8 19:26:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.54600000000000004,"15min":0.54900000000000004},"min":{"1min":0.39700000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.59099999999999997,"15min":0.59999999999999998},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.70099999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.42799999999999999}]},{"osd":4,"last update":"Wed Nov 8 19:26:44 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.55100000000000005,"15min":0.55900000000000005},"min":{"1min":0.38600000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.63300000000000001,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.39100000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.71399999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.40999999999999998}]},{"osd":6,"last update":"Wed Nov 8 19:26:44 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.60899999999999999,"15min":0.61199999999999999},"min":{"1min":0.42099999999999999,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.61099999999999999,"15min":0.61499999999999999},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.67700000000000005,"5min":0.745,"15min":0.745},"last":0.59999999999999998}]},{"osd":7,"last update":"Wed Nov 8 19:26:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.60299999999999998,"15min":0.60899999999999999},"min":{"1min":0.371,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.63900000000000001,"15min":0.64700000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.77100000000000002,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.57499999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-08T19:27:02.237 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-08T19:27:02.389 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.386+0000 7f27fcf1f700 1 Processor -- start 2023-11-08T19:27:02.389 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.386+0000 7f27fcf1f700 1 -- start start 2023-11-08T19:27:02.393 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.387+0000 7f27fcf1f700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f27f80cf090 0x7f27f80d1f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:02.393 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.387+0000 7f27fcf1f700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27f80d3200 con 0x7f27f815dc80 2023-11-08T19:27:02.393 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.387+0000 7f27fcf1f700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27f815e060 con 0x7f27f80d3690 2023-11-08T19:27:02.394 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.387+0000 7f27fcf1f700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27f80d25f0 con 0x7f27f80cf090 2023-11-08T19:27:02.394 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.387+0000 7f27f659c700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f27f815dc80 0x7f27f80ce980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:48776/0 (socket says 172.21.15.143:48776) 2023-11-08T19:27:02.394 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.387+0000 7f27f659c700 1 -- 172.21.15.143:0/3081562534 learned_addr learned my addr 172.21.15.143:0/3081562534 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:27:02.394 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.388+0000 7f27f559a700 1 -- 172.21.15.143:0/3081562534 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3806410970 0 0) 0x7f27f80d3200 con 0x7f27f815dc80 2023-11-08T19:27:02.395 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.388+0000 7f27f5d9b700 1 --2- 172.21.15.143:0/3081562534 >> v2:172.21.15.174:3300/0 conn(0x7f27f80cf090 0x7f27f80d1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:02.395 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.388+0000 7f27f559a700 1 -- 172.21.15.143:0/3081562534 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f27e00035b0 con 0x7f27f815dc80 2023-11-08T19:27:02.395 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.388+0000 7f27f559a700 1 -- 172.21.15.143:0/3081562534 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 51099802 0 0) 0x7f27e00035b0 con 0x7f27f815dc80 2023-11-08T19:27:02.395 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.388+0000 7f27f559a700 1 -- 172.21.15.143:0/3081562534 >> v2:172.21.15.174:3300/0 conn(0x7f27f80cf090 msgr2=0x7f27f80d1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:02.395 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.388+0000 7f27f559a700 1 --2- 172.21.15.143:0/3081562534 >> v2:172.21.15.174:3300/0 conn(0x7f27f80cf090 0x7f27f80d1f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:02.396 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.388+0000 7f27f559a700 1 -- 172.21.15.143:0/3081562534 >> v1:172.21.15.174:6789/0 conn(0x7f27f80d3690 legacy=0x7f27f80b6ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:02.396 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.388+0000 7f27f559a700 1 -- 172.21.15.143:0/3081562534 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27f80d28d0 con 0x7f27f815dc80 2023-11-08T19:27:02.396 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.388+0000 7f27f559a700 1 -- 172.21.15.143:0/3081562534 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4147756303 0 0) 0x7f27f0001670 con 0x7f27f815dc80 2023-11-08T19:27:02.396 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.388+0000 7f27f559a700 1 -- 172.21.15.143:0/3081562534 >> v1:172.21.15.143:6789/0 conn(0x7f27f815dc80 legacy=0x7f27f80ce980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:02.396 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.388+0000 7f27f559a700 1 --2- 172.21.15.143:0/3081562534 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f27e0003a00 0x7f27e0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:02.397 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.388+0000 7f27f559a700 1 --2- 172.21.15.143:0/3081562534 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f27e0004850 0x7f27e0004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:02.397 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.388+0000 7f27f559a700 1 --2- 172.21.15.143:0/3081562534 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f27e0005210 0x7f27e000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:02.397 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.388+0000 7f27f559a700 1 -- 172.21.15.143:0/3081562534 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27f80d25f0 con 0x7f27e0005210 2023-11-08T19:27:02.398 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.388+0000 7f27f559a700 1 -- 172.21.15.143:0/3081562534 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27e000c1b0 con 0x7f27e0003a00 2023-11-08T19:27:02.398 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.388+0000 7f27f559a700 1 -- 172.21.15.143:0/3081562534 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27e000c370 con 0x7f27e0004850 2023-11-08T19:27:02.398 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.388+0000 7f27f659c700 1 --2- 172.21.15.143:0/3081562534 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f27e0003a00 0x7f27e0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:02.398 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.388+0000 7f27f6d9d700 1 --2- 172.21.15.143:0/3081562534 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f27e0005210 0x7f27e000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:02.398 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.388+0000 7f27f5d9b700 1 --2- 172.21.15.143:0/3081562534 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f27e0004850 0x7f27e0004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:02.399 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.388+0000 7f27f6d9d700 1 -- 172.21.15.143:0/3081562534 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f27e0004850 msgr2=0x7f27e0004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:02.399 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.388+0000 7f27f6d9d700 1 --2- 172.21.15.143:0/3081562534 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f27e0004850 0x7f27e0004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:02.399 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.388+0000 7f27f6d9d700 1 -- 172.21.15.143:0/3081562534 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f27e0003a00 msgr2=0x7f27e0003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:02.399 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.388+0000 7f27f6d9d700 1 --2- 172.21.15.143:0/3081562534 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f27e0003a00 0x7f27e0003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:02.399 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.388+0000 7f27f6d9d700 1 -- 172.21.15.143:0/3081562534 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27e00035b0 con 0x7f27e0005210 2023-11-08T19:27:02.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.388+0000 7f27f6d9d700 1 -- 172.21.15.143:0/3081562534 reap_dead start 2023-11-08T19:27:02.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.389+0000 7f27f6d9d700 1 --2- 172.21.15.143:0/3081562534 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f27e0005210 0x7f27e000ba10 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f27e800c160 tx=0x7f27e0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e0e0bc540168c9b server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:02.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.389+0000 7f27f559a700 1 -- 172.21.15.143:0/3081562534 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27e8001db0 con 0x7f27e0005210 2023-11-08T19:27:02.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.389+0000 7f27f559a700 1 -- 172.21.15.143:0/3081562534 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27e800ccc0 con 0x7f27e0005210 2023-11-08T19:27:02.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.389+0000 7f27f559a700 1 -- 172.21.15.143:0/3081562534 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27e8003c40 con 0x7f27e0005210 2023-11-08T19:27:02.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.389+0000 7f27fcf1f700 1 -- 172.21.15.143:0/3081562534 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f27e0005210 msgr2=0x7f27e000ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:02.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.389+0000 7f27fcf1f700 1 --2- 172.21.15.143:0/3081562534 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f27e0005210 0x7f27e000ba10 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f27e800c160 tx=0x7f27e0005880 comp rx=0 tx=0).stop 2023-11-08T19:27:02.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.389+0000 7f27fcf1f700 1 -- 172.21.15.143:0/3081562534 shutdown_connections 2023-11-08T19:27:02.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.389+0000 7f27fcf1f700 1 --2- 172.21.15.143:0/3081562534 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f27e0005210 0x7f27e000ba10 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:02.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.389+0000 7f27fcf1f700 1 -- 172.21.15.143:0/3081562534 >> 172.21.15.143:0/3081562534 conn(0x7f27f80668e0 msgr2=0x7f27f8066cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:02.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.389+0000 7f27fcf1f700 1 -- 172.21.15.143:0/3081562534 shutdown_connections 2023-11-08T19:27:02.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.389+0000 7f27fcf1f700 1 -- 172.21.15.143:0/3081562534 wait complete. 2023-11-08T19:27:02.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.391+0000 7f27fcf1f700 1 Processor -- start 2023-11-08T19:27:02.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.391+0000 7f27fcf1f700 1 -- start start 2023-11-08T19:27:02.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.391+0000 7f27fcf1f700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f27e0005210 0x7f27f8153c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:02.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.391+0000 7f27fcf1f700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f27f81f4280 0x7f27f8160af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:02.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.391+0000 7f27fcf1f700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f27f80b5810 0x7f27f80b5c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:02.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.391+0000 7f27fcf1f700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27f80b61d0 con 0x7f27e0005210 2023-11-08T19:27:02.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.391+0000 7f27fcf1f700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27f81f4700 con 0x7f27f80b5810 2023-11-08T19:27:02.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.391+0000 7f27fcf1f700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27f8155be0 con 0x7f27f81f4280 2023-11-08T19:27:02.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.391+0000 7f27f659c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f27e0005210 0x7f27f8153c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:02.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.391+0000 7f27f659c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f27e0005210 0x7f27f8153c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52634/0 (socket says 172.21.15.143:52634) 2023-11-08T19:27:02.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.391+0000 7f27f659c700 1 -- 172.21.15.143:0/2266786012 learned_addr learned my addr 172.21.15.143:0/2266786012 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:02.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.392+0000 7f27f659c700 1 -- 172.21.15.143:0/2266786012 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f27f81f4280 msgr2=0x7f27f8160af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:02.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.392+0000 7f27f659c700 1 --2- 172.21.15.143:0/2266786012 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f27f81f4280 0x7f27f8160af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:02.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.392+0000 7f27f659c700 1 -- 172.21.15.143:0/2266786012 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f27f80b5810 msgr2=0x7f27f80b5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:02.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.392+0000 7f27f659c700 1 --2- 172.21.15.143:0/2266786012 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f27f80b5810 0x7f27f80b5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:02.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.392+0000 7f27f659c700 1 -- 172.21.15.143:0/2266786012 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27f8155ec0 con 0x7f27e0005210 2023-11-08T19:27:02.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.392+0000 7f27f659c700 1 --2- 172.21.15.143:0/2266786012 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f27e0005210 0x7f27f8153c00 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f27f00072c0 tx=0x7f27f0007680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:02.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.392+0000 7f27e6ffd700 1 -- 172.21.15.143:0/2266786012 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f00077a0 con 0x7f27e0005210 2023-11-08T19:27:02.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.392+0000 7f27e6ffd700 1 -- 172.21.15.143:0/2266786012 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27f000f8d0 con 0x7f27e0005210 2023-11-08T19:27:02.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.392+0000 7f27fcf1f700 1 -- 172.21.15.143:0/2266786012 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27f81561a0 con 0x7f27e0005210 2023-11-08T19:27:02.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.392+0000 7f27e6ffd700 1 -- 172.21.15.143:0/2266786012 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f000fc80 con 0x7f27e0005210 2023-11-08T19:27:02.409 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.392+0000 7f27fcf1f700 1 -- 172.21.15.143:0/2266786012 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27f8156760 con 0x7f27e0005210 2023-11-08T19:27:02.409 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.393+0000 7f27fcf1f700 1 -- 172.21.15.143:0/2266786012 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27f804eee0 con 0x7f27e0005210 2023-11-08T19:27:02.409 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.393+0000 7f27e6ffd700 1 -- 172.21.15.143:0/2266786012 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f27f001d460 con 0x7f27e0005210 2023-11-08T19:27:02.409 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.394+0000 7f27e6ffd700 1 --2- 172.21.15.143:0/2266786012 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f27d806af00 0x7f27d806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:02.409 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.394+0000 7f27e6ffd700 1 -- 172.21.15.143:0/2266786012 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f27f00896f0 con 0x7f27e0005210 2023-11-08T19:27:02.410 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.394+0000 7f27f5d9b700 1 --2- 172.21.15.143:0/2266786012 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f27d806af00 0x7f27d806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:02.410 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.394+0000 7f27f5d9b700 1 --2- 172.21.15.143:0/2266786012 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f27d806af00 0x7f27d806d3c0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f27f80c7550 tx=0x7f27ec0093c0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:02.410 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.398+0000 7f27e6ffd700 1 -- 172.21.15.143:0/2266786012 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f27f00546d0 con 0x7f27e0005210 2023-11-08T19:27:02.586 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.583+0000 7f27fcf1f700 1 -- 172.21.15.143:0/2266786012 --> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f27f80c7780 con 0x7f27d806af00 2023-11-08T19:27:02.596 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.594+0000 7f27e6ffd700 1 -- 172.21.15.143:0/2266786012 <== mgr.4099 v2:172.21.15.174:6832/4037280178 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+370633 (secure 0 0 0) 0x7f27f80c7780 con 0x7f27d806af00 2023-11-08T19:27:02.601 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:27:02.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.598+0000 7f27fcf1f700 1 -- 172.21.15.143:0/2266786012 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f27d806af00 msgr2=0x7f27d806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:02.604 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.598+0000 7f27fcf1f700 1 --2- 172.21.15.143:0/2266786012 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f27d806af00 0x7f27d806d3c0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f27f80c7550 tx=0x7f27ec0093c0 comp rx=0 tx=0).stop 2023-11-08T19:27:02.604 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.598+0000 7f27fcf1f700 1 -- 172.21.15.143:0/2266786012 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f27e0005210 msgr2=0x7f27f8153c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:02.604 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.598+0000 7f27fcf1f700 1 --2- 172.21.15.143:0/2266786012 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f27e0005210 0x7f27f8153c00 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f27f00072c0 tx=0x7f27f0007680 comp rx=0 tx=0).stop 2023-11-08T19:27:02.604 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.598+0000 7f27fcf1f700 1 -- 172.21.15.143:0/2266786012 shutdown_connections 2023-11-08T19:27:02.605 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.598+0000 7f27fcf1f700 1 --2- 172.21.15.143:0/2266786012 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f27d806af00 0x7f27d806d3c0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:02.605 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.598+0000 7f27fcf1f700 1 --2- 172.21.15.143:0/2266786012 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f27f80b5810 0x7f27f80b5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:02.605 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.598+0000 7f27fcf1f700 1 --2- 172.21.15.143:0/2266786012 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f27e0005210 0x7f27f8153c00 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:02.605 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.598+0000 7f27fcf1f700 1 --2- 172.21.15.143:0/2266786012 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f27f81f4280 0x7f27f8160af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:02.605 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.598+0000 7f27fcf1f700 1 -- 172.21.15.143:0/2266786012 >> 172.21.15.143:0/2266786012 conn(0x7f27f80668e0 msgr2=0x7f27f80b8b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:02.606 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.598+0000 7f27fcf1f700 1 -- 172.21.15.143:0/2266786012 shutdown_connections 2023-11-08T19:27:02.606 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.598+0000 7f27fcf1f700 1 -- 172.21.15.143:0/2266786012 wait complete. 2023-11-08T19:27:02.606 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2023-11-08T19:27:02.612 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":124,"stamp":"2023-11-08T19:27:02.543649+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":258,"acting":258,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":205,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":223184,"kb_used_data":6976,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754751536,"statfs":{"total":773094113280,"available":772865572864,"internally_reserved":0,"allocated":7143424,"data_stored":3751294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001861"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.608559+0000","last_change":"2023-11-08T19:26:57.607231+0000","last_active":"2023-11-08T19:26:57.608559+0000","last_peered":"2023-11-08T19:26:57.608559+0000","last_clean":"2023-11-08T19:26:57.608559+0000","last_became_active":"2023-11-08T19:26:57.606996+0000","last_became_peered":"2023-11-08T19:26:57.606996+0000","last_unstale":"2023-11-08T19:26:57.608559+0000","last_undegraded":"2023-11-08T19:26:57.608559+0000","last_fullsized":"2023-11-08T19:26:57.608559+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:26:42.768409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.595787+0000","last_change":"2023-11-08T19:26:57.592516+0000","last_active":"2023-11-08T19:26:57.595787+0000","last_peered":"2023-11-08T19:26:57.595787+0000","last_clean":"2023-11-08T19:26:57.595787+0000","last_became_active":"2023-11-08T19:26:57.592236+0000","last_became_peered":"2023-11-08T19:26:57.592236+0000","last_unstale":"2023-11-08T19:26:57.595787+0000","last_undegraded":"2023-11-08T19:26:57.595787+0000","last_fullsized":"2023-11-08T19:26:57.595787+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:21:04.810797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-08T19:26:58.584656+0000","last_change":"2023-11-08T19:26:57.591582+0000","last_active":"2023-11-08T19:26:58.584656+0000","last_peered":"2023-11-08T19:26:58.584656+0000","last_clean":"2023-11-08T19:26:58.584656+0000","last_became_active":"2023-11-08T19:26:57.591303+0000","last_became_peered":"2023-11-08T19:26:57.591303+0000","last_unstale":"2023-11-08T19:26:58.584656+0000","last_undegraded":"2023-11-08T19:26:58.584656+0000","last_fullsized":"2023-11-08T19:26:58.584656+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:50:28.981159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.608194+0000","last_change":"2023-11-08T19:26:57.603108+0000","last_active":"2023-11-08T19:26:57.608194+0000","last_peered":"2023-11-08T19:26:57.608194+0000","last_clean":"2023-11-08T19:26:57.608194+0000","last_became_active":"2023-11-08T19:26:57.602693+0000","last_became_peered":"2023-11-08T19:26:57.602693+0000","last_unstale":"2023-11-08T19:26:57.608194+0000","last_undegraded":"2023-11-08T19:26:57.608194+0000","last_fullsized":"2023-11-08T19:26:57.608194+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:43:20.750289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-08T19:26:58.585310+0000","last_change":"2023-11-08T19:26:57.601753+0000","last_active":"2023-11-08T19:26:58.585310+0000","last_peered":"2023-11-08T19:26:58.585310+0000","last_clean":"2023-11-08T19:26:58.585310+0000","last_became_active":"2023-11-08T19:26:57.601442+0000","last_became_peered":"2023-11-08T19:26:57.601442+0000","last_unstale":"2023-11-08T19:26:58.585310+0000","last_undegraded":"2023-11-08T19:26:58.585310+0000","last_fullsized":"2023-11-08T19:26:58.585310+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:57:17.262278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.598707+0000","last_change":"2023-11-08T19:26:57.595873+0000","last_active":"2023-11-08T19:26:57.598707+0000","last_peered":"2023-11-08T19:26:57.598707+0000","last_clean":"2023-11-08T19:26:57.598707+0000","last_became_active":"2023-11-08T19:26:57.595600+0000","last_became_peered":"2023-11-08T19:26:57.595600+0000","last_unstale":"2023-11-08T19:26:57.598707+0000","last_undegraded":"2023-11-08T19:26:57.598707+0000","last_fullsized":"2023-11-08T19:26:57.598707+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:22:30.255080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.597912+0000","last_change":"2023-11-08T19:26:57.594848+0000","last_active":"2023-11-08T19:26:57.597912+0000","last_peered":"2023-11-08T19:26:57.597912+0000","last_clean":"2023-11-08T19:26:57.597912+0000","last_became_active":"2023-11-08T19:26:57.594595+0000","last_became_peered":"2023-11-08T19:26:57.594595+0000","last_unstale":"2023-11-08T19:26:57.597912+0000","last_undegraded":"2023-11-08T19:26:57.597912+0000","last_fullsized":"2023-11-08T19:26:57.597912+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:41:02.530230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-08T19:26:58.584211+0000","last_change":"2023-11-08T19:26:57.606413+0000","last_active":"2023-11-08T19:26:58.584211+0000","last_peered":"2023-11-08T19:26:58.584211+0000","last_clean":"2023-11-08T19:26:58.584211+0000","last_became_active":"2023-11-08T19:26:57.605975+0000","last_became_peered":"2023-11-08T19:26:57.605975+0000","last_unstale":"2023-11-08T19:26:58.584211+0000","last_undegraded":"2023-11-08T19:26:58.584211+0000","last_fullsized":"2023-11-08T19:26:58.584211+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:12:16.951947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-08T19:26:58.585424+0000","last_change":"2023-11-08T19:26:57.602119+0000","last_active":"2023-11-08T19:26:58.585424+0000","last_peered":"2023-11-08T19:26:58.585424+0000","last_clean":"2023-11-08T19:26:58.585424+0000","last_became_active":"2023-11-08T19:26:57.601800+0000","last_became_peered":"2023-11-08T19:26:57.601800+0000","last_unstale":"2023-11-08T19:26:58.585424+0000","last_undegraded":"2023-11-08T19:26:58.585424+0000","last_fullsized":"2023-11-08T19:26:58.585424+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:00:45.833292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-08T19:26:58.584872+0000","last_change":"2023-11-08T19:26:57.594212+0000","last_active":"2023-11-08T19:26:58.584872+0000","last_peered":"2023-11-08T19:26:58.584872+0000","last_clean":"2023-11-08T19:26:58.584872+0000","last_became_active":"2023-11-08T19:26:57.593917+0000","last_became_peered":"2023-11-08T19:26:57.593917+0000","last_unstale":"2023-11-08T19:26:58.584872+0000","last_undegraded":"2023-11-08T19:26:58.584872+0000","last_fullsized":"2023-11-08T19:26:58.584872+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:25:18.573108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.605497+0000","last_change":"2023-11-08T19:26:57.601350+0000","last_active":"2023-11-08T19:26:57.605497+0000","last_peered":"2023-11-08T19:26:57.605497+0000","last_clean":"2023-11-08T19:26:57.605497+0000","last_became_active":"2023-11-08T19:26:57.600360+0000","last_became_peered":"2023-11-08T19:26:57.600360+0000","last_unstale":"2023-11-08T19:26:57.605497+0000","last_undegraded":"2023-11-08T19:26:57.605497+0000","last_fullsized":"2023-11-08T19:26:57.605497+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:44:44.307857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-08T19:26:58.584601+0000","last_change":"2023-11-08T19:26:57.606323+0000","last_active":"2023-11-08T19:26:58.584601+0000","last_peered":"2023-11-08T19:26:58.584601+0000","last_clean":"2023-11-08T19:26:58.584601+0000","last_became_active":"2023-11-08T19:26:57.605882+0000","last_became_peered":"2023-11-08T19:26:57.605882+0000","last_unstale":"2023-11-08T19:26:58.584601+0000","last_undegraded":"2023-11-08T19:26:58.584601+0000","last_fullsized":"2023-11-08T19:26:58.584601+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:11:09.736119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-08T19:26:58.584436+0000","last_change":"2023-11-08T19:26:57.607261+0000","last_active":"2023-11-08T19:26:58.584436+0000","last_peered":"2023-11-08T19:26:58.584436+0000","last_clean":"2023-11-08T19:26:58.584436+0000","last_became_active":"2023-11-08T19:26:57.606604+0000","last_became_peered":"2023-11-08T19:26:57.606604+0000","last_unstale":"2023-11-08T19:26:58.584436+0000","last_undegraded":"2023-11-08T19:26:58.584436+0000","last_fullsized":"2023-11-08T19:26:58.584436+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:41:44.310351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-08T19:26:58.585652+0000","last_change":"2023-11-08T19:26:57.591002+0000","last_active":"2023-11-08T19:26:58.585652+0000","last_peered":"2023-11-08T19:26:58.585652+0000","last_clean":"2023-11-08T19:26:58.585652+0000","last_became_active":"2023-11-08T19:26:57.590729+0000","last_became_peered":"2023-11-08T19:26:57.590729+0000","last_unstale":"2023-11-08T19:26:58.585652+0000","last_undegraded":"2023-11-08T19:26:58.585652+0000","last_fullsized":"2023-11-08T19:26:58.585652+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:18:10.924385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-08T19:26:58.585887+0000","last_change":"2023-11-08T19:26:57.592868+0000","last_active":"2023-11-08T19:26:58.585887+0000","last_peered":"2023-11-08T19:26:58.585887+0000","last_clean":"2023-11-08T19:26:58.585887+0000","last_became_active":"2023-11-08T19:26:57.592584+0000","last_became_peered":"2023-11-08T19:26:57.592584+0000","last_unstale":"2023-11-08T19:26:58.585887+0000","last_undegraded":"2023-11-08T19:26:58.585887+0000","last_fullsized":"2023-11-08T19:26:58.585887+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:35:20.028780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.607796+0000","last_change":"2023-11-08T19:26:57.606188+0000","last_active":"2023-11-08T19:26:57.607796+0000","last_peered":"2023-11-08T19:26:57.607796+0000","last_clean":"2023-11-08T19:26:57.607796+0000","last_became_active":"2023-11-08T19:26:57.605938+0000","last_became_peered":"2023-11-08T19:26:57.605938+0000","last_unstale":"2023-11-08T19:26:57.607796+0000","last_undegraded":"2023-11-08T19:26:57.607796+0000","last_fullsized":"2023-11-08T19:26:57.607796+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:35:55.850578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.602400+0000","last_change":"2023-11-08T19:26:57.601197+0000","last_active":"2023-11-08T19:26:57.602400+0000","last_peered":"2023-11-08T19:26:57.602400+0000","last_clean":"2023-11-08T19:26:57.602400+0000","last_became_active":"2023-11-08T19:26:57.600828+0000","last_became_peered":"2023-11-08T19:26:57.600828+0000","last_unstale":"2023-11-08T19:26:57.602400+0000","last_undegraded":"2023-11-08T19:26:57.602400+0000","last_fullsized":"2023-11-08T19:26:57.602400+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:40:19.853844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.608091+0000","last_change":"2023-11-08T19:26:57.600944+0000","last_active":"2023-11-08T19:26:57.608091+0000","last_peered":"2023-11-08T19:26:57.608091+0000","last_clean":"2023-11-08T19:26:57.608091+0000","last_became_active":"2023-11-08T19:26:57.600673+0000","last_became_peered":"2023-11-08T19:26:57.600673+0000","last_unstale":"2023-11-08T19:26:57.608091+0000","last_undegraded":"2023-11-08T19:26:57.608091+0000","last_fullsized":"2023-11-08T19:26:57.608091+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:03:53.043225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-08T19:26:58.584996+0000","last_change":"2023-11-08T19:26:57.595187+0000","last_active":"2023-11-08T19:26:58.584996+0000","last_peered":"2023-11-08T19:26:58.584996+0000","last_clean":"2023-11-08T19:26:58.584996+0000","last_became_active":"2023-11-08T19:26:57.594245+0000","last_became_peered":"2023-11-08T19:26:57.594245+0000","last_unstale":"2023-11-08T19:26:58.584996+0000","last_undegraded":"2023-11-08T19:26:58.584996+0000","last_fullsized":"2023-11-08T19:26:58.584996+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:20:08.283676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.597754+0000","last_change":"2023-11-08T19:26:57.596204+0000","last_active":"2023-11-08T19:26:57.597754+0000","last_peered":"2023-11-08T19:26:57.597754+0000","last_clean":"2023-11-08T19:26:57.597754+0000","last_became_active":"2023-11-08T19:26:57.595948+0000","last_became_peered":"2023-11-08T19:26:57.595948+0000","last_unstale":"2023-11-08T19:26:57.597754+0000","last_undegraded":"2023-11-08T19:26:57.597754+0000","last_fullsized":"2023-11-08T19:26:57.597754+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:00:45.833292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-08T19:26:58.584380+0000","last_change":"2023-11-08T19:26:57.607025+0000","last_active":"2023-11-08T19:26:58.584380+0000","last_peered":"2023-11-08T19:26:58.584380+0000","last_clean":"2023-11-08T19:26:58.584380+0000","last_became_active":"2023-11-08T19:26:57.606722+0000","last_became_peered":"2023-11-08T19:26:57.606722+0000","last_unstale":"2023-11-08T19:26:58.584380+0000","last_undegraded":"2023-11-08T19:26:58.584380+0000","last_fullsized":"2023-11-08T19:26:58.584380+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:36:03.215026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.596164+0000","last_change":"2023-11-08T19:26:57.594306+0000","last_active":"2023-11-08T19:26:57.596164+0000","last_peered":"2023-11-08T19:26:57.596164+0000","last_clean":"2023-11-08T19:26:57.596164+0000","last_became_active":"2023-11-08T19:26:57.594040+0000","last_became_peered":"2023-11-08T19:26:57.594040+0000","last_unstale":"2023-11-08T19:26:57.596164+0000","last_undegraded":"2023-11-08T19:26:57.596164+0000","last_fullsized":"2023-11-08T19:26:57.596164+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:58:39.317164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.596291+0000","last_change":"2023-11-08T19:26:57.594290+0000","last_active":"2023-11-08T19:26:57.596291+0000","last_peered":"2023-11-08T19:26:57.596291+0000","last_clean":"2023-11-08T19:26:57.596291+0000","last_became_active":"2023-11-08T19:26:57.594014+0000","last_became_peered":"2023-11-08T19:26:57.594014+0000","last_unstale":"2023-11-08T19:26:57.596291+0000","last_undegraded":"2023-11-08T19:26:57.596291+0000","last_fullsized":"2023-11-08T19:26:57.596291+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:46:31.276014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-08T19:26:58.585761+0000","last_change":"2023-11-08T19:26:57.591067+0000","last_active":"2023-11-08T19:26:58.585761+0000","last_peered":"2023-11-08T19:26:58.585761+0000","last_clean":"2023-11-08T19:26:58.585761+0000","last_became_active":"2023-11-08T19:26:57.590802+0000","last_became_peered":"2023-11-08T19:26:57.590802+0000","last_unstale":"2023-11-08T19:26:58.585761+0000","last_undegraded":"2023-11-08T19:26:58.585761+0000","last_fullsized":"2023-11-08T19:26:58.585761+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:14:08.564956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-08T19:26:58.584702+0000","last_change":"2023-11-08T19:26:57.606394+0000","last_active":"2023-11-08T19:26:58.584702+0000","last_peered":"2023-11-08T19:26:58.584702+0000","last_clean":"2023-11-08T19:26:58.584702+0000","last_became_active":"2023-11-08T19:26:57.606021+0000","last_became_peered":"2023-11-08T19:26:57.606021+0000","last_unstale":"2023-11-08T19:26:58.584702+0000","last_undegraded":"2023-11-08T19:26:58.584702+0000","last_fullsized":"2023-11-08T19:26:58.584702+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:05:13.476820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-08T19:26:58.584853+0000","last_change":"2023-11-08T19:26:57.607036+0000","last_active":"2023-11-08T19:26:58.584853+0000","last_peered":"2023-11-08T19:26:58.584853+0000","last_clean":"2023-11-08T19:26:58.584853+0000","last_became_active":"2023-11-08T19:26:57.606757+0000","last_became_peered":"2023-11-08T19:26:57.606757+0000","last_unstale":"2023-11-08T19:26:58.584853+0000","last_undegraded":"2023-11-08T19:26:58.584853+0000","last_fullsized":"2023-11-08T19:26:58.584853+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:02:00.818297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.605540+0000","last_change":"2023-11-08T19:26:57.601522+0000","last_active":"2023-11-08T19:26:57.605540+0000","last_peered":"2023-11-08T19:26:57.605540+0000","last_clean":"2023-11-08T19:26:57.605540+0000","last_became_active":"2023-11-08T19:26:57.600770+0000","last_became_peered":"2023-11-08T19:26:57.600770+0000","last_unstale":"2023-11-08T19:26:57.605540+0000","last_undegraded":"2023-11-08T19:26:57.605540+0000","last_fullsized":"2023-11-08T19:26:57.605540+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:33:38.123215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.598998+0000","last_change":"2023-11-08T19:26:57.597582+0000","last_active":"2023-11-08T19:26:57.598998+0000","last_peered":"2023-11-08T19:26:57.598998+0000","last_clean":"2023-11-08T19:26:57.598998+0000","last_became_active":"2023-11-08T19:26:57.597249+0000","last_became_peered":"2023-11-08T19:26:57.597249+0000","last_unstale":"2023-11-08T19:26:57.598998+0000","last_undegraded":"2023-11-08T19:26:57.598998+0000","last_fullsized":"2023-11-08T19:26:57.598998+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:19:20.298031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-08T19:26:58.584541+0000","last_change":"2023-11-08T19:26:57.601661+0000","last_active":"2023-11-08T19:26:58.584541+0000","last_peered":"2023-11-08T19:26:58.584541+0000","last_clean":"2023-11-08T19:26:58.584541+0000","last_became_active":"2023-11-08T19:26:57.601350+0000","last_became_peered":"2023-11-08T19:26:57.601350+0000","last_unstale":"2023-11-08T19:26:58.584541+0000","last_undegraded":"2023-11-08T19:26:58.584541+0000","last_fullsized":"2023-11-08T19:26:58.584541+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:55:27.608415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-08T19:26:58.585868+0000","last_change":"2023-11-08T19:26:57.607208+0000","last_active":"2023-11-08T19:26:58.585868+0000","last_peered":"2023-11-08T19:26:58.585868+0000","last_clean":"2023-11-08T19:26:58.585868+0000","last_became_active":"2023-11-08T19:26:57.606672+0000","last_became_peered":"2023-11-08T19:26:57.606672+0000","last_unstale":"2023-11-08T19:26:58.585868+0000","last_undegraded":"2023-11-08T19:26:58.585868+0000","last_fullsized":"2023-11-08T19:26:58.585868+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:08:41.819313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.602731+0000","last_change":"2023-11-08T19:26:57.597658+0000","last_active":"2023-11-08T19:26:57.602731+0000","last_peered":"2023-11-08T19:26:57.602731+0000","last_clean":"2023-11-08T19:26:57.602731+0000","last_became_active":"2023-11-08T19:26:57.597396+0000","last_became_peered":"2023-11-08T19:26:57.597396+0000","last_unstale":"2023-11-08T19:26:57.602731+0000","last_undegraded":"2023-11-08T19:26:57.602731+0000","last_fullsized":"2023-11-08T19:26:57.602731+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:06:04.323890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.597110+0000","last_change":"2023-11-08T19:26:57.593868+0000","last_active":"2023-11-08T19:26:57.597110+0000","last_peered":"2023-11-08T19:26:57.597110+0000","last_clean":"2023-11-08T19:26:57.597110+0000","last_became_active":"2023-11-08T19:26:57.593635+0000","last_became_peered":"2023-11-08T19:26:57.593635+0000","last_unstale":"2023-11-08T19:26:57.597110+0000","last_undegraded":"2023-11-08T19:26:57.597110+0000","last_fullsized":"2023-11-08T19:26:57.597110+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:08:41.819313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-08T19:26:58.584691+0000","last_change":"2023-11-08T19:26:57.606432+0000","last_active":"2023-11-08T19:26:58.584691+0000","last_peered":"2023-11-08T19:26:58.584691+0000","last_clean":"2023-11-08T19:26:58.584691+0000","last_became_active":"2023-11-08T19:26:57.606146+0000","last_became_peered":"2023-11-08T19:26:57.606146+0000","last_unstale":"2023-11-08T19:26:58.584691+0000","last_undegraded":"2023-11-08T19:26:58.584691+0000","last_fullsized":"2023-11-08T19:26:58.584691+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:15:10.416716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-08T19:26:58.584127+0000","last_change":"2023-11-08T19:26:57.148136+0000","last_active":"2023-11-08T19:26:58.584127+0000","last_peered":"2023-11-08T19:26:58.584127+0000","last_clean":"2023-11-08T19:26:58.584127+0000","last_became_active":"2023-11-08T19:26:57.147859+0000","last_became_peered":"2023-11-08T19:26:57.147859+0000","last_unstale":"2023-11-08T19:26:58.584127+0000","last_undegraded":"2023-11-08T19:26:58.584127+0000","last_fullsized":"2023-11-08T19:26:58.584127+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_clean_scrub_stamp":"2023-11-08T19:26:56.117060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:31:40.425519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":36,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.149273+0000","last_change":"2023-11-08T19:26:06.297255+0000","last_active":"2023-11-08T19:26:57.149273+0000","last_peered":"2023-11-08T19:26:57.149273+0000","last_clean":"2023-11-08T19:26:57.149273+0000","last_became_active":"2023-11-08T19:26:04.603979+0000","last_became_peered":"2023-11-08T19:26:04.603979+0000","last_unstale":"2023-11-08T19:26:57.149273+0000","last_undegraded":"2023-11-08T19:26:57.149273+0000","last_fullsized":"2023-11-08T19:26:57.149273+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:40:11.164951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079032099999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.605481+0000","last_change":"2023-11-08T19:26:57.600053+0000","last_active":"2023-11-08T19:26:57.605481+0000","last_peered":"2023-11-08T19:26:57.605481+0000","last_clean":"2023-11-08T19:26:57.605481+0000","last_became_active":"2023-11-08T19:26:57.599773+0000","last_became_peered":"2023-11-08T19:26:57.599773+0000","last_unstale":"2023-11-08T19:26:57.605481+0000","last_undegraded":"2023-11-08T19:26:57.605481+0000","last_fullsized":"2023-11-08T19:26:57.605481+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:50:26.288669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"61'2","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-08T19:26:58.584309+0000","last_change":"2023-11-08T19:26:57.144661+0000","last_active":"2023-11-08T19:26:58.584309+0000","last_peered":"2023-11-08T19:26:58.584309+0000","last_clean":"2023-11-08T19:26:58.584309+0000","last_became_active":"2023-11-08T19:26:57.144233+0000","last_became_peered":"2023-11-08T19:26:57.144233+0000","last_unstale":"2023-11-08T19:26:58.584309+0000","last_undegraded":"2023-11-08T19:26:58.584309+0000","last_fullsized":"2023-11-08T19:26:58.584309+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_clean_scrub_stamp":"2023-11-08T19:26:56.117060+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:11:08.730999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":42,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.584567+0000","last_change":"2023-11-08T19:26:05.609625+0000","last_active":"2023-11-08T19:26:57.584567+0000","last_peered":"2023-11-08T19:26:57.584567+0000","last_clean":"2023-11-08T19:26:57.584567+0000","last_became_active":"2023-11-08T19:26:04.606312+0000","last_became_peered":"2023-11-08T19:26:04.606312+0000","last_unstale":"2023-11-08T19:26:57.584567+0000","last_undegraded":"2023-11-08T19:26:57.584567+0000","last_fullsized":"2023-11-08T19:26:57.584567+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:54:42.032901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000505935,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.598184+0000","last_change":"2023-11-08T19:26:57.595669+0000","last_active":"2023-11-08T19:26:57.598184+0000","last_peered":"2023-11-08T19:26:57.598184+0000","last_clean":"2023-11-08T19:26:57.598184+0000","last_became_active":"2023-11-08T19:26:57.595394+0000","last_became_peered":"2023-11-08T19:26:57.595394+0000","last_unstale":"2023-11-08T19:26:57.598184+0000","last_undegraded":"2023-11-08T19:26:57.598184+0000","last_fullsized":"2023-11-08T19:26:57.598184+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:59:19.833259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"61'8","reported_seq":19,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:58.145042+0000","last_change":"2023-11-08T19:26:57.144627+0000","last_active":"2023-11-08T19:26:58.145042+0000","last_peered":"2023-11-08T19:26:58.145042+0000","last_clean":"2023-11-08T19:26:58.145042+0000","last_became_active":"2023-11-08T19:26:57.144310+0000","last_became_peered":"2023-11-08T19:26:57.144310+0000","last_unstale":"2023-11-08T19:26:58.145042+0000","last_undegraded":"2023-11-08T19:26:58.145042+0000","last_fullsized":"2023-11-08T19:26:58.145042+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_clean_scrub_stamp":"2023-11-08T19:26:56.117060+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:49:53.766699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":37,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.139034+0000","last_change":"2023-11-08T19:26:06.425133+0000","last_active":"2023-11-08T19:26:57.139034+0000","last_peered":"2023-11-08T19:26:57.139034+0000","last_clean":"2023-11-08T19:26:57.139034+0000","last_became_active":"2023-11-08T19:26:04.601103+0000","last_became_peered":"2023-11-08T19:26:04.601103+0000","last_unstale":"2023-11-08T19:26:57.139034+0000","last_undegraded":"2023-11-08T19:26:57.139034+0000","last_fullsized":"2023-11-08T19:26:57.139034+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:31:14.042561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000229244,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-08T19:26:58.585203+0000","last_change":"2023-11-08T19:26:57.595589+0000","last_active":"2023-11-08T19:26:58.585203+0000","last_peered":"2023-11-08T19:26:58.585203+0000","last_clean":"2023-11-08T19:26:58.585203+0000","last_became_active":"2023-11-08T19:26:57.594674+0000","last_became_peered":"2023-11-08T19:26:57.594674+0000","last_unstale":"2023-11-08T19:26:58.585203+0000","last_undegraded":"2023-11-08T19:26:58.585203+0000","last_fullsized":"2023-11-08T19:26:58.585203+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:03:09.281780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-08T19:26:58.585716+0000","last_change":"2023-11-08T19:26:57.142517+0000","last_active":"2023-11-08T19:26:58.585716+0000","last_peered":"2023-11-08T19:26:58.585716+0000","last_clean":"2023-11-08T19:26:58.585716+0000","last_became_active":"2023-11-08T19:26:57.142257+0000","last_became_peered":"2023-11-08T19:26:57.142257+0000","last_unstale":"2023-11-08T19:26:58.585716+0000","last_undegraded":"2023-11-08T19:26:58.585716+0000","last_fullsized":"2023-11-08T19:26:58.585716+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_clean_scrub_stamp":"2023-11-08T19:26:56.117060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:56:54.191245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":35,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.140494+0000","last_change":"2023-11-08T19:26:06.265690+0000","last_active":"2023-11-08T19:26:57.140494+0000","last_peered":"2023-11-08T19:26:57.140494+0000","last_clean":"2023-11-08T19:26:57.140494+0000","last_became_active":"2023-11-08T19:26:04.603397+0000","last_became_peered":"2023-11-08T19:26:04.603397+0000","last_unstale":"2023-11-08T19:26:57.140494+0000","last_undegraded":"2023-11-08T19:26:57.140494+0000","last_fullsized":"2023-11-08T19:26:57.140494+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:50:43.827570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053361499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.607522+0000","last_change":"2023-11-08T19:26:57.606678+0000","last_active":"2023-11-08T19:26:57.607522+0000","last_peered":"2023-11-08T19:26:57.607522+0000","last_clean":"2023-11-08T19:26:57.607522+0000","last_became_active":"2023-11-08T19:26:57.606388+0000","last_became_peered":"2023-11-08T19:26:57.606388+0000","last_unstale":"2023-11-08T19:26:57.607522+0000","last_undegraded":"2023-11-08T19:26:57.607522+0000","last_fullsized":"2023-11-08T19:26:57.607522+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:14:08.564956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.599700+0000","last_change":"2023-11-08T19:26:57.154506+0000","last_active":"2023-11-08T19:26:57.599700+0000","last_peered":"2023-11-08T19:26:57.599700+0000","last_clean":"2023-11-08T19:26:57.599700+0000","last_became_active":"2023-11-08T19:26:57.154236+0000","last_became_peered":"2023-11-08T19:26:57.154236+0000","last_unstale":"2023-11-08T19:26:57.599700+0000","last_undegraded":"2023-11-08T19:26:57.599700+0000","last_fullsized":"2023-11-08T19:26:57.599700+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_clean_scrub_stamp":"2023-11-08T19:26:56.117060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:59:53.470375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":38,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.598883+0000","last_change":"2023-11-08T19:26:06.541338+0000","last_active":"2023-11-08T19:26:57.598883+0000","last_peered":"2023-11-08T19:26:57.598883+0000","last_clean":"2023-11-08T19:26:57.598883+0000","last_became_active":"2023-11-08T19:26:04.602183+0000","last_became_peered":"2023-11-08T19:26:04.602183+0000","last_unstale":"2023-11-08T19:26:57.598883+0000","last_undegraded":"2023-11-08T19:26:57.598883+0000","last_fullsized":"2023-11-08T19:26:57.598883+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:06:47.446686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052029099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.597659+0000","last_change":"2023-11-08T19:26:57.594369+0000","last_active":"2023-11-08T19:26:57.597659+0000","last_peered":"2023-11-08T19:26:57.597659+0000","last_clean":"2023-11-08T19:26:57.597659+0000","last_became_active":"2023-11-08T19:26:57.593927+0000","last_became_peered":"2023-11-08T19:26:57.593927+0000","last_unstale":"2023-11-08T19:26:57.597659+0000","last_undegraded":"2023-11-08T19:26:57.597659+0000","last_fullsized":"2023-11-08T19:26:57.597659+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:31:41.430639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-08T19:26:58.585847+0000","last_change":"2023-11-08T19:26:57.154295+0000","last_active":"2023-11-08T19:26:58.585847+0000","last_peered":"2023-11-08T19:26:58.585847+0000","last_clean":"2023-11-08T19:26:58.585847+0000","last_became_active":"2023-11-08T19:26:57.153971+0000","last_became_peered":"2023-11-08T19:26:57.153971+0000","last_unstale":"2023-11-08T19:26:58.585847+0000","last_undegraded":"2023-11-08T19:26:58.585847+0000","last_fullsized":"2023-11-08T19:26:58.585847+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_clean_scrub_stamp":"2023-11-08T19:26:56.117060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:49:05.408409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":35,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.138718+0000","last_change":"2023-11-08T19:26:06.394081+0000","last_active":"2023-11-08T19:26:57.138718+0000","last_peered":"2023-11-08T19:26:57.138718+0000","last_clean":"2023-11-08T19:26:57.138718+0000","last_became_active":"2023-11-08T19:26:04.602137+0000","last_became_peered":"2023-11-08T19:26:04.602137+0000","last_unstale":"2023-11-08T19:26:57.138718+0000","last_undegraded":"2023-11-08T19:26:57.138718+0000","last_fullsized":"2023-11-08T19:26:57.138718+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:05:46.951663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056998200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.605523+0000","last_change":"2023-11-08T19:26:57.601714+0000","last_active":"2023-11-08T19:26:57.605523+0000","last_peered":"2023-11-08T19:26:57.605523+0000","last_clean":"2023-11-08T19:26:57.605523+0000","last_became_active":"2023-11-08T19:26:57.599850+0000","last_became_peered":"2023-11-08T19:26:57.599850+0000","last_unstale":"2023-11-08T19:26:57.605523+0000","last_undegraded":"2023-11-08T19:26:57.605523+0000","last_fullsized":"2023-11-08T19:26:57.605523+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:17:48.204358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"61'6","reported_seq":17,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:58.147383+0000","last_change":"2023-11-08T19:26:57.150200+0000","last_active":"2023-11-08T19:26:58.147383+0000","last_peered":"2023-11-08T19:26:58.147383+0000","last_clean":"2023-11-08T19:26:58.147383+0000","last_became_active":"2023-11-08T19:26:57.149918+0000","last_became_peered":"2023-11-08T19:26:57.149918+0000","last_unstale":"2023-11-08T19:26:58.147383+0000","last_undegraded":"2023-11-08T19:26:58.147383+0000","last_fullsized":"2023-11-08T19:26:58.147383+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_clean_scrub_stamp":"2023-11-08T19:26:56.117060+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:35:09.627917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":37,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.138819+0000","last_change":"2023-11-08T19:26:06.423863+0000","last_active":"2023-11-08T19:26:57.138819+0000","last_peered":"2023-11-08T19:26:57.138819+0000","last_clean":"2023-11-08T19:26:57.138819+0000","last_became_active":"2023-11-08T19:26:04.608639+0000","last_became_peered":"2023-11-08T19:26:04.608639+0000","last_unstale":"2023-11-08T19:26:57.138819+0000","last_undegraded":"2023-11-08T19:26:57.138819+0000","last_fullsized":"2023-11-08T19:26:57.138819+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:04:43.631102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044384999999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-08T19:26:58.585525+0000","last_change":"2023-11-08T19:26:57.608608+0000","last_active":"2023-11-08T19:26:58.585525+0000","last_peered":"2023-11-08T19:26:58.585525+0000","last_clean":"2023-11-08T19:26:58.585525+0000","last_became_active":"2023-11-08T19:26:57.608344+0000","last_became_peered":"2023-11-08T19:26:57.608344+0000","last_unstale":"2023-11-08T19:26:58.585525+0000","last_undegraded":"2023-11-08T19:26:58.585525+0000","last_fullsized":"2023-11-08T19:26:58.585525+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:19:20.298031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"61'3","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-08T19:26:58.584509+0000","last_change":"2023-11-08T19:26:57.154362+0000","last_active":"2023-11-08T19:26:58.584509+0000","last_peered":"2023-11-08T19:26:58.584509+0000","last_clean":"2023-11-08T19:26:58.584509+0000","last_became_active":"2023-11-08T19:26:57.154116+0000","last_became_peered":"2023-11-08T19:26:57.154116+0000","last_unstale":"2023-11-08T19:26:58.584509+0000","last_undegraded":"2023-11-08T19:26:58.584509+0000","last_fullsized":"2023-11-08T19:26:58.584509+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_clean_scrub_stamp":"2023-11-08T19:26:56.117060+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:41:43.305231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":35,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.135363+0000","last_change":"2023-11-08T19:26:06.265318+0000","last_active":"2023-11-08T19:26:57.135363+0000","last_peered":"2023-11-08T19:26:57.135363+0000","last_clean":"2023-11-08T19:26:57.135363+0000","last_became_active":"2023-11-08T19:26:04.603409+0000","last_became_peered":"2023-11-08T19:26:04.603409+0000","last_unstale":"2023-11-08T19:26:57.135363+0000","last_undegraded":"2023-11-08T19:26:57.135363+0000","last_fullsized":"2023-11-08T19:26:57.135363+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:27:24.933111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065684300000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.606871+0000","last_change":"2023-11-08T19:26:57.606073+0000","last_active":"2023-11-08T19:26:57.606871+0000","last_peered":"2023-11-08T19:26:57.606871+0000","last_clean":"2023-11-08T19:26:57.606871+0000","last_became_active":"2023-11-08T19:26:57.605705+0000","last_became_peered":"2023-11-08T19:26:57.605705+0000","last_unstale":"2023-11-08T19:26:57.606871+0000","last_undegraded":"2023-11-08T19:26:57.606871+0000","last_fullsized":"2023-11-08T19:26:57.606871+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:03:37.833816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"61'1","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:58.136433+0000","last_change":"2023-11-08T19:26:57.146499+0000","last_active":"2023-11-08T19:26:58.136433+0000","last_peered":"2023-11-08T19:26:58.136433+0000","last_clean":"2023-11-08T19:26:58.136433+0000","last_became_active":"2023-11-08T19:26:57.146173+0000","last_became_peered":"2023-11-08T19:26:57.146173+0000","last_unstale":"2023-11-08T19:26:58.136433+0000","last_undegraded":"2023-11-08T19:26:58.136433+0000","last_fullsized":"2023-11-08T19:26:58.136433+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_clean_scrub_stamp":"2023-11-08T19:26:56.117060+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:18:09.919265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":42,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.585155+0000","last_change":"2023-11-08T19:26:05.610012+0000","last_active":"2023-11-08T19:26:57.585155+0000","last_peered":"2023-11-08T19:26:57.585155+0000","last_clean":"2023-11-08T19:26:57.585155+0000","last_became_active":"2023-11-08T19:26:04.603209+0000","last_became_peered":"2023-11-08T19:26:04.603209+0000","last_unstale":"2023-11-08T19:26:57.585155+0000","last_undegraded":"2023-11-08T19:26:57.585155+0000","last_fullsized":"2023-11-08T19:26:57.585155+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:35:02.306997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053335100000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.594769+0000","last_change":"2023-11-08T19:26:57.593598+0000","last_active":"2023-11-08T19:26:57.594769+0000","last_peered":"2023-11-08T19:26:57.594769+0000","last_clean":"2023-11-08T19:26:57.594769+0000","last_became_active":"2023-11-08T19:26:57.593055+0000","last_became_peered":"2023-11-08T19:26:57.593055+0000","last_unstale":"2023-11-08T19:26:57.594769+0000","last_undegraded":"2023-11-08T19:26:57.594769+0000","last_fullsized":"2023-11-08T19:26:57.594769+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:55:35.576482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"61'1","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:58.145688+0000","last_change":"2023-11-08T19:26:57.150582+0000","last_active":"2023-11-08T19:26:58.145688+0000","last_peered":"2023-11-08T19:26:58.145688+0000","last_clean":"2023-11-08T19:26:58.145688+0000","last_became_active":"2023-11-08T19:26:57.150316+0000","last_became_peered":"2023-11-08T19:26:57.150316+0000","last_unstale":"2023-11-08T19:26:58.145688+0000","last_undegraded":"2023-11-08T19:26:58.145688+0000","last_fullsized":"2023-11-08T19:26:58.145688+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_clean_scrub_stamp":"2023-11-08T19:26:56.117060+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:40:37.959491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":36,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.141723+0000","last_change":"2023-11-08T19:26:05.606946+0000","last_active":"2023-11-08T19:26:57.141723+0000","last_peered":"2023-11-08T19:26:57.141723+0000","last_clean":"2023-11-08T19:26:57.141723+0000","last_became_active":"2023-11-08T19:26:04.598775+0000","last_became_peered":"2023-11-08T19:26:04.598775+0000","last_unstale":"2023-11-08T19:26:57.141723+0000","last_undegraded":"2023-11-08T19:26:57.141723+0000","last_fullsized":"2023-11-08T19:26:57.141723+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:20:56.898927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049870399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.601528+0000","last_change":"2023-11-08T19:26:57.601148+0000","last_active":"2023-11-08T19:26:57.601528+0000","last_peered":"2023-11-08T19:26:57.601528+0000","last_clean":"2023-11-08T19:26:57.601528+0000","last_became_active":"2023-11-08T19:26:57.600907+0000","last_became_peered":"2023-11-08T19:26:57.600907+0000","last_unstale":"2023-11-08T19:26:57.601528+0000","last_undegraded":"2023-11-08T19:26:57.601528+0000","last_fullsized":"2023-11-08T19:26:57.601528+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:25:47.748513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-08T19:26:58.584514+0000","last_change":"2023-11-08T19:26:57.143692+0000","last_active":"2023-11-08T19:26:58.584514+0000","last_peered":"2023-11-08T19:26:58.584514+0000","last_clean":"2023-11-08T19:26:58.584514+0000","last_became_active":"2023-11-08T19:26:57.142738+0000","last_became_peered":"2023-11-08T19:26:57.142738+0000","last_unstale":"2023-11-08T19:26:58.584514+0000","last_undegraded":"2023-11-08T19:26:58.584514+0000","last_fullsized":"2023-11-08T19:26:58.584514+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_clean_scrub_stamp":"2023-11-08T19:26:56.117060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:44:43.302738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":35,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.139559+0000","last_change":"2023-11-08T19:26:06.265907+0000","last_active":"2023-11-08T19:26:57.139559+0000","last_peered":"2023-11-08T19:26:57.139559+0000","last_clean":"2023-11-08T19:26:57.139559+0000","last_became_active":"2023-11-08T19:26:04.604772+0000","last_became_peered":"2023-11-08T19:26:04.604772+0000","last_unstale":"2023-11-08T19:26:57.139559+0000","last_undegraded":"2023-11-08T19:26:57.139559+0000","last_fullsized":"2023-11-08T19:26:57.139559+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:44:16.139794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066875300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-08T19:26:58.585041+0000","last_change":"2023-11-08T19:26:57.590852+0000","last_active":"2023-11-08T19:26:58.585041+0000","last_peered":"2023-11-08T19:26:58.585041+0000","last_clean":"2023-11-08T19:26:58.585041+0000","last_became_active":"2023-11-08T19:26:57.590503+0000","last_became_peered":"2023-11-08T19:26:57.590503+0000","last_unstale":"2023-11-08T19:26:58.585041+0000","last_undegraded":"2023-11-08T19:26:58.585041+0000","last_fullsized":"2023-11-08T19:26:58.585041+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:59:54.475495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.588456+0000","last_change":"2023-11-08T19:26:57.143857+0000","last_active":"2023-11-08T19:26:57.588456+0000","last_peered":"2023-11-08T19:26:57.588456+0000","last_clean":"2023-11-08T19:26:57.588456+0000","last_became_active":"2023-11-08T19:26:57.143525+0000","last_became_peered":"2023-11-08T19:26:57.143525+0000","last_unstale":"2023-11-08T19:26:57.588456+0000","last_undegraded":"2023-11-08T19:26:57.588456+0000","last_fullsized":"2023-11-08T19:26:57.588456+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_clean_scrub_stamp":"2023-11-08T19:26:56.117060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:31:06.035836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":35,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.139273+0000","last_change":"2023-11-08T19:26:06.394488+0000","last_active":"2023-11-08T19:26:57.139273+0000","last_peered":"2023-11-08T19:26:57.139273+0000","last_clean":"2023-11-08T19:26:57.139273+0000","last_became_active":"2023-11-08T19:26:04.606805+0000","last_became_peered":"2023-11-08T19:26:04.606805+0000","last_unstale":"2023-11-08T19:26:57.139273+0000","last_undegraded":"2023-11-08T19:26:57.139273+0000","last_fullsized":"2023-11-08T19:26:57.139273+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:25:20.857133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045317000000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.608238+0000","last_change":"2023-11-08T19:26:57.603508+0000","last_active":"2023-11-08T19:26:57.608238+0000","last_peered":"2023-11-08T19:26:57.608238+0000","last_clean":"2023-11-08T19:26:57.608238+0000","last_became_active":"2023-11-08T19:26:57.603223+0000","last_became_peered":"2023-11-08T19:26:57.603223+0000","last_unstale":"2023-11-08T19:26:57.608238+0000","last_undegraded":"2023-11-08T19:26:57.608238+0000","last_fullsized":"2023-11-08T19:26:57.608238+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:07:17.979415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"61'1","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:58.145500+0000","last_change":"2023-11-08T19:26:57.150422+0000","last_active":"2023-11-08T19:26:58.145500+0000","last_peered":"2023-11-08T19:26:58.145500+0000","last_clean":"2023-11-08T19:26:58.145500+0000","last_became_active":"2023-11-08T19:26:57.150147+0000","last_became_peered":"2023-11-08T19:26:57.150147+0000","last_unstale":"2023-11-08T19:26:58.145500+0000","last_undegraded":"2023-11-08T19:26:58.145500+0000","last_fullsized":"2023-11-08T19:26:58.145500+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_clean_scrub_stamp":"2023-11-08T19:26:56.117060+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:00:44.828172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":37,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.138770+0000","last_change":"2023-11-08T19:26:06.423754+0000","last_active":"2023-11-08T19:26:57.138770+0000","last_peered":"2023-11-08T19:26:57.138770+0000","last_clean":"2023-11-08T19:26:57.138770+0000","last_became_active":"2023-11-08T19:26:04.599322+0000","last_became_peered":"2023-11-08T19:26:04.599322+0000","last_unstale":"2023-11-08T19:26:57.138770+0000","last_undegraded":"2023-11-08T19:26:57.138770+0000","last_fullsized":"2023-11-08T19:26:57.138770+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:13:08.689128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042670000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-08T19:26:58.584626+0000","last_change":"2023-11-08T19:26:57.595871+0000","last_active":"2023-11-08T19:26:58.584626+0000","last_peered":"2023-11-08T19:26:58.584626+0000","last_clean":"2023-11-08T19:26:58.584626+0000","last_became_active":"2023-11-08T19:26:57.595610+0000","last_became_peered":"2023-11-08T19:26:57.595610+0000","last_unstale":"2023-11-08T19:26:58.584626+0000","last_undegraded":"2023-11-08T19:26:58.584626+0000","last_fullsized":"2023-11-08T19:26:58.584626+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:43:20.750289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"61'4","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:58.142409+0000","last_change":"2023-11-08T19:26:57.151102+0000","last_active":"2023-11-08T19:26:58.142409+0000","last_peered":"2023-11-08T19:26:58.142409+0000","last_clean":"2023-11-08T19:26:58.142409+0000","last_became_active":"2023-11-08T19:26:57.150727+0000","last_became_peered":"2023-11-08T19:26:57.150727+0000","last_unstale":"2023-11-08T19:26:58.142409+0000","last_undegraded":"2023-11-08T19:26:58.142409+0000","last_fullsized":"2023-11-08T19:26:58.142409+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_clean_scrub_stamp":"2023-11-08T19:26:56.117060+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:56:54.191245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":35,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.138522+0000","last_change":"2023-11-08T19:26:06.394320+0000","last_active":"2023-11-08T19:26:57.138522+0000","last_peered":"2023-11-08T19:26:57.138522+0000","last_clean":"2023-11-08T19:26:57.138522+0000","last_became_active":"2023-11-08T19:26:04.603212+0000","last_became_peered":"2023-11-08T19:26:04.603212+0000","last_unstale":"2023-11-08T19:26:57.138522+0000","last_undegraded":"2023-11-08T19:26:57.138522+0000","last_fullsized":"2023-11-08T19:26:57.138522+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:25:29.947079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054695399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-08T19:26:58.585584+0000","last_change":"2023-11-08T19:26:57.592878+0000","last_active":"2023-11-08T19:26:58.585584+0000","last_peered":"2023-11-08T19:26:58.585584+0000","last_clean":"2023-11-08T19:26:58.585584+0000","last_became_active":"2023-11-08T19:26:57.592598+0000","last_became_peered":"2023-11-08T19:26:57.592598+0000","last_unstale":"2023-11-08T19:26:58.585584+0000","last_undegraded":"2023-11-08T19:26:58.585584+0000","last_fullsized":"2023-11-08T19:26:58.585584+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:58:39.317164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"61'2","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:58.145630+0000","last_change":"2023-11-08T19:26:57.154544+0000","last_active":"2023-11-08T19:26:58.145630+0000","last_peered":"2023-11-08T19:26:58.145630+0000","last_clean":"2023-11-08T19:26:58.145630+0000","last_became_active":"2023-11-08T19:26:57.154258+0000","last_became_peered":"2023-11-08T19:26:57.154258+0000","last_unstale":"2023-11-08T19:26:58.145630+0000","last_undegraded":"2023-11-08T19:26:58.145630+0000","last_fullsized":"2023-11-08T19:26:58.145630+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_clean_scrub_stamp":"2023-11-08T19:26:56.117060+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:25:46.743393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":37,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.138987+0000","last_change":"2023-11-08T19:26:06.424073+0000","last_active":"2023-11-08T19:26:57.138987+0000","last_peered":"2023-11-08T19:26:57.138987+0000","last_clean":"2023-11-08T19:26:57.138987+0000","last_became_active":"2023-11-08T19:26:04.599046+0000","last_became_peered":"2023-11-08T19:26:04.599046+0000","last_unstale":"2023-11-08T19:26:57.138987+0000","last_undegraded":"2023-11-08T19:26:57.138987+0000","last_fullsized":"2023-11-08T19:26:57.138987+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:04:43.562566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035092199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.597168+0000","last_change":"2023-11-08T19:26:57.594451+0000","last_active":"2023-11-08T19:26:57.597168+0000","last_peered":"2023-11-08T19:26:57.597168+0000","last_clean":"2023-11-08T19:26:57.597168+0000","last_became_active":"2023-11-08T19:26:57.594233+0000","last_became_peered":"2023-11-08T19:26:57.594233+0000","last_unstale":"2023-11-08T19:26:57.597168+0000","last_undegraded":"2023-11-08T19:26:57.597168+0000","last_fullsized":"2023-11-08T19:26:57.597168+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:50:28.981159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"61'2","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:58.145610+0000","last_change":"2023-11-08T19:26:57.146870+0000","last_active":"2023-11-08T19:26:58.145610+0000","last_peered":"2023-11-08T19:26:58.145610+0000","last_clean":"2023-11-08T19:26:58.145610+0000","last_became_active":"2023-11-08T19:26:57.146618+0000","last_became_peered":"2023-11-08T19:26:57.146618+0000","last_unstale":"2023-11-08T19:26:58.145610+0000","last_undegraded":"2023-11-08T19:26:58.145610+0000","last_fullsized":"2023-11-08T19:26:58.145610+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_clean_scrub_stamp":"2023-11-08T19:26:56.117060+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:55:54.246933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":42,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.584338+0000","last_change":"2023-11-08T19:26:05.608980+0000","last_active":"2023-11-08T19:26:57.584338+0000","last_peered":"2023-11-08T19:26:57.584338+0000","last_clean":"2023-11-08T19:26:57.584338+0000","last_became_active":"2023-11-08T19:26:04.606386+0000","last_became_peered":"2023-11-08T19:26:04.606386+0000","last_unstale":"2023-11-08T19:26:57.584338+0000","last_undegraded":"2023-11-08T19:26:57.584338+0000","last_fullsized":"2023-11-08T19:26:57.584338+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:41:50.771457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037950599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.605420+0000","last_change":"2023-11-08T19:26:57.600033+0000","last_active":"2023-11-08T19:26:57.605420+0000","last_peered":"2023-11-08T19:26:57.605420+0000","last_clean":"2023-11-08T19:26:57.605420+0000","last_became_active":"2023-11-08T19:26:57.599766+0000","last_became_peered":"2023-11-08T19:26:57.599766+0000","last_unstale":"2023-11-08T19:26:57.605420+0000","last_undegraded":"2023-11-08T19:26:57.605420+0000","last_fullsized":"2023-11-08T19:26:57.605420+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:25:49.906352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.587451+0000","last_change":"2023-11-08T19:26:57.148628+0000","last_active":"2023-11-08T19:26:57.587451+0000","last_peered":"2023-11-08T19:26:57.587451+0000","last_clean":"2023-11-08T19:26:57.587451+0000","last_became_active":"2023-11-08T19:26:57.148355+0000","last_became_peered":"2023-11-08T19:26:57.148355+0000","last_unstale":"2023-11-08T19:26:57.587451+0000","last_undegraded":"2023-11-08T19:26:57.587451+0000","last_fullsized":"2023-11-08T19:26:57.587451+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_clean_scrub_stamp":"2023-11-08T19:26:56.117060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:35:19.023660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":38,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.598895+0000","last_change":"2023-11-08T19:26:06.541623+0000","last_active":"2023-11-08T19:26:57.598895+0000","last_peered":"2023-11-08T19:26:57.598895+0000","last_clean":"2023-11-08T19:26:57.598895+0000","last_became_active":"2023-11-08T19:26:04.601961+0000","last_became_peered":"2023-11-08T19:26:04.601961+0000","last_unstale":"2023-11-08T19:26:57.598895+0000","last_undegraded":"2023-11-08T19:26:57.598895+0000","last_fullsized":"2023-11-08T19:26:57.598895+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:11:26.508634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000466386,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-08T19:26:58.584423+0000","last_change":"2023-11-08T19:26:57.595480+0000","last_active":"2023-11-08T19:26:58.584423+0000","last_peered":"2023-11-08T19:26:58.584423+0000","last_clean":"2023-11-08T19:26:58.584423+0000","last_became_active":"2023-11-08T19:26:57.595097+0000","last_became_peered":"2023-11-08T19:26:57.595097+0000","last_unstale":"2023-11-08T19:26:58.584423+0000","last_undegraded":"2023-11-08T19:26:58.584423+0000","last_fullsized":"2023-11-08T19:26:58.584423+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:03:53.043225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-08T19:26:58.584678+0000","last_change":"2023-11-08T19:26:57.142090+0000","last_active":"2023-11-08T19:26:58.584678+0000","last_peered":"2023-11-08T19:26:58.584678+0000","last_clean":"2023-11-08T19:26:58.584678+0000","last_became_active":"2023-11-08T19:26:57.141726+0000","last_became_peered":"2023-11-08T19:26:57.141726+0000","last_unstale":"2023-11-08T19:26:58.584678+0000","last_undegraded":"2023-11-08T19:26:58.584678+0000","last_fullsized":"2023-11-08T19:26:58.584678+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_clean_scrub_stamp":"2023-11-08T19:26:56.117060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:45:33.328935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":37,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.139040+0000","last_change":"2023-11-08T19:26:06.424464+0000","last_active":"2023-11-08T19:26:57.139040+0000","last_peered":"2023-11-08T19:26:57.139040+0000","last_clean":"2023-11-08T19:26:57.139040+0000","last_became_active":"2023-11-08T19:26:04.603144+0000","last_became_peered":"2023-11-08T19:26:04.603144+0000","last_unstale":"2023-11-08T19:26:57.139040+0000","last_undegraded":"2023-11-08T19:26:57.139040+0000","last_fullsized":"2023-11-08T19:26:57.139040+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:42:55.269365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054090799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.607827+0000","last_change":"2023-11-08T19:26:57.606630+0000","last_active":"2023-11-08T19:26:57.607827+0000","last_peered":"2023-11-08T19:26:57.607827+0000","last_clean":"2023-11-08T19:26:57.607827+0000","last_became_active":"2023-11-08T19:26:57.606394+0000","last_became_peered":"2023-11-08T19:26:57.606394+0000","last_unstale":"2023-11-08T19:26:57.607827+0000","last_undegraded":"2023-11-08T19:26:57.607827+0000","last_fullsized":"2023-11-08T19:26:57.607827+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:06:39.520196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.587796+0000","last_change":"2023-11-08T19:26:57.148121+0000","last_active":"2023-11-08T19:26:57.587796+0000","last_peered":"2023-11-08T19:26:57.587796+0000","last_clean":"2023-11-08T19:26:57.587796+0000","last_became_active":"2023-11-08T19:26:57.147858+0000","last_became_peered":"2023-11-08T19:26:57.147858+0000","last_unstale":"2023-11-08T19:26:57.587796+0000","last_undegraded":"2023-11-08T19:26:57.587796+0000","last_fullsized":"2023-11-08T19:26:57.587796+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_clean_scrub_stamp":"2023-11-08T19:26:56.117060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:35:07.925172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":36,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.141849+0000","last_change":"2023-11-08T19:26:05.606898+0000","last_active":"2023-11-08T19:26:57.141849+0000","last_peered":"2023-11-08T19:26:57.141849+0000","last_clean":"2023-11-08T19:26:57.141849+0000","last_became_active":"2023-11-08T19:26:04.603955+0000","last_became_peered":"2023-11-08T19:26:04.603955+0000","last_unstale":"2023-11-08T19:26:57.141849+0000","last_undegraded":"2023-11-08T19:26:57.141849+0000","last_fullsized":"2023-11-08T19:26:57.141849+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:19:11.589559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036473499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.600408+0000","last_change":"2023-11-08T19:26:57.600008+0000","last_active":"2023-11-08T19:26:57.600408+0000","last_peered":"2023-11-08T19:26:57.600408+0000","last_clean":"2023-11-08T19:26:57.600408+0000","last_became_active":"2023-11-08T19:26:57.599778+0000","last_became_peered":"2023-11-08T19:26:57.599778+0000","last_unstale":"2023-11-08T19:26:57.600408+0000","last_undegraded":"2023-11-08T19:26:57.600408+0000","last_fullsized":"2023-11-08T19:26:57.600408+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:00:17.866236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.588330+0000","last_change":"2023-11-08T19:26:57.143777+0000","last_active":"2023-11-08T19:26:57.588330+0000","last_peered":"2023-11-08T19:26:57.588330+0000","last_clean":"2023-11-08T19:26:57.588330+0000","last_became_active":"2023-11-08T19:26:57.143173+0000","last_became_peered":"2023-11-08T19:26:57.143173+0000","last_unstale":"2023-11-08T19:26:57.588330+0000","last_undegraded":"2023-11-08T19:26:57.588330+0000","last_fullsized":"2023-11-08T19:26:57.588330+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_clean_scrub_stamp":"2023-11-08T19:26:56.117060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:35:04.669889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":35,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.140339+0000","last_change":"2023-11-08T19:26:06.265686+0000","last_active":"2023-11-08T19:26:57.140339+0000","last_peered":"2023-11-08T19:26:57.140339+0000","last_clean":"2023-11-08T19:26:57.140339+0000","last_became_active":"2023-11-08T19:26:04.602111+0000","last_became_peered":"2023-11-08T19:26:04.602111+0000","last_unstale":"2023-11-08T19:26:57.140339+0000","last_undegraded":"2023-11-08T19:26:57.140339+0000","last_fullsized":"2023-11-08T19:26:57.140339+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:40:11.164951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056197899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.594800+0000","last_change":"2023-11-08T19:26:57.593896+0000","last_active":"2023-11-08T19:26:57.594800+0000","last_peered":"2023-11-08T19:26:57.594800+0000","last_clean":"2023-11-08T19:26:57.594800+0000","last_became_active":"2023-11-08T19:26:57.593676+0000","last_became_peered":"2023-11-08T19:26:57.593676+0000","last_unstale":"2023-11-08T19:26:57.594800+0000","last_undegraded":"2023-11-08T19:26:57.594800+0000","last_fullsized":"2023-11-08T19:26:57.594800+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:48:43.587422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-08T19:26:58.585176+0000","last_change":"2023-11-08T19:26:57.146397+0000","last_active":"2023-11-08T19:26:58.585176+0000","last_peered":"2023-11-08T19:26:58.585176+0000","last_clean":"2023-11-08T19:26:58.585176+0000","last_became_active":"2023-11-08T19:26:57.146037+0000","last_became_peered":"2023-11-08T19:26:57.146037+0000","last_unstale":"2023-11-08T19:26:58.585176+0000","last_undegraded":"2023-11-08T19:26:58.585176+0000","last_fullsized":"2023-11-08T19:26:58.585176+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_clean_scrub_stamp":"2023-11-08T19:26:56.117060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:44:00.118685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":35,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.138609+0000","last_change":"2023-11-08T19:26:06.394075+0000","last_active":"2023-11-08T19:26:57.138609+0000","last_peered":"2023-11-08T19:26:57.138609+0000","last_clean":"2023-11-08T19:26:57.138609+0000","last_became_active":"2023-11-08T19:26:04.606967+0000","last_became_peered":"2023-11-08T19:26:04.606967+0000","last_unstale":"2023-11-08T19:26:57.138609+0000","last_undegraded":"2023-11-08T19:26:57.138609+0000","last_fullsized":"2023-11-08T19:26:57.138609+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:19:56.286835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040971999999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.602438+0000","last_change":"2023-11-08T19:26:57.601327+0000","last_active":"2023-11-08T19:26:57.602438+0000","last_peered":"2023-11-08T19:26:57.602438+0000","last_clean":"2023-11-08T19:26:57.602438+0000","last_became_active":"2023-11-08T19:26:57.600975+0000","last_became_peered":"2023-11-08T19:26:57.600975+0000","last_unstale":"2023-11-08T19:26:57.602438+0000","last_undegraded":"2023-11-08T19:26:57.602438+0000","last_fullsized":"2023-11-08T19:26:57.602438+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:55:27.608415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"61'2","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-08T19:26:58.584883+0000","last_change":"2023-11-08T19:26:57.143247+0000","last_active":"2023-11-08T19:26:58.584883+0000","last_peered":"2023-11-08T19:26:58.584883+0000","last_clean":"2023-11-08T19:26:58.584883+0000","last_became_active":"2023-11-08T19:26:57.142961+0000","last_became_peered":"2023-11-08T19:26:57.142961+0000","last_unstale":"2023-11-08T19:26:58.584883+0000","last_undegraded":"2023-11-08T19:26:58.584883+0000","last_fullsized":"2023-11-08T19:26:58.584883+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_clean_scrub_stamp":"2023-11-08T19:26:56.117060+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:21:03.805677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":42,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.584093+0000","last_change":"2023-11-08T19:26:05.608892+0000","last_active":"2023-11-08T19:26:57.584093+0000","last_peered":"2023-11-08T19:26:57.584093+0000","last_clean":"2023-11-08T19:26:57.584093+0000","last_became_active":"2023-11-08T19:26:04.604519+0000","last_became_peered":"2023-11-08T19:26:04.604519+0000","last_unstale":"2023-11-08T19:26:57.584093+0000","last_undegraded":"2023-11-08T19:26:57.584093+0000","last_fullsized":"2023-11-08T19:26:57.584093+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:11:26.508634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051248099999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-08T19:26:58.585731+0000","last_change":"2023-11-08T19:26:57.592692+0000","last_active":"2023-11-08T19:26:58.585731+0000","last_peered":"2023-11-08T19:26:58.585731+0000","last_clean":"2023-11-08T19:26:58.585731+0000","last_became_active":"2023-11-08T19:26:57.592401+0000","last_became_peered":"2023-11-08T19:26:57.592401+0000","last_unstale":"2023-11-08T19:26:58.585731+0000","last_undegraded":"2023-11-08T19:26:58.585731+0000","last_fullsized":"2023-11-08T19:26:58.585731+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:03:37.833816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.589194+0000","last_change":"2023-11-08T19:26:57.150117+0000","last_active":"2023-11-08T19:26:57.589194+0000","last_peered":"2023-11-08T19:26:57.589194+0000","last_clean":"2023-11-08T19:26:57.589194+0000","last_became_active":"2023-11-08T19:26:57.149674+0000","last_became_peered":"2023-11-08T19:26:57.149674+0000","last_unstale":"2023-11-08T19:26:57.589194+0000","last_undegraded":"2023-11-08T19:26:57.589194+0000","last_fullsized":"2023-11-08T19:26:57.589194+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_clean_scrub_stamp":"2023-11-08T19:26:56.117060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:44:00.118685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":37,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.138897+0000","last_change":"2023-11-08T19:26:06.424236+0000","last_active":"2023-11-08T19:26:57.138897+0000","last_peered":"2023-11-08T19:26:57.138897+0000","last_clean":"2023-11-08T19:26:57.138897+0000","last_became_active":"2023-11-08T19:26:04.600115+0000","last_became_peered":"2023-11-08T19:26:04.600115+0000","last_unstale":"2023-11-08T19:26:57.138897+0000","last_undegraded":"2023-11-08T19:26:57.138897+0000","last_fullsized":"2023-11-08T19:26:57.138897+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:18:01.876605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065803900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-08T19:26:58.585003+0000","last_change":"2023-11-08T19:26:57.607268+0000","last_active":"2023-11-08T19:26:58.585003+0000","last_peered":"2023-11-08T19:26:58.585003+0000","last_clean":"2023-11-08T19:26:58.585003+0000","last_became_active":"2023-11-08T19:26:57.606979+0000","last_became_peered":"2023-11-08T19:26:57.606979+0000","last_unstale":"2023-11-08T19:26:58.585003+0000","last_undegraded":"2023-11-08T19:26:58.585003+0000","last_fullsized":"2023-11-08T19:26:58.585003+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:32:57.602413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.584015+0000","last_change":"2023-11-08T19:26:57.146108+0000","last_active":"2023-11-08T19:26:57.584015+0000","last_peered":"2023-11-08T19:26:57.584015+0000","last_clean":"2023-11-08T19:26:57.584015+0000","last_became_active":"2023-11-08T19:26:57.145813+0000","last_became_peered":"2023-11-08T19:26:57.145813+0000","last_unstale":"2023-11-08T19:26:57.584015+0000","last_undegraded":"2023-11-08T19:26:57.584015+0000","last_fullsized":"2023-11-08T19:26:57.584015+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_clean_scrub_stamp":"2023-11-08T19:26:56.117060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:25:04.442302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":38,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.599951+0000","last_change":"2023-11-08T19:26:06.541360+0000","last_active":"2023-11-08T19:26:57.599951+0000","last_peered":"2023-11-08T19:26:57.599951+0000","last_clean":"2023-11-08T19:26:57.599951+0000","last_became_active":"2023-11-08T19:26:04.603725+0000","last_became_peered":"2023-11-08T19:26:04.603725+0000","last_unstale":"2023-11-08T19:26:57.599951+0000","last_undegraded":"2023-11-08T19:26:57.599951+0000","last_fullsized":"2023-11-08T19:26:57.599951+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:28:48.576742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053365400000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-08T19:26:58.585692+0000","last_change":"2023-11-08T19:26:57.604025+0000","last_active":"2023-11-08T19:26:58.585692+0000","last_peered":"2023-11-08T19:26:58.585692+0000","last_clean":"2023-11-08T19:26:58.585692+0000","last_became_active":"2023-11-08T19:26:57.603781+0000","last_became_peered":"2023-11-08T19:26:57.603781+0000","last_unstale":"2023-11-08T19:26:58.585692+0000","last_undegraded":"2023-11-08T19:26:58.585692+0000","last_fullsized":"2023-11-08T19:26:58.585692+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:45:34.334055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-08T19:26:58.585726+0000","last_change":"2023-11-08T19:26:57.148107+0000","last_active":"2023-11-08T19:26:58.585726+0000","last_peered":"2023-11-08T19:26:58.585726+0000","last_clean":"2023-11-08T19:26:58.585726+0000","last_became_active":"2023-11-08T19:26:57.147847+0000","last_became_peered":"2023-11-08T19:26:57.147847+0000","last_unstale":"2023-11-08T19:26:58.585726+0000","last_undegraded":"2023-11-08T19:26:58.585726+0000","last_fullsized":"2023-11-08T19:26:58.585726+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_clean_scrub_stamp":"2023-11-08T19:26:56.117060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:56:54.191245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":42,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.585071+0000","last_change":"2023-11-08T19:26:05.609824+0000","last_active":"2023-11-08T19:26:57.585071+0000","last_peered":"2023-11-08T19:26:57.585071+0000","last_clean":"2023-11-08T19:26:57.585071+0000","last_became_active":"2023-11-08T19:26:04.603320+0000","last_became_peered":"2023-11-08T19:26:04.603320+0000","last_unstale":"2023-11-08T19:26:57.585071+0000","last_undegraded":"2023-11-08T19:26:57.585071+0000","last_fullsized":"2023-11-08T19:26:57.585071+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:05:45.976615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063225,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.598146+0000","last_change":"2023-11-08T19:26:57.595984+0000","last_active":"2023-11-08T19:26:57.598146+0000","last_peered":"2023-11-08T19:26:57.598146+0000","last_clean":"2023-11-08T19:26:57.598146+0000","last_became_active":"2023-11-08T19:26:57.595187+0000","last_became_peered":"2023-11-08T19:26:57.595187+0000","last_unstale":"2023-11-08T19:26:57.598146+0000","last_undegraded":"2023-11-08T19:26:57.598146+0000","last_fullsized":"2023-11-08T19:26:57.598146+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:25:18.573108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"61'2","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:58.145607+0000","last_change":"2023-11-08T19:26:57.144514+0000","last_active":"2023-11-08T19:26:58.145607+0000","last_peered":"2023-11-08T19:26:58.145607+0000","last_clean":"2023-11-08T19:26:58.145607+0000","last_became_active":"2023-11-08T19:26:57.144231+0000","last_became_peered":"2023-11-08T19:26:57.144231+0000","last_unstale":"2023-11-08T19:26:58.145607+0000","last_undegraded":"2023-11-08T19:26:58.145607+0000","last_fullsized":"2023-11-08T19:26:58.145607+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_clean_scrub_stamp":"2023-11-08T19:26:56.117060+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:49:05.408409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":36,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.147690+0000","last_change":"2023-11-08T19:26:06.296937+0000","last_active":"2023-11-08T19:26:57.147690+0000","last_peered":"2023-11-08T19:26:57.147690+0000","last_clean":"2023-11-08T19:26:57.147690+0000","last_became_active":"2023-11-08T19:26:04.599999+0000","last_became_peered":"2023-11-08T19:26:04.599999+0000","last_unstale":"2023-11-08T19:26:57.147690+0000","last_undegraded":"2023-11-08T19:26:57.147690+0000","last_fullsized":"2023-11-08T19:26:57.147690+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:50:43.827570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025605700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-08T19:26:58.585354+0000","last_change":"2023-11-08T19:26:57.587741+0000","last_active":"2023-11-08T19:26:58.585354+0000","last_peered":"2023-11-08T19:26:58.585354+0000","last_clean":"2023-11-08T19:26:58.585354+0000","last_became_active":"2023-11-08T19:26:57.587468+0000","last_became_peered":"2023-11-08T19:26:57.587468+0000","last_unstale":"2023-11-08T19:26:58.585354+0000","last_undegraded":"2023-11-08T19:26:58.585354+0000","last_fullsized":"2023-11-08T19:26:58.585354+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:37:13.974293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.583790+0000","last_change":"2023-11-08T19:26:57.150200+0000","last_active":"2023-11-08T19:26:57.583790+0000","last_peered":"2023-11-08T19:26:57.583790+0000","last_clean":"2023-11-08T19:26:57.583790+0000","last_became_active":"2023-11-08T19:26:57.149872+0000","last_became_peered":"2023-11-08T19:26:57.149872+0000","last_unstale":"2023-11-08T19:26:57.583790+0000","last_undegraded":"2023-11-08T19:26:57.583790+0000","last_fullsized":"2023-11-08T19:26:57.583790+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_clean_scrub_stamp":"2023-11-08T19:26:56.117060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:45:33.328935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":37,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.138914+0000","last_change":"2023-11-08T19:26:06.424511+0000","last_active":"2023-11-08T19:26:57.138914+0000","last_peered":"2023-11-08T19:26:57.138914+0000","last_clean":"2023-11-08T19:26:57.138914+0000","last_became_active":"2023-11-08T19:26:04.598973+0000","last_became_peered":"2023-11-08T19:26:04.598973+0000","last_unstale":"2023-11-08T19:26:57.138914+0000","last_undegraded":"2023-11-08T19:26:57.138914+0000","last_fullsized":"2023-11-08T19:26:57.138914+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:41:53.497456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031755900000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-08T19:26:58.585138+0000","last_change":"2023-11-08T19:26:57.603772+0000","last_active":"2023-11-08T19:26:58.585138+0000","last_peered":"2023-11-08T19:26:58.585138+0000","last_clean":"2023-11-08T19:26:58.585138+0000","last_became_active":"2023-11-08T19:26:57.603487+0000","last_became_peered":"2023-11-08T19:26:57.603487+0000","last_unstale":"2023-11-08T19:26:58.585138+0000","last_undegraded":"2023-11-08T19:26:58.585138+0000","last_fullsized":"2023-11-08T19:26:58.585138+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:52:20.655917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"61'2","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-08T19:26:58.585095+0000","last_change":"2023-11-08T19:26:57.148644+0000","last_active":"2023-11-08T19:26:58.585095+0000","last_peered":"2023-11-08T19:26:58.585095+0000","last_clean":"2023-11-08T19:26:58.585095+0000","last_became_active":"2023-11-08T19:26:57.148334+0000","last_became_peered":"2023-11-08T19:26:57.148334+0000","last_unstale":"2023-11-08T19:26:58.585095+0000","last_undegraded":"2023-11-08T19:26:58.585095+0000","last_fullsized":"2023-11-08T19:26:58.585095+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_clean_scrub_stamp":"2023-11-08T19:26:56.117060+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:32:56.597293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":36,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.146253+0000","last_change":"2023-11-08T19:26:06.296634+0000","last_active":"2023-11-08T19:26:57.146253+0000","last_peered":"2023-11-08T19:26:57.146253+0000","last_clean":"2023-11-08T19:26:57.146253+0000","last_became_active":"2023-11-08T19:26:04.603088+0000","last_became_peered":"2023-11-08T19:26:04.603088+0000","last_unstale":"2023-11-08T19:26:57.146253+0000","last_undegraded":"2023-11-08T19:26:57.146253+0000","last_fullsized":"2023-11-08T19:26:57.146253+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:01:56.991346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038180899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-08T19:26:58.585061+0000","last_change":"2023-11-08T19:26:57.602122+0000","last_active":"2023-11-08T19:26:58.585061+0000","last_peered":"2023-11-08T19:26:58.585061+0000","last_clean":"2023-11-08T19:26:58.585061+0000","last_became_active":"2023-11-08T19:26:57.601572+0000","last_became_peered":"2023-11-08T19:26:57.601572+0000","last_unstale":"2023-11-08T19:26:58.585061+0000","last_undegraded":"2023-11-08T19:26:58.585061+0000","last_fullsized":"2023-11-08T19:26:58.585061+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:25:47.748513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"61'1","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-08T19:26:58.584790+0000","last_change":"2023-11-08T19:26:57.148794+0000","last_active":"2023-11-08T19:26:58.584790+0000","last_peered":"2023-11-08T19:26:58.584790+0000","last_clean":"2023-11-08T19:26:58.584790+0000","last_became_active":"2023-11-08T19:26:57.148474+0000","last_became_peered":"2023-11-08T19:26:57.148474+0000","last_unstale":"2023-11-08T19:26:58.584790+0000","last_undegraded":"2023-11-08T19:26:58.584790+0000","last_fullsized":"2023-11-08T19:26:58.584790+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_clean_scrub_stamp":"2023-11-08T19:26:56.117060+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:26:41.763289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":35,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.138624+0000","last_change":"2023-11-08T19:26:06.394300+0000","last_active":"2023-11-08T19:26:57.138624+0000","last_peered":"2023-11-08T19:26:57.138624+0000","last_clean":"2023-11-08T19:26:57.138624+0000","last_became_active":"2023-11-08T19:26:04.601685+0000","last_became_peered":"2023-11-08T19:26:04.601685+0000","last_unstale":"2023-11-08T19:26:57.138624+0000","last_undegraded":"2023-11-08T19:26:57.138624+0000","last_fullsized":"2023-11-08T19:26:57.138624+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:51:06.626474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061044099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-08T19:26:58.585687+0000","last_change":"2023-11-08T19:26:57.593050+0000","last_active":"2023-11-08T19:26:58.585687+0000","last_peered":"2023-11-08T19:26:58.585687+0000","last_clean":"2023-11-08T19:26:58.585687+0000","last_became_active":"2023-11-08T19:26:57.592759+0000","last_became_peered":"2023-11-08T19:26:57.592759+0000","last_unstale":"2023-11-08T19:26:58.585687+0000","last_undegraded":"2023-11-08T19:26:58.585687+0000","last_fullsized":"2023-11-08T19:26:58.585687+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:40:19.853844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.587822+0000","last_change":"2023-11-08T19:26:57.148765+0000","last_active":"2023-11-08T19:26:57.587822+0000","last_peered":"2023-11-08T19:26:57.587822+0000","last_clean":"2023-11-08T19:26:57.587822+0000","last_became_active":"2023-11-08T19:26:57.148468+0000","last_became_peered":"2023-11-08T19:26:57.148468+0000","last_unstale":"2023-11-08T19:26:57.587822+0000","last_undegraded":"2023-11-08T19:26:57.587822+0000","last_fullsized":"2023-11-08T19:26:57.587822+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_clean_scrub_stamp":"2023-11-08T19:26:56.117060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:26:21.727377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":42,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.584432+0000","last_change":"2023-11-08T19:26:05.609570+0000","last_active":"2023-11-08T19:26:57.584432+0000","last_peered":"2023-11-08T19:26:57.584432+0000","last_clean":"2023-11-08T19:26:57.584432+0000","last_became_active":"2023-11-08T19:26:04.608379+0000","last_became_peered":"2023-11-08T19:26:04.608379+0000","last_unstale":"2023-11-08T19:26:57.584432+0000","last_undegraded":"2023-11-08T19:26:57.584432+0000","last_fullsized":"2023-11-08T19:26:57.584432+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:40:08.986649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000529657,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.608296+0000","last_change":"2023-11-08T19:26:57.604293+0000","last_active":"2023-11-08T19:26:57.608296+0000","last_peered":"2023-11-08T19:26:57.608296+0000","last_clean":"2023-11-08T19:26:57.608296+0000","last_became_active":"2023-11-08T19:26:57.603462+0000","last_became_peered":"2023-11-08T19:26:57.603462+0000","last_unstale":"2023-11-08T19:26:57.608296+0000","last_undegraded":"2023-11-08T19:26:57.608296+0000","last_fullsized":"2023-11-08T19:26:57.608296+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:20:08.283676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"61'1","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-08T19:26:58.584751+0000","last_change":"2023-11-08T19:26:57.154388+0000","last_active":"2023-11-08T19:26:58.584751+0000","last_peered":"2023-11-08T19:26:58.584751+0000","last_clean":"2023-11-08T19:26:58.584751+0000","last_became_active":"2023-11-08T19:26:57.154161+0000","last_became_peered":"2023-11-08T19:26:57.154161+0000","last_unstale":"2023-11-08T19:26:58.584751+0000","last_undegraded":"2023-11-08T19:26:58.584751+0000","last_fullsized":"2023-11-08T19:26:58.584751+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_clean_scrub_stamp":"2023-11-08T19:26:56.117060+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:15:09.411596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":35,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.135452+0000","last_change":"2023-11-08T19:26:06.265419+0000","last_active":"2023-11-08T19:26:57.135452+0000","last_peered":"2023-11-08T19:26:57.135452+0000","last_clean":"2023-11-08T19:26:57.135452+0000","last_became_active":"2023-11-08T19:26:04.605333+0000","last_became_peered":"2023-11-08T19:26:04.605333+0000","last_unstale":"2023-11-08T19:26:57.135452+0000","last_undegraded":"2023-11-08T19:26:57.135452+0000","last_fullsized":"2023-11-08T19:26:57.135452+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:01:56.991346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064606400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-08T19:26:58.585469+0000","last_change":"2023-11-08T19:26:57.594992+0000","last_active":"2023-11-08T19:26:58.585469+0000","last_peered":"2023-11-08T19:26:58.585469+0000","last_clean":"2023-11-08T19:26:58.585469+0000","last_became_active":"2023-11-08T19:26:57.594693+0000","last_became_peered":"2023-11-08T19:26:57.594693+0000","last_unstale":"2023-11-08T19:26:58.585469+0000","last_undegraded":"2023-11-08T19:26:58.585469+0000","last_fullsized":"2023-11-08T19:26:58.585469+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:57.122180+0000","last_clean_scrub_stamp":"2023-11-08T19:26:57.122180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:00:17.866236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.586797+0000","last_change":"2023-11-08T19:26:57.150678+0000","last_active":"2023-11-08T19:26:57.586797+0000","last_peered":"2023-11-08T19:26:57.586797+0000","last_clean":"2023-11-08T19:26:57.586797+0000","last_became_active":"2023-11-08T19:26:57.150388+0000","last_became_peered":"2023-11-08T19:26:57.150388+0000","last_unstale":"2023-11-08T19:26:57.586797+0000","last_undegraded":"2023-11-08T19:26:57.586797+0000","last_fullsized":"2023-11-08T19:26:57.586797+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:56.117060+0000","last_clean_scrub_stamp":"2023-11-08T19:26:56.117060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:25:17.567988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":38,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.599249+0000","last_change":"2023-11-08T19:26:06.542044+0000","last_active":"2023-11-08T19:26:57.599249+0000","last_peered":"2023-11-08T19:26:57.599249+0000","last_clean":"2023-11-08T19:26:57.599249+0000","last_became_active":"2023-11-08T19:26:04.603056+0000","last_became_peered":"2023-11-08T19:26:04.603056+0000","last_unstale":"2023-11-08T19:26:57.599249+0000","last_undegraded":"2023-11-08T19:26:57.599249+0000","last_fullsized":"2023-11-08T19:26:57.599249+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:41:50.771457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000517414,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":208,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-08T19:26:57.584161+0000","last_change":"2023-11-08T19:24:45.888122+0000","last_active":"2023-11-08T19:26:57.584161+0000","last_peered":"2023-11-08T19:26:57.584161+0000","last_clean":"2023-11-08T19:26:57.584161+0000","last_became_active":"2023-11-08T19:24:45.887048+0000","last_became_peered":"2023-11-08T19:24:45.887048+0000","last_unstale":"2023-11-08T19:26:57.584161+0000","last_undegraded":"2023-11-08T19:26:57.584161+0000","last_fullsized":"2023-11-08T19:26:57.584161+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:44.575133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:44.575133+0000","last_clean_scrub_stamp":"2023-11-08T19:24:44.575133+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:19:20.202062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574879,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28356,"kb_used_data":1324,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343484,"statfs":{"total":96636764160,"available":96607727616,"internally_reserved":0,"allocated":1355776,"data_stored":916851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:26:48 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.504,"15min":0.498},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.78600000000000003,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.51700000000000002,"15min":0.51500000000000001},"min":{"1min":0.35099999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.65400000000000003,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.69099999999999995}]},{"osd":1,"last update":"Wed Nov 8 19:26:48 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.56999999999999995,"15min":0.56599999999999995},"min":{"1min":0.42099999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.87},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.56100000000000005,"15min":0.55500000000000005},"min":{"1min":0.432,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.76100000000000001,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.89500000000000002}]},{"osd":2,"last update":"Wed Nov 8 19:26:48 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.501,"15min":0.49099999999999999},"min":{"1min":0.39500000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.55400000000000005,"15min":0.54900000000000004},"min":{"1min":0.42999999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.80300000000000005}]},{"osd":3,"last update":"Wed Nov 8 19:26:48 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56000000000000005,"15min":0.55900000000000005},"min":{"1min":0.442,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.58899999999999997,"15min":0.57699999999999996},"min":{"1min":0.47899999999999998,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.77000000000000002}]},{"osd":4,"last update":"Wed Nov 8 19:26:48 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.51600000000000001,"15min":0.51000000000000001},"min":{"1min":0.38300000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.73299999999999998,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.54500000000000004,"15min":0.54100000000000004},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.753}]},{"osd":5,"last update":"Wed Nov 8 19:26:48 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.58699999999999997,"15min":0.58099999999999996},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.64700000000000002,"15min":0.64300000000000002},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.78300000000000003,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.84499999999999997}]},{"osd":6,"last update":"Wed Nov 8 19:26:48 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.60999999999999999,"15min":0.60699999999999998},"min":{"1min":0.47599999999999998,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.81100000000000005,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.61699999999999999,"15min":0.61199999999999999},"min":{"1min":0.47699999999999998,"5min":0.443,"15min":0.443},"max":{"1min":0.78800000000000003,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.82999999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574879,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27760,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344080,"statfs":{"total":96636764160,"available":96608337920,"internally_reserved":0,"allocated":749568,"data_stored":325741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:26:47 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":1.26,"15min":1.3799999999999999},"min":{"1min":0.33900000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.91700000000000004,"5min":21.359999999999999,"15min":21.359999999999999},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.86099999999999999,"15min":0.91300000000000003},"min":{"1min":0.38100000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.90300000000000002,"5min":10.329000000000001,"15min":10.329000000000001},"last":0.628}]},{"osd":1,"last update":"Wed Nov 8 19:26:47 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.60299999999999998,"15min":0.61699999999999999},"min":{"1min":0.35999999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.86299999999999999,"5min":3.4239999999999999,"15min":3.4239999999999999},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.63100000000000001,"15min":0.64300000000000002},"min":{"1min":0.40999999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.94999999999999996,"5min":3.4809999999999999,"15min":3.4809999999999999},"last":0.56100000000000005}]},{"osd":2,"last update":"Wed Nov 8 19:26:47 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.879,"15min":0.93100000000000005},"min":{"1min":0.38500000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.96399999999999997,"5min":10.398,"15min":10.398},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.67800000000000005,"15min":0.69799999999999995},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.89100000000000001,"5min":5.0170000000000003,"15min":5.0170000000000003},"last":0.40799999999999997}]},{"osd":3,"last update":"Wed Nov 8 19:26:47 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.65600000000000003,"15min":0.66900000000000004},"min":{"1min":0.34599999999999997,"5min":0.317,"15min":0.317},"max":{"1min":0.877,"5min":3.5710000000000002,"15min":3.5710000000000002},"last":0.496},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.91000000000000003,"15min":0.96399999999999997},"min":{"1min":0.41099999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.93500000000000005,"5min":10.368,"15min":10.368},"last":0.54000000000000004}]},{"osd":4,"last update":"Wed Nov 8 19:26:47 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.52500000000000002,"15min":0.51300000000000001},"min":{"1min":0.36899999999999999,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.56499999999999995,"15min":0.56000000000000005},"min":{"1min":0.39200000000000002,"5min":0.373,"15min":0.373},"max":{"1min":0.753,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.47899999999999998}]},{"osd":5,"last update":"Wed Nov 8 19:26:47 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.53900000000000003,"15min":0.52900000000000003},"min":{"1min":0.41599999999999998,"5min":0.307,"15min":0.307},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.58999999999999997,"15min":0.58099999999999996},"min":{"1min":0.51600000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.64000000000000001}]},{"osd":7,"last update":"Wed Nov 8 19:26:47 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.58499999999999996,"15min":0.57499999999999996},"min":{"1min":0.48599999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.84199999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.59799999999999998,"15min":0.58999999999999997},"min":{"1min":0.498,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.80500000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.57799999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574879,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27768,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344072,"statfs":{"total":96636764160,"available":96608329728,"internally_reserved":0,"allocated":761856,"data_stored":321430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:26:47 2023","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.377,"15min":0.38100000000000001},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.499,"5min":1.389,"15min":1.389},"last":0.28899999999999998},{"interface":"front","average":{"1min":0.34899999999999998,"5min":0.66500000000000004,"15min":0.71699999999999997},"min":{"1min":0.23499999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.55200000000000005,"5min":9.1370000000000005,"15min":9.1370000000000005},"last":0.25900000000000001}]},{"osd":2,"last update":"Wed Nov 8 19:26:47 2023","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.35999999999999999,"15min":0.36499999999999999},"min":{"1min":0.25600000000000001,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.47799999999999998,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.42299999999999999,"15min":0.42899999999999999},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.56399999999999995,"5min":1.425,"15min":1.425},"last":0.36099999999999999}]},{"osd":3,"last update":"Wed Nov 8 19:26:47 2023","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.80700000000000005,"15min":0.877},"min":{"1min":0.28000000000000003,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.54300000000000004,"5min":11.957000000000001,"15min":11.957000000000001},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.379,"5min":1.0209999999999999,"15min":1.1279999999999999},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.54400000000000004,"5min":18.039000000000001,"15min":18.039000000000001},"last":0.312}]},{"osd":4,"last update":"Wed Nov 8 19:26:47 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.78300000000000003,"15min":0.81399999999999995},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.4370000000000001,"5min":6.181,"15min":6.181},"last":0.442},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.64300000000000002,"15min":0.64100000000000001},"min":{"1min":0.45200000000000001,"5min":0.377,"15min":0.377},"max":{"1min":1.411,"5min":1.488,"15min":1.488},"last":0.498}]},{"osd":5,"last update":"Wed Nov 8 19:26:47 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.92000000000000004,"15min":0.96399999999999997},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.421,"5min":9.2420000000000009,"15min":9.2420000000000009},"last":0.437},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.83299999999999996,"15min":0.86499999999999999},"min":{"1min":0.439,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.2969999999999999,"5min":6.2110000000000003,"15min":6.2110000000000003},"last":0.51200000000000001}]},{"osd":6,"last update":"Wed Nov 8 19:26:47 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.92800000000000005,"15min":0.96699999999999997},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.4530000000000001,"5min":9.1880000000000006,"15min":9.1880000000000006},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.272,"5min":1.462,"15min":1.462},"last":0.52900000000000003}]},{"osd":7,"last update":"Wed Nov 8 19:26:47 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.82699999999999996,"15min":0.85499999999999998},"min":{"1min":0.46899999999999997,"5min":0.435,"15min":0.435},"max":{"1min":1.3979999999999999,"5min":6.1479999999999997,"15min":6.1479999999999997},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.95699999999999996,"15min":1.004},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.353,"5min":9.2170000000000005,"15min":9.2170000000000005},"last":0.46100000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574879,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28324,"kb_used_data":1296,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343516,"statfs":{"total":96636764160,"available":96607760384,"internally_reserved":0,"allocated":1327104,"data_stored":909422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Nov 8 19:26:46 2023","interfaces":[{"interface":"back","average":{"1min":0.31900000000000001,"5min":0.48199999999999998,"15min":0.50900000000000001},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.45200000000000001,"5min":4.1200000000000001,"15min":4.1200000000000001},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.36199999999999999,"5min":1.026,"15min":1.137},"min":{"1min":0.26200000000000001,"5min":0.253,"15min":0.253},"max":{"1min":0.52200000000000002,"5min":16.940999999999999,"15min":16.940999999999999},"last":0.317}]},{"osd":2,"last update":"Wed Nov 8 19:26:46 2023","interfaces":[{"interface":"back","average":{"1min":0.34200000000000003,"5min":0.39100000000000001,"15min":0.39900000000000002},"min":{"1min":0.25700000000000001,"5min":0.246,"15min":0.246},"max":{"1min":0.54500000000000004,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.22},{"interface":"front","average":{"1min":0.378,"5min":0.64500000000000002,"15min":0.68899999999999995},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.56899999999999995,"5min":7.0209999999999999,"15min":7.0209999999999999},"last":0.27300000000000002}]},{"osd":3,"last update":"Wed Nov 8 19:26:46 2023","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.53700000000000003,"15min":0.56299999999999994},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.51500000000000001,"5min":4.0940000000000003,"15min":4.0940000000000003},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.439,"15min":0.44700000000000001},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.62,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.29199999999999998}]},{"osd":4,"last update":"Wed Nov 8 19:26:46 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.73599999999999999,"15min":0.76600000000000001},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.70799999999999996,"5min":4.0049999999999999,"15min":4.0049999999999999},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.63400000000000001,"15min":0.64000000000000001},"min":{"1min":0.45700000000000002,"5min":0.436,"15min":0.436},"max":{"1min":0.79200000000000004,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.51300000000000001}]},{"osd":5,"last update":"Wed Nov 8 19:26:46 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.753,"15min":0.78000000000000003},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.81699999999999995,"5min":4.3090000000000002,"15min":4.3090000000000002},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.745,"15min":0.76900000000000002},"min":{"1min":0.496,"5min":0.437,"15min":0.437},"max":{"1min":0.77200000000000002,"5min":4.0709999999999997,"15min":4.0709999999999997},"last":0.69999999999999996}]},{"osd":6,"last update":"Wed Nov 8 19:26:46 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.66500000000000004,"15min":0.67300000000000004},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.82299999999999995,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.80900000000000005,"15min":0.83799999999999997},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.76000000000000001,"5min":4.2560000000000002,"15min":4.2560000000000002},"last":0.62}]},{"osd":7,"last update":"Wed Nov 8 19:26:46 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.76600000000000001,"15min":0.79000000000000004},"min":{"1min":0.49299999999999999,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.81299999999999994,"5min":4.2850000000000001,"15min":4.2850000000000001},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.79200000000000004,"15min":0.81799999999999995},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.85699999999999998,"5min":4.0460000000000003,"15min":4.0460000000000003},"last":0.71699999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574879,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27740,"kb_used_data":716,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344100,"statfs":{"total":96636764160,"available":96608358400,"internally_reserved":0,"allocated":733184,"data_stored":319054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:26:46 2023","interfaces":[{"interface":"back","average":{"1min":0.34300000000000003,"5min":0.33400000000000002,"15min":0.33200000000000002},"min":{"1min":0.254,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.496,"5min":0.496,"15min":0.496},"last":0.21099999999999999},{"interface":"front","average":{"1min":0.309,"5min":0.58899999999999997,"15min":0.63600000000000001},"min":{"1min":0.23400000000000001,"5min":0.218,"15min":0.218},"max":{"1min":0.45600000000000002,"5min":6.0999999999999996,"15min":6.0999999999999996},"last":0.23000000000000001}]},{"osd":1,"last update":"Wed Nov 8 19:26:46 2023","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.44,"5min":0.48299999999999998,"15min":0.48299999999999998},"last":0.27100000000000002},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.38600000000000001,"15min":0.38900000000000001},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.48899999999999999,"5min":0.624,"15min":0.624},"last":0.28699999999999998}]},{"osd":3,"last update":"Wed Nov 8 19:26:46 2023","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.38100000000000001,"15min":0.38400000000000001},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.46000000000000002,"5min":0.53700000000000003,"15min":0.53700000000000003},"last":0.317},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.39000000000000001,"15min":0.39400000000000002},"min":{"1min":0.29399999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.439,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.30099999999999999}]},{"osd":4,"last update":"Wed Nov 8 19:26:46 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.57899999999999996,"15min":0.58299999999999996},"min":{"1min":0.46000000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.71999999999999997,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.57999999999999996,"15min":0.58299999999999996},"min":{"1min":0.40999999999999998,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.74199999999999999,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.498}]},{"osd":5,"last update":"Wed Nov 8 19:26:46 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.58599999999999997,"15min":0.58299999999999996},"min":{"1min":0.50700000000000001,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.76400000000000001,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.60899999999999999,"15min":0.61199999999999999},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.76100000000000001,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.41999999999999998}]},{"osd":6,"last update":"Wed Nov 8 19:26:46 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.59799999999999998,"15min":0.59999999999999998},"min":{"1min":0.47699999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.77300000000000002,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.40300000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.70699999999999996,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.55000000000000004}]},{"osd":7,"last update":"Wed Nov 8 19:26:46 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.58199999999999996,"15min":0.57999999999999996},"min":{"1min":0.497,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.81499999999999995,"5min":0.878,"15min":0.878},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.60099999999999998,"15min":0.60299999999999998},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.80000000000000004,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.53100000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574880,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27740,"kb_used_data":716,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344100,"statfs":{"total":96636764160,"available":96608358400,"internally_reserved":0,"allocated":733184,"data_stored":319054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:26:49 2023","interfaces":[{"interface":"back","average":{"1min":0.32100000000000001,"5min":0.33800000000000002,"15min":0.34100000000000003},"min":{"1min":0.20300000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.42299999999999999,"5min":0.626,"15min":0.626},"last":0.30399999999999999},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.34999999999999998,"15min":0.34899999999999998},"min":{"1min":0.28100000000000003,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.46899999999999997,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.33100000000000002}]},{"osd":1,"last update":"Wed Nov 8 19:26:49 2023","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.35099999999999998,"15min":0.35299999999999998},"min":{"1min":0.24099999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.42699999999999999,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"min":{"1min":0.27900000000000003,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.49199999999999999,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.36399999999999999}]},{"osd":2,"last update":"Wed Nov 8 19:26:49 2023","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.374,"15min":0.371},"min":{"1min":0.30199999999999999,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.48199999999999998,"5min":0.58499999999999996,"15min":0.58499999999999996},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.40500000000000003,"15min":0.40400000000000003},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"last":0.40100000000000002}]},{"osd":4,"last update":"Wed Nov 8 19:26:49 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.60399999999999998,"15min":0.59999999999999998},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.80200000000000005,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.628},{"interface":"front","average":{"1min":0.624,"5min":0.60799999999999998,"15min":0.60599999999999998},"min":{"1min":0.45000000000000001,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.79000000000000004,"5min":0.871,"15min":0.871},"last":0.67800000000000005}]},{"osd":5,"last update":"Wed Nov 8 19:26:49 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.58299999999999996,"15min":0.57699999999999996},"min":{"1min":0.39800000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.625,"5min":0.58599999999999997,"15min":0.57999999999999996},"min":{"1min":0.499,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.81999999999999995,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.59099999999999997}]},{"osd":6,"last update":"Wed Nov 8 19:26:49 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.58399999999999996,"15min":0.58099999999999996},"min":{"1min":0.42999999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.61299999999999999,"15min":0.60699999999999998},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.84099999999999997,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.52900000000000003}]},{"osd":7,"last update":"Wed Nov 8 19:26:49 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.59599999999999997,"15min":0.58999999999999997},"min":{"1min":0.46400000000000002,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.88100000000000001,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.59799999999999998,"15min":0.59599999999999997},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.76600000000000001,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.57299999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574879,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27752,"kb_used_data":728,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344088,"statfs":{"total":96636764160,"available":96608346112,"internally_reserved":0,"allocated":745472,"data_stored":320666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:26:50 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.73099999999999998,"15min":0.76000000000000001},"min":{"1min":0.40999999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.78100000000000003,"5min":6.5970000000000004,"15min":6.5970000000000004},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.56599999999999995,"15min":0.57399999999999995},"min":{"1min":0.378,"5min":0.317,"15min":0.317},"max":{"1min":0.75900000000000001,"5min":2.5030000000000001,"15min":2.5030000000000001},"last":0.436}]},{"osd":1,"last update":"Wed Nov 8 19:26:50 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.50700000000000001,"15min":0.499},"min":{"1min":0.372,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.51600000000000001,"15min":0.51000000000000001},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.46400000000000002}]},{"osd":2,"last update":"Wed Nov 8 19:26:50 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.55500000000000005,"15min":0.55200000000000005},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.623,"5min":0.80100000000000005,"15min":0.83099999999999996},"min":{"1min":0.45900000000000002,"5min":0.435,"15min":0.435},"max":{"1min":0.871,"5min":6.6920000000000002,"15min":6.6920000000000002},"last":0.38600000000000001}]},{"osd":3,"last update":"Wed Nov 8 19:26:50 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58799999999999997,"15min":0.58899999999999997},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.71099999999999997,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.627,"5min":0.61399999999999999,"15min":0.61199999999999999},"min":{"1min":0.46800000000000003,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.48799999999999999}]},{"osd":5,"last update":"Wed Nov 8 19:26:50 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.34300000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.59099999999999997,"15min":0.59499999999999997},"min":{"1min":0.42999999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.876,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.505}]},{"osd":6,"last update":"Wed Nov 8 19:26:50 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.61799999999999999,"15min":0.61499999999999999},"min":{"1min":0.38900000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.78100000000000003,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.56200000000000006}]},{"osd":7,"last update":"Wed Nov 8 19:26:50 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.625,"15min":0.63200000000000001},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.82699999999999996,"5min":1.141,"15min":1.141},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.63900000000000001,"15min":0.64500000000000002},"min":{"1min":0.38700000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.85299999999999998,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.58899999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574879,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27744,"kb_used_data":720,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344096,"statfs":{"total":96636764160,"available":96608354304,"internally_reserved":0,"allocated":737280,"data_stored":319076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:26:44 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.51700000000000002,"15min":0.52100000000000002},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.502},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.51100000000000001,"15min":0.50900000000000001},"min":{"1min":0.373,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.65100000000000002,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.58499999999999996}]},{"osd":1,"last update":"Wed Nov 8 19:26:44 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.52200000000000002,"15min":0.52300000000000002},"min":{"1min":0.42099999999999999,"5min":0.371,"15min":0.371},"max":{"1min":0.65300000000000002,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53500000000000003,"15min":0.53600000000000003},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.44800000000000001}]},{"osd":2,"last update":"Wed Nov 8 19:26:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.53800000000000003,"15min":0.53600000000000003},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.56599999999999995,"15min":0.57299999999999995},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.69599999999999995,"5min":0.747,"15min":0.747},"last":0.627}]},{"osd":3,"last update":"Wed Nov 8 19:26:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.54600000000000004,"15min":0.54900000000000004},"min":{"1min":0.39700000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.59099999999999997,"15min":0.59999999999999998},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.70099999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.42799999999999999}]},{"osd":4,"last update":"Wed Nov 8 19:26:44 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.55100000000000005,"15min":0.55900000000000005},"min":{"1min":0.38600000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.63300000000000001,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.39100000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.71399999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.40999999999999998}]},{"osd":6,"last update":"Wed Nov 8 19:26:44 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.60899999999999999,"15min":0.61199999999999999},"min":{"1min":0.42099999999999999,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.61099999999999999,"15min":0.61499999999999999},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.67700000000000005,"5min":0.745,"15min":0.745},"last":0.59999999999999998}]},{"osd":7,"last update":"Wed Nov 8 19:26:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.60299999999999998,"15min":0.60899999999999999},"min":{"1min":0.371,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.63900000000000001,"15min":0.64700000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.77100000000000002,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.57499999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-08T19:27:02.620 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-08T19:27:02.620 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-08T19:27:02.779 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.777+0000 7f464442f700 1 Processor -- start 2023-11-08T19:27:02.779 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.777+0000 7f464442f700 1 -- start start 2023-11-08T19:27:02.782 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.777+0000 7f464442f700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f463c15dba0 0x7f463c06c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:02.783 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.778+0000 7f464442f700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f463c06b240 0x7f463c0caff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:02.783 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.778+0000 7f464442f700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f463c161890 con 0x7f463c0cb530 2023-11-08T19:27:02.783 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.778+0000 7f464442f700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f463c06b740 con 0x7f463c15dba0 2023-11-08T19:27:02.783 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.778+0000 7f464442f700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f463c06ba20 con 0x7f463c06b240 2023-11-08T19:27:02.783 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.778+0000 7f46421cb700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f463c15dba0 0x7f463c06c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:02.784 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.778+0000 7f46421cb700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f463c15dba0 0x7f463c06c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52642/0 (socket says 172.21.15.143:52642) 2023-11-08T19:27:02.784 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.778+0000 7f46421cb700 1 -- 172.21.15.143:0/2615492842 learned_addr learned my addr 172.21.15.143:0/2615492842 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:02.784 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.778+0000 7f46419ca700 1 --2- 172.21.15.143:0/2615492842 >> v2:172.21.15.174:3300/0 conn(0x7f463c06b240 0x7f463c0caff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:02.784 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.778+0000 7f46421cb700 1 -- 172.21.15.143:0/2615492842 >> v2:172.21.15.174:3300/0 conn(0x7f463c06b240 msgr2=0x7f463c0caff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:02.784 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.778+0000 7f46421cb700 1 --2- 172.21.15.143:0/2615492842 >> v2:172.21.15.174:3300/0 conn(0x7f463c06b240 0x7f463c0caff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:02.785 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.778+0000 7f46421cb700 1 -- 172.21.15.143:0/2615492842 >> v1:172.21.15.174:6789/0 conn(0x7f463c0cb530 legacy=0x7f463c0b22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:02.785 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.778+0000 7f46421cb700 1 -- 172.21.15.143:0/2615492842 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f463c06bd00 con 0x7f463c15dba0 2023-11-08T19:27:02.785 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.778+0000 7f46421cb700 1 --2- 172.21.15.143:0/2615492842 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f463c15dba0 0x7f463c06c610 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f463c0d0810 tx=0x7f4638009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2c59bd54be010c8 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:02.785 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.778+0000 7f46411c9700 1 -- 172.21.15.143:0/2615492842 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4638004230 con 0x7f463c15dba0 2023-11-08T19:27:02.785 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.778+0000 7f46411c9700 1 -- 172.21.15.143:0/2615492842 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4638004390 con 0x7f463c15dba0 2023-11-08T19:27:02.786 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.778+0000 7f46411c9700 1 -- 172.21.15.143:0/2615492842 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46380036a0 con 0x7f463c15dba0 2023-11-08T19:27:02.786 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.779+0000 7f464442f700 1 -- 172.21.15.143:0/2615492842 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f463c15dba0 msgr2=0x7f463c06c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:02.786 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.779+0000 7f464442f700 1 --2- 172.21.15.143:0/2615492842 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f463c15dba0 0x7f463c06c610 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f463c0d0810 tx=0x7f4638009bd0 comp rx=0 tx=0).stop 2023-11-08T19:27:02.786 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.779+0000 7f464442f700 1 -- 172.21.15.143:0/2615492842 shutdown_connections 2023-11-08T19:27:02.786 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.779+0000 7f464442f700 1 --2- 172.21.15.143:0/2615492842 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f463c15dba0 0x7f463c06c610 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:02.787 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.779+0000 7f464442f700 1 --2- 172.21.15.143:0/2615492842 >> v2:172.21.15.174:3300/0 conn(0x7f463c06b240 0x7f463c0caff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:02.787 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.779+0000 7f464442f700 1 -- 172.21.15.143:0/2615492842 >> 172.21.15.143:0/2615492842 conn(0x7f463c0cd2e0 msgr2=0x7f463c0cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:02.787 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.779+0000 7f464442f700 1 -- 172.21.15.143:0/2615492842 shutdown_connections 2023-11-08T19:27:02.787 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.779+0000 7f464442f700 1 -- 172.21.15.143:0/2615492842 wait complete. 2023-11-08T19:27:02.787 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.780+0000 7f464442f700 1 Processor -- start 2023-11-08T19:27:02.788 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.780+0000 7f464442f700 1 -- start start 2023-11-08T19:27:02.790 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.781+0000 7f464442f700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f463c06b240 0x7f463c0ad470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:02.790 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.781+0000 7f464442f700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f463c0cb530 0x7f463c15a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:02.790 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.781+0000 7f464442f700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f463c15dba0 0x7f463c15a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:02.791 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.781+0000 7f464442f700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f463c0b4ef0 con 0x7f463c06b240 2023-11-08T19:27:02.791 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.781+0000 7f464442f700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f463c0ac990 con 0x7f463c15dba0 2023-11-08T19:27:02.791 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.781+0000 7f464442f700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f463c0acc70 con 0x7f463c0cb530 2023-11-08T19:27:02.791 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.781+0000 7f46421cb700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f463c06b240 0x7f463c0ad470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:02.791 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.781+0000 7f46421cb700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f463c06b240 0x7f463c0ad470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52648/0 (socket says 172.21.15.143:52648) 2023-11-08T19:27:02.792 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.781+0000 7f46421cb700 1 -- 172.21.15.143:0/668720333 learned_addr learned my addr 172.21.15.143:0/668720333 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:02.792 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.781+0000 7f46419ca700 1 --2- 172.21.15.143:0/668720333 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f463c0cb530 0x7f463c15a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:02.792 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.781+0000 7f46429cc700 1 --2- 172.21.15.143:0/668720333 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f463c15dba0 0x7f463c15a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:02.792 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.781+0000 7f46421cb700 1 -- 172.21.15.143:0/668720333 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f463c0cb530 msgr2=0x7f463c15a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:02.792 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.781+0000 7f46421cb700 1 --2- 172.21.15.143:0/668720333 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f463c0cb530 0x7f463c15a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:02.793 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.781+0000 7f46421cb700 1 -- 172.21.15.143:0/668720333 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f463c15dba0 msgr2=0x7f463c15a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:02.793 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.781+0000 7f46421cb700 1 --2- 172.21.15.143:0/668720333 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f463c15dba0 0x7f463c15a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:02.793 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.781+0000 7f46421cb700 1 -- 172.21.15.143:0/668720333 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4638009550 con 0x7f463c06b240 2023-11-08T19:27:02.793 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.781+0000 7f46419ca700 1 --2- 172.21.15.143:0/668720333 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f463c0cb530 0x7f463c15a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:27:02.793 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.781+0000 7f46421cb700 1 --2- 172.21.15.143:0/668720333 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f463c06b240 0x7f463c0ad470 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f4638009130 tx=0x7f4638004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:02.794 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.781+0000 7f4632ffd700 1 -- 172.21.15.143:0/668720333 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4638011630 con 0x7f463c06b240 2023-11-08T19:27:02.794 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.782+0000 7f4632ffd700 1 -- 172.21.15.143:0/668720333 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4638011790 con 0x7f463c06b240 2023-11-08T19:27:02.794 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.782+0000 7f4632ffd700 1 -- 172.21.15.143:0/668720333 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4638011b40 con 0x7f463c06b240 2023-11-08T19:27:02.794 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.782+0000 7f464442f700 1 -- 172.21.15.143:0/668720333 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f463c0acf50 con 0x7f463c06b240 2023-11-08T19:27:02.794 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.782+0000 7f464442f700 1 -- 172.21.15.143:0/668720333 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f463c0cc550 con 0x7f463c06b240 2023-11-08T19:27:02.795 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.782+0000 7f464442f700 1 -- 172.21.15.143:0/668720333 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f463c0ad160 con 0x7f463c06b240 2023-11-08T19:27:02.795 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.783+0000 7f4632ffd700 1 -- 172.21.15.143:0/668720333 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4638011ca0 con 0x7f463c06b240 2023-11-08T19:27:02.795 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.783+0000 7f4632ffd700 1 --2- 172.21.15.143:0/668720333 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f462006af00 0x7f462006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:02.795 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.783+0000 7f4632ffd700 1 -- 172.21.15.143:0/668720333 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f463808ac70 con 0x7f463c06b240 2023-11-08T19:27:02.795 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.783+0000 7f46419ca700 1 --2- 172.21.15.143:0/668720333 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f462006af00 0x7f462006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:02.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.784+0000 7f46419ca700 1 --2- 172.21.15.143:0/668720333 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f462006af00 0x7f462006d3c0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f462c0099f0 tx=0x7f462c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:02.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:02.788+0000 7f4632ffd700 1 -- 172.21.15.143:0/668720333 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4638055d00 con 0x7f463c06b240 2023-11-08T19:27:03.057 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.054+0000 7f464442f700 1 -- 172.21.15.143:0/668720333 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f463c04eee0 con 0x7f463c06b240 2023-11-08T19:27:03.059 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.056+0000 7f4632ffd700 1 -- 172.21.15.143:0/668720333 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v11) v1 ==== 190+0+147 (secure 0 0 0) 0x7f463801f0e0 con 0x7f463c06b240 2023-11-08T19:27:03.059 INFO:teuthology.orchestra.run.smithi143.stdout:[client.0] 2023-11-08T19:27:03.059 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQBP4Etl0GdSFxAAVkuHL0EnjrtTUMkj/afPkg== 2023-11-08T19:27:03.060 INFO:teuthology.orchestra.run.smithi143.stdout: caps mds = "allow" 2023-11-08T19:27:03.060 INFO:teuthology.orchestra.run.smithi143.stdout: caps mon = "allow r" 2023-11-08T19:27:03.060 INFO:teuthology.orchestra.run.smithi143.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-11-08T19:27:03.062 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.059+0000 7f464442f700 1 -- 172.21.15.143:0/668720333 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f462006af00 msgr2=0x7f462006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:03.063 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.059+0000 7f464442f700 1 --2- 172.21.15.143:0/668720333 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f462006af00 0x7f462006d3c0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f462c0099f0 tx=0x7f462c008040 comp rx=0 tx=0).stop 2023-11-08T19:27:03.063 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.059+0000 7f464442f700 1 -- 172.21.15.143:0/668720333 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f463c06b240 msgr2=0x7f463c0ad470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:03.063 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.059+0000 7f464442f700 1 --2- 172.21.15.143:0/668720333 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f463c06b240 0x7f463c0ad470 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f4638009130 tx=0x7f4638004750 comp rx=0 tx=0).stop 2023-11-08T19:27:03.063 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.059+0000 7f464442f700 1 -- 172.21.15.143:0/668720333 shutdown_connections 2023-11-08T19:27:03.064 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.059+0000 7f464442f700 1 --2- 172.21.15.143:0/668720333 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f462006af00 0x7f462006d3c0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:03.064 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.059+0000 7f464442f700 1 --2- 172.21.15.143:0/668720333 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f463c15dba0 0x7f463c15a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:03.064 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.059+0000 7f464442f700 1 --2- 172.21.15.143:0/668720333 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f463c06b240 0x7f463c0ad470 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:03.064 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.059+0000 7f464442f700 1 --2- 172.21.15.143:0/668720333 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f463c0cb530 0x7f463c15a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:03.065 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.059+0000 7f464442f700 1 -- 172.21.15.143:0/668720333 >> 172.21.15.143:0/668720333 conn(0x7f463c0cd2e0 msgr2=0x7f463c15ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:03.065 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.059+0000 7f464442f700 1 -- 172.21.15.143:0/668720333 shutdown_connections 2023-11-08T19:27:03.065 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.059+0000 7f464442f700 1 -- 172.21.15.143:0/668720333 wait complete. 2023-11-08T19:27:03.065 INFO:teuthology.orchestra.run.smithi143.stderr:updated caps for client.0 2023-11-08T19:27:03.074 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:27:03.114 INFO:tasks.ceph.mds.c.smithi143.stderr:2023-11-08T19:27:03.112+0000 7fc300b59700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-08T19:27:03.196 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.194+0000 7f7331988700 1 Processor -- start 2023-11-08T19:27:03.196 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.194+0000 7f7331988700 1 -- start start 2023-11-08T19:27:03.199 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.195+0000 7f7331988700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f732c15dc80 0x7f732c0d1f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:03.199 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.195+0000 7f7331988700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f732c0d3690 0x7f732c0b9690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:03.199 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.195+0000 7f7331988700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f732c0b7cf0 con 0x7f732c0ce980 2023-11-08T19:27:03.199 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.195+0000 7f7331988700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f732c0b9bd0 con 0x7f732c15dc80 2023-11-08T19:27:03.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.195+0000 7f7331988700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f732c0d3200 con 0x7f732c0d3690 2023-11-08T19:27:03.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.195+0000 7f732affd700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f732c15dc80 0x7f732c0d1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:03.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.195+0000 7f732affd700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f732c15dc80 0x7f732c0d1f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52656/0 (socket says 172.21.15.143:52656) 2023-11-08T19:27:03.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.195+0000 7f732affd700 1 -- 172.21.15.143:0/1689926184 learned_addr learned my addr 172.21.15.143:0/1689926184 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:03.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.195+0000 7f732a7fc700 1 --2- 172.21.15.143:0/1689926184 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f732c0d3690 0x7f732c0b9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:03.201 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.195+0000 7f732affd700 1 -- 172.21.15.143:0/1689926184 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f732c0d3690 msgr2=0x7f732c0b9690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:03.201 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.195+0000 7f732affd700 1 --2- 172.21.15.143:0/1689926184 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f732c0d3690 0x7f732c0b9690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:03.201 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.195+0000 7f732affd700 1 -- 172.21.15.143:0/1689926184 >> v1:172.21.15.174:6789/0 conn(0x7f732c0ce980 legacy=0x7f732c0b5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:03.201 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.195+0000 7f732affd700 1 -- 172.21.15.143:0/1689926184 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f732c15e060 con 0x7f732c15dc80 2023-11-08T19:27:03.201 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.195+0000 7f732affd700 1 --2- 172.21.15.143:0/1689926184 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f732c15dc80 0x7f732c0d1f80 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f731c0098f0 tx=0x7f731c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dde24c88f0b4675 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:03.201 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.195+0000 7f7329ffb700 1 -- 172.21.15.143:0/1689926184 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f731c011830 con 0x7f732c15dc80 2023-11-08T19:27:03.202 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.195+0000 7f7329ffb700 1 -- 172.21.15.143:0/1689926184 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f731c011990 con 0x7f732c15dc80 2023-11-08T19:27:03.202 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.195+0000 7f7329ffb700 1 -- 172.21.15.143:0/1689926184 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f731c011d20 con 0x7f732c15dc80 2023-11-08T19:27:03.202 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.196+0000 7f7331988700 1 -- 172.21.15.143:0/1689926184 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f732c15dc80 msgr2=0x7f732c0d1f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:03.202 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.196+0000 7f7331988700 1 --2- 172.21.15.143:0/1689926184 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f732c15dc80 0x7f732c0d1f80 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f731c0098f0 tx=0x7f731c009c00 comp rx=0 tx=0).stop 2023-11-08T19:27:03.202 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.196+0000 7f7331988700 1 -- 172.21.15.143:0/1689926184 shutdown_connections 2023-11-08T19:27:03.203 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.196+0000 7f7331988700 1 --2- 172.21.15.143:0/1689926184 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f732c15dc80 0x7f732c0d1f80 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:03.203 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.196+0000 7f7331988700 1 --2- 172.21.15.143:0/1689926184 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f732c0d3690 0x7f732c0b9690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:03.203 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.196+0000 7f7331988700 1 -- 172.21.15.143:0/1689926184 >> 172.21.15.143:0/1689926184 conn(0x7f732c0668e0 msgr2=0x7f732c066cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:03.203 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.196+0000 7f7331988700 1 -- 172.21.15.143:0/1689926184 shutdown_connections 2023-11-08T19:27:03.203 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.196+0000 7f7331988700 1 -- 172.21.15.143:0/1689926184 wait complete. 2023-11-08T19:27:03.206 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.197+0000 7f7331988700 1 Processor -- start 2023-11-08T19:27:03.206 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.197+0000 7f7331988700 1 -- start start 2023-11-08T19:27:03.207 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.198+0000 7f7331988700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f732c0ce980 0x7f732c14f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:03.207 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.198+0000 7f7331988700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f732c0d3690 0x7f732c160af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:03.207 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.198+0000 7f7331988700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f732c15dc80 0x7f732c14fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:03.207 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.198+0000 7f7331988700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f732c0c9de0 con 0x7f732c0d3690 2023-11-08T19:27:03.207 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.198+0000 7f7331988700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f732c14f640 con 0x7f732c15dc80 2023-11-08T19:27:03.208 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.198+0000 7f7331988700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f732c0c8f30 con 0x7f732c0ce980 2023-11-08T19:27:03.208 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.198+0000 7f732a7fc700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f732c0d3690 0x7f732c160af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:03.208 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.198+0000 7f732a7fc700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f732c0d3690 0x7f732c160af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52664/0 (socket says 172.21.15.143:52664) 2023-11-08T19:27:03.208 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.198+0000 7f732a7fc700 1 -- 172.21.15.143:0/458511546 learned_addr learned my addr 172.21.15.143:0/458511546 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:03.209 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.198+0000 7f732affd700 1 --2- 172.21.15.143:0/458511546 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f732c0ce980 0x7f732c14f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:03.209 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.198+0000 7f732b7fe700 1 --2- 172.21.15.143:0/458511546 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f732c15dc80 0x7f732c14fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:03.209 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.198+0000 7f732a7fc700 1 -- 172.21.15.143:0/458511546 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f732c0ce980 msgr2=0x7f732c14f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:03.209 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.198+0000 7f732a7fc700 1 --2- 172.21.15.143:0/458511546 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f732c0ce980 0x7f732c14f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:03.209 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.198+0000 7f732a7fc700 1 -- 172.21.15.143:0/458511546 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f732c15dc80 msgr2=0x7f732c14fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:03.210 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.198+0000 7f732a7fc700 1 --2- 172.21.15.143:0/458511546 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f732c15dc80 0x7f732c14fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:03.210 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.198+0000 7f732a7fc700 1 -- 172.21.15.143:0/458511546 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f731c009550 con 0x7f732c0d3690 2023-11-08T19:27:03.210 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.198+0000 7f732b7fe700 1 --2- 172.21.15.143:0/458511546 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f732c15dc80 0x7f732c14fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:27:03.210 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.199+0000 7f732a7fc700 1 --2- 172.21.15.143:0/458511546 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f732c0d3690 0x7f732c160af0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f732000e9b0 tx=0x7f732000ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:03.211 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.199+0000 7f7313fff700 1 -- 172.21.15.143:0/458511546 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7320009710 con 0x7f732c0d3690 2023-11-08T19:27:03.211 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.199+0000 7f7313fff700 1 -- 172.21.15.143:0/458511546 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f732000c850 con 0x7f732c0d3690 2023-11-08T19:27:03.211 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.199+0000 7f7331988700 1 -- 172.21.15.143:0/458511546 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f732c0c91d0 con 0x7f732c0d3690 2023-11-08T19:27:03.211 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.199+0000 7f7313fff700 1 -- 172.21.15.143:0/458511546 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f732000cc00 con 0x7f732c0d3690 2023-11-08T19:27:03.211 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.199+0000 7f7331988700 1 -- 172.21.15.143:0/458511546 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f732c0c9790 con 0x7f732c0d3690 2023-11-08T19:27:03.212 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.200+0000 7f7331988700 1 -- 172.21.15.143:0/458511546 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f732c0c9f90 con 0x7f732c0d3690 2023-11-08T19:27:03.212 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.200+0000 7f7313fff700 1 -- 172.21.15.143:0/458511546 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f732000cd60 con 0x7f732c0d3690 2023-11-08T19:27:03.212 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.200+0000 7f7313fff700 1 --2- 172.21.15.143:0/458511546 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f731806af00 0x7f731806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:03.212 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.201+0000 7f7313fff700 1 -- 172.21.15.143:0/458511546 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f73200897a0 con 0x7f732c0d3690 2023-11-08T19:27:03.212 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.201+0000 7f732affd700 1 --2- 172.21.15.143:0/458511546 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f731806af00 0x7f731806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:03.213 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.201+0000 7f732affd700 1 --2- 172.21.15.143:0/458511546 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f731806af00 0x7f731806d3c0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f731c014040 tx=0x7f731c008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:03.213 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.206+0000 7f7313fff700 1 -- 172.21.15.143:0/458511546 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f732c0c9f90 con 0x7f732c0d3690 2023-11-08T19:27:03.438 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.435+0000 7f7331988700 1 -- 172.21.15.143:0/458511546 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f732c0c9f90 con 0x7f732c0d3690 2023-11-08T19:27:03.439 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.436+0000 7f7313fff700 1 -- 172.21.15.143:0/458511546 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+4923 (secure 0 0 0) 0x7f732c0c9f90 con 0x7f732c0d3690 2023-11-08T19:27:03.439 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:27:03.439 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":33,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5231,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6836/4083537882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":4083537882},{"type":"v1","addr":"172.21.15.174:6836","nonce":4083537882}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":5234,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/494923804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6835","nonce":494923804},{"type":"v1","addr":"172.21.15.174:6837","nonce":494923804}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":6009,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6834/3383638390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":3383638390},{"type":"v1","addr":"172.21.15.143:6834","nonce":3383638390}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T19:26:57.522785+0000","modified":"2023-11-08T19:27:00.536968+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6012},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6012":{"gid":6012,"name":"c","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.143:6835/2613282947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":2613282947},{"type":"v1","addr":"172.21.15.143:6835","nonce":2613282947}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-08T19:27:03.441 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.438+0000 7f7331988700 1 -- 172.21.15.143:0/458511546 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f731806af00 msgr2=0x7f731806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:03.441 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.438+0000 7f7331988700 1 --2- 172.21.15.143:0/458511546 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f731806af00 0x7f731806d3c0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f731c014040 tx=0x7f731c008030 comp rx=0 tx=0).stop 2023-11-08T19:27:03.441 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.438+0000 7f7331988700 1 -- 172.21.15.143:0/458511546 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f732c0d3690 msgr2=0x7f732c160af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:03.441 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.438+0000 7f7331988700 1 --2- 172.21.15.143:0/458511546 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f732c0d3690 0x7f732c160af0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f732000e9b0 tx=0x7f732000ecc0 comp rx=0 tx=0).stop 2023-11-08T19:27:03.442 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.438+0000 7f7331988700 1 -- 172.21.15.143:0/458511546 shutdown_connections 2023-11-08T19:27:03.442 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.438+0000 7f7331988700 1 --2- 172.21.15.143:0/458511546 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f731806af00 0x7f731806d3c0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:03.442 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.438+0000 7f7331988700 1 --2- 172.21.15.143:0/458511546 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f732c0ce980 0x7f732c14f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:03.442 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.438+0000 7f7331988700 1 --2- 172.21.15.143:0/458511546 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f732c15dc80 0x7f732c14fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:03.442 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.438+0000 7f7331988700 1 --2- 172.21.15.143:0/458511546 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f732c0d3690 0x7f732c160af0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:03.443 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.438+0000 7f7331988700 1 -- 172.21.15.143:0/458511546 >> 172.21.15.143:0/458511546 conn(0x7f732c0668e0 msgr2=0x7f732c0d4300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:03.443 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.438+0000 7f7331988700 1 -- 172.21.15.143:0/458511546 shutdown_connections 2023-11-08T19:27:03.443 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.438+0000 7f7331988700 1 -- 172.21.15.143:0/458511546 wait complete. 2023-11-08T19:27:03.443 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 33 2023-11-08T19:27:03.855 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 33, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-08T19:26:57.522785+0000', 'modified': '2023-11-08T19:27:00.536968+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6012}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6012': {'gid': 6012, 'name': 'c', 'rank': 0, 'incarnation': 29, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.143:6835/2613282947', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.143:6833', 'nonce': 2613282947}, {'type': 'v1', 'addr': '172.21.15.143:6835', 'nonce': 2613282947}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-08T19:27:03.855 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-08T19:27:03.856 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-11-08T19:27:03.979 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.977+0000 7fa0b71c5700 1 Processor -- start 2023-11-08T19:27:03.980 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.977+0000 7fa0b71c5700 1 -- start start 2023-11-08T19:27:03.983 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.978+0000 7fa0b71c5700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa0b006b320 0x7fa0b0168c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:03.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.978+0000 7fa0b71c5700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0b0161df0 con 0x7fa0b01618f0 2023-11-08T19:27:03.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.978+0000 7fa0b71c5700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0b01620d0 con 0x7fa0b015dd00 2023-11-08T19:27:03.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.978+0000 7fa0b71c5700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b00cad50 con 0x7fa0b006b320 2023-11-08T19:27:03.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.978+0000 7fa0affff700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fa0b01618f0 0x7fa0b006ac10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:48824/0 (socket says 172.21.15.143:48824) 2023-11-08T19:27:03.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.978+0000 7fa0affff700 1 -- 172.21.15.143:0/3006611420 learned_addr learned my addr 172.21.15.143:0/3006611420 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:27:03.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.978+0000 7fa0af7fe700 1 -- 172.21.15.143:0/3006611420 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 456827189 0 0) 0x7fa0b0161df0 con 0x7fa0b01618f0 2023-11-08T19:27:03.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.978+0000 7fa0af7fe700 1 -- 172.21.15.143:0/3006611420 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa0a00035b0 con 0x7fa0b01618f0 2023-11-08T19:27:03.986 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.978+0000 7fa0b5762700 1 --2- 172.21.15.143:0/3006611420 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa0b006b320 0x7fa0b0168c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:03.986 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.978+0000 7fa0af7fe700 1 -- 172.21.15.143:0/3006611420 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2477776841 0 0) 0x7fa0a00035b0 con 0x7fa0b01618f0 2023-11-08T19:27:03.986 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.978+0000 7fa0af7fe700 1 -- 172.21.15.143:0/3006611420 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa0b006b320 msgr2=0x7fa0b0168c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:03.986 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.978+0000 7fa0af7fe700 1 --2- 172.21.15.143:0/3006611420 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa0b006b320 0x7fa0b0168c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:03.986 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.978+0000 7fa0af7fe700 1 -- 172.21.15.143:0/3006611420 >> v1:172.21.15.174:6789/0 conn(0x7fa0b015dd00 legacy=0x7fa0b00cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:03.987 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.978+0000 7fa0af7fe700 1 -- 172.21.15.143:0/3006611420 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0b01623b0 con 0x7fa0b01618f0 2023-11-08T19:27:03.987 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.978+0000 7fa0af7fe700 1 -- 172.21.15.143:0/3006611420 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4147756303 0 0) 0x7fa09c001670 con 0x7fa0b01618f0 2023-11-08T19:27:03.987 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.978+0000 7fa0b5762700 1 --2- 172.21.15.143:0/3006611420 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa0b006b320 0x7fa0b0168c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:27:03.987 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.978+0000 7fa0af7fe700 1 -- 172.21.15.143:0/3006611420 >> v1:172.21.15.143:6789/0 conn(0x7fa0b01618f0 legacy=0x7fa0b006ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:03.987 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.978+0000 7fa0af7fe700 1 --2- 172.21.15.143:0/3006611420 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa0a0003a30 0x7fa0a0003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:03.988 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.978+0000 7fa0af7fe700 1 --2- 172.21.15.143:0/3006611420 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa0a0004850 0x7fa0a0004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:03.988 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.978+0000 7fa0af7fe700 1 --2- 172.21.15.143:0/3006611420 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa0b006b320 0x7fa0a0005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:03.988 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.978+0000 7fa0af7fe700 1 -- 172.21.15.143:0/3006611420 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a000bb10 con 0x7fa0a0004850 2023-11-08T19:27:03.988 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.978+0000 7fa0af7fe700 1 -- 172.21.15.143:0/3006611420 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a000bc80 con 0x7fa0a0003a30 2023-11-08T19:27:03.988 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.978+0000 7fa0af7fe700 1 -- 172.21.15.143:0/3006611420 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a000be40 con 0x7fa0b006b320 2023-11-08T19:27:03.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.979+0000 7fa0affff700 1 --2- 172.21.15.143:0/3006611420 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa0a0004850 0x7fa0a0004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:03.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.979+0000 7fa0b4f61700 1 --2- 172.21.15.143:0/3006611420 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa0a0003a30 0x7fa0a0003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:03.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.979+0000 7fa0b5762700 1 --2- 172.21.15.143:0/3006611420 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa0b006b320 0x7fa0a0005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:03.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.979+0000 7fa0affff700 1 -- 172.21.15.143:0/3006611420 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa0b006b320 msgr2=0x7fa0a0005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:03.990 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.979+0000 7fa0affff700 1 --2- 172.21.15.143:0/3006611420 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa0b006b320 0x7fa0a0005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:03.990 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.979+0000 7fa0affff700 1 -- 172.21.15.143:0/3006611420 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa0a0003a30 msgr2=0x7fa0a0003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:03.990 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.979+0000 7fa0affff700 1 --2- 172.21.15.143:0/3006611420 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa0a0003a30 0x7fa0a0003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:03.990 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.979+0000 7fa0affff700 1 -- 172.21.15.143:0/3006611420 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0a00035b0 con 0x7fa0a0004850 2023-11-08T19:27:03.990 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.979+0000 7fa0affff700 1 --2- 172.21.15.143:0/3006611420 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa0a0004850 0x7fa0a0004cd0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fa09c00c240 tx=0x7fa0a0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d918f6a3877064f0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:03.991 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.979+0000 7fa0af7fe700 1 -- 172.21.15.143:0/3006611420 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa09c007c10 con 0x7fa0a0004850 2023-11-08T19:27:03.991 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.979+0000 7fa0af7fe700 1 -- 172.21.15.143:0/3006611420 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa09c00ce00 con 0x7fa0a0004850 2023-11-08T19:27:03.991 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.979+0000 7fa0af7fe700 1 -- 172.21.15.143:0/3006611420 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa09c005ba0 con 0x7fa0a0004850 2023-11-08T19:27:03.991 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.980+0000 7fa0b71c5700 1 -- 172.21.15.143:0/3006611420 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa0a0004850 msgr2=0x7fa0a0004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:03.991 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.980+0000 7fa0b71c5700 1 --2- 172.21.15.143:0/3006611420 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa0a0004850 0x7fa0a0004cd0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fa09c00c240 tx=0x7fa0a0005880 comp rx=0 tx=0).stop 2023-11-08T19:27:03.992 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.980+0000 7fa0b5762700 1 -- 172.21.15.143:0/3006611420 reap_dead start 2023-11-08T19:27:03.992 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.980+0000 7fa0b71c5700 1 -- 172.21.15.143:0/3006611420 shutdown_connections 2023-11-08T19:27:03.992 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.980+0000 7fa0b71c5700 1 -- 172.21.15.143:0/3006611420 >> 172.21.15.143:0/3006611420 conn(0x7fa0b0099ab0 msgr2=0x7fa0b00d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:03.992 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.980+0000 7fa0b71c5700 1 -- 172.21.15.143:0/3006611420 shutdown_connections 2023-11-08T19:27:03.992 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.980+0000 7fa0b71c5700 1 -- 172.21.15.143:0/3006611420 wait complete. 2023-11-08T19:27:03.993 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.981+0000 7fa0b71c5700 1 Processor -- start 2023-11-08T19:27:03.993 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.981+0000 7fa0b71c5700 1 -- start start 2023-11-08T19:27:04.001 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.982+0000 7fa0b71c5700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa0a0004850 0x7fa0b00c6a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:04.001 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.982+0000 7fa0b71c5700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa0b00b4fb0 0x7fa0b00b5430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:04.002 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.982+0000 7fa0b71c5700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa0b00b5970 0x7fa0b00c46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:04.002 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.982+0000 7fa0b71c5700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b00b5e20 con 0x7fa0b00b5970 2023-11-08T19:27:04.002 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.982+0000 7fa0b71c5700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b00b3650 con 0x7fa0b00b4fb0 2023-11-08T19:27:04.002 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.982+0000 7fa0b71c5700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b0157790 con 0x7fa0a0004850 2023-11-08T19:27:04.002 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.982+0000 7fa0b5762700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa0b00b5970 0x7fa0b00c46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:04.003 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.982+0000 7fa0b5762700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa0b00b5970 0x7fa0b00c46e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52686/0 (socket says 172.21.15.143:52686) 2023-11-08T19:27:04.003 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.982+0000 7fa0b5762700 1 -- 172.21.15.143:0/982582007 learned_addr learned my addr 172.21.15.143:0/982582007 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:04.003 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.982+0000 7fa0b4f61700 1 --2- 172.21.15.143:0/982582007 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa0a0004850 0x7fa0b00c6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:04.003 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.982+0000 7fa0affff700 1 --2- 172.21.15.143:0/982582007 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa0b00b4fb0 0x7fa0b00b5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:04.003 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.982+0000 7fa0b5762700 1 -- 172.21.15.143:0/982582007 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa0a0004850 msgr2=0x7fa0b00c6a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:04.004 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.982+0000 7fa0b5762700 1 --2- 172.21.15.143:0/982582007 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa0a0004850 0x7fa0b00c6a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:04.004 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.982+0000 7fa0b5762700 1 -- 172.21.15.143:0/982582007 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa0b00b4fb0 msgr2=0x7fa0b00b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:04.004 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.982+0000 7fa0b5762700 1 --2- 172.21.15.143:0/982582007 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa0b00b4fb0 0x7fa0b00b5430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:04.004 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.982+0000 7fa0b5762700 1 -- 172.21.15.143:0/982582007 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0b015e260 con 0x7fa0b00b5970 2023-11-08T19:27:04.004 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.982+0000 7fa0b5762700 1 --2- 172.21.15.143:0/982582007 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa0b00b5970 0x7fa0b00c46e0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fa0b0073130 tx=0x7fa0a80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:04.005 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.983+0000 7fa0ad7fa700 1 -- 172.21.15.143:0/982582007 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0a8007c00 con 0x7fa0b00b5970 2023-11-08T19:27:04.005 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.983+0000 7fa0ad7fa700 1 -- 172.21.15.143:0/982582007 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0a800f050 con 0x7fa0b00b5970 2023-11-08T19:27:04.005 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.983+0000 7fa0b71c5700 1 -- 172.21.15.143:0/982582007 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0b0157a70 con 0x7fa0b00b5970 2023-11-08T19:27:04.005 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.983+0000 7fa0ad7fa700 1 -- 172.21.15.143:0/982582007 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0a8014410 con 0x7fa0b00b5970 2023-11-08T19:27:04.005 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.983+0000 7fa0b71c5700 1 -- 172.21.15.143:0/982582007 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0b0158030 con 0x7fa0b00b5970 2023-11-08T19:27:04.006 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.985+0000 7fa0ad7fa700 1 -- 172.21.15.143:0/982582007 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa0a8014570 con 0x7fa0b00b5970 2023-11-08T19:27:04.006 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.986+0000 7fa0ad7fa700 1 --2- 172.21.15.143:0/982582007 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa09406af00 0x7fa09406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:04.006 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.986+0000 7fa0ad7fa700 1 -- 172.21.15.143:0/982582007 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa0a8089b70 con 0x7fa0b00b5970 2023-11-08T19:27:04.006 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.986+0000 7fa0b4f61700 1 --2- 172.21.15.143:0/982582007 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa09406af00 0x7fa09406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:04.006 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.986+0000 7fa0b4f61700 1 --2- 172.21.15.143:0/982582007 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa09406af00 0x7fa09406d3c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fa0b0158640 tx=0x7fa0a4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:04.007 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.988+0000 7fa0c33beb80 1 Processor -- start 2023-11-08T19:27:04.007 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.988+0000 7fa0c33beb80 1 -- start start 2023-11-08T19:27:04.007 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.988+0000 7fa0c33beb80 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55edba5426f0 0x55edba572450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:04.007 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.988+0000 7fa0c33beb80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55edba544c30 0x55edba572990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:04.007 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.988+0000 7fa0c33beb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55edba542ad0 0x55edba55cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:04.008 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.988+0000 7fa0c33beb80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edba4ae6e0 con 0x55edba544c30 2023-11-08T19:27:04.008 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.988+0000 7fa0c33beb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edba4a23b0 con 0x55edba542ad0 2023-11-08T19:27:04.008 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.988+0000 7fa0c33beb80 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x55edba49fcd0 con 0x55edba5426f0 2023-11-08T19:27:04.008 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.988+0000 7fa0affff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55edba544c30 0x55edba572990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:04.008 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.988+0000 7fa0affff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55edba544c30 0x55edba572990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52692/0 (socket says 172.21.15.143:52692) 2023-11-08T19:27:04.009 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.988+0000 7fa0affff700 1 -- 172.21.15.143:0/92829083 learned_addr learned my addr 172.21.15.143:0/92829083 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:04.009 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.988+0000 7fa0b4f61700 1 --2- 172.21.15.143:0/92829083 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55edba5426f0 0x55edba572450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:04.009 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.989+0000 7fa0b5762700 1 --2- 172.21.15.143:0/92829083 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55edba542ad0 0x55edba55cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:04.009 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.989+0000 7fa0affff700 1 -- 172.21.15.143:0/92829083 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55edba5426f0 msgr2=0x55edba572450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:04.009 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.989+0000 7fa0affff700 1 --2- 172.21.15.143:0/92829083 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55edba5426f0 0x55edba572450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:04.009 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.989+0000 7fa0affff700 1 -- 172.21.15.143:0/92829083 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55edba542ad0 msgr2=0x55edba55cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:04.010 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.989+0000 7fa0affff700 1 --2- 172.21.15.143:0/92829083 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55edba542ad0 0x55edba55cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:04.010 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.989+0000 7fa0affff700 1 -- 172.21.15.143:0/92829083 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55edba3185e0 con 0x55edba544c30 2023-11-08T19:27:04.010 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.989+0000 7fa0affff700 1 --2- 172.21.15.143:0/92829083 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55edba544c30 0x55edba572990 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fa09c011000 tx=0x7fa09c00a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30846d61ca4409f server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:04.010 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.989+0000 7fa09a7fc700 1 -- 172.21.15.143:0/92829083 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa09c007b30 con 0x55edba544c30 2023-11-08T19:27:04.010 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.989+0000 7fa09a7fc700 1 -- 172.21.15.143:0/92829083 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa09c001430 con 0x55edba544c30 2023-11-08T19:27:04.011 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.989+0000 7fa09a7fc700 1 -- 172.21.15.143:0/92829083 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa09c0017a0 con 0x55edba544c30 2023-11-08T19:27:04.011 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.990+0000 7fa0c33beb80 1 -- 172.21.15.143:0/92829083 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55edba544c30 msgr2=0x55edba572990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:04.011 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.990+0000 7fa0c33beb80 1 --2- 172.21.15.143:0/92829083 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55edba544c30 0x55edba572990 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fa09c011000 tx=0x7fa09c00a000 comp rx=0 tx=0).stop 2023-11-08T19:27:04.011 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.990+0000 7fa0c33beb80 1 -- 172.21.15.143:0/92829083 shutdown_connections 2023-11-08T19:27:04.011 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.990+0000 7fa0c33beb80 1 --2- 172.21.15.143:0/92829083 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55edba5426f0 0x55edba572450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:04.012 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.990+0000 7fa0c33beb80 1 --2- 172.21.15.143:0/92829083 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55edba542ad0 0x55edba55cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:04.012 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.990+0000 7fa0c33beb80 1 --2- 172.21.15.143:0/92829083 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55edba544c30 0x55edba572990 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:04.012 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.990+0000 7fa0c33beb80 1 -- 172.21.15.143:0/92829083 >> 172.21.15.143:0/92829083 conn(0x55edba4bd660 msgr2=0x55edba5483c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:04.012 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.990+0000 7fa0c33beb80 1 -- 172.21.15.143:0/92829083 shutdown_connections 2023-11-08T19:27:04.012 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.990+0000 7fa0c33beb80 1 -- 172.21.15.143:0/92829083 wait complete. 2023-11-08T19:27:04.013 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.990+0000 7fa0c33beb80 1 -- start start 2023-11-08T19:27:04.013 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.990+0000 7fa0c33beb80 1 Processor -- start 2023-11-08T19:27:04.013 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.990+0000 7fa0c33beb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55edba5426f0 0x55edba59d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:04.013 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.990+0000 7fa0c33beb80 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55edba542ad0 0x55edba585990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:04.013 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.990+0000 7fa0c33beb80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55edba585ed0 0x55edba587fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:04.014 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.990+0000 7fa0c33beb80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edba49b5c0 con 0x55edba585ed0 2023-11-08T19:27:04.014 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.990+0000 7fa0c33beb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edba4370d0 con 0x55edba5426f0 2023-11-08T19:27:04.014 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.990+0000 7fa0c33beb80 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x55edba436d70 con 0x55edba542ad0 2023-11-08T19:27:04.014 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.990+0000 7fa0c33beb80 10 client.0 _ll_register_callbacks cb 0x55edba2937b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T19:27:04.014 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.990+0000 7fa0b5762700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55edba585ed0 0x55edba587fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:04.014 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.990+0000 7fa0b5762700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55edba585ed0 0x55edba587fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52700/0 (socket says 172.21.15.143:52700) 2023-11-08T19:27:04.015 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.990+0000 7fa0b5762700 1 -- 172.21.15.143:0/3035852892 learned_addr learned my addr 172.21.15.143:0/3035852892 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:04.015 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.990+0000 7fa0affff700 1 --2- 172.21.15.143:0/3035852892 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55edba542ad0 0x55edba585990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:04.015 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.990+0000 7fa0b4f61700 1 --2- 172.21.15.143:0/3035852892 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55edba5426f0 0x55edba59d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:04.015 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.992+0000 7fa0b5762700 1 -- 172.21.15.143:0/3035852892 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55edba542ad0 msgr2=0x55edba585990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:04.015 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.992+0000 7fa0b5762700 1 --2- 172.21.15.143:0/3035852892 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55edba542ad0 0x55edba585990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:04.016 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.992+0000 7fa0b5762700 1 -- 172.21.15.143:0/3035852892 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55edba5426f0 msgr2=0x55edba59d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:04.016 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.992+0000 7fa0b5762700 1 --2- 172.21.15.143:0/3035852892 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55edba5426f0 0x55edba59d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:04.016 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.992+0000 7fa0b5762700 1 -- 172.21.15.143:0/3035852892 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0a00035b0 con 0x55edba585ed0 2023-11-08T19:27:04.016 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.992+0000 7fa0b4f61700 1 --2- 172.21.15.143:0/3035852892 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55edba5426f0 0x55edba59d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:27:04.016 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.992+0000 7fa0affff700 1 --2- 172.21.15.143:0/3035852892 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55edba542ad0 0x55edba585990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:27:04.017 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.992+0000 7fa0b5762700 1 --2- 172.21.15.143:0/3035852892 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55edba585ed0 0x55edba587fb0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fa0a8060550 tx=0x7fa0a8058cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=759bcdef396cd69d server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:04.017 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.992+0000 7fa08f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2023-11-08T19:27:04.017 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.992+0000 7fa08f7fe700 1 -- 172.21.15.143:0/3035852892 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0a80607f0 con 0x55edba585ed0 2023-11-08T19:27:04.017 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.992+0000 7fa08f7fe700 1 -- 172.21.15.143:0/3035852892 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(62..62 src has 1..62) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa0a8060a80 con 0x55edba585ed0 2023-11-08T19:27:04.017 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.992+0000 7fa0c33beb80 1 -- 172.21.15.143:0/3035852892 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55edba53a880 con 0x55edba585ed0 2023-11-08T19:27:04.018 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.992+0000 7fa08f7fe700 1 -- 172.21.15.143:0/3035852892 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa0a8054570 con 0x55edba585ed0 2023-11-08T19:27:04.018 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.992+0000 7fa0c33beb80 10 client.6156 fetch_fsmap learned FSMap version 33 2023-11-08T19:27:04.018 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.992+0000 7fa0c33beb80 1 -- 172.21.15.143:0/3035852892 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) v3 -- 0x55edba2c4d90 con 0x55edba585ed0 2023-11-08T19:27:04.018 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.992+0000 7fa08f7fe700 1 -- 172.21.15.143:0/3035852892 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55edba53a880 con 0x55edba585ed0 2023-11-08T19:27:04.018 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.992+0000 7fa08f7fe700 1 -- 172.21.15.143:0/3035852892 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 33) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fa0a80546d0 con 0x55edba585ed0 2023-11-08T19:27:04.019 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.992+0000 7fa0c33beb80 10 client.6156 fetch_fsmap finished waiting for FSMap version 33 2023-11-08T19:27:04.019 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.992+0000 7fa0c33beb80 10 client.6156 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-08T19:27:04.019 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.992+0000 7fa0c33beb80 20 client.6156 populate_metadata read hostname 'smithi143' 2023-11-08T19:27:04.019 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.992+0000 7fa0c33beb80 1 --2- 172.21.15.143:0/3035852892 >> [v2:172.21.15.143:6833/2613282947,v1:172.21.15.143:6835/2613282947] conn(0x55edba5a0bb0 0x55edba5aa380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:04.019 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.992+0000 7fa0c33beb80 4 client.6156 mds_command: new command op to 6012 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-08T19:27:04.020 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.992+0000 7fa0c33beb80 1 -- 172.21.15.143:0/3035852892 --> [v2:172.21.15.143:6833/2613282947,v1:172.21.15.143:6835/2613282947] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55edba516a30 con 0x55edba5a0bb0 2023-11-08T19:27:04.020 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.992+0000 7fa0b4f61700 1 --2- 172.21.15.143:0/3035852892 >> [v2:172.21.15.143:6833/2613282947,v1:172.21.15.143:6835/2613282947] conn(0x55edba5a0bb0 0x55edba5aa380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:04.020 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.992+0000 7fa08f7fe700 10 client.6156 ms_handle_connect on v2:172.21.15.143:6833/2613282947 2023-11-08T19:27:04.020 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.992+0000 7fa0b4f61700 1 --2- 172.21.15.143:0/3035852892 >> [v2:172.21.15.143:6833/2613282947,v1:172.21.15.143:6835/2613282947] conn(0x55edba5a0bb0 0x55edba5aa380 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc8f1a59bbdd8137 server_cookie=9288aa88cac7c874 in_seq=0 out_seq=0 2023-11-08T19:27:04.020 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.992+0000 7fa08f7fe700 1 -- 172.21.15.143:0/3035852892 <== mds.0 v2:172.21.15.143:6833/2613282947 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55edba516a30 con 0x55edba5a0bb0 2023-11-08T19:27:04.021 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.992+0000 7fa08f7fe700 10 client.6156 handle_command_reply: tid=0 2023-11-08T19:27:04.021 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.992+0000 7fa0c33beb80 1 client.6156 shutdown 2023-11-08T19:27:04.021 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.992+0000 7fa0c33beb80 1 -- 172.21.15.143:0/3035852892 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55edba585ed0 msgr2=0x55edba587fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:04.021 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.993+0000 7fa0c33beb80 1 --2- 172.21.15.143:0/3035852892 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55edba585ed0 0x55edba587fb0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fa0a8060550 tx=0x7fa0a8058cc0 comp rx=0 tx=0).stop 2023-11-08T19:27:04.021 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.993+0000 7fa0c33beb80 1 -- 172.21.15.143:0/3035852892 shutdown_connections 2023-11-08T19:27:04.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.993+0000 7fa0c33beb80 1 --2- 172.21.15.143:0/3035852892 >> [v2:172.21.15.143:6833/2613282947,v1:172.21.15.143:6835/2613282947] conn(0x55edba5a0bb0 0x55edba5aa380 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:04.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.993+0000 7fa0c33beb80 1 --2- 172.21.15.143:0/3035852892 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55edba585ed0 0x55edba587fb0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:04.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.993+0000 7fa0c33beb80 1 --2- 172.21.15.143:0/3035852892 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55edba5426f0 0x55edba59d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:04.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.993+0000 7fa0c33beb80 1 --2- 172.21.15.143:0/3035852892 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55edba542ad0 0x55edba585990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:04.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.993+0000 7fa0c33beb80 1 -- 172.21.15.143:0/3035852892 >> 172.21.15.143:0/3035852892 conn(0x55edba4bd660 msgr2=0x55edba543020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:04.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.993+0000 7fa08f7fe700 0 client.6156 ms_handle_reset on v2:172.21.15.143:6833/2613282947 2023-11-08T19:27:04.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.993+0000 7fa0c33beb80 1 -- 172.21.15.143:0/3035852892 shutdown_connections 2023-11-08T19:27:04.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.993+0000 7fa0c33beb80 1 -- 172.21.15.143:0/3035852892 wait complete. 2023-11-08T19:27:04.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:03.993+0000 7fa0c33beb80 20 client.6156 trim_cache size 0 max 16384 2023-11-08T19:27:04.029 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.012+0000 7fa0c33beb80 1 Processor -- start 2023-11-08T19:27:04.029 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.012+0000 7fa0c33beb80 1 -- start start 2023-11-08T19:27:04.029 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.013+0000 7fa0c33beb80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55edba5426f0 0x55edba541500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:04.029 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.013+0000 7fa0c33beb80 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55edba542ad0 0x55edba541a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:04.030 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.013+0000 7fa0c33beb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55edba585ed0 0x55edba58f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:04.030 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.013+0000 7fa0c33beb80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edba31a8d0 con 0x55edba5426f0 2023-11-08T19:27:04.030 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.013+0000 7fa0c33beb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edba588a50 con 0x55edba585ed0 2023-11-08T19:27:04.030 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.013+0000 7fa0c33beb80 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x55edba588390 con 0x55edba542ad0 2023-11-08T19:27:04.030 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.013+0000 7fa0b4f61700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55edba5426f0 0x55edba541500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:04.031 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.013+0000 7fa0b4f61700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55edba5426f0 0x55edba541500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52704/0 (socket says 172.21.15.143:52704) 2023-11-08T19:27:04.031 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.013+0000 7fa0b4f61700 1 -- 172.21.15.143:0/2710776297 learned_addr learned my addr 172.21.15.143:0/2710776297 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:04.031 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.013+0000 7fa0b5762700 1 --2- 172.21.15.143:0/2710776297 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55edba585ed0 0x55edba58f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:04.031 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.013+0000 7fa0affff700 1 --2- 172.21.15.143:0/2710776297 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55edba542ad0 0x55edba541a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:04.031 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.013+0000 7fa0b4f61700 1 -- 172.21.15.143:0/2710776297 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55edba542ad0 msgr2=0x55edba541a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:04.032 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.013+0000 7fa0b4f61700 1 --2- 172.21.15.143:0/2710776297 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55edba542ad0 0x55edba541a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:04.032 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.013+0000 7fa0b4f61700 1 -- 172.21.15.143:0/2710776297 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55edba585ed0 msgr2=0x55edba58f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:04.032 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.013+0000 7fa0b4f61700 1 --2- 172.21.15.143:0/2710776297 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55edba585ed0 0x55edba58f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:04.032 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.013+0000 7fa0b4f61700 1 -- 172.21.15.143:0/2710776297 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55edba2adc10 con 0x55edba5426f0 2023-11-08T19:27:04.032 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.013+0000 7fa0b4f61700 1 --2- 172.21.15.143:0/2710776297 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55edba5426f0 0x55edba541500 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fa0a4003df0 tx=0x7fa0a4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35cb17b579d77571 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:04.033 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.014+0000 7fa09affd700 1 -- 172.21.15.143:0/2710776297 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0a4032440 con 0x55edba5426f0 2023-11-08T19:27:04.033 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.014+0000 7fa09affd700 1 -- 172.21.15.143:0/2710776297 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0a40325a0 con 0x55edba5426f0 2023-11-08T19:27:04.033 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.014+0000 7fa09affd700 1 -- 172.21.15.143:0/2710776297 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0a4032930 con 0x55edba5426f0 2023-11-08T19:27:04.033 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.014+0000 7fa0c33beb80 1 -- 172.21.15.143:0/2710776297 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55edba5426f0 msgr2=0x55edba541500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:04.033 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.014+0000 7fa0c33beb80 1 --2- 172.21.15.143:0/2710776297 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55edba5426f0 0x55edba541500 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fa0a4003df0 tx=0x7fa0a4014000 comp rx=0 tx=0).stop 2023-11-08T19:27:04.034 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.014+0000 7fa0c33beb80 1 -- 172.21.15.143:0/2710776297 shutdown_connections 2023-11-08T19:27:04.034 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.014+0000 7fa0c33beb80 1 --2- 172.21.15.143:0/2710776297 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55edba585ed0 0x55edba58f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:04.034 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.014+0000 7fa0c33beb80 1 --2- 172.21.15.143:0/2710776297 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55edba5426f0 0x55edba541500 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:04.034 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.014+0000 7fa0c33beb80 1 --2- 172.21.15.143:0/2710776297 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55edba542ad0 0x55edba541a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:04.034 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.014+0000 7fa0c33beb80 1 -- 172.21.15.143:0/2710776297 >> 172.21.15.143:0/2710776297 conn(0x55edba4bd660 msgr2=0x55edba53e7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:04.035 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.014+0000 7fa0c33beb80 1 -- 172.21.15.143:0/2710776297 shutdown_connections 2023-11-08T19:27:04.035 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.014+0000 7fa0c33beb80 1 -- 172.21.15.143:0/2710776297 wait complete. 2023-11-08T19:27:04.035 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.015+0000 7fa0c33beb80 1 -- start start 2023-11-08T19:27:04.035 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.015+0000 7fa0c33beb80 1 Processor -- start 2023-11-08T19:27:04.035 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.016+0000 7fa0c33beb80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55edba5426f0 0x55edba5f63c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:04.036 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.016+0000 7fa0c33beb80 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55edba542ad0 0x55edba583500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:04.036 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.016+0000 7fa0c33beb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55edba585ed0 0x55edba583a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:04.036 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.016+0000 7fa0c33beb80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edba5a2280 con 0x55edba5426f0 2023-11-08T19:27:04.036 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.016+0000 7fa0c33beb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edba5f6900 con 0x55edba585ed0 2023-11-08T19:27:04.036 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.016+0000 7fa0c33beb80 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x55edba576630 con 0x55edba542ad0 2023-11-08T19:27:04.037 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.016+0000 7fa0c33beb80 10 client.0 _ll_register_callbacks cb 0x7fa0a8058f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T19:27:04.037 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.016+0000 7fa0b4f61700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55edba5426f0 0x55edba5f63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:04.037 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.016+0000 7fa0b4f61700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55edba5426f0 0x55edba5f63c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52708/0 (socket says 172.21.15.143:52708) 2023-11-08T19:27:04.037 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.016+0000 7fa0b4f61700 1 -- 172.21.15.143:0/3778953073 learned_addr learned my addr 172.21.15.143:0/3778953073 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:04.037 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.016+0000 7fa0affff700 1 --2- 172.21.15.143:0/3778953073 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55edba542ad0 0x55edba583500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:04.037 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.016+0000 7fa0b5762700 1 --2- 172.21.15.143:0/3778953073 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55edba585ed0 0x55edba583a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:04.038 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.016+0000 7fa0b4f61700 1 -- 172.21.15.143:0/3778953073 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55edba542ad0 msgr2=0x55edba583500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:04.038 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.016+0000 7fa0b4f61700 1 --2- 172.21.15.143:0/3778953073 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55edba542ad0 0x55edba583500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:04.038 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.016+0000 7fa0b4f61700 1 -- 172.21.15.143:0/3778953073 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55edba585ed0 msgr2=0x55edba583a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:04.038 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.016+0000 7fa0b4f61700 1 --2- 172.21.15.143:0/3778953073 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55edba585ed0 0x55edba583a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:04.038 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.016+0000 7fa0b4f61700 1 -- 172.21.15.143:0/3778953073 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0b01620d0 con 0x55edba5426f0 2023-11-08T19:27:04.039 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.016+0000 7fa0b4f61700 1 --2- 172.21.15.143:0/3778953073 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55edba5426f0 0x55edba5f63c0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fa0a4003ef0 tx=0x7fa0a4001170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88ea3e36718e373d server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:04.039 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.016+0000 7fa08f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2023-11-08T19:27:04.039 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.016+0000 7fa08f7fe700 1 -- 172.21.15.143:0/3778953073 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0a4032440 con 0x55edba5426f0 2023-11-08T19:27:04.039 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.016+0000 7fa08f7fe700 1 -- 172.21.15.143:0/3778953073 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(62..62 src has 1..62) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa0a401d040 con 0x55edba5426f0 2023-11-08T19:27:04.039 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.016+0000 7fa0c33beb80 1 -- 172.21.15.143:0/3778953073 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55edba59c980 con 0x55edba5426f0 2023-11-08T19:27:04.040 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.016+0000 7fa08f7fe700 1 -- 172.21.15.143:0/3778953073 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa0a4015610 con 0x55edba5426f0 2023-11-08T19:27:04.040 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.017+0000 7fa0c33beb80 10 client.6162 fetch_fsmap learned FSMap version 33 2023-11-08T19:27:04.040 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.017+0000 7fa0c33beb80 1 -- 172.21.15.143:0/3778953073 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) v3 -- 0x55edba2eb200 con 0x55edba5426f0 2023-11-08T19:27:04.040 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.017+0000 7fa08f7fe700 1 -- 172.21.15.143:0/3778953073 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55edba59c980 con 0x55edba5426f0 2023-11-08T19:27:04.040 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.017+0000 7fa08f7fe700 1 -- 172.21.15.143:0/3778953073 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 33) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fa0a4024ca0 con 0x55edba5426f0 2023-11-08T19:27:04.041 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.017+0000 7fa0c33beb80 10 client.6162 fetch_fsmap finished waiting for FSMap version 33 2023-11-08T19:27:04.041 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.017+0000 7fa0c33beb80 10 client.6162 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-08T19:27:04.041 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.017+0000 7fa0c33beb80 20 client.6162 populate_metadata read hostname 'smithi143' 2023-11-08T19:27:04.041 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.017+0000 7fa0c33beb80 1 --2- 172.21.15.143:0/3778953073 >> [v2:172.21.15.143:6833/2613282947,v1:172.21.15.143:6835/2613282947] conn(0x55edba55c880 0x55edba55cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:04.041 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.017+0000 7fa0c33beb80 4 client.6162 mds_command: new command op to 6012 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-08T19:27:04.042 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.017+0000 7fa0c33beb80 1 -- 172.21.15.143:0/3778953073 --> [v2:172.21.15.143:6833/2613282947,v1:172.21.15.143:6835/2613282947] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7fa09c001430 con 0x55edba55c880 2023-11-08T19:27:04.042 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.017+0000 7fa0affff700 1 --2- 172.21.15.143:0/3778953073 >> [v2:172.21.15.143:6833/2613282947,v1:172.21.15.143:6835/2613282947] conn(0x55edba55c880 0x55edba55cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:04.042 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.017+0000 7fa08f7fe700 10 client.6162 ms_handle_connect on v2:172.21.15.143:6833/2613282947 2023-11-08T19:27:04.042 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.017+0000 7fa0affff700 1 --2- 172.21.15.143:0/3778953073 >> [v2:172.21.15.143:6833/2613282947,v1:172.21.15.143:6835/2613282947] conn(0x55edba55c880 0x55edba55cc60 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ba34ff2cf203c4f server_cookie=51973c43f98f91f2 in_seq=0 out_seq=0 2023-11-08T19:27:04.042 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.017+0000 7fa08f7fe700 1 -- 172.21.15.143:0/3778953073 <== mds.0 v2:172.21.15.143:6833/2613282947 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x7fa09c001430 con 0x55edba55c880 2023-11-08T19:27:04.043 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.017+0000 7fa08f7fe700 10 client.6162 handle_command_reply: tid=0 2023-11-08T19:27:04.043 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.017+0000 7fa0c33beb80 1 client.6162 shutdown 2023-11-08T19:27:04.043 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.018+0000 7fa0c33beb80 1 -- 172.21.15.143:0/3778953073 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55edba5426f0 msgr2=0x55edba5f63c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:04.043 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.018+0000 7fa0c33beb80 1 --2- 172.21.15.143:0/3778953073 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55edba5426f0 0x55edba5f63c0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fa0a4003ef0 tx=0x7fa0a4001170 comp rx=0 tx=0).stop 2023-11-08T19:27:04.043 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.018+0000 7fa0c33beb80 1 -- 172.21.15.143:0/3778953073 shutdown_connections 2023-11-08T19:27:04.043 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.018+0000 7fa0c33beb80 1 --2- 172.21.15.143:0/3778953073 >> [v2:172.21.15.143:6833/2613282947,v1:172.21.15.143:6835/2613282947] conn(0x55edba55c880 0x55edba55cc60 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:04.044 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.018+0000 7fa0c33beb80 1 --2- 172.21.15.143:0/3778953073 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55edba585ed0 0x55edba583a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:04.044 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.018+0000 7fa08f7fe700 0 client.6162 ms_handle_reset on v2:172.21.15.143:6833/2613282947 2023-11-08T19:27:04.044 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.018+0000 7fa0c33beb80 1 --2- 172.21.15.143:0/3778953073 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55edba5426f0 0x55edba5f63c0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:04.044 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.018+0000 7fa0c33beb80 1 --2- 172.21.15.143:0/3778953073 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55edba542ad0 0x55edba583500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:04.044 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.018+0000 7fa0c33beb80 1 -- 172.21.15.143:0/3778953073 >> 172.21.15.143:0/3778953073 conn(0x55edba4bd660 msgr2=0x55edba541500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:04.045 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.018+0000 7fa0c33beb80 1 -- 172.21.15.143:0/3778953073 shutdown_connections 2023-11-08T19:27:04.045 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.018+0000 7fa0c33beb80 1 -- 172.21.15.143:0/3778953073 wait complete. 2023-11-08T19:27:04.045 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.018+0000 7fa0c33beb80 20 client.6162 trim_cache size 0 max 16384 2023-11-08T19:27:04.045 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2023-11-08T19:27:04.046 INFO:teuthology.orchestra.run.smithi143.stdout: "cluster_fsid": "968483af-77fc-4068-ad7c-5c2a1a4613d7", 2023-11-08T19:27:04.046 INFO:teuthology.orchestra.run.smithi143.stdout: "whoami": 0, 2023-11-08T19:27:04.046 INFO:teuthology.orchestra.run.smithi143.stdout: "id": 6012, 2023-11-08T19:27:04.047 INFO:teuthology.orchestra.run.smithi143.stdout: "want_state": "up:active", 2023-11-08T19:27:04.047 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "up:active", 2023-11-08T19:27:04.047 INFO:teuthology.orchestra.run.smithi143.stdout: "fs_name": "cephfs", 2023-11-08T19:27:04.047 INFO:teuthology.orchestra.run.smithi143.stdout: "rank_uptime": 5.8891306490000002, 2023-11-08T19:27:04.047 INFO:teuthology.orchestra.run.smithi143.stdout: "mdsmap_epoch": 33, 2023-11-08T19:27:04.048 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_epoch": 62, 2023-11-08T19:27:04.048 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_epoch_barrier": 62, 2023-11-08T19:27:04.048 INFO:teuthology.orchestra.run.smithi143.stdout: "uptime": 11.836297432 2023-11-08T19:27:04.048 INFO:teuthology.orchestra.run.smithi143.stdout:} 2023-11-08T19:27:04.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.018+0000 7fa0b71c5700 1 -- 172.21.15.143:0/982582007 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa09406af00 msgr2=0x7fa09406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:04.050 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.018+0000 7fa0b71c5700 1 --2- 172.21.15.143:0/982582007 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa09406af00 0x7fa09406d3c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fa0b0158640 tx=0x7fa0a4009040 comp rx=0 tx=0).stop 2023-11-08T19:27:04.050 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.018+0000 7fa0b71c5700 1 -- 172.21.15.143:0/982582007 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa0b00b5970 msgr2=0x7fa0b00c46e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:04.050 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.018+0000 7fa0b71c5700 1 --2- 172.21.15.143:0/982582007 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa0b00b5970 0x7fa0b00c46e0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fa0b0073130 tx=0x7fa0a80047c0 comp rx=0 tx=0).stop 2023-11-08T19:27:04.050 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.018+0000 7fa0b71c5700 1 -- 172.21.15.143:0/982582007 shutdown_connections 2023-11-08T19:27:04.051 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.018+0000 7fa0b71c5700 1 --2- 172.21.15.143:0/982582007 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa09406af00 0x7fa09406d3c0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:04.051 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.018+0000 7fa0b71c5700 1 --2- 172.21.15.143:0/982582007 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa0a0004850 0x7fa0b00c6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:04.051 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.018+0000 7fa0b71c5700 1 --2- 172.21.15.143:0/982582007 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa0b00b5970 0x7fa0b00c46e0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:04.051 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.018+0000 7fa0b71c5700 1 --2- 172.21.15.143:0/982582007 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa0b00b4fb0 0x7fa0b00b5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:04.051 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.018+0000 7fa0b71c5700 1 -- 172.21.15.143:0/982582007 >> 172.21.15.143:0/982582007 conn(0x7fa0b0099ab0 msgr2=0x7fa0b00ad460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:04.051 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.018+0000 7fa0b71c5700 1 -- 172.21.15.143:0/982582007 shutdown_connections 2023-11-08T19:27:04.052 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.018+0000 7fa0b71c5700 1 -- 172.21.15.143:0/982582007 wait complete. 2023-11-08T19:27:04.052 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-08T19:27:04.173 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.171+0000 7f9d594f6700 1 Processor -- start 2023-11-08T19:27:04.173 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.172+0000 7f9d594f6700 1 -- start start 2023-11-08T19:27:04.176 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.172+0000 7f9d594f6700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f9d5415dc80 0x7f9d540d1f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:04.176 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.172+0000 7f9d594f6700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9d540d3690 0x7f9d540b9690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:04.176 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.172+0000 7f9d594f6700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f9d540ce980 0x7f9d540cee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:04.177 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.172+0000 7f9d594f6700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d540cf370 con 0x7f9d540ce980 2023-11-08T19:27:04.177 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.172+0000 7f9d594f6700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d540b9bd0 con 0x7f9d5415dc80 2023-11-08T19:27:04.177 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.172+0000 7f9d594f6700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d540d3200 con 0x7f9d540d3690 2023-11-08T19:27:04.177 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.172+0000 7f9d537fe700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f9d540ce980 0x7f9d540cee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:04.177 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.173+0000 7f9d537fe700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f9d540ce980 0x7f9d540cee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52718/0 (socket says 172.21.15.143:52718) 2023-11-08T19:27:04.178 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.173+0000 7f9d537fe700 1 -- 172.21.15.143:0/3168446459 learned_addr learned my addr 172.21.15.143:0/3168446459 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:04.178 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.173+0000 7f9d527fc700 1 --2- 172.21.15.143:0/3168446459 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9d540d3690 0x7f9d540b9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:04.178 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.173+0000 7f9d52ffd700 1 --2- 172.21.15.143:0/3168446459 >> v2:172.21.15.174:3300/0 conn(0x7f9d5415dc80 0x7f9d540d1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:04.178 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.173+0000 7f9d537fe700 1 -- 172.21.15.143:0/3168446459 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9d540d3690 msgr2=0x7f9d540b9690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:04.178 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.173+0000 7f9d537fe700 1 --2- 172.21.15.143:0/3168446459 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9d540d3690 0x7f9d540b9690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:04.179 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.173+0000 7f9d537fe700 1 -- 172.21.15.143:0/3168446459 >> v2:172.21.15.174:3300/0 conn(0x7f9d5415dc80 msgr2=0x7f9d540d1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:04.179 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.173+0000 7f9d537fe700 1 --2- 172.21.15.143:0/3168446459 >> v2:172.21.15.174:3300/0 conn(0x7f9d5415dc80 0x7f9d540d1f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:04.179 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.173+0000 7f9d537fe700 1 -- 172.21.15.143:0/3168446459 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d5415e060 con 0x7f9d540ce980 2023-11-08T19:27:04.179 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.173+0000 7f9d537fe700 1 --2- 172.21.15.143:0/3168446459 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9d540ce980 0x7f9d540cee30 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f9d540b6710 tx=0x7f9d4c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=417fa4cfc03d1c43 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:04.179 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.173+0000 7f9d51ffb700 1 -- 172.21.15.143:0/3168446459 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d4c00e070 con 0x7f9d540ce980 2023-11-08T19:27:04.180 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.173+0000 7f9d51ffb700 1 -- 172.21.15.143:0/3168446459 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d4c00b9c0 con 0x7f9d540ce980 2023-11-08T19:27:04.180 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.173+0000 7f9d51ffb700 1 -- 172.21.15.143:0/3168446459 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d4c00bd70 con 0x7f9d540ce980 2023-11-08T19:27:04.180 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.174+0000 7f9d594f6700 1 -- 172.21.15.143:0/3168446459 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9d540ce980 msgr2=0x7f9d540cee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:04.180 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.174+0000 7f9d594f6700 1 --2- 172.21.15.143:0/3168446459 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9d540ce980 0x7f9d540cee30 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f9d540b6710 tx=0x7f9d4c00b330 comp rx=0 tx=0).stop 2023-11-08T19:27:04.180 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.174+0000 7f9d594f6700 1 -- 172.21.15.143:0/3168446459 shutdown_connections 2023-11-08T19:27:04.180 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.174+0000 7f9d594f6700 1 --2- 172.21.15.143:0/3168446459 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9d540ce980 0x7f9d540cee30 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:04.181 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.174+0000 7f9d594f6700 1 --2- 172.21.15.143:0/3168446459 >> v2:172.21.15.174:3300/0 conn(0x7f9d5415dc80 0x7f9d540d1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:04.181 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.174+0000 7f9d594f6700 1 --2- 172.21.15.143:0/3168446459 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9d540d3690 0x7f9d540b9690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:04.181 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.174+0000 7f9d594f6700 1 -- 172.21.15.143:0/3168446459 >> 172.21.15.143:0/3168446459 conn(0x7f9d540668e0 msgr2=0x7f9d54066cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:04.181 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.174+0000 7f9d594f6700 1 -- 172.21.15.143:0/3168446459 shutdown_connections 2023-11-08T19:27:04.181 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.174+0000 7f9d594f6700 1 -- 172.21.15.143:0/3168446459 wait complete. 2023-11-08T19:27:04.184 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.175+0000 7f9d594f6700 1 Processor -- start 2023-11-08T19:27:04.184 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.175+0000 7f9d594f6700 1 -- start start 2023-11-08T19:27:04.185 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.176+0000 7f9d594f6700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9d540ce980 0x7f9d5414f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:04.185 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.176+0000 7f9d594f6700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9d540d3690 0x7f9d540c9710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:04.185 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.176+0000 7f9d594f6700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9d5415dc80 0x7f9d540c9c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:04.185 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.176+0000 7f9d594f6700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d5414f860 con 0x7f9d540ce980 2023-11-08T19:27:04.185 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.176+0000 7f9d594f6700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d540c8670 con 0x7f9d5415dc80 2023-11-08T19:27:04.186 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.176+0000 7f9d594f6700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d540c8950 con 0x7f9d540d3690 2023-11-08T19:27:04.186 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.176+0000 7f9d52ffd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9d540ce980 0x7f9d5414f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:04.186 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.176+0000 7f9d52ffd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9d540ce980 0x7f9d5414f320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52722/0 (socket says 172.21.15.143:52722) 2023-11-08T19:27:04.186 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.176+0000 7f9d52ffd700 1 -- 172.21.15.143:0/474958708 learned_addr learned my addr 172.21.15.143:0/474958708 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:04.186 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.177+0000 7f9d527fc700 1 --2- 172.21.15.143:0/474958708 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9d540d3690 0x7f9d540c9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:04.187 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.177+0000 7f9d537fe700 1 --2- 172.21.15.143:0/474958708 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9d5415dc80 0x7f9d540c9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:04.187 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.177+0000 7f9d52ffd700 1 -- 172.21.15.143:0/474958708 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9d540d3690 msgr2=0x7f9d540c9710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:04.187 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.177+0000 7f9d52ffd700 1 --2- 172.21.15.143:0/474958708 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9d540d3690 0x7f9d540c9710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:04.187 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.177+0000 7f9d52ffd700 1 -- 172.21.15.143:0/474958708 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9d5415dc80 msgr2=0x7f9d540c9c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:04.187 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.177+0000 7f9d52ffd700 1 --2- 172.21.15.143:0/474958708 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9d5415dc80 0x7f9d540c9c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:04.188 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.177+0000 7f9d52ffd700 1 -- 172.21.15.143:0/474958708 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d4c009b30 con 0x7f9d540ce980 2023-11-08T19:27:04.188 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.177+0000 7f9d52ffd700 1 --2- 172.21.15.143:0/474958708 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9d540ce980 0x7f9d5414f320 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c00c7f0 tx=0x7f9d3c00cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:04.188 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.177+0000 7f9d4b7fe700 1 -- 172.21.15.143:0/474958708 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d3c011800 con 0x7f9d540ce980 2023-11-08T19:27:04.188 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.177+0000 7f9d4b7fe700 1 -- 172.21.15.143:0/474958708 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d3c011960 con 0x7f9d540ce980 2023-11-08T19:27:04.188 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.177+0000 7f9d594f6700 1 -- 172.21.15.143:0/474958708 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d540c8c30 con 0x7f9d540ce980 2023-11-08T19:27:04.189 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.177+0000 7f9d4b7fe700 1 -- 172.21.15.143:0/474958708 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d3c011cd0 con 0x7f9d540ce980 2023-11-08T19:27:04.189 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.177+0000 7f9d594f6700 1 -- 172.21.15.143:0/474958708 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d540c91f0 con 0x7f9d540ce980 2023-11-08T19:27:04.189 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.178+0000 7f9d594f6700 1 -- 172.21.15.143:0/474958708 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d540b8c20 con 0x7f9d540ce980 2023-11-08T19:27:04.189 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.179+0000 7f9d4b7fe700 1 -- 172.21.15.143:0/474958708 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9d3c004020 con 0x7f9d540ce980 2023-11-08T19:27:04.189 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.179+0000 7f9d4b7fe700 1 --2- 172.21.15.143:0/474958708 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f9d3406af00 0x7f9d3406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:04.190 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.179+0000 7f9d4b7fe700 1 -- 172.21.15.143:0/474958708 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9d3c0891a0 con 0x7f9d540ce980 2023-11-08T19:27:04.190 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.179+0000 7f9d527fc700 1 --2- 172.21.15.143:0/474958708 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f9d3406af00 0x7f9d3406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:04.190 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.180+0000 7f9d527fc700 1 --2- 172.21.15.143:0/474958708 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f9d3406af00 0x7f9d3406d3c0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f9d540b9950 tx=0x7f9d44008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:04.191 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.183+0000 7f9d4b7fe700 1 -- 172.21.15.143:0/474958708 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9d3c0540f0 con 0x7f9d540ce980 2023-11-08T19:27:04.419 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.418+0000 7f9d594f6700 1 -- 172.21.15.143:0/474958708 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f9d5404eee0 con 0x7f9d540ce980 2023-11-08T19:27:04.419 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.418+0000 7f9d4b7fe700 1 -- 172.21.15.143:0/474958708 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v33) v1 ==== 53+0+73 (secure 0 0 0) 0x7f9d3c057710 con 0x7f9d540ce980 2023-11-08T19:27:04.420 INFO:teuthology.orchestra.run.smithi143.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-08T19:27:04.422 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.420+0000 7f9d594f6700 1 -- 172.21.15.143:0/474958708 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f9d3406af00 msgr2=0x7f9d3406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:04.422 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.420+0000 7f9d594f6700 1 --2- 172.21.15.143:0/474958708 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f9d3406af00 0x7f9d3406d3c0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f9d540b9950 tx=0x7f9d44008040 comp rx=0 tx=0).stop 2023-11-08T19:27:04.422 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.420+0000 7f9d594f6700 1 -- 172.21.15.143:0/474958708 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9d540ce980 msgr2=0x7f9d5414f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:04.422 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.420+0000 7f9d594f6700 1 --2- 172.21.15.143:0/474958708 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9d540ce980 0x7f9d5414f320 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c00c7f0 tx=0x7f9d3c00cb00 comp rx=0 tx=0).stop 2023-11-08T19:27:04.423 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.420+0000 7f9d594f6700 1 -- 172.21.15.143:0/474958708 shutdown_connections 2023-11-08T19:27:04.423 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.420+0000 7f9d594f6700 1 --2- 172.21.15.143:0/474958708 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f9d3406af00 0x7f9d3406d3c0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:04.423 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.420+0000 7f9d594f6700 1 --2- 172.21.15.143:0/474958708 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9d5415dc80 0x7f9d540c9c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:04.423 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.420+0000 7f9d594f6700 1 --2- 172.21.15.143:0/474958708 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9d540ce980 0x7f9d5414f320 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:04.423 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.420+0000 7f9d594f6700 1 --2- 172.21.15.143:0/474958708 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9d540d3690 0x7f9d540c9710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:04.423 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.420+0000 7f9d594f6700 1 -- 172.21.15.143:0/474958708 >> 172.21.15.143:0/474958708 conn(0x7f9d540668e0 msgr2=0x7f9d5415c020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:04.424 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.420+0000 7f9d594f6700 1 -- 172.21.15.143:0/474958708 shutdown_connections 2023-11-08T19:27:04.424 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:04.420+0000 7f9d594f6700 1 -- 172.21.15.143:0/474958708 wait complete. 2023-11-08T19:27:04.433 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-08T19:27:04.434 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-08T19:27:04.434 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi143.front.sepia.ceph.com 2023-11-08T19:27:04.435 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-08T19:27:04.435 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-08T19:27:04.435 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-08T19:27:04.436 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-08T19:27:04.436 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-08T19:27:04.436 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-08T19:27:04.437 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:27:04.437 DEBUG:teuthology.orchestra.run.smithi143:> ip addr 2023-11-08T19:27:04.454 INFO:teuthology.orchestra.run.smithi143.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-08T19:27:04.454 INFO:teuthology.orchestra.run.smithi143.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-08T19:27:04.455 INFO:teuthology.orchestra.run.smithi143.stdout: inet 127.0.0.1/8 scope host lo 2023-11-08T19:27:04.455 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:27:04.455 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 ::1/128 scope host 2023-11-08T19:27:04.455 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:27:04.456 INFO:teuthology.orchestra.run.smithi143.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-08T19:27:04.456 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:d9:3b:94 brd ff:ff:ff:ff:ff:ff 2023-11-08T19:27:04.456 INFO:teuthology.orchestra.run.smithi143.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-08T19:27:04.456 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:d9:3b:95 brd ff:ff:ff:ff:ff:ff 2023-11-08T19:27:04.456 INFO:teuthology.orchestra.run.smithi143.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-08T19:27:04.456 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:88:77:82 brd ff:ff:ff:ff:ff:ff 2023-11-08T19:27:04.457 INFO:teuthology.orchestra.run.smithi143.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-08T19:27:04.457 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:88:77:83 brd ff:ff:ff:ff:ff:ff 2023-11-08T19:27:04.457 INFO:teuthology.orchestra.run.smithi143.stdout: inet 172.21.15.143/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-08T19:27:04.457 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft 42243sec preferred_lft 42243sec 2023-11-08T19:27:04.458 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 fe80::e628:d259:2f76:52a6/64 scope link noprefixroute 2023-11-08T19:27:04.458 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:27:04.458 INFO:teuthology.orchestra.run.smithi143.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-08T19:27:04.458 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 46:e3:3e:66:77:ea brd ff:ff:ff:ff:ff:ff 2023-11-08T19:27:04.458 INFO:teuthology.orchestra.run.smithi143.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-08T19:27:04.459 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:27:04.459 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 fe80::749a:8aff:fe59:325c/64 scope link 2023-11-08T19:27:04.459 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:27:04.459 INFO:teuthology.orchestra.run.smithi143.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-08T19:27:04.459 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 46:e3:3e:66:77:ea brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-08T19:27:04.460 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 fe80::44e3:3eff:fe66:77ea/64 scope link 2023-11-08T19:27:04.460 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:27:04.460 INFO:teuthology.orchestra.run.smithi143.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-08T19:27:04.460 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether ba:92:23:3f:5e:c8 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-08T19:27:04.460 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 fe80::b892:23ff:fe3f:5ec8/64 scope link 2023-11-08T19:27:04.461 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:27:04.461 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:27:04.461 DEBUG:teuthology.orchestra.run.smithi143:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-08T19:27:04.519 INFO:teuthology.orchestra.run.smithi143.stdout:1 2023-11-08T19:27:04.520 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:27:04.520 DEBUG:teuthology.orchestra.run.smithi143:> route 2023-11-08T19:27:04.579 INFO:teuthology.orchestra.run.smithi143.stdout:Kernel IP routing table 2023-11-08T19:27:04.579 INFO:teuthology.orchestra.run.smithi143.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-08T19:27:04.580 INFO:teuthology.orchestra.run.smithi143.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-08T19:27:04.580 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-08T19:27:04.580 INFO:teuthology.orchestra.run.smithi143.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-08T19:27:04.580 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:27:04.581 DEBUG:teuthology.orchestra.run.smithi143:> (cd / && exec bash -c ' 2023-11-08T19:27:04.581 DEBUG:teuthology.orchestra.run.smithi143:> set -e 2023-11-08T19:27:04.582 DEBUG:teuthology.orchestra.run.smithi143:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-08T19:27:04.582 DEBUG:teuthology.orchestra.run.smithi143:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-08T19:27:04.583 DEBUG:teuthology.orchestra.run.smithi143:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-08T19:27:04.583 DEBUG:teuthology.orchestra.run.smithi143:> ') 2023-11-08T19:27:04.683 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:27:04.684 DEBUG:teuthology.orchestra.run.smithi143:> ip netns list 2023-11-08T19:27:04.700 INFO:teuthology.orchestra.run.smithi143.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-08T19:27:04.701 INFO:teuthology.orchestra.run.smithi143.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-08T19:27:04.701 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-08T19:27:04.701 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:27:04.702 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip link set brx.0 up 2023-11-08T19:27:04.767 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-08T19:27:04.768 DEBUG:teuthology.orchestra.run.smithi143:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-08T19:27:04.824 INFO:teuthology.orchestra.run.smithi143.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-08T19:27:04.824 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-08T19:27:04.824 DEBUG:teuthology.orchestra.run.smithi143:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-08T19:27:04.881 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-11-08T19:27:04.881 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:27:04.882 DEBUG:teuthology.orchestra.run.smithi143:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-08T19:27:04.966 INFO:teuthology.orchestra.run.smithi143.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-08T19:27:04.967 INFO:teuthology.orchestra.run.smithi143.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-11-08T19:27:04.967 INFO:teuthology.orchestra.run.smithi143.stdout:invalid new device string format 2023-11-08T19:27:04.975 INFO:teuthology.orchestra.run.smithi143.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-08T19:27:04.975 INFO:teuthology.orchestra.run.smithi143.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-11-08T19:27:04.975 INFO:teuthology.orchestra.run.smithi143.stdout:invalid new device string format 2023-11-08T19:27:04.976 INFO:teuthology.orchestra.run.smithi143.stdout:mount.ceph: resolved to: "172.21.15.143:3300,172.21.15.174:3300,172.21.15.174:3301" 2023-11-08T19:27:04.976 INFO:teuthology.orchestra.run.smithi143.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-08T19:27:05.049 INFO:teuthology.orchestra.run.smithi143.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-08T19:27:05.050 INFO:teuthology.orchestra.run.smithi143.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-11-08T19:27:05.050 INFO:teuthology.orchestra.run.smithi143.stdout:invalid new device string format 2023-11-08T19:27:05.050 INFO:teuthology.orchestra.run.smithi143.stdout:mount.ceph: resolved to: "172.21.15.143:3300,172.21.15.174:3300,172.21.15.174:3301" 2023-11-08T19:27:05.050 INFO:teuthology.orchestra.run.smithi143.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-08T19:27:05.050 INFO:teuthology.orchestra.run.smithi143.stdout:mount.ceph: resolved to: "172.21.15.143:6789,172.21.15.174:6789,172.21.15.174:6790" 2023-11-08T19:27:05.051 INFO:teuthology.orchestra.run.smithi143.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.143:6789,172.21.15.174:6789,172.21.15.174:6790:/ 2023-11-08T19:27:05.051 INFO:teuthology.orchestra.run.smithi143.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=968483af-77fc-4068-ad7c-5c2a1a4613d7" will pass to kernel 2023-11-08T19:27:05.052 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-08T19:27:05.052 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:27:05.053 DEBUG:teuthology.orchestra.run.smithi143:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-08T19:27:05.080 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:27:05.080 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-08T19:27:05.163 INFO:teuthology.orchestra.run.smithi143.stdout:client5324 2023-11-08T19:27:05.163 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-08T19:27:05.164 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:27:05.221 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create test_data_pool 2023-11-08T19:27:05.384 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:05.382+0000 7f06f7c17700 1 Processor -- start 2023-11-08T19:27:05.384 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:05.383+0000 7f06f7c17700 1 -- start start 2023-11-08T19:27:05.390 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:05.383+0000 7f06f7c17700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f06f00b22e0 0x7f06f00b2750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:05.390 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:05.383+0000 7f06f7c17700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06f0161890 con 0x7f06f015dba0 2023-11-08T19:27:05.390 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:05.383+0000 7f06f7c17700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06f006b740 con 0x7f06f006b240 2023-11-08T19:27:05.390 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:05.383+0000 7f06f7c17700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06f006cd20 con 0x7f06f00b22e0 2023-11-08T19:27:05.391 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:05.384+0000 7f06f59b3700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f06f015dba0 0x7f06f006c610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:48888/0 (socket says 172.21.15.143:48888) 2023-11-08T19:27:05.391 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:05.384+0000 7f06f59b3700 1 -- 172.21.15.143:0/1369771232 learned_addr learned my addr 172.21.15.143:0/1369771232 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:27:05.391 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:05.384+0000 7f06f61b4700 1 --2- 172.21.15.143:0/1369771232 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f06f00b22e0 0x7f06f00b2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:05.391 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:05.384+0000 7f06f49b1700 1 -- 172.21.15.143:0/1369771232 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3891618366 0 0) 0x7f06f0161890 con 0x7f06f015dba0 2023-11-08T19:27:05.391 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:05.384+0000 7f06f49b1700 1 -- 172.21.15.143:0/1369771232 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f06dc0035b0 con 0x7f06f015dba0 2023-11-08T19:27:05.392 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:05.384+0000 7f06f49b1700 1 -- 172.21.15.143:0/1369771232 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 348830691 0 0) 0x7f06dc0035b0 con 0x7f06f015dba0 2023-11-08T19:27:05.392 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:05.384+0000 7f06f49b1700 1 -- 172.21.15.143:0/1369771232 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f06f00b22e0 msgr2=0x7f06f00b2750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:05.392 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:05.384+0000 7f06f49b1700 1 --2- 172.21.15.143:0/1369771232 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f06f00b22e0 0x7f06f00b2750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:05.392 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:05.384+0000 7f06f49b1700 1 -- 172.21.15.143:0/1369771232 >> v1:172.21.15.174:6789/0 conn(0x7f06f006b240 legacy=0x7f06f00caff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:05.392 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:05.384+0000 7f06f49b1700 1 -- 172.21.15.143:0/1369771232 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06f006ba20 con 0x7f06f015dba0 2023-11-08T19:27:05.393 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:05.384+0000 7f06f49b1700 1 -- 172.21.15.143:0/1369771232 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4147756303 0 0) 0x7f06e0001670 con 0x7f06f015dba0 2023-11-08T19:27:05.393 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:05.384+0000 7f06f49b1700 1 -- 172.21.15.143:0/1369771232 >> v1:172.21.15.143:6789/0 conn(0x7f06f015dba0 legacy=0x7f06f006c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:05.393 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:05.384+0000 7f06f49b1700 1 --2- 172.21.15.143:0/1369771232 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f06f00b22e0 0x7f06dc003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:05.393 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:05.384+0000 7f06f49b1700 1 --2- 172.21.15.143:0/1369771232 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f06dc003fb0 0x7f06dc004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:05.393 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:05.384+0000 7f06f49b1700 1 --2- 172.21.15.143:0/1369771232 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f06dc004d90 0x7f06dc005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:05.393 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:05.384+0000 7f06f49b1700 1 -- 172.21.15.143:0/1369771232 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06f006cd20 con 0x7f06dc004d90 2023-11-08T19:27:05.394 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:05.384+0000 7f06f49b1700 1 -- 172.21.15.143:0/1369771232 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06dc00baf0 con 0x7f06dc003fb0 2023-11-08T19:27:05.394 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:05.384+0000 7f06f49b1700 1 -- 172.21.15.143:0/1369771232 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06dc00bcb0 con 0x7f06f00b22e0 2023-11-08T19:27:05.394 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:05.384+0000 7f06f61b4700 1 --2- 172.21.15.143:0/1369771232 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f06dc004d90 0x7f06dc005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:05.395 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:05.384+0000 7f06f59b3700 1 --2- 172.21.15.143:0/1369771232 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f06f00b22e0 0x7f06dc003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:05.395 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:05.384+0000 7f06f51b2700 1 --2- 172.21.15.143:0/1369771232 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f06dc003fb0 0x7f06dc004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:05.395 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:05.385+0000 7f06f61b4700 1 -- 172.21.15.143:0/1369771232 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f06f00b22e0 msgr2=0x7f06dc003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:05.395 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:05.385+0000 7f06f61b4700 1 --2- 172.21.15.143:0/1369771232 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f06f00b22e0 0x7f06dc003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:05.396 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:05.385+0000 7f06f61b4700 1 -- 172.21.15.143:0/1369771232 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f06dc003fb0 msgr2=0x7f06dc004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:05.396 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:05.385+0000 7f06f61b4700 1 --2- 172.21.15.143:0/1369771232 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f06dc003fb0 0x7f06dc004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:05.396 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:05.385+0000 7f06f61b4700 1 -- 172.21.15.143:0/1369771232 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06dc0035b0 con 0x7f06dc004d90 2023-11-08T19:27:05.397 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:05.385+0000 7f06f61b4700 1 --2- 172.21.15.143:0/1369771232 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f06dc004d90 0x7f06dc005210 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f06e8009fc0 tx=0x7f06dc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2412961c7030a20a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:05.397 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:05.385+0000 7f06f49b1700 1 -- 172.21.15.143:0/1369771232 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06e800ec30 con 0x7f06dc004d90 2023-11-08T19:27:05.397 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:05.385+0000 7f06f49b1700 1 -- 172.21.15.143:0/1369771232 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06e800f040 con 0x7f06dc004d90 2023-11-08T19:27:05.398 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:05.385+0000 7f06f49b1700 1 -- 172.21.15.143:0/1369771232 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06e8014480 con 0x7f06dc004d90 2023-11-08T19:27:05.398 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:05.385+0000 7f06f7c17700 1 -- 172.21.15.143:0/1369771232 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f06dc004d90 msgr2=0x7f06dc005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:05.398 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:05.385+0000 7f06f7c17700 1 --2- 172.21.15.143:0/1369771232 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f06dc004d90 0x7f06dc005210 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f06e8009fc0 tx=0x7f06dc005880 comp rx=0 tx=0).stop 2023-11-08T19:27:05.399 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:05.385+0000 7f06f61b4700 1 -- 172.21.15.143:0/1369771232 reap_dead start 2023-11-08T19:27:05.399 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:05.385+0000 7f06f7c17700 1 -- 172.21.15.143:0/1369771232 shutdown_connections 2023-11-08T19:27:05.399 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:05.385+0000 7f06f7c17700 1 -- 172.21.15.143:0/1369771232 >> 172.21.15.143:0/1369771232 conn(0x7f06f00cd2e0 msgr2=0x7f06f00cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:05.399 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:05.386+0000 7f06f7c17700 1 -- 172.21.15.143:0/1369771232 shutdown_connections 2023-11-08T19:27:05.399 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:05.386+0000 7f06f7c17700 1 -- 172.21.15.143:0/1369771232 wait complete. 2023-11-08T19:27:05.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:05.387+0000 7f06f7c17700 1 Processor -- start 2023-11-08T19:27:05.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:05.387+0000 7f06f7c17700 1 -- start start 2023-11-08T19:27:05.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:05.387+0000 7f06f7c17700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f06dc004d90 0x7f06f00c9e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:05.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:05.387+0000 7f06f7c17700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f06f00c5bf0 0x7f06f00c4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:05.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:05.387+0000 7f06f7c17700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f06f00c6a80 0x7f06f00c7930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:05.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:05.388+0000 7f06f7c17700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06f00c57d0 con 0x7f06dc004d90 2023-11-08T19:27:05.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:05.388+0000 7f06f7c17700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06f00c48d0 con 0x7f06f00c5bf0 2023-11-08T19:27:05.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:05.388+0000 7f06f7c17700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06f00c4b50 con 0x7f06f00c6a80 2023-11-08T19:27:05.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:05.388+0000 7f06f59b3700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f06dc004d90 0x7f06f00c9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:05.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:05.388+0000 7f06f59b3700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f06dc004d90 0x7f06f00c9e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52746/0 (socket says 172.21.15.143:52746) 2023-11-08T19:27:05.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:05.388+0000 7f06f59b3700 1 -- 172.21.15.143:0/3339800232 learned_addr learned my addr 172.21.15.143:0/3339800232 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:05.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:05.388+0000 7f06f59b3700 1 -- 172.21.15.143:0/3339800232 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f06f00c6a80 msgr2=0x7f06f00c7930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:05.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:05.388+0000 7f06f59b3700 1 --2- 172.21.15.143:0/3339800232 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f06f00c6a80 0x7f06f00c7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:05.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:05.388+0000 7f06f59b3700 1 -- 172.21.15.143:0/3339800232 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f06f00c5bf0 msgr2=0x7f06f00c4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:05.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:05.388+0000 7f06f59b3700 1 --2- 172.21.15.143:0/3339800232 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f06f00c5bf0 0x7f06f00c4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:05.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:05.388+0000 7f06f59b3700 1 -- 172.21.15.143:0/3339800232 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06f00c4e30 con 0x7f06dc004d90 2023-11-08T19:27:05.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:05.388+0000 7f06f59b3700 1 --2- 172.21.15.143:0/3339800232 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f06dc004d90 0x7f06f00c9e70 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f06e00017a0 tx=0x7f06e0007600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:05.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:05.388+0000 7f06e67fc700 1 -- 172.21.15.143:0/3339800232 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06e0001540 con 0x7f06dc004d90 2023-11-08T19:27:05.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:05.388+0000 7f06e67fc700 1 -- 172.21.15.143:0/3339800232 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06e000f7d0 con 0x7f06dc004d90 2023-11-08T19:27:05.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:05.388+0000 7f06e67fc700 1 -- 172.21.15.143:0/3339800232 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06e000fb80 con 0x7f06dc004d90 2023-11-08T19:27:05.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:05.388+0000 7f06f7c17700 1 -- 172.21.15.143:0/3339800232 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06f00c5110 con 0x7f06dc004d90 2023-11-08T19:27:05.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:05.388+0000 7f06f7c17700 1 -- 172.21.15.143:0/3339800232 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06f00c6ee0 con 0x7f06dc004d90 2023-11-08T19:27:05.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:05.389+0000 7f06f7c17700 1 -- 172.21.15.143:0/3339800232 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06f00cf640 con 0x7f06dc004d90 2023-11-08T19:27:05.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:05.390+0000 7f06e67fc700 1 -- 172.21.15.143:0/3339800232 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f06e000fce0 con 0x7f06dc004d90 2023-11-08T19:27:05.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:05.390+0000 7f06e67fc700 1 --2- 172.21.15.143:0/3339800232 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f06d406af00 0x7f06d406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:05.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:05.390+0000 7f06e67fc700 1 -- 172.21.15.143:0/3339800232 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f06e0089700 con 0x7f06dc004d90 2023-11-08T19:27:05.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:05.390+0000 7f06f51b2700 1 --2- 172.21.15.143:0/3339800232 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f06d406af00 0x7f06d406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:05.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:05.391+0000 7f06f51b2700 1 --2- 172.21.15.143:0/3339800232 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f06d406af00 0x7f06d406d3c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f06f00ced60 tx=0x7f06ec009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:05.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:05.394+0000 7f06e67fc700 1 -- 172.21.15.143:0/3339800232 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f06e00546e0 con 0x7f06dc004d90 2023-11-08T19:27:05.583 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:05.582+0000 7f06f7c17700 1 -- 172.21.15.143:0/3339800232 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "test_data_pool"} v 0) v1 -- 0x7f06f004eee0 con 0x7f06dc004d90 2023-11-08T19:27:06.167 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:06.166+0000 7f06e67fc700 1 -- 172.21.15.143:0/3339800232 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "test_data_pool"}]=0 pool 'test_data_pool' created v63) v1 ==== 118+0+0 (secure 0 0 0) 0x7f06e0057d00 con 0x7f06dc004d90 2023-11-08T19:27:06.170 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:06.169+0000 7f06f7c17700 1 -- 172.21.15.143:0/3339800232 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f06d406af00 msgr2=0x7f06d406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:06.171 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:06.169+0000 7f06f7c17700 1 --2- 172.21.15.143:0/3339800232 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f06d406af00 0x7f06d406d3c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f06f00ced60 tx=0x7f06ec009040 comp rx=0 tx=0).stop 2023-11-08T19:27:06.171 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:06.169+0000 7f06f7c17700 1 -- 172.21.15.143:0/3339800232 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f06dc004d90 msgr2=0x7f06f00c9e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:06.171 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:06.169+0000 7f06f7c17700 1 --2- 172.21.15.143:0/3339800232 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f06dc004d90 0x7f06f00c9e70 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f06e00017a0 tx=0x7f06e0007600 comp rx=0 tx=0).stop 2023-11-08T19:27:06.171 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:06.169+0000 7f06f7c17700 1 -- 172.21.15.143:0/3339800232 shutdown_connections 2023-11-08T19:27:06.171 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:06.169+0000 7f06f7c17700 1 --2- 172.21.15.143:0/3339800232 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f06d406af00 0x7f06d406d3c0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:06.172 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:06.169+0000 7f06f7c17700 1 --2- 172.21.15.143:0/3339800232 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f06f00c6a80 0x7f06f00c7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:06.172 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:06.169+0000 7f06f7c17700 1 --2- 172.21.15.143:0/3339800232 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f06dc004d90 0x7f06f00c9e70 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:06.172 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:06.169+0000 7f06f7c17700 1 --2- 172.21.15.143:0/3339800232 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f06f00c5bf0 0x7f06f00c4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:06.172 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:06.169+0000 7f06f7c17700 1 -- 172.21.15.143:0/3339800232 >> 172.21.15.143:0/3339800232 conn(0x7f06f00cd2e0 msgr2=0x7f06f0072030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:06.172 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:06.169+0000 7f06f7c17700 1 -- 172.21.15.143:0/3339800232 shutdown_connections 2023-11-08T19:27:06.173 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:06.169+0000 7f06f7c17700 1 -- 172.21.15.143:0/3339800232 wait complete. 2023-11-08T19:27:06.173 INFO:teuthology.orchestra.run.smithi143.stderr:pool 'test_data_pool' created 2023-11-08T19:27:06.189 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create test_metadata_pool 2023-11-08T19:27:06.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:06.315+0000 7f3a79af7700 1 Processor -- start 2023-11-08T19:27:06.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:06.315+0000 7f3a79af7700 1 -- start start 2023-11-08T19:27:06.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:06.316+0000 7f3a79af7700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f3a7415dd40 0x7f3a740b21c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:06.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:06.316+0000 7f3a79af7700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3a740b4880 0x7f3a7406b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:06.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:06.316+0000 7f3a79af7700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a74162690 con 0x7f3a7406ac10 2023-11-08T19:27:06.321 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:06.316+0000 7f3a79af7700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a7415e120 con 0x7f3a7415dd40 2023-11-08T19:27:06.321 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:06.316+0000 7f3a79af7700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a7406b5c0 con 0x7f3a740b4880 2023-11-08T19:27:06.321 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:06.316+0000 7f3a737fe700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f3a7415dd40 0x7f3a740b21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:06.321 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:06.316+0000 7f3a737fe700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f3a7415dd40 0x7f3a740b21c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52754/0 (socket says 172.21.15.143:52754) 2023-11-08T19:27:06.322 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:06.316+0000 7f3a737fe700 1 -- 172.21.15.143:0/3789036062 learned_addr learned my addr 172.21.15.143:0/3789036062 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:06.322 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:06.316+0000 7f3a73fff700 1 --2- 172.21.15.143:0/3789036062 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3a740b4880 0x7f3a7406b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:06.322 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:06.317+0000 7f3a737fe700 1 -- 172.21.15.143:0/3789036062 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3a740b4880 msgr2=0x7f3a7406b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:06.322 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:06.317+0000 7f3a737fe700 1 --2- 172.21.15.143:0/3789036062 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3a740b4880 0x7f3a7406b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:06.323 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:06.317+0000 7f3a737fe700 1 -- 172.21.15.143:0/3789036062 >> v1:172.21.15.174:6789/0 conn(0x7f3a7406ac10 legacy=0x7f3a74071fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:06.323 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:06.317+0000 7f3a737fe700 1 -- 172.21.15.143:0/3789036062 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a740b2700 con 0x7f3a7415dd40 2023-11-08T19:27:06.323 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:06.317+0000 7f3a737fe700 1 --2- 172.21.15.143:0/3789036062 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3a7415dd40 0x7f3a740b21c0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f3a680098f0 tx=0x7f3a68009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=769e53f54ebca762 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:06.323 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:06.317+0000 7f3a727fc700 1 -- 172.21.15.143:0/3789036062 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a6800b8a0 con 0x7f3a7415dd40 2023-11-08T19:27:06.323 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:06.317+0000 7f3a727fc700 1 -- 172.21.15.143:0/3789036062 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a6800ba00 con 0x7f3a7415dd40 2023-11-08T19:27:06.324 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:06.317+0000 7f3a727fc700 1 -- 172.21.15.143:0/3789036062 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a6800bd90 con 0x7f3a7415dd40 2023-11-08T19:27:06.324 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:06.317+0000 7f3a79af7700 1 -- 172.21.15.143:0/3789036062 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3a7415dd40 msgr2=0x7f3a740b21c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:06.324 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:06.317+0000 7f3a79af7700 1 --2- 172.21.15.143:0/3789036062 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3a7415dd40 0x7f3a740b21c0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f3a680098f0 tx=0x7f3a68009c00 comp rx=0 tx=0).stop 2023-11-08T19:27:06.324 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:06.318+0000 7f3a79af7700 1 -- 172.21.15.143:0/3789036062 shutdown_connections 2023-11-08T19:27:06.324 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:06.318+0000 7f3a79af7700 1 --2- 172.21.15.143:0/3789036062 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3a740b4880 0x7f3a7406b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:06.325 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:06.318+0000 7f3a79af7700 1 --2- 172.21.15.143:0/3789036062 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3a7415dd40 0x7f3a740b21c0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:06.325 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:06.318+0000 7f3a79af7700 1 -- 172.21.15.143:0/3789036062 >> 172.21.15.143:0/3789036062 conn(0x7f3a7406dfe0 msgr2=0x7f3a7406e3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:06.325 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:06.318+0000 7f3a79af7700 1 -- 172.21.15.143:0/3789036062 shutdown_connections 2023-11-08T19:27:06.325 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:06.318+0000 7f3a79af7700 1 -- 172.21.15.143:0/3789036062 wait complete. 2023-11-08T19:27:06.325 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:06.319+0000 7f3a79af7700 1 Processor -- start 2023-11-08T19:27:06.326 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:06.319+0000 7f3a79af7700 1 -- start start 2023-11-08T19:27:06.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:06.320+0000 7f3a79af7700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3a7406ac10 0x7f3a741517a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:06.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:06.320+0000 7f3a79af7700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a740b4880 0x7f3a7415ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:06.329 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:06.320+0000 7f3a79af7700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3a7415dd40 0x7f3a7415b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:06.329 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:06.320+0000 7f3a79af7700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a74160700 con 0x7f3a7406ac10 2023-11-08T19:27:06.329 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:06.320+0000 7f3a79af7700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a7415f870 con 0x7f3a740b4880 2023-11-08T19:27:06.329 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:06.320+0000 7f3a79af7700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a7415faf0 con 0x7f3a7415dd40 2023-11-08T19:27:06.329 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:06.320+0000 7f3a737fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3a7406ac10 0x7f3a741517a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:06.330 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:06.320+0000 7f3a737fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3a7406ac10 0x7f3a741517a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52760/0 (socket says 172.21.15.143:52760) 2023-11-08T19:27:06.330 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:06.320+0000 7f3a737fe700 1 -- 172.21.15.143:0/2511039593 learned_addr learned my addr 172.21.15.143:0/2511039593 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:06.330 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:06.320+0000 7f3a72ffd700 1 --2- 172.21.15.143:0/2511039593 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a740b4880 0x7f3a7415ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:06.330 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:06.320+0000 7f3a73fff700 1 --2- 172.21.15.143:0/2511039593 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3a7415dd40 0x7f3a7415b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:06.331 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:06.320+0000 7f3a737fe700 1 -- 172.21.15.143:0/2511039593 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3a7415dd40 msgr2=0x7f3a7415b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:06.331 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:06.320+0000 7f3a737fe700 1 --2- 172.21.15.143:0/2511039593 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3a7415dd40 0x7f3a7415b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:06.331 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:06.320+0000 7f3a737fe700 1 -- 172.21.15.143:0/2511039593 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a740b4880 msgr2=0x7f3a7415ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:06.331 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:06.320+0000 7f3a737fe700 1 --2- 172.21.15.143:0/2511039593 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a740b4880 0x7f3a7415ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:06.331 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:06.320+0000 7f3a737fe700 1 -- 172.21.15.143:0/2511039593 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a68009550 con 0x7f3a7406ac10 2023-11-08T19:27:06.332 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:06.320+0000 7f3a72ffd700 1 --2- 172.21.15.143:0/2511039593 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a740b4880 0x7f3a7415ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:27:06.332 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:06.320+0000 7f3a737fe700 1 --2- 172.21.15.143:0/2511039593 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3a7406ac10 0x7f3a741517a0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f3a68014040 tx=0x7f3a68004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:06.332 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:06.320+0000 7f3a5bfff700 1 -- 172.21.15.143:0/2511039593 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a680045d0 con 0x7f3a7406ac10 2023-11-08T19:27:06.332 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:06.320+0000 7f3a5bfff700 1 -- 172.21.15.143:0/2511039593 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a6801e070 con 0x7f3a7406ac10 2023-11-08T19:27:06.332 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:06.320+0000 7f3a79af7700 1 -- 172.21.15.143:0/2511039593 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a7415fdd0 con 0x7f3a7406ac10 2023-11-08T19:27:06.332 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:06.320+0000 7f3a5bfff700 1 -- 172.21.15.143:0/2511039593 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a680038c0 con 0x7f3a7406ac10 2023-11-08T19:27:06.333 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:06.320+0000 7f3a79af7700 1 -- 172.21.15.143:0/2511039593 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a74160390 con 0x7f3a7406ac10 2023-11-08T19:27:06.333 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:06.321+0000 7f3a79af7700 1 -- 172.21.15.143:0/2511039593 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a7404eee0 con 0x7f3a7406ac10 2023-11-08T19:27:06.333 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:06.322+0000 7f3a5bfff700 1 -- 172.21.15.143:0/2511039593 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3a68003a20 con 0x7f3a7406ac10 2023-11-08T19:27:06.333 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:06.322+0000 7f3a5bfff700 1 --2- 172.21.15.143:0/2511039593 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f3a6006af00 0x7f3a6006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:06.333 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:06.322+0000 7f3a5bfff700 1 -- 172.21.15.143:0/2511039593 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6415+0+0 (secure 0 0 0) 0x7f3a68020070 con 0x7f3a7406ac10 2023-11-08T19:27:06.334 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:06.323+0000 7f3a72ffd700 1 --2- 172.21.15.143:0/2511039593 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f3a6006af00 0x7f3a6006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:06.334 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:06.323+0000 7f3a72ffd700 1 --2- 172.21.15.143:0/2511039593 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f3a6006af00 0x7f3a6006d3c0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f3a740c7650 tx=0x7f3a64009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:06.334 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:06.327+0000 7f3a5bfff700 1 -- 172.21.15.143:0/2511039593 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3a68055bb0 con 0x7f3a7406ac10 2023-11-08T19:27:06.518 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:06.517+0000 7f3a79af7700 1 -- 172.21.15.143:0/2511039593 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "test_metadata_pool"} v 0) v1 -- 0x7f3a740afce0 con 0x7f3a7406ac10 2023-11-08T19:27:07.172 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:07.171+0000 7f3a5bfff700 1 -- 172.21.15.143:0/2511039593 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "test_metadata_pool"}]=0 pool 'test_metadata_pool' created v64) v1 ==== 126+0+0 (secure 0 0 0) 0x7f3a680150a0 con 0x7f3a7406ac10 2023-11-08T19:27:07.178 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:07.176+0000 7f3a79af7700 1 -- 172.21.15.143:0/2511039593 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f3a6006af00 msgr2=0x7f3a6006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:07.178 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:07.176+0000 7f3a79af7700 1 --2- 172.21.15.143:0/2511039593 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f3a6006af00 0x7f3a6006d3c0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f3a740c7650 tx=0x7f3a64009040 comp rx=0 tx=0).stop 2023-11-08T19:27:07.178 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:07.176+0000 7f3a79af7700 1 -- 172.21.15.143:0/2511039593 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3a7406ac10 msgr2=0x7f3a741517a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:07.179 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:07.176+0000 7f3a79af7700 1 --2- 172.21.15.143:0/2511039593 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3a7406ac10 0x7f3a741517a0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f3a68014040 tx=0x7f3a68004750 comp rx=0 tx=0).stop 2023-11-08T19:27:07.179 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:07.176+0000 7f3a79af7700 1 -- 172.21.15.143:0/2511039593 shutdown_connections 2023-11-08T19:27:07.179 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:07.176+0000 7f3a79af7700 1 --2- 172.21.15.143:0/2511039593 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f3a6006af00 0x7f3a6006d3c0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:07.179 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:07.176+0000 7f3a79af7700 1 --2- 172.21.15.143:0/2511039593 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3a7415dd40 0x7f3a7415b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:07.179 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:07.176+0000 7f3a79af7700 1 --2- 172.21.15.143:0/2511039593 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3a7406ac10 0x7f3a741517a0 secure :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f3a68014040 tx=0x7f3a68004750 comp rx=0 tx=0).stop 2023-11-08T19:27:07.180 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:07.176+0000 7f3a79af7700 1 --2- 172.21.15.143:0/2511039593 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a740b4880 0x7f3a7415ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:07.180 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:07.176+0000 7f3a79af7700 1 -- 172.21.15.143:0/2511039593 >> 172.21.15.143:0/2511039593 conn(0x7f3a7406dfe0 msgr2=0x7f3a740ce570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:07.180 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:07.177+0000 7f3a79af7700 1 -- 172.21.15.143:0/2511039593 shutdown_connections 2023-11-08T19:27:07.180 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:07.177+0000 7f3a79af7700 1 -- 172.21.15.143:0/2511039593 wait complete. 2023-11-08T19:27:07.181 INFO:teuthology.orchestra.run.smithi143.stderr:pool 'test_metadata_pool' created 2023-11-08T19:27:07.202 DEBUG:teuthology.orchestra.run.smithi143:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool test_data_pool mksnap snap4 2023-11-08T19:27:07.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:07.247+0000 7f1496ae1080 1 Processor -- start 2023-11-08T19:27:07.249 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:07.247+0000 7f1496ae1080 1 -- start start 2023-11-08T19:27:07.253 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:07.248+0000 7f1496ae1080 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f147c0030b0 0x55979d2c2360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:07.253 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:07.248+0000 7f1496ae1080 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55979d2c5650 0x55979d2bdf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:07.253 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:07.248+0000 7f1496ae1080 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55979d2338e0 0x55979d2be4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:07.254 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:07.248+0000 7f1496ae1080 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x55979d19ec30 con 0x55979d2338e0 2023-11-08T19:27:07.254 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:07.248+0000 7f1496ae1080 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55979d19a520 con 0x55979d2c5650 2023-11-08T19:27:07.254 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:07.248+0000 7f1496ae1080 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x55979d134740 con 0x7f147c0030b0 2023-11-08T19:27:07.254 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:07.248+0000 7f148d377700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55979d2338e0 0x55979d2be4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:07.254 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:07.248+0000 7f148d377700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55979d2338e0 0x55979d2be4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52782/0 (socket says 172.21.15.143:52782) 2023-11-08T19:27:07.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:07.248+0000 7f148d377700 1 -- 172.21.15.143:0/1377380698 learned_addr learned my addr 172.21.15.143:0/1377380698 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:07.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:07.248+0000 7f148cb76700 1 --2- 172.21.15.143:0/1377380698 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f147c0030b0 0x55979d2c2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:07.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:07.248+0000 7f1487fff700 1 --2- 172.21.15.143:0/1377380698 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55979d2c5650 0x55979d2bdf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:07.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:07.248+0000 7f148d377700 1 -- 172.21.15.143:0/1377380698 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f147c0030b0 msgr2=0x55979d2c2360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:07.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:07.248+0000 7f148d377700 1 --2- 172.21.15.143:0/1377380698 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f147c0030b0 0x55979d2c2360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:07.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:07.248+0000 7f148d377700 1 -- 172.21.15.143:0/1377380698 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55979d2c5650 msgr2=0x55979d2bdf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:07.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:07.248+0000 7f148d377700 1 --2- 172.21.15.143:0/1377380698 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55979d2c5650 0x55979d2bdf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:07.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:07.248+0000 7f148d377700 1 -- 172.21.15.143:0/1377380698 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55979d133b70 con 0x55979d2338e0 2023-11-08T19:27:07.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:07.249+0000 7f148d377700 1 --2- 172.21.15.143:0/1377380698 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55979d2338e0 0x55979d2be4d0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x55979d21eb50 tx=0x7f1488005d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=393894887e5d86f3 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:07.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:07.249+0000 7f1486ffd700 1 -- 172.21.15.143:0/1377380698 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f148800ae40 con 0x55979d2338e0 2023-11-08T19:27:07.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:07.249+0000 7f1486ffd700 1 -- 172.21.15.143:0/1377380698 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1488013650 con 0x55979d2338e0 2023-11-08T19:27:07.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:07.249+0000 7f1486ffd700 1 -- 172.21.15.143:0/1377380698 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14880139c0 con 0x55979d2338e0 2023-11-08T19:27:07.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:07.249+0000 7f1496ae1080 1 -- 172.21.15.143:0/1377380698 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55979d2338e0 msgr2=0x55979d2be4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:07.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:07.249+0000 7f1496ae1080 1 --2- 172.21.15.143:0/1377380698 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55979d2338e0 0x55979d2be4d0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x55979d21eb50 tx=0x7f1488005d80 comp rx=0 tx=0).stop 2023-11-08T19:27:07.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:07.249+0000 7f1496ae1080 1 -- 172.21.15.143:0/1377380698 shutdown_connections 2023-11-08T19:27:07.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:07.249+0000 7f1496ae1080 1 --2- 172.21.15.143:0/1377380698 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f147c0030b0 0x55979d2c2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:07.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:07.249+0000 7f1496ae1080 1 --2- 172.21.15.143:0/1377380698 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55979d2338e0 0x55979d2be4d0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:07.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:07.249+0000 7f1496ae1080 1 --2- 172.21.15.143:0/1377380698 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55979d2c5650 0x55979d2bdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:07.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:07.249+0000 7f1496ae1080 1 -- 172.21.15.143:0/1377380698 >> 172.21.15.143:0/1377380698 conn(0x55979d23b3c0 msgr2=0x55979d1f0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:07.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:07.249+0000 7f1496ae1080 1 -- 172.21.15.143:0/1377380698 shutdown_connections 2023-11-08T19:27:07.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:07.249+0000 7f1496ae1080 1 -- 172.21.15.143:0/1377380698 wait complete. 2023-11-08T19:27:07.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:07.250+0000 7f1496ae1080 1 Processor -- start 2023-11-08T19:27:07.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:07.250+0000 7f1496ae1080 1 -- start start 2023-11-08T19:27:07.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:07.251+0000 7f1496ae1080 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55979d2338e0 0x55979d2c6e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:07.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:07.251+0000 7f1496ae1080 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55979d2c5650 0x55979d2c49b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:07.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:07.251+0000 7f1496ae1080 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f147c0030b0 0x55979d2319a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:07.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:07.251+0000 7f1496ae1080 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x55979d1343e0 con 0x55979d2338e0 2023-11-08T19:27:07.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:07.251+0000 7f1496ae1080 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55979d128680 con 0x7f147c0030b0 2023-11-08T19:27:07.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:07.251+0000 7f1496ae1080 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x55979d2b9810 con 0x55979d2c5650 2023-11-08T19:27:07.261 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:07.251+0000 7f148cb76700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55979d2338e0 0x55979d2c6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:07.261 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:07.251+0000 7f148cb76700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55979d2338e0 0x55979d2c6e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52784/0 (socket says 172.21.15.143:52784) 2023-11-08T19:27:07.261 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:07.251+0000 7f148cb76700 1 -- 172.21.15.143:0/1162769072 learned_addr learned my addr 172.21.15.143:0/1162769072 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:07.261 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:07.251+0000 7f1487fff700 1 --2- 172.21.15.143:0/1162769072 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55979d2c5650 0x55979d2c49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:07.261 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:07.251+0000 7f148d377700 1 --2- 172.21.15.143:0/1162769072 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f147c0030b0 0x55979d2319a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:07.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:07.251+0000 7f148cb76700 1 -- 172.21.15.143:0/1162769072 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55979d2c5650 msgr2=0x55979d2c49b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:07.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:07.251+0000 7f148cb76700 1 --2- 172.21.15.143:0/1162769072 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55979d2c5650 0x55979d2c49b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:07.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:07.251+0000 7f148cb76700 1 -- 172.21.15.143:0/1162769072 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f147c0030b0 msgr2=0x55979d2319a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:07.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:07.251+0000 7f148cb76700 1 --2- 172.21.15.143:0/1162769072 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f147c0030b0 0x55979d2319a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:07.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:07.251+0000 7f148cb76700 1 -- 172.21.15.143:0/1162769072 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f148800a390 con 0x55979d2338e0 2023-11-08T19:27:07.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:07.251+0000 7f148cb76700 1 --2- 172.21.15.143:0/1162769072 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55979d2338e0 0x55979d2c6e20 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f147800f680 tx=0x7f147800f990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:07.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:07.251+0000 7f1485ffb700 1 -- 172.21.15.143:0/1162769072 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1478014070 con 0x55979d2338e0 2023-11-08T19:27:07.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:07.251+0000 7f1485ffb700 1 -- 172.21.15.143:0/1162769072 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1478005040 con 0x55979d2338e0 2023-11-08T19:27:07.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:07.251+0000 7f1496ae1080 1 -- 172.21.15.143:0/1162769072 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55979d1335d0 con 0x55979d2338e0 2023-11-08T19:27:07.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:07.251+0000 7f1485ffb700 1 -- 172.21.15.143:0/1162769072 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f147800e880 con 0x55979d2338e0 2023-11-08T19:27:07.264 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:07.251+0000 7f1496ae1080 1 -- 172.21.15.143:0/1162769072 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55979d2b9dd0 con 0x55979d2338e0 2023-11-08T19:27:07.265 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:07.253+0000 7f1485ffb700 1 -- 172.21.15.143:0/1162769072 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f147800e9e0 con 0x55979d2338e0 2023-11-08T19:27:07.265 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:07.253+0000 7f1485ffb700 1 --2- 172.21.15.143:0/1162769072 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f1470069820 0x7f147006bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:07.265 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:07.253+0000 7f1485ffb700 1 -- 172.21.15.143:0/1162769072 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6772+0+0 (secure 0 0 0) 0x7f1478089240 con 0x55979d2338e0 2023-11-08T19:27:07.265 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:07.254+0000 7f1487fff700 1 --2- 172.21.15.143:0/1162769072 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f1470069820 0x7f147006bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:07.266 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:07.254+0000 7f1496ae1080 1 -- 172.21.15.143:0/1162769072 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- pool_op(create snap pool 11 tid 1 name snap4 v0) v4 -- 0x55979d1a44a0 con 0x55979d2338e0 2023-11-08T19:27:07.266 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:07.254+0000 7f1487fff700 1 --2- 172.21.15.143:0/1162769072 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f1470069820 0x7f147006bce0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f1480000c00 tx=0x7f1480009000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:08.178 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.177+0000 7f1485ffb700 1 -- 172.21.15.143:0/1162769072 <== mon.0 v2:172.21.15.143:3300/0 6 ==== pool_op_reply(tid 1 (0) Success v65) v1 ==== 43+0+0 (secure 0 0 0) 0x7f1478053eb0 con 0x55979d2338e0 2023-11-08T19:27:08.179 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.177+0000 7f1485ffb700 1 -- 172.21.15.143:0/1162769072 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=65}) v3 -- 0x7f1470073a50 con 0x55979d2338e0 2023-11-08T19:27:08.179 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.177+0000 7f1485ffb700 1 -- 172.21.15.143:0/1162769072 <== mon.0 v2:172.21.15.143:3300/0 7 ==== osd_map(65..65 src has 1..65) v4 ==== 718+0+0 (secure 0 0 0) 0x7f147800c360 con 0x55979d2338e0 2023-11-08T19:27:08.180 INFO:teuthology.orchestra.run.smithi143.stdout:created pool test_data_pool snap snap4 2023-11-08T19:27:08.180 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.177+0000 7f1496ae1080 1 -- 172.21.15.143:0/1162769072 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f1470069820 msgr2=0x7f147006bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:08.180 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.177+0000 7f1496ae1080 1 --2- 172.21.15.143:0/1162769072 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f1470069820 0x7f147006bce0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f1480000c00 tx=0x7f1480009000 comp rx=0 tx=0).stop 2023-11-08T19:27:08.181 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.177+0000 7f1496ae1080 1 -- 172.21.15.143:0/1162769072 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55979d2338e0 msgr2=0x55979d2c6e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:08.181 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.177+0000 7f1496ae1080 1 --2- 172.21.15.143:0/1162769072 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55979d2338e0 0x55979d2c6e20 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f147800f680 tx=0x7f147800f990 comp rx=0 tx=0).stop 2023-11-08T19:27:08.181 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.179+0000 7f1496ae1080 1 -- 172.21.15.143:0/1162769072 shutdown_connections 2023-11-08T19:27:08.182 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.179+0000 7f1496ae1080 1 --2- 172.21.15.143:0/1162769072 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f1470069820 0x7f147006bce0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:08.182 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.179+0000 7f1496ae1080 1 --2- 172.21.15.143:0/1162769072 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f147c0030b0 0x55979d2319a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:08.182 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.179+0000 7f1496ae1080 1 --2- 172.21.15.143:0/1162769072 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55979d2338e0 0x55979d2c6e20 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:08.182 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.179+0000 7f1496ae1080 1 --2- 172.21.15.143:0/1162769072 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55979d2c5650 0x55979d2c49b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:08.183 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.179+0000 7f1496ae1080 1 -- 172.21.15.143:0/1162769072 >> 172.21.15.143:0/1162769072 conn(0x55979d23b3c0 msgr2=0x55979d2c2360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:08.184 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.180+0000 7f1496ae1080 1 -- 172.21.15.143:0/1162769072 shutdown_connections 2023-11-08T19:27:08.184 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.181+0000 7f1496ae1080 1 -- 172.21.15.143:0/1162769072 wait complete. 2023-11-08T19:27:08.195 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new testfs test_metadata_pool test_data_pool 2023-11-08T19:27:08.325 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.323+0000 7f610a0dc700 1 Processor -- start 2023-11-08T19:27:08.326 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.323+0000 7f610a0dc700 1 -- start start 2023-11-08T19:27:08.333 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.324+0000 7f610a0dc700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f610406c0c0 0x7f61040ccfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:08.333 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.324+0000 7f610a0dc700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f610415ce60 0x7f61041522a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:08.333 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.324+0000 7f610a0dc700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61040ced30 con 0x7f610415c9b0 2023-11-08T19:27:08.333 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.324+0000 7f610a0dc700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61040cd4f0 con 0x7f610406c0c0 2023-11-08T19:27:08.334 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.324+0000 7f610a0dc700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61040cdf60 con 0x7f610415ce60 2023-11-08T19:27:08.334 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.324+0000 7f6102ffd700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f610415c9b0 0x7f6104071900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:48946/0 (socket says 172.21.15.143:48946) 2023-11-08T19:27:08.334 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.324+0000 7f6102ffd700 1 -- 172.21.15.143:0/2671495462 learned_addr learned my addr 172.21.15.143:0/2671495462 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:27:08.334 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.325+0000 7f61037fe700 1 --2- 172.21.15.143:0/2671495462 >> v2:172.21.15.174:3300/0 conn(0x7f610406c0c0 0x7f61040ccfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:08.335 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.325+0000 7f6103fff700 1 --2- 172.21.15.143:0/2671495462 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f610415ce60 0x7f61041522a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:08.335 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.325+0000 7f61027fc700 1 -- 172.21.15.143:0/2671495462 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3325293799 0 0) 0x7f61040ced30 con 0x7f610415c9b0 2023-11-08T19:27:08.335 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.325+0000 7f61027fc700 1 -- 172.21.15.143:0/2671495462 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f60f00035b0 con 0x7f610415c9b0 2023-11-08T19:27:08.335 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.325+0000 7f61027fc700 1 -- 172.21.15.143:0/2671495462 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3524595058 0 0) 0x7f60f00035b0 con 0x7f610415c9b0 2023-11-08T19:27:08.335 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.325+0000 7f61027fc700 1 -- 172.21.15.143:0/2671495462 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f610415ce60 msgr2=0x7f61041522a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:08.336 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.325+0000 7f61027fc700 1 --2- 172.21.15.143:0/2671495462 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f610415ce60 0x7f61041522a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:08.336 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.325+0000 7f61027fc700 1 -- 172.21.15.143:0/2671495462 >> v2:172.21.15.174:3300/0 conn(0x7f610406c0c0 msgr2=0x7f61040ccfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:08.336 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.325+0000 7f61027fc700 1 --2- 172.21.15.143:0/2671495462 >> v2:172.21.15.174:3300/0 conn(0x7f610406c0c0 0x7f61040ccfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:08.336 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.325+0000 7f61027fc700 1 -- 172.21.15.143:0/2671495462 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61040cea50 con 0x7f610415c9b0 2023-11-08T19:27:08.337 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.325+0000 7f61027fc700 1 -- 172.21.15.143:0/2671495462 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4147756303 0 0) 0x7f60f4001670 con 0x7f610415c9b0 2023-11-08T19:27:08.337 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.325+0000 7f61027fc700 1 -- 172.21.15.143:0/2671495462 >> v1:172.21.15.143:6789/0 conn(0x7f610415c9b0 legacy=0x7f6104071900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:08.337 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.325+0000 7f61027fc700 1 --2- 172.21.15.143:0/2671495462 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f60f0003a00 0x7f60f0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:08.337 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.325+0000 7f61027fc700 1 --2- 172.21.15.143:0/2671495462 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f610415ce60 0x7f60f0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:08.337 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.325+0000 7f61027fc700 1 --2- 172.21.15.143:0/2671495462 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f60f0004d90 0x7f60f0005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:08.338 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.325+0000 7f61027fc700 1 -- 172.21.15.143:0/2671495462 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61040cd4f0 con 0x7f60f0003a00 2023-11-08T19:27:08.338 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.325+0000 7f61027fc700 1 -- 172.21.15.143:0/2671495462 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61040cdf60 con 0x7f60f0004d90 2023-11-08T19:27:08.339 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.325+0000 7f61027fc700 1 -- 172.21.15.143:0/2671495462 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60f000ba10 con 0x7f610415ce60 2023-11-08T19:27:08.339 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.325+0000 7f61037fe700 1 --2- 172.21.15.143:0/2671495462 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f60f0003a00 0x7f60f0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:08.339 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.325+0000 7f6102ffd700 1 --2- 172.21.15.143:0/2671495462 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f610415ce60 0x7f60f0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:08.339 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.325+0000 7f6103fff700 1 --2- 172.21.15.143:0/2671495462 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f60f0004d90 0x7f60f0005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:08.339 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.325+0000 7f61037fe700 1 -- 172.21.15.143:0/2671495462 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f610415ce60 msgr2=0x7f60f0004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:08.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.325+0000 7f61037fe700 1 --2- 172.21.15.143:0/2671495462 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f610415ce60 0x7f60f0004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:08.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.325+0000 7f61037fe700 1 -- 172.21.15.143:0/2671495462 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f60f0004d90 msgr2=0x7f60f0005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:08.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.325+0000 7f61037fe700 1 --2- 172.21.15.143:0/2671495462 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f60f0004d90 0x7f60f0005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:08.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.325+0000 7f61037fe700 1 -- 172.21.15.143:0/2671495462 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60f00035b0 con 0x7f60f0003a00 2023-11-08T19:27:08.341 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.325+0000 7f61037fe700 1 --2- 172.21.15.143:0/2671495462 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f60f0003a00 0x7f60f0003e20 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f60ec009b60 tx=0x7f60f0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f74a1ef5e948a4e7 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:08.341 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.326+0000 7f61027fc700 1 -- 172.21.15.143:0/2671495462 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60ec00e6c0 con 0x7f60f0003a00 2023-11-08T19:27:08.341 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.326+0000 7f61027fc700 1 -- 172.21.15.143:0/2671495462 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60ec00e820 con 0x7f60f0003a00 2023-11-08T19:27:08.342 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.326+0000 7f61027fc700 1 -- 172.21.15.143:0/2671495462 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60ec00eb90 con 0x7f60f0003a00 2023-11-08T19:27:08.342 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.326+0000 7f610a0dc700 1 -- 172.21.15.143:0/2671495462 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f60f0003a00 msgr2=0x7f60f0003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:08.342 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.326+0000 7f610a0dc700 1 --2- 172.21.15.143:0/2671495462 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f60f0003a00 0x7f60f0003e20 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f60ec009b60 tx=0x7f60f0005880 comp rx=0 tx=0).stop 2023-11-08T19:27:08.343 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.326+0000 7f6103fff700 1 -- 172.21.15.143:0/2671495462 reap_dead start 2023-11-08T19:27:08.343 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.326+0000 7f610a0dc700 1 -- 172.21.15.143:0/2671495462 shutdown_connections 2023-11-08T19:27:08.343 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.326+0000 7f610a0dc700 1 -- 172.21.15.143:0/2671495462 >> 172.21.15.143:0/2671495462 conn(0x7f61040b1c70 msgr2=0x7f61040b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:08.343 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.326+0000 7f610a0dc700 1 -- 172.21.15.143:0/2671495462 shutdown_connections 2023-11-08T19:27:08.343 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.326+0000 7f610a0dc700 1 -- 172.21.15.143:0/2671495462 wait complete. 2023-11-08T19:27:08.345 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.327+0000 7f610a0dc700 1 Processor -- start 2023-11-08T19:27:08.345 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.328+0000 7f610a0dc700 1 -- start start 2023-11-08T19:27:08.345 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.328+0000 7f610a0dc700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f60f0003a00 0x7f61041f5800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:08.345 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.328+0000 7f610a0dc700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f61041539b0 0x7f6104071a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:08.345 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.328+0000 7f610a0dc700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f61040b4f80 0x7f610414fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:08.346 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.328+0000 7f610a0dc700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61041f5d40 con 0x7f60f0003a00 2023-11-08T19:27:08.346 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.328+0000 7f610a0dc700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61041f4db0 con 0x7f61041539b0 2023-11-08T19:27:08.346 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.328+0000 7f610a0dc700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61041f5370 con 0x7f61040b4f80 2023-11-08T19:27:08.346 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.328+0000 7f61037fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f60f0003a00 0x7f61041f5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:08.347 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.328+0000 7f61037fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f60f0003a00 0x7f61041f5800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52804/0 (socket says 172.21.15.143:52804) 2023-11-08T19:27:08.347 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.328+0000 7f61037fe700 1 -- 172.21.15.143:0/3856598253 learned_addr learned my addr 172.21.15.143:0/3856598253 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:08.347 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.329+0000 7f6103fff700 1 --2- 172.21.15.143:0/3856598253 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f61040b4f80 0x7f610414fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:08.347 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.329+0000 7f6102ffd700 1 --2- 172.21.15.143:0/3856598253 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f61041539b0 0x7f6104071a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:08.347 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.329+0000 7f61037fe700 1 -- 172.21.15.143:0/3856598253 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f61040b4f80 msgr2=0x7f610414fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:08.348 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.329+0000 7f61037fe700 1 --2- 172.21.15.143:0/3856598253 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f61040b4f80 0x7f610414fdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:08.348 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.329+0000 7f61037fe700 1 -- 172.21.15.143:0/3856598253 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f61041539b0 msgr2=0x7f6104071a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:08.348 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.329+0000 7f61037fe700 1 --2- 172.21.15.143:0/3856598253 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f61041539b0 0x7f6104071a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:08.348 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.329+0000 7f61037fe700 1 -- 172.21.15.143:0/3856598253 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61040b59b0 con 0x7f60f0003a00 2023-11-08T19:27:08.348 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.329+0000 7f61037fe700 1 --2- 172.21.15.143:0/3856598253 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f60f0003a00 0x7f61041f5800 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f60ec00e4b0 tx=0x7f60ec0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:08.349 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.329+0000 7f60fbfff700 1 -- 172.21.15.143:0/3856598253 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60ec00e6c0 con 0x7f60f0003a00 2023-11-08T19:27:08.349 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.329+0000 7f60fbfff700 1 -- 172.21.15.143:0/3856598253 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60ec003ad0 con 0x7f60f0003a00 2023-11-08T19:27:08.349 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.329+0000 7f60fbfff700 1 -- 172.21.15.143:0/3856598253 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60ec003e80 con 0x7f60f0003a00 2023-11-08T19:27:08.349 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.329+0000 7f610a0dc700 1 -- 172.21.15.143:0/3856598253 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61040b5c90 con 0x7f60f0003a00 2023-11-08T19:27:08.349 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.329+0000 7f610a0dc700 1 -- 172.21.15.143:0/3856598253 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6104071fb0 con 0x7f60f0003a00 2023-11-08T19:27:08.349 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.330+0000 7f610a0dc700 1 -- 172.21.15.143:0/3856598253 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6104072170 con 0x7f60f0003a00 2023-11-08T19:27:08.350 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.330+0000 7f60fbfff700 1 -- 172.21.15.143:0/3856598253 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f60ec013470 con 0x7f60f0003a00 2023-11-08T19:27:08.350 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.331+0000 7f60fbfff700 1 --2- 172.21.15.143:0/3856598253 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f60e406af00 0x7f60e406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:08.350 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.331+0000 7f60fbfff700 1 -- 172.21.15.143:0/3856598253 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6811+0+0 (secure 0 0 0) 0x7f60ec08b280 con 0x7f60f0003a00 2023-11-08T19:27:08.350 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.331+0000 7f6102ffd700 1 --2- 172.21.15.143:0/3856598253 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f60e406af00 0x7f60e406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:08.351 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.331+0000 7f6102ffd700 1 --2- 172.21.15.143:0/3856598253 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f60e406af00 0x7f60e406d3c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f60f4001050 tx=0x7f60f4001110 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:08.351 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.335+0000 7f60fbfff700 1 -- 172.21.15.143:0/3856598253 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f60ec056090 con 0x7f60f0003a00 2023-11-08T19:27:08.569 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.568+0000 7f610a0dc700 1 -- 172.21.15.143:0/3856598253 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "testfs", "metadata": "test_metadata_pool", "data": "test_data_pool"} v 0) v1 -- 0x7f610404eee0 con 0x7f60f0003a00 2023-11-08T19:27:08.570 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.568+0000 7f60fbfff700 1 -- 172.21.15.143:0/3856598253 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "testfs", "metadata": "test_metadata_pool", "data": "test_data_pool"}]=-95 pool(11) already has mon-managed snaps; can't attach pool to fs v33) v1 ==== 198+0+0 (secure 0 0 0) 0x7f60ec0596b0 con 0x7f60f0003a00 2023-11-08T19:27:08.571 INFO:teuthology.orchestra.run.smithi143.stderr:Error ENOTSUP: pool(11) already has mon-managed snaps; can't attach pool to fs 2023-11-08T19:27:08.575 DEBUG:teuthology.orchestra.run:got remote process result: 95 2023-11-08T19:27:08.576 DEBUG:teuthology.orchestra.run.smithi143:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool test_data_pool rmsnap snap4 2023-11-08T19:27:08.619 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.618+0000 7f233324f080 1 Processor -- start 2023-11-08T19:27:08.620 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.618+0000 7f233324f080 1 -- start start 2023-11-08T19:27:08.624 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.619+0000 7f233324f080 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abbfed4b20 0x55abbffac1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:08.624 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.619+0000 7f233324f080 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55abbff17800 0x55abbff00780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:08.624 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.619+0000 7f233324f080 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55abbfed4f00 0x55abbff00cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:08.624 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.619+0000 7f233324f080 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abbfe7f520 con 0x55abbfed4f00 2023-11-08T19:27:08.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.619+0000 7f233324f080 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abbfe19740 con 0x55abbfed4b20 2023-11-08T19:27:08.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.619+0000 7f233324f080 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x55abbfe193e0 con 0x55abbff17800 2023-11-08T19:27:08.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.619+0000 7f2329ae5700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55abbfed4f00 0x55abbff00cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:08.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.619+0000 7f2329ae5700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55abbfed4f00 0x55abbff00cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52828/0 (socket says 172.21.15.143:52828) 2023-11-08T19:27:08.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.619+0000 7f2329ae5700 1 -- 172.21.15.143:0/2339689899 learned_addr learned my addr 172.21.15.143:0/2339689899 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:08.626 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.619+0000 7f23292e4700 1 --2- 172.21.15.143:0/2339689899 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abbfed4b20 0x55abbffac1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:08.626 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.619+0000 7f2328ae3700 1 --2- 172.21.15.143:0/2339689899 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55abbff17800 0x55abbff00780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:08.626 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.619+0000 7f2329ae5700 1 -- 172.21.15.143:0/2339689899 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55abbff17800 msgr2=0x55abbff00780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:08.626 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.619+0000 7f2329ae5700 1 --2- 172.21.15.143:0/2339689899 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55abbff17800 0x55abbff00780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:08.626 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.619+0000 7f2329ae5700 1 -- 172.21.15.143:0/2339689899 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abbfed4b20 msgr2=0x55abbffac1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:08.627 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.619+0000 7f2329ae5700 1 --2- 172.21.15.143:0/2339689899 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abbfed4b20 0x55abbffac1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:08.627 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.619+0000 7f2329ae5700 1 -- 172.21.15.143:0/2339689899 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55abbfe257d0 con 0x55abbfed4f00 2023-11-08T19:27:08.627 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.619+0000 7f2329ae5700 1 --2- 172.21.15.143:0/2339689899 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55abbfed4f00 0x55abbff00cc0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x55abbfed6db0 tx=0x7f232400e570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=454cc0a24ec5efc server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:08.627 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.619+0000 7f231b7fe700 1 -- 172.21.15.143:0/2339689899 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2324018070 con 0x55abbfed4f00 2023-11-08T19:27:08.627 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.619+0000 7f231b7fe700 1 -- 172.21.15.143:0/2339689899 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f232400ec20 con 0x55abbfed4f00 2023-11-08T19:27:08.628 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.619+0000 7f231b7fe700 1 -- 172.21.15.143:0/2339689899 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2324013460 con 0x55abbfed4f00 2023-11-08T19:27:08.628 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.620+0000 7f233324f080 1 -- 172.21.15.143:0/2339689899 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55abbfed4f00 msgr2=0x55abbff00cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:08.628 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.620+0000 7f233324f080 1 --2- 172.21.15.143:0/2339689899 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55abbfed4f00 0x55abbff00cc0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x55abbfed6db0 tx=0x7f232400e570 comp rx=0 tx=0).stop 2023-11-08T19:27:08.628 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.620+0000 7f233324f080 1 -- 172.21.15.143:0/2339689899 shutdown_connections 2023-11-08T19:27:08.629 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.620+0000 7f233324f080 1 --2- 172.21.15.143:0/2339689899 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55abbfed4f00 0x55abbff00cc0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:08.629 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.620+0000 7f233324f080 1 --2- 172.21.15.143:0/2339689899 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abbfed4b20 0x55abbffac1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:08.629 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.620+0000 7f233324f080 1 --2- 172.21.15.143:0/2339689899 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55abbff17800 0x55abbff00780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:08.629 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.620+0000 7f233324f080 1 -- 172.21.15.143:0/2339689899 >> 172.21.15.143:0/2339689899 conn(0x55abbff20c40 msgr2=0x55abbff02080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:08.629 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.620+0000 7f233324f080 1 -- 172.21.15.143:0/2339689899 shutdown_connections 2023-11-08T19:27:08.629 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.620+0000 7f233324f080 1 -- 172.21.15.143:0/2339689899 wait complete. 2023-11-08T19:27:08.630 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.620+0000 7f233324f080 1 Processor -- start 2023-11-08T19:27:08.630 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.621+0000 7f233324f080 1 -- start start 2023-11-08T19:27:08.630 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.621+0000 7f233324f080 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abbfed4b20 0x55abbffaa560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:08.630 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.621+0000 7f233324f080 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55abbfed4f00 0x55abbff1f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:08.631 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.621+0000 7f233324f080 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55abbff1f670 0x55abbffa3080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:08.631 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.621+0000 7f233324f080 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abbfe0d680 con 0x55abbfed4f00 2023-11-08T19:27:08.631 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.621+0000 7f233324f080 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abbff04910 con 0x55abbfed4b20 2023-11-08T19:27:08.631 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.621+0000 7f233324f080 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x55abbffa3cd0 con 0x55abbff1f670 2023-11-08T19:27:08.631 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.621+0000 7f2328ae3700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55abbfed4f00 0x55abbff1f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:08.631 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.621+0000 7f2328ae3700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55abbfed4f00 0x55abbff1f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52832/0 (socket says 172.21.15.143:52832) 2023-11-08T19:27:08.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.621+0000 7f2328ae3700 1 -- 172.21.15.143:0/1462699432 learned_addr learned my addr 172.21.15.143:0/1462699432 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:08.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.621+0000 7f2329ae5700 1 --2- 172.21.15.143:0/1462699432 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55abbff1f670 0x55abbffa3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:08.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.621+0000 7f23292e4700 1 --2- 172.21.15.143:0/1462699432 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abbfed4b20 0x55abbffaa560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:08.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.622+0000 7f2328ae3700 1 -- 172.21.15.143:0/1462699432 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55abbff1f670 msgr2=0x55abbffa3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:08.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.622+0000 7f2328ae3700 1 --2- 172.21.15.143:0/1462699432 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55abbff1f670 0x55abbffa3080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:08.633 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.622+0000 7f2328ae3700 1 -- 172.21.15.143:0/1462699432 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abbfed4b20 msgr2=0x55abbffaa560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:08.633 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.622+0000 7f2328ae3700 1 --2- 172.21.15.143:0/1462699432 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abbfed4b20 0x55abbffaa560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:08.633 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.622+0000 7f2328ae3700 1 -- 172.21.15.143:0/1462699432 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f232400d040 con 0x55abbfed4f00 2023-11-08T19:27:08.633 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.622+0000 7f2328ae3700 1 --2- 172.21.15.143:0/1462699432 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55abbfed4f00 0x55abbff1f130 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f2314004050 tx=0x7f231c001a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:08.633 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.622+0000 7f231a7fc700 1 -- 172.21.15.143:0/1462699432 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f231c00ba30 con 0x55abbfed4f00 2023-11-08T19:27:08.634 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.622+0000 7f231a7fc700 1 -- 172.21.15.143:0/1462699432 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f231c000ed0 con 0x55abbfed4f00 2023-11-08T19:27:08.634 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.622+0000 7f233324f080 1 -- 172.21.15.143:0/1462699432 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55abbfe18b70 con 0x55abbfed4f00 2023-11-08T19:27:08.634 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.622+0000 7f231a7fc700 1 -- 172.21.15.143:0/1462699432 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f231c00e6b0 con 0x55abbfed4f00 2023-11-08T19:27:08.634 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.622+0000 7f233324f080 1 -- 172.21.15.143:0/1462699432 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55abbfe185d0 con 0x55abbfed4f00 2023-11-08T19:27:08.635 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.623+0000 7f231a7fc700 1 -- 172.21.15.143:0/1462699432 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f231c00e810 con 0x55abbfed4f00 2023-11-08T19:27:08.635 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.624+0000 7f231a7fc700 1 --2- 172.21.15.143:0/1462699432 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f230c069820 0x7f230c06bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:08.635 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.624+0000 7f231a7fc700 1 -- 172.21.15.143:0/1462699432 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6811+0+0 (secure 0 0 0) 0x7f231c004070 con 0x55abbfed4f00 2023-11-08T19:27:08.636 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.624+0000 7f233324f080 1 -- 172.21.15.143:0/1462699432 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- pool_op(delete snap pool 11 tid 1 name snap4 v0) v4 -- 0x55abbfe894a0 con 0x55abbfed4f00 2023-11-08T19:27:08.636 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.624+0000 7f23292e4700 1 --2- 172.21.15.143:0/1462699432 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f230c069820 0x7f230c06bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:08.636 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:08.624+0000 7f23292e4700 1 --2- 172.21.15.143:0/1462699432 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f230c069820 0x7f230c06bce0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x55abbfed5760 tx=0x7f23200057b0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:09.179 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.177+0000 7f231a7fc700 1 -- 172.21.15.143:0/1462699432 <== mon.0 v2:172.21.15.143:3300/0 6 ==== pool_op_reply(tid 1 (0) Success v66) v1 ==== 43+0+0 (secure 0 0 0) 0x7f231c055030 con 0x55abbfed4f00 2023-11-08T19:27:09.179 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.177+0000 7f231a7fc700 1 -- 172.21.15.143:0/1462699432 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=66}) v3 -- 0x7f230c073a50 con 0x55abbfed4f00 2023-11-08T19:27:09.179 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.177+0000 7f231a7fc700 1 -- 172.21.15.143:0/1462699432 <== mon.0 v2:172.21.15.143:3300/0 7 ==== osd_map(66..66 src has 1..66) v4 ==== 970+0+0 (secure 0 0 0) 0x7f231c0593f0 con 0x55abbfed4f00 2023-11-08T19:27:09.180 INFO:teuthology.orchestra.run.smithi143.stdout:removed pool test_data_pool snap snap4 2023-11-08T19:27:09.180 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.178+0000 7f233324f080 1 -- 172.21.15.143:0/1462699432 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f230c069820 msgr2=0x7f230c06bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:09.181 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.178+0000 7f233324f080 1 --2- 172.21.15.143:0/1462699432 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f230c069820 0x7f230c06bce0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x55abbfed5760 tx=0x7f23200057b0 comp rx=0 tx=0).stop 2023-11-08T19:27:09.181 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.178+0000 7f233324f080 1 -- 172.21.15.143:0/1462699432 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55abbfed4f00 msgr2=0x55abbff1f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:09.181 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.178+0000 7f233324f080 1 --2- 172.21.15.143:0/1462699432 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55abbfed4f00 0x55abbff1f130 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f2314004050 tx=0x7f231c001a70 comp rx=0 tx=0).stop 2023-11-08T19:27:09.181 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.178+0000 7f233324f080 1 -- 172.21.15.143:0/1462699432 shutdown_connections 2023-11-08T19:27:09.182 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.178+0000 7f233324f080 1 --2- 172.21.15.143:0/1462699432 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f230c069820 0x7f230c06bce0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:09.182 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.178+0000 7f233324f080 1 --2- 172.21.15.143:0/1462699432 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55abbff1f670 0x55abbffa3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:09.182 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.178+0000 7f233324f080 1 --2- 172.21.15.143:0/1462699432 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abbfed4b20 0x55abbffaa560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:09.182 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.178+0000 7f233324f080 1 --2- 172.21.15.143:0/1462699432 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55abbfed4f00 0x55abbff1f130 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:09.182 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.178+0000 7f233324f080 1 -- 172.21.15.143:0/1462699432 >> 172.21.15.143:0/1462699432 conn(0x55abbff20c40 msgr2=0x55abbfed5940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:09.183 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.178+0000 7f233324f080 1 -- 172.21.15.143:0/1462699432 shutdown_connections 2023-11-08T19:27:09.183 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.178+0000 7f233324f080 1 -- 172.21.15.143:0/1462699432 wait complete. 2023-11-08T19:27:09.184 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete test_data_pool 2023-11-08T19:27:09.351 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.349+0000 7f03e832b700 1 Processor -- start 2023-11-08T19:27:09.351 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.349+0000 7f03e832b700 1 -- start start 2023-11-08T19:27:09.353 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.350+0000 7f03e832b700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f03e015ba10 0x7f03e0072bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:09.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.350+0000 7f03e832b700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03e015f6a0 con 0x7f03e015bdf0 2023-11-08T19:27:09.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.350+0000 7f03e832b700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03e006b320 con 0x7f03e00b2300 2023-11-08T19:27:09.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.350+0000 7f03e832b700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03e0073110 con 0x7f03e015ba10 2023-11-08T19:27:09.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.350+0000 7f03e60c7700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f03e015ba10 0x7f03e0072bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:09.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.350+0000 7f03e58c6700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f03e015bdf0 0x7f03e006ac10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:48994/0 (socket says 172.21.15.143:48994) 2023-11-08T19:27:09.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.350+0000 7f03e58c6700 1 -- 172.21.15.143:0/1050547047 learned_addr learned my addr 172.21.15.143:0/1050547047 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:27:09.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.350+0000 7f03e60c7700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f03e015ba10 0x7f03e0072bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52838/0 (socket says 172.21.15.143:52838) 2023-11-08T19:27:09.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.351+0000 7f03e60c7700 1 -- 172.21.15.143:0/1050547047 >> v1:172.21.15.174:6789/0 conn(0x7f03e00b2300 legacy=0x7f03e015fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:09.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.351+0000 7f03e60c7700 1 -- 172.21.15.143:0/1050547047 >> v1:172.21.15.143:6789/0 conn(0x7f03e015bdf0 legacy=0x7f03e006ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:09.356 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.351+0000 7f03e60c7700 1 -- 172.21.15.143:0/1050547047 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03e015c640 con 0x7f03e015ba10 2023-11-08T19:27:09.356 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.351+0000 7f03e50c5700 1 -- 172.21.15.143:0/1050547047 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 907609234 0 0) 0x7f03e015f6a0 con 0x7f03e015bdf0 2023-11-08T19:27:09.356 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.351+0000 7f03e60c7700 1 --2- 172.21.15.143:0/1050547047 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f03e015ba10 0x7f03e0072bd0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f03d0009840 tx=0x7f03d0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c4d467d3969fe0e server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:09.356 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.351+0000 7f03e50c5700 1 -- 172.21.15.143:0/1050547047 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03d0004230 con 0x7f03e015ba10 2023-11-08T19:27:09.356 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.351+0000 7f03e50c5700 1 -- 172.21.15.143:0/1050547047 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03d0004390 con 0x7f03e015ba10 2023-11-08T19:27:09.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.351+0000 7f03e50c5700 1 -- 172.21.15.143:0/1050547047 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03d00036a0 con 0x7f03e015ba10 2023-11-08T19:27:09.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.351+0000 7f03e832b700 1 -- 172.21.15.143:0/1050547047 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f03e015ba10 msgr2=0x7f03e0072bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:09.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.351+0000 7f03e832b700 1 --2- 172.21.15.143:0/1050547047 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f03e015ba10 0x7f03e0072bd0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f03d0009840 tx=0x7f03d0009b50 comp rx=0 tx=0).stop 2023-11-08T19:27:09.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.352+0000 7f03e832b700 1 -- 172.21.15.143:0/1050547047 shutdown_connections 2023-11-08T19:27:09.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.352+0000 7f03e832b700 1 --2- 172.21.15.143:0/1050547047 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f03e015ba10 0x7f03e0072bd0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:09.358 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.352+0000 7f03e832b700 1 -- 172.21.15.143:0/1050547047 >> 172.21.15.143:0/1050547047 conn(0x7f03e00cd300 msgr2=0x7f03e00cd710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:09.358 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.352+0000 7f03e832b700 1 -- 172.21.15.143:0/1050547047 shutdown_connections 2023-11-08T19:27:09.358 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.352+0000 7f03e832b700 1 -- 172.21.15.143:0/1050547047 wait complete. 2023-11-08T19:27:09.361 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.353+0000 7f03e832b700 1 Processor -- start 2023-11-08T19:27:09.361 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.353+0000 7f03e832b700 1 -- start start 2023-11-08T19:27:09.361 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.353+0000 7f03e832b700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f03e00b2300 0x7f03e014fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:09.361 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.353+0000 7f03e832b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f03e015ba10 0x7f03e01e5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:09.362 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.353+0000 7f03e832b700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f03e015bdf0 0x7f03e00b1170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:09.362 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.353+0000 7f03e832b700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03e0151800 con 0x7f03e00b2300 2023-11-08T19:27:09.362 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.353+0000 7f03e832b700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03e01e5b80 con 0x7f03e015ba10 2023-11-08T19:27:09.362 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.353+0000 7f03e832b700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03e0151140 con 0x7f03e015bdf0 2023-11-08T19:27:09.363 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.354+0000 7f03e60c7700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f03e00b2300 0x7f03e014fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:09.363 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.354+0000 7f03e60c7700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f03e00b2300 0x7f03e014fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52844/0 (socket says 172.21.15.143:52844) 2023-11-08T19:27:09.363 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.354+0000 7f03e60c7700 1 -- 172.21.15.143:0/2376419429 learned_addr learned my addr 172.21.15.143:0/2376419429 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:09.363 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.354+0000 7f03e58c6700 1 --2- 172.21.15.143:0/2376419429 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f03e015ba10 0x7f03e01e5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:09.363 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.354+0000 7f03e68c8700 1 --2- 172.21.15.143:0/2376419429 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f03e015bdf0 0x7f03e00b1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:09.363 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.354+0000 7f03e60c7700 1 -- 172.21.15.143:0/2376419429 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f03e015bdf0 msgr2=0x7f03e00b1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:09.364 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.354+0000 7f03e60c7700 1 --2- 172.21.15.143:0/2376419429 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f03e015bdf0 0x7f03e00b1170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:09.364 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.354+0000 7f03e60c7700 1 -- 172.21.15.143:0/2376419429 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f03e015ba10 msgr2=0x7f03e01e5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:09.364 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.354+0000 7f03e60c7700 1 --2- 172.21.15.143:0/2376419429 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f03e015ba10 0x7f03e01e5220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:09.364 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.354+0000 7f03e60c7700 1 -- 172.21.15.143:0/2376419429 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03d00094a0 con 0x7f03e00b2300 2023-11-08T19:27:09.364 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.354+0000 7f03e60c7700 1 --2- 172.21.15.143:0/2376419429 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f03e00b2300 0x7f03e014fca0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f03d00090b0 tx=0x7f03d0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:09.365 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.354+0000 7f03d6ffd700 1 -- 172.21.15.143:0/2376419429 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03d0011830 con 0x7f03e00b2300 2023-11-08T19:27:09.365 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.354+0000 7f03d6ffd700 1 -- 172.21.15.143:0/2376419429 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03d001e070 con 0x7f03e00b2300 2023-11-08T19:27:09.365 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.354+0000 7f03e832b700 1 -- 172.21.15.143:0/2376419429 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03e0151420 con 0x7f03e00b2300 2023-11-08T19:27:09.365 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.354+0000 7f03d6ffd700 1 -- 172.21.15.143:0/2376419429 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03d0011bb0 con 0x7f03e00b2300 2023-11-08T19:27:09.365 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.354+0000 7f03e832b700 1 -- 172.21.15.143:0/2376419429 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03e0150d60 con 0x7f03e00b2300 2023-11-08T19:27:09.366 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.355+0000 7f03e832b700 1 -- 172.21.15.143:0/2376419429 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03e0150fb0 con 0x7f03e00b2300 2023-11-08T19:27:09.366 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.356+0000 7f03d6ffd700 1 -- 172.21.15.143:0/2376419429 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f03d0011d10 con 0x7f03e00b2300 2023-11-08T19:27:09.366 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.356+0000 7f03d6ffd700 1 --2- 172.21.15.143:0/2376419429 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f03c406af00 0x7f03c406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:09.366 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.356+0000 7f03d6ffd700 1 -- 172.21.15.143:0/2376419429 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6828+0+0 (secure 0 0 0) 0x7f03d008ad80 con 0x7f03e00b2300 2023-11-08T19:27:09.366 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.356+0000 7f03e58c6700 1 --2- 172.21.15.143:0/2376419429 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f03c406af00 0x7f03c406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:09.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.357+0000 7f03e58c6700 1 --2- 172.21.15.143:0/2376419429 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f03c406af00 0x7f03c406d3c0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f03dc000f60 tx=0x7f03dc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:09.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.361+0000 7f03d6ffd700 1 -- 172.21.15.143:0/2376419429 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f03d0055a60 con 0x7f03e00b2300 2023-11-08T19:27:09.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.554+0000 7f03e832b700 1 -- 172.21.15.143:0/2376419429 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "test_data_pool"} v 0) v1 -- 0x7f03e004eee0 con 0x7f03e00b2300 2023-11-08T19:27:09.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.554+0000 7f03d6ffd700 1 -- 172.21.15.143:0/2376419429 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "test_data_pool"}]=-1 WARNING: this will *PERMANENTLY DESTROY* all data stored in pool test_data_pool. If you are *ABSOLUTELY CERTAIN* that is what you want, pass the pool name *twice*, followed by --yes-i-really-really-mean-it. v66) v1 ==== 296+0+0 (secure 0 0 0) 0x7f03d0020020 con 0x7f03e00b2300 2023-11-08T19:27:09.557 INFO:teuthology.orchestra.run.smithi143.stderr:Error EPERM: WARNING: this will *PERMANENTLY DESTROY* all data stored in pool test_data_pool. If you are *ABSOLUTELY CERTAIN* that is what you want, pass the pool name *twice*, followed by --yes-i-really-really-mean-it. 2023-11-08T19:27:09.560 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-08T19:27:09.561 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete test_metadata_pool 2023-11-08T19:27:09.684 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.683+0000 7f87a6a3f700 1 Processor -- start 2023-11-08T19:27:09.685 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.683+0000 7f87a6a3f700 1 -- start start 2023-11-08T19:27:09.688 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.684+0000 7f87a6a3f700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f87a015bc80 0x7f87a00b6280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:09.688 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.684+0000 7f87a6a3f700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f87a006b0b0 0x7f87a006c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:09.688 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.684+0000 7f87a6a3f700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87a00b67c0 con 0x7f87a00b22e0 2023-11-08T19:27:09.689 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.684+0000 7f87a6a3f700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87a00ce290 con 0x7f87a006b0b0 2023-11-08T19:27:09.689 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.684+0000 7f87a6a3f700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87a015c060 con 0x7f87a015bc80 2023-11-08T19:27:09.689 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.684+0000 7f87a4fdc700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f87a00b22e0 0x7f87a00b37b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:49008/0 (socket says 172.21.15.143:49008) 2023-11-08T19:27:09.689 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.684+0000 7f87a4fdc700 1 -- 172.21.15.143:0/1726277001 learned_addr learned my addr 172.21.15.143:0/1726277001 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:27:09.689 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.684+0000 7f879f7fe700 1 --2- 172.21.15.143:0/1726277001 >> v2:172.21.15.174:3300/0 conn(0x7f87a006b0b0 0x7f87a006c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:09.690 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.684+0000 7f879ffff700 1 --2- 172.21.15.143:0/1726277001 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f87a015bc80 0x7f87a00b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:09.690 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.684+0000 7f879effd700 1 -- 172.21.15.143:0/1726277001 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 636205749 0 0) 0x7f87a00b67c0 con 0x7f87a00b22e0 2023-11-08T19:27:09.690 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.684+0000 7f879effd700 1 -- 172.21.15.143:0/1726277001 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87880035b0 con 0x7f87a00b22e0 2023-11-08T19:27:09.690 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.684+0000 7f879effd700 1 -- 172.21.15.143:0/1726277001 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1285373070 0 0) 0x7f87880035b0 con 0x7f87a00b22e0 2023-11-08T19:27:09.690 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.684+0000 7f879effd700 1 -- 172.21.15.143:0/1726277001 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f87a015bc80 msgr2=0x7f87a00b6280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:09.691 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.684+0000 7f879effd700 1 --2- 172.21.15.143:0/1726277001 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f87a015bc80 0x7f87a00b6280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:09.691 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.684+0000 7f879effd700 1 -- 172.21.15.143:0/1726277001 >> v2:172.21.15.174:3300/0 conn(0x7f87a006b0b0 msgr2=0x7f87a006c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:09.691 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.684+0000 7f879effd700 1 --2- 172.21.15.143:0/1726277001 >> v2:172.21.15.174:3300/0 conn(0x7f87a006b0b0 0x7f87a006c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:09.691 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.684+0000 7f879effd700 1 -- 172.21.15.143:0/1726277001 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87a006cc50 con 0x7f87a00b22e0 2023-11-08T19:27:09.691 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.684+0000 7f879effd700 1 -- 172.21.15.143:0/1726277001 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4147756303 0 0) 0x7f87980019f0 con 0x7f87a00b22e0 2023-11-08T19:27:09.692 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.685+0000 7f879effd700 1 -- 172.21.15.143:0/1726277001 >> v1:172.21.15.143:6789/0 conn(0x7f87a00b22e0 legacy=0x7f87a00b37b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:09.692 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.685+0000 7f879effd700 1 --2- 172.21.15.143:0/1726277001 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8788003a30 0x7f8788003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:09.692 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.685+0000 7f879effd700 1 --2- 172.21.15.143:0/1726277001 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f87a015bc80 0x7f8788004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:09.692 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.685+0000 7f879effd700 1 --2- 172.21.15.143:0/1726277001 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8788004d90 0x7f8788005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:09.692 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.685+0000 7f879effd700 1 -- 172.21.15.143:0/1726277001 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87a00ce290 con 0x7f8788004d90 2023-11-08T19:27:09.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.685+0000 7f879effd700 1 -- 172.21.15.143:0/1726277001 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87a015c060 con 0x7f8788003a30 2023-11-08T19:27:09.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.685+0000 7f879effd700 1 -- 172.21.15.143:0/1726277001 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87880059e0 con 0x7f87a015bc80 2023-11-08T19:27:09.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.685+0000 7f87a4fdc700 1 --2- 172.21.15.143:0/1726277001 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8788004d90 0x7f8788005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:09.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.685+0000 7f879f7fe700 1 --2- 172.21.15.143:0/1726277001 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f87a015bc80 0x7f8788004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:09.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.685+0000 7f879ffff700 1 --2- 172.21.15.143:0/1726277001 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8788003a30 0x7f8788003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:09.694 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.685+0000 7f87a4fdc700 1 -- 172.21.15.143:0/1726277001 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f87a015bc80 msgr2=0x7f8788004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:09.694 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.685+0000 7f87a4fdc700 1 --2- 172.21.15.143:0/1726277001 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f87a015bc80 0x7f8788004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:09.694 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.685+0000 7f87a4fdc700 1 -- 172.21.15.143:0/1726277001 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8788003a30 msgr2=0x7f8788003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:09.694 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.685+0000 7f87a4fdc700 1 --2- 172.21.15.143:0/1726277001 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8788003a30 0x7f8788003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:09.694 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.685+0000 7f87a4fdc700 1 -- 172.21.15.143:0/1726277001 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87880035b0 con 0x7f8788004d90 2023-11-08T19:27:09.695 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.685+0000 7f87a4fdc700 1 --2- 172.21.15.143:0/1726277001 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8788004d90 0x7f8788005210 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f879800a830 tx=0x7f8788005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ad68d330c4646ec server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:09.695 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.685+0000 7f879effd700 1 -- 172.21.15.143:0/1726277001 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f879800a2f0 con 0x7f8788004d90 2023-11-08T19:27:09.695 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.685+0000 7f879effd700 1 -- 172.21.15.143:0/1726277001 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8798012600 con 0x7f8788004d90 2023-11-08T19:27:09.695 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.685+0000 7f879effd700 1 -- 172.21.15.143:0/1726277001 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87980129b0 con 0x7f8788004d90 2023-11-08T19:27:09.695 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.686+0000 7f87a6a3f700 1 -- 172.21.15.143:0/1726277001 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8788004d90 msgr2=0x7f8788005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:09.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.686+0000 7f87a6a3f700 1 --2- 172.21.15.143:0/1726277001 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8788004d90 0x7f8788005210 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f879800a830 tx=0x7f8788005880 comp rx=0 tx=0).stop 2023-11-08T19:27:09.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.686+0000 7f87a4fdc700 1 -- 172.21.15.143:0/1726277001 reap_dead start 2023-11-08T19:27:09.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.686+0000 7f87a6a3f700 1 -- 172.21.15.143:0/1726277001 shutdown_connections 2023-11-08T19:27:09.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.686+0000 7f87a6a3f700 1 -- 172.21.15.143:0/1726277001 >> 172.21.15.143:0/1726277001 conn(0x7f87a00cd2e0 msgr2=0x7f87a00cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:09.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.686+0000 7f87a6a3f700 1 -- 172.21.15.143:0/1726277001 shutdown_connections 2023-11-08T19:27:09.697 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.686+0000 7f87a6a3f700 1 -- 172.21.15.143:0/1726277001 wait complete. 2023-11-08T19:27:09.699 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.687+0000 7f87a6a3f700 1 Processor -- start 2023-11-08T19:27:09.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.687+0000 7f87a6a3f700 1 -- start start 2023-11-08T19:27:09.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.688+0000 7f87a6a3f700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8788004d90 0x7f87a00b1440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:09.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.688+0000 7f87a6a3f700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f87a01f0090 0x7f87a00b1980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:09.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.688+0000 7f87a6a3f700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f87a01f1f50 0x7f87a015df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:09.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.688+0000 7f87a6a3f700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87a01ef1d0 con 0x7f87a01f0090 2023-11-08T19:27:09.701 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.688+0000 7f87a6a3f700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87a01ef4b0 con 0x7f8788004d90 2023-11-08T19:27:09.701 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.688+0000 7f87a6a3f700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87a01ef790 con 0x7f87a01f1f50 2023-11-08T19:27:09.701 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.688+0000 7f879f7fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f87a01f0090 0x7f87a00b1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:09.701 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.688+0000 7f879f7fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f87a01f0090 0x7f87a00b1980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52866/0 (socket says 172.21.15.143:52866) 2023-11-08T19:27:09.701 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.688+0000 7f879f7fe700 1 -- 172.21.15.143:0/2157896301 learned_addr learned my addr 172.21.15.143:0/2157896301 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:09.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.688+0000 7f87a4fdc700 1 --2- 172.21.15.143:0/2157896301 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f87a01f1f50 0x7f87a015df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:09.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.688+0000 7f879ffff700 1 --2- 172.21.15.143:0/2157896301 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8788004d90 0x7f87a00b1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:09.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.688+0000 7f879f7fe700 1 -- 172.21.15.143:0/2157896301 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f87a01f1f50 msgr2=0x7f87a015df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:09.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.688+0000 7f879f7fe700 1 --2- 172.21.15.143:0/2157896301 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f87a01f1f50 0x7f87a015df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:09.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.688+0000 7f879f7fe700 1 -- 172.21.15.143:0/2157896301 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8788004d90 msgr2=0x7f87a00b1440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:09.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.688+0000 7f879f7fe700 1 --2- 172.21.15.143:0/2157896301 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8788004d90 0x7f87a00b1440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:09.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.688+0000 7f879f7fe700 1 -- 172.21.15.143:0/2157896301 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87a01efa70 con 0x7f87a01f0090 2023-11-08T19:27:09.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.689+0000 7f879f7fe700 1 --2- 172.21.15.143:0/2157896301 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f87a01f0090 0x7f87a00b1980 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f8790011aa0 tx=0x7f8790011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:09.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.689+0000 7f879cff9700 1 -- 172.21.15.143:0/2157896301 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f879000f9e0 con 0x7f87a01f0090 2023-11-08T19:27:09.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.689+0000 7f879cff9700 1 -- 172.21.15.143:0/2157896301 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f879000fb40 con 0x7f87a01f0090 2023-11-08T19:27:09.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.689+0000 7f87a6a3f700 1 -- 172.21.15.143:0/2157896301 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87a01efd50 con 0x7f87a01f0090 2023-11-08T19:27:09.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.689+0000 7f879cff9700 1 -- 172.21.15.143:0/2157896301 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8790014440 con 0x7f87a01f0090 2023-11-08T19:27:09.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.689+0000 7f87a6a3f700 1 -- 172.21.15.143:0/2157896301 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87a015e4d0 con 0x7f87a01f0090 2023-11-08T19:27:09.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.690+0000 7f87a6a3f700 1 -- 172.21.15.143:0/2157896301 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87a004eee0 con 0x7f87a01f0090 2023-11-08T19:27:09.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.690+0000 7f879cff9700 1 -- 172.21.15.143:0/2157896301 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f879001d400 con 0x7f87a01f0090 2023-11-08T19:27:09.705 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.691+0000 7f879cff9700 1 --2- 172.21.15.143:0/2157896301 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f878c06af00 0x7f878c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:09.705 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.691+0000 7f879cff9700 1 -- 172.21.15.143:0/2157896301 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6828+0+0 (secure 0 0 0) 0x7f879008a870 con 0x7f87a01f0090 2023-11-08T19:27:09.705 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.691+0000 7f879ffff700 1 --2- 172.21.15.143:0/2157896301 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f878c06af00 0x7f878c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:09.705 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.691+0000 7f879ffff700 1 --2- 172.21.15.143:0/2157896301 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f878c06af00 0x7f878c06d3c0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f87a00b1fa0 tx=0x7f8794008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:09.705 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.694+0000 7f879cff9700 1 -- 172.21.15.143:0/2157896301 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f87900555a0 con 0x7f87a01f0090 2023-11-08T19:27:09.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.885+0000 7f87a6a3f700 1 -- 172.21.15.143:0/2157896301 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "test_metadata_pool"} v 0) v1 -- 0x7f87a00af810 con 0x7f87a01f0090 2023-11-08T19:27:09.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:09.886+0000 7f879cff9700 1 -- 172.21.15.143:0/2157896301 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "test_metadata_pool"}]=-1 WARNING: this will *PERMANENTLY DESTROY* all data stored in pool test_metadata_pool. If you are *ABSOLUTELY CERTAIN* that is what you want, pass the pool name *twice*, followed by --yes-i-really-really-mean-it. v66) v1 ==== 304+0+0 (secure 0 0 0) 0x7f8790058bc0 con 0x7f87a01f0090 2023-11-08T19:27:09.891 INFO:teuthology.orchestra.run.smithi143.stderr:Error EPERM: WARNING: this will *PERMANENTLY DESTROY* all data stored in pool test_metadata_pool. If you are *ABSOLUTELY CERTAIN* that is what you want, pass the pool name *twice*, followed by --yes-i-really-really-mean-it. 2023-11-08T19:27:09.892 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-08T19:27:09.894 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-08T19:27:09.894 DEBUG:teuthology.orchestra.run.smithi143:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-08T19:27:09.939 DEBUG:teuthology.orchestra.run.smithi174:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-08T19:27:09.989 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-08T19:27:09.990 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-08T19:27:09.990 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:27:10.008 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-08T19:27:10.008 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:27:10.065 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:27:10.226 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.224+0000 7fc97933f700 1 Processor -- start 2023-11-08T19:27:10.226 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.225+0000 7fc97933f700 1 -- start start 2023-11-08T19:27:10.230 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.225+0000 7fc97933f700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc97415aa30 0x7fc9740cf510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:10.230 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.225+0000 7fc97933f700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9740cf080 con 0x7fc9740cc530 2023-11-08T19:27:10.230 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.225+0000 7fc97933f700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9740ce470 con 0x7fc9740b2210 2023-11-08T19:27:10.231 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.225+0000 7fc97933f700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9740ce750 con 0x7fc97415aa30 2023-11-08T19:27:10.231 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.225+0000 7fc9737fe700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fc9740cc530 0x7fc97406db40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:49028/0 (socket says 172.21.15.143:49028) 2023-11-08T19:27:10.231 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.225+0000 7fc9737fe700 1 -- 172.21.15.143:0/2479324621 learned_addr learned my addr 172.21.15.143:0/2479324621 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:27:10.231 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.226+0000 7fc972ffd700 1 --2- 172.21.15.143:0/2479324621 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc97415aa30 0x7fc9740cf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:10.232 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.226+0000 7fc971ffb700 1 -- 172.21.15.143:0/2479324621 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4116533925 0 0) 0x7fc9740cf080 con 0x7fc9740cc530 2023-11-08T19:27:10.232 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.226+0000 7fc971ffb700 1 -- 172.21.15.143:0/2479324621 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc95c0035b0 con 0x7fc9740cc530 2023-11-08T19:27:10.232 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.226+0000 7fc971ffb700 1 -- 172.21.15.143:0/2479324621 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 688615127 0 0) 0x7fc95c0035b0 con 0x7fc9740cc530 2023-11-08T19:27:10.233 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.226+0000 7fc971ffb700 1 -- 172.21.15.143:0/2479324621 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc97415aa30 msgr2=0x7fc9740cf510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:10.233 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.226+0000 7fc971ffb700 1 --2- 172.21.15.143:0/2479324621 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc97415aa30 0x7fc9740cf510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:10.233 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.226+0000 7fc971ffb700 1 -- 172.21.15.143:0/2479324621 >> v1:172.21.15.174:6789/0 conn(0x7fc9740b2210 legacy=0x7fc9740cbe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:10.233 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.226+0000 7fc971ffb700 1 -- 172.21.15.143:0/2479324621 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9740cea30 con 0x7fc9740cc530 2023-11-08T19:27:10.233 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.226+0000 7fc971ffb700 1 -- 172.21.15.143:0/2479324621 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4147756303 0 0) 0x7fc96c0019f0 con 0x7fc9740cc530 2023-11-08T19:27:10.234 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.226+0000 7fc971ffb700 1 -- 172.21.15.143:0/2479324621 >> v1:172.21.15.143:6789/0 conn(0x7fc9740cc530 legacy=0x7fc97406db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:10.234 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.226+0000 7fc971ffb700 1 --2- 172.21.15.143:0/2479324621 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc95c003a30 0x7fc95c003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:10.234 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.226+0000 7fc971ffb700 1 --2- 172.21.15.143:0/2479324621 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc97415aa30 0x7fc95c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:10.234 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.226+0000 7fc971ffb700 1 --2- 172.21.15.143:0/2479324621 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc95c004d90 0x7fc95c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:10.235 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.226+0000 7fc971ffb700 1 -- 172.21.15.143:0/2479324621 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9740ce750 con 0x7fc95c003a30 2023-11-08T19:27:10.235 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.226+0000 7fc971ffb700 1 -- 172.21.15.143:0/2479324621 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc95c00bac0 con 0x7fc95c004d90 2023-11-08T19:27:10.235 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.226+0000 7fc971ffb700 1 -- 172.21.15.143:0/2479324621 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc95c00de90 con 0x7fc97415aa30 2023-11-08T19:27:10.235 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.226+0000 7fc972ffd700 1 --2- 172.21.15.143:0/2479324621 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc95c003a30 0x7fc95c003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:10.236 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.226+0000 7fc9727fc700 1 --2- 172.21.15.143:0/2479324621 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc97415aa30 0x7fc95c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:10.236 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.226+0000 7fc9737fe700 1 --2- 172.21.15.143:0/2479324621 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc95c004d90 0x7fc95c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:10.236 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.226+0000 7fc972ffd700 1 -- 172.21.15.143:0/2479324621 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc97415aa30 msgr2=0x7fc95c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:10.236 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.226+0000 7fc972ffd700 1 --2- 172.21.15.143:0/2479324621 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc97415aa30 0x7fc95c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:10.237 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.226+0000 7fc972ffd700 1 -- 172.21.15.143:0/2479324621 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc95c004d90 msgr2=0x7fc95c005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:10.237 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.226+0000 7fc972ffd700 1 --2- 172.21.15.143:0/2479324621 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc95c004d90 0x7fc95c005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:10.237 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.226+0000 7fc972ffd700 1 -- 172.21.15.143:0/2479324621 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc95c0035b0 con 0x7fc95c003a30 2023-11-08T19:27:10.237 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.226+0000 7fc9737fe700 1 --2- 172.21.15.143:0/2479324621 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc95c004d90 0x7fc95c005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:27:10.237 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.227+0000 7fc972ffd700 1 --2- 172.21.15.143:0/2479324621 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc95c003a30 0x7fc95c003e50 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fc968009b10 tx=0x7fc95c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=844bd30b8aec3712 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:10.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.227+0000 7fc971ffb700 1 -- 172.21.15.143:0/2479324621 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc968005a20 con 0x7fc95c003a30 2023-11-08T19:27:10.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.227+0000 7fc971ffb700 1 -- 172.21.15.143:0/2479324621 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc968005b80 con 0x7fc95c003a30 2023-11-08T19:27:10.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.227+0000 7fc971ffb700 1 -- 172.21.15.143:0/2479324621 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc96800e6c0 con 0x7fc95c003a30 2023-11-08T19:27:10.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.227+0000 7fc97933f700 1 -- 172.21.15.143:0/2479324621 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc95c003a30 msgr2=0x7fc95c003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:10.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.227+0000 7fc97933f700 1 --2- 172.21.15.143:0/2479324621 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc95c003a30 0x7fc95c003e50 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fc968009b10 tx=0x7fc95c005880 comp rx=0 tx=0).stop 2023-11-08T19:27:10.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.227+0000 7fc9737fe700 1 -- 172.21.15.143:0/2479324621 reap_dead start 2023-11-08T19:27:10.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.227+0000 7fc97933f700 1 -- 172.21.15.143:0/2479324621 shutdown_connections 2023-11-08T19:27:10.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.227+0000 7fc97933f700 1 -- 172.21.15.143:0/2479324621 >> 172.21.15.143:0/2479324621 conn(0x7fc974099ab0 msgr2=0x7fc9740d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:10.240 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.227+0000 7fc97933f700 1 -- 172.21.15.143:0/2479324621 shutdown_connections 2023-11-08T19:27:10.240 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.227+0000 7fc97933f700 1 -- 172.21.15.143:0/2479324621 wait complete. 2023-11-08T19:27:10.240 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.229+0000 7fc97933f700 1 Processor -- start 2023-11-08T19:27:10.240 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.229+0000 7fc97933f700 1 -- start start 2023-11-08T19:27:10.243 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.229+0000 7fc97933f700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc95c003a30 0x7fc97415d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:10.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.229+0000 7fc97933f700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc97415be60 0x7fc9740af390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:10.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.229+0000 7fc97933f700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc97415c2e0 0x7fc9740af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:10.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.229+0000 7fc97933f700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9741ef450 con 0x7fc95c003a30 2023-11-08T19:27:10.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.229+0000 7fc97933f700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9741ef710 con 0x7fc97415c2e0 2023-11-08T19:27:10.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.229+0000 7fc97933f700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9741ef9f0 con 0x7fc97415be60 2023-11-08T19:27:10.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.229+0000 7fc972ffd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc95c003a30 0x7fc97415d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:10.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.230+0000 7fc972ffd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc95c003a30 0x7fc97415d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52882/0 (socket says 172.21.15.143:52882) 2023-11-08T19:27:10.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.230+0000 7fc972ffd700 1 -- 172.21.15.143:0/3825828124 learned_addr learned my addr 172.21.15.143:0/3825828124 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:10.246 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.230+0000 7fc972ffd700 1 -- 172.21.15.143:0/3825828124 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc97415be60 msgr2=0x7fc9740af390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:10.246 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.230+0000 7fc972ffd700 1 --2- 172.21.15.143:0/3825828124 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc97415be60 0x7fc9740af390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:10.246 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.230+0000 7fc972ffd700 1 -- 172.21.15.143:0/3825828124 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc97415c2e0 msgr2=0x7fc9740af8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:10.246 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.230+0000 7fc972ffd700 1 --2- 172.21.15.143:0/3825828124 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc97415c2e0 0x7fc9740af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:10.246 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.230+0000 7fc972ffd700 1 -- 172.21.15.143:0/3825828124 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9741efcd0 con 0x7fc95c003a30 2023-11-08T19:27:10.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.230+0000 7fc972ffd700 1 --2- 172.21.15.143:0/3825828124 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc95c003a30 0x7fc97415d950 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fc95c0047b0 tx=0x7fc9680047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:10.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.230+0000 7fc95b7fe700 1 -- 172.21.15.143:0/3825828124 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc968020070 con 0x7fc95c003a30 2023-11-08T19:27:10.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.230+0000 7fc95b7fe700 1 -- 172.21.15.143:0/3825828124 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc968004030 con 0x7fc95c003a30 2023-11-08T19:27:10.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.230+0000 7fc97933f700 1 -- 172.21.15.143:0/3825828124 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9741effb0 con 0x7fc95c003a30 2023-11-08T19:27:10.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.230+0000 7fc95b7fe700 1 -- 172.21.15.143:0/3825828124 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9680041e0 con 0x7fc95c003a30 2023-11-08T19:27:10.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.230+0000 7fc97933f700 1 -- 172.21.15.143:0/3825828124 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc97415de90 con 0x7fc95c003a30 2023-11-08T19:27:10.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.231+0000 7fc97933f700 1 -- 172.21.15.143:0/3825828124 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9740c9290 con 0x7fc95c003a30 2023-11-08T19:27:10.249 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.232+0000 7fc95b7fe700 1 -- 172.21.15.143:0/3825828124 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc968004340 con 0x7fc95c003a30 2023-11-08T19:27:10.249 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.232+0000 7fc95b7fe700 1 --2- 172.21.15.143:0/3825828124 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc96006af00 0x7fc96006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:10.249 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.232+0000 7fc95b7fe700 1 -- 172.21.15.143:0/3825828124 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6828+0+0 (secure 0 0 0) 0x7fc96808aaf0 con 0x7fc95c003a30 2023-11-08T19:27:10.249 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.232+0000 7fc9727fc700 1 --2- 172.21.15.143:0/3825828124 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc96006af00 0x7fc96006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:10.249 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.233+0000 7fc9727fc700 1 --2- 172.21.15.143:0/3825828124 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc96006af00 0x7fc96006d3c0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fc964000c00 tx=0x7fc964009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:10.250 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.237+0000 7fc95b7fe700 1 -- 172.21.15.143:0/3825828124 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc9680557d0 con 0x7fc95c003a30 2023-11-08T19:27:10.432 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.430+0000 7fc97933f700 1 -- 172.21.15.143:0/3825828124 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc97415e020 con 0x7fc95c003a30 2023-11-08T19:27:10.433 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.432+0000 7fc95b7fe700 1 -- 172.21.15.143:0/3825828124 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) v1 ==== 74+0+21891 (secure 0 0 0) 0x7fc97415e020 con 0x7fc95c003a30 2023-11-08T19:27:10.433 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:27:10.434 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":66,"fsid":"968483af-77fc-4068-ad7c-5c2a1a4613d7","created":"2023-11-08T19:24:32.826484+0000","modified":"2023-11-08T19:27:09.174117+0000","last_up_change":"2023-11-08T19:24:43.573880+0000","last_in_change":"2023-11-08T19:24:35.314525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:44.551530+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"snap-test-pool","create_time":"2023-11-08T19:26:02.874384+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":36,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-11-08T19:26:55.754227+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-11-08T19:26:56.486110+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"test_data_pool","create_time":"2023-11-08T19:27:05.601949+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":66,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_metadata_pool","create_time":"2023-11-08T19:27:06.536369+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bae98ec1-d94e-403d-a220-a522a53ea6f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6809","nonce":1487830566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6811","nonce":1487830566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6815","nonce":1487830566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6813","nonce":1487830566}]},"public_addr":"172.21.15.143:6809/1487830566","cluster_addr":"172.21.15.143:6811/1487830566","heartbeat_back_addr":"172.21.15.143:6815/1487830566","heartbeat_front_addr":"172.21.15.143:6813/1487830566","state":["exists","up"]},{"osd":1,"uuid":"5cb6538a-1ccb-4322-8633-09f66af0b7e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6801","nonce":787341397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6803","nonce":787341397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6807","nonce":787341397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6805","nonce":787341397}]},"public_addr":"172.21.15.143:6801/787341397","cluster_addr":"172.21.15.143:6803/787341397","heartbeat_back_addr":"172.21.15.143:6807/787341397","heartbeat_front_addr":"172.21.15.143:6805/787341397","state":["exists","up"]},{"osd":2,"uuid":"cdbf522c-60ef-4d60-9653-8dcb529c101a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6822","nonce":2301325597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6826","nonce":2301325597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6831","nonce":2301325597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6829","nonce":2301325597}]},"public_addr":"172.21.15.143:6822/2301325597","cluster_addr":"172.21.15.143:6826/2301325597","heartbeat_back_addr":"172.21.15.143:6831/2301325597","heartbeat_front_addr":"172.21.15.143:6829/2301325597","state":["exists","up"]},{"osd":3,"uuid":"4e2783cf-ebb9-4dd8-8245-bc1d17ce4325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6817","nonce":2021504428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6819","nonce":2021504428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6827","nonce":2021504428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6823","nonce":2021504428}]},"public_addr":"172.21.15.143:6817/2021504428","cluster_addr":"172.21.15.143:6819/2021504428","heartbeat_back_addr":"172.21.15.143:6827/2021504428","heartbeat_front_addr":"172.21.15.143:6823/2021504428","state":["exists","up"]},{"osd":4,"uuid":"691256fa-9b77-49b9-a346-7ae1adf0ca80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6801","nonce":1993380595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6803","nonce":1993380595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6807","nonce":1993380595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6805","nonce":1993380595}]},"public_addr":"172.21.15.174:6801/1993380595","cluster_addr":"172.21.15.174:6803/1993380595","heartbeat_back_addr":"172.21.15.174:6807/1993380595","heartbeat_front_addr":"172.21.15.174:6805/1993380595","state":["exists","up"]},{"osd":5,"uuid":"756fc352-48c9-46ce-b52e-0541fe6f8525","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6809","nonce":1777923303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6811","nonce":1777923303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6815","nonce":1777923303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6813","nonce":1777923303}]},"public_addr":"172.21.15.174:6809/1777923303","cluster_addr":"172.21.15.174:6811/1777923303","heartbeat_back_addr":"172.21.15.174:6815/1777923303","heartbeat_front_addr":"172.21.15.174:6813/1777923303","state":["exists","up"]},{"osd":6,"uuid":"fab2ac2b-ed97-4cde-9825-b480eabf8b8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6825","nonce":2755130316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6827","nonce":2755130316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6831","nonce":2755130316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6829","nonce":2755130316}]},"public_addr":"172.21.15.174:6825/2755130316","cluster_addr":"172.21.15.174:6827/2755130316","heartbeat_back_addr":"172.21.15.174:6831/2755130316","heartbeat_front_addr":"172.21.15.174:6829/2755130316","state":["exists","up"]},{"osd":7,"uuid":"3de9b797-2ab6-4d41-a863-7e7d638fd703","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6817","nonce":2847141411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6819","nonce":2847141411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6823","nonce":2847141411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6821","nonce":2847141411}]},"public_addr":"172.21.15.174:6817/2847141411","cluster_addr":"172.21.15.174:6819/2847141411","heartbeat_back_addr":"172.21.15.174:6823/2847141411","heartbeat_front_addr":"172.21.15.174:6821/2847141411","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.477076+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.496890+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.539305+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.454673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.538881+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.521294+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.520712+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.540404+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":11,"snaps":[{"begin":1,"length":1}]}],"new_removed_snaps":[{"pool":11,"snaps":[{"begin":1,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:27:10.435 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.434+0000 7fc97933f700 1 -- 172.21.15.143:0/3825828124 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc96006af00 msgr2=0x7fc96006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:10.436 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.434+0000 7fc97933f700 1 --2- 172.21.15.143:0/3825828124 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc96006af00 0x7fc96006d3c0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fc964000c00 tx=0x7fc964009040 comp rx=0 tx=0).stop 2023-11-08T19:27:10.436 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.434+0000 7fc97933f700 1 -- 172.21.15.143:0/3825828124 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc95c003a30 msgr2=0x7fc97415d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:10.436 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.434+0000 7fc97933f700 1 --2- 172.21.15.143:0/3825828124 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc95c003a30 0x7fc97415d950 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fc95c0047b0 tx=0x7fc9680047c0 comp rx=0 tx=0).stop 2023-11-08T19:27:10.436 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.434+0000 7fc97933f700 1 -- 172.21.15.143:0/3825828124 shutdown_connections 2023-11-08T19:27:10.436 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.434+0000 7fc97933f700 1 --2- 172.21.15.143:0/3825828124 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc96006af00 0x7fc96006d3c0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:10.437 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.434+0000 7fc97933f700 1 --2- 172.21.15.143:0/3825828124 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc97415c2e0 0x7fc9740af8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:10.437 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.434+0000 7fc97933f700 1 --2- 172.21.15.143:0/3825828124 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc95c003a30 0x7fc97415d950 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:10.437 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.434+0000 7fc97933f700 1 --2- 172.21.15.143:0/3825828124 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc97415be60 0x7fc9740af390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:10.437 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.434+0000 7fc97933f700 1 -- 172.21.15.143:0/3825828124 >> 172.21.15.143:0/3825828124 conn(0x7fc974099ab0 msgr2=0x7fc974159c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:10.437 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.434+0000 7fc97933f700 1 -- 172.21.15.143:0/3825828124 shutdown_connections 2023-11-08T19:27:10.438 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.434+0000 7fc97933f700 1 -- 172.21.15.143:0/3825828124 wait complete. 2023-11-08T19:27:10.448 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T19:24:44.551530+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'snap-test-pool', 'create_time': '2023-11-08T19:26:02.874384+0000', 'flags': 16385, 'flags_names': 'hashpspool,pool_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '36', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'pool', 'snap_seq': 2, 'snap_epoch': 36, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-08T19:26:55.754227+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '61', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2023-11-08T19:26:56.486110+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '62', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'test_data_pool', 'create_time': '2023-11-08T19:27:05.601949+0000', 'flags': 16385, 'flags_names': 'hashpspool,pool_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'pool', 'snap_seq': 2, 'snap_epoch': 66, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'test_metadata_pool', 'create_time': '2023-11-08T19:27:06.536369+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-08T19:27:10.449 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T19:27:10.573 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.571+0000 7f4b0cae3700 1 Processor -- start 2023-11-08T19:27:10.574 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.571+0000 7f4b0cae3700 1 -- start start 2023-11-08T19:27:10.583 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.572+0000 7f4b0cae3700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f4b0815df20 0x7f4b080b25f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:10.584 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.572+0000 7f4b0cae3700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4b0806bd20 0x7f4b080d19c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:10.584 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.572+0000 7f4b0cae3700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b080b2b30 con 0x7f4b080d1f00 2023-11-08T19:27:10.585 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.572+0000 7f4b0cae3700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b080d2390 con 0x7f4b0815df20 2023-11-08T19:27:10.585 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.572+0000 7f4b0cae3700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b0806c2c0 con 0x7f4b0806bd20 2023-11-08T19:27:10.585 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.573+0000 7f4b0659c700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f4b0815df20 0x7f4b080b25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:10.586 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.573+0000 7f4b0659c700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f4b0815df20 0x7f4b080b25f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52890/0 (socket says 172.21.15.143:52890) 2023-11-08T19:27:10.586 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.573+0000 7f4b0659c700 1 -- 172.21.15.143:0/386532188 learned_addr learned my addr 172.21.15.143:0/386532188 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:10.587 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.573+0000 7f4b05d9b700 1 --2- 172.21.15.143:0/386532188 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4b0806bd20 0x7f4b080d19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:10.587 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.573+0000 7f4b0659c700 1 -- 172.21.15.143:0/386532188 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4b0806bd20 msgr2=0x7f4b080d19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:10.587 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.573+0000 7f4b0659c700 1 --2- 172.21.15.143:0/386532188 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4b0806bd20 0x7f4b080d19c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:10.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.573+0000 7f4b0659c700 1 -- 172.21.15.143:0/386532188 >> v1:172.21.15.174:6789/0 conn(0x7f4b080d1f00 legacy=0x7f4b080ce2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:10.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.573+0000 7f4b0659c700 1 -- 172.21.15.143:0/386532188 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b0806c5a0 con 0x7f4b0815df20 2023-11-08T19:27:10.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.573+0000 7f4b0659c700 1 --2- 172.21.15.143:0/386532188 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4b0815df20 0x7f4b080b25f0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f4afc0098f0 tx=0x7f4afc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa0f1480dfdcabeb server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:10.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.573+0000 7f4b0559a700 1 -- 172.21.15.143:0/386532188 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4afc00b8a0 con 0x7f4b0815df20 2023-11-08T19:27:10.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.573+0000 7f4b0559a700 1 -- 172.21.15.143:0/386532188 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4afc00ba00 con 0x7f4b0815df20 2023-11-08T19:27:10.590 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.573+0000 7f4b0559a700 1 -- 172.21.15.143:0/386532188 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4afc00bd90 con 0x7f4b0815df20 2023-11-08T19:27:10.590 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.574+0000 7f4b0cae3700 1 -- 172.21.15.143:0/386532188 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4b0815df20 msgr2=0x7f4b080b25f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:10.591 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.574+0000 7f4b0cae3700 1 --2- 172.21.15.143:0/386532188 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4b0815df20 0x7f4b080b25f0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f4afc0098f0 tx=0x7f4afc009c00 comp rx=0 tx=0).stop 2023-11-08T19:27:10.591 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.574+0000 7f4b0cae3700 1 -- 172.21.15.143:0/386532188 shutdown_connections 2023-11-08T19:27:10.591 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.574+0000 7f4b0cae3700 1 --2- 172.21.15.143:0/386532188 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4b0815df20 0x7f4b080b25f0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:10.592 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.574+0000 7f4b0cae3700 1 --2- 172.21.15.143:0/386532188 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4b0806bd20 0x7f4b080d19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:10.592 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.574+0000 7f4b0cae3700 1 -- 172.21.15.143:0/386532188 >> 172.21.15.143:0/386532188 conn(0x7f4b0808b000 msgr2=0x7f4b0809ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:10.593 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.574+0000 7f4b0cae3700 1 -- 172.21.15.143:0/386532188 shutdown_connections 2023-11-08T19:27:10.593 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.574+0000 7f4b0cae3700 1 -- 172.21.15.143:0/386532188 wait complete. 2023-11-08T19:27:10.593 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.575+0000 7f4b0cae3700 1 Processor -- start 2023-11-08T19:27:10.594 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.575+0000 7f4b0cae3700 1 -- start start 2023-11-08T19:27:10.594 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.576+0000 7f4b0cae3700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4b0806bd20 0x7f4b080af000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:10.595 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.576+0000 7f4b0cae3700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4b080d1f00 0x7f4b0814ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:10.595 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.576+0000 7f4b0cae3700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4b0815df20 0x7f4b080cac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:10.595 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.576+0000 7f4b0cae3700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b080cba90 con 0x7f4b080d1f00 2023-11-08T19:27:10.596 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.576+0000 7f4b0cae3700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b080aff80 con 0x7f4b0815df20 2023-11-08T19:27:10.596 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.576+0000 7f4b0cae3700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b080af6e0 con 0x7f4b0806bd20 2023-11-08T19:27:10.596 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.576+0000 7f4b05d9b700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4b080d1f00 0x7f4b0814ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:10.596 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.576+0000 7f4b05d9b700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4b080d1f00 0x7f4b0814ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52898/0 (socket says 172.21.15.143:52898) 2023-11-08T19:27:10.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.576+0000 7f4b05d9b700 1 -- 172.21.15.143:0/2498068732 learned_addr learned my addr 172.21.15.143:0/2498068732 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:10.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.576+0000 7f4b0659c700 1 --2- 172.21.15.143:0/2498068732 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4b0806bd20 0x7f4b080af000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:10.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.576+0000 7f4b06d9d700 1 --2- 172.21.15.143:0/2498068732 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4b0815df20 0x7f4b080cac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:10.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.576+0000 7f4b05d9b700 1 -- 172.21.15.143:0/2498068732 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4b0806bd20 msgr2=0x7f4b080af000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:10.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.576+0000 7f4b05d9b700 1 --2- 172.21.15.143:0/2498068732 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4b0806bd20 0x7f4b080af000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:10.598 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.576+0000 7f4b05d9b700 1 -- 172.21.15.143:0/2498068732 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4b0815df20 msgr2=0x7f4b080cac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:10.598 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.576+0000 7f4b05d9b700 1 --2- 172.21.15.143:0/2498068732 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4b0815df20 0x7f4b080cac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:10.598 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.576+0000 7f4b05d9b700 1 -- 172.21.15.143:0/2498068732 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4afc009550 con 0x7f4b080d1f00 2023-11-08T19:27:10.598 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.576+0000 7f4b05d9b700 1 --2- 172.21.15.143:0/2498068732 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4b080d1f00 0x7f4b0814ed70 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f4af800d800 tx=0x7f4af800db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:10.598 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.577+0000 7f4af6ffd700 1 -- 172.21.15.143:0/2498068732 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af80126b0 con 0x7f4b080d1f00 2023-11-08T19:27:10.599 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.577+0000 7f4af6ffd700 1 -- 172.21.15.143:0/2498068732 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4af8012810 con 0x7f4b080d1f00 2023-11-08T19:27:10.599 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.577+0000 7f4b0cae3700 1 -- 172.21.15.143:0/2498068732 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b080af9c0 con 0x7f4b080d1f00 2023-11-08T19:27:10.599 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.577+0000 7f4af6ffd700 1 -- 172.21.15.143:0/2498068732 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af8012bc0 con 0x7f4b080d1f00 2023-11-08T19:27:10.599 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.577+0000 7f4b0cae3700 1 -- 172.21.15.143:0/2498068732 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b080c8c30 con 0x7f4b080d1f00 2023-11-08T19:27:10.599 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.578+0000 7f4b0cae3700 1 -- 172.21.15.143:0/2498068732 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b080c8e50 con 0x7f4b080d1f00 2023-11-08T19:27:10.600 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.578+0000 7f4af6ffd700 1 -- 172.21.15.143:0/2498068732 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4af8012d20 con 0x7f4b080d1f00 2023-11-08T19:27:10.600 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.578+0000 7f4af6ffd700 1 --2- 172.21.15.143:0/2498068732 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f4ae806af00 0x7f4ae806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:10.600 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.578+0000 7f4af6ffd700 1 -- 172.21.15.143:0/2498068732 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6828+0+0 (secure 0 0 0) 0x7f4af8089a50 con 0x7f4b080d1f00 2023-11-08T19:27:10.600 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.579+0000 7f4b0659c700 1 --2- 172.21.15.143:0/2498068732 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f4ae806af00 0x7f4ae806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:10.600 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.579+0000 7f4b0659c700 1 --2- 172.21.15.143:0/2498068732 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f4ae806af00 0x7f4ae806d3c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f4afc009c40 tx=0x7f4afc008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:10.601 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.583+0000 7f4af6ffd700 1 -- 172.21.15.143:0/2498068732 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4af8054730 con 0x7f4b080d1f00 2023-11-08T19:27:10.773 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.772+0000 7f4b0cae3700 1 -- 172.21.15.143:0/2498068732 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4b0804eee0 con 0x7f4b080d1f00 2023-11-08T19:27:10.773 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.772+0000 7f4af6ffd700 1 -- 172.21.15.143:0/2498068732 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v66) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4af8057d50 con 0x7f4b080d1f00 2023-11-08T19:27:10.774 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2023-11-08T19:27:10.776 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.774+0000 7f4b0cae3700 1 -- 172.21.15.143:0/2498068732 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f4ae806af00 msgr2=0x7f4ae806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:10.776 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.774+0000 7f4b0cae3700 1 --2- 172.21.15.143:0/2498068732 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f4ae806af00 0x7f4ae806d3c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f4afc009c40 tx=0x7f4afc008030 comp rx=0 tx=0).stop 2023-11-08T19:27:10.777 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.774+0000 7f4b0cae3700 1 -- 172.21.15.143:0/2498068732 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4b080d1f00 msgr2=0x7f4b0814ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:10.777 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.774+0000 7f4b0cae3700 1 --2- 172.21.15.143:0/2498068732 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4b080d1f00 0x7f4b0814ed70 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f4af800d800 tx=0x7f4af800db10 comp rx=0 tx=0).stop 2023-11-08T19:27:10.777 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.774+0000 7f4b0cae3700 1 -- 172.21.15.143:0/2498068732 shutdown_connections 2023-11-08T19:27:10.778 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.774+0000 7f4b0cae3700 1 --2- 172.21.15.143:0/2498068732 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f4ae806af00 0x7f4ae806d3c0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:10.778 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.774+0000 7f4b0cae3700 1 --2- 172.21.15.143:0/2498068732 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4b0806bd20 0x7f4b080af000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:10.778 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.774+0000 7f4b0cae3700 1 --2- 172.21.15.143:0/2498068732 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4b0815df20 0x7f4b080cac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:10.779 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.774+0000 7f4b0cae3700 1 --2- 172.21.15.143:0/2498068732 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4b080d1f00 0x7f4b0814ed70 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:10.779 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.774+0000 7f4b0cae3700 1 -- 172.21.15.143:0/2498068732 >> 172.21.15.143:0/2498068732 conn(0x7f4b0808b000 msgr2=0x7f4b080b32c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:10.779 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.774+0000 7f4b0cae3700 1 -- 172.21.15.143:0/2498068732 shutdown_connections 2023-11-08T19:27:10.780 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.774+0000 7f4b0cae3700 1 -- 172.21.15.143:0/2498068732 wait complete. 2023-11-08T19:27:10.787 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get snap-test-pool pg_num 2023-11-08T19:27:10.910 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.909+0000 7f1feb8b5700 1 Processor -- start 2023-11-08T19:27:10.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.909+0000 7f1feb8b5700 1 -- start start 2023-11-08T19:27:10.920 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.910+0000 7f1feb8b5700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f1fe406bd20 0x7f1fe40d01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:10.920 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.910+0000 7f1feb8b5700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fe40cc2a0 con 0x7f1fe40b61e0 2023-11-08T19:27:10.920 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.910+0000 7f1feb8b5700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fe406c1b0 con 0x7f1fe415a950 2023-11-08T19:27:10.920 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.910+0000 7f1feb8b5700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fe40cc130 con 0x7f1fe406bd20 2023-11-08T19:27:10.921 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.910+0000 7f1fe9e52700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f1fe40b61e0 0x7f1fe40cc730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:49062/0 (socket says 172.21.15.143:49062) 2023-11-08T19:27:10.921 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.910+0000 7f1fe9e52700 1 -- 172.21.15.143:0/2733498 learned_addr learned my addr 172.21.15.143:0/2733498 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:27:10.921 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.910+0000 7f1fe8e50700 1 --2- 172.21.15.143:0/2733498 >> v2:172.21.15.174:3300/0 conn(0x7f1fe406bd20 0x7f1fe40d01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:10.921 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.910+0000 7f1fdbfff700 1 -- 172.21.15.143:0/2733498 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 548881760 0 0) 0x7f1fe40cc2a0 con 0x7f1fe40b61e0 2023-11-08T19:27:10.921 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.910+0000 7f1fdbfff700 1 -- 172.21.15.143:0/2733498 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1fd00035b0 con 0x7f1fe40b61e0 2023-11-08T19:27:10.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.910+0000 7f1fdbfff700 1 -- 172.21.15.143:0/2733498 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1705927445 0 0) 0x7f1fd00035b0 con 0x7f1fe40b61e0 2023-11-08T19:27:10.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.910+0000 7f1fdbfff700 1 -- 172.21.15.143:0/2733498 >> v2:172.21.15.174:3300/0 conn(0x7f1fe406bd20 msgr2=0x7f1fe40d01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:10.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.910+0000 7f1fdbfff700 1 --2- 172.21.15.143:0/2733498 >> v2:172.21.15.174:3300/0 conn(0x7f1fe406bd20 0x7f1fe40d01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:10.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.910+0000 7f1fdbfff700 1 -- 172.21.15.143:0/2733498 >> v1:172.21.15.174:6789/0 conn(0x7f1fe415a950 legacy=0x7f1fe4072370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:10.923 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.910+0000 7f1fdbfff700 1 -- 172.21.15.143:0/2733498 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fe406c490 con 0x7f1fe40b61e0 2023-11-08T19:27:10.923 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.910+0000 7f1fdbfff700 1 -- 172.21.15.143:0/2733498 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4147756303 0 0) 0x7f1fdc0019f0 con 0x7f1fe40b61e0 2023-11-08T19:27:10.923 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.910+0000 7f1fdbfff700 1 -- 172.21.15.143:0/2733498 >> v1:172.21.15.143:6789/0 conn(0x7f1fe40b61e0 legacy=0x7f1fe40cc730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:10.923 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.911+0000 7f1fdbfff700 1 --2- 172.21.15.143:0/2733498 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1fd0003a00 0x7f1fd0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:10.923 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.911+0000 7f1fdbfff700 1 --2- 172.21.15.143:0/2733498 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1fd0004850 0x7f1fd0004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:10.924 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.911+0000 7f1fdbfff700 1 --2- 172.21.15.143:0/2733498 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1fd0005210 0x7f1fd000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:10.924 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.911+0000 7f1fdbfff700 1 -- 172.21.15.143:0/2733498 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd000c080 con 0x7f1fd0003a00 2023-11-08T19:27:10.924 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.911+0000 7f1fdbfff700 1 -- 172.21.15.143:0/2733498 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd000c1f0 con 0x7f1fd0004850 2023-11-08T19:27:10.924 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.911+0000 7f1fdbfff700 1 -- 172.21.15.143:0/2733498 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd000c3b0 con 0x7f1fd0005210 2023-11-08T19:27:10.924 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.911+0000 7f1fe9651700 1 --2- 172.21.15.143:0/2733498 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1fd0003a00 0x7f1fd0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:10.924 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.911+0000 7f1fe8e50700 1 --2- 172.21.15.143:0/2733498 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1fd0004850 0x7f1fd0004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:10.925 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.911+0000 7f1fe9e52700 1 --2- 172.21.15.143:0/2733498 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1fd0005210 0x7f1fd000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:10.925 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.911+0000 7f1fe9651700 1 -- 172.21.15.143:0/2733498 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1fd0005210 msgr2=0x7f1fd000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:10.925 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.911+0000 7f1fe9651700 1 --2- 172.21.15.143:0/2733498 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1fd0005210 0x7f1fd000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:10.925 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.911+0000 7f1fe9651700 1 -- 172.21.15.143:0/2733498 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1fd0004850 msgr2=0x7f1fd0004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:10.925 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.911+0000 7f1fe9651700 1 --2- 172.21.15.143:0/2733498 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1fd0004850 0x7f1fd0004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:10.926 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.911+0000 7f1fe9651700 1 -- 172.21.15.143:0/2733498 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fd00035b0 con 0x7f1fd0003a00 2023-11-08T19:27:10.926 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.911+0000 7f1fe9e52700 1 --2- 172.21.15.143:0/2733498 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1fd0005210 0x7f1fd000ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:27:10.926 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.911+0000 7f1fe9e52700 1 -- 172.21.15.143:0/2733498 reap_dead start 2023-11-08T19:27:10.926 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.911+0000 7f1fe9651700 1 --2- 172.21.15.143:0/2733498 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1fd0003a00 0x7f1fd0003e20 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f1fe00017c0 tx=0x7f1fd0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4885f2128198c1d6 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:10.927 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.911+0000 7f1fdbfff700 1 -- 172.21.15.143:0/2733498 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe00052d0 con 0x7f1fd0003a00 2023-11-08T19:27:10.927 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.911+0000 7f1fdbfff700 1 -- 172.21.15.143:0/2733498 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fe0005430 con 0x7f1fd0003a00 2023-11-08T19:27:10.927 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.911+0000 7f1fdbfff700 1 -- 172.21.15.143:0/2733498 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe00038c0 con 0x7f1fd0003a00 2023-11-08T19:27:10.927 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.912+0000 7f1feb8b5700 1 -- 172.21.15.143:0/2733498 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1fd0003a00 msgr2=0x7f1fd0003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:10.927 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.912+0000 7f1feb8b5700 1 --2- 172.21.15.143:0/2733498 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1fd0003a00 0x7f1fd0003e20 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f1fe00017c0 tx=0x7f1fd0005880 comp rx=0 tx=0).stop 2023-11-08T19:27:10.928 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.912+0000 7f1feb8b5700 1 -- 172.21.15.143:0/2733498 shutdown_connections 2023-11-08T19:27:10.928 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.912+0000 7f1feb8b5700 1 --2- 172.21.15.143:0/2733498 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1fd0003a00 0x7f1fd0003e20 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:10.928 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.912+0000 7f1feb8b5700 1 -- 172.21.15.143:0/2733498 >> 172.21.15.143:0/2733498 conn(0x7f1fe40cd2e0 msgr2=0x7f1fe40cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:10.928 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.912+0000 7f1feb8b5700 1 -- 172.21.15.143:0/2733498 shutdown_connections 2023-11-08T19:27:10.928 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.912+0000 7f1feb8b5700 1 -- 172.21.15.143:0/2733498 wait complete. 2023-11-08T19:27:10.929 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.913+0000 7f1feb8b5700 1 Processor -- start 2023-11-08T19:27:10.929 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.913+0000 7f1feb8b5700 1 -- start start 2023-11-08T19:27:10.929 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.914+0000 7f1feb8b5700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1fd0003a00 0x7f1fe41576f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:10.929 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.914+0000 7f1feb8b5700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1fe40c7930 0x7f1fe40aa1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:10.929 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.914+0000 7f1feb8b5700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1fe40aa6e0 0x7f1fe41f3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:10.930 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.914+0000 7f1feb8b5700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe4157c30 con 0x7f1fe40aa6e0 2023-11-08T19:27:10.930 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.914+0000 7f1feb8b5700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe40a84e0 con 0x7f1fd0003a00 2023-11-08T19:27:10.930 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.914+0000 7f1feb8b5700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe40aab60 con 0x7f1fe40c7930 2023-11-08T19:27:10.930 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.914+0000 7f1fe9e52700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1fe40aa6e0 0x7f1fe41f3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:10.930 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.914+0000 7f1fe9e52700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1fe40aa6e0 0x7f1fe41f3b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52920/0 (socket says 172.21.15.143:52920) 2023-11-08T19:27:10.931 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.914+0000 7f1fe9e52700 1 -- 172.21.15.143:0/4040284663 learned_addr learned my addr 172.21.15.143:0/4040284663 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:10.931 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.914+0000 7f1fe9651700 1 --2- 172.21.15.143:0/4040284663 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1fd0003a00 0x7f1fe41576f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:10.931 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.914+0000 7f1fe8e50700 1 --2- 172.21.15.143:0/4040284663 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1fe40c7930 0x7f1fe40aa1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:10.931 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.914+0000 7f1fe9e52700 1 -- 172.21.15.143:0/4040284663 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1fe40c7930 msgr2=0x7f1fe40aa1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:10.931 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.914+0000 7f1fe9e52700 1 --2- 172.21.15.143:0/4040284663 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1fe40c7930 0x7f1fe40aa1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:10.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.914+0000 7f1fe9e52700 1 -- 172.21.15.143:0/4040284663 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1fd0003a00 msgr2=0x7f1fe41576f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:10.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.914+0000 7f1fe9e52700 1 --2- 172.21.15.143:0/4040284663 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1fd0003a00 0x7f1fe41576f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:10.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.914+0000 7f1fe9e52700 1 -- 172.21.15.143:0/4040284663 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fe40ca7c0 con 0x7f1fe40aa6e0 2023-11-08T19:27:10.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.914+0000 7f1fe9e52700 1 --2- 172.21.15.143:0/4040284663 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1fe40aa6e0 0x7f1fe41f3b70 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f1fe40cbbc0 tx=0x7f1fdc00df30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:10.933 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.914+0000 7f1fd9ffb700 1 -- 172.21.15.143:0/4040284663 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fdc00bb10 con 0x7f1fe40aa6e0 2023-11-08T19:27:10.933 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.914+0000 7f1fd9ffb700 1 -- 172.21.15.143:0/4040284663 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fdc00bc70 con 0x7f1fe40aa6e0 2023-11-08T19:27:10.933 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.914+0000 7f1fd9ffb700 1 -- 172.21.15.143:0/4040284663 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fdc007030 con 0x7f1fe40aa6e0 2023-11-08T19:27:10.933 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.914+0000 7f1feb8b5700 1 -- 172.21.15.143:0/4040284663 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fe41f48a0 con 0x7f1fe40aa6e0 2023-11-08T19:27:10.933 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.915+0000 7f1feb8b5700 1 -- 172.21.15.143:0/4040284663 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fe40a89d0 con 0x7f1fe40aa6e0 2023-11-08T19:27:10.934 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.916+0000 7f1feb8b5700 1 -- 172.21.15.143:0/4040284663 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fe404eee0 con 0x7f1fe40aa6e0 2023-11-08T19:27:10.934 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.916+0000 7f1fd9ffb700 1 -- 172.21.15.143:0/4040284663 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1fdc007190 con 0x7f1fe40aa6e0 2023-11-08T19:27:10.934 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.916+0000 7f1fd9ffb700 1 --2- 172.21.15.143:0/4040284663 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f1fc806af00 0x7f1fc806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:10.934 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.916+0000 7f1fd9ffb700 1 -- 172.21.15.143:0/4040284663 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6828+0+0 (secure 0 0 0) 0x7f1fdc089610 con 0x7f1fe40aa6e0 2023-11-08T19:27:10.934 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.916+0000 7f1fe9651700 1 --2- 172.21.15.143:0/4040284663 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f1fc806af00 0x7f1fc806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:10.935 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.917+0000 7f1fe9651700 1 --2- 172.21.15.143:0/4040284663 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f1fc806af00 0x7f1fc806d3c0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f1fe0015040 tx=0x7f1fe0009030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:10.935 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:10.921+0000 7f1fd9ffb700 1 -- 172.21.15.143:0/4040284663 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1fdc054260 con 0x7f1fe40aa6e0 2023-11-08T19:27:11.110 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.109+0000 7f1feb8b5700 1 -- 172.21.15.143:0/4040284663 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"} v 0) v1 -- 0x7f1fe40ca7c0 con 0x7f1fe40aa6e0 2023-11-08T19:27:11.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.109+0000 7f1fd9ffb700 1 -- 172.21.15.143:0/4040284663 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"}]=0 v66) v1 ==== 103+0+11 (secure 0 0 0) 0x7f1fe40ca7c0 con 0x7f1fe40aa6e0 2023-11-08T19:27:11.111 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2023-11-08T19:27:11.113 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.111+0000 7f1feb8b5700 1 -- 172.21.15.143:0/4040284663 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f1fc806af00 msgr2=0x7f1fc806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:11.113 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.111+0000 7f1feb8b5700 1 --2- 172.21.15.143:0/4040284663 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f1fc806af00 0x7f1fc806d3c0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f1fe0015040 tx=0x7f1fe0009030 comp rx=0 tx=0).stop 2023-11-08T19:27:11.113 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.111+0000 7f1feb8b5700 1 -- 172.21.15.143:0/4040284663 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1fe40aa6e0 msgr2=0x7f1fe41f3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:11.114 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.111+0000 7f1feb8b5700 1 --2- 172.21.15.143:0/4040284663 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1fe40aa6e0 0x7f1fe41f3b70 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f1fe40cbbc0 tx=0x7f1fdc00df30 comp rx=0 tx=0).stop 2023-11-08T19:27:11.114 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.111+0000 7f1feb8b5700 1 -- 172.21.15.143:0/4040284663 shutdown_connections 2023-11-08T19:27:11.114 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.111+0000 7f1feb8b5700 1 --2- 172.21.15.143:0/4040284663 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f1fc806af00 0x7f1fc806d3c0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:11.114 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.111+0000 7f1feb8b5700 1 --2- 172.21.15.143:0/4040284663 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1fe40aa6e0 0x7f1fe41f3b70 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:11.115 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.111+0000 7f1feb8b5700 1 --2- 172.21.15.143:0/4040284663 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1fd0003a00 0x7f1fe41576f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:11.115 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.111+0000 7f1feb8b5700 1 --2- 172.21.15.143:0/4040284663 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1fe40c7930 0x7f1fe40aa1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:11.115 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.111+0000 7f1feb8b5700 1 -- 172.21.15.143:0/4040284663 >> 172.21.15.143:0/4040284663 conn(0x7f1fe40cd2e0 msgr2=0x7f1fe40b4b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:11.115 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.112+0000 7f1feb8b5700 1 -- 172.21.15.143:0/4040284663 shutdown_connections 2023-11-08T19:27:11.115 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.112+0000 7f1feb8b5700 1 -- 172.21.15.143:0/4040284663 wait complete. 2023-11-08T19:27:11.125 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-08T19:27:11.249 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.248+0000 7f0aadf1f700 1 Processor -- start 2023-11-08T19:27:11.250 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.248+0000 7f0aadf1f700 1 -- start start 2023-11-08T19:27:11.252 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.249+0000 7f0aadf1f700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f0aa815dba0 0x7f0aa806c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:11.252 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.249+0000 7f0aadf1f700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0aa8161890 con 0x7f0aa80b22e0 2023-11-08T19:27:11.252 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.249+0000 7f0aadf1f700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0aa806b740 con 0x7f0aa806b240 2023-11-08T19:27:11.252 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.249+0000 7f0aadf1f700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0aa806ba20 con 0x7f0aa815dba0 2023-11-08T19:27:11.253 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.249+0000 7f0aa77fe700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f0aa815dba0 0x7f0aa806c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:11.253 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.249+0000 7f0aa7fff700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f0aa80b22e0 0x7f0aa80b2700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:49082/0 (socket says 172.21.15.143:49082) 2023-11-08T19:27:11.253 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.249+0000 7f0aa7fff700 1 -- 172.21.15.143:0/3607360575 learned_addr learned my addr 172.21.15.143:0/3607360575 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:27:11.253 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.249+0000 7f0aa77fe700 1 -- 172.21.15.143:0/3607360575 >> v1:172.21.15.174:6789/0 conn(0x7f0aa806b240 legacy=0x7f0aa80caff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:11.253 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.249+0000 7f0aa77fe700 1 -- 172.21.15.143:0/3607360575 >> v1:172.21.15.143:6789/0 conn(0x7f0aa80b22e0 legacy=0x7f0aa80b2700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:11.254 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.249+0000 7f0aa77fe700 1 -- 172.21.15.143:0/3607360575 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0aa806bd00 con 0x7f0aa815dba0 2023-11-08T19:27:11.254 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.249+0000 7f0aa67fc700 1 -- 172.21.15.143:0/3607360575 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2458566078 0 0) 0x7f0aa8161890 con 0x7f0aa80b22e0 2023-11-08T19:27:11.254 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.249+0000 7f0aa77fe700 1 --2- 172.21.15.143:0/3607360575 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0aa815dba0 0x7f0aa806c610 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f0a98009840 tx=0x7f0a98009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e710bb185a971c16 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:11.254 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.249+0000 7f0aa67fc700 1 -- 172.21.15.143:0/3607360575 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a98004230 con 0x7f0aa815dba0 2023-11-08T19:27:11.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.249+0000 7f0aa67fc700 1 -- 172.21.15.143:0/3607360575 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a98004390 con 0x7f0aa815dba0 2023-11-08T19:27:11.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.249+0000 7f0aa67fc700 1 -- 172.21.15.143:0/3607360575 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a980036a0 con 0x7f0aa815dba0 2023-11-08T19:27:11.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.250+0000 7f0aadf1f700 1 -- 172.21.15.143:0/3607360575 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0aa815dba0 msgr2=0x7f0aa806c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:11.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.250+0000 7f0aadf1f700 1 --2- 172.21.15.143:0/3607360575 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0aa815dba0 0x7f0aa806c610 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f0a98009840 tx=0x7f0a98009b50 comp rx=0 tx=0).stop 2023-11-08T19:27:11.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.250+0000 7f0aadf1f700 1 -- 172.21.15.143:0/3607360575 shutdown_connections 2023-11-08T19:27:11.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.250+0000 7f0aadf1f700 1 --2- 172.21.15.143:0/3607360575 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0aa815dba0 0x7f0aa806c610 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:11.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.250+0000 7f0aadf1f700 1 -- 172.21.15.143:0/3607360575 >> 172.21.15.143:0/3607360575 conn(0x7f0aa80cd2e0 msgr2=0x7f0aa80cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:11.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.250+0000 7f0aadf1f700 1 -- 172.21.15.143:0/3607360575 shutdown_connections 2023-11-08T19:27:11.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.250+0000 7f0aadf1f700 1 -- 172.21.15.143:0/3607360575 wait complete. 2023-11-08T19:27:11.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.251+0000 7f0aadf1f700 1 Processor -- start 2023-11-08T19:27:11.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.251+0000 7f0aadf1f700 1 -- start start 2023-11-08T19:27:11.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.252+0000 7f0aadf1f700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0aa806b240 0x7f0aa815ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:11.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.252+0000 7f0aadf1f700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0aa80b22e0 0x7f0aa80b3e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:11.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.252+0000 7f0aadf1f700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0aa815dba0 0x7f0aa80d07a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:11.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.252+0000 7f0aadf1f700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa80b5f50 con 0x7f0aa806b240 2023-11-08T19:27:11.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.252+0000 7f0aadf1f700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa80c8680 con 0x7f0aa80b22e0 2023-11-08T19:27:11.261 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.252+0000 7f0aadf1f700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa815b350 con 0x7f0aa815dba0 2023-11-08T19:27:11.261 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.252+0000 7f0aa77fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0aa806b240 0x7f0aa815ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:11.261 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.252+0000 7f0aa77fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0aa806b240 0x7f0aa815ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52930/0 (socket says 172.21.15.143:52930) 2023-11-08T19:27:11.261 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.252+0000 7f0aa77fe700 1 -- 172.21.15.143:0/2748176401 learned_addr learned my addr 172.21.15.143:0/2748176401 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:11.261 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.252+0000 7f0aa6ffd700 1 --2- 172.21.15.143:0/2748176401 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0aa80b22e0 0x7f0aa80b3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:11.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.252+0000 7f0aa7fff700 1 --2- 172.21.15.143:0/2748176401 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0aa815dba0 0x7f0aa80d07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:11.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.252+0000 7f0aa77fe700 1 -- 172.21.15.143:0/2748176401 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0aa815dba0 msgr2=0x7f0aa80d07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:11.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.252+0000 7f0aa77fe700 1 --2- 172.21.15.143:0/2748176401 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0aa815dba0 0x7f0aa80d07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:11.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.252+0000 7f0aa77fe700 1 -- 172.21.15.143:0/2748176401 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0aa80b22e0 msgr2=0x7f0aa80b3e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:11.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.252+0000 7f0aa77fe700 1 --2- 172.21.15.143:0/2748176401 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0aa80b22e0 0x7f0aa80b3e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:11.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.252+0000 7f0aa77fe700 1 -- 172.21.15.143:0/2748176401 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a980094a0 con 0x7f0aa806b240 2023-11-08T19:27:11.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.252+0000 7f0aa7fff700 1 --2- 172.21.15.143:0/2748176401 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0aa815dba0 0x7f0aa80d07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:27:11.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.253+0000 7f0aa77fe700 1 --2- 172.21.15.143:0/2748176401 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0aa806b240 0x7f0aa815ae10 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f0a980090b0 tx=0x7f0a980047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:11.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.253+0000 7f0a8ffff700 1 -- 172.21.15.143:0/2748176401 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a98011630 con 0x7f0aa806b240 2023-11-08T19:27:11.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.253+0000 7f0a8ffff700 1 -- 172.21.15.143:0/2748176401 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a98011790 con 0x7f0aa806b240 2023-11-08T19:27:11.264 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.253+0000 7f0aadf1f700 1 -- 172.21.15.143:0/2748176401 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0aa80c88d0 con 0x7f0aa806b240 2023-11-08T19:27:11.264 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.253+0000 7f0a8ffff700 1 -- 172.21.15.143:0/2748176401 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a98011b40 con 0x7f0aa806b240 2023-11-08T19:27:11.264 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.253+0000 7f0aadf1f700 1 -- 172.21.15.143:0/2748176401 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0aa80c8e90 con 0x7f0aa806b240 2023-11-08T19:27:11.264 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.254+0000 7f0aadf1f700 1 -- 172.21.15.143:0/2748176401 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0aa804eee0 con 0x7f0aa806b240 2023-11-08T19:27:11.264 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.254+0000 7f0a8ffff700 1 -- 172.21.15.143:0/2748176401 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0a98011ca0 con 0x7f0aa806b240 2023-11-08T19:27:11.265 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.254+0000 7f0a8ffff700 1 --2- 172.21.15.143:0/2748176401 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f0a9406af00 0x7f0a9406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:11.265 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.254+0000 7f0a8ffff700 1 -- 172.21.15.143:0/2748176401 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6828+0+0 (secure 0 0 0) 0x7f0a9808b2f0 con 0x7f0aa806b240 2023-11-08T19:27:11.265 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.255+0000 7f0aa6ffd700 1 --2- 172.21.15.143:0/2748176401 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f0a9406af00 0x7f0a9406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:11.265 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.255+0000 7f0aa6ffd700 1 --2- 172.21.15.143:0/2748176401 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f0a9406af00 0x7f0a9406d3c0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f0aa80b6510 tx=0x7f0a9c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:11.266 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.259+0000 7f0a8ffff700 1 -- 172.21.15.143:0/2748176401 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0a980560f0 con 0x7f0aa806b240 2023-11-08T19:27:11.451 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.450+0000 7f0aadf1f700 1 -- 172.21.15.143:0/2748176401 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0aa80b67e0 con 0x7f0aa806b240 2023-11-08T19:27:11.452 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.450+0000 7f0a8ffff700 1 -- 172.21.15.143:0/2748176401 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v66) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0a98059710 con 0x7f0aa806b240 2023-11-08T19:27:11.452 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2023-11-08T19:27:11.454 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.452+0000 7f0aadf1f700 1 -- 172.21.15.143:0/2748176401 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f0a9406af00 msgr2=0x7f0a9406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:11.454 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.452+0000 7f0aadf1f700 1 --2- 172.21.15.143:0/2748176401 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f0a9406af00 0x7f0a9406d3c0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f0aa80b6510 tx=0x7f0a9c009040 comp rx=0 tx=0).stop 2023-11-08T19:27:11.454 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.452+0000 7f0aadf1f700 1 -- 172.21.15.143:0/2748176401 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0aa806b240 msgr2=0x7f0aa815ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:11.455 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.452+0000 7f0aadf1f700 1 --2- 172.21.15.143:0/2748176401 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0aa806b240 0x7f0aa815ae10 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f0a980090b0 tx=0x7f0a980047c0 comp rx=0 tx=0).stop 2023-11-08T19:27:11.455 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.452+0000 7f0aadf1f700 1 -- 172.21.15.143:0/2748176401 shutdown_connections 2023-11-08T19:27:11.455 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.452+0000 7f0aadf1f700 1 --2- 172.21.15.143:0/2748176401 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f0a9406af00 0x7f0a9406d3c0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:11.455 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.452+0000 7f0aadf1f700 1 --2- 172.21.15.143:0/2748176401 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0aa815dba0 0x7f0aa80d07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:11.455 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.452+0000 7f0aadf1f700 1 --2- 172.21.15.143:0/2748176401 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0aa806b240 0x7f0aa815ae10 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:11.455 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.452+0000 7f0aadf1f700 1 --2- 172.21.15.143:0/2748176401 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0aa80b22e0 0x7f0aa80b3e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:11.456 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.452+0000 7f0aadf1f700 1 -- 172.21.15.143:0/2748176401 >> 172.21.15.143:0/2748176401 conn(0x7f0aa80cd2e0 msgr2=0x7f0aa815ccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:11.456 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.452+0000 7f0aadf1f700 1 -- 172.21.15.143:0/2748176401 shutdown_connections 2023-11-08T19:27:11.456 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.452+0000 7f0aadf1f700 1 -- 172.21.15.143:0/2748176401 wait complete. 2023-11-08T19:27:11.466 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-08T19:27:11.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.587+0000 7fe4f1a6e700 1 Processor -- start 2023-11-08T19:27:11.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.587+0000 7fe4f1a6e700 1 -- start start 2023-11-08T19:27:11.591 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.588+0000 7fe4f1a6e700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe4ec15dba0 0x7fe4ec06c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:11.591 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.588+0000 7fe4f1a6e700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fe4ec0b22e0 0x7fe4ec0b2770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:11.592 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.588+0000 7fe4f1a6e700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4ec161890 con 0x7fe4ec06b240 2023-11-08T19:27:11.592 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.588+0000 7fe4f1a6e700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4ec0b2cb0 con 0x7fe4ec0b22e0 2023-11-08T19:27:11.592 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.588+0000 7fe4f1a6e700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4ec06b740 con 0x7fe4ec15dba0 2023-11-08T19:27:11.592 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.588+0000 7fe4ebfff700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fe4ec0b22e0 0x7fe4ec0b2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:11.592 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.588+0000 7fe4eb7fe700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe4ec15dba0 0x7fe4ec06c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:11.593 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.588+0000 7fe4ebfff700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fe4ec0b22e0 0x7fe4ec0b2770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.143:36438/0 (socket says 172.21.15.143:36438) 2023-11-08T19:27:11.593 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.588+0000 7fe4ebfff700 1 -- 172.21.15.143:0/1601030027 learned_addr learned my addr 172.21.15.143:0/1601030027 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:11.593 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.588+0000 7fe4eb7fe700 1 -- 172.21.15.143:0/1601030027 >> v2:172.21.15.174:3300/0 conn(0x7fe4ec0b22e0 msgr2=0x7fe4ec0b2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:11.593 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.588+0000 7fe4eb7fe700 1 --2- 172.21.15.143:0/1601030027 >> v2:172.21.15.174:3300/0 conn(0x7fe4ec0b22e0 0x7fe4ec0b2770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:11.593 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.588+0000 7fe4eb7fe700 1 -- 172.21.15.143:0/1601030027 >> v1:172.21.15.174:6789/0 conn(0x7fe4ec06b240 legacy=0x7fe4ec0caff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:11.594 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.588+0000 7fe4eb7fe700 1 -- 172.21.15.143:0/1601030027 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4ec06ba20 con 0x7fe4ec15dba0 2023-11-08T19:27:11.594 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.588+0000 7fe4ebfff700 1 --2- 172.21.15.143:0/1601030027 >> v2:172.21.15.174:3300/0 conn(0x7fe4ec0b22e0 0x7fe4ec0b2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T19:27:11.594 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.589+0000 7fe4eb7fe700 1 --2- 172.21.15.143:0/1601030027 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe4ec15dba0 0x7fe4ec06c610 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe4e0009840 tx=0x7fe4e0009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31e1ae28675bc7ac server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:11.594 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.589+0000 7fe4ea7fc700 1 -- 172.21.15.143:0/1601030027 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e0011830 con 0x7fe4ec15dba0 2023-11-08T19:27:11.595 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.589+0000 7fe4ea7fc700 1 -- 172.21.15.143:0/1601030027 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4e0011990 con 0x7fe4ec15dba0 2023-11-08T19:27:11.595 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.589+0000 7fe4ea7fc700 1 -- 172.21.15.143:0/1601030027 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e0011d20 con 0x7fe4ec15dba0 2023-11-08T19:27:11.595 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.589+0000 7fe4f1a6e700 1 -- 172.21.15.143:0/1601030027 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe4ec15dba0 msgr2=0x7fe4ec06c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:11.595 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.589+0000 7fe4f1a6e700 1 --2- 172.21.15.143:0/1601030027 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe4ec15dba0 0x7fe4ec06c610 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe4e0009840 tx=0x7fe4e0009b50 comp rx=0 tx=0).stop 2023-11-08T19:27:11.595 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.589+0000 7fe4f1a6e700 1 -- 172.21.15.143:0/1601030027 shutdown_connections 2023-11-08T19:27:11.595 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.589+0000 7fe4f1a6e700 1 --2- 172.21.15.143:0/1601030027 >> v2:172.21.15.174:3300/0 conn(0x7fe4ec0b22e0 0x7fe4ec0b2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:11.596 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.589+0000 7fe4f1a6e700 1 --2- 172.21.15.143:0/1601030027 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe4ec15dba0 0x7fe4ec06c610 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:11.596 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.589+0000 7fe4f1a6e700 1 -- 172.21.15.143:0/1601030027 >> 172.21.15.143:0/1601030027 conn(0x7fe4ec0cd2e0 msgr2=0x7fe4ec0cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:11.596 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.589+0000 7fe4f1a6e700 1 -- 172.21.15.143:0/1601030027 shutdown_connections 2023-11-08T19:27:11.596 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.589+0000 7fe4f1a6e700 1 -- 172.21.15.143:0/1601030027 wait complete. 2023-11-08T19:27:11.599 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.590+0000 7fe4f1a6e700 1 Processor -- start 2023-11-08T19:27:11.599 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.591+0000 7fe4f1a6e700 1 -- start start 2023-11-08T19:27:11.599 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.591+0000 7fe4f1a6e700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe4ec06b240 0x7fe4ec0b3040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:11.599 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.591+0000 7fe4f1a6e700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe4ec0b22e0 0x7fe4ec1607e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:11.600 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.591+0000 7fe4f1a6e700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe4ec15dba0 0x7fe4ec15ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:11.600 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.591+0000 7fe4f1a6e700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4ec15e700 con 0x7fe4ec0b22e0 2023-11-08T19:27:11.600 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.591+0000 7fe4f1a6e700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4ec0b4230 con 0x7fe4ec06b240 2023-11-08T19:27:11.600 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.591+0000 7fe4f1a6e700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4ec160d20 con 0x7fe4ec15dba0 2023-11-08T19:27:11.600 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.592+0000 7fe4eaffd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe4ec0b22e0 0x7fe4ec1607e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:11.601 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.592+0000 7fe4eaffd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe4ec0b22e0 0x7fe4ec1607e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52944/0 (socket says 172.21.15.143:52944) 2023-11-08T19:27:11.601 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.592+0000 7fe4eaffd700 1 -- 172.21.15.143:0/3424828070 learned_addr learned my addr 172.21.15.143:0/3424828070 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:11.601 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.592+0000 7fe4eb7fe700 1 --2- 172.21.15.143:0/3424828070 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe4ec06b240 0x7fe4ec0b3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:11.601 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.592+0000 7fe4ebfff700 1 --2- 172.21.15.143:0/3424828070 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe4ec15dba0 0x7fe4ec15ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:11.601 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.592+0000 7fe4eaffd700 1 -- 172.21.15.143:0/3424828070 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe4ec15dba0 msgr2=0x7fe4ec15ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:11.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.592+0000 7fe4eaffd700 1 --2- 172.21.15.143:0/3424828070 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe4ec15dba0 0x7fe4ec15ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:11.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.592+0000 7fe4eaffd700 1 -- 172.21.15.143:0/3424828070 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe4ec06b240 msgr2=0x7fe4ec0b3040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:11.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.592+0000 7fe4eaffd700 1 --2- 172.21.15.143:0/3424828070 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe4ec06b240 0x7fe4ec0b3040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:11.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.592+0000 7fe4eaffd700 1 -- 172.21.15.143:0/3424828070 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4e4009b30 con 0x7fe4ec0b22e0 2023-11-08T19:27:11.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.592+0000 7fe4eaffd700 1 --2- 172.21.15.143:0/3424828070 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe4ec0b22e0 0x7fe4ec1607e0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc00c4e0 tx=0x7fe4dc00c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:11.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.592+0000 7fe4d3fff700 1 -- 172.21.15.143:0/3424828070 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4dc0052d0 con 0x7fe4ec0b22e0 2023-11-08T19:27:11.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.592+0000 7fe4d3fff700 1 -- 172.21.15.143:0/3424828070 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4dc005430 con 0x7fe4ec0b22e0 2023-11-08T19:27:11.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.592+0000 7fe4f1a6e700 1 -- 172.21.15.143:0/3424828070 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4e00094a0 con 0x7fe4ec0b22e0 2023-11-08T19:27:11.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.592+0000 7fe4d3fff700 1 -- 172.21.15.143:0/3424828070 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4dc0038c0 con 0x7fe4ec0b22e0 2023-11-08T19:27:11.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.592+0000 7fe4f1a6e700 1 -- 172.21.15.143:0/3424828070 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4ec0b3900 con 0x7fe4ec0b22e0 2023-11-08T19:27:11.604 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.593+0000 7fe4f1a6e700 1 -- 172.21.15.143:0/3424828070 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4ec04eee0 con 0x7fe4ec0b22e0 2023-11-08T19:27:11.604 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.594+0000 7fe4d3fff700 1 -- 172.21.15.143:0/3424828070 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe4dc003a20 con 0x7fe4ec0b22e0 2023-11-08T19:27:11.604 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.594+0000 7fe4d3fff700 1 --2- 172.21.15.143:0/3424828070 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fe4d806af00 0x7fe4d806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:11.604 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.594+0000 7fe4d3fff700 1 -- 172.21.15.143:0/3424828070 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6828+0+0 (secure 0 0 0) 0x7fe4dc088dd0 con 0x7fe4ec0b22e0 2023-11-08T19:27:11.604 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.594+0000 7fe4eb7fe700 1 --2- 172.21.15.143:0/3424828070 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fe4d806af00 0x7fe4d806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:11.605 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.595+0000 7fe4eb7fe700 1 --2- 172.21.15.143:0/3424828070 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fe4d806af00 0x7fe4d806d3c0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fe4e0000c00 tx=0x7fe4e001c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:11.605 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.599+0000 7fe4d3fff700 1 -- 172.21.15.143:0/3424828070 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe4dc053a10 con 0x7fe4ec0b22e0 2023-11-08T19:27:11.787 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.786+0000 7fe4f1a6e700 1 -- 172.21.15.143:0/3424828070 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe4ec0b4370 con 0x7fe4ec0b22e0 2023-11-08T19:27:11.788 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.786+0000 7fe4d3fff700 1 -- 172.21.15.143:0/3424828070 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v66) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe4dc0170a0 con 0x7fe4ec0b22e0 2023-11-08T19:27:11.788 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 64 2023-11-08T19:27:11.790 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.788+0000 7fe4f1a6e700 1 -- 172.21.15.143:0/3424828070 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fe4d806af00 msgr2=0x7fe4d806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:11.790 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.788+0000 7fe4f1a6e700 1 --2- 172.21.15.143:0/3424828070 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fe4d806af00 0x7fe4d806d3c0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fe4e0000c00 tx=0x7fe4e001c040 comp rx=0 tx=0).stop 2023-11-08T19:27:11.791 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.788+0000 7fe4f1a6e700 1 -- 172.21.15.143:0/3424828070 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe4ec0b22e0 msgr2=0x7fe4ec1607e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:11.791 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.788+0000 7fe4f1a6e700 1 --2- 172.21.15.143:0/3424828070 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe4ec0b22e0 0x7fe4ec1607e0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc00c4e0 tx=0x7fe4dc00c7f0 comp rx=0 tx=0).stop 2023-11-08T19:27:11.791 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.788+0000 7fe4f1a6e700 1 -- 172.21.15.143:0/3424828070 shutdown_connections 2023-11-08T19:27:11.791 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.788+0000 7fe4f1a6e700 1 --2- 172.21.15.143:0/3424828070 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fe4d806af00 0x7fe4d806d3c0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:11.791 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.788+0000 7fe4f1a6e700 1 --2- 172.21.15.143:0/3424828070 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe4ec15dba0 0x7fe4ec15ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:11.792 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.788+0000 7fe4f1a6e700 1 --2- 172.21.15.143:0/3424828070 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe4ec06b240 0x7fe4ec0b3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:11.792 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.788+0000 7fe4f1a6e700 1 --2- 172.21.15.143:0/3424828070 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe4ec0b22e0 0x7fe4ec1607e0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:11.792 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.788+0000 7fe4f1a6e700 1 -- 172.21.15.143:0/3424828070 >> 172.21.15.143:0/3424828070 conn(0x7fe4ec0cd2e0 msgr2=0x7fe4ec0cc8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:11.792 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.789+0000 7fe4f1a6e700 1 -- 172.21.15.143:0/3424828070 shutdown_connections 2023-11-08T19:27:11.792 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.789+0000 7fe4f1a6e700 1 -- 172.21.15.143:0/3424828070 wait complete. 2023-11-08T19:27:11.802 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get test_data_pool pg_num 2023-11-08T19:27:11.925 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.924+0000 7fb9f752e700 1 Processor -- start 2023-11-08T19:27:11.926 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.924+0000 7fb9f752e700 1 -- start start 2023-11-08T19:27:11.928 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.925+0000 7fb9f752e700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb9f00b2a10 0x7fb9f01623f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:11.928 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.925+0000 7fb9f752e700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fb9f00ce660 0x7fb9f006bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:11.928 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.925+0000 7fb9f752e700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9f0162930 con 0x7fb9f015dd60 2023-11-08T19:27:11.929 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.925+0000 7fb9f752e700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9f006b2b0 con 0x7fb9f00b2a10 2023-11-08T19:27:11.929 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.925+0000 7fb9f752e700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9f00cf000 con 0x7fb9f00ce660 2023-11-08T19:27:11.929 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.925+0000 7fb9f4ac9700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb9f00b2a10 0x7fb9f01623f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:11.929 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.925+0000 7fb9f4ac9700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb9f00b2a10 0x7fb9f01623f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52950/0 (socket says 172.21.15.143:52950) 2023-11-08T19:27:11.930 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.925+0000 7fb9f4ac9700 1 -- 172.21.15.143:0/2186276885 learned_addr learned my addr 172.21.15.143:0/2186276885 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:11.930 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.925+0000 7fb9f52ca700 1 --1- 172.21.15.143:0/2186276885 >> v1:172.21.15.143:6789/0 conn(0x7fb9f015dd60 0x7fb9f00b2300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:49104/0 (socket says 172.21.15.143:49104) 2023-11-08T19:27:11.930 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.926+0000 7fb9f5acb700 1 --2- 172.21.15.143:0/2186276885 >> v2:172.21.15.174:3300/0 conn(0x7fb9f00ce660 0x7fb9f006bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:11.930 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.926+0000 7fb9f4ac9700 1 -- 172.21.15.143:0/2186276885 >> v2:172.21.15.174:3300/0 conn(0x7fb9f00ce660 msgr2=0x7fb9f006bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:11.931 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.926+0000 7fb9f4ac9700 1 --2- 172.21.15.143:0/2186276885 >> v2:172.21.15.174:3300/0 conn(0x7fb9f00ce660 0x7fb9f006bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:11.931 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.926+0000 7fb9f4ac9700 1 -- 172.21.15.143:0/2186276885 >> v1:172.21.15.143:6789/0 conn(0x7fb9f015dd60 legacy=0x7fb9f00b2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:11.931 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.926+0000 7fb9f4ac9700 1 -- 172.21.15.143:0/2186276885 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9f00cf2e0 con 0x7fb9f00b2a10 2023-11-08T19:27:11.931 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.926+0000 7fb9e7fff700 1 -- 172.21.15.143:0/2186276885 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1946601543 0 0) 0x7fb9e0001110 con 0x7fb9f015dd60 2023-11-08T19:27:11.931 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.926+0000 7fb9f4ac9700 1 --2- 172.21.15.143:0/2186276885 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb9f00b2a10 0x7fb9f01623f0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fb9f0161920 tx=0x7fb9ec009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11fa20a031037b2 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:11.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.926+0000 7fb9e7fff700 1 -- 172.21.15.143:0/2186276885 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9ec00b870 con 0x7fb9f00b2a10 2023-11-08T19:27:11.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.926+0000 7fb9e7fff700 1 -- 172.21.15.143:0/2186276885 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9ec00b9d0 con 0x7fb9f00b2a10 2023-11-08T19:27:11.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.926+0000 7fb9e7fff700 1 -- 172.21.15.143:0/2186276885 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9ec00bd60 con 0x7fb9f00b2a10 2023-11-08T19:27:11.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.926+0000 7fb9f752e700 1 -- 172.21.15.143:0/2186276885 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb9f00b2a10 msgr2=0x7fb9f01623f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:11.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.926+0000 7fb9f752e700 1 --2- 172.21.15.143:0/2186276885 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb9f00b2a10 0x7fb9f01623f0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fb9f0161920 tx=0x7fb9ec009bd0 comp rx=0 tx=0).stop 2023-11-08T19:27:11.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.926+0000 7fb9f752e700 1 -- 172.21.15.143:0/2186276885 shutdown_connections 2023-11-08T19:27:11.933 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.926+0000 7fb9f752e700 1 --2- 172.21.15.143:0/2186276885 >> v2:172.21.15.174:3300/0 conn(0x7fb9f00ce660 0x7fb9f006bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:11.933 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.926+0000 7fb9f752e700 1 --2- 172.21.15.143:0/2186276885 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb9f00b2a10 0x7fb9f01623f0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:11.933 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.926+0000 7fb9f752e700 1 -- 172.21.15.143:0/2186276885 >> 172.21.15.143:0/2186276885 conn(0x7fb9f00cd300 msgr2=0x7fb9f00cd710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:11.933 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.927+0000 7fb9f752e700 1 -- 172.21.15.143:0/2186276885 shutdown_connections 2023-11-08T19:27:11.933 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.927+0000 7fb9f752e700 1 -- 172.21.15.143:0/2186276885 wait complete. 2023-11-08T19:27:11.936 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.928+0000 7fb9f752e700 1 Processor -- start 2023-11-08T19:27:11.937 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.928+0000 7fb9f752e700 1 -- start start 2023-11-08T19:27:11.937 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.928+0000 7fb9f752e700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb9f00b2a10 0x7fb9f00ca410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:11.937 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.928+0000 7fb9f752e700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb9f00ce660 0x7fb9f00c9340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:11.937 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.928+0000 7fb9f752e700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb9f015dd60 0x7fb9f00b4da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:11.938 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.928+0000 7fb9f752e700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9f00b5c60 con 0x7fb9f00ce660 2023-11-08T19:27:11.938 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.928+0000 7fb9f752e700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9f0158870 con 0x7fb9f015dd60 2023-11-08T19:27:11.938 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.928+0000 7fb9f752e700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9f00c8eb0 con 0x7fb9f00b2a10 2023-11-08T19:27:11.938 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.929+0000 7fb9f4ac9700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb9f00ce660 0x7fb9f00c9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:11.938 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.929+0000 7fb9f4ac9700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb9f00ce660 0x7fb9f00c9340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52956/0 (socket says 172.21.15.143:52956) 2023-11-08T19:27:11.939 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.929+0000 7fb9f4ac9700 1 -- 172.21.15.143:0/1640460137 learned_addr learned my addr 172.21.15.143:0/1640460137 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:11.939 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.929+0000 7fb9f5acb700 1 --2- 172.21.15.143:0/1640460137 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb9f015dd60 0x7fb9f00b4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:11.939 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.929+0000 7fb9f52ca700 1 --2- 172.21.15.143:0/1640460137 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb9f00b2a10 0x7fb9f00ca410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:11.939 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.929+0000 7fb9f4ac9700 1 -- 172.21.15.143:0/1640460137 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb9f00b2a10 msgr2=0x7fb9f00ca410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:11.939 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.929+0000 7fb9f4ac9700 1 --2- 172.21.15.143:0/1640460137 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb9f00b2a10 0x7fb9f00ca410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:11.940 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.929+0000 7fb9f4ac9700 1 -- 172.21.15.143:0/1640460137 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb9f015dd60 msgr2=0x7fb9f00b4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:11.940 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.929+0000 7fb9f4ac9700 1 --2- 172.21.15.143:0/1640460137 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb9f015dd60 0x7fb9f00b4da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:11.940 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.929+0000 7fb9f4ac9700 1 -- 172.21.15.143:0/1640460137 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9ec009550 con 0x7fb9f00ce660 2023-11-08T19:27:11.940 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.929+0000 7fb9f4ac9700 1 --2- 172.21.15.143:0/1640460137 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb9f00ce660 0x7fb9f00c9340 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec014040 tx=0x7fb9ec004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:11.940 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.929+0000 7fb9e5ffb700 1 -- 172.21.15.143:0/1640460137 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9ec0129c0 con 0x7fb9f00ce660 2023-11-08T19:27:11.941 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.929+0000 7fb9e5ffb700 1 -- 172.21.15.143:0/1640460137 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9ec01e070 con 0x7fb9f00ce660 2023-11-08T19:27:11.941 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.929+0000 7fb9e5ffb700 1 -- 172.21.15.143:0/1640460137 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9ec012d60 con 0x7fb9f00ce660 2023-11-08T19:27:11.941 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.929+0000 7fb9f752e700 1 -- 172.21.15.143:0/1640460137 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9f00c9880 con 0x7fb9f00ce660 2023-11-08T19:27:11.941 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.929+0000 7fb9f752e700 1 -- 172.21.15.143:0/1640460137 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9f00b5980 con 0x7fb9f00ce660 2023-11-08T19:27:11.941 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.930+0000 7fb9d37fe700 1 -- 172.21.15.143:0/1640460137 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9cc0052f0 con 0x7fb9f00ce660 2023-11-08T19:27:11.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.931+0000 7fb9e5ffb700 1 -- 172.21.15.143:0/1640460137 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb9ec02d450 con 0x7fb9f00ce660 2023-11-08T19:27:11.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.931+0000 7fb9e5ffb700 1 --2- 172.21.15.143:0/1640460137 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fb9d406af00 0x7fb9d406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:11.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.931+0000 7fb9e5ffb700 1 -- 172.21.15.143:0/1640460137 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6828+0+0 (secure 0 0 0) 0x7fb9ec08ac00 con 0x7fb9f00ce660 2023-11-08T19:27:11.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.931+0000 7fb9f52ca700 1 --2- 172.21.15.143:0/1640460137 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fb9d406af00 0x7fb9d406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:11.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.932+0000 7fb9f52ca700 1 --2- 172.21.15.143:0/1640460137 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fb9d406af00 0x7fb9d406d3c0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fb9e0000f60 tx=0x7fb9e0007a40 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:11.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:11.936+0000 7fb9e5ffb700 1 -- 172.21.15.143:0/1640460137 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb9ec0558f0 con 0x7fb9f00ce660 2023-11-08T19:27:12.127 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.125+0000 7fb9d37fe700 1 -- 172.21.15.143:0/1640460137 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "test_data_pool", "var": "pg_num"} v 0) v1 -- 0x7fb9cc005b20 con 0x7fb9f00ce660 2023-11-08T19:27:12.127 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.126+0000 7fb9e5ffb700 1 -- 172.21.15.143:0/1640460137 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "test_data_pool", "var": "pg_num"}]=0 v66) v1 ==== 103+0+11 (secure 0 0 0) 0x7fb9ec0150a0 con 0x7fb9f00ce660 2023-11-08T19:27:12.127 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2023-11-08T19:27:12.130 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.128+0000 7fb9d37fe700 1 -- 172.21.15.143:0/1640460137 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fb9d406af00 msgr2=0x7fb9d406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:12.130 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.128+0000 7fb9d37fe700 1 --2- 172.21.15.143:0/1640460137 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fb9d406af00 0x7fb9d406d3c0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fb9e0000f60 tx=0x7fb9e0007a40 comp rx=0 tx=0).stop 2023-11-08T19:27:12.130 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.128+0000 7fb9d37fe700 1 -- 172.21.15.143:0/1640460137 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb9f00ce660 msgr2=0x7fb9f00c9340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:12.130 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.128+0000 7fb9d37fe700 1 --2- 172.21.15.143:0/1640460137 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb9f00ce660 0x7fb9f00c9340 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec014040 tx=0x7fb9ec004750 comp rx=0 tx=0).stop 2023-11-08T19:27:12.130 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.128+0000 7fb9d37fe700 1 -- 172.21.15.143:0/1640460137 shutdown_connections 2023-11-08T19:27:12.131 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.128+0000 7fb9d37fe700 1 --2- 172.21.15.143:0/1640460137 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fb9d406af00 0x7fb9d406d3c0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:12.131 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.128+0000 7fb9d37fe700 1 --2- 172.21.15.143:0/1640460137 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb9f00b2a10 0x7fb9f00ca410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:12.131 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.128+0000 7fb9d37fe700 1 --2- 172.21.15.143:0/1640460137 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb9f015dd60 0x7fb9f00b4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:12.131 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.128+0000 7fb9d37fe700 1 --2- 172.21.15.143:0/1640460137 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb9f00ce660 0x7fb9f00c9340 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:12.131 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.128+0000 7fb9d37fe700 1 -- 172.21.15.143:0/1640460137 >> 172.21.15.143:0/1640460137 conn(0x7fb9f00cd300 msgr2=0x7fb9f00cbe80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:12.132 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.128+0000 7fb9d37fe700 1 -- 172.21.15.143:0/1640460137 shutdown_connections 2023-11-08T19:27:12.132 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.128+0000 7fb9d37fe700 1 -- 172.21.15.143:0/1640460137 wait complete. 2023-11-08T19:27:12.141 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get test_metadata_pool pg_num 2023-11-08T19:27:12.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.265+0000 7f5c76e9c700 1 Processor -- start 2023-11-08T19:27:12.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.265+0000 7f5c76e9c700 1 -- start start 2023-11-08T19:27:12.270 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.266+0000 7f5c76e9c700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5c7015a950 0x7f5c700d01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:12.270 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.266+0000 7f5c76e9c700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f5c700b61e0 0x7f5c700cc730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:12.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.266+0000 7f5c76e9c700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c700cc2a0 con 0x7f5c7006bd20 2023-11-08T19:27:12.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.266+0000 7f5c76e9c700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c700cc130 con 0x7f5c700b61e0 2023-11-08T19:27:12.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.266+0000 7f5c76e9c700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c7006c1b0 con 0x7f5c7015a950 2023-11-08T19:27:12.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.266+0000 7f5c75439700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f5c700b61e0 0x7f5c700cc730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:12.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.266+0000 7f5c75439700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f5c700b61e0 0x7f5c700cc730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52964/0 (socket says 172.21.15.143:52964) 2023-11-08T19:27:12.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.266+0000 7f5c75439700 1 -- 172.21.15.143:0/316644716 learned_addr learned my addr 172.21.15.143:0/316644716 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:12.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.266+0000 7f5c74c38700 1 --2- 172.21.15.143:0/316644716 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5c7015a950 0x7f5c700d01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:12.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.266+0000 7f5c75439700 1 -- 172.21.15.143:0/316644716 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5c7015a950 msgr2=0x7f5c700d01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:12.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.266+0000 7f5c75439700 1 --2- 172.21.15.143:0/316644716 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5c7015a950 0x7f5c700d01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:12.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.267+0000 7f5c75439700 1 -- 172.21.15.143:0/316644716 >> v1:172.21.15.174:6789/0 conn(0x7f5c7006bd20 legacy=0x7f5c70072370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:12.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.267+0000 7f5c75439700 1 -- 172.21.15.143:0/316644716 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c7006c490 con 0x7f5c700b61e0 2023-11-08T19:27:12.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.267+0000 7f5c75439700 1 --2- 172.21.15.143:0/316644716 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c700b61e0 0x7f5c700cc730 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f5c6800d150 tx=0x7f5c6800d460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bc8ab1a179117d9 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:12.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.267+0000 7f5c6f7fe700 1 -- 172.21.15.143:0/316644716 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c68010070 con 0x7f5c700b61e0 2023-11-08T19:27:12.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.267+0000 7f5c6f7fe700 1 -- 172.21.15.143:0/316644716 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c6800dd10 con 0x7f5c700b61e0 2023-11-08T19:27:12.274 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.267+0000 7f5c6f7fe700 1 -- 172.21.15.143:0/316644716 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c6800b310 con 0x7f5c700b61e0 2023-11-08T19:27:12.274 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.267+0000 7f5c76e9c700 1 -- 172.21.15.143:0/316644716 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c700b61e0 msgr2=0x7f5c700cc730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:12.274 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.267+0000 7f5c76e9c700 1 --2- 172.21.15.143:0/316644716 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c700b61e0 0x7f5c700cc730 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f5c6800d150 tx=0x7f5c6800d460 comp rx=0 tx=0).stop 2023-11-08T19:27:12.274 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.267+0000 7f5c76e9c700 1 -- 172.21.15.143:0/316644716 shutdown_connections 2023-11-08T19:27:12.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.267+0000 7f5c76e9c700 1 --2- 172.21.15.143:0/316644716 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c700b61e0 0x7f5c700cc730 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:12.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.267+0000 7f5c76e9c700 1 --2- 172.21.15.143:0/316644716 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5c7015a950 0x7f5c700d01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:12.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.267+0000 7f5c76e9c700 1 -- 172.21.15.143:0/316644716 >> 172.21.15.143:0/316644716 conn(0x7f5c700cd2e0 msgr2=0x7f5c700cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:12.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.267+0000 7f5c76e9c700 1 -- 172.21.15.143:0/316644716 shutdown_connections 2023-11-08T19:27:12.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.267+0000 7f5c76e9c700 1 -- 172.21.15.143:0/316644716 wait complete. 2023-11-08T19:27:12.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.269+0000 7f5c76e9c700 1 Processor -- start 2023-11-08T19:27:12.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.269+0000 7f5c76e9c700 1 -- start start 2023-11-08T19:27:12.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.269+0000 7f5c76e9c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c700b61e0 0x7f5c70158000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:12.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.269+0000 7f5c76e9c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5c7015a950 0x7f5c700b0990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:12.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.269+0000 7f5c76e9c700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5c700c6f00 0x7f5c700b1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:12.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.270+0000 7f5c76e9c700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c700b3720 con 0x7f5c700b61e0 2023-11-08T19:27:12.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.270+0000 7f5c76e9c700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c700b3dd0 con 0x7f5c7015a950 2023-11-08T19:27:12.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.270+0000 7f5c76e9c700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c700b40b0 con 0x7f5c700c6f00 2023-11-08T19:27:12.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.270+0000 7f5c74c38700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c700b61e0 0x7f5c70158000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:12.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.270+0000 7f5c74c38700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c700b61e0 0x7f5c70158000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52966/0 (socket says 172.21.15.143:52966) 2023-11-08T19:27:12.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.270+0000 7f5c74c38700 1 -- 172.21.15.143:0/2407120568 learned_addr learned my addr 172.21.15.143:0/2407120568 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:12.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.270+0000 7f5c74c38700 1 -- 172.21.15.143:0/2407120568 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5c700c6f00 msgr2=0x7f5c700b1bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:12.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.270+0000 7f5c74c38700 1 --2- 172.21.15.143:0/2407120568 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5c700c6f00 0x7f5c700b1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:12.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.270+0000 7f5c74c38700 1 -- 172.21.15.143:0/2407120568 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5c7015a950 msgr2=0x7f5c700b0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:12.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.270+0000 7f5c74c38700 1 --2- 172.21.15.143:0/2407120568 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5c7015a950 0x7f5c700b0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:12.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.270+0000 7f5c74c38700 1 -- 172.21.15.143:0/2407120568 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c6800bd40 con 0x7f5c700b61e0 2023-11-08T19:27:12.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.270+0000 7f5c74c38700 1 --2- 172.21.15.143:0/2407120568 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c700b61e0 0x7f5c70158000 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f5c6400e870 tx=0x7f5c6400ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:12.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.270+0000 7f5c6d7fa700 1 -- 172.21.15.143:0/2407120568 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c64009590 con 0x7f5c700b61e0 2023-11-08T19:27:12.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.270+0000 7f5c6d7fa700 1 -- 172.21.15.143:0/2407120568 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c6400ee70 con 0x7f5c700b61e0 2023-11-08T19:27:12.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.270+0000 7f5c76e9c700 1 -- 172.21.15.143:0/2407120568 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c700b4390 con 0x7f5c700b61e0 2023-11-08T19:27:12.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.270+0000 7f5c6d7fa700 1 -- 172.21.15.143:0/2407120568 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c6400ca50 con 0x7f5c700b61e0 2023-11-08T19:27:12.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.270+0000 7f5c76e9c700 1 -- 172.21.15.143:0/2407120568 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c700b4950 con 0x7f5c700b61e0 2023-11-08T19:27:12.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.271+0000 7f5c76e9c700 1 -- 172.21.15.143:0/2407120568 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c700b5ec0 con 0x7f5c700b61e0 2023-11-08T19:27:12.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.271+0000 7f5c6d7fa700 1 -- 172.21.15.143:0/2407120568 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5c6400cbb0 con 0x7f5c700b61e0 2023-11-08T19:27:12.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.272+0000 7f5c6d7fa700 1 --2- 172.21.15.143:0/2407120568 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f5c5406af00 0x7f5c5406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:12.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.272+0000 7f5c6d7fa700 1 -- 172.21.15.143:0/2407120568 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6828+0+0 (secure 0 0 0) 0x7f5c640899c0 con 0x7f5c700b61e0 2023-11-08T19:27:12.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.272+0000 7f5c6ffff700 1 --2- 172.21.15.143:0/2407120568 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f5c5406af00 0x7f5c5406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:12.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.272+0000 7f5c6ffff700 1 --2- 172.21.15.143:0/2407120568 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f5c5406af00 0x7f5c5406d3c0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f5c5c000f60 tx=0x7f5c5c007a40 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:12.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.276+0000 7f5c6d7fa700 1 -- 172.21.15.143:0/2407120568 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5c64054610 con 0x7f5c700b61e0 2023-11-08T19:27:12.467 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.465+0000 7f5c76e9c700 1 -- 172.21.15.143:0/2407120568 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "test_metadata_pool", "var": "pg_num"} v 0) v1 -- 0x7f5c7004eee0 con 0x7f5c700b61e0 2023-11-08T19:27:12.467 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.465+0000 7f5c6d7fa700 1 -- 172.21.15.143:0/2407120568 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "test_metadata_pool", "var": "pg_num"}]=0 v66) v1 ==== 107+0+11 (secure 0 0 0) 0x7f5c6400ce60 con 0x7f5c700b61e0 2023-11-08T19:27:12.468 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2023-11-08T19:27:12.469 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.467+0000 7f5c76e9c700 1 -- 172.21.15.143:0/2407120568 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f5c5406af00 msgr2=0x7f5c5406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:12.469 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.467+0000 7f5c76e9c700 1 --2- 172.21.15.143:0/2407120568 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f5c5406af00 0x7f5c5406d3c0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f5c5c000f60 tx=0x7f5c5c007a40 comp rx=0 tx=0).stop 2023-11-08T19:27:12.470 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.467+0000 7f5c76e9c700 1 -- 172.21.15.143:0/2407120568 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c700b61e0 msgr2=0x7f5c70158000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:12.470 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.467+0000 7f5c76e9c700 1 --2- 172.21.15.143:0/2407120568 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c700b61e0 0x7f5c70158000 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f5c6400e870 tx=0x7f5c6400ec30 comp rx=0 tx=0).stop 2023-11-08T19:27:12.470 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.467+0000 7f5c76e9c700 1 -- 172.21.15.143:0/2407120568 shutdown_connections 2023-11-08T19:27:12.470 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.467+0000 7f5c76e9c700 1 --2- 172.21.15.143:0/2407120568 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f5c5406af00 0x7f5c5406d3c0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:12.470 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.467+0000 7f5c76e9c700 1 --2- 172.21.15.143:0/2407120568 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5c700c6f00 0x7f5c700b1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:12.471 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.467+0000 7f5c76e9c700 1 --2- 172.21.15.143:0/2407120568 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c700b61e0 0x7f5c70158000 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:12.471 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.467+0000 7f5c76e9c700 1 --2- 172.21.15.143:0/2407120568 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5c7015a950 0x7f5c700b0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:12.471 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.467+0000 7f5c76e9c700 1 -- 172.21.15.143:0/2407120568 >> 172.21.15.143:0/2407120568 conn(0x7f5c700cd2e0 msgr2=0x7f5c7015af70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:12.471 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.467+0000 7f5c76e9c700 1 -- 172.21.15.143:0/2407120568 shutdown_connections 2023-11-08T19:27:12.471 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.467+0000 7f5c76e9c700 1 -- 172.21.15.143:0/2407120568 wait complete. 2023-11-08T19:27:12.481 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-08T19:27:12.604 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.602+0000 7fbeb7675700 1 Processor -- start 2023-11-08T19:27:12.604 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.602+0000 7fbeb7675700 1 -- start start 2023-11-08T19:27:12.606 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.603+0000 7fbeb7675700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fbeb006b320 0x7fbeb0168c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:12.607 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.603+0000 7fbeb7675700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbeb0161df0 con 0x7fbeb01618f0 2023-11-08T19:27:12.607 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.603+0000 7fbeb7675700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbeb01620d0 con 0x7fbeb015dd00 2023-11-08T19:27:12.607 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.603+0000 7fbeb7675700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbeb00cad50 con 0x7fbeb006b320 2023-11-08T19:27:12.607 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.603+0000 7fbeb5c12700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fbeb006b320 0x7fbeb0168c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:12.607 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.603+0000 7fbeb4c10700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fbeb01618f0 0x7fbeb006ac10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:49130/0 (socket says 172.21.15.143:49130) 2023-11-08T19:27:12.608 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.603+0000 7fbeb4c10700 1 -- 172.21.15.143:0/1987902761 learned_addr learned my addr 172.21.15.143:0/1987902761 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:27:12.608 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.604+0000 7fbeb5c12700 1 -- 172.21.15.143:0/1987902761 >> v1:172.21.15.174:6789/0 conn(0x7fbeb015dd00 legacy=0x7fbeb00cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:12.608 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.604+0000 7fbeb5c12700 1 -- 172.21.15.143:0/1987902761 >> v1:172.21.15.143:6789/0 conn(0x7fbeb01618f0 legacy=0x7fbeb006ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:12.608 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.604+0000 7fbeb5c12700 1 -- 172.21.15.143:0/1987902761 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbeb01623b0 con 0x7fbeb006b320 2023-11-08T19:27:12.608 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.604+0000 7fbea7fff700 1 -- 172.21.15.143:0/1987902761 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1345425773 0 0) 0x7fbeb0161df0 con 0x7fbeb01618f0 2023-11-08T19:27:12.609 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.604+0000 7fbeb5c12700 1 --2- 172.21.15.143:0/1987902761 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbeb006b320 0x7fbeb0168c60 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fbea8009d90 tx=0x7fbea800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c9da51cd111ed2a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:12.609 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.604+0000 7fbea7fff700 1 -- 172.21.15.143:0/1987902761 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea800e070 con 0x7fbeb006b320 2023-11-08T19:27:12.609 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.604+0000 7fbea7fff700 1 -- 172.21.15.143:0/1987902761 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbea800bbd0 con 0x7fbeb006b320 2023-11-08T19:27:12.609 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.604+0000 7fbea7fff700 1 -- 172.21.15.143:0/1987902761 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea8012430 con 0x7fbeb006b320 2023-11-08T19:27:12.609 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.605+0000 7fbeb7675700 1 -- 172.21.15.143:0/1987902761 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbeb006b320 msgr2=0x7fbeb0168c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:12.610 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.605+0000 7fbeb7675700 1 --2- 172.21.15.143:0/1987902761 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbeb006b320 0x7fbeb0168c60 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fbea8009d90 tx=0x7fbea800b330 comp rx=0 tx=0).stop 2023-11-08T19:27:12.610 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.605+0000 7fbeb7675700 1 -- 172.21.15.143:0/1987902761 shutdown_connections 2023-11-08T19:27:12.610 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.605+0000 7fbeb7675700 1 --2- 172.21.15.143:0/1987902761 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbeb006b320 0x7fbeb0168c60 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:12.610 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.605+0000 7fbeb7675700 1 -- 172.21.15.143:0/1987902761 >> 172.21.15.143:0/1987902761 conn(0x7fbeb0099ab0 msgr2=0x7fbeb00d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:12.610 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.605+0000 7fbeb7675700 1 -- 172.21.15.143:0/1987902761 shutdown_connections 2023-11-08T19:27:12.611 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.605+0000 7fbeb7675700 1 -- 172.21.15.143:0/1987902761 wait complete. 2023-11-08T19:27:12.613 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.606+0000 7fbeb7675700 1 Processor -- start 2023-11-08T19:27:12.614 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.606+0000 7fbeb7675700 1 -- start start 2023-11-08T19:27:12.614 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.607+0000 7fbeb7675700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbeb006b320 0x7fbeb00af280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:12.614 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.607+0000 7fbeb7675700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbeb015dd00 0x7fbeb015a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:12.614 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.607+0000 7fbeb7675700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbeb01618f0 0x7fbeb015a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:12.614 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.607+0000 7fbeb7675700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb00b1e40 con 0x7fbeb015dd00 2023-11-08T19:27:12.615 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.607+0000 7fbeb7675700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb00b5f80 con 0x7fbeb006b320 2023-11-08T19:27:12.615 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.607+0000 7fbeb7675700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb015ae60 con 0x7fbeb01618f0 2023-11-08T19:27:12.615 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.607+0000 7fbeb4c10700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbeb015dd00 0x7fbeb015a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:12.615 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.607+0000 7fbeb4c10700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbeb015dd00 0x7fbeb015a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52982/0 (socket says 172.21.15.143:52982) 2023-11-08T19:27:12.616 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.607+0000 7fbeb4c10700 1 -- 172.21.15.143:0/3520427129 learned_addr learned my addr 172.21.15.143:0/3520427129 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:12.616 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.607+0000 7fbeb5c12700 1 --2- 172.21.15.143:0/3520427129 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbeb01618f0 0x7fbeb015a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:12.616 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.607+0000 7fbeb5411700 1 --2- 172.21.15.143:0/3520427129 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbeb006b320 0x7fbeb00af280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:12.616 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.607+0000 7fbeb4c10700 1 -- 172.21.15.143:0/3520427129 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbeb01618f0 msgr2=0x7fbeb015a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:12.616 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.607+0000 7fbeb4c10700 1 --2- 172.21.15.143:0/3520427129 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbeb01618f0 0x7fbeb015a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:12.616 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.607+0000 7fbeb4c10700 1 -- 172.21.15.143:0/3520427129 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbeb006b320 msgr2=0x7fbeb00af280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:12.617 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.607+0000 7fbeb4c10700 1 --2- 172.21.15.143:0/3520427129 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbeb006b320 0x7fbeb00af280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:12.617 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.607+0000 7fbeb4c10700 1 -- 172.21.15.143:0/3520427129 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbea8009a70 con 0x7fbeb015dd00 2023-11-08T19:27:12.617 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.607+0000 7fbeb5411700 1 --2- 172.21.15.143:0/3520427129 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbeb006b320 0x7fbeb00af280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:27:12.617 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.607+0000 7fbeb4c10700 1 --2- 172.21.15.143:0/3520427129 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbeb015dd00 0x7fbeb015a3e0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fbeac00c520 tx=0x7fbeac00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:12.617 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.607+0000 7fbea5ffb700 1 -- 172.21.15.143:0/3520427129 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeac004c80 con 0x7fbeb015dd00 2023-11-08T19:27:12.618 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.607+0000 7fbea5ffb700 1 -- 172.21.15.143:0/3520427129 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbeac004de0 con 0x7fbeb015dd00 2023-11-08T19:27:12.618 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.607+0000 7fbeb7675700 1 -- 172.21.15.143:0/3520427129 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbeb015e260 con 0x7fbeb015dd00 2023-11-08T19:27:12.618 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.607+0000 7fbea5ffb700 1 -- 172.21.15.143:0/3520427129 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeac007310 con 0x7fbeb015dd00 2023-11-08T19:27:12.618 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.608+0000 7fbeb7675700 1 -- 172.21.15.143:0/3520427129 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbeb01f3780 con 0x7fbeb015dd00 2023-11-08T19:27:12.618 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.609+0000 7fbeb7675700 1 -- 172.21.15.143:0/3520427129 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbeb004eee0 con 0x7fbeb015dd00 2023-11-08T19:27:12.619 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.609+0000 7fbea5ffb700 1 -- 172.21.15.143:0/3520427129 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbeac007470 con 0x7fbeb015dd00 2023-11-08T19:27:12.619 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.609+0000 7fbea5ffb700 1 --2- 172.21.15.143:0/3520427129 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fbe9406af00 0x7fbe9406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:12.619 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.609+0000 7fbea5ffb700 1 -- 172.21.15.143:0/3520427129 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6828+0+0 (secure 0 0 0) 0x7fbeac089620 con 0x7fbeb015dd00 2023-11-08T19:27:12.619 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.609+0000 7fbeb5411700 1 --2- 172.21.15.143:0/3520427129 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fbe9406af00 0x7fbe9406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:12.619 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.610+0000 7fbeb5411700 1 --2- 172.21.15.143:0/3520427129 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fbe9406af00 0x7fbe9406d3c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fbeb00c7250 tx=0x7fbea0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:12.620 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.614+0000 7fbea5ffb700 1 -- 172.21.15.143:0/3520427129 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbeac054270 con 0x7fbeb015dd00 2023-11-08T19:27:12.821 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.819+0000 7fbeb7675700 1 -- 172.21.15.143:0/3520427129 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fbeb006b920 con 0x7fbeb015dd00 2023-11-08T19:27:12.821 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.820+0000 7fbea5ffb700 1 -- 172.21.15.143:0/3520427129 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v66) v1 ==== 80+0+0 (secure 0 0 0) 0x7fbeac057890 con 0x7fbeb015dd00 2023-11-08T19:27:12.824 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.822+0000 7fbeb7675700 1 -- 172.21.15.143:0/3520427129 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fbe9406af00 msgr2=0x7fbe9406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:12.824 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.822+0000 7fbeb7675700 1 --2- 172.21.15.143:0/3520427129 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fbe9406af00 0x7fbe9406d3c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fbeb00c7250 tx=0x7fbea0009040 comp rx=0 tx=0).stop 2023-11-08T19:27:12.824 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.822+0000 7fbeb7675700 1 -- 172.21.15.143:0/3520427129 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbeb015dd00 msgr2=0x7fbeb015a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:12.824 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.822+0000 7fbeb7675700 1 --2- 172.21.15.143:0/3520427129 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbeb015dd00 0x7fbeb015a3e0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fbeac00c520 tx=0x7fbeac00c830 comp rx=0 tx=0).stop 2023-11-08T19:27:12.824 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.822+0000 7fbeb7675700 1 -- 172.21.15.143:0/3520427129 shutdown_connections 2023-11-08T19:27:12.825 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.822+0000 7fbeb7675700 1 --2- 172.21.15.143:0/3520427129 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fbe9406af00 0x7fbe9406d3c0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:12.825 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.822+0000 7fbeb7675700 1 --2- 172.21.15.143:0/3520427129 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbeb01618f0 0x7fbeb015a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:12.825 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.822+0000 7fbeb7675700 1 --2- 172.21.15.143:0/3520427129 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbeb006b320 0x7fbeb00af280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:12.825 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.822+0000 7fbeb7675700 1 --2- 172.21.15.143:0/3520427129 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbeb015dd00 0x7fbeb015a3e0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:12.826 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.822+0000 7fbeb7675700 1 -- 172.21.15.143:0/3520427129 >> 172.21.15.143:0/3520427129 conn(0x7fbeb0099ab0 msgr2=0x7fbeb00cb6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:12.826 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.822+0000 7fbeb7675700 1 -- 172.21.15.143:0/3520427129 shutdown_connections 2023-11-08T19:27:12.826 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:12.822+0000 7fbeb7675700 1 -- 172.21.15.143:0/3520427129 wait complete. 2023-11-08T19:27:12.826 INFO:teuthology.orchestra.run.smithi143.stderr:listed 0 entries 2023-11-08T19:27:12.835 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-08T19:27:12.836 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:27:12.836 DEBUG:teuthology.orchestra.run.smithi143:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-11-08T19:27:12.875 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi143.front.sepia.ceph.com 2023-11-08T19:27:12.876 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:27:12.876 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-08T19:27:12.932 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2023-11-08T19:27:12.933 DEBUG:teuthology.orchestra.run.smithi174:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:27:12.954 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284224k,nr_inodes=4071056,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=31,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19549 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260544k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-08T19:27:12.954 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-08T19:27:12.955 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:27:12.989 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284264k,nr_inodes=4071066,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=44,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20643 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-08T19:27:12.989 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2023-11-08T19:27:12.990 DEBUG:teuthology.orchestra.run.smithi174:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:27:13.015 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284224k,nr_inodes=4071056,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=31,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19549 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260544k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-08T19:27:13.016 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:27:13.149 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.148+0000 7f1579bea700 1 Processor -- start 2023-11-08T19:27:13.149 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.148+0000 7f1579bea700 1 -- start start 2023-11-08T19:27:13.152 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.148+0000 7f1579bea700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f157415dd00 0x7f15740cca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:13.152 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.148+0000 7f1579bea700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f15741618f0 0x7f157406ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:13.152 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.148+0000 7f1579bea700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f157406b150 0x7f15740b2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:13.152 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.148+0000 7f1579bea700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f157406b600 con 0x7f15741618f0 2023-11-08T19:27:13.152 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.148+0000 7f1579bea700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15740cad50 con 0x7f157406b150 2023-11-08T19:27:13.153 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.148+0000 7f1579bea700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15740ccfb0 con 0x7f157415dd00 2023-11-08T19:27:13.153 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.149+0000 7f1572ffd700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f15741618f0 0x7f157406ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:13.153 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.149+0000 7f1572ffd700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f15741618f0 0x7f157406ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52990/0 (socket says 172.21.15.143:52990) 2023-11-08T19:27:13.153 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.149+0000 7f1572ffd700 1 -- 172.21.15.143:0/3683876120 learned_addr learned my addr 172.21.15.143:0/3683876120 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:13.153 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.149+0000 7f15737fe700 1 --2- 172.21.15.143:0/3683876120 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f157415dd00 0x7f15740cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:13.154 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.149+0000 7f1573fff700 1 --2- 172.21.15.143:0/3683876120 >> v2:172.21.15.174:3300/0 conn(0x7f157406b150 0x7f15740b2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:13.154 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.149+0000 7f1572ffd700 1 -- 172.21.15.143:0/3683876120 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f157415dd00 msgr2=0x7f15740cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:13.154 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.149+0000 7f1572ffd700 1 --2- 172.21.15.143:0/3683876120 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f157415dd00 0x7f15740cca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:13.154 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.149+0000 7f1572ffd700 1 -- 172.21.15.143:0/3683876120 >> v2:172.21.15.174:3300/0 conn(0x7f157406b150 msgr2=0x7f15740b2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:13.154 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.149+0000 7f1572ffd700 1 --2- 172.21.15.143:0/3683876120 >> v2:172.21.15.174:3300/0 conn(0x7f157406b150 0x7f15740b2210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:13.155 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.149+0000 7f1572ffd700 1 -- 172.21.15.143:0/3683876120 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1574161df0 con 0x7f15741618f0 2023-11-08T19:27:13.155 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.149+0000 7f1573fff700 1 --2- 172.21.15.143:0/3683876120 >> v2:172.21.15.174:3300/0 conn(0x7f157406b150 0x7f15740b2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:27:13.155 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.149+0000 7f1572ffd700 1 --2- 172.21.15.143:0/3683876120 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f15741618f0 0x7f157406ac10 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f15740708b0 tx=0x7f1568009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11f7541dc189fec1 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:13.155 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.149+0000 7f15727fc700 1 -- 172.21.15.143:0/3683876120 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1568011830 con 0x7f15741618f0 2023-11-08T19:27:13.155 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.149+0000 7f15727fc700 1 -- 172.21.15.143:0/3683876120 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1568011990 con 0x7f15741618f0 2023-11-08T19:27:13.156 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.149+0000 7f15727fc700 1 -- 172.21.15.143:0/3683876120 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1568011d20 con 0x7f15741618f0 2023-11-08T19:27:13.156 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.150+0000 7f1579bea700 1 -- 172.21.15.143:0/3683876120 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f15741618f0 msgr2=0x7f157406ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:13.156 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.150+0000 7f1579bea700 1 --2- 172.21.15.143:0/3683876120 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f15741618f0 0x7f157406ac10 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f15740708b0 tx=0x7f1568009bd0 comp rx=0 tx=0).stop 2023-11-08T19:27:13.156 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.150+0000 7f1579bea700 1 -- 172.21.15.143:0/3683876120 shutdown_connections 2023-11-08T19:27:13.156 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.150+0000 7f1579bea700 1 --2- 172.21.15.143:0/3683876120 >> v2:172.21.15.174:3300/0 conn(0x7f157406b150 0x7f15740b2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:13.157 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.150+0000 7f1579bea700 1 --2- 172.21.15.143:0/3683876120 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f157415dd00 0x7f15740cca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:13.157 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.150+0000 7f1579bea700 1 --2- 172.21.15.143:0/3683876120 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f15741618f0 0x7f157406ac10 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:13.157 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.150+0000 7f1579bea700 1 -- 172.21.15.143:0/3683876120 >> 172.21.15.143:0/3683876120 conn(0x7f1574099ab0 msgr2=0x7f15740d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:13.157 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.150+0000 7f1579bea700 1 -- 172.21.15.143:0/3683876120 shutdown_connections 2023-11-08T19:27:13.157 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.150+0000 7f1579bea700 1 -- 172.21.15.143:0/3683876120 wait complete. 2023-11-08T19:27:13.160 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.151+0000 7f1579bea700 1 Processor -- start 2023-11-08T19:27:13.160 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.151+0000 7f1579bea700 1 -- start start 2023-11-08T19:27:13.160 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.152+0000 7f1579bea700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f157406b150 0x7f15741f2e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:13.161 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.152+0000 7f1579bea700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f157415dd00 0x7f15741f6c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:13.161 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.152+0000 7f1579bea700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f15741618f0 0x7f15741f7170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:13.161 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.152+0000 7f1579bea700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15740b3a40 con 0x7f157406b150 2023-11-08T19:27:13.161 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.152+0000 7f1579bea700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15740b2080 con 0x7f15741618f0 2023-11-08T19:27:13.161 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.152+0000 7f1579bea700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15740b2360 con 0x7f157415dd00 2023-11-08T19:27:13.162 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.152+0000 7f15737fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f157406b150 0x7f15741f2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:13.162 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.152+0000 7f15737fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f157406b150 0x7f15741f2e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52994/0 (socket says 172.21.15.143:52994) 2023-11-08T19:27:13.162 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.152+0000 7f15737fe700 1 -- 172.21.15.143:0/2770150051 learned_addr learned my addr 172.21.15.143:0/2770150051 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:13.162 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.152+0000 7f1573fff700 1 --2- 172.21.15.143:0/2770150051 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f15741618f0 0x7f15741f7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:13.162 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.152+0000 7f1572ffd700 1 --2- 172.21.15.143:0/2770150051 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f157415dd00 0x7f15741f6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:13.163 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.153+0000 7f15737fe700 1 -- 172.21.15.143:0/2770150051 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f157415dd00 msgr2=0x7f15741f6c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:13.163 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.153+0000 7f15737fe700 1 --2- 172.21.15.143:0/2770150051 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f157415dd00 0x7f15741f6c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:13.163 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.153+0000 7f15737fe700 1 -- 172.21.15.143:0/2770150051 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f15741618f0 msgr2=0x7f15741f7170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:13.163 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.153+0000 7f15737fe700 1 --2- 172.21.15.143:0/2770150051 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f15741618f0 0x7f15741f7170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:13.163 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.153+0000 7f15737fe700 1 -- 172.21.15.143:0/2770150051 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1568009550 con 0x7f157406b150 2023-11-08T19:27:13.164 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.153+0000 7f1572ffd700 1 --2- 172.21.15.143:0/2770150051 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f157415dd00 0x7f15741f6c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:27:13.164 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.153+0000 7f15737fe700 1 --2- 172.21.15.143:0/2770150051 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f157406b150 0x7f15741f2e10 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f156400e990 tx=0x7f156400eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:13.164 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.153+0000 7f155bfff700 1 -- 172.21.15.143:0/2770150051 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15640096f0 con 0x7f157406b150 2023-11-08T19:27:13.164 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.153+0000 7f155bfff700 1 -- 172.21.15.143:0/2770150051 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f156400c850 con 0x7f157406b150 2023-11-08T19:27:13.164 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.153+0000 7f155bfff700 1 -- 172.21.15.143:0/2770150051 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f156400eee0 con 0x7f157406b150 2023-11-08T19:27:13.165 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.153+0000 7f1579bea700 1 -- 172.21.15.143:0/2770150051 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15740b2640 con 0x7f157406b150 2023-11-08T19:27:13.165 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.153+0000 7f1579bea700 1 -- 172.21.15.143:0/2770150051 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15740c9fc0 con 0x7f157406b150 2023-11-08T19:27:13.165 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.154+0000 7f1579bea700 1 -- 172.21.15.143:0/2770150051 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15740aff60 con 0x7f157406b150 2023-11-08T19:27:13.165 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.154+0000 7f155bfff700 1 -- 172.21.15.143:0/2770150051 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f156400cbc0 con 0x7f157406b150 2023-11-08T19:27:13.165 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.155+0000 7f155bfff700 1 --2- 172.21.15.143:0/2770150051 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f156006af00 0x7f156006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:13.166 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.155+0000 7f155bfff700 1 -- 172.21.15.143:0/2770150051 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6828+0+0 (secure 0 0 0) 0x7f1564089e10 con 0x7f157406b150 2023-11-08T19:27:13.166 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.155+0000 7f1572ffd700 1 --2- 172.21.15.143:0/2770150051 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f156006af00 0x7f156006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:13.166 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.155+0000 7f1572ffd700 1 --2- 172.21.15.143:0/2770150051 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f156006af00 0x7f156006d3c0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f1568009c10 tx=0x7f1568008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:13.166 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.159+0000 7f155bfff700 1 -- 172.21.15.143:0/2770150051 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1564054a60 con 0x7f157406b150 2023-11-08T19:27:13.395 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.394+0000 7f1579bea700 1 -- 172.21.15.143:0/2770150051 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f157404eee0 con 0x7f157406b150 2023-11-08T19:27:13.396 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.395+0000 7f155bfff700 1 -- 172.21.15.143:0/2770150051 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+4923 (secure 0 0 0) 0x7f1564021090 con 0x7f157406b150 2023-11-08T19:27:13.397 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:27:13.397 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":33,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5231,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6836/4083537882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":4083537882},{"type":"v1","addr":"172.21.15.174:6836","nonce":4083537882}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":5234,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/494923804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6835","nonce":494923804},{"type":"v1","addr":"172.21.15.174:6837","nonce":494923804}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":6009,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6834/3383638390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":3383638390},{"type":"v1","addr":"172.21.15.143:6834","nonce":3383638390}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T19:26:57.522785+0000","modified":"2023-11-08T19:27:00.536968+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6012},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6012":{"gid":6012,"name":"c","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.143:6835/2613282947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":2613282947},{"type":"v1","addr":"172.21.15.143:6835","nonce":2613282947}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-08T19:27:13.399 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.397+0000 7f1579bea700 1 -- 172.21.15.143:0/2770150051 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f156006af00 msgr2=0x7f156006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:13.399 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.397+0000 7f1579bea700 1 --2- 172.21.15.143:0/2770150051 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f156006af00 0x7f156006d3c0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f1568009c10 tx=0x7f1568008030 comp rx=0 tx=0).stop 2023-11-08T19:27:13.399 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.397+0000 7f1579bea700 1 -- 172.21.15.143:0/2770150051 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f157406b150 msgr2=0x7f15741f2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:13.399 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.397+0000 7f1579bea700 1 --2- 172.21.15.143:0/2770150051 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f157406b150 0x7f15741f2e10 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f156400e990 tx=0x7f156400eca0 comp rx=0 tx=0).stop 2023-11-08T19:27:13.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.397+0000 7f1579bea700 1 -- 172.21.15.143:0/2770150051 shutdown_connections 2023-11-08T19:27:13.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.397+0000 7f1579bea700 1 --2- 172.21.15.143:0/2770150051 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f156006af00 0x7f156006d3c0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:13.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.397+0000 7f1579bea700 1 --2- 172.21.15.143:0/2770150051 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f15741618f0 0x7f15741f7170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:13.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.397+0000 7f1579bea700 1 --2- 172.21.15.143:0/2770150051 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f157406b150 0x7f15741f2e10 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:13.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.397+0000 7f1579bea700 1 --2- 172.21.15.143:0/2770150051 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f157415dd00 0x7f15741f6c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:13.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.397+0000 7f1579bea700 1 -- 172.21.15.143:0/2770150051 >> 172.21.15.143:0/2770150051 conn(0x7f1574099ab0 msgr2=0x7f15740ad3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:13.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.397+0000 7f1579bea700 1 -- 172.21.15.143:0/2770150051 shutdown_connections 2023-11-08T19:27:13.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.397+0000 7f1579bea700 1 -- 172.21.15.143:0/2770150051 wait complete. 2023-11-08T19:27:13.401 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 33 2023-11-08T19:27:13.411 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:27:13.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.530+0000 7f02ee046700 1 Processor -- start 2023-11-08T19:27:13.532 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.530+0000 7f02ee046700 1 -- start start 2023-11-08T19:27:13.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.531+0000 7f02ee046700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f02e815dd00 0x7f02e80cca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:13.540 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.531+0000 7f02ee046700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02e80ccfb0 con 0x7f02e81618f0 2023-11-08T19:27:13.540 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.531+0000 7f02ee046700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02e8161df0 con 0x7f02e806b320 2023-11-08T19:27:13.540 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.531+0000 7f02ee046700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02e80cad50 con 0x7f02e815dd00 2023-11-08T19:27:13.541 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.531+0000 7f02e6ffd700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f02e81618f0 0x7f02e806ac10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:49158/0 (socket says 172.21.15.143:49158) 2023-11-08T19:27:13.541 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.531+0000 7f02e6ffd700 1 -- 172.21.15.143:0/3803426939 learned_addr learned my addr 172.21.15.143:0/3803426939 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:27:13.541 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.531+0000 7f02e77fe700 1 --2- 172.21.15.143:0/3803426939 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f02e815dd00 0x7f02e80cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:13.541 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.531+0000 7f02e67fc700 1 -- 172.21.15.143:0/3803426939 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3758692987 0 0) 0x7f02e80ccfb0 con 0x7f02e81618f0 2023-11-08T19:27:13.541 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.531+0000 7f02e67fc700 1 -- 172.21.15.143:0/3803426939 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f02d40035b0 con 0x7f02e81618f0 2023-11-08T19:27:13.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.531+0000 7f02e67fc700 1 -- 172.21.15.143:0/3803426939 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1065596379 0 0) 0x7f02d40035b0 con 0x7f02e81618f0 2023-11-08T19:27:13.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.531+0000 7f02e67fc700 1 -- 172.21.15.143:0/3803426939 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f02e815dd00 msgr2=0x7f02e80cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:13.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.531+0000 7f02e67fc700 1 --2- 172.21.15.143:0/3803426939 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f02e815dd00 0x7f02e80cca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:13.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.532+0000 7f02e67fc700 1 -- 172.21.15.143:0/3803426939 >> v1:172.21.15.174:6789/0 conn(0x7f02e806b320 legacy=0x7f02e80cb1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:13.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.532+0000 7f02e67fc700 1 -- 172.21.15.143:0/3803426939 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02e81620d0 con 0x7f02e81618f0 2023-11-08T19:27:13.543 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.532+0000 7f02e67fc700 1 -- 172.21.15.143:0/3803426939 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4147756303 0 0) 0x7f02d8001670 con 0x7f02e81618f0 2023-11-08T19:27:13.543 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.532+0000 7f02e67fc700 1 -- 172.21.15.143:0/3803426939 >> v1:172.21.15.143:6789/0 conn(0x7f02e81618f0 legacy=0x7f02e806ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:13.543 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.532+0000 7f02e67fc700 1 --2- 172.21.15.143:0/3803426939 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f02d4003a30 0x7f02d4003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:13.543 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.532+0000 7f02e67fc700 1 --2- 172.21.15.143:0/3803426939 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f02d40043e0 0x7f02d40048b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:13.543 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.532+0000 7f02e67fc700 1 --2- 172.21.15.143:0/3803426939 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f02e815dd00 0x7f02d4005110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:13.544 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.532+0000 7f02e67fc700 1 -- 172.21.15.143:0/3803426939 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02d400ba10 con 0x7f02d40043e0 2023-11-08T19:27:13.544 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.532+0000 7f02e67fc700 1 -- 172.21.15.143:0/3803426939 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02d400bb50 con 0x7f02d4003a30 2023-11-08T19:27:13.544 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.532+0000 7f02e67fc700 1 -- 172.21.15.143:0/3803426939 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02d400bd10 con 0x7f02e815dd00 2023-11-08T19:27:13.544 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.532+0000 7f02e6ffd700 1 --2- 172.21.15.143:0/3803426939 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f02d40043e0 0x7f02d40048b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:13.544 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.532+0000 7f02e77fe700 1 --2- 172.21.15.143:0/3803426939 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f02d4003a30 0x7f02d4003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:13.544 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.532+0000 7f02e7fff700 1 --2- 172.21.15.143:0/3803426939 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f02e815dd00 0x7f02d4005110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:13.545 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.532+0000 7f02e6ffd700 1 -- 172.21.15.143:0/3803426939 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f02e815dd00 msgr2=0x7f02d4005110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:13.545 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.532+0000 7f02e6ffd700 1 --2- 172.21.15.143:0/3803426939 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f02e815dd00 0x7f02d4005110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:13.545 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.532+0000 7f02e6ffd700 1 -- 172.21.15.143:0/3803426939 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f02d4003a30 msgr2=0x7f02d4003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:13.545 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.532+0000 7f02e6ffd700 1 --2- 172.21.15.143:0/3803426939 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f02d4003a30 0x7f02d4003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:13.545 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.532+0000 7f02e6ffd700 1 -- 172.21.15.143:0/3803426939 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02d40035b0 con 0x7f02d40043e0 2023-11-08T19:27:13.546 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.532+0000 7f02e77fe700 1 --2- 172.21.15.143:0/3803426939 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f02d4003a30 0x7f02d4003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:27:13.546 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.532+0000 7f02e6ffd700 1 --2- 172.21.15.143:0/3803426939 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f02d40043e0 0x7f02d40048b0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f02d800c240 tx=0x7f02d4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39b9e612d078f901 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:13.546 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.532+0000 7f02e67fc700 1 -- 172.21.15.143:0/3803426939 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d800cd80 con 0x7f02d40043e0 2023-11-08T19:27:13.546 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.532+0000 7f02e67fc700 1 -- 172.21.15.143:0/3803426939 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02d8005960 con 0x7f02d40043e0 2023-11-08T19:27:13.546 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.532+0000 7f02e67fc700 1 -- 172.21.15.143:0/3803426939 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d8005d10 con 0x7f02d40043e0 2023-11-08T19:27:13.547 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.533+0000 7f02ee046700 1 -- 172.21.15.143:0/3803426939 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f02d40043e0 msgr2=0x7f02d40048b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:13.547 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.533+0000 7f02ee046700 1 --2- 172.21.15.143:0/3803426939 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f02d40043e0 0x7f02d40048b0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f02d800c240 tx=0x7f02d4005880 comp rx=0 tx=0).stop 2023-11-08T19:27:13.547 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.533+0000 7f02e7fff700 1 -- 172.21.15.143:0/3803426939 reap_dead start 2023-11-08T19:27:13.547 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.533+0000 7f02ee046700 1 -- 172.21.15.143:0/3803426939 shutdown_connections 2023-11-08T19:27:13.548 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.533+0000 7f02ee046700 1 -- 172.21.15.143:0/3803426939 >> 172.21.15.143:0/3803426939 conn(0x7f02e8099ab0 msgr2=0x7f02e80d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:13.548 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.533+0000 7f02ee046700 1 -- 172.21.15.143:0/3803426939 shutdown_connections 2023-11-08T19:27:13.548 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.533+0000 7f02ee046700 1 -- 172.21.15.143:0/3803426939 wait complete. 2023-11-08T19:27:13.548 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.534+0000 7f02ee046700 1 Processor -- start 2023-11-08T19:27:13.548 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.534+0000 7f02ee046700 1 -- start start 2023-11-08T19:27:13.549 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.535+0000 7f02ee046700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f02d40043e0 0x7f02e8158050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:13.549 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.535+0000 7f02ee046700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f02e815fe00 0x7f02e8160280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:13.549 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.535+0000 7f02ee046700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f02e80b3460 0x7f02e80b38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:13.549 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.535+0000 7f02ee046700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02e80afab0 con 0x7f02d40043e0 2023-11-08T19:27:13.549 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.535+0000 7f02ee046700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02e80b3e20 con 0x7f02e815fe00 2023-11-08T19:27:13.549 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.535+0000 7f02ee046700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02e80c4230 con 0x7f02e80b3460 2023-11-08T19:27:13.550 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.535+0000 7f02e77fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f02d40043e0 0x7f02e8158050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:13.550 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.535+0000 7f02e77fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f02d40043e0 0x7f02e8158050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53014/0 (socket says 172.21.15.143:53014) 2023-11-08T19:27:13.550 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.535+0000 7f02e77fe700 1 -- 172.21.15.143:0/3577584707 learned_addr learned my addr 172.21.15.143:0/3577584707 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:13.550 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.535+0000 7f02e7fff700 1 --2- 172.21.15.143:0/3577584707 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f02e80b3460 0x7f02e80b38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:13.551 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.535+0000 7f02e6ffd700 1 --2- 172.21.15.143:0/3577584707 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f02e815fe00 0x7f02e8160280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:13.551 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.535+0000 7f02e77fe700 1 -- 172.21.15.143:0/3577584707 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f02e80b3460 msgr2=0x7f02e80b38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:13.551 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.535+0000 7f02e77fe700 1 --2- 172.21.15.143:0/3577584707 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f02e80b3460 0x7f02e80b38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:13.551 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.535+0000 7f02e77fe700 1 -- 172.21.15.143:0/3577584707 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f02e815fe00 msgr2=0x7f02e8160280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:13.551 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.535+0000 7f02e77fe700 1 --2- 172.21.15.143:0/3577584707 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f02e815fe00 0x7f02e8160280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:13.552 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.535+0000 7f02e77fe700 1 -- 172.21.15.143:0/3577584707 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02e80c4460 con 0x7f02d40043e0 2023-11-08T19:27:13.552 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.536+0000 7f02e77fe700 1 --2- 172.21.15.143:0/3577584707 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f02d40043e0 0x7f02e8158050 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f02d0011680 tx=0x7f02d0011a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:13.552 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.536+0000 7f02dffff700 1 -- 172.21.15.143:0/3577584707 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d0004220 con 0x7f02d40043e0 2023-11-08T19:27:13.552 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.536+0000 7f02dffff700 1 -- 172.21.15.143:0/3577584707 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02d0004380 con 0x7f02d40043e0 2023-11-08T19:27:13.552 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.536+0000 7f02ee046700 1 -- 172.21.15.143:0/3577584707 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02e80c4740 con 0x7f02d40043e0 2023-11-08T19:27:13.553 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.536+0000 7f02dffff700 1 -- 172.21.15.143:0/3577584707 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d00036a0 con 0x7f02d40043e0 2023-11-08T19:27:13.553 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.536+0000 7f02ee046700 1 -- 172.21.15.143:0/3577584707 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02e80c4d00 con 0x7f02d40043e0 2023-11-08T19:27:13.553 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.537+0000 7f02ee046700 1 -- 172.21.15.143:0/3577584707 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02e804eee0 con 0x7f02d40043e0 2023-11-08T19:27:13.553 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.537+0000 7f02dffff700 1 -- 172.21.15.143:0/3577584707 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f02d0003800 con 0x7f02d40043e0 2023-11-08T19:27:13.553 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.537+0000 7f02dffff700 1 --2- 172.21.15.143:0/3577584707 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f02c806af30 0x7f02c806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:13.554 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.538+0000 7f02dffff700 1 -- 172.21.15.143:0/3577584707 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6828+0+0 (secure 0 0 0) 0x7f02d008a2d0 con 0x7f02d40043e0 2023-11-08T19:27:13.554 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.538+0000 7f02e6ffd700 1 --2- 172.21.15.143:0/3577584707 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f02c806af30 0x7f02c806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:13.554 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.538+0000 7f02e6ffd700 1 --2- 172.21.15.143:0/3577584707 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f02c806af30 0x7f02c806d3f0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f02d80018b0 tx=0x7f02d800a000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:13.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.542+0000 7f02dffff700 1 -- 172.21.15.143:0/3577584707 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f02d0054f30 con 0x7f02d40043e0 2023-11-08T19:27:13.737 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.735+0000 7f02ee046700 1 -- 172.21.15.143:0/3577584707 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f02e806b920 con 0x7f02d40043e0 2023-11-08T19:27:13.738 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.737+0000 7f02dffff700 1 -- 172.21.15.143:0/3577584707 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) v1 ==== 74+0+21891 (secure 0 0 0) 0x7f02d0058550 con 0x7f02d40043e0 2023-11-08T19:27:13.738 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:27:13.739 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":66,"fsid":"968483af-77fc-4068-ad7c-5c2a1a4613d7","created":"2023-11-08T19:24:32.826484+0000","modified":"2023-11-08T19:27:09.174117+0000","last_up_change":"2023-11-08T19:24:43.573880+0000","last_in_change":"2023-11-08T19:24:35.314525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:44.551530+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"snap-test-pool","create_time":"2023-11-08T19:26:02.874384+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":36,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-11-08T19:26:55.754227+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-11-08T19:26:56.486110+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"test_data_pool","create_time":"2023-11-08T19:27:05.601949+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":66,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_metadata_pool","create_time":"2023-11-08T19:27:06.536369+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bae98ec1-d94e-403d-a220-a522a53ea6f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6809","nonce":1487830566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6811","nonce":1487830566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6815","nonce":1487830566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6813","nonce":1487830566}]},"public_addr":"172.21.15.143:6809/1487830566","cluster_addr":"172.21.15.143:6811/1487830566","heartbeat_back_addr":"172.21.15.143:6815/1487830566","heartbeat_front_addr":"172.21.15.143:6813/1487830566","state":["exists","up"]},{"osd":1,"uuid":"5cb6538a-1ccb-4322-8633-09f66af0b7e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6801","nonce":787341397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6803","nonce":787341397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6807","nonce":787341397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6805","nonce":787341397}]},"public_addr":"172.21.15.143:6801/787341397","cluster_addr":"172.21.15.143:6803/787341397","heartbeat_back_addr":"172.21.15.143:6807/787341397","heartbeat_front_addr":"172.21.15.143:6805/787341397","state":["exists","up"]},{"osd":2,"uuid":"cdbf522c-60ef-4d60-9653-8dcb529c101a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6822","nonce":2301325597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6826","nonce":2301325597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6831","nonce":2301325597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6829","nonce":2301325597}]},"public_addr":"172.21.15.143:6822/2301325597","cluster_addr":"172.21.15.143:6826/2301325597","heartbeat_back_addr":"172.21.15.143:6831/2301325597","heartbeat_front_addr":"172.21.15.143:6829/2301325597","state":["exists","up"]},{"osd":3,"uuid":"4e2783cf-ebb9-4dd8-8245-bc1d17ce4325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6817","nonce":2021504428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6819","nonce":2021504428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6827","nonce":2021504428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6823","nonce":2021504428}]},"public_addr":"172.21.15.143:6817/2021504428","cluster_addr":"172.21.15.143:6819/2021504428","heartbeat_back_addr":"172.21.15.143:6827/2021504428","heartbeat_front_addr":"172.21.15.143:6823/2021504428","state":["exists","up"]},{"osd":4,"uuid":"691256fa-9b77-49b9-a346-7ae1adf0ca80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6801","nonce":1993380595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6803","nonce":1993380595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6807","nonce":1993380595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6805","nonce":1993380595}]},"public_addr":"172.21.15.174:6801/1993380595","cluster_addr":"172.21.15.174:6803/1993380595","heartbeat_back_addr":"172.21.15.174:6807/1993380595","heartbeat_front_addr":"172.21.15.174:6805/1993380595","state":["exists","up"]},{"osd":5,"uuid":"756fc352-48c9-46ce-b52e-0541fe6f8525","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6809","nonce":1777923303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6811","nonce":1777923303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6815","nonce":1777923303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6813","nonce":1777923303}]},"public_addr":"172.21.15.174:6809/1777923303","cluster_addr":"172.21.15.174:6811/1777923303","heartbeat_back_addr":"172.21.15.174:6815/1777923303","heartbeat_front_addr":"172.21.15.174:6813/1777923303","state":["exists","up"]},{"osd":6,"uuid":"fab2ac2b-ed97-4cde-9825-b480eabf8b8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6825","nonce":2755130316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6827","nonce":2755130316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6831","nonce":2755130316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6829","nonce":2755130316}]},"public_addr":"172.21.15.174:6825/2755130316","cluster_addr":"172.21.15.174:6827/2755130316","heartbeat_back_addr":"172.21.15.174:6831/2755130316","heartbeat_front_addr":"172.21.15.174:6829/2755130316","state":["exists","up"]},{"osd":7,"uuid":"3de9b797-2ab6-4d41-a863-7e7d638fd703","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6817","nonce":2847141411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6819","nonce":2847141411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6823","nonce":2847141411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6821","nonce":2847141411}]},"public_addr":"172.21.15.174:6817/2847141411","cluster_addr":"172.21.15.174:6819/2847141411","heartbeat_back_addr":"172.21.15.174:6823/2847141411","heartbeat_front_addr":"172.21.15.174:6821/2847141411","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.477076+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.496890+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.539305+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.454673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.538881+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.521294+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.520712+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.540404+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":11,"snaps":[{"begin":1,"length":1}]}],"new_removed_snaps":[{"pool":11,"snaps":[{"begin":1,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:27:13.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.739+0000 7f02ee046700 1 -- 172.21.15.143:0/3577584707 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f02c806af30 msgr2=0x7f02c806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:13.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.739+0000 7f02ee046700 1 --2- 172.21.15.143:0/3577584707 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f02c806af30 0x7f02c806d3f0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f02d80018b0 tx=0x7f02d800a000 comp rx=0 tx=0).stop 2023-11-08T19:27:13.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.739+0000 7f02ee046700 1 -- 172.21.15.143:0/3577584707 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f02d40043e0 msgr2=0x7f02e8158050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:13.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.739+0000 7f02ee046700 1 --2- 172.21.15.143:0/3577584707 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f02d40043e0 0x7f02e8158050 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f02d0011680 tx=0x7f02d0011a40 comp rx=0 tx=0).stop 2023-11-08T19:27:13.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.739+0000 7f02ee046700 1 -- 172.21.15.143:0/3577584707 shutdown_connections 2023-11-08T19:27:13.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.739+0000 7f02ee046700 1 --2- 172.21.15.143:0/3577584707 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f02c806af30 0x7f02c806d3f0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:13.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.739+0000 7f02ee046700 1 --2- 172.21.15.143:0/3577584707 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f02e80b3460 0x7f02e80b38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:13.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.739+0000 7f02ee046700 1 --2- 172.21.15.143:0/3577584707 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f02d40043e0 0x7f02e8158050 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:13.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.739+0000 7f02ee046700 1 --2- 172.21.15.143:0/3577584707 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f02e815fe00 0x7f02e8160280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:13.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.739+0000 7f02ee046700 1 -- 172.21.15.143:0/3577584707 >> 172.21.15.143:0/3577584707 conn(0x7f02e8099ab0 msgr2=0x7f02e806f450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:13.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.739+0000 7f02ee046700 1 -- 172.21.15.143:0/3577584707 shutdown_connections 2023-11-08T19:27:13.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.739+0000 7f02ee046700 1 -- 172.21.15.143:0/3577584707 wait complete. 2023-11-08T19:27:13.753 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T19:24:44.551530+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'snap-test-pool', 'create_time': '2023-11-08T19:26:02.874384+0000', 'flags': 16385, 'flags_names': 'hashpspool,pool_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '36', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'pool', 'snap_seq': 2, 'snap_epoch': 36, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-08T19:26:55.754227+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '61', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2023-11-08T19:26:56.486110+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '62', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'test_data_pool', 'create_time': '2023-11-08T19:27:05.601949+0000', 'flags': 16385, 'flags_names': 'hashpspool,pool_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'pool', 'snap_seq': 2, 'snap_epoch': 66, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'test_metadata_pool', 'create_time': '2023-11-08T19:27:06.536369+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-08T19:27:13.754 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T19:27:13.876 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.875+0000 7f861e6e2700 1 Processor -- start 2023-11-08T19:27:13.877 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.875+0000 7f861e6e2700 1 -- start start 2023-11-08T19:27:13.880 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.876+0000 7f861e6e2700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f86180b1730 0x7f86180b2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:13.881 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.876+0000 7f861e6e2700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86180d0420 con 0x7f86180d1d80 2023-11-08T19:27:13.881 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.876+0000 7f861e6e2700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86180cf810 con 0x7f861815bb20 2023-11-08T19:27:13.881 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.876+0000 7f861e6e2700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86180cf600 con 0x7f86180b1730 2023-11-08T19:27:13.881 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.876+0000 7f86177fe700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f86180d1d80 0x7f86180cc4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:49176/0 (socket says 172.21.15.143:49176) 2023-11-08T19:27:13.881 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.876+0000 7f86177fe700 1 -- 172.21.15.143:0/3567515452 learned_addr learned my addr 172.21.15.143:0/3567515452 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:27:13.882 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.876+0000 7f861cc7f700 1 --2- 172.21.15.143:0/3567515452 >> v2:172.21.15.174:3300/0 conn(0x7f86180b1730 0x7f86180b2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:13.882 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.876+0000 7f8616ffd700 1 -- 172.21.15.143:0/3567515452 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1639954095 0 0) 0x7f86180d0420 con 0x7f86180d1d80 2023-11-08T19:27:13.882 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.876+0000 7f8616ffd700 1 -- 172.21.15.143:0/3567515452 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86000035b0 con 0x7f86180d1d80 2023-11-08T19:27:13.882 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.876+0000 7f8616ffd700 1 -- 172.21.15.143:0/3567515452 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1520803498 0 0) 0x7f86000035b0 con 0x7f86180d1d80 2023-11-08T19:27:13.882 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.876+0000 7f8616ffd700 1 -- 172.21.15.143:0/3567515452 >> v2:172.21.15.174:3300/0 conn(0x7f86180b1730 msgr2=0x7f86180b2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:13.883 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.876+0000 7f8616ffd700 1 --2- 172.21.15.143:0/3567515452 >> v2:172.21.15.174:3300/0 conn(0x7f86180b1730 0x7f86180b2670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:13.883 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.876+0000 7f8616ffd700 1 -- 172.21.15.143:0/3567515452 >> v1:172.21.15.174:6789/0 conn(0x7f861815bb20 legacy=0x7f86180d1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:13.883 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.876+0000 7f8616ffd700 1 -- 172.21.15.143:0/3567515452 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86180cfaf0 con 0x7f86180d1d80 2023-11-08T19:27:13.883 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.876+0000 7f8616ffd700 1 -- 172.21.15.143:0/3567515452 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4147756303 0 0) 0x7f8608001670 con 0x7f86180d1d80 2023-11-08T19:27:13.884 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.877+0000 7f8616ffd700 1 -- 172.21.15.143:0/3567515452 >> v1:172.21.15.143:6789/0 conn(0x7f86180d1d80 legacy=0x7f86180cc4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:13.884 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.877+0000 7f8616ffd700 1 --2- 172.21.15.143:0/3567515452 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8600003a00 0x7f8600003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:13.884 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.877+0000 7f8616ffd700 1 --2- 172.21.15.143:0/3567515452 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8600004400 0x7f86000048d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:13.885 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.877+0000 7f8616ffd700 1 --2- 172.21.15.143:0/3567515452 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86000050d0 0x7f860000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:13.885 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.877+0000 7f8616ffd700 1 -- 172.21.15.143:0/3567515452 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86180cf600 con 0x7f86000050d0 2023-11-08T19:27:13.885 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.877+0000 7f8616ffd700 1 -- 172.21.15.143:0/3567515452 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f860000c030 con 0x7f8600004400 2023-11-08T19:27:13.885 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.877+0000 7f8616ffd700 1 -- 172.21.15.143:0/3567515452 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f860000c1f0 con 0x7f8600003a00 2023-11-08T19:27:13.885 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.877+0000 7f861cc7f700 1 --2- 172.21.15.143:0/3567515452 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86000050d0 0x7f860000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:13.886 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.877+0000 7f86177fe700 1 --2- 172.21.15.143:0/3567515452 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8600004400 0x7f86000048d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:13.886 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.877+0000 7f8617fff700 1 --2- 172.21.15.143:0/3567515452 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8600003a00 0x7f8600003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:13.886 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.877+0000 7f861cc7f700 1 -- 172.21.15.143:0/3567515452 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8600003a00 msgr2=0x7f8600003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:13.886 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.877+0000 7f861cc7f700 1 --2- 172.21.15.143:0/3567515452 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8600003a00 0x7f8600003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:13.887 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.877+0000 7f861cc7f700 1 -- 172.21.15.143:0/3567515452 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8600004400 msgr2=0x7f86000048d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:13.887 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.877+0000 7f861cc7f700 1 --2- 172.21.15.143:0/3567515452 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8600004400 0x7f86000048d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:13.887 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.877+0000 7f861cc7f700 1 -- 172.21.15.143:0/3567515452 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86000035b0 con 0x7f86000050d0 2023-11-08T19:27:13.887 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.877+0000 7f8617fff700 1 --2- 172.21.15.143:0/3567515452 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8600003a00 0x7f8600003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:27:13.887 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.877+0000 7f861cc7f700 1 -- 172.21.15.143:0/3567515452 reap_dead start 2023-11-08T19:27:13.888 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.877+0000 7f861cc7f700 1 --2- 172.21.15.143:0/3567515452 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86000050d0 0x7f860000ba10 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f8610009fc0 tx=0x7f8600005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47360b803d8322a4 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:13.888 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.877+0000 7f8616ffd700 1 -- 172.21.15.143:0/3567515452 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8610003a20 con 0x7f86000050d0 2023-11-08T19:27:13.889 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.877+0000 7f8616ffd700 1 -- 172.21.15.143:0/3567515452 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f861000f040 con 0x7f86000050d0 2023-11-08T19:27:13.889 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.877+0000 7f8616ffd700 1 -- 172.21.15.143:0/3567515452 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8610003de0 con 0x7f86000050d0 2023-11-08T19:27:13.889 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.878+0000 7f861e6e2700 1 -- 172.21.15.143:0/3567515452 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86000050d0 msgr2=0x7f860000ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:13.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.878+0000 7f861e6e2700 1 --2- 172.21.15.143:0/3567515452 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86000050d0 0x7f860000ba10 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f8610009fc0 tx=0x7f8600005880 comp rx=0 tx=0).stop 2023-11-08T19:27:13.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.878+0000 7f861e6e2700 1 -- 172.21.15.143:0/3567515452 shutdown_connections 2023-11-08T19:27:13.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.878+0000 7f861e6e2700 1 --2- 172.21.15.143:0/3567515452 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86000050d0 0x7f860000ba10 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:13.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.878+0000 7f861e6e2700 1 -- 172.21.15.143:0/3567515452 >> 172.21.15.143:0/3567515452 conn(0x7f861808b000 msgr2=0x7f861809ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:13.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.878+0000 7f861e6e2700 1 -- 172.21.15.143:0/3567515452 shutdown_connections 2023-11-08T19:27:13.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.878+0000 7f861e6e2700 1 -- 172.21.15.143:0/3567515452 wait complete. 2023-11-08T19:27:13.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.879+0000 7f861e6e2700 1 Processor -- start 2023-11-08T19:27:13.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.879+0000 7f861e6e2700 1 -- start start 2023-11-08T19:27:13.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.880+0000 7f861e6e2700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f86000050d0 0x7f86180b4650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:13.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.880+0000 7f861e6e2700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86180cddc0 0x7f86180b7300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:13.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.880+0000 7f861e6e2700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f86180ccea0 0x7f86180b3540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:13.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.880+0000 7f861e6e2700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86180c6150 con 0x7f86180cddc0 2023-11-08T19:27:13.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.880+0000 7f861e6e2700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86180b7840 con 0x7f86180ccea0 2023-11-08T19:27:13.895 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.880+0000 7f861e6e2700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86180c5540 con 0x7f86000050d0 2023-11-08T19:27:13.895 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.880+0000 7f86177fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86180cddc0 0x7f86180b7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:13.895 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.880+0000 7f86177fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86180cddc0 0x7f86180b7300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53036/0 (socket says 172.21.15.143:53036) 2023-11-08T19:27:13.895 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.880+0000 7f86177fe700 1 -- 172.21.15.143:0/2646904712 learned_addr learned my addr 172.21.15.143:0/2646904712 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:13.895 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.880+0000 7f8617fff700 1 --2- 172.21.15.143:0/2646904712 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f86000050d0 0x7f86180b4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:13.896 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.880+0000 7f861cc7f700 1 --2- 172.21.15.143:0/2646904712 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f86180ccea0 0x7f86180b3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:13.896 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.880+0000 7f86177fe700 1 -- 172.21.15.143:0/2646904712 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f86000050d0 msgr2=0x7f86180b4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:13.896 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.880+0000 7f86177fe700 1 --2- 172.21.15.143:0/2646904712 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f86000050d0 0x7f86180b4650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:13.896 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.880+0000 7f86177fe700 1 -- 172.21.15.143:0/2646904712 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f86180ccea0 msgr2=0x7f86180b3540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:13.896 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.880+0000 7f86177fe700 1 --2- 172.21.15.143:0/2646904712 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f86180ccea0 0x7f86180b3540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:13.897 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.880+0000 7f86177fe700 1 -- 172.21.15.143:0/2646904712 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86180c5820 con 0x7f86180cddc0 2023-11-08T19:27:13.897 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.880+0000 7f861cc7f700 1 --2- 172.21.15.143:0/2646904712 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f86180ccea0 0x7f86180b3540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:27:13.897 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.881+0000 7f86177fe700 1 --2- 172.21.15.143:0/2646904712 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86180cddc0 0x7f86180b7300 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f8608007290 tx=0x7f86080075a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:13.897 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.881+0000 7f8614ff9700 1 -- 172.21.15.143:0/2646904712 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f860800f6c0 con 0x7f86180cddc0 2023-11-08T19:27:13.897 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.881+0000 7f8614ff9700 1 -- 172.21.15.143:0/2646904712 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f860800f820 con 0x7f86180cddc0 2023-11-08T19:27:13.898 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.881+0000 7f861e6e2700 1 -- 172.21.15.143:0/2646904712 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86180c5b00 con 0x7f86180cddc0 2023-11-08T19:27:13.898 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.881+0000 7f8614ff9700 1 -- 172.21.15.143:0/2646904712 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f860800fbd0 con 0x7f86180cddc0 2023-11-08T19:27:13.898 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.881+0000 7f861e6e2700 1 -- 172.21.15.143:0/2646904712 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86180cd320 con 0x7f86180cddc0 2023-11-08T19:27:13.898 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.882+0000 7f861e6e2700 1 -- 172.21.15.143:0/2646904712 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86180d0890 con 0x7f86180cddc0 2023-11-08T19:27:13.898 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.882+0000 7f8614ff9700 1 -- 172.21.15.143:0/2646904712 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f860800fd30 con 0x7f86180cddc0 2023-11-08T19:27:13.898 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.882+0000 7f8614ff9700 1 --2- 172.21.15.143:0/2646904712 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f860406af00 0x7f860406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:13.899 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.883+0000 7f8614ff9700 1 -- 172.21.15.143:0/2646904712 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6828+0+0 (secure 0 0 0) 0x7f8608089a50 con 0x7f86180cddc0 2023-11-08T19:27:13.899 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.883+0000 7f8617fff700 1 --2- 172.21.15.143:0/2646904712 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f860406af00 0x7f860406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:13.899 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.883+0000 7f8617fff700 1 --2- 172.21.15.143:0/2646904712 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f860406af00 0x7f860406d3c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f860c007fd0 tx=0x7f860c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:13.899 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:13.887+0000 7f8614ff9700 1 -- 172.21.15.143:0/2646904712 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8608054690 con 0x7f86180cddc0 2023-11-08T19:27:14.077 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.075+0000 7f861e6e2700 1 -- 172.21.15.143:0/2646904712 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f861804eee0 con 0x7f86180cddc0 2023-11-08T19:27:14.077 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.075+0000 7f8614ff9700 1 -- 172.21.15.143:0/2646904712 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v66) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8608057cb0 con 0x7f86180cddc0 2023-11-08T19:27:14.077 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2023-11-08T19:27:14.079 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.078+0000 7f861e6e2700 1 -- 172.21.15.143:0/2646904712 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f860406af00 msgr2=0x7f860406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:14.080 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.078+0000 7f861e6e2700 1 --2- 172.21.15.143:0/2646904712 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f860406af00 0x7f860406d3c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f860c007fd0 tx=0x7f860c009040 comp rx=0 tx=0).stop 2023-11-08T19:27:14.080 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.078+0000 7f861e6e2700 1 -- 172.21.15.143:0/2646904712 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86180cddc0 msgr2=0x7f86180b7300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:14.080 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.078+0000 7f861e6e2700 1 --2- 172.21.15.143:0/2646904712 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86180cddc0 0x7f86180b7300 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f8608007290 tx=0x7f86080075a0 comp rx=0 tx=0).stop 2023-11-08T19:27:14.080 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.078+0000 7f861e6e2700 1 -- 172.21.15.143:0/2646904712 shutdown_connections 2023-11-08T19:27:14.081 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.078+0000 7f861e6e2700 1 --2- 172.21.15.143:0/2646904712 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f860406af00 0x7f860406d3c0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:14.081 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.078+0000 7f861e6e2700 1 --2- 172.21.15.143:0/2646904712 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f86000050d0 0x7f86180b4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:14.081 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.078+0000 7f861e6e2700 1 --2- 172.21.15.143:0/2646904712 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f86180ccea0 0x7f86180b3540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:14.081 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.078+0000 7f861e6e2700 1 --2- 172.21.15.143:0/2646904712 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86180cddc0 0x7f86180b7300 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:14.081 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.078+0000 7f861e6e2700 1 -- 172.21.15.143:0/2646904712 >> 172.21.15.143:0/2646904712 conn(0x7f861808b000 msgr2=0x7f86180cb3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:14.081 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.078+0000 7f861e6e2700 1 -- 172.21.15.143:0/2646904712 shutdown_connections 2023-11-08T19:27:14.082 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.078+0000 7f861e6e2700 1 -- 172.21.15.143:0/2646904712 wait complete. 2023-11-08T19:27:14.091 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get snap-test-pool pg_num 2023-11-08T19:27:14.214 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.213+0000 7fac2c621700 1 Processor -- start 2023-11-08T19:27:14.215 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.213+0000 7fac2c621700 1 -- start start 2023-11-08T19:27:14.221 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.214+0000 7fac2c621700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fac2406bd20 0x7fac240d01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:14.221 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.214+0000 7fac2c621700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fac240b61e0 0x7fac240cc730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:14.222 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.214+0000 7fac2c621700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac240cc2a0 con 0x7fac2415a950 2023-11-08T19:27:14.222 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.214+0000 7fac2c621700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac240cc130 con 0x7fac240b61e0 2023-11-08T19:27:14.222 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.214+0000 7fac2c621700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac2406c1b0 con 0x7fac2406bd20 2023-11-08T19:27:14.222 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.214+0000 7fac2abbe700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fac240b61e0 0x7fac240cc730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:14.223 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.214+0000 7fac2a3bd700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fac2415a950 0x7fac24072370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:49196/0 (socket says 172.21.15.143:49196) 2023-11-08T19:27:14.223 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.214+0000 7fac2a3bd700 1 -- 172.21.15.143:0/2790046201 learned_addr learned my addr 172.21.15.143:0/2790046201 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:27:14.223 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.214+0000 7fac29bbc700 1 --2- 172.21.15.143:0/2790046201 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fac2406bd20 0x7fac240d01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:14.223 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.214+0000 7fac2abbe700 1 -- 172.21.15.143:0/2790046201 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fac2406bd20 msgr2=0x7fac240d01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:14.224 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.214+0000 7fac2abbe700 1 --2- 172.21.15.143:0/2790046201 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fac2406bd20 0x7fac240d01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:14.224 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.214+0000 7fac2abbe700 1 -- 172.21.15.143:0/2790046201 >> v1:172.21.15.143:6789/0 conn(0x7fac2415a950 legacy=0x7fac24072370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:14.224 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.214+0000 7fac2abbe700 1 -- 172.21.15.143:0/2790046201 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac2406c490 con 0x7fac240b61e0 2023-11-08T19:27:14.225 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.214+0000 7fac293bb700 1 -- 172.21.15.143:0/2790046201 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 715313576 0 0) 0x7fac240cc2a0 con 0x7fac2415a950 2023-11-08T19:27:14.225 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.214+0000 7fac2abbe700 1 --2- 172.21.15.143:0/2790046201 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fac240b61e0 0x7fac240cc730 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fac1c009d90 tx=0x7fac1c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=408101fb294f4c51 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:14.225 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.214+0000 7fac293bb700 1 -- 172.21.15.143:0/2790046201 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac1c00e070 con 0x7fac240b61e0 2023-11-08T19:27:14.225 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.215+0000 7fac293bb700 1 -- 172.21.15.143:0/2790046201 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac1c00bb20 con 0x7fac240b61e0 2023-11-08T19:27:14.226 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.215+0000 7fac293bb700 1 -- 172.21.15.143:0/2790046201 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac1c012430 con 0x7fac240b61e0 2023-11-08T19:27:14.226 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.215+0000 7fac2c621700 1 -- 172.21.15.143:0/2790046201 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fac240b61e0 msgr2=0x7fac240cc730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:14.226 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.215+0000 7fac2c621700 1 --2- 172.21.15.143:0/2790046201 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fac240b61e0 0x7fac240cc730 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fac1c009d90 tx=0x7fac1c00b330 comp rx=0 tx=0).stop 2023-11-08T19:27:14.226 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.215+0000 7fac2c621700 1 -- 172.21.15.143:0/2790046201 shutdown_connections 2023-11-08T19:27:14.227 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.215+0000 7fac2c621700 1 --2- 172.21.15.143:0/2790046201 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fac240b61e0 0x7fac240cc730 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:14.227 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.215+0000 7fac2c621700 1 --2- 172.21.15.143:0/2790046201 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fac2406bd20 0x7fac240d01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:14.227 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.215+0000 7fac2c621700 1 -- 172.21.15.143:0/2790046201 >> 172.21.15.143:0/2790046201 conn(0x7fac240cd2e0 msgr2=0x7fac240cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:14.227 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.215+0000 7fac2c621700 1 -- 172.21.15.143:0/2790046201 shutdown_connections 2023-11-08T19:27:14.228 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.215+0000 7fac2c621700 1 -- 172.21.15.143:0/2790046201 wait complete. 2023-11-08T19:27:14.228 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.216+0000 7fac2c621700 1 Processor -- start 2023-11-08T19:27:14.228 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.216+0000 7fac2c621700 1 -- start start 2023-11-08T19:27:14.228 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.217+0000 7fac2c621700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fac240b61e0 0x7fac2414fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:14.229 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.217+0000 7fac2c621700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fac2415a950 0x7fac240b46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:14.229 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.217+0000 7fac2c621700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fac240b4bf0 0x7fac2415c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:14.229 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.217+0000 7fac2c621700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac240b5a10 con 0x7fac2415a950 2023-11-08T19:27:14.229 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.217+0000 7fac2c621700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac241502c0 con 0x7fac240b61e0 2023-11-08T19:27:14.230 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.217+0000 7fac2c621700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac240c4b40 con 0x7fac240b4bf0 2023-11-08T19:27:14.230 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.217+0000 7fac29bbc700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fac2415a950 0x7fac240b46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:14.230 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.217+0000 7fac29bbc700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fac2415a950 0x7fac240b46b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53050/0 (socket says 172.21.15.143:53050) 2023-11-08T19:27:14.230 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.217+0000 7fac29bbc700 1 -- 172.21.15.143:0/1648363884 learned_addr learned my addr 172.21.15.143:0/1648363884 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:14.231 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.217+0000 7fac2a3bd700 1 --2- 172.21.15.143:0/1648363884 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fac240b61e0 0x7fac2414fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:14.231 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.217+0000 7fac2abbe700 1 --2- 172.21.15.143:0/1648363884 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fac240b4bf0 0x7fac2415c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:14.231 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.218+0000 7fac29bbc700 1 -- 172.21.15.143:0/1648363884 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fac240b4bf0 msgr2=0x7fac2415c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:14.231 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.218+0000 7fac29bbc700 1 --2- 172.21.15.143:0/1648363884 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fac240b4bf0 0x7fac2415c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:14.232 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.218+0000 7fac29bbc700 1 -- 172.21.15.143:0/1648363884 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fac240b61e0 msgr2=0x7fac2414fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:14.232 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.218+0000 7fac29bbc700 1 --2- 172.21.15.143:0/1648363884 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fac240b61e0 0x7fac2414fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:14.232 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.218+0000 7fac29bbc700 1 -- 172.21.15.143:0/1648363884 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac1c009a70 con 0x7fac2415a950 2023-11-08T19:27:14.232 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.218+0000 7fac2abbe700 1 --2- 172.21.15.143:0/1648363884 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fac240b4bf0 0x7fac2415c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:27:14.233 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.218+0000 7fac29bbc700 1 --2- 172.21.15.143:0/1648363884 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fac2415a950 0x7fac240b46b0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fac1400e9b0 tx=0x7fac1400ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:14.233 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.218+0000 7fac1affd700 1 -- 172.21.15.143:0/1648363884 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac14009710 con 0x7fac2415a950 2023-11-08T19:27:14.233 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.218+0000 7fac1affd700 1 -- 172.21.15.143:0/1648363884 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac1400c850 con 0x7fac2415a950 2023-11-08T19:27:14.233 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.218+0000 7fac1affd700 1 -- 172.21.15.143:0/1648363884 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac1400cbc0 con 0x7fac2415a950 2023-11-08T19:27:14.233 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.218+0000 7fac2c621700 1 -- 172.21.15.143:0/1648363884 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac240b5170 con 0x7fac2415a950 2023-11-08T19:27:14.234 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.218+0000 7fac2c621700 1 -- 172.21.15.143:0/1648363884 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac240b5730 con 0x7fac2415a950 2023-11-08T19:27:14.234 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.219+0000 7fac2c621700 1 -- 172.21.15.143:0/1648363884 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac2404eee0 con 0x7fac2415a950 2023-11-08T19:27:14.234 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.219+0000 7fac1affd700 1 -- 172.21.15.143:0/1648363884 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fac1400cd20 con 0x7fac2415a950 2023-11-08T19:27:14.234 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.220+0000 7fac1affd700 1 --2- 172.21.15.143:0/1648363884 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fac0806af30 0x7fac0806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:14.234 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.220+0000 7fac1affd700 1 -- 172.21.15.143:0/1648363884 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6828+0+0 (secure 0 0 0) 0x7fac14089b30 con 0x7fac2415a950 2023-11-08T19:27:14.235 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.220+0000 7fac2a3bd700 1 --2- 172.21.15.143:0/1648363884 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fac0806af30 0x7fac0806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:14.235 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.220+0000 7fac2a3bd700 1 --2- 172.21.15.143:0/1648363884 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fac0806af30 0x7fac0806d3f0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fac20000f60 tx=0x7fac20007e70 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:14.236 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.224+0000 7fac1affd700 1 -- 172.21.15.143:0/1648363884 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fac140546f0 con 0x7fac2415a950 2023-11-08T19:27:14.409 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.408+0000 7fac2c621700 1 -- 172.21.15.143:0/1648363884 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"} v 0) v1 -- 0x7fac240afff0 con 0x7fac2415a950 2023-11-08T19:27:14.410 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.409+0000 7fac1affd700 1 -- 172.21.15.143:0/1648363884 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"}]=0 v66) v1 ==== 103+0+11 (secure 0 0 0) 0x7fac14021020 con 0x7fac2415a950 2023-11-08T19:27:14.411 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2023-11-08T19:27:14.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.411+0000 7fac2c621700 1 -- 172.21.15.143:0/1648363884 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fac0806af30 msgr2=0x7fac0806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:14.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.411+0000 7fac2c621700 1 --2- 172.21.15.143:0/1648363884 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fac0806af30 0x7fac0806d3f0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fac20000f60 tx=0x7fac20007e70 comp rx=0 tx=0).stop 2023-11-08T19:27:14.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.411+0000 7fac2c621700 1 -- 172.21.15.143:0/1648363884 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fac2415a950 msgr2=0x7fac240b46b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:14.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.411+0000 7fac2c621700 1 --2- 172.21.15.143:0/1648363884 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fac2415a950 0x7fac240b46b0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fac1400e9b0 tx=0x7fac1400ecc0 comp rx=0 tx=0).stop 2023-11-08T19:27:14.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.411+0000 7fac2c621700 1 -- 172.21.15.143:0/1648363884 shutdown_connections 2023-11-08T19:27:14.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.411+0000 7fac2c621700 1 --2- 172.21.15.143:0/1648363884 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fac0806af30 0x7fac0806d3f0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:14.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.411+0000 7fac2c621700 1 --2- 172.21.15.143:0/1648363884 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fac240b4bf0 0x7fac2415c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:14.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.411+0000 7fac2c621700 1 --2- 172.21.15.143:0/1648363884 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fac240b61e0 0x7fac2414fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:14.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.411+0000 7fac2c621700 1 --2- 172.21.15.143:0/1648363884 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fac2415a950 0x7fac240b46b0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:14.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.411+0000 7fac2c621700 1 -- 172.21.15.143:0/1648363884 >> 172.21.15.143:0/1648363884 conn(0x7fac240cd2e0 msgr2=0x7fac2406b410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:14.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.411+0000 7fac2c621700 1 -- 172.21.15.143:0/1648363884 shutdown_connections 2023-11-08T19:27:14.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.411+0000 7fac2c621700 1 -- 172.21.15.143:0/1648363884 wait complete. 2023-11-08T19:27:14.424 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-08T19:27:14.548 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.546+0000 7f66b2ce0700 1 Processor -- start 2023-11-08T19:27:14.548 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.547+0000 7f66b2ce0700 1 -- start start 2023-11-08T19:27:14.551 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.547+0000 7f66b2ce0700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f66ac15bc80 0x7f66ac0b6280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:14.552 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.547+0000 7f66b2ce0700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f66ac06b0b0 0x7f66ac06c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:14.552 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.547+0000 7f66b2ce0700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f66ac0b22e0 0x7f66ac0b2750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:14.552 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.547+0000 7f66b2ce0700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66ac0ce290 con 0x7f66ac0b22e0 2023-11-08T19:27:14.552 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.547+0000 7f66b2ce0700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66ac15c060 con 0x7f66ac15bc80 2023-11-08T19:27:14.552 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.547+0000 7f66b2ce0700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66ac0b2c90 con 0x7f66ac06b0b0 2023-11-08T19:27:14.553 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.547+0000 7f66b127d700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f66ac0b22e0 0x7f66ac0b2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:14.553 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.547+0000 7f66b127d700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f66ac0b22e0 0x7f66ac0b2750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53060/0 (socket says 172.21.15.143:53060) 2023-11-08T19:27:14.553 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.547+0000 7f66b127d700 1 -- 172.21.15.143:0/4014790218 learned_addr learned my addr 172.21.15.143:0/4014790218 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:14.553 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.548+0000 7f66b0a7c700 1 --2- 172.21.15.143:0/4014790218 >> v2:172.21.15.174:3300/0 conn(0x7f66ac15bc80 0x7f66ac0b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:14.553 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.548+0000 7f66abfff700 1 --2- 172.21.15.143:0/4014790218 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f66ac06b0b0 0x7f66ac06c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:14.554 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.548+0000 7f66b127d700 1 -- 172.21.15.143:0/4014790218 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f66ac06b0b0 msgr2=0x7f66ac06c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:14.554 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.548+0000 7f66b127d700 1 --2- 172.21.15.143:0/4014790218 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f66ac06b0b0 0x7f66ac06c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:14.554 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.548+0000 7f66b127d700 1 -- 172.21.15.143:0/4014790218 >> v2:172.21.15.174:3300/0 conn(0x7f66ac15bc80 msgr2=0x7f66ac0b6280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:14.554 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.548+0000 7f66b127d700 1 --2- 172.21.15.143:0/4014790218 >> v2:172.21.15.174:3300/0 conn(0x7f66ac15bc80 0x7f66ac0b6280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:14.554 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.548+0000 7f66b127d700 1 -- 172.21.15.143:0/4014790218 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66ac0b67c0 con 0x7f66ac0b22e0 2023-11-08T19:27:14.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.548+0000 7f66b127d700 1 --2- 172.21.15.143:0/4014790218 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f66ac0b22e0 0x7f66ac0b2750 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f66ac0673b0 tx=0x7f66a400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f3ab53511b323a7 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:14.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.548+0000 7f66ab7fe700 1 -- 172.21.15.143:0/4014790218 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66a400e070 con 0x7f66ac0b22e0 2023-11-08T19:27:14.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.548+0000 7f66ab7fe700 1 -- 172.21.15.143:0/4014790218 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66a400b890 con 0x7f66ac0b22e0 2023-11-08T19:27:14.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.548+0000 7f66ab7fe700 1 -- 172.21.15.143:0/4014790218 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66a400bc40 con 0x7f66ac0b22e0 2023-11-08T19:27:14.556 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.549+0000 7f66b2ce0700 1 -- 172.21.15.143:0/4014790218 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f66ac0b22e0 msgr2=0x7f66ac0b2750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:14.556 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.549+0000 7f66b2ce0700 1 --2- 172.21.15.143:0/4014790218 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f66ac0b22e0 0x7f66ac0b2750 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f66ac0673b0 tx=0x7f66a400b330 comp rx=0 tx=0).stop 2023-11-08T19:27:14.556 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.549+0000 7f66b2ce0700 1 -- 172.21.15.143:0/4014790218 shutdown_connections 2023-11-08T19:27:14.556 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.549+0000 7f66b2ce0700 1 --2- 172.21.15.143:0/4014790218 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f66ac0b22e0 0x7f66ac0b2750 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:14.556 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.549+0000 7f66b2ce0700 1 --2- 172.21.15.143:0/4014790218 >> v2:172.21.15.174:3300/0 conn(0x7f66ac15bc80 0x7f66ac0b6280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:14.556 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.549+0000 7f66b2ce0700 1 --2- 172.21.15.143:0/4014790218 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f66ac06b0b0 0x7f66ac06c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:14.557 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.549+0000 7f66b2ce0700 1 -- 172.21.15.143:0/4014790218 >> 172.21.15.143:0/4014790218 conn(0x7f66ac0cd2e0 msgr2=0x7f66ac0cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:14.557 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.549+0000 7f66b2ce0700 1 -- 172.21.15.143:0/4014790218 shutdown_connections 2023-11-08T19:27:14.557 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.549+0000 7f66b2ce0700 1 -- 172.21.15.143:0/4014790218 wait complete. 2023-11-08T19:27:14.557 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.550+0000 7f66b2ce0700 1 Processor -- start 2023-11-08T19:27:14.558 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.550+0000 7f66b2ce0700 1 -- start start 2023-11-08T19:27:14.560 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.551+0000 7f66b2ce0700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f66ac06b0b0 0x7f66ac14fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:14.560 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.551+0000 7f66b2ce0700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f66ac0b22e0 0x7f66ac15d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:14.561 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.551+0000 7f66b2ce0700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f66ac15bc80 0x7f66ac0ae690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:14.561 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.551+0000 7f66b2ce0700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66ac0c8a30 con 0x7f66ac06b0b0 2023-11-08T19:27:14.561 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.551+0000 7f66b2ce0700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66ac1502a0 con 0x7f66ac0b22e0 2023-11-08T19:27:14.561 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.551+0000 7f66b2ce0700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66ac0ce840 con 0x7f66ac15bc80 2023-11-08T19:27:14.561 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.551+0000 7f66b0a7c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f66ac06b0b0 0x7f66ac14fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:14.562 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.551+0000 7f66b0a7c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f66ac06b0b0 0x7f66ac14fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53062/0 (socket says 172.21.15.143:53062) 2023-11-08T19:27:14.562 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.551+0000 7f66b0a7c700 1 -- 172.21.15.143:0/1993123463 learned_addr learned my addr 172.21.15.143:0/1993123463 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:14.562 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.551+0000 7f66abfff700 1 --2- 172.21.15.143:0/1993123463 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f66ac0b22e0 0x7f66ac15d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:14.562 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.551+0000 7f66b127d700 1 --2- 172.21.15.143:0/1993123463 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f66ac15bc80 0x7f66ac0ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:14.562 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.551+0000 7f66b0a7c700 1 -- 172.21.15.143:0/1993123463 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f66ac15bc80 msgr2=0x7f66ac0ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:14.563 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.551+0000 7f66b0a7c700 1 --2- 172.21.15.143:0/1993123463 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f66ac15bc80 0x7f66ac0ae690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:14.563 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.551+0000 7f66b0a7c700 1 -- 172.21.15.143:0/1993123463 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f66ac0b22e0 msgr2=0x7f66ac15d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:14.563 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.551+0000 7f66b0a7c700 1 --2- 172.21.15.143:0/1993123463 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f66ac0b22e0 0x7f66ac15d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:14.563 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.551+0000 7f66b0a7c700 1 -- 172.21.15.143:0/1993123463 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66a4009b30 con 0x7f66ac06b0b0 2023-11-08T19:27:14.563 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.552+0000 7f66b0a7c700 1 --2- 172.21.15.143:0/1993123463 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f66ac06b0b0 0x7f66ac14fd60 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f66a000e870 tx=0x7f66a000eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:14.564 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.552+0000 7f66a97fa700 1 -- 172.21.15.143:0/1993123463 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66a0009590 con 0x7f66ac06b0b0 2023-11-08T19:27:14.564 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.552+0000 7f66a97fa700 1 -- 172.21.15.143:0/1993123463 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66a0013680 con 0x7f66ac06b0b0 2023-11-08T19:27:14.564 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.552+0000 7f66b2ce0700 1 -- 172.21.15.143:0/1993123463 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66ac15b000 con 0x7f66ac06b0b0 2023-11-08T19:27:14.564 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.552+0000 7f66a97fa700 1 -- 172.21.15.143:0/1993123463 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66a00139f0 con 0x7f66ac06b0b0 2023-11-08T19:27:14.565 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.552+0000 7f66b2ce0700 1 -- 172.21.15.143:0/1993123463 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66ac0ad880 con 0x7f66ac06b0b0 2023-11-08T19:27:14.565 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.553+0000 7f66b2ce0700 1 -- 172.21.15.143:0/1993123463 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66ac0ceb20 con 0x7f66ac06b0b0 2023-11-08T19:27:14.565 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.553+0000 7f66a97fa700 1 -- 172.21.15.143:0/1993123463 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f66a0013b50 con 0x7f66ac06b0b0 2023-11-08T19:27:14.565 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.553+0000 7f66a97fa700 1 --2- 172.21.15.143:0/1993123463 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f669006af00 0x7f669006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:14.565 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.554+0000 7f66a97fa700 1 -- 172.21.15.143:0/1993123463 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6828+0+0 (secure 0 0 0) 0x7f66a0089bf0 con 0x7f66ac06b0b0 2023-11-08T19:27:14.566 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.554+0000 7f66abfff700 1 --2- 172.21.15.143:0/1993123463 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f669006af00 0x7f669006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:14.566 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.554+0000 7f66abfff700 1 --2- 172.21.15.143:0/1993123463 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f669006af00 0x7f669006d3c0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f66ac0cc6b0 tx=0x7f6698009210 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:14.566 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.558+0000 7f66a97fa700 1 -- 172.21.15.143:0/1993123463 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f66a0054840 con 0x7f66ac06b0b0 2023-11-08T19:27:14.750 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.748+0000 7f66b2ce0700 1 -- 172.21.15.143:0/1993123463 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f66ac04eee0 con 0x7f66ac06b0b0 2023-11-08T19:27:14.750 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.749+0000 7f66a97fa700 1 -- 172.21.15.143:0/1993123463 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v66) v1 ==== 104+0+11 (secure 0 0 0) 0x7f66a0017020 con 0x7f66ac06b0b0 2023-11-08T19:27:14.751 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2023-11-08T19:27:14.752 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.751+0000 7f66b2ce0700 1 -- 172.21.15.143:0/1993123463 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f669006af00 msgr2=0x7f669006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:14.753 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.751+0000 7f66b2ce0700 1 --2- 172.21.15.143:0/1993123463 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f669006af00 0x7f669006d3c0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f66ac0cc6b0 tx=0x7f6698009210 comp rx=0 tx=0).stop 2023-11-08T19:27:14.753 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.751+0000 7f66b2ce0700 1 -- 172.21.15.143:0/1993123463 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f66ac06b0b0 msgr2=0x7f66ac14fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:14.753 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.751+0000 7f66b2ce0700 1 --2- 172.21.15.143:0/1993123463 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f66ac06b0b0 0x7f66ac14fd60 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f66a000e870 tx=0x7f66a000eb80 comp rx=0 tx=0).stop 2023-11-08T19:27:14.753 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.751+0000 7f66b2ce0700 1 -- 172.21.15.143:0/1993123463 shutdown_connections 2023-11-08T19:27:14.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.751+0000 7f66b2ce0700 1 --2- 172.21.15.143:0/1993123463 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f669006af00 0x7f669006d3c0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:14.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.751+0000 7f66b2ce0700 1 --2- 172.21.15.143:0/1993123463 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f66ac15bc80 0x7f66ac0ae690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:14.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.751+0000 7f66b2ce0700 1 --2- 172.21.15.143:0/1993123463 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f66ac06b0b0 0x7f66ac14fd60 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:14.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.751+0000 7f66b2ce0700 1 --2- 172.21.15.143:0/1993123463 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f66ac0b22e0 0x7f66ac15d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:14.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.751+0000 7f66b2ce0700 1 -- 172.21.15.143:0/1993123463 >> 172.21.15.143:0/1993123463 conn(0x7f66ac0cd2e0 msgr2=0x7f66ac0b17b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:14.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.751+0000 7f66b2ce0700 1 -- 172.21.15.143:0/1993123463 shutdown_connections 2023-11-08T19:27:14.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.751+0000 7f66b2ce0700 1 -- 172.21.15.143:0/1993123463 wait complete. 2023-11-08T19:27:14.764 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-08T19:27:14.888 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.887+0000 7f0706559700 1 Processor -- start 2023-11-08T19:27:14.888 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.887+0000 7f0706559700 1 -- start start 2023-11-08T19:27:14.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.887+0000 7f0706559700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f070015dba0 0x7f070006c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:14.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.887+0000 7f0706559700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0700161890 con 0x7f07000b22e0 2023-11-08T19:27:14.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.887+0000 7f0706559700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f070006b740 con 0x7f070006b240 2023-11-08T19:27:14.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.887+0000 7f0706559700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f070006ba20 con 0x7f070015dba0 2023-11-08T19:27:14.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.888+0000 7f06fffff700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f070015dba0 0x7f070006c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:14.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.888+0000 7f0704af6700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f07000b22e0 0x7f07000b2700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:49226/0 (socket says 172.21.15.143:49226) 2023-11-08T19:27:14.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.888+0000 7f0704af6700 1 -- 172.21.15.143:0/1401441497 learned_addr learned my addr 172.21.15.143:0/1401441497 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:27:14.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.888+0000 7f06fffff700 1 -- 172.21.15.143:0/1401441497 >> v1:172.21.15.174:6789/0 conn(0x7f070006b240 legacy=0x7f07000caff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:14.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.888+0000 7f06fffff700 1 -- 172.21.15.143:0/1401441497 >> v1:172.21.15.143:6789/0 conn(0x7f07000b22e0 legacy=0x7f07000b2700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:14.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.888+0000 7f06feffd700 1 -- 172.21.15.143:0/1401441497 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3733597702 0 0) 0x7f0700161890 con 0x7f07000b22e0 2023-11-08T19:27:14.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.888+0000 7f06fffff700 1 -- 172.21.15.143:0/1401441497 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f070006bd00 con 0x7f070015dba0 2023-11-08T19:27:14.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.888+0000 7f06fffff700 1 --2- 172.21.15.143:0/1401441497 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f070015dba0 0x7f070006c610 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f06f4009840 tx=0x7f06f4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5cda798aa3832d8 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:14.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.888+0000 7f06feffd700 1 -- 172.21.15.143:0/1401441497 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06f4004230 con 0x7f070015dba0 2023-11-08T19:27:14.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.888+0000 7f06feffd700 1 -- 172.21.15.143:0/1401441497 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06f4004390 con 0x7f070015dba0 2023-11-08T19:27:14.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.888+0000 7f06feffd700 1 -- 172.21.15.143:0/1401441497 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06f40036a0 con 0x7f070015dba0 2023-11-08T19:27:14.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.889+0000 7f0706559700 1 -- 172.21.15.143:0/1401441497 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f070015dba0 msgr2=0x7f070006c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:14.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.889+0000 7f0706559700 1 --2- 172.21.15.143:0/1401441497 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f070015dba0 0x7f070006c610 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f06f4009840 tx=0x7f06f4009b50 comp rx=0 tx=0).stop 2023-11-08T19:27:14.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.889+0000 7f0706559700 1 -- 172.21.15.143:0/1401441497 shutdown_connections 2023-11-08T19:27:14.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.889+0000 7f0706559700 1 --2- 172.21.15.143:0/1401441497 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f070015dba0 0x7f070006c610 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:14.895 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.889+0000 7f0706559700 1 -- 172.21.15.143:0/1401441497 >> 172.21.15.143:0/1401441497 conn(0x7f07000cd2e0 msgr2=0x7f07000cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:14.895 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.889+0000 7f0706559700 1 -- 172.21.15.143:0/1401441497 shutdown_connections 2023-11-08T19:27:14.895 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.889+0000 7f0706559700 1 -- 172.21.15.143:0/1401441497 wait complete. 2023-11-08T19:27:14.898 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.890+0000 7f0706559700 1 Processor -- start 2023-11-08T19:27:14.898 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.891+0000 7f0706559700 1 -- start start 2023-11-08T19:27:14.898 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.891+0000 7f0706559700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f070006b240 0x7f07000ca5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:14.898 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.891+0000 7f0706559700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f07000b22e0 0x7f07000c5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:14.898 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.891+0000 7f0706559700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f070015dba0 0x7f07001e6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:14.899 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.891+0000 7f0706559700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07000b4b70 con 0x7f070006b240 2023-11-08T19:27:14.899 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.891+0000 7f0706559700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07000b55c0 con 0x7f070015dba0 2023-11-08T19:27:14.899 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.891+0000 7f0706559700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07000b58a0 con 0x7f07000b22e0 2023-11-08T19:27:14.899 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.891+0000 7f06fffff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f070006b240 0x7f07000ca5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:14.899 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.892+0000 7f06fffff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f070006b240 0x7f07000ca5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53076/0 (socket says 172.21.15.143:53076) 2023-11-08T19:27:14.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.892+0000 7f06fffff700 1 -- 172.21.15.143:0/1631993689 learned_addr learned my addr 172.21.15.143:0/1631993689 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:14.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.892+0000 7f06fffff700 1 -- 172.21.15.143:0/1631993689 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f07000b22e0 msgr2=0x7f07000c5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:14.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.892+0000 7f06fffff700 1 --2- 172.21.15.143:0/1631993689 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f07000b22e0 0x7f07000c5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:14.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.892+0000 7f06fffff700 1 -- 172.21.15.143:0/1631993689 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f070015dba0 msgr2=0x7f07001e6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:14.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.892+0000 7f06fffff700 1 --2- 172.21.15.143:0/1631993689 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f070015dba0 0x7f07001e6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:14.901 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.892+0000 7f06fffff700 1 -- 172.21.15.143:0/1631993689 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06f40094a0 con 0x7f070006b240 2023-11-08T19:27:14.901 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.892+0000 7f06fffff700 1 --2- 172.21.15.143:0/1631993689 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f070006b240 0x7f07000ca5b0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f06f40090b0 tx=0x7f06f40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:14.901 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.892+0000 7f06fcff9700 1 -- 172.21.15.143:0/1631993689 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06f4011630 con 0x7f070006b240 2023-11-08T19:27:14.901 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.892+0000 7f06fcff9700 1 -- 172.21.15.143:0/1631993689 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06f4011790 con 0x7f070006b240 2023-11-08T19:27:14.901 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.892+0000 7f0706559700 1 -- 172.21.15.143:0/1631993689 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07000b5b80 con 0x7f070006b240 2023-11-08T19:27:14.902 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.892+0000 7f06fcff9700 1 -- 172.21.15.143:0/1631993689 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06f4011b40 con 0x7f070006b240 2023-11-08T19:27:14.902 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.892+0000 7f0706559700 1 -- 172.21.15.143:0/1631993689 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07000b6140 con 0x7f070006b240 2023-11-08T19:27:14.902 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.893+0000 7f0706559700 1 -- 172.21.15.143:0/1631993689 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f070004eee0 con 0x7f070006b240 2023-11-08T19:27:14.902 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.894+0000 7f06fcff9700 1 -- 172.21.15.143:0/1631993689 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f06f4011ca0 con 0x7f070006b240 2023-11-08T19:27:14.902 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.894+0000 7f06fcff9700 1 --2- 172.21.15.143:0/1631993689 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f06ec06af00 0x7f06ec06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:14.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.894+0000 7f06fcff9700 1 -- 172.21.15.143:0/1631993689 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6828+0+0 (secure 0 0 0) 0x7f06f408b300 con 0x7f070006b240 2023-11-08T19:27:14.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.894+0000 7f06ff7fe700 1 --2- 172.21.15.143:0/1631993689 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f06ec06af00 0x7f06ec06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:14.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.895+0000 7f06ff7fe700 1 --2- 172.21.15.143:0/1631993689 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f06ec06af00 0x7f06ec06d3c0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f07000b4f00 tx=0x7f06f0007a20 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:14.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:14.898+0000 7f06fcff9700 1 -- 172.21.15.143:0/1631993689 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f06f4055fe0 con 0x7f070006b240 2023-11-08T19:27:15.087 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.085+0000 7f0706559700 1 -- 172.21.15.143:0/1631993689 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0700161610 con 0x7f070006b240 2023-11-08T19:27:15.087 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.086+0000 7f06fcff9700 1 -- 172.21.15.143:0/1631993689 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v66) v1 ==== 100+0+11 (secure 0 0 0) 0x7f06f4059600 con 0x7f070006b240 2023-11-08T19:27:15.088 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 64 2023-11-08T19:27:15.089 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.088+0000 7f0706559700 1 -- 172.21.15.143:0/1631993689 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f06ec06af00 msgr2=0x7f06ec06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:15.090 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.088+0000 7f0706559700 1 --2- 172.21.15.143:0/1631993689 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f06ec06af00 0x7f06ec06d3c0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f07000b4f00 tx=0x7f06f0007a20 comp rx=0 tx=0).stop 2023-11-08T19:27:15.090 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.088+0000 7f0706559700 1 -- 172.21.15.143:0/1631993689 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f070006b240 msgr2=0x7f07000ca5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:15.090 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.088+0000 7f0706559700 1 --2- 172.21.15.143:0/1631993689 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f070006b240 0x7f07000ca5b0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f06f40090b0 tx=0x7f06f40047c0 comp rx=0 tx=0).stop 2023-11-08T19:27:15.090 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.088+0000 7f0706559700 1 -- 172.21.15.143:0/1631993689 shutdown_connections 2023-11-08T19:27:15.091 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.088+0000 7f0706559700 1 --2- 172.21.15.143:0/1631993689 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f06ec06af00 0x7f06ec06d3c0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:15.091 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.088+0000 7f0706559700 1 --2- 172.21.15.143:0/1631993689 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f070015dba0 0x7f07001e6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:15.091 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.088+0000 7f0706559700 1 --2- 172.21.15.143:0/1631993689 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f070006b240 0x7f07000ca5b0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:15.091 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.088+0000 7f0706559700 1 --2- 172.21.15.143:0/1631993689 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f07000b22e0 0x7f07000c5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:15.091 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.088+0000 7f0706559700 1 -- 172.21.15.143:0/1631993689 >> 172.21.15.143:0/1631993689 conn(0x7f07000cd2e0 msgr2=0x7f07000b10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:15.092 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.088+0000 7f0706559700 1 -- 172.21.15.143:0/1631993689 shutdown_connections 2023-11-08T19:27:15.092 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.088+0000 7f0706559700 1 -- 172.21.15.143:0/1631993689 wait complete. 2023-11-08T19:27:15.101 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get test_data_pool pg_num 2023-11-08T19:27:15.227 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.226+0000 7f68fd08f700 1 Processor -- start 2023-11-08T19:27:15.228 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.226+0000 7f68fd08f700 1 -- start start 2023-11-08T19:27:15.231 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.227+0000 7f68fd08f700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f68f80b22e0 0x7f68f80b2700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:15.231 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.227+0000 7f68fd08f700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68f8161890 con 0x7f68f815dba0 2023-11-08T19:27:15.232 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.227+0000 7f68fd08f700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68f80b2c40 con 0x7f68f806b240 2023-11-08T19:27:15.232 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.227+0000 7f68fd08f700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68f806cd20 con 0x7f68f80b22e0 2023-11-08T19:27:15.232 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.227+0000 7f68f6d9d700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f68f815dba0 0x7f68f806c610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:49238/0 (socket says 172.21.15.143:49238) 2023-11-08T19:27:15.232 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.227+0000 7f68f6d9d700 1 -- 172.21.15.143:0/1572094490 learned_addr learned my addr 172.21.15.143:0/1572094490 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:27:15.232 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.227+0000 7f68f759e700 1 --2- 172.21.15.143:0/1572094490 >> v2:172.21.15.174:3300/0 conn(0x7f68f80b22e0 0x7f68f80b2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:15.233 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.227+0000 7f68f5d9b700 1 -- 172.21.15.143:0/1572094490 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3073716519 0 0) 0x7f68f8161890 con 0x7f68f815dba0 2023-11-08T19:27:15.233 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.227+0000 7f68f5d9b700 1 -- 172.21.15.143:0/1572094490 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68e00035b0 con 0x7f68f815dba0 2023-11-08T19:27:15.233 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.227+0000 7f68f5d9b700 1 -- 172.21.15.143:0/1572094490 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1550726442 0 0) 0x7f68e00035b0 con 0x7f68f815dba0 2023-11-08T19:27:15.233 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.227+0000 7f68f5d9b700 1 -- 172.21.15.143:0/1572094490 >> v2:172.21.15.174:3300/0 conn(0x7f68f80b22e0 msgr2=0x7f68f80b2700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:15.233 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.227+0000 7f68f5d9b700 1 --2- 172.21.15.143:0/1572094490 >> v2:172.21.15.174:3300/0 conn(0x7f68f80b22e0 0x7f68f80b2700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:15.234 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.227+0000 7f68f5d9b700 1 -- 172.21.15.143:0/1572094490 >> v1:172.21.15.174:6789/0 conn(0x7f68f806b240 legacy=0x7f68f80caff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:15.234 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.227+0000 7f68f5d9b700 1 -- 172.21.15.143:0/1572094490 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68f806b740 con 0x7f68f815dba0 2023-11-08T19:27:15.234 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.227+0000 7f68f5d9b700 1 -- 172.21.15.143:0/1572094490 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4147756303 0 0) 0x7f68e8001670 con 0x7f68f815dba0 2023-11-08T19:27:15.234 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.227+0000 7f68f5d9b700 1 -- 172.21.15.143:0/1572094490 >> v1:172.21.15.143:6789/0 conn(0x7f68f815dba0 legacy=0x7f68f806c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:15.234 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.227+0000 7f68f5d9b700 1 --2- 172.21.15.143:0/1572094490 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f68e0003a00 0x7f68e0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:15.235 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.227+0000 7f68f5d9b700 1 --2- 172.21.15.143:0/1572094490 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f68e0004850 0x7f68e0004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:15.235 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.227+0000 7f68f5d9b700 1 --2- 172.21.15.143:0/1572094490 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f68e0005210 0x7f68e000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:15.235 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.227+0000 7f68f5d9b700 1 -- 172.21.15.143:0/1572094490 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68f806cd20 con 0x7f68e0003a00 2023-11-08T19:27:15.235 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.227+0000 7f68f5d9b700 1 -- 172.21.15.143:0/1572094490 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68e000c0b0 con 0x7f68e0004850 2023-11-08T19:27:15.235 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.227+0000 7f68f5d9b700 1 -- 172.21.15.143:0/1572094490 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68e000c270 con 0x7f68e0005210 2023-11-08T19:27:15.236 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.227+0000 7f68f6d9d700 1 --2- 172.21.15.143:0/1572094490 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f68e0003a00 0x7f68e0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:15.236 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.228+0000 7f68f659c700 1 --2- 172.21.15.143:0/1572094490 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f68e0004850 0x7f68e0004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:15.236 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.228+0000 7f68f759e700 1 --2- 172.21.15.143:0/1572094490 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f68e0005210 0x7f68e000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:15.236 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.228+0000 7f68f6d9d700 1 -- 172.21.15.143:0/1572094490 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f68e0005210 msgr2=0x7f68e000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:15.236 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.228+0000 7f68f6d9d700 1 --2- 172.21.15.143:0/1572094490 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f68e0005210 0x7f68e000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:15.237 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.228+0000 7f68f6d9d700 1 -- 172.21.15.143:0/1572094490 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f68e0004850 msgr2=0x7f68e0004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:15.237 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.228+0000 7f68f6d9d700 1 --2- 172.21.15.143:0/1572094490 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f68e0004850 0x7f68e0004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:15.237 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.228+0000 7f68f6d9d700 1 -- 172.21.15.143:0/1572094490 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68e00035b0 con 0x7f68e0003a00 2023-11-08T19:27:15.237 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.228+0000 7f68f759e700 1 -- 172.21.15.143:0/1572094490 reap_dead start 2023-11-08T19:27:15.237 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.228+0000 7f68f6d9d700 1 --2- 172.21.15.143:0/1572094490 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f68e0003a00 0x7f68e0003e20 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f68e8001050 tx=0x7f68e0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac6863fdc15a5dd0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:15.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.228+0000 7f68f5d9b700 1 -- 172.21.15.143:0/1572094490 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68e800cd00 con 0x7f68e0003a00 2023-11-08T19:27:15.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.228+0000 7f68f5d9b700 1 -- 172.21.15.143:0/1572094490 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68e800ce60 con 0x7f68e0003a00 2023-11-08T19:27:15.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.228+0000 7f68f5d9b700 1 -- 172.21.15.143:0/1572094490 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68e8005ba0 con 0x7f68e0003a00 2023-11-08T19:27:15.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.228+0000 7f68fd08f700 1 -- 172.21.15.143:0/1572094490 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f68e0003a00 msgr2=0x7f68e0003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:15.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.228+0000 7f68fd08f700 1 --2- 172.21.15.143:0/1572094490 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f68e0003a00 0x7f68e0003e20 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f68e8001050 tx=0x7f68e0005880 comp rx=0 tx=0).stop 2023-11-08T19:27:15.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.228+0000 7f68fd08f700 1 -- 172.21.15.143:0/1572094490 shutdown_connections 2023-11-08T19:27:15.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.229+0000 7f68fd08f700 1 --2- 172.21.15.143:0/1572094490 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f68e0003a00 0x7f68e0003e20 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:15.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.229+0000 7f68fd08f700 1 -- 172.21.15.143:0/1572094490 >> 172.21.15.143:0/1572094490 conn(0x7f68f80cd2e0 msgr2=0x7f68f80cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:15.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.229+0000 7f68fd08f700 1 -- 172.21.15.143:0/1572094490 shutdown_connections 2023-11-08T19:27:15.240 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.229+0000 7f68fd08f700 1 -- 172.21.15.143:0/1572094490 wait complete. 2023-11-08T19:27:15.240 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.230+0000 7f68fd08f700 1 Processor -- start 2023-11-08T19:27:15.240 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.230+0000 7f68fd08f700 1 -- start start 2023-11-08T19:27:15.243 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.231+0000 7f68fd08f700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f68e0003a00 0x7f68f80c9a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:15.243 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.231+0000 7f68fd08f700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f68f8155cc0 0x7f68f80cc040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:15.243 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.231+0000 7f68fd08f700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f68f80cc580 0x7f68f80b2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:15.243 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.231+0000 7f68fd08f700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68f80b50e0 con 0x7f68f80cc580 2023-11-08T19:27:15.243 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.231+0000 7f68fd08f700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68f80cca30 con 0x7f68f8155cc0 2023-11-08T19:27:15.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.231+0000 7f68fd08f700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68f80b5500 con 0x7f68e0003a00 2023-11-08T19:27:15.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.231+0000 7f68f759e700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f68f80cc580 0x7f68f80b2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:15.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.231+0000 7f68f759e700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f68f80cc580 0x7f68f80b2ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53100/0 (socket says 172.21.15.143:53100) 2023-11-08T19:27:15.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.231+0000 7f68f759e700 1 -- 172.21.15.143:0/3843762477 learned_addr learned my addr 172.21.15.143:0/3843762477 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:15.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.231+0000 7f68f659c700 1 --2- 172.21.15.143:0/3843762477 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f68f8155cc0 0x7f68f80cc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:15.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.231+0000 7f68f6d9d700 1 --2- 172.21.15.143:0/3843762477 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f68e0003a00 0x7f68f80c9a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:15.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.231+0000 7f68f759e700 1 -- 172.21.15.143:0/3843762477 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f68e0003a00 msgr2=0x7f68f80c9a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:15.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.231+0000 7f68f759e700 1 --2- 172.21.15.143:0/3843762477 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f68e0003a00 0x7f68f80c9a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:15.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.231+0000 7f68f759e700 1 -- 172.21.15.143:0/3843762477 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f68f8155cc0 msgr2=0x7f68f80cc040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:15.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.231+0000 7f68f759e700 1 --2- 172.21.15.143:0/3843762477 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f68f8155cc0 0x7f68f80cc040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:15.246 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.231+0000 7f68f759e700 1 -- 172.21.15.143:0/3843762477 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68f80b4210 con 0x7f68f80cc580 2023-11-08T19:27:15.246 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.231+0000 7f68f759e700 1 --2- 172.21.15.143:0/3843762477 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f68f80cc580 0x7f68f80b2ff0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f68f80b2fc0 tx=0x7f68f0011e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:15.246 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.232+0000 7f68df7fe700 1 -- 172.21.15.143:0/3843762477 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68f000e040 con 0x7f68f80cc580 2023-11-08T19:27:15.246 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.232+0000 7f68df7fe700 1 -- 172.21.15.143:0/3843762477 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68f000f040 con 0x7f68f80cc580 2023-11-08T19:27:15.246 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.232+0000 7f68df7fe700 1 -- 172.21.15.143:0/3843762477 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68f000fc70 con 0x7f68f80cc580 2023-11-08T19:27:15.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.232+0000 7f68fd08f700 1 -- 172.21.15.143:0/3843762477 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68f80b4460 con 0x7f68f80cc580 2023-11-08T19:27:15.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.232+0000 7f68fd08f700 1 -- 172.21.15.143:0/3843762477 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68f80b4a20 con 0x7f68f80cc580 2023-11-08T19:27:15.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.233+0000 7f68fd08f700 1 -- 172.21.15.143:0/3843762477 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68f80b5220 con 0x7f68f80cc580 2023-11-08T19:27:15.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.233+0000 7f68df7fe700 1 -- 172.21.15.143:0/3843762477 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f68f0022460 con 0x7f68f80cc580 2023-11-08T19:27:15.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.233+0000 7f68df7fe700 1 --2- 172.21.15.143:0/3843762477 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f68e406af00 0x7f68e406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:15.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.233+0000 7f68df7fe700 1 -- 172.21.15.143:0/3843762477 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6828+0+0 (secure 0 0 0) 0x7f68f0089d80 con 0x7f68f80cc580 2023-11-08T19:27:15.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.234+0000 7f68f6d9d700 1 --2- 172.21.15.143:0/3843762477 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f68e406af00 0x7f68e406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:15.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.234+0000 7f68f6d9d700 1 --2- 172.21.15.143:0/3843762477 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f68e406af00 0x7f68e406d3c0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f68e8000c00 tx=0x7f68e800a000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:15.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.238+0000 7f68df7fe700 1 -- 172.21.15.143:0/3843762477 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f68f80b5220 con 0x7f68f80cc580 2023-11-08T19:27:15.430 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.426+0000 7f68fd08f700 1 -- 172.21.15.143:0/3843762477 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "test_data_pool", "var": "pg_num"} v 0) v1 -- 0x7f68f80b5220 con 0x7f68f80cc580 2023-11-08T19:27:15.430 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.426+0000 7f68df7fe700 1 -- 172.21.15.143:0/3843762477 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "test_data_pool", "var": "pg_num"}]=0 v66) v1 ==== 103+0+11 (secure 0 0 0) 0x7f68f00549d0 con 0x7f68f80cc580 2023-11-08T19:27:15.431 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2023-11-08T19:27:15.432 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.428+0000 7f68fd08f700 1 -- 172.21.15.143:0/3843762477 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f68e406af00 msgr2=0x7f68e406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:15.432 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.428+0000 7f68fd08f700 1 --2- 172.21.15.143:0/3843762477 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f68e406af00 0x7f68e406d3c0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f68e8000c00 tx=0x7f68e800a000 comp rx=0 tx=0).stop 2023-11-08T19:27:15.432 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.428+0000 7f68fd08f700 1 -- 172.21.15.143:0/3843762477 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f68f80cc580 msgr2=0x7f68f80b2ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:15.432 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.428+0000 7f68fd08f700 1 --2- 172.21.15.143:0/3843762477 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f68f80cc580 0x7f68f80b2ff0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f68f80b2fc0 tx=0x7f68f0011e20 comp rx=0 tx=0).stop 2023-11-08T19:27:15.433 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.428+0000 7f68fd08f700 1 -- 172.21.15.143:0/3843762477 shutdown_connections 2023-11-08T19:27:15.433 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.428+0000 7f68fd08f700 1 --2- 172.21.15.143:0/3843762477 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f68e406af00 0x7f68e406d3c0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:15.433 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.428+0000 7f68fd08f700 1 --2- 172.21.15.143:0/3843762477 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f68e0003a00 0x7f68f80c9a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:15.433 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.428+0000 7f68fd08f700 1 --2- 172.21.15.143:0/3843762477 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f68f80cc580 0x7f68f80b2ff0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:15.433 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.428+0000 7f68fd08f700 1 --2- 172.21.15.143:0/3843762477 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f68f8155cc0 0x7f68f80cc040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:15.434 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.428+0000 7f68fd08f700 1 -- 172.21.15.143:0/3843762477 >> 172.21.15.143:0/3843762477 conn(0x7f68f80cd2e0 msgr2=0x7f68f8072450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:15.434 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.428+0000 7f68fd08f700 1 -- 172.21.15.143:0/3843762477 shutdown_connections 2023-11-08T19:27:15.434 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.428+0000 7f68fd08f700 1 -- 172.21.15.143:0/3843762477 wait complete. 2023-11-08T19:27:15.442 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get test_metadata_pool pg_num 2023-11-08T19:27:15.566 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.564+0000 7fc245148700 1 Processor -- start 2023-11-08T19:27:15.566 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.564+0000 7fc245148700 1 -- start start 2023-11-08T19:27:15.568 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.565+0000 7fc245148700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fc24015dc20 0x7fc2400c4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:15.569 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.565+0000 7fc245148700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc2400cc9e0 0x7fc2400cce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:15.569 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.565+0000 7fc245148700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc24006ad90 con 0x7fc24015e000 2023-11-08T19:27:15.569 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.565+0000 7fc245148700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc24006bb50 con 0x7fc2400cc9e0 2023-11-08T19:27:15.569 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.565+0000 7fc245148700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2400cd390 con 0x7fc24015dc20 2023-11-08T19:27:15.569 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.565+0000 7fc23f59e700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc2400cc9e0 0x7fc2400cce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:15.570 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.565+0000 7fc23f59e700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc2400cc9e0 0x7fc2400cce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53108/0 (socket says 172.21.15.143:53108) 2023-11-08T19:27:15.570 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.565+0000 7fc23f59e700 1 -- 172.21.15.143:0/1141909108 learned_addr learned my addr 172.21.15.143:0/1141909108 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:15.570 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.565+0000 7fc23ed9d700 1 --2- 172.21.15.143:0/1141909108 >> v2:172.21.15.174:3300/0 conn(0x7fc24015dc20 0x7fc2400c4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:15.570 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.565+0000 7fc23f59e700 1 -- 172.21.15.143:0/1141909108 >> v2:172.21.15.174:3300/0 conn(0x7fc24015dc20 msgr2=0x7fc2400c4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:15.570 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.565+0000 7fc23f59e700 1 --2- 172.21.15.143:0/1141909108 >> v2:172.21.15.174:3300/0 conn(0x7fc24015dc20 0x7fc2400c4ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:15.571 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.565+0000 7fc23f59e700 1 -- 172.21.15.143:0/1141909108 >> v1:172.21.15.174:6789/0 conn(0x7fc24015e000 legacy=0x7fc2400c5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:15.571 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.565+0000 7fc23f59e700 1 -- 172.21.15.143:0/1141909108 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc24006b070 con 0x7fc2400cc9e0 2023-11-08T19:27:15.571 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.566+0000 7fc23f59e700 1 --2- 172.21.15.143:0/1141909108 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc2400cc9e0 0x7fc2400cce50 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fc240072b10 tx=0x7fc23800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a923a1381ff2e3b server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:15.571 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.566+0000 7fc23dd9b700 1 -- 172.21.15.143:0/1141909108 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc23800e070 con 0x7fc2400cc9e0 2023-11-08T19:27:15.571 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.566+0000 7fc23dd9b700 1 -- 172.21.15.143:0/1141909108 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc23800b9c0 con 0x7fc2400cc9e0 2023-11-08T19:27:15.572 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.566+0000 7fc23dd9b700 1 -- 172.21.15.143:0/1141909108 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc23800bd70 con 0x7fc2400cc9e0 2023-11-08T19:27:15.572 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.566+0000 7fc245148700 1 -- 172.21.15.143:0/1141909108 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc2400cc9e0 msgr2=0x7fc2400cce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:15.572 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.566+0000 7fc245148700 1 --2- 172.21.15.143:0/1141909108 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc2400cc9e0 0x7fc2400cce50 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fc240072b10 tx=0x7fc23800b330 comp rx=0 tx=0).stop 2023-11-08T19:27:15.572 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.566+0000 7fc245148700 1 -- 172.21.15.143:0/1141909108 shutdown_connections 2023-11-08T19:27:15.572 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.566+0000 7fc245148700 1 --2- 172.21.15.143:0/1141909108 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc2400cc9e0 0x7fc2400cce50 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:15.573 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.566+0000 7fc245148700 1 --2- 172.21.15.143:0/1141909108 >> v2:172.21.15.174:3300/0 conn(0x7fc24015dc20 0x7fc2400c4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:15.573 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.566+0000 7fc245148700 1 -- 172.21.15.143:0/1141909108 >> 172.21.15.143:0/1141909108 conn(0x7fc2400b1c70 msgr2=0x7fc2400b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:15.573 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.566+0000 7fc245148700 1 -- 172.21.15.143:0/1141909108 shutdown_connections 2023-11-08T19:27:15.573 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.566+0000 7fc245148700 1 -- 172.21.15.143:0/1141909108 wait complete. 2023-11-08T19:27:15.576 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.568+0000 7fc245148700 1 Processor -- start 2023-11-08T19:27:15.577 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.568+0000 7fc245148700 1 -- start start 2023-11-08T19:27:15.577 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.569+0000 7fc245148700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc2400cc9e0 0x7fc2401538e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:15.577 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.569+0000 7fc245148700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc24015dc20 0x7fc2400c7110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:15.577 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.569+0000 7fc245148700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc24015e000 0x7fc2400c9410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:15.577 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.569+0000 7fc245148700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2401555f0 con 0x7fc24015dc20 2023-11-08T19:27:15.578 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.569+0000 7fc245148700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2400c7fd0 con 0x7fc2400cc9e0 2023-11-08T19:27:15.578 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.569+0000 7fc245148700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc240153e20 con 0x7fc24015e000 2023-11-08T19:27:15.578 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.569+0000 7fc23e59c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc24015dc20 0x7fc2400c7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:15.578 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.569+0000 7fc23e59c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc24015dc20 0x7fc2400c7110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53112/0 (socket says 172.21.15.143:53112) 2023-11-08T19:27:15.578 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.569+0000 7fc23e59c700 1 -- 172.21.15.143:0/2579057060 learned_addr learned my addr 172.21.15.143:0/2579057060 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:15.579 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.569+0000 7fc23ed9d700 1 --2- 172.21.15.143:0/2579057060 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc2400cc9e0 0x7fc2401538e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:15.579 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.569+0000 7fc23f59e700 1 --2- 172.21.15.143:0/2579057060 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc24015e000 0x7fc2400c9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:15.579 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.569+0000 7fc23e59c700 1 -- 172.21.15.143:0/2579057060 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc24015e000 msgr2=0x7fc2400c9410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:15.579 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.569+0000 7fc23e59c700 1 --2- 172.21.15.143:0/2579057060 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc24015e000 0x7fc2400c9410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:15.580 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.569+0000 7fc23e59c700 1 -- 172.21.15.143:0/2579057060 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc2400cc9e0 msgr2=0x7fc2401538e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:15.580 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.569+0000 7fc23e59c700 1 --2- 172.21.15.143:0/2579057060 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc2400cc9e0 0x7fc2401538e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:15.580 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.569+0000 7fc23e59c700 1 -- 172.21.15.143:0/2579057060 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc238009b30 con 0x7fc24015dc20 2023-11-08T19:27:15.580 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.569+0000 7fc23ed9d700 1 --2- 172.21.15.143:0/2579057060 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc2400cc9e0 0x7fc2401538e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:27:15.580 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.569+0000 7fc23f59e700 1 --2- 172.21.15.143:0/2579057060 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc24015e000 0x7fc2400c9410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:27:15.581 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.569+0000 7fc23e59c700 1 --2- 172.21.15.143:0/2579057060 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc24015dc20 0x7fc2400c7110 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fc23000c500 tx=0x7fc23000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:15.581 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.569+0000 7fc2277fe700 1 -- 172.21.15.143:0/2579057060 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2300052d0 con 0x7fc24015dc20 2023-11-08T19:27:15.581 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.569+0000 7fc2277fe700 1 -- 172.21.15.143:0/2579057060 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc230005430 con 0x7fc24015dc20 2023-11-08T19:27:15.581 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.569+0000 7fc245148700 1 -- 172.21.15.143:0/2579057060 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2400c7a10 con 0x7fc24015dc20 2023-11-08T19:27:15.581 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.569+0000 7fc2277fe700 1 -- 172.21.15.143:0/2579057060 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc230006aa0 con 0x7fc24015dc20 2023-11-08T19:27:15.582 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.569+0000 7fc245148700 1 -- 172.21.15.143:0/2579057060 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc240155210 con 0x7fc24015dc20 2023-11-08T19:27:15.582 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.570+0000 7fc245148700 1 -- 172.21.15.143:0/2579057060 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc240155730 con 0x7fc24015dc20 2023-11-08T19:27:15.582 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.571+0000 7fc2277fe700 1 -- 172.21.15.143:0/2579057060 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc230003680 con 0x7fc24015dc20 2023-11-08T19:27:15.582 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.571+0000 7fc2277fe700 1 --2- 172.21.15.143:0/2579057060 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc22c06af00 0x7fc22c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:15.582 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.571+0000 7fc2277fe700 1 -- 172.21.15.143:0/2579057060 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6828+0+0 (secure 0 0 0) 0x7fc230088ce0 con 0x7fc24015dc20 2023-11-08T19:27:15.583 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.571+0000 7fc23ed9d700 1 --2- 172.21.15.143:0/2579057060 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc22c06af00 0x7fc22c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:15.583 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.572+0000 7fc23ed9d700 1 --2- 172.21.15.143:0/2579057060 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc22c06af00 0x7fc22c06d3c0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fc234009a00 tx=0x7fc234008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:15.583 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.575+0000 7fc2277fe700 1 -- 172.21.15.143:0/2579057060 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc240155730 con 0x7fc24015dc20 2023-11-08T19:27:15.765 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.764+0000 7fc245148700 1 -- 172.21.15.143:0/2579057060 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "test_metadata_pool", "var": "pg_num"} v 0) v1 -- 0x7fc2401553d0 con 0x7fc24015dc20 2023-11-08T19:27:15.766 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.764+0000 7fc2277fe700 1 -- 172.21.15.143:0/2579057060 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "test_metadata_pool", "var": "pg_num"}]=0 v66) v1 ==== 107+0+11 (secure 0 0 0) 0x7fc230053930 con 0x7fc24015dc20 2023-11-08T19:27:15.766 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2023-11-08T19:27:15.768 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.766+0000 7fc245148700 1 -- 172.21.15.143:0/2579057060 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc22c06af00 msgr2=0x7fc22c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:15.768 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.766+0000 7fc245148700 1 --2- 172.21.15.143:0/2579057060 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc22c06af00 0x7fc22c06d3c0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fc234009a00 tx=0x7fc234008040 comp rx=0 tx=0).stop 2023-11-08T19:27:15.768 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.766+0000 7fc245148700 1 -- 172.21.15.143:0/2579057060 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc24015dc20 msgr2=0x7fc2400c7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:15.769 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.766+0000 7fc245148700 1 --2- 172.21.15.143:0/2579057060 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc24015dc20 0x7fc2400c7110 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fc23000c500 tx=0x7fc23000c8c0 comp rx=0 tx=0).stop 2023-11-08T19:27:15.769 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.766+0000 7fc245148700 1 -- 172.21.15.143:0/2579057060 shutdown_connections 2023-11-08T19:27:15.769 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.766+0000 7fc245148700 1 --2- 172.21.15.143:0/2579057060 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc22c06af00 0x7fc22c06d3c0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:15.769 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.766+0000 7fc245148700 1 --2- 172.21.15.143:0/2579057060 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc24015e000 0x7fc2400c9410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:15.769 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.766+0000 7fc245148700 1 --2- 172.21.15.143:0/2579057060 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc2400cc9e0 0x7fc2401538e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:15.770 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.766+0000 7fc245148700 1 --2- 172.21.15.143:0/2579057060 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc24015dc20 0x7fc2400c7110 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:15.770 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.766+0000 7fc245148700 1 -- 172.21.15.143:0/2579057060 >> 172.21.15.143:0/2579057060 conn(0x7fc2400b1c70 msgr2=0x7fc2400ca4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:15.770 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.767+0000 7fc245148700 1 -- 172.21.15.143:0/2579057060 shutdown_connections 2023-11-08T19:27:15.770 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.767+0000 7fc245148700 1 -- 172.21.15.143:0/2579057060 wait complete. 2023-11-08T19:27:15.780 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:27:15.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.898+0000 7f9338161700 1 Processor -- start 2023-11-08T19:27:15.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.899+0000 7f9338161700 1 -- start start 2023-11-08T19:27:15.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.899+0000 7f9338161700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f93301618f0 0x7f933006ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:15.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.899+0000 7f9338161700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f933006b150 0x7f93300cb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:15.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.899+0000 7f9338161700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9330161df0 con 0x7f933015dd00 2023-11-08T19:27:15.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.899+0000 7f9338161700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f933006b600 con 0x7f93301618f0 2023-11-08T19:27:15.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.899+0000 7f9338161700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93300cad50 con 0x7f933006b150 2023-11-08T19:27:15.904 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.900+0000 7f93356fc700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f93301618f0 0x7f933006ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:15.904 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.900+0000 7f93356fc700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f93301618f0 0x7f933006ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53120/0 (socket says 172.21.15.143:53120) 2023-11-08T19:27:15.904 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.900+0000 7f93356fc700 1 -- 172.21.15.143:0/364137964 learned_addr learned my addr 172.21.15.143:0/364137964 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:15.904 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.900+0000 7f93366fe700 1 --2- 172.21.15.143:0/364137964 >> v2:172.21.15.174:3300/0 conn(0x7f933006b150 0x7f93300cb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:15.904 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.900+0000 7f93356fc700 1 -- 172.21.15.143:0/364137964 >> v2:172.21.15.174:3300/0 conn(0x7f933006b150 msgr2=0x7f93300cb1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:15.905 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.900+0000 7f93356fc700 1 --2- 172.21.15.143:0/364137964 >> v2:172.21.15.174:3300/0 conn(0x7f933006b150 0x7f93300cb1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:15.905 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.900+0000 7f93356fc700 1 -- 172.21.15.143:0/364137964 >> v1:172.21.15.174:6789/0 conn(0x7f933015dd00 legacy=0x7f93300cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:15.905 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.900+0000 7f93356fc700 1 -- 172.21.15.143:0/364137964 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93301620d0 con 0x7f93301618f0 2023-11-08T19:27:15.905 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.900+0000 7f93366fe700 1 --2- 172.21.15.143:0/364137964 >> v2:172.21.15.174:3300/0 conn(0x7f933006b150 0x7f93300cb1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:27:15.905 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.900+0000 7f93356fc700 1 --2- 172.21.15.143:0/364137964 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f93301618f0 0x7f933006ac10 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f9330099f70 tx=0x7f932c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3b632092e48ddb9 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:15.906 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.900+0000 7f9334efb700 1 -- 172.21.15.143:0/364137964 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f932c004230 con 0x7f93301618f0 2023-11-08T19:27:15.906 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.900+0000 7f9334efb700 1 -- 172.21.15.143:0/364137964 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f932c004390 con 0x7f93301618f0 2023-11-08T19:27:15.906 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.900+0000 7f9334efb700 1 -- 172.21.15.143:0/364137964 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f932c0036a0 con 0x7f93301618f0 2023-11-08T19:27:15.906 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.901+0000 7f9338161700 1 -- 172.21.15.143:0/364137964 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f93301618f0 msgr2=0x7f933006ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:15.907 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.901+0000 7f9338161700 1 --2- 172.21.15.143:0/364137964 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f93301618f0 0x7f933006ac10 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f9330099f70 tx=0x7f932c009bd0 comp rx=0 tx=0).stop 2023-11-08T19:27:15.907 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.901+0000 7f9338161700 1 -- 172.21.15.143:0/364137964 shutdown_connections 2023-11-08T19:27:15.907 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.901+0000 7f9338161700 1 --2- 172.21.15.143:0/364137964 >> v2:172.21.15.174:3300/0 conn(0x7f933006b150 0x7f93300cb1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:15.907 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.901+0000 7f9338161700 1 --2- 172.21.15.143:0/364137964 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f93301618f0 0x7f933006ac10 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:15.907 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.901+0000 7f9338161700 1 -- 172.21.15.143:0/364137964 >> 172.21.15.143:0/364137964 conn(0x7f9330099ab0 msgr2=0x7f93300d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:15.908 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.901+0000 7f9338161700 1 -- 172.21.15.143:0/364137964 shutdown_connections 2023-11-08T19:27:15.908 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.901+0000 7f9338161700 1 -- 172.21.15.143:0/364137964 wait complete. 2023-11-08T19:27:15.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.902+0000 7f9338161700 1 Processor -- start 2023-11-08T19:27:15.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.902+0000 7f9338161700 1 -- start start 2023-11-08T19:27:15.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.903+0000 7f9338161700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f933006b150 0x7f9330160e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:15.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.903+0000 7f9338161700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f933015dd00 0x7f93300c69c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:15.912 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.903+0000 7f9338161700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f93301618f0 0x7f93301f7630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:15.912 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.903+0000 7f9338161700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93301f7b70 con 0x7f933015dd00 2023-11-08T19:27:15.912 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.903+0000 7f9338161700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93300c70c0 con 0x7f933006b150 2023-11-08T19:27:15.912 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.903+0000 7f9338161700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93301600d0 con 0x7f93301618f0 2023-11-08T19:27:15.912 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.903+0000 7f93356fc700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f933015dd00 0x7f93300c69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:15.913 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.903+0000 7f93356fc700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f933015dd00 0x7f93300c69c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53126/0 (socket says 172.21.15.143:53126) 2023-11-08T19:27:15.913 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.903+0000 7f93356fc700 1 -- 172.21.15.143:0/2743589843 learned_addr learned my addr 172.21.15.143:0/2743589843 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:15.913 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.903+0000 7f9335efd700 1 --2- 172.21.15.143:0/2743589843 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f933006b150 0x7f9330160e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:15.913 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.903+0000 7f93366fe700 1 --2- 172.21.15.143:0/2743589843 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f93301618f0 0x7f93301f7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:15.913 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.903+0000 7f93356fc700 1 -- 172.21.15.143:0/2743589843 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f93301618f0 msgr2=0x7f93301f7630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:15.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.903+0000 7f93356fc700 1 --2- 172.21.15.143:0/2743589843 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f93301618f0 0x7f93301f7630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:15.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.903+0000 7f93356fc700 1 -- 172.21.15.143:0/2743589843 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f933006b150 msgr2=0x7f9330160e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:15.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.903+0000 7f93356fc700 1 --2- 172.21.15.143:0/2743589843 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f933006b150 0x7f9330160e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:15.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.903+0000 7f93356fc700 1 -- 172.21.15.143:0/2743589843 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f932c009550 con 0x7f933015dd00 2023-11-08T19:27:15.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.903+0000 7f93356fc700 1 --2- 172.21.15.143:0/2743589843 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f933015dd00 0x7f93300c69c0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f9330070a90 tx=0x7f932c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:15.915 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.904+0000 7f93267fc700 1 -- 172.21.15.143:0/2743589843 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f932c003ea0 con 0x7f933015dd00 2023-11-08T19:27:15.915 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.904+0000 7f93267fc700 1 -- 172.21.15.143:0/2743589843 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f932c011630 con 0x7f933015dd00 2023-11-08T19:27:15.915 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.904+0000 7f9338161700 1 -- 172.21.15.143:0/2743589843 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93301603b0 con 0x7f933015dd00 2023-11-08T19:27:15.915 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.904+0000 7f93267fc700 1 -- 172.21.15.143:0/2743589843 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f932c0119e0 con 0x7f933015dd00 2023-11-08T19:27:15.915 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.904+0000 7f9338161700 1 -- 172.21.15.143:0/2743589843 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9330160970 con 0x7f933015dd00 2023-11-08T19:27:15.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.904+0000 7f9338161700 1 -- 172.21.15.143:0/2743589843 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93300aff60 con 0x7f933015dd00 2023-11-08T19:27:15.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.905+0000 7f93267fc700 1 -- 172.21.15.143:0/2743589843 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f932c011b40 con 0x7f933015dd00 2023-11-08T19:27:15.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.905+0000 7f93267fc700 1 --2- 172.21.15.143:0/2743589843 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f931406af00 0x7f931406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:15.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.905+0000 7f93267fc700 1 -- 172.21.15.143:0/2743589843 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6828+0+0 (secure 0 0 0) 0x7f932c08b010 con 0x7f933015dd00 2023-11-08T19:27:15.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.906+0000 7f9335efd700 1 --2- 172.21.15.143:0/2743589843 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f931406af00 0x7f931406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:15.917 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.906+0000 7f9335efd700 1 --2- 172.21.15.143:0/2743589843 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f931406af00 0x7f931406d3c0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f933006dbb0 tx=0x7f9320007f10 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:15.917 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:15.910+0000 7f93267fc700 1 -- 172.21.15.143:0/2743589843 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f932c055cf0 con 0x7f933015dd00 2023-11-08T19:27:16.140 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.138+0000 7f9338161700 1 -- 172.21.15.143:0/2743589843 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f933004eee0 con 0x7f933015dd00 2023-11-08T19:27:16.140 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.139+0000 7f93267fc700 1 -- 172.21.15.143:0/2743589843 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+4923 (secure 0 0 0) 0x7f932c059310 con 0x7f933015dd00 2023-11-08T19:27:16.141 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:27:16.141 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":33,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5231,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6836/4083537882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":4083537882},{"type":"v1","addr":"172.21.15.174:6836","nonce":4083537882}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":5234,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/494923804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6835","nonce":494923804},{"type":"v1","addr":"172.21.15.174:6837","nonce":494923804}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":6009,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6834/3383638390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":3383638390},{"type":"v1","addr":"172.21.15.143:6834","nonce":3383638390}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T19:26:57.522785+0000","modified":"2023-11-08T19:27:00.536968+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6012},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6012":{"gid":6012,"name":"c","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.143:6835/2613282947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":2613282947},{"type":"v1","addr":"172.21.15.143:6835","nonce":2613282947}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-08T19:27:16.143 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.141+0000 7f9338161700 1 -- 172.21.15.143:0/2743589843 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f931406af00 msgr2=0x7f931406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:16.143 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.141+0000 7f9338161700 1 --2- 172.21.15.143:0/2743589843 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f931406af00 0x7f931406d3c0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f933006dbb0 tx=0x7f9320007f10 comp rx=0 tx=0).stop 2023-11-08T19:27:16.143 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.141+0000 7f9338161700 1 -- 172.21.15.143:0/2743589843 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f933015dd00 msgr2=0x7f93300c69c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:16.143 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.141+0000 7f9338161700 1 --2- 172.21.15.143:0/2743589843 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f933015dd00 0x7f93300c69c0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f9330070a90 tx=0x7f932c004750 comp rx=0 tx=0).stop 2023-11-08T19:27:16.144 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.141+0000 7f9338161700 1 -- 172.21.15.143:0/2743589843 shutdown_connections 2023-11-08T19:27:16.144 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.141+0000 7f9338161700 1 --2- 172.21.15.143:0/2743589843 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f931406af00 0x7f931406d3c0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:16.144 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.141+0000 7f9338161700 1 --2- 172.21.15.143:0/2743589843 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f93301618f0 0x7f93301f7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:16.144 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.141+0000 7f9338161700 1 --2- 172.21.15.143:0/2743589843 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f933006b150 0x7f9330160e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:16.144 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.141+0000 7f9338161700 1 --2- 172.21.15.143:0/2743589843 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f933015dd00 0x7f93300c69c0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:16.145 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.141+0000 7f9338161700 1 -- 172.21.15.143:0/2743589843 >> 172.21.15.143:0/2743589843 conn(0x7f9330099ab0 msgr2=0x7f93300d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:16.145 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.141+0000 7f9338161700 1 -- 172.21.15.143:0/2743589843 shutdown_connections 2023-11-08T19:27:16.145 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.141+0000 7f9338161700 1 -- 172.21.15.143:0/2743589843 wait complete. 2023-11-08T19:27:16.145 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 33 2023-11-08T19:27:16.154 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:27:16.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.275+0000 7f7bfcd68700 1 Processor -- start 2023-11-08T19:27:16.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.275+0000 7f7bfcd68700 1 -- start start 2023-11-08T19:27:16.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.276+0000 7f7bfcd68700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f7bf80b2920 0x7f7bf80cf510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:16.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.276+0000 7f7bfcd68700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f7bf80cbe20 0x7f7bf80cc290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:16.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.276+0000 7f7bfcd68700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bf80cf080 con 0x7f7bf815aa30 2023-11-08T19:27:16.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.276+0000 7f7bfcd68700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bf80cc7d0 con 0x7f7bf80cbe20 2023-11-08T19:27:16.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.276+0000 7f7bfcd68700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bf80ce470 con 0x7f7bf80b2920 2023-11-08T19:27:16.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.276+0000 7f7bf6d9d700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f7bf80cbe20 0x7f7bf80cc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:16.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.277+0000 7f7bf659c700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f7bf815aa30 0x7f7bf80b2210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:49286/0 (socket says 172.21.15.143:49286) 2023-11-08T19:27:16.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.277+0000 7f7bf659c700 1 -- 172.21.15.143:0/575934981 learned_addr learned my addr 172.21.15.143:0/575934981 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:27:16.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.277+0000 7f7bf5d9b700 1 --2- 172.21.15.143:0/575934981 >> v2:172.21.15.174:3300/0 conn(0x7f7bf80b2920 0x7f7bf80cf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:16.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.277+0000 7f7bf6d9d700 1 -- 172.21.15.143:0/575934981 >> v2:172.21.15.174:3300/0 conn(0x7f7bf80b2920 msgr2=0x7f7bf80cf510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:16.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.277+0000 7f7bf6d9d700 1 --2- 172.21.15.143:0/575934981 >> v2:172.21.15.174:3300/0 conn(0x7f7bf80b2920 0x7f7bf80cf510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:16.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.277+0000 7f7bf6d9d700 1 -- 172.21.15.143:0/575934981 >> v1:172.21.15.143:6789/0 conn(0x7f7bf815aa30 legacy=0x7f7bf80b2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:16.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.277+0000 7f7bf559a700 1 -- 172.21.15.143:0/575934981 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3581010250 0 0) 0x7f7bf80cf080 con 0x7f7bf815aa30 2023-11-08T19:27:16.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.277+0000 7f7bf6d9d700 1 -- 172.21.15.143:0/575934981 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bf80ce750 con 0x7f7bf80cbe20 2023-11-08T19:27:16.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.277+0000 7f7bf6d9d700 1 --2- 172.21.15.143:0/575934981 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7bf80cbe20 0x7f7bf80cc290 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f7bf80ce260 tx=0x7f7bf000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a845c2cbd6db2b1e server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:16.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.277+0000 7f7bf559a700 1 -- 172.21.15.143:0/575934981 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf000e070 con 0x7f7bf80cbe20 2023-11-08T19:27:16.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.277+0000 7f7bf559a700 1 -- 172.21.15.143:0/575934981 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bf000bbd0 con 0x7f7bf80cbe20 2023-11-08T19:27:16.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.277+0000 7f7bf559a700 1 -- 172.21.15.143:0/575934981 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf0012430 con 0x7f7bf80cbe20 2023-11-08T19:27:16.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.278+0000 7f7bfcd68700 1 -- 172.21.15.143:0/575934981 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7bf80cbe20 msgr2=0x7f7bf80cc290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:16.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.278+0000 7f7bfcd68700 1 --2- 172.21.15.143:0/575934981 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7bf80cbe20 0x7f7bf80cc290 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f7bf80ce260 tx=0x7f7bf000b330 comp rx=0 tx=0).stop 2023-11-08T19:27:16.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.278+0000 7f7bfcd68700 1 -- 172.21.15.143:0/575934981 shutdown_connections 2023-11-08T19:27:16.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.278+0000 7f7bfcd68700 1 --2- 172.21.15.143:0/575934981 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7bf80cbe20 0x7f7bf80cc290 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:16.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.278+0000 7f7bfcd68700 1 --2- 172.21.15.143:0/575934981 >> v2:172.21.15.174:3300/0 conn(0x7f7bf80b2920 0x7f7bf80cf510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:16.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.278+0000 7f7bfcd68700 1 -- 172.21.15.143:0/575934981 >> 172.21.15.143:0/575934981 conn(0x7f7bf8099ab0 msgr2=0x7f7bf80d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:16.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.278+0000 7f7bfcd68700 1 -- 172.21.15.143:0/575934981 shutdown_connections 2023-11-08T19:27:16.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.278+0000 7f7bfcd68700 1 -- 172.21.15.143:0/575934981 wait complete. 2023-11-08T19:27:16.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.279+0000 7f7bfcd68700 1 Processor -- start 2023-11-08T19:27:16.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.279+0000 7f7bfcd68700 1 -- start start 2023-11-08T19:27:16.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.280+0000 7f7bfcd68700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7bf80b2920 0x7f7bf8153720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:16.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.280+0000 7f7bfcd68700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7bf80cbe20 0x7f7bf814f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:16.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.280+0000 7f7bfcd68700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7bf815aa30 0x7f7bf814fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:16.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.280+0000 7f7bfcd68700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bf80b5ef0 con 0x7f7bf815aa30 2023-11-08T19:27:16.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.280+0000 7f7bfcd68700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bf80b61d0 con 0x7f7bf80cbe20 2023-11-08T19:27:16.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.280+0000 7f7bfcd68700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bf815b280 con 0x7f7bf80b2920 2023-11-08T19:27:16.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.280+0000 7f7bf6d9d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7bf815aa30 0x7f7bf814fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:16.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.280+0000 7f7bf6d9d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7bf815aa30 0x7f7bf814fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53142/0 (socket says 172.21.15.143:53142) 2023-11-08T19:27:16.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.280+0000 7f7bf6d9d700 1 -- 172.21.15.143:0/2847224277 learned_addr learned my addr 172.21.15.143:0/2847224277 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:16.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.280+0000 7f7bf5d9b700 1 --2- 172.21.15.143:0/2847224277 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7bf80cbe20 0x7f7bf814f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:16.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.280+0000 7f7bf659c700 1 --2- 172.21.15.143:0/2847224277 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7bf80b2920 0x7f7bf8153720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:16.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.280+0000 7f7bf6d9d700 1 -- 172.21.15.143:0/2847224277 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7bf80b2920 msgr2=0x7f7bf8153720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:16.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.280+0000 7f7bf6d9d700 1 --2- 172.21.15.143:0/2847224277 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7bf80b2920 0x7f7bf8153720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:16.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.280+0000 7f7bf6d9d700 1 -- 172.21.15.143:0/2847224277 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7bf80cbe20 msgr2=0x7f7bf814f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:16.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.280+0000 7f7bf6d9d700 1 --2- 172.21.15.143:0/2847224277 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7bf80cbe20 0x7f7bf814f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:16.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.280+0000 7f7bf6d9d700 1 -- 172.21.15.143:0/2847224277 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bf0009a70 con 0x7f7bf815aa30 2023-11-08T19:27:16.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.280+0000 7f7bf5d9b700 1 --2- 172.21.15.143:0/2847224277 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7bf80cbe20 0x7f7bf814f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:27:16.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.280+0000 7f7bf659c700 1 --2- 172.21.15.143:0/2847224277 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7bf80b2920 0x7f7bf8153720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:27:16.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.280+0000 7f7bf6d9d700 1 --2- 172.21.15.143:0/2847224277 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7bf815aa30 0x7f7bf814fbb0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f7bf8070ae0 tx=0x7f7bf0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:16.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.281+0000 7f7be6ffd700 1 -- 172.21.15.143:0/2847224277 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf0012430 con 0x7f7bf815aa30 2023-11-08T19:27:16.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.281+0000 7f7be6ffd700 1 -- 172.21.15.143:0/2847224277 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bf0020070 con 0x7f7bf815aa30 2023-11-08T19:27:16.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.281+0000 7f7bfcd68700 1 -- 172.21.15.143:0/2847224277 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bf815b560 con 0x7f7bf815aa30 2023-11-08T19:27:16.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.281+0000 7f7be6ffd700 1 -- 172.21.15.143:0/2847224277 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf0004280 con 0x7f7bf815aa30 2023-11-08T19:27:16.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.281+0000 7f7bfcd68700 1 -- 172.21.15.143:0/2847224277 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bf80c5a00 con 0x7f7bf815aa30 2023-11-08T19:27:16.293 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.281+0000 7f7bfcd68700 1 -- 172.21.15.143:0/2847224277 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bf815b0e0 con 0x7f7bf815aa30 2023-11-08T19:27:16.293 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.282+0000 7f7be6ffd700 1 -- 172.21.15.143:0/2847224277 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7bf00043e0 con 0x7f7bf815aa30 2023-11-08T19:27:16.293 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.282+0000 7f7be6ffd700 1 --2- 172.21.15.143:0/2847224277 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f7bd806af00 0x7f7bd806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:16.293 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.282+0000 7f7be6ffd700 1 -- 172.21.15.143:0/2847224277 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6828+0+0 (secure 0 0 0) 0x7f7bf0022070 con 0x7f7bf815aa30 2023-11-08T19:27:16.293 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.283+0000 7f7bf659c700 1 --2- 172.21.15.143:0/2847224277 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f7bd806af00 0x7f7bd806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:16.294 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.283+0000 7f7bf659c700 1 --2- 172.21.15.143:0/2847224277 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f7bd806af00 0x7f7bd806d3c0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f7bf80cdd20 tx=0x7f7bec009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:16.294 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.287+0000 7f7be6ffd700 1 -- 172.21.15.143:0/2847224277 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7bf815b0e0 con 0x7f7bf815aa30 2023-11-08T19:27:16.478 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.476+0000 7f7bfcd68700 1 -- 172.21.15.143:0/2847224277 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7bf815b0e0 con 0x7f7bf815aa30 2023-11-08T19:27:16.479 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.478+0000 7f7be6ffd700 1 -- 172.21.15.143:0/2847224277 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) v1 ==== 74+0+21891 (secure 0 0 0) 0x7f7bf815b0e0 con 0x7f7bf815aa30 2023-11-08T19:27:16.480 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:27:16.480 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":66,"fsid":"968483af-77fc-4068-ad7c-5c2a1a4613d7","created":"2023-11-08T19:24:32.826484+0000","modified":"2023-11-08T19:27:09.174117+0000","last_up_change":"2023-11-08T19:24:43.573880+0000","last_in_change":"2023-11-08T19:24:35.314525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:44.551530+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"snap-test-pool","create_time":"2023-11-08T19:26:02.874384+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":36,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-11-08T19:26:55.754227+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-11-08T19:26:56.486110+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"test_data_pool","create_time":"2023-11-08T19:27:05.601949+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":66,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_metadata_pool","create_time":"2023-11-08T19:27:06.536369+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bae98ec1-d94e-403d-a220-a522a53ea6f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6809","nonce":1487830566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6811","nonce":1487830566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6815","nonce":1487830566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6813","nonce":1487830566}]},"public_addr":"172.21.15.143:6809/1487830566","cluster_addr":"172.21.15.143:6811/1487830566","heartbeat_back_addr":"172.21.15.143:6815/1487830566","heartbeat_front_addr":"172.21.15.143:6813/1487830566","state":["exists","up"]},{"osd":1,"uuid":"5cb6538a-1ccb-4322-8633-09f66af0b7e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6801","nonce":787341397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6803","nonce":787341397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6807","nonce":787341397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6805","nonce":787341397}]},"public_addr":"172.21.15.143:6801/787341397","cluster_addr":"172.21.15.143:6803/787341397","heartbeat_back_addr":"172.21.15.143:6807/787341397","heartbeat_front_addr":"172.21.15.143:6805/787341397","state":["exists","up"]},{"osd":2,"uuid":"cdbf522c-60ef-4d60-9653-8dcb529c101a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6822","nonce":2301325597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6826","nonce":2301325597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6831","nonce":2301325597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6829","nonce":2301325597}]},"public_addr":"172.21.15.143:6822/2301325597","cluster_addr":"172.21.15.143:6826/2301325597","heartbeat_back_addr":"172.21.15.143:6831/2301325597","heartbeat_front_addr":"172.21.15.143:6829/2301325597","state":["exists","up"]},{"osd":3,"uuid":"4e2783cf-ebb9-4dd8-8245-bc1d17ce4325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6817","nonce":2021504428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6819","nonce":2021504428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6827","nonce":2021504428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6823","nonce":2021504428}]},"public_addr":"172.21.15.143:6817/2021504428","cluster_addr":"172.21.15.143:6819/2021504428","heartbeat_back_addr":"172.21.15.143:6827/2021504428","heartbeat_front_addr":"172.21.15.143:6823/2021504428","state":["exists","up"]},{"osd":4,"uuid":"691256fa-9b77-49b9-a346-7ae1adf0ca80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6801","nonce":1993380595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6803","nonce":1993380595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6807","nonce":1993380595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6805","nonce":1993380595}]},"public_addr":"172.21.15.174:6801/1993380595","cluster_addr":"172.21.15.174:6803/1993380595","heartbeat_back_addr":"172.21.15.174:6807/1993380595","heartbeat_front_addr":"172.21.15.174:6805/1993380595","state":["exists","up"]},{"osd":5,"uuid":"756fc352-48c9-46ce-b52e-0541fe6f8525","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6809","nonce":1777923303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6811","nonce":1777923303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6815","nonce":1777923303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6813","nonce":1777923303}]},"public_addr":"172.21.15.174:6809/1777923303","cluster_addr":"172.21.15.174:6811/1777923303","heartbeat_back_addr":"172.21.15.174:6815/1777923303","heartbeat_front_addr":"172.21.15.174:6813/1777923303","state":["exists","up"]},{"osd":6,"uuid":"fab2ac2b-ed97-4cde-9825-b480eabf8b8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6825","nonce":2755130316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6827","nonce":2755130316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6831","nonce":2755130316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6829","nonce":2755130316}]},"public_addr":"172.21.15.174:6825/2755130316","cluster_addr":"172.21.15.174:6827/2755130316","heartbeat_back_addr":"172.21.15.174:6831/2755130316","heartbeat_front_addr":"172.21.15.174:6829/2755130316","state":["exists","up"]},{"osd":7,"uuid":"3de9b797-2ab6-4d41-a863-7e7d638fd703","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6817","nonce":2847141411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6819","nonce":2847141411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6823","nonce":2847141411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6821","nonce":2847141411}]},"public_addr":"172.21.15.174:6817/2847141411","cluster_addr":"172.21.15.174:6819/2847141411","heartbeat_back_addr":"172.21.15.174:6823/2847141411","heartbeat_front_addr":"172.21.15.174:6821/2847141411","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.477076+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.496890+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.539305+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.454673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.538881+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.521294+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.520712+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.540404+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":11,"snaps":[{"begin":1,"length":1}]}],"new_removed_snaps":[{"pool":11,"snaps":[{"begin":1,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:27:16.481 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.480+0000 7f7bfcd68700 1 -- 172.21.15.143:0/2847224277 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f7bd806af00 msgr2=0x7f7bd806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:16.482 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.480+0000 7f7bfcd68700 1 --2- 172.21.15.143:0/2847224277 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f7bd806af00 0x7f7bd806d3c0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f7bf80cdd20 tx=0x7f7bec009040 comp rx=0 tx=0).stop 2023-11-08T19:27:16.482 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.480+0000 7f7bfcd68700 1 -- 172.21.15.143:0/2847224277 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7bf815aa30 msgr2=0x7f7bf814fbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:16.482 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.480+0000 7f7bfcd68700 1 --2- 172.21.15.143:0/2847224277 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7bf815aa30 0x7f7bf814fbb0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f7bf8070ae0 tx=0x7f7bf0004ad0 comp rx=0 tx=0).stop 2023-11-08T19:27:16.482 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.480+0000 7f7bfcd68700 1 -- 172.21.15.143:0/2847224277 shutdown_connections 2023-11-08T19:27:16.482 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.480+0000 7f7bfcd68700 1 --2- 172.21.15.143:0/2847224277 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f7bd806af00 0x7f7bd806d3c0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:16.483 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.480+0000 7f7bfcd68700 1 --2- 172.21.15.143:0/2847224277 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7bf80b2920 0x7f7bf8153720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:16.483 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.480+0000 7f7bfcd68700 1 --2- 172.21.15.143:0/2847224277 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7bf815aa30 0x7f7bf814fbb0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:16.483 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.480+0000 7f7bfcd68700 1 --2- 172.21.15.143:0/2847224277 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7bf80cbe20 0x7f7bf814f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:16.483 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.480+0000 7f7bfcd68700 1 -- 172.21.15.143:0/2847224277 >> 172.21.15.143:0/2847224277 conn(0x7f7bf8099ab0 msgr2=0x7f7bf80b0e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:16.483 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.480+0000 7f7bfcd68700 1 -- 172.21.15.143:0/2847224277 shutdown_connections 2023-11-08T19:27:16.484 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.480+0000 7f7bfcd68700 1 -- 172.21.15.143:0/2847224277 wait complete. 2023-11-08T19:27:16.494 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-08T19:27:16.494 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:27:16.615 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.614+0000 7efdfa7bf700 1 Processor -- start 2023-11-08T19:27:16.616 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.614+0000 7efdfa7bf700 1 -- start start 2023-11-08T19:27:16.618 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.615+0000 7efdfa7bf700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efdf40d1c70 0x7efdf40b14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:16.618 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.615+0000 7efdfa7bf700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7efdf40d03a0 0x7efdf40cc5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:16.618 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.615+0000 7efdfa7bf700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdf40b1bc0 con 0x7efdf415ba30 2023-11-08T19:27:16.619 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.615+0000 7efdfa7bf700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdf40b1ea0 con 0x7efdf40d03a0 2023-11-08T19:27:16.619 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.615+0000 7efdfa7bf700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdf40b2180 con 0x7efdf40d1c70 2023-11-08T19:27:16.619 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.615+0000 7efdf8d5c700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7efdf40d03a0 0x7efdf40cc5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:16.619 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.615+0000 7efdf3fff700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7efdf415ba30 0x7efdf40d1560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:49300/0 (socket says 172.21.15.143:49300) 2023-11-08T19:27:16.619 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.615+0000 7efdf8d5c700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7efdf40d03a0 0x7efdf40cc5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53150/0 (socket says 172.21.15.143:53150) 2023-11-08T19:27:16.620 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.615+0000 7efdf3fff700 1 -- 172.21.15.143:0/3095605824 learned_addr learned my addr 172.21.15.143:0/3095605824 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:27:16.620 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.615+0000 7efdf37fe700 1 --2- 172.21.15.143:0/3095605824 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efdf40d1c70 0x7efdf40b14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:16.620 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.615+0000 7efdf8d5c700 1 -- 172.21.15.143:0/3095605824 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efdf40d1c70 msgr2=0x7efdf40b14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:16.620 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.615+0000 7efdf8d5c700 1 --2- 172.21.15.143:0/3095605824 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efdf40d1c70 0x7efdf40b14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:16.620 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.615+0000 7efdf8d5c700 1 -- 172.21.15.143:0/3095605824 >> v1:172.21.15.143:6789/0 conn(0x7efdf415ba30 legacy=0x7efdf40d1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:16.621 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.615+0000 7efdf8d5c700 1 -- 172.21.15.143:0/3095605824 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdf40b2460 con 0x7efdf40d03a0 2023-11-08T19:27:16.621 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.615+0000 7efdf2ffd700 1 -- 172.21.15.143:0/3095605824 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2848639109 0 0) 0x7efdf40b1bc0 con 0x7efdf415ba30 2023-11-08T19:27:16.621 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.616+0000 7efdf8d5c700 1 --2- 172.21.15.143:0/3095605824 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efdf40d03a0 0x7efdf40cc5a0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7efdec009d90 tx=0x7efdec00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90adf6c257913bac server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:16.621 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.616+0000 7efdf2ffd700 1 -- 172.21.15.143:0/3095605824 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdec00e070 con 0x7efdf40d03a0 2023-11-08T19:27:16.621 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.616+0000 7efdf2ffd700 1 -- 172.21.15.143:0/3095605824 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdec00bbd0 con 0x7efdf40d03a0 2023-11-08T19:27:16.622 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.616+0000 7efdf2ffd700 1 -- 172.21.15.143:0/3095605824 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdec012430 con 0x7efdf40d03a0 2023-11-08T19:27:16.622 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.616+0000 7efdfa7bf700 1 -- 172.21.15.143:0/3095605824 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efdf40d03a0 msgr2=0x7efdf40cc5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:16.622 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.616+0000 7efdfa7bf700 1 --2- 172.21.15.143:0/3095605824 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efdf40d03a0 0x7efdf40cc5a0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7efdec009d90 tx=0x7efdec00b330 comp rx=0 tx=0).stop 2023-11-08T19:27:16.622 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.616+0000 7efdfa7bf700 1 -- 172.21.15.143:0/3095605824 shutdown_connections 2023-11-08T19:27:16.622 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.616+0000 7efdfa7bf700 1 --2- 172.21.15.143:0/3095605824 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efdf40d03a0 0x7efdf40cc5a0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:16.622 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.616+0000 7efdfa7bf700 1 --2- 172.21.15.143:0/3095605824 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efdf40d1c70 0x7efdf40b14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:16.623 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.616+0000 7efdfa7bf700 1 -- 172.21.15.143:0/3095605824 >> 172.21.15.143:0/3095605824 conn(0x7efdf408b000 msgr2=0x7efdf409ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:16.623 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.616+0000 7efdfa7bf700 1 -- 172.21.15.143:0/3095605824 shutdown_connections 2023-11-08T19:27:16.623 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.616+0000 7efdfa7bf700 1 -- 172.21.15.143:0/3095605824 wait complete. 2023-11-08T19:27:16.626 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.618+0000 7efdfa7bf700 1 Processor -- start 2023-11-08T19:27:16.626 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.618+0000 7efdfa7bf700 1 -- start start 2023-11-08T19:27:16.626 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.618+0000 7efdfa7bf700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efdf40d03a0 0x7efdf40c53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:16.626 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.619+0000 7efdfa7bf700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efdf40d1c70 0x7efdf41f7510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:16.627 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.619+0000 7efdfa7bf700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efdf415ba30 0x7efdf4159380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:16.627 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.619+0000 7efdfa7bf700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdf415dee0 con 0x7efdf40d03a0 2023-11-08T19:27:16.627 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.619+0000 7efdfa7bf700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdf415d8a0 con 0x7efdf415ba30 2023-11-08T19:27:16.627 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.619+0000 7efdfa7bf700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdf40c7d60 con 0x7efdf40d1c70 2023-11-08T19:27:16.627 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.619+0000 7efdf3fff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efdf40d03a0 0x7efdf40c53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:16.628 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.619+0000 7efdf3fff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efdf40d03a0 0x7efdf40c53e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53152/0 (socket says 172.21.15.143:53152) 2023-11-08T19:27:16.628 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.619+0000 7efdf3fff700 1 -- 172.21.15.143:0/1378318052 learned_addr learned my addr 172.21.15.143:0/1378318052 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:16.628 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.619+0000 7efdf37fe700 1 --2- 172.21.15.143:0/1378318052 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efdf40d1c70 0x7efdf41f7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:16.628 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.619+0000 7efdf8d5c700 1 --2- 172.21.15.143:0/1378318052 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efdf415ba30 0x7efdf4159380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:16.628 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.619+0000 7efdf3fff700 1 -- 172.21.15.143:0/1378318052 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efdf40d1c70 msgr2=0x7efdf41f7510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:16.629 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.619+0000 7efdf3fff700 1 --2- 172.21.15.143:0/1378318052 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efdf40d1c70 0x7efdf41f7510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:16.629 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.619+0000 7efdf3fff700 1 -- 172.21.15.143:0/1378318052 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efdf415ba30 msgr2=0x7efdf4159380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:16.629 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.619+0000 7efdf3fff700 1 --2- 172.21.15.143:0/1378318052 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efdf415ba30 0x7efdf4159380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:16.629 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.619+0000 7efdf3fff700 1 -- 172.21.15.143:0/1378318052 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdec009a70 con 0x7efdf40d03a0 2023-11-08T19:27:16.629 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.619+0000 7efdf37fe700 1 --2- 172.21.15.143:0/1378318052 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efdf40d1c70 0x7efdf41f7510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:27:16.629 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.619+0000 7efdf8d5c700 1 --2- 172.21.15.143:0/1378318052 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efdf415ba30 0x7efdf4159380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:27:16.630 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.619+0000 7efdf3fff700 1 --2- 172.21.15.143:0/1378318052 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efdf40d03a0 0x7efdf40c53e0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7efde800c550 tx=0x7efde800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:16.630 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.619+0000 7efdf0ff9700 1 -- 172.21.15.143:0/1378318052 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efde8004c80 con 0x7efdf40d03a0 2023-11-08T19:27:16.630 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.619+0000 7efdf0ff9700 1 -- 172.21.15.143:0/1378318052 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efde8004e00 con 0x7efdf40d03a0 2023-11-08T19:27:16.630 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.619+0000 7efdfa7bf700 1 -- 172.21.15.143:0/1378318052 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdf40c8020 con 0x7efdf40d03a0 2023-11-08T19:27:16.630 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.619+0000 7efdf0ff9700 1 -- 172.21.15.143:0/1378318052 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efde8007310 con 0x7efdf40d03a0 2023-11-08T19:27:16.631 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.619+0000 7efdfa7bf700 1 -- 172.21.15.143:0/1378318052 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdf41598c0 con 0x7efdf40d03a0 2023-11-08T19:27:16.631 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.620+0000 7efdfa7bf700 1 -- 172.21.15.143:0/1378318052 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdf40cc4c0 con 0x7efdf40d03a0 2023-11-08T19:27:16.631 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.621+0000 7efdf0ff9700 1 -- 172.21.15.143:0/1378318052 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7efde8007470 con 0x7efdf40d03a0 2023-11-08T19:27:16.631 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.621+0000 7efdf0ff9700 1 --2- 172.21.15.143:0/1378318052 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7efde006af00 0x7efde006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:16.631 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.621+0000 7efdf0ff9700 1 -- 172.21.15.143:0/1378318052 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6828+0+0 (secure 0 0 0) 0x7efde8089730 con 0x7efdf40d03a0 2023-11-08T19:27:16.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.621+0000 7efdf37fe700 1 --2- 172.21.15.143:0/1378318052 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7efde006af00 0x7efde006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:16.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.622+0000 7efdf37fe700 1 --2- 172.21.15.143:0/1378318052 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7efde006af00 0x7efde006d3c0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7efdf415f190 tx=0x7efde4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:16.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.625+0000 7efdf0ff9700 1 -- 172.21.15.143:0/1378318052 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7efde8054380 con 0x7efdf40d03a0 2023-11-08T19:27:16.860 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.858+0000 7efdfa7bf700 1 -- 172.21.15.143:0/1378318052 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efdf40c8210 con 0x7efdf40d03a0 2023-11-08T19:27:16.860 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.859+0000 7efdf0ff9700 1 -- 172.21.15.143:0/1378318052 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+4923 (secure 0 0 0) 0x7efde80579a0 con 0x7efdf40d03a0 2023-11-08T19:27:16.861 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:27:16.861 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":33,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5231,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6836/4083537882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":4083537882},{"type":"v1","addr":"172.21.15.174:6836","nonce":4083537882}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":5234,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/494923804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6835","nonce":494923804},{"type":"v1","addr":"172.21.15.174:6837","nonce":494923804}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":6009,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6834/3383638390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":3383638390},{"type":"v1","addr":"172.21.15.143:6834","nonce":3383638390}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T19:26:57.522785+0000","modified":"2023-11-08T19:27:00.536968+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6012},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6012":{"gid":6012,"name":"c","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.143:6835/2613282947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":2613282947},{"type":"v1","addr":"172.21.15.143:6835","nonce":2613282947}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-08T19:27:16.863 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.861+0000 7efdfa7bf700 1 -- 172.21.15.143:0/1378318052 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7efde006af00 msgr2=0x7efde006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:16.863 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.861+0000 7efdfa7bf700 1 --2- 172.21.15.143:0/1378318052 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7efde006af00 0x7efde006d3c0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7efdf415f190 tx=0x7efde4008040 comp rx=0 tx=0).stop 2023-11-08T19:27:16.863 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.861+0000 7efdfa7bf700 1 -- 172.21.15.143:0/1378318052 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efdf40d03a0 msgr2=0x7efdf40c53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:16.863 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.861+0000 7efdfa7bf700 1 --2- 172.21.15.143:0/1378318052 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efdf40d03a0 0x7efdf40c53e0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7efde800c550 tx=0x7efde800c910 comp rx=0 tx=0).stop 2023-11-08T19:27:16.863 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.861+0000 7efdfa7bf700 1 -- 172.21.15.143:0/1378318052 shutdown_connections 2023-11-08T19:27:16.864 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.861+0000 7efdfa7bf700 1 --2- 172.21.15.143:0/1378318052 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7efde006af00 0x7efde006d3c0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:16.864 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.861+0000 7efdfa7bf700 1 --2- 172.21.15.143:0/1378318052 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efdf415ba30 0x7efdf4159380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:16.864 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.861+0000 7efdfa7bf700 1 --2- 172.21.15.143:0/1378318052 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efdf40d03a0 0x7efdf40c53e0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:16.864 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.861+0000 7efdfa7bf700 1 --2- 172.21.15.143:0/1378318052 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efdf40d1c70 0x7efdf41f7510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:16.864 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.861+0000 7efdfa7bf700 1 -- 172.21.15.143:0/1378318052 >> 172.21.15.143:0/1378318052 conn(0x7efdf408b000 msgr2=0x7efdf40cc980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:16.865 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.861+0000 7efdfa7bf700 1 -- 172.21.15.143:0/1378318052 shutdown_connections 2023-11-08T19:27:16.865 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.861+0000 7efdfa7bf700 1 -- 172.21.15.143:0/1378318052 wait complete. 2023-11-08T19:27:16.865 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 33 2023-11-08T19:27:16.874 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:27:16.995 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.994+0000 7f3fe86a1700 1 Processor -- start 2023-11-08T19:27:16.995 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.994+0000 7f3fe86a1700 1 -- start start 2023-11-08T19:27:16.998 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.994+0000 7f3fe86a1700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f3fe015ba30 0x7f3fe00b14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:16.998 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.994+0000 7f3fe86a1700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f3fe00d03a0 0x7f3fe00d1560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:16.998 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.994+0000 7f3fe86a1700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3fe00d1aa0 0x7f3fe00ce2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:16.998 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.995+0000 7f3fe86a1700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fe00d1f10 con 0x7f3fe015ba30 2023-11-08T19:27:16.999 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.995+0000 7f3fe86a1700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fe00b1bc0 con 0x7f3fe00d03a0 2023-11-08T19:27:16.999 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.995+0000 7f3fe86a1700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe00b1ea0 con 0x7f3fe00d1aa0 2023-11-08T19:27:16.999 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.995+0000 7f3fe643d700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f3fe015ba30 0x7f3fe00b14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:16.999 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.995+0000 7f3fe643d700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f3fe015ba30 0x7f3fe00b14c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53160/0 (socket says 172.21.15.143:53160) 2023-11-08T19:27:16.999 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.995+0000 7f3fe643d700 1 -- 172.21.15.143:0/3809821429 learned_addr learned my addr 172.21.15.143:0/3809821429 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:17.000 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.995+0000 7f3fe5c3c700 1 --2- 172.21.15.143:0/3809821429 >> v2:172.21.15.174:3300/0 conn(0x7f3fe00d03a0 0x7f3fe00d1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:17.000 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.995+0000 7f3fe6c3e700 1 --2- 172.21.15.143:0/3809821429 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3fe00d1aa0 0x7f3fe00ce2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:17.000 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.995+0000 7f3fe643d700 1 -- 172.21.15.143:0/3809821429 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3fe00d1aa0 msgr2=0x7f3fe00ce2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:17.000 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.995+0000 7f3fe643d700 1 --2- 172.21.15.143:0/3809821429 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3fe00d1aa0 0x7f3fe00ce2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:17.000 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.995+0000 7f3fe643d700 1 -- 172.21.15.143:0/3809821429 >> v2:172.21.15.174:3300/0 conn(0x7f3fe00d03a0 msgr2=0x7f3fe00d1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:17.001 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.995+0000 7f3fe643d700 1 --2- 172.21.15.143:0/3809821429 >> v2:172.21.15.174:3300/0 conn(0x7f3fe00d03a0 0x7f3fe00d1560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:17.001 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.995+0000 7f3fe643d700 1 -- 172.21.15.143:0/3809821429 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fe00b2180 con 0x7f3fe015ba30 2023-11-08T19:27:17.001 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.995+0000 7f3fe643d700 1 --2- 172.21.15.143:0/3809821429 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3fe015ba30 0x7f3fe00b14c0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f3fe00cbe60 tx=0x7f3fd4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6cd3e10de196c9d server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:17.001 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.995+0000 7f3fe543b700 1 -- 172.21.15.143:0/3809821429 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd400b870 con 0x7f3fe015ba30 2023-11-08T19:27:17.001 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.995+0000 7f3fe543b700 1 -- 172.21.15.143:0/3809821429 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fd400b9d0 con 0x7f3fe015ba30 2023-11-08T19:27:17.002 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.995+0000 7f3fe543b700 1 -- 172.21.15.143:0/3809821429 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd400bd60 con 0x7f3fe015ba30 2023-11-08T19:27:17.002 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.996+0000 7f3fe86a1700 1 -- 172.21.15.143:0/3809821429 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3fe015ba30 msgr2=0x7f3fe00b14c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:17.002 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.996+0000 7f3fe86a1700 1 --2- 172.21.15.143:0/3809821429 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3fe015ba30 0x7f3fe00b14c0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f3fe00cbe60 tx=0x7f3fd4009bd0 comp rx=0 tx=0).stop 2023-11-08T19:27:17.002 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.996+0000 7f3fe86a1700 1 -- 172.21.15.143:0/3809821429 shutdown_connections 2023-11-08T19:27:17.002 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.996+0000 7f3fe86a1700 1 --2- 172.21.15.143:0/3809821429 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3fe00d1aa0 0x7f3fe00ce2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:17.003 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.996+0000 7f3fe86a1700 1 --2- 172.21.15.143:0/3809821429 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3fe015ba30 0x7f3fe00b14c0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:17.003 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.996+0000 7f3fe86a1700 1 --2- 172.21.15.143:0/3809821429 >> v2:172.21.15.174:3300/0 conn(0x7f3fe00d03a0 0x7f3fe00d1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:17.003 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.996+0000 7f3fe86a1700 1 -- 172.21.15.143:0/3809821429 >> 172.21.15.143:0/3809821429 conn(0x7f3fe008b000 msgr2=0x7f3fe009ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:17.003 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.996+0000 7f3fe86a1700 1 -- 172.21.15.143:0/3809821429 shutdown_connections 2023-11-08T19:27:17.003 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.996+0000 7f3fe86a1700 1 -- 172.21.15.143:0/3809821429 wait complete. 2023-11-08T19:27:17.006 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.997+0000 7f3fe86a1700 1 Processor -- start 2023-11-08T19:27:17.006 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.997+0000 7f3fe86a1700 1 -- start start 2023-11-08T19:27:17.007 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.998+0000 7f3fe86a1700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3fe00d03a0 0x7f3fe014f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:17.007 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.998+0000 7f3fe86a1700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3fe00d1aa0 0x7f3fe014c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:17.007 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.998+0000 7f3fe86a1700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3fe015ba30 0x7f3fe015e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:17.007 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.998+0000 7f3fe86a1700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe015e300 con 0x7f3fe00d03a0 2023-11-08T19:27:17.007 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.998+0000 7f3fe86a1700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe014f840 con 0x7f3fe00d1aa0 2023-11-08T19:27:17.008 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.998+0000 7f3fe86a1700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe014c550 con 0x7f3fe015ba30 2023-11-08T19:27:17.008 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.998+0000 7f3fe643d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3fe00d03a0 0x7f3fe014f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:17.008 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.998+0000 7f3fe643d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3fe00d03a0 0x7f3fe014f300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53166/0 (socket says 172.21.15.143:53166) 2023-11-08T19:27:17.008 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.998+0000 7f3fe643d700 1 -- 172.21.15.143:0/3024719356 learned_addr learned my addr 172.21.15.143:0/3024719356 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:17.008 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.998+0000 7f3fe6c3e700 1 --2- 172.21.15.143:0/3024719356 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3fe015ba30 0x7f3fe015e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:17.009 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.998+0000 7f3fe5c3c700 1 --2- 172.21.15.143:0/3024719356 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3fe00d1aa0 0x7f3fe014c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:17.009 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.998+0000 7f3fe643d700 1 -- 172.21.15.143:0/3024719356 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3fe015ba30 msgr2=0x7f3fe015e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:17.009 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.998+0000 7f3fe643d700 1 --2- 172.21.15.143:0/3024719356 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3fe015ba30 0x7f3fe015e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:17.009 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.998+0000 7f3fe643d700 1 -- 172.21.15.143:0/3024719356 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3fe00d1aa0 msgr2=0x7f3fe014c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:17.009 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.998+0000 7f3fe643d700 1 --2- 172.21.15.143:0/3024719356 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3fe00d1aa0 0x7f3fe014c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:17.010 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.998+0000 7f3fe643d700 1 -- 172.21.15.143:0/3024719356 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fd4009550 con 0x7f3fe00d03a0 2023-11-08T19:27:17.010 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.998+0000 7f3fe5c3c700 1 --2- 172.21.15.143:0/3024719356 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3fe00d1aa0 0x7f3fe014c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:27:17.010 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.998+0000 7f3fe643d700 1 --2- 172.21.15.143:0/3024719356 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3fe00d03a0 0x7f3fe014f300 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f3fd4000c00 tx=0x7f3fd4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:17.010 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.999+0000 7f3fd2ffd700 1 -- 172.21.15.143:0/3024719356 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd4004520 con 0x7f3fe00d03a0 2023-11-08T19:27:17.010 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.999+0000 7f3fd2ffd700 1 -- 172.21.15.143:0/3024719356 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fd401e070 con 0x7f3fe00d03a0 2023-11-08T19:27:17.011 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.999+0000 7f3fd2ffd700 1 -- 172.21.15.143:0/3024719356 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd40038c0 con 0x7f3fe00d03a0 2023-11-08T19:27:17.011 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.999+0000 7f3fe86a1700 1 -- 172.21.15.143:0/3024719356 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fe015d6f0 con 0x7f3fe00d03a0 2023-11-08T19:27:17.011 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.999+0000 7f3fe86a1700 1 -- 172.21.15.143:0/3024719356 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fe015dcb0 con 0x7f3fe00d03a0 2023-11-08T19:27:17.011 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:16.999+0000 7f3fe86a1700 1 -- 172.21.15.143:0/3024719356 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fe004eee0 con 0x7f3fe00d03a0 2023-11-08T19:27:17.011 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.000+0000 7f3fd2ffd700 1 -- 172.21.15.143:0/3024719356 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3fd4003a20 con 0x7f3fe00d03a0 2023-11-08T19:27:17.012 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.000+0000 7f3fd2ffd700 1 --2- 172.21.15.143:0/3024719356 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f3fc406af00 0x7f3fc406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:17.012 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.000+0000 7f3fd2ffd700 1 -- 172.21.15.143:0/3024719356 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6828+0+0 (secure 0 0 0) 0x7f3fd4020070 con 0x7f3fe00d03a0 2023-11-08T19:27:17.012 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.001+0000 7f3fe5c3c700 1 --2- 172.21.15.143:0/3024719356 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f3fc406af00 0x7f3fc406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:17.012 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.001+0000 7f3fe5c3c700 1 --2- 172.21.15.143:0/3024719356 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f3fc406af00 0x7f3fc406d3c0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f3fe00cfee0 tx=0x7f3fdc008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:17.012 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.005+0000 7f3fd2ffd700 1 -- 172.21.15.143:0/3024719356 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3fd4055bb0 con 0x7f3fe00d03a0 2023-11-08T19:27:17.237 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.236+0000 7f3fe86a1700 1 -- 172.21.15.143:0/3024719356 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3fe00ae720 con 0x7f3fe00d03a0 2023-11-08T19:27:17.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.236+0000 7f3fd2ffd700 1 -- 172.21.15.143:0/3024719356 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+4923 (secure 0 0 0) 0x7f3fd402a090 con 0x7f3fe00d03a0 2023-11-08T19:27:17.238 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:27:17.238 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":33,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5231,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6836/4083537882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":4083537882},{"type":"v1","addr":"172.21.15.174:6836","nonce":4083537882}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":5234,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/494923804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6835","nonce":494923804},{"type":"v1","addr":"172.21.15.174:6837","nonce":494923804}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":6009,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6834/3383638390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":3383638390},{"type":"v1","addr":"172.21.15.143:6834","nonce":3383638390}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T19:26:57.522785+0000","modified":"2023-11-08T19:27:00.536968+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6012},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6012":{"gid":6012,"name":"c","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.143:6835/2613282947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":2613282947},{"type":"v1","addr":"172.21.15.143:6835","nonce":2613282947}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-08T19:27:17.240 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.238+0000 7f3fe86a1700 1 -- 172.21.15.143:0/3024719356 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f3fc406af00 msgr2=0x7f3fc406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:17.240 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.238+0000 7f3fe86a1700 1 --2- 172.21.15.143:0/3024719356 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f3fc406af00 0x7f3fc406d3c0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f3fe00cfee0 tx=0x7f3fdc008040 comp rx=0 tx=0).stop 2023-11-08T19:27:17.240 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.238+0000 7f3fe86a1700 1 -- 172.21.15.143:0/3024719356 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3fe00d03a0 msgr2=0x7f3fe014f300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:17.241 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.238+0000 7f3fe86a1700 1 --2- 172.21.15.143:0/3024719356 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3fe00d03a0 0x7f3fe014f300 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f3fd4000c00 tx=0x7f3fd4004750 comp rx=0 tx=0).stop 2023-11-08T19:27:17.241 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.238+0000 7f3fe86a1700 1 -- 172.21.15.143:0/3024719356 shutdown_connections 2023-11-08T19:27:17.241 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.238+0000 7f3fe86a1700 1 --2- 172.21.15.143:0/3024719356 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f3fc406af00 0x7f3fc406d3c0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:17.241 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.238+0000 7f3fe86a1700 1 --2- 172.21.15.143:0/3024719356 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3fe015ba30 0x7f3fe015e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:17.241 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.239+0000 7f3fe86a1700 1 --2- 172.21.15.143:0/3024719356 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3fe00d03a0 0x7f3fe014f300 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:17.242 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.239+0000 7f3fe86a1700 1 --2- 172.21.15.143:0/3024719356 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3fe00d1aa0 0x7f3fe014c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:17.242 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.239+0000 7f3fe86a1700 1 -- 172.21.15.143:0/3024719356 >> 172.21.15.143:0/3024719356 conn(0x7f3fe008b000 msgr2=0x7f3fe00aea60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:17.242 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.239+0000 7f3fe86a1700 1 -- 172.21.15.143:0/3024719356 shutdown_connections 2023-11-08T19:27:17.242 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.239+0000 7f3fe86a1700 1 -- 172.21.15.143:0/3024719356 wait complete. 2023-11-08T19:27:17.243 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 33 2023-11-08T19:27:17.252 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:27:17.372 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.371+0000 7f14e1553700 1 Processor -- start 2023-11-08T19:27:17.372 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.371+0000 7f14e1553700 1 -- start start 2023-11-08T19:27:17.376 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.371+0000 7f14e1553700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f14dc0b1810 0x7f14dc06e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:17.376 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.371+0000 7f14e1553700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14dc0cbe70 con 0x7f14dc0cadc0 2023-11-08T19:27:17.377 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.371+0000 7f14e1553700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14dc0cb260 con 0x7f14dc15a9a0 2023-11-08T19:27:17.377 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.371+0000 7f14e1553700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14dc06e890 con 0x7f14dc0b1810 2023-11-08T19:27:17.377 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.372+0000 7f14da7fc700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f14dc0cadc0 0x7f14dc06cc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:49326/0 (socket says 172.21.15.143:49326) 2023-11-08T19:27:17.377 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.372+0000 7f14da7fc700 1 -- 172.21.15.143:0/2340491409 learned_addr learned my addr 172.21.15.143:0/2340491409 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:27:17.377 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.372+0000 7f14db7fe700 1 --2- 172.21.15.143:0/2340491409 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f14dc0b1810 0x7f14dc06e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:17.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.372+0000 7f14d9ffb700 1 -- 172.21.15.143:0/2340491409 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2760396626 0 0) 0x7f14dc0cbe70 con 0x7f14dc0cadc0 2023-11-08T19:27:17.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.372+0000 7f14d9ffb700 1 -- 172.21.15.143:0/2340491409 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f14c40035b0 con 0x7f14dc0cadc0 2023-11-08T19:27:17.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.372+0000 7f14d9ffb700 1 -- 172.21.15.143:0/2340491409 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1108130977 0 0) 0x7f14c40035b0 con 0x7f14dc0cadc0 2023-11-08T19:27:17.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.372+0000 7f14d9ffb700 1 -- 172.21.15.143:0/2340491409 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f14dc0b1810 msgr2=0x7f14dc06e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:17.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.372+0000 7f14d9ffb700 1 --2- 172.21.15.143:0/2340491409 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f14dc0b1810 0x7f14dc06e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:17.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.372+0000 7f14d9ffb700 1 -- 172.21.15.143:0/2340491409 >> v1:172.21.15.174:6789/0 conn(0x7f14dc15a9a0 legacy=0x7f14dc0b1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:17.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.372+0000 7f14d9ffb700 1 -- 172.21.15.143:0/2340491409 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14dc0cb540 con 0x7f14dc0cadc0 2023-11-08T19:27:17.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.372+0000 7f14d9ffb700 1 -- 172.21.15.143:0/2340491409 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4147756303 0 0) 0x7f14d0001670 con 0x7f14dc0cadc0 2023-11-08T19:27:17.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.372+0000 7f14d9ffb700 1 -- 172.21.15.143:0/2340491409 >> v1:172.21.15.143:6789/0 conn(0x7f14dc0cadc0 legacy=0x7f14dc06cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:17.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.372+0000 7f14d9ffb700 1 --2- 172.21.15.143:0/2340491409 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f14c4003a30 0x7f14c4003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:17.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.372+0000 7f14d9ffb700 1 --2- 172.21.15.143:0/2340491409 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f14c4004850 0x7f14c4004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:17.380 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.372+0000 7f14d9ffb700 1 --2- 172.21.15.143:0/2340491409 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f14dc0b1810 0x7f14c4005250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:17.380 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.372+0000 7f14d9ffb700 1 -- 172.21.15.143:0/2340491409 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14dc06e890 con 0x7f14c4004850 2023-11-08T19:27:17.380 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.372+0000 7f14d9ffb700 1 -- 172.21.15.143:0/2340491409 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14c400bb70 con 0x7f14c4003a30 2023-11-08T19:27:17.380 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.372+0000 7f14d9ffb700 1 -- 172.21.15.143:0/2340491409 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14c400bd30 con 0x7f14dc0b1810 2023-11-08T19:27:17.381 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.372+0000 7f14da7fc700 1 --2- 172.21.15.143:0/2340491409 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f14c4004850 0x7f14c4004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:17.381 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.372+0000 7f14daffd700 1 --2- 172.21.15.143:0/2340491409 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f14c4003a30 0x7f14c4003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:17.381 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.372+0000 7f14db7fe700 1 --2- 172.21.15.143:0/2340491409 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f14dc0b1810 0x7f14c4005250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:17.381 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.373+0000 7f14da7fc700 1 -- 172.21.15.143:0/2340491409 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f14dc0b1810 msgr2=0x7f14c4005250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:17.381 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.373+0000 7f14da7fc700 1 --2- 172.21.15.143:0/2340491409 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f14dc0b1810 0x7f14c4005250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:17.382 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.373+0000 7f14da7fc700 1 -- 172.21.15.143:0/2340491409 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f14c4003a30 msgr2=0x7f14c4003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:17.382 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.373+0000 7f14da7fc700 1 --2- 172.21.15.143:0/2340491409 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f14c4003a30 0x7f14c4003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:17.382 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.373+0000 7f14da7fc700 1 -- 172.21.15.143:0/2340491409 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14c40035b0 con 0x7f14c4004850 2023-11-08T19:27:17.382 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.373+0000 7f14da7fc700 1 --2- 172.21.15.143:0/2340491409 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f14c4004850 0x7f14c4004cd0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f14d000c240 tx=0x7f14c4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75a2963633df102a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:17.383 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.373+0000 7f14d9ffb700 1 -- 172.21.15.143:0/2340491409 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14d0007c10 con 0x7f14c4004850 2023-11-08T19:27:17.383 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.373+0000 7f14d9ffb700 1 -- 172.21.15.143:0/2340491409 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14d000cde0 con 0x7f14c4004850 2023-11-08T19:27:17.383 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.373+0000 7f14d9ffb700 1 -- 172.21.15.143:0/2340491409 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14d0005ba0 con 0x7f14c4004850 2023-11-08T19:27:17.383 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.373+0000 7f14e1553700 1 -- 172.21.15.143:0/2340491409 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f14c4004850 msgr2=0x7f14c4004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:17.383 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.373+0000 7f14e1553700 1 --2- 172.21.15.143:0/2340491409 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f14c4004850 0x7f14c4004cd0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f14d000c240 tx=0x7f14c4005880 comp rx=0 tx=0).stop 2023-11-08T19:27:17.384 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.373+0000 7f14db7fe700 1 -- 172.21.15.143:0/2340491409 reap_dead start 2023-11-08T19:27:17.384 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.373+0000 7f14e1553700 1 -- 172.21.15.143:0/2340491409 shutdown_connections 2023-11-08T19:27:17.384 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.373+0000 7f14e1553700 1 -- 172.21.15.143:0/2340491409 >> 172.21.15.143:0/2340491409 conn(0x7f14dc06b4c0 msgr2=0x7f14dc0cf680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:17.384 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.374+0000 7f14e1553700 1 -- 172.21.15.143:0/2340491409 shutdown_connections 2023-11-08T19:27:17.385 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.374+0000 7f14e1553700 1 -- 172.21.15.143:0/2340491409 wait complete. 2023-11-08T19:27:17.385 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.375+0000 7f14e1553700 1 Processor -- start 2023-11-08T19:27:17.385 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.375+0000 7f14e1553700 1 -- start start 2023-11-08T19:27:17.388 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.376+0000 7f14e1553700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f14c4004850 0x7f14dc14fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:17.388 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.376+0000 7f14e1553700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f14dc1f4840 0x7f14dc159190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:17.389 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.376+0000 7f14e1553700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f14dc1f4cc0 0x7f14dc1596d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:17.389 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.376+0000 7f14e1553700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14dc0acad0 con 0x7f14dc1f4cc0 2023-11-08T19:27:17.389 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.376+0000 7f14e1553700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14dc0acd70 con 0x7f14c4004850 2023-11-08T19:27:17.389 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.376+0000 7f14e1553700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14dc0ad050 con 0x7f14dc1f4840 2023-11-08T19:27:17.389 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.376+0000 7f14db7fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f14dc1f4cc0 0x7f14dc1596d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:17.390 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.376+0000 7f14db7fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f14dc1f4cc0 0x7f14dc1596d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53188/0 (socket says 172.21.15.143:53188) 2023-11-08T19:27:17.390 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.376+0000 7f14db7fe700 1 -- 172.21.15.143:0/3376308368 learned_addr learned my addr 172.21.15.143:0/3376308368 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:17.390 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.376+0000 7f14daffd700 1 --2- 172.21.15.143:0/3376308368 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f14c4004850 0x7f14dc14fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:17.390 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.376+0000 7f14da7fc700 1 --2- 172.21.15.143:0/3376308368 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f14dc1f4840 0x7f14dc159190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:17.390 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.376+0000 7f14db7fe700 1 -- 172.21.15.143:0/3376308368 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f14dc1f4840 msgr2=0x7f14dc159190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:17.391 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.376+0000 7f14db7fe700 1 --2- 172.21.15.143:0/3376308368 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f14dc1f4840 0x7f14dc159190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:17.391 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.376+0000 7f14db7fe700 1 -- 172.21.15.143:0/3376308368 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f14c4004850 msgr2=0x7f14dc14fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:17.391 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.376+0000 7f14db7fe700 1 --2- 172.21.15.143:0/3376308368 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f14c4004850 0x7f14dc14fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:17.391 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.376+0000 7f14db7fe700 1 -- 172.21.15.143:0/3376308368 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14dc0ad330 con 0x7f14dc1f4cc0 2023-11-08T19:27:17.391 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.376+0000 7f14db7fe700 1 --2- 172.21.15.143:0/3376308368 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f14dc1f4cc0 0x7f14dc1596d0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f14dc0c6670 tx=0x7f14d400f490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:17.391 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.376+0000 7f14c37fe700 1 -- 172.21.15.143:0/3376308368 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14d400e040 con 0x7f14dc1f4cc0 2023-11-08T19:27:17.392 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.376+0000 7f14c37fe700 1 -- 172.21.15.143:0/3376308368 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14d400f040 con 0x7f14dc1f4cc0 2023-11-08T19:27:17.392 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.376+0000 7f14e1553700 1 -- 172.21.15.143:0/3376308368 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14dc0ad610 con 0x7f14dc1f4cc0 2023-11-08T19:27:17.392 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.376+0000 7f14c37fe700 1 -- 172.21.15.143:0/3376308368 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14d400fc30 con 0x7f14dc1f4cc0 2023-11-08T19:27:17.392 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.376+0000 7f14e1553700 1 -- 172.21.15.143:0/3376308368 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14dc0add10 con 0x7f14dc1f4cc0 2023-11-08T19:27:17.392 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.377+0000 7f14e1553700 1 -- 172.21.15.143:0/3376308368 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14dc04eee0 con 0x7f14dc1f4cc0 2023-11-08T19:27:17.393 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.378+0000 7f14c37fe700 1 -- 172.21.15.143:0/3376308368 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f14d400fd90 con 0x7f14dc1f4cc0 2023-11-08T19:27:17.393 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.378+0000 7f14c37fe700 1 --2- 172.21.15.143:0/3376308368 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f14c806af00 0x7f14c806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:17.393 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.378+0000 7f14c37fe700 1 -- 172.21.15.143:0/3376308368 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6828+0+0 (secure 0 0 0) 0x7f14d408a170 con 0x7f14dc1f4cc0 2023-11-08T19:27:17.393 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.378+0000 7f14daffd700 1 --2- 172.21.15.143:0/3376308368 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f14c806af00 0x7f14c806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:17.393 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.379+0000 7f14daffd700 1 --2- 172.21.15.143:0/3376308368 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f14c806af00 0x7f14c806d3c0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f14dc159d30 tx=0x7f14cc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:17.394 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.382+0000 7f14c37fe700 1 -- 172.21.15.143:0/3376308368 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f14d4054e50 con 0x7f14dc1f4cc0 2023-11-08T19:27:17.573 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.571+0000 7f14e1553700 1 -- 172.21.15.143:0/3376308368 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f14dc0ca7a0 con 0x7f14dc1f4cc0 2023-11-08T19:27:17.574 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.573+0000 7f14c37fe700 1 -- 172.21.15.143:0/3376308368 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) v1 ==== 74+0+21891 (secure 0 0 0) 0x7f14d4058470 con 0x7f14dc1f4cc0 2023-11-08T19:27:17.575 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:27:17.575 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":66,"fsid":"968483af-77fc-4068-ad7c-5c2a1a4613d7","created":"2023-11-08T19:24:32.826484+0000","modified":"2023-11-08T19:27:09.174117+0000","last_up_change":"2023-11-08T19:24:43.573880+0000","last_in_change":"2023-11-08T19:24:35.314525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:44.551530+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"snap-test-pool","create_time":"2023-11-08T19:26:02.874384+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":36,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-11-08T19:26:55.754227+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-11-08T19:26:56.486110+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"test_data_pool","create_time":"2023-11-08T19:27:05.601949+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":66,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_metadata_pool","create_time":"2023-11-08T19:27:06.536369+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bae98ec1-d94e-403d-a220-a522a53ea6f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6809","nonce":1487830566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6811","nonce":1487830566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6815","nonce":1487830566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6813","nonce":1487830566}]},"public_addr":"172.21.15.143:6809/1487830566","cluster_addr":"172.21.15.143:6811/1487830566","heartbeat_back_addr":"172.21.15.143:6815/1487830566","heartbeat_front_addr":"172.21.15.143:6813/1487830566","state":["exists","up"]},{"osd":1,"uuid":"5cb6538a-1ccb-4322-8633-09f66af0b7e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6801","nonce":787341397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6803","nonce":787341397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6807","nonce":787341397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6805","nonce":787341397}]},"public_addr":"172.21.15.143:6801/787341397","cluster_addr":"172.21.15.143:6803/787341397","heartbeat_back_addr":"172.21.15.143:6807/787341397","heartbeat_front_addr":"172.21.15.143:6805/787341397","state":["exists","up"]},{"osd":2,"uuid":"cdbf522c-60ef-4d60-9653-8dcb529c101a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6822","nonce":2301325597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6826","nonce":2301325597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6831","nonce":2301325597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6829","nonce":2301325597}]},"public_addr":"172.21.15.143:6822/2301325597","cluster_addr":"172.21.15.143:6826/2301325597","heartbeat_back_addr":"172.21.15.143:6831/2301325597","heartbeat_front_addr":"172.21.15.143:6829/2301325597","state":["exists","up"]},{"osd":3,"uuid":"4e2783cf-ebb9-4dd8-8245-bc1d17ce4325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6817","nonce":2021504428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6819","nonce":2021504428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6827","nonce":2021504428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6823","nonce":2021504428}]},"public_addr":"172.21.15.143:6817/2021504428","cluster_addr":"172.21.15.143:6819/2021504428","heartbeat_back_addr":"172.21.15.143:6827/2021504428","heartbeat_front_addr":"172.21.15.143:6823/2021504428","state":["exists","up"]},{"osd":4,"uuid":"691256fa-9b77-49b9-a346-7ae1adf0ca80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6801","nonce":1993380595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6803","nonce":1993380595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6807","nonce":1993380595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6805","nonce":1993380595}]},"public_addr":"172.21.15.174:6801/1993380595","cluster_addr":"172.21.15.174:6803/1993380595","heartbeat_back_addr":"172.21.15.174:6807/1993380595","heartbeat_front_addr":"172.21.15.174:6805/1993380595","state":["exists","up"]},{"osd":5,"uuid":"756fc352-48c9-46ce-b52e-0541fe6f8525","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6809","nonce":1777923303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6811","nonce":1777923303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6815","nonce":1777923303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6813","nonce":1777923303}]},"public_addr":"172.21.15.174:6809/1777923303","cluster_addr":"172.21.15.174:6811/1777923303","heartbeat_back_addr":"172.21.15.174:6815/1777923303","heartbeat_front_addr":"172.21.15.174:6813/1777923303","state":["exists","up"]},{"osd":6,"uuid":"fab2ac2b-ed97-4cde-9825-b480eabf8b8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6825","nonce":2755130316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6827","nonce":2755130316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6831","nonce":2755130316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6829","nonce":2755130316}]},"public_addr":"172.21.15.174:6825/2755130316","cluster_addr":"172.21.15.174:6827/2755130316","heartbeat_back_addr":"172.21.15.174:6831/2755130316","heartbeat_front_addr":"172.21.15.174:6829/2755130316","state":["exists","up"]},{"osd":7,"uuid":"3de9b797-2ab6-4d41-a863-7e7d638fd703","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6817","nonce":2847141411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6819","nonce":2847141411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6823","nonce":2847141411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6821","nonce":2847141411}]},"public_addr":"172.21.15.174:6817/2847141411","cluster_addr":"172.21.15.174:6819/2847141411","heartbeat_back_addr":"172.21.15.174:6823/2847141411","heartbeat_front_addr":"172.21.15.174:6821/2847141411","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.477076+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.496890+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.539305+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.454673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.538881+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.521294+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.520712+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.540404+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":11,"snaps":[{"begin":1,"length":1}]}],"new_removed_snaps":[{"pool":11,"snaps":[{"begin":1,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:27:17.577 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.575+0000 7f14e1553700 1 -- 172.21.15.143:0/3376308368 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f14c806af00 msgr2=0x7f14c806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:17.577 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.575+0000 7f14e1553700 1 --2- 172.21.15.143:0/3376308368 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f14c806af00 0x7f14c806d3c0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f14dc159d30 tx=0x7f14cc009040 comp rx=0 tx=0).stop 2023-11-08T19:27:17.577 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.575+0000 7f14e1553700 1 -- 172.21.15.143:0/3376308368 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f14dc1f4cc0 msgr2=0x7f14dc1596d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:17.578 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.575+0000 7f14e1553700 1 --2- 172.21.15.143:0/3376308368 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f14dc1f4cc0 0x7f14dc1596d0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f14dc0c6670 tx=0x7f14d400f490 comp rx=0 tx=0).stop 2023-11-08T19:27:17.578 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.575+0000 7f14e1553700 1 -- 172.21.15.143:0/3376308368 shutdown_connections 2023-11-08T19:27:17.578 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.575+0000 7f14e1553700 1 --2- 172.21.15.143:0/3376308368 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f14c806af00 0x7f14c806d3c0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:17.578 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.575+0000 7f14e1553700 1 --2- 172.21.15.143:0/3376308368 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f14dc1f4cc0 0x7f14dc1596d0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:17.578 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.575+0000 7f14e1553700 1 --2- 172.21.15.143:0/3376308368 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f14c4004850 0x7f14dc14fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:17.579 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.575+0000 7f14e1553700 1 --2- 172.21.15.143:0/3376308368 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f14dc1f4840 0x7f14dc159190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:17.579 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.575+0000 7f14e1553700 1 -- 172.21.15.143:0/3376308368 >> 172.21.15.143:0/3376308368 conn(0x7f14dc06b4c0 msgr2=0x7f14dc0cd1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:17.579 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.575+0000 7f14e1553700 1 -- 172.21.15.143:0/3376308368 shutdown_connections 2023-11-08T19:27:17.579 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.575+0000 7f14e1553700 1 -- 172.21.15.143:0/3376308368 wait complete. 2023-11-08T19:27:17.590 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-08T19:27:17.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.710+0000 7f36f2aa4700 1 Processor -- start 2023-11-08T19:27:17.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.710+0000 7f36f2aa4700 1 -- start start 2023-11-08T19:27:17.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.711+0000 7f36f2aa4700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f36ec15dd00 0x7f36ec0cca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:17.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.711+0000 7f36f2aa4700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f36ec06b320 0x7f36ec0cb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:17.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.711+0000 7f36f2aa4700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36ec0ccfb0 con 0x7f36ec1618f0 2023-11-08T19:27:17.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.711+0000 7f36f2aa4700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36ec0cad50 con 0x7f36ec06b320 2023-11-08T19:27:17.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.711+0000 7f36f2aa4700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36ec161df0 con 0x7f36ec15dd00 2023-11-08T19:27:17.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.711+0000 7f36f1041700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f36ec06b320 0x7f36ec0cb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:17.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.711+0000 7f36ebfff700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f36ec1618f0 0x7f36ec06ac10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:49346/0 (socket says 172.21.15.143:49346) 2023-11-08T19:27:17.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.711+0000 7f36ebfff700 1 -- 172.21.15.143:0/3446169922 learned_addr learned my addr 172.21.15.143:0/3446169922 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:27:17.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.711+0000 7f36f0840700 1 --2- 172.21.15.143:0/3446169922 >> v2:172.21.15.174:3300/0 conn(0x7f36ec15dd00 0x7f36ec0cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:17.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.711+0000 7f36f1041700 1 -- 172.21.15.143:0/3446169922 >> v2:172.21.15.174:3300/0 conn(0x7f36ec15dd00 msgr2=0x7f36ec0cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:17.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.711+0000 7f36f1041700 1 --2- 172.21.15.143:0/3446169922 >> v2:172.21.15.174:3300/0 conn(0x7f36ec15dd00 0x7f36ec0cca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:17.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.711+0000 7f36f1041700 1 -- 172.21.15.143:0/3446169922 >> v1:172.21.15.143:6789/0 conn(0x7f36ec1618f0 legacy=0x7f36ec06ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:17.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.711+0000 7f36f1041700 1 -- 172.21.15.143:0/3446169922 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36ec1620d0 con 0x7f36ec06b320 2023-11-08T19:27:17.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.711+0000 7f36f0840700 1 --2- 172.21.15.143:0/3446169922 >> v2:172.21.15.174:3300/0 conn(0x7f36ec15dd00 0x7f36ec0cca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:27:17.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.711+0000 7f36f1041700 1 --2- 172.21.15.143:0/3446169922 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f36ec06b320 0x7f36ec0cb1a0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f36ec070a60 tx=0x7f36e400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0e28d57efd45a9a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:17.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.711+0000 7f36eb7fe700 1 -- 172.21.15.143:0/3446169922 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36e400e070 con 0x7f36ec06b320 2023-11-08T19:27:17.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.711+0000 7f36eb7fe700 1 -- 172.21.15.143:0/3446169922 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36e400bbd0 con 0x7f36ec06b320 2023-11-08T19:27:17.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.711+0000 7f36eb7fe700 1 -- 172.21.15.143:0/3446169922 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36e4012430 con 0x7f36ec06b320 2023-11-08T19:27:17.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.712+0000 7f36f2aa4700 1 -- 172.21.15.143:0/3446169922 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f36ec06b320 msgr2=0x7f36ec0cb1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:17.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.712+0000 7f36f2aa4700 1 --2- 172.21.15.143:0/3446169922 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f36ec06b320 0x7f36ec0cb1a0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f36ec070a60 tx=0x7f36e400b330 comp rx=0 tx=0).stop 2023-11-08T19:27:17.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.712+0000 7f36f2aa4700 1 -- 172.21.15.143:0/3446169922 shutdown_connections 2023-11-08T19:27:17.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.712+0000 7f36f2aa4700 1 --2- 172.21.15.143:0/3446169922 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f36ec06b320 0x7f36ec0cb1a0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:17.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.712+0000 7f36f2aa4700 1 --2- 172.21.15.143:0/3446169922 >> v2:172.21.15.174:3300/0 conn(0x7f36ec15dd00 0x7f36ec0cca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:17.721 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.712+0000 7f36f2aa4700 1 -- 172.21.15.143:0/3446169922 >> 172.21.15.143:0/3446169922 conn(0x7f36ec099ab0 msgr2=0x7f36ec0d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:17.721 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.712+0000 7f36f2aa4700 1 -- 172.21.15.143:0/3446169922 shutdown_connections 2023-11-08T19:27:17.722 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.712+0000 7f36f2aa4700 1 -- 172.21.15.143:0/3446169922 wait complete. 2023-11-08T19:27:17.724 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.713+0000 7f36f2aa4700 1 Processor -- start 2023-11-08T19:27:17.725 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.713+0000 7f36f2aa4700 1 -- start start 2023-11-08T19:27:17.725 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.714+0000 7f36f2aa4700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f36ec06b320 0x7f36ec0b5e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:17.725 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.714+0000 7f36f2aa4700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f36ec15dd00 0x7f36ec157ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:17.725 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.714+0000 7f36f2aa4700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f36ec1618f0 0x7f36ec0ae610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:17.726 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.714+0000 7f36f2aa4700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36ec1587c0 con 0x7f36ec06b320 2023-11-08T19:27:17.726 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.714+0000 7f36f2aa4700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36ec0b63a0 con 0x7f36ec15dd00 2023-11-08T19:27:17.726 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.714+0000 7f36f2aa4700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36ec0b4530 con 0x7f36ec1618f0 2023-11-08T19:27:17.726 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.714+0000 7f36f0840700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f36ec06b320 0x7f36ec0b5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:17.726 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.714+0000 7f36f0840700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f36ec06b320 0x7f36ec0b5e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53198/0 (socket says 172.21.15.143:53198) 2023-11-08T19:27:17.727 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.714+0000 7f36f0840700 1 -- 172.21.15.143:0/4117934539 learned_addr learned my addr 172.21.15.143:0/4117934539 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:17.727 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.714+0000 7f36ebfff700 1 --2- 172.21.15.143:0/4117934539 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f36ec15dd00 0x7f36ec157ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:17.727 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.715+0000 7f36f0840700 1 -- 172.21.15.143:0/4117934539 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f36ec1618f0 msgr2=0x7f36ec0ae610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:17.727 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.715+0000 7f36f0840700 1 --2- 172.21.15.143:0/4117934539 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f36ec1618f0 0x7f36ec0ae610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:17.727 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.715+0000 7f36f0840700 1 -- 172.21.15.143:0/4117934539 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f36ec15dd00 msgr2=0x7f36ec157ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:17.728 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.715+0000 7f36f0840700 1 --2- 172.21.15.143:0/4117934539 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f36ec15dd00 0x7f36ec157ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:17.728 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.715+0000 7f36f0840700 1 -- 172.21.15.143:0/4117934539 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36e4009a70 con 0x7f36ec06b320 2023-11-08T19:27:17.728 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.715+0000 7f36f0840700 1 --2- 172.21.15.143:0/4117934539 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f36ec06b320 0x7f36ec0b5e60 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f36e000c820 tx=0x7f36e000cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:17.728 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.715+0000 7f36e97fa700 1 -- 172.21.15.143:0/4117934539 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36e0005c20 con 0x7f36ec06b320 2023-11-08T19:27:17.728 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.715+0000 7f36e97fa700 1 -- 172.21.15.143:0/4117934539 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36e0005d80 con 0x7f36ec06b320 2023-11-08T19:27:17.729 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.715+0000 7f36f2aa4700 1 -- 172.21.15.143:0/4117934539 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36ec1581e0 con 0x7f36ec06b320 2023-11-08T19:27:17.729 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.715+0000 7f36e97fa700 1 -- 172.21.15.143:0/4117934539 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36e0011820 con 0x7f36ec06b320 2023-11-08T19:27:17.729 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.715+0000 7f36f2aa4700 1 -- 172.21.15.143:0/4117934539 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36ec0b3c00 con 0x7f36ec06b320 2023-11-08T19:27:17.729 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.716+0000 7f36f2aa4700 1 -- 172.21.15.143:0/4117934539 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36ec0aff60 con 0x7f36ec06b320 2023-11-08T19:27:17.730 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.716+0000 7f36e97fa700 1 -- 172.21.15.143:0/4117934539 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f36e0011980 con 0x7f36ec06b320 2023-11-08T19:27:17.730 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.717+0000 7f36e97fa700 1 --2- 172.21.15.143:0/4117934539 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f36d006af00 0x7f36d006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:17.730 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.717+0000 7f36e97fa700 1 -- 172.21.15.143:0/4117934539 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6800+0+0 (secure 0 0 0) 0x7f36e0089c10 con 0x7f36ec06b320 2023-11-08T19:27:17.730 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.717+0000 7f36ebfff700 1 --2- 172.21.15.143:0/4117934539 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f36d006af00 0x7f36d006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:17.731 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.718+0000 7f36ebfff700 1 --2- 172.21.15.143:0/4117934539 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f36d006af00 0x7f36d006d3c0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f36d8005910 tx=0x7f36d8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:17.731 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.721+0000 7f36e97fa700 1 -- 172.21.15.143:0/4117934539 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f36e0054910 con 0x7f36ec06b320 2023-11-08T19:27:17.956 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.954+0000 7f36f2aa4700 1 -- 172.21.15.143:0/4117934539 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f36ec04eee0 con 0x7f36ec06b320 2023-11-08T19:27:17.956 INFO:tasks.ceph.mon.a.smithi143.stderr:2023-11-08T19:27:17.955+0000 7f65b584e700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-08T19:27:17.957 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.956+0000 7f36e97fa700 1 -- 172.21.15.143:0/4117934539 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v34) v1 ==== 161+0+0 (secure 0 0 0) 0x7f36e0057f30 con 0x7f36ec06b320 2023-11-08T19:27:17.961 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.959+0000 7f36f2aa4700 1 -- 172.21.15.143:0/4117934539 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f36d006af00 msgr2=0x7f36d006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:17.961 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.959+0000 7f36f2aa4700 1 --2- 172.21.15.143:0/4117934539 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f36d006af00 0x7f36d006d3c0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f36d8005910 tx=0x7f36d8009040 comp rx=0 tx=0).stop 2023-11-08T19:27:17.961 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.959+0000 7f36f2aa4700 1 -- 172.21.15.143:0/4117934539 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f36ec06b320 msgr2=0x7f36ec0b5e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:17.961 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.959+0000 7f36f2aa4700 1 --2- 172.21.15.143:0/4117934539 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f36ec06b320 0x7f36ec0b5e60 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f36e000c820 tx=0x7f36e000cbe0 comp rx=0 tx=0).stop 2023-11-08T19:27:17.962 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.959+0000 7f36f2aa4700 1 -- 172.21.15.143:0/4117934539 shutdown_connections 2023-11-08T19:27:17.962 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.959+0000 7f36f2aa4700 1 --2- 172.21.15.143:0/4117934539 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f36d006af00 0x7f36d006d3c0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:17.962 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.960+0000 7f36f2aa4700 1 --2- 172.21.15.143:0/4117934539 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f36ec1618f0 0x7f36ec0ae610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:17.962 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.960+0000 7f36f2aa4700 1 --2- 172.21.15.143:0/4117934539 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f36ec06b320 0x7f36ec0b5e60 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:17.963 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.960+0000 7f36f2aa4700 1 --2- 172.21.15.143:0/4117934539 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f36ec15dd00 0x7f36ec157ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:17.963 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.960+0000 7f36f2aa4700 1 -- 172.21.15.143:0/4117934539 >> 172.21.15.143:0/4117934539 conn(0x7f36ec099ab0 msgr2=0x7f36ec15b8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:17.963 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.960+0000 7f36f2aa4700 1 -- 172.21.15.143:0/4117934539 shutdown_connections 2023-11-08T19:27:17.963 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:17.960+0000 7f36f2aa4700 1 -- 172.21.15.143:0/4117934539 wait complete. 2023-11-08T19:27:17.964 INFO:teuthology.orchestra.run.smithi143.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-08T19:27:17.964 INFO:tasks.ceph.mds.c.smithi143.stderr: -3019> 2023-11-08T19:27:03.112+0000 7fc300b59700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-08T19:27:17.974 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-08T19:27:17.990 INFO:tasks.ceph.mds.c.smithi143.stdout:starting mds.c at 2023-11-08T19:27:18.098 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:18.097+0000 7fc39ab04700 1 Processor -- start 2023-11-08T19:27:18.099 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:18.097+0000 7fc39ab04700 1 -- start start 2023-11-08T19:27:18.101 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:18.098+0000 7fc39ab04700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc39406b240 0x7fc3940caff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:18.101 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:18.098+0000 7fc39ab04700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fc3940cb530 0x7fc3940b22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:18.102 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:18.098+0000 7fc39ab04700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc394161890 con 0x7fc39415dba0 2023-11-08T19:27:18.102 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:18.098+0000 7fc39ab04700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc39406cd20 con 0x7fc3940cb530 2023-11-08T19:27:18.102 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:18.098+0000 7fc39ab04700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc39406b740 con 0x7fc39406b240 2023-11-08T19:27:18.102 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:18.098+0000 7fc3988a0700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fc39415dba0 0x7fc39406c610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:49372/0 (socket says 172.21.15.143:49372) 2023-11-08T19:27:18.103 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:18.098+0000 7fc3988a0700 1 -- 172.21.15.143:0/1056543028 learned_addr learned my addr 172.21.15.143:0/1056543028 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:27:18.103 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:18.098+0000 7fc393fff700 1 --2- 172.21.15.143:0/1056543028 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc39406b240 0x7fc3940caff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:18.103 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:18.098+0000 7fc3990a1700 1 --2- 172.21.15.143:0/1056543028 >> v2:172.21.15.174:3300/0 conn(0x7fc3940cb530 0x7fc3940b22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:18.103 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:18.098+0000 7fc3937fe700 1 -- 172.21.15.143:0/1056543028 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 93676350 0 0) 0x7fc394161890 con 0x7fc39415dba0 2023-11-08T19:27:18.103 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:18.098+0000 7fc3937fe700 1 -- 172.21.15.143:0/1056543028 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc3840035b0 con 0x7fc39415dba0 2023-11-08T19:27:18.104 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:18.098+0000 7fc3990a1700 1 -- 172.21.15.143:0/1056543028 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc39406b240 msgr2=0x7fc3940caff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:18.104 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:18.098+0000 7fc3990a1700 1 --2- 172.21.15.143:0/1056543028 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc39406b240 0x7fc3940caff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:18.104 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:18.098+0000 7fc3990a1700 1 -- 172.21.15.143:0/1056543028 >> v1:172.21.15.143:6789/0 conn(0x7fc39415dba0 legacy=0x7fc39406c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:18.104 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:18.098+0000 7fc3990a1700 1 -- 172.21.15.143:0/1056543028 --> v2:172.21.15.174:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc39406ba20 con 0x7fc3940cb530 2023-11-08T19:27:18.104 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:18.098+0000 7fc393fff700 1 --2- 172.21.15.143:0/1056543028 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc39406b240 0x7fc3940caff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T19:27:18.105 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:18.099+0000 7fc3990a1700 1 --2- 172.21.15.143:0/1056543028 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc3940cb530 0x7fc3940b22e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fc38c009d90 tx=0x7fc38c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e04568405884740 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:18.105 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:18.099+0000 7fc3937fe700 1 -- 172.21.15.143:0/1056543028 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc38c00e070 con 0x7fc3940cb530 2023-11-08T19:27:18.105 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:18.099+0000 7fc3937fe700 1 -- 172.21.15.143:0/1056543028 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc38c00bb20 con 0x7fc3940cb530 2023-11-08T19:27:18.105 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:18.099+0000 7fc3937fe700 1 -- 172.21.15.143:0/1056543028 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc38c012430 con 0x7fc3940cb530 2023-11-08T19:27:18.105 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:18.099+0000 7fc39ab04700 1 -- 172.21.15.143:0/1056543028 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc3940cb530 msgr2=0x7fc3940b22e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:18.106 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:18.099+0000 7fc39ab04700 1 --2- 172.21.15.143:0/1056543028 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc3940cb530 0x7fc3940b22e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fc38c009d90 tx=0x7fc38c00b330 comp rx=0 tx=0).stop 2023-11-08T19:27:18.106 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:18.099+0000 7fc39ab04700 1 -- 172.21.15.143:0/1056543028 shutdown_connections 2023-11-08T19:27:18.106 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:18.099+0000 7fc39ab04700 1 --2- 172.21.15.143:0/1056543028 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc3940cb530 0x7fc3940b22e0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:18.106 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:18.099+0000 7fc39ab04700 1 --2- 172.21.15.143:0/1056543028 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc39406b240 0x7fc3940caff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:18.106 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:18.099+0000 7fc39ab04700 1 -- 172.21.15.143:0/1056543028 >> 172.21.15.143:0/1056543028 conn(0x7fc3940cd2e0 msgr2=0x7fc3940cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:18.107 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:18.100+0000 7fc39ab04700 1 -- 172.21.15.143:0/1056543028 shutdown_connections 2023-11-08T19:27:18.107 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:18.100+0000 7fc39ab04700 1 -- 172.21.15.143:0/1056543028 wait complete. 2023-11-08T19:27:18.110 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:18.101+0000 7fc39ab04700 1 Processor -- start 2023-11-08T19:27:18.110 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:18.101+0000 7fc39ab04700 1 -- start start 2023-11-08T19:27:18.110 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:18.102+0000 7fc39ab04700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc39406b240 0x7fc39415c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:18.110 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:18.102+0000 7fc39ab04700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc3940cb530 0x7fc3940c4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:18.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:18.102+0000 7fc39ab04700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc39415dba0 0x7fc3940c48d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:18.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:18.102+0000 7fc39ab04700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc39415d2a0 con 0x7fc3940cb530 2023-11-08T19:27:18.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:18.102+0000 7fc39ab04700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3940c4f00 con 0x7fc39415dba0 2023-11-08T19:27:18.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:18.102+0000 7fc39ab04700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3940c51e0 con 0x7fc39406b240 2023-11-08T19:27:18.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:18.102+0000 7fc393fff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc3940cb530 0x7fc3940c4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:18.112 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:18.102+0000 7fc393fff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc3940cb530 0x7fc3940c4390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53226/0 (socket says 172.21.15.143:53226) 2023-11-08T19:27:18.112 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:18.102+0000 7fc393fff700 1 -- 172.21.15.143:0/3856061933 learned_addr learned my addr 172.21.15.143:0/3856061933 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:18.112 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:18.102+0000 7fc3988a0700 1 --2- 172.21.15.143:0/3856061933 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc39406b240 0x7fc39415c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:18.112 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:18.102+0000 7fc3990a1700 1 --2- 172.21.15.143:0/3856061933 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc39415dba0 0x7fc3940c48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:18.112 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:18.102+0000 7fc393fff700 1 -- 172.21.15.143:0/3856061933 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc39406b240 msgr2=0x7fc39415c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:18.113 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:18.102+0000 7fc393fff700 1 --2- 172.21.15.143:0/3856061933 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc39406b240 0x7fc39415c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:18.113 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:18.102+0000 7fc393fff700 1 -- 172.21.15.143:0/3856061933 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc39415dba0 msgr2=0x7fc3940c48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:18.113 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:18.102+0000 7fc393fff700 1 --2- 172.21.15.143:0/3856061933 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc39415dba0 0x7fc3940c48d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:18.113 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:18.102+0000 7fc393fff700 1 -- 172.21.15.143:0/3856061933 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3800094a0 con 0x7fc3940cb530 2023-11-08T19:27:18.113 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:18.102+0000 7fc393fff700 1 --2- 172.21.15.143:0/3856061933 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc3940cb530 0x7fc3940c4390 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fc38000ebe0 tx=0x7fc38000eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:18.114 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:18.102+0000 7fc3917fa700 1 -- 172.21.15.143:0/3856061933 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc380009800 con 0x7fc3940cb530 2023-11-08T19:27:18.114 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:18.102+0000 7fc3917fa700 1 -- 172.21.15.143:0/3856061933 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc38000c910 con 0x7fc3940cb530 2023-11-08T19:27:18.114 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:18.102+0000 7fc39ab04700 1 -- 172.21.15.143:0/3856061933 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc38c009a70 con 0x7fc3940cb530 2023-11-08T19:27:18.114 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:18.102+0000 7fc3917fa700 1 -- 172.21.15.143:0/3856061933 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc38000cc80 con 0x7fc3940cb530 2023-11-08T19:27:18.114 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:18.102+0000 7fc39ab04700 1 -- 172.21.15.143:0/3856061933 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3940c57a0 con 0x7fc3940cb530 2023-11-08T19:27:18.115 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:18.103+0000 7fc39ab04700 1 -- 172.21.15.143:0/3856061933 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc39404eee0 con 0x7fc3940cb530 2023-11-08T19:27:18.115 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:18.104+0000 7fc3917fa700 1 -- 172.21.15.143:0/3856061933 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc380013470 con 0x7fc3940cb530 2023-11-08T19:27:18.115 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:18.104+0000 7fc3917fa700 1 --2- 172.21.15.143:0/3856061933 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc37806af00 0x7fc37806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:18.115 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:18.104+0000 7fc3917fa700 1 -- 172.21.15.143:0/3856061933 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6858+0+0 (secure 0 0 0) 0x7fc380089cb0 con 0x7fc3940cb530 2023-11-08T19:27:18.115 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:18.104+0000 7fc3988a0700 1 --2- 172.21.15.143:0/3856061933 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc37806af00 0x7fc37806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:18.116 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:18.105+0000 7fc3988a0700 1 --2- 172.21.15.143:0/3856061933 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc37806af00 0x7fc37806d3c0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fc388007fd0 tx=0x7fc388007dc0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:18.116 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:18.108+0000 7fc3917fa700 1 -- 172.21.15.143:0/3856061933 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc380054840 con 0x7fc3940cb530 2023-11-08T19:27:18.343 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:18.342+0000 7fc39ab04700 1 -- 172.21.15.143:0/3856061933 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fc3940b67e0 con 0x7fc3940cb530 2023-11-08T19:27:18.959 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:18.958+0000 7fc3917fa700 1 -- 172.21.15.143:0/3856061933 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v35) v1 ==== 104+0+0 (secure 0 0 0) 0x7fc380057e60 con 0x7fc3940cb530 2023-11-08T19:27:18.962 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:18.960+0000 7fc39ab04700 1 -- 172.21.15.143:0/3856061933 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc37806af00 msgr2=0x7fc37806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:18.962 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:18.960+0000 7fc39ab04700 1 --2- 172.21.15.143:0/3856061933 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc37806af00 0x7fc37806d3c0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fc388007fd0 tx=0x7fc388007dc0 comp rx=0 tx=0).stop 2023-11-08T19:27:18.963 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:18.960+0000 7fc39ab04700 1 -- 172.21.15.143:0/3856061933 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc3940cb530 msgr2=0x7fc3940c4390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:18.963 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:18.960+0000 7fc39ab04700 1 --2- 172.21.15.143:0/3856061933 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc3940cb530 0x7fc3940c4390 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fc38000ebe0 tx=0x7fc38000eef0 comp rx=0 tx=0).stop 2023-11-08T19:27:18.963 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:18.961+0000 7fc39ab04700 1 -- 172.21.15.143:0/3856061933 shutdown_connections 2023-11-08T19:27:18.963 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:18.961+0000 7fc39ab04700 1 --2- 172.21.15.143:0/3856061933 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc37806af00 0x7fc37806d3c0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:18.963 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:18.961+0000 7fc39ab04700 1 --2- 172.21.15.143:0/3856061933 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc39406b240 0x7fc39415c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:18.964 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:18.961+0000 7fc39ab04700 1 --2- 172.21.15.143:0/3856061933 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc39415dba0 0x7fc3940c48d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:18.964 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:18.961+0000 7fc39ab04700 1 --2- 172.21.15.143:0/3856061933 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc3940cb530 0x7fc3940c4390 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:18.964 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:18.961+0000 7fc39ab04700 1 -- 172.21.15.143:0/3856061933 >> 172.21.15.143:0/3856061933 conn(0x7fc3940cd2e0 msgr2=0x7fc39406c2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:18.964 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:18.961+0000 7fc39ab04700 1 -- 172.21.15.143:0/3856061933 shutdown_connections 2023-11-08T19:27:18.964 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:18.961+0000 7fc39ab04700 1 -- 172.21.15.143:0/3856061933 wait complete. 2023-11-08T19:27:18.977 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-08T19:27:19.144 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:19.143+0000 7f4b0c85e700 1 Processor -- start 2023-11-08T19:27:19.144 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:19.143+0000 7f4b0c85e700 1 -- start start 2023-11-08T19:27:19.147 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:19.144+0000 7f4b0c85e700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f4b0815a950 0x7f4b080d01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:19.147 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:19.144+0000 7f4b0c85e700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4b0806bd20 0x7f4b08072370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:19.147 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:19.144+0000 7f4b0c85e700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b080cc2a0 con 0x7f4b080728b0 2023-11-08T19:27:19.148 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:19.144+0000 7f4b0c85e700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b080cc130 con 0x7f4b0815a950 2023-11-08T19:27:19.148 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:19.144+0000 7f4b0c85e700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b0806c1b0 con 0x7f4b0806bd20 2023-11-08T19:27:19.148 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:19.144+0000 7f4b06d9d700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f4b080728b0 0x7f4b080cc730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:49390/0 (socket says 172.21.15.143:49390) 2023-11-08T19:27:19.148 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:19.144+0000 7f4b06d9d700 1 -- 172.21.15.143:0/4065088704 learned_addr learned my addr 172.21.15.143:0/4065088704 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:27:19.148 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:19.144+0000 7f4b0659c700 1 --2- 172.21.15.143:0/4065088704 >> v2:172.21.15.174:3300/0 conn(0x7f4b0815a950 0x7f4b080d01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:19.149 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:19.144+0000 7f4b05d9b700 1 --2- 172.21.15.143:0/4065088704 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4b0806bd20 0x7f4b08072370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:19.149 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:19.144+0000 7f4b0559a700 1 -- 172.21.15.143:0/4065088704 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2200685086 0 0) 0x7f4b080cc2a0 con 0x7f4b080728b0 2023-11-08T19:27:19.149 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:19.144+0000 7f4b0559a700 1 -- 172.21.15.143:0/4065088704 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4af00035b0 con 0x7f4b080728b0 2023-11-08T19:27:19.149 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:19.145+0000 7f4b0659c700 1 -- 172.21.15.143:0/4065088704 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4b0806bd20 msgr2=0x7f4b08072370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:19.149 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:19.145+0000 7f4b0659c700 1 --2- 172.21.15.143:0/4065088704 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4b0806bd20 0x7f4b08072370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:19.150 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:19.145+0000 7f4b0659c700 1 -- 172.21.15.143:0/4065088704 >> v1:172.21.15.143:6789/0 conn(0x7f4b080728b0 legacy=0x7f4b080cc730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:19.150 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:19.145+0000 7f4b0559a700 1 -- 172.21.15.143:0/4065088704 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 616312138 0 0) 0x7f4af00035b0 con 0x7f4b080728b0 2023-11-08T19:27:19.150 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:19.145+0000 7f4b0659c700 1 -- 172.21.15.143:0/4065088704 --> v2:172.21.15.174:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b0806c490 con 0x7f4b0815a950 2023-11-08T19:27:19.150 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:19.145+0000 7f4b05d9b700 1 --2- 172.21.15.143:0/4065088704 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4b0806bd20 0x7f4b08072370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T19:27:19.150 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:19.145+0000 7f4b0659c700 1 --2- 172.21.15.143:0/4065088704 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4b0815a950 0x7f4b080d01b0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f4afc009840 tx=0x7f4afc009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=16cd0ab183c158b server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:19.151 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:19.145+0000 7f4b0559a700 1 -- 172.21.15.143:0/4065088704 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b000019f0 con 0x7f4b0815a950 2023-11-08T19:27:19.151 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:19.145+0000 7f4b0559a700 1 -- 172.21.15.143:0/4065088704 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4afc004030 con 0x7f4b0815a950 2023-11-08T19:27:19.151 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:19.145+0000 7f4b0559a700 1 -- 172.21.15.143:0/4065088704 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4afc0043a0 con 0x7f4b0815a950 2023-11-08T19:27:19.151 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:19.146+0000 7f4b0c85e700 1 -- 172.21.15.143:0/4065088704 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4b0815a950 msgr2=0x7f4b080d01b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:19.151 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:19.146+0000 7f4b0c85e700 1 --2- 172.21.15.143:0/4065088704 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4b0815a950 0x7f4b080d01b0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f4afc009840 tx=0x7f4afc009b50 comp rx=0 tx=0).stop 2023-11-08T19:27:19.152 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:19.146+0000 7f4b0c85e700 1 -- 172.21.15.143:0/4065088704 shutdown_connections 2023-11-08T19:27:19.152 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:19.146+0000 7f4b0c85e700 1 --2- 172.21.15.143:0/4065088704 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4b0815a950 0x7f4b080d01b0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:19.152 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:19.146+0000 7f4b0c85e700 1 --2- 172.21.15.143:0/4065088704 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4b0806bd20 0x7f4b08072370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:19.152 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:19.146+0000 7f4b0c85e700 1 -- 172.21.15.143:0/4065088704 >> 172.21.15.143:0/4065088704 conn(0x7f4b080cd2e0 msgr2=0x7f4b080cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:19.152 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:19.146+0000 7f4b0c85e700 1 -- 172.21.15.143:0/4065088704 shutdown_connections 2023-11-08T19:27:19.153 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:19.146+0000 7f4b0c85e700 1 -- 172.21.15.143:0/4065088704 wait complete. 2023-11-08T19:27:19.156 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:19.147+0000 7f4b0c85e700 1 Processor -- start 2023-11-08T19:27:19.156 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:19.147+0000 7f4b0c85e700 1 -- start start 2023-11-08T19:27:19.156 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:19.148+0000 7f4b0c85e700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4b080728b0 0x7f4b081526e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:19.156 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:19.148+0000 7f4b0c85e700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4b0815a950 0x7f4b08158e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:19.157 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:19.148+0000 7f4b0c85e700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4b0815af70 0x7f4b0815e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:19.157 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:19.148+0000 7f4b0c85e700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b0815b530 con 0x7f4b0815af70 2023-11-08T19:27:19.157 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:19.148+0000 7f4b0c85e700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b0815bf50 con 0x7f4b080728b0 2023-11-08T19:27:19.157 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:19.148+0000 7f4b0c85e700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b081668c0 con 0x7f4b0815a950 2023-11-08T19:27:19.157 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:19.148+0000 7f4b06d9d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4b0815af70 0x7f4b0815e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:19.158 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:19.148+0000 7f4b06d9d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4b0815af70 0x7f4b0815e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53242/0 (socket says 172.21.15.143:53242) 2023-11-08T19:27:19.158 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:19.148+0000 7f4b06d9d700 1 -- 172.21.15.143:0/4112192323 learned_addr learned my addr 172.21.15.143:0/4112192323 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:19.158 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:19.149+0000 7f4b05d9b700 1 --2- 172.21.15.143:0/4112192323 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4b0815a950 0x7f4b08158e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:19.158 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:19.149+0000 7f4b0659c700 1 --2- 172.21.15.143:0/4112192323 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4b080728b0 0x7f4b081526e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:19.158 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:19.149+0000 7f4b06d9d700 1 -- 172.21.15.143:0/4112192323 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4b0815a950 msgr2=0x7f4b08158e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:19.159 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:19.149+0000 7f4b06d9d700 1 --2- 172.21.15.143:0/4112192323 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4b0815a950 0x7f4b08158e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:19.159 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:19.149+0000 7f4b06d9d700 1 -- 172.21.15.143:0/4112192323 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4b080728b0 msgr2=0x7f4b081526e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:19.159 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:19.149+0000 7f4b06d9d700 1 --2- 172.21.15.143:0/4112192323 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4b080728b0 0x7f4b081526e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:19.159 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:19.149+0000 7f4b06d9d700 1 -- 172.21.15.143:0/4112192323 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4af80094a0 con 0x7f4b0815af70 2023-11-08T19:27:19.159 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:19.149+0000 7f4b0659c700 1 --2- 172.21.15.143:0/4112192323 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4b080728b0 0x7f4b081526e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:27:19.160 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:19.149+0000 7f4b06d9d700 1 --2- 172.21.15.143:0/4112192323 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4b0815af70 0x7f4b0815e290 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f4b0000cca0 tx=0x7f4b0000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:19.160 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:19.149+0000 7f4af6ffd700 1 -- 172.21.15.143:0/4112192323 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b0000aba0 con 0x7f4b0815af70 2023-11-08T19:27:19.160 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:19.149+0000 7f4af6ffd700 1 -- 172.21.15.143:0/4112192323 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b0000ad00 con 0x7f4b0815af70 2023-11-08T19:27:19.160 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:19.149+0000 7f4b0c85e700 1 -- 172.21.15.143:0/4112192323 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4afc0094a0 con 0x7f4b0815af70 2023-11-08T19:27:19.160 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:19.149+0000 7f4af6ffd700 1 -- 172.21.15.143:0/4112192323 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b00007310 con 0x7f4b0815af70 2023-11-08T19:27:19.161 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:19.149+0000 7f4b0c85e700 1 -- 172.21.15.143:0/4112192323 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b0815bc70 con 0x7f4b0815af70 2023-11-08T19:27:19.161 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:19.150+0000 7f4b0c85e700 1 -- 172.21.15.143:0/4112192323 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b0804eee0 con 0x7f4b0815af70 2023-11-08T19:27:19.161 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:19.151+0000 7f4af6ffd700 1 -- 172.21.15.143:0/4112192323 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4b00007470 con 0x7f4b0815af70 2023-11-08T19:27:19.162 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:19.151+0000 7f4af6ffd700 1 --2- 172.21.15.143:0/4112192323 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f4ae806af00 0x7f4ae806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:19.162 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:19.151+0000 7f4af6ffd700 1 -- 172.21.15.143:0/4112192323 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6858+0+0 (secure 0 0 0) 0x7f4b00089900 con 0x7f4b0815af70 2023-11-08T19:27:19.162 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:19.151+0000 7f4b0659c700 1 --2- 172.21.15.143:0/4112192323 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f4ae806af00 0x7f4ae806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:19.162 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:19.151+0000 7f4b0659c700 1 --2- 172.21.15.143:0/4112192323 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f4ae806af00 0x7f4ae806d3c0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f4afc000c00 tx=0x7f4afc013040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:19.163 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:19.155+0000 7f4af6ffd700 1 -- 172.21.15.143:0/4112192323 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4b00054530 con 0x7f4b0815af70 2023-11-08T19:27:19.348 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:19.347+0000 7f4b0c85e700 1 -- 172.21.15.143:0/4112192323 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f4b08072150 con 0x7f4b0815af70 2023-11-08T19:27:20.243 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:20.242+0000 7f4af6ffd700 1 -- 172.21.15.143:0/4112192323 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v69) v1 ==== 181+0+0 (secure 0 0 0) 0x7f4b000180d0 con 0x7f4b0815af70 2023-11-08T19:27:20.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:20.245+0000 7f4b0c85e700 1 -- 172.21.15.143:0/4112192323 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f4ae806af00 msgr2=0x7f4ae806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:20.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:20.245+0000 7f4b0c85e700 1 --2- 172.21.15.143:0/4112192323 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f4ae806af00 0x7f4ae806d3c0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f4afc000c00 tx=0x7f4afc013040 comp rx=0 tx=0).stop 2023-11-08T19:27:20.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:20.245+0000 7f4b0c85e700 1 -- 172.21.15.143:0/4112192323 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4b0815af70 msgr2=0x7f4b0815e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:20.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:20.245+0000 7f4b0c85e700 1 --2- 172.21.15.143:0/4112192323 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4b0815af70 0x7f4b0815e290 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f4b0000cca0 tx=0x7f4b0000a060 comp rx=0 tx=0).stop 2023-11-08T19:27:20.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:20.245+0000 7f4b0c85e700 1 -- 172.21.15.143:0/4112192323 shutdown_connections 2023-11-08T19:27:20.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:20.245+0000 7f4b0c85e700 1 --2- 172.21.15.143:0/4112192323 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f4ae806af00 0x7f4ae806d3c0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:20.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:20.245+0000 7f4b0c85e700 1 --2- 172.21.15.143:0/4112192323 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4b0815af70 0x7f4b0815e290 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:20.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:20.245+0000 7f4b0c85e700 1 --2- 172.21.15.143:0/4112192323 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4b080728b0 0x7f4b081526e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:20.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:20.245+0000 7f4b0c85e700 1 --2- 172.21.15.143:0/4112192323 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4b0815a950 0x7f4b08158e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:20.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:20.245+0000 7f4b0c85e700 1 -- 172.21.15.143:0/4112192323 >> 172.21.15.143:0/4112192323 conn(0x7f4b080cd2e0 msgr2=0x7f4b081536e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:20.249 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:20.245+0000 7f4b0c85e700 1 -- 172.21.15.143:0/4112192323 shutdown_connections 2023-11-08T19:27:20.249 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:20.245+0000 7f4b0c85e700 1 -- 172.21.15.143:0/4112192323 wait complete. 2023-11-08T19:27:20.249 INFO:teuthology.orchestra.run.smithi143.stderr:pool 'cephfs_metadata' removed 2023-11-08T19:27:20.262 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-08T19:27:20.397 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:20.395+0000 7f38185a5700 1 Processor -- start 2023-11-08T19:27:20.397 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:20.395+0000 7f38185a5700 1 -- start start 2023-11-08T19:27:20.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:20.396+0000 7f38185a5700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f381015a950 0x7f38100d01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:20.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:20.396+0000 7f38185a5700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f38100b61e0 0x7f38100cc730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:20.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:20.396+0000 7f38185a5700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38100cc2a0 con 0x7f381006bd20 2023-11-08T19:27:20.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:20.396+0000 7f38185a5700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38100cc130 con 0x7f38100b61e0 2023-11-08T19:27:20.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:20.396+0000 7f38185a5700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f381006c1b0 con 0x7f381015a950 2023-11-08T19:27:20.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:20.396+0000 7f3815b40700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f381006bd20 0x7f3810072370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:49404/0 (socket says 172.21.15.143:49404) 2023-11-08T19:27:20.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:20.396+0000 7f3815b40700 1 -- 172.21.15.143:0/1120960667 learned_addr learned my addr 172.21.15.143:0/1120960667 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:27:20.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:20.397+0000 7f381533f700 1 -- 172.21.15.143:0/1120960667 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3709825602 0 0) 0x7f38100cc2a0 con 0x7f381006bd20 2023-11-08T19:27:20.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:20.397+0000 7f381533f700 1 -- 172.21.15.143:0/1120960667 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f37fc0035b0 con 0x7f381006bd20 2023-11-08T19:27:20.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:20.397+0000 7f3816341700 1 --2- 172.21.15.143:0/1120960667 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f381015a950 0x7f38100d01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:20.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:20.397+0000 7f3816b42700 1 --2- 172.21.15.143:0/1120960667 >> v2:172.21.15.174:3300/0 conn(0x7f38100b61e0 0x7f38100cc730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:20.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:20.397+0000 7f381533f700 1 -- 172.21.15.143:0/1120960667 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 987441387 0 0) 0x7f37fc0035b0 con 0x7f381006bd20 2023-11-08T19:27:20.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:20.397+0000 7f381533f700 1 -- 172.21.15.143:0/1120960667 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f381015a950 msgr2=0x7f38100d01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:20.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:20.397+0000 7f381533f700 1 --2- 172.21.15.143:0/1120960667 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f381015a950 0x7f38100d01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:20.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:20.397+0000 7f381533f700 1 -- 172.21.15.143:0/1120960667 >> v2:172.21.15.174:3300/0 conn(0x7f38100b61e0 msgr2=0x7f38100cc730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:20.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:20.397+0000 7f381533f700 1 --2- 172.21.15.143:0/1120960667 >> v2:172.21.15.174:3300/0 conn(0x7f38100b61e0 0x7f38100cc730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:20.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:20.397+0000 7f381533f700 1 -- 172.21.15.143:0/1120960667 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f381006c490 con 0x7f381006bd20 2023-11-08T19:27:20.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:20.397+0000 7f381533f700 1 -- 172.21.15.143:0/1120960667 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4147756303 0 0) 0x7f3800001670 con 0x7f381006bd20 2023-11-08T19:27:20.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:20.397+0000 7f381533f700 1 -- 172.21.15.143:0/1120960667 >> v1:172.21.15.143:6789/0 conn(0x7f381006bd20 legacy=0x7f3810072370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:20.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:20.397+0000 7f381533f700 1 --2- 172.21.15.143:0/1120960667 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f37fc003a00 0x7f37fc003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:20.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:20.397+0000 7f381533f700 1 --2- 172.21.15.143:0/1120960667 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f37fc004850 0x7f37fc004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:20.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:20.397+0000 7f381533f700 1 --2- 172.21.15.143:0/1120960667 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f381015a950 0x7f37fc005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:20.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:20.397+0000 7f381533f700 1 -- 172.21.15.143:0/1120960667 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f381006c1b0 con 0x7f37fc004850 2023-11-08T19:27:20.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:20.397+0000 7f381533f700 1 -- 172.21.15.143:0/1120960667 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37fc00ba10 con 0x7f37fc003a00 2023-11-08T19:27:20.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:20.397+0000 7f381533f700 1 -- 172.21.15.143:0/1120960667 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37fc00bb50 con 0x7f381015a950 2023-11-08T19:27:20.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:20.397+0000 7f3815b40700 1 --2- 172.21.15.143:0/1120960667 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f37fc004850 0x7f37fc004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:20.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:20.397+0000 7f3816341700 1 --2- 172.21.15.143:0/1120960667 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f37fc003a00 0x7f37fc003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:20.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:20.397+0000 7f3816b42700 1 --2- 172.21.15.143:0/1120960667 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f381015a950 0x7f37fc005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:20.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:20.397+0000 7f3815b40700 1 -- 172.21.15.143:0/1120960667 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f381015a950 msgr2=0x7f37fc005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:20.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:20.397+0000 7f3815b40700 1 --2- 172.21.15.143:0/1120960667 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f381015a950 0x7f37fc005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:20.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:20.397+0000 7f3815b40700 1 -- 172.21.15.143:0/1120960667 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f37fc003a00 msgr2=0x7f37fc003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:20.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:20.397+0000 7f3815b40700 1 --2- 172.21.15.143:0/1120960667 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f37fc003a00 0x7f37fc003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:20.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:20.397+0000 7f3815b40700 1 -- 172.21.15.143:0/1120960667 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37fc0035b0 con 0x7f37fc004850 2023-11-08T19:27:20.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:20.398+0000 7f3815b40700 1 --2- 172.21.15.143:0/1120960667 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f37fc004850 0x7f37fc004cd0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f380000c350 tx=0x7f37fc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e664366c60d4bdad server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:20.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:20.398+0000 7f381533f700 1 -- 172.21.15.143:0/1120960667 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f380000ce50 con 0x7f37fc004850 2023-11-08T19:27:20.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:20.398+0000 7f381533f700 1 -- 172.21.15.143:0/1120960667 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3800005960 con 0x7f37fc004850 2023-11-08T19:27:20.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:20.398+0000 7f381533f700 1 -- 172.21.15.143:0/1120960667 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3800005d10 con 0x7f37fc004850 2023-11-08T19:27:20.409 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:20.398+0000 7f38185a5700 1 -- 172.21.15.143:0/1120960667 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f37fc004850 msgr2=0x7f37fc004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:20.409 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:20.398+0000 7f38185a5700 1 --2- 172.21.15.143:0/1120960667 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f37fc004850 0x7f37fc004cd0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f380000c350 tx=0x7f37fc005880 comp rx=0 tx=0).stop 2023-11-08T19:27:20.409 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:20.398+0000 7f3816b42700 1 -- 172.21.15.143:0/1120960667 reap_dead start 2023-11-08T19:27:20.409 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:20.398+0000 7f38185a5700 1 -- 172.21.15.143:0/1120960667 shutdown_connections 2023-11-08T19:27:20.409 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:20.398+0000 7f38185a5700 1 -- 172.21.15.143:0/1120960667 >> 172.21.15.143:0/1120960667 conn(0x7f38100cd2e0 msgr2=0x7f38100cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:20.410 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:20.398+0000 7f38185a5700 1 -- 172.21.15.143:0/1120960667 shutdown_connections 2023-11-08T19:27:20.410 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:20.398+0000 7f38185a5700 1 -- 172.21.15.143:0/1120960667 wait complete. 2023-11-08T19:27:20.410 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:20.400+0000 7f38185a5700 1 Processor -- start 2023-11-08T19:27:20.410 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:20.400+0000 7f38185a5700 1 -- start start 2023-11-08T19:27:20.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:20.400+0000 7f38185a5700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f37fc004850 0x7f38101f34a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:20.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:20.400+0000 7f38185a5700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f38100c63c0 0x7f38101f6a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:20.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:20.400+0000 7f38185a5700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f38100c6840 0x7f3810159a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:20.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:20.400+0000 7f38185a5700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38100c7930 con 0x7f37fc004850 2023-11-08T19:27:20.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:20.400+0000 7f38185a5700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38101f74f0 con 0x7f38100c63c0 2023-11-08T19:27:20.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:20.400+0000 7f38185a5700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38101f77d0 con 0x7f38100c6840 2023-11-08T19:27:20.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:20.400+0000 7f3816341700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f37fc004850 0x7f38101f34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:20.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:20.400+0000 7f3816341700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f37fc004850 0x7f38101f34a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53258/0 (socket says 172.21.15.143:53258) 2023-11-08T19:27:20.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:20.400+0000 7f3816341700 1 -- 172.21.15.143:0/22306832 learned_addr learned my addr 172.21.15.143:0/22306832 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:20.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:20.401+0000 7f3816b42700 1 --2- 172.21.15.143:0/22306832 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f38100c6840 0x7f3810159a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:20.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:20.401+0000 7f3815b40700 1 --2- 172.21.15.143:0/22306832 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f38100c63c0 0x7f38101f6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:20.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:20.401+0000 7f3816341700 1 -- 172.21.15.143:0/22306832 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f38100c6840 msgr2=0x7f3810159a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:20.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:20.401+0000 7f3816341700 1 --2- 172.21.15.143:0/22306832 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f38100c6840 0x7f3810159a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:20.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:20.401+0000 7f3816341700 1 -- 172.21.15.143:0/22306832 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f38100c63c0 msgr2=0x7f38101f6a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:20.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:20.401+0000 7f3816341700 1 --2- 172.21.15.143:0/22306832 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f38100c63c0 0x7f38101f6a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:20.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:20.401+0000 7f3816341700 1 -- 172.21.15.143:0/22306832 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38100c7b40 con 0x7f37fc004850 2023-11-08T19:27:20.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:20.401+0000 7f3816341700 1 --2- 172.21.15.143:0/22306832 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f37fc004850 0x7f38101f34a0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f380c0119a0 tx=0x7f380c011d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:20.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:20.401+0000 7f3806ffd700 1 -- 172.21.15.143:0/22306832 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f380c00f940 con 0x7f37fc004850 2023-11-08T19:27:20.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:20.401+0000 7f3806ffd700 1 -- 172.21.15.143:0/22306832 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f380c00faa0 con 0x7f37fc004850 2023-11-08T19:27:20.417 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:20.401+0000 7f38185a5700 1 -- 172.21.15.143:0/22306832 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38100c7e20 con 0x7f37fc004850 2023-11-08T19:27:20.417 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:20.401+0000 7f3806ffd700 1 -- 172.21.15.143:0/22306832 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f380c00fe30 con 0x7f37fc004850 2023-11-08T19:27:20.417 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:20.401+0000 7f38185a5700 1 -- 172.21.15.143:0/22306832 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38101f39e0 con 0x7f37fc004850 2023-11-08T19:27:20.417 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:20.402+0000 7f38185a5700 1 -- 172.21.15.143:0/22306832 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38101f7990 con 0x7f37fc004850 2023-11-08T19:27:20.417 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:20.403+0000 7f3806ffd700 1 -- 172.21.15.143:0/22306832 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f380c014440 con 0x7f37fc004850 2023-11-08T19:27:20.418 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:20.403+0000 7f3806ffd700 1 --2- 172.21.15.143:0/22306832 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f37f406ae90 0x7f37f406d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:20.418 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:20.403+0000 7f3806ffd700 1 -- 172.21.15.143:0/22306832 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6420+0+0 (secure 0 0 0) 0x7f380c089f70 con 0x7f37fc004850 2023-11-08T19:27:20.418 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:20.403+0000 7f3815b40700 1 --2- 172.21.15.143:0/22306832 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f37f406ae90 0x7f37f406d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:20.418 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:20.404+0000 7f3815b40700 1 --2- 172.21.15.143:0/22306832 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f37f406ae90 0x7f37f406d350 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f3800008fd0 tx=0x7f380000a000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:20.418 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:20.407+0000 7f3806ffd700 1 -- 172.21.15.143:0/22306832 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f380c0bcca0 con 0x7f37fc004850 2023-11-08T19:27:20.598 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:20.597+0000 7f38185a5700 1 -- 172.21.15.143:0/22306832 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f381004eee0 con 0x7f37fc004850 2023-11-08T19:27:21.251 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.250+0000 7f3806ffd700 1 -- 172.21.15.143:0/22306832 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v70) v1 ==== 169+0+0 (secure 0 0 0) 0x7f380c054e00 con 0x7f37fc004850 2023-11-08T19:27:21.254 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.252+0000 7f38185a5700 1 -- 172.21.15.143:0/22306832 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f37f406ae90 msgr2=0x7f37f406d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:21.254 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.252+0000 7f38185a5700 1 --2- 172.21.15.143:0/22306832 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f37f406ae90 0x7f37f406d350 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f3800008fd0 tx=0x7f380000a000 comp rx=0 tx=0).stop 2023-11-08T19:27:21.254 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.252+0000 7f38185a5700 1 -- 172.21.15.143:0/22306832 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f37fc004850 msgr2=0x7f38101f34a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:21.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.252+0000 7f38185a5700 1 --2- 172.21.15.143:0/22306832 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f37fc004850 0x7f38101f34a0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f380c0119a0 tx=0x7f380c011d60 comp rx=0 tx=0).stop 2023-11-08T19:27:21.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.252+0000 7f38185a5700 1 -- 172.21.15.143:0/22306832 shutdown_connections 2023-11-08T19:27:21.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.252+0000 7f38185a5700 1 --2- 172.21.15.143:0/22306832 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f37f406ae90 0x7f37f406d350 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:21.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.252+0000 7f38185a5700 1 --2- 172.21.15.143:0/22306832 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f38100c6840 0x7f3810159a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:21.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.252+0000 7f38185a5700 1 --2- 172.21.15.143:0/22306832 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f37fc004850 0x7f38101f34a0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:21.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.252+0000 7f38185a5700 1 --2- 172.21.15.143:0/22306832 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f38100c63c0 0x7f38101f6a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:21.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.252+0000 7f38185a5700 1 -- 172.21.15.143:0/22306832 >> 172.21.15.143:0/22306832 conn(0x7f38100cd2e0 msgr2=0x7f38100d01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:21.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.252+0000 7f38185a5700 1 -- 172.21.15.143:0/22306832 shutdown_connections 2023-11-08T19:27:21.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.252+0000 7f38185a5700 1 -- 172.21.15.143:0/22306832 wait complete. 2023-11-08T19:27:21.257 INFO:teuthology.orchestra.run.smithi143.stderr:pool 'cephfs_data' removed 2023-11-08T19:27:21.266 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-08T19:27:21.397 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.396+0000 7f39d5192700 1 Processor -- start 2023-11-08T19:27:21.398 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.396+0000 7f39d5192700 1 -- start start 2023-11-08T19:27:21.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.397+0000 7f39d5192700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f39d015dca0 0x7f39d00b93f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:21.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.397+0000 7f39d5192700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f39d00b5630 0x7f39d00b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:21.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.397+0000 7f39d5192700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f39d00b6b70 0x7f39d00b6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:21.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.397+0000 7f39d5192700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39d00b3360 con 0x7f39d00b6b70 2023-11-08T19:27:21.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.397+0000 7f39d5192700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39d00b5ff0 con 0x7f39d015dca0 2023-11-08T19:27:21.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.397+0000 7f39d5192700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39d00b8f50 con 0x7f39d00b5630 2023-11-08T19:27:21.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.397+0000 7f39cf59e700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f39d00b6b70 0x7f39d00b6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:21.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.397+0000 7f39cf59e700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f39d00b6b70 0x7f39d00b6fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53270/0 (socket says 172.21.15.143:53270) 2023-11-08T19:27:21.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.397+0000 7f39cf59e700 1 -- 172.21.15.143:0/218050249 learned_addr learned my addr 172.21.15.143:0/218050249 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:21.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.397+0000 7f39ced9d700 1 --2- 172.21.15.143:0/218050249 >> v2:172.21.15.174:3300/0 conn(0x7f39d015dca0 0x7f39d00b93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:21.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.397+0000 7f39ce59c700 1 --2- 172.21.15.143:0/218050249 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f39d00b5630 0x7f39d00b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:21.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.397+0000 7f39cf59e700 1 -- 172.21.15.143:0/218050249 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f39d00b5630 msgr2=0x7f39d00b5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:21.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.397+0000 7f39cf59e700 1 --2- 172.21.15.143:0/218050249 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f39d00b5630 0x7f39d00b5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:21.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.397+0000 7f39cf59e700 1 -- 172.21.15.143:0/218050249 >> v2:172.21.15.174:3300/0 conn(0x7f39d015dca0 msgr2=0x7f39d00b93f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:21.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.397+0000 7f39cf59e700 1 --2- 172.21.15.143:0/218050249 >> v2:172.21.15.174:3300/0 conn(0x7f39d015dca0 0x7f39d00b93f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:21.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.397+0000 7f39cf59e700 1 -- 172.21.15.143:0/218050249 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39d00b7eb0 con 0x7f39d00b6b70 2023-11-08T19:27:21.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.397+0000 7f39cf59e700 1 --2- 172.21.15.143:0/218050249 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f39d00b6b70 0x7f39d00b6fc0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f39d00cf650 tx=0x7f39c800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e325fd8319b90c00 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:21.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.397+0000 7f39cdd9b700 1 -- 172.21.15.143:0/218050249 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39c800e070 con 0x7f39d00b6b70 2023-11-08T19:27:21.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.397+0000 7f39cdd9b700 1 -- 172.21.15.143:0/218050249 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39c800b9c0 con 0x7f39d00b6b70 2023-11-08T19:27:21.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.397+0000 7f39cdd9b700 1 -- 172.21.15.143:0/218050249 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39c800bd70 con 0x7f39d00b6b70 2023-11-08T19:27:21.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.398+0000 7f39d5192700 1 -- 172.21.15.143:0/218050249 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f39d00b6b70 msgr2=0x7f39d00b6fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:21.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.398+0000 7f39d5192700 1 --2- 172.21.15.143:0/218050249 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f39d00b6b70 0x7f39d00b6fc0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f39d00cf650 tx=0x7f39c800b330 comp rx=0 tx=0).stop 2023-11-08T19:27:21.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.398+0000 7f39d5192700 1 -- 172.21.15.143:0/218050249 shutdown_connections 2023-11-08T19:27:21.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.398+0000 7f39d5192700 1 --2- 172.21.15.143:0/218050249 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f39d00b6b70 0x7f39d00b6fc0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:21.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.398+0000 7f39d5192700 1 --2- 172.21.15.143:0/218050249 >> v2:172.21.15.174:3300/0 conn(0x7f39d015dca0 0x7f39d00b93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:21.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.398+0000 7f39d5192700 1 --2- 172.21.15.143:0/218050249 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f39d00b5630 0x7f39d00b5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:21.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.398+0000 7f39d5192700 1 -- 172.21.15.143:0/218050249 >> 172.21.15.143:0/218050249 conn(0x7f39d0066d00 msgr2=0x7f39d0067110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:21.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.398+0000 7f39d5192700 1 -- 172.21.15.143:0/218050249 shutdown_connections 2023-11-08T19:27:21.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.398+0000 7f39d5192700 1 -- 172.21.15.143:0/218050249 wait complete. 2023-11-08T19:27:21.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.400+0000 7f39d5192700 1 Processor -- start 2023-11-08T19:27:21.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.400+0000 7f39d5192700 1 -- start start 2023-11-08T19:27:21.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.400+0000 7f39d5192700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f39d00b5630 0x7f39d01604a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:21.409 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.400+0000 7f39d5192700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f39d00b6b70 0x7f39d0153840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:21.409 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.400+0000 7f39d5192700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f39d015dca0 0x7f39d015ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:21.409 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.400+0000 7f39d5192700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39d00cac80 con 0x7f39d00b6b70 2023-11-08T19:27:21.409 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.400+0000 7f39d5192700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39d00caed0 con 0x7f39d00b5630 2023-11-08T19:27:21.409 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.400+0000 7f39d5192700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39d00cb1b0 con 0x7f39d015dca0 2023-11-08T19:27:21.410 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.400+0000 7f39ce59c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f39d00b6b70 0x7f39d0153840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:21.410 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.400+0000 7f39ce59c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f39d00b6b70 0x7f39d0153840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53274/0 (socket says 172.21.15.143:53274) 2023-11-08T19:27:21.410 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.400+0000 7f39ce59c700 1 -- 172.21.15.143:0/2446287924 learned_addr learned my addr 172.21.15.143:0/2446287924 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:21.410 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.401+0000 7f39cf59e700 1 --2- 172.21.15.143:0/2446287924 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f39d015dca0 0x7f39d015ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:21.410 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.401+0000 7f39ced9d700 1 --2- 172.21.15.143:0/2446287924 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f39d00b5630 0x7f39d01604a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:21.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.401+0000 7f39ce59c700 1 -- 172.21.15.143:0/2446287924 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f39d015dca0 msgr2=0x7f39d015ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:21.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.401+0000 7f39ce59c700 1 --2- 172.21.15.143:0/2446287924 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f39d015dca0 0x7f39d015ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:21.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.401+0000 7f39ce59c700 1 -- 172.21.15.143:0/2446287924 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f39d00b5630 msgr2=0x7f39d01604a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:21.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.401+0000 7f39ce59c700 1 --2- 172.21.15.143:0/2446287924 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f39d00b5630 0x7f39d01604a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:21.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.401+0000 7f39ce59c700 1 -- 172.21.15.143:0/2446287924 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39c8009b30 con 0x7f39d00b6b70 2023-11-08T19:27:21.412 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.401+0000 7f39ce59c700 1 --2- 172.21.15.143:0/2446287924 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f39d00b6b70 0x7f39d0153840 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f39c400e810 tx=0x7f39c400eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:21.412 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.401+0000 7f39b77fe700 1 -- 172.21.15.143:0/2446287924 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39c400c070 con 0x7f39d00b6b70 2023-11-08T19:27:21.412 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.401+0000 7f39b77fe700 1 -- 172.21.15.143:0/2446287924 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39c4013610 con 0x7f39d00b6b70 2023-11-08T19:27:21.412 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.401+0000 7f39d5192700 1 -- 172.21.15.143:0/2446287924 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39d00cb490 con 0x7f39d00b6b70 2023-11-08T19:27:21.412 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.401+0000 7f39b77fe700 1 -- 172.21.15.143:0/2446287924 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39c40139a0 con 0x7f39d00b6b70 2023-11-08T19:27:21.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.401+0000 7f39d5192700 1 -- 172.21.15.143:0/2446287924 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39d0153d80 con 0x7f39d00b6b70 2023-11-08T19:27:21.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.402+0000 7f39d5192700 1 -- 172.21.15.143:0/2446287924 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39d004eee0 con 0x7f39d00b6b70 2023-11-08T19:27:21.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.403+0000 7f39b77fe700 1 -- 172.21.15.143:0/2446287924 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f39c4013b00 con 0x7f39d00b6b70 2023-11-08T19:27:21.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.403+0000 7f39b77fe700 1 --2- 172.21.15.143:0/2446287924 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f39bc06af00 0x7f39bc06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:21.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.403+0000 7f39b77fe700 1 -- 172.21.15.143:0/2446287924 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6006+0+0 (secure 0 0 0) 0x7f39c4089370 con 0x7f39d00b6b70 2023-11-08T19:27:21.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.403+0000 7f39ced9d700 1 --2- 172.21.15.143:0/2446287924 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f39bc06af00 0x7f39bc06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:21.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.404+0000 7f39ced9d700 1 --2- 172.21.15.143:0/2446287924 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f39bc06af00 0x7f39bc06d3c0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f39d00cdd80 tx=0x7f39c0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:21.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.407+0000 7f39b77fe700 1 -- 172.21.15.143:0/2446287924 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f39c4054380 con 0x7f39d00b6b70 2023-11-08T19:27:21.598 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.597+0000 7f39d5192700 1 -- 172.21.15.143:0/2446287924 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f39d006ca60 con 0x7f39d00b6b70 2023-11-08T19:27:21.601 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.600+0000 7f39b77fe700 1 -- 172.21.15.143:0/2446287924 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v16) v1 ==== 104+0+0 (secure 0 0 0) 0x7f39c40579a0 con 0x7f39d00b6b70 2023-11-08T19:27:21.604 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.603+0000 7f39d5192700 1 -- 172.21.15.143:0/2446287924 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f39bc06af00 msgr2=0x7f39bc06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:21.604 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.603+0000 7f39d5192700 1 --2- 172.21.15.143:0/2446287924 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f39bc06af00 0x7f39bc06d3c0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f39d00cdd80 tx=0x7f39c0008040 comp rx=0 tx=0).stop 2023-11-08T19:27:21.605 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.603+0000 7f39d5192700 1 -- 172.21.15.143:0/2446287924 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f39d00b6b70 msgr2=0x7f39d0153840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:21.605 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.603+0000 7f39d5192700 1 --2- 172.21.15.143:0/2446287924 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f39d00b6b70 0x7f39d0153840 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f39c400e810 tx=0x7f39c400eb20 comp rx=0 tx=0).stop 2023-11-08T19:27:21.605 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.603+0000 7f39d5192700 1 -- 172.21.15.143:0/2446287924 shutdown_connections 2023-11-08T19:27:21.605 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.603+0000 7f39d5192700 1 --2- 172.21.15.143:0/2446287924 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f39bc06af00 0x7f39bc06d3c0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:21.606 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.603+0000 7f39d5192700 1 --2- 172.21.15.143:0/2446287924 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f39d015dca0 0x7f39d015ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:21.606 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.603+0000 7f39d5192700 1 --2- 172.21.15.143:0/2446287924 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f39d00b5630 0x7f39d01604a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:21.606 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.603+0000 7f39d5192700 1 --2- 172.21.15.143:0/2446287924 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f39d00b6b70 0x7f39d0153840 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:21.606 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.603+0000 7f39d5192700 1 -- 172.21.15.143:0/2446287924 >> 172.21.15.143:0/2446287924 conn(0x7f39d0066d00 msgr2=0x7f39d00b3610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:21.606 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.603+0000 7f39d5192700 1 -- 172.21.15.143:0/2446287924 shutdown_connections 2023-11-08T19:27:21.607 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.603+0000 7f39d5192700 1 -- 172.21.15.143:0/2446287924 wait complete. 2023-11-08T19:27:21.616 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snapshots.TestMonSnapsAndFsPools.test_using_pool_with_snap_fails_fs_creation' 2023-11-08T19:27:21.736 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.735+0000 7f37fb89c700 1 Processor -- start 2023-11-08T19:27:21.737 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.735+0000 7f37fb89c700 1 -- start start 2023-11-08T19:27:21.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.736+0000 7f37fb89c700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f37f415aa30 0x7f37f40cf510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:21.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.736+0000 7f37fb89c700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37f40cf080 con 0x7f37f40b2210 2023-11-08T19:27:21.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.736+0000 7f37fb89c700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37f40ce470 con 0x7f37f40cc530 2023-11-08T19:27:21.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.736+0000 7f37fb89c700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37f40ce750 con 0x7f37f415aa30 2023-11-08T19:27:21.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.736+0000 7f37f8e37700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f37f40b2210 0x7f37f40cbe20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:49434/0 (socket says 172.21.15.143:49434) 2023-11-08T19:27:21.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.736+0000 7f37f8e37700 1 -- 172.21.15.143:0/1646612120 learned_addr learned my addr 172.21.15.143:0/1646612120 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:27:21.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.736+0000 7f37ebfff700 1 -- 172.21.15.143:0/1646612120 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4245760424 0 0) 0x7f37f40cf080 con 0x7f37f40b2210 2023-11-08T19:27:21.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.736+0000 7f37ebfff700 1 -- 172.21.15.143:0/1646612120 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f37e00035b0 con 0x7f37f40b2210 2023-11-08T19:27:21.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.736+0000 7f37f9638700 1 --2- 172.21.15.143:0/1646612120 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f37f415aa30 0x7f37f40cf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:21.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.737+0000 7f37ebfff700 1 -- 172.21.15.143:0/1646612120 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3265059741 0 0) 0x7f37e00035b0 con 0x7f37f40b2210 2023-11-08T19:27:21.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.737+0000 7f37ebfff700 1 -- 172.21.15.143:0/1646612120 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f37f415aa30 msgr2=0x7f37f40cf510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:21.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.737+0000 7f37ebfff700 1 --2- 172.21.15.143:0/1646612120 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f37f415aa30 0x7f37f40cf510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:21.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.737+0000 7f37ebfff700 1 -- 172.21.15.143:0/1646612120 >> v1:172.21.15.174:6789/0 conn(0x7f37f40cc530 legacy=0x7f37f406db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:21.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.737+0000 7f37ebfff700 1 -- 172.21.15.143:0/1646612120 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37f40cea30 con 0x7f37f40b2210 2023-11-08T19:27:21.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.737+0000 7f37ebfff700 1 -- 172.21.15.143:0/1646612120 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4147756303 0 0) 0x7f37f0001670 con 0x7f37f40b2210 2023-11-08T19:27:21.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.737+0000 7f37ebfff700 1 -- 172.21.15.143:0/1646612120 >> v1:172.21.15.143:6789/0 conn(0x7f37f40b2210 legacy=0x7f37f40cbe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:21.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.737+0000 7f37ebfff700 1 --2- 172.21.15.143:0/1646612120 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f37e0003a30 0x7f37e0003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:21.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.737+0000 7f37ebfff700 1 --2- 172.21.15.143:0/1646612120 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f37f415aa30 0x7f37e0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:21.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.737+0000 7f37ebfff700 1 --2- 172.21.15.143:0/1646612120 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f37e0004d90 0x7f37e0005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:21.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.737+0000 7f37ebfff700 1 -- 172.21.15.143:0/1646612120 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37f40ce750 con 0x7f37e0004d90 2023-11-08T19:27:21.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.737+0000 7f37ebfff700 1 -- 172.21.15.143:0/1646612120 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37e000bac0 con 0x7f37e0003a30 2023-11-08T19:27:21.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.737+0000 7f37ebfff700 1 -- 172.21.15.143:0/1646612120 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37e000de90 con 0x7f37f415aa30 2023-11-08T19:27:21.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.737+0000 7f37f9e39700 1 --2- 172.21.15.143:0/1646612120 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f37e0004d90 0x7f37e0005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:21.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.737+0000 7f37f9638700 1 --2- 172.21.15.143:0/1646612120 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f37e0003a30 0x7f37e0003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:21.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.737+0000 7f37f8e37700 1 --2- 172.21.15.143:0/1646612120 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f37f415aa30 0x7f37e0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:21.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.737+0000 7f37f9e39700 1 -- 172.21.15.143:0/1646612120 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f37f415aa30 msgr2=0x7f37e0004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:21.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.737+0000 7f37f9e39700 1 --2- 172.21.15.143:0/1646612120 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f37f415aa30 0x7f37e0004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:21.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.737+0000 7f37f9e39700 1 -- 172.21.15.143:0/1646612120 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f37e0003a30 msgr2=0x7f37e0003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:21.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.737+0000 7f37f9e39700 1 --2- 172.21.15.143:0/1646612120 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f37e0003a30 0x7f37e0003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:21.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.737+0000 7f37f9e39700 1 -- 172.21.15.143:0/1646612120 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37e00035b0 con 0x7f37e0004d90 2023-11-08T19:27:21.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.737+0000 7f37f9e39700 1 --2- 172.21.15.143:0/1646612120 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f37e0004d90 0x7f37e0005210 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f37ec00c160 tx=0x7f37e0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1220cc92862a6e18 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:21.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.737+0000 7f37ebfff700 1 -- 172.21.15.143:0/1646612120 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37ec001db0 con 0x7f37e0004d90 2023-11-08T19:27:21.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.737+0000 7f37ebfff700 1 -- 172.21.15.143:0/1646612120 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37ec00cca0 con 0x7f37e0004d90 2023-11-08T19:27:21.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.737+0000 7f37ebfff700 1 -- 172.21.15.143:0/1646612120 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37ec003c40 con 0x7f37e0004d90 2023-11-08T19:27:21.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.738+0000 7f37fb89c700 1 -- 172.21.15.143:0/1646612120 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f37e0004d90 msgr2=0x7f37e0005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:21.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.738+0000 7f37fb89c700 1 --2- 172.21.15.143:0/1646612120 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f37e0004d90 0x7f37e0005210 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f37ec00c160 tx=0x7f37e0005880 comp rx=0 tx=0).stop 2023-11-08T19:27:21.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.738+0000 7f37f9e39700 1 -- 172.21.15.143:0/1646612120 reap_dead start 2023-11-08T19:27:21.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.738+0000 7f37fb89c700 1 -- 172.21.15.143:0/1646612120 shutdown_connections 2023-11-08T19:27:21.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.738+0000 7f37fb89c700 1 -- 172.21.15.143:0/1646612120 >> 172.21.15.143:0/1646612120 conn(0x7f37f4099ab0 msgr2=0x7f37f40d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:21.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.738+0000 7f37fb89c700 1 -- 172.21.15.143:0/1646612120 shutdown_connections 2023-11-08T19:27:21.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.738+0000 7f37fb89c700 1 -- 172.21.15.143:0/1646612120 wait complete. 2023-11-08T19:27:21.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.739+0000 7f37fb89c700 1 Processor -- start 2023-11-08T19:27:21.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.739+0000 7f37fb89c700 1 -- start start 2023-11-08T19:27:21.752 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.740+0000 7f37fb89c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f37e0004d90 0x7f37f40b5f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:21.752 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.740+0000 7f37fb89c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f37f40c6de0 0x7f37f40b33b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:21.752 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.740+0000 7f37fb89c700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f37f40b38f0 0x7f37f40b3d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:21.752 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.740+0000 7f37fb89c700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37f415af90 con 0x7f37f40c6de0 2023-11-08T19:27:21.753 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.740+0000 7f37fb89c700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37f40aec10 con 0x7f37e0004d90 2023-11-08T19:27:21.753 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.740+0000 7f37fb89c700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37f40aeef0 con 0x7f37f40b38f0 2023-11-08T19:27:21.753 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.740+0000 7f37f8e37700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f37f40c6de0 0x7f37f40b33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:21.753 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.740+0000 7f37f8e37700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f37f40c6de0 0x7f37f40b33b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53294/0 (socket says 172.21.15.143:53294) 2023-11-08T19:27:21.753 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.740+0000 7f37f8e37700 1 -- 172.21.15.143:0/2568456479 learned_addr learned my addr 172.21.15.143:0/2568456479 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:21.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.740+0000 7f37f9e39700 1 --2- 172.21.15.143:0/2568456479 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f37f40b38f0 0x7f37f40b3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:21.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.740+0000 7f37f9638700 1 --2- 172.21.15.143:0/2568456479 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f37e0004d90 0x7f37f40b5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:21.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.740+0000 7f37f8e37700 1 -- 172.21.15.143:0/2568456479 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f37f40b38f0 msgr2=0x7f37f40b3d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:21.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.740+0000 7f37f8e37700 1 --2- 172.21.15.143:0/2568456479 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f37f40b38f0 0x7f37f40b3d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:21.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.740+0000 7f37f8e37700 1 -- 172.21.15.143:0/2568456479 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f37e0004d90 msgr2=0x7f37f40b5f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:21.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.740+0000 7f37f8e37700 1 --2- 172.21.15.143:0/2568456479 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f37e0004d90 0x7f37f40b5f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:21.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.740+0000 7f37f8e37700 1 -- 172.21.15.143:0/2568456479 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37f40c5040 con 0x7f37f40c6de0 2023-11-08T19:27:21.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.740+0000 7f37f8e37700 1 --2- 172.21.15.143:0/2568456479 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f37f40c6de0 0x7f37f40b33b0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f37f00072c0 tx=0x7f37f00075d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:21.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.741+0000 7f37e9ffb700 1 -- 172.21.15.143:0/2568456479 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f00014a0 con 0x7f37f40c6de0 2023-11-08T19:27:21.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.741+0000 7f37e9ffb700 1 -- 172.21.15.143:0/2568456479 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37f000f660 con 0x7f37f40c6de0 2023-11-08T19:27:21.756 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.741+0000 7f37e9ffb700 1 -- 172.21.15.143:0/2568456479 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f000fa10 con 0x7f37f40c6de0 2023-11-08T19:27:21.756 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.741+0000 7f37fb89c700 1 -- 172.21.15.143:0/2568456479 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37f40c5320 con 0x7f37f40c6de0 2023-11-08T19:27:21.756 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.741+0000 7f37fb89c700 1 -- 172.21.15.143:0/2568456479 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37f40c5a20 con 0x7f37f40c6de0 2023-11-08T19:27:21.756 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.742+0000 7f37fb89c700 1 -- 172.21.15.143:0/2568456479 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37f40c4ea0 con 0x7f37f40c6de0 2023-11-08T19:27:21.756 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.742+0000 7f37e9ffb700 1 -- 172.21.15.143:0/2568456479 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f37f000fb70 con 0x7f37f40c6de0 2023-11-08T19:27:21.756 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.742+0000 7f37e9ffb700 1 --2- 172.21.15.143:0/2568456479 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f37d806af00 0x7f37d806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:21.757 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.743+0000 7f37e9ffb700 1 -- 172.21.15.143:0/2568456479 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6006+0+0 (secure 0 0 0) 0x7f37f0089660 con 0x7f37f40c6de0 2023-11-08T19:27:21.757 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.743+0000 7f37f9638700 1 --2- 172.21.15.143:0/2568456479 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f37d806af00 0x7f37d806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:21.757 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.743+0000 7f37f9638700 1 --2- 172.21.15.143:0/2568456479 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f37d806af00 0x7f37d806d3c0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f37f40c4df0 tx=0x7f37e40091c0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:21.757 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:21.747+0000 7f37e9ffb700 1 -- 172.21.15.143:0/2568456479 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f37f40c4ea0 con 0x7f37f40c6de0 2023-11-08T19:27:22.014 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.013+0000 7f37fb89c700 1 -- 172.21.15.143:0/2568456479 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snapshots.TestMonSnapsAndFsPools.test_using_pool_with_snap_fails_fs_creation"]} v 0) v1 -- 0x7f37f40c4ea0 con 0x7f37f40c6de0 2023-11-08T19:27:22.253 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.251+0000 7f37e9ffb700 1 -- 172.21.15.143:0/2568456479 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snapshots.TestMonSnapsAndFsPools.test_using_pool_with_snap_fails_fs_creation"]}]=0 v163) v1 ==== 173+0+0 (secure 0 0 0) 0x7f37f40c4ea0 con 0x7f37f40c6de0 2023-11-08T19:27:22.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.254+0000 7f37fb89c700 1 -- 172.21.15.143:0/2568456479 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f37d806af00 msgr2=0x7f37d806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:22.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.254+0000 7f37fb89c700 1 --2- 172.21.15.143:0/2568456479 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f37d806af00 0x7f37d806d3c0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f37f40c4df0 tx=0x7f37e40091c0 comp rx=0 tx=0).stop 2023-11-08T19:27:22.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.254+0000 7f37fb89c700 1 -- 172.21.15.143:0/2568456479 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f37f40c6de0 msgr2=0x7f37f40b33b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:22.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.254+0000 7f37fb89c700 1 --2- 172.21.15.143:0/2568456479 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f37f40c6de0 0x7f37f40b33b0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f37f00072c0 tx=0x7f37f00075d0 comp rx=0 tx=0).stop 2023-11-08T19:27:22.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.254+0000 7f37fb89c700 1 -- 172.21.15.143:0/2568456479 shutdown_connections 2023-11-08T19:27:22.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.254+0000 7f37fb89c700 1 --2- 172.21.15.143:0/2568456479 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f37d806af00 0x7f37d806d3c0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:22.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.254+0000 7f37fb89c700 1 --2- 172.21.15.143:0/2568456479 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f37f40b38f0 0x7f37f40b3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:22.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.254+0000 7f37fb89c700 1 --2- 172.21.15.143:0/2568456479 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f37e0004d90 0x7f37f40b5f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:22.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.254+0000 7f37fb89c700 1 --2- 172.21.15.143:0/2568456479 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f37f40c6de0 0x7f37f40b33b0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:22.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.254+0000 7f37fb89c700 1 -- 172.21.15.143:0/2568456479 >> 172.21.15.143:0/2568456479 conn(0x7f37f4099ab0 msgr2=0x7f37f40cb310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:22.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.254+0000 7f37fb89c700 1 -- 172.21.15.143:0/2568456479 shutdown_connections 2023-11-08T19:27:22.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.254+0000 7f37fb89c700 1 -- 172.21.15.143:0/2568456479 wait complete. 2023-11-08T19:27:22.268 INFO:tasks.cephfs_test_runner:Test that using a pool with snaps for fs creation fails ... ok 2023-11-08T19:27:22.268 INFO:tasks.cephfs_test_runner:Starting test: test_allow_new_snaps_config (tasks.cephfs.test_snapshots.TestSnapshots) Check whether 'allow_new_snaps' setting works 2023-11-08T19:27:22.269 INFO:tasks.cephfs_test_runner:test_allow_new_snaps_config (tasks.cephfs.test_snapshots.TestSnapshots) 2023-11-08T19:27:22.269 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:27:22.389 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.388+0000 7f2e06ea8700 1 Processor -- start 2023-11-08T19:27:22.389 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.388+0000 7f2e06ea8700 1 -- start start 2023-11-08T19:27:22.392 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.389+0000 7f2e06ea8700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2e0015dd00 0x7f2e000cca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:22.392 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.389+0000 7f2e06ea8700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f2e001618f0 0x7f2e0006ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:22.392 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.389+0000 7f2e06ea8700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f2e0006b150 0x7f2e000b2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:22.392 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.389+0000 7f2e06ea8700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e0006b600 con 0x7f2e0006b150 2023-11-08T19:27:22.393 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.389+0000 7f2e06ea8700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e000cad50 con 0x7f2e001618f0 2023-11-08T19:27:22.393 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.389+0000 7f2e06ea8700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e000ccfb0 con 0x7f2e0015dd00 2023-11-08T19:27:22.393 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.389+0000 7f2e05445700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f2e0006b150 0x7f2e000b2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:22.393 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.389+0000 7f2e05445700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f2e0006b150 0x7f2e000b2210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53304/0 (socket says 172.21.15.143:53304) 2023-11-08T19:27:22.393 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.389+0000 7f2e05445700 1 -- 172.21.15.143:0/3234583366 learned_addr learned my addr 172.21.15.143:0/3234583366 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:22.394 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.389+0000 7f2e04c44700 1 --2- 172.21.15.143:0/3234583366 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2e0015dd00 0x7f2e000cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:22.394 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.389+0000 7f2dfffff700 1 --2- 172.21.15.143:0/3234583366 >> v2:172.21.15.174:3300/0 conn(0x7f2e001618f0 0x7f2e0006ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:22.394 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.389+0000 7f2e05445700 1 -- 172.21.15.143:0/3234583366 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2e0015dd00 msgr2=0x7f2e000cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:22.394 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.389+0000 7f2e05445700 1 --2- 172.21.15.143:0/3234583366 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2e0015dd00 0x7f2e000cca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:22.394 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.389+0000 7f2e05445700 1 -- 172.21.15.143:0/3234583366 >> v2:172.21.15.174:3300/0 conn(0x7f2e001618f0 msgr2=0x7f2e0006ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:22.395 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.389+0000 7f2e05445700 1 --2- 172.21.15.143:0/3234583366 >> v2:172.21.15.174:3300/0 conn(0x7f2e001618f0 0x7f2e0006ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:22.395 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.389+0000 7f2e05445700 1 -- 172.21.15.143:0/3234583366 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e00161df0 con 0x7f2e0006b150 2023-11-08T19:27:22.395 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.389+0000 7f2dfffff700 1 --2- 172.21.15.143:0/3234583366 >> v2:172.21.15.174:3300/0 conn(0x7f2e001618f0 0x7f2e0006ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:27:22.395 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.389+0000 7f2e05445700 1 --2- 172.21.15.143:0/3234583366 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2e0006b150 0x7f2e000b2210 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f2e00071c00 tx=0x7f2df800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=788c438b9f1dee5e server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:22.395 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.389+0000 7f2dff7fe700 1 -- 172.21.15.143:0/3234583366 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2df800e070 con 0x7f2e0006b150 2023-11-08T19:27:22.395 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.389+0000 7f2dff7fe700 1 -- 172.21.15.143:0/3234583366 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2df800b910 con 0x7f2e0006b150 2023-11-08T19:27:22.396 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.389+0000 7f2dff7fe700 1 -- 172.21.15.143:0/3234583366 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2df800bcc0 con 0x7f2e0006b150 2023-11-08T19:27:22.396 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.390+0000 7f2e06ea8700 1 -- 172.21.15.143:0/3234583366 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2e0006b150 msgr2=0x7f2e000b2210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:22.396 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.390+0000 7f2e06ea8700 1 --2- 172.21.15.143:0/3234583366 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2e0006b150 0x7f2e000b2210 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f2e00071c00 tx=0x7f2df800b330 comp rx=0 tx=0).stop 2023-11-08T19:27:22.396 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.390+0000 7f2e06ea8700 1 -- 172.21.15.143:0/3234583366 shutdown_connections 2023-11-08T19:27:22.396 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.390+0000 7f2e06ea8700 1 --2- 172.21.15.143:0/3234583366 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2e0006b150 0x7f2e000b2210 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:22.397 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.390+0000 7f2e06ea8700 1 --2- 172.21.15.143:0/3234583366 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2e0015dd00 0x7f2e000cca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:22.397 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.390+0000 7f2e06ea8700 1 --2- 172.21.15.143:0/3234583366 >> v2:172.21.15.174:3300/0 conn(0x7f2e001618f0 0x7f2e0006ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:22.397 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.390+0000 7f2e06ea8700 1 -- 172.21.15.143:0/3234583366 >> 172.21.15.143:0/3234583366 conn(0x7f2e00099ab0 msgr2=0x7f2e000d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:22.397 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.390+0000 7f2e06ea8700 1 -- 172.21.15.143:0/3234583366 shutdown_connections 2023-11-08T19:27:22.397 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.390+0000 7f2e06ea8700 1 -- 172.21.15.143:0/3234583366 wait complete. 2023-11-08T19:27:22.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.392+0000 7f2e06ea8700 1 Processor -- start 2023-11-08T19:27:22.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.392+0000 7f2e06ea8700 1 -- start start 2023-11-08T19:27:22.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.393+0000 7f2e06ea8700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2e0006b150 0x7f2e00152100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:22.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.393+0000 7f2e06ea8700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2e0015dd00 0x7f2e001f6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:22.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.393+0000 7f2e06ea8700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2e001618f0 0x7f2e000c9c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:22.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.393+0000 7f2e06ea8700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e000c5c80 con 0x7f2e0006b150 2023-11-08T19:27:22.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.393+0000 7f2e06ea8700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e000c5670 con 0x7f2e0015dd00 2023-11-08T19:27:22.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.393+0000 7f2e06ea8700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e001f7de0 con 0x7f2e001618f0 2023-11-08T19:27:22.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.393+0000 7f2e04c44700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2e0006b150 0x7f2e00152100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:22.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.393+0000 7f2e04c44700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2e0006b150 0x7f2e00152100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53306/0 (socket says 172.21.15.143:53306) 2023-11-08T19:27:22.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.393+0000 7f2e04c44700 1 -- 172.21.15.143:0/833830336 learned_addr learned my addr 172.21.15.143:0/833830336 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:22.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.393+0000 7f2dfffff700 1 --2- 172.21.15.143:0/833830336 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2e0015dd00 0x7f2e001f6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:22.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.393+0000 7f2e05445700 1 --2- 172.21.15.143:0/833830336 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2e001618f0 0x7f2e000c9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:22.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.393+0000 7f2e04c44700 1 -- 172.21.15.143:0/833830336 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2e001618f0 msgr2=0x7f2e000c9c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:22.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.393+0000 7f2e04c44700 1 --2- 172.21.15.143:0/833830336 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2e001618f0 0x7f2e000c9c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:22.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.393+0000 7f2e04c44700 1 -- 172.21.15.143:0/833830336 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2e0015dd00 msgr2=0x7f2e001f6eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:22.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.393+0000 7f2e04c44700 1 --2- 172.21.15.143:0/833830336 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2e0015dd00 0x7f2e001f6eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:22.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.393+0000 7f2e04c44700 1 -- 172.21.15.143:0/833830336 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2df8009b30 con 0x7f2e0006b150 2023-11-08T19:27:22.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.393+0000 7f2e04c44700 1 --2- 172.21.15.143:0/833830336 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2e0006b150 0x7f2e00152100 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f2df400e9b0 tx=0x7f2df400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:22.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.393+0000 7f2dfd7fa700 1 -- 172.21.15.143:0/833830336 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2df4009670 con 0x7f2e0006b150 2023-11-08T19:27:22.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.393+0000 7f2dfd7fa700 1 -- 172.21.15.143:0/833830336 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2df4005a20 con 0x7f2e0006b150 2023-11-08T19:27:22.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.393+0000 7f2e06ea8700 1 -- 172.21.15.143:0/833830336 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e00152640 con 0x7f2e0006b150 2023-11-08T19:27:22.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.393+0000 7f2dfd7fa700 1 -- 172.21.15.143:0/833830336 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2df4005db0 con 0x7f2e0006b150 2023-11-08T19:27:22.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.394+0000 7f2e06ea8700 1 -- 172.21.15.143:0/833830336 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e001f7b00 con 0x7f2e0006b150 2023-11-08T19:27:22.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.394+0000 7f2e06ea8700 1 -- 172.21.15.143:0/833830336 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e0015d840 con 0x7f2e0006b150 2023-11-08T19:27:22.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.395+0000 7f2dfd7fa700 1 -- 172.21.15.143:0/833830336 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2df400c850 con 0x7f2e0006b150 2023-11-08T19:27:22.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.395+0000 7f2dfd7fa700 1 --2- 172.21.15.143:0/833830336 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f2de406af00 0x7f2de406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:22.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.395+0000 7f2dfd7fa700 1 -- 172.21.15.143:0/833830336 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6006+0+0 (secure 0 0 0) 0x7f2df4089950 con 0x7f2e0006b150 2023-11-08T19:27:22.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.395+0000 7f2dfffff700 1 --2- 172.21.15.143:0/833830336 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f2de406af00 0x7f2de406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:22.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.396+0000 7f2dfffff700 1 --2- 172.21.15.143:0/833830336 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f2de406af00 0x7f2de406d3c0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f2e00161820 tx=0x7f2dec008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:22.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.400+0000 7f2dfd7fa700 1 -- 172.21.15.143:0/833830336 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2df40548c0 con 0x7f2e0006b150 2023-11-08T19:27:22.594 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.592+0000 7f2e06ea8700 1 -- 172.21.15.143:0/833830336 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2e000aff60 con 0x7f2e0006b150 2023-11-08T19:27:22.595 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.594+0000 7f2dfd7fa700 1 -- 172.21.15.143:0/833830336 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) v1 ==== 74+0+18281 (secure 0 0 0) 0x7f2df4020020 con 0x7f2e0006b150 2023-11-08T19:27:22.595 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:27:22.595 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":70,"fsid":"968483af-77fc-4068-ad7c-5c2a1a4613d7","created":"2023-11-08T19:24:32.826484+0000","modified":"2023-11-08T19:27:21.243025+0000","last_up_change":"2023-11-08T19:24:43.573880+0000","last_in_change":"2023-11-08T19:24:35.314525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:44.551530+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"snap-test-pool","create_time":"2023-11-08T19:26:02.874384+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":36,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"test_data_pool","create_time":"2023-11-08T19:27:05.601949+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":66,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_metadata_pool","create_time":"2023-11-08T19:27:06.536369+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bae98ec1-d94e-403d-a220-a522a53ea6f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6809","nonce":1487830566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6811","nonce":1487830566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6815","nonce":1487830566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6813","nonce":1487830566}]},"public_addr":"172.21.15.143:6809/1487830566","cluster_addr":"172.21.15.143:6811/1487830566","heartbeat_back_addr":"172.21.15.143:6815/1487830566","heartbeat_front_addr":"172.21.15.143:6813/1487830566","state":["exists","up"]},{"osd":1,"uuid":"5cb6538a-1ccb-4322-8633-09f66af0b7e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6801","nonce":787341397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6803","nonce":787341397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6807","nonce":787341397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6805","nonce":787341397}]},"public_addr":"172.21.15.143:6801/787341397","cluster_addr":"172.21.15.143:6803/787341397","heartbeat_back_addr":"172.21.15.143:6807/787341397","heartbeat_front_addr":"172.21.15.143:6805/787341397","state":["exists","up"]},{"osd":2,"uuid":"cdbf522c-60ef-4d60-9653-8dcb529c101a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6822","nonce":2301325597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6826","nonce":2301325597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6831","nonce":2301325597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6829","nonce":2301325597}]},"public_addr":"172.21.15.143:6822/2301325597","cluster_addr":"172.21.15.143:6826/2301325597","heartbeat_back_addr":"172.21.15.143:6831/2301325597","heartbeat_front_addr":"172.21.15.143:6829/2301325597","state":["exists","up"]},{"osd":3,"uuid":"4e2783cf-ebb9-4dd8-8245-bc1d17ce4325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6817","nonce":2021504428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6819","nonce":2021504428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6827","nonce":2021504428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6823","nonce":2021504428}]},"public_addr":"172.21.15.143:6817/2021504428","cluster_addr":"172.21.15.143:6819/2021504428","heartbeat_back_addr":"172.21.15.143:6827/2021504428","heartbeat_front_addr":"172.21.15.143:6823/2021504428","state":["exists","up"]},{"osd":4,"uuid":"691256fa-9b77-49b9-a346-7ae1adf0ca80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6801","nonce":1993380595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6803","nonce":1993380595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6807","nonce":1993380595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6805","nonce":1993380595}]},"public_addr":"172.21.15.174:6801/1993380595","cluster_addr":"172.21.15.174:6803/1993380595","heartbeat_back_addr":"172.21.15.174:6807/1993380595","heartbeat_front_addr":"172.21.15.174:6805/1993380595","state":["exists","up"]},{"osd":5,"uuid":"756fc352-48c9-46ce-b52e-0541fe6f8525","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6809","nonce":1777923303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6811","nonce":1777923303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6815","nonce":1777923303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6813","nonce":1777923303}]},"public_addr":"172.21.15.174:6809/1777923303","cluster_addr":"172.21.15.174:6811/1777923303","heartbeat_back_addr":"172.21.15.174:6815/1777923303","heartbeat_front_addr":"172.21.15.174:6813/1777923303","state":["exists","up"]},{"osd":6,"uuid":"fab2ac2b-ed97-4cde-9825-b480eabf8b8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6825","nonce":2755130316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6827","nonce":2755130316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6831","nonce":2755130316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6829","nonce":2755130316}]},"public_addr":"172.21.15.174:6825/2755130316","cluster_addr":"172.21.15.174:6827/2755130316","heartbeat_back_addr":"172.21.15.174:6831/2755130316","heartbeat_front_addr":"172.21.15.174:6829/2755130316","state":["exists","up"]},{"osd":7,"uuid":"3de9b797-2ab6-4d41-a863-7e7d638fd703","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6817","nonce":2847141411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6819","nonce":2847141411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6823","nonce":2847141411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6821","nonce":2847141411}]},"public_addr":"172.21.15.174:6817/2847141411","cluster_addr":"172.21.15.174:6819/2847141411","heartbeat_back_addr":"172.21.15.174:6823/2847141411","heartbeat_front_addr":"172.21.15.174:6821/2847141411","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.477076+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.496890+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.539305+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.454673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.538881+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.521294+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.520712+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.540404+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.143:6835/2613282947":"2023-11-09T19:27:17.956060+0000","172.21.15.143:6833/2613282947":"2023-11-09T19:27:17.956060+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:27:22.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.596+0000 7f2e06ea8700 1 -- 172.21.15.143:0/833830336 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f2de406af00 msgr2=0x7f2de406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:22.598 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.596+0000 7f2e06ea8700 1 --2- 172.21.15.143:0/833830336 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f2de406af00 0x7f2de406d3c0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f2e00161820 tx=0x7f2dec008040 comp rx=0 tx=0).stop 2023-11-08T19:27:22.598 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.596+0000 7f2e06ea8700 1 -- 172.21.15.143:0/833830336 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2e0006b150 msgr2=0x7f2e00152100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:22.598 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.596+0000 7f2e06ea8700 1 --2- 172.21.15.143:0/833830336 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2e0006b150 0x7f2e00152100 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f2df400e9b0 tx=0x7f2df400ed70 comp rx=0 tx=0).stop 2023-11-08T19:27:22.598 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.596+0000 7f2e06ea8700 1 -- 172.21.15.143:0/833830336 shutdown_connections 2023-11-08T19:27:22.598 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.596+0000 7f2e06ea8700 1 --2- 172.21.15.143:0/833830336 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f2de406af00 0x7f2de406d3c0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:22.599 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.596+0000 7f2e06ea8700 1 --2- 172.21.15.143:0/833830336 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2e001618f0 0x7f2e000c9c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:22.599 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.596+0000 7f2e06ea8700 1 --2- 172.21.15.143:0/833830336 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2e0006b150 0x7f2e00152100 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:22.599 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.596+0000 7f2e06ea8700 1 --2- 172.21.15.143:0/833830336 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2e0015dd00 0x7f2e001f6eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:22.599 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.596+0000 7f2e06ea8700 1 -- 172.21.15.143:0/833830336 >> 172.21.15.143:0/833830336 conn(0x7f2e00099ab0 msgr2=0x7f2e000ad5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:22.601 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.596+0000 7f2e06ea8700 1 -- 172.21.15.143:0/833830336 shutdown_connections 2023-11-08T19:27:22.601 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.596+0000 7f2e06ea8700 1 -- 172.21.15.143:0/833830336 wait complete. 2023-11-08T19:27:22.610 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T19:24:44.551530+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'snap-test-pool', 'create_time': '2023-11-08T19:26:02.874384+0000', 'flags': 16385, 'flags_names': 'hashpspool,pool_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '36', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'pool', 'snap_seq': 2, 'snap_epoch': 36, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'test_data_pool', 'create_time': '2023-11-08T19:27:05.601949+0000', 'flags': 16385, 'flags_names': 'hashpspool,pool_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'pool', 'snap_seq': 2, 'snap_epoch': 66, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'test_metadata_pool', 'create_time': '2023-11-08T19:27:06.536369+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-08T19:27:22.611 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T19:27:22.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.732+0000 7fa83be44700 1 Processor -- start 2023-11-08T19:27:22.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.733+0000 7fa83be44700 1 -- start start 2023-11-08T19:27:22.736 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.733+0000 7fa83be44700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa83406bd20 0x7fa8340d01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:22.736 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.733+0000 7fa83be44700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8340cc2a0 con 0x7fa83415a950 2023-11-08T19:27:22.737 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.733+0000 7fa83be44700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa83406c1b0 con 0x7fa8340b61e0 2023-11-08T19:27:22.737 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.733+0000 7fa83be44700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8340cc130 con 0x7fa83406bd20 2023-11-08T19:27:22.737 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.734+0000 7fa8393df700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa83406bd20 0x7fa8340d01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:22.737 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.734+0000 7fa839be0700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fa83415a950 0x7fa834072370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:49468/0 (socket says 172.21.15.143:49468) 2023-11-08T19:27:22.737 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.734+0000 7fa839be0700 1 -- 172.21.15.143:0/3066413305 learned_addr learned my addr 172.21.15.143:0/3066413305 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:27:22.738 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.734+0000 7fa8393df700 1 -- 172.21.15.143:0/3066413305 >> v1:172.21.15.174:6789/0 conn(0x7fa8340b61e0 legacy=0x7fa8340cc730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:22.738 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.734+0000 7fa8393df700 1 -- 172.21.15.143:0/3066413305 >> v1:172.21.15.143:6789/0 conn(0x7fa83415a950 legacy=0x7fa834072370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:22.738 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.734+0000 7fa8393df700 1 -- 172.21.15.143:0/3066413305 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa83406c490 con 0x7fa83406bd20 2023-11-08T19:27:22.738 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.734+0000 7fa838bde700 1 -- 172.21.15.143:0/3066413305 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2275452400 0 0) 0x7fa8340cc2a0 con 0x7fa83415a950 2023-11-08T19:27:22.738 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.734+0000 7fa8393df700 1 --2- 172.21.15.143:0/3066413305 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa83406bd20 0x7fa8340d01b0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fa83000bae0 tx=0x7fa83000bdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6187df653898ee7b server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:22.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.734+0000 7fa838bde700 1 -- 172.21.15.143:0/3066413305 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa83000dcf0 con 0x7fa83406bd20 2023-11-08T19:27:22.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.734+0000 7fa838bde700 1 -- 172.21.15.143:0/3066413305 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa83000de50 con 0x7fa83406bd20 2023-11-08T19:27:22.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.734+0000 7fa838bde700 1 -- 172.21.15.143:0/3066413305 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa830004250 con 0x7fa83406bd20 2023-11-08T19:27:22.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.735+0000 7fa83be44700 1 -- 172.21.15.143:0/3066413305 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa83406bd20 msgr2=0x7fa8340d01b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:22.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.735+0000 7fa83be44700 1 --2- 172.21.15.143:0/3066413305 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa83406bd20 0x7fa8340d01b0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fa83000bae0 tx=0x7fa83000bdf0 comp rx=0 tx=0).stop 2023-11-08T19:27:22.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.735+0000 7fa83be44700 1 -- 172.21.15.143:0/3066413305 shutdown_connections 2023-11-08T19:27:22.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.735+0000 7fa83be44700 1 --2- 172.21.15.143:0/3066413305 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa83406bd20 0x7fa8340d01b0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:22.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.735+0000 7fa83be44700 1 -- 172.21.15.143:0/3066413305 >> 172.21.15.143:0/3066413305 conn(0x7fa8340cd2e0 msgr2=0x7fa8340cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:22.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.735+0000 7fa83be44700 1 -- 172.21.15.143:0/3066413305 shutdown_connections 2023-11-08T19:27:22.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.735+0000 7fa83be44700 1 -- 172.21.15.143:0/3066413305 wait complete. 2023-11-08T19:27:22.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.736+0000 7fa83be44700 1 Processor -- start 2023-11-08T19:27:22.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.736+0000 7fa83be44700 1 -- start start 2023-11-08T19:27:22.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.737+0000 7fa83be44700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa8340b61e0 0x7fa8341550b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:22.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.737+0000 7fa83be44700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa83415a950 0x7fa8340c8340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:22.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.737+0000 7fa83be44700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa8341ef670 0x7fa83406ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:22.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.737+0000 7fa83be44700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8340c44d0 con 0x7fa8340b61e0 2023-11-08T19:27:22.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.737+0000 7fa83be44700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8340c47b0 con 0x7fa8341ef670 2023-11-08T19:27:22.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.737+0000 7fa83be44700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8340c4a90 con 0x7fa83415a950 2023-11-08T19:27:22.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.737+0000 7fa839be0700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa8340b61e0 0x7fa8341550b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:22.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.737+0000 7fa839be0700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa8340b61e0 0x7fa8341550b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53320/0 (socket says 172.21.15.143:53320) 2023-11-08T19:27:22.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.737+0000 7fa839be0700 1 -- 172.21.15.143:0/39566183 learned_addr learned my addr 172.21.15.143:0/39566183 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:22.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.737+0000 7fa8393df700 1 --2- 172.21.15.143:0/39566183 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa83415a950 0x7fa8340c8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:22.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.737+0000 7fa83a3e1700 1 --2- 172.21.15.143:0/39566183 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa8341ef670 0x7fa83406ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:22.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.737+0000 7fa839be0700 1 -- 172.21.15.143:0/39566183 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa83415a950 msgr2=0x7fa8340c8340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:22.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.737+0000 7fa839be0700 1 --2- 172.21.15.143:0/39566183 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa83415a950 0x7fa8340c8340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:22.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.737+0000 7fa839be0700 1 -- 172.21.15.143:0/39566183 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa8341ef670 msgr2=0x7fa83406ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:22.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.737+0000 7fa839be0700 1 --2- 172.21.15.143:0/39566183 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa8341ef670 0x7fa83406ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:22.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.737+0000 7fa839be0700 1 -- 172.21.15.143:0/39566183 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa83000b740 con 0x7fa8340b61e0 2023-11-08T19:27:22.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.737+0000 7fa8393df700 1 --2- 172.21.15.143:0/39566183 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa83415a950 0x7fa8340c8340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:27:22.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.737+0000 7fa839be0700 1 --2- 172.21.15.143:0/39566183 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa8340b61e0 0x7fa8341550b0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fa82400c550 tx=0x7fa82400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:22.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.737+0000 7fa82a7fc700 1 -- 172.21.15.143:0/39566183 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa824004c80 con 0x7fa8340b61e0 2023-11-08T19:27:22.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.738+0000 7fa82a7fc700 1 -- 172.21.15.143:0/39566183 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa824004e00 con 0x7fa8340b61e0 2023-11-08T19:27:22.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.738+0000 7fa83be44700 1 -- 172.21.15.143:0/39566183 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8340c4d70 con 0x7fa8340b61e0 2023-11-08T19:27:22.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.738+0000 7fa82a7fc700 1 -- 172.21.15.143:0/39566183 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa824007310 con 0x7fa8340b61e0 2023-11-08T19:27:22.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.738+0000 7fa83be44700 1 -- 172.21.15.143:0/39566183 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8341555f0 con 0x7fa8340b61e0 2023-11-08T19:27:22.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.738+0000 7fa83be44700 1 -- 172.21.15.143:0/39566183 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8340c5290 con 0x7fa8340b61e0 2023-11-08T19:27:22.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.739+0000 7fa82a7fc700 1 -- 172.21.15.143:0/39566183 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa824007470 con 0x7fa8340b61e0 2023-11-08T19:27:22.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.739+0000 7fa82a7fc700 1 --2- 172.21.15.143:0/39566183 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa81806af00 0x7fa81806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:22.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.739+0000 7fa82a7fc700 1 -- 172.21.15.143:0/39566183 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6006+0+0 (secure 0 0 0) 0x7fa8240893e0 con 0x7fa8340b61e0 2023-11-08T19:27:22.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.740+0000 7fa8393df700 1 --2- 172.21.15.143:0/39566183 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa81806af00 0x7fa81806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:22.750 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.740+0000 7fa8393df700 1 --2- 172.21.15.143:0/39566183 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa81806af00 0x7fa81806d3c0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fa83000b2c0 tx=0x7fa83001b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:22.750 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.744+0000 7fa82a7fc700 1 -- 172.21.15.143:0/39566183 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa8340c5290 con 0x7fa8340b61e0 2023-11-08T19:27:22.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.930+0000 7fa83be44700 1 -- 172.21.15.143:0/39566183 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa83404eee0 con 0x7fa8340b61e0 2023-11-08T19:27:22.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.930+0000 7fa82a7fc700 1 -- 172.21.15.143:0/39566183 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v70) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa824054360 con 0x7fa8340b61e0 2023-11-08T19:27:22.932 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2023-11-08T19:27:22.934 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.932+0000 7fa83be44700 1 -- 172.21.15.143:0/39566183 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa81806af00 msgr2=0x7fa81806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:22.934 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.932+0000 7fa83be44700 1 --2- 172.21.15.143:0/39566183 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa81806af00 0x7fa81806d3c0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fa83000b2c0 tx=0x7fa83001b040 comp rx=0 tx=0).stop 2023-11-08T19:27:22.935 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.932+0000 7fa83be44700 1 -- 172.21.15.143:0/39566183 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa8340b61e0 msgr2=0x7fa8341550b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:22.935 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.932+0000 7fa83be44700 1 --2- 172.21.15.143:0/39566183 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa8340b61e0 0x7fa8341550b0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fa82400c550 tx=0x7fa82400c910 comp rx=0 tx=0).stop 2023-11-08T19:27:22.935 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.932+0000 7fa83be44700 1 -- 172.21.15.143:0/39566183 shutdown_connections 2023-11-08T19:27:22.935 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.932+0000 7fa83be44700 1 --2- 172.21.15.143:0/39566183 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa81806af00 0x7fa81806d3c0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:22.935 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.932+0000 7fa83be44700 1 --2- 172.21.15.143:0/39566183 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa8341ef670 0x7fa83406ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:22.936 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.932+0000 7fa83be44700 1 --2- 172.21.15.143:0/39566183 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa8340b61e0 0x7fa8341550b0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:22.936 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.932+0000 7fa83be44700 1 --2- 172.21.15.143:0/39566183 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa83415a950 0x7fa8340c8340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:22.936 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.932+0000 7fa83be44700 1 -- 172.21.15.143:0/39566183 >> 172.21.15.143:0/39566183 conn(0x7fa8340cd2e0 msgr2=0x7fa8340d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:22.936 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.933+0000 7fa83be44700 1 -- 172.21.15.143:0/39566183 shutdown_connections 2023-11-08T19:27:22.936 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:22.933+0000 7fa83be44700 1 -- 172.21.15.143:0/39566183 wait complete. 2023-11-08T19:27:22.946 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get snap-test-pool pg_num 2023-11-08T19:27:23.069 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.068+0000 7f64ca785700 1 Processor -- start 2023-11-08T19:27:23.070 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.068+0000 7f64ca785700 1 -- start start 2023-11-08T19:27:23.072 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.069+0000 7f64ca785700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f64c415a950 0x7f64c40d01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:23.072 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.069+0000 7f64ca785700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f64c406bd20 0x7f64c4072370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:23.072 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.069+0000 7f64ca785700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64c40cc2a0 con 0x7f64c40728b0 2023-11-08T19:27:23.072 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.069+0000 7f64ca785700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64c40cc130 con 0x7f64c406bd20 2023-11-08T19:27:23.073 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.069+0000 7f64ca785700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64c406c1b0 con 0x7f64c415a950 2023-11-08T19:27:23.073 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.069+0000 7f64c37fe700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f64c406bd20 0x7f64c4072370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:23.073 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.069+0000 7f64c37fe700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f64c406bd20 0x7f64c4072370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53330/0 (socket says 172.21.15.143:53330) 2023-11-08T19:27:23.073 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.069+0000 7f64c37fe700 1 -- 172.21.15.143:0/4159059167 learned_addr learned my addr 172.21.15.143:0/4159059167 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:23.074 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.069+0000 7f64c3fff700 1 --2- 172.21.15.143:0/4159059167 >> v2:172.21.15.174:3300/0 conn(0x7f64c415a950 0x7f64c40d01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:23.074 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.069+0000 7f64c37fe700 1 -- 172.21.15.143:0/4159059167 >> v2:172.21.15.174:3300/0 conn(0x7f64c415a950 msgr2=0x7f64c40d01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:23.074 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.069+0000 7f64c37fe700 1 --2- 172.21.15.143:0/4159059167 >> v2:172.21.15.174:3300/0 conn(0x7f64c415a950 0x7f64c40d01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:23.074 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.069+0000 7f64c37fe700 1 -- 172.21.15.143:0/4159059167 >> v1:172.21.15.174:6789/0 conn(0x7f64c40728b0 legacy=0x7f64c40cc730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:23.074 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.069+0000 7f64c37fe700 1 -- 172.21.15.143:0/4159059167 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64c406c490 con 0x7f64c406bd20 2023-11-08T19:27:23.075 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.069+0000 7f64c37fe700 1 --2- 172.21.15.143:0/4159059167 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f64c406bd20 0x7f64c4072370 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f64c40cdf90 tx=0x7f64b4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16066db72800f5ca server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:23.075 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.069+0000 7f64c2ffd700 1 -- 172.21.15.143:0/4159059167 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64b4011830 con 0x7f64c406bd20 2023-11-08T19:27:23.075 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.070+0000 7f64c2ffd700 1 -- 172.21.15.143:0/4159059167 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64b4011990 con 0x7f64c406bd20 2023-11-08T19:27:23.075 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.070+0000 7f64c2ffd700 1 -- 172.21.15.143:0/4159059167 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64b4011d20 con 0x7f64c406bd20 2023-11-08T19:27:23.075 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.070+0000 7f64ca785700 1 -- 172.21.15.143:0/4159059167 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f64c406bd20 msgr2=0x7f64c4072370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:23.076 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.070+0000 7f64ca785700 1 --2- 172.21.15.143:0/4159059167 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f64c406bd20 0x7f64c4072370 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f64c40cdf90 tx=0x7f64b4009bd0 comp rx=0 tx=0).stop 2023-11-08T19:27:23.076 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.070+0000 7f64ca785700 1 -- 172.21.15.143:0/4159059167 shutdown_connections 2023-11-08T19:27:23.076 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.070+0000 7f64ca785700 1 --2- 172.21.15.143:0/4159059167 >> v2:172.21.15.174:3300/0 conn(0x7f64c415a950 0x7f64c40d01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:23.076 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.070+0000 7f64ca785700 1 --2- 172.21.15.143:0/4159059167 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f64c406bd20 0x7f64c4072370 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:23.076 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.070+0000 7f64ca785700 1 -- 172.21.15.143:0/4159059167 >> 172.21.15.143:0/4159059167 conn(0x7f64c40cd2e0 msgr2=0x7f64c40cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:23.076 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.070+0000 7f64ca785700 1 -- 172.21.15.143:0/4159059167 shutdown_connections 2023-11-08T19:27:23.077 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.070+0000 7f64ca785700 1 -- 172.21.15.143:0/4159059167 wait complete. 2023-11-08T19:27:23.080 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.071+0000 7f64ca785700 1 Processor -- start 2023-11-08T19:27:23.080 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.072+0000 7f64ca785700 1 -- start start 2023-11-08T19:27:23.080 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.072+0000 7f64ca785700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f64c40728b0 0x7f64c40c6e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:23.080 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.072+0000 7f64ca785700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f64c415a950 0x7f64c414ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:23.081 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.072+0000 7f64ca785700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f64c40a8ef0 0x7f64c40a9380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:23.081 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.072+0000 7f64ca785700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64c40a98c0 con 0x7f64c40a8ef0 2023-11-08T19:27:23.081 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.072+0000 7f64ca785700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64c40c6a00 con 0x7f64c415a950 2023-11-08T19:27:23.081 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.072+0000 7f64ca785700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64c41593d0 con 0x7f64c40728b0 2023-11-08T19:27:23.082 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.072+0000 7f64c8d22700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f64c40a8ef0 0x7f64c40a9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:23.082 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.072+0000 7f64c8d22700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f64c40a8ef0 0x7f64c40a9380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53338/0 (socket says 172.21.15.143:53338) 2023-11-08T19:27:23.082 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.072+0000 7f64c8d22700 1 -- 172.21.15.143:0/3532265283 learned_addr learned my addr 172.21.15.143:0/3532265283 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:23.082 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.073+0000 7f64c37fe700 1 --2- 172.21.15.143:0/3532265283 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f64c415a950 0x7f64c414ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:23.082 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.073+0000 7f64c3fff700 1 --2- 172.21.15.143:0/3532265283 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f64c40728b0 0x7f64c40c6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:23.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.073+0000 7f64c8d22700 1 -- 172.21.15.143:0/3532265283 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f64c40728b0 msgr2=0x7f64c40c6e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:23.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.073+0000 7f64c8d22700 1 --2- 172.21.15.143:0/3532265283 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f64c40728b0 0x7f64c40c6e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:23.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.073+0000 7f64c8d22700 1 -- 172.21.15.143:0/3532265283 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f64c415a950 msgr2=0x7f64c414ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:23.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.073+0000 7f64c8d22700 1 --2- 172.21.15.143:0/3532265283 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f64c415a950 0x7f64c414ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:23.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.073+0000 7f64c8d22700 1 -- 172.21.15.143:0/3532265283 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64b4009550 con 0x7f64c40a8ef0 2023-11-08T19:27:23.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.073+0000 7f64c37fe700 1 --2- 172.21.15.143:0/3532265283 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f64c415a950 0x7f64c414ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:27:23.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.073+0000 7f64c8d22700 1 --2- 172.21.15.143:0/3532265283 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f64c40a8ef0 0x7f64c40a9380 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f64bc00cc50 tx=0x7f64bc00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:23.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.073+0000 7f64c0ff9700 1 -- 172.21.15.143:0/3532265283 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64bc00ad80 con 0x7f64c40a8ef0 2023-11-08T19:27:23.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.073+0000 7f64c0ff9700 1 -- 172.21.15.143:0/3532265283 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64bc00ce50 con 0x7f64c40a8ef0 2023-11-08T19:27:23.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.073+0000 7f64c0ff9700 1 -- 172.21.15.143:0/3532265283 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64bc003c40 con 0x7f64c40a8ef0 2023-11-08T19:27:23.085 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.073+0000 7f64ca785700 1 -- 172.21.15.143:0/3532265283 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64c40c5df0 con 0x7f64c40a8ef0 2023-11-08T19:27:23.085 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.073+0000 7f64ca785700 1 -- 172.21.15.143:0/3532265283 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64c40c63b0 con 0x7f64c40a8ef0 2023-11-08T19:27:23.085 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.074+0000 7f64ca785700 1 -- 172.21.15.143:0/3532265283 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64c404eee0 con 0x7f64c40a8ef0 2023-11-08T19:27:23.085 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.074+0000 7f64c0ff9700 1 -- 172.21.15.143:0/3532265283 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f64bc0050d0 con 0x7f64c40a8ef0 2023-11-08T19:27:23.085 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.075+0000 7f64c0ff9700 1 --2- 172.21.15.143:0/3532265283 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f64b006af00 0x7f64b006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:23.086 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.075+0000 7f64c0ff9700 1 -- 172.21.15.143:0/3532265283 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6006+0+0 (secure 0 0 0) 0x7f64bc0898c0 con 0x7f64c40a8ef0 2023-11-08T19:27:23.086 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.075+0000 7f64c3fff700 1 --2- 172.21.15.143:0/3532265283 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f64b006af00 0x7f64b006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:23.086 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.075+0000 7f64c3fff700 1 --2- 172.21.15.143:0/3532265283 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f64b006af00 0x7f64b006d3c0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f64b800bb70 tx=0x7f64b8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:23.087 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.079+0000 7f64c0ff9700 1 -- 172.21.15.143:0/3532265283 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f64bc0547a0 con 0x7f64c40a8ef0 2023-11-08T19:27:23.274 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.272+0000 7f64ca785700 1 -- 172.21.15.143:0/3532265283 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"} v 0) v1 -- 0x7f64c4072150 con 0x7f64c40a8ef0 2023-11-08T19:27:23.274 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.273+0000 7f64c0ff9700 1 -- 172.21.15.143:0/3532265283 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"}]=0 v70) v1 ==== 103+0+11 (secure 0 0 0) 0x7f64bc057dc0 con 0x7f64c40a8ef0 2023-11-08T19:27:23.275 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2023-11-08T19:27:23.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.275+0000 7f64ca785700 1 -- 172.21.15.143:0/3532265283 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f64b006af00 msgr2=0x7f64b006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:23.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.275+0000 7f64ca785700 1 --2- 172.21.15.143:0/3532265283 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f64b006af00 0x7f64b006d3c0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f64b800bb70 tx=0x7f64b8008040 comp rx=0 tx=0).stop 2023-11-08T19:27:23.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.275+0000 7f64ca785700 1 -- 172.21.15.143:0/3532265283 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f64c40a8ef0 msgr2=0x7f64c40a9380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:23.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.275+0000 7f64ca785700 1 --2- 172.21.15.143:0/3532265283 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f64c40a8ef0 0x7f64c40a9380 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f64bc00cc50 tx=0x7f64bc00a340 comp rx=0 tx=0).stop 2023-11-08T19:27:23.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.275+0000 7f64ca785700 1 -- 172.21.15.143:0/3532265283 shutdown_connections 2023-11-08T19:27:23.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.275+0000 7f64ca785700 1 --2- 172.21.15.143:0/3532265283 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f64b006af00 0x7f64b006d3c0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:23.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.275+0000 7f64ca785700 1 --2- 172.21.15.143:0/3532265283 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f64c40728b0 0x7f64c40c6e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:23.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.275+0000 7f64ca785700 1 --2- 172.21.15.143:0/3532265283 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f64c40a8ef0 0x7f64c40a9380 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:23.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.275+0000 7f64ca785700 1 --2- 172.21.15.143:0/3532265283 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f64c415a950 0x7f64c414ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:23.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.275+0000 7f64ca785700 1 -- 172.21.15.143:0/3532265283 >> 172.21.15.143:0/3532265283 conn(0x7f64c40cd2e0 msgr2=0x7f64c40cc630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:23.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.275+0000 7f64ca785700 1 -- 172.21.15.143:0/3532265283 shutdown_connections 2023-11-08T19:27:23.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.275+0000 7f64ca785700 1 -- 172.21.15.143:0/3532265283 wait complete. 2023-11-08T19:27:23.288 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get test_data_pool pg_num 2023-11-08T19:27:23.410 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.409+0000 7f8fe7a83700 1 Processor -- start 2023-11-08T19:27:23.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.409+0000 7f8fe7a83700 1 -- start start 2023-11-08T19:27:23.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.410+0000 7f8fe7a83700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f8fe015dba0 0x7f8fe006c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:23.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.410+0000 7f8fe7a83700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8fe006b240 0x7f8fe00caff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:23.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.410+0000 7f8fe7a83700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fe0161890 con 0x7f8fe00cb530 2023-11-08T19:27:23.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.410+0000 7f8fe7a83700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fe006b740 con 0x7f8fe015dba0 2023-11-08T19:27:23.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.410+0000 7f8fe7a83700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fe006ba20 con 0x7f8fe006b240 2023-11-08T19:27:23.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.410+0000 7f8fe581f700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f8fe015dba0 0x7f8fe006c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:23.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.410+0000 7f8fe581f700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f8fe015dba0 0x7f8fe006c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53342/0 (socket says 172.21.15.143:53342) 2023-11-08T19:27:23.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.410+0000 7f8fe581f700 1 -- 172.21.15.143:0/1198924009 learned_addr learned my addr 172.21.15.143:0/1198924009 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:23.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.410+0000 7f8fe501e700 1 --2- 172.21.15.143:0/1198924009 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8fe006b240 0x7f8fe00caff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:23.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.410+0000 7f8fe581f700 1 -- 172.21.15.143:0/1198924009 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8fe006b240 msgr2=0x7f8fe00caff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:23.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.410+0000 7f8fe581f700 1 --2- 172.21.15.143:0/1198924009 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8fe006b240 0x7f8fe00caff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:23.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.410+0000 7f8fe581f700 1 -- 172.21.15.143:0/1198924009 >> v1:172.21.15.174:6789/0 conn(0x7f8fe00cb530 legacy=0x7f8fe00b22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:23.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.410+0000 7f8fe581f700 1 -- 172.21.15.143:0/1198924009 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fe006bd00 con 0x7f8fe015dba0 2023-11-08T19:27:23.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.411+0000 7f8fe581f700 1 --2- 172.21.15.143:0/1198924009 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8fe015dba0 0x7f8fe006c610 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f8fd00098f0 tx=0x7f8fd0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3a3fa07d65c2a5d server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:23.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.411+0000 7f8fe481d700 1 -- 172.21.15.143:0/1198924009 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fd0004230 con 0x7f8fe015dba0 2023-11-08T19:27:23.417 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.411+0000 7f8fe481d700 1 -- 172.21.15.143:0/1198924009 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fd0004390 con 0x7f8fe015dba0 2023-11-08T19:27:23.417 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.411+0000 7f8fe481d700 1 -- 172.21.15.143:0/1198924009 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fd00036a0 con 0x7f8fe015dba0 2023-11-08T19:27:23.417 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.411+0000 7f8fe7a83700 1 -- 172.21.15.143:0/1198924009 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8fe015dba0 msgr2=0x7f8fe006c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:23.417 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.411+0000 7f8fe7a83700 1 --2- 172.21.15.143:0/1198924009 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8fe015dba0 0x7f8fe006c610 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f8fd00098f0 tx=0x7f8fd0009c00 comp rx=0 tx=0).stop 2023-11-08T19:27:23.417 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.411+0000 7f8fe7a83700 1 -- 172.21.15.143:0/1198924009 shutdown_connections 2023-11-08T19:27:23.418 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.411+0000 7f8fe7a83700 1 --2- 172.21.15.143:0/1198924009 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8fe015dba0 0x7f8fe006c610 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:23.418 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.411+0000 7f8fe7a83700 1 --2- 172.21.15.143:0/1198924009 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8fe006b240 0x7f8fe00caff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:23.418 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.411+0000 7f8fe7a83700 1 -- 172.21.15.143:0/1198924009 >> 172.21.15.143:0/1198924009 conn(0x7f8fe00cd2e0 msgr2=0x7f8fe00cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:23.418 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.411+0000 7f8fe7a83700 1 -- 172.21.15.143:0/1198924009 shutdown_connections 2023-11-08T19:27:23.419 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.411+0000 7f8fe7a83700 1 -- 172.21.15.143:0/1198924009 wait complete. 2023-11-08T19:27:23.424 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.413+0000 7f8fe7a83700 1 Processor -- start 2023-11-08T19:27:23.424 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.413+0000 7f8fe7a83700 1 -- start start 2023-11-08T19:27:23.424 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.413+0000 7f8fe7a83700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8fe006b240 0x7f8fe014ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:23.424 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.413+0000 7f8fe7a83700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8fe00cb530 0x7f8fe00af340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:23.425 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.413+0000 7f8fe7a83700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8fe015dba0 0x7f8fe0072030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:23.425 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.413+0000 7f8fe7a83700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fe00b5a90 con 0x7f8fe015dba0 2023-11-08T19:27:23.425 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.413+0000 7f8fe7a83700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fe00b5d30 con 0x7f8fe006b240 2023-11-08T19:27:23.425 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.413+0000 7f8fe7a83700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fe00b6010 con 0x7f8fe00cb530 2023-11-08T19:27:23.425 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.413+0000 7f8fe6020700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8fe015dba0 0x7f8fe0072030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:23.426 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.413+0000 7f8fe6020700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8fe015dba0 0x7f8fe0072030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53352/0 (socket says 172.21.15.143:53352) 2023-11-08T19:27:23.426 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.413+0000 7f8fe6020700 1 -- 172.21.15.143:0/1661564114 learned_addr learned my addr 172.21.15.143:0/1661564114 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:23.426 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.413+0000 7f8fe501e700 1 --2- 172.21.15.143:0/1661564114 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8fe00cb530 0x7f8fe00af340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:23.426 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.413+0000 7f8fe581f700 1 --2- 172.21.15.143:0/1661564114 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8fe006b240 0x7f8fe014ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:23.426 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.414+0000 7f8fe6020700 1 -- 172.21.15.143:0/1661564114 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8fe00cb530 msgr2=0x7f8fe00af340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:23.427 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.414+0000 7f8fe6020700 1 --2- 172.21.15.143:0/1661564114 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8fe00cb530 0x7f8fe00af340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:23.427 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.414+0000 7f8fe6020700 1 -- 172.21.15.143:0/1661564114 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8fe006b240 msgr2=0x7f8fe014ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:23.427 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.414+0000 7f8fe6020700 1 --2- 172.21.15.143:0/1661564114 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8fe006b240 0x7f8fe014ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:23.427 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.414+0000 7f8fe6020700 1 -- 172.21.15.143:0/1661564114 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fd0009550 con 0x7f8fe015dba0 2023-11-08T19:27:23.427 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.414+0000 7f8fe6020700 1 --2- 172.21.15.143:0/1661564114 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8fe015dba0 0x7f8fe0072030 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f8fd800c780 tx=0x7f8fd800ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:23.428 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.414+0000 7f8fd67fc700 1 -- 172.21.15.143:0/1661564114 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fd8003e10 con 0x7f8fe015dba0 2023-11-08T19:27:23.428 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.414+0000 7f8fd67fc700 1 -- 172.21.15.143:0/1661564114 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fd8005070 con 0x7f8fe015dba0 2023-11-08T19:27:23.428 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.414+0000 7f8fe7a83700 1 -- 172.21.15.143:0/1661564114 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fe00b62f0 con 0x7f8fe015dba0 2023-11-08T19:27:23.428 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.414+0000 7f8fd67fc700 1 -- 172.21.15.143:0/1661564114 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fd8005420 con 0x7f8fe015dba0 2023-11-08T19:27:23.429 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.414+0000 7f8fe7a83700 1 -- 172.21.15.143:0/1661564114 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fe00af880 con 0x7f8fe015dba0 2023-11-08T19:27:23.429 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.415+0000 7f8fe7a83700 1 -- 172.21.15.143:0/1661564114 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fe00ace30 con 0x7f8fe015dba0 2023-11-08T19:27:23.429 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.415+0000 7f8fd67fc700 1 -- 172.21.15.143:0/1661564114 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8fd8005580 con 0x7f8fe015dba0 2023-11-08T19:27:23.429 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.416+0000 7f8fd67fc700 1 --2- 172.21.15.143:0/1661564114 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f8fc406af00 0x7f8fc406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:23.429 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.416+0000 7f8fd67fc700 1 -- 172.21.15.143:0/1661564114 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6006+0+0 (secure 0 0 0) 0x7f8fd80887d0 con 0x7f8fe015dba0 2023-11-08T19:27:23.430 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.416+0000 7f8fe581f700 1 --2- 172.21.15.143:0/1661564114 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f8fc406af00 0x7f8fc406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:23.430 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.416+0000 7f8fe581f700 1 --2- 172.21.15.143:0/1661564114 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f8fc406af00 0x7f8fc406d3c0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f8fd0009130 tx=0x7f8fd0008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:23.430 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.420+0000 7f8fd67fc700 1 -- 172.21.15.143:0/1661564114 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8fe00ace30 con 0x7f8fe015dba0 2023-11-08T19:27:23.609 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.607+0000 7f8fe7a83700 1 -- 172.21.15.143:0/1661564114 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "test_data_pool", "var": "pg_num"} v 0) v1 -- 0x7f8fe00ace30 con 0x7f8fe015dba0 2023-11-08T19:27:23.609 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.607+0000 7f8fd67fc700 1 -- 172.21.15.143:0/1661564114 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "test_data_pool", "var": "pg_num"}]=0 v70) v1 ==== 103+0+11 (secure 0 0 0) 0x7f8fe00ace30 con 0x7f8fe015dba0 2023-11-08T19:27:23.610 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2023-11-08T19:27:23.611 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.609+0000 7f8fe7a83700 1 -- 172.21.15.143:0/1661564114 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f8fc406af00 msgr2=0x7f8fc406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:23.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.609+0000 7f8fe7a83700 1 --2- 172.21.15.143:0/1661564114 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f8fc406af00 0x7f8fc406d3c0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f8fd0009130 tx=0x7f8fd0008030 comp rx=0 tx=0).stop 2023-11-08T19:27:23.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.609+0000 7f8fe7a83700 1 -- 172.21.15.143:0/1661564114 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8fe015dba0 msgr2=0x7f8fe0072030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:23.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.609+0000 7f8fe7a83700 1 --2- 172.21.15.143:0/1661564114 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8fe015dba0 0x7f8fe0072030 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f8fd800c780 tx=0x7f8fd800ca90 comp rx=0 tx=0).stop 2023-11-08T19:27:23.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.609+0000 7f8fe7a83700 1 -- 172.21.15.143:0/1661564114 shutdown_connections 2023-11-08T19:27:23.613 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.609+0000 7f8fe7a83700 1 --2- 172.21.15.143:0/1661564114 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f8fc406af00 0x7f8fc406d3c0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:23.613 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.609+0000 7f8fe7a83700 1 --2- 172.21.15.143:0/1661564114 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8fe015dba0 0x7f8fe0072030 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:23.613 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.609+0000 7f8fe7a83700 1 --2- 172.21.15.143:0/1661564114 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8fe006b240 0x7f8fe014ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:23.613 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.609+0000 7f8fe7a83700 1 --2- 172.21.15.143:0/1661564114 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8fe00cb530 0x7f8fe00af340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:23.614 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.609+0000 7f8fe7a83700 1 -- 172.21.15.143:0/1661564114 >> 172.21.15.143:0/1661564114 conn(0x7f8fe00cd2e0 msgr2=0x7f8fe015cc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:23.614 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.610+0000 7f8fe7a83700 1 -- 172.21.15.143:0/1661564114 shutdown_connections 2023-11-08T19:27:23.614 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.610+0000 7f8fe7a83700 1 -- 172.21.15.143:0/1661564114 wait complete. 2023-11-08T19:27:23.623 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get test_metadata_pool pg_num 2023-11-08T19:27:23.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.746+0000 7f9b743ac700 1 Processor -- start 2023-11-08T19:27:23.937 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.746+0000 7f9b743ac700 1 -- start start 2023-11-08T19:27:23.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.747+0000 7f9b743ac700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9b6c15bb20 0x7f9b6c0b2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:23.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.747+0000 7f9b743ac700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b6c0d0420 con 0x7f9b6c0d1d80 2023-11-08T19:27:23.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.747+0000 7f9b743ac700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b6c0cf810 con 0x7f9b6c0b1730 2023-11-08T19:27:23.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.747+0000 7f9b743ac700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b6c0cf600 con 0x7f9b6c15bb20 2023-11-08T19:27:23.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.747+0000 7f9b72949700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f9b6c0d1d80 0x7f9b6c0cc4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:49514/0 (socket says 172.21.15.143:49514) 2023-11-08T19:27:23.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.747+0000 7f9b72949700 1 -- 172.21.15.143:0/23633281 learned_addr learned my addr 172.21.15.143:0/23633281 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:27:23.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.747+0000 7f9b72148700 1 --2- 172.21.15.143:0/23633281 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9b6c15bb20 0x7f9b6c0b2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:23.944 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.747+0000 7f9b71146700 1 -- 172.21.15.143:0/23633281 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3254722162 0 0) 0x7f9b6c0d0420 con 0x7f9b6c0d1d80 2023-11-08T19:27:23.944 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.747+0000 7f9b71146700 1 -- 172.21.15.143:0/23633281 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9b580035b0 con 0x7f9b6c0d1d80 2023-11-08T19:27:23.944 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.747+0000 7f9b71146700 1 -- 172.21.15.143:0/23633281 <== mon.1 v1:172.21.15.174:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1391964173 0 0) 0x7f9b6c0cf810 con 0x7f9b6c0b1730 2023-11-08T19:27:23.944 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.747+0000 7f9b71146700 1 -- 172.21.15.143:0/23633281 --> v1:172.21.15.174:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9b6c0d0420 con 0x7f9b6c0b1730 2023-11-08T19:27:23.944 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.747+0000 7f9b72148700 1 -- 172.21.15.143:0/23633281 >> v1:172.21.15.174:6789/0 conn(0x7f9b6c0b1730 legacy=0x7f9b6c0d1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:23.945 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.747+0000 7f9b72148700 1 -- 172.21.15.143:0/23633281 >> v1:172.21.15.143:6789/0 conn(0x7f9b6c0d1d80 legacy=0x7f9b6c0cc4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:23.945 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.747+0000 7f9b71146700 1 -- 172.21.15.143:0/23633281 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 599429073 0 0) 0x7f9b580035b0 con 0x7f9b6c0d1d80 2023-11-08T19:27:23.945 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.747+0000 7f9b72148700 1 -- 172.21.15.143:0/23633281 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b6c0cfaf0 con 0x7f9b6c15bb20 2023-11-08T19:27:23.945 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.748+0000 7f9b72148700 1 --2- 172.21.15.143:0/23633281 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9b6c15bb20 0x7f9b6c0b2670 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9b5c009870 tx=0x7f9b5c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bcb8f02ab05e3c34 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:23.945 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.748+0000 7f9b71146700 1 -- 172.21.15.143:0/23633281 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b640019f0 con 0x7f9b6c15bb20 2023-11-08T19:27:23.946 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.748+0000 7f9b71146700 1 -- 172.21.15.143:0/23633281 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b5c004030 con 0x7f9b6c15bb20 2023-11-08T19:27:23.946 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.748+0000 7f9b71146700 1 -- 172.21.15.143:0/23633281 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b5c0043a0 con 0x7f9b6c15bb20 2023-11-08T19:27:23.946 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.748+0000 7f9b743ac700 1 -- 172.21.15.143:0/23633281 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9b6c15bb20 msgr2=0x7f9b6c0b2670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:23.946 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.748+0000 7f9b743ac700 1 --2- 172.21.15.143:0/23633281 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9b6c15bb20 0x7f9b6c0b2670 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9b5c009870 tx=0x7f9b5c009b80 comp rx=0 tx=0).stop 2023-11-08T19:27:23.947 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.748+0000 7f9b743ac700 1 -- 172.21.15.143:0/23633281 shutdown_connections 2023-11-08T19:27:23.947 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.748+0000 7f9b743ac700 1 --2- 172.21.15.143:0/23633281 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9b6c15bb20 0x7f9b6c0b2670 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:23.947 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.748+0000 7f9b743ac700 1 -- 172.21.15.143:0/23633281 >> 172.21.15.143:0/23633281 conn(0x7f9b6c08b000 msgr2=0x7f9b6c09ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:23.947 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.748+0000 7f9b743ac700 1 -- 172.21.15.143:0/23633281 shutdown_connections 2023-11-08T19:27:23.947 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.748+0000 7f9b743ac700 1 -- 172.21.15.143:0/23633281 wait complete. 2023-11-08T19:27:23.948 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.750+0000 7f9b743ac700 1 Processor -- start 2023-11-08T19:27:23.948 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.750+0000 7f9b743ac700 1 -- start start 2023-11-08T19:27:23.948 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.750+0000 7f9b743ac700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9b6c0b1730 0x7f9b6c154440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:23.948 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.750+0000 7f9b743ac700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9b6c0d1d80 0x7f9b6c1e3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:23.948 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.750+0000 7f9b743ac700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9b6c15bb20 0x7f9b6c1e59d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:23.949 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.750+0000 7f9b743ac700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b6c0cd260 con 0x7f9b6c0d1d80 2023-11-08T19:27:23.949 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.750+0000 7f9b743ac700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b6c0b6560 con 0x7f9b6c15bb20 2023-11-08T19:27:23.949 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.750+0000 7f9b743ac700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b6c0b6840 con 0x7f9b6c0b1730 2023-11-08T19:27:23.949 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.750+0000 7f9b71947700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9b6c0d1d80 0x7f9b6c1e3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:23.949 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.750+0000 7f9b71947700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9b6c0d1d80 0x7f9b6c1e3cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53364/0 (socket says 172.21.15.143:53364) 2023-11-08T19:27:23.950 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.750+0000 7f9b71947700 1 -- 172.21.15.143:0/2992132172 learned_addr learned my addr 172.21.15.143:0/2992132172 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:23.950 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.751+0000 7f9b72949700 1 --2- 172.21.15.143:0/2992132172 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9b6c15bb20 0x7f9b6c1e59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:23.950 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.751+0000 7f9b72148700 1 --2- 172.21.15.143:0/2992132172 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9b6c0b1730 0x7f9b6c154440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:23.950 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.751+0000 7f9b71947700 1 -- 172.21.15.143:0/2992132172 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9b6c0b1730 msgr2=0x7f9b6c154440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:23.950 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.751+0000 7f9b71947700 1 --2- 172.21.15.143:0/2992132172 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9b6c0b1730 0x7f9b6c154440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:23.951 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.751+0000 7f9b71947700 1 -- 172.21.15.143:0/2992132172 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9b6c15bb20 msgr2=0x7f9b6c1e59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:23.951 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.751+0000 7f9b71947700 1 --2- 172.21.15.143:0/2992132172 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9b6c15bb20 0x7f9b6c1e59d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:23.951 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.751+0000 7f9b71947700 1 -- 172.21.15.143:0/2992132172 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b5c0094a0 con 0x7f9b6c0d1d80 2023-11-08T19:27:23.951 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.751+0000 7f9b71947700 1 --2- 172.21.15.143:0/2992132172 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9b6c0d1d80 0x7f9b6c1e3cf0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f9b6800c520 tx=0x7f9b6800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:23.951 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.751+0000 7f9b62ffd700 1 -- 172.21.15.143:0/2992132172 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b680052d0 con 0x7f9b6c0d1d80 2023-11-08T19:27:23.952 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.751+0000 7f9b62ffd700 1 -- 172.21.15.143:0/2992132172 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b68005430 con 0x7f9b6c0d1d80 2023-11-08T19:27:23.952 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.751+0000 7f9b743ac700 1 -- 172.21.15.143:0/2992132172 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b6c0b6b20 con 0x7f9b6c0d1d80 2023-11-08T19:27:23.952 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.751+0000 7f9b62ffd700 1 -- 172.21.15.143:0/2992132172 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b680038c0 con 0x7f9b6c0d1d80 2023-11-08T19:27:23.952 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.751+0000 7f9b743ac700 1 -- 172.21.15.143:0/2992132172 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b6c154980 con 0x7f9b6c0d1d80 2023-11-08T19:27:23.952 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.752+0000 7f9b743ac700 1 -- 172.21.15.143:0/2992132172 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b6c154b40 con 0x7f9b6c0d1d80 2023-11-08T19:27:23.952 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.752+0000 7f9b62ffd700 1 -- 172.21.15.143:0/2992132172 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9b68003a20 con 0x7f9b6c0d1d80 2023-11-08T19:27:23.953 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.753+0000 7f9b62ffd700 1 --2- 172.21.15.143:0/2992132172 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f9b5006af00 0x7f9b5006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:23.953 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.753+0000 7f9b62ffd700 1 -- 172.21.15.143:0/2992132172 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6006+0+0 (secure 0 0 0) 0x7f9b68088be0 con 0x7f9b6c0d1d80 2023-11-08T19:27:23.953 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.753+0000 7f9b72148700 1 --2- 172.21.15.143:0/2992132172 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f9b5006af00 0x7f9b5006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:23.953 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.753+0000 7f9b72148700 1 --2- 172.21.15.143:0/2992132172 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f9b5006af00 0x7f9b5006d3c0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f9b5c0090b0 tx=0x7f9b5c008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:23.953 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.757+0000 7f9b62ffd700 1 -- 172.21.15.143:0/2992132172 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9b68053ad0 con 0x7f9b6c0d1d80 2023-11-08T19:27:23.954 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.943+0000 7f9b743ac700 1 -- 172.21.15.143:0/2992132172 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "test_metadata_pool", "var": "pg_num"} v 0) v1 -- 0x7f9b6c0cd3a0 con 0x7f9b6c0d1d80 2023-11-08T19:27:23.954 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.944+0000 7f9b62ffd700 1 -- 172.21.15.143:0/2992132172 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "test_metadata_pool", "var": "pg_num"}]=0 v70) v1 ==== 107+0+11 (secure 0 0 0) 0x7f9b680570f0 con 0x7f9b6c0d1d80 2023-11-08T19:27:23.955 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2023-11-08T19:27:23.956 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.946+0000 7f9b743ac700 1 -- 172.21.15.143:0/2992132172 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f9b5006af00 msgr2=0x7f9b5006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:23.956 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.946+0000 7f9b743ac700 1 --2- 172.21.15.143:0/2992132172 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f9b5006af00 0x7f9b5006d3c0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f9b5c0090b0 tx=0x7f9b5c008030 comp rx=0 tx=0).stop 2023-11-08T19:27:23.956 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.946+0000 7f9b743ac700 1 -- 172.21.15.143:0/2992132172 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9b6c0d1d80 msgr2=0x7f9b6c1e3cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:23.956 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.946+0000 7f9b743ac700 1 --2- 172.21.15.143:0/2992132172 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9b6c0d1d80 0x7f9b6c1e3cf0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f9b6800c520 tx=0x7f9b6800c830 comp rx=0 tx=0).stop 2023-11-08T19:27:23.956 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.946+0000 7f9b743ac700 1 -- 172.21.15.143:0/2992132172 shutdown_connections 2023-11-08T19:27:23.957 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.946+0000 7f9b743ac700 1 --2- 172.21.15.143:0/2992132172 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f9b5006af00 0x7f9b5006d3c0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:23.957 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.946+0000 7f9b743ac700 1 --2- 172.21.15.143:0/2992132172 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9b6c0b1730 0x7f9b6c154440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:23.957 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.946+0000 7f9b743ac700 1 --2- 172.21.15.143:0/2992132172 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9b6c15bb20 0x7f9b6c1e59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:23.957 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.946+0000 7f9b743ac700 1 --2- 172.21.15.143:0/2992132172 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9b6c0d1d80 0x7f9b6c1e3cf0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:23.957 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.946+0000 7f9b743ac700 1 -- 172.21.15.143:0/2992132172 >> 172.21.15.143:0/2992132172 conn(0x7f9b6c08b000 msgr2=0x7f9b6c0b2c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:23.958 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.946+0000 7f9b743ac700 1 -- 172.21.15.143:0/2992132172 shutdown_connections 2023-11-08T19:27:23.958 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:23.946+0000 7f9b743ac700 1 -- 172.21.15.143:0/2992132172 wait complete. 2023-11-08T19:27:23.959 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snapshots.TestSnapshots.test_allow_new_snaps_config' 2023-11-08T19:27:24.080 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:24.079+0000 7f8c40e89700 1 Processor -- start 2023-11-08T19:27:24.081 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:24.079+0000 7f8c40e89700 1 -- start start 2023-11-08T19:27:24.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:24.080+0000 7f8c40e89700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8c3c15ba30 0x7f8c3c0b14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:24.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:24.080+0000 7f8c40e89700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f8c3c0d03a0 0x7f8c3c0d1560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:24.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:24.080+0000 7f8c40e89700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c3c0d1f10 con 0x7f8c3c0d1aa0 2023-11-08T19:27:24.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:24.080+0000 7f8c40e89700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c3c0b1bc0 con 0x7f8c3c0d03a0 2023-11-08T19:27:24.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:24.080+0000 7f8c40e89700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c3c0b1ea0 con 0x7f8c3c15ba30 2023-11-08T19:27:24.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:24.080+0000 7f8c39d9b700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f8c3c0d03a0 0x7f8c3c0d1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:24.085 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:24.080+0000 7f8c39d9b700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f8c3c0d03a0 0x7f8c3c0d1560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53370/0 (socket says 172.21.15.143:53370) 2023-11-08T19:27:24.085 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:24.080+0000 7f8c39d9b700 1 -- 172.21.15.143:0/1713314480 learned_addr learned my addr 172.21.15.143:0/1713314480 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:24.085 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:24.080+0000 7f8c3a59c700 1 --2- 172.21.15.143:0/1713314480 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8c3c15ba30 0x7f8c3c0b14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:24.085 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:24.080+0000 7f8c39d9b700 1 -- 172.21.15.143:0/1713314480 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8c3c15ba30 msgr2=0x7f8c3c0b14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:24.085 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:24.080+0000 7f8c39d9b700 1 --2- 172.21.15.143:0/1713314480 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8c3c15ba30 0x7f8c3c0b14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:24.086 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:24.080+0000 7f8c39d9b700 1 -- 172.21.15.143:0/1713314480 >> v1:172.21.15.174:6789/0 conn(0x7f8c3c0d1aa0 legacy=0x7f8c3c0ce2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:24.086 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:24.080+0000 7f8c39d9b700 1 -- 172.21.15.143:0/1713314480 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c3c0b2180 con 0x7f8c3c0d03a0 2023-11-08T19:27:24.086 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:24.080+0000 7f8c39d9b700 1 --2- 172.21.15.143:0/1713314480 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8c3c0d03a0 0x7f8c3c0d1560 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f8c300098f0 tx=0x7f8c30009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dafb35afd92d000 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:24.086 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:24.080+0000 7f8c3959a700 1 -- 172.21.15.143:0/1713314480 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c30011830 con 0x7f8c3c0d03a0 2023-11-08T19:27:24.087 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:24.081+0000 7f8c3959a700 1 -- 172.21.15.143:0/1713314480 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c30011990 con 0x7f8c3c0d03a0 2023-11-08T19:27:24.087 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:24.081+0000 7f8c3959a700 1 -- 172.21.15.143:0/1713314480 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c30011d20 con 0x7f8c3c0d03a0 2023-11-08T19:27:24.087 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:24.081+0000 7f8c40e89700 1 -- 172.21.15.143:0/1713314480 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8c3c0d03a0 msgr2=0x7f8c3c0d1560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:24.087 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:24.081+0000 7f8c40e89700 1 --2- 172.21.15.143:0/1713314480 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8c3c0d03a0 0x7f8c3c0d1560 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f8c300098f0 tx=0x7f8c30009c00 comp rx=0 tx=0).stop 2023-11-08T19:27:24.087 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:24.081+0000 7f8c40e89700 1 -- 172.21.15.143:0/1713314480 shutdown_connections 2023-11-08T19:27:24.088 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:24.081+0000 7f8c40e89700 1 --2- 172.21.15.143:0/1713314480 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8c3c15ba30 0x7f8c3c0b14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:24.088 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:24.081+0000 7f8c40e89700 1 --2- 172.21.15.143:0/1713314480 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8c3c0d03a0 0x7f8c3c0d1560 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:24.088 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:24.081+0000 7f8c40e89700 1 -- 172.21.15.143:0/1713314480 >> 172.21.15.143:0/1713314480 conn(0x7f8c3c08b000 msgr2=0x7f8c3c09ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:24.088 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:24.081+0000 7f8c40e89700 1 -- 172.21.15.143:0/1713314480 shutdown_connections 2023-11-08T19:27:24.089 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:24.081+0000 7f8c40e89700 1 -- 172.21.15.143:0/1713314480 wait complete. 2023-11-08T19:27:24.092 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:24.083+0000 7f8c40e89700 1 Processor -- start 2023-11-08T19:27:24.092 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:24.083+0000 7f8c40e89700 1 -- start start 2023-11-08T19:27:24.092 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:24.083+0000 7f8c40e89700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c3c0d03a0 0x7f8c3c0cb010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:24.092 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:24.083+0000 7f8c40e89700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8c3c0d1aa0 0x7f8c3c15c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:24.093 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:24.083+0000 7f8c40e89700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8c3c15ba30 0x7f8c3c15d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:24.093 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:24.083+0000 7f8c40e89700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c3c15ddb0 con 0x7f8c3c0d1aa0 2023-11-08T19:27:24.097 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:24.083+0000 7f8c40e89700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c3c15eab0 con 0x7f8c3c0d03a0 2023-11-08T19:27:24.098 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:24.083+0000 7f8c40e89700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c3c15db50 con 0x7f8c3c15ba30 2023-11-08T19:27:24.098 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:24.083+0000 7f8c39d9b700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8c3c0d1aa0 0x7f8c3c15c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:24.098 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:24.083+0000 7f8c39d9b700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8c3c0d1aa0 0x7f8c3c15c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53376/0 (socket says 172.21.15.143:53376) 2023-11-08T19:27:24.098 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:24.083+0000 7f8c39d9b700 1 -- 172.21.15.143:0/843056041 learned_addr learned my addr 172.21.15.143:0/843056041 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:24.099 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:24.084+0000 7f8c3a59c700 1 --2- 172.21.15.143:0/843056041 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c3c0d03a0 0x7f8c3c0cb010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:24.099 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:24.084+0000 7f8c39d9b700 1 -- 172.21.15.143:0/843056041 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8c3c15ba30 msgr2=0x7f8c3c15d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:24.099 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:24.084+0000 7f8c39d9b700 1 --2- 172.21.15.143:0/843056041 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8c3c15ba30 0x7f8c3c15d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:24.099 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:24.084+0000 7f8c39d9b700 1 -- 172.21.15.143:0/843056041 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c3c0d03a0 msgr2=0x7f8c3c0cb010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:24.100 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:24.084+0000 7f8c39d9b700 1 --2- 172.21.15.143:0/843056041 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c3c0d03a0 0x7f8c3c0cb010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:24.100 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:24.084+0000 7f8c39d9b700 1 -- 172.21.15.143:0/843056041 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c30009550 con 0x7f8c3c0d1aa0 2023-11-08T19:27:24.100 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:24.084+0000 7f8c3a59c700 1 --2- 172.21.15.143:0/843056041 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c3c0d03a0 0x7f8c3c0cb010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:27:24.100 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:24.084+0000 7f8c39d9b700 1 --2- 172.21.15.143:0/843056041 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8c3c0d1aa0 0x7f8c3c15c430 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f8c30000c00 tx=0x7f8c30004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:24.100 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:24.084+0000 7f8c2affd700 1 -- 172.21.15.143:0/843056041 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c30004570 con 0x7f8c3c0d1aa0 2023-11-08T19:27:24.101 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:24.084+0000 7f8c2affd700 1 -- 172.21.15.143:0/843056041 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c3001f070 con 0x7f8c3c0d1aa0 2023-11-08T19:27:24.101 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:24.084+0000 7f8c40e89700 1 -- 172.21.15.143:0/843056041 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c3c0b4c30 con 0x7f8c3c0d1aa0 2023-11-08T19:27:24.101 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:24.084+0000 7f8c2affd700 1 -- 172.21.15.143:0/843056041 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c300038c0 con 0x7f8c3c0d1aa0 2023-11-08T19:27:24.101 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:24.084+0000 7f8c40e89700 1 -- 172.21.15.143:0/843056041 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c3c0b4950 con 0x7f8c3c0d1aa0 2023-11-08T19:27:24.101 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:24.085+0000 7f8c40e89700 1 -- 172.21.15.143:0/843056041 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c3c15e880 con 0x7f8c3c0d1aa0 2023-11-08T19:27:24.102 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:24.085+0000 7f8c2affd700 1 -- 172.21.15.143:0/843056041 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8c30021030 con 0x7f8c3c0d1aa0 2023-11-08T19:27:24.102 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:24.086+0000 7f8c2affd700 1 --2- 172.21.15.143:0/843056041 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f8c1c06af00 0x7f8c1c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:24.102 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:24.086+0000 7f8c2affd700 1 -- 172.21.15.143:0/843056041 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6006+0+0 (secure 0 0 0) 0x7f8c3008ab40 con 0x7f8c3c0d1aa0 2023-11-08T19:27:24.102 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:24.086+0000 7f8c3a59c700 1 --2- 172.21.15.143:0/843056041 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f8c1c06af00 0x7f8c1c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:24.103 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:24.086+0000 7f8c3a59c700 1 --2- 172.21.15.143:0/843056041 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f8c1c06af00 0x7f8c1c06d3c0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f8c2c009ac0 tx=0x7f8c2c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:24.103 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:24.090+0000 7f8c2affd700 1 -- 172.21.15.143:0/843056041 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8c3c15e880 con 0x7f8c3c0d1aa0 2023-11-08T19:27:24.362 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:24.361+0000 7f8c40e89700 1 -- 172.21.15.143:0/843056041 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snapshots.TestSnapshots.test_allow_new_snaps_config"]} v 0) v1 -- 0x7f8c3c15ec20 con 0x7f8c3c0d1aa0 2023-11-08T19:27:25.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.268+0000 7f8c2affd700 1 -- 172.21.15.143:0/843056041 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snapshots.TestSnapshots.test_allow_new_snaps_config"]}]=0 v166) v1 ==== 151+0+0 (secure 0 0 0) 0x7f8c3c15ec20 con 0x7f8c3c0d1aa0 2023-11-08T19:27:25.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.271+0000 7f8c40e89700 1 -- 172.21.15.143:0/843056041 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f8c1c06af00 msgr2=0x7f8c1c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:25.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.271+0000 7f8c40e89700 1 --2- 172.21.15.143:0/843056041 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f8c1c06af00 0x7f8c1c06d3c0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f8c2c009ac0 tx=0x7f8c2c008040 comp rx=0 tx=0).stop 2023-11-08T19:27:25.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.271+0000 7f8c40e89700 1 -- 172.21.15.143:0/843056041 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8c3c0d1aa0 msgr2=0x7f8c3c15c430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:25.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.271+0000 7f8c40e89700 1 --2- 172.21.15.143:0/843056041 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8c3c0d1aa0 0x7f8c3c15c430 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f8c30000c00 tx=0x7f8c30004750 comp rx=0 tx=0).stop 2023-11-08T19:27:25.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.271+0000 7f8c40e89700 1 -- 172.21.15.143:0/843056041 shutdown_connections 2023-11-08T19:27:25.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.271+0000 7f8c40e89700 1 --2- 172.21.15.143:0/843056041 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f8c1c06af00 0x7f8c1c06d3c0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:25.274 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.271+0000 7f8c40e89700 1 --2- 172.21.15.143:0/843056041 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8c3c15ba30 0x7f8c3c15d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:25.274 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.271+0000 7f8c40e89700 1 --2- 172.21.15.143:0/843056041 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c3c0d03a0 0x7f8c3c0cb010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:25.274 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.271+0000 7f8c40e89700 1 --2- 172.21.15.143:0/843056041 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8c3c0d1aa0 0x7f8c3c15c430 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:25.274 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.271+0000 7f8c40e89700 1 -- 172.21.15.143:0/843056041 >> 172.21.15.143:0/843056041 conn(0x7f8c3c08b000 msgr2=0x7f8c3c0aecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:25.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.271+0000 7f8c40e89700 1 -- 172.21.15.143:0/843056041 shutdown_connections 2023-11-08T19:27:25.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.271+0000 7f8c40e89700 1 -- 172.21.15.143:0/843056041 wait complete. 2023-11-08T19:27:25.287 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-08T19:27:25.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.414+0000 7f4129cba700 1 Processor -- start 2023-11-08T19:27:25.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.415+0000 7f4129cba700 1 -- start start 2023-11-08T19:27:25.418 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.415+0000 7f4129cba700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f412415db30 0x7f4124162200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:25.419 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.415+0000 7f4129cba700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f41240ceab0 0x7f412406cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:25.419 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.415+0000 7f4129cba700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4124162740 con 0x7f412415df10 2023-11-08T19:27:25.419 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.415+0000 7f4129cba700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41240cf960 con 0x7f412415db30 2023-11-08T19:27:25.419 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.415+0000 7f4129cba700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41240cfc40 con 0x7f41240ceab0 2023-11-08T19:27:25.419 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.416+0000 7f41237fe700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f412415db30 0x7f4124162200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:25.420 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.416+0000 7f41237fe700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f412415db30 0x7f4124162200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53382/0 (socket says 172.21.15.143:53382) 2023-11-08T19:27:25.420 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.416+0000 7f41237fe700 1 -- 172.21.15.143:0/3086060769 learned_addr learned my addr 172.21.15.143:0/3086060769 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:25.420 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.416+0000 7f4123fff700 1 --2- 172.21.15.143:0/3086060769 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f41240ceab0 0x7f412406cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:25.421 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.416+0000 7f41237fe700 1 -- 172.21.15.143:0/3086060769 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f41240ceab0 msgr2=0x7f412406cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:25.421 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.416+0000 7f41237fe700 1 --2- 172.21.15.143:0/3086060769 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f41240ceab0 0x7f412406cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:25.421 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.416+0000 7f41237fe700 1 -- 172.21.15.143:0/3086060769 >> v1:172.21.15.174:6789/0 conn(0x7f412415df10 legacy=0x7f41240ce3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:25.421 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.416+0000 7f41237fe700 1 -- 172.21.15.143:0/3086060769 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f412406c240 con 0x7f412415db30 2023-11-08T19:27:25.421 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.416+0000 7f41237fe700 1 --2- 172.21.15.143:0/3086060769 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f412415db30 0x7f4124162200 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f410c009920 tx=0x7f410c009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4daf1d5e90a72f19 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:25.422 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.416+0000 7f41227fc700 1 -- 172.21.15.143:0/3086060769 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f410c004230 con 0x7f412415db30 2023-11-08T19:27:25.422 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.416+0000 7f41227fc700 1 -- 172.21.15.143:0/3086060769 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f410c004390 con 0x7f412415db30 2023-11-08T19:27:25.423 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.416+0000 7f41227fc700 1 -- 172.21.15.143:0/3086060769 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f410c009e70 con 0x7f412415db30 2023-11-08T19:27:25.423 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.417+0000 7f4129cba700 1 -- 172.21.15.143:0/3086060769 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f412415db30 msgr2=0x7f4124162200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:25.423 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.417+0000 7f4129cba700 1 --2- 172.21.15.143:0/3086060769 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f412415db30 0x7f4124162200 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f410c009920 tx=0x7f410c009c30 comp rx=0 tx=0).stop 2023-11-08T19:27:25.423 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.417+0000 7f4129cba700 1 -- 172.21.15.143:0/3086060769 shutdown_connections 2023-11-08T19:27:25.423 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.417+0000 7f4129cba700 1 --2- 172.21.15.143:0/3086060769 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f41240ceab0 0x7f412406cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:25.424 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.417+0000 7f4129cba700 1 --2- 172.21.15.143:0/3086060769 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f412415db30 0x7f4124162200 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:25.424 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.417+0000 7f4129cba700 1 -- 172.21.15.143:0/3086060769 >> 172.21.15.143:0/3086060769 conn(0x7f412406f2e0 msgr2=0x7f412406f6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:25.424 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.417+0000 7f4129cba700 1 -- 172.21.15.143:0/3086060769 shutdown_connections 2023-11-08T19:27:25.424 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.417+0000 7f4129cba700 1 -- 172.21.15.143:0/3086060769 wait complete. 2023-11-08T19:27:25.427 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.418+0000 7f4129cba700 1 Processor -- start 2023-11-08T19:27:25.427 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.418+0000 7f4129cba700 1 -- start start 2023-11-08T19:27:25.427 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.419+0000 7f4129cba700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f41240ceab0 0x7f41241e8690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:25.427 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.419+0000 7f4129cba700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f412415db30 0x7f412415a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:25.428 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.419+0000 7f4129cba700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f412415df10 0x7f4124157a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:25.428 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.419+0000 7f4129cba700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f412415abe0 con 0x7f412415db30 2023-11-08T19:27:25.428 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.419+0000 7f4129cba700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41241e99a0 con 0x7f41240ceab0 2023-11-08T19:27:25.428 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.419+0000 7f4129cba700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41241e8d90 con 0x7f412415df10 2023-11-08T19:27:25.428 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.419+0000 7f4122ffd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f412415db30 0x7f412415a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:25.429 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.419+0000 7f4122ffd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f412415db30 0x7f412415a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53392/0 (socket says 172.21.15.143:53392) 2023-11-08T19:27:25.429 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.419+0000 7f4122ffd700 1 -- 172.21.15.143:0/2924955332 learned_addr learned my addr 172.21.15.143:0/2924955332 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:25.429 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.419+0000 7f4123fff700 1 --2- 172.21.15.143:0/2924955332 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f412415df10 0x7f4124157a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:25.429 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.419+0000 7f41237fe700 1 --2- 172.21.15.143:0/2924955332 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f41240ceab0 0x7f41241e8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:25.430 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.419+0000 7f4122ffd700 1 -- 172.21.15.143:0/2924955332 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f412415df10 msgr2=0x7f4124157a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:25.430 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.419+0000 7f4122ffd700 1 --2- 172.21.15.143:0/2924955332 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f412415df10 0x7f4124157a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:25.430 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.419+0000 7f4122ffd700 1 -- 172.21.15.143:0/2924955332 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f41240ceab0 msgr2=0x7f41241e8690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:25.430 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.419+0000 7f4122ffd700 1 --2- 172.21.15.143:0/2924955332 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f41240ceab0 0x7f41241e8690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:25.431 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.419+0000 7f4122ffd700 1 -- 172.21.15.143:0/2924955332 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f410c009550 con 0x7f412415db30 2023-11-08T19:27:25.431 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.420+0000 7f4122ffd700 1 --2- 172.21.15.143:0/2924955332 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f412415db30 0x7f412415a6a0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f411400b450 tx=0x7f411400b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:25.431 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.420+0000 7f411bfff700 1 -- 172.21.15.143:0/2924955332 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4114011830 con 0x7f412415db30 2023-11-08T19:27:25.431 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.420+0000 7f411bfff700 1 -- 172.21.15.143:0/2924955332 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4114011990 con 0x7f412415db30 2023-11-08T19:27:25.431 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.420+0000 7f4129cba700 1 -- 172.21.15.143:0/2924955332 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41241e9070 con 0x7f412415db30 2023-11-08T19:27:25.432 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.420+0000 7f411bfff700 1 -- 172.21.15.143:0/2924955332 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4114011d40 con 0x7f412415db30 2023-11-08T19:27:25.432 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.420+0000 7f4129cba700 1 -- 172.21.15.143:0/2924955332 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41241e9630 con 0x7f412415db30 2023-11-08T19:27:25.432 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.421+0000 7f4129cba700 1 -- 172.21.15.143:0/2924955332 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41241e9b30 con 0x7f412415db30 2023-11-08T19:27:25.432 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.421+0000 7f411bfff700 1 -- 172.21.15.143:0/2924955332 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f411401a400 con 0x7f412415db30 2023-11-08T19:27:25.432 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.421+0000 7f411bfff700 1 --2- 172.21.15.143:0/2924955332 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f410406af00 0x7f410406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:25.433 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.422+0000 7f411bfff700 1 -- 172.21.15.143:0/2924955332 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6006+0+0 (secure 0 0 0) 0x7f4114089420 con 0x7f412415db30 2023-11-08T19:27:25.433 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.422+0000 7f41237fe700 1 --2- 172.21.15.143:0/2924955332 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f410406af00 0x7f410406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:25.433 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.422+0000 7f41237fe700 1 --2- 172.21.15.143:0/2924955332 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f410406af00 0x7f410406d3c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f410c009130 tx=0x7f410c008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:25.434 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.426+0000 7f411bfff700 1 -- 172.21.15.143:0/2924955332 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f41140543a0 con 0x7f412415db30 2023-11-08T19:27:25.618 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.616+0000 7f4129cba700 1 -- 172.21.15.143:0/2924955332 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f412404eee0 con 0x7f412415db30 2023-11-08T19:27:25.623 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.622+0000 7f411bfff700 1 -- 172.21.15.143:0/2924955332 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v17)=0 v17) v1 ==== 122+0+0 (secure 0 0 0) 0x7f41140579c0 con 0x7f412415db30 2023-11-08T19:27:25.626 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.624+0000 7f4129cba700 1 -- 172.21.15.143:0/2924955332 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f410406af00 msgr2=0x7f410406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:25.626 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.624+0000 7f4129cba700 1 --2- 172.21.15.143:0/2924955332 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f410406af00 0x7f410406d3c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f410c009130 tx=0x7f410c008030 comp rx=0 tx=0).stop 2023-11-08T19:27:25.626 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.624+0000 7f4129cba700 1 -- 172.21.15.143:0/2924955332 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f412415db30 msgr2=0x7f412415a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:25.626 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.624+0000 7f4129cba700 1 --2- 172.21.15.143:0/2924955332 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f412415db30 0x7f412415a6a0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f411400b450 tx=0x7f411400b810 comp rx=0 tx=0).stop 2023-11-08T19:27:25.627 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.624+0000 7f4129cba700 1 -- 172.21.15.143:0/2924955332 shutdown_connections 2023-11-08T19:27:25.627 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.624+0000 7f4129cba700 1 --2- 172.21.15.143:0/2924955332 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f410406af00 0x7f410406d3c0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:25.627 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.624+0000 7f4129cba700 1 --2- 172.21.15.143:0/2924955332 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f412415df10 0x7f4124157a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:25.627 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.624+0000 7f4129cba700 1 --2- 172.21.15.143:0/2924955332 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f41240ceab0 0x7f41241e8690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:25.627 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.624+0000 7f4129cba700 1 --2- 172.21.15.143:0/2924955332 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f412415db30 0x7f412415a6a0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:25.628 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.624+0000 7f4129cba700 1 -- 172.21.15.143:0/2924955332 >> 172.21.15.143:0/2924955332 conn(0x7f412406f2e0 msgr2=0x7f4124160ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:25.628 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.624+0000 7f4129cba700 1 -- 172.21.15.143:0/2924955332 shutdown_connections 2023-11-08T19:27:25.628 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.624+0000 7f4129cba700 1 -- 172.21.15.143:0/2924955332 wait complete. 2023-11-08T19:27:25.638 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-08T19:27:25.638 DEBUG:teuthology.orchestra.run.smithi143:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-08T19:27:25.678 DEBUG:teuthology.orchestra.run.smithi174:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-08T19:27:25.726 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-08T19:27:25.727 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-08T19:27:25.727 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:27:25.745 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284264k,nr_inodes=4071066,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=44,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20643 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-08T19:27:25.746 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2023-11-08T19:27:25.746 DEBUG:teuthology.orchestra.run.smithi174:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:27:25.765 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284224k,nr_inodes=4071056,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=31,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19549 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260544k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-08T19:27:25.766 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-08T19:27:25.766 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:27:25.802 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284264k,nr_inodes=4071066,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=44,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20643 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-08T19:27:25.803 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2023-11-08T19:27:25.803 DEBUG:teuthology.orchestra.run.smithi174:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:27:25.824 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284224k,nr_inodes=4071056,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=31,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19549 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260544k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-08T19:27:25.825 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-08T19:27:25.825 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:27:25.962 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.960+0000 7fea553b7700 1 Processor -- start 2023-11-08T19:27:25.962 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.960+0000 7fea553b7700 1 -- start start 2023-11-08T19:27:25.964 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.961+0000 7fea553b7700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fea5015dc80 0x7fea500d1f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:25.965 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.961+0000 7fea553b7700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea500d3200 con 0x7fea500cf090 2023-11-08T19:27:25.965 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.961+0000 7fea553b7700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea5015e060 con 0x7fea500d3690 2023-11-08T19:27:25.965 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.961+0000 7fea553b7700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea500d25f0 con 0x7fea5015dc80 2023-11-08T19:27:25.965 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.961+0000 7fea4f7fe700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fea500cf090 0x7fea500b6ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:49554/0 (socket says 172.21.15.143:49554) 2023-11-08T19:27:25.965 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.961+0000 7fea4f7fe700 1 -- 172.21.15.143:0/1081256504 learned_addr learned my addr 172.21.15.143:0/1081256504 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:27:25.966 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.961+0000 7fea4effd700 1 --2- 172.21.15.143:0/1081256504 >> v2:172.21.15.174:3300/0 conn(0x7fea5015dc80 0x7fea500d1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:25.966 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.961+0000 7fea4dffb700 1 -- 172.21.15.143:0/1081256504 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1111316445 0 0) 0x7fea500d3200 con 0x7fea500cf090 2023-11-08T19:27:25.966 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.961+0000 7fea4dffb700 1 -- 172.21.15.143:0/1081256504 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fea3c0035b0 con 0x7fea500cf090 2023-11-08T19:27:25.966 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.962+0000 7fea4effd700 1 -- 172.21.15.143:0/1081256504 >> v1:172.21.15.174:6789/0 conn(0x7fea500d3690 legacy=0x7fea500ce980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:25.966 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.962+0000 7fea4dffb700 1 -- 172.21.15.143:0/1081256504 <== mon.1 v1:172.21.15.174:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 941133815 0 0) 0x7fea5015e060 con 0x7fea500d3690 2023-11-08T19:27:25.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.962+0000 7fea4effd700 1 -- 172.21.15.143:0/1081256504 >> v1:172.21.15.143:6789/0 conn(0x7fea500cf090 legacy=0x7fea500b6ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:25.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.962+0000 7fea4effd700 1 -- 172.21.15.143:0/1081256504 --> v2:172.21.15.174:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea500d28d0 con 0x7fea5015dc80 2023-11-08T19:27:25.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.962+0000 7fea4effd700 1 --2- 172.21.15.143:0/1081256504 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fea5015dc80 0x7fea500d1f80 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fea38009840 tx=0x7fea38009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de0a2098dd9f2b79 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:25.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.962+0000 7fea4dffb700 1 -- 172.21.15.143:0/1081256504 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea38011830 con 0x7fea5015dc80 2023-11-08T19:27:25.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.962+0000 7fea4dffb700 1 -- 172.21.15.143:0/1081256504 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea38011990 con 0x7fea5015dc80 2023-11-08T19:27:25.968 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.962+0000 7fea4dffb700 1 -- 172.21.15.143:0/1081256504 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea38011d20 con 0x7fea5015dc80 2023-11-08T19:27:25.968 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.963+0000 7fea553b7700 1 -- 172.21.15.143:0/1081256504 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fea5015dc80 msgr2=0x7fea500d1f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:25.968 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.963+0000 7fea553b7700 1 --2- 172.21.15.143:0/1081256504 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fea5015dc80 0x7fea500d1f80 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fea38009840 tx=0x7fea38009b50 comp rx=0 tx=0).stop 2023-11-08T19:27:25.968 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.963+0000 7fea553b7700 1 -- 172.21.15.143:0/1081256504 shutdown_connections 2023-11-08T19:27:25.968 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.963+0000 7fea553b7700 1 --2- 172.21.15.143:0/1081256504 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fea5015dc80 0x7fea500d1f80 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:25.969 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.963+0000 7fea553b7700 1 -- 172.21.15.143:0/1081256504 >> 172.21.15.143:0/1081256504 conn(0x7fea500668e0 msgr2=0x7fea50066cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:25.969 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.963+0000 7fea553b7700 1 -- 172.21.15.143:0/1081256504 shutdown_connections 2023-11-08T19:27:25.969 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.963+0000 7fea553b7700 1 -- 172.21.15.143:0/1081256504 wait complete. 2023-11-08T19:27:25.972 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.964+0000 7fea553b7700 1 Processor -- start 2023-11-08T19:27:25.972 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.964+0000 7fea553b7700 1 -- start start 2023-11-08T19:27:25.972 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.965+0000 7fea553b7700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fea500cf090 0x7fea500b5db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:25.972 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.965+0000 7fea553b7700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fea500d3690 0x7fea501f4470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:25.972 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.965+0000 7fea553b7700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fea5015dc80 0x7fea501f3260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:25.973 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.965+0000 7fea553b7700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea500c76e0 con 0x7fea500cf090 2023-11-08T19:27:25.973 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.965+0000 7fea553b7700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea500c8260 con 0x7fea500d3690 2023-11-08T19:27:25.973 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.965+0000 7fea553b7700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea500c7930 con 0x7fea5015dc80 2023-11-08T19:27:25.973 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.965+0000 7fea4effd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fea500cf090 0x7fea500b5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:25.974 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.965+0000 7fea4effd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fea500cf090 0x7fea500b5db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53402/0 (socket says 172.21.15.143:53402) 2023-11-08T19:27:25.974 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.965+0000 7fea4effd700 1 -- 172.21.15.143:0/1126015908 learned_addr learned my addr 172.21.15.143:0/1126015908 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:25.974 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.965+0000 7fea4e7fc700 1 --2- 172.21.15.143:0/1126015908 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fea500d3690 0x7fea501f4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:25.974 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.965+0000 7fea4f7fe700 1 --2- 172.21.15.143:0/1126015908 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fea5015dc80 0x7fea501f3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:25.974 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.965+0000 7fea4effd700 1 -- 172.21.15.143:0/1126015908 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fea5015dc80 msgr2=0x7fea501f3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:25.975 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.965+0000 7fea4effd700 1 --2- 172.21.15.143:0/1126015908 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fea5015dc80 0x7fea501f3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:25.975 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.965+0000 7fea4effd700 1 -- 172.21.15.143:0/1126015908 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fea500d3690 msgr2=0x7fea501f4470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:25.975 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.965+0000 7fea4effd700 1 --2- 172.21.15.143:0/1126015908 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fea500d3690 0x7fea501f4470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:25.975 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.965+0000 7fea4effd700 1 -- 172.21.15.143:0/1126015908 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea380094a0 con 0x7fea500cf090 2023-11-08T19:27:25.975 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.965+0000 7fea4effd700 1 --2- 172.21.15.143:0/1126015908 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fea500cf090 0x7fea500b5db0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fea38009b90 tx=0x7fea38004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:25.976 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.965+0000 7fea477fe700 1 -- 172.21.15.143:0/1126015908 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea3801ab20 con 0x7fea500cf090 2023-11-08T19:27:25.976 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.965+0000 7fea477fe700 1 -- 172.21.15.143:0/1126015908 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea3801ac80 con 0x7fea500cf090 2023-11-08T19:27:25.976 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.965+0000 7fea477fe700 1 -- 172.21.15.143:0/1126015908 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea380243f0 con 0x7fea500cf090 2023-11-08T19:27:25.976 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.965+0000 7fea553b7700 1 -- 172.21.15.143:0/1126015908 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea500c7c10 con 0x7fea500cf090 2023-11-08T19:27:25.976 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.965+0000 7fea553b7700 1 -- 172.21.15.143:0/1126015908 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea501f49b0 con 0x7fea500cf090 2023-11-08T19:27:25.976 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.966+0000 7fea553b7700 1 -- 172.21.15.143:0/1126015908 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea500d15d0 con 0x7fea500cf090 2023-11-08T19:27:25.977 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.967+0000 7fea477fe700 1 -- 172.21.15.143:0/1126015908 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fea3802d450 con 0x7fea500cf090 2023-11-08T19:27:25.977 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.967+0000 7fea477fe700 1 --2- 172.21.15.143:0/1126015908 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fea3006af30 0x7fea3006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:25.977 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.967+0000 7fea477fe700 1 -- 172.21.15.143:0/1126015908 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6006+0+0 (secure 0 0 0) 0x7fea3808aab0 con 0x7fea500cf090 2023-11-08T19:27:25.977 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.967+0000 7fea4e7fc700 1 --2- 172.21.15.143:0/1126015908 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fea3006af30 0x7fea3006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:25.977 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.968+0000 7fea4e7fc700 1 --2- 172.21.15.143:0/1126015908 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fea3006af30 0x7fea3006d3f0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fea50157a70 tx=0x7fea40009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:25.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:25.972+0000 7fea477fe700 1 -- 172.21.15.143:0/1126015908 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fea38055ac0 con 0x7fea500cf090 2023-11-08T19:27:26.205 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:26.204+0000 7fea553b7700 1 -- 172.21.15.143:0/1126015908 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fea5004eee0 con 0x7fea500cf090 2023-11-08T19:27:26.206 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:26.204+0000 7fea477fe700 1 -- 172.21.15.143:0/1126015908 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+3708 (secure 0 0 0) 0x7fea38015020 con 0x7fea500cf090 2023-11-08T19:27:26.206 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:27:26.207 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":35,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5231,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6836/4083537882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":4083537882},{"type":"v1","addr":"172.21.15.174:6836","nonce":4083537882}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":5234,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/494923804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6835","nonce":494923804},{"type":"v1","addr":"172.21.15.174:6837","nonce":494923804}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":6009,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6834/3383638390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":3383638390},{"type":"v1","addr":"172.21.15.143:6834","nonce":3383638390}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":6381,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/1999441712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":1999441712},{"type":"v1","addr":"172.21.15.143:6835","nonce":1999441712}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35}],"filesystems":[]} 2023-11-08T19:27:26.208 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:26.206+0000 7fea553b7700 1 -- 172.21.15.143:0/1126015908 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fea3006af30 msgr2=0x7fea3006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:26.208 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:26.206+0000 7fea553b7700 1 --2- 172.21.15.143:0/1126015908 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fea3006af30 0x7fea3006d3f0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fea50157a70 tx=0x7fea40009040 comp rx=0 tx=0).stop 2023-11-08T19:27:26.208 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:26.206+0000 7fea553b7700 1 -- 172.21.15.143:0/1126015908 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fea500cf090 msgr2=0x7fea500b5db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:26.209 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:26.206+0000 7fea553b7700 1 --2- 172.21.15.143:0/1126015908 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fea500cf090 0x7fea500b5db0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fea38009b90 tx=0x7fea38004750 comp rx=0 tx=0).stop 2023-11-08T19:27:26.209 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:26.207+0000 7fea553b7700 1 -- 172.21.15.143:0/1126015908 shutdown_connections 2023-11-08T19:27:26.209 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:26.207+0000 7fea553b7700 1 --2- 172.21.15.143:0/1126015908 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fea3006af30 0x7fea3006d3f0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:26.209 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:26.207+0000 7fea553b7700 1 --2- 172.21.15.143:0/1126015908 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fea5015dc80 0x7fea501f3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:26.209 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:26.207+0000 7fea553b7700 1 --2- 172.21.15.143:0/1126015908 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fea500cf090 0x7fea500b5db0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:26.210 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:26.207+0000 7fea553b7700 1 --2- 172.21.15.143:0/1126015908 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fea500d3690 0x7fea501f4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:26.210 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:26.207+0000 7fea553b7700 1 -- 172.21.15.143:0/1126015908 >> 172.21.15.143:0/1126015908 conn(0x7fea500668e0 msgr2=0x7fea50159740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:26.210 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:26.207+0000 7fea553b7700 1 -- 172.21.15.143:0/1126015908 shutdown_connections 2023-11-08T19:27:26.210 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:26.207+0000 7fea553b7700 1 -- 172.21.15.143:0/1126015908 wait complete. 2023-11-08T19:27:26.211 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 35 2023-11-08T19:27:26.222 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-08T19:27:26.222 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-08T19:27:26.223 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-08T19:27:26.223 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T19:27:26.223 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-08T19:27:26.223 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-08T19:27:26.224 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-08T19:27:26.224 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T19:27:26.224 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-08T19:27:26.225 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-08T19:27:26.225 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-08T19:27:26.225 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T19:27:26.225 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-08T19:27:26.226 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-08T19:27:26.226 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-08T19:27:26.226 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T19:27:26.232 INFO:tasks.ceph.mds.a:Stopped 2023-11-08T19:27:26.232 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-08T19:27:26.233 INFO:tasks.ceph.mds.c:Stopped 2023-11-08T19:27:26.234 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-08T19:27:26.237 INFO:tasks.ceph.mds.b:Stopped 2023-11-08T19:27:26.237 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-08T19:27:26.238 INFO:tasks.ceph.mds.d:Stopped 2023-11-08T19:27:26.239 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-08T19:27:26.275 INFO:tasks.ceph.mds.a:Started 2023-11-08T19:27:26.276 DEBUG:teuthology.parallel:result is None 2023-11-08T19:27:26.277 INFO:tasks.ceph.mds.c:Started 2023-11-08T19:27:26.277 DEBUG:teuthology.parallel:result is None 2023-11-08T19:27:26.280 INFO:tasks.ceph.mds.b:Started 2023-11-08T19:27:26.280 DEBUG:teuthology.parallel:result is None 2023-11-08T19:27:26.281 INFO:tasks.ceph.mds.d:Started 2023-11-08T19:27:26.281 DEBUG:teuthology.parallel:result is None 2023-11-08T19:27:26.281 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-08T19:27:26.354 INFO:tasks.ceph.mds.a.smithi143.stdout:starting mds.a at 2023-11-08T19:27:26.355 INFO:tasks.ceph.mds.c.smithi143.stdout:starting mds.c at 2023-11-08T19:27:26.364 INFO:tasks.ceph.mds.d.smithi174.stdout:starting mds.d at 2023-11-08T19:27:26.364 INFO:tasks.ceph.mds.b.smithi174.stdout:starting mds.b at 2023-11-08T19:27:26.409 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:26.407+0000 7f8c41036700 1 Processor -- start 2023-11-08T19:27:26.409 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:26.407+0000 7f8c41036700 1 -- start start 2023-11-08T19:27:26.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:26.408+0000 7f8c41036700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f8c3c15aa30 0x7f8c3c0cf510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:26.412 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:26.408+0000 7f8c41036700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c3c0cf080 con 0x7f8c3c0b2210 2023-11-08T19:27:26.412 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:26.408+0000 7f8c41036700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c3c0ce470 con 0x7f8c3c0cc530 2023-11-08T19:27:26.412 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:26.408+0000 7f8c41036700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c3c0ce750 con 0x7f8c3c15aa30 2023-11-08T19:27:26.412 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:26.408+0000 7f8c3a59c700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f8c3c0b2210 0x7f8c3c0cbe20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:49588/0 (socket says 172.21.15.143:49588) 2023-11-08T19:27:26.412 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:26.409+0000 7f8c3a59c700 1 -- 172.21.15.143:0/2801570909 learned_addr learned my addr 172.21.15.143:0/2801570909 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:27:26.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:26.409+0000 7f8c3ad9d700 1 --2- 172.21.15.143:0/2801570909 >> v2:172.21.15.174:3300/0 conn(0x7f8c3c15aa30 0x7f8c3c0cf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:26.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:26.409+0000 7f8c39d9b700 1 -- 172.21.15.143:0/2801570909 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3141463893 0 0) 0x7f8c3c0cf080 con 0x7f8c3c0b2210 2023-11-08T19:27:26.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:26.409+0000 7f8c39d9b700 1 -- 172.21.15.143:0/2801570909 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c240035b0 con 0x7f8c3c0b2210 2023-11-08T19:27:26.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:26.409+0000 7f8c3ad9d700 1 -- 172.21.15.143:0/2801570909 >> v1:172.21.15.174:6789/0 conn(0x7f8c3c0cc530 legacy=0x7f8c3c06db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:26.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:26.409+0000 7f8c39d9b700 1 -- 172.21.15.143:0/2801570909 <== mon.1 v1:172.21.15.174:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1612324757 0 0) 0x7f8c3c0ce470 con 0x7f8c3c0cc530 2023-11-08T19:27:26.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:26.409+0000 7f8c3ad9d700 1 -- 172.21.15.143:0/2801570909 >> v1:172.21.15.143:6789/0 conn(0x7f8c3c0b2210 legacy=0x7f8c3c0cbe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:26.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:26.409+0000 7f8c3ad9d700 1 -- 172.21.15.143:0/2801570909 --> v2:172.21.15.174:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c3c0cea30 con 0x7f8c3c15aa30 2023-11-08T19:27:26.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:26.409+0000 7f8c3ad9d700 1 --2- 172.21.15.143:0/2801570909 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c3c15aa30 0x7f8c3c0cf510 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f8c30009820 tx=0x7f8c30009b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5608bdc4d7a408cb server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:26.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:26.409+0000 7f8c39d9b700 1 -- 172.21.15.143:0/2801570909 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c30005e30 con 0x7f8c3c15aa30 2023-11-08T19:27:26.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:26.409+0000 7f8c39d9b700 1 -- 172.21.15.143:0/2801570909 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c30006030 con 0x7f8c3c15aa30 2023-11-08T19:27:26.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:26.409+0000 7f8c39d9b700 1 -- 172.21.15.143:0/2801570909 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c300063c0 con 0x7f8c3c15aa30 2023-11-08T19:27:26.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:26.410+0000 7f8c41036700 1 -- 172.21.15.143:0/2801570909 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c3c15aa30 msgr2=0x7f8c3c0cf510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:26.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:26.410+0000 7f8c41036700 1 --2- 172.21.15.143:0/2801570909 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c3c15aa30 0x7f8c3c0cf510 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f8c30009820 tx=0x7f8c30009b30 comp rx=0 tx=0).stop 2023-11-08T19:27:26.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:26.410+0000 7f8c41036700 1 -- 172.21.15.143:0/2801570909 shutdown_connections 2023-11-08T19:27:26.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:26.410+0000 7f8c41036700 1 --2- 172.21.15.143:0/2801570909 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c3c15aa30 0x7f8c3c0cf510 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:26.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:26.410+0000 7f8c41036700 1 -- 172.21.15.143:0/2801570909 >> 172.21.15.143:0/2801570909 conn(0x7f8c3c099ab0 msgr2=0x7f8c3c0d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:26.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:26.410+0000 7f8c41036700 1 -- 172.21.15.143:0/2801570909 shutdown_connections 2023-11-08T19:27:26.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:26.410+0000 7f8c41036700 1 -- 172.21.15.143:0/2801570909 wait complete. 2023-11-08T19:27:26.419 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:26.412+0000 7f8c41036700 1 Processor -- start 2023-11-08T19:27:26.420 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:26.412+0000 7f8c41036700 1 -- start start 2023-11-08T19:27:26.420 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:26.413+0000 7f8c41036700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8c3c0b2210 0x7f8c3c14fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:26.420 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:26.413+0000 7f8c41036700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8c3c0cc530 0x7f8c3c15d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:26.420 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:26.413+0000 7f8c41036700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c3c15aa30 0x7f8c3c0c99e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:26.421 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:26.413+0000 7f8c41036700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c3c0c4f40 con 0x7f8c3c0b2210 2023-11-08T19:27:26.421 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:26.413+0000 7f8c41036700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c3c1f1840 con 0x7f8c3c15aa30 2023-11-08T19:27:26.421 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:26.413+0000 7f8c41036700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c3c1f1b20 con 0x7f8c3c0cc530 2023-11-08T19:27:26.421 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:26.413+0000 7f8c3ad9d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8c3c0b2210 0x7f8c3c14fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:26.421 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:26.413+0000 7f8c3ad9d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8c3c0b2210 0x7f8c3c14fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53440/0 (socket says 172.21.15.143:53440) 2023-11-08T19:27:26.422 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:26.413+0000 7f8c3ad9d700 1 -- 172.21.15.143:0/3798259928 learned_addr learned my addr 172.21.15.143:0/3798259928 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:26.422 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:26.413+0000 7f8c3a59c700 1 --2- 172.21.15.143:0/3798259928 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8c3c0cc530 0x7f8c3c15d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:26.422 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:26.413+0000 7f8c3b59e700 1 --2- 172.21.15.143:0/3798259928 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c3c15aa30 0x7f8c3c0c99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:26.422 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:26.413+0000 7f8c3ad9d700 1 -- 172.21.15.143:0/3798259928 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8c3c0cc530 msgr2=0x7f8c3c15d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:26.422 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:26.413+0000 7f8c3ad9d700 1 --2- 172.21.15.143:0/3798259928 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8c3c0cc530 0x7f8c3c15d9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:26.423 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:26.413+0000 7f8c3ad9d700 1 -- 172.21.15.143:0/3798259928 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c3c15aa30 msgr2=0x7f8c3c0c99e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:26.423 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:26.413+0000 7f8c3ad9d700 1 --2- 172.21.15.143:0/3798259928 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c3c15aa30 0x7f8c3c0c99e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:26.423 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:26.413+0000 7f8c3ad9d700 1 -- 172.21.15.143:0/3798259928 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c30009430 con 0x7f8c3c0b2210 2023-11-08T19:27:26.423 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:26.413+0000 7f8c3b59e700 1 --2- 172.21.15.143:0/3798259928 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c3c15aa30 0x7f8c3c0c99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:27:26.423 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:26.413+0000 7f8c3ad9d700 1 --2- 172.21.15.143:0/3798259928 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8c3c0b2210 0x7f8c3c14fd20 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f8c30013040 tx=0x7f8c30004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:26.424 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:26.413+0000 7f8c237fe700 1 -- 172.21.15.143:0/3798259928 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c30025430 con 0x7f8c3c0b2210 2023-11-08T19:27:26.424 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:26.413+0000 7f8c237fe700 1 -- 172.21.15.143:0/3798259928 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c30020070 con 0x7f8c3c0b2210 2023-11-08T19:27:26.424 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:26.414+0000 7f8c41036700 1 -- 172.21.15.143:0/3798259928 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c3c1f1e00 con 0x7f8c3c0b2210 2023-11-08T19:27:26.424 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:26.414+0000 7f8c237fe700 1 -- 172.21.15.143:0/3798259928 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c300257f0 con 0x7f8c3c0b2210 2023-11-08T19:27:26.424 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:26.414+0000 7f8c41036700 1 -- 172.21.15.143:0/3798259928 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c3c150260 con 0x7f8c3c0b2210 2023-11-08T19:27:26.425 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:26.415+0000 7f8c41036700 1 -- 172.21.15.143:0/3798259928 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c3c04eee0 con 0x7f8c3c0b2210 2023-11-08T19:27:26.425 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:26.415+0000 7f8c237fe700 1 -- 172.21.15.143:0/3798259928 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8c30022030 con 0x7f8c3c0b2210 2023-11-08T19:27:26.425 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:26.415+0000 7f8c237fe700 1 --2- 172.21.15.143:0/3798259928 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f8c2806af00 0x7f8c2806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:26.425 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:26.416+0000 7f8c237fe700 1 -- 172.21.15.143:0/3798259928 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6006+0+0 (secure 0 0 0) 0x7f8c3008abf0 con 0x7f8c3c0b2210 2023-11-08T19:27:26.425 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:26.416+0000 7f8c3a59c700 1 --2- 172.21.15.143:0/3798259928 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f8c2806af00 0x7f8c2806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:26.426 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:26.416+0000 7f8c3a59c700 1 --2- 172.21.15.143:0/3798259928 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f8c2806af00 0x7f8c2806d3c0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c0afa10 tx=0x7f8c2c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:26.426 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:26.420+0000 7f8c237fe700 1 -- 172.21.15.143:0/3798259928 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8c30055c00 con 0x7f8c3c0b2210 2023-11-08T19:27:26.626 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:26.624+0000 7f8c41036700 1 -- 172.21.15.143:0/3798259928 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f8c3c0cac30 con 0x7f8c3c0b2210 2023-11-08T19:27:26.953 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T19:27:26.956 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T19:27:27.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.288+0000 7f8c237fe700 1 -- 172.21.15.143:0/3798259928 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v72) v1 ==== 97+0+0 (secure 0 0 0) 0x7f8c30059220 con 0x7f8c3c0b2210 2023-11-08T19:27:27.293 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.292+0000 7f8c41036700 1 -- 172.21.15.143:0/3798259928 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f8c2806af00 msgr2=0x7f8c2806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:27.294 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.292+0000 7f8c41036700 1 --2- 172.21.15.143:0/3798259928 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f8c2806af00 0x7f8c2806d3c0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c0afa10 tx=0x7f8c2c009040 comp rx=0 tx=0).stop 2023-11-08T19:27:27.294 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.292+0000 7f8c41036700 1 -- 172.21.15.143:0/3798259928 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8c3c0b2210 msgr2=0x7f8c3c14fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:27.294 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.292+0000 7f8c41036700 1 --2- 172.21.15.143:0/3798259928 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8c3c0b2210 0x7f8c3c14fd20 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f8c30013040 tx=0x7f8c30004750 comp rx=0 tx=0).stop 2023-11-08T19:27:27.294 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.292+0000 7f8c41036700 1 -- 172.21.15.143:0/3798259928 shutdown_connections 2023-11-08T19:27:27.294 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.292+0000 7f8c41036700 1 --2- 172.21.15.143:0/3798259928 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f8c2806af00 0x7f8c2806d3c0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:27.295 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.292+0000 7f8c41036700 1 --2- 172.21.15.143:0/3798259928 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c3c15aa30 0x7f8c3c0c99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:27.295 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.292+0000 7f8c41036700 1 --2- 172.21.15.143:0/3798259928 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8c3c0b2210 0x7f8c3c14fd20 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:27.295 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.292+0000 7f8c41036700 1 --2- 172.21.15.143:0/3798259928 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8c3c0cc530 0x7f8c3c15d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:27.295 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.292+0000 7f8c41036700 1 -- 172.21.15.143:0/3798259928 >> 172.21.15.143:0/3798259928 conn(0x7f8c3c099ab0 msgr2=0x7f8c3c0adcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:27.295 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.292+0000 7f8c41036700 1 -- 172.21.15.143:0/3798259928 shutdown_connections 2023-11-08T19:27:27.296 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.292+0000 7f8c41036700 1 -- 172.21.15.143:0/3798259928 wait complete. 2023-11-08T19:27:27.296 INFO:teuthology.orchestra.run.smithi143.stderr: removed all blocklist entries 2023-11-08T19:27:27.307 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-08T19:27:27.428 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.426+0000 7f91208cf700 1 Processor -- start 2023-11-08T19:27:27.428 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.426+0000 7f91208cf700 1 -- start start 2023-11-08T19:27:27.431 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.427+0000 7f91208cf700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f911c1585e0 0x7f911c06ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:27.431 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.428+0000 7f91208cf700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f911c0b2510 0x7f911c0cec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:27.431 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.428+0000 7f91208cf700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f911c1589c0 con 0x7f911c15e530 2023-11-08T19:27:27.431 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.428+0000 7f91208cf700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f911c15bf30 con 0x7f911c1585e0 2023-11-08T19:27:27.432 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.428+0000 7f91208cf700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f911c0cf9f0 con 0x7f911c0b2510 2023-11-08T19:27:27.432 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.428+0000 7f911ad9d700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f911c15e530 0x7f911c0cb900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:49608/0 (socket says 172.21.15.143:49608) 2023-11-08T19:27:27.432 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.428+0000 7f911ad9d700 1 -- 172.21.15.143:0/4022353484 learned_addr learned my addr 172.21.15.143:0/4022353484 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:27:27.433 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.428+0000 7f911a59c700 1 --2- 172.21.15.143:0/4022353484 >> v2:172.21.15.174:3300/0 conn(0x7f911c1585e0 0x7f911c06ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:27.433 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.428+0000 7f9119d9b700 1 --2- 172.21.15.143:0/4022353484 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f911c0b2510 0x7f911c0cec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:27.433 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.428+0000 7f911959a700 1 -- 172.21.15.143:0/4022353484 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2540431633 0 0) 0x7f9114001490 con 0x7f911c15e530 2023-11-08T19:27:27.433 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.428+0000 7f911959a700 1 -- 172.21.15.143:0/4022353484 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91040035b0 con 0x7f911c15e530 2023-11-08T19:27:27.433 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.428+0000 7f9119d9b700 1 -- 172.21.15.143:0/4022353484 >> v2:172.21.15.174:3300/0 conn(0x7f911c1585e0 msgr2=0x7f911c06ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:27.434 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.428+0000 7f911959a700 1 -- 172.21.15.143:0/4022353484 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3899798591 0 0) 0x7f91040035b0 con 0x7f911c15e530 2023-11-08T19:27:27.434 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.428+0000 7f9119d9b700 1 --2- 172.21.15.143:0/4022353484 >> v2:172.21.15.174:3300/0 conn(0x7f911c1585e0 0x7f911c06ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:27.434 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.428+0000 7f9119d9b700 1 -- 172.21.15.143:0/4022353484 >> v1:172.21.15.143:6789/0 conn(0x7f911c15e530 legacy=0x7f911c0cb900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:27.435 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.428+0000 7f9119d9b700 1 -- 172.21.15.143:0/4022353484 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f911c15b320 con 0x7f911c0b2510 2023-11-08T19:27:27.435 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.428+0000 7f911a59c700 1 --2- 172.21.15.143:0/4022353484 >> v2:172.21.15.174:3300/0 conn(0x7f911c1585e0 0x7f911c06ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T19:27:27.435 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.428+0000 7f9119d9b700 1 --2- 172.21.15.143:0/4022353484 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f911c0b2510 0x7f911c0cec10 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f910c009870 tx=0x7f910c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a08354989a432f6b server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:27.435 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.429+0000 7f911959a700 1 -- 172.21.15.143:0/4022353484 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9114001b80 con 0x7f911c0b2510 2023-11-08T19:27:27.436 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.429+0000 7f911959a700 1 -- 172.21.15.143:0/4022353484 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f910c004030 con 0x7f911c0b2510 2023-11-08T19:27:27.436 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.429+0000 7f911959a700 1 -- 172.21.15.143:0/4022353484 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f910c0043c0 con 0x7f911c0b2510 2023-11-08T19:27:27.436 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.429+0000 7f91208cf700 1 -- 172.21.15.143:0/4022353484 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f911c0b2510 msgr2=0x7f911c0cec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:27.436 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.429+0000 7f91208cf700 1 --2- 172.21.15.143:0/4022353484 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f911c0b2510 0x7f911c0cec10 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f910c009870 tx=0x7f910c009b80 comp rx=0 tx=0).stop 2023-11-08T19:27:27.436 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.429+0000 7f91208cf700 1 -- 172.21.15.143:0/4022353484 shutdown_connections 2023-11-08T19:27:27.437 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.429+0000 7f91208cf700 1 --2- 172.21.15.143:0/4022353484 >> v2:172.21.15.174:3300/0 conn(0x7f911c1585e0 0x7f911c06ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:27.437 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.429+0000 7f91208cf700 1 --2- 172.21.15.143:0/4022353484 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f911c0b2510 0x7f911c0cec10 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:27.437 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.429+0000 7f91208cf700 1 -- 172.21.15.143:0/4022353484 >> 172.21.15.143:0/4022353484 conn(0x7f911c0b1a20 msgr2=0x7f911c0b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:27.437 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.429+0000 7f91208cf700 1 -- 172.21.15.143:0/4022353484 shutdown_connections 2023-11-08T19:27:27.437 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.429+0000 7f91208cf700 1 -- 172.21.15.143:0/4022353484 wait complete. 2023-11-08T19:27:27.440 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.431+0000 7f91208cf700 1 Processor -- start 2023-11-08T19:27:27.440 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.431+0000 7f91208cf700 1 -- start start 2023-11-08T19:27:27.440 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.431+0000 7f91208cf700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f911c0b2510 0x7f911c14e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:27.441 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.431+0000 7f91208cf700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f911c1585e0 0x7f911c1502a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:27.441 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.431+0000 7f91208cf700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f911c15e530 0x7f911c14b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:27.441 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.431+0000 7f91208cf700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f911c166ca0 con 0x7f911c0b2510 2023-11-08T19:27:27.441 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.431+0000 7f91208cf700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f911c166970 con 0x7f911c15e530 2023-11-08T19:27:27.442 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.431+0000 7f91208cf700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f911c14bd70 con 0x7f911c1585e0 2023-11-08T19:27:27.442 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.431+0000 7f911a59c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f911c0b2510 0x7f911c14e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:27.442 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.431+0000 7f911a59c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f911c0b2510 0x7f911c14e4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53456/0 (socket says 172.21.15.143:53456) 2023-11-08T19:27:27.442 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.431+0000 7f911a59c700 1 -- 172.21.15.143:0/2657968734 learned_addr learned my addr 172.21.15.143:0/2657968734 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:27.442 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.432+0000 7f911ad9d700 1 --2- 172.21.15.143:0/2657968734 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f911c15e530 0x7f911c14b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:27.443 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.432+0000 7f911a59c700 1 -- 172.21.15.143:0/2657968734 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f911c1585e0 msgr2=0x7f911c1502a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:27.443 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.432+0000 7f911a59c700 1 --2- 172.21.15.143:0/2657968734 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f911c1585e0 0x7f911c1502a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:27.443 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.432+0000 7f911a59c700 1 -- 172.21.15.143:0/2657968734 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f911c15e530 msgr2=0x7f911c14b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:27.443 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.432+0000 7f911a59c700 1 --2- 172.21.15.143:0/2657968734 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f911c15e530 0x7f911c14b830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:27.443 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.432+0000 7f911a59c700 1 -- 172.21.15.143:0/2657968734 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91100094a0 con 0x7f911c0b2510 2023-11-08T19:27:27.444 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.432+0000 7f911a59c700 1 --2- 172.21.15.143:0/2657968734 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f911c0b2510 0x7f911c14e4c0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f911000ebe0 tx=0x7f9110009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:27.444 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.432+0000 7f910affd700 1 -- 172.21.15.143:0/2657968734 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f911000c910 con 0x7f911c0b2510 2023-11-08T19:27:27.444 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.432+0000 7f910affd700 1 -- 172.21.15.143:0/2657968734 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f911000ca70 con 0x7f911c0b2510 2023-11-08T19:27:27.445 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.432+0000 7f91208cf700 1 -- 172.21.15.143:0/2657968734 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f910c0094a0 con 0x7f911c0b2510 2023-11-08T19:27:27.445 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.432+0000 7f910affd700 1 -- 172.21.15.143:0/2657968734 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f911000cde0 con 0x7f911c0b2510 2023-11-08T19:27:27.445 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.432+0000 7f91208cf700 1 -- 172.21.15.143:0/2657968734 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f911c150c00 con 0x7f911c0b2510 2023-11-08T19:27:27.445 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.433+0000 7f91208cf700 1 -- 172.21.15.143:0/2657968734 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f911c04eee0 con 0x7f911c0b2510 2023-11-08T19:27:27.445 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.434+0000 7f910affd700 1 -- 172.21.15.143:0/2657968734 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9110013470 con 0x7f911c0b2510 2023-11-08T19:27:27.446 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.434+0000 7f910affd700 1 --2- 172.21.15.143:0/2657968734 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f90fc06af00 0x7f90fc06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:27.446 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.434+0000 7f910affd700 1 -- 172.21.15.143:0/2657968734 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5920+0+0 (secure 0 0 0) 0x7f9110089c10 con 0x7f911c0b2510 2023-11-08T19:27:27.446 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.434+0000 7f9119d9b700 1 --2- 172.21.15.143:0/2657968734 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f90fc06af00 0x7f90fc06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:27.446 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.435+0000 7f9119d9b700 1 --2- 172.21.15.143:0/2657968734 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f90fc06af00 0x7f90fc06d3c0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f911c06c200 tx=0x7f910c010000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:27.446 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.438+0000 7f910affd700 1 -- 172.21.15.143:0/2657968734 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9110054bf0 con 0x7f911c0b2510 2023-11-08T19:27:27.705 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.703+0000 7f91208cf700 1 -- 172.21.15.143:0/2657968734 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f911c072930 con 0x7f911c0b2510 2023-11-08T19:27:27.705 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.704+0000 7f910affd700 1 -- 172.21.15.143:0/2657968734 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v11) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f9110058210 con 0x7f911c0b2510 2023-11-08T19:27:27.706 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:27:27.706 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2023-11-08T19:27:27.706 INFO:teuthology.orchestra.run.smithi143.stdout: "auth_dump": [ 2023-11-08T19:27:27.706 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-08T19:27:27.707 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "mds.a", 2023-11-08T19:27:27.707 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQBO4EtlqQPkOhAACwt3IOZLFtEQle9/FFliYA==", 2023-11-08T19:27:27.707 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-08T19:27:27.707 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2023-11-08T19:27:27.708 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow *", 2023-11-08T19:27:27.708 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow *", 2023-11-08T19:27:27.708 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-08T19:27:27.708 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-08T19:27:27.708 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-08T19:27:27.709 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-08T19:27:27.709 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "mds.b", 2023-11-08T19:27:27.709 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQBP4EtlHJqqCxAAxgT7Ce6QICWZ0qebHWZhMQ==", 2023-11-08T19:27:27.709 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-08T19:27:27.709 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2023-11-08T19:27:27.710 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow *", 2023-11-08T19:27:27.710 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow *", 2023-11-08T19:27:27.710 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-08T19:27:27.710 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-08T19:27:27.711 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-08T19:27:27.711 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-08T19:27:27.711 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "mds.c", 2023-11-08T19:27:27.711 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQBP4EtlGEeDBhAAihr82JiBJZR0rRkQj2n8wA==", 2023-11-08T19:27:27.711 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-08T19:27:27.711 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2023-11-08T19:27:27.712 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow *", 2023-11-08T19:27:27.712 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow *", 2023-11-08T19:27:27.712 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-08T19:27:27.712 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-08T19:27:27.712 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-08T19:27:27.713 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-08T19:27:27.713 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "mds.d", 2023-11-08T19:27:27.713 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQBP4EtlW5//EhAArVOkzNpCYd6Skl6eh5EYUQ==", 2023-11-08T19:27:27.713 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-08T19:27:27.713 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2023-11-08T19:27:27.714 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow *", 2023-11-08T19:27:27.714 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow *", 2023-11-08T19:27:27.714 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-08T19:27:27.714 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-08T19:27:27.714 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-08T19:27:27.715 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-08T19:27:27.715 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.0", 2023-11-08T19:27:27.715 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQBS4EtloVd4KRAAi5nj/vB9UK/mWZ9lkaFAxg==", 2023-11-08T19:27:27.715 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-08T19:27:27.716 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2023-11-08T19:27:27.716 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2023-11-08T19:27:27.716 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-08T19:27:27.716 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-08T19:27:27.716 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-08T19:27:27.717 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-08T19:27:27.717 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.1", 2023-11-08T19:27:27.717 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQBV4EtlWlOaKRAAooz5P+AoTgyuL/QAv4REKA==", 2023-11-08T19:27:27.717 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-08T19:27:27.718 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2023-11-08T19:27:27.718 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2023-11-08T19:27:27.718 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-08T19:27:27.719 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-08T19:27:27.719 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-08T19:27:27.719 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-08T19:27:27.719 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.2", 2023-11-08T19:27:27.720 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQBY4EtlBpaWKhAAmob9TfEKw5x6dEIu8ALVGA==", 2023-11-08T19:27:27.720 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-08T19:27:27.720 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2023-11-08T19:27:27.720 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2023-11-08T19:27:27.720 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-08T19:27:27.721 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-08T19:27:27.721 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-08T19:27:27.721 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-08T19:27:27.721 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.3", 2023-11-08T19:27:27.721 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQBb4Etl5pkrKhAA38ATMC3NkZ4plRRA69i0vw==", 2023-11-08T19:27:27.722 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-08T19:27:27.722 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2023-11-08T19:27:27.722 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2023-11-08T19:27:27.722 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-08T19:27:27.722 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-08T19:27:27.723 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-08T19:27:27.723 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-08T19:27:27.723 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.4", 2023-11-08T19:27:27.723 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQBh4EtlqLnUMBAABIrSskahtrZR0ClfczX5Ag==", 2023-11-08T19:27:27.723 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-08T19:27:27.724 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2023-11-08T19:27:27.724 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2023-11-08T19:27:27.724 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-08T19:27:27.724 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-08T19:27:27.724 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-08T19:27:27.725 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-08T19:27:27.725 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.5", 2023-11-08T19:27:27.725 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQBk4Etlt9PTMBAAz55IHg6Jrin3BUAvhOM/Fg==", 2023-11-08T19:27:27.725 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-08T19:27:27.725 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2023-11-08T19:27:27.725 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2023-11-08T19:27:27.726 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-08T19:27:27.726 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-08T19:27:27.726 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-08T19:27:27.727 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-08T19:27:27.727 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.6", 2023-11-08T19:27:27.727 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQBn4EtlcTkyMBAA5fD3h4s+99rgd6fqHHCFMw==", 2023-11-08T19:27:27.727 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-08T19:27:27.727 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2023-11-08T19:27:27.728 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2023-11-08T19:27:27.728 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-08T19:27:27.728 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-08T19:27:27.728 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-08T19:27:27.728 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-08T19:27:27.729 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.7", 2023-11-08T19:27:27.729 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQBq4EtldEviMBAAyGYrHVGwxHdmtDOwWCSHDQ==", 2023-11-08T19:27:27.729 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-08T19:27:27.729 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2023-11-08T19:27:27.729 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2023-11-08T19:27:27.730 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-08T19:27:27.730 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-08T19:27:27.730 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-08T19:27:27.731 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-08T19:27:27.731 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "client.0", 2023-11-08T19:27:27.731 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQBP4Etl0GdSFxAAVkuHL0EnjrtTUMkj/afPkg==", 2023-11-08T19:27:27.731 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-08T19:27:27.732 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2023-11-08T19:27:27.732 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow r", 2023-11-08T19:27:27.732 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-11-08T19:27:27.732 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-08T19:27:27.732 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-08T19:27:27.733 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-08T19:27:27.733 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "client.1", 2023-11-08T19:27:27.733 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQBP4EtlPQ5yIBAAiyQaqCJph2KOPBZPSIh4ZQ==", 2023-11-08T19:27:27.733 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-08T19:27:27.733 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2023-11-08T19:27:27.734 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow r", 2023-11-08T19:27:27.734 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow rw", 2023-11-08T19:27:27.734 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow rwx" 2023-11-08T19:27:27.734 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-08T19:27:27.735 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-08T19:27:27.735 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-08T19:27:27.735 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "client.2", 2023-11-08T19:27:27.735 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQBP4Etlm+8SHRAAAwxcy2EzUvtFw9/mTzyk7w==", 2023-11-08T19:27:27.735 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-08T19:27:27.735 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2023-11-08T19:27:27.736 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow r", 2023-11-08T19:27:27.736 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow rw", 2023-11-08T19:27:27.736 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow rwx" 2023-11-08T19:27:27.736 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-08T19:27:27.736 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-08T19:27:27.737 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-08T19:27:27.737 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "client.3", 2023-11-08T19:27:27.737 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQBP4EtlNDC8IxAACBkCM+3TXhJv5eE9xLvwKA==", 2023-11-08T19:27:27.737 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-08T19:27:27.737 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2023-11-08T19:27:27.738 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow r", 2023-11-08T19:27:27.738 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow rw", 2023-11-08T19:27:27.738 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow rwx" 2023-11-08T19:27:27.738 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-08T19:27:27.741 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-08T19:27:27.741 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-08T19:27:27.741 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "client.admin", 2023-11-08T19:27:27.742 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQBO4Etlr5yuHRAAblpqD18z9bT3Wmgn74dehg==", 2023-11-08T19:27:27.742 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-08T19:27:27.742 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow *", 2023-11-08T19:27:27.742 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow *", 2023-11-08T19:27:27.742 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow *", 2023-11-08T19:27:27.743 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-08T19:27:27.743 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-08T19:27:27.743 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-08T19:27:27.743 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-08T19:27:27.743 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "mgr.x", 2023-11-08T19:27:27.744 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQBO4EtlzrqBNxAA5rtcgkzroSTKuBBmeQRiig==", 2023-11-08T19:27:27.744 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-08T19:27:27.744 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow *", 2023-11-08T19:27:27.744 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile mgr", 2023-11-08T19:27:27.744 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-08T19:27:27.745 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-08T19:27:27.745 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-08T19:27:27.745 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-08T19:27:27.745 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "mgr.y", 2023-11-08T19:27:27.745 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQBO4Etl4Gl4MxAArxdUnjLDhTLO3feXHV4HyQ==", 2023-11-08T19:27:27.746 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-08T19:27:27.746 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow *", 2023-11-08T19:27:27.746 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile mgr", 2023-11-08T19:27:27.746 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-08T19:27:27.747 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-08T19:27:27.747 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-08T19:27:27.747 INFO:teuthology.orchestra.run.smithi143.stdout: ] 2023-11-08T19:27:27.747 INFO:teuthology.orchestra.run.smithi143.stdout:} 2023-11-08T19:27:27.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.706+0000 7f91208cf700 1 -- 172.21.15.143:0/2657968734 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f90fc06af00 msgr2=0x7f90fc06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:27.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.706+0000 7f91208cf700 1 --2- 172.21.15.143:0/2657968734 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f90fc06af00 0x7f90fc06d3c0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f911c06c200 tx=0x7f910c010000 comp rx=0 tx=0).stop 2023-11-08T19:27:27.750 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.706+0000 7f91208cf700 1 -- 172.21.15.143:0/2657968734 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f911c0b2510 msgr2=0x7f911c14e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:27.750 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.706+0000 7f91208cf700 1 --2- 172.21.15.143:0/2657968734 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f911c0b2510 0x7f911c14e4c0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f911000ebe0 tx=0x7f9110009ec0 comp rx=0 tx=0).stop 2023-11-08T19:27:27.750 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.706+0000 7f91208cf700 1 -- 172.21.15.143:0/2657968734 shutdown_connections 2023-11-08T19:27:27.750 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.706+0000 7f91208cf700 1 --2- 172.21.15.143:0/2657968734 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f90fc06af00 0x7f90fc06d3c0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:27.750 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.706+0000 7f91208cf700 1 --2- 172.21.15.143:0/2657968734 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f911c15e530 0x7f911c14b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:27.751 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.706+0000 7f91208cf700 1 --2- 172.21.15.143:0/2657968734 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f911c0b2510 0x7f911c14e4c0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:27.751 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.706+0000 7f91208cf700 1 --2- 172.21.15.143:0/2657968734 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f911c1585e0 0x7f911c1502a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:27.751 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.706+0000 7f91208cf700 1 -- 172.21.15.143:0/2657968734 >> 172.21.15.143:0/2657968734 conn(0x7f911c0b1a20 msgr2=0x7f911c0721b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:27.751 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.706+0000 7f91208cf700 1 -- 172.21.15.143:0/2657968734 shutdown_connections 2023-11-08T19:27:27.751 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.706+0000 7f91208cf700 1 -- 172.21.15.143:0/2657968734 wait complete. 2023-11-08T19:27:27.752 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:27:27.874 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.873+0000 7fbe0a001700 1 Processor -- start 2023-11-08T19:27:27.874 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.873+0000 7fbe0a001700 1 -- start start 2023-11-08T19:27:27.882 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.873+0000 7fbe0a001700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbe040cad70 0x7fbe040b2b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:27.882 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.873+0000 7fbe0a001700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe04071320 con 0x7fbe040ca990 2023-11-08T19:27:27.882 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.873+0000 7fbe0a001700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe040b5390 con 0x7fbe0415f580 2023-11-08T19:27:27.883 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.873+0000 7fbe0a001700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe040b5630 con 0x7fbe040cad70 2023-11-08T19:27:27.883 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.874+0000 7fbe037fe700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fbe040ca990 0x7fbe0415ee70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:49618/0 (socket says 172.21.15.143:49618) 2023-11-08T19:27:27.883 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.874+0000 7fbe037fe700 1 -- 172.21.15.143:0/730880036 learned_addr learned my addr 172.21.15.143:0/730880036 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:27:27.883 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.874+0000 7fbe03fff700 1 --2- 172.21.15.143:0/730880036 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbe040cad70 0x7fbe040b2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:27.884 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.874+0000 7fbe027fc700 1 -- 172.21.15.143:0/730880036 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2725750282 0 0) 0x7fbe04071320 con 0x7fbe040ca990 2023-11-08T19:27:27.884 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.874+0000 7fbe027fc700 1 -- 172.21.15.143:0/730880036 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbdf00035b0 con 0x7fbe040ca990 2023-11-08T19:27:27.884 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.874+0000 7fbe027fc700 1 -- 172.21.15.143:0/730880036 <== mon.1 v1:172.21.15.174:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1418781502 0 0) 0x7fbe040b5390 con 0x7fbe0415f580 2023-11-08T19:27:27.884 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.874+0000 7fbe027fc700 1 -- 172.21.15.143:0/730880036 --> v1:172.21.15.174:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe04071320 con 0x7fbe0415f580 2023-11-08T19:27:27.885 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.874+0000 7fbe027fc700 1 -- 172.21.15.143:0/730880036 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 141178346 0 0) 0x7fbdf00035b0 con 0x7fbe040ca990 2023-11-08T19:27:27.885 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.874+0000 7fbe027fc700 1 -- 172.21.15.143:0/730880036 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbe040cad70 msgr2=0x7fbe040b2b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:27.885 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.874+0000 7fbe027fc700 1 --2- 172.21.15.143:0/730880036 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbe040cad70 0x7fbe040b2b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:27.885 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.874+0000 7fbe027fc700 1 -- 172.21.15.143:0/730880036 >> v1:172.21.15.174:6789/0 conn(0x7fbe0415f580 legacy=0x7fbe040717b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:27.886 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.874+0000 7fbe027fc700 1 -- 172.21.15.143:0/730880036 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe040b5910 con 0x7fbe040ca990 2023-11-08T19:27:27.886 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.874+0000 7fbe027fc700 1 -- 172.21.15.143:0/730880036 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4147756303 0 0) 0x7fbdf4001670 con 0x7fbe040ca990 2023-11-08T19:27:27.886 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.874+0000 7fbe027fc700 1 -- 172.21.15.143:0/730880036 >> v1:172.21.15.143:6789/0 conn(0x7fbe040ca990 legacy=0x7fbe0415ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:27.886 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.874+0000 7fbe027fc700 1 --2- 172.21.15.143:0/730880036 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbdf0003bc0 0x7fbdf0003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:27.886 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.874+0000 7fbe027fc700 1 --2- 172.21.15.143:0/730880036 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbdf0004850 0x7fbdf0004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:27.887 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.874+0000 7fbe027fc700 1 --2- 172.21.15.143:0/730880036 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbe040cad70 0x7fbdf0005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:27.887 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.874+0000 7fbe027fc700 1 -- 172.21.15.143:0/730880036 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe040b5630 con 0x7fbdf0003bc0 2023-11-08T19:27:27.887 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.874+0000 7fbe027fc700 1 -- 172.21.15.143:0/730880036 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdf000bac0 con 0x7fbdf0004850 2023-11-08T19:27:27.887 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.874+0000 7fbe027fc700 1 -- 172.21.15.143:0/730880036 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdf000bc50 con 0x7fbe040cad70 2023-11-08T19:27:27.887 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.874+0000 7fbe037fe700 1 --2- 172.21.15.143:0/730880036 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbdf0003bc0 0x7fbdf0003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:27.888 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.874+0000 7fbe03fff700 1 --2- 172.21.15.143:0/730880036 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbe040cad70 0x7fbdf0005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:27.888 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.874+0000 7fbe02ffd700 1 --2- 172.21.15.143:0/730880036 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbdf0004850 0x7fbdf0004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:27.888 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.874+0000 7fbe037fe700 1 -- 172.21.15.143:0/730880036 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbe040cad70 msgr2=0x7fbdf0005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:27.888 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.874+0000 7fbe037fe700 1 --2- 172.21.15.143:0/730880036 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbe040cad70 0x7fbdf0005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:27.888 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.874+0000 7fbe037fe700 1 -- 172.21.15.143:0/730880036 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbdf0004850 msgr2=0x7fbdf0004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:27.889 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.874+0000 7fbe037fe700 1 --2- 172.21.15.143:0/730880036 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbdf0004850 0x7fbdf0004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:27.889 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.874+0000 7fbe037fe700 1 -- 172.21.15.143:0/730880036 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdf00035b0 con 0x7fbdf0003bc0 2023-11-08T19:27:27.889 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.875+0000 7fbe037fe700 1 --2- 172.21.15.143:0/730880036 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbdf0003bc0 0x7fbdf0003fc0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fbdf400c210 tx=0x7fbdf0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc0ff7720840b1a4 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:27.889 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.875+0000 7fbe027fc700 1 -- 172.21.15.143:0/730880036 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdf400cd70 con 0x7fbdf0003bc0 2023-11-08T19:27:27.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.875+0000 7fbe027fc700 1 -- 172.21.15.143:0/730880036 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdf4001af0 con 0x7fbdf0003bc0 2023-11-08T19:27:27.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.875+0000 7fbe027fc700 1 -- 172.21.15.143:0/730880036 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdf4005ba0 con 0x7fbdf0003bc0 2023-11-08T19:27:27.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.875+0000 7fbe0a001700 1 -- 172.21.15.143:0/730880036 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbdf0003bc0 msgr2=0x7fbdf0003fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:27.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.875+0000 7fbe0a001700 1 --2- 172.21.15.143:0/730880036 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbdf0003bc0 0x7fbdf0003fc0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fbdf400c210 tx=0x7fbdf0005880 comp rx=0 tx=0).stop 2023-11-08T19:27:27.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.875+0000 7fbe03fff700 1 -- 172.21.15.143:0/730880036 reap_dead start 2023-11-08T19:27:27.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.875+0000 7fbe0a001700 1 -- 172.21.15.143:0/730880036 shutdown_connections 2023-11-08T19:27:27.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.875+0000 7fbe0a001700 1 -- 172.21.15.143:0/730880036 >> 172.21.15.143:0/730880036 conn(0x7fbe040b1650 msgr2=0x7fbe040b1a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:27.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.875+0000 7fbe0a001700 1 -- 172.21.15.143:0/730880036 shutdown_connections 2023-11-08T19:27:27.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.875+0000 7fbe0a001700 1 -- 172.21.15.143:0/730880036 wait complete. 2023-11-08T19:27:27.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.877+0000 7fbe0a001700 1 Processor -- start 2023-11-08T19:27:27.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.877+0000 7fbe0a001700 1 -- start start 2023-11-08T19:27:27.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.878+0000 7fbe0a001700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbdf0003bc0 0x7fbe041f2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:27.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.878+0000 7fbe0a001700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbe0415ded0 0x7fbe0406d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:27.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.878+0000 7fbe0a001700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbe0415e350 0x7fbe0406dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:27.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.878+0000 7fbe0a001700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe040c7d10 con 0x7fbdf0003bc0 2023-11-08T19:27:27.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.878+0000 7fbe0a001700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe040cd9e0 con 0x7fbe0415e350 2023-11-08T19:27:27.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.878+0000 7fbe0a001700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe040cdcc0 con 0x7fbe0415ded0 2023-11-08T19:27:27.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.878+0000 7fbe037fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbdf0003bc0 0x7fbe041f2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:27.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.878+0000 7fbe037fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbdf0003bc0 0x7fbe041f2ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53476/0 (socket says 172.21.15.143:53476) 2023-11-08T19:27:27.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.878+0000 7fbe037fe700 1 -- 172.21.15.143:0/3910313297 learned_addr learned my addr 172.21.15.143:0/3910313297 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:27.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.878+0000 7fbe02ffd700 1 --2- 172.21.15.143:0/3910313297 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbe0415ded0 0x7fbe0406d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:27.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.878+0000 7fbe03fff700 1 --2- 172.21.15.143:0/3910313297 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbe0415e350 0x7fbe0406dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:27.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.878+0000 7fbe037fe700 1 -- 172.21.15.143:0/3910313297 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbe0415ded0 msgr2=0x7fbe0406d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:27.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.878+0000 7fbe037fe700 1 --2- 172.21.15.143:0/3910313297 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbe0415ded0 0x7fbe0406d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:27.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.878+0000 7fbe037fe700 1 -- 172.21.15.143:0/3910313297 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbe0415e350 msgr2=0x7fbe0406dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:27.895 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.878+0000 7fbe037fe700 1 --2- 172.21.15.143:0/3910313297 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbe0415e350 0x7fbe0406dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:27.895 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.878+0000 7fbe037fe700 1 -- 172.21.15.143:0/3910313297 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe040cdfa0 con 0x7fbdf0003bc0 2023-11-08T19:27:27.895 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.878+0000 7fbe037fe700 1 --2- 172.21.15.143:0/3910313297 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbdf0003bc0 0x7fbe041f2ef0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fbdf400c800 tx=0x7fbdf4001980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:27.895 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.878+0000 7fbdfbfff700 1 -- 172.21.15.143:0/3910313297 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdf4004d00 con 0x7fbdf0003bc0 2023-11-08T19:27:27.895 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.878+0000 7fbdfbfff700 1 -- 172.21.15.143:0/3910313297 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdf4004e60 con 0x7fbdf0003bc0 2023-11-08T19:27:27.896 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.878+0000 7fbe0a001700 1 -- 172.21.15.143:0/3910313297 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe040ce280 con 0x7fbdf0003bc0 2023-11-08T19:27:27.896 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.878+0000 7fbdfbfff700 1 -- 172.21.15.143:0/3910313297 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdf40072d0 con 0x7fbdf0003bc0 2023-11-08T19:27:27.896 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.878+0000 7fbe0a001700 1 -- 172.21.15.143:0/3910313297 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe041f3430 con 0x7fbdf0003bc0 2023-11-08T19:27:27.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.879+0000 7fbe0a001700 1 -- 172.21.15.143:0/3910313297 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe0404eee0 con 0x7fbdf0003bc0 2023-11-08T19:27:27.902 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.880+0000 7fbdfbfff700 1 -- 172.21.15.143:0/3910313297 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbdf4007430 con 0x7fbdf0003bc0 2023-11-08T19:27:27.902 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.880+0000 7fbdfbfff700 1 --2- 172.21.15.143:0/3910313297 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fbde406af00 0x7fbde406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:27.902 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.880+0000 7fbdfbfff700 1 -- 172.21.15.143:0/3910313297 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5920+0+0 (secure 0 0 0) 0x7fbdf408a8b0 con 0x7fbdf0003bc0 2023-11-08T19:27:27.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.880+0000 7fbe02ffd700 1 --2- 172.21.15.143:0/3910313297 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fbde406af00 0x7fbde406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:27.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.881+0000 7fbe02ffd700 1 --2- 172.21.15.143:0/3910313297 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fbde406af00 0x7fbde406d3c0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fbdec001120 tx=0x7fbdec009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:27.904 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:27.885+0000 7fbdfbfff700 1 -- 172.21.15.143:0/3910313297 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbdf40558a0 con 0x7fbdf0003bc0 2023-11-08T19:27:28.079 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.077+0000 7fbe0a001700 1 -- 172.21.15.143:0/3910313297 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbe04070e70 con 0x7fbdf0003bc0 2023-11-08T19:27:28.080 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.079+0000 7fbdfbfff700 1 -- 172.21.15.143:0/3910313297 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) v1 ==== 74+0+18150 (secure 0 0 0) 0x7fbdf4058ec0 con 0x7fbdf0003bc0 2023-11-08T19:27:28.081 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:27:28.081 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":74,"fsid":"968483af-77fc-4068-ad7c-5c2a1a4613d7","created":"2023-11-08T19:24:32.826484+0000","modified":"2023-11-08T19:27:27.301151+0000","last_up_change":"2023-11-08T19:24:43.573880+0000","last_in_change":"2023-11-08T19:24:35.314525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:44.551530+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"snap-test-pool","create_time":"2023-11-08T19:26:02.874384+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":36,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"test_data_pool","create_time":"2023-11-08T19:27:05.601949+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":66,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_metadata_pool","create_time":"2023-11-08T19:27:06.536369+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bae98ec1-d94e-403d-a220-a522a53ea6f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6809","nonce":1487830566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6811","nonce":1487830566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6815","nonce":1487830566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6813","nonce":1487830566}]},"public_addr":"172.21.15.143:6809/1487830566","cluster_addr":"172.21.15.143:6811/1487830566","heartbeat_back_addr":"172.21.15.143:6815/1487830566","heartbeat_front_addr":"172.21.15.143:6813/1487830566","state":["exists","up"]},{"osd":1,"uuid":"5cb6538a-1ccb-4322-8633-09f66af0b7e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6801","nonce":787341397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6803","nonce":787341397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6807","nonce":787341397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6805","nonce":787341397}]},"public_addr":"172.21.15.143:6801/787341397","cluster_addr":"172.21.15.143:6803/787341397","heartbeat_back_addr":"172.21.15.143:6807/787341397","heartbeat_front_addr":"172.21.15.143:6805/787341397","state":["exists","up"]},{"osd":2,"uuid":"cdbf522c-60ef-4d60-9653-8dcb529c101a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6822","nonce":2301325597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6826","nonce":2301325597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6831","nonce":2301325597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6829","nonce":2301325597}]},"public_addr":"172.21.15.143:6822/2301325597","cluster_addr":"172.21.15.143:6826/2301325597","heartbeat_back_addr":"172.21.15.143:6831/2301325597","heartbeat_front_addr":"172.21.15.143:6829/2301325597","state":["exists","up"]},{"osd":3,"uuid":"4e2783cf-ebb9-4dd8-8245-bc1d17ce4325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6817","nonce":2021504428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6819","nonce":2021504428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6827","nonce":2021504428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6823","nonce":2021504428}]},"public_addr":"172.21.15.143:6817/2021504428","cluster_addr":"172.21.15.143:6819/2021504428","heartbeat_back_addr":"172.21.15.143:6827/2021504428","heartbeat_front_addr":"172.21.15.143:6823/2021504428","state":["exists","up"]},{"osd":4,"uuid":"691256fa-9b77-49b9-a346-7ae1adf0ca80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6801","nonce":1993380595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6803","nonce":1993380595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6807","nonce":1993380595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6805","nonce":1993380595}]},"public_addr":"172.21.15.174:6801/1993380595","cluster_addr":"172.21.15.174:6803/1993380595","heartbeat_back_addr":"172.21.15.174:6807/1993380595","heartbeat_front_addr":"172.21.15.174:6805/1993380595","state":["exists","up"]},{"osd":5,"uuid":"756fc352-48c9-46ce-b52e-0541fe6f8525","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6809","nonce":1777923303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6811","nonce":1777923303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6815","nonce":1777923303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6813","nonce":1777923303}]},"public_addr":"172.21.15.174:6809/1777923303","cluster_addr":"172.21.15.174:6811/1777923303","heartbeat_back_addr":"172.21.15.174:6815/1777923303","heartbeat_front_addr":"172.21.15.174:6813/1777923303","state":["exists","up"]},{"osd":6,"uuid":"fab2ac2b-ed97-4cde-9825-b480eabf8b8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6825","nonce":2755130316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6827","nonce":2755130316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6831","nonce":2755130316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6829","nonce":2755130316}]},"public_addr":"172.21.15.174:6825/2755130316","cluster_addr":"172.21.15.174:6827/2755130316","heartbeat_back_addr":"172.21.15.174:6831/2755130316","heartbeat_front_addr":"172.21.15.174:6829/2755130316","state":["exists","up"]},{"osd":7,"uuid":"3de9b797-2ab6-4d41-a863-7e7d638fd703","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6817","nonce":2847141411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6819","nonce":2847141411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6823","nonce":2847141411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6821","nonce":2847141411}]},"public_addr":"172.21.15.174:6817/2847141411","cluster_addr":"172.21.15.174:6819/2847141411","heartbeat_back_addr":"172.21.15.174:6823/2847141411","heartbeat_front_addr":"172.21.15.174:6821/2847141411","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.477076+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.496890+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.539305+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.454673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.538881+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.521294+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.520712+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.540404+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:27:28.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.081+0000 7fbe0a001700 1 -- 172.21.15.143:0/3910313297 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fbde406af00 msgr2=0x7fbde406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:28.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.081+0000 7fbe0a001700 1 --2- 172.21.15.143:0/3910313297 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fbde406af00 0x7fbde406d3c0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fbdec001120 tx=0x7fbdec009040 comp rx=0 tx=0).stop 2023-11-08T19:27:28.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.081+0000 7fbe0a001700 1 -- 172.21.15.143:0/3910313297 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbdf0003bc0 msgr2=0x7fbe041f2ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:28.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.081+0000 7fbe0a001700 1 --2- 172.21.15.143:0/3910313297 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbdf0003bc0 0x7fbe041f2ef0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fbdf400c800 tx=0x7fbdf4001980 comp rx=0 tx=0).stop 2023-11-08T19:27:28.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.081+0000 7fbe0a001700 1 -- 172.21.15.143:0/3910313297 shutdown_connections 2023-11-08T19:27:28.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.081+0000 7fbe0a001700 1 --2- 172.21.15.143:0/3910313297 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fbde406af00 0x7fbde406d3c0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:28.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.081+0000 7fbe0a001700 1 --2- 172.21.15.143:0/3910313297 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbe0415e350 0x7fbe0406dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:28.085 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.081+0000 7fbe0a001700 1 --2- 172.21.15.143:0/3910313297 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbdf0003bc0 0x7fbe041f2ef0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:28.085 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.081+0000 7fbe0a001700 1 --2- 172.21.15.143:0/3910313297 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbe0415ded0 0x7fbe0406d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:28.085 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.081+0000 7fbe0a001700 1 -- 172.21.15.143:0/3910313297 >> 172.21.15.143:0/3910313297 conn(0x7fbe040b1650 msgr2=0x7fbe0415b580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:28.085 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.081+0000 7fbe0a001700 1 -- 172.21.15.143:0/3910313297 shutdown_connections 2023-11-08T19:27:28.085 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.081+0000 7fbe0a001700 1 -- 172.21.15.143:0/3910313297 wait complete. 2023-11-08T19:27:28.095 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T19:24:44.551530+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'snap-test-pool', 'create_time': '2023-11-08T19:26:02.874384+0000', 'flags': 16385, 'flags_names': 'hashpspool,pool_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '36', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'pool', 'snap_seq': 2, 'snap_epoch': 36, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'test_data_pool', 'create_time': '2023-11-08T19:27:05.601949+0000', 'flags': 16385, 'flags_names': 'hashpspool,pool_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'pool', 'snap_seq': 2, 'snap_epoch': 66, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'test_metadata_pool', 'create_time': '2023-11-08T19:27:06.536369+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-08T19:27:28.096 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T19:27:28.219 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.218+0000 7f9d9c044700 1 Processor -- start 2023-11-08T19:27:28.220 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.218+0000 7f9d9c044700 1 -- start start 2023-11-08T19:27:28.222 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.219+0000 7f9d9c044700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9d9406b0b0 0x7f9d9406c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:28.222 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.219+0000 7f9d9c044700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f9d940b22e0 0x7f9d940b2770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:28.222 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.219+0000 7f9d9c044700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d9406cc50 con 0x7f9d9415bc80 2023-11-08T19:27:28.223 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.219+0000 7f9d9c044700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d940b2cb0 con 0x7f9d940b22e0 2023-11-08T19:27:28.225 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.219+0000 7f9d9c044700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d940ce290 con 0x7f9d9406b0b0 2023-11-08T19:27:28.225 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.219+0000 7f9d9a5e1700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f9d940b22e0 0x7f9d940b2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:28.226 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.219+0000 7f9d9a5e1700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f9d940b22e0 0x7f9d940b2770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53488/0 (socket says 172.21.15.143:53488) 2023-11-08T19:27:28.226 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.219+0000 7f9d9a5e1700 1 -- 172.21.15.143:0/113238325 learned_addr learned my addr 172.21.15.143:0/113238325 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:28.226 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.219+0000 7f9d995df700 1 --2- 172.21.15.143:0/113238325 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9d9406b0b0 0x7f9d9406c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:28.227 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.219+0000 7f9d9a5e1700 1 -- 172.21.15.143:0/113238325 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9d9406b0b0 msgr2=0x7f9d9406c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:28.227 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.219+0000 7f9d9a5e1700 1 --2- 172.21.15.143:0/113238325 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9d9406b0b0 0x7f9d9406c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:28.227 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.219+0000 7f9d9a5e1700 1 -- 172.21.15.143:0/113238325 >> v1:172.21.15.174:6789/0 conn(0x7f9d9415bc80 legacy=0x7f9d940b6280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:28.228 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.219+0000 7f9d9a5e1700 1 -- 172.21.15.143:0/113238325 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d9406b670 con 0x7f9d940b22e0 2023-11-08T19:27:28.228 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.219+0000 7f9d995df700 1 --2- 172.21.15.143:0/113238325 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9d9406b0b0 0x7f9d9406c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:27:28.229 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.219+0000 7f9d9a5e1700 1 --2- 172.21.15.143:0/113238325 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9d940b22e0 0x7f9d940b2770 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c009e80 tx=0x7f9d8c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51c75e5828e9a7eb server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:28.229 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.220+0000 7f9d98dde700 1 -- 172.21.15.143:0/113238325 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d8c00e070 con 0x7f9d940b22e0 2023-11-08T19:27:28.229 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.220+0000 7f9d98dde700 1 -- 172.21.15.143:0/113238325 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d8c00b980 con 0x7f9d940b22e0 2023-11-08T19:27:28.230 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.220+0000 7f9d98dde700 1 -- 172.21.15.143:0/113238325 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d8c00bd30 con 0x7f9d940b22e0 2023-11-08T19:27:28.230 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.220+0000 7f9d9c044700 1 -- 172.21.15.143:0/113238325 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9d940b22e0 msgr2=0x7f9d940b2770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:28.230 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.220+0000 7f9d9c044700 1 --2- 172.21.15.143:0/113238325 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9d940b22e0 0x7f9d940b2770 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c009e80 tx=0x7f9d8c00b330 comp rx=0 tx=0).stop 2023-11-08T19:27:28.230 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.220+0000 7f9d9c044700 1 -- 172.21.15.143:0/113238325 shutdown_connections 2023-11-08T19:27:28.231 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.220+0000 7f9d9c044700 1 --2- 172.21.15.143:0/113238325 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9d940b22e0 0x7f9d940b2770 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:28.231 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.220+0000 7f9d9c044700 1 --2- 172.21.15.143:0/113238325 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9d9406b0b0 0x7f9d9406c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:28.231 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.220+0000 7f9d9c044700 1 -- 172.21.15.143:0/113238325 >> 172.21.15.143:0/113238325 conn(0x7f9d940cd2e0 msgr2=0x7f9d940cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:28.232 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.220+0000 7f9d9c044700 1 -- 172.21.15.143:0/113238325 shutdown_connections 2023-11-08T19:27:28.232 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.220+0000 7f9d9c044700 1 -- 172.21.15.143:0/113238325 wait complete. 2023-11-08T19:27:28.236 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.221+0000 7f9d9c044700 1 Processor -- start 2023-11-08T19:27:28.236 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.221+0000 7f9d9c044700 1 -- start start 2023-11-08T19:27:28.236 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.222+0000 7f9d9c044700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9d9406b0b0 0x7f9d94153b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:28.236 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.222+0000 7f9d9c044700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9d940b22e0 0x7f9d940c6f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:28.237 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.222+0000 7f9d9c044700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9d9415bc80 0x7f9d94156a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:28.237 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.222+0000 7f9d9c044700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d940c5520 con 0x7f9d9415bc80 2023-11-08T19:27:28.237 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.222+0000 7f9d9c044700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d940c5800 con 0x7f9d940b22e0 2023-11-08T19:27:28.237 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.222+0000 7f9d9c044700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d940c5ae0 con 0x7f9d9406b0b0 2023-11-08T19:27:28.237 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.222+0000 7f9d9a5e1700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9d9415bc80 0x7f9d94156a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:28.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.222+0000 7f9d9a5e1700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9d9415bc80 0x7f9d94156a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53494/0 (socket says 172.21.15.143:53494) 2023-11-08T19:27:28.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.222+0000 7f9d9a5e1700 1 -- 172.21.15.143:0/3296903319 learned_addr learned my addr 172.21.15.143:0/3296903319 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:28.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.222+0000 7f9d995df700 1 --2- 172.21.15.143:0/3296903319 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9d940b22e0 0x7f9d940c6f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:28.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.223+0000 7f9d99de0700 1 --2- 172.21.15.143:0/3296903319 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9d9406b0b0 0x7f9d94153b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:28.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.223+0000 7f9d9a5e1700 1 -- 172.21.15.143:0/3296903319 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9d9406b0b0 msgr2=0x7f9d94153b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:28.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.223+0000 7f9d9a5e1700 1 --2- 172.21.15.143:0/3296903319 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9d9406b0b0 0x7f9d94153b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:28.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.223+0000 7f9d9a5e1700 1 -- 172.21.15.143:0/3296903319 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9d940b22e0 msgr2=0x7f9d940c6f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:28.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.223+0000 7f9d9a5e1700 1 --2- 172.21.15.143:0/3296903319 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9d940b22e0 0x7f9d940c6f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:28.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.223+0000 7f9d9a5e1700 1 -- 172.21.15.143:0/3296903319 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d8c009b30 con 0x7f9d9415bc80 2023-11-08T19:27:28.240 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.223+0000 7f9d995df700 1 --2- 172.21.15.143:0/3296903319 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9d940b22e0 0x7f9d940c6f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:27:28.240 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.223+0000 7f9d9a5e1700 1 --2- 172.21.15.143:0/3296903319 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9d9415bc80 0x7f9d94156a30 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f9d9406f460 tx=0x7f9d8c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:28.240 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.223+0000 7f9d8a7fc700 1 -- 172.21.15.143:0/3296903319 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d8c004960 con 0x7f9d9415bc80 2023-11-08T19:27:28.240 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.223+0000 7f9d8a7fc700 1 -- 172.21.15.143:0/3296903319 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d8c0099b0 con 0x7f9d9415bc80 2023-11-08T19:27:28.241 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.223+0000 7f9d9c044700 1 -- 172.21.15.143:0/3296903319 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d940c5dc0 con 0x7f9d9415bc80 2023-11-08T19:27:28.241 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.223+0000 7f9d8a7fc700 1 -- 172.21.15.143:0/3296903319 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d8c01c680 con 0x7f9d9415bc80 2023-11-08T19:27:28.241 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.223+0000 7f9d9c044700 1 -- 172.21.15.143:0/3296903319 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d940c60a0 con 0x7f9d9415bc80 2023-11-08T19:27:28.241 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.224+0000 7f9d9c044700 1 -- 172.21.15.143:0/3296903319 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d9404eee0 con 0x7f9d9415bc80 2023-11-08T19:27:28.241 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.224+0000 7f9d8a7fc700 1 -- 172.21.15.143:0/3296903319 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9d8c01c7e0 con 0x7f9d9415bc80 2023-11-08T19:27:28.242 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.225+0000 7f9d8a7fc700 1 --2- 172.21.15.143:0/3296903319 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f9d7806af00 0x7f9d7806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:28.242 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.225+0000 7f9d8a7fc700 1 -- 172.21.15.143:0/3296903319 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5920+0+0 (secure 0 0 0) 0x7f9d8c08b0c0 con 0x7f9d9415bc80 2023-11-08T19:27:28.242 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.225+0000 7f9d99de0700 1 --2- 172.21.15.143:0/3296903319 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f9d7806af00 0x7f9d7806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:28.242 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.225+0000 7f9d99de0700 1 --2- 172.21.15.143:0/3296903319 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f9d7806af00 0x7f9d7806d3c0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f9d90000f60 tx=0x7f9d90009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:28.243 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.229+0000 7f9d8a7fc700 1 -- 172.21.15.143:0/3296903319 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9d8c059bc0 con 0x7f9d9415bc80 2023-11-08T19:27:28.425 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.423+0000 7f9d9c044700 1 -- 172.21.15.143:0/3296903319 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9d940af810 con 0x7f9d9415bc80 2023-11-08T19:27:28.425 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.423+0000 7f9d8a7fc700 1 -- 172.21.15.143:0/3296903319 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v74) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9d8c059750 con 0x7f9d9415bc80 2023-11-08T19:27:28.425 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2023-11-08T19:27:28.427 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.425+0000 7f9d9c044700 1 -- 172.21.15.143:0/3296903319 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f9d7806af00 msgr2=0x7f9d7806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:28.427 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.425+0000 7f9d9c044700 1 --2- 172.21.15.143:0/3296903319 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f9d7806af00 0x7f9d7806d3c0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f9d90000f60 tx=0x7f9d90009040 comp rx=0 tx=0).stop 2023-11-08T19:27:28.428 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.425+0000 7f9d9c044700 1 -- 172.21.15.143:0/3296903319 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9d9415bc80 msgr2=0x7f9d94156a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:28.428 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.425+0000 7f9d9c044700 1 --2- 172.21.15.143:0/3296903319 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9d9415bc80 0x7f9d94156a30 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f9d9406f460 tx=0x7f9d8c004ad0 comp rx=0 tx=0).stop 2023-11-08T19:27:28.428 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.426+0000 7f9d9c044700 1 -- 172.21.15.143:0/3296903319 shutdown_connections 2023-11-08T19:27:28.428 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.426+0000 7f9d9c044700 1 --2- 172.21.15.143:0/3296903319 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f9d7806af00 0x7f9d7806d3c0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:28.429 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.426+0000 7f9d9c044700 1 --2- 172.21.15.143:0/3296903319 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9d9406b0b0 0x7f9d94153b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:28.429 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.426+0000 7f9d9c044700 1 --2- 172.21.15.143:0/3296903319 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9d9415bc80 0x7f9d94156a30 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:28.429 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.426+0000 7f9d9c044700 1 --2- 172.21.15.143:0/3296903319 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9d940b22e0 0x7f9d940c6f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:28.429 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.426+0000 7f9d9c044700 1 -- 172.21.15.143:0/3296903319 >> 172.21.15.143:0/3296903319 conn(0x7f9d940cd2e0 msgr2=0x7f9d940b1650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:28.430 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.426+0000 7f9d9c044700 1 -- 172.21.15.143:0/3296903319 shutdown_connections 2023-11-08T19:27:28.430 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.426+0000 7f9d9c044700 1 -- 172.21.15.143:0/3296903319 wait complete. 2023-11-08T19:27:28.439 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get snap-test-pool pg_num 2023-11-08T19:27:28.563 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.562+0000 7febb0a08700 1 Processor -- start 2023-11-08T19:27:28.564 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.562+0000 7febb0a08700 1 -- start start 2023-11-08T19:27:28.567 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.563+0000 7febb0a08700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7febac0b61e0 0x7febac0d01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:28.568 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.563+0000 7febb0a08700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febac0cc2a0 con 0x7febac06bd20 2023-11-08T19:27:28.568 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.563+0000 7febb0a08700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febac06c1b0 con 0x7febac15a950 2023-11-08T19:27:28.568 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.563+0000 7febb0a08700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febac0cc130 con 0x7febac0b61e0 2023-11-08T19:27:28.569 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.563+0000 7feba9d9b700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7febac06bd20 0x7febac0cc730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:49654/0 (socket says 172.21.15.143:49654) 2023-11-08T19:27:28.569 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.563+0000 7feba9d9b700 1 -- 172.21.15.143:0/2179794151 learned_addr learned my addr 172.21.15.143:0/2179794151 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:27:28.569 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.563+0000 7febaad9d700 1 --2- 172.21.15.143:0/2179794151 >> v2:172.21.15.174:3300/0 conn(0x7febac0b61e0 0x7febac0d01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:28.569 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.563+0000 7feba959a700 1 -- 172.21.15.143:0/2179794151 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1196248171 0 0) 0x7febac0cc2a0 con 0x7febac06bd20 2023-11-08T19:27:28.570 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.563+0000 7feba959a700 1 -- 172.21.15.143:0/2179794151 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb940035b0 con 0x7febac06bd20 2023-11-08T19:27:28.570 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.563+0000 7feba959a700 1 -- 172.21.15.143:0/2179794151 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2403122713 0 0) 0x7feb940035b0 con 0x7febac06bd20 2023-11-08T19:27:28.570 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.563+0000 7feba959a700 1 -- 172.21.15.143:0/2179794151 >> v2:172.21.15.174:3300/0 conn(0x7febac0b61e0 msgr2=0x7febac0d01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:28.570 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.563+0000 7feba959a700 1 --2- 172.21.15.143:0/2179794151 >> v2:172.21.15.174:3300/0 conn(0x7febac0b61e0 0x7febac0d01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:28.571 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.563+0000 7feba959a700 1 -- 172.21.15.143:0/2179794151 >> v1:172.21.15.174:6789/0 conn(0x7febac15a950 legacy=0x7febac072370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:28.571 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.563+0000 7feba959a700 1 -- 172.21.15.143:0/2179794151 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febac06c490 con 0x7febac06bd20 2023-11-08T19:27:28.571 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.563+0000 7feba959a700 1 -- 172.21.15.143:0/2179794151 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4147756303 0 0) 0x7feba0001670 con 0x7febac06bd20 2023-11-08T19:27:28.572 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.563+0000 7feba959a700 1 -- 172.21.15.143:0/2179794151 >> v1:172.21.15.143:6789/0 conn(0x7febac06bd20 legacy=0x7febac0cc730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:28.572 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.563+0000 7feba959a700 1 --2- 172.21.15.143:0/2179794151 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feb94003a00 0x7feb94003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:28.572 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.563+0000 7feba959a700 1 --2- 172.21.15.143:0/2179794151 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb94004850 0x7feb94004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:28.573 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.563+0000 7feba959a700 1 --2- 172.21.15.143:0/2179794151 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7feb94005210 0x7feb9400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:28.573 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.563+0000 7feba959a700 1 -- 172.21.15.143:0/2179794151 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb9400c080 con 0x7feb94003a00 2023-11-08T19:27:28.573 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.563+0000 7feba959a700 1 -- 172.21.15.143:0/2179794151 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb9400c1f0 con 0x7feb94004850 2023-11-08T19:27:28.574 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.563+0000 7feba959a700 1 -- 172.21.15.143:0/2179794151 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb9400c3b0 con 0x7feb94005210 2023-11-08T19:27:28.574 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.564+0000 7febaa59c700 1 --2- 172.21.15.143:0/2179794151 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feb94003a00 0x7feb94003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:28.574 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.564+0000 7feba9d9b700 1 --2- 172.21.15.143:0/2179794151 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb94004850 0x7feb94004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:28.574 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.564+0000 7febaad9d700 1 --2- 172.21.15.143:0/2179794151 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7feb94005210 0x7feb9400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:28.575 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.564+0000 7febaa59c700 1 -- 172.21.15.143:0/2179794151 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7feb94005210 msgr2=0x7feb9400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:28.575 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.564+0000 7febaa59c700 1 --2- 172.21.15.143:0/2179794151 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7feb94005210 0x7feb9400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:28.575 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.564+0000 7febaa59c700 1 -- 172.21.15.143:0/2179794151 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb94004850 msgr2=0x7feb94004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:28.575 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.564+0000 7febaa59c700 1 --2- 172.21.15.143:0/2179794151 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb94004850 0x7feb94004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:28.576 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.564+0000 7febaa59c700 1 -- 172.21.15.143:0/2179794151 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb940035b0 con 0x7feb94003a00 2023-11-08T19:27:28.576 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.564+0000 7febaad9d700 1 -- 172.21.15.143:0/2179794151 reap_dead start 2023-11-08T19:27:28.576 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.564+0000 7febaa59c700 1 --2- 172.21.15.143:0/2179794151 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feb94003a00 0x7feb94003e20 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7feb9c0017c0 tx=0x7feb94005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a155ea7099d008b3 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:28.577 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.564+0000 7feba959a700 1 -- 172.21.15.143:0/2179794151 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb9c0052d0 con 0x7feb94003a00 2023-11-08T19:27:28.577 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.564+0000 7feba959a700 1 -- 172.21.15.143:0/2179794151 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb9c005430 con 0x7feb94003a00 2023-11-08T19:27:28.577 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.564+0000 7feba959a700 1 -- 172.21.15.143:0/2179794151 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb9c0038c0 con 0x7feb94003a00 2023-11-08T19:27:28.577 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.565+0000 7febb0a08700 1 -- 172.21.15.143:0/2179794151 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feb94003a00 msgr2=0x7feb94003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:28.578 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.565+0000 7febb0a08700 1 --2- 172.21.15.143:0/2179794151 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feb94003a00 0x7feb94003e20 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7feb9c0017c0 tx=0x7feb94005880 comp rx=0 tx=0).stop 2023-11-08T19:27:28.578 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.565+0000 7febb0a08700 1 -- 172.21.15.143:0/2179794151 shutdown_connections 2023-11-08T19:27:28.578 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.565+0000 7febb0a08700 1 --2- 172.21.15.143:0/2179794151 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feb94003a00 0x7feb94003e20 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:28.578 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.565+0000 7febb0a08700 1 -- 172.21.15.143:0/2179794151 >> 172.21.15.143:0/2179794151 conn(0x7febac0cd2e0 msgr2=0x7febac0cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:28.579 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.565+0000 7febb0a08700 1 -- 172.21.15.143:0/2179794151 shutdown_connections 2023-11-08T19:27:28.579 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.565+0000 7febb0a08700 1 -- 172.21.15.143:0/2179794151 wait complete. 2023-11-08T19:27:28.579 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.566+0000 7febb0a08700 1 Processor -- start 2023-11-08T19:27:28.579 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.566+0000 7febb0a08700 1 -- start start 2023-11-08T19:27:28.583 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.567+0000 7febb0a08700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7feb94003a00 0x7febac154c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:28.584 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.567+0000 7febb0a08700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7febac15e830 0x7febac15ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:28.584 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.567+0000 7febb0a08700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7febac0c6150 0x7febac0c65d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:28.584 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.567+0000 7febb0a08700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febac15f1f0 con 0x7febac0c6150 2023-11-08T19:27:28.584 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.567+0000 7febb0a08700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febac0b4a00 con 0x7febac15e830 2023-11-08T19:27:28.585 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.567+0000 7febb0a08700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febac0c6e10 con 0x7feb94003a00 2023-11-08T19:27:28.585 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.567+0000 7febaad9d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7febac0c6150 0x7febac0c65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:28.585 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.567+0000 7febaad9d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7febac0c6150 0x7febac0c65d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53516/0 (socket says 172.21.15.143:53516) 2023-11-08T19:27:28.585 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.567+0000 7febaad9d700 1 -- 172.21.15.143:0/588396066 learned_addr learned my addr 172.21.15.143:0/588396066 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:28.586 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.567+0000 7feba9d9b700 1 --2- 172.21.15.143:0/588396066 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7febac15e830 0x7febac15ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:28.586 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.567+0000 7febaa59c700 1 --2- 172.21.15.143:0/588396066 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7feb94003a00 0x7febac154c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:28.586 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.567+0000 7febaad9d700 1 -- 172.21.15.143:0/588396066 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7feb94003a00 msgr2=0x7febac154c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:28.586 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.567+0000 7febaad9d700 1 --2- 172.21.15.143:0/588396066 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7feb94003a00 0x7febac154c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:28.587 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.567+0000 7febaad9d700 1 -- 172.21.15.143:0/588396066 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7febac15e830 msgr2=0x7febac15ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:28.587 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.567+0000 7febaad9d700 1 --2- 172.21.15.143:0/588396066 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7febac15e830 0x7febac15ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:28.587 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.567+0000 7febaad9d700 1 -- 172.21.15.143:0/588396066 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febac0c70f0 con 0x7febac0c6150 2023-11-08T19:27:28.587 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.568+0000 7febaad9d700 1 --2- 172.21.15.143:0/588396066 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7febac0c6150 0x7febac0c65d0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7febac0cae30 tx=0x7feba4011e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:28.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.568+0000 7feb9affd700 1 -- 172.21.15.143:0/588396066 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba400e040 con 0x7febac0c6150 2023-11-08T19:27:28.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.568+0000 7feb9affd700 1 -- 172.21.15.143:0/588396066 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feba400f040 con 0x7febac0c6150 2023-11-08T19:27:28.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.568+0000 7feb9affd700 1 -- 172.21.15.143:0/588396066 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba400fc70 con 0x7febac0c6150 2023-11-08T19:27:28.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.568+0000 7febb0a08700 1 -- 172.21.15.143:0/588396066 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febac155680 con 0x7febac0c6150 2023-11-08T19:27:28.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.568+0000 7febb0a08700 1 -- 172.21.15.143:0/588396066 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febac155c40 con 0x7febac0c6150 2023-11-08T19:27:28.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.569+0000 7febb0a08700 1 -- 172.21.15.143:0/588396066 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febac04eee0 con 0x7febac0c6150 2023-11-08T19:27:28.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.569+0000 7feb9affd700 1 -- 172.21.15.143:0/588396066 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7feba4022460 con 0x7febac0c6150 2023-11-08T19:27:28.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.569+0000 7feb9affd700 1 --2- 172.21.15.143:0/588396066 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7feb8c06af00 0x7feb8c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:28.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.570+0000 7feb9affd700 1 -- 172.21.15.143:0/588396066 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5920+0+0 (secure 0 0 0) 0x7feba40899c0 con 0x7febac0c6150 2023-11-08T19:27:28.590 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.570+0000 7febaa59c700 1 --2- 172.21.15.143:0/588396066 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7feb8c06af00 0x7feb8c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:28.590 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.570+0000 7febaa59c700 1 --2- 172.21.15.143:0/588396066 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7feb8c06af00 0x7feb8c06d3c0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7feb9c005040 tx=0x7feb9c009030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:28.590 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.574+0000 7feb9affd700 1 -- 172.21.15.143:0/588396066 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7feba40549a0 con 0x7febac0c6150 2023-11-08T19:27:28.764 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.763+0000 7febb0a08700 1 -- 172.21.15.143:0/588396066 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"} v 0) v1 -- 0x7febac0afd20 con 0x7febac0c6150 2023-11-08T19:27:28.765 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.763+0000 7feb9affd700 1 -- 172.21.15.143:0/588396066 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"}]=0 v74) v1 ==== 103+0+11 (secure 0 0 0) 0x7feba4057fc0 con 0x7febac0c6150 2023-11-08T19:27:28.765 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2023-11-08T19:27:28.767 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.765+0000 7febb0a08700 1 -- 172.21.15.143:0/588396066 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7feb8c06af00 msgr2=0x7feb8c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:28.767 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.765+0000 7febb0a08700 1 --2- 172.21.15.143:0/588396066 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7feb8c06af00 0x7feb8c06d3c0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7feb9c005040 tx=0x7feb9c009030 comp rx=0 tx=0).stop 2023-11-08T19:27:28.768 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.765+0000 7febb0a08700 1 -- 172.21.15.143:0/588396066 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7febac0c6150 msgr2=0x7febac0c65d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:28.768 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.765+0000 7febb0a08700 1 --2- 172.21.15.143:0/588396066 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7febac0c6150 0x7febac0c65d0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7febac0cae30 tx=0x7feba4011e20 comp rx=0 tx=0).stop 2023-11-08T19:27:28.768 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.765+0000 7febb0a08700 1 -- 172.21.15.143:0/588396066 shutdown_connections 2023-11-08T19:27:28.768 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.765+0000 7febb0a08700 1 --2- 172.21.15.143:0/588396066 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7feb8c06af00 0x7feb8c06d3c0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:28.769 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.765+0000 7febb0a08700 1 --2- 172.21.15.143:0/588396066 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7feb94003a00 0x7febac154c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:28.769 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.765+0000 7febb0a08700 1 --2- 172.21.15.143:0/588396066 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7febac0c6150 0x7febac0c65d0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:28.769 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.765+0000 7febb0a08700 1 --2- 172.21.15.143:0/588396066 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7febac15e830 0x7febac15ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:28.769 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.765+0000 7febb0a08700 1 -- 172.21.15.143:0/588396066 >> 172.21.15.143:0/588396066 conn(0x7febac0cd2e0 msgr2=0x7febac0ced20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:28.770 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.765+0000 7febb0a08700 1 -- 172.21.15.143:0/588396066 shutdown_connections 2023-11-08T19:27:28.770 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.765+0000 7febb0a08700 1 -- 172.21.15.143:0/588396066 wait complete. 2023-11-08T19:27:28.778 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get test_data_pool pg_num 2023-11-08T19:27:28.901 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.900+0000 7f96a3a8d700 1 Processor -- start 2023-11-08T19:27:28.902 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.900+0000 7f96a3a8d700 1 -- start start 2023-11-08T19:27:28.905 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.901+0000 7f96a3a8d700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f969c15dba0 0x7f969c06c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:28.905 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.901+0000 7f96a3a8d700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f969c06b240 0x7f969c0caff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:28.906 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.901+0000 7f96a3a8d700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f969c161890 con 0x7f969c0cb530 2023-11-08T19:27:28.906 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.901+0000 7f96a3a8d700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f969c06b740 con 0x7f969c15dba0 2023-11-08T19:27:28.906 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.901+0000 7f96a3a8d700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f969c06ba20 con 0x7f969c06b240 2023-11-08T19:27:28.907 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.901+0000 7f96a202a700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f969c0cb530 0x7f969c0b22e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:49674/0 (socket says 172.21.15.143:49674) 2023-11-08T19:27:28.907 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.901+0000 7f96a202a700 1 -- 172.21.15.143:0/1781114128 learned_addr learned my addr 172.21.15.143:0/1781114128 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:27:28.907 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.901+0000 7f96a1829700 1 --2- 172.21.15.143:0/1781114128 >> v2:172.21.15.174:3300/0 conn(0x7f969c15dba0 0x7f969c06c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:28.908 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.901+0000 7f96a1028700 1 --2- 172.21.15.143:0/1781114128 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f969c06b240 0x7f969c0caff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:28.908 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.901+0000 7f96a0827700 1 -- 172.21.15.143:0/1781114128 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3773961719 0 0) 0x7f969c161890 con 0x7f969c0cb530 2023-11-08T19:27:28.908 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.901+0000 7f96a0827700 1 -- 172.21.15.143:0/1781114128 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f96880035b0 con 0x7f969c0cb530 2023-11-08T19:27:28.909 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.901+0000 7f96a0827700 1 -- 172.21.15.143:0/1781114128 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4208340272 0 0) 0x7f96880035b0 con 0x7f969c0cb530 2023-11-08T19:27:28.909 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.901+0000 7f96a1829700 1 -- 172.21.15.143:0/1781114128 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f969c06b240 msgr2=0x7f969c0caff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:28.909 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.901+0000 7f96a1829700 1 --2- 172.21.15.143:0/1781114128 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f969c06b240 0x7f969c0caff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:28.909 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.901+0000 7f96a1829700 1 -- 172.21.15.143:0/1781114128 >> v1:172.21.15.143:6789/0 conn(0x7f969c0cb530 legacy=0x7f969c0b22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:28.910 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.901+0000 7f96a1829700 1 -- 172.21.15.143:0/1781114128 --> v2:172.21.15.174:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f969c06bd00 con 0x7f969c15dba0 2023-11-08T19:27:28.910 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.902+0000 7f96a1829700 1 --2- 172.21.15.143:0/1781114128 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f969c15dba0 0x7f969c06c610 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f968c009840 tx=0x7f968c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b38aee96fe430201 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:28.910 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.902+0000 7f96a0827700 1 -- 172.21.15.143:0/1781114128 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96940019f0 con 0x7f969c15dba0 2023-11-08T19:27:28.910 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.902+0000 7f96a0827700 1 -- 172.21.15.143:0/1781114128 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f968c004030 con 0x7f969c15dba0 2023-11-08T19:27:28.910 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.902+0000 7f96a0827700 1 -- 172.21.15.143:0/1781114128 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f968c0043c0 con 0x7f969c15dba0 2023-11-08T19:27:28.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.902+0000 7f96a3a8d700 1 -- 172.21.15.143:0/1781114128 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f969c15dba0 msgr2=0x7f969c06c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:28.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.902+0000 7f96a3a8d700 1 --2- 172.21.15.143:0/1781114128 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f969c15dba0 0x7f969c06c610 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f968c009840 tx=0x7f968c009b50 comp rx=0 tx=0).stop 2023-11-08T19:27:28.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.902+0000 7f96a3a8d700 1 -- 172.21.15.143:0/1781114128 shutdown_connections 2023-11-08T19:27:28.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.902+0000 7f96a3a8d700 1 --2- 172.21.15.143:0/1781114128 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f969c15dba0 0x7f969c06c610 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:28.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.902+0000 7f96a3a8d700 1 --2- 172.21.15.143:0/1781114128 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f969c06b240 0x7f969c0caff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:28.912 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.902+0000 7f96a3a8d700 1 -- 172.21.15.143:0/1781114128 >> 172.21.15.143:0/1781114128 conn(0x7f969c0cd2e0 msgr2=0x7f969c0cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:28.912 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.902+0000 7f96a3a8d700 1 -- 172.21.15.143:0/1781114128 shutdown_connections 2023-11-08T19:27:28.912 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.903+0000 7f96a3a8d700 1 -- 172.21.15.143:0/1781114128 wait complete. 2023-11-08T19:27:28.912 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.904+0000 7f96a3a8d700 1 Processor -- start 2023-11-08T19:27:28.912 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.904+0000 7f96a3a8d700 1 -- start start 2023-11-08T19:27:28.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.904+0000 7f96a3a8d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f969c06b240 0x7f969c1f7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:28.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.904+0000 7f96a3a8d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f969c0cb530 0x7f969c15ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:28.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.904+0000 7f96a3a8d700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f969c15dba0 0x7f969c1f3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:28.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.904+0000 7f96a3a8d700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f969c1f6d70 con 0x7f969c06b240 2023-11-08T19:27:28.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.904+0000 7f96a3a8d700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f969c1f42b0 con 0x7f969c0cb530 2023-11-08T19:27:28.917 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.904+0000 7f96a3a8d700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f969c1f4590 con 0x7f969c15dba0 2023-11-08T19:27:28.917 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.904+0000 7f96a1829700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f969c06b240 0x7f969c1f7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:28.917 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.904+0000 7f96a1829700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f969c06b240 0x7f969c1f7830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53524/0 (socket says 172.21.15.143:53524) 2023-11-08T19:27:28.917 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.904+0000 7f96a1829700 1 -- 172.21.15.143:0/1634179441 learned_addr learned my addr 172.21.15.143:0/1634179441 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:28.917 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.905+0000 7f96a1028700 1 --2- 172.21.15.143:0/1634179441 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f969c0cb530 0x7f969c15ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:28.918 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.905+0000 7f96a202a700 1 --2- 172.21.15.143:0/1634179441 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f969c15dba0 0x7f969c1f3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:28.918 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.905+0000 7f96a1829700 1 -- 172.21.15.143:0/1634179441 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f969c15dba0 msgr2=0x7f969c1f3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:28.918 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.905+0000 7f96a1829700 1 --2- 172.21.15.143:0/1634179441 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f969c15dba0 0x7f969c1f3750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:28.919 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.905+0000 7f96a1829700 1 -- 172.21.15.143:0/1634179441 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f969c0cb530 msgr2=0x7f969c15ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:28.919 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.905+0000 7f96a1829700 1 --2- 172.21.15.143:0/1634179441 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f969c0cb530 0x7f969c15ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:28.919 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.905+0000 7f96a1829700 1 -- 172.21.15.143:0/1634179441 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f968c0094a0 con 0x7f969c06b240 2023-11-08T19:27:28.919 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.905+0000 7f96a1028700 1 --2- 172.21.15.143:0/1634179441 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f969c0cb530 0x7f969c15ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:27:28.920 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.905+0000 7f96a1829700 1 --2- 172.21.15.143:0/1634179441 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f969c06b240 0x7f969c1f7830 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f968c00a010 tx=0x7f968c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:28.920 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.905+0000 7f96927fc700 1 -- 172.21.15.143:0/1634179441 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f968c003c00 con 0x7f969c06b240 2023-11-08T19:27:28.920 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.905+0000 7f96927fc700 1 -- 172.21.15.143:0/1634179441 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f968c01f070 con 0x7f969c06b240 2023-11-08T19:27:28.921 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.905+0000 7f96a3a8d700 1 -- 172.21.15.143:0/1634179441 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f969c1f7030 con 0x7f969c06b240 2023-11-08T19:27:28.921 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.905+0000 7f96927fc700 1 -- 172.21.15.143:0/1634179441 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f968c011630 con 0x7f969c06b240 2023-11-08T19:27:28.921 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.905+0000 7f96a3a8d700 1 -- 172.21.15.143:0/1634179441 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f969c160e80 con 0x7f969c06b240 2023-11-08T19:27:28.921 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.906+0000 7f96a3a8d700 1 -- 172.21.15.143:0/1634179441 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f969c04eee0 con 0x7f969c06b240 2023-11-08T19:27:28.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.906+0000 7f96927fc700 1 -- 172.21.15.143:0/1634179441 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f968c021030 con 0x7f969c06b240 2023-11-08T19:27:28.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.907+0000 7f96927fc700 1 --2- 172.21.15.143:0/1634179441 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f968006af00 0x7f968006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:28.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.907+0000 7f96927fc700 1 -- 172.21.15.143:0/1634179441 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5920+0+0 (secure 0 0 0) 0x7f968c08aa70 con 0x7f969c06b240 2023-11-08T19:27:28.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.907+0000 7f96a1028700 1 --2- 172.21.15.143:0/1634179441 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f968006af00 0x7f968006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:28.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.907+0000 7f96a1028700 1 --2- 172.21.15.143:0/1634179441 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f968006af00 0x7f968006d3c0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f969c15b430 tx=0x7f9698008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:28.923 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:28.911+0000 7f96927fc700 1 -- 172.21.15.143:0/1634179441 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f968c055c00 con 0x7f969c06b240 2023-11-08T19:27:29.099 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.098+0000 7f96a3a8d700 1 -- 172.21.15.143:0/1634179441 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "test_data_pool", "var": "pg_num"} v 0) v1 -- 0x7f969c0b67e0 con 0x7f969c06b240 2023-11-08T19:27:29.099 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.098+0000 7f96927fc700 1 -- 172.21.15.143:0/1634179441 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "test_data_pool", "var": "pg_num"}]=0 v74) v1 ==== 103+0+11 (secure 0 0 0) 0x7f968c059220 con 0x7f969c06b240 2023-11-08T19:27:29.100 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2023-11-08T19:27:29.102 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.100+0000 7f96a3a8d700 1 -- 172.21.15.143:0/1634179441 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f968006af00 msgr2=0x7f968006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:29.102 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.100+0000 7f96a3a8d700 1 --2- 172.21.15.143:0/1634179441 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f968006af00 0x7f968006d3c0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f969c15b430 tx=0x7f9698008040 comp rx=0 tx=0).stop 2023-11-08T19:27:29.102 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.100+0000 7f96a3a8d700 1 -- 172.21.15.143:0/1634179441 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f969c06b240 msgr2=0x7f969c1f7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:29.102 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.100+0000 7f96a3a8d700 1 --2- 172.21.15.143:0/1634179441 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f969c06b240 0x7f969c1f7830 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f968c00a010 tx=0x7f968c004750 comp rx=0 tx=0).stop 2023-11-08T19:27:29.103 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.100+0000 7f96a3a8d700 1 -- 172.21.15.143:0/1634179441 shutdown_connections 2023-11-08T19:27:29.103 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.100+0000 7f96a3a8d700 1 --2- 172.21.15.143:0/1634179441 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f968006af00 0x7f968006d3c0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:29.103 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.100+0000 7f96a3a8d700 1 --2- 172.21.15.143:0/1634179441 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f969c15dba0 0x7f969c1f3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:29.103 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.100+0000 7f96a3a8d700 1 --2- 172.21.15.143:0/1634179441 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f969c06b240 0x7f969c1f7830 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:29.104 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.100+0000 7f96a3a8d700 1 --2- 172.21.15.143:0/1634179441 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f969c0cb530 0x7f969c15ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:29.104 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.100+0000 7f96a3a8d700 1 -- 172.21.15.143:0/1634179441 >> 172.21.15.143:0/1634179441 conn(0x7f969c0cd2e0 msgr2=0x7f969c0b08a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:29.104 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.100+0000 7f96a3a8d700 1 -- 172.21.15.143:0/1634179441 shutdown_connections 2023-11-08T19:27:29.105 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.100+0000 7f96a3a8d700 1 -- 172.21.15.143:0/1634179441 wait complete. 2023-11-08T19:27:29.114 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get test_metadata_pool pg_num 2023-11-08T19:27:29.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.237+0000 7fd0910cd700 1 Processor -- start 2023-11-08T19:27:29.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.237+0000 7fd0910cd700 1 -- start start 2023-11-08T19:27:29.241 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.237+0000 7fd0910cd700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fd08c15dba0 0x7fd08c06c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:29.242 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.237+0000 7fd0910cd700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd08c0b22e0 0x7fd08c0b2750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:29.242 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.238+0000 7fd0910cd700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd08c161890 con 0x7fd08c06b240 2023-11-08T19:27:29.242 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.238+0000 7fd0910cd700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd08c0b2c90 con 0x7fd08c15dba0 2023-11-08T19:27:29.243 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.238+0000 7fd0910cd700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd08c06b740 con 0x7fd08c0b22e0 2023-11-08T19:27:29.243 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.238+0000 7fd08a59c700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fd08c06b240 0x7fd08c0caff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:49686/0 (socket says 172.21.15.143:49686) 2023-11-08T19:27:29.243 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.238+0000 7fd08a59c700 1 -- 172.21.15.143:0/3914713896 learned_addr learned my addr 172.21.15.143:0/3914713896 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:27:29.243 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.238+0000 7fd08ad9d700 1 --2- 172.21.15.143:0/3914713896 >> v2:172.21.15.174:3300/0 conn(0x7fd08c15dba0 0x7fd08c06c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:29.243 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.238+0000 7fd08b59e700 1 --2- 172.21.15.143:0/3914713896 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd08c0b22e0 0x7fd08c0b2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:29.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.238+0000 7fd089d9b700 1 -- 172.21.15.143:0/3914713896 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3615849085 0 0) 0x7fd08c161890 con 0x7fd08c06b240 2023-11-08T19:27:29.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.238+0000 7fd089d9b700 1 -- 172.21.15.143:0/3914713896 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd0740035b0 con 0x7fd08c06b240 2023-11-08T19:27:29.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.238+0000 7fd089d9b700 1 -- 172.21.15.143:0/3914713896 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4173669318 0 0) 0x7fd0740035b0 con 0x7fd08c06b240 2023-11-08T19:27:29.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.238+0000 7fd08ad9d700 1 -- 172.21.15.143:0/3914713896 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd08c0b22e0 msgr2=0x7fd08c0b2750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:29.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.238+0000 7fd08ad9d700 1 --2- 172.21.15.143:0/3914713896 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd08c0b22e0 0x7fd08c0b2750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:29.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.238+0000 7fd08ad9d700 1 -- 172.21.15.143:0/3914713896 >> v1:172.21.15.143:6789/0 conn(0x7fd08c06b240 legacy=0x7fd08c0caff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:29.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.238+0000 7fd08ad9d700 1 -- 172.21.15.143:0/3914713896 --> v2:172.21.15.174:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd08c06ba20 con 0x7fd08c15dba0 2023-11-08T19:27:29.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.238+0000 7fd08b59e700 1 --2- 172.21.15.143:0/3914713896 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd08c0b22e0 0x7fd08c0b2750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T19:27:29.246 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.238+0000 7fd08ad9d700 1 --2- 172.21.15.143:0/3914713896 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd08c15dba0 0x7fd08c06c610 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fd07c009840 tx=0x7fd07c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa5d6e438e7125f4 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:29.246 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.239+0000 7fd089d9b700 1 -- 172.21.15.143:0/3914713896 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd080001670 con 0x7fd08c15dba0 2023-11-08T19:27:29.246 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.239+0000 7fd089d9b700 1 -- 172.21.15.143:0/3914713896 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd07c004030 con 0x7fd08c15dba0 2023-11-08T19:27:29.246 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.239+0000 7fd089d9b700 1 -- 172.21.15.143:0/3914713896 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd07c0043c0 con 0x7fd08c15dba0 2023-11-08T19:27:29.246 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.239+0000 7fd0910cd700 1 -- 172.21.15.143:0/3914713896 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd08c15dba0 msgr2=0x7fd08c06c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:29.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.239+0000 7fd0910cd700 1 --2- 172.21.15.143:0/3914713896 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd08c15dba0 0x7fd08c06c610 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fd07c009840 tx=0x7fd07c009b50 comp rx=0 tx=0).stop 2023-11-08T19:27:29.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.239+0000 7fd0910cd700 1 -- 172.21.15.143:0/3914713896 shutdown_connections 2023-11-08T19:27:29.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.239+0000 7fd0910cd700 1 --2- 172.21.15.143:0/3914713896 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd08c0b22e0 0x7fd08c0b2750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:29.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.239+0000 7fd0910cd700 1 --2- 172.21.15.143:0/3914713896 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd08c15dba0 0x7fd08c06c610 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:29.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.239+0000 7fd0910cd700 1 -- 172.21.15.143:0/3914713896 >> 172.21.15.143:0/3914713896 conn(0x7fd08c0cd2e0 msgr2=0x7fd08c0cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:29.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.239+0000 7fd0910cd700 1 -- 172.21.15.143:0/3914713896 shutdown_connections 2023-11-08T19:27:29.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.239+0000 7fd0910cd700 1 -- 172.21.15.143:0/3914713896 wait complete. 2023-11-08T19:27:29.251 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.241+0000 7fd0910cd700 1 Processor -- start 2023-11-08T19:27:29.252 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.241+0000 7fd0910cd700 1 -- start start 2023-11-08T19:27:29.252 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.241+0000 7fd0910cd700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd08c06b240 0x7fd08c15f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:29.253 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.241+0000 7fd0910cd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd08c0b22e0 0x7fd08c160bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:29.253 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.241+0000 7fd0910cd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd08c15dba0 0x7fd08c153270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:29.253 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.241+0000 7fd0910cd700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd08c161110 con 0x7fd08c15dba0 2023-11-08T19:27:29.253 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.241+0000 7fd0910cd700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd08c0b3720 con 0x7fd08c0b22e0 2023-11-08T19:27:29.254 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.241+0000 7fd0910cd700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd08c15fd80 con 0x7fd08c06b240 2023-11-08T19:27:29.254 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.241+0000 7fd08b59e700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd08c15dba0 0x7fd08c153270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:29.254 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.241+0000 7fd08b59e700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd08c15dba0 0x7fd08c153270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53542/0 (socket says 172.21.15.143:53542) 2023-11-08T19:27:29.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.242+0000 7fd08b59e700 1 -- 172.21.15.143:0/3004794644 learned_addr learned my addr 172.21.15.143:0/3004794644 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:29.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.242+0000 7fd08ad9d700 1 --2- 172.21.15.143:0/3004794644 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd08c06b240 0x7fd08c15f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:29.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.242+0000 7fd08a59c700 1 --2- 172.21.15.143:0/3004794644 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd08c0b22e0 0x7fd08c160bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:29.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.242+0000 7fd08b59e700 1 -- 172.21.15.143:0/3004794644 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd08c06b240 msgr2=0x7fd08c15f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:29.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.242+0000 7fd08b59e700 1 --2- 172.21.15.143:0/3004794644 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd08c06b240 0x7fd08c15f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:29.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.242+0000 7fd08b59e700 1 -- 172.21.15.143:0/3004794644 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd08c0b22e0 msgr2=0x7fd08c160bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:29.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.242+0000 7fd08b59e700 1 --2- 172.21.15.143:0/3004794644 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd08c0b22e0 0x7fd08c160bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:29.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.242+0000 7fd08b59e700 1 -- 172.21.15.143:0/3004794644 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd084009a70 con 0x7fd08c15dba0 2023-11-08T19:27:29.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.242+0000 7fd08b59e700 1 --2- 172.21.15.143:0/3004794644 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd08c15dba0 0x7fd08c153270 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fd08400efc0 tx=0x7fd08400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:29.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.242+0000 7fd0737fe700 1 -- 172.21.15.143:0/3004794644 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd08400ca90 con 0x7fd08c15dba0 2023-11-08T19:27:29.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.242+0000 7fd0737fe700 1 -- 172.21.15.143:0/3004794644 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd08400f040 con 0x7fd08c15dba0 2023-11-08T19:27:29.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.242+0000 7fd0910cd700 1 -- 172.21.15.143:0/3004794644 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd07c0094a0 con 0x7fd08c15dba0 2023-11-08T19:27:29.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.242+0000 7fd0737fe700 1 -- 172.21.15.143:0/3004794644 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd08400ce10 con 0x7fd08c15dba0 2023-11-08T19:27:29.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.242+0000 7fd0910cd700 1 -- 172.21.15.143:0/3004794644 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd08c0b3cc0 con 0x7fd08c15dba0 2023-11-08T19:27:29.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.243+0000 7fd0910cd700 1 -- 172.21.15.143:0/3004794644 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd08c04eee0 con 0x7fd08c15dba0 2023-11-08T19:27:29.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.244+0000 7fd0737fe700 1 -- 172.21.15.143:0/3004794644 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd084014480 con 0x7fd08c15dba0 2023-11-08T19:27:29.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.244+0000 7fd0737fe700 1 --2- 172.21.15.143:0/3004794644 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd07806af00 0x7fd07806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:29.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.244+0000 7fd0737fe700 1 -- 172.21.15.143:0/3004794644 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5920+0+0 (secure 0 0 0) 0x7fd084089d20 con 0x7fd08c15dba0 2023-11-08T19:27:29.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.244+0000 7fd08ad9d700 1 --2- 172.21.15.143:0/3004794644 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd07806af00 0x7fd07806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:29.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.245+0000 7fd08ad9d700 1 --2- 172.21.15.143:0/3004794644 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd07806af00 0x7fd07806d3c0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fd07c0090b0 tx=0x7fd07c008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:29.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.248+0000 7fd0737fe700 1 -- 172.21.15.143:0/3004794644 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd084054d00 con 0x7fd08c15dba0 2023-11-08T19:27:29.441 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.439+0000 7fd0910cd700 1 -- 172.21.15.143:0/3004794644 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "test_metadata_pool", "var": "pg_num"} v 0) v1 -- 0x7fd08c0b67e0 con 0x7fd08c15dba0 2023-11-08T19:27:29.441 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.439+0000 7fd0737fe700 1 -- 172.21.15.143:0/3004794644 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "test_metadata_pool", "var": "pg_num"}]=0 v74) v1 ==== 107+0+11 (secure 0 0 0) 0x7fd084009d70 con 0x7fd08c15dba0 2023-11-08T19:27:29.442 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2023-11-08T19:27:29.444 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.442+0000 7fd0910cd700 1 -- 172.21.15.143:0/3004794644 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd07806af00 msgr2=0x7fd07806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:29.444 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.442+0000 7fd0910cd700 1 --2- 172.21.15.143:0/3004794644 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd07806af00 0x7fd07806d3c0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fd07c0090b0 tx=0x7fd07c008030 comp rx=0 tx=0).stop 2023-11-08T19:27:29.444 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.442+0000 7fd0910cd700 1 -- 172.21.15.143:0/3004794644 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd08c15dba0 msgr2=0x7fd08c153270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:29.444 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.442+0000 7fd0910cd700 1 --2- 172.21.15.143:0/3004794644 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd08c15dba0 0x7fd08c153270 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fd08400efc0 tx=0x7fd08400c3a0 comp rx=0 tx=0).stop 2023-11-08T19:27:29.445 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.442+0000 7fd0910cd700 1 -- 172.21.15.143:0/3004794644 shutdown_connections 2023-11-08T19:27:29.445 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.442+0000 7fd0910cd700 1 --2- 172.21.15.143:0/3004794644 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd07806af00 0x7fd07806d3c0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:29.445 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.442+0000 7fd0910cd700 1 --2- 172.21.15.143:0/3004794644 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd08c06b240 0x7fd08c15f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:29.445 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.442+0000 7fd0910cd700 1 --2- 172.21.15.143:0/3004794644 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd08c15dba0 0x7fd08c153270 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:29.445 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.442+0000 7fd0910cd700 1 --2- 172.21.15.143:0/3004794644 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd08c0b22e0 0x7fd08c160bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:29.446 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.442+0000 7fd0910cd700 1 -- 172.21.15.143:0/3004794644 >> 172.21.15.143:0/3004794644 conn(0x7fd08c0cd2e0 msgr2=0x7fd08c0b1220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:29.446 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.442+0000 7fd0910cd700 1 -- 172.21.15.143:0/3004794644 shutdown_connections 2023-11-08T19:27:29.446 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.442+0000 7fd0910cd700 1 -- 172.21.15.143:0/3004794644 wait complete. 2023-11-08T19:27:29.456 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-08T19:27:29.577 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.575+0000 7fd723f4a700 1 Processor -- start 2023-11-08T19:27:29.577 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.576+0000 7fd723f4a700 1 -- start start 2023-11-08T19:27:29.579 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.576+0000 7fd723f4a700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fd71c06c030 0x7fd71c0cd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:29.580 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.576+0000 7fd723f4a700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd71c15de80 0x7fd71c0cbf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:29.580 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.576+0000 7fd723f4a700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd71c0cd480 con 0x7fd71c06c730 2023-11-08T19:27:29.580 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.576+0000 7fd723f4a700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd71c0cc870 con 0x7fd71c06c030 2023-11-08T19:27:29.580 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.576+0000 7fd723f4a700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd71c0ccb50 con 0x7fd71c15de80 2023-11-08T19:27:29.580 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.577+0000 7fd7224e7700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd71c15de80 0x7fd71c0cbf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:29.581 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.577+0000 7fd7214e5700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fd71c06c030 0x7fd71c0cd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:29.581 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.577+0000 7fd7224e7700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd71c15de80 0x7fd71c0cbf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.143:41188/0 (socket says 172.21.15.143:41188) 2023-11-08T19:27:29.581 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.577+0000 7fd7214e5700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fd71c06c030 0x7fd71c0cd910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.143:37044/0 (socket says 172.21.15.143:37044) 2023-11-08T19:27:29.581 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.577+0000 7fd7224e7700 1 -- 172.21.15.143:0/1402022286 learned_addr learned my addr 172.21.15.143:0/1402022286 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:29.581 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.577+0000 7fd7224e7700 1 -- 172.21.15.143:0/1402022286 >> v2:172.21.15.174:3300/0 conn(0x7fd71c06c030 msgr2=0x7fd71c0cd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:29.582 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.577+0000 7fd7224e7700 1 --2- 172.21.15.143:0/1402022286 >> v2:172.21.15.174:3300/0 conn(0x7fd71c06c030 0x7fd71c0cd910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:29.582 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.577+0000 7fd7224e7700 1 -- 172.21.15.143:0/1402022286 >> v1:172.21.15.174:6789/0 conn(0x7fd71c06c730 legacy=0x7fd71c06cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:29.582 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.577+0000 7fd7224e7700 1 -- 172.21.15.143:0/1402022286 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd71c0cce30 con 0x7fd71c15de80 2023-11-08T19:27:29.582 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.577+0000 7fd7214e5700 1 --2- 172.21.15.143:0/1402022286 >> v2:172.21.15.174:3300/0 conn(0x7fd71c06c030 0x7fd71c0cd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T19:27:29.582 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.577+0000 7fd7224e7700 1 --2- 172.21.15.143:0/1402022286 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd71c15de80 0x7fd71c0cbf00 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd714009e50 tx=0x7fd71400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa7c79897a2f3a50 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:29.583 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.578+0000 7fd720ce4700 1 -- 172.21.15.143:0/1402022286 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd71400e070 con 0x7fd71c15de80 2023-11-08T19:27:29.583 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.578+0000 7fd720ce4700 1 -- 172.21.15.143:0/1402022286 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd71400b910 con 0x7fd71c15de80 2023-11-08T19:27:29.583 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.578+0000 7fd720ce4700 1 -- 172.21.15.143:0/1402022286 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd71400bcc0 con 0x7fd71c15de80 2023-11-08T19:27:29.583 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.578+0000 7fd723f4a700 1 -- 172.21.15.143:0/1402022286 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd71c15de80 msgr2=0x7fd71c0cbf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:29.583 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.578+0000 7fd723f4a700 1 --2- 172.21.15.143:0/1402022286 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd71c15de80 0x7fd71c0cbf00 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd714009e50 tx=0x7fd71400b330 comp rx=0 tx=0).stop 2023-11-08T19:27:29.584 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.578+0000 7fd723f4a700 1 -- 172.21.15.143:0/1402022286 shutdown_connections 2023-11-08T19:27:29.584 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.578+0000 7fd723f4a700 1 --2- 172.21.15.143:0/1402022286 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd71c15de80 0x7fd71c0cbf00 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:29.584 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.578+0000 7fd723f4a700 1 --2- 172.21.15.143:0/1402022286 >> v2:172.21.15.174:3300/0 conn(0x7fd71c06c030 0x7fd71c0cd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:29.584 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.578+0000 7fd723f4a700 1 -- 172.21.15.143:0/1402022286 >> 172.21.15.143:0/1402022286 conn(0x7fd71c08b000 msgr2=0x7fd71c09ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:29.584 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.578+0000 7fd723f4a700 1 -- 172.21.15.143:0/1402022286 shutdown_connections 2023-11-08T19:27:29.585 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.578+0000 7fd723f4a700 1 -- 172.21.15.143:0/1402022286 wait complete. 2023-11-08T19:27:29.587 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.579+0000 7fd723f4a700 1 Processor -- start 2023-11-08T19:27:29.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.579+0000 7fd723f4a700 1 -- start start 2023-11-08T19:27:29.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.580+0000 7fd723f4a700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd71c06c030 0x7fd71c15d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:29.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.580+0000 7fd723f4a700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd71c06c730 0x7fd71c159b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:29.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.580+0000 7fd723f4a700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd71c15de80 0x7fd71c14bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:29.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.580+0000 7fd723f4a700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd71c15ac80 con 0x7fd71c06c030 2023-11-08T19:27:29.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.580+0000 7fd723f4a700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd71c14d430 con 0x7fd71c15de80 2023-11-08T19:27:29.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.580+0000 7fd723f4a700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd71c15a070 con 0x7fd71c06c730 2023-11-08T19:27:29.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.580+0000 7fd721ce6700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd71c06c030 0x7fd71c15d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:29.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.580+0000 7fd721ce6700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd71c06c030 0x7fd71c15d330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53552/0 (socket says 172.21.15.143:53552) 2023-11-08T19:27:29.590 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.580+0000 7fd721ce6700 1 -- 172.21.15.143:0/2919762615 learned_addr learned my addr 172.21.15.143:0/2919762615 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:29.590 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.580+0000 7fd7224e7700 1 --2- 172.21.15.143:0/2919762615 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd71c15de80 0x7fd71c14bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:29.590 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.580+0000 7fd7214e5700 1 --2- 172.21.15.143:0/2919762615 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd71c06c730 0x7fd71c159b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:29.590 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.581+0000 7fd721ce6700 1 -- 172.21.15.143:0/2919762615 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd71c06c730 msgr2=0x7fd71c159b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:29.590 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.581+0000 7fd721ce6700 1 --2- 172.21.15.143:0/2919762615 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd71c06c730 0x7fd71c159b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:29.590 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.581+0000 7fd721ce6700 1 -- 172.21.15.143:0/2919762615 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd71c15de80 msgr2=0x7fd71c14bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:29.591 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.581+0000 7fd721ce6700 1 --2- 172.21.15.143:0/2919762615 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd71c15de80 0x7fd71c14bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:29.591 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.581+0000 7fd721ce6700 1 -- 172.21.15.143:0/2919762615 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7180094a0 con 0x7fd71c06c030 2023-11-08T19:27:29.591 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.581+0000 7fd721ce6700 1 --2- 172.21.15.143:0/2919762615 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd71c06c030 0x7fd71c15d330 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fd70c00c510 tx=0x7fd70c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:29.591 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.581+0000 7fd7127fc700 1 -- 172.21.15.143:0/2919762615 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd70c0052d0 con 0x7fd71c06c030 2023-11-08T19:27:29.591 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.581+0000 7fd7127fc700 1 -- 172.21.15.143:0/2919762615 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd70c005430 con 0x7fd71c06c030 2023-11-08T19:27:29.592 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.581+0000 7fd723f4a700 1 -- 172.21.15.143:0/2919762615 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd714009b30 con 0x7fd71c06c030 2023-11-08T19:27:29.592 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.581+0000 7fd7127fc700 1 -- 172.21.15.143:0/2919762615 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd70c0038c0 con 0x7fd71c06c030 2023-11-08T19:27:29.592 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.581+0000 7fd723f4a700 1 -- 172.21.15.143:0/2919762615 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd71c15a9a0 con 0x7fd71c06c030 2023-11-08T19:27:29.592 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.582+0000 7fd723f4a700 1 -- 172.21.15.143:0/2919762615 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd71c04eee0 con 0x7fd71c06c030 2023-11-08T19:27:29.592 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.582+0000 7fd7127fc700 1 -- 172.21.15.143:0/2919762615 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd70c003a20 con 0x7fd71c06c030 2023-11-08T19:27:29.593 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.583+0000 7fd7127fc700 1 --2- 172.21.15.143:0/2919762615 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd70006af30 0x7fd70006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:29.593 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.583+0000 7fd7127fc700 1 -- 172.21.15.143:0/2919762615 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5920+0+0 (secure 0 0 0) 0x7fd70c088b00 con 0x7fd71c06c030 2023-11-08T19:27:29.593 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.583+0000 7fd7214e5700 1 --2- 172.21.15.143:0/2919762615 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd70006af30 0x7fd70006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:29.593 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.583+0000 7fd7214e5700 1 --2- 172.21.15.143:0/2919762615 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd70006af30 0x7fd70006d3f0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fd71c15bf40 tx=0x7fd718008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:29.594 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.587+0000 7fd7127fc700 1 -- 172.21.15.143:0/2919762615 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd70c053b70 con 0x7fd71c06c030 2023-11-08T19:27:29.781 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.780+0000 7fd723f4a700 1 -- 172.21.15.143:0/2919762615 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fd71c06f3b0 con 0x7fd71c06c030 2023-11-08T19:27:29.782 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.780+0000 7fd7127fc700 1 -- 172.21.15.143:0/2919762615 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v74) v1 ==== 84+0+287 (secure 0 0 0) 0x7fd70c017090 con 0x7fd71c06c030 2023-11-08T19:27:29.782 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:27:29.782 INFO:teuthology.orchestra.run.smithi143.stdout:[ 2023-11-08T19:27:29.782 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-08T19:27:29.783 INFO:teuthology.orchestra.run.smithi143.stdout: "poolnum": 1, 2023-11-08T19:27:29.783 INFO:teuthology.orchestra.run.smithi143.stdout: "poolname": ".mgr" 2023-11-08T19:27:29.783 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-08T19:27:29.783 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-08T19:27:29.784 INFO:teuthology.orchestra.run.smithi143.stdout: "poolnum": 6, 2023-11-08T19:27:29.784 INFO:teuthology.orchestra.run.smithi143.stdout: "poolname": "snap-test-pool" 2023-11-08T19:27:29.784 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-08T19:27:29.784 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-08T19:27:29.784 INFO:teuthology.orchestra.run.smithi143.stdout: "poolnum": 11, 2023-11-08T19:27:29.784 INFO:teuthology.orchestra.run.smithi143.stdout: "poolname": "test_data_pool" 2023-11-08T19:27:29.785 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-08T19:27:29.785 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-08T19:27:29.785 INFO:teuthology.orchestra.run.smithi143.stdout: "poolnum": 12, 2023-11-08T19:27:29.785 INFO:teuthology.orchestra.run.smithi143.stdout: "poolname": "test_metadata_pool" 2023-11-08T19:27:29.785 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-08T19:27:29.786 INFO:teuthology.orchestra.run.smithi143.stdout:] 2023-11-08T19:27:29.787 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.782+0000 7fd723f4a700 1 -- 172.21.15.143:0/2919762615 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd70006af30 msgr2=0x7fd70006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:29.787 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.782+0000 7fd723f4a700 1 --2- 172.21.15.143:0/2919762615 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd70006af30 0x7fd70006d3f0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fd71c15bf40 tx=0x7fd718008040 comp rx=0 tx=0).stop 2023-11-08T19:27:29.788 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.782+0000 7fd723f4a700 1 -- 172.21.15.143:0/2919762615 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd71c06c030 msgr2=0x7fd71c15d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:29.788 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.782+0000 7fd723f4a700 1 --2- 172.21.15.143:0/2919762615 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd71c06c030 0x7fd71c15d330 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fd70c00c510 tx=0x7fd70c00c8d0 comp rx=0 tx=0).stop 2023-11-08T19:27:29.788 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.782+0000 7fd723f4a700 1 -- 172.21.15.143:0/2919762615 shutdown_connections 2023-11-08T19:27:29.788 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.782+0000 7fd723f4a700 1 --2- 172.21.15.143:0/2919762615 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd70006af30 0x7fd70006d3f0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:29.788 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.782+0000 7fd723f4a700 1 --2- 172.21.15.143:0/2919762615 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd71c15de80 0x7fd71c14bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:29.789 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.782+0000 7fd723f4a700 1 --2- 172.21.15.143:0/2919762615 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd71c06c030 0x7fd71c15d330 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:29.789 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.782+0000 7fd723f4a700 1 --2- 172.21.15.143:0/2919762615 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd71c06c730 0x7fd71c159b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:29.789 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.782+0000 7fd723f4a700 1 -- 172.21.15.143:0/2919762615 >> 172.21.15.143:0/2919762615 conn(0x7fd71c08b000 msgr2=0x7fd71c0ca990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:29.789 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.782+0000 7fd723f4a700 1 -- 172.21.15.143:0/2919762615 shutdown_connections 2023-11-08T19:27:29.789 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.782+0000 7fd723f4a700 1 -- 172.21.15.143:0/2919762615 wait complete. 2023-11-08T19:27:29.796 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-08T19:27:29.796 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-08T19:27:29.921 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.920+0000 7f2fbdceb700 1 Processor -- start 2023-11-08T19:27:29.921 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.920+0000 7f2fbdceb700 1 -- start start 2023-11-08T19:27:29.924 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.920+0000 7f2fbdceb700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f2fb815dba0 0x7f2fb806c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:29.924 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.920+0000 7f2fbdceb700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2fb806b240 0x7f2fb80caff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:29.925 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.920+0000 7f2fbdceb700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f2fb80cb530 0x7f2fb80b22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:29.925 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.920+0000 7f2fbdceb700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fb8161890 con 0x7f2fb815dba0 2023-11-08T19:27:29.925 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.920+0000 7f2fbdceb700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fb806b740 con 0x7f2fb80cb530 2023-11-08T19:27:29.925 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.920+0000 7f2fbdceb700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb806ba20 con 0x7f2fb806b240 2023-11-08T19:27:29.925 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.921+0000 7f2fb77fe700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f2fb815dba0 0x7f2fb806c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:29.926 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.921+0000 7f2fb77fe700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f2fb815dba0 0x7f2fb806c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53560/0 (socket says 172.21.15.143:53560) 2023-11-08T19:27:29.926 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.921+0000 7f2fb77fe700 1 -- 172.21.15.143:0/2898566865 learned_addr learned my addr 172.21.15.143:0/2898566865 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:29.926 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.921+0000 7f2fb7fff700 1 --2- 172.21.15.143:0/2898566865 >> v2:172.21.15.174:3300/0 conn(0x7f2fb80cb530 0x7f2fb80b22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:29.926 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.921+0000 7f2fb6ffd700 1 --2- 172.21.15.143:0/2898566865 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2fb806b240 0x7f2fb80caff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:29.926 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.921+0000 7f2fb77fe700 1 -- 172.21.15.143:0/2898566865 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2fb806b240 msgr2=0x7f2fb80caff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:29.927 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.921+0000 7f2fb77fe700 1 --2- 172.21.15.143:0/2898566865 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2fb806b240 0x7f2fb80caff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:29.927 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.921+0000 7f2fb77fe700 1 -- 172.21.15.143:0/2898566865 >> v2:172.21.15.174:3300/0 conn(0x7f2fb80cb530 msgr2=0x7f2fb80b22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:29.927 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.921+0000 7f2fb77fe700 1 --2- 172.21.15.143:0/2898566865 >> v2:172.21.15.174:3300/0 conn(0x7f2fb80cb530 0x7f2fb80b22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:29.927 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.921+0000 7f2fb77fe700 1 -- 172.21.15.143:0/2898566865 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fb806bd00 con 0x7f2fb815dba0 2023-11-08T19:27:29.927 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.921+0000 7f2fb7fff700 1 --2- 172.21.15.143:0/2898566865 >> v2:172.21.15.174:3300/0 conn(0x7f2fb80cb530 0x7f2fb80b22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:27:29.928 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.921+0000 7f2fb6ffd700 1 --2- 172.21.15.143:0/2898566865 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2fb806b240 0x7f2fb80caff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:27:29.928 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.921+0000 7f2fb77fe700 1 --2- 172.21.15.143:0/2898566865 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2fb815dba0 0x7f2fb806c610 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f2fb806f460 tx=0x7f2fac009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9844e5cb45bbde7 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:29.928 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.921+0000 7f2fb67fc700 1 -- 172.21.15.143:0/2898566865 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fac00b870 con 0x7f2fb815dba0 2023-11-08T19:27:29.928 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.921+0000 7f2fb67fc700 1 -- 172.21.15.143:0/2898566865 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fac00b9d0 con 0x7f2fb815dba0 2023-11-08T19:27:29.928 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.921+0000 7f2fb67fc700 1 -- 172.21.15.143:0/2898566865 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fac00bd60 con 0x7f2fb815dba0 2023-11-08T19:27:29.929 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.922+0000 7f2fbdceb700 1 -- 172.21.15.143:0/2898566865 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2fb815dba0 msgr2=0x7f2fb806c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:29.929 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.922+0000 7f2fbdceb700 1 --2- 172.21.15.143:0/2898566865 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2fb815dba0 0x7f2fb806c610 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f2fb806f460 tx=0x7f2fac009bd0 comp rx=0 tx=0).stop 2023-11-08T19:27:29.929 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.922+0000 7f2fbdceb700 1 -- 172.21.15.143:0/2898566865 shutdown_connections 2023-11-08T19:27:29.929 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.922+0000 7f2fbdceb700 1 --2- 172.21.15.143:0/2898566865 >> v2:172.21.15.174:3300/0 conn(0x7f2fb80cb530 0x7f2fb80b22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:29.929 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.922+0000 7f2fbdceb700 1 --2- 172.21.15.143:0/2898566865 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2fb815dba0 0x7f2fb806c610 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:29.930 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.922+0000 7f2fbdceb700 1 --2- 172.21.15.143:0/2898566865 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2fb806b240 0x7f2fb80caff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:29.930 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.922+0000 7f2fbdceb700 1 -- 172.21.15.143:0/2898566865 >> 172.21.15.143:0/2898566865 conn(0x7f2fb80cd2e0 msgr2=0x7f2fb80cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:29.930 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.922+0000 7f2fbdceb700 1 -- 172.21.15.143:0/2898566865 shutdown_connections 2023-11-08T19:27:29.930 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.922+0000 7f2fbdceb700 1 -- 172.21.15.143:0/2898566865 wait complete. 2023-11-08T19:27:29.930 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.923+0000 7f2fbdceb700 1 Processor -- start 2023-11-08T19:27:29.930 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.923+0000 7f2fbdceb700 1 -- start start 2023-11-08T19:27:29.933 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.924+0000 7f2fbdceb700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2fb806b240 0x7f2fb80b54b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:29.933 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.924+0000 7f2fbdceb700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2fb80cb530 0x7f2fb8154210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:29.934 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.924+0000 7f2fbdceb700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2fb815dba0 0x7f2fb80b1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:29.934 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.924+0000 7f2fbdceb700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb815df80 con 0x7f2fb80cb530 2023-11-08T19:27:29.934 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.924+0000 7f2fbdceb700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb80b63a0 con 0x7f2fb806b240 2023-11-08T19:27:29.934 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.924+0000 7f2fbdceb700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb80b5de0 con 0x7f2fb815dba0 2023-11-08T19:27:29.934 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.924+0000 7f2fb6ffd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2fb80cb530 0x7f2fb8154210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:29.935 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.924+0000 7f2fb6ffd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2fb80cb530 0x7f2fb8154210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53568/0 (socket says 172.21.15.143:53568) 2023-11-08T19:27:29.935 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.924+0000 7f2fb6ffd700 1 -- 172.21.15.143:0/855859094 learned_addr learned my addr 172.21.15.143:0/855859094 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:29.935 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.924+0000 7f2fb7fff700 1 --2- 172.21.15.143:0/855859094 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2fb815dba0 0x7f2fb80b1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:29.935 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.924+0000 7f2fb77fe700 1 --2- 172.21.15.143:0/855859094 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2fb806b240 0x7f2fb80b54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:29.936 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.924+0000 7f2fb6ffd700 1 -- 172.21.15.143:0/855859094 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2fb815dba0 msgr2=0x7f2fb80b1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:29.936 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.924+0000 7f2fb6ffd700 1 --2- 172.21.15.143:0/855859094 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2fb815dba0 0x7f2fb80b1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:29.936 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.924+0000 7f2fb6ffd700 1 -- 172.21.15.143:0/855859094 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2fb806b240 msgr2=0x7f2fb80b54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:29.936 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.924+0000 7f2fb6ffd700 1 --2- 172.21.15.143:0/855859094 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2fb806b240 0x7f2fb80b54b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:29.936 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.924+0000 7f2fb6ffd700 1 -- 172.21.15.143:0/855859094 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fac009550 con 0x7f2fb80cb530 2023-11-08T19:27:29.936 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.924+0000 7f2fb6ffd700 1 --2- 172.21.15.143:0/855859094 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2fb80cb530 0x7f2fb8154210 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f2fa800c740 tx=0x7f2fa800cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:29.937 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.925+0000 7f2f9ffff700 1 -- 172.21.15.143:0/855859094 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa8011800 con 0x7f2fb80cb530 2023-11-08T19:27:29.937 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.925+0000 7f2f9ffff700 1 -- 172.21.15.143:0/855859094 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fa8011960 con 0x7f2fb80cb530 2023-11-08T19:27:29.937 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.925+0000 7f2fbdceb700 1 -- 172.21.15.143:0/855859094 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fb80b60c0 con 0x7f2fb80cb530 2023-11-08T19:27:29.937 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.925+0000 7f2f9ffff700 1 -- 172.21.15.143:0/855859094 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa8011cd0 con 0x7f2fb80cb530 2023-11-08T19:27:29.937 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.925+0000 7f2fbdceb700 1 -- 172.21.15.143:0/855859094 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fb80acd00 con 0x7f2fb80cb530 2023-11-08T19:27:29.938 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.926+0000 7f2fbdceb700 1 -- 172.21.15.143:0/855859094 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fb80acf20 con 0x7f2fb80cb530 2023-11-08T19:27:29.938 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.926+0000 7f2f9ffff700 1 -- 172.21.15.143:0/855859094 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2fa8023480 con 0x7f2fb80cb530 2023-11-08T19:27:29.938 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.927+0000 7f2f9ffff700 1 --2- 172.21.15.143:0/855859094 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f2fa406af00 0x7f2fa406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:29.938 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.927+0000 7f2f9ffff700 1 -- 172.21.15.143:0/855859094 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5920+0+0 (secure 0 0 0) 0x7f2fa80899f0 con 0x7f2fb80cb530 2023-11-08T19:27:29.939 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.927+0000 7f2fb77fe700 1 --2- 172.21.15.143:0/855859094 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f2fa406af00 0x7f2fa406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:29.939 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.927+0000 7f2fb77fe700 1 --2- 172.21.15.143:0/855859094 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f2fa406af00 0x7f2fa406d3c0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f2fac014040 tx=0x7f2fac008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:29.939 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:29.931+0000 7f2f9ffff700 1 -- 172.21.15.143:0/855859094 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2fa8054a60 con 0x7f2fb80cb530 2023-11-08T19:27:30.119 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:30.118+0000 7f2fbdceb700 1 -- 172.21.15.143:0/855859094 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f2fb80afce0 con 0x7f2fb80cb530 2023-11-08T19:27:30.294 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:30.293+0000 7f2f9ffff700 1 -- 172.21.15.143:0/855859094 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v75) v1 ==== 138+0+0 (secure 0 0 0) 0x7f2fa8058080 con 0x7f2fb80cb530 2023-11-08T19:27:30.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:30.296+0000 7f2fbdceb700 1 -- 172.21.15.143:0/855859094 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f2fa406af00 msgr2=0x7f2fa406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:30.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:30.296+0000 7f2fbdceb700 1 --2- 172.21.15.143:0/855859094 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f2fa406af00 0x7f2fa406d3c0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f2fac014040 tx=0x7f2fac008030 comp rx=0 tx=0).stop 2023-11-08T19:27:30.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:30.296+0000 7f2fbdceb700 1 -- 172.21.15.143:0/855859094 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2fb80cb530 msgr2=0x7f2fb8154210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:30.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:30.296+0000 7f2fbdceb700 1 --2- 172.21.15.143:0/855859094 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2fb80cb530 0x7f2fb8154210 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f2fa800c740 tx=0x7f2fa800cb00 comp rx=0 tx=0).stop 2023-11-08T19:27:30.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:30.296+0000 7f2fbdceb700 1 -- 172.21.15.143:0/855859094 shutdown_connections 2023-11-08T19:27:30.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:30.296+0000 7f2fbdceb700 1 --2- 172.21.15.143:0/855859094 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f2fa406af00 0x7f2fa406d3c0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:30.299 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:30.296+0000 7f2fbdceb700 1 --2- 172.21.15.143:0/855859094 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2fb815dba0 0x7f2fb80b1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:30.299 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:30.296+0000 7f2fbdceb700 1 --2- 172.21.15.143:0/855859094 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2fb806b240 0x7f2fb80b54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:30.299 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:30.296+0000 7f2fbdceb700 1 --2- 172.21.15.143:0/855859094 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2fb80cb530 0x7f2fb8154210 secure :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f2fa800c740 tx=0x7f2fa800cb00 comp rx=0 tx=0).stop 2023-11-08T19:27:30.299 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:30.296+0000 7f2fbdceb700 1 -- 172.21.15.143:0/855859094 >> 172.21.15.143:0/855859094 conn(0x7f2fb80cd2e0 msgr2=0x7f2fb80cea90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:30.299 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:30.296+0000 7f2fbdceb700 1 -- 172.21.15.143:0/855859094 shutdown_connections 2023-11-08T19:27:30.300 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:30.296+0000 7f2fbdceb700 1 -- 172.21.15.143:0/855859094 wait complete. 2023-11-08T19:27:30.300 INFO:teuthology.orchestra.run.smithi143.stderr:pool 'cephfs_metadata' created 2023-11-08T19:27:30.313 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-08T19:27:30.441 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:30.440+0000 7f8108173700 1 Processor -- start 2023-11-08T19:27:30.441 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:30.440+0000 7f8108173700 1 -- start start 2023-11-08T19:27:30.444 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:30.441+0000 7f8108173700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f810015dba0 0x7f810006c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:30.444 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:30.441+0000 7f8108173700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f810006b240 0x7f81000caff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:30.444 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:30.441+0000 7f8108173700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8100161890 con 0x7f81000cb530 2023-11-08T19:27:30.445 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:30.441+0000 7f8108173700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f810006b740 con 0x7f810015dba0 2023-11-08T19:27:30.445 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:30.441+0000 7f8108173700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f810006ba20 con 0x7f810006b240 2023-11-08T19:27:30.445 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:30.441+0000 7f8105f0f700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f810015dba0 0x7f810006c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:30.445 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:30.441+0000 7f8105f0f700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f810015dba0 0x7f810006c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53574/0 (socket says 172.21.15.143:53574) 2023-11-08T19:27:30.446 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:30.441+0000 7f8105f0f700 1 -- 172.21.15.143:0/1326780830 learned_addr learned my addr 172.21.15.143:0/1326780830 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:30.446 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:30.441+0000 7f8105f0f700 1 -- 172.21.15.143:0/1326780830 >> v2:172.21.15.174:3300/0 conn(0x7f810006b240 msgr2=0x7f81000caff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:30.446 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:30.441+0000 7f8105f0f700 1 --2- 172.21.15.143:0/1326780830 >> v2:172.21.15.174:3300/0 conn(0x7f810006b240 0x7f81000caff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:30.446 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:30.441+0000 7f8105f0f700 1 -- 172.21.15.143:0/1326780830 >> v1:172.21.15.174:6789/0 conn(0x7f81000cb530 legacy=0x7f81000b22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:30.447 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:30.441+0000 7f8105f0f700 1 -- 172.21.15.143:0/1326780830 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f810006bd00 con 0x7f810015dba0 2023-11-08T19:27:30.447 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:30.441+0000 7f8105f0f700 1 --2- 172.21.15.143:0/1326780830 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f810015dba0 0x7f810006c610 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f81000d0810 tx=0x7f80fc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=601fff5228e86e8c server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:30.448 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:30.441+0000 7f8104f0d700 1 -- 172.21.15.143:0/1326780830 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80fc004230 con 0x7f810015dba0 2023-11-08T19:27:30.448 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:30.441+0000 7f8104f0d700 1 -- 172.21.15.143:0/1326780830 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80fc004390 con 0x7f810015dba0 2023-11-08T19:27:30.448 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:30.441+0000 7f8104f0d700 1 -- 172.21.15.143:0/1326780830 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80fc0036a0 con 0x7f810015dba0 2023-11-08T19:27:30.448 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:30.442+0000 7f8108173700 1 -- 172.21.15.143:0/1326780830 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f810015dba0 msgr2=0x7f810006c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:30.449 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:30.442+0000 7f8108173700 1 --2- 172.21.15.143:0/1326780830 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f810015dba0 0x7f810006c610 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f81000d0810 tx=0x7f80fc009bd0 comp rx=0 tx=0).stop 2023-11-08T19:27:30.449 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:30.442+0000 7f8108173700 1 -- 172.21.15.143:0/1326780830 shutdown_connections 2023-11-08T19:27:30.449 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:30.442+0000 7f8108173700 1 --2- 172.21.15.143:0/1326780830 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f810015dba0 0x7f810006c610 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:30.449 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:30.442+0000 7f8108173700 1 --2- 172.21.15.143:0/1326780830 >> v2:172.21.15.174:3300/0 conn(0x7f810006b240 0x7f81000caff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:30.449 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:30.442+0000 7f8108173700 1 -- 172.21.15.143:0/1326780830 >> 172.21.15.143:0/1326780830 conn(0x7f81000cd2e0 msgr2=0x7f81000cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:30.450 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:30.442+0000 7f8108173700 1 -- 172.21.15.143:0/1326780830 shutdown_connections 2023-11-08T19:27:30.450 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:30.442+0000 7f8108173700 1 -- 172.21.15.143:0/1326780830 wait complete. 2023-11-08T19:27:30.453 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:30.443+0000 7f8108173700 1 Processor -- start 2023-11-08T19:27:30.453 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:30.443+0000 7f8108173700 1 -- start start 2023-11-08T19:27:30.453 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:30.444+0000 7f8108173700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f810006b240 0x7f81000ae6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:30.453 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:30.444+0000 7f8108173700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f81000cb530 0x7f81001f6e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:30.453 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:30.444+0000 7f8108173700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f810015dba0 0x7f81001f7370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:30.454 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:30.444+0000 7f8108173700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81000ad320 con 0x7f81000cb530 2023-11-08T19:27:30.454 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:30.444+0000 7f8108173700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f810014ee50 con 0x7f810006b240 2023-11-08T19:27:30.454 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:30.444+0000 7f8108173700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81000c8980 con 0x7f810015dba0 2023-11-08T19:27:30.454 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:30.444+0000 7f810570e700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f81000cb530 0x7f81001f6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:30.455 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:30.444+0000 7f810570e700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f81000cb530 0x7f81001f6e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53582/0 (socket says 172.21.15.143:53582) 2023-11-08T19:27:30.455 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:30.444+0000 7f810570e700 1 -- 172.21.15.143:0/2149011966 learned_addr learned my addr 172.21.15.143:0/2149011966 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:30.455 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:30.444+0000 7f8105f0f700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f810006b240 0x7f81000ae6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:30.455 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:30.444+0000 7f8106710700 1 --2- 172.21.15.143:0/2149011966 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f810015dba0 0x7f81001f7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:30.455 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:30.444+0000 7f810570e700 1 -- 172.21.15.143:0/2149011966 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f810015dba0 msgr2=0x7f81001f7370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:30.456 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:30.444+0000 7f810570e700 1 --2- 172.21.15.143:0/2149011966 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f810015dba0 0x7f81001f7370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:30.456 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:30.444+0000 7f810570e700 1 -- 172.21.15.143:0/2149011966 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f810006b240 msgr2=0x7f81000ae6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:30.456 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:30.444+0000 7f810570e700 1 --2- 172.21.15.143:0/2149011966 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f810006b240 0x7f81000ae6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:30.456 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:30.444+0000 7f810570e700 1 -- 172.21.15.143:0/2149011966 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80fc009550 con 0x7f81000cb530 2023-11-08T19:27:30.456 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:30.445+0000 7f810570e700 1 --2- 172.21.15.143:0/2149011966 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f81000cb530 0x7f81001f6e30 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f80f000b530 tx=0x7f80f000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:30.457 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:30.445+0000 7f80f67fc700 1 -- 172.21.15.143:0/2149011966 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80f0011830 con 0x7f81000cb530 2023-11-08T19:27:30.457 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:30.445+0000 7f80f67fc700 1 -- 172.21.15.143:0/2149011966 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80f0011990 con 0x7f81000cb530 2023-11-08T19:27:30.457 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:30.445+0000 7f8108173700 1 -- 172.21.15.143:0/2149011966 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81000af020 con 0x7f81000cb530 2023-11-08T19:27:30.457 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:30.445+0000 7f80f67fc700 1 -- 172.21.15.143:0/2149011966 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80f0011d40 con 0x7f81000cb530 2023-11-08T19:27:30.458 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:30.445+0000 7f8108173700 1 -- 172.21.15.143:0/2149011966 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81000af5e0 con 0x7f81000cb530 2023-11-08T19:27:30.458 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:30.446+0000 7f8108173700 1 -- 172.21.15.143:0/2149011966 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81000c8ba0 con 0x7f81000cb530 2023-11-08T19:27:30.458 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:30.446+0000 7f80f67fc700 1 -- 172.21.15.143:0/2149011966 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f80f001a400 con 0x7f81000cb530 2023-11-08T19:27:30.458 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:30.447+0000 7f80f67fc700 1 --2- 172.21.15.143:0/2149011966 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f80e406af00 0x7f80e406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:30.458 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:30.447+0000 7f80f67fc700 1 -- 172.21.15.143:0/2149011966 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6290+0+0 (secure 0 0 0) 0x7f80f0089b60 con 0x7f81000cb530 2023-11-08T19:27:30.459 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:30.447+0000 7f8105f0f700 1 --2- 172.21.15.143:0/2149011966 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f80e406af00 0x7f80e406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:30.459 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:30.447+0000 7f8105f0f700 1 --2- 172.21.15.143:0/2149011966 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f80e406af00 0x7f80e406d3c0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f80fc009130 tx=0x7f80fc008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:30.459 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:30.451+0000 7f80f67fc700 1 -- 172.21.15.143:0/2149011966 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f80f0054b60 con 0x7f81000cb530 2023-11-08T19:27:30.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:30.641+0000 7f8108173700 1 -- 172.21.15.143:0/2149011966 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f810004eee0 con 0x7f81000cb530 2023-11-08T19:27:31.302 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:31.300+0000 7f80f67fc700 1 -- 172.21.15.143:0/2149011966 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v76) v1 ==== 170+0+0 (secure 0 0 0) 0x7f80f00150d0 con 0x7f81000cb530 2023-11-08T19:27:31.305 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:31.303+0000 7f8108173700 1 -- 172.21.15.143:0/2149011966 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f80e406af00 msgr2=0x7f80e406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:31.305 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:31.303+0000 7f8108173700 1 --2- 172.21.15.143:0/2149011966 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f80e406af00 0x7f80e406d3c0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f80fc009130 tx=0x7f80fc008030 comp rx=0 tx=0).stop 2023-11-08T19:27:31.305 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:31.303+0000 7f8108173700 1 -- 172.21.15.143:0/2149011966 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f81000cb530 msgr2=0x7f81001f6e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:31.305 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:31.303+0000 7f8108173700 1 --2- 172.21.15.143:0/2149011966 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f81000cb530 0x7f81001f6e30 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f80f000b530 tx=0x7f80f000b840 comp rx=0 tx=0).stop 2023-11-08T19:27:31.306 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:31.304+0000 7f8108173700 1 -- 172.21.15.143:0/2149011966 shutdown_connections 2023-11-08T19:27:31.306 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:31.304+0000 7f8108173700 1 --2- 172.21.15.143:0/2149011966 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f80e406af00 0x7f80e406d3c0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:31.306 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:31.304+0000 7f8108173700 1 --2- 172.21.15.143:0/2149011966 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f810015dba0 0x7f81001f7370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:31.307 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:31.304+0000 7f8108173700 1 --2- 172.21.15.143:0/2149011966 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f810006b240 0x7f81000ae6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:31.307 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:31.304+0000 7f8108173700 1 --2- 172.21.15.143:0/2149011966 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f81000cb530 0x7f81001f6e30 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:31.307 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:31.304+0000 7f8108173700 1 -- 172.21.15.143:0/2149011966 >> 172.21.15.143:0/2149011966 conn(0x7f81000cd2e0 msgr2=0x7f81000b0280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:31.307 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:31.305+0000 7f8108173700 1 -- 172.21.15.143:0/2149011966 shutdown_connections 2023-11-08T19:27:31.307 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:31.305+0000 7f8108173700 1 -- 172.21.15.143:0/2149011966 wait complete. 2023-11-08T19:27:31.316 INFO:teuthology.orchestra.run.smithi143.stderr:pool 'cephfs_data' created 2023-11-08T19:27:31.334 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-08T19:27:31.460 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:31.458+0000 7fa6bb10e700 1 Processor -- start 2023-11-08T19:27:31.460 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:31.458+0000 7fa6bb10e700 1 -- start start 2023-11-08T19:27:31.463 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:31.459+0000 7fa6bb10e700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa6b40b22e0 0x7fa6b40b2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:31.463 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:31.459+0000 7fa6bb10e700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6b40b2c70 con 0x7fa6b406b0b0 2023-11-08T19:27:31.463 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:31.459+0000 7fa6bb10e700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6b406b670 con 0x7fa6b415bc80 2023-11-08T19:27:31.463 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:31.459+0000 7fa6bb10e700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6b40ce290 con 0x7fa6b40b22e0 2023-11-08T19:27:31.464 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:31.459+0000 7fa6b96ab700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa6b40b22e0 0x7fa6b40b2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:31.464 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:31.459+0000 7fa6b96ab700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa6b40b22e0 0x7fa6b40b2730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53592/0 (socket says 172.21.15.143:53592) 2023-11-08T19:27:31.464 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:31.459+0000 7fa6b3fff700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fa6b406b0b0 0x7fa6b406c710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:49744/0 (socket says 172.21.15.143:49744) 2023-11-08T19:27:31.464 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:31.459+0000 7fa6b96ab700 1 -- 172.21.15.143:0/229682476 learned_addr learned my addr 172.21.15.143:0/229682476 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:31.465 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:31.460+0000 7fa6b37fe700 1 -- 172.21.15.143:0/229682476 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3395454813 0 0) 0x7fa6a0001110 con 0x7fa6b406b0b0 2023-11-08T19:27:31.465 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:31.460+0000 7fa6b37fe700 1 -- 172.21.15.143:0/229682476 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6a40035b0 con 0x7fa6b406b0b0 2023-11-08T19:27:31.465 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:31.460+0000 7fa6b96ab700 1 -- 172.21.15.143:0/229682476 >> v1:172.21.15.174:6789/0 conn(0x7fa6b415bc80 legacy=0x7fa6b40b6280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:31.465 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:31.460+0000 7fa6b96ab700 1 -- 172.21.15.143:0/229682476 >> v1:172.21.15.143:6789/0 conn(0x7fa6b406b0b0 legacy=0x7fa6b406c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:31.465 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:31.460+0000 7fa6b96ab700 1 -- 172.21.15.143:0/229682476 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6b406b950 con 0x7fa6b40b22e0 2023-11-08T19:27:31.466 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:31.460+0000 7fa6b96ab700 1 --2- 172.21.15.143:0/229682476 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6b40b22e0 0x7fa6b40b2730 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac009e80 tx=0x7fa6ac00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76f4333158cae312 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:31.466 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:31.460+0000 7fa6b37fe700 1 -- 172.21.15.143:0/229682476 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6ac00e070 con 0x7fa6b40b22e0 2023-11-08T19:27:31.466 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:31.460+0000 7fa6b37fe700 1 -- 172.21.15.143:0/229682476 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6ac00b8d0 con 0x7fa6b40b22e0 2023-11-08T19:27:31.466 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:31.460+0000 7fa6b37fe700 1 -- 172.21.15.143:0/229682476 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6ac00bc80 con 0x7fa6b40b22e0 2023-11-08T19:27:31.466 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:31.461+0000 7fa6bb10e700 1 -- 172.21.15.143:0/229682476 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6b40b22e0 msgr2=0x7fa6b40b2730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:31.467 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:31.461+0000 7fa6bb10e700 1 --2- 172.21.15.143:0/229682476 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6b40b22e0 0x7fa6b40b2730 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac009e80 tx=0x7fa6ac00b330 comp rx=0 tx=0).stop 2023-11-08T19:27:31.467 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:31.461+0000 7fa6bb10e700 1 -- 172.21.15.143:0/229682476 shutdown_connections 2023-11-08T19:27:31.467 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:31.461+0000 7fa6bb10e700 1 --2- 172.21.15.143:0/229682476 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6b40b22e0 0x7fa6b40b2730 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:31.467 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:31.461+0000 7fa6bb10e700 1 -- 172.21.15.143:0/229682476 >> 172.21.15.143:0/229682476 conn(0x7fa6b40cd2e0 msgr2=0x7fa6b40cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:31.467 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:31.461+0000 7fa6bb10e700 1 -- 172.21.15.143:0/229682476 shutdown_connections 2023-11-08T19:27:31.468 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:31.461+0000 7fa6bb10e700 1 -- 172.21.15.143:0/229682476 wait complete. 2023-11-08T19:27:31.468 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:31.462+0000 7fa6bb10e700 1 Processor -- start 2023-11-08T19:27:31.468 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:31.462+0000 7fa6bb10e700 1 -- start start 2023-11-08T19:27:31.471 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:31.463+0000 7fa6bb10e700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa6b406b0b0 0x7fa6b40c6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:31.471 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:31.463+0000 7fa6bb10e700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa6b40b22e0 0x7fa6b40c7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:31.471 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:31.463+0000 7fa6bb10e700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6b415bc80 0x7fa6b414bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:31.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:31.463+0000 7fa6bb10e700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6b414a800 con 0x7fa6b415bc80 2023-11-08T19:27:31.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:31.463+0000 7fa6bb10e700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6b414ba40 con 0x7fa6b40b22e0 2023-11-08T19:27:31.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:31.463+0000 7fa6bb10e700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6b414ae30 con 0x7fa6b406b0b0 2023-11-08T19:27:31.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:31.463+0000 7fa6b96ab700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6b415bc80 0x7fa6b414bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:31.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:31.463+0000 7fa6b96ab700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6b415bc80 0x7fa6b414bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53598/0 (socket says 172.21.15.143:53598) 2023-11-08T19:27:31.473 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:31.463+0000 7fa6b96ab700 1 -- 172.21.15.143:0/3178458527 learned_addr learned my addr 172.21.15.143:0/3178458527 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:31.473 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:31.463+0000 7fa6b3fff700 1 --2- 172.21.15.143:0/3178458527 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa6b40b22e0 0x7fa6b40c7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:31.473 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:31.463+0000 7fa6b8eaa700 1 --2- 172.21.15.143:0/3178458527 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa6b406b0b0 0x7fa6b40c6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:31.473 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:31.463+0000 7fa6b96ab700 1 -- 172.21.15.143:0/3178458527 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa6b406b0b0 msgr2=0x7fa6b40c6bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:31.473 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:31.463+0000 7fa6b96ab700 1 --2- 172.21.15.143:0/3178458527 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa6b406b0b0 0x7fa6b40c6bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:31.474 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:31.463+0000 7fa6b96ab700 1 -- 172.21.15.143:0/3178458527 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa6b40b22e0 msgr2=0x7fa6b40c7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:31.474 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:31.463+0000 7fa6b96ab700 1 --2- 172.21.15.143:0/3178458527 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa6b40b22e0 0x7fa6b40c7bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:31.474 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:31.463+0000 7fa6b96ab700 1 -- 172.21.15.143:0/3178458527 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6ac009b30 con 0x7fa6b415bc80 2023-11-08T19:27:31.474 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:31.463+0000 7fa6b8eaa700 1 --2- 172.21.15.143:0/3178458527 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa6b406b0b0 0x7fa6b40c6bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:27:31.474 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:31.463+0000 7fa6b96ab700 1 --2- 172.21.15.143:0/3178458527 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6b415bc80 0x7fa6b414bed0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac00b680 tx=0x7fa6ac004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:31.475 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:31.463+0000 7fa6b17fa700 1 -- 172.21.15.143:0/3178458527 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6ac0099b0 con 0x7fa6b415bc80 2023-11-08T19:27:31.475 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:31.463+0000 7fa6b17fa700 1 -- 172.21.15.143:0/3178458527 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6ac0092e0 con 0x7fa6b415bc80 2023-11-08T19:27:31.475 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:31.463+0000 7fa6bb10e700 1 -- 172.21.15.143:0/3178458527 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6b414b110 con 0x7fa6b415bc80 2023-11-08T19:27:31.475 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:31.463+0000 7fa6b17fa700 1 -- 172.21.15.143:0/3178458527 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6ac01c640 con 0x7fa6b415bc80 2023-11-08T19:27:31.475 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:31.464+0000 7fa6bb10e700 1 -- 172.21.15.143:0/3178458527 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6b414b6d0 con 0x7fa6b415bc80 2023-11-08T19:27:31.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:31.464+0000 7fa6bb10e700 1 -- 172.21.15.143:0/3178458527 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6b404eee0 con 0x7fa6b415bc80 2023-11-08T19:27:31.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:31.465+0000 7fa6b17fa700 1 -- 172.21.15.143:0/3178458527 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa6ac01c7a0 con 0x7fa6b415bc80 2023-11-08T19:27:31.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:31.465+0000 7fa6b17fa700 1 --2- 172.21.15.143:0/3178458527 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa69806af00 0x7fa69806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:31.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:31.465+0000 7fa6b17fa700 1 -- 172.21.15.143:0/3178458527 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6672+0+0 (secure 0 0 0) 0x7fa6ac08b890 con 0x7fa6b415bc80 2023-11-08T19:27:31.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:31.466+0000 7fa6b8eaa700 1 --2- 172.21.15.143:0/3178458527 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa69806af00 0x7fa69806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:31.477 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:31.466+0000 7fa6b8eaa700 1 --2- 172.21.15.143:0/3178458527 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa69806af00 0x7fa69806d3c0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fa6a8000f60 tx=0x7fa6a8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:31.477 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:31.469+0000 7fa6b17fa700 1 -- 172.21.15.143:0/3178458527 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa6ac056610 con 0x7fa6b415bc80 2023-11-08T19:27:31.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:31.700+0000 7fa6bb10e700 1 -- 172.21.15.143:0/3178458527 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fa6b414bb80 con 0x7fa6b415bc80 2023-11-08T19:27:32.303 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.301+0000 7fa6b17fa700 1 -- 172.21.15.143:0/3178458527 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 13 and data pool 14 v38) v1 ==== 174+0+0 (secure 0 0 0) 0x7fa6ac059c30 con 0x7fa6b415bc80 2023-11-08T19:27:32.306 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.305+0000 7fa6bb10e700 1 -- 172.21.15.143:0/3178458527 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa69806af00 msgr2=0x7fa69806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:32.307 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.305+0000 7fa6bb10e700 1 --2- 172.21.15.143:0/3178458527 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa69806af00 0x7fa69806d3c0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fa6a8000f60 tx=0x7fa6a8009040 comp rx=0 tx=0).stop 2023-11-08T19:27:32.307 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.305+0000 7fa6bb10e700 1 -- 172.21.15.143:0/3178458527 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6b415bc80 msgr2=0x7fa6b414bed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:32.307 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.305+0000 7fa6bb10e700 1 --2- 172.21.15.143:0/3178458527 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6b415bc80 0x7fa6b414bed0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac00b680 tx=0x7fa6ac004ad0 comp rx=0 tx=0).stop 2023-11-08T19:27:32.307 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.305+0000 7fa6bb10e700 1 -- 172.21.15.143:0/3178458527 shutdown_connections 2023-11-08T19:27:32.308 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.305+0000 7fa6bb10e700 1 --2- 172.21.15.143:0/3178458527 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa69806af00 0x7fa69806d3c0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:32.308 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.305+0000 7fa6bb10e700 1 --2- 172.21.15.143:0/3178458527 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa6b406b0b0 0x7fa6b40c6bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:32.308 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.305+0000 7fa6bb10e700 1 --2- 172.21.15.143:0/3178458527 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6b415bc80 0x7fa6b414bed0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:32.308 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.305+0000 7fa6bb10e700 1 --2- 172.21.15.143:0/3178458527 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa6b40b22e0 0x7fa6b40c7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:32.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.305+0000 7fa6bb10e700 1 -- 172.21.15.143:0/3178458527 >> 172.21.15.143:0/3178458527 conn(0x7fa6b40cd2e0 msgr2=0x7fa6b40b0110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:32.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.306+0000 7fa6bb10e700 1 -- 172.21.15.143:0/3178458527 shutdown_connections 2023-11-08T19:27:32.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.306+0000 7fa6bb10e700 1 -- 172.21.15.143:0/3178458527 wait complete. 2023-11-08T19:27:32.324 INFO:teuthology.orchestra.run.smithi143.stderr:new fs with metadata pool 13 and data pool 14 2023-11-08T19:27:32.333 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:27:32.454 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.453+0000 7fd703c9b700 1 Processor -- start 2023-11-08T19:27:32.454 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.453+0000 7fd703c9b700 1 -- start start 2023-11-08T19:27:32.457 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.454+0000 7fd703c9b700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fd6fc06dbb0 0x7fd6fc0ca470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:32.457 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.454+0000 7fd703c9b700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd6fc0668e0 0x7fd6fc0ca9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:32.457 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.454+0000 7fd703c9b700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6fc0ce7f0 con 0x7fd6fc06df90 2023-11-08T19:27:32.457 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.454+0000 7fd703c9b700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6fc066db0 con 0x7fd6fc0668e0 2023-11-08T19:27:32.458 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.454+0000 7fd703c9b700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6fc0af5e0 con 0x7fd6fc06dbb0 2023-11-08T19:27:32.458 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.454+0000 7fd701236700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd6fc0668e0 0x7fd6fc0ca9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:32.458 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.454+0000 7fd701236700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd6fc0668e0 0x7fd6fc0ca9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53602/0 (socket says 172.21.15.143:53602) 2023-11-08T19:27:32.458 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.454+0000 7fd701236700 1 -- 172.21.15.143:0/2534461082 learned_addr learned my addr 172.21.15.143:0/2534461082 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:32.459 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.454+0000 7fd701236700 1 -- 172.21.15.143:0/2534461082 >> v2:172.21.15.174:3300/0 conn(0x7fd6fc06dbb0 msgr2=0x7fd6fc0ca470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:32.459 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.454+0000 7fd701a37700 1 --2- 172.21.15.143:0/2534461082 >> v2:172.21.15.174:3300/0 conn(0x7fd6fc06dbb0 0x7fd6fc0ca470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:32.459 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.454+0000 7fd701236700 1 --2- 172.21.15.143:0/2534461082 >> v2:172.21.15.174:3300/0 conn(0x7fd6fc06dbb0 0x7fd6fc0ca470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:32.459 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.454+0000 7fd701236700 1 -- 172.21.15.143:0/2534461082 >> v1:172.21.15.174:6789/0 conn(0x7fd6fc06df90 legacy=0x7fd6fc1621f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:32.459 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.454+0000 7fd701236700 1 -- 172.21.15.143:0/2534461082 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6fc0cb520 con 0x7fd6fc0668e0 2023-11-08T19:27:32.460 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.454+0000 7fd701236700 1 --2- 172.21.15.143:0/2534461082 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd6fc0668e0 0x7fd6fc0ca9b0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fd6fc0cfe10 tx=0x7fd6ec009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b34834b9c5eea10 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:32.460 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.454+0000 7fd700a35700 1 -- 172.21.15.143:0/2534461082 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6ec004230 con 0x7fd6fc0668e0 2023-11-08T19:27:32.460 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.454+0000 7fd700a35700 1 -- 172.21.15.143:0/2534461082 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6ec004390 con 0x7fd6fc0668e0 2023-11-08T19:27:32.460 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.454+0000 7fd700a35700 1 -- 172.21.15.143:0/2534461082 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6ec0036a0 con 0x7fd6fc0668e0 2023-11-08T19:27:32.461 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.455+0000 7fd703c9b700 1 -- 172.21.15.143:0/2534461082 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd6fc0668e0 msgr2=0x7fd6fc0ca9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:32.461 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.455+0000 7fd703c9b700 1 --2- 172.21.15.143:0/2534461082 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd6fc0668e0 0x7fd6fc0ca9b0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fd6fc0cfe10 tx=0x7fd6ec009bd0 comp rx=0 tx=0).stop 2023-11-08T19:27:32.461 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.455+0000 7fd703c9b700 1 -- 172.21.15.143:0/2534461082 shutdown_connections 2023-11-08T19:27:32.461 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.455+0000 7fd703c9b700 1 --2- 172.21.15.143:0/2534461082 >> v2:172.21.15.174:3300/0 conn(0x7fd6fc06dbb0 0x7fd6fc0ca470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:32.462 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.455+0000 7fd703c9b700 1 --2- 172.21.15.143:0/2534461082 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd6fc0668e0 0x7fd6fc0ca9b0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:32.462 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.455+0000 7fd703c9b700 1 -- 172.21.15.143:0/2534461082 >> 172.21.15.143:0/2534461082 conn(0x7fd6fc0b1a20 msgr2=0x7fd6fc0b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:32.462 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.455+0000 7fd703c9b700 1 -- 172.21.15.143:0/2534461082 shutdown_connections 2023-11-08T19:27:32.462 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.455+0000 7fd703c9b700 1 -- 172.21.15.143:0/2534461082 wait complete. 2023-11-08T19:27:32.465 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.456+0000 7fd703c9b700 1 Processor -- start 2023-11-08T19:27:32.465 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.456+0000 7fd703c9b700 1 -- start start 2023-11-08T19:27:32.466 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.457+0000 7fd703c9b700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd6fc0668e0 0x7fd6fc155960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:32.466 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.457+0000 7fd703c9b700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd6fc06dbb0 0x7fd6fc0c5b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:32.466 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.457+0000 7fd703c9b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd6fc06df90 0x7fd6fc1e81c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:32.466 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.457+0000 7fd703c9b700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6fc0c3c20 con 0x7fd6fc0668e0 2023-11-08T19:27:32.466 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.457+0000 7fd703c9b700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6fc160da0 con 0x7fd6fc06df90 2023-11-08T19:27:32.467 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.457+0000 7fd703c9b700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6fc161080 con 0x7fd6fc06dbb0 2023-11-08T19:27:32.467 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.457+0000 7fd701a37700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd6fc0668e0 0x7fd6fc155960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:32.467 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.457+0000 7fd701a37700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd6fc0668e0 0x7fd6fc155960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53606/0 (socket says 172.21.15.143:53606) 2023-11-08T19:27:32.467 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.457+0000 7fd701a37700 1 -- 172.21.15.143:0/550671980 learned_addr learned my addr 172.21.15.143:0/550671980 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:32.467 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.457+0000 7fd701236700 1 --2- 172.21.15.143:0/550671980 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd6fc06dbb0 0x7fd6fc0c5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:32.468 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.457+0000 7fd702238700 1 --2- 172.21.15.143:0/550671980 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd6fc06df90 0x7fd6fc1e81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:32.468 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.457+0000 7fd701a37700 1 -- 172.21.15.143:0/550671980 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd6fc06dbb0 msgr2=0x7fd6fc0c5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:32.468 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.457+0000 7fd701a37700 1 --2- 172.21.15.143:0/550671980 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd6fc06dbb0 0x7fd6fc0c5b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:32.468 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.457+0000 7fd701a37700 1 -- 172.21.15.143:0/550671980 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd6fc06df90 msgr2=0x7fd6fc1e81c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:32.468 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.457+0000 7fd701a37700 1 --2- 172.21.15.143:0/550671980 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd6fc06df90 0x7fd6fc1e81c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:32.469 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.457+0000 7fd701a37700 1 -- 172.21.15.143:0/550671980 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6ec009550 con 0x7fd6fc0668e0 2023-11-08T19:27:32.469 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.458+0000 7fd701a37700 1 --2- 172.21.15.143:0/550671980 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd6fc0668e0 0x7fd6fc155960 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fd6f800c7f0 tx=0x7fd6f800cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:32.469 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.458+0000 7fd6f27fc700 1 -- 172.21.15.143:0/550671980 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f8011800 con 0x7fd6fc0668e0 2023-11-08T19:27:32.469 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.458+0000 7fd6f27fc700 1 -- 172.21.15.143:0/550671980 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6f8011960 con 0x7fd6fc0668e0 2023-11-08T19:27:32.469 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.458+0000 7fd703c9b700 1 -- 172.21.15.143:0/550671980 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6fc161360 con 0x7fd6fc0668e0 2023-11-08T19:27:32.470 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.458+0000 7fd6f27fc700 1 -- 172.21.15.143:0/550671980 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f8011cf0 con 0x7fd6fc0668e0 2023-11-08T19:27:32.470 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.458+0000 7fd703c9b700 1 -- 172.21.15.143:0/550671980 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6fc0c67b0 con 0x7fd6fc0668e0 2023-11-08T19:27:32.470 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.459+0000 7fd703c9b700 1 -- 172.21.15.143:0/550671980 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6fc06d1a0 con 0x7fd6fc0668e0 2023-11-08T19:27:32.470 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.459+0000 7fd6f27fc700 1 -- 172.21.15.143:0/550671980 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd6f8018360 con 0x7fd6fc0668e0 2023-11-08T19:27:32.470 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.459+0000 7fd6f27fc700 1 --2- 172.21.15.143:0/550671980 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd6e006af00 0x7fd6e006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:32.471 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.460+0000 7fd6f27fc700 1 -- 172.21.15.143:0/550671980 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6772+0+0 (secure 0 0 0) 0x7fd6f80899f0 con 0x7fd6fc0668e0 2023-11-08T19:27:32.471 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.460+0000 7fd701236700 1 --2- 172.21.15.143:0/550671980 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd6e006af00 0x7fd6e006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:32.471 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.460+0000 7fd701236700 1 --2- 172.21.15.143:0/550671980 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd6e006af00 0x7fd6e006d3c0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fd6fc0c3b20 tx=0x7fd6ec008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:32.471 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.464+0000 7fd6f27fc700 1 -- 172.21.15.143:0/550671980 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd6f8054680 con 0x7fd6fc0668e0 2023-11-08T19:27:32.660 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.658+0000 7fd703c9b700 1 -- 172.21.15.143:0/550671980 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd6fc04eee0 con 0x7fd6fc0668e0 2023-11-08T19:27:32.661 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.660+0000 7fd6f27fc700 1 -- 172.21.15.143:0/550671980 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) v1 ==== 74+0+21817 (secure 0 0 0) 0x7fd6f8057ca0 con 0x7fd6fc0668e0 2023-11-08T19:27:32.662 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:27:32.662 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":77,"fsid":"968483af-77fc-4068-ad7c-5c2a1a4613d7","created":"2023-11-08T19:24:32.826484+0000","modified":"2023-11-08T19:27:31.702217+0000","last_up_change":"2023-11-08T19:24:43.573880+0000","last_in_change":"2023-11-08T19:24:35.314525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:44.551530+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"snap-test-pool","create_time":"2023-11-08T19:26:02.874384+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":36,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"test_data_pool","create_time":"2023-11-08T19:27:05.601949+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":66,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_metadata_pool","create_time":"2023-11-08T19:27:06.536369+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-11-08T19:27:30.137037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-11-08T19:27:30.660318+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bae98ec1-d94e-403d-a220-a522a53ea6f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6809","nonce":1487830566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6811","nonce":1487830566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6815","nonce":1487830566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6813","nonce":1487830566}]},"public_addr":"172.21.15.143:6809/1487830566","cluster_addr":"172.21.15.143:6811/1487830566","heartbeat_back_addr":"172.21.15.143:6815/1487830566","heartbeat_front_addr":"172.21.15.143:6813/1487830566","state":["exists","up"]},{"osd":1,"uuid":"5cb6538a-1ccb-4322-8633-09f66af0b7e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6801","nonce":787341397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6803","nonce":787341397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6807","nonce":787341397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6805","nonce":787341397}]},"public_addr":"172.21.15.143:6801/787341397","cluster_addr":"172.21.15.143:6803/787341397","heartbeat_back_addr":"172.21.15.143:6807/787341397","heartbeat_front_addr":"172.21.15.143:6805/787341397","state":["exists","up"]},{"osd":2,"uuid":"cdbf522c-60ef-4d60-9653-8dcb529c101a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6822","nonce":2301325597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6826","nonce":2301325597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6831","nonce":2301325597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6829","nonce":2301325597}]},"public_addr":"172.21.15.143:6822/2301325597","cluster_addr":"172.21.15.143:6826/2301325597","heartbeat_back_addr":"172.21.15.143:6831/2301325597","heartbeat_front_addr":"172.21.15.143:6829/2301325597","state":["exists","up"]},{"osd":3,"uuid":"4e2783cf-ebb9-4dd8-8245-bc1d17ce4325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6817","nonce":2021504428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6819","nonce":2021504428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6827","nonce":2021504428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6823","nonce":2021504428}]},"public_addr":"172.21.15.143:6817/2021504428","cluster_addr":"172.21.15.143:6819/2021504428","heartbeat_back_addr":"172.21.15.143:6827/2021504428","heartbeat_front_addr":"172.21.15.143:6823/2021504428","state":["exists","up"]},{"osd":4,"uuid":"691256fa-9b77-49b9-a346-7ae1adf0ca80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6801","nonce":1993380595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6803","nonce":1993380595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6807","nonce":1993380595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6805","nonce":1993380595}]},"public_addr":"172.21.15.174:6801/1993380595","cluster_addr":"172.21.15.174:6803/1993380595","heartbeat_back_addr":"172.21.15.174:6807/1993380595","heartbeat_front_addr":"172.21.15.174:6805/1993380595","state":["exists","up"]},{"osd":5,"uuid":"756fc352-48c9-46ce-b52e-0541fe6f8525","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6809","nonce":1777923303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6811","nonce":1777923303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6815","nonce":1777923303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6813","nonce":1777923303}]},"public_addr":"172.21.15.174:6809/1777923303","cluster_addr":"172.21.15.174:6811/1777923303","heartbeat_back_addr":"172.21.15.174:6815/1777923303","heartbeat_front_addr":"172.21.15.174:6813/1777923303","state":["exists","up"]},{"osd":6,"uuid":"fab2ac2b-ed97-4cde-9825-b480eabf8b8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6825","nonce":2755130316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6827","nonce":2755130316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6831","nonce":2755130316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6829","nonce":2755130316}]},"public_addr":"172.21.15.174:6825/2755130316","cluster_addr":"172.21.15.174:6827/2755130316","heartbeat_back_addr":"172.21.15.174:6831/2755130316","heartbeat_front_addr":"172.21.15.174:6829/2755130316","state":["exists","up"]},{"osd":7,"uuid":"3de9b797-2ab6-4d41-a863-7e7d638fd703","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6817","nonce":2847141411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6819","nonce":2847141411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6823","nonce":2847141411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6821","nonce":2847141411}]},"public_addr":"172.21.15.174:6817/2847141411","cluster_addr":"172.21.15.174:6819/2847141411","heartbeat_back_addr":"172.21.15.174:6823/2847141411","heartbeat_front_addr":"172.21.15.174:6821/2847141411","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.477076+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.496890+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.539305+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.454673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.538881+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.521294+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.520712+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.540404+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:27:32.664 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.662+0000 7fd703c9b700 1 -- 172.21.15.143:0/550671980 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd6e006af00 msgr2=0x7fd6e006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:32.664 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.662+0000 7fd703c9b700 1 --2- 172.21.15.143:0/550671980 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd6e006af00 0x7fd6e006d3c0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fd6fc0c3b20 tx=0x7fd6ec008030 comp rx=0 tx=0).stop 2023-11-08T19:27:32.664 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.662+0000 7fd703c9b700 1 -- 172.21.15.143:0/550671980 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd6fc0668e0 msgr2=0x7fd6fc155960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:32.664 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.662+0000 7fd703c9b700 1 --2- 172.21.15.143:0/550671980 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd6fc0668e0 0x7fd6fc155960 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fd6f800c7f0 tx=0x7fd6f800cbb0 comp rx=0 tx=0).stop 2023-11-08T19:27:32.665 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.662+0000 7fd703c9b700 1 -- 172.21.15.143:0/550671980 shutdown_connections 2023-11-08T19:27:32.665 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.662+0000 7fd703c9b700 1 --2- 172.21.15.143:0/550671980 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd6e006af00 0x7fd6e006d3c0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:32.665 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.662+0000 7fd703c9b700 1 --2- 172.21.15.143:0/550671980 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd6fc06df90 0x7fd6fc1e81c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:32.665 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.662+0000 7fd703c9b700 1 --2- 172.21.15.143:0/550671980 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd6fc0668e0 0x7fd6fc155960 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:32.666 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.662+0000 7fd703c9b700 1 --2- 172.21.15.143:0/550671980 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd6fc06dbb0 0x7fd6fc0c5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:32.668 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.662+0000 7fd703c9b700 1 -- 172.21.15.143:0/550671980 >> 172.21.15.143:0/550671980 conn(0x7fd6fc0b1a20 msgr2=0x7fd6fc157610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:32.668 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.662+0000 7fd703c9b700 1 -- 172.21.15.143:0/550671980 shutdown_connections 2023-11-08T19:27:32.668 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.662+0000 7fd703c9b700 1 -- 172.21.15.143:0/550671980 wait complete. 2023-11-08T19:27:32.676 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:27:32.798 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.797+0000 7fc978eae700 1 Processor -- start 2023-11-08T19:27:32.798 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.797+0000 7fc978eae700 1 -- start start 2023-11-08T19:27:32.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.798+0000 7fc978eae700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc97406dbb0 0x7fc9740ca470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:32.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.798+0000 7fc978eae700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fc9740668e0 0x7fc9740ca9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:32.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.798+0000 7fc978eae700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9740ce7f0 con 0x7fc97406df90 2023-11-08T19:27:32.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.798+0000 7fc978eae700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc974066db0 con 0x7fc9740668e0 2023-11-08T19:27:32.802 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.798+0000 7fc978eae700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9740af5e0 con 0x7fc97406dbb0 2023-11-08T19:27:32.802 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.798+0000 7fc971d9b700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fc9740668e0 0x7fc9740ca9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:32.802 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.798+0000 7fc97259c700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc97406dbb0 0x7fc9740ca470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:32.802 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.798+0000 7fc971d9b700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fc9740668e0 0x7fc9740ca9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.143:37114/0 (socket says 172.21.15.143:37114) 2023-11-08T19:27:32.802 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.798+0000 7fc971d9b700 1 -- 172.21.15.143:0/3850564828 learned_addr learned my addr 172.21.15.143:0/3850564828 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:32.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.798+0000 7fc97259c700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc97406dbb0 0x7fc9740ca470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.143:41252/0 (socket says 172.21.15.143:41252) 2023-11-08T19:27:32.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.798+0000 7fc972d9d700 1 --1- 172.21.15.143:0/3850564828 >> v1:172.21.15.174:6789/0 conn(0x7fc97406df90 0x7fc9741621f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.174:6789/0 says I am v1:172.21.15.143:60706/0 (socket says 172.21.15.143:60706) 2023-11-08T19:27:32.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.798+0000 7fc97259c700 1 -- 172.21.15.143:0/3850564828 >> v2:172.21.15.174:3300/0 conn(0x7fc9740668e0 msgr2=0x7fc9740ca9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:32.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.798+0000 7fc97259c700 1 --2- 172.21.15.143:0/3850564828 >> v2:172.21.15.174:3300/0 conn(0x7fc9740668e0 0x7fc9740ca9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:32.804 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.798+0000 7fc97259c700 1 -- 172.21.15.143:0/3850564828 >> v1:172.21.15.174:6789/0 conn(0x7fc97406df90 legacy=0x7fc9741621f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:32.804 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.798+0000 7fc97259c700 1 -- 172.21.15.143:0/3850564828 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9740cb520 con 0x7fc97406dbb0 2023-11-08T19:27:32.804 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.798+0000 7fc971d9b700 1 --2- 172.21.15.143:0/3850564828 >> v2:172.21.15.174:3300/0 conn(0x7fc9740668e0 0x7fc9740ca9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T19:27:32.804 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.799+0000 7fc97259c700 1 --2- 172.21.15.143:0/3850564828 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc97406dbb0 0x7fc9740ca470 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc964009840 tx=0x7fc964009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8919dd205da3e512 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:32.804 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.799+0000 7fc97159a700 1 -- 172.21.15.143:0/3850564828 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc964011830 con 0x7fc97406dbb0 2023-11-08T19:27:32.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.799+0000 7fc97159a700 1 -- 172.21.15.143:0/3850564828 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc964011990 con 0x7fc97406dbb0 2023-11-08T19:27:32.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.799+0000 7fc97159a700 1 -- 172.21.15.143:0/3850564828 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc964011d20 con 0x7fc97406dbb0 2023-11-08T19:27:32.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.799+0000 7fc978eae700 1 -- 172.21.15.143:0/3850564828 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc97406dbb0 msgr2=0x7fc9740ca470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:32.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.799+0000 7fc978eae700 1 --2- 172.21.15.143:0/3850564828 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc97406dbb0 0x7fc9740ca470 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc964009840 tx=0x7fc964009b50 comp rx=0 tx=0).stop 2023-11-08T19:27:32.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.799+0000 7fc978eae700 1 -- 172.21.15.143:0/3850564828 shutdown_connections 2023-11-08T19:27:32.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.799+0000 7fc978eae700 1 --2- 172.21.15.143:0/3850564828 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc97406dbb0 0x7fc9740ca470 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:32.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.799+0000 7fc978eae700 1 --2- 172.21.15.143:0/3850564828 >> v2:172.21.15.174:3300/0 conn(0x7fc9740668e0 0x7fc9740ca9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:32.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.799+0000 7fc978eae700 1 -- 172.21.15.143:0/3850564828 >> 172.21.15.143:0/3850564828 conn(0x7fc9740b1a20 msgr2=0x7fc9740b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:32.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.799+0000 7fc978eae700 1 -- 172.21.15.143:0/3850564828 shutdown_connections 2023-11-08T19:27:32.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.799+0000 7fc978eae700 1 -- 172.21.15.143:0/3850564828 wait complete. 2023-11-08T19:27:32.809 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.801+0000 7fc978eae700 1 Processor -- start 2023-11-08T19:27:32.810 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.801+0000 7fc978eae700 1 -- start start 2023-11-08T19:27:32.810 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.801+0000 7fc978eae700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc9740668e0 0x7fc9741f6b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:32.810 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.801+0000 7fc978eae700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc97406dbb0 0x7fc9740b4190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:32.810 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.801+0000 7fc978eae700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc97406df90 0x7fc97415efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:32.811 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.801+0000 7fc978eae700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9741f7070 con 0x7fc9740668e0 2023-11-08T19:27:32.811 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.801+0000 7fc978eae700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9740b4ac0 con 0x7fc97406dbb0 2023-11-08T19:27:32.811 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.801+0000 7fc978eae700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9740b4da0 con 0x7fc97406df90 2023-11-08T19:27:32.811 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.801+0000 7fc97259c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc9740668e0 0x7fc9741f6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:32.811 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.801+0000 7fc97259c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc9740668e0 0x7fc9741f6b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53620/0 (socket says 172.21.15.143:53620) 2023-11-08T19:27:32.812 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.801+0000 7fc97259c700 1 -- 172.21.15.143:0/2490485900 learned_addr learned my addr 172.21.15.143:0/2490485900 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:32.812 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.802+0000 7fc971d9b700 1 --2- 172.21.15.143:0/2490485900 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc97406dbb0 0x7fc9740b4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:32.812 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.802+0000 7fc972d9d700 1 --2- 172.21.15.143:0/2490485900 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc97406df90 0x7fc97415efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:32.812 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.802+0000 7fc97259c700 1 -- 172.21.15.143:0/2490485900 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc97406df90 msgr2=0x7fc97415efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:32.812 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.802+0000 7fc97259c700 1 --2- 172.21.15.143:0/2490485900 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc97406df90 0x7fc97415efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:32.813 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.802+0000 7fc97259c700 1 -- 172.21.15.143:0/2490485900 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc97406dbb0 msgr2=0x7fc9740b4190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:32.813 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.802+0000 7fc97259c700 1 --2- 172.21.15.143:0/2490485900 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc97406dbb0 0x7fc9740b4190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:32.813 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.802+0000 7fc97259c700 1 -- 172.21.15.143:0/2490485900 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc968009550 con 0x7fc9740668e0 2023-11-08T19:27:32.813 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.802+0000 7fc971d9b700 1 --2- 172.21.15.143:0/2490485900 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc97406dbb0 0x7fc9740b4190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:27:32.813 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.802+0000 7fc972d9d700 1 --2- 172.21.15.143:0/2490485900 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc97406df90 0x7fc97415efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:27:32.814 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.802+0000 7fc97259c700 1 --2- 172.21.15.143:0/2490485900 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc9740668e0 0x7fc9741f6b30 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fc964009fd0 tx=0x7fc964004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:32.814 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.802+0000 7fc962ffd700 1 -- 172.21.15.143:0/2490485900 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9640036a0 con 0x7fc9740668e0 2023-11-08T19:27:32.814 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.802+0000 7fc962ffd700 1 -- 172.21.15.143:0/2490485900 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc96401f070 con 0x7fc9740668e0 2023-11-08T19:27:32.814 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.802+0000 7fc978eae700 1 -- 172.21.15.143:0/2490485900 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9640094a0 con 0x7fc9740668e0 2023-11-08T19:27:32.814 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.802+0000 7fc962ffd700 1 -- 172.21.15.143:0/2490485900 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc964003a60 con 0x7fc9740668e0 2023-11-08T19:27:32.815 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.802+0000 7fc978eae700 1 -- 172.21.15.143:0/2490485900 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9740b5360 con 0x7fc9740668e0 2023-11-08T19:27:32.815 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.804+0000 7fc962ffd700 1 -- 172.21.15.143:0/2490485900 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc964003bc0 con 0x7fc9740668e0 2023-11-08T19:27:32.815 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.804+0000 7fc978eae700 1 -- 172.21.15.143:0/2490485900 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc97406d1a0 con 0x7fc9740668e0 2023-11-08T19:27:32.815 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.804+0000 7fc962ffd700 1 --2- 172.21.15.143:0/2490485900 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc95406af00 0x7fc95406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:32.815 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.804+0000 7fc962ffd700 1 -- 172.21.15.143:0/2490485900 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6772+0+0 (secure 0 0 0) 0x7fc964021070 con 0x7fc9740668e0 2023-11-08T19:27:32.816 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.804+0000 7fc971d9b700 1 --2- 172.21.15.143:0/2490485900 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc95406af00 0x7fc95406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:32.816 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.805+0000 7fc971d9b700 1 --2- 172.21.15.143:0/2490485900 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc95406af00 0x7fc95406d3c0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fc97406c530 tx=0x7fc968008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:32.816 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.808+0000 7fc962ffd700 1 -- 172.21.15.143:0/2490485900 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc964055fd0 con 0x7fc9740668e0 2023-11-08T19:27:32.999 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.998+0000 7fc978eae700 1 -- 172.21.15.143:0/2490485900 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc9740b59b0 con 0x7fc9740668e0 2023-11-08T19:27:33.001 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:32.999+0000 7fc962ffd700 1 -- 172.21.15.143:0/2490485900 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) v1 ==== 74+0+21817 (secure 0 0 0) 0x7fc9640595f0 con 0x7fc9740668e0 2023-11-08T19:27:33.001 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:27:33.001 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":77,"fsid":"968483af-77fc-4068-ad7c-5c2a1a4613d7","created":"2023-11-08T19:24:32.826484+0000","modified":"2023-11-08T19:27:31.702217+0000","last_up_change":"2023-11-08T19:24:43.573880+0000","last_in_change":"2023-11-08T19:24:35.314525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:44.551530+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"snap-test-pool","create_time":"2023-11-08T19:26:02.874384+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":36,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"test_data_pool","create_time":"2023-11-08T19:27:05.601949+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":66,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_metadata_pool","create_time":"2023-11-08T19:27:06.536369+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-11-08T19:27:30.137037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-11-08T19:27:30.660318+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bae98ec1-d94e-403d-a220-a522a53ea6f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6809","nonce":1487830566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6811","nonce":1487830566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6815","nonce":1487830566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6813","nonce":1487830566}]},"public_addr":"172.21.15.143:6809/1487830566","cluster_addr":"172.21.15.143:6811/1487830566","heartbeat_back_addr":"172.21.15.143:6815/1487830566","heartbeat_front_addr":"172.21.15.143:6813/1487830566","state":["exists","up"]},{"osd":1,"uuid":"5cb6538a-1ccb-4322-8633-09f66af0b7e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6801","nonce":787341397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6803","nonce":787341397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6807","nonce":787341397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6805","nonce":787341397}]},"public_addr":"172.21.15.143:6801/787341397","cluster_addr":"172.21.15.143:6803/787341397","heartbeat_back_addr":"172.21.15.143:6807/787341397","heartbeat_front_addr":"172.21.15.143:6805/787341397","state":["exists","up"]},{"osd":2,"uuid":"cdbf522c-60ef-4d60-9653-8dcb529c101a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6822","nonce":2301325597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6826","nonce":2301325597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6831","nonce":2301325597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6829","nonce":2301325597}]},"public_addr":"172.21.15.143:6822/2301325597","cluster_addr":"172.21.15.143:6826/2301325597","heartbeat_back_addr":"172.21.15.143:6831/2301325597","heartbeat_front_addr":"172.21.15.143:6829/2301325597","state":["exists","up"]},{"osd":3,"uuid":"4e2783cf-ebb9-4dd8-8245-bc1d17ce4325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6817","nonce":2021504428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6819","nonce":2021504428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6827","nonce":2021504428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6823","nonce":2021504428}]},"public_addr":"172.21.15.143:6817/2021504428","cluster_addr":"172.21.15.143:6819/2021504428","heartbeat_back_addr":"172.21.15.143:6827/2021504428","heartbeat_front_addr":"172.21.15.143:6823/2021504428","state":["exists","up"]},{"osd":4,"uuid":"691256fa-9b77-49b9-a346-7ae1adf0ca80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6801","nonce":1993380595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6803","nonce":1993380595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6807","nonce":1993380595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6805","nonce":1993380595}]},"public_addr":"172.21.15.174:6801/1993380595","cluster_addr":"172.21.15.174:6803/1993380595","heartbeat_back_addr":"172.21.15.174:6807/1993380595","heartbeat_front_addr":"172.21.15.174:6805/1993380595","state":["exists","up"]},{"osd":5,"uuid":"756fc352-48c9-46ce-b52e-0541fe6f8525","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6809","nonce":1777923303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6811","nonce":1777923303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6815","nonce":1777923303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6813","nonce":1777923303}]},"public_addr":"172.21.15.174:6809/1777923303","cluster_addr":"172.21.15.174:6811/1777923303","heartbeat_back_addr":"172.21.15.174:6815/1777923303","heartbeat_front_addr":"172.21.15.174:6813/1777923303","state":["exists","up"]},{"osd":6,"uuid":"fab2ac2b-ed97-4cde-9825-b480eabf8b8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6825","nonce":2755130316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6827","nonce":2755130316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6831","nonce":2755130316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6829","nonce":2755130316}]},"public_addr":"172.21.15.174:6825/2755130316","cluster_addr":"172.21.15.174:6827/2755130316","heartbeat_back_addr":"172.21.15.174:6831/2755130316","heartbeat_front_addr":"172.21.15.174:6829/2755130316","state":["exists","up"]},{"osd":7,"uuid":"3de9b797-2ab6-4d41-a863-7e7d638fd703","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6817","nonce":2847141411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6819","nonce":2847141411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6823","nonce":2847141411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6821","nonce":2847141411}]},"public_addr":"172.21.15.174:6817/2847141411","cluster_addr":"172.21.15.174:6819/2847141411","heartbeat_back_addr":"172.21.15.174:6823/2847141411","heartbeat_front_addr":"172.21.15.174:6821/2847141411","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.477076+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.496890+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.539305+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.454673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.538881+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.521294+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.520712+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.540404+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:27:33.003 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:33.001+0000 7fc978eae700 1 -- 172.21.15.143:0/2490485900 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc95406af00 msgr2=0x7fc95406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:33.003 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:33.001+0000 7fc978eae700 1 --2- 172.21.15.143:0/2490485900 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc95406af00 0x7fc95406d3c0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fc97406c530 tx=0x7fc968008040 comp rx=0 tx=0).stop 2023-11-08T19:27:33.003 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:33.001+0000 7fc978eae700 1 -- 172.21.15.143:0/2490485900 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc9740668e0 msgr2=0x7fc9741f6b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:33.004 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:33.001+0000 7fc978eae700 1 --2- 172.21.15.143:0/2490485900 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc9740668e0 0x7fc9741f6b30 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fc964009fd0 tx=0x7fc964004750 comp rx=0 tx=0).stop 2023-11-08T19:27:33.004 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:33.001+0000 7fc978eae700 1 -- 172.21.15.143:0/2490485900 shutdown_connections 2023-11-08T19:27:33.004 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:33.001+0000 7fc978eae700 1 --2- 172.21.15.143:0/2490485900 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc95406af00 0x7fc95406d3c0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:33.004 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:33.001+0000 7fc978eae700 1 --2- 172.21.15.143:0/2490485900 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc97406df90 0x7fc97415efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:33.004 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:33.001+0000 7fc978eae700 1 --2- 172.21.15.143:0/2490485900 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc9740668e0 0x7fc9741f6b30 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:33.005 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:33.001+0000 7fc978eae700 1 --2- 172.21.15.143:0/2490485900 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc97406dbb0 0x7fc9740b4190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:33.005 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:33.001+0000 7fc978eae700 1 -- 172.21.15.143:0/2490485900 >> 172.21.15.143:0/2490485900 conn(0x7fc9740b1a20 msgr2=0x7fc9740b01e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:33.005 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:33.002+0000 7fc978eae700 1 -- 172.21.15.143:0/2490485900 shutdown_connections 2023-11-08T19:27:33.006 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:33.002+0000 7fc978eae700 1 -- 172.21.15.143:0/2490485900 wait complete. 2023-11-08T19:27:33.016 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-08T19:27:33.139 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:33.137+0000 7fcbd8117700 1 Processor -- start 2023-11-08T19:27:33.139 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:33.138+0000 7fcbd8117700 1 -- start start 2023-11-08T19:27:33.141 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:33.138+0000 7fcbd8117700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fcbd006b240 0x7fcbd00caff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:33.141 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:33.138+0000 7fcbd8117700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbd0161890 con 0x7fcbd015dba0 2023-11-08T19:27:33.142 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:33.138+0000 7fcbd8117700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbd006b740 con 0x7fcbd00cb530 2023-11-08T19:27:33.142 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:33.138+0000 7fcbd8117700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbd006cd20 con 0x7fcbd006b240 2023-11-08T19:27:33.142 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:33.138+0000 7fcbd56b2700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fcbd006b240 0x7fcbd00caff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:33.142 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:33.139+0000 7fcbd5eb3700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fcbd015dba0 0x7fcbd006c610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:49782/0 (socket says 172.21.15.143:49782) 2023-11-08T19:27:33.142 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:33.139+0000 7fcbd5eb3700 1 -- 172.21.15.143:0/2123634515 learned_addr learned my addr 172.21.15.143:0/2123634515 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:27:33.143 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:33.139+0000 7fcbd56b2700 1 -- 172.21.15.143:0/2123634515 >> v1:172.21.15.174:6789/0 conn(0x7fcbd00cb530 legacy=0x7fcbd00b22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:33.143 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:33.139+0000 7fcbd56b2700 1 -- 172.21.15.143:0/2123634515 >> v1:172.21.15.143:6789/0 conn(0x7fcbd015dba0 legacy=0x7fcbd006c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:33.143 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:33.139+0000 7fcbd56b2700 1 -- 172.21.15.143:0/2123634515 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbd006ba20 con 0x7fcbd006b240 2023-11-08T19:27:33.143 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:33.139+0000 7fcbd4eb1700 1 -- 172.21.15.143:0/2123634515 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 280270344 0 0) 0x7fcbd0161890 con 0x7fcbd015dba0 2023-11-08T19:27:33.143 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:33.139+0000 7fcbd56b2700 1 --2- 172.21.15.143:0/2123634515 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcbd006b240 0x7fcbd00caff0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc009840 tx=0x7fcbcc009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87cafbc69f85b1b6 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:33.144 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:33.139+0000 7fcbd4eb1700 1 -- 172.21.15.143:0/2123634515 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbcc004230 con 0x7fcbd006b240 2023-11-08T19:27:33.144 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:33.139+0000 7fcbd4eb1700 1 -- 172.21.15.143:0/2123634515 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbcc004390 con 0x7fcbd006b240 2023-11-08T19:27:33.144 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:33.139+0000 7fcbd4eb1700 1 -- 172.21.15.143:0/2123634515 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbcc0036a0 con 0x7fcbd006b240 2023-11-08T19:27:33.144 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:33.140+0000 7fcbd8117700 1 -- 172.21.15.143:0/2123634515 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcbd006b240 msgr2=0x7fcbd00caff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:33.145 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:33.140+0000 7fcbd8117700 1 --2- 172.21.15.143:0/2123634515 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcbd006b240 0x7fcbd00caff0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc009840 tx=0x7fcbcc009b50 comp rx=0 tx=0).stop 2023-11-08T19:27:33.145 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:33.140+0000 7fcbd8117700 1 -- 172.21.15.143:0/2123634515 shutdown_connections 2023-11-08T19:27:33.145 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:33.140+0000 7fcbd8117700 1 --2- 172.21.15.143:0/2123634515 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcbd006b240 0x7fcbd00caff0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:33.145 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:33.140+0000 7fcbd8117700 1 -- 172.21.15.143:0/2123634515 >> 172.21.15.143:0/2123634515 conn(0x7fcbd00cd2e0 msgr2=0x7fcbd00cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:33.145 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:33.140+0000 7fcbd8117700 1 -- 172.21.15.143:0/2123634515 shutdown_connections 2023-11-08T19:27:33.145 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:33.140+0000 7fcbd8117700 1 -- 172.21.15.143:0/2123634515 wait complete. 2023-11-08T19:27:33.148 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:33.141+0000 7fcbd8117700 1 Processor -- start 2023-11-08T19:27:33.149 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:33.141+0000 7fcbd8117700 1 -- start start 2023-11-08T19:27:33.149 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:33.142+0000 7fcbd8117700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcbd006b240 0x7fcbd00b40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:33.149 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:33.142+0000 7fcbd8117700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcbd00cb530 0x7fcbd015f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:33.149 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:33.142+0000 7fcbd8117700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcbd015dba0 0x7fcbd015fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:33.149 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:33.142+0000 7fcbd8117700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd00c5530 con 0x7fcbd00cb530 2023-11-08T19:27:33.150 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:33.142+0000 7fcbd8117700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd0160150 con 0x7fcbd006b240 2023-11-08T19:27:33.150 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:33.142+0000 7fcbd8117700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd0160430 con 0x7fcbd015dba0 2023-11-08T19:27:33.150 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:33.142+0000 7fcbd56b2700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcbd00cb530 0x7fcbd015f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:33.150 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:33.142+0000 7fcbd56b2700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcbd00cb530 0x7fcbd015f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53636/0 (socket says 172.21.15.143:53636) 2023-11-08T19:27:33.150 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:33.142+0000 7fcbd56b2700 1 -- 172.21.15.143:0/3573954304 learned_addr learned my addr 172.21.15.143:0/3573954304 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:33.151 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:33.142+0000 7fcbd66b4700 1 --2- 172.21.15.143:0/3573954304 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcbd015dba0 0x7fcbd015fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:33.151 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:33.142+0000 7fcbd5eb3700 1 --2- 172.21.15.143:0/3573954304 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcbd006b240 0x7fcbd00b40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:33.151 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:33.142+0000 7fcbd56b2700 1 -- 172.21.15.143:0/3573954304 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcbd015dba0 msgr2=0x7fcbd015fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:33.151 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:33.142+0000 7fcbd56b2700 1 --2- 172.21.15.143:0/3573954304 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcbd015dba0 0x7fcbd015fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:33.151 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:33.142+0000 7fcbd56b2700 1 -- 172.21.15.143:0/3573954304 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcbd006b240 msgr2=0x7fcbd00b40e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:33.152 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:33.142+0000 7fcbd56b2700 1 --2- 172.21.15.143:0/3573954304 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcbd006b240 0x7fcbd00b40e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:33.152 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:33.142+0000 7fcbd56b2700 1 -- 172.21.15.143:0/3573954304 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbcc0094a0 con 0x7fcbd00cb530 2023-11-08T19:27:33.152 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:33.142+0000 7fcbd5eb3700 1 --2- 172.21.15.143:0/3573954304 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcbd006b240 0x7fcbd00b40e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:27:33.152 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:33.142+0000 7fcbd56b2700 1 --2- 172.21.15.143:0/3573954304 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcbd00cb530 0x7fcbd015f5e0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc009b90 tx=0x7fcbcc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:33.152 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:33.142+0000 7fcbc67fc700 1 -- 172.21.15.143:0/3573954304 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbcc011630 con 0x7fcbd00cb530 2023-11-08T19:27:33.152 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:33.142+0000 7fcbc67fc700 1 -- 172.21.15.143:0/3573954304 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbcc01e070 con 0x7fcbd00cb530 2023-11-08T19:27:33.153 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:33.142+0000 7fcbc67fc700 1 -- 172.21.15.143:0/3573954304 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbcc0119b0 con 0x7fcbd00cb530 2023-11-08T19:27:33.153 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:33.142+0000 7fcbd8117700 1 -- 172.21.15.143:0/3573954304 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbd0160710 con 0x7fcbd00cb530 2023-11-08T19:27:33.153 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:33.142+0000 7fcbd8117700 1 -- 172.21.15.143:0/3573954304 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbd0160cd0 con 0x7fcbd00cb530 2023-11-08T19:27:33.153 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:33.144+0000 7fcbd8117700 1 -- 172.21.15.143:0/3573954304 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbd0160ec0 con 0x7fcbd00cb530 2023-11-08T19:27:33.153 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:33.144+0000 7fcbc67fc700 1 -- 172.21.15.143:0/3573954304 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fcbcc011b10 con 0x7fcbd00cb530 2023-11-08T19:27:33.154 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:33.144+0000 7fcbc67fc700 1 --2- 172.21.15.143:0/3573954304 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fcbb406af00 0x7fcbb406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:33.154 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:33.144+0000 7fcbc67fc700 1 -- 172.21.15.143:0/3573954304 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6772+0+0 (secure 0 0 0) 0x7fcbcc08adb0 con 0x7fcbd00cb530 2023-11-08T19:27:33.154 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:33.144+0000 7fcbd5eb3700 1 --2- 172.21.15.143:0/3573954304 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fcbb406af00 0x7fcbb406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:33.154 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:33.145+0000 7fcbd5eb3700 1 --2- 172.21.15.143:0/3573954304 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fcbb406af00 0x7fcbb406d3c0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fcbc0007fd0 tx=0x7fcbc0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:33.155 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:33.148+0000 7fcbc67fc700 1 -- 172.21.15.143:0/3573954304 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcbcc055ac0 con 0x7fcbd00cb530 2023-11-08T19:27:33.291 INFO:tasks.ceph.mds.d.smithi174.stderr:2023-11-08T19:27:33.290+0000 7f24365d5700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-08T19:27:33.387 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:33.386+0000 7fcbd8117700 1 -- 172.21.15.143:0/3573954304 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fcbd004eee0 con 0x7fcbd00cb530 2023-11-08T19:27:34.333 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:34.332+0000 7fcbc67fc700 1 -- 172.21.15.143:0/3573954304 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v40) v1 ==== 118+0+0 (secure 0 0 0) 0x7fcbcc020020 con 0x7fcbd00cb530 2023-11-08T19:27:34.336 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:34.335+0000 7fcbd8117700 1 -- 172.21.15.143:0/3573954304 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fcbb406af00 msgr2=0x7fcbb406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:34.336 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:34.335+0000 7fcbd8117700 1 --2- 172.21.15.143:0/3573954304 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fcbb406af00 0x7fcbb406d3c0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fcbc0007fd0 tx=0x7fcbc0009040 comp rx=0 tx=0).stop 2023-11-08T19:27:34.337 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:34.335+0000 7fcbd8117700 1 -- 172.21.15.143:0/3573954304 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcbd00cb530 msgr2=0x7fcbd015f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:34.337 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:34.335+0000 7fcbd8117700 1 --2- 172.21.15.143:0/3573954304 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcbd00cb530 0x7fcbd015f5e0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc009b90 tx=0x7fcbcc004750 comp rx=0 tx=0).stop 2023-11-08T19:27:34.337 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:34.335+0000 7fcbd8117700 1 -- 172.21.15.143:0/3573954304 shutdown_connections 2023-11-08T19:27:34.337 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:34.335+0000 7fcbd8117700 1 --2- 172.21.15.143:0/3573954304 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fcbb406af00 0x7fcbb406d3c0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:34.337 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:34.335+0000 7fcbd8117700 1 --2- 172.21.15.143:0/3573954304 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcbd015dba0 0x7fcbd015fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:34.338 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:34.335+0000 7fcbd8117700 1 --2- 172.21.15.143:0/3573954304 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcbd006b240 0x7fcbd00b40e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:34.338 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:34.335+0000 7fcbd8117700 1 --2- 172.21.15.143:0/3573954304 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcbd00cb530 0x7fcbd015f5e0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:34.338 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:34.335+0000 7fcbd8117700 1 -- 172.21.15.143:0/3573954304 >> 172.21.15.143:0/3573954304 conn(0x7fcbd00cd2e0 msgr2=0x7fcbd00b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:34.338 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:34.335+0000 7fcbd8117700 1 -- 172.21.15.143:0/3573954304 shutdown_connections 2023-11-08T19:27:34.338 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:34.335+0000 7fcbd8117700 1 -- 172.21.15.143:0/3573954304 wait complete. 2023-11-08T19:27:34.351 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-08T19:27:34.351 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-08T19:27:34.479 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:34.478+0000 7f586b5f3700 1 Processor -- start 2023-11-08T19:27:34.479 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:34.478+0000 7f586b5f3700 1 -- start start 2023-11-08T19:27:34.483 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:34.479+0000 7f586b5f3700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f58640cc350 0x7f586415b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:34.483 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:34.479+0000 7f586b5f3700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5864072c40 con 0x7f5864072010 2023-11-08T19:27:34.483 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:34.479+0000 7f586b5f3700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f586415be90 con 0x7f5864158610 2023-11-08T19:27:34.483 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:34.479+0000 7f586b5f3700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58640cf150 con 0x7f58640cc350 2023-11-08T19:27:34.484 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:34.479+0000 7f5868b8e700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f5864072010 0x7f586406aff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:49798/0 (socket says 172.21.15.143:49798) 2023-11-08T19:27:34.484 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:34.479+0000 7f5868b8e700 1 -- 172.21.15.143:0/148811308 learned_addr learned my addr 172.21.15.143:0/148811308 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:27:34.484 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:34.479+0000 7f5869b90700 1 --2- 172.21.15.143:0/148811308 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f58640cc350 0x7f586415b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:34.485 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:34.479+0000 7f585bfff700 1 -- 172.21.15.143:0/148811308 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1695091767 0 0) 0x7f5864072c40 con 0x7f5864072010 2023-11-08T19:27:34.485 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:34.479+0000 7f585bfff700 1 -- 172.21.15.143:0/148811308 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f58500035b0 con 0x7f5864072010 2023-11-08T19:27:34.485 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:34.479+0000 7f585bfff700 1 -- 172.21.15.143:0/148811308 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4131734989 0 0) 0x7f58500035b0 con 0x7f5864072010 2023-11-08T19:27:34.485 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:34.479+0000 7f585bfff700 1 -- 172.21.15.143:0/148811308 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f58640cc350 msgr2=0x7f586415b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:34.485 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:34.479+0000 7f585bfff700 1 --2- 172.21.15.143:0/148811308 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f58640cc350 0x7f586415b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:34.486 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:34.479+0000 7f585bfff700 1 -- 172.21.15.143:0/148811308 >> v1:172.21.15.174:6789/0 conn(0x7f5864158610 legacy=0x7f5864071900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:34.486 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:34.479+0000 7f585bfff700 1 -- 172.21.15.143:0/148811308 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58640cf360 con 0x7f5864072010 2023-11-08T19:27:34.486 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:34.479+0000 7f585bfff700 1 -- 172.21.15.143:0/148811308 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4147756303 0 0) 0x7f5854001670 con 0x7f5864072010 2023-11-08T19:27:34.486 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:34.480+0000 7f585bfff700 1 -- 172.21.15.143:0/148811308 >> v1:172.21.15.143:6789/0 conn(0x7f5864072010 legacy=0x7f586406aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:34.486 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:34.480+0000 7f585bfff700 1 --2- 172.21.15.143:0/148811308 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f58640cc350 0x7f5850003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:34.487 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:34.480+0000 7f585bfff700 1 --2- 172.21.15.143:0/148811308 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5850003f70 0x7f5850004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:34.487 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:34.480+0000 7f585bfff700 1 --2- 172.21.15.143:0/148811308 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5850004d90 0x7f5850005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:34.487 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:34.480+0000 7f585bfff700 1 -- 172.21.15.143:0/148811308 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58640cf150 con 0x7f5850004d90 2023-11-08T19:27:34.487 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:34.480+0000 7f585bfff700 1 -- 172.21.15.143:0/148811308 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f585000ba70 con 0x7f5850003f70 2023-11-08T19:27:34.487 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:34.480+0000 7f585bfff700 1 -- 172.21.15.143:0/148811308 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f585000bc30 con 0x7f58640cc350 2023-11-08T19:27:34.488 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:34.480+0000 7f5869b90700 1 --2- 172.21.15.143:0/148811308 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5850004d90 0x7f5850005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:34.488 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:34.480+0000 7f586938f700 1 --2- 172.21.15.143:0/148811308 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f58640cc350 0x7f5850003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:34.488 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:34.480+0000 7f5868b8e700 1 --2- 172.21.15.143:0/148811308 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5850003f70 0x7f5850004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:34.488 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:34.480+0000 7f5869b90700 1 -- 172.21.15.143:0/148811308 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f58640cc350 msgr2=0x7f5850003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:34.488 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:34.480+0000 7f5869b90700 1 --2- 172.21.15.143:0/148811308 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f58640cc350 0x7f5850003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:34.489 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:34.480+0000 7f5869b90700 1 -- 172.21.15.143:0/148811308 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5850003f70 msgr2=0x7f5850004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:34.489 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:34.480+0000 7f5869b90700 1 --2- 172.21.15.143:0/148811308 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5850003f70 0x7f5850004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:34.489 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:34.480+0000 7f5869b90700 1 -- 172.21.15.143:0/148811308 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58500035b0 con 0x7f5850004d90 2023-11-08T19:27:34.489 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:34.480+0000 7f5869b90700 1 --2- 172.21.15.143:0/148811308 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5850004d90 0x7f5850005210 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f585c009f90 tx=0x7f5850005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12f03bf7e82192c4 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:34.489 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:34.480+0000 7f585bfff700 1 -- 172.21.15.143:0/148811308 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f585c00ebd0 con 0x7f5850004d90 2023-11-08T19:27:34.489 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:34.480+0000 7f585bfff700 1 -- 172.21.15.143:0/148811308 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f585c00f040 con 0x7f5850004d90 2023-11-08T19:27:34.490 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:34.480+0000 7f585bfff700 1 -- 172.21.15.143:0/148811308 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f585c014480 con 0x7f5850004d90 2023-11-08T19:27:34.490 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:34.481+0000 7f586b5f3700 1 -- 172.21.15.143:0/148811308 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5850004d90 msgr2=0x7f5850005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:34.490 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:34.481+0000 7f586b5f3700 1 --2- 172.21.15.143:0/148811308 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5850004d90 0x7f5850005210 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f585c009f90 tx=0x7f5850005880 comp rx=0 tx=0).stop 2023-11-08T19:27:34.490 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:34.481+0000 7f5869b90700 1 -- 172.21.15.143:0/148811308 reap_dead start 2023-11-08T19:27:34.491 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:34.481+0000 7f586b5f3700 1 -- 172.21.15.143:0/148811308 shutdown_connections 2023-11-08T19:27:34.491 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:34.481+0000 7f586b5f3700 1 -- 172.21.15.143:0/148811308 >> 172.21.15.143:0/148811308 conn(0x7f58640b1c70 msgr2=0x7f58640b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:34.491 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:34.481+0000 7f586b5f3700 1 -- 172.21.15.143:0/148811308 shutdown_connections 2023-11-08T19:27:34.492 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:34.481+0000 7f586b5f3700 1 -- 172.21.15.143:0/148811308 wait complete. 2023-11-08T19:27:34.495 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:34.482+0000 7f586b5f3700 1 Processor -- start 2023-11-08T19:27:34.495 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:34.482+0000 7f586b5f3700 1 -- start start 2023-11-08T19:27:34.495 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:34.483+0000 7f586b5f3700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5864147d20 0x7f5864148100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:34.495 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:34.483+0000 7f586b5f3700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f586415b2e0 0x7f58640c9360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:34.497 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:34.483+0000 7f586b5f3700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f58640c98a0 0x7f58641589f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:34.497 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:34.483+0000 7f586b5f3700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5864148640 con 0x7f586415b2e0 2023-11-08T19:27:34.497 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:34.483+0000 7f586b5f3700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f586415a820 con 0x7f5864147d20 2023-11-08T19:27:34.498 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:34.483+0000 7f586b5f3700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5864148830 con 0x7f58640c98a0 2023-11-08T19:27:34.498 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:34.483+0000 7f5868b8e700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f586415b2e0 0x7f58640c9360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:34.498 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:34.483+0000 7f5868b8e700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f586415b2e0 0x7f58640c9360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53656/0 (socket says 172.21.15.143:53656) 2023-11-08T19:27:34.498 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:34.483+0000 7f5868b8e700 1 -- 172.21.15.143:0/3231394106 learned_addr learned my addr 172.21.15.143:0/3231394106 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:34.498 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:34.483+0000 7f586938f700 1 --2- 172.21.15.143:0/3231394106 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5864147d20 0x7f5864148100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:34.499 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:34.483+0000 7f5869b90700 1 --2- 172.21.15.143:0/3231394106 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f58640c98a0 0x7f58641589f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:34.499 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:34.483+0000 7f5868b8e700 1 -- 172.21.15.143:0/3231394106 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f58640c98a0 msgr2=0x7f58641589f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:34.499 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:34.483+0000 7f5868b8e700 1 --2- 172.21.15.143:0/3231394106 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f58640c98a0 0x7f58641589f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:34.499 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:34.483+0000 7f5868b8e700 1 -- 172.21.15.143:0/3231394106 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5864147d20 msgr2=0x7f5864148100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:34.499 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:34.483+0000 7f5868b8e700 1 --2- 172.21.15.143:0/3231394106 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5864147d20 0x7f5864148100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:34.500 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:34.483+0000 7f5868b8e700 1 -- 172.21.15.143:0/3231394106 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5864148b10 con 0x7f586415b2e0 2023-11-08T19:27:34.500 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:34.483+0000 7f5868b8e700 1 --2- 172.21.15.143:0/3231394106 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f586415b2e0 0x7f58640c9360 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f58540011e0 tx=0x7f5854007570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:34.500 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:34.484+0000 7f5859ffb700 1 -- 172.21.15.143:0/3231394106 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f585400f5e0 con 0x7f586415b2e0 2023-11-08T19:27:34.500 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:34.484+0000 7f5859ffb700 1 -- 172.21.15.143:0/3231394106 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f585400f760 con 0x7f586415b2e0 2023-11-08T19:27:34.500 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:34.484+0000 7f586b5f3700 1 -- 172.21.15.143:0/3231394106 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5864148df0 con 0x7f586415b2e0 2023-11-08T19:27:34.501 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:34.484+0000 7f5859ffb700 1 -- 172.21.15.143:0/3231394106 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f585400fb10 con 0x7f586415b2e0 2023-11-08T19:27:34.501 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:34.484+0000 7f586b5f3700 1 -- 172.21.15.143:0/3231394106 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f586413f470 con 0x7f586415b2e0 2023-11-08T19:27:34.501 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:34.485+0000 7f586b5f3700 1 -- 172.21.15.143:0/3231394106 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f586404eee0 con 0x7f586415b2e0 2023-11-08T19:27:34.501 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:34.485+0000 7f5859ffb700 1 -- 172.21.15.143:0/3231394106 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f585400fc70 con 0x7f586415b2e0 2023-11-08T19:27:34.501 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:34.486+0000 7f5859ffb700 1 --2- 172.21.15.143:0/3231394106 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f584806af00 0x7f584806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:34.502 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:34.486+0000 7f5859ffb700 1 -- 172.21.15.143:0/3231394106 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6772+0+0 (secure 0 0 0) 0x7f5854089a30 con 0x7f586415b2e0 2023-11-08T19:27:34.502 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:34.486+0000 7f586938f700 1 --2- 172.21.15.143:0/3231394106 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f584806af00 0x7f584806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:34.502 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:34.486+0000 7f586938f700 1 --2- 172.21.15.143:0/3231394106 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f584806af00 0x7f584806d3c0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f5860007fd0 tx=0x7f5860009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:34.502 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:34.490+0000 7f5859ffb700 1 -- 172.21.15.143:0/3231394106 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5854054620 con 0x7f586415b2e0 2023-11-08T19:27:34.725 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:34.724+0000 7f586b5f3700 1 -- 172.21.15.143:0/3231394106 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f5864072720 con 0x7f586415b2e0 2023-11-08T19:27:35.334 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.333+0000 7f5859ffb700 1 -- 172.21.15.143:0/3231394106 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v41) v1 ==== 122+0+0 (secure 0 0 0) 0x7f585405e020 con 0x7f586415b2e0 2023-11-08T19:27:35.338 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.336+0000 7f586b5f3700 1 -- 172.21.15.143:0/3231394106 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f584806af00 msgr2=0x7f584806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:35.338 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.336+0000 7f586b5f3700 1 --2- 172.21.15.143:0/3231394106 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f584806af00 0x7f584806d3c0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f5860007fd0 tx=0x7f5860009040 comp rx=0 tx=0).stop 2023-11-08T19:27:35.338 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.336+0000 7f586b5f3700 1 -- 172.21.15.143:0/3231394106 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f586415b2e0 msgr2=0x7f58640c9360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:35.338 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.336+0000 7f586b5f3700 1 --2- 172.21.15.143:0/3231394106 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f586415b2e0 0x7f58640c9360 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f58540011e0 tx=0x7f5854007570 comp rx=0 tx=0).stop 2023-11-08T19:27:35.339 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.336+0000 7f586b5f3700 1 -- 172.21.15.143:0/3231394106 shutdown_connections 2023-11-08T19:27:35.339 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.336+0000 7f586b5f3700 1 --2- 172.21.15.143:0/3231394106 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f584806af00 0x7f584806d3c0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:35.339 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.336+0000 7f586b5f3700 1 --2- 172.21.15.143:0/3231394106 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f58640c98a0 0x7f58641589f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:35.339 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.336+0000 7f586b5f3700 1 --2- 172.21.15.143:0/3231394106 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5864147d20 0x7f5864148100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:35.339 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.336+0000 7f586b5f3700 1 --2- 172.21.15.143:0/3231394106 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f586415b2e0 0x7f58640c9360 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:35.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.336+0000 7f586b5f3700 1 -- 172.21.15.143:0/3231394106 >> 172.21.15.143:0/3231394106 conn(0x7f58640b1c70 msgr2=0x7f58640ca690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:35.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.336+0000 7f586b5f3700 1 -- 172.21.15.143:0/3231394106 shutdown_connections 2023-11-08T19:27:35.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.336+0000 7f586b5f3700 1 -- 172.21.15.143:0/3231394106 wait complete. 2023-11-08T19:27:35.352 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-08T19:27:35.352 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:27:35.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.474+0000 7f76f7608700 1 Processor -- start 2023-11-08T19:27:35.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.475+0000 7f76f7608700 1 -- start start 2023-11-08T19:27:35.479 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.475+0000 7f76f7608700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f76f015aa30 0x7f76f00cf510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:35.479 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.475+0000 7f76f7608700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f76f00b2210 0x7f76f00b26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:35.479 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.475+0000 7f76f7608700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f76f00cbe20 0x7f76f00cc270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:35.479 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.475+0000 7f76f7608700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76f00b2be0 con 0x7f76f00b2210 2023-11-08T19:27:35.479 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.475+0000 7f76f7608700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76f00cf080 con 0x7f76f015aa30 2023-11-08T19:27:35.480 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.475+0000 7f76f7608700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76f00ce470 con 0x7f76f00cbe20 2023-11-08T19:27:35.480 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.476+0000 7f76f4ba3700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f76f00b2210 0x7f76f00b26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:35.480 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.476+0000 7f76f4ba3700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f76f00b2210 0x7f76f00b26a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53662/0 (socket says 172.21.15.143:53662) 2023-11-08T19:27:35.480 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.476+0000 7f76f4ba3700 1 -- 172.21.15.143:0/3469960902 learned_addr learned my addr 172.21.15.143:0/3469960902 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:35.480 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.476+0000 7f76f4ba3700 1 -- 172.21.15.143:0/3469960902 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f76f00cbe20 msgr2=0x7f76f00cc270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:35.481 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.476+0000 7f76f4ba3700 1 --2- 172.21.15.143:0/3469960902 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f76f00cbe20 0x7f76f00cc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:35.481 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.476+0000 7f76f4ba3700 1 -- 172.21.15.143:0/3469960902 >> v2:172.21.15.174:3300/0 conn(0x7f76f015aa30 msgr2=0x7f76f00cf510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:35.481 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.476+0000 7f76f53a4700 1 --2- 172.21.15.143:0/3469960902 >> v2:172.21.15.174:3300/0 conn(0x7f76f015aa30 0x7f76f00cf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:35.481 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.476+0000 7f76f4ba3700 1 --2- 172.21.15.143:0/3469960902 >> v2:172.21.15.174:3300/0 conn(0x7f76f015aa30 0x7f76f00cf510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:35.481 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.476+0000 7f76f4ba3700 1 -- 172.21.15.143:0/3469960902 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76f00ce750 con 0x7f76f00b2210 2023-11-08T19:27:35.482 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.476+0000 7f76f4ba3700 1 --2- 172.21.15.143:0/3469960902 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f76f00b2210 0x7f76f00b26a0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f76f0071c90 tx=0x7f76ec009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=948b8109d35dcece server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:35.482 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.476+0000 7f76e7fff700 1 -- 172.21.15.143:0/3469960902 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76ec00b870 con 0x7f76f00b2210 2023-11-08T19:27:35.482 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.476+0000 7f76e7fff700 1 -- 172.21.15.143:0/3469960902 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76ec00b9d0 con 0x7f76f00b2210 2023-11-08T19:27:35.483 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.476+0000 7f76e7fff700 1 -- 172.21.15.143:0/3469960902 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76ec00bd60 con 0x7f76f00b2210 2023-11-08T19:27:35.483 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.477+0000 7f76f7608700 1 -- 172.21.15.143:0/3469960902 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f76f00b2210 msgr2=0x7f76f00b26a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:35.483 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.477+0000 7f76f7608700 1 --2- 172.21.15.143:0/3469960902 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f76f00b2210 0x7f76f00b26a0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f76f0071c90 tx=0x7f76ec009bd0 comp rx=0 tx=0).stop 2023-11-08T19:27:35.483 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.477+0000 7f76f7608700 1 -- 172.21.15.143:0/3469960902 shutdown_connections 2023-11-08T19:27:35.483 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.477+0000 7f76f7608700 1 --2- 172.21.15.143:0/3469960902 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f76f00cbe20 0x7f76f00cc270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:35.483 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.477+0000 7f76f7608700 1 --2- 172.21.15.143:0/3469960902 >> v2:172.21.15.174:3300/0 conn(0x7f76f015aa30 0x7f76f00cf510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:35.484 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.477+0000 7f76f7608700 1 --2- 172.21.15.143:0/3469960902 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f76f00b2210 0x7f76f00b26a0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:35.484 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.477+0000 7f76f7608700 1 -- 172.21.15.143:0/3469960902 >> 172.21.15.143:0/3469960902 conn(0x7f76f0099ab0 msgr2=0x7f76f00d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:35.484 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.477+0000 7f76f7608700 1 -- 172.21.15.143:0/3469960902 shutdown_connections 2023-11-08T19:27:35.484 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.477+0000 7f76f7608700 1 -- 172.21.15.143:0/3469960902 wait complete. 2023-11-08T19:27:35.487 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.478+0000 7f76f7608700 1 Processor -- start 2023-11-08T19:27:35.487 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.478+0000 7f76f7608700 1 -- start start 2023-11-08T19:27:35.488 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.479+0000 7f76f7608700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f76f00b2210 0x7f76f00c66b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:35.488 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.479+0000 7f76f7608700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f76f00cbe20 0x7f76f00b5e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:35.488 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.479+0000 7f76f7608700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f76f015aa30 0x7f76f01ee7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:35.488 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.479+0000 7f76f7608700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76f01f26e0 con 0x7f76f00cbe20 2023-11-08T19:27:35.489 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.479+0000 7f76f7608700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76f01f1ad0 con 0x7f76f015aa30 2023-11-08T19:27:35.489 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.479+0000 7f76f7608700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76f01f1db0 con 0x7f76f00b2210 2023-11-08T19:27:35.489 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.479+0000 7f76f4ba3700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f76f00cbe20 0x7f76f00b5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:35.489 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.479+0000 7f76f4ba3700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f76f00cbe20 0x7f76f00b5e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53670/0 (socket says 172.21.15.143:53670) 2023-11-08T19:27:35.489 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.479+0000 7f76f4ba3700 1 -- 172.21.15.143:0/1255311116 learned_addr learned my addr 172.21.15.143:0/1255311116 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:35.490 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.479+0000 7f76f53a4700 1 --2- 172.21.15.143:0/1255311116 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f76f00b2210 0x7f76f00c66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:35.490 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.479+0000 7f76f5ba5700 1 --2- 172.21.15.143:0/1255311116 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f76f015aa30 0x7f76f01ee7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:35.490 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.479+0000 7f76f4ba3700 1 -- 172.21.15.143:0/1255311116 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f76f00b2210 msgr2=0x7f76f00c66b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:35.490 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.479+0000 7f76f4ba3700 1 --2- 172.21.15.143:0/1255311116 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f76f00b2210 0x7f76f00c66b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:35.490 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.479+0000 7f76f4ba3700 1 -- 172.21.15.143:0/1255311116 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f76f015aa30 msgr2=0x7f76f01ee7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:35.491 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.479+0000 7f76f4ba3700 1 --2- 172.21.15.143:0/1255311116 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f76f015aa30 0x7f76f01ee7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:35.491 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.479+0000 7f76f4ba3700 1 -- 172.21.15.143:0/1255311116 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76ec009550 con 0x7f76f00cbe20 2023-11-08T19:27:35.491 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.479+0000 7f76f4ba3700 1 --2- 172.21.15.143:0/1255311116 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f76f00cbe20 0x7f76f00b5e50 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f76f0070ae0 tx=0x7f76ec004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:35.491 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.479+0000 7f76e5ffb700 1 -- 172.21.15.143:0/1255311116 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76ec004310 con 0x7f76f00cbe20 2023-11-08T19:27:35.491 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.479+0000 7f76e5ffb700 1 -- 172.21.15.143:0/1255311116 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76ec01f070 con 0x7f76f00cbe20 2023-11-08T19:27:35.492 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.479+0000 7f76e5ffb700 1 -- 172.21.15.143:0/1255311116 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76ec0036a0 con 0x7f76f00cbe20 2023-11-08T19:27:35.492 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.479+0000 7f76f7608700 1 -- 172.21.15.143:0/1255311116 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76f01f2090 con 0x7f76f00cbe20 2023-11-08T19:27:35.492 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.480+0000 7f76f7608700 1 -- 172.21.15.143:0/1255311116 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76f01f16f0 con 0x7f76f00cbe20 2023-11-08T19:27:35.493 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.481+0000 7f76f7608700 1 -- 172.21.15.143:0/1255311116 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76f00c77d0 con 0x7f76f00cbe20 2023-11-08T19:27:35.493 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.481+0000 7f76e5ffb700 1 -- 172.21.15.143:0/1255311116 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f76ec021030 con 0x7f76f00cbe20 2023-11-08T19:27:35.493 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.482+0000 7f76e5ffb700 1 --2- 172.21.15.143:0/1255311116 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f76d406af00 0x7f76d406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:35.493 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.482+0000 7f76e5ffb700 1 -- 172.21.15.143:0/1255311116 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6772+0+0 (secure 0 0 0) 0x7f76ec08b090 con 0x7f76f00cbe20 2023-11-08T19:27:35.494 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.482+0000 7f76f53a4700 1 --2- 172.21.15.143:0/1255311116 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f76d406af00 0x7f76d406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:35.494 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.482+0000 7f76f53a4700 1 --2- 172.21.15.143:0/1255311116 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f76d406af00 0x7f76d406d3c0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f76e0006fd0 tx=0x7f76e0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:35.494 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.486+0000 7f76e5ffb700 1 -- 172.21.15.143:0/1255311116 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f76ec055e80 con 0x7f76f00cbe20 2023-11-08T19:27:35.722 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.718+0000 7f76f7608700 1 -- 172.21.15.143:0/1255311116 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f76f00c77d0 con 0x7f76f00cbe20 2023-11-08T19:27:35.722 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.719+0000 7f76e5ffb700 1 -- 172.21.15.143:0/1255311116 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+4921 (secure 0 0 0) 0x7f76f00c77d0 con 0x7f76f00cbe20 2023-11-08T19:27:35.723 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:27:35.723 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":41,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5510,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6836/3796015890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3796015890},{"type":"v1","addr":"172.21.15.174:6836","nonce":3796015890}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37},{"gid":6471,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6834/432456394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":432456394},{"type":"v1","addr":"172.21.15.143:6834","nonce":432456394}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37},{"gid":6474,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/310920972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":310920972},{"type":"v1","addr":"172.21.15.143:6835","nonce":310920972}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T19:27:31.702539+0000","modified":"2023-11-08T19:27:34.725623+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6489},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6489":{"gid":6489,"name":"d","rank":0,"incarnation":38,"state":"up:active","state_seq":3,"addr":"172.21.15.174:6837/1950356087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6835","nonce":1950356087},{"type":"v1","addr":"172.21.15.174:6837","nonce":1950356087}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-08T19:27:35.724 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.721+0000 7f76f7608700 1 -- 172.21.15.143:0/1255311116 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f76d406af00 msgr2=0x7f76d406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:35.724 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.721+0000 7f76f7608700 1 --2- 172.21.15.143:0/1255311116 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f76d406af00 0x7f76d406d3c0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f76e0006fd0 tx=0x7f76e0008040 comp rx=0 tx=0).stop 2023-11-08T19:27:35.724 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.721+0000 7f76f7608700 1 -- 172.21.15.143:0/1255311116 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f76f00cbe20 msgr2=0x7f76f00b5e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:35.725 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.721+0000 7f76f7608700 1 --2- 172.21.15.143:0/1255311116 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f76f00cbe20 0x7f76f00b5e50 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f76f0070ae0 tx=0x7f76ec004750 comp rx=0 tx=0).stop 2023-11-08T19:27:35.725 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.721+0000 7f76f7608700 1 -- 172.21.15.143:0/1255311116 shutdown_connections 2023-11-08T19:27:35.725 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.721+0000 7f76f7608700 1 --2- 172.21.15.143:0/1255311116 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f76d406af00 0x7f76d406d3c0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:35.725 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.721+0000 7f76f7608700 1 --2- 172.21.15.143:0/1255311116 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f76f00b2210 0x7f76f00c66b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:35.725 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.721+0000 7f76f7608700 1 --2- 172.21.15.143:0/1255311116 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f76f015aa30 0x7f76f01ee7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:35.726 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.721+0000 7f76f7608700 1 --2- 172.21.15.143:0/1255311116 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f76f00cbe20 0x7f76f00b5e50 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:35.726 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.721+0000 7f76f7608700 1 -- 172.21.15.143:0/1255311116 >> 172.21.15.143:0/1255311116 conn(0x7f76f0099ab0 msgr2=0x7f76f0159d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:35.726 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.721+0000 7f76f7608700 1 -- 172.21.15.143:0/1255311116 shutdown_connections 2023-11-08T19:27:35.726 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.721+0000 7f76f7608700 1 -- 172.21.15.143:0/1255311116 wait complete. 2023-11-08T19:27:35.726 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 41 2023-11-08T19:27:35.735 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:27:35.855 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.853+0000 7fa87090c700 1 Processor -- start 2023-11-08T19:27:35.855 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.853+0000 7fa87090c700 1 -- start start 2023-11-08T19:27:35.857 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.854+0000 7fa87090c700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa86c15dd00 0x7fa86c0cca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:35.858 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.854+0000 7fa87090c700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa86c06b320 0x7fa86c0cb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:35.858 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.854+0000 7fa87090c700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa86c0ccfb0 con 0x7fa86c1618f0 2023-11-08T19:27:35.858 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.854+0000 7fa87090c700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa86c0cad50 con 0x7fa86c15dd00 2023-11-08T19:27:35.858 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.854+0000 7fa87090c700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa86c161df0 con 0x7fa86c06b320 2023-11-08T19:27:35.859 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.855+0000 7fa86a59c700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa86c15dd00 0x7fa86c0cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:35.859 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.855+0000 7fa86a59c700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa86c15dd00 0x7fa86c0cca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53676/0 (socket says 172.21.15.143:53676) 2023-11-08T19:27:35.859 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.855+0000 7fa86a59c700 1 -- 172.21.15.143:0/1488748896 learned_addr learned my addr 172.21.15.143:0/1488748896 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:35.859 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.855+0000 7fa86ad9d700 1 --2- 172.21.15.143:0/1488748896 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa86c06b320 0x7fa86c0cb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:35.859 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.855+0000 7fa86a59c700 1 -- 172.21.15.143:0/1488748896 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa86c06b320 msgr2=0x7fa86c0cb1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:35.860 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.855+0000 7fa86a59c700 1 --2- 172.21.15.143:0/1488748896 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa86c06b320 0x7fa86c0cb1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:35.860 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.855+0000 7fa86a59c700 1 -- 172.21.15.143:0/1488748896 >> v1:172.21.15.174:6789/0 conn(0x7fa86c1618f0 legacy=0x7fa86c06ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:35.860 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.855+0000 7fa86a59c700 1 -- 172.21.15.143:0/1488748896 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa86c1620d0 con 0x7fa86c15dd00 2023-11-08T19:27:35.860 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.855+0000 7fa86a59c700 1 --2- 172.21.15.143:0/1488748896 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa86c15dd00 0x7fa86c0cca70 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fa85c006fd0 tx=0x7fa85c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23dd9e4cc612d80 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:35.860 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.855+0000 7fa86959a700 1 -- 172.21.15.143:0/1488748896 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa85c005e30 con 0x7fa86c15dd00 2023-11-08T19:27:35.861 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.855+0000 7fa86959a700 1 -- 172.21.15.143:0/1488748896 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa85c004030 con 0x7fa86c15dd00 2023-11-08T19:27:35.861 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.855+0000 7fa86959a700 1 -- 172.21.15.143:0/1488748896 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa85c0043e0 con 0x7fa86c15dd00 2023-11-08T19:27:35.861 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.856+0000 7fa87090c700 1 -- 172.21.15.143:0/1488748896 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa86c15dd00 msgr2=0x7fa86c0cca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:35.861 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.856+0000 7fa87090c700 1 --2- 172.21.15.143:0/1488748896 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa86c15dd00 0x7fa86c0cca70 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fa85c006fd0 tx=0x7fa85c009bd0 comp rx=0 tx=0).stop 2023-11-08T19:27:35.861 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.856+0000 7fa87090c700 1 -- 172.21.15.143:0/1488748896 shutdown_connections 2023-11-08T19:27:35.862 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.856+0000 7fa87090c700 1 --2- 172.21.15.143:0/1488748896 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa86c06b320 0x7fa86c0cb1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:35.862 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.856+0000 7fa87090c700 1 --2- 172.21.15.143:0/1488748896 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa86c15dd00 0x7fa86c0cca70 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:35.862 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.856+0000 7fa87090c700 1 -- 172.21.15.143:0/1488748896 >> 172.21.15.143:0/1488748896 conn(0x7fa86c099ab0 msgr2=0x7fa86c0d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:35.862 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.856+0000 7fa87090c700 1 -- 172.21.15.143:0/1488748896 shutdown_connections 2023-11-08T19:27:35.862 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.856+0000 7fa87090c700 1 -- 172.21.15.143:0/1488748896 wait complete. 2023-11-08T19:27:35.865 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.857+0000 7fa87090c700 1 Processor -- start 2023-11-08T19:27:35.866 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.857+0000 7fa87090c700 1 -- start start 2023-11-08T19:27:35.866 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.858+0000 7fa87090c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa86c06b320 0x7fa86c0b0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:35.866 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.858+0000 7fa87090c700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa86c15dd00 0x7fa86c0b2560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:35.866 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.858+0000 7fa87090c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa86c1618f0 0x7fa86c0b4070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:35.867 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.858+0000 7fa87090c700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa86c0b14e0 con 0x7fa86c1618f0 2023-11-08T19:27:35.867 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.858+0000 7fa87090c700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa86c0b2aa0 con 0x7fa86c06b320 2023-11-08T19:27:35.867 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.858+0000 7fa87090c700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa86c0b45b0 con 0x7fa86c15dd00 2023-11-08T19:27:35.867 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.858+0000 7fa86ad9d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa86c1618f0 0x7fa86c0b4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:35.867 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.858+0000 7fa86a59c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa86c06b320 0x7fa86c0b0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:35.868 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.858+0000 7fa86ad9d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa86c1618f0 0x7fa86c0b4070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53686/0 (socket says 172.21.15.143:53686) 2023-11-08T19:27:35.868 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.858+0000 7fa86ad9d700 1 -- 172.21.15.143:0/969703555 learned_addr learned my addr 172.21.15.143:0/969703555 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:35.868 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.858+0000 7fa869d9b700 1 --2- 172.21.15.143:0/969703555 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa86c15dd00 0x7fa86c0b2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:35.868 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.858+0000 7fa86ad9d700 1 -- 172.21.15.143:0/969703555 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa86c15dd00 msgr2=0x7fa86c0b2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:35.868 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.858+0000 7fa86ad9d700 1 --2- 172.21.15.143:0/969703555 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa86c15dd00 0x7fa86c0b2560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:35.869 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.858+0000 7fa86ad9d700 1 -- 172.21.15.143:0/969703555 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa86c06b320 msgr2=0x7fa86c0b0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:35.869 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.858+0000 7fa86ad9d700 1 --2- 172.21.15.143:0/969703555 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa86c06b320 0x7fa86c0b0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:35.869 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.858+0000 7fa86ad9d700 1 -- 172.21.15.143:0/969703555 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa85c009500 con 0x7fa86c1618f0 2023-11-08T19:27:35.869 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.859+0000 7fa86ad9d700 1 --2- 172.21.15.143:0/969703555 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa86c1618f0 0x7fa86c0b4070 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fa864007e30 tx=0x7fa86400e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:35.869 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.859+0000 7fa85affd700 1 -- 172.21.15.143:0/969703555 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa86400ed30 con 0x7fa86c1618f0 2023-11-08T19:27:35.870 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.859+0000 7fa85affd700 1 -- 172.21.15.143:0/969703555 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa86400ee90 con 0x7fa86c1618f0 2023-11-08T19:27:35.870 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.859+0000 7fa87090c700 1 -- 172.21.15.143:0/969703555 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa86c0b1200 con 0x7fa86c1618f0 2023-11-08T19:27:35.870 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.859+0000 7fa85affd700 1 -- 172.21.15.143:0/969703555 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa864014660 con 0x7fa86c1618f0 2023-11-08T19:27:35.870 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.859+0000 7fa87090c700 1 -- 172.21.15.143:0/969703555 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa86c0c46b0 con 0x7fa86c1618f0 2023-11-08T19:27:35.871 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.860+0000 7fa87090c700 1 -- 172.21.15.143:0/969703555 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa86c0aff60 con 0x7fa86c1618f0 2023-11-08T19:27:35.871 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.860+0000 7fa85affd700 1 -- 172.21.15.143:0/969703555 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa8640147c0 con 0x7fa86c1618f0 2023-11-08T19:27:35.871 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.860+0000 7fa85affd700 1 --2- 172.21.15.143:0/969703555 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa84c06af00 0x7fa84c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:35.871 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.861+0000 7fa85affd700 1 -- 172.21.15.143:0/969703555 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6772+0+0 (secure 0 0 0) 0x7fa86408a060 con 0x7fa86c1618f0 2023-11-08T19:27:35.871 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.861+0000 7fa86a59c700 1 --2- 172.21.15.143:0/969703555 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa84c06af00 0x7fa84c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:35.872 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.861+0000 7fa86a59c700 1 --2- 172.21.15.143:0/969703555 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa84c06af00 0x7fa84c06d3c0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fa85c000c00 tx=0x7fa85c008070 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:35.872 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:35.865+0000 7fa85affd700 1 -- 172.21.15.143:0/969703555 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa864054ce0 con 0x7fa86c1618f0 2023-11-08T19:27:36.057 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.056+0000 7fa87090c700 1 -- 172.21.15.143:0/969703555 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa86c0c4230 con 0x7fa86c1618f0 2023-11-08T19:27:36.059 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.058+0000 7fa85affd700 1 -- 172.21.15.143:0/969703555 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) v1 ==== 74+0+21804 (secure 0 0 0) 0x7fa86c0c4230 con 0x7fa86c1618f0 2023-11-08T19:27:36.059 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:27:36.060 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":78,"fsid":"968483af-77fc-4068-ad7c-5c2a1a4613d7","created":"2023-11-08T19:24:32.826484+0000","modified":"2023-11-08T19:27:33.302832+0000","last_up_change":"2023-11-08T19:24:43.573880+0000","last_in_change":"2023-11-08T19:24:35.314525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:44.551530+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"snap-test-pool","create_time":"2023-11-08T19:26:02.874384+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":36,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"test_data_pool","create_time":"2023-11-08T19:27:05.601949+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":66,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_metadata_pool","create_time":"2023-11-08T19:27:06.536369+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-11-08T19:27:30.137037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-11-08T19:27:30.660318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bae98ec1-d94e-403d-a220-a522a53ea6f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6809","nonce":1487830566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6811","nonce":1487830566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6815","nonce":1487830566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6813","nonce":1487830566}]},"public_addr":"172.21.15.143:6809/1487830566","cluster_addr":"172.21.15.143:6811/1487830566","heartbeat_back_addr":"172.21.15.143:6815/1487830566","heartbeat_front_addr":"172.21.15.143:6813/1487830566","state":["exists","up"]},{"osd":1,"uuid":"5cb6538a-1ccb-4322-8633-09f66af0b7e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6801","nonce":787341397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6803","nonce":787341397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6807","nonce":787341397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6805","nonce":787341397}]},"public_addr":"172.21.15.143:6801/787341397","cluster_addr":"172.21.15.143:6803/787341397","heartbeat_back_addr":"172.21.15.143:6807/787341397","heartbeat_front_addr":"172.21.15.143:6805/787341397","state":["exists","up"]},{"osd":2,"uuid":"cdbf522c-60ef-4d60-9653-8dcb529c101a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6822","nonce":2301325597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6826","nonce":2301325597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6831","nonce":2301325597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6829","nonce":2301325597}]},"public_addr":"172.21.15.143:6822/2301325597","cluster_addr":"172.21.15.143:6826/2301325597","heartbeat_back_addr":"172.21.15.143:6831/2301325597","heartbeat_front_addr":"172.21.15.143:6829/2301325597","state":["exists","up"]},{"osd":3,"uuid":"4e2783cf-ebb9-4dd8-8245-bc1d17ce4325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6817","nonce":2021504428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6819","nonce":2021504428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6827","nonce":2021504428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6823","nonce":2021504428}]},"public_addr":"172.21.15.143:6817/2021504428","cluster_addr":"172.21.15.143:6819/2021504428","heartbeat_back_addr":"172.21.15.143:6827/2021504428","heartbeat_front_addr":"172.21.15.143:6823/2021504428","state":["exists","up"]},{"osd":4,"uuid":"691256fa-9b77-49b9-a346-7ae1adf0ca80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6801","nonce":1993380595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6803","nonce":1993380595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6807","nonce":1993380595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6805","nonce":1993380595}]},"public_addr":"172.21.15.174:6801/1993380595","cluster_addr":"172.21.15.174:6803/1993380595","heartbeat_back_addr":"172.21.15.174:6807/1993380595","heartbeat_front_addr":"172.21.15.174:6805/1993380595","state":["exists","up"]},{"osd":5,"uuid":"756fc352-48c9-46ce-b52e-0541fe6f8525","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6809","nonce":1777923303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6811","nonce":1777923303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6815","nonce":1777923303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6813","nonce":1777923303}]},"public_addr":"172.21.15.174:6809/1777923303","cluster_addr":"172.21.15.174:6811/1777923303","heartbeat_back_addr":"172.21.15.174:6815/1777923303","heartbeat_front_addr":"172.21.15.174:6813/1777923303","state":["exists","up"]},{"osd":6,"uuid":"fab2ac2b-ed97-4cde-9825-b480eabf8b8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6825","nonce":2755130316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6827","nonce":2755130316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6831","nonce":2755130316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6829","nonce":2755130316}]},"public_addr":"172.21.15.174:6825/2755130316","cluster_addr":"172.21.15.174:6827/2755130316","heartbeat_back_addr":"172.21.15.174:6831/2755130316","heartbeat_front_addr":"172.21.15.174:6829/2755130316","state":["exists","up"]},{"osd":7,"uuid":"3de9b797-2ab6-4d41-a863-7e7d638fd703","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6817","nonce":2847141411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6819","nonce":2847141411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6823","nonce":2847141411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6821","nonce":2847141411}]},"public_addr":"172.21.15.174:6817/2847141411","cluster_addr":"172.21.15.174:6819/2847141411","heartbeat_back_addr":"172.21.15.174:6823/2847141411","heartbeat_front_addr":"172.21.15.174:6821/2847141411","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.477076+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.496890+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.539305+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.454673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.538881+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.521294+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.520712+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.540404+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:27:36.062 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.060+0000 7fa87090c700 1 -- 172.21.15.143:0/969703555 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa84c06af00 msgr2=0x7fa84c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:36.062 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.060+0000 7fa87090c700 1 --2- 172.21.15.143:0/969703555 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa84c06af00 0x7fa84c06d3c0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fa85c000c00 tx=0x7fa85c008070 comp rx=0 tx=0).stop 2023-11-08T19:27:36.062 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.060+0000 7fa87090c700 1 -- 172.21.15.143:0/969703555 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa86c1618f0 msgr2=0x7fa86c0b4070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:36.062 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.060+0000 7fa87090c700 1 --2- 172.21.15.143:0/969703555 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa86c1618f0 0x7fa86c0b4070 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fa864007e30 tx=0x7fa86400e3c0 comp rx=0 tx=0).stop 2023-11-08T19:27:36.063 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.060+0000 7fa87090c700 1 -- 172.21.15.143:0/969703555 shutdown_connections 2023-11-08T19:27:36.063 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.060+0000 7fa87090c700 1 --2- 172.21.15.143:0/969703555 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa84c06af00 0x7fa84c06d3c0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:36.063 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.060+0000 7fa87090c700 1 --2- 172.21.15.143:0/969703555 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa86c1618f0 0x7fa86c0b4070 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:36.063 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.060+0000 7fa87090c700 1 --2- 172.21.15.143:0/969703555 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa86c06b320 0x7fa86c0b0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:36.063 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.060+0000 7fa87090c700 1 --2- 172.21.15.143:0/969703555 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa86c15dd00 0x7fa86c0b2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:36.063 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.060+0000 7fa87090c700 1 -- 172.21.15.143:0/969703555 >> 172.21.15.143:0/969703555 conn(0x7fa86c099ab0 msgr2=0x7fa86c0ad380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:36.064 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.060+0000 7fa87090c700 1 -- 172.21.15.143:0/969703555 shutdown_connections 2023-11-08T19:27:36.064 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.060+0000 7fa87090c700 1 -- 172.21.15.143:0/969703555 wait complete. 2023-11-08T19:27:36.074 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-08T19:27:36.075 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-08T19:27:36.195 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.193+0000 7f73e56ea700 1 Processor -- start 2023-11-08T19:27:36.195 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.194+0000 7f73e56ea700 1 -- start start 2023-11-08T19:27:36.198 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.194+0000 7f73e56ea700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f73e006c730 0x7f73e00cd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:36.198 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.194+0000 7f73e56ea700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f73e006c030 0x7f73e006cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:36.199 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.194+0000 7f73e56ea700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f73e015de80 0x7f73e00cbf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:36.199 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.194+0000 7f73e56ea700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73e00cd480 con 0x7f73e015de80 2023-11-08T19:27:36.199 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.194+0000 7f73e56ea700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73e006d050 con 0x7f73e006c730 2023-11-08T19:27:36.199 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.194+0000 7f73e56ea700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73e00cc870 con 0x7f73e006c030 2023-11-08T19:27:36.199 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.194+0000 7f73df7fe700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f73e015de80 0x7f73e00cbf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:36.199 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.194+0000 7f73df7fe700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f73e015de80 0x7f73e00cbf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53692/0 (socket says 172.21.15.143:53692) 2023-11-08T19:27:36.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.194+0000 7f73df7fe700 1 -- 172.21.15.143:0/3112441278 learned_addr learned my addr 172.21.15.143:0/3112441278 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:36.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.195+0000 7f73df7fe700 1 -- 172.21.15.143:0/3112441278 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f73e006c030 msgr2=0x7f73e006cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:36.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.195+0000 7f73de7fc700 1 --2- 172.21.15.143:0/3112441278 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f73e006c030 0x7f73e006cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:36.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.195+0000 7f73deffd700 1 --2- 172.21.15.143:0/3112441278 >> v2:172.21.15.174:3300/0 conn(0x7f73e006c730 0x7f73e00cd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:36.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.195+0000 7f73df7fe700 1 --2- 172.21.15.143:0/3112441278 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f73e006c030 0x7f73e006cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:36.201 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.195+0000 7f73df7fe700 1 -- 172.21.15.143:0/3112441278 >> v2:172.21.15.174:3300/0 conn(0x7f73e006c730 msgr2=0x7f73e00cd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:36.201 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.195+0000 7f73df7fe700 1 --2- 172.21.15.143:0/3112441278 >> v2:172.21.15.174:3300/0 conn(0x7f73e006c730 0x7f73e00cd910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:36.201 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.195+0000 7f73df7fe700 1 -- 172.21.15.143:0/3112441278 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73e00ccb50 con 0x7f73e015de80 2023-11-08T19:27:36.201 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.195+0000 7f73deffd700 1 --2- 172.21.15.143:0/3112441278 >> v2:172.21.15.174:3300/0 conn(0x7f73e006c730 0x7f73e00cd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:27:36.201 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.195+0000 7f73df7fe700 1 --2- 172.21.15.143:0/3112441278 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f73e015de80 0x7f73e00cbf00 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f73e008bd60 tx=0x7f73d800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c40e205ee6a91068 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:36.202 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.195+0000 7f73ddffb700 1 -- 172.21.15.143:0/3112441278 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73d800e070 con 0x7f73e015de80 2023-11-08T19:27:36.202 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.195+0000 7f73ddffb700 1 -- 172.21.15.143:0/3112441278 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73d800b9c0 con 0x7f73e015de80 2023-11-08T19:27:36.202 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.195+0000 7f73ddffb700 1 -- 172.21.15.143:0/3112441278 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73d800bd70 con 0x7f73e015de80 2023-11-08T19:27:36.202 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.196+0000 7f73e56ea700 1 -- 172.21.15.143:0/3112441278 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f73e015de80 msgr2=0x7f73e00cbf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:36.202 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.196+0000 7f73e56ea700 1 --2- 172.21.15.143:0/3112441278 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f73e015de80 0x7f73e00cbf00 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f73e008bd60 tx=0x7f73d800b330 comp rx=0 tx=0).stop 2023-11-08T19:27:36.203 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.196+0000 7f73e56ea700 1 -- 172.21.15.143:0/3112441278 shutdown_connections 2023-11-08T19:27:36.203 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.196+0000 7f73e56ea700 1 --2- 172.21.15.143:0/3112441278 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f73e015de80 0x7f73e00cbf00 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:36.203 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.196+0000 7f73e56ea700 1 --2- 172.21.15.143:0/3112441278 >> v2:172.21.15.174:3300/0 conn(0x7f73e006c730 0x7f73e00cd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:36.203 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.196+0000 7f73e56ea700 1 --2- 172.21.15.143:0/3112441278 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f73e006c030 0x7f73e006cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:36.203 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.196+0000 7f73e56ea700 1 -- 172.21.15.143:0/3112441278 >> 172.21.15.143:0/3112441278 conn(0x7f73e008b000 msgr2=0x7f73e009ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:36.204 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.196+0000 7f73e56ea700 1 -- 172.21.15.143:0/3112441278 shutdown_connections 2023-11-08T19:27:36.204 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.196+0000 7f73e56ea700 1 -- 172.21.15.143:0/3112441278 wait complete. 2023-11-08T19:27:36.204 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.197+0000 7f73e56ea700 1 Processor -- start 2023-11-08T19:27:36.204 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.197+0000 7f73e56ea700 1 -- start start 2023-11-08T19:27:36.207 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.198+0000 7f73e56ea700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f73e006c030 0x7f73e00b72c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:36.207 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.198+0000 7f73e56ea700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f73e006c730 0x7f73e0151fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:36.207 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.198+0000 7f73e56ea700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f73e015de80 0x7f73e00c99d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:36.208 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.198+0000 7f73e56ea700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73e0152520 con 0x7f73e006c730 2023-11-08T19:27:36.208 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.198+0000 7f73e56ea700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73e015f160 con 0x7f73e006c030 2023-11-08T19:27:36.208 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.198+0000 7f73e56ea700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73e015e5e0 con 0x7f73e015de80 2023-11-08T19:27:36.208 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.198+0000 7f73de7fc700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f73e006c730 0x7f73e0151fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:36.208 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.198+0000 7f73de7fc700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f73e006c730 0x7f73e0151fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53696/0 (socket says 172.21.15.143:53696) 2023-11-08T19:27:36.208 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.198+0000 7f73de7fc700 1 -- 172.21.15.143:0/306400928 learned_addr learned my addr 172.21.15.143:0/306400928 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:36.209 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.198+0000 7f73deffd700 1 --2- 172.21.15.143:0/306400928 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f73e006c030 0x7f73e00b72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:36.209 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.198+0000 7f73df7fe700 1 --2- 172.21.15.143:0/306400928 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f73e015de80 0x7f73e00c99d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:36.209 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.198+0000 7f73de7fc700 1 -- 172.21.15.143:0/306400928 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f73e015de80 msgr2=0x7f73e00c99d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:36.209 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.198+0000 7f73de7fc700 1 --2- 172.21.15.143:0/306400928 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f73e015de80 0x7f73e00c99d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:36.209 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.198+0000 7f73de7fc700 1 -- 172.21.15.143:0/306400928 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f73e006c030 msgr2=0x7f73e00b72c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:36.210 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.198+0000 7f73de7fc700 1 --2- 172.21.15.143:0/306400928 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f73e006c030 0x7f73e00b72c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:36.210 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.198+0000 7f73de7fc700 1 -- 172.21.15.143:0/306400928 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73d8009b30 con 0x7f73e006c730 2023-11-08T19:27:36.210 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.198+0000 7f73de7fc700 1 --2- 172.21.15.143:0/306400928 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f73e006c730 0x7f73e0151fe0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f73d000c7f0 tx=0x7f73d000cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:36.210 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.198+0000 7f73c77fe700 1 -- 172.21.15.143:0/306400928 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73d0011800 con 0x7f73e006c730 2023-11-08T19:27:36.210 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.198+0000 7f73c77fe700 1 -- 172.21.15.143:0/306400928 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73d0011960 con 0x7f73e006c730 2023-11-08T19:27:36.211 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.198+0000 7f73e56ea700 1 -- 172.21.15.143:0/306400928 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73e015e8c0 con 0x7f73e006c730 2023-11-08T19:27:36.211 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.198+0000 7f73c77fe700 1 -- 172.21.15.143:0/306400928 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73d0011cd0 con 0x7f73e006c730 2023-11-08T19:27:36.211 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.198+0000 7f73e56ea700 1 -- 172.21.15.143:0/306400928 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73e015ee80 con 0x7f73e006c730 2023-11-08T19:27:36.211 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.199+0000 7f73e56ea700 1 -- 172.21.15.143:0/306400928 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73e004eee0 con 0x7f73e006c730 2023-11-08T19:27:36.211 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.200+0000 7f73c77fe700 1 -- 172.21.15.143:0/306400928 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f73d0022350 con 0x7f73e006c730 2023-11-08T19:27:36.212 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.200+0000 7f73c77fe700 1 --2- 172.21.15.143:0/306400928 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f73cc06af00 0x7f73cc06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:36.212 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.200+0000 7f73c77fe700 1 -- 172.21.15.143:0/306400928 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6772+0+0 (secure 0 0 0) 0x7f73d0089960 con 0x7f73e006c730 2023-11-08T19:27:36.212 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.200+0000 7f73deffd700 1 --2- 172.21.15.143:0/306400928 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f73cc06af00 0x7f73cc06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:36.212 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.201+0000 7f73deffd700 1 --2- 172.21.15.143:0/306400928 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f73cc06af00 0x7f73cc06d3c0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f73d4006fd0 tx=0x7f73d4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:36.212 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.205+0000 7f73c77fe700 1 -- 172.21.15.143:0/306400928 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f73d0054550 con 0x7f73e006c730 2023-11-08T19:27:36.391 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.390+0000 7f73e56ea700 1 -- 172.21.15.143:0/306400928 --> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f73e00cce30 con 0x7f73cc06af00 2023-11-08T19:27:36.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.405+0000 7f73c77fe700 1 -- 172.21.15.143:0/306400928 <== mgr.4099 v2:172.21.15.174:6832/4037280178 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+528905 (secure 0 0 0) 0x7f73d400c050 con 0x7f73cc06af00 2023-11-08T19:27:36.418 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:27:36.425 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.409+0000 7f73e56ea700 1 -- 172.21.15.143:0/306400928 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f73cc06af00 msgr2=0x7f73cc06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:36.426 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.409+0000 7f73e56ea700 1 --2- 172.21.15.143:0/306400928 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f73cc06af00 0x7f73cc06d3c0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f73d4006fd0 tx=0x7f73d4008040 comp rx=0 tx=0).stop 2023-11-08T19:27:36.426 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.409+0000 7f73e56ea700 1 -- 172.21.15.143:0/306400928 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f73e006c730 msgr2=0x7f73e0151fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:36.427 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.409+0000 7f73e56ea700 1 --2- 172.21.15.143:0/306400928 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f73e006c730 0x7f73e0151fe0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f73d000c7f0 tx=0x7f73d000cbb0 comp rx=0 tx=0).stop 2023-11-08T19:27:36.427 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.409+0000 7f73e56ea700 1 -- 172.21.15.143:0/306400928 shutdown_connections 2023-11-08T19:27:36.427 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.409+0000 7f73e56ea700 1 --2- 172.21.15.143:0/306400928 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f73cc06af00 0x7f73cc06d3c0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:36.428 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.409+0000 7f73e56ea700 1 --2- 172.21.15.143:0/306400928 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f73e015de80 0x7f73e00c99d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:36.428 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.409+0000 7f73e56ea700 1 --2- 172.21.15.143:0/306400928 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f73e006c030 0x7f73e00b72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:36.428 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.409+0000 7f73e56ea700 1 --2- 172.21.15.143:0/306400928 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f73e006c730 0x7f73e0151fe0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:36.428 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.409+0000 7f73e56ea700 1 -- 172.21.15.143:0/306400928 >> 172.21.15.143:0/306400928 conn(0x7f73e008b000 msgr2=0x7f73e00d0640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:36.429 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.409+0000 7f73e56ea700 1 -- 172.21.15.143:0/306400928 shutdown_connections 2023-11-08T19:27:36.429 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.409+0000 7f73e56ea700 1 -- 172.21.15.143:0/306400928 wait complete. 2023-11-08T19:27:36.429 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2023-11-08T19:27:36.430 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":156,"stamp":"2023-11-08T19:27:34.553276+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592140,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":183,"num_write_kb":2421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":140,"ondisk_log_size":140,"up":344,"acting":344,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":251,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":224676,"kb_used_data":8452,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754750044,"statfs":{"total":773094113280,"available":772864045056,"internally_reserved":0,"allocated":8654848,"data_stored":4577708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1772,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.257715"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.335728+0000","last_change":"2023-11-08T19:27:32.328141+0000","last_active":"2023-11-08T19:27:32.335728+0000","last_peered":"2023-11-08T19:27:32.335728+0000","last_clean":"2023-11-08T19:27:32.335728+0000","last_became_active":"2023-11-08T19:27:32.327903+0000","last_became_peered":"2023-11-08T19:27:32.327903+0000","last_unstale":"2023-11-08T19:27:32.335728+0000","last_undegraded":"2023-11-08T19:27:32.335728+0000","last_fullsized":"2023-11-08T19:27:32.335728+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:05:01.010467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.331281+0000","last_change":"2023-11-08T19:27:32.324220+0000","last_active":"2023-11-08T19:27:32.331281+0000","last_peered":"2023-11-08T19:27:32.331281+0000","last_clean":"2023-11-08T19:27:32.331281+0000","last_became_active":"2023-11-08T19:27:32.323918+0000","last_became_peered":"2023-11-08T19:27:32.323918+0000","last_unstale":"2023-11-08T19:27:32.331281+0000","last_undegraded":"2023-11-08T19:27:32.331281+0000","last_fullsized":"2023-11-08T19:27:32.331281+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:46:55.585097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.336191+0000","last_change":"2023-11-08T19:27:32.335224+0000","last_active":"2023-11-08T19:27:32.336191+0000","last_peered":"2023-11-08T19:27:32.336191+0000","last_clean":"2023-11-08T19:27:32.336191+0000","last_became_active":"2023-11-08T19:27:32.334895+0000","last_became_peered":"2023-11-08T19:27:32.334895+0000","last_unstale":"2023-11-08T19:27:32.336191+0000","last_undegraded":"2023-11-08T19:27:32.336191+0000","last_fullsized":"2023-11-08T19:27:32.336191+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:28:17.551060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.332448+0000","last_change":"2023-11-08T19:27:32.325238+0000","last_active":"2023-11-08T19:27:32.332448+0000","last_peered":"2023-11-08T19:27:32.332448+0000","last_clean":"2023-11-08T19:27:32.332448+0000","last_became_active":"2023-11-08T19:27:32.324599+0000","last_became_peered":"2023-11-08T19:27:32.324599+0000","last_unstale":"2023-11-08T19:27:32.332448+0000","last_undegraded":"2023-11-08T19:27:32.332448+0000","last_fullsized":"2023-11-08T19:27:32.332448+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:18:25.098882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-08T19:27:33.308729+0000","last_change":"2023-11-08T19:27:32.328764+0000","last_active":"2023-11-08T19:27:33.308729+0000","last_peered":"2023-11-08T19:27:33.308729+0000","last_clean":"2023-11-08T19:27:33.308729+0000","last_became_active":"2023-11-08T19:27:32.328599+0000","last_became_peered":"2023-11-08T19:27:32.328599+0000","last_unstale":"2023-11-08T19:27:33.308729+0000","last_undegraded":"2023-11-08T19:27:33.308729+0000","last_fullsized":"2023-11-08T19:27:33.308729+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:41:33.954786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.332186+0000","last_change":"2023-11-08T19:27:32.326288+0000","last_active":"2023-11-08T19:27:32.332186+0000","last_peered":"2023-11-08T19:27:32.332186+0000","last_clean":"2023-11-08T19:27:32.332186+0000","last_became_active":"2023-11-08T19:27:32.326019+0000","last_became_peered":"2023-11-08T19:27:32.326019+0000","last_unstale":"2023-11-08T19:27:32.332186+0000","last_undegraded":"2023-11-08T19:27:32.332186+0000","last_fullsized":"2023-11-08T19:27:32.332186+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:38:32.359142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:27:31.294701+0000","last_change":"2023-11-08T19:27:31.294701+0000","last_active":"2023-11-08T19:27:31.294701+0000","last_peered":"2023-11-08T19:27:31.294701+0000","last_clean":"2023-11-08T19:27:31.294701+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:27:31.294701+0000","last_undegraded":"2023-11-08T19:27:31.294701+0000","last_fullsized":"2023-11-08T19:27:31.294701+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.335638+0000","last_change":"2023-11-08T19:27:32.332920+0000","last_active":"2023-11-08T19:27:32.335638+0000","last_peered":"2023-11-08T19:27:32.335638+0000","last_clean":"2023-11-08T19:27:32.335638+0000","last_became_active":"2023-11-08T19:27:32.331279+0000","last_became_peered":"2023-11-08T19:27:32.331279+0000","last_unstale":"2023-11-08T19:27:32.335638+0000","last_undegraded":"2023-11-08T19:27:32.335638+0000","last_fullsized":"2023-11-08T19:27:32.335638+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:43:12.503877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-08T19:27:33.309872+0000","last_change":"2023-11-08T19:27:32.323553+0000","last_active":"2023-11-08T19:27:33.309872+0000","last_peered":"2023-11-08T19:27:33.309872+0000","last_clean":"2023-11-08T19:27:33.309872+0000","last_became_active":"2023-11-08T19:27:32.323106+0000","last_became_peered":"2023-11-08T19:27:32.323106+0000","last_unstale":"2023-11-08T19:27:33.309872+0000","last_undegraded":"2023-11-08T19:27:33.309872+0000","last_fullsized":"2023-11-08T19:27:33.309872+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:13:16.149790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.321253+0000","last_change":"2023-11-08T19:27:32.317700+0000","last_active":"2023-11-08T19:27:32.321253+0000","last_peered":"2023-11-08T19:27:32.321253+0000","last_clean":"2023-11-08T19:27:32.321253+0000","last_became_active":"2023-11-08T19:27:32.317372+0000","last_became_peered":"2023-11-08T19:27:32.317372+0000","last_unstale":"2023-11-08T19:27:32.321253+0000","last_undegraded":"2023-11-08T19:27:32.321253+0000","last_fullsized":"2023-11-08T19:27:32.321253+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:00:28.648016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-08T19:27:33.310558+0000","last_change":"2023-11-08T19:27:32.324920+0000","last_active":"2023-11-08T19:27:33.310558+0000","last_peered":"2023-11-08T19:27:33.310558+0000","last_clean":"2023-11-08T19:27:33.310558+0000","last_became_active":"2023-11-08T19:27:32.324665+0000","last_became_peered":"2023-11-08T19:27:32.324665+0000","last_unstale":"2023-11-08T19:27:33.310558+0000","last_undegraded":"2023-11-08T19:27:33.310558+0000","last_fullsized":"2023-11-08T19:27:33.310558+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:21:39.056390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.331630+0000","last_change":"2023-11-08T19:27:32.327016+0000","last_active":"2023-11-08T19:27:32.331630+0000","last_peered":"2023-11-08T19:27:32.331630+0000","last_clean":"2023-11-08T19:27:32.331630+0000","last_became_active":"2023-11-08T19:27:32.326728+0000","last_became_peered":"2023-11-08T19:27:32.326728+0000","last_unstale":"2023-11-08T19:27:32.331630+0000","last_undegraded":"2023-11-08T19:27:32.331630+0000","last_fullsized":"2023-11-08T19:27:32.331630+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:00:52.038756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.333149+0000","last_change":"2023-11-08T19:27:32.330157+0000","last_active":"2023-11-08T19:27:32.333149+0000","last_peered":"2023-11-08T19:27:32.333149+0000","last_clean":"2023-11-08T19:27:32.333149+0000","last_became_active":"2023-11-08T19:27:32.329459+0000","last_became_peered":"2023-11-08T19:27:32.329459+0000","last_unstale":"2023-11-08T19:27:32.333149+0000","last_undegraded":"2023-11-08T19:27:32.333149+0000","last_fullsized":"2023-11-08T19:27:32.333149+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:21:03.212549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-08T19:27:33.308505+0000","last_change":"2023-11-08T19:27:32.333313+0000","last_active":"2023-11-08T19:27:33.308505+0000","last_peered":"2023-11-08T19:27:33.308505+0000","last_clean":"2023-11-08T19:27:33.308505+0000","last_became_active":"2023-11-08T19:27:32.329612+0000","last_became_peered":"2023-11-08T19:27:32.329612+0000","last_unstale":"2023-11-08T19:27:33.308505+0000","last_undegraded":"2023-11-08T19:27:33.308505+0000","last_fullsized":"2023-11-08T19:27:33.308505+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:46:55.585097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.332859+0000","last_change":"2023-11-08T19:27:32.328748+0000","last_active":"2023-11-08T19:27:32.332859+0000","last_peered":"2023-11-08T19:27:32.332859+0000","last_clean":"2023-11-08T19:27:32.332859+0000","last_became_active":"2023-11-08T19:27:32.328457+0000","last_became_peered":"2023-11-08T19:27:32.328457+0000","last_unstale":"2023-11-08T19:27:32.332859+0000","last_undegraded":"2023-11-08T19:27:32.332859+0000","last_fullsized":"2023-11-08T19:27:32.332859+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:04:14.754128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.332386+0000","last_change":"2023-11-08T19:27:32.328017+0000","last_active":"2023-11-08T19:27:32.332386+0000","last_peered":"2023-11-08T19:27:32.332386+0000","last_clean":"2023-11-08T19:27:32.332386+0000","last_became_active":"2023-11-08T19:27:32.327751+0000","last_became_peered":"2023-11-08T19:27:32.327751+0000","last_unstale":"2023-11-08T19:27:32.332386+0000","last_undegraded":"2023-11-08T19:27:32.332386+0000","last_fullsized":"2023-11-08T19:27:32.332386+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:33:46.073688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.334543+0000","last_change":"2023-11-08T19:27:32.333889+0000","last_active":"2023-11-08T19:27:32.334543+0000","last_peered":"2023-11-08T19:27:32.334543+0000","last_clean":"2023-11-08T19:27:32.334543+0000","last_became_active":"2023-11-08T19:27:32.333661+0000","last_became_peered":"2023-11-08T19:27:32.333661+0000","last_unstale":"2023-11-08T19:27:32.334543+0000","last_undegraded":"2023-11-08T19:27:32.334543+0000","last_fullsized":"2023-11-08T19:27:32.334543+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:10:39.776305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.335761+0000","last_change":"2023-11-08T19:27:32.324868+0000","last_active":"2023-11-08T19:27:32.335761+0000","last_peered":"2023-11-08T19:27:32.335761+0000","last_clean":"2023-11-08T19:27:32.335761+0000","last_became_active":"2023-11-08T19:27:32.323651+0000","last_became_peered":"2023-11-08T19:27:32.323651+0000","last_unstale":"2023-11-08T19:27:32.335761+0000","last_undegraded":"2023-11-08T19:27:32.335761+0000","last_fullsized":"2023-11-08T19:27:32.335761+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:50:42.784058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-08T19:27:33.310723+0000","last_change":"2023-11-08T19:27:32.333127+0000","last_active":"2023-11-08T19:27:33.310723+0000","last_peered":"2023-11-08T19:27:33.310723+0000","last_clean":"2023-11-08T19:27:33.310723+0000","last_became_active":"2023-11-08T19:27:32.329831+0000","last_became_peered":"2023-11-08T19:27:32.329831+0000","last_unstale":"2023-11-08T19:27:33.310723+0000","last_undegraded":"2023-11-08T19:27:33.310723+0000","last_fullsized":"2023-11-08T19:27:33.310723+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:32:28.687246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.333223+0000","last_change":"2023-11-08T19:27:32.330456+0000","last_active":"2023-11-08T19:27:32.333223+0000","last_peered":"2023-11-08T19:27:32.333223+0000","last_clean":"2023-11-08T19:27:32.333223+0000","last_became_active":"2023-11-08T19:27:32.329082+0000","last_became_peered":"2023-11-08T19:27:32.329082+0000","last_unstale":"2023-11-08T19:27:32.333223+0000","last_undegraded":"2023-11-08T19:27:32.333223+0000","last_fullsized":"2023-11-08T19:27:32.333223+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:52:05.082696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-08T19:27:33.308892+0000","last_change":"2023-11-08T19:27:32.329643+0000","last_active":"2023-11-08T19:27:33.308892+0000","last_peered":"2023-11-08T19:27:33.308892+0000","last_clean":"2023-11-08T19:27:33.308892+0000","last_became_active":"2023-11-08T19:27:32.329325+0000","last_became_peered":"2023-11-08T19:27:32.329325+0000","last_unstale":"2023-11-08T19:27:33.308892+0000","last_undegraded":"2023-11-08T19:27:33.308892+0000","last_fullsized":"2023-11-08T19:27:33.308892+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:45:46.211431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-08T19:27:33.310102+0000","last_change":"2023-11-08T19:27:32.329346+0000","last_active":"2023-11-08T19:27:33.310102+0000","last_peered":"2023-11-08T19:27:33.310102+0000","last_clean":"2023-11-08T19:27:33.310102+0000","last_became_active":"2023-11-08T19:27:32.329063+0000","last_became_peered":"2023-11-08T19:27:32.329063+0000","last_unstale":"2023-11-08T19:27:33.310102+0000","last_undegraded":"2023-11-08T19:27:33.310102+0000","last_fullsized":"2023-11-08T19:27:33.310102+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:12:54.533391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:27:31.294701+0000","last_change":"2023-11-08T19:27:31.294701+0000","last_active":"2023-11-08T19:27:31.294701+0000","last_peered":"2023-11-08T19:27:31.294701+0000","last_clean":"2023-11-08T19:27:31.294701+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:27:31.294701+0000","last_undegraded":"2023-11-08T19:27:31.294701+0000","last_fullsized":"2023-11-08T19:27:31.294701+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.335828+0000","last_change":"2023-11-08T19:27:32.332025+0000","last_active":"2023-11-08T19:27:32.335828+0000","last_peered":"2023-11-08T19:27:32.335828+0000","last_clean":"2023-11-08T19:27:32.335828+0000","last_became_active":"2023-11-08T19:27:32.331723+0000","last_became_peered":"2023-11-08T19:27:32.331723+0000","last_unstale":"2023-11-08T19:27:32.335828+0000","last_undegraded":"2023-11-08T19:27:32.335828+0000","last_fullsized":"2023-11-08T19:27:32.335828+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:00:21.681748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.331572+0000","last_change":"2023-11-08T19:27:32.326008+0000","last_active":"2023-11-08T19:27:32.331572+0000","last_peered":"2023-11-08T19:27:32.331572+0000","last_clean":"2023-11-08T19:27:32.331572+0000","last_became_active":"2023-11-08T19:27:32.325759+0000","last_became_peered":"2023-11-08T19:27:32.325759+0000","last_unstale":"2023-11-08T19:27:32.331572+0000","last_undegraded":"2023-11-08T19:27:32.331572+0000","last_fullsized":"2023-11-08T19:27:32.331572+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:51:54.379524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-08T19:27:33.308689+0000","last_change":"2023-11-08T19:27:32.333144+0000","last_active":"2023-11-08T19:27:33.308689+0000","last_peered":"2023-11-08T19:27:33.308689+0000","last_clean":"2023-11-08T19:27:33.308689+0000","last_became_active":"2023-11-08T19:27:32.329041+0000","last_became_peered":"2023-11-08T19:27:32.329041+0000","last_unstale":"2023-11-08T19:27:33.308689+0000","last_undegraded":"2023-11-08T19:27:33.308689+0000","last_fullsized":"2023-11-08T19:27:33.308689+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:51:54.379524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.324310+0000","last_change":"2023-11-08T19:27:32.322409+0000","last_active":"2023-11-08T19:27:32.324310+0000","last_peered":"2023-11-08T19:27:32.324310+0000","last_clean":"2023-11-08T19:27:32.324310+0000","last_became_active":"2023-11-08T19:27:32.322144+0000","last_became_peered":"2023-11-08T19:27:32.322144+0000","last_unstale":"2023-11-08T19:27:32.324310+0000","last_undegraded":"2023-11-08T19:27:32.324310+0000","last_fullsized":"2023-11-08T19:27:32.324310+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:52:54.828438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.332426+0000","last_change":"2023-11-08T19:27:32.326284+0000","last_active":"2023-11-08T19:27:32.332426+0000","last_peered":"2023-11-08T19:27:32.332426+0000","last_clean":"2023-11-08T19:27:32.332426+0000","last_became_active":"2023-11-08T19:27:32.326071+0000","last_became_peered":"2023-11-08T19:27:32.326071+0000","last_unstale":"2023-11-08T19:27:32.332426+0000","last_undegraded":"2023-11-08T19:27:32.332426+0000","last_fullsized":"2023-11-08T19:27:32.332426+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:30:14.195303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:27:31.294701+0000","last_change":"2023-11-08T19:27:31.294701+0000","last_active":"2023-11-08T19:27:31.294701+0000","last_peered":"2023-11-08T19:27:31.294701+0000","last_clean":"2023-11-08T19:27:31.294701+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:27:31.294701+0000","last_undegraded":"2023-11-08T19:27:31.294701+0000","last_fullsized":"2023-11-08T19:27:31.294701+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-08T19:27:33.310582+0000","last_change":"2023-11-08T19:27:32.323841+0000","last_active":"2023-11-08T19:27:33.310582+0000","last_peered":"2023-11-08T19:27:33.310582+0000","last_clean":"2023-11-08T19:27:33.310582+0000","last_became_active":"2023-11-08T19:27:32.323593+0000","last_became_peered":"2023-11-08T19:27:32.323593+0000","last_unstale":"2023-11-08T19:27:33.310582+0000","last_undegraded":"2023-11-08T19:27:33.310582+0000","last_fullsized":"2023-11-08T19:27:33.310582+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:36:25.623933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.322600+0000","last_change":"2023-11-08T19:27:32.318553+0000","last_active":"2023-11-08T19:27:32.322600+0000","last_peered":"2023-11-08T19:27:32.322600+0000","last_clean":"2023-11-08T19:27:32.322600+0000","last_became_active":"2023-11-08T19:27:32.318309+0000","last_became_peered":"2023-11-08T19:27:32.318309+0000","last_unstale":"2023-11-08T19:27:32.322600+0000","last_undegraded":"2023-11-08T19:27:32.322600+0000","last_fullsized":"2023-11-08T19:27:32.322600+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:30:14.195303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.332370+0000","last_change":"2023-11-08T19:27:32.327734+0000","last_active":"2023-11-08T19:27:32.332370+0000","last_peered":"2023-11-08T19:27:32.332370+0000","last_clean":"2023-11-08T19:27:32.332370+0000","last_became_active":"2023-11-08T19:27:32.327412+0000","last_became_peered":"2023-11-08T19:27:32.327412+0000","last_unstale":"2023-11-08T19:27:32.332370+0000","last_undegraded":"2023-11-08T19:27:32.332370+0000","last_fullsized":"2023-11-08T19:27:32.332370+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:50:42.784058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:27:31.294701+0000","last_change":"2023-11-08T19:27:31.294701+0000","last_active":"2023-11-08T19:27:31.294701+0000","last_peered":"2023-11-08T19:27:31.294701+0000","last_clean":"2023-11-08T19:27:31.294701+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:27:31.294701+0000","last_undegraded":"2023-11-08T19:27:31.294701+0000","last_fullsized":"2023-11-08T19:27:31.294701+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.314151+0000","last_change":"2023-11-08T19:27:31.323578+0000","last_active":"2023-11-08T19:27:32.314151+0000","last_peered":"2023-11-08T19:27:32.314151+0000","last_clean":"2023-11-08T19:27:32.314151+0000","last_became_active":"2023-11-08T19:27:31.323254+0000","last_became_peered":"2023-11-08T19:27:31.323254+0000","last_unstale":"2023-11-08T19:27:32.314151+0000","last_undegraded":"2023-11-08T19:27:32.314151+0000","last_fullsized":"2023-11-08T19:27:32.314151+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_clean_scrub_stamp":"2023-11-08T19:27:30.289497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:45:45.206227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":19,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-08T19:27:27.305121+0000","last_change":"2023-11-08T19:27:08.190826+0000","last_active":"2023-11-08T19:27:27.305121+0000","last_peered":"2023-11-08T19:27:27.305121+0000","last_clean":"2023-11-08T19:27:27.305121+0000","last_became_active":"2023-11-08T19:27:08.190440+0000","last_became_peered":"2023-11-08T19:27:08.190440+0000","last_unstale":"2023-11-08T19:27:27.305121+0000","last_undegraded":"2023-11-08T19:27:27.305121+0000","last_fullsized":"2023-11-08T19:27:27.305121+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:42:48.376462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":220,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-08T19:27:27.304931+0000","last_change":"2023-11-08T19:24:45.888122+0000","last_active":"2023-11-08T19:27:27.304931+0000","last_peered":"2023-11-08T19:27:27.304931+0000","last_clean":"2023-11-08T19:27:27.304931+0000","last_became_active":"2023-11-08T19:24:45.887048+0000","last_became_peered":"2023-11-08T19:24:45.887048+0000","last_unstale":"2023-11-08T19:27:27.304931+0000","last_undegraded":"2023-11-08T19:27:27.304931+0000","last_fullsized":"2023-11-08T19:27:27.304931+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:44.575133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:44.575133+0000","last_clean_scrub_stamp":"2023-11-08T19:24:44.575133+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:19:20.202062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":51,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-08T19:27:31.313997+0000","last_change":"2023-11-08T19:26:06.542044+0000","last_active":"2023-11-08T19:27:31.313997+0000","last_peered":"2023-11-08T19:27:31.313997+0000","last_clean":"2023-11-08T19:27:31.313997+0000","last_became_active":"2023-11-08T19:26:04.603056+0000","last_became_peered":"2023-11-08T19:26:04.603056+0000","last_unstale":"2023-11-08T19:27:31.313997+0000","last_undegraded":"2023-11-08T19:27:31.313997+0000","last_fullsized":"2023-11-08T19:27:31.313997+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:41:50.771457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000517414,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":28,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-08T19:27:31.313164+0000","last_change":"2023-11-08T19:27:09.496842+0000","last_active":"2023-11-08T19:27:31.313164+0000","last_peered":"2023-11-08T19:27:31.313164+0000","last_clean":"2023-11-08T19:27:31.313164+0000","last_became_active":"2023-11-08T19:27:07.185776+0000","last_became_peered":"2023-11-08T19:27:07.185776+0000","last_unstale":"2023-11-08T19:27:31.313164+0000","last_undegraded":"2023-11-08T19:27:31.313164+0000","last_fullsized":"2023-11-08T19:27:31.313164+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:21:13.923970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031747200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.327909+0000","last_change":"2023-11-08T19:27:32.321094+0000","last_active":"2023-11-08T19:27:32.327909+0000","last_peered":"2023-11-08T19:27:32.327909+0000","last_clean":"2023-11-08T19:27:32.327909+0000","last_became_active":"2023-11-08T19:27:32.320655+0000","last_became_peered":"2023-11-08T19:27:32.320655+0000","last_unstale":"2023-11-08T19:27:32.327909+0000","last_undegraded":"2023-11-08T19:27:32.327909+0000","last_fullsized":"2023-11-08T19:27:32.327909+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:41:33.954786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.308411+0000","last_change":"2023-11-08T19:27:31.322203+0000","last_active":"2023-11-08T19:27:32.308411+0000","last_peered":"2023-11-08T19:27:32.308411+0000","last_clean":"2023-11-08T19:27:32.308411+0000","last_became_active":"2023-11-08T19:27:31.321851+0000","last_became_peered":"2023-11-08T19:27:31.321851+0000","last_unstale":"2023-11-08T19:27:32.308411+0000","last_undegraded":"2023-11-08T19:27:32.308411+0000","last_fullsized":"2023-11-08T19:27:32.308411+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_clean_scrub_stamp":"2023-11-08T19:27:30.289497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:18:24.093678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":21,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-08T19:27:31.313099+0000","last_change":"2023-11-08T19:27:08.188241+0000","last_active":"2023-11-08T19:27:31.313099+0000","last_peered":"2023-11-08T19:27:31.313099+0000","last_clean":"2023-11-08T19:27:31.313099+0000","last_became_active":"2023-11-08T19:27:08.187863+0000","last_became_peered":"2023-11-08T19:27:08.187863+0000","last_unstale":"2023-11-08T19:27:31.313099+0000","last_undegraded":"2023-11-08T19:27:31.313099+0000","last_fullsized":"2023-11-08T19:27:31.313099+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:45:22.084015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":50,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.311772+0000","last_change":"2023-11-08T19:26:06.265419+0000","last_active":"2023-11-08T19:27:32.311772+0000","last_peered":"2023-11-08T19:27:32.311772+0000","last_clean":"2023-11-08T19:27:32.311772+0000","last_became_active":"2023-11-08T19:26:04.605333+0000","last_became_peered":"2023-11-08T19:26:04.605333+0000","last_unstale":"2023-11-08T19:27:32.311772+0000","last_undegraded":"2023-11-08T19:27:32.311772+0000","last_fullsized":"2023-11-08T19:27:32.311772+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:01:56.991346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064606400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":26,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-08T19:27:27.305282+0000","last_change":"2023-11-08T19:27:09.182113+0000","last_active":"2023-11-08T19:27:27.305282+0000","last_peered":"2023-11-08T19:27:27.305282+0000","last_clean":"2023-11-08T19:27:27.305282+0000","last_became_active":"2023-11-08T19:27:07.189939+0000","last_became_peered":"2023-11-08T19:27:07.189939+0000","last_unstale":"2023-11-08T19:27:27.305282+0000","last_undegraded":"2023-11-08T19:27:27.305282+0000","last_fullsized":"2023-11-08T19:27:27.305282+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:41:08.822366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036507500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.324310+0000","last_change":"2023-11-08T19:27:32.320256+0000","last_active":"2023-11-08T19:27:32.324310+0000","last_peered":"2023-11-08T19:27:32.324310+0000","last_clean":"2023-11-08T19:27:32.324310+0000","last_became_active":"2023-11-08T19:27:32.319830+0000","last_became_peered":"2023-11-08T19:27:32.319830+0000","last_unstale":"2023-11-08T19:27:32.324310+0000","last_undegraded":"2023-11-08T19:27:32.324310+0000","last_fullsized":"2023-11-08T19:27:32.324310+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:00:21.681748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.310481+0000","last_change":"2023-11-08T19:27:31.314348+0000","last_active":"2023-11-08T19:27:32.310481+0000","last_peered":"2023-11-08T19:27:32.310481+0000","last_clean":"2023-11-08T19:27:32.310481+0000","last_became_active":"2023-11-08T19:27:31.314099+0000","last_became_peered":"2023-11-08T19:27:31.314099+0000","last_unstale":"2023-11-08T19:27:32.310481+0000","last_undegraded":"2023-11-08T19:27:32.310481+0000","last_fullsized":"2023-11-08T19:27:32.310481+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_clean_scrub_stamp":"2023-11-08T19:27:30.289497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:37:47.141610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":19,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-08T19:27:31.303468+0000","last_change":"2023-11-08T19:27:08.187127+0000","last_active":"2023-11-08T19:27:31.303468+0000","last_peered":"2023-11-08T19:27:31.303468+0000","last_clean":"2023-11-08T19:27:31.303468+0000","last_became_active":"2023-11-08T19:27:08.182648+0000","last_became_peered":"2023-11-08T19:27:08.182648+0000","last_unstale":"2023-11-08T19:27:31.303468+0000","last_undegraded":"2023-11-08T19:27:31.303468+0000","last_fullsized":"2023-11-08T19:27:31.303468+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:50:12.101878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":54,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-08T19:27:27.305236+0000","last_change":"2023-11-08T19:26:05.609570+0000","last_active":"2023-11-08T19:27:27.305236+0000","last_peered":"2023-11-08T19:27:27.305236+0000","last_clean":"2023-11-08T19:27:27.305236+0000","last_became_active":"2023-11-08T19:26:04.608379+0000","last_became_peered":"2023-11-08T19:26:04.608379+0000","last_unstale":"2023-11-08T19:27:27.305236+0000","last_undegraded":"2023-11-08T19:27:27.305236+0000","last_fullsized":"2023-11-08T19:27:27.305236+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:40:08.986649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000529657,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.312068+0000","last_change":"2023-11-08T19:27:09.211991+0000","last_active":"2023-11-08T19:27:32.312068+0000","last_peered":"2023-11-08T19:27:32.312068+0000","last_clean":"2023-11-08T19:27:32.312068+0000","last_became_active":"2023-11-08T19:27:07.195231+0000","last_became_peered":"2023-11-08T19:27:07.195231+0000","last_unstale":"2023-11-08T19:27:32.312068+0000","last_undegraded":"2023-11-08T19:27:32.312068+0000","last_fullsized":"2023-11-08T19:27:32.312068+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:25:14.487524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000441221,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.326577+0000","last_change":"2023-11-08T19:27:32.317869+0000","last_active":"2023-11-08T19:27:32.326577+0000","last_peered":"2023-11-08T19:27:32.326577+0000","last_clean":"2023-11-08T19:27:32.326577+0000","last_became_active":"2023-11-08T19:27:32.317588+0000","last_became_peered":"2023-11-08T19:27:32.317588+0000","last_unstale":"2023-11-08T19:27:32.326577+0000","last_undegraded":"2023-11-08T19:27:32.326577+0000","last_fullsized":"2023-11-08T19:27:32.326577+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:28:17.551060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"77'2","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.333389+0000","last_change":"2023-11-08T19:27:31.327747+0000","last_active":"2023-11-08T19:27:32.333389+0000","last_peered":"2023-11-08T19:27:32.333389+0000","last_clean":"2023-11-08T19:27:32.333389+0000","last_became_active":"2023-11-08T19:27:31.327536+0000","last_became_peered":"2023-11-08T19:27:31.327536+0000","last_unstale":"2023-11-08T19:27:32.333389+0000","last_undegraded":"2023-11-08T19:27:32.333389+0000","last_fullsized":"2023-11-08T19:27:32.333389+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_clean_scrub_stamp":"2023-11-08T19:27:30.289497+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:28:21.515340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":19,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-08T19:27:31.304153+0000","last_change":"2023-11-08T19:27:08.195364+0000","last_active":"2023-11-08T19:27:31.304153+0000","last_peered":"2023-11-08T19:27:31.304153+0000","last_clean":"2023-11-08T19:27:31.304153+0000","last_became_active":"2023-11-08T19:27:08.195074+0000","last_became_peered":"2023-11-08T19:27:08.195074+0000","last_unstale":"2023-11-08T19:27:31.304153+0000","last_undegraded":"2023-11-08T19:27:31.304153+0000","last_fullsized":"2023-11-08T19:27:31.304153+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:02:57.552873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":49,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-08T19:27:31.326311+0000","last_change":"2023-11-08T19:26:06.394300+0000","last_active":"2023-11-08T19:27:31.326311+0000","last_peered":"2023-11-08T19:27:31.326311+0000","last_clean":"2023-11-08T19:27:31.326311+0000","last_became_active":"2023-11-08T19:26:04.601685+0000","last_became_peered":"2023-11-08T19:26:04.601685+0000","last_unstale":"2023-11-08T19:27:31.326311+0000","last_undegraded":"2023-11-08T19:27:31.326311+0000","last_fullsized":"2023-11-08T19:27:31.326311+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:51:06.626474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061044099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-08T19:27:31.304181+0000","last_change":"2023-11-08T19:27:09.178466+0000","last_active":"2023-11-08T19:27:31.304181+0000","last_peered":"2023-11-08T19:27:31.304181+0000","last_clean":"2023-11-08T19:27:31.304181+0000","last_became_active":"2023-11-08T19:27:07.189386+0000","last_became_peered":"2023-11-08T19:27:07.189386+0000","last_unstale":"2023-11-08T19:27:31.304181+0000","last_undegraded":"2023-11-08T19:27:31.304181+0000","last_fullsized":"2023-11-08T19:27:31.304181+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:48:51.139820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049390100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.337392+0000","last_change":"2023-11-08T19:27:32.333807+0000","last_active":"2023-11-08T19:27:32.337392+0000","last_peered":"2023-11-08T19:27:32.337392+0000","last_clean":"2023-11-08T19:27:32.337392+0000","last_became_active":"2023-11-08T19:27:32.333527+0000","last_became_peered":"2023-11-08T19:27:32.333527+0000","last_unstale":"2023-11-08T19:27:32.337392+0000","last_undegraded":"2023-11-08T19:27:32.337392+0000","last_fullsized":"2023-11-08T19:27:32.337392+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:43:12.503877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.310400+0000","last_change":"2023-11-08T19:27:31.316250+0000","last_active":"2023-11-08T19:27:32.310400+0000","last_peered":"2023-11-08T19:27:32.310400+0000","last_clean":"2023-11-08T19:27:32.310400+0000","last_became_active":"2023-11-08T19:27:31.315970+0000","last_became_peered":"2023-11-08T19:27:31.315970+0000","last_unstale":"2023-11-08T19:27:32.310400+0000","last_undegraded":"2023-11-08T19:27:32.310400+0000","last_fullsized":"2023-11-08T19:27:32.310400+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_clean_scrub_stamp":"2023-11-08T19:27:30.289497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:57:50.429595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":21,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-08T19:27:31.311591+0000","last_change":"2023-11-08T19:27:08.189452+0000","last_active":"2023-11-08T19:27:31.311591+0000","last_peered":"2023-11-08T19:27:31.311591+0000","last_clean":"2023-11-08T19:27:31.311591+0000","last_became_active":"2023-11-08T19:27:08.189088+0000","last_became_peered":"2023-11-08T19:27:08.189088+0000","last_unstale":"2023-11-08T19:27:31.311591+0000","last_undegraded":"2023-11-08T19:27:31.311591+0000","last_fullsized":"2023-11-08T19:27:31.311591+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:25:18.963220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":50,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-08T19:27:31.307657+0000","last_change":"2023-11-08T19:26:06.296634+0000","last_active":"2023-11-08T19:27:31.307657+0000","last_peered":"2023-11-08T19:27:31.307657+0000","last_clean":"2023-11-08T19:27:31.307657+0000","last_became_active":"2023-11-08T19:26:04.603088+0000","last_became_peered":"2023-11-08T19:26:04.603088+0000","last_unstale":"2023-11-08T19:27:31.307657+0000","last_undegraded":"2023-11-08T19:27:31.307657+0000","last_fullsized":"2023-11-08T19:27:31.307657+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:01:56.991346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038180899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-08T19:27:31.309270+0000","last_change":"2023-11-08T19:27:09.431220+0000","last_active":"2023-11-08T19:27:31.309270+0000","last_peered":"2023-11-08T19:27:31.309270+0000","last_clean":"2023-11-08T19:27:31.309270+0000","last_became_active":"2023-11-08T19:27:07.191589+0000","last_became_peered":"2023-11-08T19:27:07.191589+0000","last_unstale":"2023-11-08T19:27:31.309270+0000","last_undegraded":"2023-11-08T19:27:31.309270+0000","last_fullsized":"2023-11-08T19:27:31.309270+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:18:19.964487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000729527,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-08T19:27:33.310067+0000","last_change":"2023-11-08T19:27:32.318855+0000","last_active":"2023-11-08T19:27:33.310067+0000","last_peered":"2023-11-08T19:27:33.310067+0000","last_clean":"2023-11-08T19:27:33.310067+0000","last_became_active":"2023-11-08T19:27:32.318553+0000","last_became_peered":"2023-11-08T19:27:32.318553+0000","last_unstale":"2023-11-08T19:27:33.310067+0000","last_undegraded":"2023-11-08T19:27:33.310067+0000","last_fullsized":"2023-11-08T19:27:33.310067+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:37:00.013636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"77'1","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-08T19:27:33.310659+0000","last_change":"2023-11-08T19:27:31.322280+0000","last_active":"2023-11-08T19:27:33.310659+0000","last_peered":"2023-11-08T19:27:33.310659+0000","last_clean":"2023-11-08T19:27:33.310659+0000","last_became_active":"2023-11-08T19:27:31.321982+0000","last_became_peered":"2023-11-08T19:27:31.321982+0000","last_unstale":"2023-11-08T19:27:33.310659+0000","last_undegraded":"2023-11-08T19:27:33.310659+0000","last_fullsized":"2023-11-08T19:27:33.310659+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_clean_scrub_stamp":"2023-11-08T19:27:30.289497+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:18:24.093678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-08T19:27:31.326935+0000","last_change":"2023-11-08T19:27:08.193337+0000","last_active":"2023-11-08T19:27:31.326935+0000","last_peered":"2023-11-08T19:27:31.326935+0000","last_clean":"2023-11-08T19:27:31.326935+0000","last_became_active":"2023-11-08T19:27:08.192997+0000","last_became_peered":"2023-11-08T19:27:08.192997+0000","last_unstale":"2023-11-08T19:27:31.326935+0000","last_undegraded":"2023-11-08T19:27:31.326935+0000","last_fullsized":"2023-11-08T19:27:31.326935+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:45:22.084015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":52,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-08T19:27:31.313803+0000","last_change":"2023-11-08T19:26:06.424511+0000","last_active":"2023-11-08T19:27:31.313803+0000","last_peered":"2023-11-08T19:27:31.313803+0000","last_clean":"2023-11-08T19:27:31.313803+0000","last_became_active":"2023-11-08T19:26:04.598973+0000","last_became_peered":"2023-11-08T19:26:04.598973+0000","last_unstale":"2023-11-08T19:27:31.313803+0000","last_undegraded":"2023-11-08T19:27:31.313803+0000","last_fullsized":"2023-11-08T19:27:31.313803+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:41:53.497456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031755900000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":28,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-08T19:27:31.311679+0000","last_change":"2023-11-08T19:27:09.497024+0000","last_active":"2023-11-08T19:27:31.311679+0000","last_peered":"2023-11-08T19:27:31.311679+0000","last_clean":"2023-11-08T19:27:31.311679+0000","last_became_active":"2023-11-08T19:27:07.190132+0000","last_became_peered":"2023-11-08T19:27:07.190132+0000","last_unstale":"2023-11-08T19:27:31.311679+0000","last_undegraded":"2023-11-08T19:27:31.311679+0000","last_fullsized":"2023-11-08T19:27:31.311679+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:36:34.881217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054126000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.324544+0000","last_change":"2023-11-08T19:27:32.320953+0000","last_active":"2023-11-08T19:27:32.324544+0000","last_peered":"2023-11-08T19:27:32.324544+0000","last_clean":"2023-11-08T19:27:32.324544+0000","last_became_active":"2023-11-08T19:27:32.320660+0000","last_became_peered":"2023-11-08T19:27:32.320660+0000","last_unstale":"2023-11-08T19:27:32.324544+0000","last_undegraded":"2023-11-08T19:27:32.324544+0000","last_fullsized":"2023-11-08T19:27:32.324544+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:54:44.277177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:27:30.289497+0000","last_change":"2023-11-08T19:27:30.289497+0000","last_active":"2023-11-08T19:27:30.289497+0000","last_peered":"2023-11-08T19:27:30.289497+0000","last_clean":"2023-11-08T19:27:30.289497+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:27:30.289497+0000","last_undegraded":"2023-11-08T19:27:30.289497+0000","last_fullsized":"2023-11-08T19:27:30.289497+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_clean_scrub_stamp":"2023-11-08T19:27:30.289497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-08T19:27:31.326274+0000","last_change":"2023-11-08T19:27:08.194196+0000","last_active":"2023-11-08T19:27:31.326274+0000","last_peered":"2023-11-08T19:27:31.326274+0000","last_clean":"2023-11-08T19:27:31.326274+0000","last_became_active":"2023-11-08T19:27:08.193912+0000","last_became_peered":"2023-11-08T19:27:08.193912+0000","last_unstale":"2023-11-08T19:27:31.326274+0000","last_undegraded":"2023-11-08T19:27:31.326274+0000","last_fullsized":"2023-11-08T19:27:31.326274+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:25:18.963220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":50,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-08T19:27:31.309352+0000","last_change":"2023-11-08T19:26:06.296937+0000","last_active":"2023-11-08T19:27:31.309352+0000","last_peered":"2023-11-08T19:27:31.309352+0000","last_clean":"2023-11-08T19:27:31.309352+0000","last_became_active":"2023-11-08T19:26:04.599999+0000","last_became_peered":"2023-11-08T19:26:04.599999+0000","last_unstale":"2023-11-08T19:27:31.309352+0000","last_undegraded":"2023-11-08T19:27:31.309352+0000","last_fullsized":"2023-11-08T19:27:31.309352+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:50:43.827570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025605700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":28,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-08T19:27:31.313345+0000","last_change":"2023-11-08T19:27:09.497040+0000","last_active":"2023-11-08T19:27:31.313345+0000","last_peered":"2023-11-08T19:27:31.313345+0000","last_clean":"2023-11-08T19:27:31.313345+0000","last_became_active":"2023-11-08T19:27:07.189135+0000","last_became_peered":"2023-11-08T19:27:07.189135+0000","last_unstale":"2023-11-08T19:27:31.313345+0000","last_undegraded":"2023-11-08T19:27:31.313345+0000","last_fullsized":"2023-11-08T19:27:31.313345+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:12:29.400972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051794299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:27:31.294701+0000","last_change":"2023-11-08T19:27:31.294701+0000","last_active":"2023-11-08T19:27:31.294701+0000","last_peered":"2023-11-08T19:27:31.294701+0000","last_clean":"2023-11-08T19:27:31.294701+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:27:31.294701+0000","last_undegraded":"2023-11-08T19:27:31.294701+0000","last_fullsized":"2023-11-08T19:27:31.294701+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:27:30.289497+0000","last_change":"2023-11-08T19:27:30.289497+0000","last_active":"2023-11-08T19:27:30.289497+0000","last_peered":"2023-11-08T19:27:30.289497+0000","last_clean":"2023-11-08T19:27:30.289497+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:27:30.289497+0000","last_undegraded":"2023-11-08T19:27:30.289497+0000","last_fullsized":"2023-11-08T19:27:30.289497+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_clean_scrub_stamp":"2023-11-08T19:27:30.289497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":19,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-08T19:27:27.306446+0000","last_change":"2023-11-08T19:27:08.187053+0000","last_active":"2023-11-08T19:27:27.306446+0000","last_peered":"2023-11-08T19:27:27.306446+0000","last_clean":"2023-11-08T19:27:27.306446+0000","last_became_active":"2023-11-08T19:27:08.186643+0000","last_became_peered":"2023-11-08T19:27:08.186643+0000","last_unstale":"2023-11-08T19:27:27.306446+0000","last_undegraded":"2023-11-08T19:27:27.306446+0000","last_fullsized":"2023-11-08T19:27:27.306446+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:35:30.073885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":54,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-08T19:27:27.305394+0000","last_change":"2023-11-08T19:26:05.609824+0000","last_active":"2023-11-08T19:27:27.305394+0000","last_peered":"2023-11-08T19:27:27.305394+0000","last_clean":"2023-11-08T19:27:27.305394+0000","last_became_active":"2023-11-08T19:26:04.603320+0000","last_became_peered":"2023-11-08T19:26:04.603320+0000","last_unstale":"2023-11-08T19:27:27.305394+0000","last_undegraded":"2023-11-08T19:27:27.305394+0000","last_fullsized":"2023-11-08T19:27:27.305394+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:05:45.976615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063225,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":28,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-08T19:27:31.313234+0000","last_change":"2023-11-08T19:27:09.496655+0000","last_active":"2023-11-08T19:27:31.313234+0000","last_peered":"2023-11-08T19:27:31.313234+0000","last_clean":"2023-11-08T19:27:31.313234+0000","last_became_active":"2023-11-08T19:27:07.189895+0000","last_became_peered":"2023-11-08T19:27:07.189895+0000","last_unstale":"2023-11-08T19:27:31.313234+0000","last_undegraded":"2023-11-08T19:27:31.313234+0000","last_fullsized":"2023-11-08T19:27:31.313234+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:41:08.822366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045654199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:27:31.294701+0000","last_change":"2023-11-08T19:27:31.294701+0000","last_active":"2023-11-08T19:27:31.294701+0000","last_peered":"2023-11-08T19:27:31.294701+0000","last_clean":"2023-11-08T19:27:31.294701+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:27:31.294701+0000","last_undegraded":"2023-11-08T19:27:31.294701+0000","last_fullsized":"2023-11-08T19:27:31.294701+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:27:30.289497+0000","last_change":"2023-11-08T19:27:30.289497+0000","last_active":"2023-11-08T19:27:30.289497+0000","last_peered":"2023-11-08T19:27:30.289497+0000","last_clean":"2023-11-08T19:27:30.289497+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:27:30.289497+0000","last_undegraded":"2023-11-08T19:27:30.289497+0000","last_fullsized":"2023-11-08T19:27:30.289497+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_clean_scrub_stamp":"2023-11-08T19:27:30.289497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":51,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-08T19:27:31.312208+0000","last_change":"2023-11-08T19:26:06.541360+0000","last_active":"2023-11-08T19:27:31.312208+0000","last_peered":"2023-11-08T19:27:31.312208+0000","last_clean":"2023-11-08T19:27:31.312208+0000","last_became_active":"2023-11-08T19:26:04.603725+0000","last_became_peered":"2023-11-08T19:26:04.603725+0000","last_unstale":"2023-11-08T19:27:31.312208+0000","last_undegraded":"2023-11-08T19:27:31.312208+0000","last_fullsized":"2023-11-08T19:27:31.312208+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:28:48.576742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053365400000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-08T19:27:31.314076+0000","last_change":"2023-11-08T19:27:09.283599+0000","last_active":"2023-11-08T19:27:31.314076+0000","last_peered":"2023-11-08T19:27:31.314076+0000","last_clean":"2023-11-08T19:27:31.314076+0000","last_became_active":"2023-11-08T19:27:07.189595+0000","last_became_peered":"2023-11-08T19:27:07.189595+0000","last_unstale":"2023-11-08T19:27:31.314076+0000","last_undegraded":"2023-11-08T19:27:31.314076+0000","last_fullsized":"2023-11-08T19:27:31.314076+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:48:20.082682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000323479,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":21,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.309454+0000","last_change":"2023-11-08T19:27:08.185395+0000","last_active":"2023-11-08T19:27:32.309454+0000","last_peered":"2023-11-08T19:27:32.309454+0000","last_clean":"2023-11-08T19:27:32.309454+0000","last_became_active":"2023-11-08T19:27:08.184765+0000","last_became_peered":"2023-11-08T19:27:08.184765+0000","last_unstale":"2023-11-08T19:27:32.309454+0000","last_undegraded":"2023-11-08T19:27:32.309454+0000","last_fullsized":"2023-11-08T19:27:32.309454+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:08:51.864418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.332314+0000","last_change":"2023-11-08T19:27:32.327550+0000","last_active":"2023-11-08T19:27:32.332314+0000","last_peered":"2023-11-08T19:27:32.332314+0000","last_clean":"2023-11-08T19:27:32.332314+0000","last_became_active":"2023-11-08T19:27:32.327314+0000","last_became_peered":"2023-11-08T19:27:32.327314+0000","last_unstale":"2023-11-08T19:27:32.332314+0000","last_undegraded":"2023-11-08T19:27:32.332314+0000","last_fullsized":"2023-11-08T19:27:32.332314+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:05:01.010467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:27:30.289497+0000","last_change":"2023-11-08T19:27:30.289497+0000","last_active":"2023-11-08T19:27:30.289497+0000","last_peered":"2023-11-08T19:27:30.289497+0000","last_clean":"2023-11-08T19:27:30.289497+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:27:30.289497+0000","last_undegraded":"2023-11-08T19:27:30.289497+0000","last_fullsized":"2023-11-08T19:27:30.289497+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_clean_scrub_stamp":"2023-11-08T19:27:30.289497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":52,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-08T19:27:31.311879+0000","last_change":"2023-11-08T19:26:06.424236+0000","last_active":"2023-11-08T19:27:31.311879+0000","last_peered":"2023-11-08T19:27:31.311879+0000","last_clean":"2023-11-08T19:27:31.311879+0000","last_became_active":"2023-11-08T19:26:04.600115+0000","last_became_peered":"2023-11-08T19:26:04.600115+0000","last_unstale":"2023-11-08T19:27:31.311879+0000","last_undegraded":"2023-11-08T19:27:31.311879+0000","last_fullsized":"2023-11-08T19:27:31.311879+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:18:01.876605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065803900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-08T19:27:31.326139+0000","last_change":"2023-11-08T19:27:09.331899+0000","last_active":"2023-11-08T19:27:31.326139+0000","last_peered":"2023-11-08T19:27:31.326139+0000","last_clean":"2023-11-08T19:27:31.326139+0000","last_became_active":"2023-11-08T19:27:07.191460+0000","last_became_peered":"2023-11-08T19:27:07.191460+0000","last_unstale":"2023-11-08T19:27:31.326139+0000","last_undegraded":"2023-11-08T19:27:31.326139+0000","last_fullsized":"2023-11-08T19:27:31.326139+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:51:29.247104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044907500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":19,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-08T19:27:27.305965+0000","last_change":"2023-11-08T19:27:08.191802+0000","last_active":"2023-11-08T19:27:27.305965+0000","last_peered":"2023-11-08T19:27:27.305965+0000","last_clean":"2023-11-08T19:27:27.305965+0000","last_became_active":"2023-11-08T19:27:08.191520+0000","last_became_peered":"2023-11-08T19:27:08.191520+0000","last_unstale":"2023-11-08T19:27:27.305965+0000","last_undegraded":"2023-11-08T19:27:27.305965+0000","last_fullsized":"2023-11-08T19:27:27.305965+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:49:16.458634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.323523+0000","last_change":"2023-11-08T19:27:32.319621+0000","last_active":"2023-11-08T19:27:32.323523+0000","last_peered":"2023-11-08T19:27:32.323523+0000","last_clean":"2023-11-08T19:27:32.323523+0000","last_became_active":"2023-11-08T19:27:32.319371+0000","last_became_peered":"2023-11-08T19:27:32.319371+0000","last_unstale":"2023-11-08T19:27:32.323523+0000","last_undegraded":"2023-11-08T19:27:32.323523+0000","last_fullsized":"2023-11-08T19:27:32.323523+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:43:54.922810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.312114+0000","last_change":"2023-11-08T19:27:31.326195+0000","last_active":"2023-11-08T19:27:32.312114+0000","last_peered":"2023-11-08T19:27:32.312114+0000","last_clean":"2023-11-08T19:27:32.312114+0000","last_became_active":"2023-11-08T19:27:31.325718+0000","last_became_peered":"2023-11-08T19:27:31.325718+0000","last_unstale":"2023-11-08T19:27:32.312114+0000","last_undegraded":"2023-11-08T19:27:32.312114+0000","last_fullsized":"2023-11-08T19:27:32.312114+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_clean_scrub_stamp":"2023-11-08T19:27:30.289497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:51:14.480725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":54,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-08T19:27:27.304861+0000","last_change":"2023-11-08T19:26:05.608892+0000","last_active":"2023-11-08T19:27:27.304861+0000","last_peered":"2023-11-08T19:27:27.304861+0000","last_clean":"2023-11-08T19:27:27.304861+0000","last_became_active":"2023-11-08T19:26:04.604519+0000","last_became_peered":"2023-11-08T19:26:04.604519+0000","last_unstale":"2023-11-08T19:27:27.304861+0000","last_undegraded":"2023-11-08T19:27:27.304861+0000","last_fullsized":"2023-11-08T19:27:27.304861+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:11:26.508634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051248099999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":26,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-08T19:27:27.305357+0000","last_change":"2023-11-08T19:27:09.181431+0000","last_active":"2023-11-08T19:27:27.305357+0000","last_peered":"2023-11-08T19:27:27.305357+0000","last_clean":"2023-11-08T19:27:27.305357+0000","last_became_active":"2023-11-08T19:27:07.190800+0000","last_became_peered":"2023-11-08T19:27:07.190800+0000","last_unstale":"2023-11-08T19:27:27.305357+0000","last_undegraded":"2023-11-08T19:27:27.305357+0000","last_fullsized":"2023-11-08T19:27:27.305357+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:46:30.452678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656099999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":21,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.312177+0000","last_change":"2023-11-08T19:27:08.187983+0000","last_active":"2023-11-08T19:27:32.312177+0000","last_peered":"2023-11-08T19:27:32.312177+0000","last_clean":"2023-11-08T19:27:32.312177+0000","last_became_active":"2023-11-08T19:27:08.187322+0000","last_became_peered":"2023-11-08T19:27:08.187322+0000","last_unstale":"2023-11-08T19:27:32.312177+0000","last_undegraded":"2023-11-08T19:27:32.312177+0000","last_fullsized":"2023-11-08T19:27:32.312177+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:33:07.647519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.336174+0000","last_change":"2023-11-08T19:27:32.332731+0000","last_active":"2023-11-08T19:27:32.336174+0000","last_peered":"2023-11-08T19:27:32.336174+0000","last_clean":"2023-11-08T19:27:32.336174+0000","last_became_active":"2023-11-08T19:27:32.332354+0000","last_became_peered":"2023-11-08T19:27:32.332354+0000","last_unstale":"2023-11-08T19:27:32.336174+0000","last_undegraded":"2023-11-08T19:27:32.336174+0000","last_fullsized":"2023-11-08T19:27:32.336174+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:15:16.572033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.311295+0000","last_change":"2023-11-08T19:27:31.327299+0000","last_active":"2023-11-08T19:27:32.311295+0000","last_peered":"2023-11-08T19:27:32.311295+0000","last_clean":"2023-11-08T19:27:32.311295+0000","last_became_active":"2023-11-08T19:27:31.326997+0000","last_became_peered":"2023-11-08T19:27:31.326997+0000","last_unstale":"2023-11-08T19:27:32.311295+0000","last_undegraded":"2023-11-08T19:27:32.311295+0000","last_fullsized":"2023-11-08T19:27:32.311295+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_clean_scrub_stamp":"2023-11-08T19:27:30.289497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:25:42.085432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":49,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-08T19:27:31.326286+0000","last_change":"2023-11-08T19:26:06.394075+0000","last_active":"2023-11-08T19:27:31.326286+0000","last_peered":"2023-11-08T19:27:31.326286+0000","last_clean":"2023-11-08T19:27:31.326286+0000","last_became_active":"2023-11-08T19:26:04.606967+0000","last_became_peered":"2023-11-08T19:26:04.606967+0000","last_unstale":"2023-11-08T19:27:31.326286+0000","last_undegraded":"2023-11-08T19:27:31.326286+0000","last_fullsized":"2023-11-08T19:27:31.326286+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:19:56.286835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040971999999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":26,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-08T19:27:27.305088+0000","last_change":"2023-11-08T19:27:09.180722+0000","last_active":"2023-11-08T19:27:27.305088+0000","last_peered":"2023-11-08T19:27:27.305088+0000","last_clean":"2023-11-08T19:27:27.305088+0000","last_became_active":"2023-11-08T19:27:07.186977+0000","last_became_peered":"2023-11-08T19:27:07.186977+0000","last_unstale":"2023-11-08T19:27:27.305088+0000","last_undegraded":"2023-11-08T19:27:27.305088+0000","last_fullsized":"2023-11-08T19:27:27.305088+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:17:59.966463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057675100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-08T19:27:31.326404+0000","last_change":"2023-11-08T19:27:08.193963+0000","last_active":"2023-11-08T19:27:31.326404+0000","last_peered":"2023-11-08T19:27:31.326404+0000","last_clean":"2023-11-08T19:27:31.326404+0000","last_became_active":"2023-11-08T19:27:08.193098+0000","last_became_peered":"2023-11-08T19:27:08.193098+0000","last_unstale":"2023-11-08T19:27:31.326404+0000","last_undegraded":"2023-11-08T19:27:31.326404+0000","last_fullsized":"2023-11-08T19:27:31.326404+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:41:09.827370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.324574+0000","last_change":"2023-11-08T19:27:32.322909+0000","last_active":"2023-11-08T19:27:32.324574+0000","last_peered":"2023-11-08T19:27:32.324574+0000","last_clean":"2023-11-08T19:27:32.324574+0000","last_became_active":"2023-11-08T19:27:32.322643+0000","last_became_peered":"2023-11-08T19:27:32.322643+0000","last_unstale":"2023-11-08T19:27:32.324574+0000","last_undegraded":"2023-11-08T19:27:32.324574+0000","last_fullsized":"2023-11-08T19:27:32.324574+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:08:48.489860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-08T19:27:33.310535+0000","last_change":"2023-11-08T19:27:31.331499+0000","last_active":"2023-11-08T19:27:33.310535+0000","last_peered":"2023-11-08T19:27:33.310535+0000","last_clean":"2023-11-08T19:27:33.310535+0000","last_became_active":"2023-11-08T19:27:31.331164+0000","last_became_peered":"2023-11-08T19:27:31.331164+0000","last_unstale":"2023-11-08T19:27:33.310535+0000","last_undegraded":"2023-11-08T19:27:33.310535+0000","last_fullsized":"2023-11-08T19:27:33.310535+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_clean_scrub_stamp":"2023-11-08T19:27:30.289497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:30:13.190099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":50,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.312402+0000","last_change":"2023-11-08T19:26:06.265686+0000","last_active":"2023-11-08T19:27:32.312402+0000","last_peered":"2023-11-08T19:27:32.312402+0000","last_clean":"2023-11-08T19:27:32.312402+0000","last_became_active":"2023-11-08T19:26:04.602111+0000","last_became_peered":"2023-11-08T19:26:04.602111+0000","last_unstale":"2023-11-08T19:27:32.312402+0000","last_undegraded":"2023-11-08T19:27:32.312402+0000","last_fullsized":"2023-11-08T19:27:32.312402+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:40:11.164951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056197899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":26,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-08T19:27:27.306032+0000","last_change":"2023-11-08T19:27:09.181333+0000","last_active":"2023-11-08T19:27:27.306032+0000","last_peered":"2023-11-08T19:27:27.306032+0000","last_clean":"2023-11-08T19:27:27.306032+0000","last_became_active":"2023-11-08T19:27:07.188657+0000","last_became_peered":"2023-11-08T19:27:07.188657+0000","last_unstale":"2023-11-08T19:27:27.306032+0000","last_undegraded":"2023-11-08T19:27:27.306032+0000","last_fullsized":"2023-11-08T19:27:27.306032+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:25:56.788615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001227393,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":19,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-08T19:27:27.306128+0000","last_change":"2023-11-08T19:27:08.188533+0000","last_active":"2023-11-08T19:27:27.306128+0000","last_peered":"2023-11-08T19:27:27.306128+0000","last_clean":"2023-11-08T19:27:27.306128+0000","last_became_active":"2023-11-08T19:27:08.188240+0000","last_became_peered":"2023-11-08T19:27:08.188240+0000","last_unstale":"2023-11-08T19:27:27.306128+0000","last_undegraded":"2023-11-08T19:27:27.306128+0000","last_fullsized":"2023-11-08T19:27:27.306128+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:55:37.653521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.327548+0000","last_change":"2023-11-08T19:27:32.320936+0000","last_active":"2023-11-08T19:27:32.327548+0000","last_peered":"2023-11-08T19:27:32.327548+0000","last_clean":"2023-11-08T19:27:32.327548+0000","last_became_active":"2023-11-08T19:27:32.320641+0000","last_became_peered":"2023-11-08T19:27:32.320641+0000","last_unstale":"2023-11-08T19:27:32.327548+0000","last_undegraded":"2023-11-08T19:27:32.327548+0000","last_fullsized":"2023-11-08T19:27:32.327548+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:21:03.212549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:27:30.289497+0000","last_change":"2023-11-08T19:27:30.289497+0000","last_active":"2023-11-08T19:27:30.289497+0000","last_peered":"2023-11-08T19:27:30.289497+0000","last_clean":"2023-11-08T19:27:30.289497+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:27:30.289497+0000","last_undegraded":"2023-11-08T19:27:30.289497+0000","last_fullsized":"2023-11-08T19:27:30.289497+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_clean_scrub_stamp":"2023-11-08T19:27:30.289497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":19,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-08T19:27:27.306315+0000","last_change":"2023-11-08T19:27:08.188458+0000","last_active":"2023-11-08T19:27:27.306315+0000","last_peered":"2023-11-08T19:27:27.306315+0000","last_clean":"2023-11-08T19:27:27.306315+0000","last_became_active":"2023-11-08T19:27:08.187880+0000","last_became_peered":"2023-11-08T19:27:08.187880+0000","last_unstale":"2023-11-08T19:27:27.306315+0000","last_undegraded":"2023-11-08T19:27:27.306315+0000","last_fullsized":"2023-11-08T19:27:27.306315+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:25:28.618214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":49,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-08T19:27:31.304123+0000","last_change":"2023-11-08T19:26:05.606898+0000","last_active":"2023-11-08T19:27:31.304123+0000","last_peered":"2023-11-08T19:27:31.304123+0000","last_clean":"2023-11-08T19:27:31.304123+0000","last_became_active":"2023-11-08T19:26:04.603955+0000","last_became_peered":"2023-11-08T19:26:04.603955+0000","last_unstale":"2023-11-08T19:27:31.304123+0000","last_undegraded":"2023-11-08T19:27:31.304123+0000","last_fullsized":"2023-11-08T19:27:31.304123+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:19:11.589559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036473499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-08T19:27:31.304009+0000","last_change":"2023-11-08T19:27:09.178770+0000","last_active":"2023-11-08T19:27:31.304009+0000","last_peered":"2023-11-08T19:27:31.304009+0000","last_clean":"2023-11-08T19:27:31.304009+0000","last_became_active":"2023-11-08T19:27:07.194992+0000","last_became_peered":"2023-11-08T19:27:07.194992+0000","last_unstale":"2023-11-08T19:27:31.304009+0000","last_undegraded":"2023-11-08T19:27:31.304009+0000","last_fullsized":"2023-11-08T19:27:31.304009+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:35:34.026910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000770285,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:27:31.294701+0000","last_change":"2023-11-08T19:27:31.294701+0000","last_active":"2023-11-08T19:27:31.294701+0000","last_peered":"2023-11-08T19:27:31.294701+0000","last_clean":"2023-11-08T19:27:31.294701+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:27:31.294701+0000","last_undegraded":"2023-11-08T19:27:31.294701+0000","last_fullsized":"2023-11-08T19:27:31.294701+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:27:30.289497+0000","last_change":"2023-11-08T19:27:30.289497+0000","last_active":"2023-11-08T19:27:30.289497+0000","last_peered":"2023-11-08T19:27:30.289497+0000","last_clean":"2023-11-08T19:27:30.289497+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:27:30.289497+0000","last_undegraded":"2023-11-08T19:27:30.289497+0000","last_fullsized":"2023-11-08T19:27:30.289497+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_clean_scrub_stamp":"2023-11-08T19:27:30.289497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-08T19:27:31.307173+0000","last_change":"2023-11-08T19:27:08.193953+0000","last_active":"2023-11-08T19:27:31.307173+0000","last_peered":"2023-11-08T19:27:31.307173+0000","last_clean":"2023-11-08T19:27:31.307173+0000","last_became_active":"2023-11-08T19:27:08.193653+0000","last_became_peered":"2023-11-08T19:27:08.193653+0000","last_unstale":"2023-11-08T19:27:31.307173+0000","last_undegraded":"2023-11-08T19:27:31.307173+0000","last_fullsized":"2023-11-08T19:27:31.307173+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:35:30.073885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":52,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-08T19:27:31.313067+0000","last_change":"2023-11-08T19:26:06.424464+0000","last_active":"2023-11-08T19:27:31.313067+0000","last_peered":"2023-11-08T19:27:31.313067+0000","last_clean":"2023-11-08T19:27:31.313067+0000","last_became_active":"2023-11-08T19:26:04.603144+0000","last_became_peered":"2023-11-08T19:26:04.603144+0000","last_unstale":"2023-11-08T19:27:31.313067+0000","last_undegraded":"2023-11-08T19:27:31.313067+0000","last_fullsized":"2023-11-08T19:27:31.313067+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:42:55.269365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054090799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":26,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-08T19:27:27.305937+0000","last_change":"2023-11-08T19:27:09.180479+0000","last_active":"2023-11-08T19:27:27.305937+0000","last_peered":"2023-11-08T19:27:27.305937+0000","last_clean":"2023-11-08T19:27:27.305937+0000","last_became_active":"2023-11-08T19:27:07.190438+0000","last_became_peered":"2023-11-08T19:27:07.190438+0000","last_unstale":"2023-11-08T19:27:27.305937+0000","last_undegraded":"2023-11-08T19:27:27.305937+0000","last_fullsized":"2023-11-08T19:27:27.305937+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:25:14.487524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050893300000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.333587+0000","last_change":"2023-11-08T19:27:32.326515+0000","last_active":"2023-11-08T19:27:32.333587+0000","last_peered":"2023-11-08T19:27:32.333587+0000","last_clean":"2023-11-08T19:27:32.333587+0000","last_became_active":"2023-11-08T19:27:32.325953+0000","last_became_peered":"2023-11-08T19:27:32.325953+0000","last_unstale":"2023-11-08T19:27:32.333587+0000","last_undegraded":"2023-11-08T19:27:32.333587+0000","last_fullsized":"2023-11-08T19:27:32.333587+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:58:18.403623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-08T19:27:33.308801+0000","last_change":"2023-11-08T19:27:31.322887+0000","last_active":"2023-11-08T19:27:33.308801+0000","last_peered":"2023-11-08T19:27:33.308801+0000","last_clean":"2023-11-08T19:27:33.308801+0000","last_became_active":"2023-11-08T19:27:31.322598+0000","last_became_peered":"2023-11-08T19:27:31.322598+0000","last_unstale":"2023-11-08T19:27:33.308801+0000","last_undegraded":"2023-11-08T19:27:33.308801+0000","last_fullsized":"2023-11-08T19:27:33.308801+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_clean_scrub_stamp":"2023-11-08T19:27:30.289497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:22:50.960778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-08T19:27:31.315030+0000","last_change":"2023-11-08T19:27:08.190530+0000","last_active":"2023-11-08T19:27:31.315030+0000","last_peered":"2023-11-08T19:27:31.315030+0000","last_clean":"2023-11-08T19:27:31.315030+0000","last_became_active":"2023-11-08T19:27:08.190203+0000","last_became_peered":"2023-11-08T19:27:08.190203+0000","last_unstale":"2023-11-08T19:27:31.315030+0000","last_undegraded":"2023-11-08T19:27:31.315030+0000","last_fullsized":"2023-11-08T19:27:31.315030+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:26:32.777602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":51,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-08T19:27:31.312599+0000","last_change":"2023-11-08T19:26:06.541623+0000","last_active":"2023-11-08T19:27:31.312599+0000","last_peered":"2023-11-08T19:27:31.312599+0000","last_clean":"2023-11-08T19:27:31.312599+0000","last_became_active":"2023-11-08T19:26:04.601961+0000","last_became_peered":"2023-11-08T19:26:04.601961+0000","last_unstale":"2023-11-08T19:27:31.312599+0000","last_undegraded":"2023-11-08T19:27:31.312599+0000","last_fullsized":"2023-11-08T19:27:31.312599+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:11:26.508634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000466386,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-08T19:27:31.313059+0000","last_change":"2023-11-08T19:27:09.431231+0000","last_active":"2023-11-08T19:27:31.313059+0000","last_peered":"2023-11-08T19:27:31.313059+0000","last_clean":"2023-11-08T19:27:31.313059+0000","last_became_active":"2023-11-08T19:27:07.191583+0000","last_became_peered":"2023-11-08T19:27:07.191583+0000","last_unstale":"2023-11-08T19:27:31.313059+0000","last_undegraded":"2023-11-08T19:27:31.313059+0000","last_fullsized":"2023-11-08T19:27:31.313059+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:58:48.357266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093695200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.324342+0000","last_change":"2023-11-08T19:27:32.320291+0000","last_active":"2023-11-08T19:27:32.324342+0000","last_peered":"2023-11-08T19:27:32.324342+0000","last_clean":"2023-11-08T19:27:32.324342+0000","last_became_active":"2023-11-08T19:27:32.320026+0000","last_became_peered":"2023-11-08T19:27:32.320026+0000","last_unstale":"2023-11-08T19:27:32.324342+0000","last_undegraded":"2023-11-08T19:27:32.324342+0000","last_fullsized":"2023-11-08T19:27:32.324342+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:15:16.572033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.320295+0000","last_change":"2023-11-08T19:27:31.314962+0000","last_active":"2023-11-08T19:27:32.320295+0000","last_peered":"2023-11-08T19:27:32.320295+0000","last_clean":"2023-11-08T19:27:32.320295+0000","last_became_active":"2023-11-08T19:27:31.314643+0000","last_became_peered":"2023-11-08T19:27:31.314643+0000","last_unstale":"2023-11-08T19:27:32.320295+0000","last_undegraded":"2023-11-08T19:27:32.320295+0000","last_fullsized":"2023-11-08T19:27:32.320295+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_clean_scrub_stamp":"2023-11-08T19:27:30.289497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:57:26.172555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-08T19:27:31.313134+0000","last_change":"2023-11-08T19:27:08.187900+0000","last_active":"2023-11-08T19:27:31.313134+0000","last_peered":"2023-11-08T19:27:31.313134+0000","last_clean":"2023-11-08T19:27:31.313134+0000","last_became_active":"2023-11-08T19:27:08.187626+0000","last_became_peered":"2023-11-08T19:27:08.187626+0000","last_unstale":"2023-11-08T19:27:31.313134+0000","last_undegraded":"2023-11-08T19:27:31.313134+0000","last_fullsized":"2023-11-08T19:27:31.313134+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:00:55.878397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":54,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-08T19:27:27.305037+0000","last_change":"2023-11-08T19:26:05.608980+0000","last_active":"2023-11-08T19:27:27.305037+0000","last_peered":"2023-11-08T19:27:27.305037+0000","last_clean":"2023-11-08T19:27:27.305037+0000","last_became_active":"2023-11-08T19:26:04.606386+0000","last_became_peered":"2023-11-08T19:26:04.606386+0000","last_unstale":"2023-11-08T19:27:27.305037+0000","last_undegraded":"2023-11-08T19:27:27.305037+0000","last_fullsized":"2023-11-08T19:27:27.305037+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:41:50.771457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037950599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-08T19:27:31.303745+0000","last_change":"2023-11-08T19:27:09.179037+0000","last_active":"2023-11-08T19:27:31.303745+0000","last_peered":"2023-11-08T19:27:31.303745+0000","last_clean":"2023-11-08T19:27:31.303745+0000","last_became_active":"2023-11-08T19:27:07.195033+0000","last_became_peered":"2023-11-08T19:27:07.195033+0000","last_unstale":"2023-11-08T19:27:31.303745+0000","last_undegraded":"2023-11-08T19:27:31.303745+0000","last_fullsized":"2023-11-08T19:27:31.303745+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:19:08.253483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041824800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.332780+0000","last_change":"2023-11-08T19:27:32.328033+0000","last_active":"2023-11-08T19:27:32.332780+0000","last_peered":"2023-11-08T19:27:32.332780+0000","last_clean":"2023-11-08T19:27:32.332780+0000","last_became_active":"2023-11-08T19:27:32.327746+0000","last_became_peered":"2023-11-08T19:27:32.327746+0000","last_unstale":"2023-11-08T19:27:32.332780+0000","last_undegraded":"2023-11-08T19:27:32.332780+0000","last_fullsized":"2023-11-08T19:27:32.332780+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:46:04.602463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:27:30.289497+0000","last_change":"2023-11-08T19:27:30.289497+0000","last_active":"2023-11-08T19:27:30.289497+0000","last_peered":"2023-11-08T19:27:30.289497+0000","last_clean":"2023-11-08T19:27:30.289497+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:27:30.289497+0000","last_undegraded":"2023-11-08T19:27:30.289497+0000","last_fullsized":"2023-11-08T19:27:30.289497+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_clean_scrub_stamp":"2023-11-08T19:27:30.289497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-08T19:27:31.326030+0000","last_change":"2023-11-08T19:27:08.193761+0000","last_active":"2023-11-08T19:27:31.326030+0000","last_peered":"2023-11-08T19:27:31.326030+0000","last_clean":"2023-11-08T19:27:31.326030+0000","last_became_active":"2023-11-08T19:27:08.193017+0000","last_became_peered":"2023-11-08T19:27:08.193017+0000","last_unstale":"2023-11-08T19:27:31.326030+0000","last_undegraded":"2023-11-08T19:27:31.326030+0000","last_fullsized":"2023-11-08T19:27:31.326030+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:18:00.971466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":52,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-08T19:27:31.311969+0000","last_change":"2023-11-08T19:26:06.424073+0000","last_active":"2023-11-08T19:27:31.311969+0000","last_peered":"2023-11-08T19:27:31.311969+0000","last_clean":"2023-11-08T19:27:31.311969+0000","last_became_active":"2023-11-08T19:26:04.599046+0000","last_became_peered":"2023-11-08T19:26:04.599046+0000","last_unstale":"2023-11-08T19:27:31.311969+0000","last_undegraded":"2023-11-08T19:27:31.311969+0000","last_fullsized":"2023-11-08T19:27:31.311969+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:04:43.562566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035092199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":26,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-08T19:27:27.305217+0000","last_change":"2023-11-08T19:27:09.181716+0000","last_active":"2023-11-08T19:27:27.305217+0000","last_peered":"2023-11-08T19:27:27.305217+0000","last_clean":"2023-11-08T19:27:27.305217+0000","last_became_active":"2023-11-08T19:27:07.187077+0000","last_became_peered":"2023-11-08T19:27:07.187077+0000","last_unstale":"2023-11-08T19:27:27.305217+0000","last_undegraded":"2023-11-08T19:27:27.305217+0000","last_fullsized":"2023-11-08T19:27:27.305217+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:51:29.247104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056673599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.338674+0000","last_change":"2023-11-08T19:27:32.334380+0000","last_active":"2023-11-08T19:27:32.338674+0000","last_peered":"2023-11-08T19:27:32.338674+0000","last_clean":"2023-11-08T19:27:32.338674+0000","last_became_active":"2023-11-08T19:27:32.334125+0000","last_became_peered":"2023-11-08T19:27:32.334125+0000","last_unstale":"2023-11-08T19:27:32.338674+0000","last_undegraded":"2023-11-08T19:27:32.338674+0000","last_fullsized":"2023-11-08T19:27:32.338674+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:46:55.585097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"77'1","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-08T19:27:33.308913+0000","last_change":"2023-11-08T19:27:31.320048+0000","last_active":"2023-11-08T19:27:33.308913+0000","last_peered":"2023-11-08T19:27:33.308913+0000","last_clean":"2023-11-08T19:27:33.308913+0000","last_became_active":"2023-11-08T19:27:31.319829+0000","last_became_peered":"2023-11-08T19:27:31.319829+0000","last_unstale":"2023-11-08T19:27:33.308913+0000","last_undegraded":"2023-11-08T19:27:33.308913+0000","last_fullsized":"2023-11-08T19:27:33.308913+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_clean_scrub_stamp":"2023-11-08T19:27:30.289497+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:48:28.492986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":21,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.311978+0000","last_change":"2023-11-08T19:27:08.188143+0000","last_active":"2023-11-08T19:27:32.311978+0000","last_peered":"2023-11-08T19:27:32.311978+0000","last_clean":"2023-11-08T19:27:32.311978+0000","last_became_active":"2023-11-08T19:27:08.187714+0000","last_became_peered":"2023-11-08T19:27:08.187714+0000","last_unstale":"2023-11-08T19:27:32.311978+0000","last_undegraded":"2023-11-08T19:27:32.311978+0000","last_fullsized":"2023-11-08T19:27:32.311978+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:02:10.863403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":49,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-08T19:27:31.325889+0000","last_change":"2023-11-08T19:26:06.394320+0000","last_active":"2023-11-08T19:27:31.325889+0000","last_peered":"2023-11-08T19:27:31.325889+0000","last_clean":"2023-11-08T19:27:31.325889+0000","last_became_active":"2023-11-08T19:26:04.603212+0000","last_became_peered":"2023-11-08T19:26:04.603212+0000","last_unstale":"2023-11-08T19:27:31.325889+0000","last_undegraded":"2023-11-08T19:27:31.325889+0000","last_fullsized":"2023-11-08T19:27:31.325889+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:25:29.947079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054695399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":26,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-08T19:27:27.305343+0000","last_change":"2023-11-08T19:27:09.181613+0000","last_active":"2023-11-08T19:27:27.305343+0000","last_peered":"2023-11-08T19:27:27.305343+0000","last_clean":"2023-11-08T19:27:27.305343+0000","last_became_active":"2023-11-08T19:27:07.190897+0000","last_became_peered":"2023-11-08T19:27:07.190897+0000","last_unstale":"2023-11-08T19:27:27.305343+0000","last_undegraded":"2023-11-08T19:27:27.305343+0000","last_fullsized":"2023-11-08T19:27:27.305343+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:27:57.388125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011172459999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:27:31.294701+0000","last_change":"2023-11-08T19:27:31.294701+0000","last_active":"2023-11-08T19:27:31.294701+0000","last_peered":"2023-11-08T19:27:31.294701+0000","last_clean":"2023-11-08T19:27:31.294701+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:27:31.294701+0000","last_undegraded":"2023-11-08T19:27:31.294701+0000","last_fullsized":"2023-11-08T19:27:31.294701+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1a","version":"77'1","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.333508+0000","last_change":"2023-11-08T19:27:31.320446+0000","last_active":"2023-11-08T19:27:32.333508+0000","last_peered":"2023-11-08T19:27:32.333508+0000","last_clean":"2023-11-08T19:27:32.333508+0000","last_became_active":"2023-11-08T19:27:31.320148+0000","last_became_peered":"2023-11-08T19:27:31.320148+0000","last_unstale":"2023-11-08T19:27:32.333508+0000","last_undegraded":"2023-11-08T19:27:32.333508+0000","last_fullsized":"2023-11-08T19:27:32.333508+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_clean_scrub_stamp":"2023-11-08T19:27:30.289497+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:41:32.949582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":21,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.311759+0000","last_change":"2023-11-08T19:27:08.188668+0000","last_active":"2023-11-08T19:27:32.311759+0000","last_peered":"2023-11-08T19:27:32.311759+0000","last_clean":"2023-11-08T19:27:32.311759+0000","last_became_active":"2023-11-08T19:27:08.188441+0000","last_became_peered":"2023-11-08T19:27:08.188441+0000","last_unstale":"2023-11-08T19:27:32.311759+0000","last_undegraded":"2023-11-08T19:27:32.311759+0000","last_fullsized":"2023-11-08T19:27:32.311759+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:45:44.379160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":52,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-08T19:27:31.313661+0000","last_change":"2023-11-08T19:26:06.423754+0000","last_active":"2023-11-08T19:27:31.313661+0000","last_peered":"2023-11-08T19:27:31.313661+0000","last_clean":"2023-11-08T19:27:31.313661+0000","last_became_active":"2023-11-08T19:26:04.599322+0000","last_became_peered":"2023-11-08T19:26:04.599322+0000","last_unstale":"2023-11-08T19:27:31.313661+0000","last_undegraded":"2023-11-08T19:27:31.313661+0000","last_fullsized":"2023-11-08T19:27:31.313661+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:13:08.689128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042670000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":26,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-08T19:27:27.306290+0000","last_change":"2023-11-08T19:27:09.181656+0000","last_active":"2023-11-08T19:27:27.306290+0000","last_peered":"2023-11-08T19:27:27.306290+0000","last_clean":"2023-11-08T19:27:27.306290+0000","last_became_active":"2023-11-08T19:27:07.187337+0000","last_became_peered":"2023-11-08T19:27:07.187337+0000","last_unstale":"2023-11-08T19:27:27.306290+0000","last_undegraded":"2023-11-08T19:27:27.306290+0000","last_fullsized":"2023-11-08T19:27:27.306290+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:14:17.605057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000346967,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:27:31.294701+0000","last_change":"2023-11-08T19:27:31.294701+0000","last_active":"2023-11-08T19:27:31.294701+0000","last_peered":"2023-11-08T19:27:31.294701+0000","last_clean":"2023-11-08T19:27:31.294701+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:27:31.294701+0000","last_undegraded":"2023-11-08T19:27:31.294701+0000","last_fullsized":"2023-11-08T19:27:31.294701+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:27:30.289497+0000","last_change":"2023-11-08T19:27:30.289497+0000","last_active":"2023-11-08T19:27:30.289497+0000","last_peered":"2023-11-08T19:27:30.289497+0000","last_clean":"2023-11-08T19:27:30.289497+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:27:30.289497+0000","last_undegraded":"2023-11-08T19:27:30.289497+0000","last_fullsized":"2023-11-08T19:27:30.289497+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_clean_scrub_stamp":"2023-11-08T19:27:30.289497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-08T19:27:31.326903+0000","last_change":"2023-11-08T19:27:08.194065+0000","last_active":"2023-11-08T19:27:31.326903+0000","last_peered":"2023-11-08T19:27:31.326903+0000","last_clean":"2023-11-08T19:27:31.326903+0000","last_became_active":"2023-11-08T19:27:08.193532+0000","last_became_peered":"2023-11-08T19:27:08.193532+0000","last_unstale":"2023-11-08T19:27:31.326903+0000","last_undegraded":"2023-11-08T19:27:31.326903+0000","last_fullsized":"2023-11-08T19:27:31.326903+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:27:58.393129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":49,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-08T19:27:31.326259+0000","last_change":"2023-11-08T19:26:06.394488+0000","last_active":"2023-11-08T19:27:31.326259+0000","last_peered":"2023-11-08T19:27:31.326259+0000","last_clean":"2023-11-08T19:27:31.326259+0000","last_became_active":"2023-11-08T19:26:04.606805+0000","last_became_peered":"2023-11-08T19:26:04.606805+0000","last_unstale":"2023-11-08T19:27:31.326259+0000","last_undegraded":"2023-11-08T19:27:31.326259+0000","last_fullsized":"2023-11-08T19:27:31.326259+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:25:20.857133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045317000000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.311868+0000","last_change":"2023-11-08T19:27:09.211647+0000","last_active":"2023-11-08T19:27:32.311868+0000","last_peered":"2023-11-08T19:27:32.311868+0000","last_clean":"2023-11-08T19:27:32.311868+0000","last_became_active":"2023-11-08T19:27:07.190309+0000","last_became_peered":"2023-11-08T19:27:07.190309+0000","last_unstale":"2023-11-08T19:27:32.311868+0000","last_undegraded":"2023-11-08T19:27:32.311868+0000","last_fullsized":"2023-11-08T19:27:32.311868+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:56:04.292154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047615199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.332615+0000","last_change":"2023-11-08T19:27:32.329293+0000","last_active":"2023-11-08T19:27:32.332615+0000","last_peered":"2023-11-08T19:27:32.332615+0000","last_clean":"2023-11-08T19:27:32.332615+0000","last_became_active":"2023-11-08T19:27:32.329046+0000","last_became_peered":"2023-11-08T19:27:32.329046+0000","last_unstale":"2023-11-08T19:27:32.332615+0000","last_undegraded":"2023-11-08T19:27:32.332615+0000","last_fullsized":"2023-11-08T19:27:32.332615+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:24:29.699037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:27:30.289497+0000","last_change":"2023-11-08T19:27:30.289497+0000","last_active":"2023-11-08T19:27:30.289497+0000","last_peered":"2023-11-08T19:27:30.289497+0000","last_clean":"2023-11-08T19:27:30.289497+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:27:30.289497+0000","last_undegraded":"2023-11-08T19:27:30.289497+0000","last_fullsized":"2023-11-08T19:27:30.289497+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_clean_scrub_stamp":"2023-11-08T19:27:30.289497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-08T19:27:31.326274+0000","last_change":"2023-11-08T19:27:08.194884+0000","last_active":"2023-11-08T19:27:31.326274+0000","last_peered":"2023-11-08T19:27:31.326274+0000","last_clean":"2023-11-08T19:27:31.326274+0000","last_became_active":"2023-11-08T19:27:08.193241+0000","last_became_peered":"2023-11-08T19:27:08.193241+0000","last_unstale":"2023-11-08T19:27:31.326274+0000","last_undegraded":"2023-11-08T19:27:31.326274+0000","last_fullsized":"2023-11-08T19:27:31.326274+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:46:31.457682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":50,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.313746+0000","last_change":"2023-11-08T19:26:06.265907+0000","last_active":"2023-11-08T19:27:32.313746+0000","last_peered":"2023-11-08T19:27:32.313746+0000","last_clean":"2023-11-08T19:27:32.313746+0000","last_became_active":"2023-11-08T19:26:04.604772+0000","last_became_peered":"2023-11-08T19:26:04.604772+0000","last_unstale":"2023-11-08T19:27:32.313746+0000","last_undegraded":"2023-11-08T19:27:32.313746+0000","last_fullsized":"2023-11-08T19:27:32.313746+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:44:16.139794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066875300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-08T19:27:31.312115+0000","last_change":"2023-11-08T19:27:09.282699+0000","last_active":"2023-11-08T19:27:31.312115+0000","last_peered":"2023-11-08T19:27:31.312115+0000","last_clean":"2023-11-08T19:27:31.312115+0000","last_became_active":"2023-11-08T19:27:07.191212+0000","last_became_peered":"2023-11-08T19:27:07.191212+0000","last_unstale":"2023-11-08T19:27:31.312115+0000","last_undegraded":"2023-11-08T19:27:31.312115+0000","last_fullsized":"2023-11-08T19:27:31.312115+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:35:14.715110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046640800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.332349+0000","last_change":"2023-11-08T19:27:32.326739+0000","last_active":"2023-11-08T19:27:32.332349+0000","last_peered":"2023-11-08T19:27:32.332349+0000","last_clean":"2023-11-08T19:27:32.332349+0000","last_became_active":"2023-11-08T19:27:32.324726+0000","last_became_peered":"2023-11-08T19:27:32.324726+0000","last_unstale":"2023-11-08T19:27:32.332349+0000","last_undegraded":"2023-11-08T19:27:32.332349+0000","last_fullsized":"2023-11-08T19:27:32.332349+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:22:41.488446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:27:30.289497+0000","last_change":"2023-11-08T19:27:30.289497+0000","last_active":"2023-11-08T19:27:30.289497+0000","last_peered":"2023-11-08T19:27:30.289497+0000","last_clean":"2023-11-08T19:27:30.289497+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:27:30.289497+0000","last_undegraded":"2023-11-08T19:27:30.289497+0000","last_fullsized":"2023-11-08T19:27:30.289497+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_clean_scrub_stamp":"2023-11-08T19:27:30.289497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":21,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.311853+0000","last_change":"2023-11-08T19:27:08.187644+0000","last_active":"2023-11-08T19:27:32.311853+0000","last_peered":"2023-11-08T19:27:32.311853+0000","last_clean":"2023-11-08T19:27:32.311853+0000","last_became_active":"2023-11-08T19:27:08.187376+0000","last_became_peered":"2023-11-08T19:27:08.187376+0000","last_unstale":"2023-11-08T19:27:32.311853+0000","last_undegraded":"2023-11-08T19:27:32.311853+0000","last_fullsized":"2023-11-08T19:27:32.311853+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:22:40.300185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":49,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-08T19:27:31.303869+0000","last_change":"2023-11-08T19:26:05.606946+0000","last_active":"2023-11-08T19:27:31.303869+0000","last_peered":"2023-11-08T19:27:31.303869+0000","last_clean":"2023-11-08T19:27:31.303869+0000","last_became_active":"2023-11-08T19:26:04.598775+0000","last_became_peered":"2023-11-08T19:26:04.598775+0000","last_unstale":"2023-11-08T19:27:31.303869+0000","last_undegraded":"2023-11-08T19:27:31.303869+0000","last_fullsized":"2023-11-08T19:27:31.303869+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:20:56.898927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049870399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-08T19:27:31.325951+0000","last_change":"2023-11-08T19:27:09.331646+0000","last_active":"2023-11-08T19:27:31.325951+0000","last_peered":"2023-11-08T19:27:31.325951+0000","last_clean":"2023-11-08T19:27:31.325951+0000","last_became_active":"2023-11-08T19:27:07.190619+0000","last_became_peered":"2023-11-08T19:27:07.190619+0000","last_unstale":"2023-11-08T19:27:31.325951+0000","last_undegraded":"2023-11-08T19:27:31.325951+0000","last_fullsized":"2023-11-08T19:27:31.325951+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:42:47.371458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065918400000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.324554+0000","last_change":"2023-11-08T19:27:32.322603+0000","last_active":"2023-11-08T19:27:32.324554+0000","last_peered":"2023-11-08T19:27:32.324554+0000","last_clean":"2023-11-08T19:27:32.324554+0000","last_became_active":"2023-11-08T19:27:32.322302+0000","last_became_peered":"2023-11-08T19:27:32.322302+0000","last_unstale":"2023-11-08T19:27:32.324554+0000","last_undegraded":"2023-11-08T19:27:32.324554+0000","last_fullsized":"2023-11-08T19:27:32.324554+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:57:27.177759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"77'1","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-08T19:27:33.309850+0000","last_change":"2023-11-08T19:27:31.320554+0000","last_active":"2023-11-08T19:27:33.309850+0000","last_peered":"2023-11-08T19:27:33.309850+0000","last_clean":"2023-11-08T19:27:33.309850+0000","last_became_active":"2023-11-08T19:27:31.320355+0000","last_became_peered":"2023-11-08T19:27:31.320355+0000","last_unstale":"2023-11-08T19:27:33.309850+0000","last_undegraded":"2023-11-08T19:27:33.309850+0000","last_fullsized":"2023-11-08T19:27:33.309850+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_clean_scrub_stamp":"2023-11-08T19:27:30.289497+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:21:02.207345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":19,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-08T19:27:27.306357+0000","last_change":"2023-11-08T19:27:08.183798+0000","last_active":"2023-11-08T19:27:27.306357+0000","last_peered":"2023-11-08T19:27:27.306357+0000","last_clean":"2023-11-08T19:27:27.306357+0000","last_became_active":"2023-11-08T19:27:08.183537+0000","last_became_peered":"2023-11-08T19:27:08.183537+0000","last_unstale":"2023-11-08T19:27:27.306357+0000","last_undegraded":"2023-11-08T19:27:27.306357+0000","last_fullsized":"2023-11-08T19:27:27.306357+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:50:39.026265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":54,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-08T19:27:27.305442+0000","last_change":"2023-11-08T19:26:05.610012+0000","last_active":"2023-11-08T19:27:27.305442+0000","last_peered":"2023-11-08T19:27:27.305442+0000","last_clean":"2023-11-08T19:27:27.305442+0000","last_became_active":"2023-11-08T19:26:04.603209+0000","last_became_peered":"2023-11-08T19:26:04.603209+0000","last_unstale":"2023-11-08T19:27:27.305442+0000","last_undegraded":"2023-11-08T19:27:27.305442+0000","last_fullsized":"2023-11-08T19:27:27.305442+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:35:02.306997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053335100000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-08T19:27:31.303605+0000","last_change":"2023-11-08T19:27:09.178405+0000","last_active":"2023-11-08T19:27:31.303605+0000","last_peered":"2023-11-08T19:27:31.303605+0000","last_clean":"2023-11-08T19:27:31.303605+0000","last_became_active":"2023-11-08T19:27:07.188169+0000","last_became_peered":"2023-11-08T19:27:07.188169+0000","last_unstale":"2023-11-08T19:27:31.303605+0000","last_undegraded":"2023-11-08T19:27:31.303605+0000","last_fullsized":"2023-11-08T19:27:31.303605+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:39:25.855467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056209700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.321278+0000","last_change":"2023-11-08T19:27:32.317565+0000","last_active":"2023-11-08T19:27:32.321278+0000","last_peered":"2023-11-08T19:27:32.321278+0000","last_clean":"2023-11-08T19:27:32.321278+0000","last_became_active":"2023-11-08T19:27:32.317307+0000","last_became_peered":"2023-11-08T19:27:32.317307+0000","last_unstale":"2023-11-08T19:27:32.321278+0000","last_undegraded":"2023-11-08T19:27:32.321278+0000","last_fullsized":"2023-11-08T19:27:32.321278+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:03:43.454301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.308725+0000","last_change":"2023-11-08T19:27:31.317219+0000","last_active":"2023-11-08T19:27:32.308725+0000","last_peered":"2023-11-08T19:27:32.308725+0000","last_clean":"2023-11-08T19:27:32.308725+0000","last_became_active":"2023-11-08T19:27:31.316299+0000","last_became_peered":"2023-11-08T19:27:31.316299+0000","last_unstale":"2023-11-08T19:27:32.308725+0000","last_undegraded":"2023-11-08T19:27:32.308725+0000","last_fullsized":"2023-11-08T19:27:32.308725+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_clean_scrub_stamp":"2023-11-08T19:27:30.289497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:22:50.960778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-08T19:27:31.325684+0000","last_change":"2023-11-08T19:27:08.193508+0000","last_active":"2023-11-08T19:27:31.325684+0000","last_peered":"2023-11-08T19:27:31.325684+0000","last_clean":"2023-11-08T19:27:31.325684+0000","last_became_active":"2023-11-08T19:27:08.193037+0000","last_became_peered":"2023-11-08T19:27:08.193037+0000","last_unstale":"2023-11-08T19:27:31.325684+0000","last_undegraded":"2023-11-08T19:27:31.325684+0000","last_fullsized":"2023-11-08T19:27:31.325684+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:29:50.067887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":50,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.311882+0000","last_change":"2023-11-08T19:26:06.265318+0000","last_active":"2023-11-08T19:27:32.311882+0000","last_peered":"2023-11-08T19:27:32.311882+0000","last_clean":"2023-11-08T19:27:32.311882+0000","last_became_active":"2023-11-08T19:26:04.603409+0000","last_became_peered":"2023-11-08T19:26:04.603409+0000","last_unstale":"2023-11-08T19:27:32.311882+0000","last_undegraded":"2023-11-08T19:27:32.311882+0000","last_fullsized":"2023-11-08T19:27:32.311882+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:27:24.933111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065684300000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":26,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-08T19:27:27.305289+0000","last_change":"2023-11-08T19:27:09.182282+0000","last_active":"2023-11-08T19:27:27.305289+0000","last_peered":"2023-11-08T19:27:27.305289+0000","last_clean":"2023-11-08T19:27:27.305289+0000","last_became_active":"2023-11-08T19:27:07.186944+0000","last_became_peered":"2023-11-08T19:27:07.186944+0000","last_unstale":"2023-11-08T19:27:27.305289+0000","last_undegraded":"2023-11-08T19:27:27.305289+0000","last_fullsized":"2023-11-08T19:27:27.305289+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:45:21.079011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00028665599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.324720+0000","last_change":"2023-11-08T19:27:32.321504+0000","last_active":"2023-11-08T19:27:32.324720+0000","last_peered":"2023-11-08T19:27:32.324720+0000","last_clean":"2023-11-08T19:27:32.324720+0000","last_became_active":"2023-11-08T19:27:32.321104+0000","last_became_peered":"2023-11-08T19:27:32.321104+0000","last_unstale":"2023-11-08T19:27:32.324720+0000","last_undegraded":"2023-11-08T19:27:32.324720+0000","last_fullsized":"2023-11-08T19:27:32.324720+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:20:42.456197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"77'8","reported_seq":19,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.334964+0000","last_change":"2023-11-08T19:27:31.320269+0000","last_active":"2023-11-08T19:27:32.334964+0000","last_peered":"2023-11-08T19:27:32.334964+0000","last_clean":"2023-11-08T19:27:32.334964+0000","last_became_active":"2023-11-08T19:27:31.319962+0000","last_became_peered":"2023-11-08T19:27:31.319962+0000","last_unstale":"2023-11-08T19:27:32.334964+0000","last_undegraded":"2023-11-08T19:27:32.334964+0000","last_fullsized":"2023-11-08T19:27:32.334964+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_clean_scrub_stamp":"2023-11-08T19:27:30.289497+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:51:53.374320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-08T19:27:31.313714+0000","last_change":"2023-11-08T19:27:08.193990+0000","last_active":"2023-11-08T19:27:31.313714+0000","last_peered":"2023-11-08T19:27:31.313714+0000","last_clean":"2023-11-08T19:27:31.313714+0000","last_became_active":"2023-11-08T19:27:08.193624+0000","last_became_peered":"2023-11-08T19:27:08.193624+0000","last_unstale":"2023-11-08T19:27:31.313714+0000","last_undegraded":"2023-11-08T19:27:31.313714+0000","last_fullsized":"2023-11-08T19:27:31.313714+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:46:41.321120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":52,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-08T19:27:31.311786+0000","last_change":"2023-11-08T19:26:06.423863+0000","last_active":"2023-11-08T19:27:31.311786+0000","last_peered":"2023-11-08T19:27:31.311786+0000","last_clean":"2023-11-08T19:27:31.311786+0000","last_became_active":"2023-11-08T19:26:04.608639+0000","last_became_peered":"2023-11-08T19:26:04.608639+0000","last_unstale":"2023-11-08T19:27:31.311786+0000","last_undegraded":"2023-11-08T19:27:31.311786+0000","last_fullsized":"2023-11-08T19:27:31.311786+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:04:43.631102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044384999999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-08T19:27:31.314667+0000","last_change":"2023-11-08T19:27:09.283232+0000","last_active":"2023-11-08T19:27:31.314667+0000","last_peered":"2023-11-08T19:27:31.314667+0000","last_clean":"2023-11-08T19:27:31.314667+0000","last_became_active":"2023-11-08T19:27:07.190931+0000","last_became_peered":"2023-11-08T19:27:07.190931+0000","last_unstale":"2023-11-08T19:27:31.314667+0000","last_undegraded":"2023-11-08T19:27:31.314667+0000","last_fullsized":"2023-11-08T19:27:31.314667+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:43:28.225967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034146799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.333802+0000","last_change":"2023-11-08T19:27:32.331085+0000","last_active":"2023-11-08T19:27:32.333802+0000","last_peered":"2023-11-08T19:27:32.333802+0000","last_clean":"2023-11-08T19:27:32.333802+0000","last_became_active":"2023-11-08T19:27:32.330779+0000","last_became_peered":"2023-11-08T19:27:32.330779+0000","last_unstale":"2023-11-08T19:27:32.333802+0000","last_undegraded":"2023-11-08T19:27:32.333802+0000","last_fullsized":"2023-11-08T19:27:32.333802+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:28:22.520545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:27:30.289497+0000","last_change":"2023-11-08T19:27:30.289497+0000","last_active":"2023-11-08T19:27:30.289497+0000","last_peered":"2023-11-08T19:27:30.289497+0000","last_clean":"2023-11-08T19:27:30.289497+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:27:30.289497+0000","last_undegraded":"2023-11-08T19:27:30.289497+0000","last_fullsized":"2023-11-08T19:27:30.289497+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_clean_scrub_stamp":"2023-11-08T19:27:30.289497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":21,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-08T19:27:31.312243+0000","last_change":"2023-11-08T19:27:08.187252+0000","last_active":"2023-11-08T19:27:31.312243+0000","last_peered":"2023-11-08T19:27:31.312243+0000","last_clean":"2023-11-08T19:27:31.312243+0000","last_became_active":"2023-11-08T19:27:08.186952+0000","last_became_peered":"2023-11-08T19:27:08.186952+0000","last_unstale":"2023-11-08T19:27:31.312243+0000","last_undegraded":"2023-11-08T19:27:31.312243+0000","last_fullsized":"2023-11-08T19:27:31.312243+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:12:52.022374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":49,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-08T19:27:31.325983+0000","last_change":"2023-11-08T19:26:06.394081+0000","last_active":"2023-11-08T19:27:31.325983+0000","last_peered":"2023-11-08T19:27:31.325983+0000","last_clean":"2023-11-08T19:27:31.325983+0000","last_became_active":"2023-11-08T19:26:04.602137+0000","last_became_peered":"2023-11-08T19:26:04.602137+0000","last_unstale":"2023-11-08T19:27:31.325983+0000","last_undegraded":"2023-11-08T19:27:31.325983+0000","last_fullsized":"2023-11-08T19:27:31.325983+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:05:46.951663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056998200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-08T19:27:31.304150+0000","last_change":"2023-11-08T19:27:09.178816+0000","last_active":"2023-11-08T19:27:31.304150+0000","last_peered":"2023-11-08T19:27:31.304150+0000","last_clean":"2023-11-08T19:27:31.304150+0000","last_became_active":"2023-11-08T19:27:07.192001+0000","last_became_peered":"2023-11-08T19:27:07.192001+0000","last_unstale":"2023-11-08T19:27:31.304150+0000","last_undegraded":"2023-11-08T19:27:31.304150+0000","last_fullsized":"2023-11-08T19:27:31.304150+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:04:12.481212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045359000000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-08T19:27:33.308306+0000","last_change":"2023-11-08T19:27:32.325945+0000","last_active":"2023-11-08T19:27:33.308306+0000","last_peered":"2023-11-08T19:27:33.308306+0000","last_clean":"2023-11-08T19:27:33.308306+0000","last_became_active":"2023-11-08T19:27:32.325695+0000","last_became_peered":"2023-11-08T19:27:32.325695+0000","last_unstale":"2023-11-08T19:27:33.308306+0000","last_undegraded":"2023-11-08T19:27:33.308306+0000","last_fullsized":"2023-11-08T19:27:33.308306+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:43:12.503877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.315606+0000","last_change":"2023-11-08T19:27:31.326547+0000","last_active":"2023-11-08T19:27:32.315606+0000","last_peered":"2023-11-08T19:27:32.315606+0000","last_clean":"2023-11-08T19:27:32.315606+0000","last_became_active":"2023-11-08T19:27:31.325766+0000","last_became_peered":"2023-11-08T19:27:31.325766+0000","last_unstale":"2023-11-08T19:27:32.315606+0000","last_undegraded":"2023-11-08T19:27:32.315606+0000","last_fullsized":"2023-11-08T19:27:32.315606+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_clean_scrub_stamp":"2023-11-08T19:27:30.289497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:28:53.687831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-08T19:27:31.310810+0000","last_change":"2023-11-08T19:27:08.193836+0000","last_active":"2023-11-08T19:27:31.310810+0000","last_peered":"2023-11-08T19:27:31.310810+0000","last_clean":"2023-11-08T19:27:31.310810+0000","last_became_active":"2023-11-08T19:27:08.193458+0000","last_became_peered":"2023-11-08T19:27:08.193458+0000","last_unstale":"2023-11-08T19:27:31.310810+0000","last_undegraded":"2023-11-08T19:27:31.310810+0000","last_fullsized":"2023-11-08T19:27:31.310810+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:14:18.610061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":51,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-08T19:27:31.313794+0000","last_change":"2023-11-08T19:26:06.541338+0000","last_active":"2023-11-08T19:27:31.313794+0000","last_peered":"2023-11-08T19:27:31.313794+0000","last_clean":"2023-11-08T19:27:31.313794+0000","last_became_active":"2023-11-08T19:26:04.602183+0000","last_became_peered":"2023-11-08T19:26:04.602183+0000","last_unstale":"2023-11-08T19:27:31.313794+0000","last_undegraded":"2023-11-08T19:27:31.313794+0000","last_fullsized":"2023-11-08T19:27:31.313794+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:06:47.446686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052029099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":26,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-08T19:27:27.306187+0000","last_change":"2023-11-08T19:27:09.181622+0000","last_active":"2023-11-08T19:27:27.306187+0000","last_peered":"2023-11-08T19:27:27.306187+0000","last_clean":"2023-11-08T19:27:27.306187+0000","last_became_active":"2023-11-08T19:27:07.184507+0000","last_became_peered":"2023-11-08T19:27:07.184507+0000","last_unstale":"2023-11-08T19:27:27.306187+0000","last_undegraded":"2023-11-08T19:27:27.306187+0000","last_fullsized":"2023-11-08T19:27:27.306187+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:58:48.357266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051924700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.335780+0000","last_change":"2023-11-08T19:27:32.327660+0000","last_active":"2023-11-08T19:27:32.335780+0000","last_peered":"2023-11-08T19:27:32.335780+0000","last_clean":"2023-11-08T19:27:32.335780+0000","last_became_active":"2023-11-08T19:27:32.327429+0000","last_became_peered":"2023-11-08T19:27:32.327429+0000","last_unstale":"2023-11-08T19:27:32.335780+0000","last_undegraded":"2023-11-08T19:27:32.335780+0000","last_fullsized":"2023-11-08T19:27:32.335780+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:33:46.073688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"77'3","reported_seq":15,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-08T19:27:33.310765+0000","last_change":"2023-11-08T19:27:31.325270+0000","last_active":"2023-11-08T19:27:33.310765+0000","last_peered":"2023-11-08T19:27:33.310765+0000","last_clean":"2023-11-08T19:27:33.310765+0000","last_became_active":"2023-11-08T19:27:31.324996+0000","last_became_peered":"2023-11-08T19:27:31.324996+0000","last_unstale":"2023-11-08T19:27:33.310765+0000","last_undegraded":"2023-11-08T19:27:33.310765+0000","last_fullsized":"2023-11-08T19:27:33.310765+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_clean_scrub_stamp":"2023-11-08T19:27:30.289497+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:28:21.515340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-08T19:27:31.313158+0000","last_change":"2023-11-08T19:27:08.194263+0000","last_active":"2023-11-08T19:27:31.313158+0000","last_peered":"2023-11-08T19:27:31.313158+0000","last_clean":"2023-11-08T19:27:31.313158+0000","last_became_active":"2023-11-08T19:27:08.193938+0000","last_became_peered":"2023-11-08T19:27:08.193938+0000","last_unstale":"2023-11-08T19:27:31.313158+0000","last_undegraded":"2023-11-08T19:27:31.313158+0000","last_fullsized":"2023-11-08T19:27:31.313158+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:40:49.009717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":50,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.312557+0000","last_change":"2023-11-08T19:26:06.265690+0000","last_active":"2023-11-08T19:27:32.312557+0000","last_peered":"2023-11-08T19:27:32.312557+0000","last_clean":"2023-11-08T19:27:32.312557+0000","last_became_active":"2023-11-08T19:26:04.603397+0000","last_became_peered":"2023-11-08T19:26:04.603397+0000","last_unstale":"2023-11-08T19:27:32.312557+0000","last_undegraded":"2023-11-08T19:27:32.312557+0000","last_fullsized":"2023-11-08T19:27:32.312557+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:50:43.827570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053361499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-08T19:27:31.312744+0000","last_change":"2023-11-08T19:27:09.282750+0000","last_active":"2023-11-08T19:27:31.312744+0000","last_peered":"2023-11-08T19:27:31.312744+0000","last_clean":"2023-11-08T19:27:31.312744+0000","last_became_active":"2023-11-08T19:27:07.189943+0000","last_became_peered":"2023-11-08T19:27:07.189943+0000","last_unstale":"2023-11-08T19:27:31.312744+0000","last_undegraded":"2023-11-08T19:27:31.312744+0000","last_fullsized":"2023-11-08T19:27:31.312744+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:31:16.081058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000398725,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.331875+0000","last_change":"2023-11-08T19:27:32.325712+0000","last_active":"2023-11-08T19:27:32.331875+0000","last_peered":"2023-11-08T19:27:32.331875+0000","last_clean":"2023-11-08T19:27:32.331875+0000","last_became_active":"2023-11-08T19:27:32.325440+0000","last_became_peered":"2023-11-08T19:27:32.325440+0000","last_unstale":"2023-11-08T19:27:32.331875+0000","last_undegraded":"2023-11-08T19:27:32.331875+0000","last_fullsized":"2023-11-08T19:27:32.331875+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:48:29.498190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"77'6","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.337588+0000","last_change":"2023-11-08T19:27:31.330983+0000","last_active":"2023-11-08T19:27:32.337588+0000","last_peered":"2023-11-08T19:27:32.337588+0000","last_clean":"2023-11-08T19:27:32.337588+0000","last_became_active":"2023-11-08T19:27:31.330683+0000","last_became_peered":"2023-11-08T19:27:31.330683+0000","last_unstale":"2023-11-08T19:27:32.337588+0000","last_undegraded":"2023-11-08T19:27:32.337588+0000","last_fullsized":"2023-11-08T19:27:32.337588+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_clean_scrub_stamp":"2023-11-08T19:27:30.289497+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:20:52.279156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":19,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-08T19:27:27.306179+0000","last_change":"2023-11-08T19:27:08.188147+0000","last_active":"2023-11-08T19:27:27.306179+0000","last_peered":"2023-11-08T19:27:27.306179+0000","last_clean":"2023-11-08T19:27:27.306179+0000","last_became_active":"2023-11-08T19:27:08.187487+0000","last_became_peered":"2023-11-08T19:27:08.187487+0000","last_unstale":"2023-11-08T19:27:27.306179+0000","last_undegraded":"2023-11-08T19:27:27.306179+0000","last_fullsized":"2023-11-08T19:27:27.306179+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:22:40.300185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":52,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-08T19:27:31.311969+0000","last_change":"2023-11-08T19:26:06.425133+0000","last_active":"2023-11-08T19:27:31.311969+0000","last_peered":"2023-11-08T19:27:31.311969+0000","last_clean":"2023-11-08T19:27:31.311969+0000","last_became_active":"2023-11-08T19:26:04.601103+0000","last_became_peered":"2023-11-08T19:26:04.601103+0000","last_unstale":"2023-11-08T19:27:31.311969+0000","last_undegraded":"2023-11-08T19:27:31.311969+0000","last_fullsized":"2023-11-08T19:27:31.311969+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:31:14.042561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000229244,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":26,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-08T19:27:27.304929+0000","last_change":"2023-11-08T19:27:09.179852+0000","last_active":"2023-11-08T19:27:27.304929+0000","last_peered":"2023-11-08T19:27:27.304929+0000","last_clean":"2023-11-08T19:27:27.304929+0000","last_became_active":"2023-11-08T19:27:07.191926+0000","last_became_peered":"2023-11-08T19:27:07.191926+0000","last_unstale":"2023-11-08T19:27:27.304929+0000","last_undegraded":"2023-11-08T19:27:27.304929+0000","last_fullsized":"2023-11-08T19:27:27.304929+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:20:38.080129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055457100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-08T19:27:33.308961+0000","last_change":"2023-11-08T19:27:32.336494+0000","last_active":"2023-11-08T19:27:33.308961+0000","last_peered":"2023-11-08T19:27:33.308961+0000","last_clean":"2023-11-08T19:27:33.308961+0000","last_became_active":"2023-11-08T19:27:32.336133+0000","last_became_peered":"2023-11-08T19:27:32.336133+0000","last_unstale":"2023-11-08T19:27:33.308961+0000","last_undegraded":"2023-11-08T19:27:33.308961+0000","last_fullsized":"2023-11-08T19:27:33.308961+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:25:43.090636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.309954+0000","last_change":"2023-11-08T19:27:31.317961+0000","last_active":"2023-11-08T19:27:32.309954+0000","last_peered":"2023-11-08T19:27:32.309954+0000","last_clean":"2023-11-08T19:27:32.309954+0000","last_became_active":"2023-11-08T19:27:31.317699+0000","last_became_peered":"2023-11-08T19:27:31.317699+0000","last_unstale":"2023-11-08T19:27:32.309954+0000","last_undegraded":"2023-11-08T19:27:32.309954+0000","last_fullsized":"2023-11-08T19:27:32.309954+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_clean_scrub_stamp":"2023-11-08T19:27:30.289497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:45:45.206227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":19,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-08T19:27:27.306353+0000","last_change":"2023-11-08T19:27:08.186054+0000","last_active":"2023-11-08T19:27:27.306353+0000","last_peered":"2023-11-08T19:27:27.306353+0000","last_clean":"2023-11-08T19:27:27.306353+0000","last_became_active":"2023-11-08T19:27:08.185581+0000","last_became_peered":"2023-11-08T19:27:08.185581+0000","last_unstale":"2023-11-08T19:27:27.306353+0000","last_undegraded":"2023-11-08T19:27:27.306353+0000","last_fullsized":"2023-11-08T19:27:27.306353+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:00:55.878397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":54,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-08T19:27:27.305097+0000","last_change":"2023-11-08T19:26:05.609625+0000","last_active":"2023-11-08T19:27:27.305097+0000","last_peered":"2023-11-08T19:27:27.305097+0000","last_clean":"2023-11-08T19:27:27.305097+0000","last_became_active":"2023-11-08T19:26:04.606312+0000","last_became_peered":"2023-11-08T19:26:04.606312+0000","last_unstale":"2023-11-08T19:27:27.305097+0000","last_undegraded":"2023-11-08T19:27:27.305097+0000","last_fullsized":"2023-11-08T19:27:27.305097+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:54:42.032901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000505935,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-08T19:27:31.303571+0000","last_change":"2023-11-08T19:27:09.177985+0000","last_active":"2023-11-08T19:27:31.303571+0000","last_peered":"2023-11-08T19:27:31.303571+0000","last_clean":"2023-11-08T19:27:31.303571+0000","last_became_active":"2023-11-08T19:27:07.193254+0000","last_became_peered":"2023-11-08T19:27:07.193254+0000","last_unstale":"2023-11-08T19:27:31.303571+0000","last_undegraded":"2023-11-08T19:27:31.303571+0000","last_fullsized":"2023-11-08T19:27:31.303571+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:23:38.662629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041830799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:27:31.294701+0000","last_change":"2023-11-08T19:27:31.294701+0000","last_active":"2023-11-08T19:27:31.294701+0000","last_peered":"2023-11-08T19:27:31.294701+0000","last_clean":"2023-11-08T19:27:31.294701+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:27:31.294701+0000","last_undegraded":"2023-11-08T19:27:31.294701+0000","last_fullsized":"2023-11-08T19:27:31.294701+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.315417+0000","last_change":"2023-11-08T19:27:31.318789+0000","last_active":"2023-11-08T19:27:32.315417+0000","last_peered":"2023-11-08T19:27:32.315417+0000","last_clean":"2023-11-08T19:27:32.315417+0000","last_became_active":"2023-11-08T19:27:31.318530+0000","last_became_peered":"2023-11-08T19:27:31.318530+0000","last_unstale":"2023-11-08T19:27:32.315417+0000","last_undegraded":"2023-11-08T19:27:32.315417+0000","last_fullsized":"2023-11-08T19:27:32.315417+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_clean_scrub_stamp":"2023-11-08T19:27:30.289497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:54:43.271973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":19,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-08T19:27:27.306412+0000","last_change":"2023-11-08T19:27:08.188860+0000","last_active":"2023-11-08T19:27:27.306412+0000","last_peered":"2023-11-08T19:27:27.306412+0000","last_clean":"2023-11-08T19:27:27.306412+0000","last_became_active":"2023-11-08T19:27:08.188560+0000","last_became_peered":"2023-11-08T19:27:08.188560+0000","last_unstale":"2023-11-08T19:27:27.306412+0000","last_undegraded":"2023-11-08T19:27:27.306412+0000","last_fullsized":"2023-11-08T19:27:27.306412+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:18:20.969491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":50,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-08T19:27:31.307866+0000","last_change":"2023-11-08T19:26:06.297255+0000","last_active":"2023-11-08T19:27:31.307866+0000","last_peered":"2023-11-08T19:27:31.307866+0000","last_clean":"2023-11-08T19:27:31.307866+0000","last_became_active":"2023-11-08T19:26:04.603979+0000","last_became_peered":"2023-11-08T19:26:04.603979+0000","last_unstale":"2023-11-08T19:27:31.307866+0000","last_undegraded":"2023-11-08T19:27:31.307866+0000","last_fullsized":"2023-11-08T19:27:31.307866+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:40:11.164951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079032099999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-08T19:27:31.325852+0000","last_change":"2023-11-08T19:27:09.331990+0000","last_active":"2023-11-08T19:27:31.325852+0000","last_peered":"2023-11-08T19:27:31.325852+0000","last_clean":"2023-11-08T19:27:31.325852+0000","last_became_active":"2023-11-08T19:27:07.185478+0000","last_became_peered":"2023-11-08T19:27:07.185478+0000","last_unstale":"2023-11-08T19:27:31.325852+0000","last_undegraded":"2023-11-08T19:27:31.325852+0000","last_fullsized":"2023-11-08T19:27:31.325852+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:20:38.080129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010661850000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":108,"acting":108,"num_store_stats":6},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":1772,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":23,"ondisk_log_size":23,"up":42,"acting":42,"num_store_stats":6},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574885,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28492,"kb_used_data":1460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343348,"statfs":{"total":96636764160,"available":96607588352,"internally_reserved":0,"allocated":1495040,"data_stored":998129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:26:48 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.504,"15min":0.498},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.78600000000000003,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.51700000000000002,"15min":0.51500000000000001},"min":{"1min":0.35099999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.65400000000000003,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.36899999999999999}]},{"osd":1,"last update":"Wed Nov 8 19:26:48 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.56999999999999995,"15min":0.56599999999999995},"min":{"1min":0.42099999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.30399999999999999},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.56100000000000005,"15min":0.55500000000000005},"min":{"1min":0.432,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.76100000000000001,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.435}]},{"osd":2,"last update":"Wed Nov 8 19:26:48 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.501,"15min":0.49099999999999999},"min":{"1min":0.39500000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.55400000000000005,"15min":0.54900000000000004},"min":{"1min":0.42999999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.54300000000000004}]},{"osd":3,"last update":"Wed Nov 8 19:26:48 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56000000000000005,"15min":0.55900000000000005},"min":{"1min":0.442,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.58899999999999997,"15min":0.57699999999999996},"min":{"1min":0.47899999999999998,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.57999999999999996}]},{"osd":4,"last update":"Wed Nov 8 19:26:48 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.51600000000000001,"15min":0.51000000000000001},"min":{"1min":0.38300000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.73299999999999998,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.54500000000000004,"15min":0.54100000000000004},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.52400000000000002}]},{"osd":5,"last update":"Wed Nov 8 19:26:48 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.58699999999999997,"15min":0.58099999999999996},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.497},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.64700000000000002,"15min":0.64300000000000002},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.78300000000000003,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.56399999999999995}]},{"osd":6,"last update":"Wed Nov 8 19:26:48 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.60999999999999999,"15min":0.60699999999999998},"min":{"1min":0.47599999999999998,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.81100000000000005,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.61699999999999999,"15min":0.61199999999999999},"min":{"1min":0.47699999999999998,"5min":0.443,"15min":0.443},"max":{"1min":0.78800000000000003,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.62}]}]},{"osd":6,"up_from":13,"seq":55834574885,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27912,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343928,"statfs":{"total":96636764160,"available":96608182272,"internally_reserved":0,"allocated":901120,"data_stored":407761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:26:47 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":1.26,"15min":1.3799999999999999},"min":{"1min":0.33900000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.91700000000000004,"5min":21.359999999999999,"15min":21.359999999999999},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.86099999999999999,"15min":0.91300000000000003},"min":{"1min":0.38100000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.90300000000000002,"5min":10.329000000000001,"15min":10.329000000000001},"last":0.73799999999999999}]},{"osd":1,"last update":"Wed Nov 8 19:26:47 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.60299999999999998,"15min":0.61699999999999999},"min":{"1min":0.35999999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.86299999999999999,"5min":3.4239999999999999,"15min":3.4239999999999999},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.63100000000000001,"15min":0.64300000000000002},"min":{"1min":0.40999999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.94999999999999996,"5min":3.4809999999999999,"15min":3.4809999999999999},"last":0.61099999999999999}]},{"osd":2,"last update":"Wed Nov 8 19:26:47 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.879,"15min":0.93100000000000005},"min":{"1min":0.38500000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.96399999999999997,"5min":10.398,"15min":10.398},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.67800000000000005,"15min":0.69799999999999995},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.89100000000000001,"5min":5.0170000000000003,"15min":5.0170000000000003},"last":0.71099999999999997}]},{"osd":3,"last update":"Wed Nov 8 19:26:47 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.65600000000000003,"15min":0.66900000000000004},"min":{"1min":0.34599999999999997,"5min":0.317,"15min":0.317},"max":{"1min":0.877,"5min":3.5710000000000002,"15min":3.5710000000000002},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.91000000000000003,"15min":0.96399999999999997},"min":{"1min":0.41099999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.93500000000000005,"5min":10.368,"15min":10.368},"last":0.76600000000000001}]},{"osd":4,"last update":"Wed Nov 8 19:26:47 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.52500000000000002,"15min":0.51300000000000001},"min":{"1min":0.36899999999999999,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.56499999999999995,"15min":0.56000000000000005},"min":{"1min":0.39200000000000002,"5min":0.373,"15min":0.373},"max":{"1min":0.753,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.52100000000000002}]},{"osd":5,"last update":"Wed Nov 8 19:26:47 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.53900000000000003,"15min":0.52900000000000003},"min":{"1min":0.41599999999999998,"5min":0.307,"15min":0.307},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.503},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.58999999999999997,"15min":0.58099999999999996},"min":{"1min":0.51600000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.67500000000000004}]},{"osd":7,"last update":"Wed Nov 8 19:26:47 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.58499999999999996,"15min":0.57499999999999996},"min":{"1min":0.48599999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.84199999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.625},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.59799999999999998,"15min":0.58999999999999997},"min":{"1min":0.498,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.80500000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.59699999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574886,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27948,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343892,"statfs":{"total":96636764160,"available":96608145408,"internally_reserved":0,"allocated":942080,"data_stored":429887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:26:47 2023","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.377,"15min":0.38100000000000001},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.499,"5min":1.389,"15min":1.389},"last":0.254},{"interface":"front","average":{"1min":0.34899999999999998,"5min":0.66500000000000004,"15min":0.71699999999999997},"min":{"1min":0.23499999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.55200000000000005,"5min":9.1370000000000005,"15min":9.1370000000000005},"last":0.36099999999999999}]},{"osd":2,"last update":"Wed Nov 8 19:26:47 2023","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.35999999999999999,"15min":0.36499999999999999},"min":{"1min":0.25600000000000001,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.47799999999999998,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.316},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.42299999999999999,"15min":0.42899999999999999},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.56399999999999995,"5min":1.425,"15min":1.425},"last":0.34799999999999998}]},{"osd":3,"last update":"Wed Nov 8 19:26:47 2023","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.80700000000000005,"15min":0.877},"min":{"1min":0.28000000000000003,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.54300000000000004,"5min":11.957000000000001,"15min":11.957000000000001},"last":0.28799999999999998},{"interface":"front","average":{"1min":0.379,"5min":1.0209999999999999,"15min":1.1279999999999999},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.54400000000000004,"5min":18.039000000000001,"15min":18.039000000000001},"last":0.27700000000000002}]},{"osd":4,"last update":"Wed Nov 8 19:26:47 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.78300000000000003,"15min":0.81399999999999995},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.4370000000000001,"5min":6.181,"15min":6.181},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.64300000000000002,"15min":0.64100000000000001},"min":{"1min":0.45200000000000001,"5min":0.377,"15min":0.377},"max":{"1min":1.411,"5min":1.488,"15min":1.488},"last":0.64700000000000002}]},{"osd":5,"last update":"Wed Nov 8 19:26:47 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.92000000000000004,"15min":0.96399999999999997},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.421,"5min":9.2420000000000009,"15min":9.2420000000000009},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.83299999999999996,"15min":0.86499999999999999},"min":{"1min":0.439,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.2969999999999999,"5min":6.2110000000000003,"15min":6.2110000000000003},"last":0.745}]},{"osd":6,"last update":"Wed Nov 8 19:26:47 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.92800000000000005,"15min":0.96699999999999997},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.4530000000000001,"5min":9.1880000000000006,"15min":9.1880000000000006},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.272,"5min":1.462,"15min":1.462},"last":0.56799999999999995}]},{"osd":7,"last update":"Wed Nov 8 19:26:47 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.82699999999999996,"15min":0.85499999999999998},"min":{"1min":0.46899999999999997,"5min":0.435,"15min":0.435},"max":{"1min":1.3979999999999999,"5min":6.1479999999999997,"15min":6.1479999999999997},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.95699999999999996,"15min":1.004},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.353,"5min":9.2170000000000005,"15min":9.2170000000000005},"last":0.55600000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574886,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28540,"kb_used_data":1520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343300,"statfs":{"total":96636764160,"available":96607539200,"internally_reserved":0,"allocated":1556480,"data_stored":1022503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Nov 8 19:26:46 2023","interfaces":[{"interface":"back","average":{"1min":0.31900000000000001,"5min":0.48199999999999998,"15min":0.50900000000000001},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.45200000000000001,"5min":4.1200000000000001,"15min":4.1200000000000001},"last":0.38},{"interface":"front","average":{"1min":0.36199999999999999,"5min":1.026,"15min":1.137},"min":{"1min":0.26200000000000001,"5min":0.253,"15min":0.253},"max":{"1min":0.52200000000000002,"5min":16.940999999999999,"15min":16.940999999999999},"last":0.26900000000000002}]},{"osd":2,"last update":"Wed Nov 8 19:26:46 2023","interfaces":[{"interface":"back","average":{"1min":0.34200000000000003,"5min":0.39100000000000001,"15min":0.39900000000000002},"min":{"1min":0.25700000000000001,"5min":0.246,"15min":0.246},"max":{"1min":0.54500000000000004,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.28899999999999998},{"interface":"front","average":{"1min":0.378,"5min":0.64500000000000002,"15min":0.68899999999999995},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.56899999999999995,"5min":7.0209999999999999,"15min":7.0209999999999999},"last":0.35699999999999998}]},{"osd":3,"last update":"Wed Nov 8 19:26:46 2023","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.53700000000000003,"15min":0.56299999999999994},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.51500000000000001,"5min":4.0940000000000003,"15min":4.0940000000000003},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.439,"15min":0.44700000000000001},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.62,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.317}]},{"osd":4,"last update":"Wed Nov 8 19:26:46 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.73599999999999999,"15min":0.76600000000000001},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.70799999999999996,"5min":4.0049999999999999,"15min":4.0049999999999999},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.63400000000000001,"15min":0.64000000000000001},"min":{"1min":0.45700000000000002,"5min":0.436,"15min":0.436},"max":{"1min":0.79200000000000004,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.59499999999999997}]},{"osd":5,"last update":"Wed Nov 8 19:26:46 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.753,"15min":0.78000000000000003},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.81699999999999995,"5min":4.3090000000000002,"15min":4.3090000000000002},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.745,"15min":0.76900000000000002},"min":{"1min":0.496,"5min":0.437,"15min":0.437},"max":{"1min":0.77200000000000002,"5min":4.0709999999999997,"15min":4.0709999999999997},"last":0.504}]},{"osd":6,"last update":"Wed Nov 8 19:26:46 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.66500000000000004,"15min":0.67300000000000004},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.82299999999999995,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.621},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.80900000000000005,"15min":0.83799999999999997},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.76000000000000001,"5min":4.2560000000000002,"15min":4.2560000000000002},"last":0.60799999999999998}]},{"osd":7,"last update":"Wed Nov 8 19:26:46 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.76600000000000001,"15min":0.79000000000000004},"min":{"1min":0.49299999999999999,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.81299999999999994,"5min":4.2850000000000001,"15min":4.2850000000000001},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.79200000000000004,"15min":0.81799999999999995},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.85699999999999998,"5min":4.0460000000000003,"15min":4.0460000000000003},"last":0.58599999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574886,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27944,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343896,"statfs":{"total":96636764160,"available":96608149504,"internally_reserved":0,"allocated":937984,"data_stored":429817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:26:46 2023","interfaces":[{"interface":"back","average":{"1min":0.34300000000000003,"5min":0.33400000000000002,"15min":0.33200000000000002},"min":{"1min":0.254,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.496,"5min":0.496,"15min":0.496},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.309,"5min":0.58899999999999997,"15min":0.63600000000000001},"min":{"1min":0.23400000000000001,"5min":0.218,"15min":0.218},"max":{"1min":0.45600000000000002,"5min":6.0999999999999996,"15min":6.0999999999999996},"last":0.36099999999999999}]},{"osd":1,"last update":"Wed Nov 8 19:26:46 2023","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.44,"5min":0.48299999999999998,"15min":0.48299999999999998},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.38600000000000001,"15min":0.38900000000000001},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.48899999999999999,"5min":0.624,"15min":0.624},"last":0.41899999999999998}]},{"osd":3,"last update":"Wed Nov 8 19:26:46 2023","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.38100000000000001,"15min":0.38400000000000001},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.46000000000000002,"5min":0.53700000000000003,"15min":0.53700000000000003},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.39000000000000001,"15min":0.39400000000000002},"min":{"1min":0.29399999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.439,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.44700000000000001}]},{"osd":4,"last update":"Wed Nov 8 19:26:46 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.57899999999999996,"15min":0.58299999999999996},"min":{"1min":0.46000000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.71999999999999997,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.57999999999999996,"15min":0.58299999999999996},"min":{"1min":0.40999999999999998,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.74199999999999999,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.68700000000000006}]},{"osd":5,"last update":"Wed Nov 8 19:26:46 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.58599999999999997,"15min":0.58299999999999996},"min":{"1min":0.50700000000000001,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.76400000000000001,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.60899999999999999,"15min":0.61199999999999999},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.76100000000000001,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.76400000000000001}]},{"osd":6,"last update":"Wed Nov 8 19:26:46 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.59799999999999998,"15min":0.59999999999999998},"min":{"1min":0.47699999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.77300000000000002,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.40300000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.70699999999999996,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.80000000000000004}]},{"osd":7,"last update":"Wed Nov 8 19:26:46 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.58199999999999996,"15min":0.57999999999999996},"min":{"1min":0.497,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.81499999999999995,"5min":0.878,"15min":0.878},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.60099999999999998,"15min":0.60299999999999998},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.80000000000000004,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.81799999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574887,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27944,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343896,"statfs":{"total":96636764160,"available":96608149504,"internally_reserved":0,"allocated":937984,"data_stored":429817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:26:49 2023","interfaces":[{"interface":"back","average":{"1min":0.32100000000000001,"5min":0.33800000000000002,"15min":0.34100000000000003},"min":{"1min":0.20300000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.42299999999999999,"5min":0.626,"15min":0.626},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.34999999999999998,"15min":0.34899999999999998},"min":{"1min":0.28100000000000003,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.46899999999999997,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.28799999999999998}]},{"osd":1,"last update":"Wed Nov 8 19:26:49 2023","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.35099999999999998,"15min":0.35299999999999998},"min":{"1min":0.24099999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.42699999999999999,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":0.25900000000000001},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"min":{"1min":0.27900000000000003,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.49199999999999999,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.39400000000000002}]},{"osd":2,"last update":"Wed Nov 8 19:26:49 2023","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.374,"15min":0.371},"min":{"1min":0.30199999999999999,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.48199999999999998,"5min":0.58499999999999996,"15min":0.58499999999999996},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.40500000000000003,"15min":0.40400000000000003},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"last":0.34499999999999997}]},{"osd":4,"last update":"Wed Nov 8 19:26:49 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.60399999999999998,"15min":0.59999999999999998},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.80200000000000005,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.624,"5min":0.60799999999999998,"15min":0.60599999999999998},"min":{"1min":0.45000000000000001,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.79000000000000004,"5min":0.871,"15min":0.871},"last":0.63900000000000001}]},{"osd":5,"last update":"Wed Nov 8 19:26:49 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.58299999999999996,"15min":0.57699999999999996},"min":{"1min":0.39800000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.625,"5min":0.58599999999999997,"15min":0.57999999999999996},"min":{"1min":0.499,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.81999999999999995,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.54100000000000004}]},{"osd":6,"last update":"Wed Nov 8 19:26:49 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.58399999999999996,"15min":0.58099999999999996},"min":{"1min":0.42999999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.61299999999999999,"15min":0.60699999999999998},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.84099999999999997,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.628}]},{"osd":7,"last update":"Wed Nov 8 19:26:49 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.59599999999999997,"15min":0.58999999999999997},"min":{"1min":0.46400000000000002,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.88100000000000001,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.59799999999999998,"15min":0.59599999999999997},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.76600000000000001,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.55300000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574886,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27952,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343888,"statfs":{"total":96636764160,"available":96608141312,"internally_reserved":0,"allocated":946176,"data_stored":429977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:26:50 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.73099999999999998,"15min":0.76000000000000001},"min":{"1min":0.40999999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.78100000000000003,"5min":6.5970000000000004,"15min":6.5970000000000004},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.56599999999999995,"15min":0.57399999999999995},"min":{"1min":0.378,"5min":0.317,"15min":0.317},"max":{"1min":0.75900000000000001,"5min":2.5030000000000001,"15min":2.5030000000000001},"last":0.60399999999999998}]},{"osd":1,"last update":"Wed Nov 8 19:26:50 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.50700000000000001,"15min":0.499},"min":{"1min":0.372,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.51600000000000001,"15min":0.51000000000000001},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.57599999999999996}]},{"osd":2,"last update":"Wed Nov 8 19:26:50 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.55500000000000005,"15min":0.55200000000000005},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.623,"5min":0.80100000000000005,"15min":0.83099999999999996},"min":{"1min":0.45900000000000002,"5min":0.435,"15min":0.435},"max":{"1min":0.871,"5min":6.6920000000000002,"15min":6.6920000000000002},"last":0.44500000000000001}]},{"osd":3,"last update":"Wed Nov 8 19:26:50 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58799999999999997,"15min":0.58899999999999997},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.71099999999999997,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.627,"5min":0.61399999999999999,"15min":0.61199999999999999},"min":{"1min":0.46800000000000003,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.627}]},{"osd":5,"last update":"Wed Nov 8 19:26:50 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.34300000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.59099999999999997,"15min":0.59499999999999997},"min":{"1min":0.42999999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.876,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.67100000000000004}]},{"osd":6,"last update":"Wed Nov 8 19:26:50 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.61799999999999999,"15min":0.61499999999999999},"min":{"1min":0.38900000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.78100000000000003,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.69599999999999995}]},{"osd":7,"last update":"Wed Nov 8 19:26:50 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.625,"15min":0.63200000000000001},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.82699999999999996,"5min":1.141,"15min":1.141},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.63900000000000001,"15min":0.64500000000000002},"min":{"1min":0.38700000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.85299999999999998,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.71099999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574886,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27944,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343896,"statfs":{"total":96636764160,"available":96608149504,"internally_reserved":0,"allocated":937984,"data_stored":429817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:26:44 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.51700000000000002,"15min":0.52100000000000002},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.51100000000000001,"15min":0.50900000000000001},"min":{"1min":0.373,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.65100000000000002,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.38500000000000001}]},{"osd":1,"last update":"Wed Nov 8 19:26:44 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.52200000000000002,"15min":0.52300000000000002},"min":{"1min":0.42099999999999999,"5min":0.371,"15min":0.371},"max":{"1min":0.65300000000000002,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53500000000000003,"15min":0.53600000000000003},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.46000000000000002}]},{"osd":2,"last update":"Wed Nov 8 19:26:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.53800000000000003,"15min":0.53600000000000003},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.441},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.56599999999999995,"15min":0.57299999999999995},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.69599999999999995,"5min":0.747,"15min":0.747},"last":0.503}]},{"osd":3,"last update":"Wed Nov 8 19:26:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.54600000000000004,"15min":0.54900000000000004},"min":{"1min":0.39700000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.59099999999999997,"15min":0.59999999999999998},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.70099999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.56299999999999994}]},{"osd":4,"last update":"Wed Nov 8 19:26:44 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.55100000000000005,"15min":0.55900000000000005},"min":{"1min":0.38600000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.63300000000000001,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.39100000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.71399999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.54900000000000004}]},{"osd":6,"last update":"Wed Nov 8 19:26:44 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.60899999999999999,"15min":0.61199999999999999},"min":{"1min":0.42099999999999999,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.61099999999999999,"15min":0.61499999999999999},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.67700000000000005,"5min":0.745,"15min":0.745},"last":0.52800000000000002}]},{"osd":7,"last update":"Wed Nov 8 19:26:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.60299999999999998,"15min":0.60899999999999999},"min":{"1min":0.371,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.63900000000000001,"15min":0.64700000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.77100000000000002,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.63400000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-08T19:27:36.443 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-08T19:27:36.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.582+0000 7fdcb6986700 1 Processor -- start 2023-11-08T19:27:36.590 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.582+0000 7fdcb6986700 1 -- start start 2023-11-08T19:27:36.590 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.583+0000 7fdcb6986700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fdcb0159870 0x7fdcb00b2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:36.590 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.583+0000 7fdcb6986700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fdcb006ed60 0x7fdcb006b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:36.590 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.583+0000 7fdcb6986700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcb00b2a60 con 0x7fdcb00cc410 2023-11-08T19:27:36.591 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.583+0000 7fdcb6986700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcb006b090 con 0x7fdcb0159870 2023-11-08T19:27:36.591 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.583+0000 7fdcb6986700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcb00b2d40 con 0x7fdcb006ed60 2023-11-08T19:27:36.591 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.583+0000 7fdcaffff700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fdcb0159870 0x7fdcb00b2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:36.591 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.583+0000 7fdcaffff700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fdcb0159870 0x7fdcb00b2210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53702/0 (socket says 172.21.15.143:53702) 2023-11-08T19:27:36.591 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.583+0000 7fdcaffff700 1 -- 172.21.15.143:0/862274890 learned_addr learned my addr 172.21.15.143:0/862274890 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:36.592 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.583+0000 7fdcb4f23700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fdcb00cc410 0x7fdcb00cc820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:49858/0 (socket says 172.21.15.143:49858) 2023-11-08T19:27:36.592 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.583+0000 7fdcaf7fe700 1 --2- 172.21.15.143:0/862274890 >> v2:172.21.15.174:3300/0 conn(0x7fdcb006ed60 0x7fdcb006b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:36.592 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.583+0000 7fdcaffff700 1 -- 172.21.15.143:0/862274890 >> v2:172.21.15.174:3300/0 conn(0x7fdcb006ed60 msgr2=0x7fdcb006b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:36.592 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.583+0000 7fdcaffff700 1 --2- 172.21.15.143:0/862274890 >> v2:172.21.15.174:3300/0 conn(0x7fdcb006ed60 0x7fdcb006b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:36.592 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.583+0000 7fdcaffff700 1 -- 172.21.15.143:0/862274890 >> v1:172.21.15.143:6789/0 conn(0x7fdcb00cc410 legacy=0x7fdcb00cc820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:36.593 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.584+0000 7fdcaeffd700 1 -- 172.21.15.143:0/862274890 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2073586129 0 0) 0x7fdcb00b2a60 con 0x7fdcb00cc410 2023-11-08T19:27:36.593 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.584+0000 7fdcaffff700 1 -- 172.21.15.143:0/862274890 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcb00b3020 con 0x7fdcb0159870 2023-11-08T19:27:36.593 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.584+0000 7fdcaffff700 1 --2- 172.21.15.143:0/862274890 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdcb0159870 0x7fdcb00b2210 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fdcb0073350 tx=0x7fdca4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=351d53fcdc7577ea server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:36.593 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.584+0000 7fdcaeffd700 1 -- 172.21.15.143:0/862274890 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdca4005e30 con 0x7fdcb0159870 2023-11-08T19:27:36.593 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.584+0000 7fdcaeffd700 1 -- 172.21.15.143:0/862274890 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdca4004030 con 0x7fdcb0159870 2023-11-08T19:27:36.594 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.584+0000 7fdcaeffd700 1 -- 172.21.15.143:0/862274890 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdca40043e0 con 0x7fdcb0159870 2023-11-08T19:27:36.594 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.584+0000 7fdcb6986700 1 -- 172.21.15.143:0/862274890 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdcb0159870 msgr2=0x7fdcb00b2210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:36.594 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.584+0000 7fdcb6986700 1 --2- 172.21.15.143:0/862274890 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdcb0159870 0x7fdcb00b2210 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fdcb0073350 tx=0x7fdca4009bd0 comp rx=0 tx=0).stop 2023-11-08T19:27:36.594 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.584+0000 7fdcb6986700 1 -- 172.21.15.143:0/862274890 shutdown_connections 2023-11-08T19:27:36.595 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.584+0000 7fdcb6986700 1 --2- 172.21.15.143:0/862274890 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdcb0159870 0x7fdcb00b2210 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:36.595 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.584+0000 7fdcb6986700 1 --2- 172.21.15.143:0/862274890 >> v2:172.21.15.174:3300/0 conn(0x7fdcb006ed60 0x7fdcb006b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:36.595 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.584+0000 7fdcb6986700 1 -- 172.21.15.143:0/862274890 >> 172.21.15.143:0/862274890 conn(0x7fdcb0099ab0 msgr2=0x7fdcb00d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:36.595 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.584+0000 7fdcb6986700 1 -- 172.21.15.143:0/862274890 shutdown_connections 2023-11-08T19:27:36.595 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.584+0000 7fdcb6986700 1 -- 172.21.15.143:0/862274890 wait complete. 2023-11-08T19:27:36.595 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.586+0000 7fdcb6986700 1 Processor -- start 2023-11-08T19:27:36.596 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.586+0000 7fdcb6986700 1 -- start start 2023-11-08T19:27:36.596 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.586+0000 7fdcb6986700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdcb006ed60 0x7fdcb00c97f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:36.596 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.586+0000 7fdcb6986700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdcb00cc410 0x7fdcb00b3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:36.596 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.586+0000 7fdcb6986700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdcb0159870 0x7fdcb00b4500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:36.596 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.586+0000 7fdcb6986700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcb00c9210 con 0x7fdcb00cc410 2023-11-08T19:27:36.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.586+0000 7fdcb6986700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcb00c8f30 con 0x7fdcb006ed60 2023-11-08T19:27:36.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.586+0000 7fdcb6986700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcb00b34e0 con 0x7fdcb0159870 2023-11-08T19:27:36.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.587+0000 7fdcaf7fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdcb00cc410 0x7fdcb00b3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:36.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.587+0000 7fdcaf7fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdcb00cc410 0x7fdcb00b3fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53710/0 (socket says 172.21.15.143:53710) 2023-11-08T19:27:36.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.587+0000 7fdcaf7fe700 1 -- 172.21.15.143:0/1075164159 learned_addr learned my addr 172.21.15.143:0/1075164159 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:36.598 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.587+0000 7fdcaffff700 1 --2- 172.21.15.143:0/1075164159 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdcb006ed60 0x7fdcb00c97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:36.598 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.587+0000 7fdcb4f23700 1 --2- 172.21.15.143:0/1075164159 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdcb0159870 0x7fdcb00b4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:36.598 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.587+0000 7fdcaf7fe700 1 -- 172.21.15.143:0/1075164159 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdcb0159870 msgr2=0x7fdcb00b4500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:36.598 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.587+0000 7fdcaf7fe700 1 --2- 172.21.15.143:0/1075164159 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdcb0159870 0x7fdcb00b4500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:36.598 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.587+0000 7fdcaf7fe700 1 -- 172.21.15.143:0/1075164159 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdcb006ed60 msgr2=0x7fdcb00c97f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:36.599 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.587+0000 7fdcaf7fe700 1 --2- 172.21.15.143:0/1075164159 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdcb006ed60 0x7fdcb00c97f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:36.599 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.587+0000 7fdcaf7fe700 1 -- 172.21.15.143:0/1075164159 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdca4009500 con 0x7fdcb00cc410 2023-11-08T19:27:36.599 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.587+0000 7fdcaf7fe700 1 --2- 172.21.15.143:0/1075164159 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdcb00cc410 0x7fdcb00b3fc0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fdca000d800 tx=0x7fdca000dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:36.599 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.587+0000 7fdcacff9700 1 -- 172.21.15.143:0/1075164159 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdca00125f0 con 0x7fdcb00cc410 2023-11-08T19:27:36.599 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.587+0000 7fdcacff9700 1 -- 172.21.15.143:0/1075164159 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdca0012750 con 0x7fdcb00cc410 2023-11-08T19:27:36.600 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.587+0000 7fdcb6986700 1 -- 172.21.15.143:0/1075164159 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcb00b37c0 con 0x7fdcb00cc410 2023-11-08T19:27:36.600 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.587+0000 7fdcacff9700 1 -- 172.21.15.143:0/1075164159 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdca0012b00 con 0x7fdcb00cc410 2023-11-08T19:27:36.600 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.587+0000 7fdcb6986700 1 -- 172.21.15.143:0/1075164159 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcb00c4330 con 0x7fdcb00cc410 2023-11-08T19:27:36.600 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.588+0000 7fdcb6986700 1 -- 172.21.15.143:0/1075164159 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcb00c44c0 con 0x7fdcb00cc410 2023-11-08T19:27:36.601 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.589+0000 7fdcacff9700 1 -- 172.21.15.143:0/1075164159 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fdca0012c60 con 0x7fdcb00cc410 2023-11-08T19:27:36.601 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.589+0000 7fdcacff9700 1 --2- 172.21.15.143:0/1075164159 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fdc9c06af00 0x7fdc9c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:36.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.589+0000 7fdcacff9700 1 -- 172.21.15.143:0/1075164159 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6772+0+0 (secure 0 0 0) 0x7fdca008a8d0 con 0x7fdcb00cc410 2023-11-08T19:27:36.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.590+0000 7fdcaffff700 1 --2- 172.21.15.143:0/1075164159 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fdc9c06af00 0x7fdc9c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:36.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.590+0000 7fdcaffff700 1 --2- 172.21.15.143:0/1075164159 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fdc9c06af00 0x7fdc9c06d3c0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fdcb0073350 tx=0x7fdca4008070 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:36.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.593+0000 7fdcacff9700 1 -- 172.21.15.143:0/1075164159 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdca00555c0 con 0x7fdcb00cc410 2023-11-08T19:27:36.778 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.777+0000 7fdcb6986700 1 -- 172.21.15.143:0/1075164159 --> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fdcb00b2750 con 0x7fdc9c06af00 2023-11-08T19:27:36.793 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.792+0000 7fdcacff9700 1 -- 172.21.15.143:0/1075164159 <== mgr.4099 v2:172.21.15.174:6832/4037280178 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+532277 (secure 0 0 0) 0x7fdca40036a0 con 0x7fdc9c06af00 2023-11-08T19:27:36.799 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:27:36.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.796+0000 7fdcb6986700 1 -- 172.21.15.143:0/1075164159 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fdc9c06af00 msgr2=0x7fdc9c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:36.804 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.796+0000 7fdcb6986700 1 --2- 172.21.15.143:0/1075164159 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fdc9c06af00 0x7fdc9c06d3c0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fdcb0073350 tx=0x7fdca4008070 comp rx=0 tx=0).stop 2023-11-08T19:27:36.804 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.796+0000 7fdcb6986700 1 -- 172.21.15.143:0/1075164159 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdcb00cc410 msgr2=0x7fdcb00b3fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:36.804 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.796+0000 7fdcb6986700 1 --2- 172.21.15.143:0/1075164159 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdcb00cc410 0x7fdcb00b3fc0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fdca000d800 tx=0x7fdca000dbc0 comp rx=0 tx=0).stop 2023-11-08T19:27:36.804 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.796+0000 7fdcb6986700 1 -- 172.21.15.143:0/1075164159 shutdown_connections 2023-11-08T19:27:36.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.796+0000 7fdcb6986700 1 --2- 172.21.15.143:0/1075164159 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fdc9c06af00 0x7fdc9c06d3c0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:36.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.796+0000 7fdcb6986700 1 --2- 172.21.15.143:0/1075164159 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdcb0159870 0x7fdcb00b4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:36.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.796+0000 7fdcb6986700 1 --2- 172.21.15.143:0/1075164159 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdcb006ed60 0x7fdcb00c97f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:36.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.796+0000 7fdcb6986700 1 --2- 172.21.15.143:0/1075164159 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdcb00cc410 0x7fdcb00b3fc0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:36.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.796+0000 7fdcb6986700 1 -- 172.21.15.143:0/1075164159 >> 172.21.15.143:0/1075164159 conn(0x7fdcb0099ab0 msgr2=0x7fdcb00b0650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:36.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.796+0000 7fdcb6986700 1 -- 172.21.15.143:0/1075164159 shutdown_connections 2023-11-08T19:27:36.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.796+0000 7fdcb6986700 1 -- 172.21.15.143:0/1075164159 wait complete. 2023-11-08T19:27:36.806 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2023-11-08T19:27:36.810 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":157,"stamp":"2023-11-08T19:27:36.554012+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":278,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":224780,"kb_used_data":8576,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754749940,"statfs":{"total":773094113280,"available":772863938560,"internally_reserved":0,"allocated":8781824,"data_stored":4638818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001760"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.335728+0000","last_change":"2023-11-08T19:27:32.328141+0000","last_active":"2023-11-08T19:27:32.335728+0000","last_peered":"2023-11-08T19:27:32.335728+0000","last_clean":"2023-11-08T19:27:32.335728+0000","last_became_active":"2023-11-08T19:27:32.327903+0000","last_became_peered":"2023-11-08T19:27:32.327903+0000","last_unstale":"2023-11-08T19:27:32.335728+0000","last_undegraded":"2023-11-08T19:27:32.335728+0000","last_fullsized":"2023-11-08T19:27:32.335728+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:05:01.010467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.331281+0000","last_change":"2023-11-08T19:27:32.324220+0000","last_active":"2023-11-08T19:27:32.331281+0000","last_peered":"2023-11-08T19:27:32.331281+0000","last_clean":"2023-11-08T19:27:32.331281+0000","last_became_active":"2023-11-08T19:27:32.323918+0000","last_became_peered":"2023-11-08T19:27:32.323918+0000","last_unstale":"2023-11-08T19:27:32.331281+0000","last_undegraded":"2023-11-08T19:27:32.331281+0000","last_fullsized":"2023-11-08T19:27:32.331281+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:46:55.585097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.336191+0000","last_change":"2023-11-08T19:27:32.335224+0000","last_active":"2023-11-08T19:27:32.336191+0000","last_peered":"2023-11-08T19:27:32.336191+0000","last_clean":"2023-11-08T19:27:32.336191+0000","last_became_active":"2023-11-08T19:27:32.334895+0000","last_became_peered":"2023-11-08T19:27:32.334895+0000","last_unstale":"2023-11-08T19:27:32.336191+0000","last_undegraded":"2023-11-08T19:27:32.336191+0000","last_fullsized":"2023-11-08T19:27:32.336191+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:28:17.551060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.332448+0000","last_change":"2023-11-08T19:27:32.325238+0000","last_active":"2023-11-08T19:27:32.332448+0000","last_peered":"2023-11-08T19:27:32.332448+0000","last_clean":"2023-11-08T19:27:32.332448+0000","last_became_active":"2023-11-08T19:27:32.324599+0000","last_became_peered":"2023-11-08T19:27:32.324599+0000","last_unstale":"2023-11-08T19:27:32.332448+0000","last_undegraded":"2023-11-08T19:27:32.332448+0000","last_fullsized":"2023-11-08T19:27:32.332448+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:18:25.098882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-08T19:27:33.308729+0000","last_change":"2023-11-08T19:27:32.328764+0000","last_active":"2023-11-08T19:27:33.308729+0000","last_peered":"2023-11-08T19:27:33.308729+0000","last_clean":"2023-11-08T19:27:33.308729+0000","last_became_active":"2023-11-08T19:27:32.328599+0000","last_became_peered":"2023-11-08T19:27:32.328599+0000","last_unstale":"2023-11-08T19:27:33.308729+0000","last_undegraded":"2023-11-08T19:27:33.308729+0000","last_fullsized":"2023-11-08T19:27:33.308729+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:41:33.954786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.332186+0000","last_change":"2023-11-08T19:27:32.326288+0000","last_active":"2023-11-08T19:27:32.332186+0000","last_peered":"2023-11-08T19:27:32.332186+0000","last_clean":"2023-11-08T19:27:32.332186+0000","last_became_active":"2023-11-08T19:27:32.326019+0000","last_became_peered":"2023-11-08T19:27:32.326019+0000","last_unstale":"2023-11-08T19:27:32.332186+0000","last_undegraded":"2023-11-08T19:27:32.332186+0000","last_fullsized":"2023-11-08T19:27:32.332186+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:38:32.359142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-08T19:27:33.310432+0000","last_change":"2023-11-08T19:27:32.337045+0000","last_active":"2023-11-08T19:27:33.310432+0000","last_peered":"2023-11-08T19:27:33.310432+0000","last_clean":"2023-11-08T19:27:33.310432+0000","last_became_active":"2023-11-08T19:27:32.336764+0000","last_became_peered":"2023-11-08T19:27:32.336764+0000","last_unstale":"2023-11-08T19:27:33.310432+0000","last_undegraded":"2023-11-08T19:27:33.310432+0000","last_fullsized":"2023-11-08T19:27:33.310432+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:20:53.284360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.335638+0000","last_change":"2023-11-08T19:27:32.332920+0000","last_active":"2023-11-08T19:27:32.335638+0000","last_peered":"2023-11-08T19:27:32.335638+0000","last_clean":"2023-11-08T19:27:32.335638+0000","last_became_active":"2023-11-08T19:27:32.331279+0000","last_became_peered":"2023-11-08T19:27:32.331279+0000","last_unstale":"2023-11-08T19:27:32.335638+0000","last_undegraded":"2023-11-08T19:27:32.335638+0000","last_fullsized":"2023-11-08T19:27:32.335638+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:43:12.503877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-08T19:27:33.309872+0000","last_change":"2023-11-08T19:27:32.323553+0000","last_active":"2023-11-08T19:27:33.309872+0000","last_peered":"2023-11-08T19:27:33.309872+0000","last_clean":"2023-11-08T19:27:33.309872+0000","last_became_active":"2023-11-08T19:27:32.323106+0000","last_became_peered":"2023-11-08T19:27:32.323106+0000","last_unstale":"2023-11-08T19:27:33.309872+0000","last_undegraded":"2023-11-08T19:27:33.309872+0000","last_fullsized":"2023-11-08T19:27:33.309872+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:13:16.149790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.321253+0000","last_change":"2023-11-08T19:27:32.317700+0000","last_active":"2023-11-08T19:27:32.321253+0000","last_peered":"2023-11-08T19:27:32.321253+0000","last_clean":"2023-11-08T19:27:32.321253+0000","last_became_active":"2023-11-08T19:27:32.317372+0000","last_became_peered":"2023-11-08T19:27:32.317372+0000","last_unstale":"2023-11-08T19:27:32.321253+0000","last_undegraded":"2023-11-08T19:27:32.321253+0000","last_fullsized":"2023-11-08T19:27:32.321253+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:00:28.648016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-08T19:27:33.310558+0000","last_change":"2023-11-08T19:27:32.324920+0000","last_active":"2023-11-08T19:27:33.310558+0000","last_peered":"2023-11-08T19:27:33.310558+0000","last_clean":"2023-11-08T19:27:33.310558+0000","last_became_active":"2023-11-08T19:27:32.324665+0000","last_became_peered":"2023-11-08T19:27:32.324665+0000","last_unstale":"2023-11-08T19:27:33.310558+0000","last_undegraded":"2023-11-08T19:27:33.310558+0000","last_fullsized":"2023-11-08T19:27:33.310558+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:21:39.056390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.331630+0000","last_change":"2023-11-08T19:27:32.327016+0000","last_active":"2023-11-08T19:27:32.331630+0000","last_peered":"2023-11-08T19:27:32.331630+0000","last_clean":"2023-11-08T19:27:32.331630+0000","last_became_active":"2023-11-08T19:27:32.326728+0000","last_became_peered":"2023-11-08T19:27:32.326728+0000","last_unstale":"2023-11-08T19:27:32.331630+0000","last_undegraded":"2023-11-08T19:27:32.331630+0000","last_fullsized":"2023-11-08T19:27:32.331630+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:00:52.038756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.333149+0000","last_change":"2023-11-08T19:27:32.330157+0000","last_active":"2023-11-08T19:27:32.333149+0000","last_peered":"2023-11-08T19:27:32.333149+0000","last_clean":"2023-11-08T19:27:32.333149+0000","last_became_active":"2023-11-08T19:27:32.329459+0000","last_became_peered":"2023-11-08T19:27:32.329459+0000","last_unstale":"2023-11-08T19:27:32.333149+0000","last_undegraded":"2023-11-08T19:27:32.333149+0000","last_fullsized":"2023-11-08T19:27:32.333149+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:21:03.212549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-08T19:27:33.308505+0000","last_change":"2023-11-08T19:27:32.333313+0000","last_active":"2023-11-08T19:27:33.308505+0000","last_peered":"2023-11-08T19:27:33.308505+0000","last_clean":"2023-11-08T19:27:33.308505+0000","last_became_active":"2023-11-08T19:27:32.329612+0000","last_became_peered":"2023-11-08T19:27:32.329612+0000","last_unstale":"2023-11-08T19:27:33.308505+0000","last_undegraded":"2023-11-08T19:27:33.308505+0000","last_fullsized":"2023-11-08T19:27:33.308505+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:46:55.585097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.332859+0000","last_change":"2023-11-08T19:27:32.328748+0000","last_active":"2023-11-08T19:27:32.332859+0000","last_peered":"2023-11-08T19:27:32.332859+0000","last_clean":"2023-11-08T19:27:32.332859+0000","last_became_active":"2023-11-08T19:27:32.328457+0000","last_became_peered":"2023-11-08T19:27:32.328457+0000","last_unstale":"2023-11-08T19:27:32.332859+0000","last_undegraded":"2023-11-08T19:27:32.332859+0000","last_fullsized":"2023-11-08T19:27:32.332859+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:04:14.754128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.332386+0000","last_change":"2023-11-08T19:27:32.328017+0000","last_active":"2023-11-08T19:27:32.332386+0000","last_peered":"2023-11-08T19:27:32.332386+0000","last_clean":"2023-11-08T19:27:32.332386+0000","last_became_active":"2023-11-08T19:27:32.327751+0000","last_became_peered":"2023-11-08T19:27:32.327751+0000","last_unstale":"2023-11-08T19:27:32.332386+0000","last_undegraded":"2023-11-08T19:27:32.332386+0000","last_fullsized":"2023-11-08T19:27:32.332386+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:33:46.073688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.334543+0000","last_change":"2023-11-08T19:27:32.333889+0000","last_active":"2023-11-08T19:27:32.334543+0000","last_peered":"2023-11-08T19:27:32.334543+0000","last_clean":"2023-11-08T19:27:32.334543+0000","last_became_active":"2023-11-08T19:27:32.333661+0000","last_became_peered":"2023-11-08T19:27:32.333661+0000","last_unstale":"2023-11-08T19:27:32.334543+0000","last_undegraded":"2023-11-08T19:27:32.334543+0000","last_fullsized":"2023-11-08T19:27:32.334543+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:10:39.776305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.335761+0000","last_change":"2023-11-08T19:27:32.324868+0000","last_active":"2023-11-08T19:27:32.335761+0000","last_peered":"2023-11-08T19:27:32.335761+0000","last_clean":"2023-11-08T19:27:32.335761+0000","last_became_active":"2023-11-08T19:27:32.323651+0000","last_became_peered":"2023-11-08T19:27:32.323651+0000","last_unstale":"2023-11-08T19:27:32.335761+0000","last_undegraded":"2023-11-08T19:27:32.335761+0000","last_fullsized":"2023-11-08T19:27:32.335761+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:50:42.784058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-08T19:27:33.310723+0000","last_change":"2023-11-08T19:27:32.333127+0000","last_active":"2023-11-08T19:27:33.310723+0000","last_peered":"2023-11-08T19:27:33.310723+0000","last_clean":"2023-11-08T19:27:33.310723+0000","last_became_active":"2023-11-08T19:27:32.329831+0000","last_became_peered":"2023-11-08T19:27:32.329831+0000","last_unstale":"2023-11-08T19:27:33.310723+0000","last_undegraded":"2023-11-08T19:27:33.310723+0000","last_fullsized":"2023-11-08T19:27:33.310723+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:32:28.687246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.333223+0000","last_change":"2023-11-08T19:27:32.330456+0000","last_active":"2023-11-08T19:27:32.333223+0000","last_peered":"2023-11-08T19:27:32.333223+0000","last_clean":"2023-11-08T19:27:32.333223+0000","last_became_active":"2023-11-08T19:27:32.329082+0000","last_became_peered":"2023-11-08T19:27:32.329082+0000","last_unstale":"2023-11-08T19:27:32.333223+0000","last_undegraded":"2023-11-08T19:27:32.333223+0000","last_fullsized":"2023-11-08T19:27:32.333223+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:52:05.082696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-08T19:27:33.308892+0000","last_change":"2023-11-08T19:27:32.329643+0000","last_active":"2023-11-08T19:27:33.308892+0000","last_peered":"2023-11-08T19:27:33.308892+0000","last_clean":"2023-11-08T19:27:33.308892+0000","last_became_active":"2023-11-08T19:27:32.329325+0000","last_became_peered":"2023-11-08T19:27:32.329325+0000","last_unstale":"2023-11-08T19:27:33.308892+0000","last_undegraded":"2023-11-08T19:27:33.308892+0000","last_fullsized":"2023-11-08T19:27:33.308892+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:45:46.211431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-08T19:27:33.310102+0000","last_change":"2023-11-08T19:27:32.329346+0000","last_active":"2023-11-08T19:27:33.310102+0000","last_peered":"2023-11-08T19:27:33.310102+0000","last_clean":"2023-11-08T19:27:33.310102+0000","last_became_active":"2023-11-08T19:27:32.329063+0000","last_became_peered":"2023-11-08T19:27:32.329063+0000","last_unstale":"2023-11-08T19:27:33.310102+0000","last_undegraded":"2023-11-08T19:27:33.310102+0000","last_fullsized":"2023-11-08T19:27:33.310102+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:12:54.533391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-08T19:27:33.310118+0000","last_change":"2023-11-08T19:27:32.324965+0000","last_active":"2023-11-08T19:27:33.310118+0000","last_peered":"2023-11-08T19:27:33.310118+0000","last_clean":"2023-11-08T19:27:33.310118+0000","last_became_active":"2023-11-08T19:27:32.324717+0000","last_became_peered":"2023-11-08T19:27:32.324717+0000","last_unstale":"2023-11-08T19:27:33.310118+0000","last_undegraded":"2023-11-08T19:27:33.310118+0000","last_fullsized":"2023-11-08T19:27:33.310118+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:13:16.149790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.335828+0000","last_change":"2023-11-08T19:27:32.332025+0000","last_active":"2023-11-08T19:27:32.335828+0000","last_peered":"2023-11-08T19:27:32.335828+0000","last_clean":"2023-11-08T19:27:32.335828+0000","last_became_active":"2023-11-08T19:27:32.331723+0000","last_became_peered":"2023-11-08T19:27:32.331723+0000","last_unstale":"2023-11-08T19:27:32.335828+0000","last_undegraded":"2023-11-08T19:27:32.335828+0000","last_fullsized":"2023-11-08T19:27:32.335828+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:00:21.681748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.331572+0000","last_change":"2023-11-08T19:27:32.326008+0000","last_active":"2023-11-08T19:27:32.331572+0000","last_peered":"2023-11-08T19:27:32.331572+0000","last_clean":"2023-11-08T19:27:32.331572+0000","last_became_active":"2023-11-08T19:27:32.325759+0000","last_became_peered":"2023-11-08T19:27:32.325759+0000","last_unstale":"2023-11-08T19:27:32.331572+0000","last_undegraded":"2023-11-08T19:27:32.331572+0000","last_fullsized":"2023-11-08T19:27:32.331572+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:51:54.379524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-08T19:27:33.308689+0000","last_change":"2023-11-08T19:27:32.333144+0000","last_active":"2023-11-08T19:27:33.308689+0000","last_peered":"2023-11-08T19:27:33.308689+0000","last_clean":"2023-11-08T19:27:33.308689+0000","last_became_active":"2023-11-08T19:27:32.329041+0000","last_became_peered":"2023-11-08T19:27:32.329041+0000","last_unstale":"2023-11-08T19:27:33.308689+0000","last_undegraded":"2023-11-08T19:27:33.308689+0000","last_fullsized":"2023-11-08T19:27:33.308689+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:51:54.379524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.324310+0000","last_change":"2023-11-08T19:27:32.322409+0000","last_active":"2023-11-08T19:27:32.324310+0000","last_peered":"2023-11-08T19:27:32.324310+0000","last_clean":"2023-11-08T19:27:32.324310+0000","last_became_active":"2023-11-08T19:27:32.322144+0000","last_became_peered":"2023-11-08T19:27:32.322144+0000","last_unstale":"2023-11-08T19:27:32.324310+0000","last_undegraded":"2023-11-08T19:27:32.324310+0000","last_fullsized":"2023-11-08T19:27:32.324310+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:52:54.828438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.332426+0000","last_change":"2023-11-08T19:27:32.326284+0000","last_active":"2023-11-08T19:27:32.332426+0000","last_peered":"2023-11-08T19:27:32.332426+0000","last_clean":"2023-11-08T19:27:32.332426+0000","last_became_active":"2023-11-08T19:27:32.326071+0000","last_became_peered":"2023-11-08T19:27:32.326071+0000","last_unstale":"2023-11-08T19:27:32.332426+0000","last_undegraded":"2023-11-08T19:27:32.332426+0000","last_fullsized":"2023-11-08T19:27:32.332426+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:30:14.195303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-08T19:27:33.308177+0000","last_change":"2023-11-08T19:27:32.323665+0000","last_active":"2023-11-08T19:27:33.308177+0000","last_peered":"2023-11-08T19:27:33.308177+0000","last_clean":"2023-11-08T19:27:33.308177+0000","last_became_active":"2023-11-08T19:27:32.323416+0000","last_became_peered":"2023-11-08T19:27:32.323416+0000","last_unstale":"2023-11-08T19:27:33.308177+0000","last_undegraded":"2023-11-08T19:27:33.308177+0000","last_fullsized":"2023-11-08T19:27:33.308177+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:10:39.776305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-08T19:27:33.310582+0000","last_change":"2023-11-08T19:27:32.323841+0000","last_active":"2023-11-08T19:27:33.310582+0000","last_peered":"2023-11-08T19:27:33.310582+0000","last_clean":"2023-11-08T19:27:33.310582+0000","last_became_active":"2023-11-08T19:27:32.323593+0000","last_became_peered":"2023-11-08T19:27:32.323593+0000","last_unstale":"2023-11-08T19:27:33.310582+0000","last_undegraded":"2023-11-08T19:27:33.310582+0000","last_fullsized":"2023-11-08T19:27:33.310582+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:36:25.623933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.322600+0000","last_change":"2023-11-08T19:27:32.318553+0000","last_active":"2023-11-08T19:27:32.322600+0000","last_peered":"2023-11-08T19:27:32.322600+0000","last_clean":"2023-11-08T19:27:32.322600+0000","last_became_active":"2023-11-08T19:27:32.318309+0000","last_became_peered":"2023-11-08T19:27:32.318309+0000","last_unstale":"2023-11-08T19:27:32.322600+0000","last_undegraded":"2023-11-08T19:27:32.322600+0000","last_fullsized":"2023-11-08T19:27:32.322600+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:30:14.195303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.332370+0000","last_change":"2023-11-08T19:27:32.327734+0000","last_active":"2023-11-08T19:27:32.332370+0000","last_peered":"2023-11-08T19:27:32.332370+0000","last_clean":"2023-11-08T19:27:32.332370+0000","last_became_active":"2023-11-08T19:27:32.327412+0000","last_became_peered":"2023-11-08T19:27:32.327412+0000","last_unstale":"2023-11-08T19:27:32.332370+0000","last_undegraded":"2023-11-08T19:27:32.332370+0000","last_fullsized":"2023-11-08T19:27:32.332370+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:50:42.784058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-08T19:27:33.309419+0000","last_change":"2023-11-08T19:27:32.336575+0000","last_active":"2023-11-08T19:27:33.309419+0000","last_peered":"2023-11-08T19:27:33.309419+0000","last_clean":"2023-11-08T19:27:33.309419+0000","last_became_active":"2023-11-08T19:27:32.336299+0000","last_became_peered":"2023-11-08T19:27:32.336299+0000","last_unstale":"2023-11-08T19:27:33.309419+0000","last_undegraded":"2023-11-08T19:27:33.309419+0000","last_fullsized":"2023-11-08T19:27:33.309419+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:28:54.693035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.314151+0000","last_change":"2023-11-08T19:27:31.323578+0000","last_active":"2023-11-08T19:27:32.314151+0000","last_peered":"2023-11-08T19:27:32.314151+0000","last_clean":"2023-11-08T19:27:32.314151+0000","last_became_active":"2023-11-08T19:27:31.323254+0000","last_became_peered":"2023-11-08T19:27:31.323254+0000","last_unstale":"2023-11-08T19:27:32.314151+0000","last_undegraded":"2023-11-08T19:27:32.314151+0000","last_fullsized":"2023-11-08T19:27:32.314151+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_clean_scrub_stamp":"2023-11-08T19:27:30.289497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:45:45.206227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.317146+0000","last_change":"2023-11-08T19:27:08.190826+0000","last_active":"2023-11-08T19:27:32.317146+0000","last_peered":"2023-11-08T19:27:32.317146+0000","last_clean":"2023-11-08T19:27:32.317146+0000","last_became_active":"2023-11-08T19:27:08.190440+0000","last_became_peered":"2023-11-08T19:27:08.190440+0000","last_unstale":"2023-11-08T19:27:32.317146+0000","last_undegraded":"2023-11-08T19:27:32.317146+0000","last_fullsized":"2023-11-08T19:27:32.317146+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:42:48.376462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":224,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.312645+0000","last_change":"2023-11-08T19:24:45.888122+0000","last_active":"2023-11-08T19:27:32.312645+0000","last_peered":"2023-11-08T19:27:32.312645+0000","last_clean":"2023-11-08T19:27:32.312645+0000","last_became_active":"2023-11-08T19:24:45.887048+0000","last_became_peered":"2023-11-08T19:24:45.887048+0000","last_unstale":"2023-11-08T19:27:32.312645+0000","last_undegraded":"2023-11-08T19:27:32.312645+0000","last_fullsized":"2023-11-08T19:27:32.312645+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:44.575133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:44.575133+0000","last_clean_scrub_stamp":"2023-11-08T19:24:44.575133+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:19:20.202062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":51,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-08T19:27:31.313997+0000","last_change":"2023-11-08T19:26:06.542044+0000","last_active":"2023-11-08T19:27:31.313997+0000","last_peered":"2023-11-08T19:27:31.313997+0000","last_clean":"2023-11-08T19:27:31.313997+0000","last_became_active":"2023-11-08T19:26:04.603056+0000","last_became_peered":"2023-11-08T19:26:04.603056+0000","last_unstale":"2023-11-08T19:27:31.313997+0000","last_undegraded":"2023-11-08T19:27:31.313997+0000","last_fullsized":"2023-11-08T19:27:31.313997+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:41:50.771457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000517414,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":28,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-08T19:27:31.313164+0000","last_change":"2023-11-08T19:27:09.496842+0000","last_active":"2023-11-08T19:27:31.313164+0000","last_peered":"2023-11-08T19:27:31.313164+0000","last_clean":"2023-11-08T19:27:31.313164+0000","last_became_active":"2023-11-08T19:27:07.185776+0000","last_became_peered":"2023-11-08T19:27:07.185776+0000","last_unstale":"2023-11-08T19:27:31.313164+0000","last_undegraded":"2023-11-08T19:27:31.313164+0000","last_fullsized":"2023-11-08T19:27:31.313164+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:21:13.923970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031747200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.327909+0000","last_change":"2023-11-08T19:27:32.321094+0000","last_active":"2023-11-08T19:27:32.327909+0000","last_peered":"2023-11-08T19:27:32.327909+0000","last_clean":"2023-11-08T19:27:32.327909+0000","last_became_active":"2023-11-08T19:27:32.320655+0000","last_became_peered":"2023-11-08T19:27:32.320655+0000","last_unstale":"2023-11-08T19:27:32.327909+0000","last_undegraded":"2023-11-08T19:27:32.327909+0000","last_fullsized":"2023-11-08T19:27:32.327909+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:41:33.954786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.308411+0000","last_change":"2023-11-08T19:27:31.322203+0000","last_active":"2023-11-08T19:27:32.308411+0000","last_peered":"2023-11-08T19:27:32.308411+0000","last_clean":"2023-11-08T19:27:32.308411+0000","last_became_active":"2023-11-08T19:27:31.321851+0000","last_became_peered":"2023-11-08T19:27:31.321851+0000","last_unstale":"2023-11-08T19:27:32.308411+0000","last_undegraded":"2023-11-08T19:27:32.308411+0000","last_fullsized":"2023-11-08T19:27:32.308411+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_clean_scrub_stamp":"2023-11-08T19:27:30.289497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:18:24.093678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":21,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-08T19:27:31.313099+0000","last_change":"2023-11-08T19:27:08.188241+0000","last_active":"2023-11-08T19:27:31.313099+0000","last_peered":"2023-11-08T19:27:31.313099+0000","last_clean":"2023-11-08T19:27:31.313099+0000","last_became_active":"2023-11-08T19:27:08.187863+0000","last_became_peered":"2023-11-08T19:27:08.187863+0000","last_unstale":"2023-11-08T19:27:31.313099+0000","last_undegraded":"2023-11-08T19:27:31.313099+0000","last_fullsized":"2023-11-08T19:27:31.313099+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:45:22.084015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":50,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.311772+0000","last_change":"2023-11-08T19:26:06.265419+0000","last_active":"2023-11-08T19:27:32.311772+0000","last_peered":"2023-11-08T19:27:32.311772+0000","last_clean":"2023-11-08T19:27:32.311772+0000","last_became_active":"2023-11-08T19:26:04.605333+0000","last_became_peered":"2023-11-08T19:26:04.605333+0000","last_unstale":"2023-11-08T19:27:32.311772+0000","last_undegraded":"2023-11-08T19:27:32.311772+0000","last_fullsized":"2023-11-08T19:27:32.311772+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:01:56.991346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064606400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.319313+0000","last_change":"2023-11-08T19:27:09.182113+0000","last_active":"2023-11-08T19:27:32.319313+0000","last_peered":"2023-11-08T19:27:32.319313+0000","last_clean":"2023-11-08T19:27:32.319313+0000","last_became_active":"2023-11-08T19:27:07.189939+0000","last_became_peered":"2023-11-08T19:27:07.189939+0000","last_unstale":"2023-11-08T19:27:32.319313+0000","last_undegraded":"2023-11-08T19:27:32.319313+0000","last_fullsized":"2023-11-08T19:27:32.319313+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:41:08.822366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036507500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.324310+0000","last_change":"2023-11-08T19:27:32.320256+0000","last_active":"2023-11-08T19:27:32.324310+0000","last_peered":"2023-11-08T19:27:32.324310+0000","last_clean":"2023-11-08T19:27:32.324310+0000","last_became_active":"2023-11-08T19:27:32.319830+0000","last_became_peered":"2023-11-08T19:27:32.319830+0000","last_unstale":"2023-11-08T19:27:32.324310+0000","last_undegraded":"2023-11-08T19:27:32.324310+0000","last_fullsized":"2023-11-08T19:27:32.324310+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:00:21.681748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.310481+0000","last_change":"2023-11-08T19:27:31.314348+0000","last_active":"2023-11-08T19:27:32.310481+0000","last_peered":"2023-11-08T19:27:32.310481+0000","last_clean":"2023-11-08T19:27:32.310481+0000","last_became_active":"2023-11-08T19:27:31.314099+0000","last_became_peered":"2023-11-08T19:27:31.314099+0000","last_unstale":"2023-11-08T19:27:32.310481+0000","last_undegraded":"2023-11-08T19:27:32.310481+0000","last_fullsized":"2023-11-08T19:27:32.310481+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_clean_scrub_stamp":"2023-11-08T19:27:30.289497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:37:47.141610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":19,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-08T19:27:31.303468+0000","last_change":"2023-11-08T19:27:08.187127+0000","last_active":"2023-11-08T19:27:31.303468+0000","last_peered":"2023-11-08T19:27:31.303468+0000","last_clean":"2023-11-08T19:27:31.303468+0000","last_became_active":"2023-11-08T19:27:08.182648+0000","last_became_peered":"2023-11-08T19:27:08.182648+0000","last_unstale":"2023-11-08T19:27:31.303468+0000","last_undegraded":"2023-11-08T19:27:31.303468+0000","last_fullsized":"2023-11-08T19:27:31.303468+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:50:12.101878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":58,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.316216+0000","last_change":"2023-11-08T19:26:05.609570+0000","last_active":"2023-11-08T19:27:32.316216+0000","last_peered":"2023-11-08T19:27:32.316216+0000","last_clean":"2023-11-08T19:27:32.316216+0000","last_became_active":"2023-11-08T19:26:04.608379+0000","last_became_peered":"2023-11-08T19:26:04.608379+0000","last_unstale":"2023-11-08T19:27:32.316216+0000","last_undegraded":"2023-11-08T19:27:32.316216+0000","last_fullsized":"2023-11-08T19:27:32.316216+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:40:08.986649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000529657,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.312068+0000","last_change":"2023-11-08T19:27:09.211991+0000","last_active":"2023-11-08T19:27:32.312068+0000","last_peered":"2023-11-08T19:27:32.312068+0000","last_clean":"2023-11-08T19:27:32.312068+0000","last_became_active":"2023-11-08T19:27:07.195231+0000","last_became_peered":"2023-11-08T19:27:07.195231+0000","last_unstale":"2023-11-08T19:27:32.312068+0000","last_undegraded":"2023-11-08T19:27:32.312068+0000","last_fullsized":"2023-11-08T19:27:32.312068+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:25:14.487524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000441221,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.326577+0000","last_change":"2023-11-08T19:27:32.317869+0000","last_active":"2023-11-08T19:27:32.326577+0000","last_peered":"2023-11-08T19:27:32.326577+0000","last_clean":"2023-11-08T19:27:32.326577+0000","last_became_active":"2023-11-08T19:27:32.317588+0000","last_became_peered":"2023-11-08T19:27:32.317588+0000","last_unstale":"2023-11-08T19:27:32.326577+0000","last_undegraded":"2023-11-08T19:27:32.326577+0000","last_fullsized":"2023-11-08T19:27:32.326577+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:28:17.551060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"77'2","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.333389+0000","last_change":"2023-11-08T19:27:31.327747+0000","last_active":"2023-11-08T19:27:32.333389+0000","last_peered":"2023-11-08T19:27:32.333389+0000","last_clean":"2023-11-08T19:27:32.333389+0000","last_became_active":"2023-11-08T19:27:31.327536+0000","last_became_peered":"2023-11-08T19:27:31.327536+0000","last_unstale":"2023-11-08T19:27:32.333389+0000","last_undegraded":"2023-11-08T19:27:32.333389+0000","last_fullsized":"2023-11-08T19:27:32.333389+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_clean_scrub_stamp":"2023-11-08T19:27:30.289497+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:28:21.515340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":19,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-08T19:27:31.304153+0000","last_change":"2023-11-08T19:27:08.195364+0000","last_active":"2023-11-08T19:27:31.304153+0000","last_peered":"2023-11-08T19:27:31.304153+0000","last_clean":"2023-11-08T19:27:31.304153+0000","last_became_active":"2023-11-08T19:27:08.195074+0000","last_became_peered":"2023-11-08T19:27:08.195074+0000","last_unstale":"2023-11-08T19:27:31.304153+0000","last_undegraded":"2023-11-08T19:27:31.304153+0000","last_fullsized":"2023-11-08T19:27:31.304153+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:02:57.552873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":49,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-08T19:27:31.326311+0000","last_change":"2023-11-08T19:26:06.394300+0000","last_active":"2023-11-08T19:27:31.326311+0000","last_peered":"2023-11-08T19:27:31.326311+0000","last_clean":"2023-11-08T19:27:31.326311+0000","last_became_active":"2023-11-08T19:26:04.601685+0000","last_became_peered":"2023-11-08T19:26:04.601685+0000","last_unstale":"2023-11-08T19:27:31.326311+0000","last_undegraded":"2023-11-08T19:27:31.326311+0000","last_fullsized":"2023-11-08T19:27:31.326311+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:51:06.626474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061044099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-08T19:27:31.304181+0000","last_change":"2023-11-08T19:27:09.178466+0000","last_active":"2023-11-08T19:27:31.304181+0000","last_peered":"2023-11-08T19:27:31.304181+0000","last_clean":"2023-11-08T19:27:31.304181+0000","last_became_active":"2023-11-08T19:27:07.189386+0000","last_became_peered":"2023-11-08T19:27:07.189386+0000","last_unstale":"2023-11-08T19:27:31.304181+0000","last_undegraded":"2023-11-08T19:27:31.304181+0000","last_fullsized":"2023-11-08T19:27:31.304181+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:48:51.139820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049390100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.337392+0000","last_change":"2023-11-08T19:27:32.333807+0000","last_active":"2023-11-08T19:27:32.337392+0000","last_peered":"2023-11-08T19:27:32.337392+0000","last_clean":"2023-11-08T19:27:32.337392+0000","last_became_active":"2023-11-08T19:27:32.333527+0000","last_became_peered":"2023-11-08T19:27:32.333527+0000","last_unstale":"2023-11-08T19:27:32.337392+0000","last_undegraded":"2023-11-08T19:27:32.337392+0000","last_fullsized":"2023-11-08T19:27:32.337392+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:43:12.503877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.310400+0000","last_change":"2023-11-08T19:27:31.316250+0000","last_active":"2023-11-08T19:27:32.310400+0000","last_peered":"2023-11-08T19:27:32.310400+0000","last_clean":"2023-11-08T19:27:32.310400+0000","last_became_active":"2023-11-08T19:27:31.315970+0000","last_became_peered":"2023-11-08T19:27:31.315970+0000","last_unstale":"2023-11-08T19:27:32.310400+0000","last_undegraded":"2023-11-08T19:27:32.310400+0000","last_fullsized":"2023-11-08T19:27:32.310400+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_clean_scrub_stamp":"2023-11-08T19:27:30.289497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:57:50.429595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":21,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-08T19:27:31.311591+0000","last_change":"2023-11-08T19:27:08.189452+0000","last_active":"2023-11-08T19:27:31.311591+0000","last_peered":"2023-11-08T19:27:31.311591+0000","last_clean":"2023-11-08T19:27:31.311591+0000","last_became_active":"2023-11-08T19:27:08.189088+0000","last_became_peered":"2023-11-08T19:27:08.189088+0000","last_unstale":"2023-11-08T19:27:31.311591+0000","last_undegraded":"2023-11-08T19:27:31.311591+0000","last_fullsized":"2023-11-08T19:27:31.311591+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:25:18.963220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":50,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-08T19:27:31.307657+0000","last_change":"2023-11-08T19:26:06.296634+0000","last_active":"2023-11-08T19:27:31.307657+0000","last_peered":"2023-11-08T19:27:31.307657+0000","last_clean":"2023-11-08T19:27:31.307657+0000","last_became_active":"2023-11-08T19:26:04.603088+0000","last_became_peered":"2023-11-08T19:26:04.603088+0000","last_unstale":"2023-11-08T19:27:31.307657+0000","last_undegraded":"2023-11-08T19:27:31.307657+0000","last_fullsized":"2023-11-08T19:27:31.307657+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:01:56.991346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038180899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-08T19:27:31.309270+0000","last_change":"2023-11-08T19:27:09.431220+0000","last_active":"2023-11-08T19:27:31.309270+0000","last_peered":"2023-11-08T19:27:31.309270+0000","last_clean":"2023-11-08T19:27:31.309270+0000","last_became_active":"2023-11-08T19:27:07.191589+0000","last_became_peered":"2023-11-08T19:27:07.191589+0000","last_unstale":"2023-11-08T19:27:31.309270+0000","last_undegraded":"2023-11-08T19:27:31.309270+0000","last_fullsized":"2023-11-08T19:27:31.309270+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:18:19.964487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000729527,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-08T19:27:33.310067+0000","last_change":"2023-11-08T19:27:32.318855+0000","last_active":"2023-11-08T19:27:33.310067+0000","last_peered":"2023-11-08T19:27:33.310067+0000","last_clean":"2023-11-08T19:27:33.310067+0000","last_became_active":"2023-11-08T19:27:32.318553+0000","last_became_peered":"2023-11-08T19:27:32.318553+0000","last_unstale":"2023-11-08T19:27:33.310067+0000","last_undegraded":"2023-11-08T19:27:33.310067+0000","last_fullsized":"2023-11-08T19:27:33.310067+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:37:00.013636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"77'1","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-08T19:27:33.310659+0000","last_change":"2023-11-08T19:27:31.322280+0000","last_active":"2023-11-08T19:27:33.310659+0000","last_peered":"2023-11-08T19:27:33.310659+0000","last_clean":"2023-11-08T19:27:33.310659+0000","last_became_active":"2023-11-08T19:27:31.321982+0000","last_became_peered":"2023-11-08T19:27:31.321982+0000","last_unstale":"2023-11-08T19:27:33.310659+0000","last_undegraded":"2023-11-08T19:27:33.310659+0000","last_fullsized":"2023-11-08T19:27:33.310659+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_clean_scrub_stamp":"2023-11-08T19:27:30.289497+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:18:24.093678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-08T19:27:31.326935+0000","last_change":"2023-11-08T19:27:08.193337+0000","last_active":"2023-11-08T19:27:31.326935+0000","last_peered":"2023-11-08T19:27:31.326935+0000","last_clean":"2023-11-08T19:27:31.326935+0000","last_became_active":"2023-11-08T19:27:08.192997+0000","last_became_peered":"2023-11-08T19:27:08.192997+0000","last_unstale":"2023-11-08T19:27:31.326935+0000","last_undegraded":"2023-11-08T19:27:31.326935+0000","last_fullsized":"2023-11-08T19:27:31.326935+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:45:22.084015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":52,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-08T19:27:31.313803+0000","last_change":"2023-11-08T19:26:06.424511+0000","last_active":"2023-11-08T19:27:31.313803+0000","last_peered":"2023-11-08T19:27:31.313803+0000","last_clean":"2023-11-08T19:27:31.313803+0000","last_became_active":"2023-11-08T19:26:04.598973+0000","last_became_peered":"2023-11-08T19:26:04.598973+0000","last_unstale":"2023-11-08T19:27:31.313803+0000","last_undegraded":"2023-11-08T19:27:31.313803+0000","last_fullsized":"2023-11-08T19:27:31.313803+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:41:53.497456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031755900000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":28,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-08T19:27:31.311679+0000","last_change":"2023-11-08T19:27:09.497024+0000","last_active":"2023-11-08T19:27:31.311679+0000","last_peered":"2023-11-08T19:27:31.311679+0000","last_clean":"2023-11-08T19:27:31.311679+0000","last_became_active":"2023-11-08T19:27:07.190132+0000","last_became_peered":"2023-11-08T19:27:07.190132+0000","last_unstale":"2023-11-08T19:27:31.311679+0000","last_undegraded":"2023-11-08T19:27:31.311679+0000","last_fullsized":"2023-11-08T19:27:31.311679+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:36:34.881217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054126000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.324544+0000","last_change":"2023-11-08T19:27:32.320953+0000","last_active":"2023-11-08T19:27:32.324544+0000","last_peered":"2023-11-08T19:27:32.324544+0000","last_clean":"2023-11-08T19:27:32.324544+0000","last_became_active":"2023-11-08T19:27:32.320660+0000","last_became_peered":"2023-11-08T19:27:32.320660+0000","last_unstale":"2023-11-08T19:27:32.324544+0000","last_undegraded":"2023-11-08T19:27:32.324544+0000","last_fullsized":"2023-11-08T19:27:32.324544+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:54:44.277177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"77'1","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-08T19:27:33.309414+0000","last_change":"2023-11-08T19:27:31.314416+0000","last_active":"2023-11-08T19:27:33.309414+0000","last_peered":"2023-11-08T19:27:33.309414+0000","last_clean":"2023-11-08T19:27:33.309414+0000","last_became_active":"2023-11-08T19:27:31.314068+0000","last_became_peered":"2023-11-08T19:27:31.314068+0000","last_unstale":"2023-11-08T19:27:33.309414+0000","last_undegraded":"2023-11-08T19:27:33.309414+0000","last_fullsized":"2023-11-08T19:27:33.309414+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_clean_scrub_stamp":"2023-11-08T19:27:30.289497+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:21:02.207345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-08T19:27:31.326274+0000","last_change":"2023-11-08T19:27:08.194196+0000","last_active":"2023-11-08T19:27:31.326274+0000","last_peered":"2023-11-08T19:27:31.326274+0000","last_clean":"2023-11-08T19:27:31.326274+0000","last_became_active":"2023-11-08T19:27:08.193912+0000","last_became_peered":"2023-11-08T19:27:08.193912+0000","last_unstale":"2023-11-08T19:27:31.326274+0000","last_undegraded":"2023-11-08T19:27:31.326274+0000","last_fullsized":"2023-11-08T19:27:31.326274+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:25:18.963220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":50,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-08T19:27:31.309352+0000","last_change":"2023-11-08T19:26:06.296937+0000","last_active":"2023-11-08T19:27:31.309352+0000","last_peered":"2023-11-08T19:27:31.309352+0000","last_clean":"2023-11-08T19:27:31.309352+0000","last_became_active":"2023-11-08T19:26:04.599999+0000","last_became_peered":"2023-11-08T19:26:04.599999+0000","last_unstale":"2023-11-08T19:27:31.309352+0000","last_undegraded":"2023-11-08T19:27:31.309352+0000","last_fullsized":"2023-11-08T19:27:31.309352+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:50:43.827570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025605700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":28,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-08T19:27:31.313345+0000","last_change":"2023-11-08T19:27:09.497040+0000","last_active":"2023-11-08T19:27:31.313345+0000","last_peered":"2023-11-08T19:27:31.313345+0000","last_clean":"2023-11-08T19:27:31.313345+0000","last_became_active":"2023-11-08T19:27:07.189135+0000","last_became_peered":"2023-11-08T19:27:07.189135+0000","last_unstale":"2023-11-08T19:27:31.313345+0000","last_undegraded":"2023-11-08T19:27:31.313345+0000","last_fullsized":"2023-11-08T19:27:31.313345+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:12:29.400972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051794299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-08T19:27:33.309188+0000","last_change":"2023-11-08T19:27:32.326129+0000","last_active":"2023-11-08T19:27:33.309188+0000","last_peered":"2023-11-08T19:27:33.309188+0000","last_clean":"2023-11-08T19:27:33.309188+0000","last_became_active":"2023-11-08T19:27:32.325853+0000","last_became_peered":"2023-11-08T19:27:32.325853+0000","last_unstale":"2023-11-08T19:27:33.309188+0000","last_undegraded":"2023-11-08T19:27:33.309188+0000","last_fullsized":"2023-11-08T19:27:33.309188+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:41:33.954786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"77'2","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-08T19:27:33.308328+0000","last_change":"2023-11-08T19:27:31.315843+0000","last_active":"2023-11-08T19:27:33.308328+0000","last_peered":"2023-11-08T19:27:33.308328+0000","last_clean":"2023-11-08T19:27:33.308328+0000","last_became_active":"2023-11-08T19:27:31.315545+0000","last_became_peered":"2023-11-08T19:27:31.315545+0000","last_unstale":"2023-11-08T19:27:33.308328+0000","last_undegraded":"2023-11-08T19:27:33.308328+0000","last_fullsized":"2023-11-08T19:27:33.308328+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_clean_scrub_stamp":"2023-11-08T19:27:30.289497+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:00:20.676544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.315365+0000","last_change":"2023-11-08T19:27:08.187053+0000","last_active":"2023-11-08T19:27:32.315365+0000","last_peered":"2023-11-08T19:27:32.315365+0000","last_clean":"2023-11-08T19:27:32.315365+0000","last_became_active":"2023-11-08T19:27:08.186643+0000","last_became_peered":"2023-11-08T19:27:08.186643+0000","last_unstale":"2023-11-08T19:27:32.315365+0000","last_undegraded":"2023-11-08T19:27:32.315365+0000","last_fullsized":"2023-11-08T19:27:32.315365+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:35:30.073885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":58,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.317345+0000","last_change":"2023-11-08T19:26:05.609824+0000","last_active":"2023-11-08T19:27:32.317345+0000","last_peered":"2023-11-08T19:27:32.317345+0000","last_clean":"2023-11-08T19:27:32.317345+0000","last_became_active":"2023-11-08T19:26:04.603320+0000","last_became_peered":"2023-11-08T19:26:04.603320+0000","last_unstale":"2023-11-08T19:27:32.317345+0000","last_undegraded":"2023-11-08T19:27:32.317345+0000","last_fullsized":"2023-11-08T19:27:32.317345+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:05:45.976615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063225,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":28,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-08T19:27:31.313234+0000","last_change":"2023-11-08T19:27:09.496655+0000","last_active":"2023-11-08T19:27:31.313234+0000","last_peered":"2023-11-08T19:27:31.313234+0000","last_clean":"2023-11-08T19:27:31.313234+0000","last_became_active":"2023-11-08T19:27:07.189895+0000","last_became_peered":"2023-11-08T19:27:07.189895+0000","last_unstale":"2023-11-08T19:27:31.313234+0000","last_undegraded":"2023-11-08T19:27:31.313234+0000","last_fullsized":"2023-11-08T19:27:31.313234+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:41:08.822366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045654199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-08T19:27:33.308011+0000","last_change":"2023-11-08T19:27:32.325048+0000","last_active":"2023-11-08T19:27:33.308011+0000","last_peered":"2023-11-08T19:27:33.308011+0000","last_clean":"2023-11-08T19:27:33.308011+0000","last_became_active":"2023-11-08T19:27:32.324780+0000","last_became_peered":"2023-11-08T19:27:32.324780+0000","last_unstale":"2023-11-08T19:27:33.308011+0000","last_undegraded":"2023-11-08T19:27:33.308011+0000","last_fullsized":"2023-11-08T19:27:33.308011+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:51:54.379524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-08T19:27:33.309903+0000","last_change":"2023-11-08T19:27:31.317717+0000","last_active":"2023-11-08T19:27:33.309903+0000","last_peered":"2023-11-08T19:27:33.309903+0000","last_clean":"2023-11-08T19:27:33.309903+0000","last_became_active":"2023-11-08T19:27:31.317433+0000","last_became_peered":"2023-11-08T19:27:31.317433+0000","last_unstale":"2023-11-08T19:27:33.309903+0000","last_undegraded":"2023-11-08T19:27:33.309903+0000","last_fullsized":"2023-11-08T19:27:33.309903+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_clean_scrub_stamp":"2023-11-08T19:27:30.289497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:43:11.498673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":51,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-08T19:27:31.312208+0000","last_change":"2023-11-08T19:26:06.541360+0000","last_active":"2023-11-08T19:27:31.312208+0000","last_peered":"2023-11-08T19:27:31.312208+0000","last_clean":"2023-11-08T19:27:31.312208+0000","last_became_active":"2023-11-08T19:26:04.603725+0000","last_became_peered":"2023-11-08T19:26:04.603725+0000","last_unstale":"2023-11-08T19:27:31.312208+0000","last_undegraded":"2023-11-08T19:27:31.312208+0000","last_fullsized":"2023-11-08T19:27:31.312208+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:28:48.576742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053365400000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-08T19:27:31.314076+0000","last_change":"2023-11-08T19:27:09.283599+0000","last_active":"2023-11-08T19:27:31.314076+0000","last_peered":"2023-11-08T19:27:31.314076+0000","last_clean":"2023-11-08T19:27:31.314076+0000","last_became_active":"2023-11-08T19:27:07.189595+0000","last_became_peered":"2023-11-08T19:27:07.189595+0000","last_unstale":"2023-11-08T19:27:31.314076+0000","last_undegraded":"2023-11-08T19:27:31.314076+0000","last_fullsized":"2023-11-08T19:27:31.314076+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:48:20.082682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000323479,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":21,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.309454+0000","last_change":"2023-11-08T19:27:08.185395+0000","last_active":"2023-11-08T19:27:32.309454+0000","last_peered":"2023-11-08T19:27:32.309454+0000","last_clean":"2023-11-08T19:27:32.309454+0000","last_became_active":"2023-11-08T19:27:08.184765+0000","last_became_peered":"2023-11-08T19:27:08.184765+0000","last_unstale":"2023-11-08T19:27:32.309454+0000","last_undegraded":"2023-11-08T19:27:32.309454+0000","last_fullsized":"2023-11-08T19:27:32.309454+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:08:51.864418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.332314+0000","last_change":"2023-11-08T19:27:32.327550+0000","last_active":"2023-11-08T19:27:32.332314+0000","last_peered":"2023-11-08T19:27:32.332314+0000","last_clean":"2023-11-08T19:27:32.332314+0000","last_became_active":"2023-11-08T19:27:32.327314+0000","last_became_peered":"2023-11-08T19:27:32.327314+0000","last_unstale":"2023-11-08T19:27:32.332314+0000","last_undegraded":"2023-11-08T19:27:32.332314+0000","last_fullsized":"2023-11-08T19:27:32.332314+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:05:01.010467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-08T19:27:33.307949+0000","last_change":"2023-11-08T19:27:31.320643+0000","last_active":"2023-11-08T19:27:33.307949+0000","last_peered":"2023-11-08T19:27:33.307949+0000","last_clean":"2023-11-08T19:27:33.307949+0000","last_became_active":"2023-11-08T19:27:31.320287+0000","last_became_peered":"2023-11-08T19:27:31.320287+0000","last_unstale":"2023-11-08T19:27:33.307949+0000","last_undegraded":"2023-11-08T19:27:33.307949+0000","last_fullsized":"2023-11-08T19:27:33.307949+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_clean_scrub_stamp":"2023-11-08T19:27:30.289497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:50:41.778854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":52,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-08T19:27:31.311879+0000","last_change":"2023-11-08T19:26:06.424236+0000","last_active":"2023-11-08T19:27:31.311879+0000","last_peered":"2023-11-08T19:27:31.311879+0000","last_clean":"2023-11-08T19:27:31.311879+0000","last_became_active":"2023-11-08T19:26:04.600115+0000","last_became_peered":"2023-11-08T19:26:04.600115+0000","last_unstale":"2023-11-08T19:27:31.311879+0000","last_undegraded":"2023-11-08T19:27:31.311879+0000","last_fullsized":"2023-11-08T19:27:31.311879+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:18:01.876605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065803900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-08T19:27:31.326139+0000","last_change":"2023-11-08T19:27:09.331899+0000","last_active":"2023-11-08T19:27:31.326139+0000","last_peered":"2023-11-08T19:27:31.326139+0000","last_clean":"2023-11-08T19:27:31.326139+0000","last_became_active":"2023-11-08T19:27:07.191460+0000","last_became_peered":"2023-11-08T19:27:07.191460+0000","last_unstale":"2023-11-08T19:27:31.326139+0000","last_undegraded":"2023-11-08T19:27:31.326139+0000","last_fullsized":"2023-11-08T19:27:31.326139+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:51:29.247104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044907500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.318270+0000","last_change":"2023-11-08T19:27:08.191802+0000","last_active":"2023-11-08T19:27:32.318270+0000","last_peered":"2023-11-08T19:27:32.318270+0000","last_clean":"2023-11-08T19:27:32.318270+0000","last_became_active":"2023-11-08T19:27:08.191520+0000","last_became_peered":"2023-11-08T19:27:08.191520+0000","last_unstale":"2023-11-08T19:27:32.318270+0000","last_undegraded":"2023-11-08T19:27:32.318270+0000","last_fullsized":"2023-11-08T19:27:32.318270+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:49:16.458634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.323523+0000","last_change":"2023-11-08T19:27:32.319621+0000","last_active":"2023-11-08T19:27:32.323523+0000","last_peered":"2023-11-08T19:27:32.323523+0000","last_clean":"2023-11-08T19:27:32.323523+0000","last_became_active":"2023-11-08T19:27:32.319371+0000","last_became_peered":"2023-11-08T19:27:32.319371+0000","last_unstale":"2023-11-08T19:27:32.323523+0000","last_undegraded":"2023-11-08T19:27:32.323523+0000","last_fullsized":"2023-11-08T19:27:32.323523+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:43:54.922810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.312114+0000","last_change":"2023-11-08T19:27:31.326195+0000","last_active":"2023-11-08T19:27:32.312114+0000","last_peered":"2023-11-08T19:27:32.312114+0000","last_clean":"2023-11-08T19:27:32.312114+0000","last_became_active":"2023-11-08T19:27:31.325718+0000","last_became_peered":"2023-11-08T19:27:31.325718+0000","last_unstale":"2023-11-08T19:27:32.312114+0000","last_undegraded":"2023-11-08T19:27:32.312114+0000","last_fullsized":"2023-11-08T19:27:32.312114+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_clean_scrub_stamp":"2023-11-08T19:27:30.289497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:51:14.480725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":58,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.312534+0000","last_change":"2023-11-08T19:26:05.608892+0000","last_active":"2023-11-08T19:27:32.312534+0000","last_peered":"2023-11-08T19:27:32.312534+0000","last_clean":"2023-11-08T19:27:32.312534+0000","last_became_active":"2023-11-08T19:26:04.604519+0000","last_became_peered":"2023-11-08T19:26:04.604519+0000","last_unstale":"2023-11-08T19:27:32.312534+0000","last_undegraded":"2023-11-08T19:27:32.312534+0000","last_fullsized":"2023-11-08T19:27:32.312534+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:11:26.508634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051248099999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.317259+0000","last_change":"2023-11-08T19:27:09.181431+0000","last_active":"2023-11-08T19:27:32.317259+0000","last_peered":"2023-11-08T19:27:32.317259+0000","last_clean":"2023-11-08T19:27:32.317259+0000","last_became_active":"2023-11-08T19:27:07.190800+0000","last_became_peered":"2023-11-08T19:27:07.190800+0000","last_unstale":"2023-11-08T19:27:32.317259+0000","last_undegraded":"2023-11-08T19:27:32.317259+0000","last_fullsized":"2023-11-08T19:27:32.317259+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:46:30.452678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656099999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":21,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.312177+0000","last_change":"2023-11-08T19:27:08.187983+0000","last_active":"2023-11-08T19:27:32.312177+0000","last_peered":"2023-11-08T19:27:32.312177+0000","last_clean":"2023-11-08T19:27:32.312177+0000","last_became_active":"2023-11-08T19:27:08.187322+0000","last_became_peered":"2023-11-08T19:27:08.187322+0000","last_unstale":"2023-11-08T19:27:32.312177+0000","last_undegraded":"2023-11-08T19:27:32.312177+0000","last_fullsized":"2023-11-08T19:27:32.312177+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:33:07.647519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.336174+0000","last_change":"2023-11-08T19:27:32.332731+0000","last_active":"2023-11-08T19:27:32.336174+0000","last_peered":"2023-11-08T19:27:32.336174+0000","last_clean":"2023-11-08T19:27:32.336174+0000","last_became_active":"2023-11-08T19:27:32.332354+0000","last_became_peered":"2023-11-08T19:27:32.332354+0000","last_unstale":"2023-11-08T19:27:32.336174+0000","last_undegraded":"2023-11-08T19:27:32.336174+0000","last_fullsized":"2023-11-08T19:27:32.336174+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:15:16.572033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.311295+0000","last_change":"2023-11-08T19:27:31.327299+0000","last_active":"2023-11-08T19:27:32.311295+0000","last_peered":"2023-11-08T19:27:32.311295+0000","last_clean":"2023-11-08T19:27:32.311295+0000","last_became_active":"2023-11-08T19:27:31.326997+0000","last_became_peered":"2023-11-08T19:27:31.326997+0000","last_unstale":"2023-11-08T19:27:32.311295+0000","last_undegraded":"2023-11-08T19:27:32.311295+0000","last_fullsized":"2023-11-08T19:27:32.311295+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_clean_scrub_stamp":"2023-11-08T19:27:30.289497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:25:42.085432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":49,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-08T19:27:31.326286+0000","last_change":"2023-11-08T19:26:06.394075+0000","last_active":"2023-11-08T19:27:31.326286+0000","last_peered":"2023-11-08T19:27:31.326286+0000","last_clean":"2023-11-08T19:27:31.326286+0000","last_became_active":"2023-11-08T19:26:04.606967+0000","last_became_peered":"2023-11-08T19:26:04.606967+0000","last_unstale":"2023-11-08T19:27:31.326286+0000","last_undegraded":"2023-11-08T19:27:31.326286+0000","last_fullsized":"2023-11-08T19:27:31.326286+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:19:56.286835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040971999999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.316668+0000","last_change":"2023-11-08T19:27:09.180722+0000","last_active":"2023-11-08T19:27:32.316668+0000","last_peered":"2023-11-08T19:27:32.316668+0000","last_clean":"2023-11-08T19:27:32.316668+0000","last_became_active":"2023-11-08T19:27:07.186977+0000","last_became_peered":"2023-11-08T19:27:07.186977+0000","last_unstale":"2023-11-08T19:27:32.316668+0000","last_undegraded":"2023-11-08T19:27:32.316668+0000","last_fullsized":"2023-11-08T19:27:32.316668+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:17:59.966463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057675100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-08T19:27:31.326404+0000","last_change":"2023-11-08T19:27:08.193963+0000","last_active":"2023-11-08T19:27:31.326404+0000","last_peered":"2023-11-08T19:27:31.326404+0000","last_clean":"2023-11-08T19:27:31.326404+0000","last_became_active":"2023-11-08T19:27:08.193098+0000","last_became_peered":"2023-11-08T19:27:08.193098+0000","last_unstale":"2023-11-08T19:27:31.326404+0000","last_undegraded":"2023-11-08T19:27:31.326404+0000","last_fullsized":"2023-11-08T19:27:31.326404+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:41:09.827370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.324574+0000","last_change":"2023-11-08T19:27:32.322909+0000","last_active":"2023-11-08T19:27:32.324574+0000","last_peered":"2023-11-08T19:27:32.324574+0000","last_clean":"2023-11-08T19:27:32.324574+0000","last_became_active":"2023-11-08T19:27:32.322643+0000","last_became_peered":"2023-11-08T19:27:32.322643+0000","last_unstale":"2023-11-08T19:27:32.324574+0000","last_undegraded":"2023-11-08T19:27:32.324574+0000","last_fullsized":"2023-11-08T19:27:32.324574+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:08:48.489860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-08T19:27:33.310535+0000","last_change":"2023-11-08T19:27:31.331499+0000","last_active":"2023-11-08T19:27:33.310535+0000","last_peered":"2023-11-08T19:27:33.310535+0000","last_clean":"2023-11-08T19:27:33.310535+0000","last_became_active":"2023-11-08T19:27:31.331164+0000","last_became_peered":"2023-11-08T19:27:31.331164+0000","last_unstale":"2023-11-08T19:27:33.310535+0000","last_undegraded":"2023-11-08T19:27:33.310535+0000","last_fullsized":"2023-11-08T19:27:33.310535+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_clean_scrub_stamp":"2023-11-08T19:27:30.289497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:30:13.190099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":50,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.312402+0000","last_change":"2023-11-08T19:26:06.265686+0000","last_active":"2023-11-08T19:27:32.312402+0000","last_peered":"2023-11-08T19:27:32.312402+0000","last_clean":"2023-11-08T19:27:32.312402+0000","last_became_active":"2023-11-08T19:26:04.602111+0000","last_became_peered":"2023-11-08T19:26:04.602111+0000","last_unstale":"2023-11-08T19:27:32.312402+0000","last_undegraded":"2023-11-08T19:27:32.312402+0000","last_fullsized":"2023-11-08T19:27:32.312402+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:40:11.164951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056197899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.318242+0000","last_change":"2023-11-08T19:27:09.181333+0000","last_active":"2023-11-08T19:27:32.318242+0000","last_peered":"2023-11-08T19:27:32.318242+0000","last_clean":"2023-11-08T19:27:32.318242+0000","last_became_active":"2023-11-08T19:27:07.188657+0000","last_became_peered":"2023-11-08T19:27:07.188657+0000","last_unstale":"2023-11-08T19:27:32.318242+0000","last_undegraded":"2023-11-08T19:27:32.318242+0000","last_fullsized":"2023-11-08T19:27:32.318242+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:25:56.788615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001227393,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.317396+0000","last_change":"2023-11-08T19:27:08.188533+0000","last_active":"2023-11-08T19:27:32.317396+0000","last_peered":"2023-11-08T19:27:32.317396+0000","last_clean":"2023-11-08T19:27:32.317396+0000","last_became_active":"2023-11-08T19:27:08.188240+0000","last_became_peered":"2023-11-08T19:27:08.188240+0000","last_unstale":"2023-11-08T19:27:32.317396+0000","last_undegraded":"2023-11-08T19:27:32.317396+0000","last_fullsized":"2023-11-08T19:27:32.317396+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:55:37.653521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.327548+0000","last_change":"2023-11-08T19:27:32.320936+0000","last_active":"2023-11-08T19:27:32.327548+0000","last_peered":"2023-11-08T19:27:32.327548+0000","last_clean":"2023-11-08T19:27:32.327548+0000","last_became_active":"2023-11-08T19:27:32.320641+0000","last_became_peered":"2023-11-08T19:27:32.320641+0000","last_unstale":"2023-11-08T19:27:32.327548+0000","last_undegraded":"2023-11-08T19:27:32.327548+0000","last_fullsized":"2023-11-08T19:27:32.327548+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:21:03.212549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-08T19:27:33.308956+0000","last_change":"2023-11-08T19:27:31.317825+0000","last_active":"2023-11-08T19:27:33.308956+0000","last_peered":"2023-11-08T19:27:33.308956+0000","last_clean":"2023-11-08T19:27:33.308956+0000","last_became_active":"2023-11-08T19:27:31.317552+0000","last_became_peered":"2023-11-08T19:27:31.317552+0000","last_unstale":"2023-11-08T19:27:33.308956+0000","last_undegraded":"2023-11-08T19:27:33.308956+0000","last_fullsized":"2023-11-08T19:27:33.308956+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_clean_scrub_stamp":"2023-11-08T19:27:30.289497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:46:54.579893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.314263+0000","last_change":"2023-11-08T19:27:08.188458+0000","last_active":"2023-11-08T19:27:32.314263+0000","last_peered":"2023-11-08T19:27:32.314263+0000","last_clean":"2023-11-08T19:27:32.314263+0000","last_became_active":"2023-11-08T19:27:08.187880+0000","last_became_peered":"2023-11-08T19:27:08.187880+0000","last_unstale":"2023-11-08T19:27:32.314263+0000","last_undegraded":"2023-11-08T19:27:32.314263+0000","last_fullsized":"2023-11-08T19:27:32.314263+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:25:28.618214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":49,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-08T19:27:31.304123+0000","last_change":"2023-11-08T19:26:05.606898+0000","last_active":"2023-11-08T19:27:31.304123+0000","last_peered":"2023-11-08T19:27:31.304123+0000","last_clean":"2023-11-08T19:27:31.304123+0000","last_became_active":"2023-11-08T19:26:04.603955+0000","last_became_peered":"2023-11-08T19:26:04.603955+0000","last_unstale":"2023-11-08T19:27:31.304123+0000","last_undegraded":"2023-11-08T19:27:31.304123+0000","last_fullsized":"2023-11-08T19:27:31.304123+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:19:11.589559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036473499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-08T19:27:31.304009+0000","last_change":"2023-11-08T19:27:09.178770+0000","last_active":"2023-11-08T19:27:31.304009+0000","last_peered":"2023-11-08T19:27:31.304009+0000","last_clean":"2023-11-08T19:27:31.304009+0000","last_became_active":"2023-11-08T19:27:07.194992+0000","last_became_peered":"2023-11-08T19:27:07.194992+0000","last_unstale":"2023-11-08T19:27:31.304009+0000","last_undegraded":"2023-11-08T19:27:31.304009+0000","last_fullsized":"2023-11-08T19:27:31.304009+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:35:34.026910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000770285,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-08T19:27:33.309306+0000","last_change":"2023-11-08T19:27:32.324329+0000","last_active":"2023-11-08T19:27:33.309306+0000","last_peered":"2023-11-08T19:27:33.309306+0000","last_clean":"2023-11-08T19:27:33.309306+0000","last_became_active":"2023-11-08T19:27:32.324044+0000","last_became_peered":"2023-11-08T19:27:32.324044+0000","last_unstale":"2023-11-08T19:27:33.309306+0000","last_undegraded":"2023-11-08T19:27:33.309306+0000","last_fullsized":"2023-11-08T19:27:33.309306+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:45:46.211431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"77'2","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-08T19:27:33.309382+0000","last_change":"2023-11-08T19:27:31.315970+0000","last_active":"2023-11-08T19:27:33.309382+0000","last_peered":"2023-11-08T19:27:33.309382+0000","last_clean":"2023-11-08T19:27:33.309382+0000","last_became_active":"2023-11-08T19:27:31.315638+0000","last_became_peered":"2023-11-08T19:27:31.315638+0000","last_unstale":"2023-11-08T19:27:33.309382+0000","last_undegraded":"2023-11-08T19:27:33.309382+0000","last_fullsized":"2023-11-08T19:27:33.309382+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_clean_scrub_stamp":"2023-11-08T19:27:30.289497+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:24:28.693833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-08T19:27:31.307173+0000","last_change":"2023-11-08T19:27:08.193953+0000","last_active":"2023-11-08T19:27:31.307173+0000","last_peered":"2023-11-08T19:27:31.307173+0000","last_clean":"2023-11-08T19:27:31.307173+0000","last_became_active":"2023-11-08T19:27:08.193653+0000","last_became_peered":"2023-11-08T19:27:08.193653+0000","last_unstale":"2023-11-08T19:27:31.307173+0000","last_undegraded":"2023-11-08T19:27:31.307173+0000","last_fullsized":"2023-11-08T19:27:31.307173+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:35:30.073885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":52,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-08T19:27:31.313067+0000","last_change":"2023-11-08T19:26:06.424464+0000","last_active":"2023-11-08T19:27:31.313067+0000","last_peered":"2023-11-08T19:27:31.313067+0000","last_clean":"2023-11-08T19:27:31.313067+0000","last_became_active":"2023-11-08T19:26:04.603144+0000","last_became_peered":"2023-11-08T19:26:04.603144+0000","last_unstale":"2023-11-08T19:27:31.313067+0000","last_undegraded":"2023-11-08T19:27:31.313067+0000","last_fullsized":"2023-11-08T19:27:31.313067+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:42:55.269365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054090799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.318172+0000","last_change":"2023-11-08T19:27:09.180479+0000","last_active":"2023-11-08T19:27:32.318172+0000","last_peered":"2023-11-08T19:27:32.318172+0000","last_clean":"2023-11-08T19:27:32.318172+0000","last_became_active":"2023-11-08T19:27:07.190438+0000","last_became_peered":"2023-11-08T19:27:07.190438+0000","last_unstale":"2023-11-08T19:27:32.318172+0000","last_undegraded":"2023-11-08T19:27:32.318172+0000","last_fullsized":"2023-11-08T19:27:32.318172+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:25:14.487524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050893300000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.333587+0000","last_change":"2023-11-08T19:27:32.326515+0000","last_active":"2023-11-08T19:27:32.333587+0000","last_peered":"2023-11-08T19:27:32.333587+0000","last_clean":"2023-11-08T19:27:32.333587+0000","last_became_active":"2023-11-08T19:27:32.325953+0000","last_became_peered":"2023-11-08T19:27:32.325953+0000","last_unstale":"2023-11-08T19:27:32.333587+0000","last_undegraded":"2023-11-08T19:27:32.333587+0000","last_fullsized":"2023-11-08T19:27:32.333587+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:58:18.403623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-08T19:27:33.308801+0000","last_change":"2023-11-08T19:27:31.322887+0000","last_active":"2023-11-08T19:27:33.308801+0000","last_peered":"2023-11-08T19:27:33.308801+0000","last_clean":"2023-11-08T19:27:33.308801+0000","last_became_active":"2023-11-08T19:27:31.322598+0000","last_became_peered":"2023-11-08T19:27:31.322598+0000","last_unstale":"2023-11-08T19:27:33.308801+0000","last_undegraded":"2023-11-08T19:27:33.308801+0000","last_fullsized":"2023-11-08T19:27:33.308801+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_clean_scrub_stamp":"2023-11-08T19:27:30.289497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:22:50.960778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-08T19:27:31.315030+0000","last_change":"2023-11-08T19:27:08.190530+0000","last_active":"2023-11-08T19:27:31.315030+0000","last_peered":"2023-11-08T19:27:31.315030+0000","last_clean":"2023-11-08T19:27:31.315030+0000","last_became_active":"2023-11-08T19:27:08.190203+0000","last_became_peered":"2023-11-08T19:27:08.190203+0000","last_unstale":"2023-11-08T19:27:31.315030+0000","last_undegraded":"2023-11-08T19:27:31.315030+0000","last_fullsized":"2023-11-08T19:27:31.315030+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:26:32.777602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":51,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-08T19:27:31.312599+0000","last_change":"2023-11-08T19:26:06.541623+0000","last_active":"2023-11-08T19:27:31.312599+0000","last_peered":"2023-11-08T19:27:31.312599+0000","last_clean":"2023-11-08T19:27:31.312599+0000","last_became_active":"2023-11-08T19:26:04.601961+0000","last_became_peered":"2023-11-08T19:26:04.601961+0000","last_unstale":"2023-11-08T19:27:31.312599+0000","last_undegraded":"2023-11-08T19:27:31.312599+0000","last_fullsized":"2023-11-08T19:27:31.312599+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:11:26.508634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000466386,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-08T19:27:31.313059+0000","last_change":"2023-11-08T19:27:09.431231+0000","last_active":"2023-11-08T19:27:31.313059+0000","last_peered":"2023-11-08T19:27:31.313059+0000","last_clean":"2023-11-08T19:27:31.313059+0000","last_became_active":"2023-11-08T19:27:07.191583+0000","last_became_peered":"2023-11-08T19:27:07.191583+0000","last_unstale":"2023-11-08T19:27:31.313059+0000","last_undegraded":"2023-11-08T19:27:31.313059+0000","last_fullsized":"2023-11-08T19:27:31.313059+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:58:48.357266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093695200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.324342+0000","last_change":"2023-11-08T19:27:32.320291+0000","last_active":"2023-11-08T19:27:32.324342+0000","last_peered":"2023-11-08T19:27:32.324342+0000","last_clean":"2023-11-08T19:27:32.324342+0000","last_became_active":"2023-11-08T19:27:32.320026+0000","last_became_peered":"2023-11-08T19:27:32.320026+0000","last_unstale":"2023-11-08T19:27:32.324342+0000","last_undegraded":"2023-11-08T19:27:32.324342+0000","last_fullsized":"2023-11-08T19:27:32.324342+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:15:16.572033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.320295+0000","last_change":"2023-11-08T19:27:31.314962+0000","last_active":"2023-11-08T19:27:32.320295+0000","last_peered":"2023-11-08T19:27:32.320295+0000","last_clean":"2023-11-08T19:27:32.320295+0000","last_became_active":"2023-11-08T19:27:31.314643+0000","last_became_peered":"2023-11-08T19:27:31.314643+0000","last_unstale":"2023-11-08T19:27:32.320295+0000","last_undegraded":"2023-11-08T19:27:32.320295+0000","last_fullsized":"2023-11-08T19:27:32.320295+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_clean_scrub_stamp":"2023-11-08T19:27:30.289497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:57:26.172555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-08T19:27:31.313134+0000","last_change":"2023-11-08T19:27:08.187900+0000","last_active":"2023-11-08T19:27:31.313134+0000","last_peered":"2023-11-08T19:27:31.313134+0000","last_clean":"2023-11-08T19:27:31.313134+0000","last_became_active":"2023-11-08T19:27:08.187626+0000","last_became_peered":"2023-11-08T19:27:08.187626+0000","last_unstale":"2023-11-08T19:27:31.313134+0000","last_undegraded":"2023-11-08T19:27:31.313134+0000","last_fullsized":"2023-11-08T19:27:31.313134+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:00:55.878397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":58,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.315423+0000","last_change":"2023-11-08T19:26:05.608980+0000","last_active":"2023-11-08T19:27:32.315423+0000","last_peered":"2023-11-08T19:27:32.315423+0000","last_clean":"2023-11-08T19:27:32.315423+0000","last_became_active":"2023-11-08T19:26:04.606386+0000","last_became_peered":"2023-11-08T19:26:04.606386+0000","last_unstale":"2023-11-08T19:27:32.315423+0000","last_undegraded":"2023-11-08T19:27:32.315423+0000","last_fullsized":"2023-11-08T19:27:32.315423+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:41:50.771457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037950599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-08T19:27:31.303745+0000","last_change":"2023-11-08T19:27:09.179037+0000","last_active":"2023-11-08T19:27:31.303745+0000","last_peered":"2023-11-08T19:27:31.303745+0000","last_clean":"2023-11-08T19:27:31.303745+0000","last_became_active":"2023-11-08T19:27:07.195033+0000","last_became_peered":"2023-11-08T19:27:07.195033+0000","last_unstale":"2023-11-08T19:27:31.303745+0000","last_undegraded":"2023-11-08T19:27:31.303745+0000","last_fullsized":"2023-11-08T19:27:31.303745+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:19:08.253483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041824800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.332780+0000","last_change":"2023-11-08T19:27:32.328033+0000","last_active":"2023-11-08T19:27:32.332780+0000","last_peered":"2023-11-08T19:27:32.332780+0000","last_clean":"2023-11-08T19:27:32.332780+0000","last_became_active":"2023-11-08T19:27:32.327746+0000","last_became_peered":"2023-11-08T19:27:32.327746+0000","last_unstale":"2023-11-08T19:27:32.332780+0000","last_undegraded":"2023-11-08T19:27:32.332780+0000","last_fullsized":"2023-11-08T19:27:32.332780+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:46:04.602463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-08T19:27:33.307752+0000","last_change":"2023-11-08T19:27:31.322428+0000","last_active":"2023-11-08T19:27:33.307752+0000","last_peered":"2023-11-08T19:27:33.307752+0000","last_clean":"2023-11-08T19:27:33.307752+0000","last_became_active":"2023-11-08T19:27:31.322190+0000","last_became_peered":"2023-11-08T19:27:31.322190+0000","last_unstale":"2023-11-08T19:27:33.307752+0000","last_undegraded":"2023-11-08T19:27:33.307752+0000","last_fullsized":"2023-11-08T19:27:33.307752+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_clean_scrub_stamp":"2023-11-08T19:27:30.289497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:33:45.068484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-08T19:27:31.326030+0000","last_change":"2023-11-08T19:27:08.193761+0000","last_active":"2023-11-08T19:27:31.326030+0000","last_peered":"2023-11-08T19:27:31.326030+0000","last_clean":"2023-11-08T19:27:31.326030+0000","last_became_active":"2023-11-08T19:27:08.193017+0000","last_became_peered":"2023-11-08T19:27:08.193017+0000","last_unstale":"2023-11-08T19:27:31.326030+0000","last_undegraded":"2023-11-08T19:27:31.326030+0000","last_fullsized":"2023-11-08T19:27:31.326030+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:18:00.971466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":52,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-08T19:27:31.311969+0000","last_change":"2023-11-08T19:26:06.424073+0000","last_active":"2023-11-08T19:27:31.311969+0000","last_peered":"2023-11-08T19:27:31.311969+0000","last_clean":"2023-11-08T19:27:31.311969+0000","last_became_active":"2023-11-08T19:26:04.599046+0000","last_became_peered":"2023-11-08T19:26:04.599046+0000","last_unstale":"2023-11-08T19:27:31.311969+0000","last_undegraded":"2023-11-08T19:27:31.311969+0000","last_fullsized":"2023-11-08T19:27:31.311969+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:04:43.562566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035092199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.317783+0000","last_change":"2023-11-08T19:27:09.181716+0000","last_active":"2023-11-08T19:27:32.317783+0000","last_peered":"2023-11-08T19:27:32.317783+0000","last_clean":"2023-11-08T19:27:32.317783+0000","last_became_active":"2023-11-08T19:27:07.187077+0000","last_became_peered":"2023-11-08T19:27:07.187077+0000","last_unstale":"2023-11-08T19:27:32.317783+0000","last_undegraded":"2023-11-08T19:27:32.317783+0000","last_fullsized":"2023-11-08T19:27:32.317783+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:51:29.247104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056673599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.338674+0000","last_change":"2023-11-08T19:27:32.334380+0000","last_active":"2023-11-08T19:27:32.338674+0000","last_peered":"2023-11-08T19:27:32.338674+0000","last_clean":"2023-11-08T19:27:32.338674+0000","last_became_active":"2023-11-08T19:27:32.334125+0000","last_became_peered":"2023-11-08T19:27:32.334125+0000","last_unstale":"2023-11-08T19:27:32.338674+0000","last_undegraded":"2023-11-08T19:27:32.338674+0000","last_fullsized":"2023-11-08T19:27:32.338674+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:46:55.585097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"77'1","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-08T19:27:33.308913+0000","last_change":"2023-11-08T19:27:31.320048+0000","last_active":"2023-11-08T19:27:33.308913+0000","last_peered":"2023-11-08T19:27:33.308913+0000","last_clean":"2023-11-08T19:27:33.308913+0000","last_became_active":"2023-11-08T19:27:31.319829+0000","last_became_peered":"2023-11-08T19:27:31.319829+0000","last_unstale":"2023-11-08T19:27:33.308913+0000","last_undegraded":"2023-11-08T19:27:33.308913+0000","last_fullsized":"2023-11-08T19:27:33.308913+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_clean_scrub_stamp":"2023-11-08T19:27:30.289497+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:48:28.492986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":21,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.311978+0000","last_change":"2023-11-08T19:27:08.188143+0000","last_active":"2023-11-08T19:27:32.311978+0000","last_peered":"2023-11-08T19:27:32.311978+0000","last_clean":"2023-11-08T19:27:32.311978+0000","last_became_active":"2023-11-08T19:27:08.187714+0000","last_became_peered":"2023-11-08T19:27:08.187714+0000","last_unstale":"2023-11-08T19:27:32.311978+0000","last_undegraded":"2023-11-08T19:27:32.311978+0000","last_fullsized":"2023-11-08T19:27:32.311978+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:02:10.863403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":49,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-08T19:27:31.325889+0000","last_change":"2023-11-08T19:26:06.394320+0000","last_active":"2023-11-08T19:27:31.325889+0000","last_peered":"2023-11-08T19:27:31.325889+0000","last_clean":"2023-11-08T19:27:31.325889+0000","last_became_active":"2023-11-08T19:26:04.603212+0000","last_became_peered":"2023-11-08T19:26:04.603212+0000","last_unstale":"2023-11-08T19:27:31.325889+0000","last_undegraded":"2023-11-08T19:27:31.325889+0000","last_fullsized":"2023-11-08T19:27:31.325889+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:25:29.947079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054695399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.318445+0000","last_change":"2023-11-08T19:27:09.181613+0000","last_active":"2023-11-08T19:27:32.318445+0000","last_peered":"2023-11-08T19:27:32.318445+0000","last_clean":"2023-11-08T19:27:32.318445+0000","last_became_active":"2023-11-08T19:27:07.190897+0000","last_became_peered":"2023-11-08T19:27:07.190897+0000","last_unstale":"2023-11-08T19:27:32.318445+0000","last_undegraded":"2023-11-08T19:27:32.318445+0000","last_fullsized":"2023-11-08T19:27:32.318445+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:27:57.388125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011172459999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-08T19:27:33.309821+0000","last_change":"2023-11-08T19:27:32.328157+0000","last_active":"2023-11-08T19:27:33.309821+0000","last_peered":"2023-11-08T19:27:33.309821+0000","last_clean":"2023-11-08T19:27:33.309821+0000","last_became_active":"2023-11-08T19:27:32.327928+0000","last_became_peered":"2023-11-08T19:27:32.327928+0000","last_unstale":"2023-11-08T19:27:33.309821+0000","last_undegraded":"2023-11-08T19:27:33.309821+0000","last_fullsized":"2023-11-08T19:27:33.309821+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:25:43.090636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"77'1","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.333508+0000","last_change":"2023-11-08T19:27:31.320446+0000","last_active":"2023-11-08T19:27:32.333508+0000","last_peered":"2023-11-08T19:27:32.333508+0000","last_clean":"2023-11-08T19:27:32.333508+0000","last_became_active":"2023-11-08T19:27:31.320148+0000","last_became_peered":"2023-11-08T19:27:31.320148+0000","last_unstale":"2023-11-08T19:27:32.333508+0000","last_undegraded":"2023-11-08T19:27:32.333508+0000","last_fullsized":"2023-11-08T19:27:32.333508+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_clean_scrub_stamp":"2023-11-08T19:27:30.289497+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:41:32.949582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":21,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.311759+0000","last_change":"2023-11-08T19:27:08.188668+0000","last_active":"2023-11-08T19:27:32.311759+0000","last_peered":"2023-11-08T19:27:32.311759+0000","last_clean":"2023-11-08T19:27:32.311759+0000","last_became_active":"2023-11-08T19:27:08.188441+0000","last_became_peered":"2023-11-08T19:27:08.188441+0000","last_unstale":"2023-11-08T19:27:32.311759+0000","last_undegraded":"2023-11-08T19:27:32.311759+0000","last_fullsized":"2023-11-08T19:27:32.311759+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:45:44.379160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":52,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-08T19:27:31.313661+0000","last_change":"2023-11-08T19:26:06.423754+0000","last_active":"2023-11-08T19:27:31.313661+0000","last_peered":"2023-11-08T19:27:31.313661+0000","last_clean":"2023-11-08T19:27:31.313661+0000","last_became_active":"2023-11-08T19:26:04.599322+0000","last_became_peered":"2023-11-08T19:26:04.599322+0000","last_unstale":"2023-11-08T19:27:31.313661+0000","last_undegraded":"2023-11-08T19:27:31.313661+0000","last_fullsized":"2023-11-08T19:27:31.313661+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:13:08.689128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042670000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.317593+0000","last_change":"2023-11-08T19:27:09.181656+0000","last_active":"2023-11-08T19:27:32.317593+0000","last_peered":"2023-11-08T19:27:32.317593+0000","last_clean":"2023-11-08T19:27:32.317593+0000","last_became_active":"2023-11-08T19:27:07.187337+0000","last_became_peered":"2023-11-08T19:27:07.187337+0000","last_unstale":"2023-11-08T19:27:32.317593+0000","last_undegraded":"2023-11-08T19:27:32.317593+0000","last_fullsized":"2023-11-08T19:27:32.317593+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:14:17.605057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000346967,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-08T19:27:33.309303+0000","last_change":"2023-11-08T19:27:32.326637+0000","last_active":"2023-11-08T19:27:33.309303+0000","last_peered":"2023-11-08T19:27:33.309303+0000","last_clean":"2023-11-08T19:27:33.309303+0000","last_became_active":"2023-11-08T19:27:32.326388+0000","last_became_peered":"2023-11-08T19:27:32.326388+0000","last_unstale":"2023-11-08T19:27:33.309303+0000","last_undegraded":"2023-11-08T19:27:33.309303+0000","last_fullsized":"2023-11-08T19:27:33.309303+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:38:32.359142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"77'2","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-08T19:27:33.308394+0000","last_change":"2023-11-08T19:27:31.317671+0000","last_active":"2023-11-08T19:27:33.308394+0000","last_peered":"2023-11-08T19:27:33.308394+0000","last_clean":"2023-11-08T19:27:33.308394+0000","last_became_active":"2023-11-08T19:27:31.317416+0000","last_became_peered":"2023-11-08T19:27:31.317416+0000","last_unstale":"2023-11-08T19:27:33.308394+0000","last_undegraded":"2023-11-08T19:27:33.308394+0000","last_fullsized":"2023-11-08T19:27:33.308394+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_clean_scrub_stamp":"2023-11-08T19:27:30.289497+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:08:47.484656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-08T19:27:31.326903+0000","last_change":"2023-11-08T19:27:08.194065+0000","last_active":"2023-11-08T19:27:31.326903+0000","last_peered":"2023-11-08T19:27:31.326903+0000","last_clean":"2023-11-08T19:27:31.326903+0000","last_became_active":"2023-11-08T19:27:08.193532+0000","last_became_peered":"2023-11-08T19:27:08.193532+0000","last_unstale":"2023-11-08T19:27:31.326903+0000","last_undegraded":"2023-11-08T19:27:31.326903+0000","last_fullsized":"2023-11-08T19:27:31.326903+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:27:58.393129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":49,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-08T19:27:31.326259+0000","last_change":"2023-11-08T19:26:06.394488+0000","last_active":"2023-11-08T19:27:31.326259+0000","last_peered":"2023-11-08T19:27:31.326259+0000","last_clean":"2023-11-08T19:27:31.326259+0000","last_became_active":"2023-11-08T19:26:04.606805+0000","last_became_peered":"2023-11-08T19:26:04.606805+0000","last_unstale":"2023-11-08T19:27:31.326259+0000","last_undegraded":"2023-11-08T19:27:31.326259+0000","last_fullsized":"2023-11-08T19:27:31.326259+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:25:20.857133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045317000000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.311868+0000","last_change":"2023-11-08T19:27:09.211647+0000","last_active":"2023-11-08T19:27:32.311868+0000","last_peered":"2023-11-08T19:27:32.311868+0000","last_clean":"2023-11-08T19:27:32.311868+0000","last_became_active":"2023-11-08T19:27:07.190309+0000","last_became_peered":"2023-11-08T19:27:07.190309+0000","last_unstale":"2023-11-08T19:27:32.311868+0000","last_undegraded":"2023-11-08T19:27:32.311868+0000","last_fullsized":"2023-11-08T19:27:32.311868+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:56:04.292154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047615199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.332615+0000","last_change":"2023-11-08T19:27:32.329293+0000","last_active":"2023-11-08T19:27:32.332615+0000","last_peered":"2023-11-08T19:27:32.332615+0000","last_clean":"2023-11-08T19:27:32.332615+0000","last_became_active":"2023-11-08T19:27:32.329046+0000","last_became_peered":"2023-11-08T19:27:32.329046+0000","last_unstale":"2023-11-08T19:27:32.332615+0000","last_undegraded":"2023-11-08T19:27:32.332615+0000","last_fullsized":"2023-11-08T19:27:32.332615+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:24:29.699037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"77'2","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-08T19:27:33.308862+0000","last_change":"2023-11-08T19:27:31.318947+0000","last_active":"2023-11-08T19:27:33.308862+0000","last_peered":"2023-11-08T19:27:33.308862+0000","last_clean":"2023-11-08T19:27:33.308862+0000","last_became_active":"2023-11-08T19:27:31.318682+0000","last_became_peered":"2023-11-08T19:27:31.318682+0000","last_unstale":"2023-11-08T19:27:33.308862+0000","last_undegraded":"2023-11-08T19:27:33.308862+0000","last_fullsized":"2023-11-08T19:27:33.308862+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_clean_scrub_stamp":"2023-11-08T19:27:30.289497+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:18:24.093678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-08T19:27:31.326274+0000","last_change":"2023-11-08T19:27:08.194884+0000","last_active":"2023-11-08T19:27:31.326274+0000","last_peered":"2023-11-08T19:27:31.326274+0000","last_clean":"2023-11-08T19:27:31.326274+0000","last_became_active":"2023-11-08T19:27:08.193241+0000","last_became_peered":"2023-11-08T19:27:08.193241+0000","last_unstale":"2023-11-08T19:27:31.326274+0000","last_undegraded":"2023-11-08T19:27:31.326274+0000","last_fullsized":"2023-11-08T19:27:31.326274+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:46:31.457682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":50,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.313746+0000","last_change":"2023-11-08T19:26:06.265907+0000","last_active":"2023-11-08T19:27:32.313746+0000","last_peered":"2023-11-08T19:27:32.313746+0000","last_clean":"2023-11-08T19:27:32.313746+0000","last_became_active":"2023-11-08T19:26:04.604772+0000","last_became_peered":"2023-11-08T19:26:04.604772+0000","last_unstale":"2023-11-08T19:27:32.313746+0000","last_undegraded":"2023-11-08T19:27:32.313746+0000","last_fullsized":"2023-11-08T19:27:32.313746+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:44:16.139794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066875300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-08T19:27:31.312115+0000","last_change":"2023-11-08T19:27:09.282699+0000","last_active":"2023-11-08T19:27:31.312115+0000","last_peered":"2023-11-08T19:27:31.312115+0000","last_clean":"2023-11-08T19:27:31.312115+0000","last_became_active":"2023-11-08T19:27:07.191212+0000","last_became_peered":"2023-11-08T19:27:07.191212+0000","last_unstale":"2023-11-08T19:27:31.312115+0000","last_undegraded":"2023-11-08T19:27:31.312115+0000","last_fullsized":"2023-11-08T19:27:31.312115+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:35:14.715110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046640800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.332349+0000","last_change":"2023-11-08T19:27:32.326739+0000","last_active":"2023-11-08T19:27:32.332349+0000","last_peered":"2023-11-08T19:27:32.332349+0000","last_clean":"2023-11-08T19:27:32.332349+0000","last_became_active":"2023-11-08T19:27:32.324726+0000","last_became_peered":"2023-11-08T19:27:32.324726+0000","last_unstale":"2023-11-08T19:27:32.332349+0000","last_undegraded":"2023-11-08T19:27:32.332349+0000","last_fullsized":"2023-11-08T19:27:32.332349+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:22:41.488446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"77'4","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-08T19:27:33.309303+0000","last_change":"2023-11-08T19:27:31.325307+0000","last_active":"2023-11-08T19:27:33.309303+0000","last_peered":"2023-11-08T19:27:33.309303+0000","last_clean":"2023-11-08T19:27:33.309303+0000","last_became_active":"2023-11-08T19:27:31.324998+0000","last_became_peered":"2023-11-08T19:27:31.324998+0000","last_unstale":"2023-11-08T19:27:33.309303+0000","last_undegraded":"2023-11-08T19:27:33.309303+0000","last_fullsized":"2023-11-08T19:27:33.309303+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_clean_scrub_stamp":"2023-11-08T19:27:30.289497+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:30:13.190099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":21,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.311853+0000","last_change":"2023-11-08T19:27:08.187644+0000","last_active":"2023-11-08T19:27:32.311853+0000","last_peered":"2023-11-08T19:27:32.311853+0000","last_clean":"2023-11-08T19:27:32.311853+0000","last_became_active":"2023-11-08T19:27:08.187376+0000","last_became_peered":"2023-11-08T19:27:08.187376+0000","last_unstale":"2023-11-08T19:27:32.311853+0000","last_undegraded":"2023-11-08T19:27:32.311853+0000","last_fullsized":"2023-11-08T19:27:32.311853+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:22:40.300185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":49,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-08T19:27:31.303869+0000","last_change":"2023-11-08T19:26:05.606946+0000","last_active":"2023-11-08T19:27:31.303869+0000","last_peered":"2023-11-08T19:27:31.303869+0000","last_clean":"2023-11-08T19:27:31.303869+0000","last_became_active":"2023-11-08T19:26:04.598775+0000","last_became_peered":"2023-11-08T19:26:04.598775+0000","last_unstale":"2023-11-08T19:27:31.303869+0000","last_undegraded":"2023-11-08T19:27:31.303869+0000","last_fullsized":"2023-11-08T19:27:31.303869+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:20:56.898927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049870399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-08T19:27:31.325951+0000","last_change":"2023-11-08T19:27:09.331646+0000","last_active":"2023-11-08T19:27:31.325951+0000","last_peered":"2023-11-08T19:27:31.325951+0000","last_clean":"2023-11-08T19:27:31.325951+0000","last_became_active":"2023-11-08T19:27:07.190619+0000","last_became_peered":"2023-11-08T19:27:07.190619+0000","last_unstale":"2023-11-08T19:27:31.325951+0000","last_undegraded":"2023-11-08T19:27:31.325951+0000","last_fullsized":"2023-11-08T19:27:31.325951+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:42:47.371458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065918400000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.324554+0000","last_change":"2023-11-08T19:27:32.322603+0000","last_active":"2023-11-08T19:27:32.324554+0000","last_peered":"2023-11-08T19:27:32.324554+0000","last_clean":"2023-11-08T19:27:32.324554+0000","last_became_active":"2023-11-08T19:27:32.322302+0000","last_became_peered":"2023-11-08T19:27:32.322302+0000","last_unstale":"2023-11-08T19:27:32.324554+0000","last_undegraded":"2023-11-08T19:27:32.324554+0000","last_fullsized":"2023-11-08T19:27:32.324554+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:57:27.177759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"77'1","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-08T19:27:33.309850+0000","last_change":"2023-11-08T19:27:31.320554+0000","last_active":"2023-11-08T19:27:33.309850+0000","last_peered":"2023-11-08T19:27:33.309850+0000","last_clean":"2023-11-08T19:27:33.309850+0000","last_became_active":"2023-11-08T19:27:31.320355+0000","last_became_peered":"2023-11-08T19:27:31.320355+0000","last_unstale":"2023-11-08T19:27:33.309850+0000","last_undegraded":"2023-11-08T19:27:33.309850+0000","last_fullsized":"2023-11-08T19:27:33.309850+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_clean_scrub_stamp":"2023-11-08T19:27:30.289497+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:21:02.207345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.318552+0000","last_change":"2023-11-08T19:27:08.183798+0000","last_active":"2023-11-08T19:27:32.318552+0000","last_peered":"2023-11-08T19:27:32.318552+0000","last_clean":"2023-11-08T19:27:32.318552+0000","last_became_active":"2023-11-08T19:27:08.183537+0000","last_became_peered":"2023-11-08T19:27:08.183537+0000","last_unstale":"2023-11-08T19:27:32.318552+0000","last_undegraded":"2023-11-08T19:27:32.318552+0000","last_fullsized":"2023-11-08T19:27:32.318552+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:50:39.026265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":58,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.317439+0000","last_change":"2023-11-08T19:26:05.610012+0000","last_active":"2023-11-08T19:27:32.317439+0000","last_peered":"2023-11-08T19:27:32.317439+0000","last_clean":"2023-11-08T19:27:32.317439+0000","last_became_active":"2023-11-08T19:26:04.603209+0000","last_became_peered":"2023-11-08T19:26:04.603209+0000","last_unstale":"2023-11-08T19:27:32.317439+0000","last_undegraded":"2023-11-08T19:27:32.317439+0000","last_fullsized":"2023-11-08T19:27:32.317439+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:35:02.306997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053335100000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-08T19:27:31.303605+0000","last_change":"2023-11-08T19:27:09.178405+0000","last_active":"2023-11-08T19:27:31.303605+0000","last_peered":"2023-11-08T19:27:31.303605+0000","last_clean":"2023-11-08T19:27:31.303605+0000","last_became_active":"2023-11-08T19:27:07.188169+0000","last_became_peered":"2023-11-08T19:27:07.188169+0000","last_unstale":"2023-11-08T19:27:31.303605+0000","last_undegraded":"2023-11-08T19:27:31.303605+0000","last_fullsized":"2023-11-08T19:27:31.303605+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:39:25.855467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056209700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.321278+0000","last_change":"2023-11-08T19:27:32.317565+0000","last_active":"2023-11-08T19:27:32.321278+0000","last_peered":"2023-11-08T19:27:32.321278+0000","last_clean":"2023-11-08T19:27:32.321278+0000","last_became_active":"2023-11-08T19:27:32.317307+0000","last_became_peered":"2023-11-08T19:27:32.317307+0000","last_unstale":"2023-11-08T19:27:32.321278+0000","last_undegraded":"2023-11-08T19:27:32.321278+0000","last_fullsized":"2023-11-08T19:27:32.321278+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:03:43.454301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.308725+0000","last_change":"2023-11-08T19:27:31.317219+0000","last_active":"2023-11-08T19:27:32.308725+0000","last_peered":"2023-11-08T19:27:32.308725+0000","last_clean":"2023-11-08T19:27:32.308725+0000","last_became_active":"2023-11-08T19:27:31.316299+0000","last_became_peered":"2023-11-08T19:27:31.316299+0000","last_unstale":"2023-11-08T19:27:32.308725+0000","last_undegraded":"2023-11-08T19:27:32.308725+0000","last_fullsized":"2023-11-08T19:27:32.308725+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_clean_scrub_stamp":"2023-11-08T19:27:30.289497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:22:50.960778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-08T19:27:31.325684+0000","last_change":"2023-11-08T19:27:08.193508+0000","last_active":"2023-11-08T19:27:31.325684+0000","last_peered":"2023-11-08T19:27:31.325684+0000","last_clean":"2023-11-08T19:27:31.325684+0000","last_became_active":"2023-11-08T19:27:08.193037+0000","last_became_peered":"2023-11-08T19:27:08.193037+0000","last_unstale":"2023-11-08T19:27:31.325684+0000","last_undegraded":"2023-11-08T19:27:31.325684+0000","last_fullsized":"2023-11-08T19:27:31.325684+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:29:50.067887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":50,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.311882+0000","last_change":"2023-11-08T19:26:06.265318+0000","last_active":"2023-11-08T19:27:32.311882+0000","last_peered":"2023-11-08T19:27:32.311882+0000","last_clean":"2023-11-08T19:27:32.311882+0000","last_became_active":"2023-11-08T19:26:04.603409+0000","last_became_peered":"2023-11-08T19:26:04.603409+0000","last_unstale":"2023-11-08T19:27:32.311882+0000","last_undegraded":"2023-11-08T19:27:32.311882+0000","last_fullsized":"2023-11-08T19:27:32.311882+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:27:24.933111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065684300000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.319225+0000","last_change":"2023-11-08T19:27:09.182282+0000","last_active":"2023-11-08T19:27:32.319225+0000","last_peered":"2023-11-08T19:27:32.319225+0000","last_clean":"2023-11-08T19:27:32.319225+0000","last_became_active":"2023-11-08T19:27:07.186944+0000","last_became_peered":"2023-11-08T19:27:07.186944+0000","last_unstale":"2023-11-08T19:27:32.319225+0000","last_undegraded":"2023-11-08T19:27:32.319225+0000","last_fullsized":"2023-11-08T19:27:32.319225+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:45:21.079011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00028665599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.324720+0000","last_change":"2023-11-08T19:27:32.321504+0000","last_active":"2023-11-08T19:27:32.324720+0000","last_peered":"2023-11-08T19:27:32.324720+0000","last_clean":"2023-11-08T19:27:32.324720+0000","last_became_active":"2023-11-08T19:27:32.321104+0000","last_became_peered":"2023-11-08T19:27:32.321104+0000","last_unstale":"2023-11-08T19:27:32.324720+0000","last_undegraded":"2023-11-08T19:27:32.324720+0000","last_fullsized":"2023-11-08T19:27:32.324720+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:20:42.456197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"77'8","reported_seq":19,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.334964+0000","last_change":"2023-11-08T19:27:31.320269+0000","last_active":"2023-11-08T19:27:32.334964+0000","last_peered":"2023-11-08T19:27:32.334964+0000","last_clean":"2023-11-08T19:27:32.334964+0000","last_became_active":"2023-11-08T19:27:31.319962+0000","last_became_peered":"2023-11-08T19:27:31.319962+0000","last_unstale":"2023-11-08T19:27:32.334964+0000","last_undegraded":"2023-11-08T19:27:32.334964+0000","last_fullsized":"2023-11-08T19:27:32.334964+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_clean_scrub_stamp":"2023-11-08T19:27:30.289497+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:51:53.374320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-08T19:27:31.313714+0000","last_change":"2023-11-08T19:27:08.193990+0000","last_active":"2023-11-08T19:27:31.313714+0000","last_peered":"2023-11-08T19:27:31.313714+0000","last_clean":"2023-11-08T19:27:31.313714+0000","last_became_active":"2023-11-08T19:27:08.193624+0000","last_became_peered":"2023-11-08T19:27:08.193624+0000","last_unstale":"2023-11-08T19:27:31.313714+0000","last_undegraded":"2023-11-08T19:27:31.313714+0000","last_fullsized":"2023-11-08T19:27:31.313714+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:46:41.321120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":52,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-08T19:27:31.311786+0000","last_change":"2023-11-08T19:26:06.423863+0000","last_active":"2023-11-08T19:27:31.311786+0000","last_peered":"2023-11-08T19:27:31.311786+0000","last_clean":"2023-11-08T19:27:31.311786+0000","last_became_active":"2023-11-08T19:26:04.608639+0000","last_became_peered":"2023-11-08T19:26:04.608639+0000","last_unstale":"2023-11-08T19:27:31.311786+0000","last_undegraded":"2023-11-08T19:27:31.311786+0000","last_fullsized":"2023-11-08T19:27:31.311786+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:04:43.631102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044384999999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-08T19:27:31.314667+0000","last_change":"2023-11-08T19:27:09.283232+0000","last_active":"2023-11-08T19:27:31.314667+0000","last_peered":"2023-11-08T19:27:31.314667+0000","last_clean":"2023-11-08T19:27:31.314667+0000","last_became_active":"2023-11-08T19:27:07.190931+0000","last_became_peered":"2023-11-08T19:27:07.190931+0000","last_unstale":"2023-11-08T19:27:31.314667+0000","last_undegraded":"2023-11-08T19:27:31.314667+0000","last_fullsized":"2023-11-08T19:27:31.314667+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:43:28.225967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034146799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.333802+0000","last_change":"2023-11-08T19:27:32.331085+0000","last_active":"2023-11-08T19:27:32.333802+0000","last_peered":"2023-11-08T19:27:32.333802+0000","last_clean":"2023-11-08T19:27:32.333802+0000","last_became_active":"2023-11-08T19:27:32.330779+0000","last_became_peered":"2023-11-08T19:27:32.330779+0000","last_unstale":"2023-11-08T19:27:32.333802+0000","last_undegraded":"2023-11-08T19:27:32.333802+0000","last_fullsized":"2023-11-08T19:27:32.333802+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:28:22.520545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"77'2","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-08T19:27:33.308970+0000","last_change":"2023-11-08T19:27:31.331312+0000","last_active":"2023-11-08T19:27:33.308970+0000","last_peered":"2023-11-08T19:27:33.308970+0000","last_clean":"2023-11-08T19:27:33.308970+0000","last_became_active":"2023-11-08T19:27:31.330915+0000","last_became_peered":"2023-11-08T19:27:31.330915+0000","last_unstale":"2023-11-08T19:27:33.308970+0000","last_undegraded":"2023-11-08T19:27:33.308970+0000","last_fullsized":"2023-11-08T19:27:33.308970+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_clean_scrub_stamp":"2023-11-08T19:27:30.289497+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:15:15.566829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":21,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-08T19:27:31.312243+0000","last_change":"2023-11-08T19:27:08.187252+0000","last_active":"2023-11-08T19:27:31.312243+0000","last_peered":"2023-11-08T19:27:31.312243+0000","last_clean":"2023-11-08T19:27:31.312243+0000","last_became_active":"2023-11-08T19:27:08.186952+0000","last_became_peered":"2023-11-08T19:27:08.186952+0000","last_unstale":"2023-11-08T19:27:31.312243+0000","last_undegraded":"2023-11-08T19:27:31.312243+0000","last_fullsized":"2023-11-08T19:27:31.312243+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:12:52.022374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":49,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-08T19:27:31.325983+0000","last_change":"2023-11-08T19:26:06.394081+0000","last_active":"2023-11-08T19:27:31.325983+0000","last_peered":"2023-11-08T19:27:31.325983+0000","last_clean":"2023-11-08T19:27:31.325983+0000","last_became_active":"2023-11-08T19:26:04.602137+0000","last_became_peered":"2023-11-08T19:26:04.602137+0000","last_unstale":"2023-11-08T19:27:31.325983+0000","last_undegraded":"2023-11-08T19:27:31.325983+0000","last_fullsized":"2023-11-08T19:27:31.325983+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:05:46.951663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056998200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-08T19:27:31.304150+0000","last_change":"2023-11-08T19:27:09.178816+0000","last_active":"2023-11-08T19:27:31.304150+0000","last_peered":"2023-11-08T19:27:31.304150+0000","last_clean":"2023-11-08T19:27:31.304150+0000","last_became_active":"2023-11-08T19:27:07.192001+0000","last_became_peered":"2023-11-08T19:27:07.192001+0000","last_unstale":"2023-11-08T19:27:31.304150+0000","last_undegraded":"2023-11-08T19:27:31.304150+0000","last_fullsized":"2023-11-08T19:27:31.304150+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:04:12.481212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045359000000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-08T19:27:33.308306+0000","last_change":"2023-11-08T19:27:32.325945+0000","last_active":"2023-11-08T19:27:33.308306+0000","last_peered":"2023-11-08T19:27:33.308306+0000","last_clean":"2023-11-08T19:27:33.308306+0000","last_became_active":"2023-11-08T19:27:32.325695+0000","last_became_peered":"2023-11-08T19:27:32.325695+0000","last_unstale":"2023-11-08T19:27:33.308306+0000","last_undegraded":"2023-11-08T19:27:33.308306+0000","last_fullsized":"2023-11-08T19:27:33.308306+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:43:12.503877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.315606+0000","last_change":"2023-11-08T19:27:31.326547+0000","last_active":"2023-11-08T19:27:32.315606+0000","last_peered":"2023-11-08T19:27:32.315606+0000","last_clean":"2023-11-08T19:27:32.315606+0000","last_became_active":"2023-11-08T19:27:31.325766+0000","last_became_peered":"2023-11-08T19:27:31.325766+0000","last_unstale":"2023-11-08T19:27:32.315606+0000","last_undegraded":"2023-11-08T19:27:32.315606+0000","last_fullsized":"2023-11-08T19:27:32.315606+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_clean_scrub_stamp":"2023-11-08T19:27:30.289497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:28:53.687831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-08T19:27:31.310810+0000","last_change":"2023-11-08T19:27:08.193836+0000","last_active":"2023-11-08T19:27:31.310810+0000","last_peered":"2023-11-08T19:27:31.310810+0000","last_clean":"2023-11-08T19:27:31.310810+0000","last_became_active":"2023-11-08T19:27:08.193458+0000","last_became_peered":"2023-11-08T19:27:08.193458+0000","last_unstale":"2023-11-08T19:27:31.310810+0000","last_undegraded":"2023-11-08T19:27:31.310810+0000","last_fullsized":"2023-11-08T19:27:31.310810+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:14:18.610061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":51,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-08T19:27:31.313794+0000","last_change":"2023-11-08T19:26:06.541338+0000","last_active":"2023-11-08T19:27:31.313794+0000","last_peered":"2023-11-08T19:27:31.313794+0000","last_clean":"2023-11-08T19:27:31.313794+0000","last_became_active":"2023-11-08T19:26:04.602183+0000","last_became_peered":"2023-11-08T19:26:04.602183+0000","last_unstale":"2023-11-08T19:27:31.313794+0000","last_undegraded":"2023-11-08T19:27:31.313794+0000","last_fullsized":"2023-11-08T19:27:31.313794+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:06:47.446686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052029099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.317519+0000","last_change":"2023-11-08T19:27:09.181622+0000","last_active":"2023-11-08T19:27:32.317519+0000","last_peered":"2023-11-08T19:27:32.317519+0000","last_clean":"2023-11-08T19:27:32.317519+0000","last_became_active":"2023-11-08T19:27:07.184507+0000","last_became_peered":"2023-11-08T19:27:07.184507+0000","last_unstale":"2023-11-08T19:27:32.317519+0000","last_undegraded":"2023-11-08T19:27:32.317519+0000","last_fullsized":"2023-11-08T19:27:32.317519+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:58:48.357266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051924700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.335780+0000","last_change":"2023-11-08T19:27:32.327660+0000","last_active":"2023-11-08T19:27:32.335780+0000","last_peered":"2023-11-08T19:27:32.335780+0000","last_clean":"2023-11-08T19:27:32.335780+0000","last_became_active":"2023-11-08T19:27:32.327429+0000","last_became_peered":"2023-11-08T19:27:32.327429+0000","last_unstale":"2023-11-08T19:27:32.335780+0000","last_undegraded":"2023-11-08T19:27:32.335780+0000","last_fullsized":"2023-11-08T19:27:32.335780+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:33:46.073688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"77'3","reported_seq":15,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-08T19:27:33.310765+0000","last_change":"2023-11-08T19:27:31.325270+0000","last_active":"2023-11-08T19:27:33.310765+0000","last_peered":"2023-11-08T19:27:33.310765+0000","last_clean":"2023-11-08T19:27:33.310765+0000","last_became_active":"2023-11-08T19:27:31.324996+0000","last_became_peered":"2023-11-08T19:27:31.324996+0000","last_unstale":"2023-11-08T19:27:33.310765+0000","last_undegraded":"2023-11-08T19:27:33.310765+0000","last_fullsized":"2023-11-08T19:27:33.310765+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_clean_scrub_stamp":"2023-11-08T19:27:30.289497+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:28:21.515340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-08T19:27:31.313158+0000","last_change":"2023-11-08T19:27:08.194263+0000","last_active":"2023-11-08T19:27:31.313158+0000","last_peered":"2023-11-08T19:27:31.313158+0000","last_clean":"2023-11-08T19:27:31.313158+0000","last_became_active":"2023-11-08T19:27:08.193938+0000","last_became_peered":"2023-11-08T19:27:08.193938+0000","last_unstale":"2023-11-08T19:27:31.313158+0000","last_undegraded":"2023-11-08T19:27:31.313158+0000","last_fullsized":"2023-11-08T19:27:31.313158+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:40:49.009717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":50,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.312557+0000","last_change":"2023-11-08T19:26:06.265690+0000","last_active":"2023-11-08T19:27:32.312557+0000","last_peered":"2023-11-08T19:27:32.312557+0000","last_clean":"2023-11-08T19:27:32.312557+0000","last_became_active":"2023-11-08T19:26:04.603397+0000","last_became_peered":"2023-11-08T19:26:04.603397+0000","last_unstale":"2023-11-08T19:27:32.312557+0000","last_undegraded":"2023-11-08T19:27:32.312557+0000","last_fullsized":"2023-11-08T19:27:32.312557+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:50:43.827570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053361499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-08T19:27:31.312744+0000","last_change":"2023-11-08T19:27:09.282750+0000","last_active":"2023-11-08T19:27:31.312744+0000","last_peered":"2023-11-08T19:27:31.312744+0000","last_clean":"2023-11-08T19:27:31.312744+0000","last_became_active":"2023-11-08T19:27:07.189943+0000","last_became_peered":"2023-11-08T19:27:07.189943+0000","last_unstale":"2023-11-08T19:27:31.312744+0000","last_undegraded":"2023-11-08T19:27:31.312744+0000","last_fullsized":"2023-11-08T19:27:31.312744+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:31:16.081058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000398725,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.331875+0000","last_change":"2023-11-08T19:27:32.325712+0000","last_active":"2023-11-08T19:27:32.331875+0000","last_peered":"2023-11-08T19:27:32.331875+0000","last_clean":"2023-11-08T19:27:32.331875+0000","last_became_active":"2023-11-08T19:27:32.325440+0000","last_became_peered":"2023-11-08T19:27:32.325440+0000","last_unstale":"2023-11-08T19:27:32.331875+0000","last_undegraded":"2023-11-08T19:27:32.331875+0000","last_fullsized":"2023-11-08T19:27:32.331875+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:48:29.498190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"77'6","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.337588+0000","last_change":"2023-11-08T19:27:31.330983+0000","last_active":"2023-11-08T19:27:32.337588+0000","last_peered":"2023-11-08T19:27:32.337588+0000","last_clean":"2023-11-08T19:27:32.337588+0000","last_became_active":"2023-11-08T19:27:31.330683+0000","last_became_peered":"2023-11-08T19:27:31.330683+0000","last_unstale":"2023-11-08T19:27:32.337588+0000","last_undegraded":"2023-11-08T19:27:32.337588+0000","last_fullsized":"2023-11-08T19:27:32.337588+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_clean_scrub_stamp":"2023-11-08T19:27:30.289497+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:20:52.279156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.313521+0000","last_change":"2023-11-08T19:27:08.188147+0000","last_active":"2023-11-08T19:27:32.313521+0000","last_peered":"2023-11-08T19:27:32.313521+0000","last_clean":"2023-11-08T19:27:32.313521+0000","last_became_active":"2023-11-08T19:27:08.187487+0000","last_became_peered":"2023-11-08T19:27:08.187487+0000","last_unstale":"2023-11-08T19:27:32.313521+0000","last_undegraded":"2023-11-08T19:27:32.313521+0000","last_fullsized":"2023-11-08T19:27:32.313521+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:22:40.300185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":52,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-08T19:27:31.311969+0000","last_change":"2023-11-08T19:26:06.425133+0000","last_active":"2023-11-08T19:27:31.311969+0000","last_peered":"2023-11-08T19:27:31.311969+0000","last_clean":"2023-11-08T19:27:31.311969+0000","last_became_active":"2023-11-08T19:26:04.601103+0000","last_became_peered":"2023-11-08T19:26:04.601103+0000","last_unstale":"2023-11-08T19:27:31.311969+0000","last_undegraded":"2023-11-08T19:27:31.311969+0000","last_fullsized":"2023-11-08T19:27:31.311969+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:31:14.042561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000229244,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.315336+0000","last_change":"2023-11-08T19:27:09.179852+0000","last_active":"2023-11-08T19:27:32.315336+0000","last_peered":"2023-11-08T19:27:32.315336+0000","last_clean":"2023-11-08T19:27:32.315336+0000","last_became_active":"2023-11-08T19:27:07.191926+0000","last_became_peered":"2023-11-08T19:27:07.191926+0000","last_unstale":"2023-11-08T19:27:32.315336+0000","last_undegraded":"2023-11-08T19:27:32.315336+0000","last_fullsized":"2023-11-08T19:27:32.315336+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:20:38.080129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055457100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-08T19:27:33.308961+0000","last_change":"2023-11-08T19:27:32.336494+0000","last_active":"2023-11-08T19:27:33.308961+0000","last_peered":"2023-11-08T19:27:33.308961+0000","last_clean":"2023-11-08T19:27:33.308961+0000","last_became_active":"2023-11-08T19:27:32.336133+0000","last_became_peered":"2023-11-08T19:27:32.336133+0000","last_unstale":"2023-11-08T19:27:33.308961+0000","last_undegraded":"2023-11-08T19:27:33.308961+0000","last_fullsized":"2023-11-08T19:27:33.308961+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:25:43.090636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.309954+0000","last_change":"2023-11-08T19:27:31.317961+0000","last_active":"2023-11-08T19:27:32.309954+0000","last_peered":"2023-11-08T19:27:32.309954+0000","last_clean":"2023-11-08T19:27:32.309954+0000","last_became_active":"2023-11-08T19:27:31.317699+0000","last_became_peered":"2023-11-08T19:27:31.317699+0000","last_unstale":"2023-11-08T19:27:32.309954+0000","last_undegraded":"2023-11-08T19:27:32.309954+0000","last_fullsized":"2023-11-08T19:27:32.309954+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_clean_scrub_stamp":"2023-11-08T19:27:30.289497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:45:45.206227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.312978+0000","last_change":"2023-11-08T19:27:08.186054+0000","last_active":"2023-11-08T19:27:32.312978+0000","last_peered":"2023-11-08T19:27:32.312978+0000","last_clean":"2023-11-08T19:27:32.312978+0000","last_became_active":"2023-11-08T19:27:08.185581+0000","last_became_peered":"2023-11-08T19:27:08.185581+0000","last_unstale":"2023-11-08T19:27:32.312978+0000","last_undegraded":"2023-11-08T19:27:32.312978+0000","last_fullsized":"2023-11-08T19:27:32.312978+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:00:55.878397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":58,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.316763+0000","last_change":"2023-11-08T19:26:05.609625+0000","last_active":"2023-11-08T19:27:32.316763+0000","last_peered":"2023-11-08T19:27:32.316763+0000","last_clean":"2023-11-08T19:27:32.316763+0000","last_became_active":"2023-11-08T19:26:04.606312+0000","last_became_peered":"2023-11-08T19:26:04.606312+0000","last_unstale":"2023-11-08T19:27:32.316763+0000","last_undegraded":"2023-11-08T19:27:32.316763+0000","last_fullsized":"2023-11-08T19:27:32.316763+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:54:42.032901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000505935,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-08T19:27:31.303571+0000","last_change":"2023-11-08T19:27:09.177985+0000","last_active":"2023-11-08T19:27:31.303571+0000","last_peered":"2023-11-08T19:27:31.303571+0000","last_clean":"2023-11-08T19:27:31.303571+0000","last_became_active":"2023-11-08T19:27:07.193254+0000","last_became_peered":"2023-11-08T19:27:07.193254+0000","last_unstale":"2023-11-08T19:27:31.303571+0000","last_undegraded":"2023-11-08T19:27:31.303571+0000","last_fullsized":"2023-11-08T19:27:31.303571+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:23:38.662629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041830799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-08T19:27:33.308285+0000","last_change":"2023-11-08T19:27:32.335865+0000","last_active":"2023-11-08T19:27:33.308285+0000","last_peered":"2023-11-08T19:27:33.308285+0000","last_clean":"2023-11-08T19:27:33.308285+0000","last_became_active":"2023-11-08T19:27:32.335585+0000","last_became_peered":"2023-11-08T19:27:32.335585+0000","last_unstale":"2023-11-08T19:27:33.308285+0000","last_undegraded":"2023-11-08T19:27:33.308285+0000","last_fullsized":"2023-11-08T19:27:33.308285+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:31.294701+0000","last_clean_scrub_stamp":"2023-11-08T19:27:31.294701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:22:41.488446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.315417+0000","last_change":"2023-11-08T19:27:31.318789+0000","last_active":"2023-11-08T19:27:32.315417+0000","last_peered":"2023-11-08T19:27:32.315417+0000","last_clean":"2023-11-08T19:27:32.315417+0000","last_became_active":"2023-11-08T19:27:31.318530+0000","last_became_peered":"2023-11-08T19:27:31.318530+0000","last_unstale":"2023-11-08T19:27:32.315417+0000","last_undegraded":"2023-11-08T19:27:32.315417+0000","last_fullsized":"2023-11-08T19:27:32.315417+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:30.289497+0000","last_clean_scrub_stamp":"2023-11-08T19:27:30.289497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:54:43.271973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-08T19:27:32.315279+0000","last_change":"2023-11-08T19:27:08.188860+0000","last_active":"2023-11-08T19:27:32.315279+0000","last_peered":"2023-11-08T19:27:32.315279+0000","last_clean":"2023-11-08T19:27:32.315279+0000","last_became_active":"2023-11-08T19:27:08.188560+0000","last_became_peered":"2023-11-08T19:27:08.188560+0000","last_unstale":"2023-11-08T19:27:32.315279+0000","last_undegraded":"2023-11-08T19:27:32.315279+0000","last_fullsized":"2023-11-08T19:27:32.315279+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:18:20.969491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":50,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-08T19:27:31.307866+0000","last_change":"2023-11-08T19:26:06.297255+0000","last_active":"2023-11-08T19:27:31.307866+0000","last_peered":"2023-11-08T19:27:31.307866+0000","last_clean":"2023-11-08T19:27:31.307866+0000","last_became_active":"2023-11-08T19:26:04.603979+0000","last_became_peered":"2023-11-08T19:26:04.603979+0000","last_unstale":"2023-11-08T19:27:31.307866+0000","last_undegraded":"2023-11-08T19:27:31.307866+0000","last_fullsized":"2023-11-08T19:27:31.307866+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:40:11.164951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079032099999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-11-08T19:27:31.325852+0000","last_change":"2023-11-08T19:27:09.331990+0000","last_active":"2023-11-08T19:27:31.325852+0000","last_peered":"2023-11-08T19:27:31.325852+0000","last_clean":"2023-11-08T19:27:31.325852+0000","last_became_active":"2023-11-08T19:27:07.185478+0000","last_became_peered":"2023-11-08T19:27:07.185478+0000","last_unstale":"2023-11-08T19:27:31.325852+0000","last_undegraded":"2023-11-08T19:27:31.325852+0000","last_fullsized":"2023-11-08T19:27:31.325852+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:20:38.080129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010661850000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574886,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28540,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343300,"statfs":{"total":96636764160,"available":96607539200,"internally_reserved":0,"allocated":1552384,"data_stored":1027604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:26:48 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.504,"15min":0.498},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.78600000000000003,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.51700000000000002,"15min":0.51500000000000001},"min":{"1min":0.35099999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.65400000000000003,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.59399999999999997}]},{"osd":1,"last update":"Wed Nov 8 19:26:48 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.56999999999999995,"15min":0.56599999999999995},"min":{"1min":0.42099999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.624},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.56100000000000005,"15min":0.55500000000000005},"min":{"1min":0.432,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.76100000000000001,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.53000000000000003}]},{"osd":2,"last update":"Wed Nov 8 19:26:48 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.501,"15min":0.49099999999999999},"min":{"1min":0.39500000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.55400000000000005,"15min":0.54900000000000004},"min":{"1min":0.42999999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.66700000000000004}]},{"osd":3,"last update":"Wed Nov 8 19:26:48 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56000000000000005,"15min":0.55900000000000005},"min":{"1min":0.442,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.58899999999999997,"15min":0.57699999999999996},"min":{"1min":0.47899999999999998,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.55500000000000005}]},{"osd":4,"last update":"Wed Nov 8 19:26:48 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.51600000000000001,"15min":0.51000000000000001},"min":{"1min":0.38300000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.73299999999999998,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.54500000000000004,"15min":0.54100000000000004},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.61299999999999999}]},{"osd":5,"last update":"Wed Nov 8 19:26:48 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.58699999999999997,"15min":0.58099999999999996},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.64700000000000002,"15min":0.64300000000000002},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.78300000000000003,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.72499999999999998}]},{"osd":6,"last update":"Wed Nov 8 19:26:48 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.60999999999999999,"15min":0.60699999999999998},"min":{"1min":0.47599999999999998,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.81100000000000005,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.61699999999999999,"15min":0.61199999999999999},"min":{"1min":0.47699999999999998,"5min":0.443,"15min":0.443},"max":{"1min":0.78800000000000003,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.57999999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574886,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27968,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343872,"statfs":{"total":96636764160,"available":96608124928,"internally_reserved":0,"allocated":970752,"data_stored":439396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:26:47 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":1.26,"15min":1.3799999999999999},"min":{"1min":0.33900000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.91700000000000004,"5min":21.359999999999999,"15min":21.359999999999999},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.86099999999999999,"15min":0.91300000000000003},"min":{"1min":0.38100000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.90300000000000002,"5min":10.329000000000001,"15min":10.329000000000001},"last":0.501}]},{"osd":1,"last update":"Wed Nov 8 19:26:47 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.60299999999999998,"15min":0.61699999999999999},"min":{"1min":0.35999999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.86299999999999999,"5min":3.4239999999999999,"15min":3.4239999999999999},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.63100000000000001,"15min":0.64300000000000002},"min":{"1min":0.40999999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.94999999999999996,"5min":3.4809999999999999,"15min":3.4809999999999999},"last":0.58599999999999997}]},{"osd":2,"last update":"Wed Nov 8 19:26:47 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.879,"15min":0.93100000000000005},"min":{"1min":0.38500000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.96399999999999997,"5min":10.398,"15min":10.398},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.67800000000000005,"15min":0.69799999999999995},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.89100000000000001,"5min":5.0170000000000003,"15min":5.0170000000000003},"last":0.52200000000000002}]},{"osd":3,"last update":"Wed Nov 8 19:26:47 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.65600000000000003,"15min":0.66900000000000004},"min":{"1min":0.34599999999999997,"5min":0.317,"15min":0.317},"max":{"1min":0.877,"5min":3.5710000000000002,"15min":3.5710000000000002},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.91000000000000003,"15min":0.96399999999999997},"min":{"1min":0.41099999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.93500000000000005,"5min":10.368,"15min":10.368},"last":0.61799999999999999}]},{"osd":4,"last update":"Wed Nov 8 19:26:47 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.52500000000000002,"15min":0.51300000000000001},"min":{"1min":0.36899999999999999,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.56499999999999995,"15min":0.56000000000000005},"min":{"1min":0.39200000000000002,"5min":0.373,"15min":0.373},"max":{"1min":0.753,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.44}]},{"osd":5,"last update":"Wed Nov 8 19:26:47 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.53900000000000003,"15min":0.52900000000000003},"min":{"1min":0.41599999999999998,"5min":0.307,"15min":0.307},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.58999999999999997,"15min":0.58099999999999996},"min":{"1min":0.51600000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.46899999999999997}]},{"osd":7,"last update":"Wed Nov 8 19:26:47 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.58499999999999996,"15min":0.57499999999999996},"min":{"1min":0.48599999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.84199999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.59799999999999998,"15min":0.58999999999999997},"min":{"1min":0.498,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.80500000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.60499999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574886,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27948,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343892,"statfs":{"total":96636764160,"available":96608145408,"internally_reserved":0,"allocated":942080,"data_stored":429887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:26:47 2023","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.377,"15min":0.38100000000000001},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.499,"5min":1.389,"15min":1.389},"last":0.254},{"interface":"front","average":{"1min":0.34899999999999998,"5min":0.66500000000000004,"15min":0.71699999999999997},"min":{"1min":0.23499999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.55200000000000005,"5min":9.1370000000000005,"15min":9.1370000000000005},"last":0.36099999999999999}]},{"osd":2,"last update":"Wed Nov 8 19:26:47 2023","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.35999999999999999,"15min":0.36499999999999999},"min":{"1min":0.25600000000000001,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.47799999999999998,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.316},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.42299999999999999,"15min":0.42899999999999999},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.56399999999999995,"5min":1.425,"15min":1.425},"last":0.34799999999999998}]},{"osd":3,"last update":"Wed Nov 8 19:26:47 2023","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.80700000000000005,"15min":0.877},"min":{"1min":0.28000000000000003,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.54300000000000004,"5min":11.957000000000001,"15min":11.957000000000001},"last":0.28799999999999998},{"interface":"front","average":{"1min":0.379,"5min":1.0209999999999999,"15min":1.1279999999999999},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.54400000000000004,"5min":18.039000000000001,"15min":18.039000000000001},"last":0.27700000000000002}]},{"osd":4,"last update":"Wed Nov 8 19:26:47 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.78300000000000003,"15min":0.81399999999999995},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.4370000000000001,"5min":6.181,"15min":6.181},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.64300000000000002,"15min":0.64100000000000001},"min":{"1min":0.45200000000000001,"5min":0.377,"15min":0.377},"max":{"1min":1.411,"5min":1.488,"15min":1.488},"last":0.64700000000000002}]},{"osd":5,"last update":"Wed Nov 8 19:26:47 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.92000000000000004,"15min":0.96399999999999997},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.421,"5min":9.2420000000000009,"15min":9.2420000000000009},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.83299999999999996,"15min":0.86499999999999999},"min":{"1min":0.439,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.2969999999999999,"5min":6.2110000000000003,"15min":6.2110000000000003},"last":0.745}]},{"osd":6,"last update":"Wed Nov 8 19:26:47 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.92800000000000005,"15min":0.96699999999999997},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.4530000000000001,"5min":9.1880000000000006,"15min":9.1880000000000006},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.272,"5min":1.462,"15min":1.462},"last":0.56799999999999995}]},{"osd":7,"last update":"Wed Nov 8 19:26:47 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.82699999999999996,"15min":0.85499999999999998},"min":{"1min":0.46899999999999997,"5min":0.435,"15min":0.435},"max":{"1min":1.3979999999999999,"5min":6.1479999999999997,"15min":6.1479999999999997},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.95699999999999996,"15min":1.004},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.353,"5min":9.2170000000000005,"15min":9.2170000000000005},"last":0.55600000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574886,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28540,"kb_used_data":1520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343300,"statfs":{"total":96636764160,"available":96607539200,"internally_reserved":0,"allocated":1556480,"data_stored":1022503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Nov 8 19:26:46 2023","interfaces":[{"interface":"back","average":{"1min":0.31900000000000001,"5min":0.48199999999999998,"15min":0.50900000000000001},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.45200000000000001,"5min":4.1200000000000001,"15min":4.1200000000000001},"last":0.38},{"interface":"front","average":{"1min":0.36199999999999999,"5min":1.026,"15min":1.137},"min":{"1min":0.26200000000000001,"5min":0.253,"15min":0.253},"max":{"1min":0.52200000000000002,"5min":16.940999999999999,"15min":16.940999999999999},"last":0.26900000000000002}]},{"osd":2,"last update":"Wed Nov 8 19:26:46 2023","interfaces":[{"interface":"back","average":{"1min":0.34200000000000003,"5min":0.39100000000000001,"15min":0.39900000000000002},"min":{"1min":0.25700000000000001,"5min":0.246,"15min":0.246},"max":{"1min":0.54500000000000004,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.28899999999999998},{"interface":"front","average":{"1min":0.378,"5min":0.64500000000000002,"15min":0.68899999999999995},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.56899999999999995,"5min":7.0209999999999999,"15min":7.0209999999999999},"last":0.35699999999999998}]},{"osd":3,"last update":"Wed Nov 8 19:26:46 2023","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.53700000000000003,"15min":0.56299999999999994},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.51500000000000001,"5min":4.0940000000000003,"15min":4.0940000000000003},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.439,"15min":0.44700000000000001},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.62,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.317}]},{"osd":4,"last update":"Wed Nov 8 19:26:46 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.73599999999999999,"15min":0.76600000000000001},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.70799999999999996,"5min":4.0049999999999999,"15min":4.0049999999999999},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.63400000000000001,"15min":0.64000000000000001},"min":{"1min":0.45700000000000002,"5min":0.436,"15min":0.436},"max":{"1min":0.79200000000000004,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.59499999999999997}]},{"osd":5,"last update":"Wed Nov 8 19:26:46 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.753,"15min":0.78000000000000003},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.81699999999999995,"5min":4.3090000000000002,"15min":4.3090000000000002},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.745,"15min":0.76900000000000002},"min":{"1min":0.496,"5min":0.437,"15min":0.437},"max":{"1min":0.77200000000000002,"5min":4.0709999999999997,"15min":4.0709999999999997},"last":0.504}]},{"osd":6,"last update":"Wed Nov 8 19:26:46 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.66500000000000004,"15min":0.67300000000000004},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.82299999999999995,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.621},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.80900000000000005,"15min":0.83799999999999997},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.76000000000000001,"5min":4.2560000000000002,"15min":4.2560000000000002},"last":0.60799999999999998}]},{"osd":7,"last update":"Wed Nov 8 19:26:46 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.76600000000000001,"15min":0.79000000000000004},"min":{"1min":0.49299999999999999,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.81299999999999994,"5min":4.2850000000000001,"15min":4.2850000000000001},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.79200000000000004,"15min":0.81799999999999995},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.85699999999999998,"5min":4.0460000000000003,"15min":4.0460000000000003},"last":0.58599999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574886,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27944,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343896,"statfs":{"total":96636764160,"available":96608149504,"internally_reserved":0,"allocated":937984,"data_stored":429817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:26:46 2023","interfaces":[{"interface":"back","average":{"1min":0.34300000000000003,"5min":0.33400000000000002,"15min":0.33200000000000002},"min":{"1min":0.254,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.496,"5min":0.496,"15min":0.496},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.309,"5min":0.58899999999999997,"15min":0.63600000000000001},"min":{"1min":0.23400000000000001,"5min":0.218,"15min":0.218},"max":{"1min":0.45600000000000002,"5min":6.0999999999999996,"15min":6.0999999999999996},"last":0.36099999999999999}]},{"osd":1,"last update":"Wed Nov 8 19:26:46 2023","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.44,"5min":0.48299999999999998,"15min":0.48299999999999998},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.38600000000000001,"15min":0.38900000000000001},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.48899999999999999,"5min":0.624,"15min":0.624},"last":0.41899999999999998}]},{"osd":3,"last update":"Wed Nov 8 19:26:46 2023","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.38100000000000001,"15min":0.38400000000000001},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.46000000000000002,"5min":0.53700000000000003,"15min":0.53700000000000003},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.39000000000000001,"15min":0.39400000000000002},"min":{"1min":0.29399999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.439,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.44700000000000001}]},{"osd":4,"last update":"Wed Nov 8 19:26:46 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.57899999999999996,"15min":0.58299999999999996},"min":{"1min":0.46000000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.71999999999999997,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.57999999999999996,"15min":0.58299999999999996},"min":{"1min":0.40999999999999998,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.74199999999999999,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.68700000000000006}]},{"osd":5,"last update":"Wed Nov 8 19:26:46 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.58599999999999997,"15min":0.58299999999999996},"min":{"1min":0.50700000000000001,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.76400000000000001,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.60899999999999999,"15min":0.61199999999999999},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.76100000000000001,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.76400000000000001}]},{"osd":6,"last update":"Wed Nov 8 19:26:46 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.59799999999999998,"15min":0.59999999999999998},"min":{"1min":0.47699999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.77300000000000002,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.40300000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.70699999999999996,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.80000000000000004}]},{"osd":7,"last update":"Wed Nov 8 19:26:46 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.58199999999999996,"15min":0.57999999999999996},"min":{"1min":0.497,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.81499999999999995,"5min":0.878,"15min":0.878},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.60099999999999998,"15min":0.60299999999999998},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.80000000000000004,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.81799999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574887,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27944,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343896,"statfs":{"total":96636764160,"available":96608149504,"internally_reserved":0,"allocated":937984,"data_stored":429817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:26:49 2023","interfaces":[{"interface":"back","average":{"1min":0.32100000000000001,"5min":0.33800000000000002,"15min":0.34100000000000003},"min":{"1min":0.20300000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.42299999999999999,"5min":0.626,"15min":0.626},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.34999999999999998,"15min":0.34899999999999998},"min":{"1min":0.28100000000000003,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.46899999999999997,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.28799999999999998}]},{"osd":1,"last update":"Wed Nov 8 19:26:49 2023","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.35099999999999998,"15min":0.35299999999999998},"min":{"1min":0.24099999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.42699999999999999,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":0.25900000000000001},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"min":{"1min":0.27900000000000003,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.49199999999999999,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.39400000000000002}]},{"osd":2,"last update":"Wed Nov 8 19:26:49 2023","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.374,"15min":0.371},"min":{"1min":0.30199999999999999,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.48199999999999998,"5min":0.58499999999999996,"15min":0.58499999999999996},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.40500000000000003,"15min":0.40400000000000003},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"last":0.34499999999999997}]},{"osd":4,"last update":"Wed Nov 8 19:26:49 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.60399999999999998,"15min":0.59999999999999998},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.80200000000000005,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.624,"5min":0.60799999999999998,"15min":0.60599999999999998},"min":{"1min":0.45000000000000001,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.79000000000000004,"5min":0.871,"15min":0.871},"last":0.63900000000000001}]},{"osd":5,"last update":"Wed Nov 8 19:26:49 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.58299999999999996,"15min":0.57699999999999996},"min":{"1min":0.39800000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.625,"5min":0.58599999999999997,"15min":0.57999999999999996},"min":{"1min":0.499,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.81999999999999995,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.54100000000000004}]},{"osd":6,"last update":"Wed Nov 8 19:26:49 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.58399999999999996,"15min":0.58099999999999996},"min":{"1min":0.42999999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.61299999999999999,"15min":0.60699999999999998},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.84099999999999997,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.628}]},{"osd":7,"last update":"Wed Nov 8 19:26:49 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.59599999999999997,"15min":0.58999999999999997},"min":{"1min":0.46400000000000002,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.88100000000000001,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.59799999999999998,"15min":0.59599999999999997},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.76600000000000001,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.55300000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574886,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27952,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343888,"statfs":{"total":96636764160,"available":96608141312,"internally_reserved":0,"allocated":946176,"data_stored":429977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:26:50 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.73099999999999998,"15min":0.76000000000000001},"min":{"1min":0.40999999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.78100000000000003,"5min":6.5970000000000004,"15min":6.5970000000000004},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.56599999999999995,"15min":0.57399999999999995},"min":{"1min":0.378,"5min":0.317,"15min":0.317},"max":{"1min":0.75900000000000001,"5min":2.5030000000000001,"15min":2.5030000000000001},"last":0.60399999999999998}]},{"osd":1,"last update":"Wed Nov 8 19:26:50 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.50700000000000001,"15min":0.499},"min":{"1min":0.372,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.51600000000000001,"15min":0.51000000000000001},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.57599999999999996}]},{"osd":2,"last update":"Wed Nov 8 19:26:50 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.55500000000000005,"15min":0.55200000000000005},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.623,"5min":0.80100000000000005,"15min":0.83099999999999996},"min":{"1min":0.45900000000000002,"5min":0.435,"15min":0.435},"max":{"1min":0.871,"5min":6.6920000000000002,"15min":6.6920000000000002},"last":0.44500000000000001}]},{"osd":3,"last update":"Wed Nov 8 19:26:50 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58799999999999997,"15min":0.58899999999999997},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.71099999999999997,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.627,"5min":0.61399999999999999,"15min":0.61199999999999999},"min":{"1min":0.46800000000000003,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.627}]},{"osd":5,"last update":"Wed Nov 8 19:26:50 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.34300000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.59099999999999997,"15min":0.59499999999999997},"min":{"1min":0.42999999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.876,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.67100000000000004}]},{"osd":6,"last update":"Wed Nov 8 19:26:50 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.61799999999999999,"15min":0.61499999999999999},"min":{"1min":0.38900000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.78100000000000003,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.69599999999999995}]},{"osd":7,"last update":"Wed Nov 8 19:26:50 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.625,"15min":0.63200000000000001},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.82699999999999996,"5min":1.141,"15min":1.141},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.63900000000000001,"15min":0.64500000000000002},"min":{"1min":0.38700000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.85299999999999998,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.71099999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574886,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27944,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343896,"statfs":{"total":96636764160,"available":96608149504,"internally_reserved":0,"allocated":937984,"data_stored":429817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:26:44 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.51700000000000002,"15min":0.52100000000000002},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.51100000000000001,"15min":0.50900000000000001},"min":{"1min":0.373,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.65100000000000002,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.38500000000000001}]},{"osd":1,"last update":"Wed Nov 8 19:26:44 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.52200000000000002,"15min":0.52300000000000002},"min":{"1min":0.42099999999999999,"5min":0.371,"15min":0.371},"max":{"1min":0.65300000000000002,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53500000000000003,"15min":0.53600000000000003},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.46000000000000002}]},{"osd":2,"last update":"Wed Nov 8 19:26:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.53800000000000003,"15min":0.53600000000000003},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.441},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.56599999999999995,"15min":0.57299999999999995},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.69599999999999995,"5min":0.747,"15min":0.747},"last":0.503}]},{"osd":3,"last update":"Wed Nov 8 19:26:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.54600000000000004,"15min":0.54900000000000004},"min":{"1min":0.39700000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.59099999999999997,"15min":0.59999999999999998},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.70099999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.56299999999999994}]},{"osd":4,"last update":"Wed Nov 8 19:26:44 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.55100000000000005,"15min":0.55900000000000005},"min":{"1min":0.38600000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.63300000000000001,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.39100000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.71399999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.54900000000000004}]},{"osd":6,"last update":"Wed Nov 8 19:26:44 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.60899999999999999,"15min":0.61199999999999999},"min":{"1min":0.42099999999999999,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.61099999999999999,"15min":0.61499999999999999},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.67700000000000005,"5min":0.745,"15min":0.745},"last":0.52800000000000002}]},{"osd":7,"last update":"Wed Nov 8 19:26:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.60299999999999998,"15min":0.60899999999999999},"min":{"1min":0.371,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.63900000000000001,"15min":0.64700000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.77100000000000002,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.63400000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-08T19:27:36.820 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-08T19:27:36.821 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-08T19:27:36.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.976+0000 7fe9b6653700 1 Processor -- start 2023-11-08T19:27:36.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.976+0000 7fe9b6653700 1 -- start start 2023-11-08T19:27:36.980 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.977+0000 7fe9b6653700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe9b00cc8c0 0x7fe9b00ccd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:36.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.977+0000 7fe9b6653700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fe9b00c5880 0x7fe9b00c5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:36.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.977+0000 7fe9b6653700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9b00c61d0 con 0x7fe9b0143a30 2023-11-08T19:27:36.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.977+0000 7fe9b6653700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9b015e110 con 0x7fe9b00cc8c0 2023-11-08T19:27:36.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.977+0000 7fe9b6653700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9b00cd290 con 0x7fe9b00c5880 2023-11-08T19:27:36.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.977+0000 7fe9af7fe700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe9b00cc8c0 0x7fe9b00ccd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:36.982 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.977+0000 7fe9affff700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fe9b0143a30 0x7fe9b006bfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:49870/0 (socket says 172.21.15.143:49870) 2023-11-08T19:27:36.982 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.977+0000 7fe9affff700 1 -- 172.21.15.143:0/291728169 learned_addr learned my addr 172.21.15.143:0/291728169 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:27:36.982 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.977+0000 7fe9b4bf0700 1 --2- 172.21.15.143:0/291728169 >> v2:172.21.15.174:3300/0 conn(0x7fe9b00c5880 0x7fe9b00c5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:36.982 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.978+0000 7fe9af7fe700 1 -- 172.21.15.143:0/291728169 >> v2:172.21.15.174:3300/0 conn(0x7fe9b00c5880 msgr2=0x7fe9b00c5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:36.982 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.978+0000 7fe9af7fe700 1 --2- 172.21.15.143:0/291728169 >> v2:172.21.15.174:3300/0 conn(0x7fe9b00c5880 0x7fe9b00c5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:36.983 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.978+0000 7fe9af7fe700 1 -- 172.21.15.143:0/291728169 >> v1:172.21.15.143:6789/0 conn(0x7fe9b0143a30 legacy=0x7fe9b006bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:36.983 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.978+0000 7fe9af7fe700 1 -- 172.21.15.143:0/291728169 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9b0143e10 con 0x7fe9b00cc8c0 2023-11-08T19:27:36.983 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.978+0000 7fe9af7fe700 1 --2- 172.21.15.143:0/291728169 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe9b00cc8c0 0x7fe9b00ccd50 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fe9b00afb90 tx=0x7fe9a000bdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4d12ccdae67d8bf server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:36.983 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.978+0000 7fe9aeffd700 1 -- 172.21.15.143:0/291728169 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a000dcf0 con 0x7fe9b00cc8c0 2023-11-08T19:27:36.983 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.978+0000 7fe9aeffd700 1 -- 172.21.15.143:0/291728169 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9a000de50 con 0x7fe9b00cc8c0 2023-11-08T19:27:36.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.978+0000 7fe9aeffd700 1 -- 172.21.15.143:0/291728169 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a0010690 con 0x7fe9b00cc8c0 2023-11-08T19:27:36.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.978+0000 7fe9b6653700 1 -- 172.21.15.143:0/291728169 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe9b00cc8c0 msgr2=0x7fe9b00ccd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:36.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.978+0000 7fe9b6653700 1 --2- 172.21.15.143:0/291728169 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe9b00cc8c0 0x7fe9b00ccd50 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fe9b00afb90 tx=0x7fe9a000bdf0 comp rx=0 tx=0).stop 2023-11-08T19:27:36.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.978+0000 7fe9b6653700 1 -- 172.21.15.143:0/291728169 shutdown_connections 2023-11-08T19:27:36.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.978+0000 7fe9b6653700 1 --2- 172.21.15.143:0/291728169 >> v2:172.21.15.174:3300/0 conn(0x7fe9b00c5880 0x7fe9b00c5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:36.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.978+0000 7fe9b6653700 1 --2- 172.21.15.143:0/291728169 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe9b00cc8c0 0x7fe9b00ccd50 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:36.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.978+0000 7fe9b6653700 1 -- 172.21.15.143:0/291728169 >> 172.21.15.143:0/291728169 conn(0x7fe9b00b1c70 msgr2=0x7fe9b00b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:36.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.979+0000 7fe9b6653700 1 -- 172.21.15.143:0/291728169 shutdown_connections 2023-11-08T19:27:36.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.979+0000 7fe9b6653700 1 -- 172.21.15.143:0/291728169 wait complete. 2023-11-08T19:27:36.988 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.980+0000 7fe9b6653700 1 Processor -- start 2023-11-08T19:27:36.988 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.980+0000 7fe9b6653700 1 -- start start 2023-11-08T19:27:36.988 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.981+0000 7fe9b6653700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe9b00c5880 0x7fe9b015ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:36.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.981+0000 7fe9b6653700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe9b00cc8c0 0x7fe9b015a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:36.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.981+0000 7fe9b6653700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe9b0143a30 0x7fe9b00b50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:36.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.981+0000 7fe9b6653700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9b015bae0 con 0x7fe9b00c5880 2023-11-08T19:27:36.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.981+0000 7fe9b6653700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9b015b240 con 0x7fe9b0143a30 2023-11-08T19:27:36.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.981+0000 7fe9b6653700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9b015b520 con 0x7fe9b00cc8c0 2023-11-08T19:27:36.990 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.981+0000 7fe9affff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe9b00c5880 0x7fe9b015ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:36.990 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.981+0000 7fe9affff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe9b00c5880 0x7fe9b015ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53722/0 (socket says 172.21.15.143:53722) 2023-11-08T19:27:36.990 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.981+0000 7fe9affff700 1 -- 172.21.15.143:0/742531237 learned_addr learned my addr 172.21.15.143:0/742531237 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:36.990 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.981+0000 7fe9b4bf0700 1 --2- 172.21.15.143:0/742531237 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe9b0143a30 0x7fe9b00b50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:36.990 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.981+0000 7fe9af7fe700 1 --2- 172.21.15.143:0/742531237 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe9b00cc8c0 0x7fe9b015a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:36.991 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.981+0000 7fe9affff700 1 -- 172.21.15.143:0/742531237 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe9b00cc8c0 msgr2=0x7fe9b015a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:36.991 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.981+0000 7fe9affff700 1 --2- 172.21.15.143:0/742531237 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe9b00cc8c0 0x7fe9b015a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:36.991 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.981+0000 7fe9affff700 1 -- 172.21.15.143:0/742531237 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe9b0143a30 msgr2=0x7fe9b00b50e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:36.991 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.981+0000 7fe9affff700 1 --2- 172.21.15.143:0/742531237 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe9b0143a30 0x7fe9b00b50e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:36.991 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.981+0000 7fe9affff700 1 -- 172.21.15.143:0/742531237 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9a000b740 con 0x7fe9b00c5880 2023-11-08T19:27:36.992 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.981+0000 7fe9af7fe700 1 --2- 172.21.15.143:0/742531237 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe9b00cc8c0 0x7fe9b015a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:27:36.992 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.981+0000 7fe9affff700 1 --2- 172.21.15.143:0/742531237 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe9b00c5880 0x7fe9b015ca10 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fe9a400c520 tx=0x7fe9a400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:36.992 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.981+0000 7fe9acff9700 1 -- 172.21.15.143:0/742531237 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a4005d70 con 0x7fe9b00c5880 2023-11-08T19:27:36.992 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.981+0000 7fe9acff9700 1 -- 172.21.15.143:0/742531237 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9a40070d0 con 0x7fe9b00c5880 2023-11-08T19:27:36.992 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.981+0000 7fe9b6653700 1 -- 172.21.15.143:0/742531237 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9b015b800 con 0x7fe9b00c5880 2023-11-08T19:27:36.993 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.981+0000 7fe9acff9700 1 -- 172.21.15.143:0/742531237 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a4007440 con 0x7fe9b00c5880 2023-11-08T19:27:36.993 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.981+0000 7fe9b6653700 1 -- 172.21.15.143:0/742531237 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9b015cf50 con 0x7fe9b00c5880 2023-11-08T19:27:36.993 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.982+0000 7fe9b6653700 1 -- 172.21.15.143:0/742531237 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9b00b3040 con 0x7fe9b00c5880 2023-11-08T19:27:36.993 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.983+0000 7fe9acff9700 1 -- 172.21.15.143:0/742531237 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe9a40075a0 con 0x7fe9b00c5880 2023-11-08T19:27:36.993 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.983+0000 7fe9acff9700 1 --2- 172.21.15.143:0/742531237 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fe99c06af00 0x7fe99c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:36.994 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.983+0000 7fe9acff9700 1 -- 172.21.15.143:0/742531237 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6772+0+0 (secure 0 0 0) 0x7fe9a4088e40 con 0x7fe9b00c5880 2023-11-08T19:27:36.994 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.983+0000 7fe9af7fe700 1 --2- 172.21.15.143:0/742531237 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fe99c06af00 0x7fe99c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:36.994 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.984+0000 7fe9af7fe700 1 --2- 172.21.15.143:0/742531237 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fe99c06af00 0x7fe99c06d3c0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fe9b015bf20 tx=0x7fe9a0013000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:36.994 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:36.988+0000 7fe9acff9700 1 -- 172.21.15.143:0/742531237 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe9b00b3040 con 0x7fe9b00c5880 2023-11-08T19:27:37.254 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:37.252+0000 7fe9b6653700 1 -- 172.21.15.143:0/742531237 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fe9b00b3040 con 0x7fe9b00c5880 2023-11-08T19:27:37.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:37.254+0000 7fe9acff9700 1 -- 172.21.15.143:0/742531237 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v12) v1 ==== 190+0+147 (secure 0 0 0) 0x7fe9b00b3040 con 0x7fe9b00c5880 2023-11-08T19:27:37.256 INFO:teuthology.orchestra.run.smithi143.stdout:[client.0] 2023-11-08T19:27:37.256 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQBP4Etl0GdSFxAAVkuHL0EnjrtTUMkj/afPkg== 2023-11-08T19:27:37.256 INFO:teuthology.orchestra.run.smithi143.stdout: caps mds = "allow" 2023-11-08T19:27:37.257 INFO:teuthology.orchestra.run.smithi143.stdout: caps mon = "allow r" 2023-11-08T19:27:37.257 INFO:teuthology.orchestra.run.smithi143.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-11-08T19:27:37.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:37.257+0000 7fe9b6653700 1 -- 172.21.15.143:0/742531237 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fe99c06af00 msgr2=0x7fe99c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:37.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:37.257+0000 7fe9b6653700 1 --2- 172.21.15.143:0/742531237 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fe99c06af00 0x7fe99c06d3c0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fe9b015bf20 tx=0x7fe9a0013000 comp rx=0 tx=0).stop 2023-11-08T19:27:37.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:37.257+0000 7fe9b6653700 1 -- 172.21.15.143:0/742531237 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe9b00c5880 msgr2=0x7fe9b015ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:37.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:37.257+0000 7fe9b6653700 1 --2- 172.21.15.143:0/742531237 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe9b00c5880 0x7fe9b015ca10 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fe9a400c520 tx=0x7fe9a400c8e0 comp rx=0 tx=0).stop 2023-11-08T19:27:37.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:37.257+0000 7fe9b6653700 1 -- 172.21.15.143:0/742531237 shutdown_connections 2023-11-08T19:27:37.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:37.257+0000 7fe9b6653700 1 --2- 172.21.15.143:0/742531237 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fe99c06af00 0x7fe99c06d3c0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:37.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:37.257+0000 7fe9b6653700 1 --2- 172.21.15.143:0/742531237 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe9b0143a30 0x7fe9b00b50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:37.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:37.257+0000 7fe9b6653700 1 --2- 172.21.15.143:0/742531237 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe9b00c5880 0x7fe9b015ca10 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:37.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:37.257+0000 7fe9b6653700 1 --2- 172.21.15.143:0/742531237 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe9b00cc8c0 0x7fe9b015a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:37.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:37.257+0000 7fe9b6653700 1 -- 172.21.15.143:0/742531237 >> 172.21.15.143:0/742531237 conn(0x7fe9b00b1c70 msgr2=0x7fe9b00b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:37.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:37.257+0000 7fe9b6653700 1 -- 172.21.15.143:0/742531237 shutdown_connections 2023-11-08T19:27:37.261 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:37.257+0000 7fe9b6653700 1 -- 172.21.15.143:0/742531237 wait complete. 2023-11-08T19:27:37.261 INFO:teuthology.orchestra.run.smithi143.stderr:updated caps for client.0 2023-11-08T19:27:37.270 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:27:37.393 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:37.392+0000 7f3c1fec8700 1 Processor -- start 2023-11-08T19:27:37.394 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:37.392+0000 7f3c1fec8700 1 -- start start 2023-11-08T19:27:37.396 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:37.393+0000 7f3c1fec8700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f3c1806dbb0 0x7f3c180ca470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:37.396 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:37.393+0000 7f3c1fec8700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3c180668e0 0x7f3c180ca9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:37.396 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:37.393+0000 7f3c1fec8700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c180ce7f0 con 0x7f3c1806df90 2023-11-08T19:27:37.397 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:37.393+0000 7f3c1fec8700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c18066db0 con 0x7f3c1806dbb0 2023-11-08T19:27:37.397 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:37.393+0000 7f3c1fec8700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c180af5e0 con 0x7f3c180668e0 2023-11-08T19:27:37.397 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:37.393+0000 7f3c1dc64700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f3c1806dbb0 0x7f3c180ca470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:37.397 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:37.393+0000 7f3c1dc64700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f3c1806dbb0 0x7f3c180ca470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53730/0 (socket says 172.21.15.143:53730) 2023-11-08T19:27:37.397 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:37.393+0000 7f3c1dc64700 1 -- 172.21.15.143:0/2364512983 learned_addr learned my addr 172.21.15.143:0/2364512983 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:37.398 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:37.393+0000 7f3c1dc64700 1 -- 172.21.15.143:0/2364512983 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3c180668e0 msgr2=0x7f3c180ca9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:37.398 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:37.393+0000 7f3c1dc64700 1 --2- 172.21.15.143:0/2364512983 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3c180668e0 0x7f3c180ca9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:37.398 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:37.393+0000 7f3c1dc64700 1 -- 172.21.15.143:0/2364512983 >> v1:172.21.15.174:6789/0 conn(0x7f3c1806df90 legacy=0x7f3c181621f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:37.398 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:37.393+0000 7f3c1dc64700 1 -- 172.21.15.143:0/2364512983 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c180cb520 con 0x7f3c1806dbb0 2023-11-08T19:27:37.398 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:37.394+0000 7f3c1dc64700 1 --2- 172.21.15.143:0/2364512983 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3c1806dbb0 0x7f3c180ca470 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f3c080098f0 tx=0x7f3c08009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3604e72b9fb6cfc8 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:37.399 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:37.394+0000 7f3c1cc62700 1 -- 172.21.15.143:0/2364512983 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c08004230 con 0x7f3c1806dbb0 2023-11-08T19:27:37.399 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:37.394+0000 7f3c1cc62700 1 -- 172.21.15.143:0/2364512983 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c08004390 con 0x7f3c1806dbb0 2023-11-08T19:27:37.399 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:37.394+0000 7f3c1cc62700 1 -- 172.21.15.143:0/2364512983 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c080036a0 con 0x7f3c1806dbb0 2023-11-08T19:27:37.399 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:37.394+0000 7f3c1fec8700 1 -- 172.21.15.143:0/2364512983 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3c1806dbb0 msgr2=0x7f3c180ca470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:37.399 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:37.394+0000 7f3c1fec8700 1 --2- 172.21.15.143:0/2364512983 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3c1806dbb0 0x7f3c180ca470 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f3c080098f0 tx=0x7f3c08009c00 comp rx=0 tx=0).stop 2023-11-08T19:27:37.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:37.394+0000 7f3c1fec8700 1 -- 172.21.15.143:0/2364512983 shutdown_connections 2023-11-08T19:27:37.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:37.394+0000 7f3c1fec8700 1 --2- 172.21.15.143:0/2364512983 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3c1806dbb0 0x7f3c180ca470 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:37.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:37.394+0000 7f3c1fec8700 1 --2- 172.21.15.143:0/2364512983 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3c180668e0 0x7f3c180ca9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:37.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:37.394+0000 7f3c1fec8700 1 -- 172.21.15.143:0/2364512983 >> 172.21.15.143:0/2364512983 conn(0x7f3c180b1a20 msgr2=0x7f3c180b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:37.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:37.394+0000 7f3c1fec8700 1 -- 172.21.15.143:0/2364512983 shutdown_connections 2023-11-08T19:27:37.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:37.394+0000 7f3c1fec8700 1 -- 172.21.15.143:0/2364512983 wait complete. 2023-11-08T19:27:37.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:37.396+0000 7f3c1fec8700 1 Processor -- start 2023-11-08T19:27:37.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:37.396+0000 7f3c1fec8700 1 -- start start 2023-11-08T19:27:37.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:37.396+0000 7f3c1fec8700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3c180668e0 0x7f3c180ae400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:37.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:37.396+0000 7f3c1fec8700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3c1806dbb0 0x7f3c180c6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:37.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:37.396+0000 7f3c1fec8700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c1806df90 0x7f3c180c7530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:37.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:37.396+0000 7f3c1fec8700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c181f7130 con 0x7f3c180668e0 2023-11-08T19:27:37.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:37.396+0000 7f3c1fec8700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c181f73d0 con 0x7f3c1806df90 2023-11-08T19:27:37.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:37.396+0000 7f3c1fec8700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c181f76b0 con 0x7f3c1806dbb0 2023-11-08T19:27:37.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:37.396+0000 7f3c1dc64700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3c180668e0 0x7f3c180ae400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:37.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:37.396+0000 7f3c1dc64700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3c180668e0 0x7f3c180ae400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53736/0 (socket says 172.21.15.143:53736) 2023-11-08T19:27:37.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:37.396+0000 7f3c1dc64700 1 -- 172.21.15.143:0/3836713364 learned_addr learned my addr 172.21.15.143:0/3836713364 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:37.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:37.397+0000 7f3c1dc64700 1 -- 172.21.15.143:0/3836713364 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3c1806dbb0 msgr2=0x7f3c180c6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:37.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:37.397+0000 7f3c1d463700 1 --2- 172.21.15.143:0/3836713364 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3c1806dbb0 0x7f3c180c6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:37.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:37.397+0000 7f3c1e465700 1 --2- 172.21.15.143:0/3836713364 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c1806df90 0x7f3c180c7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:37.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:37.397+0000 7f3c1dc64700 1 --2- 172.21.15.143:0/3836713364 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3c1806dbb0 0x7f3c180c6ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:37.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:37.397+0000 7f3c1dc64700 1 -- 172.21.15.143:0/3836713364 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c1806df90 msgr2=0x7f3c180c7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:37.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:37.397+0000 7f3c1dc64700 1 --2- 172.21.15.143:0/3836713364 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c1806df90 0x7f3c180c7530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:37.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:37.397+0000 7f3c1dc64700 1 -- 172.21.15.143:0/3836713364 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c08009550 con 0x7f3c180668e0 2023-11-08T19:27:37.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:37.397+0000 7f3c1dc64700 1 --2- 172.21.15.143:0/3836713364 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3c180668e0 0x7f3c180ae400 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f3c08009130 tx=0x7f3c08004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:37.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:37.397+0000 7f3c0e7fc700 1 -- 172.21.15.143:0/3836713364 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c08003e70 con 0x7f3c180668e0 2023-11-08T19:27:37.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:37.397+0000 7f3c0e7fc700 1 -- 172.21.15.143:0/3836713364 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c0801f070 con 0x7f3c180668e0 2023-11-08T19:27:37.409 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:37.397+0000 7f3c1fec8700 1 -- 172.21.15.143:0/3836713364 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c181f7990 con 0x7f3c180668e0 2023-11-08T19:27:37.409 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:37.397+0000 7f3c0e7fc700 1 -- 172.21.15.143:0/3836713364 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c0800b8a0 con 0x7f3c180668e0 2023-11-08T19:27:37.409 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:37.397+0000 7f3c1fec8700 1 -- 172.21.15.143:0/3836713364 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c180ae940 con 0x7f3c180668e0 2023-11-08T19:27:37.409 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:37.398+0000 7f3c1fec8700 1 -- 172.21.15.143:0/3836713364 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c1806d1a0 con 0x7f3c180668e0 2023-11-08T19:27:37.409 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:37.399+0000 7f3c0e7fc700 1 -- 172.21.15.143:0/3836713364 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3c08021030 con 0x7f3c180668e0 2023-11-08T19:27:37.410 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:37.399+0000 7f3c0e7fc700 1 --2- 172.21.15.143:0/3836713364 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f3bfc06af00 0x7f3bfc06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:37.410 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:37.399+0000 7f3c0e7fc700 1 -- 172.21.15.143:0/3836713364 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6772+0+0 (secure 0 0 0) 0x7f3c0808b110 con 0x7f3c180668e0 2023-11-08T19:27:37.410 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:37.399+0000 7f3c1d463700 1 --2- 172.21.15.143:0/3836713364 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f3bfc06af00 0x7f3bfc06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:37.410 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:37.400+0000 7f3c1d463700 1 --2- 172.21.15.143:0/3836713364 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f3bfc06af00 0x7f3bfc06d3c0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f3c14006fd0 tx=0x7f3c14008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:37.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:37.403+0000 7f3c0e7fc700 1 -- 172.21.15.143:0/3836713364 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3c08055e20 con 0x7f3c180668e0 2023-11-08T19:27:37.641 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:37.640+0000 7f3c1fec8700 1 -- 172.21.15.143:0/3836713364 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3c1806d1a0 con 0x7f3c180668e0 2023-11-08T19:27:37.642 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:37.641+0000 7f3c0e7fc700 1 -- 172.21.15.143:0/3836713364 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+4921 (secure 0 0 0) 0x7f3c08059440 con 0x7f3c180668e0 2023-11-08T19:27:37.642 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:27:37.643 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":41,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5510,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6836/3796015890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3796015890},{"type":"v1","addr":"172.21.15.174:6836","nonce":3796015890}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37},{"gid":6471,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6834/432456394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":432456394},{"type":"v1","addr":"172.21.15.143:6834","nonce":432456394}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37},{"gid":6474,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/310920972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":310920972},{"type":"v1","addr":"172.21.15.143:6835","nonce":310920972}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T19:27:31.702539+0000","modified":"2023-11-08T19:27:34.725623+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6489},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6489":{"gid":6489,"name":"d","rank":0,"incarnation":38,"state":"up:active","state_seq":3,"addr":"172.21.15.174:6837/1950356087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6835","nonce":1950356087},{"type":"v1","addr":"172.21.15.174:6837","nonce":1950356087}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-08T19:27:37.645 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:37.643+0000 7f3c1fec8700 1 -- 172.21.15.143:0/3836713364 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f3bfc06af00 msgr2=0x7f3bfc06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:37.645 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:37.643+0000 7f3c1fec8700 1 --2- 172.21.15.143:0/3836713364 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f3bfc06af00 0x7f3bfc06d3c0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f3c14006fd0 tx=0x7f3c14008040 comp rx=0 tx=0).stop 2023-11-08T19:27:37.645 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:37.643+0000 7f3c1fec8700 1 -- 172.21.15.143:0/3836713364 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3c180668e0 msgr2=0x7f3c180ae400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:37.645 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:37.643+0000 7f3c1fec8700 1 --2- 172.21.15.143:0/3836713364 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3c180668e0 0x7f3c180ae400 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f3c08009130 tx=0x7f3c08004750 comp rx=0 tx=0).stop 2023-11-08T19:27:37.646 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:37.643+0000 7f3c1fec8700 1 -- 172.21.15.143:0/3836713364 shutdown_connections 2023-11-08T19:27:37.646 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:37.643+0000 7f3c1fec8700 1 --2- 172.21.15.143:0/3836713364 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f3bfc06af00 0x7f3bfc06d3c0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:37.646 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:37.643+0000 7f3c1fec8700 1 --2- 172.21.15.143:0/3836713364 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c1806df90 0x7f3c180c7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:37.646 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:37.643+0000 7f3c1fec8700 1 --2- 172.21.15.143:0/3836713364 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3c180668e0 0x7f3c180ae400 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:37.646 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:37.643+0000 7f3c1fec8700 1 --2- 172.21.15.143:0/3836713364 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3c1806dbb0 0x7f3c180c6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:37.647 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:37.643+0000 7f3c1fec8700 1 -- 172.21.15.143:0/3836713364 >> 172.21.15.143:0/3836713364 conn(0x7f3c180b1a20 msgr2=0x7f3c1806e470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:37.647 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:37.643+0000 7f3c1fec8700 1 -- 172.21.15.143:0/3836713364 shutdown_connections 2023-11-08T19:27:37.647 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:37.643+0000 7f3c1fec8700 1 -- 172.21.15.143:0/3836713364 wait complete. 2023-11-08T19:27:37.647 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 41 2023-11-08T19:27:38.059 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 41, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-08T19:27:31.702539+0000', 'modified': '2023-11-08T19:27:34.725623+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6489}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6489': {'gid': 6489, 'name': 'd', 'rank': 0, 'incarnation': 38, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.174:6837/1950356087', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.174:6835', 'nonce': 1950356087}, {'type': 'v1', 'addr': '172.21.15.174:6837', 'nonce': 1950356087}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-08T19:27:38.059 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-08T19:27:38.060 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-11-08T19:27:38.183 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.182+0000 7f5ae3083700 1 Processor -- start 2023-11-08T19:27:38.183 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.182+0000 7f5ae3083700 1 -- start start 2023-11-08T19:27:38.185 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.182+0000 7f5ae3083700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f5adc06dbb0 0x7f5adc15fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:38.186 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.183+0000 7f5ae3083700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f5adc06e9a0 0x7f5adc0b3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:38.186 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.183+0000 7f5ae3083700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5adc06bd50 con 0x7f5adc072860 2023-11-08T19:27:38.186 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.183+0000 7f5ae3083700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5adc0cb1c0 con 0x7f5adc06e9a0 2023-11-08T19:27:38.186 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.183+0000 7f5ae3083700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5adc06bff0 con 0x7f5adc06dbb0 2023-11-08T19:27:38.187 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.183+0000 7f5ae1620700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f5adc06e9a0 0x7f5adc0b3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:38.187 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.183+0000 7f5ae1620700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f5adc06e9a0 0x7f5adc0b3240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53748/0 (socket says 172.21.15.143:53748) 2023-11-08T19:27:38.187 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.183+0000 7f5ae1620700 1 -- 172.21.15.143:0/2014922601 learned_addr learned my addr 172.21.15.143:0/2014922601 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:38.187 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.183+0000 7f5ae0e1f700 1 --2- 172.21.15.143:0/2014922601 >> v2:172.21.15.174:3300/0 conn(0x7f5adc06dbb0 0x7f5adc15fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:38.187 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.183+0000 7f5ae1620700 1 -- 172.21.15.143:0/2014922601 >> v2:172.21.15.174:3300/0 conn(0x7f5adc06dbb0 msgr2=0x7f5adc15fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:38.189 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.183+0000 7f5ae1620700 1 --2- 172.21.15.143:0/2014922601 >> v2:172.21.15.174:3300/0 conn(0x7f5adc06dbb0 0x7f5adc15fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:38.189 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.183+0000 7f5ae1620700 1 -- 172.21.15.143:0/2014922601 >> v1:172.21.15.174:6789/0 conn(0x7f5adc072860 legacy=0x7f5adc15ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:38.189 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.183+0000 7f5ae1620700 1 -- 172.21.15.143:0/2014922601 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5adc06c2d0 con 0x7f5adc06e9a0 2023-11-08T19:27:38.190 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.183+0000 7f5ae1620700 1 --2- 172.21.15.143:0/2014922601 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5adc06e9a0 0x7f5adc0b3240 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f5adc062890 tx=0x7f5ad400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b53103ea752ee855 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:38.190 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.183+0000 7f5adb7fe700 1 -- 172.21.15.143:0/2014922601 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad400e070 con 0x7f5adc06e9a0 2023-11-08T19:27:38.190 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.183+0000 7f5adb7fe700 1 -- 172.21.15.143:0/2014922601 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ad4003a20 con 0x7f5adc06e9a0 2023-11-08T19:27:38.190 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.183+0000 7f5adb7fe700 1 -- 172.21.15.143:0/2014922601 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad4003dd0 con 0x7f5adc06e9a0 2023-11-08T19:27:38.190 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.184+0000 7f5ae3083700 1 -- 172.21.15.143:0/2014922601 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5adc06e9a0 msgr2=0x7f5adc0b3240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:38.191 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.184+0000 7f5ae3083700 1 --2- 172.21.15.143:0/2014922601 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5adc06e9a0 0x7f5adc0b3240 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f5adc062890 tx=0x7f5ad400b330 comp rx=0 tx=0).stop 2023-11-08T19:27:38.191 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.184+0000 7f5ae3083700 1 -- 172.21.15.143:0/2014922601 shutdown_connections 2023-11-08T19:27:38.191 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.184+0000 7f5ae3083700 1 --2- 172.21.15.143:0/2014922601 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5adc06e9a0 0x7f5adc0b3240 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:38.191 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.184+0000 7f5ae3083700 1 --2- 172.21.15.143:0/2014922601 >> v2:172.21.15.174:3300/0 conn(0x7f5adc06dbb0 0x7f5adc15fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:38.191 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.184+0000 7f5ae3083700 1 -- 172.21.15.143:0/2014922601 >> 172.21.15.143:0/2014922601 conn(0x7f5adc0b1a20 msgr2=0x7f5adc0b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:38.192 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.184+0000 7f5ae3083700 1 -- 172.21.15.143:0/2014922601 shutdown_connections 2023-11-08T19:27:38.192 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.184+0000 7f5ae3083700 1 -- 172.21.15.143:0/2014922601 wait complete. 2023-11-08T19:27:38.201 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.185+0000 7f5ae3083700 1 Processor -- start 2023-11-08T19:27:38.201 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.185+0000 7f5ae3083700 1 -- start start 2023-11-08T19:27:38.201 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.186+0000 7f5ae3083700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5adc06dbb0 0x7f5adc156850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:38.202 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.186+0000 7f5ae3083700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5adc06e9a0 0x7f5adc1575d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:38.202 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.186+0000 7f5ae3083700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5adc072860 0x7f5adc0c82b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:38.202 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.186+0000 7f5ae3083700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5adc156d90 con 0x7f5adc06e9a0 2023-11-08T19:27:38.202 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.186+0000 7f5ae3083700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5adc0c7530 con 0x7f5adc072860 2023-11-08T19:27:38.202 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.186+0000 7f5ae3083700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5adc0c77d0 con 0x7f5adc06dbb0 2023-11-08T19:27:38.203 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.186+0000 7f5adbfff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5adc06e9a0 0x7f5adc1575d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:38.203 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.186+0000 7f5adbfff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5adc06e9a0 0x7f5adc1575d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53752/0 (socket says 172.21.15.143:53752) 2023-11-08T19:27:38.203 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.186+0000 7f5adbfff700 1 -- 172.21.15.143:0/3421000394 learned_addr learned my addr 172.21.15.143:0/3421000394 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:38.203 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.186+0000 7f5ae1620700 1 --2- 172.21.15.143:0/3421000394 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5adc072860 0x7f5adc0c82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:38.203 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.186+0000 7f5ae0e1f700 1 --2- 172.21.15.143:0/3421000394 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5adc06dbb0 0x7f5adc156850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:38.204 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.186+0000 7f5adbfff700 1 -- 172.21.15.143:0/3421000394 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5adc06dbb0 msgr2=0x7f5adc156850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:38.204 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.186+0000 7f5adbfff700 1 --2- 172.21.15.143:0/3421000394 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5adc06dbb0 0x7f5adc156850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:38.204 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.186+0000 7f5adbfff700 1 -- 172.21.15.143:0/3421000394 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5adc072860 msgr2=0x7f5adc0c82b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:38.204 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.186+0000 7f5adbfff700 1 --2- 172.21.15.143:0/3421000394 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5adc072860 0x7f5adc0c82b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:38.204 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.186+0000 7f5adbfff700 1 -- 172.21.15.143:0/3421000394 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ad4009b30 con 0x7f5adc06e9a0 2023-11-08T19:27:38.205 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.187+0000 7f5ae1620700 1 --2- 172.21.15.143:0/3421000394 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5adc072860 0x7f5adc0c82b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:27:38.205 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.187+0000 7f5adbfff700 1 --2- 172.21.15.143:0/3421000394 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5adc06e9a0 0x7f5adc1575d0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f5ac800b4a0 tx=0x7f5ac800b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:38.205 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.187+0000 7f5ad97fa700 1 -- 172.21.15.143:0/3421000394 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ac8011830 con 0x7f5adc06e9a0 2023-11-08T19:27:38.205 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.187+0000 7f5ad97fa700 1 -- 172.21.15.143:0/3421000394 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ac8011990 con 0x7f5adc06e9a0 2023-11-08T19:27:38.205 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.187+0000 7f5ae3083700 1 -- 172.21.15.143:0/3421000394 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5adc0c7ab0 con 0x7f5adc06e9a0 2023-11-08T19:27:38.205 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.187+0000 7f5ad97fa700 1 -- 172.21.15.143:0/3421000394 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ac8011d40 con 0x7f5adc06e9a0 2023-11-08T19:27:38.206 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.187+0000 7f5ae3083700 1 -- 172.21.15.143:0/3421000394 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5adc157f20 con 0x7f5adc06e9a0 2023-11-08T19:27:38.206 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.188+0000 7f5ad97fa700 1 -- 172.21.15.143:0/3421000394 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5ac801a400 con 0x7f5adc06e9a0 2023-11-08T19:27:38.206 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.189+0000 7f5ad97fa700 1 --2- 172.21.15.143:0/3421000394 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f5ac006af00 0x7f5ac006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:38.206 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.189+0000 7f5ad97fa700 1 -- 172.21.15.143:0/3421000394 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6772+0+0 (secure 0 0 0) 0x7f5ac80896f0 con 0x7f5adc06e9a0 2023-11-08T19:27:38.207 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.189+0000 7f5ae0e1f700 1 --2- 172.21.15.143:0/3421000394 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f5ac006af00 0x7f5ac006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:38.207 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.189+0000 7f5ae0e1f700 1 --2- 172.21.15.143:0/3421000394 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f5ac006af00 0x7f5ac006d3c0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f5adc0c8d10 tx=0x7f5ad0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:38.207 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.191+0000 7f5aef27cb80 1 Processor -- start 2023-11-08T19:27:38.207 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.191+0000 7f5aef27cb80 1 -- start start 2023-11-08T19:27:38.207 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.191+0000 7f5aef27cb80 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55cc4b20e6f0 0x55cc4b23e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:38.208 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.191+0000 7f5aef27cb80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55cc4b210c30 0x55cc4b23e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:38.208 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.191+0000 7f5aef27cb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cc4b20ead0 0x55cc4b228f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:38.208 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.191+0000 7f5aef27cb80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc4b17a6e0 con 0x55cc4b210c30 2023-11-08T19:27:38.208 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.191+0000 7f5aef27cb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc4b16e3b0 con 0x55cc4b20ead0 2023-11-08T19:27:38.208 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.191+0000 7f5aef27cb80 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cc4b16bcd0 con 0x55cc4b20e6f0 2023-11-08T19:27:38.208 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.192+0000 7f5adbfff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55cc4b210c30 0x55cc4b23e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:38.209 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.192+0000 7f5adbfff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55cc4b210c30 0x55cc4b23e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53760/0 (socket says 172.21.15.143:53760) 2023-11-08T19:27:38.209 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.192+0000 7f5adbfff700 1 -- 172.21.15.143:0/663407032 learned_addr learned my addr 172.21.15.143:0/663407032 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:38.209 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.192+0000 7f5ae1620700 1 --2- 172.21.15.143:0/663407032 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cc4b20ead0 0x55cc4b228f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:38.209 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.192+0000 7f5adbfff700 1 -- 172.21.15.143:0/663407032 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55cc4b20e6f0 msgr2=0x55cc4b23e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:38.210 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.192+0000 7f5ae0e1f700 1 --2- 172.21.15.143:0/663407032 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55cc4b20e6f0 0x55cc4b23e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:38.210 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.192+0000 7f5adbfff700 1 --2- 172.21.15.143:0/663407032 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55cc4b20e6f0 0x55cc4b23e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:38.210 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.192+0000 7f5adbfff700 1 -- 172.21.15.143:0/663407032 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cc4b20ead0 msgr2=0x55cc4b228f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:38.210 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.192+0000 7f5adbfff700 1 --2- 172.21.15.143:0/663407032 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cc4b20ead0 0x55cc4b228f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:38.210 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.192+0000 7f5adbfff700 1 -- 172.21.15.143:0/663407032 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cc4afe45e0 con 0x55cc4b210c30 2023-11-08T19:27:38.211 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.192+0000 7f5adbfff700 1 --2- 172.21.15.143:0/663407032 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55cc4b210c30 0x55cc4b23e990 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f5ac800bf20 tx=0x7f5ac8057ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61d72e20e4cfa37f server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:38.211 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.192+0000 7f5ac67fc700 1 -- 172.21.15.143:0/663407032 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ac8060320 con 0x55cc4b210c30 2023-11-08T19:27:38.211 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.192+0000 7f5ac67fc700 1 -- 172.21.15.143:0/663407032 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ac8056470 con 0x55cc4b210c30 2023-11-08T19:27:38.211 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.192+0000 7f5ac67fc700 1 -- 172.21.15.143:0/663407032 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ac8058330 con 0x55cc4b210c30 2023-11-08T19:27:38.211 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.193+0000 7f5aef27cb80 1 -- 172.21.15.143:0/663407032 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55cc4b210c30 msgr2=0x55cc4b23e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:38.212 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.193+0000 7f5aef27cb80 1 --2- 172.21.15.143:0/663407032 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55cc4b210c30 0x55cc4b23e990 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f5ac800bf20 tx=0x7f5ac8057ab0 comp rx=0 tx=0).stop 2023-11-08T19:27:38.212 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.193+0000 7f5aef27cb80 1 -- 172.21.15.143:0/663407032 shutdown_connections 2023-11-08T19:27:38.212 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.193+0000 7f5aef27cb80 1 --2- 172.21.15.143:0/663407032 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55cc4b20e6f0 0x55cc4b23e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:38.212 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.193+0000 7f5aef27cb80 1 --2- 172.21.15.143:0/663407032 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cc4b20ead0 0x55cc4b228f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:38.212 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.193+0000 7f5aef27cb80 1 --2- 172.21.15.143:0/663407032 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55cc4b210c30 0x55cc4b23e990 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:38.212 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.193+0000 7f5aef27cb80 1 -- 172.21.15.143:0/663407032 >> 172.21.15.143:0/663407032 conn(0x55cc4b189660 msgr2=0x55cc4b2143c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:38.213 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.193+0000 7f5aef27cb80 1 -- 172.21.15.143:0/663407032 shutdown_connections 2023-11-08T19:27:38.213 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.193+0000 7f5aef27cb80 1 -- 172.21.15.143:0/663407032 wait complete. 2023-11-08T19:27:38.213 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.193+0000 7f5aef27cb80 1 -- start start 2023-11-08T19:27:38.213 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.194+0000 7f5aef27cb80 1 Processor -- start 2023-11-08T19:27:38.214 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.194+0000 7f5aef27cb80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55cc4b20e6f0 0x55cc4b254490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:38.214 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.194+0000 7f5aef27cb80 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55cc4b20ead0 0x55cc4b2296c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:38.214 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.194+0000 7f5aef27cb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cc4b242230 0x55cc4b255d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:38.214 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.194+0000 7f5aef27cb80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc4b1675c0 con 0x55cc4b20e6f0 2023-11-08T19:27:38.214 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.194+0000 7f5aef27cb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc4b1030d0 con 0x55cc4b242230 2023-11-08T19:27:38.215 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.194+0000 7f5aef27cb80 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cc4b102d70 con 0x55cc4b20ead0 2023-11-08T19:27:38.215 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.194+0000 7f5aef27cb80 10 client.0 _ll_register_callbacks cb 0x55cc4af5f7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T19:27:38.215 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.194+0000 7f5ae0e1f700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55cc4b20e6f0 0x55cc4b254490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:38.215 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.194+0000 7f5ae0e1f700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55cc4b20e6f0 0x55cc4b254490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53764/0 (socket says 172.21.15.143:53764) 2023-11-08T19:27:38.215 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.194+0000 7f5ae0e1f700 1 -- 172.21.15.143:0/1395005543 learned_addr learned my addr 172.21.15.143:0/1395005543 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:38.216 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.195+0000 7f5adbfff700 1 --2- 172.21.15.143:0/1395005543 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55cc4b20ead0 0x55cc4b2296c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:38.216 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.195+0000 7f5ae1620700 1 --2- 172.21.15.143:0/1395005543 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cc4b242230 0x55cc4b255d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:38.216 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.195+0000 7f5ae0e1f700 1 -- 172.21.15.143:0/1395005543 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55cc4b20ead0 msgr2=0x55cc4b2296c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:38.216 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.195+0000 7f5ae0e1f700 1 --2- 172.21.15.143:0/1395005543 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55cc4b20ead0 0x55cc4b2296c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:38.216 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.195+0000 7f5ae0e1f700 1 -- 172.21.15.143:0/1395005543 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cc4b242230 msgr2=0x55cc4b255d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:38.217 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.195+0000 7f5ae0e1f700 1 --2- 172.21.15.143:0/1395005543 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cc4b242230 0x55cc4b255d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:38.217 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.195+0000 7f5ae0e1f700 1 -- 172.21.15.143:0/1395005543 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5adc157f20 con 0x55cc4b20e6f0 2023-11-08T19:27:38.217 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.195+0000 7f5ae0e1f700 1 --2- 172.21.15.143:0/1395005543 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55cc4b20e6f0 0x55cc4b254490 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f5ad000fef0 tx=0x7f5ad00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e22afb115eea000a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:38.217 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.195+0000 7f5abb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2023-11-08T19:27:38.217 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.195+0000 7f5abb7fe700 1 -- 172.21.15.143:0/1395005543 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad0021610 con 0x55cc4b20e6f0 2023-11-08T19:27:38.218 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.195+0000 7f5abb7fe700 1 -- 172.21.15.143:0/1395005543 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(78..78 src has 1..78) v4 ==== 6772+0+0 (secure 0 0 0) 0x7f5ad0021770 con 0x55cc4b20e6f0 2023-11-08T19:27:38.218 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.195+0000 7f5aef27cb80 1 -- 172.21.15.143:0/1395005543 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cc4b206880 con 0x55cc4b20e6f0 2023-11-08T19:27:38.218 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.195+0000 7f5abb7fe700 1 -- 172.21.15.143:0/1395005543 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5ad0014e90 con 0x55cc4b20e6f0 2023-11-08T19:27:38.218 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.195+0000 7f5aef27cb80 10 client.6633 fetch_fsmap learned FSMap version 41 2023-11-08T19:27:38.218 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.195+0000 7f5aef27cb80 1 -- 172.21.15.143:0/1395005543 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=2+}) v3 -- 0x55cc4af90d90 con 0x55cc4b20e6f0 2023-11-08T19:27:38.219 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.195+0000 7f5abb7fe700 1 -- 172.21.15.143:0/1395005543 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55cc4b206880 con 0x55cc4b20e6f0 2023-11-08T19:27:38.219 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.195+0000 7f5abb7fe700 1 -- 172.21.15.143:0/1395005543 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 41) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f5ad0021930 con 0x55cc4b20e6f0 2023-11-08T19:27:38.219 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.195+0000 7f5aef27cb80 10 client.6633 fetch_fsmap finished waiting for FSMap version 41 2023-11-08T19:27:38.219 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.195+0000 7f5aef27cb80 10 client.6633 resolve_mds: resolved name 'd' to daemon mds.d 2023-11-08T19:27:38.219 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.195+0000 7f5aef27cb80 20 client.6633 populate_metadata read hostname 'smithi143' 2023-11-08T19:27:38.219 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.195+0000 7f5aef27cb80 1 --2- 172.21.15.143:0/1395005543 >> [v2:172.21.15.174:6835/1950356087,v1:172.21.15.174:6837/1950356087] conn(0x55cc4b240d40 0x55cc4b276520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:38.220 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.195+0000 7f5aef27cb80 4 client.6633 mds_command: new command op to 6489 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-08T19:27:38.220 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.195+0000 7f5aef27cb80 1 -- 172.21.15.143:0/1395005543 --> [v2:172.21.15.174:6835/1950356087,v1:172.21.15.174:6837/1950356087] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cc4b1e2a30 con 0x55cc4b240d40 2023-11-08T19:27:38.220 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.196+0000 7f5adbfff700 1 --2- 172.21.15.143:0/1395005543 >> [v2:172.21.15.174:6835/1950356087,v1:172.21.15.174:6837/1950356087] conn(0x55cc4b240d40 0x55cc4b276520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:38.220 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.196+0000 7f5abb7fe700 10 client.6633 ms_handle_connect on v2:172.21.15.174:6835/1950356087 2023-11-08T19:27:38.221 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.196+0000 7f5adbfff700 1 --2- 172.21.15.143:0/1395005543 >> [v2:172.21.15.174:6835/1950356087,v1:172.21.15.174:6837/1950356087] conn(0x55cc4b240d40 0x55cc4b276520 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e75868019cca945f server_cookie=435de69186226642 in_seq=0 out_seq=0 2023-11-08T19:27:38.221 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.197+0000 7f5abb7fe700 1 -- 172.21.15.143:0/1395005543 <== mds.0 v2:172.21.15.174:6835/1950356087 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55cc4b1e2a30 con 0x55cc4b240d40 2023-11-08T19:27:38.221 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.197+0000 7f5abb7fe700 10 client.6633 handle_command_reply: tid=0 2023-11-08T19:27:38.221 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.197+0000 7f5aef27cb80 1 client.6633 shutdown 2023-11-08T19:27:38.221 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.197+0000 7f5aef27cb80 1 -- 172.21.15.143:0/1395005543 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55cc4b20e6f0 msgr2=0x55cc4b254490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:38.222 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.197+0000 7f5aef27cb80 1 --2- 172.21.15.143:0/1395005543 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55cc4b20e6f0 0x55cc4b254490 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f5ad000fef0 tx=0x7f5ad00047c0 comp rx=0 tx=0).stop 2023-11-08T19:27:38.222 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.197+0000 7f5aef27cb80 1 -- 172.21.15.143:0/1395005543 shutdown_connections 2023-11-08T19:27:38.222 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.197+0000 7f5aef27cb80 1 --2- 172.21.15.143:0/1395005543 >> [v2:172.21.15.174:6835/1950356087,v1:172.21.15.174:6837/1950356087] conn(0x55cc4b240d40 0x55cc4b276520 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:38.222 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.197+0000 7f5aef27cb80 1 --2- 172.21.15.143:0/1395005543 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cc4b242230 0x55cc4b255d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:38.222 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.197+0000 7f5abb7fe700 0 client.6633 ms_handle_reset on v2:172.21.15.174:6835/1950356087 2023-11-08T19:27:38.223 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.197+0000 7f5aef27cb80 1 --2- 172.21.15.143:0/1395005543 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55cc4b20e6f0 0x55cc4b254490 secure :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f5ad000fef0 tx=0x7f5ad00047c0 comp rx=0 tx=0).stop 2023-11-08T19:27:38.223 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.197+0000 7f5aef27cb80 1 --2- 172.21.15.143:0/1395005543 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55cc4b20ead0 0x55cc4b2296c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:38.223 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.197+0000 7f5aef27cb80 1 -- 172.21.15.143:0/1395005543 >> 172.21.15.143:0/1395005543 conn(0x55cc4b189660 msgr2=0x55cc4b23e9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:38.223 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.197+0000 7f5aef27cb80 1 -- 172.21.15.143:0/1395005543 shutdown_connections 2023-11-08T19:27:38.223 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.197+0000 7f5aef27cb80 1 -- 172.21.15.143:0/1395005543 wait complete. 2023-11-08T19:27:38.223 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.197+0000 7f5aef27cb80 20 client.6633 trim_cache size 0 max 16384 2023-11-08T19:27:38.229 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.215+0000 7f5aef27cb80 1 Processor -- start 2023-11-08T19:27:38.229 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.215+0000 7f5aef27cb80 1 -- start start 2023-11-08T19:27:38.229 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.216+0000 7f5aef27cb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cc4b20ead0 0x55cc4b278540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:38.230 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.216+0000 7f5aef27cb80 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55cc4b242230 0x55cc4b278a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:38.230 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.216+0000 7f5aef27cb80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55cc4b2572a0 0x55cc4b252100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:38.230 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.216+0000 7f5aef27cb80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc4afe68d0 con 0x55cc4b2572a0 2023-11-08T19:27:38.230 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.216+0000 7f5aef27cb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc4b23eb50 con 0x55cc4b20ead0 2023-11-08T19:27:38.231 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.216+0000 7f5aef27cb80 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cc4b2157c0 con 0x55cc4b242230 2023-11-08T19:27:38.231 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.216+0000 7f5ae1620700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55cc4b2572a0 0x55cc4b252100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:38.231 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.216+0000 7f5ae1620700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55cc4b2572a0 0x55cc4b252100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53776/0 (socket says 172.21.15.143:53776) 2023-11-08T19:27:38.231 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.216+0000 7f5ae1620700 1 -- 172.21.15.143:0/3206483057 learned_addr learned my addr 172.21.15.143:0/3206483057 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:38.231 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.216+0000 7f5adbfff700 1 --2- 172.21.15.143:0/3206483057 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55cc4b242230 0x55cc4b278a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:38.232 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.216+0000 7f5ae1620700 1 -- 172.21.15.143:0/3206483057 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55cc4b242230 msgr2=0x55cc4b278a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:38.232 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.216+0000 7f5ae0e1f700 1 --2- 172.21.15.143:0/3206483057 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cc4b20ead0 0x55cc4b278540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:38.232 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.216+0000 7f5ae1620700 1 --2- 172.21.15.143:0/3206483057 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55cc4b242230 0x55cc4b278a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:38.232 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.216+0000 7f5ae1620700 1 -- 172.21.15.143:0/3206483057 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cc4b20ead0 msgr2=0x55cc4b278540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:38.232 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.216+0000 7f5ae1620700 1 --2- 172.21.15.143:0/3206483057 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cc4b20ead0 0x55cc4b278540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:38.233 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.216+0000 7f5ae1620700 1 -- 172.21.15.143:0/3206483057 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cc4af79c10 con 0x55cc4b2572a0 2023-11-08T19:27:38.233 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.216+0000 7f5ae1620700 1 --2- 172.21.15.143:0/3206483057 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55cc4b2572a0 0x55cc4b252100 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f5ad400b730 tx=0x7f5ad4010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e1eef9d098f3c94 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:38.233 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.216+0000 7f5ac6ffd700 1 -- 172.21.15.143:0/3206483057 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad4003e90 con 0x55cc4b2572a0 2023-11-08T19:27:38.233 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.216+0000 7f5ac6ffd700 1 -- 172.21.15.143:0/3206483057 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ad400b980 con 0x55cc4b2572a0 2023-11-08T19:27:38.233 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.216+0000 7f5ac6ffd700 1 -- 172.21.15.143:0/3206483057 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad400bd30 con 0x55cc4b2572a0 2023-11-08T19:27:38.233 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.217+0000 7f5aef27cb80 1 -- 172.21.15.143:0/3206483057 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55cc4b2572a0 msgr2=0x55cc4b252100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:38.234 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.217+0000 7f5aef27cb80 1 --2- 172.21.15.143:0/3206483057 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55cc4b2572a0 0x55cc4b252100 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f5ad400b730 tx=0x7f5ad4010000 comp rx=0 tx=0).stop 2023-11-08T19:27:38.234 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.217+0000 7f5aef27cb80 1 -- 172.21.15.143:0/3206483057 shutdown_connections 2023-11-08T19:27:38.234 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.217+0000 7f5aef27cb80 1 --2- 172.21.15.143:0/3206483057 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55cc4b2572a0 0x55cc4b252100 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:38.234 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.217+0000 7f5aef27cb80 1 --2- 172.21.15.143:0/3206483057 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cc4b20ead0 0x55cc4b278540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:38.234 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.217+0000 7f5aef27cb80 1 --2- 172.21.15.143:0/3206483057 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55cc4b242230 0x55cc4b278a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:38.235 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.217+0000 7f5aef27cb80 1 -- 172.21.15.143:0/3206483057 >> 172.21.15.143:0/3206483057 conn(0x55cc4b189660 msgr2=0x55cc4b2414c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:38.235 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.217+0000 7f5aef27cb80 1 -- 172.21.15.143:0/3206483057 shutdown_connections 2023-11-08T19:27:38.235 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.217+0000 7f5aef27cb80 1 -- 172.21.15.143:0/3206483057 wait complete. 2023-11-08T19:27:38.235 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.217+0000 7f5aef27cb80 1 -- start start 2023-11-08T19:27:38.235 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.218+0000 7f5aef27cb80 1 Processor -- start 2023-11-08T19:27:38.236 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.218+0000 7f5aef27cb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cc4b20ead0 0x55cc4b267d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:38.236 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.218+0000 7f5aef27cb80 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55cc4b242230 0x55cc4b268260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:38.236 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.218+0000 7f5aef27cb80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55cc4b2572a0 0x55cc4b259ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:38.236 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.218+0000 7f5aef27cb80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc4b215f30 con 0x55cc4b2572a0 2023-11-08T19:27:38.236 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.218+0000 7f5aef27cb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc4b276be0 con 0x55cc4b20ead0 2023-11-08T19:27:38.237 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.218+0000 7f5aef27cb80 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cc4b215ce0 con 0x55cc4b242230 2023-11-08T19:27:38.237 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.218+0000 7f5aef27cb80 10 client.0 _ll_register_callbacks cb 0x7f5ad000fb40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T19:27:38.237 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.218+0000 7f5ae1620700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55cc4b2572a0 0x55cc4b259ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:38.237 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.218+0000 7f5ae1620700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55cc4b2572a0 0x55cc4b259ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53782/0 (socket says 172.21.15.143:53782) 2023-11-08T19:27:38.237 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.218+0000 7f5ae1620700 1 -- 172.21.15.143:0/184128583 learned_addr learned my addr 172.21.15.143:0/184128583 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:38.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.219+0000 7f5ae0e1f700 1 --2- 172.21.15.143:0/184128583 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cc4b20ead0 0x55cc4b267d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:38.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.219+0000 7f5adbfff700 1 --2- 172.21.15.143:0/184128583 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55cc4b242230 0x55cc4b268260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:38.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.219+0000 7f5ae1620700 1 -- 172.21.15.143:0/184128583 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55cc4b242230 msgr2=0x55cc4b268260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:38.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.219+0000 7f5ae1620700 1 --2- 172.21.15.143:0/184128583 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55cc4b242230 0x55cc4b268260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:38.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.219+0000 7f5ae1620700 1 -- 172.21.15.143:0/184128583 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cc4b20ead0 msgr2=0x55cc4b267d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:38.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.219+0000 7f5ae1620700 1 --2- 172.21.15.143:0/184128583 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cc4b20ead0 0x55cc4b267d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:38.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.219+0000 7f5ae1620700 1 -- 172.21.15.143:0/184128583 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5adc06c2d0 con 0x55cc4b2572a0 2023-11-08T19:27:38.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.219+0000 7f5ae1620700 1 --2- 172.21.15.143:0/184128583 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55cc4b2572a0 0x55cc4b259ed0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f5ad400b370 tx=0x7f5ad4004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c8420c73af8a39b server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:38.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.219+0000 7f5abb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2023-11-08T19:27:38.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.219+0000 7f5abb7fe700 1 -- 172.21.15.143:0/184128583 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad401dc70 con 0x55cc4b2572a0 2023-11-08T19:27:38.240 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.219+0000 7f5abb7fe700 1 -- 172.21.15.143:0/184128583 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(78..78 src has 1..78) v4 ==== 6772+0+0 (secure 0 0 0) 0x7f5ad401ddd0 con 0x55cc4b2572a0 2023-11-08T19:27:38.240 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.219+0000 7f5aef27cb80 1 -- 172.21.15.143:0/184128583 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5ac8056470 con 0x55cc4b2572a0 2023-11-08T19:27:38.240 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.219+0000 7f5abb7fe700 1 -- 172.21.15.143:0/184128583 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5ad4012430 con 0x55cc4b2572a0 2023-11-08T19:27:38.240 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.219+0000 7f5aef27cb80 10 client.6639 fetch_fsmap learned FSMap version 41 2023-11-08T19:27:38.240 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.219+0000 7f5aef27cb80 1 -- 172.21.15.143:0/184128583 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=2+}) v3 -- 0x55cc4afb7200 con 0x55cc4b2572a0 2023-11-08T19:27:38.241 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.219+0000 7f5abb7fe700 1 -- 172.21.15.143:0/184128583 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ac8056470 con 0x55cc4b2572a0 2023-11-08T19:27:38.241 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.219+0000 7f5abb7fe700 1 -- 172.21.15.143:0/184128583 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 41) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f5ad4023020 con 0x55cc4b2572a0 2023-11-08T19:27:38.241 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.219+0000 7f5aef27cb80 10 client.6639 fetch_fsmap finished waiting for FSMap version 41 2023-11-08T19:27:38.241 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.219+0000 7f5aef27cb80 10 client.6639 resolve_mds: resolved name 'd' to daemon mds.d 2023-11-08T19:27:38.241 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.219+0000 7f5aef27cb80 20 client.6639 populate_metadata read hostname 'smithi143' 2023-11-08T19:27:38.242 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.219+0000 7f5aef27cb80 1 --2- 172.21.15.143:0/184128583 >> [v2:172.21.15.174:6835/1950356087,v1:172.21.15.174:6837/1950356087] conn(0x55cc4b2c2f10 0x55cc4b25a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:38.242 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.220+0000 7f5aef27cb80 4 client.6639 mds_command: new command op to 6489 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-08T19:27:38.242 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.220+0000 7f5aef27cb80 1 -- 172.21.15.143:0/184128583 --> [v2:172.21.15.174:6835/1950356087,v1:172.21.15.174:6837/1950356087] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55cc4b1605a0 con 0x55cc4b2c2f10 2023-11-08T19:27:38.242 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.220+0000 7f5ae0e1f700 1 --2- 172.21.15.143:0/184128583 >> [v2:172.21.15.174:6835/1950356087,v1:172.21.15.174:6837/1950356087] conn(0x55cc4b2c2f10 0x55cc4b25a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:38.242 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.220+0000 7f5ae0e1f700 1 --2- 172.21.15.143:0/184128583 >> [v2:172.21.15.174:6835/1950356087,v1:172.21.15.174:6837/1950356087] conn(0x55cc4b2c2f10 0x55cc4b25a410 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5571d614f375ff50 server_cookie=c7db2527e95f930e in_seq=0 out_seq=0 2023-11-08T19:27:38.243 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.220+0000 7f5abb7fe700 10 client.6639 ms_handle_connect on v2:172.21.15.174:6835/1950356087 2023-11-08T19:27:38.243 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.220+0000 7f5abb7fe700 1 -- 172.21.15.143:0/184128583 <== mds.0 v2:172.21.15.174:6835/1950356087 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55cc4b1605a0 con 0x55cc4b2c2f10 2023-11-08T19:27:38.243 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.220+0000 7f5abb7fe700 10 client.6639 handle_command_reply: tid=0 2023-11-08T19:27:38.243 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.220+0000 7f5aef27cb80 1 client.6639 shutdown 2023-11-08T19:27:38.243 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.220+0000 7f5aef27cb80 1 -- 172.21.15.143:0/184128583 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55cc4b2572a0 msgr2=0x55cc4b259ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:38.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.220+0000 7f5aef27cb80 1 --2- 172.21.15.143:0/184128583 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55cc4b2572a0 0x55cc4b259ed0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f5ad400b370 tx=0x7f5ad4004b40 comp rx=0 tx=0).stop 2023-11-08T19:27:38.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.220+0000 7f5aef27cb80 1 -- 172.21.15.143:0/184128583 shutdown_connections 2023-11-08T19:27:38.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.220+0000 7f5aef27cb80 1 --2- 172.21.15.143:0/184128583 >> [v2:172.21.15.174:6835/1950356087,v1:172.21.15.174:6837/1950356087] conn(0x55cc4b2c2f10 0x55cc4b25a410 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:38.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.220+0000 7f5aef27cb80 1 --2- 172.21.15.143:0/184128583 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55cc4b2572a0 0x55cc4b259ed0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:38.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.220+0000 7f5aef27cb80 1 --2- 172.21.15.143:0/184128583 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cc4b20ead0 0x55cc4b267d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:38.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.220+0000 7f5abb7fe700 0 client.6639 ms_handle_reset on v2:172.21.15.174:6835/1950356087 2023-11-08T19:27:38.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.220+0000 7f5aef27cb80 1 --2- 172.21.15.143:0/184128583 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55cc4b242230 0x55cc4b268260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:38.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.220+0000 7f5aef27cb80 1 -- 172.21.15.143:0/184128583 >> 172.21.15.143:0/184128583 conn(0x55cc4b189660 msgr2=0x55cc4b241180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:38.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.221+0000 7f5aef27cb80 1 -- 172.21.15.143:0/184128583 shutdown_connections 2023-11-08T19:27:38.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.221+0000 7f5aef27cb80 1 -- 172.21.15.143:0/184128583 wait complete. 2023-11-08T19:27:38.246 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.221+0000 7f5aef27cb80 20 client.6639 trim_cache size 0 max 16384 2023-11-08T19:27:38.246 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2023-11-08T19:27:38.246 INFO:teuthology.orchestra.run.smithi143.stdout: "cluster_fsid": "968483af-77fc-4068-ad7c-5c2a1a4613d7", 2023-11-08T19:27:38.247 INFO:teuthology.orchestra.run.smithi143.stdout: "whoami": 0, 2023-11-08T19:27:38.247 INFO:teuthology.orchestra.run.smithi143.stdout: "id": 6489, 2023-11-08T19:27:38.247 INFO:teuthology.orchestra.run.smithi143.stdout: "want_state": "up:active", 2023-11-08T19:27:38.247 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "up:active", 2023-11-08T19:27:38.247 INFO:teuthology.orchestra.run.smithi143.stdout: "fs_name": "cephfs", 2023-11-08T19:27:38.247 INFO:teuthology.orchestra.run.smithi143.stdout: "rank_uptime": 5.9179533119999999, 2023-11-08T19:27:38.248 INFO:teuthology.orchestra.run.smithi143.stdout: "mdsmap_epoch": 41, 2023-11-08T19:27:38.248 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_epoch": 78, 2023-11-08T19:27:38.248 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_epoch_barrier": 78, 2023-11-08T19:27:38.248 INFO:teuthology.orchestra.run.smithi143.stdout: "uptime": 11.855742496 2023-11-08T19:27:38.248 INFO:teuthology.orchestra.run.smithi143.stdout:} 2023-11-08T19:27:38.250 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.221+0000 7f5ae3083700 1 -- 172.21.15.143:0/3421000394 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f5ac006af00 msgr2=0x7f5ac006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:38.250 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.221+0000 7f5ae3083700 1 --2- 172.21.15.143:0/3421000394 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f5ac006af00 0x7f5ac006d3c0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f5adc0c8d10 tx=0x7f5ad0008040 comp rx=0 tx=0).stop 2023-11-08T19:27:38.250 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.221+0000 7f5ae3083700 1 -- 172.21.15.143:0/3421000394 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5adc06e9a0 msgr2=0x7f5adc1575d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:38.251 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.221+0000 7f5ae3083700 1 --2- 172.21.15.143:0/3421000394 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5adc06e9a0 0x7f5adc1575d0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f5ac800b4a0 tx=0x7f5ac800b860 comp rx=0 tx=0).stop 2023-11-08T19:27:38.251 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.221+0000 7f5ae3083700 1 -- 172.21.15.143:0/3421000394 shutdown_connections 2023-11-08T19:27:38.251 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.221+0000 7f5ae3083700 1 --2- 172.21.15.143:0/3421000394 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f5ac006af00 0x7f5ac006d3c0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:38.251 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.221+0000 7f5ae3083700 1 --2- 172.21.15.143:0/3421000394 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5adc06dbb0 0x7f5adc156850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:38.252 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.221+0000 7f5ae3083700 1 --2- 172.21.15.143:0/3421000394 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5adc072860 0x7f5adc0c82b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:38.252 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.221+0000 7f5ae3083700 1 --2- 172.21.15.143:0/3421000394 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5adc06e9a0 0x7f5adc1575d0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:38.252 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.221+0000 7f5ae3083700 1 -- 172.21.15.143:0/3421000394 >> 172.21.15.143:0/3421000394 conn(0x7f5adc0b1a20 msgr2=0x7f5adc0b4510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:38.252 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.221+0000 7f5ae3083700 1 -- 172.21.15.143:0/3421000394 shutdown_connections 2023-11-08T19:27:38.252 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.221+0000 7f5ae3083700 1 -- 172.21.15.143:0/3421000394 wait complete. 2023-11-08T19:27:38.253 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-08T19:27:38.374 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.373+0000 7fd146a1f700 1 Processor -- start 2023-11-08T19:27:38.374 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.373+0000 7fd146a1f700 1 -- start start 2023-11-08T19:27:38.377 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.373+0000 7fd146a1f700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd1400b14c0 0x7fd1400d03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:38.377 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.373+0000 7fd146a1f700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1400d08e0 con 0x7fd1400d1c70 2023-11-08T19:27:38.377 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.373+0000 7fd146a1f700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1400b1910 con 0x7fd14015ba30 2023-11-08T19:27:38.377 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.373+0000 7fd146a1f700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1400b1bc0 con 0x7fd1400b14c0 2023-11-08T19:27:38.377 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.374+0000 7fd144fbc700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd1400b14c0 0x7fd1400d03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:38.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.374+0000 7fd13f7fe700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fd1400d1c70 0x7fd1400cc5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:49940/0 (socket says 172.21.15.143:49940) 2023-11-08T19:27:38.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.374+0000 7fd13f7fe700 1 -- 172.21.15.143:0/2728634844 learned_addr learned my addr 172.21.15.143:0/2728634844 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:27:38.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.374+0000 7fd144fbc700 1 -- 172.21.15.143:0/2728634844 >> v1:172.21.15.174:6789/0 conn(0x7fd14015ba30 legacy=0x7fd1400d1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:38.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.374+0000 7fd144fbc700 1 -- 172.21.15.143:0/2728634844 >> v1:172.21.15.143:6789/0 conn(0x7fd1400d1c70 legacy=0x7fd1400cc5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:38.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.374+0000 7fd144fbc700 1 -- 172.21.15.143:0/2728634844 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1400b1ea0 con 0x7fd1400b14c0 2023-11-08T19:27:38.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.374+0000 7fd144fbc700 1 --2- 172.21.15.143:0/2728634844 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd1400b14c0 0x7fd1400d03a0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fd138009d90 tx=0x7fd13800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad9062c50e4a1b2b server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:38.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.374+0000 7fd13effd700 1 -- 172.21.15.143:0/2728634844 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd13800e070 con 0x7fd1400b14c0 2023-11-08T19:27:38.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.374+0000 7fd13effd700 1 -- 172.21.15.143:0/2728634844 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd13800bad0 con 0x7fd1400b14c0 2023-11-08T19:27:38.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.374+0000 7fd13effd700 1 -- 172.21.15.143:0/2728634844 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd13800be80 con 0x7fd1400b14c0 2023-11-08T19:27:38.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.375+0000 7fd146a1f700 1 -- 172.21.15.143:0/2728634844 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd1400b14c0 msgr2=0x7fd1400d03a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:38.380 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.375+0000 7fd146a1f700 1 --2- 172.21.15.143:0/2728634844 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd1400b14c0 0x7fd1400d03a0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fd138009d90 tx=0x7fd13800b330 comp rx=0 tx=0).stop 2023-11-08T19:27:38.380 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.375+0000 7fd146a1f700 1 -- 172.21.15.143:0/2728634844 shutdown_connections 2023-11-08T19:27:38.380 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.375+0000 7fd146a1f700 1 --2- 172.21.15.143:0/2728634844 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd1400b14c0 0x7fd1400d03a0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:38.380 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.375+0000 7fd146a1f700 1 -- 172.21.15.143:0/2728634844 >> 172.21.15.143:0/2728634844 conn(0x7fd14008b000 msgr2=0x7fd14009ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:38.380 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.375+0000 7fd146a1f700 1 -- 172.21.15.143:0/2728634844 shutdown_connections 2023-11-08T19:27:38.381 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.375+0000 7fd146a1f700 1 -- 172.21.15.143:0/2728634844 wait complete. 2023-11-08T19:27:38.383 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.376+0000 7fd146a1f700 1 Processor -- start 2023-11-08T19:27:38.383 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.376+0000 7fd146a1f700 1 -- start start 2023-11-08T19:27:38.384 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.377+0000 7fd146a1f700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd1400b14c0 0x7fd1400c7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:38.384 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.377+0000 7fd146a1f700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd1400d1c70 0x7fd14006bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:38.384 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.377+0000 7fd146a1f700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd14015ba30 0x7fd14006d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:38.384 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.377+0000 7fd146a1f700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd14015c810 con 0x7fd1400d1c70 2023-11-08T19:27:38.384 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.377+0000 7fd146a1f700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd14006c420 con 0x7fd1400b14c0 2023-11-08T19:27:38.385 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.377+0000 7fd146a1f700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd14006c700 con 0x7fd14015ba30 2023-11-08T19:27:38.385 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.377+0000 7fd13f7fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd1400d1c70 0x7fd14006bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:38.385 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.377+0000 7fd13f7fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd1400d1c70 0x7fd14006bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53792/0 (socket says 172.21.15.143:53792) 2023-11-08T19:27:38.385 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.377+0000 7fd13f7fe700 1 -- 172.21.15.143:0/1457992900 learned_addr learned my addr 172.21.15.143:0/1457992900 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:38.385 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.377+0000 7fd144fbc700 1 --2- 172.21.15.143:0/1457992900 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd14015ba30 0x7fd14006d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:38.386 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.377+0000 7fd13ffff700 1 --2- 172.21.15.143:0/1457992900 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd1400b14c0 0x7fd1400c7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:38.386 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.377+0000 7fd13f7fe700 1 -- 172.21.15.143:0/1457992900 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd14015ba30 msgr2=0x7fd14006d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:38.386 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.377+0000 7fd13f7fe700 1 --2- 172.21.15.143:0/1457992900 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd14015ba30 0x7fd14006d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:38.386 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.377+0000 7fd13f7fe700 1 -- 172.21.15.143:0/1457992900 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd1400b14c0 msgr2=0x7fd1400c7ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:38.387 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.377+0000 7fd13f7fe700 1 --2- 172.21.15.143:0/1457992900 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd1400b14c0 0x7fd1400c7ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:38.387 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.377+0000 7fd13f7fe700 1 -- 172.21.15.143:0/1457992900 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd138009a70 con 0x7fd1400d1c70 2023-11-08T19:27:38.387 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.377+0000 7fd13ffff700 1 --2- 172.21.15.143:0/1457992900 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd1400b14c0 0x7fd1400c7ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:27:38.387 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.378+0000 7fd13f7fe700 1 --2- 172.21.15.143:0/1457992900 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd1400d1c70 0x7fd14006bd20 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fd13400c850 tx=0x7fd13400cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:38.387 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.378+0000 7fd13cff9700 1 -- 172.21.15.143:0/1457992900 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd134004c80 con 0x7fd1400d1c70 2023-11-08T19:27:38.388 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.378+0000 7fd13cff9700 1 -- 172.21.15.143:0/1457992900 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd134004de0 con 0x7fd1400d1c70 2023-11-08T19:27:38.388 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.378+0000 7fd146a1f700 1 -- 172.21.15.143:0/1457992900 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd14006c9e0 con 0x7fd1400d1c70 2023-11-08T19:27:38.388 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.378+0000 7fd13cff9700 1 -- 172.21.15.143:0/1457992900 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd134007310 con 0x7fd1400d1c70 2023-11-08T19:27:38.388 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.378+0000 7fd146a1f700 1 -- 172.21.15.143:0/1457992900 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd14006cfa0 con 0x7fd1400d1c70 2023-11-08T19:27:38.388 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.379+0000 7fd146a1f700 1 -- 172.21.15.143:0/1457992900 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd14006d1c0 con 0x7fd1400d1c70 2023-11-08T19:27:38.389 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.379+0000 7fd13cff9700 1 -- 172.21.15.143:0/1457992900 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd134007470 con 0x7fd1400d1c70 2023-11-08T19:27:38.389 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.379+0000 7fd13cff9700 1 --2- 172.21.15.143:0/1457992900 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd12c06af00 0x7fd12c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:38.389 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.380+0000 7fd13cff9700 1 -- 172.21.15.143:0/1457992900 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6772+0+0 (secure 0 0 0) 0x7fd134088f30 con 0x7fd1400d1c70 2023-11-08T19:27:38.389 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.380+0000 7fd13ffff700 1 --2- 172.21.15.143:0/1457992900 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd12c06af00 0x7fd12c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:38.389 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.380+0000 7fd13ffff700 1 --2- 172.21.15.143:0/1457992900 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd12c06af00 0x7fd12c06d3c0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fd1300016a0 tx=0x7fd130009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:38.390 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.384+0000 7fd13cff9700 1 -- 172.21.15.143:0/1457992900 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd134053bb0 con 0x7fd1400d1c70 2023-11-08T19:27:38.618 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.617+0000 7fd146a1f700 1 -- 172.21.15.143:0/1457992900 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fd1400d08e0 con 0x7fd1400d1c70 2023-11-08T19:27:38.619 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.617+0000 7fd13cff9700 1 -- 172.21.15.143:0/1457992900 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v41) v1 ==== 53+0+73 (secure 0 0 0) 0x7fd1400d08e0 con 0x7fd1400d1c70 2023-11-08T19:27:38.619 INFO:teuthology.orchestra.run.smithi143.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-08T19:27:38.621 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.619+0000 7fd146a1f700 1 -- 172.21.15.143:0/1457992900 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd12c06af00 msgr2=0x7fd12c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:38.621 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.619+0000 7fd146a1f700 1 --2- 172.21.15.143:0/1457992900 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd12c06af00 0x7fd12c06d3c0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fd1300016a0 tx=0x7fd130009040 comp rx=0 tx=0).stop 2023-11-08T19:27:38.622 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.619+0000 7fd146a1f700 1 -- 172.21.15.143:0/1457992900 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd1400d1c70 msgr2=0x7fd14006bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:38.622 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.619+0000 7fd146a1f700 1 --2- 172.21.15.143:0/1457992900 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd1400d1c70 0x7fd14006bd20 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fd13400c850 tx=0x7fd13400cc10 comp rx=0 tx=0).stop 2023-11-08T19:27:38.622 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.620+0000 7fd146a1f700 1 -- 172.21.15.143:0/1457992900 shutdown_connections 2023-11-08T19:27:38.622 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.620+0000 7fd146a1f700 1 --2- 172.21.15.143:0/1457992900 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd12c06af00 0x7fd12c06d3c0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:38.622 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.620+0000 7fd146a1f700 1 --2- 172.21.15.143:0/1457992900 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd14015ba30 0x7fd14006d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:38.623 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.620+0000 7fd146a1f700 1 --2- 172.21.15.143:0/1457992900 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd1400b14c0 0x7fd1400c7ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:38.623 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.620+0000 7fd146a1f700 1 --2- 172.21.15.143:0/1457992900 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd1400d1c70 0x7fd14006bd20 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:38.623 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.620+0000 7fd146a1f700 1 -- 172.21.15.143:0/1457992900 >> 172.21.15.143:0/1457992900 conn(0x7fd14008b000 msgr2=0x7fd1400aeaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:38.623 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.620+0000 7fd146a1f700 1 -- 172.21.15.143:0/1457992900 shutdown_connections 2023-11-08T19:27:38.623 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:38.620+0000 7fd146a1f700 1 -- 172.21.15.143:0/1457992900 wait complete. 2023-11-08T19:27:38.633 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-08T19:27:38.633 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-08T19:27:38.634 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi143.front.sepia.ceph.com 2023-11-08T19:27:38.634 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-08T19:27:38.634 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-08T19:27:38.635 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-08T19:27:38.635 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-08T19:27:38.635 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-08T19:27:38.635 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-08T19:27:38.636 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:27:38.636 DEBUG:teuthology.orchestra.run.smithi143:> ip addr 2023-11-08T19:27:38.654 INFO:teuthology.orchestra.run.smithi143.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-08T19:27:38.655 INFO:teuthology.orchestra.run.smithi143.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-08T19:27:38.655 INFO:teuthology.orchestra.run.smithi143.stdout: inet 127.0.0.1/8 scope host lo 2023-11-08T19:27:38.655 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:27:38.655 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 ::1/128 scope host 2023-11-08T19:27:38.655 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:27:38.656 INFO:teuthology.orchestra.run.smithi143.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-08T19:27:38.656 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:d9:3b:94 brd ff:ff:ff:ff:ff:ff 2023-11-08T19:27:38.656 INFO:teuthology.orchestra.run.smithi143.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-08T19:27:38.656 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:d9:3b:95 brd ff:ff:ff:ff:ff:ff 2023-11-08T19:27:38.656 INFO:teuthology.orchestra.run.smithi143.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-08T19:27:38.656 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:88:77:82 brd ff:ff:ff:ff:ff:ff 2023-11-08T19:27:38.657 INFO:teuthology.orchestra.run.smithi143.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-08T19:27:38.657 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:88:77:83 brd ff:ff:ff:ff:ff:ff 2023-11-08T19:27:38.657 INFO:teuthology.orchestra.run.smithi143.stdout: inet 172.21.15.143/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-08T19:27:38.657 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft 42209sec preferred_lft 42209sec 2023-11-08T19:27:38.657 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 fe80::e628:d259:2f76:52a6/64 scope link noprefixroute 2023-11-08T19:27:38.658 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:27:38.658 INFO:teuthology.orchestra.run.smithi143.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-08T19:27:38.658 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 46:e3:3e:66:77:ea brd ff:ff:ff:ff:ff:ff 2023-11-08T19:27:38.658 INFO:teuthology.orchestra.run.smithi143.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-08T19:27:38.658 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:27:38.658 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 fe80::749a:8aff:fe59:325c/64 scope link 2023-11-08T19:27:38.659 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:27:38.659 INFO:teuthology.orchestra.run.smithi143.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-08T19:27:38.659 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 46:e3:3e:66:77:ea brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-08T19:27:38.659 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 fe80::44e3:3eff:fe66:77ea/64 scope link 2023-11-08T19:27:38.659 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:27:38.660 INFO:teuthology.orchestra.run.smithi143.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-08T19:27:38.660 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether ba:92:23:3f:5e:c8 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-08T19:27:38.660 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 fe80::b892:23ff:fe3f:5ec8/64 scope link 2023-11-08T19:27:38.660 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:27:38.660 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:27:38.661 DEBUG:teuthology.orchestra.run.smithi143:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-08T19:27:38.719 INFO:teuthology.orchestra.run.smithi143.stdout:1 2023-11-08T19:27:38.720 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:27:38.721 DEBUG:teuthology.orchestra.run.smithi143:> route 2023-11-08T19:27:38.779 INFO:teuthology.orchestra.run.smithi143.stdout:Kernel IP routing table 2023-11-08T19:27:38.780 INFO:teuthology.orchestra.run.smithi143.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-08T19:27:38.780 INFO:teuthology.orchestra.run.smithi143.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-08T19:27:38.780 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-08T19:27:38.780 INFO:teuthology.orchestra.run.smithi143.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-08T19:27:38.781 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:27:38.781 DEBUG:teuthology.orchestra.run.smithi143:> (cd / && exec bash -c ' 2023-11-08T19:27:38.782 DEBUG:teuthology.orchestra.run.smithi143:> set -e 2023-11-08T19:27:38.782 DEBUG:teuthology.orchestra.run.smithi143:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-08T19:27:38.783 DEBUG:teuthology.orchestra.run.smithi143:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-08T19:27:38.783 DEBUG:teuthology.orchestra.run.smithi143:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-08T19:27:38.784 DEBUG:teuthology.orchestra.run.smithi143:> ') 2023-11-08T19:27:38.884 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:27:38.885 DEBUG:teuthology.orchestra.run.smithi143:> ip netns list 2023-11-08T19:27:38.902 INFO:teuthology.orchestra.run.smithi143.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-08T19:27:38.903 INFO:teuthology.orchestra.run.smithi143.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-08T19:27:38.903 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-08T19:27:38.903 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:27:38.904 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip link set brx.0 up 2023-11-08T19:27:38.968 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-08T19:27:38.969 DEBUG:teuthology.orchestra.run.smithi143:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-08T19:27:39.029 INFO:teuthology.orchestra.run.smithi143.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-08T19:27:39.029 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-08T19:27:39.030 DEBUG:teuthology.orchestra.run.smithi143:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-08T19:27:39.085 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-11-08T19:27:39.086 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:27:39.086 DEBUG:teuthology.orchestra.run.smithi143:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-08T19:27:39.172 INFO:teuthology.orchestra.run.smithi143.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-08T19:27:39.172 INFO:teuthology.orchestra.run.smithi143.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-11-08T19:27:39.172 INFO:teuthology.orchestra.run.smithi143.stdout:invalid new device string format 2023-11-08T19:27:39.180 INFO:teuthology.orchestra.run.smithi143.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-08T19:27:39.180 INFO:teuthology.orchestra.run.smithi143.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-11-08T19:27:39.181 INFO:teuthology.orchestra.run.smithi143.stdout:invalid new device string format 2023-11-08T19:27:39.181 INFO:teuthology.orchestra.run.smithi143.stdout:mount.ceph: resolved to: "172.21.15.143:3300,172.21.15.174:3300,172.21.15.174:3301" 2023-11-08T19:27:39.181 INFO:teuthology.orchestra.run.smithi143.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-08T19:27:39.255 INFO:teuthology.orchestra.run.smithi143.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-08T19:27:39.255 INFO:teuthology.orchestra.run.smithi143.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-11-08T19:27:39.256 INFO:teuthology.orchestra.run.smithi143.stdout:invalid new device string format 2023-11-08T19:27:39.256 INFO:teuthology.orchestra.run.smithi143.stdout:mount.ceph: resolved to: "172.21.15.143:3300,172.21.15.174:3300,172.21.15.174:3301" 2023-11-08T19:27:39.256 INFO:teuthology.orchestra.run.smithi143.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-08T19:27:39.256 INFO:teuthology.orchestra.run.smithi143.stdout:mount.ceph: resolved to: "172.21.15.143:6789,172.21.15.174:6789,172.21.15.174:6790" 2023-11-08T19:27:39.257 INFO:teuthology.orchestra.run.smithi143.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.143:6789,172.21.15.174:6789,172.21.15.174:6790:/ 2023-11-08T19:27:39.257 INFO:teuthology.orchestra.run.smithi143.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=968483af-77fc-4068-ad7c-5c2a1a4613d7" will pass to kernel 2023-11-08T19:27:39.258 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-08T19:27:39.258 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:27:39.259 DEBUG:teuthology.orchestra.run.smithi143:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-08T19:27:39.301 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:27:39.301 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-08T19:27:39.398 INFO:teuthology.orchestra.run.smithi143.stdout:client5588 2023-11-08T19:27:39.399 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-08T19:27:39.399 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:27:39.420 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config get mds_max_snaps_per_dir 2023-11-08T19:27:39.602 INFO:teuthology.orchestra.run.smithi143.stdout:{"mds_max_snaps_per_dir":"100"} 2023-11-08T19:27:39.602 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_max_snaps_per_dir": "100" } 2023-11-08T19:27:39.603 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:27:39.603 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir test-allow-snaps) 2023-11-08T19:27:39.631 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_new_snaps false --yes-i-really-mean-it 2023-11-08T19:27:39.799 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:39.797+0000 7f8120936700 1 Processor -- start 2023-11-08T19:27:39.799 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:39.797+0000 7f8120936700 1 -- start start 2023-11-08T19:27:39.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:39.798+0000 7f8120936700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f811c15dba0 0x7f811c06c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:39.804 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:39.798+0000 7f8120936700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f811c06b240 0x7f811c0caff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:39.804 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:39.798+0000 7f8120936700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f811c161890 con 0x7f811c0cb530 2023-11-08T19:27:39.804 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:39.798+0000 7f8120936700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f811c06b740 con 0x7f811c15dba0 2023-11-08T19:27:39.804 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:39.798+0000 7f8120936700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f811c06ba20 con 0x7f811c06b240 2023-11-08T19:27:39.804 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:39.798+0000 7f811ad9d700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f811c0cb530 0x7f811c0b22e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:49958/0 (socket says 172.21.15.143:49958) 2023-11-08T19:27:39.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:39.798+0000 7f811ad9d700 1 -- 172.21.15.143:0/3258908653 learned_addr learned my addr 172.21.15.143:0/3258908653 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:27:39.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:39.798+0000 7f8119d9b700 1 --2- 172.21.15.143:0/3258908653 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f811c06b240 0x7f811c0caff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:39.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:39.798+0000 7f811a59c700 1 --2- 172.21.15.143:0/3258908653 >> v2:172.21.15.174:3300/0 conn(0x7f811c15dba0 0x7f811c06c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:39.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:39.798+0000 7f811959a700 1 -- 172.21.15.143:0/3258908653 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 810407933 0 0) 0x7f811c161890 con 0x7f811c0cb530 2023-11-08T19:27:39.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:39.798+0000 7f811959a700 1 -- 172.21.15.143:0/3258908653 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81040035b0 con 0x7f811c0cb530 2023-11-08T19:27:39.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:39.798+0000 7f811959a700 1 -- 172.21.15.143:0/3258908653 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1015796088 0 0) 0x7f81040035b0 con 0x7f811c0cb530 2023-11-08T19:27:39.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:39.798+0000 7f811959a700 1 -- 172.21.15.143:0/3258908653 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f811c06b240 msgr2=0x7f811c0caff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:39.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:39.798+0000 7f811959a700 1 --2- 172.21.15.143:0/3258908653 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f811c06b240 0x7f811c0caff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:39.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:39.798+0000 7f811959a700 1 -- 172.21.15.143:0/3258908653 >> v2:172.21.15.174:3300/0 conn(0x7f811c15dba0 msgr2=0x7f811c06c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:39.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:39.798+0000 7f811959a700 1 --2- 172.21.15.143:0/3258908653 >> v2:172.21.15.174:3300/0 conn(0x7f811c15dba0 0x7f811c06c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:39.807 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:39.798+0000 7f811959a700 1 -- 172.21.15.143:0/3258908653 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f811c06bd00 con 0x7f811c0cb530 2023-11-08T19:27:39.807 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:39.798+0000 7f811959a700 1 -- 172.21.15.143:0/3258908653 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4147756303 0 0) 0x7f81140019f0 con 0x7f811c0cb530 2023-11-08T19:27:39.807 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:39.798+0000 7f811959a700 1 -- 172.21.15.143:0/3258908653 >> v1:172.21.15.143:6789/0 conn(0x7f811c0cb530 legacy=0x7f811c0b22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:39.807 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:39.798+0000 7f811959a700 1 --2- 172.21.15.143:0/3258908653 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8104003a00 0x7f8104003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:39.807 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:39.798+0000 7f811959a700 1 --2- 172.21.15.143:0/3258908653 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f811c06b240 0x7f8104004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:39.807 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:39.798+0000 7f811959a700 1 --2- 172.21.15.143:0/3258908653 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8104004d90 0x7f8104005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:39.808 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:39.798+0000 7f811959a700 1 -- 172.21.15.143:0/3258908653 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f811c06b740 con 0x7f8104004d90 2023-11-08T19:27:39.808 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:39.798+0000 7f811959a700 1 -- 172.21.15.143:0/3258908653 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f811c06ba20 con 0x7f8104003a00 2023-11-08T19:27:39.808 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:39.798+0000 7f811959a700 1 -- 172.21.15.143:0/3258908653 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f810400ba10 con 0x7f811c06b240 2023-11-08T19:27:39.808 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:39.798+0000 7f811ad9d700 1 --2- 172.21.15.143:0/3258908653 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8104004d90 0x7f8104005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:39.808 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:39.799+0000 7f8119d9b700 1 --2- 172.21.15.143:0/3258908653 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f811c06b240 0x7f8104004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:39.809 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:39.799+0000 7f811a59c700 1 --2- 172.21.15.143:0/3258908653 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8104003a00 0x7f8104003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:39.809 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:39.799+0000 7f811ad9d700 1 -- 172.21.15.143:0/3258908653 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f811c06b240 msgr2=0x7f8104004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:39.809 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:39.799+0000 7f811ad9d700 1 --2- 172.21.15.143:0/3258908653 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f811c06b240 0x7f8104004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:39.809 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:39.799+0000 7f811ad9d700 1 -- 172.21.15.143:0/3258908653 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8104003a00 msgr2=0x7f8104003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:39.810 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:39.799+0000 7f811ad9d700 1 --2- 172.21.15.143:0/3258908653 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8104003a00 0x7f8104003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:39.810 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:39.799+0000 7f811ad9d700 1 -- 172.21.15.143:0/3258908653 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81040035b0 con 0x7f8104004d90 2023-11-08T19:27:39.810 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:39.799+0000 7f811ad9d700 1 --2- 172.21.15.143:0/3258908653 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8104004d90 0x7f8104005210 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f811400a830 tx=0x7f8104005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62beb75c22f6c220 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:39.810 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:39.799+0000 7f811959a700 1 -- 172.21.15.143:0/3258908653 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8114012600 con 0x7f8104004d90 2023-11-08T19:27:39.810 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:39.799+0000 7f811959a700 1 -- 172.21.15.143:0/3258908653 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81140127a0 con 0x7f8104004d90 2023-11-08T19:27:39.811 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:39.799+0000 7f811959a700 1 -- 172.21.15.143:0/3258908653 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8114012b50 con 0x7f8104004d90 2023-11-08T19:27:39.811 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:39.799+0000 7f8120936700 1 -- 172.21.15.143:0/3258908653 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8104004d90 msgr2=0x7f8104005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:39.811 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:39.799+0000 7f8120936700 1 --2- 172.21.15.143:0/3258908653 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8104004d90 0x7f8104005210 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f811400a830 tx=0x7f8104005880 comp rx=0 tx=0).stop 2023-11-08T19:27:39.811 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:39.800+0000 7f811ad9d700 1 -- 172.21.15.143:0/3258908653 reap_dead start 2023-11-08T19:27:39.811 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:39.800+0000 7f8120936700 1 -- 172.21.15.143:0/3258908653 shutdown_connections 2023-11-08T19:27:39.812 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:39.800+0000 7f8120936700 1 -- 172.21.15.143:0/3258908653 >> 172.21.15.143:0/3258908653 conn(0x7f811c0cd2e0 msgr2=0x7f811c0cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:39.812 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:39.800+0000 7f8120936700 1 -- 172.21.15.143:0/3258908653 shutdown_connections 2023-11-08T19:27:39.812 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:39.800+0000 7f8120936700 1 -- 172.21.15.143:0/3258908653 wait complete. 2023-11-08T19:27:39.812 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:39.801+0000 7f8120936700 1 Processor -- start 2023-11-08T19:27:39.812 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:39.801+0000 7f8120936700 1 -- start start 2023-11-08T19:27:39.815 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:39.802+0000 7f8120936700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8104004d90 0x7f811c15e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:39.815 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:39.802+0000 7f8120936700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f811c0c4390 0x7f811c0b2220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:39.816 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:39.802+0000 7f8120936700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f811c158aa0 0x7f811c158f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:39.816 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:39.802+0000 7f8120936700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f811c15a700 con 0x7f8104004d90 2023-11-08T19:27:39.816 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:39.802+0000 7f8120936700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f811c159460 con 0x7f811c0c4390 2023-11-08T19:27:39.816 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:39.802+0000 7f8120936700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f811c160760 con 0x7f811c158aa0 2023-11-08T19:27:39.816 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:39.802+0000 7f811a59c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8104004d90 0x7f811c15e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:39.817 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:39.802+0000 7f811a59c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8104004d90 0x7f811c15e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53814/0 (socket says 172.21.15.143:53814) 2023-11-08T19:27:39.817 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:39.802+0000 7f811a59c700 1 -- 172.21.15.143:0/1427135142 learned_addr learned my addr 172.21.15.143:0/1427135142 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:39.817 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:39.802+0000 7f811ad9d700 1 --2- 172.21.15.143:0/1427135142 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f811c158aa0 0x7f811c158f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:39.817 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:39.802+0000 7f8119d9b700 1 --2- 172.21.15.143:0/1427135142 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f811c0c4390 0x7f811c0b2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:39.817 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:39.802+0000 7f811a59c700 1 -- 172.21.15.143:0/1427135142 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f811c158aa0 msgr2=0x7f811c158f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:39.818 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:39.802+0000 7f811a59c700 1 --2- 172.21.15.143:0/1427135142 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f811c158aa0 0x7f811c158f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:39.818 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:39.802+0000 7f811a59c700 1 -- 172.21.15.143:0/1427135142 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f811c0c4390 msgr2=0x7f811c0b2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:39.818 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:39.802+0000 7f811a59c700 1 --2- 172.21.15.143:0/1427135142 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f811c0c4390 0x7f811c0b2220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:39.818 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:39.802+0000 7f811a59c700 1 -- 172.21.15.143:0/1427135142 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f811c160a40 con 0x7f8104004d90 2023-11-08T19:27:39.818 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:39.802+0000 7f8119d9b700 1 --2- 172.21.15.143:0/1427135142 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f811c0c4390 0x7f811c0b2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:27:39.819 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:39.802+0000 7f811a59c700 1 --2- 172.21.15.143:0/1427135142 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8104004d90 0x7f811c15e890 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f810c011ad0 tx=0x7f810c011de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:39.819 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:39.802+0000 7f810affd700 1 -- 172.21.15.143:0/1427135142 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f810c00f940 con 0x7f8104004d90 2023-11-08T19:27:39.819 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:39.802+0000 7f810affd700 1 -- 172.21.15.143:0/1427135142 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f810c00fac0 con 0x7f8104004d90 2023-11-08T19:27:39.819 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:39.802+0000 7f8120936700 1 -- 172.21.15.143:0/1427135142 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f811c160d20 con 0x7f8104004d90 2023-11-08T19:27:39.819 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:39.802+0000 7f810affd700 1 -- 172.21.15.143:0/1427135142 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f810c00fe70 con 0x7f8104004d90 2023-11-08T19:27:39.820 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:39.802+0000 7f8120936700 1 -- 172.21.15.143:0/1427135142 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f811c161000 con 0x7f8104004d90 2023-11-08T19:27:39.820 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:39.803+0000 7f8120936700 1 -- 172.21.15.143:0/1427135142 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f811c06bfc0 con 0x7f8104004d90 2023-11-08T19:27:39.820 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:39.804+0000 7f810affd700 1 -- 172.21.15.143:0/1427135142 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f810c0273f0 con 0x7f8104004d90 2023-11-08T19:27:39.820 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:39.805+0000 7f810affd700 1 --2- 172.21.15.143:0/1427135142 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f80fc06af00 0x7f80fc06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:39.820 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:39.805+0000 7f810affd700 1 -- 172.21.15.143:0/1427135142 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6888+0+0 (secure 0 0 0) 0x7f810c08a250 con 0x7f8104004d90 2023-11-08T19:27:39.821 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:39.805+0000 7f8119d9b700 1 --2- 172.21.15.143:0/1427135142 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f80fc06af00 0x7f80fc06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:39.821 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:39.805+0000 7f8119d9b700 1 --2- 172.21.15.143:0/1427135142 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f80fc06af00 0x7f80fc06d3c0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f811c0b5c10 tx=0x7f8110008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:39.821 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:39.808+0000 7f810affd700 1 -- 172.21.15.143:0/1427135142 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f810c055000 con 0x7f8104004d90 2023-11-08T19:27:40.044 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:40.042+0000 7f8120936700 1 -- 172.21.15.143:0/1427135142 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "false", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f811c06bfc0 con 0x7f8104004d90 2023-11-08T19:27:40.361 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:40.358+0000 7f810affd700 1 -- 172.21.15.143:0/1427135142 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "false", "yes_i_really_mean_it": true}]=0 disabled new snapshots v42) v1 ==== 169+0+0 (secure 0 0 0) 0x7f811c06bfc0 con 0x7f8104004d90 2023-11-08T19:27:40.363 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:40.361+0000 7f8120936700 1 -- 172.21.15.143:0/1427135142 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f80fc06af00 msgr2=0x7f80fc06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:40.364 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:40.361+0000 7f8120936700 1 --2- 172.21.15.143:0/1427135142 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f80fc06af00 0x7f80fc06d3c0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f811c0b5c10 tx=0x7f8110008040 comp rx=0 tx=0).stop 2023-11-08T19:27:40.364 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:40.361+0000 7f8120936700 1 -- 172.21.15.143:0/1427135142 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8104004d90 msgr2=0x7f811c15e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:40.364 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:40.361+0000 7f8120936700 1 --2- 172.21.15.143:0/1427135142 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8104004d90 0x7f811c15e890 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f810c011ad0 tx=0x7f810c011de0 comp rx=0 tx=0).stop 2023-11-08T19:27:40.364 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:40.361+0000 7f8120936700 1 -- 172.21.15.143:0/1427135142 shutdown_connections 2023-11-08T19:27:40.365 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:40.361+0000 7f8120936700 1 --2- 172.21.15.143:0/1427135142 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f80fc06af00 0x7f80fc06d3c0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:40.365 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:40.361+0000 7f8120936700 1 --2- 172.21.15.143:0/1427135142 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f811c158aa0 0x7f811c158f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:40.365 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:40.361+0000 7f8120936700 1 --2- 172.21.15.143:0/1427135142 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8104004d90 0x7f811c15e890 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:40.365 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:40.361+0000 7f8120936700 1 --2- 172.21.15.143:0/1427135142 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f811c0c4390 0x7f811c0b2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:40.365 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:40.361+0000 7f8120936700 1 -- 172.21.15.143:0/1427135142 >> 172.21.15.143:0/1427135142 conn(0x7f811c0cd2e0 msgr2=0x7f811c0c7930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:40.366 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:40.361+0000 7f8120936700 1 -- 172.21.15.143:0/1427135142 shutdown_connections 2023-11-08T19:27:40.366 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:40.361+0000 7f8120936700 1 -- 172.21.15.143:0/1427135142 wait complete. 2023-11-08T19:27:40.366 INFO:teuthology.orchestra.run.smithi143.stderr:disabled new snapshots 2023-11-08T19:27:40.376 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:27:40.377 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir test-allow-snaps/.snap/snap00) 2023-11-08T19:27:40.395 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-08T19:27:40.396 INFO:teuthology.orchestra.run.smithi143.stderr:mkdir: cannot create directory ‘test-allow-snaps/.snap/snap00’: Operation not permitted 2023-11-08T19:27:40.396 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_new_snaps true --yes-i-really-mean-it 2023-11-08T19:27:40.561 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:40.559+0000 7f392f1f8700 1 Processor -- start 2023-11-08T19:27:40.562 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:40.559+0000 7f392f1f8700 1 -- start start 2023-11-08T19:27:40.564 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:40.560+0000 7f392f1f8700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f392815dba0 0x7f392806c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:40.565 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:40.560+0000 7f392f1f8700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f392806b240 0x7f39280caff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:40.565 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:40.560+0000 7f392f1f8700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3928161890 con 0x7f39280cb530 2023-11-08T19:27:40.565 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:40.560+0000 7f392f1f8700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f392806b740 con 0x7f392806b240 2023-11-08T19:27:40.565 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:40.560+0000 7f392f1f8700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f392806ba20 con 0x7f392815dba0 2023-11-08T19:27:40.565 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:40.560+0000 7f3927fff700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f392806b240 0x7f39280caff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:40.566 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:40.560+0000 7f392d795700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f39280cb530 0x7f39280b22e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:49980/0 (socket says 172.21.15.143:49980) 2023-11-08T19:27:40.566 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:40.560+0000 7f392d795700 1 -- 172.21.15.143:0/2085151018 learned_addr learned my addr 172.21.15.143:0/2085151018 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:27:40.566 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:40.560+0000 7f392cf94700 1 --2- 172.21.15.143:0/2085151018 >> v2:172.21.15.174:3300/0 conn(0x7f392815dba0 0x7f392806c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:40.566 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:40.560+0000 7f3927fff700 1 -- 172.21.15.143:0/2085151018 >> v2:172.21.15.174:3300/0 conn(0x7f392815dba0 msgr2=0x7f392806c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:40.566 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:40.560+0000 7f3927fff700 1 --2- 172.21.15.143:0/2085151018 >> v2:172.21.15.174:3300/0 conn(0x7f392815dba0 0x7f392806c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:40.567 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:40.560+0000 7f3927fff700 1 -- 172.21.15.143:0/2085151018 >> v1:172.21.15.143:6789/0 conn(0x7f39280cb530 legacy=0x7f39280b22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:40.567 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:40.560+0000 7f39277fe700 1 -- 172.21.15.143:0/2085151018 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1226315303 0 0) 0x7f3928161890 con 0x7f39280cb530 2023-11-08T19:27:40.567 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:40.560+0000 7f3927fff700 1 -- 172.21.15.143:0/2085151018 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f392806bd00 con 0x7f392806b240 2023-11-08T19:27:40.567 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:40.560+0000 7f3927fff700 1 --2- 172.21.15.143:0/2085151018 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f392806b240 0x7f39280caff0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f3928067460 tx=0x7f3914009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39d56abd6d5091e6 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:40.568 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:40.561+0000 7f39277fe700 1 -- 172.21.15.143:0/2085151018 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3914011830 con 0x7f392806b240 2023-11-08T19:27:40.568 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:40.561+0000 7f39277fe700 1 -- 172.21.15.143:0/2085151018 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3914011990 con 0x7f392806b240 2023-11-08T19:27:40.568 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:40.561+0000 7f39277fe700 1 -- 172.21.15.143:0/2085151018 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3914011d20 con 0x7f392806b240 2023-11-08T19:27:40.568 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:40.561+0000 7f392f1f8700 1 -- 172.21.15.143:0/2085151018 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f392806b240 msgr2=0x7f39280caff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:40.568 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:40.561+0000 7f392f1f8700 1 --2- 172.21.15.143:0/2085151018 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f392806b240 0x7f39280caff0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f3928067460 tx=0x7f3914009b20 comp rx=0 tx=0).stop 2023-11-08T19:27:40.569 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:40.561+0000 7f392f1f8700 1 -- 172.21.15.143:0/2085151018 shutdown_connections 2023-11-08T19:27:40.569 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:40.561+0000 7f392f1f8700 1 --2- 172.21.15.143:0/2085151018 >> v2:172.21.15.174:3300/0 conn(0x7f392815dba0 0x7f392806c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:40.569 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:40.561+0000 7f392f1f8700 1 --2- 172.21.15.143:0/2085151018 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f392806b240 0x7f39280caff0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:40.569 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:40.561+0000 7f392f1f8700 1 -- 172.21.15.143:0/2085151018 >> 172.21.15.143:0/2085151018 conn(0x7f39280cd2e0 msgr2=0x7f39280cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:40.569 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:40.561+0000 7f392f1f8700 1 -- 172.21.15.143:0/2085151018 shutdown_connections 2023-11-08T19:27:40.570 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:40.561+0000 7f392f1f8700 1 -- 172.21.15.143:0/2085151018 wait complete. 2023-11-08T19:27:40.570 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:40.562+0000 7f392f1f8700 1 Processor -- start 2023-11-08T19:27:40.570 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:40.562+0000 7f392f1f8700 1 -- start start 2023-11-08T19:27:40.573 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:40.563+0000 7f392f1f8700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f392806b240 0x7f39280b3180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:40.573 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:40.563+0000 7f392f1f8700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f39280cb530 0x7f39280af360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:40.573 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:40.563+0000 7f392f1f8700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f392815dba0 0x7f39280b39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:40.573 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:40.563+0000 7f392f1f8700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39280c57f0 con 0x7f39280cb530 2023-11-08T19:27:40.574 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:40.563+0000 7f392f1f8700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39281f6d20 con 0x7f392815dba0 2023-11-08T19:27:40.574 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:40.563+0000 7f392f1f8700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39281f7000 con 0x7f392806b240 2023-11-08T19:27:40.574 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:40.563+0000 7f3927fff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f39280cb530 0x7f39280af360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:40.574 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:40.563+0000 7f3927fff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f39280cb530 0x7f39280af360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53830/0 (socket says 172.21.15.143:53830) 2023-11-08T19:27:40.574 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:40.563+0000 7f3927fff700 1 -- 172.21.15.143:0/579779538 learned_addr learned my addr 172.21.15.143:0/579779538 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:40.575 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:40.563+0000 7f392d795700 1 --2- 172.21.15.143:0/579779538 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f392815dba0 0x7f39280b39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:40.575 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:40.564+0000 7f392cf94700 1 --2- 172.21.15.143:0/579779538 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f392806b240 0x7f39280b3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:40.575 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:40.564+0000 7f3927fff700 1 -- 172.21.15.143:0/579779538 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f392806b240 msgr2=0x7f39280b3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:40.575 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:40.564+0000 7f3927fff700 1 --2- 172.21.15.143:0/579779538 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f392806b240 0x7f39280b3180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:40.575 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:40.564+0000 7f3927fff700 1 -- 172.21.15.143:0/579779538 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f392815dba0 msgr2=0x7f39280b39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:40.576 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:40.564+0000 7f3927fff700 1 --2- 172.21.15.143:0/579779538 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f392815dba0 0x7f39280b39b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:40.576 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:40.564+0000 7f3927fff700 1 -- 172.21.15.143:0/579779538 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39140094a0 con 0x7f39280cb530 2023-11-08T19:27:40.576 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:40.564+0000 7f392cf94700 1 --2- 172.21.15.143:0/579779538 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f392806b240 0x7f39280b3180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:27:40.576 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:40.564+0000 7f392d795700 1 --2- 172.21.15.143:0/579779538 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f392815dba0 0x7f39280b39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:27:40.576 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:40.564+0000 7f3927fff700 1 --2- 172.21.15.143:0/579779538 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f39280cb530 0x7f39280af360 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f3914000c00 tx=0x7f3914004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:40.577 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:40.564+0000 7f39257fa700 1 -- 172.21.15.143:0/579779538 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39140038a0 con 0x7f39280cb530 2023-11-08T19:27:40.577 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:40.564+0000 7f39257fa700 1 -- 172.21.15.143:0/579779538 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f391401f070 con 0x7f39280cb530 2023-11-08T19:27:40.577 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:40.564+0000 7f392f1f8700 1 -- 172.21.15.143:0/579779538 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39281f72e0 con 0x7f39280cb530 2023-11-08T19:27:40.577 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:40.564+0000 7f39257fa700 1 -- 172.21.15.143:0/579779538 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3914003c40 con 0x7f39280cb530 2023-11-08T19:27:40.577 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:40.564+0000 7f392f1f8700 1 -- 172.21.15.143:0/579779538 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39280b36c0 con 0x7f39280cb530 2023-11-08T19:27:40.578 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:40.565+0000 7f392f1f8700 1 -- 172.21.15.143:0/579779538 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39280c5960 con 0x7f39280cb530 2023-11-08T19:27:40.578 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:40.565+0000 7f39257fa700 1 -- 172.21.15.143:0/579779538 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3914026480 con 0x7f39280cb530 2023-11-08T19:27:40.578 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:40.566+0000 7f39257fa700 1 --2- 172.21.15.143:0/579779538 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f390c06af00 0x7f390c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:40.579 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:40.566+0000 7f39257fa700 1 -- 172.21.15.143:0/579779538 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6888+0+0 (secure 0 0 0) 0x7f3914021070 con 0x7f39280cb530 2023-11-08T19:27:40.579 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:40.566+0000 7f392cf94700 1 --2- 172.21.15.143:0/579779538 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f390c06af00 0x7f390c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:40.579 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:40.566+0000 7f392cf94700 1 --2- 172.21.15.143:0/579779538 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f390c06af00 0x7f390c06d3c0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f391c00a9b0 tx=0x7f391c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:40.580 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:40.570+0000 7f39257fa700 1 -- 172.21.15.143:0/579779538 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3914056170 con 0x7f39280cb530 2023-11-08T19:27:40.808 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:40.806+0000 7f392f1f8700 1 -- 172.21.15.143:0/579779538 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f392804eee0 con 0x7f39280cb530 2023-11-08T19:27:41.366 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:41.363+0000 7f39257fa700 1 -- 172.21.15.143:0/579779538 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true}]=0 enabled new snapshots v43) v1 ==== 167+0+0 (secure 0 0 0) 0x7f3914059790 con 0x7f39280cb530 2023-11-08T19:27:41.370 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:41.367+0000 7f392f1f8700 1 -- 172.21.15.143:0/579779538 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f390c06af00 msgr2=0x7f390c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:41.370 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:41.367+0000 7f392f1f8700 1 --2- 172.21.15.143:0/579779538 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f390c06af00 0x7f390c06d3c0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f391c00a9b0 tx=0x7f391c008040 comp rx=0 tx=0).stop 2023-11-08T19:27:41.370 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:41.367+0000 7f392f1f8700 1 -- 172.21.15.143:0/579779538 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f39280cb530 msgr2=0x7f39280af360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:41.370 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:41.367+0000 7f392f1f8700 1 --2- 172.21.15.143:0/579779538 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f39280cb530 0x7f39280af360 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f3914000c00 tx=0x7f3914004750 comp rx=0 tx=0).stop 2023-11-08T19:27:41.370 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:41.367+0000 7f392f1f8700 1 -- 172.21.15.143:0/579779538 shutdown_connections 2023-11-08T19:27:41.371 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:41.367+0000 7f392f1f8700 1 --2- 172.21.15.143:0/579779538 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f390c06af00 0x7f390c06d3c0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:41.371 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:41.367+0000 7f392f1f8700 1 --2- 172.21.15.143:0/579779538 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f392806b240 0x7f39280b3180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:41.371 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:41.367+0000 7f392f1f8700 1 --2- 172.21.15.143:0/579779538 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f392815dba0 0x7f39280b39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:41.371 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:41.367+0000 7f392f1f8700 1 --2- 172.21.15.143:0/579779538 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f39280cb530 0x7f39280af360 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:41.371 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:41.367+0000 7f392f1f8700 1 -- 172.21.15.143:0/579779538 >> 172.21.15.143:0/579779538 conn(0x7f39280cd2e0 msgr2=0x7f392815cd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:41.372 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:41.367+0000 7f392f1f8700 1 -- 172.21.15.143:0/579779538 shutdown_connections 2023-11-08T19:27:41.372 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:41.367+0000 7f392f1f8700 1 -- 172.21.15.143:0/579779538 wait complete. 2023-11-08T19:27:41.372 INFO:teuthology.orchestra.run.smithi143.stderr:enabled new snapshots 2023-11-08T19:27:41.383 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:27:41.384 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir test-allow-snaps/.snap/snap00) 2023-11-08T19:27:41.424 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:27:41.425 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir test-allow-snaps/.snap/snap00) 2023-11-08T19:27:41.488 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:27:41.489 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir test-allow-snaps) 2023-11-08T19:27:41.547 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-08T19:27:41.547 DEBUG:teuthology.orchestra.run.smithi143:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-08T19:27:41.633 DEBUG:teuthology.orchestra.run.smithi174:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-08T19:27:41.681 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-08T19:27:41.682 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-08T19:27:41.682 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:27:41.700 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-08T19:27:41.700 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:27:41.757 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:27:41.917 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:41.916+0000 7f96d5bd3700 1 Processor -- start 2023-11-08T19:27:41.917 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:41.916+0000 7f96d5bd3700 1 -- start start 2023-11-08T19:27:41.920 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:41.916+0000 7f96d5bd3700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f96d015aa30 0x7f96d00cf510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:41.920 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:41.916+0000 7f96d5bd3700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f96d00b2210 0x7f96d00b2690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:41.920 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:41.916+0000 7f96d5bd3700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96d00cf080 con 0x7f96d00cbe20 2023-11-08T19:27:41.920 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:41.916+0000 7f96d5bd3700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96d00b2bd0 con 0x7f96d00b2210 2023-11-08T19:27:41.920 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:41.916+0000 7f96d5bd3700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96d00ce470 con 0x7f96d015aa30 2023-11-08T19:27:41.921 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:41.916+0000 7f96ceffd700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f96d00b2210 0x7f96d00b2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:41.921 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:41.916+0000 7f96cf7fe700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f96d015aa30 0x7f96d00cf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:41.921 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:41.916+0000 7f96ceffd700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f96d00b2210 0x7f96d00b2690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.143:37336/0 (socket says 172.21.15.143:37336) 2023-11-08T19:27:41.921 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:41.916+0000 7f96cf7fe700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f96d015aa30 0x7f96d00cf510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.143:41474/0 (socket says 172.21.15.143:41474) 2023-11-08T19:27:41.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:41.916+0000 7f96ceffd700 1 -- 172.21.15.143:0/2067209202 learned_addr learned my addr 172.21.15.143:0/2067209202 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:41.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:41.916+0000 7f96cf7fe700 1 -- 172.21.15.143:0/2067209202 >> v2:172.21.15.174:3300/0 conn(0x7f96d00b2210 msgr2=0x7f96d00b2690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:41.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:41.916+0000 7f96cf7fe700 1 --2- 172.21.15.143:0/2067209202 >> v2:172.21.15.174:3300/0 conn(0x7f96d00b2210 0x7f96d00b2690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:41.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:41.916+0000 7f96cf7fe700 1 -- 172.21.15.143:0/2067209202 >> v1:172.21.15.174:6789/0 conn(0x7f96d00cbe20 legacy=0x7f96d00cc270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:41.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:41.916+0000 7f96cf7fe700 1 -- 172.21.15.143:0/2067209202 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96d00ce750 con 0x7f96d015aa30 2023-11-08T19:27:41.923 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:41.916+0000 7f96ceffd700 1 --2- 172.21.15.143:0/2067209202 >> v2:172.21.15.174:3300/0 conn(0x7f96d00b2210 0x7f96d00b2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T19:27:41.923 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:41.916+0000 7f96cf7fe700 1 --2- 172.21.15.143:0/2067209202 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f96d015aa30 0x7f96d00cf510 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f96c0009820 tx=0x7f96c0009b30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39dd8d2799470438 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:41.923 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:41.916+0000 7f96ce7fc700 1 -- 172.21.15.143:0/2067209202 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96c0005e30 con 0x7f96d015aa30 2023-11-08T19:27:41.923 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:41.917+0000 7f96ce7fc700 1 -- 172.21.15.143:0/2067209202 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96c0006030 con 0x7f96d015aa30 2023-11-08T19:27:41.923 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:41.917+0000 7f96ce7fc700 1 -- 172.21.15.143:0/2067209202 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96c00063c0 con 0x7f96d015aa30 2023-11-08T19:27:41.923 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:41.917+0000 7f96d5bd3700 1 -- 172.21.15.143:0/2067209202 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f96d015aa30 msgr2=0x7f96d00cf510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:41.924 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:41.917+0000 7f96d5bd3700 1 --2- 172.21.15.143:0/2067209202 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f96d015aa30 0x7f96d00cf510 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f96c0009820 tx=0x7f96c0009b30 comp rx=0 tx=0).stop 2023-11-08T19:27:41.924 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:41.917+0000 7f96d5bd3700 1 -- 172.21.15.143:0/2067209202 shutdown_connections 2023-11-08T19:27:41.924 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:41.917+0000 7f96d5bd3700 1 --2- 172.21.15.143:0/2067209202 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f96d015aa30 0x7f96d00cf510 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:41.924 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:41.917+0000 7f96d5bd3700 1 --2- 172.21.15.143:0/2067209202 >> v2:172.21.15.174:3300/0 conn(0x7f96d00b2210 0x7f96d00b2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:41.924 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:41.917+0000 7f96d5bd3700 1 -- 172.21.15.143:0/2067209202 >> 172.21.15.143:0/2067209202 conn(0x7f96d0099ab0 msgr2=0x7f96d00d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:41.925 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:41.917+0000 7f96d5bd3700 1 -- 172.21.15.143:0/2067209202 shutdown_connections 2023-11-08T19:27:41.925 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:41.917+0000 7f96d5bd3700 1 -- 172.21.15.143:0/2067209202 wait complete. 2023-11-08T19:27:41.927 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:41.918+0000 7f96d5bd3700 1 Processor -- start 2023-11-08T19:27:41.928 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:41.919+0000 7f96d5bd3700 1 -- start start 2023-11-08T19:27:41.928 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:41.919+0000 7f96d5bd3700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f96d00b2210 0x7f96d015b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:41.928 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:41.919+0000 7f96d5bd3700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f96d00cbe20 0x7f96d015d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:41.928 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:41.919+0000 7f96d5bd3700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f96d015aa30 0x7f96d014fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:41.928 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:41.919+0000 7f96d5bd3700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96d015ba90 con 0x7f96d015aa30 2023-11-08T19:27:41.929 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:41.919+0000 7f96d5bd3700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96d00ad760 con 0x7f96d00cbe20 2023-11-08T19:27:41.929 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:41.919+0000 7f96d5bd3700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96d00ada40 con 0x7f96d00b2210 2023-11-08T19:27:41.929 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:41.919+0000 7f96cffff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f96d015aa30 0x7f96d014fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:41.929 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:41.919+0000 7f96cffff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f96d015aa30 0x7f96d014fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53846/0 (socket says 172.21.15.143:53846) 2023-11-08T19:27:41.929 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:41.919+0000 7f96cffff700 1 -- 172.21.15.143:0/3314153574 learned_addr learned my addr 172.21.15.143:0/3314153574 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:41.930 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:41.919+0000 7f96cf7fe700 1 --2- 172.21.15.143:0/3314153574 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f96d00b2210 0x7f96d015b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:41.930 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:41.919+0000 7f96ceffd700 1 --2- 172.21.15.143:0/3314153574 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f96d00cbe20 0x7f96d015d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:41.930 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:41.920+0000 7f96cffff700 1 -- 172.21.15.143:0/3314153574 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f96d00b2210 msgr2=0x7f96d015b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:41.930 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:41.920+0000 7f96cffff700 1 --2- 172.21.15.143:0/3314153574 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f96d00b2210 0x7f96d015b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:41.930 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:41.920+0000 7f96cffff700 1 -- 172.21.15.143:0/3314153574 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f96d00cbe20 msgr2=0x7f96d015d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:41.931 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:41.920+0000 7f96cffff700 1 --2- 172.21.15.143:0/3314153574 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f96d00cbe20 0x7f96d015d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:41.931 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:41.920+0000 7f96cffff700 1 -- 172.21.15.143:0/3314153574 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96c4009550 con 0x7f96d015aa30 2023-11-08T19:27:41.931 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:41.920+0000 7f96cffff700 1 --2- 172.21.15.143:0/3314153574 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f96d015aa30 0x7f96d014fd10 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f96c800cc50 tx=0x7f96c800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:41.931 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:41.920+0000 7f96b7fff700 1 -- 172.21.15.143:0/3314153574 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96c8003c00 con 0x7f96d015aa30 2023-11-08T19:27:41.931 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:41.920+0000 7f96b7fff700 1 -- 172.21.15.143:0/3314153574 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96c8003d60 con 0x7f96d015aa30 2023-11-08T19:27:41.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:41.920+0000 7f96d5bd3700 1 -- 172.21.15.143:0/3314153574 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96c0009430 con 0x7f96d015aa30 2023-11-08T19:27:41.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:41.920+0000 7f96b7fff700 1 -- 172.21.15.143:0/3314153574 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96c800aab0 con 0x7f96d015aa30 2023-11-08T19:27:41.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:41.920+0000 7f96d5bd3700 1 -- 172.21.15.143:0/3314153574 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96d00ae000 con 0x7f96d015aa30 2023-11-08T19:27:41.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:41.921+0000 7f96d5bd3700 1 -- 172.21.15.143:0/3314153574 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96d004eee0 con 0x7f96d015aa30 2023-11-08T19:27:41.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:41.921+0000 7f96b7fff700 1 -- 172.21.15.143:0/3314153574 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f96c800ac10 con 0x7f96d015aa30 2023-11-08T19:27:41.933 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:41.922+0000 7f96b7fff700 1 --2- 172.21.15.143:0/3314153574 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f96bc06af00 0x7f96bc06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:41.933 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:41.922+0000 7f96b7fff700 1 -- 172.21.15.143:0/3314153574 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6888+0+0 (secure 0 0 0) 0x7f96c8089a50 con 0x7f96d015aa30 2023-11-08T19:27:41.933 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:41.922+0000 7f96cf7fe700 1 --2- 172.21.15.143:0/3314153574 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f96bc06af00 0x7f96bc06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:41.933 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:41.922+0000 7f96cf7fe700 1 --2- 172.21.15.143:0/3314153574 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f96bc06af00 0x7f96bc06d3c0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f96c0000c00 tx=0x7f96c0008070 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:41.934 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:41.927+0000 7f96b7fff700 1 -- 172.21.15.143:0/3314153574 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f96c8054660 con 0x7f96d015aa30 2023-11-08T19:27:42.118 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.116+0000 7f96d5bd3700 1 -- 172.21.15.143:0/3314153574 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f96d00c7590 con 0x7f96d015aa30 2023-11-08T19:27:42.119 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.118+0000 7f96b7fff700 1 -- 172.21.15.143:0/3314153574 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+21997 (secure 0 0 0) 0x7f96c8057c80 con 0x7f96d015aa30 2023-11-08T19:27:42.120 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:27:42.120 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":80,"fsid":"968483af-77fc-4068-ad7c-5c2a1a4613d7","created":"2023-11-08T19:24:32.826484+0000","modified":"2023-11-08T19:27:40.353576+0000","last_up_change":"2023-11-08T19:24:43.573880+0000","last_in_change":"2023-11-08T19:24:35.314525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:44.551530+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"snap-test-pool","create_time":"2023-11-08T19:26:02.874384+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":36,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"test_data_pool","create_time":"2023-11-08T19:27:05.601949+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":66,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_metadata_pool","create_time":"2023-11-08T19:27:06.536369+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-11-08T19:27:30.137037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-11-08T19:27:30.660318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bae98ec1-d94e-403d-a220-a522a53ea6f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6809","nonce":1487830566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6811","nonce":1487830566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6815","nonce":1487830566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6813","nonce":1487830566}]},"public_addr":"172.21.15.143:6809/1487830566","cluster_addr":"172.21.15.143:6811/1487830566","heartbeat_back_addr":"172.21.15.143:6815/1487830566","heartbeat_front_addr":"172.21.15.143:6813/1487830566","state":["exists","up"]},{"osd":1,"uuid":"5cb6538a-1ccb-4322-8633-09f66af0b7e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6801","nonce":787341397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6803","nonce":787341397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6807","nonce":787341397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6805","nonce":787341397}]},"public_addr":"172.21.15.143:6801/787341397","cluster_addr":"172.21.15.143:6803/787341397","heartbeat_back_addr":"172.21.15.143:6807/787341397","heartbeat_front_addr":"172.21.15.143:6805/787341397","state":["exists","up"]},{"osd":2,"uuid":"cdbf522c-60ef-4d60-9653-8dcb529c101a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6822","nonce":2301325597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6826","nonce":2301325597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6831","nonce":2301325597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6829","nonce":2301325597}]},"public_addr":"172.21.15.143:6822/2301325597","cluster_addr":"172.21.15.143:6826/2301325597","heartbeat_back_addr":"172.21.15.143:6831/2301325597","heartbeat_front_addr":"172.21.15.143:6829/2301325597","state":["exists","up"]},{"osd":3,"uuid":"4e2783cf-ebb9-4dd8-8245-bc1d17ce4325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6817","nonce":2021504428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6819","nonce":2021504428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6827","nonce":2021504428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6823","nonce":2021504428}]},"public_addr":"172.21.15.143:6817/2021504428","cluster_addr":"172.21.15.143:6819/2021504428","heartbeat_back_addr":"172.21.15.143:6827/2021504428","heartbeat_front_addr":"172.21.15.143:6823/2021504428","state":["exists","up"]},{"osd":4,"uuid":"691256fa-9b77-49b9-a346-7ae1adf0ca80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6801","nonce":1993380595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6803","nonce":1993380595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6807","nonce":1993380595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6805","nonce":1993380595}]},"public_addr":"172.21.15.174:6801/1993380595","cluster_addr":"172.21.15.174:6803/1993380595","heartbeat_back_addr":"172.21.15.174:6807/1993380595","heartbeat_front_addr":"172.21.15.174:6805/1993380595","state":["exists","up"]},{"osd":5,"uuid":"756fc352-48c9-46ce-b52e-0541fe6f8525","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6809","nonce":1777923303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6811","nonce":1777923303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6815","nonce":1777923303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6813","nonce":1777923303}]},"public_addr":"172.21.15.174:6809/1777923303","cluster_addr":"172.21.15.174:6811/1777923303","heartbeat_back_addr":"172.21.15.174:6815/1777923303","heartbeat_front_addr":"172.21.15.174:6813/1777923303","state":["exists","up"]},{"osd":6,"uuid":"fab2ac2b-ed97-4cde-9825-b480eabf8b8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6825","nonce":2755130316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6827","nonce":2755130316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6831","nonce":2755130316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6829","nonce":2755130316}]},"public_addr":"172.21.15.174:6825/2755130316","cluster_addr":"172.21.15.174:6827/2755130316","heartbeat_back_addr":"172.21.15.174:6831/2755130316","heartbeat_front_addr":"172.21.15.174:6829/2755130316","state":["exists","up"]},{"osd":7,"uuid":"3de9b797-2ab6-4d41-a863-7e7d638fd703","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6817","nonce":2847141411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6819","nonce":2847141411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6823","nonce":2847141411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6821","nonce":2847141411}]},"public_addr":"172.21.15.174:6817/2847141411","cluster_addr":"172.21.15.174:6819/2847141411","heartbeat_back_addr":"172.21.15.174:6823/2847141411","heartbeat_front_addr":"172.21.15.174:6821/2847141411","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.477076+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.496890+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.539305+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.454673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.538881+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.521294+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.520712+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.540404+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.a","mappings":[{"from":1,"to":7}]},{"pgid":"14.17","mappings":[{"from":1,"to":7}]},{"pgid":"14.29","mappings":[{"from":1,"to":2}]},{"pgid":"14.32","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:27:42.122 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.120+0000 7f96d5bd3700 1 -- 172.21.15.143:0/3314153574 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f96bc06af00 msgr2=0x7f96bc06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:42.122 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.120+0000 7f96d5bd3700 1 --2- 172.21.15.143:0/3314153574 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f96bc06af00 0x7f96bc06d3c0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f96c0000c00 tx=0x7f96c0008070 comp rx=0 tx=0).stop 2023-11-08T19:27:42.122 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.120+0000 7f96d5bd3700 1 -- 172.21.15.143:0/3314153574 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f96d015aa30 msgr2=0x7f96d014fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:42.123 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.120+0000 7f96d5bd3700 1 --2- 172.21.15.143:0/3314153574 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f96d015aa30 0x7f96d014fd10 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f96c800cc50 tx=0x7f96c800cf60 comp rx=0 tx=0).stop 2023-11-08T19:27:42.123 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.120+0000 7f96d5bd3700 1 -- 172.21.15.143:0/3314153574 shutdown_connections 2023-11-08T19:27:42.123 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.120+0000 7f96d5bd3700 1 --2- 172.21.15.143:0/3314153574 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f96bc06af00 0x7f96bc06d3c0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:42.123 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.120+0000 7f96d5bd3700 1 --2- 172.21.15.143:0/3314153574 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f96d00b2210 0x7f96d015b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:42.123 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.120+0000 7f96d5bd3700 1 --2- 172.21.15.143:0/3314153574 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f96d015aa30 0x7f96d014fd10 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:42.124 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.120+0000 7f96d5bd3700 1 --2- 172.21.15.143:0/3314153574 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f96d00cbe20 0x7f96d015d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:42.124 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.120+0000 7f96d5bd3700 1 -- 172.21.15.143:0/3314153574 >> 172.21.15.143:0/3314153574 conn(0x7f96d0099ab0 msgr2=0x7f96d006f320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:42.124 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.120+0000 7f96d5bd3700 1 -- 172.21.15.143:0/3314153574 shutdown_connections 2023-11-08T19:27:42.124 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.120+0000 7f96d5bd3700 1 -- 172.21.15.143:0/3314153574 wait complete. 2023-11-08T19:27:42.134 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T19:24:44.551530+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'snap-test-pool', 'create_time': '2023-11-08T19:26:02.874384+0000', 'flags': 16385, 'flags_names': 'hashpspool,pool_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '36', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'pool', 'snap_seq': 2, 'snap_epoch': 36, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'test_data_pool', 'create_time': '2023-11-08T19:27:05.601949+0000', 'flags': 16385, 'flags_names': 'hashpspool,pool_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'pool', 'snap_seq': 2, 'snap_epoch': 66, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'test_metadata_pool', 'create_time': '2023-11-08T19:27:06.536369+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-08T19:27:30.137037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2023-11-08T19:27:30.660318+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-08T19:27:42.135 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T19:27:42.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.258+0000 7f0abc650700 1 Processor -- start 2023-11-08T19:27:42.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.258+0000 7f0abc650700 1 -- start start 2023-11-08T19:27:42.261 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.258+0000 7f0abc650700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f0ab415dba0 0x7f0ab406c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:42.261 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.259+0000 7f0abc650700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0ab40b22e0 0x7f0ab40b2750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:42.261 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.259+0000 7f0abc650700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ab4161890 con 0x7f0ab406b240 2023-11-08T19:27:42.261 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.259+0000 7f0abc650700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ab40b2c90 con 0x7f0ab415dba0 2023-11-08T19:27:42.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.259+0000 7f0abc650700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab406b740 con 0x7f0ab40b22e0 2023-11-08T19:27:42.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.259+0000 7f0ababed700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0ab40b22e0 0x7f0ab40b2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:42.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.259+0000 7f0aba3ec700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f0ab415dba0 0x7f0ab406c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:42.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.259+0000 7f0ababed700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0ab40b22e0 0x7f0ab40b2750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.143:41488/0 (socket says 172.21.15.143:41488) 2023-11-08T19:27:42.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.259+0000 7f0aba3ec700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f0ab415dba0 0x7f0ab406c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.143:37348/0 (socket says 172.21.15.143:37348) 2023-11-08T19:27:42.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.259+0000 7f0ababed700 1 -- 172.21.15.143:0/1823325593 learned_addr learned my addr 172.21.15.143:0/1823325593 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:42.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.259+0000 7f0ababed700 1 -- 172.21.15.143:0/1823325593 >> v2:172.21.15.174:3300/0 conn(0x7f0ab415dba0 msgr2=0x7f0ab406c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:42.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.259+0000 7f0ababed700 1 --2- 172.21.15.143:0/1823325593 >> v2:172.21.15.174:3300/0 conn(0x7f0ab415dba0 0x7f0ab406c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:42.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.259+0000 7f0ababed700 1 -- 172.21.15.143:0/1823325593 >> v1:172.21.15.174:6789/0 conn(0x7f0ab406b240 legacy=0x7f0ab40caff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:42.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.259+0000 7f0ababed700 1 -- 172.21.15.143:0/1823325593 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ab406ba20 con 0x7f0ab40b22e0 2023-11-08T19:27:42.264 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.259+0000 7f0ababed700 1 --2- 172.21.15.143:0/1823325593 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0ab40b22e0 0x7f0ab40b2750 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0aac009e50 tx=0x7f0aac00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cfa653baa2f008e5 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:42.264 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.259+0000 7f0ab93ea700 1 -- 172.21.15.143:0/1823325593 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aac00e070 con 0x7f0ab40b22e0 2023-11-08T19:27:42.264 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.260+0000 7f0ab93ea700 1 -- 172.21.15.143:0/1823325593 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0aac00b910 con 0x7f0ab40b22e0 2023-11-08T19:27:42.264 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.260+0000 7f0ab93ea700 1 -- 172.21.15.143:0/1823325593 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aac00bcc0 con 0x7f0ab40b22e0 2023-11-08T19:27:42.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.260+0000 7f0abc650700 1 -- 172.21.15.143:0/1823325593 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0ab40b22e0 msgr2=0x7f0ab40b2750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:42.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.260+0000 7f0abc650700 1 --2- 172.21.15.143:0/1823325593 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0ab40b22e0 0x7f0ab40b2750 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0aac009e50 tx=0x7f0aac00b330 comp rx=0 tx=0).stop 2023-11-08T19:27:42.268 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.260+0000 7f0abc650700 1 -- 172.21.15.143:0/1823325593 shutdown_connections 2023-11-08T19:27:42.268 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.260+0000 7f0abc650700 1 --2- 172.21.15.143:0/1823325593 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0ab40b22e0 0x7f0ab40b2750 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:42.268 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.260+0000 7f0abc650700 1 --2- 172.21.15.143:0/1823325593 >> v2:172.21.15.174:3300/0 conn(0x7f0ab415dba0 0x7f0ab406c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:42.268 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.260+0000 7f0abc650700 1 -- 172.21.15.143:0/1823325593 >> 172.21.15.143:0/1823325593 conn(0x7f0ab40cd2e0 msgr2=0x7f0ab40cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:42.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.260+0000 7f0abc650700 1 -- 172.21.15.143:0/1823325593 shutdown_connections 2023-11-08T19:27:42.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.260+0000 7f0abc650700 1 -- 172.21.15.143:0/1823325593 wait complete. 2023-11-08T19:27:42.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.261+0000 7f0abc650700 1 Processor -- start 2023-11-08T19:27:42.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.261+0000 7f0abc650700 1 -- start start 2023-11-08T19:27:42.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.262+0000 7f0abc650700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0ab406b240 0x7f0ab415a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:42.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.262+0000 7f0abc650700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0ab40b22e0 0x7f0ab415aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:42.270 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.262+0000 7f0abc650700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0ab415dba0 0x7f0ab415b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:42.270 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.262+0000 7f0abc650700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab40c8090 con 0x7f0ab415dba0 2023-11-08T19:27:42.270 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.262+0000 7f0abc650700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab40ca4c0 con 0x7f0ab40b22e0 2023-11-08T19:27:42.270 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.262+0000 7f0abc650700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab40c5f60 con 0x7f0ab406b240 2023-11-08T19:27:42.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.262+0000 7f0ababed700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0ab415dba0 0x7f0ab415b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:42.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.262+0000 7f0ababed700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0ab415dba0 0x7f0ab415b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53858/0 (socket says 172.21.15.143:53858) 2023-11-08T19:27:42.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.262+0000 7f0ababed700 1 -- 172.21.15.143:0/1832998577 learned_addr learned my addr 172.21.15.143:0/1832998577 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:42.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.262+0000 7f0ab9beb700 1 --2- 172.21.15.143:0/1832998577 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0ab40b22e0 0x7f0ab415aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:42.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.262+0000 7f0aba3ec700 1 --2- 172.21.15.143:0/1832998577 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0ab406b240 0x7f0ab415a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:42.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.263+0000 7f0ababed700 1 -- 172.21.15.143:0/1832998577 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0ab406b240 msgr2=0x7f0ab415a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:42.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.263+0000 7f0ababed700 1 --2- 172.21.15.143:0/1832998577 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0ab406b240 0x7f0ab415a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:42.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.263+0000 7f0ababed700 1 -- 172.21.15.143:0/1832998577 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0ab40b22e0 msgr2=0x7f0ab415aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:42.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.263+0000 7f0ababed700 1 --2- 172.21.15.143:0/1832998577 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0ab40b22e0 0x7f0ab415aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:42.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.263+0000 7f0ababed700 1 -- 172.21.15.143:0/1832998577 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0aac009b30 con 0x7f0ab415dba0 2023-11-08T19:27:42.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.263+0000 7f0ababed700 1 --2- 172.21.15.143:0/1832998577 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0ab415dba0 0x7f0ab415b400 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f0ab406c2f0 tx=0x7f0aac004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:42.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.263+0000 7f0aaaffd700 1 -- 172.21.15.143:0/1832998577 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aac004800 con 0x7f0ab415dba0 2023-11-08T19:27:42.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.263+0000 7f0aaaffd700 1 -- 172.21.15.143:0/1832998577 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0aac004960 con 0x7f0ab415dba0 2023-11-08T19:27:42.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.263+0000 7f0abc650700 1 -- 172.21.15.143:0/1832998577 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ab40c6240 con 0x7f0ab415dba0 2023-11-08T19:27:42.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.263+0000 7f0aaaffd700 1 -- 172.21.15.143:0/1832998577 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aac01c680 con 0x7f0ab415dba0 2023-11-08T19:27:42.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.263+0000 7f0abc650700 1 -- 172.21.15.143:0/1832998577 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ab40c7cb0 con 0x7f0ab415dba0 2023-11-08T19:27:42.274 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.264+0000 7f0abc650700 1 -- 172.21.15.143:0/1832998577 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ab40c81d0 con 0x7f0ab415dba0 2023-11-08T19:27:42.274 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.264+0000 7f0aaaffd700 1 -- 172.21.15.143:0/1832998577 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0aac01c7e0 con 0x7f0ab415dba0 2023-11-08T19:27:42.274 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.265+0000 7f0aaaffd700 1 --2- 172.21.15.143:0/1832998577 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f0a9806af00 0x7f0a9806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:42.274 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.265+0000 7f0aaaffd700 1 -- 172.21.15.143:0/1832998577 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6888+0+0 (secure 0 0 0) 0x7f0aac08b620 con 0x7f0ab415dba0 2023-11-08T19:27:42.274 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.265+0000 7f0aba3ec700 1 --2- 172.21.15.143:0/1832998577 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f0a9806af00 0x7f0a9806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:42.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.265+0000 7f0aba3ec700 1 --2- 172.21.15.143:0/1832998577 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f0a9806af00 0x7f0a9806d3c0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f0aa4009b40 tx=0x7f0aa4009210 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:42.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.269+0000 7f0aaaffd700 1 -- 172.21.15.143:0/1832998577 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0ab40c81d0 con 0x7f0ab415dba0 2023-11-08T19:27:42.461 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.459+0000 7f0abc650700 1 -- 172.21.15.143:0/1832998577 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0ab40c81d0 con 0x7f0ab415dba0 2023-11-08T19:27:42.462 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.460+0000 7f0aaaffd700 1 -- 172.21.15.143:0/1832998577 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v80) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0ab40c81d0 con 0x7f0ab415dba0 2023-11-08T19:27:42.462 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2023-11-08T19:27:42.464 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.462+0000 7f0abc650700 1 -- 172.21.15.143:0/1832998577 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f0a9806af00 msgr2=0x7f0a9806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:42.464 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.462+0000 7f0abc650700 1 --2- 172.21.15.143:0/1832998577 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f0a9806af00 0x7f0a9806d3c0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f0aa4009b40 tx=0x7f0aa4009210 comp rx=0 tx=0).stop 2023-11-08T19:27:42.464 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.462+0000 7f0abc650700 1 -- 172.21.15.143:0/1832998577 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0ab415dba0 msgr2=0x7f0ab415b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:42.464 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.462+0000 7f0abc650700 1 --2- 172.21.15.143:0/1832998577 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0ab415dba0 0x7f0ab415b400 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f0ab406c2f0 tx=0x7f0aac004ad0 comp rx=0 tx=0).stop 2023-11-08T19:27:42.464 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.462+0000 7f0abc650700 1 -- 172.21.15.143:0/1832998577 shutdown_connections 2023-11-08T19:27:42.465 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.462+0000 7f0abc650700 1 --2- 172.21.15.143:0/1832998577 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f0a9806af00 0x7f0a9806d3c0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:42.465 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.462+0000 7f0abc650700 1 --2- 172.21.15.143:0/1832998577 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0ab406b240 0x7f0ab415a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:42.465 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.462+0000 7f0abc650700 1 --2- 172.21.15.143:0/1832998577 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0ab415dba0 0x7f0ab415b400 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:42.465 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.462+0000 7f0abc650700 1 --2- 172.21.15.143:0/1832998577 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0ab40b22e0 0x7f0ab415aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:42.465 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.462+0000 7f0abc650700 1 -- 172.21.15.143:0/1832998577 >> 172.21.15.143:0/1832998577 conn(0x7f0ab40cd2e0 msgr2=0x7f0ab40b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:42.466 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.462+0000 7f0abc650700 1 -- 172.21.15.143:0/1832998577 shutdown_connections 2023-11-08T19:27:42.466 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.462+0000 7f0abc650700 1 -- 172.21.15.143:0/1832998577 wait complete. 2023-11-08T19:27:42.476 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get snap-test-pool pg_num 2023-11-08T19:27:42.599 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.598+0000 7f7216a5d700 1 Processor -- start 2023-11-08T19:27:42.599 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.598+0000 7f7216a5d700 1 -- start start 2023-11-08T19:27:42.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.599+0000 7f7216a5d700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f721015e250 0x7f72100d17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:42.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.599+0000 7f7216a5d700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72100d2990 con 0x7f721015de70 2023-11-08T19:27:42.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.599+0000 7f7216a5d700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72100d1d80 con 0x7f72101626d0 2023-11-08T19:27:42.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.599+0000 7f7216a5d700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72100d2060 con 0x7f721015e250 2023-11-08T19:27:42.604 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.599+0000 7f720ffff700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f721015de70 0x7f72100d2e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:50018/0 (socket says 172.21.15.143:50018) 2023-11-08T19:27:42.604 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.599+0000 7f720ffff700 1 -- 172.21.15.143:0/2977420003 learned_addr learned my addr 172.21.15.143:0/2977420003 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:27:42.604 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.599+0000 7f720f7fe700 1 --2- 172.21.15.143:0/2977420003 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f721015e250 0x7f72100d17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:42.604 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.599+0000 7f720effd700 1 -- 172.21.15.143:0/2977420003 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3299103958 0 0) 0x7f72100d2990 con 0x7f721015de70 2023-11-08T19:27:42.604 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.599+0000 7f720effd700 1 -- 172.21.15.143:0/2977420003 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f71f80035b0 con 0x7f721015de70 2023-11-08T19:27:42.605 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.599+0000 7f720effd700 1 -- 172.21.15.143:0/2977420003 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3407099027 0 0) 0x7f71f80035b0 con 0x7f721015de70 2023-11-08T19:27:42.605 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.599+0000 7f720effd700 1 -- 172.21.15.143:0/2977420003 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f721015e250 msgr2=0x7f72100d17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:42.605 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.599+0000 7f720effd700 1 --2- 172.21.15.143:0/2977420003 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f721015e250 0x7f72100d17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:42.605 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.599+0000 7f720effd700 1 -- 172.21.15.143:0/2977420003 >> v1:172.21.15.174:6789/0 conn(0x7f72101626d0 legacy=0x7f721006c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:42.605 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.599+0000 7f720effd700 1 -- 172.21.15.143:0/2977420003 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72100d2340 con 0x7f721015de70 2023-11-08T19:27:42.606 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.599+0000 7f720effd700 1 -- 172.21.15.143:0/2977420003 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4147756303 0 0) 0x7f7200001670 con 0x7f721015de70 2023-11-08T19:27:42.606 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.599+0000 7f720effd700 1 -- 172.21.15.143:0/2977420003 >> v1:172.21.15.143:6789/0 conn(0x7f721015de70 legacy=0x7f72100d2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:42.606 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.599+0000 7f720effd700 1 --2- 172.21.15.143:0/2977420003 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f71f8003a30 0x7f71f8003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:42.606 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.599+0000 7f720effd700 1 --2- 172.21.15.143:0/2977420003 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f721015e250 0x7f71f8004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:42.607 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.599+0000 7f720effd700 1 --2- 172.21.15.143:0/2977420003 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f71f8004d90 0x7f71f8005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:42.607 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.599+0000 7f720effd700 1 -- 172.21.15.143:0/2977420003 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72100d2060 con 0x7f71f8003a30 2023-11-08T19:27:42.607 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.599+0000 7f720effd700 1 -- 172.21.15.143:0/2977420003 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71f800ba10 con 0x7f71f8004d90 2023-11-08T19:27:42.607 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.599+0000 7f720effd700 1 -- 172.21.15.143:0/2977420003 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71f800bbd0 con 0x7f721015e250 2023-11-08T19:27:42.607 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.599+0000 7f720ffff700 1 --2- 172.21.15.143:0/2977420003 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f71f8003a30 0x7f71f8003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:42.607 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.599+0000 7f720f7fe700 1 --2- 172.21.15.143:0/2977420003 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f721015e250 0x7f71f8004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:42.608 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.599+0000 7f7214ffa700 1 --2- 172.21.15.143:0/2977420003 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f71f8004d90 0x7f71f8005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:42.608 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.600+0000 7f720ffff700 1 -- 172.21.15.143:0/2977420003 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f721015e250 msgr2=0x7f71f8004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:42.608 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.600+0000 7f720ffff700 1 --2- 172.21.15.143:0/2977420003 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f721015e250 0x7f71f8004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:42.608 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.600+0000 7f720ffff700 1 -- 172.21.15.143:0/2977420003 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f71f8004d90 msgr2=0x7f71f8005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:42.608 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.600+0000 7f720ffff700 1 --2- 172.21.15.143:0/2977420003 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f71f8004d90 0x7f71f8005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:42.609 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.600+0000 7f720ffff700 1 -- 172.21.15.143:0/2977420003 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71f80035b0 con 0x7f71f8003a30 2023-11-08T19:27:42.609 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.600+0000 7f720ffff700 1 --2- 172.21.15.143:0/2977420003 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f71f8003a30 0x7f71f8003e10 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f720000c240 tx=0x7f71f8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=421be1bd6094420a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:42.609 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.600+0000 7f720effd700 1 -- 172.21.15.143:0/2977420003 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f720000cd20 con 0x7f71f8003a30 2023-11-08T19:27:42.609 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.600+0000 7f720effd700 1 -- 172.21.15.143:0/2977420003 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f720000ce80 con 0x7f71f8003a30 2023-11-08T19:27:42.609 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.600+0000 7f720effd700 1 -- 172.21.15.143:0/2977420003 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7200005ba0 con 0x7f71f8003a30 2023-11-08T19:27:42.610 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.600+0000 7f7216a5d700 1 -- 172.21.15.143:0/2977420003 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f71f8003a30 msgr2=0x7f71f8003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:42.610 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.600+0000 7f7216a5d700 1 --2- 172.21.15.143:0/2977420003 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f71f8003a30 0x7f71f8003e10 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f720000c240 tx=0x7f71f8005880 comp rx=0 tx=0).stop 2023-11-08T19:27:42.610 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.600+0000 7f7214ffa700 1 -- 172.21.15.143:0/2977420003 reap_dead start 2023-11-08T19:27:42.610 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.600+0000 7f7216a5d700 1 -- 172.21.15.143:0/2977420003 shutdown_connections 2023-11-08T19:27:42.610 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.600+0000 7f7216a5d700 1 -- 172.21.15.143:0/2977420003 >> 172.21.15.143:0/2977420003 conn(0x7f72100d06e0 msgr2=0x7f72100d0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:42.611 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.601+0000 7f7216a5d700 1 -- 172.21.15.143:0/2977420003 shutdown_connections 2023-11-08T19:27:42.611 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.601+0000 7f7216a5d700 1 -- 172.21.15.143:0/2977420003 wait complete. 2023-11-08T19:27:42.614 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.602+0000 7f7216a5d700 1 Processor -- start 2023-11-08T19:27:42.614 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.602+0000 7f7216a5d700 1 -- start start 2023-11-08T19:27:42.614 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.603+0000 7f7216a5d700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f71f8003a30 0x7f7210153000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:42.614 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.603+0000 7f7216a5d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7210154780 0x7f72101573a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:42.614 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.603+0000 7f7216a5d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7210158350 0x7f72101587d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:42.615 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.603+0000 7f7216a5d700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7210152710 con 0x7f7210154780 2023-11-08T19:27:42.615 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.603+0000 7f7216a5d700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72101578e0 con 0x7f7210158350 2023-11-08T19:27:42.615 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.603+0000 7f7216a5d700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72101536e0 con 0x7f71f8003a30 2023-11-08T19:27:42.615 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.603+0000 7f720f7fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7210154780 0x7f72101573a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:42.615 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.603+0000 7f720f7fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7210154780 0x7f72101573a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53878/0 (socket says 172.21.15.143:53878) 2023-11-08T19:27:42.616 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.603+0000 7f720f7fe700 1 -- 172.21.15.143:0/3027814151 learned_addr learned my addr 172.21.15.143:0/3027814151 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:42.616 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.603+0000 7f720ffff700 1 --2- 172.21.15.143:0/3027814151 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f71f8003a30 0x7f7210153000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:42.616 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.603+0000 7f7214ffa700 1 --2- 172.21.15.143:0/3027814151 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7210158350 0x7f72101587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:42.616 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.603+0000 7f720f7fe700 1 -- 172.21.15.143:0/3027814151 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f71f8003a30 msgr2=0x7f7210153000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:42.616 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.603+0000 7f720f7fe700 1 --2- 172.21.15.143:0/3027814151 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f71f8003a30 0x7f7210153000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:42.617 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.603+0000 7f720f7fe700 1 -- 172.21.15.143:0/3027814151 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7210158350 msgr2=0x7f72101587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:42.617 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.603+0000 7f720f7fe700 1 --2- 172.21.15.143:0/3027814151 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7210158350 0x7f72101587d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:42.617 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.604+0000 7f720f7fe700 1 -- 172.21.15.143:0/3027814151 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72101539c0 con 0x7f7210154780 2023-11-08T19:27:42.617 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.604+0000 7f720f7fe700 1 --2- 172.21.15.143:0/3027814151 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7210154780 0x7f72101573a0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f7204011aa0 tx=0x7f7204011db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:42.617 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.604+0000 7f720cff9700 1 -- 172.21.15.143:0/3027814151 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f720400f8e0 con 0x7f7210154780 2023-11-08T19:27:42.618 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.604+0000 7f720cff9700 1 -- 172.21.15.143:0/3027814151 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f720400fa40 con 0x7f7210154780 2023-11-08T19:27:42.618 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.604+0000 7f720cff9700 1 -- 172.21.15.143:0/3027814151 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f720400fdf0 con 0x7f7210154780 2023-11-08T19:27:42.618 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.604+0000 7f7216a5d700 1 -- 172.21.15.143:0/3027814151 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7210153ca0 con 0x7f7210154780 2023-11-08T19:27:42.618 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.604+0000 7f7216a5d700 1 -- 172.21.15.143:0/3027814151 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7210154260 con 0x7f7210154780 2023-11-08T19:27:42.618 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.605+0000 7f7216a5d700 1 -- 172.21.15.143:0/3027814151 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f721004eee0 con 0x7f7210154780 2023-11-08T19:27:42.619 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.606+0000 7f720cff9700 1 -- 172.21.15.143:0/3027814151 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7204014440 con 0x7f7210154780 2023-11-08T19:27:42.619 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.606+0000 7f720cff9700 1 --2- 172.21.15.143:0/3027814151 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f71fc06af00 0x7f71fc06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:42.619 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.606+0000 7f720cff9700 1 -- 172.21.15.143:0/3027814151 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6888+0+0 (secure 0 0 0) 0x7f720408a4a0 con 0x7f7210154780 2023-11-08T19:27:42.619 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.606+0000 7f720ffff700 1 --2- 172.21.15.143:0/3027814151 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f71fc06af00 0x7f71fc06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:42.619 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.606+0000 7f720ffff700 1 --2- 172.21.15.143:0/3027814151 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f71fc06af00 0x7f71fc06d3c0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f7200001930 tx=0x7f7200007000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:42.619 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.610+0000 7f720cff9700 1 -- 172.21.15.143:0/3027814151 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7204055390 con 0x7f7210154780 2023-11-08T19:27:42.799 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.798+0000 7f7216a5d700 1 -- 172.21.15.143:0/3027814151 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"} v 0) v1 -- 0x7f72100d1510 con 0x7f7210154780 2023-11-08T19:27:42.800 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.799+0000 7f720cff9700 1 -- 172.21.15.143:0/3027814151 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"}]=0 v80) v1 ==== 103+0+11 (secure 0 0 0) 0x7f72040589b0 con 0x7f7210154780 2023-11-08T19:27:42.800 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2023-11-08T19:27:42.802 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.800+0000 7f7216a5d700 1 -- 172.21.15.143:0/3027814151 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f71fc06af00 msgr2=0x7f71fc06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:42.802 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.800+0000 7f7216a5d700 1 --2- 172.21.15.143:0/3027814151 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f71fc06af00 0x7f71fc06d3c0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f7200001930 tx=0x7f7200007000 comp rx=0 tx=0).stop 2023-11-08T19:27:42.802 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.800+0000 7f7216a5d700 1 -- 172.21.15.143:0/3027814151 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7210154780 msgr2=0x7f72101573a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:42.802 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.800+0000 7f7216a5d700 1 --2- 172.21.15.143:0/3027814151 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7210154780 0x7f72101573a0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f7204011aa0 tx=0x7f7204011db0 comp rx=0 tx=0).stop 2023-11-08T19:27:42.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.800+0000 7f7216a5d700 1 -- 172.21.15.143:0/3027814151 shutdown_connections 2023-11-08T19:27:42.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.800+0000 7f7216a5d700 1 --2- 172.21.15.143:0/3027814151 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f71fc06af00 0x7f71fc06d3c0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:42.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.800+0000 7f7216a5d700 1 --2- 172.21.15.143:0/3027814151 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f71f8003a30 0x7f7210153000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:42.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.801+0000 7f7216a5d700 1 --2- 172.21.15.143:0/3027814151 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7210158350 0x7f72101587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:42.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.801+0000 7f7216a5d700 1 --2- 172.21.15.143:0/3027814151 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7210154780 0x7f72101573a0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:42.804 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.801+0000 7f7216a5d700 1 -- 172.21.15.143:0/3027814151 >> 172.21.15.143:0/3027814151 conn(0x7f72100d06e0 msgr2=0x7f7210160360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:42.804 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.801+0000 7f7216a5d700 1 -- 172.21.15.143:0/3027814151 shutdown_connections 2023-11-08T19:27:42.804 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.801+0000 7f7216a5d700 1 -- 172.21.15.143:0/3027814151 wait complete. 2023-11-08T19:27:42.815 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get test_data_pool pg_num 2023-11-08T19:27:42.938 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.936+0000 7f48a9b87700 1 Processor -- start 2023-11-08T19:27:42.938 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.936+0000 7f48a9b87700 1 -- start start 2023-11-08T19:27:42.946 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.937+0000 7f48a9b87700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f48a4072010 0x7f48a41522a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:42.946 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.937+0000 7f48a9b87700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48a415d0c0 con 0x7f48a406c0c0 2023-11-08T19:27:42.946 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.937+0000 7f48a9b87700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48a40ced30 con 0x7f48a40ccfb0 2023-11-08T19:27:42.947 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.937+0000 7f48a9b87700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48a40cd460 con 0x7f48a4072010 2023-11-08T19:27:42.947 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.937+0000 7f48a3fff700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f48a4072010 0x7f48a41522a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:42.947 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.937+0000 7f48a37fe700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f48a406c0c0 0x7f48a415c9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:50038/0 (socket says 172.21.15.143:50038) 2023-11-08T19:27:42.947 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.937+0000 7f48a37fe700 1 -- 172.21.15.143:0/2203056507 learned_addr learned my addr 172.21.15.143:0/2203056507 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:27:42.947 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.937+0000 7f48a3fff700 1 -- 172.21.15.143:0/2203056507 >> v1:172.21.15.174:6789/0 conn(0x7f48a40ccfb0 legacy=0x7f48a4071900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:42.948 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.937+0000 7f48a3fff700 1 -- 172.21.15.143:0/2203056507 >> v1:172.21.15.143:6789/0 conn(0x7f48a406c0c0 legacy=0x7f48a415c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:42.948 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.938+0000 7f48a3fff700 1 -- 172.21.15.143:0/2203056507 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48a40cdf60 con 0x7f48a4072010 2023-11-08T19:27:42.948 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.938+0000 7f48a3fff700 1 --2- 172.21.15.143:0/2203056507 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f48a4072010 0x7f48a41522a0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f489c009d90 tx=0x7f489c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=968a3edc42236733 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:42.948 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.938+0000 7f48a27fc700 1 -- 172.21.15.143:0/2203056507 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f489c00e070 con 0x7f48a4072010 2023-11-08T19:27:42.949 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.938+0000 7f48a27fc700 1 -- 172.21.15.143:0/2203056507 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f489c00bad0 con 0x7f48a4072010 2023-11-08T19:27:42.949 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.938+0000 7f48a27fc700 1 -- 172.21.15.143:0/2203056507 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f489c00be60 con 0x7f48a4072010 2023-11-08T19:27:42.949 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.938+0000 7f48a9b87700 1 -- 172.21.15.143:0/2203056507 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f48a4072010 msgr2=0x7f48a41522a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:42.949 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.938+0000 7f48a9b87700 1 --2- 172.21.15.143:0/2203056507 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f48a4072010 0x7f48a41522a0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f489c009d90 tx=0x7f489c00b330 comp rx=0 tx=0).stop 2023-11-08T19:27:42.949 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.938+0000 7f48a9b87700 1 -- 172.21.15.143:0/2203056507 shutdown_connections 2023-11-08T19:27:42.950 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.938+0000 7f48a9b87700 1 --2- 172.21.15.143:0/2203056507 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f48a4072010 0x7f48a41522a0 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:42.950 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.938+0000 7f48a9b87700 1 -- 172.21.15.143:0/2203056507 >> 172.21.15.143:0/2203056507 conn(0x7f48a40b1c70 msgr2=0x7f48a40b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:42.950 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.938+0000 7f48a9b87700 1 -- 172.21.15.143:0/2203056507 shutdown_connections 2023-11-08T19:27:42.950 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.938+0000 7f48a9b87700 1 -- 172.21.15.143:0/2203056507 wait complete. 2023-11-08T19:27:42.951 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.940+0000 7f48a9b87700 1 Processor -- start 2023-11-08T19:27:42.951 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.940+0000 7f48a9b87700 1 -- start start 2023-11-08T19:27:42.951 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.940+0000 7f48a9b87700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f48a406c0c0 0x7f48a414e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:42.951 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.940+0000 7f48a9b87700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f48a4072010 0x7f48a4153a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:42.951 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.940+0000 7f48a9b87700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f48a40ccfb0 0x7f48a41e5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:42.952 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.940+0000 7f48a9b87700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48a40b31e0 con 0x7f48a4072010 2023-11-08T19:27:42.952 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.940+0000 7f48a9b87700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48a40c74f0 con 0x7f48a40ccfb0 2023-11-08T19:27:42.952 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.940+0000 7f48a9b87700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48a40c77d0 con 0x7f48a406c0c0 2023-11-08T19:27:42.952 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.941+0000 7f48a2ffd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f48a4072010 0x7f48a4153a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:42.953 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.941+0000 7f48a2ffd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f48a4072010 0x7f48a4153a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53892/0 (socket says 172.21.15.143:53892) 2023-11-08T19:27:42.953 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.941+0000 7f48a2ffd700 1 -- 172.21.15.143:0/3165264051 learned_addr learned my addr 172.21.15.143:0/3165264051 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:42.953 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.941+0000 7f48a3fff700 1 --2- 172.21.15.143:0/3165264051 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f48a40ccfb0 0x7f48a41e5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:42.953 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.941+0000 7f48a37fe700 1 --2- 172.21.15.143:0/3165264051 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f48a406c0c0 0x7f48a414e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:42.954 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.941+0000 7f48a2ffd700 1 -- 172.21.15.143:0/3165264051 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f48a406c0c0 msgr2=0x7f48a414e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:42.954 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.941+0000 7f48a2ffd700 1 --2- 172.21.15.143:0/3165264051 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f48a406c0c0 0x7f48a414e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:42.954 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.941+0000 7f48a2ffd700 1 -- 172.21.15.143:0/3165264051 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f48a40ccfb0 msgr2=0x7f48a41e5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:42.954 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.941+0000 7f48a2ffd700 1 --2- 172.21.15.143:0/3165264051 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f48a40ccfb0 0x7f48a41e5500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:42.955 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.941+0000 7f48a2ffd700 1 -- 172.21.15.143:0/3165264051 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f489c009a70 con 0x7f48a4072010 2023-11-08T19:27:42.955 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.941+0000 7f48a3fff700 1 --2- 172.21.15.143:0/3165264051 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f48a40ccfb0 0x7f48a41e5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:27:42.956 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.941+0000 7f48a2ffd700 1 --2- 172.21.15.143:0/3165264051 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f48a4072010 0x7f48a4153a20 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f489400b4a0 tx=0x7f489400b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:42.956 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.941+0000 7f488bfff700 1 -- 172.21.15.143:0/3165264051 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4894011830 con 0x7f48a4072010 2023-11-08T19:27:42.956 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.941+0000 7f488bfff700 1 -- 172.21.15.143:0/3165264051 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4894011990 con 0x7f48a4072010 2023-11-08T19:27:42.957 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.941+0000 7f48a9b87700 1 -- 172.21.15.143:0/3165264051 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48a40c7ab0 con 0x7f48a4072010 2023-11-08T19:27:42.957 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.941+0000 7f488bfff700 1 -- 172.21.15.143:0/3165264051 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4894011d40 con 0x7f48a4072010 2023-11-08T19:27:42.957 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.941+0000 7f48a9b87700 1 -- 172.21.15.143:0/3165264051 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48a40c8070 con 0x7f48a4072010 2023-11-08T19:27:42.958 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.942+0000 7f48a9b87700 1 -- 172.21.15.143:0/3165264051 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48a40b29d0 con 0x7f48a4072010 2023-11-08T19:27:42.958 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.943+0000 7f488bfff700 1 -- 172.21.15.143:0/3165264051 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f489401a400 con 0x7f48a4072010 2023-11-08T19:27:42.958 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.943+0000 7f488bfff700 1 --2- 172.21.15.143:0/3165264051 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f489006af00 0x7f489006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:42.959 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.943+0000 7f488bfff700 1 -- 172.21.15.143:0/3165264051 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6888+0+0 (secure 0 0 0) 0x7f4894089690 con 0x7f48a4072010 2023-11-08T19:27:42.959 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.943+0000 7f48a37fe700 1 --2- 172.21.15.143:0/3165264051 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f489006af00 0x7f489006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:42.959 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.944+0000 7f48a37fe700 1 --2- 172.21.15.143:0/3165264051 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f489006af00 0x7f489006d3c0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f4898007fd0 tx=0x7f4898009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:42.960 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:42.947+0000 7f488bfff700 1 -- 172.21.15.143:0/3165264051 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4894054330 con 0x7f48a4072010 2023-11-08T19:27:43.134 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.133+0000 7f48a9b87700 1 -- 172.21.15.143:0/3165264051 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "test_data_pool", "var": "pg_num"} v 0) v1 -- 0x7f48a404eee0 con 0x7f48a4072010 2023-11-08T19:27:43.135 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.133+0000 7f488bfff700 1 -- 172.21.15.143:0/3165264051 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "test_data_pool", "var": "pg_num"}]=0 v80) v1 ==== 103+0+11 (secure 0 0 0) 0x7f4894057950 con 0x7f48a4072010 2023-11-08T19:27:43.135 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2023-11-08T19:27:43.138 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.136+0000 7f48a9b87700 1 -- 172.21.15.143:0/3165264051 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f489006af00 msgr2=0x7f489006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:43.138 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.136+0000 7f48a9b87700 1 --2- 172.21.15.143:0/3165264051 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f489006af00 0x7f489006d3c0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f4898007fd0 tx=0x7f4898009040 comp rx=0 tx=0).stop 2023-11-08T19:27:43.138 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.136+0000 7f48a9b87700 1 -- 172.21.15.143:0/3165264051 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f48a4072010 msgr2=0x7f48a4153a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:43.138 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.136+0000 7f48a9b87700 1 --2- 172.21.15.143:0/3165264051 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f48a4072010 0x7f48a4153a20 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f489400b4a0 tx=0x7f489400b860 comp rx=0 tx=0).stop 2023-11-08T19:27:43.138 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.136+0000 7f48a9b87700 1 -- 172.21.15.143:0/3165264051 shutdown_connections 2023-11-08T19:27:43.139 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.136+0000 7f48a9b87700 1 --2- 172.21.15.143:0/3165264051 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f489006af00 0x7f489006d3c0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:43.139 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.136+0000 7f48a9b87700 1 --2- 172.21.15.143:0/3165264051 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f48a406c0c0 0x7f48a414e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:43.139 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.136+0000 7f48a9b87700 1 --2- 172.21.15.143:0/3165264051 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f48a40ccfb0 0x7f48a41e5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:43.139 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.136+0000 7f48a9b87700 1 --2- 172.21.15.143:0/3165264051 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f48a4072010 0x7f48a4153a20 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:43.139 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.136+0000 7f48a9b87700 1 -- 172.21.15.143:0/3165264051 >> 172.21.15.143:0/3165264051 conn(0x7f48a40b1c70 msgr2=0x7f48a40cad10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:43.140 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.136+0000 7f48a9b87700 1 -- 172.21.15.143:0/3165264051 shutdown_connections 2023-11-08T19:27:43.140 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.136+0000 7f48a9b87700 1 -- 172.21.15.143:0/3165264051 wait complete. 2023-11-08T19:27:43.150 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get test_metadata_pool pg_num 2023-11-08T19:27:43.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.271+0000 7fb589f61700 1 Processor -- start 2023-11-08T19:27:43.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.271+0000 7fb589f61700 1 -- start start 2023-11-08T19:27:43.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.272+0000 7fb589f61700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb58415df10 0x7fb5840ce3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:43.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.272+0000 7fb589f61700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fb5840ce8e0 0x7fb58406cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:43.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.272+0000 7fb589f61700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5840cf960 con 0x7fb58415db30 2023-11-08T19:27:43.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.272+0000 7fb589f61700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5840ced50 con 0x7fb5840ce8e0 2023-11-08T19:27:43.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.272+0000 7fb589f61700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5840cfc40 con 0x7fb58415df10 2023-11-08T19:27:43.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.273+0000 7fb583fff700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fb5840ce8e0 0x7fb58406cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:43.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.273+0000 7fb582ffd700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb58415df10 0x7fb5840ce3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:43.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.273+0000 7fb583fff700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fb5840ce8e0 0x7fb58406cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.143:37396/0 (socket says 172.21.15.143:37396) 2023-11-08T19:27:43.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.273+0000 7fb583fff700 1 -- 172.21.15.143:0/3932974637 learned_addr learned my addr 172.21.15.143:0/3932974637 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:43.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.273+0000 7fb582ffd700 1 -- 172.21.15.143:0/3932974637 >> v2:172.21.15.174:3300/0 conn(0x7fb5840ce8e0 msgr2=0x7fb58406cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:43.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.273+0000 7fb582ffd700 1 --2- 172.21.15.143:0/3932974637 >> v2:172.21.15.174:3300/0 conn(0x7fb5840ce8e0 0x7fb58406cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:43.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.273+0000 7fb582ffd700 1 -- 172.21.15.143:0/3932974637 >> v1:172.21.15.174:6789/0 conn(0x7fb58415db30 legacy=0x7fb584162200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:43.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.273+0000 7fb582ffd700 1 -- 172.21.15.143:0/3932974637 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb58406c240 con 0x7fb58415df10 2023-11-08T19:27:43.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.273+0000 7fb582ffd700 1 --2- 172.21.15.143:0/3932974637 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb58415df10 0x7fb5840ce3a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb574009840 tx=0x7fb574009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cbb781b0ca6c2ff8 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:43.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.273+0000 7fb5827fc700 1 -- 172.21.15.143:0/3932974637 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb574011830 con 0x7fb58415df10 2023-11-08T19:27:43.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.273+0000 7fb5827fc700 1 -- 172.21.15.143:0/3932974637 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb574011990 con 0x7fb58415df10 2023-11-08T19:27:43.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.273+0000 7fb5827fc700 1 -- 172.21.15.143:0/3932974637 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb574011d20 con 0x7fb58415df10 2023-11-08T19:27:43.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.274+0000 7fb589f61700 1 -- 172.21.15.143:0/3932974637 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb58415df10 msgr2=0x7fb5840ce3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:43.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.274+0000 7fb589f61700 1 --2- 172.21.15.143:0/3932974637 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb58415df10 0x7fb5840ce3a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb574009840 tx=0x7fb574009b50 comp rx=0 tx=0).stop 2023-11-08T19:27:43.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.274+0000 7fb589f61700 1 -- 172.21.15.143:0/3932974637 shutdown_connections 2023-11-08T19:27:43.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.274+0000 7fb589f61700 1 --2- 172.21.15.143:0/3932974637 >> v2:172.21.15.174:3300/0 conn(0x7fb5840ce8e0 0x7fb58406cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:43.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.274+0000 7fb589f61700 1 --2- 172.21.15.143:0/3932974637 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb58415df10 0x7fb5840ce3a0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:43.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.274+0000 7fb589f61700 1 -- 172.21.15.143:0/3932974637 >> 172.21.15.143:0/3932974637 conn(0x7fb58406f2e0 msgr2=0x7fb58406f6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:43.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.274+0000 7fb589f61700 1 -- 172.21.15.143:0/3932974637 shutdown_connections 2023-11-08T19:27:43.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.274+0000 7fb589f61700 1 -- 172.21.15.143:0/3932974637 wait complete. 2023-11-08T19:27:43.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.275+0000 7fb589f61700 1 Processor -- start 2023-11-08T19:27:43.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.275+0000 7fb589f61700 1 -- start start 2023-11-08T19:27:43.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.276+0000 7fb589f61700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb5840ce8e0 0x7fb5841565e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:43.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.276+0000 7fb589f61700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb58415db30 0x7fb5841e96c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:43.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.276+0000 7fb589f61700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb58415df10 0x7fb5840b1070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:43.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.276+0000 7fb589f61700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5840c9510 con 0x7fb58415df10 2023-11-08T19:27:43.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.276+0000 7fb589f61700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5840c97f0 con 0x7fb5840ce8e0 2023-11-08T19:27:43.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.276+0000 7fb589f61700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5840c9ad0 con 0x7fb58415db30 2023-11-08T19:27:43.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.276+0000 7fb583fff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb58415df10 0x7fb5840b1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:43.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.276+0000 7fb583fff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb58415df10 0x7fb5840b1070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53908/0 (socket says 172.21.15.143:53908) 2023-11-08T19:27:43.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.276+0000 7fb583fff700 1 -- 172.21.15.143:0/3619784898 learned_addr learned my addr 172.21.15.143:0/3619784898 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:43.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.276+0000 7fb5837fe700 1 --2- 172.21.15.143:0/3619784898 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb5840ce8e0 0x7fb5841565e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:43.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.276+0000 7fb582ffd700 1 --2- 172.21.15.143:0/3619784898 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb58415db30 0x7fb5841e96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:43.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.276+0000 7fb583fff700 1 -- 172.21.15.143:0/3619784898 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb58415db30 msgr2=0x7fb5841e96c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:43.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.277+0000 7fb583fff700 1 --2- 172.21.15.143:0/3619784898 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb58415db30 0x7fb5841e96c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:43.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.277+0000 7fb583fff700 1 -- 172.21.15.143:0/3619784898 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb5840ce8e0 msgr2=0x7fb5841565e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:43.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.277+0000 7fb583fff700 1 --2- 172.21.15.143:0/3619784898 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb5840ce8e0 0x7fb5841565e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:43.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.277+0000 7fb583fff700 1 -- 172.21.15.143:0/3619784898 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5740094a0 con 0x7fb58415df10 2023-11-08T19:27:43.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.277+0000 7fb5837fe700 1 --2- 172.21.15.143:0/3619784898 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb5840ce8e0 0x7fb5841565e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:27:43.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.277+0000 7fb582ffd700 1 --2- 172.21.15.143:0/3619784898 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb58415db30 0x7fb5841e96c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:27:43.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.277+0000 7fb583fff700 1 --2- 172.21.15.143:0/3619784898 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb58415df10 0x7fb5840b1070 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fb57c00efc0 tx=0x7fb57c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:43.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.277+0000 7fb56bfff700 1 -- 172.21.15.143:0/3619784898 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb57c003c20 con 0x7fb58415df10 2023-11-08T19:27:43.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.277+0000 7fb56bfff700 1 -- 172.21.15.143:0/3619784898 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb57c00f040 con 0x7fb58415df10 2023-11-08T19:27:43.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.277+0000 7fb589f61700 1 -- 172.21.15.143:0/3619784898 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5840c9db0 con 0x7fb58415df10 2023-11-08T19:27:43.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.277+0000 7fb56bfff700 1 -- 172.21.15.143:0/3619784898 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb57c00c930 con 0x7fb58415df10 2023-11-08T19:27:43.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.277+0000 7fb589f61700 1 -- 172.21.15.143:0/3619784898 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb584156b20 con 0x7fb58415df10 2023-11-08T19:27:43.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.278+0000 7fb589f61700 1 -- 172.21.15.143:0/3619784898 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5840ca2d0 con 0x7fb58415df10 2023-11-08T19:27:43.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.278+0000 7fb56bfff700 1 -- 172.21.15.143:0/3619784898 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb57c00ca90 con 0x7fb58415df10 2023-11-08T19:27:43.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.279+0000 7fb56bfff700 1 --2- 172.21.15.143:0/3619784898 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fb57006af00 0x7fb57006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:43.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.279+0000 7fb56bfff700 1 -- 172.21.15.143:0/3619784898 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6888+0+0 (secure 0 0 0) 0x7fb57c089f10 con 0x7fb58415df10 2023-11-08T19:27:43.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.279+0000 7fb5837fe700 1 --2- 172.21.15.143:0/3619784898 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fb57006af00 0x7fb57006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:43.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.279+0000 7fb5837fe700 1 --2- 172.21.15.143:0/3619784898 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fb57006af00 0x7fb57006d3c0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fb578000f60 tx=0x7fb578009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:43.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.283+0000 7fb56bfff700 1 -- 172.21.15.143:0/3619784898 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb5840ca2d0 con 0x7fb58415df10 2023-11-08T19:27:43.474 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.472+0000 7fb589f61700 1 -- 172.21.15.143:0/3619784898 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "test_metadata_pool", "var": "pg_num"} v 0) v1 -- 0x7fb5840ca2d0 con 0x7fb58415df10 2023-11-08T19:27:43.474 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.472+0000 7fb56bfff700 1 -- 172.21.15.143:0/3619784898 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "test_metadata_pool", "var": "pg_num"}]=0 v80) v1 ==== 107+0+11 (secure 0 0 0) 0x7fb5840ca2d0 con 0x7fb58415df10 2023-11-08T19:27:43.474 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2023-11-08T19:27:43.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.474+0000 7fb589f61700 1 -- 172.21.15.143:0/3619784898 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fb57006af00 msgr2=0x7fb57006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:43.477 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.474+0000 7fb589f61700 1 --2- 172.21.15.143:0/3619784898 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fb57006af00 0x7fb57006d3c0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fb578000f60 tx=0x7fb578009040 comp rx=0 tx=0).stop 2023-11-08T19:27:43.477 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.474+0000 7fb589f61700 1 -- 172.21.15.143:0/3619784898 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb58415df10 msgr2=0x7fb5840b1070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:43.477 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.474+0000 7fb589f61700 1 --2- 172.21.15.143:0/3619784898 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb58415df10 0x7fb5840b1070 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fb57c00efc0 tx=0x7fb57c00c3a0 comp rx=0 tx=0).stop 2023-11-08T19:27:43.477 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.474+0000 7fb589f61700 1 -- 172.21.15.143:0/3619784898 shutdown_connections 2023-11-08T19:27:43.478 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.474+0000 7fb589f61700 1 --2- 172.21.15.143:0/3619784898 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fb57006af00 0x7fb57006d3c0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:43.478 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.474+0000 7fb589f61700 1 --2- 172.21.15.143:0/3619784898 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb58415df10 0x7fb5840b1070 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:43.478 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.475+0000 7fb589f61700 1 --2- 172.21.15.143:0/3619784898 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb5840ce8e0 0x7fb5841565e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:43.478 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.475+0000 7fb589f61700 1 --2- 172.21.15.143:0/3619784898 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb58415db30 0x7fb5841e96c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:43.479 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.475+0000 7fb589f61700 1 -- 172.21.15.143:0/3619784898 >> 172.21.15.143:0/3619784898 conn(0x7fb58406f2e0 msgr2=0x7fb5840b5b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:43.479 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.475+0000 7fb589f61700 1 -- 172.21.15.143:0/3619784898 shutdown_connections 2023-11-08T19:27:43.479 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.475+0000 7fb589f61700 1 -- 172.21.15.143:0/3619784898 wait complete. 2023-11-08T19:27:43.488 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-08T19:27:43.611 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.610+0000 7f6a68159700 1 Processor -- start 2023-11-08T19:27:43.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.610+0000 7f6a68159700 1 -- start start 2023-11-08T19:27:43.614 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.611+0000 7f6a68159700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f6a600b61e0 0x7f6a600d01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:43.614 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.611+0000 7f6a68159700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a600cc2a0 con 0x7f6a6006bd20 2023-11-08T19:27:43.614 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.611+0000 7f6a68159700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a6006c1b0 con 0x7f6a6015a950 2023-11-08T19:27:43.614 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.611+0000 7f6a68159700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a600cc130 con 0x7f6a600b61e0 2023-11-08T19:27:43.615 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.611+0000 7f6a666f6700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f6a600b61e0 0x7f6a600d01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:43.615 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.611+0000 7f6a656f4700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f6a6006bd20 0x7f6a600cc730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:50066/0 (socket says 172.21.15.143:50066) 2023-11-08T19:27:43.615 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.611+0000 7f6a656f4700 1 -- 172.21.15.143:0/3510442010 learned_addr learned my addr 172.21.15.143:0/3510442010 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:27:43.615 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.611+0000 7f6a666f6700 1 -- 172.21.15.143:0/3510442010 >> v1:172.21.15.174:6789/0 conn(0x7f6a6015a950 legacy=0x7f6a60072370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:43.615 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.611+0000 7f6a666f6700 1 -- 172.21.15.143:0/3510442010 >> v1:172.21.15.143:6789/0 conn(0x7f6a6006bd20 legacy=0x7f6a600cc730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:43.616 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.611+0000 7f6a666f6700 1 -- 172.21.15.143:0/3510442010 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a6006c490 con 0x7f6a600b61e0 2023-11-08T19:27:43.616 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.611+0000 7f6a64ef3700 1 -- 172.21.15.143:0/3510442010 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3569537841 0 0) 0x7f6a600cc2a0 con 0x7f6a6006bd20 2023-11-08T19:27:43.616 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.611+0000 7f6a666f6700 1 --2- 172.21.15.143:0/3510442010 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a600b61e0 0x7f6a600d01b0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f6a58009d90 tx=0x7f6a5800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b99a7a88335e1088 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:43.616 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.611+0000 7f6a64ef3700 1 -- 172.21.15.143:0/3510442010 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a5800e070 con 0x7f6a600b61e0 2023-11-08T19:27:43.616 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.612+0000 7f6a64ef3700 1 -- 172.21.15.143:0/3510442010 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a5800bbd0 con 0x7f6a600b61e0 2023-11-08T19:27:43.616 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.612+0000 7f6a64ef3700 1 -- 172.21.15.143:0/3510442010 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a58012430 con 0x7f6a600b61e0 2023-11-08T19:27:43.617 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.612+0000 7f6a68159700 1 -- 172.21.15.143:0/3510442010 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a600b61e0 msgr2=0x7f6a600d01b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:43.617 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.612+0000 7f6a68159700 1 --2- 172.21.15.143:0/3510442010 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a600b61e0 0x7f6a600d01b0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f6a58009d90 tx=0x7f6a5800b330 comp rx=0 tx=0).stop 2023-11-08T19:27:43.617 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.612+0000 7f6a68159700 1 -- 172.21.15.143:0/3510442010 shutdown_connections 2023-11-08T19:27:43.617 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.612+0000 7f6a68159700 1 --2- 172.21.15.143:0/3510442010 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a600b61e0 0x7f6a600d01b0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:43.617 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.612+0000 7f6a68159700 1 -- 172.21.15.143:0/3510442010 >> 172.21.15.143:0/3510442010 conn(0x7f6a600cd2e0 msgr2=0x7f6a600cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:43.618 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.612+0000 7f6a68159700 1 -- 172.21.15.143:0/3510442010 shutdown_connections 2023-11-08T19:27:43.618 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.612+0000 7f6a68159700 1 -- 172.21.15.143:0/3510442010 wait complete. 2023-11-08T19:27:43.621 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.614+0000 7f6a68159700 1 Processor -- start 2023-11-08T19:27:43.621 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.614+0000 7f6a68159700 1 -- start start 2023-11-08T19:27:43.621 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.615+0000 7f6a68159700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a600b61e0 0x7f6a600c6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:43.621 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.615+0000 7f6a68159700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6a6015a950 0x7f6a600c8c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:43.622 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.615+0000 7f6a68159700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6a601f4660 0x7f6a601f3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:43.622 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.615+0000 7f6a68159700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a600c8850 con 0x7f6a600b61e0 2023-11-08T19:27:43.622 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.615+0000 7f6a68159700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a600c7230 con 0x7f6a6015a950 2023-11-08T19:27:43.622 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.615+0000 7f6a68159700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a600c7930 con 0x7f6a601f4660 2023-11-08T19:27:43.622 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.615+0000 7f6a65ef5700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a600b61e0 0x7f6a600c6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:43.623 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.615+0000 7f6a65ef5700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a600b61e0 0x7f6a600c6cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53918/0 (socket says 172.21.15.143:53918) 2023-11-08T19:27:43.623 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.615+0000 7f6a65ef5700 1 -- 172.21.15.143:0/4178501081 learned_addr learned my addr 172.21.15.143:0/4178501081 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:43.623 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.615+0000 7f6a666f6700 1 --2- 172.21.15.143:0/4178501081 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6a601f4660 0x7f6a601f3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:43.623 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.615+0000 7f6a656f4700 1 --2- 172.21.15.143:0/4178501081 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6a6015a950 0x7f6a600c8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:43.623 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.615+0000 7f6a65ef5700 1 -- 172.21.15.143:0/4178501081 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6a601f4660 msgr2=0x7f6a601f3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:43.624 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.615+0000 7f6a65ef5700 1 --2- 172.21.15.143:0/4178501081 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6a601f4660 0x7f6a601f3cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:43.624 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.615+0000 7f6a65ef5700 1 -- 172.21.15.143:0/4178501081 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6a6015a950 msgr2=0x7f6a600c8c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:43.624 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.615+0000 7f6a65ef5700 1 --2- 172.21.15.143:0/4178501081 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6a6015a950 0x7f6a600c8c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:43.624 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.615+0000 7f6a65ef5700 1 -- 172.21.15.143:0/4178501081 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a58009a70 con 0x7f6a600b61e0 2023-11-08T19:27:43.624 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.615+0000 7f6a666f6700 1 --2- 172.21.15.143:0/4178501081 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6a601f4660 0x7f6a601f3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:27:43.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.615+0000 7f6a65ef5700 1 --2- 172.21.15.143:0/4178501081 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a600b61e0 0x7f6a600c6cf0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f6a5c00c4e0 tx=0x7f6a5c00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:43.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.615+0000 7f6a567fc700 1 -- 172.21.15.143:0/4178501081 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a5c0052d0 con 0x7f6a600b61e0 2023-11-08T19:27:43.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.615+0000 7f6a567fc700 1 -- 172.21.15.143:0/4178501081 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a5c005430 con 0x7f6a600b61e0 2023-11-08T19:27:43.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.615+0000 7f6a567fc700 1 -- 172.21.15.143:0/4178501081 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a5c0038c0 con 0x7f6a600b61e0 2023-11-08T19:27:43.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.615+0000 7f6a68159700 1 -- 172.21.15.143:0/4178501081 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a600c7bd0 con 0x7f6a600b61e0 2023-11-08T19:27:43.626 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.615+0000 7f6a68159700 1 -- 172.21.15.143:0/4178501081 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a600c8190 con 0x7f6a600b61e0 2023-11-08T19:27:43.626 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.616+0000 7f6a68159700 1 -- 172.21.15.143:0/4178501081 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a600c8ac0 con 0x7f6a600b61e0 2023-11-08T19:27:43.626 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.617+0000 7f6a567fc700 1 -- 172.21.15.143:0/4178501081 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6a5c003a20 con 0x7f6a600b61e0 2023-11-08T19:27:43.626 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.617+0000 7f6a567fc700 1 --2- 172.21.15.143:0/4178501081 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f6a4406af00 0x7f6a4406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:43.626 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.617+0000 7f6a567fc700 1 -- 172.21.15.143:0/4178501081 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6888+0+0 (secure 0 0 0) 0x7f6a5c088f50 con 0x7f6a600b61e0 2023-11-08T19:27:43.627 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.617+0000 7f6a656f4700 1 --2- 172.21.15.143:0/4178501081 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f6a4406af00 0x7f6a4406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:43.627 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.618+0000 7f6a656f4700 1 --2- 172.21.15.143:0/4178501081 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f6a4406af00 0x7f6a4406d3c0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f6a600c6540 tx=0x7f6a50007d90 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:43.627 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.621+0000 7f6a567fc700 1 -- 172.21.15.143:0/4178501081 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6a5c053b60 con 0x7f6a600b61e0 2023-11-08T19:27:43.812 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.810+0000 7f6a68159700 1 -- 172.21.15.143:0/4178501081 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6a6004eee0 con 0x7f6a600b61e0 2023-11-08T19:27:43.812 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.811+0000 7f6a567fc700 1 -- 172.21.15.143:0/4178501081 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v80) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6a5c017020 con 0x7f6a600b61e0 2023-11-08T19:27:43.813 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2023-11-08T19:27:43.814 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.813+0000 7f6a68159700 1 -- 172.21.15.143:0/4178501081 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f6a4406af00 msgr2=0x7f6a4406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:43.815 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.813+0000 7f6a68159700 1 --2- 172.21.15.143:0/4178501081 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f6a4406af00 0x7f6a4406d3c0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f6a600c6540 tx=0x7f6a50007d90 comp rx=0 tx=0).stop 2023-11-08T19:27:43.815 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.813+0000 7f6a68159700 1 -- 172.21.15.143:0/4178501081 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a600b61e0 msgr2=0x7f6a600c6cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:43.815 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.813+0000 7f6a68159700 1 --2- 172.21.15.143:0/4178501081 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a600b61e0 0x7f6a600c6cf0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f6a5c00c4e0 tx=0x7f6a5c00c8a0 comp rx=0 tx=0).stop 2023-11-08T19:27:43.815 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.813+0000 7f6a68159700 1 -- 172.21.15.143:0/4178501081 shutdown_connections 2023-11-08T19:27:43.815 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.813+0000 7f6a68159700 1 --2- 172.21.15.143:0/4178501081 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f6a4406af00 0x7f6a4406d3c0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:43.816 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.813+0000 7f6a68159700 1 --2- 172.21.15.143:0/4178501081 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6a601f4660 0x7f6a601f3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:43.816 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.813+0000 7f6a68159700 1 --2- 172.21.15.143:0/4178501081 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a600b61e0 0x7f6a600c6cf0 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:43.816 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.813+0000 7f6a68159700 1 --2- 172.21.15.143:0/4178501081 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6a6015a950 0x7f6a600c8c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:43.816 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.813+0000 7f6a68159700 1 -- 172.21.15.143:0/4178501081 >> 172.21.15.143:0/4178501081 conn(0x7f6a600cd2e0 msgr2=0x7f6a6015d650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:43.816 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.813+0000 7f6a68159700 1 -- 172.21.15.143:0/4178501081 shutdown_connections 2023-11-08T19:27:43.816 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.813+0000 7f6a68159700 1 -- 172.21.15.143:0/4178501081 wait complete. 2023-11-08T19:27:43.826 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-08T19:27:43.956 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.948+0000 7f0f3ef01700 1 Processor -- start 2023-11-08T19:27:43.956 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.948+0000 7f0f3ef01700 1 -- start start 2023-11-08T19:27:43.957 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.949+0000 7f0f3ef01700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f0f3815bb20 0x7f0f380b2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:43.958 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.949+0000 7f0f3ef01700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f380d0420 con 0x7f0f380b1730 2023-11-08T19:27:43.958 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.949+0000 7f0f3ef01700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f380cf810 con 0x7f0f380d1d80 2023-11-08T19:27:43.958 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.949+0000 7f0f3ef01700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f380cf600 con 0x7f0f3815bb20 2023-11-08T19:27:43.958 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.949+0000 7f0f37fff700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f0f380b1730 0x7f0f380d1670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:50080/0 (socket says 172.21.15.143:50080) 2023-11-08T19:27:43.959 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.949+0000 7f0f37fff700 1 -- 172.21.15.143:0/45415385 learned_addr learned my addr 172.21.15.143:0/45415385 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:27:43.959 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.949+0000 7f0f3cc9d700 1 --2- 172.21.15.143:0/45415385 >> v2:172.21.15.174:3300/0 conn(0x7f0f3815bb20 0x7f0f380b2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:43.959 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.949+0000 7f0f377fe700 1 -- 172.21.15.143:0/45415385 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3455601862 0 0) 0x7f0f380d0420 con 0x7f0f380b1730 2023-11-08T19:27:43.959 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.949+0000 7f0f377fe700 1 -- 172.21.15.143:0/45415385 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0f280035b0 con 0x7f0f380b1730 2023-11-08T19:27:43.959 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.950+0000 7f0f377fe700 1 -- 172.21.15.143:0/45415385 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2395698746 0 0) 0x7f0f280035b0 con 0x7f0f380b1730 2023-11-08T19:27:43.960 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.950+0000 7f0f377fe700 1 -- 172.21.15.143:0/45415385 >> v2:172.21.15.174:3300/0 conn(0x7f0f3815bb20 msgr2=0x7f0f380b2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:43.960 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.950+0000 7f0f377fe700 1 --2- 172.21.15.143:0/45415385 >> v2:172.21.15.174:3300/0 conn(0x7f0f3815bb20 0x7f0f380b2670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:43.960 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.950+0000 7f0f377fe700 1 -- 172.21.15.143:0/45415385 >> v1:172.21.15.174:6789/0 conn(0x7f0f380d1d80 legacy=0x7f0f380cc4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:43.960 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.950+0000 7f0f377fe700 1 -- 172.21.15.143:0/45415385 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f380cfaf0 con 0x7f0f380b1730 2023-11-08T19:27:43.960 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.950+0000 7f0f377fe700 1 -- 172.21.15.143:0/45415385 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4147756303 0 0) 0x7f0f24001670 con 0x7f0f380b1730 2023-11-08T19:27:43.961 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.950+0000 7f0f377fe700 1 -- 172.21.15.143:0/45415385 >> v1:172.21.15.143:6789/0 conn(0x7f0f380b1730 legacy=0x7f0f380d1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:43.961 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.950+0000 7f0f377fe700 1 --2- 172.21.15.143:0/45415385 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0f28003a00 0x7f0f28003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:43.961 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.950+0000 7f0f377fe700 1 --2- 172.21.15.143:0/45415385 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0f28004400 0x7f0f280048d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:43.961 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.950+0000 7f0f377fe700 1 --2- 172.21.15.143:0/45415385 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0f28005170 0x7f0f2800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:43.961 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.950+0000 7f0f377fe700 1 -- 172.21.15.143:0/45415385 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f380cf600 con 0x7f0f28004400 2023-11-08T19:27:43.962 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.950+0000 7f0f377fe700 1 -- 172.21.15.143:0/45415385 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f2800c0b0 con 0x7f0f28003a00 2023-11-08T19:27:43.962 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.950+0000 7f0f377fe700 1 -- 172.21.15.143:0/45415385 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f2800c270 con 0x7f0f28005170 2023-11-08T19:27:43.962 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.950+0000 7f0f37fff700 1 --2- 172.21.15.143:0/45415385 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0f28004400 0x7f0f280048d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:43.962 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.950+0000 7f0f3cc9d700 1 --2- 172.21.15.143:0/45415385 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0f28003a00 0x7f0f28003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:43.962 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.950+0000 7f0f3d49e700 1 --2- 172.21.15.143:0/45415385 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0f28005170 0x7f0f2800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:43.963 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.950+0000 7f0f37fff700 1 -- 172.21.15.143:0/45415385 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0f28005170 msgr2=0x7f0f2800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:43.963 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.950+0000 7f0f37fff700 1 --2- 172.21.15.143:0/45415385 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0f28005170 0x7f0f2800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:43.963 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.950+0000 7f0f37fff700 1 -- 172.21.15.143:0/45415385 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0f28003a00 msgr2=0x7f0f28003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:43.963 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.950+0000 7f0f37fff700 1 --2- 172.21.15.143:0/45415385 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0f28003a00 0x7f0f28003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:43.963 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.950+0000 7f0f37fff700 1 -- 172.21.15.143:0/45415385 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f280035b0 con 0x7f0f28004400 2023-11-08T19:27:43.964 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.950+0000 7f0f3d49e700 1 -- 172.21.15.143:0/45415385 reap_dead start 2023-11-08T19:27:43.964 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.950+0000 7f0f37fff700 1 --2- 172.21.15.143:0/45415385 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0f28004400 0x7f0f280048d0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f0f2400c2f0 tx=0x7f0f28005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f2abb98a05b4fea server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:43.964 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.950+0000 7f0f377fe700 1 -- 172.21.15.143:0/45415385 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f2400ce30 con 0x7f0f28004400 2023-11-08T19:27:43.964 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.950+0000 7f0f377fe700 1 -- 172.21.15.143:0/45415385 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f24005960 con 0x7f0f28004400 2023-11-08T19:27:43.964 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.950+0000 7f0f377fe700 1 -- 172.21.15.143:0/45415385 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f24005d10 con 0x7f0f28004400 2023-11-08T19:27:43.964 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.951+0000 7f0f3ef01700 1 -- 172.21.15.143:0/45415385 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0f28004400 msgr2=0x7f0f280048d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:43.965 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.951+0000 7f0f3ef01700 1 --2- 172.21.15.143:0/45415385 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0f28004400 0x7f0f280048d0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f0f2400c2f0 tx=0x7f0f28005880 comp rx=0 tx=0).stop 2023-11-08T19:27:43.965 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.951+0000 7f0f3ef01700 1 -- 172.21.15.143:0/45415385 shutdown_connections 2023-11-08T19:27:43.965 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.951+0000 7f0f3ef01700 1 --2- 172.21.15.143:0/45415385 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0f28004400 0x7f0f280048d0 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:43.965 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.951+0000 7f0f3ef01700 1 -- 172.21.15.143:0/45415385 >> 172.21.15.143:0/45415385 conn(0x7f0f3808b000 msgr2=0x7f0f3809ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:43.965 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.951+0000 7f0f3ef01700 1 -- 172.21.15.143:0/45415385 shutdown_connections 2023-11-08T19:27:43.966 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.951+0000 7f0f3ef01700 1 -- 172.21.15.143:0/45415385 wait complete. 2023-11-08T19:27:43.966 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.952+0000 7f0f3ef01700 1 Processor -- start 2023-11-08T19:27:43.966 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.952+0000 7f0f3ef01700 1 -- start start 2023-11-08T19:27:43.966 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.953+0000 7f0f3ef01700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0f28004400 0x7f0f3806d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:43.966 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.953+0000 7f0f3ef01700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0f381e3190 0x7f0f38153ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:43.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.953+0000 7f0f3ef01700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0f381e5c30 0x7f0f380b5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:43.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.953+0000 7f0f3ef01700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f380cd4e0 con 0x7f0f28004400 2023-11-08T19:27:43.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.953+0000 7f0f3ef01700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f380cdaa0 con 0x7f0f381e5c30 2023-11-08T19:27:43.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.953+0000 7f0f3ef01700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f380c4c90 con 0x7f0f381e3190 2023-11-08T19:27:43.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.953+0000 7f0f3cc9d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0f28004400 0x7f0f3806d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:43.968 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.953+0000 7f0f3cc9d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0f28004400 0x7f0f3806d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53938/0 (socket says 172.21.15.143:53938) 2023-11-08T19:27:43.968 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.953+0000 7f0f3cc9d700 1 -- 172.21.15.143:0/3946022397 learned_addr learned my addr 172.21.15.143:0/3946022397 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:43.968 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.953+0000 7f0f3cc9d700 1 -- 172.21.15.143:0/3946022397 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0f381e3190 msgr2=0x7f0f38153ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:43.968 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.953+0000 7f0f3cc9d700 1 --2- 172.21.15.143:0/3946022397 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0f381e3190 0x7f0f38153ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:43.968 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.954+0000 7f0f3cc9d700 1 -- 172.21.15.143:0/3946022397 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0f381e5c30 msgr2=0x7f0f380b5780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:43.969 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.954+0000 7f0f3cc9d700 1 --2- 172.21.15.143:0/3946022397 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0f381e5c30 0x7f0f380b5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:43.969 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.954+0000 7f0f3cc9d700 1 -- 172.21.15.143:0/3946022397 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f3806d820 con 0x7f0f28004400 2023-11-08T19:27:43.969 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.954+0000 7f0f3cc9d700 1 --2- 172.21.15.143:0/3946022397 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0f28004400 0x7f0f3806d120 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c011b00 tx=0x7f0f2c011ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:43.969 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.954+0000 7f0f357fa700 1 -- 172.21.15.143:0/3946022397 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f2c00f9e0 con 0x7f0f28004400 2023-11-08T19:27:43.969 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.954+0000 7f0f357fa700 1 -- 172.21.15.143:0/3946022397 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f2c00fb40 con 0x7f0f28004400 2023-11-08T19:27:43.970 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.954+0000 7f0f3ef01700 1 -- 172.21.15.143:0/3946022397 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f3806db00 con 0x7f0f28004400 2023-11-08T19:27:43.970 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.954+0000 7f0f357fa700 1 -- 172.21.15.143:0/3946022397 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f2c014440 con 0x7f0f28004400 2023-11-08T19:27:43.970 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.954+0000 7f0f3ef01700 1 -- 172.21.15.143:0/3946022397 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f38154430 con 0x7f0f28004400 2023-11-08T19:27:43.970 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.955+0000 7f0f3ef01700 1 -- 172.21.15.143:0/3946022397 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f3804eee0 con 0x7f0f28004400 2023-11-08T19:27:43.971 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.955+0000 7f0f357fa700 1 -- 172.21.15.143:0/3946022397 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0f2c0145a0 con 0x7f0f28004400 2023-11-08T19:27:43.971 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.956+0000 7f0f357fa700 1 --2- 172.21.15.143:0/3946022397 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f0f1c06af30 0x7f0f1c06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:43.972 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.956+0000 7f0f357fa700 1 -- 172.21.15.143:0/3946022397 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6888+0+0 (secure 0 0 0) 0x7f0f2c08a300 con 0x7f0f28004400 2023-11-08T19:27:43.972 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.956+0000 7f0f37fff700 1 --2- 172.21.15.143:0/3946022397 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f0f1c06af30 0x7f0f1c06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:43.972 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.956+0000 7f0f37fff700 1 --2- 172.21.15.143:0/3946022397 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f0f1c06af30 0x7f0f1c06d3f0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f0f38158160 tx=0x7f0f2400a000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:43.972 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:43.960+0000 7f0f357fa700 1 -- 172.21.15.143:0/3946022397 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0f2c054e70 con 0x7f0f28004400 2023-11-08T19:27:44.150 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.149+0000 7f0f3ef01700 1 -- 172.21.15.143:0/3946022397 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0f381e10d0 con 0x7f0f28004400 2023-11-08T19:27:44.151 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.149+0000 7f0f357fa700 1 -- 172.21.15.143:0/3946022397 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v80) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0f2c058490 con 0x7f0f28004400 2023-11-08T19:27:44.151 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 64 2023-11-08T19:27:44.153 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.151+0000 7f0f3ef01700 1 -- 172.21.15.143:0/3946022397 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f0f1c06af30 msgr2=0x7f0f1c06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:44.153 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.151+0000 7f0f3ef01700 1 --2- 172.21.15.143:0/3946022397 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f0f1c06af30 0x7f0f1c06d3f0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f0f38158160 tx=0x7f0f2400a000 comp rx=0 tx=0).stop 2023-11-08T19:27:44.153 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.151+0000 7f0f3ef01700 1 -- 172.21.15.143:0/3946022397 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0f28004400 msgr2=0x7f0f3806d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:44.154 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.151+0000 7f0f3ef01700 1 --2- 172.21.15.143:0/3946022397 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0f28004400 0x7f0f3806d120 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c011b00 tx=0x7f0f2c011ec0 comp rx=0 tx=0).stop 2023-11-08T19:27:44.154 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.151+0000 7f0f3ef01700 1 -- 172.21.15.143:0/3946022397 shutdown_connections 2023-11-08T19:27:44.154 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.151+0000 7f0f3ef01700 1 --2- 172.21.15.143:0/3946022397 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f0f1c06af30 0x7f0f1c06d3f0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:44.154 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.151+0000 7f0f3ef01700 1 --2- 172.21.15.143:0/3946022397 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0f381e5c30 0x7f0f380b5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:44.154 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.151+0000 7f0f3ef01700 1 --2- 172.21.15.143:0/3946022397 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0f28004400 0x7f0f3806d120 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:44.155 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.151+0000 7f0f3ef01700 1 --2- 172.21.15.143:0/3946022397 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0f381e3190 0x7f0f38153ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:44.155 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.151+0000 7f0f3ef01700 1 -- 172.21.15.143:0/3946022397 >> 172.21.15.143:0/3946022397 conn(0x7f0f3808b000 msgr2=0x7f0f380af450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:44.155 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.151+0000 7f0f3ef01700 1 -- 172.21.15.143:0/3946022397 shutdown_connections 2023-11-08T19:27:44.155 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.151+0000 7f0f3ef01700 1 -- 172.21.15.143:0/3946022397 wait complete. 2023-11-08T19:27:44.165 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-08T19:27:44.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.285+0000 7f1241c3b700 1 Processor -- start 2023-11-08T19:27:44.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.285+0000 7f1241c3b700 1 -- start start 2023-11-08T19:27:44.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.286+0000 7f1241c3b700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f123c1618f0 0x7f123c06ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:44.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.286+0000 7f1241c3b700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f123c06b150 0x7f123c0cb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:44.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.286+0000 7f1241c3b700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f123c161df0 con 0x7f123c15dd00 2023-11-08T19:27:44.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.286+0000 7f1241c3b700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f123c06b600 con 0x7f123c1618f0 2023-11-08T19:27:44.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.286+0000 7f1241c3b700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f123c0cad50 con 0x7f123c06b150 2023-11-08T19:27:44.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.286+0000 7f123affd700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f123c1618f0 0x7f123c06ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:44.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.286+0000 7f123b7fe700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f123c15dd00 0x7f123c0cca70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:50100/0 (socket says 172.21.15.143:50100) 2023-11-08T19:27:44.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.286+0000 7f123b7fe700 1 -- 172.21.15.143:0/1289202129 learned_addr learned my addr 172.21.15.143:0/1289202129 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:27:44.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.286+0000 7f123bfff700 1 --2- 172.21.15.143:0/1289202129 >> v2:172.21.15.174:3300/0 conn(0x7f123c06b150 0x7f123c0cb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:44.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.286+0000 7f123affd700 1 -- 172.21.15.143:0/1289202129 >> v2:172.21.15.174:3300/0 conn(0x7f123c06b150 msgr2=0x7f123c0cb1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:44.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.286+0000 7f123affd700 1 --2- 172.21.15.143:0/1289202129 >> v2:172.21.15.174:3300/0 conn(0x7f123c06b150 0x7f123c0cb1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:44.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.286+0000 7f123affd700 1 -- 172.21.15.143:0/1289202129 >> v1:172.21.15.143:6789/0 conn(0x7f123c15dd00 legacy=0x7f123c0cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:44.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.286+0000 7f123affd700 1 -- 172.21.15.143:0/1289202129 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f123c1620d0 con 0x7f123c1618f0 2023-11-08T19:27:44.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.286+0000 7f123a7fc700 1 -- 172.21.15.143:0/1289202129 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1110393295 0 0) 0x7f123c161df0 con 0x7f123c15dd00 2023-11-08T19:27:44.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.286+0000 7f123bfff700 1 --2- 172.21.15.143:0/1289202129 >> v2:172.21.15.174:3300/0 conn(0x7f123c06b150 0x7f123c0cb1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:27:44.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.286+0000 7f123affd700 1 --2- 172.21.15.143:0/1289202129 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f123c1618f0 0x7f123c06ac10 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f123c099f70 tx=0x7f122c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1562c47cb9191ab9 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:44.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.286+0000 7f123a7fc700 1 -- 172.21.15.143:0/1289202129 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f122c011830 con 0x7f123c1618f0 2023-11-08T19:27:44.293 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.287+0000 7f123a7fc700 1 -- 172.21.15.143:0/1289202129 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f122c011990 con 0x7f123c1618f0 2023-11-08T19:27:44.293 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.287+0000 7f123a7fc700 1 -- 172.21.15.143:0/1289202129 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f122c011d20 con 0x7f123c1618f0 2023-11-08T19:27:44.293 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.287+0000 7f1241c3b700 1 -- 172.21.15.143:0/1289202129 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f123c1618f0 msgr2=0x7f123c06ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:44.293 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.287+0000 7f1241c3b700 1 --2- 172.21.15.143:0/1289202129 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f123c1618f0 0x7f123c06ac10 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f123c099f70 tx=0x7f122c009b20 comp rx=0 tx=0).stop 2023-11-08T19:27:44.293 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.287+0000 7f1241c3b700 1 -- 172.21.15.143:0/1289202129 shutdown_connections 2023-11-08T19:27:44.294 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.287+0000 7f1241c3b700 1 --2- 172.21.15.143:0/1289202129 >> v2:172.21.15.174:3300/0 conn(0x7f123c06b150 0x7f123c0cb1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:44.294 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.287+0000 7f1241c3b700 1 --2- 172.21.15.143:0/1289202129 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f123c1618f0 0x7f123c06ac10 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:44.294 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.287+0000 7f1241c3b700 1 -- 172.21.15.143:0/1289202129 >> 172.21.15.143:0/1289202129 conn(0x7f123c099ab0 msgr2=0x7f123c0d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:44.294 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.287+0000 7f1241c3b700 1 -- 172.21.15.143:0/1289202129 shutdown_connections 2023-11-08T19:27:44.294 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.287+0000 7f1241c3b700 1 -- 172.21.15.143:0/1289202129 wait complete. 2023-11-08T19:27:44.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.288+0000 7f1241c3b700 1 Processor -- start 2023-11-08T19:27:44.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.288+0000 7f1241c3b700 1 -- start start 2023-11-08T19:27:44.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.289+0000 7f1241c3b700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f123c06b150 0x7f123c0ca050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:44.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.289+0000 7f1241c3b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f123c15dd00 0x7f123c1f3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:44.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.289+0000 7f1241c3b700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f123c1618f0 0x7f123c15bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:44.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.289+0000 7f1241c3b700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f123c0b2df0 con 0x7f123c1618f0 2023-11-08T19:27:44.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.289+0000 7f1241c3b700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f123c1f44e0 con 0x7f123c15dd00 2023-11-08T19:27:44.299 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.289+0000 7f1241c3b700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f123c0c5740 con 0x7f123c06b150 2023-11-08T19:27:44.299 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.289+0000 7f123bfff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f123c1618f0 0x7f123c15bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:44.299 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.289+0000 7f123bfff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f123c1618f0 0x7f123c15bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53956/0 (socket says 172.21.15.143:53956) 2023-11-08T19:27:44.299 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.289+0000 7f123bfff700 1 -- 172.21.15.143:0/1698126618 learned_addr learned my addr 172.21.15.143:0/1698126618 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:44.299 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.289+0000 7f123affd700 1 --2- 172.21.15.143:0/1698126618 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f123c15dd00 0x7f123c1f3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:44.300 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.289+0000 7f123b7fe700 1 --2- 172.21.15.143:0/1698126618 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f123c06b150 0x7f123c0ca050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:44.300 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.290+0000 7f123bfff700 1 -- 172.21.15.143:0/1698126618 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f123c06b150 msgr2=0x7f123c0ca050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:44.300 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.290+0000 7f123bfff700 1 --2- 172.21.15.143:0/1698126618 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f123c06b150 0x7f123c0ca050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:44.300 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.290+0000 7f123bfff700 1 -- 172.21.15.143:0/1698126618 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f123c15dd00 msgr2=0x7f123c1f3fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:44.300 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.290+0000 7f123bfff700 1 --2- 172.21.15.143:0/1698126618 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f123c15dd00 0x7f123c1f3fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:44.301 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.290+0000 7f123bfff700 1 -- 172.21.15.143:0/1698126618 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f122c0094a0 con 0x7f123c1618f0 2023-11-08T19:27:44.301 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.290+0000 7f123bfff700 1 --2- 172.21.15.143:0/1698126618 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f123c1618f0 0x7f123c15bd30 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f1234007ef0 tx=0x7f123400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:44.301 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.290+0000 7f1223fff700 1 -- 172.21.15.143:0/1698126618 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f123400de60 con 0x7f123c1618f0 2023-11-08T19:27:44.301 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.290+0000 7f1223fff700 1 -- 172.21.15.143:0/1698126618 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f123400f040 con 0x7f123c1618f0 2023-11-08T19:27:44.301 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.290+0000 7f1241c3b700 1 -- 172.21.15.143:0/1698126618 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f123c0c5a20 con 0x7f123c1618f0 2023-11-08T19:27:44.302 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.290+0000 7f1223fff700 1 -- 172.21.15.143:0/1698126618 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1234003c60 con 0x7f123c1618f0 2023-11-08T19:27:44.302 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.290+0000 7f1241c3b700 1 -- 172.21.15.143:0/1698126618 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f123c0c5fe0 con 0x7f123c1618f0 2023-11-08T19:27:44.302 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.291+0000 7f1241c3b700 1 -- 172.21.15.143:0/1698126618 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f123c0aff60 con 0x7f123c1618f0 2023-11-08T19:27:44.302 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.291+0000 7f1223fff700 1 -- 172.21.15.143:0/1698126618 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1234004030 con 0x7f123c1618f0 2023-11-08T19:27:44.302 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.292+0000 7f1223fff700 1 --2- 172.21.15.143:0/1698126618 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f122806af00 0x7f122806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:44.303 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.292+0000 7f1223fff700 1 -- 172.21.15.143:0/1698126618 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6888+0+0 (secure 0 0 0) 0x7f1234089c70 con 0x7f123c1618f0 2023-11-08T19:27:44.303 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.292+0000 7f123b7fe700 1 --2- 172.21.15.143:0/1698126618 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f122806af00 0x7f122806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:44.303 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.292+0000 7f123b7fe700 1 --2- 172.21.15.143:0/1698126618 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f122806af00 0x7f122806d3c0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f1230007fd0 tx=0x7f1230009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:44.303 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.296+0000 7f1223fff700 1 -- 172.21.15.143:0/1698126618 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1234054880 con 0x7f123c1618f0 2023-11-08T19:27:44.503 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.502+0000 7f1241c3b700 1 -- 172.21.15.143:0/1698126618 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f123c04eee0 con 0x7f123c1618f0 2023-11-08T19:27:44.504 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.502+0000 7f1223fff700 1 -- 172.21.15.143:0/1698126618 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v80) v1 ==== 80+0+0 (secure 0 0 0) 0x7f123401a080 con 0x7f123c1618f0 2023-11-08T19:27:44.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.504+0000 7f1241c3b700 1 -- 172.21.15.143:0/1698126618 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f122806af00 msgr2=0x7f122806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:44.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.504+0000 7f1241c3b700 1 --2- 172.21.15.143:0/1698126618 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f122806af00 0x7f122806d3c0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f1230007fd0 tx=0x7f1230009040 comp rx=0 tx=0).stop 2023-11-08T19:27:44.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.504+0000 7f1241c3b700 1 -- 172.21.15.143:0/1698126618 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f123c1618f0 msgr2=0x7f123c15bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:44.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.504+0000 7f1241c3b700 1 --2- 172.21.15.143:0/1698126618 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f123c1618f0 0x7f123c15bd30 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f1234007ef0 tx=0x7f123400d3b0 comp rx=0 tx=0).stop 2023-11-08T19:27:44.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.504+0000 7f1241c3b700 1 -- 172.21.15.143:0/1698126618 shutdown_connections 2023-11-08T19:27:44.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.504+0000 7f1241c3b700 1 --2- 172.21.15.143:0/1698126618 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f122806af00 0x7f122806d3c0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:44.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.504+0000 7f1241c3b700 1 --2- 172.21.15.143:0/1698126618 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f123c06b150 0x7f123c0ca050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:44.508 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.504+0000 7f1241c3b700 1 --2- 172.21.15.143:0/1698126618 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f123c1618f0 0x7f123c15bd30 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:44.508 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.504+0000 7f1241c3b700 1 --2- 172.21.15.143:0/1698126618 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f123c15dd00 0x7f123c1f3fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:44.508 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.504+0000 7f1241c3b700 1 -- 172.21.15.143:0/1698126618 >> 172.21.15.143:0/1698126618 conn(0x7f123c099ab0 msgr2=0x7f123c0ad3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:44.508 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.504+0000 7f1241c3b700 1 -- 172.21.15.143:0/1698126618 shutdown_connections 2023-11-08T19:27:44.508 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.504+0000 7f1241c3b700 1 -- 172.21.15.143:0/1698126618 wait complete. 2023-11-08T19:27:44.509 INFO:teuthology.orchestra.run.smithi143.stderr:listed 0 entries 2023-11-08T19:27:44.518 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-08T19:27:44.518 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:27:44.519 DEBUG:teuthology.orchestra.run.smithi143:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-11-08T19:27:44.563 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi143.front.sepia.ceph.com 2023-11-08T19:27:44.564 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:27:44.564 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-08T19:27:44.620 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2023-11-08T19:27:44.620 DEBUG:teuthology.orchestra.run.smithi174:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:27:44.642 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284224k,nr_inodes=4071056,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=31,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19549 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260544k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-08T19:27:44.642 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-08T19:27:44.643 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:27:44.677 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284264k,nr_inodes=4071066,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=44,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20643 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-08T19:27:44.677 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2023-11-08T19:27:44.678 DEBUG:teuthology.orchestra.run.smithi174:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:27:44.703 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284224k,nr_inodes=4071056,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=31,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19549 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260544k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-08T19:27:44.704 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:27:44.837 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.836+0000 7f20f15fe700 1 Processor -- start 2023-11-08T19:27:44.837 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.836+0000 7f20f15fe700 1 -- start start 2023-11-08T19:27:44.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.836+0000 7f20f15fe700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f20ec157e30 0x7f20ec071aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:44.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.836+0000 7f20f15fe700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f20ec0cde00 0x7f20ec0cd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:44.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.836+0000 7f20f15fe700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f20ec0ce290 0x7f20ec0cd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:44.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.836+0000 7f20f15fe700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20ec071680 con 0x7f20ec157e30 2023-11-08T19:27:44.841 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.836+0000 7f20f15fe700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20ec0ceb30 con 0x7f20ec0cde00 2023-11-08T19:27:44.841 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.836+0000 7f20f15fe700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20ec070fe0 con 0x7f20ec0ce290 2023-11-08T19:27:44.841 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.837+0000 7f20eaffd700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f20ec157e30 0x7f20ec071aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:44.841 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.837+0000 7f20eaffd700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f20ec157e30 0x7f20ec071aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53960/0 (socket says 172.21.15.143:53960) 2023-11-08T19:27:44.842 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.837+0000 7f20eaffd700 1 -- 172.21.15.143:0/3322818294 learned_addr learned my addr 172.21.15.143:0/3322818294 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:44.842 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.837+0000 7f20eaffd700 1 -- 172.21.15.143:0/3322818294 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f20ec0ce290 msgr2=0x7f20ec0cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:44.842 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.837+0000 7f20eaffd700 1 --2- 172.21.15.143:0/3322818294 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f20ec0ce290 0x7f20ec0cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:44.842 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.837+0000 7f20eaffd700 1 -- 172.21.15.143:0/3322818294 >> v2:172.21.15.174:3300/0 conn(0x7f20ec0cde00 msgr2=0x7f20ec0cd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:44.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.837+0000 7f20ea7fc700 1 --2- 172.21.15.143:0/3322818294 >> v2:172.21.15.174:3300/0 conn(0x7f20ec0cde00 0x7f20ec0cd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:44.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.837+0000 7f20eaffd700 1 --2- 172.21.15.143:0/3322818294 >> v2:172.21.15.174:3300/0 conn(0x7f20ec0cde00 0x7f20ec0cd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:44.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.837+0000 7f20eaffd700 1 -- 172.21.15.143:0/3322818294 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20ec0ce850 con 0x7f20ec157e30 2023-11-08T19:27:44.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.837+0000 7f20ea7fc700 1 --2- 172.21.15.143:0/3322818294 >> v2:172.21.15.174:3300/0 conn(0x7f20ec0cde00 0x7f20ec0cd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:27:44.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.837+0000 7f20eaffd700 1 --2- 172.21.15.143:0/3322818294 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f20ec157e30 0x7f20ec071aa0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f20ec0cf630 tx=0x7f20e0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bdd6b60bbf5c667 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:44.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.837+0000 7f20e9ffb700 1 -- 172.21.15.143:0/3322818294 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e000b870 con 0x7f20ec157e30 2023-11-08T19:27:44.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.837+0000 7f20e9ffb700 1 -- 172.21.15.143:0/3322818294 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20e000b9d0 con 0x7f20ec157e30 2023-11-08T19:27:44.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.837+0000 7f20e9ffb700 1 -- 172.21.15.143:0/3322818294 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e000bd60 con 0x7f20ec157e30 2023-11-08T19:27:44.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.838+0000 7f20f15fe700 1 -- 172.21.15.143:0/3322818294 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f20ec157e30 msgr2=0x7f20ec071aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:44.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.838+0000 7f20f15fe700 1 --2- 172.21.15.143:0/3322818294 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f20ec157e30 0x7f20ec071aa0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f20ec0cf630 tx=0x7f20e0009bd0 comp rx=0 tx=0).stop 2023-11-08T19:27:44.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.838+0000 7f20f15fe700 1 -- 172.21.15.143:0/3322818294 shutdown_connections 2023-11-08T19:27:44.845 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.838+0000 7f20f15fe700 1 --2- 172.21.15.143:0/3322818294 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f20ec0ce290 0x7f20ec0cd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:44.845 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.838+0000 7f20f15fe700 1 --2- 172.21.15.143:0/3322818294 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f20ec157e30 0x7f20ec071aa0 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:44.845 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.838+0000 7f20f15fe700 1 --2- 172.21.15.143:0/3322818294 >> v2:172.21.15.174:3300/0 conn(0x7f20ec0cde00 0x7f20ec0cd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:44.845 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.838+0000 7f20f15fe700 1 -- 172.21.15.143:0/3322818294 >> 172.21.15.143:0/3322818294 conn(0x7f20ec06b4c0 msgr2=0x7f20ec0cf680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:44.845 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.838+0000 7f20f15fe700 1 -- 172.21.15.143:0/3322818294 shutdown_connections 2023-11-08T19:27:44.846 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.838+0000 7f20f15fe700 1 -- 172.21.15.143:0/3322818294 wait complete. 2023-11-08T19:27:44.848 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.839+0000 7f20f15fe700 1 Processor -- start 2023-11-08T19:27:44.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.839+0000 7f20f15fe700 1 -- start start 2023-11-08T19:27:44.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.840+0000 7f20f15fe700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f20ec0cde00 0x7f20ec154c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:44.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.840+0000 7f20f15fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f20ec0ce290 0x7f20ec1f44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:44.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.840+0000 7f20f15fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f20ec157e30 0x7f20ec0ca550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:44.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.840+0000 7f20f15fe700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20ec154870 con 0x7f20ec157e30 2023-11-08T19:27:44.850 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.840+0000 7f20f15fe700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20ec1f49f0 con 0x7f20ec0ce290 2023-11-08T19:27:44.850 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.840+0000 7f20f15fe700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20ec1541d0 con 0x7f20ec0cde00 2023-11-08T19:27:44.850 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.840+0000 7f20eb7fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f20ec157e30 0x7f20ec0ca550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:44.850 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.840+0000 7f20eb7fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f20ec157e30 0x7f20ec0ca550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53970/0 (socket says 172.21.15.143:53970) 2023-11-08T19:27:44.850 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.840+0000 7f20eb7fe700 1 -- 172.21.15.143:0/214294629 learned_addr learned my addr 172.21.15.143:0/214294629 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:44.851 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.840+0000 7f20eaffd700 1 --2- 172.21.15.143:0/214294629 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f20ec0cde00 0x7f20ec154c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:44.851 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.840+0000 7f20ea7fc700 1 --2- 172.21.15.143:0/214294629 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f20ec0ce290 0x7f20ec1f44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:44.851 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.840+0000 7f20eb7fe700 1 -- 172.21.15.143:0/214294629 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f20ec0cde00 msgr2=0x7f20ec154c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:44.851 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.840+0000 7f20eb7fe700 1 --2- 172.21.15.143:0/214294629 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f20ec0cde00 0x7f20ec154c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:44.851 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.840+0000 7f20eb7fe700 1 -- 172.21.15.143:0/214294629 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f20ec0ce290 msgr2=0x7f20ec1f44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:44.851 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.840+0000 7f20eb7fe700 1 --2- 172.21.15.143:0/214294629 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f20ec0ce290 0x7f20ec1f44b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:44.852 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.840+0000 7f20eb7fe700 1 -- 172.21.15.143:0/214294629 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20e0009550 con 0x7f20ec157e30 2023-11-08T19:27:44.852 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.840+0000 7f20eb7fe700 1 --2- 172.21.15.143:0/214294629 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f20ec157e30 0x7f20ec0ca550 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f20e400b9a0 tx=0x7f20e400bcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:44.852 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.841+0000 7f20d37fe700 1 -- 172.21.15.143:0/214294629 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e400d850 con 0x7f20ec157e30 2023-11-08T19:27:44.852 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.841+0000 7f20d37fe700 1 -- 172.21.15.143:0/214294629 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20e400d9d0 con 0x7f20ec157e30 2023-11-08T19:27:44.852 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.841+0000 7f20f15fe700 1 -- 172.21.15.143:0/214294629 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20ec1f7d40 con 0x7f20ec157e30 2023-11-08T19:27:44.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.841+0000 7f20d37fe700 1 -- 172.21.15.143:0/214294629 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e400dd80 con 0x7f20ec157e30 2023-11-08T19:27:44.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.841+0000 7f20f15fe700 1 -- 172.21.15.143:0/214294629 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20ec0caa90 con 0x7f20ec157e30 2023-11-08T19:27:44.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.842+0000 7f20f15fe700 1 -- 172.21.15.143:0/214294629 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20ec1549b0 con 0x7f20ec157e30 2023-11-08T19:27:44.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.842+0000 7f20d37fe700 1 -- 172.21.15.143:0/214294629 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f20e400e420 con 0x7f20ec157e30 2023-11-08T19:27:44.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.842+0000 7f20d37fe700 1 --2- 172.21.15.143:0/214294629 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f20d806af00 0x7f20d806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:44.854 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.842+0000 7f20d37fe700 1 -- 172.21.15.143:0/214294629 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6888+0+0 (secure 0 0 0) 0x7f20e4006e40 con 0x7f20ec157e30 2023-11-08T19:27:44.854 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.843+0000 7f20eaffd700 1 --2- 172.21.15.143:0/214294629 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f20d806af00 0x7f20d806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:44.854 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.843+0000 7f20eaffd700 1 --2- 172.21.15.143:0/214294629 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f20d806af00 0x7f20d806d3c0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f20e0000c00 tx=0x7f20e0008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:44.854 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:44.847+0000 7f20d37fe700 1 -- 172.21.15.143:0/214294629 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f20ec1549b0 con 0x7f20ec157e30 2023-11-08T19:27:45.078 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.076+0000 7f20f15fe700 1 -- 172.21.15.143:0/214294629 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f20ec1549b0 con 0x7f20ec157e30 2023-11-08T19:27:45.078 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.076+0000 7f20d37fe700 1 -- 172.21.15.143:0/214294629 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+4921 (secure 0 0 0) 0x7f20ec1549b0 con 0x7f20ec157e30 2023-11-08T19:27:45.078 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:27:45.079 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":43,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5510,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6836/3796015890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3796015890},{"type":"v1","addr":"172.21.15.174:6836","nonce":3796015890}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37},{"gid":6471,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6834/432456394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":432456394},{"type":"v1","addr":"172.21.15.143:6834","nonce":432456394}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37},{"gid":6474,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/310920972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":310920972},{"type":"v1","addr":"172.21.15.143:6835","nonce":310920972}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:27:31.702539+0000","modified":"2023-11-08T19:27:40.808769+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6489},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6489":{"gid":6489,"name":"d","rank":0,"incarnation":38,"state":"up:active","state_seq":3,"addr":"172.21.15.174:6837/1950356087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6835","nonce":1950356087},{"type":"v1","addr":"172.21.15.174:6837","nonce":1950356087}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-08T19:27:45.080 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.078+0000 7f20f15fe700 1 -- 172.21.15.143:0/214294629 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f20d806af00 msgr2=0x7f20d806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:45.080 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.078+0000 7f20f15fe700 1 --2- 172.21.15.143:0/214294629 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f20d806af00 0x7f20d806d3c0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f20e0000c00 tx=0x7f20e0008030 comp rx=0 tx=0).stop 2023-11-08T19:27:45.081 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.078+0000 7f20f15fe700 1 -- 172.21.15.143:0/214294629 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f20ec157e30 msgr2=0x7f20ec0ca550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:45.081 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.078+0000 7f20f15fe700 1 --2- 172.21.15.143:0/214294629 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f20ec157e30 0x7f20ec0ca550 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f20e400b9a0 tx=0x7f20e400bcb0 comp rx=0 tx=0).stop 2023-11-08T19:27:45.081 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.078+0000 7f20f15fe700 1 -- 172.21.15.143:0/214294629 shutdown_connections 2023-11-08T19:27:45.082 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.078+0000 7f20f15fe700 1 --2- 172.21.15.143:0/214294629 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f20d806af00 0x7f20d806d3c0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:45.082 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.078+0000 7f20f15fe700 1 --2- 172.21.15.143:0/214294629 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f20ec0cde00 0x7f20ec154c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:45.082 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.078+0000 7f20f15fe700 1 --2- 172.21.15.143:0/214294629 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f20ec157e30 0x7f20ec0ca550 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:45.082 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.078+0000 7f20f15fe700 1 --2- 172.21.15.143:0/214294629 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f20ec0ce290 0x7f20ec1f44b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:45.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.078+0000 7f20f15fe700 1 -- 172.21.15.143:0/214294629 >> 172.21.15.143:0/214294629 conn(0x7f20ec06b4c0 msgr2=0x7f20ec0b08f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:45.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.078+0000 7f20f15fe700 1 -- 172.21.15.143:0/214294629 shutdown_connections 2023-11-08T19:27:45.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.078+0000 7f20f15fe700 1 -- 172.21.15.143:0/214294629 wait complete. 2023-11-08T19:27:45.083 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 43 2023-11-08T19:27:45.092 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:27:45.212 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.210+0000 7f874b793700 1 Processor -- start 2023-11-08T19:27:45.212 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.211+0000 7f874b793700 1 -- start start 2023-11-08T19:27:45.214 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.211+0000 7f874b793700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f87441618f0 0x7f874406ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:45.214 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.211+0000 7f874b793700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f874406b150 0x7f87440cb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:45.215 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.211+0000 7f874b793700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8744161df0 con 0x7f874415dd00 2023-11-08T19:27:45.215 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.211+0000 7f874b793700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f874406b600 con 0x7f874406b150 2023-11-08T19:27:45.215 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.211+0000 7f874b793700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87440cad50 con 0x7f87441618f0 2023-11-08T19:27:45.215 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.212+0000 7f8749d30700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f874406b150 0x7f87440cb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:45.216 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.212+0000 7f8749d30700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f874406b150 0x7f87440cb1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53978/0 (socket says 172.21.15.143:53978) 2023-11-08T19:27:45.216 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.212+0000 7f8749d30700 1 -- 172.21.15.143:0/743978229 learned_addr learned my addr 172.21.15.143:0/743978229 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:45.216 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.212+0000 7f8748d2e700 1 --2- 172.21.15.143:0/743978229 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f87441618f0 0x7f874406ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:45.216 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.212+0000 7f8749d30700 1 -- 172.21.15.143:0/743978229 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f87441618f0 msgr2=0x7f874406ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:45.216 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.212+0000 7f8749d30700 1 --2- 172.21.15.143:0/743978229 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f87441618f0 0x7f874406ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:45.217 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.212+0000 7f8749d30700 1 -- 172.21.15.143:0/743978229 >> v1:172.21.15.174:6789/0 conn(0x7f874415dd00 legacy=0x7f87440cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:45.217 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.212+0000 7f8749d30700 1 -- 172.21.15.143:0/743978229 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87441620d0 con 0x7f874406b150 2023-11-08T19:27:45.217 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.212+0000 7f8749d30700 1 --2- 172.21.15.143:0/743978229 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f874406b150 0x7f87440cb1a0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f873c009e50 tx=0x7f873c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5174d7c0d62f7af1 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:45.217 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.212+0000 7f873bfff700 1 -- 172.21.15.143:0/743978229 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f873c00e070 con 0x7f874406b150 2023-11-08T19:27:45.217 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.212+0000 7f873bfff700 1 -- 172.21.15.143:0/743978229 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f873c003a20 con 0x7f874406b150 2023-11-08T19:27:45.217 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.212+0000 7f873bfff700 1 -- 172.21.15.143:0/743978229 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f873c003dd0 con 0x7f874406b150 2023-11-08T19:27:45.218 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.213+0000 7f874b793700 1 -- 172.21.15.143:0/743978229 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f874406b150 msgr2=0x7f87440cb1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:45.218 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.213+0000 7f874b793700 1 --2- 172.21.15.143:0/743978229 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f874406b150 0x7f87440cb1a0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f873c009e50 tx=0x7f873c00b330 comp rx=0 tx=0).stop 2023-11-08T19:27:45.218 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.213+0000 7f874b793700 1 -- 172.21.15.143:0/743978229 shutdown_connections 2023-11-08T19:27:45.218 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.213+0000 7f874b793700 1 --2- 172.21.15.143:0/743978229 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f874406b150 0x7f87440cb1a0 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:45.219 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.213+0000 7f874b793700 1 --2- 172.21.15.143:0/743978229 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f87441618f0 0x7f874406ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:45.219 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.213+0000 7f874b793700 1 -- 172.21.15.143:0/743978229 >> 172.21.15.143:0/743978229 conn(0x7f8744099ab0 msgr2=0x7f87440d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:45.219 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.213+0000 7f874b793700 1 -- 172.21.15.143:0/743978229 shutdown_connections 2023-11-08T19:27:45.219 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.213+0000 7f874b793700 1 -- 172.21.15.143:0/743978229 wait complete. 2023-11-08T19:27:45.222 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.214+0000 7f874b793700 1 Processor -- start 2023-11-08T19:27:45.222 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.214+0000 7f874b793700 1 -- start start 2023-11-08T19:27:45.222 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.215+0000 7f874b793700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f874406b150 0x7f8744149b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:45.223 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.215+0000 7f874b793700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f874415dd00 0x7f87440c67f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:45.223 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.215+0000 7f874b793700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f87441618f0 0x7f87440c6d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:45.223 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.215+0000 7f874b793700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f874415f2c0 con 0x7f87441618f0 2023-11-08T19:27:45.223 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.215+0000 7f874b793700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f874415e3d0 con 0x7f874406b150 2023-11-08T19:27:45.223 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.215+0000 7f874b793700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f874415e6b0 con 0x7f874415dd00 2023-11-08T19:27:45.224 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.215+0000 7f8749d30700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f87441618f0 0x7f87440c6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:45.224 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.215+0000 7f8749d30700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f87441618f0 0x7f87440c6d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53984/0 (socket says 172.21.15.143:53984) 2023-11-08T19:27:45.224 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.215+0000 7f8749d30700 1 -- 172.21.15.143:0/3975337344 learned_addr learned my addr 172.21.15.143:0/3975337344 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:45.224 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.215+0000 7f8748d2e700 1 --2- 172.21.15.143:0/3975337344 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f874415dd00 0x7f87440c67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:45.224 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.215+0000 7f8749d30700 1 -- 172.21.15.143:0/3975337344 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f874415dd00 msgr2=0x7f87440c67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:45.225 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.215+0000 7f874952f700 1 --2- 172.21.15.143:0/3975337344 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f874406b150 0x7f8744149b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:45.225 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.215+0000 7f8749d30700 1 --2- 172.21.15.143:0/3975337344 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f874415dd00 0x7f87440c67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:45.225 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.215+0000 7f8749d30700 1 -- 172.21.15.143:0/3975337344 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f874406b150 msgr2=0x7f8744149b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:45.225 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.215+0000 7f8749d30700 1 --2- 172.21.15.143:0/3975337344 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f874406b150 0x7f8744149b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:45.225 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.215+0000 7f8749d30700 1 -- 172.21.15.143:0/3975337344 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f873c009b30 con 0x7f87441618f0 2023-11-08T19:27:45.226 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.215+0000 7f8748d2e700 1 --2- 172.21.15.143:0/3975337344 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f874415dd00 0x7f87440c67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:27:45.226 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.215+0000 7f874952f700 1 --2- 172.21.15.143:0/3975337344 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f874406b150 0x7f8744149b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:27:45.226 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.215+0000 7f8749d30700 1 --2- 172.21.15.143:0/3975337344 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f87441618f0 0x7f87440c6d30 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f8744070a90 tx=0x7f873c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:45.226 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.215+0000 7f8739ffb700 1 -- 172.21.15.143:0/3975337344 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f873c004440 con 0x7f87441618f0 2023-11-08T19:27:45.226 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.215+0000 7f8739ffb700 1 -- 172.21.15.143:0/3975337344 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f873c0045a0 con 0x7f87441618f0 2023-11-08T19:27:45.227 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.215+0000 7f874b793700 1 -- 172.21.15.143:0/3975337344 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f874415e990 con 0x7f87441618f0 2023-11-08T19:27:45.227 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.215+0000 7f8739ffb700 1 -- 172.21.15.143:0/3975337344 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f873c004950 con 0x7f87441618f0 2023-11-08T19:27:45.227 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.216+0000 7f874b793700 1 -- 172.21.15.143:0/3975337344 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f874415ef50 con 0x7f87441618f0 2023-11-08T19:27:45.227 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.217+0000 7f874b793700 1 -- 172.21.15.143:0/3975337344 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87440aff60 con 0x7f87441618f0 2023-11-08T19:27:45.228 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.217+0000 7f8739ffb700 1 -- 172.21.15.143:0/3975337344 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f873c025450 con 0x7f87441618f0 2023-11-08T19:27:45.229 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.217+0000 7f8739ffb700 1 --2- 172.21.15.143:0/3975337344 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f872806af00 0x7f872806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:45.229 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.218+0000 7f8739ffb700 1 -- 172.21.15.143:0/3975337344 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6888+0+0 (secure 0 0 0) 0x7f873c08b340 con 0x7f87441618f0 2023-11-08T19:27:45.230 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.218+0000 7f874952f700 1 --2- 172.21.15.143:0/3975337344 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f872806af00 0x7f872806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:45.230 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.218+0000 7f874952f700 1 --2- 172.21.15.143:0/3975337344 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f872806af00 0x7f872806d3c0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f87340017f0 tx=0x7f8734009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:45.231 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.222+0000 7f8739ffb700 1 -- 172.21.15.143:0/3975337344 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f873c055f60 con 0x7f87441618f0 2023-11-08T19:27:45.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.411+0000 7f874b793700 1 -- 172.21.15.143:0/3975337344 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f874406af70 con 0x7f87441618f0 2023-11-08T19:27:45.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.413+0000 7f8739ffb700 1 -- 172.21.15.143:0/3975337344 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+21997 (secure 0 0 0) 0x7f873c059580 con 0x7f87441618f0 2023-11-08T19:27:45.415 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:27:45.415 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":80,"fsid":"968483af-77fc-4068-ad7c-5c2a1a4613d7","created":"2023-11-08T19:24:32.826484+0000","modified":"2023-11-08T19:27:40.353576+0000","last_up_change":"2023-11-08T19:24:43.573880+0000","last_in_change":"2023-11-08T19:24:35.314525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:44.551530+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"snap-test-pool","create_time":"2023-11-08T19:26:02.874384+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":36,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"test_data_pool","create_time":"2023-11-08T19:27:05.601949+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":66,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_metadata_pool","create_time":"2023-11-08T19:27:06.536369+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-11-08T19:27:30.137037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-11-08T19:27:30.660318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bae98ec1-d94e-403d-a220-a522a53ea6f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6809","nonce":1487830566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6811","nonce":1487830566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6815","nonce":1487830566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6813","nonce":1487830566}]},"public_addr":"172.21.15.143:6809/1487830566","cluster_addr":"172.21.15.143:6811/1487830566","heartbeat_back_addr":"172.21.15.143:6815/1487830566","heartbeat_front_addr":"172.21.15.143:6813/1487830566","state":["exists","up"]},{"osd":1,"uuid":"5cb6538a-1ccb-4322-8633-09f66af0b7e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6801","nonce":787341397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6803","nonce":787341397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6807","nonce":787341397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6805","nonce":787341397}]},"public_addr":"172.21.15.143:6801/787341397","cluster_addr":"172.21.15.143:6803/787341397","heartbeat_back_addr":"172.21.15.143:6807/787341397","heartbeat_front_addr":"172.21.15.143:6805/787341397","state":["exists","up"]},{"osd":2,"uuid":"cdbf522c-60ef-4d60-9653-8dcb529c101a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6822","nonce":2301325597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6826","nonce":2301325597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6831","nonce":2301325597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6829","nonce":2301325597}]},"public_addr":"172.21.15.143:6822/2301325597","cluster_addr":"172.21.15.143:6826/2301325597","heartbeat_back_addr":"172.21.15.143:6831/2301325597","heartbeat_front_addr":"172.21.15.143:6829/2301325597","state":["exists","up"]},{"osd":3,"uuid":"4e2783cf-ebb9-4dd8-8245-bc1d17ce4325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6817","nonce":2021504428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6819","nonce":2021504428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6827","nonce":2021504428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6823","nonce":2021504428}]},"public_addr":"172.21.15.143:6817/2021504428","cluster_addr":"172.21.15.143:6819/2021504428","heartbeat_back_addr":"172.21.15.143:6827/2021504428","heartbeat_front_addr":"172.21.15.143:6823/2021504428","state":["exists","up"]},{"osd":4,"uuid":"691256fa-9b77-49b9-a346-7ae1adf0ca80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6801","nonce":1993380595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6803","nonce":1993380595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6807","nonce":1993380595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6805","nonce":1993380595}]},"public_addr":"172.21.15.174:6801/1993380595","cluster_addr":"172.21.15.174:6803/1993380595","heartbeat_back_addr":"172.21.15.174:6807/1993380595","heartbeat_front_addr":"172.21.15.174:6805/1993380595","state":["exists","up"]},{"osd":5,"uuid":"756fc352-48c9-46ce-b52e-0541fe6f8525","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6809","nonce":1777923303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6811","nonce":1777923303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6815","nonce":1777923303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6813","nonce":1777923303}]},"public_addr":"172.21.15.174:6809/1777923303","cluster_addr":"172.21.15.174:6811/1777923303","heartbeat_back_addr":"172.21.15.174:6815/1777923303","heartbeat_front_addr":"172.21.15.174:6813/1777923303","state":["exists","up"]},{"osd":6,"uuid":"fab2ac2b-ed97-4cde-9825-b480eabf8b8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6825","nonce":2755130316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6827","nonce":2755130316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6831","nonce":2755130316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6829","nonce":2755130316}]},"public_addr":"172.21.15.174:6825/2755130316","cluster_addr":"172.21.15.174:6827/2755130316","heartbeat_back_addr":"172.21.15.174:6831/2755130316","heartbeat_front_addr":"172.21.15.174:6829/2755130316","state":["exists","up"]},{"osd":7,"uuid":"3de9b797-2ab6-4d41-a863-7e7d638fd703","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6817","nonce":2847141411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6819","nonce":2847141411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6823","nonce":2847141411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6821","nonce":2847141411}]},"public_addr":"172.21.15.174:6817/2847141411","cluster_addr":"172.21.15.174:6819/2847141411","heartbeat_back_addr":"172.21.15.174:6823/2847141411","heartbeat_front_addr":"172.21.15.174:6821/2847141411","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.477076+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.496890+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.539305+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.454673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.538881+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.521294+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.520712+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.540404+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.a","mappings":[{"from":1,"to":7}]},{"pgid":"14.17","mappings":[{"from":1,"to":7}]},{"pgid":"14.29","mappings":[{"from":1,"to":2}]},{"pgid":"14.32","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:27:45.417 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.415+0000 7f874b793700 1 -- 172.21.15.143:0/3975337344 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f872806af00 msgr2=0x7f872806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:45.417 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.415+0000 7f874b793700 1 --2- 172.21.15.143:0/3975337344 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f872806af00 0x7f872806d3c0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f87340017f0 tx=0x7f8734009040 comp rx=0 tx=0).stop 2023-11-08T19:27:45.417 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.415+0000 7f874b793700 1 -- 172.21.15.143:0/3975337344 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f87441618f0 msgr2=0x7f87440c6d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:45.417 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.415+0000 7f874b793700 1 --2- 172.21.15.143:0/3975337344 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f87441618f0 0x7f87440c6d30 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f8744070a90 tx=0x7f873c004ad0 comp rx=0 tx=0).stop 2023-11-08T19:27:45.417 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.415+0000 7f874b793700 1 -- 172.21.15.143:0/3975337344 shutdown_connections 2023-11-08T19:27:45.418 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.415+0000 7f874b793700 1 --2- 172.21.15.143:0/3975337344 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f872806af00 0x7f872806d3c0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:45.419 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.415+0000 7f874b793700 1 --2- 172.21.15.143:0/3975337344 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f87441618f0 0x7f87440c6d30 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:45.419 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.415+0000 7f874b793700 1 --2- 172.21.15.143:0/3975337344 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f874406b150 0x7f8744149b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:45.420 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.415+0000 7f874b793700 1 --2- 172.21.15.143:0/3975337344 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f874415dd00 0x7f87440c67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:45.421 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.415+0000 7f874b793700 1 -- 172.21.15.143:0/3975337344 >> 172.21.15.143:0/3975337344 conn(0x7f8744099ab0 msgr2=0x7f874406ece0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:45.421 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.415+0000 7f874b793700 1 -- 172.21.15.143:0/3975337344 shutdown_connections 2023-11-08T19:27:45.421 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.415+0000 7f874b793700 1 -- 172.21.15.143:0/3975337344 wait complete. 2023-11-08T19:27:45.429 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T19:24:44.551530+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'snap-test-pool', 'create_time': '2023-11-08T19:26:02.874384+0000', 'flags': 16385, 'flags_names': 'hashpspool,pool_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '36', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'pool', 'snap_seq': 2, 'snap_epoch': 36, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'test_data_pool', 'create_time': '2023-11-08T19:27:05.601949+0000', 'flags': 16385, 'flags_names': 'hashpspool,pool_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'pool', 'snap_seq': 2, 'snap_epoch': 66, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'test_metadata_pool', 'create_time': '2023-11-08T19:27:06.536369+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-08T19:27:30.137037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2023-11-08T19:27:30.660318+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-08T19:27:45.430 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T19:27:45.553 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.552+0000 7f862fc62700 1 Processor -- start 2023-11-08T19:27:45.554 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.552+0000 7f862fc62700 1 -- start start 2023-11-08T19:27:45.556 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.553+0000 7f862fc62700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f862815d4f0 0x7f86280cb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:45.556 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.553+0000 7f862fc62700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f86280ca7e0 0x7f86280ce660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:45.556 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.553+0000 7f862fc62700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86280cd2e0 con 0x7f8628070ff0 2023-11-08T19:27:45.556 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.553+0000 7f862fc62700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86280cac60 con 0x7f862815d4f0 2023-11-08T19:27:45.557 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.553+0000 7f862fc62700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86280ceba0 con 0x7f86280ca7e0 2023-11-08T19:27:45.557 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.553+0000 7f862d9fe700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f862815d4f0 0x7f86280cb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:45.557 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.553+0000 7f862d1fd700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f86280ca7e0 0x7f86280ce660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:45.557 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.553+0000 7f862e1ff700 1 --1- >> v1:172.21.15.174:6789/0 conn(0x7f8628070ff0 0x7f862806cac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.174:6789/0 says I am v1:172.21.15.143:32848/0 (socket says 172.21.15.143:32848) 2023-11-08T19:27:45.557 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.553+0000 7f862e1ff700 1 -- 172.21.15.143:0/991023743 learned_addr learned my addr 172.21.15.143:0/991023743 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:27:45.558 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.554+0000 7f862d9fe700 1 -- 172.21.15.143:0/991023743 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f86280ca7e0 msgr2=0x7f86280ce660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:45.558 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.554+0000 7f862d9fe700 1 --2- 172.21.15.143:0/991023743 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f86280ca7e0 0x7f86280ce660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:45.558 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.554+0000 7f862d9fe700 1 -- 172.21.15.143:0/991023743 >> v1:172.21.15.174:6789/0 conn(0x7f8628070ff0 legacy=0x7f862806cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:45.558 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.554+0000 7f862c9fc700 1 -- 172.21.15.143:0/991023743 <== mon.1 v1:172.21.15.174:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 712783113 0 0) 0x7f86280cd2e0 con 0x7f8628070ff0 2023-11-08T19:27:45.558 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.554+0000 7f862d9fe700 1 -- 172.21.15.143:0/991023743 --> v2:172.21.15.174:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86280cd5c0 con 0x7f862815d4f0 2023-11-08T19:27:45.559 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.554+0000 7f862d1fd700 1 --2- 172.21.15.143:0/991023743 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f86280ca7e0 0x7f86280ce660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T19:27:45.559 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.554+0000 7f862d9fe700 1 --2- 172.21.15.143:0/991023743 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f862815d4f0 0x7f86280cb7d0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f862400aa10 tx=0x7f862400ad20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=607bc1ac37e2aaee server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:45.559 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.554+0000 7f862c9fc700 1 -- 172.21.15.143:0/991023743 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f862400cb30 con 0x7f862815d4f0 2023-11-08T19:27:45.561 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.554+0000 7f862c9fc700 1 -- 172.21.15.143:0/991023743 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f862400cc90 con 0x7f862815d4f0 2023-11-08T19:27:45.561 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.554+0000 7f862c9fc700 1 -- 172.21.15.143:0/991023743 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8624018400 con 0x7f862815d4f0 2023-11-08T19:27:45.562 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.555+0000 7f862fc62700 1 -- 172.21.15.143:0/991023743 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f862815d4f0 msgr2=0x7f86280cb7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:45.562 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.555+0000 7f862fc62700 1 --2- 172.21.15.143:0/991023743 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f862815d4f0 0x7f86280cb7d0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f862400aa10 tx=0x7f862400ad20 comp rx=0 tx=0).stop 2023-11-08T19:27:45.562 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.555+0000 7f862fc62700 1 -- 172.21.15.143:0/991023743 shutdown_connections 2023-11-08T19:27:45.562 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.555+0000 7f862fc62700 1 --2- 172.21.15.143:0/991023743 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f862815d4f0 0x7f86280cb7d0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:45.562 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.555+0000 7f862fc62700 1 --2- 172.21.15.143:0/991023743 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f86280ca7e0 0x7f86280ce660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:45.563 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.555+0000 7f862fc62700 1 -- 172.21.15.143:0/991023743 >> 172.21.15.143:0/991023743 conn(0x7f86280cf710 msgr2=0x7f86280cfb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:45.563 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.555+0000 7f862fc62700 1 -- 172.21.15.143:0/991023743 shutdown_connections 2023-11-08T19:27:45.563 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.555+0000 7f862fc62700 1 -- 172.21.15.143:0/991023743 wait complete. 2023-11-08T19:27:45.566 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.556+0000 7f862fc62700 1 Processor -- start 2023-11-08T19:27:45.566 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.556+0000 7f862fc62700 1 -- start start 2023-11-08T19:27:45.567 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.557+0000 7f862fc62700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8628070ff0 0x7f86280c83e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:45.567 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.557+0000 7f862fc62700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86280ca7e0 0x7f8628150800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:45.567 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.557+0000 7f862fc62700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f862815d4f0 0x7f8628150f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:45.567 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.557+0000 7f862fc62700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86280c95f0 con 0x7f86280ca7e0 2023-11-08T19:27:45.567 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.557+0000 7f862fc62700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86280c9a80 con 0x7f8628070ff0 2023-11-08T19:27:45.568 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.557+0000 7f862fc62700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86280c89e0 con 0x7f862815d4f0 2023-11-08T19:27:45.568 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.557+0000 7f862d1fd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86280ca7e0 0x7f8628150800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:45.568 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.557+0000 7f862d1fd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86280ca7e0 0x7f8628150800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53996/0 (socket says 172.21.15.143:53996) 2023-11-08T19:27:45.568 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.557+0000 7f862d1fd700 1 -- 172.21.15.143:0/212194745 learned_addr learned my addr 172.21.15.143:0/212194745 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:45.568 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.557+0000 7f862e1ff700 1 --2- 172.21.15.143:0/212194745 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f862815d4f0 0x7f8628150f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:45.569 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.557+0000 7f862d9fe700 1 --2- 172.21.15.143:0/212194745 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8628070ff0 0x7f86280c83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:45.569 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.557+0000 7f862d1fd700 1 -- 172.21.15.143:0/212194745 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f862815d4f0 msgr2=0x7f8628150f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:45.569 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.557+0000 7f862d1fd700 1 --2- 172.21.15.143:0/212194745 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f862815d4f0 0x7f8628150f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:45.569 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.557+0000 7f862d1fd700 1 -- 172.21.15.143:0/212194745 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8628070ff0 msgr2=0x7f86280c83e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:45.569 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.557+0000 7f862d1fd700 1 --2- 172.21.15.143:0/212194745 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8628070ff0 0x7f86280c83e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:45.570 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.557+0000 7f862d1fd700 1 -- 172.21.15.143:0/212194745 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86180094a0 con 0x7f86280ca7e0 2023-11-08T19:27:45.570 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.557+0000 7f862d1fd700 1 --2- 172.21.15.143:0/212194745 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86280ca7e0 0x7f8628150800 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f861800ebe0 tx=0x7f861800eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:45.570 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.557+0000 7f861e7fc700 1 -- 172.21.15.143:0/212194745 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f861800c850 con 0x7f86280ca7e0 2023-11-08T19:27:45.570 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.557+0000 7f862fc62700 1 -- 172.21.15.143:0/212194745 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f862400a640 con 0x7f86280ca7e0 2023-11-08T19:27:45.570 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.557+0000 7f861e7fc700 1 -- 172.21.15.143:0/212194745 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f861800c9b0 con 0x7f86280ca7e0 2023-11-08T19:27:45.570 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.557+0000 7f861e7fc700 1 -- 172.21.15.143:0/212194745 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f861800cd40 con 0x7f86280ca7e0 2023-11-08T19:27:45.571 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.557+0000 7f862fc62700 1 -- 172.21.15.143:0/212194745 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86280c8fa0 con 0x7f86280ca7e0 2023-11-08T19:27:45.571 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.558+0000 7f862fc62700 1 -- 172.21.15.143:0/212194745 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86280c97a0 con 0x7f86280ca7e0 2023-11-08T19:27:45.571 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.559+0000 7f861e7fc700 1 -- 172.21.15.143:0/212194745 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8618013470 con 0x7f86280ca7e0 2023-11-08T19:27:45.571 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.559+0000 7f861e7fc700 1 --2- 172.21.15.143:0/212194745 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f860c06af30 0x7f860c06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:45.571 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.559+0000 7f861e7fc700 1 -- 172.21.15.143:0/212194745 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6888+0+0 (secure 0 0 0) 0x7f8618089ab0 con 0x7f86280ca7e0 2023-11-08T19:27:45.572 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.559+0000 7f862d9fe700 1 --2- 172.21.15.143:0/212194745 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f860c06af30 0x7f860c06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:45.572 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.560+0000 7f862d9fe700 1 --2- 172.21.15.143:0/212194745 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f860c06af30 0x7f860c06d3f0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f8624000c00 tx=0x7f8624012000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:45.572 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.563+0000 7f861e7fc700 1 -- 172.21.15.143:0/212194745 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f86280c97a0 con 0x7f86280ca7e0 2023-11-08T19:27:45.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.748+0000 7f862fc62700 1 -- 172.21.15.143:0/212194745 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f862804eee0 con 0x7f86280ca7e0 2023-11-08T19:27:45.750 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.749+0000 7f861e7fc700 1 -- 172.21.15.143:0/212194745 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v80) v1 ==== 93+0+10 (secure 0 0 0) 0x7f86180546d0 con 0x7f86280ca7e0 2023-11-08T19:27:45.750 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2023-11-08T19:27:45.752 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.751+0000 7f862fc62700 1 -- 172.21.15.143:0/212194745 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f860c06af30 msgr2=0x7f860c06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:45.752 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.751+0000 7f862fc62700 1 --2- 172.21.15.143:0/212194745 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f860c06af30 0x7f860c06d3f0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f8624000c00 tx=0x7f8624012000 comp rx=0 tx=0).stop 2023-11-08T19:27:45.753 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.751+0000 7f862fc62700 1 -- 172.21.15.143:0/212194745 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86280ca7e0 msgr2=0x7f8628150800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:45.753 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.751+0000 7f862fc62700 1 --2- 172.21.15.143:0/212194745 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86280ca7e0 0x7f8628150800 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f861800ebe0 tx=0x7f861800eef0 comp rx=0 tx=0).stop 2023-11-08T19:27:45.753 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.751+0000 7f862fc62700 1 -- 172.21.15.143:0/212194745 shutdown_connections 2023-11-08T19:27:45.753 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.751+0000 7f862fc62700 1 --2- 172.21.15.143:0/212194745 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f860c06af30 0x7f860c06d3f0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:45.753 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.751+0000 7f862fc62700 1 --2- 172.21.15.143:0/212194745 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f862815d4f0 0x7f8628150f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:45.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.751+0000 7f862fc62700 1 --2- 172.21.15.143:0/212194745 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8628070ff0 0x7f86280c83e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:45.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.751+0000 7f862fc62700 1 --2- 172.21.15.143:0/212194745 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86280ca7e0 0x7f8628150800 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:45.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.751+0000 7f862fc62700 1 -- 172.21.15.143:0/212194745 >> 172.21.15.143:0/212194745 conn(0x7f86280cf710 msgr2=0x7f8628074340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:45.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.751+0000 7f862fc62700 1 -- 172.21.15.143:0/212194745 shutdown_connections 2023-11-08T19:27:45.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.751+0000 7f862fc62700 1 -- 172.21.15.143:0/212194745 wait complete. 2023-11-08T19:27:45.765 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get snap-test-pool pg_num 2023-11-08T19:27:45.888 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.887+0000 7fd7caa25700 1 Processor -- start 2023-11-08T19:27:45.889 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.887+0000 7fd7caa25700 1 -- start start 2023-11-08T19:27:45.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.888+0000 7fd7caa25700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fd7c4143a30 0x7fd7c406bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:45.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.888+0000 7fd7caa25700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7c406c510 con 0x7fd7c40c5f90 2023-11-08T19:27:45.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.888+0000 7fd7caa25700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7c4143e10 con 0x7fd7c40cc8c0 2023-11-08T19:27:45.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.888+0000 7fd7caa25700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7c415e110 con 0x7fd7c4143a30 2023-11-08T19:27:45.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.888+0000 7fd7c8fc2700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fd7c40c5f90 0x7fd7c40ce800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:50160/0 (socket says 172.21.15.143:50160) 2023-11-08T19:27:45.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.888+0000 7fd7c8fc2700 1 -- 172.21.15.143:0/1045696657 learned_addr learned my addr 172.21.15.143:0/1045696657 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:27:45.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.888+0000 7fd7c3fff700 1 --2- 172.21.15.143:0/1045696657 >> v2:172.21.15.174:3300/0 conn(0x7fd7c4143a30 0x7fd7c406bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:45.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.888+0000 7fd7c2ffd700 1 -- 172.21.15.143:0/1045696657 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1050834261 0 0) 0x7fd7c406c510 con 0x7fd7c40c5f90 2023-11-08T19:27:45.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.888+0000 7fd7c2ffd700 1 -- 172.21.15.143:0/1045696657 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7ac0035b0 con 0x7fd7c40c5f90 2023-11-08T19:27:45.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.888+0000 7fd7c3fff700 1 -- 172.21.15.143:0/1045696657 >> v1:172.21.15.174:6789/0 conn(0x7fd7c40cc8c0 legacy=0x7fd7c40c5880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:45.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.888+0000 7fd7c3fff700 1 -- 172.21.15.143:0/1045696657 >> v1:172.21.15.143:6789/0 conn(0x7fd7c40c5f90 legacy=0x7fd7c40ce800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:45.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.888+0000 7fd7c3fff700 1 -- 172.21.15.143:0/1045696657 --> v2:172.21.15.174:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7c40c6460 con 0x7fd7c4143a30 2023-11-08T19:27:45.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.889+0000 7fd7c3fff700 1 --2- 172.21.15.143:0/1045696657 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd7c4143a30 0x7fd7c406bfd0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fd7b8009840 tx=0x7fd7b8009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a4e3b1a42dd18cd server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:45.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.889+0000 7fd7c2ffd700 1 -- 172.21.15.143:0/1045696657 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b8011830 con 0x7fd7c4143a30 2023-11-08T19:27:45.895 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.889+0000 7fd7c2ffd700 1 -- 172.21.15.143:0/1045696657 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7b8011990 con 0x7fd7c4143a30 2023-11-08T19:27:45.895 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.889+0000 7fd7c2ffd700 1 -- 172.21.15.143:0/1045696657 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b8011d00 con 0x7fd7c4143a30 2023-11-08T19:27:45.895 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.889+0000 7fd7caa25700 1 -- 172.21.15.143:0/1045696657 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd7c4143a30 msgr2=0x7fd7c406bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:45.895 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.889+0000 7fd7caa25700 1 --2- 172.21.15.143:0/1045696657 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd7c4143a30 0x7fd7c406bfd0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fd7b8009840 tx=0x7fd7b8009b50 comp rx=0 tx=0).stop 2023-11-08T19:27:45.896 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.889+0000 7fd7caa25700 1 -- 172.21.15.143:0/1045696657 shutdown_connections 2023-11-08T19:27:45.896 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.889+0000 7fd7caa25700 1 --2- 172.21.15.143:0/1045696657 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd7c4143a30 0x7fd7c406bfd0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:45.896 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.889+0000 7fd7caa25700 1 -- 172.21.15.143:0/1045696657 >> 172.21.15.143:0/1045696657 conn(0x7fd7c40b1c70 msgr2=0x7fd7c40b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:45.896 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.889+0000 7fd7caa25700 1 -- 172.21.15.143:0/1045696657 shutdown_connections 2023-11-08T19:27:45.896 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.889+0000 7fd7caa25700 1 -- 172.21.15.143:0/1045696657 wait complete. 2023-11-08T19:27:45.899 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.891+0000 7fd7caa25700 1 Processor -- start 2023-11-08T19:27:45.899 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.891+0000 7fd7caa25700 1 -- start start 2023-11-08T19:27:45.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.891+0000 7fd7caa25700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd7c40c5f90 0x7fd7c40b57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:45.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.891+0000 7fd7caa25700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd7c40cc8c0 0x7fd7c41545d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:45.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.891+0000 7fd7caa25700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd7c4143a30 0x7fd7c40b0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:45.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.891+0000 7fd7caa25700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c40c7f90 con 0x7fd7c4143a30 2023-11-08T19:27:45.901 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.891+0000 7fd7caa25700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c40c9e00 con 0x7fd7c40cc8c0 2023-11-08T19:27:45.901 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.891+0000 7fd7caa25700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c415a070 con 0x7fd7c40c5f90 2023-11-08T19:27:45.901 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.892+0000 7fd7c8fc2700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd7c4143a30 0x7fd7c40b0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:45.901 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.892+0000 7fd7c8fc2700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd7c4143a30 0x7fd7c40b0410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54012/0 (socket says 172.21.15.143:54012) 2023-11-08T19:27:45.901 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.892+0000 7fd7c8fc2700 1 -- 172.21.15.143:0/2355090163 learned_addr learned my addr 172.21.15.143:0/2355090163 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:45.902 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.892+0000 7fd7c3fff700 1 --2- 172.21.15.143:0/2355090163 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd7c40c5f90 0x7fd7c40b57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:45.902 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.892+0000 7fd7c37fe700 1 --2- 172.21.15.143:0/2355090163 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd7c40cc8c0 0x7fd7c41545d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:45.902 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.892+0000 7fd7c8fc2700 1 -- 172.21.15.143:0/2355090163 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd7c40c5f90 msgr2=0x7fd7c40b57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:45.902 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.892+0000 7fd7c8fc2700 1 --2- 172.21.15.143:0/2355090163 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd7c40c5f90 0x7fd7c40b57c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:45.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.892+0000 7fd7c8fc2700 1 -- 172.21.15.143:0/2355090163 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd7c40cc8c0 msgr2=0x7fd7c41545d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:45.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.892+0000 7fd7c8fc2700 1 --2- 172.21.15.143:0/2355090163 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd7c40cc8c0 0x7fd7c41545d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:45.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.892+0000 7fd7c8fc2700 1 -- 172.21.15.143:0/2355090163 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7b80094a0 con 0x7fd7c4143a30 2023-11-08T19:27:45.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.892+0000 7fd7c8fc2700 1 --2- 172.21.15.143:0/2355090163 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd7c4143a30 0x7fd7c40b0410 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc00caa0 tx=0x7fd7bc00ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:45.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.892+0000 7fd7c0ff9700 1 -- 172.21.15.143:0/2355090163 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7bc00aa80 con 0x7fd7c4143a30 2023-11-08T19:27:45.904 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.892+0000 7fd7c0ff9700 1 -- 172.21.15.143:0/2355090163 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7bc00abe0 con 0x7fd7c4143a30 2023-11-08T19:27:45.904 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.892+0000 7fd7caa25700 1 -- 172.21.15.143:0/2355090163 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7c40c9510 con 0x7fd7c4143a30 2023-11-08T19:27:45.904 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.892+0000 7fd7c0ff9700 1 -- 172.21.15.143:0/2355090163 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7bc012400 con 0x7fd7c4143a30 2023-11-08T19:27:45.904 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.892+0000 7fd7caa25700 1 -- 172.21.15.143:0/2355090163 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7c40c9a90 con 0x7fd7c4143a30 2023-11-08T19:27:45.904 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.893+0000 7fd7caa25700 1 -- 172.21.15.143:0/2355090163 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7c40c9fb0 con 0x7fd7c4143a30 2023-11-08T19:27:45.905 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.894+0000 7fd7c0ff9700 1 -- 172.21.15.143:0/2355090163 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd7bc01b3b0 con 0x7fd7c4143a30 2023-11-08T19:27:45.905 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.894+0000 7fd7c0ff9700 1 --2- 172.21.15.143:0/2355090163 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd7b006af00 0x7fd7b006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:45.905 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.894+0000 7fd7c0ff9700 1 -- 172.21.15.143:0/2355090163 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6888+0+0 (secure 0 0 0) 0x7fd7bc0897c0 con 0x7fd7c4143a30 2023-11-08T19:27:45.905 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.894+0000 7fd7c3fff700 1 --2- 172.21.15.143:0/2355090163 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd7b006af00 0x7fd7b006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:45.905 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.895+0000 7fd7c3fff700 1 --2- 172.21.15.143:0/2355090163 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd7b006af00 0x7fd7b006d3c0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fd7c415a590 tx=0x7fd7b8008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:45.906 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:45.899+0000 7fd7c0ff9700 1 -- 172.21.15.143:0/2355090163 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd7c40c9fb0 con 0x7fd7c4143a30 2023-11-08T19:27:46.091 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.089+0000 7fd7caa25700 1 -- 172.21.15.143:0/2355090163 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"} v 0) v1 -- 0x7fd7c40ca190 con 0x7fd7c4143a30 2023-11-08T19:27:46.091 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.090+0000 7fd7c0ff9700 1 -- 172.21.15.143:0/2355090163 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"}]=0 v80) v1 ==== 103+0+11 (secure 0 0 0) 0x7fd7bc0543d0 con 0x7fd7c4143a30 2023-11-08T19:27:46.092 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2023-11-08T19:27:46.094 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.092+0000 7fd7caa25700 1 -- 172.21.15.143:0/2355090163 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd7b006af00 msgr2=0x7fd7b006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:46.094 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.092+0000 7fd7caa25700 1 --2- 172.21.15.143:0/2355090163 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd7b006af00 0x7fd7b006d3c0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fd7c415a590 tx=0x7fd7b8008030 comp rx=0 tx=0).stop 2023-11-08T19:27:46.094 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.092+0000 7fd7caa25700 1 -- 172.21.15.143:0/2355090163 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd7c4143a30 msgr2=0x7fd7c40b0410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:46.094 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.092+0000 7fd7caa25700 1 --2- 172.21.15.143:0/2355090163 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd7c4143a30 0x7fd7c40b0410 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc00caa0 tx=0x7fd7bc00ce60 comp rx=0 tx=0).stop 2023-11-08T19:27:46.094 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.092+0000 7fd7caa25700 1 -- 172.21.15.143:0/2355090163 shutdown_connections 2023-11-08T19:27:46.095 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.092+0000 7fd7caa25700 1 --2- 172.21.15.143:0/2355090163 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd7b006af00 0x7fd7b006d3c0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:46.095 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.092+0000 7fd7caa25700 1 --2- 172.21.15.143:0/2355090163 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd7c40c5f90 0x7fd7c40b57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:46.095 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.092+0000 7fd7caa25700 1 --2- 172.21.15.143:0/2355090163 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd7c4143a30 0x7fd7c40b0410 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:46.095 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.092+0000 7fd7caa25700 1 --2- 172.21.15.143:0/2355090163 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd7c40cc8c0 0x7fd7c41545d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:46.096 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.092+0000 7fd7caa25700 1 -- 172.21.15.143:0/2355090163 >> 172.21.15.143:0/2355090163 conn(0x7fd7c40b1c70 msgr2=0x7fd7c415e2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:46.096 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.092+0000 7fd7caa25700 1 -- 172.21.15.143:0/2355090163 shutdown_connections 2023-11-08T19:27:46.096 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.092+0000 7fd7caa25700 1 -- 172.21.15.143:0/2355090163 wait complete. 2023-11-08T19:27:46.105 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get test_data_pool pg_num 2023-11-08T19:27:46.228 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.227+0000 7fb3923ea700 1 Processor -- start 2023-11-08T19:27:46.229 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.227+0000 7fb3923ea700 1 -- start start 2023-11-08T19:27:46.231 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.228+0000 7fb3923ea700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb38c0af760 0x7fb38c0afbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:46.231 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.228+0000 7fb3923ea700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb38c0ce450 0x7fb38c0ce880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:46.231 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.228+0000 7fb3923ea700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb38c0b2dc0 con 0x7fb38c15a9e0 2023-11-08T19:27:46.232 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.228+0000 7fb3923ea700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb38c0b0130 con 0x7fb38c0af760 2023-11-08T19:27:46.232 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.228+0000 7fb3923ea700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb38c073020 con 0x7fb38c0ce450 2023-11-08T19:27:46.232 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.228+0000 7fb38b7fe700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb38c0af760 0x7fb38c0afbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:46.232 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.228+0000 7fb38bfff700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fb38c15a9e0 0x7fb38c0b2300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:50170/0 (socket says 172.21.15.143:50170) 2023-11-08T19:27:46.232 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.228+0000 7fb38bfff700 1 -- 172.21.15.143:0/2679136282 learned_addr learned my addr 172.21.15.143:0/2679136282 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:27:46.233 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.228+0000 7fb390987700 1 --2- 172.21.15.143:0/2679136282 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb38c0ce450 0x7fb38c0ce880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:46.233 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.228+0000 7fb38b7fe700 1 -- 172.21.15.143:0/2679136282 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb38c0ce450 msgr2=0x7fb38c0ce880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:46.233 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.228+0000 7fb38b7fe700 1 --2- 172.21.15.143:0/2679136282 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb38c0ce450 0x7fb38c0ce880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:46.233 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.228+0000 7fb38b7fe700 1 -- 172.21.15.143:0/2679136282 >> v1:172.21.15.143:6789/0 conn(0x7fb38c15a9e0 legacy=0x7fb38c0b2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:46.233 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.228+0000 7fb38affd700 1 -- 172.21.15.143:0/2679136282 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3342124181 0 0) 0x7fb38c0b2dc0 con 0x7fb38c15a9e0 2023-11-08T19:27:46.234 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.228+0000 7fb38b7fe700 1 -- 172.21.15.143:0/2679136282 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb38c072260 con 0x7fb38c0af760 2023-11-08T19:27:46.234 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.228+0000 7fb390987700 1 --2- 172.21.15.143:0/2679136282 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb38c0ce450 0x7fb38c0ce880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:27:46.234 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.228+0000 7fb38b7fe700 1 --2- 172.21.15.143:0/2679136282 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb38c0af760 0x7fb38c0afbf0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fb37c009840 tx=0x7fb37c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=142e1e5f6edffc75 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:46.234 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.228+0000 7fb38affd700 1 -- 172.21.15.143:0/2679136282 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb37c011830 con 0x7fb38c0af760 2023-11-08T19:27:46.234 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.229+0000 7fb38affd700 1 -- 172.21.15.143:0/2679136282 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb37c011990 con 0x7fb38c0af760 2023-11-08T19:27:46.235 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.229+0000 7fb38affd700 1 -- 172.21.15.143:0/2679136282 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb37c011d20 con 0x7fb38c0af760 2023-11-08T19:27:46.235 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.229+0000 7fb3923ea700 1 -- 172.21.15.143:0/2679136282 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb38c0af760 msgr2=0x7fb38c0afbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:46.235 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.229+0000 7fb3923ea700 1 --2- 172.21.15.143:0/2679136282 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb38c0af760 0x7fb38c0afbf0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fb37c009840 tx=0x7fb37c009b50 comp rx=0 tx=0).stop 2023-11-08T19:27:46.235 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.229+0000 7fb3923ea700 1 -- 172.21.15.143:0/2679136282 shutdown_connections 2023-11-08T19:27:46.236 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.229+0000 7fb3923ea700 1 --2- 172.21.15.143:0/2679136282 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb38c0ce450 0x7fb38c0ce880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:46.236 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.229+0000 7fb3923ea700 1 --2- 172.21.15.143:0/2679136282 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb38c0af760 0x7fb38c0afbf0 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:46.236 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.229+0000 7fb3923ea700 1 -- 172.21.15.143:0/2679136282 >> 172.21.15.143:0/2679136282 conn(0x7fb38c0cd300 msgr2=0x7fb38c0cd710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:46.236 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.229+0000 7fb3923ea700 1 -- 172.21.15.143:0/2679136282 shutdown_connections 2023-11-08T19:27:46.236 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.229+0000 7fb3923ea700 1 -- 172.21.15.143:0/2679136282 wait complete. 2023-11-08T19:27:46.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.230+0000 7fb3923ea700 1 Processor -- start 2023-11-08T19:27:46.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.230+0000 7fb3923ea700 1 -- start start 2023-11-08T19:27:46.240 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.231+0000 7fb3923ea700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb38c0af760 0x7fb38c0c4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:46.240 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.231+0000 7fb3923ea700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb38c0ce450 0x7fb38c15adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:46.240 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.231+0000 7fb3923ea700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb38c15a9e0 0x7fb38c15b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:46.240 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.231+0000 7fb3923ea700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb38c0b4190 con 0x7fb38c15a9e0 2023-11-08T19:27:46.240 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.231+0000 7fb3923ea700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb38c0c53a0 con 0x7fb38c0ce450 2023-11-08T19:27:46.241 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.231+0000 7fb3923ea700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb38c0c5680 con 0x7fb38c0af760 2023-11-08T19:27:46.241 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.231+0000 7fb390987700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb38c15a9e0 0x7fb38c15b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:46.241 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.231+0000 7fb390987700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb38c15a9e0 0x7fb38c15b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54026/0 (socket says 172.21.15.143:54026) 2023-11-08T19:27:46.241 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.231+0000 7fb390987700 1 -- 172.21.15.143:0/2643409706 learned_addr learned my addr 172.21.15.143:0/2643409706 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:46.241 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.231+0000 7fb38b7fe700 1 --2- 172.21.15.143:0/2643409706 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb38c0ce450 0x7fb38c15adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:46.242 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.231+0000 7fb38bfff700 1 --2- 172.21.15.143:0/2643409706 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb38c0af760 0x7fb38c0c4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:46.242 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.232+0000 7fb390987700 1 -- 172.21.15.143:0/2643409706 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb38c0af760 msgr2=0x7fb38c0c4dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:46.242 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.232+0000 7fb390987700 1 --2- 172.21.15.143:0/2643409706 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb38c0af760 0x7fb38c0c4dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:46.242 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.232+0000 7fb390987700 1 -- 172.21.15.143:0/2643409706 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb38c0ce450 msgr2=0x7fb38c15adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:46.242 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.232+0000 7fb390987700 1 --2- 172.21.15.143:0/2643409706 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb38c0ce450 0x7fb38c15adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:46.243 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.232+0000 7fb390987700 1 -- 172.21.15.143:0/2643409706 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb37c0094a0 con 0x7fb38c15a9e0 2023-11-08T19:27:46.243 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.232+0000 7fb390987700 1 --2- 172.21.15.143:0/2643409706 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb38c15a9e0 0x7fb38c15b300 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fb38400cef0 tx=0x7fb38400cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:46.243 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.232+0000 7fb388ff9700 1 -- 172.21.15.143:0/2643409706 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb38400ad00 con 0x7fb38c15a9e0 2023-11-08T19:27:46.243 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.232+0000 7fb388ff9700 1 -- 172.21.15.143:0/2643409706 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb38400ae60 con 0x7fb38c15a9e0 2023-11-08T19:27:46.243 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.232+0000 7fb3923ea700 1 -- 172.21.15.143:0/2643409706 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb38c0c5960 con 0x7fb38c15a9e0 2023-11-08T19:27:46.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.232+0000 7fb388ff9700 1 -- 172.21.15.143:0/2643409706 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb384015680 con 0x7fb38c15a9e0 2023-11-08T19:27:46.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.232+0000 7fb3923ea700 1 -- 172.21.15.143:0/2643409706 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb38c15bb10 con 0x7fb38c15a9e0 2023-11-08T19:27:46.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.233+0000 7fb3923ea700 1 -- 172.21.15.143:0/2643409706 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb38c04eee0 con 0x7fb38c15a9e0 2023-11-08T19:27:46.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.233+0000 7fb388ff9700 1 -- 172.21.15.143:0/2643409706 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb38400d430 con 0x7fb38c15a9e0 2023-11-08T19:27:46.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.234+0000 7fb388ff9700 1 --2- 172.21.15.143:0/2643409706 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fb37806af00 0x7fb37806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:46.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.234+0000 7fb388ff9700 1 -- 172.21.15.143:0/2643409706 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6888+0+0 (secure 0 0 0) 0x7fb384089f70 con 0x7fb38c15a9e0 2023-11-08T19:27:46.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.234+0000 7fb38bfff700 1 --2- 172.21.15.143:0/2643409706 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fb37806af00 0x7fb37806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:46.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.234+0000 7fb38bfff700 1 --2- 172.21.15.143:0/2643409706 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fb37806af00 0x7fb37806d3c0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fb380007fd0 tx=0x7fb380007de0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:46.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.238+0000 7fb388ff9700 1 -- 172.21.15.143:0/2643409706 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb3840bcca0 con 0x7fb38c15a9e0 2023-11-08T19:27:46.430 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.428+0000 7fb3923ea700 1 -- 172.21.15.143:0/2643409706 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "test_data_pool", "var": "pg_num"} v 0) v1 -- 0x7fb38c0d11f0 con 0x7fb38c15a9e0 2023-11-08T19:27:46.430 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.429+0000 7fb388ff9700 1 -- 172.21.15.143:0/2643409706 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "test_data_pool", "var": "pg_num"}]=0 v81) v1 ==== 103+0+11 (secure 0 0 0) 0x7fb384054b90 con 0x7fb38c15a9e0 2023-11-08T19:27:46.431 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2023-11-08T19:27:46.433 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.432+0000 7fb3923ea700 1 -- 172.21.15.143:0/2643409706 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fb37806af00 msgr2=0x7fb37806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:46.434 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.432+0000 7fb3923ea700 1 --2- 172.21.15.143:0/2643409706 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fb37806af00 0x7fb37806d3c0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fb380007fd0 tx=0x7fb380007de0 comp rx=0 tx=0).stop 2023-11-08T19:27:46.434 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.432+0000 7fb3923ea700 1 -- 172.21.15.143:0/2643409706 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb38c15a9e0 msgr2=0x7fb38c15b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:46.434 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.432+0000 7fb3923ea700 1 --2- 172.21.15.143:0/2643409706 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb38c15a9e0 0x7fb38c15b300 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fb38400cef0 tx=0x7fb38400cf20 comp rx=0 tx=0).stop 2023-11-08T19:27:46.434 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.432+0000 7fb3923ea700 1 -- 172.21.15.143:0/2643409706 shutdown_connections 2023-11-08T19:27:46.434 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.432+0000 7fb3923ea700 1 --2- 172.21.15.143:0/2643409706 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fb37806af00 0x7fb37806d3c0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:46.435 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.432+0000 7fb3923ea700 1 --2- 172.21.15.143:0/2643409706 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb38c0af760 0x7fb38c0c4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:46.435 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.432+0000 7fb3923ea700 1 --2- 172.21.15.143:0/2643409706 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb38c15a9e0 0x7fb38c15b300 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:46.435 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.432+0000 7fb3923ea700 1 --2- 172.21.15.143:0/2643409706 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb38c0ce450 0x7fb38c15adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:46.435 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.432+0000 7fb3923ea700 1 -- 172.21.15.143:0/2643409706 >> 172.21.15.143:0/2643409706 conn(0x7fb38c0cd300 msgr2=0x7fb38c159c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:46.436 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.432+0000 7fb3923ea700 1 -- 172.21.15.143:0/2643409706 shutdown_connections 2023-11-08T19:27:46.436 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.433+0000 7fb3923ea700 1 -- 172.21.15.143:0/2643409706 wait complete. 2023-11-08T19:27:46.446 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get test_metadata_pool pg_num 2023-11-08T19:27:46.577 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.575+0000 7f91bde21700 1 Processor -- start 2023-11-08T19:27:46.577 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.575+0000 7f91bde21700 1 -- start start 2023-11-08T19:27:46.580 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.576+0000 7f91bde21700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f91b815bb20 0x7f91b80b2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:46.580 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.576+0000 7f91bde21700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f91b80b1730 0x7f91b80d1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:46.580 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.576+0000 7f91bde21700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91b80d0420 con 0x7f91b80d1bb0 2023-11-08T19:27:46.580 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.576+0000 7f91bde21700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91b80cf600 con 0x7f91b80b1730 2023-11-08T19:27:46.580 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.576+0000 7f91bde21700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91b80cf810 con 0x7f91b815bb20 2023-11-08T19:27:46.581 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.576+0000 7f91b6ffd700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f91b80b1730 0x7f91b80d1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:46.581 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.576+0000 7f91b6ffd700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f91b80b1730 0x7f91b80d1670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54030/0 (socket says 172.21.15.143:54030) 2023-11-08T19:27:46.581 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.576+0000 7f91b7fff700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f91b80d1bb0 0x7f91b80ce420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:50188/0 (socket says 172.21.15.143:50188) 2023-11-08T19:27:46.581 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.576+0000 7f91b6ffd700 1 -- 172.21.15.143:0/3193984775 learned_addr learned my addr 172.21.15.143:0/3193984775 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:46.581 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.577+0000 7f91b77fe700 1 --2- 172.21.15.143:0/3193984775 >> v2:172.21.15.174:3300/0 conn(0x7f91b815bb20 0x7f91b80b2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:46.582 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.577+0000 7f91b6ffd700 1 -- 172.21.15.143:0/3193984775 >> v2:172.21.15.174:3300/0 conn(0x7f91b815bb20 msgr2=0x7f91b80b2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:46.582 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.577+0000 7f91b6ffd700 1 --2- 172.21.15.143:0/3193984775 >> v2:172.21.15.174:3300/0 conn(0x7f91b815bb20 0x7f91b80b2670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:46.582 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.577+0000 7f91b6ffd700 1 -- 172.21.15.143:0/3193984775 >> v1:172.21.15.143:6789/0 conn(0x7f91b80d1bb0 legacy=0x7f91b80ce420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:46.582 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.577+0000 7f91b6ffd700 1 -- 172.21.15.143:0/3193984775 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91b80cfaf0 con 0x7f91b80b1730 2023-11-08T19:27:46.583 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.577+0000 7f91b67fc700 1 -- 172.21.15.143:0/3193984775 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2847386667 0 0) 0x7f91b80d0420 con 0x7f91b80d1bb0 2023-11-08T19:27:46.583 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.577+0000 7f91b6ffd700 1 --2- 172.21.15.143:0/3193984775 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f91b80b1730 0x7f91b80d1670 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f91b808b850 tx=0x7f91a8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2de524fe862b9f71 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:46.583 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.577+0000 7f91b67fc700 1 -- 172.21.15.143:0/3193984775 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91a800b870 con 0x7f91b80b1730 2023-11-08T19:27:46.583 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.577+0000 7f91b67fc700 1 -- 172.21.15.143:0/3193984775 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91a800b9d0 con 0x7f91b80b1730 2023-11-08T19:27:46.583 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.577+0000 7f91b67fc700 1 -- 172.21.15.143:0/3193984775 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91a800bd60 con 0x7f91b80b1730 2023-11-08T19:27:46.583 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.578+0000 7f91bde21700 1 -- 172.21.15.143:0/3193984775 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f91b80b1730 msgr2=0x7f91b80d1670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:46.584 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.578+0000 7f91bde21700 1 --2- 172.21.15.143:0/3193984775 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f91b80b1730 0x7f91b80d1670 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f91b808b850 tx=0x7f91a8009bd0 comp rx=0 tx=0).stop 2023-11-08T19:27:46.584 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.578+0000 7f91bde21700 1 -- 172.21.15.143:0/3193984775 shutdown_connections 2023-11-08T19:27:46.584 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.578+0000 7f91bde21700 1 --2- 172.21.15.143:0/3193984775 >> v2:172.21.15.174:3300/0 conn(0x7f91b815bb20 0x7f91b80b2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:46.584 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.578+0000 7f91bde21700 1 --2- 172.21.15.143:0/3193984775 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f91b80b1730 0x7f91b80d1670 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:46.584 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.578+0000 7f91bde21700 1 -- 172.21.15.143:0/3193984775 >> 172.21.15.143:0/3193984775 conn(0x7f91b808b000 msgr2=0x7f91b809ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:46.585 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.578+0000 7f91bde21700 1 -- 172.21.15.143:0/3193984775 shutdown_connections 2023-11-08T19:27:46.585 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.578+0000 7f91bde21700 1 -- 172.21.15.143:0/3193984775 wait complete. 2023-11-08T19:27:46.590 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.579+0000 7f91bde21700 1 Processor -- start 2023-11-08T19:27:46.590 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.579+0000 7f91bde21700 1 -- start start 2023-11-08T19:27:46.590 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.580+0000 7f91bde21700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f91b80b1730 0x7f91b80b56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:46.591 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.580+0000 7f91bde21700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f91b80d1bb0 0x7f91b80cb050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:46.591 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.580+0000 7f91bde21700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f91b815bb20 0x7f91b8154610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:46.591 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.580+0000 7f91bde21700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91b8154b50 con 0x7f91b80b1730 2023-11-08T19:27:46.591 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.580+0000 7f91bde21700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91b80b5c80 con 0x7f91b815bb20 2023-11-08T19:27:46.591 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.580+0000 7f91bde21700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91b80b5f60 con 0x7f91b80d1bb0 2023-11-08T19:27:46.592 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.580+0000 7f91b77fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f91b80b1730 0x7f91b80b56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:46.592 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.580+0000 7f91b77fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f91b80b1730 0x7f91b80b56a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54036/0 (socket says 172.21.15.143:54036) 2023-11-08T19:27:46.592 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.580+0000 7f91b77fe700 1 -- 172.21.15.143:0/1526148894 learned_addr learned my addr 172.21.15.143:0/1526148894 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:46.592 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.581+0000 7f91b7fff700 1 --2- 172.21.15.143:0/1526148894 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f91b815bb20 0x7f91b8154610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:46.592 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.581+0000 7f91b6ffd700 1 --2- 172.21.15.143:0/1526148894 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f91b80d1bb0 0x7f91b80cb050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:46.593 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.581+0000 7f91b77fe700 1 -- 172.21.15.143:0/1526148894 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f91b80d1bb0 msgr2=0x7f91b80cb050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:46.593 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.581+0000 7f91b77fe700 1 --2- 172.21.15.143:0/1526148894 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f91b80d1bb0 0x7f91b80cb050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:46.593 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.581+0000 7f91b77fe700 1 -- 172.21.15.143:0/1526148894 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f91b815bb20 msgr2=0x7f91b8154610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:46.593 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.581+0000 7f91b77fe700 1 --2- 172.21.15.143:0/1526148894 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f91b815bb20 0x7f91b8154610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:46.593 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.581+0000 7f91b77fe700 1 -- 172.21.15.143:0/1526148894 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91a8009550 con 0x7f91b80b1730 2023-11-08T19:27:46.594 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.581+0000 7f91b6ffd700 1 --2- 172.21.15.143:0/1526148894 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f91b80d1bb0 0x7f91b80cb050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:27:46.594 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.581+0000 7f91b77fe700 1 --2- 172.21.15.143:0/1526148894 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f91b80b1730 0x7f91b80b56a0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f91a000d800 tx=0x7f91a000dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:46.594 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.581+0000 7f91affff700 1 -- 172.21.15.143:0/1526148894 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91a00125f0 con 0x7f91b80b1730 2023-11-08T19:27:46.594 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.581+0000 7f91affff700 1 -- 172.21.15.143:0/1526148894 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91a0012750 con 0x7f91b80b1730 2023-11-08T19:27:46.594 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.581+0000 7f91bde21700 1 -- 172.21.15.143:0/1526148894 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91b80b6240 con 0x7f91b80b1730 2023-11-08T19:27:46.595 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.581+0000 7f91affff700 1 -- 172.21.15.143:0/1526148894 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91a0012b00 con 0x7f91b80b1730 2023-11-08T19:27:46.595 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.581+0000 7f91bde21700 1 -- 172.21.15.143:0/1526148894 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91b80b6800 con 0x7f91b80b1730 2023-11-08T19:27:46.595 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.582+0000 7f91bde21700 1 -- 172.21.15.143:0/1526148894 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91b80b6990 con 0x7f91b80b1730 2023-11-08T19:27:46.595 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.583+0000 7f91affff700 1 -- 172.21.15.143:0/1526148894 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f91a0012c60 con 0x7f91b80b1730 2023-11-08T19:27:46.595 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.583+0000 7f91affff700 1 --2- 172.21.15.143:0/1526148894 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f919806af00 0x7f919806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:46.596 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.583+0000 7f91affff700 1 -- 172.21.15.143:0/1526148894 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6944+0+0 (secure 0 0 0) 0x7f91a0089e70 con 0x7f91b80b1730 2023-11-08T19:27:46.596 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.583+0000 7f91b6ffd700 1 --2- 172.21.15.143:0/1526148894 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f919806af00 0x7f919806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:46.596 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.584+0000 7f91b6ffd700 1 --2- 172.21.15.143:0/1526148894 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f919806af00 0x7f919806d3c0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f91b80c4d00 tx=0x7f91a8008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:46.596 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.587+0000 7f91affff700 1 -- 172.21.15.143:0/1526148894 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f91a0054a50 con 0x7f91b80b1730 2023-11-08T19:27:46.775 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.774+0000 7f91bde21700 1 -- 172.21.15.143:0/1526148894 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "test_metadata_pool", "var": "pg_num"} v 0) v1 -- 0x7f91b804eee0 con 0x7f91b80b1730 2023-11-08T19:27:46.776 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.774+0000 7f91affff700 1 -- 172.21.15.143:0/1526148894 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "test_metadata_pool", "var": "pg_num"}]=0 v81) v1 ==== 107+0+11 (secure 0 0 0) 0x7f91a0058070 con 0x7f91b80b1730 2023-11-08T19:27:46.776 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2023-11-08T19:27:46.778 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.776+0000 7f91bde21700 1 -- 172.21.15.143:0/1526148894 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f919806af00 msgr2=0x7f919806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:46.778 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.776+0000 7f91bde21700 1 --2- 172.21.15.143:0/1526148894 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f919806af00 0x7f919806d3c0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f91b80c4d00 tx=0x7f91a8008030 comp rx=0 tx=0).stop 2023-11-08T19:27:46.778 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.776+0000 7f91bde21700 1 -- 172.21.15.143:0/1526148894 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f91b80b1730 msgr2=0x7f91b80b56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:46.778 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.776+0000 7f91bde21700 1 --2- 172.21.15.143:0/1526148894 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f91b80b1730 0x7f91b80b56a0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f91a000d800 tx=0x7f91a000dbc0 comp rx=0 tx=0).stop 2023-11-08T19:27:46.779 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.776+0000 7f91bde21700 1 -- 172.21.15.143:0/1526148894 shutdown_connections 2023-11-08T19:27:46.779 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.776+0000 7f91bde21700 1 --2- 172.21.15.143:0/1526148894 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f919806af00 0x7f919806d3c0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:46.779 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.776+0000 7f91bde21700 1 --2- 172.21.15.143:0/1526148894 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f91b815bb20 0x7f91b8154610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:46.779 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.776+0000 7f91bde21700 1 --2- 172.21.15.143:0/1526148894 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f91b80b1730 0x7f91b80b56a0 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:46.779 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.776+0000 7f91bde21700 1 --2- 172.21.15.143:0/1526148894 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f91b80d1bb0 0x7f91b80cb050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:46.780 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.776+0000 7f91bde21700 1 -- 172.21.15.143:0/1526148894 >> 172.21.15.143:0/1526148894 conn(0x7f91b808b000 msgr2=0x7f91b806fdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:46.780 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.776+0000 7f91bde21700 1 -- 172.21.15.143:0/1526148894 shutdown_connections 2023-11-08T19:27:46.780 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.777+0000 7f91bde21700 1 -- 172.21.15.143:0/1526148894 wait complete. 2023-11-08T19:27:46.790 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-08T19:27:46.913 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.911+0000 7f0fad7c1700 1 Processor -- start 2023-11-08T19:27:46.913 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.912+0000 7f0fad7c1700 1 -- start start 2023-11-08T19:27:46.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.912+0000 7f0fad7c1700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f0fa80b61e0 0x7f0fa80d01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:46.923 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.912+0000 7f0fad7c1700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fa80cc2a0 con 0x7f0fa806bd20 2023-11-08T19:27:46.923 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.912+0000 7f0fad7c1700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fa806c1b0 con 0x7f0fa815a950 2023-11-08T19:27:46.923 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.912+0000 7f0fad7c1700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fa80cc130 con 0x7f0fa80b61e0 2023-11-08T19:27:46.924 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.912+0000 7f0fa67fc700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f0fa806bd20 0x7f0fa80cc730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:50198/0 (socket says 172.21.15.143:50198) 2023-11-08T19:27:46.924 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.912+0000 7f0fa67fc700 1 -- 172.21.15.143:0/3881725622 learned_addr learned my addr 172.21.15.143:0/3881725622 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:27:46.925 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.913+0000 7f0fa5ffb700 1 -- 172.21.15.143:0/3881725622 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 107001593 0 0) 0x7f0fa80cc2a0 con 0x7f0fa806bd20 2023-11-08T19:27:46.925 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.913+0000 7f0fa77fe700 1 --2- 172.21.15.143:0/3881725622 >> v2:172.21.15.174:3300/0 conn(0x7f0fa80b61e0 0x7f0fa80d01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:46.925 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.913+0000 7f0fa5ffb700 1 -- 172.21.15.143:0/3881725622 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0f900035b0 con 0x7f0fa806bd20 2023-11-08T19:27:46.925 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.913+0000 7f0fa5ffb700 1 -- 172.21.15.143:0/3881725622 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1664834519 0 0) 0x7f0f900035b0 con 0x7f0fa806bd20 2023-11-08T19:27:46.926 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.913+0000 7f0fa5ffb700 1 -- 172.21.15.143:0/3881725622 >> v2:172.21.15.174:3300/0 conn(0x7f0fa80b61e0 msgr2=0x7f0fa80d01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:46.926 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.913+0000 7f0fa5ffb700 1 --2- 172.21.15.143:0/3881725622 >> v2:172.21.15.174:3300/0 conn(0x7f0fa80b61e0 0x7f0fa80d01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:46.926 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.913+0000 7f0fa5ffb700 1 -- 172.21.15.143:0/3881725622 >> v1:172.21.15.174:6789/0 conn(0x7f0fa815a950 legacy=0x7f0fa8072370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:46.926 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.913+0000 7f0fa5ffb700 1 -- 172.21.15.143:0/3881725622 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fa806c490 con 0x7f0fa806bd20 2023-11-08T19:27:46.927 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.913+0000 7f0fa5ffb700 1 -- 172.21.15.143:0/3881725622 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4147756303 0 0) 0x7f0f98001670 con 0x7f0fa806bd20 2023-11-08T19:27:46.927 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.913+0000 7f0fa5ffb700 1 -- 172.21.15.143:0/3881725622 >> v1:172.21.15.143:6789/0 conn(0x7f0fa806bd20 legacy=0x7f0fa80cc730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:46.927 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.913+0000 7f0fa5ffb700 1 --2- 172.21.15.143:0/3881725622 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0f90003a00 0x7f0f90003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:46.927 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.913+0000 7f0fa5ffb700 1 --2- 172.21.15.143:0/3881725622 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0f90004850 0x7f0f90004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:46.927 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.913+0000 7f0fa5ffb700 1 --2- 172.21.15.143:0/3881725622 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0f90005210 0x7f0f9000dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:46.928 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.913+0000 7f0fa5ffb700 1 -- 172.21.15.143:0/3881725622 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f9000e290 con 0x7f0f90003a00 2023-11-08T19:27:46.928 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.913+0000 7f0fa5ffb700 1 -- 172.21.15.143:0/3881725622 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f9000e400 con 0x7f0f90004850 2023-11-08T19:27:46.928 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.913+0000 7f0fa5ffb700 1 -- 172.21.15.143:0/3881725622 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f9000e5c0 con 0x7f0f90005210 2023-11-08T19:27:46.928 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.913+0000 7f0fa6ffd700 1 --2- 172.21.15.143:0/3881725622 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0f90003a00 0x7f0f90003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:46.928 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.913+0000 7f0fa67fc700 1 --2- 172.21.15.143:0/3881725622 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0f90004850 0x7f0f90004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:46.929 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.913+0000 7f0fa77fe700 1 --2- 172.21.15.143:0/3881725622 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0f90005210 0x7f0f9000dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:46.929 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.913+0000 7f0fa6ffd700 1 -- 172.21.15.143:0/3881725622 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0f90005210 msgr2=0x7f0f9000dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:46.929 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.913+0000 7f0fa6ffd700 1 --2- 172.21.15.143:0/3881725622 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0f90005210 0x7f0f9000dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:46.929 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.913+0000 7f0fa6ffd700 1 -- 172.21.15.143:0/3881725622 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0f90004850 msgr2=0x7f0f90004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:46.929 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.913+0000 7f0fa6ffd700 1 --2- 172.21.15.143:0/3881725622 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0f90004850 0x7f0f90004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:46.930 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.913+0000 7f0fa77fe700 1 -- 172.21.15.143:0/3881725622 reap_dead start 2023-11-08T19:27:46.930 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.913+0000 7f0fa6ffd700 1 -- 172.21.15.143:0/3881725622 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f900035b0 con 0x7f0f90003a00 2023-11-08T19:27:46.930 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.914+0000 7f0fa6ffd700 1 --2- 172.21.15.143:0/3881725622 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0f90003a00 0x7f0f90003e20 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c0017c0 tx=0x7f0f90005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a318e176e0975d0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:46.930 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.914+0000 7f0fa5ffb700 1 -- 172.21.15.143:0/3881725622 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f9c0052d0 con 0x7f0f90003a00 2023-11-08T19:27:46.930 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.914+0000 7f0fa5ffb700 1 -- 172.21.15.143:0/3881725622 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f9c005430 con 0x7f0f90003a00 2023-11-08T19:27:46.931 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.914+0000 7f0fa5ffb700 1 -- 172.21.15.143:0/3881725622 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f9c0038c0 con 0x7f0f90003a00 2023-11-08T19:27:46.931 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.914+0000 7f0fad7c1700 1 -- 172.21.15.143:0/3881725622 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0f90003a00 msgr2=0x7f0f90003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:46.931 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.914+0000 7f0fad7c1700 1 --2- 172.21.15.143:0/3881725622 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0f90003a00 0x7f0f90003e20 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c0017c0 tx=0x7f0f90005880 comp rx=0 tx=0).stop 2023-11-08T19:27:46.931 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.914+0000 7f0fad7c1700 1 -- 172.21.15.143:0/3881725622 shutdown_connections 2023-11-08T19:27:46.931 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.914+0000 7f0fad7c1700 1 --2- 172.21.15.143:0/3881725622 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0f90003a00 0x7f0f90003e20 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:46.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.914+0000 7f0fad7c1700 1 -- 172.21.15.143:0/3881725622 >> 172.21.15.143:0/3881725622 conn(0x7f0fa80cd2e0 msgr2=0x7f0fa80cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:46.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.914+0000 7f0fad7c1700 1 -- 172.21.15.143:0/3881725622 shutdown_connections 2023-11-08T19:27:46.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.914+0000 7f0fad7c1700 1 -- 172.21.15.143:0/3881725622 wait complete. 2023-11-08T19:27:46.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.916+0000 7f0fad7c1700 1 Processor -- start 2023-11-08T19:27:46.933 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.916+0000 7f0fad7c1700 1 -- start start 2023-11-08T19:27:46.933 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.916+0000 7f0fad7c1700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0f90003a00 0x7f0fa80c4400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:46.933 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.916+0000 7f0fad7c1700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0fa80c4940 0x7f0fa806af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:46.933 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.916+0000 7f0fad7c1700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0fa8144830 0x7f0fa806b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:46.933 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.916+0000 7f0fad7c1700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fa80c5fc0 con 0x7f0fa80c4940 2023-11-08T19:27:46.934 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.916+0000 7f0fad7c1700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fa80c61f0 con 0x7f0f90003a00 2023-11-08T19:27:46.934 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.916+0000 7f0fad7c1700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fa80c64d0 con 0x7f0fa8144830 2023-11-08T19:27:46.934 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.916+0000 7f0fa67fc700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0fa80c4940 0x7f0fa806af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:46.934 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.917+0000 7f0fa67fc700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0fa80c4940 0x7f0fa806af70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54058/0 (socket says 172.21.15.143:54058) 2023-11-08T19:27:46.935 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.917+0000 7f0fa67fc700 1 -- 172.21.15.143:0/2136292055 learned_addr learned my addr 172.21.15.143:0/2136292055 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:46.935 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.917+0000 7f0fa6ffd700 1 --2- 172.21.15.143:0/2136292055 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0f90003a00 0x7f0fa80c4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:46.935 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.917+0000 7f0fa77fe700 1 --2- 172.21.15.143:0/2136292055 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0fa8144830 0x7f0fa806b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:46.935 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.917+0000 7f0fa67fc700 1 -- 172.21.15.143:0/2136292055 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0fa8144830 msgr2=0x7f0fa806b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:46.936 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.917+0000 7f0fa67fc700 1 --2- 172.21.15.143:0/2136292055 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0fa8144830 0x7f0fa806b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:46.936 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.917+0000 7f0fa67fc700 1 -- 172.21.15.143:0/2136292055 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0f90003a00 msgr2=0x7f0fa80c4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:46.936 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.917+0000 7f0fa67fc700 1 --2- 172.21.15.143:0/2136292055 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0f90003a00 0x7f0fa80c4400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:46.936 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.917+0000 7f0fa67fc700 1 -- 172.21.15.143:0/2136292055 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fa80c67b0 con 0x7f0fa80c4940 2023-11-08T19:27:46.936 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.917+0000 7f0fa6ffd700 1 --2- 172.21.15.143:0/2136292055 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0f90003a00 0x7f0fa80c4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:27:46.937 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.917+0000 7f0fa67fc700 1 --2- 172.21.15.143:0/2136292055 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0fa80c4940 0x7f0fa806af70 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f0f980072c0 tx=0x7f0f98007680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:46.937 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.917+0000 7f0f8f7fe700 1 -- 172.21.15.143:0/2136292055 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f980077a0 con 0x7f0fa80c4940 2023-11-08T19:27:46.937 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.917+0000 7f0f8f7fe700 1 -- 172.21.15.143:0/2136292055 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f9800f8d0 con 0x7f0fa80c4940 2023-11-08T19:27:46.937 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.917+0000 7f0fad7c1700 1 -- 172.21.15.143:0/2136292055 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fa80c6a90 con 0x7f0fa80c4940 2023-11-08T19:27:46.938 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.917+0000 7f0f8f7fe700 1 -- 172.21.15.143:0/2136292055 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f9800fc80 con 0x7f0fa80c4940 2023-11-08T19:27:46.938 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.917+0000 7f0fad7c1700 1 -- 172.21.15.143:0/2136292055 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fa8154d80 con 0x7f0fa80c4940 2023-11-08T19:27:46.938 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.918+0000 7f0fad7c1700 1 -- 172.21.15.143:0/2136292055 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fa80c6fb0 con 0x7f0fa80c4940 2023-11-08T19:27:46.938 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.919+0000 7f0f8f7fe700 1 -- 172.21.15.143:0/2136292055 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0f9801d460 con 0x7f0fa80c4940 2023-11-08T19:27:46.939 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.919+0000 7f0f8f7fe700 1 --2- 172.21.15.143:0/2136292055 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f0f9406af00 0x7f0f9406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:46.939 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.919+0000 7f0f8f7fe700 1 -- 172.21.15.143:0/2136292055 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6944+0+0 (secure 0 0 0) 0x7f0f98089a90 con 0x7f0fa80c4940 2023-11-08T19:27:46.939 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.919+0000 7f0fa6ffd700 1 --2- 172.21.15.143:0/2136292055 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f0f9406af00 0x7f0f9406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:46.939 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.919+0000 7f0fa6ffd700 1 --2- 172.21.15.143:0/2136292055 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f0f9406af00 0x7f0f9406d3c0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c005040 tx=0x7f0f9c009030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:46.940 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:46.923+0000 7f0f8f7fe700 1 -- 172.21.15.143:0/2136292055 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0fa80c6fb0 con 0x7f0fa80c4940 2023-11-08T19:27:47.114 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.113+0000 7f0fad7c1700 1 -- 172.21.15.143:0/2136292055 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0fa80c6fb0 con 0x7f0fa80c4940 2023-11-08T19:27:47.114 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.113+0000 7f0f8f7fe700 1 -- 172.21.15.143:0/2136292055 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v81) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0fa80c6fb0 con 0x7f0fa80c4940 2023-11-08T19:27:47.115 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2023-11-08T19:27:47.117 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.115+0000 7f0fad7c1700 1 -- 172.21.15.143:0/2136292055 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f0f9406af00 msgr2=0x7f0f9406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:47.117 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.115+0000 7f0fad7c1700 1 --2- 172.21.15.143:0/2136292055 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f0f9406af00 0x7f0f9406d3c0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c005040 tx=0x7f0f9c009030 comp rx=0 tx=0).stop 2023-11-08T19:27:47.117 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.115+0000 7f0fad7c1700 1 -- 172.21.15.143:0/2136292055 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0fa80c4940 msgr2=0x7f0fa806af70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:47.117 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.115+0000 7f0fad7c1700 1 --2- 172.21.15.143:0/2136292055 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0fa80c4940 0x7f0fa806af70 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f0f980072c0 tx=0x7f0f98007680 comp rx=0 tx=0).stop 2023-11-08T19:27:47.117 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.115+0000 7f0fad7c1700 1 -- 172.21.15.143:0/2136292055 shutdown_connections 2023-11-08T19:27:47.118 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.115+0000 7f0fad7c1700 1 --2- 172.21.15.143:0/2136292055 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f0f9406af00 0x7f0f9406d3c0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:47.118 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.115+0000 7f0fad7c1700 1 --2- 172.21.15.143:0/2136292055 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0fa8144830 0x7f0fa806b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:47.118 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.115+0000 7f0fad7c1700 1 --2- 172.21.15.143:0/2136292055 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0f90003a00 0x7f0fa80c4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:47.118 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.115+0000 7f0fad7c1700 1 --2- 172.21.15.143:0/2136292055 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0fa80c4940 0x7f0fa806af70 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:47.119 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.115+0000 7f0fad7c1700 1 -- 172.21.15.143:0/2136292055 >> 172.21.15.143:0/2136292055 conn(0x7f0fa80cd2e0 msgr2=0x7f0fa80af760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:47.119 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.115+0000 7f0fad7c1700 1 -- 172.21.15.143:0/2136292055 shutdown_connections 2023-11-08T19:27:47.119 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.115+0000 7f0fad7c1700 1 -- 172.21.15.143:0/2136292055 wait complete. 2023-11-08T19:27:47.128 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-08T19:27:47.254 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.252+0000 7f4d52bd2700 1 Processor -- start 2023-11-08T19:27:47.254 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.252+0000 7f4d52bd2700 1 -- start start 2023-11-08T19:27:47.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.253+0000 7f4d52bd2700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f4d4c15a950 0x7f4d4c0d01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:47.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.253+0000 7f4d52bd2700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f4d4c0b61e0 0x7f4d4c0cc730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:47.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.253+0000 7f4d52bd2700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d4c0cc2a0 con 0x7f4d4c06bd20 2023-11-08T19:27:47.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.253+0000 7f4d52bd2700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d4c0cc130 con 0x7f4d4c0b61e0 2023-11-08T19:27:47.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.253+0000 7f4d52bd2700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d4c06c1b0 con 0x7f4d4c15a950 2023-11-08T19:27:47.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.254+0000 7f4d5116f700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f4d4c0b61e0 0x7f4d4c0cc730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:47.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.254+0000 7f4d5116f700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f4d4c0b61e0 0x7f4d4c0cc730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54068/0 (socket says 172.21.15.143:54068) 2023-11-08T19:27:47.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.254+0000 7f4d5116f700 1 -- 172.21.15.143:0/1513281236 learned_addr learned my addr 172.21.15.143:0/1513281236 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:47.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.254+0000 7f4d5096e700 1 --2- 172.21.15.143:0/1513281236 >> v2:172.21.15.174:3300/0 conn(0x7f4d4c15a950 0x7f4d4c0d01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:47.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.254+0000 7f4d5116f700 1 -- 172.21.15.143:0/1513281236 >> v2:172.21.15.174:3300/0 conn(0x7f4d4c15a950 msgr2=0x7f4d4c0d01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:47.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.254+0000 7f4d5116f700 1 --2- 172.21.15.143:0/1513281236 >> v2:172.21.15.174:3300/0 conn(0x7f4d4c15a950 0x7f4d4c0d01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:47.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.254+0000 7f4d5116f700 1 -- 172.21.15.143:0/1513281236 >> v1:172.21.15.174:6789/0 conn(0x7f4d4c06bd20 legacy=0x7f4d4c072370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:47.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.254+0000 7f4d5116f700 1 -- 172.21.15.143:0/1513281236 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d4c06c490 con 0x7f4d4c0b61e0 2023-11-08T19:27:47.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.254+0000 7f4d5116f700 1 --2- 172.21.15.143:0/1513281236 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4d4c0b61e0 0x7f4d4c0cc730 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f4d4c0cdf90 tx=0x7f4d4400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac8f62acc0f3898e server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:47.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.254+0000 7f4d4b7fe700 1 -- 172.21.15.143:0/1513281236 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d4400e070 con 0x7f4d4c0b61e0 2023-11-08T19:27:47.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.254+0000 7f4d4b7fe700 1 -- 172.21.15.143:0/1513281236 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d44003a20 con 0x7f4d4c0b61e0 2023-11-08T19:27:47.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.254+0000 7f4d4b7fe700 1 -- 172.21.15.143:0/1513281236 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d44003dd0 con 0x7f4d4c0b61e0 2023-11-08T19:27:47.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.255+0000 7f4d52bd2700 1 -- 172.21.15.143:0/1513281236 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4d4c0b61e0 msgr2=0x7f4d4c0cc730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:47.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.255+0000 7f4d52bd2700 1 --2- 172.21.15.143:0/1513281236 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4d4c0b61e0 0x7f4d4c0cc730 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f4d4c0cdf90 tx=0x7f4d4400b330 comp rx=0 tx=0).stop 2023-11-08T19:27:47.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.255+0000 7f4d52bd2700 1 -- 172.21.15.143:0/1513281236 shutdown_connections 2023-11-08T19:27:47.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.255+0000 7f4d52bd2700 1 --2- 172.21.15.143:0/1513281236 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4d4c0b61e0 0x7f4d4c0cc730 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:47.261 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.255+0000 7f4d52bd2700 1 --2- 172.21.15.143:0/1513281236 >> v2:172.21.15.174:3300/0 conn(0x7f4d4c15a950 0x7f4d4c0d01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:47.261 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.255+0000 7f4d52bd2700 1 -- 172.21.15.143:0/1513281236 >> 172.21.15.143:0/1513281236 conn(0x7f4d4c0cd2e0 msgr2=0x7f4d4c0cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:47.261 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.255+0000 7f4d52bd2700 1 -- 172.21.15.143:0/1513281236 shutdown_connections 2023-11-08T19:27:47.261 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.255+0000 7f4d52bd2700 1 -- 172.21.15.143:0/1513281236 wait complete. 2023-11-08T19:27:47.265 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.256+0000 7f4d52bd2700 1 Processor -- start 2023-11-08T19:27:47.265 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.256+0000 7f4d52bd2700 1 -- start start 2023-11-08T19:27:47.266 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.257+0000 7f4d52bd2700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4d4c0b61e0 0x7f4d4c14ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:47.266 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.257+0000 7f4d52bd2700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4d4c15a950 0x7f4d4c0c6940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:47.266 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.257+0000 7f4d52bd2700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4d4c0c6e80 0x7f4d4c0b4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:47.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.257+0000 7f4d52bd2700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d4c1f47f0 con 0x7f4d4c15a950 2023-11-08T19:27:47.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.257+0000 7f4d52bd2700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d4c1f3b70 con 0x7f4d4c0c6e80 2023-11-08T19:27:47.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.257+0000 7f4d52bd2700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d4c1f3e50 con 0x7f4d4c0b61e0 2023-11-08T19:27:47.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.257+0000 7f4d4bfff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4d4c15a950 0x7f4d4c0c6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:47.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.257+0000 7f4d4bfff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4d4c15a950 0x7f4d4c0c6940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54070/0 (socket says 172.21.15.143:54070) 2023-11-08T19:27:47.268 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.257+0000 7f4d4bfff700 1 -- 172.21.15.143:0/2981281699 learned_addr learned my addr 172.21.15.143:0/2981281699 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:47.268 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.257+0000 7f4d5116f700 1 --2- 172.21.15.143:0/2981281699 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4d4c0c6e80 0x7f4d4c0b4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:47.268 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.257+0000 7f4d5096e700 1 --2- 172.21.15.143:0/2981281699 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4d4c0b61e0 0x7f4d4c14ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:47.268 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.257+0000 7f4d4bfff700 1 -- 172.21.15.143:0/2981281699 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4d4c0b61e0 msgr2=0x7f4d4c14ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:47.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.257+0000 7f4d4bfff700 1 --2- 172.21.15.143:0/2981281699 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4d4c0b61e0 0x7f4d4c14ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:47.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.257+0000 7f4d4bfff700 1 -- 172.21.15.143:0/2981281699 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4d4c0c6e80 msgr2=0x7f4d4c0b4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:47.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.257+0000 7f4d4bfff700 1 --2- 172.21.15.143:0/2981281699 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4d4c0c6e80 0x7f4d4c0b4390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:47.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.257+0000 7f4d4bfff700 1 -- 172.21.15.143:0/2981281699 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d44009b30 con 0x7f4d4c15a950 2023-11-08T19:27:47.270 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.257+0000 7f4d4bfff700 1 --2- 172.21.15.143:0/2981281699 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4d4c15a950 0x7f4d4c0c6940 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f4d3800c4d0 tx=0x7f4d3800c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:47.270 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.258+0000 7f4d497fa700 1 -- 172.21.15.143:0/2981281699 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d38011790 con 0x7f4d4c15a950 2023-11-08T19:27:47.270 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.258+0000 7f4d497fa700 1 -- 172.21.15.143:0/2981281699 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d380118f0 con 0x7f4d4c15a950 2023-11-08T19:27:47.270 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.258+0000 7f4d52bd2700 1 -- 172.21.15.143:0/2981281699 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d4c1f4130 con 0x7f4d4c15a950 2023-11-08T19:27:47.270 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.258+0000 7f4d497fa700 1 -- 172.21.15.143:0/2981281699 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d38011ca0 con 0x7f4d4c15a950 2023-11-08T19:27:47.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.258+0000 7f4d52bd2700 1 -- 172.21.15.143:0/2981281699 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d4c1f37a0 con 0x7f4d4c15a950 2023-11-08T19:27:47.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.258+0000 7f4d52bd2700 1 -- 172.21.15.143:0/2981281699 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d4c1f4930 con 0x7f4d4c15a950 2023-11-08T19:27:47.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.259+0000 7f4d497fa700 1 -- 172.21.15.143:0/2981281699 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4d3801c460 con 0x7f4d4c15a950 2023-11-08T19:27:47.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.259+0000 7f4d497fa700 1 --2- 172.21.15.143:0/2981281699 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f4d3006af00 0x7f4d3006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:47.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.259+0000 7f4d497fa700 1 -- 172.21.15.143:0/2981281699 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6944+0+0 (secure 0 0 0) 0x7f4d38089700 con 0x7f4d4c15a950 2023-11-08T19:27:47.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.260+0000 7f4d5096e700 1 --2- 172.21.15.143:0/2981281699 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f4d3006af00 0x7f4d3006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:47.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.260+0000 7f4d5096e700 1 --2- 172.21.15.143:0/2981281699 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f4d3006af00 0x7f4d3006d3c0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f4d40006b20 tx=0x7f4d40008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:47.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.264+0000 7f4d497fa700 1 -- 172.21.15.143:0/2981281699 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4d4c1f4930 con 0x7f4d4c15a950 2023-11-08T19:27:47.456 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.455+0000 7f4d52bd2700 1 -- 172.21.15.143:0/2981281699 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4d4c1f4930 con 0x7f4d4c15a950 2023-11-08T19:27:47.457 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.455+0000 7f4d497fa700 1 -- 172.21.15.143:0/2981281699 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v81) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4d4c1f4930 con 0x7f4d4c15a950 2023-11-08T19:27:47.457 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 64 2023-11-08T19:27:47.459 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.457+0000 7f4d52bd2700 1 -- 172.21.15.143:0/2981281699 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f4d3006af00 msgr2=0x7f4d3006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:47.459 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.458+0000 7f4d52bd2700 1 --2- 172.21.15.143:0/2981281699 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f4d3006af00 0x7f4d3006d3c0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f4d40006b20 tx=0x7f4d40008040 comp rx=0 tx=0).stop 2023-11-08T19:27:47.460 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.458+0000 7f4d52bd2700 1 -- 172.21.15.143:0/2981281699 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4d4c15a950 msgr2=0x7f4d4c0c6940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:47.460 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.458+0000 7f4d52bd2700 1 --2- 172.21.15.143:0/2981281699 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4d4c15a950 0x7f4d4c0c6940 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f4d3800c4d0 tx=0x7f4d3800c7e0 comp rx=0 tx=0).stop 2023-11-08T19:27:47.460 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.458+0000 7f4d52bd2700 1 -- 172.21.15.143:0/2981281699 shutdown_connections 2023-11-08T19:27:47.460 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.458+0000 7f4d52bd2700 1 --2- 172.21.15.143:0/2981281699 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f4d3006af00 0x7f4d3006d3c0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:47.461 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.458+0000 7f4d52bd2700 1 --2- 172.21.15.143:0/2981281699 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4d4c0b61e0 0x7f4d4c14ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:47.461 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.458+0000 7f4d52bd2700 1 --2- 172.21.15.143:0/2981281699 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4d4c0c6e80 0x7f4d4c0b4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:47.461 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.458+0000 7f4d52bd2700 1 --2- 172.21.15.143:0/2981281699 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4d4c15a950 0x7f4d4c0c6940 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:47.461 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.458+0000 7f4d52bd2700 1 -- 172.21.15.143:0/2981281699 >> 172.21.15.143:0/2981281699 conn(0x7f4d4c0cd2e0 msgr2=0x7f4d4c15c7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:47.461 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.458+0000 7f4d52bd2700 1 -- 172.21.15.143:0/2981281699 shutdown_connections 2023-11-08T19:27:47.462 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.458+0000 7f4d52bd2700 1 -- 172.21.15.143:0/2981281699 wait complete. 2023-11-08T19:27:47.471 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:27:47.592 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.590+0000 7fdcb32b1700 1 Processor -- start 2023-11-08T19:27:47.592 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.590+0000 7fdcb32b1700 1 -- start start 2023-11-08T19:27:47.594 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.591+0000 7fdcb32b1700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdcac159870 0x7fdcac0b2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:47.595 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.591+0000 7fdcb32b1700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fdcac0cc410 0x7fdcac0cc820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:47.595 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.591+0000 7fdcb32b1700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcac0b2a60 con 0x7fdcac06ed60 2023-11-08T19:27:47.595 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.591+0000 7fdcb32b1700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcac06b090 con 0x7fdcac0cc410 2023-11-08T19:27:47.595 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.591+0000 7fdcb32b1700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcac06bbc0 con 0x7fdcac159870 2023-11-08T19:27:47.596 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.591+0000 7fdcb184e700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fdcac0cc410 0x7fdcac0cc820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:47.596 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.591+0000 7fdcb184e700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fdcac0cc410 0x7fdcac0cc820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54082/0 (socket says 172.21.15.143:54082) 2023-11-08T19:27:47.596 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.591+0000 7fdcb184e700 1 -- 172.21.15.143:0/911004248 learned_addr learned my addr 172.21.15.143:0/911004248 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:47.596 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.591+0000 7fdcb084c700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fdcac06ed60 0x7fdcac06b4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:50232/0 (socket says 172.21.15.143:50232) 2023-11-08T19:27:47.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.592+0000 7fdcb104d700 1 --2- 172.21.15.143:0/911004248 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdcac159870 0x7fdcac0b2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:47.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.592+0000 7fdcb184e700 1 -- 172.21.15.143:0/911004248 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdcac159870 msgr2=0x7fdcac0b2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:47.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.592+0000 7fdcb184e700 1 --2- 172.21.15.143:0/911004248 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdcac159870 0x7fdcac0b2210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:47.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.592+0000 7fdcb184e700 1 -- 172.21.15.143:0/911004248 >> v1:172.21.15.143:6789/0 conn(0x7fdcac06ed60 legacy=0x7fdcac06b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:47.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.592+0000 7fdcb184e700 1 -- 172.21.15.143:0/911004248 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcac0b2d40 con 0x7fdcac0cc410 2023-11-08T19:27:47.598 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.592+0000 7fdcabfff700 1 -- 172.21.15.143:0/911004248 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1493642769 0 0) 0x7fdcac0b2a60 con 0x7fdcac06ed60 2023-11-08T19:27:47.598 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.592+0000 7fdcb184e700 1 --2- 172.21.15.143:0/911004248 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdcac0cc410 0x7fdcac0cc820 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c0098f0 tx=0x7fdc9c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=321b25cde0320e3e server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:47.598 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.592+0000 7fdcabfff700 1 -- 172.21.15.143:0/911004248 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc9c011830 con 0x7fdcac0cc410 2023-11-08T19:27:47.598 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.592+0000 7fdcabfff700 1 -- 172.21.15.143:0/911004248 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc9c011990 con 0x7fdcac0cc410 2023-11-08T19:27:47.598 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.592+0000 7fdcabfff700 1 -- 172.21.15.143:0/911004248 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc9c011d20 con 0x7fdcac0cc410 2023-11-08T19:27:47.599 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.593+0000 7fdcb32b1700 1 -- 172.21.15.143:0/911004248 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdcac0cc410 msgr2=0x7fdcac0cc820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:47.599 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.593+0000 7fdcb32b1700 1 --2- 172.21.15.143:0/911004248 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdcac0cc410 0x7fdcac0cc820 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c0098f0 tx=0x7fdc9c009c00 comp rx=0 tx=0).stop 2023-11-08T19:27:47.599 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.593+0000 7fdcb32b1700 1 -- 172.21.15.143:0/911004248 shutdown_connections 2023-11-08T19:27:47.599 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.593+0000 7fdcb32b1700 1 --2- 172.21.15.143:0/911004248 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdcac0cc410 0x7fdcac0cc820 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:47.599 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.593+0000 7fdcb32b1700 1 --2- 172.21.15.143:0/911004248 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdcac159870 0x7fdcac0b2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:47.600 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.593+0000 7fdcb32b1700 1 -- 172.21.15.143:0/911004248 >> 172.21.15.143:0/911004248 conn(0x7fdcac099ab0 msgr2=0x7fdcac0d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:47.600 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.593+0000 7fdcb32b1700 1 -- 172.21.15.143:0/911004248 shutdown_connections 2023-11-08T19:27:47.600 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.593+0000 7fdcb32b1700 1 -- 172.21.15.143:0/911004248 wait complete. 2023-11-08T19:27:47.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.594+0000 7fdcb32b1700 1 Processor -- start 2023-11-08T19:27:47.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.594+0000 7fdcb32b1700 1 -- start start 2023-11-08T19:27:47.604 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.595+0000 7fdcb32b1700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdcac06ed60 0x7fdcac0c77d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:47.604 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.595+0000 7fdcb32b1700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdcac0cc410 0x7fdcac157850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:47.604 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.595+0000 7fdcb32b1700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdcac159870 0x7fdcac15c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:47.604 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.595+0000 7fdcb32b1700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcac0c7d10 con 0x7fdcac0cc410 2023-11-08T19:27:47.604 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.595+0000 7fdcb32b1700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcac157d90 con 0x7fdcac06ed60 2023-11-08T19:27:47.605 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.595+0000 7fdcb32b1700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcac15b380 con 0x7fdcac159870 2023-11-08T19:27:47.605 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.595+0000 7fdcb084c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdcac0cc410 0x7fdcac157850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:47.605 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.595+0000 7fdcb084c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdcac0cc410 0x7fdcac157850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54086/0 (socket says 172.21.15.143:54086) 2023-11-08T19:27:47.605 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.595+0000 7fdcb084c700 1 -- 172.21.15.143:0/4154648242 learned_addr learned my addr 172.21.15.143:0/4154648242 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:47.605 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.595+0000 7fdcb184e700 1 --2- 172.21.15.143:0/4154648242 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdcac159870 0x7fdcac15c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:47.606 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.595+0000 7fdcb104d700 1 --2- 172.21.15.143:0/4154648242 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdcac06ed60 0x7fdcac0c77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:47.606 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.595+0000 7fdcb084c700 1 -- 172.21.15.143:0/4154648242 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdcac159870 msgr2=0x7fdcac15c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:47.606 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.595+0000 7fdcb084c700 1 --2- 172.21.15.143:0/4154648242 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdcac159870 0x7fdcac15c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:47.606 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.595+0000 7fdcb084c700 1 -- 172.21.15.143:0/4154648242 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdcac06ed60 msgr2=0x7fdcac0c77d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:47.606 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.595+0000 7fdcb084c700 1 --2- 172.21.15.143:0/4154648242 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdcac06ed60 0x7fdcac0c77d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:47.607 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.595+0000 7fdcb084c700 1 -- 172.21.15.143:0/4154648242 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc9c009550 con 0x7fdcac0cc410 2023-11-08T19:27:47.607 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.595+0000 7fdcb104d700 1 --2- 172.21.15.143:0/4154648242 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdcac06ed60 0x7fdcac0c77d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:27:47.607 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.595+0000 7fdcb084c700 1 --2- 172.21.15.143:0/4154648242 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdcac0cc410 0x7fdcac157850 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fdca000c550 tx=0x7fdca000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:47.607 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.595+0000 7fdca9ffb700 1 -- 172.21.15.143:0/4154648242 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdca0004c80 con 0x7fdcac0cc410 2023-11-08T19:27:47.607 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.595+0000 7fdca9ffb700 1 -- 172.21.15.143:0/4154648242 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdca0004e00 con 0x7fdcac0cc410 2023-11-08T19:27:47.608 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.595+0000 7fdcb32b1700 1 -- 172.21.15.143:0/4154648242 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcac15b660 con 0x7fdcac0cc410 2023-11-08T19:27:47.608 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.595+0000 7fdca9ffb700 1 -- 172.21.15.143:0/4154648242 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdca0007310 con 0x7fdcac0cc410 2023-11-08T19:27:47.608 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.596+0000 7fdcb32b1700 1 -- 172.21.15.143:0/4154648242 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcac15bc20 con 0x7fdcac0cc410 2023-11-08T19:27:47.608 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.596+0000 7fdcb32b1700 1 -- 172.21.15.143:0/4154648242 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcac04eee0 con 0x7fdcac0cc410 2023-11-08T19:27:47.608 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.597+0000 7fdca9ffb700 1 -- 172.21.15.143:0/4154648242 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fdca0007470 con 0x7fdcac0cc410 2023-11-08T19:27:47.609 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.597+0000 7fdca9ffb700 1 --2- 172.21.15.143:0/4154648242 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fdc9806af00 0x7fdc9806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:47.609 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.597+0000 7fdca9ffb700 1 -- 172.21.15.143:0/4154648242 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6944+0+0 (secure 0 0 0) 0x7fdca00896f0 con 0x7fdcac0cc410 2023-11-08T19:27:47.609 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.597+0000 7fdcb104d700 1 --2- 172.21.15.143:0/4154648242 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fdc9806af00 0x7fdc9806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:47.609 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.598+0000 7fdcb104d700 1 --2- 172.21.15.143:0/4154648242 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fdc9806af00 0x7fdc9806d3c0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fdca4007500 tx=0x7fdca400b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:47.609 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.602+0000 7fdca9ffb700 1 -- 172.21.15.143:0/4154648242 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdca00542d0 con 0x7fdcac0cc410 2023-11-08T19:27:47.832 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.831+0000 7fdcb32b1700 1 -- 172.21.15.143:0/4154648242 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdcac0cdaa0 con 0x7fdcac0cc410 2023-11-08T19:27:47.832 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.831+0000 7fdca9ffb700 1 -- 172.21.15.143:0/4154648242 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+4921 (secure 0 0 0) 0x7fdca00578f0 con 0x7fdcac0cc410 2023-11-08T19:27:47.833 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:27:47.833 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":43,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5510,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6836/3796015890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3796015890},{"type":"v1","addr":"172.21.15.174:6836","nonce":3796015890}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37},{"gid":6471,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6834/432456394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":432456394},{"type":"v1","addr":"172.21.15.143:6834","nonce":432456394}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37},{"gid":6474,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/310920972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":310920972},{"type":"v1","addr":"172.21.15.143:6835","nonce":310920972}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:27:31.702539+0000","modified":"2023-11-08T19:27:40.808769+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6489},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6489":{"gid":6489,"name":"d","rank":0,"incarnation":38,"state":"up:active","state_seq":3,"addr":"172.21.15.174:6837/1950356087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6835","nonce":1950356087},{"type":"v1","addr":"172.21.15.174:6837","nonce":1950356087}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-08T19:27:47.835 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.833+0000 7fdcb32b1700 1 -- 172.21.15.143:0/4154648242 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fdc9806af00 msgr2=0x7fdc9806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:47.835 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.833+0000 7fdcb32b1700 1 --2- 172.21.15.143:0/4154648242 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fdc9806af00 0x7fdc9806d3c0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fdca4007500 tx=0x7fdca400b040 comp rx=0 tx=0).stop 2023-11-08T19:27:47.836 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.833+0000 7fdcb32b1700 1 -- 172.21.15.143:0/4154648242 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdcac0cc410 msgr2=0x7fdcac157850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:47.836 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.833+0000 7fdcb32b1700 1 --2- 172.21.15.143:0/4154648242 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdcac0cc410 0x7fdcac157850 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fdca000c550 tx=0x7fdca000c860 comp rx=0 tx=0).stop 2023-11-08T19:27:47.836 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.833+0000 7fdcb32b1700 1 -- 172.21.15.143:0/4154648242 shutdown_connections 2023-11-08T19:27:47.836 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.833+0000 7fdcb32b1700 1 --2- 172.21.15.143:0/4154648242 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fdc9806af00 0x7fdc9806d3c0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:47.836 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.833+0000 7fdcb32b1700 1 --2- 172.21.15.143:0/4154648242 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdcac159870 0x7fdcac15c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:47.837 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.833+0000 7fdcb32b1700 1 --2- 172.21.15.143:0/4154648242 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdcac06ed60 0x7fdcac0c77d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:47.837 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.833+0000 7fdcb32b1700 1 --2- 172.21.15.143:0/4154648242 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdcac0cc410 0x7fdcac157850 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:47.837 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.833+0000 7fdcb32b1700 1 -- 172.21.15.143:0/4154648242 >> 172.21.15.143:0/4154648242 conn(0x7fdcac099ab0 msgr2=0x7fdcac0b00d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:47.837 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.833+0000 7fdcb32b1700 1 -- 172.21.15.143:0/4154648242 shutdown_connections 2023-11-08T19:27:47.837 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.833+0000 7fdcb32b1700 1 -- 172.21.15.143:0/4154648242 wait complete. 2023-11-08T19:27:47.838 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 43 2023-11-08T19:27:47.847 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:27:47.968 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.967+0000 7f52fbb8d700 1 Processor -- start 2023-11-08T19:27:47.969 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.967+0000 7f52fbb8d700 1 -- start start 2023-11-08T19:27:47.971 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.968+0000 7f52fbb8d700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f52f40d1c70 0x7f52f40b14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:47.971 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.968+0000 7f52fbb8d700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52f40b1bc0 con 0x7f52f415ba30 2023-11-08T19:27:47.972 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.968+0000 7f52fbb8d700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52f40b1ea0 con 0x7f52f40d03a0 2023-11-08T19:27:47.972 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.968+0000 7f52fbb8d700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52f40b2180 con 0x7f52f40d1c70 2023-11-08T19:27:47.972 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.968+0000 7f52f9929700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f52f415ba30 0x7f52f40d1560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:50246/0 (socket says 172.21.15.143:50246) 2023-11-08T19:27:47.973 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.968+0000 7f52f9929700 1 -- 172.21.15.143:0/1419274277 learned_addr learned my addr 172.21.15.143:0/1419274277 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:27:47.973 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.968+0000 7f52f9128700 1 --2- 172.21.15.143:0/1419274277 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f52f40d1c70 0x7f52f40b14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:47.973 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.968+0000 7f52f8927700 1 -- 172.21.15.143:0/1419274277 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2042219391 0 0) 0x7f52f40b1bc0 con 0x7f52f415ba30 2023-11-08T19:27:47.974 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.968+0000 7f52f8927700 1 -- 172.21.15.143:0/1419274277 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52e00035b0 con 0x7f52f415ba30 2023-11-08T19:27:47.974 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.968+0000 7f52f8927700 1 -- 172.21.15.143:0/1419274277 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1487658684 0 0) 0x7f52e00035b0 con 0x7f52f415ba30 2023-11-08T19:27:47.974 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.968+0000 7f52f9128700 1 -- 172.21.15.143:0/1419274277 >> v1:172.21.15.174:6789/0 conn(0x7f52f40d03a0 legacy=0x7f52f40cc5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:47.975 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.968+0000 7f52f9128700 1 -- 172.21.15.143:0/1419274277 >> v1:172.21.15.143:6789/0 conn(0x7f52f415ba30 legacy=0x7f52f40d1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:47.975 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.968+0000 7f52f9128700 1 -- 172.21.15.143:0/1419274277 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52f40b2460 con 0x7f52f40d1c70 2023-11-08T19:27:47.975 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.969+0000 7f52f9128700 1 --2- 172.21.15.143:0/1419274277 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f52f40d1c70 0x7f52f40b14c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f52e4009870 tx=0x7f52e4009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=669066e20a6514c7 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:47.976 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.969+0000 7f52f8927700 1 -- 172.21.15.143:0/1419274277 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52f0001670 con 0x7f52f40d1c70 2023-11-08T19:27:47.976 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.969+0000 7f52f8927700 1 -- 172.21.15.143:0/1419274277 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52e4004030 con 0x7f52f40d1c70 2023-11-08T19:27:47.977 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.969+0000 7f52f8927700 1 -- 172.21.15.143:0/1419274277 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52e40043c0 con 0x7f52f40d1c70 2023-11-08T19:27:47.977 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.969+0000 7f52fbb8d700 1 -- 172.21.15.143:0/1419274277 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f52f40d1c70 msgr2=0x7f52f40b14c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:47.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.969+0000 7f52fbb8d700 1 --2- 172.21.15.143:0/1419274277 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f52f40d1c70 0x7f52f40b14c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f52e4009870 tx=0x7f52e4009b80 comp rx=0 tx=0).stop 2023-11-08T19:27:47.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.969+0000 7f52fbb8d700 1 -- 172.21.15.143:0/1419274277 shutdown_connections 2023-11-08T19:27:47.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.969+0000 7f52fbb8d700 1 --2- 172.21.15.143:0/1419274277 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f52f40d1c70 0x7f52f40b14c0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:47.979 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.969+0000 7f52fbb8d700 1 -- 172.21.15.143:0/1419274277 >> 172.21.15.143:0/1419274277 conn(0x7f52f408b000 msgr2=0x7f52f409ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:47.979 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.969+0000 7f52fbb8d700 1 -- 172.21.15.143:0/1419274277 shutdown_connections 2023-11-08T19:27:47.980 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.969+0000 7f52fbb8d700 1 -- 172.21.15.143:0/1419274277 wait complete. 2023-11-08T19:27:47.983 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.971+0000 7f52fbb8d700 1 Processor -- start 2023-11-08T19:27:47.983 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.971+0000 7f52fbb8d700 1 -- start start 2023-11-08T19:27:47.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.971+0000 7f52fbb8d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f52f40d03a0 0x7f52f40c5c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:47.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.971+0000 7f52fbb8d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f52f40d1c70 0x7f52f4157db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:47.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.971+0000 7f52fbb8d700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f52f415ba30 0x7f52f40ce1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:47.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.971+0000 7f52fbb8d700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52f40c6d20 con 0x7f52f40d03a0 2023-11-08T19:27:47.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.971+0000 7f52fbb8d700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52f40c6480 con 0x7f52f40d1c70 2023-11-08T19:27:47.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.971+0000 7f52fbb8d700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52f40c6760 con 0x7f52f415ba30 2023-11-08T19:27:47.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.971+0000 7f52f9929700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f52f40d03a0 0x7f52f40c5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:47.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.971+0000 7f52f9929700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f52f40d03a0 0x7f52f40c5c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54098/0 (socket says 172.21.15.143:54098) 2023-11-08T19:27:47.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.971+0000 7f52f9929700 1 -- 172.21.15.143:0/1054071736 learned_addr learned my addr 172.21.15.143:0/1054071736 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:47.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.972+0000 7f52f9128700 1 --2- 172.21.15.143:0/1054071736 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f52f40d1c70 0x7f52f4157db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:47.986 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.972+0000 7f52fa12a700 1 --2- 172.21.15.143:0/1054071736 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f52f415ba30 0x7f52f40ce1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:47.986 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.972+0000 7f52f9929700 1 -- 172.21.15.143:0/1054071736 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f52f415ba30 msgr2=0x7f52f40ce1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:47.986 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.972+0000 7f52f9929700 1 --2- 172.21.15.143:0/1054071736 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f52f415ba30 0x7f52f40ce1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:47.986 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.972+0000 7f52f9929700 1 -- 172.21.15.143:0/1054071736 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f52f40d1c70 msgr2=0x7f52f4157db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:47.986 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.972+0000 7f52f9929700 1 --2- 172.21.15.143:0/1054071736 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f52f40d1c70 0x7f52f4157db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:47.987 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.972+0000 7f52f9929700 1 -- 172.21.15.143:0/1054071736 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52e40094a0 con 0x7f52f40d03a0 2023-11-08T19:27:47.987 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.972+0000 7f52f9128700 1 --2- 172.21.15.143:0/1054071736 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f52f40d1c70 0x7f52f4157db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:27:47.987 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.972+0000 7f52f9929700 1 --2- 172.21.15.143:0/1054071736 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f52f40d03a0 0x7f52f40c5c50 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f52f000c730 tx=0x7f52f000caf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:47.987 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.972+0000 7f52ea7fc700 1 -- 172.21.15.143:0/1054071736 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52f0004c80 con 0x7f52f40d03a0 2023-11-08T19:27:47.987 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.972+0000 7f52ea7fc700 1 -- 172.21.15.143:0/1054071736 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52f0004de0 con 0x7f52f40d03a0 2023-11-08T19:27:47.988 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.972+0000 7f52ea7fc700 1 -- 172.21.15.143:0/1054071736 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52f0007310 con 0x7f52f40d03a0 2023-11-08T19:27:47.988 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.972+0000 7f52fbb8d700 1 -- 172.21.15.143:0/1054071736 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52f40c6a40 con 0x7f52f40d03a0 2023-11-08T19:27:47.988 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.972+0000 7f52fbb8d700 1 -- 172.21.15.143:0/1054071736 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52f40c6190 con 0x7f52f40d03a0 2023-11-08T19:27:47.988 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.973+0000 7f52fbb8d700 1 -- 172.21.15.143:0/1054071736 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52f40b77a0 con 0x7f52f40d03a0 2023-11-08T19:27:47.988 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.973+0000 7f52ea7fc700 1 -- 172.21.15.143:0/1054071736 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f52f0007470 con 0x7f52f40d03a0 2023-11-08T19:27:47.988 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.974+0000 7f52ea7fc700 1 --2- 172.21.15.143:0/1054071736 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f52d806af00 0x7f52d806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:47.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.974+0000 7f52ea7fc700 1 -- 172.21.15.143:0/1054071736 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6944+0+0 (secure 0 0 0) 0x7f52f00898f0 con 0x7f52f40d03a0 2023-11-08T19:27:47.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.974+0000 7f52f9128700 1 --2- 172.21.15.143:0/1054071736 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f52d806af00 0x7f52d806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:47.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.974+0000 7f52f9128700 1 --2- 172.21.15.143:0/1054071736 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f52d806af00 0x7f52d806d3c0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f52e40097c0 tx=0x7f52e4008000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:47.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:47.978+0000 7f52ea7fc700 1 -- 172.21.15.143:0/1054071736 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f52f00544d0 con 0x7f52f40d03a0 2023-11-08T19:27:48.168 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.167+0000 7f52fbb8d700 1 -- 172.21.15.143:0/1054071736 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f52f404eee0 con 0x7f52f40d03a0 2023-11-08T19:27:48.169 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.168+0000 7f52ea7fc700 1 -- 172.21.15.143:0/1054071736 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+22107 (secure 0 0 0) 0x7f52f0057af0 con 0x7f52f40d03a0 2023-11-08T19:27:48.170 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:27:48.170 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":81,"fsid":"968483af-77fc-4068-ad7c-5c2a1a4613d7","created":"2023-11-08T19:24:32.826484+0000","modified":"2023-11-08T19:27:46.378783+0000","last_up_change":"2023-11-08T19:24:43.573880+0000","last_in_change":"2023-11-08T19:24:35.314525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:44.551530+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"snap-test-pool","create_time":"2023-11-08T19:26:02.874384+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":36,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"test_data_pool","create_time":"2023-11-08T19:27:05.601949+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":66,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_metadata_pool","create_time":"2023-11-08T19:27:06.536369+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-11-08T19:27:30.137037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-11-08T19:27:30.660318+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":81,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bae98ec1-d94e-403d-a220-a522a53ea6f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6809","nonce":1487830566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6811","nonce":1487830566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6815","nonce":1487830566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6813","nonce":1487830566}]},"public_addr":"172.21.15.143:6809/1487830566","cluster_addr":"172.21.15.143:6811/1487830566","heartbeat_back_addr":"172.21.15.143:6815/1487830566","heartbeat_front_addr":"172.21.15.143:6813/1487830566","state":["exists","up"]},{"osd":1,"uuid":"5cb6538a-1ccb-4322-8633-09f66af0b7e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6801","nonce":787341397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6803","nonce":787341397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6807","nonce":787341397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6805","nonce":787341397}]},"public_addr":"172.21.15.143:6801/787341397","cluster_addr":"172.21.15.143:6803/787341397","heartbeat_back_addr":"172.21.15.143:6807/787341397","heartbeat_front_addr":"172.21.15.143:6805/787341397","state":["exists","up"]},{"osd":2,"uuid":"cdbf522c-60ef-4d60-9653-8dcb529c101a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6822","nonce":2301325597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6826","nonce":2301325597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6831","nonce":2301325597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6829","nonce":2301325597}]},"public_addr":"172.21.15.143:6822/2301325597","cluster_addr":"172.21.15.143:6826/2301325597","heartbeat_back_addr":"172.21.15.143:6831/2301325597","heartbeat_front_addr":"172.21.15.143:6829/2301325597","state":["exists","up"]},{"osd":3,"uuid":"4e2783cf-ebb9-4dd8-8245-bc1d17ce4325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6817","nonce":2021504428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6819","nonce":2021504428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6827","nonce":2021504428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6823","nonce":2021504428}]},"public_addr":"172.21.15.143:6817/2021504428","cluster_addr":"172.21.15.143:6819/2021504428","heartbeat_back_addr":"172.21.15.143:6827/2021504428","heartbeat_front_addr":"172.21.15.143:6823/2021504428","state":["exists","up"]},{"osd":4,"uuid":"691256fa-9b77-49b9-a346-7ae1adf0ca80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6801","nonce":1993380595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6803","nonce":1993380595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6807","nonce":1993380595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6805","nonce":1993380595}]},"public_addr":"172.21.15.174:6801/1993380595","cluster_addr":"172.21.15.174:6803/1993380595","heartbeat_back_addr":"172.21.15.174:6807/1993380595","heartbeat_front_addr":"172.21.15.174:6805/1993380595","state":["exists","up"]},{"osd":5,"uuid":"756fc352-48c9-46ce-b52e-0541fe6f8525","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6809","nonce":1777923303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6811","nonce":1777923303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6815","nonce":1777923303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6813","nonce":1777923303}]},"public_addr":"172.21.15.174:6809/1777923303","cluster_addr":"172.21.15.174:6811/1777923303","heartbeat_back_addr":"172.21.15.174:6815/1777923303","heartbeat_front_addr":"172.21.15.174:6813/1777923303","state":["exists","up"]},{"osd":6,"uuid":"fab2ac2b-ed97-4cde-9825-b480eabf8b8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6825","nonce":2755130316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6827","nonce":2755130316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6831","nonce":2755130316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6829","nonce":2755130316}]},"public_addr":"172.21.15.174:6825/2755130316","cluster_addr":"172.21.15.174:6827/2755130316","heartbeat_back_addr":"172.21.15.174:6831/2755130316","heartbeat_front_addr":"172.21.15.174:6829/2755130316","state":["exists","up"]},{"osd":7,"uuid":"3de9b797-2ab6-4d41-a863-7e7d638fd703","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6817","nonce":2847141411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6819","nonce":2847141411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6823","nonce":2847141411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6821","nonce":2847141411}]},"public_addr":"172.21.15.174:6817/2847141411","cluster_addr":"172.21.15.174:6819/2847141411","heartbeat_back_addr":"172.21.15.174:6823/2847141411","heartbeat_front_addr":"172.21.15.174:6821/2847141411","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.477076+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.496890+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.539305+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.454673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.538881+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.521294+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.520712+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.540404+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.a","mappings":[{"from":1,"to":7}]},{"pgid":"14.17","mappings":[{"from":1,"to":7}]},{"pgid":"14.29","mappings":[{"from":1,"to":2}]},{"pgid":"14.32","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":14,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":14,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:27:48.172 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.170+0000 7f52fbb8d700 1 -- 172.21.15.143:0/1054071736 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f52d806af00 msgr2=0x7f52d806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:48.172 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.170+0000 7f52fbb8d700 1 --2- 172.21.15.143:0/1054071736 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f52d806af00 0x7f52d806d3c0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f52e40097c0 tx=0x7f52e4008000 comp rx=0 tx=0).stop 2023-11-08T19:27:48.172 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.170+0000 7f52fbb8d700 1 -- 172.21.15.143:0/1054071736 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f52f40d03a0 msgr2=0x7f52f40c5c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:48.173 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.170+0000 7f52fbb8d700 1 --2- 172.21.15.143:0/1054071736 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f52f40d03a0 0x7f52f40c5c50 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f52f000c730 tx=0x7f52f000caf0 comp rx=0 tx=0).stop 2023-11-08T19:27:48.173 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.170+0000 7f52fbb8d700 1 -- 172.21.15.143:0/1054071736 shutdown_connections 2023-11-08T19:27:48.173 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.170+0000 7f52fbb8d700 1 --2- 172.21.15.143:0/1054071736 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f52d806af00 0x7f52d806d3c0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:48.173 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.170+0000 7f52fbb8d700 1 --2- 172.21.15.143:0/1054071736 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f52f415ba30 0x7f52f40ce1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:48.173 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.170+0000 7f52fbb8d700 1 --2- 172.21.15.143:0/1054071736 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f52f40d03a0 0x7f52f40c5c50 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:48.173 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.170+0000 7f52fbb8d700 1 --2- 172.21.15.143:0/1054071736 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f52f40d1c70 0x7f52f4157db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:48.174 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.170+0000 7f52fbb8d700 1 -- 172.21.15.143:0/1054071736 >> 172.21.15.143:0/1054071736 conn(0x7f52f408b000 msgr2=0x7f52f40aeaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:48.174 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.170+0000 7f52fbb8d700 1 -- 172.21.15.143:0/1054071736 shutdown_connections 2023-11-08T19:27:48.174 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.170+0000 7f52fbb8d700 1 -- 172.21.15.143:0/1054071736 wait complete. 2023-11-08T19:27:48.184 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-08T19:27:48.185 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:27:48.306 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.304+0000 7f2206f0d700 1 Processor -- start 2023-11-08T19:27:48.306 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.304+0000 7f2206f0d700 1 -- start start 2023-11-08T19:27:48.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.305+0000 7f2206f0d700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f220006dbb0 0x7f22000ca470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:48.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.305+0000 7f2206f0d700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f22000668e0 0x7f22000ca9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:48.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.305+0000 7f2206f0d700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22000ce7f0 con 0x7f220006df90 2023-11-08T19:27:48.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.305+0000 7f2206f0d700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2200066db0 con 0x7f220006dbb0 2023-11-08T19:27:48.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.305+0000 7f2206f0d700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22000af5e0 con 0x7f22000668e0 2023-11-08T19:27:48.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.305+0000 7f2204ca9700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f220006dbb0 0x7f22000ca470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:48.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.305+0000 7f22054aa700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f220006df90 0x7f22001621f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:50264/0 (socket says 172.21.15.143:50264) 2023-11-08T19:27:48.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.305+0000 7f22054aa700 1 -- 172.21.15.143:0/2584951892 learned_addr learned my addr 172.21.15.143:0/2584951892 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:27:48.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.306+0000 7f21fffff700 1 --2- 172.21.15.143:0/2584951892 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f22000668e0 0x7f22000ca9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:48.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.306+0000 7f2204ca9700 1 -- 172.21.15.143:0/2584951892 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f22000668e0 msgr2=0x7f22000ca9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:48.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.306+0000 7f2204ca9700 1 --2- 172.21.15.143:0/2584951892 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f22000668e0 0x7f22000ca9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:48.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.306+0000 7f2204ca9700 1 -- 172.21.15.143:0/2584951892 >> v1:172.21.15.143:6789/0 conn(0x7f220006df90 legacy=0x7f22001621f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:48.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.306+0000 7f21ff7fe700 1 -- 172.21.15.143:0/2584951892 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2150175390 0 0) 0x7f22000ce7f0 con 0x7f220006df90 2023-11-08T19:27:48.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.306+0000 7f2204ca9700 1 -- 172.21.15.143:0/2584951892 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22000cb520 con 0x7f220006dbb0 2023-11-08T19:27:48.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.306+0000 7f2204ca9700 1 --2- 172.21.15.143:0/2584951892 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f220006dbb0 0x7f22000ca470 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f21f4009840 tx=0x7f21f4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=790cd855906c0da0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:48.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.306+0000 7f21ff7fe700 1 -- 172.21.15.143:0/2584951892 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f4011830 con 0x7f220006dbb0 2023-11-08T19:27:48.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.306+0000 7f21ff7fe700 1 -- 172.21.15.143:0/2584951892 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21f4011990 con 0x7f220006dbb0 2023-11-08T19:27:48.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.306+0000 7f21ff7fe700 1 -- 172.21.15.143:0/2584951892 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f4011d20 con 0x7f220006dbb0 2023-11-08T19:27:48.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.307+0000 7f2206f0d700 1 -- 172.21.15.143:0/2584951892 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f220006dbb0 msgr2=0x7f22000ca470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:48.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.307+0000 7f2206f0d700 1 --2- 172.21.15.143:0/2584951892 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f220006dbb0 0x7f22000ca470 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f21f4009840 tx=0x7f21f4009b50 comp rx=0 tx=0).stop 2023-11-08T19:27:48.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.307+0000 7f2206f0d700 1 -- 172.21.15.143:0/2584951892 shutdown_connections 2023-11-08T19:27:48.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.307+0000 7f2206f0d700 1 --2- 172.21.15.143:0/2584951892 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f220006dbb0 0x7f22000ca470 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:48.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.307+0000 7f2206f0d700 1 --2- 172.21.15.143:0/2584951892 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f22000668e0 0x7f22000ca9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:48.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.307+0000 7f2206f0d700 1 -- 172.21.15.143:0/2584951892 >> 172.21.15.143:0/2584951892 conn(0x7f22000b1a20 msgr2=0x7f22000b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:48.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.307+0000 7f2206f0d700 1 -- 172.21.15.143:0/2584951892 shutdown_connections 2023-11-08T19:27:48.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.307+0000 7f2206f0d700 1 -- 172.21.15.143:0/2584951892 wait complete. 2023-11-08T19:27:48.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.308+0000 7f2206f0d700 1 Processor -- start 2023-11-08T19:27:48.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.308+0000 7f2206f0d700 1 -- start start 2023-11-08T19:27:48.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.309+0000 7f2206f0d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f22000668e0 0x7f220015da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:48.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.309+0000 7f2206f0d700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f220006dbb0 0x7f22001585e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:48.318 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.309+0000 7f2206f0d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f220006df90 0x7f22000b3490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:48.318 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.309+0000 7f2206f0d700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f220015dfa0 con 0x7f220006df90 2023-11-08T19:27:48.318 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.309+0000 7f2206f0d700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22000b42e0 con 0x7f22000668e0 2023-11-08T19:27:48.318 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.309+0000 7f2206f0d700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22000c3af0 con 0x7f220006dbb0 2023-11-08T19:27:48.318 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.309+0000 7f22054aa700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f220006df90 0x7f22000b3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:48.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.309+0000 7f22054aa700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f220006df90 0x7f22000b3490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54116/0 (socket says 172.21.15.143:54116) 2023-11-08T19:27:48.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.309+0000 7f22054aa700 1 -- 172.21.15.143:0/1363321884 learned_addr learned my addr 172.21.15.143:0/1363321884 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:48.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.309+0000 7f21fffff700 1 --2- 172.21.15.143:0/1363321884 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f220006dbb0 0x7f22001585e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:48.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.309+0000 7f2204ca9700 1 --2- 172.21.15.143:0/1363321884 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f22000668e0 0x7f220015da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:48.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.309+0000 7f22054aa700 1 -- 172.21.15.143:0/1363321884 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f220006dbb0 msgr2=0x7f22001585e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:48.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.309+0000 7f22054aa700 1 --2- 172.21.15.143:0/1363321884 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f220006dbb0 0x7f22001585e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:48.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.309+0000 7f22054aa700 1 -- 172.21.15.143:0/1363321884 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f22000668e0 msgr2=0x7f220015da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:48.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.309+0000 7f22054aa700 1 --2- 172.21.15.143:0/1363321884 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f22000668e0 0x7f220015da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:48.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.309+0000 7f22054aa700 1 -- 172.21.15.143:0/1363321884 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21f40094a0 con 0x7f220006df90 2023-11-08T19:27:48.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.309+0000 7f22054aa700 1 --2- 172.21.15.143:0/1363321884 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f220006df90 0x7f22000b3490 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f21f800cba0 tx=0x7f21f800ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:48.321 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.309+0000 7f21fd7fa700 1 -- 172.21.15.143:0/1363321884 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f800aaa0 con 0x7f220006df90 2023-11-08T19:27:48.321 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.310+0000 7f21fd7fa700 1 -- 172.21.15.143:0/1363321884 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21f800ac20 con 0x7f220006df90 2023-11-08T19:27:48.321 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.310+0000 7f21fd7fa700 1 -- 172.21.15.143:0/1363321884 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f80070d0 con 0x7f220006df90 2023-11-08T19:27:48.321 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.310+0000 7f2206f0d700 1 -- 172.21.15.143:0/1363321884 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22000b4000 con 0x7f220006df90 2023-11-08T19:27:48.321 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.310+0000 7f2206f0d700 1 -- 172.21.15.143:0/1363321884 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22000c4070 con 0x7f220006df90 2023-11-08T19:27:48.322 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.311+0000 7f2206f0d700 1 -- 172.21.15.143:0/1363321884 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f220006d1a0 con 0x7f220006df90 2023-11-08T19:27:48.322 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.311+0000 7f21fd7fa700 1 -- 172.21.15.143:0/1363321884 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f21f8007230 con 0x7f220006df90 2023-11-08T19:27:48.322 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.311+0000 7f21fd7fa700 1 --2- 172.21.15.143:0/1363321884 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f21e406af00 0x7f21e406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:48.322 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.311+0000 7f21fd7fa700 1 -- 172.21.15.143:0/1363321884 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6944+0+0 (secure 0 0 0) 0x7f21f8089510 con 0x7f220006df90 2023-11-08T19:27:48.322 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.312+0000 7f2204ca9700 1 --2- 172.21.15.143:0/1363321884 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f21e406af00 0x7f21e406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:48.323 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.312+0000 7f2204ca9700 1 --2- 172.21.15.143:0/1363321884 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f21e406af00 0x7f21e406d3c0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f21f40090b0 tx=0x7f21f4008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:48.323 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.316+0000 7f21fd7fa700 1 -- 172.21.15.143:0/1363321884 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f21f80540f0 con 0x7f220006df90 2023-11-08T19:27:48.550 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.549+0000 7f2206f0d700 1 -- 172.21.15.143:0/1363321884 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f220004eee0 con 0x7f220006df90 2023-11-08T19:27:48.551 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.549+0000 7f21fd7fa700 1 -- 172.21.15.143:0/1363321884 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+4921 (secure 0 0 0) 0x7f21f8057710 con 0x7f220006df90 2023-11-08T19:27:48.551 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:27:48.551 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":43,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5510,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6836/3796015890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3796015890},{"type":"v1","addr":"172.21.15.174:6836","nonce":3796015890}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37},{"gid":6471,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6834/432456394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":432456394},{"type":"v1","addr":"172.21.15.143:6834","nonce":432456394}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37},{"gid":6474,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/310920972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":310920972},{"type":"v1","addr":"172.21.15.143:6835","nonce":310920972}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:27:31.702539+0000","modified":"2023-11-08T19:27:40.808769+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6489},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6489":{"gid":6489,"name":"d","rank":0,"incarnation":38,"state":"up:active","state_seq":3,"addr":"172.21.15.174:6837/1950356087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6835","nonce":1950356087},{"type":"v1","addr":"172.21.15.174:6837","nonce":1950356087}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-08T19:27:48.553 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.551+0000 7f2206f0d700 1 -- 172.21.15.143:0/1363321884 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f21e406af00 msgr2=0x7f21e406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:48.553 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.551+0000 7f2206f0d700 1 --2- 172.21.15.143:0/1363321884 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f21e406af00 0x7f21e406d3c0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f21f40090b0 tx=0x7f21f4008030 comp rx=0 tx=0).stop 2023-11-08T19:27:48.553 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.551+0000 7f2206f0d700 1 -- 172.21.15.143:0/1363321884 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f220006df90 msgr2=0x7f22000b3490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:48.553 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.551+0000 7f2206f0d700 1 --2- 172.21.15.143:0/1363321884 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f220006df90 0x7f22000b3490 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f21f800cba0 tx=0x7f21f800ceb0 comp rx=0 tx=0).stop 2023-11-08T19:27:48.554 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.551+0000 7f2206f0d700 1 -- 172.21.15.143:0/1363321884 shutdown_connections 2023-11-08T19:27:48.554 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.551+0000 7f2206f0d700 1 --2- 172.21.15.143:0/1363321884 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f21e406af00 0x7f21e406d3c0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:48.554 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.551+0000 7f2206f0d700 1 --2- 172.21.15.143:0/1363321884 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f220006df90 0x7f22000b3490 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:48.554 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.551+0000 7f2206f0d700 1 --2- 172.21.15.143:0/1363321884 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f22000668e0 0x7f220015da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:48.554 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.551+0000 7f2206f0d700 1 --2- 172.21.15.143:0/1363321884 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f220006dbb0 0x7f22001585e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:48.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.551+0000 7f2206f0d700 1 -- 172.21.15.143:0/1363321884 >> 172.21.15.143:0/1363321884 conn(0x7f22000b1a20 msgr2=0x7f22001617e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:48.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.551+0000 7f2206f0d700 1 -- 172.21.15.143:0/1363321884 shutdown_connections 2023-11-08T19:27:48.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.551+0000 7f2206f0d700 1 -- 172.21.15.143:0/1363321884 wait complete. 2023-11-08T19:27:48.555 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 43 2023-11-08T19:27:48.565 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:27:48.687 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.685+0000 7f0b19dae700 1 Processor -- start 2023-11-08T19:27:48.687 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.685+0000 7f0b19dae700 1 -- start start 2023-11-08T19:27:48.689 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.686+0000 7f0b19dae700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f0b1415dd00 0x7f0b140cca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:48.689 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.686+0000 7f0b19dae700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b140ccfb0 con 0x7f0b1406b320 2023-11-08T19:27:48.689 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.686+0000 7f0b19dae700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b14161df0 con 0x7f0b141618f0 2023-11-08T19:27:48.690 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.686+0000 7f0b19dae700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b140cad50 con 0x7f0b1415dd00 2023-11-08T19:27:48.690 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.686+0000 7f0b137fe700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f0b1415dd00 0x7f0b140cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:48.690 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.686+0000 7f0b137fe700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f0b1415dd00 0x7f0b140cca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54120/0 (socket says 172.21.15.143:54120) 2023-11-08T19:27:48.690 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.686+0000 7f0b137fe700 1 -- 172.21.15.143:0/3470641609 learned_addr learned my addr 172.21.15.143:0/3470641609 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:48.690 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.686+0000 7f0b13fff700 1 --1- 172.21.15.143:0/3470641609 >> v1:172.21.15.143:6789/0 conn(0x7f0b1406b320 0x7f0b140cb1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:50278/0 (socket says 172.21.15.143:50278) 2023-11-08T19:27:48.691 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.687+0000 7f0b137fe700 1 -- 172.21.15.143:0/3470641609 >> v1:172.21.15.174:6789/0 conn(0x7f0b141618f0 legacy=0x7f0b1406ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:48.691 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.687+0000 7f0b137fe700 1 -- 172.21.15.143:0/3470641609 >> v1:172.21.15.143:6789/0 conn(0x7f0b1406b320 legacy=0x7f0b140cb1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:48.691 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.687+0000 7f0b127fc700 1 -- 172.21.15.143:0/3470641609 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1315871943 0 0) 0x7f0b140ccfb0 con 0x7f0b1406b320 2023-11-08T19:27:48.691 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.687+0000 7f0b137fe700 1 -- 172.21.15.143:0/3470641609 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b141620d0 con 0x7f0b1415dd00 2023-11-08T19:27:48.691 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.687+0000 7f0b137fe700 1 --2- 172.21.15.143:0/3470641609 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0b1415dd00 0x7f0b140cca70 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f0b08006fd0 tx=0x7f0b08009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6ac35192cf6d290 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:48.692 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.687+0000 7f0b127fc700 1 -- 172.21.15.143:0/3470641609 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b08005e30 con 0x7f0b1415dd00 2023-11-08T19:27:48.692 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.687+0000 7f0b127fc700 1 -- 172.21.15.143:0/3470641609 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b08004030 con 0x7f0b1415dd00 2023-11-08T19:27:48.692 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.687+0000 7f0b127fc700 1 -- 172.21.15.143:0/3470641609 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b080043e0 con 0x7f0b1415dd00 2023-11-08T19:27:48.692 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.687+0000 7f0b19dae700 1 -- 172.21.15.143:0/3470641609 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0b1415dd00 msgr2=0x7f0b140cca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:48.692 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.687+0000 7f0b19dae700 1 --2- 172.21.15.143:0/3470641609 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0b1415dd00 0x7f0b140cca70 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f0b08006fd0 tx=0x7f0b08009bd0 comp rx=0 tx=0).stop 2023-11-08T19:27:48.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.687+0000 7f0b19dae700 1 -- 172.21.15.143:0/3470641609 shutdown_connections 2023-11-08T19:27:48.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.687+0000 7f0b19dae700 1 --2- 172.21.15.143:0/3470641609 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0b1415dd00 0x7f0b140cca70 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:48.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.687+0000 7f0b19dae700 1 -- 172.21.15.143:0/3470641609 >> 172.21.15.143:0/3470641609 conn(0x7f0b14099ab0 msgr2=0x7f0b140d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:48.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.687+0000 7f0b19dae700 1 -- 172.21.15.143:0/3470641609 shutdown_connections 2023-11-08T19:27:48.694 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.688+0000 7f0b19dae700 1 -- 172.21.15.143:0/3470641609 wait complete. 2023-11-08T19:27:48.697 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.689+0000 7f0b19dae700 1 Processor -- start 2023-11-08T19:27:48.697 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.689+0000 7f0b19dae700 1 -- start start 2023-11-08T19:27:48.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.690+0000 7f0b19dae700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0b1406b320 0x7f0b141608d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:48.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.690+0000 7f0b19dae700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0b1415dd00 0x7f0b14160e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:48.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.690+0000 7f0b19dae700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0b141618f0 0x7f0b141f6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:48.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.690+0000 7f0b19dae700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b141f7f30 con 0x7f0b1406b320 2023-11-08T19:27:48.699 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.690+0000 7f0b19dae700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b141f7c50 con 0x7f0b1415dd00 2023-11-08T19:27:48.699 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.690+0000 7f0b19dae700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b140b6110 con 0x7f0b141618f0 2023-11-08T19:27:48.699 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.690+0000 7f0b137fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0b1406b320 0x7f0b141608d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:48.699 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.690+0000 7f0b137fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0b1406b320 0x7f0b141608d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54126/0 (socket says 172.21.15.143:54126) 2023-11-08T19:27:48.699 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.690+0000 7f0b137fe700 1 -- 172.21.15.143:0/4086009153 learned_addr learned my addr 172.21.15.143:0/4086009153 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:48.699 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.690+0000 7f0b12ffd700 1 --2- 172.21.15.143:0/4086009153 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0b1415dd00 0x7f0b14160e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:48.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.690+0000 7f0b13fff700 1 --2- 172.21.15.143:0/4086009153 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0b141618f0 0x7f0b141f6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:48.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.690+0000 7f0b137fe700 1 -- 172.21.15.143:0/4086009153 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0b141618f0 msgr2=0x7f0b141f6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:48.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.690+0000 7f0b137fe700 1 --2- 172.21.15.143:0/4086009153 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0b141618f0 0x7f0b141f6ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:48.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.690+0000 7f0b137fe700 1 -- 172.21.15.143:0/4086009153 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0b1415dd00 msgr2=0x7f0b14160e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:48.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.690+0000 7f0b137fe700 1 --2- 172.21.15.143:0/4086009153 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0b1415dd00 0x7f0b14160e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:48.701 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.690+0000 7f0b137fe700 1 -- 172.21.15.143:0/4086009153 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b08009500 con 0x7f0b1406b320 2023-11-08T19:27:48.701 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.690+0000 7f0b12ffd700 1 --2- 172.21.15.143:0/4086009153 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0b1415dd00 0x7f0b14160e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:27:48.701 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.690+0000 7f0b137fe700 1 --2- 172.21.15.143:0/4086009153 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0b1406b320 0x7f0b141608d0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f0b14070a90 tx=0x7f0b08004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:48.701 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.690+0000 7f0afbfff700 1 -- 172.21.15.143:0/4086009153 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b08003e90 con 0x7f0b1406b320 2023-11-08T19:27:48.701 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.690+0000 7f0afbfff700 1 -- 172.21.15.143:0/4086009153 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b08011420 con 0x7f0b1406b320 2023-11-08T19:27:48.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.690+0000 7f0afbfff700 1 -- 172.21.15.143:0/4086009153 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b08011790 con 0x7f0b1406b320 2023-11-08T19:27:48.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.690+0000 7f0b19dae700 1 -- 172.21.15.143:0/4086009153 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b140b0ba0 con 0x7f0b1406b320 2023-11-08T19:27:48.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.690+0000 7f0b19dae700 1 -- 172.21.15.143:0/4086009153 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b140b10c0 con 0x7f0b1406b320 2023-11-08T19:27:48.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.691+0000 7f0b19dae700 1 -- 172.21.15.143:0/4086009153 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b140b12b0 con 0x7f0b1406b320 2023-11-08T19:27:48.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.692+0000 7f0afbfff700 1 -- 172.21.15.143:0/4086009153 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0b080118f0 con 0x7f0b1406b320 2023-11-08T19:27:48.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.692+0000 7f0afbfff700 1 --2- 172.21.15.143:0/4086009153 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f0b0006af00 0x7f0b0006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:48.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.692+0000 7f0afbfff700 1 -- 172.21.15.143:0/4086009153 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6944+0+0 (secure 0 0 0) 0x7f0b0808ab50 con 0x7f0b1406b320 2023-11-08T19:27:48.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.692+0000 7f0b12ffd700 1 --2- 172.21.15.143:0/4086009153 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f0b0006af00 0x7f0b0006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:48.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.693+0000 7f0b12ffd700 1 --2- 172.21.15.143:0/4086009153 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f0b0006af00 0x7f0b0006d3c0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f0b1406f3f0 tx=0x7f0b04009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:48.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.697+0000 7f0afbfff700 1 -- 172.21.15.143:0/4086009153 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0b080557c0 con 0x7f0b1406b320 2023-11-08T19:27:48.926 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.925+0000 7f0b19dae700 1 -- 172.21.15.143:0/4086009153 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0b140b5f70 con 0x7f0b1406b320 2023-11-08T19:27:48.927 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.925+0000 7f0afbfff700 1 -- 172.21.15.143:0/4086009153 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+4921 (secure 0 0 0) 0x7f0b140b5f70 con 0x7f0b1406b320 2023-11-08T19:27:48.927 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:27:48.927 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":43,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5510,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6836/3796015890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3796015890},{"type":"v1","addr":"172.21.15.174:6836","nonce":3796015890}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37},{"gid":6471,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6834/432456394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":432456394},{"type":"v1","addr":"172.21.15.143:6834","nonce":432456394}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37},{"gid":6474,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/310920972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":310920972},{"type":"v1","addr":"172.21.15.143:6835","nonce":310920972}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:27:31.702539+0000","modified":"2023-11-08T19:27:40.808769+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6489},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6489":{"gid":6489,"name":"d","rank":0,"incarnation":38,"state":"up:active","state_seq":3,"addr":"172.21.15.174:6837/1950356087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6835","nonce":1950356087},{"type":"v1","addr":"172.21.15.174:6837","nonce":1950356087}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-08T19:27:48.929 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.928+0000 7f0b19dae700 1 -- 172.21.15.143:0/4086009153 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f0b0006af00 msgr2=0x7f0b0006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:48.929 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.928+0000 7f0b19dae700 1 --2- 172.21.15.143:0/4086009153 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f0b0006af00 0x7f0b0006d3c0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f0b1406f3f0 tx=0x7f0b04009040 comp rx=0 tx=0).stop 2023-11-08T19:27:48.930 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.928+0000 7f0b19dae700 1 -- 172.21.15.143:0/4086009153 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0b1406b320 msgr2=0x7f0b141608d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:48.930 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.928+0000 7f0b19dae700 1 --2- 172.21.15.143:0/4086009153 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0b1406b320 0x7f0b141608d0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f0b14070a90 tx=0x7f0b08004750 comp rx=0 tx=0).stop 2023-11-08T19:27:48.930 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.928+0000 7f0b19dae700 1 -- 172.21.15.143:0/4086009153 shutdown_connections 2023-11-08T19:27:48.930 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.928+0000 7f0b19dae700 1 --2- 172.21.15.143:0/4086009153 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f0b0006af00 0x7f0b0006d3c0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:48.931 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.928+0000 7f0b19dae700 1 --2- 172.21.15.143:0/4086009153 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0b141618f0 0x7f0b141f6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:48.931 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.928+0000 7f0b19dae700 1 --2- 172.21.15.143:0/4086009153 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0b1406b320 0x7f0b141608d0 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:48.931 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.928+0000 7f0b19dae700 1 --2- 172.21.15.143:0/4086009153 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0b1415dd00 0x7f0b14160e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:48.931 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.928+0000 7f0b19dae700 1 -- 172.21.15.143:0/4086009153 >> 172.21.15.143:0/4086009153 conn(0x7f0b14099ab0 msgr2=0x7f0b1406f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:48.931 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.928+0000 7f0b19dae700 1 -- 172.21.15.143:0/4086009153 shutdown_connections 2023-11-08T19:27:48.931 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:48.928+0000 7f0b19dae700 1 -- 172.21.15.143:0/4086009153 wait complete. 2023-11-08T19:27:48.932 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 43 2023-11-08T19:27:48.941 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:27:49.062 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.060+0000 7f1bb7f01700 1 Processor -- start 2023-11-08T19:27:49.062 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.061+0000 7f1bb7f01700 1 -- start start 2023-11-08T19:27:49.064 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.061+0000 7f1bb7f01700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f1bb00d1c70 0x7f1bb00b14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:49.065 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.061+0000 7f1bb7f01700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1bb00d03a0 0x7f1bb00cc5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:49.065 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.061+0000 7f1bb7f01700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1bb00b1bc0 con 0x7f1bb015ba30 2023-11-08T19:27:49.065 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.061+0000 7f1bb7f01700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bb00b1ea0 con 0x7f1bb00d1c70 2023-11-08T19:27:49.065 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.061+0000 7f1bb7f01700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb00b2180 con 0x7f1bb00d03a0 2023-11-08T19:27:49.065 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.062+0000 7f1bb649e700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1bb00d03a0 0x7f1bb00cc5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:49.066 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.062+0000 7f1bb549c700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f1bb00d1c70 0x7f1bb00b14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:49.066 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.062+0000 7f1bb649e700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1bb00d03a0 0x7f1bb00cc5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.143:41776/0 (socket says 172.21.15.143:41776) 2023-11-08T19:27:49.066 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.062+0000 7f1bb649e700 1 -- 172.21.15.143:0/1506360704 learned_addr learned my addr 172.21.15.143:0/1506360704 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:49.066 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.062+0000 7f1bb649e700 1 -- 172.21.15.143:0/1506360704 >> v2:172.21.15.174:3300/0 conn(0x7f1bb00d1c70 msgr2=0x7f1bb00b14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:49.066 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.062+0000 7f1bb649e700 1 --2- 172.21.15.143:0/1506360704 >> v2:172.21.15.174:3300/0 conn(0x7f1bb00d1c70 0x7f1bb00b14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:49.067 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.062+0000 7f1bb649e700 1 -- 172.21.15.143:0/1506360704 >> v1:172.21.15.174:6789/0 conn(0x7f1bb015ba30 legacy=0x7f1bb00d1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:49.067 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.062+0000 7f1bb649e700 1 -- 172.21.15.143:0/1506360704 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bb00b2460 con 0x7f1bb00d03a0 2023-11-08T19:27:49.067 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.062+0000 7f1bb549c700 1 --2- 172.21.15.143:0/1506360704 >> v2:172.21.15.174:3300/0 conn(0x7f1bb00d1c70 0x7f1bb00b14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T19:27:49.067 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.062+0000 7f1bb649e700 1 --2- 172.21.15.143:0/1506360704 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1bb00d03a0 0x7f1bb00cc5a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1ba8009e50 tx=0x7f1ba800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e52d3840e2df015b server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:49.068 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.062+0000 7f1bb4c9b700 1 -- 172.21.15.143:0/1506360704 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ba800e070 con 0x7f1bb00d03a0 2023-11-08T19:27:49.068 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.062+0000 7f1bb4c9b700 1 -- 172.21.15.143:0/1506360704 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ba800b910 con 0x7f1bb00d03a0 2023-11-08T19:27:49.068 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.062+0000 7f1bb4c9b700 1 -- 172.21.15.143:0/1506360704 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ba800bcc0 con 0x7f1bb00d03a0 2023-11-08T19:27:49.068 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.063+0000 7f1bb7f01700 1 -- 172.21.15.143:0/1506360704 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1bb00d03a0 msgr2=0x7f1bb00cc5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:49.068 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.063+0000 7f1bb7f01700 1 --2- 172.21.15.143:0/1506360704 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1bb00d03a0 0x7f1bb00cc5a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1ba8009e50 tx=0x7f1ba800b330 comp rx=0 tx=0).stop 2023-11-08T19:27:49.069 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.063+0000 7f1bb7f01700 1 -- 172.21.15.143:0/1506360704 shutdown_connections 2023-11-08T19:27:49.069 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.063+0000 7f1bb7f01700 1 --2- 172.21.15.143:0/1506360704 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1bb00d03a0 0x7f1bb00cc5a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:49.069 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.063+0000 7f1bb7f01700 1 --2- 172.21.15.143:0/1506360704 >> v2:172.21.15.174:3300/0 conn(0x7f1bb00d1c70 0x7f1bb00b14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:49.069 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.063+0000 7f1bb7f01700 1 -- 172.21.15.143:0/1506360704 >> 172.21.15.143:0/1506360704 conn(0x7f1bb008b000 msgr2=0x7f1bb009ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:49.069 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.063+0000 7f1bb7f01700 1 -- 172.21.15.143:0/1506360704 shutdown_connections 2023-11-08T19:27:49.070 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.063+0000 7f1bb7f01700 1 -- 172.21.15.143:0/1506360704 wait complete. 2023-11-08T19:27:49.072 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.064+0000 7f1bb7f01700 1 Processor -- start 2023-11-08T19:27:49.073 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.064+0000 7f1bb7f01700 1 -- start start 2023-11-08T19:27:49.073 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.065+0000 7f1bb7f01700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1bb00d03a0 0x7f1bb01571f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:49.073 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.065+0000 7f1bb7f01700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1bb00d1c70 0x7f1bb00ce680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:49.073 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.065+0000 7f1bb7f01700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1bb015ba30 0x7f1bb01e7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:49.073 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.065+0000 7f1bb7f01700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb00ccac0 con 0x7f1bb00d1c70 2023-11-08T19:27:49.074 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.065+0000 7f1bb7f01700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb00cd360 con 0x7f1bb00d03a0 2023-11-08T19:27:49.074 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.065+0000 7f1bb7f01700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb0157730 con 0x7f1bb015ba30 2023-11-08T19:27:49.074 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.065+0000 7f1bb549c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1bb00d1c70 0x7f1bb00ce680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:49.074 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.065+0000 7f1bb549c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1bb00d1c70 0x7f1bb00ce680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54144/0 (socket says 172.21.15.143:54144) 2023-11-08T19:27:49.074 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.065+0000 7f1bb549c700 1 -- 172.21.15.143:0/1049385046 learned_addr learned my addr 172.21.15.143:0/1049385046 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:49.075 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.065+0000 7f1bb5c9d700 1 --2- 172.21.15.143:0/1049385046 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1bb00d03a0 0x7f1bb01571f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:49.075 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.065+0000 7f1bb649e700 1 --2- 172.21.15.143:0/1049385046 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1bb015ba30 0x7f1bb01e7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:49.075 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.066+0000 7f1bb549c700 1 -- 172.21.15.143:0/1049385046 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1bb015ba30 msgr2=0x7f1bb01e7040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:49.075 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.066+0000 7f1bb549c700 1 --2- 172.21.15.143:0/1049385046 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1bb015ba30 0x7f1bb01e7040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:49.075 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.066+0000 7f1bb549c700 1 -- 172.21.15.143:0/1049385046 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1bb00d03a0 msgr2=0x7f1bb01571f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:49.076 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.066+0000 7f1bb549c700 1 --2- 172.21.15.143:0/1049385046 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1bb00d03a0 0x7f1bb01571f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:49.076 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.066+0000 7f1bb549c700 1 -- 172.21.15.143:0/1049385046 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bac0094a0 con 0x7f1bb00d1c70 2023-11-08T19:27:49.076 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.066+0000 7f1bb549c700 1 --2- 172.21.15.143:0/1049385046 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1bb00d1c70 0x7f1bb00ce680 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f1bac00ebe0 tx=0x7f1bac009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:49.076 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.066+0000 7f1ba67fc700 1 -- 172.21.15.143:0/1049385046 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bac00c910 con 0x7f1bb00d1c70 2023-11-08T19:27:49.076 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.066+0000 7f1ba67fc700 1 -- 172.21.15.143:0/1049385046 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bac00ca70 con 0x7f1bb00d1c70 2023-11-08T19:27:49.076 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.066+0000 7f1ba67fc700 1 -- 172.21.15.143:0/1049385046 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bac00cde0 con 0x7f1bb00d1c70 2023-11-08T19:27:49.077 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.066+0000 7f1bb7f01700 1 -- 172.21.15.143:0/1049385046 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ba8009b30 con 0x7f1bb00d1c70 2023-11-08T19:27:49.077 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.066+0000 7f1bb7f01700 1 -- 172.21.15.143:0/1049385046 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bb00ccd10 con 0x7f1bb00d1c70 2023-11-08T19:27:49.077 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.067+0000 7f1bb7f01700 1 -- 172.21.15.143:0/1049385046 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bb00cd4d0 con 0x7f1bb00d1c70 2023-11-08T19:27:49.077 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.067+0000 7f1ba67fc700 1 -- 172.21.15.143:0/1049385046 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1bac013470 con 0x7f1bb00d1c70 2023-11-08T19:27:49.077 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.068+0000 7f1ba67fc700 1 --2- 172.21.15.143:0/1049385046 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f1b9406af00 0x7f1b9406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:49.078 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.068+0000 7f1ba67fc700 1 -- 172.21.15.143:0/1049385046 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6944+0+0 (secure 0 0 0) 0x7f1bac089f80 con 0x7f1bb00d1c70 2023-11-08T19:27:49.078 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.068+0000 7f1bb5c9d700 1 --2- 172.21.15.143:0/1049385046 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f1b9406af00 0x7f1b9406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:49.078 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.068+0000 7f1bb5c9d700 1 --2- 172.21.15.143:0/1049385046 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f1b9406af00 0x7f1b9406d3c0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f1bb01e7c00 tx=0x7f1ba0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:49.079 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.072+0000 7f1ba67fc700 1 -- 172.21.15.143:0/1049385046 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1bac0bcca0 con 0x7f1bb00d1c70 2023-11-08T19:27:49.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.259+0000 7f1bb7f01700 1 -- 172.21.15.143:0/1049385046 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1bb004eee0 con 0x7f1bb00d1c70 2023-11-08T19:27:49.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.261+0000 7f1ba67fc700 1 -- 172.21.15.143:0/1049385046 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+22107 (secure 0 0 0) 0x7f1bac017090 con 0x7f1bb00d1c70 2023-11-08T19:27:49.262 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:27:49.263 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":81,"fsid":"968483af-77fc-4068-ad7c-5c2a1a4613d7","created":"2023-11-08T19:24:32.826484+0000","modified":"2023-11-08T19:27:46.378783+0000","last_up_change":"2023-11-08T19:24:43.573880+0000","last_in_change":"2023-11-08T19:24:35.314525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:44.551530+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"snap-test-pool","create_time":"2023-11-08T19:26:02.874384+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":36,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"test_data_pool","create_time":"2023-11-08T19:27:05.601949+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":66,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_metadata_pool","create_time":"2023-11-08T19:27:06.536369+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-11-08T19:27:30.137037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-11-08T19:27:30.660318+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":81,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bae98ec1-d94e-403d-a220-a522a53ea6f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6809","nonce":1487830566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6811","nonce":1487830566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6815","nonce":1487830566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6813","nonce":1487830566}]},"public_addr":"172.21.15.143:6809/1487830566","cluster_addr":"172.21.15.143:6811/1487830566","heartbeat_back_addr":"172.21.15.143:6815/1487830566","heartbeat_front_addr":"172.21.15.143:6813/1487830566","state":["exists","up"]},{"osd":1,"uuid":"5cb6538a-1ccb-4322-8633-09f66af0b7e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6801","nonce":787341397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6803","nonce":787341397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6807","nonce":787341397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6805","nonce":787341397}]},"public_addr":"172.21.15.143:6801/787341397","cluster_addr":"172.21.15.143:6803/787341397","heartbeat_back_addr":"172.21.15.143:6807/787341397","heartbeat_front_addr":"172.21.15.143:6805/787341397","state":["exists","up"]},{"osd":2,"uuid":"cdbf522c-60ef-4d60-9653-8dcb529c101a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6822","nonce":2301325597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6826","nonce":2301325597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6831","nonce":2301325597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6829","nonce":2301325597}]},"public_addr":"172.21.15.143:6822/2301325597","cluster_addr":"172.21.15.143:6826/2301325597","heartbeat_back_addr":"172.21.15.143:6831/2301325597","heartbeat_front_addr":"172.21.15.143:6829/2301325597","state":["exists","up"]},{"osd":3,"uuid":"4e2783cf-ebb9-4dd8-8245-bc1d17ce4325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6817","nonce":2021504428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6819","nonce":2021504428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6827","nonce":2021504428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6823","nonce":2021504428}]},"public_addr":"172.21.15.143:6817/2021504428","cluster_addr":"172.21.15.143:6819/2021504428","heartbeat_back_addr":"172.21.15.143:6827/2021504428","heartbeat_front_addr":"172.21.15.143:6823/2021504428","state":["exists","up"]},{"osd":4,"uuid":"691256fa-9b77-49b9-a346-7ae1adf0ca80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6801","nonce":1993380595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6803","nonce":1993380595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6807","nonce":1993380595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6805","nonce":1993380595}]},"public_addr":"172.21.15.174:6801/1993380595","cluster_addr":"172.21.15.174:6803/1993380595","heartbeat_back_addr":"172.21.15.174:6807/1993380595","heartbeat_front_addr":"172.21.15.174:6805/1993380595","state":["exists","up"]},{"osd":5,"uuid":"756fc352-48c9-46ce-b52e-0541fe6f8525","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6809","nonce":1777923303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6811","nonce":1777923303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6815","nonce":1777923303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6813","nonce":1777923303}]},"public_addr":"172.21.15.174:6809/1777923303","cluster_addr":"172.21.15.174:6811/1777923303","heartbeat_back_addr":"172.21.15.174:6815/1777923303","heartbeat_front_addr":"172.21.15.174:6813/1777923303","state":["exists","up"]},{"osd":6,"uuid":"fab2ac2b-ed97-4cde-9825-b480eabf8b8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6825","nonce":2755130316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6827","nonce":2755130316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6831","nonce":2755130316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6829","nonce":2755130316}]},"public_addr":"172.21.15.174:6825/2755130316","cluster_addr":"172.21.15.174:6827/2755130316","heartbeat_back_addr":"172.21.15.174:6831/2755130316","heartbeat_front_addr":"172.21.15.174:6829/2755130316","state":["exists","up"]},{"osd":7,"uuid":"3de9b797-2ab6-4d41-a863-7e7d638fd703","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6817","nonce":2847141411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6819","nonce":2847141411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6823","nonce":2847141411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6821","nonce":2847141411}]},"public_addr":"172.21.15.174:6817/2847141411","cluster_addr":"172.21.15.174:6819/2847141411","heartbeat_back_addr":"172.21.15.174:6823/2847141411","heartbeat_front_addr":"172.21.15.174:6821/2847141411","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.477076+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.496890+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.539305+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.454673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.538881+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.521294+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.520712+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.540404+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.a","mappings":[{"from":1,"to":7}]},{"pgid":"14.17","mappings":[{"from":1,"to":7}]},{"pgid":"14.29","mappings":[{"from":1,"to":2}]},{"pgid":"14.32","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":14,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":14,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:27:49.264 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.263+0000 7f1bb7f01700 1 -- 172.21.15.143:0/1049385046 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f1b9406af00 msgr2=0x7f1b9406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:49.265 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.263+0000 7f1bb7f01700 1 --2- 172.21.15.143:0/1049385046 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f1b9406af00 0x7f1b9406d3c0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f1bb01e7c00 tx=0x7f1ba0009040 comp rx=0 tx=0).stop 2023-11-08T19:27:49.265 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.263+0000 7f1bb7f01700 1 -- 172.21.15.143:0/1049385046 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1bb00d1c70 msgr2=0x7f1bb00ce680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:49.265 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.263+0000 7f1bb7f01700 1 --2- 172.21.15.143:0/1049385046 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1bb00d1c70 0x7f1bb00ce680 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f1bac00ebe0 tx=0x7f1bac009ec0 comp rx=0 tx=0).stop 2023-11-08T19:27:49.265 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.263+0000 7f1bb7f01700 1 -- 172.21.15.143:0/1049385046 shutdown_connections 2023-11-08T19:27:49.265 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.263+0000 7f1bb7f01700 1 --2- 172.21.15.143:0/1049385046 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f1b9406af00 0x7f1b9406d3c0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:49.266 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.263+0000 7f1bb7f01700 1 --2- 172.21.15.143:0/1049385046 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1bb015ba30 0x7f1bb01e7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:49.266 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.263+0000 7f1bb7f01700 1 --2- 172.21.15.143:0/1049385046 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1bb00d03a0 0x7f1bb01571f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:49.266 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.263+0000 7f1bb7f01700 1 --2- 172.21.15.143:0/1049385046 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1bb00d1c70 0x7f1bb00ce680 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:49.266 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.263+0000 7f1bb7f01700 1 -- 172.21.15.143:0/1049385046 >> 172.21.15.143:0/1049385046 conn(0x7f1bb008b000 msgr2=0x7f1bb00d1540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:49.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.263+0000 7f1bb7f01700 1 -- 172.21.15.143:0/1049385046 shutdown_connections 2023-11-08T19:27:49.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.263+0000 7f1bb7f01700 1 -- 172.21.15.143:0/1049385046 wait complete. 2023-11-08T19:27:49.277 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-08T19:27:49.397 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.396+0000 7f4b5c968700 1 Processor -- start 2023-11-08T19:27:49.398 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.396+0000 7f4b5c968700 1 -- start start 2023-11-08T19:27:49.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.397+0000 7f4b5c968700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f4b5815dd00 0x7f4b580cca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:49.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.397+0000 7f4b5c968700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f4b581618f0 0x7f4b5806ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:49.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.397+0000 7f4b5c968700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b580ccfb0 con 0x7f4b5806b150 2023-11-08T19:27:49.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.397+0000 7f4b5c968700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b5806b600 con 0x7f4b581618f0 2023-11-08T19:27:49.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.397+0000 7f4b5c968700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b580cad50 con 0x7f4b5815dd00 2023-11-08T19:27:49.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.397+0000 7f4b55d9b700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f4b581618f0 0x7f4b5806ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:49.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.397+0000 7f4b55d9b700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f4b581618f0 0x7f4b5806ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54150/0 (socket says 172.21.15.143:54150) 2023-11-08T19:27:49.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.397+0000 7f4b55d9b700 1 -- 172.21.15.143:0/2485122099 learned_addr learned my addr 172.21.15.143:0/2485122099 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:49.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.397+0000 7f4b56d9d700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f4b5806b150 0x7f4b580b2210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:50306/0 (socket says 172.21.15.143:50306) 2023-11-08T19:27:49.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.397+0000 7f4b5659c700 1 --2- 172.21.15.143:0/2485122099 >> v2:172.21.15.174:3300/0 conn(0x7f4b5815dd00 0x7f4b580cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:49.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.398+0000 7f4b55d9b700 1 -- 172.21.15.143:0/2485122099 >> v2:172.21.15.174:3300/0 conn(0x7f4b5815dd00 msgr2=0x7f4b580cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:49.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.398+0000 7f4b55d9b700 1 --2- 172.21.15.143:0/2485122099 >> v2:172.21.15.174:3300/0 conn(0x7f4b5815dd00 0x7f4b580cca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:49.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.398+0000 7f4b55d9b700 1 -- 172.21.15.143:0/2485122099 >> v1:172.21.15.143:6789/0 conn(0x7f4b5806b150 legacy=0x7f4b580b2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:49.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.398+0000 7f4b55d9b700 1 -- 172.21.15.143:0/2485122099 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b58161df0 con 0x7f4b581618f0 2023-11-08T19:27:49.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.398+0000 7f4b55d9b700 1 --2- 172.21.15.143:0/2485122099 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4b581618f0 0x7f4b5806ac10 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f4b58070a60 tx=0x7f4b4c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d83cff30383b03d1 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:49.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.398+0000 7f4b5559a700 1 -- 172.21.15.143:0/2485122099 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b4c011830 con 0x7f4b581618f0 2023-11-08T19:27:49.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.398+0000 7f4b5559a700 1 -- 172.21.15.143:0/2485122099 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b4c011990 con 0x7f4b581618f0 2023-11-08T19:27:49.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.398+0000 7f4b5559a700 1 -- 172.21.15.143:0/2485122099 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b4c011d20 con 0x7f4b581618f0 2023-11-08T19:27:49.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.399+0000 7f4b5c968700 1 -- 172.21.15.143:0/2485122099 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4b581618f0 msgr2=0x7f4b5806ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:49.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.399+0000 7f4b5c968700 1 --2- 172.21.15.143:0/2485122099 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4b581618f0 0x7f4b5806ac10 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f4b58070a60 tx=0x7f4b4c009bd0 comp rx=0 tx=0).stop 2023-11-08T19:27:49.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.399+0000 7f4b5c968700 1 -- 172.21.15.143:0/2485122099 shutdown_connections 2023-11-08T19:27:49.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.399+0000 7f4b5c968700 1 --2- 172.21.15.143:0/2485122099 >> v2:172.21.15.174:3300/0 conn(0x7f4b5815dd00 0x7f4b580cca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:49.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.399+0000 7f4b5c968700 1 --2- 172.21.15.143:0/2485122099 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4b581618f0 0x7f4b5806ac10 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:49.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.399+0000 7f4b5c968700 1 -- 172.21.15.143:0/2485122099 >> 172.21.15.143:0/2485122099 conn(0x7f4b58099ab0 msgr2=0x7f4b580d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:49.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.399+0000 7f4b5c968700 1 -- 172.21.15.143:0/2485122099 shutdown_connections 2023-11-08T19:27:49.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.399+0000 7f4b5c968700 1 -- 172.21.15.143:0/2485122099 wait complete. 2023-11-08T19:27:49.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.400+0000 7f4b5c968700 1 Processor -- start 2023-11-08T19:27:49.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.400+0000 7f4b5c968700 1 -- start start 2023-11-08T19:27:49.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.401+0000 7f4b5c968700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4b5806b150 0x7f4b58151e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:49.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.401+0000 7f4b5c968700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4b5815dd00 0x7f4b58147520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:49.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.401+0000 7f4b5c968700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4b581618f0 0x7f4b580c4230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:49.412 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.401+0000 7f4b5c968700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b580b62a0 con 0x7f4b5806b150 2023-11-08T19:27:49.412 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.401+0000 7f4b5c968700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b580b2a30 con 0x7f4b581618f0 2023-11-08T19:27:49.412 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.401+0000 7f4b5c968700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b580c52e0 con 0x7f4b5815dd00 2023-11-08T19:27:49.412 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.401+0000 7f4b5659c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4b5806b150 0x7f4b58151e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:49.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.401+0000 7f4b5659c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4b5806b150 0x7f4b58151e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54154/0 (socket says 172.21.15.143:54154) 2023-11-08T19:27:49.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.401+0000 7f4b5659c700 1 -- 172.21.15.143:0/1477305445 learned_addr learned my addr 172.21.15.143:0/1477305445 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:49.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.401+0000 7f4b56d9d700 1 --2- 172.21.15.143:0/1477305445 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4b581618f0 0x7f4b580c4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:49.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.401+0000 7f4b55d9b700 1 --2- 172.21.15.143:0/1477305445 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4b5815dd00 0x7f4b58147520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:49.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.402+0000 7f4b5659c700 1 -- 172.21.15.143:0/1477305445 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4b5815dd00 msgr2=0x7f4b58147520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:49.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.402+0000 7f4b5659c700 1 --2- 172.21.15.143:0/1477305445 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4b5815dd00 0x7f4b58147520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:49.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.402+0000 7f4b5659c700 1 -- 172.21.15.143:0/1477305445 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4b581618f0 msgr2=0x7f4b580c4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:49.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.402+0000 7f4b5659c700 1 --2- 172.21.15.143:0/1477305445 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4b581618f0 0x7f4b580c4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:49.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.402+0000 7f4b5659c700 1 -- 172.21.15.143:0/1477305445 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b4c009550 con 0x7f4b5806b150 2023-11-08T19:27:49.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.402+0000 7f4b55d9b700 1 --2- 172.21.15.143:0/1477305445 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4b5815dd00 0x7f4b58147520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:27:49.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.402+0000 7f4b5659c700 1 --2- 172.21.15.143:0/1477305445 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4b5806b150 0x7f4b58151e00 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f4b4800e990 tx=0x7f4b4800eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:49.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.402+0000 7f4b46ffd700 1 -- 172.21.15.143:0/1477305445 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b480096f0 con 0x7f4b5806b150 2023-11-08T19:27:49.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.402+0000 7f4b46ffd700 1 -- 172.21.15.143:0/1477305445 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b4800c850 con 0x7f4b5806b150 2023-11-08T19:27:49.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.402+0000 7f4b5c968700 1 -- 172.21.15.143:0/1477305445 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b580c55c0 con 0x7f4b5806b150 2023-11-08T19:27:49.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.402+0000 7f4b46ffd700 1 -- 172.21.15.143:0/1477305445 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b4800eee0 con 0x7f4b5806b150 2023-11-08T19:27:49.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.402+0000 7f4b5c968700 1 -- 172.21.15.143:0/1477305445 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b580c5b80 con 0x7f4b5806b150 2023-11-08T19:27:49.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.403+0000 7f4b5c968700 1 -- 172.21.15.143:0/1477305445 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b580aff60 con 0x7f4b5806b150 2023-11-08T19:27:49.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.403+0000 7f4b46ffd700 1 -- 172.21.15.143:0/1477305445 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4b4800cbc0 con 0x7f4b5806b150 2023-11-08T19:27:49.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.404+0000 7f4b46ffd700 1 --2- 172.21.15.143:0/1477305445 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f4b3806af00 0x7f4b3806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:49.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.404+0000 7f4b46ffd700 1 -- 172.21.15.143:0/1477305445 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6944+0+0 (secure 0 0 0) 0x7f4b48089f10 con 0x7f4b5806b150 2023-11-08T19:27:49.417 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.404+0000 7f4b55d9b700 1 --2- 172.21.15.143:0/1477305445 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f4b3806af00 0x7f4b3806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:49.417 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.404+0000 7f4b55d9b700 1 --2- 172.21.15.143:0/1477305445 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f4b3806af00 0x7f4b3806d3c0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c009c10 tx=0x7f4b4c008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:49.417 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.408+0000 7f4b46ffd700 1 -- 172.21.15.143:0/1477305445 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4b48054a60 con 0x7f4b5806b150 2023-11-08T19:27:49.641 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.640+0000 7f4b5c968700 1 -- 172.21.15.143:0/1477305445 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f4b580b33b0 con 0x7f4b5806b150 2023-11-08T19:27:49.642 INFO:tasks.ceph.mon.a.smithi143.stderr:2023-11-08T19:27:49.641+0000 7f65b584e700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-08T19:27:49.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.642+0000 7f4b46ffd700 1 -- 172.21.15.143:0/1477305445 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v44) v1 ==== 161+0+0 (secure 0 0 0) 0x7f4b48021020 con 0x7f4b5806b150 2023-11-08T19:27:49.647 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.646+0000 7f4b5c968700 1 -- 172.21.15.143:0/1477305445 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f4b3806af00 msgr2=0x7f4b3806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:49.648 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.646+0000 7f4b5c968700 1 --2- 172.21.15.143:0/1477305445 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f4b3806af00 0x7f4b3806d3c0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c009c10 tx=0x7f4b4c008030 comp rx=0 tx=0).stop 2023-11-08T19:27:49.648 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.646+0000 7f4b5c968700 1 -- 172.21.15.143:0/1477305445 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4b5806b150 msgr2=0x7f4b58151e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:49.648 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.646+0000 7f4b5c968700 1 --2- 172.21.15.143:0/1477305445 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4b5806b150 0x7f4b58151e00 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f4b4800e990 tx=0x7f4b4800eca0 comp rx=0 tx=0).stop 2023-11-08T19:27:49.648 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.646+0000 7f4b5c968700 1 -- 172.21.15.143:0/1477305445 shutdown_connections 2023-11-08T19:27:49.648 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.646+0000 7f4b5c968700 1 --2- 172.21.15.143:0/1477305445 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f4b3806af00 0x7f4b3806d3c0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:49.649 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.646+0000 7f4b5c968700 1 --2- 172.21.15.143:0/1477305445 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4b581618f0 0x7f4b580c4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:49.649 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.646+0000 7f4b5c968700 1 --2- 172.21.15.143:0/1477305445 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4b5806b150 0x7f4b58151e00 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:49.649 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.646+0000 7f4b5c968700 1 --2- 172.21.15.143:0/1477305445 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4b5815dd00 0x7f4b58147520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:49.649 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.646+0000 7f4b5c968700 1 -- 172.21.15.143:0/1477305445 >> 172.21.15.143:0/1477305445 conn(0x7f4b58099ab0 msgr2=0x7f4b5815b990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:49.649 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.646+0000 7f4b5c968700 1 -- 172.21.15.143:0/1477305445 shutdown_connections 2023-11-08T19:27:49.650 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.646+0000 7f4b5c968700 1 -- 172.21.15.143:0/1477305445 wait complete. 2023-11-08T19:27:49.650 INFO:tasks.ceph.mds.d.smithi174.stderr: -4811> 2023-11-08T19:27:33.290+0000 7f24365d5700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-08T19:27:49.651 INFO:teuthology.orchestra.run.smithi143.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-08T19:27:49.659 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-08T19:27:49.681 INFO:tasks.ceph.mds.d.smithi174.stdout:starting mds.d at 2023-11-08T19:27:49.782 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.781+0000 7eff5d16d700 1 Processor -- start 2023-11-08T19:27:49.782 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.781+0000 7eff5d16d700 1 -- start start 2023-11-08T19:27:49.784 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.781+0000 7eff5d16d700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7eff5815eca0 0x7eff580b1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:49.785 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.781+0000 7eff5d16d700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff5815e200 con 0x7eff58158140 2023-11-08T19:27:49.785 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.781+0000 7eff5d16d700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff580b23d0 con 0x7eff580b1ff0 2023-11-08T19:27:49.785 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.781+0000 7eff5d16d700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff5815f130 con 0x7eff5815eca0 2023-11-08T19:27:49.785 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.782+0000 7eff5659c700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7eff5815eca0 0x7eff580b1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:49.786 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.782+0000 7eff56d9d700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7eff58158140 0x7eff5806b620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:50316/0 (socket says 172.21.15.143:50316) 2023-11-08T19:27:49.786 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.782+0000 7eff56d9d700 1 -- 172.21.15.143:0/3547970870 learned_addr learned my addr 172.21.15.143:0/3547970870 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:27:49.786 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.782+0000 7eff5659c700 1 -- 172.21.15.143:0/3547970870 >> v1:172.21.15.174:6789/0 conn(0x7eff580b1ff0 legacy=0x7eff581616e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:49.786 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.782+0000 7eff5659c700 1 -- 172.21.15.143:0/3547970870 >> v1:172.21.15.143:6789/0 conn(0x7eff58158140 legacy=0x7eff5806b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:49.786 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.782+0000 7eff5659c700 1 -- 172.21.15.143:0/3547970870 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff580ce740 con 0x7eff5815eca0 2023-11-08T19:27:49.787 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.782+0000 7eff55d9b700 1 -- 172.21.15.143:0/3547970870 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1556931526 0 0) 0x7eff5815e200 con 0x7eff58158140 2023-11-08T19:27:49.787 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.782+0000 7eff5659c700 1 --2- 172.21.15.143:0/3547970870 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7eff5815eca0 0x7eff580b1ab0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7eff40009840 tx=0x7eff40009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d37afb91104b5454 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:49.787 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.782+0000 7eff55d9b700 1 -- 172.21.15.143:0/3547970870 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff40004230 con 0x7eff5815eca0 2023-11-08T19:27:49.787 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.782+0000 7eff55d9b700 1 -- 172.21.15.143:0/3547970870 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff40004390 con 0x7eff5815eca0 2023-11-08T19:27:49.787 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.782+0000 7eff55d9b700 1 -- 172.21.15.143:0/3547970870 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff400036a0 con 0x7eff5815eca0 2023-11-08T19:27:49.788 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.783+0000 7eff5d16d700 1 -- 172.21.15.143:0/3547970870 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7eff5815eca0 msgr2=0x7eff580b1ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:49.788 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.783+0000 7eff5d16d700 1 --2- 172.21.15.143:0/3547970870 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7eff5815eca0 0x7eff580b1ab0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7eff40009840 tx=0x7eff40009b50 comp rx=0 tx=0).stop 2023-11-08T19:27:49.788 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.783+0000 7eff5d16d700 1 -- 172.21.15.143:0/3547970870 shutdown_connections 2023-11-08T19:27:49.788 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.783+0000 7eff5d16d700 1 --2- 172.21.15.143:0/3547970870 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7eff5815eca0 0x7eff580b1ab0 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:49.788 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.783+0000 7eff5d16d700 1 -- 172.21.15.143:0/3547970870 >> 172.21.15.143:0/3547970870 conn(0x7eff580d0630 msgr2=0x7eff580cc9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:49.789 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.783+0000 7eff5d16d700 1 -- 172.21.15.143:0/3547970870 shutdown_connections 2023-11-08T19:27:49.789 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.783+0000 7eff5d16d700 1 -- 172.21.15.143:0/3547970870 wait complete. 2023-11-08T19:27:49.792 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.784+0000 7eff5d16d700 1 Processor -- start 2023-11-08T19:27:49.792 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.784+0000 7eff5d16d700 1 -- start start 2023-11-08T19:27:49.792 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.785+0000 7eff5d16d700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7eff580b1ff0 0x7eff580c6f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:49.792 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.785+0000 7eff5d16d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7eff58158140 0x7eff580af230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:49.793 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.785+0000 7eff5d16d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7eff5815eca0 0x7eff580c8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:49.793 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.785+0000 7eff5d16d700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff580c7690 con 0x7eff58158140 2023-11-08T19:27:49.793 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.785+0000 7eff5d16d700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff581594b0 con 0x7eff5815eca0 2023-11-08T19:27:49.793 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.785+0000 7eff5d16d700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff58159790 con 0x7eff580b1ff0 2023-11-08T19:27:49.794 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.785+0000 7eff5659c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7eff58158140 0x7eff580af230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:49.794 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.785+0000 7eff5659c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7eff58158140 0x7eff580af230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54170/0 (socket says 172.21.15.143:54170) 2023-11-08T19:27:49.794 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.785+0000 7eff5659c700 1 -- 172.21.15.143:0/494885173 learned_addr learned my addr 172.21.15.143:0/494885173 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:49.794 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.785+0000 7eff5759e700 1 --2- 172.21.15.143:0/494885173 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7eff5815eca0 0x7eff580c8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:49.794 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.785+0000 7eff56d9d700 1 --2- 172.21.15.143:0/494885173 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7eff580b1ff0 0x7eff580c6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:49.795 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.785+0000 7eff5659c700 1 -- 172.21.15.143:0/494885173 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7eff580b1ff0 msgr2=0x7eff580c6f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:49.795 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.785+0000 7eff5659c700 1 --2- 172.21.15.143:0/494885173 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7eff580b1ff0 0x7eff580c6f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:49.795 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.785+0000 7eff5659c700 1 -- 172.21.15.143:0/494885173 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7eff5815eca0 msgr2=0x7eff580c8c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:49.795 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.785+0000 7eff5659c700 1 --2- 172.21.15.143:0/494885173 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7eff5815eca0 0x7eff580c8c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:49.795 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.785+0000 7eff5659c700 1 -- 172.21.15.143:0/494885173 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff400094a0 con 0x7eff58158140 2023-11-08T19:27:49.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.785+0000 7eff5759e700 1 --2- 172.21.15.143:0/494885173 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7eff5815eca0 0x7eff580c8c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:27:49.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.786+0000 7eff5659c700 1 --2- 172.21.15.143:0/494885173 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7eff58158140 0x7eff580af230 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7eff400090b0 tx=0x7eff400047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:49.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.786+0000 7eff4f7fe700 1 -- 172.21.15.143:0/494885173 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff40011630 con 0x7eff58158140 2023-11-08T19:27:49.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.786+0000 7eff4f7fe700 1 -- 172.21.15.143:0/494885173 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff40011790 con 0x7eff58158140 2023-11-08T19:27:49.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.786+0000 7eff5d16d700 1 -- 172.21.15.143:0/494885173 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff58159a70 con 0x7eff58158140 2023-11-08T19:27:49.797 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.786+0000 7eff4f7fe700 1 -- 172.21.15.143:0/494885173 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff40011b40 con 0x7eff58158140 2023-11-08T19:27:49.797 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.786+0000 7eff5d16d700 1 -- 172.21.15.143:0/494885173 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff580af770 con 0x7eff58158140 2023-11-08T19:27:49.797 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.787+0000 7eff5d16d700 1 -- 172.21.15.143:0/494885173 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff5804eee0 con 0x7eff58158140 2023-11-08T19:27:49.797 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.787+0000 7eff4f7fe700 1 -- 172.21.15.143:0/494885173 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7eff40011ca0 con 0x7eff58158140 2023-11-08T19:27:49.797 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.787+0000 7eff4f7fe700 1 --2- 172.21.15.143:0/494885173 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7eff3806af00 0x7eff3806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:49.798 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.788+0000 7eff4f7fe700 1 -- 172.21.15.143:0/494885173 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7002+0+0 (secure 0 0 0) 0x7eff4008b370 con 0x7eff58158140 2023-11-08T19:27:49.798 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.788+0000 7eff56d9d700 1 --2- 172.21.15.143:0/494885173 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7eff3806af00 0x7eff3806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:49.798 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.788+0000 7eff56d9d700 1 --2- 172.21.15.143:0/494885173 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7eff3806af00 0x7eff3806d3c0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7eff48000f60 tx=0x7eff48009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:49.799 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:49.792+0000 7eff4f7fe700 1 -- 172.21.15.143:0/494885173 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7eff40055fa0 con 0x7eff58158140 2023-11-08T19:27:50.027 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:50.026+0000 7eff5d16d700 1 -- 172.21.15.143:0/494885173 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7eff580cd290 con 0x7eff58158140 2023-11-08T19:27:50.645 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:50.644+0000 7eff4f7fe700 1 -- 172.21.15.143:0/494885173 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v45) v1 ==== 104+0+0 (secure 0 0 0) 0x7eff400595c0 con 0x7eff58158140 2023-11-08T19:27:50.648 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:50.646+0000 7eff5d16d700 1 -- 172.21.15.143:0/494885173 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7eff3806af00 msgr2=0x7eff3806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:50.648 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:50.646+0000 7eff5d16d700 1 --2- 172.21.15.143:0/494885173 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7eff3806af00 0x7eff3806d3c0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7eff48000f60 tx=0x7eff48009040 comp rx=0 tx=0).stop 2023-11-08T19:27:50.648 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:50.646+0000 7eff5d16d700 1 -- 172.21.15.143:0/494885173 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7eff58158140 msgr2=0x7eff580af230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:50.649 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:50.646+0000 7eff5d16d700 1 --2- 172.21.15.143:0/494885173 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7eff58158140 0x7eff580af230 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7eff400090b0 tx=0x7eff400047c0 comp rx=0 tx=0).stop 2023-11-08T19:27:50.649 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:50.646+0000 7eff5d16d700 1 -- 172.21.15.143:0/494885173 shutdown_connections 2023-11-08T19:27:50.649 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:50.646+0000 7eff5d16d700 1 --2- 172.21.15.143:0/494885173 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7eff3806af00 0x7eff3806d3c0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:50.649 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:50.646+0000 7eff5d16d700 1 --2- 172.21.15.143:0/494885173 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7eff580b1ff0 0x7eff580c6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:50.649 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:50.646+0000 7eff5d16d700 1 --2- 172.21.15.143:0/494885173 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7eff5815eca0 0x7eff580c8c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:50.650 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:50.646+0000 7eff5d16d700 1 --2- 172.21.15.143:0/494885173 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7eff58158140 0x7eff580af230 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:50.650 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:50.646+0000 7eff5d16d700 1 -- 172.21.15.143:0/494885173 >> 172.21.15.143:0/494885173 conn(0x7eff580d0630 msgr2=0x7eff58071800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:50.650 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:50.647+0000 7eff5d16d700 1 -- 172.21.15.143:0/494885173 shutdown_connections 2023-11-08T19:27:50.650 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:50.647+0000 7eff5d16d700 1 -- 172.21.15.143:0/494885173 wait complete. 2023-11-08T19:27:50.661 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-08T19:27:50.787 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:50.786+0000 7fe55aa2a700 1 Processor -- start 2023-11-08T19:27:50.788 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:50.786+0000 7fe55aa2a700 1 -- start start 2023-11-08T19:27:50.791 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:50.787+0000 7fe55aa2a700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe5540b22e0 0x7fe5540b2700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:50.791 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:50.787+0000 7fe55aa2a700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe554161890 con 0x7fe55415dba0 2023-11-08T19:27:50.791 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:50.787+0000 7fe55aa2a700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5540b2c40 con 0x7fe55406b240 2023-11-08T19:27:50.792 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:50.787+0000 7fe55aa2a700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe55406cd20 con 0x7fe5540b22e0 2023-11-08T19:27:50.792 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:50.787+0000 7fe558fc7700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe5540b22e0 0x7fe5540b2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:50.792 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:50.787+0000 7fe558fc7700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe5540b22e0 0x7fe5540b2700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54180/0 (socket says 172.21.15.143:54180) 2023-11-08T19:27:50.792 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:50.787+0000 7fe553fff700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fe55415dba0 0x7fe55406c610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:50330/0 (socket says 172.21.15.143:50330) 2023-11-08T19:27:50.792 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:50.787+0000 7fe558fc7700 1 -- 172.21.15.143:0/974189833 learned_addr learned my addr 172.21.15.143:0/974189833 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:50.793 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:50.787+0000 7fe558fc7700 1 -- 172.21.15.143:0/974189833 >> v1:172.21.15.174:6789/0 conn(0x7fe55406b240 legacy=0x7fe5540caff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:50.793 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:50.787+0000 7fe558fc7700 1 -- 172.21.15.143:0/974189833 >> v1:172.21.15.143:6789/0 conn(0x7fe55415dba0 legacy=0x7fe55406c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:50.793 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:50.787+0000 7fe558fc7700 1 -- 172.21.15.143:0/974189833 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe55406b740 con 0x7fe5540b22e0 2023-11-08T19:27:50.793 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:50.787+0000 7fe558fc7700 1 --2- 172.21.15.143:0/974189833 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe5540b22e0 0x7fe5540b2700 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fe54c009e50 tx=0x7fe54c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c49f90857d177144 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:50.794 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:50.787+0000 7fe552ffd700 1 -- 172.21.15.143:0/974189833 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe54c00e070 con 0x7fe5540b22e0 2023-11-08T19:27:50.794 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:50.787+0000 7fe552ffd700 1 -- 172.21.15.143:0/974189833 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe54c00b9c0 con 0x7fe5540b22e0 2023-11-08T19:27:50.794 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:50.787+0000 7fe552ffd700 1 -- 172.21.15.143:0/974189833 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe54c00bd70 con 0x7fe5540b22e0 2023-11-08T19:27:50.794 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:50.788+0000 7fe55aa2a700 1 -- 172.21.15.143:0/974189833 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe5540b22e0 msgr2=0x7fe5540b2700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:50.795 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:50.788+0000 7fe55aa2a700 1 --2- 172.21.15.143:0/974189833 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe5540b22e0 0x7fe5540b2700 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fe54c009e50 tx=0x7fe54c00b330 comp rx=0 tx=0).stop 2023-11-08T19:27:50.795 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:50.788+0000 7fe55aa2a700 1 -- 172.21.15.143:0/974189833 shutdown_connections 2023-11-08T19:27:50.795 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:50.788+0000 7fe55aa2a700 1 --2- 172.21.15.143:0/974189833 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe5540b22e0 0x7fe5540b2700 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:50.795 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:50.788+0000 7fe55aa2a700 1 -- 172.21.15.143:0/974189833 >> 172.21.15.143:0/974189833 conn(0x7fe5540cd2e0 msgr2=0x7fe5540cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:50.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:50.788+0000 7fe55aa2a700 1 -- 172.21.15.143:0/974189833 shutdown_connections 2023-11-08T19:27:50.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:50.788+0000 7fe55aa2a700 1 -- 172.21.15.143:0/974189833 wait complete. 2023-11-08T19:27:50.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:50.789+0000 7fe55aa2a700 1 Processor -- start 2023-11-08T19:27:50.797 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:50.789+0000 7fe55aa2a700 1 -- start start 2023-11-08T19:27:50.800 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:50.790+0000 7fe55aa2a700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe55406b240 0x7fe5540ca640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:50.800 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:50.790+0000 7fe55aa2a700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe5540b22e0 0x7fe5540c4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:50.800 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:50.790+0000 7fe55aa2a700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe55415dba0 0x7fe55415abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:50.800 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:50.790+0000 7fe55aa2a700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5540c5150 con 0x7fe55415dba0 2023-11-08T19:27:50.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:50.790+0000 7fe55aa2a700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5540ad760 con 0x7fe55406b240 2023-11-08T19:27:50.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:50.790+0000 7fe55aa2a700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5540ada40 con 0x7fe5540b22e0 2023-11-08T19:27:50.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:50.790+0000 7fe558fc7700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe55415dba0 0x7fe55415abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:50.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:50.790+0000 7fe558fc7700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe55415dba0 0x7fe55415abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54186/0 (socket says 172.21.15.143:54186) 2023-11-08T19:27:50.802 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:50.790+0000 7fe558fc7700 1 -- 172.21.15.143:0/3957045101 learned_addr learned my addr 172.21.15.143:0/3957045101 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:50.802 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:50.790+0000 7fe553fff700 1 --2- 172.21.15.143:0/3957045101 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe55406b240 0x7fe5540ca640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:50.802 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:50.790+0000 7fe5537fe700 1 --2- 172.21.15.143:0/3957045101 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe5540b22e0 0x7fe5540c4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:50.802 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:50.790+0000 7fe558fc7700 1 -- 172.21.15.143:0/3957045101 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe5540b22e0 msgr2=0x7fe5540c4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:50.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:50.790+0000 7fe558fc7700 1 --2- 172.21.15.143:0/3957045101 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe5540b22e0 0x7fe5540c4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:50.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:50.790+0000 7fe558fc7700 1 -- 172.21.15.143:0/3957045101 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe55406b240 msgr2=0x7fe5540ca640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:50.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:50.790+0000 7fe558fc7700 1 --2- 172.21.15.143:0/3957045101 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe55406b240 0x7fe5540ca640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:50.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:50.790+0000 7fe558fc7700 1 -- 172.21.15.143:0/3957045101 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe54c009b30 con 0x7fe55415dba0 2023-11-08T19:27:50.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:50.790+0000 7fe553fff700 1 --2- 172.21.15.143:0/3957045101 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe55406b240 0x7fe5540ca640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:27:50.804 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:50.791+0000 7fe558fc7700 1 --2- 172.21.15.143:0/3957045101 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe55415dba0 0x7fe55415abe0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fe554067530 tx=0x7fe54c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:50.804 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:50.791+0000 7fe550ff9700 1 -- 172.21.15.143:0/3957045101 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe54c004960 con 0x7fe55415dba0 2023-11-08T19:27:50.804 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:50.791+0000 7fe550ff9700 1 -- 172.21.15.143:0/3957045101 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe54c0099b0 con 0x7fe55415dba0 2023-11-08T19:27:50.804 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:50.791+0000 7fe550ff9700 1 -- 172.21.15.143:0/3957045101 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe54c01c680 con 0x7fe55415dba0 2023-11-08T19:27:50.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:50.791+0000 7fe55aa2a700 1 -- 172.21.15.143:0/3957045101 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5540add20 con 0x7fe55415dba0 2023-11-08T19:27:50.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:50.791+0000 7fe55aa2a700 1 -- 172.21.15.143:0/3957045101 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe55415b120 con 0x7fe55415dba0 2023-11-08T19:27:50.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:50.792+0000 7fe55aa2a700 1 -- 172.21.15.143:0/3957045101 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5540ae1e0 con 0x7fe55415dba0 2023-11-08T19:27:50.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:50.792+0000 7fe550ff9700 1 -- 172.21.15.143:0/3957045101 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe54c01c7e0 con 0x7fe55415dba0 2023-11-08T19:27:50.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:50.793+0000 7fe550ff9700 1 --2- 172.21.15.143:0/3957045101 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fe54006af00 0x7fe54006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:50.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:50.793+0000 7fe550ff9700 1 -- 172.21.15.143:0/3957045101 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7002+0+0 (secure 0 0 0) 0x7fe54c08b9a0 con 0x7fe55415dba0 2023-11-08T19:27:50.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:50.793+0000 7fe553fff700 1 --2- 172.21.15.143:0/3957045101 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fe54006af00 0x7fe54006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:50.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:50.793+0000 7fe553fff700 1 --2- 172.21.15.143:0/3957045101 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fe54006af00 0x7fe54006d3c0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fe548000f60 tx=0x7fe548007dd0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:50.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:50.797+0000 7fe550ff9700 1 -- 172.21.15.143:0/3957045101 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe54c059fc0 con 0x7fe55415dba0 2023-11-08T19:27:50.987 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:50.985+0000 7fe55aa2a700 1 -- 172.21.15.143:0/3957045101 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe55404eee0 con 0x7fe55415dba0 2023-11-08T19:27:51.649 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:51.647+0000 7fe550ff9700 1 -- 172.21.15.143:0/3957045101 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v83) v1 ==== 181+0+0 (secure 0 0 0) 0x7fe54c059b50 con 0x7fe55415dba0 2023-11-08T19:27:51.652 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:51.650+0000 7fe55aa2a700 1 -- 172.21.15.143:0/3957045101 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fe54006af00 msgr2=0x7fe54006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:51.652 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:51.650+0000 7fe55aa2a700 1 --2- 172.21.15.143:0/3957045101 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fe54006af00 0x7fe54006d3c0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fe548000f60 tx=0x7fe548007dd0 comp rx=0 tx=0).stop 2023-11-08T19:27:51.652 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:51.650+0000 7fe55aa2a700 1 -- 172.21.15.143:0/3957045101 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe55415dba0 msgr2=0x7fe55415abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:51.653 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:51.650+0000 7fe55aa2a700 1 --2- 172.21.15.143:0/3957045101 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe55415dba0 0x7fe55415abe0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fe554067530 tx=0x7fe54c004ad0 comp rx=0 tx=0).stop 2023-11-08T19:27:51.653 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:51.650+0000 7fe55aa2a700 1 -- 172.21.15.143:0/3957045101 shutdown_connections 2023-11-08T19:27:51.653 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:51.650+0000 7fe55aa2a700 1 --2- 172.21.15.143:0/3957045101 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fe54006af00 0x7fe54006d3c0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:51.653 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:51.650+0000 7fe55aa2a700 1 --2- 172.21.15.143:0/3957045101 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe55415dba0 0x7fe55415abe0 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:51.654 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:51.650+0000 7fe55aa2a700 1 --2- 172.21.15.143:0/3957045101 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe55406b240 0x7fe5540ca640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:51.654 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:51.650+0000 7fe55aa2a700 1 --2- 172.21.15.143:0/3957045101 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe5540b22e0 0x7fe5540c4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:51.654 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:51.650+0000 7fe55aa2a700 1 -- 172.21.15.143:0/3957045101 >> 172.21.15.143:0/3957045101 conn(0x7fe5540cd2e0 msgr2=0x7fe5540cd8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:51.654 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:51.651+0000 7fe55aa2a700 1 -- 172.21.15.143:0/3957045101 shutdown_connections 2023-11-08T19:27:51.654 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:51.651+0000 7fe55aa2a700 1 -- 172.21.15.143:0/3957045101 wait complete. 2023-11-08T19:27:51.655 INFO:teuthology.orchestra.run.smithi143.stderr:pool 'cephfs_metadata' removed 2023-11-08T19:27:51.668 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-08T19:27:51.798 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:51.797+0000 7f89a31e4700 1 Processor -- start 2023-11-08T19:27:51.798 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:51.797+0000 7f89a31e4700 1 -- start start 2023-11-08T19:27:51.802 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:51.798+0000 7f89a31e4700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f899c06bd20 0x7f899c0d01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:51.802 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:51.798+0000 7f89a31e4700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f899c0cc2a0 con 0x7f899c15a950 2023-11-08T19:27:51.802 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:51.798+0000 7f89a31e4700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f899c06c1b0 con 0x7f899c0b61e0 2023-11-08T19:27:51.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:51.798+0000 7f89a31e4700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f899c0cc130 con 0x7f899c06bd20 2023-11-08T19:27:51.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:51.798+0000 7f89a0f80700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f899c15a950 0x7f899c072370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:50344/0 (socket says 172.21.15.143:50344) 2023-11-08T19:27:51.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:51.798+0000 7f89a0f80700 1 -- 172.21.15.143:0/649303618 learned_addr learned my addr 172.21.15.143:0/649303618 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:27:51.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:51.798+0000 7f899b7fe700 1 -- 172.21.15.143:0/649303618 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3291353097 0 0) 0x7f899c0cc2a0 con 0x7f899c15a950 2023-11-08T19:27:51.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:51.798+0000 7f899b7fe700 1 -- 172.21.15.143:0/649303618 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f898c0035b0 con 0x7f899c15a950 2023-11-08T19:27:51.804 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:51.798+0000 7f899bfff700 1 --2- 172.21.15.143:0/649303618 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f899c06bd20 0x7f899c0d01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:51.804 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:51.798+0000 7f899b7fe700 1 -- 172.21.15.143:0/649303618 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1284363061 0 0) 0x7f898c0035b0 con 0x7f899c15a950 2023-11-08T19:27:51.804 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:51.798+0000 7f899b7fe700 1 -- 172.21.15.143:0/649303618 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f899c06bd20 msgr2=0x7f899c0d01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:51.804 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:51.798+0000 7f899b7fe700 1 --2- 172.21.15.143:0/649303618 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f899c06bd20 0x7f899c0d01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:51.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:51.798+0000 7f899b7fe700 1 -- 172.21.15.143:0/649303618 >> v1:172.21.15.174:6789/0 conn(0x7f899c0b61e0 legacy=0x7f899c0cc730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:51.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:51.798+0000 7f899b7fe700 1 -- 172.21.15.143:0/649303618 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f899c06c490 con 0x7f899c15a950 2023-11-08T19:27:51.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:51.798+0000 7f899b7fe700 1 -- 172.21.15.143:0/649303618 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4147756303 0 0) 0x7f8990001670 con 0x7f899c15a950 2023-11-08T19:27:51.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:51.799+0000 7f899b7fe700 1 -- 172.21.15.143:0/649303618 >> v1:172.21.15.143:6789/0 conn(0x7f899c15a950 legacy=0x7f899c072370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:51.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:51.799+0000 7f899b7fe700 1 --2- 172.21.15.143:0/649303618 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f898c003a30 0x7f898c003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:51.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:51.799+0000 7f899b7fe700 1 --2- 172.21.15.143:0/649303618 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f898c004850 0x7f898c004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:51.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:51.799+0000 7f899b7fe700 1 --2- 172.21.15.143:0/649303618 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f898c005330 0x7f898c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:51.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:51.799+0000 7f899b7fe700 1 -- 172.21.15.143:0/649303618 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f898c00c000 con 0x7f898c004850 2023-11-08T19:27:51.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:51.799+0000 7f899b7fe700 1 -- 172.21.15.143:0/649303618 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f898c00c140 con 0x7f898c003a30 2023-11-08T19:27:51.807 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:51.799+0000 7f899b7fe700 1 -- 172.21.15.143:0/649303618 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f898c00c2d0 con 0x7f898c005330 2023-11-08T19:27:51.807 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:51.799+0000 7f899bfff700 1 --2- 172.21.15.143:0/649303618 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f898c004850 0x7f898c004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:51.807 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:51.799+0000 7f89a0f80700 1 --2- 172.21.15.143:0/649303618 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f898c003a30 0x7f898c003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:51.807 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:51.799+0000 7f89a1781700 1 --2- 172.21.15.143:0/649303618 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f898c005330 0x7f898c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:51.807 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:51.799+0000 7f899bfff700 1 -- 172.21.15.143:0/649303618 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f898c005330 msgr2=0x7f898c00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:51.808 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:51.799+0000 7f899bfff700 1 --2- 172.21.15.143:0/649303618 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f898c005330 0x7f898c00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:51.808 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:51.799+0000 7f899bfff700 1 -- 172.21.15.143:0/649303618 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f898c003a30 msgr2=0x7f898c003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:51.808 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:51.799+0000 7f899bfff700 1 --2- 172.21.15.143:0/649303618 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f898c003a30 0x7f898c003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:51.808 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:51.799+0000 7f899bfff700 1 -- 172.21.15.143:0/649303618 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f898c0035b0 con 0x7f898c004850 2023-11-08T19:27:51.808 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:51.799+0000 7f899bfff700 1 --2- 172.21.15.143:0/649303618 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f898c004850 0x7f898c004cd0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f8988009890 tx=0x7f898c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e17c487bc7075c6 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:51.809 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:51.799+0000 7f899b7fe700 1 -- 172.21.15.143:0/649303618 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f898800c070 con 0x7f898c004850 2023-11-08T19:27:51.809 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:51.799+0000 7f899b7fe700 1 -- 172.21.15.143:0/649303618 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8988013610 con 0x7f898c004850 2023-11-08T19:27:51.809 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:51.799+0000 7f899b7fe700 1 -- 172.21.15.143:0/649303618 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89880139a0 con 0x7f898c004850 2023-11-08T19:27:51.809 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:51.800+0000 7f89a31e4700 1 -- 172.21.15.143:0/649303618 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f898c004850 msgr2=0x7f898c004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:51.809 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:51.800+0000 7f89a31e4700 1 --2- 172.21.15.143:0/649303618 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f898c004850 0x7f898c004cd0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f8988009890 tx=0x7f898c005880 comp rx=0 tx=0).stop 2023-11-08T19:27:51.810 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:51.800+0000 7f89a1781700 1 -- 172.21.15.143:0/649303618 reap_dead start 2023-11-08T19:27:51.810 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:51.800+0000 7f89a31e4700 1 -- 172.21.15.143:0/649303618 shutdown_connections 2023-11-08T19:27:51.810 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:51.800+0000 7f89a31e4700 1 -- 172.21.15.143:0/649303618 >> 172.21.15.143:0/649303618 conn(0x7f899c0cd2e0 msgr2=0x7f899c0cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:51.810 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:51.800+0000 7f89a31e4700 1 -- 172.21.15.143:0/649303618 shutdown_connections 2023-11-08T19:27:51.810 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:51.800+0000 7f89a31e4700 1 -- 172.21.15.143:0/649303618 wait complete. 2023-11-08T19:27:51.813 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:51.801+0000 7f89a31e4700 1 Processor -- start 2023-11-08T19:27:51.814 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:51.802+0000 7f89a31e4700 1 -- start start 2023-11-08T19:27:51.814 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:51.803+0000 7f89a31e4700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f898c004850 0x7f899c1594b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:51.814 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:51.803+0000 7f89a31e4700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f899c1f1b90 0x7f899c0c5c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:51.814 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:51.803+0000 7f89a31e4700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f899c0c61d0 0x7f899c1f4500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:51.814 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:51.803+0000 7f89a31e4700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f899c0c6680 con 0x7f898c004850 2023-11-08T19:27:51.815 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:51.803+0000 7f89a31e4700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f899c1f53c0 con 0x7f899c1f1b90 2023-11-08T19:27:51.815 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:51.803+0000 7f89a31e4700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f899c1513a0 con 0x7f899c0c61d0 2023-11-08T19:27:51.815 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:51.803+0000 7f89a0f80700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f898c004850 0x7f899c1594b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:51.815 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:51.803+0000 7f89a0f80700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f898c004850 0x7f899c1594b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54202/0 (socket says 172.21.15.143:54202) 2023-11-08T19:27:51.815 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:51.803+0000 7f89a0f80700 1 -- 172.21.15.143:0/3169501307 learned_addr learned my addr 172.21.15.143:0/3169501307 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:51.816 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:51.803+0000 7f899bfff700 1 --2- 172.21.15.143:0/3169501307 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f899c1f1b90 0x7f899c0c5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:51.816 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:51.803+0000 7f89a1781700 1 --2- 172.21.15.143:0/3169501307 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f899c0c61d0 0x7f899c1f4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:51.816 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:51.803+0000 7f89a0f80700 1 -- 172.21.15.143:0/3169501307 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f899c0c61d0 msgr2=0x7f899c1f4500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:51.816 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:51.803+0000 7f89a0f80700 1 --2- 172.21.15.143:0/3169501307 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f899c0c61d0 0x7f899c1f4500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:51.816 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:51.803+0000 7f89a0f80700 1 -- 172.21.15.143:0/3169501307 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f899c1f1b90 msgr2=0x7f899c0c5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:51.817 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:51.803+0000 7f89a0f80700 1 --2- 172.21.15.143:0/3169501307 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f899c1f1b90 0x7f899c0c5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:51.817 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:51.803+0000 7f89a0f80700 1 -- 172.21.15.143:0/3169501307 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f899c1f50e0 con 0x7f898c004850 2023-11-08T19:27:51.817 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:51.803+0000 7f899bfff700 1 --2- 172.21.15.143:0/3169501307 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f899c1f1b90 0x7f899c0c5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:27:51.817 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:51.803+0000 7f89a0f80700 1 --2- 172.21.15.143:0/3169501307 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f898c004850 0x7f899c1594b0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f89900072f0 tx=0x7f89900076b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:51.817 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:51.803+0000 7f89997fa700 1 -- 172.21.15.143:0/3169501307 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8990001450 con 0x7f898c004850 2023-11-08T19:27:51.818 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:51.803+0000 7f89997fa700 1 -- 172.21.15.143:0/3169501307 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f899000f850 con 0x7f898c004850 2023-11-08T19:27:51.818 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:51.803+0000 7f89a31e4700 1 -- 172.21.15.143:0/3169501307 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f899c1f10b0 con 0x7f898c004850 2023-11-08T19:27:51.818 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:51.803+0000 7f89997fa700 1 -- 172.21.15.143:0/3169501307 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f899000fc00 con 0x7f898c004850 2023-11-08T19:27:51.818 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:51.803+0000 7f89a31e4700 1 -- 172.21.15.143:0/3169501307 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f899c1f1670 con 0x7f898c004850 2023-11-08T19:27:51.818 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:51.804+0000 7f89a31e4700 1 -- 172.21.15.143:0/3169501307 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f899c1f0f10 con 0x7f898c004850 2023-11-08T19:27:51.819 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:51.805+0000 7f89997fa700 1 -- 172.21.15.143:0/3169501307 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f899000fd60 con 0x7f898c004850 2023-11-08T19:27:51.819 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:51.805+0000 7f89997fa700 1 --2- 172.21.15.143:0/3169501307 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f898006af00 0x7f898006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:51.819 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:51.805+0000 7f89997fa700 1 -- 172.21.15.143:0/3169501307 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6564+0+0 (secure 0 0 0) 0x7f89900899f0 con 0x7f898c004850 2023-11-08T19:27:51.819 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:51.805+0000 7f899bfff700 1 --2- 172.21.15.143:0/3169501307 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f898006af00 0x7f898006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:51.819 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:51.806+0000 7f899bfff700 1 --2- 172.21.15.143:0/3169501307 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f898006af00 0x7f898006d3c0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f8988008010 tx=0x7f898800c000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:51.820 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:51.809+0000 7f89997fa700 1 -- 172.21.15.143:0/3169501307 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f899c1f0f10 con 0x7f898c004850 2023-11-08T19:27:52.001 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:52.000+0000 7f89a31e4700 1 -- 172.21.15.143:0/3169501307 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f899c1f0f10 con 0x7f898c004850 2023-11-08T19:27:52.590 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:52.589+0000 7f89997fa700 1 -- 172.21.15.143:0/3169501307 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v84) v1 ==== 169+0+0 (secure 0 0 0) 0x7f899c1f0f10 con 0x7f898c004850 2023-11-08T19:27:52.593 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:52.591+0000 7f89a31e4700 1 -- 172.21.15.143:0/3169501307 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f898006af00 msgr2=0x7f898006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:52.593 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:52.591+0000 7f89a31e4700 1 --2- 172.21.15.143:0/3169501307 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f898006af00 0x7f898006d3c0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f8988008010 tx=0x7f898800c000 comp rx=0 tx=0).stop 2023-11-08T19:27:52.593 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:52.591+0000 7f89a31e4700 1 -- 172.21.15.143:0/3169501307 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f898c004850 msgr2=0x7f899c1594b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:52.594 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:52.591+0000 7f89a31e4700 1 --2- 172.21.15.143:0/3169501307 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f898c004850 0x7f899c1594b0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f89900072f0 tx=0x7f89900076b0 comp rx=0 tx=0).stop 2023-11-08T19:27:52.594 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:52.591+0000 7f89a31e4700 1 -- 172.21.15.143:0/3169501307 shutdown_connections 2023-11-08T19:27:52.594 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:52.591+0000 7f89a31e4700 1 --2- 172.21.15.143:0/3169501307 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f898006af00 0x7f898006d3c0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:52.594 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:52.591+0000 7f89a31e4700 1 --2- 172.21.15.143:0/3169501307 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f899c0c61d0 0x7f899c1f4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:52.594 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:52.591+0000 7f89a31e4700 1 --2- 172.21.15.143:0/3169501307 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f898c004850 0x7f899c1594b0 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:52.595 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:52.591+0000 7f89a31e4700 1 --2- 172.21.15.143:0/3169501307 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f899c1f1b90 0x7f899c0c5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:52.595 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:52.591+0000 7f89a31e4700 1 -- 172.21.15.143:0/3169501307 >> 172.21.15.143:0/3169501307 conn(0x7f899c0cd2e0 msgr2=0x7f899c0cc730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:52.595 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:52.591+0000 7f89a31e4700 1 -- 172.21.15.143:0/3169501307 shutdown_connections 2023-11-08T19:27:52.595 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:52.591+0000 7f89a31e4700 1 -- 172.21.15.143:0/3169501307 wait complete. 2023-11-08T19:27:52.596 INFO:teuthology.orchestra.run.smithi143.stderr:pool 'cephfs_data' removed 2023-11-08T19:27:52.609 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-08T19:27:52.733 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:52.731+0000 7fda535ec700 1 Processor -- start 2023-11-08T19:27:52.733 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:52.732+0000 7fda535ec700 1 -- start start 2023-11-08T19:27:52.735 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:52.732+0000 7fda535ec700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fda4c15a950 0x7fda4c0d01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:52.736 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:52.732+0000 7fda535ec700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fda4c0b61e0 0x7fda4c0cc730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:52.736 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:52.732+0000 7fda535ec700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda4c0cc2a0 con 0x7fda4c06bd20 2023-11-08T19:27:52.736 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:52.732+0000 7fda535ec700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda4c0cc130 con 0x7fda4c0b61e0 2023-11-08T19:27:52.736 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:52.732+0000 7fda535ec700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda4c06c1b0 con 0x7fda4c15a950 2023-11-08T19:27:52.737 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:52.732+0000 7fda51b89700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fda4c0b61e0 0x7fda4c0cc730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:52.737 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:52.733+0000 7fda50b87700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fda4c06bd20 0x7fda4c072370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:50366/0 (socket says 172.21.15.143:50366) 2023-11-08T19:27:52.737 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:52.733+0000 7fda50b87700 1 -- 172.21.15.143:0/2678787129 learned_addr learned my addr 172.21.15.143:0/2678787129 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:27:52.737 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:52.733+0000 7fda51b89700 1 -- 172.21.15.143:0/2678787129 >> v2:172.21.15.174:3300/0 conn(0x7fda4c15a950 msgr2=0x7fda4c0d01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:52.737 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:52.733+0000 7fda51388700 1 --2- 172.21.15.143:0/2678787129 >> v2:172.21.15.174:3300/0 conn(0x7fda4c15a950 0x7fda4c0d01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:52.737 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:52.733+0000 7fda43fff700 1 -- 172.21.15.143:0/2678787129 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4138650208 0 0) 0x7fda4c0cc2a0 con 0x7fda4c06bd20 2023-11-08T19:27:52.738 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:52.733+0000 7fda51b89700 1 --2- 172.21.15.143:0/2678787129 >> v2:172.21.15.174:3300/0 conn(0x7fda4c15a950 0x7fda4c0d01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:52.738 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:52.733+0000 7fda51b89700 1 -- 172.21.15.143:0/2678787129 >> v1:172.21.15.143:6789/0 conn(0x7fda4c06bd20 legacy=0x7fda4c072370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:52.738 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:52.733+0000 7fda51b89700 1 -- 172.21.15.143:0/2678787129 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda4c06c490 con 0x7fda4c0b61e0 2023-11-08T19:27:52.738 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:52.733+0000 7fda51388700 1 --2- 172.21.15.143:0/2678787129 >> v2:172.21.15.174:3300/0 conn(0x7fda4c15a950 0x7fda4c0d01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:27:52.738 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:52.733+0000 7fda51b89700 1 --2- 172.21.15.143:0/2678787129 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fda4c0b61e0 0x7fda4c0cc730 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fda4c0cdf90 tx=0x7fda4400d330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eca0b12aa11fbaa4 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:52.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:52.733+0000 7fda43fff700 1 -- 172.21.15.143:0/2678787129 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda44010070 con 0x7fda4c0b61e0 2023-11-08T19:27:52.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:52.733+0000 7fda43fff700 1 -- 172.21.15.143:0/2678787129 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda4400b2e0 con 0x7fda4c0b61e0 2023-11-08T19:27:52.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:52.733+0000 7fda43fff700 1 -- 172.21.15.143:0/2678787129 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda4400dcc0 con 0x7fda4c0b61e0 2023-11-08T19:27:52.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:52.734+0000 7fda535ec700 1 -- 172.21.15.143:0/2678787129 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fda4c0b61e0 msgr2=0x7fda4c0cc730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:52.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:52.734+0000 7fda535ec700 1 --2- 172.21.15.143:0/2678787129 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fda4c0b61e0 0x7fda4c0cc730 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fda4c0cdf90 tx=0x7fda4400d330 comp rx=0 tx=0).stop 2023-11-08T19:27:52.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:52.734+0000 7fda535ec700 1 -- 172.21.15.143:0/2678787129 shutdown_connections 2023-11-08T19:27:52.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:52.734+0000 7fda535ec700 1 --2- 172.21.15.143:0/2678787129 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fda4c0b61e0 0x7fda4c0cc730 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:52.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:52.734+0000 7fda535ec700 1 --2- 172.21.15.143:0/2678787129 >> v2:172.21.15.174:3300/0 conn(0x7fda4c15a950 0x7fda4c0d01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:52.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:52.734+0000 7fda535ec700 1 -- 172.21.15.143:0/2678787129 >> 172.21.15.143:0/2678787129 conn(0x7fda4c0cd2e0 msgr2=0x7fda4c0cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:52.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:52.734+0000 7fda535ec700 1 -- 172.21.15.143:0/2678787129 shutdown_connections 2023-11-08T19:27:52.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:52.734+0000 7fda535ec700 1 -- 172.21.15.143:0/2678787129 wait complete. 2023-11-08T19:27:52.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:52.735+0000 7fda535ec700 1 Processor -- start 2023-11-08T19:27:52.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:52.735+0000 7fda535ec700 1 -- start start 2023-11-08T19:27:52.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:52.736+0000 7fda535ec700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fda4c0b61e0 0x7fda4c157450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:52.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:52.736+0000 7fda535ec700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fda4c15a950 0x7fda4c15c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:52.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:52.736+0000 7fda535ec700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fda4c0b4820 0x7fda4c0c9940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:52.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:52.736+0000 7fda535ec700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda4c0b0090 con 0x7fda4c0b61e0 2023-11-08T19:27:52.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:52.736+0000 7fda535ec700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda4c15cd90 con 0x7fda4c0b4820 2023-11-08T19:27:52.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:52.736+0000 7fda535ec700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda4c15d040 con 0x7fda4c15a950 2023-11-08T19:27:52.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:52.736+0000 7fda51388700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fda4c0b61e0 0x7fda4c157450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:52.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:52.736+0000 7fda51388700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fda4c0b61e0 0x7fda4c157450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54216/0 (socket says 172.21.15.143:54216) 2023-11-08T19:27:52.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:52.736+0000 7fda51388700 1 -- 172.21.15.143:0/309022513 learned_addr learned my addr 172.21.15.143:0/309022513 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:52.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:52.736+0000 7fda51388700 1 -- 172.21.15.143:0/309022513 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fda4c15a950 msgr2=0x7fda4c15c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:52.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:52.736+0000 7fda51388700 1 --2- 172.21.15.143:0/309022513 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fda4c15a950 0x7fda4c15c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:52.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:52.736+0000 7fda51388700 1 -- 172.21.15.143:0/309022513 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fda4c0b4820 msgr2=0x7fda4c0c9940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:52.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:52.736+0000 7fda51388700 1 --2- 172.21.15.143:0/309022513 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fda4c0b4820 0x7fda4c0c9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:52.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:52.736+0000 7fda51388700 1 -- 172.21.15.143:0/309022513 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda4400bc80 con 0x7fda4c0b61e0 2023-11-08T19:27:52.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:52.736+0000 7fda51388700 1 --2- 172.21.15.143:0/309022513 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fda4c0b61e0 0x7fda4c157450 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fda3c00c7f0 tx=0x7fda3c00cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:52.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:52.736+0000 7fda41ffb700 1 -- 172.21.15.143:0/309022513 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda3c011800 con 0x7fda4c0b61e0 2023-11-08T19:27:52.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:52.736+0000 7fda41ffb700 1 -- 172.21.15.143:0/309022513 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda3c011960 con 0x7fda4c0b61e0 2023-11-08T19:27:52.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:52.736+0000 7fda535ec700 1 -- 172.21.15.143:0/309022513 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda4c15d320 con 0x7fda4c0b61e0 2023-11-08T19:27:52.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:52.736+0000 7fda41ffb700 1 -- 172.21.15.143:0/309022513 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda3c011d10 con 0x7fda4c0b61e0 2023-11-08T19:27:52.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:52.737+0000 7fda535ec700 1 -- 172.21.15.143:0/309022513 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda4c15d8e0 con 0x7fda4c0b61e0 2023-11-08T19:27:52.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:52.738+0000 7fda535ec700 1 -- 172.21.15.143:0/309022513 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda4c04eee0 con 0x7fda4c0b61e0 2023-11-08T19:27:52.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:52.738+0000 7fda41ffb700 1 -- 172.21.15.143:0/309022513 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fda3c018360 con 0x7fda4c0b61e0 2023-11-08T19:27:52.750 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:52.738+0000 7fda41ffb700 1 --2- 172.21.15.143:0/309022513 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fda3006af00 0x7fda3006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:52.750 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:52.738+0000 7fda41ffb700 1 -- 172.21.15.143:0/309022513 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6034+0+0 (secure 0 0 0) 0x7fda3c089760 con 0x7fda4c0b61e0 2023-11-08T19:27:52.750 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:52.738+0000 7fda50b87700 1 --2- 172.21.15.143:0/309022513 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fda3006af00 0x7fda3006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:52.750 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:52.739+0000 7fda50b87700 1 --2- 172.21.15.143:0/309022513 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fda3006af00 0x7fda3006d3c0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fda480011e0 tx=0x7fda480079b0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:52.751 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:52.743+0000 7fda41ffb700 1 -- 172.21.15.143:0/309022513 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fda3c054630 con 0x7fda4c0b61e0 2023-11-08T19:27:52.947 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:52.945+0000 7fda535ec700 1 -- 172.21.15.143:0/309022513 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fda4c072150 con 0x7fda4c0b61e0 2023-11-08T19:27:52.949 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:52.948+0000 7fda41ffb700 1 -- 172.21.15.143:0/309022513 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v18) v1 ==== 104+0+0 (secure 0 0 0) 0x7fda3c057c50 con 0x7fda4c0b61e0 2023-11-08T19:27:52.952 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:52.950+0000 7fda535ec700 1 -- 172.21.15.143:0/309022513 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fda3006af00 msgr2=0x7fda3006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:52.952 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:52.950+0000 7fda535ec700 1 --2- 172.21.15.143:0/309022513 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fda3006af00 0x7fda3006d3c0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fda480011e0 tx=0x7fda480079b0 comp rx=0 tx=0).stop 2023-11-08T19:27:52.952 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:52.950+0000 7fda535ec700 1 -- 172.21.15.143:0/309022513 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fda4c0b61e0 msgr2=0x7fda4c157450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:52.953 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:52.950+0000 7fda535ec700 1 --2- 172.21.15.143:0/309022513 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fda4c0b61e0 0x7fda4c157450 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fda3c00c7f0 tx=0x7fda3c00cbb0 comp rx=0 tx=0).stop 2023-11-08T19:27:52.953 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:52.950+0000 7fda535ec700 1 -- 172.21.15.143:0/309022513 shutdown_connections 2023-11-08T19:27:52.953 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:52.950+0000 7fda535ec700 1 --2- 172.21.15.143:0/309022513 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fda3006af00 0x7fda3006d3c0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:52.953 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:52.950+0000 7fda535ec700 1 --2- 172.21.15.143:0/309022513 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fda4c0b4820 0x7fda4c0c9940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:52.953 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:52.950+0000 7fda535ec700 1 --2- 172.21.15.143:0/309022513 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fda4c0b61e0 0x7fda4c157450 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:52.954 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:52.950+0000 7fda535ec700 1 --2- 172.21.15.143:0/309022513 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fda4c15a950 0x7fda4c15c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:52.954 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:52.950+0000 7fda535ec700 1 -- 172.21.15.143:0/309022513 >> 172.21.15.143:0/309022513 conn(0x7fda4c0cd2e0 msgr2=0x7fda4c0d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:52.954 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:52.950+0000 7fda535ec700 1 -- 172.21.15.143:0/309022513 shutdown_connections 2023-11-08T19:27:52.954 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:52.950+0000 7fda535ec700 1 -- 172.21.15.143:0/309022513 wait complete. 2023-11-08T19:27:52.964 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snapshots.TestSnapshots.test_allow_new_snaps_config' 2023-11-08T19:27:53.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.083+0000 7f252b60e700 1 Processor -- start 2023-11-08T19:27:53.085 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.083+0000 7f252b60e700 1 -- start start 2023-11-08T19:27:53.087 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.084+0000 7f252b60e700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f252415dd00 0x7f25240cca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:53.087 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.084+0000 7f252b60e700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f25241618f0 0x7f252406ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:53.087 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.084+0000 7f252b60e700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25240ccfb0 con 0x7f252406b150 2023-11-08T19:27:53.088 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.084+0000 7f252b60e700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f252406b600 con 0x7f252415dd00 2023-11-08T19:27:53.088 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.084+0000 7f252b60e700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25240cad50 con 0x7f25241618f0 2023-11-08T19:27:53.088 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.084+0000 7f2528ba9700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f25241618f0 0x7f252406ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:53.088 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.084+0000 7f2528ba9700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f25241618f0 0x7f252406ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.143:41862/0 (socket says 172.21.15.143:41862) 2023-11-08T19:27:53.088 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.084+0000 7f2528ba9700 1 -- 172.21.15.143:0/3372158786 learned_addr learned my addr 172.21.15.143:0/3372158786 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:53.089 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.084+0000 7f25293aa700 1 --2- 172.21.15.143:0/3372158786 >> v2:172.21.15.174:3300/0 conn(0x7f252415dd00 0x7f25240cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:53.089 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.084+0000 7f2528ba9700 1 -- 172.21.15.143:0/3372158786 >> v2:172.21.15.174:3300/0 conn(0x7f252415dd00 msgr2=0x7f25240cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:53.089 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.084+0000 7f2528ba9700 1 --2- 172.21.15.143:0/3372158786 >> v2:172.21.15.174:3300/0 conn(0x7f252415dd00 0x7f25240cca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:53.089 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.084+0000 7f2528ba9700 1 -- 172.21.15.143:0/3372158786 >> v1:172.21.15.174:6789/0 conn(0x7f252406b150 legacy=0x7f25240b2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:53.089 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.084+0000 7f2528ba9700 1 -- 172.21.15.143:0/3372158786 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2524161df0 con 0x7f25241618f0 2023-11-08T19:27:53.090 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.085+0000 7f2528ba9700 1 --2- 172.21.15.143:0/3372158786 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f25241618f0 0x7f252406ac10 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f25140098f0 tx=0x7f2514009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=681f7cd6b5fd8c8d server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:53.090 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.085+0000 7f251bfff700 1 -- 172.21.15.143:0/3372158786 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2514011830 con 0x7f25241618f0 2023-11-08T19:27:53.090 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.085+0000 7f251bfff700 1 -- 172.21.15.143:0/3372158786 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2514011990 con 0x7f25241618f0 2023-11-08T19:27:53.090 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.085+0000 7f251bfff700 1 -- 172.21.15.143:0/3372158786 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2514011d20 con 0x7f25241618f0 2023-11-08T19:27:53.090 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.085+0000 7f252b60e700 1 -- 172.21.15.143:0/3372158786 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f25241618f0 msgr2=0x7f252406ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:53.091 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.085+0000 7f252b60e700 1 --2- 172.21.15.143:0/3372158786 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f25241618f0 0x7f252406ac10 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f25140098f0 tx=0x7f2514009c00 comp rx=0 tx=0).stop 2023-11-08T19:27:53.091 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.085+0000 7f252b60e700 1 -- 172.21.15.143:0/3372158786 shutdown_connections 2023-11-08T19:27:53.091 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.085+0000 7f252b60e700 1 --2- 172.21.15.143:0/3372158786 >> v2:172.21.15.174:3300/0 conn(0x7f252415dd00 0x7f25240cca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:53.091 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.085+0000 7f252b60e700 1 --2- 172.21.15.143:0/3372158786 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f25241618f0 0x7f252406ac10 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:53.091 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.085+0000 7f252b60e700 1 -- 172.21.15.143:0/3372158786 >> 172.21.15.143:0/3372158786 conn(0x7f2524099ab0 msgr2=0x7f25240d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:53.091 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.085+0000 7f252b60e700 1 -- 172.21.15.143:0/3372158786 shutdown_connections 2023-11-08T19:27:53.092 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.085+0000 7f252b60e700 1 -- 172.21.15.143:0/3372158786 wait complete. 2023-11-08T19:27:53.093 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.087+0000 7f252b60e700 1 Processor -- start 2023-11-08T19:27:53.093 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.087+0000 7f252b60e700 1 -- start start 2023-11-08T19:27:53.094 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.087+0000 7f252b60e700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f252406b150 0x7f252415af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:53.094 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.087+0000 7f252b60e700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f252415dd00 0x7f25240b00f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:53.094 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.087+0000 7f252b60e700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f25241618f0 0x7f25240cb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:53.095 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.087+0000 7f252b60e700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25240c8130 con 0x7f252406b150 2023-11-08T19:27:53.095 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.087+0000 7f252b60e700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25240b1350 con 0x7f252415dd00 2023-11-08T19:27:53.095 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.087+0000 7f252b60e700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25240b0ab0 con 0x7f25241618f0 2023-11-08T19:27:53.095 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.087+0000 7f25293aa700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f252406b150 0x7f252415af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:53.095 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.087+0000 7f25293aa700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f252406b150 0x7f252415af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54230/0 (socket says 172.21.15.143:54230) 2023-11-08T19:27:53.096 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.087+0000 7f25293aa700 1 -- 172.21.15.143:0/4221782814 learned_addr learned my addr 172.21.15.143:0/4221782814 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:53.096 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.088+0000 7f2529bab700 1 --2- 172.21.15.143:0/4221782814 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f25241618f0 0x7f25240cb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:53.096 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.088+0000 7f2528ba9700 1 --2- 172.21.15.143:0/4221782814 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f252415dd00 0x7f25240b00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:53.098 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.088+0000 7f25293aa700 1 -- 172.21.15.143:0/4221782814 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f25241618f0 msgr2=0x7f25240cb0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:53.098 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.088+0000 7f25293aa700 1 --2- 172.21.15.143:0/4221782814 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f25241618f0 0x7f25240cb0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:53.099 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.088+0000 7f25293aa700 1 -- 172.21.15.143:0/4221782814 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f252415dd00 msgr2=0x7f25240b00f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:53.099 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.088+0000 7f25293aa700 1 --2- 172.21.15.143:0/4221782814 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f252415dd00 0x7f25240b00f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:53.099 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.088+0000 7f25293aa700 1 -- 172.21.15.143:0/4221782814 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2514009550 con 0x7f252406b150 2023-11-08T19:27:53.099 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.088+0000 7f25293aa700 1 --2- 172.21.15.143:0/4221782814 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f252406b150 0x7f252415af40 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f252000e990 tx=0x7f252000eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:53.099 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.088+0000 7f2519ffb700 1 -- 172.21.15.143:0/4221782814 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25200096f0 con 0x7f252406b150 2023-11-08T19:27:53.100 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.088+0000 7f2519ffb700 1 -- 172.21.15.143:0/4221782814 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f252000c690 con 0x7f252406b150 2023-11-08T19:27:53.100 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.088+0000 7f252b60e700 1 -- 172.21.15.143:0/4221782814 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25240b0d90 con 0x7f252406b150 2023-11-08T19:27:53.100 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.088+0000 7f2519ffb700 1 -- 172.21.15.143:0/4221782814 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f252000ca00 con 0x7f252406b150 2023-11-08T19:27:53.100 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.088+0000 7f252b60e700 1 -- 172.21.15.143:0/4221782814 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25240c6110 con 0x7f252406b150 2023-11-08T19:27:53.100 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.089+0000 7f252b60e700 1 -- 172.21.15.143:0/4221782814 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25240aff60 con 0x7f252406b150 2023-11-08T19:27:53.101 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.090+0000 7f2519ffb700 1 -- 172.21.15.143:0/4221782814 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f252000cb60 con 0x7f252406b150 2023-11-08T19:27:53.101 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.090+0000 7f2519ffb700 1 --2- 172.21.15.143:0/4221782814 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f250806af30 0x7f250806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:53.101 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.090+0000 7f2519ffb700 1 -- 172.21.15.143:0/4221782814 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6034+0+0 (secure 0 0 0) 0x7f2520089a00 con 0x7f252406b150 2023-11-08T19:27:53.101 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.090+0000 7f2528ba9700 1 --2- 172.21.15.143:0/4221782814 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f250806af30 0x7f250806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:53.101 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.091+0000 7f2528ba9700 1 --2- 172.21.15.143:0/4221782814 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f250806af30 0x7f250806d3f0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f2514014040 tx=0x7f2514008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:53.102 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.095+0000 7f2519ffb700 1 -- 172.21.15.143:0/4221782814 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2520054970 con 0x7f252406b150 2023-11-08T19:27:53.364 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.363+0000 7f252b60e700 1 -- 172.21.15.143:0/4221782814 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snapshots.TestSnapshots.test_allow_new_snaps_config"]} v 0) v1 -- 0x7f25240c7e90 con 0x7f252406b150 2023-11-08T19:27:53.656 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.655+0000 7f2519ffb700 1 -- 172.21.15.143:0/4221782814 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snapshots.TestSnapshots.test_allow_new_snaps_config"]}]=0 v194) v1 ==== 148+0+0 (secure 0 0 0) 0x7f25240c7e90 con 0x7f252406b150 2023-11-08T19:27:53.659 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.658+0000 7f252b60e700 1 -- 172.21.15.143:0/4221782814 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f250806af30 msgr2=0x7f250806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:53.659 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.658+0000 7f252b60e700 1 --2- 172.21.15.143:0/4221782814 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f250806af30 0x7f250806d3f0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f2514014040 tx=0x7f2514008030 comp rx=0 tx=0).stop 2023-11-08T19:27:53.660 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.658+0000 7f252b60e700 1 -- 172.21.15.143:0/4221782814 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f252406b150 msgr2=0x7f252415af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:53.660 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.658+0000 7f252b60e700 1 --2- 172.21.15.143:0/4221782814 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f252406b150 0x7f252415af40 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f252000e990 tx=0x7f252000eca0 comp rx=0 tx=0).stop 2023-11-08T19:27:53.660 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.658+0000 7f252b60e700 1 -- 172.21.15.143:0/4221782814 shutdown_connections 2023-11-08T19:27:53.660 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.658+0000 7f252b60e700 1 --2- 172.21.15.143:0/4221782814 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f250806af30 0x7f250806d3f0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:53.660 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.658+0000 7f252b60e700 1 --2- 172.21.15.143:0/4221782814 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f25241618f0 0x7f25240cb0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:53.661 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.658+0000 7f252b60e700 1 --2- 172.21.15.143:0/4221782814 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f252406b150 0x7f252415af40 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:53.661 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.658+0000 7f252b60e700 1 --2- 172.21.15.143:0/4221782814 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f252415dd00 0x7f25240b00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:53.661 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.658+0000 7f252b60e700 1 -- 172.21.15.143:0/4221782814 >> 172.21.15.143:0/4221782814 conn(0x7f2524099ab0 msgr2=0x7f25240d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:53.661 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.658+0000 7f252b60e700 1 -- 172.21.15.143:0/4221782814 shutdown_connections 2023-11-08T19:27:53.661 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.658+0000 7f252b60e700 1 -- 172.21.15.143:0/4221782814 wait complete. 2023-11-08T19:27:53.671 INFO:tasks.cephfs_test_runner:Check whether 'allow_new_snaps' setting works ... ok 2023-11-08T19:27:53.671 INFO:tasks.cephfs_test_runner:Starting test: test_kill_mdstable (tasks.cephfs.test_snapshots.TestSnapshots) check snaptable transcation 2023-11-08T19:27:53.672 INFO:tasks.cephfs_test_runner:test_kill_mdstable (tasks.cephfs.test_snapshots.TestSnapshots) 2023-11-08T19:27:53.672 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:27:53.792 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.791+0000 7f30346c3700 1 Processor -- start 2023-11-08T19:27:53.793 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.791+0000 7f30346c3700 1 -- start start 2023-11-08T19:27:53.795 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.792+0000 7f30346c3700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f302c15dd00 0x7f302c0cca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:53.795 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.792+0000 7f30346c3700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f302c1618f0 0x7f302c06ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:53.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.792+0000 7f30346c3700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f302c0ccfb0 con 0x7f302c06b150 2023-11-08T19:27:53.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.792+0000 7f30346c3700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f302c06b600 con 0x7f302c1618f0 2023-11-08T19:27:53.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.792+0000 7f30346c3700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f302c0cad50 con 0x7f302c15dd00 2023-11-08T19:27:53.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.792+0000 7f3032c60700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f302c06b150 0x7f302c0b2210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:50396/0 (socket says 172.21.15.143:50396) 2023-11-08T19:27:53.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.792+0000 7f3032c60700 1 -- 172.21.15.143:0/3626988302 learned_addr learned my addr 172.21.15.143:0/3626988302 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:27:53.797 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.792+0000 7f3031c5e700 1 --2- 172.21.15.143:0/3626988302 >> v2:172.21.15.174:3300/0 conn(0x7f302c1618f0 0x7f302c06ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:53.797 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.792+0000 7f303245f700 1 --2- 172.21.15.143:0/3626988302 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f302c15dd00 0x7f302c0cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:53.797 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.792+0000 7f303145d700 1 -- 172.21.15.143:0/3626988302 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2477878220 0 0) 0x7f302c0ccfb0 con 0x7f302c06b150 2023-11-08T19:27:53.797 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.792+0000 7f303145d700 1 -- 172.21.15.143:0/3626988302 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f30180035b0 con 0x7f302c06b150 2023-11-08T19:27:53.797 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.793+0000 7f3031c5e700 1 -- 172.21.15.143:0/3626988302 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f302c15dd00 msgr2=0x7f302c0cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:53.798 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.793+0000 7f3031c5e700 1 --2- 172.21.15.143:0/3626988302 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f302c15dd00 0x7f302c0cca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:53.798 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.793+0000 7f303145d700 1 -- 172.21.15.143:0/3626988302 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2361189952 0 0) 0x7f30180035b0 con 0x7f302c06b150 2023-11-08T19:27:53.798 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.793+0000 7f3031c5e700 1 -- 172.21.15.143:0/3626988302 >> v1:172.21.15.143:6789/0 conn(0x7f302c06b150 legacy=0x7f302c0b2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:53.798 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.793+0000 7f3031c5e700 1 -- 172.21.15.143:0/3626988302 --> v2:172.21.15.174:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f302c161df0 con 0x7f302c1618f0 2023-11-08T19:27:53.798 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.793+0000 7f303245f700 1 --2- 172.21.15.143:0/3626988302 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f302c15dd00 0x7f302c0cca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T19:27:53.799 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.793+0000 7f3031c5e700 1 --2- 172.21.15.143:0/3626988302 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f302c1618f0 0x7f302c06ac10 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f3028009840 tx=0x7f3028009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=15142033e44c424e server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:53.799 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.793+0000 7f303145d700 1 -- 172.21.15.143:0/3626988302 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30240019f0 con 0x7f302c1618f0 2023-11-08T19:27:53.799 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.793+0000 7f303145d700 1 -- 172.21.15.143:0/3626988302 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3028004030 con 0x7f302c1618f0 2023-11-08T19:27:53.799 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.793+0000 7f303145d700 1 -- 172.21.15.143:0/3626988302 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30280043c0 con 0x7f302c1618f0 2023-11-08T19:27:53.800 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.794+0000 7f30346c3700 1 -- 172.21.15.143:0/3626988302 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f302c1618f0 msgr2=0x7f302c06ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:53.800 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.794+0000 7f30346c3700 1 --2- 172.21.15.143:0/3626988302 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f302c1618f0 0x7f302c06ac10 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f3028009840 tx=0x7f3028009b50 comp rx=0 tx=0).stop 2023-11-08T19:27:53.800 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.794+0000 7f30346c3700 1 -- 172.21.15.143:0/3626988302 shutdown_connections 2023-11-08T19:27:53.800 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.794+0000 7f30346c3700 1 --2- 172.21.15.143:0/3626988302 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f302c15dd00 0x7f302c0cca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:53.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.794+0000 7f30346c3700 1 --2- 172.21.15.143:0/3626988302 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f302c1618f0 0x7f302c06ac10 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:53.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.794+0000 7f30346c3700 1 -- 172.21.15.143:0/3626988302 >> 172.21.15.143:0/3626988302 conn(0x7f302c099ab0 msgr2=0x7f302c0d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:53.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.794+0000 7f30346c3700 1 -- 172.21.15.143:0/3626988302 shutdown_connections 2023-11-08T19:27:53.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.794+0000 7f30346c3700 1 -- 172.21.15.143:0/3626988302 wait complete. 2023-11-08T19:27:53.804 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.795+0000 7f30346c3700 1 Processor -- start 2023-11-08T19:27:53.804 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.795+0000 7f30346c3700 1 -- start start 2023-11-08T19:27:53.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.796+0000 7f30346c3700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f302c06b150 0x7f302c1545f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:53.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.796+0000 7f30346c3700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f302c15dd00 0x7f302c0c77d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:53.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.796+0000 7f30346c3700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f302c1618f0 0x7f302c0c7d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:53.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.796+0000 7f30346c3700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f302c0ca110 con 0x7f302c1618f0 2023-11-08T19:27:53.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.796+0000 7f30346c3700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f302c1551e0 con 0x7f302c15dd00 2023-11-08T19:27:53.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.796+0000 7f30346c3700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f302c0c9490 con 0x7f302c06b150 2023-11-08T19:27:53.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.796+0000 7f3032c60700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f302c1618f0 0x7f302c0c7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:53.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.796+0000 7f3032c60700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f302c1618f0 0x7f302c0c7d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54248/0 (socket says 172.21.15.143:54248) 2023-11-08T19:27:53.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.796+0000 7f3032c60700 1 -- 172.21.15.143:0/994419399 learned_addr learned my addr 172.21.15.143:0/994419399 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:53.807 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.796+0000 7f3031c5e700 1 --2- 172.21.15.143:0/994419399 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f302c15dd00 0x7f302c0c77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:53.807 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.796+0000 7f303245f700 1 --2- 172.21.15.143:0/994419399 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f302c06b150 0x7f302c1545f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:53.807 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.796+0000 7f3032c60700 1 -- 172.21.15.143:0/994419399 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f302c06b150 msgr2=0x7f302c1545f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:53.807 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.796+0000 7f3032c60700 1 --2- 172.21.15.143:0/994419399 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f302c06b150 0x7f302c1545f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:53.807 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.796+0000 7f3032c60700 1 -- 172.21.15.143:0/994419399 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f302c15dd00 msgr2=0x7f302c0c77d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:53.808 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.796+0000 7f3032c60700 1 --2- 172.21.15.143:0/994419399 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f302c15dd00 0x7f302c0c77d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:53.808 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.796+0000 7f3032c60700 1 -- 172.21.15.143:0/994419399 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f301c009430 con 0x7f302c1618f0 2023-11-08T19:27:53.808 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.796+0000 7f3032c60700 1 --2- 172.21.15.143:0/994419399 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f302c1618f0 0x7f302c0c7d10 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f302400cca0 tx=0x7f302400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:53.808 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.796+0000 7f3022ffd700 1 -- 172.21.15.143:0/994419399 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f302400ab70 con 0x7f302c1618f0 2023-11-08T19:27:53.808 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.796+0000 7f3022ffd700 1 -- 172.21.15.143:0/994419399 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f302400acf0 con 0x7f302c1618f0 2023-11-08T19:27:53.809 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.796+0000 7f30346c3700 1 -- 172.21.15.143:0/994419399 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30280094a0 con 0x7f302c1618f0 2023-11-08T19:27:53.809 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.796+0000 7f3022ffd700 1 -- 172.21.15.143:0/994419399 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3024007310 con 0x7f302c1618f0 2023-11-08T19:27:53.809 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.796+0000 7f30346c3700 1 -- 172.21.15.143:0/994419399 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f302c0c9a50 con 0x7f302c1618f0 2023-11-08T19:27:53.809 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.797+0000 7f30346c3700 1 -- 172.21.15.143:0/994419399 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f302c0aff60 con 0x7f302c1618f0 2023-11-08T19:27:53.809 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.798+0000 7f3022ffd700 1 -- 172.21.15.143:0/994419399 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3024007470 con 0x7f302c1618f0 2023-11-08T19:27:53.810 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.798+0000 7f3022ffd700 1 --2- 172.21.15.143:0/994419399 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f301006af00 0x7f301006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:53.810 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.798+0000 7f3022ffd700 1 -- 172.21.15.143:0/994419399 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6034+0+0 (secure 0 0 0) 0x7f30240895e0 con 0x7f302c1618f0 2023-11-08T19:27:53.810 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.799+0000 7f303245f700 1 --2- 172.21.15.143:0/994419399 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f301006af00 0x7f301006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:53.810 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.799+0000 7f303245f700 1 --2- 172.21.15.143:0/994419399 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f301006af00 0x7f301006d3c0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f301c009d50 tx=0x7f301c0091c0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:53.810 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.803+0000 7f3022ffd700 1 -- 172.21.15.143:0/994419399 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3024054530 con 0x7f302c1618f0 2023-11-08T19:27:53.993 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.992+0000 7f30346c3700 1 -- 172.21.15.143:0/994419399 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f302c0ca250 con 0x7f302c1618f0 2023-11-08T19:27:53.994 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.993+0000 7f3022ffd700 1 -- 172.21.15.143:0/994419399 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+18325 (secure 0 0 0) 0x7f302c0ca250 con 0x7f302c1618f0 2023-11-08T19:27:53.995 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:27:53.995 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":84,"fsid":"968483af-77fc-4068-ad7c-5c2a1a4613d7","created":"2023-11-08T19:24:32.826484+0000","modified":"2023-11-08T19:27:52.588541+0000","last_up_change":"2023-11-08T19:24:43.573880+0000","last_in_change":"2023-11-08T19:24:35.314525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:44.551530+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"snap-test-pool","create_time":"2023-11-08T19:26:02.874384+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":36,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"test_data_pool","create_time":"2023-11-08T19:27:05.601949+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":66,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_metadata_pool","create_time":"2023-11-08T19:27:06.536369+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bae98ec1-d94e-403d-a220-a522a53ea6f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6809","nonce":1487830566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6811","nonce":1487830566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6815","nonce":1487830566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6813","nonce":1487830566}]},"public_addr":"172.21.15.143:6809/1487830566","cluster_addr":"172.21.15.143:6811/1487830566","heartbeat_back_addr":"172.21.15.143:6815/1487830566","heartbeat_front_addr":"172.21.15.143:6813/1487830566","state":["exists","up"]},{"osd":1,"uuid":"5cb6538a-1ccb-4322-8633-09f66af0b7e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6801","nonce":787341397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6803","nonce":787341397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6807","nonce":787341397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6805","nonce":787341397}]},"public_addr":"172.21.15.143:6801/787341397","cluster_addr":"172.21.15.143:6803/787341397","heartbeat_back_addr":"172.21.15.143:6807/787341397","heartbeat_front_addr":"172.21.15.143:6805/787341397","state":["exists","up"]},{"osd":2,"uuid":"cdbf522c-60ef-4d60-9653-8dcb529c101a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6822","nonce":2301325597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6826","nonce":2301325597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6831","nonce":2301325597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6829","nonce":2301325597}]},"public_addr":"172.21.15.143:6822/2301325597","cluster_addr":"172.21.15.143:6826/2301325597","heartbeat_back_addr":"172.21.15.143:6831/2301325597","heartbeat_front_addr":"172.21.15.143:6829/2301325597","state":["exists","up"]},{"osd":3,"uuid":"4e2783cf-ebb9-4dd8-8245-bc1d17ce4325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6817","nonce":2021504428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6819","nonce":2021504428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6827","nonce":2021504428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6823","nonce":2021504428}]},"public_addr":"172.21.15.143:6817/2021504428","cluster_addr":"172.21.15.143:6819/2021504428","heartbeat_back_addr":"172.21.15.143:6827/2021504428","heartbeat_front_addr":"172.21.15.143:6823/2021504428","state":["exists","up"]},{"osd":4,"uuid":"691256fa-9b77-49b9-a346-7ae1adf0ca80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6801","nonce":1993380595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6803","nonce":1993380595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6807","nonce":1993380595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6805","nonce":1993380595}]},"public_addr":"172.21.15.174:6801/1993380595","cluster_addr":"172.21.15.174:6803/1993380595","heartbeat_back_addr":"172.21.15.174:6807/1993380595","heartbeat_front_addr":"172.21.15.174:6805/1993380595","state":["exists","up"]},{"osd":5,"uuid":"756fc352-48c9-46ce-b52e-0541fe6f8525","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6809","nonce":1777923303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6811","nonce":1777923303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6815","nonce":1777923303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6813","nonce":1777923303}]},"public_addr":"172.21.15.174:6809/1777923303","cluster_addr":"172.21.15.174:6811/1777923303","heartbeat_back_addr":"172.21.15.174:6815/1777923303","heartbeat_front_addr":"172.21.15.174:6813/1777923303","state":["exists","up"]},{"osd":6,"uuid":"fab2ac2b-ed97-4cde-9825-b480eabf8b8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6825","nonce":2755130316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6827","nonce":2755130316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6831","nonce":2755130316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6829","nonce":2755130316}]},"public_addr":"172.21.15.174:6825/2755130316","cluster_addr":"172.21.15.174:6827/2755130316","heartbeat_back_addr":"172.21.15.174:6831/2755130316","heartbeat_front_addr":"172.21.15.174:6829/2755130316","state":["exists","up"]},{"osd":7,"uuid":"3de9b797-2ab6-4d41-a863-7e7d638fd703","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6817","nonce":2847141411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6819","nonce":2847141411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6823","nonce":2847141411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6821","nonce":2847141411}]},"public_addr":"172.21.15.174:6817/2847141411","cluster_addr":"172.21.15.174:6819/2847141411","heartbeat_back_addr":"172.21.15.174:6823/2847141411","heartbeat_front_addr":"172.21.15.174:6821/2847141411","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.477076+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.496890+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.539305+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.454673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.538881+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.521294+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.520712+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.540404+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.174:6837/1950356087":"2023-11-09T19:27:49.641952+0000","172.21.15.174:6835/1950356087":"2023-11-09T19:27:49.641952+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":14,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:27:53.997 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.995+0000 7f30346c3700 1 -- 172.21.15.143:0/994419399 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f301006af00 msgr2=0x7f301006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:53.997 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.995+0000 7f30346c3700 1 --2- 172.21.15.143:0/994419399 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f301006af00 0x7f301006d3c0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f301c009d50 tx=0x7f301c0091c0 comp rx=0 tx=0).stop 2023-11-08T19:27:53.997 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.995+0000 7f30346c3700 1 -- 172.21.15.143:0/994419399 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f302c1618f0 msgr2=0x7f302c0c7d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:53.998 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.995+0000 7f30346c3700 1 --2- 172.21.15.143:0/994419399 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f302c1618f0 0x7f302c0c7d10 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f302400cca0 tx=0x7f302400a060 comp rx=0 tx=0).stop 2023-11-08T19:27:53.998 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.995+0000 7f30346c3700 1 -- 172.21.15.143:0/994419399 shutdown_connections 2023-11-08T19:27:53.998 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.995+0000 7f30346c3700 1 --2- 172.21.15.143:0/994419399 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f301006af00 0x7f301006d3c0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:53.998 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.995+0000 7f30346c3700 1 --2- 172.21.15.143:0/994419399 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f302c06b150 0x7f302c1545f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:53.998 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.995+0000 7f30346c3700 1 --2- 172.21.15.143:0/994419399 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f302c1618f0 0x7f302c0c7d10 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:53.999 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.995+0000 7f30346c3700 1 --2- 172.21.15.143:0/994419399 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f302c15dd00 0x7f302c0c77d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:53.999 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.995+0000 7f30346c3700 1 -- 172.21.15.143:0/994419399 >> 172.21.15.143:0/994419399 conn(0x7f302c099ab0 msgr2=0x7f302c0ad380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:53.999 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.995+0000 7f30346c3700 1 -- 172.21.15.143:0/994419399 shutdown_connections 2023-11-08T19:27:53.999 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:53.995+0000 7f30346c3700 1 -- 172.21.15.143:0/994419399 wait complete. 2023-11-08T19:27:54.009 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T19:24:44.551530+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'snap-test-pool', 'create_time': '2023-11-08T19:26:02.874384+0000', 'flags': 16385, 'flags_names': 'hashpspool,pool_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '36', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'pool', 'snap_seq': 2, 'snap_epoch': 36, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'test_data_pool', 'create_time': '2023-11-08T19:27:05.601949+0000', 'flags': 16385, 'flags_names': 'hashpspool,pool_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'pool', 'snap_seq': 2, 'snap_epoch': 66, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'test_metadata_pool', 'create_time': '2023-11-08T19:27:06.536369+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-08T19:27:54.010 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T19:27:54.133 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.131+0000 7f6d67315700 1 Processor -- start 2023-11-08T19:27:54.133 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.132+0000 7f6d67315700 1 -- start start 2023-11-08T19:27:54.135 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.132+0000 7f6d67315700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f6d6015dba0 0x7f6d6006c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:54.135 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.132+0000 7f6d67315700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f6d6006b240 0x7f6d600caff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:54.136 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.132+0000 7f6d67315700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d60161890 con 0x7f6d600cb530 2023-11-08T19:27:54.136 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.132+0000 7f6d67315700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d6006b740 con 0x7f6d6015dba0 2023-11-08T19:27:54.136 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.132+0000 7f6d67315700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d6006ba20 con 0x7f6d6006b240 2023-11-08T19:27:54.136 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.132+0000 7f6d650b1700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f6d6015dba0 0x7f6d6006c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:54.137 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.132+0000 7f6d658b2700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f6d600cb530 0x7f6d600b22e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:50410/0 (socket says 172.21.15.143:50410) 2023-11-08T19:27:54.137 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.133+0000 7f6d658b2700 1 -- 172.21.15.143:0/877656767 learned_addr learned my addr 172.21.15.143:0/877656767 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:27:54.137 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.133+0000 7f6d648b0700 1 --2- 172.21.15.143:0/877656767 >> v2:172.21.15.174:3300/0 conn(0x7f6d6006b240 0x7f6d600caff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:54.137 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.133+0000 7f6d650b1700 1 -- 172.21.15.143:0/877656767 >> v2:172.21.15.174:3300/0 conn(0x7f6d6006b240 msgr2=0x7f6d600caff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:54.137 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.133+0000 7f6d650b1700 1 --2- 172.21.15.143:0/877656767 >> v2:172.21.15.174:3300/0 conn(0x7f6d6006b240 0x7f6d600caff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:54.138 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.133+0000 7f6d650b1700 1 -- 172.21.15.143:0/877656767 >> v1:172.21.15.143:6789/0 conn(0x7f6d600cb530 legacy=0x7f6d600b22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:54.138 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.133+0000 7f6d57fff700 1 -- 172.21.15.143:0/877656767 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4178367746 0 0) 0x7f6d60161890 con 0x7f6d600cb530 2023-11-08T19:27:54.138 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.133+0000 7f6d650b1700 1 -- 172.21.15.143:0/877656767 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d6006bd00 con 0x7f6d6015dba0 2023-11-08T19:27:54.138 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.133+0000 7f6d648b0700 1 --2- 172.21.15.143:0/877656767 >> v2:172.21.15.174:3300/0 conn(0x7f6d6006b240 0x7f6d600caff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:27:54.138 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.133+0000 7f6d650b1700 1 --2- 172.21.15.143:0/877656767 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d6015dba0 0x7f6d6006c610 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f6d600d0810 tx=0x7f6d50009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cedaad57b210eba server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:54.139 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.133+0000 7f6d57fff700 1 -- 172.21.15.143:0/877656767 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d50011830 con 0x7f6d6015dba0 2023-11-08T19:27:54.139 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.133+0000 7f6d57fff700 1 -- 172.21.15.143:0/877656767 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d50011990 con 0x7f6d6015dba0 2023-11-08T19:27:54.139 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.133+0000 7f6d57fff700 1 -- 172.21.15.143:0/877656767 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d50011d20 con 0x7f6d6015dba0 2023-11-08T19:27:54.139 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.134+0000 7f6d67315700 1 -- 172.21.15.143:0/877656767 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d6015dba0 msgr2=0x7f6d6006c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:54.139 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.134+0000 7f6d67315700 1 --2- 172.21.15.143:0/877656767 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d6015dba0 0x7f6d6006c610 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f6d600d0810 tx=0x7f6d50009b20 comp rx=0 tx=0).stop 2023-11-08T19:27:54.140 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.134+0000 7f6d67315700 1 -- 172.21.15.143:0/877656767 shutdown_connections 2023-11-08T19:27:54.140 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.134+0000 7f6d67315700 1 --2- 172.21.15.143:0/877656767 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d6015dba0 0x7f6d6006c610 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:54.140 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.134+0000 7f6d67315700 1 --2- 172.21.15.143:0/877656767 >> v2:172.21.15.174:3300/0 conn(0x7f6d6006b240 0x7f6d600caff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:54.140 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.134+0000 7f6d67315700 1 -- 172.21.15.143:0/877656767 >> 172.21.15.143:0/877656767 conn(0x7f6d600cd2e0 msgr2=0x7f6d600cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:54.140 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.134+0000 7f6d67315700 1 -- 172.21.15.143:0/877656767 shutdown_connections 2023-11-08T19:27:54.141 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.134+0000 7f6d67315700 1 -- 172.21.15.143:0/877656767 wait complete. 2023-11-08T19:27:54.143 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.135+0000 7f6d67315700 1 Processor -- start 2023-11-08T19:27:54.143 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.135+0000 7f6d67315700 1 -- start start 2023-11-08T19:27:54.144 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.136+0000 7f6d67315700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6d6006b240 0x7f6d600c7070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:54.144 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.136+0000 7f6d67315700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d600cb530 0x7f6d600b3480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:54.144 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.136+0000 7f6d67315700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6d6015dba0 0x7f6d6015a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:54.144 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.136+0000 7f6d67315700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d600b3cd0 con 0x7f6d600cb530 2023-11-08T19:27:54.145 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.136+0000 7f6d67315700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d600b3fb0 con 0x7f6d6006b240 2023-11-08T19:27:54.145 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.136+0000 7f6d67315700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d600c4fa0 con 0x7f6d6015dba0 2023-11-08T19:27:54.145 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.136+0000 7f6d648b0700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d600cb530 0x7f6d600b3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:54.145 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.136+0000 7f6d648b0700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d600cb530 0x7f6d600b3480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54260/0 (socket says 172.21.15.143:54260) 2023-11-08T19:27:54.145 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.136+0000 7f6d648b0700 1 -- 172.21.15.143:0/3474453843 learned_addr learned my addr 172.21.15.143:0/3474453843 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:54.146 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.136+0000 7f6d650b1700 1 --2- 172.21.15.143:0/3474453843 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6d6006b240 0x7f6d600c7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:54.146 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.136+0000 7f6d658b2700 1 --2- 172.21.15.143:0/3474453843 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6d6015dba0 0x7f6d6015a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:54.146 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.136+0000 7f6d648b0700 1 -- 172.21.15.143:0/3474453843 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6d6015dba0 msgr2=0x7f6d6015a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:54.146 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.136+0000 7f6d648b0700 1 --2- 172.21.15.143:0/3474453843 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6d6015dba0 0x7f6d6015a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:54.146 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.136+0000 7f6d648b0700 1 -- 172.21.15.143:0/3474453843 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6d6006b240 msgr2=0x7f6d600c7070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:54.147 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.136+0000 7f6d648b0700 1 --2- 172.21.15.143:0/3474453843 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6d6006b240 0x7f6d600c7070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:54.147 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.136+0000 7f6d648b0700 1 -- 172.21.15.143:0/3474453843 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d500094a0 con 0x7f6d600cb530 2023-11-08T19:27:54.147 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.136+0000 7f6d648b0700 1 --2- 172.21.15.143:0/3474453843 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d600cb530 0x7f6d600b3480 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c00e810 tx=0x7f6d5c00ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:54.147 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.136+0000 7f6d55ffb700 1 -- 172.21.15.143:0/3474453843 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d5c004230 con 0x7f6d600cb530 2023-11-08T19:27:54.147 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.136+0000 7f6d55ffb700 1 -- 172.21.15.143:0/3474453843 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d5c004390 con 0x7f6d600cb530 2023-11-08T19:27:54.148 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.136+0000 7f6d67315700 1 -- 172.21.15.143:0/3474453843 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d600c5280 con 0x7f6d600cb530 2023-11-08T19:27:54.148 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.136+0000 7f6d55ffb700 1 -- 172.21.15.143:0/3474453843 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d5c0036a0 con 0x7f6d600cb530 2023-11-08T19:27:54.148 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.137+0000 7f6d67315700 1 -- 172.21.15.143:0/3474453843 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d6015af20 con 0x7f6d600cb530 2023-11-08T19:27:54.148 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.137+0000 7f6d67315700 1 -- 172.21.15.143:0/3474453843 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d6004eee0 con 0x7f6d600cb530 2023-11-08T19:27:54.148 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.138+0000 7f6d55ffb700 1 -- 172.21.15.143:0/3474453843 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6d5c003800 con 0x7f6d600cb530 2023-11-08T19:27:54.149 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.138+0000 7f6d55ffb700 1 --2- 172.21.15.143:0/3474453843 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f6d4406af00 0x7f6d4406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:54.149 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.138+0000 7f6d55ffb700 1 -- 172.21.15.143:0/3474453843 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6034+0+0 (secure 0 0 0) 0x7f6d5c089200 con 0x7f6d600cb530 2023-11-08T19:27:54.149 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.138+0000 7f6d650b1700 1 --2- 172.21.15.143:0/3474453843 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f6d4406af00 0x7f6d4406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:54.149 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.139+0000 7f6d650b1700 1 --2- 172.21.15.143:0/3474453843 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f6d4406af00 0x7f6d4406d3c0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f6d500090b0 tx=0x7f6d5001c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:54.150 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.143+0000 7f6d55ffb700 1 -- 172.21.15.143:0/3474453843 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6d5c054160 con 0x7f6d600cb530 2023-11-08T19:27:54.330 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.328+0000 7f6d67315700 1 -- 172.21.15.143:0/3474453843 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6d600b67e0 con 0x7f6d600cb530 2023-11-08T19:27:54.330 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.329+0000 7f6d55ffb700 1 -- 172.21.15.143:0/3474453843 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6d5c057780 con 0x7f6d600cb530 2023-11-08T19:27:54.331 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2023-11-08T19:27:54.333 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.331+0000 7f6d67315700 1 -- 172.21.15.143:0/3474453843 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f6d4406af00 msgr2=0x7f6d4406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:54.333 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.331+0000 7f6d67315700 1 --2- 172.21.15.143:0/3474453843 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f6d4406af00 0x7f6d4406d3c0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f6d500090b0 tx=0x7f6d5001c040 comp rx=0 tx=0).stop 2023-11-08T19:27:54.333 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.331+0000 7f6d67315700 1 -- 172.21.15.143:0/3474453843 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d600cb530 msgr2=0x7f6d600b3480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:54.333 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.331+0000 7f6d67315700 1 --2- 172.21.15.143:0/3474453843 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d600cb530 0x7f6d600b3480 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c00e810 tx=0x7f6d5c00ebd0 comp rx=0 tx=0).stop 2023-11-08T19:27:54.333 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.331+0000 7f6d67315700 1 -- 172.21.15.143:0/3474453843 shutdown_connections 2023-11-08T19:27:54.334 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.331+0000 7f6d67315700 1 --2- 172.21.15.143:0/3474453843 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f6d4406af00 0x7f6d4406d3c0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:54.334 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.331+0000 7f6d67315700 1 --2- 172.21.15.143:0/3474453843 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6d6015dba0 0x7f6d6015a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:54.334 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.331+0000 7f6d67315700 1 --2- 172.21.15.143:0/3474453843 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6d6006b240 0x7f6d600c7070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:54.334 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.331+0000 7f6d67315700 1 --2- 172.21.15.143:0/3474453843 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d600cb530 0x7f6d600b3480 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:54.334 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.331+0000 7f6d67315700 1 -- 172.21.15.143:0/3474453843 >> 172.21.15.143:0/3474453843 conn(0x7f6d600cd2e0 msgr2=0x7f6d60072030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:54.335 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.331+0000 7f6d67315700 1 -- 172.21.15.143:0/3474453843 shutdown_connections 2023-11-08T19:27:54.335 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.331+0000 7f6d67315700 1 -- 172.21.15.143:0/3474453843 wait complete. 2023-11-08T19:27:54.345 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get snap-test-pool pg_num 2023-11-08T19:27:54.468 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.466+0000 7f14922f0700 1 Processor -- start 2023-11-08T19:27:54.468 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.466+0000 7f14922f0700 1 -- start start 2023-11-08T19:27:54.471 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.467+0000 7f14922f0700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f148c15dba0 0x7f148c06c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:54.471 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.467+0000 7f14922f0700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f148c06b240 0x7f148c0caff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:54.471 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.467+0000 7f14922f0700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f148c0cb530 0x7f148c0b22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:54.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.467+0000 7f14922f0700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f148c161890 con 0x7f148c0cb530 2023-11-08T19:27:54.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.467+0000 7f14922f0700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f148c06b740 con 0x7f148c15dba0 2023-11-08T19:27:54.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.467+0000 7f14922f0700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f148c06ba20 con 0x7f148c06b240 2023-11-08T19:27:54.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.467+0000 7f149088d700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f148c0cb530 0x7f148c0b22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:54.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.467+0000 7f149088d700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f148c0cb530 0x7f148c0b22e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54270/0 (socket says 172.21.15.143:54270) 2023-11-08T19:27:54.473 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.467+0000 7f149088d700 1 -- 172.21.15.143:0/965241097 learned_addr learned my addr 172.21.15.143:0/965241097 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:54.473 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.468+0000 7f148bfff700 1 --2- 172.21.15.143:0/965241097 >> v2:172.21.15.174:3300/0 conn(0x7f148c15dba0 0x7f148c06c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:54.473 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.468+0000 7f148b7fe700 1 --2- 172.21.15.143:0/965241097 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f148c06b240 0x7f148c0caff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:54.473 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.468+0000 7f149088d700 1 -- 172.21.15.143:0/965241097 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f148c06b240 msgr2=0x7f148c0caff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:54.474 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.468+0000 7f149088d700 1 --2- 172.21.15.143:0/965241097 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f148c06b240 0x7f148c0caff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:54.474 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.468+0000 7f149088d700 1 -- 172.21.15.143:0/965241097 >> v2:172.21.15.174:3300/0 conn(0x7f148c15dba0 msgr2=0x7f148c06c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:54.474 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.468+0000 7f149088d700 1 --2- 172.21.15.143:0/965241097 >> v2:172.21.15.174:3300/0 conn(0x7f148c15dba0 0x7f148c06c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:54.474 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.468+0000 7f149088d700 1 -- 172.21.15.143:0/965241097 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f148c06bd00 con 0x7f148c0cb530 2023-11-08T19:27:54.474 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.468+0000 7f149088d700 1 --2- 172.21.15.143:0/965241097 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f148c0cb530 0x7f148c0b22e0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f148c067460 tx=0x7f148400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb16a6e02bf3eb7e server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:54.475 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.468+0000 7f148affd700 1 -- 172.21.15.143:0/965241097 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f148400e070 con 0x7f148c0cb530 2023-11-08T19:27:54.475 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.468+0000 7f148affd700 1 -- 172.21.15.143:0/965241097 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f148400b9c0 con 0x7f148c0cb530 2023-11-08T19:27:54.475 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.468+0000 7f148affd700 1 -- 172.21.15.143:0/965241097 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f148400bd70 con 0x7f148c0cb530 2023-11-08T19:27:54.475 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.468+0000 7f14922f0700 1 -- 172.21.15.143:0/965241097 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f148c0cb530 msgr2=0x7f148c0b22e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:54.475 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.468+0000 7f14922f0700 1 --2- 172.21.15.143:0/965241097 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f148c0cb530 0x7f148c0b22e0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f148c067460 tx=0x7f148400b330 comp rx=0 tx=0).stop 2023-11-08T19:27:54.475 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.469+0000 7f14922f0700 1 -- 172.21.15.143:0/965241097 shutdown_connections 2023-11-08T19:27:54.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.469+0000 7f14922f0700 1 --2- 172.21.15.143:0/965241097 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f148c0cb530 0x7f148c0b22e0 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:54.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.469+0000 7f14922f0700 1 --2- 172.21.15.143:0/965241097 >> v2:172.21.15.174:3300/0 conn(0x7f148c15dba0 0x7f148c06c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:54.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.469+0000 7f14922f0700 1 --2- 172.21.15.143:0/965241097 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f148c06b240 0x7f148c0caff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:54.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.469+0000 7f14922f0700 1 -- 172.21.15.143:0/965241097 >> 172.21.15.143:0/965241097 conn(0x7f148c0cd2e0 msgr2=0x7f148c0cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:54.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.469+0000 7f14922f0700 1 -- 172.21.15.143:0/965241097 shutdown_connections 2023-11-08T19:27:54.477 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.469+0000 7f14922f0700 1 -- 172.21.15.143:0/965241097 wait complete. 2023-11-08T19:27:54.477 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.470+0000 7f14922f0700 1 Processor -- start 2023-11-08T19:27:54.477 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.470+0000 7f14922f0700 1 -- start start 2023-11-08T19:27:54.480 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.471+0000 7f14922f0700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f148c06b240 0x7f148c14b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:54.480 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.471+0000 7f14922f0700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f148c0cb530 0x7f148c0c7f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:54.480 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.471+0000 7f14922f0700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f148c15dba0 0x7f148c0b02c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:54.480 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.471+0000 7f14922f0700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f148c14cf60 con 0x7f148c06b240 2023-11-08T19:27:54.481 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.471+0000 7f14922f0700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f148c0c8490 con 0x7f148c15dba0 2023-11-08T19:27:54.481 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.471+0000 7f14922f0700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f148c14c2e0 con 0x7f148c0cb530 2023-11-08T19:27:54.481 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.471+0000 7f148bfff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f148c06b240 0x7f148c14b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:54.481 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.471+0000 7f148bfff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f148c06b240 0x7f148c14b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54272/0 (socket says 172.21.15.143:54272) 2023-11-08T19:27:54.481 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.471+0000 7f148bfff700 1 -- 172.21.15.143:0/2334486417 learned_addr learned my addr 172.21.15.143:0/2334486417 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:54.482 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.471+0000 7f149088d700 1 --2- 172.21.15.143:0/2334486417 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f148c15dba0 0x7f148c0b02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:54.482 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.471+0000 7f148b7fe700 1 --2- 172.21.15.143:0/2334486417 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f148c0cb530 0x7f148c0c7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:54.482 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.471+0000 7f148bfff700 1 -- 172.21.15.143:0/2334486417 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f148c0cb530 msgr2=0x7f148c0c7f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:54.482 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.471+0000 7f148bfff700 1 --2- 172.21.15.143:0/2334486417 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f148c0cb530 0x7f148c0c7f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:54.482 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.471+0000 7f148bfff700 1 -- 172.21.15.143:0/2334486417 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f148c15dba0 msgr2=0x7f148c0b02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:54.483 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.471+0000 7f148bfff700 1 --2- 172.21.15.143:0/2334486417 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f148c15dba0 0x7f148c0b02c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:54.483 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.471+0000 7f148bfff700 1 -- 172.21.15.143:0/2334486417 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1484009b30 con 0x7f148c06b240 2023-11-08T19:27:54.483 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.471+0000 7f149088d700 1 --2- 172.21.15.143:0/2334486417 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f148c15dba0 0x7f148c0b02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:27:54.483 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.471+0000 7f148bfff700 1 --2- 172.21.15.143:0/2334486417 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f148c06b240 0x7f148c14b9f0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f148000d800 tx=0x7f148000dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:54.483 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.471+0000 7f1488ff9700 1 -- 172.21.15.143:0/2334486417 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14800125f0 con 0x7f148c06b240 2023-11-08T19:27:54.484 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.471+0000 7f1488ff9700 1 -- 172.21.15.143:0/2334486417 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1480012750 con 0x7f148c06b240 2023-11-08T19:27:54.484 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.471+0000 7f14922f0700 1 -- 172.21.15.143:0/2334486417 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f148c14c5c0 con 0x7f148c06b240 2023-11-08T19:27:54.484 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.471+0000 7f1488ff9700 1 -- 172.21.15.143:0/2334486417 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1480012b00 con 0x7f148c06b240 2023-11-08T19:27:54.484 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.471+0000 7f14922f0700 1 -- 172.21.15.143:0/2334486417 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f148c14cb80 con 0x7f148c06b240 2023-11-08T19:27:54.484 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.472+0000 7f14922f0700 1 -- 172.21.15.143:0/2334486417 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f148c14d1d0 con 0x7f148c06b240 2023-11-08T19:27:54.485 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.473+0000 7f1488ff9700 1 -- 172.21.15.143:0/2334486417 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1480012c60 con 0x7f148c06b240 2023-11-08T19:27:54.485 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.473+0000 7f1488ff9700 1 --2- 172.21.15.143:0/2334486417 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f147806af00 0x7f147806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:54.485 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.473+0000 7f1488ff9700 1 -- 172.21.15.143:0/2334486417 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6034+0+0 (secure 0 0 0) 0x7f1480089a90 con 0x7f148c06b240 2023-11-08T19:27:54.485 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.473+0000 7f148b7fe700 1 --2- 172.21.15.143:0/2334486417 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f147806af00 0x7f147806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:54.485 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.474+0000 7f148b7fe700 1 --2- 172.21.15.143:0/2334486417 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f147806af00 0x7f147806d3c0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f148c0caef0 tx=0x7f147c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:54.486 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.478+0000 7f1488ff9700 1 -- 172.21.15.143:0/2334486417 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1480054a80 con 0x7f148c06b240 2023-11-08T19:27:54.668 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.666+0000 7f14922f0700 1 -- 172.21.15.143:0/2334486417 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"} v 0) v1 -- 0x7f148c04eee0 con 0x7f148c06b240 2023-11-08T19:27:54.668 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.666+0000 7f1488ff9700 1 -- 172.21.15.143:0/2334486417 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"}]=0 v84) v1 ==== 103+0+11 (secure 0 0 0) 0x7f14800580a0 con 0x7f148c06b240 2023-11-08T19:27:54.668 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2023-11-08T19:27:54.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.669+0000 7f14922f0700 1 -- 172.21.15.143:0/2334486417 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f147806af00 msgr2=0x7f147806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:54.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.669+0000 7f14922f0700 1 --2- 172.21.15.143:0/2334486417 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f147806af00 0x7f147806d3c0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f148c0caef0 tx=0x7f147c008040 comp rx=0 tx=0).stop 2023-11-08T19:27:54.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.669+0000 7f14922f0700 1 -- 172.21.15.143:0/2334486417 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f148c06b240 msgr2=0x7f148c14b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:54.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.669+0000 7f14922f0700 1 --2- 172.21.15.143:0/2334486417 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f148c06b240 0x7f148c14b9f0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f148000d800 tx=0x7f148000dbc0 comp rx=0 tx=0).stop 2023-11-08T19:27:54.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.669+0000 7f14922f0700 1 -- 172.21.15.143:0/2334486417 shutdown_connections 2023-11-08T19:27:54.672 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.669+0000 7f14922f0700 1 --2- 172.21.15.143:0/2334486417 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f147806af00 0x7f147806d3c0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:54.672 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.669+0000 7f14922f0700 1 --2- 172.21.15.143:0/2334486417 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f148c15dba0 0x7f148c0b02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:54.672 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.669+0000 7f14922f0700 1 --2- 172.21.15.143:0/2334486417 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f148c06b240 0x7f148c14b9f0 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:54.672 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.669+0000 7f14922f0700 1 --2- 172.21.15.143:0/2334486417 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f148c0cb530 0x7f148c0c7f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:54.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.669+0000 7f14922f0700 1 -- 172.21.15.143:0/2334486417 >> 172.21.15.143:0/2334486417 conn(0x7f148c0cd2e0 msgr2=0x7f148c15cd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:54.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.669+0000 7f14922f0700 1 -- 172.21.15.143:0/2334486417 shutdown_connections 2023-11-08T19:27:54.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.669+0000 7f14922f0700 1 -- 172.21.15.143:0/2334486417 wait complete. 2023-11-08T19:27:54.682 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get test_data_pool pg_num 2023-11-08T19:27:54.807 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.804+0000 7f9d3c43f700 1 Processor -- start 2023-11-08T19:27:54.808 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.804+0000 7f9d3c43f700 1 -- start start 2023-11-08T19:27:54.808 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.805+0000 7f9d3c43f700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9d3415dba0 0x7f9d3406c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:54.808 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.805+0000 7f9d3c43f700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f9d340b22e0 0x7f9d340b2770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:54.808 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.805+0000 7f9d3c43f700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d34161890 con 0x7f9d3406b240 2023-11-08T19:27:54.809 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.805+0000 7f9d3c43f700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d340b2cb0 con 0x7f9d340b22e0 2023-11-08T19:27:54.809 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.805+0000 7f9d3c43f700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d3406b740 con 0x7f9d3415dba0 2023-11-08T19:27:54.809 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.805+0000 7f9d3a9dc700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f9d340b22e0 0x7f9d340b2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:54.809 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.805+0000 7f9d3a9dc700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f9d340b22e0 0x7f9d340b2770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54284/0 (socket says 172.21.15.143:54284) 2023-11-08T19:27:54.809 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.805+0000 7f9d3a9dc700 1 -- 172.21.15.143:0/4056292204 learned_addr learned my addr 172.21.15.143:0/4056292204 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:54.810 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.806+0000 7f9d3a1db700 1 --2- 172.21.15.143:0/4056292204 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9d3415dba0 0x7f9d3406c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:54.810 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.806+0000 7f9d3a9dc700 1 -- 172.21.15.143:0/4056292204 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9d3415dba0 msgr2=0x7f9d3406c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:54.810 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.806+0000 7f9d3a9dc700 1 --2- 172.21.15.143:0/4056292204 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9d3415dba0 0x7f9d3406c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:54.810 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.806+0000 7f9d3a9dc700 1 -- 172.21.15.143:0/4056292204 >> v1:172.21.15.174:6789/0 conn(0x7f9d3406b240 legacy=0x7f9d340caff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:54.810 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.806+0000 7f9d3a9dc700 1 -- 172.21.15.143:0/4056292204 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d3406ba20 con 0x7f9d340b22e0 2023-11-08T19:27:54.811 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.806+0000 7f9d3a9dc700 1 --2- 172.21.15.143:0/4056292204 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9d340b22e0 0x7f9d340b2770 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c009e50 tx=0x7f9d2c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b79a690a9665860e server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:54.811 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.806+0000 7f9d391d9700 1 -- 172.21.15.143:0/4056292204 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d2c00e070 con 0x7f9d340b22e0 2023-11-08T19:27:54.811 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.806+0000 7f9d391d9700 1 -- 172.21.15.143:0/4056292204 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d2c003a20 con 0x7f9d340b22e0 2023-11-08T19:27:54.811 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.806+0000 7f9d391d9700 1 -- 172.21.15.143:0/4056292204 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d2c003dd0 con 0x7f9d340b22e0 2023-11-08T19:27:54.814 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.807+0000 7f9d3c43f700 1 -- 172.21.15.143:0/4056292204 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9d340b22e0 msgr2=0x7f9d340b2770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:54.815 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.807+0000 7f9d3c43f700 1 --2- 172.21.15.143:0/4056292204 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9d340b22e0 0x7f9d340b2770 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c009e50 tx=0x7f9d2c00b330 comp rx=0 tx=0).stop 2023-11-08T19:27:54.815 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.807+0000 7f9d3c43f700 1 -- 172.21.15.143:0/4056292204 shutdown_connections 2023-11-08T19:27:54.815 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.807+0000 7f9d3c43f700 1 --2- 172.21.15.143:0/4056292204 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9d340b22e0 0x7f9d340b2770 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:54.816 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.807+0000 7f9d3c43f700 1 --2- 172.21.15.143:0/4056292204 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9d3415dba0 0x7f9d3406c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:54.816 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.807+0000 7f9d3c43f700 1 -- 172.21.15.143:0/4056292204 >> 172.21.15.143:0/4056292204 conn(0x7f9d340cd2e0 msgr2=0x7f9d340cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:54.816 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.807+0000 7f9d3c43f700 1 -- 172.21.15.143:0/4056292204 shutdown_connections 2023-11-08T19:27:54.816 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.807+0000 7f9d3c43f700 1 -- 172.21.15.143:0/4056292204 wait complete. 2023-11-08T19:27:54.816 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.808+0000 7f9d3c43f700 1 Processor -- start 2023-11-08T19:27:54.817 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.808+0000 7f9d3c43f700 1 -- start start 2023-11-08T19:27:54.817 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.809+0000 7f9d3c43f700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9d3406b240 0x7f9d340b4b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:54.817 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.809+0000 7f9d3c43f700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9d340b22e0 0x7f9d3414cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:54.817 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.809+0000 7f9d3c43f700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9d3415dba0 0x7f9d3414d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:54.818 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.809+0000 7f9d3c43f700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d341505a0 con 0x7f9d3415dba0 2023-11-08T19:27:54.818 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.809+0000 7f9d3c43f700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d340b5090 con 0x7f9d3406b240 2023-11-08T19:27:54.818 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.809+0000 7f9d3c43f700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d340c5440 con 0x7f9d340b22e0 2023-11-08T19:27:54.818 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.809+0000 7f9d3a9dc700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9d3415dba0 0x7f9d3414d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:54.819 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.809+0000 7f9d3a9dc700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9d3415dba0 0x7f9d3414d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54290/0 (socket says 172.21.15.143:54290) 2023-11-08T19:27:54.819 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.809+0000 7f9d3a9dc700 1 -- 172.21.15.143:0/3122023505 learned_addr learned my addr 172.21.15.143:0/3122023505 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:54.819 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.809+0000 7f9d3a1db700 1 --2- 172.21.15.143:0/3122023505 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9d3406b240 0x7f9d340b4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:54.819 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.809+0000 7f9d399da700 1 --2- 172.21.15.143:0/3122023505 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9d340b22e0 0x7f9d3414cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:54.819 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.809+0000 7f9d3a9dc700 1 -- 172.21.15.143:0/3122023505 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9d340b22e0 msgr2=0x7f9d3414cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:54.820 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.809+0000 7f9d3a9dc700 1 --2- 172.21.15.143:0/3122023505 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9d340b22e0 0x7f9d3414cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:54.820 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.809+0000 7f9d3a9dc700 1 -- 172.21.15.143:0/3122023505 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9d3406b240 msgr2=0x7f9d340b4b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:54.820 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.809+0000 7f9d3a9dc700 1 --2- 172.21.15.143:0/3122023505 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9d3406b240 0x7f9d340b4b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:54.820 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.809+0000 7f9d3a9dc700 1 -- 172.21.15.143:0/3122023505 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d2c009b30 con 0x7f9d3415dba0 2023-11-08T19:27:54.820 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.809+0000 7f9d3a1db700 1 --2- 172.21.15.143:0/3122023505 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9d3406b240 0x7f9d340b4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:27:54.821 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.809+0000 7f9d3a9dc700 1 --2- 172.21.15.143:0/3122023505 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9d3415dba0 0x7f9d3414d710 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f9d34067530 tx=0x7f9d2c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:54.821 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.809+0000 7f9d2affd700 1 -- 172.21.15.143:0/3122023505 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d2c004440 con 0x7f9d3415dba0 2023-11-08T19:27:54.821 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.809+0000 7f9d2affd700 1 -- 172.21.15.143:0/3122023505 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d2c0045a0 con 0x7f9d3415dba0 2023-11-08T19:27:54.821 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.809+0000 7f9d3c43f700 1 -- 172.21.15.143:0/3122023505 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d340c5720 con 0x7f9d3415dba0 2023-11-08T19:27:54.821 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.809+0000 7f9d2affd700 1 -- 172.21.15.143:0/3122023505 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d2c004950 con 0x7f9d3415dba0 2023-11-08T19:27:54.821 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.810+0000 7f9d3c43f700 1 -- 172.21.15.143:0/3122023505 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d340b5580 con 0x7f9d3415dba0 2023-11-08T19:27:54.822 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.810+0000 7f9d3c43f700 1 -- 172.21.15.143:0/3122023505 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d3404eee0 con 0x7f9d3415dba0 2023-11-08T19:27:54.822 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.811+0000 7f9d2affd700 1 -- 172.21.15.143:0/3122023505 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9d2c012430 con 0x7f9d3415dba0 2023-11-08T19:27:54.822 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.811+0000 7f9d2affd700 1 --2- 172.21.15.143:0/3122023505 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f9d1806af00 0x7f9d1806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:54.822 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.811+0000 7f9d2affd700 1 -- 172.21.15.143:0/3122023505 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6034+0+0 (secure 0 0 0) 0x7f9d2c08abf0 con 0x7f9d3415dba0 2023-11-08T19:27:54.822 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.811+0000 7f9d3a1db700 1 --2- 172.21.15.143:0/3122023505 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f9d1806af00 0x7f9d1806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:54.823 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.812+0000 7f9d3a1db700 1 --2- 172.21.15.143:0/3122023505 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f9d1806af00 0x7f9d1806d3c0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f9d24009820 tx=0x7f9d24008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:54.823 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:54.815+0000 7f9d2affd700 1 -- 172.21.15.143:0/3122023505 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9d2c055b50 con 0x7f9d3415dba0 2023-11-08T19:27:55.006 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.004+0000 7f9d3c43f700 1 -- 172.21.15.143:0/3122023505 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "test_data_pool", "var": "pg_num"} v 0) v1 -- 0x7f9d340b67e0 con 0x7f9d3415dba0 2023-11-08T19:27:55.499 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.004+0000 7f9d2affd700 1 -- 172.21.15.143:0/3122023505 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "test_data_pool", "var": "pg_num"}]=0 v84) v1 ==== 103+0+11 (secure 0 0 0) 0x7f9d2c0170a0 con 0x7f9d3415dba0 2023-11-08T19:27:55.500 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2023-11-08T19:27:55.503 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.006+0000 7f9d3c43f700 1 -- 172.21.15.143:0/3122023505 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f9d1806af00 msgr2=0x7f9d1806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:55.503 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.006+0000 7f9d3c43f700 1 --2- 172.21.15.143:0/3122023505 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f9d1806af00 0x7f9d1806d3c0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f9d24009820 tx=0x7f9d24008040 comp rx=0 tx=0).stop 2023-11-08T19:27:55.503 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.006+0000 7f9d3c43f700 1 -- 172.21.15.143:0/3122023505 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9d3415dba0 msgr2=0x7f9d3414d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:55.504 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.006+0000 7f9d3c43f700 1 --2- 172.21.15.143:0/3122023505 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9d3415dba0 0x7f9d3414d710 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f9d34067530 tx=0x7f9d2c004ad0 comp rx=0 tx=0).stop 2023-11-08T19:27:55.504 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.006+0000 7f9d3c43f700 1 -- 172.21.15.143:0/3122023505 shutdown_connections 2023-11-08T19:27:55.504 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.006+0000 7f9d3c43f700 1 --2- 172.21.15.143:0/3122023505 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f9d1806af00 0x7f9d1806d3c0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:55.505 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.006+0000 7f9d3c43f700 1 --2- 172.21.15.143:0/3122023505 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9d3415dba0 0x7f9d3414d710 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:55.505 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.006+0000 7f9d3c43f700 1 --2- 172.21.15.143:0/3122023505 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9d3406b240 0x7f9d340b4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:55.505 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.006+0000 7f9d3c43f700 1 --2- 172.21.15.143:0/3122023505 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9d340b22e0 0x7f9d3414cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:55.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.006+0000 7f9d3c43f700 1 -- 172.21.15.143:0/3122023505 >> 172.21.15.143:0/3122023505 conn(0x7f9d340cd2e0 msgr2=0x7f9d340b0660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:55.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.007+0000 7f9d3c43f700 1 -- 172.21.15.143:0/3122023505 shutdown_connections 2023-11-08T19:27:55.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.007+0000 7f9d3c43f700 1 -- 172.21.15.143:0/3122023505 wait complete. 2023-11-08T19:27:55.507 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get test_metadata_pool pg_num 2023-11-08T19:27:55.638 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.637+0000 7f5d12299700 1 Processor -- start 2023-11-08T19:27:55.639 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.637+0000 7f5d12299700 1 -- start start 2023-11-08T19:27:55.640 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.638+0000 7f5d12299700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f5d0c06b240 0x7f5d0c0caff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:55.641 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.638+0000 7f5d12299700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5d0c0cb530 0x7f5d0c0b22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:55.641 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.638+0000 7f5d12299700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d0c161890 con 0x7f5d0c15dba0 2023-11-08T19:27:55.641 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.638+0000 7f5d12299700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d0c06cd20 con 0x7f5d0c06b240 2023-11-08T19:27:55.641 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.638+0000 7f5d12299700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d0c06b740 con 0x7f5d0c0cb530 2023-11-08T19:27:55.641 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.638+0000 7f5d0b7fe700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f5d0c06b240 0x7f5d0c0caff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:55.642 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.638+0000 7f5d10836700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5d0c0cb530 0x7f5d0c0b22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:55.642 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.638+0000 7f5d0b7fe700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f5d0c06b240 0x7f5d0c0caff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.143:37794/0 (socket says 172.21.15.143:37794) 2023-11-08T19:27:55.642 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.638+0000 7f5d0b7fe700 1 -- 172.21.15.143:0/1375956196 learned_addr learned my addr 172.21.15.143:0/1375956196 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:55.642 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.639+0000 7f5d10836700 1 -- 172.21.15.143:0/1375956196 >> v2:172.21.15.174:3300/0 conn(0x7f5d0c06b240 msgr2=0x7f5d0c0caff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:55.642 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.639+0000 7f5d10836700 1 --2- 172.21.15.143:0/1375956196 >> v2:172.21.15.174:3300/0 conn(0x7f5d0c06b240 0x7f5d0c0caff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:55.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.639+0000 7f5d10836700 1 -- 172.21.15.143:0/1375956196 >> v1:172.21.15.174:6789/0 conn(0x7f5d0c15dba0 legacy=0x7f5d0c06c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:55.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.639+0000 7f5d10836700 1 -- 172.21.15.143:0/1375956196 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d0c06ba20 con 0x7f5d0c0cb530 2023-11-08T19:27:55.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.639+0000 7f5d0b7fe700 1 --2- 172.21.15.143:0/1375956196 >> v2:172.21.15.174:3300/0 conn(0x7f5d0c06b240 0x7f5d0c0caff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T19:27:55.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.639+0000 7f5d10836700 1 --2- 172.21.15.143:0/1375956196 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5d0c0cb530 0x7f5d0c0b22e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5d04009d90 tx=0x7f5d0400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba76cc82b100f353 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:55.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.639+0000 7f5d0affd700 1 -- 172.21.15.143:0/1375956196 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d0400e070 con 0x7f5d0c0cb530 2023-11-08T19:27:55.644 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.639+0000 7f5d0affd700 1 -- 172.21.15.143:0/1375956196 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d0400bb20 con 0x7f5d0c0cb530 2023-11-08T19:27:55.644 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.639+0000 7f5d0affd700 1 -- 172.21.15.143:0/1375956196 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d04012430 con 0x7f5d0c0cb530 2023-11-08T19:27:55.647 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.640+0000 7f5d12299700 1 -- 172.21.15.143:0/1375956196 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5d0c0cb530 msgr2=0x7f5d0c0b22e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:55.647 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.640+0000 7f5d12299700 1 --2- 172.21.15.143:0/1375956196 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5d0c0cb530 0x7f5d0c0b22e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5d04009d90 tx=0x7f5d0400b330 comp rx=0 tx=0).stop 2023-11-08T19:27:55.647 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.640+0000 7f5d12299700 1 -- 172.21.15.143:0/1375956196 shutdown_connections 2023-11-08T19:27:55.647 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.640+0000 7f5d12299700 1 --2- 172.21.15.143:0/1375956196 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5d0c0cb530 0x7f5d0c0b22e0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:55.648 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.640+0000 7f5d12299700 1 --2- 172.21.15.143:0/1375956196 >> v2:172.21.15.174:3300/0 conn(0x7f5d0c06b240 0x7f5d0c0caff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:55.648 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.640+0000 7f5d12299700 1 -- 172.21.15.143:0/1375956196 >> 172.21.15.143:0/1375956196 conn(0x7f5d0c0cd2e0 msgr2=0x7f5d0c0cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:55.648 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.640+0000 7f5d12299700 1 -- 172.21.15.143:0/1375956196 shutdown_connections 2023-11-08T19:27:55.648 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.640+0000 7f5d12299700 1 -- 172.21.15.143:0/1375956196 wait complete. 2023-11-08T19:27:55.648 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.641+0000 7f5d12299700 1 Processor -- start 2023-11-08T19:27:55.649 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.641+0000 7f5d12299700 1 -- start start 2023-11-08T19:27:55.649 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.642+0000 7f5d12299700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5d0c06b240 0x7f5d0c15e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:55.649 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.642+0000 7f5d12299700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5d0c0cb530 0x7f5d0c15b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:55.649 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.642+0000 7f5d12299700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5d0c15dba0 0x7f5d0c0c8e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:55.649 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.642+0000 7f5d12299700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d0c0c9cd0 con 0x7f5d0c06b240 2023-11-08T19:27:55.649 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.642+0000 7f5d12299700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d0c0c87b0 con 0x7f5d0c0cb530 2023-11-08T19:27:55.650 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.642+0000 7f5d12299700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d0c0c99f0 con 0x7f5d0c15dba0 2023-11-08T19:27:55.650 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.642+0000 7f5d0bfff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5d0c06b240 0x7f5d0c15e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:55.650 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.642+0000 7f5d0bfff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5d0c06b240 0x7f5d0c15e2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54300/0 (socket says 172.21.15.143:54300) 2023-11-08T19:27:55.650 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.642+0000 7f5d10836700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5d0c15dba0 0x7f5d0c0c8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:55.650 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.642+0000 7f5d0bfff700 1 -- 172.21.15.143:0/1317924861 learned_addr learned my addr 172.21.15.143:0/1317924861 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:55.651 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.642+0000 7f5d0b7fe700 1 --2- 172.21.15.143:0/1317924861 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5d0c0cb530 0x7f5d0c15b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:55.651 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.642+0000 7f5d0bfff700 1 -- 172.21.15.143:0/1317924861 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5d0c15dba0 msgr2=0x7f5d0c0c8e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:55.651 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.642+0000 7f5d0bfff700 1 --2- 172.21.15.143:0/1317924861 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5d0c15dba0 0x7f5d0c0c8e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:55.651 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.642+0000 7f5d0bfff700 1 -- 172.21.15.143:0/1317924861 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5d0c0cb530 msgr2=0x7f5d0c15b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:55.651 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.642+0000 7f5d0bfff700 1 --2- 172.21.15.143:0/1317924861 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5d0c0cb530 0x7f5d0c15b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:55.652 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.642+0000 7f5d0bfff700 1 -- 172.21.15.143:0/1317924861 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cfc009550 con 0x7f5d0c06b240 2023-11-08T19:27:55.652 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.642+0000 7f5d0bfff700 1 --2- 172.21.15.143:0/1317924861 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5d0c06b240 0x7f5d0c15e2e0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f5d0000c450 tx=0x7f5d0000c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:55.652 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.643+0000 7f5d08ff9700 1 -- 172.21.15.143:0/1317924861 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d000052d0 con 0x7f5d0c06b240 2023-11-08T19:27:55.652 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.643+0000 7f5d08ff9700 1 -- 172.21.15.143:0/1317924861 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d00005430 con 0x7f5d0c06b240 2023-11-08T19:27:55.652 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.643+0000 7f5d08ff9700 1 -- 172.21.15.143:0/1317924861 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d000038c0 con 0x7f5d0c06b240 2023-11-08T19:27:55.652 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.643+0000 7f5d12299700 1 -- 172.21.15.143:0/1317924861 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d04009a70 con 0x7f5d0c06b240 2023-11-08T19:27:55.653 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.643+0000 7f5d12299700 1 -- 172.21.15.143:0/1317924861 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d0c1e6e20 con 0x7f5d0c06b240 2023-11-08T19:27:55.653 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.643+0000 7f5d12299700 1 -- 172.21.15.143:0/1317924861 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d0c04eee0 con 0x7f5d0c06b240 2023-11-08T19:27:55.653 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.644+0000 7f5d08ff9700 1 -- 172.21.15.143:0/1317924861 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5d00003a20 con 0x7f5d0c06b240 2023-11-08T19:27:55.653 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.644+0000 7f5d08ff9700 1 --2- 172.21.15.143:0/1317924861 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f5cf806af00 0x7f5cf806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:55.654 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.644+0000 7f5d08ff9700 1 -- 172.21.15.143:0/1317924861 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6034+0+0 (secure 0 0 0) 0x7f5d00088a50 con 0x7f5d0c06b240 2023-11-08T19:27:55.654 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.645+0000 7f5d0b7fe700 1 --2- 172.21.15.143:0/1317924861 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f5cf806af00 0x7f5cf806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:55.654 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.645+0000 7f5d0b7fe700 1 --2- 172.21.15.143:0/1317924861 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f5cf806af00 0x7f5cf806d3c0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f5d0c0b53f0 tx=0x7f5cfc0092c0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:55.655 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.649+0000 7f5d08ff9700 1 -- 172.21.15.143:0/1317924861 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5d000539a0 con 0x7f5d0c06b240 2023-11-08T19:27:55.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.838+0000 7f5d12299700 1 -- 172.21.15.143:0/1317924861 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "test_metadata_pool", "var": "pg_num"} v 0) v1 -- 0x7f5d0c0b67e0 con 0x7f5d0c06b240 2023-11-08T19:27:55.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.839+0000 7f5d08ff9700 1 -- 172.21.15.143:0/1317924861 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "test_metadata_pool", "var": "pg_num"}]=0 v84) v1 ==== 107+0+11 (secure 0 0 0) 0x7f5d000170b0 con 0x7f5d0c06b240 2023-11-08T19:27:55.841 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2023-11-08T19:27:55.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.841+0000 7f5d12299700 1 -- 172.21.15.143:0/1317924861 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f5cf806af00 msgr2=0x7f5cf806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:55.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.841+0000 7f5d12299700 1 --2- 172.21.15.143:0/1317924861 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f5cf806af00 0x7f5cf806d3c0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f5d0c0b53f0 tx=0x7f5cfc0092c0 comp rx=0 tx=0).stop 2023-11-08T19:27:55.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.841+0000 7f5d12299700 1 -- 172.21.15.143:0/1317924861 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5d0c06b240 msgr2=0x7f5d0c15e2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:55.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.841+0000 7f5d12299700 1 --2- 172.21.15.143:0/1317924861 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5d0c06b240 0x7f5d0c15e2e0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f5d0000c450 tx=0x7f5d0000c760 comp rx=0 tx=0).stop 2023-11-08T19:27:55.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.841+0000 7f5d12299700 1 -- 172.21.15.143:0/1317924861 shutdown_connections 2023-11-08T19:27:55.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.841+0000 7f5d12299700 1 --2- 172.21.15.143:0/1317924861 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f5cf806af00 0x7f5cf806d3c0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:55.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.841+0000 7f5d12299700 1 --2- 172.21.15.143:0/1317924861 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5d0c15dba0 0x7f5d0c0c8e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:55.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.841+0000 7f5d12299700 1 --2- 172.21.15.143:0/1317924861 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5d0c06b240 0x7f5d0c15e2e0 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:55.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.841+0000 7f5d12299700 1 --2- 172.21.15.143:0/1317924861 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5d0c0cb530 0x7f5d0c15b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:55.845 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.841+0000 7f5d12299700 1 -- 172.21.15.143:0/1317924861 >> 172.21.15.143:0/1317924861 conn(0x7f5d0c0cd2e0 msgr2=0x7f5d0c15cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:55.845 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.841+0000 7f5d12299700 1 -- 172.21.15.143:0/1317924861 shutdown_connections 2023-11-08T19:27:55.845 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.841+0000 7f5d12299700 1 -- 172.21.15.143:0/1317924861 wait complete. 2023-11-08T19:27:55.854 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snapshots.TestSnapshots.test_kill_mdstable' 2023-11-08T19:27:55.975 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.974+0000 7fc44d2db700 1 Processor -- start 2023-11-08T19:27:55.975 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.974+0000 7fc44d2db700 1 -- start start 2023-11-08T19:27:55.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.974+0000 7fc44d2db700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc44815ba40 0x7fc44815f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:55.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.974+0000 7fc44d2db700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc44806e470 0x7fc4480b24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:55.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.974+0000 7fc44d2db700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc44815be20 con 0x7fc4480b1100 2023-11-08T19:27:55.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.974+0000 7fc44d2db700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc448070f00 con 0x7fc44815ba40 2023-11-08T19:27:55.979 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.974+0000 7fc44d2db700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc44815fb60 con 0x7fc44806e470 2023-11-08T19:27:55.979 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.975+0000 7fc446ffd700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc44815ba40 0x7fc44815f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:55.979 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.975+0000 7fc4477fe700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fc4480b1100 0x7fc4480713e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:50464/0 (socket says 172.21.15.143:50464) 2023-11-08T19:27:55.979 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.975+0000 7fc4477fe700 1 -- 172.21.15.143:0/3926392792 learned_addr learned my addr 172.21.15.143:0/3926392792 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:27:55.979 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.975+0000 7fc4467fc700 1 --2- 172.21.15.143:0/3926392792 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc44806e470 0x7fc4480b24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:55.980 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.975+0000 7fc446ffd700 1 -- 172.21.15.143:0/3926392792 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc44806e470 msgr2=0x7fc4480b24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:55.980 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.975+0000 7fc446ffd700 1 --2- 172.21.15.143:0/3926392792 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc44806e470 0x7fc4480b24d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:55.980 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.975+0000 7fc446ffd700 1 -- 172.21.15.143:0/3926392792 >> v1:172.21.15.143:6789/0 conn(0x7fc4480b1100 legacy=0x7fc4480713e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:55.980 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.975+0000 7fc446ffd700 1 -- 172.21.15.143:0/3926392792 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc44815fe40 con 0x7fc44815ba40 2023-11-08T19:27:55.980 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.975+0000 7fc445ffb700 1 -- 172.21.15.143:0/3926392792 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4253252538 0 0) 0x7fc44815be20 con 0x7fc4480b1100 2023-11-08T19:27:55.980 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.975+0000 7fc446ffd700 1 --2- 172.21.15.143:0/3926392792 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc44815ba40 0x7fc44815f510 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fc438009840 tx=0x7fc438009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7491f71463ca272 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:55.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.975+0000 7fc445ffb700 1 -- 172.21.15.143:0/3926392792 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc438011830 con 0x7fc44815ba40 2023-11-08T19:27:55.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.975+0000 7fc445ffb700 1 -- 172.21.15.143:0/3926392792 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc438011990 con 0x7fc44815ba40 2023-11-08T19:27:55.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.975+0000 7fc445ffb700 1 -- 172.21.15.143:0/3926392792 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc438011d20 con 0x7fc44815ba40 2023-11-08T19:27:55.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.976+0000 7fc44d2db700 1 -- 172.21.15.143:0/3926392792 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc44815ba40 msgr2=0x7fc44815f510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:55.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.976+0000 7fc44d2db700 1 --2- 172.21.15.143:0/3926392792 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc44815ba40 0x7fc44815f510 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fc438009840 tx=0x7fc438009b50 comp rx=0 tx=0).stop 2023-11-08T19:27:55.982 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.976+0000 7fc44d2db700 1 -- 172.21.15.143:0/3926392792 shutdown_connections 2023-11-08T19:27:55.982 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.976+0000 7fc44d2db700 1 --2- 172.21.15.143:0/3926392792 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc44815ba40 0x7fc44815f510 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:55.982 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.976+0000 7fc44d2db700 1 --2- 172.21.15.143:0/3926392792 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc44806e470 0x7fc4480b24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:55.982 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.976+0000 7fc44d2db700 1 -- 172.21.15.143:0/3926392792 >> 172.21.15.143:0/3926392792 conn(0x7fc44806b4c0 msgr2=0x7fc4480cf680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:55.982 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.976+0000 7fc44d2db700 1 -- 172.21.15.143:0/3926392792 shutdown_connections 2023-11-08T19:27:55.983 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.976+0000 7fc44d2db700 1 -- 172.21.15.143:0/3926392792 wait complete. 2023-11-08T19:27:55.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.977+0000 7fc44d2db700 1 Processor -- start 2023-11-08T19:27:55.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.977+0000 7fc44d2db700 1 -- start start 2023-11-08T19:27:55.986 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.978+0000 7fc44d2db700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc44806e470 0x7fc44814c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:55.986 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.978+0000 7fc44d2db700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc4480b1100 0x7fc4480b0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:55.986 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.978+0000 7fc44d2db700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc44815ba40 0x7fc4480b0860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:55.986 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.978+0000 7fc44d2db700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4480c5390 con 0x7fc44806e470 2023-11-08T19:27:55.987 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.978+0000 7fc44d2db700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4480c5670 con 0x7fc44815ba40 2023-11-08T19:27:55.987 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.978+0000 7fc44d2db700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4480c5950 con 0x7fc4480b1100 2023-11-08T19:27:55.987 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.978+0000 7fc446ffd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc44806e470 0x7fc44814c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:55.987 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.978+0000 7fc446ffd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc44806e470 0x7fc44814c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54312/0 (socket says 172.21.15.143:54312) 2023-11-08T19:27:55.987 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.978+0000 7fc446ffd700 1 -- 172.21.15.143:0/2951026563 learned_addr learned my addr 172.21.15.143:0/2951026563 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:55.988 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.978+0000 7fc4467fc700 1 --2- 172.21.15.143:0/2951026563 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc4480b1100 0x7fc4480b0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:55.988 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.978+0000 7fc4477fe700 1 --2- 172.21.15.143:0/2951026563 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc44815ba40 0x7fc4480b0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:55.988 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.978+0000 7fc446ffd700 1 -- 172.21.15.143:0/2951026563 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc4480b1100 msgr2=0x7fc4480b0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:55.988 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.978+0000 7fc446ffd700 1 --2- 172.21.15.143:0/2951026563 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc4480b1100 0x7fc4480b0320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:55.988 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.978+0000 7fc446ffd700 1 -- 172.21.15.143:0/2951026563 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc44815ba40 msgr2=0x7fc4480b0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:55.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.978+0000 7fc446ffd700 1 --2- 172.21.15.143:0/2951026563 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc44815ba40 0x7fc4480b0860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:55.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.978+0000 7fc446ffd700 1 -- 172.21.15.143:0/2951026563 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4380094a0 con 0x7fc44806e470 2023-11-08T19:27:55.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.978+0000 7fc446ffd700 1 --2- 172.21.15.143:0/2951026563 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc44806e470 0x7fc44814c9c0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fc438009b90 tx=0x7fc438004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:55.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.978+0000 7fc42f7fe700 1 -- 172.21.15.143:0/2951026563 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4380045d0 con 0x7fc44806e470 2023-11-08T19:27:55.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.979+0000 7fc42f7fe700 1 -- 172.21.15.143:0/2951026563 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc43801f070 con 0x7fc44806e470 2023-11-08T19:27:55.990 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.979+0000 7fc44d2db700 1 -- 172.21.15.143:0/2951026563 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4481f26d0 con 0x7fc44806e470 2023-11-08T19:27:55.990 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.979+0000 7fc42f7fe700 1 -- 172.21.15.143:0/2951026563 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4380038c0 con 0x7fc44806e470 2023-11-08T19:27:55.990 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.979+0000 7fc44d2db700 1 -- 172.21.15.143:0/2951026563 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4480c45d0 con 0x7fc44806e470 2023-11-08T19:27:55.990 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.979+0000 7fc44d2db700 1 -- 172.21.15.143:0/2951026563 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4480c5b70 con 0x7fc44806e470 2023-11-08T19:27:55.990 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.980+0000 7fc42f7fe700 1 -- 172.21.15.143:0/2951026563 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc438003a20 con 0x7fc44806e470 2023-11-08T19:27:55.991 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.980+0000 7fc42f7fe700 1 --2- 172.21.15.143:0/2951026563 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc43406af00 0x7fc43406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:55.991 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.980+0000 7fc42f7fe700 1 -- 172.21.15.143:0/2951026563 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6034+0+0 (secure 0 0 0) 0x7fc438021070 con 0x7fc44806e470 2023-11-08T19:27:55.991 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.981+0000 7fc4467fc700 1 --2- 172.21.15.143:0/2951026563 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc43406af00 0x7fc43406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:55.991 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.981+0000 7fc4467fc700 1 --2- 172.21.15.143:0/2951026563 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc43406af00 0x7fc43406d3c0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fc4480b0ee0 tx=0x7fc43c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:55.992 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:55.985+0000 7fc42f7fe700 1 -- 172.21.15.143:0/2951026563 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc438056030 con 0x7fc44806e470 2023-11-08T19:27:56.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:56.259+0000 7fc44d2db700 1 -- 172.21.15.143:0/2951026563 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snapshots.TestSnapshots.test_kill_mdstable"]} v 0) v1 -- 0x7fc44804eee0 con 0x7fc44806e470 2023-11-08T19:27:56.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:56.665+0000 7fc42f7fe700 1 -- 172.21.15.143:0/2951026563 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snapshots.TestSnapshots.test_kill_mdstable"]}]=0 v197) v1 ==== 142+0+0 (secure 0 0 0) 0x7fc4380150d0 con 0x7fc44806e470 2023-11-08T19:27:56.669 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:56.668+0000 7fc44d2db700 1 -- 172.21.15.143:0/2951026563 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc43406af00 msgr2=0x7fc43406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:56.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:56.668+0000 7fc44d2db700 1 --2- 172.21.15.143:0/2951026563 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc43406af00 0x7fc43406d3c0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fc4480b0ee0 tx=0x7fc43c008040 comp rx=0 tx=0).stop 2023-11-08T19:27:56.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:56.668+0000 7fc44d2db700 1 -- 172.21.15.143:0/2951026563 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc44806e470 msgr2=0x7fc44814c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:56.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:56.668+0000 7fc44d2db700 1 --2- 172.21.15.143:0/2951026563 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc44806e470 0x7fc44814c9c0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fc438009b90 tx=0x7fc438004750 comp rx=0 tx=0).stop 2023-11-08T19:27:56.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:56.668+0000 7fc44d2db700 1 -- 172.21.15.143:0/2951026563 shutdown_connections 2023-11-08T19:27:56.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:56.668+0000 7fc44d2db700 1 --2- 172.21.15.143:0/2951026563 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc43406af00 0x7fc43406d3c0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:56.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:56.668+0000 7fc44d2db700 1 --2- 172.21.15.143:0/2951026563 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc44815ba40 0x7fc4480b0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:56.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:56.668+0000 7fc44d2db700 1 --2- 172.21.15.143:0/2951026563 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc44806e470 0x7fc44814c9c0 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:56.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:56.668+0000 7fc44d2db700 1 --2- 172.21.15.143:0/2951026563 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc4480b1100 0x7fc4480b0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:56.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:56.668+0000 7fc44d2db700 1 -- 172.21.15.143:0/2951026563 >> 172.21.15.143:0/2951026563 conn(0x7fc44806b4c0 msgr2=0x7fc44815cb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:56.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:56.668+0000 7fc44d2db700 1 -- 172.21.15.143:0/2951026563 shutdown_connections 2023-11-08T19:27:56.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:56.668+0000 7fc44d2db700 1 -- 172.21.15.143:0/2951026563 wait complete. 2023-11-08T19:27:56.682 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-08T19:27:56.807 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:56.805+0000 7f8d5a7d2700 1 Processor -- start 2023-11-08T19:27:56.807 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:56.805+0000 7f8d5a7d2700 1 -- start start 2023-11-08T19:27:56.809 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:56.806+0000 7f8d5a7d2700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f8d54143a30 0x7f8d5406bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:56.810 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:56.806+0000 7f8d5a7d2700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f8d540cc8c0 0x7f8d540ccd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:56.810 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:56.806+0000 7f8d5a7d2700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d5406c510 con 0x7f8d540c5880 2023-11-08T19:27:56.810 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:56.806+0000 7f8d5a7d2700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d540cd2b0 con 0x7f8d540cc8c0 2023-11-08T19:27:56.810 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:56.806+0000 7f8d5a7d2700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d5415e110 con 0x7f8d54143a30 2023-11-08T19:27:56.810 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:56.806+0000 7f8d537fe700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f8d540cc8c0 0x7f8d540ccd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:56.811 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:56.806+0000 7f8d537fe700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f8d540cc8c0 0x7f8d540ccd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54320/0 (socket says 172.21.15.143:54320) 2023-11-08T19:27:56.811 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:56.806+0000 7f8d537fe700 1 -- 172.21.15.143:0/1381039171 learned_addr learned my addr 172.21.15.143:0/1381039171 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:56.811 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:56.807+0000 7f8d53fff700 1 --2- 172.21.15.143:0/1381039171 >> v2:172.21.15.174:3300/0 conn(0x7f8d54143a30 0x7f8d5406bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:56.811 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:56.807+0000 7f8d537fe700 1 -- 172.21.15.143:0/1381039171 >> v2:172.21.15.174:3300/0 conn(0x7f8d54143a30 msgr2=0x7f8d5406bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:56.811 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:56.807+0000 7f8d537fe700 1 --2- 172.21.15.143:0/1381039171 >> v2:172.21.15.174:3300/0 conn(0x7f8d54143a30 0x7f8d5406bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:56.812 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:56.807+0000 7f8d537fe700 1 -- 172.21.15.143:0/1381039171 >> v1:172.21.15.174:6789/0 conn(0x7f8d540c5880 legacy=0x7f8d540c5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:56.812 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:56.807+0000 7f8d537fe700 1 -- 172.21.15.143:0/1381039171 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d54143e10 con 0x7f8d540cc8c0 2023-11-08T19:27:56.812 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:56.807+0000 7f8d537fe700 1 --2- 172.21.15.143:0/1381039171 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8d540cc8c0 0x7f8d540ccd70 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f8d54072980 tx=0x7f8d48009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b770b493b2feb83 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:56.812 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:56.807+0000 7f8d52ffd700 1 -- 172.21.15.143:0/1381039171 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d4800b870 con 0x7f8d540cc8c0 2023-11-08T19:27:56.812 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:56.807+0000 7f8d52ffd700 1 -- 172.21.15.143:0/1381039171 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d4800b9d0 con 0x7f8d540cc8c0 2023-11-08T19:27:56.812 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:56.807+0000 7f8d52ffd700 1 -- 172.21.15.143:0/1381039171 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d4800bd60 con 0x7f8d540cc8c0 2023-11-08T19:27:56.813 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:56.808+0000 7f8d5a7d2700 1 -- 172.21.15.143:0/1381039171 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8d540cc8c0 msgr2=0x7f8d540ccd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:56.813 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:56.808+0000 7f8d5a7d2700 1 --2- 172.21.15.143:0/1381039171 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8d540cc8c0 0x7f8d540ccd70 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f8d54072980 tx=0x7f8d48009bd0 comp rx=0 tx=0).stop 2023-11-08T19:27:56.813 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:56.808+0000 7f8d5a7d2700 1 -- 172.21.15.143:0/1381039171 shutdown_connections 2023-11-08T19:27:56.813 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:56.808+0000 7f8d5a7d2700 1 --2- 172.21.15.143:0/1381039171 >> v2:172.21.15.174:3300/0 conn(0x7f8d54143a30 0x7f8d5406bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:56.813 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:56.808+0000 7f8d5a7d2700 1 --2- 172.21.15.143:0/1381039171 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8d540cc8c0 0x7f8d540ccd70 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:56.814 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:56.808+0000 7f8d5a7d2700 1 -- 172.21.15.143:0/1381039171 >> 172.21.15.143:0/1381039171 conn(0x7f8d540b1c70 msgr2=0x7f8d540b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:56.814 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:56.808+0000 7f8d5a7d2700 1 -- 172.21.15.143:0/1381039171 shutdown_connections 2023-11-08T19:27:56.814 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:56.808+0000 7f8d5a7d2700 1 -- 172.21.15.143:0/1381039171 wait complete. 2023-11-08T19:27:56.817 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:56.809+0000 7f8d5a7d2700 1 Processor -- start 2023-11-08T19:27:56.817 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:56.809+0000 7f8d5a7d2700 1 -- start start 2023-11-08T19:27:56.817 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:56.810+0000 7f8d5a7d2700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8d540c5880 0x7f8d540c8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:56.817 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:56.810+0000 7f8d5a7d2700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8d540cc8c0 0x7f8d540b2d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:56.817 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:56.810+0000 7f8d5a7d2700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8d54143a30 0x7f8d54159640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:56.818 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:56.810+0000 7f8d5a7d2700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d540b32a0 con 0x7f8d540cc8c0 2023-11-08T19:27:56.818 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:56.810+0000 7f8d5a7d2700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d54140a90 con 0x7f8d54143a30 2023-11-08T19:27:56.818 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:56.810+0000 7f8d5a7d2700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d540b3520 con 0x7f8d540c5880 2023-11-08T19:27:56.819 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:56.810+0000 7f8d537fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8d540cc8c0 0x7f8d540b2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:56.819 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:56.810+0000 7f8d537fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8d540cc8c0 0x7f8d540b2d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54328/0 (socket says 172.21.15.143:54328) 2023-11-08T19:27:56.819 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:56.810+0000 7f8d537fe700 1 -- 172.21.15.143:0/4272182447 learned_addr learned my addr 172.21.15.143:0/4272182447 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:56.819 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:56.810+0000 7f8d58d6f700 1 --2- 172.21.15.143:0/4272182447 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8d54143a30 0x7f8d54159640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:56.819 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:56.810+0000 7f8d53fff700 1 --2- 172.21.15.143:0/4272182447 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8d540c5880 0x7f8d540c8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:56.820 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:56.810+0000 7f8d537fe700 1 -- 172.21.15.143:0/4272182447 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8d540c5880 msgr2=0x7f8d540c8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:56.820 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:56.810+0000 7f8d537fe700 1 --2- 172.21.15.143:0/4272182447 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8d540c5880 0x7f8d540c8ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:56.820 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:56.810+0000 7f8d537fe700 1 -- 172.21.15.143:0/4272182447 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8d54143a30 msgr2=0x7f8d54159640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:56.820 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:56.810+0000 7f8d537fe700 1 --2- 172.21.15.143:0/4272182447 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8d54143a30 0x7f8d54159640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:56.820 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:56.810+0000 7f8d537fe700 1 -- 172.21.15.143:0/4272182447 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d48009550 con 0x7f8d540cc8c0 2023-11-08T19:27:56.821 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:56.810+0000 7f8d53fff700 1 --2- 172.21.15.143:0/4272182447 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8d540c5880 0x7f8d540c8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:27:56.821 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:56.810+0000 7f8d537fe700 1 --2- 172.21.15.143:0/4272182447 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8d540cc8c0 0x7f8d540b2d60 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f8d48000c00 tx=0x7f8d48004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:56.821 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:56.810+0000 7f8d50ff9700 1 -- 172.21.15.143:0/4272182447 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d48004540 con 0x7f8d540cc8c0 2023-11-08T19:27:56.821 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:56.810+0000 7f8d50ff9700 1 -- 172.21.15.143:0/4272182447 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d480036a0 con 0x7f8d540cc8c0 2023-11-08T19:27:56.821 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:56.810+0000 7f8d5a7d2700 1 -- 172.21.15.143:0/4272182447 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d540b3800 con 0x7f8d540cc8c0 2023-11-08T19:27:56.821 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:56.810+0000 7f8d50ff9700 1 -- 172.21.15.143:0/4272182447 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d48003a10 con 0x7f8d540cc8c0 2023-11-08T19:27:56.822 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:56.810+0000 7f8d5a7d2700 1 -- 172.21.15.143:0/4272182447 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d54159b80 con 0x7f8d540cc8c0 2023-11-08T19:27:56.822 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:56.811+0000 7f8d5a7d2700 1 -- 172.21.15.143:0/4272182447 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d54140f70 con 0x7f8d540cc8c0 2023-11-08T19:27:56.822 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:56.812+0000 7f8d50ff9700 1 -- 172.21.15.143:0/4272182447 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8d48021070 con 0x7f8d540cc8c0 2023-11-08T19:27:56.822 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:56.812+0000 7f8d50ff9700 1 --2- 172.21.15.143:0/4272182447 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f8d4006af00 0x7f8d4006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:56.823 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:56.813+0000 7f8d50ff9700 1 -- 172.21.15.143:0/4272182447 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6034+0+0 (secure 0 0 0) 0x7f8d4808ad40 con 0x7f8d540cc8c0 2023-11-08T19:27:56.823 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:56.813+0000 7f8d53fff700 1 --2- 172.21.15.143:0/4272182447 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f8d4006af00 0x7f8d4006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:56.823 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:56.813+0000 7f8d53fff700 1 --2- 172.21.15.143:0/4272182447 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f8d4006af00 0x7f8d4006d3c0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f8d540cee30 tx=0x7f8d44008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:56.823 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:56.817+0000 7f8d50ff9700 1 -- 172.21.15.143:0/4272182447 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8d54140f70 con 0x7f8d540cc8c0 2023-11-08T19:27:57.007 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.006+0000 7f8d5a7d2700 1 -- 172.21.15.143:0/4272182447 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f8d54140f70 con 0x7f8d540cc8c0 2023-11-08T19:27:57.010 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.009+0000 7f8d50ff9700 1 -- 172.21.15.143:0/4272182447 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v19)=0 v19) v1 ==== 122+0+0 (secure 0 0 0) 0x7f8d54140f70 con 0x7f8d540cc8c0 2023-11-08T19:27:57.013 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.011+0000 7f8d5a7d2700 1 -- 172.21.15.143:0/4272182447 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f8d4006af00 msgr2=0x7f8d4006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:57.013 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.011+0000 7f8d5a7d2700 1 --2- 172.21.15.143:0/4272182447 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f8d4006af00 0x7f8d4006d3c0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f8d540cee30 tx=0x7f8d44008040 comp rx=0 tx=0).stop 2023-11-08T19:27:57.013 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.011+0000 7f8d5a7d2700 1 -- 172.21.15.143:0/4272182447 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8d540cc8c0 msgr2=0x7f8d540b2d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:57.014 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.011+0000 7f8d5a7d2700 1 --2- 172.21.15.143:0/4272182447 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8d540cc8c0 0x7f8d540b2d60 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f8d48000c00 tx=0x7f8d48004750 comp rx=0 tx=0).stop 2023-11-08T19:27:57.014 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.011+0000 7f8d5a7d2700 1 -- 172.21.15.143:0/4272182447 shutdown_connections 2023-11-08T19:27:57.014 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.011+0000 7f8d5a7d2700 1 --2- 172.21.15.143:0/4272182447 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f8d4006af00 0x7f8d4006d3c0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:57.014 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.011+0000 7f8d5a7d2700 1 --2- 172.21.15.143:0/4272182447 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8d540c5880 0x7f8d540c8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:57.014 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.011+0000 7f8d5a7d2700 1 --2- 172.21.15.143:0/4272182447 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8d54143a30 0x7f8d54159640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:57.015 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.011+0000 7f8d5a7d2700 1 --2- 172.21.15.143:0/4272182447 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8d540cc8c0 0x7f8d540b2d60 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:57.015 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.011+0000 7f8d5a7d2700 1 -- 172.21.15.143:0/4272182447 >> 172.21.15.143:0/4272182447 conn(0x7f8d540b1c70 msgr2=0x7f8d540cf030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:57.015 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.011+0000 7f8d5a7d2700 1 -- 172.21.15.143:0/4272182447 shutdown_connections 2023-11-08T19:27:57.015 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.012+0000 7f8d5a7d2700 1 -- 172.21.15.143:0/4272182447 wait complete. 2023-11-08T19:27:57.025 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-08T19:27:57.026 DEBUG:teuthology.orchestra.run.smithi143:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-08T19:27:57.028 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T19:27:57.071 DEBUG:teuthology.orchestra.run.smithi174:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-08T19:27:57.072 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T19:27:57.122 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-08T19:27:57.123 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-08T19:27:57.123 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:27:57.180 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284264k,nr_inodes=4071066,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=44,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20643 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-08T19:27:57.181 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2023-11-08T19:27:57.181 DEBUG:teuthology.orchestra.run.smithi174:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:27:57.201 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284224k,nr_inodes=4071056,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=31,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19549 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260544k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-08T19:27:57.202 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-08T19:27:57.202 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:27:57.238 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284264k,nr_inodes=4071066,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=44,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20643 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-08T19:27:57.239 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2023-11-08T19:27:57.239 DEBUG:teuthology.orchestra.run.smithi174:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:27:57.260 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284224k,nr_inodes=4071056,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=31,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19549 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260544k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-08T19:27:57.261 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-08T19:27:57.261 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:27:57.398 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.397+0000 7fcacfe0a700 1 Processor -- start 2023-11-08T19:27:57.399 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.397+0000 7fcacfe0a700 1 -- start start 2023-11-08T19:27:57.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.398+0000 7fcacfe0a700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcac8159870 0x7fcac80b2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:57.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.398+0000 7fcacfe0a700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fcac80cc410 0x7fcac80cc820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:57.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.398+0000 7fcacfe0a700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcac80b2a60 con 0x7fcac806ed60 2023-11-08T19:27:57.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.398+0000 7fcacfe0a700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcac806b090 con 0x7fcac80cc410 2023-11-08T19:27:57.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.398+0000 7fcacfe0a700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcac806bbc0 con 0x7fcac8159870 2023-11-08T19:27:57.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.398+0000 7fcace3a7700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fcac80cc410 0x7fcac80cc820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:57.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.398+0000 7fcacdba6700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcac8159870 0x7fcac80b2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:57.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.398+0000 7fcacd3a5700 1 --1- >> v1:172.21.15.174:6789/0 conn(0x7fcac806ed60 0x7fcac806b4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.174:6789/0 says I am v1:172.21.15.143:33190/0 (socket says 172.21.15.143:33190) 2023-11-08T19:27:57.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.398+0000 7fcace3a7700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fcac80cc410 0x7fcac80cc820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.143:37836/0 (socket says 172.21.15.143:37836) 2023-11-08T19:27:57.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.398+0000 7fcacd3a5700 1 -- 172.21.15.143:0/2284726918 learned_addr learned my addr 172.21.15.143:0/2284726918 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:27:57.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.399+0000 7fcacdba6700 1 -- 172.21.15.143:0/2284726918 >> v2:172.21.15.174:3300/0 conn(0x7fcac80cc410 msgr2=0x7fcac80cc820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:57.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.399+0000 7fcacdba6700 1 --2- 172.21.15.143:0/2284726918 >> v2:172.21.15.174:3300/0 conn(0x7fcac80cc410 0x7fcac80cc820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:57.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.399+0000 7fcacdba6700 1 -- 172.21.15.143:0/2284726918 >> v1:172.21.15.174:6789/0 conn(0x7fcac806ed60 legacy=0x7fcac806b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:57.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.399+0000 7fcacdba6700 1 -- 172.21.15.143:0/2284726918 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcac80b2d40 con 0x7fcac8159870 2023-11-08T19:27:57.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.399+0000 7fcace3a7700 1 --2- 172.21.15.143:0/2284726918 >> v2:172.21.15.174:3300/0 conn(0x7fcac80cc410 0x7fcac80cc820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T19:27:57.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.399+0000 7fcacdba6700 1 --2- 172.21.15.143:0/2284726918 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcac8159870 0x7fcac80b2210 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fcab8009820 tx=0x7fcab8009b30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9496b74c935f8592 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:57.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.399+0000 7fcaccba4700 1 -- 172.21.15.143:0/2284726918 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab8005e30 con 0x7fcac8159870 2023-11-08T19:27:57.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.399+0000 7fcaccba4700 1 -- 172.21.15.143:0/2284726918 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcab8006030 con 0x7fcac8159870 2023-11-08T19:27:57.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.399+0000 7fcaccba4700 1 -- 172.21.15.143:0/2284726918 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab80063c0 con 0x7fcac8159870 2023-11-08T19:27:57.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.399+0000 7fcacfe0a700 1 -- 172.21.15.143:0/2284726918 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcac8159870 msgr2=0x7fcac80b2210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:57.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.399+0000 7fcacfe0a700 1 --2- 172.21.15.143:0/2284726918 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcac8159870 0x7fcac80b2210 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fcab8009820 tx=0x7fcab8009b30 comp rx=0 tx=0).stop 2023-11-08T19:27:57.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.399+0000 7fcacfe0a700 1 -- 172.21.15.143:0/2284726918 shutdown_connections 2023-11-08T19:27:57.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.399+0000 7fcacfe0a700 1 --2- 172.21.15.143:0/2284726918 >> v2:172.21.15.174:3300/0 conn(0x7fcac80cc410 0x7fcac80cc820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:57.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.399+0000 7fcacfe0a700 1 --2- 172.21.15.143:0/2284726918 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcac8159870 0x7fcac80b2210 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:57.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.399+0000 7fcacfe0a700 1 -- 172.21.15.143:0/2284726918 >> 172.21.15.143:0/2284726918 conn(0x7fcac8099ab0 msgr2=0x7fcac80d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:57.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.399+0000 7fcacfe0a700 1 -- 172.21.15.143:0/2284726918 shutdown_connections 2023-11-08T19:27:57.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.400+0000 7fcacfe0a700 1 -- 172.21.15.143:0/2284726918 wait complete. 2023-11-08T19:27:57.410 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.401+0000 7fcacfe0a700 1 Processor -- start 2023-11-08T19:27:57.410 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.401+0000 7fcacfe0a700 1 -- start start 2023-11-08T19:27:57.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.401+0000 7fcacfe0a700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcac806ed60 0x7fcac814f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:57.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.401+0000 7fcacfe0a700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcac80cc410 0x7fcac80c4470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:57.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.401+0000 7fcacfe0a700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcac8159870 0x7fcac80c49b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:57.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.401+0000 7fcacfe0a700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcac80c77d0 con 0x7fcac8159870 2023-11-08T19:27:57.412 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.401+0000 7fcacfe0a700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcac80c7ab0 con 0x7fcac80cc410 2023-11-08T19:27:57.412 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.401+0000 7fcacfe0a700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcac80c7d90 con 0x7fcac806ed60 2023-11-08T19:27:57.412 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.401+0000 7fcace3a7700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcac8159870 0x7fcac80c49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:57.412 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.402+0000 7fcace3a7700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcac8159870 0x7fcac80c49b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54344/0 (socket says 172.21.15.143:54344) 2023-11-08T19:27:57.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.402+0000 7fcace3a7700 1 -- 172.21.15.143:0/3059263869 learned_addr learned my addr 172.21.15.143:0/3059263869 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:57.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.402+0000 7fcacd3a5700 1 --2- 172.21.15.143:0/3059263869 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcac80cc410 0x7fcac80c4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:57.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.402+0000 7fcacdba6700 1 --2- 172.21.15.143:0/3059263869 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcac806ed60 0x7fcac814f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:57.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.402+0000 7fcace3a7700 1 -- 172.21.15.143:0/3059263869 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcac806ed60 msgr2=0x7fcac814f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:57.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.402+0000 7fcace3a7700 1 --2- 172.21.15.143:0/3059263869 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcac806ed60 0x7fcac814f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:57.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.402+0000 7fcace3a7700 1 -- 172.21.15.143:0/3059263869 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcac80cc410 msgr2=0x7fcac80c4470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:57.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.402+0000 7fcace3a7700 1 --2- 172.21.15.143:0/3059263869 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcac80cc410 0x7fcac80c4470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:57.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.402+0000 7fcace3a7700 1 -- 172.21.15.143:0/3059263869 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcac0009a70 con 0x7fcac8159870 2023-11-08T19:27:57.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.402+0000 7fcace3a7700 1 --2- 172.21.15.143:0/3059263869 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcac8159870 0x7fcac80c49b0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fcac000efc0 tx=0x7fcac000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:57.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.402+0000 7fcabe7fc700 1 -- 172.21.15.143:0/3059263869 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcac000ca90 con 0x7fcac8159870 2023-11-08T19:27:57.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.402+0000 7fcabe7fc700 1 -- 172.21.15.143:0/3059263869 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcac000f040 con 0x7fcac8159870 2023-11-08T19:27:57.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.402+0000 7fcacfe0a700 1 -- 172.21.15.143:0/3059263869 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcab8009430 con 0x7fcac8159870 2023-11-08T19:27:57.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.402+0000 7fcabe7fc700 1 -- 172.21.15.143:0/3059263869 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcac000ce30 con 0x7fcac8159870 2023-11-08T19:27:57.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.402+0000 7fcacfe0a700 1 -- 172.21.15.143:0/3059263869 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcac80c8350 con 0x7fcac8159870 2023-11-08T19:27:57.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.403+0000 7fcacfe0a700 1 -- 172.21.15.143:0/3059263869 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcac804eee0 con 0x7fcac8159870 2023-11-08T19:27:57.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.404+0000 7fcabe7fc700 1 -- 172.21.15.143:0/3059263869 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fcac0014480 con 0x7fcac8159870 2023-11-08T19:27:57.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.404+0000 7fcabe7fc700 1 --2- 172.21.15.143:0/3059263869 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fcaac06af00 0x7fcaac06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:57.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.404+0000 7fcabe7fc700 1 -- 172.21.15.143:0/3059263869 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6034+0+0 (secure 0 0 0) 0x7fcac0089da0 con 0x7fcac8159870 2023-11-08T19:27:57.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.404+0000 7fcacdba6700 1 --2- 172.21.15.143:0/3059263869 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fcaac06af00 0x7fcaac06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:57.417 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.405+0000 7fcacdba6700 1 --2- 172.21.15.143:0/3059263869 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fcaac06af00 0x7fcaac06d3c0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fcab8000c00 tx=0x7fcab8008070 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:57.417 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.408+0000 7fcabe7fc700 1 -- 172.21.15.143:0/3059263869 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcac0054d00 con 0x7fcac8159870 2023-11-08T19:27:57.640 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.639+0000 7fcacfe0a700 1 -- 172.21.15.143:0/3059263869 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcac80c3f30 con 0x7fcac8159870 2023-11-08T19:27:57.641 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.639+0000 7fcabe7fc700 1 -- 172.21.15.143:0/3059263869 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) v1 ==== 94+0+3702 (secure 0 0 0) 0x7fcac0058320 con 0x7fcac8159870 2023-11-08T19:27:57.641 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:27:57.642 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":45,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5510,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6836/3796015890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3796015890},{"type":"v1","addr":"172.21.15.174:6836","nonce":3796015890}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37},{"gid":6157,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/926208760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6835","nonce":926208760},{"type":"v1","addr":"172.21.15.174:6837","nonce":926208760}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":45},{"gid":6471,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6834/432456394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":432456394},{"type":"v1","addr":"172.21.15.143:6834","nonce":432456394}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37},{"gid":6474,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/310920972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":310920972},{"type":"v1","addr":"172.21.15.143:6835","nonce":310920972}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36}],"filesystems":[]} 2023-11-08T19:27:57.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.641+0000 7fcacfe0a700 1 -- 172.21.15.143:0/3059263869 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fcaac06af00 msgr2=0x7fcaac06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:57.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.641+0000 7fcacfe0a700 1 --2- 172.21.15.143:0/3059263869 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fcaac06af00 0x7fcaac06d3c0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fcab8000c00 tx=0x7fcab8008070 comp rx=0 tx=0).stop 2023-11-08T19:27:57.644 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.641+0000 7fcacfe0a700 1 -- 172.21.15.143:0/3059263869 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcac8159870 msgr2=0x7fcac80c49b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:57.644 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.641+0000 7fcacfe0a700 1 --2- 172.21.15.143:0/3059263869 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcac8159870 0x7fcac80c49b0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fcac000efc0 tx=0x7fcac000c3a0 comp rx=0 tx=0).stop 2023-11-08T19:27:57.644 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.641+0000 7fcacfe0a700 1 -- 172.21.15.143:0/3059263869 shutdown_connections 2023-11-08T19:27:57.644 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.641+0000 7fcacfe0a700 1 --2- 172.21.15.143:0/3059263869 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fcaac06af00 0x7fcaac06d3c0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:57.644 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.641+0000 7fcacfe0a700 1 --2- 172.21.15.143:0/3059263869 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcac806ed60 0x7fcac814f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:57.645 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.641+0000 7fcacfe0a700 1 --2- 172.21.15.143:0/3059263869 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcac8159870 0x7fcac80c49b0 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:57.645 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.641+0000 7fcacfe0a700 1 --2- 172.21.15.143:0/3059263869 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcac80cc410 0x7fcac80c4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:57.645 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.642+0000 7fcacfe0a700 1 -- 172.21.15.143:0/3059263869 >> 172.21.15.143:0/3059263869 conn(0x7fcac8099ab0 msgr2=0x7fcac80b1940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:57.645 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.642+0000 7fcacfe0a700 1 -- 172.21.15.143:0/3059263869 shutdown_connections 2023-11-08T19:27:57.646 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.642+0000 7fcacfe0a700 1 -- 172.21.15.143:0/3059263869 wait complete. 2023-11-08T19:27:57.646 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 45 2023-11-08T19:27:57.655 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-08T19:27:57.656 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-08T19:27:57.656 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-08T19:27:57.656 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T19:27:57.657 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-08T19:27:57.657 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-08T19:27:57.657 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-08T19:27:57.657 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T19:27:57.658 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-08T19:27:57.658 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-08T19:27:57.658 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-08T19:27:57.659 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T19:27:57.659 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-08T19:27:57.659 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-08T19:27:57.660 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-08T19:27:57.660 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T19:27:57.667 INFO:tasks.ceph.mds.a:Stopped 2023-11-08T19:27:57.668 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-08T19:27:57.668 INFO:tasks.ceph.mds.c:Stopped 2023-11-08T19:27:57.669 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-08T19:27:57.671 INFO:tasks.ceph.mds.b:Stopped 2023-11-08T19:27:57.671 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-08T19:27:57.672 INFO:tasks.ceph.mds.d:Stopped 2023-11-08T19:27:57.672 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-08T19:27:57.710 INFO:tasks.ceph.mds.a:Started 2023-11-08T19:27:57.710 DEBUG:teuthology.parallel:result is None 2023-11-08T19:27:57.711 INFO:tasks.ceph.mds.c:Started 2023-11-08T19:27:57.711 DEBUG:teuthology.parallel:result is None 2023-11-08T19:27:57.713 INFO:tasks.ceph.mds.b:Started 2023-11-08T19:27:57.714 DEBUG:teuthology.parallel:result is None 2023-11-08T19:27:57.714 INFO:tasks.ceph.mds.d:Started 2023-11-08T19:27:57.715 DEBUG:teuthology.parallel:result is None 2023-11-08T19:27:57.715 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-08T19:27:57.788 INFO:tasks.ceph.mds.c.smithi143.stdout:starting mds.c at 2023-11-08T19:27:57.789 INFO:tasks.ceph.mds.a.smithi143.stdout:starting mds.a at 2023-11-08T19:27:57.802 INFO:tasks.ceph.mds.b.smithi174.stdout:starting mds.b at 2023-11-08T19:27:57.803 INFO:tasks.ceph.mds.d.smithi174.stdout:starting mds.d at 2023-11-08T19:27:57.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.841+0000 7f0533c8b700 1 Processor -- start 2023-11-08T19:27:57.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.842+0000 7f0533c8b700 1 -- start start 2023-11-08T19:27:57.845 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.842+0000 7f0533c8b700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f052c06c730 0x7f052c0cd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:57.846 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.842+0000 7f0533c8b700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f052c15de80 0x7f052c0cbf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:57.846 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.842+0000 7f0533c8b700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f052c0cd480 con 0x7f052c06c030 2023-11-08T19:27:57.846 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.842+0000 7f0533c8b700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f052c0cc870 con 0x7f052c15de80 2023-11-08T19:27:57.846 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.842+0000 7f0533c8b700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f052c0ccb50 con 0x7f052c06c730 2023-11-08T19:27:57.846 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.842+0000 7f0532228700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f052c15de80 0x7f052c0cbf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:57.847 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.842+0000 7f0531226700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f052c06c030 0x7f052c06cb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:50528/0 (socket says 172.21.15.143:50528) 2023-11-08T19:27:57.847 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.842+0000 7f0531226700 1 -- 172.21.15.143:0/1080100972 learned_addr learned my addr 172.21.15.143:0/1080100972 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:27:57.847 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.842+0000 7f0532228700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f052c15de80 0x7f052c0cbf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54376/0 (socket says 172.21.15.143:54376) 2023-11-08T19:27:57.847 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.843+0000 7f0531a27700 1 --2- 172.21.15.143:0/1080100972 >> v2:172.21.15.174:3300/0 conn(0x7f052c06c730 0x7f052c0cd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:57.847 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.843+0000 7f0532228700 1 -- 172.21.15.143:0/1080100972 >> v2:172.21.15.174:3300/0 conn(0x7f052c06c730 msgr2=0x7f052c0cd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:57.848 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.843+0000 7f0532228700 1 --2- 172.21.15.143:0/1080100972 >> v2:172.21.15.174:3300/0 conn(0x7f052c06c730 0x7f052c0cd910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:57.848 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.843+0000 7f0532228700 1 -- 172.21.15.143:0/1080100972 >> v1:172.21.15.143:6789/0 conn(0x7f052c06c030 legacy=0x7f052c06cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:57.848 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.843+0000 7f0530a25700 1 -- 172.21.15.143:0/1080100972 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2540143360 0 0) 0x7f052c0cd480 con 0x7f052c06c030 2023-11-08T19:27:57.848 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.843+0000 7f0532228700 1 -- 172.21.15.143:0/1080100972 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f052c0cce30 con 0x7f052c15de80 2023-11-08T19:27:57.848 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.843+0000 7f0531a27700 1 --2- 172.21.15.143:0/1080100972 >> v2:172.21.15.174:3300/0 conn(0x7f052c06c730 0x7f052c0cd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:27:57.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.843+0000 7f0532228700 1 --2- 172.21.15.143:0/1080100972 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f052c15de80 0x7f052c0cbf00 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f052c08bd60 tx=0x7f051c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceff968a9fedc85a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:57.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.843+0000 7f0530a25700 1 -- 172.21.15.143:0/1080100972 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f051c011830 con 0x7f052c15de80 2023-11-08T19:27:57.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.843+0000 7f0530a25700 1 -- 172.21.15.143:0/1080100972 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f051c011990 con 0x7f052c15de80 2023-11-08T19:27:57.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.843+0000 7f0530a25700 1 -- 172.21.15.143:0/1080100972 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f051c011d20 con 0x7f052c15de80 2023-11-08T19:27:57.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.844+0000 7f0533c8b700 1 -- 172.21.15.143:0/1080100972 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f052c15de80 msgr2=0x7f052c0cbf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:57.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.844+0000 7f0533c8b700 1 --2- 172.21.15.143:0/1080100972 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f052c15de80 0x7f052c0cbf00 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f052c08bd60 tx=0x7f051c009b20 comp rx=0 tx=0).stop 2023-11-08T19:27:57.850 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.844+0000 7f0533c8b700 1 -- 172.21.15.143:0/1080100972 shutdown_connections 2023-11-08T19:27:57.850 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.844+0000 7f0533c8b700 1 --2- 172.21.15.143:0/1080100972 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f052c15de80 0x7f052c0cbf00 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:57.850 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.844+0000 7f0533c8b700 1 --2- 172.21.15.143:0/1080100972 >> v2:172.21.15.174:3300/0 conn(0x7f052c06c730 0x7f052c0cd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:57.850 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.844+0000 7f0533c8b700 1 -- 172.21.15.143:0/1080100972 >> 172.21.15.143:0/1080100972 conn(0x7f052c08b000 msgr2=0x7f052c09ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:57.850 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.844+0000 7f0533c8b700 1 -- 172.21.15.143:0/1080100972 shutdown_connections 2023-11-08T19:27:57.851 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.844+0000 7f0533c8b700 1 -- 172.21.15.143:0/1080100972 wait complete. 2023-11-08T19:27:57.854 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.845+0000 7f0533c8b700 1 Processor -- start 2023-11-08T19:27:57.854 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.845+0000 7f0533c8b700 1 -- start start 2023-11-08T19:27:57.855 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.846+0000 7f0533c8b700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f052c06c030 0x7f052c150ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:57.855 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.846+0000 7f0533c8b700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f052c06c730 0x7f052c1506c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:57.855 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.846+0000 7f0533c8b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f052c15de80 0x7f052c15c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:57.855 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.846+0000 7f0533c8b700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f052c1f3560 con 0x7f052c06c730 2023-11-08T19:27:57.855 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.846+0000 7f0533c8b700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f052c0c92d0 con 0x7f052c15de80 2023-11-08T19:27:57.855 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.846+0000 7f0533c8b700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f052c0c95b0 con 0x7f052c06c030 2023-11-08T19:27:57.856 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.846+0000 7f0531226700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f052c06c730 0x7f052c1506c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:57.856 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.846+0000 7f0531226700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f052c06c730 0x7f052c1506c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54380/0 (socket says 172.21.15.143:54380) 2023-11-08T19:27:57.856 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.846+0000 7f0531226700 1 -- 172.21.15.143:0/2947966884 learned_addr learned my addr 172.21.15.143:0/2947966884 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:57.856 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.846+0000 7f0531a27700 1 --2- 172.21.15.143:0/2947966884 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f052c06c030 0x7f052c150ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:57.856 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.846+0000 7f0532228700 1 --2- 172.21.15.143:0/2947966884 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f052c15de80 0x7f052c15c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:57.857 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.846+0000 7f0531226700 1 -- 172.21.15.143:0/2947966884 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f052c06c030 msgr2=0x7f052c150ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:57.857 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.846+0000 7f0531226700 1 --2- 172.21.15.143:0/2947966884 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f052c06c030 0x7f052c150ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:57.857 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.846+0000 7f0531226700 1 -- 172.21.15.143:0/2947966884 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f052c15de80 msgr2=0x7f052c15c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:57.857 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.846+0000 7f0531226700 1 --2- 172.21.15.143:0/2947966884 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f052c15de80 0x7f052c15c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:57.857 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.846+0000 7f0531226700 1 -- 172.21.15.143:0/2947966884 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f051c0094a0 con 0x7f052c06c730 2023-11-08T19:27:57.858 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.846+0000 7f0531226700 1 --2- 172.21.15.143:0/2947966884 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f052c06c730 0x7f052c1506c0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f052000c540 tx=0x7f052000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:57.858 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.846+0000 7f052a7fc700 1 -- 172.21.15.143:0/2947966884 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0520004c80 con 0x7f052c06c730 2023-11-08T19:27:57.858 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.846+0000 7f052a7fc700 1 -- 172.21.15.143:0/2947966884 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0520004de0 con 0x7f052c06c730 2023-11-08T19:27:57.858 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.846+0000 7f0533c8b700 1 -- 172.21.15.143:0/2947966884 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f052c0c9890 con 0x7f052c06c730 2023-11-08T19:27:57.858 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.846+0000 7f052a7fc700 1 -- 172.21.15.143:0/2947966884 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0520007310 con 0x7f052c06c730 2023-11-08T19:27:57.859 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.846+0000 7f0533c8b700 1 -- 172.21.15.143:0/2947966884 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f052c0c9e50 con 0x7f052c06c730 2023-11-08T19:27:57.859 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.847+0000 7f0533c8b700 1 -- 172.21.15.143:0/2947966884 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f052c0ca070 con 0x7f052c06c730 2023-11-08T19:27:57.859 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.848+0000 7f052a7fc700 1 -- 172.21.15.143:0/2947966884 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0520007470 con 0x7f052c06c730 2023-11-08T19:27:57.859 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.848+0000 7f052a7fc700 1 --2- 172.21.15.143:0/2947966884 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f051806af00 0x7f051806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:57.859 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.848+0000 7f052a7fc700 1 -- 172.21.15.143:0/2947966884 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6034+0+0 (secure 0 0 0) 0x7f05200892a0 con 0x7f052c06c730 2023-11-08T19:27:57.860 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.848+0000 7f0531a27700 1 --2- 172.21.15.143:0/2947966884 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f051806af00 0x7f051806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:57.860 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.849+0000 7f0531a27700 1 --2- 172.21.15.143:0/2947966884 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f051806af00 0x7f051806d3c0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f052400d620 tx=0x7f0524007810 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:57.860 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:57.853+0000 7f052a7fc700 1 -- 172.21.15.143:0/2947966884 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0520054290 con 0x7f052c06c730 2023-11-08T19:27:58.059 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:58.058+0000 7f0533c8b700 1 -- 172.21.15.143:0/2947966884 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f052c04eee0 con 0x7f052c06c730 2023-11-08T19:27:58.682 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:58.681+0000 7f052a7fc700 1 -- 172.21.15.143:0/2947966884 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v86) v1 ==== 97+0+0 (secure 0 0 0) 0x7f05200578b0 con 0x7f052c06c730 2023-11-08T19:27:58.686 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:58.684+0000 7f0533c8b700 1 -- 172.21.15.143:0/2947966884 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f051806af00 msgr2=0x7f051806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:58.686 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:58.684+0000 7f0533c8b700 1 --2- 172.21.15.143:0/2947966884 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f051806af00 0x7f051806d3c0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f052400d620 tx=0x7f0524007810 comp rx=0 tx=0).stop 2023-11-08T19:27:58.686 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:58.684+0000 7f0533c8b700 1 -- 172.21.15.143:0/2947966884 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f052c06c730 msgr2=0x7f052c1506c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:58.686 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:58.684+0000 7f0533c8b700 1 --2- 172.21.15.143:0/2947966884 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f052c06c730 0x7f052c1506c0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f052000c540 tx=0x7f052000c850 comp rx=0 tx=0).stop 2023-11-08T19:27:58.687 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:58.684+0000 7f0533c8b700 1 -- 172.21.15.143:0/2947966884 shutdown_connections 2023-11-08T19:27:58.687 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:58.684+0000 7f0533c8b700 1 --2- 172.21.15.143:0/2947966884 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f051806af00 0x7f051806d3c0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:58.687 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:58.684+0000 7f0533c8b700 1 --2- 172.21.15.143:0/2947966884 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f052c06c030 0x7f052c150ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:58.687 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:58.684+0000 7f0533c8b700 1 --2- 172.21.15.143:0/2947966884 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f052c15de80 0x7f052c15c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:58.687 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:58.684+0000 7f0533c8b700 1 --2- 172.21.15.143:0/2947966884 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f052c06c730 0x7f052c1506c0 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:58.688 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:58.684+0000 7f0533c8b700 1 -- 172.21.15.143:0/2947966884 >> 172.21.15.143:0/2947966884 conn(0x7f052c08b000 msgr2=0x7f052c1591a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:58.688 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:58.684+0000 7f0533c8b700 1 -- 172.21.15.143:0/2947966884 shutdown_connections 2023-11-08T19:27:58.688 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:58.684+0000 7f0533c8b700 1 -- 172.21.15.143:0/2947966884 wait complete. 2023-11-08T19:27:58.689 INFO:teuthology.orchestra.run.smithi143.stderr: removed all blocklist entries 2023-11-08T19:27:58.703 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-08T19:27:58.829 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:58.828+0000 7f1151216700 1 Processor -- start 2023-11-08T19:27:58.829 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:58.828+0000 7f1151216700 1 -- start start 2023-11-08T19:27:58.832 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:58.828+0000 7f1151216700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f114c15dd00 0x7f114c0cca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:58.832 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:58.828+0000 7f1151216700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f114c1618f0 0x7f114c06ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:58.832 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:58.828+0000 7f1151216700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f114c06b150 0x7f114c0b2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:58.832 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:58.828+0000 7f1151216700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f114c06b600 con 0x7f114c15dd00 2023-11-08T19:27:58.832 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:58.828+0000 7f1151216700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f114c0cad50 con 0x7f114c06b150 2023-11-08T19:27:58.833 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:58.828+0000 7f1151216700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f114c0ccfb0 con 0x7f114c1618f0 2023-11-08T19:27:58.833 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:58.828+0000 7f114ad9d700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f114c15dd00 0x7f114c0cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:58.833 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:58.828+0000 7f114ad9d700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f114c15dd00 0x7f114c0cca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54388/0 (socket says 172.21.15.143:54388) 2023-11-08T19:27:58.833 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:58.828+0000 7f114ad9d700 1 -- 172.21.15.143:0/3533393608 learned_addr learned my addr 172.21.15.143:0/3533393608 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:58.834 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:58.829+0000 7f114ad9d700 1 -- 172.21.15.143:0/3533393608 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f114c1618f0 msgr2=0x7f114c06ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:58.834 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:58.829+0000 7f114ad9d700 1 --2- 172.21.15.143:0/3533393608 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f114c1618f0 0x7f114c06ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:58.834 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:58.829+0000 7f114ad9d700 1 -- 172.21.15.143:0/3533393608 >> v2:172.21.15.174:3300/0 conn(0x7f114c06b150 msgr2=0x7f114c0b2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:58.834 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:58.829+0000 7f114ad9d700 1 --2- 172.21.15.143:0/3533393608 >> v2:172.21.15.174:3300/0 conn(0x7f114c06b150 0x7f114c0b2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:58.834 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:58.829+0000 7f114ad9d700 1 -- 172.21.15.143:0/3533393608 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f114c161df0 con 0x7f114c15dd00 2023-11-08T19:27:58.835 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:58.829+0000 7f114ad9d700 1 --2- 172.21.15.143:0/3533393608 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f114c15dd00 0x7f114c0cca70 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f114c0708b0 tx=0x7f1134009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f160516f0f8720d0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:58.835 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:58.829+0000 7f1149d9b700 1 -- 172.21.15.143:0/3533393608 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1134005e30 con 0x7f114c15dd00 2023-11-08T19:27:58.835 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:58.829+0000 7f1149d9b700 1 -- 172.21.15.143:0/3533393608 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1134004030 con 0x7f114c15dd00 2023-11-08T19:27:58.835 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:58.829+0000 7f1149d9b700 1 -- 172.21.15.143:0/3533393608 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11340043e0 con 0x7f114c15dd00 2023-11-08T19:27:58.835 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:58.829+0000 7f1151216700 1 -- 172.21.15.143:0/3533393608 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f114c15dd00 msgr2=0x7f114c0cca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:58.836 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:58.829+0000 7f1151216700 1 --2- 172.21.15.143:0/3533393608 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f114c15dd00 0x7f114c0cca70 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f114c0708b0 tx=0x7f1134009bd0 comp rx=0 tx=0).stop 2023-11-08T19:27:58.836 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:58.830+0000 7f1151216700 1 -- 172.21.15.143:0/3533393608 shutdown_connections 2023-11-08T19:27:58.836 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:58.830+0000 7f1151216700 1 --2- 172.21.15.143:0/3533393608 >> v2:172.21.15.174:3300/0 conn(0x7f114c06b150 0x7f114c0b2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:58.836 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:58.830+0000 7f1151216700 1 --2- 172.21.15.143:0/3533393608 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f114c15dd00 0x7f114c0cca70 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:58.836 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:58.830+0000 7f1151216700 1 --2- 172.21.15.143:0/3533393608 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f114c1618f0 0x7f114c06ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:58.837 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:58.830+0000 7f1151216700 1 -- 172.21.15.143:0/3533393608 >> 172.21.15.143:0/3533393608 conn(0x7f114c099ab0 msgr2=0x7f114c0d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:58.837 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:58.830+0000 7f1151216700 1 -- 172.21.15.143:0/3533393608 shutdown_connections 2023-11-08T19:27:58.837 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:58.830+0000 7f1151216700 1 -- 172.21.15.143:0/3533393608 wait complete. 2023-11-08T19:27:58.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:58.831+0000 7f1151216700 1 Processor -- start 2023-11-08T19:27:58.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:58.831+0000 7f1151216700 1 -- start start 2023-11-08T19:27:58.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:58.832+0000 7f1151216700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f114c06b150 0x7f114c151350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:58.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:58.832+0000 7f1151216700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f114c15dd00 0x7f114c0b2a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:58.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:58.832+0000 7f1151216700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f114c1618f0 0x7f114c0c5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:58.841 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:58.832+0000 7f1151216700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f114c0b1990 con 0x7f114c06b150 2023-11-08T19:27:58.841 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:58.832+0000 7f1151216700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f114c0b1c70 con 0x7f114c1618f0 2023-11-08T19:27:58.841 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:58.832+0000 7f1151216700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f114c0b1f50 con 0x7f114c15dd00 2023-11-08T19:27:58.841 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:58.832+0000 7f114ad9d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f114c06b150 0x7f114c151350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:58.841 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:58.832+0000 7f114ad9d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f114c06b150 0x7f114c151350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54392/0 (socket says 172.21.15.143:54392) 2023-11-08T19:27:58.842 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:58.832+0000 7f114ad9d700 1 -- 172.21.15.143:0/602650702 learned_addr learned my addr 172.21.15.143:0/602650702 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:58.842 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:58.832+0000 7f114a59c700 1 --2- 172.21.15.143:0/602650702 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f114c15dd00 0x7f114c0b2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:58.842 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:58.832+0000 7f114b59e700 1 --2- 172.21.15.143:0/602650702 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f114c1618f0 0x7f114c0c5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:58.842 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:58.832+0000 7f114ad9d700 1 -- 172.21.15.143:0/602650702 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f114c15dd00 msgr2=0x7f114c0b2a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:58.842 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:58.832+0000 7f114ad9d700 1 --2- 172.21.15.143:0/602650702 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f114c15dd00 0x7f114c0b2a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:58.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:58.832+0000 7f114ad9d700 1 -- 172.21.15.143:0/602650702 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f114c1618f0 msgr2=0x7f114c0c5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:58.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:58.832+0000 7f114ad9d700 1 --2- 172.21.15.143:0/602650702 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f114c1618f0 0x7f114c0c5ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:58.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:58.832+0000 7f114ad9d700 1 -- 172.21.15.143:0/602650702 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1134009500 con 0x7f114c06b150 2023-11-08T19:27:58.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:58.832+0000 7f114ad9d700 1 --2- 172.21.15.143:0/602650702 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f114c06b150 0x7f114c151350 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f1134009130 tx=0x7f1134004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:58.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:58.833+0000 7f11437fe700 1 -- 172.21.15.143:0/602650702 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11340045a0 con 0x7f114c06b150 2023-11-08T19:27:58.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:58.833+0000 7f11437fe700 1 -- 172.21.15.143:0/602650702 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f113401d430 con 0x7f114c06b150 2023-11-08T19:27:58.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:58.833+0000 7f11437fe700 1 -- 172.21.15.143:0/602650702 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f113401d7e0 con 0x7f114c06b150 2023-11-08T19:27:58.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:58.833+0000 7f1151216700 1 -- 172.21.15.143:0/602650702 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f114c0b2230 con 0x7f114c06b150 2023-11-08T19:27:58.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:58.833+0000 7f1151216700 1 -- 172.21.15.143:0/602650702 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f114c151890 con 0x7f114c06b150 2023-11-08T19:27:58.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:58.833+0000 7f1151216700 1 -- 172.21.15.143:0/602650702 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f114c0b2750 con 0x7f114c06b150 2023-11-08T19:27:58.845 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:58.834+0000 7f11437fe700 1 -- 172.21.15.143:0/602650702 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f113401d940 con 0x7f114c06b150 2023-11-08T19:27:58.845 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:58.834+0000 7f11437fe700 1 --2- 172.21.15.143:0/602650702 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f112c06af30 0x7f112c06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:58.845 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:58.834+0000 7f11437fe700 1 -- 172.21.15.143:0/602650702 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5920+0+0 (secure 0 0 0) 0x7f113408aef0 con 0x7f114c06b150 2023-11-08T19:27:58.845 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:58.835+0000 7f114a59c700 1 --2- 172.21.15.143:0/602650702 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f112c06af30 0x7f112c06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:58.845 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:58.835+0000 7f114a59c700 1 --2- 172.21.15.143:0/602650702 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f112c06af30 0x7f112c06d3f0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f114c0ca040 tx=0x7f113c009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:58.846 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:58.839+0000 7f11437fe700 1 -- 172.21.15.143:0/602650702 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f114c0b2750 con 0x7f114c06b150 2023-11-08T19:27:59.112 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.110+0000 7f1151216700 1 -- 172.21.15.143:0/602650702 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f114c0b2750 con 0x7f114c06b150 2023-11-08T19:27:59.113 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.111+0000 7f11437fe700 1 -- 172.21.15.143:0/602650702 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v12) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f114c0b2750 con 0x7f114c06b150 2023-11-08T19:27:59.113 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:27:59.113 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2023-11-08T19:27:59.113 INFO:teuthology.orchestra.run.smithi143.stdout: "auth_dump": [ 2023-11-08T19:27:59.114 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-08T19:27:59.114 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "mds.a", 2023-11-08T19:27:59.114 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQBO4EtlqQPkOhAACwt3IOZLFtEQle9/FFliYA==", 2023-11-08T19:27:59.114 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-08T19:27:59.114 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2023-11-08T19:27:59.115 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow *", 2023-11-08T19:27:59.115 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow *", 2023-11-08T19:27:59.115 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-08T19:27:59.115 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-08T19:27:59.115 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-08T19:27:59.116 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-08T19:27:59.116 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "mds.b", 2023-11-08T19:27:59.116 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQBP4EtlHJqqCxAAxgT7Ce6QICWZ0qebHWZhMQ==", 2023-11-08T19:27:59.116 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-08T19:27:59.117 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2023-11-08T19:27:59.117 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow *", 2023-11-08T19:27:59.117 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow *", 2023-11-08T19:27:59.117 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-08T19:27:59.117 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-08T19:27:59.118 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-08T19:27:59.118 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-08T19:27:59.118 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "mds.c", 2023-11-08T19:27:59.118 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQBP4EtlGEeDBhAAihr82JiBJZR0rRkQj2n8wA==", 2023-11-08T19:27:59.118 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-08T19:27:59.119 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2023-11-08T19:27:59.119 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow *", 2023-11-08T19:27:59.119 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow *", 2023-11-08T19:27:59.119 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-08T19:27:59.119 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-08T19:27:59.119 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-08T19:27:59.120 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-08T19:27:59.120 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "mds.d", 2023-11-08T19:27:59.120 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQBP4EtlW5//EhAArVOkzNpCYd6Skl6eh5EYUQ==", 2023-11-08T19:27:59.120 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-08T19:27:59.120 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2023-11-08T19:27:59.121 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow *", 2023-11-08T19:27:59.121 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow *", 2023-11-08T19:27:59.121 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-08T19:27:59.121 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-08T19:27:59.121 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-08T19:27:59.122 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-08T19:27:59.122 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.0", 2023-11-08T19:27:59.122 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQBS4EtloVd4KRAAi5nj/vB9UK/mWZ9lkaFAxg==", 2023-11-08T19:27:59.122 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-08T19:27:59.122 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2023-11-08T19:27:59.123 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2023-11-08T19:27:59.123 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-08T19:27:59.123 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-08T19:27:59.123 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-08T19:27:59.123 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-08T19:27:59.123 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.1", 2023-11-08T19:27:59.124 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQBV4EtlWlOaKRAAooz5P+AoTgyuL/QAv4REKA==", 2023-11-08T19:27:59.124 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-08T19:27:59.124 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2023-11-08T19:27:59.124 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2023-11-08T19:27:59.124 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-08T19:27:59.125 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-08T19:27:59.125 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-08T19:27:59.125 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-08T19:27:59.125 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.2", 2023-11-08T19:27:59.125 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQBY4EtlBpaWKhAAmob9TfEKw5x6dEIu8ALVGA==", 2023-11-08T19:27:59.126 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-08T19:27:59.126 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2023-11-08T19:27:59.126 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2023-11-08T19:27:59.126 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-08T19:27:59.126 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-08T19:27:59.127 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-08T19:27:59.127 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-08T19:27:59.127 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.3", 2023-11-08T19:27:59.127 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQBb4Etl5pkrKhAA38ATMC3NkZ4plRRA69i0vw==", 2023-11-08T19:27:59.127 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-08T19:27:59.127 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2023-11-08T19:27:59.128 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2023-11-08T19:27:59.128 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-08T19:27:59.128 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-08T19:27:59.128 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-08T19:27:59.128 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-08T19:27:59.129 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.4", 2023-11-08T19:27:59.129 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQBh4EtlqLnUMBAABIrSskahtrZR0ClfczX5Ag==", 2023-11-08T19:27:59.129 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-08T19:27:59.129 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2023-11-08T19:27:59.129 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2023-11-08T19:27:59.129 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-08T19:27:59.130 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-08T19:27:59.130 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-08T19:27:59.130 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-08T19:27:59.130 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.5", 2023-11-08T19:27:59.130 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQBk4Etlt9PTMBAAz55IHg6Jrin3BUAvhOM/Fg==", 2023-11-08T19:27:59.131 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-08T19:27:59.131 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2023-11-08T19:27:59.131 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2023-11-08T19:27:59.131 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-08T19:27:59.131 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-08T19:27:59.132 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-08T19:27:59.132 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-08T19:27:59.132 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.6", 2023-11-08T19:27:59.132 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQBn4EtlcTkyMBAA5fD3h4s+99rgd6fqHHCFMw==", 2023-11-08T19:27:59.132 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-08T19:27:59.133 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2023-11-08T19:27:59.133 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2023-11-08T19:27:59.133 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-08T19:27:59.133 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-08T19:27:59.133 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-08T19:27:59.133 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-08T19:27:59.134 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.7", 2023-11-08T19:27:59.134 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQBq4EtldEviMBAAyGYrHVGwxHdmtDOwWCSHDQ==", 2023-11-08T19:27:59.134 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-08T19:27:59.134 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2023-11-08T19:27:59.134 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2023-11-08T19:27:59.135 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-08T19:27:59.135 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-08T19:27:59.135 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-08T19:27:59.135 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-08T19:27:59.135 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "client.0", 2023-11-08T19:27:59.136 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQBP4Etl0GdSFxAAVkuHL0EnjrtTUMkj/afPkg==", 2023-11-08T19:27:59.136 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-08T19:27:59.136 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2023-11-08T19:27:59.136 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow r", 2023-11-08T19:27:59.136 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-11-08T19:27:59.136 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-08T19:27:59.137 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-08T19:27:59.137 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-08T19:27:59.137 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "client.1", 2023-11-08T19:27:59.137 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQBP4EtlPQ5yIBAAiyQaqCJph2KOPBZPSIh4ZQ==", 2023-11-08T19:27:59.137 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-08T19:27:59.138 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2023-11-08T19:27:59.138 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow r", 2023-11-08T19:27:59.138 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow rw", 2023-11-08T19:27:59.138 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow rwx" 2023-11-08T19:27:59.138 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-08T19:27:59.139 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-08T19:27:59.139 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-08T19:27:59.139 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "client.2", 2023-11-08T19:27:59.139 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQBP4Etlm+8SHRAAAwxcy2EzUvtFw9/mTzyk7w==", 2023-11-08T19:27:59.139 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-08T19:27:59.140 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2023-11-08T19:27:59.140 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow r", 2023-11-08T19:27:59.140 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow rw", 2023-11-08T19:27:59.140 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow rwx" 2023-11-08T19:27:59.140 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-08T19:27:59.141 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-08T19:27:59.141 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-08T19:27:59.141 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "client.3", 2023-11-08T19:27:59.141 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQBP4EtlNDC8IxAACBkCM+3TXhJv5eE9xLvwKA==", 2023-11-08T19:27:59.141 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-08T19:27:59.142 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2023-11-08T19:27:59.142 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow r", 2023-11-08T19:27:59.142 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow rw", 2023-11-08T19:27:59.142 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow rwx" 2023-11-08T19:27:59.143 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-08T19:27:59.143 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-08T19:27:59.143 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-08T19:27:59.143 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "client.admin", 2023-11-08T19:27:59.143 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQBO4Etlr5yuHRAAblpqD18z9bT3Wmgn74dehg==", 2023-11-08T19:27:59.144 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-08T19:27:59.144 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow *", 2023-11-08T19:27:59.144 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow *", 2023-11-08T19:27:59.144 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow *", 2023-11-08T19:27:59.144 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-08T19:27:59.145 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-08T19:27:59.145 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-08T19:27:59.145 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-08T19:27:59.145 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "mgr.x", 2023-11-08T19:27:59.145 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQBO4EtlzrqBNxAA5rtcgkzroSTKuBBmeQRiig==", 2023-11-08T19:27:59.145 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-08T19:27:59.146 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow *", 2023-11-08T19:27:59.146 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile mgr", 2023-11-08T19:27:59.146 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-08T19:27:59.146 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-08T19:27:59.146 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-08T19:27:59.147 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-08T19:27:59.147 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "mgr.y", 2023-11-08T19:27:59.147 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQBO4Etl4Gl4MxAArxdUnjLDhTLO3feXHV4HyQ==", 2023-11-08T19:27:59.147 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2023-11-08T19:27:59.147 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow *", 2023-11-08T19:27:59.148 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile mgr", 2023-11-08T19:27:59.148 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2023-11-08T19:27:59.148 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-08T19:27:59.148 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-08T19:27:59.148 INFO:teuthology.orchestra.run.smithi143.stdout: ] 2023-11-08T19:27:59.149 INFO:teuthology.orchestra.run.smithi143.stdout:} 2023-11-08T19:27:59.150 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.113+0000 7f1151216700 1 -- 172.21.15.143:0/602650702 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f112c06af30 msgr2=0x7f112c06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:59.151 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.113+0000 7f1151216700 1 --2- 172.21.15.143:0/602650702 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f112c06af30 0x7f112c06d3f0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f114c0ca040 tx=0x7f113c009190 comp rx=0 tx=0).stop 2023-11-08T19:27:59.151 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.113+0000 7f1151216700 1 -- 172.21.15.143:0/602650702 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f114c06b150 msgr2=0x7f114c151350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:59.151 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.113+0000 7f1151216700 1 --2- 172.21.15.143:0/602650702 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f114c06b150 0x7f114c151350 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f1134009130 tx=0x7f1134004750 comp rx=0 tx=0).stop 2023-11-08T19:27:59.151 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.113+0000 7f1151216700 1 -- 172.21.15.143:0/602650702 shutdown_connections 2023-11-08T19:27:59.151 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.113+0000 7f1151216700 1 --2- 172.21.15.143:0/602650702 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f112c06af30 0x7f112c06d3f0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:59.152 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.113+0000 7f1151216700 1 --2- 172.21.15.143:0/602650702 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f114c1618f0 0x7f114c0c5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:59.152 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.113+0000 7f1151216700 1 --2- 172.21.15.143:0/602650702 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f114c06b150 0x7f114c151350 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:59.152 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.113+0000 7f1151216700 1 --2- 172.21.15.143:0/602650702 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f114c15dd00 0x7f114c0b2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:59.152 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.113+0000 7f1151216700 1 -- 172.21.15.143:0/602650702 >> 172.21.15.143:0/602650702 conn(0x7f114c099ab0 msgr2=0x7f114c15d030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:59.152 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.113+0000 7f1151216700 1 -- 172.21.15.143:0/602650702 shutdown_connections 2023-11-08T19:27:59.153 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.113+0000 7f1151216700 1 -- 172.21.15.143:0/602650702 wait complete. 2023-11-08T19:27:59.153 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:27:59.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.272+0000 7f8688f56700 1 Processor -- start 2023-11-08T19:27:59.274 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.272+0000 7f8688f56700 1 -- start start 2023-11-08T19:27:59.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.273+0000 7f8688f56700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f86840b2920 0x7f86840cf510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:59.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.273+0000 7f8688f56700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f86840cbe20 0x7f86840cc290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:59.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.273+0000 7f8688f56700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86840cf080 con 0x7f868415aa30 2023-11-08T19:27:59.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.273+0000 7f8688f56700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86840cc7d0 con 0x7f86840b2920 2023-11-08T19:27:59.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.273+0000 7f8688f56700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86840ce470 con 0x7f86840cbe20 2023-11-08T19:27:59.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.273+0000 7f868259c700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f868415aa30 0x7f86840b2210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:50554/0 (socket says 172.21.15.143:50554) 2023-11-08T19:27:59.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.273+0000 7f868259c700 1 -- 172.21.15.143:0/2464414242 learned_addr learned my addr 172.21.15.143:0/2464414242 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:27:59.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.273+0000 7f8682d9d700 1 --2- 172.21.15.143:0/2464414242 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f86840cbe20 0x7f86840cc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:59.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.273+0000 7f8681d9b700 1 --2- 172.21.15.143:0/2464414242 >> v2:172.21.15.174:3300/0 conn(0x7f86840b2920 0x7f86840cf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:59.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.273+0000 7f868159a700 1 -- 172.21.15.143:0/2464414242 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 656402185 0 0) 0x7f86840cf080 con 0x7f868415aa30 2023-11-08T19:27:59.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.273+0000 7f868159a700 1 -- 172.21.15.143:0/2464414242 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f866c0035b0 con 0x7f868415aa30 2023-11-08T19:27:59.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.274+0000 7f8682d9d700 1 -- 172.21.15.143:0/2464414242 >> v2:172.21.15.174:3300/0 conn(0x7f86840b2920 msgr2=0x7f86840cf510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:59.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.274+0000 7f8682d9d700 1 --2- 172.21.15.143:0/2464414242 >> v2:172.21.15.174:3300/0 conn(0x7f86840b2920 0x7f86840cf510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:59.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.274+0000 7f8682d9d700 1 -- 172.21.15.143:0/2464414242 >> v1:172.21.15.143:6789/0 conn(0x7f868415aa30 legacy=0x7f86840b2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:59.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.274+0000 7f8682d9d700 1 -- 172.21.15.143:0/2464414242 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86840ce750 con 0x7f86840cbe20 2023-11-08T19:27:59.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.274+0000 7f8682d9d700 1 --2- 172.21.15.143:0/2464414242 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f86840cbe20 0x7f86840cc290 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f867c00bfa0 tx=0x7f867c00d330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be1dedcfc55afe54 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:59.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.274+0000 7f868159a700 1 -- 172.21.15.143:0/2464414242 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f867c010070 con 0x7f86840cbe20 2023-11-08T19:27:59.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.274+0000 7f868159a700 1 -- 172.21.15.143:0/2464414242 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f867c00b2e0 con 0x7f86840cbe20 2023-11-08T19:27:59.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.274+0000 7f868159a700 1 -- 172.21.15.143:0/2464414242 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f867c00dc40 con 0x7f86840cbe20 2023-11-08T19:27:59.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.274+0000 7f8688f56700 1 -- 172.21.15.143:0/2464414242 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f86840cbe20 msgr2=0x7f86840cc290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:59.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.274+0000 7f8688f56700 1 --2- 172.21.15.143:0/2464414242 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f86840cbe20 0x7f86840cc290 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f867c00bfa0 tx=0x7f867c00d330 comp rx=0 tx=0).stop 2023-11-08T19:27:59.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.275+0000 7f8688f56700 1 -- 172.21.15.143:0/2464414242 shutdown_connections 2023-11-08T19:27:59.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.275+0000 7f8688f56700 1 --2- 172.21.15.143:0/2464414242 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f86840cbe20 0x7f86840cc290 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:59.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.275+0000 7f8688f56700 1 --2- 172.21.15.143:0/2464414242 >> v2:172.21.15.174:3300/0 conn(0x7f86840b2920 0x7f86840cf510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:59.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.275+0000 7f8688f56700 1 -- 172.21.15.143:0/2464414242 >> 172.21.15.143:0/2464414242 conn(0x7f8684099ab0 msgr2=0x7f86840d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:59.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.275+0000 7f8688f56700 1 -- 172.21.15.143:0/2464414242 shutdown_connections 2023-11-08T19:27:59.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.275+0000 7f8688f56700 1 -- 172.21.15.143:0/2464414242 wait complete. 2023-11-08T19:27:59.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.276+0000 7f8688f56700 1 Processor -- start 2023-11-08T19:27:59.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.276+0000 7f8688f56700 1 -- start start 2023-11-08T19:27:59.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.277+0000 7f8688f56700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f86840b2920 0x7f86841ee980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:59.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.277+0000 7f8688f56700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86840cbe20 0x7f86840b3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:59.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.277+0000 7f8688f56700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f868415aa30 0x7f86841f2620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:59.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.277+0000 7f8688f56700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86840b64a0 con 0x7f86840cbe20 2023-11-08T19:27:59.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.277+0000 7f8688f56700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f868415cb40 con 0x7f86840b2920 2023-11-08T19:27:59.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.277+0000 7f8688f56700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86841eeec0 con 0x7f868415aa30 2023-11-08T19:27:59.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.277+0000 7f8681d9b700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86840cbe20 0x7f86840b3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:59.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.277+0000 7f8681d9b700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86840cbe20 0x7f86840b3710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54408/0 (socket says 172.21.15.143:54408) 2023-11-08T19:27:59.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.277+0000 7f8681d9b700 1 -- 172.21.15.143:0/78984553 learned_addr learned my addr 172.21.15.143:0/78984553 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:59.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.277+0000 7f868259c700 1 --2- 172.21.15.143:0/78984553 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f86840b2920 0x7f86841ee980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:59.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.277+0000 7f8681d9b700 1 -- 172.21.15.143:0/78984553 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f868415aa30 msgr2=0x7f86841f2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:59.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.277+0000 7f8681d9b700 1 --2- 172.21.15.143:0/78984553 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f868415aa30 0x7f86841f2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:59.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.277+0000 7f8681d9b700 1 -- 172.21.15.143:0/78984553 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f86840b2920 msgr2=0x7f86841ee980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:59.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.277+0000 7f8681d9b700 1 --2- 172.21.15.143:0/78984553 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f86840b2920 0x7f86841ee980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:59.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.277+0000 7f8681d9b700 1 -- 172.21.15.143:0/78984553 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f867c00bc80 con 0x7f86840cbe20 2023-11-08T19:27:59.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.277+0000 7f8681d9b700 1 --2- 172.21.15.143:0/78984553 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86840cbe20 0x7f86840b3710 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f867400e9b0 tx=0x7f867400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:59.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.277+0000 7f8672ffd700 1 -- 172.21.15.143:0/78984553 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8674009710 con 0x7f86840cbe20 2023-11-08T19:27:59.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.277+0000 7f8672ffd700 1 -- 172.21.15.143:0/78984553 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f867400c850 con 0x7f86840cbe20 2023-11-08T19:27:59.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.277+0000 7f8688f56700 1 -- 172.21.15.143:0/78984553 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86840b5ee0 con 0x7f86840cbe20 2023-11-08T19:27:59.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.277+0000 7f8672ffd700 1 -- 172.21.15.143:0/78984553 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f867400cbc0 con 0x7f86840cbe20 2023-11-08T19:27:59.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.277+0000 7f8688f56700 1 -- 172.21.15.143:0/78984553 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86840c43d0 con 0x7f86840cbe20 2023-11-08T19:27:59.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.278+0000 7f8688f56700 1 -- 172.21.15.143:0/78984553 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86840c4230 con 0x7f86840cbe20 2023-11-08T19:27:59.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.279+0000 7f8672ffd700 1 -- 172.21.15.143:0/78984553 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f867400cd20 con 0x7f86840cbe20 2023-11-08T19:27:59.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.279+0000 7f8672ffd700 1 --2- 172.21.15.143:0/78984553 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f866406af00 0x7f866406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:59.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.279+0000 7f8672ffd700 1 -- 172.21.15.143:0/78984553 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5920+0+0 (secure 0 0 0) 0x7f8674089710 con 0x7f86840cbe20 2023-11-08T19:27:59.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.280+0000 7f868259c700 1 --2- 172.21.15.143:0/78984553 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f866406af00 0x7f866406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:59.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.280+0000 7f868259c700 1 --2- 172.21.15.143:0/78984553 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f866406af00 0x7f866406d3c0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f8678001360 tx=0x7f8678009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:59.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.284+0000 7f8672ffd700 1 -- 172.21.15.143:0/78984553 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f86840c4230 con 0x7f86840cbe20 2023-11-08T19:27:59.473 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.472+0000 7f8688f56700 1 -- 172.21.15.143:0/78984553 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f868415ccf0 con 0x7f86840cbe20 2023-11-08T19:27:59.474 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.473+0000 7f8672ffd700 1 -- 172.21.15.143:0/78984553 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+18150 (secure 0 0 0) 0x7f868415ccf0 con 0x7f86840cbe20 2023-11-08T19:27:59.475 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:27:59.475 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":88,"fsid":"968483af-77fc-4068-ad7c-5c2a1a4613d7","created":"2023-11-08T19:24:32.826484+0000","modified":"2023-11-08T19:27:58.690720+0000","last_up_change":"2023-11-08T19:24:43.573880+0000","last_in_change":"2023-11-08T19:24:35.314525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:44.551530+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"snap-test-pool","create_time":"2023-11-08T19:26:02.874384+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":36,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"test_data_pool","create_time":"2023-11-08T19:27:05.601949+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":66,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_metadata_pool","create_time":"2023-11-08T19:27:06.536369+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bae98ec1-d94e-403d-a220-a522a53ea6f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6809","nonce":1487830566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6811","nonce":1487830566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6815","nonce":1487830566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6813","nonce":1487830566}]},"public_addr":"172.21.15.143:6809/1487830566","cluster_addr":"172.21.15.143:6811/1487830566","heartbeat_back_addr":"172.21.15.143:6815/1487830566","heartbeat_front_addr":"172.21.15.143:6813/1487830566","state":["exists","up"]},{"osd":1,"uuid":"5cb6538a-1ccb-4322-8633-09f66af0b7e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6801","nonce":787341397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6803","nonce":787341397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6807","nonce":787341397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6805","nonce":787341397}]},"public_addr":"172.21.15.143:6801/787341397","cluster_addr":"172.21.15.143:6803/787341397","heartbeat_back_addr":"172.21.15.143:6807/787341397","heartbeat_front_addr":"172.21.15.143:6805/787341397","state":["exists","up"]},{"osd":2,"uuid":"cdbf522c-60ef-4d60-9653-8dcb529c101a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6822","nonce":2301325597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6826","nonce":2301325597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6831","nonce":2301325597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6829","nonce":2301325597}]},"public_addr":"172.21.15.143:6822/2301325597","cluster_addr":"172.21.15.143:6826/2301325597","heartbeat_back_addr":"172.21.15.143:6831/2301325597","heartbeat_front_addr":"172.21.15.143:6829/2301325597","state":["exists","up"]},{"osd":3,"uuid":"4e2783cf-ebb9-4dd8-8245-bc1d17ce4325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6817","nonce":2021504428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6819","nonce":2021504428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6827","nonce":2021504428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6823","nonce":2021504428}]},"public_addr":"172.21.15.143:6817/2021504428","cluster_addr":"172.21.15.143:6819/2021504428","heartbeat_back_addr":"172.21.15.143:6827/2021504428","heartbeat_front_addr":"172.21.15.143:6823/2021504428","state":["exists","up"]},{"osd":4,"uuid":"691256fa-9b77-49b9-a346-7ae1adf0ca80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6801","nonce":1993380595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6803","nonce":1993380595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6807","nonce":1993380595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6805","nonce":1993380595}]},"public_addr":"172.21.15.174:6801/1993380595","cluster_addr":"172.21.15.174:6803/1993380595","heartbeat_back_addr":"172.21.15.174:6807/1993380595","heartbeat_front_addr":"172.21.15.174:6805/1993380595","state":["exists","up"]},{"osd":5,"uuid":"756fc352-48c9-46ce-b52e-0541fe6f8525","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6809","nonce":1777923303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6811","nonce":1777923303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6815","nonce":1777923303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6813","nonce":1777923303}]},"public_addr":"172.21.15.174:6809/1777923303","cluster_addr":"172.21.15.174:6811/1777923303","heartbeat_back_addr":"172.21.15.174:6815/1777923303","heartbeat_front_addr":"172.21.15.174:6813/1777923303","state":["exists","up"]},{"osd":6,"uuid":"fab2ac2b-ed97-4cde-9825-b480eabf8b8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6825","nonce":2755130316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6827","nonce":2755130316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6831","nonce":2755130316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6829","nonce":2755130316}]},"public_addr":"172.21.15.174:6825/2755130316","cluster_addr":"172.21.15.174:6827/2755130316","heartbeat_back_addr":"172.21.15.174:6831/2755130316","heartbeat_front_addr":"172.21.15.174:6829/2755130316","state":["exists","up"]},{"osd":7,"uuid":"3de9b797-2ab6-4d41-a863-7e7d638fd703","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6817","nonce":2847141411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6819","nonce":2847141411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6823","nonce":2847141411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6821","nonce":2847141411}]},"public_addr":"172.21.15.174:6817/2847141411","cluster_addr":"172.21.15.174:6819/2847141411","heartbeat_back_addr":"172.21.15.174:6823/2847141411","heartbeat_front_addr":"172.21.15.174:6821/2847141411","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.477076+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.496890+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.539305+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.454673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.538881+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.521294+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.520712+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.540404+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:27:59.477 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.475+0000 7f8688f56700 1 -- 172.21.15.143:0/78984553 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f866406af00 msgr2=0x7f866406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:59.477 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.475+0000 7f8688f56700 1 --2- 172.21.15.143:0/78984553 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f866406af00 0x7f866406d3c0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f8678001360 tx=0x7f8678009040 comp rx=0 tx=0).stop 2023-11-08T19:27:59.477 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.475+0000 7f8688f56700 1 -- 172.21.15.143:0/78984553 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86840cbe20 msgr2=0x7f86840b3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:59.478 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.475+0000 7f8688f56700 1 --2- 172.21.15.143:0/78984553 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86840cbe20 0x7f86840b3710 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f867400e9b0 tx=0x7f867400ed70 comp rx=0 tx=0).stop 2023-11-08T19:27:59.478 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.475+0000 7f8688f56700 1 -- 172.21.15.143:0/78984553 shutdown_connections 2023-11-08T19:27:59.478 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.475+0000 7f8688f56700 1 --2- 172.21.15.143:0/78984553 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f866406af00 0x7f866406d3c0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:59.478 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.475+0000 7f8688f56700 1 --2- 172.21.15.143:0/78984553 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f868415aa30 0x7f86841f2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:59.478 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.475+0000 7f8688f56700 1 --2- 172.21.15.143:0/78984553 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f86840b2920 0x7f86841ee980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:59.479 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.475+0000 7f8688f56700 1 --2- 172.21.15.143:0/78984553 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86840cbe20 0x7f86840b3710 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:59.479 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.476+0000 7f8688f56700 1 -- 172.21.15.143:0/78984553 >> 172.21.15.143:0/78984553 conn(0x7f8684099ab0 msgr2=0x7f86840af390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:59.479 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.476+0000 7f8688f56700 1 -- 172.21.15.143:0/78984553 shutdown_connections 2023-11-08T19:27:59.479 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.476+0000 7f8688f56700 1 -- 172.21.15.143:0/78984553 wait complete. 2023-11-08T19:27:59.489 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T19:24:44.551530+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'snap-test-pool', 'create_time': '2023-11-08T19:26:02.874384+0000', 'flags': 16385, 'flags_names': 'hashpspool,pool_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '36', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'pool', 'snap_seq': 2, 'snap_epoch': 36, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'test_data_pool', 'create_time': '2023-11-08T19:27:05.601949+0000', 'flags': 16385, 'flags_names': 'hashpspool,pool_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'pool', 'snap_seq': 2, 'snap_epoch': 66, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'test_metadata_pool', 'create_time': '2023-11-08T19:27:06.536369+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-08T19:27:59.490 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T19:27:59.614 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.612+0000 7f92ec71b700 1 Processor -- start 2023-11-08T19:27:59.614 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.612+0000 7f92ec71b700 1 -- start start 2023-11-08T19:27:59.621 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.613+0000 7f92ec71b700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f92e406b240 0x7f92e40caff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:59.621 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.613+0000 7f92ec71b700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92e4161890 con 0x7f92e40cb530 2023-11-08T19:27:59.621 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.613+0000 7f92ec71b700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92e406b740 con 0x7f92e415dba0 2023-11-08T19:27:59.622 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.613+0000 7f92ec71b700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92e406cd20 con 0x7f92e406b240 2023-11-08T19:27:59.622 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.613+0000 7f92eacb8700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f92e40cb530 0x7f92e40b22e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:50572/0 (socket says 172.21.15.143:50572) 2023-11-08T19:27:59.622 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.613+0000 7f92eacb8700 1 -- 172.21.15.143:0/116364688 learned_addr learned my addr 172.21.15.143:0/116364688 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:27:59.622 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.613+0000 7f92e9cb6700 1 --2- 172.21.15.143:0/116364688 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f92e406b240 0x7f92e40caff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:59.622 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.613+0000 7f92e94b5700 1 -- 172.21.15.143:0/116364688 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 734059536 0 0) 0x7f92e4161890 con 0x7f92e40cb530 2023-11-08T19:27:59.623 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.613+0000 7f92e94b5700 1 -- 172.21.15.143:0/116364688 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f92d00035b0 con 0x7f92e40cb530 2023-11-08T19:27:59.623 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.614+0000 7f92e94b5700 1 -- 172.21.15.143:0/116364688 <== mon.1 v1:172.21.15.174:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4078056469 0 0) 0x7f92e406b740 con 0x7f92e415dba0 2023-11-08T19:27:59.623 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.614+0000 7f92e94b5700 1 -- 172.21.15.143:0/116364688 --> v1:172.21.15.174:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f92e4161890 con 0x7f92e415dba0 2023-11-08T19:27:59.623 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.614+0000 7f92e9cb6700 1 -- 172.21.15.143:0/116364688 >> v1:172.21.15.174:6789/0 conn(0x7f92e415dba0 legacy=0x7f92e406c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:59.624 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.614+0000 7f92e9cb6700 1 -- 172.21.15.143:0/116364688 >> v1:172.21.15.143:6789/0 conn(0x7f92e40cb530 legacy=0x7f92e40b22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:59.624 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.614+0000 7f92e94b5700 1 -- 172.21.15.143:0/116364688 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 652090612 0 0) 0x7f92d00035b0 con 0x7f92e40cb530 2023-11-08T19:27:59.624 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.614+0000 7f92e9cb6700 1 -- 172.21.15.143:0/116364688 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92e406ba20 con 0x7f92e406b240 2023-11-08T19:27:59.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.614+0000 7f92e9cb6700 1 --2- 172.21.15.143:0/116364688 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f92e406b240 0x7f92e40caff0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f92e0009870 tx=0x7f92e0009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd742644532b8089 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:59.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.614+0000 7f92e94b5700 1 -- 172.21.15.143:0/116364688 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92dc0019f0 con 0x7f92e406b240 2023-11-08T19:27:59.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.614+0000 7f92e94b5700 1 -- 172.21.15.143:0/116364688 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92e0004030 con 0x7f92e406b240 2023-11-08T19:27:59.626 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.614+0000 7f92e94b5700 1 -- 172.21.15.143:0/116364688 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92e00043c0 con 0x7f92e406b240 2023-11-08T19:27:59.626 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.615+0000 7f92ec71b700 1 -- 172.21.15.143:0/116364688 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f92e406b240 msgr2=0x7f92e40caff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:59.626 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.615+0000 7f92ec71b700 1 --2- 172.21.15.143:0/116364688 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f92e406b240 0x7f92e40caff0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f92e0009870 tx=0x7f92e0009b80 comp rx=0 tx=0).stop 2023-11-08T19:27:59.626 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.615+0000 7f92ec71b700 1 -- 172.21.15.143:0/116364688 shutdown_connections 2023-11-08T19:27:59.627 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.615+0000 7f92ec71b700 1 --2- 172.21.15.143:0/116364688 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f92e406b240 0x7f92e40caff0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:59.627 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.615+0000 7f92ec71b700 1 -- 172.21.15.143:0/116364688 >> 172.21.15.143:0/116364688 conn(0x7f92e40cd2e0 msgr2=0x7f92e40cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:59.627 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.615+0000 7f92ec71b700 1 -- 172.21.15.143:0/116364688 shutdown_connections 2023-11-08T19:27:59.628 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.615+0000 7f92ec71b700 1 -- 172.21.15.143:0/116364688 wait complete. 2023-11-08T19:27:59.628 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.616+0000 7f92ec71b700 1 Processor -- start 2023-11-08T19:27:59.628 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.616+0000 7f92ec71b700 1 -- start start 2023-11-08T19:27:59.629 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.617+0000 7f92ec71b700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f92e406b240 0x7f92e41f7540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:59.629 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.617+0000 7f92ec71b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f92e40cb530 0x7f92e415bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:59.629 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.617+0000 7f92ec71b700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f92e415dba0 0x7f92e4160440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:59.629 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.617+0000 7f92ec71b700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92e41f7a80 con 0x7f92e406b240 2023-11-08T19:27:59.630 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.617+0000 7f92ec71b700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92e41f3650 con 0x7f92e40cb530 2023-11-08T19:27:59.630 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.617+0000 7f92ec71b700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92e40c9730 con 0x7f92e415dba0 2023-11-08T19:27:59.630 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.617+0000 7f92ea4b7700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f92e406b240 0x7f92e41f7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:59.631 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.617+0000 7f92ea4b7700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f92e406b240 0x7f92e41f7540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54420/0 (socket says 172.21.15.143:54420) 2023-11-08T19:27:59.631 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.617+0000 7f92ea4b7700 1 -- 172.21.15.143:0/1600939037 learned_addr learned my addr 172.21.15.143:0/1600939037 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:59.631 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.617+0000 7f92e9cb6700 1 --2- 172.21.15.143:0/1600939037 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f92e40cb530 0x7f92e415bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:59.631 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.617+0000 7f92eacb8700 1 --2- 172.21.15.143:0/1600939037 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f92e415dba0 0x7f92e4160440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:59.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.617+0000 7f92ea4b7700 1 -- 172.21.15.143:0/1600939037 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f92e415dba0 msgr2=0x7f92e4160440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:59.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.617+0000 7f92ea4b7700 1 --2- 172.21.15.143:0/1600939037 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f92e415dba0 0x7f92e4160440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:59.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.617+0000 7f92ea4b7700 1 -- 172.21.15.143:0/1600939037 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f92e40cb530 msgr2=0x7f92e415bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:59.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.617+0000 7f92ea4b7700 1 --2- 172.21.15.143:0/1600939037 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f92e40cb530 0x7f92e415bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:59.633 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.617+0000 7f92ea4b7700 1 -- 172.21.15.143:0/1600939037 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92e00094a0 con 0x7f92e406b240 2023-11-08T19:27:59.633 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.617+0000 7f92ea4b7700 1 --2- 172.21.15.143:0/1600939037 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f92e406b240 0x7f92e41f7540 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f92d400c520 tx=0x7f92d400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:59.633 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.617+0000 7f92daffd700 1 -- 172.21.15.143:0/1600939037 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92d40052d0 con 0x7f92e406b240 2023-11-08T19:27:59.634 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.617+0000 7f92daffd700 1 -- 172.21.15.143:0/1600939037 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92d4005430 con 0x7f92e406b240 2023-11-08T19:27:59.634 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.617+0000 7f92ec71b700 1 -- 172.21.15.143:0/1600939037 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92e41f38f0 con 0x7f92e406b240 2023-11-08T19:27:59.634 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.617+0000 7f92daffd700 1 -- 172.21.15.143:0/1600939037 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92d40038c0 con 0x7f92e406b240 2023-11-08T19:27:59.634 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.618+0000 7f92ec71b700 1 -- 172.21.15.143:0/1600939037 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92e40c9a10 con 0x7f92e406b240 2023-11-08T19:27:59.635 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.618+0000 7f92ec71b700 1 -- 172.21.15.143:0/1600939037 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92e404eee0 con 0x7f92e406b240 2023-11-08T19:27:59.635 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.619+0000 7f92daffd700 1 -- 172.21.15.143:0/1600939037 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f92d4003a20 con 0x7f92e406b240 2023-11-08T19:27:59.635 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.619+0000 7f92daffd700 1 --2- 172.21.15.143:0/1600939037 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f92c806af00 0x7f92c806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:59.636 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.619+0000 7f92daffd700 1 -- 172.21.15.143:0/1600939037 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5920+0+0 (secure 0 0 0) 0x7f92d4088a90 con 0x7f92e406b240 2023-11-08T19:27:59.636 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.619+0000 7f92e9cb6700 1 --2- 172.21.15.143:0/1600939037 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f92c806af00 0x7f92c806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:59.636 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.620+0000 7f92e9cb6700 1 --2- 172.21.15.143:0/1600939037 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f92c806af00 0x7f92c806d3c0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f92e00090b0 tx=0x7f92e0008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:59.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.623+0000 7f92daffd700 1 -- 172.21.15.143:0/1600939037 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f92d4053b00 con 0x7f92e406b240 2023-11-08T19:27:59.814 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.813+0000 7f92ec71b700 1 -- 172.21.15.143:0/1600939037 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f92e406bcd0 con 0x7f92e406b240 2023-11-08T19:27:59.815 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.814+0000 7f92daffd700 1 -- 172.21.15.143:0/1600939037 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7f92d40170a0 con 0x7f92e406b240 2023-11-08T19:27:59.816 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2023-11-08T19:27:59.818 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.816+0000 7f92ec71b700 1 -- 172.21.15.143:0/1600939037 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f92c806af00 msgr2=0x7f92c806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:59.818 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.816+0000 7f92ec71b700 1 --2- 172.21.15.143:0/1600939037 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f92c806af00 0x7f92c806d3c0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f92e00090b0 tx=0x7f92e0008030 comp rx=0 tx=0).stop 2023-11-08T19:27:59.818 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.816+0000 7f92ec71b700 1 -- 172.21.15.143:0/1600939037 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f92e406b240 msgr2=0x7f92e41f7540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:59.819 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.816+0000 7f92ec71b700 1 --2- 172.21.15.143:0/1600939037 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f92e406b240 0x7f92e41f7540 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f92d400c520 tx=0x7f92d400c830 comp rx=0 tx=0).stop 2023-11-08T19:27:59.819 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.816+0000 7f92ec71b700 1 -- 172.21.15.143:0/1600939037 shutdown_connections 2023-11-08T19:27:59.819 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.816+0000 7f92ec71b700 1 --2- 172.21.15.143:0/1600939037 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f92c806af00 0x7f92c806d3c0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:59.819 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.816+0000 7f92ec71b700 1 --2- 172.21.15.143:0/1600939037 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f92e415dba0 0x7f92e4160440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:59.820 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.816+0000 7f92ec71b700 1 --2- 172.21.15.143:0/1600939037 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f92e406b240 0x7f92e41f7540 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:59.820 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.816+0000 7f92ec71b700 1 --2- 172.21.15.143:0/1600939037 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f92e40cb530 0x7f92e415bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:59.820 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.816+0000 7f92ec71b700 1 -- 172.21.15.143:0/1600939037 >> 172.21.15.143:0/1600939037 conn(0x7f92e40cd2e0 msgr2=0x7f92e40b0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:59.820 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.816+0000 7f92ec71b700 1 -- 172.21.15.143:0/1600939037 shutdown_connections 2023-11-08T19:27:59.821 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.816+0000 7f92ec71b700 1 -- 172.21.15.143:0/1600939037 wait complete. 2023-11-08T19:27:59.829 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get snap-test-pool pg_num 2023-11-08T19:27:59.953 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.951+0000 7f62b9005700 1 Processor -- start 2023-11-08T19:27:59.953 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.951+0000 7f62b9005700 1 -- start start 2023-11-08T19:27:59.955 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.952+0000 7f62b9005700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f62b415dc20 0x7f62b40c4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:59.956 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.952+0000 7f62b9005700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f62b40cc9e0 0x7f62b40cce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:59.956 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.952+0000 7f62b9005700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62b406ad90 con 0x7f62b415e000 2023-11-08T19:27:59.956 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.952+0000 7f62b9005700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62b406bb50 con 0x7f62b415dc20 2023-11-08T19:27:59.956 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.952+0000 7f62b9005700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62b40cd390 con 0x7f62b40cc9e0 2023-11-08T19:27:59.956 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.952+0000 7f62b2d9d700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f62b415dc20 0x7f62b40c4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:59.957 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.952+0000 7f62b2d9d700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f62b415dc20 0x7f62b40c4ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54430/0 (socket says 172.21.15.143:54430) 2023-11-08T19:27:59.957 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.952+0000 7f62b2d9d700 1 -- 172.21.15.143:0/2099613298 learned_addr learned my addr 172.21.15.143:0/2099613298 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:59.957 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.953+0000 7f62b359e700 1 --2- 172.21.15.143:0/2099613298 >> v2:172.21.15.174:3300/0 conn(0x7f62b40cc9e0 0x7f62b40cce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:59.957 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.953+0000 7f62b2d9d700 1 -- 172.21.15.143:0/2099613298 >> v2:172.21.15.174:3300/0 conn(0x7f62b40cc9e0 msgr2=0x7f62b40cce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:59.957 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.953+0000 7f62b2d9d700 1 --2- 172.21.15.143:0/2099613298 >> v2:172.21.15.174:3300/0 conn(0x7f62b40cc9e0 0x7f62b40cce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:59.958 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.953+0000 7f62b2d9d700 1 -- 172.21.15.143:0/2099613298 >> v1:172.21.15.174:6789/0 conn(0x7f62b415e000 legacy=0x7f62b40c5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:59.958 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.953+0000 7f62b2d9d700 1 -- 172.21.15.143:0/2099613298 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62b406b070 con 0x7f62b415dc20 2023-11-08T19:27:59.958 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.953+0000 7f62b359e700 1 --2- 172.21.15.143:0/2099613298 >> v2:172.21.15.174:3300/0 conn(0x7f62b40cc9e0 0x7f62b40cce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:27:59.958 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.953+0000 7f62b2d9d700 1 --2- 172.21.15.143:0/2099613298 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f62b415dc20 0x7f62b40c4ee0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f62b40cdd30 tx=0x7f62a8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b492c8b84f27353f server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:59.958 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.953+0000 7f62b1d9b700 1 -- 172.21.15.143:0/2099613298 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62a8004230 con 0x7f62b415dc20 2023-11-08T19:27:59.959 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.953+0000 7f62b1d9b700 1 -- 172.21.15.143:0/2099613298 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62a8004390 con 0x7f62b415dc20 2023-11-08T19:27:59.959 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.953+0000 7f62b1d9b700 1 -- 172.21.15.143:0/2099613298 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62a80036a0 con 0x7f62b415dc20 2023-11-08T19:27:59.959 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.954+0000 7f62b9005700 1 -- 172.21.15.143:0/2099613298 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f62b415dc20 msgr2=0x7f62b40c4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:59.959 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.954+0000 7f62b9005700 1 --2- 172.21.15.143:0/2099613298 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f62b415dc20 0x7f62b40c4ee0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f62b40cdd30 tx=0x7f62a8009bd0 comp rx=0 tx=0).stop 2023-11-08T19:27:59.959 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.954+0000 7f62b9005700 1 -- 172.21.15.143:0/2099613298 shutdown_connections 2023-11-08T19:27:59.960 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.954+0000 7f62b9005700 1 --2- 172.21.15.143:0/2099613298 >> v2:172.21.15.174:3300/0 conn(0x7f62b40cc9e0 0x7f62b40cce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:59.960 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.954+0000 7f62b9005700 1 --2- 172.21.15.143:0/2099613298 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f62b415dc20 0x7f62b40c4ee0 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:59.960 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.954+0000 7f62b9005700 1 -- 172.21.15.143:0/2099613298 >> 172.21.15.143:0/2099613298 conn(0x7f62b40b1c70 msgr2=0x7f62b40b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:59.960 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.954+0000 7f62b9005700 1 -- 172.21.15.143:0/2099613298 shutdown_connections 2023-11-08T19:27:59.961 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.954+0000 7f62b9005700 1 -- 172.21.15.143:0/2099613298 wait complete. 2023-11-08T19:27:59.964 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.955+0000 7f62b9005700 1 Processor -- start 2023-11-08T19:27:59.964 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.955+0000 7f62b9005700 1 -- start start 2023-11-08T19:27:59.964 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.956+0000 7f62b9005700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f62b40cc9e0 0x7f62b41f47d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:59.964 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.956+0000 7f62b9005700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f62b415dc20 0x7f62b4140dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:59.964 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.956+0000 7f62b9005700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f62b415e000 0x7f62b4140620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:59.965 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.956+0000 7f62b9005700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62b4140b60 con 0x7f62b415dc20 2023-11-08T19:27:59.965 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.956+0000 7f62b9005700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62b41f7f80 con 0x7f62b40cc9e0 2023-11-08T19:27:59.965 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.956+0000 7f62b9005700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62b41f7370 con 0x7f62b415e000 2023-11-08T19:27:59.965 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.956+0000 7f62b259c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f62b415dc20 0x7f62b4140dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:59.965 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.956+0000 7f62b259c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f62b415dc20 0x7f62b4140dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54438/0 (socket says 172.21.15.143:54438) 2023-11-08T19:27:59.966 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.956+0000 7f62b259c700 1 -- 172.21.15.143:0/19446906 learned_addr learned my addr 172.21.15.143:0/19446906 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:27:59.966 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.956+0000 7f62b2d9d700 1 --2- 172.21.15.143:0/19446906 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f62b40cc9e0 0x7f62b41f47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:59.966 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.956+0000 7f62b359e700 1 --2- 172.21.15.143:0/19446906 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f62b415e000 0x7f62b4140620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:59.966 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.956+0000 7f62b259c700 1 -- 172.21.15.143:0/19446906 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f62b415e000 msgr2=0x7f62b4140620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:59.966 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.956+0000 7f62b259c700 1 --2- 172.21.15.143:0/19446906 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f62b415e000 0x7f62b4140620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:59.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.956+0000 7f62b259c700 1 -- 172.21.15.143:0/19446906 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f62b40cc9e0 msgr2=0x7f62b41f47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:59.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.956+0000 7f62b259c700 1 --2- 172.21.15.143:0/19446906 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f62b40cc9e0 0x7f62b41f47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:59.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.956+0000 7f62b259c700 1 -- 172.21.15.143:0/19446906 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62a8009550 con 0x7f62b415dc20 2023-11-08T19:27:59.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.956+0000 7f62b359e700 1 --2- 172.21.15.143:0/19446906 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f62b415e000 0x7f62b4140620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:27:59.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.957+0000 7f62b259c700 1 --2- 172.21.15.143:0/19446906 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f62b415dc20 0x7f62b4140dc0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f62a400c4d0 tx=0x7f62a400c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:59.968 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.957+0000 7f629b7fe700 1 -- 172.21.15.143:0/19446906 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62a4011790 con 0x7f62b415dc20 2023-11-08T19:27:59.968 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.957+0000 7f629b7fe700 1 -- 172.21.15.143:0/19446906 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62a40118f0 con 0x7f62b415dc20 2023-11-08T19:27:59.968 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.957+0000 7f629b7fe700 1 -- 172.21.15.143:0/19446906 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62a4011ca0 con 0x7f62b415dc20 2023-11-08T19:27:59.968 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.957+0000 7f62b9005700 1 -- 172.21.15.143:0/19446906 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62b41f7650 con 0x7f62b415dc20 2023-11-08T19:27:59.968 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.957+0000 7f62b9005700 1 -- 172.21.15.143:0/19446906 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62b41f7c10 con 0x7f62b415dc20 2023-11-08T19:27:59.969 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.958+0000 7f62b9005700 1 -- 172.21.15.143:0/19446906 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62b404eee0 con 0x7f62b415dc20 2023-11-08T19:27:59.969 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.958+0000 7f629b7fe700 1 -- 172.21.15.143:0/19446906 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f62a401c460 con 0x7f62b415dc20 2023-11-08T19:27:59.969 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.958+0000 7f629b7fe700 1 --2- 172.21.15.143:0/19446906 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f62a006af00 0x7f62a006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:59.969 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.959+0000 7f629b7fe700 1 -- 172.21.15.143:0/19446906 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5920+0+0 (secure 0 0 0) 0x7f62a40893d0 con 0x7f62b415dc20 2023-11-08T19:27:59.969 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.959+0000 7f62b2d9d700 1 --2- 172.21.15.143:0/19446906 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f62a006af00 0x7f62a006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:59.970 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.959+0000 7f62b2d9d700 1 --2- 172.21.15.143:0/19446906 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f62a006af00 0x7f62a006d3c0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f62a8009130 tx=0x7f62a8008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:59.970 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:27:59.963+0000 7f629b7fe700 1 -- 172.21.15.143:0/19446906 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f62a40543b0 con 0x7f62b415dc20 2023-11-08T19:28:00.154 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.151+0000 7f62b9005700 1 -- 172.21.15.143:0/19446906 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"} v 0) v1 -- 0x7f62b41f80c0 con 0x7f62b415dc20 2023-11-08T19:28:00.155 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.151+0000 7f629b7fe700 1 -- 172.21.15.143:0/19446906 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"}]=0 v88) v1 ==== 103+0+11 (secure 0 0 0) 0x7f62b41f80c0 con 0x7f62b415dc20 2023-11-08T19:28:00.156 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2023-11-08T19:28:00.158 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.154+0000 7f62b9005700 1 -- 172.21.15.143:0/19446906 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f62a006af00 msgr2=0x7f62a006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:00.158 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.154+0000 7f62b9005700 1 --2- 172.21.15.143:0/19446906 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f62a006af00 0x7f62a006d3c0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f62a8009130 tx=0x7f62a8008030 comp rx=0 tx=0).stop 2023-11-08T19:28:00.158 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.154+0000 7f62b9005700 1 -- 172.21.15.143:0/19446906 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f62b415dc20 msgr2=0x7f62b4140dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:00.159 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.154+0000 7f62b9005700 1 --2- 172.21.15.143:0/19446906 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f62b415dc20 0x7f62b4140dc0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f62a400c4d0 tx=0x7f62a400c7e0 comp rx=0 tx=0).stop 2023-11-08T19:28:00.159 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.154+0000 7f62b9005700 1 -- 172.21.15.143:0/19446906 shutdown_connections 2023-11-08T19:28:00.159 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.154+0000 7f62b9005700 1 --2- 172.21.15.143:0/19446906 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f62a006af00 0x7f62a006d3c0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:00.159 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.154+0000 7f62b9005700 1 --2- 172.21.15.143:0/19446906 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f62b415e000 0x7f62b4140620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:00.159 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.154+0000 7f62b9005700 1 --2- 172.21.15.143:0/19446906 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f62b40cc9e0 0x7f62b41f47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:00.160 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.154+0000 7f62b9005700 1 --2- 172.21.15.143:0/19446906 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f62b415dc20 0x7f62b4140dc0 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:00.160 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.154+0000 7f62b9005700 1 -- 172.21.15.143:0/19446906 >> 172.21.15.143:0/19446906 conn(0x7f62b40b1c70 msgr2=0x7f62b40ca4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:00.160 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.154+0000 7f62b9005700 1 -- 172.21.15.143:0/19446906 shutdown_connections 2023-11-08T19:28:00.160 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.154+0000 7f62b9005700 1 -- 172.21.15.143:0/19446906 wait complete. 2023-11-08T19:28:00.168 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get test_data_pool pg_num 2023-11-08T19:28:00.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.296+0000 7fe9736ca700 1 Processor -- start 2023-11-08T19:28:00.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.297+0000 7fe9736ca700 1 -- start start 2023-11-08T19:28:00.301 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.297+0000 7fe9736ca700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe96c15bb20 0x7fe96c0b2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:00.301 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.297+0000 7fe9736ca700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe96c0d1d80 0x7fe96c0cc4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:00.302 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.297+0000 7fe9736ca700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe96c0d0420 con 0x7fe96c0b1730 2023-11-08T19:28:00.302 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.297+0000 7fe9736ca700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe96c0cf600 con 0x7fe96c0d1d80 2023-11-08T19:28:00.302 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.297+0000 7fe9736ca700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe96c0cf810 con 0x7fe96c15bb20 2023-11-08T19:28:00.302 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.297+0000 7fe971c67700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe96c0d1d80 0x7fe96c0cc4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:00.302 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.297+0000 7fe970c65700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fe96c0b1730 0x7fe96c0d1670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:50598/0 (socket says 172.21.15.143:50598) 2023-11-08T19:28:00.303 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.297+0000 7fe970c65700 1 -- 172.21.15.143:0/4288920352 learned_addr learned my addr 172.21.15.143:0/4288920352 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:28:00.303 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.298+0000 7fe971466700 1 --2- 172.21.15.143:0/4288920352 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe96c15bb20 0x7fe96c0b2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:00.303 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.298+0000 7fe971c67700 1 -- 172.21.15.143:0/4288920352 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe96c15bb20 msgr2=0x7fe96c0b2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:00.303 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.298+0000 7fe971c67700 1 --2- 172.21.15.143:0/4288920352 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe96c15bb20 0x7fe96c0b2670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:00.303 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.298+0000 7fe971c67700 1 -- 172.21.15.143:0/4288920352 >> v1:172.21.15.143:6789/0 conn(0x7fe96c0b1730 legacy=0x7fe96c0d1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:00.304 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.298+0000 7fe971c67700 1 -- 172.21.15.143:0/4288920352 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe96c0cfaf0 con 0x7fe96c0d1d80 2023-11-08T19:28:00.304 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.298+0000 7fe971c67700 1 --2- 172.21.15.143:0/4288920352 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe96c0d1d80 0x7fe96c0cc4d0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fe964009d90 tx=0x7fe96400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7da2392c5454b45 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:00.304 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.298+0000 7fe963fff700 1 -- 172.21.15.143:0/4288920352 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe96400e070 con 0x7fe96c0d1d80 2023-11-08T19:28:00.304 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.298+0000 7fe963fff700 1 -- 172.21.15.143:0/4288920352 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe96400bbd0 con 0x7fe96c0d1d80 2023-11-08T19:28:00.305 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.298+0000 7fe963fff700 1 -- 172.21.15.143:0/4288920352 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe964012430 con 0x7fe96c0d1d80 2023-11-08T19:28:00.305 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.299+0000 7fe9736ca700 1 -- 172.21.15.143:0/4288920352 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe96c0d1d80 msgr2=0x7fe96c0cc4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:00.305 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.299+0000 7fe9736ca700 1 --2- 172.21.15.143:0/4288920352 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe96c0d1d80 0x7fe96c0cc4d0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fe964009d90 tx=0x7fe96400b330 comp rx=0 tx=0).stop 2023-11-08T19:28:00.305 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.299+0000 7fe9736ca700 1 -- 172.21.15.143:0/4288920352 shutdown_connections 2023-11-08T19:28:00.306 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.299+0000 7fe9736ca700 1 --2- 172.21.15.143:0/4288920352 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe96c0d1d80 0x7fe96c0cc4d0 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:00.306 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.299+0000 7fe9736ca700 1 --2- 172.21.15.143:0/4288920352 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe96c15bb20 0x7fe96c0b2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:00.306 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.299+0000 7fe9736ca700 1 -- 172.21.15.143:0/4288920352 >> 172.21.15.143:0/4288920352 conn(0x7fe96c08b000 msgr2=0x7fe96c09ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:00.306 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.299+0000 7fe9736ca700 1 -- 172.21.15.143:0/4288920352 shutdown_connections 2023-11-08T19:28:00.307 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.299+0000 7fe9736ca700 1 -- 172.21.15.143:0/4288920352 wait complete. 2023-11-08T19:28:00.307 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.300+0000 7fe9736ca700 1 Processor -- start 2023-11-08T19:28:00.307 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.300+0000 7fe9736ca700 1 -- start start 2023-11-08T19:28:00.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.301+0000 7fe9736ca700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe96c0b1730 0x7fe96c0cc4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:00.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.301+0000 7fe9736ca700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe96c0d1d80 0x7fe96c1535a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:00.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.301+0000 7fe9736ca700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe96c15bb20 0x7fe96c144950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:00.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.301+0000 7fe9736ca700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe96c06d5e0 con 0x7fe96c0b1730 2023-11-08T19:28:00.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.301+0000 7fe9736ca700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe96c06c730 con 0x7fe96c15bb20 2023-11-08T19:28:00.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.301+0000 7fe9736ca700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe96c0cca10 con 0x7fe96c0d1d80 2023-11-08T19:28:00.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.301+0000 7fe971466700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe96c0b1730 0x7fe96c0cc4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:00.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.301+0000 7fe971466700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe96c0b1730 0x7fe96c0cc4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54448/0 (socket says 172.21.15.143:54448) 2023-11-08T19:28:00.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.301+0000 7fe971466700 1 -- 172.21.15.143:0/1474242631 learned_addr learned my addr 172.21.15.143:0/1474242631 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:28:00.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.301+0000 7fe970c65700 1 --2- 172.21.15.143:0/1474242631 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe96c0d1d80 0x7fe96c1535a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:00.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.301+0000 7fe971c67700 1 --2- 172.21.15.143:0/1474242631 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe96c15bb20 0x7fe96c144950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:00.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.301+0000 7fe971466700 1 -- 172.21.15.143:0/1474242631 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe96c0d1d80 msgr2=0x7fe96c1535a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:00.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.301+0000 7fe971466700 1 --2- 172.21.15.143:0/1474242631 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe96c0d1d80 0x7fe96c1535a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:00.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.301+0000 7fe971466700 1 -- 172.21.15.143:0/1474242631 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe96c15bb20 msgr2=0x7fe96c144950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:00.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.301+0000 7fe971466700 1 --2- 172.21.15.143:0/1474242631 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe96c15bb20 0x7fe96c144950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:00.315 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.301+0000 7fe971466700 1 -- 172.21.15.143:0/1474242631 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe964009a70 con 0x7fe96c0b1730 2023-11-08T19:28:00.315 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.301+0000 7fe971c67700 1 --2- 172.21.15.143:0/1474242631 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe96c15bb20 0x7fe96c144950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:28:00.315 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.301+0000 7fe971466700 1 --2- 172.21.15.143:0/1474242631 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe96c0b1730 0x7fe96c0cc4d0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fe95c00c7f0 tx=0x7fe95c00cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:00.315 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.301+0000 7fe961ffb700 1 -- 172.21.15.143:0/1474242631 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe95c011800 con 0x7fe96c0b1730 2023-11-08T19:28:00.315 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.301+0000 7fe961ffb700 1 -- 172.21.15.143:0/1474242631 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe95c011960 con 0x7fe96c0b1730 2023-11-08T19:28:00.316 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.301+0000 7fe9736ca700 1 -- 172.21.15.143:0/1474242631 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe96c06c9d0 con 0x7fe96c0b1730 2023-11-08T19:28:00.316 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.301+0000 7fe961ffb700 1 -- 172.21.15.143:0/1474242631 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe95c011cf0 con 0x7fe96c0b1730 2023-11-08T19:28:00.316 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.302+0000 7fe9736ca700 1 -- 172.21.15.143:0/1474242631 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe96c06cf90 con 0x7fe96c0b1730 2023-11-08T19:28:00.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.302+0000 7fe9736ca700 1 -- 172.21.15.143:0/1474242631 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe96c04eee0 con 0x7fe96c0b1730 2023-11-08T19:28:00.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.303+0000 7fe961ffb700 1 -- 172.21.15.143:0/1474242631 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe95c018360 con 0x7fe96c0b1730 2023-11-08T19:28:00.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.303+0000 7fe961ffb700 1 --2- 172.21.15.143:0/1474242631 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fe95006af00 0x7fe95006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:00.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.303+0000 7fe961ffb700 1 -- 172.21.15.143:0/1474242631 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5920+0+0 (secure 0 0 0) 0x7fe95c089620 con 0x7fe96c0b1730 2023-11-08T19:28:00.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.303+0000 7fe970c65700 1 --2- 172.21.15.143:0/1474242631 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fe95006af00 0x7fe95006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:00.318 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.304+0000 7fe970c65700 1 --2- 172.21.15.143:0/1474242631 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fe95006af00 0x7fe95006d3c0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fe96c1f67e0 tx=0x7fe968009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:00.318 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.307+0000 7fe961ffb700 1 -- 172.21.15.143:0/1474242631 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe95c054610 con 0x7fe96c0b1730 2023-11-08T19:28:00.498 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.497+0000 7fe9736ca700 1 -- 172.21.15.143:0/1474242631 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "test_data_pool", "var": "pg_num"} v 0) v1 -- 0x7fe96c0af2c0 con 0x7fe96c0b1730 2023-11-08T19:28:00.499 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.497+0000 7fe961ffb700 1 -- 172.21.15.143:0/1474242631 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "test_data_pool", "var": "pg_num"}]=0 v88) v1 ==== 103+0+11 (secure 0 0 0) 0x7fe95c057c30 con 0x7fe96c0b1730 2023-11-08T19:28:00.499 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2023-11-08T19:28:00.501 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.499+0000 7fe9736ca700 1 -- 172.21.15.143:0/1474242631 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fe95006af00 msgr2=0x7fe95006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:00.501 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.499+0000 7fe9736ca700 1 --2- 172.21.15.143:0/1474242631 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fe95006af00 0x7fe95006d3c0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fe96c1f67e0 tx=0x7fe968009040 comp rx=0 tx=0).stop 2023-11-08T19:28:00.502 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.499+0000 7fe9736ca700 1 -- 172.21.15.143:0/1474242631 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe96c0b1730 msgr2=0x7fe96c0cc4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:00.502 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.499+0000 7fe9736ca700 1 --2- 172.21.15.143:0/1474242631 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe96c0b1730 0x7fe96c0cc4d0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fe95c00c7f0 tx=0x7fe95c00cbb0 comp rx=0 tx=0).stop 2023-11-08T19:28:00.502 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.499+0000 7fe9736ca700 1 -- 172.21.15.143:0/1474242631 shutdown_connections 2023-11-08T19:28:00.502 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.499+0000 7fe9736ca700 1 --2- 172.21.15.143:0/1474242631 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fe95006af00 0x7fe95006d3c0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:00.503 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.499+0000 7fe9736ca700 1 --2- 172.21.15.143:0/1474242631 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe96c15bb20 0x7fe96c144950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:00.503 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.499+0000 7fe9736ca700 1 --2- 172.21.15.143:0/1474242631 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe96c0b1730 0x7fe96c0cc4d0 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:00.503 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.499+0000 7fe9736ca700 1 --2- 172.21.15.143:0/1474242631 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe96c0d1d80 0x7fe96c1535a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:00.503 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.499+0000 7fe9736ca700 1 -- 172.21.15.143:0/1474242631 >> 172.21.15.143:0/1474242631 conn(0x7fe96c08b000 msgr2=0x7fe96c0ae5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:00.504 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.499+0000 7fe9736ca700 1 -- 172.21.15.143:0/1474242631 shutdown_connections 2023-11-08T19:28:00.504 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.499+0000 7fe9736ca700 1 -- 172.21.15.143:0/1474242631 wait complete. 2023-11-08T19:28:00.513 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get test_metadata_pool pg_num 2023-11-08T19:28:00.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.635+0000 7fe384803700 1 Processor -- start 2023-11-08T19:28:00.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.635+0000 7fe384803700 1 -- start start 2023-11-08T19:28:00.640 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.636+0000 7fe384803700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe38015dba0 0x7fe38006c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:00.640 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.636+0000 7fe384803700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe38006b240 0x7fe3800caff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:00.640 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.636+0000 7fe384803700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fe3800cb530 0x7fe3800b22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:00.640 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.636+0000 7fe384803700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe380161890 con 0x7fe38006b240 2023-11-08T19:28:00.641 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.636+0000 7fe384803700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe38006b740 con 0x7fe3800cb530 2023-11-08T19:28:00.641 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.636+0000 7fe384803700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe38006ba20 con 0x7fe38015dba0 2023-11-08T19:28:00.641 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.637+0000 7fe37dd9b700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe38006b240 0x7fe3800caff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:00.641 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.637+0000 7fe37dd9b700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe38006b240 0x7fe3800caff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54460/0 (socket says 172.21.15.143:54460) 2023-11-08T19:28:00.641 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.637+0000 7fe37dd9b700 1 -- 172.21.15.143:0/1562354689 learned_addr learned my addr 172.21.15.143:0/1562354689 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:28:00.642 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.637+0000 7fe37e59c700 1 --2- 172.21.15.143:0/1562354689 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe38015dba0 0x7fe38006c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:00.642 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.637+0000 7fe37ed9d700 1 --2- 172.21.15.143:0/1562354689 >> v2:172.21.15.174:3300/0 conn(0x7fe3800cb530 0x7fe3800b22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:00.642 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.637+0000 7fe37dd9b700 1 -- 172.21.15.143:0/1562354689 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe38015dba0 msgr2=0x7fe38006c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:00.642 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.637+0000 7fe37dd9b700 1 --2- 172.21.15.143:0/1562354689 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe38015dba0 0x7fe38006c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:00.642 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.637+0000 7fe37dd9b700 1 -- 172.21.15.143:0/1562354689 >> v2:172.21.15.174:3300/0 conn(0x7fe3800cb530 msgr2=0x7fe3800b22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:00.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.637+0000 7fe37dd9b700 1 --2- 172.21.15.143:0/1562354689 >> v2:172.21.15.174:3300/0 conn(0x7fe3800cb530 0x7fe3800b22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:00.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.637+0000 7fe37dd9b700 1 -- 172.21.15.143:0/1562354689 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe38006bd00 con 0x7fe38006b240 2023-11-08T19:28:00.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.637+0000 7fe37ed9d700 1 --2- 172.21.15.143:0/1562354689 >> v2:172.21.15.174:3300/0 conn(0x7fe3800cb530 0x7fe3800b22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:28:00.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.637+0000 7fe37dd9b700 1 --2- 172.21.15.143:0/1562354689 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe38006b240 0x7fe3800caff0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fe38006f460 tx=0x7fe370009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e4e8df6f55bd5bb server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:00.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.637+0000 7fe37d59a700 1 -- 172.21.15.143:0/1562354689 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe37000b870 con 0x7fe38006b240 2023-11-08T19:28:00.644 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.637+0000 7fe37d59a700 1 -- 172.21.15.143:0/1562354689 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe37000b9d0 con 0x7fe38006b240 2023-11-08T19:28:00.644 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.637+0000 7fe37d59a700 1 -- 172.21.15.143:0/1562354689 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe37000bd60 con 0x7fe38006b240 2023-11-08T19:28:00.644 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.638+0000 7fe384803700 1 -- 172.21.15.143:0/1562354689 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe38006b240 msgr2=0x7fe3800caff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:00.644 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.638+0000 7fe384803700 1 --2- 172.21.15.143:0/1562354689 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe38006b240 0x7fe3800caff0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fe38006f460 tx=0x7fe370009bd0 comp rx=0 tx=0).stop 2023-11-08T19:28:00.645 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.638+0000 7fe384803700 1 -- 172.21.15.143:0/1562354689 shutdown_connections 2023-11-08T19:28:00.645 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.638+0000 7fe384803700 1 --2- 172.21.15.143:0/1562354689 >> v2:172.21.15.174:3300/0 conn(0x7fe3800cb530 0x7fe3800b22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:00.645 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.638+0000 7fe384803700 1 --2- 172.21.15.143:0/1562354689 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe38015dba0 0x7fe38006c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:00.645 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.638+0000 7fe384803700 1 --2- 172.21.15.143:0/1562354689 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe38006b240 0x7fe3800caff0 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:00.645 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.638+0000 7fe384803700 1 -- 172.21.15.143:0/1562354689 >> 172.21.15.143:0/1562354689 conn(0x7fe3800cd2e0 msgr2=0x7fe3800cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:00.645 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.638+0000 7fe384803700 1 -- 172.21.15.143:0/1562354689 shutdown_connections 2023-11-08T19:28:00.646 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.638+0000 7fe384803700 1 -- 172.21.15.143:0/1562354689 wait complete. 2023-11-08T19:28:00.648 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.639+0000 7fe384803700 1 Processor -- start 2023-11-08T19:28:00.648 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.639+0000 7fe384803700 1 -- start start 2023-11-08T19:28:00.649 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.640+0000 7fe384803700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe38006b240 0x7fe38015b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:00.649 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.640+0000 7fe384803700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe3800cb530 0x7fe3801e7420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:00.649 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.640+0000 7fe384803700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe38015dba0 0x7fe3800ca360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:00.649 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.640+0000 7fe384803700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3800adfb0 con 0x7fe38015dba0 2023-11-08T19:28:00.649 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.640+0000 7fe384803700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe38015b6e0 con 0x7fe3800cb530 2023-11-08T19:28:00.650 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.640+0000 7fe384803700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3801e7c70 con 0x7fe38006b240 2023-11-08T19:28:00.650 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.640+0000 7fe37ed9d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe38015dba0 0x7fe3800ca360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:00.650 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.640+0000 7fe37ed9d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe38015dba0 0x7fe3800ca360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54466/0 (socket says 172.21.15.143:54466) 2023-11-08T19:28:00.650 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.640+0000 7fe37ed9d700 1 -- 172.21.15.143:0/3454684246 learned_addr learned my addr 172.21.15.143:0/3454684246 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:28:00.650 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.640+0000 7fe37dd9b700 1 --2- 172.21.15.143:0/3454684246 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe3800cb530 0x7fe3801e7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:00.651 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.640+0000 7fe37e59c700 1 --2- 172.21.15.143:0/3454684246 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe38006b240 0x7fe38015b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:00.651 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.640+0000 7fe37ed9d700 1 -- 172.21.15.143:0/3454684246 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe38006b240 msgr2=0x7fe38015b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:00.651 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.640+0000 7fe37ed9d700 1 --2- 172.21.15.143:0/3454684246 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe38006b240 0x7fe38015b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:00.651 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.640+0000 7fe37ed9d700 1 -- 172.21.15.143:0/3454684246 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe3800cb530 msgr2=0x7fe3801e7420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:00.651 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.640+0000 7fe37ed9d700 1 --2- 172.21.15.143:0/3454684246 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe3800cb530 0x7fe3801e7420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:00.652 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.640+0000 7fe37ed9d700 1 -- 172.21.15.143:0/3454684246 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe370009550 con 0x7fe38015dba0 2023-11-08T19:28:00.652 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.640+0000 7fe37ed9d700 1 --2- 172.21.15.143:0/3454684246 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe38015dba0 0x7fe3800ca360 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fe378007df0 tx=0x7fe37800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:00.652 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.641+0000 7fe36effd700 1 -- 172.21.15.143:0/3454684246 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe37800de90 con 0x7fe38015dba0 2023-11-08T19:28:00.652 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.641+0000 7fe36effd700 1 -- 172.21.15.143:0/3454684246 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe37800f040 con 0x7fe38015dba0 2023-11-08T19:28:00.652 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.641+0000 7fe384803700 1 -- 172.21.15.143:0/3454684246 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3801e7f50 con 0x7fe38015dba0 2023-11-08T19:28:00.653 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.641+0000 7fe36effd700 1 -- 172.21.15.143:0/3454684246 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe378015620 con 0x7fe38015dba0 2023-11-08T19:28:00.653 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.641+0000 7fe384803700 1 -- 172.21.15.143:0/3454684246 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3800ae4e0 con 0x7fe38015dba0 2023-11-08T19:28:00.653 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.641+0000 7fe384803700 1 -- 172.21.15.143:0/3454684246 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3800ae700 con 0x7fe38015dba0 2023-11-08T19:28:00.653 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.642+0000 7fe36effd700 1 -- 172.21.15.143:0/3454684246 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe37801e3b0 con 0x7fe38015dba0 2023-11-08T19:28:00.653 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.642+0000 7fe36effd700 1 --2- 172.21.15.143:0/3454684246 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fe36006af00 0x7fe36006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:00.653 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.643+0000 7fe36effd700 1 -- 172.21.15.143:0/3454684246 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5920+0+0 (secure 0 0 0) 0x7fe378089910 con 0x7fe38015dba0 2023-11-08T19:28:00.654 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.643+0000 7fe37e59c700 1 --2- 172.21.15.143:0/3454684246 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fe36006af00 0x7fe36006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:00.654 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.643+0000 7fe37e59c700 1 --2- 172.21.15.143:0/3454684246 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fe36006af00 0x7fe36006d3c0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fe37400b9b0 tx=0x7fe374008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:00.654 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.647+0000 7fe36effd700 1 -- 172.21.15.143:0/3454684246 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe3780548f0 con 0x7fe38015dba0 2023-11-08T19:28:00.838 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.837+0000 7fe384803700 1 -- 172.21.15.143:0/3454684246 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "test_metadata_pool", "var": "pg_num"} v 0) v1 -- 0x7fe38004eee0 con 0x7fe38015dba0 2023-11-08T19:28:00.839 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.837+0000 7fe36effd700 1 -- 172.21.15.143:0/3454684246 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "test_metadata_pool", "var": "pg_num"}]=0 v88) v1 ==== 107+0+11 (secure 0 0 0) 0x7fe37801a0b0 con 0x7fe38015dba0 2023-11-08T19:28:00.839 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2023-11-08T19:28:00.841 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.839+0000 7fe384803700 1 -- 172.21.15.143:0/3454684246 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fe36006af00 msgr2=0x7fe36006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:00.841 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.839+0000 7fe384803700 1 --2- 172.21.15.143:0/3454684246 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fe36006af00 0x7fe36006d3c0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fe37400b9b0 tx=0x7fe374008040 comp rx=0 tx=0).stop 2023-11-08T19:28:00.841 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.839+0000 7fe384803700 1 -- 172.21.15.143:0/3454684246 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe38015dba0 msgr2=0x7fe3800ca360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:00.841 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.839+0000 7fe384803700 1 --2- 172.21.15.143:0/3454684246 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe38015dba0 0x7fe3800ca360 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fe378007df0 tx=0x7fe37800d3b0 comp rx=0 tx=0).stop 2023-11-08T19:28:00.841 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.839+0000 7fe384803700 1 -- 172.21.15.143:0/3454684246 shutdown_connections 2023-11-08T19:28:00.842 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.839+0000 7fe384803700 1 --2- 172.21.15.143:0/3454684246 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fe36006af00 0x7fe36006d3c0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:00.842 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.839+0000 7fe384803700 1 --2- 172.21.15.143:0/3454684246 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe38006b240 0x7fe38015b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:00.842 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.839+0000 7fe384803700 1 --2- 172.21.15.143:0/3454684246 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe38015dba0 0x7fe3800ca360 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:00.842 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.839+0000 7fe384803700 1 --2- 172.21.15.143:0/3454684246 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe3800cb530 0x7fe3801e7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:00.842 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.839+0000 7fe384803700 1 -- 172.21.15.143:0/3454684246 >> 172.21.15.143:0/3454684246 conn(0x7fe3800cd2e0 msgr2=0x7fe3800b1e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:00.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.839+0000 7fe384803700 1 -- 172.21.15.143:0/3454684246 shutdown_connections 2023-11-08T19:28:00.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.839+0000 7fe384803700 1 -- 172.21.15.143:0/3454684246 wait complete. 2023-11-08T19:28:00.853 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-08T19:28:00.974 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.972+0000 7f4df5bfc700 1 Processor -- start 2023-11-08T19:28:00.974 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.973+0000 7f4df5bfc700 1 -- start start 2023-11-08T19:28:00.977 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.973+0000 7f4df5bfc700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4df0072860 0x7f4df0160170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:00.977 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.973+0000 7f4df5bfc700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f4df006e9a0 0x7f4df00b3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:00.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.973+0000 7f4df5bfc700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4df006bd50 con 0x7f4df006dbb0 2023-11-08T19:28:00.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.973+0000 7f4df5bfc700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4df00cb1c0 con 0x7f4df006e9a0 2023-11-08T19:28:00.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.973+0000 7f4df5bfc700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4df006bff0 con 0x7f4df0072860 2023-11-08T19:28:00.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.974+0000 7f4deffff700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f4df006e9a0 0x7f4df00b3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:00.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.974+0000 7f4deeffd700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4df0072860 0x7f4df0160170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:00.979 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.974+0000 7f4deeffd700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4df0072860 0x7f4df0160170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.143:42108/0 (socket says 172.21.15.143:42108) 2023-11-08T19:28:00.979 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.974+0000 7f4deeffd700 1 -- 172.21.15.143:0/3830450966 learned_addr learned my addr 172.21.15.143:0/3830450966 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:28:00.979 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.974+0000 7f4deffff700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f4df006e9a0 0x7f4df00b3240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.143:37968/0 (socket says 172.21.15.143:37968) 2023-11-08T19:28:00.979 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.974+0000 7f4def7fe700 1 --1- 172.21.15.143:0/3830450966 >> v1:172.21.15.174:6789/0 conn(0x7f4df006dbb0 0x7f4df015fa60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.174:6789/0 says I am v1:172.21.15.143:33326/0 (socket says 172.21.15.143:33326) 2023-11-08T19:28:00.979 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.974+0000 7f4deeffd700 1 -- 172.21.15.143:0/3830450966 >> v2:172.21.15.174:3300/0 conn(0x7f4df006e9a0 msgr2=0x7f4df00b3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:00.980 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.974+0000 7f4deeffd700 1 --2- 172.21.15.143:0/3830450966 >> v2:172.21.15.174:3300/0 conn(0x7f4df006e9a0 0x7f4df00b3240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:00.980 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.974+0000 7f4deeffd700 1 -- 172.21.15.143:0/3830450966 >> v1:172.21.15.174:6789/0 conn(0x7f4df006dbb0 legacy=0x7f4df015fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:00.980 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.974+0000 7f4deeffd700 1 -- 172.21.15.143:0/3830450966 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4df006c2d0 con 0x7f4df0072860 2023-11-08T19:28:00.980 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.974+0000 7f4deffff700 1 --2- 172.21.15.143:0/3830450966 >> v2:172.21.15.174:3300/0 conn(0x7f4df006e9a0 0x7f4df00b3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T19:28:00.980 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.974+0000 7f4deeffd700 1 --2- 172.21.15.143:0/3830450966 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4df0072860 0x7f4df0160170 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4dd80098f0 tx=0x7f4dd8009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1677692119509f3c server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:00.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.974+0000 7f4dee7fc700 1 -- 172.21.15.143:0/3830450966 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dd8011830 con 0x7f4df0072860 2023-11-08T19:28:00.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.974+0000 7f4dee7fc700 1 -- 172.21.15.143:0/3830450966 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dd8011990 con 0x7f4df0072860 2023-11-08T19:28:00.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.974+0000 7f4dee7fc700 1 -- 172.21.15.143:0/3830450966 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dd8011d20 con 0x7f4df0072860 2023-11-08T19:28:00.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.975+0000 7f4df5bfc700 1 -- 172.21.15.143:0/3830450966 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4df0072860 msgr2=0x7f4df0160170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:00.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.975+0000 7f4df5bfc700 1 --2- 172.21.15.143:0/3830450966 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4df0072860 0x7f4df0160170 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4dd80098f0 tx=0x7f4dd8009c00 comp rx=0 tx=0).stop 2023-11-08T19:28:00.982 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.975+0000 7f4df5bfc700 1 -- 172.21.15.143:0/3830450966 shutdown_connections 2023-11-08T19:28:00.982 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.975+0000 7f4df5bfc700 1 --2- 172.21.15.143:0/3830450966 >> v2:172.21.15.174:3300/0 conn(0x7f4df006e9a0 0x7f4df00b3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:00.982 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.975+0000 7f4df5bfc700 1 --2- 172.21.15.143:0/3830450966 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4df0072860 0x7f4df0160170 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:00.982 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.975+0000 7f4df5bfc700 1 -- 172.21.15.143:0/3830450966 >> 172.21.15.143:0/3830450966 conn(0x7f4df00b1a20 msgr2=0x7f4df00b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:00.982 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.975+0000 7f4df5bfc700 1 -- 172.21.15.143:0/3830450966 shutdown_connections 2023-11-08T19:28:00.983 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.975+0000 7f4df5bfc700 1 -- 172.21.15.143:0/3830450966 wait complete. 2023-11-08T19:28:00.983 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.976+0000 7f4df5bfc700 1 Processor -- start 2023-11-08T19:28:00.986 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.976+0000 7f4df5bfc700 1 -- start start 2023-11-08T19:28:00.986 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.977+0000 7f4df5bfc700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4df006dbb0 0x7f4df00ae430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:00.986 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.977+0000 7f4df5bfc700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4df006e9a0 0x7f4df00ce440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:00.986 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.977+0000 7f4df5bfc700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4df0072860 0x7f4df0158140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:00.987 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.977+0000 7f4df5bfc700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4df00ce980 con 0x7f4df006e9a0 2023-11-08T19:28:00.987 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.977+0000 7f4df5bfc700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4df00cfa60 con 0x7f4df006dbb0 2023-11-08T19:28:00.987 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.977+0000 7f4df5bfc700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4df00cd9f0 con 0x7f4df0072860 2023-11-08T19:28:00.987 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.977+0000 7f4deeffd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4df006e9a0 0x7f4df00ce440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:00.987 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.977+0000 7f4deeffd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4df006e9a0 0x7f4df00ce440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54478/0 (socket says 172.21.15.143:54478) 2023-11-08T19:28:00.988 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.977+0000 7f4deeffd700 1 -- 172.21.15.143:0/583822839 learned_addr learned my addr 172.21.15.143:0/583822839 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:28:00.988 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.977+0000 7f4deffff700 1 --2- 172.21.15.143:0/583822839 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4df0072860 0x7f4df0158140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:00.988 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.977+0000 7f4def7fe700 1 --2- 172.21.15.143:0/583822839 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4df006dbb0 0x7f4df00ae430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:00.988 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.977+0000 7f4deeffd700 1 -- 172.21.15.143:0/583822839 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4df0072860 msgr2=0x7f4df0158140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:00.988 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.977+0000 7f4deeffd700 1 --2- 172.21.15.143:0/583822839 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4df0072860 0x7f4df0158140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:00.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.977+0000 7f4deeffd700 1 -- 172.21.15.143:0/583822839 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4df006dbb0 msgr2=0x7f4df00ae430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:00.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.977+0000 7f4deeffd700 1 --2- 172.21.15.143:0/583822839 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4df006dbb0 0x7f4df00ae430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:00.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.977+0000 7f4deeffd700 1 -- 172.21.15.143:0/583822839 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4de8009a70 con 0x7f4df006e9a0 2023-11-08T19:28:00.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.977+0000 7f4deffff700 1 --2- 172.21.15.143:0/583822839 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4df0072860 0x7f4df0158140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:28:00.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.977+0000 7f4def7fe700 1 --2- 172.21.15.143:0/583822839 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4df006dbb0 0x7f4df00ae430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:28:00.990 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.978+0000 7f4deeffd700 1 --2- 172.21.15.143:0/583822839 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4df006e9a0 0x7f4df00ce440 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f4df006c130 tx=0x7f4dd8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:00.990 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.978+0000 7f4de7fff700 1 -- 172.21.15.143:0/583822839 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dd80036a0 con 0x7f4df006e9a0 2023-11-08T19:28:00.990 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.978+0000 7f4de7fff700 1 -- 172.21.15.143:0/583822839 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dd8003800 con 0x7f4df006e9a0 2023-11-08T19:28:00.990 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.978+0000 7f4df5bfc700 1 -- 172.21.15.143:0/583822839 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4dd8009550 con 0x7f4df006e9a0 2023-11-08T19:28:00.991 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.978+0000 7f4de7fff700 1 -- 172.21.15.143:0/583822839 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dd8003b90 con 0x7f4df006e9a0 2023-11-08T19:28:00.991 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.978+0000 7f4df5bfc700 1 -- 172.21.15.143:0/583822839 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4df00cdfb0 con 0x7f4df006e9a0 2023-11-08T19:28:00.991 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.979+0000 7f4df5bfc700 1 -- 172.21.15.143:0/583822839 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4df00ce140 con 0x7f4df006e9a0 2023-11-08T19:28:00.991 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.979+0000 7f4de7fff700 1 -- 172.21.15.143:0/583822839 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4dd8003cf0 con 0x7f4df006e9a0 2023-11-08T19:28:00.991 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.980+0000 7f4de7fff700 1 --2- 172.21.15.143:0/583822839 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f4dd006af00 0x7f4dd006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:00.991 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.980+0000 7f4de7fff700 1 -- 172.21.15.143:0/583822839 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5920+0+0 (secure 0 0 0) 0x7f4dd808a920 con 0x7f4df006e9a0 2023-11-08T19:28:00.992 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.980+0000 7f4def7fe700 1 --2- 172.21.15.143:0/583822839 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f4dd006af00 0x7f4dd006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:00.992 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.980+0000 7f4def7fe700 1 --2- 172.21.15.143:0/583822839 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f4dd006af00 0x7f4dd006d3c0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f4de0000f60 tx=0x7f4de0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:00.992 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:00.984+0000 7f4de7fff700 1 -- 172.21.15.143:0/583822839 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4dd8055990 con 0x7f4df006e9a0 2023-11-08T19:28:01.176 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.174+0000 7f4df5bfc700 1 -- 172.21.15.143:0/583822839 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f4df004eee0 con 0x7f4df006e9a0 2023-11-08T19:28:01.176 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.174+0000 7f4de7fff700 1 -- 172.21.15.143:0/583822839 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v88) v1 ==== 84+0+287 (secure 0 0 0) 0x7f4dd8028130 con 0x7f4df006e9a0 2023-11-08T19:28:01.177 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:28:01.177 INFO:teuthology.orchestra.run.smithi143.stdout:[ 2023-11-08T19:28:01.177 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-08T19:28:01.177 INFO:teuthology.orchestra.run.smithi143.stdout: "poolnum": 1, 2023-11-08T19:28:01.177 INFO:teuthology.orchestra.run.smithi143.stdout: "poolname": ".mgr" 2023-11-08T19:28:01.178 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-08T19:28:01.178 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-08T19:28:01.178 INFO:teuthology.orchestra.run.smithi143.stdout: "poolnum": 6, 2023-11-08T19:28:01.178 INFO:teuthology.orchestra.run.smithi143.stdout: "poolname": "snap-test-pool" 2023-11-08T19:28:01.178 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-08T19:28:01.179 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-08T19:28:01.179 INFO:teuthology.orchestra.run.smithi143.stdout: "poolnum": 11, 2023-11-08T19:28:01.179 INFO:teuthology.orchestra.run.smithi143.stdout: "poolname": "test_data_pool" 2023-11-08T19:28:01.179 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-11-08T19:28:01.179 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-11-08T19:28:01.180 INFO:teuthology.orchestra.run.smithi143.stdout: "poolnum": 12, 2023-11-08T19:28:01.180 INFO:teuthology.orchestra.run.smithi143.stdout: "poolname": "test_metadata_pool" 2023-11-08T19:28:01.180 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-11-08T19:28:01.180 INFO:teuthology.orchestra.run.smithi143.stdout:] 2023-11-08T19:28:01.181 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.176+0000 7f4df5bfc700 1 -- 172.21.15.143:0/583822839 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f4dd006af00 msgr2=0x7f4dd006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:01.181 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.176+0000 7f4df5bfc700 1 --2- 172.21.15.143:0/583822839 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f4dd006af00 0x7f4dd006d3c0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f4de0000f60 tx=0x7f4de0009040 comp rx=0 tx=0).stop 2023-11-08T19:28:01.182 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.176+0000 7f4df5bfc700 1 -- 172.21.15.143:0/583822839 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4df006e9a0 msgr2=0x7f4df00ce440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:01.182 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.177+0000 7f4df5bfc700 1 --2- 172.21.15.143:0/583822839 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4df006e9a0 0x7f4df00ce440 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f4df006c130 tx=0x7f4dd8004750 comp rx=0 tx=0).stop 2023-11-08T19:28:01.182 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.177+0000 7f4df5bfc700 1 -- 172.21.15.143:0/583822839 shutdown_connections 2023-11-08T19:28:01.182 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.177+0000 7f4df5bfc700 1 --2- 172.21.15.143:0/583822839 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f4dd006af00 0x7f4dd006d3c0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:01.182 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.177+0000 7f4df5bfc700 1 --2- 172.21.15.143:0/583822839 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4df0072860 0x7f4df0158140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:01.183 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.177+0000 7f4df5bfc700 1 --2- 172.21.15.143:0/583822839 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4df006dbb0 0x7f4df00ae430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:01.183 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.177+0000 7f4df5bfc700 1 --2- 172.21.15.143:0/583822839 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4df006e9a0 0x7f4df00ce440 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:01.184 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.177+0000 7f4df5bfc700 1 -- 172.21.15.143:0/583822839 >> 172.21.15.143:0/583822839 conn(0x7f4df00b1a20 msgr2=0x7f4df006df90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:01.184 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.177+0000 7f4df5bfc700 1 -- 172.21.15.143:0/583822839 shutdown_connections 2023-11-08T19:28:01.184 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.177+0000 7f4df5bfc700 1 -- 172.21.15.143:0/583822839 wait complete. 2023-11-08T19:28:01.190 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-08T19:28:01.191 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-08T19:28:01.315 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.313+0000 7f33046ab700 1 Processor -- start 2023-11-08T19:28:01.315 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.313+0000 7f33046ab700 1 -- start start 2023-11-08T19:28:01.318 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.315+0000 7f33046ab700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f32fc06bd20 0x7f32fc0d01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:01.318 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.315+0000 7f33046ab700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f32fc0b61e0 0x7f32fc0cc730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:01.318 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.315+0000 7f33046ab700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32fc0cc2a0 con 0x7f32fc15a950 2023-11-08T19:28:01.318 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.315+0000 7f33046ab700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32fc0cc130 con 0x7f32fc0b61e0 2023-11-08T19:28:01.318 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.315+0000 7f33046ab700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32fc06c1b0 con 0x7f32fc06bd20 2023-11-08T19:28:01.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.315+0000 7f3302c48700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f32fc0b61e0 0x7f32fc0cc730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:01.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.315+0000 7f3302c48700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f32fc0b61e0 0x7f32fc0cc730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54488/0 (socket says 172.21.15.143:54488) 2023-11-08T19:28:01.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.315+0000 7f3302c48700 1 -- 172.21.15.143:0/1980199785 learned_addr learned my addr 172.21.15.143:0/1980199785 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:28:01.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.315+0000 7f3301c46700 1 --2- 172.21.15.143:0/1980199785 >> v2:172.21.15.174:3300/0 conn(0x7f32fc06bd20 0x7f32fc0d01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:01.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.315+0000 7f3302c48700 1 -- 172.21.15.143:0/1980199785 >> v2:172.21.15.174:3300/0 conn(0x7f32fc06bd20 msgr2=0x7f32fc0d01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:01.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.315+0000 7f3302c48700 1 --2- 172.21.15.143:0/1980199785 >> v2:172.21.15.174:3300/0 conn(0x7f32fc06bd20 0x7f32fc0d01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:01.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.315+0000 7f3302c48700 1 -- 172.21.15.143:0/1980199785 >> v1:172.21.15.174:6789/0 conn(0x7f32fc15a950 legacy=0x7f32fc072370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:01.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.315+0000 7f3302c48700 1 -- 172.21.15.143:0/1980199785 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32fc06c490 con 0x7f32fc0b61e0 2023-11-08T19:28:01.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.315+0000 7f3302c48700 1 --2- 172.21.15.143:0/1980199785 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f32fc0b61e0 0x7f32fc0cc730 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f32fc0cd240 tx=0x7f32f400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c1f5a310c398070 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:01.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.316+0000 7f3301445700 1 -- 172.21.15.143:0/1980199785 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32f400e070 con 0x7f32fc0b61e0 2023-11-08T19:28:01.321 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.316+0000 7f3301445700 1 -- 172.21.15.143:0/1980199785 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32f400b910 con 0x7f32fc0b61e0 2023-11-08T19:28:01.321 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.316+0000 7f3301445700 1 -- 172.21.15.143:0/1980199785 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32f400bcc0 con 0x7f32fc0b61e0 2023-11-08T19:28:01.321 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.316+0000 7f33046ab700 1 -- 172.21.15.143:0/1980199785 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f32fc0b61e0 msgr2=0x7f32fc0cc730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:01.321 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.316+0000 7f33046ab700 1 --2- 172.21.15.143:0/1980199785 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f32fc0b61e0 0x7f32fc0cc730 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f32fc0cd240 tx=0x7f32f400b330 comp rx=0 tx=0).stop 2023-11-08T19:28:01.321 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.316+0000 7f33046ab700 1 -- 172.21.15.143:0/1980199785 shutdown_connections 2023-11-08T19:28:01.322 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.316+0000 7f33046ab700 1 --2- 172.21.15.143:0/1980199785 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f32fc0b61e0 0x7f32fc0cc730 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:01.322 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.316+0000 7f33046ab700 1 --2- 172.21.15.143:0/1980199785 >> v2:172.21.15.174:3300/0 conn(0x7f32fc06bd20 0x7f32fc0d01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:01.322 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.316+0000 7f33046ab700 1 -- 172.21.15.143:0/1980199785 >> 172.21.15.143:0/1980199785 conn(0x7f32fc0cd2e0 msgr2=0x7f32fc0cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:01.322 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.316+0000 7f33046ab700 1 -- 172.21.15.143:0/1980199785 shutdown_connections 2023-11-08T19:28:01.322 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.316+0000 7f33046ab700 1 -- 172.21.15.143:0/1980199785 wait complete. 2023-11-08T19:28:01.325 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.317+0000 7f33046ab700 1 Processor -- start 2023-11-08T19:28:01.326 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.317+0000 7f33046ab700 1 -- start start 2023-11-08T19:28:01.326 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.318+0000 7f33046ab700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f32fc0b61e0 0x7f32fc150d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:01.326 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.318+0000 7f33046ab700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f32fc15a950 0x7f32fc0d0f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:01.326 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.318+0000 7f33046ab700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f32fc0d14a0 0x7f32fc158ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:01.326 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.318+0000 7f33046ab700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32fc0ca800 con 0x7f32fc15a950 2023-11-08T19:28:01.327 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.318+0000 7f33046ab700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32fc0b0990 con 0x7f32fc0b61e0 2023-11-08T19:28:01.327 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.318+0000 7f33046ab700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32fc0b0c30 con 0x7f32fc0d14a0 2023-11-08T19:28:01.327 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.318+0000 7f3301c46700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f32fc15a950 0x7f32fc0d0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:01.327 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.318+0000 7f3301c46700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f32fc15a950 0x7f32fc0d0f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54492/0 (socket says 172.21.15.143:54492) 2023-11-08T19:28:01.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.318+0000 7f3301c46700 1 -- 172.21.15.143:0/46876552 learned_addr learned my addr 172.21.15.143:0/46876552 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:28:01.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.318+0000 7f3302447700 1 --2- 172.21.15.143:0/46876552 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f32fc0b61e0 0x7f32fc150d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:01.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.318+0000 7f3302c48700 1 --2- 172.21.15.143:0/46876552 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f32fc0d14a0 0x7f32fc158ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:01.329 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.318+0000 7f3301c46700 1 -- 172.21.15.143:0/46876552 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f32fc0d14a0 msgr2=0x7f32fc158ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:01.329 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.319+0000 7f3301c46700 1 --2- 172.21.15.143:0/46876552 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f32fc0d14a0 0x7f32fc158ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:01.329 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.319+0000 7f3301c46700 1 -- 172.21.15.143:0/46876552 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f32fc0b61e0 msgr2=0x7f32fc150d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:01.329 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.319+0000 7f3301c46700 1 --2- 172.21.15.143:0/46876552 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f32fc0b61e0 0x7f32fc150d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:01.330 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.319+0000 7f3301c46700 1 -- 172.21.15.143:0/46876552 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32f4009b30 con 0x7f32fc15a950 2023-11-08T19:28:01.330 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.319+0000 7f3302447700 1 --2- 172.21.15.143:0/46876552 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f32fc0b61e0 0x7f32fc150d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:28:01.330 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.319+0000 7f3301c46700 1 --2- 172.21.15.143:0/46876552 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f32fc15a950 0x7f32fc0d0f60 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f32f800e9b0 tx=0x7f32f800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:01.330 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.319+0000 7f32f2ffd700 1 -- 172.21.15.143:0/46876552 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32f8009710 con 0x7f32fc15a950 2023-11-08T19:28:01.331 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.319+0000 7f32f2ffd700 1 -- 172.21.15.143:0/46876552 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32f800c850 con 0x7f32fc15a950 2023-11-08T19:28:01.331 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.319+0000 7f33046ab700 1 -- 172.21.15.143:0/46876552 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32fc0b0f10 con 0x7f32fc15a950 2023-11-08T19:28:01.331 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.319+0000 7f32f2ffd700 1 -- 172.21.15.143:0/46876552 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32f800cbc0 con 0x7f32fc15a950 2023-11-08T19:28:01.331 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.319+0000 7f33046ab700 1 -- 172.21.15.143:0/46876552 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32fc0b14d0 con 0x7f32fc15a950 2023-11-08T19:28:01.332 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.320+0000 7f33046ab700 1 -- 172.21.15.143:0/46876552 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32fc0b1b20 con 0x7f32fc15a950 2023-11-08T19:28:01.332 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.320+0000 7f32f2ffd700 1 -- 172.21.15.143:0/46876552 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f32f800cd20 con 0x7f32fc15a950 2023-11-08T19:28:01.332 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.321+0000 7f32f2ffd700 1 --2- 172.21.15.143:0/46876552 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f32e006af00 0x7f32e006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:01.332 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.321+0000 7f32f2ffd700 1 -- 172.21.15.143:0/46876552 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5920+0+0 (secure 0 0 0) 0x7f32f8089800 con 0x7f32fc15a950 2023-11-08T19:28:01.333 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.321+0000 7f3302447700 1 --2- 172.21.15.143:0/46876552 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f32e006af00 0x7f32e006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:01.333 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.321+0000 7f3302447700 1 --2- 172.21.15.143:0/46876552 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f32e006af00 0x7f32e006d3c0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f32ec001670 tx=0x7f32ec009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:01.333 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.325+0000 7f32f2ffd700 1 -- 172.21.15.143:0/46876552 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f32f80547e0 con 0x7f32fc15a950 2023-11-08T19:28:01.516 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.514+0000 7f33046ab700 1 -- 172.21.15.143:0/46876552 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f32fc04eee0 con 0x7f32fc15a950 2023-11-08T19:28:01.688 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.687+0000 7f32f2ffd700 1 -- 172.21.15.143:0/46876552 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v89) v1 ==== 138+0+0 (secure 0 0 0) 0x7f32f80210b0 con 0x7f32fc15a950 2023-11-08T19:28:01.691 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.690+0000 7f33046ab700 1 -- 172.21.15.143:0/46876552 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f32e006af00 msgr2=0x7f32e006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:01.691 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.690+0000 7f33046ab700 1 --2- 172.21.15.143:0/46876552 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f32e006af00 0x7f32e006d3c0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f32ec001670 tx=0x7f32ec009040 comp rx=0 tx=0).stop 2023-11-08T19:28:01.692 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.690+0000 7f33046ab700 1 -- 172.21.15.143:0/46876552 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f32fc15a950 msgr2=0x7f32fc0d0f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:01.692 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.690+0000 7f33046ab700 1 --2- 172.21.15.143:0/46876552 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f32fc15a950 0x7f32fc0d0f60 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f32f800e9b0 tx=0x7f32f800ed70 comp rx=0 tx=0).stop 2023-11-08T19:28:01.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.690+0000 7f33046ab700 1 -- 172.21.15.143:0/46876552 shutdown_connections 2023-11-08T19:28:01.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.690+0000 7f33046ab700 1 --2- 172.21.15.143:0/46876552 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f32e006af00 0x7f32e006d3c0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:01.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.690+0000 7f33046ab700 1 --2- 172.21.15.143:0/46876552 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f32fc0d14a0 0x7f32fc158ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:01.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.690+0000 7f33046ab700 1 --2- 172.21.15.143:0/46876552 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f32fc0b61e0 0x7f32fc150d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:01.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.690+0000 7f33046ab700 1 --2- 172.21.15.143:0/46876552 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f32fc15a950 0x7f32fc0d0f60 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:01.694 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.690+0000 7f33046ab700 1 -- 172.21.15.143:0/46876552 >> 172.21.15.143:0/46876552 conn(0x7f32fc0cd2e0 msgr2=0x7f32fc15af40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:01.694 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.691+0000 7f33046ab700 1 -- 172.21.15.143:0/46876552 shutdown_connections 2023-11-08T19:28:01.694 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.691+0000 7f33046ab700 1 -- 172.21.15.143:0/46876552 wait complete. 2023-11-08T19:28:01.698 INFO:teuthology.orchestra.run.smithi143.stderr:pool 'cephfs_metadata' created 2023-11-08T19:28:01.709 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-08T19:28:01.836 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.835+0000 7fc33fbc7700 1 Processor -- start 2023-11-08T19:28:01.837 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.835+0000 7fc33fbc7700 1 -- start start 2023-11-08T19:28:01.839 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.836+0000 7fc33fbc7700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc3380d1d80 0x7fc3380b2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:01.839 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.836+0000 7fc33fbc7700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3380d0420 con 0x7fc33815bb20 2023-11-08T19:28:01.839 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.836+0000 7fc33fbc7700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3380cf810 con 0x7fc3380b1730 2023-11-08T19:28:01.839 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.836+0000 7fc33fbc7700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3380cf600 con 0x7fc3380d1d80 2023-11-08T19:28:01.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.836+0000 7fc33d162700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc3380d1d80 0x7fc3380b2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:01.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.836+0000 7fc33d162700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc3380d1d80 0x7fc3380b2670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54500/0 (socket says 172.21.15.143:54500) 2023-11-08T19:28:01.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.836+0000 7fc33d162700 1 -- 172.21.15.143:0/1919489889 learned_addr learned my addr 172.21.15.143:0/1919489889 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:28:01.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.836+0000 7fc33d963700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fc33815bb20 0x7fc3380d1670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:50652/0 (socket says 172.21.15.143:50652) 2023-11-08T19:28:01.841 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.836+0000 7fc33d162700 1 -- 172.21.15.143:0/1919489889 >> v1:172.21.15.174:6789/0 conn(0x7fc3380b1730 legacy=0x7fc3380cc4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:01.841 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.836+0000 7fc33d162700 1 -- 172.21.15.143:0/1919489889 >> v1:172.21.15.143:6789/0 conn(0x7fc33815bb20 legacy=0x7fc3380d1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:01.841 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.836+0000 7fc33d162700 1 -- 172.21.15.143:0/1919489889 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3380cfaf0 con 0x7fc3380d1d80 2023-11-08T19:28:01.841 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.836+0000 7fc33c961700 1 -- 172.21.15.143:0/1919489889 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3746916694 0 0) 0x7fc3380d0420 con 0x7fc33815bb20 2023-11-08T19:28:01.842 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.837+0000 7fc33d162700 1 --2- 172.21.15.143:0/1919489889 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc3380d1d80 0x7fc3380b2670 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fc3340098f0 tx=0x7fc334009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20ade509ee78b610 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:01.842 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.837+0000 7fc33c961700 1 -- 172.21.15.143:0/1919489889 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc334004230 con 0x7fc3380d1d80 2023-11-08T19:28:01.842 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.837+0000 7fc33c961700 1 -- 172.21.15.143:0/1919489889 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc334004390 con 0x7fc3380d1d80 2023-11-08T19:28:01.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.837+0000 7fc33c961700 1 -- 172.21.15.143:0/1919489889 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3340036a0 con 0x7fc3380d1d80 2023-11-08T19:28:01.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.837+0000 7fc33fbc7700 1 -- 172.21.15.143:0/1919489889 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc3380d1d80 msgr2=0x7fc3380b2670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:01.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.837+0000 7fc33fbc7700 1 --2- 172.21.15.143:0/1919489889 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc3380d1d80 0x7fc3380b2670 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fc3340098f0 tx=0x7fc334009c00 comp rx=0 tx=0).stop 2023-11-08T19:28:01.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.837+0000 7fc33fbc7700 1 -- 172.21.15.143:0/1919489889 shutdown_connections 2023-11-08T19:28:01.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.837+0000 7fc33fbc7700 1 --2- 172.21.15.143:0/1919489889 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc3380d1d80 0x7fc3380b2670 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:01.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.837+0000 7fc33fbc7700 1 -- 172.21.15.143:0/1919489889 >> 172.21.15.143:0/1919489889 conn(0x7fc33808b000 msgr2=0x7fc33809ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:01.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.837+0000 7fc33fbc7700 1 -- 172.21.15.143:0/1919489889 shutdown_connections 2023-11-08T19:28:01.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.837+0000 7fc33fbc7700 1 -- 172.21.15.143:0/1919489889 wait complete. 2023-11-08T19:28:01.847 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.838+0000 7fc33fbc7700 1 Processor -- start 2023-11-08T19:28:01.847 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.839+0000 7fc33fbc7700 1 -- start start 2023-11-08T19:28:01.847 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.839+0000 7fc33fbc7700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc3380b1730 0x7fc3380ccee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:01.847 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.839+0000 7fc33fbc7700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc3380d1d80 0x7fc3381538f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:01.848 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.839+0000 7fc33fbc7700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc33815bb20 0x7fc3380c6c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:01.848 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.839+0000 7fc33fbc7700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3380c7b30 con 0x7fc3380b1730 2023-11-08T19:28:01.848 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.839+0000 7fc33fbc7700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3380cc4d0 con 0x7fc33815bb20 2023-11-08T19:28:01.848 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.839+0000 7fc33fbc7700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3380cca50 con 0x7fc3380d1d80 2023-11-08T19:28:01.848 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.839+0000 7fc33d963700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc3380b1730 0x7fc3380ccee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:01.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.839+0000 7fc33d963700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc3380b1730 0x7fc3380ccee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54504/0 (socket says 172.21.15.143:54504) 2023-11-08T19:28:01.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.839+0000 7fc33d963700 1 -- 172.21.15.143:0/3209031273 learned_addr learned my addr 172.21.15.143:0/3209031273 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:28:01.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.840+0000 7fc33d963700 1 -- 172.21.15.143:0/3209031273 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc3380d1d80 msgr2=0x7fc3381538f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:01.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.840+0000 7fc33d963700 1 --2- 172.21.15.143:0/3209031273 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc3380d1d80 0x7fc3381538f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:01.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.840+0000 7fc33d963700 1 -- 172.21.15.143:0/3209031273 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc33815bb20 msgr2=0x7fc3380c6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:01.850 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.840+0000 7fc33e164700 1 --2- 172.21.15.143:0/3209031273 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc33815bb20 0x7fc3380c6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:01.850 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.840+0000 7fc33d963700 1 --2- 172.21.15.143:0/3209031273 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc33815bb20 0x7fc3380c6c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:01.850 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.840+0000 7fc33d963700 1 -- 172.21.15.143:0/3209031273 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc334009550 con 0x7fc3380b1730 2023-11-08T19:28:01.850 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.840+0000 7fc33d963700 1 --2- 172.21.15.143:0/3209031273 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc3380b1730 0x7fc3380ccee0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fc32800c550 tx=0x7fc32800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:01.851 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.840+0000 7fc32e7fc700 1 -- 172.21.15.143:0/3209031273 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc328004c80 con 0x7fc3380b1730 2023-11-08T19:28:01.851 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.840+0000 7fc32e7fc700 1 -- 172.21.15.143:0/3209031273 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc328004e00 con 0x7fc3380b1730 2023-11-08T19:28:01.851 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.840+0000 7fc33fbc7700 1 -- 172.21.15.143:0/3209031273 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3380c7570 con 0x7fc3380b1730 2023-11-08T19:28:01.851 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.840+0000 7fc32e7fc700 1 -- 172.21.15.143:0/3209031273 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc328007310 con 0x7fc3380b1730 2023-11-08T19:28:01.852 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.840+0000 7fc33fbc7700 1 -- 172.21.15.143:0/3209031273 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3380cc6e0 con 0x7fc3380b1730 2023-11-08T19:28:01.852 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.841+0000 7fc33fbc7700 1 -- 172.21.15.143:0/3209031273 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3380cc8a0 con 0x7fc3380b1730 2023-11-08T19:28:01.852 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.841+0000 7fc32e7fc700 1 -- 172.21.15.143:0/3209031273 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc328007470 con 0x7fc3380b1730 2023-11-08T19:28:01.852 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.842+0000 7fc32e7fc700 1 --2- 172.21.15.143:0/3209031273 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc31c06af00 0x7fc31c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:01.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.842+0000 7fc32e7fc700 1 -- 172.21.15.143:0/3209031273 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6290+0+0 (secure 0 0 0) 0x7fc3280895b0 con 0x7fc3380b1730 2023-11-08T19:28:01.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.842+0000 7fc33d162700 1 --2- 172.21.15.143:0/3209031273 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc31c06af00 0x7fc31c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:01.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.842+0000 7fc33d162700 1 --2- 172.21.15.143:0/3209031273 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc31c06af00 0x7fc31c06d3c0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fc3381e7770 tx=0x7fc334010000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:01.854 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:01.846+0000 7fc32e7fc700 1 -- 172.21.15.143:0/3209031273 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc328054380 con 0x7fc3380b1730 2023-11-08T19:28:02.036 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:02.034+0000 7fc33fbc7700 1 -- 172.21.15.143:0/3209031273 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fc3380ccd30 con 0x7fc3380b1730 2023-11-08T19:28:02.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:02.691+0000 7fc32e7fc700 1 -- 172.21.15.143:0/3209031273 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v90) v1 ==== 170+0+0 (secure 0 0 0) 0x7fc3280579a0 con 0x7fc3380b1730 2023-11-08T19:28:02.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:02.694+0000 7fc33fbc7700 1 -- 172.21.15.143:0/3209031273 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc31c06af00 msgr2=0x7fc31c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:02.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:02.694+0000 7fc33fbc7700 1 --2- 172.21.15.143:0/3209031273 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc31c06af00 0x7fc31c06d3c0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fc3381e7770 tx=0x7fc334010000 comp rx=0 tx=0).stop 2023-11-08T19:28:02.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:02.694+0000 7fc33fbc7700 1 -- 172.21.15.143:0/3209031273 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc3380b1730 msgr2=0x7fc3380ccee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:02.697 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:02.694+0000 7fc33fbc7700 1 --2- 172.21.15.143:0/3209031273 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc3380b1730 0x7fc3380ccee0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fc32800c550 tx=0x7fc32800c910 comp rx=0 tx=0).stop 2023-11-08T19:28:02.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:02.695+0000 7fc33fbc7700 1 -- 172.21.15.143:0/3209031273 shutdown_connections 2023-11-08T19:28:02.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:02.695+0000 7fc33fbc7700 1 --2- 172.21.15.143:0/3209031273 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc31c06af00 0x7fc31c06d3c0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:02.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:02.695+0000 7fc33fbc7700 1 --2- 172.21.15.143:0/3209031273 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc33815bb20 0x7fc3380c6c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:02.699 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:02.695+0000 7fc33fbc7700 1 --2- 172.21.15.143:0/3209031273 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc3380b1730 0x7fc3380ccee0 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:02.699 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:02.695+0000 7fc33fbc7700 1 --2- 172.21.15.143:0/3209031273 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc3380d1d80 0x7fc3381538f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:02.699 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:02.695+0000 7fc33fbc7700 1 -- 172.21.15.143:0/3209031273 >> 172.21.15.143:0/3209031273 conn(0x7fc33808b000 msgr2=0x7fc338070b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:02.699 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:02.696+0000 7fc33fbc7700 1 -- 172.21.15.143:0/3209031273 shutdown_connections 2023-11-08T19:28:02.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:02.696+0000 7fc33fbc7700 1 -- 172.21.15.143:0/3209031273 wait complete. 2023-11-08T19:28:02.701 INFO:teuthology.orchestra.run.smithi143.stderr:pool 'cephfs_data' created 2023-11-08T19:28:02.719 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-08T19:28:02.846 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:02.845+0000 7f49e4604700 1 Processor -- start 2023-11-08T19:28:02.847 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:02.845+0000 7f49e4604700 1 -- start start 2023-11-08T19:28:02.848 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:02.846+0000 7f49e4604700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f49dc06bd20 0x7f49dc0d01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:02.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:02.846+0000 7f49e4604700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49dc0cc2a0 con 0x7f49dc15a950 2023-11-08T19:28:02.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:02.846+0000 7f49e4604700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49dc06c1b0 con 0x7f49dc0b61e0 2023-11-08T19:28:02.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:02.846+0000 7f49e4604700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49dc0cc130 con 0x7f49dc06bd20 2023-11-08T19:28:02.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:02.846+0000 7f49e23a0700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f49dc15a950 0x7f49dc072370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:50666/0 (socket says 172.21.15.143:50666) 2023-11-08T19:28:02.850 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:02.846+0000 7f49e23a0700 1 -- 172.21.15.143:0/644514793 learned_addr learned my addr 172.21.15.143:0/644514793 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:28:02.850 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:02.846+0000 7f49e1b9f700 1 --2- 172.21.15.143:0/644514793 >> v2:172.21.15.174:3300/0 conn(0x7f49dc06bd20 0x7f49dc0d01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:02.850 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:02.846+0000 7f49e139e700 1 -- 172.21.15.143:0/644514793 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 724030848 0 0) 0x7f49dc0cc2a0 con 0x7f49dc15a950 2023-11-08T19:28:02.850 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:02.846+0000 7f49e139e700 1 -- 172.21.15.143:0/644514793 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49c80035b0 con 0x7f49dc15a950 2023-11-08T19:28:02.851 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:02.846+0000 7f49e1b9f700 1 -- 172.21.15.143:0/644514793 >> v1:172.21.15.174:6789/0 conn(0x7f49dc0b61e0 legacy=0x7f49dc0cc730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:02.851 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:02.846+0000 7f49e1b9f700 1 -- 172.21.15.143:0/644514793 >> v1:172.21.15.143:6789/0 conn(0x7f49dc15a950 legacy=0x7f49dc072370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:02.851 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:02.846+0000 7f49e139e700 1 -- 172.21.15.143:0/644514793 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3057905301 0 0) 0x7f49c80035b0 con 0x7f49dc15a950 2023-11-08T19:28:02.851 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:02.846+0000 7f49e1b9f700 1 -- 172.21.15.143:0/644514793 --> v2:172.21.15.174:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49dc06c490 con 0x7f49dc06bd20 2023-11-08T19:28:02.851 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:02.847+0000 7f49e1b9f700 1 --2- 172.21.15.143:0/644514793 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f49dc06bd20 0x7f49dc0d01b0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f49cc009840 tx=0x7f49cc009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=984af81962a32ba1 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:02.852 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:02.847+0000 7f49e139e700 1 -- 172.21.15.143:0/644514793 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49cc011830 con 0x7f49dc06bd20 2023-11-08T19:28:02.852 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:02.847+0000 7f49e139e700 1 -- 172.21.15.143:0/644514793 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49cc011990 con 0x7f49dc06bd20 2023-11-08T19:28:02.852 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:02.847+0000 7f49e139e700 1 -- 172.21.15.143:0/644514793 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49cc011d20 con 0x7f49dc06bd20 2023-11-08T19:28:02.852 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:02.847+0000 7f49e4604700 1 -- 172.21.15.143:0/644514793 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f49dc06bd20 msgr2=0x7f49dc0d01b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:02.852 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:02.847+0000 7f49e4604700 1 --2- 172.21.15.143:0/644514793 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f49dc06bd20 0x7f49dc0d01b0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f49cc009840 tx=0x7f49cc009b50 comp rx=0 tx=0).stop 2023-11-08T19:28:02.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:02.847+0000 7f49e4604700 1 -- 172.21.15.143:0/644514793 shutdown_connections 2023-11-08T19:28:02.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:02.847+0000 7f49e4604700 1 --2- 172.21.15.143:0/644514793 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f49dc06bd20 0x7f49dc0d01b0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:02.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:02.847+0000 7f49e4604700 1 -- 172.21.15.143:0/644514793 >> 172.21.15.143:0/644514793 conn(0x7f49dc0cd2e0 msgr2=0x7f49dc0cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:02.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:02.847+0000 7f49e4604700 1 -- 172.21.15.143:0/644514793 shutdown_connections 2023-11-08T19:28:02.854 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:02.847+0000 7f49e4604700 1 -- 172.21.15.143:0/644514793 wait complete. 2023-11-08T19:28:02.857 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:02.849+0000 7f49e4604700 1 Processor -- start 2023-11-08T19:28:02.857 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:02.849+0000 7f49e4604700 1 -- start start 2023-11-08T19:28:02.857 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:02.849+0000 7f49e4604700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f49dc0b61e0 0x7f49dc148e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:02.857 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:02.849+0000 7f49e4604700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f49dc15a950 0x7f49dc1f3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:02.858 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:02.849+0000 7f49e4604700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f49dc15d630 0x7f49dc15dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:02.858 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:02.849+0000 7f49e4604700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49dc1f3ec0 con 0x7f49dc0b61e0 2023-11-08T19:28:02.858 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:02.849+0000 7f49e4604700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49dc0c54e0 con 0x7f49dc15d630 2023-11-08T19:28:02.858 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:02.849+0000 7f49e4604700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49dc0c57c0 con 0x7f49dc15a950 2023-11-08T19:28:02.859 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:02.849+0000 7f49e23a0700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f49dc0b61e0 0x7f49dc148e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:02.859 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:02.850+0000 7f49e23a0700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f49dc0b61e0 0x7f49dc148e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54520/0 (socket says 172.21.15.143:54520) 2023-11-08T19:28:02.859 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:02.850+0000 7f49e23a0700 1 -- 172.21.15.143:0/1197037841 learned_addr learned my addr 172.21.15.143:0/1197037841 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:28:02.859 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:02.850+0000 7f49e1b9f700 1 --2- 172.21.15.143:0/1197037841 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f49dc15a950 0x7f49dc1f3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:02.859 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:02.850+0000 7f49e2ba1700 1 --2- 172.21.15.143:0/1197037841 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f49dc15d630 0x7f49dc15dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:02.860 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:02.850+0000 7f49e23a0700 1 -- 172.21.15.143:0/1197037841 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f49dc15a950 msgr2=0x7f49dc1f3980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:02.860 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:02.850+0000 7f49e23a0700 1 --2- 172.21.15.143:0/1197037841 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f49dc15a950 0x7f49dc1f3980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:02.860 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:02.850+0000 7f49e23a0700 1 -- 172.21.15.143:0/1197037841 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f49dc15d630 msgr2=0x7f49dc15dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:02.860 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:02.850+0000 7f49e23a0700 1 --2- 172.21.15.143:0/1197037841 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f49dc15d630 0x7f49dc15dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:02.860 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:02.850+0000 7f49e23a0700 1 -- 172.21.15.143:0/1197037841 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49cc0094a0 con 0x7f49dc0b61e0 2023-11-08T19:28:02.860 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:02.850+0000 7f49e2ba1700 1 --2- 172.21.15.143:0/1197037841 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f49dc15d630 0x7f49dc15dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:28:02.861 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:02.850+0000 7f49e23a0700 1 --2- 172.21.15.143:0/1197037841 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f49dc0b61e0 0x7f49dc148e10 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f49d800c780 tx=0x7f49d800ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:02.861 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:02.850+0000 7f49d2ffd700 1 -- 172.21.15.143:0/1197037841 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d8005d70 con 0x7f49dc0b61e0 2023-11-08T19:28:02.861 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:02.850+0000 7f49d2ffd700 1 -- 172.21.15.143:0/1197037841 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49d80070d0 con 0x7f49dc0b61e0 2023-11-08T19:28:02.861 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:02.850+0000 7f49e4604700 1 -- 172.21.15.143:0/1197037841 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49dc0c5aa0 con 0x7f49dc0b61e0 2023-11-08T19:28:02.861 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:02.850+0000 7f49d2ffd700 1 -- 172.21.15.143:0/1197037841 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d8007480 con 0x7f49dc0b61e0 2023-11-08T19:28:02.862 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:02.850+0000 7f49e4604700 1 -- 172.21.15.143:0/1197037841 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49dc0c6060 con 0x7f49dc0b61e0 2023-11-08T19:28:02.862 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:02.851+0000 7f49e4604700 1 -- 172.21.15.143:0/1197037841 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49dc0d04e0 con 0x7f49dc0b61e0 2023-11-08T19:28:02.862 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:02.851+0000 7f49d2ffd700 1 -- 172.21.15.143:0/1197037841 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f49d80075e0 con 0x7f49dc0b61e0 2023-11-08T19:28:02.862 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:02.852+0000 7f49d2ffd700 1 --2- 172.21.15.143:0/1197037841 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f49c006af00 0x7f49c006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:02.862 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:02.852+0000 7f49d2ffd700 1 -- 172.21.15.143:0/1197037841 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f49d8088f50 con 0x7f49dc0b61e0 2023-11-08T19:28:02.863 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:02.852+0000 7f49e1b9f700 1 --2- 172.21.15.143:0/1197037841 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f49c006af00 0x7f49c006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:02.863 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:02.853+0000 7f49e1b9f700 1 --2- 172.21.15.143:0/1197037841 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f49c006af00 0x7f49c006d3c0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f49cc009fd0 tx=0x7f49cc01c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:02.863 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:02.857+0000 7f49d2ffd700 1 -- 172.21.15.143:0/1197037841 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f49d8053c40 con 0x7f49dc0b61e0 2023-11-08T19:28:03.091 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:03.090+0000 7f49e4604700 1 -- 172.21.15.143:0/1197037841 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f49dc04eee0 con 0x7f49dc0b61e0 2023-11-08T19:28:03.694 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:03.693+0000 7f49d2ffd700 1 -- 172.21.15.143:0/1197037841 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 15 and data pool 16 v48) v1 ==== 174+0+0 (secure 0 0 0) 0x7f49d80170c0 con 0x7f49dc0b61e0 2023-11-08T19:28:03.699 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:03.696+0000 7f49e4604700 1 -- 172.21.15.143:0/1197037841 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f49c006af00 msgr2=0x7f49c006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:03.699 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:03.696+0000 7f49e4604700 1 --2- 172.21.15.143:0/1197037841 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f49c006af00 0x7f49c006d3c0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f49cc009fd0 tx=0x7f49cc01c040 comp rx=0 tx=0).stop 2023-11-08T19:28:03.699 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:03.696+0000 7f49e4604700 1 -- 172.21.15.143:0/1197037841 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f49dc0b61e0 msgr2=0x7f49dc148e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:03.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:03.696+0000 7f49e4604700 1 --2- 172.21.15.143:0/1197037841 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f49dc0b61e0 0x7f49dc148e10 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f49d800c780 tx=0x7f49d800ca90 comp rx=0 tx=0).stop 2023-11-08T19:28:03.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:03.697+0000 7f49e4604700 1 -- 172.21.15.143:0/1197037841 shutdown_connections 2023-11-08T19:28:03.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:03.697+0000 7f49e4604700 1 --2- 172.21.15.143:0/1197037841 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f49c006af00 0x7f49c006d3c0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:03.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:03.697+0000 7f49e4604700 1 --2- 172.21.15.143:0/1197037841 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f49dc15d630 0x7f49dc15dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:03.701 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:03.697+0000 7f49e4604700 1 --2- 172.21.15.143:0/1197037841 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f49dc0b61e0 0x7f49dc148e10 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:03.701 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:03.697+0000 7f49e4604700 1 --2- 172.21.15.143:0/1197037841 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f49dc15a950 0x7f49dc1f3980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:03.701 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:03.697+0000 7f49e4604700 1 -- 172.21.15.143:0/1197037841 >> 172.21.15.143:0/1197037841 conn(0x7f49dc0cd2e0 msgr2=0x7f49dc0ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:03.701 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:03.697+0000 7f49e4604700 1 -- 172.21.15.143:0/1197037841 shutdown_connections 2023-11-08T19:28:03.701 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:03.698+0000 7f49e4604700 1 -- 172.21.15.143:0/1197037841 wait complete. 2023-11-08T19:28:03.703 INFO:teuthology.orchestra.run.smithi143.stderr:new fs with metadata pool 15 and data pool 16 2023-11-08T19:28:03.724 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:28:03.846 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:03.844+0000 7f8c8ed66700 1 Processor -- start 2023-11-08T19:28:03.846 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:03.844+0000 7f8c8ed66700 1 -- start start 2023-11-08T19:28:03.850 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:03.845+0000 7f8c8ed66700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8c8815dd00 0x7f8c880cca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:03.850 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:03.845+0000 7f8c8ed66700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f8c8806b320 0x7f8c880cb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:03.850 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:03.845+0000 7f8c8ed66700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c880ccfb0 con 0x7f8c881618f0 2023-11-08T19:28:03.850 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:03.845+0000 7f8c8ed66700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c880cad50 con 0x7f8c8806b320 2023-11-08T19:28:03.850 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:03.845+0000 7f8c8ed66700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c88161df0 con 0x7f8c8815dd00 2023-11-08T19:28:03.851 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:03.845+0000 7f8c87fff700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f8c881618f0 0x7f8c8806ac10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:50680/0 (socket says 172.21.15.143:50680) 2023-11-08T19:28:03.851 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:03.845+0000 7f8c87fff700 1 -- 172.21.15.143:0/444189763 learned_addr learned my addr 172.21.15.143:0/444189763 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:28:03.851 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:03.845+0000 7f8c877fe700 1 -- 172.21.15.143:0/444189763 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 339302142 0 0) 0x7f8c880ccfb0 con 0x7f8c881618f0 2023-11-08T19:28:03.851 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:03.845+0000 7f8c8d303700 1 --2- 172.21.15.143:0/444189763 >> v2:172.21.15.174:3300/0 conn(0x7f8c8806b320 0x7f8c880cb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:03.851 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:03.845+0000 7f8c8cb02700 1 --2- 172.21.15.143:0/444189763 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8c8815dd00 0x7f8c880cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:03.852 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:03.845+0000 7f8c877fe700 1 -- 172.21.15.143:0/444189763 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c780035b0 con 0x7f8c881618f0 2023-11-08T19:28:03.852 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:03.846+0000 7f8c877fe700 1 -- 172.21.15.143:0/444189763 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3934063987 0 0) 0x7f8c780035b0 con 0x7f8c881618f0 2023-11-08T19:28:03.852 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:03.846+0000 7f8c877fe700 1 -- 172.21.15.143:0/444189763 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8c8815dd00 msgr2=0x7f8c880cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:03.852 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:03.846+0000 7f8c877fe700 1 --2- 172.21.15.143:0/444189763 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8c8815dd00 0x7f8c880cca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:03.852 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:03.846+0000 7f8c877fe700 1 -- 172.21.15.143:0/444189763 >> v2:172.21.15.174:3300/0 conn(0x7f8c8806b320 msgr2=0x7f8c880cb1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:03.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:03.846+0000 7f8c877fe700 1 --2- 172.21.15.143:0/444189763 >> v2:172.21.15.174:3300/0 conn(0x7f8c8806b320 0x7f8c880cb1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:03.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:03.846+0000 7f8c877fe700 1 -- 172.21.15.143:0/444189763 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c881620d0 con 0x7f8c881618f0 2023-11-08T19:28:03.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:03.846+0000 7f8c877fe700 1 -- 172.21.15.143:0/444189763 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4147756303 0 0) 0x7f8c74001670 con 0x7f8c881618f0 2023-11-08T19:28:03.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:03.846+0000 7f8c877fe700 1 -- 172.21.15.143:0/444189763 >> v1:172.21.15.143:6789/0 conn(0x7f8c881618f0 legacy=0x7f8c8806ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:03.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:03.846+0000 7f8c877fe700 1 --2- 172.21.15.143:0/444189763 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8c8815dd00 0x7f8c78003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:03.854 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:03.846+0000 7f8c877fe700 1 --2- 172.21.15.143:0/444189763 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8c78003f80 0x7f8c78004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:03.854 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:03.846+0000 7f8c877fe700 1 --2- 172.21.15.143:0/444189763 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c78004d90 0x7f8c78005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:03.854 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:03.846+0000 7f8c877fe700 1 -- 172.21.15.143:0/444189763 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c88161df0 con 0x7f8c78003f80 2023-11-08T19:28:03.854 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:03.846+0000 7f8c877fe700 1 -- 172.21.15.143:0/444189763 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c7800ba10 con 0x7f8c78004d90 2023-11-08T19:28:03.854 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:03.846+0000 7f8c877fe700 1 -- 172.21.15.143:0/444189763 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c7800bb50 con 0x7f8c8815dd00 2023-11-08T19:28:03.855 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:03.846+0000 7f8c87fff700 1 --2- 172.21.15.143:0/444189763 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8c78003f80 0x7f8c78004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:03.855 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:03.846+0000 7f8c8cb02700 1 --2- 172.21.15.143:0/444189763 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8c8815dd00 0x7f8c78003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:03.855 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:03.846+0000 7f8c8d303700 1 --2- 172.21.15.143:0/444189763 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c78004d90 0x7f8c78005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:03.855 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:03.846+0000 7f8c87fff700 1 -- 172.21.15.143:0/444189763 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8c8815dd00 msgr2=0x7f8c78003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:03.855 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:03.846+0000 7f8c87fff700 1 --2- 172.21.15.143:0/444189763 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8c8815dd00 0x7f8c78003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:03.856 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:03.846+0000 7f8c87fff700 1 -- 172.21.15.143:0/444189763 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c78004d90 msgr2=0x7f8c78005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:03.856 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:03.846+0000 7f8c87fff700 1 --2- 172.21.15.143:0/444189763 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c78004d90 0x7f8c78005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:03.856 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:03.846+0000 7f8c87fff700 1 -- 172.21.15.143:0/444189763 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c780035b0 con 0x7f8c78003f80 2023-11-08T19:28:03.856 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:03.846+0000 7f8c8cb02700 1 --2- 172.21.15.143:0/444189763 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8c8815dd00 0x7f8c78003a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T19:28:03.856 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:03.846+0000 7f8c8d303700 1 --2- 172.21.15.143:0/444189763 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c78004d90 0x7f8c78005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T19:28:03.856 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:03.846+0000 7f8c87fff700 1 --2- 172.21.15.143:0/444189763 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8c78003f80 0x7f8c78004850 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f8c7400c370 tx=0x7f8c78005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44020b32b4536074 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:03.857 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:03.846+0000 7f8c877fe700 1 -- 172.21.15.143:0/444189763 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c74007c60 con 0x7f8c78003f80 2023-11-08T19:28:03.857 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:03.846+0000 7f8c877fe700 1 -- 172.21.15.143:0/444189763 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c7400ce50 con 0x7f8c78003f80 2023-11-08T19:28:03.857 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:03.847+0000 7f8c877fe700 1 -- 172.21.15.143:0/444189763 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c74005ba0 con 0x7f8c78003f80 2023-11-08T19:28:03.857 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:03.847+0000 7f8c8ed66700 1 -- 172.21.15.143:0/444189763 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8c78003f80 msgr2=0x7f8c78004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:03.857 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:03.847+0000 7f8c8ed66700 1 --2- 172.21.15.143:0/444189763 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8c78003f80 0x7f8c78004850 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f8c7400c370 tx=0x7f8c78005880 comp rx=0 tx=0).stop 2023-11-08T19:28:03.858 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:03.847+0000 7f8c8d303700 1 -- 172.21.15.143:0/444189763 reap_dead start 2023-11-08T19:28:03.858 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:03.847+0000 7f8c8ed66700 1 -- 172.21.15.143:0/444189763 shutdown_connections 2023-11-08T19:28:03.858 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:03.847+0000 7f8c8ed66700 1 -- 172.21.15.143:0/444189763 >> 172.21.15.143:0/444189763 conn(0x7f8c88099ab0 msgr2=0x7f8c880d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:03.858 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:03.847+0000 7f8c8ed66700 1 -- 172.21.15.143:0/444189763 shutdown_connections 2023-11-08T19:28:03.858 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:03.847+0000 7f8c8ed66700 1 -- 172.21.15.143:0/444189763 wait complete. 2023-11-08T19:28:03.859 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:03.848+0000 7f8c8ed66700 1 Processor -- start 2023-11-08T19:28:03.859 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:03.848+0000 7f8c8ed66700 1 -- start start 2023-11-08T19:28:03.861 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:03.849+0000 7f8c8ed66700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c78003f80 0x7f8c880c91f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:03.862 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:03.849+0000 7f8c8ed66700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8c880b2c80 0x7f8c88159b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:03.862 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:03.849+0000 7f8c8ed66700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8c880b1a00 0x7f8c880ae760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:03.862 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:03.849+0000 7f8c8ed66700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c880ca550 con 0x7f8c880b1a00 2023-11-08T19:28:03.862 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:03.849+0000 7f8c8ed66700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c880c81e0 con 0x7f8c78003f80 2023-11-08T19:28:03.862 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:03.849+0000 7f8c8ed66700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c880c8d60 con 0x7f8c880b2c80 2023-11-08T19:28:03.863 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:03.849+0000 7f8c8d303700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8c880b1a00 0x7f8c880ae760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:03.863 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:03.849+0000 7f8c8d303700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8c880b1a00 0x7f8c880ae760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54542/0 (socket says 172.21.15.143:54542) 2023-11-08T19:28:03.863 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:03.849+0000 7f8c8d303700 1 -- 172.21.15.143:0/1022502447 learned_addr learned my addr 172.21.15.143:0/1022502447 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:28:03.863 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:03.850+0000 7f8c87fff700 1 --2- 172.21.15.143:0/1022502447 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8c880b2c80 0x7f8c88159b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:03.863 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:03.850+0000 7f8c8cb02700 1 --2- 172.21.15.143:0/1022502447 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c78003f80 0x7f8c880c91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:03.864 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:03.850+0000 7f8c8d303700 1 -- 172.21.15.143:0/1022502447 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8c880b2c80 msgr2=0x7f8c88159b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:03.864 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:03.850+0000 7f8c8d303700 1 --2- 172.21.15.143:0/1022502447 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8c880b2c80 0x7f8c88159b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:03.864 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:03.850+0000 7f8c8d303700 1 -- 172.21.15.143:0/1022502447 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c78003f80 msgr2=0x7f8c880c91f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:03.864 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:03.850+0000 7f8c8d303700 1 --2- 172.21.15.143:0/1022502447 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c78003f80 0x7f8c880c91f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:03.864 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:03.850+0000 7f8c8d303700 1 -- 172.21.15.143:0/1022502447 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c880c8430 con 0x7f8c880b1a00 2023-11-08T19:28:03.864 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:03.850+0000 7f8c8cb02700 1 --2- 172.21.15.143:0/1022502447 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c78003f80 0x7f8c880c91f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:28:03.865 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:03.850+0000 7f8c8d303700 1 --2- 172.21.15.143:0/1022502447 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8c880b1a00 0x7f8c880ae760 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f8c88073130 tx=0x7f8c8000f490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:03.865 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:03.850+0000 7f8c857fa700 1 -- 172.21.15.143:0/1022502447 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c8001c070 con 0x7f8c880b1a00 2023-11-08T19:28:03.865 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:03.850+0000 7f8c857fa700 1 -- 172.21.15.143:0/1022502447 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c8000f040 con 0x7f8c880b1a00 2023-11-08T19:28:03.865 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:03.850+0000 7f8c8ed66700 1 -- 172.21.15.143:0/1022502447 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c880c8710 con 0x7f8c880b1a00 2023-11-08T19:28:03.865 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:03.850+0000 7f8c857fa700 1 -- 172.21.15.143:0/1022502447 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c8000e040 con 0x7f8c880b1a00 2023-11-08T19:28:03.866 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:03.850+0000 7f8c8ed66700 1 -- 172.21.15.143:0/1022502447 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c8815a0d0 con 0x7f8c880b1a00 2023-11-08T19:28:03.866 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:03.851+0000 7f8c8ed66700 1 -- 172.21.15.143:0/1022502447 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c8804eee0 con 0x7f8c880b1a00 2023-11-08T19:28:03.866 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:03.851+0000 7f8c857fa700 1 -- 172.21.15.143:0/1022502447 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8c80015430 con 0x7f8c880b1a00 2023-11-08T19:28:03.866 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:03.852+0000 7f8c857fa700 1 --2- 172.21.15.143:0/1022502447 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f8c6c06af00 0x7f8c6c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:03.866 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:03.852+0000 7f8c857fa700 1 -- 172.21.15.143:0/1022502447 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6772+0+0 (secure 0 0 0) 0x7f8c800895d0 con 0x7f8c880b1a00 2023-11-08T19:28:03.867 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:03.852+0000 7f8c8cb02700 1 --2- 172.21.15.143:0/1022502447 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f8c6c06af00 0x7f8c6c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:03.867 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:03.852+0000 7f8c8cb02700 1 --2- 172.21.15.143:0/1022502447 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f8c6c06af00 0x7f8c6c06d3c0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c009a60 tx=0x7f8c7c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:03.867 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:03.856+0000 7f8c857fa700 1 -- 172.21.15.143:0/1022502447 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8c80054250 con 0x7f8c880b1a00 2023-11-08T19:28:04.046 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.045+0000 7f8c8ed66700 1 -- 172.21.15.143:0/1022502447 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8c880cab50 con 0x7f8c880b1a00 2023-11-08T19:28:04.048 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.047+0000 7f8c857fa700 1 -- 172.21.15.143:0/1022502447 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+21805 (secure 0 0 0) 0x7f8c880cab50 con 0x7f8c880b1a00 2023-11-08T19:28:04.048 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:28:04.049 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":91,"fsid":"968483af-77fc-4068-ad7c-5c2a1a4613d7","created":"2023-11-08T19:24:32.826484+0000","modified":"2023-11-08T19:28:03.091905+0000","last_up_change":"2023-11-08T19:24:43.573880+0000","last_in_change":"2023-11-08T19:24:35.314525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:44.551530+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"snap-test-pool","create_time":"2023-11-08T19:26:02.874384+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":36,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"test_data_pool","create_time":"2023-11-08T19:27:05.601949+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":66,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_metadata_pool","create_time":"2023-11-08T19:27:06.536369+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-11-08T19:28:01.533566+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-11-08T19:28:02.053445+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bae98ec1-d94e-403d-a220-a522a53ea6f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6809","nonce":1487830566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6811","nonce":1487830566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6815","nonce":1487830566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6813","nonce":1487830566}]},"public_addr":"172.21.15.143:6809/1487830566","cluster_addr":"172.21.15.143:6811/1487830566","heartbeat_back_addr":"172.21.15.143:6815/1487830566","heartbeat_front_addr":"172.21.15.143:6813/1487830566","state":["exists","up"]},{"osd":1,"uuid":"5cb6538a-1ccb-4322-8633-09f66af0b7e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6801","nonce":787341397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6803","nonce":787341397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6807","nonce":787341397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6805","nonce":787341397}]},"public_addr":"172.21.15.143:6801/787341397","cluster_addr":"172.21.15.143:6803/787341397","heartbeat_back_addr":"172.21.15.143:6807/787341397","heartbeat_front_addr":"172.21.15.143:6805/787341397","state":["exists","up"]},{"osd":2,"uuid":"cdbf522c-60ef-4d60-9653-8dcb529c101a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6822","nonce":2301325597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6826","nonce":2301325597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6831","nonce":2301325597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6829","nonce":2301325597}]},"public_addr":"172.21.15.143:6822/2301325597","cluster_addr":"172.21.15.143:6826/2301325597","heartbeat_back_addr":"172.21.15.143:6831/2301325597","heartbeat_front_addr":"172.21.15.143:6829/2301325597","state":["exists","up"]},{"osd":3,"uuid":"4e2783cf-ebb9-4dd8-8245-bc1d17ce4325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6817","nonce":2021504428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6819","nonce":2021504428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6827","nonce":2021504428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6823","nonce":2021504428}]},"public_addr":"172.21.15.143:6817/2021504428","cluster_addr":"172.21.15.143:6819/2021504428","heartbeat_back_addr":"172.21.15.143:6827/2021504428","heartbeat_front_addr":"172.21.15.143:6823/2021504428","state":["exists","up"]},{"osd":4,"uuid":"691256fa-9b77-49b9-a346-7ae1adf0ca80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6801","nonce":1993380595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6803","nonce":1993380595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6807","nonce":1993380595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6805","nonce":1993380595}]},"public_addr":"172.21.15.174:6801/1993380595","cluster_addr":"172.21.15.174:6803/1993380595","heartbeat_back_addr":"172.21.15.174:6807/1993380595","heartbeat_front_addr":"172.21.15.174:6805/1993380595","state":["exists","up"]},{"osd":5,"uuid":"756fc352-48c9-46ce-b52e-0541fe6f8525","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6809","nonce":1777923303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6811","nonce":1777923303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6815","nonce":1777923303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6813","nonce":1777923303}]},"public_addr":"172.21.15.174:6809/1777923303","cluster_addr":"172.21.15.174:6811/1777923303","heartbeat_back_addr":"172.21.15.174:6815/1777923303","heartbeat_front_addr":"172.21.15.174:6813/1777923303","state":["exists","up"]},{"osd":6,"uuid":"fab2ac2b-ed97-4cde-9825-b480eabf8b8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6825","nonce":2755130316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6827","nonce":2755130316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6831","nonce":2755130316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6829","nonce":2755130316}]},"public_addr":"172.21.15.174:6825/2755130316","cluster_addr":"172.21.15.174:6827/2755130316","heartbeat_back_addr":"172.21.15.174:6831/2755130316","heartbeat_front_addr":"172.21.15.174:6829/2755130316","state":["exists","up"]},{"osd":7,"uuid":"3de9b797-2ab6-4d41-a863-7e7d638fd703","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6817","nonce":2847141411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6819","nonce":2847141411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6823","nonce":2847141411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6821","nonce":2847141411}]},"public_addr":"172.21.15.174:6817/2847141411","cluster_addr":"172.21.15.174:6819/2847141411","heartbeat_back_addr":"172.21.15.174:6823/2847141411","heartbeat_front_addr":"172.21.15.174:6821/2847141411","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.477076+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.496890+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.539305+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.454673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.538881+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.521294+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.520712+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.540404+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:28:04.050 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.049+0000 7f8c8ed66700 1 -- 172.21.15.143:0/1022502447 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f8c6c06af00 msgr2=0x7f8c6c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:04.051 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.049+0000 7f8c8ed66700 1 --2- 172.21.15.143:0/1022502447 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f8c6c06af00 0x7f8c6c06d3c0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c009a60 tx=0x7f8c7c008040 comp rx=0 tx=0).stop 2023-11-08T19:28:04.051 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.049+0000 7f8c8ed66700 1 -- 172.21.15.143:0/1022502447 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8c880b1a00 msgr2=0x7f8c880ae760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:04.051 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.049+0000 7f8c8ed66700 1 --2- 172.21.15.143:0/1022502447 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8c880b1a00 0x7f8c880ae760 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f8c88073130 tx=0x7f8c8000f490 comp rx=0 tx=0).stop 2023-11-08T19:28:04.051 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.049+0000 7f8c8ed66700 1 -- 172.21.15.143:0/1022502447 shutdown_connections 2023-11-08T19:28:04.051 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.049+0000 7f8c8ed66700 1 --2- 172.21.15.143:0/1022502447 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f8c6c06af00 0x7f8c6c06d3c0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:04.052 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.049+0000 7f8c8ed66700 1 --2- 172.21.15.143:0/1022502447 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8c880b1a00 0x7f8c880ae760 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:04.052 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.049+0000 7f8c8ed66700 1 --2- 172.21.15.143:0/1022502447 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c78003f80 0x7f8c880c91f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:04.052 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.049+0000 7f8c8ed66700 1 --2- 172.21.15.143:0/1022502447 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8c880b2c80 0x7f8c88159b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:04.052 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.049+0000 7f8c8ed66700 1 -- 172.21.15.143:0/1022502447 >> 172.21.15.143:0/1022502447 conn(0x7f8c88099ab0 msgr2=0x7f8c8806e790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:04.052 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.049+0000 7f8c8ed66700 1 -- 172.21.15.143:0/1022502447 shutdown_connections 2023-11-08T19:28:04.053 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.049+0000 7f8c8ed66700 1 -- 172.21.15.143:0/1022502447 wait complete. 2023-11-08T19:28:04.062 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:28:04.182 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.181+0000 7f7e58738700 1 Processor -- start 2023-11-08T19:28:04.183 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.181+0000 7f7e58738700 1 -- start start 2023-11-08T19:28:04.185 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.182+0000 7f7e58738700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f7e5015dd00 0x7f7e500cca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:04.185 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.182+0000 7f7e58738700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f7e5006b320 0x7f7e500cb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:04.185 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.182+0000 7f7e58738700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e500ccfb0 con 0x7f7e501618f0 2023-11-08T19:28:04.185 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.182+0000 7f7e58738700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e500cad50 con 0x7f7e5015dd00 2023-11-08T19:28:04.186 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.182+0000 7f7e58738700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e50161df0 con 0x7f7e5006b320 2023-11-08T19:28:04.186 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.182+0000 7f7e564d4700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f7e5015dd00 0x7f7e500cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:04.186 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.182+0000 7f7e564d4700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f7e5015dd00 0x7f7e500cca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54546/0 (socket says 172.21.15.143:54546) 2023-11-08T19:28:04.186 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.182+0000 7f7e564d4700 1 -- 172.21.15.143:0/3212051285 learned_addr learned my addr 172.21.15.143:0/3212051285 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:28:04.186 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.182+0000 7f7e56cd5700 1 --2- 172.21.15.143:0/3212051285 >> v2:172.21.15.174:3300/0 conn(0x7f7e5006b320 0x7f7e500cb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:04.187 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.182+0000 7f7e564d4700 1 -- 172.21.15.143:0/3212051285 >> v2:172.21.15.174:3300/0 conn(0x7f7e5006b320 msgr2=0x7f7e500cb1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:04.187 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.182+0000 7f7e564d4700 1 --2- 172.21.15.143:0/3212051285 >> v2:172.21.15.174:3300/0 conn(0x7f7e5006b320 0x7f7e500cb1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:04.187 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.182+0000 7f7e564d4700 1 -- 172.21.15.143:0/3212051285 >> v1:172.21.15.174:6789/0 conn(0x7f7e501618f0 legacy=0x7f7e5006ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:04.187 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.182+0000 7f7e564d4700 1 -- 172.21.15.143:0/3212051285 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e501620d0 con 0x7f7e5015dd00 2023-11-08T19:28:04.187 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.182+0000 7f7e564d4700 1 --2- 172.21.15.143:0/3212051285 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7e5015dd00 0x7f7e500cca70 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f7e500ce8d0 tx=0x7f7e40009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9804550428c44847 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:04.188 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.183+0000 7f7e554d2700 1 -- 172.21.15.143:0/3212051285 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e40005e30 con 0x7f7e5015dd00 2023-11-08T19:28:04.188 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.183+0000 7f7e554d2700 1 -- 172.21.15.143:0/3212051285 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e40004030 con 0x7f7e5015dd00 2023-11-08T19:28:04.188 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.183+0000 7f7e554d2700 1 -- 172.21.15.143:0/3212051285 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e400043e0 con 0x7f7e5015dd00 2023-11-08T19:28:04.188 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.183+0000 7f7e58738700 1 -- 172.21.15.143:0/3212051285 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7e5015dd00 msgr2=0x7f7e500cca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:04.188 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.183+0000 7f7e58738700 1 --2- 172.21.15.143:0/3212051285 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7e5015dd00 0x7f7e500cca70 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f7e500ce8d0 tx=0x7f7e40009bd0 comp rx=0 tx=0).stop 2023-11-08T19:28:04.189 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.183+0000 7f7e58738700 1 -- 172.21.15.143:0/3212051285 shutdown_connections 2023-11-08T19:28:04.189 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.183+0000 7f7e58738700 1 --2- 172.21.15.143:0/3212051285 >> v2:172.21.15.174:3300/0 conn(0x7f7e5006b320 0x7f7e500cb1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:04.189 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.183+0000 7f7e58738700 1 --2- 172.21.15.143:0/3212051285 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7e5015dd00 0x7f7e500cca70 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:04.189 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.183+0000 7f7e58738700 1 -- 172.21.15.143:0/3212051285 >> 172.21.15.143:0/3212051285 conn(0x7f7e50099ab0 msgr2=0x7f7e500d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:04.189 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.183+0000 7f7e58738700 1 -- 172.21.15.143:0/3212051285 shutdown_connections 2023-11-08T19:28:04.190 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.183+0000 7f7e58738700 1 -- 172.21.15.143:0/3212051285 wait complete. 2023-11-08T19:28:04.192 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.185+0000 7f7e58738700 1 Processor -- start 2023-11-08T19:28:04.192 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.185+0000 7f7e58738700 1 -- start start 2023-11-08T19:28:04.193 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.185+0000 7f7e58738700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7e5006b320 0x7f7e501f3430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:04.193 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.185+0000 7f7e58738700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7e5015dd00 0x7f7e500c61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:04.193 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.185+0000 7f7e58738700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7e501618f0 0x7f7e500c4230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:04.193 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.185+0000 7f7e58738700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e5015e380 con 0x7f7e5015dd00 2023-11-08T19:28:04.193 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.185+0000 7f7e58738700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e501f3970 con 0x7f7e501618f0 2023-11-08T19:28:04.194 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.185+0000 7f7e58738700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e500c6be0 con 0x7f7e5006b320 2023-11-08T19:28:04.194 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.185+0000 7f7e55cd3700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7e5015dd00 0x7f7e500c61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:04.194 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.185+0000 7f7e55cd3700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7e5015dd00 0x7f7e500c61e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54554/0 (socket says 172.21.15.143:54554) 2023-11-08T19:28:04.194 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.185+0000 7f7e55cd3700 1 -- 172.21.15.143:0/3878973000 learned_addr learned my addr 172.21.15.143:0/3878973000 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:28:04.194 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.185+0000 7f7e564d4700 1 --2- 172.21.15.143:0/3878973000 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7e5006b320 0x7f7e501f3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:04.195 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.185+0000 7f7e56cd5700 1 --2- 172.21.15.143:0/3878973000 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7e501618f0 0x7f7e500c4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:04.195 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.186+0000 7f7e55cd3700 1 -- 172.21.15.143:0/3878973000 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7e5006b320 msgr2=0x7f7e501f3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:04.195 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.186+0000 7f7e55cd3700 1 --2- 172.21.15.143:0/3878973000 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7e5006b320 0x7f7e501f3430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:04.195 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.186+0000 7f7e55cd3700 1 -- 172.21.15.143:0/3878973000 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7e501618f0 msgr2=0x7f7e500c4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:04.195 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.186+0000 7f7e55cd3700 1 --2- 172.21.15.143:0/3878973000 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7e501618f0 0x7f7e500c4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:04.195 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.186+0000 7f7e55cd3700 1 -- 172.21.15.143:0/3878973000 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e40009500 con 0x7f7e5015dd00 2023-11-08T19:28:04.196 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.186+0000 7f7e55cd3700 1 --2- 172.21.15.143:0/3878973000 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7e5015dd00 0x7f7e500c61e0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f7e4c00c450 tx=0x7f7e4c00c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:04.196 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.186+0000 7f7e46ffd700 1 -- 172.21.15.143:0/3878973000 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e4c0052d0 con 0x7f7e5015dd00 2023-11-08T19:28:04.196 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.186+0000 7f7e46ffd700 1 -- 172.21.15.143:0/3878973000 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e4c005430 con 0x7f7e5015dd00 2023-11-08T19:28:04.196 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.186+0000 7f7e58738700 1 -- 172.21.15.143:0/3878973000 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e500c6ec0 con 0x7f7e5015dd00 2023-11-08T19:28:04.196 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.186+0000 7f7e46ffd700 1 -- 172.21.15.143:0/3878973000 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e4c0038c0 con 0x7f7e5015dd00 2023-11-08T19:28:04.197 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.186+0000 7f7e58738700 1 -- 172.21.15.143:0/3878973000 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e5015e870 con 0x7f7e5015dd00 2023-11-08T19:28:04.197 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.187+0000 7f7e58738700 1 -- 172.21.15.143:0/3878973000 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e500aff60 con 0x7f7e5015dd00 2023-11-08T19:28:04.197 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.188+0000 7f7e46ffd700 1 -- 172.21.15.143:0/3878973000 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7e4c003a20 con 0x7f7e5015dd00 2023-11-08T19:28:04.197 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.188+0000 7f7e46ffd700 1 --2- 172.21.15.143:0/3878973000 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f7e3406af00 0x7f7e3406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:04.197 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.188+0000 7f7e46ffd700 1 -- 172.21.15.143:0/3878973000 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6772+0+0 (secure 0 0 0) 0x7f7e4c088cf0 con 0x7f7e5015dd00 2023-11-08T19:28:04.198 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.188+0000 7f7e564d4700 1 --2- 172.21.15.143:0/3878973000 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f7e3406af00 0x7f7e3406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:04.198 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.189+0000 7f7e564d4700 1 --2- 172.21.15.143:0/3878973000 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f7e3406af00 0x7f7e3406d3c0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f7e40009c10 tx=0x7f7e40008070 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:04.198 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.193+0000 7f7e46ffd700 1 -- 172.21.15.143:0/3878973000 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7e4c08d070 con 0x7f7e5015dd00 2023-11-08T19:28:04.385 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.384+0000 7f7e58738700 1 -- 172.21.15.143:0/3878973000 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7e5004eee0 con 0x7f7e5015dd00 2023-11-08T19:28:04.387 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.385+0000 7f7e46ffd700 1 -- 172.21.15.143:0/3878973000 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+21805 (secure 0 0 0) 0x7f7e4c017090 con 0x7f7e5015dd00 2023-11-08T19:28:04.387 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:28:04.387 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":91,"fsid":"968483af-77fc-4068-ad7c-5c2a1a4613d7","created":"2023-11-08T19:24:32.826484+0000","modified":"2023-11-08T19:28:03.091905+0000","last_up_change":"2023-11-08T19:24:43.573880+0000","last_in_change":"2023-11-08T19:24:35.314525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:44.551530+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"snap-test-pool","create_time":"2023-11-08T19:26:02.874384+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":36,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"test_data_pool","create_time":"2023-11-08T19:27:05.601949+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":66,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_metadata_pool","create_time":"2023-11-08T19:27:06.536369+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-11-08T19:28:01.533566+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-11-08T19:28:02.053445+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bae98ec1-d94e-403d-a220-a522a53ea6f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6809","nonce":1487830566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6811","nonce":1487830566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6815","nonce":1487830566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6813","nonce":1487830566}]},"public_addr":"172.21.15.143:6809/1487830566","cluster_addr":"172.21.15.143:6811/1487830566","heartbeat_back_addr":"172.21.15.143:6815/1487830566","heartbeat_front_addr":"172.21.15.143:6813/1487830566","state":["exists","up"]},{"osd":1,"uuid":"5cb6538a-1ccb-4322-8633-09f66af0b7e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6801","nonce":787341397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6803","nonce":787341397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6807","nonce":787341397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6805","nonce":787341397}]},"public_addr":"172.21.15.143:6801/787341397","cluster_addr":"172.21.15.143:6803/787341397","heartbeat_back_addr":"172.21.15.143:6807/787341397","heartbeat_front_addr":"172.21.15.143:6805/787341397","state":["exists","up"]},{"osd":2,"uuid":"cdbf522c-60ef-4d60-9653-8dcb529c101a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6822","nonce":2301325597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6826","nonce":2301325597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6831","nonce":2301325597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6829","nonce":2301325597}]},"public_addr":"172.21.15.143:6822/2301325597","cluster_addr":"172.21.15.143:6826/2301325597","heartbeat_back_addr":"172.21.15.143:6831/2301325597","heartbeat_front_addr":"172.21.15.143:6829/2301325597","state":["exists","up"]},{"osd":3,"uuid":"4e2783cf-ebb9-4dd8-8245-bc1d17ce4325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6817","nonce":2021504428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6819","nonce":2021504428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6827","nonce":2021504428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6823","nonce":2021504428}]},"public_addr":"172.21.15.143:6817/2021504428","cluster_addr":"172.21.15.143:6819/2021504428","heartbeat_back_addr":"172.21.15.143:6827/2021504428","heartbeat_front_addr":"172.21.15.143:6823/2021504428","state":["exists","up"]},{"osd":4,"uuid":"691256fa-9b77-49b9-a346-7ae1adf0ca80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6801","nonce":1993380595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6803","nonce":1993380595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6807","nonce":1993380595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6805","nonce":1993380595}]},"public_addr":"172.21.15.174:6801/1993380595","cluster_addr":"172.21.15.174:6803/1993380595","heartbeat_back_addr":"172.21.15.174:6807/1993380595","heartbeat_front_addr":"172.21.15.174:6805/1993380595","state":["exists","up"]},{"osd":5,"uuid":"756fc352-48c9-46ce-b52e-0541fe6f8525","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6809","nonce":1777923303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6811","nonce":1777923303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6815","nonce":1777923303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6813","nonce":1777923303}]},"public_addr":"172.21.15.174:6809/1777923303","cluster_addr":"172.21.15.174:6811/1777923303","heartbeat_back_addr":"172.21.15.174:6815/1777923303","heartbeat_front_addr":"172.21.15.174:6813/1777923303","state":["exists","up"]},{"osd":6,"uuid":"fab2ac2b-ed97-4cde-9825-b480eabf8b8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6825","nonce":2755130316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6827","nonce":2755130316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6831","nonce":2755130316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6829","nonce":2755130316}]},"public_addr":"172.21.15.174:6825/2755130316","cluster_addr":"172.21.15.174:6827/2755130316","heartbeat_back_addr":"172.21.15.174:6831/2755130316","heartbeat_front_addr":"172.21.15.174:6829/2755130316","state":["exists","up"]},{"osd":7,"uuid":"3de9b797-2ab6-4d41-a863-7e7d638fd703","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6817","nonce":2847141411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6819","nonce":2847141411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6823","nonce":2847141411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6821","nonce":2847141411}]},"public_addr":"172.21.15.174:6817/2847141411","cluster_addr":"172.21.15.174:6819/2847141411","heartbeat_back_addr":"172.21.15.174:6823/2847141411","heartbeat_front_addr":"172.21.15.174:6821/2847141411","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.477076+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.496890+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.539305+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.454673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.538881+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.521294+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.520712+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.540404+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:28:04.389 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.387+0000 7f7e58738700 1 -- 172.21.15.143:0/3878973000 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f7e3406af00 msgr2=0x7f7e3406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:04.389 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.387+0000 7f7e58738700 1 --2- 172.21.15.143:0/3878973000 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f7e3406af00 0x7f7e3406d3c0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f7e40009c10 tx=0x7f7e40008070 comp rx=0 tx=0).stop 2023-11-08T19:28:04.389 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.387+0000 7f7e58738700 1 -- 172.21.15.143:0/3878973000 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7e5015dd00 msgr2=0x7f7e500c61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:04.390 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.387+0000 7f7e58738700 1 --2- 172.21.15.143:0/3878973000 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7e5015dd00 0x7f7e500c61e0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f7e4c00c450 tx=0x7f7e4c00c760 comp rx=0 tx=0).stop 2023-11-08T19:28:04.390 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.387+0000 7f7e58738700 1 -- 172.21.15.143:0/3878973000 shutdown_connections 2023-11-08T19:28:04.390 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.388+0000 7f7e58738700 1 --2- 172.21.15.143:0/3878973000 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f7e3406af00 0x7f7e3406d3c0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:04.390 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.388+0000 7f7e58738700 1 --2- 172.21.15.143:0/3878973000 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7e5006b320 0x7f7e501f3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:04.391 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.388+0000 7f7e58738700 1 --2- 172.21.15.143:0/3878973000 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7e501618f0 0x7f7e500c4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:04.391 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.388+0000 7f7e58738700 1 --2- 172.21.15.143:0/3878973000 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7e5015dd00 0x7f7e500c61e0 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:04.391 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.388+0000 7f7e58738700 1 -- 172.21.15.143:0/3878973000 >> 172.21.15.143:0/3878973000 conn(0x7f7e50099ab0 msgr2=0x7f7e5006f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:04.391 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.388+0000 7f7e58738700 1 -- 172.21.15.143:0/3878973000 shutdown_connections 2023-11-08T19:28:04.391 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.388+0000 7f7e58738700 1 -- 172.21.15.143:0/3878973000 wait complete. 2023-11-08T19:28:04.402 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-08T19:28:04.525 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.524+0000 7f2f4c889700 1 Processor -- start 2023-11-08T19:28:04.526 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.524+0000 7f2f4c889700 1 -- start start 2023-11-08T19:28:04.528 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.525+0000 7f2f4c889700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f2f4815ba10 0x7f2f48072bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:04.528 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.525+0000 7f2f4c889700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f4815f6a0 con 0x7f2f480b2300 2023-11-08T19:28:04.528 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.525+0000 7f2f4c889700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f4806b320 con 0x7f2f4815bdf0 2023-11-08T19:28:04.529 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.525+0000 7f2f4c889700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f48073110 con 0x7f2f4815ba10 2023-11-08T19:28:04.529 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.525+0000 7f2f4659c700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f2f4815ba10 0x7f2f48072bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:04.529 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.525+0000 7f2f46d9d700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f2f480b2300 0x7f2f4815fb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:50718/0 (socket says 172.21.15.143:50718) 2023-11-08T19:28:04.529 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.525+0000 7f2f46d9d700 1 -- 172.21.15.143:0/4254714328 learned_addr learned my addr 172.21.15.143:0/4254714328 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:28:04.529 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.525+0000 7f2f4659c700 1 --2- 172.21.15.143:0/4254714328 >> v2:172.21.15.143:3300/0 conn(0x7f2f4815ba10 0x7f2f48072bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54560/0 (socket says 172.21.15.143:54560) 2023-11-08T19:28:04.530 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.525+0000 7f2f4659c700 1 -- 172.21.15.143:0/4254714328 >> v1:172.21.15.174:6789/0 conn(0x7f2f4815bdf0 legacy=0x7f2f4806ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:04.530 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.525+0000 7f2f4659c700 1 -- 172.21.15.143:0/4254714328 >> v1:172.21.15.143:6789/0 conn(0x7f2f480b2300 legacy=0x7f2f4815fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:04.530 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.525+0000 7f2f4659c700 1 -- 172.21.15.143:0/4254714328 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f4815c640 con 0x7f2f4815ba10 2023-11-08T19:28:04.530 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.525+0000 7f2f4559a700 1 -- 172.21.15.143:0/4254714328 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4133999400 0 0) 0x7f2f4815f6a0 con 0x7f2f480b2300 2023-11-08T19:28:04.530 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.525+0000 7f2f4659c700 1 --2- 172.21.15.143:0/4254714328 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2f4815ba10 0x7f2f48072bd0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f2f38009840 tx=0x7f2f38009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8d78c0a7afb4973 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:04.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.525+0000 7f2f4559a700 1 -- 172.21.15.143:0/4254714328 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f38004230 con 0x7f2f4815ba10 2023-11-08T19:28:04.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.526+0000 7f2f4559a700 1 -- 172.21.15.143:0/4254714328 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f38004390 con 0x7f2f4815ba10 2023-11-08T19:28:04.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.526+0000 7f2f4559a700 1 -- 172.21.15.143:0/4254714328 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f380036a0 con 0x7f2f4815ba10 2023-11-08T19:28:04.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.526+0000 7f2f4c889700 1 -- 172.21.15.143:0/4254714328 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2f4815ba10 msgr2=0x7f2f48072bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:04.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.526+0000 7f2f4c889700 1 --2- 172.21.15.143:0/4254714328 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2f4815ba10 0x7f2f48072bd0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f2f38009840 tx=0x7f2f38009b50 comp rx=0 tx=0).stop 2023-11-08T19:28:04.532 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.526+0000 7f2f4c889700 1 -- 172.21.15.143:0/4254714328 shutdown_connections 2023-11-08T19:28:04.532 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.526+0000 7f2f4c889700 1 --2- 172.21.15.143:0/4254714328 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2f4815ba10 0x7f2f48072bd0 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:04.532 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.526+0000 7f2f4c889700 1 -- 172.21.15.143:0/4254714328 >> 172.21.15.143:0/4254714328 conn(0x7f2f480cd300 msgr2=0x7f2f480cd710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:04.532 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.526+0000 7f2f4c889700 1 -- 172.21.15.143:0/4254714328 shutdown_connections 2023-11-08T19:28:04.532 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.526+0000 7f2f4c889700 1 -- 172.21.15.143:0/4254714328 wait complete. 2023-11-08T19:28:04.535 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.527+0000 7f2f4c889700 1 Processor -- start 2023-11-08T19:28:04.535 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.528+0000 7f2f4c889700 1 -- start start 2023-11-08T19:28:04.535 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.528+0000 7f2f4c889700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2f480b2300 0x7f2f480cfc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:04.536 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.528+0000 7f2f4c889700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2f4815ba10 0x7f2f481e63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:04.536 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.528+0000 7f2f4c889700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2f4815bdf0 0x7f2f480af080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:04.536 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.528+0000 7f2f4c889700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f481e68f0 con 0x7f2f4815bdf0 2023-11-08T19:28:04.536 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.528+0000 7f2f4c889700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f480ce450 con 0x7f2f4815ba10 2023-11-08T19:28:04.536 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.528+0000 7f2f4c889700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f480af5c0 con 0x7f2f480b2300 2023-11-08T19:28:04.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.528+0000 7f2f46d9d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2f4815bdf0 0x7f2f480af080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:04.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.528+0000 7f2f46d9d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2f4815bdf0 0x7f2f480af080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54570/0 (socket says 172.21.15.143:54570) 2023-11-08T19:28:04.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.528+0000 7f2f46d9d700 1 -- 172.21.15.143:0/2935668187 learned_addr learned my addr 172.21.15.143:0/2935668187 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:28:04.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.528+0000 7f2f4659c700 1 --2- 172.21.15.143:0/2935668187 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2f480b2300 0x7f2f480cfc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:04.538 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.528+0000 7f2f45d9b700 1 --2- 172.21.15.143:0/2935668187 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2f4815ba10 0x7f2f481e63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:04.538 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.529+0000 7f2f46d9d700 1 -- 172.21.15.143:0/2935668187 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2f480b2300 msgr2=0x7f2f480cfc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:04.538 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.529+0000 7f2f46d9d700 1 --2- 172.21.15.143:0/2935668187 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2f480b2300 0x7f2f480cfc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:04.538 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.529+0000 7f2f46d9d700 1 -- 172.21.15.143:0/2935668187 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2f4815ba10 msgr2=0x7f2f481e63b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:04.538 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.529+0000 7f2f46d9d700 1 --2- 172.21.15.143:0/2935668187 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2f4815ba10 0x7f2f481e63b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:04.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.529+0000 7f2f46d9d700 1 -- 172.21.15.143:0/2935668187 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f380094a0 con 0x7f2f4815bdf0 2023-11-08T19:28:04.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.529+0000 7f2f46d9d700 1 --2- 172.21.15.143:0/2935668187 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2f4815bdf0 0x7f2f480af080 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f2f4000cb90 tx=0x7f2f4000cea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:04.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.529+0000 7f2f36ffd700 1 -- 172.21.15.143:0/2935668187 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f4000aa60 con 0x7f2f4815bdf0 2023-11-08T19:28:04.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.529+0000 7f2f36ffd700 1 -- 172.21.15.143:0/2935668187 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f4000abc0 con 0x7f2f4815bdf0 2023-11-08T19:28:04.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.529+0000 7f2f4c889700 1 -- 172.21.15.143:0/2935668187 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f480d02e0 con 0x7f2f4815bdf0 2023-11-08T19:28:04.540 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.529+0000 7f2f36ffd700 1 -- 172.21.15.143:0/2935668187 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f400070d0 con 0x7f2f4815bdf0 2023-11-08T19:28:04.540 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.529+0000 7f2f4c889700 1 -- 172.21.15.143:0/2935668187 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f480ce980 con 0x7f2f4815bdf0 2023-11-08T19:28:04.540 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.530+0000 7f2f4c889700 1 -- 172.21.15.143:0/2935668187 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f4804eee0 con 0x7f2f4815bdf0 2023-11-08T19:28:04.540 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.530+0000 7f2f36ffd700 1 -- 172.21.15.143:0/2935668187 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2f40007230 con 0x7f2f4815bdf0 2023-11-08T19:28:04.540 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.531+0000 7f2f36ffd700 1 --2- 172.21.15.143:0/2935668187 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f2f2806af00 0x7f2f2806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:04.541 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.531+0000 7f2f36ffd700 1 -- 172.21.15.143:0/2935668187 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6772+0+0 (secure 0 0 0) 0x7f2f40089620 con 0x7f2f4815bdf0 2023-11-08T19:28:04.541 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.531+0000 7f2f4659c700 1 --2- 172.21.15.143:0/2935668187 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f2f2806af00 0x7f2f2806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:04.541 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.531+0000 7f2f4659c700 1 --2- 172.21.15.143:0/2935668187 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f2f2806af00 0x7f2f2806d3c0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f2f380090b0 tx=0x7f2f3801c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:04.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.535+0000 7f2f36ffd700 1 -- 172.21.15.143:0/2935668187 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2f40054330 con 0x7f2f4815bdf0 2023-11-08T19:28:04.776 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:04.775+0000 7f2f4c889700 1 -- 172.21.15.143:0/2935668187 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f2f480afaa0 con 0x7f2f4815bdf0 2023-11-08T19:28:05.729 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:05.727+0000 7f2f36ffd700 1 -- 172.21.15.143:0/2935668187 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v51) v1 ==== 118+0+0 (secure 0 0 0) 0x7f2f40057950 con 0x7f2f4815bdf0 2023-11-08T19:28:05.731 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:05.730+0000 7f2f4c889700 1 -- 172.21.15.143:0/2935668187 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f2f2806af00 msgr2=0x7f2f2806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:05.732 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:05.730+0000 7f2f4c889700 1 --2- 172.21.15.143:0/2935668187 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f2f2806af00 0x7f2f2806d3c0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f2f380090b0 tx=0x7f2f3801c040 comp rx=0 tx=0).stop 2023-11-08T19:28:05.732 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:05.730+0000 7f2f4c889700 1 -- 172.21.15.143:0/2935668187 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2f4815bdf0 msgr2=0x7f2f480af080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:05.732 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:05.730+0000 7f2f4c889700 1 --2- 172.21.15.143:0/2935668187 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2f4815bdf0 0x7f2f480af080 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f2f4000cb90 tx=0x7f2f4000cea0 comp rx=0 tx=0).stop 2023-11-08T19:28:05.732 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:05.730+0000 7f2f4c889700 1 -- 172.21.15.143:0/2935668187 shutdown_connections 2023-11-08T19:28:05.733 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:05.730+0000 7f2f4c889700 1 --2- 172.21.15.143:0/2935668187 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f2f2806af00 0x7f2f2806d3c0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:05.733 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:05.730+0000 7f2f4c889700 1 --2- 172.21.15.143:0/2935668187 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2f480b2300 0x7f2f480cfc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:05.733 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:05.730+0000 7f2f4c889700 1 --2- 172.21.15.143:0/2935668187 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2f4815bdf0 0x7f2f480af080 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:05.733 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:05.730+0000 7f2f4c889700 1 --2- 172.21.15.143:0/2935668187 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2f4815ba10 0x7f2f481e63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:05.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:05.730+0000 7f2f4c889700 1 -- 172.21.15.143:0/2935668187 >> 172.21.15.143:0/2935668187 conn(0x7f2f480cd300 msgr2=0x7f2f48153f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:05.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:05.730+0000 7f2f4c889700 1 -- 172.21.15.143:0/2935668187 shutdown_connections 2023-11-08T19:28:05.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:05.730+0000 7f2f4c889700 1 -- 172.21.15.143:0/2935668187 wait complete. 2023-11-08T19:28:05.746 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-08T19:28:05.747 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-08T19:28:05.877 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:05.876+0000 7fd9ba345700 1 Processor -- start 2023-11-08T19:28:05.878 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:05.876+0000 7fd9ba345700 1 -- start start 2023-11-08T19:28:05.880 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:05.877+0000 7fd9ba345700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd9b415dba0 0x7fd9b406c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:05.880 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:05.877+0000 7fd9ba345700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd9b406b240 0x7fd9b40caff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:05.881 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:05.877+0000 7fd9ba345700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fd9b40cb530 0x7fd9b40b22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:05.881 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:05.877+0000 7fd9ba345700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9b4161890 con 0x7fd9b406b240 2023-11-08T19:28:05.881 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:05.877+0000 7fd9ba345700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9b406b740 con 0x7fd9b40cb530 2023-11-08T19:28:05.881 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:05.877+0000 7fd9ba345700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b406ba20 con 0x7fd9b415dba0 2023-11-08T19:28:05.881 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:05.877+0000 7fd9b37fe700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd9b406b240 0x7fd9b40caff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:05.882 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:05.877+0000 7fd9b37fe700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd9b406b240 0x7fd9b40caff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54574/0 (socket says 172.21.15.143:54574) 2023-11-08T19:28:05.882 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:05.877+0000 7fd9b37fe700 1 -- 172.21.15.143:0/4058960308 learned_addr learned my addr 172.21.15.143:0/4058960308 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:28:05.882 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:05.877+0000 7fd9b88e2700 1 --2- 172.21.15.143:0/4058960308 >> v2:172.21.15.174:3300/0 conn(0x7fd9b40cb530 0x7fd9b40b22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:05.882 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:05.877+0000 7fd9b3fff700 1 --2- 172.21.15.143:0/4058960308 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd9b415dba0 0x7fd9b406c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:05.882 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:05.877+0000 7fd9b37fe700 1 -- 172.21.15.143:0/4058960308 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd9b415dba0 msgr2=0x7fd9b406c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:05.883 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:05.877+0000 7fd9b37fe700 1 --2- 172.21.15.143:0/4058960308 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd9b415dba0 0x7fd9b406c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:05.883 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:05.877+0000 7fd9b37fe700 1 -- 172.21.15.143:0/4058960308 >> v2:172.21.15.174:3300/0 conn(0x7fd9b40cb530 msgr2=0x7fd9b40b22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:05.883 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:05.877+0000 7fd9b37fe700 1 --2- 172.21.15.143:0/4058960308 >> v2:172.21.15.174:3300/0 conn(0x7fd9b40cb530 0x7fd9b40b22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:05.883 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:05.877+0000 7fd9b37fe700 1 -- 172.21.15.143:0/4058960308 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9b406bd00 con 0x7fd9b406b240 2023-11-08T19:28:05.883 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:05.877+0000 7fd9b37fe700 1 --2- 172.21.15.143:0/4058960308 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd9b406b240 0x7fd9b40caff0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fd9b406f460 tx=0x7fd9a4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=904cfe411c5f2159 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:05.884 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:05.878+0000 7fd9b2ffd700 1 -- 172.21.15.143:0/4058960308 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a4011830 con 0x7fd9b406b240 2023-11-08T19:28:05.884 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:05.878+0000 7fd9b2ffd700 1 -- 172.21.15.143:0/4058960308 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9a4011990 con 0x7fd9b406b240 2023-11-08T19:28:05.884 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:05.878+0000 7fd9b2ffd700 1 -- 172.21.15.143:0/4058960308 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a4011d20 con 0x7fd9b406b240 2023-11-08T19:28:05.884 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:05.878+0000 7fd9ba345700 1 -- 172.21.15.143:0/4058960308 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd9b406b240 msgr2=0x7fd9b40caff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:05.884 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:05.878+0000 7fd9ba345700 1 --2- 172.21.15.143:0/4058960308 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd9b406b240 0x7fd9b40caff0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fd9b406f460 tx=0x7fd9a4009bd0 comp rx=0 tx=0).stop 2023-11-08T19:28:05.885 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:05.878+0000 7fd9ba345700 1 -- 172.21.15.143:0/4058960308 shutdown_connections 2023-11-08T19:28:05.885 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:05.878+0000 7fd9ba345700 1 --2- 172.21.15.143:0/4058960308 >> v2:172.21.15.174:3300/0 conn(0x7fd9b40cb530 0x7fd9b40b22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:05.885 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:05.878+0000 7fd9ba345700 1 --2- 172.21.15.143:0/4058960308 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd9b415dba0 0x7fd9b406c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:05.885 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:05.878+0000 7fd9ba345700 1 --2- 172.21.15.143:0/4058960308 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd9b406b240 0x7fd9b40caff0 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:05.886 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:05.878+0000 7fd9ba345700 1 -- 172.21.15.143:0/4058960308 >> 172.21.15.143:0/4058960308 conn(0x7fd9b40cd2e0 msgr2=0x7fd9b40cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:05.886 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:05.878+0000 7fd9ba345700 1 -- 172.21.15.143:0/4058960308 shutdown_connections 2023-11-08T19:28:05.886 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:05.878+0000 7fd9ba345700 1 -- 172.21.15.143:0/4058960308 wait complete. 2023-11-08T19:28:05.889 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:05.880+0000 7fd9ba345700 1 Processor -- start 2023-11-08T19:28:05.889 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:05.880+0000 7fd9ba345700 1 -- start start 2023-11-08T19:28:05.889 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:05.881+0000 7fd9ba345700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd9b406b240 0x7fd9b415b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:05.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:05.881+0000 7fd9ba345700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd9b40cb530 0x7fd9b41e7420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:05.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:05.881+0000 7fd9ba345700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd9b415dba0 0x7fd9b40ca360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:05.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:05.881+0000 7fd9ba345700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b40adfb0 con 0x7fd9b406b240 2023-11-08T19:28:05.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:05.881+0000 7fd9ba345700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b415b6e0 con 0x7fd9b40cb530 2023-11-08T19:28:05.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:05.881+0000 7fd9ba345700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b41e7c70 con 0x7fd9b415dba0 2023-11-08T19:28:05.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:05.881+0000 7fd9b3fff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd9b406b240 0x7fd9b415b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:05.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:05.881+0000 7fd9b3fff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd9b406b240 0x7fd9b415b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54580/0 (socket says 172.21.15.143:54580) 2023-11-08T19:28:05.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:05.881+0000 7fd9b3fff700 1 -- 172.21.15.143:0/1652616179 learned_addr learned my addr 172.21.15.143:0/1652616179 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:28:05.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:05.881+0000 7fd9b37fe700 1 --2- 172.21.15.143:0/1652616179 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd9b40cb530 0x7fd9b41e7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:05.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:05.881+0000 7fd9b88e2700 1 --2- 172.21.15.143:0/1652616179 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd9b415dba0 0x7fd9b40ca360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:05.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:05.881+0000 7fd9b3fff700 1 -- 172.21.15.143:0/1652616179 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd9b415dba0 msgr2=0x7fd9b40ca360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:05.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:05.881+0000 7fd9b3fff700 1 --2- 172.21.15.143:0/1652616179 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd9b415dba0 0x7fd9b40ca360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:05.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:05.881+0000 7fd9b3fff700 1 -- 172.21.15.143:0/1652616179 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd9b40cb530 msgr2=0x7fd9b41e7420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:05.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:05.881+0000 7fd9b3fff700 1 --2- 172.21.15.143:0/1652616179 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd9b40cb530 0x7fd9b41e7420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:05.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:05.881+0000 7fd9b3fff700 1 -- 172.21.15.143:0/1652616179 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9a4009550 con 0x7fd9b406b240 2023-11-08T19:28:05.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:05.881+0000 7fd9b88e2700 1 --2- 172.21.15.143:0/1652616179 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd9b415dba0 0x7fd9b40ca360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:28:05.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:05.881+0000 7fd9b3fff700 1 --2- 172.21.15.143:0/1652616179 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd9b406b240 0x7fd9b415b1a0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fd9a800e870 tx=0x7fd9a800eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:05.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:05.881+0000 7fd9b0ff9700 1 -- 172.21.15.143:0/1652616179 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a8009590 con 0x7fd9b406b240 2023-11-08T19:28:05.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:05.881+0000 7fd9b0ff9700 1 -- 172.21.15.143:0/1652616179 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9a8013680 con 0x7fd9b406b240 2023-11-08T19:28:05.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:05.882+0000 7fd9b0ff9700 1 -- 172.21.15.143:0/1652616179 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a80139f0 con 0x7fd9b406b240 2023-11-08T19:28:05.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:05.882+0000 7fd9ba345700 1 -- 172.21.15.143:0/1652616179 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9b41e7f50 con 0x7fd9b406b240 2023-11-08T19:28:05.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:05.882+0000 7fd9ba345700 1 -- 172.21.15.143:0/1652616179 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9b40ae4e0 con 0x7fd9b406b240 2023-11-08T19:28:05.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:05.883+0000 7fd9ba345700 1 -- 172.21.15.143:0/1652616179 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9b404eee0 con 0x7fd9b406b240 2023-11-08T19:28:05.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:05.883+0000 7fd9b0ff9700 1 -- 172.21.15.143:0/1652616179 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd9a8013b50 con 0x7fd9b406b240 2023-11-08T19:28:05.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:05.883+0000 7fd9b0ff9700 1 --2- 172.21.15.143:0/1652616179 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd9a006af00 0x7fd9a006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:05.895 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:05.883+0000 7fd9b0ff9700 1 -- 172.21.15.143:0/1652616179 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6772+0+0 (secure 0 0 0) 0x7fd9a8089900 con 0x7fd9b406b240 2023-11-08T19:28:05.895 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:05.884+0000 7fd9b37fe700 1 --2- 172.21.15.143:0/1652616179 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd9a006af00 0x7fd9a006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:05.895 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:05.884+0000 7fd9b37fe700 1 --2- 172.21.15.143:0/1652616179 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd9a006af00 0x7fd9a006d3c0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fd9a4009c10 tx=0x7fd9a4008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:05.895 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:05.888+0000 7fd9b0ff9700 1 -- 172.21.15.143:0/1652616179 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd9a8054580 con 0x7fd9b406b240 2023-11-08T19:28:06.121 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:06.119+0000 7fd9ba345700 1 -- 172.21.15.143:0/1652616179 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fd9b40b67e0 con 0x7fd9b406b240 2023-11-08T19:28:06.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:06.733+0000 7fd9b0ff9700 1 -- 172.21.15.143:0/1652616179 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v52) v1 ==== 122+0+0 (secure 0 0 0) 0x7fd9a8057ba0 con 0x7fd9b406b240 2023-11-08T19:28:06.737 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:06.736+0000 7fd9ba345700 1 -- 172.21.15.143:0/1652616179 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd9a006af00 msgr2=0x7fd9a006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:06.738 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:06.736+0000 7fd9ba345700 1 --2- 172.21.15.143:0/1652616179 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd9a006af00 0x7fd9a006d3c0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fd9a4009c10 tx=0x7fd9a4008030 comp rx=0 tx=0).stop 2023-11-08T19:28:06.738 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:06.736+0000 7fd9ba345700 1 -- 172.21.15.143:0/1652616179 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd9b406b240 msgr2=0x7fd9b415b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:06.738 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:06.736+0000 7fd9ba345700 1 --2- 172.21.15.143:0/1652616179 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd9b406b240 0x7fd9b415b1a0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fd9a800e870 tx=0x7fd9a800eb80 comp rx=0 tx=0).stop 2023-11-08T19:28:06.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:06.736+0000 7fd9ba345700 1 -- 172.21.15.143:0/1652616179 shutdown_connections 2023-11-08T19:28:06.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:06.736+0000 7fd9ba345700 1 --2- 172.21.15.143:0/1652616179 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd9a006af00 0x7fd9a006d3c0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:06.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:06.736+0000 7fd9ba345700 1 --2- 172.21.15.143:0/1652616179 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd9b415dba0 0x7fd9b40ca360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:06.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:06.736+0000 7fd9ba345700 1 --2- 172.21.15.143:0/1652616179 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd9b406b240 0x7fd9b415b1a0 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:06.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:06.736+0000 7fd9ba345700 1 --2- 172.21.15.143:0/1652616179 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd9b40cb530 0x7fd9b41e7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:06.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:06.736+0000 7fd9ba345700 1 -- 172.21.15.143:0/1652616179 >> 172.21.15.143:0/1652616179 conn(0x7fd9b40cd2e0 msgr2=0x7fd9b40b1e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:06.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:06.736+0000 7fd9ba345700 1 -- 172.21.15.143:0/1652616179 shutdown_connections 2023-11-08T19:28:06.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:06.736+0000 7fd9ba345700 1 -- 172.21.15.143:0/1652616179 wait complete. 2023-11-08T19:28:06.751 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-08T19:28:06.752 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:28:06.875 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:06.874+0000 7f7f30bb4700 1 Processor -- start 2023-11-08T19:28:06.876 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:06.874+0000 7f7f30bb4700 1 -- start start 2023-11-08T19:28:06.878 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:06.875+0000 7f7f30bb4700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f7f2c15aa30 0x7f7f2c0cf510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:06.878 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:06.875+0000 7f7f30bb4700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7f2c0b2210 0x7f7f2c0b2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:06.878 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:06.875+0000 7f7f30bb4700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f2c0b2bb0 con 0x7f7f2c0cbe20 2023-11-08T19:28:06.878 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:06.875+0000 7f7f30bb4700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f2c0cf080 con 0x7f7f2c15aa30 2023-11-08T19:28:06.879 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:06.875+0000 7f7f30bb4700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f2c0ce470 con 0x7f7f2c0b2210 2023-11-08T19:28:06.879 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:06.875+0000 7f7f2a59c700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f7f2c15aa30 0x7f7f2c0cf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:06.879 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:06.875+0000 7f7f2ad9d700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f7f2c0cbe20 0x7f7f2c0cc270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:50744/0 (socket says 172.21.15.143:50744) 2023-11-08T19:28:06.879 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:06.875+0000 7f7f2a59c700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f7f2c15aa30 0x7f7f2c0cf510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54588/0 (socket says 172.21.15.143:54588) 2023-11-08T19:28:06.879 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:06.875+0000 7f7f2ad9d700 1 -- 172.21.15.143:0/1856222706 learned_addr learned my addr 172.21.15.143:0/1856222706 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:28:06.879 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:06.875+0000 7f7f29d9b700 1 --2- 172.21.15.143:0/1856222706 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7f2c0b2210 0x7f7f2c0b2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:06.880 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:06.875+0000 7f7f2a59c700 1 -- 172.21.15.143:0/1856222706 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7f2c0b2210 msgr2=0x7f7f2c0b2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:06.880 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:06.875+0000 7f7f2a59c700 1 --2- 172.21.15.143:0/1856222706 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7f2c0b2210 0x7f7f2c0b2670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:06.880 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:06.875+0000 7f7f2a59c700 1 -- 172.21.15.143:0/1856222706 >> v1:172.21.15.143:6789/0 conn(0x7f7f2c0cbe20 legacy=0x7f7f2c0cc270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:06.880 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:06.875+0000 7f7f2959a700 1 -- 172.21.15.143:0/1856222706 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1628291797 0 0) 0x7f7f2c0b2bb0 con 0x7f7f2c0cbe20 2023-11-08T19:28:06.880 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:06.875+0000 7f7f2a59c700 1 -- 172.21.15.143:0/1856222706 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f2c0ce750 con 0x7f7f2c15aa30 2023-11-08T19:28:06.881 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:06.875+0000 7f7f2a59c700 1 --2- 172.21.15.143:0/1856222706 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f2c15aa30 0x7f7f2c0cf510 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c009820 tx=0x7f7f1c009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d7d3b0f9b6ca287 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:06.881 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:06.875+0000 7f7f2959a700 1 -- 172.21.15.143:0/1856222706 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f1c005e30 con 0x7f7f2c15aa30 2023-11-08T19:28:06.881 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:06.875+0000 7f7f2959a700 1 -- 172.21.15.143:0/1856222706 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f1c006030 con 0x7f7f2c15aa30 2023-11-08T19:28:06.881 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:06.876+0000 7f7f2959a700 1 -- 172.21.15.143:0/1856222706 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f1c0063e0 con 0x7f7f2c15aa30 2023-11-08T19:28:06.881 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:06.876+0000 7f7f30bb4700 1 -- 172.21.15.143:0/1856222706 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f2c15aa30 msgr2=0x7f7f2c0cf510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:06.882 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:06.876+0000 7f7f30bb4700 1 --2- 172.21.15.143:0/1856222706 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f2c15aa30 0x7f7f2c0cf510 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c009820 tx=0x7f7f1c009b30 comp rx=0 tx=0).stop 2023-11-08T19:28:06.882 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:06.876+0000 7f7f30bb4700 1 -- 172.21.15.143:0/1856222706 shutdown_connections 2023-11-08T19:28:06.882 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:06.876+0000 7f7f30bb4700 1 --2- 172.21.15.143:0/1856222706 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f2c15aa30 0x7f7f2c0cf510 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:06.882 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:06.876+0000 7f7f30bb4700 1 --2- 172.21.15.143:0/1856222706 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7f2c0b2210 0x7f7f2c0b2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:06.882 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:06.876+0000 7f7f30bb4700 1 -- 172.21.15.143:0/1856222706 >> 172.21.15.143:0/1856222706 conn(0x7f7f2c099ab0 msgr2=0x7f7f2c0d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:06.883 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:06.876+0000 7f7f30bb4700 1 -- 172.21.15.143:0/1856222706 shutdown_connections 2023-11-08T19:28:06.883 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:06.876+0000 7f7f30bb4700 1 -- 172.21.15.143:0/1856222706 wait complete. 2023-11-08T19:28:06.885 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:06.878+0000 7f7f30bb4700 1 Processor -- start 2023-11-08T19:28:06.885 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:06.878+0000 7f7f30bb4700 1 -- start start 2023-11-08T19:28:06.886 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:06.879+0000 7f7f30bb4700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f2c0b2210 0x7f7f2c0b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:06.886 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:06.879+0000 7f7f30bb4700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7f2c0cbe20 0x7f7f2c147900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:06.886 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:06.879+0000 7f7f30bb4700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7f2c15aa30 0x7f7f2c0c6400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:06.886 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:06.879+0000 7f7f30bb4700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f2c14b120 con 0x7f7f2c0b2210 2023-11-08T19:28:06.887 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:06.879+0000 7f7f30bb4700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f2c147e40 con 0x7f7f2c15aa30 2023-11-08T19:28:06.887 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:06.879+0000 7f7f30bb4700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f2c14a5a0 con 0x7f7f2c0cbe20 2023-11-08T19:28:06.887 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:06.879+0000 7f7f2a59c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f2c0b2210 0x7f7f2c0b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:06.887 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:06.879+0000 7f7f2a59c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f2c0b2210 0x7f7f2c0b6200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54594/0 (socket says 172.21.15.143:54594) 2023-11-08T19:28:06.887 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:06.879+0000 7f7f2a59c700 1 -- 172.21.15.143:0/2937737290 learned_addr learned my addr 172.21.15.143:0/2937737290 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:28:06.888 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:06.879+0000 7f7f2a59c700 1 -- 172.21.15.143:0/2937737290 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7f2c0cbe20 msgr2=0x7f7f2c147900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:06.888 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:06.879+0000 7f7f2a59c700 1 --2- 172.21.15.143:0/2937737290 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7f2c0cbe20 0x7f7f2c147900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:06.888 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:06.879+0000 7f7f2a59c700 1 -- 172.21.15.143:0/2937737290 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7f2c15aa30 msgr2=0x7f7f2c0c6400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:06.888 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:06.879+0000 7f7f2ad9d700 1 --2- 172.21.15.143:0/2937737290 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7f2c15aa30 0x7f7f2c0c6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:06.889 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:06.879+0000 7f7f2a59c700 1 --2- 172.21.15.143:0/2937737290 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7f2c15aa30 0x7f7f2c0c6400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:06.889 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:06.879+0000 7f7f2a59c700 1 -- 172.21.15.143:0/2937737290 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f1c009430 con 0x7f7f2c0b2210 2023-11-08T19:28:06.889 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:06.879+0000 7f7f2a59c700 1 --2- 172.21.15.143:0/2937737290 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f2c0b2210 0x7f7f2c0b6200 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c009f30 tx=0x7f7f1c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:06.889 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:06.879+0000 7f7f1affd700 1 -- 172.21.15.143:0/2937737290 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f1c025630 con 0x7f7f2c0b2210 2023-11-08T19:28:06.889 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:06.879+0000 7f7f1affd700 1 -- 172.21.15.143:0/2937737290 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f1c020070 con 0x7f7f2c0b2210 2023-11-08T19:28:06.889 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:06.880+0000 7f7f30bb4700 1 -- 172.21.15.143:0/2937737290 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f2c14a880 con 0x7f7f2c0b2210 2023-11-08T19:28:06.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:06.880+0000 7f7f1affd700 1 -- 172.21.15.143:0/2937737290 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f1c0259d0 con 0x7f7f2c0b2210 2023-11-08T19:28:06.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:06.880+0000 7f7f30bb4700 1 -- 172.21.15.143:0/2937737290 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f2c14ae40 con 0x7f7f2c0b2210 2023-11-08T19:28:06.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:06.880+0000 7f7f30bb4700 1 -- 172.21.15.143:0/2937737290 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f2c04eee0 con 0x7f7f2c0b2210 2023-11-08T19:28:06.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:06.881+0000 7f7f1affd700 1 -- 172.21.15.143:0/2937737290 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7f1c022030 con 0x7f7f2c0b2210 2023-11-08T19:28:06.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:06.881+0000 7f7f1affd700 1 --2- 172.21.15.143:0/2937737290 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f7f0c06af00 0x7f7f0c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:06.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:06.881+0000 7f7f1affd700 1 -- 172.21.15.143:0/2937737290 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6772+0+0 (secure 0 0 0) 0x7f7f1c08b0a0 con 0x7f7f2c0b2210 2023-11-08T19:28:06.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:06.882+0000 7f7f29d9b700 1 --2- 172.21.15.143:0/2937737290 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f7f0c06af00 0x7f7f0c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:06.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:06.882+0000 7f7f29d9b700 1 --2- 172.21.15.143:0/2937737290 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f7f0c06af00 0x7f7f0c06d3c0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f7f2c1f2c20 tx=0x7f7f20008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:06.895 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:06.886+0000 7f7f1affd700 1 -- 172.21.15.143:0/2937737290 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7f1c055db0 con 0x7f7f2c0b2210 2023-11-08T19:28:07.121 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.120+0000 7f7f30bb4700 1 -- 172.21.15.143:0/2937737290 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7f2c0b2ff0 con 0x7f7f2c0b2210 2023-11-08T19:28:07.122 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.120+0000 7f7f1affd700 1 -- 172.21.15.143:0/2937737290 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) v1 ==== 94+0+4927 (secure 0 0 0) 0x7f7f1c0593d0 con 0x7f7f2c0b2210 2023-11-08T19:28:07.122 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:28:07.122 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":52,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6906,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6834/3172664377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":3172664377},{"type":"v1","addr":"172.21.15.143:6834","nonce":3172664377}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47},{"gid":6909,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3953059900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":3953059900},{"type":"v1","addr":"172.21.15.143:6835","nonce":3953059900}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":46},{"gid":6918,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6836/3601088618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3601088618},{"type":"v1","addr":"172.21.15.174:6836","nonce":3601088618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T19:28:03.092245+0000","modified":"2023-11-08T19:28:06.121216+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6921},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6921":{"gid":6921,"name":"d","rank":0,"incarnation":48,"state":"up:active","state_seq":3,"addr":"172.21.15.174:6837/2201447629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6835","nonce":2201447629},{"type":"v1","addr":"172.21.15.174:6837","nonce":2201447629}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:28:07.124 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.123+0000 7f7f30bb4700 1 -- 172.21.15.143:0/2937737290 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f7f0c06af00 msgr2=0x7f7f0c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:07.125 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.123+0000 7f7f30bb4700 1 --2- 172.21.15.143:0/2937737290 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f7f0c06af00 0x7f7f0c06d3c0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f7f2c1f2c20 tx=0x7f7f20008040 comp rx=0 tx=0).stop 2023-11-08T19:28:07.125 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.123+0000 7f7f30bb4700 1 -- 172.21.15.143:0/2937737290 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f2c0b2210 msgr2=0x7f7f2c0b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:07.125 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.123+0000 7f7f30bb4700 1 --2- 172.21.15.143:0/2937737290 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f2c0b2210 0x7f7f2c0b6200 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c009f30 tx=0x7f7f1c004750 comp rx=0 tx=0).stop 2023-11-08T19:28:07.125 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.123+0000 7f7f30bb4700 1 -- 172.21.15.143:0/2937737290 shutdown_connections 2023-11-08T19:28:07.125 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.123+0000 7f7f30bb4700 1 --2- 172.21.15.143:0/2937737290 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f7f0c06af00 0x7f7f0c06d3c0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:07.126 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.123+0000 7f7f30bb4700 1 --2- 172.21.15.143:0/2937737290 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7f2c15aa30 0x7f7f2c0c6400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:07.126 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.123+0000 7f7f30bb4700 1 --2- 172.21.15.143:0/2937737290 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f2c0b2210 0x7f7f2c0b6200 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:07.126 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.123+0000 7f7f30bb4700 1 --2- 172.21.15.143:0/2937737290 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7f2c0cbe20 0x7f7f2c147900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:07.126 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.123+0000 7f7f30bb4700 1 -- 172.21.15.143:0/2937737290 >> 172.21.15.143:0/2937737290 conn(0x7f7f2c099ab0 msgr2=0x7f7f2c0cb2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:07.126 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.123+0000 7f7f30bb4700 1 -- 172.21.15.143:0/2937737290 shutdown_connections 2023-11-08T19:28:07.127 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.123+0000 7f7f30bb4700 1 -- 172.21.15.143:0/2937737290 wait complete. 2023-11-08T19:28:07.127 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 52 2023-11-08T19:28:07.137 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:28:07.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.255+0000 7f519d4ba700 1 Processor -- start 2023-11-08T19:28:07.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.255+0000 7f519d4ba700 1 -- start start 2023-11-08T19:28:07.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.256+0000 7f519d4ba700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f51980cf090 0x7f51980d1f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:07.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.256+0000 7f519d4ba700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f51980d3690 0x7f51980b9690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:07.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.256+0000 7f519d4ba700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51980d3200 con 0x7f519815dc80 2023-11-08T19:28:07.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.256+0000 7f519d4ba700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51980b9bd0 con 0x7f51980d3690 2023-11-08T19:28:07.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.256+0000 7f519d4ba700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51980b7cf0 con 0x7f51980cf090 2023-11-08T19:28:07.261 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.256+0000 7f51977fe700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f51980d3690 0x7f51980b9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:07.261 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.256+0000 7f51977fe700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f51980d3690 0x7f51980b9690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54606/0 (socket says 172.21.15.143:54606) 2023-11-08T19:28:07.261 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.256+0000 7f5196ffd700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f519815dc80 0x7f51980ce980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:50756/0 (socket says 172.21.15.143:50756) 2023-11-08T19:28:07.261 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.256+0000 7f51977fe700 1 -- 172.21.15.143:0/3123921815 learned_addr learned my addr 172.21.15.143:0/3123921815 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:28:07.261 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.256+0000 7f51967fc700 1 --2- 172.21.15.143:0/3123921815 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f51980cf090 0x7f51980d1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:07.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.256+0000 7f51977fe700 1 -- 172.21.15.143:0/3123921815 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f51980cf090 msgr2=0x7f51980d1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:07.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.256+0000 7f51977fe700 1 --2- 172.21.15.143:0/3123921815 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f51980cf090 0x7f51980d1f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:07.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.256+0000 7f51977fe700 1 -- 172.21.15.143:0/3123921815 >> v1:172.21.15.143:6789/0 conn(0x7f519815dc80 legacy=0x7f51980ce980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:07.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.256+0000 7f51977fe700 1 -- 172.21.15.143:0/3123921815 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f519815e060 con 0x7f51980d3690 2023-11-08T19:28:07.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.257+0000 7f51977fe700 1 --2- 172.21.15.143:0/3123921815 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f51980d3690 0x7f51980b9690 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f5190009e50 tx=0x7f519000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3972f6e56f938d11 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:07.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.257+0000 7f5195ffb700 1 -- 172.21.15.143:0/3123921815 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f519000e070 con 0x7f51980d3690 2023-11-08T19:28:07.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.257+0000 7f5195ffb700 1 -- 172.21.15.143:0/3123921815 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f519000b9c0 con 0x7f51980d3690 2023-11-08T19:28:07.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.257+0000 7f5195ffb700 1 -- 172.21.15.143:0/3123921815 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f519000bd70 con 0x7f51980d3690 2023-11-08T19:28:07.264 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.257+0000 7f519d4ba700 1 -- 172.21.15.143:0/3123921815 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f51980d3690 msgr2=0x7f51980b9690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:07.264 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.257+0000 7f519d4ba700 1 --2- 172.21.15.143:0/3123921815 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f51980d3690 0x7f51980b9690 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f5190009e50 tx=0x7f519000b330 comp rx=0 tx=0).stop 2023-11-08T19:28:07.264 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.257+0000 7f519d4ba700 1 -- 172.21.15.143:0/3123921815 shutdown_connections 2023-11-08T19:28:07.264 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.257+0000 7f519d4ba700 1 --2- 172.21.15.143:0/3123921815 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f51980d3690 0x7f51980b9690 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:07.264 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.257+0000 7f519d4ba700 1 --2- 172.21.15.143:0/3123921815 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f51980cf090 0x7f51980d1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:07.265 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.257+0000 7f519d4ba700 1 -- 172.21.15.143:0/3123921815 >> 172.21.15.143:0/3123921815 conn(0x7f51980668e0 msgr2=0x7f5198066cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:07.265 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.257+0000 7f519d4ba700 1 -- 172.21.15.143:0/3123921815 shutdown_connections 2023-11-08T19:28:07.265 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.257+0000 7f519d4ba700 1 -- 172.21.15.143:0/3123921815 wait complete. 2023-11-08T19:28:07.268 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.259+0000 7f519d4ba700 1 Processor -- start 2023-11-08T19:28:07.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.259+0000 7f519d4ba700 1 -- start start 2023-11-08T19:28:07.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.259+0000 7f519d4ba700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f51980cf090 0x7f51980c9090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:07.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.259+0000 7f519d4ba700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f51980d3690 0x7f51981f2a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:07.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.260+0000 7f519d4ba700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f519815dc80 0x7f51980c76a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:07.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.260+0000 7f519d4ba700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51981f2fc0 con 0x7f51980cf090 2023-11-08T19:28:07.270 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.260+0000 7f519d4ba700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51980cb290 con 0x7f519815dc80 2023-11-08T19:28:07.270 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.260+0000 7f519d4ba700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51980cb570 con 0x7f51980d3690 2023-11-08T19:28:07.270 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.260+0000 7f5196ffd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f51980cf090 0x7f51980c9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:07.270 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.260+0000 7f5196ffd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f51980cf090 0x7f51980c9090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54608/0 (socket says 172.21.15.143:54608) 2023-11-08T19:28:07.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.260+0000 7f5196ffd700 1 -- 172.21.15.143:0/2198931655 learned_addr learned my addr 172.21.15.143:0/2198931655 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:28:07.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.260+0000 7f51967fc700 1 --2- 172.21.15.143:0/2198931655 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f51980d3690 0x7f51981f2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:07.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.260+0000 7f51977fe700 1 --2- 172.21.15.143:0/2198931655 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f519815dc80 0x7f51980c76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:07.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.260+0000 7f5196ffd700 1 -- 172.21.15.143:0/2198931655 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f51980d3690 msgr2=0x7f51981f2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:07.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.260+0000 7f5196ffd700 1 --2- 172.21.15.143:0/2198931655 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f51980d3690 0x7f51981f2a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:07.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.260+0000 7f5196ffd700 1 -- 172.21.15.143:0/2198931655 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f519815dc80 msgr2=0x7f51980c76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:07.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.260+0000 7f5196ffd700 1 --2- 172.21.15.143:0/2198931655 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f519815dc80 0x7f51980c76a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:07.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.260+0000 7f5196ffd700 1 -- 172.21.15.143:0/2198931655 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5190009b30 con 0x7f51980cf090 2023-11-08T19:28:07.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.260+0000 7f5196ffd700 1 --2- 172.21.15.143:0/2198931655 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f51980cf090 0x7f51980c9090 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f518c00c500 tx=0x7f518c00c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:07.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.260+0000 7f517f7fe700 1 -- 172.21.15.143:0/2198931655 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f518c005cb0 con 0x7f51980cf090 2023-11-08T19:28:07.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.260+0000 7f517f7fe700 1 -- 172.21.15.143:0/2198931655 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f518c005e10 con 0x7f51980cf090 2023-11-08T19:28:07.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.260+0000 7f519d4ba700 1 -- 172.21.15.143:0/2198931655 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51980cb850 con 0x7f51980cf090 2023-11-08T19:28:07.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.260+0000 7f517f7fe700 1 -- 172.21.15.143:0/2198931655 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f518c0072d0 con 0x7f51980cf090 2023-11-08T19:28:07.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.260+0000 7f519d4ba700 1 -- 172.21.15.143:0/2198931655 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51980bda00 con 0x7f51980cf090 2023-11-08T19:28:07.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.261+0000 7f519d4ba700 1 -- 172.21.15.143:0/2198931655 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51980cb3d0 con 0x7f51980cf090 2023-11-08T19:28:07.274 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.262+0000 7f517f7fe700 1 -- 172.21.15.143:0/2198931655 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f518c007430 con 0x7f51980cf090 2023-11-08T19:28:07.274 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.262+0000 7f517f7fe700 1 --2- 172.21.15.143:0/2198931655 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f518406af30 0x7f518406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:07.274 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.262+0000 7f517f7fe700 1 -- 172.21.15.143:0/2198931655 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6772+0+0 (secure 0 0 0) 0x7f518c088ca0 con 0x7f51980cf090 2023-11-08T19:28:07.274 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.262+0000 7f51967fc700 1 --2- 172.21.15.143:0/2198931655 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f518406af30 0x7f518406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:07.274 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.263+0000 7f51967fc700 1 --2- 172.21.15.143:0/2198931655 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f518406af30 0x7f518406d3f0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f51980d2200 tx=0x7f5188008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:07.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.266+0000 7f517f7fe700 1 -- 172.21.15.143:0/2198931655 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f51980cb3d0 con 0x7f51980cf090 2023-11-08T19:28:07.458 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.457+0000 7f519d4ba700 1 -- 172.21.15.143:0/2198931655 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f51980cbd70 con 0x7f51980cf090 2023-11-08T19:28:07.459 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.458+0000 7f517f7fe700 1 -- 172.21.15.143:0/2198931655 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+21792 (secure 0 0 0) 0x7f51980cbd70 con 0x7f51980cf090 2023-11-08T19:28:07.460 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:28:07.460 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":92,"fsid":"968483af-77fc-4068-ad7c-5c2a1a4613d7","created":"2023-11-08T19:24:32.826484+0000","modified":"2023-11-08T19:28:04.694036+0000","last_up_change":"2023-11-08T19:24:43.573880+0000","last_in_change":"2023-11-08T19:24:35.314525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:44.551530+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"snap-test-pool","create_time":"2023-11-08T19:26:02.874384+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":36,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"test_data_pool","create_time":"2023-11-08T19:27:05.601949+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":66,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_metadata_pool","create_time":"2023-11-08T19:27:06.536369+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-11-08T19:28:01.533566+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-11-08T19:28:02.053445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bae98ec1-d94e-403d-a220-a522a53ea6f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6809","nonce":1487830566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6811","nonce":1487830566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6815","nonce":1487830566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6813","nonce":1487830566}]},"public_addr":"172.21.15.143:6809/1487830566","cluster_addr":"172.21.15.143:6811/1487830566","heartbeat_back_addr":"172.21.15.143:6815/1487830566","heartbeat_front_addr":"172.21.15.143:6813/1487830566","state":["exists","up"]},{"osd":1,"uuid":"5cb6538a-1ccb-4322-8633-09f66af0b7e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6801","nonce":787341397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6803","nonce":787341397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6807","nonce":787341397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6805","nonce":787341397}]},"public_addr":"172.21.15.143:6801/787341397","cluster_addr":"172.21.15.143:6803/787341397","heartbeat_back_addr":"172.21.15.143:6807/787341397","heartbeat_front_addr":"172.21.15.143:6805/787341397","state":["exists","up"]},{"osd":2,"uuid":"cdbf522c-60ef-4d60-9653-8dcb529c101a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6822","nonce":2301325597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6826","nonce":2301325597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6831","nonce":2301325597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6829","nonce":2301325597}]},"public_addr":"172.21.15.143:6822/2301325597","cluster_addr":"172.21.15.143:6826/2301325597","heartbeat_back_addr":"172.21.15.143:6831/2301325597","heartbeat_front_addr":"172.21.15.143:6829/2301325597","state":["exists","up"]},{"osd":3,"uuid":"4e2783cf-ebb9-4dd8-8245-bc1d17ce4325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6817","nonce":2021504428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6819","nonce":2021504428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6827","nonce":2021504428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6823","nonce":2021504428}]},"public_addr":"172.21.15.143:6817/2021504428","cluster_addr":"172.21.15.143:6819/2021504428","heartbeat_back_addr":"172.21.15.143:6827/2021504428","heartbeat_front_addr":"172.21.15.143:6823/2021504428","state":["exists","up"]},{"osd":4,"uuid":"691256fa-9b77-49b9-a346-7ae1adf0ca80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6801","nonce":1993380595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6803","nonce":1993380595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6807","nonce":1993380595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6805","nonce":1993380595}]},"public_addr":"172.21.15.174:6801/1993380595","cluster_addr":"172.21.15.174:6803/1993380595","heartbeat_back_addr":"172.21.15.174:6807/1993380595","heartbeat_front_addr":"172.21.15.174:6805/1993380595","state":["exists","up"]},{"osd":5,"uuid":"756fc352-48c9-46ce-b52e-0541fe6f8525","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6809","nonce":1777923303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6811","nonce":1777923303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6815","nonce":1777923303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6813","nonce":1777923303}]},"public_addr":"172.21.15.174:6809/1777923303","cluster_addr":"172.21.15.174:6811/1777923303","heartbeat_back_addr":"172.21.15.174:6815/1777923303","heartbeat_front_addr":"172.21.15.174:6813/1777923303","state":["exists","up"]},{"osd":6,"uuid":"fab2ac2b-ed97-4cde-9825-b480eabf8b8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6825","nonce":2755130316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6827","nonce":2755130316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6831","nonce":2755130316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6829","nonce":2755130316}]},"public_addr":"172.21.15.174:6825/2755130316","cluster_addr":"172.21.15.174:6827/2755130316","heartbeat_back_addr":"172.21.15.174:6831/2755130316","heartbeat_front_addr":"172.21.15.174:6829/2755130316","state":["exists","up"]},{"osd":7,"uuid":"3de9b797-2ab6-4d41-a863-7e7d638fd703","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6817","nonce":2847141411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6819","nonce":2847141411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6823","nonce":2847141411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6821","nonce":2847141411}]},"public_addr":"172.21.15.174:6817/2847141411","cluster_addr":"172.21.15.174:6819/2847141411","heartbeat_back_addr":"172.21.15.174:6823/2847141411","heartbeat_front_addr":"172.21.15.174:6821/2847141411","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.477076+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.496890+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.539305+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.454673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.538881+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.521294+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.520712+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.540404+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:28:07.462 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.460+0000 7f519d4ba700 1 -- 172.21.15.143:0/2198931655 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f518406af30 msgr2=0x7f518406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:07.462 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.460+0000 7f519d4ba700 1 --2- 172.21.15.143:0/2198931655 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f518406af30 0x7f518406d3f0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f51980d2200 tx=0x7f5188008040 comp rx=0 tx=0).stop 2023-11-08T19:28:07.463 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.460+0000 7f519d4ba700 1 -- 172.21.15.143:0/2198931655 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f51980cf090 msgr2=0x7f51980c9090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:07.463 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.460+0000 7f519d4ba700 1 --2- 172.21.15.143:0/2198931655 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f51980cf090 0x7f51980c9090 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f518c00c500 tx=0x7f518c00c810 comp rx=0 tx=0).stop 2023-11-08T19:28:07.463 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.460+0000 7f519d4ba700 1 -- 172.21.15.143:0/2198931655 shutdown_connections 2023-11-08T19:28:07.463 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.460+0000 7f519d4ba700 1 --2- 172.21.15.143:0/2198931655 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f518406af30 0x7f518406d3f0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:07.463 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.460+0000 7f519d4ba700 1 --2- 172.21.15.143:0/2198931655 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f519815dc80 0x7f51980c76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:07.464 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.460+0000 7f519d4ba700 1 --2- 172.21.15.143:0/2198931655 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f51980cf090 0x7f51980c9090 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:07.464 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.460+0000 7f519d4ba700 1 --2- 172.21.15.143:0/2198931655 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f51980d3690 0x7f51981f2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:07.464 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.460+0000 7f519d4ba700 1 -- 172.21.15.143:0/2198931655 >> 172.21.15.143:0/2198931655 conn(0x7f51980668e0 msgr2=0x7f519815b900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:07.464 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.460+0000 7f519d4ba700 1 -- 172.21.15.143:0/2198931655 shutdown_connections 2023-11-08T19:28:07.464 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.460+0000 7f519d4ba700 1 -- 172.21.15.143:0/2198931655 wait complete. 2023-11-08T19:28:07.474 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-08T19:28:07.475 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-08T19:28:07.596 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.594+0000 7f09163ec700 1 Processor -- start 2023-11-08T19:28:07.596 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.594+0000 7f09163ec700 1 -- start start 2023-11-08T19:28:07.598 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.595+0000 7f09163ec700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f091015aa30 0x7f09100cf510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:07.599 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.595+0000 7f09163ec700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f09100cc530 0x7f09100b2690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:07.599 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.595+0000 7f09163ec700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09100cf080 con 0x7f09100b2210 2023-11-08T19:28:07.599 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.595+0000 7f09163ec700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09100b2bd0 con 0x7f09100cc530 2023-11-08T19:28:07.599 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.595+0000 7f09163ec700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09100ce470 con 0x7f091015aa30 2023-11-08T19:28:07.599 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.595+0000 7f090f7fe700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f09100b2210 0x7f09100cbe20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:50770/0 (socket says 172.21.15.143:50770) 2023-11-08T19:28:07.600 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.595+0000 7f090f7fe700 1 -- 172.21.15.143:0/142676144 learned_addr learned my addr 172.21.15.143:0/142676144 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:28:07.600 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.595+0000 7f0914989700 1 --2- 172.21.15.143:0/142676144 >> v2:172.21.15.174:3300/0 conn(0x7f09100cc530 0x7f09100b2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:07.600 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.595+0000 7f090ffff700 1 --2- 172.21.15.143:0/142676144 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f091015aa30 0x7f09100cf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:07.600 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.595+0000 7f090effd700 1 -- 172.21.15.143:0/142676144 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 887755537 0 0) 0x7f09100cf080 con 0x7f09100b2210 2023-11-08T19:28:07.600 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.596+0000 7f090effd700 1 -- 172.21.15.143:0/142676144 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f08f80035b0 con 0x7f09100b2210 2023-11-08T19:28:07.601 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.596+0000 7f090ffff700 1 -- 172.21.15.143:0/142676144 >> v2:172.21.15.174:3300/0 conn(0x7f09100cc530 msgr2=0x7f09100b2690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:07.601 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.596+0000 7f090ffff700 1 --2- 172.21.15.143:0/142676144 >> v2:172.21.15.174:3300/0 conn(0x7f09100cc530 0x7f09100b2690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:07.601 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.596+0000 7f090ffff700 1 -- 172.21.15.143:0/142676144 >> v1:172.21.15.143:6789/0 conn(0x7f09100b2210 legacy=0x7f09100cbe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:07.601 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.596+0000 7f090ffff700 1 -- 172.21.15.143:0/142676144 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09100ce750 con 0x7f091015aa30 2023-11-08T19:28:07.601 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.596+0000 7f0914989700 1 --2- 172.21.15.143:0/142676144 >> v2:172.21.15.174:3300/0 conn(0x7f09100cc530 0x7f09100b2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T19:28:07.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.596+0000 7f090ffff700 1 --2- 172.21.15.143:0/142676144 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f091015aa30 0x7f09100cf510 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0900009820 tx=0x7f0900009b30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2362cd2ac98cb561 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:07.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.596+0000 7f090effd700 1 -- 172.21.15.143:0/142676144 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0900005e30 con 0x7f091015aa30 2023-11-08T19:28:07.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.596+0000 7f090effd700 1 -- 172.21.15.143:0/142676144 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0900006030 con 0x7f091015aa30 2023-11-08T19:28:07.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.596+0000 7f090effd700 1 -- 172.21.15.143:0/142676144 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09000063c0 con 0x7f091015aa30 2023-11-08T19:28:07.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.597+0000 7f09163ec700 1 -- 172.21.15.143:0/142676144 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f091015aa30 msgr2=0x7f09100cf510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:07.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.597+0000 7f09163ec700 1 --2- 172.21.15.143:0/142676144 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f091015aa30 0x7f09100cf510 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0900009820 tx=0x7f0900009b30 comp rx=0 tx=0).stop 2023-11-08T19:28:07.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.597+0000 7f09163ec700 1 -- 172.21.15.143:0/142676144 shutdown_connections 2023-11-08T19:28:07.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.597+0000 7f09163ec700 1 --2- 172.21.15.143:0/142676144 >> v2:172.21.15.174:3300/0 conn(0x7f09100cc530 0x7f09100b2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:07.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.597+0000 7f09163ec700 1 --2- 172.21.15.143:0/142676144 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f091015aa30 0x7f09100cf510 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:07.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.597+0000 7f09163ec700 1 -- 172.21.15.143:0/142676144 >> 172.21.15.143:0/142676144 conn(0x7f0910099ab0 msgr2=0x7f09100d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:07.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.597+0000 7f09163ec700 1 -- 172.21.15.143:0/142676144 shutdown_connections 2023-11-08T19:28:07.604 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.597+0000 7f09163ec700 1 -- 172.21.15.143:0/142676144 wait complete. 2023-11-08T19:28:07.606 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.598+0000 7f09163ec700 1 Processor -- start 2023-11-08T19:28:07.607 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.598+0000 7f09163ec700 1 -- start start 2023-11-08T19:28:07.607 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.599+0000 7f09163ec700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f09100b2210 0x7f091006ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:07.607 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.599+0000 7f09163ec700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f09100cc530 0x7f09101f5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:07.607 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.599+0000 7f09163ec700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f091015aa30 0x7f09101f6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:07.607 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.599+0000 7f09163ec700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f091015d4f0 con 0x7f091015aa30 2023-11-08T19:28:07.608 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.599+0000 7f09163ec700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0910152680 con 0x7f09100cc530 2023-11-08T19:28:07.608 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.599+0000 7f09163ec700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0910152960 con 0x7f09100b2210 2023-11-08T19:28:07.608 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.599+0000 7f0914989700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f091015aa30 0x7f09101f6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:07.608 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.599+0000 7f0914989700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f091015aa30 0x7f09101f6460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54626/0 (socket says 172.21.15.143:54626) 2023-11-08T19:28:07.608 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.599+0000 7f0914989700 1 -- 172.21.15.143:0/2053460508 learned_addr learned my addr 172.21.15.143:0/2053460508 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:28:07.609 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.599+0000 7f090f7fe700 1 --2- 172.21.15.143:0/2053460508 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f09100cc530 0x7f09101f5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:07.609 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.599+0000 7f090ffff700 1 --2- 172.21.15.143:0/2053460508 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f09100b2210 0x7f091006ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:07.609 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.599+0000 7f0914989700 1 -- 172.21.15.143:0/2053460508 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f09100b2210 msgr2=0x7f091006ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:07.609 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.599+0000 7f0914989700 1 --2- 172.21.15.143:0/2053460508 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f09100b2210 0x7f091006ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:07.609 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.599+0000 7f0914989700 1 -- 172.21.15.143:0/2053460508 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f09100cc530 msgr2=0x7f09101f5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:07.610 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.599+0000 7f0914989700 1 --2- 172.21.15.143:0/2053460508 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f09100cc530 0x7f09101f5f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:07.610 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.599+0000 7f0914989700 1 -- 172.21.15.143:0/2053460508 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0908009a70 con 0x7f091015aa30 2023-11-08T19:28:07.610 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.599+0000 7f090ffff700 1 --2- 172.21.15.143:0/2053460508 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f09100b2210 0x7f091006ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:28:07.610 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.599+0000 7f0914989700 1 --2- 172.21.15.143:0/2053460508 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f091015aa30 0x7f09101f6460 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f090800efc0 tx=0x7f090800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:07.610 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.599+0000 7f090cff9700 1 -- 172.21.15.143:0/2053460508 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f090800cb40 con 0x7f091015aa30 2023-11-08T19:28:07.611 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.600+0000 7f090cff9700 1 -- 172.21.15.143:0/2053460508 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f090800f040 con 0x7f091015aa30 2023-11-08T19:28:07.611 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.600+0000 7f09163ec700 1 -- 172.21.15.143:0/2053460508 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0900009430 con 0x7f091015aa30 2023-11-08T19:28:07.611 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.600+0000 7f090cff9700 1 -- 172.21.15.143:0/2053460508 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0908003a20 con 0x7f091015aa30 2023-11-08T19:28:07.611 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.600+0000 7f09163ec700 1 -- 172.21.15.143:0/2053460508 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09101e6810 con 0x7f091015aa30 2023-11-08T19:28:07.611 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.600+0000 7f09163ec700 1 -- 172.21.15.143:0/2053460508 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f091004eee0 con 0x7f091015aa30 2023-11-08T19:28:07.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.601+0000 7f090cff9700 1 -- 172.21.15.143:0/2053460508 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0908014480 con 0x7f091015aa30 2023-11-08T19:28:07.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.601+0000 7f090cff9700 1 --2- 172.21.15.143:0/2053460508 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f08fc06af00 0x7f08fc06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:07.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.601+0000 7f090cff9700 1 -- 172.21.15.143:0/2053460508 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6772+0+0 (secure 0 0 0) 0x7f090808a320 con 0x7f091015aa30 2023-11-08T19:28:07.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.602+0000 7f090ffff700 1 --2- 172.21.15.143:0/2053460508 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f08fc06af00 0x7f08fc06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:07.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.602+0000 7f090ffff700 1 --2- 172.21.15.143:0/2053460508 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f08fc06af00 0x7f08fc06d3c0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f0900000c00 tx=0x7f0900008070 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:07.613 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.606+0000 7f090cff9700 1 -- 172.21.15.143:0/2053460508 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0908055030 con 0x7f091015aa30 2023-11-08T19:28:07.792 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.791+0000 7f09163ec700 1 -- 172.21.15.143:0/2053460508 --> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f09100ced10 con 0x7f08fc06af00 2023-11-08T19:28:07.807 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.806+0000 7f090cff9700 1 -- 172.21.15.143:0/2053460508 <== mgr.4099 v2:172.21.15.174:6832/4037280178 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+528138 (secure 0 0 0) 0x7f0900005cf0 con 0x7f08fc06af00 2023-11-08T19:28:07.813 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:28:07.820 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.810+0000 7f09163ec700 1 -- 172.21.15.143:0/2053460508 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f08fc06af00 msgr2=0x7f08fc06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:07.820 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.810+0000 7f09163ec700 1 --2- 172.21.15.143:0/2053460508 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f08fc06af00 0x7f08fc06d3c0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f0900000c00 tx=0x7f0900008070 comp rx=0 tx=0).stop 2023-11-08T19:28:07.821 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.810+0000 7f09163ec700 1 -- 172.21.15.143:0/2053460508 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f091015aa30 msgr2=0x7f09101f6460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:07.821 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.810+0000 7f09163ec700 1 --2- 172.21.15.143:0/2053460508 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f091015aa30 0x7f09101f6460 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f090800efc0 tx=0x7f090800c3a0 comp rx=0 tx=0).stop 2023-11-08T19:28:07.821 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.810+0000 7f09163ec700 1 -- 172.21.15.143:0/2053460508 shutdown_connections 2023-11-08T19:28:07.822 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.810+0000 7f09163ec700 1 --2- 172.21.15.143:0/2053460508 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f08fc06af00 0x7f08fc06d3c0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:07.822 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.810+0000 7f09163ec700 1 --2- 172.21.15.143:0/2053460508 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f09100b2210 0x7f091006ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:07.822 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.810+0000 7f09163ec700 1 --2- 172.21.15.143:0/2053460508 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f091015aa30 0x7f09101f6460 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:07.823 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.810+0000 7f09163ec700 1 --2- 172.21.15.143:0/2053460508 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f09100cc530 0x7f09101f5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:07.823 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.810+0000 7f09163ec700 1 -- 172.21.15.143:0/2053460508 >> 172.21.15.143:0/2053460508 conn(0x7f0910099ab0 msgr2=0x7f09101e4d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:07.823 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.810+0000 7f09163ec700 1 -- 172.21.15.143:0/2053460508 shutdown_connections 2023-11-08T19:28:07.824 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.810+0000 7f09163ec700 1 -- 172.21.15.143:0/2053460508 wait complete. 2023-11-08T19:28:07.824 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2023-11-08T19:28:07.826 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":186,"stamp":"2023-11-08T19:28:06.562183+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592162,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":171,"num_write_kb":2422,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":129,"ondisk_log_size":129,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":236,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":226020,"kb_used_data":9840,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754748700,"statfs":{"total":773094113280,"available":772862668800,"internally_reserved":0,"allocated":10076160,"data_stored":5344818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1794,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.868593"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.701553+0000","last_change":"2023-11-08T19:28:02.701553+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.701553+0000","last_undegraded":"2023-11-08T19:28:02.701553+0000","last_fullsized":"2023-11-08T19:28:02.701553+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.719263+0000","last_change":"2023-11-08T19:28:03.715294+0000","last_active":"2023-11-08T19:28:03.719263+0000","last_peered":"2023-11-08T19:28:03.719263+0000","last_clean":"2023-11-08T19:28:03.719263+0000","last_became_active":"2023-11-08T19:28:03.715007+0000","last_became_peered":"2023-11-08T19:28:03.715007+0000","last_unstale":"2023-11-08T19:28:03.719263+0000","last_undegraded":"2023-11-08T19:28:03.719263+0000","last_fullsized":"2023-11-08T19:28:03.719263+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:51:46.879726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.700726+0000","last_change":"2023-11-08T19:28:02.700726+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.700726+0000","last_undegraded":"2023-11-08T19:28:02.700726+0000","last_fullsized":"2023-11-08T19:28:02.700726+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.699799+0000","last_change":"2023-11-08T19:28:02.699799+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.699799+0000","last_undegraded":"2023-11-08T19:28:02.699799+0000","last_fullsized":"2023-11-08T19:28:02.699799+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.704338+0000","last_change":"2023-11-08T19:28:02.704338+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.704338+0000","last_undegraded":"2023-11-08T19:28:02.704338+0000","last_fullsized":"2023-11-08T19:28:02.704338+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.722681+0000","last_change":"2023-11-08T19:28:03.718208+0000","last_active":"2023-11-08T19:28:03.722681+0000","last_peered":"2023-11-08T19:28:03.722681+0000","last_clean":"2023-11-08T19:28:03.722681+0000","last_became_active":"2023-11-08T19:28:03.716630+0000","last_became_peered":"2023-11-08T19:28:03.716630+0000","last_unstale":"2023-11-08T19:28:03.722681+0000","last_undegraded":"2023-11-08T19:28:03.722681+0000","last_fullsized":"2023-11-08T19:28:03.722681+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:21:24.678157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.712530+0000","last_change":"2023-11-08T19:28:03.708477+0000","last_active":"2023-11-08T19:28:03.712530+0000","last_peered":"2023-11-08T19:28:03.712530+0000","last_clean":"2023-11-08T19:28:03.712530+0000","last_became_active":"2023-11-08T19:28:03.708188+0000","last_became_peered":"2023-11-08T19:28:03.708188+0000","last_unstale":"2023-11-08T19:28:03.712530+0000","last_undegraded":"2023-11-08T19:28:03.712530+0000","last_fullsized":"2023-11-08T19:28:03.712530+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:57:58.571556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.714751+0000","last_change":"2023-11-08T19:28:02.714751+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.714751+0000","last_undegraded":"2023-11-08T19:28:02.714751+0000","last_fullsized":"2023-11-08T19:28:02.714751+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.722749+0000","last_change":"2023-11-08T19:28:03.718615+0000","last_active":"2023-11-08T19:28:03.722749+0000","last_peered":"2023-11-08T19:28:03.722749+0000","last_clean":"2023-11-08T19:28:03.722749+0000","last_became_active":"2023-11-08T19:28:03.718295+0000","last_became_peered":"2023-11-08T19:28:03.718295+0000","last_unstale":"2023-11-08T19:28:03.722749+0000","last_undegraded":"2023-11-08T19:28:03.722749+0000","last_fullsized":"2023-11-08T19:28:03.722749+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:23:12.882243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.702139+0000","last_change":"2023-11-08T19:28:02.702139+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.702139+0000","last_undegraded":"2023-11-08T19:28:02.702139+0000","last_fullsized":"2023-11-08T19:28:02.702139+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.705435+0000","last_change":"2023-11-08T19:28:02.705435+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.705435+0000","last_undegraded":"2023-11-08T19:28:02.705435+0000","last_fullsized":"2023-11-08T19:28:02.705435+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.703962+0000","last_change":"2023-11-08T19:28:02.703962+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.703962+0000","last_undegraded":"2023-11-08T19:28:02.703962+0000","last_fullsized":"2023-11-08T19:28:02.703962+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.697723+0000","last_change":"2023-11-08T19:28:02.697723+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.697723+0000","last_undegraded":"2023-11-08T19:28:02.697723+0000","last_fullsized":"2023-11-08T19:28:02.697723+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.721906+0000","last_change":"2023-11-08T19:28:03.718346+0000","last_active":"2023-11-08T19:28:03.721906+0000","last_peered":"2023-11-08T19:28:03.721906+0000","last_clean":"2023-11-08T19:28:03.721906+0000","last_became_active":"2023-11-08T19:28:03.718068+0000","last_became_peered":"2023-11-08T19:28:03.718068+0000","last_unstale":"2023-11-08T19:28:03.721906+0000","last_undegraded":"2023-11-08T19:28:03.721906+0000","last_fullsized":"2023-11-08T19:28:03.721906+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:29:02.099134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.721945+0000","last_change":"2023-11-08T19:28:03.717576+0000","last_active":"2023-11-08T19:28:03.721945+0000","last_peered":"2023-11-08T19:28:03.721945+0000","last_clean":"2023-11-08T19:28:03.721945+0000","last_became_active":"2023-11-08T19:28:03.715708+0000","last_became_peered":"2023-11-08T19:28:03.715708+0000","last_unstale":"2023-11-08T19:28:03.721945+0000","last_undegraded":"2023-11-08T19:28:03.721945+0000","last_fullsized":"2023-11-08T19:28:03.721945+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:40:03.163555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.711516+0000","last_change":"2023-11-08T19:28:03.703641+0000","last_active":"2023-11-08T19:28:03.711516+0000","last_peered":"2023-11-08T19:28:03.711516+0000","last_clean":"2023-11-08T19:28:03.711516+0000","last_became_active":"2023-11-08T19:28:03.703331+0000","last_became_peered":"2023-11-08T19:28:03.703331+0000","last_unstale":"2023-11-08T19:28:03.711516+0000","last_undegraded":"2023-11-08T19:28:03.711516+0000","last_fullsized":"2023-11-08T19:28:03.711516+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:32:03.694798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.700364+0000","last_change":"2023-11-08T19:28:02.700364+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.700364+0000","last_undegraded":"2023-11-08T19:28:02.700364+0000","last_fullsized":"2023-11-08T19:28:02.700364+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.719244+0000","last_change":"2023-11-08T19:28:03.715221+0000","last_active":"2023-11-08T19:28:03.719244+0000","last_peered":"2023-11-08T19:28:03.719244+0000","last_clean":"2023-11-08T19:28:03.719244+0000","last_became_active":"2023-11-08T19:28:03.714929+0000","last_became_peered":"2023-11-08T19:28:03.714929+0000","last_unstale":"2023-11-08T19:28:03.719244+0000","last_undegraded":"2023-11-08T19:28:03.719244+0000","last_fullsized":"2023-11-08T19:28:03.719244+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:58:49.797420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.700365+0000","last_change":"2023-11-08T19:28:02.700365+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.700365+0000","last_undegraded":"2023-11-08T19:28:02.700365+0000","last_fullsized":"2023-11-08T19:28:02.700365+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.704383+0000","last_change":"2023-11-08T19:28:02.704383+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.704383+0000","last_undegraded":"2023-11-08T19:28:02.704383+0000","last_fullsized":"2023-11-08T19:28:02.704383+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.713555+0000","last_change":"2023-11-08T19:28:02.713555+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.713555+0000","last_undegraded":"2023-11-08T19:28:02.713555+0000","last_fullsized":"2023-11-08T19:28:02.713555+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.702640+0000","last_change":"2023-11-08T19:28:02.702640+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.702640+0000","last_undegraded":"2023-11-08T19:28:02.702640+0000","last_fullsized":"2023-11-08T19:28:02.702640+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.699841+0000","last_change":"2023-11-08T19:28:02.699841+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.699841+0000","last_undegraded":"2023-11-08T19:28:02.699841+0000","last_fullsized":"2023-11-08T19:28:02.699841+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.702861+0000","last_change":"2023-11-08T19:28:02.702861+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.702861+0000","last_undegraded":"2023-11-08T19:28:02.702861+0000","last_fullsized":"2023-11-08T19:28:02.702861+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.707625+0000","last_change":"2023-11-08T19:28:02.707625+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.707625+0000","last_undegraded":"2023-11-08T19:28:02.707625+0000","last_fullsized":"2023-11-08T19:28:02.707625+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.710228+0000","last_change":"2023-11-08T19:28:02.710228+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.710228+0000","last_undegraded":"2023-11-08T19:28:02.710228+0000","last_fullsized":"2023-11-08T19:28:02.710228+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.704524+0000","last_change":"2023-11-08T19:28:02.704524+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.704524+0000","last_undegraded":"2023-11-08T19:28:02.704524+0000","last_fullsized":"2023-11-08T19:28:02.704524+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.725514+0000","last_change":"2023-11-08T19:28:03.721162+0000","last_active":"2023-11-08T19:28:03.725514+0000","last_peered":"2023-11-08T19:28:03.725514+0000","last_clean":"2023-11-08T19:28:03.725514+0000","last_became_active":"2023-11-08T19:28:03.720956+0000","last_became_peered":"2023-11-08T19:28:03.720956+0000","last_unstale":"2023-11-08T19:28:03.725514+0000","last_undegraded":"2023-11-08T19:28:03.725514+0000","last_fullsized":"2023-11-08T19:28:03.725514+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:09:19.883657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.712428+0000","last_change":"2023-11-08T19:28:02.712428+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.712428+0000","last_undegraded":"2023-11-08T19:28:02.712428+0000","last_fullsized":"2023-11-08T19:28:02.712428+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.705928+0000","last_change":"2023-11-08T19:28:02.705928+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.705928+0000","last_undegraded":"2023-11-08T19:28:02.705928+0000","last_fullsized":"2023-11-08T19:28:02.705928+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.712510+0000","last_change":"2023-11-08T19:28:03.707881+0000","last_active":"2023-11-08T19:28:03.712510+0000","last_peered":"2023-11-08T19:28:03.712510+0000","last_clean":"2023-11-08T19:28:03.712510+0000","last_became_active":"2023-11-08T19:28:03.707569+0000","last_became_peered":"2023-11-08T19:28:03.707569+0000","last_unstale":"2023-11-08T19:28:03.712510+0000","last_undegraded":"2023-11-08T19:28:03.712510+0000","last_fullsized":"2023-11-08T19:28:03.712510+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:00:53.075545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.711643+0000","last_change":"2023-11-08T19:28:02.711643+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.711643+0000","last_undegraded":"2023-11-08T19:28:02.711643+0000","last_fullsized":"2023-11-08T19:28:02.711643+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.702089+0000","last_change":"2023-11-08T19:28:02.702089+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.702089+0000","last_undegraded":"2023-11-08T19:28:02.702089+0000","last_fullsized":"2023-11-08T19:28:02.702089+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.725513+0000","last_change":"2023-11-08T19:28:02.720536+0000","last_active":"2023-11-08T19:28:02.725513+0000","last_peered":"2023-11-08T19:28:02.725513+0000","last_clean":"2023-11-08T19:28:02.725513+0000","last_became_active":"2023-11-08T19:28:02.720236+0000","last_became_peered":"2023-11-08T19:28:02.720236+0000","last_unstale":"2023-11-08T19:28:02.725513+0000","last_undegraded":"2023-11-08T19:28:02.725513+0000","last_fullsized":"2023-11-08T19:28:02.725513+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:56:59.813550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.700732+0000","last_change":"2023-11-08T19:27:08.190826+0000","last_active":"2023-11-08T19:28:02.700732+0000","last_peered":"2023-11-08T19:28:02.700732+0000","last_clean":"2023-11-08T19:28:02.700732+0000","last_became_active":"2023-11-08T19:27:08.190440+0000","last_became_peered":"2023-11-08T19:27:08.190440+0000","last_unstale":"2023-11-08T19:28:02.700732+0000","last_undegraded":"2023-11-08T19:28:02.700732+0000","last_fullsized":"2023-11-08T19:28:02.700732+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:42:48.376462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":237,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.699373+0000","last_change":"2023-11-08T19:24:45.888122+0000","last_active":"2023-11-08T19:28:02.699373+0000","last_peered":"2023-11-08T19:28:02.699373+0000","last_clean":"2023-11-08T19:28:02.699373+0000","last_became_active":"2023-11-08T19:24:45.887048+0000","last_became_peered":"2023-11-08T19:24:45.887048+0000","last_unstale":"2023-11-08T19:28:02.699373+0000","last_undegraded":"2023-11-08T19:28:02.699373+0000","last_fullsized":"2023-11-08T19:28:02.699373+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:44.575133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:44.575133+0000","last_clean_scrub_stamp":"2023-11-08T19:24:44.575133+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:19:20.202062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":60,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.698526+0000","last_change":"2023-11-08T19:26:06.542044+0000","last_active":"2023-11-08T19:28:01.698526+0000","last_peered":"2023-11-08T19:28:01.698526+0000","last_clean":"2023-11-08T19:28:01.698526+0000","last_became_active":"2023-11-08T19:26:04.603056+0000","last_became_peered":"2023-11-08T19:26:04.603056+0000","last_unstale":"2023-11-08T19:28:01.698526+0000","last_undegraded":"2023-11-08T19:28:01.698526+0000","last_fullsized":"2023-11-08T19:28:01.698526+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:41:50.771457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000517414,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.694743+0000","last_change":"2023-11-08T19:27:09.496842+0000","last_active":"2023-11-08T19:28:01.694743+0000","last_peered":"2023-11-08T19:28:01.694743+0000","last_clean":"2023-11-08T19:28:01.694743+0000","last_became_active":"2023-11-08T19:27:07.185776+0000","last_became_peered":"2023-11-08T19:27:07.185776+0000","last_unstale":"2023-11-08T19:28:01.694743+0000","last_undegraded":"2023-11-08T19:28:01.694743+0000","last_fullsized":"2023-11-08T19:28:01.694743+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:21:13.923970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031747200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.698950+0000","last_change":"2023-11-08T19:28:02.698950+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.698950+0000","last_undegraded":"2023-11-08T19:28:02.698950+0000","last_fullsized":"2023-11-08T19:28:02.698950+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.725538+0000","last_change":"2023-11-08T19:28:02.720512+0000","last_active":"2023-11-08T19:28:02.725538+0000","last_peered":"2023-11-08T19:28:02.725538+0000","last_clean":"2023-11-08T19:28:02.725538+0000","last_became_active":"2023-11-08T19:28:02.720202+0000","last_became_peered":"2023-11-08T19:28:02.720202+0000","last_unstale":"2023-11-08T19:28:02.725538+0000","last_undegraded":"2023-11-08T19:28:02.725538+0000","last_fullsized":"2023-11-08T19:28:02.725538+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:26:10.008919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.694327+0000","last_change":"2023-11-08T19:27:08.188241+0000","last_active":"2023-11-08T19:28:01.694327+0000","last_peered":"2023-11-08T19:28:01.694327+0000","last_clean":"2023-11-08T19:28:01.694327+0000","last_became_active":"2023-11-08T19:27:08.187863+0000","last_became_peered":"2023-11-08T19:27:08.187863+0000","last_unstale":"2023-11-08T19:28:01.694327+0000","last_undegraded":"2023-11-08T19:28:01.694327+0000","last_fullsized":"2023-11-08T19:28:01.694327+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:45:22.084015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":58,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.696493+0000","last_change":"2023-11-08T19:26:06.265419+0000","last_active":"2023-11-08T19:28:01.696493+0000","last_peered":"2023-11-08T19:28:01.696493+0000","last_clean":"2023-11-08T19:28:01.696493+0000","last_became_active":"2023-11-08T19:26:04.605333+0000","last_became_peered":"2023-11-08T19:26:04.605333+0000","last_unstale":"2023-11-08T19:28:01.696493+0000","last_undegraded":"2023-11-08T19:28:01.696493+0000","last_fullsized":"2023-11-08T19:28:01.696493+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:01:56.991346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064606400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.701529+0000","last_change":"2023-11-08T19:27:09.182113+0000","last_active":"2023-11-08T19:28:02.701529+0000","last_peered":"2023-11-08T19:28:02.701529+0000","last_clean":"2023-11-08T19:28:02.701529+0000","last_became_active":"2023-11-08T19:27:07.189939+0000","last_became_peered":"2023-11-08T19:27:07.189939+0000","last_unstale":"2023-11-08T19:28:02.701529+0000","last_undegraded":"2023-11-08T19:28:02.701529+0000","last_fullsized":"2023-11-08T19:28:02.701529+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:41:08.822366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036507500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.699139+0000","last_change":"2023-11-08T19:28:02.699139+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.699139+0000","last_undegraded":"2023-11-08T19:28:02.699139+0000","last_fullsized":"2023-11-08T19:28:02.699139+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.725537+0000","last_change":"2023-11-08T19:28:02.720708+0000","last_active":"2023-11-08T19:28:02.725537+0000","last_peered":"2023-11-08T19:28:02.725537+0000","last_clean":"2023-11-08T19:28:02.725537+0000","last_became_active":"2023-11-08T19:28:02.720263+0000","last_became_peered":"2023-11-08T19:28:02.720263+0000","last_unstale":"2023-11-08T19:28:02.725537+0000","last_undegraded":"2023-11-08T19:28:02.725537+0000","last_fullsized":"2023-11-08T19:28:02.725537+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:56:32.169912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.691014+0000","last_change":"2023-11-08T19:27:08.187127+0000","last_active":"2023-11-08T19:28:01.691014+0000","last_peered":"2023-11-08T19:28:01.691014+0000","last_clean":"2023-11-08T19:28:01.691014+0000","last_became_active":"2023-11-08T19:27:08.182648+0000","last_became_peered":"2023-11-08T19:27:08.182648+0000","last_unstale":"2023-11-08T19:28:01.691014+0000","last_undegraded":"2023-11-08T19:28:01.691014+0000","last_fullsized":"2023-11-08T19:28:01.691014+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:50:12.101878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":71,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.700875+0000","last_change":"2023-11-08T19:26:05.609570+0000","last_active":"2023-11-08T19:28:02.700875+0000","last_peered":"2023-11-08T19:28:02.700875+0000","last_clean":"2023-11-08T19:28:02.700875+0000","last_became_active":"2023-11-08T19:26:04.608379+0000","last_became_peered":"2023-11-08T19:26:04.608379+0000","last_unstale":"2023-11-08T19:28:02.700875+0000","last_undegraded":"2023-11-08T19:28:02.700875+0000","last_fullsized":"2023-11-08T19:28:02.700875+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:40:08.986649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000529657,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.696937+0000","last_change":"2023-11-08T19:27:09.211991+0000","last_active":"2023-11-08T19:28:01.696937+0000","last_peered":"2023-11-08T19:28:01.696937+0000","last_clean":"2023-11-08T19:28:01.696937+0000","last_became_active":"2023-11-08T19:27:07.195231+0000","last_became_peered":"2023-11-08T19:27:07.195231+0000","last_unstale":"2023-11-08T19:28:01.696937+0000","last_undegraded":"2023-11-08T19:28:01.696937+0000","last_fullsized":"2023-11-08T19:28:01.696937+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:25:14.487524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000441221,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.697661+0000","last_change":"2023-11-08T19:28:02.697661+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.697661+0000","last_undegraded":"2023-11-08T19:28:02.697661+0000","last_fullsized":"2023-11-08T19:28:02.697661+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.705119+0000","last_change":"2023-11-08T19:28:02.707543+0000","last_active":"2023-11-08T19:28:03.705119+0000","last_peered":"2023-11-08T19:28:03.705119+0000","last_clean":"2023-11-08T19:28:03.705119+0000","last_became_active":"2023-11-08T19:28:02.707034+0000","last_became_peered":"2023-11-08T19:28:02.707034+0000","last_unstale":"2023-11-08T19:28:03.705119+0000","last_undegraded":"2023-11-08T19:28:03.705119+0000","last_fullsized":"2023-11-08T19:28:03.705119+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:06:46.912027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.690189+0000","last_change":"2023-11-08T19:27:08.195364+0000","last_active":"2023-11-08T19:28:01.690189+0000","last_peered":"2023-11-08T19:28:01.690189+0000","last_clean":"2023-11-08T19:28:01.690189+0000","last_became_active":"2023-11-08T19:27:08.195074+0000","last_became_peered":"2023-11-08T19:27:08.195074+0000","last_unstale":"2023-11-08T19:28:01.690189+0000","last_undegraded":"2023-11-08T19:28:01.690189+0000","last_fullsized":"2023-11-08T19:28:01.690189+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:02:57.552873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":59,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.700695+0000","last_change":"2023-11-08T19:26:06.394300+0000","last_active":"2023-11-08T19:28:02.700695+0000","last_peered":"2023-11-08T19:28:02.700695+0000","last_clean":"2023-11-08T19:28:02.700695+0000","last_became_active":"2023-11-08T19:26:04.601685+0000","last_became_peered":"2023-11-08T19:26:04.601685+0000","last_unstale":"2023-11-08T19:28:02.700695+0000","last_undegraded":"2023-11-08T19:28:02.700695+0000","last_fullsized":"2023-11-08T19:28:02.700695+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:51:06.626474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061044099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.690413+0000","last_change":"2023-11-08T19:27:09.178466+0000","last_active":"2023-11-08T19:28:01.690413+0000","last_peered":"2023-11-08T19:28:01.690413+0000","last_clean":"2023-11-08T19:28:01.690413+0000","last_became_active":"2023-11-08T19:27:07.189386+0000","last_became_peered":"2023-11-08T19:27:07.189386+0000","last_unstale":"2023-11-08T19:28:01.690413+0000","last_undegraded":"2023-11-08T19:28:01.690413+0000","last_fullsized":"2023-11-08T19:28:01.690413+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:48:51.139820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049390100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.713309+0000","last_change":"2023-11-08T19:28:02.713309+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.713309+0000","last_undegraded":"2023-11-08T19:28:02.713309+0000","last_fullsized":"2023-11-08T19:28:02.713309+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.727972+0000","last_change":"2023-11-08T19:28:02.723021+0000","last_active":"2023-11-08T19:28:02.727972+0000","last_peered":"2023-11-08T19:28:02.727972+0000","last_clean":"2023-11-08T19:28:02.727972+0000","last_became_active":"2023-11-08T19:28:02.722748+0000","last_became_peered":"2023-11-08T19:28:02.722748+0000","last_unstale":"2023-11-08T19:28:02.727972+0000","last_undegraded":"2023-11-08T19:28:02.727972+0000","last_fullsized":"2023-11-08T19:28:02.727972+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:26:52.310010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.695135+0000","last_change":"2023-11-08T19:27:08.189452+0000","last_active":"2023-11-08T19:28:01.695135+0000","last_peered":"2023-11-08T19:28:01.695135+0000","last_clean":"2023-11-08T19:28:01.695135+0000","last_became_active":"2023-11-08T19:27:08.189088+0000","last_became_peered":"2023-11-08T19:27:08.189088+0000","last_unstale":"2023-11-08T19:28:01.695135+0000","last_undegraded":"2023-11-08T19:28:01.695135+0000","last_fullsized":"2023-11-08T19:28:01.695135+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:25:18.963220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":59,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.694898+0000","last_change":"2023-11-08T19:26:06.296634+0000","last_active":"2023-11-08T19:28:01.694898+0000","last_peered":"2023-11-08T19:28:01.694898+0000","last_clean":"2023-11-08T19:28:01.694898+0000","last_became_active":"2023-11-08T19:26:04.603088+0000","last_became_peered":"2023-11-08T19:26:04.603088+0000","last_unstale":"2023-11-08T19:28:01.694898+0000","last_undegraded":"2023-11-08T19:28:01.694898+0000","last_fullsized":"2023-11-08T19:28:01.694898+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:01:56.991346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038180899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.695512+0000","last_change":"2023-11-08T19:27:09.431220+0000","last_active":"2023-11-08T19:28:01.695512+0000","last_peered":"2023-11-08T19:28:01.695512+0000","last_clean":"2023-11-08T19:28:01.695512+0000","last_became_active":"2023-11-08T19:27:07.191589+0000","last_became_peered":"2023-11-08T19:27:07.191589+0000","last_unstale":"2023-11-08T19:28:01.695512+0000","last_undegraded":"2023-11-08T19:28:01.695512+0000","last_fullsized":"2023-11-08T19:28:01.695512+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:18:19.964487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000729527,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.714312+0000","last_change":"2023-11-08T19:28:02.714312+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.714312+0000","last_undegraded":"2023-11-08T19:28:02.714312+0000","last_fullsized":"2023-11-08T19:28:02.714312+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"91'1","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.719567+0000","last_change":"2023-11-08T19:28:02.712293+0000","last_active":"2023-11-08T19:28:03.719567+0000","last_peered":"2023-11-08T19:28:03.719567+0000","last_clean":"2023-11-08T19:28:03.719567+0000","last_became_active":"2023-11-08T19:28:02.712002+0000","last_became_peered":"2023-11-08T19:28:02.712002+0000","last_unstale":"2023-11-08T19:28:03.719567+0000","last_undegraded":"2023-11-08T19:28:03.719567+0000","last_fullsized":"2023-11-08T19:28:03.719567+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:24:47.603187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.694985+0000","last_change":"2023-11-08T19:27:08.193337+0000","last_active":"2023-11-08T19:28:01.694985+0000","last_peered":"2023-11-08T19:28:01.694985+0000","last_clean":"2023-11-08T19:28:01.694985+0000","last_became_active":"2023-11-08T19:27:08.192997+0000","last_became_peered":"2023-11-08T19:27:08.192997+0000","last_unstale":"2023-11-08T19:28:01.694985+0000","last_undegraded":"2023-11-08T19:28:01.694985+0000","last_fullsized":"2023-11-08T19:28:01.694985+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:45:22.084015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":64,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.696307+0000","last_change":"2023-11-08T19:26:06.424511+0000","last_active":"2023-11-08T19:28:01.696307+0000","last_peered":"2023-11-08T19:28:01.696307+0000","last_clean":"2023-11-08T19:28:01.696307+0000","last_became_active":"2023-11-08T19:26:04.598973+0000","last_became_peered":"2023-11-08T19:26:04.598973+0000","last_unstale":"2023-11-08T19:28:01.696307+0000","last_undegraded":"2023-11-08T19:28:01.696307+0000","last_fullsized":"2023-11-08T19:28:01.696307+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:41:53.497456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031755900000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.694476+0000","last_change":"2023-11-08T19:27:09.497024+0000","last_active":"2023-11-08T19:28:01.694476+0000","last_peered":"2023-11-08T19:28:01.694476+0000","last_clean":"2023-11-08T19:28:01.694476+0000","last_became_active":"2023-11-08T19:27:07.190132+0000","last_became_peered":"2023-11-08T19:27:07.190132+0000","last_unstale":"2023-11-08T19:28:01.694476+0000","last_undegraded":"2023-11-08T19:28:01.694476+0000","last_fullsized":"2023-11-08T19:28:01.694476+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:36:34.881217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054126000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.718435+0000","last_change":"2023-11-08T19:28:03.713184+0000","last_active":"2023-11-08T19:28:03.718435+0000","last_peered":"2023-11-08T19:28:03.718435+0000","last_clean":"2023-11-08T19:28:03.718435+0000","last_became_active":"2023-11-08T19:28:03.712864+0000","last_became_peered":"2023-11-08T19:28:03.712864+0000","last_unstale":"2023-11-08T19:28:03.718435+0000","last_undegraded":"2023-11-08T19:28:03.718435+0000","last_fullsized":"2023-11-08T19:28:03.718435+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:38:54.805747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"91'1","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.716012+0000","last_change":"2023-11-08T19:28:02.713126+0000","last_active":"2023-11-08T19:28:03.716012+0000","last_peered":"2023-11-08T19:28:03.716012+0000","last_clean":"2023-11-08T19:28:03.716012+0000","last_became_active":"2023-11-08T19:28:02.712836+0000","last_became_peered":"2023-11-08T19:28:02.712836+0000","last_unstale":"2023-11-08T19:28:03.716012+0000","last_undegraded":"2023-11-08T19:28:03.716012+0000","last_fullsized":"2023-11-08T19:28:03.716012+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:29:31.276279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.695424+0000","last_change":"2023-11-08T19:27:08.194196+0000","last_active":"2023-11-08T19:28:01.695424+0000","last_peered":"2023-11-08T19:28:01.695424+0000","last_clean":"2023-11-08T19:28:01.695424+0000","last_became_active":"2023-11-08T19:27:08.193912+0000","last_became_peered":"2023-11-08T19:27:08.193912+0000","last_unstale":"2023-11-08T19:28:01.695424+0000","last_undegraded":"2023-11-08T19:28:01.695424+0000","last_fullsized":"2023-11-08T19:28:01.695424+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:25:18.963220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":59,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.695591+0000","last_change":"2023-11-08T19:26:06.296937+0000","last_active":"2023-11-08T19:28:01.695591+0000","last_peered":"2023-11-08T19:28:01.695591+0000","last_clean":"2023-11-08T19:28:01.695591+0000","last_became_active":"2023-11-08T19:26:04.599999+0000","last_became_peered":"2023-11-08T19:26:04.599999+0000","last_unstale":"2023-11-08T19:28:01.695591+0000","last_undegraded":"2023-11-08T19:28:01.695591+0000","last_fullsized":"2023-11-08T19:28:01.695591+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:50:43.827570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025605700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.695067+0000","last_change":"2023-11-08T19:27:09.497040+0000","last_active":"2023-11-08T19:28:01.695067+0000","last_peered":"2023-11-08T19:28:01.695067+0000","last_clean":"2023-11-08T19:28:01.695067+0000","last_became_active":"2023-11-08T19:27:07.189135+0000","last_became_peered":"2023-11-08T19:27:07.189135+0000","last_unstale":"2023-11-08T19:28:01.695067+0000","last_undegraded":"2023-11-08T19:28:01.695067+0000","last_fullsized":"2023-11-08T19:28:01.695067+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:12:29.400972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051794299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.703964+0000","last_change":"2023-11-08T19:28:02.703964+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.703964+0000","last_undegraded":"2023-11-08T19:28:02.703964+0000","last_fullsized":"2023-11-08T19:28:02.703964+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.710152+0000","last_change":"2023-11-08T19:28:02.705153+0000","last_active":"2023-11-08T19:28:02.710152+0000","last_peered":"2023-11-08T19:28:02.710152+0000","last_clean":"2023-11-08T19:28:02.710152+0000","last_became_active":"2023-11-08T19:28:02.704854+0000","last_became_peered":"2023-11-08T19:28:02.704854+0000","last_unstale":"2023-11-08T19:28:02.710152+0000","last_undegraded":"2023-11-08T19:28:02.710152+0000","last_fullsized":"2023-11-08T19:28:02.710152+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:25:00.088013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.703721+0000","last_change":"2023-11-08T19:27:08.187053+0000","last_active":"2023-11-08T19:28:02.703721+0000","last_peered":"2023-11-08T19:28:02.703721+0000","last_clean":"2023-11-08T19:28:02.703721+0000","last_became_active":"2023-11-08T19:27:08.186643+0000","last_became_peered":"2023-11-08T19:27:08.186643+0000","last_unstale":"2023-11-08T19:28:02.703721+0000","last_undegraded":"2023-11-08T19:28:02.703721+0000","last_fullsized":"2023-11-08T19:28:02.703721+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:35:30.073885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":71,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.700976+0000","last_change":"2023-11-08T19:26:05.609824+0000","last_active":"2023-11-08T19:28:02.700976+0000","last_peered":"2023-11-08T19:28:02.700976+0000","last_clean":"2023-11-08T19:28:02.700976+0000","last_became_active":"2023-11-08T19:26:04.603320+0000","last_became_peered":"2023-11-08T19:26:04.603320+0000","last_unstale":"2023-11-08T19:28:02.700976+0000","last_undegraded":"2023-11-08T19:28:02.700976+0000","last_fullsized":"2023-11-08T19:28:02.700976+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:05:45.976615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063225,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.694457+0000","last_change":"2023-11-08T19:27:09.496655+0000","last_active":"2023-11-08T19:28:01.694457+0000","last_peered":"2023-11-08T19:28:01.694457+0000","last_clean":"2023-11-08T19:28:01.694457+0000","last_became_active":"2023-11-08T19:27:07.189895+0000","last_became_peered":"2023-11-08T19:27:07.189895+0000","last_unstale":"2023-11-08T19:28:01.694457+0000","last_undegraded":"2023-11-08T19:28:01.694457+0000","last_fullsized":"2023-11-08T19:28:01.694457+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:41:08.822366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045654199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.711687+0000","last_change":"2023-11-08T19:28:02.711687+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.711687+0000","last_undegraded":"2023-11-08T19:28:02.711687+0000","last_fullsized":"2023-11-08T19:28:02.711687+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.711905+0000","last_change":"2023-11-08T19:28:02.709891+0000","last_active":"2023-11-08T19:28:02.711905+0000","last_peered":"2023-11-08T19:28:02.711905+0000","last_clean":"2023-11-08T19:28:02.711905+0000","last_became_active":"2023-11-08T19:28:02.709612+0000","last_became_peered":"2023-11-08T19:28:02.709612+0000","last_unstale":"2023-11-08T19:28:02.711905+0000","last_undegraded":"2023-11-08T19:28:02.711905+0000","last_fullsized":"2023-11-08T19:28:02.711905+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:24:47.603187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":60,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.698243+0000","last_change":"2023-11-08T19:26:06.541360+0000","last_active":"2023-11-08T19:28:01.698243+0000","last_peered":"2023-11-08T19:28:01.698243+0000","last_clean":"2023-11-08T19:28:01.698243+0000","last_became_active":"2023-11-08T19:26:04.603725+0000","last_became_peered":"2023-11-08T19:26:04.603725+0000","last_unstale":"2023-11-08T19:28:01.698243+0000","last_undegraded":"2023-11-08T19:28:01.698243+0000","last_fullsized":"2023-11-08T19:28:01.698243+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:28:48.576742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053365400000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.698592+0000","last_change":"2023-11-08T19:27:09.283599+0000","last_active":"2023-11-08T19:28:01.698592+0000","last_peered":"2023-11-08T19:28:01.698592+0000","last_clean":"2023-11-08T19:28:01.698592+0000","last_became_active":"2023-11-08T19:27:07.189595+0000","last_became_peered":"2023-11-08T19:27:07.189595+0000","last_unstale":"2023-11-08T19:28:01.698592+0000","last_undegraded":"2023-11-08T19:28:01.698592+0000","last_fullsized":"2023-11-08T19:28:01.698592+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:48:20.082682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000323479,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.696394+0000","last_change":"2023-11-08T19:27:08.185395+0000","last_active":"2023-11-08T19:28:01.696394+0000","last_peered":"2023-11-08T19:28:01.696394+0000","last_clean":"2023-11-08T19:28:01.696394+0000","last_became_active":"2023-11-08T19:27:08.184765+0000","last_became_peered":"2023-11-08T19:27:08.184765+0000","last_unstale":"2023-11-08T19:28:01.696394+0000","last_undegraded":"2023-11-08T19:28:01.696394+0000","last_fullsized":"2023-11-08T19:28:01.696394+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:08:51.864418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.712306+0000","last_change":"2023-11-08T19:28:02.712306+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.712306+0000","last_undegraded":"2023-11-08T19:28:02.712306+0000","last_fullsized":"2023-11-08T19:28:02.712306+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.725515+0000","last_change":"2023-11-08T19:28:02.720596+0000","last_active":"2023-11-08T19:28:02.725515+0000","last_peered":"2023-11-08T19:28:02.725515+0000","last_clean":"2023-11-08T19:28:02.725515+0000","last_became_active":"2023-11-08T19:28:02.720144+0000","last_became_peered":"2023-11-08T19:28:02.720144+0000","last_unstale":"2023-11-08T19:28:02.725515+0000","last_undegraded":"2023-11-08T19:28:02.725515+0000","last_fullsized":"2023-11-08T19:28:02.725515+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:51:33.542656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":64,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.694624+0000","last_change":"2023-11-08T19:26:06.424236+0000","last_active":"2023-11-08T19:28:01.694624+0000","last_peered":"2023-11-08T19:28:01.694624+0000","last_clean":"2023-11-08T19:28:01.694624+0000","last_became_active":"2023-11-08T19:26:04.600115+0000","last_became_peered":"2023-11-08T19:26:04.600115+0000","last_unstale":"2023-11-08T19:28:01.694624+0000","last_undegraded":"2023-11-08T19:28:01.694624+0000","last_fullsized":"2023-11-08T19:28:01.694624+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:18:01.876605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065803900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.695283+0000","last_change":"2023-11-08T19:27:09.331899+0000","last_active":"2023-11-08T19:28:01.695283+0000","last_peered":"2023-11-08T19:28:01.695283+0000","last_clean":"2023-11-08T19:28:01.695283+0000","last_became_active":"2023-11-08T19:27:07.191460+0000","last_became_peered":"2023-11-08T19:27:07.191460+0000","last_unstale":"2023-11-08T19:28:01.695283+0000","last_undegraded":"2023-11-08T19:28:01.695283+0000","last_fullsized":"2023-11-08T19:28:01.695283+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:51:29.247104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044907500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.701089+0000","last_change":"2023-11-08T19:27:08.191802+0000","last_active":"2023-11-08T19:28:02.701089+0000","last_peered":"2023-11-08T19:28:02.701089+0000","last_clean":"2023-11-08T19:28:02.701089+0000","last_became_active":"2023-11-08T19:27:08.191520+0000","last_became_peered":"2023-11-08T19:27:08.191520+0000","last_unstale":"2023-11-08T19:28:02.701089+0000","last_undegraded":"2023-11-08T19:28:02.701089+0000","last_fullsized":"2023-11-08T19:28:02.701089+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:49:16.458634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.724704+0000","last_change":"2023-11-08T19:28:03.722086+0000","last_active":"2023-11-08T19:28:03.724704+0000","last_peered":"2023-11-08T19:28:03.724704+0000","last_clean":"2023-11-08T19:28:03.724704+0000","last_became_active":"2023-11-08T19:28:03.721713+0000","last_became_peered":"2023-11-08T19:28:03.721713+0000","last_unstale":"2023-11-08T19:28:03.724704+0000","last_undegraded":"2023-11-08T19:28:03.724704+0000","last_fullsized":"2023-11-08T19:28:03.724704+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:34:22.087376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.705286+0000","last_change":"2023-11-08T19:28:02.707105+0000","last_active":"2023-11-08T19:28:03.705286+0000","last_peered":"2023-11-08T19:28:03.705286+0000","last_clean":"2023-11-08T19:28:03.705286+0000","last_became_active":"2023-11-08T19:28:02.706615+0000","last_became_peered":"2023-11-08T19:28:02.706615+0000","last_unstale":"2023-11-08T19:28:03.705286+0000","last_undegraded":"2023-11-08T19:28:03.705286+0000","last_fullsized":"2023-11-08T19:28:03.705286+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:18:55.364855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":71,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.699281+0000","last_change":"2023-11-08T19:26:05.608892+0000","last_active":"2023-11-08T19:28:02.699281+0000","last_peered":"2023-11-08T19:28:02.699281+0000","last_clean":"2023-11-08T19:28:02.699281+0000","last_became_active":"2023-11-08T19:26:04.604519+0000","last_became_peered":"2023-11-08T19:26:04.604519+0000","last_unstale":"2023-11-08T19:28:02.699281+0000","last_undegraded":"2023-11-08T19:28:02.699281+0000","last_fullsized":"2023-11-08T19:28:02.699281+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:11:26.508634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051248099999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.700897+0000","last_change":"2023-11-08T19:27:09.181431+0000","last_active":"2023-11-08T19:28:02.700897+0000","last_peered":"2023-11-08T19:28:02.700897+0000","last_clean":"2023-11-08T19:28:02.700897+0000","last_became_active":"2023-11-08T19:27:07.190800+0000","last_became_peered":"2023-11-08T19:27:07.190800+0000","last_unstale":"2023-11-08T19:28:02.700897+0000","last_undegraded":"2023-11-08T19:28:02.700897+0000","last_fullsized":"2023-11-08T19:28:02.700897+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:46:30.452678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656099999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.696596+0000","last_change":"2023-11-08T19:27:08.187983+0000","last_active":"2023-11-08T19:28:01.696596+0000","last_peered":"2023-11-08T19:28:01.696596+0000","last_clean":"2023-11-08T19:28:01.696596+0000","last_became_active":"2023-11-08T19:27:08.187322+0000","last_became_peered":"2023-11-08T19:27:08.187322+0000","last_unstale":"2023-11-08T19:28:01.696596+0000","last_undegraded":"2023-11-08T19:28:01.696596+0000","last_fullsized":"2023-11-08T19:28:01.696596+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:33:07.647519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.704461+0000","last_change":"2023-11-08T19:28:02.704461+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.704461+0000","last_undegraded":"2023-11-08T19:28:02.704461+0000","last_fullsized":"2023-11-08T19:28:02.704461+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.725558+0000","last_change":"2023-11-08T19:28:02.720466+0000","last_active":"2023-11-08T19:28:02.725558+0000","last_peered":"2023-11-08T19:28:02.725558+0000","last_clean":"2023-11-08T19:28:02.725558+0000","last_became_active":"2023-11-08T19:28:02.720193+0000","last_became_peered":"2023-11-08T19:28:02.720193+0000","last_unstale":"2023-11-08T19:28:02.725558+0000","last_undegraded":"2023-11-08T19:28:02.725558+0000","last_fullsized":"2023-11-08T19:28:02.725558+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:26:23.134605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":58,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.694087+0000","last_change":"2023-11-08T19:26:06.394075+0000","last_active":"2023-11-08T19:28:01.694087+0000","last_peered":"2023-11-08T19:28:01.694087+0000","last_clean":"2023-11-08T19:28:01.694087+0000","last_became_active":"2023-11-08T19:26:04.606967+0000","last_became_peered":"2023-11-08T19:26:04.606967+0000","last_unstale":"2023-11-08T19:28:01.694087+0000","last_undegraded":"2023-11-08T19:28:01.694087+0000","last_fullsized":"2023-11-08T19:28:01.694087+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:19:56.286835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040971999999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.701127+0000","last_change":"2023-11-08T19:27:09.180722+0000","last_active":"2023-11-08T19:28:02.701127+0000","last_peered":"2023-11-08T19:28:02.701127+0000","last_clean":"2023-11-08T19:28:02.701127+0000","last_became_active":"2023-11-08T19:27:07.186977+0000","last_became_peered":"2023-11-08T19:27:07.186977+0000","last_unstale":"2023-11-08T19:28:02.701127+0000","last_undegraded":"2023-11-08T19:28:02.701127+0000","last_fullsized":"2023-11-08T19:28:02.701127+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:17:59.966463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057675100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.694729+0000","last_change":"2023-11-08T19:27:08.193963+0000","last_active":"2023-11-08T19:28:01.694729+0000","last_peered":"2023-11-08T19:28:01.694729+0000","last_clean":"2023-11-08T19:28:01.694729+0000","last_became_active":"2023-11-08T19:27:08.193098+0000","last_became_peered":"2023-11-08T19:27:08.193098+0000","last_unstale":"2023-11-08T19:28:01.694729+0000","last_undegraded":"2023-11-08T19:28:01.694729+0000","last_fullsized":"2023-11-08T19:28:01.694729+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:41:09.827370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.699774+0000","last_change":"2023-11-08T19:28:02.699774+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.699774+0000","last_undegraded":"2023-11-08T19:28:02.699774+0000","last_fullsized":"2023-11-08T19:28:02.699774+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.722618+0000","last_change":"2023-11-08T19:28:02.718059+0000","last_active":"2023-11-08T19:28:02.722618+0000","last_peered":"2023-11-08T19:28:02.722618+0000","last_clean":"2023-11-08T19:28:02.722618+0000","last_became_active":"2023-11-08T19:28:02.717775+0000","last_became_peered":"2023-11-08T19:28:02.717775+0000","last_unstale":"2023-11-08T19:28:02.722618+0000","last_undegraded":"2023-11-08T19:28:02.722618+0000","last_fullsized":"2023-11-08T19:28:02.722618+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:29:25.082011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":58,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.697065+0000","last_change":"2023-11-08T19:26:06.265686+0000","last_active":"2023-11-08T19:28:01.697065+0000","last_peered":"2023-11-08T19:28:01.697065+0000","last_clean":"2023-11-08T19:28:01.697065+0000","last_became_active":"2023-11-08T19:26:04.602111+0000","last_became_peered":"2023-11-08T19:26:04.602111+0000","last_unstale":"2023-11-08T19:28:01.697065+0000","last_undegraded":"2023-11-08T19:28:01.697065+0000","last_fullsized":"2023-11-08T19:28:01.697065+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:40:11.164951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056197899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.701328+0000","last_change":"2023-11-08T19:27:09.181333+0000","last_active":"2023-11-08T19:28:02.701328+0000","last_peered":"2023-11-08T19:28:02.701328+0000","last_clean":"2023-11-08T19:28:02.701328+0000","last_became_active":"2023-11-08T19:27:07.188657+0000","last_became_peered":"2023-11-08T19:27:07.188657+0000","last_unstale":"2023-11-08T19:28:02.701328+0000","last_undegraded":"2023-11-08T19:28:02.701328+0000","last_fullsized":"2023-11-08T19:28:02.701328+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:25:56.788615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001227393,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.701395+0000","last_change":"2023-11-08T19:27:08.188533+0000","last_active":"2023-11-08T19:28:02.701395+0000","last_peered":"2023-11-08T19:28:02.701395+0000","last_clean":"2023-11-08T19:28:02.701395+0000","last_became_active":"2023-11-08T19:27:08.188240+0000","last_became_peered":"2023-11-08T19:27:08.188240+0000","last_unstale":"2023-11-08T19:28:02.701395+0000","last_undegraded":"2023-11-08T19:28:02.701395+0000","last_fullsized":"2023-11-08T19:28:02.701395+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:55:37.653521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.701682+0000","last_change":"2023-11-08T19:28:02.701682+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.701682+0000","last_undegraded":"2023-11-08T19:28:02.701682+0000","last_fullsized":"2023-11-08T19:28:02.701682+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.714684+0000","last_change":"2023-11-08T19:28:02.710864+0000","last_active":"2023-11-08T19:28:02.714684+0000","last_peered":"2023-11-08T19:28:02.714684+0000","last_clean":"2023-11-08T19:28:02.714684+0000","last_became_active":"2023-11-08T19:28:02.710597+0000","last_became_peered":"2023-11-08T19:28:02.710597+0000","last_unstale":"2023-11-08T19:28:02.714684+0000","last_undegraded":"2023-11-08T19:28:02.714684+0000","last_fullsized":"2023-11-08T19:28:02.714684+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:02:43.242472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.702305+0000","last_change":"2023-11-08T19:27:08.188458+0000","last_active":"2023-11-08T19:28:02.702305+0000","last_peered":"2023-11-08T19:28:02.702305+0000","last_clean":"2023-11-08T19:28:02.702305+0000","last_became_active":"2023-11-08T19:27:08.187880+0000","last_became_peered":"2023-11-08T19:27:08.187880+0000","last_unstale":"2023-11-08T19:28:02.702305+0000","last_undegraded":"2023-11-08T19:28:02.702305+0000","last_fullsized":"2023-11-08T19:28:02.702305+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:25:28.618214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":59,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.690530+0000","last_change":"2023-11-08T19:26:05.606898+0000","last_active":"2023-11-08T19:28:01.690530+0000","last_peered":"2023-11-08T19:28:01.690530+0000","last_clean":"2023-11-08T19:28:01.690530+0000","last_became_active":"2023-11-08T19:26:04.603955+0000","last_became_peered":"2023-11-08T19:26:04.603955+0000","last_unstale":"2023-11-08T19:28:01.690530+0000","last_undegraded":"2023-11-08T19:28:01.690530+0000","last_fullsized":"2023-11-08T19:28:01.690530+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:19:11.589559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036473499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.690241+0000","last_change":"2023-11-08T19:27:09.178770+0000","last_active":"2023-11-08T19:28:01.690241+0000","last_peered":"2023-11-08T19:28:01.690241+0000","last_clean":"2023-11-08T19:28:01.690241+0000","last_became_active":"2023-11-08T19:27:07.194992+0000","last_became_peered":"2023-11-08T19:27:07.194992+0000","last_unstale":"2023-11-08T19:28:01.690241+0000","last_undegraded":"2023-11-08T19:28:01.690241+0000","last_fullsized":"2023-11-08T19:28:01.690241+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:35:34.026910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000770285,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.697682+0000","last_change":"2023-11-08T19:28:02.697682+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.697682+0000","last_undegraded":"2023-11-08T19:28:02.697682+0000","last_fullsized":"2023-11-08T19:28:02.697682+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.703765+0000","last_change":"2023-11-08T19:28:02.707011+0000","last_active":"2023-11-08T19:28:03.703765+0000","last_peered":"2023-11-08T19:28:03.703765+0000","last_clean":"2023-11-08T19:28:03.703765+0000","last_became_active":"2023-11-08T19:28:02.706729+0000","last_became_peered":"2023-11-08T19:28:02.706729+0000","last_unstale":"2023-11-08T19:28:03.703765+0000","last_undegraded":"2023-11-08T19:28:03.703765+0000","last_fullsized":"2023-11-08T19:28:03.703765+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:29:25.082011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.695698+0000","last_change":"2023-11-08T19:27:08.193953+0000","last_active":"2023-11-08T19:28:01.695698+0000","last_peered":"2023-11-08T19:28:01.695698+0000","last_clean":"2023-11-08T19:28:01.695698+0000","last_became_active":"2023-11-08T19:27:08.193653+0000","last_became_peered":"2023-11-08T19:27:08.193653+0000","last_unstale":"2023-11-08T19:28:01.695698+0000","last_undegraded":"2023-11-08T19:28:01.695698+0000","last_fullsized":"2023-11-08T19:28:01.695698+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:35:30.073885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":64,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.694881+0000","last_change":"2023-11-08T19:26:06.424464+0000","last_active":"2023-11-08T19:28:01.694881+0000","last_peered":"2023-11-08T19:28:01.694881+0000","last_clean":"2023-11-08T19:28:01.694881+0000","last_became_active":"2023-11-08T19:26:04.603144+0000","last_became_peered":"2023-11-08T19:26:04.603144+0000","last_unstale":"2023-11-08T19:28:01.694881+0000","last_undegraded":"2023-11-08T19:28:01.694881+0000","last_fullsized":"2023-11-08T19:28:01.694881+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:42:55.269365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054090799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.701001+0000","last_change":"2023-11-08T19:27:09.180479+0000","last_active":"2023-11-08T19:28:02.701001+0000","last_peered":"2023-11-08T19:28:02.701001+0000","last_clean":"2023-11-08T19:28:02.701001+0000","last_became_active":"2023-11-08T19:27:07.190438+0000","last_became_peered":"2023-11-08T19:27:07.190438+0000","last_unstale":"2023-11-08T19:28:02.701001+0000","last_undegraded":"2023-11-08T19:28:02.701001+0000","last_fullsized":"2023-11-08T19:28:02.701001+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:25:14.487524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050893300000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.704857+0000","last_change":"2023-11-08T19:28:02.704857+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.704857+0000","last_undegraded":"2023-11-08T19:28:02.704857+0000","last_fullsized":"2023-11-08T19:28:02.704857+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.715287+0000","last_change":"2023-11-08T19:28:02.708336+0000","last_active":"2023-11-08T19:28:02.715287+0000","last_peered":"2023-11-08T19:28:02.715287+0000","last_clean":"2023-11-08T19:28:02.715287+0000","last_became_active":"2023-11-08T19:28:02.708016+0000","last_became_peered":"2023-11-08T19:28:02.708016+0000","last_unstale":"2023-11-08T19:28:02.715287+0000","last_undegraded":"2023-11-08T19:28:02.715287+0000","last_fullsized":"2023-11-08T19:28:02.715287+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:39:02.748118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.696341+0000","last_change":"2023-11-08T19:27:08.190530+0000","last_active":"2023-11-08T19:28:01.696341+0000","last_peered":"2023-11-08T19:28:01.696341+0000","last_clean":"2023-11-08T19:28:01.696341+0000","last_became_active":"2023-11-08T19:27:08.190203+0000","last_became_peered":"2023-11-08T19:27:08.190203+0000","last_unstale":"2023-11-08T19:28:01.696341+0000","last_undegraded":"2023-11-08T19:28:01.696341+0000","last_fullsized":"2023-11-08T19:28:01.696341+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:26:32.777602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":60,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.698474+0000","last_change":"2023-11-08T19:26:06.541623+0000","last_active":"2023-11-08T19:28:01.698474+0000","last_peered":"2023-11-08T19:28:01.698474+0000","last_clean":"2023-11-08T19:28:01.698474+0000","last_became_active":"2023-11-08T19:26:04.601961+0000","last_became_peered":"2023-11-08T19:26:04.601961+0000","last_unstale":"2023-11-08T19:28:01.698474+0000","last_undegraded":"2023-11-08T19:28:01.698474+0000","last_fullsized":"2023-11-08T19:28:01.698474+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:11:26.508634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000466386,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.694709+0000","last_change":"2023-11-08T19:27:09.431231+0000","last_active":"2023-11-08T19:28:01.694709+0000","last_peered":"2023-11-08T19:28:01.694709+0000","last_clean":"2023-11-08T19:28:01.694709+0000","last_became_active":"2023-11-08T19:27:07.191583+0000","last_became_peered":"2023-11-08T19:27:07.191583+0000","last_unstale":"2023-11-08T19:28:01.694709+0000","last_undegraded":"2023-11-08T19:28:01.694709+0000","last_fullsized":"2023-11-08T19:28:01.694709+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:58:48.357266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093695200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.698528+0000","last_change":"2023-11-08T19:28:02.698528+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.698528+0000","last_undegraded":"2023-11-08T19:28:02.698528+0000","last_fullsized":"2023-11-08T19:28:02.698528+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"91'1","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.717319+0000","last_change":"2023-11-08T19:28:02.708326+0000","last_active":"2023-11-08T19:28:03.717319+0000","last_peered":"2023-11-08T19:28:03.717319+0000","last_clean":"2023-11-08T19:28:03.717319+0000","last_became_active":"2023-11-08T19:28:02.708055+0000","last_became_peered":"2023-11-08T19:28:02.708055+0000","last_unstale":"2023-11-08T19:28:03.717319+0000","last_undegraded":"2023-11-08T19:28:03.717319+0000","last_fullsized":"2023-11-08T19:28:03.717319+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:50:41.723104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.694787+0000","last_change":"2023-11-08T19:27:08.187900+0000","last_active":"2023-11-08T19:28:01.694787+0000","last_peered":"2023-11-08T19:28:01.694787+0000","last_clean":"2023-11-08T19:28:01.694787+0000","last_became_active":"2023-11-08T19:27:08.187626+0000","last_became_peered":"2023-11-08T19:27:08.187626+0000","last_unstale":"2023-11-08T19:28:01.694787+0000","last_undegraded":"2023-11-08T19:28:01.694787+0000","last_fullsized":"2023-11-08T19:28:01.694787+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:00:55.878397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":71,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.701751+0000","last_change":"2023-11-08T19:26:05.608980+0000","last_active":"2023-11-08T19:28:02.701751+0000","last_peered":"2023-11-08T19:28:02.701751+0000","last_clean":"2023-11-08T19:28:02.701751+0000","last_became_active":"2023-11-08T19:26:04.606386+0000","last_became_peered":"2023-11-08T19:26:04.606386+0000","last_unstale":"2023-11-08T19:28:02.701751+0000","last_undegraded":"2023-11-08T19:28:02.701751+0000","last_fullsized":"2023-11-08T19:28:02.701751+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:41:50.771457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037950599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.690460+0000","last_change":"2023-11-08T19:27:09.179037+0000","last_active":"2023-11-08T19:28:01.690460+0000","last_peered":"2023-11-08T19:28:01.690460+0000","last_clean":"2023-11-08T19:28:01.690460+0000","last_became_active":"2023-11-08T19:27:07.195033+0000","last_became_peered":"2023-11-08T19:27:07.195033+0000","last_unstale":"2023-11-08T19:28:01.690460+0000","last_undegraded":"2023-11-08T19:28:01.690460+0000","last_fullsized":"2023-11-08T19:28:01.690460+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:19:08.253483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041824800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.697868+0000","last_change":"2023-11-08T19:28:02.697868+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.697868+0000","last_undegraded":"2023-11-08T19:28:02.697868+0000","last_fullsized":"2023-11-08T19:28:02.697868+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.713986+0000","last_change":"2023-11-08T19:28:02.712619+0000","last_active":"2023-11-08T19:28:02.713986+0000","last_peered":"2023-11-08T19:28:02.713986+0000","last_clean":"2023-11-08T19:28:02.713986+0000","last_became_active":"2023-11-08T19:28:02.712342+0000","last_became_peered":"2023-11-08T19:28:02.712342+0000","last_unstale":"2023-11-08T19:28:02.713986+0000","last_undegraded":"2023-11-08T19:28:02.713986+0000","last_fullsized":"2023-11-08T19:28:02.713986+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:11:10.165281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.693976+0000","last_change":"2023-11-08T19:27:08.193761+0000","last_active":"2023-11-08T19:28:01.693976+0000","last_peered":"2023-11-08T19:28:01.693976+0000","last_clean":"2023-11-08T19:28:01.693976+0000","last_became_active":"2023-11-08T19:27:08.193017+0000","last_became_peered":"2023-11-08T19:27:08.193017+0000","last_unstale":"2023-11-08T19:28:01.693976+0000","last_undegraded":"2023-11-08T19:28:01.693976+0000","last_fullsized":"2023-11-08T19:28:01.693976+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:18:00.971466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":64,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.694700+0000","last_change":"2023-11-08T19:26:06.424073+0000","last_active":"2023-11-08T19:28:01.694700+0000","last_peered":"2023-11-08T19:28:01.694700+0000","last_clean":"2023-11-08T19:28:01.694700+0000","last_became_active":"2023-11-08T19:26:04.599046+0000","last_became_peered":"2023-11-08T19:26:04.599046+0000","last_unstale":"2023-11-08T19:28:01.694700+0000","last_undegraded":"2023-11-08T19:28:01.694700+0000","last_fullsized":"2023-11-08T19:28:01.694700+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:04:43.562566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035092199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.700565+0000","last_change":"2023-11-08T19:27:09.181716+0000","last_active":"2023-11-08T19:28:02.700565+0000","last_peered":"2023-11-08T19:28:02.700565+0000","last_clean":"2023-11-08T19:28:02.700565+0000","last_became_active":"2023-11-08T19:27:07.187077+0000","last_became_peered":"2023-11-08T19:27:07.187077+0000","last_unstale":"2023-11-08T19:28:02.700565+0000","last_undegraded":"2023-11-08T19:28:02.700565+0000","last_fullsized":"2023-11-08T19:28:02.700565+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:51:29.247104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056673599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.726360+0000","last_change":"2023-11-08T19:28:03.724351+0000","last_active":"2023-11-08T19:28:03.726360+0000","last_peered":"2023-11-08T19:28:03.726360+0000","last_clean":"2023-11-08T19:28:03.726360+0000","last_became_active":"2023-11-08T19:28:03.724109+0000","last_became_peered":"2023-11-08T19:28:03.724109+0000","last_unstale":"2023-11-08T19:28:03.726360+0000","last_undegraded":"2023-11-08T19:28:03.726360+0000","last_fullsized":"2023-11-08T19:28:03.726360+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:11:11.170102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.716963+0000","last_change":"2023-11-08T19:28:02.711594+0000","last_active":"2023-11-08T19:28:02.716963+0000","last_peered":"2023-11-08T19:28:02.716963+0000","last_clean":"2023-11-08T19:28:02.716963+0000","last_became_active":"2023-11-08T19:28:02.711204+0000","last_became_peered":"2023-11-08T19:28:02.711204+0000","last_unstale":"2023-11-08T19:28:02.716963+0000","last_undegraded":"2023-11-08T19:28:02.716963+0000","last_fullsized":"2023-11-08T19:28:02.716963+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:21:23.673336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.696597+0000","last_change":"2023-11-08T19:27:08.188143+0000","last_active":"2023-11-08T19:28:01.696597+0000","last_peered":"2023-11-08T19:28:01.696597+0000","last_clean":"2023-11-08T19:28:01.696597+0000","last_became_active":"2023-11-08T19:27:08.187714+0000","last_became_peered":"2023-11-08T19:27:08.187714+0000","last_unstale":"2023-11-08T19:28:01.696597+0000","last_undegraded":"2023-11-08T19:28:01.696597+0000","last_fullsized":"2023-11-08T19:28:01.696597+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:02:10.863403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":58,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.693970+0000","last_change":"2023-11-08T19:26:06.394320+0000","last_active":"2023-11-08T19:28:01.693970+0000","last_peered":"2023-11-08T19:28:01.693970+0000","last_clean":"2023-11-08T19:28:01.693970+0000","last_became_active":"2023-11-08T19:26:04.603212+0000","last_became_peered":"2023-11-08T19:26:04.603212+0000","last_unstale":"2023-11-08T19:28:01.693970+0000","last_undegraded":"2023-11-08T19:28:01.693970+0000","last_fullsized":"2023-11-08T19:28:01.693970+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:25:29.947079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054695399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.700819+0000","last_change":"2023-11-08T19:27:09.181613+0000","last_active":"2023-11-08T19:28:02.700819+0000","last_peered":"2023-11-08T19:28:02.700819+0000","last_clean":"2023-11-08T19:28:02.700819+0000","last_became_active":"2023-11-08T19:27:07.190897+0000","last_became_peered":"2023-11-08T19:27:07.190897+0000","last_unstale":"2023-11-08T19:28:02.700819+0000","last_undegraded":"2023-11-08T19:28:02.700819+0000","last_fullsized":"2023-11-08T19:28:02.700819+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:27:57.388125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011172459999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.698922+0000","last_change":"2023-11-08T19:28:02.698922+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.698922+0000","last_undegraded":"2023-11-08T19:28:02.698922+0000","last_fullsized":"2023-11-08T19:28:02.698922+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.711981+0000","last_change":"2023-11-08T19:28:02.707358+0000","last_active":"2023-11-08T19:28:02.711981+0000","last_peered":"2023-11-08T19:28:02.711981+0000","last_clean":"2023-11-08T19:28:02.711981+0000","last_became_active":"2023-11-08T19:28:02.707109+0000","last_became_peered":"2023-11-08T19:28:02.707109+0000","last_unstale":"2023-11-08T19:28:02.711981+0000","last_undegraded":"2023-11-08T19:28:02.711981+0000","last_fullsized":"2023-11-08T19:28:02.711981+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:32:42.691100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.696667+0000","last_change":"2023-11-08T19:27:08.188668+0000","last_active":"2023-11-08T19:28:01.696667+0000","last_peered":"2023-11-08T19:28:01.696667+0000","last_clean":"2023-11-08T19:28:01.696667+0000","last_became_active":"2023-11-08T19:27:08.188441+0000","last_became_peered":"2023-11-08T19:27:08.188441+0000","last_unstale":"2023-11-08T19:28:01.696667+0000","last_undegraded":"2023-11-08T19:28:01.696667+0000","last_fullsized":"2023-11-08T19:28:01.696667+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:45:44.379160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":64,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.694995+0000","last_change":"2023-11-08T19:26:06.423754+0000","last_active":"2023-11-08T19:28:01.694995+0000","last_peered":"2023-11-08T19:28:01.694995+0000","last_clean":"2023-11-08T19:28:01.694995+0000","last_became_active":"2023-11-08T19:26:04.599322+0000","last_became_peered":"2023-11-08T19:26:04.599322+0000","last_unstale":"2023-11-08T19:28:01.694995+0000","last_undegraded":"2023-11-08T19:28:01.694995+0000","last_fullsized":"2023-11-08T19:28:01.694995+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:13:08.689128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042670000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.701576+0000","last_change":"2023-11-08T19:27:09.181656+0000","last_active":"2023-11-08T19:28:02.701576+0000","last_peered":"2023-11-08T19:28:02.701576+0000","last_clean":"2023-11-08T19:28:02.701576+0000","last_became_active":"2023-11-08T19:27:07.187337+0000","last_became_peered":"2023-11-08T19:27:07.187337+0000","last_unstale":"2023-11-08T19:28:02.701576+0000","last_undegraded":"2023-11-08T19:28:02.701576+0000","last_fullsized":"2023-11-08T19:28:02.701576+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:14:17.605057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000346967,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.698224+0000","last_change":"2023-11-08T19:28:02.698224+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.698224+0000","last_undegraded":"2023-11-08T19:28:02.698224+0000","last_fullsized":"2023-11-08T19:28:02.698224+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.712011+0000","last_change":"2023-11-08T19:28:02.704760+0000","last_active":"2023-11-08T19:28:02.712011+0000","last_peered":"2023-11-08T19:28:02.712011+0000","last_clean":"2023-11-08T19:28:02.712011+0000","last_became_active":"2023-11-08T19:28:02.704410+0000","last_became_peered":"2023-11-08T19:28:02.704410+0000","last_unstale":"2023-11-08T19:28:02.712011+0000","last_undegraded":"2023-11-08T19:28:02.712011+0000","last_fullsized":"2023-11-08T19:28:02.712011+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:00:10.329151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.694901+0000","last_change":"2023-11-08T19:27:08.194065+0000","last_active":"2023-11-08T19:28:01.694901+0000","last_peered":"2023-11-08T19:28:01.694901+0000","last_clean":"2023-11-08T19:28:01.694901+0000","last_became_active":"2023-11-08T19:27:08.193532+0000","last_became_peered":"2023-11-08T19:27:08.193532+0000","last_unstale":"2023-11-08T19:28:01.694901+0000","last_undegraded":"2023-11-08T19:28:01.694901+0000","last_fullsized":"2023-11-08T19:28:01.694901+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:27:58.393129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":58,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.695354+0000","last_change":"2023-11-08T19:26:06.394488+0000","last_active":"2023-11-08T19:28:01.695354+0000","last_peered":"2023-11-08T19:28:01.695354+0000","last_clean":"2023-11-08T19:28:01.695354+0000","last_became_active":"2023-11-08T19:26:04.606805+0000","last_became_peered":"2023-11-08T19:26:04.606805+0000","last_unstale":"2023-11-08T19:28:01.695354+0000","last_undegraded":"2023-11-08T19:28:01.695354+0000","last_fullsized":"2023-11-08T19:28:01.695354+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:25:20.857133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045317000000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.696700+0000","last_change":"2023-11-08T19:27:09.211647+0000","last_active":"2023-11-08T19:28:01.696700+0000","last_peered":"2023-11-08T19:28:01.696700+0000","last_clean":"2023-11-08T19:28:01.696700+0000","last_became_active":"2023-11-08T19:27:07.190309+0000","last_became_peered":"2023-11-08T19:27:07.190309+0000","last_unstale":"2023-11-08T19:28:01.696700+0000","last_undegraded":"2023-11-08T19:28:01.696700+0000","last_fullsized":"2023-11-08T19:28:01.696700+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:56:04.292154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047615199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.711664+0000","last_change":"2023-11-08T19:28:02.711664+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.711664+0000","last_undegraded":"2023-11-08T19:28:02.711664+0000","last_fullsized":"2023-11-08T19:28:02.711664+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.722917+0000","last_change":"2023-11-08T19:28:02.717508+0000","last_active":"2023-11-08T19:28:02.722917+0000","last_peered":"2023-11-08T19:28:02.722917+0000","last_clean":"2023-11-08T19:28:02.722917+0000","last_became_active":"2023-11-08T19:28:02.717126+0000","last_became_peered":"2023-11-08T19:28:02.717126+0000","last_unstale":"2023-11-08T19:28:02.722917+0000","last_undegraded":"2023-11-08T19:28:02.722917+0000","last_fullsized":"2023-11-08T19:28:02.722917+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:23:34.816577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.695560+0000","last_change":"2023-11-08T19:27:08.194884+0000","last_active":"2023-11-08T19:28:01.695560+0000","last_peered":"2023-11-08T19:28:01.695560+0000","last_clean":"2023-11-08T19:28:01.695560+0000","last_became_active":"2023-11-08T19:27:08.193241+0000","last_became_peered":"2023-11-08T19:27:08.193241+0000","last_unstale":"2023-11-08T19:28:01.695560+0000","last_undegraded":"2023-11-08T19:28:01.695560+0000","last_fullsized":"2023-11-08T19:28:01.695560+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:46:31.457682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":58,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.696871+0000","last_change":"2023-11-08T19:26:06.265907+0000","last_active":"2023-11-08T19:28:01.696871+0000","last_peered":"2023-11-08T19:28:01.696871+0000","last_clean":"2023-11-08T19:28:01.696871+0000","last_became_active":"2023-11-08T19:26:04.604772+0000","last_became_peered":"2023-11-08T19:26:04.604772+0000","last_unstale":"2023-11-08T19:28:01.696871+0000","last_undegraded":"2023-11-08T19:28:01.696871+0000","last_fullsized":"2023-11-08T19:28:01.696871+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:44:16.139794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066875300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.698318+0000","last_change":"2023-11-08T19:27:09.282699+0000","last_active":"2023-11-08T19:28:01.698318+0000","last_peered":"2023-11-08T19:28:01.698318+0000","last_clean":"2023-11-08T19:28:01.698318+0000","last_became_active":"2023-11-08T19:27:07.191212+0000","last_became_peered":"2023-11-08T19:27:07.191212+0000","last_unstale":"2023-11-08T19:28:01.698318+0000","last_undegraded":"2023-11-08T19:28:01.698318+0000","last_fullsized":"2023-11-08T19:28:01.698318+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:35:14.715110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046640800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.705457+0000","last_change":"2023-11-08T19:28:02.705457+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.705457+0000","last_undegraded":"2023-11-08T19:28:02.705457+0000","last_fullsized":"2023-11-08T19:28:02.705457+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.717068+0000","last_change":"2023-11-08T19:28:02.712356+0000","last_active":"2023-11-08T19:28:02.717068+0000","last_peered":"2023-11-08T19:28:02.717068+0000","last_clean":"2023-11-08T19:28:02.717068+0000","last_became_active":"2023-11-08T19:28:02.712073+0000","last_became_peered":"2023-11-08T19:28:02.712073+0000","last_unstale":"2023-11-08T19:28:02.717068+0000","last_undegraded":"2023-11-08T19:28:02.717068+0000","last_fullsized":"2023-11-08T19:28:02.717068+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:06:46.912027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.696801+0000","last_change":"2023-11-08T19:27:08.187644+0000","last_active":"2023-11-08T19:28:01.696801+0000","last_peered":"2023-11-08T19:28:01.696801+0000","last_clean":"2023-11-08T19:28:01.696801+0000","last_became_active":"2023-11-08T19:27:08.187376+0000","last_became_peered":"2023-11-08T19:27:08.187376+0000","last_unstale":"2023-11-08T19:28:01.696801+0000","last_undegraded":"2023-11-08T19:28:01.696801+0000","last_fullsized":"2023-11-08T19:28:01.696801+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:22:40.300185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":59,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.690307+0000","last_change":"2023-11-08T19:26:05.606946+0000","last_active":"2023-11-08T19:28:01.690307+0000","last_peered":"2023-11-08T19:28:01.690307+0000","last_clean":"2023-11-08T19:28:01.690307+0000","last_became_active":"2023-11-08T19:26:04.598775+0000","last_became_peered":"2023-11-08T19:26:04.598775+0000","last_unstale":"2023-11-08T19:28:01.690307+0000","last_undegraded":"2023-11-08T19:28:01.690307+0000","last_fullsized":"2023-11-08T19:28:01.690307+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:20:56.898927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049870399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.693887+0000","last_change":"2023-11-08T19:27:09.331646+0000","last_active":"2023-11-08T19:28:01.693887+0000","last_peered":"2023-11-08T19:28:01.693887+0000","last_clean":"2023-11-08T19:28:01.693887+0000","last_became_active":"2023-11-08T19:27:07.190619+0000","last_became_peered":"2023-11-08T19:27:07.190619+0000","last_unstale":"2023-11-08T19:28:01.693887+0000","last_undegraded":"2023-11-08T19:28:01.693887+0000","last_fullsized":"2023-11-08T19:28:01.693887+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:42:47.371458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065918400000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.702920+0000","last_change":"2023-11-08T19:28:02.702920+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.702920+0000","last_undegraded":"2023-11-08T19:28:02.702920+0000","last_fullsized":"2023-11-08T19:28:02.702920+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.719716+0000","last_change":"2023-11-08T19:28:02.710012+0000","last_active":"2023-11-08T19:28:02.719716+0000","last_peered":"2023-11-08T19:28:02.719716+0000","last_clean":"2023-11-08T19:28:02.719716+0000","last_became_active":"2023-11-08T19:28:02.709715+0000","last_became_peered":"2023-11-08T19:28:02.709715+0000","last_unstale":"2023-11-08T19:28:02.719716+0000","last_undegraded":"2023-11-08T19:28:02.719716+0000","last_fullsized":"2023-11-08T19:28:02.719716+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:23:11.877422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.703535+0000","last_change":"2023-11-08T19:27:08.183798+0000","last_active":"2023-11-08T19:28:02.703535+0000","last_peered":"2023-11-08T19:28:02.703535+0000","last_clean":"2023-11-08T19:28:02.703535+0000","last_became_active":"2023-11-08T19:27:08.183537+0000","last_became_peered":"2023-11-08T19:27:08.183537+0000","last_unstale":"2023-11-08T19:28:02.703535+0000","last_undegraded":"2023-11-08T19:28:02.703535+0000","last_fullsized":"2023-11-08T19:28:02.703535+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:50:39.026265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":71,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.701309+0000","last_change":"2023-11-08T19:26:05.610012+0000","last_active":"2023-11-08T19:28:02.701309+0000","last_peered":"2023-11-08T19:28:02.701309+0000","last_clean":"2023-11-08T19:28:02.701309+0000","last_became_active":"2023-11-08T19:26:04.603209+0000","last_became_peered":"2023-11-08T19:26:04.603209+0000","last_unstale":"2023-11-08T19:28:02.701309+0000","last_undegraded":"2023-11-08T19:28:02.701309+0000","last_fullsized":"2023-11-08T19:28:02.701309+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:35:02.306997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053335100000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.690186+0000","last_change":"2023-11-08T19:27:09.178405+0000","last_active":"2023-11-08T19:28:01.690186+0000","last_peered":"2023-11-08T19:28:01.690186+0000","last_clean":"2023-11-08T19:28:01.690186+0000","last_became_active":"2023-11-08T19:27:07.188169+0000","last_became_peered":"2023-11-08T19:27:07.188169+0000","last_unstale":"2023-11-08T19:28:01.690186+0000","last_undegraded":"2023-11-08T19:28:01.690186+0000","last_fullsized":"2023-11-08T19:28:01.690186+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:39:25.855467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056209700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.703242+0000","last_change":"2023-11-08T19:28:02.703242+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.703242+0000","last_undegraded":"2023-11-08T19:28:02.703242+0000","last_fullsized":"2023-11-08T19:28:02.703242+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.722598+0000","last_change":"2023-11-08T19:28:02.716988+0000","last_active":"2023-11-08T19:28:02.722598+0000","last_peered":"2023-11-08T19:28:02.722598+0000","last_clean":"2023-11-08T19:28:02.722598+0000","last_became_active":"2023-11-08T19:28:02.716687+0000","last_became_peered":"2023-11-08T19:28:02.716687+0000","last_unstale":"2023-11-08T19:28:02.722598+0000","last_undegraded":"2023-11-08T19:28:02.722598+0000","last_fullsized":"2023-11-08T19:28:02.722598+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:21:23.673336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.693769+0000","last_change":"2023-11-08T19:27:08.193508+0000","last_active":"2023-11-08T19:28:01.693769+0000","last_peered":"2023-11-08T19:28:01.693769+0000","last_clean":"2023-11-08T19:28:01.693769+0000","last_became_active":"2023-11-08T19:27:08.193037+0000","last_became_peered":"2023-11-08T19:27:08.193037+0000","last_unstale":"2023-11-08T19:28:01.693769+0000","last_undegraded":"2023-11-08T19:28:01.693769+0000","last_fullsized":"2023-11-08T19:28:01.693769+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:29:50.067887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":58,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.696488+0000","last_change":"2023-11-08T19:26:06.265318+0000","last_active":"2023-11-08T19:28:01.696488+0000","last_peered":"2023-11-08T19:28:01.696488+0000","last_clean":"2023-11-08T19:28:01.696488+0000","last_became_active":"2023-11-08T19:26:04.603409+0000","last_became_peered":"2023-11-08T19:26:04.603409+0000","last_unstale":"2023-11-08T19:28:01.696488+0000","last_undegraded":"2023-11-08T19:28:01.696488+0000","last_fullsized":"2023-11-08T19:28:01.696488+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:27:24.933111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065684300000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.701449+0000","last_change":"2023-11-08T19:27:09.182282+0000","last_active":"2023-11-08T19:28:02.701449+0000","last_peered":"2023-11-08T19:28:02.701449+0000","last_clean":"2023-11-08T19:28:02.701449+0000","last_became_active":"2023-11-08T19:27:07.186944+0000","last_became_peered":"2023-11-08T19:27:07.186944+0000","last_unstale":"2023-11-08T19:28:02.701449+0000","last_undegraded":"2023-11-08T19:28:02.701449+0000","last_fullsized":"2023-11-08T19:28:02.701449+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:45:21.079011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00028665599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.720726+0000","last_change":"2023-11-08T19:28:03.716823+0000","last_active":"2023-11-08T19:28:03.720726+0000","last_peered":"2023-11-08T19:28:03.720726+0000","last_clean":"2023-11-08T19:28:03.720726+0000","last_became_active":"2023-11-08T19:28:03.716545+0000","last_became_peered":"2023-11-08T19:28:03.716545+0000","last_unstale":"2023-11-08T19:28:03.720726+0000","last_undegraded":"2023-11-08T19:28:03.720726+0000","last_fullsized":"2023-11-08T19:28:03.720726+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:39:03.752939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.712391+0000","last_change":"2023-11-08T19:28:02.703978+0000","last_active":"2023-11-08T19:28:02.712391+0000","last_peered":"2023-11-08T19:28:02.712391+0000","last_clean":"2023-11-08T19:28:02.712391+0000","last_became_active":"2023-11-08T19:28:02.703654+0000","last_became_peered":"2023-11-08T19:28:02.703654+0000","last_unstale":"2023-11-08T19:28:02.712391+0000","last_undegraded":"2023-11-08T19:28:02.712391+0000","last_fullsized":"2023-11-08T19:28:02.712391+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:39:48.728129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.698064+0000","last_change":"2023-11-08T19:27:08.193990+0000","last_active":"2023-11-08T19:28:01.698064+0000","last_peered":"2023-11-08T19:28:01.698064+0000","last_clean":"2023-11-08T19:28:01.698064+0000","last_became_active":"2023-11-08T19:27:08.193624+0000","last_became_peered":"2023-11-08T19:27:08.193624+0000","last_unstale":"2023-11-08T19:28:01.698064+0000","last_undegraded":"2023-11-08T19:28:01.698064+0000","last_fullsized":"2023-11-08T19:28:01.698064+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:46:41.321120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":64,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.694552+0000","last_change":"2023-11-08T19:26:06.423863+0000","last_active":"2023-11-08T19:28:01.694552+0000","last_peered":"2023-11-08T19:28:01.694552+0000","last_clean":"2023-11-08T19:28:01.694552+0000","last_became_active":"2023-11-08T19:26:04.608639+0000","last_became_peered":"2023-11-08T19:26:04.608639+0000","last_unstale":"2023-11-08T19:28:01.694552+0000","last_undegraded":"2023-11-08T19:28:01.694552+0000","last_fullsized":"2023-11-08T19:28:01.694552+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:04:43.631102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044384999999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.698449+0000","last_change":"2023-11-08T19:27:09.283232+0000","last_active":"2023-11-08T19:28:01.698449+0000","last_peered":"2023-11-08T19:28:01.698449+0000","last_clean":"2023-11-08T19:28:01.698449+0000","last_became_active":"2023-11-08T19:27:07.190931+0000","last_became_peered":"2023-11-08T19:27:07.190931+0000","last_unstale":"2023-11-08T19:28:01.698449+0000","last_undegraded":"2023-11-08T19:28:01.698449+0000","last_fullsized":"2023-11-08T19:28:01.698449+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:43:28.225967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034146799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.713166+0000","last_change":"2023-11-08T19:28:02.713166+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.713166+0000","last_undegraded":"2023-11-08T19:28:02.713166+0000","last_fullsized":"2023-11-08T19:28:02.713166+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.722936+0000","last_change":"2023-11-08T19:28:02.717768+0000","last_active":"2023-11-08T19:28:02.722936+0000","last_peered":"2023-11-08T19:28:02.722936+0000","last_clean":"2023-11-08T19:28:02.722936+0000","last_became_active":"2023-11-08T19:28:02.717553+0000","last_became_peered":"2023-11-08T19:28:02.717553+0000","last_unstale":"2023-11-08T19:28:02.722936+0000","last_undegraded":"2023-11-08T19:28:02.722936+0000","last_fullsized":"2023-11-08T19:28:02.722936+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:50:10.975026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.694667+0000","last_change":"2023-11-08T19:27:08.187252+0000","last_active":"2023-11-08T19:28:01.694667+0000","last_peered":"2023-11-08T19:28:01.694667+0000","last_clean":"2023-11-08T19:28:01.694667+0000","last_became_active":"2023-11-08T19:27:08.186952+0000","last_became_peered":"2023-11-08T19:27:08.186952+0000","last_unstale":"2023-11-08T19:28:01.694667+0000","last_undegraded":"2023-11-08T19:28:01.694667+0000","last_fullsized":"2023-11-08T19:28:01.694667+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:12:52.022374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":58,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.694052+0000","last_change":"2023-11-08T19:26:06.394081+0000","last_active":"2023-11-08T19:28:01.694052+0000","last_peered":"2023-11-08T19:28:01.694052+0000","last_clean":"2023-11-08T19:28:01.694052+0000","last_became_active":"2023-11-08T19:26:04.602137+0000","last_became_peered":"2023-11-08T19:26:04.602137+0000","last_unstale":"2023-11-08T19:28:01.694052+0000","last_undegraded":"2023-11-08T19:28:01.694052+0000","last_fullsized":"2023-11-08T19:28:01.694052+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:05:46.951663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056998200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.690432+0000","last_change":"2023-11-08T19:27:09.178816+0000","last_active":"2023-11-08T19:28:01.690432+0000","last_peered":"2023-11-08T19:28:01.690432+0000","last_clean":"2023-11-08T19:28:01.690432+0000","last_became_active":"2023-11-08T19:27:07.192001+0000","last_became_peered":"2023-11-08T19:27:07.192001+0000","last_unstale":"2023-11-08T19:28:01.690432+0000","last_undegraded":"2023-11-08T19:28:01.690432+0000","last_fullsized":"2023-11-08T19:28:01.690432+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:04:12.481212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045359000000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.698299+0000","last_change":"2023-11-08T19:28:02.698299+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.698299+0000","last_undegraded":"2023-11-08T19:28:02.698299+0000","last_fullsized":"2023-11-08T19:28:02.698299+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.716969+0000","last_change":"2023-11-08T19:28:02.711841+0000","last_active":"2023-11-08T19:28:02.716969+0000","last_peered":"2023-11-08T19:28:02.716969+0000","last_clean":"2023-11-08T19:28:02.716969+0000","last_became_active":"2023-11-08T19:28:02.711455+0000","last_became_peered":"2023-11-08T19:28:02.711455+0000","last_unstale":"2023-11-08T19:28:02.716969+0000","last_undegraded":"2023-11-08T19:28:02.716969+0000","last_fullsized":"2023-11-08T19:28:02.716969+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:29:25.082011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.695522+0000","last_change":"2023-11-08T19:27:08.193836+0000","last_active":"2023-11-08T19:28:01.695522+0000","last_peered":"2023-11-08T19:28:01.695522+0000","last_clean":"2023-11-08T19:28:01.695522+0000","last_became_active":"2023-11-08T19:27:08.193458+0000","last_became_peered":"2023-11-08T19:27:08.193458+0000","last_unstale":"2023-11-08T19:28:01.695522+0000","last_undegraded":"2023-11-08T19:28:01.695522+0000","last_fullsized":"2023-11-08T19:28:01.695522+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:14:18.610061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":60,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.698115+0000","last_change":"2023-11-08T19:26:06.541338+0000","last_active":"2023-11-08T19:28:01.698115+0000","last_peered":"2023-11-08T19:28:01.698115+0000","last_clean":"2023-11-08T19:28:01.698115+0000","last_became_active":"2023-11-08T19:26:04.602183+0000","last_became_peered":"2023-11-08T19:26:04.602183+0000","last_unstale":"2023-11-08T19:28:01.698115+0000","last_undegraded":"2023-11-08T19:28:01.698115+0000","last_fullsized":"2023-11-08T19:28:01.698115+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:06:47.446686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052029099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.701560+0000","last_change":"2023-11-08T19:27:09.181622+0000","last_active":"2023-11-08T19:28:02.701560+0000","last_peered":"2023-11-08T19:28:02.701560+0000","last_clean":"2023-11-08T19:28:02.701560+0000","last_became_active":"2023-11-08T19:27:07.184507+0000","last_became_peered":"2023-11-08T19:27:07.184507+0000","last_unstale":"2023-11-08T19:28:02.701560+0000","last_undegraded":"2023-11-08T19:28:02.701560+0000","last_fullsized":"2023-11-08T19:28:02.701560+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:58:48.357266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051924700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.699401+0000","last_change":"2023-11-08T19:28:02.699401+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.699401+0000","last_undegraded":"2023-11-08T19:28:02.699401+0000","last_fullsized":"2023-11-08T19:28:02.699401+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.717064+0000","last_change":"2023-11-08T19:28:02.713407+0000","last_active":"2023-11-08T19:28:02.717064+0000","last_peered":"2023-11-08T19:28:02.717064+0000","last_clean":"2023-11-08T19:28:02.717064+0000","last_became_active":"2023-11-08T19:28:02.713144+0000","last_became_peered":"2023-11-08T19:28:02.713144+0000","last_unstale":"2023-11-08T19:28:02.717064+0000","last_undegraded":"2023-11-08T19:28:02.717064+0000","last_fullsized":"2023-11-08T19:28:02.717064+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:38:53.800926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.698458+0000","last_change":"2023-11-08T19:27:08.194263+0000","last_active":"2023-11-08T19:28:01.698458+0000","last_peered":"2023-11-08T19:28:01.698458+0000","last_clean":"2023-11-08T19:28:01.698458+0000","last_became_active":"2023-11-08T19:27:08.193938+0000","last_became_peered":"2023-11-08T19:27:08.193938+0000","last_unstale":"2023-11-08T19:28:01.698458+0000","last_undegraded":"2023-11-08T19:28:01.698458+0000","last_fullsized":"2023-11-08T19:28:01.698458+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:40:49.009717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":59,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.711234+0000","last_change":"2023-11-08T19:26:06.265690+0000","last_active":"2023-11-08T19:28:02.711234+0000","last_peered":"2023-11-08T19:28:02.711234+0000","last_clean":"2023-11-08T19:28:02.711234+0000","last_became_active":"2023-11-08T19:26:04.603397+0000","last_became_peered":"2023-11-08T19:26:04.603397+0000","last_unstale":"2023-11-08T19:28:02.711234+0000","last_undegraded":"2023-11-08T19:28:02.711234+0000","last_fullsized":"2023-11-08T19:28:02.711234+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:50:43.827570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053361499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.698140+0000","last_change":"2023-11-08T19:27:09.282750+0000","last_active":"2023-11-08T19:28:01.698140+0000","last_peered":"2023-11-08T19:28:01.698140+0000","last_clean":"2023-11-08T19:28:01.698140+0000","last_became_active":"2023-11-08T19:27:07.189943+0000","last_became_peered":"2023-11-08T19:27:07.189943+0000","last_unstale":"2023-11-08T19:28:01.698140+0000","last_undegraded":"2023-11-08T19:28:01.698140+0000","last_fullsized":"2023-11-08T19:28:01.698140+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:31:16.081058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000398725,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.702677+0000","last_change":"2023-11-08T19:28:02.702677+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.702677+0000","last_undegraded":"2023-11-08T19:28:02.702677+0000","last_fullsized":"2023-11-08T19:28:02.702677+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.724982+0000","last_change":"2023-11-08T19:28:02.715521+0000","last_active":"2023-11-08T19:28:02.724982+0000","last_peered":"2023-11-08T19:28:02.724982+0000","last_clean":"2023-11-08T19:28:02.724982+0000","last_became_active":"2023-11-08T19:28:02.715145+0000","last_became_peered":"2023-11-08T19:28:02.715145+0000","last_unstale":"2023-11-08T19:28:02.724982+0000","last_undegraded":"2023-11-08T19:28:02.724982+0000","last_fullsized":"2023-11-08T19:28:02.724982+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:52:23.978416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.702217+0000","last_change":"2023-11-08T19:27:08.188147+0000","last_active":"2023-11-08T19:28:02.702217+0000","last_peered":"2023-11-08T19:28:02.702217+0000","last_clean":"2023-11-08T19:28:02.702217+0000","last_became_active":"2023-11-08T19:27:08.187487+0000","last_became_peered":"2023-11-08T19:27:08.187487+0000","last_unstale":"2023-11-08T19:28:02.702217+0000","last_undegraded":"2023-11-08T19:28:02.702217+0000","last_fullsized":"2023-11-08T19:28:02.702217+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:22:40.300185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":64,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.696443+0000","last_change":"2023-11-08T19:26:06.425133+0000","last_active":"2023-11-08T19:28:01.696443+0000","last_peered":"2023-11-08T19:28:01.696443+0000","last_clean":"2023-11-08T19:28:01.696443+0000","last_became_active":"2023-11-08T19:26:04.601103+0000","last_became_peered":"2023-11-08T19:26:04.601103+0000","last_unstale":"2023-11-08T19:28:01.696443+0000","last_undegraded":"2023-11-08T19:28:01.696443+0000","last_fullsized":"2023-11-08T19:28:01.696443+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:31:14.042561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000229244,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.701647+0000","last_change":"2023-11-08T19:27:09.179852+0000","last_active":"2023-11-08T19:28:02.701647+0000","last_peered":"2023-11-08T19:28:02.701647+0000","last_clean":"2023-11-08T19:28:02.701647+0000","last_became_active":"2023-11-08T19:27:07.191926+0000","last_became_peered":"2023-11-08T19:27:07.191926+0000","last_unstale":"2023-11-08T19:28:02.701647+0000","last_undegraded":"2023-11-08T19:28:02.701647+0000","last_fullsized":"2023-11-08T19:28:02.701647+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:20:38.080129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055457100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.711863+0000","last_change":"2023-11-08T19:28:02.711863+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.711863+0000","last_undegraded":"2023-11-08T19:28:02.711863+0000","last_fullsized":"2023-11-08T19:28:02.711863+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"91'3","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.713504+0000","last_change":"2023-11-08T19:28:02.708691+0000","last_active":"2023-11-08T19:28:03.713504+0000","last_peered":"2023-11-08T19:28:03.713504+0000","last_clean":"2023-11-08T19:28:03.713504+0000","last_became_active":"2023-11-08T19:28:02.708037+0000","last_became_peered":"2023-11-08T19:28:02.708037+0000","last_unstale":"2023-11-08T19:28:03.713504+0000","last_undegraded":"2023-11-08T19:28:03.713504+0000","last_fullsized":"2023-11-08T19:28:03.713504+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:07:33.241500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.702633+0000","last_change":"2023-11-08T19:27:08.186054+0000","last_active":"2023-11-08T19:28:02.702633+0000","last_peered":"2023-11-08T19:28:02.702633+0000","last_clean":"2023-11-08T19:28:02.702633+0000","last_became_active":"2023-11-08T19:27:08.185581+0000","last_became_peered":"2023-11-08T19:27:08.185581+0000","last_unstale":"2023-11-08T19:28:02.702633+0000","last_undegraded":"2023-11-08T19:28:02.702633+0000","last_fullsized":"2023-11-08T19:28:02.702633+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:00:55.878397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":71,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.701211+0000","last_change":"2023-11-08T19:26:05.609625+0000","last_active":"2023-11-08T19:28:02.701211+0000","last_peered":"2023-11-08T19:28:02.701211+0000","last_clean":"2023-11-08T19:28:02.701211+0000","last_became_active":"2023-11-08T19:26:04.606312+0000","last_became_peered":"2023-11-08T19:26:04.606312+0000","last_unstale":"2023-11-08T19:28:02.701211+0000","last_undegraded":"2023-11-08T19:28:02.701211+0000","last_fullsized":"2023-11-08T19:28:02.701211+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:54:42.032901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000505935,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.690085+0000","last_change":"2023-11-08T19:27:09.177985+0000","last_active":"2023-11-08T19:28:01.690085+0000","last_peered":"2023-11-08T19:28:01.690085+0000","last_clean":"2023-11-08T19:28:01.690085+0000","last_became_active":"2023-11-08T19:27:07.193254+0000","last_became_peered":"2023-11-08T19:27:07.193254+0000","last_unstale":"2023-11-08T19:28:01.690085+0000","last_undegraded":"2023-11-08T19:28:01.690085+0000","last_fullsized":"2023-11-08T19:28:01.690085+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:23:38.662629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041830799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.703260+0000","last_change":"2023-11-08T19:28:02.703260+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.703260+0000","last_undegraded":"2023-11-08T19:28:02.703260+0000","last_fullsized":"2023-11-08T19:28:02.703260+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"91'6","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.727979+0000","last_change":"2023-11-08T19:28:02.700819+0000","last_active":"2023-11-08T19:28:03.727979+0000","last_peered":"2023-11-08T19:28:03.727979+0000","last_clean":"2023-11-08T19:28:03.727979+0000","last_became_active":"2023-11-08T19:28:02.700537+0000","last_became_peered":"2023-11-08T19:28:02.700537+0000","last_unstale":"2023-11-08T19:28:03.727979+0000","last_undegraded":"2023-11-08T19:28:03.727979+0000","last_fullsized":"2023-11-08T19:28:03.727979+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:25:00.088013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.703640+0000","last_change":"2023-11-08T19:27:08.188860+0000","last_active":"2023-11-08T19:28:02.703640+0000","last_peered":"2023-11-08T19:28:02.703640+0000","last_clean":"2023-11-08T19:28:02.703640+0000","last_became_active":"2023-11-08T19:27:08.188560+0000","last_became_peered":"2023-11-08T19:27:08.188560+0000","last_unstale":"2023-11-08T19:28:02.703640+0000","last_undegraded":"2023-11-08T19:28:02.703640+0000","last_fullsized":"2023-11-08T19:28:02.703640+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:18:20.969491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":59,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.695852+0000","last_change":"2023-11-08T19:26:06.297255+0000","last_active":"2023-11-08T19:28:01.695852+0000","last_peered":"2023-11-08T19:28:01.695852+0000","last_clean":"2023-11-08T19:28:01.695852+0000","last_became_active":"2023-11-08T19:26:04.603979+0000","last_became_peered":"2023-11-08T19:26:04.603979+0000","last_unstale":"2023-11-08T19:28:01.695852+0000","last_undegraded":"2023-11-08T19:28:01.695852+0000","last_fullsized":"2023-11-08T19:28:01.695852+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:40:11.164951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079032099999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.693851+0000","last_change":"2023-11-08T19:27:09.331990+0000","last_active":"2023-11-08T19:28:01.693851+0000","last_peered":"2023-11-08T19:28:01.693851+0000","last_clean":"2023-11-08T19:28:01.693851+0000","last_became_active":"2023-11-08T19:27:07.185478+0000","last_became_peered":"2023-11-08T19:27:07.185478+0000","last_unstale":"2023-11-08T19:28:01.693851+0000","last_undegraded":"2023-11-08T19:28:01.693851+0000","last_fullsized":"2023-11-08T19:28:01.693851+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:20:38.080129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010661850000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":1794,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12,"ondisk_log_size":12,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574892,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28704,"kb_used_data":1680,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343136,"statfs":{"total":96636764160,"available":96607371264,"internally_reserved":0,"allocated":1720320,"data_stored":1117537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:27:53 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.50600000000000001,"15min":0.498},"min":{"1min":0.34300000000000003,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.72899999999999998,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.437},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.51900000000000002,"15min":0.51500000000000001},"min":{"1min":0.36899999999999999,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.69099999999999995,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.621}]},{"osd":1,"last update":"Wed Nov 8 19:27:53 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56899999999999995,"15min":0.56599999999999995},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.56399999999999995,"15min":0.55600000000000005},"min":{"1min":0.435,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.59199999999999997}]},{"osd":2,"last update":"Wed Nov 8 19:27:53 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.50900000000000001,"15min":0.49299999999999999},"min":{"1min":0.38700000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.55000000000000004,"15min":0.54800000000000004},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.38500000000000001}]},{"osd":3,"last update":"Wed Nov 8 19:27:53 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.55300000000000005,"15min":0.55700000000000005},"min":{"1min":0.375,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.66200000000000003,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59399999999999997,"15min":0.57799999999999996},"min":{"1min":0.49399999999999999,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.77000000000000002,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.5}]},{"osd":4,"last update":"Wed Nov 8 19:27:53 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.50900000000000001,"15min":0.50700000000000001},"min":{"1min":0.36299999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.64500000000000002,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.54600000000000004,"15min":0.54100000000000004},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.41099999999999998}]},{"osd":5,"last update":"Wed Nov 8 19:27:53 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.58599999999999997,"15min":0.58099999999999996},"min":{"1min":0.41199999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.71999999999999997,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.62,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.84499999999999997,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.64300000000000002}]},{"osd":6,"last update":"Wed Nov 8 19:27:53 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.60199999999999998,"15min":0.60399999999999998},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.78400000000000003,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.61499999999999999,"15min":0.61099999999999999},"min":{"1min":0.5,"5min":0.443,"15min":0.443},"max":{"1min":0.82999999999999996,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.57299999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574892,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28132,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343708,"statfs":{"total":96636764160,"available":96607956992,"internally_reserved":0,"allocated":1138688,"data_stored":529225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":1.0780000000000001,"15min":1.319},"min":{"1min":0.33800000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.67800000000000005,"5min":21.359999999999999,"15min":21.359999999999999},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.78300000000000003,"15min":0.88700000000000001},"min":{"1min":0.378,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.73799999999999999,"5min":10.329000000000001,"15min":10.329000000000001},"last":0.434}]},{"osd":1,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.57899999999999996,"15min":0.60899999999999999},"min":{"1min":0.36899999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.71899999999999997,"5min":3.4239999999999999,"15min":3.4239999999999999},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.61699999999999999,"15min":0.63800000000000001},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.73199999999999998,"5min":3.4809999999999999,"15min":3.4809999999999999},"last":0.45800000000000002}]},{"osd":2,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.80100000000000005,"15min":0.90500000000000003},"min":{"1min":0.40999999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.69299999999999995,"5min":10.398,"15min":10.398},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.65400000000000003,"15min":0.68999999999999995},"min":{"1min":0.40799999999999997,"5min":0.318,"15min":0.318},"max":{"1min":0.746,"5min":5.0170000000000003,"15min":5.0170000000000003},"last":0.39300000000000002}]},{"osd":3,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.63900000000000001,"15min":0.66300000000000003},"min":{"1min":0.39500000000000002,"5min":0.317,"15min":0.317},"max":{"1min":0.77300000000000002,"5min":3.5710000000000002,"15min":3.5710000000000002},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.82999999999999996,"15min":0.93799999999999994},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.86799999999999999,"5min":10.368,"15min":10.368},"last":0.56299999999999994}]},{"osd":4,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.53700000000000003,"15min":0.51700000000000002},"min":{"1min":0.38,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.56999999999999995,"15min":0.56200000000000006},"min":{"1min":0.40600000000000003,"5min":0.373,"15min":0.373},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.53300000000000003}]},{"osd":5,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.55000000000000004,"15min":0.53200000000000003},"min":{"1min":0.40699999999999997,"5min":0.307,"15min":0.307},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.503},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.60599999999999998,"15min":0.58599999999999997},"min":{"1min":0.42899999999999999,"5min":0.378,"15min":0.378},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.55200000000000005}]},{"osd":7,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.59899999999999998,"15min":0.57899999999999996},"min":{"1min":0.41299999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.81200000000000006,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.61099999999999999,"15min":0.59399999999999997},"min":{"1min":0.54000000000000004,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.47499999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574892,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28104,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343736,"statfs":{"total":96636764160,"available":96607985664,"internally_reserved":0,"allocated":1105920,"data_stored":519171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:27:47 2023","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.36899999999999999,"15min":0.379},"min":{"1min":0.22800000000000001,"5min":0.219,"15min":0.219},"max":{"1min":0.52400000000000002,"5min":1.389,"15min":1.389},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.33800000000000002,"5min":0.58299999999999996,"15min":0.68999999999999995},"min":{"1min":0.25900000000000001,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.504,"5min":9.1370000000000005,"15min":9.1370000000000005},"last":0.36499999999999999}]},{"osd":2,"last update":"Wed Nov 8 19:27:47 2023","interfaces":[{"interface":"back","average":{"1min":0.33600000000000002,"5min":0.35399999999999998,"15min":0.36299999999999999},"min":{"1min":0.254,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.50600000000000001,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.373,"5min":0.41099999999999998,"15min":0.42499999999999999},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.58299999999999996,"5min":1.425,"15min":1.425},"last":0.47899999999999998}]},{"osd":3,"last update":"Wed Nov 8 19:27:47 2023","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.69999999999999996,"15min":0.84099999999999997},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.54600000000000004,"5min":11.957000000000001,"15min":11.957000000000001},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.85799999999999998,"15min":1.0740000000000001},"min":{"1min":0.27700000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.56299999999999994,"5min":18.039000000000001,"15min":18.039000000000001},"last":0.437}]},{"osd":4,"last update":"Wed Nov 8 19:27:47 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.73399999999999999,"15min":0.79800000000000004},"min":{"1min":0.43099999999999999,"5min":0.372,"15min":0.372},"max":{"1min":0.77800000000000002,"5min":6.181,"15min":6.181},"last":0.94699999999999995},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.629,"15min":0.63700000000000001},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.76900000000000002,"5min":1.488,"15min":1.488},"last":0.996}]},{"osd":5,"last update":"Wed Nov 8 19:27:47 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.84099999999999997,"15min":0.93700000000000006},"min":{"1min":0.437,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.74199999999999999,"5min":9.2420000000000009,"15min":9.2420000000000009},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.77200000000000002,"15min":0.84399999999999997},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.745,"5min":6.2110000000000003,"15min":6.2110000000000003},"last":0.878}]},{"osd":6,"last update":"Wed Nov 8 19:27:47 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.84899999999999998,"15min":0.94099999999999995},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.751,"5min":9.1880000000000006,"15min":9.1880000000000006},"last":0.755},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.65500000000000003,"15min":0.66300000000000003},"min":{"1min":0.38800000000000001,"5min":0.38,"15min":0.38},"max":{"1min":0.79400000000000004,"5min":1.462,"15min":1.462},"last":0.85799999999999998}]},{"osd":7,"last update":"Wed Nov 8 19:27:47 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.77400000000000002,"15min":0.83799999999999997},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.76600000000000001,"5min":6.1479999999999997,"15min":6.1479999999999997},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.872,"15min":0.97599999999999998},"min":{"1min":0.46100000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.76400000000000001,"5min":9.2170000000000005,"15min":9.2170000000000005},"last":0.90600000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574892,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28680,"kb_used_data":1660,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343160,"statfs":{"total":96636764160,"available":96607395840,"internally_reserved":0,"allocated":1699840,"data_stored":1109539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Nov 8 19:27:50 2023","interfaces":[{"interface":"back","average":{"1min":0.32100000000000001,"5min":0.441,"15min":0.495},"min":{"1min":0.23599999999999999,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.48799999999999999,"5min":4.1200000000000001,"15min":4.1200000000000001},"last":0.247},{"interface":"front","average":{"1min":0.32300000000000001,"5min":0.85199999999999998,"15min":1.079},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.51200000000000001,"5min":16.940999999999999,"15min":16.940999999999999},"last":0.34000000000000002}]},{"osd":2,"last update":"Wed Nov 8 19:27:50 2023","interfaces":[{"interface":"back","average":{"1min":0.318,"5min":0.374,"15min":0.39400000000000002},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.45600000000000002,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.57499999999999996,"15min":0.66600000000000004},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.58299999999999996,"5min":7.0209999999999999,"15min":7.0209999999999999},"last":0.28699999999999998}]},{"osd":3,"last update":"Wed Nov 8 19:27:50 2023","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.49299999999999999,"15min":0.54800000000000004},"min":{"1min":0.25,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.56699999999999995,"5min":4.0940000000000003,"15min":4.0940000000000003},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.37,"5min":0.42299999999999999,"15min":0.442},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.53600000000000003,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.36099999999999999}]},{"osd":4,"last update":"Wed Nov 8 19:27:50 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.69299999999999995,"15min":0.751},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.76800000000000002,"5min":4.0049999999999999,"15min":4.0049999999999999},"last":0.375},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.62,"15min":0.63500000000000001},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.77500000000000002,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.40000000000000002}]},{"osd":5,"last update":"Wed Nov 8 19:27:50 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.71199999999999997,"15min":0.76600000000000001},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.79200000000000004,"5min":4.3090000000000002,"15min":4.3090000000000002},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.70899999999999996,"15min":0.75700000000000001},"min":{"1min":0.45600000000000002,"5min":0.437,"15min":0.437},"max":{"1min":0.84699999999999998,"5min":4.0709999999999997,"15min":4.0709999999999997},"last":0.441}]},{"osd":6,"last update":"Wed Nov 8 19:27:50 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.65500000000000003,"15min":0.66900000000000004},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.81200000000000006,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.76800000000000002,"15min":0.82399999999999995},"min":{"1min":0.47699999999999998,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.84199999999999997,"5min":4.2560000000000002,"15min":4.2560000000000002},"last":0.56200000000000006}]},{"osd":7,"last update":"Wed Nov 8 19:27:50 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.72699999999999998,"15min":0.77700000000000002},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.78700000000000003,"5min":4.2850000000000001,"15min":4.2850000000000001},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.753,"15min":0.80500000000000005},"min":{"1min":0.48699999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.82499999999999996,"5min":4.0460000000000003,"15min":4.0460000000000003},"last":0.41399999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574892,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28088,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343752,"statfs":{"total":96636764160,"available":96608002048,"internally_reserved":0,"allocated":1093632,"data_stored":511833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:27:47 2023","interfaces":[{"interface":"back","average":{"1min":0.32100000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.47299999999999998,"5min":0.496,"15min":0.496},"last":0.28499999999999998},{"interface":"front","average":{"1min":0.31,"5min":0.51900000000000002,"15min":0.61299999999999999},"min":{"1min":0.219,"5min":0.218,"15min":0.218},"max":{"1min":0.501,"5min":6.0999999999999996,"15min":6.0999999999999996},"last":0.23000000000000001}]},{"osd":1,"last update":"Wed Nov 8 19:27:47 2023","interfaces":[{"interface":"back","average":{"1min":0.309,"5min":0.33400000000000002,"15min":0.33800000000000002},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.42699999999999999,"5min":0.48299999999999998,"15min":0.48299999999999998},"last":0.26300000000000001},{"interface":"front","average":{"1min":0.32500000000000001,"5min":0.372,"15min":0.38400000000000001},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.41899999999999998,"5min":0.624,"15min":0.624},"last":0.32600000000000001}]},{"osd":3,"last update":"Wed Nov 8 19:27:47 2023","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.376,"15min":0.38200000000000001},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.46500000000000002,"5min":0.53700000000000003,"15min":0.53700000000000003},"last":0.375},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.38300000000000001,"15min":0.39200000000000002},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.53400000000000003,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.34899999999999998}]},{"osd":4,"last update":"Wed Nov 8 19:27:47 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.56699999999999995,"15min":0.57899999999999996},"min":{"1min":0.39000000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.57699999999999996,"15min":0.58199999999999996},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.77100000000000002,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.5}]},{"osd":5,"last update":"Wed Nov 8 19:27:47 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.59199999999999997,"15min":0.58499999999999996},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.78100000000000003,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.628},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.60499999999999998,"15min":0.61099999999999999},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.76400000000000001,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.57199999999999995}]},{"osd":6,"last update":"Wed Nov 8 19:27:47 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59799999999999998,"15min":0.59999999999999998},"min":{"1min":0.47199999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.79700000000000004,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57799999999999996,"15min":0.57899999999999996},"min":{"1min":0.41499999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.80000000000000004,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.58899999999999997}]},{"osd":7,"last update":"Wed Nov 8 19:27:47 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.58199999999999996,"15min":0.57999999999999996},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.73199999999999998,"5min":0.878,"15min":0.878},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60099999999999998,"15min":0.60299999999999998},"min":{"1min":0.45300000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.82199999999999995,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.65700000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574893,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28104,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343736,"statfs":{"total":96636764160,"available":96607985664,"internally_reserved":0,"allocated":1105920,"data_stored":519171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.318,"5min":0.33300000000000002,"15min":0.33900000000000002},"min":{"1min":0.224,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.49099999999999999,"5min":0.626,"15min":0.626},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.34999999999999998,"15min":0.34899999999999998},"min":{"1min":0.28000000000000003,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.48699999999999999,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.26200000000000001}]},{"osd":1,"last update":"Wed Nov 8 19:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.34699999999999998,"15min":0.35199999999999998},"min":{"1min":0.25800000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.42299999999999999,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":0.29599999999999999},{"interface":"front","average":{"1min":0.38,"5min":0.38300000000000001,"15min":0.38400000000000001},"min":{"1min":0.28000000000000003,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.48699999999999999,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.27900000000000003}]},{"osd":2,"last update":"Wed Nov 8 19:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.376,"15min":0.372},"min":{"1min":0.32900000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.46300000000000002,"5min":0.58499999999999996,"15min":0.58499999999999996},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.40400000000000003,"15min":0.40400000000000003},"min":{"1min":0.32700000000000001,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.47399999999999998,"5min":0.58499999999999996,"15min":0.58499999999999996},"last":0.32000000000000001}]},{"osd":4,"last update":"Wed Nov 8 19:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.60799999999999998,"15min":0.60099999999999998},"min":{"1min":0.45300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.624,"5min":0.61199999999999999,"15min":0.60699999999999998},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.84499999999999997,"5min":0.871,"15min":0.871},"last":0.56100000000000005}]},{"osd":5,"last update":"Wed Nov 8 19:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.58699999999999997,"15min":0.57899999999999996},"min":{"1min":0.46700000000000003,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.76600000000000001,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.58899999999999997,"15min":0.58099999999999996},"min":{"1min":0.42399999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.78600000000000003,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.54700000000000004}]},{"osd":6,"last update":"Wed Nov 8 19:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.58799999999999997,"15min":0.58199999999999996},"min":{"1min":0.436,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.73299999999999998,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.61299999999999999,"15min":0.60699999999999998},"min":{"1min":0.49099999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.75900000000000001,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.57399999999999995}]},{"osd":7,"last update":"Wed Nov 8 19:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.59899999999999998,"15min":0.59099999999999997},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.72199999999999998,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.60199999999999998,"15min":0.59799999999999998},"min":{"1min":0.45800000000000002,"5min":0.432,"15min":0.432},"max":{"1min":0.80700000000000005,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.58299999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574892,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28104,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343736,"statfs":{"total":96636764160,"available":96607985664,"internally_reserved":0,"allocated":1105920,"data_stored":519171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:27:55 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.68200000000000005,"15min":0.74399999999999999},"min":{"1min":0.35699999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.72099999999999997,"5min":6.5970000000000004,"15min":6.5970000000000004},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.55500000000000005,"15min":0.56999999999999995},"min":{"1min":0.35799999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.70999999999999996,"5min":2.5030000000000001,"15min":2.5030000000000001},"last":0.46899999999999997}]},{"osd":1,"last update":"Wed Nov 8 19:27:55 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.51300000000000001,"15min":0.501},"min":{"1min":0.36199999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.75800000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.629},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.52100000000000002,"15min":0.51200000000000001},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.70199999999999996}]},{"osd":2,"last update":"Wed Nov 8 19:27:55 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.55400000000000005,"15min":0.55200000000000005},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.80600000000000005,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.748,"15min":0.81299999999999994},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.86199999999999999,"5min":6.6920000000000002,"15min":6.6920000000000002},"last":0.66900000000000004}]},{"osd":3,"last update":"Wed Nov 8 19:27:55 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.58299999999999996,"15min":0.58699999999999997},"min":{"1min":0.378,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.79800000000000004,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.60899999999999999,"15min":0.60999999999999999},"min":{"1min":0.42499999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.71399999999999997,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.68500000000000005}]},{"osd":5,"last update":"Wed Nov 8 19:27:55 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.55000000000000004,"15min":0.54800000000000004},"min":{"1min":0.35099999999999998,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.81599999999999995,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.58499999999999996,"15min":0.59299999999999997},"min":{"1min":0.39400000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.745,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.53000000000000003}]},{"osd":6,"last update":"Wed Nov 8 19:27:55 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58599999999999997,"15min":0.58499999999999996},"min":{"1min":0.373,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.627,"15min":0.61799999999999999},"min":{"1min":0.46400000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.73099999999999998}]},{"osd":7,"last update":"Wed Nov 8 19:27:55 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.61599999999999999,"15min":0.629},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.84499999999999997,"5min":1.141,"15min":1.141},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.64100000000000001,"15min":0.64600000000000002},"min":{"1min":0.42399999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.86399999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.74199999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574892,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28104,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343736,"statfs":{"total":96636764160,"available":96607985664,"internally_reserved":0,"allocated":1105920,"data_stored":519171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:27:46 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.51400000000000001,"15min":0.52000000000000002},"min":{"1min":0.32200000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.73999999999999999,"5min":0.753,"15min":0.753},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.51900000000000002,"15min":0.51200000000000001},"min":{"1min":0.38500000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.69399999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.495}]},{"osd":1,"last update":"Wed Nov 8 19:27:46 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.51900000000000002,"15min":0.52200000000000002},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.70799999999999996,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.53400000000000003,"15min":0.53600000000000003},"min":{"1min":0.38400000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.70499999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.56899999999999995}]},{"osd":2,"last update":"Wed Nov 8 19:27:46 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.53600000000000003,"15min":0.53500000000000003},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.68000000000000005,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.55800000000000005,"15min":0.56999999999999995},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.69799999999999995,"5min":0.747,"15min":0.747},"last":0.47699999999999998}]},{"osd":3,"last update":"Wed Nov 8 19:27:46 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.54700000000000004,"15min":0.54900000000000004},"min":{"1min":0.40899999999999997,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.72199999999999998,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.57899999999999996,"15min":0.59599999999999997},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.69099999999999995,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.627}]},{"osd":4,"last update":"Wed Nov 8 19:27:46 2023","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.53200000000000003,"15min":0.55200000000000005},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.67200000000000004,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.56200000000000006,"15min":0.56899999999999995},"min":{"1min":0.36599999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.37}]},{"osd":6,"last update":"Wed Nov 8 19:27:46 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.59399999999999997,"15min":0.60699999999999998},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.755,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.59799999999999998,"15min":0.61099999999999999},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.72099999999999997,"5min":0.745,"15min":0.745},"last":0.60999999999999999}]},{"osd":7,"last update":"Wed Nov 8 19:27:46 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.59399999999999997,"15min":0.60599999999999998},"min":{"1min":0.44500000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.74199999999999999,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.625,"15min":0.64200000000000002},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.752,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.59399999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-08T19:28:07.845 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-08T19:28:07.986 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.985+0000 7f9eb216e700 1 Processor -- start 2023-11-08T19:28:07.986 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.985+0000 7f9eb216e700 1 -- start start 2023-11-08T19:28:07.990 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.985+0000 7f9eb216e700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9eac0cc530 0x7f9eac0cf510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:07.990 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.985+0000 7f9eb216e700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9eac0cf080 con 0x7f9eac15aa30 2023-11-08T19:28:07.991 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.985+0000 7f9eb216e700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9eac0ce470 con 0x7f9eac0b2920 2023-11-08T19:28:07.991 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.985+0000 7f9eb216e700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9eac0ce750 con 0x7f9eac0cc530 2023-11-08T19:28:07.991 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.986+0000 7f9eab7fe700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f9eac15aa30 0x7f9eac0b2210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:50784/0 (socket says 172.21.15.143:50784) 2023-11-08T19:28:07.991 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.986+0000 7f9eab7fe700 1 -- 172.21.15.143:0/2063876188 learned_addr learned my addr 172.21.15.143:0/2063876188 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:28:07.992 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.986+0000 7f9eabfff700 1 --2- 172.21.15.143:0/2063876188 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9eac0cc530 0x7f9eac0cf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:07.992 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.986+0000 7f9eaa7fc700 1 -- 172.21.15.143:0/2063876188 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 491828133 0 0) 0x7f9eac0cf080 con 0x7f9eac15aa30 2023-11-08T19:28:07.992 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.986+0000 7f9eaa7fc700 1 -- 172.21.15.143:0/2063876188 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e940035b0 con 0x7f9eac15aa30 2023-11-08T19:28:07.992 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.986+0000 7f9eaa7fc700 1 -- 172.21.15.143:0/2063876188 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 933622087 0 0) 0x7f9e940035b0 con 0x7f9eac15aa30 2023-11-08T19:28:07.992 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.986+0000 7f9eaa7fc700 1 -- 172.21.15.143:0/2063876188 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9eac0cc530 msgr2=0x7f9eac0cf510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:07.993 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.986+0000 7f9eaa7fc700 1 --2- 172.21.15.143:0/2063876188 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9eac0cc530 0x7f9eac0cf510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:07.993 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.986+0000 7f9eaa7fc700 1 -- 172.21.15.143:0/2063876188 >> v1:172.21.15.174:6789/0 conn(0x7f9eac0b2920 legacy=0x7f9eac0cbe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:07.993 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.986+0000 7f9eaa7fc700 1 -- 172.21.15.143:0/2063876188 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9eac0cea30 con 0x7f9eac15aa30 2023-11-08T19:28:07.993 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.986+0000 7f9eaa7fc700 1 -- 172.21.15.143:0/2063876188 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4147756303 0 0) 0x7f9e9c0017f0 con 0x7f9eac15aa30 2023-11-08T19:28:07.993 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.986+0000 7f9eaa7fc700 1 -- 172.21.15.143:0/2063876188 >> v1:172.21.15.143:6789/0 conn(0x7f9eac15aa30 legacy=0x7f9eac0b2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:07.993 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.986+0000 7f9eaa7fc700 1 --2- 172.21.15.143:0/2063876188 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9e94003a30 0x7f9e94003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:07.994 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.986+0000 7f9eaa7fc700 1 --2- 172.21.15.143:0/2063876188 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9e940043e0 0x7f9e940048b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:07.994 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.986+0000 7f9eaa7fc700 1 --2- 172.21.15.143:0/2063876188 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9eac0cc530 0x7f9e94005110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:07.994 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.986+0000 7f9eaa7fc700 1 -- 172.21.15.143:0/2063876188 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eac0ce750 con 0x7f9e94003a30 2023-11-08T19:28:07.994 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.986+0000 7f9eaa7fc700 1 -- 172.21.15.143:0/2063876188 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e9400ba10 con 0x7f9e940043e0 2023-11-08T19:28:07.995 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.986+0000 7f9eaa7fc700 1 -- 172.21.15.143:0/2063876188 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e9400bbd0 con 0x7f9eac0cc530 2023-11-08T19:28:07.995 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.986+0000 7f9eab7fe700 1 --2- 172.21.15.143:0/2063876188 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9e94003a30 0x7f9e94003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:07.995 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.986+0000 7f9eabfff700 1 --2- 172.21.15.143:0/2063876188 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9eac0cc530 0x7f9e94005110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:07.995 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.986+0000 7f9eaaffd700 1 --2- 172.21.15.143:0/2063876188 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9e940043e0 0x7f9e940048b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:07.995 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.987+0000 7f9eab7fe700 1 -- 172.21.15.143:0/2063876188 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9eac0cc530 msgr2=0x7f9e94005110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:07.996 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.987+0000 7f9eab7fe700 1 --2- 172.21.15.143:0/2063876188 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9eac0cc530 0x7f9e94005110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:07.996 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.987+0000 7f9eab7fe700 1 -- 172.21.15.143:0/2063876188 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9e940043e0 msgr2=0x7f9e940048b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:07.996 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.987+0000 7f9eab7fe700 1 --2- 172.21.15.143:0/2063876188 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9e940043e0 0x7f9e940048b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:07.996 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.987+0000 7f9eab7fe700 1 -- 172.21.15.143:0/2063876188 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e940035b0 con 0x7f9e94003a30 2023-11-08T19:28:07.996 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.987+0000 7f9eab7fe700 1 --2- 172.21.15.143:0/2063876188 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9e94003a30 0x7f9e94003e10 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c00c240 tx=0x7f9e94005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cce86c222a8eb56 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:07.997 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.987+0000 7f9eaa7fc700 1 -- 172.21.15.143:0/2063876188 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e9c00cde0 con 0x7f9e94003a30 2023-11-08T19:28:07.997 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.987+0000 7f9eaa7fc700 1 -- 172.21.15.143:0/2063876188 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e9c007ba0 con 0x7f9e94003a30 2023-11-08T19:28:07.997 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.987+0000 7f9eaa7fc700 1 -- 172.21.15.143:0/2063876188 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e9c0072f0 con 0x7f9e94003a30 2023-11-08T19:28:07.997 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.987+0000 7f9eb216e700 1 -- 172.21.15.143:0/2063876188 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9e94003a30 msgr2=0x7f9e94003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:07.997 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.987+0000 7f9eb216e700 1 --2- 172.21.15.143:0/2063876188 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9e94003a30 0x7f9e94003e10 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c00c240 tx=0x7f9e94005880 comp rx=0 tx=0).stop 2023-11-08T19:28:07.998 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.987+0000 7f9eabfff700 1 -- 172.21.15.143:0/2063876188 reap_dead start 2023-11-08T19:28:07.998 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.987+0000 7f9eb216e700 1 -- 172.21.15.143:0/2063876188 shutdown_connections 2023-11-08T19:28:07.998 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.987+0000 7f9eb216e700 1 -- 172.21.15.143:0/2063876188 >> 172.21.15.143:0/2063876188 conn(0x7f9eac099ab0 msgr2=0x7f9eac0d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:07.998 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.987+0000 7f9eb216e700 1 -- 172.21.15.143:0/2063876188 shutdown_connections 2023-11-08T19:28:07.998 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.988+0000 7f9eb216e700 1 -- 172.21.15.143:0/2063876188 wait complete. 2023-11-08T19:28:07.999 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.989+0000 7f9eb216e700 1 Processor -- start 2023-11-08T19:28:07.999 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.989+0000 7f9eb216e700 1 -- start start 2023-11-08T19:28:08.002 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.990+0000 7f9eb216e700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9e94003a30 0x7f9eac1f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:08.002 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.990+0000 7f9eb216e700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9eac0b5db0 0x7f9eac0b42f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:08.002 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.990+0000 7f9eb216e700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9eac14fb20 0x7f9eac06ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:08.002 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.990+0000 7f9eb216e700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eac0b3530 con 0x7f9e94003a30 2023-11-08T19:28:08.003 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.990+0000 7f9eb216e700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eac0b3810 con 0x7f9eac0b5db0 2023-11-08T19:28:08.003 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.990+0000 7f9eb216e700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9eac0b3af0 con 0x7f9eac14fb20 2023-11-08T19:28:08.003 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.990+0000 7f9eab7fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9e94003a30 0x7f9eac1f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:08.003 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.990+0000 7f9eab7fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9e94003a30 0x7f9eac1f20e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54642/0 (socket says 172.21.15.143:54642) 2023-11-08T19:28:08.003 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.990+0000 7f9eab7fe700 1 -- 172.21.15.143:0/1814441954 learned_addr learned my addr 172.21.15.143:0/1814441954 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:28:08.004 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.990+0000 7f9eabfff700 1 --2- 172.21.15.143:0/1814441954 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9eac14fb20 0x7f9eac06ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:08.004 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.990+0000 7f9eaaffd700 1 --2- 172.21.15.143:0/1814441954 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9eac0b5db0 0x7f9eac0b42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:08.004 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.990+0000 7f9eab7fe700 1 -- 172.21.15.143:0/1814441954 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9eac14fb20 msgr2=0x7f9eac06ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:08.004 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.990+0000 7f9eab7fe700 1 --2- 172.21.15.143:0/1814441954 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9eac14fb20 0x7f9eac06ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:08.006 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.990+0000 7f9eab7fe700 1 -- 172.21.15.143:0/1814441954 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9eac0b5db0 msgr2=0x7f9eac0b42f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:08.006 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.990+0000 7f9eab7fe700 1 --2- 172.21.15.143:0/1814441954 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9eac0b5db0 0x7f9eac0b42f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:08.006 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.990+0000 7f9eab7fe700 1 -- 172.21.15.143:0/1814441954 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9eac0b3dd0 con 0x7f9e94003a30 2023-11-08T19:28:08.006 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.990+0000 7f9eaaffd700 1 --2- 172.21.15.143:0/1814441954 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9eac0b5db0 0x7f9eac0b42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:28:08.007 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.990+0000 7f9eab7fe700 1 --2- 172.21.15.143:0/1814441954 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9e94003a30 0x7f9eac1f20e0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c0017c0 tx=0x7f9e9c001b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:08.007 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.990+0000 7f9e93fff700 1 -- 172.21.15.143:0/1814441954 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e9c0074e0 con 0x7f9e94003a30 2023-11-08T19:28:08.007 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.990+0000 7f9e93fff700 1 -- 172.21.15.143:0/1814441954 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e9c007640 con 0x7f9e94003a30 2023-11-08T19:28:08.007 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.990+0000 7f9eb216e700 1 -- 172.21.15.143:0/1814441954 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9eac0b4830 con 0x7f9e94003a30 2023-11-08T19:28:08.008 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.990+0000 7f9e93fff700 1 -- 172.21.15.143:0/1814441954 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e9c005a30 con 0x7f9e94003a30 2023-11-08T19:28:08.008 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.990+0000 7f9eb216e700 1 -- 172.21.15.143:0/1814441954 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9eac0b6410 con 0x7f9e94003a30 2023-11-08T19:28:08.008 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.991+0000 7f9eb216e700 1 -- 172.21.15.143:0/1814441954 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9eac04eee0 con 0x7f9e94003a30 2023-11-08T19:28:08.008 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.992+0000 7f9e93fff700 1 -- 172.21.15.143:0/1814441954 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9e9c019030 con 0x7f9e94003a30 2023-11-08T19:28:08.008 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.992+0000 7f9e93fff700 1 --2- 172.21.15.143:0/1814441954 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f9e9806af00 0x7f9e9806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:08.009 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.992+0000 7f9e93fff700 1 -- 172.21.15.143:0/1814441954 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6772+0+0 (secure 0 0 0) 0x7f9e9c08a730 con 0x7f9e94003a30 2023-11-08T19:28:08.009 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.993+0000 7f9eaaffd700 1 --2- 172.21.15.143:0/1814441954 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f9e9806af00 0x7f9e9806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:08.009 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.993+0000 7f9eaaffd700 1 --2- 172.21.15.143:0/1814441954 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f9e9806af00 0x7f9e9806d3c0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f9eac153990 tx=0x7f9ea0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:08.009 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:07.997+0000 7f9e93fff700 1 -- 172.21.15.143:0/1814441954 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9e9c055500 con 0x7f9e94003a30 2023-11-08T19:28:08.184 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.182+0000 7f9eb216e700 1 -- 172.21.15.143:0/1814441954 --> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9eac0b3980 con 0x7f9e9806af00 2023-11-08T19:28:08.198 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.197+0000 7f9e93fff700 1 -- 172.21.15.143:0/1814441954 <== mgr.4099 v2:172.21.15.174:6832/4037280178 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+528138 (secure 0 0 0) 0x7f9eac0b3980 con 0x7f9e9806af00 2023-11-08T19:28:08.204 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:28:08.210 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.201+0000 7f9eb216e700 1 -- 172.21.15.143:0/1814441954 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f9e9806af00 msgr2=0x7f9e9806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:08.211 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.201+0000 7f9eb216e700 1 --2- 172.21.15.143:0/1814441954 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f9e9806af00 0x7f9e9806d3c0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f9eac153990 tx=0x7f9ea0009040 comp rx=0 tx=0).stop 2023-11-08T19:28:08.211 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.201+0000 7f9eb216e700 1 -- 172.21.15.143:0/1814441954 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9e94003a30 msgr2=0x7f9eac1f20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:08.211 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.201+0000 7f9eb216e700 1 --2- 172.21.15.143:0/1814441954 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9e94003a30 0x7f9eac1f20e0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c0017c0 tx=0x7f9e9c001b80 comp rx=0 tx=0).stop 2023-11-08T19:28:08.211 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.201+0000 7f9eb216e700 1 -- 172.21.15.143:0/1814441954 shutdown_connections 2023-11-08T19:28:08.211 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.201+0000 7f9eb216e700 1 --2- 172.21.15.143:0/1814441954 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f9e9806af00 0x7f9e9806d3c0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:08.212 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.201+0000 7f9eb216e700 1 --2- 172.21.15.143:0/1814441954 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9eac14fb20 0x7f9eac06ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:08.212 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.201+0000 7f9eb216e700 1 --2- 172.21.15.143:0/1814441954 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9e94003a30 0x7f9eac1f20e0 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:08.212 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.201+0000 7f9eb216e700 1 --2- 172.21.15.143:0/1814441954 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9eac0b5db0 0x7f9eac0b42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:08.212 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.201+0000 7f9eb216e700 1 -- 172.21.15.143:0/1814441954 >> 172.21.15.143:0/1814441954 conn(0x7f9eac099ab0 msgr2=0x7f9eac06e120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:08.212 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.202+0000 7f9eb216e700 1 -- 172.21.15.143:0/1814441954 shutdown_connections 2023-11-08T19:28:08.213 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.202+0000 7f9eb216e700 1 -- 172.21.15.143:0/1814441954 wait complete. 2023-11-08T19:28:08.213 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2023-11-08T19:28:08.216 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":186,"stamp":"2023-11-08T19:28:06.562183+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592162,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":171,"num_write_kb":2422,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":129,"ondisk_log_size":129,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":236,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":226020,"kb_used_data":9840,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754748700,"statfs":{"total":773094113280,"available":772862668800,"internally_reserved":0,"allocated":10076160,"data_stored":5344818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1794,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.868593"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.701553+0000","last_change":"2023-11-08T19:28:02.701553+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.701553+0000","last_undegraded":"2023-11-08T19:28:02.701553+0000","last_fullsized":"2023-11-08T19:28:02.701553+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.719263+0000","last_change":"2023-11-08T19:28:03.715294+0000","last_active":"2023-11-08T19:28:03.719263+0000","last_peered":"2023-11-08T19:28:03.719263+0000","last_clean":"2023-11-08T19:28:03.719263+0000","last_became_active":"2023-11-08T19:28:03.715007+0000","last_became_peered":"2023-11-08T19:28:03.715007+0000","last_unstale":"2023-11-08T19:28:03.719263+0000","last_undegraded":"2023-11-08T19:28:03.719263+0000","last_fullsized":"2023-11-08T19:28:03.719263+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:51:46.879726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.700726+0000","last_change":"2023-11-08T19:28:02.700726+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.700726+0000","last_undegraded":"2023-11-08T19:28:02.700726+0000","last_fullsized":"2023-11-08T19:28:02.700726+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.699799+0000","last_change":"2023-11-08T19:28:02.699799+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.699799+0000","last_undegraded":"2023-11-08T19:28:02.699799+0000","last_fullsized":"2023-11-08T19:28:02.699799+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.704338+0000","last_change":"2023-11-08T19:28:02.704338+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.704338+0000","last_undegraded":"2023-11-08T19:28:02.704338+0000","last_fullsized":"2023-11-08T19:28:02.704338+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.722681+0000","last_change":"2023-11-08T19:28:03.718208+0000","last_active":"2023-11-08T19:28:03.722681+0000","last_peered":"2023-11-08T19:28:03.722681+0000","last_clean":"2023-11-08T19:28:03.722681+0000","last_became_active":"2023-11-08T19:28:03.716630+0000","last_became_peered":"2023-11-08T19:28:03.716630+0000","last_unstale":"2023-11-08T19:28:03.722681+0000","last_undegraded":"2023-11-08T19:28:03.722681+0000","last_fullsized":"2023-11-08T19:28:03.722681+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:21:24.678157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.712530+0000","last_change":"2023-11-08T19:28:03.708477+0000","last_active":"2023-11-08T19:28:03.712530+0000","last_peered":"2023-11-08T19:28:03.712530+0000","last_clean":"2023-11-08T19:28:03.712530+0000","last_became_active":"2023-11-08T19:28:03.708188+0000","last_became_peered":"2023-11-08T19:28:03.708188+0000","last_unstale":"2023-11-08T19:28:03.712530+0000","last_undegraded":"2023-11-08T19:28:03.712530+0000","last_fullsized":"2023-11-08T19:28:03.712530+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:57:58.571556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.714751+0000","last_change":"2023-11-08T19:28:02.714751+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.714751+0000","last_undegraded":"2023-11-08T19:28:02.714751+0000","last_fullsized":"2023-11-08T19:28:02.714751+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.722749+0000","last_change":"2023-11-08T19:28:03.718615+0000","last_active":"2023-11-08T19:28:03.722749+0000","last_peered":"2023-11-08T19:28:03.722749+0000","last_clean":"2023-11-08T19:28:03.722749+0000","last_became_active":"2023-11-08T19:28:03.718295+0000","last_became_peered":"2023-11-08T19:28:03.718295+0000","last_unstale":"2023-11-08T19:28:03.722749+0000","last_undegraded":"2023-11-08T19:28:03.722749+0000","last_fullsized":"2023-11-08T19:28:03.722749+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:23:12.882243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.702139+0000","last_change":"2023-11-08T19:28:02.702139+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.702139+0000","last_undegraded":"2023-11-08T19:28:02.702139+0000","last_fullsized":"2023-11-08T19:28:02.702139+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.705435+0000","last_change":"2023-11-08T19:28:02.705435+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.705435+0000","last_undegraded":"2023-11-08T19:28:02.705435+0000","last_fullsized":"2023-11-08T19:28:02.705435+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.703962+0000","last_change":"2023-11-08T19:28:02.703962+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.703962+0000","last_undegraded":"2023-11-08T19:28:02.703962+0000","last_fullsized":"2023-11-08T19:28:02.703962+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.697723+0000","last_change":"2023-11-08T19:28:02.697723+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.697723+0000","last_undegraded":"2023-11-08T19:28:02.697723+0000","last_fullsized":"2023-11-08T19:28:02.697723+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.721906+0000","last_change":"2023-11-08T19:28:03.718346+0000","last_active":"2023-11-08T19:28:03.721906+0000","last_peered":"2023-11-08T19:28:03.721906+0000","last_clean":"2023-11-08T19:28:03.721906+0000","last_became_active":"2023-11-08T19:28:03.718068+0000","last_became_peered":"2023-11-08T19:28:03.718068+0000","last_unstale":"2023-11-08T19:28:03.721906+0000","last_undegraded":"2023-11-08T19:28:03.721906+0000","last_fullsized":"2023-11-08T19:28:03.721906+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:29:02.099134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.721945+0000","last_change":"2023-11-08T19:28:03.717576+0000","last_active":"2023-11-08T19:28:03.721945+0000","last_peered":"2023-11-08T19:28:03.721945+0000","last_clean":"2023-11-08T19:28:03.721945+0000","last_became_active":"2023-11-08T19:28:03.715708+0000","last_became_peered":"2023-11-08T19:28:03.715708+0000","last_unstale":"2023-11-08T19:28:03.721945+0000","last_undegraded":"2023-11-08T19:28:03.721945+0000","last_fullsized":"2023-11-08T19:28:03.721945+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:40:03.163555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.711516+0000","last_change":"2023-11-08T19:28:03.703641+0000","last_active":"2023-11-08T19:28:03.711516+0000","last_peered":"2023-11-08T19:28:03.711516+0000","last_clean":"2023-11-08T19:28:03.711516+0000","last_became_active":"2023-11-08T19:28:03.703331+0000","last_became_peered":"2023-11-08T19:28:03.703331+0000","last_unstale":"2023-11-08T19:28:03.711516+0000","last_undegraded":"2023-11-08T19:28:03.711516+0000","last_fullsized":"2023-11-08T19:28:03.711516+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:32:03.694798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.700364+0000","last_change":"2023-11-08T19:28:02.700364+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.700364+0000","last_undegraded":"2023-11-08T19:28:02.700364+0000","last_fullsized":"2023-11-08T19:28:02.700364+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.719244+0000","last_change":"2023-11-08T19:28:03.715221+0000","last_active":"2023-11-08T19:28:03.719244+0000","last_peered":"2023-11-08T19:28:03.719244+0000","last_clean":"2023-11-08T19:28:03.719244+0000","last_became_active":"2023-11-08T19:28:03.714929+0000","last_became_peered":"2023-11-08T19:28:03.714929+0000","last_unstale":"2023-11-08T19:28:03.719244+0000","last_undegraded":"2023-11-08T19:28:03.719244+0000","last_fullsized":"2023-11-08T19:28:03.719244+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:58:49.797420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.700365+0000","last_change":"2023-11-08T19:28:02.700365+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.700365+0000","last_undegraded":"2023-11-08T19:28:02.700365+0000","last_fullsized":"2023-11-08T19:28:02.700365+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.704383+0000","last_change":"2023-11-08T19:28:02.704383+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.704383+0000","last_undegraded":"2023-11-08T19:28:02.704383+0000","last_fullsized":"2023-11-08T19:28:02.704383+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.713555+0000","last_change":"2023-11-08T19:28:02.713555+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.713555+0000","last_undegraded":"2023-11-08T19:28:02.713555+0000","last_fullsized":"2023-11-08T19:28:02.713555+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.702640+0000","last_change":"2023-11-08T19:28:02.702640+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.702640+0000","last_undegraded":"2023-11-08T19:28:02.702640+0000","last_fullsized":"2023-11-08T19:28:02.702640+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.699841+0000","last_change":"2023-11-08T19:28:02.699841+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.699841+0000","last_undegraded":"2023-11-08T19:28:02.699841+0000","last_fullsized":"2023-11-08T19:28:02.699841+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.702861+0000","last_change":"2023-11-08T19:28:02.702861+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.702861+0000","last_undegraded":"2023-11-08T19:28:02.702861+0000","last_fullsized":"2023-11-08T19:28:02.702861+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.707625+0000","last_change":"2023-11-08T19:28:02.707625+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.707625+0000","last_undegraded":"2023-11-08T19:28:02.707625+0000","last_fullsized":"2023-11-08T19:28:02.707625+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.710228+0000","last_change":"2023-11-08T19:28:02.710228+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.710228+0000","last_undegraded":"2023-11-08T19:28:02.710228+0000","last_fullsized":"2023-11-08T19:28:02.710228+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.704524+0000","last_change":"2023-11-08T19:28:02.704524+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.704524+0000","last_undegraded":"2023-11-08T19:28:02.704524+0000","last_fullsized":"2023-11-08T19:28:02.704524+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.725514+0000","last_change":"2023-11-08T19:28:03.721162+0000","last_active":"2023-11-08T19:28:03.725514+0000","last_peered":"2023-11-08T19:28:03.725514+0000","last_clean":"2023-11-08T19:28:03.725514+0000","last_became_active":"2023-11-08T19:28:03.720956+0000","last_became_peered":"2023-11-08T19:28:03.720956+0000","last_unstale":"2023-11-08T19:28:03.725514+0000","last_undegraded":"2023-11-08T19:28:03.725514+0000","last_fullsized":"2023-11-08T19:28:03.725514+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:09:19.883657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.712428+0000","last_change":"2023-11-08T19:28:02.712428+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.712428+0000","last_undegraded":"2023-11-08T19:28:02.712428+0000","last_fullsized":"2023-11-08T19:28:02.712428+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.705928+0000","last_change":"2023-11-08T19:28:02.705928+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.705928+0000","last_undegraded":"2023-11-08T19:28:02.705928+0000","last_fullsized":"2023-11-08T19:28:02.705928+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.712510+0000","last_change":"2023-11-08T19:28:03.707881+0000","last_active":"2023-11-08T19:28:03.712510+0000","last_peered":"2023-11-08T19:28:03.712510+0000","last_clean":"2023-11-08T19:28:03.712510+0000","last_became_active":"2023-11-08T19:28:03.707569+0000","last_became_peered":"2023-11-08T19:28:03.707569+0000","last_unstale":"2023-11-08T19:28:03.712510+0000","last_undegraded":"2023-11-08T19:28:03.712510+0000","last_fullsized":"2023-11-08T19:28:03.712510+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:00:53.075545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.711643+0000","last_change":"2023-11-08T19:28:02.711643+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.711643+0000","last_undegraded":"2023-11-08T19:28:02.711643+0000","last_fullsized":"2023-11-08T19:28:02.711643+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.702089+0000","last_change":"2023-11-08T19:28:02.702089+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.702089+0000","last_undegraded":"2023-11-08T19:28:02.702089+0000","last_fullsized":"2023-11-08T19:28:02.702089+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.725513+0000","last_change":"2023-11-08T19:28:02.720536+0000","last_active":"2023-11-08T19:28:02.725513+0000","last_peered":"2023-11-08T19:28:02.725513+0000","last_clean":"2023-11-08T19:28:02.725513+0000","last_became_active":"2023-11-08T19:28:02.720236+0000","last_became_peered":"2023-11-08T19:28:02.720236+0000","last_unstale":"2023-11-08T19:28:02.725513+0000","last_undegraded":"2023-11-08T19:28:02.725513+0000","last_fullsized":"2023-11-08T19:28:02.725513+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:56:59.813550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.700732+0000","last_change":"2023-11-08T19:27:08.190826+0000","last_active":"2023-11-08T19:28:02.700732+0000","last_peered":"2023-11-08T19:28:02.700732+0000","last_clean":"2023-11-08T19:28:02.700732+0000","last_became_active":"2023-11-08T19:27:08.190440+0000","last_became_peered":"2023-11-08T19:27:08.190440+0000","last_unstale":"2023-11-08T19:28:02.700732+0000","last_undegraded":"2023-11-08T19:28:02.700732+0000","last_fullsized":"2023-11-08T19:28:02.700732+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:42:48.376462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":237,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.699373+0000","last_change":"2023-11-08T19:24:45.888122+0000","last_active":"2023-11-08T19:28:02.699373+0000","last_peered":"2023-11-08T19:28:02.699373+0000","last_clean":"2023-11-08T19:28:02.699373+0000","last_became_active":"2023-11-08T19:24:45.887048+0000","last_became_peered":"2023-11-08T19:24:45.887048+0000","last_unstale":"2023-11-08T19:28:02.699373+0000","last_undegraded":"2023-11-08T19:28:02.699373+0000","last_fullsized":"2023-11-08T19:28:02.699373+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:44.575133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:44.575133+0000","last_clean_scrub_stamp":"2023-11-08T19:24:44.575133+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:19:20.202062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":60,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.698526+0000","last_change":"2023-11-08T19:26:06.542044+0000","last_active":"2023-11-08T19:28:01.698526+0000","last_peered":"2023-11-08T19:28:01.698526+0000","last_clean":"2023-11-08T19:28:01.698526+0000","last_became_active":"2023-11-08T19:26:04.603056+0000","last_became_peered":"2023-11-08T19:26:04.603056+0000","last_unstale":"2023-11-08T19:28:01.698526+0000","last_undegraded":"2023-11-08T19:28:01.698526+0000","last_fullsized":"2023-11-08T19:28:01.698526+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:41:50.771457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000517414,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.694743+0000","last_change":"2023-11-08T19:27:09.496842+0000","last_active":"2023-11-08T19:28:01.694743+0000","last_peered":"2023-11-08T19:28:01.694743+0000","last_clean":"2023-11-08T19:28:01.694743+0000","last_became_active":"2023-11-08T19:27:07.185776+0000","last_became_peered":"2023-11-08T19:27:07.185776+0000","last_unstale":"2023-11-08T19:28:01.694743+0000","last_undegraded":"2023-11-08T19:28:01.694743+0000","last_fullsized":"2023-11-08T19:28:01.694743+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:21:13.923970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031747200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.698950+0000","last_change":"2023-11-08T19:28:02.698950+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.698950+0000","last_undegraded":"2023-11-08T19:28:02.698950+0000","last_fullsized":"2023-11-08T19:28:02.698950+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.725538+0000","last_change":"2023-11-08T19:28:02.720512+0000","last_active":"2023-11-08T19:28:02.725538+0000","last_peered":"2023-11-08T19:28:02.725538+0000","last_clean":"2023-11-08T19:28:02.725538+0000","last_became_active":"2023-11-08T19:28:02.720202+0000","last_became_peered":"2023-11-08T19:28:02.720202+0000","last_unstale":"2023-11-08T19:28:02.725538+0000","last_undegraded":"2023-11-08T19:28:02.725538+0000","last_fullsized":"2023-11-08T19:28:02.725538+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:26:10.008919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.694327+0000","last_change":"2023-11-08T19:27:08.188241+0000","last_active":"2023-11-08T19:28:01.694327+0000","last_peered":"2023-11-08T19:28:01.694327+0000","last_clean":"2023-11-08T19:28:01.694327+0000","last_became_active":"2023-11-08T19:27:08.187863+0000","last_became_peered":"2023-11-08T19:27:08.187863+0000","last_unstale":"2023-11-08T19:28:01.694327+0000","last_undegraded":"2023-11-08T19:28:01.694327+0000","last_fullsized":"2023-11-08T19:28:01.694327+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:45:22.084015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":58,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.696493+0000","last_change":"2023-11-08T19:26:06.265419+0000","last_active":"2023-11-08T19:28:01.696493+0000","last_peered":"2023-11-08T19:28:01.696493+0000","last_clean":"2023-11-08T19:28:01.696493+0000","last_became_active":"2023-11-08T19:26:04.605333+0000","last_became_peered":"2023-11-08T19:26:04.605333+0000","last_unstale":"2023-11-08T19:28:01.696493+0000","last_undegraded":"2023-11-08T19:28:01.696493+0000","last_fullsized":"2023-11-08T19:28:01.696493+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:01:56.991346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064606400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.701529+0000","last_change":"2023-11-08T19:27:09.182113+0000","last_active":"2023-11-08T19:28:02.701529+0000","last_peered":"2023-11-08T19:28:02.701529+0000","last_clean":"2023-11-08T19:28:02.701529+0000","last_became_active":"2023-11-08T19:27:07.189939+0000","last_became_peered":"2023-11-08T19:27:07.189939+0000","last_unstale":"2023-11-08T19:28:02.701529+0000","last_undegraded":"2023-11-08T19:28:02.701529+0000","last_fullsized":"2023-11-08T19:28:02.701529+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:41:08.822366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036507500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.699139+0000","last_change":"2023-11-08T19:28:02.699139+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.699139+0000","last_undegraded":"2023-11-08T19:28:02.699139+0000","last_fullsized":"2023-11-08T19:28:02.699139+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.725537+0000","last_change":"2023-11-08T19:28:02.720708+0000","last_active":"2023-11-08T19:28:02.725537+0000","last_peered":"2023-11-08T19:28:02.725537+0000","last_clean":"2023-11-08T19:28:02.725537+0000","last_became_active":"2023-11-08T19:28:02.720263+0000","last_became_peered":"2023-11-08T19:28:02.720263+0000","last_unstale":"2023-11-08T19:28:02.725537+0000","last_undegraded":"2023-11-08T19:28:02.725537+0000","last_fullsized":"2023-11-08T19:28:02.725537+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:56:32.169912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.691014+0000","last_change":"2023-11-08T19:27:08.187127+0000","last_active":"2023-11-08T19:28:01.691014+0000","last_peered":"2023-11-08T19:28:01.691014+0000","last_clean":"2023-11-08T19:28:01.691014+0000","last_became_active":"2023-11-08T19:27:08.182648+0000","last_became_peered":"2023-11-08T19:27:08.182648+0000","last_unstale":"2023-11-08T19:28:01.691014+0000","last_undegraded":"2023-11-08T19:28:01.691014+0000","last_fullsized":"2023-11-08T19:28:01.691014+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:50:12.101878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":71,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.700875+0000","last_change":"2023-11-08T19:26:05.609570+0000","last_active":"2023-11-08T19:28:02.700875+0000","last_peered":"2023-11-08T19:28:02.700875+0000","last_clean":"2023-11-08T19:28:02.700875+0000","last_became_active":"2023-11-08T19:26:04.608379+0000","last_became_peered":"2023-11-08T19:26:04.608379+0000","last_unstale":"2023-11-08T19:28:02.700875+0000","last_undegraded":"2023-11-08T19:28:02.700875+0000","last_fullsized":"2023-11-08T19:28:02.700875+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:40:08.986649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000529657,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.696937+0000","last_change":"2023-11-08T19:27:09.211991+0000","last_active":"2023-11-08T19:28:01.696937+0000","last_peered":"2023-11-08T19:28:01.696937+0000","last_clean":"2023-11-08T19:28:01.696937+0000","last_became_active":"2023-11-08T19:27:07.195231+0000","last_became_peered":"2023-11-08T19:27:07.195231+0000","last_unstale":"2023-11-08T19:28:01.696937+0000","last_undegraded":"2023-11-08T19:28:01.696937+0000","last_fullsized":"2023-11-08T19:28:01.696937+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:25:14.487524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000441221,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.697661+0000","last_change":"2023-11-08T19:28:02.697661+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.697661+0000","last_undegraded":"2023-11-08T19:28:02.697661+0000","last_fullsized":"2023-11-08T19:28:02.697661+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.705119+0000","last_change":"2023-11-08T19:28:02.707543+0000","last_active":"2023-11-08T19:28:03.705119+0000","last_peered":"2023-11-08T19:28:03.705119+0000","last_clean":"2023-11-08T19:28:03.705119+0000","last_became_active":"2023-11-08T19:28:02.707034+0000","last_became_peered":"2023-11-08T19:28:02.707034+0000","last_unstale":"2023-11-08T19:28:03.705119+0000","last_undegraded":"2023-11-08T19:28:03.705119+0000","last_fullsized":"2023-11-08T19:28:03.705119+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:06:46.912027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.690189+0000","last_change":"2023-11-08T19:27:08.195364+0000","last_active":"2023-11-08T19:28:01.690189+0000","last_peered":"2023-11-08T19:28:01.690189+0000","last_clean":"2023-11-08T19:28:01.690189+0000","last_became_active":"2023-11-08T19:27:08.195074+0000","last_became_peered":"2023-11-08T19:27:08.195074+0000","last_unstale":"2023-11-08T19:28:01.690189+0000","last_undegraded":"2023-11-08T19:28:01.690189+0000","last_fullsized":"2023-11-08T19:28:01.690189+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:02:57.552873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":59,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.700695+0000","last_change":"2023-11-08T19:26:06.394300+0000","last_active":"2023-11-08T19:28:02.700695+0000","last_peered":"2023-11-08T19:28:02.700695+0000","last_clean":"2023-11-08T19:28:02.700695+0000","last_became_active":"2023-11-08T19:26:04.601685+0000","last_became_peered":"2023-11-08T19:26:04.601685+0000","last_unstale":"2023-11-08T19:28:02.700695+0000","last_undegraded":"2023-11-08T19:28:02.700695+0000","last_fullsized":"2023-11-08T19:28:02.700695+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:51:06.626474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061044099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.690413+0000","last_change":"2023-11-08T19:27:09.178466+0000","last_active":"2023-11-08T19:28:01.690413+0000","last_peered":"2023-11-08T19:28:01.690413+0000","last_clean":"2023-11-08T19:28:01.690413+0000","last_became_active":"2023-11-08T19:27:07.189386+0000","last_became_peered":"2023-11-08T19:27:07.189386+0000","last_unstale":"2023-11-08T19:28:01.690413+0000","last_undegraded":"2023-11-08T19:28:01.690413+0000","last_fullsized":"2023-11-08T19:28:01.690413+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:48:51.139820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049390100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.713309+0000","last_change":"2023-11-08T19:28:02.713309+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.713309+0000","last_undegraded":"2023-11-08T19:28:02.713309+0000","last_fullsized":"2023-11-08T19:28:02.713309+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.727972+0000","last_change":"2023-11-08T19:28:02.723021+0000","last_active":"2023-11-08T19:28:02.727972+0000","last_peered":"2023-11-08T19:28:02.727972+0000","last_clean":"2023-11-08T19:28:02.727972+0000","last_became_active":"2023-11-08T19:28:02.722748+0000","last_became_peered":"2023-11-08T19:28:02.722748+0000","last_unstale":"2023-11-08T19:28:02.727972+0000","last_undegraded":"2023-11-08T19:28:02.727972+0000","last_fullsized":"2023-11-08T19:28:02.727972+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:26:52.310010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.695135+0000","last_change":"2023-11-08T19:27:08.189452+0000","last_active":"2023-11-08T19:28:01.695135+0000","last_peered":"2023-11-08T19:28:01.695135+0000","last_clean":"2023-11-08T19:28:01.695135+0000","last_became_active":"2023-11-08T19:27:08.189088+0000","last_became_peered":"2023-11-08T19:27:08.189088+0000","last_unstale":"2023-11-08T19:28:01.695135+0000","last_undegraded":"2023-11-08T19:28:01.695135+0000","last_fullsized":"2023-11-08T19:28:01.695135+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:25:18.963220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":59,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.694898+0000","last_change":"2023-11-08T19:26:06.296634+0000","last_active":"2023-11-08T19:28:01.694898+0000","last_peered":"2023-11-08T19:28:01.694898+0000","last_clean":"2023-11-08T19:28:01.694898+0000","last_became_active":"2023-11-08T19:26:04.603088+0000","last_became_peered":"2023-11-08T19:26:04.603088+0000","last_unstale":"2023-11-08T19:28:01.694898+0000","last_undegraded":"2023-11-08T19:28:01.694898+0000","last_fullsized":"2023-11-08T19:28:01.694898+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:01:56.991346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038180899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.695512+0000","last_change":"2023-11-08T19:27:09.431220+0000","last_active":"2023-11-08T19:28:01.695512+0000","last_peered":"2023-11-08T19:28:01.695512+0000","last_clean":"2023-11-08T19:28:01.695512+0000","last_became_active":"2023-11-08T19:27:07.191589+0000","last_became_peered":"2023-11-08T19:27:07.191589+0000","last_unstale":"2023-11-08T19:28:01.695512+0000","last_undegraded":"2023-11-08T19:28:01.695512+0000","last_fullsized":"2023-11-08T19:28:01.695512+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:18:19.964487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000729527,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.714312+0000","last_change":"2023-11-08T19:28:02.714312+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.714312+0000","last_undegraded":"2023-11-08T19:28:02.714312+0000","last_fullsized":"2023-11-08T19:28:02.714312+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"91'1","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.719567+0000","last_change":"2023-11-08T19:28:02.712293+0000","last_active":"2023-11-08T19:28:03.719567+0000","last_peered":"2023-11-08T19:28:03.719567+0000","last_clean":"2023-11-08T19:28:03.719567+0000","last_became_active":"2023-11-08T19:28:02.712002+0000","last_became_peered":"2023-11-08T19:28:02.712002+0000","last_unstale":"2023-11-08T19:28:03.719567+0000","last_undegraded":"2023-11-08T19:28:03.719567+0000","last_fullsized":"2023-11-08T19:28:03.719567+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:24:47.603187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.694985+0000","last_change":"2023-11-08T19:27:08.193337+0000","last_active":"2023-11-08T19:28:01.694985+0000","last_peered":"2023-11-08T19:28:01.694985+0000","last_clean":"2023-11-08T19:28:01.694985+0000","last_became_active":"2023-11-08T19:27:08.192997+0000","last_became_peered":"2023-11-08T19:27:08.192997+0000","last_unstale":"2023-11-08T19:28:01.694985+0000","last_undegraded":"2023-11-08T19:28:01.694985+0000","last_fullsized":"2023-11-08T19:28:01.694985+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:45:22.084015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":64,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.696307+0000","last_change":"2023-11-08T19:26:06.424511+0000","last_active":"2023-11-08T19:28:01.696307+0000","last_peered":"2023-11-08T19:28:01.696307+0000","last_clean":"2023-11-08T19:28:01.696307+0000","last_became_active":"2023-11-08T19:26:04.598973+0000","last_became_peered":"2023-11-08T19:26:04.598973+0000","last_unstale":"2023-11-08T19:28:01.696307+0000","last_undegraded":"2023-11-08T19:28:01.696307+0000","last_fullsized":"2023-11-08T19:28:01.696307+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:41:53.497456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031755900000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.694476+0000","last_change":"2023-11-08T19:27:09.497024+0000","last_active":"2023-11-08T19:28:01.694476+0000","last_peered":"2023-11-08T19:28:01.694476+0000","last_clean":"2023-11-08T19:28:01.694476+0000","last_became_active":"2023-11-08T19:27:07.190132+0000","last_became_peered":"2023-11-08T19:27:07.190132+0000","last_unstale":"2023-11-08T19:28:01.694476+0000","last_undegraded":"2023-11-08T19:28:01.694476+0000","last_fullsized":"2023-11-08T19:28:01.694476+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:36:34.881217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054126000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.718435+0000","last_change":"2023-11-08T19:28:03.713184+0000","last_active":"2023-11-08T19:28:03.718435+0000","last_peered":"2023-11-08T19:28:03.718435+0000","last_clean":"2023-11-08T19:28:03.718435+0000","last_became_active":"2023-11-08T19:28:03.712864+0000","last_became_peered":"2023-11-08T19:28:03.712864+0000","last_unstale":"2023-11-08T19:28:03.718435+0000","last_undegraded":"2023-11-08T19:28:03.718435+0000","last_fullsized":"2023-11-08T19:28:03.718435+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:38:54.805747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"91'1","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.716012+0000","last_change":"2023-11-08T19:28:02.713126+0000","last_active":"2023-11-08T19:28:03.716012+0000","last_peered":"2023-11-08T19:28:03.716012+0000","last_clean":"2023-11-08T19:28:03.716012+0000","last_became_active":"2023-11-08T19:28:02.712836+0000","last_became_peered":"2023-11-08T19:28:02.712836+0000","last_unstale":"2023-11-08T19:28:03.716012+0000","last_undegraded":"2023-11-08T19:28:03.716012+0000","last_fullsized":"2023-11-08T19:28:03.716012+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:29:31.276279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.695424+0000","last_change":"2023-11-08T19:27:08.194196+0000","last_active":"2023-11-08T19:28:01.695424+0000","last_peered":"2023-11-08T19:28:01.695424+0000","last_clean":"2023-11-08T19:28:01.695424+0000","last_became_active":"2023-11-08T19:27:08.193912+0000","last_became_peered":"2023-11-08T19:27:08.193912+0000","last_unstale":"2023-11-08T19:28:01.695424+0000","last_undegraded":"2023-11-08T19:28:01.695424+0000","last_fullsized":"2023-11-08T19:28:01.695424+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:25:18.963220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":59,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.695591+0000","last_change":"2023-11-08T19:26:06.296937+0000","last_active":"2023-11-08T19:28:01.695591+0000","last_peered":"2023-11-08T19:28:01.695591+0000","last_clean":"2023-11-08T19:28:01.695591+0000","last_became_active":"2023-11-08T19:26:04.599999+0000","last_became_peered":"2023-11-08T19:26:04.599999+0000","last_unstale":"2023-11-08T19:28:01.695591+0000","last_undegraded":"2023-11-08T19:28:01.695591+0000","last_fullsized":"2023-11-08T19:28:01.695591+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:50:43.827570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025605700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.695067+0000","last_change":"2023-11-08T19:27:09.497040+0000","last_active":"2023-11-08T19:28:01.695067+0000","last_peered":"2023-11-08T19:28:01.695067+0000","last_clean":"2023-11-08T19:28:01.695067+0000","last_became_active":"2023-11-08T19:27:07.189135+0000","last_became_peered":"2023-11-08T19:27:07.189135+0000","last_unstale":"2023-11-08T19:28:01.695067+0000","last_undegraded":"2023-11-08T19:28:01.695067+0000","last_fullsized":"2023-11-08T19:28:01.695067+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:12:29.400972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051794299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.703964+0000","last_change":"2023-11-08T19:28:02.703964+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.703964+0000","last_undegraded":"2023-11-08T19:28:02.703964+0000","last_fullsized":"2023-11-08T19:28:02.703964+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.710152+0000","last_change":"2023-11-08T19:28:02.705153+0000","last_active":"2023-11-08T19:28:02.710152+0000","last_peered":"2023-11-08T19:28:02.710152+0000","last_clean":"2023-11-08T19:28:02.710152+0000","last_became_active":"2023-11-08T19:28:02.704854+0000","last_became_peered":"2023-11-08T19:28:02.704854+0000","last_unstale":"2023-11-08T19:28:02.710152+0000","last_undegraded":"2023-11-08T19:28:02.710152+0000","last_fullsized":"2023-11-08T19:28:02.710152+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:25:00.088013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.703721+0000","last_change":"2023-11-08T19:27:08.187053+0000","last_active":"2023-11-08T19:28:02.703721+0000","last_peered":"2023-11-08T19:28:02.703721+0000","last_clean":"2023-11-08T19:28:02.703721+0000","last_became_active":"2023-11-08T19:27:08.186643+0000","last_became_peered":"2023-11-08T19:27:08.186643+0000","last_unstale":"2023-11-08T19:28:02.703721+0000","last_undegraded":"2023-11-08T19:28:02.703721+0000","last_fullsized":"2023-11-08T19:28:02.703721+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:35:30.073885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":71,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.700976+0000","last_change":"2023-11-08T19:26:05.609824+0000","last_active":"2023-11-08T19:28:02.700976+0000","last_peered":"2023-11-08T19:28:02.700976+0000","last_clean":"2023-11-08T19:28:02.700976+0000","last_became_active":"2023-11-08T19:26:04.603320+0000","last_became_peered":"2023-11-08T19:26:04.603320+0000","last_unstale":"2023-11-08T19:28:02.700976+0000","last_undegraded":"2023-11-08T19:28:02.700976+0000","last_fullsized":"2023-11-08T19:28:02.700976+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:05:45.976615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063225,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.694457+0000","last_change":"2023-11-08T19:27:09.496655+0000","last_active":"2023-11-08T19:28:01.694457+0000","last_peered":"2023-11-08T19:28:01.694457+0000","last_clean":"2023-11-08T19:28:01.694457+0000","last_became_active":"2023-11-08T19:27:07.189895+0000","last_became_peered":"2023-11-08T19:27:07.189895+0000","last_unstale":"2023-11-08T19:28:01.694457+0000","last_undegraded":"2023-11-08T19:28:01.694457+0000","last_fullsized":"2023-11-08T19:28:01.694457+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:41:08.822366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045654199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.711687+0000","last_change":"2023-11-08T19:28:02.711687+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.711687+0000","last_undegraded":"2023-11-08T19:28:02.711687+0000","last_fullsized":"2023-11-08T19:28:02.711687+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.711905+0000","last_change":"2023-11-08T19:28:02.709891+0000","last_active":"2023-11-08T19:28:02.711905+0000","last_peered":"2023-11-08T19:28:02.711905+0000","last_clean":"2023-11-08T19:28:02.711905+0000","last_became_active":"2023-11-08T19:28:02.709612+0000","last_became_peered":"2023-11-08T19:28:02.709612+0000","last_unstale":"2023-11-08T19:28:02.711905+0000","last_undegraded":"2023-11-08T19:28:02.711905+0000","last_fullsized":"2023-11-08T19:28:02.711905+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:24:47.603187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":60,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.698243+0000","last_change":"2023-11-08T19:26:06.541360+0000","last_active":"2023-11-08T19:28:01.698243+0000","last_peered":"2023-11-08T19:28:01.698243+0000","last_clean":"2023-11-08T19:28:01.698243+0000","last_became_active":"2023-11-08T19:26:04.603725+0000","last_became_peered":"2023-11-08T19:26:04.603725+0000","last_unstale":"2023-11-08T19:28:01.698243+0000","last_undegraded":"2023-11-08T19:28:01.698243+0000","last_fullsized":"2023-11-08T19:28:01.698243+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:28:48.576742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053365400000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.698592+0000","last_change":"2023-11-08T19:27:09.283599+0000","last_active":"2023-11-08T19:28:01.698592+0000","last_peered":"2023-11-08T19:28:01.698592+0000","last_clean":"2023-11-08T19:28:01.698592+0000","last_became_active":"2023-11-08T19:27:07.189595+0000","last_became_peered":"2023-11-08T19:27:07.189595+0000","last_unstale":"2023-11-08T19:28:01.698592+0000","last_undegraded":"2023-11-08T19:28:01.698592+0000","last_fullsized":"2023-11-08T19:28:01.698592+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:48:20.082682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000323479,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.696394+0000","last_change":"2023-11-08T19:27:08.185395+0000","last_active":"2023-11-08T19:28:01.696394+0000","last_peered":"2023-11-08T19:28:01.696394+0000","last_clean":"2023-11-08T19:28:01.696394+0000","last_became_active":"2023-11-08T19:27:08.184765+0000","last_became_peered":"2023-11-08T19:27:08.184765+0000","last_unstale":"2023-11-08T19:28:01.696394+0000","last_undegraded":"2023-11-08T19:28:01.696394+0000","last_fullsized":"2023-11-08T19:28:01.696394+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:08:51.864418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.712306+0000","last_change":"2023-11-08T19:28:02.712306+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.712306+0000","last_undegraded":"2023-11-08T19:28:02.712306+0000","last_fullsized":"2023-11-08T19:28:02.712306+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.725515+0000","last_change":"2023-11-08T19:28:02.720596+0000","last_active":"2023-11-08T19:28:02.725515+0000","last_peered":"2023-11-08T19:28:02.725515+0000","last_clean":"2023-11-08T19:28:02.725515+0000","last_became_active":"2023-11-08T19:28:02.720144+0000","last_became_peered":"2023-11-08T19:28:02.720144+0000","last_unstale":"2023-11-08T19:28:02.725515+0000","last_undegraded":"2023-11-08T19:28:02.725515+0000","last_fullsized":"2023-11-08T19:28:02.725515+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:51:33.542656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":64,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.694624+0000","last_change":"2023-11-08T19:26:06.424236+0000","last_active":"2023-11-08T19:28:01.694624+0000","last_peered":"2023-11-08T19:28:01.694624+0000","last_clean":"2023-11-08T19:28:01.694624+0000","last_became_active":"2023-11-08T19:26:04.600115+0000","last_became_peered":"2023-11-08T19:26:04.600115+0000","last_unstale":"2023-11-08T19:28:01.694624+0000","last_undegraded":"2023-11-08T19:28:01.694624+0000","last_fullsized":"2023-11-08T19:28:01.694624+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:18:01.876605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065803900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.695283+0000","last_change":"2023-11-08T19:27:09.331899+0000","last_active":"2023-11-08T19:28:01.695283+0000","last_peered":"2023-11-08T19:28:01.695283+0000","last_clean":"2023-11-08T19:28:01.695283+0000","last_became_active":"2023-11-08T19:27:07.191460+0000","last_became_peered":"2023-11-08T19:27:07.191460+0000","last_unstale":"2023-11-08T19:28:01.695283+0000","last_undegraded":"2023-11-08T19:28:01.695283+0000","last_fullsized":"2023-11-08T19:28:01.695283+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:51:29.247104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044907500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.701089+0000","last_change":"2023-11-08T19:27:08.191802+0000","last_active":"2023-11-08T19:28:02.701089+0000","last_peered":"2023-11-08T19:28:02.701089+0000","last_clean":"2023-11-08T19:28:02.701089+0000","last_became_active":"2023-11-08T19:27:08.191520+0000","last_became_peered":"2023-11-08T19:27:08.191520+0000","last_unstale":"2023-11-08T19:28:02.701089+0000","last_undegraded":"2023-11-08T19:28:02.701089+0000","last_fullsized":"2023-11-08T19:28:02.701089+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:49:16.458634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.724704+0000","last_change":"2023-11-08T19:28:03.722086+0000","last_active":"2023-11-08T19:28:03.724704+0000","last_peered":"2023-11-08T19:28:03.724704+0000","last_clean":"2023-11-08T19:28:03.724704+0000","last_became_active":"2023-11-08T19:28:03.721713+0000","last_became_peered":"2023-11-08T19:28:03.721713+0000","last_unstale":"2023-11-08T19:28:03.724704+0000","last_undegraded":"2023-11-08T19:28:03.724704+0000","last_fullsized":"2023-11-08T19:28:03.724704+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:34:22.087376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.705286+0000","last_change":"2023-11-08T19:28:02.707105+0000","last_active":"2023-11-08T19:28:03.705286+0000","last_peered":"2023-11-08T19:28:03.705286+0000","last_clean":"2023-11-08T19:28:03.705286+0000","last_became_active":"2023-11-08T19:28:02.706615+0000","last_became_peered":"2023-11-08T19:28:02.706615+0000","last_unstale":"2023-11-08T19:28:03.705286+0000","last_undegraded":"2023-11-08T19:28:03.705286+0000","last_fullsized":"2023-11-08T19:28:03.705286+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:18:55.364855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":71,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.699281+0000","last_change":"2023-11-08T19:26:05.608892+0000","last_active":"2023-11-08T19:28:02.699281+0000","last_peered":"2023-11-08T19:28:02.699281+0000","last_clean":"2023-11-08T19:28:02.699281+0000","last_became_active":"2023-11-08T19:26:04.604519+0000","last_became_peered":"2023-11-08T19:26:04.604519+0000","last_unstale":"2023-11-08T19:28:02.699281+0000","last_undegraded":"2023-11-08T19:28:02.699281+0000","last_fullsized":"2023-11-08T19:28:02.699281+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:11:26.508634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051248099999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.700897+0000","last_change":"2023-11-08T19:27:09.181431+0000","last_active":"2023-11-08T19:28:02.700897+0000","last_peered":"2023-11-08T19:28:02.700897+0000","last_clean":"2023-11-08T19:28:02.700897+0000","last_became_active":"2023-11-08T19:27:07.190800+0000","last_became_peered":"2023-11-08T19:27:07.190800+0000","last_unstale":"2023-11-08T19:28:02.700897+0000","last_undegraded":"2023-11-08T19:28:02.700897+0000","last_fullsized":"2023-11-08T19:28:02.700897+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:46:30.452678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656099999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.696596+0000","last_change":"2023-11-08T19:27:08.187983+0000","last_active":"2023-11-08T19:28:01.696596+0000","last_peered":"2023-11-08T19:28:01.696596+0000","last_clean":"2023-11-08T19:28:01.696596+0000","last_became_active":"2023-11-08T19:27:08.187322+0000","last_became_peered":"2023-11-08T19:27:08.187322+0000","last_unstale":"2023-11-08T19:28:01.696596+0000","last_undegraded":"2023-11-08T19:28:01.696596+0000","last_fullsized":"2023-11-08T19:28:01.696596+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:33:07.647519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.704461+0000","last_change":"2023-11-08T19:28:02.704461+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.704461+0000","last_undegraded":"2023-11-08T19:28:02.704461+0000","last_fullsized":"2023-11-08T19:28:02.704461+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.725558+0000","last_change":"2023-11-08T19:28:02.720466+0000","last_active":"2023-11-08T19:28:02.725558+0000","last_peered":"2023-11-08T19:28:02.725558+0000","last_clean":"2023-11-08T19:28:02.725558+0000","last_became_active":"2023-11-08T19:28:02.720193+0000","last_became_peered":"2023-11-08T19:28:02.720193+0000","last_unstale":"2023-11-08T19:28:02.725558+0000","last_undegraded":"2023-11-08T19:28:02.725558+0000","last_fullsized":"2023-11-08T19:28:02.725558+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:26:23.134605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":58,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.694087+0000","last_change":"2023-11-08T19:26:06.394075+0000","last_active":"2023-11-08T19:28:01.694087+0000","last_peered":"2023-11-08T19:28:01.694087+0000","last_clean":"2023-11-08T19:28:01.694087+0000","last_became_active":"2023-11-08T19:26:04.606967+0000","last_became_peered":"2023-11-08T19:26:04.606967+0000","last_unstale":"2023-11-08T19:28:01.694087+0000","last_undegraded":"2023-11-08T19:28:01.694087+0000","last_fullsized":"2023-11-08T19:28:01.694087+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:19:56.286835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040971999999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.701127+0000","last_change":"2023-11-08T19:27:09.180722+0000","last_active":"2023-11-08T19:28:02.701127+0000","last_peered":"2023-11-08T19:28:02.701127+0000","last_clean":"2023-11-08T19:28:02.701127+0000","last_became_active":"2023-11-08T19:27:07.186977+0000","last_became_peered":"2023-11-08T19:27:07.186977+0000","last_unstale":"2023-11-08T19:28:02.701127+0000","last_undegraded":"2023-11-08T19:28:02.701127+0000","last_fullsized":"2023-11-08T19:28:02.701127+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:17:59.966463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057675100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.694729+0000","last_change":"2023-11-08T19:27:08.193963+0000","last_active":"2023-11-08T19:28:01.694729+0000","last_peered":"2023-11-08T19:28:01.694729+0000","last_clean":"2023-11-08T19:28:01.694729+0000","last_became_active":"2023-11-08T19:27:08.193098+0000","last_became_peered":"2023-11-08T19:27:08.193098+0000","last_unstale":"2023-11-08T19:28:01.694729+0000","last_undegraded":"2023-11-08T19:28:01.694729+0000","last_fullsized":"2023-11-08T19:28:01.694729+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:41:09.827370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.699774+0000","last_change":"2023-11-08T19:28:02.699774+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.699774+0000","last_undegraded":"2023-11-08T19:28:02.699774+0000","last_fullsized":"2023-11-08T19:28:02.699774+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.722618+0000","last_change":"2023-11-08T19:28:02.718059+0000","last_active":"2023-11-08T19:28:02.722618+0000","last_peered":"2023-11-08T19:28:02.722618+0000","last_clean":"2023-11-08T19:28:02.722618+0000","last_became_active":"2023-11-08T19:28:02.717775+0000","last_became_peered":"2023-11-08T19:28:02.717775+0000","last_unstale":"2023-11-08T19:28:02.722618+0000","last_undegraded":"2023-11-08T19:28:02.722618+0000","last_fullsized":"2023-11-08T19:28:02.722618+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:29:25.082011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":58,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.697065+0000","last_change":"2023-11-08T19:26:06.265686+0000","last_active":"2023-11-08T19:28:01.697065+0000","last_peered":"2023-11-08T19:28:01.697065+0000","last_clean":"2023-11-08T19:28:01.697065+0000","last_became_active":"2023-11-08T19:26:04.602111+0000","last_became_peered":"2023-11-08T19:26:04.602111+0000","last_unstale":"2023-11-08T19:28:01.697065+0000","last_undegraded":"2023-11-08T19:28:01.697065+0000","last_fullsized":"2023-11-08T19:28:01.697065+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:40:11.164951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056197899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.701328+0000","last_change":"2023-11-08T19:27:09.181333+0000","last_active":"2023-11-08T19:28:02.701328+0000","last_peered":"2023-11-08T19:28:02.701328+0000","last_clean":"2023-11-08T19:28:02.701328+0000","last_became_active":"2023-11-08T19:27:07.188657+0000","last_became_peered":"2023-11-08T19:27:07.188657+0000","last_unstale":"2023-11-08T19:28:02.701328+0000","last_undegraded":"2023-11-08T19:28:02.701328+0000","last_fullsized":"2023-11-08T19:28:02.701328+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:25:56.788615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001227393,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.701395+0000","last_change":"2023-11-08T19:27:08.188533+0000","last_active":"2023-11-08T19:28:02.701395+0000","last_peered":"2023-11-08T19:28:02.701395+0000","last_clean":"2023-11-08T19:28:02.701395+0000","last_became_active":"2023-11-08T19:27:08.188240+0000","last_became_peered":"2023-11-08T19:27:08.188240+0000","last_unstale":"2023-11-08T19:28:02.701395+0000","last_undegraded":"2023-11-08T19:28:02.701395+0000","last_fullsized":"2023-11-08T19:28:02.701395+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:55:37.653521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.701682+0000","last_change":"2023-11-08T19:28:02.701682+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.701682+0000","last_undegraded":"2023-11-08T19:28:02.701682+0000","last_fullsized":"2023-11-08T19:28:02.701682+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.714684+0000","last_change":"2023-11-08T19:28:02.710864+0000","last_active":"2023-11-08T19:28:02.714684+0000","last_peered":"2023-11-08T19:28:02.714684+0000","last_clean":"2023-11-08T19:28:02.714684+0000","last_became_active":"2023-11-08T19:28:02.710597+0000","last_became_peered":"2023-11-08T19:28:02.710597+0000","last_unstale":"2023-11-08T19:28:02.714684+0000","last_undegraded":"2023-11-08T19:28:02.714684+0000","last_fullsized":"2023-11-08T19:28:02.714684+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:02:43.242472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.702305+0000","last_change":"2023-11-08T19:27:08.188458+0000","last_active":"2023-11-08T19:28:02.702305+0000","last_peered":"2023-11-08T19:28:02.702305+0000","last_clean":"2023-11-08T19:28:02.702305+0000","last_became_active":"2023-11-08T19:27:08.187880+0000","last_became_peered":"2023-11-08T19:27:08.187880+0000","last_unstale":"2023-11-08T19:28:02.702305+0000","last_undegraded":"2023-11-08T19:28:02.702305+0000","last_fullsized":"2023-11-08T19:28:02.702305+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:25:28.618214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":59,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.690530+0000","last_change":"2023-11-08T19:26:05.606898+0000","last_active":"2023-11-08T19:28:01.690530+0000","last_peered":"2023-11-08T19:28:01.690530+0000","last_clean":"2023-11-08T19:28:01.690530+0000","last_became_active":"2023-11-08T19:26:04.603955+0000","last_became_peered":"2023-11-08T19:26:04.603955+0000","last_unstale":"2023-11-08T19:28:01.690530+0000","last_undegraded":"2023-11-08T19:28:01.690530+0000","last_fullsized":"2023-11-08T19:28:01.690530+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:19:11.589559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036473499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.690241+0000","last_change":"2023-11-08T19:27:09.178770+0000","last_active":"2023-11-08T19:28:01.690241+0000","last_peered":"2023-11-08T19:28:01.690241+0000","last_clean":"2023-11-08T19:28:01.690241+0000","last_became_active":"2023-11-08T19:27:07.194992+0000","last_became_peered":"2023-11-08T19:27:07.194992+0000","last_unstale":"2023-11-08T19:28:01.690241+0000","last_undegraded":"2023-11-08T19:28:01.690241+0000","last_fullsized":"2023-11-08T19:28:01.690241+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:35:34.026910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000770285,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.697682+0000","last_change":"2023-11-08T19:28:02.697682+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.697682+0000","last_undegraded":"2023-11-08T19:28:02.697682+0000","last_fullsized":"2023-11-08T19:28:02.697682+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.703765+0000","last_change":"2023-11-08T19:28:02.707011+0000","last_active":"2023-11-08T19:28:03.703765+0000","last_peered":"2023-11-08T19:28:03.703765+0000","last_clean":"2023-11-08T19:28:03.703765+0000","last_became_active":"2023-11-08T19:28:02.706729+0000","last_became_peered":"2023-11-08T19:28:02.706729+0000","last_unstale":"2023-11-08T19:28:03.703765+0000","last_undegraded":"2023-11-08T19:28:03.703765+0000","last_fullsized":"2023-11-08T19:28:03.703765+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:29:25.082011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.695698+0000","last_change":"2023-11-08T19:27:08.193953+0000","last_active":"2023-11-08T19:28:01.695698+0000","last_peered":"2023-11-08T19:28:01.695698+0000","last_clean":"2023-11-08T19:28:01.695698+0000","last_became_active":"2023-11-08T19:27:08.193653+0000","last_became_peered":"2023-11-08T19:27:08.193653+0000","last_unstale":"2023-11-08T19:28:01.695698+0000","last_undegraded":"2023-11-08T19:28:01.695698+0000","last_fullsized":"2023-11-08T19:28:01.695698+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:35:30.073885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":64,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.694881+0000","last_change":"2023-11-08T19:26:06.424464+0000","last_active":"2023-11-08T19:28:01.694881+0000","last_peered":"2023-11-08T19:28:01.694881+0000","last_clean":"2023-11-08T19:28:01.694881+0000","last_became_active":"2023-11-08T19:26:04.603144+0000","last_became_peered":"2023-11-08T19:26:04.603144+0000","last_unstale":"2023-11-08T19:28:01.694881+0000","last_undegraded":"2023-11-08T19:28:01.694881+0000","last_fullsized":"2023-11-08T19:28:01.694881+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:42:55.269365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054090799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.701001+0000","last_change":"2023-11-08T19:27:09.180479+0000","last_active":"2023-11-08T19:28:02.701001+0000","last_peered":"2023-11-08T19:28:02.701001+0000","last_clean":"2023-11-08T19:28:02.701001+0000","last_became_active":"2023-11-08T19:27:07.190438+0000","last_became_peered":"2023-11-08T19:27:07.190438+0000","last_unstale":"2023-11-08T19:28:02.701001+0000","last_undegraded":"2023-11-08T19:28:02.701001+0000","last_fullsized":"2023-11-08T19:28:02.701001+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:25:14.487524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050893300000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.704857+0000","last_change":"2023-11-08T19:28:02.704857+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.704857+0000","last_undegraded":"2023-11-08T19:28:02.704857+0000","last_fullsized":"2023-11-08T19:28:02.704857+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.715287+0000","last_change":"2023-11-08T19:28:02.708336+0000","last_active":"2023-11-08T19:28:02.715287+0000","last_peered":"2023-11-08T19:28:02.715287+0000","last_clean":"2023-11-08T19:28:02.715287+0000","last_became_active":"2023-11-08T19:28:02.708016+0000","last_became_peered":"2023-11-08T19:28:02.708016+0000","last_unstale":"2023-11-08T19:28:02.715287+0000","last_undegraded":"2023-11-08T19:28:02.715287+0000","last_fullsized":"2023-11-08T19:28:02.715287+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:39:02.748118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.696341+0000","last_change":"2023-11-08T19:27:08.190530+0000","last_active":"2023-11-08T19:28:01.696341+0000","last_peered":"2023-11-08T19:28:01.696341+0000","last_clean":"2023-11-08T19:28:01.696341+0000","last_became_active":"2023-11-08T19:27:08.190203+0000","last_became_peered":"2023-11-08T19:27:08.190203+0000","last_unstale":"2023-11-08T19:28:01.696341+0000","last_undegraded":"2023-11-08T19:28:01.696341+0000","last_fullsized":"2023-11-08T19:28:01.696341+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:26:32.777602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":60,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.698474+0000","last_change":"2023-11-08T19:26:06.541623+0000","last_active":"2023-11-08T19:28:01.698474+0000","last_peered":"2023-11-08T19:28:01.698474+0000","last_clean":"2023-11-08T19:28:01.698474+0000","last_became_active":"2023-11-08T19:26:04.601961+0000","last_became_peered":"2023-11-08T19:26:04.601961+0000","last_unstale":"2023-11-08T19:28:01.698474+0000","last_undegraded":"2023-11-08T19:28:01.698474+0000","last_fullsized":"2023-11-08T19:28:01.698474+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:11:26.508634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000466386,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.694709+0000","last_change":"2023-11-08T19:27:09.431231+0000","last_active":"2023-11-08T19:28:01.694709+0000","last_peered":"2023-11-08T19:28:01.694709+0000","last_clean":"2023-11-08T19:28:01.694709+0000","last_became_active":"2023-11-08T19:27:07.191583+0000","last_became_peered":"2023-11-08T19:27:07.191583+0000","last_unstale":"2023-11-08T19:28:01.694709+0000","last_undegraded":"2023-11-08T19:28:01.694709+0000","last_fullsized":"2023-11-08T19:28:01.694709+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:58:48.357266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093695200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.698528+0000","last_change":"2023-11-08T19:28:02.698528+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.698528+0000","last_undegraded":"2023-11-08T19:28:02.698528+0000","last_fullsized":"2023-11-08T19:28:02.698528+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"91'1","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.717319+0000","last_change":"2023-11-08T19:28:02.708326+0000","last_active":"2023-11-08T19:28:03.717319+0000","last_peered":"2023-11-08T19:28:03.717319+0000","last_clean":"2023-11-08T19:28:03.717319+0000","last_became_active":"2023-11-08T19:28:02.708055+0000","last_became_peered":"2023-11-08T19:28:02.708055+0000","last_unstale":"2023-11-08T19:28:03.717319+0000","last_undegraded":"2023-11-08T19:28:03.717319+0000","last_fullsized":"2023-11-08T19:28:03.717319+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:50:41.723104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.694787+0000","last_change":"2023-11-08T19:27:08.187900+0000","last_active":"2023-11-08T19:28:01.694787+0000","last_peered":"2023-11-08T19:28:01.694787+0000","last_clean":"2023-11-08T19:28:01.694787+0000","last_became_active":"2023-11-08T19:27:08.187626+0000","last_became_peered":"2023-11-08T19:27:08.187626+0000","last_unstale":"2023-11-08T19:28:01.694787+0000","last_undegraded":"2023-11-08T19:28:01.694787+0000","last_fullsized":"2023-11-08T19:28:01.694787+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:00:55.878397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":71,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.701751+0000","last_change":"2023-11-08T19:26:05.608980+0000","last_active":"2023-11-08T19:28:02.701751+0000","last_peered":"2023-11-08T19:28:02.701751+0000","last_clean":"2023-11-08T19:28:02.701751+0000","last_became_active":"2023-11-08T19:26:04.606386+0000","last_became_peered":"2023-11-08T19:26:04.606386+0000","last_unstale":"2023-11-08T19:28:02.701751+0000","last_undegraded":"2023-11-08T19:28:02.701751+0000","last_fullsized":"2023-11-08T19:28:02.701751+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:41:50.771457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037950599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.690460+0000","last_change":"2023-11-08T19:27:09.179037+0000","last_active":"2023-11-08T19:28:01.690460+0000","last_peered":"2023-11-08T19:28:01.690460+0000","last_clean":"2023-11-08T19:28:01.690460+0000","last_became_active":"2023-11-08T19:27:07.195033+0000","last_became_peered":"2023-11-08T19:27:07.195033+0000","last_unstale":"2023-11-08T19:28:01.690460+0000","last_undegraded":"2023-11-08T19:28:01.690460+0000","last_fullsized":"2023-11-08T19:28:01.690460+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:19:08.253483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041824800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.697868+0000","last_change":"2023-11-08T19:28:02.697868+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.697868+0000","last_undegraded":"2023-11-08T19:28:02.697868+0000","last_fullsized":"2023-11-08T19:28:02.697868+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.713986+0000","last_change":"2023-11-08T19:28:02.712619+0000","last_active":"2023-11-08T19:28:02.713986+0000","last_peered":"2023-11-08T19:28:02.713986+0000","last_clean":"2023-11-08T19:28:02.713986+0000","last_became_active":"2023-11-08T19:28:02.712342+0000","last_became_peered":"2023-11-08T19:28:02.712342+0000","last_unstale":"2023-11-08T19:28:02.713986+0000","last_undegraded":"2023-11-08T19:28:02.713986+0000","last_fullsized":"2023-11-08T19:28:02.713986+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:11:10.165281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.693976+0000","last_change":"2023-11-08T19:27:08.193761+0000","last_active":"2023-11-08T19:28:01.693976+0000","last_peered":"2023-11-08T19:28:01.693976+0000","last_clean":"2023-11-08T19:28:01.693976+0000","last_became_active":"2023-11-08T19:27:08.193017+0000","last_became_peered":"2023-11-08T19:27:08.193017+0000","last_unstale":"2023-11-08T19:28:01.693976+0000","last_undegraded":"2023-11-08T19:28:01.693976+0000","last_fullsized":"2023-11-08T19:28:01.693976+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:18:00.971466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":64,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.694700+0000","last_change":"2023-11-08T19:26:06.424073+0000","last_active":"2023-11-08T19:28:01.694700+0000","last_peered":"2023-11-08T19:28:01.694700+0000","last_clean":"2023-11-08T19:28:01.694700+0000","last_became_active":"2023-11-08T19:26:04.599046+0000","last_became_peered":"2023-11-08T19:26:04.599046+0000","last_unstale":"2023-11-08T19:28:01.694700+0000","last_undegraded":"2023-11-08T19:28:01.694700+0000","last_fullsized":"2023-11-08T19:28:01.694700+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:04:43.562566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035092199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.700565+0000","last_change":"2023-11-08T19:27:09.181716+0000","last_active":"2023-11-08T19:28:02.700565+0000","last_peered":"2023-11-08T19:28:02.700565+0000","last_clean":"2023-11-08T19:28:02.700565+0000","last_became_active":"2023-11-08T19:27:07.187077+0000","last_became_peered":"2023-11-08T19:27:07.187077+0000","last_unstale":"2023-11-08T19:28:02.700565+0000","last_undegraded":"2023-11-08T19:28:02.700565+0000","last_fullsized":"2023-11-08T19:28:02.700565+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:51:29.247104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056673599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.726360+0000","last_change":"2023-11-08T19:28:03.724351+0000","last_active":"2023-11-08T19:28:03.726360+0000","last_peered":"2023-11-08T19:28:03.726360+0000","last_clean":"2023-11-08T19:28:03.726360+0000","last_became_active":"2023-11-08T19:28:03.724109+0000","last_became_peered":"2023-11-08T19:28:03.724109+0000","last_unstale":"2023-11-08T19:28:03.726360+0000","last_undegraded":"2023-11-08T19:28:03.726360+0000","last_fullsized":"2023-11-08T19:28:03.726360+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:11:11.170102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.716963+0000","last_change":"2023-11-08T19:28:02.711594+0000","last_active":"2023-11-08T19:28:02.716963+0000","last_peered":"2023-11-08T19:28:02.716963+0000","last_clean":"2023-11-08T19:28:02.716963+0000","last_became_active":"2023-11-08T19:28:02.711204+0000","last_became_peered":"2023-11-08T19:28:02.711204+0000","last_unstale":"2023-11-08T19:28:02.716963+0000","last_undegraded":"2023-11-08T19:28:02.716963+0000","last_fullsized":"2023-11-08T19:28:02.716963+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:21:23.673336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.696597+0000","last_change":"2023-11-08T19:27:08.188143+0000","last_active":"2023-11-08T19:28:01.696597+0000","last_peered":"2023-11-08T19:28:01.696597+0000","last_clean":"2023-11-08T19:28:01.696597+0000","last_became_active":"2023-11-08T19:27:08.187714+0000","last_became_peered":"2023-11-08T19:27:08.187714+0000","last_unstale":"2023-11-08T19:28:01.696597+0000","last_undegraded":"2023-11-08T19:28:01.696597+0000","last_fullsized":"2023-11-08T19:28:01.696597+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:02:10.863403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":58,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.693970+0000","last_change":"2023-11-08T19:26:06.394320+0000","last_active":"2023-11-08T19:28:01.693970+0000","last_peered":"2023-11-08T19:28:01.693970+0000","last_clean":"2023-11-08T19:28:01.693970+0000","last_became_active":"2023-11-08T19:26:04.603212+0000","last_became_peered":"2023-11-08T19:26:04.603212+0000","last_unstale":"2023-11-08T19:28:01.693970+0000","last_undegraded":"2023-11-08T19:28:01.693970+0000","last_fullsized":"2023-11-08T19:28:01.693970+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:25:29.947079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054695399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.700819+0000","last_change":"2023-11-08T19:27:09.181613+0000","last_active":"2023-11-08T19:28:02.700819+0000","last_peered":"2023-11-08T19:28:02.700819+0000","last_clean":"2023-11-08T19:28:02.700819+0000","last_became_active":"2023-11-08T19:27:07.190897+0000","last_became_peered":"2023-11-08T19:27:07.190897+0000","last_unstale":"2023-11-08T19:28:02.700819+0000","last_undegraded":"2023-11-08T19:28:02.700819+0000","last_fullsized":"2023-11-08T19:28:02.700819+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:27:57.388125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011172459999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.698922+0000","last_change":"2023-11-08T19:28:02.698922+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.698922+0000","last_undegraded":"2023-11-08T19:28:02.698922+0000","last_fullsized":"2023-11-08T19:28:02.698922+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.711981+0000","last_change":"2023-11-08T19:28:02.707358+0000","last_active":"2023-11-08T19:28:02.711981+0000","last_peered":"2023-11-08T19:28:02.711981+0000","last_clean":"2023-11-08T19:28:02.711981+0000","last_became_active":"2023-11-08T19:28:02.707109+0000","last_became_peered":"2023-11-08T19:28:02.707109+0000","last_unstale":"2023-11-08T19:28:02.711981+0000","last_undegraded":"2023-11-08T19:28:02.711981+0000","last_fullsized":"2023-11-08T19:28:02.711981+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:32:42.691100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.696667+0000","last_change":"2023-11-08T19:27:08.188668+0000","last_active":"2023-11-08T19:28:01.696667+0000","last_peered":"2023-11-08T19:28:01.696667+0000","last_clean":"2023-11-08T19:28:01.696667+0000","last_became_active":"2023-11-08T19:27:08.188441+0000","last_became_peered":"2023-11-08T19:27:08.188441+0000","last_unstale":"2023-11-08T19:28:01.696667+0000","last_undegraded":"2023-11-08T19:28:01.696667+0000","last_fullsized":"2023-11-08T19:28:01.696667+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:45:44.379160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":64,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.694995+0000","last_change":"2023-11-08T19:26:06.423754+0000","last_active":"2023-11-08T19:28:01.694995+0000","last_peered":"2023-11-08T19:28:01.694995+0000","last_clean":"2023-11-08T19:28:01.694995+0000","last_became_active":"2023-11-08T19:26:04.599322+0000","last_became_peered":"2023-11-08T19:26:04.599322+0000","last_unstale":"2023-11-08T19:28:01.694995+0000","last_undegraded":"2023-11-08T19:28:01.694995+0000","last_fullsized":"2023-11-08T19:28:01.694995+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:13:08.689128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042670000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.701576+0000","last_change":"2023-11-08T19:27:09.181656+0000","last_active":"2023-11-08T19:28:02.701576+0000","last_peered":"2023-11-08T19:28:02.701576+0000","last_clean":"2023-11-08T19:28:02.701576+0000","last_became_active":"2023-11-08T19:27:07.187337+0000","last_became_peered":"2023-11-08T19:27:07.187337+0000","last_unstale":"2023-11-08T19:28:02.701576+0000","last_undegraded":"2023-11-08T19:28:02.701576+0000","last_fullsized":"2023-11-08T19:28:02.701576+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:14:17.605057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000346967,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.698224+0000","last_change":"2023-11-08T19:28:02.698224+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.698224+0000","last_undegraded":"2023-11-08T19:28:02.698224+0000","last_fullsized":"2023-11-08T19:28:02.698224+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.712011+0000","last_change":"2023-11-08T19:28:02.704760+0000","last_active":"2023-11-08T19:28:02.712011+0000","last_peered":"2023-11-08T19:28:02.712011+0000","last_clean":"2023-11-08T19:28:02.712011+0000","last_became_active":"2023-11-08T19:28:02.704410+0000","last_became_peered":"2023-11-08T19:28:02.704410+0000","last_unstale":"2023-11-08T19:28:02.712011+0000","last_undegraded":"2023-11-08T19:28:02.712011+0000","last_fullsized":"2023-11-08T19:28:02.712011+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:00:10.329151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.694901+0000","last_change":"2023-11-08T19:27:08.194065+0000","last_active":"2023-11-08T19:28:01.694901+0000","last_peered":"2023-11-08T19:28:01.694901+0000","last_clean":"2023-11-08T19:28:01.694901+0000","last_became_active":"2023-11-08T19:27:08.193532+0000","last_became_peered":"2023-11-08T19:27:08.193532+0000","last_unstale":"2023-11-08T19:28:01.694901+0000","last_undegraded":"2023-11-08T19:28:01.694901+0000","last_fullsized":"2023-11-08T19:28:01.694901+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:27:58.393129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":58,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.695354+0000","last_change":"2023-11-08T19:26:06.394488+0000","last_active":"2023-11-08T19:28:01.695354+0000","last_peered":"2023-11-08T19:28:01.695354+0000","last_clean":"2023-11-08T19:28:01.695354+0000","last_became_active":"2023-11-08T19:26:04.606805+0000","last_became_peered":"2023-11-08T19:26:04.606805+0000","last_unstale":"2023-11-08T19:28:01.695354+0000","last_undegraded":"2023-11-08T19:28:01.695354+0000","last_fullsized":"2023-11-08T19:28:01.695354+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:25:20.857133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045317000000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.696700+0000","last_change":"2023-11-08T19:27:09.211647+0000","last_active":"2023-11-08T19:28:01.696700+0000","last_peered":"2023-11-08T19:28:01.696700+0000","last_clean":"2023-11-08T19:28:01.696700+0000","last_became_active":"2023-11-08T19:27:07.190309+0000","last_became_peered":"2023-11-08T19:27:07.190309+0000","last_unstale":"2023-11-08T19:28:01.696700+0000","last_undegraded":"2023-11-08T19:28:01.696700+0000","last_fullsized":"2023-11-08T19:28:01.696700+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:56:04.292154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047615199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.711664+0000","last_change":"2023-11-08T19:28:02.711664+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.711664+0000","last_undegraded":"2023-11-08T19:28:02.711664+0000","last_fullsized":"2023-11-08T19:28:02.711664+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.722917+0000","last_change":"2023-11-08T19:28:02.717508+0000","last_active":"2023-11-08T19:28:02.722917+0000","last_peered":"2023-11-08T19:28:02.722917+0000","last_clean":"2023-11-08T19:28:02.722917+0000","last_became_active":"2023-11-08T19:28:02.717126+0000","last_became_peered":"2023-11-08T19:28:02.717126+0000","last_unstale":"2023-11-08T19:28:02.722917+0000","last_undegraded":"2023-11-08T19:28:02.722917+0000","last_fullsized":"2023-11-08T19:28:02.722917+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:23:34.816577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.695560+0000","last_change":"2023-11-08T19:27:08.194884+0000","last_active":"2023-11-08T19:28:01.695560+0000","last_peered":"2023-11-08T19:28:01.695560+0000","last_clean":"2023-11-08T19:28:01.695560+0000","last_became_active":"2023-11-08T19:27:08.193241+0000","last_became_peered":"2023-11-08T19:27:08.193241+0000","last_unstale":"2023-11-08T19:28:01.695560+0000","last_undegraded":"2023-11-08T19:28:01.695560+0000","last_fullsized":"2023-11-08T19:28:01.695560+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:46:31.457682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":58,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.696871+0000","last_change":"2023-11-08T19:26:06.265907+0000","last_active":"2023-11-08T19:28:01.696871+0000","last_peered":"2023-11-08T19:28:01.696871+0000","last_clean":"2023-11-08T19:28:01.696871+0000","last_became_active":"2023-11-08T19:26:04.604772+0000","last_became_peered":"2023-11-08T19:26:04.604772+0000","last_unstale":"2023-11-08T19:28:01.696871+0000","last_undegraded":"2023-11-08T19:28:01.696871+0000","last_fullsized":"2023-11-08T19:28:01.696871+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:44:16.139794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066875300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.698318+0000","last_change":"2023-11-08T19:27:09.282699+0000","last_active":"2023-11-08T19:28:01.698318+0000","last_peered":"2023-11-08T19:28:01.698318+0000","last_clean":"2023-11-08T19:28:01.698318+0000","last_became_active":"2023-11-08T19:27:07.191212+0000","last_became_peered":"2023-11-08T19:27:07.191212+0000","last_unstale":"2023-11-08T19:28:01.698318+0000","last_undegraded":"2023-11-08T19:28:01.698318+0000","last_fullsized":"2023-11-08T19:28:01.698318+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:35:14.715110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046640800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.705457+0000","last_change":"2023-11-08T19:28:02.705457+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.705457+0000","last_undegraded":"2023-11-08T19:28:02.705457+0000","last_fullsized":"2023-11-08T19:28:02.705457+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.717068+0000","last_change":"2023-11-08T19:28:02.712356+0000","last_active":"2023-11-08T19:28:02.717068+0000","last_peered":"2023-11-08T19:28:02.717068+0000","last_clean":"2023-11-08T19:28:02.717068+0000","last_became_active":"2023-11-08T19:28:02.712073+0000","last_became_peered":"2023-11-08T19:28:02.712073+0000","last_unstale":"2023-11-08T19:28:02.717068+0000","last_undegraded":"2023-11-08T19:28:02.717068+0000","last_fullsized":"2023-11-08T19:28:02.717068+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:06:46.912027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.696801+0000","last_change":"2023-11-08T19:27:08.187644+0000","last_active":"2023-11-08T19:28:01.696801+0000","last_peered":"2023-11-08T19:28:01.696801+0000","last_clean":"2023-11-08T19:28:01.696801+0000","last_became_active":"2023-11-08T19:27:08.187376+0000","last_became_peered":"2023-11-08T19:27:08.187376+0000","last_unstale":"2023-11-08T19:28:01.696801+0000","last_undegraded":"2023-11-08T19:28:01.696801+0000","last_fullsized":"2023-11-08T19:28:01.696801+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:22:40.300185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":59,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.690307+0000","last_change":"2023-11-08T19:26:05.606946+0000","last_active":"2023-11-08T19:28:01.690307+0000","last_peered":"2023-11-08T19:28:01.690307+0000","last_clean":"2023-11-08T19:28:01.690307+0000","last_became_active":"2023-11-08T19:26:04.598775+0000","last_became_peered":"2023-11-08T19:26:04.598775+0000","last_unstale":"2023-11-08T19:28:01.690307+0000","last_undegraded":"2023-11-08T19:28:01.690307+0000","last_fullsized":"2023-11-08T19:28:01.690307+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:20:56.898927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049870399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.693887+0000","last_change":"2023-11-08T19:27:09.331646+0000","last_active":"2023-11-08T19:28:01.693887+0000","last_peered":"2023-11-08T19:28:01.693887+0000","last_clean":"2023-11-08T19:28:01.693887+0000","last_became_active":"2023-11-08T19:27:07.190619+0000","last_became_peered":"2023-11-08T19:27:07.190619+0000","last_unstale":"2023-11-08T19:28:01.693887+0000","last_undegraded":"2023-11-08T19:28:01.693887+0000","last_fullsized":"2023-11-08T19:28:01.693887+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:42:47.371458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065918400000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.702920+0000","last_change":"2023-11-08T19:28:02.702920+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.702920+0000","last_undegraded":"2023-11-08T19:28:02.702920+0000","last_fullsized":"2023-11-08T19:28:02.702920+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.719716+0000","last_change":"2023-11-08T19:28:02.710012+0000","last_active":"2023-11-08T19:28:02.719716+0000","last_peered":"2023-11-08T19:28:02.719716+0000","last_clean":"2023-11-08T19:28:02.719716+0000","last_became_active":"2023-11-08T19:28:02.709715+0000","last_became_peered":"2023-11-08T19:28:02.709715+0000","last_unstale":"2023-11-08T19:28:02.719716+0000","last_undegraded":"2023-11-08T19:28:02.719716+0000","last_fullsized":"2023-11-08T19:28:02.719716+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:23:11.877422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.703535+0000","last_change":"2023-11-08T19:27:08.183798+0000","last_active":"2023-11-08T19:28:02.703535+0000","last_peered":"2023-11-08T19:28:02.703535+0000","last_clean":"2023-11-08T19:28:02.703535+0000","last_became_active":"2023-11-08T19:27:08.183537+0000","last_became_peered":"2023-11-08T19:27:08.183537+0000","last_unstale":"2023-11-08T19:28:02.703535+0000","last_undegraded":"2023-11-08T19:28:02.703535+0000","last_fullsized":"2023-11-08T19:28:02.703535+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:50:39.026265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":71,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.701309+0000","last_change":"2023-11-08T19:26:05.610012+0000","last_active":"2023-11-08T19:28:02.701309+0000","last_peered":"2023-11-08T19:28:02.701309+0000","last_clean":"2023-11-08T19:28:02.701309+0000","last_became_active":"2023-11-08T19:26:04.603209+0000","last_became_peered":"2023-11-08T19:26:04.603209+0000","last_unstale":"2023-11-08T19:28:02.701309+0000","last_undegraded":"2023-11-08T19:28:02.701309+0000","last_fullsized":"2023-11-08T19:28:02.701309+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:35:02.306997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053335100000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.690186+0000","last_change":"2023-11-08T19:27:09.178405+0000","last_active":"2023-11-08T19:28:01.690186+0000","last_peered":"2023-11-08T19:28:01.690186+0000","last_clean":"2023-11-08T19:28:01.690186+0000","last_became_active":"2023-11-08T19:27:07.188169+0000","last_became_peered":"2023-11-08T19:27:07.188169+0000","last_unstale":"2023-11-08T19:28:01.690186+0000","last_undegraded":"2023-11-08T19:28:01.690186+0000","last_fullsized":"2023-11-08T19:28:01.690186+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:39:25.855467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056209700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.703242+0000","last_change":"2023-11-08T19:28:02.703242+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.703242+0000","last_undegraded":"2023-11-08T19:28:02.703242+0000","last_fullsized":"2023-11-08T19:28:02.703242+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.722598+0000","last_change":"2023-11-08T19:28:02.716988+0000","last_active":"2023-11-08T19:28:02.722598+0000","last_peered":"2023-11-08T19:28:02.722598+0000","last_clean":"2023-11-08T19:28:02.722598+0000","last_became_active":"2023-11-08T19:28:02.716687+0000","last_became_peered":"2023-11-08T19:28:02.716687+0000","last_unstale":"2023-11-08T19:28:02.722598+0000","last_undegraded":"2023-11-08T19:28:02.722598+0000","last_fullsized":"2023-11-08T19:28:02.722598+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:21:23.673336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.693769+0000","last_change":"2023-11-08T19:27:08.193508+0000","last_active":"2023-11-08T19:28:01.693769+0000","last_peered":"2023-11-08T19:28:01.693769+0000","last_clean":"2023-11-08T19:28:01.693769+0000","last_became_active":"2023-11-08T19:27:08.193037+0000","last_became_peered":"2023-11-08T19:27:08.193037+0000","last_unstale":"2023-11-08T19:28:01.693769+0000","last_undegraded":"2023-11-08T19:28:01.693769+0000","last_fullsized":"2023-11-08T19:28:01.693769+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:29:50.067887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":58,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.696488+0000","last_change":"2023-11-08T19:26:06.265318+0000","last_active":"2023-11-08T19:28:01.696488+0000","last_peered":"2023-11-08T19:28:01.696488+0000","last_clean":"2023-11-08T19:28:01.696488+0000","last_became_active":"2023-11-08T19:26:04.603409+0000","last_became_peered":"2023-11-08T19:26:04.603409+0000","last_unstale":"2023-11-08T19:28:01.696488+0000","last_undegraded":"2023-11-08T19:28:01.696488+0000","last_fullsized":"2023-11-08T19:28:01.696488+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:27:24.933111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065684300000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.701449+0000","last_change":"2023-11-08T19:27:09.182282+0000","last_active":"2023-11-08T19:28:02.701449+0000","last_peered":"2023-11-08T19:28:02.701449+0000","last_clean":"2023-11-08T19:28:02.701449+0000","last_became_active":"2023-11-08T19:27:07.186944+0000","last_became_peered":"2023-11-08T19:27:07.186944+0000","last_unstale":"2023-11-08T19:28:02.701449+0000","last_undegraded":"2023-11-08T19:28:02.701449+0000","last_fullsized":"2023-11-08T19:28:02.701449+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:45:21.079011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00028665599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.720726+0000","last_change":"2023-11-08T19:28:03.716823+0000","last_active":"2023-11-08T19:28:03.720726+0000","last_peered":"2023-11-08T19:28:03.720726+0000","last_clean":"2023-11-08T19:28:03.720726+0000","last_became_active":"2023-11-08T19:28:03.716545+0000","last_became_peered":"2023-11-08T19:28:03.716545+0000","last_unstale":"2023-11-08T19:28:03.720726+0000","last_undegraded":"2023-11-08T19:28:03.720726+0000","last_fullsized":"2023-11-08T19:28:03.720726+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:39:03.752939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.712391+0000","last_change":"2023-11-08T19:28:02.703978+0000","last_active":"2023-11-08T19:28:02.712391+0000","last_peered":"2023-11-08T19:28:02.712391+0000","last_clean":"2023-11-08T19:28:02.712391+0000","last_became_active":"2023-11-08T19:28:02.703654+0000","last_became_peered":"2023-11-08T19:28:02.703654+0000","last_unstale":"2023-11-08T19:28:02.712391+0000","last_undegraded":"2023-11-08T19:28:02.712391+0000","last_fullsized":"2023-11-08T19:28:02.712391+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:39:48.728129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.698064+0000","last_change":"2023-11-08T19:27:08.193990+0000","last_active":"2023-11-08T19:28:01.698064+0000","last_peered":"2023-11-08T19:28:01.698064+0000","last_clean":"2023-11-08T19:28:01.698064+0000","last_became_active":"2023-11-08T19:27:08.193624+0000","last_became_peered":"2023-11-08T19:27:08.193624+0000","last_unstale":"2023-11-08T19:28:01.698064+0000","last_undegraded":"2023-11-08T19:28:01.698064+0000","last_fullsized":"2023-11-08T19:28:01.698064+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:46:41.321120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":64,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.694552+0000","last_change":"2023-11-08T19:26:06.423863+0000","last_active":"2023-11-08T19:28:01.694552+0000","last_peered":"2023-11-08T19:28:01.694552+0000","last_clean":"2023-11-08T19:28:01.694552+0000","last_became_active":"2023-11-08T19:26:04.608639+0000","last_became_peered":"2023-11-08T19:26:04.608639+0000","last_unstale":"2023-11-08T19:28:01.694552+0000","last_undegraded":"2023-11-08T19:28:01.694552+0000","last_fullsized":"2023-11-08T19:28:01.694552+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:04:43.631102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044384999999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.698449+0000","last_change":"2023-11-08T19:27:09.283232+0000","last_active":"2023-11-08T19:28:01.698449+0000","last_peered":"2023-11-08T19:28:01.698449+0000","last_clean":"2023-11-08T19:28:01.698449+0000","last_became_active":"2023-11-08T19:27:07.190931+0000","last_became_peered":"2023-11-08T19:27:07.190931+0000","last_unstale":"2023-11-08T19:28:01.698449+0000","last_undegraded":"2023-11-08T19:28:01.698449+0000","last_fullsized":"2023-11-08T19:28:01.698449+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:43:28.225967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034146799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.713166+0000","last_change":"2023-11-08T19:28:02.713166+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.713166+0000","last_undegraded":"2023-11-08T19:28:02.713166+0000","last_fullsized":"2023-11-08T19:28:02.713166+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.722936+0000","last_change":"2023-11-08T19:28:02.717768+0000","last_active":"2023-11-08T19:28:02.722936+0000","last_peered":"2023-11-08T19:28:02.722936+0000","last_clean":"2023-11-08T19:28:02.722936+0000","last_became_active":"2023-11-08T19:28:02.717553+0000","last_became_peered":"2023-11-08T19:28:02.717553+0000","last_unstale":"2023-11-08T19:28:02.722936+0000","last_undegraded":"2023-11-08T19:28:02.722936+0000","last_fullsized":"2023-11-08T19:28:02.722936+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:50:10.975026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.694667+0000","last_change":"2023-11-08T19:27:08.187252+0000","last_active":"2023-11-08T19:28:01.694667+0000","last_peered":"2023-11-08T19:28:01.694667+0000","last_clean":"2023-11-08T19:28:01.694667+0000","last_became_active":"2023-11-08T19:27:08.186952+0000","last_became_peered":"2023-11-08T19:27:08.186952+0000","last_unstale":"2023-11-08T19:28:01.694667+0000","last_undegraded":"2023-11-08T19:28:01.694667+0000","last_fullsized":"2023-11-08T19:28:01.694667+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:12:52.022374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":58,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.694052+0000","last_change":"2023-11-08T19:26:06.394081+0000","last_active":"2023-11-08T19:28:01.694052+0000","last_peered":"2023-11-08T19:28:01.694052+0000","last_clean":"2023-11-08T19:28:01.694052+0000","last_became_active":"2023-11-08T19:26:04.602137+0000","last_became_peered":"2023-11-08T19:26:04.602137+0000","last_unstale":"2023-11-08T19:28:01.694052+0000","last_undegraded":"2023-11-08T19:28:01.694052+0000","last_fullsized":"2023-11-08T19:28:01.694052+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:05:46.951663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056998200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.690432+0000","last_change":"2023-11-08T19:27:09.178816+0000","last_active":"2023-11-08T19:28:01.690432+0000","last_peered":"2023-11-08T19:28:01.690432+0000","last_clean":"2023-11-08T19:28:01.690432+0000","last_became_active":"2023-11-08T19:27:07.192001+0000","last_became_peered":"2023-11-08T19:27:07.192001+0000","last_unstale":"2023-11-08T19:28:01.690432+0000","last_undegraded":"2023-11-08T19:28:01.690432+0000","last_fullsized":"2023-11-08T19:28:01.690432+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:04:12.481212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045359000000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.698299+0000","last_change":"2023-11-08T19:28:02.698299+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.698299+0000","last_undegraded":"2023-11-08T19:28:02.698299+0000","last_fullsized":"2023-11-08T19:28:02.698299+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.716969+0000","last_change":"2023-11-08T19:28:02.711841+0000","last_active":"2023-11-08T19:28:02.716969+0000","last_peered":"2023-11-08T19:28:02.716969+0000","last_clean":"2023-11-08T19:28:02.716969+0000","last_became_active":"2023-11-08T19:28:02.711455+0000","last_became_peered":"2023-11-08T19:28:02.711455+0000","last_unstale":"2023-11-08T19:28:02.716969+0000","last_undegraded":"2023-11-08T19:28:02.716969+0000","last_fullsized":"2023-11-08T19:28:02.716969+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:29:25.082011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.695522+0000","last_change":"2023-11-08T19:27:08.193836+0000","last_active":"2023-11-08T19:28:01.695522+0000","last_peered":"2023-11-08T19:28:01.695522+0000","last_clean":"2023-11-08T19:28:01.695522+0000","last_became_active":"2023-11-08T19:27:08.193458+0000","last_became_peered":"2023-11-08T19:27:08.193458+0000","last_unstale":"2023-11-08T19:28:01.695522+0000","last_undegraded":"2023-11-08T19:28:01.695522+0000","last_fullsized":"2023-11-08T19:28:01.695522+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:14:18.610061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":60,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.698115+0000","last_change":"2023-11-08T19:26:06.541338+0000","last_active":"2023-11-08T19:28:01.698115+0000","last_peered":"2023-11-08T19:28:01.698115+0000","last_clean":"2023-11-08T19:28:01.698115+0000","last_became_active":"2023-11-08T19:26:04.602183+0000","last_became_peered":"2023-11-08T19:26:04.602183+0000","last_unstale":"2023-11-08T19:28:01.698115+0000","last_undegraded":"2023-11-08T19:28:01.698115+0000","last_fullsized":"2023-11-08T19:28:01.698115+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:06:47.446686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052029099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.701560+0000","last_change":"2023-11-08T19:27:09.181622+0000","last_active":"2023-11-08T19:28:02.701560+0000","last_peered":"2023-11-08T19:28:02.701560+0000","last_clean":"2023-11-08T19:28:02.701560+0000","last_became_active":"2023-11-08T19:27:07.184507+0000","last_became_peered":"2023-11-08T19:27:07.184507+0000","last_unstale":"2023-11-08T19:28:02.701560+0000","last_undegraded":"2023-11-08T19:28:02.701560+0000","last_fullsized":"2023-11-08T19:28:02.701560+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:58:48.357266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051924700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.699401+0000","last_change":"2023-11-08T19:28:02.699401+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.699401+0000","last_undegraded":"2023-11-08T19:28:02.699401+0000","last_fullsized":"2023-11-08T19:28:02.699401+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.717064+0000","last_change":"2023-11-08T19:28:02.713407+0000","last_active":"2023-11-08T19:28:02.717064+0000","last_peered":"2023-11-08T19:28:02.717064+0000","last_clean":"2023-11-08T19:28:02.717064+0000","last_became_active":"2023-11-08T19:28:02.713144+0000","last_became_peered":"2023-11-08T19:28:02.713144+0000","last_unstale":"2023-11-08T19:28:02.717064+0000","last_undegraded":"2023-11-08T19:28:02.717064+0000","last_fullsized":"2023-11-08T19:28:02.717064+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:38:53.800926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.698458+0000","last_change":"2023-11-08T19:27:08.194263+0000","last_active":"2023-11-08T19:28:01.698458+0000","last_peered":"2023-11-08T19:28:01.698458+0000","last_clean":"2023-11-08T19:28:01.698458+0000","last_became_active":"2023-11-08T19:27:08.193938+0000","last_became_peered":"2023-11-08T19:27:08.193938+0000","last_unstale":"2023-11-08T19:28:01.698458+0000","last_undegraded":"2023-11-08T19:28:01.698458+0000","last_fullsized":"2023-11-08T19:28:01.698458+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:40:49.009717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":59,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.711234+0000","last_change":"2023-11-08T19:26:06.265690+0000","last_active":"2023-11-08T19:28:02.711234+0000","last_peered":"2023-11-08T19:28:02.711234+0000","last_clean":"2023-11-08T19:28:02.711234+0000","last_became_active":"2023-11-08T19:26:04.603397+0000","last_became_peered":"2023-11-08T19:26:04.603397+0000","last_unstale":"2023-11-08T19:28:02.711234+0000","last_undegraded":"2023-11-08T19:28:02.711234+0000","last_fullsized":"2023-11-08T19:28:02.711234+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:50:43.827570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053361499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.698140+0000","last_change":"2023-11-08T19:27:09.282750+0000","last_active":"2023-11-08T19:28:01.698140+0000","last_peered":"2023-11-08T19:28:01.698140+0000","last_clean":"2023-11-08T19:28:01.698140+0000","last_became_active":"2023-11-08T19:27:07.189943+0000","last_became_peered":"2023-11-08T19:27:07.189943+0000","last_unstale":"2023-11-08T19:28:01.698140+0000","last_undegraded":"2023-11-08T19:28:01.698140+0000","last_fullsized":"2023-11-08T19:28:01.698140+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:31:16.081058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000398725,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.702677+0000","last_change":"2023-11-08T19:28:02.702677+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.702677+0000","last_undegraded":"2023-11-08T19:28:02.702677+0000","last_fullsized":"2023-11-08T19:28:02.702677+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.724982+0000","last_change":"2023-11-08T19:28:02.715521+0000","last_active":"2023-11-08T19:28:02.724982+0000","last_peered":"2023-11-08T19:28:02.724982+0000","last_clean":"2023-11-08T19:28:02.724982+0000","last_became_active":"2023-11-08T19:28:02.715145+0000","last_became_peered":"2023-11-08T19:28:02.715145+0000","last_unstale":"2023-11-08T19:28:02.724982+0000","last_undegraded":"2023-11-08T19:28:02.724982+0000","last_fullsized":"2023-11-08T19:28:02.724982+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:52:23.978416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.702217+0000","last_change":"2023-11-08T19:27:08.188147+0000","last_active":"2023-11-08T19:28:02.702217+0000","last_peered":"2023-11-08T19:28:02.702217+0000","last_clean":"2023-11-08T19:28:02.702217+0000","last_became_active":"2023-11-08T19:27:08.187487+0000","last_became_peered":"2023-11-08T19:27:08.187487+0000","last_unstale":"2023-11-08T19:28:02.702217+0000","last_undegraded":"2023-11-08T19:28:02.702217+0000","last_fullsized":"2023-11-08T19:28:02.702217+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:22:40.300185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":64,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.696443+0000","last_change":"2023-11-08T19:26:06.425133+0000","last_active":"2023-11-08T19:28:01.696443+0000","last_peered":"2023-11-08T19:28:01.696443+0000","last_clean":"2023-11-08T19:28:01.696443+0000","last_became_active":"2023-11-08T19:26:04.601103+0000","last_became_peered":"2023-11-08T19:26:04.601103+0000","last_unstale":"2023-11-08T19:28:01.696443+0000","last_undegraded":"2023-11-08T19:28:01.696443+0000","last_fullsized":"2023-11-08T19:28:01.696443+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:31:14.042561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000229244,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.701647+0000","last_change":"2023-11-08T19:27:09.179852+0000","last_active":"2023-11-08T19:28:02.701647+0000","last_peered":"2023-11-08T19:28:02.701647+0000","last_clean":"2023-11-08T19:28:02.701647+0000","last_became_active":"2023-11-08T19:27:07.191926+0000","last_became_peered":"2023-11-08T19:27:07.191926+0000","last_unstale":"2023-11-08T19:28:02.701647+0000","last_undegraded":"2023-11-08T19:28:02.701647+0000","last_fullsized":"2023-11-08T19:28:02.701647+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:20:38.080129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055457100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.711863+0000","last_change":"2023-11-08T19:28:02.711863+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.711863+0000","last_undegraded":"2023-11-08T19:28:02.711863+0000","last_fullsized":"2023-11-08T19:28:02.711863+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"91'3","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.713504+0000","last_change":"2023-11-08T19:28:02.708691+0000","last_active":"2023-11-08T19:28:03.713504+0000","last_peered":"2023-11-08T19:28:03.713504+0000","last_clean":"2023-11-08T19:28:03.713504+0000","last_became_active":"2023-11-08T19:28:02.708037+0000","last_became_peered":"2023-11-08T19:28:02.708037+0000","last_unstale":"2023-11-08T19:28:03.713504+0000","last_undegraded":"2023-11-08T19:28:03.713504+0000","last_fullsized":"2023-11-08T19:28:03.713504+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:07:33.241500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.702633+0000","last_change":"2023-11-08T19:27:08.186054+0000","last_active":"2023-11-08T19:28:02.702633+0000","last_peered":"2023-11-08T19:28:02.702633+0000","last_clean":"2023-11-08T19:28:02.702633+0000","last_became_active":"2023-11-08T19:27:08.185581+0000","last_became_peered":"2023-11-08T19:27:08.185581+0000","last_unstale":"2023-11-08T19:28:02.702633+0000","last_undegraded":"2023-11-08T19:28:02.702633+0000","last_fullsized":"2023-11-08T19:28:02.702633+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:00:55.878397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":71,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.701211+0000","last_change":"2023-11-08T19:26:05.609625+0000","last_active":"2023-11-08T19:28:02.701211+0000","last_peered":"2023-11-08T19:28:02.701211+0000","last_clean":"2023-11-08T19:28:02.701211+0000","last_became_active":"2023-11-08T19:26:04.606312+0000","last_became_peered":"2023-11-08T19:26:04.606312+0000","last_unstale":"2023-11-08T19:28:02.701211+0000","last_undegraded":"2023-11-08T19:28:02.701211+0000","last_fullsized":"2023-11-08T19:28:02.701211+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:54:42.032901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000505935,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.690085+0000","last_change":"2023-11-08T19:27:09.177985+0000","last_active":"2023-11-08T19:28:01.690085+0000","last_peered":"2023-11-08T19:28:01.690085+0000","last_clean":"2023-11-08T19:28:01.690085+0000","last_became_active":"2023-11-08T19:27:07.193254+0000","last_became_peered":"2023-11-08T19:27:07.193254+0000","last_unstale":"2023-11-08T19:28:01.690085+0000","last_undegraded":"2023-11-08T19:28:01.690085+0000","last_fullsized":"2023-11-08T19:28:01.690085+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:23:38.662629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041830799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.703260+0000","last_change":"2023-11-08T19:28:02.703260+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.703260+0000","last_undegraded":"2023-11-08T19:28:02.703260+0000","last_fullsized":"2023-11-08T19:28:02.703260+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"91'6","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.727979+0000","last_change":"2023-11-08T19:28:02.700819+0000","last_active":"2023-11-08T19:28:03.727979+0000","last_peered":"2023-11-08T19:28:03.727979+0000","last_clean":"2023-11-08T19:28:03.727979+0000","last_became_active":"2023-11-08T19:28:02.700537+0000","last_became_peered":"2023-11-08T19:28:02.700537+0000","last_unstale":"2023-11-08T19:28:03.727979+0000","last_undegraded":"2023-11-08T19:28:03.727979+0000","last_fullsized":"2023-11-08T19:28:03.727979+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:25:00.088013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.703640+0000","last_change":"2023-11-08T19:27:08.188860+0000","last_active":"2023-11-08T19:28:02.703640+0000","last_peered":"2023-11-08T19:28:02.703640+0000","last_clean":"2023-11-08T19:28:02.703640+0000","last_became_active":"2023-11-08T19:27:08.188560+0000","last_became_peered":"2023-11-08T19:27:08.188560+0000","last_unstale":"2023-11-08T19:28:02.703640+0000","last_undegraded":"2023-11-08T19:28:02.703640+0000","last_fullsized":"2023-11-08T19:28:02.703640+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:18:20.969491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":59,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.695852+0000","last_change":"2023-11-08T19:26:06.297255+0000","last_active":"2023-11-08T19:28:01.695852+0000","last_peered":"2023-11-08T19:28:01.695852+0000","last_clean":"2023-11-08T19:28:01.695852+0000","last_became_active":"2023-11-08T19:26:04.603979+0000","last_became_peered":"2023-11-08T19:26:04.603979+0000","last_unstale":"2023-11-08T19:28:01.695852+0000","last_undegraded":"2023-11-08T19:28:01.695852+0000","last_fullsized":"2023-11-08T19:28:01.695852+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:40:11.164951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079032099999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.693851+0000","last_change":"2023-11-08T19:27:09.331990+0000","last_active":"2023-11-08T19:28:01.693851+0000","last_peered":"2023-11-08T19:28:01.693851+0000","last_clean":"2023-11-08T19:28:01.693851+0000","last_became_active":"2023-11-08T19:27:07.185478+0000","last_became_peered":"2023-11-08T19:27:07.185478+0000","last_unstale":"2023-11-08T19:28:01.693851+0000","last_undegraded":"2023-11-08T19:28:01.693851+0000","last_fullsized":"2023-11-08T19:28:01.693851+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:20:38.080129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010661850000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":1794,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12,"ondisk_log_size":12,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574892,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28704,"kb_used_data":1680,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343136,"statfs":{"total":96636764160,"available":96607371264,"internally_reserved":0,"allocated":1720320,"data_stored":1117537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:27:53 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.50600000000000001,"15min":0.498},"min":{"1min":0.34300000000000003,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.72899999999999998,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.437},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.51900000000000002,"15min":0.51500000000000001},"min":{"1min":0.36899999999999999,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.69099999999999995,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.621}]},{"osd":1,"last update":"Wed Nov 8 19:27:53 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56899999999999995,"15min":0.56599999999999995},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.56399999999999995,"15min":0.55600000000000005},"min":{"1min":0.435,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.59199999999999997}]},{"osd":2,"last update":"Wed Nov 8 19:27:53 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.50900000000000001,"15min":0.49299999999999999},"min":{"1min":0.38700000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.55000000000000004,"15min":0.54800000000000004},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.38500000000000001}]},{"osd":3,"last update":"Wed Nov 8 19:27:53 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.55300000000000005,"15min":0.55700000000000005},"min":{"1min":0.375,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.66200000000000003,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59399999999999997,"15min":0.57799999999999996},"min":{"1min":0.49399999999999999,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.77000000000000002,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.5}]},{"osd":4,"last update":"Wed Nov 8 19:27:53 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.50900000000000001,"15min":0.50700000000000001},"min":{"1min":0.36299999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.64500000000000002,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.54600000000000004,"15min":0.54100000000000004},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.41099999999999998}]},{"osd":5,"last update":"Wed Nov 8 19:27:53 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.58599999999999997,"15min":0.58099999999999996},"min":{"1min":0.41199999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.71999999999999997,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.62,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.84499999999999997,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.64300000000000002}]},{"osd":6,"last update":"Wed Nov 8 19:27:53 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.60199999999999998,"15min":0.60399999999999998},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.78400000000000003,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.61499999999999999,"15min":0.61099999999999999},"min":{"1min":0.5,"5min":0.443,"15min":0.443},"max":{"1min":0.82999999999999996,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.57299999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574892,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28132,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343708,"statfs":{"total":96636764160,"available":96607956992,"internally_reserved":0,"allocated":1138688,"data_stored":529225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":1.0780000000000001,"15min":1.319},"min":{"1min":0.33800000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.67800000000000005,"5min":21.359999999999999,"15min":21.359999999999999},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.78300000000000003,"15min":0.88700000000000001},"min":{"1min":0.378,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.73799999999999999,"5min":10.329000000000001,"15min":10.329000000000001},"last":0.434}]},{"osd":1,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.57899999999999996,"15min":0.60899999999999999},"min":{"1min":0.36899999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.71899999999999997,"5min":3.4239999999999999,"15min":3.4239999999999999},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.61699999999999999,"15min":0.63800000000000001},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.73199999999999998,"5min":3.4809999999999999,"15min":3.4809999999999999},"last":0.45800000000000002}]},{"osd":2,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.80100000000000005,"15min":0.90500000000000003},"min":{"1min":0.40999999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.69299999999999995,"5min":10.398,"15min":10.398},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.65400000000000003,"15min":0.68999999999999995},"min":{"1min":0.40799999999999997,"5min":0.318,"15min":0.318},"max":{"1min":0.746,"5min":5.0170000000000003,"15min":5.0170000000000003},"last":0.39300000000000002}]},{"osd":3,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.63900000000000001,"15min":0.66300000000000003},"min":{"1min":0.39500000000000002,"5min":0.317,"15min":0.317},"max":{"1min":0.77300000000000002,"5min":3.5710000000000002,"15min":3.5710000000000002},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.82999999999999996,"15min":0.93799999999999994},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.86799999999999999,"5min":10.368,"15min":10.368},"last":0.56299999999999994}]},{"osd":4,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.53700000000000003,"15min":0.51700000000000002},"min":{"1min":0.38,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.56999999999999995,"15min":0.56200000000000006},"min":{"1min":0.40600000000000003,"5min":0.373,"15min":0.373},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.53300000000000003}]},{"osd":5,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.55000000000000004,"15min":0.53200000000000003},"min":{"1min":0.40699999999999997,"5min":0.307,"15min":0.307},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.503},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.60599999999999998,"15min":0.58599999999999997},"min":{"1min":0.42899999999999999,"5min":0.378,"15min":0.378},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.55200000000000005}]},{"osd":7,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.59899999999999998,"15min":0.57899999999999996},"min":{"1min":0.41299999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.81200000000000006,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.61099999999999999,"15min":0.59399999999999997},"min":{"1min":0.54000000000000004,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.47499999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574892,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28104,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343736,"statfs":{"total":96636764160,"available":96607985664,"internally_reserved":0,"allocated":1105920,"data_stored":519171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:27:47 2023","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.36899999999999999,"15min":0.379},"min":{"1min":0.22800000000000001,"5min":0.219,"15min":0.219},"max":{"1min":0.52400000000000002,"5min":1.389,"15min":1.389},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.33800000000000002,"5min":0.58299999999999996,"15min":0.68999999999999995},"min":{"1min":0.25900000000000001,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.504,"5min":9.1370000000000005,"15min":9.1370000000000005},"last":0.36499999999999999}]},{"osd":2,"last update":"Wed Nov 8 19:27:47 2023","interfaces":[{"interface":"back","average":{"1min":0.33600000000000002,"5min":0.35399999999999998,"15min":0.36299999999999999},"min":{"1min":0.254,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.50600000000000001,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.373,"5min":0.41099999999999998,"15min":0.42499999999999999},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.58299999999999996,"5min":1.425,"15min":1.425},"last":0.47899999999999998}]},{"osd":3,"last update":"Wed Nov 8 19:27:47 2023","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.69999999999999996,"15min":0.84099999999999997},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.54600000000000004,"5min":11.957000000000001,"15min":11.957000000000001},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.85799999999999998,"15min":1.0740000000000001},"min":{"1min":0.27700000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.56299999999999994,"5min":18.039000000000001,"15min":18.039000000000001},"last":0.437}]},{"osd":4,"last update":"Wed Nov 8 19:27:47 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.73399999999999999,"15min":0.79800000000000004},"min":{"1min":0.43099999999999999,"5min":0.372,"15min":0.372},"max":{"1min":0.77800000000000002,"5min":6.181,"15min":6.181},"last":0.94699999999999995},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.629,"15min":0.63700000000000001},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.76900000000000002,"5min":1.488,"15min":1.488},"last":0.996}]},{"osd":5,"last update":"Wed Nov 8 19:27:47 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.84099999999999997,"15min":0.93700000000000006},"min":{"1min":0.437,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.74199999999999999,"5min":9.2420000000000009,"15min":9.2420000000000009},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.77200000000000002,"15min":0.84399999999999997},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.745,"5min":6.2110000000000003,"15min":6.2110000000000003},"last":0.878}]},{"osd":6,"last update":"Wed Nov 8 19:27:47 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.84899999999999998,"15min":0.94099999999999995},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.751,"5min":9.1880000000000006,"15min":9.1880000000000006},"last":0.755},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.65500000000000003,"15min":0.66300000000000003},"min":{"1min":0.38800000000000001,"5min":0.38,"15min":0.38},"max":{"1min":0.79400000000000004,"5min":1.462,"15min":1.462},"last":0.85799999999999998}]},{"osd":7,"last update":"Wed Nov 8 19:27:47 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.77400000000000002,"15min":0.83799999999999997},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.76600000000000001,"5min":6.1479999999999997,"15min":6.1479999999999997},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.872,"15min":0.97599999999999998},"min":{"1min":0.46100000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.76400000000000001,"5min":9.2170000000000005,"15min":9.2170000000000005},"last":0.90600000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574892,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28680,"kb_used_data":1660,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343160,"statfs":{"total":96636764160,"available":96607395840,"internally_reserved":0,"allocated":1699840,"data_stored":1109539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Nov 8 19:27:50 2023","interfaces":[{"interface":"back","average":{"1min":0.32100000000000001,"5min":0.441,"15min":0.495},"min":{"1min":0.23599999999999999,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.48799999999999999,"5min":4.1200000000000001,"15min":4.1200000000000001},"last":0.247},{"interface":"front","average":{"1min":0.32300000000000001,"5min":0.85199999999999998,"15min":1.079},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.51200000000000001,"5min":16.940999999999999,"15min":16.940999999999999},"last":0.34000000000000002}]},{"osd":2,"last update":"Wed Nov 8 19:27:50 2023","interfaces":[{"interface":"back","average":{"1min":0.318,"5min":0.374,"15min":0.39400000000000002},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.45600000000000002,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.57499999999999996,"15min":0.66600000000000004},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.58299999999999996,"5min":7.0209999999999999,"15min":7.0209999999999999},"last":0.28699999999999998}]},{"osd":3,"last update":"Wed Nov 8 19:27:50 2023","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.49299999999999999,"15min":0.54800000000000004},"min":{"1min":0.25,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.56699999999999995,"5min":4.0940000000000003,"15min":4.0940000000000003},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.37,"5min":0.42299999999999999,"15min":0.442},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.53600000000000003,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.36099999999999999}]},{"osd":4,"last update":"Wed Nov 8 19:27:50 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.69299999999999995,"15min":0.751},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.76800000000000002,"5min":4.0049999999999999,"15min":4.0049999999999999},"last":0.375},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.62,"15min":0.63500000000000001},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.77500000000000002,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.40000000000000002}]},{"osd":5,"last update":"Wed Nov 8 19:27:50 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.71199999999999997,"15min":0.76600000000000001},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.79200000000000004,"5min":4.3090000000000002,"15min":4.3090000000000002},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.70899999999999996,"15min":0.75700000000000001},"min":{"1min":0.45600000000000002,"5min":0.437,"15min":0.437},"max":{"1min":0.84699999999999998,"5min":4.0709999999999997,"15min":4.0709999999999997},"last":0.441}]},{"osd":6,"last update":"Wed Nov 8 19:27:50 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.65500000000000003,"15min":0.66900000000000004},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.81200000000000006,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.76800000000000002,"15min":0.82399999999999995},"min":{"1min":0.47699999999999998,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.84199999999999997,"5min":4.2560000000000002,"15min":4.2560000000000002},"last":0.56200000000000006}]},{"osd":7,"last update":"Wed Nov 8 19:27:50 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.72699999999999998,"15min":0.77700000000000002},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.78700000000000003,"5min":4.2850000000000001,"15min":4.2850000000000001},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.753,"15min":0.80500000000000005},"min":{"1min":0.48699999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.82499999999999996,"5min":4.0460000000000003,"15min":4.0460000000000003},"last":0.41399999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574892,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28088,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343752,"statfs":{"total":96636764160,"available":96608002048,"internally_reserved":0,"allocated":1093632,"data_stored":511833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:27:47 2023","interfaces":[{"interface":"back","average":{"1min":0.32100000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.47299999999999998,"5min":0.496,"15min":0.496},"last":0.28499999999999998},{"interface":"front","average":{"1min":0.31,"5min":0.51900000000000002,"15min":0.61299999999999999},"min":{"1min":0.219,"5min":0.218,"15min":0.218},"max":{"1min":0.501,"5min":6.0999999999999996,"15min":6.0999999999999996},"last":0.23000000000000001}]},{"osd":1,"last update":"Wed Nov 8 19:27:47 2023","interfaces":[{"interface":"back","average":{"1min":0.309,"5min":0.33400000000000002,"15min":0.33800000000000002},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.42699999999999999,"5min":0.48299999999999998,"15min":0.48299999999999998},"last":0.26300000000000001},{"interface":"front","average":{"1min":0.32500000000000001,"5min":0.372,"15min":0.38400000000000001},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.41899999999999998,"5min":0.624,"15min":0.624},"last":0.32600000000000001}]},{"osd":3,"last update":"Wed Nov 8 19:27:47 2023","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.376,"15min":0.38200000000000001},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.46500000000000002,"5min":0.53700000000000003,"15min":0.53700000000000003},"last":0.375},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.38300000000000001,"15min":0.39200000000000002},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.53400000000000003,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.34899999999999998}]},{"osd":4,"last update":"Wed Nov 8 19:27:47 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.56699999999999995,"15min":0.57899999999999996},"min":{"1min":0.39000000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.57699999999999996,"15min":0.58199999999999996},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.77100000000000002,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.5}]},{"osd":5,"last update":"Wed Nov 8 19:27:47 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.59199999999999997,"15min":0.58499999999999996},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.78100000000000003,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.628},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.60499999999999998,"15min":0.61099999999999999},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.76400000000000001,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.57199999999999995}]},{"osd":6,"last update":"Wed Nov 8 19:27:47 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59799999999999998,"15min":0.59999999999999998},"min":{"1min":0.47199999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.79700000000000004,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57799999999999996,"15min":0.57899999999999996},"min":{"1min":0.41499999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.80000000000000004,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.58899999999999997}]},{"osd":7,"last update":"Wed Nov 8 19:27:47 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.58199999999999996,"15min":0.57999999999999996},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.73199999999999998,"5min":0.878,"15min":0.878},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60099999999999998,"15min":0.60299999999999998},"min":{"1min":0.45300000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.82199999999999995,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.65700000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574893,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28104,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343736,"statfs":{"total":96636764160,"available":96607985664,"internally_reserved":0,"allocated":1105920,"data_stored":519171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.318,"5min":0.33300000000000002,"15min":0.33900000000000002},"min":{"1min":0.224,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.49099999999999999,"5min":0.626,"15min":0.626},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.34999999999999998,"15min":0.34899999999999998},"min":{"1min":0.28000000000000003,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.48699999999999999,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.26200000000000001}]},{"osd":1,"last update":"Wed Nov 8 19:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.34699999999999998,"15min":0.35199999999999998},"min":{"1min":0.25800000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.42299999999999999,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":0.29599999999999999},{"interface":"front","average":{"1min":0.38,"5min":0.38300000000000001,"15min":0.38400000000000001},"min":{"1min":0.28000000000000003,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.48699999999999999,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.27900000000000003}]},{"osd":2,"last update":"Wed Nov 8 19:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.376,"15min":0.372},"min":{"1min":0.32900000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.46300000000000002,"5min":0.58499999999999996,"15min":0.58499999999999996},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.40400000000000003,"15min":0.40400000000000003},"min":{"1min":0.32700000000000001,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.47399999999999998,"5min":0.58499999999999996,"15min":0.58499999999999996},"last":0.32000000000000001}]},{"osd":4,"last update":"Wed Nov 8 19:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.60799999999999998,"15min":0.60099999999999998},"min":{"1min":0.45300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.624,"5min":0.61199999999999999,"15min":0.60699999999999998},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.84499999999999997,"5min":0.871,"15min":0.871},"last":0.56100000000000005}]},{"osd":5,"last update":"Wed Nov 8 19:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.58699999999999997,"15min":0.57899999999999996},"min":{"1min":0.46700000000000003,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.76600000000000001,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.58899999999999997,"15min":0.58099999999999996},"min":{"1min":0.42399999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.78600000000000003,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.54700000000000004}]},{"osd":6,"last update":"Wed Nov 8 19:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.58799999999999997,"15min":0.58199999999999996},"min":{"1min":0.436,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.73299999999999998,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.61299999999999999,"15min":0.60699999999999998},"min":{"1min":0.49099999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.75900000000000001,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.57399999999999995}]},{"osd":7,"last update":"Wed Nov 8 19:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.59899999999999998,"15min":0.59099999999999997},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.72199999999999998,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.60199999999999998,"15min":0.59799999999999998},"min":{"1min":0.45800000000000002,"5min":0.432,"15min":0.432},"max":{"1min":0.80700000000000005,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.58299999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574892,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28104,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343736,"statfs":{"total":96636764160,"available":96607985664,"internally_reserved":0,"allocated":1105920,"data_stored":519171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:27:55 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.68200000000000005,"15min":0.74399999999999999},"min":{"1min":0.35699999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.72099999999999997,"5min":6.5970000000000004,"15min":6.5970000000000004},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.55500000000000005,"15min":0.56999999999999995},"min":{"1min":0.35799999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.70999999999999996,"5min":2.5030000000000001,"15min":2.5030000000000001},"last":0.46899999999999997}]},{"osd":1,"last update":"Wed Nov 8 19:27:55 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.51300000000000001,"15min":0.501},"min":{"1min":0.36199999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.75800000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.629},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.52100000000000002,"15min":0.51200000000000001},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.70199999999999996}]},{"osd":2,"last update":"Wed Nov 8 19:27:55 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.55400000000000005,"15min":0.55200000000000005},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.80600000000000005,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.748,"15min":0.81299999999999994},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.86199999999999999,"5min":6.6920000000000002,"15min":6.6920000000000002},"last":0.66900000000000004}]},{"osd":3,"last update":"Wed Nov 8 19:27:55 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.58299999999999996,"15min":0.58699999999999997},"min":{"1min":0.378,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.79800000000000004,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.60899999999999999,"15min":0.60999999999999999},"min":{"1min":0.42499999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.71399999999999997,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.68500000000000005}]},{"osd":5,"last update":"Wed Nov 8 19:27:55 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.55000000000000004,"15min":0.54800000000000004},"min":{"1min":0.35099999999999998,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.81599999999999995,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.58499999999999996,"15min":0.59299999999999997},"min":{"1min":0.39400000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.745,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.53000000000000003}]},{"osd":6,"last update":"Wed Nov 8 19:27:55 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58599999999999997,"15min":0.58499999999999996},"min":{"1min":0.373,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.627,"15min":0.61799999999999999},"min":{"1min":0.46400000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.73099999999999998}]},{"osd":7,"last update":"Wed Nov 8 19:27:55 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.61599999999999999,"15min":0.629},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.84499999999999997,"5min":1.141,"15min":1.141},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.64100000000000001,"15min":0.64600000000000002},"min":{"1min":0.42399999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.86399999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.74199999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574892,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28104,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343736,"statfs":{"total":96636764160,"available":96607985664,"internally_reserved":0,"allocated":1105920,"data_stored":519171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:27:46 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.51400000000000001,"15min":0.52000000000000002},"min":{"1min":0.32200000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.73999999999999999,"5min":0.753,"15min":0.753},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.51900000000000002,"15min":0.51200000000000001},"min":{"1min":0.38500000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.69399999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.495}]},{"osd":1,"last update":"Wed Nov 8 19:27:46 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.51900000000000002,"15min":0.52200000000000002},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.70799999999999996,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.53400000000000003,"15min":0.53600000000000003},"min":{"1min":0.38400000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.70499999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.56899999999999995}]},{"osd":2,"last update":"Wed Nov 8 19:27:46 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.53600000000000003,"15min":0.53500000000000003},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.68000000000000005,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.55800000000000005,"15min":0.56999999999999995},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.69799999999999995,"5min":0.747,"15min":0.747},"last":0.47699999999999998}]},{"osd":3,"last update":"Wed Nov 8 19:27:46 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.54700000000000004,"15min":0.54900000000000004},"min":{"1min":0.40899999999999997,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.72199999999999998,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.57899999999999996,"15min":0.59599999999999997},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.69099999999999995,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.627}]},{"osd":4,"last update":"Wed Nov 8 19:27:46 2023","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.53200000000000003,"15min":0.55200000000000005},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.67200000000000004,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.56200000000000006,"15min":0.56899999999999995},"min":{"1min":0.36599999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.37}]},{"osd":6,"last update":"Wed Nov 8 19:27:46 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.59399999999999997,"15min":0.60699999999999998},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.755,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.59799999999999998,"15min":0.61099999999999999},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.72099999999999997,"5min":0.745,"15min":0.745},"last":0.60999999999999999}]},{"osd":7,"last update":"Wed Nov 8 19:27:46 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.59399999999999997,"15min":0.60599999999999998},"min":{"1min":0.44500000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.74199999999999999,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.625,"15min":0.64200000000000002},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.752,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.59399999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-08T19:28:08.228 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-08T19:28:08.377 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.375+0000 7f48309cc700 1 Processor -- start 2023-11-08T19:28:08.377 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.375+0000 7f48309cc700 1 -- start start 2023-11-08T19:28:08.381 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.377+0000 7f48309cc700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f482c0cc530 0x7f482c0cf510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:08.381 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.377+0000 7f48309cc700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f482c0cf080 con 0x7f482c15aa30 2023-11-08T19:28:08.381 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.377+0000 7f48309cc700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f482c0ce470 con 0x7f482c0b2920 2023-11-08T19:28:08.381 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.377+0000 7f48309cc700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f482c0ce750 con 0x7f482c0cc530 2023-11-08T19:28:08.381 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.377+0000 7f482a59c700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f482c15aa30 0x7f482c0b2210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:50804/0 (socket says 172.21.15.143:50804) 2023-11-08T19:28:08.382 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.377+0000 7f482a59c700 1 -- 172.21.15.143:0/3004681144 learned_addr learned my addr 172.21.15.143:0/3004681144 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:28:08.382 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.377+0000 7f482ad9d700 1 --2- 172.21.15.143:0/3004681144 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f482c0cc530 0x7f482c0cf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:08.382 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.377+0000 7f482959a700 1 -- 172.21.15.143:0/3004681144 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1513666441 0 0) 0x7f482c0cf080 con 0x7f482c15aa30 2023-11-08T19:28:08.383 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.377+0000 7f482959a700 1 -- 172.21.15.143:0/3004681144 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48140035b0 con 0x7f482c15aa30 2023-11-08T19:28:08.383 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.377+0000 7f482959a700 1 -- 172.21.15.143:0/3004681144 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 505973610 0 0) 0x7f48140035b0 con 0x7f482c15aa30 2023-11-08T19:28:08.383 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.377+0000 7f482959a700 1 -- 172.21.15.143:0/3004681144 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f482c0cc530 msgr2=0x7f482c0cf510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:08.383 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.377+0000 7f482959a700 1 --2- 172.21.15.143:0/3004681144 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f482c0cc530 0x7f482c0cf510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:08.383 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.377+0000 7f482959a700 1 -- 172.21.15.143:0/3004681144 >> v1:172.21.15.174:6789/0 conn(0x7f482c0b2920 legacy=0x7f482c0cbe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:08.384 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.377+0000 7f482959a700 1 -- 172.21.15.143:0/3004681144 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f482c0cea30 con 0x7f482c15aa30 2023-11-08T19:28:08.384 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.377+0000 7f482959a700 1 -- 172.21.15.143:0/3004681144 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4147756303 0 0) 0x7f48200017f0 con 0x7f482c15aa30 2023-11-08T19:28:08.384 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.377+0000 7f482959a700 1 -- 172.21.15.143:0/3004681144 >> v1:172.21.15.143:6789/0 conn(0x7f482c15aa30 legacy=0x7f482c0b2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:08.384 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.377+0000 7f482959a700 1 --2- 172.21.15.143:0/3004681144 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4814003a30 0x7f4814003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:08.384 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.377+0000 7f482959a700 1 --2- 172.21.15.143:0/3004681144 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f48140043e0 0x7f48140048b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:08.385 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.377+0000 7f482959a700 1 --2- 172.21.15.143:0/3004681144 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f482c0cc530 0x7f4814005110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:08.385 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.377+0000 7f482959a700 1 -- 172.21.15.143:0/3004681144 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f482c0ce750 con 0x7f4814003a30 2023-11-08T19:28:08.385 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.377+0000 7f482959a700 1 -- 172.21.15.143:0/3004681144 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f481400ba10 con 0x7f48140043e0 2023-11-08T19:28:08.385 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.377+0000 7f482959a700 1 -- 172.21.15.143:0/3004681144 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f481400dde0 con 0x7f482c0cc530 2023-11-08T19:28:08.385 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.377+0000 7f482a59c700 1 --2- 172.21.15.143:0/3004681144 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4814003a30 0x7f4814003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:08.386 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.378+0000 7f482ad9d700 1 --2- 172.21.15.143:0/3004681144 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f482c0cc530 0x7f4814005110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:08.386 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.378+0000 7f4829d9b700 1 --2- 172.21.15.143:0/3004681144 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f48140043e0 0x7f48140048b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:08.386 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.378+0000 7f482a59c700 1 -- 172.21.15.143:0/3004681144 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f482c0cc530 msgr2=0x7f4814005110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:08.386 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.378+0000 7f482a59c700 1 --2- 172.21.15.143:0/3004681144 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f482c0cc530 0x7f4814005110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:08.386 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.378+0000 7f482a59c700 1 -- 172.21.15.143:0/3004681144 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f48140043e0 msgr2=0x7f48140048b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:08.387 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.378+0000 7f482a59c700 1 --2- 172.21.15.143:0/3004681144 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f48140043e0 0x7f48140048b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:08.387 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.378+0000 7f482a59c700 1 -- 172.21.15.143:0/3004681144 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48140035b0 con 0x7f4814003a30 2023-11-08T19:28:08.387 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.378+0000 7f482a59c700 1 --2- 172.21.15.143:0/3004681144 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4814003a30 0x7f4814003e10 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f482000c240 tx=0x7f4814005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6989d05c07af2c2 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:08.387 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.378+0000 7f482959a700 1 -- 172.21.15.143:0/3004681144 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f482000cde0 con 0x7f4814003a30 2023-11-08T19:28:08.387 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.378+0000 7f482959a700 1 -- 172.21.15.143:0/3004681144 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4820007ba0 con 0x7f4814003a30 2023-11-08T19:28:08.388 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.378+0000 7f482959a700 1 -- 172.21.15.143:0/3004681144 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48200072f0 con 0x7f4814003a30 2023-11-08T19:28:08.388 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.378+0000 7f48309cc700 1 -- 172.21.15.143:0/3004681144 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4814003a30 msgr2=0x7f4814003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:08.388 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.378+0000 7f48309cc700 1 --2- 172.21.15.143:0/3004681144 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4814003a30 0x7f4814003e10 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f482000c240 tx=0x7f4814005880 comp rx=0 tx=0).stop 2023-11-08T19:28:08.388 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.378+0000 7f482ad9d700 1 -- 172.21.15.143:0/3004681144 reap_dead start 2023-11-08T19:28:08.388 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.379+0000 7f48309cc700 1 -- 172.21.15.143:0/3004681144 shutdown_connections 2023-11-08T19:28:08.389 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.379+0000 7f48309cc700 1 -- 172.21.15.143:0/3004681144 >> 172.21.15.143:0/3004681144 conn(0x7f482c099ab0 msgr2=0x7f482c0d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:08.389 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.379+0000 7f48309cc700 1 -- 172.21.15.143:0/3004681144 shutdown_connections 2023-11-08T19:28:08.389 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.379+0000 7f48309cc700 1 -- 172.21.15.143:0/3004681144 wait complete. 2023-11-08T19:28:08.392 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.380+0000 7f48309cc700 1 Processor -- start 2023-11-08T19:28:08.392 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.380+0000 7f48309cc700 1 -- start start 2023-11-08T19:28:08.392 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.381+0000 7f48309cc700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4814003a30 0x7f482c15c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:08.393 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.381+0000 7f48309cc700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f482c1587d0 0x7f482c0b1c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:08.393 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.381+0000 7f48309cc700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f482c0b21d0 0x7f482c0b3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:08.393 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.381+0000 7f48309cc700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f482c1f73b0 con 0x7f482c0b21d0 2023-11-08T19:28:08.393 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.381+0000 7f48309cc700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f482c1f7690 con 0x7f482c1587d0 2023-11-08T19:28:08.393 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.381+0000 7f48309cc700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f482c1f7970 con 0x7f4814003a30 2023-11-08T19:28:08.394 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.381+0000 7f482ad9d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f482c0b21d0 0x7f482c0b3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:08.394 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.381+0000 7f482ad9d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f482c0b21d0 0x7f482c0b3cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54666/0 (socket says 172.21.15.143:54666) 2023-11-08T19:28:08.394 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.381+0000 7f482ad9d700 1 -- 172.21.15.143:0/2288452733 learned_addr learned my addr 172.21.15.143:0/2288452733 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:28:08.394 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.381+0000 7f4829d9b700 1 --2- 172.21.15.143:0/2288452733 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f482c1587d0 0x7f482c0b1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:08.394 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.381+0000 7f482a59c700 1 --2- 172.21.15.143:0/2288452733 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4814003a30 0x7f482c15c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:08.395 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.381+0000 7f482ad9d700 1 -- 172.21.15.143:0/2288452733 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4814003a30 msgr2=0x7f482c15c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:08.395 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.381+0000 7f482ad9d700 1 --2- 172.21.15.143:0/2288452733 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4814003a30 0x7f482c15c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:08.395 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.381+0000 7f482ad9d700 1 -- 172.21.15.143:0/2288452733 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f482c1587d0 msgr2=0x7f482c0b1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:08.395 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.381+0000 7f482ad9d700 1 --2- 172.21.15.143:0/2288452733 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f482c1587d0 0x7f482c0b1c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:08.395 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.381+0000 7f482ad9d700 1 -- 172.21.15.143:0/2288452733 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f482c1f7c50 con 0x7f482c0b21d0 2023-11-08T19:28:08.396 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.382+0000 7f482ad9d700 1 --2- 172.21.15.143:0/2288452733 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f482c0b21d0 0x7f482c0b3cf0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f482c072e70 tx=0x7f4824011f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:08.396 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.382+0000 7f481affd700 1 -- 172.21.15.143:0/2288452733 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f482400e040 con 0x7f482c0b21d0 2023-11-08T19:28:08.396 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.382+0000 7f481affd700 1 -- 172.21.15.143:0/2288452733 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f482400f040 con 0x7f482c0b21d0 2023-11-08T19:28:08.396 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.382+0000 7f48309cc700 1 -- 172.21.15.143:0/2288452733 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f482c1f7f30 con 0x7f482c0b21d0 2023-11-08T19:28:08.396 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.382+0000 7f481affd700 1 -- 172.21.15.143:0/2288452733 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f482400fe40 con 0x7f482c0b21d0 2023-11-08T19:28:08.397 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.382+0000 7f48309cc700 1 -- 172.21.15.143:0/2288452733 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f482c1f7040 con 0x7f482c0b21d0 2023-11-08T19:28:08.397 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.383+0000 7f48309cc700 1 -- 172.21.15.143:0/2288452733 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f482c04eee0 con 0x7f482c0b21d0 2023-11-08T19:28:08.397 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.383+0000 7f481affd700 1 -- 172.21.15.143:0/2288452733 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4824022460 con 0x7f482c0b21d0 2023-11-08T19:28:08.397 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.383+0000 7f481affd700 1 --2- 172.21.15.143:0/2288452733 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f480c06af00 0x7f480c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:08.397 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.384+0000 7f481affd700 1 -- 172.21.15.143:0/2288452733 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6772+0+0 (secure 0 0 0) 0x7f4824089e40 con 0x7f482c0b21d0 2023-11-08T19:28:08.397 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.384+0000 7f482a59c700 1 --2- 172.21.15.143:0/2288452733 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f480c06af00 0x7f480c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:08.398 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.384+0000 7f482a59c700 1 --2- 172.21.15.143:0/2288452733 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f480c06af00 0x7f480c06d3c0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f4820007d50 tx=0x7f4820007000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:08.398 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.388+0000 7f481affd700 1 -- 172.21.15.143:0/2288452733 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4824054ac0 con 0x7f482c0b21d0 2023-11-08T19:28:08.572 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.571+0000 7f48309cc700 1 -- 172.21.15.143:0/2288452733 --> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f482c0ced10 con 0x7f480c06af00 2023-11-08T19:28:08.587 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.586+0000 7f481affd700 1 -- 172.21.15.143:0/2288452733 <== mgr.4099 v2:172.21.15.174:6832/4037280178 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+528138 (secure 0 0 0) 0x7f4820023070 con 0x7f480c06af00 2023-11-08T19:28:08.593 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:28:08.598 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.590+0000 7f48309cc700 1 -- 172.21.15.143:0/2288452733 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f480c06af00 msgr2=0x7f480c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:08.598 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.590+0000 7f48309cc700 1 --2- 172.21.15.143:0/2288452733 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f480c06af00 0x7f480c06d3c0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f4820007d50 tx=0x7f4820007000 comp rx=0 tx=0).stop 2023-11-08T19:28:08.599 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.590+0000 7f48309cc700 1 -- 172.21.15.143:0/2288452733 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f482c0b21d0 msgr2=0x7f482c0b3cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:08.599 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.590+0000 7f48309cc700 1 --2- 172.21.15.143:0/2288452733 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f482c0b21d0 0x7f482c0b3cf0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f482c072e70 tx=0x7f4824011f60 comp rx=0 tx=0).stop 2023-11-08T19:28:08.599 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.590+0000 7f48309cc700 1 -- 172.21.15.143:0/2288452733 shutdown_connections 2023-11-08T19:28:08.599 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.590+0000 7f48309cc700 1 --2- 172.21.15.143:0/2288452733 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f480c06af00 0x7f480c06d3c0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:08.599 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.590+0000 7f48309cc700 1 --2- 172.21.15.143:0/2288452733 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4814003a30 0x7f482c15c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:08.600 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.590+0000 7f48309cc700 1 --2- 172.21.15.143:0/2288452733 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f482c0b21d0 0x7f482c0b3cf0 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:08.600 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.590+0000 7f48309cc700 1 --2- 172.21.15.143:0/2288452733 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f482c1587d0 0x7f482c0b1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:08.600 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.590+0000 7f48309cc700 1 -- 172.21.15.143:0/2288452733 >> 172.21.15.143:0/2288452733 conn(0x7f482c099ab0 msgr2=0x7f482c0b01f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:08.600 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.590+0000 7f48309cc700 1 -- 172.21.15.143:0/2288452733 shutdown_connections 2023-11-08T19:28:08.600 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.590+0000 7f48309cc700 1 -- 172.21.15.143:0/2288452733 wait complete. 2023-11-08T19:28:08.601 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2023-11-08T19:28:08.604 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":187,"stamp":"2023-11-08T19:28:08.562651+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592162,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":171,"num_write_kb":2422,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":129,"ondisk_log_size":129,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":236,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":226020,"kb_used_data":9840,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754748700,"statfs":{"total":773094113280,"available":772862668800,"internally_reserved":0,"allocated":10076160,"data_stored":5344818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1794,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001373"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.701553+0000","last_change":"2023-11-08T19:28:02.701553+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.701553+0000","last_undegraded":"2023-11-08T19:28:02.701553+0000","last_fullsized":"2023-11-08T19:28:02.701553+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.719263+0000","last_change":"2023-11-08T19:28:03.715294+0000","last_active":"2023-11-08T19:28:03.719263+0000","last_peered":"2023-11-08T19:28:03.719263+0000","last_clean":"2023-11-08T19:28:03.719263+0000","last_became_active":"2023-11-08T19:28:03.715007+0000","last_became_peered":"2023-11-08T19:28:03.715007+0000","last_unstale":"2023-11-08T19:28:03.719263+0000","last_undegraded":"2023-11-08T19:28:03.719263+0000","last_fullsized":"2023-11-08T19:28:03.719263+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:51:46.879726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.700726+0000","last_change":"2023-11-08T19:28:02.700726+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.700726+0000","last_undegraded":"2023-11-08T19:28:02.700726+0000","last_fullsized":"2023-11-08T19:28:02.700726+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.699799+0000","last_change":"2023-11-08T19:28:02.699799+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.699799+0000","last_undegraded":"2023-11-08T19:28:02.699799+0000","last_fullsized":"2023-11-08T19:28:02.699799+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.704338+0000","last_change":"2023-11-08T19:28:02.704338+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.704338+0000","last_undegraded":"2023-11-08T19:28:02.704338+0000","last_fullsized":"2023-11-08T19:28:02.704338+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.722681+0000","last_change":"2023-11-08T19:28:03.718208+0000","last_active":"2023-11-08T19:28:03.722681+0000","last_peered":"2023-11-08T19:28:03.722681+0000","last_clean":"2023-11-08T19:28:03.722681+0000","last_became_active":"2023-11-08T19:28:03.716630+0000","last_became_peered":"2023-11-08T19:28:03.716630+0000","last_unstale":"2023-11-08T19:28:03.722681+0000","last_undegraded":"2023-11-08T19:28:03.722681+0000","last_fullsized":"2023-11-08T19:28:03.722681+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:21:24.678157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.712530+0000","last_change":"2023-11-08T19:28:03.708477+0000","last_active":"2023-11-08T19:28:03.712530+0000","last_peered":"2023-11-08T19:28:03.712530+0000","last_clean":"2023-11-08T19:28:03.712530+0000","last_became_active":"2023-11-08T19:28:03.708188+0000","last_became_peered":"2023-11-08T19:28:03.708188+0000","last_unstale":"2023-11-08T19:28:03.712530+0000","last_undegraded":"2023-11-08T19:28:03.712530+0000","last_fullsized":"2023-11-08T19:28:03.712530+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:57:58.571556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.714751+0000","last_change":"2023-11-08T19:28:02.714751+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.714751+0000","last_undegraded":"2023-11-08T19:28:02.714751+0000","last_fullsized":"2023-11-08T19:28:02.714751+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.722749+0000","last_change":"2023-11-08T19:28:03.718615+0000","last_active":"2023-11-08T19:28:03.722749+0000","last_peered":"2023-11-08T19:28:03.722749+0000","last_clean":"2023-11-08T19:28:03.722749+0000","last_became_active":"2023-11-08T19:28:03.718295+0000","last_became_peered":"2023-11-08T19:28:03.718295+0000","last_unstale":"2023-11-08T19:28:03.722749+0000","last_undegraded":"2023-11-08T19:28:03.722749+0000","last_fullsized":"2023-11-08T19:28:03.722749+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:23:12.882243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.702139+0000","last_change":"2023-11-08T19:28:02.702139+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.702139+0000","last_undegraded":"2023-11-08T19:28:02.702139+0000","last_fullsized":"2023-11-08T19:28:02.702139+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.705435+0000","last_change":"2023-11-08T19:28:02.705435+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.705435+0000","last_undegraded":"2023-11-08T19:28:02.705435+0000","last_fullsized":"2023-11-08T19:28:02.705435+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.703962+0000","last_change":"2023-11-08T19:28:02.703962+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.703962+0000","last_undegraded":"2023-11-08T19:28:02.703962+0000","last_fullsized":"2023-11-08T19:28:02.703962+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.697723+0000","last_change":"2023-11-08T19:28:02.697723+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.697723+0000","last_undegraded":"2023-11-08T19:28:02.697723+0000","last_fullsized":"2023-11-08T19:28:02.697723+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.721906+0000","last_change":"2023-11-08T19:28:03.718346+0000","last_active":"2023-11-08T19:28:03.721906+0000","last_peered":"2023-11-08T19:28:03.721906+0000","last_clean":"2023-11-08T19:28:03.721906+0000","last_became_active":"2023-11-08T19:28:03.718068+0000","last_became_peered":"2023-11-08T19:28:03.718068+0000","last_unstale":"2023-11-08T19:28:03.721906+0000","last_undegraded":"2023-11-08T19:28:03.721906+0000","last_fullsized":"2023-11-08T19:28:03.721906+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:29:02.099134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.721945+0000","last_change":"2023-11-08T19:28:03.717576+0000","last_active":"2023-11-08T19:28:03.721945+0000","last_peered":"2023-11-08T19:28:03.721945+0000","last_clean":"2023-11-08T19:28:03.721945+0000","last_became_active":"2023-11-08T19:28:03.715708+0000","last_became_peered":"2023-11-08T19:28:03.715708+0000","last_unstale":"2023-11-08T19:28:03.721945+0000","last_undegraded":"2023-11-08T19:28:03.721945+0000","last_fullsized":"2023-11-08T19:28:03.721945+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:40:03.163555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.711516+0000","last_change":"2023-11-08T19:28:03.703641+0000","last_active":"2023-11-08T19:28:03.711516+0000","last_peered":"2023-11-08T19:28:03.711516+0000","last_clean":"2023-11-08T19:28:03.711516+0000","last_became_active":"2023-11-08T19:28:03.703331+0000","last_became_peered":"2023-11-08T19:28:03.703331+0000","last_unstale":"2023-11-08T19:28:03.711516+0000","last_undegraded":"2023-11-08T19:28:03.711516+0000","last_fullsized":"2023-11-08T19:28:03.711516+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:32:03.694798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.700364+0000","last_change":"2023-11-08T19:28:02.700364+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.700364+0000","last_undegraded":"2023-11-08T19:28:02.700364+0000","last_fullsized":"2023-11-08T19:28:02.700364+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.719244+0000","last_change":"2023-11-08T19:28:03.715221+0000","last_active":"2023-11-08T19:28:03.719244+0000","last_peered":"2023-11-08T19:28:03.719244+0000","last_clean":"2023-11-08T19:28:03.719244+0000","last_became_active":"2023-11-08T19:28:03.714929+0000","last_became_peered":"2023-11-08T19:28:03.714929+0000","last_unstale":"2023-11-08T19:28:03.719244+0000","last_undegraded":"2023-11-08T19:28:03.719244+0000","last_fullsized":"2023-11-08T19:28:03.719244+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:58:49.797420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.700365+0000","last_change":"2023-11-08T19:28:02.700365+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.700365+0000","last_undegraded":"2023-11-08T19:28:02.700365+0000","last_fullsized":"2023-11-08T19:28:02.700365+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.704383+0000","last_change":"2023-11-08T19:28:02.704383+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.704383+0000","last_undegraded":"2023-11-08T19:28:02.704383+0000","last_fullsized":"2023-11-08T19:28:02.704383+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.713555+0000","last_change":"2023-11-08T19:28:02.713555+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.713555+0000","last_undegraded":"2023-11-08T19:28:02.713555+0000","last_fullsized":"2023-11-08T19:28:02.713555+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.702640+0000","last_change":"2023-11-08T19:28:02.702640+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.702640+0000","last_undegraded":"2023-11-08T19:28:02.702640+0000","last_fullsized":"2023-11-08T19:28:02.702640+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.699841+0000","last_change":"2023-11-08T19:28:02.699841+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.699841+0000","last_undegraded":"2023-11-08T19:28:02.699841+0000","last_fullsized":"2023-11-08T19:28:02.699841+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.702861+0000","last_change":"2023-11-08T19:28:02.702861+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.702861+0000","last_undegraded":"2023-11-08T19:28:02.702861+0000","last_fullsized":"2023-11-08T19:28:02.702861+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.707625+0000","last_change":"2023-11-08T19:28:02.707625+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.707625+0000","last_undegraded":"2023-11-08T19:28:02.707625+0000","last_fullsized":"2023-11-08T19:28:02.707625+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.710228+0000","last_change":"2023-11-08T19:28:02.710228+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.710228+0000","last_undegraded":"2023-11-08T19:28:02.710228+0000","last_fullsized":"2023-11-08T19:28:02.710228+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.704524+0000","last_change":"2023-11-08T19:28:02.704524+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.704524+0000","last_undegraded":"2023-11-08T19:28:02.704524+0000","last_fullsized":"2023-11-08T19:28:02.704524+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.725514+0000","last_change":"2023-11-08T19:28:03.721162+0000","last_active":"2023-11-08T19:28:03.725514+0000","last_peered":"2023-11-08T19:28:03.725514+0000","last_clean":"2023-11-08T19:28:03.725514+0000","last_became_active":"2023-11-08T19:28:03.720956+0000","last_became_peered":"2023-11-08T19:28:03.720956+0000","last_unstale":"2023-11-08T19:28:03.725514+0000","last_undegraded":"2023-11-08T19:28:03.725514+0000","last_fullsized":"2023-11-08T19:28:03.725514+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:09:19.883657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.712428+0000","last_change":"2023-11-08T19:28:02.712428+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.712428+0000","last_undegraded":"2023-11-08T19:28:02.712428+0000","last_fullsized":"2023-11-08T19:28:02.712428+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.705928+0000","last_change":"2023-11-08T19:28:02.705928+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.705928+0000","last_undegraded":"2023-11-08T19:28:02.705928+0000","last_fullsized":"2023-11-08T19:28:02.705928+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.712510+0000","last_change":"2023-11-08T19:28:03.707881+0000","last_active":"2023-11-08T19:28:03.712510+0000","last_peered":"2023-11-08T19:28:03.712510+0000","last_clean":"2023-11-08T19:28:03.712510+0000","last_became_active":"2023-11-08T19:28:03.707569+0000","last_became_peered":"2023-11-08T19:28:03.707569+0000","last_unstale":"2023-11-08T19:28:03.712510+0000","last_undegraded":"2023-11-08T19:28:03.712510+0000","last_fullsized":"2023-11-08T19:28:03.712510+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:00:53.075545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.711643+0000","last_change":"2023-11-08T19:28:02.711643+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.711643+0000","last_undegraded":"2023-11-08T19:28:02.711643+0000","last_fullsized":"2023-11-08T19:28:02.711643+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.702089+0000","last_change":"2023-11-08T19:28:02.702089+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.702089+0000","last_undegraded":"2023-11-08T19:28:02.702089+0000","last_fullsized":"2023-11-08T19:28:02.702089+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.725513+0000","last_change":"2023-11-08T19:28:02.720536+0000","last_active":"2023-11-08T19:28:02.725513+0000","last_peered":"2023-11-08T19:28:02.725513+0000","last_clean":"2023-11-08T19:28:02.725513+0000","last_became_active":"2023-11-08T19:28:02.720236+0000","last_became_peered":"2023-11-08T19:28:02.720236+0000","last_unstale":"2023-11-08T19:28:02.725513+0000","last_undegraded":"2023-11-08T19:28:02.725513+0000","last_fullsized":"2023-11-08T19:28:02.725513+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:56:59.813550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.700732+0000","last_change":"2023-11-08T19:27:08.190826+0000","last_active":"2023-11-08T19:28:02.700732+0000","last_peered":"2023-11-08T19:28:02.700732+0000","last_clean":"2023-11-08T19:28:02.700732+0000","last_became_active":"2023-11-08T19:27:08.190440+0000","last_became_peered":"2023-11-08T19:27:08.190440+0000","last_unstale":"2023-11-08T19:28:02.700732+0000","last_undegraded":"2023-11-08T19:28:02.700732+0000","last_fullsized":"2023-11-08T19:28:02.700732+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:42:48.376462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":237,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.699373+0000","last_change":"2023-11-08T19:24:45.888122+0000","last_active":"2023-11-08T19:28:02.699373+0000","last_peered":"2023-11-08T19:28:02.699373+0000","last_clean":"2023-11-08T19:28:02.699373+0000","last_became_active":"2023-11-08T19:24:45.887048+0000","last_became_peered":"2023-11-08T19:24:45.887048+0000","last_unstale":"2023-11-08T19:28:02.699373+0000","last_undegraded":"2023-11-08T19:28:02.699373+0000","last_fullsized":"2023-11-08T19:28:02.699373+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:44.575133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:44.575133+0000","last_clean_scrub_stamp":"2023-11-08T19:24:44.575133+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:19:20.202062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":60,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.698526+0000","last_change":"2023-11-08T19:26:06.542044+0000","last_active":"2023-11-08T19:28:01.698526+0000","last_peered":"2023-11-08T19:28:01.698526+0000","last_clean":"2023-11-08T19:28:01.698526+0000","last_became_active":"2023-11-08T19:26:04.603056+0000","last_became_peered":"2023-11-08T19:26:04.603056+0000","last_unstale":"2023-11-08T19:28:01.698526+0000","last_undegraded":"2023-11-08T19:28:01.698526+0000","last_fullsized":"2023-11-08T19:28:01.698526+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:41:50.771457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000517414,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.694743+0000","last_change":"2023-11-08T19:27:09.496842+0000","last_active":"2023-11-08T19:28:01.694743+0000","last_peered":"2023-11-08T19:28:01.694743+0000","last_clean":"2023-11-08T19:28:01.694743+0000","last_became_active":"2023-11-08T19:27:07.185776+0000","last_became_peered":"2023-11-08T19:27:07.185776+0000","last_unstale":"2023-11-08T19:28:01.694743+0000","last_undegraded":"2023-11-08T19:28:01.694743+0000","last_fullsized":"2023-11-08T19:28:01.694743+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:21:13.923970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031747200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.698950+0000","last_change":"2023-11-08T19:28:02.698950+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.698950+0000","last_undegraded":"2023-11-08T19:28:02.698950+0000","last_fullsized":"2023-11-08T19:28:02.698950+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.725538+0000","last_change":"2023-11-08T19:28:02.720512+0000","last_active":"2023-11-08T19:28:02.725538+0000","last_peered":"2023-11-08T19:28:02.725538+0000","last_clean":"2023-11-08T19:28:02.725538+0000","last_became_active":"2023-11-08T19:28:02.720202+0000","last_became_peered":"2023-11-08T19:28:02.720202+0000","last_unstale":"2023-11-08T19:28:02.725538+0000","last_undegraded":"2023-11-08T19:28:02.725538+0000","last_fullsized":"2023-11-08T19:28:02.725538+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:26:10.008919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.694327+0000","last_change":"2023-11-08T19:27:08.188241+0000","last_active":"2023-11-08T19:28:01.694327+0000","last_peered":"2023-11-08T19:28:01.694327+0000","last_clean":"2023-11-08T19:28:01.694327+0000","last_became_active":"2023-11-08T19:27:08.187863+0000","last_became_peered":"2023-11-08T19:27:08.187863+0000","last_unstale":"2023-11-08T19:28:01.694327+0000","last_undegraded":"2023-11-08T19:28:01.694327+0000","last_fullsized":"2023-11-08T19:28:01.694327+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:45:22.084015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":58,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.696493+0000","last_change":"2023-11-08T19:26:06.265419+0000","last_active":"2023-11-08T19:28:01.696493+0000","last_peered":"2023-11-08T19:28:01.696493+0000","last_clean":"2023-11-08T19:28:01.696493+0000","last_became_active":"2023-11-08T19:26:04.605333+0000","last_became_peered":"2023-11-08T19:26:04.605333+0000","last_unstale":"2023-11-08T19:28:01.696493+0000","last_undegraded":"2023-11-08T19:28:01.696493+0000","last_fullsized":"2023-11-08T19:28:01.696493+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:01:56.991346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064606400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.701529+0000","last_change":"2023-11-08T19:27:09.182113+0000","last_active":"2023-11-08T19:28:02.701529+0000","last_peered":"2023-11-08T19:28:02.701529+0000","last_clean":"2023-11-08T19:28:02.701529+0000","last_became_active":"2023-11-08T19:27:07.189939+0000","last_became_peered":"2023-11-08T19:27:07.189939+0000","last_unstale":"2023-11-08T19:28:02.701529+0000","last_undegraded":"2023-11-08T19:28:02.701529+0000","last_fullsized":"2023-11-08T19:28:02.701529+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:41:08.822366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036507500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.699139+0000","last_change":"2023-11-08T19:28:02.699139+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.699139+0000","last_undegraded":"2023-11-08T19:28:02.699139+0000","last_fullsized":"2023-11-08T19:28:02.699139+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.725537+0000","last_change":"2023-11-08T19:28:02.720708+0000","last_active":"2023-11-08T19:28:02.725537+0000","last_peered":"2023-11-08T19:28:02.725537+0000","last_clean":"2023-11-08T19:28:02.725537+0000","last_became_active":"2023-11-08T19:28:02.720263+0000","last_became_peered":"2023-11-08T19:28:02.720263+0000","last_unstale":"2023-11-08T19:28:02.725537+0000","last_undegraded":"2023-11-08T19:28:02.725537+0000","last_fullsized":"2023-11-08T19:28:02.725537+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:56:32.169912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.691014+0000","last_change":"2023-11-08T19:27:08.187127+0000","last_active":"2023-11-08T19:28:01.691014+0000","last_peered":"2023-11-08T19:28:01.691014+0000","last_clean":"2023-11-08T19:28:01.691014+0000","last_became_active":"2023-11-08T19:27:08.182648+0000","last_became_peered":"2023-11-08T19:27:08.182648+0000","last_unstale":"2023-11-08T19:28:01.691014+0000","last_undegraded":"2023-11-08T19:28:01.691014+0000","last_fullsized":"2023-11-08T19:28:01.691014+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:50:12.101878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":71,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.700875+0000","last_change":"2023-11-08T19:26:05.609570+0000","last_active":"2023-11-08T19:28:02.700875+0000","last_peered":"2023-11-08T19:28:02.700875+0000","last_clean":"2023-11-08T19:28:02.700875+0000","last_became_active":"2023-11-08T19:26:04.608379+0000","last_became_peered":"2023-11-08T19:26:04.608379+0000","last_unstale":"2023-11-08T19:28:02.700875+0000","last_undegraded":"2023-11-08T19:28:02.700875+0000","last_fullsized":"2023-11-08T19:28:02.700875+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:40:08.986649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000529657,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.696937+0000","last_change":"2023-11-08T19:27:09.211991+0000","last_active":"2023-11-08T19:28:01.696937+0000","last_peered":"2023-11-08T19:28:01.696937+0000","last_clean":"2023-11-08T19:28:01.696937+0000","last_became_active":"2023-11-08T19:27:07.195231+0000","last_became_peered":"2023-11-08T19:27:07.195231+0000","last_unstale":"2023-11-08T19:28:01.696937+0000","last_undegraded":"2023-11-08T19:28:01.696937+0000","last_fullsized":"2023-11-08T19:28:01.696937+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:25:14.487524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000441221,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.697661+0000","last_change":"2023-11-08T19:28:02.697661+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.697661+0000","last_undegraded":"2023-11-08T19:28:02.697661+0000","last_fullsized":"2023-11-08T19:28:02.697661+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.705119+0000","last_change":"2023-11-08T19:28:02.707543+0000","last_active":"2023-11-08T19:28:03.705119+0000","last_peered":"2023-11-08T19:28:03.705119+0000","last_clean":"2023-11-08T19:28:03.705119+0000","last_became_active":"2023-11-08T19:28:02.707034+0000","last_became_peered":"2023-11-08T19:28:02.707034+0000","last_unstale":"2023-11-08T19:28:03.705119+0000","last_undegraded":"2023-11-08T19:28:03.705119+0000","last_fullsized":"2023-11-08T19:28:03.705119+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:06:46.912027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.690189+0000","last_change":"2023-11-08T19:27:08.195364+0000","last_active":"2023-11-08T19:28:01.690189+0000","last_peered":"2023-11-08T19:28:01.690189+0000","last_clean":"2023-11-08T19:28:01.690189+0000","last_became_active":"2023-11-08T19:27:08.195074+0000","last_became_peered":"2023-11-08T19:27:08.195074+0000","last_unstale":"2023-11-08T19:28:01.690189+0000","last_undegraded":"2023-11-08T19:28:01.690189+0000","last_fullsized":"2023-11-08T19:28:01.690189+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:02:57.552873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":59,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.700695+0000","last_change":"2023-11-08T19:26:06.394300+0000","last_active":"2023-11-08T19:28:02.700695+0000","last_peered":"2023-11-08T19:28:02.700695+0000","last_clean":"2023-11-08T19:28:02.700695+0000","last_became_active":"2023-11-08T19:26:04.601685+0000","last_became_peered":"2023-11-08T19:26:04.601685+0000","last_unstale":"2023-11-08T19:28:02.700695+0000","last_undegraded":"2023-11-08T19:28:02.700695+0000","last_fullsized":"2023-11-08T19:28:02.700695+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:51:06.626474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061044099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.690413+0000","last_change":"2023-11-08T19:27:09.178466+0000","last_active":"2023-11-08T19:28:01.690413+0000","last_peered":"2023-11-08T19:28:01.690413+0000","last_clean":"2023-11-08T19:28:01.690413+0000","last_became_active":"2023-11-08T19:27:07.189386+0000","last_became_peered":"2023-11-08T19:27:07.189386+0000","last_unstale":"2023-11-08T19:28:01.690413+0000","last_undegraded":"2023-11-08T19:28:01.690413+0000","last_fullsized":"2023-11-08T19:28:01.690413+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:48:51.139820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049390100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.713309+0000","last_change":"2023-11-08T19:28:02.713309+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.713309+0000","last_undegraded":"2023-11-08T19:28:02.713309+0000","last_fullsized":"2023-11-08T19:28:02.713309+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.727972+0000","last_change":"2023-11-08T19:28:02.723021+0000","last_active":"2023-11-08T19:28:02.727972+0000","last_peered":"2023-11-08T19:28:02.727972+0000","last_clean":"2023-11-08T19:28:02.727972+0000","last_became_active":"2023-11-08T19:28:02.722748+0000","last_became_peered":"2023-11-08T19:28:02.722748+0000","last_unstale":"2023-11-08T19:28:02.727972+0000","last_undegraded":"2023-11-08T19:28:02.727972+0000","last_fullsized":"2023-11-08T19:28:02.727972+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:26:52.310010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.695135+0000","last_change":"2023-11-08T19:27:08.189452+0000","last_active":"2023-11-08T19:28:01.695135+0000","last_peered":"2023-11-08T19:28:01.695135+0000","last_clean":"2023-11-08T19:28:01.695135+0000","last_became_active":"2023-11-08T19:27:08.189088+0000","last_became_peered":"2023-11-08T19:27:08.189088+0000","last_unstale":"2023-11-08T19:28:01.695135+0000","last_undegraded":"2023-11-08T19:28:01.695135+0000","last_fullsized":"2023-11-08T19:28:01.695135+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:25:18.963220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":59,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.694898+0000","last_change":"2023-11-08T19:26:06.296634+0000","last_active":"2023-11-08T19:28:01.694898+0000","last_peered":"2023-11-08T19:28:01.694898+0000","last_clean":"2023-11-08T19:28:01.694898+0000","last_became_active":"2023-11-08T19:26:04.603088+0000","last_became_peered":"2023-11-08T19:26:04.603088+0000","last_unstale":"2023-11-08T19:28:01.694898+0000","last_undegraded":"2023-11-08T19:28:01.694898+0000","last_fullsized":"2023-11-08T19:28:01.694898+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:01:56.991346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038180899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.695512+0000","last_change":"2023-11-08T19:27:09.431220+0000","last_active":"2023-11-08T19:28:01.695512+0000","last_peered":"2023-11-08T19:28:01.695512+0000","last_clean":"2023-11-08T19:28:01.695512+0000","last_became_active":"2023-11-08T19:27:07.191589+0000","last_became_peered":"2023-11-08T19:27:07.191589+0000","last_unstale":"2023-11-08T19:28:01.695512+0000","last_undegraded":"2023-11-08T19:28:01.695512+0000","last_fullsized":"2023-11-08T19:28:01.695512+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:18:19.964487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000729527,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.714312+0000","last_change":"2023-11-08T19:28:02.714312+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.714312+0000","last_undegraded":"2023-11-08T19:28:02.714312+0000","last_fullsized":"2023-11-08T19:28:02.714312+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"91'1","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.719567+0000","last_change":"2023-11-08T19:28:02.712293+0000","last_active":"2023-11-08T19:28:03.719567+0000","last_peered":"2023-11-08T19:28:03.719567+0000","last_clean":"2023-11-08T19:28:03.719567+0000","last_became_active":"2023-11-08T19:28:02.712002+0000","last_became_peered":"2023-11-08T19:28:02.712002+0000","last_unstale":"2023-11-08T19:28:03.719567+0000","last_undegraded":"2023-11-08T19:28:03.719567+0000","last_fullsized":"2023-11-08T19:28:03.719567+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:24:47.603187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.694985+0000","last_change":"2023-11-08T19:27:08.193337+0000","last_active":"2023-11-08T19:28:01.694985+0000","last_peered":"2023-11-08T19:28:01.694985+0000","last_clean":"2023-11-08T19:28:01.694985+0000","last_became_active":"2023-11-08T19:27:08.192997+0000","last_became_peered":"2023-11-08T19:27:08.192997+0000","last_unstale":"2023-11-08T19:28:01.694985+0000","last_undegraded":"2023-11-08T19:28:01.694985+0000","last_fullsized":"2023-11-08T19:28:01.694985+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:45:22.084015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":64,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.696307+0000","last_change":"2023-11-08T19:26:06.424511+0000","last_active":"2023-11-08T19:28:01.696307+0000","last_peered":"2023-11-08T19:28:01.696307+0000","last_clean":"2023-11-08T19:28:01.696307+0000","last_became_active":"2023-11-08T19:26:04.598973+0000","last_became_peered":"2023-11-08T19:26:04.598973+0000","last_unstale":"2023-11-08T19:28:01.696307+0000","last_undegraded":"2023-11-08T19:28:01.696307+0000","last_fullsized":"2023-11-08T19:28:01.696307+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:41:53.497456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031755900000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.694476+0000","last_change":"2023-11-08T19:27:09.497024+0000","last_active":"2023-11-08T19:28:01.694476+0000","last_peered":"2023-11-08T19:28:01.694476+0000","last_clean":"2023-11-08T19:28:01.694476+0000","last_became_active":"2023-11-08T19:27:07.190132+0000","last_became_peered":"2023-11-08T19:27:07.190132+0000","last_unstale":"2023-11-08T19:28:01.694476+0000","last_undegraded":"2023-11-08T19:28:01.694476+0000","last_fullsized":"2023-11-08T19:28:01.694476+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:36:34.881217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054126000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.718435+0000","last_change":"2023-11-08T19:28:03.713184+0000","last_active":"2023-11-08T19:28:03.718435+0000","last_peered":"2023-11-08T19:28:03.718435+0000","last_clean":"2023-11-08T19:28:03.718435+0000","last_became_active":"2023-11-08T19:28:03.712864+0000","last_became_peered":"2023-11-08T19:28:03.712864+0000","last_unstale":"2023-11-08T19:28:03.718435+0000","last_undegraded":"2023-11-08T19:28:03.718435+0000","last_fullsized":"2023-11-08T19:28:03.718435+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:38:54.805747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"91'1","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.716012+0000","last_change":"2023-11-08T19:28:02.713126+0000","last_active":"2023-11-08T19:28:03.716012+0000","last_peered":"2023-11-08T19:28:03.716012+0000","last_clean":"2023-11-08T19:28:03.716012+0000","last_became_active":"2023-11-08T19:28:02.712836+0000","last_became_peered":"2023-11-08T19:28:02.712836+0000","last_unstale":"2023-11-08T19:28:03.716012+0000","last_undegraded":"2023-11-08T19:28:03.716012+0000","last_fullsized":"2023-11-08T19:28:03.716012+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:29:31.276279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.695424+0000","last_change":"2023-11-08T19:27:08.194196+0000","last_active":"2023-11-08T19:28:01.695424+0000","last_peered":"2023-11-08T19:28:01.695424+0000","last_clean":"2023-11-08T19:28:01.695424+0000","last_became_active":"2023-11-08T19:27:08.193912+0000","last_became_peered":"2023-11-08T19:27:08.193912+0000","last_unstale":"2023-11-08T19:28:01.695424+0000","last_undegraded":"2023-11-08T19:28:01.695424+0000","last_fullsized":"2023-11-08T19:28:01.695424+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:25:18.963220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":59,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.695591+0000","last_change":"2023-11-08T19:26:06.296937+0000","last_active":"2023-11-08T19:28:01.695591+0000","last_peered":"2023-11-08T19:28:01.695591+0000","last_clean":"2023-11-08T19:28:01.695591+0000","last_became_active":"2023-11-08T19:26:04.599999+0000","last_became_peered":"2023-11-08T19:26:04.599999+0000","last_unstale":"2023-11-08T19:28:01.695591+0000","last_undegraded":"2023-11-08T19:28:01.695591+0000","last_fullsized":"2023-11-08T19:28:01.695591+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:50:43.827570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025605700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.695067+0000","last_change":"2023-11-08T19:27:09.497040+0000","last_active":"2023-11-08T19:28:01.695067+0000","last_peered":"2023-11-08T19:28:01.695067+0000","last_clean":"2023-11-08T19:28:01.695067+0000","last_became_active":"2023-11-08T19:27:07.189135+0000","last_became_peered":"2023-11-08T19:27:07.189135+0000","last_unstale":"2023-11-08T19:28:01.695067+0000","last_undegraded":"2023-11-08T19:28:01.695067+0000","last_fullsized":"2023-11-08T19:28:01.695067+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:12:29.400972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051794299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.703964+0000","last_change":"2023-11-08T19:28:02.703964+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.703964+0000","last_undegraded":"2023-11-08T19:28:02.703964+0000","last_fullsized":"2023-11-08T19:28:02.703964+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.710152+0000","last_change":"2023-11-08T19:28:02.705153+0000","last_active":"2023-11-08T19:28:02.710152+0000","last_peered":"2023-11-08T19:28:02.710152+0000","last_clean":"2023-11-08T19:28:02.710152+0000","last_became_active":"2023-11-08T19:28:02.704854+0000","last_became_peered":"2023-11-08T19:28:02.704854+0000","last_unstale":"2023-11-08T19:28:02.710152+0000","last_undegraded":"2023-11-08T19:28:02.710152+0000","last_fullsized":"2023-11-08T19:28:02.710152+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:25:00.088013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.703721+0000","last_change":"2023-11-08T19:27:08.187053+0000","last_active":"2023-11-08T19:28:02.703721+0000","last_peered":"2023-11-08T19:28:02.703721+0000","last_clean":"2023-11-08T19:28:02.703721+0000","last_became_active":"2023-11-08T19:27:08.186643+0000","last_became_peered":"2023-11-08T19:27:08.186643+0000","last_unstale":"2023-11-08T19:28:02.703721+0000","last_undegraded":"2023-11-08T19:28:02.703721+0000","last_fullsized":"2023-11-08T19:28:02.703721+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:35:30.073885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":71,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.700976+0000","last_change":"2023-11-08T19:26:05.609824+0000","last_active":"2023-11-08T19:28:02.700976+0000","last_peered":"2023-11-08T19:28:02.700976+0000","last_clean":"2023-11-08T19:28:02.700976+0000","last_became_active":"2023-11-08T19:26:04.603320+0000","last_became_peered":"2023-11-08T19:26:04.603320+0000","last_unstale":"2023-11-08T19:28:02.700976+0000","last_undegraded":"2023-11-08T19:28:02.700976+0000","last_fullsized":"2023-11-08T19:28:02.700976+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:05:45.976615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063225,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.694457+0000","last_change":"2023-11-08T19:27:09.496655+0000","last_active":"2023-11-08T19:28:01.694457+0000","last_peered":"2023-11-08T19:28:01.694457+0000","last_clean":"2023-11-08T19:28:01.694457+0000","last_became_active":"2023-11-08T19:27:07.189895+0000","last_became_peered":"2023-11-08T19:27:07.189895+0000","last_unstale":"2023-11-08T19:28:01.694457+0000","last_undegraded":"2023-11-08T19:28:01.694457+0000","last_fullsized":"2023-11-08T19:28:01.694457+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:41:08.822366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045654199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.711687+0000","last_change":"2023-11-08T19:28:02.711687+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.711687+0000","last_undegraded":"2023-11-08T19:28:02.711687+0000","last_fullsized":"2023-11-08T19:28:02.711687+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.711905+0000","last_change":"2023-11-08T19:28:02.709891+0000","last_active":"2023-11-08T19:28:02.711905+0000","last_peered":"2023-11-08T19:28:02.711905+0000","last_clean":"2023-11-08T19:28:02.711905+0000","last_became_active":"2023-11-08T19:28:02.709612+0000","last_became_peered":"2023-11-08T19:28:02.709612+0000","last_unstale":"2023-11-08T19:28:02.711905+0000","last_undegraded":"2023-11-08T19:28:02.711905+0000","last_fullsized":"2023-11-08T19:28:02.711905+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:24:47.603187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":60,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.698243+0000","last_change":"2023-11-08T19:26:06.541360+0000","last_active":"2023-11-08T19:28:01.698243+0000","last_peered":"2023-11-08T19:28:01.698243+0000","last_clean":"2023-11-08T19:28:01.698243+0000","last_became_active":"2023-11-08T19:26:04.603725+0000","last_became_peered":"2023-11-08T19:26:04.603725+0000","last_unstale":"2023-11-08T19:28:01.698243+0000","last_undegraded":"2023-11-08T19:28:01.698243+0000","last_fullsized":"2023-11-08T19:28:01.698243+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:28:48.576742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053365400000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.698592+0000","last_change":"2023-11-08T19:27:09.283599+0000","last_active":"2023-11-08T19:28:01.698592+0000","last_peered":"2023-11-08T19:28:01.698592+0000","last_clean":"2023-11-08T19:28:01.698592+0000","last_became_active":"2023-11-08T19:27:07.189595+0000","last_became_peered":"2023-11-08T19:27:07.189595+0000","last_unstale":"2023-11-08T19:28:01.698592+0000","last_undegraded":"2023-11-08T19:28:01.698592+0000","last_fullsized":"2023-11-08T19:28:01.698592+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:48:20.082682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000323479,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.696394+0000","last_change":"2023-11-08T19:27:08.185395+0000","last_active":"2023-11-08T19:28:01.696394+0000","last_peered":"2023-11-08T19:28:01.696394+0000","last_clean":"2023-11-08T19:28:01.696394+0000","last_became_active":"2023-11-08T19:27:08.184765+0000","last_became_peered":"2023-11-08T19:27:08.184765+0000","last_unstale":"2023-11-08T19:28:01.696394+0000","last_undegraded":"2023-11-08T19:28:01.696394+0000","last_fullsized":"2023-11-08T19:28:01.696394+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:08:51.864418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.712306+0000","last_change":"2023-11-08T19:28:02.712306+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.712306+0000","last_undegraded":"2023-11-08T19:28:02.712306+0000","last_fullsized":"2023-11-08T19:28:02.712306+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.725515+0000","last_change":"2023-11-08T19:28:02.720596+0000","last_active":"2023-11-08T19:28:02.725515+0000","last_peered":"2023-11-08T19:28:02.725515+0000","last_clean":"2023-11-08T19:28:02.725515+0000","last_became_active":"2023-11-08T19:28:02.720144+0000","last_became_peered":"2023-11-08T19:28:02.720144+0000","last_unstale":"2023-11-08T19:28:02.725515+0000","last_undegraded":"2023-11-08T19:28:02.725515+0000","last_fullsized":"2023-11-08T19:28:02.725515+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:51:33.542656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":64,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.694624+0000","last_change":"2023-11-08T19:26:06.424236+0000","last_active":"2023-11-08T19:28:01.694624+0000","last_peered":"2023-11-08T19:28:01.694624+0000","last_clean":"2023-11-08T19:28:01.694624+0000","last_became_active":"2023-11-08T19:26:04.600115+0000","last_became_peered":"2023-11-08T19:26:04.600115+0000","last_unstale":"2023-11-08T19:28:01.694624+0000","last_undegraded":"2023-11-08T19:28:01.694624+0000","last_fullsized":"2023-11-08T19:28:01.694624+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:18:01.876605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065803900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.695283+0000","last_change":"2023-11-08T19:27:09.331899+0000","last_active":"2023-11-08T19:28:01.695283+0000","last_peered":"2023-11-08T19:28:01.695283+0000","last_clean":"2023-11-08T19:28:01.695283+0000","last_became_active":"2023-11-08T19:27:07.191460+0000","last_became_peered":"2023-11-08T19:27:07.191460+0000","last_unstale":"2023-11-08T19:28:01.695283+0000","last_undegraded":"2023-11-08T19:28:01.695283+0000","last_fullsized":"2023-11-08T19:28:01.695283+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:51:29.247104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044907500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.701089+0000","last_change":"2023-11-08T19:27:08.191802+0000","last_active":"2023-11-08T19:28:02.701089+0000","last_peered":"2023-11-08T19:28:02.701089+0000","last_clean":"2023-11-08T19:28:02.701089+0000","last_became_active":"2023-11-08T19:27:08.191520+0000","last_became_peered":"2023-11-08T19:27:08.191520+0000","last_unstale":"2023-11-08T19:28:02.701089+0000","last_undegraded":"2023-11-08T19:28:02.701089+0000","last_fullsized":"2023-11-08T19:28:02.701089+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:49:16.458634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.724704+0000","last_change":"2023-11-08T19:28:03.722086+0000","last_active":"2023-11-08T19:28:03.724704+0000","last_peered":"2023-11-08T19:28:03.724704+0000","last_clean":"2023-11-08T19:28:03.724704+0000","last_became_active":"2023-11-08T19:28:03.721713+0000","last_became_peered":"2023-11-08T19:28:03.721713+0000","last_unstale":"2023-11-08T19:28:03.724704+0000","last_undegraded":"2023-11-08T19:28:03.724704+0000","last_fullsized":"2023-11-08T19:28:03.724704+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:34:22.087376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.705286+0000","last_change":"2023-11-08T19:28:02.707105+0000","last_active":"2023-11-08T19:28:03.705286+0000","last_peered":"2023-11-08T19:28:03.705286+0000","last_clean":"2023-11-08T19:28:03.705286+0000","last_became_active":"2023-11-08T19:28:02.706615+0000","last_became_peered":"2023-11-08T19:28:02.706615+0000","last_unstale":"2023-11-08T19:28:03.705286+0000","last_undegraded":"2023-11-08T19:28:03.705286+0000","last_fullsized":"2023-11-08T19:28:03.705286+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:18:55.364855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":71,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.699281+0000","last_change":"2023-11-08T19:26:05.608892+0000","last_active":"2023-11-08T19:28:02.699281+0000","last_peered":"2023-11-08T19:28:02.699281+0000","last_clean":"2023-11-08T19:28:02.699281+0000","last_became_active":"2023-11-08T19:26:04.604519+0000","last_became_peered":"2023-11-08T19:26:04.604519+0000","last_unstale":"2023-11-08T19:28:02.699281+0000","last_undegraded":"2023-11-08T19:28:02.699281+0000","last_fullsized":"2023-11-08T19:28:02.699281+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:11:26.508634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051248099999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.700897+0000","last_change":"2023-11-08T19:27:09.181431+0000","last_active":"2023-11-08T19:28:02.700897+0000","last_peered":"2023-11-08T19:28:02.700897+0000","last_clean":"2023-11-08T19:28:02.700897+0000","last_became_active":"2023-11-08T19:27:07.190800+0000","last_became_peered":"2023-11-08T19:27:07.190800+0000","last_unstale":"2023-11-08T19:28:02.700897+0000","last_undegraded":"2023-11-08T19:28:02.700897+0000","last_fullsized":"2023-11-08T19:28:02.700897+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:46:30.452678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656099999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.696596+0000","last_change":"2023-11-08T19:27:08.187983+0000","last_active":"2023-11-08T19:28:01.696596+0000","last_peered":"2023-11-08T19:28:01.696596+0000","last_clean":"2023-11-08T19:28:01.696596+0000","last_became_active":"2023-11-08T19:27:08.187322+0000","last_became_peered":"2023-11-08T19:27:08.187322+0000","last_unstale":"2023-11-08T19:28:01.696596+0000","last_undegraded":"2023-11-08T19:28:01.696596+0000","last_fullsized":"2023-11-08T19:28:01.696596+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:33:07.647519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.704461+0000","last_change":"2023-11-08T19:28:02.704461+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.704461+0000","last_undegraded":"2023-11-08T19:28:02.704461+0000","last_fullsized":"2023-11-08T19:28:02.704461+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.725558+0000","last_change":"2023-11-08T19:28:02.720466+0000","last_active":"2023-11-08T19:28:02.725558+0000","last_peered":"2023-11-08T19:28:02.725558+0000","last_clean":"2023-11-08T19:28:02.725558+0000","last_became_active":"2023-11-08T19:28:02.720193+0000","last_became_peered":"2023-11-08T19:28:02.720193+0000","last_unstale":"2023-11-08T19:28:02.725558+0000","last_undegraded":"2023-11-08T19:28:02.725558+0000","last_fullsized":"2023-11-08T19:28:02.725558+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:26:23.134605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":58,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.694087+0000","last_change":"2023-11-08T19:26:06.394075+0000","last_active":"2023-11-08T19:28:01.694087+0000","last_peered":"2023-11-08T19:28:01.694087+0000","last_clean":"2023-11-08T19:28:01.694087+0000","last_became_active":"2023-11-08T19:26:04.606967+0000","last_became_peered":"2023-11-08T19:26:04.606967+0000","last_unstale":"2023-11-08T19:28:01.694087+0000","last_undegraded":"2023-11-08T19:28:01.694087+0000","last_fullsized":"2023-11-08T19:28:01.694087+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:19:56.286835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040971999999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.701127+0000","last_change":"2023-11-08T19:27:09.180722+0000","last_active":"2023-11-08T19:28:02.701127+0000","last_peered":"2023-11-08T19:28:02.701127+0000","last_clean":"2023-11-08T19:28:02.701127+0000","last_became_active":"2023-11-08T19:27:07.186977+0000","last_became_peered":"2023-11-08T19:27:07.186977+0000","last_unstale":"2023-11-08T19:28:02.701127+0000","last_undegraded":"2023-11-08T19:28:02.701127+0000","last_fullsized":"2023-11-08T19:28:02.701127+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:17:59.966463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057675100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.694729+0000","last_change":"2023-11-08T19:27:08.193963+0000","last_active":"2023-11-08T19:28:01.694729+0000","last_peered":"2023-11-08T19:28:01.694729+0000","last_clean":"2023-11-08T19:28:01.694729+0000","last_became_active":"2023-11-08T19:27:08.193098+0000","last_became_peered":"2023-11-08T19:27:08.193098+0000","last_unstale":"2023-11-08T19:28:01.694729+0000","last_undegraded":"2023-11-08T19:28:01.694729+0000","last_fullsized":"2023-11-08T19:28:01.694729+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:41:09.827370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.699774+0000","last_change":"2023-11-08T19:28:02.699774+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.699774+0000","last_undegraded":"2023-11-08T19:28:02.699774+0000","last_fullsized":"2023-11-08T19:28:02.699774+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.722618+0000","last_change":"2023-11-08T19:28:02.718059+0000","last_active":"2023-11-08T19:28:02.722618+0000","last_peered":"2023-11-08T19:28:02.722618+0000","last_clean":"2023-11-08T19:28:02.722618+0000","last_became_active":"2023-11-08T19:28:02.717775+0000","last_became_peered":"2023-11-08T19:28:02.717775+0000","last_unstale":"2023-11-08T19:28:02.722618+0000","last_undegraded":"2023-11-08T19:28:02.722618+0000","last_fullsized":"2023-11-08T19:28:02.722618+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:29:25.082011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":58,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.697065+0000","last_change":"2023-11-08T19:26:06.265686+0000","last_active":"2023-11-08T19:28:01.697065+0000","last_peered":"2023-11-08T19:28:01.697065+0000","last_clean":"2023-11-08T19:28:01.697065+0000","last_became_active":"2023-11-08T19:26:04.602111+0000","last_became_peered":"2023-11-08T19:26:04.602111+0000","last_unstale":"2023-11-08T19:28:01.697065+0000","last_undegraded":"2023-11-08T19:28:01.697065+0000","last_fullsized":"2023-11-08T19:28:01.697065+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:40:11.164951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056197899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.701328+0000","last_change":"2023-11-08T19:27:09.181333+0000","last_active":"2023-11-08T19:28:02.701328+0000","last_peered":"2023-11-08T19:28:02.701328+0000","last_clean":"2023-11-08T19:28:02.701328+0000","last_became_active":"2023-11-08T19:27:07.188657+0000","last_became_peered":"2023-11-08T19:27:07.188657+0000","last_unstale":"2023-11-08T19:28:02.701328+0000","last_undegraded":"2023-11-08T19:28:02.701328+0000","last_fullsized":"2023-11-08T19:28:02.701328+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:25:56.788615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001227393,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.701395+0000","last_change":"2023-11-08T19:27:08.188533+0000","last_active":"2023-11-08T19:28:02.701395+0000","last_peered":"2023-11-08T19:28:02.701395+0000","last_clean":"2023-11-08T19:28:02.701395+0000","last_became_active":"2023-11-08T19:27:08.188240+0000","last_became_peered":"2023-11-08T19:27:08.188240+0000","last_unstale":"2023-11-08T19:28:02.701395+0000","last_undegraded":"2023-11-08T19:28:02.701395+0000","last_fullsized":"2023-11-08T19:28:02.701395+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:55:37.653521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.701682+0000","last_change":"2023-11-08T19:28:02.701682+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.701682+0000","last_undegraded":"2023-11-08T19:28:02.701682+0000","last_fullsized":"2023-11-08T19:28:02.701682+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.714684+0000","last_change":"2023-11-08T19:28:02.710864+0000","last_active":"2023-11-08T19:28:02.714684+0000","last_peered":"2023-11-08T19:28:02.714684+0000","last_clean":"2023-11-08T19:28:02.714684+0000","last_became_active":"2023-11-08T19:28:02.710597+0000","last_became_peered":"2023-11-08T19:28:02.710597+0000","last_unstale":"2023-11-08T19:28:02.714684+0000","last_undegraded":"2023-11-08T19:28:02.714684+0000","last_fullsized":"2023-11-08T19:28:02.714684+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:02:43.242472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.702305+0000","last_change":"2023-11-08T19:27:08.188458+0000","last_active":"2023-11-08T19:28:02.702305+0000","last_peered":"2023-11-08T19:28:02.702305+0000","last_clean":"2023-11-08T19:28:02.702305+0000","last_became_active":"2023-11-08T19:27:08.187880+0000","last_became_peered":"2023-11-08T19:27:08.187880+0000","last_unstale":"2023-11-08T19:28:02.702305+0000","last_undegraded":"2023-11-08T19:28:02.702305+0000","last_fullsized":"2023-11-08T19:28:02.702305+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:25:28.618214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":59,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.690530+0000","last_change":"2023-11-08T19:26:05.606898+0000","last_active":"2023-11-08T19:28:01.690530+0000","last_peered":"2023-11-08T19:28:01.690530+0000","last_clean":"2023-11-08T19:28:01.690530+0000","last_became_active":"2023-11-08T19:26:04.603955+0000","last_became_peered":"2023-11-08T19:26:04.603955+0000","last_unstale":"2023-11-08T19:28:01.690530+0000","last_undegraded":"2023-11-08T19:28:01.690530+0000","last_fullsized":"2023-11-08T19:28:01.690530+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:19:11.589559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036473499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.690241+0000","last_change":"2023-11-08T19:27:09.178770+0000","last_active":"2023-11-08T19:28:01.690241+0000","last_peered":"2023-11-08T19:28:01.690241+0000","last_clean":"2023-11-08T19:28:01.690241+0000","last_became_active":"2023-11-08T19:27:07.194992+0000","last_became_peered":"2023-11-08T19:27:07.194992+0000","last_unstale":"2023-11-08T19:28:01.690241+0000","last_undegraded":"2023-11-08T19:28:01.690241+0000","last_fullsized":"2023-11-08T19:28:01.690241+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:35:34.026910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000770285,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.697682+0000","last_change":"2023-11-08T19:28:02.697682+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.697682+0000","last_undegraded":"2023-11-08T19:28:02.697682+0000","last_fullsized":"2023-11-08T19:28:02.697682+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.703765+0000","last_change":"2023-11-08T19:28:02.707011+0000","last_active":"2023-11-08T19:28:03.703765+0000","last_peered":"2023-11-08T19:28:03.703765+0000","last_clean":"2023-11-08T19:28:03.703765+0000","last_became_active":"2023-11-08T19:28:02.706729+0000","last_became_peered":"2023-11-08T19:28:02.706729+0000","last_unstale":"2023-11-08T19:28:03.703765+0000","last_undegraded":"2023-11-08T19:28:03.703765+0000","last_fullsized":"2023-11-08T19:28:03.703765+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:29:25.082011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.695698+0000","last_change":"2023-11-08T19:27:08.193953+0000","last_active":"2023-11-08T19:28:01.695698+0000","last_peered":"2023-11-08T19:28:01.695698+0000","last_clean":"2023-11-08T19:28:01.695698+0000","last_became_active":"2023-11-08T19:27:08.193653+0000","last_became_peered":"2023-11-08T19:27:08.193653+0000","last_unstale":"2023-11-08T19:28:01.695698+0000","last_undegraded":"2023-11-08T19:28:01.695698+0000","last_fullsized":"2023-11-08T19:28:01.695698+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:35:30.073885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":64,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.694881+0000","last_change":"2023-11-08T19:26:06.424464+0000","last_active":"2023-11-08T19:28:01.694881+0000","last_peered":"2023-11-08T19:28:01.694881+0000","last_clean":"2023-11-08T19:28:01.694881+0000","last_became_active":"2023-11-08T19:26:04.603144+0000","last_became_peered":"2023-11-08T19:26:04.603144+0000","last_unstale":"2023-11-08T19:28:01.694881+0000","last_undegraded":"2023-11-08T19:28:01.694881+0000","last_fullsized":"2023-11-08T19:28:01.694881+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:42:55.269365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054090799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.701001+0000","last_change":"2023-11-08T19:27:09.180479+0000","last_active":"2023-11-08T19:28:02.701001+0000","last_peered":"2023-11-08T19:28:02.701001+0000","last_clean":"2023-11-08T19:28:02.701001+0000","last_became_active":"2023-11-08T19:27:07.190438+0000","last_became_peered":"2023-11-08T19:27:07.190438+0000","last_unstale":"2023-11-08T19:28:02.701001+0000","last_undegraded":"2023-11-08T19:28:02.701001+0000","last_fullsized":"2023-11-08T19:28:02.701001+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:25:14.487524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050893300000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.704857+0000","last_change":"2023-11-08T19:28:02.704857+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.704857+0000","last_undegraded":"2023-11-08T19:28:02.704857+0000","last_fullsized":"2023-11-08T19:28:02.704857+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.715287+0000","last_change":"2023-11-08T19:28:02.708336+0000","last_active":"2023-11-08T19:28:02.715287+0000","last_peered":"2023-11-08T19:28:02.715287+0000","last_clean":"2023-11-08T19:28:02.715287+0000","last_became_active":"2023-11-08T19:28:02.708016+0000","last_became_peered":"2023-11-08T19:28:02.708016+0000","last_unstale":"2023-11-08T19:28:02.715287+0000","last_undegraded":"2023-11-08T19:28:02.715287+0000","last_fullsized":"2023-11-08T19:28:02.715287+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:39:02.748118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.696341+0000","last_change":"2023-11-08T19:27:08.190530+0000","last_active":"2023-11-08T19:28:01.696341+0000","last_peered":"2023-11-08T19:28:01.696341+0000","last_clean":"2023-11-08T19:28:01.696341+0000","last_became_active":"2023-11-08T19:27:08.190203+0000","last_became_peered":"2023-11-08T19:27:08.190203+0000","last_unstale":"2023-11-08T19:28:01.696341+0000","last_undegraded":"2023-11-08T19:28:01.696341+0000","last_fullsized":"2023-11-08T19:28:01.696341+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:26:32.777602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":60,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.698474+0000","last_change":"2023-11-08T19:26:06.541623+0000","last_active":"2023-11-08T19:28:01.698474+0000","last_peered":"2023-11-08T19:28:01.698474+0000","last_clean":"2023-11-08T19:28:01.698474+0000","last_became_active":"2023-11-08T19:26:04.601961+0000","last_became_peered":"2023-11-08T19:26:04.601961+0000","last_unstale":"2023-11-08T19:28:01.698474+0000","last_undegraded":"2023-11-08T19:28:01.698474+0000","last_fullsized":"2023-11-08T19:28:01.698474+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:11:26.508634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000466386,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.694709+0000","last_change":"2023-11-08T19:27:09.431231+0000","last_active":"2023-11-08T19:28:01.694709+0000","last_peered":"2023-11-08T19:28:01.694709+0000","last_clean":"2023-11-08T19:28:01.694709+0000","last_became_active":"2023-11-08T19:27:07.191583+0000","last_became_peered":"2023-11-08T19:27:07.191583+0000","last_unstale":"2023-11-08T19:28:01.694709+0000","last_undegraded":"2023-11-08T19:28:01.694709+0000","last_fullsized":"2023-11-08T19:28:01.694709+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:58:48.357266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093695200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.698528+0000","last_change":"2023-11-08T19:28:02.698528+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.698528+0000","last_undegraded":"2023-11-08T19:28:02.698528+0000","last_fullsized":"2023-11-08T19:28:02.698528+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"91'1","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.717319+0000","last_change":"2023-11-08T19:28:02.708326+0000","last_active":"2023-11-08T19:28:03.717319+0000","last_peered":"2023-11-08T19:28:03.717319+0000","last_clean":"2023-11-08T19:28:03.717319+0000","last_became_active":"2023-11-08T19:28:02.708055+0000","last_became_peered":"2023-11-08T19:28:02.708055+0000","last_unstale":"2023-11-08T19:28:03.717319+0000","last_undegraded":"2023-11-08T19:28:03.717319+0000","last_fullsized":"2023-11-08T19:28:03.717319+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:50:41.723104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.694787+0000","last_change":"2023-11-08T19:27:08.187900+0000","last_active":"2023-11-08T19:28:01.694787+0000","last_peered":"2023-11-08T19:28:01.694787+0000","last_clean":"2023-11-08T19:28:01.694787+0000","last_became_active":"2023-11-08T19:27:08.187626+0000","last_became_peered":"2023-11-08T19:27:08.187626+0000","last_unstale":"2023-11-08T19:28:01.694787+0000","last_undegraded":"2023-11-08T19:28:01.694787+0000","last_fullsized":"2023-11-08T19:28:01.694787+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:00:55.878397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":71,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.701751+0000","last_change":"2023-11-08T19:26:05.608980+0000","last_active":"2023-11-08T19:28:02.701751+0000","last_peered":"2023-11-08T19:28:02.701751+0000","last_clean":"2023-11-08T19:28:02.701751+0000","last_became_active":"2023-11-08T19:26:04.606386+0000","last_became_peered":"2023-11-08T19:26:04.606386+0000","last_unstale":"2023-11-08T19:28:02.701751+0000","last_undegraded":"2023-11-08T19:28:02.701751+0000","last_fullsized":"2023-11-08T19:28:02.701751+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:41:50.771457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037950599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.690460+0000","last_change":"2023-11-08T19:27:09.179037+0000","last_active":"2023-11-08T19:28:01.690460+0000","last_peered":"2023-11-08T19:28:01.690460+0000","last_clean":"2023-11-08T19:28:01.690460+0000","last_became_active":"2023-11-08T19:27:07.195033+0000","last_became_peered":"2023-11-08T19:27:07.195033+0000","last_unstale":"2023-11-08T19:28:01.690460+0000","last_undegraded":"2023-11-08T19:28:01.690460+0000","last_fullsized":"2023-11-08T19:28:01.690460+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:19:08.253483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041824800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.697868+0000","last_change":"2023-11-08T19:28:02.697868+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.697868+0000","last_undegraded":"2023-11-08T19:28:02.697868+0000","last_fullsized":"2023-11-08T19:28:02.697868+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.713986+0000","last_change":"2023-11-08T19:28:02.712619+0000","last_active":"2023-11-08T19:28:02.713986+0000","last_peered":"2023-11-08T19:28:02.713986+0000","last_clean":"2023-11-08T19:28:02.713986+0000","last_became_active":"2023-11-08T19:28:02.712342+0000","last_became_peered":"2023-11-08T19:28:02.712342+0000","last_unstale":"2023-11-08T19:28:02.713986+0000","last_undegraded":"2023-11-08T19:28:02.713986+0000","last_fullsized":"2023-11-08T19:28:02.713986+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:11:10.165281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.693976+0000","last_change":"2023-11-08T19:27:08.193761+0000","last_active":"2023-11-08T19:28:01.693976+0000","last_peered":"2023-11-08T19:28:01.693976+0000","last_clean":"2023-11-08T19:28:01.693976+0000","last_became_active":"2023-11-08T19:27:08.193017+0000","last_became_peered":"2023-11-08T19:27:08.193017+0000","last_unstale":"2023-11-08T19:28:01.693976+0000","last_undegraded":"2023-11-08T19:28:01.693976+0000","last_fullsized":"2023-11-08T19:28:01.693976+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:18:00.971466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":64,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.694700+0000","last_change":"2023-11-08T19:26:06.424073+0000","last_active":"2023-11-08T19:28:01.694700+0000","last_peered":"2023-11-08T19:28:01.694700+0000","last_clean":"2023-11-08T19:28:01.694700+0000","last_became_active":"2023-11-08T19:26:04.599046+0000","last_became_peered":"2023-11-08T19:26:04.599046+0000","last_unstale":"2023-11-08T19:28:01.694700+0000","last_undegraded":"2023-11-08T19:28:01.694700+0000","last_fullsized":"2023-11-08T19:28:01.694700+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:04:43.562566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035092199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.700565+0000","last_change":"2023-11-08T19:27:09.181716+0000","last_active":"2023-11-08T19:28:02.700565+0000","last_peered":"2023-11-08T19:28:02.700565+0000","last_clean":"2023-11-08T19:28:02.700565+0000","last_became_active":"2023-11-08T19:27:07.187077+0000","last_became_peered":"2023-11-08T19:27:07.187077+0000","last_unstale":"2023-11-08T19:28:02.700565+0000","last_undegraded":"2023-11-08T19:28:02.700565+0000","last_fullsized":"2023-11-08T19:28:02.700565+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:51:29.247104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056673599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.726360+0000","last_change":"2023-11-08T19:28:03.724351+0000","last_active":"2023-11-08T19:28:03.726360+0000","last_peered":"2023-11-08T19:28:03.726360+0000","last_clean":"2023-11-08T19:28:03.726360+0000","last_became_active":"2023-11-08T19:28:03.724109+0000","last_became_peered":"2023-11-08T19:28:03.724109+0000","last_unstale":"2023-11-08T19:28:03.726360+0000","last_undegraded":"2023-11-08T19:28:03.726360+0000","last_fullsized":"2023-11-08T19:28:03.726360+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:11:11.170102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.716963+0000","last_change":"2023-11-08T19:28:02.711594+0000","last_active":"2023-11-08T19:28:02.716963+0000","last_peered":"2023-11-08T19:28:02.716963+0000","last_clean":"2023-11-08T19:28:02.716963+0000","last_became_active":"2023-11-08T19:28:02.711204+0000","last_became_peered":"2023-11-08T19:28:02.711204+0000","last_unstale":"2023-11-08T19:28:02.716963+0000","last_undegraded":"2023-11-08T19:28:02.716963+0000","last_fullsized":"2023-11-08T19:28:02.716963+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:21:23.673336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.696597+0000","last_change":"2023-11-08T19:27:08.188143+0000","last_active":"2023-11-08T19:28:01.696597+0000","last_peered":"2023-11-08T19:28:01.696597+0000","last_clean":"2023-11-08T19:28:01.696597+0000","last_became_active":"2023-11-08T19:27:08.187714+0000","last_became_peered":"2023-11-08T19:27:08.187714+0000","last_unstale":"2023-11-08T19:28:01.696597+0000","last_undegraded":"2023-11-08T19:28:01.696597+0000","last_fullsized":"2023-11-08T19:28:01.696597+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:02:10.863403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":58,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.693970+0000","last_change":"2023-11-08T19:26:06.394320+0000","last_active":"2023-11-08T19:28:01.693970+0000","last_peered":"2023-11-08T19:28:01.693970+0000","last_clean":"2023-11-08T19:28:01.693970+0000","last_became_active":"2023-11-08T19:26:04.603212+0000","last_became_peered":"2023-11-08T19:26:04.603212+0000","last_unstale":"2023-11-08T19:28:01.693970+0000","last_undegraded":"2023-11-08T19:28:01.693970+0000","last_fullsized":"2023-11-08T19:28:01.693970+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:25:29.947079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054695399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.700819+0000","last_change":"2023-11-08T19:27:09.181613+0000","last_active":"2023-11-08T19:28:02.700819+0000","last_peered":"2023-11-08T19:28:02.700819+0000","last_clean":"2023-11-08T19:28:02.700819+0000","last_became_active":"2023-11-08T19:27:07.190897+0000","last_became_peered":"2023-11-08T19:27:07.190897+0000","last_unstale":"2023-11-08T19:28:02.700819+0000","last_undegraded":"2023-11-08T19:28:02.700819+0000","last_fullsized":"2023-11-08T19:28:02.700819+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:27:57.388125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011172459999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.698922+0000","last_change":"2023-11-08T19:28:02.698922+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.698922+0000","last_undegraded":"2023-11-08T19:28:02.698922+0000","last_fullsized":"2023-11-08T19:28:02.698922+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.711981+0000","last_change":"2023-11-08T19:28:02.707358+0000","last_active":"2023-11-08T19:28:02.711981+0000","last_peered":"2023-11-08T19:28:02.711981+0000","last_clean":"2023-11-08T19:28:02.711981+0000","last_became_active":"2023-11-08T19:28:02.707109+0000","last_became_peered":"2023-11-08T19:28:02.707109+0000","last_unstale":"2023-11-08T19:28:02.711981+0000","last_undegraded":"2023-11-08T19:28:02.711981+0000","last_fullsized":"2023-11-08T19:28:02.711981+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:32:42.691100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.696667+0000","last_change":"2023-11-08T19:27:08.188668+0000","last_active":"2023-11-08T19:28:01.696667+0000","last_peered":"2023-11-08T19:28:01.696667+0000","last_clean":"2023-11-08T19:28:01.696667+0000","last_became_active":"2023-11-08T19:27:08.188441+0000","last_became_peered":"2023-11-08T19:27:08.188441+0000","last_unstale":"2023-11-08T19:28:01.696667+0000","last_undegraded":"2023-11-08T19:28:01.696667+0000","last_fullsized":"2023-11-08T19:28:01.696667+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:45:44.379160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":64,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.694995+0000","last_change":"2023-11-08T19:26:06.423754+0000","last_active":"2023-11-08T19:28:01.694995+0000","last_peered":"2023-11-08T19:28:01.694995+0000","last_clean":"2023-11-08T19:28:01.694995+0000","last_became_active":"2023-11-08T19:26:04.599322+0000","last_became_peered":"2023-11-08T19:26:04.599322+0000","last_unstale":"2023-11-08T19:28:01.694995+0000","last_undegraded":"2023-11-08T19:28:01.694995+0000","last_fullsized":"2023-11-08T19:28:01.694995+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:13:08.689128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042670000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.701576+0000","last_change":"2023-11-08T19:27:09.181656+0000","last_active":"2023-11-08T19:28:02.701576+0000","last_peered":"2023-11-08T19:28:02.701576+0000","last_clean":"2023-11-08T19:28:02.701576+0000","last_became_active":"2023-11-08T19:27:07.187337+0000","last_became_peered":"2023-11-08T19:27:07.187337+0000","last_unstale":"2023-11-08T19:28:02.701576+0000","last_undegraded":"2023-11-08T19:28:02.701576+0000","last_fullsized":"2023-11-08T19:28:02.701576+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:14:17.605057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000346967,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.698224+0000","last_change":"2023-11-08T19:28:02.698224+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.698224+0000","last_undegraded":"2023-11-08T19:28:02.698224+0000","last_fullsized":"2023-11-08T19:28:02.698224+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.712011+0000","last_change":"2023-11-08T19:28:02.704760+0000","last_active":"2023-11-08T19:28:02.712011+0000","last_peered":"2023-11-08T19:28:02.712011+0000","last_clean":"2023-11-08T19:28:02.712011+0000","last_became_active":"2023-11-08T19:28:02.704410+0000","last_became_peered":"2023-11-08T19:28:02.704410+0000","last_unstale":"2023-11-08T19:28:02.712011+0000","last_undegraded":"2023-11-08T19:28:02.712011+0000","last_fullsized":"2023-11-08T19:28:02.712011+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:00:10.329151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.694901+0000","last_change":"2023-11-08T19:27:08.194065+0000","last_active":"2023-11-08T19:28:01.694901+0000","last_peered":"2023-11-08T19:28:01.694901+0000","last_clean":"2023-11-08T19:28:01.694901+0000","last_became_active":"2023-11-08T19:27:08.193532+0000","last_became_peered":"2023-11-08T19:27:08.193532+0000","last_unstale":"2023-11-08T19:28:01.694901+0000","last_undegraded":"2023-11-08T19:28:01.694901+0000","last_fullsized":"2023-11-08T19:28:01.694901+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:27:58.393129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":58,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.695354+0000","last_change":"2023-11-08T19:26:06.394488+0000","last_active":"2023-11-08T19:28:01.695354+0000","last_peered":"2023-11-08T19:28:01.695354+0000","last_clean":"2023-11-08T19:28:01.695354+0000","last_became_active":"2023-11-08T19:26:04.606805+0000","last_became_peered":"2023-11-08T19:26:04.606805+0000","last_unstale":"2023-11-08T19:28:01.695354+0000","last_undegraded":"2023-11-08T19:28:01.695354+0000","last_fullsized":"2023-11-08T19:28:01.695354+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:25:20.857133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045317000000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.696700+0000","last_change":"2023-11-08T19:27:09.211647+0000","last_active":"2023-11-08T19:28:01.696700+0000","last_peered":"2023-11-08T19:28:01.696700+0000","last_clean":"2023-11-08T19:28:01.696700+0000","last_became_active":"2023-11-08T19:27:07.190309+0000","last_became_peered":"2023-11-08T19:27:07.190309+0000","last_unstale":"2023-11-08T19:28:01.696700+0000","last_undegraded":"2023-11-08T19:28:01.696700+0000","last_fullsized":"2023-11-08T19:28:01.696700+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:56:04.292154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047615199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.711664+0000","last_change":"2023-11-08T19:28:02.711664+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.711664+0000","last_undegraded":"2023-11-08T19:28:02.711664+0000","last_fullsized":"2023-11-08T19:28:02.711664+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.722917+0000","last_change":"2023-11-08T19:28:02.717508+0000","last_active":"2023-11-08T19:28:02.722917+0000","last_peered":"2023-11-08T19:28:02.722917+0000","last_clean":"2023-11-08T19:28:02.722917+0000","last_became_active":"2023-11-08T19:28:02.717126+0000","last_became_peered":"2023-11-08T19:28:02.717126+0000","last_unstale":"2023-11-08T19:28:02.722917+0000","last_undegraded":"2023-11-08T19:28:02.722917+0000","last_fullsized":"2023-11-08T19:28:02.722917+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:23:34.816577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.695560+0000","last_change":"2023-11-08T19:27:08.194884+0000","last_active":"2023-11-08T19:28:01.695560+0000","last_peered":"2023-11-08T19:28:01.695560+0000","last_clean":"2023-11-08T19:28:01.695560+0000","last_became_active":"2023-11-08T19:27:08.193241+0000","last_became_peered":"2023-11-08T19:27:08.193241+0000","last_unstale":"2023-11-08T19:28:01.695560+0000","last_undegraded":"2023-11-08T19:28:01.695560+0000","last_fullsized":"2023-11-08T19:28:01.695560+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:46:31.457682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":58,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.696871+0000","last_change":"2023-11-08T19:26:06.265907+0000","last_active":"2023-11-08T19:28:01.696871+0000","last_peered":"2023-11-08T19:28:01.696871+0000","last_clean":"2023-11-08T19:28:01.696871+0000","last_became_active":"2023-11-08T19:26:04.604772+0000","last_became_peered":"2023-11-08T19:26:04.604772+0000","last_unstale":"2023-11-08T19:28:01.696871+0000","last_undegraded":"2023-11-08T19:28:01.696871+0000","last_fullsized":"2023-11-08T19:28:01.696871+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:44:16.139794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066875300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.698318+0000","last_change":"2023-11-08T19:27:09.282699+0000","last_active":"2023-11-08T19:28:01.698318+0000","last_peered":"2023-11-08T19:28:01.698318+0000","last_clean":"2023-11-08T19:28:01.698318+0000","last_became_active":"2023-11-08T19:27:07.191212+0000","last_became_peered":"2023-11-08T19:27:07.191212+0000","last_unstale":"2023-11-08T19:28:01.698318+0000","last_undegraded":"2023-11-08T19:28:01.698318+0000","last_fullsized":"2023-11-08T19:28:01.698318+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:35:14.715110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046640800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.705457+0000","last_change":"2023-11-08T19:28:02.705457+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.705457+0000","last_undegraded":"2023-11-08T19:28:02.705457+0000","last_fullsized":"2023-11-08T19:28:02.705457+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.717068+0000","last_change":"2023-11-08T19:28:02.712356+0000","last_active":"2023-11-08T19:28:02.717068+0000","last_peered":"2023-11-08T19:28:02.717068+0000","last_clean":"2023-11-08T19:28:02.717068+0000","last_became_active":"2023-11-08T19:28:02.712073+0000","last_became_peered":"2023-11-08T19:28:02.712073+0000","last_unstale":"2023-11-08T19:28:02.717068+0000","last_undegraded":"2023-11-08T19:28:02.717068+0000","last_fullsized":"2023-11-08T19:28:02.717068+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:06:46.912027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.696801+0000","last_change":"2023-11-08T19:27:08.187644+0000","last_active":"2023-11-08T19:28:01.696801+0000","last_peered":"2023-11-08T19:28:01.696801+0000","last_clean":"2023-11-08T19:28:01.696801+0000","last_became_active":"2023-11-08T19:27:08.187376+0000","last_became_peered":"2023-11-08T19:27:08.187376+0000","last_unstale":"2023-11-08T19:28:01.696801+0000","last_undegraded":"2023-11-08T19:28:01.696801+0000","last_fullsized":"2023-11-08T19:28:01.696801+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:22:40.300185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":59,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.690307+0000","last_change":"2023-11-08T19:26:05.606946+0000","last_active":"2023-11-08T19:28:01.690307+0000","last_peered":"2023-11-08T19:28:01.690307+0000","last_clean":"2023-11-08T19:28:01.690307+0000","last_became_active":"2023-11-08T19:26:04.598775+0000","last_became_peered":"2023-11-08T19:26:04.598775+0000","last_unstale":"2023-11-08T19:28:01.690307+0000","last_undegraded":"2023-11-08T19:28:01.690307+0000","last_fullsized":"2023-11-08T19:28:01.690307+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:20:56.898927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049870399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.693887+0000","last_change":"2023-11-08T19:27:09.331646+0000","last_active":"2023-11-08T19:28:01.693887+0000","last_peered":"2023-11-08T19:28:01.693887+0000","last_clean":"2023-11-08T19:28:01.693887+0000","last_became_active":"2023-11-08T19:27:07.190619+0000","last_became_peered":"2023-11-08T19:27:07.190619+0000","last_unstale":"2023-11-08T19:28:01.693887+0000","last_undegraded":"2023-11-08T19:28:01.693887+0000","last_fullsized":"2023-11-08T19:28:01.693887+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:42:47.371458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065918400000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.702920+0000","last_change":"2023-11-08T19:28:02.702920+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.702920+0000","last_undegraded":"2023-11-08T19:28:02.702920+0000","last_fullsized":"2023-11-08T19:28:02.702920+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.719716+0000","last_change":"2023-11-08T19:28:02.710012+0000","last_active":"2023-11-08T19:28:02.719716+0000","last_peered":"2023-11-08T19:28:02.719716+0000","last_clean":"2023-11-08T19:28:02.719716+0000","last_became_active":"2023-11-08T19:28:02.709715+0000","last_became_peered":"2023-11-08T19:28:02.709715+0000","last_unstale":"2023-11-08T19:28:02.719716+0000","last_undegraded":"2023-11-08T19:28:02.719716+0000","last_fullsized":"2023-11-08T19:28:02.719716+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:23:11.877422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.703535+0000","last_change":"2023-11-08T19:27:08.183798+0000","last_active":"2023-11-08T19:28:02.703535+0000","last_peered":"2023-11-08T19:28:02.703535+0000","last_clean":"2023-11-08T19:28:02.703535+0000","last_became_active":"2023-11-08T19:27:08.183537+0000","last_became_peered":"2023-11-08T19:27:08.183537+0000","last_unstale":"2023-11-08T19:28:02.703535+0000","last_undegraded":"2023-11-08T19:28:02.703535+0000","last_fullsized":"2023-11-08T19:28:02.703535+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:50:39.026265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":71,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.701309+0000","last_change":"2023-11-08T19:26:05.610012+0000","last_active":"2023-11-08T19:28:02.701309+0000","last_peered":"2023-11-08T19:28:02.701309+0000","last_clean":"2023-11-08T19:28:02.701309+0000","last_became_active":"2023-11-08T19:26:04.603209+0000","last_became_peered":"2023-11-08T19:26:04.603209+0000","last_unstale":"2023-11-08T19:28:02.701309+0000","last_undegraded":"2023-11-08T19:28:02.701309+0000","last_fullsized":"2023-11-08T19:28:02.701309+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:35:02.306997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053335100000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.690186+0000","last_change":"2023-11-08T19:27:09.178405+0000","last_active":"2023-11-08T19:28:01.690186+0000","last_peered":"2023-11-08T19:28:01.690186+0000","last_clean":"2023-11-08T19:28:01.690186+0000","last_became_active":"2023-11-08T19:27:07.188169+0000","last_became_peered":"2023-11-08T19:27:07.188169+0000","last_unstale":"2023-11-08T19:28:01.690186+0000","last_undegraded":"2023-11-08T19:28:01.690186+0000","last_fullsized":"2023-11-08T19:28:01.690186+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:39:25.855467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056209700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.703242+0000","last_change":"2023-11-08T19:28:02.703242+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.703242+0000","last_undegraded":"2023-11-08T19:28:02.703242+0000","last_fullsized":"2023-11-08T19:28:02.703242+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.722598+0000","last_change":"2023-11-08T19:28:02.716988+0000","last_active":"2023-11-08T19:28:02.722598+0000","last_peered":"2023-11-08T19:28:02.722598+0000","last_clean":"2023-11-08T19:28:02.722598+0000","last_became_active":"2023-11-08T19:28:02.716687+0000","last_became_peered":"2023-11-08T19:28:02.716687+0000","last_unstale":"2023-11-08T19:28:02.722598+0000","last_undegraded":"2023-11-08T19:28:02.722598+0000","last_fullsized":"2023-11-08T19:28:02.722598+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:21:23.673336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.693769+0000","last_change":"2023-11-08T19:27:08.193508+0000","last_active":"2023-11-08T19:28:01.693769+0000","last_peered":"2023-11-08T19:28:01.693769+0000","last_clean":"2023-11-08T19:28:01.693769+0000","last_became_active":"2023-11-08T19:27:08.193037+0000","last_became_peered":"2023-11-08T19:27:08.193037+0000","last_unstale":"2023-11-08T19:28:01.693769+0000","last_undegraded":"2023-11-08T19:28:01.693769+0000","last_fullsized":"2023-11-08T19:28:01.693769+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:29:50.067887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":58,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.696488+0000","last_change":"2023-11-08T19:26:06.265318+0000","last_active":"2023-11-08T19:28:01.696488+0000","last_peered":"2023-11-08T19:28:01.696488+0000","last_clean":"2023-11-08T19:28:01.696488+0000","last_became_active":"2023-11-08T19:26:04.603409+0000","last_became_peered":"2023-11-08T19:26:04.603409+0000","last_unstale":"2023-11-08T19:28:01.696488+0000","last_undegraded":"2023-11-08T19:28:01.696488+0000","last_fullsized":"2023-11-08T19:28:01.696488+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:27:24.933111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065684300000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.701449+0000","last_change":"2023-11-08T19:27:09.182282+0000","last_active":"2023-11-08T19:28:02.701449+0000","last_peered":"2023-11-08T19:28:02.701449+0000","last_clean":"2023-11-08T19:28:02.701449+0000","last_became_active":"2023-11-08T19:27:07.186944+0000","last_became_peered":"2023-11-08T19:27:07.186944+0000","last_unstale":"2023-11-08T19:28:02.701449+0000","last_undegraded":"2023-11-08T19:28:02.701449+0000","last_fullsized":"2023-11-08T19:28:02.701449+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:45:21.079011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00028665599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.720726+0000","last_change":"2023-11-08T19:28:03.716823+0000","last_active":"2023-11-08T19:28:03.720726+0000","last_peered":"2023-11-08T19:28:03.720726+0000","last_clean":"2023-11-08T19:28:03.720726+0000","last_became_active":"2023-11-08T19:28:03.716545+0000","last_became_peered":"2023-11-08T19:28:03.716545+0000","last_unstale":"2023-11-08T19:28:03.720726+0000","last_undegraded":"2023-11-08T19:28:03.720726+0000","last_fullsized":"2023-11-08T19:28:03.720726+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:39:03.752939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.712391+0000","last_change":"2023-11-08T19:28:02.703978+0000","last_active":"2023-11-08T19:28:02.712391+0000","last_peered":"2023-11-08T19:28:02.712391+0000","last_clean":"2023-11-08T19:28:02.712391+0000","last_became_active":"2023-11-08T19:28:02.703654+0000","last_became_peered":"2023-11-08T19:28:02.703654+0000","last_unstale":"2023-11-08T19:28:02.712391+0000","last_undegraded":"2023-11-08T19:28:02.712391+0000","last_fullsized":"2023-11-08T19:28:02.712391+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:39:48.728129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.698064+0000","last_change":"2023-11-08T19:27:08.193990+0000","last_active":"2023-11-08T19:28:01.698064+0000","last_peered":"2023-11-08T19:28:01.698064+0000","last_clean":"2023-11-08T19:28:01.698064+0000","last_became_active":"2023-11-08T19:27:08.193624+0000","last_became_peered":"2023-11-08T19:27:08.193624+0000","last_unstale":"2023-11-08T19:28:01.698064+0000","last_undegraded":"2023-11-08T19:28:01.698064+0000","last_fullsized":"2023-11-08T19:28:01.698064+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:46:41.321120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":64,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.694552+0000","last_change":"2023-11-08T19:26:06.423863+0000","last_active":"2023-11-08T19:28:01.694552+0000","last_peered":"2023-11-08T19:28:01.694552+0000","last_clean":"2023-11-08T19:28:01.694552+0000","last_became_active":"2023-11-08T19:26:04.608639+0000","last_became_peered":"2023-11-08T19:26:04.608639+0000","last_unstale":"2023-11-08T19:28:01.694552+0000","last_undegraded":"2023-11-08T19:28:01.694552+0000","last_fullsized":"2023-11-08T19:28:01.694552+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:04:43.631102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044384999999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.698449+0000","last_change":"2023-11-08T19:27:09.283232+0000","last_active":"2023-11-08T19:28:01.698449+0000","last_peered":"2023-11-08T19:28:01.698449+0000","last_clean":"2023-11-08T19:28:01.698449+0000","last_became_active":"2023-11-08T19:27:07.190931+0000","last_became_peered":"2023-11-08T19:27:07.190931+0000","last_unstale":"2023-11-08T19:28:01.698449+0000","last_undegraded":"2023-11-08T19:28:01.698449+0000","last_fullsized":"2023-11-08T19:28:01.698449+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:43:28.225967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034146799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.713166+0000","last_change":"2023-11-08T19:28:02.713166+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.713166+0000","last_undegraded":"2023-11-08T19:28:02.713166+0000","last_fullsized":"2023-11-08T19:28:02.713166+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.722936+0000","last_change":"2023-11-08T19:28:02.717768+0000","last_active":"2023-11-08T19:28:02.722936+0000","last_peered":"2023-11-08T19:28:02.722936+0000","last_clean":"2023-11-08T19:28:02.722936+0000","last_became_active":"2023-11-08T19:28:02.717553+0000","last_became_peered":"2023-11-08T19:28:02.717553+0000","last_unstale":"2023-11-08T19:28:02.722936+0000","last_undegraded":"2023-11-08T19:28:02.722936+0000","last_fullsized":"2023-11-08T19:28:02.722936+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:50:10.975026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.694667+0000","last_change":"2023-11-08T19:27:08.187252+0000","last_active":"2023-11-08T19:28:01.694667+0000","last_peered":"2023-11-08T19:28:01.694667+0000","last_clean":"2023-11-08T19:28:01.694667+0000","last_became_active":"2023-11-08T19:27:08.186952+0000","last_became_peered":"2023-11-08T19:27:08.186952+0000","last_unstale":"2023-11-08T19:28:01.694667+0000","last_undegraded":"2023-11-08T19:28:01.694667+0000","last_fullsized":"2023-11-08T19:28:01.694667+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:12:52.022374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":58,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.694052+0000","last_change":"2023-11-08T19:26:06.394081+0000","last_active":"2023-11-08T19:28:01.694052+0000","last_peered":"2023-11-08T19:28:01.694052+0000","last_clean":"2023-11-08T19:28:01.694052+0000","last_became_active":"2023-11-08T19:26:04.602137+0000","last_became_peered":"2023-11-08T19:26:04.602137+0000","last_unstale":"2023-11-08T19:28:01.694052+0000","last_undegraded":"2023-11-08T19:28:01.694052+0000","last_fullsized":"2023-11-08T19:28:01.694052+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:05:46.951663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056998200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.690432+0000","last_change":"2023-11-08T19:27:09.178816+0000","last_active":"2023-11-08T19:28:01.690432+0000","last_peered":"2023-11-08T19:28:01.690432+0000","last_clean":"2023-11-08T19:28:01.690432+0000","last_became_active":"2023-11-08T19:27:07.192001+0000","last_became_peered":"2023-11-08T19:27:07.192001+0000","last_unstale":"2023-11-08T19:28:01.690432+0000","last_undegraded":"2023-11-08T19:28:01.690432+0000","last_fullsized":"2023-11-08T19:28:01.690432+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:04:12.481212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045359000000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.698299+0000","last_change":"2023-11-08T19:28:02.698299+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.698299+0000","last_undegraded":"2023-11-08T19:28:02.698299+0000","last_fullsized":"2023-11-08T19:28:02.698299+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.716969+0000","last_change":"2023-11-08T19:28:02.711841+0000","last_active":"2023-11-08T19:28:02.716969+0000","last_peered":"2023-11-08T19:28:02.716969+0000","last_clean":"2023-11-08T19:28:02.716969+0000","last_became_active":"2023-11-08T19:28:02.711455+0000","last_became_peered":"2023-11-08T19:28:02.711455+0000","last_unstale":"2023-11-08T19:28:02.716969+0000","last_undegraded":"2023-11-08T19:28:02.716969+0000","last_fullsized":"2023-11-08T19:28:02.716969+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:29:25.082011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.695522+0000","last_change":"2023-11-08T19:27:08.193836+0000","last_active":"2023-11-08T19:28:01.695522+0000","last_peered":"2023-11-08T19:28:01.695522+0000","last_clean":"2023-11-08T19:28:01.695522+0000","last_became_active":"2023-11-08T19:27:08.193458+0000","last_became_peered":"2023-11-08T19:27:08.193458+0000","last_unstale":"2023-11-08T19:28:01.695522+0000","last_undegraded":"2023-11-08T19:28:01.695522+0000","last_fullsized":"2023-11-08T19:28:01.695522+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:14:18.610061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":60,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.698115+0000","last_change":"2023-11-08T19:26:06.541338+0000","last_active":"2023-11-08T19:28:01.698115+0000","last_peered":"2023-11-08T19:28:01.698115+0000","last_clean":"2023-11-08T19:28:01.698115+0000","last_became_active":"2023-11-08T19:26:04.602183+0000","last_became_peered":"2023-11-08T19:26:04.602183+0000","last_unstale":"2023-11-08T19:28:01.698115+0000","last_undegraded":"2023-11-08T19:28:01.698115+0000","last_fullsized":"2023-11-08T19:28:01.698115+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:06:47.446686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052029099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.701560+0000","last_change":"2023-11-08T19:27:09.181622+0000","last_active":"2023-11-08T19:28:02.701560+0000","last_peered":"2023-11-08T19:28:02.701560+0000","last_clean":"2023-11-08T19:28:02.701560+0000","last_became_active":"2023-11-08T19:27:07.184507+0000","last_became_peered":"2023-11-08T19:27:07.184507+0000","last_unstale":"2023-11-08T19:28:02.701560+0000","last_undegraded":"2023-11-08T19:28:02.701560+0000","last_fullsized":"2023-11-08T19:28:02.701560+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:58:48.357266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051924700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.699401+0000","last_change":"2023-11-08T19:28:02.699401+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.699401+0000","last_undegraded":"2023-11-08T19:28:02.699401+0000","last_fullsized":"2023-11-08T19:28:02.699401+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.717064+0000","last_change":"2023-11-08T19:28:02.713407+0000","last_active":"2023-11-08T19:28:02.717064+0000","last_peered":"2023-11-08T19:28:02.717064+0000","last_clean":"2023-11-08T19:28:02.717064+0000","last_became_active":"2023-11-08T19:28:02.713144+0000","last_became_peered":"2023-11-08T19:28:02.713144+0000","last_unstale":"2023-11-08T19:28:02.717064+0000","last_undegraded":"2023-11-08T19:28:02.717064+0000","last_fullsized":"2023-11-08T19:28:02.717064+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:38:53.800926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.698458+0000","last_change":"2023-11-08T19:27:08.194263+0000","last_active":"2023-11-08T19:28:01.698458+0000","last_peered":"2023-11-08T19:28:01.698458+0000","last_clean":"2023-11-08T19:28:01.698458+0000","last_became_active":"2023-11-08T19:27:08.193938+0000","last_became_peered":"2023-11-08T19:27:08.193938+0000","last_unstale":"2023-11-08T19:28:01.698458+0000","last_undegraded":"2023-11-08T19:28:01.698458+0000","last_fullsized":"2023-11-08T19:28:01.698458+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:40:49.009717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":59,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.711234+0000","last_change":"2023-11-08T19:26:06.265690+0000","last_active":"2023-11-08T19:28:02.711234+0000","last_peered":"2023-11-08T19:28:02.711234+0000","last_clean":"2023-11-08T19:28:02.711234+0000","last_became_active":"2023-11-08T19:26:04.603397+0000","last_became_peered":"2023-11-08T19:26:04.603397+0000","last_unstale":"2023-11-08T19:28:02.711234+0000","last_undegraded":"2023-11-08T19:28:02.711234+0000","last_fullsized":"2023-11-08T19:28:02.711234+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:50:43.827570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053361499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.698140+0000","last_change":"2023-11-08T19:27:09.282750+0000","last_active":"2023-11-08T19:28:01.698140+0000","last_peered":"2023-11-08T19:28:01.698140+0000","last_clean":"2023-11-08T19:28:01.698140+0000","last_became_active":"2023-11-08T19:27:07.189943+0000","last_became_peered":"2023-11-08T19:27:07.189943+0000","last_unstale":"2023-11-08T19:28:01.698140+0000","last_undegraded":"2023-11-08T19:28:01.698140+0000","last_fullsized":"2023-11-08T19:28:01.698140+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:31:16.081058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000398725,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.702677+0000","last_change":"2023-11-08T19:28:02.702677+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.702677+0000","last_undegraded":"2023-11-08T19:28:02.702677+0000","last_fullsized":"2023-11-08T19:28:02.702677+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.724982+0000","last_change":"2023-11-08T19:28:02.715521+0000","last_active":"2023-11-08T19:28:02.724982+0000","last_peered":"2023-11-08T19:28:02.724982+0000","last_clean":"2023-11-08T19:28:02.724982+0000","last_became_active":"2023-11-08T19:28:02.715145+0000","last_became_peered":"2023-11-08T19:28:02.715145+0000","last_unstale":"2023-11-08T19:28:02.724982+0000","last_undegraded":"2023-11-08T19:28:02.724982+0000","last_fullsized":"2023-11-08T19:28:02.724982+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:52:23.978416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.702217+0000","last_change":"2023-11-08T19:27:08.188147+0000","last_active":"2023-11-08T19:28:02.702217+0000","last_peered":"2023-11-08T19:28:02.702217+0000","last_clean":"2023-11-08T19:28:02.702217+0000","last_became_active":"2023-11-08T19:27:08.187487+0000","last_became_peered":"2023-11-08T19:27:08.187487+0000","last_unstale":"2023-11-08T19:28:02.702217+0000","last_undegraded":"2023-11-08T19:28:02.702217+0000","last_fullsized":"2023-11-08T19:28:02.702217+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:22:40.300185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":64,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.696443+0000","last_change":"2023-11-08T19:26:06.425133+0000","last_active":"2023-11-08T19:28:01.696443+0000","last_peered":"2023-11-08T19:28:01.696443+0000","last_clean":"2023-11-08T19:28:01.696443+0000","last_became_active":"2023-11-08T19:26:04.601103+0000","last_became_peered":"2023-11-08T19:26:04.601103+0000","last_unstale":"2023-11-08T19:28:01.696443+0000","last_undegraded":"2023-11-08T19:28:01.696443+0000","last_fullsized":"2023-11-08T19:28:01.696443+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:31:14.042561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000229244,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.701647+0000","last_change":"2023-11-08T19:27:09.179852+0000","last_active":"2023-11-08T19:28:02.701647+0000","last_peered":"2023-11-08T19:28:02.701647+0000","last_clean":"2023-11-08T19:28:02.701647+0000","last_became_active":"2023-11-08T19:27:07.191926+0000","last_became_peered":"2023-11-08T19:27:07.191926+0000","last_unstale":"2023-11-08T19:28:02.701647+0000","last_undegraded":"2023-11-08T19:28:02.701647+0000","last_fullsized":"2023-11-08T19:28:02.701647+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:20:38.080129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055457100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.711863+0000","last_change":"2023-11-08T19:28:02.711863+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.711863+0000","last_undegraded":"2023-11-08T19:28:02.711863+0000","last_fullsized":"2023-11-08T19:28:02.711863+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"91'3","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.713504+0000","last_change":"2023-11-08T19:28:02.708691+0000","last_active":"2023-11-08T19:28:03.713504+0000","last_peered":"2023-11-08T19:28:03.713504+0000","last_clean":"2023-11-08T19:28:03.713504+0000","last_became_active":"2023-11-08T19:28:02.708037+0000","last_became_peered":"2023-11-08T19:28:02.708037+0000","last_unstale":"2023-11-08T19:28:03.713504+0000","last_undegraded":"2023-11-08T19:28:03.713504+0000","last_fullsized":"2023-11-08T19:28:03.713504+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:07:33.241500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.702633+0000","last_change":"2023-11-08T19:27:08.186054+0000","last_active":"2023-11-08T19:28:02.702633+0000","last_peered":"2023-11-08T19:28:02.702633+0000","last_clean":"2023-11-08T19:28:02.702633+0000","last_became_active":"2023-11-08T19:27:08.185581+0000","last_became_peered":"2023-11-08T19:27:08.185581+0000","last_unstale":"2023-11-08T19:28:02.702633+0000","last_undegraded":"2023-11-08T19:28:02.702633+0000","last_fullsized":"2023-11-08T19:28:02.702633+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:00:55.878397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":71,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.701211+0000","last_change":"2023-11-08T19:26:05.609625+0000","last_active":"2023-11-08T19:28:02.701211+0000","last_peered":"2023-11-08T19:28:02.701211+0000","last_clean":"2023-11-08T19:28:02.701211+0000","last_became_active":"2023-11-08T19:26:04.606312+0000","last_became_peered":"2023-11-08T19:26:04.606312+0000","last_unstale":"2023-11-08T19:28:02.701211+0000","last_undegraded":"2023-11-08T19:28:02.701211+0000","last_fullsized":"2023-11-08T19:28:02.701211+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:54:42.032901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000505935,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.690085+0000","last_change":"2023-11-08T19:27:09.177985+0000","last_active":"2023-11-08T19:28:01.690085+0000","last_peered":"2023-11-08T19:28:01.690085+0000","last_clean":"2023-11-08T19:28:01.690085+0000","last_became_active":"2023-11-08T19:27:07.193254+0000","last_became_peered":"2023-11-08T19:27:07.193254+0000","last_unstale":"2023-11-08T19:28:01.690085+0000","last_undegraded":"2023-11-08T19:28:01.690085+0000","last_fullsized":"2023-11-08T19:28:01.690085+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:23:38.662629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041830799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.703260+0000","last_change":"2023-11-08T19:28:02.703260+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.703260+0000","last_undegraded":"2023-11-08T19:28:02.703260+0000","last_fullsized":"2023-11-08T19:28:02.703260+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"91'6","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.727979+0000","last_change":"2023-11-08T19:28:02.700819+0000","last_active":"2023-11-08T19:28:03.727979+0000","last_peered":"2023-11-08T19:28:03.727979+0000","last_clean":"2023-11-08T19:28:03.727979+0000","last_became_active":"2023-11-08T19:28:02.700537+0000","last_became_peered":"2023-11-08T19:28:02.700537+0000","last_unstale":"2023-11-08T19:28:03.727979+0000","last_undegraded":"2023-11-08T19:28:03.727979+0000","last_fullsized":"2023-11-08T19:28:03.727979+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:25:00.088013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.703640+0000","last_change":"2023-11-08T19:27:08.188860+0000","last_active":"2023-11-08T19:28:02.703640+0000","last_peered":"2023-11-08T19:28:02.703640+0000","last_clean":"2023-11-08T19:28:02.703640+0000","last_became_active":"2023-11-08T19:27:08.188560+0000","last_became_peered":"2023-11-08T19:27:08.188560+0000","last_unstale":"2023-11-08T19:28:02.703640+0000","last_undegraded":"2023-11-08T19:28:02.703640+0000","last_fullsized":"2023-11-08T19:28:02.703640+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:18:20.969491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":59,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.695852+0000","last_change":"2023-11-08T19:26:06.297255+0000","last_active":"2023-11-08T19:28:01.695852+0000","last_peered":"2023-11-08T19:28:01.695852+0000","last_clean":"2023-11-08T19:28:01.695852+0000","last_became_active":"2023-11-08T19:26:04.603979+0000","last_became_peered":"2023-11-08T19:26:04.603979+0000","last_unstale":"2023-11-08T19:28:01.695852+0000","last_undegraded":"2023-11-08T19:28:01.695852+0000","last_fullsized":"2023-11-08T19:28:01.695852+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:40:11.164951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079032099999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.693851+0000","last_change":"2023-11-08T19:27:09.331990+0000","last_active":"2023-11-08T19:28:01.693851+0000","last_peered":"2023-11-08T19:28:01.693851+0000","last_clean":"2023-11-08T19:28:01.693851+0000","last_became_active":"2023-11-08T19:27:07.185478+0000","last_became_peered":"2023-11-08T19:27:07.185478+0000","last_unstale":"2023-11-08T19:28:01.693851+0000","last_undegraded":"2023-11-08T19:28:01.693851+0000","last_fullsized":"2023-11-08T19:28:01.693851+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:20:38.080129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010661850000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":1794,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12,"ondisk_log_size":12,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574892,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28704,"kb_used_data":1680,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343136,"statfs":{"total":96636764160,"available":96607371264,"internally_reserved":0,"allocated":1720320,"data_stored":1117537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:27:53 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.50600000000000001,"15min":0.498},"min":{"1min":0.34300000000000003,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.72899999999999998,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.437},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.51900000000000002,"15min":0.51500000000000001},"min":{"1min":0.36899999999999999,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.69099999999999995,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.621}]},{"osd":1,"last update":"Wed Nov 8 19:27:53 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56899999999999995,"15min":0.56599999999999995},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.56399999999999995,"15min":0.55600000000000005},"min":{"1min":0.435,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.59199999999999997}]},{"osd":2,"last update":"Wed Nov 8 19:27:53 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.50900000000000001,"15min":0.49299999999999999},"min":{"1min":0.38700000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.55000000000000004,"15min":0.54800000000000004},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.38500000000000001}]},{"osd":3,"last update":"Wed Nov 8 19:27:53 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.55300000000000005,"15min":0.55700000000000005},"min":{"1min":0.375,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.66200000000000003,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59399999999999997,"15min":0.57799999999999996},"min":{"1min":0.49399999999999999,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.77000000000000002,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.5}]},{"osd":4,"last update":"Wed Nov 8 19:27:53 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.50900000000000001,"15min":0.50700000000000001},"min":{"1min":0.36299999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.64500000000000002,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.54600000000000004,"15min":0.54100000000000004},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.41099999999999998}]},{"osd":5,"last update":"Wed Nov 8 19:27:53 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.58599999999999997,"15min":0.58099999999999996},"min":{"1min":0.41199999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.71999999999999997,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.62,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.84499999999999997,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.64300000000000002}]},{"osd":6,"last update":"Wed Nov 8 19:27:53 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.60199999999999998,"15min":0.60399999999999998},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.78400000000000003,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.61499999999999999,"15min":0.61099999999999999},"min":{"1min":0.5,"5min":0.443,"15min":0.443},"max":{"1min":0.82999999999999996,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.57299999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574892,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28132,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343708,"statfs":{"total":96636764160,"available":96607956992,"internally_reserved":0,"allocated":1138688,"data_stored":529225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":1.0780000000000001,"15min":1.319},"min":{"1min":0.33800000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.67800000000000005,"5min":21.359999999999999,"15min":21.359999999999999},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.78300000000000003,"15min":0.88700000000000001},"min":{"1min":0.378,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.73799999999999999,"5min":10.329000000000001,"15min":10.329000000000001},"last":0.434}]},{"osd":1,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.57899999999999996,"15min":0.60899999999999999},"min":{"1min":0.36899999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.71899999999999997,"5min":3.4239999999999999,"15min":3.4239999999999999},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.61699999999999999,"15min":0.63800000000000001},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.73199999999999998,"5min":3.4809999999999999,"15min":3.4809999999999999},"last":0.45800000000000002}]},{"osd":2,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.80100000000000005,"15min":0.90500000000000003},"min":{"1min":0.40999999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.69299999999999995,"5min":10.398,"15min":10.398},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.65400000000000003,"15min":0.68999999999999995},"min":{"1min":0.40799999999999997,"5min":0.318,"15min":0.318},"max":{"1min":0.746,"5min":5.0170000000000003,"15min":5.0170000000000003},"last":0.39300000000000002}]},{"osd":3,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.63900000000000001,"15min":0.66300000000000003},"min":{"1min":0.39500000000000002,"5min":0.317,"15min":0.317},"max":{"1min":0.77300000000000002,"5min":3.5710000000000002,"15min":3.5710000000000002},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.82999999999999996,"15min":0.93799999999999994},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.86799999999999999,"5min":10.368,"15min":10.368},"last":0.56299999999999994}]},{"osd":4,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.53700000000000003,"15min":0.51700000000000002},"min":{"1min":0.38,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.56999999999999995,"15min":0.56200000000000006},"min":{"1min":0.40600000000000003,"5min":0.373,"15min":0.373},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.53300000000000003}]},{"osd":5,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.55000000000000004,"15min":0.53200000000000003},"min":{"1min":0.40699999999999997,"5min":0.307,"15min":0.307},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.503},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.60599999999999998,"15min":0.58599999999999997},"min":{"1min":0.42899999999999999,"5min":0.378,"15min":0.378},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.55200000000000005}]},{"osd":7,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.59899999999999998,"15min":0.57899999999999996},"min":{"1min":0.41299999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.81200000000000006,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.61099999999999999,"15min":0.59399999999999997},"min":{"1min":0.54000000000000004,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.47499999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574892,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28104,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343736,"statfs":{"total":96636764160,"available":96607985664,"internally_reserved":0,"allocated":1105920,"data_stored":519171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:27:47 2023","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.36899999999999999,"15min":0.379},"min":{"1min":0.22800000000000001,"5min":0.219,"15min":0.219},"max":{"1min":0.52400000000000002,"5min":1.389,"15min":1.389},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.33800000000000002,"5min":0.58299999999999996,"15min":0.68999999999999995},"min":{"1min":0.25900000000000001,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.504,"5min":9.1370000000000005,"15min":9.1370000000000005},"last":0.36499999999999999}]},{"osd":2,"last update":"Wed Nov 8 19:27:47 2023","interfaces":[{"interface":"back","average":{"1min":0.33600000000000002,"5min":0.35399999999999998,"15min":0.36299999999999999},"min":{"1min":0.254,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.50600000000000001,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.373,"5min":0.41099999999999998,"15min":0.42499999999999999},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.58299999999999996,"5min":1.425,"15min":1.425},"last":0.47899999999999998}]},{"osd":3,"last update":"Wed Nov 8 19:27:47 2023","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.69999999999999996,"15min":0.84099999999999997},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.54600000000000004,"5min":11.957000000000001,"15min":11.957000000000001},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.85799999999999998,"15min":1.0740000000000001},"min":{"1min":0.27700000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.56299999999999994,"5min":18.039000000000001,"15min":18.039000000000001},"last":0.437}]},{"osd":4,"last update":"Wed Nov 8 19:27:47 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.73399999999999999,"15min":0.79800000000000004},"min":{"1min":0.43099999999999999,"5min":0.372,"15min":0.372},"max":{"1min":0.77800000000000002,"5min":6.181,"15min":6.181},"last":0.94699999999999995},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.629,"15min":0.63700000000000001},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.76900000000000002,"5min":1.488,"15min":1.488},"last":0.996}]},{"osd":5,"last update":"Wed Nov 8 19:27:47 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.84099999999999997,"15min":0.93700000000000006},"min":{"1min":0.437,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.74199999999999999,"5min":9.2420000000000009,"15min":9.2420000000000009},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.77200000000000002,"15min":0.84399999999999997},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.745,"5min":6.2110000000000003,"15min":6.2110000000000003},"last":0.878}]},{"osd":6,"last update":"Wed Nov 8 19:27:47 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.84899999999999998,"15min":0.94099999999999995},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.751,"5min":9.1880000000000006,"15min":9.1880000000000006},"last":0.755},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.65500000000000003,"15min":0.66300000000000003},"min":{"1min":0.38800000000000001,"5min":0.38,"15min":0.38},"max":{"1min":0.79400000000000004,"5min":1.462,"15min":1.462},"last":0.85799999999999998}]},{"osd":7,"last update":"Wed Nov 8 19:27:47 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.77400000000000002,"15min":0.83799999999999997},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.76600000000000001,"5min":6.1479999999999997,"15min":6.1479999999999997},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.872,"15min":0.97599999999999998},"min":{"1min":0.46100000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.76400000000000001,"5min":9.2170000000000005,"15min":9.2170000000000005},"last":0.90600000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574892,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28680,"kb_used_data":1660,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343160,"statfs":{"total":96636764160,"available":96607395840,"internally_reserved":0,"allocated":1699840,"data_stored":1109539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Nov 8 19:27:50 2023","interfaces":[{"interface":"back","average":{"1min":0.32100000000000001,"5min":0.441,"15min":0.495},"min":{"1min":0.23599999999999999,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.48799999999999999,"5min":4.1200000000000001,"15min":4.1200000000000001},"last":0.247},{"interface":"front","average":{"1min":0.32300000000000001,"5min":0.85199999999999998,"15min":1.079},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.51200000000000001,"5min":16.940999999999999,"15min":16.940999999999999},"last":0.34000000000000002}]},{"osd":2,"last update":"Wed Nov 8 19:27:50 2023","interfaces":[{"interface":"back","average":{"1min":0.318,"5min":0.374,"15min":0.39400000000000002},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.45600000000000002,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.57499999999999996,"15min":0.66600000000000004},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.58299999999999996,"5min":7.0209999999999999,"15min":7.0209999999999999},"last":0.28699999999999998}]},{"osd":3,"last update":"Wed Nov 8 19:27:50 2023","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.49299999999999999,"15min":0.54800000000000004},"min":{"1min":0.25,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.56699999999999995,"5min":4.0940000000000003,"15min":4.0940000000000003},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.37,"5min":0.42299999999999999,"15min":0.442},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.53600000000000003,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.36099999999999999}]},{"osd":4,"last update":"Wed Nov 8 19:27:50 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.69299999999999995,"15min":0.751},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.76800000000000002,"5min":4.0049999999999999,"15min":4.0049999999999999},"last":0.375},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.62,"15min":0.63500000000000001},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.77500000000000002,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.40000000000000002}]},{"osd":5,"last update":"Wed Nov 8 19:27:50 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.71199999999999997,"15min":0.76600000000000001},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.79200000000000004,"5min":4.3090000000000002,"15min":4.3090000000000002},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.70899999999999996,"15min":0.75700000000000001},"min":{"1min":0.45600000000000002,"5min":0.437,"15min":0.437},"max":{"1min":0.84699999999999998,"5min":4.0709999999999997,"15min":4.0709999999999997},"last":0.441}]},{"osd":6,"last update":"Wed Nov 8 19:27:50 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.65500000000000003,"15min":0.66900000000000004},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.81200000000000006,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.76800000000000002,"15min":0.82399999999999995},"min":{"1min":0.47699999999999998,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.84199999999999997,"5min":4.2560000000000002,"15min":4.2560000000000002},"last":0.56200000000000006}]},{"osd":7,"last update":"Wed Nov 8 19:27:50 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.72699999999999998,"15min":0.77700000000000002},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.78700000000000003,"5min":4.2850000000000001,"15min":4.2850000000000001},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.753,"15min":0.80500000000000005},"min":{"1min":0.48699999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.82499999999999996,"5min":4.0460000000000003,"15min":4.0460000000000003},"last":0.41399999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574892,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28088,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343752,"statfs":{"total":96636764160,"available":96608002048,"internally_reserved":0,"allocated":1093632,"data_stored":511833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:27:47 2023","interfaces":[{"interface":"back","average":{"1min":0.32100000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.47299999999999998,"5min":0.496,"15min":0.496},"last":0.28499999999999998},{"interface":"front","average":{"1min":0.31,"5min":0.51900000000000002,"15min":0.61299999999999999},"min":{"1min":0.219,"5min":0.218,"15min":0.218},"max":{"1min":0.501,"5min":6.0999999999999996,"15min":6.0999999999999996},"last":0.23000000000000001}]},{"osd":1,"last update":"Wed Nov 8 19:27:47 2023","interfaces":[{"interface":"back","average":{"1min":0.309,"5min":0.33400000000000002,"15min":0.33800000000000002},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.42699999999999999,"5min":0.48299999999999998,"15min":0.48299999999999998},"last":0.26300000000000001},{"interface":"front","average":{"1min":0.32500000000000001,"5min":0.372,"15min":0.38400000000000001},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.41899999999999998,"5min":0.624,"15min":0.624},"last":0.32600000000000001}]},{"osd":3,"last update":"Wed Nov 8 19:27:47 2023","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.376,"15min":0.38200000000000001},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.46500000000000002,"5min":0.53700000000000003,"15min":0.53700000000000003},"last":0.375},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.38300000000000001,"15min":0.39200000000000002},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.53400000000000003,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.34899999999999998}]},{"osd":4,"last update":"Wed Nov 8 19:27:47 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.56699999999999995,"15min":0.57899999999999996},"min":{"1min":0.39000000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.57699999999999996,"15min":0.58199999999999996},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.77100000000000002,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.5}]},{"osd":5,"last update":"Wed Nov 8 19:27:47 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.59199999999999997,"15min":0.58499999999999996},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.78100000000000003,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.628},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.60499999999999998,"15min":0.61099999999999999},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.76400000000000001,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.57199999999999995}]},{"osd":6,"last update":"Wed Nov 8 19:27:47 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59799999999999998,"15min":0.59999999999999998},"min":{"1min":0.47199999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.79700000000000004,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57799999999999996,"15min":0.57899999999999996},"min":{"1min":0.41499999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.80000000000000004,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.58899999999999997}]},{"osd":7,"last update":"Wed Nov 8 19:27:47 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.58199999999999996,"15min":0.57999999999999996},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.73199999999999998,"5min":0.878,"15min":0.878},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60099999999999998,"15min":0.60299999999999998},"min":{"1min":0.45300000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.82199999999999995,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.65700000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574893,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28104,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343736,"statfs":{"total":96636764160,"available":96607985664,"internally_reserved":0,"allocated":1105920,"data_stored":519171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.318,"5min":0.33300000000000002,"15min":0.33900000000000002},"min":{"1min":0.224,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.49099999999999999,"5min":0.626,"15min":0.626},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.34999999999999998,"15min":0.34899999999999998},"min":{"1min":0.28000000000000003,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.48699999999999999,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.26200000000000001}]},{"osd":1,"last update":"Wed Nov 8 19:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.34699999999999998,"15min":0.35199999999999998},"min":{"1min":0.25800000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.42299999999999999,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":0.29599999999999999},{"interface":"front","average":{"1min":0.38,"5min":0.38300000000000001,"15min":0.38400000000000001},"min":{"1min":0.28000000000000003,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.48699999999999999,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.27900000000000003}]},{"osd":2,"last update":"Wed Nov 8 19:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.376,"15min":0.372},"min":{"1min":0.32900000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.46300000000000002,"5min":0.58499999999999996,"15min":0.58499999999999996},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.40400000000000003,"15min":0.40400000000000003},"min":{"1min":0.32700000000000001,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.47399999999999998,"5min":0.58499999999999996,"15min":0.58499999999999996},"last":0.32000000000000001}]},{"osd":4,"last update":"Wed Nov 8 19:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.60799999999999998,"15min":0.60099999999999998},"min":{"1min":0.45300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.624,"5min":0.61199999999999999,"15min":0.60699999999999998},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.84499999999999997,"5min":0.871,"15min":0.871},"last":0.56100000000000005}]},{"osd":5,"last update":"Wed Nov 8 19:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.58699999999999997,"15min":0.57899999999999996},"min":{"1min":0.46700000000000003,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.76600000000000001,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.58899999999999997,"15min":0.58099999999999996},"min":{"1min":0.42399999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.78600000000000003,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.54700000000000004}]},{"osd":6,"last update":"Wed Nov 8 19:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.58799999999999997,"15min":0.58199999999999996},"min":{"1min":0.436,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.73299999999999998,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.61299999999999999,"15min":0.60699999999999998},"min":{"1min":0.49099999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.75900000000000001,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.57399999999999995}]},{"osd":7,"last update":"Wed Nov 8 19:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.59899999999999998,"15min":0.59099999999999997},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.72199999999999998,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.60199999999999998,"15min":0.59799999999999998},"min":{"1min":0.45800000000000002,"5min":0.432,"15min":0.432},"max":{"1min":0.80700000000000005,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.58299999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574892,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28104,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343736,"statfs":{"total":96636764160,"available":96607985664,"internally_reserved":0,"allocated":1105920,"data_stored":519171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:27:55 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.68200000000000005,"15min":0.74399999999999999},"min":{"1min":0.35699999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.72099999999999997,"5min":6.5970000000000004,"15min":6.5970000000000004},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.55500000000000005,"15min":0.56999999999999995},"min":{"1min":0.35799999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.70999999999999996,"5min":2.5030000000000001,"15min":2.5030000000000001},"last":0.46899999999999997}]},{"osd":1,"last update":"Wed Nov 8 19:27:55 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.51300000000000001,"15min":0.501},"min":{"1min":0.36199999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.75800000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.629},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.52100000000000002,"15min":0.51200000000000001},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.70199999999999996}]},{"osd":2,"last update":"Wed Nov 8 19:27:55 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.55400000000000005,"15min":0.55200000000000005},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.80600000000000005,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.748,"15min":0.81299999999999994},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.86199999999999999,"5min":6.6920000000000002,"15min":6.6920000000000002},"last":0.66900000000000004}]},{"osd":3,"last update":"Wed Nov 8 19:27:55 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.58299999999999996,"15min":0.58699999999999997},"min":{"1min":0.378,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.79800000000000004,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.60899999999999999,"15min":0.60999999999999999},"min":{"1min":0.42499999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.71399999999999997,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.68500000000000005}]},{"osd":5,"last update":"Wed Nov 8 19:27:55 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.55000000000000004,"15min":0.54800000000000004},"min":{"1min":0.35099999999999998,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.81599999999999995,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.58499999999999996,"15min":0.59299999999999997},"min":{"1min":0.39400000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.745,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.53000000000000003}]},{"osd":6,"last update":"Wed Nov 8 19:27:55 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58599999999999997,"15min":0.58499999999999996},"min":{"1min":0.373,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.627,"15min":0.61799999999999999},"min":{"1min":0.46400000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.73099999999999998}]},{"osd":7,"last update":"Wed Nov 8 19:27:55 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.61599999999999999,"15min":0.629},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.84499999999999997,"5min":1.141,"15min":1.141},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.64100000000000001,"15min":0.64600000000000002},"min":{"1min":0.42399999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.86399999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.74199999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574892,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28104,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343736,"statfs":{"total":96636764160,"available":96607985664,"internally_reserved":0,"allocated":1105920,"data_stored":519171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:27:46 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.51400000000000001,"15min":0.52000000000000002},"min":{"1min":0.32200000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.73999999999999999,"5min":0.753,"15min":0.753},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.51900000000000002,"15min":0.51200000000000001},"min":{"1min":0.38500000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.69399999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.495}]},{"osd":1,"last update":"Wed Nov 8 19:27:46 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.51900000000000002,"15min":0.52200000000000002},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.70799999999999996,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.53400000000000003,"15min":0.53600000000000003},"min":{"1min":0.38400000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.70499999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.56899999999999995}]},{"osd":2,"last update":"Wed Nov 8 19:27:46 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.53600000000000003,"15min":0.53500000000000003},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.68000000000000005,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.55800000000000005,"15min":0.56999999999999995},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.69799999999999995,"5min":0.747,"15min":0.747},"last":0.47699999999999998}]},{"osd":3,"last update":"Wed Nov 8 19:27:46 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.54700000000000004,"15min":0.54900000000000004},"min":{"1min":0.40899999999999997,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.72199999999999998,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.57899999999999996,"15min":0.59599999999999997},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.69099999999999995,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.627}]},{"osd":4,"last update":"Wed Nov 8 19:27:46 2023","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.53200000000000003,"15min":0.55200000000000005},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.67200000000000004,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.56200000000000006,"15min":0.56899999999999995},"min":{"1min":0.36599999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.37}]},{"osd":6,"last update":"Wed Nov 8 19:27:46 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.59399999999999997,"15min":0.60699999999999998},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.755,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.59799999999999998,"15min":0.61099999999999999},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.72099999999999997,"5min":0.745,"15min":0.745},"last":0.60999999999999999}]},{"osd":7,"last update":"Wed Nov 8 19:27:46 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.59399999999999997,"15min":0.60599999999999998},"min":{"1min":0.44500000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.74199999999999999,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.625,"15min":0.64200000000000002},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.752,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.59399999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-08T19:28:08.615 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.3f is not active+clean 2023-11-08T19:28:08.616 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2023-11-08T19:28:02.701553+0000', 'last_change': '2023-11-08T19:28:02.701553+0000', 'last_active': '2023-11-08T19:28:02.688498+0000', 'last_peered': '2023-11-08T19:28:02.688498+0000', 'last_clean': '2023-11-08T19:28:02.688498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:02.701553+0000', 'last_undegraded': '2023-11-08T19:28:02.701553+0000', 'last_fullsized': '2023-11-08T19:28:02.701553+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-08T19:28:08.616 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.3d is not active+clean 2023-11-08T19:28:08.616 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2023-11-08T19:28:02.700726+0000', 'last_change': '2023-11-08T19:28:02.700726+0000', 'last_active': '2023-11-08T19:28:02.688498+0000', 'last_peered': '2023-11-08T19:28:02.688498+0000', 'last_clean': '2023-11-08T19:28:02.688498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:02.700726+0000', 'last_undegraded': '2023-11-08T19:28:02.700726+0000', 'last_fullsized': '2023-11-08T19:28:02.700726+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-08T19:28:08.617 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.3c is not active+clean 2023-11-08T19:28:08.617 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2023-11-08T19:28:02.699799+0000', 'last_change': '2023-11-08T19:28:02.699799+0000', 'last_active': '2023-11-08T19:28:02.688498+0000', 'last_peered': '2023-11-08T19:28:02.688498+0000', 'last_clean': '2023-11-08T19:28:02.688498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:02.699799+0000', 'last_undegraded': '2023-11-08T19:28:02.699799+0000', 'last_fullsized': '2023-11-08T19:28:02.699799+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-08T19:28:08.618 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.3b is not active+clean 2023-11-08T19:28:08.618 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2023-11-08T19:28:02.704338+0000', 'last_change': '2023-11-08T19:28:02.704338+0000', 'last_active': '2023-11-08T19:28:02.688498+0000', 'last_peered': '2023-11-08T19:28:02.688498+0000', 'last_clean': '2023-11-08T19:28:02.688498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:02.704338+0000', 'last_undegraded': '2023-11-08T19:28:02.704338+0000', 'last_fullsized': '2023-11-08T19:28:02.704338+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-08T19:28:08.618 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.38 is not active+clean 2023-11-08T19:28:08.619 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2023-11-08T19:28:02.714751+0000', 'last_change': '2023-11-08T19:28:02.714751+0000', 'last_active': '2023-11-08T19:28:02.688498+0000', 'last_peered': '2023-11-08T19:28:02.688498+0000', 'last_clean': '2023-11-08T19:28:02.688498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:02.714751+0000', 'last_undegraded': '2023-11-08T19:28:02.714751+0000', 'last_fullsized': '2023-11-08T19:28:02.714751+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-08T19:28:08.620 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.36 is not active+clean 2023-11-08T19:28:08.622 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2023-11-08T19:28:02.702139+0000', 'last_change': '2023-11-08T19:28:02.702139+0000', 'last_active': '2023-11-08T19:28:02.688498+0000', 'last_peered': '2023-11-08T19:28:02.688498+0000', 'last_clean': '2023-11-08T19:28:02.688498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:02.702139+0000', 'last_undegraded': '2023-11-08T19:28:02.702139+0000', 'last_fullsized': '2023-11-08T19:28:02.702139+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-08T19:28:08.623 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.35 is not active+clean 2023-11-08T19:28:08.623 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2023-11-08T19:28:02.705435+0000', 'last_change': '2023-11-08T19:28:02.705435+0000', 'last_active': '2023-11-08T19:28:02.688498+0000', 'last_peered': '2023-11-08T19:28:02.688498+0000', 'last_clean': '2023-11-08T19:28:02.688498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:02.705435+0000', 'last_undegraded': '2023-11-08T19:28:02.705435+0000', 'last_fullsized': '2023-11-08T19:28:02.705435+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-08T19:28:08.624 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.34 is not active+clean 2023-11-08T19:28:08.624 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2023-11-08T19:28:02.703962+0000', 'last_change': '2023-11-08T19:28:02.703962+0000', 'last_active': '2023-11-08T19:28:02.688498+0000', 'last_peered': '2023-11-08T19:28:02.688498+0000', 'last_clean': '2023-11-08T19:28:02.688498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:02.703962+0000', 'last_undegraded': '2023-11-08T19:28:02.703962+0000', 'last_fullsized': '2023-11-08T19:28:02.703962+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-08T19:28:08.625 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.33 is not active+clean 2023-11-08T19:28:08.625 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2023-11-08T19:28:02.697723+0000', 'last_change': '2023-11-08T19:28:02.697723+0000', 'last_active': '2023-11-08T19:28:02.688498+0000', 'last_peered': '2023-11-08T19:28:02.688498+0000', 'last_clean': '2023-11-08T19:28:02.688498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:02.697723+0000', 'last_undegraded': '2023-11-08T19:28:02.697723+0000', 'last_fullsized': '2023-11-08T19:28:02.697723+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-08T19:28:08.626 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.2f is not active+clean 2023-11-08T19:28:08.626 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2023-11-08T19:28:02.700364+0000', 'last_change': '2023-11-08T19:28:02.700364+0000', 'last_active': '2023-11-08T19:28:02.688498+0000', 'last_peered': '2023-11-08T19:28:02.688498+0000', 'last_clean': '2023-11-08T19:28:02.688498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:02.700364+0000', 'last_undegraded': '2023-11-08T19:28:02.700364+0000', 'last_fullsized': '2023-11-08T19:28:02.700364+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-08T19:28:08.626 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.2d is not active+clean 2023-11-08T19:28:08.627 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2023-11-08T19:28:02.700365+0000', 'last_change': '2023-11-08T19:28:02.700365+0000', 'last_active': '2023-11-08T19:28:02.688498+0000', 'last_peered': '2023-11-08T19:28:02.688498+0000', 'last_clean': '2023-11-08T19:28:02.688498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:02.700365+0000', 'last_undegraded': '2023-11-08T19:28:02.700365+0000', 'last_fullsized': '2023-11-08T19:28:02.700365+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-08T19:28:08.627 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.2c is not active+clean 2023-11-08T19:28:08.628 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2023-11-08T19:28:02.704383+0000', 'last_change': '2023-11-08T19:28:02.704383+0000', 'last_active': '2023-11-08T19:28:02.688498+0000', 'last_peered': '2023-11-08T19:28:02.688498+0000', 'last_clean': '2023-11-08T19:28:02.688498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:02.704383+0000', 'last_undegraded': '2023-11-08T19:28:02.704383+0000', 'last_fullsized': '2023-11-08T19:28:02.704383+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-08T19:28:08.628 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.2b is not active+clean 2023-11-08T19:28:08.628 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2023-11-08T19:28:02.713555+0000', 'last_change': '2023-11-08T19:28:02.713555+0000', 'last_active': '2023-11-08T19:28:02.688498+0000', 'last_peered': '2023-11-08T19:28:02.688498+0000', 'last_clean': '2023-11-08T19:28:02.688498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:02.713555+0000', 'last_undegraded': '2023-11-08T19:28:02.713555+0000', 'last_fullsized': '2023-11-08T19:28:02.713555+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-08T19:28:08.629 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.2a is not active+clean 2023-11-08T19:28:08.629 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2023-11-08T19:28:02.702640+0000', 'last_change': '2023-11-08T19:28:02.702640+0000', 'last_active': '2023-11-08T19:28:02.688498+0000', 'last_peered': '2023-11-08T19:28:02.688498+0000', 'last_clean': '2023-11-08T19:28:02.688498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:02.702640+0000', 'last_undegraded': '2023-11-08T19:28:02.702640+0000', 'last_fullsized': '2023-11-08T19:28:02.702640+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-08T19:28:08.630 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.29 is not active+clean 2023-11-08T19:28:08.630 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2023-11-08T19:28:02.699841+0000', 'last_change': '2023-11-08T19:28:02.699841+0000', 'last_active': '2023-11-08T19:28:02.688498+0000', 'last_peered': '2023-11-08T19:28:02.688498+0000', 'last_clean': '2023-11-08T19:28:02.688498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:02.699841+0000', 'last_undegraded': '2023-11-08T19:28:02.699841+0000', 'last_fullsized': '2023-11-08T19:28:02.699841+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-08T19:28:08.630 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.28 is not active+clean 2023-11-08T19:28:08.631 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2023-11-08T19:28:02.702861+0000', 'last_change': '2023-11-08T19:28:02.702861+0000', 'last_active': '2023-11-08T19:28:02.688498+0000', 'last_peered': '2023-11-08T19:28:02.688498+0000', 'last_clean': '2023-11-08T19:28:02.688498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:02.702861+0000', 'last_undegraded': '2023-11-08T19:28:02.702861+0000', 'last_fullsized': '2023-11-08T19:28:02.702861+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-08T19:28:08.631 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.27 is not active+clean 2023-11-08T19:28:08.632 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2023-11-08T19:28:02.707625+0000', 'last_change': '2023-11-08T19:28:02.707625+0000', 'last_active': '2023-11-08T19:28:02.688498+0000', 'last_peered': '2023-11-08T19:28:02.688498+0000', 'last_clean': '2023-11-08T19:28:02.688498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:02.707625+0000', 'last_undegraded': '2023-11-08T19:28:02.707625+0000', 'last_fullsized': '2023-11-08T19:28:02.707625+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-08T19:28:08.632 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.26 is not active+clean 2023-11-08T19:28:08.632 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2023-11-08T19:28:02.710228+0000', 'last_change': '2023-11-08T19:28:02.710228+0000', 'last_active': '2023-11-08T19:28:02.688498+0000', 'last_peered': '2023-11-08T19:28:02.688498+0000', 'last_clean': '2023-11-08T19:28:02.688498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:02.710228+0000', 'last_undegraded': '2023-11-08T19:28:02.710228+0000', 'last_fullsized': '2023-11-08T19:28:02.710228+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-08T19:28:08.633 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.25 is not active+clean 2023-11-08T19:28:08.633 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2023-11-08T19:28:02.704524+0000', 'last_change': '2023-11-08T19:28:02.704524+0000', 'last_active': '2023-11-08T19:28:02.688498+0000', 'last_peered': '2023-11-08T19:28:02.688498+0000', 'last_clean': '2023-11-08T19:28:02.688498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:02.704524+0000', 'last_undegraded': '2023-11-08T19:28:02.704524+0000', 'last_fullsized': '2023-11-08T19:28:02.704524+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-08T19:28:08.634 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.23 is not active+clean 2023-11-08T19:28:08.634 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2023-11-08T19:28:02.712428+0000', 'last_change': '2023-11-08T19:28:02.712428+0000', 'last_active': '2023-11-08T19:28:02.688498+0000', 'last_peered': '2023-11-08T19:28:02.688498+0000', 'last_clean': '2023-11-08T19:28:02.688498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:02.712428+0000', 'last_undegraded': '2023-11-08T19:28:02.712428+0000', 'last_fullsized': '2023-11-08T19:28:02.712428+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-08T19:28:08.634 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.22 is not active+clean 2023-11-08T19:28:08.635 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2023-11-08T19:28:02.705928+0000', 'last_change': '2023-11-08T19:28:02.705928+0000', 'last_active': '2023-11-08T19:28:02.688498+0000', 'last_peered': '2023-11-08T19:28:02.688498+0000', 'last_clean': '2023-11-08T19:28:02.688498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:02.705928+0000', 'last_undegraded': '2023-11-08T19:28:02.705928+0000', 'last_fullsized': '2023-11-08T19:28:02.705928+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-08T19:28:08.635 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.20 is not active+clean 2023-11-08T19:28:08.635 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2023-11-08T19:28:02.711643+0000', 'last_change': '2023-11-08T19:28:02.711643+0000', 'last_active': '2023-11-08T19:28:02.688498+0000', 'last_peered': '2023-11-08T19:28:02.688498+0000', 'last_clean': '2023-11-08T19:28:02.688498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:02.711643+0000', 'last_undegraded': '2023-11-08T19:28:02.711643+0000', 'last_fullsized': '2023-11-08T19:28:02.711643+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-08T19:28:08.636 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.11 is not active+clean 2023-11-08T19:28:08.636 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2023-11-08T19:28:02.702089+0000', 'last_change': '2023-11-08T19:28:02.702089+0000', 'last_active': '2023-11-08T19:28:02.688498+0000', 'last_peered': '2023-11-08T19:28:02.688498+0000', 'last_clean': '2023-11-08T19:28:02.688498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:02.702089+0000', 'last_undegraded': '2023-11-08T19:28:02.702089+0000', 'last_fullsized': '2023-11-08T19:28:02.702089+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-08T19:28:08.637 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.14 is not active+clean 2023-11-08T19:28:08.637 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2023-11-08T19:28:02.698950+0000', 'last_change': '2023-11-08T19:28:02.698950+0000', 'last_active': '2023-11-08T19:28:02.688498+0000', 'last_peered': '2023-11-08T19:28:02.688498+0000', 'last_clean': '2023-11-08T19:28:02.688498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:02.698950+0000', 'last_undegraded': '2023-11-08T19:28:02.698950+0000', 'last_fullsized': '2023-11-08T19:28:02.698950+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-08T19:28:08.637 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.15 is not active+clean 2023-11-08T19:28:08.638 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2023-11-08T19:28:02.699139+0000', 'last_change': '2023-11-08T19:28:02.699139+0000', 'last_active': '2023-11-08T19:28:02.688498+0000', 'last_peered': '2023-11-08T19:28:02.688498+0000', 'last_clean': '2023-11-08T19:28:02.688498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:02.699139+0000', 'last_undegraded': '2023-11-08T19:28:02.699139+0000', 'last_fullsized': '2023-11-08T19:28:02.699139+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-08T19:28:08.638 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.16 is not active+clean 2023-11-08T19:28:08.639 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2023-11-08T19:28:02.697661+0000', 'last_change': '2023-11-08T19:28:02.697661+0000', 'last_active': '2023-11-08T19:28:02.688498+0000', 'last_peered': '2023-11-08T19:28:02.688498+0000', 'last_clean': '2023-11-08T19:28:02.688498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:02.697661+0000', 'last_undegraded': '2023-11-08T19:28:02.697661+0000', 'last_fullsized': '2023-11-08T19:28:02.697661+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-08T19:28:08.639 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.17 is not active+clean 2023-11-08T19:28:08.640 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2023-11-08T19:28:02.713309+0000', 'last_change': '2023-11-08T19:28:02.713309+0000', 'last_active': '2023-11-08T19:28:02.688498+0000', 'last_peered': '2023-11-08T19:28:02.688498+0000', 'last_clean': '2023-11-08T19:28:02.688498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:02.713309+0000', 'last_undegraded': '2023-11-08T19:28:02.713309+0000', 'last_fullsized': '2023-11-08T19:28:02.713309+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-08T19:28:08.640 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.10 is not active+clean 2023-11-08T19:28:08.640 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2023-11-08T19:28:02.714312+0000', 'last_change': '2023-11-08T19:28:02.714312+0000', 'last_active': '2023-11-08T19:28:02.688498+0000', 'last_peered': '2023-11-08T19:28:02.688498+0000', 'last_clean': '2023-11-08T19:28:02.688498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:02.714312+0000', 'last_undegraded': '2023-11-08T19:28:02.714312+0000', 'last_fullsized': '2023-11-08T19:28:02.714312+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-08T19:28:08.641 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.13 is not active+clean 2023-11-08T19:28:08.641 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2023-11-08T19:28:02.703964+0000', 'last_change': '2023-11-08T19:28:02.703964+0000', 'last_active': '2023-11-08T19:28:02.688498+0000', 'last_peered': '2023-11-08T19:28:02.688498+0000', 'last_clean': '2023-11-08T19:28:02.688498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:02.703964+0000', 'last_undegraded': '2023-11-08T19:28:02.703964+0000', 'last_fullsized': '2023-11-08T19:28:02.703964+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-08T19:28:08.642 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.1c is not active+clean 2023-11-08T19:28:08.642 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2023-11-08T19:28:02.711687+0000', 'last_change': '2023-11-08T19:28:02.711687+0000', 'last_active': '2023-11-08T19:28:02.688498+0000', 'last_peered': '2023-11-08T19:28:02.688498+0000', 'last_clean': '2023-11-08T19:28:02.688498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:02.711687+0000', 'last_undegraded': '2023-11-08T19:28:02.711687+0000', 'last_fullsized': '2023-11-08T19:28:02.711687+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-08T19:28:08.642 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.1d is not active+clean 2023-11-08T19:28:08.643 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2023-11-08T19:28:02.712306+0000', 'last_change': '2023-11-08T19:28:02.712306+0000', 'last_active': '2023-11-08T19:28:02.688498+0000', 'last_peered': '2023-11-08T19:28:02.688498+0000', 'last_clean': '2023-11-08T19:28:02.688498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:02.712306+0000', 'last_undegraded': '2023-11-08T19:28:02.712306+0000', 'last_fullsized': '2023-11-08T19:28:02.712306+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-08T19:28:08.643 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.1f is not active+clean 2023-11-08T19:28:08.643 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2023-11-08T19:28:02.704461+0000', 'last_change': '2023-11-08T19:28:02.704461+0000', 'last_active': '2023-11-08T19:28:02.688498+0000', 'last_peered': '2023-11-08T19:28:02.688498+0000', 'last_clean': '2023-11-08T19:28:02.688498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:02.704461+0000', 'last_undegraded': '2023-11-08T19:28:02.704461+0000', 'last_fullsized': '2023-11-08T19:28:02.704461+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-08T19:28:08.644 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.18 is not active+clean 2023-11-08T19:28:08.644 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2023-11-08T19:28:02.699774+0000', 'last_change': '2023-11-08T19:28:02.699774+0000', 'last_active': '2023-11-08T19:28:02.688498+0000', 'last_peered': '2023-11-08T19:28:02.688498+0000', 'last_clean': '2023-11-08T19:28:02.688498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:02.699774+0000', 'last_undegraded': '2023-11-08T19:28:02.699774+0000', 'last_fullsized': '2023-11-08T19:28:02.699774+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-08T19:28:08.645 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.19 is not active+clean 2023-11-08T19:28:08.645 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2023-11-08T19:28:02.701682+0000', 'last_change': '2023-11-08T19:28:02.701682+0000', 'last_active': '2023-11-08T19:28:02.688498+0000', 'last_peered': '2023-11-08T19:28:02.688498+0000', 'last_clean': '2023-11-08T19:28:02.688498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:02.701682+0000', 'last_undegraded': '2023-11-08T19:28:02.701682+0000', 'last_fullsized': '2023-11-08T19:28:02.701682+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-08T19:28:08.645 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.1a is not active+clean 2023-11-08T19:28:08.646 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2023-11-08T19:28:02.697682+0000', 'last_change': '2023-11-08T19:28:02.697682+0000', 'last_active': '2023-11-08T19:28:02.688498+0000', 'last_peered': '2023-11-08T19:28:02.688498+0000', 'last_clean': '2023-11-08T19:28:02.688498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:02.697682+0000', 'last_undegraded': '2023-11-08T19:28:02.697682+0000', 'last_fullsized': '2023-11-08T19:28:02.697682+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-08T19:28:08.646 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.1b is not active+clean 2023-11-08T19:28:08.647 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2023-11-08T19:28:02.704857+0000', 'last_change': '2023-11-08T19:28:02.704857+0000', 'last_active': '2023-11-08T19:28:02.688498+0000', 'last_peered': '2023-11-08T19:28:02.688498+0000', 'last_clean': '2023-11-08T19:28:02.688498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:02.704857+0000', 'last_undegraded': '2023-11-08T19:28:02.704857+0000', 'last_fullsized': '2023-11-08T19:28:02.704857+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-08T19:28:08.647 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.4 is not active+clean 2023-11-08T19:28:08.647 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2023-11-08T19:28:02.698528+0000', 'last_change': '2023-11-08T19:28:02.698528+0000', 'last_active': '2023-11-08T19:28:02.688498+0000', 'last_peered': '2023-11-08T19:28:02.688498+0000', 'last_clean': '2023-11-08T19:28:02.688498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:02.698528+0000', 'last_undegraded': '2023-11-08T19:28:02.698528+0000', 'last_fullsized': '2023-11-08T19:28:02.698528+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-08T19:28:08.648 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.5 is not active+clean 2023-11-08T19:28:08.648 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2023-11-08T19:28:02.697868+0000', 'last_change': '2023-11-08T19:28:02.697868+0000', 'last_active': '2023-11-08T19:28:02.688498+0000', 'last_peered': '2023-11-08T19:28:02.688498+0000', 'last_clean': '2023-11-08T19:28:02.688498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:02.697868+0000', 'last_undegraded': '2023-11-08T19:28:02.697868+0000', 'last_fullsized': '2023-11-08T19:28:02.697868+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-08T19:28:08.649 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.7 is not active+clean 2023-11-08T19:28:08.649 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2023-11-08T19:28:02.698922+0000', 'last_change': '2023-11-08T19:28:02.698922+0000', 'last_active': '2023-11-08T19:28:02.688498+0000', 'last_peered': '2023-11-08T19:28:02.688498+0000', 'last_clean': '2023-11-08T19:28:02.688498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:02.698922+0000', 'last_undegraded': '2023-11-08T19:28:02.698922+0000', 'last_fullsized': '2023-11-08T19:28:02.698922+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-08T19:28:08.649 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.0 is not active+clean 2023-11-08T19:28:08.650 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2023-11-08T19:28:02.698224+0000', 'last_change': '2023-11-08T19:28:02.698224+0000', 'last_active': '2023-11-08T19:28:02.688498+0000', 'last_peered': '2023-11-08T19:28:02.688498+0000', 'last_clean': '2023-11-08T19:28:02.688498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:02.698224+0000', 'last_undegraded': '2023-11-08T19:28:02.698224+0000', 'last_fullsized': '2023-11-08T19:28:02.698224+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-08T19:28:08.650 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.1 is not active+clean 2023-11-08T19:28:08.650 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2023-11-08T19:28:02.711664+0000', 'last_change': '2023-11-08T19:28:02.711664+0000', 'last_active': '2023-11-08T19:28:02.688498+0000', 'last_peered': '2023-11-08T19:28:02.688498+0000', 'last_clean': '2023-11-08T19:28:02.688498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:02.711664+0000', 'last_undegraded': '2023-11-08T19:28:02.711664+0000', 'last_fullsized': '2023-11-08T19:28:02.711664+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-08T19:28:08.651 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.2 is not active+clean 2023-11-08T19:28:08.651 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2023-11-08T19:28:02.705457+0000', 'last_change': '2023-11-08T19:28:02.705457+0000', 'last_active': '2023-11-08T19:28:02.688498+0000', 'last_peered': '2023-11-08T19:28:02.688498+0000', 'last_clean': '2023-11-08T19:28:02.688498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:02.705457+0000', 'last_undegraded': '2023-11-08T19:28:02.705457+0000', 'last_fullsized': '2023-11-08T19:28:02.705457+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-08T19:28:08.652 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.3 is not active+clean 2023-11-08T19:28:08.652 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2023-11-08T19:28:02.702920+0000', 'last_change': '2023-11-08T19:28:02.702920+0000', 'last_active': '2023-11-08T19:28:02.688498+0000', 'last_peered': '2023-11-08T19:28:02.688498+0000', 'last_clean': '2023-11-08T19:28:02.688498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:02.702920+0000', 'last_undegraded': '2023-11-08T19:28:02.702920+0000', 'last_fullsized': '2023-11-08T19:28:02.702920+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-08T19:28:08.652 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.c is not active+clean 2023-11-08T19:28:08.653 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2023-11-08T19:28:02.703242+0000', 'last_change': '2023-11-08T19:28:02.703242+0000', 'last_active': '2023-11-08T19:28:02.688498+0000', 'last_peered': '2023-11-08T19:28:02.688498+0000', 'last_clean': '2023-11-08T19:28:02.688498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:02.703242+0000', 'last_undegraded': '2023-11-08T19:28:02.703242+0000', 'last_fullsized': '2023-11-08T19:28:02.703242+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-08T19:28:08.653 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.e is not active+clean 2023-11-08T19:28:08.654 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2023-11-08T19:28:02.713166+0000', 'last_change': '2023-11-08T19:28:02.713166+0000', 'last_active': '2023-11-08T19:28:02.688498+0000', 'last_peered': '2023-11-08T19:28:02.688498+0000', 'last_clean': '2023-11-08T19:28:02.688498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:02.713166+0000', 'last_undegraded': '2023-11-08T19:28:02.713166+0000', 'last_fullsized': '2023-11-08T19:28:02.713166+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-08T19:28:08.654 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.f is not active+clean 2023-11-08T19:28:08.654 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2023-11-08T19:28:02.698299+0000', 'last_change': '2023-11-08T19:28:02.698299+0000', 'last_active': '2023-11-08T19:28:02.688498+0000', 'last_peered': '2023-11-08T19:28:02.688498+0000', 'last_clean': '2023-11-08T19:28:02.688498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:02.698299+0000', 'last_undegraded': '2023-11-08T19:28:02.698299+0000', 'last_fullsized': '2023-11-08T19:28:02.698299+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-08T19:28:08.655 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.8 is not active+clean 2023-11-08T19:28:08.655 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2023-11-08T19:28:02.699401+0000', 'last_change': '2023-11-08T19:28:02.699401+0000', 'last_active': '2023-11-08T19:28:02.688498+0000', 'last_peered': '2023-11-08T19:28:02.688498+0000', 'last_clean': '2023-11-08T19:28:02.688498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:02.699401+0000', 'last_undegraded': '2023-11-08T19:28:02.699401+0000', 'last_fullsized': '2023-11-08T19:28:02.699401+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-08T19:28:08.656 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.9 is not active+clean 2023-11-08T19:28:08.656 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2023-11-08T19:28:02.702677+0000', 'last_change': '2023-11-08T19:28:02.702677+0000', 'last_active': '2023-11-08T19:28:02.688498+0000', 'last_peered': '2023-11-08T19:28:02.688498+0000', 'last_clean': '2023-11-08T19:28:02.688498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:02.702677+0000', 'last_undegraded': '2023-11-08T19:28:02.702677+0000', 'last_fullsized': '2023-11-08T19:28:02.702677+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-08T19:28:08.656 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.a is not active+clean 2023-11-08T19:28:08.657 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2023-11-08T19:28:02.711863+0000', 'last_change': '2023-11-08T19:28:02.711863+0000', 'last_active': '2023-11-08T19:28:02.688498+0000', 'last_peered': '2023-11-08T19:28:02.688498+0000', 'last_clean': '2023-11-08T19:28:02.688498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:02.711863+0000', 'last_undegraded': '2023-11-08T19:28:02.711863+0000', 'last_fullsized': '2023-11-08T19:28:02.711863+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-08T19:28:08.657 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.b is not active+clean 2023-11-08T19:28:08.657 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 90, 'state': 'creating+peering', 'last_fresh': '2023-11-08T19:28:02.703260+0000', 'last_change': '2023-11-08T19:28:02.703260+0000', 'last_active': '2023-11-08T19:28:02.688498+0000', 'last_peered': '2023-11-08T19:28:02.688498+0000', 'last_clean': '2023-11-08T19:28:02.688498+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:02.703260+0000', 'last_undegraded': '2023-11-08T19:28:02.703260+0000', 'last_fullsized': '2023-11-08T19:28:02.703260+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 90, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:02.688498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-08T19:28:08.659 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-11-08T19:28:08.779 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.778+0000 7f6467540700 1 Processor -- start 2023-11-08T19:28:08.780 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.778+0000 7f6467540700 1 -- start start 2023-11-08T19:28:08.783 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.779+0000 7f6467540700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f64600d1170 0x7f646006ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:08.783 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.779+0000 7f6467540700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f64600ce300 0x7f646006b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:08.784 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.779+0000 7f6467540700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f646006e250 con 0x7f646015de50 2023-11-08T19:28:08.784 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.779+0000 7f6467540700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f646015e230 con 0x7f64600ce300 2023-11-08T19:28:08.784 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.779+0000 7f6467540700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f646006f750 con 0x7f64600d1170 2023-11-08T19:28:08.784 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.779+0000 7f64652dc700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f646015de50 0x7f64600cad80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:50824/0 (socket says 172.21.15.143:50824) 2023-11-08T19:28:08.785 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.779+0000 7f64652dc700 1 -- 172.21.15.143:0/1509275065 learned_addr learned my addr 172.21.15.143:0/1509275065 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:28:08.785 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.779+0000 7f6465add700 1 --2- 172.21.15.143:0/1509275065 >> v2:172.21.15.174:3300/0 conn(0x7f64600ce300 0x7f646006b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:08.785 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.779+0000 7f6464adb700 1 --2- 172.21.15.143:0/1509275065 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f64600d1170 0x7f646006ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:08.785 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.779+0000 7f6457fff700 1 -- 172.21.15.143:0/1509275065 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 719669565 0 0) 0x7f646006e250 con 0x7f646015de50 2023-11-08T19:28:08.785 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.779+0000 7f6457fff700 1 -- 172.21.15.143:0/1509275065 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f644c0035b0 con 0x7f646015de50 2023-11-08T19:28:08.786 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.780+0000 7f6457fff700 1 -- 172.21.15.143:0/1509275065 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 235075923 0 0) 0x7f644c0035b0 con 0x7f646015de50 2023-11-08T19:28:08.786 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.780+0000 7f6457fff700 1 -- 172.21.15.143:0/1509275065 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f64600d1170 msgr2=0x7f646006ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:08.786 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.780+0000 7f6457fff700 1 --2- 172.21.15.143:0/1509275065 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f64600d1170 0x7f646006ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:08.786 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.780+0000 7f6457fff700 1 -- 172.21.15.143:0/1509275065 >> v2:172.21.15.174:3300/0 conn(0x7f64600ce300 msgr2=0x7f646006b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:08.786 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.780+0000 7f6457fff700 1 --2- 172.21.15.143:0/1509275065 >> v2:172.21.15.174:3300/0 conn(0x7f64600ce300 0x7f646006b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:08.787 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.780+0000 7f6457fff700 1 -- 172.21.15.143:0/1509275065 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f646006fa30 con 0x7f646015de50 2023-11-08T19:28:08.787 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.780+0000 7f6457fff700 1 -- 172.21.15.143:0/1509275065 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4147756303 0 0) 0x7f64500017f0 con 0x7f646015de50 2023-11-08T19:28:08.787 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.780+0000 7f6465add700 1 --2- 172.21.15.143:0/1509275065 >> v2:172.21.15.174:3300/0 conn(0x7f64600ce300 0x7f646006b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T19:28:08.787 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.780+0000 7f6457fff700 1 -- 172.21.15.143:0/1509275065 >> v1:172.21.15.143:6789/0 conn(0x7f646015de50 legacy=0x7f64600cad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:08.787 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.780+0000 7f6457fff700 1 --2- 172.21.15.143:0/1509275065 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f64600d1170 0x7f644c003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:08.788 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.780+0000 7f6457fff700 1 --2- 172.21.15.143:0/1509275065 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f644c003f80 0x7f644c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:08.788 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.780+0000 7f6457fff700 1 --2- 172.21.15.143:0/1509275065 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f644c004d90 0x7f644c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:08.788 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.780+0000 7f6457fff700 1 -- 172.21.15.143:0/1509275065 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f646015e230 con 0x7f644c003f80 2023-11-08T19:28:08.788 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.780+0000 7f6457fff700 1 -- 172.21.15.143:0/1509275065 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f646006f750 con 0x7f644c004d90 2023-11-08T19:28:08.788 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.780+0000 7f6457fff700 1 -- 172.21.15.143:0/1509275065 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f644c00ba60 con 0x7f64600d1170 2023-11-08T19:28:08.789 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.780+0000 7f6464adb700 1 --2- 172.21.15.143:0/1509275065 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f644c003f80 0x7f644c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:08.789 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.780+0000 7f64652dc700 1 --2- 172.21.15.143:0/1509275065 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f64600d1170 0x7f644c003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:08.789 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.780+0000 7f6465add700 1 --2- 172.21.15.143:0/1509275065 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f644c004d90 0x7f644c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:08.789 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.780+0000 7f6464adb700 1 -- 172.21.15.143:0/1509275065 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f64600d1170 msgr2=0x7f644c003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:08.789 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.780+0000 7f6464adb700 1 --2- 172.21.15.143:0/1509275065 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f64600d1170 0x7f644c003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:08.789 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.780+0000 7f6464adb700 1 -- 172.21.15.143:0/1509275065 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f644c004d90 msgr2=0x7f644c005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:08.790 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.780+0000 7f6464adb700 1 --2- 172.21.15.143:0/1509275065 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f644c004d90 0x7f644c005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:08.790 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.780+0000 7f6464adb700 1 -- 172.21.15.143:0/1509275065 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f644c0035b0 con 0x7f644c003f80 2023-11-08T19:28:08.790 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.780+0000 7f6464adb700 1 --2- 172.21.15.143:0/1509275065 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f644c003f80 0x7f644c004850 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f645c009b90 tx=0x7f644c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd7b2725eaa5263f server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:08.790 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.780+0000 7f6457fff700 1 -- 172.21.15.143:0/1509275065 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f645c00e6c0 con 0x7f644c003f80 2023-11-08T19:28:08.790 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.780+0000 7f6457fff700 1 -- 172.21.15.143:0/1509275065 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f645c00e820 con 0x7f644c003f80 2023-11-08T19:28:08.791 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.780+0000 7f6457fff700 1 -- 172.21.15.143:0/1509275065 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f645c00eb90 con 0x7f644c003f80 2023-11-08T19:28:08.791 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.781+0000 7f6467540700 1 -- 172.21.15.143:0/1509275065 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f644c003f80 msgr2=0x7f644c004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:08.791 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.781+0000 7f6467540700 1 --2- 172.21.15.143:0/1509275065 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f644c003f80 0x7f644c004850 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f645c009b90 tx=0x7f644c005880 comp rx=0 tx=0).stop 2023-11-08T19:28:08.791 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.781+0000 7f6465add700 1 -- 172.21.15.143:0/1509275065 reap_dead start 2023-11-08T19:28:08.791 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.781+0000 7f6467540700 1 -- 172.21.15.143:0/1509275065 shutdown_connections 2023-11-08T19:28:08.792 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.781+0000 7f6467540700 1 -- 172.21.15.143:0/1509275065 >> 172.21.15.143:0/1509275065 conn(0x7f6460099a70 msgr2=0x7f64600d07c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:08.792 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.781+0000 7f6467540700 1 -- 172.21.15.143:0/1509275065 shutdown_connections 2023-11-08T19:28:08.792 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.781+0000 7f6467540700 1 -- 172.21.15.143:0/1509275065 wait complete. 2023-11-08T19:28:08.795 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.782+0000 7f6467540700 1 Processor -- start 2023-11-08T19:28:08.795 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.782+0000 7f6467540700 1 -- start start 2023-11-08T19:28:08.795 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.783+0000 7f6467540700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f644c003f80 0x7f64601f7240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:08.795 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.783+0000 7f6467540700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f64600c4260 0x7f64600c46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:08.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.783+0000 7f6467540700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f64600b33b0 0x7f64600b3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:08.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.783+0000 7f6467540700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64600b5560 con 0x7f64600c4260 2023-11-08T19:28:08.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.783+0000 7f6467540700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64600b3e50 con 0x7f644c003f80 2023-11-08T19:28:08.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.783+0000 7f6467540700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64600b4130 con 0x7f64600b33b0 2023-11-08T19:28:08.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.783+0000 7f6464adb700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f64600c4260 0x7f64600c46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:08.797 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.783+0000 7f6464adb700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f64600c4260 0x7f64600c46e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54684/0 (socket says 172.21.15.143:54684) 2023-11-08T19:28:08.797 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.783+0000 7f6464adb700 1 -- 172.21.15.143:0/556292911 learned_addr learned my addr 172.21.15.143:0/556292911 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:28:08.797 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.783+0000 7f6465add700 1 --2- 172.21.15.143:0/556292911 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f64600b33b0 0x7f64600b3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:08.797 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.783+0000 7f64652dc700 1 --2- 172.21.15.143:0/556292911 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f644c003f80 0x7f64601f7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:08.797 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.783+0000 7f6464adb700 1 -- 172.21.15.143:0/556292911 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f64600b33b0 msgr2=0x7f64600b3830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:08.798 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.783+0000 7f6464adb700 1 --2- 172.21.15.143:0/556292911 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f64600b33b0 0x7f64600b3830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:08.798 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.783+0000 7f6464adb700 1 -- 172.21.15.143:0/556292911 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f644c003f80 msgr2=0x7f64601f7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:08.798 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.783+0000 7f6464adb700 1 --2- 172.21.15.143:0/556292911 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f644c003f80 0x7f64601f7240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:08.798 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.783+0000 7f6464adb700 1 -- 172.21.15.143:0/556292911 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64600b51f0 con 0x7f64600c4260 2023-11-08T19:28:08.798 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.784+0000 7f6464adb700 1 --2- 172.21.15.143:0/556292911 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f64600c4260 0x7f64600c46e0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f644c004790 tx=0x7f645c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:08.798 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.784+0000 7f6455ffb700 1 -- 172.21.15.143:0/556292911 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f645c021070 con 0x7f64600c4260 2023-11-08T19:28:08.799 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.784+0000 7f6455ffb700 1 -- 172.21.15.143:0/556292911 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f645c004550 con 0x7f64600c4260 2023-11-08T19:28:08.799 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.784+0000 7f6467540700 1 -- 172.21.15.143:0/556292911 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64600b5560 con 0x7f64600c4260 2023-11-08T19:28:08.799 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.784+0000 7f6455ffb700 1 -- 172.21.15.143:0/556292911 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f645c00e6c0 con 0x7f64600c4260 2023-11-08T19:28:08.799 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.784+0000 7f6467540700 1 -- 172.21.15.143:0/556292911 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64601f6bb0 con 0x7f64600c4260 2023-11-08T19:28:08.799 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.785+0000 7f6467540700 1 -- 172.21.15.143:0/556292911 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64600b5050 con 0x7f64600c4260 2023-11-08T19:28:08.800 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.785+0000 7f6455ffb700 1 -- 172.21.15.143:0/556292911 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f645c025070 con 0x7f64600c4260 2023-11-08T19:28:08.800 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.786+0000 7f6455ffb700 1 --2- 172.21.15.143:0/556292911 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f644406af00 0x7f644406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:08.800 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.786+0000 7f6455ffb700 1 -- 172.21.15.143:0/556292911 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6772+0+0 (secure 0 0 0) 0x7f645c08aa30 con 0x7f64600c4260 2023-11-08T19:28:08.800 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.786+0000 7f64652dc700 1 --2- 172.21.15.143:0/556292911 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f644406af00 0x7f644406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:08.800 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.786+0000 7f64652dc700 1 --2- 172.21.15.143:0/556292911 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f644406af00 0x7f644406d3c0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f6450000c00 tx=0x7f6450009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:08.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:08.790+0000 7f6455ffb700 1 -- 172.21.15.143:0/556292911 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f64600b5050 con 0x7f64600c4260 2023-11-08T19:28:09.060 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:09.059+0000 7f6467540700 1 -- 172.21.15.143:0/556292911 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f64600b5050 con 0x7f64600c4260 2023-11-08T19:28:09.061 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:09.059+0000 7f6455ffb700 1 -- 172.21.15.143:0/556292911 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1662 (secure 0 0 0) 0x7f645c017690 con 0x7f64600c4260 2023-11-08T19:28:09.061 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:28:09.062 INFO:teuthology.orchestra.run.smithi143.stdout:{"fsid":"968483af-77fc-4068-ad7c-5c2a1a4613d7","health":{"status":"HEALTH_WARN","checks":{"POOL_APP_NOT_ENABLED":{"severity":"HEALTH_WARN","summary":{"message":"3 pool(s) do not have an application enabled","count":3},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":216,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":92,"num_osds":8,"num_up_osds":8,"osd_up_since":1699471483,"num_in_osds":8,"osd_in_since":1699471475,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":143},{"state_name":"creating+peering","count":50}],"num_pgs":193,"num_pools":6,"num_objects":10,"data_bytes":592162,"bytes_used":231444480,"bytes_avail":772862668800,"bytes_total":773094113280,"inactive_pgs_ratio":0.2590673565864563,"write_bytes_sec":1023,"read_op_per_sec":0,"write_op_per_sec":1},"fsmap":{"epoch":52,"id":6,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":6,"rank":0,"name":"d","status":"up:active","gid":6921}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2023-11-08T19:27:50.558120+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-11-08T19:28:09.063 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:09.062+0000 7f6467540700 1 -- 172.21.15.143:0/556292911 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f644406af00 msgr2=0x7f644406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:09.063 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:09.062+0000 7f6467540700 1 --2- 172.21.15.143:0/556292911 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f644406af00 0x7f644406d3c0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f6450000c00 tx=0x7f6450009040 comp rx=0 tx=0).stop 2023-11-08T19:28:09.064 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:09.062+0000 7f6467540700 1 -- 172.21.15.143:0/556292911 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f64600c4260 msgr2=0x7f64600c46e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:09.064 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:09.062+0000 7f6467540700 1 --2- 172.21.15.143:0/556292911 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f64600c4260 0x7f64600c46e0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f644c004790 tx=0x7f645c0047c0 comp rx=0 tx=0).stop 2023-11-08T19:28:09.064 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:09.062+0000 7f6467540700 1 -- 172.21.15.143:0/556292911 shutdown_connections 2023-11-08T19:28:09.064 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:09.062+0000 7f6467540700 1 --2- 172.21.15.143:0/556292911 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f644406af00 0x7f644406d3c0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:09.064 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:09.062+0000 7f6467540700 1 --2- 172.21.15.143:0/556292911 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f64600b33b0 0x7f64600b3830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:09.065 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:09.062+0000 7f6467540700 1 --2- 172.21.15.143:0/556292911 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f644c003f80 0x7f64601f7240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:09.065 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:09.062+0000 7f6467540700 1 --2- 172.21.15.143:0/556292911 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f64600c4260 0x7f64600c46e0 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:09.065 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:09.062+0000 7f6467540700 1 -- 172.21.15.143:0/556292911 >> 172.21.15.143:0/556292911 conn(0x7f6460099a70 msgr2=0x7f6460152ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:09.065 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:09.062+0000 7f6467540700 1 -- 172.21.15.143:0/556292911 shutdown_connections 2023-11-08T19:28:09.065 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:09.062+0000 7f6467540700 1 -- 172.21.15.143:0/556292911 wait complete. 2023-11-08T19:28:09.075 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-11-08T19:28:09.075 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-08T19:28:09.195 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:09.194+0000 7fa200e11700 1 Processor -- start 2023-11-08T19:28:09.196 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:09.194+0000 7fa200e11700 1 -- start start 2023-11-08T19:28:09.198 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:09.195+0000 7fa200e11700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa1fc0b2920 0x7fa1fc0cf510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:09.198 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:09.195+0000 7fa200e11700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fa1fc0cbe20 0x7fa1fc0cc290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:09.199 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:09.195+0000 7fa200e11700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1fc0cf080 con 0x7fa1fc15aa30 2023-11-08T19:28:09.199 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:09.195+0000 7fa200e11700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1fc0cc7d0 con 0x7fa1fc0b2920 2023-11-08T19:28:09.199 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:09.195+0000 7fa200e11700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1fc0ce470 con 0x7fa1fc0cbe20 2023-11-08T19:28:09.199 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:09.195+0000 7fa1f9d9b700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa1fc0b2920 0x7fa1fc0cf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:09.199 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:09.195+0000 7fa1f9d9b700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa1fc0b2920 0x7fa1fc0cf510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54692/0 (socket says 172.21.15.143:54692) 2023-11-08T19:28:09.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:09.195+0000 7fa1f9d9b700 1 -- 172.21.15.143:0/1741313999 learned_addr learned my addr 172.21.15.143:0/1741313999 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:28:09.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:09.195+0000 7fa1fad9d700 1 --2- 172.21.15.143:0/1741313999 >> v2:172.21.15.174:3300/0 conn(0x7fa1fc0cbe20 0x7fa1fc0cc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:09.201 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:09.196+0000 7fa1f9d9b700 1 -- 172.21.15.143:0/1741313999 >> v2:172.21.15.174:3300/0 conn(0x7fa1fc0cbe20 msgr2=0x7fa1fc0cc290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:09.201 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:09.196+0000 7fa1f9d9b700 1 --2- 172.21.15.143:0/1741313999 >> v2:172.21.15.174:3300/0 conn(0x7fa1fc0cbe20 0x7fa1fc0cc290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:09.201 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:09.196+0000 7fa1f9d9b700 1 -- 172.21.15.143:0/1741313999 >> v1:172.21.15.174:6789/0 conn(0x7fa1fc15aa30 legacy=0x7fa1fc0b2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:09.201 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:09.196+0000 7fa1f9d9b700 1 -- 172.21.15.143:0/1741313999 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1fc0ce750 con 0x7fa1fc0b2920 2023-11-08T19:28:09.201 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:09.196+0000 7fa1f9d9b700 1 --2- 172.21.15.143:0/1741313999 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa1fc0b2920 0x7fa1fc0cf510 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc070a10 tx=0x7fa1ec009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfc50f60e5d87aa6 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:09.202 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:09.196+0000 7fa1f959a700 1 -- 172.21.15.143:0/1741313999 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1ec011830 con 0x7fa1fc0b2920 2023-11-08T19:28:09.202 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:09.196+0000 7fa1f959a700 1 -- 172.21.15.143:0/1741313999 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1ec011990 con 0x7fa1fc0b2920 2023-11-08T19:28:09.202 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:09.196+0000 7fa1f959a700 1 -- 172.21.15.143:0/1741313999 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1ec011d20 con 0x7fa1fc0b2920 2023-11-08T19:28:09.202 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:09.196+0000 7fa200e11700 1 -- 172.21.15.143:0/1741313999 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa1fc0b2920 msgr2=0x7fa1fc0cf510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:09.202 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:09.196+0000 7fa200e11700 1 --2- 172.21.15.143:0/1741313999 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa1fc0b2920 0x7fa1fc0cf510 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc070a10 tx=0x7fa1ec009bd0 comp rx=0 tx=0).stop 2023-11-08T19:28:09.203 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:09.197+0000 7fa200e11700 1 -- 172.21.15.143:0/1741313999 shutdown_connections 2023-11-08T19:28:09.203 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:09.197+0000 7fa200e11700 1 --2- 172.21.15.143:0/1741313999 >> v2:172.21.15.174:3300/0 conn(0x7fa1fc0cbe20 0x7fa1fc0cc290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:09.203 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:09.197+0000 7fa200e11700 1 --2- 172.21.15.143:0/1741313999 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa1fc0b2920 0x7fa1fc0cf510 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:09.203 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:09.197+0000 7fa200e11700 1 -- 172.21.15.143:0/1741313999 >> 172.21.15.143:0/1741313999 conn(0x7fa1fc099ab0 msgr2=0x7fa1fc0d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:09.203 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:09.197+0000 7fa200e11700 1 -- 172.21.15.143:0/1741313999 shutdown_connections 2023-11-08T19:28:09.204 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:09.197+0000 7fa200e11700 1 -- 172.21.15.143:0/1741313999 wait complete. 2023-11-08T19:28:09.206 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:09.198+0000 7fa200e11700 1 Processor -- start 2023-11-08T19:28:09.207 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:09.198+0000 7fa200e11700 1 -- start start 2023-11-08T19:28:09.207 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:09.199+0000 7fa200e11700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa1fc0b2920 0x7fa1fc0b5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:09.207 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:09.199+0000 7fa200e11700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa1fc0cbe20 0x7fa1fc1f4a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:09.207 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:09.199+0000 7fa200e11700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa1fc15aa30 0x7fa1fc06ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:09.207 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:09.199+0000 7fa200e11700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1fc0b5130 con 0x7fa1fc0cbe20 2023-11-08T19:28:09.208 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:09.199+0000 7fa200e11700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1fc1f3840 con 0x7fa1fc0b2920 2023-11-08T19:28:09.208 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:09.199+0000 7fa200e11700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1fc0c8d20 con 0x7fa1fc15aa30 2023-11-08T19:28:09.208 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:09.199+0000 7fa1f9d9b700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa1fc0cbe20 0x7fa1fc1f4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:09.208 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:09.199+0000 7fa1f9d9b700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa1fc0cbe20 0x7fa1fc1f4a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54698/0 (socket says 172.21.15.143:54698) 2023-11-08T19:28:09.208 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:09.199+0000 7fa1f9d9b700 1 -- 172.21.15.143:0/3653006161 learned_addr learned my addr 172.21.15.143:0/3653006161 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:28:09.209 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:09.199+0000 7fa1fa59c700 1 --2- 172.21.15.143:0/3653006161 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa1fc0b2920 0x7fa1fc0b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:09.209 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:09.199+0000 7fa1fad9d700 1 --2- 172.21.15.143:0/3653006161 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa1fc15aa30 0x7fa1fc06ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:09.209 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:09.199+0000 7fa1f9d9b700 1 -- 172.21.15.143:0/3653006161 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa1fc15aa30 msgr2=0x7fa1fc06ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:09.209 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:09.199+0000 7fa1f9d9b700 1 --2- 172.21.15.143:0/3653006161 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa1fc15aa30 0x7fa1fc06ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:09.209 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:09.199+0000 7fa1f9d9b700 1 -- 172.21.15.143:0/3653006161 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa1fc0b2920 msgr2=0x7fa1fc0b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:09.210 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:09.199+0000 7fa1f9d9b700 1 --2- 172.21.15.143:0/3653006161 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa1fc0b2920 0x7fa1fc0b5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:09.210 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:09.199+0000 7fa1f9d9b700 1 -- 172.21.15.143:0/3653006161 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1ec009550 con 0x7fa1fc0cbe20 2023-11-08T19:28:09.210 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:09.199+0000 7fa1f9d9b700 1 --2- 172.21.15.143:0/3653006161 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa1fc0cbe20 0x7fa1fc1f4a00 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec000c00 tx=0x7fa1ec004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:09.210 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:09.199+0000 7fa1eaffd700 1 -- 172.21.15.143:0/3653006161 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1ec004380 con 0x7fa1fc0cbe20 2023-11-08T19:28:09.210 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:09.199+0000 7fa1eaffd700 1 -- 172.21.15.143:0/3653006161 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1ec01f070 con 0x7fa1fc0cbe20 2023-11-08T19:28:09.211 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:09.199+0000 7fa1eaffd700 1 -- 172.21.15.143:0/3653006161 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1ec0036a0 con 0x7fa1fc0cbe20 2023-11-08T19:28:09.211 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:09.199+0000 7fa200e11700 1 -- 172.21.15.143:0/3653006161 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1fc0c9000 con 0x7fa1fc0cbe20 2023-11-08T19:28:09.211 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:09.199+0000 7fa200e11700 1 -- 172.21.15.143:0/3653006161 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1fc0c95c0 con 0x7fa1fc0cbe20 2023-11-08T19:28:09.212 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:09.200+0000 7fa200e11700 1 -- 172.21.15.143:0/3653006161 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1fc1f3a60 con 0x7fa1fc0cbe20 2023-11-08T19:28:09.212 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:09.201+0000 7fa1eaffd700 1 -- 172.21.15.143:0/3653006161 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa1ec021030 con 0x7fa1fc0cbe20 2023-11-08T19:28:09.212 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:09.201+0000 7fa1eaffd700 1 --2- 172.21.15.143:0/3653006161 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa1dc06af00 0x7fa1dc06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:09.212 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:09.201+0000 7fa1eaffd700 1 -- 172.21.15.143:0/3653006161 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6772+0+0 (secure 0 0 0) 0x7fa1ec08adb0 con 0x7fa1fc0cbe20 2023-11-08T19:28:09.212 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:09.201+0000 7fa1fa59c700 1 --2- 172.21.15.143:0/3653006161 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa1dc06af00 0x7fa1dc06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:09.213 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:09.202+0000 7fa1fa59c700 1 --2- 172.21.15.143:0/3653006161 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa1dc06af00 0x7fa1dc06d3c0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fa1f0000fb0 tx=0x7fa1f0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:09.213 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:09.206+0000 7fa1eaffd700 1 -- 172.21.15.143:0/3653006161 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa1fc1f3a60 con 0x7fa1fc0cbe20 2023-11-08T19:28:09.393 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:09.391+0000 7fa200e11700 1 -- 172.21.15.143:0/3653006161 --> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa1fc0ced10 con 0x7fa1dc06af00 2023-11-08T19:28:09.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:09.407+0000 7fa1eaffd700 1 -- 172.21.15.143:0/3653006161 <== mgr.4099 v2:172.21.15.174:6832/4037280178 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+528138 (secure 0 0 0) 0x7fa1f0001cd0 con 0x7fa1dc06af00 2023-11-08T19:28:09.414 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:28:09.419 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:09.411+0000 7fa200e11700 1 -- 172.21.15.143:0/3653006161 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa1dc06af00 msgr2=0x7fa1dc06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:09.419 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:09.411+0000 7fa200e11700 1 --2- 172.21.15.143:0/3653006161 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa1dc06af00 0x7fa1dc06d3c0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fa1f0000fb0 tx=0x7fa1f0009040 comp rx=0 tx=0).stop 2023-11-08T19:28:09.419 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:09.411+0000 7fa200e11700 1 -- 172.21.15.143:0/3653006161 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa1fc0cbe20 msgr2=0x7fa1fc1f4a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:09.419 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:09.411+0000 7fa200e11700 1 --2- 172.21.15.143:0/3653006161 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa1fc0cbe20 0x7fa1fc1f4a00 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec000c00 tx=0x7fa1ec004750 comp rx=0 tx=0).stop 2023-11-08T19:28:09.420 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:09.411+0000 7fa200e11700 1 -- 172.21.15.143:0/3653006161 shutdown_connections 2023-11-08T19:28:09.420 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:09.411+0000 7fa200e11700 1 --2- 172.21.15.143:0/3653006161 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa1dc06af00 0x7fa1dc06d3c0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:09.420 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:09.411+0000 7fa200e11700 1 --2- 172.21.15.143:0/3653006161 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa1fc15aa30 0x7fa1fc06ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:09.420 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:09.411+0000 7fa200e11700 1 --2- 172.21.15.143:0/3653006161 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa1fc0b2920 0x7fa1fc0b5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:09.420 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:09.411+0000 7fa200e11700 1 --2- 172.21.15.143:0/3653006161 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa1fc0cbe20 0x7fa1fc1f4a00 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:09.421 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:09.411+0000 7fa200e11700 1 -- 172.21.15.143:0/3653006161 >> 172.21.15.143:0/3653006161 conn(0x7fa1fc099ab0 msgr2=0x7fa1fc06ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:09.421 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:09.411+0000 7fa200e11700 1 -- 172.21.15.143:0/3653006161 shutdown_connections 2023-11-08T19:28:09.421 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:09.411+0000 7fa200e11700 1 -- 172.21.15.143:0/3653006161 wait complete. 2023-11-08T19:28:09.421 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2023-11-08T19:28:09.424 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":187,"stamp":"2023-11-08T19:28:08.562651+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592162,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":171,"num_write_kb":2422,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":129,"ondisk_log_size":129,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":236,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":226020,"kb_used_data":9840,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754748700,"statfs":{"total":773094113280,"available":772862668800,"internally_reserved":0,"allocated":10076160,"data_stored":5344818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1794,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001373"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.701553+0000","last_change":"2023-11-08T19:28:02.701553+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.701553+0000","last_undegraded":"2023-11-08T19:28:02.701553+0000","last_fullsized":"2023-11-08T19:28:02.701553+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.719263+0000","last_change":"2023-11-08T19:28:03.715294+0000","last_active":"2023-11-08T19:28:03.719263+0000","last_peered":"2023-11-08T19:28:03.719263+0000","last_clean":"2023-11-08T19:28:03.719263+0000","last_became_active":"2023-11-08T19:28:03.715007+0000","last_became_peered":"2023-11-08T19:28:03.715007+0000","last_unstale":"2023-11-08T19:28:03.719263+0000","last_undegraded":"2023-11-08T19:28:03.719263+0000","last_fullsized":"2023-11-08T19:28:03.719263+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:51:46.879726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.700726+0000","last_change":"2023-11-08T19:28:02.700726+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.700726+0000","last_undegraded":"2023-11-08T19:28:02.700726+0000","last_fullsized":"2023-11-08T19:28:02.700726+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.699799+0000","last_change":"2023-11-08T19:28:02.699799+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.699799+0000","last_undegraded":"2023-11-08T19:28:02.699799+0000","last_fullsized":"2023-11-08T19:28:02.699799+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.704338+0000","last_change":"2023-11-08T19:28:02.704338+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.704338+0000","last_undegraded":"2023-11-08T19:28:02.704338+0000","last_fullsized":"2023-11-08T19:28:02.704338+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.722681+0000","last_change":"2023-11-08T19:28:03.718208+0000","last_active":"2023-11-08T19:28:03.722681+0000","last_peered":"2023-11-08T19:28:03.722681+0000","last_clean":"2023-11-08T19:28:03.722681+0000","last_became_active":"2023-11-08T19:28:03.716630+0000","last_became_peered":"2023-11-08T19:28:03.716630+0000","last_unstale":"2023-11-08T19:28:03.722681+0000","last_undegraded":"2023-11-08T19:28:03.722681+0000","last_fullsized":"2023-11-08T19:28:03.722681+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:21:24.678157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.712530+0000","last_change":"2023-11-08T19:28:03.708477+0000","last_active":"2023-11-08T19:28:03.712530+0000","last_peered":"2023-11-08T19:28:03.712530+0000","last_clean":"2023-11-08T19:28:03.712530+0000","last_became_active":"2023-11-08T19:28:03.708188+0000","last_became_peered":"2023-11-08T19:28:03.708188+0000","last_unstale":"2023-11-08T19:28:03.712530+0000","last_undegraded":"2023-11-08T19:28:03.712530+0000","last_fullsized":"2023-11-08T19:28:03.712530+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:57:58.571556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.714751+0000","last_change":"2023-11-08T19:28:02.714751+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.714751+0000","last_undegraded":"2023-11-08T19:28:02.714751+0000","last_fullsized":"2023-11-08T19:28:02.714751+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.722749+0000","last_change":"2023-11-08T19:28:03.718615+0000","last_active":"2023-11-08T19:28:03.722749+0000","last_peered":"2023-11-08T19:28:03.722749+0000","last_clean":"2023-11-08T19:28:03.722749+0000","last_became_active":"2023-11-08T19:28:03.718295+0000","last_became_peered":"2023-11-08T19:28:03.718295+0000","last_unstale":"2023-11-08T19:28:03.722749+0000","last_undegraded":"2023-11-08T19:28:03.722749+0000","last_fullsized":"2023-11-08T19:28:03.722749+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:23:12.882243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.702139+0000","last_change":"2023-11-08T19:28:02.702139+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.702139+0000","last_undegraded":"2023-11-08T19:28:02.702139+0000","last_fullsized":"2023-11-08T19:28:02.702139+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.705435+0000","last_change":"2023-11-08T19:28:02.705435+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.705435+0000","last_undegraded":"2023-11-08T19:28:02.705435+0000","last_fullsized":"2023-11-08T19:28:02.705435+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.703962+0000","last_change":"2023-11-08T19:28:02.703962+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.703962+0000","last_undegraded":"2023-11-08T19:28:02.703962+0000","last_fullsized":"2023-11-08T19:28:02.703962+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.697723+0000","last_change":"2023-11-08T19:28:02.697723+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.697723+0000","last_undegraded":"2023-11-08T19:28:02.697723+0000","last_fullsized":"2023-11-08T19:28:02.697723+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.721906+0000","last_change":"2023-11-08T19:28:03.718346+0000","last_active":"2023-11-08T19:28:03.721906+0000","last_peered":"2023-11-08T19:28:03.721906+0000","last_clean":"2023-11-08T19:28:03.721906+0000","last_became_active":"2023-11-08T19:28:03.718068+0000","last_became_peered":"2023-11-08T19:28:03.718068+0000","last_unstale":"2023-11-08T19:28:03.721906+0000","last_undegraded":"2023-11-08T19:28:03.721906+0000","last_fullsized":"2023-11-08T19:28:03.721906+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:29:02.099134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.721945+0000","last_change":"2023-11-08T19:28:03.717576+0000","last_active":"2023-11-08T19:28:03.721945+0000","last_peered":"2023-11-08T19:28:03.721945+0000","last_clean":"2023-11-08T19:28:03.721945+0000","last_became_active":"2023-11-08T19:28:03.715708+0000","last_became_peered":"2023-11-08T19:28:03.715708+0000","last_unstale":"2023-11-08T19:28:03.721945+0000","last_undegraded":"2023-11-08T19:28:03.721945+0000","last_fullsized":"2023-11-08T19:28:03.721945+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:40:03.163555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.711516+0000","last_change":"2023-11-08T19:28:03.703641+0000","last_active":"2023-11-08T19:28:03.711516+0000","last_peered":"2023-11-08T19:28:03.711516+0000","last_clean":"2023-11-08T19:28:03.711516+0000","last_became_active":"2023-11-08T19:28:03.703331+0000","last_became_peered":"2023-11-08T19:28:03.703331+0000","last_unstale":"2023-11-08T19:28:03.711516+0000","last_undegraded":"2023-11-08T19:28:03.711516+0000","last_fullsized":"2023-11-08T19:28:03.711516+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:32:03.694798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.700364+0000","last_change":"2023-11-08T19:28:02.700364+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.700364+0000","last_undegraded":"2023-11-08T19:28:02.700364+0000","last_fullsized":"2023-11-08T19:28:02.700364+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.719244+0000","last_change":"2023-11-08T19:28:03.715221+0000","last_active":"2023-11-08T19:28:03.719244+0000","last_peered":"2023-11-08T19:28:03.719244+0000","last_clean":"2023-11-08T19:28:03.719244+0000","last_became_active":"2023-11-08T19:28:03.714929+0000","last_became_peered":"2023-11-08T19:28:03.714929+0000","last_unstale":"2023-11-08T19:28:03.719244+0000","last_undegraded":"2023-11-08T19:28:03.719244+0000","last_fullsized":"2023-11-08T19:28:03.719244+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:58:49.797420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.700365+0000","last_change":"2023-11-08T19:28:02.700365+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.700365+0000","last_undegraded":"2023-11-08T19:28:02.700365+0000","last_fullsized":"2023-11-08T19:28:02.700365+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.704383+0000","last_change":"2023-11-08T19:28:02.704383+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.704383+0000","last_undegraded":"2023-11-08T19:28:02.704383+0000","last_fullsized":"2023-11-08T19:28:02.704383+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.713555+0000","last_change":"2023-11-08T19:28:02.713555+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.713555+0000","last_undegraded":"2023-11-08T19:28:02.713555+0000","last_fullsized":"2023-11-08T19:28:02.713555+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.702640+0000","last_change":"2023-11-08T19:28:02.702640+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.702640+0000","last_undegraded":"2023-11-08T19:28:02.702640+0000","last_fullsized":"2023-11-08T19:28:02.702640+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.699841+0000","last_change":"2023-11-08T19:28:02.699841+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.699841+0000","last_undegraded":"2023-11-08T19:28:02.699841+0000","last_fullsized":"2023-11-08T19:28:02.699841+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.702861+0000","last_change":"2023-11-08T19:28:02.702861+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.702861+0000","last_undegraded":"2023-11-08T19:28:02.702861+0000","last_fullsized":"2023-11-08T19:28:02.702861+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.707625+0000","last_change":"2023-11-08T19:28:02.707625+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.707625+0000","last_undegraded":"2023-11-08T19:28:02.707625+0000","last_fullsized":"2023-11-08T19:28:02.707625+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.710228+0000","last_change":"2023-11-08T19:28:02.710228+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.710228+0000","last_undegraded":"2023-11-08T19:28:02.710228+0000","last_fullsized":"2023-11-08T19:28:02.710228+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.704524+0000","last_change":"2023-11-08T19:28:02.704524+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.704524+0000","last_undegraded":"2023-11-08T19:28:02.704524+0000","last_fullsized":"2023-11-08T19:28:02.704524+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.725514+0000","last_change":"2023-11-08T19:28:03.721162+0000","last_active":"2023-11-08T19:28:03.725514+0000","last_peered":"2023-11-08T19:28:03.725514+0000","last_clean":"2023-11-08T19:28:03.725514+0000","last_became_active":"2023-11-08T19:28:03.720956+0000","last_became_peered":"2023-11-08T19:28:03.720956+0000","last_unstale":"2023-11-08T19:28:03.725514+0000","last_undegraded":"2023-11-08T19:28:03.725514+0000","last_fullsized":"2023-11-08T19:28:03.725514+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:09:19.883657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.712428+0000","last_change":"2023-11-08T19:28:02.712428+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.712428+0000","last_undegraded":"2023-11-08T19:28:02.712428+0000","last_fullsized":"2023-11-08T19:28:02.712428+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.705928+0000","last_change":"2023-11-08T19:28:02.705928+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.705928+0000","last_undegraded":"2023-11-08T19:28:02.705928+0000","last_fullsized":"2023-11-08T19:28:02.705928+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.712510+0000","last_change":"2023-11-08T19:28:03.707881+0000","last_active":"2023-11-08T19:28:03.712510+0000","last_peered":"2023-11-08T19:28:03.712510+0000","last_clean":"2023-11-08T19:28:03.712510+0000","last_became_active":"2023-11-08T19:28:03.707569+0000","last_became_peered":"2023-11-08T19:28:03.707569+0000","last_unstale":"2023-11-08T19:28:03.712510+0000","last_undegraded":"2023-11-08T19:28:03.712510+0000","last_fullsized":"2023-11-08T19:28:03.712510+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:00:53.075545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.711643+0000","last_change":"2023-11-08T19:28:02.711643+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.711643+0000","last_undegraded":"2023-11-08T19:28:02.711643+0000","last_fullsized":"2023-11-08T19:28:02.711643+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.702089+0000","last_change":"2023-11-08T19:28:02.702089+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.702089+0000","last_undegraded":"2023-11-08T19:28:02.702089+0000","last_fullsized":"2023-11-08T19:28:02.702089+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.725513+0000","last_change":"2023-11-08T19:28:02.720536+0000","last_active":"2023-11-08T19:28:02.725513+0000","last_peered":"2023-11-08T19:28:02.725513+0000","last_clean":"2023-11-08T19:28:02.725513+0000","last_became_active":"2023-11-08T19:28:02.720236+0000","last_became_peered":"2023-11-08T19:28:02.720236+0000","last_unstale":"2023-11-08T19:28:02.725513+0000","last_undegraded":"2023-11-08T19:28:02.725513+0000","last_fullsized":"2023-11-08T19:28:02.725513+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:56:59.813550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.700732+0000","last_change":"2023-11-08T19:27:08.190826+0000","last_active":"2023-11-08T19:28:02.700732+0000","last_peered":"2023-11-08T19:28:02.700732+0000","last_clean":"2023-11-08T19:28:02.700732+0000","last_became_active":"2023-11-08T19:27:08.190440+0000","last_became_peered":"2023-11-08T19:27:08.190440+0000","last_unstale":"2023-11-08T19:28:02.700732+0000","last_undegraded":"2023-11-08T19:28:02.700732+0000","last_fullsized":"2023-11-08T19:28:02.700732+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:42:48.376462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":237,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.699373+0000","last_change":"2023-11-08T19:24:45.888122+0000","last_active":"2023-11-08T19:28:02.699373+0000","last_peered":"2023-11-08T19:28:02.699373+0000","last_clean":"2023-11-08T19:28:02.699373+0000","last_became_active":"2023-11-08T19:24:45.887048+0000","last_became_peered":"2023-11-08T19:24:45.887048+0000","last_unstale":"2023-11-08T19:28:02.699373+0000","last_undegraded":"2023-11-08T19:28:02.699373+0000","last_fullsized":"2023-11-08T19:28:02.699373+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:44.575133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:44.575133+0000","last_clean_scrub_stamp":"2023-11-08T19:24:44.575133+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:19:20.202062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":60,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.698526+0000","last_change":"2023-11-08T19:26:06.542044+0000","last_active":"2023-11-08T19:28:01.698526+0000","last_peered":"2023-11-08T19:28:01.698526+0000","last_clean":"2023-11-08T19:28:01.698526+0000","last_became_active":"2023-11-08T19:26:04.603056+0000","last_became_peered":"2023-11-08T19:26:04.603056+0000","last_unstale":"2023-11-08T19:28:01.698526+0000","last_undegraded":"2023-11-08T19:28:01.698526+0000","last_fullsized":"2023-11-08T19:28:01.698526+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:41:50.771457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000517414,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.694743+0000","last_change":"2023-11-08T19:27:09.496842+0000","last_active":"2023-11-08T19:28:01.694743+0000","last_peered":"2023-11-08T19:28:01.694743+0000","last_clean":"2023-11-08T19:28:01.694743+0000","last_became_active":"2023-11-08T19:27:07.185776+0000","last_became_peered":"2023-11-08T19:27:07.185776+0000","last_unstale":"2023-11-08T19:28:01.694743+0000","last_undegraded":"2023-11-08T19:28:01.694743+0000","last_fullsized":"2023-11-08T19:28:01.694743+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:21:13.923970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031747200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.698950+0000","last_change":"2023-11-08T19:28:02.698950+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.698950+0000","last_undegraded":"2023-11-08T19:28:02.698950+0000","last_fullsized":"2023-11-08T19:28:02.698950+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.725538+0000","last_change":"2023-11-08T19:28:02.720512+0000","last_active":"2023-11-08T19:28:02.725538+0000","last_peered":"2023-11-08T19:28:02.725538+0000","last_clean":"2023-11-08T19:28:02.725538+0000","last_became_active":"2023-11-08T19:28:02.720202+0000","last_became_peered":"2023-11-08T19:28:02.720202+0000","last_unstale":"2023-11-08T19:28:02.725538+0000","last_undegraded":"2023-11-08T19:28:02.725538+0000","last_fullsized":"2023-11-08T19:28:02.725538+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:26:10.008919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.694327+0000","last_change":"2023-11-08T19:27:08.188241+0000","last_active":"2023-11-08T19:28:01.694327+0000","last_peered":"2023-11-08T19:28:01.694327+0000","last_clean":"2023-11-08T19:28:01.694327+0000","last_became_active":"2023-11-08T19:27:08.187863+0000","last_became_peered":"2023-11-08T19:27:08.187863+0000","last_unstale":"2023-11-08T19:28:01.694327+0000","last_undegraded":"2023-11-08T19:28:01.694327+0000","last_fullsized":"2023-11-08T19:28:01.694327+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:45:22.084015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":58,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.696493+0000","last_change":"2023-11-08T19:26:06.265419+0000","last_active":"2023-11-08T19:28:01.696493+0000","last_peered":"2023-11-08T19:28:01.696493+0000","last_clean":"2023-11-08T19:28:01.696493+0000","last_became_active":"2023-11-08T19:26:04.605333+0000","last_became_peered":"2023-11-08T19:26:04.605333+0000","last_unstale":"2023-11-08T19:28:01.696493+0000","last_undegraded":"2023-11-08T19:28:01.696493+0000","last_fullsized":"2023-11-08T19:28:01.696493+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:01:56.991346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064606400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.701529+0000","last_change":"2023-11-08T19:27:09.182113+0000","last_active":"2023-11-08T19:28:02.701529+0000","last_peered":"2023-11-08T19:28:02.701529+0000","last_clean":"2023-11-08T19:28:02.701529+0000","last_became_active":"2023-11-08T19:27:07.189939+0000","last_became_peered":"2023-11-08T19:27:07.189939+0000","last_unstale":"2023-11-08T19:28:02.701529+0000","last_undegraded":"2023-11-08T19:28:02.701529+0000","last_fullsized":"2023-11-08T19:28:02.701529+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:41:08.822366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036507500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.699139+0000","last_change":"2023-11-08T19:28:02.699139+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.699139+0000","last_undegraded":"2023-11-08T19:28:02.699139+0000","last_fullsized":"2023-11-08T19:28:02.699139+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.725537+0000","last_change":"2023-11-08T19:28:02.720708+0000","last_active":"2023-11-08T19:28:02.725537+0000","last_peered":"2023-11-08T19:28:02.725537+0000","last_clean":"2023-11-08T19:28:02.725537+0000","last_became_active":"2023-11-08T19:28:02.720263+0000","last_became_peered":"2023-11-08T19:28:02.720263+0000","last_unstale":"2023-11-08T19:28:02.725537+0000","last_undegraded":"2023-11-08T19:28:02.725537+0000","last_fullsized":"2023-11-08T19:28:02.725537+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:56:32.169912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.691014+0000","last_change":"2023-11-08T19:27:08.187127+0000","last_active":"2023-11-08T19:28:01.691014+0000","last_peered":"2023-11-08T19:28:01.691014+0000","last_clean":"2023-11-08T19:28:01.691014+0000","last_became_active":"2023-11-08T19:27:08.182648+0000","last_became_peered":"2023-11-08T19:27:08.182648+0000","last_unstale":"2023-11-08T19:28:01.691014+0000","last_undegraded":"2023-11-08T19:28:01.691014+0000","last_fullsized":"2023-11-08T19:28:01.691014+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:50:12.101878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":71,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.700875+0000","last_change":"2023-11-08T19:26:05.609570+0000","last_active":"2023-11-08T19:28:02.700875+0000","last_peered":"2023-11-08T19:28:02.700875+0000","last_clean":"2023-11-08T19:28:02.700875+0000","last_became_active":"2023-11-08T19:26:04.608379+0000","last_became_peered":"2023-11-08T19:26:04.608379+0000","last_unstale":"2023-11-08T19:28:02.700875+0000","last_undegraded":"2023-11-08T19:28:02.700875+0000","last_fullsized":"2023-11-08T19:28:02.700875+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:40:08.986649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000529657,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.696937+0000","last_change":"2023-11-08T19:27:09.211991+0000","last_active":"2023-11-08T19:28:01.696937+0000","last_peered":"2023-11-08T19:28:01.696937+0000","last_clean":"2023-11-08T19:28:01.696937+0000","last_became_active":"2023-11-08T19:27:07.195231+0000","last_became_peered":"2023-11-08T19:27:07.195231+0000","last_unstale":"2023-11-08T19:28:01.696937+0000","last_undegraded":"2023-11-08T19:28:01.696937+0000","last_fullsized":"2023-11-08T19:28:01.696937+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:25:14.487524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000441221,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.697661+0000","last_change":"2023-11-08T19:28:02.697661+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.697661+0000","last_undegraded":"2023-11-08T19:28:02.697661+0000","last_fullsized":"2023-11-08T19:28:02.697661+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.705119+0000","last_change":"2023-11-08T19:28:02.707543+0000","last_active":"2023-11-08T19:28:03.705119+0000","last_peered":"2023-11-08T19:28:03.705119+0000","last_clean":"2023-11-08T19:28:03.705119+0000","last_became_active":"2023-11-08T19:28:02.707034+0000","last_became_peered":"2023-11-08T19:28:02.707034+0000","last_unstale":"2023-11-08T19:28:03.705119+0000","last_undegraded":"2023-11-08T19:28:03.705119+0000","last_fullsized":"2023-11-08T19:28:03.705119+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:06:46.912027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.690189+0000","last_change":"2023-11-08T19:27:08.195364+0000","last_active":"2023-11-08T19:28:01.690189+0000","last_peered":"2023-11-08T19:28:01.690189+0000","last_clean":"2023-11-08T19:28:01.690189+0000","last_became_active":"2023-11-08T19:27:08.195074+0000","last_became_peered":"2023-11-08T19:27:08.195074+0000","last_unstale":"2023-11-08T19:28:01.690189+0000","last_undegraded":"2023-11-08T19:28:01.690189+0000","last_fullsized":"2023-11-08T19:28:01.690189+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:02:57.552873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":59,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.700695+0000","last_change":"2023-11-08T19:26:06.394300+0000","last_active":"2023-11-08T19:28:02.700695+0000","last_peered":"2023-11-08T19:28:02.700695+0000","last_clean":"2023-11-08T19:28:02.700695+0000","last_became_active":"2023-11-08T19:26:04.601685+0000","last_became_peered":"2023-11-08T19:26:04.601685+0000","last_unstale":"2023-11-08T19:28:02.700695+0000","last_undegraded":"2023-11-08T19:28:02.700695+0000","last_fullsized":"2023-11-08T19:28:02.700695+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:51:06.626474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061044099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.690413+0000","last_change":"2023-11-08T19:27:09.178466+0000","last_active":"2023-11-08T19:28:01.690413+0000","last_peered":"2023-11-08T19:28:01.690413+0000","last_clean":"2023-11-08T19:28:01.690413+0000","last_became_active":"2023-11-08T19:27:07.189386+0000","last_became_peered":"2023-11-08T19:27:07.189386+0000","last_unstale":"2023-11-08T19:28:01.690413+0000","last_undegraded":"2023-11-08T19:28:01.690413+0000","last_fullsized":"2023-11-08T19:28:01.690413+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:48:51.139820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049390100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.713309+0000","last_change":"2023-11-08T19:28:02.713309+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.713309+0000","last_undegraded":"2023-11-08T19:28:02.713309+0000","last_fullsized":"2023-11-08T19:28:02.713309+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.727972+0000","last_change":"2023-11-08T19:28:02.723021+0000","last_active":"2023-11-08T19:28:02.727972+0000","last_peered":"2023-11-08T19:28:02.727972+0000","last_clean":"2023-11-08T19:28:02.727972+0000","last_became_active":"2023-11-08T19:28:02.722748+0000","last_became_peered":"2023-11-08T19:28:02.722748+0000","last_unstale":"2023-11-08T19:28:02.727972+0000","last_undegraded":"2023-11-08T19:28:02.727972+0000","last_fullsized":"2023-11-08T19:28:02.727972+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:26:52.310010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.695135+0000","last_change":"2023-11-08T19:27:08.189452+0000","last_active":"2023-11-08T19:28:01.695135+0000","last_peered":"2023-11-08T19:28:01.695135+0000","last_clean":"2023-11-08T19:28:01.695135+0000","last_became_active":"2023-11-08T19:27:08.189088+0000","last_became_peered":"2023-11-08T19:27:08.189088+0000","last_unstale":"2023-11-08T19:28:01.695135+0000","last_undegraded":"2023-11-08T19:28:01.695135+0000","last_fullsized":"2023-11-08T19:28:01.695135+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:25:18.963220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":59,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.694898+0000","last_change":"2023-11-08T19:26:06.296634+0000","last_active":"2023-11-08T19:28:01.694898+0000","last_peered":"2023-11-08T19:28:01.694898+0000","last_clean":"2023-11-08T19:28:01.694898+0000","last_became_active":"2023-11-08T19:26:04.603088+0000","last_became_peered":"2023-11-08T19:26:04.603088+0000","last_unstale":"2023-11-08T19:28:01.694898+0000","last_undegraded":"2023-11-08T19:28:01.694898+0000","last_fullsized":"2023-11-08T19:28:01.694898+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:01:56.991346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038180899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.695512+0000","last_change":"2023-11-08T19:27:09.431220+0000","last_active":"2023-11-08T19:28:01.695512+0000","last_peered":"2023-11-08T19:28:01.695512+0000","last_clean":"2023-11-08T19:28:01.695512+0000","last_became_active":"2023-11-08T19:27:07.191589+0000","last_became_peered":"2023-11-08T19:27:07.191589+0000","last_unstale":"2023-11-08T19:28:01.695512+0000","last_undegraded":"2023-11-08T19:28:01.695512+0000","last_fullsized":"2023-11-08T19:28:01.695512+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:18:19.964487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000729527,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.714312+0000","last_change":"2023-11-08T19:28:02.714312+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.714312+0000","last_undegraded":"2023-11-08T19:28:02.714312+0000","last_fullsized":"2023-11-08T19:28:02.714312+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"91'1","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.719567+0000","last_change":"2023-11-08T19:28:02.712293+0000","last_active":"2023-11-08T19:28:03.719567+0000","last_peered":"2023-11-08T19:28:03.719567+0000","last_clean":"2023-11-08T19:28:03.719567+0000","last_became_active":"2023-11-08T19:28:02.712002+0000","last_became_peered":"2023-11-08T19:28:02.712002+0000","last_unstale":"2023-11-08T19:28:03.719567+0000","last_undegraded":"2023-11-08T19:28:03.719567+0000","last_fullsized":"2023-11-08T19:28:03.719567+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:24:47.603187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.694985+0000","last_change":"2023-11-08T19:27:08.193337+0000","last_active":"2023-11-08T19:28:01.694985+0000","last_peered":"2023-11-08T19:28:01.694985+0000","last_clean":"2023-11-08T19:28:01.694985+0000","last_became_active":"2023-11-08T19:27:08.192997+0000","last_became_peered":"2023-11-08T19:27:08.192997+0000","last_unstale":"2023-11-08T19:28:01.694985+0000","last_undegraded":"2023-11-08T19:28:01.694985+0000","last_fullsized":"2023-11-08T19:28:01.694985+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:45:22.084015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":64,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.696307+0000","last_change":"2023-11-08T19:26:06.424511+0000","last_active":"2023-11-08T19:28:01.696307+0000","last_peered":"2023-11-08T19:28:01.696307+0000","last_clean":"2023-11-08T19:28:01.696307+0000","last_became_active":"2023-11-08T19:26:04.598973+0000","last_became_peered":"2023-11-08T19:26:04.598973+0000","last_unstale":"2023-11-08T19:28:01.696307+0000","last_undegraded":"2023-11-08T19:28:01.696307+0000","last_fullsized":"2023-11-08T19:28:01.696307+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:41:53.497456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031755900000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.694476+0000","last_change":"2023-11-08T19:27:09.497024+0000","last_active":"2023-11-08T19:28:01.694476+0000","last_peered":"2023-11-08T19:28:01.694476+0000","last_clean":"2023-11-08T19:28:01.694476+0000","last_became_active":"2023-11-08T19:27:07.190132+0000","last_became_peered":"2023-11-08T19:27:07.190132+0000","last_unstale":"2023-11-08T19:28:01.694476+0000","last_undegraded":"2023-11-08T19:28:01.694476+0000","last_fullsized":"2023-11-08T19:28:01.694476+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:36:34.881217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054126000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.718435+0000","last_change":"2023-11-08T19:28:03.713184+0000","last_active":"2023-11-08T19:28:03.718435+0000","last_peered":"2023-11-08T19:28:03.718435+0000","last_clean":"2023-11-08T19:28:03.718435+0000","last_became_active":"2023-11-08T19:28:03.712864+0000","last_became_peered":"2023-11-08T19:28:03.712864+0000","last_unstale":"2023-11-08T19:28:03.718435+0000","last_undegraded":"2023-11-08T19:28:03.718435+0000","last_fullsized":"2023-11-08T19:28:03.718435+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:38:54.805747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"91'1","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.716012+0000","last_change":"2023-11-08T19:28:02.713126+0000","last_active":"2023-11-08T19:28:03.716012+0000","last_peered":"2023-11-08T19:28:03.716012+0000","last_clean":"2023-11-08T19:28:03.716012+0000","last_became_active":"2023-11-08T19:28:02.712836+0000","last_became_peered":"2023-11-08T19:28:02.712836+0000","last_unstale":"2023-11-08T19:28:03.716012+0000","last_undegraded":"2023-11-08T19:28:03.716012+0000","last_fullsized":"2023-11-08T19:28:03.716012+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:29:31.276279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.695424+0000","last_change":"2023-11-08T19:27:08.194196+0000","last_active":"2023-11-08T19:28:01.695424+0000","last_peered":"2023-11-08T19:28:01.695424+0000","last_clean":"2023-11-08T19:28:01.695424+0000","last_became_active":"2023-11-08T19:27:08.193912+0000","last_became_peered":"2023-11-08T19:27:08.193912+0000","last_unstale":"2023-11-08T19:28:01.695424+0000","last_undegraded":"2023-11-08T19:28:01.695424+0000","last_fullsized":"2023-11-08T19:28:01.695424+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:25:18.963220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":59,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.695591+0000","last_change":"2023-11-08T19:26:06.296937+0000","last_active":"2023-11-08T19:28:01.695591+0000","last_peered":"2023-11-08T19:28:01.695591+0000","last_clean":"2023-11-08T19:28:01.695591+0000","last_became_active":"2023-11-08T19:26:04.599999+0000","last_became_peered":"2023-11-08T19:26:04.599999+0000","last_unstale":"2023-11-08T19:28:01.695591+0000","last_undegraded":"2023-11-08T19:28:01.695591+0000","last_fullsized":"2023-11-08T19:28:01.695591+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:50:43.827570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025605700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.695067+0000","last_change":"2023-11-08T19:27:09.497040+0000","last_active":"2023-11-08T19:28:01.695067+0000","last_peered":"2023-11-08T19:28:01.695067+0000","last_clean":"2023-11-08T19:28:01.695067+0000","last_became_active":"2023-11-08T19:27:07.189135+0000","last_became_peered":"2023-11-08T19:27:07.189135+0000","last_unstale":"2023-11-08T19:28:01.695067+0000","last_undegraded":"2023-11-08T19:28:01.695067+0000","last_fullsized":"2023-11-08T19:28:01.695067+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:12:29.400972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051794299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.703964+0000","last_change":"2023-11-08T19:28:02.703964+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.703964+0000","last_undegraded":"2023-11-08T19:28:02.703964+0000","last_fullsized":"2023-11-08T19:28:02.703964+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.710152+0000","last_change":"2023-11-08T19:28:02.705153+0000","last_active":"2023-11-08T19:28:02.710152+0000","last_peered":"2023-11-08T19:28:02.710152+0000","last_clean":"2023-11-08T19:28:02.710152+0000","last_became_active":"2023-11-08T19:28:02.704854+0000","last_became_peered":"2023-11-08T19:28:02.704854+0000","last_unstale":"2023-11-08T19:28:02.710152+0000","last_undegraded":"2023-11-08T19:28:02.710152+0000","last_fullsized":"2023-11-08T19:28:02.710152+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:25:00.088013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.703721+0000","last_change":"2023-11-08T19:27:08.187053+0000","last_active":"2023-11-08T19:28:02.703721+0000","last_peered":"2023-11-08T19:28:02.703721+0000","last_clean":"2023-11-08T19:28:02.703721+0000","last_became_active":"2023-11-08T19:27:08.186643+0000","last_became_peered":"2023-11-08T19:27:08.186643+0000","last_unstale":"2023-11-08T19:28:02.703721+0000","last_undegraded":"2023-11-08T19:28:02.703721+0000","last_fullsized":"2023-11-08T19:28:02.703721+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:35:30.073885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":71,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.700976+0000","last_change":"2023-11-08T19:26:05.609824+0000","last_active":"2023-11-08T19:28:02.700976+0000","last_peered":"2023-11-08T19:28:02.700976+0000","last_clean":"2023-11-08T19:28:02.700976+0000","last_became_active":"2023-11-08T19:26:04.603320+0000","last_became_peered":"2023-11-08T19:26:04.603320+0000","last_unstale":"2023-11-08T19:28:02.700976+0000","last_undegraded":"2023-11-08T19:28:02.700976+0000","last_fullsized":"2023-11-08T19:28:02.700976+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:05:45.976615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063225,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.694457+0000","last_change":"2023-11-08T19:27:09.496655+0000","last_active":"2023-11-08T19:28:01.694457+0000","last_peered":"2023-11-08T19:28:01.694457+0000","last_clean":"2023-11-08T19:28:01.694457+0000","last_became_active":"2023-11-08T19:27:07.189895+0000","last_became_peered":"2023-11-08T19:27:07.189895+0000","last_unstale":"2023-11-08T19:28:01.694457+0000","last_undegraded":"2023-11-08T19:28:01.694457+0000","last_fullsized":"2023-11-08T19:28:01.694457+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:41:08.822366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045654199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.711687+0000","last_change":"2023-11-08T19:28:02.711687+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.711687+0000","last_undegraded":"2023-11-08T19:28:02.711687+0000","last_fullsized":"2023-11-08T19:28:02.711687+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.711905+0000","last_change":"2023-11-08T19:28:02.709891+0000","last_active":"2023-11-08T19:28:02.711905+0000","last_peered":"2023-11-08T19:28:02.711905+0000","last_clean":"2023-11-08T19:28:02.711905+0000","last_became_active":"2023-11-08T19:28:02.709612+0000","last_became_peered":"2023-11-08T19:28:02.709612+0000","last_unstale":"2023-11-08T19:28:02.711905+0000","last_undegraded":"2023-11-08T19:28:02.711905+0000","last_fullsized":"2023-11-08T19:28:02.711905+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:24:47.603187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":60,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.698243+0000","last_change":"2023-11-08T19:26:06.541360+0000","last_active":"2023-11-08T19:28:01.698243+0000","last_peered":"2023-11-08T19:28:01.698243+0000","last_clean":"2023-11-08T19:28:01.698243+0000","last_became_active":"2023-11-08T19:26:04.603725+0000","last_became_peered":"2023-11-08T19:26:04.603725+0000","last_unstale":"2023-11-08T19:28:01.698243+0000","last_undegraded":"2023-11-08T19:28:01.698243+0000","last_fullsized":"2023-11-08T19:28:01.698243+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:28:48.576742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053365400000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.698592+0000","last_change":"2023-11-08T19:27:09.283599+0000","last_active":"2023-11-08T19:28:01.698592+0000","last_peered":"2023-11-08T19:28:01.698592+0000","last_clean":"2023-11-08T19:28:01.698592+0000","last_became_active":"2023-11-08T19:27:07.189595+0000","last_became_peered":"2023-11-08T19:27:07.189595+0000","last_unstale":"2023-11-08T19:28:01.698592+0000","last_undegraded":"2023-11-08T19:28:01.698592+0000","last_fullsized":"2023-11-08T19:28:01.698592+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:48:20.082682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000323479,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.696394+0000","last_change":"2023-11-08T19:27:08.185395+0000","last_active":"2023-11-08T19:28:01.696394+0000","last_peered":"2023-11-08T19:28:01.696394+0000","last_clean":"2023-11-08T19:28:01.696394+0000","last_became_active":"2023-11-08T19:27:08.184765+0000","last_became_peered":"2023-11-08T19:27:08.184765+0000","last_unstale":"2023-11-08T19:28:01.696394+0000","last_undegraded":"2023-11-08T19:28:01.696394+0000","last_fullsized":"2023-11-08T19:28:01.696394+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:08:51.864418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.712306+0000","last_change":"2023-11-08T19:28:02.712306+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.712306+0000","last_undegraded":"2023-11-08T19:28:02.712306+0000","last_fullsized":"2023-11-08T19:28:02.712306+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.725515+0000","last_change":"2023-11-08T19:28:02.720596+0000","last_active":"2023-11-08T19:28:02.725515+0000","last_peered":"2023-11-08T19:28:02.725515+0000","last_clean":"2023-11-08T19:28:02.725515+0000","last_became_active":"2023-11-08T19:28:02.720144+0000","last_became_peered":"2023-11-08T19:28:02.720144+0000","last_unstale":"2023-11-08T19:28:02.725515+0000","last_undegraded":"2023-11-08T19:28:02.725515+0000","last_fullsized":"2023-11-08T19:28:02.725515+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:51:33.542656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":64,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.694624+0000","last_change":"2023-11-08T19:26:06.424236+0000","last_active":"2023-11-08T19:28:01.694624+0000","last_peered":"2023-11-08T19:28:01.694624+0000","last_clean":"2023-11-08T19:28:01.694624+0000","last_became_active":"2023-11-08T19:26:04.600115+0000","last_became_peered":"2023-11-08T19:26:04.600115+0000","last_unstale":"2023-11-08T19:28:01.694624+0000","last_undegraded":"2023-11-08T19:28:01.694624+0000","last_fullsized":"2023-11-08T19:28:01.694624+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:18:01.876605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065803900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.695283+0000","last_change":"2023-11-08T19:27:09.331899+0000","last_active":"2023-11-08T19:28:01.695283+0000","last_peered":"2023-11-08T19:28:01.695283+0000","last_clean":"2023-11-08T19:28:01.695283+0000","last_became_active":"2023-11-08T19:27:07.191460+0000","last_became_peered":"2023-11-08T19:27:07.191460+0000","last_unstale":"2023-11-08T19:28:01.695283+0000","last_undegraded":"2023-11-08T19:28:01.695283+0000","last_fullsized":"2023-11-08T19:28:01.695283+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:51:29.247104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044907500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.701089+0000","last_change":"2023-11-08T19:27:08.191802+0000","last_active":"2023-11-08T19:28:02.701089+0000","last_peered":"2023-11-08T19:28:02.701089+0000","last_clean":"2023-11-08T19:28:02.701089+0000","last_became_active":"2023-11-08T19:27:08.191520+0000","last_became_peered":"2023-11-08T19:27:08.191520+0000","last_unstale":"2023-11-08T19:28:02.701089+0000","last_undegraded":"2023-11-08T19:28:02.701089+0000","last_fullsized":"2023-11-08T19:28:02.701089+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:49:16.458634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.724704+0000","last_change":"2023-11-08T19:28:03.722086+0000","last_active":"2023-11-08T19:28:03.724704+0000","last_peered":"2023-11-08T19:28:03.724704+0000","last_clean":"2023-11-08T19:28:03.724704+0000","last_became_active":"2023-11-08T19:28:03.721713+0000","last_became_peered":"2023-11-08T19:28:03.721713+0000","last_unstale":"2023-11-08T19:28:03.724704+0000","last_undegraded":"2023-11-08T19:28:03.724704+0000","last_fullsized":"2023-11-08T19:28:03.724704+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:34:22.087376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.705286+0000","last_change":"2023-11-08T19:28:02.707105+0000","last_active":"2023-11-08T19:28:03.705286+0000","last_peered":"2023-11-08T19:28:03.705286+0000","last_clean":"2023-11-08T19:28:03.705286+0000","last_became_active":"2023-11-08T19:28:02.706615+0000","last_became_peered":"2023-11-08T19:28:02.706615+0000","last_unstale":"2023-11-08T19:28:03.705286+0000","last_undegraded":"2023-11-08T19:28:03.705286+0000","last_fullsized":"2023-11-08T19:28:03.705286+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:18:55.364855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":71,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.699281+0000","last_change":"2023-11-08T19:26:05.608892+0000","last_active":"2023-11-08T19:28:02.699281+0000","last_peered":"2023-11-08T19:28:02.699281+0000","last_clean":"2023-11-08T19:28:02.699281+0000","last_became_active":"2023-11-08T19:26:04.604519+0000","last_became_peered":"2023-11-08T19:26:04.604519+0000","last_unstale":"2023-11-08T19:28:02.699281+0000","last_undegraded":"2023-11-08T19:28:02.699281+0000","last_fullsized":"2023-11-08T19:28:02.699281+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:11:26.508634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051248099999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.700897+0000","last_change":"2023-11-08T19:27:09.181431+0000","last_active":"2023-11-08T19:28:02.700897+0000","last_peered":"2023-11-08T19:28:02.700897+0000","last_clean":"2023-11-08T19:28:02.700897+0000","last_became_active":"2023-11-08T19:27:07.190800+0000","last_became_peered":"2023-11-08T19:27:07.190800+0000","last_unstale":"2023-11-08T19:28:02.700897+0000","last_undegraded":"2023-11-08T19:28:02.700897+0000","last_fullsized":"2023-11-08T19:28:02.700897+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:46:30.452678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656099999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.696596+0000","last_change":"2023-11-08T19:27:08.187983+0000","last_active":"2023-11-08T19:28:01.696596+0000","last_peered":"2023-11-08T19:28:01.696596+0000","last_clean":"2023-11-08T19:28:01.696596+0000","last_became_active":"2023-11-08T19:27:08.187322+0000","last_became_peered":"2023-11-08T19:27:08.187322+0000","last_unstale":"2023-11-08T19:28:01.696596+0000","last_undegraded":"2023-11-08T19:28:01.696596+0000","last_fullsized":"2023-11-08T19:28:01.696596+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:33:07.647519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.704461+0000","last_change":"2023-11-08T19:28:02.704461+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.704461+0000","last_undegraded":"2023-11-08T19:28:02.704461+0000","last_fullsized":"2023-11-08T19:28:02.704461+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.725558+0000","last_change":"2023-11-08T19:28:02.720466+0000","last_active":"2023-11-08T19:28:02.725558+0000","last_peered":"2023-11-08T19:28:02.725558+0000","last_clean":"2023-11-08T19:28:02.725558+0000","last_became_active":"2023-11-08T19:28:02.720193+0000","last_became_peered":"2023-11-08T19:28:02.720193+0000","last_unstale":"2023-11-08T19:28:02.725558+0000","last_undegraded":"2023-11-08T19:28:02.725558+0000","last_fullsized":"2023-11-08T19:28:02.725558+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:26:23.134605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":58,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.694087+0000","last_change":"2023-11-08T19:26:06.394075+0000","last_active":"2023-11-08T19:28:01.694087+0000","last_peered":"2023-11-08T19:28:01.694087+0000","last_clean":"2023-11-08T19:28:01.694087+0000","last_became_active":"2023-11-08T19:26:04.606967+0000","last_became_peered":"2023-11-08T19:26:04.606967+0000","last_unstale":"2023-11-08T19:28:01.694087+0000","last_undegraded":"2023-11-08T19:28:01.694087+0000","last_fullsized":"2023-11-08T19:28:01.694087+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:19:56.286835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040971999999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.701127+0000","last_change":"2023-11-08T19:27:09.180722+0000","last_active":"2023-11-08T19:28:02.701127+0000","last_peered":"2023-11-08T19:28:02.701127+0000","last_clean":"2023-11-08T19:28:02.701127+0000","last_became_active":"2023-11-08T19:27:07.186977+0000","last_became_peered":"2023-11-08T19:27:07.186977+0000","last_unstale":"2023-11-08T19:28:02.701127+0000","last_undegraded":"2023-11-08T19:28:02.701127+0000","last_fullsized":"2023-11-08T19:28:02.701127+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:17:59.966463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057675100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.694729+0000","last_change":"2023-11-08T19:27:08.193963+0000","last_active":"2023-11-08T19:28:01.694729+0000","last_peered":"2023-11-08T19:28:01.694729+0000","last_clean":"2023-11-08T19:28:01.694729+0000","last_became_active":"2023-11-08T19:27:08.193098+0000","last_became_peered":"2023-11-08T19:27:08.193098+0000","last_unstale":"2023-11-08T19:28:01.694729+0000","last_undegraded":"2023-11-08T19:28:01.694729+0000","last_fullsized":"2023-11-08T19:28:01.694729+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:41:09.827370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.699774+0000","last_change":"2023-11-08T19:28:02.699774+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.699774+0000","last_undegraded":"2023-11-08T19:28:02.699774+0000","last_fullsized":"2023-11-08T19:28:02.699774+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.722618+0000","last_change":"2023-11-08T19:28:02.718059+0000","last_active":"2023-11-08T19:28:02.722618+0000","last_peered":"2023-11-08T19:28:02.722618+0000","last_clean":"2023-11-08T19:28:02.722618+0000","last_became_active":"2023-11-08T19:28:02.717775+0000","last_became_peered":"2023-11-08T19:28:02.717775+0000","last_unstale":"2023-11-08T19:28:02.722618+0000","last_undegraded":"2023-11-08T19:28:02.722618+0000","last_fullsized":"2023-11-08T19:28:02.722618+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:29:25.082011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":58,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.697065+0000","last_change":"2023-11-08T19:26:06.265686+0000","last_active":"2023-11-08T19:28:01.697065+0000","last_peered":"2023-11-08T19:28:01.697065+0000","last_clean":"2023-11-08T19:28:01.697065+0000","last_became_active":"2023-11-08T19:26:04.602111+0000","last_became_peered":"2023-11-08T19:26:04.602111+0000","last_unstale":"2023-11-08T19:28:01.697065+0000","last_undegraded":"2023-11-08T19:28:01.697065+0000","last_fullsized":"2023-11-08T19:28:01.697065+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:40:11.164951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056197899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.701328+0000","last_change":"2023-11-08T19:27:09.181333+0000","last_active":"2023-11-08T19:28:02.701328+0000","last_peered":"2023-11-08T19:28:02.701328+0000","last_clean":"2023-11-08T19:28:02.701328+0000","last_became_active":"2023-11-08T19:27:07.188657+0000","last_became_peered":"2023-11-08T19:27:07.188657+0000","last_unstale":"2023-11-08T19:28:02.701328+0000","last_undegraded":"2023-11-08T19:28:02.701328+0000","last_fullsized":"2023-11-08T19:28:02.701328+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:25:56.788615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001227393,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.701395+0000","last_change":"2023-11-08T19:27:08.188533+0000","last_active":"2023-11-08T19:28:02.701395+0000","last_peered":"2023-11-08T19:28:02.701395+0000","last_clean":"2023-11-08T19:28:02.701395+0000","last_became_active":"2023-11-08T19:27:08.188240+0000","last_became_peered":"2023-11-08T19:27:08.188240+0000","last_unstale":"2023-11-08T19:28:02.701395+0000","last_undegraded":"2023-11-08T19:28:02.701395+0000","last_fullsized":"2023-11-08T19:28:02.701395+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:55:37.653521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.701682+0000","last_change":"2023-11-08T19:28:02.701682+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.701682+0000","last_undegraded":"2023-11-08T19:28:02.701682+0000","last_fullsized":"2023-11-08T19:28:02.701682+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.714684+0000","last_change":"2023-11-08T19:28:02.710864+0000","last_active":"2023-11-08T19:28:02.714684+0000","last_peered":"2023-11-08T19:28:02.714684+0000","last_clean":"2023-11-08T19:28:02.714684+0000","last_became_active":"2023-11-08T19:28:02.710597+0000","last_became_peered":"2023-11-08T19:28:02.710597+0000","last_unstale":"2023-11-08T19:28:02.714684+0000","last_undegraded":"2023-11-08T19:28:02.714684+0000","last_fullsized":"2023-11-08T19:28:02.714684+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:02:43.242472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.702305+0000","last_change":"2023-11-08T19:27:08.188458+0000","last_active":"2023-11-08T19:28:02.702305+0000","last_peered":"2023-11-08T19:28:02.702305+0000","last_clean":"2023-11-08T19:28:02.702305+0000","last_became_active":"2023-11-08T19:27:08.187880+0000","last_became_peered":"2023-11-08T19:27:08.187880+0000","last_unstale":"2023-11-08T19:28:02.702305+0000","last_undegraded":"2023-11-08T19:28:02.702305+0000","last_fullsized":"2023-11-08T19:28:02.702305+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:25:28.618214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":59,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.690530+0000","last_change":"2023-11-08T19:26:05.606898+0000","last_active":"2023-11-08T19:28:01.690530+0000","last_peered":"2023-11-08T19:28:01.690530+0000","last_clean":"2023-11-08T19:28:01.690530+0000","last_became_active":"2023-11-08T19:26:04.603955+0000","last_became_peered":"2023-11-08T19:26:04.603955+0000","last_unstale":"2023-11-08T19:28:01.690530+0000","last_undegraded":"2023-11-08T19:28:01.690530+0000","last_fullsized":"2023-11-08T19:28:01.690530+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:19:11.589559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036473499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.690241+0000","last_change":"2023-11-08T19:27:09.178770+0000","last_active":"2023-11-08T19:28:01.690241+0000","last_peered":"2023-11-08T19:28:01.690241+0000","last_clean":"2023-11-08T19:28:01.690241+0000","last_became_active":"2023-11-08T19:27:07.194992+0000","last_became_peered":"2023-11-08T19:27:07.194992+0000","last_unstale":"2023-11-08T19:28:01.690241+0000","last_undegraded":"2023-11-08T19:28:01.690241+0000","last_fullsized":"2023-11-08T19:28:01.690241+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:35:34.026910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000770285,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.697682+0000","last_change":"2023-11-08T19:28:02.697682+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.697682+0000","last_undegraded":"2023-11-08T19:28:02.697682+0000","last_fullsized":"2023-11-08T19:28:02.697682+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.703765+0000","last_change":"2023-11-08T19:28:02.707011+0000","last_active":"2023-11-08T19:28:03.703765+0000","last_peered":"2023-11-08T19:28:03.703765+0000","last_clean":"2023-11-08T19:28:03.703765+0000","last_became_active":"2023-11-08T19:28:02.706729+0000","last_became_peered":"2023-11-08T19:28:02.706729+0000","last_unstale":"2023-11-08T19:28:03.703765+0000","last_undegraded":"2023-11-08T19:28:03.703765+0000","last_fullsized":"2023-11-08T19:28:03.703765+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:29:25.082011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.695698+0000","last_change":"2023-11-08T19:27:08.193953+0000","last_active":"2023-11-08T19:28:01.695698+0000","last_peered":"2023-11-08T19:28:01.695698+0000","last_clean":"2023-11-08T19:28:01.695698+0000","last_became_active":"2023-11-08T19:27:08.193653+0000","last_became_peered":"2023-11-08T19:27:08.193653+0000","last_unstale":"2023-11-08T19:28:01.695698+0000","last_undegraded":"2023-11-08T19:28:01.695698+0000","last_fullsized":"2023-11-08T19:28:01.695698+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:35:30.073885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":64,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.694881+0000","last_change":"2023-11-08T19:26:06.424464+0000","last_active":"2023-11-08T19:28:01.694881+0000","last_peered":"2023-11-08T19:28:01.694881+0000","last_clean":"2023-11-08T19:28:01.694881+0000","last_became_active":"2023-11-08T19:26:04.603144+0000","last_became_peered":"2023-11-08T19:26:04.603144+0000","last_unstale":"2023-11-08T19:28:01.694881+0000","last_undegraded":"2023-11-08T19:28:01.694881+0000","last_fullsized":"2023-11-08T19:28:01.694881+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:42:55.269365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054090799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.701001+0000","last_change":"2023-11-08T19:27:09.180479+0000","last_active":"2023-11-08T19:28:02.701001+0000","last_peered":"2023-11-08T19:28:02.701001+0000","last_clean":"2023-11-08T19:28:02.701001+0000","last_became_active":"2023-11-08T19:27:07.190438+0000","last_became_peered":"2023-11-08T19:27:07.190438+0000","last_unstale":"2023-11-08T19:28:02.701001+0000","last_undegraded":"2023-11-08T19:28:02.701001+0000","last_fullsized":"2023-11-08T19:28:02.701001+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:25:14.487524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050893300000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.704857+0000","last_change":"2023-11-08T19:28:02.704857+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.704857+0000","last_undegraded":"2023-11-08T19:28:02.704857+0000","last_fullsized":"2023-11-08T19:28:02.704857+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.715287+0000","last_change":"2023-11-08T19:28:02.708336+0000","last_active":"2023-11-08T19:28:02.715287+0000","last_peered":"2023-11-08T19:28:02.715287+0000","last_clean":"2023-11-08T19:28:02.715287+0000","last_became_active":"2023-11-08T19:28:02.708016+0000","last_became_peered":"2023-11-08T19:28:02.708016+0000","last_unstale":"2023-11-08T19:28:02.715287+0000","last_undegraded":"2023-11-08T19:28:02.715287+0000","last_fullsized":"2023-11-08T19:28:02.715287+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:39:02.748118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.696341+0000","last_change":"2023-11-08T19:27:08.190530+0000","last_active":"2023-11-08T19:28:01.696341+0000","last_peered":"2023-11-08T19:28:01.696341+0000","last_clean":"2023-11-08T19:28:01.696341+0000","last_became_active":"2023-11-08T19:27:08.190203+0000","last_became_peered":"2023-11-08T19:27:08.190203+0000","last_unstale":"2023-11-08T19:28:01.696341+0000","last_undegraded":"2023-11-08T19:28:01.696341+0000","last_fullsized":"2023-11-08T19:28:01.696341+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:26:32.777602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":60,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.698474+0000","last_change":"2023-11-08T19:26:06.541623+0000","last_active":"2023-11-08T19:28:01.698474+0000","last_peered":"2023-11-08T19:28:01.698474+0000","last_clean":"2023-11-08T19:28:01.698474+0000","last_became_active":"2023-11-08T19:26:04.601961+0000","last_became_peered":"2023-11-08T19:26:04.601961+0000","last_unstale":"2023-11-08T19:28:01.698474+0000","last_undegraded":"2023-11-08T19:28:01.698474+0000","last_fullsized":"2023-11-08T19:28:01.698474+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:11:26.508634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000466386,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.694709+0000","last_change":"2023-11-08T19:27:09.431231+0000","last_active":"2023-11-08T19:28:01.694709+0000","last_peered":"2023-11-08T19:28:01.694709+0000","last_clean":"2023-11-08T19:28:01.694709+0000","last_became_active":"2023-11-08T19:27:07.191583+0000","last_became_peered":"2023-11-08T19:27:07.191583+0000","last_unstale":"2023-11-08T19:28:01.694709+0000","last_undegraded":"2023-11-08T19:28:01.694709+0000","last_fullsized":"2023-11-08T19:28:01.694709+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:58:48.357266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093695200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.698528+0000","last_change":"2023-11-08T19:28:02.698528+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.698528+0000","last_undegraded":"2023-11-08T19:28:02.698528+0000","last_fullsized":"2023-11-08T19:28:02.698528+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"91'1","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.717319+0000","last_change":"2023-11-08T19:28:02.708326+0000","last_active":"2023-11-08T19:28:03.717319+0000","last_peered":"2023-11-08T19:28:03.717319+0000","last_clean":"2023-11-08T19:28:03.717319+0000","last_became_active":"2023-11-08T19:28:02.708055+0000","last_became_peered":"2023-11-08T19:28:02.708055+0000","last_unstale":"2023-11-08T19:28:03.717319+0000","last_undegraded":"2023-11-08T19:28:03.717319+0000","last_fullsized":"2023-11-08T19:28:03.717319+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:50:41.723104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.694787+0000","last_change":"2023-11-08T19:27:08.187900+0000","last_active":"2023-11-08T19:28:01.694787+0000","last_peered":"2023-11-08T19:28:01.694787+0000","last_clean":"2023-11-08T19:28:01.694787+0000","last_became_active":"2023-11-08T19:27:08.187626+0000","last_became_peered":"2023-11-08T19:27:08.187626+0000","last_unstale":"2023-11-08T19:28:01.694787+0000","last_undegraded":"2023-11-08T19:28:01.694787+0000","last_fullsized":"2023-11-08T19:28:01.694787+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:00:55.878397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":71,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.701751+0000","last_change":"2023-11-08T19:26:05.608980+0000","last_active":"2023-11-08T19:28:02.701751+0000","last_peered":"2023-11-08T19:28:02.701751+0000","last_clean":"2023-11-08T19:28:02.701751+0000","last_became_active":"2023-11-08T19:26:04.606386+0000","last_became_peered":"2023-11-08T19:26:04.606386+0000","last_unstale":"2023-11-08T19:28:02.701751+0000","last_undegraded":"2023-11-08T19:28:02.701751+0000","last_fullsized":"2023-11-08T19:28:02.701751+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:41:50.771457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037950599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.690460+0000","last_change":"2023-11-08T19:27:09.179037+0000","last_active":"2023-11-08T19:28:01.690460+0000","last_peered":"2023-11-08T19:28:01.690460+0000","last_clean":"2023-11-08T19:28:01.690460+0000","last_became_active":"2023-11-08T19:27:07.195033+0000","last_became_peered":"2023-11-08T19:27:07.195033+0000","last_unstale":"2023-11-08T19:28:01.690460+0000","last_undegraded":"2023-11-08T19:28:01.690460+0000","last_fullsized":"2023-11-08T19:28:01.690460+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:19:08.253483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041824800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.697868+0000","last_change":"2023-11-08T19:28:02.697868+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.697868+0000","last_undegraded":"2023-11-08T19:28:02.697868+0000","last_fullsized":"2023-11-08T19:28:02.697868+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.713986+0000","last_change":"2023-11-08T19:28:02.712619+0000","last_active":"2023-11-08T19:28:02.713986+0000","last_peered":"2023-11-08T19:28:02.713986+0000","last_clean":"2023-11-08T19:28:02.713986+0000","last_became_active":"2023-11-08T19:28:02.712342+0000","last_became_peered":"2023-11-08T19:28:02.712342+0000","last_unstale":"2023-11-08T19:28:02.713986+0000","last_undegraded":"2023-11-08T19:28:02.713986+0000","last_fullsized":"2023-11-08T19:28:02.713986+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:11:10.165281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.693976+0000","last_change":"2023-11-08T19:27:08.193761+0000","last_active":"2023-11-08T19:28:01.693976+0000","last_peered":"2023-11-08T19:28:01.693976+0000","last_clean":"2023-11-08T19:28:01.693976+0000","last_became_active":"2023-11-08T19:27:08.193017+0000","last_became_peered":"2023-11-08T19:27:08.193017+0000","last_unstale":"2023-11-08T19:28:01.693976+0000","last_undegraded":"2023-11-08T19:28:01.693976+0000","last_fullsized":"2023-11-08T19:28:01.693976+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:18:00.971466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":64,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.694700+0000","last_change":"2023-11-08T19:26:06.424073+0000","last_active":"2023-11-08T19:28:01.694700+0000","last_peered":"2023-11-08T19:28:01.694700+0000","last_clean":"2023-11-08T19:28:01.694700+0000","last_became_active":"2023-11-08T19:26:04.599046+0000","last_became_peered":"2023-11-08T19:26:04.599046+0000","last_unstale":"2023-11-08T19:28:01.694700+0000","last_undegraded":"2023-11-08T19:28:01.694700+0000","last_fullsized":"2023-11-08T19:28:01.694700+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:04:43.562566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035092199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.700565+0000","last_change":"2023-11-08T19:27:09.181716+0000","last_active":"2023-11-08T19:28:02.700565+0000","last_peered":"2023-11-08T19:28:02.700565+0000","last_clean":"2023-11-08T19:28:02.700565+0000","last_became_active":"2023-11-08T19:27:07.187077+0000","last_became_peered":"2023-11-08T19:27:07.187077+0000","last_unstale":"2023-11-08T19:28:02.700565+0000","last_undegraded":"2023-11-08T19:28:02.700565+0000","last_fullsized":"2023-11-08T19:28:02.700565+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:51:29.247104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056673599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.726360+0000","last_change":"2023-11-08T19:28:03.724351+0000","last_active":"2023-11-08T19:28:03.726360+0000","last_peered":"2023-11-08T19:28:03.726360+0000","last_clean":"2023-11-08T19:28:03.726360+0000","last_became_active":"2023-11-08T19:28:03.724109+0000","last_became_peered":"2023-11-08T19:28:03.724109+0000","last_unstale":"2023-11-08T19:28:03.726360+0000","last_undegraded":"2023-11-08T19:28:03.726360+0000","last_fullsized":"2023-11-08T19:28:03.726360+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:11:11.170102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.716963+0000","last_change":"2023-11-08T19:28:02.711594+0000","last_active":"2023-11-08T19:28:02.716963+0000","last_peered":"2023-11-08T19:28:02.716963+0000","last_clean":"2023-11-08T19:28:02.716963+0000","last_became_active":"2023-11-08T19:28:02.711204+0000","last_became_peered":"2023-11-08T19:28:02.711204+0000","last_unstale":"2023-11-08T19:28:02.716963+0000","last_undegraded":"2023-11-08T19:28:02.716963+0000","last_fullsized":"2023-11-08T19:28:02.716963+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:21:23.673336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.696597+0000","last_change":"2023-11-08T19:27:08.188143+0000","last_active":"2023-11-08T19:28:01.696597+0000","last_peered":"2023-11-08T19:28:01.696597+0000","last_clean":"2023-11-08T19:28:01.696597+0000","last_became_active":"2023-11-08T19:27:08.187714+0000","last_became_peered":"2023-11-08T19:27:08.187714+0000","last_unstale":"2023-11-08T19:28:01.696597+0000","last_undegraded":"2023-11-08T19:28:01.696597+0000","last_fullsized":"2023-11-08T19:28:01.696597+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:02:10.863403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":58,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.693970+0000","last_change":"2023-11-08T19:26:06.394320+0000","last_active":"2023-11-08T19:28:01.693970+0000","last_peered":"2023-11-08T19:28:01.693970+0000","last_clean":"2023-11-08T19:28:01.693970+0000","last_became_active":"2023-11-08T19:26:04.603212+0000","last_became_peered":"2023-11-08T19:26:04.603212+0000","last_unstale":"2023-11-08T19:28:01.693970+0000","last_undegraded":"2023-11-08T19:28:01.693970+0000","last_fullsized":"2023-11-08T19:28:01.693970+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:25:29.947079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054695399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.700819+0000","last_change":"2023-11-08T19:27:09.181613+0000","last_active":"2023-11-08T19:28:02.700819+0000","last_peered":"2023-11-08T19:28:02.700819+0000","last_clean":"2023-11-08T19:28:02.700819+0000","last_became_active":"2023-11-08T19:27:07.190897+0000","last_became_peered":"2023-11-08T19:27:07.190897+0000","last_unstale":"2023-11-08T19:28:02.700819+0000","last_undegraded":"2023-11-08T19:28:02.700819+0000","last_fullsized":"2023-11-08T19:28:02.700819+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:27:57.388125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011172459999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.698922+0000","last_change":"2023-11-08T19:28:02.698922+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.698922+0000","last_undegraded":"2023-11-08T19:28:02.698922+0000","last_fullsized":"2023-11-08T19:28:02.698922+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.711981+0000","last_change":"2023-11-08T19:28:02.707358+0000","last_active":"2023-11-08T19:28:02.711981+0000","last_peered":"2023-11-08T19:28:02.711981+0000","last_clean":"2023-11-08T19:28:02.711981+0000","last_became_active":"2023-11-08T19:28:02.707109+0000","last_became_peered":"2023-11-08T19:28:02.707109+0000","last_unstale":"2023-11-08T19:28:02.711981+0000","last_undegraded":"2023-11-08T19:28:02.711981+0000","last_fullsized":"2023-11-08T19:28:02.711981+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:32:42.691100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.696667+0000","last_change":"2023-11-08T19:27:08.188668+0000","last_active":"2023-11-08T19:28:01.696667+0000","last_peered":"2023-11-08T19:28:01.696667+0000","last_clean":"2023-11-08T19:28:01.696667+0000","last_became_active":"2023-11-08T19:27:08.188441+0000","last_became_peered":"2023-11-08T19:27:08.188441+0000","last_unstale":"2023-11-08T19:28:01.696667+0000","last_undegraded":"2023-11-08T19:28:01.696667+0000","last_fullsized":"2023-11-08T19:28:01.696667+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:45:44.379160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":64,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.694995+0000","last_change":"2023-11-08T19:26:06.423754+0000","last_active":"2023-11-08T19:28:01.694995+0000","last_peered":"2023-11-08T19:28:01.694995+0000","last_clean":"2023-11-08T19:28:01.694995+0000","last_became_active":"2023-11-08T19:26:04.599322+0000","last_became_peered":"2023-11-08T19:26:04.599322+0000","last_unstale":"2023-11-08T19:28:01.694995+0000","last_undegraded":"2023-11-08T19:28:01.694995+0000","last_fullsized":"2023-11-08T19:28:01.694995+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:13:08.689128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042670000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.701576+0000","last_change":"2023-11-08T19:27:09.181656+0000","last_active":"2023-11-08T19:28:02.701576+0000","last_peered":"2023-11-08T19:28:02.701576+0000","last_clean":"2023-11-08T19:28:02.701576+0000","last_became_active":"2023-11-08T19:27:07.187337+0000","last_became_peered":"2023-11-08T19:27:07.187337+0000","last_unstale":"2023-11-08T19:28:02.701576+0000","last_undegraded":"2023-11-08T19:28:02.701576+0000","last_fullsized":"2023-11-08T19:28:02.701576+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:14:17.605057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000346967,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.698224+0000","last_change":"2023-11-08T19:28:02.698224+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.698224+0000","last_undegraded":"2023-11-08T19:28:02.698224+0000","last_fullsized":"2023-11-08T19:28:02.698224+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.712011+0000","last_change":"2023-11-08T19:28:02.704760+0000","last_active":"2023-11-08T19:28:02.712011+0000","last_peered":"2023-11-08T19:28:02.712011+0000","last_clean":"2023-11-08T19:28:02.712011+0000","last_became_active":"2023-11-08T19:28:02.704410+0000","last_became_peered":"2023-11-08T19:28:02.704410+0000","last_unstale":"2023-11-08T19:28:02.712011+0000","last_undegraded":"2023-11-08T19:28:02.712011+0000","last_fullsized":"2023-11-08T19:28:02.712011+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:00:10.329151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.694901+0000","last_change":"2023-11-08T19:27:08.194065+0000","last_active":"2023-11-08T19:28:01.694901+0000","last_peered":"2023-11-08T19:28:01.694901+0000","last_clean":"2023-11-08T19:28:01.694901+0000","last_became_active":"2023-11-08T19:27:08.193532+0000","last_became_peered":"2023-11-08T19:27:08.193532+0000","last_unstale":"2023-11-08T19:28:01.694901+0000","last_undegraded":"2023-11-08T19:28:01.694901+0000","last_fullsized":"2023-11-08T19:28:01.694901+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:27:58.393129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":58,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.695354+0000","last_change":"2023-11-08T19:26:06.394488+0000","last_active":"2023-11-08T19:28:01.695354+0000","last_peered":"2023-11-08T19:28:01.695354+0000","last_clean":"2023-11-08T19:28:01.695354+0000","last_became_active":"2023-11-08T19:26:04.606805+0000","last_became_peered":"2023-11-08T19:26:04.606805+0000","last_unstale":"2023-11-08T19:28:01.695354+0000","last_undegraded":"2023-11-08T19:28:01.695354+0000","last_fullsized":"2023-11-08T19:28:01.695354+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:25:20.857133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045317000000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.696700+0000","last_change":"2023-11-08T19:27:09.211647+0000","last_active":"2023-11-08T19:28:01.696700+0000","last_peered":"2023-11-08T19:28:01.696700+0000","last_clean":"2023-11-08T19:28:01.696700+0000","last_became_active":"2023-11-08T19:27:07.190309+0000","last_became_peered":"2023-11-08T19:27:07.190309+0000","last_unstale":"2023-11-08T19:28:01.696700+0000","last_undegraded":"2023-11-08T19:28:01.696700+0000","last_fullsized":"2023-11-08T19:28:01.696700+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:56:04.292154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047615199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.711664+0000","last_change":"2023-11-08T19:28:02.711664+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.711664+0000","last_undegraded":"2023-11-08T19:28:02.711664+0000","last_fullsized":"2023-11-08T19:28:02.711664+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.722917+0000","last_change":"2023-11-08T19:28:02.717508+0000","last_active":"2023-11-08T19:28:02.722917+0000","last_peered":"2023-11-08T19:28:02.722917+0000","last_clean":"2023-11-08T19:28:02.722917+0000","last_became_active":"2023-11-08T19:28:02.717126+0000","last_became_peered":"2023-11-08T19:28:02.717126+0000","last_unstale":"2023-11-08T19:28:02.722917+0000","last_undegraded":"2023-11-08T19:28:02.722917+0000","last_fullsized":"2023-11-08T19:28:02.722917+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:23:34.816577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.695560+0000","last_change":"2023-11-08T19:27:08.194884+0000","last_active":"2023-11-08T19:28:01.695560+0000","last_peered":"2023-11-08T19:28:01.695560+0000","last_clean":"2023-11-08T19:28:01.695560+0000","last_became_active":"2023-11-08T19:27:08.193241+0000","last_became_peered":"2023-11-08T19:27:08.193241+0000","last_unstale":"2023-11-08T19:28:01.695560+0000","last_undegraded":"2023-11-08T19:28:01.695560+0000","last_fullsized":"2023-11-08T19:28:01.695560+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:46:31.457682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":58,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.696871+0000","last_change":"2023-11-08T19:26:06.265907+0000","last_active":"2023-11-08T19:28:01.696871+0000","last_peered":"2023-11-08T19:28:01.696871+0000","last_clean":"2023-11-08T19:28:01.696871+0000","last_became_active":"2023-11-08T19:26:04.604772+0000","last_became_peered":"2023-11-08T19:26:04.604772+0000","last_unstale":"2023-11-08T19:28:01.696871+0000","last_undegraded":"2023-11-08T19:28:01.696871+0000","last_fullsized":"2023-11-08T19:28:01.696871+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:44:16.139794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066875300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.698318+0000","last_change":"2023-11-08T19:27:09.282699+0000","last_active":"2023-11-08T19:28:01.698318+0000","last_peered":"2023-11-08T19:28:01.698318+0000","last_clean":"2023-11-08T19:28:01.698318+0000","last_became_active":"2023-11-08T19:27:07.191212+0000","last_became_peered":"2023-11-08T19:27:07.191212+0000","last_unstale":"2023-11-08T19:28:01.698318+0000","last_undegraded":"2023-11-08T19:28:01.698318+0000","last_fullsized":"2023-11-08T19:28:01.698318+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:35:14.715110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046640800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.705457+0000","last_change":"2023-11-08T19:28:02.705457+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.705457+0000","last_undegraded":"2023-11-08T19:28:02.705457+0000","last_fullsized":"2023-11-08T19:28:02.705457+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.717068+0000","last_change":"2023-11-08T19:28:02.712356+0000","last_active":"2023-11-08T19:28:02.717068+0000","last_peered":"2023-11-08T19:28:02.717068+0000","last_clean":"2023-11-08T19:28:02.717068+0000","last_became_active":"2023-11-08T19:28:02.712073+0000","last_became_peered":"2023-11-08T19:28:02.712073+0000","last_unstale":"2023-11-08T19:28:02.717068+0000","last_undegraded":"2023-11-08T19:28:02.717068+0000","last_fullsized":"2023-11-08T19:28:02.717068+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:06:46.912027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.696801+0000","last_change":"2023-11-08T19:27:08.187644+0000","last_active":"2023-11-08T19:28:01.696801+0000","last_peered":"2023-11-08T19:28:01.696801+0000","last_clean":"2023-11-08T19:28:01.696801+0000","last_became_active":"2023-11-08T19:27:08.187376+0000","last_became_peered":"2023-11-08T19:27:08.187376+0000","last_unstale":"2023-11-08T19:28:01.696801+0000","last_undegraded":"2023-11-08T19:28:01.696801+0000","last_fullsized":"2023-11-08T19:28:01.696801+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:22:40.300185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":59,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.690307+0000","last_change":"2023-11-08T19:26:05.606946+0000","last_active":"2023-11-08T19:28:01.690307+0000","last_peered":"2023-11-08T19:28:01.690307+0000","last_clean":"2023-11-08T19:28:01.690307+0000","last_became_active":"2023-11-08T19:26:04.598775+0000","last_became_peered":"2023-11-08T19:26:04.598775+0000","last_unstale":"2023-11-08T19:28:01.690307+0000","last_undegraded":"2023-11-08T19:28:01.690307+0000","last_fullsized":"2023-11-08T19:28:01.690307+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:20:56.898927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049870399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.693887+0000","last_change":"2023-11-08T19:27:09.331646+0000","last_active":"2023-11-08T19:28:01.693887+0000","last_peered":"2023-11-08T19:28:01.693887+0000","last_clean":"2023-11-08T19:28:01.693887+0000","last_became_active":"2023-11-08T19:27:07.190619+0000","last_became_peered":"2023-11-08T19:27:07.190619+0000","last_unstale":"2023-11-08T19:28:01.693887+0000","last_undegraded":"2023-11-08T19:28:01.693887+0000","last_fullsized":"2023-11-08T19:28:01.693887+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:42:47.371458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065918400000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.702920+0000","last_change":"2023-11-08T19:28:02.702920+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.702920+0000","last_undegraded":"2023-11-08T19:28:02.702920+0000","last_fullsized":"2023-11-08T19:28:02.702920+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.719716+0000","last_change":"2023-11-08T19:28:02.710012+0000","last_active":"2023-11-08T19:28:02.719716+0000","last_peered":"2023-11-08T19:28:02.719716+0000","last_clean":"2023-11-08T19:28:02.719716+0000","last_became_active":"2023-11-08T19:28:02.709715+0000","last_became_peered":"2023-11-08T19:28:02.709715+0000","last_unstale":"2023-11-08T19:28:02.719716+0000","last_undegraded":"2023-11-08T19:28:02.719716+0000","last_fullsized":"2023-11-08T19:28:02.719716+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:23:11.877422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.703535+0000","last_change":"2023-11-08T19:27:08.183798+0000","last_active":"2023-11-08T19:28:02.703535+0000","last_peered":"2023-11-08T19:28:02.703535+0000","last_clean":"2023-11-08T19:28:02.703535+0000","last_became_active":"2023-11-08T19:27:08.183537+0000","last_became_peered":"2023-11-08T19:27:08.183537+0000","last_unstale":"2023-11-08T19:28:02.703535+0000","last_undegraded":"2023-11-08T19:28:02.703535+0000","last_fullsized":"2023-11-08T19:28:02.703535+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:50:39.026265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":71,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.701309+0000","last_change":"2023-11-08T19:26:05.610012+0000","last_active":"2023-11-08T19:28:02.701309+0000","last_peered":"2023-11-08T19:28:02.701309+0000","last_clean":"2023-11-08T19:28:02.701309+0000","last_became_active":"2023-11-08T19:26:04.603209+0000","last_became_peered":"2023-11-08T19:26:04.603209+0000","last_unstale":"2023-11-08T19:28:02.701309+0000","last_undegraded":"2023-11-08T19:28:02.701309+0000","last_fullsized":"2023-11-08T19:28:02.701309+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:35:02.306997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053335100000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.690186+0000","last_change":"2023-11-08T19:27:09.178405+0000","last_active":"2023-11-08T19:28:01.690186+0000","last_peered":"2023-11-08T19:28:01.690186+0000","last_clean":"2023-11-08T19:28:01.690186+0000","last_became_active":"2023-11-08T19:27:07.188169+0000","last_became_peered":"2023-11-08T19:27:07.188169+0000","last_unstale":"2023-11-08T19:28:01.690186+0000","last_undegraded":"2023-11-08T19:28:01.690186+0000","last_fullsized":"2023-11-08T19:28:01.690186+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:39:25.855467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056209700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.703242+0000","last_change":"2023-11-08T19:28:02.703242+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.703242+0000","last_undegraded":"2023-11-08T19:28:02.703242+0000","last_fullsized":"2023-11-08T19:28:02.703242+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.722598+0000","last_change":"2023-11-08T19:28:02.716988+0000","last_active":"2023-11-08T19:28:02.722598+0000","last_peered":"2023-11-08T19:28:02.722598+0000","last_clean":"2023-11-08T19:28:02.722598+0000","last_became_active":"2023-11-08T19:28:02.716687+0000","last_became_peered":"2023-11-08T19:28:02.716687+0000","last_unstale":"2023-11-08T19:28:02.722598+0000","last_undegraded":"2023-11-08T19:28:02.722598+0000","last_fullsized":"2023-11-08T19:28:02.722598+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:21:23.673336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.693769+0000","last_change":"2023-11-08T19:27:08.193508+0000","last_active":"2023-11-08T19:28:01.693769+0000","last_peered":"2023-11-08T19:28:01.693769+0000","last_clean":"2023-11-08T19:28:01.693769+0000","last_became_active":"2023-11-08T19:27:08.193037+0000","last_became_peered":"2023-11-08T19:27:08.193037+0000","last_unstale":"2023-11-08T19:28:01.693769+0000","last_undegraded":"2023-11-08T19:28:01.693769+0000","last_fullsized":"2023-11-08T19:28:01.693769+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:29:50.067887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":58,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.696488+0000","last_change":"2023-11-08T19:26:06.265318+0000","last_active":"2023-11-08T19:28:01.696488+0000","last_peered":"2023-11-08T19:28:01.696488+0000","last_clean":"2023-11-08T19:28:01.696488+0000","last_became_active":"2023-11-08T19:26:04.603409+0000","last_became_peered":"2023-11-08T19:26:04.603409+0000","last_unstale":"2023-11-08T19:28:01.696488+0000","last_undegraded":"2023-11-08T19:28:01.696488+0000","last_fullsized":"2023-11-08T19:28:01.696488+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:27:24.933111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065684300000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.701449+0000","last_change":"2023-11-08T19:27:09.182282+0000","last_active":"2023-11-08T19:28:02.701449+0000","last_peered":"2023-11-08T19:28:02.701449+0000","last_clean":"2023-11-08T19:28:02.701449+0000","last_became_active":"2023-11-08T19:27:07.186944+0000","last_became_peered":"2023-11-08T19:27:07.186944+0000","last_unstale":"2023-11-08T19:28:02.701449+0000","last_undegraded":"2023-11-08T19:28:02.701449+0000","last_fullsized":"2023-11-08T19:28:02.701449+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:45:21.079011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00028665599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.720726+0000","last_change":"2023-11-08T19:28:03.716823+0000","last_active":"2023-11-08T19:28:03.720726+0000","last_peered":"2023-11-08T19:28:03.720726+0000","last_clean":"2023-11-08T19:28:03.720726+0000","last_became_active":"2023-11-08T19:28:03.716545+0000","last_became_peered":"2023-11-08T19:28:03.716545+0000","last_unstale":"2023-11-08T19:28:03.720726+0000","last_undegraded":"2023-11-08T19:28:03.720726+0000","last_fullsized":"2023-11-08T19:28:03.720726+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:39:03.752939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.712391+0000","last_change":"2023-11-08T19:28:02.703978+0000","last_active":"2023-11-08T19:28:02.712391+0000","last_peered":"2023-11-08T19:28:02.712391+0000","last_clean":"2023-11-08T19:28:02.712391+0000","last_became_active":"2023-11-08T19:28:02.703654+0000","last_became_peered":"2023-11-08T19:28:02.703654+0000","last_unstale":"2023-11-08T19:28:02.712391+0000","last_undegraded":"2023-11-08T19:28:02.712391+0000","last_fullsized":"2023-11-08T19:28:02.712391+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:39:48.728129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.698064+0000","last_change":"2023-11-08T19:27:08.193990+0000","last_active":"2023-11-08T19:28:01.698064+0000","last_peered":"2023-11-08T19:28:01.698064+0000","last_clean":"2023-11-08T19:28:01.698064+0000","last_became_active":"2023-11-08T19:27:08.193624+0000","last_became_peered":"2023-11-08T19:27:08.193624+0000","last_unstale":"2023-11-08T19:28:01.698064+0000","last_undegraded":"2023-11-08T19:28:01.698064+0000","last_fullsized":"2023-11-08T19:28:01.698064+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:46:41.321120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":64,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.694552+0000","last_change":"2023-11-08T19:26:06.423863+0000","last_active":"2023-11-08T19:28:01.694552+0000","last_peered":"2023-11-08T19:28:01.694552+0000","last_clean":"2023-11-08T19:28:01.694552+0000","last_became_active":"2023-11-08T19:26:04.608639+0000","last_became_peered":"2023-11-08T19:26:04.608639+0000","last_unstale":"2023-11-08T19:28:01.694552+0000","last_undegraded":"2023-11-08T19:28:01.694552+0000","last_fullsized":"2023-11-08T19:28:01.694552+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:04:43.631102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044384999999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.698449+0000","last_change":"2023-11-08T19:27:09.283232+0000","last_active":"2023-11-08T19:28:01.698449+0000","last_peered":"2023-11-08T19:28:01.698449+0000","last_clean":"2023-11-08T19:28:01.698449+0000","last_became_active":"2023-11-08T19:27:07.190931+0000","last_became_peered":"2023-11-08T19:27:07.190931+0000","last_unstale":"2023-11-08T19:28:01.698449+0000","last_undegraded":"2023-11-08T19:28:01.698449+0000","last_fullsized":"2023-11-08T19:28:01.698449+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:43:28.225967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034146799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.713166+0000","last_change":"2023-11-08T19:28:02.713166+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.713166+0000","last_undegraded":"2023-11-08T19:28:02.713166+0000","last_fullsized":"2023-11-08T19:28:02.713166+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.722936+0000","last_change":"2023-11-08T19:28:02.717768+0000","last_active":"2023-11-08T19:28:02.722936+0000","last_peered":"2023-11-08T19:28:02.722936+0000","last_clean":"2023-11-08T19:28:02.722936+0000","last_became_active":"2023-11-08T19:28:02.717553+0000","last_became_peered":"2023-11-08T19:28:02.717553+0000","last_unstale":"2023-11-08T19:28:02.722936+0000","last_undegraded":"2023-11-08T19:28:02.722936+0000","last_fullsized":"2023-11-08T19:28:02.722936+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:50:10.975026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.694667+0000","last_change":"2023-11-08T19:27:08.187252+0000","last_active":"2023-11-08T19:28:01.694667+0000","last_peered":"2023-11-08T19:28:01.694667+0000","last_clean":"2023-11-08T19:28:01.694667+0000","last_became_active":"2023-11-08T19:27:08.186952+0000","last_became_peered":"2023-11-08T19:27:08.186952+0000","last_unstale":"2023-11-08T19:28:01.694667+0000","last_undegraded":"2023-11-08T19:28:01.694667+0000","last_fullsized":"2023-11-08T19:28:01.694667+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:12:52.022374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":58,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.694052+0000","last_change":"2023-11-08T19:26:06.394081+0000","last_active":"2023-11-08T19:28:01.694052+0000","last_peered":"2023-11-08T19:28:01.694052+0000","last_clean":"2023-11-08T19:28:01.694052+0000","last_became_active":"2023-11-08T19:26:04.602137+0000","last_became_peered":"2023-11-08T19:26:04.602137+0000","last_unstale":"2023-11-08T19:28:01.694052+0000","last_undegraded":"2023-11-08T19:28:01.694052+0000","last_fullsized":"2023-11-08T19:28:01.694052+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:05:46.951663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056998200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.690432+0000","last_change":"2023-11-08T19:27:09.178816+0000","last_active":"2023-11-08T19:28:01.690432+0000","last_peered":"2023-11-08T19:28:01.690432+0000","last_clean":"2023-11-08T19:28:01.690432+0000","last_became_active":"2023-11-08T19:27:07.192001+0000","last_became_peered":"2023-11-08T19:27:07.192001+0000","last_unstale":"2023-11-08T19:28:01.690432+0000","last_undegraded":"2023-11-08T19:28:01.690432+0000","last_fullsized":"2023-11-08T19:28:01.690432+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:04:12.481212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045359000000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.698299+0000","last_change":"2023-11-08T19:28:02.698299+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.698299+0000","last_undegraded":"2023-11-08T19:28:02.698299+0000","last_fullsized":"2023-11-08T19:28:02.698299+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.716969+0000","last_change":"2023-11-08T19:28:02.711841+0000","last_active":"2023-11-08T19:28:02.716969+0000","last_peered":"2023-11-08T19:28:02.716969+0000","last_clean":"2023-11-08T19:28:02.716969+0000","last_became_active":"2023-11-08T19:28:02.711455+0000","last_became_peered":"2023-11-08T19:28:02.711455+0000","last_unstale":"2023-11-08T19:28:02.716969+0000","last_undegraded":"2023-11-08T19:28:02.716969+0000","last_fullsized":"2023-11-08T19:28:02.716969+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:29:25.082011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.695522+0000","last_change":"2023-11-08T19:27:08.193836+0000","last_active":"2023-11-08T19:28:01.695522+0000","last_peered":"2023-11-08T19:28:01.695522+0000","last_clean":"2023-11-08T19:28:01.695522+0000","last_became_active":"2023-11-08T19:27:08.193458+0000","last_became_peered":"2023-11-08T19:27:08.193458+0000","last_unstale":"2023-11-08T19:28:01.695522+0000","last_undegraded":"2023-11-08T19:28:01.695522+0000","last_fullsized":"2023-11-08T19:28:01.695522+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:14:18.610061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":60,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.698115+0000","last_change":"2023-11-08T19:26:06.541338+0000","last_active":"2023-11-08T19:28:01.698115+0000","last_peered":"2023-11-08T19:28:01.698115+0000","last_clean":"2023-11-08T19:28:01.698115+0000","last_became_active":"2023-11-08T19:26:04.602183+0000","last_became_peered":"2023-11-08T19:26:04.602183+0000","last_unstale":"2023-11-08T19:28:01.698115+0000","last_undegraded":"2023-11-08T19:28:01.698115+0000","last_fullsized":"2023-11-08T19:28:01.698115+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:06:47.446686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052029099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.701560+0000","last_change":"2023-11-08T19:27:09.181622+0000","last_active":"2023-11-08T19:28:02.701560+0000","last_peered":"2023-11-08T19:28:02.701560+0000","last_clean":"2023-11-08T19:28:02.701560+0000","last_became_active":"2023-11-08T19:27:07.184507+0000","last_became_peered":"2023-11-08T19:27:07.184507+0000","last_unstale":"2023-11-08T19:28:02.701560+0000","last_undegraded":"2023-11-08T19:28:02.701560+0000","last_fullsized":"2023-11-08T19:28:02.701560+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:58:48.357266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051924700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.699401+0000","last_change":"2023-11-08T19:28:02.699401+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.699401+0000","last_undegraded":"2023-11-08T19:28:02.699401+0000","last_fullsized":"2023-11-08T19:28:02.699401+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.717064+0000","last_change":"2023-11-08T19:28:02.713407+0000","last_active":"2023-11-08T19:28:02.717064+0000","last_peered":"2023-11-08T19:28:02.717064+0000","last_clean":"2023-11-08T19:28:02.717064+0000","last_became_active":"2023-11-08T19:28:02.713144+0000","last_became_peered":"2023-11-08T19:28:02.713144+0000","last_unstale":"2023-11-08T19:28:02.717064+0000","last_undegraded":"2023-11-08T19:28:02.717064+0000","last_fullsized":"2023-11-08T19:28:02.717064+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:38:53.800926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.698458+0000","last_change":"2023-11-08T19:27:08.194263+0000","last_active":"2023-11-08T19:28:01.698458+0000","last_peered":"2023-11-08T19:28:01.698458+0000","last_clean":"2023-11-08T19:28:01.698458+0000","last_became_active":"2023-11-08T19:27:08.193938+0000","last_became_peered":"2023-11-08T19:27:08.193938+0000","last_unstale":"2023-11-08T19:28:01.698458+0000","last_undegraded":"2023-11-08T19:28:01.698458+0000","last_fullsized":"2023-11-08T19:28:01.698458+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:40:49.009717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":59,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.711234+0000","last_change":"2023-11-08T19:26:06.265690+0000","last_active":"2023-11-08T19:28:02.711234+0000","last_peered":"2023-11-08T19:28:02.711234+0000","last_clean":"2023-11-08T19:28:02.711234+0000","last_became_active":"2023-11-08T19:26:04.603397+0000","last_became_peered":"2023-11-08T19:26:04.603397+0000","last_unstale":"2023-11-08T19:28:02.711234+0000","last_undegraded":"2023-11-08T19:28:02.711234+0000","last_fullsized":"2023-11-08T19:28:02.711234+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:50:43.827570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053361499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.698140+0000","last_change":"2023-11-08T19:27:09.282750+0000","last_active":"2023-11-08T19:28:01.698140+0000","last_peered":"2023-11-08T19:28:01.698140+0000","last_clean":"2023-11-08T19:28:01.698140+0000","last_became_active":"2023-11-08T19:27:07.189943+0000","last_became_peered":"2023-11-08T19:27:07.189943+0000","last_unstale":"2023-11-08T19:28:01.698140+0000","last_undegraded":"2023-11-08T19:28:01.698140+0000","last_fullsized":"2023-11-08T19:28:01.698140+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:31:16.081058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000398725,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.702677+0000","last_change":"2023-11-08T19:28:02.702677+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.702677+0000","last_undegraded":"2023-11-08T19:28:02.702677+0000","last_fullsized":"2023-11-08T19:28:02.702677+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.724982+0000","last_change":"2023-11-08T19:28:02.715521+0000","last_active":"2023-11-08T19:28:02.724982+0000","last_peered":"2023-11-08T19:28:02.724982+0000","last_clean":"2023-11-08T19:28:02.724982+0000","last_became_active":"2023-11-08T19:28:02.715145+0000","last_became_peered":"2023-11-08T19:28:02.715145+0000","last_unstale":"2023-11-08T19:28:02.724982+0000","last_undegraded":"2023-11-08T19:28:02.724982+0000","last_fullsized":"2023-11-08T19:28:02.724982+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:52:23.978416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.702217+0000","last_change":"2023-11-08T19:27:08.188147+0000","last_active":"2023-11-08T19:28:02.702217+0000","last_peered":"2023-11-08T19:28:02.702217+0000","last_clean":"2023-11-08T19:28:02.702217+0000","last_became_active":"2023-11-08T19:27:08.187487+0000","last_became_peered":"2023-11-08T19:27:08.187487+0000","last_unstale":"2023-11-08T19:28:02.702217+0000","last_undegraded":"2023-11-08T19:28:02.702217+0000","last_fullsized":"2023-11-08T19:28:02.702217+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:22:40.300185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":64,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.696443+0000","last_change":"2023-11-08T19:26:06.425133+0000","last_active":"2023-11-08T19:28:01.696443+0000","last_peered":"2023-11-08T19:28:01.696443+0000","last_clean":"2023-11-08T19:28:01.696443+0000","last_became_active":"2023-11-08T19:26:04.601103+0000","last_became_peered":"2023-11-08T19:26:04.601103+0000","last_unstale":"2023-11-08T19:28:01.696443+0000","last_undegraded":"2023-11-08T19:28:01.696443+0000","last_fullsized":"2023-11-08T19:28:01.696443+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:31:14.042561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000229244,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.701647+0000","last_change":"2023-11-08T19:27:09.179852+0000","last_active":"2023-11-08T19:28:02.701647+0000","last_peered":"2023-11-08T19:28:02.701647+0000","last_clean":"2023-11-08T19:28:02.701647+0000","last_became_active":"2023-11-08T19:27:07.191926+0000","last_became_peered":"2023-11-08T19:27:07.191926+0000","last_unstale":"2023-11-08T19:28:02.701647+0000","last_undegraded":"2023-11-08T19:28:02.701647+0000","last_fullsized":"2023-11-08T19:28:02.701647+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:20:38.080129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055457100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.711863+0000","last_change":"2023-11-08T19:28:02.711863+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.711863+0000","last_undegraded":"2023-11-08T19:28:02.711863+0000","last_fullsized":"2023-11-08T19:28:02.711863+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"91'3","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.713504+0000","last_change":"2023-11-08T19:28:02.708691+0000","last_active":"2023-11-08T19:28:03.713504+0000","last_peered":"2023-11-08T19:28:03.713504+0000","last_clean":"2023-11-08T19:28:03.713504+0000","last_became_active":"2023-11-08T19:28:02.708037+0000","last_became_peered":"2023-11-08T19:28:02.708037+0000","last_unstale":"2023-11-08T19:28:03.713504+0000","last_undegraded":"2023-11-08T19:28:03.713504+0000","last_fullsized":"2023-11-08T19:28:03.713504+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:07:33.241500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.702633+0000","last_change":"2023-11-08T19:27:08.186054+0000","last_active":"2023-11-08T19:28:02.702633+0000","last_peered":"2023-11-08T19:28:02.702633+0000","last_clean":"2023-11-08T19:28:02.702633+0000","last_became_active":"2023-11-08T19:27:08.185581+0000","last_became_peered":"2023-11-08T19:27:08.185581+0000","last_unstale":"2023-11-08T19:28:02.702633+0000","last_undegraded":"2023-11-08T19:28:02.702633+0000","last_fullsized":"2023-11-08T19:28:02.702633+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:00:55.878397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":71,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.701211+0000","last_change":"2023-11-08T19:26:05.609625+0000","last_active":"2023-11-08T19:28:02.701211+0000","last_peered":"2023-11-08T19:28:02.701211+0000","last_clean":"2023-11-08T19:28:02.701211+0000","last_became_active":"2023-11-08T19:26:04.606312+0000","last_became_peered":"2023-11-08T19:26:04.606312+0000","last_unstale":"2023-11-08T19:28:02.701211+0000","last_undegraded":"2023-11-08T19:28:02.701211+0000","last_fullsized":"2023-11-08T19:28:02.701211+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:54:42.032901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000505935,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.690085+0000","last_change":"2023-11-08T19:27:09.177985+0000","last_active":"2023-11-08T19:28:01.690085+0000","last_peered":"2023-11-08T19:28:01.690085+0000","last_clean":"2023-11-08T19:28:01.690085+0000","last_became_active":"2023-11-08T19:27:07.193254+0000","last_became_peered":"2023-11-08T19:27:07.193254+0000","last_unstale":"2023-11-08T19:28:01.690085+0000","last_undegraded":"2023-11-08T19:28:01.690085+0000","last_fullsized":"2023-11-08T19:28:01.690085+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:23:38.662629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041830799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2023-11-08T19:28:02.703260+0000","last_change":"2023-11-08T19:28:02.703260+0000","last_active":"2023-11-08T19:28:02.688498+0000","last_peered":"2023-11-08T19:28:02.688498+0000","last_clean":"2023-11-08T19:28:02.688498+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:02.703260+0000","last_undegraded":"2023-11-08T19:28:02.703260+0000","last_fullsized":"2023-11-08T19:28:02.703260+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"91'6","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.727979+0000","last_change":"2023-11-08T19:28:02.700819+0000","last_active":"2023-11-08T19:28:03.727979+0000","last_peered":"2023-11-08T19:28:03.727979+0000","last_clean":"2023-11-08T19:28:03.727979+0000","last_became_active":"2023-11-08T19:28:02.700537+0000","last_became_peered":"2023-11-08T19:28:02.700537+0000","last_unstale":"2023-11-08T19:28:03.727979+0000","last_undegraded":"2023-11-08T19:28:03.727979+0000","last_fullsized":"2023-11-08T19:28:03.727979+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:25:00.088013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-08T19:28:02.703640+0000","last_change":"2023-11-08T19:27:08.188860+0000","last_active":"2023-11-08T19:28:02.703640+0000","last_peered":"2023-11-08T19:28:02.703640+0000","last_clean":"2023-11-08T19:28:02.703640+0000","last_became_active":"2023-11-08T19:27:08.188560+0000","last_became_peered":"2023-11-08T19:27:08.188560+0000","last_unstale":"2023-11-08T19:28:02.703640+0000","last_undegraded":"2023-11-08T19:28:02.703640+0000","last_fullsized":"2023-11-08T19:28:02.703640+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:18:20.969491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":59,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.695852+0000","last_change":"2023-11-08T19:26:06.297255+0000","last_active":"2023-11-08T19:28:01.695852+0000","last_peered":"2023-11-08T19:28:01.695852+0000","last_clean":"2023-11-08T19:28:01.695852+0000","last_became_active":"2023-11-08T19:26:04.603979+0000","last_became_peered":"2023-11-08T19:26:04.603979+0000","last_unstale":"2023-11-08T19:28:01.695852+0000","last_undegraded":"2023-11-08T19:28:01.695852+0000","last_fullsized":"2023-11-08T19:28:01.695852+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:40:11.164951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079032099999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-08T19:28:01.693851+0000","last_change":"2023-11-08T19:27:09.331990+0000","last_active":"2023-11-08T19:28:01.693851+0000","last_peered":"2023-11-08T19:28:01.693851+0000","last_clean":"2023-11-08T19:28:01.693851+0000","last_became_active":"2023-11-08T19:27:07.185478+0000","last_became_peered":"2023-11-08T19:27:07.185478+0000","last_unstale":"2023-11-08T19:28:01.693851+0000","last_undegraded":"2023-11-08T19:28:01.693851+0000","last_fullsized":"2023-11-08T19:28:01.693851+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:20:38.080129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010661850000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":1794,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12,"ondisk_log_size":12,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574892,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28704,"kb_used_data":1680,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343136,"statfs":{"total":96636764160,"available":96607371264,"internally_reserved":0,"allocated":1720320,"data_stored":1117537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:27:53 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.50600000000000001,"15min":0.498},"min":{"1min":0.34300000000000003,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.72899999999999998,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.437},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.51900000000000002,"15min":0.51500000000000001},"min":{"1min":0.36899999999999999,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.69099999999999995,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.621}]},{"osd":1,"last update":"Wed Nov 8 19:27:53 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56899999999999995,"15min":0.56599999999999995},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.56399999999999995,"15min":0.55600000000000005},"min":{"1min":0.435,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.59199999999999997}]},{"osd":2,"last update":"Wed Nov 8 19:27:53 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.50900000000000001,"15min":0.49299999999999999},"min":{"1min":0.38700000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.55000000000000004,"15min":0.54800000000000004},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.38500000000000001}]},{"osd":3,"last update":"Wed Nov 8 19:27:53 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.55300000000000005,"15min":0.55700000000000005},"min":{"1min":0.375,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.66200000000000003,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59399999999999997,"15min":0.57799999999999996},"min":{"1min":0.49399999999999999,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.77000000000000002,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.5}]},{"osd":4,"last update":"Wed Nov 8 19:27:53 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.50900000000000001,"15min":0.50700000000000001},"min":{"1min":0.36299999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.64500000000000002,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.54600000000000004,"15min":0.54100000000000004},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.41099999999999998}]},{"osd":5,"last update":"Wed Nov 8 19:27:53 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.58599999999999997,"15min":0.58099999999999996},"min":{"1min":0.41199999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.71999999999999997,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.62,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.84499999999999997,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.64300000000000002}]},{"osd":6,"last update":"Wed Nov 8 19:27:53 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.60199999999999998,"15min":0.60399999999999998},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.78400000000000003,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.61499999999999999,"15min":0.61099999999999999},"min":{"1min":0.5,"5min":0.443,"15min":0.443},"max":{"1min":0.82999999999999996,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.57299999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574892,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28132,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343708,"statfs":{"total":96636764160,"available":96607956992,"internally_reserved":0,"allocated":1138688,"data_stored":529225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":1.0780000000000001,"15min":1.319},"min":{"1min":0.33800000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.67800000000000005,"5min":21.359999999999999,"15min":21.359999999999999},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.78300000000000003,"15min":0.88700000000000001},"min":{"1min":0.378,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.73799999999999999,"5min":10.329000000000001,"15min":10.329000000000001},"last":0.434}]},{"osd":1,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.57899999999999996,"15min":0.60899999999999999},"min":{"1min":0.36899999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.71899999999999997,"5min":3.4239999999999999,"15min":3.4239999999999999},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.61699999999999999,"15min":0.63800000000000001},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.73199999999999998,"5min":3.4809999999999999,"15min":3.4809999999999999},"last":0.45800000000000002}]},{"osd":2,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.80100000000000005,"15min":0.90500000000000003},"min":{"1min":0.40999999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.69299999999999995,"5min":10.398,"15min":10.398},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.65400000000000003,"15min":0.68999999999999995},"min":{"1min":0.40799999999999997,"5min":0.318,"15min":0.318},"max":{"1min":0.746,"5min":5.0170000000000003,"15min":5.0170000000000003},"last":0.39300000000000002}]},{"osd":3,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.63900000000000001,"15min":0.66300000000000003},"min":{"1min":0.39500000000000002,"5min":0.317,"15min":0.317},"max":{"1min":0.77300000000000002,"5min":3.5710000000000002,"15min":3.5710000000000002},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.82999999999999996,"15min":0.93799999999999994},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.86799999999999999,"5min":10.368,"15min":10.368},"last":0.56299999999999994}]},{"osd":4,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.53700000000000003,"15min":0.51700000000000002},"min":{"1min":0.38,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.56999999999999995,"15min":0.56200000000000006},"min":{"1min":0.40600000000000003,"5min":0.373,"15min":0.373},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.53300000000000003}]},{"osd":5,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.55000000000000004,"15min":0.53200000000000003},"min":{"1min":0.40699999999999997,"5min":0.307,"15min":0.307},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.503},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.60599999999999998,"15min":0.58599999999999997},"min":{"1min":0.42899999999999999,"5min":0.378,"15min":0.378},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.55200000000000005}]},{"osd":7,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.59899999999999998,"15min":0.57899999999999996},"min":{"1min":0.41299999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.81200000000000006,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.61099999999999999,"15min":0.59399999999999997},"min":{"1min":0.54000000000000004,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.47499999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574892,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28104,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343736,"statfs":{"total":96636764160,"available":96607985664,"internally_reserved":0,"allocated":1105920,"data_stored":519171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:27:47 2023","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.36899999999999999,"15min":0.379},"min":{"1min":0.22800000000000001,"5min":0.219,"15min":0.219},"max":{"1min":0.52400000000000002,"5min":1.389,"15min":1.389},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.33800000000000002,"5min":0.58299999999999996,"15min":0.68999999999999995},"min":{"1min":0.25900000000000001,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.504,"5min":9.1370000000000005,"15min":9.1370000000000005},"last":0.36499999999999999}]},{"osd":2,"last update":"Wed Nov 8 19:27:47 2023","interfaces":[{"interface":"back","average":{"1min":0.33600000000000002,"5min":0.35399999999999998,"15min":0.36299999999999999},"min":{"1min":0.254,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.50600000000000001,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.373,"5min":0.41099999999999998,"15min":0.42499999999999999},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.58299999999999996,"5min":1.425,"15min":1.425},"last":0.47899999999999998}]},{"osd":3,"last update":"Wed Nov 8 19:27:47 2023","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.69999999999999996,"15min":0.84099999999999997},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.54600000000000004,"5min":11.957000000000001,"15min":11.957000000000001},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.85799999999999998,"15min":1.0740000000000001},"min":{"1min":0.27700000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.56299999999999994,"5min":18.039000000000001,"15min":18.039000000000001},"last":0.437}]},{"osd":4,"last update":"Wed Nov 8 19:27:47 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.73399999999999999,"15min":0.79800000000000004},"min":{"1min":0.43099999999999999,"5min":0.372,"15min":0.372},"max":{"1min":0.77800000000000002,"5min":6.181,"15min":6.181},"last":0.94699999999999995},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.629,"15min":0.63700000000000001},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.76900000000000002,"5min":1.488,"15min":1.488},"last":0.996}]},{"osd":5,"last update":"Wed Nov 8 19:27:47 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.84099999999999997,"15min":0.93700000000000006},"min":{"1min":0.437,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.74199999999999999,"5min":9.2420000000000009,"15min":9.2420000000000009},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.77200000000000002,"15min":0.84399999999999997},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.745,"5min":6.2110000000000003,"15min":6.2110000000000003},"last":0.878}]},{"osd":6,"last update":"Wed Nov 8 19:27:47 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.84899999999999998,"15min":0.94099999999999995},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.751,"5min":9.1880000000000006,"15min":9.1880000000000006},"last":0.755},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.65500000000000003,"15min":0.66300000000000003},"min":{"1min":0.38800000000000001,"5min":0.38,"15min":0.38},"max":{"1min":0.79400000000000004,"5min":1.462,"15min":1.462},"last":0.85799999999999998}]},{"osd":7,"last update":"Wed Nov 8 19:27:47 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.77400000000000002,"15min":0.83799999999999997},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.76600000000000001,"5min":6.1479999999999997,"15min":6.1479999999999997},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.872,"15min":0.97599999999999998},"min":{"1min":0.46100000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.76400000000000001,"5min":9.2170000000000005,"15min":9.2170000000000005},"last":0.90600000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574892,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28680,"kb_used_data":1660,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343160,"statfs":{"total":96636764160,"available":96607395840,"internally_reserved":0,"allocated":1699840,"data_stored":1109539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Nov 8 19:27:50 2023","interfaces":[{"interface":"back","average":{"1min":0.32100000000000001,"5min":0.441,"15min":0.495},"min":{"1min":0.23599999999999999,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.48799999999999999,"5min":4.1200000000000001,"15min":4.1200000000000001},"last":0.247},{"interface":"front","average":{"1min":0.32300000000000001,"5min":0.85199999999999998,"15min":1.079},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.51200000000000001,"5min":16.940999999999999,"15min":16.940999999999999},"last":0.34000000000000002}]},{"osd":2,"last update":"Wed Nov 8 19:27:50 2023","interfaces":[{"interface":"back","average":{"1min":0.318,"5min":0.374,"15min":0.39400000000000002},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.45600000000000002,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.57499999999999996,"15min":0.66600000000000004},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.58299999999999996,"5min":7.0209999999999999,"15min":7.0209999999999999},"last":0.28699999999999998}]},{"osd":3,"last update":"Wed Nov 8 19:27:50 2023","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.49299999999999999,"15min":0.54800000000000004},"min":{"1min":0.25,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.56699999999999995,"5min":4.0940000000000003,"15min":4.0940000000000003},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.37,"5min":0.42299999999999999,"15min":0.442},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.53600000000000003,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.36099999999999999}]},{"osd":4,"last update":"Wed Nov 8 19:27:50 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.69299999999999995,"15min":0.751},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.76800000000000002,"5min":4.0049999999999999,"15min":4.0049999999999999},"last":0.375},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.62,"15min":0.63500000000000001},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.77500000000000002,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.40000000000000002}]},{"osd":5,"last update":"Wed Nov 8 19:27:50 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.71199999999999997,"15min":0.76600000000000001},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.79200000000000004,"5min":4.3090000000000002,"15min":4.3090000000000002},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.70899999999999996,"15min":0.75700000000000001},"min":{"1min":0.45600000000000002,"5min":0.437,"15min":0.437},"max":{"1min":0.84699999999999998,"5min":4.0709999999999997,"15min":4.0709999999999997},"last":0.441}]},{"osd":6,"last update":"Wed Nov 8 19:27:50 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.65500000000000003,"15min":0.66900000000000004},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.81200000000000006,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.76800000000000002,"15min":0.82399999999999995},"min":{"1min":0.47699999999999998,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.84199999999999997,"5min":4.2560000000000002,"15min":4.2560000000000002},"last":0.56200000000000006}]},{"osd":7,"last update":"Wed Nov 8 19:27:50 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.72699999999999998,"15min":0.77700000000000002},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.78700000000000003,"5min":4.2850000000000001,"15min":4.2850000000000001},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.753,"15min":0.80500000000000005},"min":{"1min":0.48699999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.82499999999999996,"5min":4.0460000000000003,"15min":4.0460000000000003},"last":0.41399999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574892,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28088,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343752,"statfs":{"total":96636764160,"available":96608002048,"internally_reserved":0,"allocated":1093632,"data_stored":511833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:27:47 2023","interfaces":[{"interface":"back","average":{"1min":0.32100000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.47299999999999998,"5min":0.496,"15min":0.496},"last":0.28499999999999998},{"interface":"front","average":{"1min":0.31,"5min":0.51900000000000002,"15min":0.61299999999999999},"min":{"1min":0.219,"5min":0.218,"15min":0.218},"max":{"1min":0.501,"5min":6.0999999999999996,"15min":6.0999999999999996},"last":0.23000000000000001}]},{"osd":1,"last update":"Wed Nov 8 19:27:47 2023","interfaces":[{"interface":"back","average":{"1min":0.309,"5min":0.33400000000000002,"15min":0.33800000000000002},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.42699999999999999,"5min":0.48299999999999998,"15min":0.48299999999999998},"last":0.26300000000000001},{"interface":"front","average":{"1min":0.32500000000000001,"5min":0.372,"15min":0.38400000000000001},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.41899999999999998,"5min":0.624,"15min":0.624},"last":0.32600000000000001}]},{"osd":3,"last update":"Wed Nov 8 19:27:47 2023","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.376,"15min":0.38200000000000001},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.46500000000000002,"5min":0.53700000000000003,"15min":0.53700000000000003},"last":0.375},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.38300000000000001,"15min":0.39200000000000002},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.53400000000000003,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.34899999999999998}]},{"osd":4,"last update":"Wed Nov 8 19:27:47 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.56699999999999995,"15min":0.57899999999999996},"min":{"1min":0.39000000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.57699999999999996,"15min":0.58199999999999996},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.77100000000000002,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.5}]},{"osd":5,"last update":"Wed Nov 8 19:27:47 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.59199999999999997,"15min":0.58499999999999996},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.78100000000000003,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.628},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.60499999999999998,"15min":0.61099999999999999},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.76400000000000001,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.57199999999999995}]},{"osd":6,"last update":"Wed Nov 8 19:27:47 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59799999999999998,"15min":0.59999999999999998},"min":{"1min":0.47199999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.79700000000000004,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57799999999999996,"15min":0.57899999999999996},"min":{"1min":0.41499999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.80000000000000004,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.58899999999999997}]},{"osd":7,"last update":"Wed Nov 8 19:27:47 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.58199999999999996,"15min":0.57999999999999996},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.73199999999999998,"5min":0.878,"15min":0.878},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60099999999999998,"15min":0.60299999999999998},"min":{"1min":0.45300000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.82199999999999995,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.65700000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574893,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28104,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343736,"statfs":{"total":96636764160,"available":96607985664,"internally_reserved":0,"allocated":1105920,"data_stored":519171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.318,"5min":0.33300000000000002,"15min":0.33900000000000002},"min":{"1min":0.224,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.49099999999999999,"5min":0.626,"15min":0.626},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.34999999999999998,"15min":0.34899999999999998},"min":{"1min":0.28000000000000003,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.48699999999999999,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.26200000000000001}]},{"osd":1,"last update":"Wed Nov 8 19:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.34699999999999998,"15min":0.35199999999999998},"min":{"1min":0.25800000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.42299999999999999,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":0.29599999999999999},{"interface":"front","average":{"1min":0.38,"5min":0.38300000000000001,"15min":0.38400000000000001},"min":{"1min":0.28000000000000003,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.48699999999999999,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.27900000000000003}]},{"osd":2,"last update":"Wed Nov 8 19:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.376,"15min":0.372},"min":{"1min":0.32900000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.46300000000000002,"5min":0.58499999999999996,"15min":0.58499999999999996},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.40400000000000003,"15min":0.40400000000000003},"min":{"1min":0.32700000000000001,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.47399999999999998,"5min":0.58499999999999996,"15min":0.58499999999999996},"last":0.32000000000000001}]},{"osd":4,"last update":"Wed Nov 8 19:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.60799999999999998,"15min":0.60099999999999998},"min":{"1min":0.45300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.624,"5min":0.61199999999999999,"15min":0.60699999999999998},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.84499999999999997,"5min":0.871,"15min":0.871},"last":0.56100000000000005}]},{"osd":5,"last update":"Wed Nov 8 19:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.58699999999999997,"15min":0.57899999999999996},"min":{"1min":0.46700000000000003,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.76600000000000001,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.58899999999999997,"15min":0.58099999999999996},"min":{"1min":0.42399999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.78600000000000003,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.54700000000000004}]},{"osd":6,"last update":"Wed Nov 8 19:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.58799999999999997,"15min":0.58199999999999996},"min":{"1min":0.436,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.73299999999999998,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.61299999999999999,"15min":0.60699999999999998},"min":{"1min":0.49099999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.75900000000000001,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.57399999999999995}]},{"osd":7,"last update":"Wed Nov 8 19:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.59899999999999998,"15min":0.59099999999999997},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.72199999999999998,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.60199999999999998,"15min":0.59799999999999998},"min":{"1min":0.45800000000000002,"5min":0.432,"15min":0.432},"max":{"1min":0.80700000000000005,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.58299999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574892,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28104,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343736,"statfs":{"total":96636764160,"available":96607985664,"internally_reserved":0,"allocated":1105920,"data_stored":519171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:27:55 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.68200000000000005,"15min":0.74399999999999999},"min":{"1min":0.35699999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.72099999999999997,"5min":6.5970000000000004,"15min":6.5970000000000004},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.55500000000000005,"15min":0.56999999999999995},"min":{"1min":0.35799999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.70999999999999996,"5min":2.5030000000000001,"15min":2.5030000000000001},"last":0.46899999999999997}]},{"osd":1,"last update":"Wed Nov 8 19:27:55 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.51300000000000001,"15min":0.501},"min":{"1min":0.36199999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.75800000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.629},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.52100000000000002,"15min":0.51200000000000001},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.70199999999999996}]},{"osd":2,"last update":"Wed Nov 8 19:27:55 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.55400000000000005,"15min":0.55200000000000005},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.80600000000000005,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.748,"15min":0.81299999999999994},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.86199999999999999,"5min":6.6920000000000002,"15min":6.6920000000000002},"last":0.66900000000000004}]},{"osd":3,"last update":"Wed Nov 8 19:27:55 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.58299999999999996,"15min":0.58699999999999997},"min":{"1min":0.378,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.79800000000000004,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.60899999999999999,"15min":0.60999999999999999},"min":{"1min":0.42499999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.71399999999999997,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.68500000000000005}]},{"osd":5,"last update":"Wed Nov 8 19:27:55 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.55000000000000004,"15min":0.54800000000000004},"min":{"1min":0.35099999999999998,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.81599999999999995,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.58499999999999996,"15min":0.59299999999999997},"min":{"1min":0.39400000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.745,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.53000000000000003}]},{"osd":6,"last update":"Wed Nov 8 19:27:55 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58599999999999997,"15min":0.58499999999999996},"min":{"1min":0.373,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.627,"15min":0.61799999999999999},"min":{"1min":0.46400000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.73099999999999998}]},{"osd":7,"last update":"Wed Nov 8 19:27:55 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.61599999999999999,"15min":0.629},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.84499999999999997,"5min":1.141,"15min":1.141},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.64100000000000001,"15min":0.64600000000000002},"min":{"1min":0.42399999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.86399999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.74199999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574892,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28104,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343736,"statfs":{"total":96636764160,"available":96607985664,"internally_reserved":0,"allocated":1105920,"data_stored":519171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:27:46 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.51400000000000001,"15min":0.52000000000000002},"min":{"1min":0.32200000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.73999999999999999,"5min":0.753,"15min":0.753},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.51900000000000002,"15min":0.51200000000000001},"min":{"1min":0.38500000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.69399999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.495}]},{"osd":1,"last update":"Wed Nov 8 19:27:46 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.51900000000000002,"15min":0.52200000000000002},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.70799999999999996,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.53400000000000003,"15min":0.53600000000000003},"min":{"1min":0.38400000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.70499999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.56899999999999995}]},{"osd":2,"last update":"Wed Nov 8 19:27:46 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.53600000000000003,"15min":0.53500000000000003},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.68000000000000005,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.55800000000000005,"15min":0.56999999999999995},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.69799999999999995,"5min":0.747,"15min":0.747},"last":0.47699999999999998}]},{"osd":3,"last update":"Wed Nov 8 19:27:46 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.54700000000000004,"15min":0.54900000000000004},"min":{"1min":0.40899999999999997,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.72199999999999998,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.57899999999999996,"15min":0.59599999999999997},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.69099999999999995,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.627}]},{"osd":4,"last update":"Wed Nov 8 19:27:46 2023","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.53200000000000003,"15min":0.55200000000000005},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.67200000000000004,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.56200000000000006,"15min":0.56899999999999995},"min":{"1min":0.36599999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.37}]},{"osd":6,"last update":"Wed Nov 8 19:27:46 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.59399999999999997,"15min":0.60699999999999998},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.755,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.59799999999999998,"15min":0.61099999999999999},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.72099999999999997,"5min":0.745,"15min":0.745},"last":0.60999999999999999}]},{"osd":7,"last update":"Wed Nov 8 19:27:46 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.59399999999999997,"15min":0.60599999999999998},"min":{"1min":0.44500000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.74199999999999999,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.625,"15min":0.64200000000000002},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.752,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.59399999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-08T19:28:09.687 INFO:tasks.ceph.mds.d.smithi174.stderr:2023-11-08T19:28:09.685+0000 7f73f8270700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-08T19:28:12.436 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-08T19:28:12.571 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.569+0000 7f966cfce700 1 Processor -- start 2023-11-08T19:28:12.571 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.569+0000 7f966cfce700 1 -- start start 2023-11-08T19:28:12.574 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.570+0000 7f966cfce700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f966815dd00 0x7f96680cca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:12.575 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.570+0000 7f966cfce700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f966806b320 0x7f96680cb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:12.575 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.570+0000 7f966cfce700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96680ccfb0 con 0x7f96681618f0 2023-11-08T19:28:12.575 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.570+0000 7f966cfce700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96680cad50 con 0x7f966815dd00 2023-11-08T19:28:12.575 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.570+0000 7f966cfce700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9668161df0 con 0x7f966806b320 2023-11-08T19:28:12.577 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.570+0000 7f966659c700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f966815dd00 0x7f96680cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:12.577 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.570+0000 7f966659c700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f966815dd00 0x7f96680cca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54704/0 (socket says 172.21.15.143:54704) 2023-11-08T19:28:12.577 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.570+0000 7f9665d9b700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f96681618f0 0x7f966806ac10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:50858/0 (socket says 172.21.15.143:50858) 2023-11-08T19:28:12.578 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.570+0000 7f966659c700 1 -- 172.21.15.143:0/2502860334 learned_addr learned my addr 172.21.15.143:0/2502860334 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:28:12.578 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.570+0000 7f9666d9d700 1 --2- 172.21.15.143:0/2502860334 >> v2:172.21.15.174:3300/0 conn(0x7f966806b320 0x7f96680cb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:12.578 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.571+0000 7f966659c700 1 -- 172.21.15.143:0/2502860334 >> v2:172.21.15.174:3300/0 conn(0x7f966806b320 msgr2=0x7f96680cb1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:12.578 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.571+0000 7f966659c700 1 --2- 172.21.15.143:0/2502860334 >> v2:172.21.15.174:3300/0 conn(0x7f966806b320 0x7f96680cb1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:12.579 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.571+0000 7f966659c700 1 -- 172.21.15.143:0/2502860334 >> v1:172.21.15.143:6789/0 conn(0x7f96681618f0 legacy=0x7f966806ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:12.579 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.571+0000 7f966659c700 1 -- 172.21.15.143:0/2502860334 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96681620d0 con 0x7f966815dd00 2023-11-08T19:28:12.579 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.571+0000 7f9666d9d700 1 --2- 172.21.15.143:0/2502860334 >> v2:172.21.15.174:3300/0 conn(0x7f966806b320 0x7f96680cb1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:28:12.579 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.571+0000 7f966659c700 1 --2- 172.21.15.143:0/2502860334 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f966815dd00 0x7f96680cca70 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f96680ce8d0 tx=0x7f965c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f692bc1e5cb4df3 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:12.579 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.571+0000 7f966559a700 1 -- 172.21.15.143:0/2502860334 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f965c005e30 con 0x7f966815dd00 2023-11-08T19:28:12.580 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.571+0000 7f966559a700 1 -- 172.21.15.143:0/2502860334 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f965c004030 con 0x7f966815dd00 2023-11-08T19:28:12.580 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.571+0000 7f966559a700 1 -- 172.21.15.143:0/2502860334 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f965c0043e0 con 0x7f966815dd00 2023-11-08T19:28:12.580 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.571+0000 7f966cfce700 1 -- 172.21.15.143:0/2502860334 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f966815dd00 msgr2=0x7f96680cca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:12.580 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.571+0000 7f966cfce700 1 --2- 172.21.15.143:0/2502860334 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f966815dd00 0x7f96680cca70 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f96680ce8d0 tx=0x7f965c009bd0 comp rx=0 tx=0).stop 2023-11-08T19:28:12.580 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.571+0000 7f966cfce700 1 -- 172.21.15.143:0/2502860334 shutdown_connections 2023-11-08T19:28:12.581 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.572+0000 7f966cfce700 1 --2- 172.21.15.143:0/2502860334 >> v2:172.21.15.174:3300/0 conn(0x7f966806b320 0x7f96680cb1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:12.581 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.572+0000 7f966cfce700 1 --2- 172.21.15.143:0/2502860334 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f966815dd00 0x7f96680cca70 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:12.581 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.572+0000 7f966cfce700 1 -- 172.21.15.143:0/2502860334 >> 172.21.15.143:0/2502860334 conn(0x7f9668099ab0 msgr2=0x7f96680d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:12.581 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.572+0000 7f966cfce700 1 -- 172.21.15.143:0/2502860334 shutdown_connections 2023-11-08T19:28:12.581 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.572+0000 7f966cfce700 1 -- 172.21.15.143:0/2502860334 wait complete. 2023-11-08T19:28:12.582 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.573+0000 7f966cfce700 1 Processor -- start 2023-11-08T19:28:12.582 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.573+0000 7f966cfce700 1 -- start start 2023-11-08T19:28:12.585 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.574+0000 7f966cfce700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f966806b320 0x7f96680c4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:12.585 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.574+0000 7f966cfce700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f966815dd00 0x7f96680b1270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:12.585 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.574+0000 7f966cfce700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f96681618f0 0x7f96680b17b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:12.585 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.574+0000 7f966cfce700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f966815e760 con 0x7f966806b320 2023-11-08T19:28:12.586 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.574+0000 7f966cfce700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96680c5470 con 0x7f96681618f0 2023-11-08T19:28:12.586 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.574+0000 7f966cfce700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96680c5750 con 0x7f966815dd00 2023-11-08T19:28:12.586 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.574+0000 7f966659c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f966806b320 0x7f96680c4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:12.586 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.574+0000 7f966659c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f966806b320 0x7f96680c4c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54710/0 (socket says 172.21.15.143:54710) 2023-11-08T19:28:12.587 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.574+0000 7f966659c700 1 -- 172.21.15.143:0/2225643250 learned_addr learned my addr 172.21.15.143:0/2225643250 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:28:12.587 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.574+0000 7f9665d9b700 1 --2- 172.21.15.143:0/2225643250 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f966815dd00 0x7f96680b1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:12.587 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.574+0000 7f9666d9d700 1 --2- 172.21.15.143:0/2225643250 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f96681618f0 0x7f96680b17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:12.587 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.574+0000 7f966659c700 1 -- 172.21.15.143:0/2225643250 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f966815dd00 msgr2=0x7f96680b1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:12.587 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.574+0000 7f966659c700 1 --2- 172.21.15.143:0/2225643250 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f966815dd00 0x7f96680b1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:12.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.574+0000 7f966659c700 1 -- 172.21.15.143:0/2225643250 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f96681618f0 msgr2=0x7f96680b17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:12.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.574+0000 7f966659c700 1 --2- 172.21.15.143:0/2225643250 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f96681618f0 0x7f96680b17b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:12.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.574+0000 7f966659c700 1 -- 172.21.15.143:0/2225643250 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f965c009500 con 0x7f966806b320 2023-11-08T19:28:12.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.574+0000 7f966659c700 1 --2- 172.21.15.143:0/2225643250 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f966806b320 0x7f96680c4c40 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f965c009c10 tx=0x7f965c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:12.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.574+0000 7f9656ffd700 1 -- 172.21.15.143:0/2225643250 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f965c01d630 con 0x7f966806b320 2023-11-08T19:28:12.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.574+0000 7f9656ffd700 1 -- 172.21.15.143:0/2225643250 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f965c01d790 con 0x7f966806b320 2023-11-08T19:28:12.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.574+0000 7f9656ffd700 1 -- 172.21.15.143:0/2225643250 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f965c01db40 con 0x7f966806b320 2023-11-08T19:28:12.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.574+0000 7f966cfce700 1 -- 172.21.15.143:0/2225643250 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96680c5a30 con 0x7f966806b320 2023-11-08T19:28:12.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.574+0000 7f966cfce700 1 -- 172.21.15.143:0/2225643250 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96680c5180 con 0x7f966806b320 2023-11-08T19:28:12.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.575+0000 7f966cfce700 1 -- 172.21.15.143:0/2225643250 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96680aff60 con 0x7f966806b320 2023-11-08T19:28:12.590 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.576+0000 7f9656ffd700 1 -- 172.21.15.143:0/2225643250 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f965c01dca0 con 0x7f966806b320 2023-11-08T19:28:12.590 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.576+0000 7f9656ffd700 1 --2- 172.21.15.143:0/2225643250 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f964806af00 0x7f964806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:12.590 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.576+0000 7f9656ffd700 1 -- 172.21.15.143:0/2225643250 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6772+0+0 (secure 0 0 0) 0x7f965c08ae90 con 0x7f966806b320 2023-11-08T19:28:12.590 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.576+0000 7f9665d9b700 1 --2- 172.21.15.143:0/2225643250 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f964806af00 0x7f964806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:12.590 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.577+0000 7f9665d9b700 1 --2- 172.21.15.143:0/2225643250 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f964806af00 0x7f964806d3c0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f96681f7170 tx=0x7f9658007d70 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:12.591 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.581+0000 7f9656ffd700 1 -- 172.21.15.143:0/2225643250 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f965c055ba0 con 0x7f966806b320 2023-11-08T19:28:12.770 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.768+0000 7f966cfce700 1 -- 172.21.15.143:0/2225643250 --> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9668162690 con 0x7f964806af00 2023-11-08T19:28:12.784 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.783+0000 7f9656ffd700 1 -- 172.21.15.143:0/2225643250 <== mgr.4099 v2:172.21.15.174:6832/4037280178 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+532304 (secure 0 0 0) 0x7f9658004870 con 0x7f964806af00 2023-11-08T19:28:12.791 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:28:12.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.788+0000 7f966cfce700 1 -- 172.21.15.143:0/2225643250 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f964806af00 msgr2=0x7f964806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:12.797 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.788+0000 7f966cfce700 1 --2- 172.21.15.143:0/2225643250 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f964806af00 0x7f964806d3c0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f96681f7170 tx=0x7f9658007d70 comp rx=0 tx=0).stop 2023-11-08T19:28:12.797 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.788+0000 7f966cfce700 1 -- 172.21.15.143:0/2225643250 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f966806b320 msgr2=0x7f96680c4c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:12.797 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.788+0000 7f966cfce700 1 --2- 172.21.15.143:0/2225643250 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f966806b320 0x7f96680c4c40 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f965c009c10 tx=0x7f965c004750 comp rx=0 tx=0).stop 2023-11-08T19:28:12.797 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.788+0000 7f966cfce700 1 -- 172.21.15.143:0/2225643250 shutdown_connections 2023-11-08T19:28:12.798 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.788+0000 7f966cfce700 1 --2- 172.21.15.143:0/2225643250 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f964806af00 0x7f964806d3c0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:12.798 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.788+0000 7f966cfce700 1 --2- 172.21.15.143:0/2225643250 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f96681618f0 0x7f96680b17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:12.798 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.788+0000 7f966cfce700 1 --2- 172.21.15.143:0/2225643250 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f966806b320 0x7f96680c4c40 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:12.798 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.788+0000 7f966cfce700 1 --2- 172.21.15.143:0/2225643250 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f966815dd00 0x7f96680b1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:12.799 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.788+0000 7f966cfce700 1 -- 172.21.15.143:0/2225643250 >> 172.21.15.143:0/2225643250 conn(0x7f9668099ab0 msgr2=0x7f96680cbfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:12.799 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.788+0000 7f966cfce700 1 -- 172.21.15.143:0/2225643250 shutdown_connections 2023-11-08T19:28:12.799 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.788+0000 7f966cfce700 1 -- 172.21.15.143:0/2225643250 wait complete. 2023-11-08T19:28:12.799 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2023-11-08T19:28:12.801 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":189,"stamp":"2023-11-08T19:28:12.564133+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":386,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":226248,"kb_used_data":10056,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754748472,"statfs":{"total":773094113280,"available":772862435328,"internally_reserved":0,"allocated":10297344,"data_stored":5460532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.858410"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:05.139926+0000","last_change":"2023-11-08T19:28:03.721117+0000","last_active":"2023-11-08T19:28:05.139926+0000","last_peered":"2023-11-08T19:28:05.139926+0000","last_clean":"2023-11-08T19:28:05.139926+0000","last_became_active":"2023-11-08T19:28:03.720824+0000","last_became_peered":"2023-11-08T19:28:03.720824+0000","last_unstale":"2023-11-08T19:28:05.139926+0000","last_undegraded":"2023-11-08T19:28:05.139926+0000","last_fullsized":"2023-11-08T19:28:05.139926+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:17:58.716366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:04.700302+0000","last_change":"2023-11-08T19:28:03.715294+0000","last_active":"2023-11-08T19:28:04.700302+0000","last_peered":"2023-11-08T19:28:04.700302+0000","last_clean":"2023-11-08T19:28:04.700302+0000","last_became_active":"2023-11-08T19:28:03.715007+0000","last_became_peered":"2023-11-08T19:28:03.715007+0000","last_unstale":"2023-11-08T19:28:04.700302+0000","last_undegraded":"2023-11-08T19:28:04.700302+0000","last_fullsized":"2023-11-08T19:28:04.700302+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:39:49.732950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:05.139786+0000","last_change":"2023-11-08T19:28:03.722902+0000","last_active":"2023-11-08T19:28:05.139786+0000","last_peered":"2023-11-08T19:28:05.139786+0000","last_clean":"2023-11-08T19:28:05.139786+0000","last_became_active":"2023-11-08T19:28:03.722433+0000","last_became_peered":"2023-11-08T19:28:03.722433+0000","last_unstale":"2023-11-08T19:28:05.139786+0000","last_undegraded":"2023-11-08T19:28:05.139786+0000","last_fullsized":"2023-11-08T19:28:05.139786+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:57:47.183121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:05.211582+0000","last_change":"2023-11-08T19:28:03.715911+0000","last_active":"2023-11-08T19:28:05.211582+0000","last_peered":"2023-11-08T19:28:05.211582+0000","last_clean":"2023-11-08T19:28:05.211582+0000","last_became_active":"2023-11-08T19:28:03.715589+0000","last_became_peered":"2023-11-08T19:28:03.715589+0000","last_unstale":"2023-11-08T19:28:05.211582+0000","last_undegraded":"2023-11-08T19:28:05.211582+0000","last_fullsized":"2023-11-08T19:28:05.211582+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:40:03.163555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:04.697488+0000","last_change":"2023-11-08T19:28:03.716455+0000","last_active":"2023-11-08T19:28:04.697488+0000","last_peered":"2023-11-08T19:28:04.697488+0000","last_clean":"2023-11-08T19:28:04.697488+0000","last_became_active":"2023-11-08T19:28:03.716145+0000","last_became_peered":"2023-11-08T19:28:03.716145+0000","last_unstale":"2023-11-08T19:28:04.697488+0000","last_undegraded":"2023-11-08T19:28:04.697488+0000","last_fullsized":"2023-11-08T19:28:04.697488+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:06:26.112939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:04.698764+0000","last_change":"2023-11-08T19:28:03.718208+0000","last_active":"2023-11-08T19:28:04.698764+0000","last_peered":"2023-11-08T19:28:04.698764+0000","last_clean":"2023-11-08T19:28:04.698764+0000","last_became_active":"2023-11-08T19:28:03.716630+0000","last_became_peered":"2023-11-08T19:28:03.716630+0000","last_unstale":"2023-11-08T19:28:04.698764+0000","last_undegraded":"2023-11-08T19:28:04.698764+0000","last_fullsized":"2023-11-08T19:28:04.698764+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:21:12.684194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:04.698964+0000","last_change":"2023-11-08T19:28:03.708477+0000","last_active":"2023-11-08T19:28:04.698964+0000","last_peered":"2023-11-08T19:28:04.698964+0000","last_clean":"2023-11-08T19:28:04.698964+0000","last_became_active":"2023-11-08T19:28:03.708188+0000","last_became_peered":"2023-11-08T19:28:03.708188+0000","last_unstale":"2023-11-08T19:28:04.698964+0000","last_undegraded":"2023-11-08T19:28:04.698964+0000","last_fullsized":"2023-11-08T19:28:04.698964+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:00:53.382966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:05.209742+0000","last_change":"2023-11-08T19:28:03.715436+0000","last_active":"2023-11-08T19:28:05.209742+0000","last_peered":"2023-11-08T19:28:05.209742+0000","last_clean":"2023-11-08T19:28:05.209742+0000","last_became_active":"2023-11-08T19:28:03.714649+0000","last_became_peered":"2023-11-08T19:28:03.714649+0000","last_unstale":"2023-11-08T19:28:05.209742+0000","last_undegraded":"2023-11-08T19:28:05.209742+0000","last_fullsized":"2023-11-08T19:28:05.209742+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:49:00.891987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:04.699504+0000","last_change":"2023-11-08T19:28:03.718615+0000","last_active":"2023-11-08T19:28:04.699504+0000","last_peered":"2023-11-08T19:28:04.699504+0000","last_clean":"2023-11-08T19:28:04.699504+0000","last_became_active":"2023-11-08T19:28:03.718295+0000","last_became_peered":"2023-11-08T19:28:03.718295+0000","last_unstale":"2023-11-08T19:28:04.699504+0000","last_undegraded":"2023-11-08T19:28:04.699504+0000","last_fullsized":"2023-11-08T19:28:04.699504+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:39:51.062357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:04.698101+0000","last_change":"2023-11-08T19:28:03.710442+0000","last_active":"2023-11-08T19:28:04.698101+0000","last_peered":"2023-11-08T19:28:04.698101+0000","last_clean":"2023-11-08T19:28:04.698101+0000","last_became_active":"2023-11-08T19:28:03.710192+0000","last_became_peered":"2023-11-08T19:28:03.710192+0000","last_unstale":"2023-11-08T19:28:04.698101+0000","last_undegraded":"2023-11-08T19:28:04.698101+0000","last_fullsized":"2023-11-08T19:28:04.698101+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:10:59.668045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:05.212009+0000","last_change":"2023-11-08T19:28:03.720743+0000","last_active":"2023-11-08T19:28:05.212009+0000","last_peered":"2023-11-08T19:28:05.212009+0000","last_clean":"2023-11-08T19:28:05.212009+0000","last_became_active":"2023-11-08T19:28:03.720389+0000","last_became_peered":"2023-11-08T19:28:03.720389+0000","last_unstale":"2023-11-08T19:28:05.212009+0000","last_undegraded":"2023-11-08T19:28:05.212009+0000","last_fullsized":"2023-11-08T19:28:05.212009+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:32:43.695921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:05.211650+0000","last_change":"2023-11-08T19:28:03.718643+0000","last_active":"2023-11-08T19:28:05.211650+0000","last_peered":"2023-11-08T19:28:05.211650+0000","last_clean":"2023-11-08T19:28:05.211650+0000","last_became_active":"2023-11-08T19:28:03.717218+0000","last_became_peered":"2023-11-08T19:28:03.717218+0000","last_unstale":"2023-11-08T19:28:05.211650+0000","last_undegraded":"2023-11-08T19:28:05.211650+0000","last_fullsized":"2023-11-08T19:28:05.211650+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:50:46.499639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:05.210270+0000","last_change":"2023-11-08T19:28:03.721165+0000","last_active":"2023-11-08T19:28:05.210270+0000","last_peered":"2023-11-08T19:28:05.210270+0000","last_clean":"2023-11-08T19:28:05.210270+0000","last_became_active":"2023-11-08T19:28:03.720785+0000","last_became_peered":"2023-11-08T19:28:03.720785+0000","last_unstale":"2023-11-08T19:28:05.210270+0000","last_undegraded":"2023-11-08T19:28:05.210270+0000","last_fullsized":"2023-11-08T19:28:05.210270+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:34:22.087376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:04.700465+0000","last_change":"2023-11-08T19:28:03.718346+0000","last_active":"2023-11-08T19:28:04.700465+0000","last_peered":"2023-11-08T19:28:04.700465+0000","last_clean":"2023-11-08T19:28:04.700465+0000","last_became_active":"2023-11-08T19:28:03.718068+0000","last_became_peered":"2023-11-08T19:28:03.718068+0000","last_unstale":"2023-11-08T19:28:04.700465+0000","last_undegraded":"2023-11-08T19:28:04.700465+0000","last_fullsized":"2023-11-08T19:28:04.700465+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:36:49.083044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:04.699975+0000","last_change":"2023-11-08T19:28:03.717576+0000","last_active":"2023-11-08T19:28:04.699975+0000","last_peered":"2023-11-08T19:28:04.699975+0000","last_clean":"2023-11-08T19:28:04.699975+0000","last_became_active":"2023-11-08T19:28:03.715708+0000","last_became_peered":"2023-11-08T19:28:03.715708+0000","last_unstale":"2023-11-08T19:28:04.699975+0000","last_undegraded":"2023-11-08T19:28:04.699975+0000","last_fullsized":"2023-11-08T19:28:04.699975+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:22:54.270760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:04.699531+0000","last_change":"2023-11-08T19:28:03.703641+0000","last_active":"2023-11-08T19:28:04.699531+0000","last_peered":"2023-11-08T19:28:04.699531+0000","last_clean":"2023-11-08T19:28:04.699531+0000","last_became_active":"2023-11-08T19:28:03.703331+0000","last_became_peered":"2023-11-08T19:28:03.703331+0000","last_unstale":"2023-11-08T19:28:04.699531+0000","last_undegraded":"2023-11-08T19:28:04.699531+0000","last_fullsized":"2023-11-08T19:28:04.699531+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:52:06.126259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:05.140126+0000","last_change":"2023-11-08T19:28:03.723973+0000","last_active":"2023-11-08T19:28:05.140126+0000","last_peered":"2023-11-08T19:28:05.140126+0000","last_clean":"2023-11-08T19:28:05.140126+0000","last_became_active":"2023-11-08T19:28:03.723710+0000","last_became_peered":"2023-11-08T19:28:03.723710+0000","last_unstale":"2023-11-08T19:28:05.140126+0000","last_undegraded":"2023-11-08T19:28:05.140126+0000","last_fullsized":"2023-11-08T19:28:05.140126+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:30:31.070280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:04.700088+0000","last_change":"2023-11-08T19:28:03.715221+0000","last_active":"2023-11-08T19:28:04.700088+0000","last_peered":"2023-11-08T19:28:04.700088+0000","last_clean":"2023-11-08T19:28:04.700088+0000","last_became_active":"2023-11-08T19:28:03.714929+0000","last_became_peered":"2023-11-08T19:28:03.714929+0000","last_unstale":"2023-11-08T19:28:04.700088+0000","last_undegraded":"2023-11-08T19:28:04.700088+0000","last_fullsized":"2023-11-08T19:28:04.700088+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:00:11.333972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:05.139756+0000","last_change":"2023-11-08T19:28:03.719248+0000","last_active":"2023-11-08T19:28:05.139756+0000","last_peered":"2023-11-08T19:28:05.139756+0000","last_clean":"2023-11-08T19:28:05.139756+0000","last_became_active":"2023-11-08T19:28:03.714110+0000","last_became_peered":"2023-11-08T19:28:03.714110+0000","last_unstale":"2023-11-08T19:28:05.139756+0000","last_undegraded":"2023-11-08T19:28:05.139756+0000","last_fullsized":"2023-11-08T19:28:05.139756+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:38:00.923155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:05.211260+0000","last_change":"2023-11-08T19:28:03.714122+0000","last_active":"2023-11-08T19:28:05.211260+0000","last_peered":"2023-11-08T19:28:05.211260+0000","last_clean":"2023-11-08T19:28:05.211260+0000","last_became_active":"2023-11-08T19:28:03.713845+0000","last_became_peered":"2023-11-08T19:28:03.713845+0000","last_unstale":"2023-11-08T19:28:05.211260+0000","last_undegraded":"2023-11-08T19:28:05.211260+0000","last_fullsized":"2023-11-08T19:28:05.211260+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:10:59.668045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:05.210270+0000","last_change":"2023-11-08T19:28:03.721893+0000","last_active":"2023-11-08T19:28:05.210270+0000","last_peered":"2023-11-08T19:28:05.210270+0000","last_clean":"2023-11-08T19:28:05.210270+0000","last_became_active":"2023-11-08T19:28:03.721639+0000","last_became_peered":"2023-11-08T19:28:03.721639+0000","last_unstale":"2023-11-08T19:28:05.210270+0000","last_undegraded":"2023-11-08T19:28:05.210270+0000","last_fullsized":"2023-11-08T19:28:05.210270+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:21:34.606346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:04.697611+0000","last_change":"2023-11-08T19:28:03.722634+0000","last_active":"2023-11-08T19:28:04.697611+0000","last_peered":"2023-11-08T19:28:04.697611+0000","last_clean":"2023-11-08T19:28:04.697611+0000","last_became_active":"2023-11-08T19:28:03.722374+0000","last_became_peered":"2023-11-08T19:28:03.722374+0000","last_unstale":"2023-11-08T19:28:04.697611+0000","last_undegraded":"2023-11-08T19:28:04.697611+0000","last_fullsized":"2023-11-08T19:28:04.697611+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:39:49.732950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:05.139446+0000","last_change":"2023-11-08T19:28:03.718751+0000","last_active":"2023-11-08T19:28:05.139446+0000","last_peered":"2023-11-08T19:28:05.139446+0000","last_clean":"2023-11-08T19:28:05.139446+0000","last_became_active":"2023-11-08T19:28:03.718450+0000","last_became_peered":"2023-11-08T19:28:03.718450+0000","last_unstale":"2023-11-08T19:28:05.139446+0000","last_undegraded":"2023-11-08T19:28:05.139446+0000","last_fullsized":"2023-11-08T19:28:05.139446+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:29:14.528608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:04.697375+0000","last_change":"2023-11-08T19:28:03.712465+0000","last_active":"2023-11-08T19:28:04.697375+0000","last_peered":"2023-11-08T19:28:04.697375+0000","last_clean":"2023-11-08T19:28:04.697375+0000","last_became_active":"2023-11-08T19:28:03.712185+0000","last_became_peered":"2023-11-08T19:28:03.712185+0000","last_unstale":"2023-11-08T19:28:04.697375+0000","last_undegraded":"2023-11-08T19:28:04.697375+0000","last_fullsized":"2023-11-08T19:28:04.697375+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:32:43.695921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:05.212347+0000","last_change":"2023-11-08T19:28:03.725958+0000","last_active":"2023-11-08T19:28:05.212347+0000","last_peered":"2023-11-08T19:28:05.212347+0000","last_clean":"2023-11-08T19:28:05.212347+0000","last_became_active":"2023-11-08T19:28:03.725715+0000","last_became_peered":"2023-11-08T19:28:03.725715+0000","last_unstale":"2023-11-08T19:28:05.212347+0000","last_undegraded":"2023-11-08T19:28:05.212347+0000","last_fullsized":"2023-11-08T19:28:05.212347+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:03:34.978474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:05.211859+0000","last_change":"2023-11-08T19:28:03.716658+0000","last_active":"2023-11-08T19:28:05.211859+0000","last_peered":"2023-11-08T19:28:05.211859+0000","last_clean":"2023-11-08T19:28:05.211859+0000","last_became_active":"2023-11-08T19:28:03.716306+0000","last_became_peered":"2023-11-08T19:28:03.716306+0000","last_unstale":"2023-11-08T19:28:05.211859+0000","last_undegraded":"2023-11-08T19:28:05.211859+0000","last_fullsized":"2023-11-08T19:28:05.211859+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:43:50.055092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:05.211785+0000","last_change":"2023-11-08T19:28:03.721232+0000","last_active":"2023-11-08T19:28:05.211785+0000","last_peered":"2023-11-08T19:28:05.211785+0000","last_clean":"2023-11-08T19:28:05.211785+0000","last_became_active":"2023-11-08T19:28:03.720946+0000","last_became_peered":"2023-11-08T19:28:03.720946+0000","last_unstale":"2023-11-08T19:28:05.211785+0000","last_undegraded":"2023-11-08T19:28:05.211785+0000","last_fullsized":"2023-11-08T19:28:05.211785+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:35:11.551335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:04.699322+0000","last_change":"2023-11-08T19:28:03.721162+0000","last_active":"2023-11-08T19:28:04.699322+0000","last_peered":"2023-11-08T19:28:04.699322+0000","last_clean":"2023-11-08T19:28:04.699322+0000","last_became_active":"2023-11-08T19:28:03.720956+0000","last_became_peered":"2023-11-08T19:28:03.720956+0000","last_unstale":"2023-11-08T19:28:04.699322+0000","last_undegraded":"2023-11-08T19:28:04.699322+0000","last_fullsized":"2023-11-08T19:28:04.699322+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:30:24.544349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:05.210293+0000","last_change":"2023-11-08T19:28:03.714955+0000","last_active":"2023-11-08T19:28:05.210293+0000","last_peered":"2023-11-08T19:28:05.210293+0000","last_clean":"2023-11-08T19:28:05.210293+0000","last_became_active":"2023-11-08T19:28:03.714381+0000","last_became_peered":"2023-11-08T19:28:03.714381+0000","last_unstale":"2023-11-08T19:28:05.210293+0000","last_undegraded":"2023-11-08T19:28:05.210293+0000","last_fullsized":"2023-11-08T19:28:05.210293+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:18:56.492679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:05.210899+0000","last_change":"2023-11-08T19:28:03.723214+0000","last_active":"2023-11-08T19:28:05.210899+0000","last_peered":"2023-11-08T19:28:05.210899+0000","last_clean":"2023-11-08T19:28:05.210899+0000","last_became_active":"2023-11-08T19:28:03.722857+0000","last_became_peered":"2023-11-08T19:28:03.722857+0000","last_unstale":"2023-11-08T19:28:05.210899+0000","last_undegraded":"2023-11-08T19:28:05.210899+0000","last_fullsized":"2023-11-08T19:28:05.210899+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:03:01.222287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:04.698561+0000","last_change":"2023-11-08T19:28:03.707881+0000","last_active":"2023-11-08T19:28:04.698561+0000","last_peered":"2023-11-08T19:28:04.698561+0000","last_clean":"2023-11-08T19:28:04.698561+0000","last_became_active":"2023-11-08T19:28:03.707569+0000","last_became_peered":"2023-11-08T19:28:03.707569+0000","last_unstale":"2023-11-08T19:28:04.698561+0000","last_undegraded":"2023-11-08T19:28:04.698561+0000","last_fullsized":"2023-11-08T19:28:04.698561+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:11:45.309372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:05.210061+0000","last_change":"2023-11-08T19:28:03.717467+0000","last_active":"2023-11-08T19:28:05.210061+0000","last_peered":"2023-11-08T19:28:05.210061+0000","last_clean":"2023-11-08T19:28:05.210061+0000","last_became_active":"2023-11-08T19:28:03.717209+0000","last_became_peered":"2023-11-08T19:28:03.717209+0000","last_unstale":"2023-11-08T19:28:05.210061+0000","last_undegraded":"2023-11-08T19:28:05.210061+0000","last_fullsized":"2023-11-08T19:28:05.210061+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:30:45.589100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:05.211077+0000","last_change":"2023-11-08T19:28:03.713590+0000","last_active":"2023-11-08T19:28:05.211077+0000","last_peered":"2023-11-08T19:28:05.211077+0000","last_clean":"2023-11-08T19:28:05.211077+0000","last_became_active":"2023-11-08T19:28:03.713312+0000","last_became_peered":"2023-11-08T19:28:03.713312+0000","last_unstale":"2023-11-08T19:28:05.211077+0000","last_undegraded":"2023-11-08T19:28:05.211077+0000","last_fullsized":"2023-11-08T19:28:05.211077+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:30:24.544349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"91'2","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.720244+0000","last_change":"2023-11-08T19:28:02.720536+0000","last_active":"2023-11-08T19:28:03.720244+0000","last_peered":"2023-11-08T19:28:03.720244+0000","last_clean":"2023-11-08T19:28:03.720244+0000","last_became_active":"2023-11-08T19:28:02.720236+0000","last_became_peered":"2023-11-08T19:28:02.720236+0000","last_unstale":"2023-11-08T19:28:03.720244+0000","last_undegraded":"2023-11-08T19:28:03.720244+0000","last_fullsized":"2023-11-08T19:28:03.720244+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:29:43.577109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.699420+0000","last_change":"2023-11-08T19:27:08.190826+0000","last_active":"2023-11-08T19:28:03.699420+0000","last_peered":"2023-11-08T19:28:03.699420+0000","last_clean":"2023-11-08T19:28:03.699420+0000","last_became_active":"2023-11-08T19:27:08.190440+0000","last_became_peered":"2023-11-08T19:27:08.190440+0000","last_unstale":"2023-11-08T19:28:03.699420+0000","last_undegraded":"2023-11-08T19:28:03.699420+0000","last_fullsized":"2023-11-08T19:28:03.699420+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:42:48.376462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":238,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.699607+0000","last_change":"2023-11-08T19:24:45.888122+0000","last_active":"2023-11-08T19:28:03.699607+0000","last_peered":"2023-11-08T19:28:03.699607+0000","last_clean":"2023-11-08T19:28:03.699607+0000","last_became_active":"2023-11-08T19:24:45.887048+0000","last_became_peered":"2023-11-08T19:24:45.887048+0000","last_unstale":"2023-11-08T19:28:03.699607+0000","last_undegraded":"2023-11-08T19:28:03.699607+0000","last_fullsized":"2023-11-08T19:28:03.699607+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:44.575133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:44.575133+0000","last_clean_scrub_stamp":"2023-11-08T19:24:44.575133+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:19:20.202062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":62,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.723099+0000","last_change":"2023-11-08T19:26:06.542044+0000","last_active":"2023-11-08T19:28:03.723099+0000","last_peered":"2023-11-08T19:28:03.723099+0000","last_clean":"2023-11-08T19:28:03.723099+0000","last_became_active":"2023-11-08T19:26:04.603056+0000","last_became_peered":"2023-11-08T19:26:04.603056+0000","last_unstale":"2023-11-08T19:28:03.723099+0000","last_undegraded":"2023-11-08T19:28:03.723099+0000","last_fullsized":"2023-11-08T19:28:03.723099+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:41:50.771457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000517414,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.708796+0000","last_change":"2023-11-08T19:27:09.496842+0000","last_active":"2023-11-08T19:28:03.708796+0000","last_peered":"2023-11-08T19:28:03.708796+0000","last_clean":"2023-11-08T19:28:03.708796+0000","last_became_active":"2023-11-08T19:27:07.185776+0000","last_became_peered":"2023-11-08T19:27:07.185776+0000","last_unstale":"2023-11-08T19:28:03.708796+0000","last_undegraded":"2023-11-08T19:28:03.708796+0000","last_fullsized":"2023-11-08T19:28:03.708796+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:21:13.923970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031747200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:05.139994+0000","last_change":"2023-11-08T19:28:03.722042+0000","last_active":"2023-11-08T19:28:05.139994+0000","last_peered":"2023-11-08T19:28:05.139994+0000","last_clean":"2023-11-08T19:28:05.139994+0000","last_became_active":"2023-11-08T19:28:03.720189+0000","last_became_peered":"2023-11-08T19:28:03.720189+0000","last_unstale":"2023-11-08T19:28:05.139994+0000","last_undegraded":"2023-11-08T19:28:05.139994+0000","last_fullsized":"2023-11-08T19:28:05.139994+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:52:29.184631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"91'2","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.725401+0000","last_change":"2023-11-08T19:28:02.720512+0000","last_active":"2023-11-08T19:28:03.725401+0000","last_peered":"2023-11-08T19:28:03.725401+0000","last_clean":"2023-11-08T19:28:03.725401+0000","last_became_active":"2023-11-08T19:28:02.720202+0000","last_became_peered":"2023-11-08T19:28:02.720202+0000","last_unstale":"2023-11-08T19:28:03.725401+0000","last_undegraded":"2023-11-08T19:28:03.725401+0000","last_fullsized":"2023-11-08T19:28:03.725401+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:49:15.604078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.700705+0000","last_change":"2023-11-08T19:27:08.188241+0000","last_active":"2023-11-08T19:28:03.700705+0000","last_peered":"2023-11-08T19:28:03.700705+0000","last_clean":"2023-11-08T19:28:03.700705+0000","last_became_active":"2023-11-08T19:27:08.187863+0000","last_became_peered":"2023-11-08T19:27:08.187863+0000","last_unstale":"2023-11-08T19:28:03.700705+0000","last_undegraded":"2023-11-08T19:28:03.700705+0000","last_fullsized":"2023-11-08T19:28:03.700705+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:45:22.084015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":61,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:05.211109+0000","last_change":"2023-11-08T19:26:06.265419+0000","last_active":"2023-11-08T19:28:05.211109+0000","last_peered":"2023-11-08T19:28:05.211109+0000","last_clean":"2023-11-08T19:28:05.211109+0000","last_became_active":"2023-11-08T19:26:04.605333+0000","last_became_peered":"2023-11-08T19:26:04.605333+0000","last_unstale":"2023-11-08T19:28:05.211109+0000","last_undegraded":"2023-11-08T19:28:05.211109+0000","last_fullsized":"2023-11-08T19:28:05.211109+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:01:56.991346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064606400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.700401+0000","last_change":"2023-11-08T19:27:09.182113+0000","last_active":"2023-11-08T19:28:03.700401+0000","last_peered":"2023-11-08T19:28:03.700401+0000","last_clean":"2023-11-08T19:28:03.700401+0000","last_became_active":"2023-11-08T19:27:07.189939+0000","last_became_peered":"2023-11-08T19:27:07.189939+0000","last_unstale":"2023-11-08T19:28:03.700401+0000","last_undegraded":"2023-11-08T19:28:03.700401+0000","last_fullsized":"2023-11-08T19:28:03.700401+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:41:08.822366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036507500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:05.139937+0000","last_change":"2023-11-08T19:28:03.718720+0000","last_active":"2023-11-08T19:28:05.139937+0000","last_peered":"2023-11-08T19:28:05.139937+0000","last_clean":"2023-11-08T19:28:05.139937+0000","last_became_active":"2023-11-08T19:28:03.714655+0000","last_became_peered":"2023-11-08T19:28:03.714655+0000","last_unstale":"2023-11-08T19:28:05.139937+0000","last_undegraded":"2023-11-08T19:28:05.139937+0000","last_fullsized":"2023-11-08T19:28:05.139937+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:37:40.998880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.696736+0000","last_change":"2023-11-08T19:28:02.720708+0000","last_active":"2023-11-08T19:28:03.696736+0000","last_peered":"2023-11-08T19:28:03.696736+0000","last_clean":"2023-11-08T19:28:03.696736+0000","last_became_active":"2023-11-08T19:28:02.720263+0000","last_became_peered":"2023-11-08T19:28:02.720263+0000","last_unstale":"2023-11-08T19:28:03.696736+0000","last_undegraded":"2023-11-08T19:28:03.696736+0000","last_fullsized":"2023-11-08T19:28:03.696736+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:36:24.590277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.696644+0000","last_change":"2023-11-08T19:27:08.187127+0000","last_active":"2023-11-08T19:28:03.696644+0000","last_peered":"2023-11-08T19:28:03.696644+0000","last_clean":"2023-11-08T19:28:03.696644+0000","last_became_active":"2023-11-08T19:27:08.182648+0000","last_became_peered":"2023-11-08T19:27:08.182648+0000","last_unstale":"2023-11-08T19:28:03.696644+0000","last_undegraded":"2023-11-08T19:28:03.696644+0000","last_fullsized":"2023-11-08T19:28:03.696644+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:50:12.101878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":73,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:04.700585+0000","last_change":"2023-11-08T19:26:05.609570+0000","last_active":"2023-11-08T19:28:04.700585+0000","last_peered":"2023-11-08T19:28:04.700585+0000","last_clean":"2023-11-08T19:28:04.700585+0000","last_became_active":"2023-11-08T19:26:04.608379+0000","last_became_peered":"2023-11-08T19:26:04.608379+0000","last_unstale":"2023-11-08T19:28:04.700585+0000","last_undegraded":"2023-11-08T19:28:04.700585+0000","last_fullsized":"2023-11-08T19:28:04.700585+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:40:08.986649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000529657,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.705202+0000","last_change":"2023-11-08T19:27:09.211991+0000","last_active":"2023-11-08T19:28:03.705202+0000","last_peered":"2023-11-08T19:28:03.705202+0000","last_clean":"2023-11-08T19:28:03.705202+0000","last_became_active":"2023-11-08T19:27:07.195231+0000","last_became_peered":"2023-11-08T19:27:07.195231+0000","last_unstale":"2023-11-08T19:28:03.705202+0000","last_undegraded":"2023-11-08T19:28:03.705202+0000","last_fullsized":"2023-11-08T19:28:03.705202+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:25:14.487524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000441221,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:05.211004+0000","last_change":"2023-11-08T19:28:03.718236+0000","last_active":"2023-11-08T19:28:05.211004+0000","last_peered":"2023-11-08T19:28:05.211004+0000","last_clean":"2023-11-08T19:28:05.211004+0000","last_became_active":"2023-11-08T19:28:03.717974+0000","last_became_peered":"2023-11-08T19:28:03.717974+0000","last_unstale":"2023-11-08T19:28:05.211004+0000","last_undegraded":"2023-11-08T19:28:05.211004+0000","last_fullsized":"2023-11-08T19:28:05.211004+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:50:42.727925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:04.596751+0000","last_change":"2023-11-08T19:28:02.707543+0000","last_active":"2023-11-08T19:28:04.596751+0000","last_peered":"2023-11-08T19:28:04.596751+0000","last_clean":"2023-11-08T19:28:04.596751+0000","last_became_active":"2023-11-08T19:28:02.707034+0000","last_became_peered":"2023-11-08T19:28:02.707034+0000","last_unstale":"2023-11-08T19:28:04.596751+0000","last_undegraded":"2023-11-08T19:28:04.596751+0000","last_fullsized":"2023-11-08T19:28:04.596751+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:06:46.912027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.697894+0000","last_change":"2023-11-08T19:27:08.195364+0000","last_active":"2023-11-08T19:28:03.697894+0000","last_peered":"2023-11-08T19:28:03.697894+0000","last_clean":"2023-11-08T19:28:03.697894+0000","last_became_active":"2023-11-08T19:27:08.195074+0000","last_became_peered":"2023-11-08T19:27:08.195074+0000","last_unstale":"2023-11-08T19:28:03.697894+0000","last_undegraded":"2023-11-08T19:28:03.697894+0000","last_fullsized":"2023-11-08T19:28:03.697894+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:02:57.552873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":61,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.703532+0000","last_change":"2023-11-08T19:26:06.394300+0000","last_active":"2023-11-08T19:28:03.703532+0000","last_peered":"2023-11-08T19:28:03.703532+0000","last_clean":"2023-11-08T19:28:03.703532+0000","last_became_active":"2023-11-08T19:26:04.601685+0000","last_became_peered":"2023-11-08T19:26:04.601685+0000","last_unstale":"2023-11-08T19:28:03.703532+0000","last_undegraded":"2023-11-08T19:28:03.703532+0000","last_fullsized":"2023-11-08T19:28:03.703532+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:51:06.626474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061044099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.697961+0000","last_change":"2023-11-08T19:27:09.178466+0000","last_active":"2023-11-08T19:28:03.697961+0000","last_peered":"2023-11-08T19:28:03.697961+0000","last_clean":"2023-11-08T19:28:03.697961+0000","last_became_active":"2023-11-08T19:27:07.189386+0000","last_became_peered":"2023-11-08T19:27:07.189386+0000","last_unstale":"2023-11-08T19:28:03.697961+0000","last_undegraded":"2023-11-08T19:28:03.697961+0000","last_fullsized":"2023-11-08T19:28:03.697961+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:48:51.139820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049390100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:05.211305+0000","last_change":"2023-11-08T19:28:03.708772+0000","last_active":"2023-11-08T19:28:05.211305+0000","last_peered":"2023-11-08T19:28:05.211305+0000","last_clean":"2023-11-08T19:28:05.211305+0000","last_became_active":"2023-11-08T19:28:03.708262+0000","last_became_peered":"2023-11-08T19:28:03.708262+0000","last_unstale":"2023-11-08T19:28:05.211305+0000","last_undegraded":"2023-11-08T19:28:05.211305+0000","last_fullsized":"2023-11-08T19:28:05.211305+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:42:05.348583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.696892+0000","last_change":"2023-11-08T19:28:02.723021+0000","last_active":"2023-11-08T19:28:03.696892+0000","last_peered":"2023-11-08T19:28:03.696892+0000","last_clean":"2023-11-08T19:28:03.696892+0000","last_became_active":"2023-11-08T19:28:02.722748+0000","last_became_peered":"2023-11-08T19:28:02.722748+0000","last_unstale":"2023-11-08T19:28:03.696892+0000","last_undegraded":"2023-11-08T19:28:03.696892+0000","last_fullsized":"2023-11-08T19:28:03.696892+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:27:27.293994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.701156+0000","last_change":"2023-11-08T19:27:08.189452+0000","last_active":"2023-11-08T19:28:03.701156+0000","last_peered":"2023-11-08T19:28:03.701156+0000","last_clean":"2023-11-08T19:28:03.701156+0000","last_became_active":"2023-11-08T19:27:08.189088+0000","last_became_peered":"2023-11-08T19:27:08.189088+0000","last_unstale":"2023-11-08T19:28:03.701156+0000","last_undegraded":"2023-11-08T19:28:03.701156+0000","last_fullsized":"2023-11-08T19:28:03.701156+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:25:18.963220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":61,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.702773+0000","last_change":"2023-11-08T19:26:06.296634+0000","last_active":"2023-11-08T19:28:03.702773+0000","last_peered":"2023-11-08T19:28:03.702773+0000","last_clean":"2023-11-08T19:28:03.702773+0000","last_became_active":"2023-11-08T19:26:04.603088+0000","last_became_peered":"2023-11-08T19:26:04.603088+0000","last_unstale":"2023-11-08T19:28:03.702773+0000","last_undegraded":"2023-11-08T19:28:03.702773+0000","last_fullsized":"2023-11-08T19:28:03.702773+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:01:56.991346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038180899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.702063+0000","last_change":"2023-11-08T19:27:09.431220+0000","last_active":"2023-11-08T19:28:03.702063+0000","last_peered":"2023-11-08T19:28:03.702063+0000","last_clean":"2023-11-08T19:28:03.702063+0000","last_became_active":"2023-11-08T19:27:07.191589+0000","last_became_peered":"2023-11-08T19:27:07.191589+0000","last_unstale":"2023-11-08T19:28:03.702063+0000","last_undegraded":"2023-11-08T19:28:03.702063+0000","last_fullsized":"2023-11-08T19:28:03.702063+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:18:19.964487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000729527,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:05.209781+0000","last_change":"2023-11-08T19:28:03.715237+0000","last_active":"2023-11-08T19:28:05.209781+0000","last_peered":"2023-11-08T19:28:05.209781+0000","last_clean":"2023-11-08T19:28:05.209781+0000","last_became_active":"2023-11-08T19:28:03.714862+0000","last_became_peered":"2023-11-08T19:28:03.714862+0000","last_unstale":"2023-11-08T19:28:05.209781+0000","last_undegraded":"2023-11-08T19:28:05.209781+0000","last_fullsized":"2023-11-08T19:28:05.209781+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:52:36.476493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"91'1","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:04.597159+0000","last_change":"2023-11-08T19:28:02.712293+0000","last_active":"2023-11-08T19:28:04.597159+0000","last_peered":"2023-11-08T19:28:04.597159+0000","last_clean":"2023-11-08T19:28:04.597159+0000","last_became_active":"2023-11-08T19:28:02.712002+0000","last_became_peered":"2023-11-08T19:28:02.712002+0000","last_unstale":"2023-11-08T19:28:04.597159+0000","last_undegraded":"2023-11-08T19:28:04.597159+0000","last_fullsized":"2023-11-08T19:28:04.597159+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:24:47.603187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.704040+0000","last_change":"2023-11-08T19:27:08.193337+0000","last_active":"2023-11-08T19:28:03.704040+0000","last_peered":"2023-11-08T19:28:03.704040+0000","last_clean":"2023-11-08T19:28:03.704040+0000","last_became_active":"2023-11-08T19:27:08.192997+0000","last_became_peered":"2023-11-08T19:27:08.192997+0000","last_unstale":"2023-11-08T19:28:03.704040+0000","last_undegraded":"2023-11-08T19:28:03.704040+0000","last_fullsized":"2023-11-08T19:28:03.704040+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:45:22.084015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":66,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.705539+0000","last_change":"2023-11-08T19:26:06.424511+0000","last_active":"2023-11-08T19:28:03.705539+0000","last_peered":"2023-11-08T19:28:03.705539+0000","last_clean":"2023-11-08T19:28:03.705539+0000","last_became_active":"2023-11-08T19:26:04.598973+0000","last_became_peered":"2023-11-08T19:26:04.598973+0000","last_unstale":"2023-11-08T19:28:03.705539+0000","last_undegraded":"2023-11-08T19:28:03.705539+0000","last_fullsized":"2023-11-08T19:28:03.705539+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:41:53.497456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031755900000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.701231+0000","last_change":"2023-11-08T19:27:09.497024+0000","last_active":"2023-11-08T19:28:03.701231+0000","last_peered":"2023-11-08T19:28:03.701231+0000","last_clean":"2023-11-08T19:28:03.701231+0000","last_became_active":"2023-11-08T19:27:07.190132+0000","last_became_peered":"2023-11-08T19:27:07.190132+0000","last_unstale":"2023-11-08T19:28:03.701231+0000","last_undegraded":"2023-11-08T19:28:03.701231+0000","last_fullsized":"2023-11-08T19:28:03.701231+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:36:34.881217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054126000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:04.700352+0000","last_change":"2023-11-08T19:28:03.713184+0000","last_active":"2023-11-08T19:28:04.700352+0000","last_peered":"2023-11-08T19:28:04.700352+0000","last_clean":"2023-11-08T19:28:04.700352+0000","last_became_active":"2023-11-08T19:28:03.712864+0000","last_became_peered":"2023-11-08T19:28:03.712864+0000","last_unstale":"2023-11-08T19:28:04.700352+0000","last_undegraded":"2023-11-08T19:28:04.700352+0000","last_fullsized":"2023-11-08T19:28:04.700352+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:33:29.952280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"91'1","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:04.596991+0000","last_change":"2023-11-08T19:28:02.713126+0000","last_active":"2023-11-08T19:28:04.596991+0000","last_peered":"2023-11-08T19:28:04.596991+0000","last_clean":"2023-11-08T19:28:04.596991+0000","last_became_active":"2023-11-08T19:28:02.712836+0000","last_became_peered":"2023-11-08T19:28:02.712836+0000","last_unstale":"2023-11-08T19:28:04.596991+0000","last_undegraded":"2023-11-08T19:28:04.596991+0000","last_fullsized":"2023-11-08T19:28:04.596991+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:29:31.276279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.704346+0000","last_change":"2023-11-08T19:27:08.194196+0000","last_active":"2023-11-08T19:28:03.704346+0000","last_peered":"2023-11-08T19:28:03.704346+0000","last_clean":"2023-11-08T19:28:03.704346+0000","last_became_active":"2023-11-08T19:27:08.193912+0000","last_became_peered":"2023-11-08T19:27:08.193912+0000","last_unstale":"2023-11-08T19:28:03.704346+0000","last_undegraded":"2023-11-08T19:28:03.704346+0000","last_fullsized":"2023-11-08T19:28:03.704346+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:25:18.963220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":61,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.702299+0000","last_change":"2023-11-08T19:26:06.296937+0000","last_active":"2023-11-08T19:28:03.702299+0000","last_peered":"2023-11-08T19:28:03.702299+0000","last_clean":"2023-11-08T19:28:03.702299+0000","last_became_active":"2023-11-08T19:26:04.599999+0000","last_became_peered":"2023-11-08T19:26:04.599999+0000","last_unstale":"2023-11-08T19:28:03.702299+0000","last_undegraded":"2023-11-08T19:28:03.702299+0000","last_fullsized":"2023-11-08T19:28:03.702299+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:50:43.827570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025605700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.705595+0000","last_change":"2023-11-08T19:27:09.497040+0000","last_active":"2023-11-08T19:28:03.705595+0000","last_peered":"2023-11-08T19:28:03.705595+0000","last_clean":"2023-11-08T19:28:03.705595+0000","last_became_active":"2023-11-08T19:27:07.189135+0000","last_became_peered":"2023-11-08T19:27:07.189135+0000","last_unstale":"2023-11-08T19:28:03.705595+0000","last_undegraded":"2023-11-08T19:28:03.705595+0000","last_fullsized":"2023-11-08T19:28:03.705595+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:12:29.400972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051794299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:04.697623+0000","last_change":"2023-11-08T19:28:03.713320+0000","last_active":"2023-11-08T19:28:04.697623+0000","last_peered":"2023-11-08T19:28:04.697623+0000","last_clean":"2023-11-08T19:28:04.697623+0000","last_became_active":"2023-11-08T19:28:03.713041+0000","last_became_peered":"2023-11-08T19:28:03.713041+0000","last_unstale":"2023-11-08T19:28:04.697623+0000","last_undegraded":"2023-11-08T19:28:04.697623+0000","last_fullsized":"2023-11-08T19:28:04.697623+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:33:29.952280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.722112+0000","last_change":"2023-11-08T19:28:02.705153+0000","last_active":"2023-11-08T19:28:03.722112+0000","last_peered":"2023-11-08T19:28:03.722112+0000","last_clean":"2023-11-08T19:28:03.722112+0000","last_became_active":"2023-11-08T19:28:02.704854+0000","last_became_peered":"2023-11-08T19:28:02.704854+0000","last_unstale":"2023-11-08T19:28:03.722112+0000","last_undegraded":"2023-11-08T19:28:03.722112+0000","last_fullsized":"2023-11-08T19:28:03.722112+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:47:48.130157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.705223+0000","last_change":"2023-11-08T19:27:08.187053+0000","last_active":"2023-11-08T19:28:03.705223+0000","last_peered":"2023-11-08T19:28:03.705223+0000","last_clean":"2023-11-08T19:28:03.705223+0000","last_became_active":"2023-11-08T19:27:08.186643+0000","last_became_peered":"2023-11-08T19:27:08.186643+0000","last_unstale":"2023-11-08T19:28:03.705223+0000","last_undegraded":"2023-11-08T19:28:03.705223+0000","last_fullsized":"2023-11-08T19:28:03.705223+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:35:30.073885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":73,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:04.701011+0000","last_change":"2023-11-08T19:26:05.609824+0000","last_active":"2023-11-08T19:28:04.701011+0000","last_peered":"2023-11-08T19:28:04.701011+0000","last_clean":"2023-11-08T19:28:04.701011+0000","last_became_active":"2023-11-08T19:26:04.603320+0000","last_became_peered":"2023-11-08T19:26:04.603320+0000","last_unstale":"2023-11-08T19:28:04.701011+0000","last_undegraded":"2023-11-08T19:28:04.701011+0000","last_fullsized":"2023-11-08T19:28:04.701011+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:05:45.976615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063225,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.700840+0000","last_change":"2023-11-08T19:27:09.496655+0000","last_active":"2023-11-08T19:28:03.700840+0000","last_peered":"2023-11-08T19:28:03.700840+0000","last_clean":"2023-11-08T19:28:03.700840+0000","last_became_active":"2023-11-08T19:27:07.189895+0000","last_became_peered":"2023-11-08T19:27:07.189895+0000","last_unstale":"2023-11-08T19:28:03.700840+0000","last_undegraded":"2023-11-08T19:28:03.700840+0000","last_fullsized":"2023-11-08T19:28:03.700840+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:41:08.822366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045654199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:05.210327+0000","last_change":"2023-11-08T19:28:03.708585+0000","last_active":"2023-11-08T19:28:05.210327+0000","last_peered":"2023-11-08T19:28:05.210327+0000","last_clean":"2023-11-08T19:28:05.210327+0000","last_became_active":"2023-11-08T19:28:03.708334+0000","last_became_peered":"2023-11-08T19:28:03.708334+0000","last_unstale":"2023-11-08T19:28:05.210327+0000","last_undegraded":"2023-11-08T19:28:05.210327+0000","last_fullsized":"2023-11-08T19:28:05.210327+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:28:53.914341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.705083+0000","last_change":"2023-11-08T19:28:02.709891+0000","last_active":"2023-11-08T19:28:03.705083+0000","last_peered":"2023-11-08T19:28:03.705083+0000","last_clean":"2023-11-08T19:28:03.705083+0000","last_became_active":"2023-11-08T19:28:02.709612+0000","last_became_peered":"2023-11-08T19:28:02.709612+0000","last_unstale":"2023-11-08T19:28:03.705083+0000","last_undegraded":"2023-11-08T19:28:03.705083+0000","last_fullsized":"2023-11-08T19:28:03.705083+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:32:42.691100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":62,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.719733+0000","last_change":"2023-11-08T19:26:06.541360+0000","last_active":"2023-11-08T19:28:03.719733+0000","last_peered":"2023-11-08T19:28:03.719733+0000","last_clean":"2023-11-08T19:28:03.719733+0000","last_became_active":"2023-11-08T19:26:04.603725+0000","last_became_peered":"2023-11-08T19:26:04.603725+0000","last_unstale":"2023-11-08T19:28:03.719733+0000","last_undegraded":"2023-11-08T19:28:03.719733+0000","last_fullsized":"2023-11-08T19:28:03.719733+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:28:48.576742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053365400000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.723315+0000","last_change":"2023-11-08T19:27:09.283599+0000","last_active":"2023-11-08T19:28:03.723315+0000","last_peered":"2023-11-08T19:28:03.723315+0000","last_clean":"2023-11-08T19:28:03.723315+0000","last_became_active":"2023-11-08T19:27:07.189595+0000","last_became_peered":"2023-11-08T19:27:07.189595+0000","last_unstale":"2023-11-08T19:28:03.723315+0000","last_undegraded":"2023-11-08T19:28:03.723315+0000","last_fullsized":"2023-11-08T19:28:03.723315+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:48:20.082682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000323479,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.704456+0000","last_change":"2023-11-08T19:27:08.185395+0000","last_active":"2023-11-08T19:28:03.704456+0000","last_peered":"2023-11-08T19:28:03.704456+0000","last_clean":"2023-11-08T19:28:03.704456+0000","last_became_active":"2023-11-08T19:27:08.184765+0000","last_became_peered":"2023-11-08T19:27:08.184765+0000","last_unstale":"2023-11-08T19:28:03.704456+0000","last_undegraded":"2023-11-08T19:28:03.704456+0000","last_fullsized":"2023-11-08T19:28:03.704456+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:08:51.864418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:05.210683+0000","last_change":"2023-11-08T19:28:03.708515+0000","last_active":"2023-11-08T19:28:05.210683+0000","last_peered":"2023-11-08T19:28:05.210683+0000","last_clean":"2023-11-08T19:28:05.210683+0000","last_became_active":"2023-11-08T19:28:03.707926+0000","last_became_peered":"2023-11-08T19:28:03.707926+0000","last_unstale":"2023-11-08T19:28:05.210683+0000","last_undegraded":"2023-11-08T19:28:05.210683+0000","last_fullsized":"2023-11-08T19:28:05.210683+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:43:43.897674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.696607+0000","last_change":"2023-11-08T19:28:02.720596+0000","last_active":"2023-11-08T19:28:03.696607+0000","last_peered":"2023-11-08T19:28:03.696607+0000","last_clean":"2023-11-08T19:28:03.696607+0000","last_became_active":"2023-11-08T19:28:02.720144+0000","last_became_peered":"2023-11-08T19:28:02.720144+0000","last_unstale":"2023-11-08T19:28:03.696607+0000","last_undegraded":"2023-11-08T19:28:03.696607+0000","last_fullsized":"2023-11-08T19:28:03.696607+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:19:15.485882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":66,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.701393+0000","last_change":"2023-11-08T19:26:06.424236+0000","last_active":"2023-11-08T19:28:03.701393+0000","last_peered":"2023-11-08T19:28:03.701393+0000","last_clean":"2023-11-08T19:28:03.701393+0000","last_became_active":"2023-11-08T19:26:04.600115+0000","last_became_peered":"2023-11-08T19:26:04.600115+0000","last_unstale":"2023-11-08T19:28:03.701393+0000","last_undegraded":"2023-11-08T19:28:03.701393+0000","last_fullsized":"2023-11-08T19:28:03.701393+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:18:01.876605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065803900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.704181+0000","last_change":"2023-11-08T19:27:09.331899+0000","last_active":"2023-11-08T19:28:03.704181+0000","last_peered":"2023-11-08T19:28:03.704181+0000","last_clean":"2023-11-08T19:28:03.704181+0000","last_became_active":"2023-11-08T19:27:07.191460+0000","last_became_peered":"2023-11-08T19:27:07.191460+0000","last_unstale":"2023-11-08T19:28:03.704181+0000","last_undegraded":"2023-11-08T19:28:03.704181+0000","last_fullsized":"2023-11-08T19:28:03.704181+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:51:29.247104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044907500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.705567+0000","last_change":"2023-11-08T19:27:08.191802+0000","last_active":"2023-11-08T19:28:03.705567+0000","last_peered":"2023-11-08T19:28:03.705567+0000","last_clean":"2023-11-08T19:28:03.705567+0000","last_became_active":"2023-11-08T19:27:08.191520+0000","last_became_peered":"2023-11-08T19:27:08.191520+0000","last_unstale":"2023-11-08T19:28:03.705567+0000","last_undegraded":"2023-11-08T19:28:03.705567+0000","last_fullsized":"2023-11-08T19:28:03.705567+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:49:16.458634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:04.701104+0000","last_change":"2023-11-08T19:28:03.722086+0000","last_active":"2023-11-08T19:28:04.701104+0000","last_peered":"2023-11-08T19:28:04.701104+0000","last_clean":"2023-11-08T19:28:04.701104+0000","last_became_active":"2023-11-08T19:28:03.721713+0000","last_became_peered":"2023-11-08T19:28:03.721713+0000","last_unstale":"2023-11-08T19:28:04.701104+0000","last_undegraded":"2023-11-08T19:28:04.701104+0000","last_fullsized":"2023-11-08T19:28:04.701104+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:02:58.243507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:04.596780+0000","last_change":"2023-11-08T19:28:02.707105+0000","last_active":"2023-11-08T19:28:04.596780+0000","last_peered":"2023-11-08T19:28:04.596780+0000","last_clean":"2023-11-08T19:28:04.596780+0000","last_became_active":"2023-11-08T19:28:02.706615+0000","last_became_peered":"2023-11-08T19:28:02.706615+0000","last_unstale":"2023-11-08T19:28:04.596780+0000","last_undegraded":"2023-11-08T19:28:04.596780+0000","last_fullsized":"2023-11-08T19:28:04.596780+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:18:55.364855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":72,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.699456+0000","last_change":"2023-11-08T19:26:05.608892+0000","last_active":"2023-11-08T19:28:03.699456+0000","last_peered":"2023-11-08T19:28:03.699456+0000","last_clean":"2023-11-08T19:28:03.699456+0000","last_became_active":"2023-11-08T19:26:04.604519+0000","last_became_peered":"2023-11-08T19:26:04.604519+0000","last_unstale":"2023-11-08T19:28:03.699456+0000","last_undegraded":"2023-11-08T19:28:03.699456+0000","last_fullsized":"2023-11-08T19:28:03.699456+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:11:26.508634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051248099999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.699608+0000","last_change":"2023-11-08T19:27:09.181431+0000","last_active":"2023-11-08T19:28:03.699608+0000","last_peered":"2023-11-08T19:28:03.699608+0000","last_clean":"2023-11-08T19:28:03.699608+0000","last_became_active":"2023-11-08T19:27:07.190800+0000","last_became_peered":"2023-11-08T19:27:07.190800+0000","last_unstale":"2023-11-08T19:28:03.699608+0000","last_undegraded":"2023-11-08T19:28:03.699608+0000","last_fullsized":"2023-11-08T19:28:03.699608+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:46:30.452678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656099999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.706227+0000","last_change":"2023-11-08T19:27:08.187983+0000","last_active":"2023-11-08T19:28:03.706227+0000","last_peered":"2023-11-08T19:28:03.706227+0000","last_clean":"2023-11-08T19:28:03.706227+0000","last_became_active":"2023-11-08T19:27:08.187322+0000","last_became_peered":"2023-11-08T19:27:08.187322+0000","last_unstale":"2023-11-08T19:28:03.706227+0000","last_undegraded":"2023-11-08T19:28:03.706227+0000","last_fullsized":"2023-11-08T19:28:03.706227+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:33:07.647519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:05.212115+0000","last_change":"2023-11-08T19:28:03.723439+0000","last_active":"2023-11-08T19:28:05.212115+0000","last_peered":"2023-11-08T19:28:05.212115+0000","last_clean":"2023-11-08T19:28:05.212115+0000","last_became_active":"2023-11-08T19:28:03.722747+0000","last_became_peered":"2023-11-08T19:28:03.722747+0000","last_unstale":"2023-11-08T19:28:05.212115+0000","last_undegraded":"2023-11-08T19:28:05.212115+0000","last_fullsized":"2023-11-08T19:28:05.212115+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:00:11.333972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.697016+0000","last_change":"2023-11-08T19:28:02.720466+0000","last_active":"2023-11-08T19:28:03.697016+0000","last_peered":"2023-11-08T19:28:03.697016+0000","last_clean":"2023-11-08T19:28:03.697016+0000","last_became_active":"2023-11-08T19:28:02.720193+0000","last_became_peered":"2023-11-08T19:28:02.720193+0000","last_unstale":"2023-11-08T19:28:03.697016+0000","last_undegraded":"2023-11-08T19:28:03.697016+0000","last_fullsized":"2023-11-08T19:28:03.697016+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:47:35.837511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":60,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.701362+0000","last_change":"2023-11-08T19:26:06.394075+0000","last_active":"2023-11-08T19:28:03.701362+0000","last_peered":"2023-11-08T19:28:03.701362+0000","last_clean":"2023-11-08T19:28:03.701362+0000","last_became_active":"2023-11-08T19:26:04.606967+0000","last_became_peered":"2023-11-08T19:26:04.606967+0000","last_unstale":"2023-11-08T19:28:03.701362+0000","last_undegraded":"2023-11-08T19:28:03.701362+0000","last_fullsized":"2023-11-08T19:28:03.701362+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:19:56.286835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040971999999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.699888+0000","last_change":"2023-11-08T19:27:09.180722+0000","last_active":"2023-11-08T19:28:03.699888+0000","last_peered":"2023-11-08T19:28:03.699888+0000","last_clean":"2023-11-08T19:28:03.699888+0000","last_became_active":"2023-11-08T19:27:07.186977+0000","last_became_peered":"2023-11-08T19:27:07.186977+0000","last_unstale":"2023-11-08T19:28:03.699888+0000","last_undegraded":"2023-11-08T19:28:03.699888+0000","last_fullsized":"2023-11-08T19:28:03.699888+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:17:59.966463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057675100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.701722+0000","last_change":"2023-11-08T19:27:08.193963+0000","last_active":"2023-11-08T19:28:03.701722+0000","last_peered":"2023-11-08T19:28:03.701722+0000","last_clean":"2023-11-08T19:28:03.701722+0000","last_became_active":"2023-11-08T19:27:08.193098+0000","last_became_peered":"2023-11-08T19:27:08.193098+0000","last_unstale":"2023-11-08T19:28:03.701722+0000","last_undegraded":"2023-11-08T19:28:03.701722+0000","last_fullsized":"2023-11-08T19:28:03.701722+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:41:09.827370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:05.139402+0000","last_change":"2023-11-08T19:28:03.720461+0000","last_active":"2023-11-08T19:28:05.139402+0000","last_peered":"2023-11-08T19:28:05.139402+0000","last_clean":"2023-11-08T19:28:05.139402+0000","last_became_active":"2023-11-08T19:28:03.720211+0000","last_became_peered":"2023-11-08T19:28:03.720211+0000","last_unstale":"2023-11-08T19:28:05.139402+0000","last_undegraded":"2023-11-08T19:28:05.139402+0000","last_fullsized":"2023-11-08T19:28:05.139402+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:32:10.892027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"91'2","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.720911+0000","last_change":"2023-11-08T19:28:02.718059+0000","last_active":"2023-11-08T19:28:03.720911+0000","last_peered":"2023-11-08T19:28:03.720911+0000","last_clean":"2023-11-08T19:28:03.720911+0000","last_became_active":"2023-11-08T19:28:02.717775+0000","last_became_peered":"2023-11-08T19:28:02.717775+0000","last_unstale":"2023-11-08T19:28:03.720911+0000","last_undegraded":"2023-11-08T19:28:03.720911+0000","last_fullsized":"2023-11-08T19:28:03.720911+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:21:11.679373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":60,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.708665+0000","last_change":"2023-11-08T19:26:06.265686+0000","last_active":"2023-11-08T19:28:03.708665+0000","last_peered":"2023-11-08T19:28:03.708665+0000","last_clean":"2023-11-08T19:28:03.708665+0000","last_became_active":"2023-11-08T19:26:04.602111+0000","last_became_peered":"2023-11-08T19:26:04.602111+0000","last_unstale":"2023-11-08T19:28:03.708665+0000","last_undegraded":"2023-11-08T19:28:03.708665+0000","last_fullsized":"2023-11-08T19:28:03.708665+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:40:11.164951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056197899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.706404+0000","last_change":"2023-11-08T19:27:09.181333+0000","last_active":"2023-11-08T19:28:03.706404+0000","last_peered":"2023-11-08T19:28:03.706404+0000","last_clean":"2023-11-08T19:28:03.706404+0000","last_became_active":"2023-11-08T19:27:07.188657+0000","last_became_peered":"2023-11-08T19:27:07.188657+0000","last_unstale":"2023-11-08T19:28:03.706404+0000","last_undegraded":"2023-11-08T19:28:03.706404+0000","last_fullsized":"2023-11-08T19:28:03.706404+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:25:56.788615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001227393,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.701047+0000","last_change":"2023-11-08T19:27:08.188533+0000","last_active":"2023-11-08T19:28:03.701047+0000","last_peered":"2023-11-08T19:28:03.701047+0000","last_clean":"2023-11-08T19:28:03.701047+0000","last_became_active":"2023-11-08T19:27:08.188240+0000","last_became_peered":"2023-11-08T19:27:08.188240+0000","last_unstale":"2023-11-08T19:28:03.701047+0000","last_undegraded":"2023-11-08T19:28:03.701047+0000","last_fullsized":"2023-11-08T19:28:03.701047+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:55:37.653521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:05.210891+0000","last_change":"2023-11-08T19:28:03.716743+0000","last_active":"2023-11-08T19:28:05.210891+0000","last_peered":"2023-11-08T19:28:05.210891+0000","last_clean":"2023-11-08T19:28:05.210891+0000","last_became_active":"2023-11-08T19:28:03.716457+0000","last_became_peered":"2023-11-08T19:28:03.716457+0000","last_unstale":"2023-11-08T19:28:05.210891+0000","last_undegraded":"2023-11-08T19:28:05.210891+0000","last_fullsized":"2023-11-08T19:28:05.210891+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:06:26.112939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"92'1","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:05.140059+0000","last_change":"2023-11-08T19:28:02.710864+0000","last_active":"2023-11-08T19:28:05.140059+0000","last_peered":"2023-11-08T19:28:05.140059+0000","last_clean":"2023-11-08T19:28:05.140059+0000","last_became_active":"2023-11-08T19:28:02.710597+0000","last_became_peered":"2023-11-08T19:28:02.710597+0000","last_unstale":"2023-11-08T19:28:05.140059+0000","last_undegraded":"2023-11-08T19:28:05.140059+0000","last_fullsized":"2023-11-08T19:28:05.140059+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:30:40.294603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.703867+0000","last_change":"2023-11-08T19:27:08.188458+0000","last_active":"2023-11-08T19:28:03.703867+0000","last_peered":"2023-11-08T19:28:03.703867+0000","last_clean":"2023-11-08T19:28:03.703867+0000","last_became_active":"2023-11-08T19:27:08.187880+0000","last_became_peered":"2023-11-08T19:27:08.187880+0000","last_unstale":"2023-11-08T19:28:03.703867+0000","last_undegraded":"2023-11-08T19:28:03.703867+0000","last_fullsized":"2023-11-08T19:28:03.703867+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:25:28.618214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":61,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.698195+0000","last_change":"2023-11-08T19:26:05.606898+0000","last_active":"2023-11-08T19:28:03.698195+0000","last_peered":"2023-11-08T19:28:03.698195+0000","last_clean":"2023-11-08T19:28:03.698195+0000","last_became_active":"2023-11-08T19:26:04.603955+0000","last_became_peered":"2023-11-08T19:26:04.603955+0000","last_unstale":"2023-11-08T19:28:03.698195+0000","last_undegraded":"2023-11-08T19:28:03.698195+0000","last_fullsized":"2023-11-08T19:28:03.698195+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:19:11.589559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036473499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.698769+0000","last_change":"2023-11-08T19:27:09.178770+0000","last_active":"2023-11-08T19:28:03.698769+0000","last_peered":"2023-11-08T19:28:03.698769+0000","last_clean":"2023-11-08T19:28:03.698769+0000","last_became_active":"2023-11-08T19:27:07.194992+0000","last_became_peered":"2023-11-08T19:27:07.194992+0000","last_unstale":"2023-11-08T19:28:03.698769+0000","last_undegraded":"2023-11-08T19:28:03.698769+0000","last_fullsized":"2023-11-08T19:28:03.698769+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:35:34.026910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000770285,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:05.210107+0000","last_change":"2023-11-08T19:28:03.716878+0000","last_active":"2023-11-08T19:28:05.210107+0000","last_peered":"2023-11-08T19:28:05.210107+0000","last_clean":"2023-11-08T19:28:05.210107+0000","last_became_active":"2023-11-08T19:28:03.716614+0000","last_became_peered":"2023-11-08T19:28:03.716614+0000","last_unstale":"2023-11-08T19:28:05.210107+0000","last_undegraded":"2023-11-08T19:28:05.210107+0000","last_fullsized":"2023-11-08T19:28:05.210107+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:07:34.246320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:04.597014+0000","last_change":"2023-11-08T19:28:02.707011+0000","last_active":"2023-11-08T19:28:04.597014+0000","last_peered":"2023-11-08T19:28:04.597014+0000","last_clean":"2023-11-08T19:28:04.597014+0000","last_became_active":"2023-11-08T19:28:02.706729+0000","last_became_peered":"2023-11-08T19:28:02.706729+0000","last_unstale":"2023-11-08T19:28:04.597014+0000","last_undegraded":"2023-11-08T19:28:04.597014+0000","last_fullsized":"2023-11-08T19:28:04.597014+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:29:25.082011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.704828+0000","last_change":"2023-11-08T19:27:08.193953+0000","last_active":"2023-11-08T19:28:03.704828+0000","last_peered":"2023-11-08T19:28:03.704828+0000","last_clean":"2023-11-08T19:28:03.704828+0000","last_became_active":"2023-11-08T19:27:08.193653+0000","last_became_peered":"2023-11-08T19:27:08.193653+0000","last_unstale":"2023-11-08T19:28:03.704828+0000","last_undegraded":"2023-11-08T19:28:03.704828+0000","last_fullsized":"2023-11-08T19:28:03.704828+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:35:30.073885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":66,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.705057+0000","last_change":"2023-11-08T19:26:06.424464+0000","last_active":"2023-11-08T19:28:03.705057+0000","last_peered":"2023-11-08T19:28:03.705057+0000","last_clean":"2023-11-08T19:28:03.705057+0000","last_became_active":"2023-11-08T19:26:04.603144+0000","last_became_peered":"2023-11-08T19:26:04.603144+0000","last_unstale":"2023-11-08T19:28:03.705057+0000","last_undegraded":"2023-11-08T19:28:03.705057+0000","last_fullsized":"2023-11-08T19:28:03.705057+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:42:55.269365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054090799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.705491+0000","last_change":"2023-11-08T19:27:09.180479+0000","last_active":"2023-11-08T19:28:03.705491+0000","last_peered":"2023-11-08T19:28:03.705491+0000","last_clean":"2023-11-08T19:28:03.705491+0000","last_became_active":"2023-11-08T19:27:07.190438+0000","last_became_peered":"2023-11-08T19:27:07.190438+0000","last_unstale":"2023-11-08T19:28:03.705491+0000","last_undegraded":"2023-11-08T19:28:03.705491+0000","last_fullsized":"2023-11-08T19:28:03.705491+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:25:14.487524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050893300000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:05.210891+0000","last_change":"2023-11-08T19:28:03.721352+0000","last_active":"2023-11-08T19:28:05.210891+0000","last_peered":"2023-11-08T19:28:05.210891+0000","last_clean":"2023-11-08T19:28:05.210891+0000","last_became_active":"2023-11-08T19:28:03.720903+0000","last_became_peered":"2023-11-08T19:28:03.720903+0000","last_unstale":"2023-11-08T19:28:05.210891+0000","last_undegraded":"2023-11-08T19:28:05.210891+0000","last_fullsized":"2023-11-08T19:28:05.210891+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:21:12.684194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.702895+0000","last_change":"2023-11-08T19:28:02.708336+0000","last_active":"2023-11-08T19:28:03.702895+0000","last_peered":"2023-11-08T19:28:03.702895+0000","last_clean":"2023-11-08T19:28:03.702895+0000","last_became_active":"2023-11-08T19:28:02.708016+0000","last_became_peered":"2023-11-08T19:28:02.708016+0000","last_unstale":"2023-11-08T19:28:03.702895+0000","last_undegraded":"2023-11-08T19:28:03.702895+0000","last_fullsized":"2023-11-08T19:28:03.702895+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:11:44.304551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.705298+0000","last_change":"2023-11-08T19:27:08.190530+0000","last_active":"2023-11-08T19:28:03.705298+0000","last_peered":"2023-11-08T19:28:03.705298+0000","last_clean":"2023-11-08T19:28:03.705298+0000","last_became_active":"2023-11-08T19:27:08.190203+0000","last_became_peered":"2023-11-08T19:27:08.190203+0000","last_unstale":"2023-11-08T19:28:03.705298+0000","last_undegraded":"2023-11-08T19:28:03.705298+0000","last_fullsized":"2023-11-08T19:28:03.705298+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:26:32.777602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":62,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.722523+0000","last_change":"2023-11-08T19:26:06.541623+0000","last_active":"2023-11-08T19:28:03.722523+0000","last_peered":"2023-11-08T19:28:03.722523+0000","last_clean":"2023-11-08T19:28:03.722523+0000","last_became_active":"2023-11-08T19:26:04.601961+0000","last_became_peered":"2023-11-08T19:26:04.601961+0000","last_unstale":"2023-11-08T19:28:03.722523+0000","last_undegraded":"2023-11-08T19:28:03.722523+0000","last_fullsized":"2023-11-08T19:28:03.722523+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:11:26.508634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000466386,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.705974+0000","last_change":"2023-11-08T19:27:09.431231+0000","last_active":"2023-11-08T19:28:03.705974+0000","last_peered":"2023-11-08T19:28:03.705974+0000","last_clean":"2023-11-08T19:28:03.705974+0000","last_became_active":"2023-11-08T19:27:07.191583+0000","last_became_peered":"2023-11-08T19:27:07.191583+0000","last_unstale":"2023-11-08T19:28:03.705974+0000","last_undegraded":"2023-11-08T19:28:03.705974+0000","last_fullsized":"2023-11-08T19:28:03.705974+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:58:48.357266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093695200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:05.211443+0000","last_change":"2023-11-08T19:28:03.713201+0000","last_active":"2023-11-08T19:28:05.211443+0000","last_peered":"2023-11-08T19:28:05.211443+0000","last_clean":"2023-11-08T19:28:05.211443+0000","last_became_active":"2023-11-08T19:28:03.712926+0000","last_became_peered":"2023-11-08T19:28:03.712926+0000","last_unstale":"2023-11-08T19:28:05.211443+0000","last_undegraded":"2023-11-08T19:28:05.211443+0000","last_fullsized":"2023-11-08T19:28:05.211443+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:29:32.281100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"91'1","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:04.596855+0000","last_change":"2023-11-08T19:28:02.708326+0000","last_active":"2023-11-08T19:28:04.596855+0000","last_peered":"2023-11-08T19:28:04.596855+0000","last_clean":"2023-11-08T19:28:04.596855+0000","last_became_active":"2023-11-08T19:28:02.708055+0000","last_became_peered":"2023-11-08T19:28:02.708055+0000","last_unstale":"2023-11-08T19:28:04.596855+0000","last_undegraded":"2023-11-08T19:28:04.596855+0000","last_fullsized":"2023-11-08T19:28:04.596855+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:50:41.723104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.706051+0000","last_change":"2023-11-08T19:27:08.187900+0000","last_active":"2023-11-08T19:28:03.706051+0000","last_peered":"2023-11-08T19:28:03.706051+0000","last_clean":"2023-11-08T19:28:03.706051+0000","last_became_active":"2023-11-08T19:27:08.187626+0000","last_became_peered":"2023-11-08T19:27:08.187626+0000","last_unstale":"2023-11-08T19:28:03.706051+0000","last_undegraded":"2023-11-08T19:28:03.706051+0000","last_fullsized":"2023-11-08T19:28:03.706051+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:00:55.878397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":72,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.700188+0000","last_change":"2023-11-08T19:26:05.608980+0000","last_active":"2023-11-08T19:28:03.700188+0000","last_peered":"2023-11-08T19:28:03.700188+0000","last_clean":"2023-11-08T19:28:03.700188+0000","last_became_active":"2023-11-08T19:26:04.606386+0000","last_became_peered":"2023-11-08T19:26:04.606386+0000","last_unstale":"2023-11-08T19:28:03.700188+0000","last_undegraded":"2023-11-08T19:28:03.700188+0000","last_fullsized":"2023-11-08T19:28:03.700188+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:41:50.771457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037950599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.698117+0000","last_change":"2023-11-08T19:27:09.179037+0000","last_active":"2023-11-08T19:28:03.698117+0000","last_peered":"2023-11-08T19:28:03.698117+0000","last_clean":"2023-11-08T19:28:03.698117+0000","last_became_active":"2023-11-08T19:27:07.195033+0000","last_became_peered":"2023-11-08T19:27:07.195033+0000","last_unstale":"2023-11-08T19:28:03.698117+0000","last_undegraded":"2023-11-08T19:28:03.698117+0000","last_fullsized":"2023-11-08T19:28:03.698117+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:19:08.253483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041824800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:05.139897+0000","last_change":"2023-11-08T19:28:03.719383+0000","last_active":"2023-11-08T19:28:05.139897+0000","last_peered":"2023-11-08T19:28:05.139897+0000","last_clean":"2023-11-08T19:28:05.139897+0000","last_became_active":"2023-11-08T19:28:03.718422+0000","last_became_peered":"2023-11-08T19:28:03.718422+0000","last_unstale":"2023-11-08T19:28:05.139897+0000","last_undegraded":"2023-11-08T19:28:05.139897+0000","last_fullsized":"2023-11-08T19:28:05.139897+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:00:39.284900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"91'1","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.726348+0000","last_change":"2023-11-08T19:28:02.712619+0000","last_active":"2023-11-08T19:28:03.726348+0000","last_peered":"2023-11-08T19:28:03.726348+0000","last_clean":"2023-11-08T19:28:03.726348+0000","last_became_active":"2023-11-08T19:28:02.712342+0000","last_became_peered":"2023-11-08T19:28:02.712342+0000","last_unstale":"2023-11-08T19:28:03.726348+0000","last_undegraded":"2023-11-08T19:28:03.726348+0000","last_fullsized":"2023-11-08T19:28:03.726348+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:18:55.364855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.701289+0000","last_change":"2023-11-08T19:27:08.193761+0000","last_active":"2023-11-08T19:28:03.701289+0000","last_peered":"2023-11-08T19:28:03.701289+0000","last_clean":"2023-11-08T19:28:03.701289+0000","last_became_active":"2023-11-08T19:27:08.193017+0000","last_became_peered":"2023-11-08T19:27:08.193017+0000","last_unstale":"2023-11-08T19:28:03.701289+0000","last_undegraded":"2023-11-08T19:28:03.701289+0000","last_fullsized":"2023-11-08T19:28:03.701289+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:18:00.971466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":66,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.701468+0000","last_change":"2023-11-08T19:26:06.424073+0000","last_active":"2023-11-08T19:28:03.701468+0000","last_peered":"2023-11-08T19:28:03.701468+0000","last_clean":"2023-11-08T19:28:03.701468+0000","last_became_active":"2023-11-08T19:26:04.599046+0000","last_became_peered":"2023-11-08T19:26:04.599046+0000","last_unstale":"2023-11-08T19:28:03.701468+0000","last_undegraded":"2023-11-08T19:28:03.701468+0000","last_fullsized":"2023-11-08T19:28:03.701468+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:04:43.562566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035092199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.699765+0000","last_change":"2023-11-08T19:27:09.181716+0000","last_active":"2023-11-08T19:28:03.699765+0000","last_peered":"2023-11-08T19:28:03.699765+0000","last_clean":"2023-11-08T19:28:03.699765+0000","last_became_active":"2023-11-08T19:27:07.187077+0000","last_became_peered":"2023-11-08T19:27:07.187077+0000","last_unstale":"2023-11-08T19:28:03.699765+0000","last_undegraded":"2023-11-08T19:28:03.699765+0000","last_fullsized":"2023-11-08T19:28:03.699765+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:51:29.247104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056673599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:04.699935+0000","last_change":"2023-11-08T19:28:03.724351+0000","last_active":"2023-11-08T19:28:04.699935+0000","last_peered":"2023-11-08T19:28:04.699935+0000","last_clean":"2023-11-08T19:28:04.699935+0000","last_became_active":"2023-11-08T19:28:03.724109+0000","last_became_peered":"2023-11-08T19:28:03.724109+0000","last_unstale":"2023-11-08T19:28:04.699935+0000","last_undegraded":"2023-11-08T19:28:04.699935+0000","last_fullsized":"2023-11-08T19:28:04.699935+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:35:11.551335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.704800+0000","last_change":"2023-11-08T19:28:02.711594+0000","last_active":"2023-11-08T19:28:03.704800+0000","last_peered":"2023-11-08T19:28:03.704800+0000","last_clean":"2023-11-08T19:28:03.704800+0000","last_became_active":"2023-11-08T19:28:02.711204+0000","last_became_peered":"2023-11-08T19:28:02.711204+0000","last_unstale":"2023-11-08T19:28:03.704800+0000","last_undegraded":"2023-11-08T19:28:03.704800+0000","last_fullsized":"2023-11-08T19:28:03.704800+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:39:48.728129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.706133+0000","last_change":"2023-11-08T19:27:08.188143+0000","last_active":"2023-11-08T19:28:03.706133+0000","last_peered":"2023-11-08T19:28:03.706133+0000","last_clean":"2023-11-08T19:28:03.706133+0000","last_became_active":"2023-11-08T19:27:08.187714+0000","last_became_peered":"2023-11-08T19:27:08.187714+0000","last_unstale":"2023-11-08T19:28:03.706133+0000","last_undegraded":"2023-11-08T19:28:03.706133+0000","last_fullsized":"2023-11-08T19:28:03.706133+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:02:10.863403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":60,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.703500+0000","last_change":"2023-11-08T19:26:06.394320+0000","last_active":"2023-11-08T19:28:03.703500+0000","last_peered":"2023-11-08T19:28:03.703500+0000","last_clean":"2023-11-08T19:28:03.703500+0000","last_became_active":"2023-11-08T19:26:04.603212+0000","last_became_peered":"2023-11-08T19:26:04.603212+0000","last_unstale":"2023-11-08T19:28:03.703500+0000","last_undegraded":"2023-11-08T19:28:03.703500+0000","last_fullsized":"2023-11-08T19:28:03.703500+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:25:29.947079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054695399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.699509+0000","last_change":"2023-11-08T19:27:09.181613+0000","last_active":"2023-11-08T19:28:03.699509+0000","last_peered":"2023-11-08T19:28:03.699509+0000","last_clean":"2023-11-08T19:28:03.699509+0000","last_became_active":"2023-11-08T19:27:07.190897+0000","last_became_peered":"2023-11-08T19:27:07.190897+0000","last_unstale":"2023-11-08T19:28:03.699509+0000","last_undegraded":"2023-11-08T19:28:03.699509+0000","last_fullsized":"2023-11-08T19:28:03.699509+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:27:57.388125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011172459999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:05.210843+0000","last_change":"2023-11-08T19:28:03.715848+0000","last_active":"2023-11-08T19:28:05.210843+0000","last_peered":"2023-11-08T19:28:05.210843+0000","last_clean":"2023-11-08T19:28:05.210843+0000","last_became_active":"2023-11-08T19:28:03.715506+0000","last_became_peered":"2023-11-08T19:28:03.715506+0000","last_unstale":"2023-11-08T19:28:05.210843+0000","last_undegraded":"2023-11-08T19:28:05.210843+0000","last_fullsized":"2023-11-08T19:28:05.210843+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:29:02.099134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.700979+0000","last_change":"2023-11-08T19:28:02.707358+0000","last_active":"2023-11-08T19:28:03.700979+0000","last_peered":"2023-11-08T19:28:03.700979+0000","last_clean":"2023-11-08T19:28:03.700979+0000","last_became_active":"2023-11-08T19:28:02.707109+0000","last_became_peered":"2023-11-08T19:28:02.707109+0000","last_unstale":"2023-11-08T19:28:03.700979+0000","last_undegraded":"2023-11-08T19:28:03.700979+0000","last_fullsized":"2023-11-08T19:28:03.700979+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:06:25.108118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.704597+0000","last_change":"2023-11-08T19:27:08.188668+0000","last_active":"2023-11-08T19:28:03.704597+0000","last_peered":"2023-11-08T19:28:03.704597+0000","last_clean":"2023-11-08T19:28:03.704597+0000","last_became_active":"2023-11-08T19:27:08.188441+0000","last_became_peered":"2023-11-08T19:27:08.188441+0000","last_unstale":"2023-11-08T19:28:03.704597+0000","last_undegraded":"2023-11-08T19:28:03.704597+0000","last_fullsized":"2023-11-08T19:28:03.704597+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:45:44.379160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":66,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.705502+0000","last_change":"2023-11-08T19:26:06.423754+0000","last_active":"2023-11-08T19:28:03.705502+0000","last_peered":"2023-11-08T19:28:03.705502+0000","last_clean":"2023-11-08T19:28:03.705502+0000","last_became_active":"2023-11-08T19:26:04.599322+0000","last_became_peered":"2023-11-08T19:26:04.599322+0000","last_unstale":"2023-11-08T19:28:03.705502+0000","last_undegraded":"2023-11-08T19:28:03.705502+0000","last_fullsized":"2023-11-08T19:28:03.705502+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:13:08.689128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042670000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.701929+0000","last_change":"2023-11-08T19:27:09.181656+0000","last_active":"2023-11-08T19:28:03.701929+0000","last_peered":"2023-11-08T19:28:03.701929+0000","last_clean":"2023-11-08T19:28:03.701929+0000","last_became_active":"2023-11-08T19:27:07.187337+0000","last_became_peered":"2023-11-08T19:27:07.187337+0000","last_unstale":"2023-11-08T19:28:03.701929+0000","last_undegraded":"2023-11-08T19:28:03.701929+0000","last_fullsized":"2023-11-08T19:28:03.701929+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:14:17.605057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000346967,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:05.139552+0000","last_change":"2023-11-08T19:28:03.714132+0000","last_active":"2023-11-08T19:28:05.139552+0000","last_peered":"2023-11-08T19:28:05.139552+0000","last_clean":"2023-11-08T19:28:05.139552+0000","last_became_active":"2023-11-08T19:28:03.713735+0000","last_became_peered":"2023-11-08T19:28:03.713735+0000","last_unstale":"2023-11-08T19:28:05.139552+0000","last_undegraded":"2023-11-08T19:28:05.139552+0000","last_fullsized":"2023-11-08T19:28:05.139552+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:27:02.474288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"91'4","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.721593+0000","last_change":"2023-11-08T19:28:02.704760+0000","last_active":"2023-11-08T19:28:03.721593+0000","last_peered":"2023-11-08T19:28:03.721593+0000","last_clean":"2023-11-08T19:28:03.721593+0000","last_became_active":"2023-11-08T19:28:02.704410+0000","last_became_peered":"2023-11-08T19:28:02.704410+0000","last_unstale":"2023-11-08T19:28:03.721593+0000","last_undegraded":"2023-11-08T19:28:03.721593+0000","last_fullsized":"2023-11-08T19:28:03.721593+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:43:49.050271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.703802+0000","last_change":"2023-11-08T19:27:08.194065+0000","last_active":"2023-11-08T19:28:03.703802+0000","last_peered":"2023-11-08T19:28:03.703802+0000","last_clean":"2023-11-08T19:28:03.703802+0000","last_became_active":"2023-11-08T19:27:08.193532+0000","last_became_peered":"2023-11-08T19:27:08.193532+0000","last_unstale":"2023-11-08T19:28:03.703802+0000","last_undegraded":"2023-11-08T19:28:03.703802+0000","last_fullsized":"2023-11-08T19:28:03.703802+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:27:58.393129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":60,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.704261+0000","last_change":"2023-11-08T19:26:06.394488+0000","last_active":"2023-11-08T19:28:03.704261+0000","last_peered":"2023-11-08T19:28:03.704261+0000","last_clean":"2023-11-08T19:28:03.704261+0000","last_became_active":"2023-11-08T19:26:04.606805+0000","last_became_peered":"2023-11-08T19:26:04.606805+0000","last_unstale":"2023-11-08T19:28:03.704261+0000","last_undegraded":"2023-11-08T19:28:03.704261+0000","last_fullsized":"2023-11-08T19:28:03.704261+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:25:20.857133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045317000000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.704686+0000","last_change":"2023-11-08T19:27:09.211647+0000","last_active":"2023-11-08T19:28:03.704686+0000","last_peered":"2023-11-08T19:28:03.704686+0000","last_clean":"2023-11-08T19:28:03.704686+0000","last_became_active":"2023-11-08T19:27:07.190309+0000","last_became_peered":"2023-11-08T19:27:07.190309+0000","last_unstale":"2023-11-08T19:28:03.704686+0000","last_undegraded":"2023-11-08T19:28:03.704686+0000","last_fullsized":"2023-11-08T19:28:03.704686+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:56:04.292154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047615199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:05.209953+0000","last_change":"2023-11-08T19:28:03.710916+0000","last_active":"2023-11-08T19:28:05.209953+0000","last_peered":"2023-11-08T19:28:05.209953+0000","last_clean":"2023-11-08T19:28:05.209953+0000","last_became_active":"2023-11-08T19:28:03.710670+0000","last_became_peered":"2023-11-08T19:28:03.710670+0000","last_unstale":"2023-11-08T19:28:05.209953+0000","last_undegraded":"2023-11-08T19:28:05.209953+0000","last_fullsized":"2023-11-08T19:28:05.209953+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:23:23.359779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"91'1","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.720329+0000","last_change":"2023-11-08T19:28:02.717508+0000","last_active":"2023-11-08T19:28:03.720329+0000","last_peered":"2023-11-08T19:28:03.720329+0000","last_clean":"2023-11-08T19:28:03.720329+0000","last_became_active":"2023-11-08T19:28:02.717126+0000","last_became_peered":"2023-11-08T19:28:02.717126+0000","last_unstale":"2023-11-08T19:28:03.720329+0000","last_undegraded":"2023-11-08T19:28:03.720329+0000","last_fullsized":"2023-11-08T19:28:03.720329+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:11:41.600313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.704822+0000","last_change":"2023-11-08T19:27:08.194884+0000","last_active":"2023-11-08T19:28:03.704822+0000","last_peered":"2023-11-08T19:28:03.704822+0000","last_clean":"2023-11-08T19:28:03.704822+0000","last_became_active":"2023-11-08T19:27:08.193241+0000","last_became_peered":"2023-11-08T19:27:08.193241+0000","last_unstale":"2023-11-08T19:28:03.704822+0000","last_undegraded":"2023-11-08T19:28:03.704822+0000","last_fullsized":"2023-11-08T19:28:03.704822+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:46:31.457682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":60,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.705026+0000","last_change":"2023-11-08T19:26:06.265907+0000","last_active":"2023-11-08T19:28:03.705026+0000","last_peered":"2023-11-08T19:28:03.705026+0000","last_clean":"2023-11-08T19:28:03.705026+0000","last_became_active":"2023-11-08T19:26:04.604772+0000","last_became_peered":"2023-11-08T19:26:04.604772+0000","last_unstale":"2023-11-08T19:28:03.705026+0000","last_undegraded":"2023-11-08T19:28:03.705026+0000","last_fullsized":"2023-11-08T19:28:03.705026+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:44:16.139794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066875300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.719805+0000","last_change":"2023-11-08T19:27:09.282699+0000","last_active":"2023-11-08T19:28:03.719805+0000","last_peered":"2023-11-08T19:28:03.719805+0000","last_clean":"2023-11-08T19:28:03.719805+0000","last_became_active":"2023-11-08T19:27:07.191212+0000","last_became_peered":"2023-11-08T19:27:07.191212+0000","last_unstale":"2023-11-08T19:28:03.719805+0000","last_undegraded":"2023-11-08T19:28:03.719805+0000","last_fullsized":"2023-11-08T19:28:03.719805+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:35:14.715110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046640800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:05.211377+0000","last_change":"2023-11-08T19:28:03.713915+0000","last_active":"2023-11-08T19:28:05.211377+0000","last_peered":"2023-11-08T19:28:05.211377+0000","last_clean":"2023-11-08T19:28:05.211377+0000","last_became_active":"2023-11-08T19:28:03.712248+0000","last_became_peered":"2023-11-08T19:28:03.712248+0000","last_unstale":"2023-11-08T19:28:05.211377+0000","last_undegraded":"2023-11-08T19:28:05.211377+0000","last_fullsized":"2023-11-08T19:28:05.211377+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:33:39.095416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"91'2","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.721546+0000","last_change":"2023-11-08T19:28:02.712356+0000","last_active":"2023-11-08T19:28:03.721546+0000","last_peered":"2023-11-08T19:28:03.721546+0000","last_clean":"2023-11-08T19:28:03.721546+0000","last_became_active":"2023-11-08T19:28:02.712073+0000","last_became_peered":"2023-11-08T19:28:02.712073+0000","last_unstale":"2023-11-08T19:28:03.721546+0000","last_undegraded":"2023-11-08T19:28:03.721546+0000","last_fullsized":"2023-11-08T19:28:03.721546+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:00:10.329151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.705078+0000","last_change":"2023-11-08T19:27:08.187644+0000","last_active":"2023-11-08T19:28:03.705078+0000","last_peered":"2023-11-08T19:28:03.705078+0000","last_clean":"2023-11-08T19:28:03.705078+0000","last_became_active":"2023-11-08T19:27:08.187376+0000","last_became_peered":"2023-11-08T19:27:08.187376+0000","last_unstale":"2023-11-08T19:28:03.705078+0000","last_undegraded":"2023-11-08T19:28:03.705078+0000","last_fullsized":"2023-11-08T19:28:03.705078+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:22:40.300185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":61,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.697967+0000","last_change":"2023-11-08T19:26:05.606946+0000","last_active":"2023-11-08T19:28:03.697967+0000","last_peered":"2023-11-08T19:28:03.697967+0000","last_clean":"2023-11-08T19:28:03.697967+0000","last_became_active":"2023-11-08T19:26:04.598775+0000","last_became_peered":"2023-11-08T19:26:04.598775+0000","last_unstale":"2023-11-08T19:28:03.697967+0000","last_undegraded":"2023-11-08T19:28:03.697967+0000","last_fullsized":"2023-11-08T19:28:03.697967+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:20:56.898927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049870399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.701433+0000","last_change":"2023-11-08T19:27:09.331646+0000","last_active":"2023-11-08T19:28:03.701433+0000","last_peered":"2023-11-08T19:28:03.701433+0000","last_clean":"2023-11-08T19:28:03.701433+0000","last_became_active":"2023-11-08T19:27:07.190619+0000","last_became_peered":"2023-11-08T19:27:07.190619+0000","last_unstale":"2023-11-08T19:28:03.701433+0000","last_undegraded":"2023-11-08T19:28:03.701433+0000","last_fullsized":"2023-11-08T19:28:03.701433+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:42:47.371458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065918400000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:05.211056+0000","last_change":"2023-11-08T19:28:03.721163+0000","last_active":"2023-11-08T19:28:05.211056+0000","last_peered":"2023-11-08T19:28:05.211056+0000","last_clean":"2023-11-08T19:28:05.211056+0000","last_became_active":"2023-11-08T19:28:03.720603+0000","last_became_peered":"2023-11-08T19:28:03.720603+0000","last_unstale":"2023-11-08T19:28:05.211056+0000","last_undegraded":"2023-11-08T19:28:05.211056+0000","last_fullsized":"2023-11-08T19:28:05.211056+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:00:53.382966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"91'2","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.716105+0000","last_change":"2023-11-08T19:28:02.710012+0000","last_active":"2023-11-08T19:28:03.716105+0000","last_peered":"2023-11-08T19:28:03.716105+0000","last_clean":"2023-11-08T19:28:03.716105+0000","last_became_active":"2023-11-08T19:28:02.709715+0000","last_became_peered":"2023-11-08T19:28:02.709715+0000","last_unstale":"2023-11-08T19:28:03.716105+0000","last_undegraded":"2023-11-08T19:28:03.716105+0000","last_fullsized":"2023-11-08T19:28:03.716105+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:18:55.364855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.704802+0000","last_change":"2023-11-08T19:27:08.183798+0000","last_active":"2023-11-08T19:28:03.704802+0000","last_peered":"2023-11-08T19:28:03.704802+0000","last_clean":"2023-11-08T19:28:03.704802+0000","last_became_active":"2023-11-08T19:27:08.183537+0000","last_became_peered":"2023-11-08T19:27:08.183537+0000","last_unstale":"2023-11-08T19:28:03.704802+0000","last_undegraded":"2023-11-08T19:28:03.704802+0000","last_fullsized":"2023-11-08T19:28:03.704802+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:50:39.026265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":72,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.700210+0000","last_change":"2023-11-08T19:26:05.610012+0000","last_active":"2023-11-08T19:28:03.700210+0000","last_peered":"2023-11-08T19:28:03.700210+0000","last_clean":"2023-11-08T19:28:03.700210+0000","last_became_active":"2023-11-08T19:26:04.603209+0000","last_became_peered":"2023-11-08T19:26:04.603209+0000","last_unstale":"2023-11-08T19:28:03.700210+0000","last_undegraded":"2023-11-08T19:28:03.700210+0000","last_fullsized":"2023-11-08T19:28:03.700210+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:35:02.306997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053335100000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.697148+0000","last_change":"2023-11-08T19:27:09.178405+0000","last_active":"2023-11-08T19:28:03.697148+0000","last_peered":"2023-11-08T19:28:03.697148+0000","last_clean":"2023-11-08T19:28:03.697148+0000","last_became_active":"2023-11-08T19:27:07.188169+0000","last_became_peered":"2023-11-08T19:27:07.188169+0000","last_unstale":"2023-11-08T19:28:03.697148+0000","last_undegraded":"2023-11-08T19:28:03.697148+0000","last_fullsized":"2023-11-08T19:28:03.697148+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:39:25.855467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056209700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:05.211269+0000","last_change":"2023-11-08T19:28:03.725822+0000","last_active":"2023-11-08T19:28:05.211269+0000","last_peered":"2023-11-08T19:28:05.211269+0000","last_clean":"2023-11-08T19:28:05.211269+0000","last_became_active":"2023-11-08T19:28:03.725537+0000","last_became_peered":"2023-11-08T19:28:03.725537+0000","last_unstale":"2023-11-08T19:28:05.211269+0000","last_undegraded":"2023-11-08T19:28:05.211269+0000","last_fullsized":"2023-11-08T19:28:05.211269+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:39:51.062357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"91'2","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.721637+0000","last_change":"2023-11-08T19:28:02.716988+0000","last_active":"2023-11-08T19:28:03.721637+0000","last_peered":"2023-11-08T19:28:03.721637+0000","last_clean":"2023-11-08T19:28:03.721637+0000","last_became_active":"2023-11-08T19:28:02.716687+0000","last_became_peered":"2023-11-08T19:28:02.716687+0000","last_unstale":"2023-11-08T19:28:03.721637+0000","last_undegraded":"2023-11-08T19:28:03.721637+0000","last_fullsized":"2023-11-08T19:28:03.721637+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:35:10.546514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.703294+0000","last_change":"2023-11-08T19:27:08.193508+0000","last_active":"2023-11-08T19:28:03.703294+0000","last_peered":"2023-11-08T19:28:03.703294+0000","last_clean":"2023-11-08T19:28:03.703294+0000","last_became_active":"2023-11-08T19:27:08.193037+0000","last_became_peered":"2023-11-08T19:27:08.193037+0000","last_unstale":"2023-11-08T19:28:03.703294+0000","last_undegraded":"2023-11-08T19:28:03.703294+0000","last_fullsized":"2023-11-08T19:28:03.703294+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:29:50.067887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":60,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.706061+0000","last_change":"2023-11-08T19:26:06.265318+0000","last_active":"2023-11-08T19:28:03.706061+0000","last_peered":"2023-11-08T19:28:03.706061+0000","last_clean":"2023-11-08T19:28:03.706061+0000","last_became_active":"2023-11-08T19:26:04.603409+0000","last_became_peered":"2023-11-08T19:26:04.603409+0000","last_unstale":"2023-11-08T19:28:03.706061+0000","last_undegraded":"2023-11-08T19:28:03.706061+0000","last_fullsized":"2023-11-08T19:28:03.706061+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:27:24.933111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065684300000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.699795+0000","last_change":"2023-11-08T19:27:09.182282+0000","last_active":"2023-11-08T19:28:03.699795+0000","last_peered":"2023-11-08T19:28:03.699795+0000","last_clean":"2023-11-08T19:28:03.699795+0000","last_became_active":"2023-11-08T19:27:07.186944+0000","last_became_peered":"2023-11-08T19:27:07.186944+0000","last_unstale":"2023-11-08T19:28:03.699795+0000","last_undegraded":"2023-11-08T19:28:03.699795+0000","last_fullsized":"2023-11-08T19:28:03.699795+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:45:21.079011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00028665599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:04.700054+0000","last_change":"2023-11-08T19:28:03.716823+0000","last_active":"2023-11-08T19:28:04.700054+0000","last_peered":"2023-11-08T19:28:04.700054+0000","last_clean":"2023-11-08T19:28:04.700054+0000","last_became_active":"2023-11-08T19:28:03.716545+0000","last_became_peered":"2023-11-08T19:28:03.716545+0000","last_unstale":"2023-11-08T19:28:04.700054+0000","last_undegraded":"2023-11-08T19:28:04.700054+0000","last_fullsized":"2023-11-08T19:28:04.700054+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:32:43.695921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.706800+0000","last_change":"2023-11-08T19:28:02.703978+0000","last_active":"2023-11-08T19:28:03.706800+0000","last_peered":"2023-11-08T19:28:03.706800+0000","last_clean":"2023-11-08T19:28:03.706800+0000","last_became_active":"2023-11-08T19:28:02.703654+0000","last_became_peered":"2023-11-08T19:28:02.703654+0000","last_unstale":"2023-11-08T19:28:03.706800+0000","last_undegraded":"2023-11-08T19:28:03.706800+0000","last_fullsized":"2023-11-08T19:28:03.706800+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:32:02.689977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.719681+0000","last_change":"2023-11-08T19:27:08.193990+0000","last_active":"2023-11-08T19:28:03.719681+0000","last_peered":"2023-11-08T19:28:03.719681+0000","last_clean":"2023-11-08T19:28:03.719681+0000","last_became_active":"2023-11-08T19:27:08.193624+0000","last_became_peered":"2023-11-08T19:27:08.193624+0000","last_unstale":"2023-11-08T19:28:03.719681+0000","last_undegraded":"2023-11-08T19:28:03.719681+0000","last_fullsized":"2023-11-08T19:28:03.719681+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:46:41.321120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":66,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.701319+0000","last_change":"2023-11-08T19:26:06.423863+0000","last_active":"2023-11-08T19:28:03.701319+0000","last_peered":"2023-11-08T19:28:03.701319+0000","last_clean":"2023-11-08T19:28:03.701319+0000","last_became_active":"2023-11-08T19:26:04.608639+0000","last_became_peered":"2023-11-08T19:26:04.608639+0000","last_unstale":"2023-11-08T19:28:03.701319+0000","last_undegraded":"2023-11-08T19:28:03.701319+0000","last_fullsized":"2023-11-08T19:28:03.701319+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:04:43.631102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044384999999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.722139+0000","last_change":"2023-11-08T19:27:09.283232+0000","last_active":"2023-11-08T19:28:03.722139+0000","last_peered":"2023-11-08T19:28:03.722139+0000","last_clean":"2023-11-08T19:28:03.722139+0000","last_became_active":"2023-11-08T19:27:07.190931+0000","last_became_peered":"2023-11-08T19:27:07.190931+0000","last_unstale":"2023-11-08T19:28:03.722139+0000","last_undegraded":"2023-11-08T19:28:03.722139+0000","last_fullsized":"2023-11-08T19:28:03.722139+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:43:28.225967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034146799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:05.210766+0000","last_change":"2023-11-08T19:28:03.720476+0000","last_active":"2023-11-08T19:28:05.210766+0000","last_peered":"2023-11-08T19:28:05.210766+0000","last_clean":"2023-11-08T19:28:05.210766+0000","last_became_active":"2023-11-08T19:28:03.719873+0000","last_became_peered":"2023-11-08T19:28:03.719873+0000","last_unstale":"2023-11-08T19:28:05.210766+0000","last_undegraded":"2023-11-08T19:28:05.210766+0000","last_fullsized":"2023-11-08T19:28:05.210766+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:47:26.978894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.696557+0000","last_change":"2023-11-08T19:28:02.717768+0000","last_active":"2023-11-08T19:28:03.696557+0000","last_peered":"2023-11-08T19:28:03.696557+0000","last_clean":"2023-11-08T19:28:03.696557+0000","last_became_active":"2023-11-08T19:28:02.717553+0000","last_became_peered":"2023-11-08T19:28:02.717553+0000","last_unstale":"2023-11-08T19:28:03.696557+0000","last_undegraded":"2023-11-08T19:28:03.696557+0000","last_fullsized":"2023-11-08T19:28:03.696557+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:01:50.394789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.707098+0000","last_change":"2023-11-08T19:27:08.187252+0000","last_active":"2023-11-08T19:28:03.707098+0000","last_peered":"2023-11-08T19:28:03.707098+0000","last_clean":"2023-11-08T19:28:03.707098+0000","last_became_active":"2023-11-08T19:27:08.186952+0000","last_became_peered":"2023-11-08T19:27:08.186952+0000","last_unstale":"2023-11-08T19:28:03.707098+0000","last_undegraded":"2023-11-08T19:28:03.707098+0000","last_fullsized":"2023-11-08T19:28:03.707098+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:12:52.022374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":60,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.703612+0000","last_change":"2023-11-08T19:26:06.394081+0000","last_active":"2023-11-08T19:28:03.703612+0000","last_peered":"2023-11-08T19:28:03.703612+0000","last_clean":"2023-11-08T19:28:03.703612+0000","last_became_active":"2023-11-08T19:26:04.602137+0000","last_became_peered":"2023-11-08T19:26:04.602137+0000","last_unstale":"2023-11-08T19:28:03.703612+0000","last_undegraded":"2023-11-08T19:28:03.703612+0000","last_fullsized":"2023-11-08T19:28:03.703612+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:05:46.951663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056998200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.698040+0000","last_change":"2023-11-08T19:27:09.178816+0000","last_active":"2023-11-08T19:28:03.698040+0000","last_peered":"2023-11-08T19:28:03.698040+0000","last_clean":"2023-11-08T19:28:03.698040+0000","last_became_active":"2023-11-08T19:27:07.192001+0000","last_became_peered":"2023-11-08T19:27:07.192001+0000","last_unstale":"2023-11-08T19:28:03.698040+0000","last_undegraded":"2023-11-08T19:28:03.698040+0000","last_fullsized":"2023-11-08T19:28:03.698040+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:04:12.481212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045359000000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:05.140134+0000","last_change":"2023-11-08T19:28:03.723644+0000","last_active":"2023-11-08T19:28:05.140134+0000","last_peered":"2023-11-08T19:28:05.140134+0000","last_clean":"2023-11-08T19:28:05.140134+0000","last_became_active":"2023-11-08T19:28:03.723370+0000","last_became_peered":"2023-11-08T19:28:03.723370+0000","last_unstale":"2023-11-08T19:28:05.140134+0000","last_undegraded":"2023-11-08T19:28:05.140134+0000","last_fullsized":"2023-11-08T19:28:05.140134+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:20:01.880569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"91'8","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.725981+0000","last_change":"2023-11-08T19:28:02.711841+0000","last_active":"2023-11-08T19:28:03.725981+0000","last_peered":"2023-11-08T19:28:03.725981+0000","last_clean":"2023-11-08T19:28:03.725981+0000","last_became_active":"2023-11-08T19:28:02.711455+0000","last_became_peered":"2023-11-08T19:28:02.711455+0000","last_unstale":"2023-11-08T19:28:03.725981+0000","last_undegraded":"2023-11-08T19:28:03.725981+0000","last_fullsized":"2023-11-08T19:28:03.725981+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:35:10.546514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.706187+0000","last_change":"2023-11-08T19:27:08.193836+0000","last_active":"2023-11-08T19:28:03.706187+0000","last_peered":"2023-11-08T19:28:03.706187+0000","last_clean":"2023-11-08T19:28:03.706187+0000","last_became_active":"2023-11-08T19:27:08.193458+0000","last_became_peered":"2023-11-08T19:27:08.193458+0000","last_unstale":"2023-11-08T19:28:03.706187+0000","last_undegraded":"2023-11-08T19:28:03.706187+0000","last_fullsized":"2023-11-08T19:28:03.706187+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:14:18.610061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":62,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.719593+0000","last_change":"2023-11-08T19:26:06.541338+0000","last_active":"2023-11-08T19:28:03.719593+0000","last_peered":"2023-11-08T19:28:03.719593+0000","last_clean":"2023-11-08T19:28:03.719593+0000","last_became_active":"2023-11-08T19:26:04.602183+0000","last_became_peered":"2023-11-08T19:26:04.602183+0000","last_unstale":"2023-11-08T19:28:03.719593+0000","last_undegraded":"2023-11-08T19:28:03.719593+0000","last_fullsized":"2023-11-08T19:28:03.719593+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:06:47.446686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052029099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.701852+0000","last_change":"2023-11-08T19:27:09.181622+0000","last_active":"2023-11-08T19:28:03.701852+0000","last_peered":"2023-11-08T19:28:03.701852+0000","last_clean":"2023-11-08T19:28:03.701852+0000","last_became_active":"2023-11-08T19:27:07.184507+0000","last_became_peered":"2023-11-08T19:27:07.184507+0000","last_unstale":"2023-11-08T19:28:03.701852+0000","last_undegraded":"2023-11-08T19:28:03.701852+0000","last_fullsized":"2023-11-08T19:28:03.701852+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:58:48.357266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051924700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:05.139464+0000","last_change":"2023-11-08T19:28:03.714548+0000","last_active":"2023-11-08T19:28:05.139464+0000","last_peered":"2023-11-08T19:28:05.139464+0000","last_clean":"2023-11-08T19:28:05.139464+0000","last_became_active":"2023-11-08T19:28:03.714261+0000","last_became_peered":"2023-11-08T19:28:03.714261+0000","last_unstale":"2023-11-08T19:28:05.139464+0000","last_undegraded":"2023-11-08T19:28:05.139464+0000","last_fullsized":"2023-11-08T19:28:05.139464+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:22:51.628132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.704944+0000","last_change":"2023-11-08T19:28:02.713407+0000","last_active":"2023-11-08T19:28:03.704944+0000","last_peered":"2023-11-08T19:28:03.704944+0000","last_clean":"2023-11-08T19:28:03.704944+0000","last_became_active":"2023-11-08T19:28:02.713144+0000","last_became_peered":"2023-11-08T19:28:02.713144+0000","last_unstale":"2023-11-08T19:28:03.704944+0000","last_undegraded":"2023-11-08T19:28:03.704944+0000","last_fullsized":"2023-11-08T19:28:03.704944+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:33:28.947459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.721968+0000","last_change":"2023-11-08T19:27:08.194263+0000","last_active":"2023-11-08T19:28:03.721968+0000","last_peered":"2023-11-08T19:28:03.721968+0000","last_clean":"2023-11-08T19:28:03.721968+0000","last_became_active":"2023-11-08T19:27:08.193938+0000","last_became_peered":"2023-11-08T19:27:08.193938+0000","last_unstale":"2023-11-08T19:28:03.721968+0000","last_undegraded":"2023-11-08T19:28:03.721968+0000","last_fullsized":"2023-11-08T19:28:03.721968+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:40:49.009717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":61,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.708586+0000","last_change":"2023-11-08T19:26:06.265690+0000","last_active":"2023-11-08T19:28:03.708586+0000","last_peered":"2023-11-08T19:28:03.708586+0000","last_clean":"2023-11-08T19:28:03.708586+0000","last_became_active":"2023-11-08T19:26:04.603397+0000","last_became_peered":"2023-11-08T19:26:04.603397+0000","last_unstale":"2023-11-08T19:28:03.708586+0000","last_undegraded":"2023-11-08T19:28:03.708586+0000","last_fullsized":"2023-11-08T19:28:03.708586+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:50:43.827570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053361499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.721780+0000","last_change":"2023-11-08T19:27:09.282750+0000","last_active":"2023-11-08T19:28:03.721780+0000","last_peered":"2023-11-08T19:28:03.721780+0000","last_clean":"2023-11-08T19:28:03.721780+0000","last_became_active":"2023-11-08T19:27:07.189943+0000","last_became_peered":"2023-11-08T19:27:07.189943+0000","last_unstale":"2023-11-08T19:28:03.721780+0000","last_undegraded":"2023-11-08T19:28:03.721780+0000","last_fullsized":"2023-11-08T19:28:03.721780+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:31:16.081058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000398725,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:04.697522+0000","last_change":"2023-11-08T19:28:03.717608+0000","last_active":"2023-11-08T19:28:04.697522+0000","last_peered":"2023-11-08T19:28:04.697522+0000","last_clean":"2023-11-08T19:28:04.697522+0000","last_became_active":"2023-11-08T19:28:03.717368+0000","last_became_peered":"2023-11-08T19:28:03.717368+0000","last_unstale":"2023-11-08T19:28:04.697522+0000","last_undegraded":"2023-11-08T19:28:04.697522+0000","last_fullsized":"2023-11-08T19:28:04.697522+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:00:11.333972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.704033+0000","last_change":"2023-11-08T19:28:02.715521+0000","last_active":"2023-11-08T19:28:03.704033+0000","last_peered":"2023-11-08T19:28:03.704033+0000","last_clean":"2023-11-08T19:28:03.704033+0000","last_became_active":"2023-11-08T19:28:02.715145+0000","last_became_peered":"2023-11-08T19:28:02.715145+0000","last_unstale":"2023-11-08T19:28:03.704033+0000","last_undegraded":"2023-11-08T19:28:03.704033+0000","last_fullsized":"2023-11-08T19:28:03.704033+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:24:20.895706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.703685+0000","last_change":"2023-11-08T19:27:08.188147+0000","last_active":"2023-11-08T19:28:03.703685+0000","last_peered":"2023-11-08T19:28:03.703685+0000","last_clean":"2023-11-08T19:28:03.703685+0000","last_became_active":"2023-11-08T19:27:08.187487+0000","last_became_peered":"2023-11-08T19:27:08.187487+0000","last_unstale":"2023-11-08T19:28:03.703685+0000","last_undegraded":"2023-11-08T19:28:03.703685+0000","last_fullsized":"2023-11-08T19:28:03.703685+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:22:40.300185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":67,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:05.211105+0000","last_change":"2023-11-08T19:26:06.425133+0000","last_active":"2023-11-08T19:28:05.211105+0000","last_peered":"2023-11-08T19:28:05.211105+0000","last_clean":"2023-11-08T19:28:05.211105+0000","last_became_active":"2023-11-08T19:26:04.601103+0000","last_became_peered":"2023-11-08T19:26:04.601103+0000","last_unstale":"2023-11-08T19:28:05.211105+0000","last_undegraded":"2023-11-08T19:28:05.211105+0000","last_fullsized":"2023-11-08T19:28:05.211105+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:31:14.042561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000229244,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.699502+0000","last_change":"2023-11-08T19:27:09.179852+0000","last_active":"2023-11-08T19:28:03.699502+0000","last_peered":"2023-11-08T19:28:03.699502+0000","last_clean":"2023-11-08T19:28:03.699502+0000","last_became_active":"2023-11-08T19:27:07.191926+0000","last_became_peered":"2023-11-08T19:27:07.191926+0000","last_unstale":"2023-11-08T19:28:03.699502+0000","last_undegraded":"2023-11-08T19:28:03.699502+0000","last_fullsized":"2023-11-08T19:28:03.699502+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:20:38.080129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055457100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:05.211295+0000","last_change":"2023-11-08T19:28:03.708655+0000","last_active":"2023-11-08T19:28:05.211295+0000","last_peered":"2023-11-08T19:28:05.211295+0000","last_clean":"2023-11-08T19:28:05.211295+0000","last_became_active":"2023-11-08T19:28:03.708233+0000","last_became_peered":"2023-11-08T19:28:03.708233+0000","last_unstale":"2023-11-08T19:28:05.211295+0000","last_undegraded":"2023-11-08T19:28:05.211295+0000","last_fullsized":"2023-11-08T19:28:05.211295+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:52:25.773321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"91'3","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:04.596910+0000","last_change":"2023-11-08T19:28:02.708691+0000","last_active":"2023-11-08T19:28:04.596910+0000","last_peered":"2023-11-08T19:28:04.596910+0000","last_clean":"2023-11-08T19:28:04.596910+0000","last_became_active":"2023-11-08T19:28:02.708037+0000","last_became_peered":"2023-11-08T19:28:02.708037+0000","last_unstale":"2023-11-08T19:28:04.596910+0000","last_undegraded":"2023-11-08T19:28:04.596910+0000","last_fullsized":"2023-11-08T19:28:04.596910+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:07:33.241500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.702669+0000","last_change":"2023-11-08T19:27:08.186054+0000","last_active":"2023-11-08T19:28:03.702669+0000","last_peered":"2023-11-08T19:28:03.702669+0000","last_clean":"2023-11-08T19:28:03.702669+0000","last_became_active":"2023-11-08T19:27:08.185581+0000","last_became_peered":"2023-11-08T19:27:08.185581+0000","last_unstale":"2023-11-08T19:28:03.702669+0000","last_undegraded":"2023-11-08T19:28:03.702669+0000","last_fullsized":"2023-11-08T19:28:03.702669+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:00:55.878397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":72,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.699946+0000","last_change":"2023-11-08T19:26:05.609625+0000","last_active":"2023-11-08T19:28:03.699946+0000","last_peered":"2023-11-08T19:28:03.699946+0000","last_clean":"2023-11-08T19:28:03.699946+0000","last_became_active":"2023-11-08T19:26:04.606312+0000","last_became_peered":"2023-11-08T19:26:04.606312+0000","last_unstale":"2023-11-08T19:28:03.699946+0000","last_undegraded":"2023-11-08T19:28:03.699946+0000","last_fullsized":"2023-11-08T19:28:03.699946+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:54:42.032901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000505935,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.696724+0000","last_change":"2023-11-08T19:27:09.177985+0000","last_active":"2023-11-08T19:28:03.696724+0000","last_peered":"2023-11-08T19:28:03.696724+0000","last_clean":"2023-11-08T19:28:03.696724+0000","last_became_active":"2023-11-08T19:27:07.193254+0000","last_became_peered":"2023-11-08T19:27:07.193254+0000","last_unstale":"2023-11-08T19:28:03.696724+0000","last_undegraded":"2023-11-08T19:28:03.696724+0000","last_fullsized":"2023-11-08T19:28:03.696724+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:23:38.662629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041830799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:04.697999+0000","last_change":"2023-11-08T19:28:03.717201+0000","last_active":"2023-11-08T19:28:04.697999+0000","last_peered":"2023-11-08T19:28:04.697999+0000","last_clean":"2023-11-08T19:28:04.697999+0000","last_became_active":"2023-11-08T19:28:03.716956+0000","last_became_peered":"2023-11-08T19:28:03.716956+0000","last_unstale":"2023-11-08T19:28:04.697999+0000","last_undegraded":"2023-11-08T19:28:04.697999+0000","last_fullsized":"2023-11-08T19:28:04.697999+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:02.688498+0000","last_clean_scrub_stamp":"2023-11-08T19:28:02.688498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:03:01.222287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"91'6","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:04.596943+0000","last_change":"2023-11-08T19:28:02.700819+0000","last_active":"2023-11-08T19:28:04.596943+0000","last_peered":"2023-11-08T19:28:04.596943+0000","last_clean":"2023-11-08T19:28:04.596943+0000","last_became_active":"2023-11-08T19:28:02.700537+0000","last_became_peered":"2023-11-08T19:28:02.700537+0000","last_unstale":"2023-11-08T19:28:04.596943+0000","last_undegraded":"2023-11-08T19:28:04.596943+0000","last_fullsized":"2023-11-08T19:28:04.596943+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:01.683677+0000","last_clean_scrub_stamp":"2023-11-08T19:28:01.683677+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:25:00.088013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.705133+0000","last_change":"2023-11-08T19:27:08.188860+0000","last_active":"2023-11-08T19:28:03.705133+0000","last_peered":"2023-11-08T19:28:03.705133+0000","last_clean":"2023-11-08T19:28:03.705133+0000","last_became_active":"2023-11-08T19:27:08.188560+0000","last_became_peered":"2023-11-08T19:27:08.188560+0000","last_unstale":"2023-11-08T19:28:03.705133+0000","last_undegraded":"2023-11-08T19:28:03.705133+0000","last_fullsized":"2023-11-08T19:28:03.705133+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:18:20.969491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":61,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.705484+0000","last_change":"2023-11-08T19:26:06.297255+0000","last_active":"2023-11-08T19:28:03.705484+0000","last_peered":"2023-11-08T19:28:03.705484+0000","last_clean":"2023-11-08T19:28:03.705484+0000","last_became_active":"2023-11-08T19:26:04.603979+0000","last_became_peered":"2023-11-08T19:26:04.603979+0000","last_unstale":"2023-11-08T19:28:03.705484+0000","last_undegraded":"2023-11-08T19:28:03.705484+0000","last_fullsized":"2023-11-08T19:28:03.705484+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:40:11.164951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079032099999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-08T19:28:03.703377+0000","last_change":"2023-11-08T19:27:09.331990+0000","last_active":"2023-11-08T19:28:03.703377+0000","last_peered":"2023-11-08T19:28:03.703377+0000","last_clean":"2023-11-08T19:28:03.703377+0000","last_became_active":"2023-11-08T19:27:07.185478+0000","last_became_peered":"2023-11-08T19:27:07.185478+0000","last_unstale":"2023-11-08T19:28:03.703377+0000","last_undegraded":"2023-11-08T19:28:03.703377+0000","last_fullsized":"2023-11-08T19:28:03.703377+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:20:38.080129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010661850000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574893,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28720,"kb_used_data":1692,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343120,"statfs":{"total":96636764160,"available":96607354880,"internally_reserved":0,"allocated":1732608,"data_stored":1124900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:27:53 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.50600000000000001,"15min":0.498},"min":{"1min":0.34300000000000003,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.72899999999999998,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.443},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.51900000000000002,"15min":0.51500000000000001},"min":{"1min":0.36899999999999999,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.69099999999999995,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.58399999999999996}]},{"osd":1,"last update":"Wed Nov 8 19:27:53 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56899999999999995,"15min":0.56599999999999995},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.56399999999999995,"15min":0.55600000000000005},"min":{"1min":0.435,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.53500000000000003}]},{"osd":2,"last update":"Wed Nov 8 19:27:53 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.50900000000000001,"15min":0.49299999999999999},"min":{"1min":0.38700000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.55000000000000004,"15min":0.54800000000000004},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.50600000000000001}]},{"osd":3,"last update":"Wed Nov 8 19:27:53 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.55300000000000005,"15min":0.55700000000000005},"min":{"1min":0.375,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.66200000000000003,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59399999999999997,"15min":0.57799999999999996},"min":{"1min":0.49399999999999999,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.77000000000000002,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.55900000000000005}]},{"osd":4,"last update":"Wed Nov 8 19:27:53 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.50900000000000001,"15min":0.50700000000000001},"min":{"1min":0.36299999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.64500000000000002,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.54600000000000004,"15min":0.54100000000000004},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.45800000000000002}]},{"osd":5,"last update":"Wed Nov 8 19:27:53 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.58599999999999997,"15min":0.58099999999999996},"min":{"1min":0.41199999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.71999999999999997,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.38},{"interface":"front","average":{"1min":0.62,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.84499999999999997,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.60899999999999999}]},{"osd":6,"last update":"Wed Nov 8 19:27:53 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.60199999999999998,"15min":0.60399999999999998},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.78400000000000003,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.61499999999999999,"15min":0.61099999999999999},"min":{"1min":0.5,"5min":0.443,"15min":0.443},"max":{"1min":0.82999999999999996,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.52000000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574893,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28148,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343692,"statfs":{"total":96636764160,"available":96607940608,"internally_reserved":0,"allocated":1150976,"data_stored":536588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":1.0780000000000001,"15min":1.319},"min":{"1min":0.33800000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.67800000000000005,"5min":21.359999999999999,"15min":21.359999999999999},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.78300000000000003,"15min":0.88700000000000001},"min":{"1min":0.378,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.73799999999999999,"5min":10.329000000000001,"15min":10.329000000000001},"last":0.46999999999999997}]},{"osd":1,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.57899999999999996,"15min":0.60899999999999999},"min":{"1min":0.36899999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.71899999999999997,"5min":3.4239999999999999,"15min":3.4239999999999999},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.61699999999999999,"15min":0.63800000000000001},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.73199999999999998,"5min":3.4809999999999999,"15min":3.4809999999999999},"last":0.58699999999999997}]},{"osd":2,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.80100000000000005,"15min":0.90500000000000003},"min":{"1min":0.40999999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.69299999999999995,"5min":10.398,"15min":10.398},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.65400000000000003,"15min":0.68999999999999995},"min":{"1min":0.40799999999999997,"5min":0.318,"15min":0.318},"max":{"1min":0.746,"5min":5.0170000000000003,"15min":5.0170000000000003},"last":0.48999999999999999}]},{"osd":3,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.63900000000000001,"15min":0.66300000000000003},"min":{"1min":0.39500000000000002,"5min":0.317,"15min":0.317},"max":{"1min":0.77300000000000002,"5min":3.5710000000000002,"15min":3.5710000000000002},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.82999999999999996,"15min":0.93799999999999994},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.86799999999999999,"5min":10.368,"15min":10.368},"last":0.56299999999999994}]},{"osd":4,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.53700000000000003,"15min":0.51700000000000002},"min":{"1min":0.38,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.56999999999999995,"15min":0.56200000000000006},"min":{"1min":0.40600000000000003,"5min":0.373,"15min":0.373},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.629}]},{"osd":5,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.55000000000000004,"15min":0.53200000000000003},"min":{"1min":0.40699999999999997,"5min":0.307,"15min":0.307},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.60599999999999998,"15min":0.58599999999999997},"min":{"1min":0.42899999999999999,"5min":0.378,"15min":0.378},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.64100000000000001}]},{"osd":7,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.59899999999999998,"15min":0.57899999999999996},"min":{"1min":0.41299999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.81200000000000006,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.61099999999999999,"15min":0.59399999999999997},"min":{"1min":0.54000000000000004,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.61299999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574893,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28132,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343708,"statfs":{"total":96636764160,"available":96607956992,"internally_reserved":0,"allocated":1134592,"data_stored":534406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:27:47 2023","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.36899999999999999,"15min":0.379},"min":{"1min":0.22800000000000001,"5min":0.219,"15min":0.219},"max":{"1min":0.52400000000000002,"5min":1.389,"15min":1.389},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.33800000000000002,"5min":0.58299999999999996,"15min":0.68999999999999995},"min":{"1min":0.25900000000000001,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.504,"5min":9.1370000000000005,"15min":9.1370000000000005},"last":0.443}]},{"osd":2,"last update":"Wed Nov 8 19:27:47 2023","interfaces":[{"interface":"back","average":{"1min":0.33600000000000002,"5min":0.35399999999999998,"15min":0.36299999999999999},"min":{"1min":0.254,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.50600000000000001,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.373,"5min":0.41099999999999998,"15min":0.42499999999999999},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.58299999999999996,"5min":1.425,"15min":1.425},"last":0.45800000000000002}]},{"osd":3,"last update":"Wed Nov 8 19:27:47 2023","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.69999999999999996,"15min":0.84099999999999997},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.54600000000000004,"5min":11.957000000000001,"15min":11.957000000000001},"last":0.504},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.85799999999999998,"15min":1.0740000000000001},"min":{"1min":0.27700000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.56299999999999994,"5min":18.039000000000001,"15min":18.039000000000001},"last":0.52300000000000002}]},{"osd":4,"last update":"Wed Nov 8 19:27:47 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.73399999999999999,"15min":0.79800000000000004},"min":{"1min":0.43099999999999999,"5min":0.372,"15min":0.372},"max":{"1min":0.77800000000000002,"5min":6.181,"15min":6.181},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.629,"15min":0.63700000000000001},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.76900000000000002,"5min":1.488,"15min":1.488},"last":0.77000000000000002}]},{"osd":5,"last update":"Wed Nov 8 19:27:47 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.84099999999999997,"15min":0.93700000000000006},"min":{"1min":0.437,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.74199999999999999,"5min":9.2420000000000009,"15min":9.2420000000000009},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.77200000000000002,"15min":0.84399999999999997},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.745,"5min":6.2110000000000003,"15min":6.2110000000000003},"last":0.71199999999999997}]},{"osd":6,"last update":"Wed Nov 8 19:27:47 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.84899999999999998,"15min":0.94099999999999995},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.751,"5min":9.1880000000000006,"15min":9.1880000000000006},"last":0.749},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.65500000000000003,"15min":0.66300000000000003},"min":{"1min":0.38800000000000001,"5min":0.38,"15min":0.38},"max":{"1min":0.79400000000000004,"5min":1.462,"15min":1.462},"last":0.86799999999999999}]},{"osd":7,"last update":"Wed Nov 8 19:27:47 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.77400000000000002,"15min":0.83799999999999997},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.76600000000000001,"5min":6.1479999999999997,"15min":6.1479999999999997},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.872,"15min":0.97599999999999998},"min":{"1min":0.46100000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.76400000000000001,"5min":9.2170000000000005,"15min":9.2170000000000005},"last":0.85799999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574893,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28720,"kb_used_data":1700,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343120,"statfs":{"total":96636764160,"available":96607354880,"internally_reserved":0,"allocated":1740800,"data_stored":1126442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Nov 8 19:27:50 2023","interfaces":[{"interface":"back","average":{"1min":0.32100000000000001,"5min":0.441,"15min":0.495},"min":{"1min":0.23599999999999999,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.48799999999999999,"5min":4.1200000000000001,"15min":4.1200000000000001},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.32300000000000001,"5min":0.85199999999999998,"15min":1.079},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.51200000000000001,"5min":16.940999999999999,"15min":16.940999999999999},"last":0.41699999999999998}]},{"osd":2,"last update":"Wed Nov 8 19:27:50 2023","interfaces":[{"interface":"back","average":{"1min":0.318,"5min":0.374,"15min":0.39400000000000002},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.45600000000000002,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.57499999999999996,"15min":0.66600000000000004},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.58299999999999996,"5min":7.0209999999999999,"15min":7.0209999999999999},"last":0.39300000000000002}]},{"osd":3,"last update":"Wed Nov 8 19:27:50 2023","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.49299999999999999,"15min":0.54800000000000004},"min":{"1min":0.25,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.56699999999999995,"5min":4.0940000000000003,"15min":4.0940000000000003},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.37,"5min":0.42299999999999999,"15min":0.442},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.53600000000000003,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.375}]},{"osd":4,"last update":"Wed Nov 8 19:27:50 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.69299999999999995,"15min":0.751},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.76800000000000002,"5min":4.0049999999999999,"15min":4.0049999999999999},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.62,"15min":0.63500000000000001},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.77500000000000002,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.624}]},{"osd":5,"last update":"Wed Nov 8 19:27:50 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.71199999999999997,"15min":0.76600000000000001},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.79200000000000004,"5min":4.3090000000000002,"15min":4.3090000000000002},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.70899999999999996,"15min":0.75700000000000001},"min":{"1min":0.45600000000000002,"5min":0.437,"15min":0.437},"max":{"1min":0.84699999999999998,"5min":4.0709999999999997,"15min":4.0709999999999997},"last":0.77800000000000002}]},{"osd":6,"last update":"Wed Nov 8 19:27:50 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.65500000000000003,"15min":0.66900000000000004},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.81200000000000006,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.76800000000000002,"15min":0.82399999999999995},"min":{"1min":0.47699999999999998,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.84199999999999997,"5min":4.2560000000000002,"15min":4.2560000000000002},"last":0.76700000000000002}]},{"osd":7,"last update":"Wed Nov 8 19:27:50 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.72699999999999998,"15min":0.77700000000000002},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.78700000000000003,"5min":4.2850000000000001,"15min":4.2850000000000001},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.753,"15min":0.80500000000000005},"min":{"1min":0.48699999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.82499999999999996,"5min":4.0460000000000003,"15min":4.0460000000000003},"last":0.72299999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574893,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28132,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343708,"statfs":{"total":96636764160,"available":96607956992,"internally_reserved":0,"allocated":1134592,"data_stored":534406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:27:47 2023","interfaces":[{"interface":"back","average":{"1min":0.32100000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.47299999999999998,"5min":0.496,"15min":0.496},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.31,"5min":0.51900000000000002,"15min":0.61299999999999999},"min":{"1min":0.219,"5min":0.218,"15min":0.218},"max":{"1min":0.501,"5min":6.0999999999999996,"15min":6.0999999999999996},"last":0.29699999999999999}]},{"osd":1,"last update":"Wed Nov 8 19:27:47 2023","interfaces":[{"interface":"back","average":{"1min":0.309,"5min":0.33400000000000002,"15min":0.33800000000000002},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.42699999999999999,"5min":0.48299999999999998,"15min":0.48299999999999998},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.32500000000000001,"5min":0.372,"15min":0.38400000000000001},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.41899999999999998,"5min":0.624,"15min":0.624},"last":0.25800000000000001}]},{"osd":3,"last update":"Wed Nov 8 19:27:47 2023","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.376,"15min":0.38200000000000001},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.46500000000000002,"5min":0.53700000000000003,"15min":0.53700000000000003},"last":0.378},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.38300000000000001,"15min":0.39200000000000002},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.53400000000000003,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.35899999999999999}]},{"osd":4,"last update":"Wed Nov 8 19:27:47 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.56699999999999995,"15min":0.57899999999999996},"min":{"1min":0.39000000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.57699999999999996,"15min":0.58199999999999996},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.77100000000000002,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.45700000000000002}]},{"osd":5,"last update":"Wed Nov 8 19:27:47 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.59199999999999997,"15min":0.58499999999999996},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.78100000000000003,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.502},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.60499999999999998,"15min":0.61099999999999999},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.76400000000000001,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.55600000000000005}]},{"osd":6,"last update":"Wed Nov 8 19:27:47 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59799999999999998,"15min":0.59999999999999998},"min":{"1min":0.47199999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.79700000000000004,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57799999999999996,"15min":0.57899999999999996},"min":{"1min":0.41499999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.80000000000000004,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.48099999999999998}]},{"osd":7,"last update":"Wed Nov 8 19:27:47 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.58199999999999996,"15min":0.57999999999999996},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.73199999999999998,"5min":0.878,"15min":0.878},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60099999999999998,"15min":0.60299999999999998},"min":{"1min":0.45300000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.82199999999999995,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.53200000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574894,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28132,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343708,"statfs":{"total":96636764160,"available":96607956992,"internally_reserved":0,"allocated":1134592,"data_stored":534954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.318,"5min":0.33300000000000002,"15min":0.33900000000000002},"min":{"1min":0.224,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.49099999999999999,"5min":0.626,"15min":0.626},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.34999999999999998,"15min":0.34899999999999998},"min":{"1min":0.28000000000000003,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.48699999999999999,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.23699999999999999}]},{"osd":1,"last update":"Wed Nov 8 19:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.34699999999999998,"15min":0.35199999999999998},"min":{"1min":0.25800000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.42299999999999999,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":0.26600000000000001},{"interface":"front","average":{"1min":0.38,"5min":0.38300000000000001,"15min":0.38400000000000001},"min":{"1min":0.28000000000000003,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.48699999999999999,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.33000000000000002}]},{"osd":2,"last update":"Wed Nov 8 19:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.376,"15min":0.372},"min":{"1min":0.32900000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.46300000000000002,"5min":0.58499999999999996,"15min":0.58499999999999996},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.40400000000000003,"15min":0.40400000000000003},"min":{"1min":0.32700000000000001,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.47399999999999998,"5min":0.58499999999999996,"15min":0.58499999999999996},"last":0.28100000000000003}]},{"osd":4,"last update":"Wed Nov 8 19:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.60799999999999998,"15min":0.60099999999999998},"min":{"1min":0.45300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.624,"5min":0.61199999999999999,"15min":0.60699999999999998},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.84499999999999997,"5min":0.871,"15min":0.871},"last":0.34799999999999998}]},{"osd":5,"last update":"Wed Nov 8 19:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.58699999999999997,"15min":0.57899999999999996},"min":{"1min":0.46700000000000003,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.76600000000000001,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.58899999999999997,"15min":0.58099999999999996},"min":{"1min":0.42399999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.78600000000000003,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.47799999999999998}]},{"osd":6,"last update":"Wed Nov 8 19:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.58799999999999997,"15min":0.58199999999999996},"min":{"1min":0.436,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.73299999999999998,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.44},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.61299999999999999,"15min":0.60699999999999998},"min":{"1min":0.49099999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.75900000000000001,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.51300000000000001}]},{"osd":7,"last update":"Wed Nov 8 19:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.59899999999999998,"15min":0.59099999999999997},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.72199999999999998,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.60199999999999998,"15min":0.59799999999999998},"min":{"1min":0.45800000000000002,"5min":0.432,"15min":0.432},"max":{"1min":0.80700000000000005,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.499}]}]},{"osd":4,"up_from":13,"seq":55834574893,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28136,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343704,"statfs":{"total":96636764160,"available":96607952896,"internally_reserved":0,"allocated":1138688,"data_stored":534464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:27:55 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.68200000000000005,"15min":0.74399999999999999},"min":{"1min":0.35699999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.72099999999999997,"5min":6.5970000000000004,"15min":6.5970000000000004},"last":0.432},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.55500000000000005,"15min":0.56999999999999995},"min":{"1min":0.35799999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.70999999999999996,"5min":2.5030000000000001,"15min":2.5030000000000001},"last":0.34899999999999998}]},{"osd":1,"last update":"Wed Nov 8 19:27:55 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.51300000000000001,"15min":0.501},"min":{"1min":0.36199999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.75800000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.52100000000000002,"15min":0.51200000000000001},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.48599999999999999}]},{"osd":2,"last update":"Wed Nov 8 19:27:55 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.55400000000000005,"15min":0.55200000000000005},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.80600000000000005,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.748,"15min":0.81299999999999994},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.86199999999999999,"5min":6.6920000000000002,"15min":6.6920000000000002},"last":0.51600000000000001}]},{"osd":3,"last update":"Wed Nov 8 19:27:55 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.58299999999999996,"15min":0.58699999999999997},"min":{"1min":0.378,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.79800000000000004,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.60899999999999999,"15min":0.60999999999999999},"min":{"1min":0.42499999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.71399999999999997,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.60399999999999998}]},{"osd":5,"last update":"Wed Nov 8 19:27:55 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.55000000000000004,"15min":0.54800000000000004},"min":{"1min":0.35099999999999998,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.81599999999999995,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.58499999999999996,"15min":0.59299999999999997},"min":{"1min":0.39400000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.745,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.498}]},{"osd":6,"last update":"Wed Nov 8 19:27:55 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58599999999999997,"15min":0.58499999999999996},"min":{"1min":0.373,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.627,"15min":0.61799999999999999},"min":{"1min":0.46400000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.53900000000000003}]},{"osd":7,"last update":"Wed Nov 8 19:27:55 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.61599999999999999,"15min":0.629},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.84499999999999997,"5min":1.141,"15min":1.141},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.64100000000000001,"15min":0.64600000000000002},"min":{"1min":0.42399999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.86399999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.64300000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574893,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28128,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343712,"statfs":{"total":96636764160,"available":96607961088,"internally_reserved":0,"allocated":1130496,"data_stored":534372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:27:46 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.51400000000000001,"15min":0.52000000000000002},"min":{"1min":0.32200000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.73999999999999999,"5min":0.753,"15min":0.753},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.51900000000000002,"15min":0.51200000000000001},"min":{"1min":0.38500000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.69399999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.62}]},{"osd":1,"last update":"Wed Nov 8 19:27:46 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.51900000000000002,"15min":0.52200000000000002},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.70799999999999996,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.53400000000000003,"15min":0.53600000000000003},"min":{"1min":0.38400000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.70499999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.46500000000000002}]},{"osd":2,"last update":"Wed Nov 8 19:27:46 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.53600000000000003,"15min":0.53500000000000003},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.68000000000000005,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.498},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.55800000000000005,"15min":0.56999999999999995},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.69799999999999995,"5min":0.747,"15min":0.747},"last":0.57299999999999995}]},{"osd":3,"last update":"Wed Nov 8 19:27:46 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.54700000000000004,"15min":0.54900000000000004},"min":{"1min":0.40899999999999997,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.72199999999999998,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.57899999999999996,"15min":0.59599999999999997},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.69099999999999995,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.52400000000000002}]},{"osd":4,"last update":"Wed Nov 8 19:27:46 2023","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.53200000000000003,"15min":0.55200000000000005},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.67200000000000004,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.56200000000000006,"15min":0.56899999999999995},"min":{"1min":0.36599999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.44400000000000001}]},{"osd":6,"last update":"Wed Nov 8 19:27:46 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.59399999999999997,"15min":0.60699999999999998},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.755,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.63},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.59799999999999998,"15min":0.61099999999999999},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.72099999999999997,"5min":0.745,"15min":0.745},"last":0.56000000000000005}]},{"osd":7,"last update":"Wed Nov 8 19:27:46 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.59399999999999997,"15min":0.60599999999999998},"min":{"1min":0.44500000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.74199999999999999,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.625,"15min":0.64200000000000002},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.752,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.47999999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-08T19:28:12.857 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-08T19:28:12.857 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-08T19:28:12.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.984+0000 7f1cd8304700 1 Processor -- start 2023-11-08T19:28:12.986 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.984+0000 7f1cd8304700 1 -- start start 2023-11-08T19:28:12.988 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.985+0000 7f1cd8304700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1cd00b22e0 0x7f1cd00b2750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:12.988 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.985+0000 7f1cd8304700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cd0161890 con 0x7f1cd015dba0 2023-11-08T19:28:12.988 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.985+0000 7f1cd8304700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cd006b740 con 0x7f1cd006b240 2023-11-08T19:28:12.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.985+0000 7f1cd8304700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd006cd20 con 0x7f1cd00b22e0 2023-11-08T19:28:12.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.985+0000 7f1cd60a0700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f1cd015dba0 0x7f1cd006c610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:50872/0 (socket says 172.21.15.143:50872) 2023-11-08T19:28:12.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.985+0000 7f1cd60a0700 1 -- 172.21.15.143:0/576875420 learned_addr learned my addr 172.21.15.143:0/576875420 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:28:12.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.985+0000 7f1cd68a1700 1 --2- 172.21.15.143:0/576875420 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1cd00b22e0 0x7f1cd00b2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:12.990 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.985+0000 7f1cd509e700 1 -- 172.21.15.143:0/576875420 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 486101403 0 0) 0x7f1cd0161890 con 0x7f1cd015dba0 2023-11-08T19:28:12.990 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.985+0000 7f1cd509e700 1 -- 172.21.15.143:0/576875420 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1cbc0035b0 con 0x7f1cd015dba0 2023-11-08T19:28:12.990 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.985+0000 7f1cd509e700 1 -- 172.21.15.143:0/576875420 <== mon.1 v1:172.21.15.174:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4247584281 0 0) 0x7f1cd006b740 con 0x7f1cd006b240 2023-11-08T19:28:12.990 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.985+0000 7f1cd509e700 1 -- 172.21.15.143:0/576875420 --> v1:172.21.15.174:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1cd0161890 con 0x7f1cd006b240 2023-11-08T19:28:12.990 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.986+0000 7f1cd68a1700 1 -- 172.21.15.143:0/576875420 >> v1:172.21.15.174:6789/0 conn(0x7f1cd006b240 legacy=0x7f1cd00caff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:12.991 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.986+0000 7f1cd68a1700 1 -- 172.21.15.143:0/576875420 >> v1:172.21.15.143:6789/0 conn(0x7f1cd015dba0 legacy=0x7f1cd006c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:12.991 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.986+0000 7f1cd509e700 1 -- 172.21.15.143:0/576875420 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 960659752 0 0) 0x7f1cbc0035b0 con 0x7f1cd015dba0 2023-11-08T19:28:12.991 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.986+0000 7f1cd68a1700 1 -- 172.21.15.143:0/576875420 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cd006ba20 con 0x7f1cd00b22e0 2023-11-08T19:28:12.991 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.986+0000 7f1cd68a1700 1 --2- 172.21.15.143:0/576875420 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1cd00b22e0 0x7f1cd00b2750 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1cc8009dc0 tx=0x7f1cc800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20392d8e92d7fcc server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:12.991 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.986+0000 7f1cd509e700 1 -- 172.21.15.143:0/576875420 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cc800e070 con 0x7f1cd00b22e0 2023-11-08T19:28:12.992 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.986+0000 7f1cd509e700 1 -- 172.21.15.143:0/576875420 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cc8003a20 con 0x7f1cd00b22e0 2023-11-08T19:28:12.992 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.986+0000 7f1cd509e700 1 -- 172.21.15.143:0/576875420 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cc0001670 con 0x7f1cd00b22e0 2023-11-08T19:28:12.992 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.986+0000 7f1cd8304700 1 -- 172.21.15.143:0/576875420 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1cd00b22e0 msgr2=0x7f1cd00b2750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:12.992 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.986+0000 7f1cd8304700 1 --2- 172.21.15.143:0/576875420 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1cd00b22e0 0x7f1cd00b2750 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1cc8009dc0 tx=0x7f1cc800b330 comp rx=0 tx=0).stop 2023-11-08T19:28:12.992 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.987+0000 7f1cd8304700 1 -- 172.21.15.143:0/576875420 shutdown_connections 2023-11-08T19:28:12.993 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.987+0000 7f1cd8304700 1 --2- 172.21.15.143:0/576875420 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1cd00b22e0 0x7f1cd00b2750 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:12.993 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.987+0000 7f1cd8304700 1 -- 172.21.15.143:0/576875420 >> 172.21.15.143:0/576875420 conn(0x7f1cd00cd2e0 msgr2=0x7f1cd00cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:12.993 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.987+0000 7f1cd8304700 1 -- 172.21.15.143:0/576875420 shutdown_connections 2023-11-08T19:28:12.993 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.987+0000 7f1cd8304700 1 -- 172.21.15.143:0/576875420 wait complete. 2023-11-08T19:28:12.996 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.988+0000 7f1cd8304700 1 Processor -- start 2023-11-08T19:28:12.996 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.988+0000 7f1cd8304700 1 -- start start 2023-11-08T19:28:12.996 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.989+0000 7f1cd8304700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1cd006b240 0x7f1cd014cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:12.996 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.989+0000 7f1cd8304700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1cd00b22e0 0x7f1cd00c8e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:12.997 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.989+0000 7f1cd8304700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1cd015dba0 0x7f1cd01e7c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:12.997 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.989+0000 7f1cd8304700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd00c5190 con 0x7f1cd006b240 2023-11-08T19:28:12.997 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.989+0000 7f1cd8304700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd00c4510 con 0x7f1cd00b22e0 2023-11-08T19:28:12.997 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.989+0000 7f1cd8304700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd00c47f0 con 0x7f1cd015dba0 2023-11-08T19:28:12.997 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.989+0000 7f1cd60a0700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1cd006b240 0x7f1cd014cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:12.998 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.989+0000 7f1cd60a0700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1cd006b240 0x7f1cd014cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54724/0 (socket says 172.21.15.143:54724) 2023-11-08T19:28:12.998 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.989+0000 7f1cd60a0700 1 -- 172.21.15.143:0/2184035130 learned_addr learned my addr 172.21.15.143:0/2184035130 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:28:12.998 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.989+0000 7f1cd589f700 1 --2- 172.21.15.143:0/2184035130 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1cd00b22e0 0x7f1cd00c8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:12.998 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.989+0000 7f1cd68a1700 1 --2- 172.21.15.143:0/2184035130 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1cd015dba0 0x7f1cd01e7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:12.998 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.989+0000 7f1cd60a0700 1 -- 172.21.15.143:0/2184035130 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1cd015dba0 msgr2=0x7f1cd01e7c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:12.999 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.989+0000 7f1cd60a0700 1 --2- 172.21.15.143:0/2184035130 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1cd015dba0 0x7f1cd01e7c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:12.999 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.989+0000 7f1cd60a0700 1 -- 172.21.15.143:0/2184035130 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1cd00b22e0 msgr2=0x7f1cd00c8e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:12.999 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.989+0000 7f1cd60a0700 1 --2- 172.21.15.143:0/2184035130 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1cd00b22e0 0x7f1cd00c8e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:12.999 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.989+0000 7f1cd60a0700 1 -- 172.21.15.143:0/2184035130 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cc8009a70 con 0x7f1cd006b240 2023-11-08T19:28:12.999 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.989+0000 7f1cd60a0700 1 --2- 172.21.15.143:0/2184035130 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1cd006b240 0x7f1cd014cb90 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f1cc000c4d0 tx=0x7f1cc000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:13.000 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.989+0000 7f1cc6ffd700 1 -- 172.21.15.143:0/2184035130 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cc0004c80 con 0x7f1cd006b240 2023-11-08T19:28:13.000 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.989+0000 7f1cc6ffd700 1 -- 172.21.15.143:0/2184035130 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cc0004e00 con 0x7f1cd006b240 2023-11-08T19:28:13.000 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.989+0000 7f1cd8304700 1 -- 172.21.15.143:0/2184035130 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cd00c4ad0 con 0x7f1cd006b240 2023-11-08T19:28:13.000 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.989+0000 7f1cc6ffd700 1 -- 172.21.15.143:0/2184035130 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cc0007310 con 0x7f1cd006b240 2023-11-08T19:28:13.000 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.989+0000 7f1cd8304700 1 -- 172.21.15.143:0/2184035130 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cd014d0d0 con 0x7f1cd006b240 2023-11-08T19:28:13.001 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.990+0000 7f1cd8304700 1 -- 172.21.15.143:0/2184035130 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cd00c5400 con 0x7f1cd006b240 2023-11-08T19:28:13.001 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.991+0000 7f1cc6ffd700 1 -- 172.21.15.143:0/2184035130 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1cc0007470 con 0x7f1cd006b240 2023-11-08T19:28:13.001 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.991+0000 7f1cc6ffd700 1 --2- 172.21.15.143:0/2184035130 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f1cb406af00 0x7f1cb406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:13.001 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.991+0000 7f1cc6ffd700 1 -- 172.21.15.143:0/2184035130 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6772+0+0 (secure 0 0 0) 0x7f1cc0089760 con 0x7f1cd006b240 2023-11-08T19:28:13.001 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.991+0000 7f1cd589f700 1 --2- 172.21.15.143:0/2184035130 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f1cb406af00 0x7f1cb406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:13.002 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.992+0000 7f1cd589f700 1 --2- 172.21.15.143:0/2184035130 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f1cb406af00 0x7f1cb406d3c0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f1cd00b6420 tx=0x7f1ccc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:13.002 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:12.995+0000 7f1cc6ffd700 1 -- 172.21.15.143:0/2184035130 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1cc00543e0 con 0x7f1cd006b240 2023-11-08T19:28:13.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:13.261+0000 7f1cd8304700 1 -- 172.21.15.143:0/2184035130 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f1cd004eee0 con 0x7f1cd006b240 2023-11-08T19:28:13.266 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:13.265+0000 7f1cc6ffd700 1 -- 172.21.15.143:0/2184035130 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) v1 ==== 190+0+147 (secure 0 0 0) 0x7f1cc0057a00 con 0x7f1cd006b240 2023-11-08T19:28:13.267 INFO:teuthology.orchestra.run.smithi143.stdout:[client.0] 2023-11-08T19:28:13.267 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQBP4Etl0GdSFxAAVkuHL0EnjrtTUMkj/afPkg== 2023-11-08T19:28:13.267 INFO:teuthology.orchestra.run.smithi143.stdout: caps mds = "allow" 2023-11-08T19:28:13.268 INFO:teuthology.orchestra.run.smithi143.stdout: caps mon = "allow r" 2023-11-08T19:28:13.268 INFO:teuthology.orchestra.run.smithi143.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-11-08T19:28:13.270 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:13.268+0000 7f1cd8304700 1 -- 172.21.15.143:0/2184035130 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f1cb406af00 msgr2=0x7f1cb406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:13.270 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:13.268+0000 7f1cd8304700 1 --2- 172.21.15.143:0/2184035130 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f1cb406af00 0x7f1cb406d3c0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f1cd00b6420 tx=0x7f1ccc009040 comp rx=0 tx=0).stop 2023-11-08T19:28:13.270 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:13.268+0000 7f1cd8304700 1 -- 172.21.15.143:0/2184035130 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1cd006b240 msgr2=0x7f1cd014cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:13.270 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:13.268+0000 7f1cd8304700 1 --2- 172.21.15.143:0/2184035130 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1cd006b240 0x7f1cd014cb90 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f1cc000c4d0 tx=0x7f1cc000c890 comp rx=0 tx=0).stop 2023-11-08T19:28:13.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:13.268+0000 7f1cd8304700 1 -- 172.21.15.143:0/2184035130 shutdown_connections 2023-11-08T19:28:13.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:13.268+0000 7f1cd8304700 1 --2- 172.21.15.143:0/2184035130 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f1cb406af00 0x7f1cb406d3c0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:13.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:13.268+0000 7f1cd8304700 1 --2- 172.21.15.143:0/2184035130 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1cd015dba0 0x7f1cd01e7c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:13.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:13.268+0000 7f1cd8304700 1 --2- 172.21.15.143:0/2184035130 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1cd006b240 0x7f1cd014cb90 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:13.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:13.268+0000 7f1cd8304700 1 --2- 172.21.15.143:0/2184035130 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1cd00b22e0 0x7f1cd00c8e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:13.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:13.268+0000 7f1cd8304700 1 -- 172.21.15.143:0/2184035130 >> 172.21.15.143:0/2184035130 conn(0x7f1cd00cd2e0 msgr2=0x7f1cd015ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:13.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:13.268+0000 7f1cd8304700 1 -- 172.21.15.143:0/2184035130 shutdown_connections 2023-11-08T19:28:13.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:13.268+0000 7f1cd8304700 1 -- 172.21.15.143:0/2184035130 wait complete. 2023-11-08T19:28:13.273 INFO:teuthology.orchestra.run.smithi143.stderr:updated caps for client.0 2023-11-08T19:28:13.282 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:28:13.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:13.401+0000 7f63b16a8700 1 Processor -- start 2023-11-08T19:28:13.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:13.401+0000 7f63b16a8700 1 -- start start 2023-11-08T19:28:13.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:13.401+0000 7f63b16a8700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f63ac15dc80 0x7f63ac0d1f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:13.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:13.401+0000 7f63b16a8700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f63ac0cf090 0x7f63ac0b9690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:13.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:13.401+0000 7f63b16a8700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63ac0d3200 con 0x7f63ac0d3690 2023-11-08T19:28:13.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:13.401+0000 7f63b16a8700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63ac0b9bd0 con 0x7f63ac0cf090 2023-11-08T19:28:13.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:13.401+0000 7f63b16a8700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63ac0b7cf0 con 0x7f63ac15dc80 2023-11-08T19:28:13.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:13.402+0000 7f63aa7fc700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f63ac0d3690 0x7f63ac0ce980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:50886/0 (socket says 172.21.15.143:50886) 2023-11-08T19:28:13.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:13.402+0000 7f63aa7fc700 1 -- 172.21.15.143:0/204055704 learned_addr learned my addr 172.21.15.143:0/204055704 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:28:13.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:13.402+0000 7f63ab7fe700 1 --2- 172.21.15.143:0/204055704 >> v2:172.21.15.174:3300/0 conn(0x7f63ac0cf090 0x7f63ac0b9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:13.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:13.402+0000 7f63a9ffb700 1 -- 172.21.15.143:0/204055704 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3216495444 0 0) 0x7f63ac0d3200 con 0x7f63ac0d3690 2023-11-08T19:28:13.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:13.402+0000 7f63aaffd700 1 --2- 172.21.15.143:0/204055704 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f63ac15dc80 0x7f63ac0d1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:13.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:13.402+0000 7f63a9ffb700 1 -- 172.21.15.143:0/204055704 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f63940035b0 con 0x7f63ac0d3690 2023-11-08T19:28:13.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:13.402+0000 7f63a9ffb700 1 -- 172.21.15.143:0/204055704 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 433679706 0 0) 0x7f63940035b0 con 0x7f63ac0d3690 2023-11-08T19:28:13.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:13.402+0000 7f63a9ffb700 1 -- 172.21.15.143:0/204055704 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f63ac15dc80 msgr2=0x7f63ac0d1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:13.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:13.402+0000 7f63a9ffb700 1 --2- 172.21.15.143:0/204055704 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f63ac15dc80 0x7f63ac0d1f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:13.409 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:13.402+0000 7f63a9ffb700 1 -- 172.21.15.143:0/204055704 >> v2:172.21.15.174:3300/0 conn(0x7f63ac0cf090 msgr2=0x7f63ac0b9690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:13.409 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:13.402+0000 7f63a9ffb700 1 --2- 172.21.15.143:0/204055704 >> v2:172.21.15.174:3300/0 conn(0x7f63ac0cf090 0x7f63ac0b9690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:13.409 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:13.402+0000 7f63a9ffb700 1 -- 172.21.15.143:0/204055704 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63ac15e060 con 0x7f63ac0d3690 2023-11-08T19:28:13.409 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:13.402+0000 7f63a9ffb700 1 -- 172.21.15.143:0/204055704 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4147756303 0 0) 0x7f639c001670 con 0x7f63ac0d3690 2023-11-08T19:28:13.409 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:13.402+0000 7f63a9ffb700 1 -- 172.21.15.143:0/204055704 >> v1:172.21.15.143:6789/0 conn(0x7f63ac0d3690 legacy=0x7f63ac0ce980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:13.409 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:13.402+0000 7f63a9ffb700 1 --2- 172.21.15.143:0/204055704 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6394003a00 0x7f6394003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:13.410 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:13.402+0000 7f63a9ffb700 1 --2- 172.21.15.143:0/204055704 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6394004850 0x7f6394004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:13.410 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:13.402+0000 7f63a9ffb700 1 --2- 172.21.15.143:0/204055704 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f63ac15dc80 0x7f6394005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:13.410 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:13.402+0000 7f63a9ffb700 1 -- 172.21.15.143:0/204055704 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63ac0b9bd0 con 0x7f6394003a00 2023-11-08T19:28:13.410 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:13.402+0000 7f63a9ffb700 1 -- 172.21.15.143:0/204055704 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63ac0b7cf0 con 0x7f6394004850 2023-11-08T19:28:13.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:13.402+0000 7f63a9ffb700 1 -- 172.21.15.143:0/204055704 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f639400ba10 con 0x7f63ac15dc80 2023-11-08T19:28:13.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:13.402+0000 7f63aaffd700 1 --2- 172.21.15.143:0/204055704 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6394003a00 0x7f6394003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:13.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:13.402+0000 7f63aa7fc700 1 --2- 172.21.15.143:0/204055704 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6394004850 0x7f6394004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:13.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:13.402+0000 7f63ab7fe700 1 --2- 172.21.15.143:0/204055704 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f63ac15dc80 0x7f6394005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:13.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:13.403+0000 7f63aaffd700 1 -- 172.21.15.143:0/204055704 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f63ac15dc80 msgr2=0x7f6394005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:13.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:13.403+0000 7f63aaffd700 1 --2- 172.21.15.143:0/204055704 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f63ac15dc80 0x7f6394005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:13.412 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:13.403+0000 7f63aaffd700 1 -- 172.21.15.143:0/204055704 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6394004850 msgr2=0x7f6394004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:13.412 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:13.403+0000 7f63aaffd700 1 --2- 172.21.15.143:0/204055704 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6394004850 0x7f6394004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:13.412 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:13.403+0000 7f63aaffd700 1 -- 172.21.15.143:0/204055704 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63940035b0 con 0x7f6394003a00 2023-11-08T19:28:13.412 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:13.403+0000 7f63ab7fe700 1 --2- 172.21.15.143:0/204055704 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f63ac15dc80 0x7f6394005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:28:13.412 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:13.403+0000 7f63aaffd700 1 --2- 172.21.15.143:0/204055704 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6394003a00 0x7f6394003e20 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f63a0009b90 tx=0x7f6394005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=424db5a36e6a34b9 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:13.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:13.403+0000 7f63a9ffb700 1 -- 172.21.15.143:0/204055704 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a000e6c0 con 0x7f6394003a00 2023-11-08T19:28:13.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:13.403+0000 7f63a9ffb700 1 -- 172.21.15.143:0/204055704 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63a000e820 con 0x7f6394003a00 2023-11-08T19:28:13.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:13.403+0000 7f63a9ffb700 1 -- 172.21.15.143:0/204055704 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a000eb90 con 0x7f6394003a00 2023-11-08T19:28:13.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:13.403+0000 7f63b16a8700 1 -- 172.21.15.143:0/204055704 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6394003a00 msgr2=0x7f6394003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:13.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:13.403+0000 7f63b16a8700 1 --2- 172.21.15.143:0/204055704 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6394003a00 0x7f6394003e20 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f63a0009b90 tx=0x7f6394005880 comp rx=0 tx=0).stop 2023-11-08T19:28:13.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:13.403+0000 7f63ab7fe700 1 -- 172.21.15.143:0/204055704 reap_dead start 2023-11-08T19:28:13.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:13.403+0000 7f63b16a8700 1 -- 172.21.15.143:0/204055704 shutdown_connections 2023-11-08T19:28:13.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:13.403+0000 7f63b16a8700 1 -- 172.21.15.143:0/204055704 >> 172.21.15.143:0/204055704 conn(0x7f63ac0668e0 msgr2=0x7f63ac066cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:13.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:13.403+0000 7f63b16a8700 1 -- 172.21.15.143:0/204055704 shutdown_connections 2023-11-08T19:28:13.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:13.403+0000 7f63b16a8700 1 -- 172.21.15.143:0/204055704 wait complete. 2023-11-08T19:28:13.417 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:13.405+0000 7f63b16a8700 1 Processor -- start 2023-11-08T19:28:13.417 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:13.405+0000 7f63b16a8700 1 -- start start 2023-11-08T19:28:13.418 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:13.406+0000 7f63b16a8700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6394003a00 0x7f63ac0cd5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:13.418 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:13.406+0000 7f63b16a8700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f63ac154cc0 0x7f63ac0c96c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:13.418 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:13.406+0000 7f63b16a8700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f63ac0d3df0 0x7f63ac0d4270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:13.418 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:13.406+0000 7f63b16a8700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63ac0d47b0 con 0x7f63ac0d3df0 2023-11-08T19:28:13.419 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:13.406+0000 7f63b16a8700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63ac0cdae0 con 0x7f6394003a00 2023-11-08T19:28:13.419 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:13.406+0000 7f63b16a8700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63ac1f73d0 con 0x7f63ac154cc0 2023-11-08T19:28:13.419 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:13.406+0000 7f63ab7fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f63ac0d3df0 0x7f63ac0d4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:13.419 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:13.406+0000 7f63ab7fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f63ac0d3df0 0x7f63ac0d4270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54748/0 (socket says 172.21.15.143:54748) 2023-11-08T19:28:13.419 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:13.406+0000 7f63ab7fe700 1 -- 172.21.15.143:0/2600707955 learned_addr learned my addr 172.21.15.143:0/2600707955 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:28:13.419 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:13.406+0000 7f63aa7fc700 1 --2- 172.21.15.143:0/2600707955 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f63ac154cc0 0x7f63ac0c96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:13.420 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:13.406+0000 7f63aaffd700 1 --2- 172.21.15.143:0/2600707955 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6394003a00 0x7f63ac0cd5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:13.420 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:13.406+0000 7f63ab7fe700 1 -- 172.21.15.143:0/2600707955 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f63ac154cc0 msgr2=0x7f63ac0c96c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:13.420 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:13.406+0000 7f63ab7fe700 1 --2- 172.21.15.143:0/2600707955 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f63ac154cc0 0x7f63ac0c96c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:13.420 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:13.406+0000 7f63ab7fe700 1 -- 172.21.15.143:0/2600707955 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6394003a00 msgr2=0x7f63ac0cd5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:13.420 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:13.406+0000 7f63ab7fe700 1 --2- 172.21.15.143:0/2600707955 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6394003a00 0x7f63ac0cd5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:13.421 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:13.406+0000 7f63ab7fe700 1 -- 172.21.15.143:0/2600707955 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63ac1f76b0 con 0x7f63ac0d3df0 2023-11-08T19:28:13.421 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:13.406+0000 7f63aa7fc700 1 --2- 172.21.15.143:0/2600707955 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f63ac154cc0 0x7f63ac0c96c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:28:13.421 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:13.406+0000 7f63ab7fe700 1 --2- 172.21.15.143:0/2600707955 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f63ac0d3df0 0x7f63ac0d4270 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f63ac06d470 tx=0x7f63a40114b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:13.421 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:13.406+0000 7f63937fe700 1 -- 172.21.15.143:0/2600707955 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a400e030 con 0x7f63ac0d3df0 2023-11-08T19:28:13.421 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:13.406+0000 7f63937fe700 1 -- 172.21.15.143:0/2600707955 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63a4011c50 con 0x7f63ac0d3df0 2023-11-08T19:28:13.422 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:13.406+0000 7f63b16a8700 1 -- 172.21.15.143:0/2600707955 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63ac1f7990 con 0x7f63ac0d3df0 2023-11-08T19:28:13.422 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:13.406+0000 7f63937fe700 1 -- 172.21.15.143:0/2600707955 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a4009b10 con 0x7f63ac0d3df0 2023-11-08T19:28:13.422 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:13.406+0000 7f63b16a8700 1 -- 172.21.15.143:0/2600707955 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63ac1f7f50 con 0x7f63ac0d3df0 2023-11-08T19:28:13.422 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:13.407+0000 7f63b16a8700 1 -- 172.21.15.143:0/2600707955 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63ac1512c0 con 0x7f63ac0d3df0 2023-11-08T19:28:13.422 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:13.408+0000 7f63937fe700 1 -- 172.21.15.143:0/2600707955 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f63a4016470 con 0x7f63ac0d3df0 2023-11-08T19:28:13.423 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:13.408+0000 7f63937fe700 1 --2- 172.21.15.143:0/2600707955 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f639806af00 0x7f639806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:13.423 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:13.408+0000 7f63937fe700 1 -- 172.21.15.143:0/2600707955 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6772+0+0 (secure 0 0 0) 0x7f63a400f070 con 0x7f63ac0d3df0 2023-11-08T19:28:13.423 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:13.408+0000 7f63aaffd700 1 --2- 172.21.15.143:0/2600707955 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f639806af00 0x7f639806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:13.423 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:13.409+0000 7f63aaffd700 1 --2- 172.21.15.143:0/2600707955 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f639806af00 0x7f639806d3c0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f63a00090b0 tx=0x7f63a0008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:13.423 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:13.413+0000 7f63937fe700 1 -- 172.21.15.143:0/2600707955 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f63ac1512c0 con 0x7f63ac0d3df0 2023-11-08T19:28:13.644 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:13.642+0000 7f63b16a8700 1 -- 172.21.15.143:0/2600707955 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f63ac1512c0 con 0x7f63ac0d3df0 2023-11-08T19:28:13.644 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:13.643+0000 7f63937fe700 1 -- 172.21.15.143:0/2600707955 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) v1 ==== 94+0+4927 (secure 0 0 0) 0x7f63ac1512c0 con 0x7f63ac0d3df0 2023-11-08T19:28:13.645 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:28:13.645 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":52,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6906,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6834/3172664377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":3172664377},{"type":"v1","addr":"172.21.15.143:6834","nonce":3172664377}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47},{"gid":6909,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3953059900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":3953059900},{"type":"v1","addr":"172.21.15.143:6835","nonce":3953059900}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":46},{"gid":6918,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6836/3601088618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3601088618},{"type":"v1","addr":"172.21.15.174:6836","nonce":3601088618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T19:28:03.092245+0000","modified":"2023-11-08T19:28:06.121216+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6921},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6921":{"gid":6921,"name":"d","rank":0,"incarnation":48,"state":"up:active","state_seq":3,"addr":"172.21.15.174:6837/2201447629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6835","nonce":2201447629},{"type":"v1","addr":"172.21.15.174:6837","nonce":2201447629}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:28:13.647 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:13.645+0000 7f63b16a8700 1 -- 172.21.15.143:0/2600707955 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f639806af00 msgr2=0x7f639806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:13.647 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:13.645+0000 7f63b16a8700 1 --2- 172.21.15.143:0/2600707955 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f639806af00 0x7f639806d3c0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f63a00090b0 tx=0x7f63a0008030 comp rx=0 tx=0).stop 2023-11-08T19:28:13.647 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:13.645+0000 7f63b16a8700 1 -- 172.21.15.143:0/2600707955 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f63ac0d3df0 msgr2=0x7f63ac0d4270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:13.647 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:13.645+0000 7f63b16a8700 1 --2- 172.21.15.143:0/2600707955 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f63ac0d3df0 0x7f63ac0d4270 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f63ac06d470 tx=0x7f63a40114b0 comp rx=0 tx=0).stop 2023-11-08T19:28:13.648 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:13.645+0000 7f63b16a8700 1 -- 172.21.15.143:0/2600707955 shutdown_connections 2023-11-08T19:28:13.648 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:13.645+0000 7f63b16a8700 1 --2- 172.21.15.143:0/2600707955 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f639806af00 0x7f639806d3c0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:13.648 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:13.645+0000 7f63b16a8700 1 --2- 172.21.15.143:0/2600707955 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f63ac0d3df0 0x7f63ac0d4270 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:13.648 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:13.645+0000 7f63b16a8700 1 --2- 172.21.15.143:0/2600707955 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6394003a00 0x7f63ac0cd5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:13.648 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:13.645+0000 7f63b16a8700 1 --2- 172.21.15.143:0/2600707955 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f63ac154cc0 0x7f63ac0c96c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:13.649 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:13.645+0000 7f63b16a8700 1 -- 172.21.15.143:0/2600707955 >> 172.21.15.143:0/2600707955 conn(0x7f63ac0668e0 msgr2=0x7f63ac15c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:13.649 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:13.645+0000 7f63b16a8700 1 -- 172.21.15.143:0/2600707955 shutdown_connections 2023-11-08T19:28:13.649 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:13.645+0000 7f63b16a8700 1 -- 172.21.15.143:0/2600707955 wait complete. 2023-11-08T19:28:13.650 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 52 2023-11-08T19:28:14.061 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 52, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-08T19:28:03.092245+0000', 'modified': '2023-11-08T19:28:06.121216+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6921}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6921': {'gid': 6921, 'name': 'd', 'rank': 0, 'incarnation': 48, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.174:6837/2201447629', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.174:6835', 'nonce': 2201447629}, {'type': 'v1', 'addr': '172.21.15.174:6837', 'nonce': 2201447629}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-08T19:28:14.065 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-08T19:28:14.065 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-11-08T19:28:14.188 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.186+0000 7f22e7c6d700 1 Processor -- start 2023-11-08T19:28:14.188 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.186+0000 7f22e7c6d700 1 -- start start 2023-11-08T19:28:14.190 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.187+0000 7f22e7c6d700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f22e015aa30 0x7f22e00cf510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:14.191 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.187+0000 7f22e7c6d700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f22e00b2210 0x7f22e00b2690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:14.191 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.187+0000 7f22e7c6d700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22e00cf080 con 0x7f22e00cbe20 2023-11-08T19:28:14.191 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.187+0000 7f22e7c6d700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22e00b2bd0 con 0x7f22e00b2210 2023-11-08T19:28:14.191 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.187+0000 7f22e7c6d700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22e00ce470 con 0x7f22e015aa30 2023-11-08T19:28:14.192 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.187+0000 7f22e5208700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f22e00b2210 0x7f22e00b2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:14.192 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.187+0000 7f22e5208700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f22e00b2210 0x7f22e00b2690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54752/0 (socket says 172.21.15.143:54752) 2023-11-08T19:28:14.192 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.187+0000 7f22e5208700 1 -- 172.21.15.143:0/3497609681 learned_addr learned my addr 172.21.15.143:0/3497609681 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:28:14.192 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.188+0000 7f22e5a09700 1 --2- 172.21.15.143:0/3497609681 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f22e015aa30 0x7f22e00cf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:14.192 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.188+0000 7f22e5208700 1 -- 172.21.15.143:0/3497609681 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f22e015aa30 msgr2=0x7f22e00cf510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:14.192 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.188+0000 7f22e5208700 1 --2- 172.21.15.143:0/3497609681 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f22e015aa30 0x7f22e00cf510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:14.193 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.188+0000 7f22e5208700 1 -- 172.21.15.143:0/3497609681 >> v1:172.21.15.174:6789/0 conn(0x7f22e00cbe20 legacy=0x7f22e00cc270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:14.193 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.188+0000 7f22e5208700 1 -- 172.21.15.143:0/3497609681 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22e00ce750 con 0x7f22e00b2210 2023-11-08T19:28:14.193 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.188+0000 7f22e5a09700 1 --2- 172.21.15.143:0/3497609681 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f22e015aa30 0x7f22e00cf510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:28:14.193 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.188+0000 7f22e5208700 1 --2- 172.21.15.143:0/3497609681 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f22e00b2210 0x7f22e00b2690 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f22dc0098f0 tx=0x7f22dc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd36e21e4795c798 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:14.193 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.188+0000 7f22e4a07700 1 -- 172.21.15.143:0/3497609681 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22dc004230 con 0x7f22e00b2210 2023-11-08T19:28:14.194 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.188+0000 7f22e4a07700 1 -- 172.21.15.143:0/3497609681 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22dc004390 con 0x7f22e00b2210 2023-11-08T19:28:14.194 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.188+0000 7f22e4a07700 1 -- 172.21.15.143:0/3497609681 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22dc0036a0 con 0x7f22e00b2210 2023-11-08T19:28:14.194 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.189+0000 7f22e7c6d700 1 -- 172.21.15.143:0/3497609681 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f22e00b2210 msgr2=0x7f22e00b2690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:14.194 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.189+0000 7f22e7c6d700 1 --2- 172.21.15.143:0/3497609681 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f22e00b2210 0x7f22e00b2690 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f22dc0098f0 tx=0x7f22dc009c00 comp rx=0 tx=0).stop 2023-11-08T19:28:14.194 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.189+0000 7f22e7c6d700 1 -- 172.21.15.143:0/3497609681 shutdown_connections 2023-11-08T19:28:14.195 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.189+0000 7f22e7c6d700 1 --2- 172.21.15.143:0/3497609681 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f22e015aa30 0x7f22e00cf510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:14.195 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.189+0000 7f22e7c6d700 1 --2- 172.21.15.143:0/3497609681 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f22e00b2210 0x7f22e00b2690 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:14.195 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.189+0000 7f22e7c6d700 1 -- 172.21.15.143:0/3497609681 >> 172.21.15.143:0/3497609681 conn(0x7f22e0099ab0 msgr2=0x7f22e00d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:14.195 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.189+0000 7f22e7c6d700 1 -- 172.21.15.143:0/3497609681 shutdown_connections 2023-11-08T19:28:14.195 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.189+0000 7f22e7c6d700 1 -- 172.21.15.143:0/3497609681 wait complete. 2023-11-08T19:28:14.204 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.190+0000 7f22e7c6d700 1 Processor -- start 2023-11-08T19:28:14.204 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.190+0000 7f22e7c6d700 1 -- start start 2023-11-08T19:28:14.204 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.191+0000 7f22e7c6d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f22e00b2210 0x7f22e00b33b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:14.205 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.191+0000 7f22e7c6d700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f22e00cbe20 0x7f22e00c5c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:14.205 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.191+0000 7f22e7c6d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f22e015aa30 0x7f22e0153df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:14.205 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.191+0000 7f22e7c6d700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e015bef0 con 0x7f22e015aa30 2023-11-08T19:28:14.205 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.191+0000 7f22e7c6d700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e015cd30 con 0x7f22e00b2210 2023-11-08T19:28:14.205 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.191+0000 7f22e7c6d700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22e00b38f0 con 0x7f22e00cbe20 2023-11-08T19:28:14.206 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.191+0000 7f22e620a700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f22e015aa30 0x7f22e0153df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:14.206 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.191+0000 7f22e620a700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f22e015aa30 0x7f22e0153df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54760/0 (socket says 172.21.15.143:54760) 2023-11-08T19:28:14.206 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.191+0000 7f22e620a700 1 -- 172.21.15.143:0/3638109487 learned_addr learned my addr 172.21.15.143:0/3638109487 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:28:14.206 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.191+0000 7f22e5208700 1 --2- 172.21.15.143:0/3638109487 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f22e00cbe20 0x7f22e00c5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:14.206 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.191+0000 7f22e620a700 1 -- 172.21.15.143:0/3638109487 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f22e00cbe20 msgr2=0x7f22e00c5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:14.207 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.191+0000 7f22e620a700 1 --2- 172.21.15.143:0/3638109487 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f22e00cbe20 0x7f22e00c5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:14.207 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.191+0000 7f22e620a700 1 -- 172.21.15.143:0/3638109487 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f22e00b2210 msgr2=0x7f22e00b33b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:14.207 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.191+0000 7f22e5a09700 1 --2- 172.21.15.143:0/3638109487 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f22e00b2210 0x7f22e00b33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:14.207 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.191+0000 7f22e620a700 1 --2- 172.21.15.143:0/3638109487 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f22e00b2210 0x7f22e00b33b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:14.207 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.191+0000 7f22e620a700 1 -- 172.21.15.143:0/3638109487 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22dc009550 con 0x7f22e015aa30 2023-11-08T19:28:14.208 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.191+0000 7f22e5208700 1 --2- 172.21.15.143:0/3638109487 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f22e00cbe20 0x7f22e00c5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:28:14.208 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.191+0000 7f22e5a09700 1 --2- 172.21.15.143:0/3638109487 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f22e00b2210 0x7f22e00b33b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:28:14.208 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.191+0000 7f22e620a700 1 --2- 172.21.15.143:0/3638109487 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f22e015aa30 0x7f22e0153df0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f22d800cc40 tx=0x7f22d800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:14.208 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.191+0000 7f22d67fc700 1 -- 172.21.15.143:0/3638109487 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d800aba0 con 0x7f22e015aa30 2023-11-08T19:28:14.208 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.191+0000 7f22d67fc700 1 -- 172.21.15.143:0/3638109487 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22d800ad00 con 0x7f22e015aa30 2023-11-08T19:28:14.209 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.191+0000 7f22e7c6d700 1 -- 172.21.15.143:0/3638109487 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22e015c120 con 0x7f22e015aa30 2023-11-08T19:28:14.209 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.191+0000 7f22d67fc700 1 -- 172.21.15.143:0/3638109487 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d8012660 con 0x7f22e015aa30 2023-11-08T19:28:14.209 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.191+0000 7f22e7c6d700 1 -- 172.21.15.143:0/3638109487 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22e015c6e0 con 0x7f22e015aa30 2023-11-08T19:28:14.209 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.193+0000 7f22d67fc700 1 -- 172.21.15.143:0/3638109487 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f22d80127c0 con 0x7f22e015aa30 2023-11-08T19:28:14.209 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.193+0000 7f22d67fc700 1 --2- 172.21.15.143:0/3638109487 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f22c406af00 0x7f22c406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:14.210 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.193+0000 7f22d67fc700 1 -- 172.21.15.143:0/3638109487 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6772+0+0 (secure 0 0 0) 0x7f22d808a1b0 con 0x7f22e015aa30 2023-11-08T19:28:14.210 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.194+0000 7f22e5a09700 1 --2- 172.21.15.143:0/3638109487 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f22c406af00 0x7f22c406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:14.210 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.194+0000 7f22e5a09700 1 --2- 172.21.15.143:0/3638109487 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f22c406af00 0x7f22c406d3c0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f22d0006fd0 tx=0x7f22d0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:14.210 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.195+0000 7f22f3e66b80 1 Processor -- start 2023-11-08T19:28:14.210 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.196+0000 7f22f3e66b80 1 -- start start 2023-11-08T19:28:14.211 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.196+0000 7f22f3e66b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559a8f9426f0 0x559a8f972450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:14.211 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.196+0000 7f22f3e66b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a8f944c30 0x559a8f972990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:14.211 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.196+0000 7f22f3e66b80 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x559a8f942ad0 0x559a8f95cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:14.211 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.196+0000 7f22f3e66b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a8f8ae6e0 con 0x559a8f9426f0 2023-11-08T19:28:14.211 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.196+0000 7f22f3e66b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a8f8a23b0 con 0x559a8f944c30 2023-11-08T19:28:14.211 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.196+0000 7f22f3e66b80 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x559a8f89fcd0 con 0x559a8f942ad0 2023-11-08T19:28:14.212 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.196+0000 7f22e5a09700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559a8f9426f0 0x559a8f972450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:14.212 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.196+0000 7f22e5a09700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559a8f9426f0 0x559a8f972450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54764/0 (socket says 172.21.15.143:54764) 2023-11-08T19:28:14.212 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.196+0000 7f22e5a09700 1 -- 172.21.15.143:0/125158542 learned_addr learned my addr 172.21.15.143:0/125158542 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:28:14.212 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.196+0000 7f22e620a700 1 --2- 172.21.15.143:0/125158542 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x559a8f942ad0 0x559a8f95cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:14.212 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.196+0000 7f22e5208700 1 --2- 172.21.15.143:0/125158542 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a8f944c30 0x559a8f972990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:14.213 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.197+0000 7f22e5a09700 1 -- 172.21.15.143:0/125158542 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x559a8f942ad0 msgr2=0x559a8f95cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:14.213 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.197+0000 7f22e5a09700 1 --2- 172.21.15.143:0/125158542 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x559a8f942ad0 0x559a8f95cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:14.213 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.197+0000 7f22e5a09700 1 -- 172.21.15.143:0/125158542 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a8f944c30 msgr2=0x559a8f972990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:14.213 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.197+0000 7f22e5a09700 1 --2- 172.21.15.143:0/125158542 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a8f944c30 0x559a8f972990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:14.213 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.197+0000 7f22e5a09700 1 -- 172.21.15.143:0/125158542 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559a8f7185e0 con 0x559a8f9426f0 2023-11-08T19:28:14.214 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.197+0000 7f22e5208700 1 --2- 172.21.15.143:0/125158542 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a8f944c30 0x559a8f972990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:28:14.214 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.197+0000 7f22e5a09700 1 --2- 172.21.15.143:0/125158542 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559a8f9426f0 0x559a8f972450 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f22d000d9d0 tx=0x7f22d0010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c07130cf8ccf039 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:14.214 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.197+0000 7f22c37fe700 1 -- 172.21.15.143:0/125158542 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d000b050 con 0x559a8f9426f0 2023-11-08T19:28:14.214 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.197+0000 7f22c37fe700 1 -- 172.21.15.143:0/125158542 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22d0017070 con 0x559a8f9426f0 2023-11-08T19:28:14.214 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.197+0000 7f22c37fe700 1 -- 172.21.15.143:0/125158542 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d0008ea0 con 0x559a8f9426f0 2023-11-08T19:28:14.215 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.198+0000 7f22f3e66b80 1 -- 172.21.15.143:0/125158542 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559a8f9426f0 msgr2=0x559a8f972450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:14.215 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.198+0000 7f22f3e66b80 1 --2- 172.21.15.143:0/125158542 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559a8f9426f0 0x559a8f972450 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f22d000d9d0 tx=0x7f22d0010000 comp rx=0 tx=0).stop 2023-11-08T19:28:14.215 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.198+0000 7f22f3e66b80 1 -- 172.21.15.143:0/125158542 shutdown_connections 2023-11-08T19:28:14.215 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.198+0000 7f22f3e66b80 1 --2- 172.21.15.143:0/125158542 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x559a8f942ad0 0x559a8f95cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:14.215 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.198+0000 7f22f3e66b80 1 --2- 172.21.15.143:0/125158542 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559a8f9426f0 0x559a8f972450 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:14.216 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.198+0000 7f22f3e66b80 1 --2- 172.21.15.143:0/125158542 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a8f944c30 0x559a8f972990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:14.216 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.198+0000 7f22f3e66b80 1 -- 172.21.15.143:0/125158542 >> 172.21.15.143:0/125158542 conn(0x559a8f8bd660 msgr2=0x559a8f9483c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:14.216 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.198+0000 7f22f3e66b80 1 -- 172.21.15.143:0/125158542 shutdown_connections 2023-11-08T19:28:14.216 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.198+0000 7f22f3e66b80 1 -- 172.21.15.143:0/125158542 wait complete. 2023-11-08T19:28:14.216 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.198+0000 7f22f3e66b80 1 -- start start 2023-11-08T19:28:14.216 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.198+0000 7f22f3e66b80 1 Processor -- start 2023-11-08T19:28:14.217 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.199+0000 7f22f3e66b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559a8f9426f0 0x559a8f987a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:14.217 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.199+0000 7f22f3e66b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a8f942ad0 0x559a8f973d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:14.217 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.199+0000 7f22f3e66b80 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x559a8f974290 0x559a8f974710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:14.217 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.199+0000 7f22f3e66b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a8f89b5c0 con 0x559a8f9426f0 2023-11-08T19:28:14.217 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.199+0000 7f22f3e66b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a8f8370d0 con 0x559a8f942ad0 2023-11-08T19:28:14.218 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.199+0000 7f22f3e66b80 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x559a8f836d70 con 0x559a8f974290 2023-11-08T19:28:14.218 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.199+0000 7f22f3e66b80 10 client.0 _ll_register_callbacks cb 0x559a8f6937b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T19:28:14.218 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.199+0000 7f22e5a09700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559a8f9426f0 0x559a8f987a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:14.218 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.199+0000 7f22e5a09700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559a8f9426f0 0x559a8f987a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54770/0 (socket says 172.21.15.143:54770) 2023-11-08T19:28:14.218 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.199+0000 7f22e5a09700 1 -- 172.21.15.143:0/3549624471 learned_addr learned my addr 172.21.15.143:0/3549624471 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:28:14.219 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.200+0000 7f22e5a09700 1 -- 172.21.15.143:0/3549624471 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x559a8f974290 msgr2=0x559a8f974710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:14.219 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.200+0000 7f22e5208700 1 --2- 172.21.15.143:0/3549624471 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a8f942ad0 0x559a8f973d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:14.219 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.200+0000 7f22e5a09700 1 --2- 172.21.15.143:0/3549624471 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x559a8f974290 0x559a8f974710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:14.219 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.200+0000 7f22e5a09700 1 -- 172.21.15.143:0/3549624471 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a8f942ad0 msgr2=0x559a8f973d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:14.219 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.200+0000 7f22e5a09700 1 --2- 172.21.15.143:0/3549624471 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a8f942ad0 0x559a8f973d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:14.220 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.200+0000 7f22e5a09700 1 -- 172.21.15.143:0/3549624471 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22d000d6b0 con 0x559a8f9426f0 2023-11-08T19:28:14.220 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.200+0000 7f22e5a09700 1 --2- 172.21.15.143:0/3549624471 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559a8f9426f0 0x559a8f987a30 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f22d0008010 tx=0x7f22d00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b474796fc3eacef server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:14.220 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.200+0000 7f22c0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2023-11-08T19:28:14.220 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.200+0000 7f22c0ff9700 1 -- 172.21.15.143:0/3549624471 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d000b040 con 0x559a8f9426f0 2023-11-08T19:28:14.220 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.200+0000 7f22c0ff9700 1 -- 172.21.15.143:0/3549624471 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 6772+0+0 (secure 0 0 0) 0x7f22d002e070 con 0x559a8f9426f0 2023-11-08T19:28:14.220 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.200+0000 7f22f3e66b80 1 -- 172.21.15.143:0/3549624471 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559a8f93a880 con 0x559a8f9426f0 2023-11-08T19:28:14.221 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.200+0000 7f22c0ff9700 1 -- 172.21.15.143:0/3549624471 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f22d001e850 con 0x559a8f9426f0 2023-11-08T19:28:14.221 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.200+0000 7f22f3e66b80 10 client.7101 fetch_fsmap learned FSMap version 52 2023-11-08T19:28:14.221 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.200+0000 7f22f3e66b80 1 -- 172.21.15.143:0/3549624471 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) v3 -- 0x559a8f6c4d90 con 0x559a8f9426f0 2023-11-08T19:28:14.221 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.200+0000 7f22c0ff9700 1 -- 172.21.15.143:0/3549624471 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559a8f93a880 con 0x559a8f9426f0 2023-11-08T19:28:14.221 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.200+0000 7f22c0ff9700 1 -- 172.21.15.143:0/3549624471 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 52) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f22d001e9b0 con 0x559a8f9426f0 2023-11-08T19:28:14.222 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.200+0000 7f22f3e66b80 10 client.7101 fetch_fsmap finished waiting for FSMap version 52 2023-11-08T19:28:14.222 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.200+0000 7f22f3e66b80 10 client.7101 resolve_mds: resolved name 'd' to daemon mds.d 2023-11-08T19:28:14.222 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.200+0000 7f22f3e66b80 20 client.7101 populate_metadata read hostname 'smithi143' 2023-11-08T19:28:14.222 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.201+0000 7f22f3e66b80 1 --2- 172.21.15.143:0/3549624471 >> [v2:172.21.15.174:6835/2201447629,v1:172.21.15.174:6837/2201447629] conn(0x559a8f99d3f0 0x559a8f9aa210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:14.222 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.201+0000 7f22f3e66b80 4 client.7101 mds_command: new command op to 6921 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-08T19:28:14.223 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.201+0000 7f22f3e66b80 1 -- 172.21.15.143:0/3549624471 --> [v2:172.21.15.174:6835/2201447629,v1:172.21.15.174:6837/2201447629] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559a8f916a30 con 0x559a8f99d3f0 2023-11-08T19:28:14.223 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.201+0000 7f22e5208700 1 --2- 172.21.15.143:0/3549624471 >> [v2:172.21.15.174:6835/2201447629,v1:172.21.15.174:6837/2201447629] conn(0x559a8f99d3f0 0x559a8f9aa210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:14.223 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.201+0000 7f22c0ff9700 10 client.7101 ms_handle_connect on v2:172.21.15.174:6835/2201447629 2023-11-08T19:28:14.223 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.201+0000 7f22e5208700 1 --2- 172.21.15.143:0/3549624471 >> [v2:172.21.15.174:6835/2201447629,v1:172.21.15.174:6837/2201447629] conn(0x559a8f99d3f0 0x559a8f9aa210 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9170ace2b33442e server_cookie=52d12291f5260fc0 in_seq=0 out_seq=0 2023-11-08T19:28:14.223 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.202+0000 7f22c0ff9700 1 -- 172.21.15.143:0/3549624471 <== mds.0 v2:172.21.15.174:6835/2201447629 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559a8f916a30 con 0x559a8f99d3f0 2023-11-08T19:28:14.224 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.202+0000 7f22c0ff9700 10 client.7101 handle_command_reply: tid=0 2023-11-08T19:28:14.224 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.202+0000 7f22f3e66b80 1 client.7101 shutdown 2023-11-08T19:28:14.224 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.202+0000 7f22f3e66b80 1 -- 172.21.15.143:0/3549624471 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559a8f9426f0 msgr2=0x559a8f987a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:14.224 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.202+0000 7f22f3e66b80 1 --2- 172.21.15.143:0/3549624471 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559a8f9426f0 0x559a8f987a30 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f22d0008010 tx=0x7f22d00047c0 comp rx=0 tx=0).stop 2023-11-08T19:28:14.224 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.202+0000 7f22f3e66b80 1 -- 172.21.15.143:0/3549624471 shutdown_connections 2023-11-08T19:28:14.224 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.202+0000 7f22f3e66b80 1 --2- 172.21.15.143:0/3549624471 >> [v2:172.21.15.174:6835/2201447629,v1:172.21.15.174:6837/2201447629] conn(0x559a8f99d3f0 0x559a8f9aa210 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:14.225 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.202+0000 7f22f3e66b80 1 --2- 172.21.15.143:0/3549624471 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x559a8f974290 0x559a8f974710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:14.225 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.202+0000 7f22c0ff9700 0 client.7101 ms_handle_reset on v2:172.21.15.174:6835/2201447629 2023-11-08T19:28:14.225 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.202+0000 7f22f3e66b80 1 --2- 172.21.15.143:0/3549624471 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559a8f9426f0 0x559a8f987a30 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:14.225 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.202+0000 7f22f3e66b80 1 --2- 172.21.15.143:0/3549624471 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a8f942ad0 0x559a8f973d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:14.225 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.202+0000 7f22f3e66b80 1 -- 172.21.15.143:0/3549624471 >> 172.21.15.143:0/3549624471 conn(0x559a8f8bd660 msgr2=0x559a8f98e580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:14.226 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.202+0000 7f22f3e66b80 1 -- 172.21.15.143:0/3549624471 shutdown_connections 2023-11-08T19:28:14.226 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.202+0000 7f22f3e66b80 1 -- 172.21.15.143:0/3549624471 wait complete. 2023-11-08T19:28:14.226 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.202+0000 7f22f3e66b80 20 client.7101 trim_cache size 0 max 16384 2023-11-08T19:28:14.233 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.220+0000 7f22f3e66b80 1 Processor -- start 2023-11-08T19:28:14.233 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.220+0000 7f22f3e66b80 1 -- start start 2023-11-08T19:28:14.233 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.221+0000 7f22f3e66b80 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x559a8f9426f0 0x559a8f9887a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:14.233 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.221+0000 7f22f3e66b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559a8f942ad0 0x559a8f972b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:14.234 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.221+0000 7f22f3e66b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a8f974290 0x559a8f973070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:14.234 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.221+0000 7f22f3e66b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a8f71a8d0 con 0x559a8f942ad0 2023-11-08T19:28:14.234 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.221+0000 7f22f3e66b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a8f945880 con 0x559a8f974290 2023-11-08T19:28:14.234 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.221+0000 7f22f3e66b80 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x559a8f976620 con 0x559a8f9426f0 2023-11-08T19:28:14.234 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.221+0000 7f22e5208700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559a8f942ad0 0x559a8f972b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:14.235 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.221+0000 7f22e5208700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559a8f942ad0 0x559a8f972b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54780/0 (socket says 172.21.15.143:54780) 2023-11-08T19:28:14.235 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.221+0000 7f22e5208700 1 -- 172.21.15.143:0/1176171319 learned_addr learned my addr 172.21.15.143:0/1176171319 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:28:14.235 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.221+0000 7f22e620a700 1 --2- 172.21.15.143:0/1176171319 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a8f974290 0x559a8f973070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:14.235 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.221+0000 7f22e5a09700 1 --2- 172.21.15.143:0/1176171319 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x559a8f9426f0 0x559a8f9887a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:14.235 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.221+0000 7f22e5208700 1 -- 172.21.15.143:0/1176171319 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x559a8f9426f0 msgr2=0x559a8f9887a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:14.236 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.221+0000 7f22e5208700 1 --2- 172.21.15.143:0/1176171319 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x559a8f9426f0 0x559a8f9887a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:14.236 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.221+0000 7f22e5208700 1 -- 172.21.15.143:0/1176171319 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a8f974290 msgr2=0x559a8f973070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:14.236 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.221+0000 7f22e5208700 1 --2- 172.21.15.143:0/1176171319 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a8f974290 0x559a8f973070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:14.236 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.221+0000 7f22e5208700 1 -- 172.21.15.143:0/1176171319 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559a8f6adc10 con 0x559a8f942ad0 2023-11-08T19:28:14.236 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.221+0000 7f22e620a700 1 --2- 172.21.15.143:0/1176171319 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a8f974290 0x559a8f973070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:28:14.237 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.221+0000 7f22e5208700 1 --2- 172.21.15.143:0/1176171319 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559a8f942ad0 0x559a8f972b30 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f22dc004000 tx=0x7f22dc010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=172960855e5a9080 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:14.237 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.222+0000 7f22c3fff700 1 -- 172.21.15.143:0/1176171319 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22dc006d60 con 0x559a8f942ad0 2023-11-08T19:28:14.237 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.222+0000 7f22c3fff700 1 -- 172.21.15.143:0/1176171319 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22dc023040 con 0x559a8f942ad0 2023-11-08T19:28:14.237 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.222+0000 7f22c3fff700 1 -- 172.21.15.143:0/1176171319 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22dc024250 con 0x559a8f942ad0 2023-11-08T19:28:14.237 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.222+0000 7f22f3e66b80 1 -- 172.21.15.143:0/1176171319 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559a8f942ad0 msgr2=0x559a8f972b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:14.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.222+0000 7f22f3e66b80 1 --2- 172.21.15.143:0/1176171319 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559a8f942ad0 0x559a8f972b30 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f22dc004000 tx=0x7f22dc010000 comp rx=0 tx=0).stop 2023-11-08T19:28:14.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.222+0000 7f22f3e66b80 1 -- 172.21.15.143:0/1176171319 shutdown_connections 2023-11-08T19:28:14.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.222+0000 7f22f3e66b80 1 --2- 172.21.15.143:0/1176171319 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x559a8f9426f0 0x559a8f9887a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:14.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.222+0000 7f22f3e66b80 1 --2- 172.21.15.143:0/1176171319 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a8f974290 0x559a8f973070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:14.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.222+0000 7f22f3e66b80 1 --2- 172.21.15.143:0/1176171319 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559a8f942ad0 0x559a8f972b30 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:14.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.222+0000 7f22f3e66b80 1 -- 172.21.15.143:0/1176171319 >> 172.21.15.143:0/1176171319 conn(0x559a8f8bd660 msgr2=0x559a8f93e7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:14.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.222+0000 7f22f3e66b80 1 -- 172.21.15.143:0/1176171319 shutdown_connections 2023-11-08T19:28:14.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.222+0000 7f22f3e66b80 1 -- 172.21.15.143:0/1176171319 wait complete. 2023-11-08T19:28:14.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.223+0000 7f22f3e66b80 1 -- start start 2023-11-08T19:28:14.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.223+0000 7f22f3e66b80 1 Processor -- start 2023-11-08T19:28:14.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.224+0000 7f22f3e66b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559a8f9426f0 0x559a8f9dc940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:14.240 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.224+0000 7f22f3e66b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a8f942ad0 0x559a8f9ae230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:14.240 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.224+0000 7f22f3e66b80 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x559a8f974290 0x559a8f9ae770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:14.240 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.224+0000 7f22f3e66b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a8f974dd0 con 0x559a8f9426f0 2023-11-08T19:28:14.240 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.224+0000 7f22f3e66b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a8f987d20 con 0x559a8f942ad0 2023-11-08T19:28:14.240 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.224+0000 7f22f3e66b80 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x559a8f9a47f0 con 0x559a8f974290 2023-11-08T19:28:14.241 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.224+0000 7f22f3e66b80 10 client.0 _ll_register_callbacks cb 0x7f22d0012ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T19:28:14.241 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.224+0000 7f22e5a09700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559a8f9426f0 0x559a8f9dc940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:14.241 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.224+0000 7f22e5a09700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559a8f9426f0 0x559a8f9dc940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54784/0 (socket says 172.21.15.143:54784) 2023-11-08T19:28:14.241 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.224+0000 7f22e5a09700 1 -- 172.21.15.143:0/3493275194 learned_addr learned my addr 172.21.15.143:0/3493275194 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:28:14.241 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.224+0000 7f22e620a700 1 --2- 172.21.15.143:0/3493275194 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x559a8f974290 0x559a8f9ae770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:14.242 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.224+0000 7f22e5208700 1 --2- 172.21.15.143:0/3493275194 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a8f942ad0 0x559a8f9ae230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:14.242 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.224+0000 7f22e5a09700 1 -- 172.21.15.143:0/3493275194 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x559a8f974290 msgr2=0x559a8f9ae770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:14.242 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.224+0000 7f22e5a09700 1 --2- 172.21.15.143:0/3493275194 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x559a8f974290 0x559a8f9ae770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:14.242 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.224+0000 7f22e5a09700 1 -- 172.21.15.143:0/3493275194 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a8f942ad0 msgr2=0x559a8f9ae230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:14.242 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.224+0000 7f22e5a09700 1 --2- 172.21.15.143:0/3493275194 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a8f942ad0 0x559a8f9ae230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:14.243 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.224+0000 7f22e5a09700 1 -- 172.21.15.143:0/3493275194 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22e00ce750 con 0x559a8f9426f0 2023-11-08T19:28:14.243 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.224+0000 7f22e620a700 1 --2- 172.21.15.143:0/3493275194 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x559a8f974290 0x559a8f9ae770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:28:14.243 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.224+0000 7f22e5208700 1 --2- 172.21.15.143:0/3493275194 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a8f942ad0 0x559a8f9ae230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:28:14.243 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.224+0000 7f22e5a09700 1 --2- 172.21.15.143:0/3493275194 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559a8f9426f0 0x559a8f9dc940 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f22d0006da0 tx=0x7f22d0013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91befd455cd227a2 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:14.243 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.224+0000 7f22c0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2023-11-08T19:28:14.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.224+0000 7f22c0ff9700 1 -- 172.21.15.143:0/3493275194 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d000b070 con 0x559a8f9426f0 2023-11-08T19:28:14.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.224+0000 7f22c0ff9700 1 -- 172.21.15.143:0/3493275194 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 6772+0+0 (secure 0 0 0) 0x7f22d002e040 con 0x559a8f9426f0 2023-11-08T19:28:14.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.224+0000 7f22f3e66b80 1 -- 172.21.15.143:0/3493275194 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f22d0017070 con 0x559a8f9426f0 2023-11-08T19:28:14.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.224+0000 7f22c0ff9700 1 -- 172.21.15.143:0/3493275194 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f22d0008e80 con 0x559a8f9426f0 2023-11-08T19:28:14.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.224+0000 7f22f3e66b80 10 client.7107 fetch_fsmap learned FSMap version 52 2023-11-08T19:28:14.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.225+0000 7f22f3e66b80 1 -- 172.21.15.143:0/3493275194 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) v3 -- 0x559a8f6eb200 con 0x559a8f9426f0 2023-11-08T19:28:14.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.225+0000 7f22c0ff9700 1 -- 172.21.15.143:0/3493275194 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22d0017070 con 0x559a8f9426f0 2023-11-08T19:28:14.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.225+0000 7f22c0ff9700 1 -- 172.21.15.143:0/3493275194 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 52) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f22d0033680 con 0x559a8f9426f0 2023-11-08T19:28:14.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.225+0000 7f22f3e66b80 10 client.7107 fetch_fsmap finished waiting for FSMap version 52 2023-11-08T19:28:14.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.225+0000 7f22f3e66b80 10 client.7107 resolve_mds: resolved name 'd' to daemon mds.d 2023-11-08T19:28:14.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.225+0000 7f22f3e66b80 20 client.7107 populate_metadata read hostname 'smithi143' 2023-11-08T19:28:14.246 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.225+0000 7f22f3e66b80 1 --2- 172.21.15.143:0/3493275194 >> [v2:172.21.15.174:6835/2201447629,v1:172.21.15.174:6837/2201447629] conn(0x559a8f9a5240 0x559a8f9a5620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:14.246 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.225+0000 7f22f3e66b80 4 client.7107 mds_command: new command op to 6921 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-08T19:28:14.246 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.225+0000 7f22f3e66b80 1 -- 172.21.15.143:0/3493275194 --> [v2:172.21.15.174:6835/2201447629,v1:172.21.15.174:6837/2201447629] -- command(tid 0: {"prefix": "status"}) v1 -- 0x559a8f8945a0 con 0x559a8f9a5240 2023-11-08T19:28:14.246 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.225+0000 7f22e5208700 1 --2- 172.21.15.143:0/3493275194 >> [v2:172.21.15.174:6835/2201447629,v1:172.21.15.174:6837/2201447629] conn(0x559a8f9a5240 0x559a8f9a5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:14.246 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.225+0000 7f22c0ff9700 10 client.7107 ms_handle_connect on v2:172.21.15.174:6835/2201447629 2023-11-08T19:28:14.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.225+0000 7f22e5208700 1 --2- 172.21.15.143:0/3493275194 >> [v2:172.21.15.174:6835/2201447629,v1:172.21.15.174:6837/2201447629] conn(0x559a8f9a5240 0x559a8f9a5620 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=371e69447ec739 server_cookie=2ce17036d597463c in_seq=0 out_seq=0 2023-11-08T19:28:14.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.225+0000 7f22c0ff9700 1 -- 172.21.15.143:0/3493275194 <== mds.0 v2:172.21.15.174:6835/2201447629 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x559a8f8945a0 con 0x559a8f9a5240 2023-11-08T19:28:14.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.225+0000 7f22c0ff9700 10 client.7107 handle_command_reply: tid=0 2023-11-08T19:28:14.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.226+0000 7f22f3e66b80 1 client.7107 shutdown 2023-11-08T19:28:14.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.226+0000 7f22f3e66b80 1 -- 172.21.15.143:0/3493275194 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559a8f9426f0 msgr2=0x559a8f9dc940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:14.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.226+0000 7f22f3e66b80 1 --2- 172.21.15.143:0/3493275194 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559a8f9426f0 0x559a8f9dc940 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f22d0006da0 tx=0x7f22d0013000 comp rx=0 tx=0).stop 2023-11-08T19:28:14.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.226+0000 7f22f3e66b80 1 -- 172.21.15.143:0/3493275194 shutdown_connections 2023-11-08T19:28:14.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.226+0000 7f22f3e66b80 1 --2- 172.21.15.143:0/3493275194 >> [v2:172.21.15.174:6835/2201447629,v1:172.21.15.174:6837/2201447629] conn(0x559a8f9a5240 0x559a8f9a5620 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:14.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.226+0000 7f22f3e66b80 1 --2- 172.21.15.143:0/3493275194 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x559a8f974290 0x559a8f9ae770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:14.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.226+0000 7f22c0ff9700 0 client.7107 ms_handle_reset on v2:172.21.15.174:6835/2201447629 2023-11-08T19:28:14.249 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.226+0000 7f22f3e66b80 1 --2- 172.21.15.143:0/3493275194 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559a8f9426f0 0x559a8f9dc940 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:14.249 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.226+0000 7f22f3e66b80 1 --2- 172.21.15.143:0/3493275194 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a8f942ad0 0x559a8f9ae230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:14.249 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.226+0000 7f22f3e66b80 1 -- 172.21.15.143:0/3493275194 >> 172.21.15.143:0/3493275194 conn(0x559a8f8bd660 msgr2=0x559a8f985210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:14.249 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.226+0000 7f22f3e66b80 1 -- 172.21.15.143:0/3493275194 shutdown_connections 2023-11-08T19:28:14.249 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.226+0000 7f22f3e66b80 1 -- 172.21.15.143:0/3493275194 wait complete. 2023-11-08T19:28:14.249 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.226+0000 7f22f3e66b80 20 client.7107 trim_cache size 0 max 16384 2023-11-08T19:28:14.250 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2023-11-08T19:28:14.250 INFO:teuthology.orchestra.run.smithi143.stdout: "cluster_fsid": "968483af-77fc-4068-ad7c-5c2a1a4613d7", 2023-11-08T19:28:14.250 INFO:teuthology.orchestra.run.smithi143.stdout: "whoami": 0, 2023-11-08T19:28:14.251 INFO:teuthology.orchestra.run.smithi143.stdout: "id": 6921, 2023-11-08T19:28:14.251 INFO:teuthology.orchestra.run.smithi143.stdout: "want_state": "up:active", 2023-11-08T19:28:14.251 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "up:active", 2023-11-08T19:28:14.251 INFO:teuthology.orchestra.run.smithi143.stdout: "fs_name": "cephfs", 2023-11-08T19:28:14.251 INFO:teuthology.orchestra.run.smithi143.stdout: "rank_uptime": 10.532095628, 2023-11-08T19:28:14.252 INFO:teuthology.orchestra.run.smithi143.stdout: "mdsmap_epoch": 52, 2023-11-08T19:28:14.252 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_epoch": 92, 2023-11-08T19:28:14.252 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_epoch_barrier": 92, 2023-11-08T19:28:14.252 INFO:teuthology.orchestra.run.smithi143.stdout: "uptime": 16.423064187000001 2023-11-08T19:28:14.252 INFO:teuthology.orchestra.run.smithi143.stdout:} 2023-11-08T19:28:14.254 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.226+0000 7f22e7c6d700 1 -- 172.21.15.143:0/3638109487 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f22c406af00 msgr2=0x7f22c406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:14.254 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.226+0000 7f22e7c6d700 1 --2- 172.21.15.143:0/3638109487 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f22c406af00 0x7f22c406d3c0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f22d0006fd0 tx=0x7f22d0008040 comp rx=0 tx=0).stop 2023-11-08T19:28:14.254 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.226+0000 7f22e7c6d700 1 -- 172.21.15.143:0/3638109487 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f22e015aa30 msgr2=0x7f22e0153df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:14.254 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.226+0000 7f22e7c6d700 1 --2- 172.21.15.143:0/3638109487 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f22e015aa30 0x7f22e0153df0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f22d800cc40 tx=0x7f22d800a060 comp rx=0 tx=0).stop 2023-11-08T19:28:14.254 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.226+0000 7f22e7c6d700 1 -- 172.21.15.143:0/3638109487 shutdown_connections 2023-11-08T19:28:14.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.226+0000 7f22e7c6d700 1 --2- 172.21.15.143:0/3638109487 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f22c406af00 0x7f22c406d3c0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:14.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.226+0000 7f22e7c6d700 1 --2- 172.21.15.143:0/3638109487 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f22e015aa30 0x7f22e0153df0 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:14.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.226+0000 7f22e7c6d700 1 --2- 172.21.15.143:0/3638109487 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f22e00b2210 0x7f22e00b33b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:14.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.226+0000 7f22e7c6d700 1 --2- 172.21.15.143:0/3638109487 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f22e00cbe20 0x7f22e00c5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:14.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.226+0000 7f22e7c6d700 1 -- 172.21.15.143:0/3638109487 >> 172.21.15.143:0/3638109487 conn(0x7f22e0099ab0 msgr2=0x7f22e00ad530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:14.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.226+0000 7f22e7c6d700 1 -- 172.21.15.143:0/3638109487 shutdown_connections 2023-11-08T19:28:14.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.226+0000 7f22e7c6d700 1 -- 172.21.15.143:0/3638109487 wait complete. 2023-11-08T19:28:14.256 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-08T19:28:14.376 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.375+0000 7f46524ba700 1 Processor -- start 2023-11-08T19:28:14.377 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.375+0000 7f46524ba700 1 -- start start 2023-11-08T19:28:14.382 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.376+0000 7f46524ba700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f464c15aa30 0x7f464c0cf510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:14.382 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.376+0000 7f46524ba700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f464c0cf080 con 0x7f464c0cc530 2023-11-08T19:28:14.383 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.376+0000 7f46524ba700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f464c0ce470 con 0x7f464c0b2210 2023-11-08T19:28:14.383 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.376+0000 7f46524ba700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f464c0ce750 con 0x7f464c15aa30 2023-11-08T19:28:14.383 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.376+0000 7f4650a57700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f464c0cc530 0x7f464c06db40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:50950/0 (socket says 172.21.15.143:50950) 2023-11-08T19:28:14.383 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.376+0000 7f4650a57700 1 -- 172.21.15.143:0/313325561 learned_addr learned my addr 172.21.15.143:0/313325561 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:28:14.383 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.376+0000 7f464bfff700 1 --2- 172.21.15.143:0/313325561 >> v2:172.21.15.174:3300/0 conn(0x7f464c15aa30 0x7f464c0cf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:14.384 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.376+0000 7f464affd700 1 -- 172.21.15.143:0/313325561 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 541648732 0 0) 0x7f464c0cf080 con 0x7f464c0cc530 2023-11-08T19:28:14.384 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.376+0000 7f464affd700 1 -- 172.21.15.143:0/313325561 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f46340035b0 con 0x7f464c0cc530 2023-11-08T19:28:14.384 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.376+0000 7f464affd700 1 -- 172.21.15.143:0/313325561 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4230121737 0 0) 0x7f46340035b0 con 0x7f464c0cc530 2023-11-08T19:28:14.384 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.376+0000 7f464affd700 1 -- 172.21.15.143:0/313325561 >> v2:172.21.15.174:3300/0 conn(0x7f464c15aa30 msgr2=0x7f464c0cf510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:14.384 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.376+0000 7f464affd700 1 --2- 172.21.15.143:0/313325561 >> v2:172.21.15.174:3300/0 conn(0x7f464c15aa30 0x7f464c0cf510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:14.385 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.376+0000 7f464affd700 1 -- 172.21.15.143:0/313325561 >> v1:172.21.15.174:6789/0 conn(0x7f464c0b2210 legacy=0x7f464c0cbe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:14.385 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.376+0000 7f464affd700 1 -- 172.21.15.143:0/313325561 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f464c0cea30 con 0x7f464c0cc530 2023-11-08T19:28:14.385 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.376+0000 7f464affd700 1 -- 172.21.15.143:0/313325561 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4147756303 0 0) 0x7f46440019f0 con 0x7f464c0cc530 2023-11-08T19:28:14.385 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.376+0000 7f464affd700 1 -- 172.21.15.143:0/313325561 >> v1:172.21.15.143:6789/0 conn(0x7f464c0cc530 legacy=0x7f464c06db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:14.385 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.376+0000 7f464affd700 1 --2- 172.21.15.143:0/313325561 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4634003a00 0x7f4634003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:14.386 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.376+0000 7f464affd700 1 --2- 172.21.15.143:0/313325561 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4634004850 0x7f4634004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:14.386 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.377+0000 7f464affd700 1 --2- 172.21.15.143:0/313325561 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4634005210 0x7f463400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:14.386 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.377+0000 7f464affd700 1 -- 172.21.15.143:0/313325561 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f464c0ce750 con 0x7f4634005210 2023-11-08T19:28:14.386 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.377+0000 7f464affd700 1 -- 172.21.15.143:0/313325561 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f463400c0b0 con 0x7f4634003a00 2023-11-08T19:28:14.386 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.377+0000 7f464affd700 1 -- 172.21.15.143:0/313325561 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f463400e480 con 0x7f4634004850 2023-11-08T19:28:14.387 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.377+0000 7f4650a57700 1 --2- 172.21.15.143:0/313325561 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4634005210 0x7f463400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:14.387 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.377+0000 7f464bfff700 1 --2- 172.21.15.143:0/313325561 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4634003a00 0x7f4634003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:14.387 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.377+0000 7f464b7fe700 1 --2- 172.21.15.143:0/313325561 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4634004850 0x7f4634004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:14.387 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.377+0000 7f4650a57700 1 -- 172.21.15.143:0/313325561 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4634004850 msgr2=0x7f4634004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:14.387 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.377+0000 7f4650a57700 1 --2- 172.21.15.143:0/313325561 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4634004850 0x7f4634004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:14.388 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.377+0000 7f4650a57700 1 -- 172.21.15.143:0/313325561 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4634003a00 msgr2=0x7f4634003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:14.388 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.377+0000 7f4650a57700 1 --2- 172.21.15.143:0/313325561 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4634003a00 0x7f4634003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:14.388 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.377+0000 7f4650a57700 1 -- 172.21.15.143:0/313325561 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46340035b0 con 0x7f4634005210 2023-11-08T19:28:14.388 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.377+0000 7f4650a57700 1 -- 172.21.15.143:0/313325561 reap_dead start 2023-11-08T19:28:14.388 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.377+0000 7f4650a57700 1 --2- 172.21.15.143:0/313325561 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4634005210 0x7f463400ba10 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f464400a830 tx=0x7f4634005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96e88ac69e3e1737 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:14.389 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.377+0000 7f464affd700 1 -- 172.21.15.143:0/313325561 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f464400a2f0 con 0x7f4634005210 2023-11-08T19:28:14.389 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.377+0000 7f464affd700 1 -- 172.21.15.143:0/313325561 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4644012620 con 0x7f4634005210 2023-11-08T19:28:14.389 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.377+0000 7f464affd700 1 -- 172.21.15.143:0/313325561 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46440129d0 con 0x7f4634005210 2023-11-08T19:28:14.389 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.378+0000 7f46524ba700 1 -- 172.21.15.143:0/313325561 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4634005210 msgr2=0x7f463400ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:14.389 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.378+0000 7f46524ba700 1 --2- 172.21.15.143:0/313325561 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4634005210 0x7f463400ba10 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f464400a830 tx=0x7f4634005880 comp rx=0 tx=0).stop 2023-11-08T19:28:14.390 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.378+0000 7f46524ba700 1 -- 172.21.15.143:0/313325561 shutdown_connections 2023-11-08T19:28:14.390 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.378+0000 7f46524ba700 1 --2- 172.21.15.143:0/313325561 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4634005210 0x7f463400ba10 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:14.390 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.378+0000 7f46524ba700 1 -- 172.21.15.143:0/313325561 >> 172.21.15.143:0/313325561 conn(0x7f464c099ab0 msgr2=0x7f464c0d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:14.390 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.378+0000 7f46524ba700 1 -- 172.21.15.143:0/313325561 shutdown_connections 2023-11-08T19:28:14.390 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.378+0000 7f46524ba700 1 -- 172.21.15.143:0/313325561 wait complete. 2023-11-08T19:28:14.391 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.379+0000 7f46524ba700 1 Processor -- start 2023-11-08T19:28:14.391 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.379+0000 7f46524ba700 1 -- start start 2023-11-08T19:28:14.391 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.380+0000 7f46524ba700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4634005210 0x7f464c152ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:14.391 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.380+0000 7f46524ba700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f464c0c5b60 0x7f464c0c5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:14.391 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.380+0000 7f46524ba700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f464c0ae980 0x7f464c0aee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:14.391 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.380+0000 7f46524ba700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f464c0b3dc0 con 0x7f4634005210 2023-11-08T19:28:14.392 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.380+0000 7f46524ba700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f464c0af340 con 0x7f464c0c5b60 2023-11-08T19:28:14.392 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.380+0000 7f46524ba700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f464c1536a0 con 0x7f464c0ae980 2023-11-08T19:28:14.392 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.380+0000 7f464bfff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4634005210 0x7f464c152ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:14.392 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.380+0000 7f464bfff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4634005210 0x7f464c152ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54804/0 (socket says 172.21.15.143:54804) 2023-11-08T19:28:14.392 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.380+0000 7f464bfff700 1 -- 172.21.15.143:0/2586705415 learned_addr learned my addr 172.21.15.143:0/2586705415 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:28:14.393 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.380+0000 7f4650a57700 1 --2- 172.21.15.143:0/2586705415 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f464c0ae980 0x7f464c0aee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:14.393 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.380+0000 7f464b7fe700 1 --2- 172.21.15.143:0/2586705415 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f464c0c5b60 0x7f464c0c5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:14.393 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.380+0000 7f464bfff700 1 -- 172.21.15.143:0/2586705415 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f464c0ae980 msgr2=0x7f464c0aee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:14.393 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.380+0000 7f464bfff700 1 --2- 172.21.15.143:0/2586705415 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f464c0ae980 0x7f464c0aee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:14.393 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.380+0000 7f464bfff700 1 -- 172.21.15.143:0/2586705415 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f464c0c5b60 msgr2=0x7f464c0c5fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:14.394 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.380+0000 7f464bfff700 1 --2- 172.21.15.143:0/2586705415 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f464c0c5b60 0x7f464c0c5fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:14.394 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.380+0000 7f464bfff700 1 -- 172.21.15.143:0/2586705415 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f464c0b3fd0 con 0x7f4634005210 2023-11-08T19:28:14.394 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.380+0000 7f464b7fe700 1 --2- 172.21.15.143:0/2586705415 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f464c0c5b60 0x7f464c0c5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:28:14.394 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.380+0000 7f4650a57700 1 --2- 172.21.15.143:0/2586705415 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f464c0ae980 0x7f464c0aee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:28:14.394 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.380+0000 7f464bfff700 1 --2- 172.21.15.143:0/2586705415 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4634005210 0x7f464c152ce0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f46400119a0 tx=0x7f4640011d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:14.395 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.380+0000 7f4648ff9700 1 -- 172.21.15.143:0/2586705415 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4640005c30 con 0x7f4634005210 2023-11-08T19:28:14.395 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.380+0000 7f4648ff9700 1 -- 172.21.15.143:0/2586705415 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4640005d90 con 0x7f4634005210 2023-11-08T19:28:14.395 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.380+0000 7f46524ba700 1 -- 172.21.15.143:0/2586705415 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f464c0b42b0 con 0x7f4634005210 2023-11-08T19:28:14.395 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.381+0000 7f4648ff9700 1 -- 172.21.15.143:0/2586705415 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f464000f910 con 0x7f4634005210 2023-11-08T19:28:14.395 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.381+0000 7f46524ba700 1 -- 172.21.15.143:0/2586705415 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f464c0b4870 con 0x7f4634005210 2023-11-08T19:28:14.397 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.382+0000 7f46524ba700 1 -- 172.21.15.143:0/2586705415 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f464c04eee0 con 0x7f4634005210 2023-11-08T19:28:14.397 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.382+0000 7f4648ff9700 1 -- 172.21.15.143:0/2586705415 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f464000fa70 con 0x7f4634005210 2023-11-08T19:28:14.397 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.382+0000 7f4648ff9700 1 --2- 172.21.15.143:0/2586705415 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f463806af00 0x7f463806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:14.397 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.382+0000 7f4648ff9700 1 -- 172.21.15.143:0/2586705415 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6772+0+0 (secure 0 0 0) 0x7f4640089fc0 con 0x7f4634005210 2023-11-08T19:28:14.398 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.383+0000 7f464b7fe700 1 --2- 172.21.15.143:0/2586705415 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f463806af00 0x7f463806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:14.398 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.383+0000 7f464b7fe700 1 --2- 172.21.15.143:0/2586705415 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f463806af00 0x7f463806d3c0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f464c0ca540 tx=0x7f463c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:14.398 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.387+0000 7f4648ff9700 1 -- 172.21.15.143:0/2586705415 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f46400bcca0 con 0x7f4634005210 2023-11-08T19:28:14.619 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.618+0000 7f46524ba700 1 -- 172.21.15.143:0/2586705415 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f464c0d1590 con 0x7f4634005210 2023-11-08T19:28:14.620 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.618+0000 7f4648ff9700 1 -- 172.21.15.143:0/2586705415 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v52) v1 ==== 53+0+73 (secure 0 0 0) 0x7f4640054d80 con 0x7f4634005210 2023-11-08T19:28:14.620 INFO:teuthology.orchestra.run.smithi143.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-08T19:28:14.622 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.620+0000 7f46524ba700 1 -- 172.21.15.143:0/2586705415 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f463806af00 msgr2=0x7f463806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:14.622 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.620+0000 7f46524ba700 1 --2- 172.21.15.143:0/2586705415 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f463806af00 0x7f463806d3c0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f464c0ca540 tx=0x7f463c009040 comp rx=0 tx=0).stop 2023-11-08T19:28:14.622 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.620+0000 7f46524ba700 1 -- 172.21.15.143:0/2586705415 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4634005210 msgr2=0x7f464c152ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:14.623 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.620+0000 7f46524ba700 1 --2- 172.21.15.143:0/2586705415 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4634005210 0x7f464c152ce0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f46400119a0 tx=0x7f4640011d60 comp rx=0 tx=0).stop 2023-11-08T19:28:14.623 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.620+0000 7f46524ba700 1 -- 172.21.15.143:0/2586705415 shutdown_connections 2023-11-08T19:28:14.623 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.620+0000 7f46524ba700 1 --2- 172.21.15.143:0/2586705415 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f463806af00 0x7f463806d3c0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:14.623 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.620+0000 7f46524ba700 1 --2- 172.21.15.143:0/2586705415 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f464c0ae980 0x7f464c0aee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:14.623 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.620+0000 7f46524ba700 1 --2- 172.21.15.143:0/2586705415 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4634005210 0x7f464c152ce0 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:14.624 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.620+0000 7f46524ba700 1 --2- 172.21.15.143:0/2586705415 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f464c0c5b60 0x7f464c0c5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:14.624 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.620+0000 7f46524ba700 1 -- 172.21.15.143:0/2586705415 >> 172.21.15.143:0/2586705415 conn(0x7f464c099ab0 msgr2=0x7f464c159ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:14.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.620+0000 7f46524ba700 1 -- 172.21.15.143:0/2586705415 shutdown_connections 2023-11-08T19:28:14.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:14.620+0000 7f46524ba700 1 -- 172.21.15.143:0/2586705415 wait complete. 2023-11-08T19:28:14.634 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-08T19:28:14.634 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-08T19:28:14.635 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi143.front.sepia.ceph.com 2023-11-08T19:28:14.635 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-08T19:28:14.635 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-08T19:28:14.636 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-08T19:28:14.636 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-08T19:28:14.636 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-08T19:28:14.637 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-08T19:28:14.637 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:28:14.638 DEBUG:teuthology.orchestra.run.smithi143:> ip addr 2023-11-08T19:28:14.655 INFO:teuthology.orchestra.run.smithi143.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-08T19:28:14.655 INFO:teuthology.orchestra.run.smithi143.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-08T19:28:14.655 INFO:teuthology.orchestra.run.smithi143.stdout: inet 127.0.0.1/8 scope host lo 2023-11-08T19:28:14.656 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:28:14.656 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 ::1/128 scope host 2023-11-08T19:28:14.656 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:28:14.656 INFO:teuthology.orchestra.run.smithi143.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-08T19:28:14.657 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:d9:3b:94 brd ff:ff:ff:ff:ff:ff 2023-11-08T19:28:14.657 INFO:teuthology.orchestra.run.smithi143.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-08T19:28:14.657 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:d9:3b:95 brd ff:ff:ff:ff:ff:ff 2023-11-08T19:28:14.657 INFO:teuthology.orchestra.run.smithi143.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-08T19:28:14.657 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:88:77:82 brd ff:ff:ff:ff:ff:ff 2023-11-08T19:28:14.658 INFO:teuthology.orchestra.run.smithi143.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-08T19:28:14.658 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:88:77:83 brd ff:ff:ff:ff:ff:ff 2023-11-08T19:28:14.658 INFO:teuthology.orchestra.run.smithi143.stdout: inet 172.21.15.143/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-08T19:28:14.658 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft 42173sec preferred_lft 42173sec 2023-11-08T19:28:14.658 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 fe80::e628:d259:2f76:52a6/64 scope link noprefixroute 2023-11-08T19:28:14.659 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:28:14.659 INFO:teuthology.orchestra.run.smithi143.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-08T19:28:14.659 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 46:e3:3e:66:77:ea brd ff:ff:ff:ff:ff:ff 2023-11-08T19:28:14.659 INFO:teuthology.orchestra.run.smithi143.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-08T19:28:14.659 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:28:14.660 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 fe80::749a:8aff:fe59:325c/64 scope link 2023-11-08T19:28:14.660 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:28:14.660 INFO:teuthology.orchestra.run.smithi143.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-08T19:28:14.660 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 46:e3:3e:66:77:ea brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-08T19:28:14.660 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 fe80::44e3:3eff:fe66:77ea/64 scope link 2023-11-08T19:28:14.661 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:28:14.661 INFO:teuthology.orchestra.run.smithi143.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-08T19:28:14.661 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether ba:92:23:3f:5e:c8 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-08T19:28:14.661 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 fe80::b892:23ff:fe3f:5ec8/64 scope link 2023-11-08T19:28:14.661 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:28:14.662 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:28:14.662 DEBUG:teuthology.orchestra.run.smithi143:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-08T19:28:14.720 INFO:teuthology.orchestra.run.smithi143.stdout:1 2023-11-08T19:28:14.721 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:28:14.722 DEBUG:teuthology.orchestra.run.smithi143:> route 2023-11-08T19:28:14.781 INFO:teuthology.orchestra.run.smithi143.stdout:Kernel IP routing table 2023-11-08T19:28:14.781 INFO:teuthology.orchestra.run.smithi143.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-08T19:28:14.781 INFO:teuthology.orchestra.run.smithi143.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-08T19:28:14.781 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-08T19:28:14.782 INFO:teuthology.orchestra.run.smithi143.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-08T19:28:14.782 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:28:14.783 DEBUG:teuthology.orchestra.run.smithi143:> (cd / && exec bash -c ' 2023-11-08T19:28:14.783 DEBUG:teuthology.orchestra.run.smithi143:> set -e 2023-11-08T19:28:14.783 DEBUG:teuthology.orchestra.run.smithi143:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-08T19:28:14.784 DEBUG:teuthology.orchestra.run.smithi143:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-08T19:28:14.784 DEBUG:teuthology.orchestra.run.smithi143:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-08T19:28:14.784 DEBUG:teuthology.orchestra.run.smithi143:> ') 2023-11-08T19:28:14.885 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:28:14.886 DEBUG:teuthology.orchestra.run.smithi143:> ip netns list 2023-11-08T19:28:14.903 INFO:teuthology.orchestra.run.smithi143.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-08T19:28:14.903 INFO:teuthology.orchestra.run.smithi143.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-08T19:28:14.903 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-08T19:28:14.904 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:28:14.904 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip link set brx.0 up 2023-11-08T19:28:14.970 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-08T19:28:14.970 DEBUG:teuthology.orchestra.run.smithi143:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-08T19:28:15.027 INFO:teuthology.orchestra.run.smithi143.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-08T19:28:15.027 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-08T19:28:15.028 DEBUG:teuthology.orchestra.run.smithi143:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-08T19:28:15.083 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-11-08T19:28:15.084 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:28:15.084 DEBUG:teuthology.orchestra.run.smithi143:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-08T19:28:15.169 INFO:teuthology.orchestra.run.smithi143.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-08T19:28:15.169 INFO:teuthology.orchestra.run.smithi143.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-11-08T19:28:15.170 INFO:teuthology.orchestra.run.smithi143.stdout:invalid new device string format 2023-11-08T19:28:15.178 INFO:teuthology.orchestra.run.smithi143.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-08T19:28:15.178 INFO:teuthology.orchestra.run.smithi143.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-11-08T19:28:15.178 INFO:teuthology.orchestra.run.smithi143.stdout:invalid new device string format 2023-11-08T19:28:15.178 INFO:teuthology.orchestra.run.smithi143.stdout:mount.ceph: resolved to: "172.21.15.143:3300,172.21.15.174:3300,172.21.15.174:3301" 2023-11-08T19:28:15.179 INFO:teuthology.orchestra.run.smithi143.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-08T19:28:15.252 INFO:teuthology.orchestra.run.smithi143.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-08T19:28:15.253 INFO:teuthology.orchestra.run.smithi143.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-11-08T19:28:15.253 INFO:teuthology.orchestra.run.smithi143.stdout:invalid new device string format 2023-11-08T19:28:15.253 INFO:teuthology.orchestra.run.smithi143.stdout:mount.ceph: resolved to: "172.21.15.143:3300,172.21.15.174:3300,172.21.15.174:3301" 2023-11-08T19:28:15.253 INFO:teuthology.orchestra.run.smithi143.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-08T19:28:15.254 INFO:teuthology.orchestra.run.smithi143.stdout:mount.ceph: resolved to: "172.21.15.143:6789,172.21.15.174:6789,172.21.15.174:6790" 2023-11-08T19:28:15.254 INFO:teuthology.orchestra.run.smithi143.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.143:6789,172.21.15.174:6789,172.21.15.174:6790:/ 2023-11-08T19:28:15.254 INFO:teuthology.orchestra.run.smithi143.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=968483af-77fc-4068-ad7c-5c2a1a4613d7" will pass to kernel 2023-11-08T19:28:15.255 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-08T19:28:15.255 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:28:15.256 DEBUG:teuthology.orchestra.run.smithi143:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-08T19:28:15.283 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:28:15.283 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-08T19:28:15.371 INFO:teuthology.orchestra.run.smithi143.stdout:client5864 2023-11-08T19:28:15.372 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-08T19:28:15.372 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:28:15.389 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config get mds_max_snaps_per_dir 2023-11-08T19:28:15.563 INFO:teuthology.orchestra.run.smithi143.stdout:{"mds_max_snaps_per_dir":"100"} 2023-11-08T19:28:15.563 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_max_snaps_per_dir": "100" } 2023-11-08T19:28:15.564 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_new_snaps true --yes-i-really-mean-it 2023-11-08T19:28:15.692 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:15.688+0000 7fa9d248d700 1 Processor -- start 2023-11-08T19:28:15.692 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:15.688+0000 7fa9d248d700 1 -- start start 2023-11-08T19:28:15.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:15.689+0000 7fa9d248d700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa9cc15df20 0x7fa9cc0b25f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:15.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:15.689+0000 7fa9d248d700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa9cc06bd20 0x7fa9cc0d19c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:15.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:15.689+0000 7fa9d248d700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9cc0b2b30 con 0x7fa9cc0d1f00 2023-11-08T19:28:15.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:15.689+0000 7fa9d248d700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9cc0d2390 con 0x7fa9cc06bd20 2023-11-08T19:28:15.694 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:15.689+0000 7fa9d248d700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9cc06c2c0 con 0x7fa9cc15df20 2023-11-08T19:28:15.694 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:15.689+0000 7fa9cb7fe700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa9cc06bd20 0x7fa9cc0d19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:15.694 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:15.689+0000 7fa9cb7fe700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa9cc06bd20 0x7fa9cc0d19c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54816/0 (socket says 172.21.15.143:54816) 2023-11-08T19:28:15.694 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:15.689+0000 7fa9cb7fe700 1 -- 172.21.15.143:0/580650228 learned_addr learned my addr 172.21.15.143:0/580650228 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:28:15.695 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:15.689+0000 7fa9cbfff700 1 --2- 172.21.15.143:0/580650228 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa9cc15df20 0x7fa9cc0b25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:15.695 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:15.689+0000 7fa9cb7fe700 1 -- 172.21.15.143:0/580650228 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa9cc15df20 msgr2=0x7fa9cc0b25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:15.695 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:15.689+0000 7fa9cb7fe700 1 --2- 172.21.15.143:0/580650228 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa9cc15df20 0x7fa9cc0b25f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:15.695 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:15.689+0000 7fa9cb7fe700 1 -- 172.21.15.143:0/580650228 >> v1:172.21.15.174:6789/0 conn(0x7fa9cc0d1f00 legacy=0x7fa9cc0ce2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:15.695 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:15.689+0000 7fa9cb7fe700 1 -- 172.21.15.143:0/580650228 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9cc06c5a0 con 0x7fa9cc06bd20 2023-11-08T19:28:15.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:15.690+0000 7fa9cb7fe700 1 --2- 172.21.15.143:0/580650228 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa9cc06bd20 0x7fa9cc0d19c0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7fa9bc0098f0 tx=0x7fa9bc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e8c6646aced4d1b server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:15.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:15.690+0000 7fa9caffd700 1 -- 172.21.15.143:0/580650228 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9bc004230 con 0x7fa9cc06bd20 2023-11-08T19:28:15.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:15.690+0000 7fa9caffd700 1 -- 172.21.15.143:0/580650228 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9bc004390 con 0x7fa9cc06bd20 2023-11-08T19:28:15.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:15.690+0000 7fa9caffd700 1 -- 172.21.15.143:0/580650228 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9bc0036a0 con 0x7fa9cc06bd20 2023-11-08T19:28:15.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:15.690+0000 7fa9d248d700 1 -- 172.21.15.143:0/580650228 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa9cc06bd20 msgr2=0x7fa9cc0d19c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:15.697 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:15.690+0000 7fa9d248d700 1 --2- 172.21.15.143:0/580650228 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa9cc06bd20 0x7fa9cc0d19c0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7fa9bc0098f0 tx=0x7fa9bc009c00 comp rx=0 tx=0).stop 2023-11-08T19:28:15.697 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:15.690+0000 7fa9d248d700 1 -- 172.21.15.143:0/580650228 shutdown_connections 2023-11-08T19:28:15.697 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:15.690+0000 7fa9d248d700 1 --2- 172.21.15.143:0/580650228 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa9cc15df20 0x7fa9cc0b25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:15.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:15.690+0000 7fa9d248d700 1 --2- 172.21.15.143:0/580650228 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa9cc06bd20 0x7fa9cc0d19c0 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:15.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:15.690+0000 7fa9d248d700 1 -- 172.21.15.143:0/580650228 >> 172.21.15.143:0/580650228 conn(0x7fa9cc08b000 msgr2=0x7fa9cc09ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:15.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:15.690+0000 7fa9d248d700 1 -- 172.21.15.143:0/580650228 shutdown_connections 2023-11-08T19:28:15.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:15.690+0000 7fa9d248d700 1 -- 172.21.15.143:0/580650228 wait complete. 2023-11-08T19:28:15.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:15.691+0000 7fa9d248d700 1 Processor -- start 2023-11-08T19:28:15.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:15.692+0000 7fa9d248d700 1 -- start start 2023-11-08T19:28:15.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:15.692+0000 7fa9d248d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa9cc06bd20 0x7fa9cc0caf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:15.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:15.692+0000 7fa9d248d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa9cc0d1f00 0x7fa9cc0b7210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:15.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:15.692+0000 7fa9d248d700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa9cc15df20 0x7fa9cc160b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:15.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:15.692+0000 7fa9d248d700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9cc15fab0 con 0x7fa9cc06bd20 2023-11-08T19:28:15.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:15.692+0000 7fa9d248d700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9cc15fd90 con 0x7fa9cc0d1f00 2023-11-08T19:28:15.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:15.692+0000 7fa9d248d700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9cc160070 con 0x7fa9cc15df20 2023-11-08T19:28:15.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:15.692+0000 7fa9cbfff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa9cc06bd20 0x7fa9cc0caf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:15.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:15.693+0000 7fa9cbfff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa9cc06bd20 0x7fa9cc0caf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54822/0 (socket says 172.21.15.143:54822) 2023-11-08T19:28:15.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:15.693+0000 7fa9cbfff700 1 -- 172.21.15.143:0/2837517932 learned_addr learned my addr 172.21.15.143:0/2837517932 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:28:15.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:15.693+0000 7fa9d0a2a700 1 --2- 172.21.15.143:0/2837517932 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa9cc15df20 0x7fa9cc160b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:15.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:15.693+0000 7fa9cb7fe700 1 --2- 172.21.15.143:0/2837517932 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa9cc0d1f00 0x7fa9cc0b7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:15.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:15.693+0000 7fa9cbfff700 1 -- 172.21.15.143:0/2837517932 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa9cc15df20 msgr2=0x7fa9cc160b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:15.705 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:15.693+0000 7fa9cbfff700 1 --2- 172.21.15.143:0/2837517932 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa9cc15df20 0x7fa9cc160b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:15.705 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:15.693+0000 7fa9cbfff700 1 -- 172.21.15.143:0/2837517932 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa9cc0d1f00 msgr2=0x7fa9cc0b7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:15.705 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:15.693+0000 7fa9cbfff700 1 --2- 172.21.15.143:0/2837517932 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa9cc0d1f00 0x7fa9cc0b7210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:15.705 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:15.693+0000 7fa9cbfff700 1 -- 172.21.15.143:0/2837517932 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9bc009550 con 0x7fa9cc06bd20 2023-11-08T19:28:15.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:15.693+0000 7fa9cb7fe700 1 --2- 172.21.15.143:0/2837517932 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa9cc0d1f00 0x7fa9cc0b7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:28:15.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:15.693+0000 7fa9cbfff700 1 --2- 172.21.15.143:0/2837517932 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa9cc06bd20 0x7fa9cc0caf40 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fa9c000c7f0 tx=0x7fa9c000cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:15.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:15.693+0000 7fa9c8ff9700 1 -- 172.21.15.143:0/2837517932 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9c0011800 con 0x7fa9cc06bd20 2023-11-08T19:28:15.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:15.693+0000 7fa9c8ff9700 1 -- 172.21.15.143:0/2837517932 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9c0011960 con 0x7fa9cc06bd20 2023-11-08T19:28:15.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:15.693+0000 7fa9d248d700 1 -- 172.21.15.143:0/2837517932 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9cc160350 con 0x7fa9cc06bd20 2023-11-08T19:28:15.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:15.693+0000 7fa9c8ff9700 1 -- 172.21.15.143:0/2837517932 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9c0011cf0 con 0x7fa9cc06bd20 2023-11-08T19:28:15.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:15.693+0000 7fa9d248d700 1 -- 172.21.15.143:0/2837517932 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9cc0cb480 con 0x7fa9cc06bd20 2023-11-08T19:28:15.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:15.694+0000 7fa9d248d700 1 -- 172.21.15.143:0/2837517932 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9cc160870 con 0x7fa9cc06bd20 2023-11-08T19:28:15.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:15.695+0000 7fa9c8ff9700 1 -- 172.21.15.143:0/2837517932 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa9c0018360 con 0x7fa9cc06bd20 2023-11-08T19:28:15.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:15.695+0000 7fa9c8ff9700 1 --2- 172.21.15.143:0/2837517932 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa9b806af00 0x7fa9b806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:15.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:15.695+0000 7fa9c8ff9700 1 -- 172.21.15.143:0/2837517932 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6772+0+0 (secure 0 0 0) 0x7fa9c0089a40 con 0x7fa9cc06bd20 2023-11-08T19:28:15.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:15.695+0000 7fa9cb7fe700 1 --2- 172.21.15.143:0/2837517932 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa9b806af00 0x7fa9b806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:15.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:15.696+0000 7fa9cb7fe700 1 --2- 172.21.15.143:0/2837517932 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa9b806af00 0x7fa9b806d3c0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc0b14c0 tx=0x7fa9bc008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:15.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:15.699+0000 7fa9c8ff9700 1 -- 172.21.15.143:0/2837517932 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa9cc160870 con 0x7fa9cc06bd20 2023-11-08T19:28:15.936 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:15.934+0000 7fa9d248d700 1 -- 172.21.15.143:0/2837517932 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fa9cc160870 con 0x7fa9cc06bd20 2023-11-08T19:28:16.777 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:16.776+0000 7fa9c8ff9700 1 -- 172.21.15.143:0/2837517932 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true}]=0 enabled new snapshots v53) v1 ==== 167+0+0 (secure 0 0 0) 0x7fa9c008be60 con 0x7fa9cc06bd20 2023-11-08T19:28:16.780 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:16.779+0000 7fa9d248d700 1 -- 172.21.15.143:0/2837517932 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa9b806af00 msgr2=0x7fa9b806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:16.781 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:16.779+0000 7fa9d248d700 1 --2- 172.21.15.143:0/2837517932 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa9b806af00 0x7fa9b806d3c0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc0b14c0 tx=0x7fa9bc008030 comp rx=0 tx=0).stop 2023-11-08T19:28:16.781 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:16.779+0000 7fa9d248d700 1 -- 172.21.15.143:0/2837517932 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa9cc06bd20 msgr2=0x7fa9cc0caf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:16.781 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:16.779+0000 7fa9d248d700 1 --2- 172.21.15.143:0/2837517932 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa9cc06bd20 0x7fa9cc0caf40 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fa9c000c7f0 tx=0x7fa9c000cbb0 comp rx=0 tx=0).stop 2023-11-08T19:28:16.782 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:16.779+0000 7fa9d248d700 1 -- 172.21.15.143:0/2837517932 shutdown_connections 2023-11-08T19:28:16.782 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:16.779+0000 7fa9d248d700 1 --2- 172.21.15.143:0/2837517932 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa9b806af00 0x7fa9b806d3c0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:16.782 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:16.779+0000 7fa9d248d700 1 --2- 172.21.15.143:0/2837517932 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa9cc15df20 0x7fa9cc160b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:16.782 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:16.779+0000 7fa9d248d700 1 --2- 172.21.15.143:0/2837517932 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa9cc06bd20 0x7fa9cc0caf40 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:16.782 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:16.779+0000 7fa9d248d700 1 --2- 172.21.15.143:0/2837517932 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa9cc0d1f00 0x7fa9cc0b7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:16.783 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:16.779+0000 7fa9d248d700 1 -- 172.21.15.143:0/2837517932 >> 172.21.15.143:0/2837517932 conn(0x7fa9cc08b000 msgr2=0x7fa9cc0cca00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:16.783 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:16.779+0000 7fa9d248d700 1 -- 172.21.15.143:0/2837517932 shutdown_connections 2023-11-08T19:28:16.783 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:16.779+0000 7fa9d248d700 1 -- 172.21.15.143:0/2837517932 wait complete. 2023-11-08T19:28:16.783 INFO:teuthology.orchestra.run.smithi143.stderr:enabled new snapshots 2023-11-08T19:28:16.795 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 2 2023-11-08T19:28:16.925 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:16.923+0000 7f0b6535d700 1 Processor -- start 2023-11-08T19:28:16.925 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:16.924+0000 7f0b6535d700 1 -- start start 2023-11-08T19:28:16.929 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:16.924+0000 7f0b6535d700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f0b600b2a10 0x7f0b601623f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:16.929 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:16.924+0000 7f0b6535d700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0b600ce660 0x7f0b6006bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:16.930 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:16.924+0000 7f0b6535d700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b60162930 con 0x7f0b6015dd60 2023-11-08T19:28:16.930 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:16.924+0000 7f0b6535d700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b6006b2b0 con 0x7f0b600b2a10 2023-11-08T19:28:16.930 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:16.924+0000 7f0b6535d700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b600cf000 con 0x7f0b600ce660 2023-11-08T19:28:16.930 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:16.924+0000 7f0b5effd700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f0b6015dd60 0x7f0b600b2300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:50984/0 (socket says 172.21.15.143:50984) 2023-11-08T19:28:16.931 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:16.924+0000 7f0b5effd700 1 -- 172.21.15.143:0/3306424198 learned_addr learned my addr 172.21.15.143:0/3306424198 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:28:16.931 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:16.925+0000 7f0b5dffb700 1 -- 172.21.15.143:0/3306424198 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3874978649 0 0) 0x7f0b54001110 con 0x7f0b6015dd60 2023-11-08T19:28:16.931 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:16.925+0000 7f0b5f7fe700 1 --2- 172.21.15.143:0/3306424198 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0b600ce660 0x7f0b6006bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:16.931 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:16.925+0000 7f0b5e7fc700 1 --2- 172.21.15.143:0/3306424198 >> v2:172.21.15.174:3300/0 conn(0x7f0b600b2a10 0x7f0b601623f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:16.931 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:16.925+0000 7f0b5dffb700 1 -- 172.21.15.143:0/3306424198 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b480035b0 con 0x7f0b6015dd60 2023-11-08T19:28:16.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:16.925+0000 7f0b5dffb700 1 -- 172.21.15.143:0/3306424198 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3254370055 0 0) 0x7f0b480035b0 con 0x7f0b6015dd60 2023-11-08T19:28:16.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:16.925+0000 7f0b5dffb700 1 -- 172.21.15.143:0/3306424198 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0b600ce660 msgr2=0x7f0b6006bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:16.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:16.925+0000 7f0b5dffb700 1 --2- 172.21.15.143:0/3306424198 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0b600ce660 0x7f0b6006bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:16.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:16.925+0000 7f0b5dffb700 1 -- 172.21.15.143:0/3306424198 >> v2:172.21.15.174:3300/0 conn(0x7f0b600b2a10 msgr2=0x7f0b601623f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:16.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:16.925+0000 7f0b5dffb700 1 --2- 172.21.15.143:0/3306424198 >> v2:172.21.15.174:3300/0 conn(0x7f0b600b2a10 0x7f0b601623f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:16.933 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:16.925+0000 7f0b5dffb700 1 -- 172.21.15.143:0/3306424198 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b600cf2e0 con 0x7f0b6015dd60 2023-11-08T19:28:16.933 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:16.925+0000 7f0b5dffb700 1 -- 172.21.15.143:0/3306424198 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4147756303 0 0) 0x7f0b54001800 con 0x7f0b6015dd60 2023-11-08T19:28:16.933 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:16.925+0000 7f0b5dffb700 1 -- 172.21.15.143:0/3306424198 >> v1:172.21.15.143:6789/0 conn(0x7f0b6015dd60 legacy=0x7f0b600b2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:16.933 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:16.925+0000 7f0b5dffb700 1 --2- 172.21.15.143:0/3306424198 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0b48003a00 0x7f0b48003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:16.933 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:16.925+0000 7f0b5dffb700 1 --2- 172.21.15.143:0/3306424198 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0b600ce660 0x7f0b48004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:16.934 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:16.925+0000 7f0b5dffb700 1 --2- 172.21.15.143:0/3306424198 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0b48004d90 0x7f0b48005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:16.934 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:16.925+0000 7f0b5dffb700 1 -- 172.21.15.143:0/3306424198 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b6006b2b0 con 0x7f0b48003a00 2023-11-08T19:28:16.934 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:16.925+0000 7f0b5dffb700 1 -- 172.21.15.143:0/3306424198 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b600cf000 con 0x7f0b48004d90 2023-11-08T19:28:16.934 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:16.925+0000 7f0b5dffb700 1 -- 172.21.15.143:0/3306424198 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b4800bb70 con 0x7f0b600ce660 2023-11-08T19:28:16.935 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:16.925+0000 7f0b5effd700 1 --2- 172.21.15.143:0/3306424198 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0b48003a00 0x7f0b48003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:16.935 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:16.925+0000 7f0b5e7fc700 1 --2- 172.21.15.143:0/3306424198 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0b600ce660 0x7f0b48004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:16.935 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:16.925+0000 7f0b5f7fe700 1 --2- 172.21.15.143:0/3306424198 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0b48004d90 0x7f0b48005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:16.935 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:16.925+0000 7f0b5effd700 1 -- 172.21.15.143:0/3306424198 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0b600ce660 msgr2=0x7f0b48004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:16.935 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:16.925+0000 7f0b5effd700 1 --2- 172.21.15.143:0/3306424198 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0b600ce660 0x7f0b48004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:16.936 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:16.925+0000 7f0b5effd700 1 -- 172.21.15.143:0/3306424198 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0b48004d90 msgr2=0x7f0b48005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:16.936 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:16.925+0000 7f0b5effd700 1 --2- 172.21.15.143:0/3306424198 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0b48004d90 0x7f0b48005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:16.936 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:16.925+0000 7f0b5effd700 1 -- 172.21.15.143:0/3306424198 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b480035b0 con 0x7f0b48003a00 2023-11-08T19:28:16.936 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:16.925+0000 7f0b5f7fe700 1 --2- 172.21.15.143:0/3306424198 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0b48004d90 0x7f0b48005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:28:16.936 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:16.926+0000 7f0b5effd700 1 --2- 172.21.15.143:0/3306424198 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0b48003a00 0x7f0b48003e20 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f0b5400c2f0 tx=0x7f0b48005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fe7f74fcc9dcc48 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:16.937 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:16.926+0000 7f0b5dffb700 1 -- 172.21.15.143:0/3306424198 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b5400ce30 con 0x7f0b48003a00 2023-11-08T19:28:16.937 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:16.926+0000 7f0b5dffb700 1 -- 172.21.15.143:0/3306424198 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b54005960 con 0x7f0b48003a00 2023-11-08T19:28:16.937 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:16.926+0000 7f0b5dffb700 1 -- 172.21.15.143:0/3306424198 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b54005d10 con 0x7f0b48003a00 2023-11-08T19:28:16.937 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:16.926+0000 7f0b6535d700 1 -- 172.21.15.143:0/3306424198 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0b48003a00 msgr2=0x7f0b48003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:16.937 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:16.926+0000 7f0b6535d700 1 --2- 172.21.15.143:0/3306424198 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0b48003a00 0x7f0b48003e20 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f0b5400c2f0 tx=0x7f0b48005880 comp rx=0 tx=0).stop 2023-11-08T19:28:16.938 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:16.926+0000 7f0b5f7fe700 1 -- 172.21.15.143:0/3306424198 reap_dead start 2023-11-08T19:28:16.938 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:16.926+0000 7f0b6535d700 1 -- 172.21.15.143:0/3306424198 shutdown_connections 2023-11-08T19:28:16.938 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:16.926+0000 7f0b6535d700 1 -- 172.21.15.143:0/3306424198 >> 172.21.15.143:0/3306424198 conn(0x7f0b600cd300 msgr2=0x7f0b600cd710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:16.938 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:16.926+0000 7f0b6535d700 1 -- 172.21.15.143:0/3306424198 shutdown_connections 2023-11-08T19:28:16.938 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:16.926+0000 7f0b6535d700 1 -- 172.21.15.143:0/3306424198 wait complete. 2023-11-08T19:28:16.939 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:16.928+0000 7f0b6535d700 1 Processor -- start 2023-11-08T19:28:16.939 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:16.928+0000 7f0b6535d700 1 -- start start 2023-11-08T19:28:16.941 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:16.928+0000 7f0b6535d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0b48003a00 0x7f0b600c4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:16.941 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:16.928+0000 7f0b6535d700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0b600aebb0 0x7f0b601f73e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:16.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:16.928+0000 7f0b6535d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0b601f7920 0x7f0b601473a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:16.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:16.928+0000 7f0b6535d700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b601f7da0 con 0x7f0b601f7920 2023-11-08T19:28:16.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:16.928+0000 7f0b6535d700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b600c7a80 con 0x7f0b48003a00 2023-11-08T19:28:16.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:16.928+0000 7f0b6535d700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b600c5cd0 con 0x7f0b600aebb0 2023-11-08T19:28:16.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:16.928+0000 7f0b5f7fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0b601f7920 0x7f0b601473a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:16.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:16.929+0000 7f0b5f7fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0b601f7920 0x7f0b601473a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54846/0 (socket says 172.21.15.143:54846) 2023-11-08T19:28:16.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:16.929+0000 7f0b5f7fe700 1 -- 172.21.15.143:0/337007648 learned_addr learned my addr 172.21.15.143:0/337007648 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:28:16.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:16.929+0000 7f0b5f7fe700 1 -- 172.21.15.143:0/337007648 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0b600aebb0 msgr2=0x7f0b601f73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:16.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:16.929+0000 7f0b5f7fe700 1 --2- 172.21.15.143:0/337007648 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0b600aebb0 0x7f0b601f73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:16.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:16.929+0000 7f0b5f7fe700 1 -- 172.21.15.143:0/337007648 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0b48003a00 msgr2=0x7f0b600c4d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:16.945 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:16.929+0000 7f0b5f7fe700 1 --2- 172.21.15.143:0/337007648 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0b48003a00 0x7f0b600c4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:16.945 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:16.929+0000 7f0b5f7fe700 1 -- 172.21.15.143:0/337007648 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b600c5430 con 0x7f0b601f7920 2023-11-08T19:28:16.946 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:16.929+0000 7f0b5f7fe700 1 --2- 172.21.15.143:0/337007648 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0b601f7920 0x7f0b601473a0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f0b600afb40 tx=0x7f0b58004970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:16.946 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:16.929+0000 7f0b477fe700 1 -- 172.21.15.143:0/337007648 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b58007b00 con 0x7f0b601f7920 2023-11-08T19:28:16.946 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:16.929+0000 7f0b477fe700 1 -- 172.21.15.143:0/337007648 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b5800f050 con 0x7f0b601f7920 2023-11-08T19:28:16.946 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:16.929+0000 7f0b477fe700 1 -- 172.21.15.143:0/337007648 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b58014410 con 0x7f0b601f7920 2023-11-08T19:28:16.946 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:16.929+0000 7f0b6535d700 1 -- 172.21.15.143:0/337007648 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b600c5710 con 0x7f0b601f7920 2023-11-08T19:28:16.947 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:16.929+0000 7f0b6535d700 1 -- 172.21.15.143:0/337007648 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b60152060 con 0x7f0b601f7920 2023-11-08T19:28:16.947 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:16.930+0000 7f0b6535d700 1 -- 172.21.15.143:0/337007648 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b6004eee0 con 0x7f0b601f7920 2023-11-08T19:28:16.947 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:16.931+0000 7f0b477fe700 1 -- 172.21.15.143:0/337007648 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0b58014570 con 0x7f0b601f7920 2023-11-08T19:28:16.947 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:16.931+0000 7f0b477fe700 1 --2- 172.21.15.143:0/337007648 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f0b4c06af00 0x7f0b4c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:16.947 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:16.931+0000 7f0b477fe700 1 -- 172.21.15.143:0/337007648 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6772+0+0 (secure 0 0 0) 0x7f0b58089cf0 con 0x7f0b601f7920 2023-11-08T19:28:16.948 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:16.931+0000 7f0b5effd700 1 --2- 172.21.15.143:0/337007648 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f0b4c06af00 0x7f0b4c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:16.948 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:16.932+0000 7f0b5effd700 1 --2- 172.21.15.143:0/337007648 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f0b4c06af00 0x7f0b4c06d3c0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f0b600728c0 tx=0x7f0b5400f000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:16.948 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:16.936+0000 7f0b477fe700 1 -- 172.21.15.143:0/337007648 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0b5808e070 con 0x7f0b601f7920 2023-11-08T19:28:17.168 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:17.167+0000 7f0b6535d700 1 -- 172.21.15.143:0/337007648 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"} v 0) v1 -- 0x7f0b600b6800 con 0x7f0b601f7920 2023-11-08T19:28:17.594 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:17.592+0000 7f0b477fe700 1 -- 172.21.15.143:0/337007648 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]=0 v54) v1 ==== 105+0+0 (secure 0 0 0) 0x7f0b58054970 con 0x7f0b601f7920 2023-11-08T19:28:17.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:17.594+0000 7f0b6535d700 1 -- 172.21.15.143:0/337007648 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f0b4c06af00 msgr2=0x7f0b4c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:17.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:17.594+0000 7f0b6535d700 1 --2- 172.21.15.143:0/337007648 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f0b4c06af00 0x7f0b4c06d3c0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f0b600728c0 tx=0x7f0b5400f000 comp rx=0 tx=0).stop 2023-11-08T19:28:17.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:17.594+0000 7f0b6535d700 1 -- 172.21.15.143:0/337007648 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0b601f7920 msgr2=0x7f0b601473a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:17.598 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:17.594+0000 7f0b6535d700 1 --2- 172.21.15.143:0/337007648 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0b601f7920 0x7f0b601473a0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f0b600afb40 tx=0x7f0b58004970 comp rx=0 tx=0).stop 2023-11-08T19:28:17.598 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:17.594+0000 7f0b6535d700 1 -- 172.21.15.143:0/337007648 shutdown_connections 2023-11-08T19:28:17.598 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:17.594+0000 7f0b6535d700 1 --2- 172.21.15.143:0/337007648 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f0b4c06af00 0x7f0b4c06d3c0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:17.598 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:17.594+0000 7f0b6535d700 1 --2- 172.21.15.143:0/337007648 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0b601f7920 0x7f0b601473a0 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:17.599 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:17.594+0000 7f0b6535d700 1 --2- 172.21.15.143:0/337007648 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0b48003a00 0x7f0b600c4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:17.599 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:17.594+0000 7f0b6535d700 1 --2- 172.21.15.143:0/337007648 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0b600aebb0 0x7f0b601f73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:17.599 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:17.594+0000 7f0b6535d700 1 -- 172.21.15.143:0/337007648 >> 172.21.15.143:0/337007648 conn(0x7f0b600cd300 msgr2=0x7f0b600cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:17.599 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:17.594+0000 7f0b6535d700 1 -- 172.21.15.143:0/337007648 shutdown_connections 2023-11-08T19:28:17.599 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:17.594+0000 7f0b6535d700 1 -- 172.21.15.143:0/337007648 wait complete. 2023-11-08T19:28:17.614 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:28:17.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:17.731+0000 7f9de4b1b700 1 Processor -- start 2023-11-08T19:28:17.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:17.732+0000 7f9de4b1b700 1 -- start start 2023-11-08T19:28:17.737 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:17.732+0000 7f9de4b1b700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f9de015aa30 0x7f9de00cf510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:17.737 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:17.732+0000 7f9de4b1b700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9de00cf080 con 0x7f9de00b2210 2023-11-08T19:28:17.737 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:17.732+0000 7f9de4b1b700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9de00ce470 con 0x7f9de00cc530 2023-11-08T19:28:17.737 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:17.732+0000 7f9de4b1b700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9de00ce750 con 0x7f9de015aa30 2023-11-08T19:28:17.738 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:17.733+0000 7f9dde59c700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f9de015aa30 0x7f9de00cf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:17.738 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:17.733+0000 7f9dde59c700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f9de015aa30 0x7f9de00cf510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54850/0 (socket says 172.21.15.143:54850) 2023-11-08T19:28:17.738 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:17.733+0000 7f9dddd9b700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f9de00b2210 0x7f9de00cbe20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:51006/0 (socket says 172.21.15.143:51006) 2023-11-08T19:28:17.738 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:17.733+0000 7f9dde59c700 1 -- 172.21.15.143:0/2104501330 learned_addr learned my addr 172.21.15.143:0/2104501330 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:28:17.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:17.733+0000 7f9dde59c700 1 -- 172.21.15.143:0/2104501330 >> v1:172.21.15.174:6789/0 conn(0x7f9de00cc530 legacy=0x7f9de006db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:17.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:17.733+0000 7f9dde59c700 1 -- 172.21.15.143:0/2104501330 >> v1:172.21.15.143:6789/0 conn(0x7f9de00b2210 legacy=0x7f9de00cbe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:17.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:17.733+0000 7f9dde59c700 1 -- 172.21.15.143:0/2104501330 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9de00cea30 con 0x7f9de015aa30 2023-11-08T19:28:17.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:17.733+0000 7f9ddd59a700 1 -- 172.21.15.143:0/2104501330 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3563855119 0 0) 0x7f9de00cf080 con 0x7f9de00b2210 2023-11-08T19:28:17.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:17.733+0000 7f9dde59c700 1 --2- 172.21.15.143:0/2104501330 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9de015aa30 0x7f9de00cf510 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f9dd4006fd0 tx=0x7f9dd4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b853f0523c56ca server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:17.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:17.733+0000 7f9ddd59a700 1 -- 172.21.15.143:0/2104501330 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dd4005e30 con 0x7f9de015aa30 2023-11-08T19:28:17.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:17.733+0000 7f9ddd59a700 1 -- 172.21.15.143:0/2104501330 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dd4006030 con 0x7f9de015aa30 2023-11-08T19:28:17.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:17.733+0000 7f9ddd59a700 1 -- 172.21.15.143:0/2104501330 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dd40063c0 con 0x7f9de015aa30 2023-11-08T19:28:17.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:17.734+0000 7f9de4b1b700 1 -- 172.21.15.143:0/2104501330 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9de015aa30 msgr2=0x7f9de00cf510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:17.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:17.734+0000 7f9de4b1b700 1 --2- 172.21.15.143:0/2104501330 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9de015aa30 0x7f9de00cf510 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f9dd4006fd0 tx=0x7f9dd4009bd0 comp rx=0 tx=0).stop 2023-11-08T19:28:17.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:17.734+0000 7f9de4b1b700 1 -- 172.21.15.143:0/2104501330 shutdown_connections 2023-11-08T19:28:17.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:17.734+0000 7f9de4b1b700 1 --2- 172.21.15.143:0/2104501330 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9de015aa30 0x7f9de00cf510 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:17.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:17.734+0000 7f9de4b1b700 1 -- 172.21.15.143:0/2104501330 >> 172.21.15.143:0/2104501330 conn(0x7f9de0099ab0 msgr2=0x7f9de00d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:17.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:17.734+0000 7f9de4b1b700 1 -- 172.21.15.143:0/2104501330 shutdown_connections 2023-11-08T19:28:17.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:17.734+0000 7f9de4b1b700 1 -- 172.21.15.143:0/2104501330 wait complete. 2023-11-08T19:28:17.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:17.735+0000 7f9de4b1b700 1 Processor -- start 2023-11-08T19:28:17.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:17.736+0000 7f9de4b1b700 1 -- start start 2023-11-08T19:28:17.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:17.736+0000 7f9de4b1b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9de00b2210 0x7f9de014fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:17.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:17.736+0000 7f9de4b1b700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9de00cc530 0x7f9de00ca160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:17.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:17.736+0000 7f9de4b1b700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9de015aa30 0x7f9de00b55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:17.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:17.736+0000 7f9de4b1b700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9de01501b0 con 0x7f9de00cc530 2023-11-08T19:28:17.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:17.736+0000 7f9de4b1b700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9de01f2920 con 0x7f9de00b2210 2023-11-08T19:28:17.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:17.736+0000 7f9de4b1b700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9de01f2be0 con 0x7f9de015aa30 2023-11-08T19:28:17.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:17.737+0000 7f9dddd9b700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9de00cc530 0x7f9de00ca160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:17.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:17.737+0000 7f9dddd9b700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9de00cc530 0x7f9de00ca160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54856/0 (socket says 172.21.15.143:54856) 2023-11-08T19:28:17.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:17.737+0000 7f9dddd9b700 1 -- 172.21.15.143:0/483803781 learned_addr learned my addr 172.21.15.143:0/483803781 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:28:17.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:17.737+0000 7f9dde59c700 1 --2- 172.21.15.143:0/483803781 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9de00b2210 0x7f9de014fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:17.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:17.737+0000 7f9dded9d700 1 --2- 172.21.15.143:0/483803781 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9de015aa30 0x7f9de00b55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:17.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:17.737+0000 7f9dddd9b700 1 -- 172.21.15.143:0/483803781 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9de015aa30 msgr2=0x7f9de00b55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:17.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:17.737+0000 7f9dddd9b700 1 --2- 172.21.15.143:0/483803781 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9de015aa30 0x7f9de00b55c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:17.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:17.737+0000 7f9dddd9b700 1 -- 172.21.15.143:0/483803781 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9de00b2210 msgr2=0x7f9de014fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:17.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:17.737+0000 7f9dddd9b700 1 --2- 172.21.15.143:0/483803781 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9de00b2210 0x7f9de014fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:17.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:17.737+0000 7f9dddd9b700 1 -- 172.21.15.143:0/483803781 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dd4009500 con 0x7f9de00cc530 2023-11-08T19:28:17.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:17.737+0000 7f9dddd9b700 1 --2- 172.21.15.143:0/483803781 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9de00cc530 0x7f9de00ca160 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f9dd00011e0 tx=0x7f9dd000cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:17.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:17.737+0000 7f9dceffd700 1 -- 172.21.15.143:0/483803781 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dd0004c80 con 0x7f9de00cc530 2023-11-08T19:28:17.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:17.737+0000 7f9dceffd700 1 -- 172.21.15.143:0/483803781 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dd0004e00 con 0x7f9de00cc530 2023-11-08T19:28:17.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:17.737+0000 7f9dceffd700 1 -- 172.21.15.143:0/483803781 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dd0007310 con 0x7f9de00cc530 2023-11-08T19:28:17.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:17.737+0000 7f9de4b1b700 1 -- 172.21.15.143:0/483803781 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9de01f2ec0 con 0x7f9de00cc530 2023-11-08T19:28:17.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:17.737+0000 7f9de4b1b700 1 -- 172.21.15.143:0/483803781 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9de01f3480 con 0x7f9de00cc530 2023-11-08T19:28:17.750 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:17.738+0000 7f9de4b1b700 1 -- 172.21.15.143:0/483803781 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9de004eee0 con 0x7f9de00cc530 2023-11-08T19:28:17.750 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:17.739+0000 7f9dceffd700 1 -- 172.21.15.143:0/483803781 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9dd0007470 con 0x7f9de00cc530 2023-11-08T19:28:17.750 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:17.739+0000 7f9dceffd700 1 --2- 172.21.15.143:0/483803781 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f9dc006af00 0x7f9dc006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:17.750 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:17.739+0000 7f9dceffd700 1 -- 172.21.15.143:0/483803781 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6772+0+0 (secure 0 0 0) 0x7f9dd0089720 con 0x7f9de00cc530 2023-11-08T19:28:17.750 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:17.739+0000 7f9dde59c700 1 --2- 172.21.15.143:0/483803781 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f9dc006af00 0x7f9dc006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:17.751 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:17.740+0000 7f9dde59c700 1 --2- 172.21.15.143:0/483803781 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f9dc006af00 0x7f9dc006d3c0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f9dd4013040 tx=0x7f9dd4008070 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:17.751 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:17.743+0000 7f9dceffd700 1 -- 172.21.15.143:0/483803781 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9dd0054430 con 0x7f9de00cc530 2023-11-08T19:28:17.975 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:17.972+0000 7f9de4b1b700 1 -- 172.21.15.143:0/483803781 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9de00ccb10 con 0x7f9de00cc530 2023-11-08T19:28:17.975 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:17.973+0000 7f9dceffd700 1 -- 172.21.15.143:0/483803781 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f9dd0057a50 con 0x7f9de00cc530 2023-11-08T19:28:17.976 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:28:17.976 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":54,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6906,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6834/3172664377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":3172664377},{"type":"v1","addr":"172.21.15.143:6834","nonce":3172664377}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47},{"gid":6909,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3953059900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":3953059900},{"type":"v1","addr":"172.21.15.143:6835","nonce":3953059900}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":46}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:03.092245+0000","modified":"2023-11-08T19:28:17.592924+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6921,"mds_1":6918},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6918":{"gid":6918,"name":"b","rank":1,"incarnation":54,"state":"up:creating","state_seq":1,"addr":"172.21.15.174:6836/3601088618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3601088618},{"type":"v1","addr":"172.21.15.174:6836","nonce":3601088618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6921":{"gid":6921,"name":"d","rank":0,"incarnation":48,"state":"up:active","state_seq":3,"addr":"172.21.15.174:6837/2201447629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6835","nonce":2201447629},{"type":"v1","addr":"172.21.15.174:6837","nonce":2201447629}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:28:17.977 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:17.975+0000 7f9de4b1b700 1 -- 172.21.15.143:0/483803781 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f9dc006af00 msgr2=0x7f9dc006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:17.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:17.975+0000 7f9de4b1b700 1 --2- 172.21.15.143:0/483803781 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f9dc006af00 0x7f9dc006d3c0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f9dd4013040 tx=0x7f9dd4008070 comp rx=0 tx=0).stop 2023-11-08T19:28:17.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:17.975+0000 7f9de4b1b700 1 -- 172.21.15.143:0/483803781 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9de00cc530 msgr2=0x7f9de00ca160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:17.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:17.975+0000 7f9de4b1b700 1 --2- 172.21.15.143:0/483803781 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9de00cc530 0x7f9de00ca160 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f9dd00011e0 tx=0x7f9dd000cae0 comp rx=0 tx=0).stop 2023-11-08T19:28:17.979 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:17.975+0000 7f9de4b1b700 1 -- 172.21.15.143:0/483803781 shutdown_connections 2023-11-08T19:28:17.979 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:17.975+0000 7f9de4b1b700 1 --2- 172.21.15.143:0/483803781 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f9dc006af00 0x7f9dc006d3c0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:17.979 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:17.975+0000 7f9de4b1b700 1 --2- 172.21.15.143:0/483803781 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9de015aa30 0x7f9de00b55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:17.979 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:17.975+0000 7f9de4b1b700 1 --2- 172.21.15.143:0/483803781 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9de00b2210 0x7f9de014fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:17.980 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:17.975+0000 7f9de4b1b700 1 --2- 172.21.15.143:0/483803781 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9de00cc530 0x7f9de00ca160 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:17.980 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:17.975+0000 7f9de4b1b700 1 -- 172.21.15.143:0/483803781 >> 172.21.15.143:0/483803781 conn(0x7f9de0099ab0 msgr2=0x7f9de0158840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:17.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:17.975+0000 7f9de4b1b700 1 -- 172.21.15.143:0/483803781 shutdown_connections 2023-11-08T19:28:17.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:17.975+0000 7f9de4b1b700 1 -- 172.21.15.143:0/483803781 wait complete. 2023-11-08T19:28:17.981 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 54 2023-11-08T19:28:18.392 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 54, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 2, 'explicitly_allowed_features': 2, 'created': '2023-11-08T19:28:03.092245+0000', 'modified': '2023-11-08T19:28:17.592924+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 2, 'in': [0, 1], 'up': {'mds_0': 6921, 'mds_1': 6918}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6918': {'gid': 6918, 'name': 'b', 'rank': 1, 'incarnation': 54, 'state': 'up:creating', 'state_seq': 1, 'addr': '172.21.15.174:6836/3601088618', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.174:6834', 'nonce': 3601088618}, {'type': 'v1', 'addr': '172.21.15.174:6836', 'nonce': 3601088618}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_6921': {'gid': 6921, 'name': 'd', 'rank': 0, 'incarnation': 48, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.174:6837/2201447629', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.174:6835', 'nonce': 2201447629}, {'type': 'v1', 'addr': '172.21.15.174:6837', 'nonce': 2201447629}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-08T19:28:18.392 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6918:up:creating 2023-11-08T19:28:19.393 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:28:19.527 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:19.526+0000 7f2a672ce700 1 Processor -- start 2023-11-08T19:28:19.528 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:19.526+0000 7f2a672ce700 1 -- start start 2023-11-08T19:28:19.530 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:19.526+0000 7f2a672ce700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f2a600b14c0 0x7f2a600d03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:19.530 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:19.526+0000 7f2a672ce700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a600d08e0 con 0x7f2a6015ba30 2023-11-08T19:28:19.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:19.526+0000 7f2a672ce700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a600b1910 con 0x7f2a600d1c70 2023-11-08T19:28:19.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:19.526+0000 7f2a672ce700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a600b1bc0 con 0x7f2a600b14c0 2023-11-08T19:28:19.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:19.527+0000 7f2a6586b700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f2a600b14c0 0x7f2a600d03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:19.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:19.527+0000 7f2a6506a700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f2a6015ba30 0x7f2a600d1560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:51018/0 (socket says 172.21.15.143:51018) 2023-11-08T19:28:19.532 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:19.527+0000 7f2a6586b700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f2a600b14c0 0x7f2a600d03a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54866/0 (socket says 172.21.15.143:54866) 2023-11-08T19:28:19.532 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:19.527+0000 7f2a6506a700 1 -- 172.21.15.143:0/2740175995 learned_addr learned my addr 172.21.15.143:0/2740175995 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:28:19.532 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:19.527+0000 7f2a6586b700 1 -- 172.21.15.143:0/2740175995 >> v1:172.21.15.174:6789/0 conn(0x7f2a600d1c70 legacy=0x7f2a600cc5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:19.532 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:19.527+0000 7f2a6586b700 1 -- 172.21.15.143:0/2740175995 >> v1:172.21.15.143:6789/0 conn(0x7f2a6015ba30 legacy=0x7f2a600d1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:19.533 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:19.527+0000 7f2a6586b700 1 -- 172.21.15.143:0/2740175995 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a600b1ea0 con 0x7f2a600b14c0 2023-11-08T19:28:19.533 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:19.527+0000 7f2a57fff700 1 -- 172.21.15.143:0/2740175995 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 609177387 0 0) 0x7f2a50001110 con 0x7f2a6015ba30 2023-11-08T19:28:19.533 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:19.527+0000 7f2a6586b700 1 --2- 172.21.15.143:0/2740175995 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2a600b14c0 0x7f2a600d03a0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f2a58009d90 tx=0x7f2a5800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47ae0e17c987696a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:19.533 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:19.527+0000 7f2a57fff700 1 -- 172.21.15.143:0/2740175995 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a5800e070 con 0x7f2a600b14c0 2023-11-08T19:28:19.533 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:19.527+0000 7f2a57fff700 1 -- 172.21.15.143:0/2740175995 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a5800bbd0 con 0x7f2a600b14c0 2023-11-08T19:28:19.534 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:19.527+0000 7f2a57fff700 1 -- 172.21.15.143:0/2740175995 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a58012430 con 0x7f2a600b14c0 2023-11-08T19:28:19.535 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:19.528+0000 7f2a672ce700 1 -- 172.21.15.143:0/2740175995 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2a600b14c0 msgr2=0x7f2a600d03a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:19.535 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:19.528+0000 7f2a672ce700 1 --2- 172.21.15.143:0/2740175995 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2a600b14c0 0x7f2a600d03a0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f2a58009d90 tx=0x7f2a5800b330 comp rx=0 tx=0).stop 2023-11-08T19:28:19.535 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:19.528+0000 7f2a672ce700 1 -- 172.21.15.143:0/2740175995 shutdown_connections 2023-11-08T19:28:19.535 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:19.528+0000 7f2a672ce700 1 --2- 172.21.15.143:0/2740175995 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2a600b14c0 0x7f2a600d03a0 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:19.536 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:19.528+0000 7f2a672ce700 1 -- 172.21.15.143:0/2740175995 >> 172.21.15.143:0/2740175995 conn(0x7f2a6008b000 msgr2=0x7f2a6009ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:19.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:19.528+0000 7f2a672ce700 1 -- 172.21.15.143:0/2740175995 shutdown_connections 2023-11-08T19:28:19.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:19.528+0000 7f2a672ce700 1 -- 172.21.15.143:0/2740175995 wait complete. 2023-11-08T19:28:19.540 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:19.529+0000 7f2a672ce700 1 Processor -- start 2023-11-08T19:28:19.540 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:19.529+0000 7f2a672ce700 1 -- start start 2023-11-08T19:28:19.540 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:19.530+0000 7f2a672ce700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2a600b14c0 0x7f2a60152310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:19.541 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:19.530+0000 7f2a672ce700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2a600d1c70 0x7f2a600cacd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:19.541 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:19.530+0000 7f2a672ce700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a6015ba30 0x7f2a6015e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:19.541 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:19.530+0000 7f2a672ce700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a600ca510 con 0x7f2a600b14c0 2023-11-08T19:28:19.541 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:19.530+0000 7f2a672ce700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a6015cb70 con 0x7f2a6015ba30 2023-11-08T19:28:19.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:19.530+0000 7f2a672ce700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a6015ce50 con 0x7f2a600d1c70 2023-11-08T19:28:19.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:19.530+0000 7f2a6506a700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2a600b14c0 0x7f2a60152310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:19.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:19.530+0000 7f2a6506a700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2a600b14c0 0x7f2a60152310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54868/0 (socket says 172.21.15.143:54868) 2023-11-08T19:28:19.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:19.530+0000 7f2a6506a700 1 -- 172.21.15.143:0/3332507868 learned_addr learned my addr 172.21.15.143:0/3332507868 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:28:19.543 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:19.530+0000 7f2a6586b700 1 --2- 172.21.15.143:0/3332507868 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a6015ba30 0x7f2a6015e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:19.543 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:19.530+0000 7f2a64869700 1 --2- 172.21.15.143:0/3332507868 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2a600d1c70 0x7f2a600cacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:19.543 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:19.530+0000 7f2a6506a700 1 -- 172.21.15.143:0/3332507868 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2a600d1c70 msgr2=0x7f2a600cacd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:19.543 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:19.530+0000 7f2a6506a700 1 --2- 172.21.15.143:0/3332507868 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2a600d1c70 0x7f2a600cacd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:19.543 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:19.530+0000 7f2a6506a700 1 -- 172.21.15.143:0/3332507868 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a6015ba30 msgr2=0x7f2a6015e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:19.544 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:19.530+0000 7f2a6506a700 1 --2- 172.21.15.143:0/3332507868 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a6015ba30 0x7f2a6015e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:19.544 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:19.530+0000 7f2a6506a700 1 -- 172.21.15.143:0/3332507868 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a58009a70 con 0x7f2a600b14c0 2023-11-08T19:28:19.544 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:19.530+0000 7f2a6506a700 1 --2- 172.21.15.143:0/3332507868 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2a600b14c0 0x7f2a60152310 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f2a5000c520 tx=0x7f2a5000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:19.544 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:19.530+0000 7f2a55ffb700 1 -- 172.21.15.143:0/3332507868 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a50004c80 con 0x7f2a600b14c0 2023-11-08T19:28:19.545 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:19.531+0000 7f2a55ffb700 1 -- 172.21.15.143:0/3332507868 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a50004e20 con 0x7f2a600b14c0 2023-11-08T19:28:19.545 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:19.531+0000 7f2a672ce700 1 -- 172.21.15.143:0/3332507868 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a600ca7b0 con 0x7f2a600b14c0 2023-11-08T19:28:19.545 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:19.531+0000 7f2a55ffb700 1 -- 172.21.15.143:0/3332507868 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a50007310 con 0x7f2a600b14c0 2023-11-08T19:28:19.545 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:19.531+0000 7f2a672ce700 1 -- 172.21.15.143:0/3332507868 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a6015d130 con 0x7f2a600b14c0 2023-11-08T19:28:19.545 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:19.532+0000 7f2a672ce700 1 -- 172.21.15.143:0/3332507868 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a6004eee0 con 0x7f2a600b14c0 2023-11-08T19:28:19.546 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:19.532+0000 7f2a55ffb700 1 -- 172.21.15.143:0/3332507868 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2a50007470 con 0x7f2a600b14c0 2023-11-08T19:28:19.546 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:19.532+0000 7f2a55ffb700 1 --2- 172.21.15.143:0/3332507868 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f2a4406af00 0x7f2a4406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:19.546 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:19.532+0000 7f2a55ffb700 1 -- 172.21.15.143:0/3332507868 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6772+0+0 (secure 0 0 0) 0x7f2a50089560 con 0x7f2a600b14c0 2023-11-08T19:28:19.546 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:19.533+0000 7f2a64869700 1 --2- 172.21.15.143:0/3332507868 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f2a4406af00 0x7f2a4406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:19.547 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:19.533+0000 7f2a64869700 1 --2- 172.21.15.143:0/3332507868 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f2a4406af00 0x7f2a4406d3c0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c000f60 tx=0x7f2a5c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:19.547 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:19.537+0000 7f2a55ffb700 1 -- 172.21.15.143:0/3332507868 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2a500541d0 con 0x7f2a600b14c0 2023-11-08T19:28:19.688 INFO:tasks.ceph.mds.d.smithi174.stderr:2023-11-08T19:28:19.686+0000 7f73f8270700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-11-08T19:28:19.774 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:19.772+0000 7f2a672ce700 1 -- 172.21.15.143:0/3332507868 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2a600d08e0 con 0x7f2a600b14c0 2023-11-08T19:28:19.775 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:19.772+0000 7f2a55ffb700 1 -- 172.21.15.143:0/3332507868 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+4941 (secure 0 0 0) 0x7f2a600d08e0 con 0x7f2a600b14c0 2023-11-08T19:28:19.775 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:28:19.776 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":55,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6906,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6834/3172664377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":3172664377},{"type":"v1","addr":"172.21.15.143:6834","nonce":3172664377}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47},{"gid":6909,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3953059900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":3953059900},{"type":"v1","addr":"172.21.15.143:6835","nonce":3953059900}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":46}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:03.092245+0000","modified":"2023-11-08T19:28:17.609590+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6921,"mds_1":6918},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6918":{"gid":6918,"name":"b","rank":1,"incarnation":54,"state":"up:active","state_seq":6,"addr":"172.21.15.174:6836/3601088618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3601088618},{"type":"v1","addr":"172.21.15.174:6836","nonce":3601088618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6921":{"gid":6921,"name":"d","rank":0,"incarnation":48,"state":"up:active","state_seq":3,"addr":"172.21.15.174:6837/2201447629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6835","nonce":2201447629},{"type":"v1","addr":"172.21.15.174:6837","nonce":2201447629}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:28:19.778 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:19.775+0000 7f2a672ce700 1 -- 172.21.15.143:0/3332507868 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f2a4406af00 msgr2=0x7f2a4406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:19.778 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:19.775+0000 7f2a672ce700 1 --2- 172.21.15.143:0/3332507868 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f2a4406af00 0x7f2a4406d3c0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c000f60 tx=0x7f2a5c009040 comp rx=0 tx=0).stop 2023-11-08T19:28:19.778 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:19.775+0000 7f2a672ce700 1 -- 172.21.15.143:0/3332507868 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2a600b14c0 msgr2=0x7f2a60152310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:19.778 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:19.775+0000 7f2a672ce700 1 --2- 172.21.15.143:0/3332507868 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2a600b14c0 0x7f2a60152310 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f2a5000c520 tx=0x7f2a5000c8e0 comp rx=0 tx=0).stop 2023-11-08T19:28:19.779 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:19.775+0000 7f2a672ce700 1 -- 172.21.15.143:0/3332507868 shutdown_connections 2023-11-08T19:28:19.779 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:19.775+0000 7f2a672ce700 1 --2- 172.21.15.143:0/3332507868 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f2a4406af00 0x7f2a4406d3c0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:19.779 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:19.775+0000 7f2a672ce700 1 --2- 172.21.15.143:0/3332507868 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a6015ba30 0x7f2a6015e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:19.779 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:19.775+0000 7f2a672ce700 1 --2- 172.21.15.143:0/3332507868 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2a600b14c0 0x7f2a60152310 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:19.779 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:19.775+0000 7f2a672ce700 1 --2- 172.21.15.143:0/3332507868 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2a600d1c70 0x7f2a600cacd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:19.780 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:19.775+0000 7f2a672ce700 1 -- 172.21.15.143:0/3332507868 >> 172.21.15.143:0/3332507868 conn(0x7f2a6008b000 msgr2=0x7f2a600cec00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:19.780 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:19.775+0000 7f2a672ce700 1 -- 172.21.15.143:0/3332507868 shutdown_connections 2023-11-08T19:28:19.780 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:19.775+0000 7f2a672ce700 1 -- 172.21.15.143:0/3332507868 wait complete. 2023-11-08T19:28:19.780 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 55 2023-11-08T19:28:20.191 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 55, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 2, 'explicitly_allowed_features': 2, 'created': '2023-11-08T19:28:03.092245+0000', 'modified': '2023-11-08T19:28:17.609590+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 2, 'in': [0, 1], 'up': {'mds_0': 6921, 'mds_1': 6918}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6918': {'gid': 6918, 'name': 'b', 'rank': 1, 'incarnation': 54, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.174:6836/3601088618', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.174:6834', 'nonce': 3601088618}, {'type': 'v1', 'addr': '172.21.15.174:6836', 'nonce': 3601088618}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_6921': {'gid': 6921, 'name': 'd', 'rank': 0, 'incarnation': 48, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.174:6837/2201447629', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.174:6835', 'nonce': 2201447629}, {'type': 'v1', 'addr': '172.21.15.174:6837', 'nonce': 2201447629}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-08T19:28:20.192 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 2/2 2023-11-08T19:28:20.192 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-11-08T19:28:20.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.316+0000 7f0ba0177700 1 Processor -- start 2023-11-08T19:28:20.318 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.316+0000 7f0ba0177700 1 -- start start 2023-11-08T19:28:20.321 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.317+0000 7f0ba0177700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0b980b14c0 0x7f0b980d03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:20.322 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.317+0000 7f0ba0177700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b980d08e0 con 0x7f0b980d1c70 2023-11-08T19:28:20.322 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.317+0000 7f0ba0177700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b980b1930 con 0x7f0b9815ba30 2023-11-08T19:28:20.322 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.317+0000 7f0ba0177700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b980b1bc0 con 0x7f0b980b14c0 2023-11-08T19:28:20.322 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.317+0000 7f0b9d712700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f0b980d1c70 0x7f0b980cc5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:51032/0 (socket says 172.21.15.143:51032) 2023-11-08T19:28:20.322 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.317+0000 7f0b9d712700 1 -- 172.21.15.143:0/1134079552 learned_addr learned my addr 172.21.15.143:0/1134079552 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:28:20.323 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.317+0000 7f0b9cf11700 1 -- 172.21.15.143:0/1134079552 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 464627521 0 0) 0x7f0b88001110 con 0x7f0b980d1c70 2023-11-08T19:28:20.323 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.317+0000 7f0b9cf11700 1 -- 172.21.15.143:0/1134079552 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b840035b0 con 0x7f0b980d1c70 2023-11-08T19:28:20.323 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.317+0000 7f0b9e714700 1 --2- 172.21.15.143:0/1134079552 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0b980b14c0 0x7f0b980d03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:20.323 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.317+0000 7f0b9cf11700 1 -- 172.21.15.143:0/1134079552 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3445041359 0 0) 0x7f0b840035b0 con 0x7f0b980d1c70 2023-11-08T19:28:20.323 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.317+0000 7f0b9cf11700 1 -- 172.21.15.143:0/1134079552 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0b980b14c0 msgr2=0x7f0b980d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:20.324 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.317+0000 7f0b9cf11700 1 --2- 172.21.15.143:0/1134079552 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0b980b14c0 0x7f0b980d03a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:20.324 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.317+0000 7f0b9cf11700 1 -- 172.21.15.143:0/1134079552 >> v1:172.21.15.174:6789/0 conn(0x7f0b9815ba30 legacy=0x7f0b980d1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:20.324 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.317+0000 7f0b9cf11700 1 -- 172.21.15.143:0/1134079552 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b980b1ea0 con 0x7f0b980d1c70 2023-11-08T19:28:20.324 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.317+0000 7f0b9cf11700 1 -- 172.21.15.143:0/1134079552 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4147756303 0 0) 0x7f0b88001800 con 0x7f0b980d1c70 2023-11-08T19:28:20.324 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.317+0000 7f0b9cf11700 1 -- 172.21.15.143:0/1134079552 >> v1:172.21.15.143:6789/0 conn(0x7f0b980d1c70 legacy=0x7f0b980cc5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:20.324 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.318+0000 7f0b9cf11700 1 --2- 172.21.15.143:0/1134079552 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0b980b14c0 0x7f0b84003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:20.325 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.318+0000 7f0b9cf11700 1 --2- 172.21.15.143:0/1134079552 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0b84003fb0 0x7f0b84004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:20.325 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.318+0000 7f0b9cf11700 1 --2- 172.21.15.143:0/1134079552 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0b84004d90 0x7f0b84005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:20.325 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.318+0000 7f0b9cf11700 1 -- 172.21.15.143:0/1134079552 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b980b1bc0 con 0x7f0b84003fb0 2023-11-08T19:28:20.325 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.318+0000 7f0b9cf11700 1 -- 172.21.15.143:0/1134079552 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b8400baf0 con 0x7f0b84004d90 2023-11-08T19:28:20.325 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.318+0000 7f0b9d712700 1 --2- 172.21.15.143:0/1134079552 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0b84003fb0 0x7f0b84004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:20.326 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.318+0000 7f0b9cf11700 1 -- 172.21.15.143:0/1134079552 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b8400bcb0 con 0x7f0b980b14c0 2023-11-08T19:28:20.326 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.318+0000 7f0b9df13700 1 --2- 172.21.15.143:0/1134079552 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0b980b14c0 0x7f0b84003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:20.326 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.318+0000 7f0b9e714700 1 --2- 172.21.15.143:0/1134079552 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0b84004d90 0x7f0b84005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:20.326 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.318+0000 7f0b9d712700 1 -- 172.21.15.143:0/1134079552 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0b980b14c0 msgr2=0x7f0b84003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:20.326 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.318+0000 7f0b9d712700 1 --2- 172.21.15.143:0/1134079552 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0b980b14c0 0x7f0b84003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:20.327 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.318+0000 7f0b9d712700 1 -- 172.21.15.143:0/1134079552 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0b84004d90 msgr2=0x7f0b84005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:20.327 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.318+0000 7f0b9d712700 1 --2- 172.21.15.143:0/1134079552 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0b84004d90 0x7f0b84005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:20.327 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.318+0000 7f0b9d712700 1 -- 172.21.15.143:0/1134079552 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b840035b0 con 0x7f0b84003fb0 2023-11-08T19:28:20.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.318+0000 7f0b9d712700 1 --2- 172.21.15.143:0/1134079552 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0b84003fb0 0x7f0b84004850 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f0b8800c240 tx=0x7f0b84005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f375897385531e8 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:20.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.318+0000 7f0b9cf11700 1 -- 172.21.15.143:0/1134079552 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b88007db0 con 0x7f0b84003fb0 2023-11-08T19:28:20.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.318+0000 7f0b9cf11700 1 -- 172.21.15.143:0/1134079552 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b8800cde0 con 0x7f0b84003fb0 2023-11-08T19:28:20.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.318+0000 7f0b9cf11700 1 -- 172.21.15.143:0/1134079552 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b88005b80 con 0x7f0b84003fb0 2023-11-08T19:28:20.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.319+0000 7f0ba0177700 1 -- 172.21.15.143:0/1134079552 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0b84003fb0 msgr2=0x7f0b84004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:20.329 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.319+0000 7f0ba0177700 1 --2- 172.21.15.143:0/1134079552 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0b84003fb0 0x7f0b84004850 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f0b8800c240 tx=0x7f0b84005880 comp rx=0 tx=0).stop 2023-11-08T19:28:20.329 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.319+0000 7f0b9e714700 1 -- 172.21.15.143:0/1134079552 reap_dead start 2023-11-08T19:28:20.329 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.319+0000 7f0ba0177700 1 -- 172.21.15.143:0/1134079552 shutdown_connections 2023-11-08T19:28:20.329 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.319+0000 7f0ba0177700 1 -- 172.21.15.143:0/1134079552 >> 172.21.15.143:0/1134079552 conn(0x7f0b9808b000 msgr2=0x7f0b9809ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:20.330 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.319+0000 7f0ba0177700 1 -- 172.21.15.143:0/1134079552 shutdown_connections 2023-11-08T19:28:20.330 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.319+0000 7f0ba0177700 1 -- 172.21.15.143:0/1134079552 wait complete. 2023-11-08T19:28:20.330 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.320+0000 7f0ba0177700 1 Processor -- start 2023-11-08T19:28:20.330 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.320+0000 7f0ba0177700 1 -- start start 2023-11-08T19:28:20.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.321+0000 7f0ba0177700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0b84003fb0 0x7f0b981578c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:20.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.321+0000 7f0ba0177700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0b98157ea0 0x7f0b980b5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:20.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.321+0000 7f0ba0177700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0b981571f0 0x7f0b980c7780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:20.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.321+0000 7f0ba0177700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b980b55d0 con 0x7f0b98157ea0 2023-11-08T19:28:20.341 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.321+0000 7f0ba0177700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b980c6b60 con 0x7f0b84003fb0 2023-11-08T19:28:20.341 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.321+0000 7f0ba0177700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b98157670 con 0x7f0b981571f0 2023-11-08T19:28:20.341 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.321+0000 7f0b9d712700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0b98157ea0 0x7f0b980b5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:20.341 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.321+0000 7f0b9d712700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0b98157ea0 0x7f0b980b5a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54890/0 (socket says 172.21.15.143:54890) 2023-11-08T19:28:20.341 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.321+0000 7f0b9d712700 1 -- 172.21.15.143:0/3137236859 learned_addr learned my addr 172.21.15.143:0/3137236859 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:28:20.342 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.321+0000 7f0b9e714700 1 --2- 172.21.15.143:0/3137236859 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0b981571f0 0x7f0b980c7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:20.342 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.321+0000 7f0b9df13700 1 --2- 172.21.15.143:0/3137236859 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0b84003fb0 0x7f0b981578c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:20.342 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.321+0000 7f0b9d712700 1 -- 172.21.15.143:0/3137236859 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0b981571f0 msgr2=0x7f0b980c7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:20.342 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.321+0000 7f0b9d712700 1 --2- 172.21.15.143:0/3137236859 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0b981571f0 0x7f0b980c7780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:20.342 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.321+0000 7f0b9d712700 1 -- 172.21.15.143:0/3137236859 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0b84003fb0 msgr2=0x7f0b981578c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:20.343 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.321+0000 7f0b9d712700 1 --2- 172.21.15.143:0/3137236859 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0b84003fb0 0x7f0b981578c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:20.343 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.321+0000 7f0b9d712700 1 -- 172.21.15.143:0/3137236859 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b980c6880 con 0x7f0b98157ea0 2023-11-08T19:28:20.343 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.321+0000 7f0b9d712700 1 --2- 172.21.15.143:0/3137236859 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0b98157ea0 0x7f0b980b5a60 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f0b88000c00 tx=0x7f0b88001b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:20.343 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.321+0000 7f0b8e7fc700 1 -- 172.21.15.143:0/3137236859 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b88004c80 con 0x7f0b98157ea0 2023-11-08T19:28:20.344 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.321+0000 7f0b8e7fc700 1 -- 172.21.15.143:0/3137236859 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b88004de0 con 0x7f0b98157ea0 2023-11-08T19:28:20.344 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.321+0000 7f0b8e7fc700 1 -- 172.21.15.143:0/3137236859 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b88007310 con 0x7f0b98157ea0 2023-11-08T19:28:20.344 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.321+0000 7f0ba0177700 1 -- 172.21.15.143:0/3137236859 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b980b4f80 con 0x7f0b98157ea0 2023-11-08T19:28:20.344 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.321+0000 7f0ba0177700 1 -- 172.21.15.143:0/3137236859 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b980c7cc0 con 0x7f0b98157ea0 2023-11-08T19:28:20.344 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.323+0000 7f0b8e7fc700 1 -- 172.21.15.143:0/3137236859 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0b88019070 con 0x7f0b98157ea0 2023-11-08T19:28:20.345 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.323+0000 7f0b8e7fc700 1 --2- 172.21.15.143:0/3137236859 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f0b7c06af00 0x7f0b7c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:20.345 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.323+0000 7f0b8e7fc700 1 -- 172.21.15.143:0/3137236859 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6772+0+0 (secure 0 0 0) 0x7f0b8808a860 con 0x7f0b98157ea0 2023-11-08T19:28:20.345 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.324+0000 7f0b9df13700 1 --2- 172.21.15.143:0/3137236859 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f0b7c06af00 0x7f0b7c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:20.345 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.324+0000 7f0b9df13700 1 --2- 172.21.15.143:0/3137236859 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f0b7c06af00 0x7f0b7c06d3c0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f0b94000c00 tx=0x7f0b94009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:20.346 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.325+0000 7f0bac370b80 1 Processor -- start 2023-11-08T19:28:20.346 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.325+0000 7f0bac370b80 1 -- start start 2023-11-08T19:28:20.346 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.326+0000 7f0bac370b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55957b3346f0 0x55957b364450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:20.346 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.326+0000 7f0bac370b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55957b336c30 0x55957b364990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:20.347 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.326+0000 7f0bac370b80 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55957b334ad0 0x55957b34ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:20.347 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.326+0000 7f0bac370b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x55957b2a06e0 con 0x55957b3346f0 2023-11-08T19:28:20.347 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.326+0000 7f0bac370b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55957b2943b0 con 0x55957b336c30 2023-11-08T19:28:20.347 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.326+0000 7f0bac370b80 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x55957b291cd0 con 0x55957b334ad0 2023-11-08T19:28:20.347 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.326+0000 7f0b9df13700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55957b3346f0 0x55957b364450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:20.348 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.326+0000 7f0b9df13700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55957b3346f0 0x55957b364450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54896/0 (socket says 172.21.15.143:54896) 2023-11-08T19:28:20.348 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.326+0000 7f0b9df13700 1 -- 172.21.15.143:0/768770783 learned_addr learned my addr 172.21.15.143:0/768770783 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:28:20.348 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.326+0000 7f0b9df13700 1 -- 172.21.15.143:0/768770783 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55957b334ad0 msgr2=0x55957b34ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:20.348 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.326+0000 7f0b9df13700 1 --2- 172.21.15.143:0/768770783 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55957b334ad0 0x55957b34ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:20.348 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.326+0000 7f0b9df13700 1 -- 172.21.15.143:0/768770783 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55957b336c30 msgr2=0x55957b364990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:20.349 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.326+0000 7f0b9df13700 1 --2- 172.21.15.143:0/768770783 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55957b336c30 0x55957b364990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:20.349 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.326+0000 7f0b9df13700 1 -- 172.21.15.143:0/768770783 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55957b10a5e0 con 0x55957b3346f0 2023-11-08T19:28:20.349 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.326+0000 7f0b9df13700 1 --2- 172.21.15.143:0/768770783 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55957b3346f0 0x55957b364450 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f0b940156f0 tx=0x7f0b94014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=535abfee0873a0e5 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:20.349 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.327+0000 7f0b7b7fe700 1 -- 172.21.15.143:0/768770783 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b94001500 con 0x55957b3346f0 2023-11-08T19:28:20.350 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.327+0000 7f0b7b7fe700 1 -- 172.21.15.143:0/768770783 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b94001660 con 0x55957b3346f0 2023-11-08T19:28:20.350 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.327+0000 7f0b7b7fe700 1 -- 172.21.15.143:0/768770783 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b940019f0 con 0x55957b3346f0 2023-11-08T19:28:20.350 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.327+0000 7f0bac370b80 1 -- 172.21.15.143:0/768770783 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55957b3346f0 msgr2=0x55957b364450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:20.350 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.327+0000 7f0bac370b80 1 --2- 172.21.15.143:0/768770783 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55957b3346f0 0x55957b364450 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f0b940156f0 tx=0x7f0b94014000 comp rx=0 tx=0).stop 2023-11-08T19:28:20.350 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.327+0000 7f0bac370b80 1 -- 172.21.15.143:0/768770783 shutdown_connections 2023-11-08T19:28:20.351 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.327+0000 7f0bac370b80 1 --2- 172.21.15.143:0/768770783 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55957b334ad0 0x55957b34ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:20.351 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.327+0000 7f0bac370b80 1 --2- 172.21.15.143:0/768770783 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55957b3346f0 0x55957b364450 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:20.351 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.327+0000 7f0bac370b80 1 --2- 172.21.15.143:0/768770783 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55957b336c30 0x55957b364990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:20.351 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.327+0000 7f0bac370b80 1 -- 172.21.15.143:0/768770783 >> 172.21.15.143:0/768770783 conn(0x55957b2af660 msgr2=0x55957b33a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:20.351 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.327+0000 7f0bac370b80 1 -- 172.21.15.143:0/768770783 shutdown_connections 2023-11-08T19:28:20.352 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.327+0000 7f0bac370b80 1 -- 172.21.15.143:0/768770783 wait complete. 2023-11-08T19:28:20.352 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.328+0000 7f0bac370b80 1 -- start start 2023-11-08T19:28:20.352 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.328+0000 7f0bac370b80 1 Processor -- start 2023-11-08T19:28:20.352 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.329+0000 7f0bac370b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55957b3346f0 0x55957b366640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:20.352 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.329+0000 7f0bac370b80 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55957b334ad0 0x55957b34f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:20.353 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.329+0000 7f0bac370b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55957b3793a0 0x55957b378720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:20.353 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.329+0000 7f0bac370b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x55957b28d5c0 con 0x55957b3793a0 2023-11-08T19:28:20.353 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.329+0000 7f0bac370b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55957b2290d0 con 0x55957b3346f0 2023-11-08T19:28:20.353 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.329+0000 7f0bac370b80 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x55957b228d70 con 0x55957b334ad0 2023-11-08T19:28:20.353 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.329+0000 7f0bac370b80 10 client.0 _ll_register_callbacks cb 0x55957b0857b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T19:28:20.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.329+0000 7f0b9e714700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55957b3793a0 0x55957b378720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:20.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.329+0000 7f0b9e714700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55957b3793a0 0x55957b378720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54906/0 (socket says 172.21.15.143:54906) 2023-11-08T19:28:20.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.329+0000 7f0b9e714700 1 -- 172.21.15.143:0/773037830 learned_addr learned my addr 172.21.15.143:0/773037830 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:28:20.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.329+0000 7f0b9df13700 1 --2- 172.21.15.143:0/773037830 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55957b3346f0 0x55957b366640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:20.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.329+0000 7f0b9d712700 1 --2- 172.21.15.143:0/773037830 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55957b334ad0 0x55957b34f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:20.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.329+0000 7f0b9e714700 1 -- 172.21.15.143:0/773037830 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55957b334ad0 msgr2=0x55957b34f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:20.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.329+0000 7f0b9e714700 1 --2- 172.21.15.143:0/773037830 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55957b334ad0 0x55957b34f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:20.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.329+0000 7f0b9e714700 1 -- 172.21.15.143:0/773037830 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55957b3346f0 msgr2=0x55957b366640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:20.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.329+0000 7f0b9e714700 1 --2- 172.21.15.143:0/773037830 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55957b3346f0 0x55957b366640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:20.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.329+0000 7f0b9e714700 1 -- 172.21.15.143:0/773037830 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b980b1930 con 0x55957b3793a0 2023-11-08T19:28:20.356 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.329+0000 7f0b9d712700 1 --2- 172.21.15.143:0/773037830 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55957b334ad0 0x55957b34f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:28:20.356 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.330+0000 7f0b9e714700 1 --2- 172.21.15.143:0/773037830 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55957b3793a0 0x55957b378720 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f0b980cf3a0 tx=0x7f0b90004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b09767cf05e5b12a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:20.356 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.330+0000 7f0b78ff9700 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2023-11-08T19:28:20.356 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.330+0000 7f0b78ff9700 1 -- 172.21.15.143:0/773037830 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b9000e030 con 0x55957b3793a0 2023-11-08T19:28:20.356 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.330+0000 7f0b78ff9700 1 -- 172.21.15.143:0/773037830 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 6772+0+0 (secure 0 0 0) 0x7f0b9000f070 con 0x55957b3793a0 2023-11-08T19:28:20.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.330+0000 7f0bac370b80 1 -- 172.21.15.143:0/773037830 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55957b32c880 con 0x55957b3793a0 2023-11-08T19:28:20.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.330+0000 7f0b78ff9700 1 -- 172.21.15.143:0/773037830 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0b90016870 con 0x55957b3793a0 2023-11-08T19:28:20.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.330+0000 7f0bac370b80 10 client.7155 fetch_fsmap learned FSMap version 55 2023-11-08T19:28:20.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.330+0000 7f0bac370b80 1 -- 172.21.15.143:0/773037830 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=2+}) v3 -- 0x55957b0b6d90 con 0x55957b3793a0 2023-11-08T19:28:20.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.330+0000 7f0b78ff9700 1 -- 172.21.15.143:0/773037830 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55957b32c880 con 0x55957b3793a0 2023-11-08T19:28:20.358 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.330+0000 7f0b78ff9700 1 -- 172.21.15.143:0/773037830 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 55) v1 ==== 3306+0+0 (secure 0 0 0) 0x7f0b90014040 con 0x55957b3793a0 2023-11-08T19:28:20.358 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.330+0000 7f0bac370b80 10 client.7155 fetch_fsmap finished waiting for FSMap version 55 2023-11-08T19:28:20.358 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.330+0000 7f0bac370b80 10 client.7155 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-08T19:28:20.358 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.330+0000 7f0bac370b80 20 client.7155 populate_metadata read hostname 'smithi143' 2023-11-08T19:28:20.358 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.330+0000 7f0bac370b80 1 --2- 172.21.15.143:0/773037830 >> [v2:172.21.15.174:6834/3601088618,v1:172.21.15.174:6836/3601088618] conn(0x55957b37b590 0x55957b376500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:20.359 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.330+0000 7f0bac370b80 4 client.7155 mds_command: new command op to 6918 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-08T19:28:20.359 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.330+0000 7f0bac370b80 1 -- 172.21.15.143:0/773037830 --> [v2:172.21.15.174:6834/3601088618,v1:172.21.15.174:6836/3601088618] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55957b308a30 con 0x55957b37b590 2023-11-08T19:28:20.359 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.330+0000 7f0b9df13700 1 --2- 172.21.15.143:0/773037830 >> [v2:172.21.15.174:6834/3601088618,v1:172.21.15.174:6836/3601088618] conn(0x55957b37b590 0x55957b376500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:20.359 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.331+0000 7f0b9df13700 1 --2- 172.21.15.143:0/773037830 >> [v2:172.21.15.174:6834/3601088618,v1:172.21.15.174:6836/3601088618] conn(0x55957b37b590 0x55957b376500 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=146d9eb17fe50a6a server_cookie=fc597d24bcebf0ae in_seq=0 out_seq=0 2023-11-08T19:28:20.359 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.331+0000 7f0b78ff9700 10 client.7155 ms_handle_connect on v2:172.21.15.174:6834/3601088618 2023-11-08T19:28:20.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.331+0000 7f0b78ff9700 1 -- 172.21.15.143:0/773037830 <== mds.1 v2:172.21.15.174:6834/3601088618 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55957b308a30 con 0x55957b37b590 2023-11-08T19:28:20.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.331+0000 7f0b78ff9700 10 client.7155 handle_command_reply: tid=0 2023-11-08T19:28:20.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.332+0000 7f0bac370b80 1 client.7155 shutdown 2023-11-08T19:28:20.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.332+0000 7f0bac370b80 1 -- 172.21.15.143:0/773037830 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55957b3793a0 msgr2=0x55957b378720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:20.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.332+0000 7f0bac370b80 1 --2- 172.21.15.143:0/773037830 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55957b3793a0 0x55957b378720 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f0b980cf3a0 tx=0x7f0b90004600 comp rx=0 tx=0).stop 2023-11-08T19:28:20.361 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.332+0000 7f0bac370b80 1 -- 172.21.15.143:0/773037830 shutdown_connections 2023-11-08T19:28:20.361 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.332+0000 7f0bac370b80 1 --2- 172.21.15.143:0/773037830 >> [v2:172.21.15.174:6834/3601088618,v1:172.21.15.174:6836/3601088618] conn(0x55957b37b590 0x55957b376500 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:20.361 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.332+0000 7f0bac370b80 1 --2- 172.21.15.143:0/773037830 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55957b3793a0 0x55957b378720 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:20.361 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.332+0000 7f0bac370b80 1 --2- 172.21.15.143:0/773037830 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55957b3346f0 0x55957b366640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:20.361 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.332+0000 7f0b78ff9700 0 client.7155 ms_handle_reset on v2:172.21.15.174:6834/3601088618 2023-11-08T19:28:20.362 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.332+0000 7f0bac370b80 1 --2- 172.21.15.143:0/773037830 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55957b334ad0 0x55957b34f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:20.362 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.332+0000 7f0bac370b80 1 -- 172.21.15.143:0/773037830 >> 172.21.15.143:0/773037830 conn(0x55957b2af660 msgr2=0x55957b335020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:20.362 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.332+0000 7f0bac370b80 1 -- 172.21.15.143:0/773037830 shutdown_connections 2023-11-08T19:28:20.362 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.332+0000 7f0bac370b80 1 -- 172.21.15.143:0/773037830 wait complete. 2023-11-08T19:28:20.363 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.332+0000 7f0bac370b80 20 client.7155 trim_cache size 0 max 16384 2023-11-08T19:28:20.370 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.350+0000 7f0bac370b80 1 Processor -- start 2023-11-08T19:28:20.370 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.350+0000 7f0bac370b80 1 -- start start 2023-11-08T19:28:20.371 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.351+0000 7f0bac370b80 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55957b3346f0 0x55957b378520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:20.371 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.351+0000 7f0bac370b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55957b334ad0 0x55957b34f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:20.371 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.351+0000 7f0bac370b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55957b3793a0 0x55957b338300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:20.372 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.351+0000 7f0bac370b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x55957b10c8d0 con 0x55957b3793a0 2023-11-08T19:28:20.372 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.351+0000 7f0bac370b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55957b381a10 con 0x55957b334ad0 2023-11-08T19:28:20.372 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.351+0000 7f0bac370b80 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x55957b378a60 con 0x55957b3346f0 2023-11-08T19:28:20.372 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.351+0000 7f0b9e714700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55957b3793a0 0x55957b338300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:20.373 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.351+0000 7f0b9e714700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55957b3793a0 0x55957b338300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54914/0 (socket says 172.21.15.143:54914) 2023-11-08T19:28:20.373 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.351+0000 7f0b9e714700 1 -- 172.21.15.143:0/584157416 learned_addr learned my addr 172.21.15.143:0/584157416 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:28:20.373 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.351+0000 7f0b9e714700 1 -- 172.21.15.143:0/584157416 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55957b3346f0 msgr2=0x55957b378520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:20.373 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.351+0000 7f0b9e714700 1 --2- 172.21.15.143:0/584157416 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55957b3346f0 0x55957b378520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:20.374 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.351+0000 7f0b9e714700 1 -- 172.21.15.143:0/584157416 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55957b334ad0 msgr2=0x55957b34f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:20.374 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.351+0000 7f0b9e714700 1 --2- 172.21.15.143:0/584157416 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55957b334ad0 0x55957b34f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:20.374 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.351+0000 7f0b9e714700 1 -- 172.21.15.143:0/584157416 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55957b09fc10 con 0x55957b3793a0 2023-11-08T19:28:20.374 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.352+0000 7f0b9e714700 1 --2- 172.21.15.143:0/584157416 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55957b3793a0 0x55957b338300 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f0b90004bc0 tx=0x7f0b9000f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce4f5e76c04b55dd server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:20.375 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.352+0000 7f0b7bfff700 1 -- 172.21.15.143:0/584157416 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b9000e070 con 0x55957b3793a0 2023-11-08T19:28:20.375 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.352+0000 7f0b7bfff700 1 -- 172.21.15.143:0/584157416 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b90003ed0 con 0x55957b3793a0 2023-11-08T19:28:20.375 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.352+0000 7f0b7bfff700 1 -- 172.21.15.143:0/584157416 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b90016670 con 0x55957b3793a0 2023-11-08T19:28:20.375 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.352+0000 7f0bac370b80 1 -- 172.21.15.143:0/584157416 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55957b3793a0 msgr2=0x55957b338300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:20.375 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.352+0000 7f0bac370b80 1 --2- 172.21.15.143:0/584157416 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55957b3793a0 0x55957b338300 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f0b90004bc0 tx=0x7f0b9000f000 comp rx=0 tx=0).stop 2023-11-08T19:28:20.376 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.352+0000 7f0bac370b80 1 -- 172.21.15.143:0/584157416 shutdown_connections 2023-11-08T19:28:20.376 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.352+0000 7f0bac370b80 1 --2- 172.21.15.143:0/584157416 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55957b3346f0 0x55957b378520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:20.376 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.352+0000 7f0bac370b80 1 --2- 172.21.15.143:0/584157416 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55957b3793a0 0x55957b338300 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:20.376 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.352+0000 7f0bac370b80 1 --2- 172.21.15.143:0/584157416 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55957b334ad0 0x55957b34f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:20.377 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.352+0000 7f0bac370b80 1 -- 172.21.15.143:0/584157416 >> 172.21.15.143:0/584157416 conn(0x55957b2af660 msgr2=0x55957b3307e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:20.377 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.352+0000 7f0bac370b80 1 -- 172.21.15.143:0/584157416 shutdown_connections 2023-11-08T19:28:20.377 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.352+0000 7f0bac370b80 1 -- 172.21.15.143:0/584157416 wait complete. 2023-11-08T19:28:20.377 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.353+0000 7f0bac370b80 1 -- start start 2023-11-08T19:28:20.377 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.353+0000 7f0bac370b80 1 Processor -- start 2023-11-08T19:28:20.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.354+0000 7f0bac370b80 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55957b3346f0 0x55957b364840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:20.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.354+0000 7f0bac370b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55957b334ad0 0x55957b3ec4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:20.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.354+0000 7f0bac370b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55957b3793a0 0x55957b3eca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:20.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.354+0000 7f0bac370b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x55957b34dee0 con 0x55957b3793a0 2023-11-08T19:28:20.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.354+0000 7f0bac370b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55957b33be90 con 0x55957b334ad0 2023-11-08T19:28:20.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.354+0000 7f0bac370b80 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x55957b3ecf50 con 0x55957b3346f0 2023-11-08T19:28:20.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.354+0000 7f0bac370b80 10 client.0 _ll_register_callbacks cb 0x7f0b90003ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T19:28:20.380 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.354+0000 7f0b9e714700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55957b3793a0 0x55957b3eca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:20.380 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.354+0000 7f0b9e714700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55957b3793a0 0x55957b3eca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54918/0 (socket says 172.21.15.143:54918) 2023-11-08T19:28:20.380 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.354+0000 7f0b9e714700 1 -- 172.21.15.143:0/3024685461 learned_addr learned my addr 172.21.15.143:0/3024685461 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:28:20.380 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.354+0000 7f0b9df13700 1 --2- 172.21.15.143:0/3024685461 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55957b3346f0 0x55957b364840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:20.380 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.354+0000 7f0b9d712700 1 --2- 172.21.15.143:0/3024685461 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55957b334ad0 0x55957b3ec4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:20.381 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.354+0000 7f0b9e714700 1 -- 172.21.15.143:0/3024685461 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55957b3346f0 msgr2=0x55957b364840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:20.381 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.354+0000 7f0b9e714700 1 --2- 172.21.15.143:0/3024685461 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55957b3346f0 0x55957b364840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:20.381 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.354+0000 7f0b9e714700 1 -- 172.21.15.143:0/3024685461 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55957b334ad0 msgr2=0x55957b3ec4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:20.382 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.354+0000 7f0b9e714700 1 --2- 172.21.15.143:0/3024685461 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55957b334ad0 0x55957b3ec4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:20.382 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.354+0000 7f0b9e714700 1 -- 172.21.15.143:0/3024685461 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55957b0b6d90 con 0x55957b3793a0 2023-11-08T19:28:20.382 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.354+0000 7f0b9e714700 1 --2- 172.21.15.143:0/3024685461 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55957b3793a0 0x55957b3eca10 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f0b90019040 tx=0x7f0b9000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a88647e16db3bf1 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:20.382 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.354+0000 7f0b78ff9700 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2023-11-08T19:28:20.383 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.355+0000 7f0b78ff9700 1 -- 172.21.15.143:0/3024685461 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b9000e040 con 0x55957b3793a0 2023-11-08T19:28:20.383 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.355+0000 7f0b78ff9700 1 -- 172.21.15.143:0/3024685461 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 6772+0+0 (secure 0 0 0) 0x7f0b90032470 con 0x55957b3793a0 2023-11-08T19:28:20.383 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.355+0000 7f0bac370b80 1 -- 172.21.15.143:0/3024685461 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0b94001660 con 0x55957b3793a0 2023-11-08T19:28:20.383 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.355+0000 7f0b78ff9700 1 -- 172.21.15.143:0/3024685461 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0b90032630 con 0x55957b3793a0 2023-11-08T19:28:20.383 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.355+0000 7f0bac370b80 10 client.7161 fetch_fsmap learned FSMap version 55 2023-11-08T19:28:20.384 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.355+0000 7f0bac370b80 1 -- 172.21.15.143:0/3024685461 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=2+}) v3 -- 0x55957b0dd200 con 0x55957b3793a0 2023-11-08T19:28:20.384 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.355+0000 7f0b78ff9700 1 -- 172.21.15.143:0/3024685461 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b94001660 con 0x55957b3793a0 2023-11-08T19:28:20.384 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.355+0000 7f0b78ff9700 1 -- 172.21.15.143:0/3024685461 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 55) v1 ==== 3306+0+0 (secure 0 0 0) 0x7f0b90014070 con 0x55957b3793a0 2023-11-08T19:28:20.384 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.355+0000 7f0bac370b80 10 client.7161 fetch_fsmap finished waiting for FSMap version 55 2023-11-08T19:28:20.384 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.355+0000 7f0bac370b80 10 client.7161 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-08T19:28:20.385 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.355+0000 7f0bac370b80 20 client.7161 populate_metadata read hostname 'smithi143' 2023-11-08T19:28:20.385 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.355+0000 7f0bac370b80 1 --2- 172.21.15.143:0/3024685461 >> [v2:172.21.15.174:6834/3601088618,v1:172.21.15.174:6836/3601088618] conn(0x55957b365780 0x55957b365b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:20.385 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.355+0000 7f0bac370b80 4 client.7161 mds_command: new command op to 6918 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-08T19:28:20.385 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.355+0000 7f0bac370b80 1 -- 172.21.15.143:0/3024685461 --> [v2:172.21.15.174:6834/3601088618,v1:172.21.15.174:6836/3601088618] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55957b2865a0 con 0x55957b365780 2023-11-08T19:28:20.385 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.355+0000 7f0b9df13700 1 --2- 172.21.15.143:0/3024685461 >> [v2:172.21.15.174:6834/3601088618,v1:172.21.15.174:6836/3601088618] conn(0x55957b365780 0x55957b365b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:20.386 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.356+0000 7f0b78ff9700 10 client.7161 ms_handle_connect on v2:172.21.15.174:6834/3601088618 2023-11-08T19:28:20.386 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.356+0000 7f0b9df13700 1 --2- 172.21.15.143:0/3024685461 >> [v2:172.21.15.174:6834/3601088618,v1:172.21.15.174:6836/3601088618] conn(0x55957b365780 0x55957b365b60 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c36ca1ce9436ca23 server_cookie=3002619b98a37dfe in_seq=0 out_seq=0 2023-11-08T19:28:20.386 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.356+0000 7f0b78ff9700 1 -- 172.21.15.143:0/3024685461 <== mds.1 v2:172.21.15.174:6834/3601088618 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55957b2865a0 con 0x55957b365780 2023-11-08T19:28:20.386 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.356+0000 7f0b78ff9700 10 client.7161 handle_command_reply: tid=0 2023-11-08T19:28:20.387 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.356+0000 7f0bac370b80 1 client.7161 shutdown 2023-11-08T19:28:20.387 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.356+0000 7f0bac370b80 1 -- 172.21.15.143:0/3024685461 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55957b3793a0 msgr2=0x55957b3eca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:20.387 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.356+0000 7f0bac370b80 1 --2- 172.21.15.143:0/3024685461 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55957b3793a0 0x55957b3eca10 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f0b90019040 tx=0x7f0b9000b040 comp rx=0 tx=0).stop 2023-11-08T19:28:20.387 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.356+0000 7f0bac370b80 1 -- 172.21.15.143:0/3024685461 shutdown_connections 2023-11-08T19:28:20.388 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.356+0000 7f0bac370b80 1 --2- 172.21.15.143:0/3024685461 >> [v2:172.21.15.174:6834/3601088618,v1:172.21.15.174:6836/3601088618] conn(0x55957b365780 0x55957b365b60 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:20.388 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.356+0000 7f0bac370b80 1 --2- 172.21.15.143:0/3024685461 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55957b3346f0 0x55957b364840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:20.388 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.356+0000 7f0bac370b80 1 --2- 172.21.15.143:0/3024685461 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55957b3793a0 0x55957b3eca10 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:20.388 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.356+0000 7f0b78ff9700 0 client.7161 ms_handle_reset on v2:172.21.15.174:6834/3601088618 2023-11-08T19:28:20.388 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.356+0000 7f0bac370b80 1 --2- 172.21.15.143:0/3024685461 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55957b334ad0 0x55957b3ec4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:20.389 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.356+0000 7f0bac370b80 1 -- 172.21.15.143:0/3024685461 >> 172.21.15.143:0/3024685461 conn(0x55957b2af660 msgr2=0x55957b338170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:20.389 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.356+0000 7f0bac370b80 1 -- 172.21.15.143:0/3024685461 shutdown_connections 2023-11-08T19:28:20.389 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.356+0000 7f0bac370b80 1 -- 172.21.15.143:0/3024685461 wait complete. 2023-11-08T19:28:20.389 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.356+0000 7f0bac370b80 20 client.7161 trim_cache size 0 max 16384 2023-11-08T19:28:20.390 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2023-11-08T19:28:20.390 INFO:teuthology.orchestra.run.smithi143.stdout: "cluster_fsid": "968483af-77fc-4068-ad7c-5c2a1a4613d7", 2023-11-08T19:28:20.390 INFO:teuthology.orchestra.run.smithi143.stdout: "whoami": 1, 2023-11-08T19:28:20.391 INFO:teuthology.orchestra.run.smithi143.stdout: "id": 6918, 2023-11-08T19:28:20.391 INFO:teuthology.orchestra.run.smithi143.stdout: "want_state": "up:active", 2023-11-08T19:28:20.391 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "up:active", 2023-11-08T19:28:20.391 INFO:teuthology.orchestra.run.smithi143.stdout: "fs_name": "cephfs", 2023-11-08T19:28:20.391 INFO:teuthology.orchestra.run.smithi143.stdout: "rank_uptime": 2.7623868520000001, 2023-11-08T19:28:20.392 INFO:teuthology.orchestra.run.smithi143.stdout: "mdsmap_epoch": 55, 2023-11-08T19:28:20.392 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_epoch": 92, 2023-11-08T19:28:20.392 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_epoch_barrier": 92, 2023-11-08T19:28:20.392 INFO:teuthology.orchestra.run.smithi143.stdout: "uptime": 22.553602479999999 2023-11-08T19:28:20.392 INFO:teuthology.orchestra.run.smithi143.stdout:} 2023-11-08T19:28:20.394 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.357+0000 7f0ba0177700 1 -- 172.21.15.143:0/3137236859 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f0b7c06af00 msgr2=0x7f0b7c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:20.394 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.357+0000 7f0ba0177700 1 --2- 172.21.15.143:0/3137236859 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f0b7c06af00 0x7f0b7c06d3c0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f0b94000c00 tx=0x7f0b94009040 comp rx=0 tx=0).stop 2023-11-08T19:28:20.394 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.357+0000 7f0ba0177700 1 -- 172.21.15.143:0/3137236859 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0b98157ea0 msgr2=0x7f0b980b5a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:20.395 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.357+0000 7f0ba0177700 1 --2- 172.21.15.143:0/3137236859 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0b98157ea0 0x7f0b980b5a60 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f0b88000c00 tx=0x7f0b88001b90 comp rx=0 tx=0).stop 2023-11-08T19:28:20.395 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.357+0000 7f0ba0177700 1 -- 172.21.15.143:0/3137236859 shutdown_connections 2023-11-08T19:28:20.395 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.357+0000 7f0ba0177700 1 --2- 172.21.15.143:0/3137236859 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f0b7c06af00 0x7f0b7c06d3c0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:20.395 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.357+0000 7f0ba0177700 1 --2- 172.21.15.143:0/3137236859 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0b981571f0 0x7f0b980c7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:20.395 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.357+0000 7f0ba0177700 1 --2- 172.21.15.143:0/3137236859 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0b84003fb0 0x7f0b981578c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:20.396 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.357+0000 7f0ba0177700 1 --2- 172.21.15.143:0/3137236859 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0b98157ea0 0x7f0b980b5a60 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:20.396 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.357+0000 7f0ba0177700 1 -- 172.21.15.143:0/3137236859 >> 172.21.15.143:0/3137236859 conn(0x7f0b9808b000 msgr2=0x7f0b980affa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:20.396 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.357+0000 7f0ba0177700 1 -- 172.21.15.143:0/3137236859 shutdown_connections 2023-11-08T19:28:20.396 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.357+0000 7f0ba0177700 1 -- 172.21.15.143:0/3137236859 wait complete. 2023-11-08T19:28:20.397 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-11-08T19:28:20.518 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.517+0000 7f7143609700 1 Processor -- start 2023-11-08T19:28:20.519 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.517+0000 7f7143609700 1 -- start start 2023-11-08T19:28:20.521 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.518+0000 7f7143609700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f713c06c730 0x7f713c0cd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:20.522 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.518+0000 7f7143609700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f713c06c030 0x7f713c06cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:20.522 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.518+0000 7f7143609700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f713c0cd480 con 0x7f713c15de80 2023-11-08T19:28:20.522 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.518+0000 7f7143609700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f713c0cc610 con 0x7f713c06c030 2023-11-08T19:28:20.522 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.518+0000 7f7143609700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f713c06d050 con 0x7f713c06c730 2023-11-08T19:28:20.523 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.518+0000 7f7140ba4700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f713c06c030 0x7f713c06cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:20.523 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.518+0000 7f7140ba4700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f713c06c030 0x7f713c06cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54926/0 (socket says 172.21.15.143:54926) 2023-11-08T19:28:20.523 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.518+0000 7f7141ba6700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f713c15de80 0x7f713c0cbf00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:51080/0 (socket says 172.21.15.143:51080) 2023-11-08T19:28:20.523 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.518+0000 7f7140ba4700 1 -- 172.21.15.143:0/2046546748 learned_addr learned my addr 172.21.15.143:0/2046546748 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:28:20.524 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.518+0000 7f71413a5700 1 --2- 172.21.15.143:0/2046546748 >> v2:172.21.15.174:3300/0 conn(0x7f713c06c730 0x7f713c0cd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:20.524 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.518+0000 7f7140ba4700 1 -- 172.21.15.143:0/2046546748 >> v2:172.21.15.174:3300/0 conn(0x7f713c06c730 msgr2=0x7f713c0cd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:20.524 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.518+0000 7f7140ba4700 1 --2- 172.21.15.143:0/2046546748 >> v2:172.21.15.174:3300/0 conn(0x7f713c06c730 0x7f713c0cd910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:20.524 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.518+0000 7f7140ba4700 1 -- 172.21.15.143:0/2046546748 >> v1:172.21.15.143:6789/0 conn(0x7f713c15de80 legacy=0x7f713c0cbf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:20.524 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.518+0000 7f7140ba4700 1 -- 172.21.15.143:0/2046546748 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f713c0cc870 con 0x7f713c06c030 2023-11-08T19:28:20.525 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.518+0000 7f7133fff700 1 -- 172.21.15.143:0/2046546748 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1970898207 0 0) 0x7f713c0cd480 con 0x7f713c15de80 2023-11-08T19:28:20.525 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.518+0000 7f71413a5700 1 --2- 172.21.15.143:0/2046546748 >> v2:172.21.15.174:3300/0 conn(0x7f713c06c730 0x7f713c0cd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:28:20.525 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.518+0000 7f7140ba4700 1 --2- 172.21.15.143:0/2046546748 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f713c06c030 0x7f713c06cb10 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f713c08bd60 tx=0x7f712c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=787c9bf717d63e3b server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:20.525 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.519+0000 7f7133fff700 1 -- 172.21.15.143:0/2046546748 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f712c00b870 con 0x7f713c06c030 2023-11-08T19:28:20.525 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.519+0000 7f7133fff700 1 -- 172.21.15.143:0/2046546748 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f712c00b9d0 con 0x7f713c06c030 2023-11-08T19:28:20.526 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.519+0000 7f7133fff700 1 -- 172.21.15.143:0/2046546748 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f712c00bd60 con 0x7f713c06c030 2023-11-08T19:28:20.526 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.519+0000 7f7143609700 1 -- 172.21.15.143:0/2046546748 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f713c06c030 msgr2=0x7f713c06cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:20.526 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.519+0000 7f7143609700 1 --2- 172.21.15.143:0/2046546748 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f713c06c030 0x7f713c06cb10 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f713c08bd60 tx=0x7f712c009bd0 comp rx=0 tx=0).stop 2023-11-08T19:28:20.526 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.519+0000 7f7143609700 1 -- 172.21.15.143:0/2046546748 shutdown_connections 2023-11-08T19:28:20.527 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.519+0000 7f7143609700 1 --2- 172.21.15.143:0/2046546748 >> v2:172.21.15.174:3300/0 conn(0x7f713c06c730 0x7f713c0cd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:20.527 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.519+0000 7f7143609700 1 --2- 172.21.15.143:0/2046546748 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f713c06c030 0x7f713c06cb10 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:20.527 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.519+0000 7f7143609700 1 -- 172.21.15.143:0/2046546748 >> 172.21.15.143:0/2046546748 conn(0x7f713c08b000 msgr2=0x7f713c09ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:20.527 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.519+0000 7f7143609700 1 -- 172.21.15.143:0/2046546748 shutdown_connections 2023-11-08T19:28:20.527 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.519+0000 7f7143609700 1 -- 172.21.15.143:0/2046546748 wait complete. 2023-11-08T19:28:20.528 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.520+0000 7f7143609700 1 Processor -- start 2023-11-08T19:28:20.528 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.520+0000 7f7143609700 1 -- start start 2023-11-08T19:28:20.536 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.521+0000 7f7143609700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f713c06c030 0x7f713c160250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:20.536 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.521+0000 7f7143609700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f713c06c730 0x7f713c15e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:20.536 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.521+0000 7f7143609700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f713c15de80 0x7f713c153920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:20.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.521+0000 7f7143609700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f713c0b6cc0 con 0x7f713c15de80 2023-11-08T19:28:20.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.521+0000 7f7143609700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f713c160790 con 0x7f713c06c730 2023-11-08T19:28:20.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.521+0000 7f7143609700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f713c0b69e0 con 0x7f713c06c030 2023-11-08T19:28:20.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.521+0000 7f7141ba6700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f713c15de80 0x7f713c153920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:20.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.521+0000 7f7141ba6700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f713c15de80 0x7f713c153920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54934/0 (socket says 172.21.15.143:54934) 2023-11-08T19:28:20.538 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.521+0000 7f7141ba6700 1 -- 172.21.15.143:0/3528147075 learned_addr learned my addr 172.21.15.143:0/3528147075 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:28:20.538 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.521+0000 7f7140ba4700 1 --2- 172.21.15.143:0/3528147075 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f713c06c730 0x7f713c15e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:20.538 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.521+0000 7f71413a5700 1 --2- 172.21.15.143:0/3528147075 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f713c06c030 0x7f713c160250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:20.538 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.521+0000 7f7141ba6700 1 -- 172.21.15.143:0/3528147075 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f713c06c030 msgr2=0x7f713c160250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:20.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.521+0000 7f7141ba6700 1 --2- 172.21.15.143:0/3528147075 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f713c06c030 0x7f713c160250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:20.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.521+0000 7f7141ba6700 1 -- 172.21.15.143:0/3528147075 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f713c06c730 msgr2=0x7f713c15e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:20.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.521+0000 7f7141ba6700 1 --2- 172.21.15.143:0/3528147075 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f713c06c730 0x7f713c15e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:20.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.522+0000 7f7141ba6700 1 -- 172.21.15.143:0/3528147075 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f712c009550 con 0x7f713c15de80 2023-11-08T19:28:20.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.522+0000 7f71413a5700 1 --2- 172.21.15.143:0/3528147075 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f713c06c030 0x7f713c160250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:28:20.540 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.522+0000 7f7140ba4700 1 --2- 172.21.15.143:0/3528147075 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f713c06c730 0x7f713c15e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:28:20.540 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.522+0000 7f7141ba6700 1 --2- 172.21.15.143:0/3528147075 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f713c15de80 0x7f713c153920 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f713400cbe0 tx=0x7f713400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:20.540 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.522+0000 7f7131ffb700 1 -- 172.21.15.143:0/3528147075 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f713400abc0 con 0x7f713c15de80 2023-11-08T19:28:20.540 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.522+0000 7f7131ffb700 1 -- 172.21.15.143:0/3528147075 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f713400ad40 con 0x7f713c15de80 2023-11-08T19:28:20.540 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.522+0000 7f7143609700 1 -- 172.21.15.143:0/3528147075 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f713c0b4670 con 0x7f713c15de80 2023-11-08T19:28:20.541 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.522+0000 7f7131ffb700 1 -- 172.21.15.143:0/3528147075 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7134007310 con 0x7f713c15de80 2023-11-08T19:28:20.541 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.522+0000 7f7143609700 1 -- 172.21.15.143:0/3528147075 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f713c0b4c30 con 0x7f713c15de80 2023-11-08T19:28:20.541 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.524+0000 7f7131ffb700 1 -- 172.21.15.143:0/3528147075 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7134007470 con 0x7f713c15de80 2023-11-08T19:28:20.541 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.524+0000 7f7131ffb700 1 --2- 172.21.15.143:0/3528147075 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f712006af00 0x7f712006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:20.541 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.524+0000 7f7131ffb700 1 -- 172.21.15.143:0/3528147075 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6772+0+0 (secure 0 0 0) 0x7f7134089800 con 0x7f713c15de80 2023-11-08T19:28:20.541 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.524+0000 7f71413a5700 1 --2- 172.21.15.143:0/3528147075 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f712006af00 0x7f712006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:20.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.525+0000 7f71413a5700 1 --2- 172.21.15.143:0/3528147075 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f712006af00 0x7f712006d3c0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f713c1595c0 tx=0x7f7138008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:20.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.526+0000 7f714f802b80 1 Processor -- start 2023-11-08T19:28:20.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.526+0000 7f714f802b80 1 -- start start 2023-11-08T19:28:20.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.527+0000 7f714f802b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556d68bd76f0 0x556d68c07450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:20.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.527+0000 7f714f802b80 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x556d68bd9c30 0x556d68c07990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:20.543 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.527+0000 7f714f802b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556d68bd7ad0 0x556d68bf1f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:20.543 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.527+0000 7f714f802b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d68b436e0 con 0x556d68bd7ad0 2023-11-08T19:28:20.543 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.527+0000 7f714f802b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d68b373b0 con 0x556d68bd76f0 2023-11-08T19:28:20.543 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.527+0000 7f714f802b80 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x556d68b34cd0 con 0x556d68bd9c30 2023-11-08T19:28:20.544 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.527+0000 7f7141ba6700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556d68bd7ad0 0x556d68bf1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:20.544 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.527+0000 7f7141ba6700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556d68bd7ad0 0x556d68bf1f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54942/0 (socket says 172.21.15.143:54942) 2023-11-08T19:28:20.544 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.527+0000 7f7141ba6700 1 -- 172.21.15.143:0/4009978373 learned_addr learned my addr 172.21.15.143:0/4009978373 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:28:20.544 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.527+0000 7f71413a5700 1 --2- 172.21.15.143:0/4009978373 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556d68bd76f0 0x556d68c07450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:20.544 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.527+0000 7f7140ba4700 1 --2- 172.21.15.143:0/4009978373 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x556d68bd9c30 0x556d68c07990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:20.545 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.527+0000 7f7141ba6700 1 -- 172.21.15.143:0/4009978373 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x556d68bd9c30 msgr2=0x556d68c07990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:20.545 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.527+0000 7f7141ba6700 1 --2- 172.21.15.143:0/4009978373 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x556d68bd9c30 0x556d68c07990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:20.545 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.527+0000 7f7141ba6700 1 -- 172.21.15.143:0/4009978373 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556d68bd76f0 msgr2=0x556d68c07450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:20.545 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.527+0000 7f7141ba6700 1 --2- 172.21.15.143:0/4009978373 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556d68bd76f0 0x556d68c07450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:20.545 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.527+0000 7f7141ba6700 1 -- 172.21.15.143:0/4009978373 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556d689ad5e0 con 0x556d68bd7ad0 2023-11-08T19:28:20.546 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.527+0000 7f7141ba6700 1 --2- 172.21.15.143:0/4009978373 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556d68bd7ad0 0x556d68bf1f90 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f7134087900 tx=0x7f7134057e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=923e60d6b198230e server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:20.546 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.528+0000 7f7126ffd700 1 -- 172.21.15.143:0/4009978373 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71340588d0 con 0x556d68bd7ad0 2023-11-08T19:28:20.546 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.528+0000 7f7126ffd700 1 -- 172.21.15.143:0/4009978373 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71340581d0 con 0x556d68bd7ad0 2023-11-08T19:28:20.546 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.528+0000 7f7126ffd700 1 -- 172.21.15.143:0/4009978373 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f713405c320 con 0x556d68bd7ad0 2023-11-08T19:28:20.546 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.528+0000 7f714f802b80 1 -- 172.21.15.143:0/4009978373 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556d68bd7ad0 msgr2=0x556d68bf1f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:20.547 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.528+0000 7f714f802b80 1 --2- 172.21.15.143:0/4009978373 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556d68bd7ad0 0x556d68bf1f90 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f7134087900 tx=0x7f7134057e10 comp rx=0 tx=0).stop 2023-11-08T19:28:20.547 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.528+0000 7f714f802b80 1 -- 172.21.15.143:0/4009978373 shutdown_connections 2023-11-08T19:28:20.547 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.528+0000 7f714f802b80 1 --2- 172.21.15.143:0/4009978373 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556d68bd7ad0 0x556d68bf1f90 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:20.547 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.528+0000 7f714f802b80 1 --2- 172.21.15.143:0/4009978373 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556d68bd76f0 0x556d68c07450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:20.547 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.528+0000 7f714f802b80 1 --2- 172.21.15.143:0/4009978373 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x556d68bd9c30 0x556d68c07990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:20.547 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.528+0000 7f714f802b80 1 -- 172.21.15.143:0/4009978373 >> 172.21.15.143:0/4009978373 conn(0x556d68b52660 msgr2=0x556d68bdd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:20.548 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.528+0000 7f714f802b80 1 -- 172.21.15.143:0/4009978373 shutdown_connections 2023-11-08T19:28:20.548 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.528+0000 7f714f802b80 1 -- 172.21.15.143:0/4009978373 wait complete. 2023-11-08T19:28:20.548 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.529+0000 7f714f802b80 1 -- start start 2023-11-08T19:28:20.548 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.529+0000 7f714f802b80 1 Processor -- start 2023-11-08T19:28:20.548 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.530+0000 7f714f802b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556d68bd76f0 0x556d68c0b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:20.549 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.530+0000 7f714f802b80 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x556d68bd7ad0 0x556d68c099f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:20.549 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.530+0000 7f714f802b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556d68c047d0 0x556d68c31780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:20.549 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.530+0000 7f714f802b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d68b305c0 con 0x556d68bd76f0 2023-11-08T19:28:20.549 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.530+0000 7f714f802b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d68acc0d0 con 0x556d68c047d0 2023-11-08T19:28:20.549 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.530+0000 7f714f802b80 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x556d68acbd70 con 0x556d68bd7ad0 2023-11-08T19:28:20.550 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.530+0000 7f714f802b80 10 client.0 _ll_register_callbacks cb 0x556d689287b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T19:28:20.550 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.530+0000 7f71413a5700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556d68bd76f0 0x556d68c0b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:20.550 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.530+0000 7f71413a5700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556d68bd76f0 0x556d68c0b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54944/0 (socket says 172.21.15.143:54944) 2023-11-08T19:28:20.550 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.530+0000 7f71413a5700 1 -- 172.21.15.143:0/2655455614 learned_addr learned my addr 172.21.15.143:0/2655455614 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:28:20.550 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.530+0000 7f7140ba4700 1 --2- 172.21.15.143:0/2655455614 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x556d68bd7ad0 0x556d68c099f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:20.551 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.530+0000 7f7141ba6700 1 --2- 172.21.15.143:0/2655455614 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556d68c047d0 0x556d68c31780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:20.551 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.530+0000 7f71413a5700 1 -- 172.21.15.143:0/2655455614 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x556d68bd7ad0 msgr2=0x556d68c099f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:20.551 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.530+0000 7f71413a5700 1 --2- 172.21.15.143:0/2655455614 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x556d68bd7ad0 0x556d68c099f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:20.552 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.530+0000 7f71413a5700 1 -- 172.21.15.143:0/2655455614 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556d68c047d0 msgr2=0x556d68c31780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:20.552 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.530+0000 7f71413a5700 1 --2- 172.21.15.143:0/2655455614 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556d68c047d0 0x556d68c31780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:20.552 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.530+0000 7f71413a5700 1 -- 172.21.15.143:0/2655455614 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f713c0b4c30 con 0x556d68bd76f0 2023-11-08T19:28:20.552 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.530+0000 7f71413a5700 1 --2- 172.21.15.143:0/2655455614 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556d68bd76f0 0x556d68c0b540 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f7138011eb0 tx=0x7f7138004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6739064c04de27a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:20.553 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.530+0000 7f711bfff700 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2023-11-08T19:28:20.553 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.530+0000 7f711bfff700 1 -- 172.21.15.143:0/2655455614 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7138011ee0 con 0x556d68bd76f0 2023-11-08T19:28:20.553 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.530+0000 7f711bfff700 1 -- 172.21.15.143:0/2655455614 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 6772+0+0 (secure 0 0 0) 0x7f7138017430 con 0x556d68bd76f0 2023-11-08T19:28:20.553 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.530+0000 7f714f802b80 1 -- 172.21.15.143:0/2655455614 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556d68bcf880 con 0x556d68bd76f0 2023-11-08T19:28:20.553 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.531+0000 7f711bfff700 1 -- 172.21.15.143:0/2655455614 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7f71380175f0 con 0x556d68bd76f0 2023-11-08T19:28:20.554 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.531+0000 7f714f802b80 10 client.7176 fetch_fsmap learned FSMap version 55 2023-11-08T19:28:20.554 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.531+0000 7f714f802b80 1 -- 172.21.15.143:0/2655455614 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=2+}) v3 -- 0x556d68959d90 con 0x556d68bd76f0 2023-11-08T19:28:20.554 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.531+0000 7f711bfff700 1 -- 172.21.15.143:0/2655455614 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556d68bcf880 con 0x556d68bd76f0 2023-11-08T19:28:20.554 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.531+0000 7f711bfff700 1 -- 172.21.15.143:0/2655455614 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 55) v1 ==== 3306+0+0 (secure 0 0 0) 0x7f7138017750 con 0x556d68bd76f0 2023-11-08T19:28:20.554 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.531+0000 7f714f802b80 10 client.7176 fetch_fsmap finished waiting for FSMap version 55 2023-11-08T19:28:20.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.531+0000 7f714f802b80 10 client.7176 resolve_mds: resolved name 'd' to daemon mds.d 2023-11-08T19:28:20.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.531+0000 7f714f802b80 20 client.7176 populate_metadata read hostname 'smithi143' 2023-11-08T19:28:20.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.531+0000 7f714f802b80 1 --2- 172.21.15.143:0/2655455614 >> [v2:172.21.15.174:6835/2201447629,v1:172.21.15.174:6837/2201447629] conn(0x556d68bd8700 0x556d68c06700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:20.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.531+0000 7f714f802b80 4 client.7176 mds_command: new command op to 6921 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-08T19:28:20.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.531+0000 7f714f802b80 1 -- 172.21.15.143:0/2655455614 --> [v2:172.21.15.174:6835/2201447629,v1:172.21.15.174:6837/2201447629] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556d68baba30 con 0x556d68bd8700 2023-11-08T19:28:20.556 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.531+0000 7f7140ba4700 1 --2- 172.21.15.143:0/2655455614 >> [v2:172.21.15.174:6835/2201447629,v1:172.21.15.174:6837/2201447629] conn(0x556d68bd8700 0x556d68c06700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:20.556 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.531+0000 7f711bfff700 10 client.7176 ms_handle_connect on v2:172.21.15.174:6835/2201447629 2023-11-08T19:28:20.556 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.531+0000 7f7140ba4700 1 --2- 172.21.15.143:0/2655455614 >> [v2:172.21.15.174:6835/2201447629,v1:172.21.15.174:6837/2201447629] conn(0x556d68bd8700 0x556d68c06700 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59ecc2a2a60b0ef5 server_cookie=9df7eb9d073ddff6 in_seq=0 out_seq=0 2023-11-08T19:28:20.556 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.532+0000 7f711bfff700 1 -- 172.21.15.143:0/2655455614 <== mds.0 v2:172.21.15.174:6835/2201447629 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556d68baba30 con 0x556d68bd8700 2023-11-08T19:28:20.556 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.532+0000 7f711bfff700 10 client.7176 handle_command_reply: tid=0 2023-11-08T19:28:20.557 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.532+0000 7f714f802b80 1 client.7176 shutdown 2023-11-08T19:28:20.557 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.532+0000 7f714f802b80 1 -- 172.21.15.143:0/2655455614 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556d68bd76f0 msgr2=0x556d68c0b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:20.557 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.532+0000 7f714f802b80 1 --2- 172.21.15.143:0/2655455614 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556d68bd76f0 0x556d68c0b540 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f7138011eb0 tx=0x7f7138004750 comp rx=0 tx=0).stop 2023-11-08T19:28:20.557 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.532+0000 7f714f802b80 1 -- 172.21.15.143:0/2655455614 shutdown_connections 2023-11-08T19:28:20.557 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.532+0000 7f714f802b80 1 --2- 172.21.15.143:0/2655455614 >> [v2:172.21.15.174:6835/2201447629,v1:172.21.15.174:6837/2201447629] conn(0x556d68bd8700 0x556d68c06700 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:20.558 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.532+0000 7f714f802b80 1 --2- 172.21.15.143:0/2655455614 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556d68c047d0 0x556d68c31780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:20.558 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.532+0000 7f711bfff700 0 client.7176 ms_handle_reset on v2:172.21.15.174:6835/2201447629 2023-11-08T19:28:20.558 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.532+0000 7f714f802b80 1 --2- 172.21.15.143:0/2655455614 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556d68bd76f0 0x556d68c0b540 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:20.558 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.532+0000 7f714f802b80 1 --2- 172.21.15.143:0/2655455614 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x556d68bd7ad0 0x556d68c099f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:20.558 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.532+0000 7f714f802b80 1 -- 172.21.15.143:0/2655455614 >> 172.21.15.143:0/2655455614 conn(0x556d68b52660 msgr2=0x556d68c24780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:20.559 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.533+0000 7f714f802b80 1 -- 172.21.15.143:0/2655455614 shutdown_connections 2023-11-08T19:28:20.559 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.533+0000 7f714f802b80 1 -- 172.21.15.143:0/2655455614 wait complete. 2023-11-08T19:28:20.559 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.533+0000 7f714f802b80 20 client.7176 trim_cache size 0 max 16384 2023-11-08T19:28:20.565 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.550+0000 7f714f802b80 1 Processor -- start 2023-11-08T19:28:20.565 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.551+0000 7f714f802b80 1 -- start start 2023-11-08T19:28:20.565 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.551+0000 7f714f802b80 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x556d68bd76f0 0x556d68c26420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:20.565 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.551+0000 7f714f802b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556d68bd7ad0 0x556d68bdea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:20.566 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.551+0000 7f714f802b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556d68c047d0 0x556d68c079a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:20.566 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.551+0000 7f714f802b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d689af8d0 con 0x556d68c047d0 2023-11-08T19:28:20.566 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.551+0000 7f714f802b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d68c08940 con 0x556d68bd7ad0 2023-11-08T19:28:20.566 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.551+0000 7f714f802b80 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x556d68c04260 con 0x556d68bd76f0 2023-11-08T19:28:20.566 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.551+0000 7f7141ba6700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556d68c047d0 0x556d68c079a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:20.567 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.551+0000 7f7141ba6700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556d68c047d0 0x556d68c079a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54952/0 (socket says 172.21.15.143:54952) 2023-11-08T19:28:20.567 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.551+0000 7f7141ba6700 1 -- 172.21.15.143:0/456566358 learned_addr learned my addr 172.21.15.143:0/456566358 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:28:20.567 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.551+0000 7f7141ba6700 1 -- 172.21.15.143:0/456566358 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x556d68bd76f0 msgr2=0x556d68c26420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:20.567 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.552+0000 7f7141ba6700 1 --2- 172.21.15.143:0/456566358 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x556d68bd76f0 0x556d68c26420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:20.567 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.552+0000 7f7141ba6700 1 -- 172.21.15.143:0/456566358 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556d68bd7ad0 msgr2=0x556d68bdea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:20.568 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.552+0000 7f7141ba6700 1 --2- 172.21.15.143:0/456566358 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556d68bd7ad0 0x556d68bdea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:20.568 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.552+0000 7f7141ba6700 1 -- 172.21.15.143:0/456566358 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556d68942c10 con 0x556d68c047d0 2023-11-08T19:28:20.568 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.552+0000 7f7141ba6700 1 --2- 172.21.15.143:0/456566358 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556d68c047d0 0x556d68c079a0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f71340879e0 tx=0x7f7134088000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a35ca38f207c2918 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:20.568 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.552+0000 7f71277fe700 1 -- 172.21.15.143:0/456566358 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71340588d0 con 0x556d68c047d0 2023-11-08T19:28:20.568 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.552+0000 7f71277fe700 1 -- 172.21.15.143:0/456566358 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71340604d0 con 0x556d68c047d0 2023-11-08T19:28:20.569 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.552+0000 7f71277fe700 1 -- 172.21.15.143:0/456566358 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f713405c320 con 0x556d68c047d0 2023-11-08T19:28:20.569 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.552+0000 7f714f802b80 1 -- 172.21.15.143:0/456566358 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556d68c047d0 msgr2=0x556d68c079a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:20.569 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.552+0000 7f714f802b80 1 --2- 172.21.15.143:0/456566358 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556d68c047d0 0x556d68c079a0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f71340879e0 tx=0x7f7134088000 comp rx=0 tx=0).stop 2023-11-08T19:28:20.569 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.552+0000 7f714f802b80 1 -- 172.21.15.143:0/456566358 shutdown_connections 2023-11-08T19:28:20.569 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.552+0000 7f714f802b80 1 --2- 172.21.15.143:0/456566358 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x556d68bd76f0 0x556d68c26420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:20.570 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.552+0000 7f714f802b80 1 --2- 172.21.15.143:0/456566358 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556d68c047d0 0x556d68c079a0 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:20.570 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.552+0000 7f714f802b80 1 --2- 172.21.15.143:0/456566358 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556d68bd7ad0 0x556d68bdea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:20.570 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.552+0000 7f714f802b80 1 -- 172.21.15.143:0/456566358 >> 172.21.15.143:0/456566358 conn(0x556d68b52660 msgr2=0x556d68bd37e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:20.570 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.552+0000 7f714f802b80 1 -- 172.21.15.143:0/456566358 shutdown_connections 2023-11-08T19:28:20.570 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.552+0000 7f714f802b80 1 -- 172.21.15.143:0/456566358 wait complete. 2023-11-08T19:28:20.571 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.553+0000 7f714f802b80 1 -- start start 2023-11-08T19:28:20.571 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.553+0000 7f714f802b80 1 Processor -- start 2023-11-08T19:28:20.571 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.554+0000 7f714f802b80 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x556d68bd76f0 0x556d68bf2f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:20.571 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.554+0000 7f714f802b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556d68bd7ad0 0x556d68c82c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:20.572 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.554+0000 7f714f802b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556d68c047d0 0x556d68bf1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:20.572 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.554+0000 7f714f802b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d68c833c0 con 0x556d68bd7ad0 2023-11-08T19:28:20.572 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.554+0000 7f714f802b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d68c187a0 con 0x556d68c047d0 2023-11-08T19:28:20.572 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.554+0000 7f714f802b80 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x556d68c8cd70 con 0x556d68bd76f0 2023-11-08T19:28:20.572 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.554+0000 7f714f802b80 10 client.0 _ll_register_callbacks cb 0x7f7138011b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T19:28:20.573 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.554+0000 7f7140ba4700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556d68bd7ad0 0x556d68c82c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:20.573 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.554+0000 7f7140ba4700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556d68bd7ad0 0x556d68c82c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54960/0 (socket says 172.21.15.143:54960) 2023-11-08T19:28:20.573 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.554+0000 7f7140ba4700 1 -- 172.21.15.143:0/2255699692 learned_addr learned my addr 172.21.15.143:0/2255699692 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:28:20.573 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.554+0000 7f71413a5700 1 --2- 172.21.15.143:0/2255699692 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x556d68bd76f0 0x556d68bf2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:20.573 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.554+0000 7f7141ba6700 1 --2- 172.21.15.143:0/2255699692 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556d68c047d0 0x556d68bf1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:20.574 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.554+0000 7f7140ba4700 1 -- 172.21.15.143:0/2255699692 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x556d68bd76f0 msgr2=0x556d68bf2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:20.574 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.554+0000 7f7140ba4700 1 --2- 172.21.15.143:0/2255699692 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x556d68bd76f0 0x556d68bf2f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:20.574 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.554+0000 7f7140ba4700 1 -- 172.21.15.143:0/2255699692 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556d68c047d0 msgr2=0x556d68bf1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:20.574 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.554+0000 7f7140ba4700 1 --2- 172.21.15.143:0/2255699692 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556d68c047d0 0x556d68bf1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:20.574 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.554+0000 7f7140ba4700 1 -- 172.21.15.143:0/2255699692 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556d689ad5e0 con 0x556d68bd7ad0 2023-11-08T19:28:20.575 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.554+0000 7f7140ba4700 1 --2- 172.21.15.143:0/2255699692 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556d68bd7ad0 0x556d68c82c80 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f712c014040 tx=0x7f712c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce472e428d42cacd server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:20.575 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.554+0000 7f711bfff700 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2023-11-08T19:28:20.575 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.554+0000 7f711bfff700 1 -- 172.21.15.143:0/2255699692 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f712c020040 con 0x556d68bd7ad0 2023-11-08T19:28:20.575 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.555+0000 7f711bfff700 1 -- 172.21.15.143:0/2255699692 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 6772+0+0 (secure 0 0 0) 0x7f712c027840 con 0x556d68bd7ad0 2023-11-08T19:28:20.576 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.555+0000 7f714f802b80 1 -- 172.21.15.143:0/2255699692 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556d68bde840 con 0x556d68bd7ad0 2023-11-08T19:28:20.576 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.555+0000 7f711bfff700 1 -- 172.21.15.143:0/2255699692 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7f712c004030 con 0x556d68bd7ad0 2023-11-08T19:28:20.576 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.555+0000 7f714f802b80 10 client.7182 fetch_fsmap learned FSMap version 55 2023-11-08T19:28:20.576 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.555+0000 7f714f802b80 1 -- 172.21.15.143:0/2255699692 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=2+}) v3 -- 0x556d68980200 con 0x556d68bd7ad0 2023-11-08T19:28:20.576 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.555+0000 7f711bfff700 1 -- 172.21.15.143:0/2255699692 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556d68bde840 con 0x556d68bd7ad0 2023-11-08T19:28:20.576 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.555+0000 7f711bfff700 1 -- 172.21.15.143:0/2255699692 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 55) v1 ==== 3306+0+0 (secure 0 0 0) 0x7f712c02e060 con 0x556d68bd7ad0 2023-11-08T19:28:20.577 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.555+0000 7f714f802b80 10 client.7182 fetch_fsmap finished waiting for FSMap version 55 2023-11-08T19:28:20.577 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.555+0000 7f714f802b80 10 client.7182 resolve_mds: resolved name 'd' to daemon mds.d 2023-11-08T19:28:20.577 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.555+0000 7f714f802b80 20 client.7182 populate_metadata read hostname 'smithi143' 2023-11-08T19:28:20.577 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.555+0000 7f714f802b80 1 --2- 172.21.15.143:0/2255699692 >> [v2:172.21.15.174:6835/2201447629,v1:172.21.15.174:6837/2201447629] conn(0x556d68c8bf20 0x556d68c8c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:20.577 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.555+0000 7f714f802b80 4 client.7182 mds_command: new command op to 6921 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-08T19:28:20.578 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.555+0000 7f714f802b80 1 -- 172.21.15.143:0/2255699692 --> [v2:172.21.15.174:6835/2201447629,v1:172.21.15.174:6837/2201447629] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7f71340581d0 con 0x556d68c8bf20 2023-11-08T19:28:20.578 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.555+0000 7f71413a5700 1 --2- 172.21.15.143:0/2255699692 >> [v2:172.21.15.174:6835/2201447629,v1:172.21.15.174:6837/2201447629] conn(0x556d68c8bf20 0x556d68c8c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:20.578 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.555+0000 7f711bfff700 10 client.7182 ms_handle_connect on v2:172.21.15.174:6835/2201447629 2023-11-08T19:28:20.578 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.555+0000 7f71413a5700 1 --2- 172.21.15.143:0/2255699692 >> [v2:172.21.15.174:6835/2201447629,v1:172.21.15.174:6837/2201447629] conn(0x556d68c8bf20 0x556d68c8c300 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7036f0f18d2ef5a8 server_cookie=136f38ff88518e0 in_seq=0 out_seq=0 2023-11-08T19:28:20.578 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.556+0000 7f711bfff700 1 -- 172.21.15.143:0/2255699692 <== mds.0 v2:172.21.15.174:6835/2201447629 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x7f71340581d0 con 0x556d68c8bf20 2023-11-08T19:28:20.579 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.556+0000 7f711bfff700 10 client.7182 handle_command_reply: tid=0 2023-11-08T19:28:20.579 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.556+0000 7f714f802b80 1 client.7182 shutdown 2023-11-08T19:28:20.579 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.556+0000 7f714f802b80 1 -- 172.21.15.143:0/2255699692 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556d68bd7ad0 msgr2=0x556d68c82c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:20.579 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.556+0000 7f714f802b80 1 --2- 172.21.15.143:0/2255699692 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556d68bd7ad0 0x556d68c82c80 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f712c014040 tx=0x7f712c004750 comp rx=0 tx=0).stop 2023-11-08T19:28:20.579 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.556+0000 7f714f802b80 1 -- 172.21.15.143:0/2255699692 shutdown_connections 2023-11-08T19:28:20.580 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.556+0000 7f714f802b80 1 --2- 172.21.15.143:0/2255699692 >> [v2:172.21.15.174:6835/2201447629,v1:172.21.15.174:6837/2201447629] conn(0x556d68c8bf20 0x556d68c8c300 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:20.580 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.556+0000 7f714f802b80 1 --2- 172.21.15.143:0/2255699692 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x556d68bd76f0 0x556d68bf2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:20.580 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.556+0000 7f711bfff700 0 client.7182 ms_handle_reset on v2:172.21.15.174:6835/2201447629 2023-11-08T19:28:20.580 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.556+0000 7f714f802b80 1 --2- 172.21.15.143:0/2255699692 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556d68c047d0 0x556d68bf1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:20.580 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.556+0000 7f714f802b80 1 --2- 172.21.15.143:0/2255699692 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556d68bd7ad0 0x556d68c82c80 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:20.581 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.556+0000 7f714f802b80 1 -- 172.21.15.143:0/2255699692 >> 172.21.15.143:0/2255699692 conn(0x556d68b52660 msgr2=0x556d68c079a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:20.581 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.556+0000 7f714f802b80 1 -- 172.21.15.143:0/2255699692 shutdown_connections 2023-11-08T19:28:20.581 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.556+0000 7f714f802b80 1 -- 172.21.15.143:0/2255699692 wait complete. 2023-11-08T19:28:20.581 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.556+0000 7f714f802b80 20 client.7182 trim_cache size 0 max 16384 2023-11-08T19:28:20.582 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2023-11-08T19:28:20.582 INFO:teuthology.orchestra.run.smithi143.stdout: "cluster_fsid": "968483af-77fc-4068-ad7c-5c2a1a4613d7", 2023-11-08T19:28:20.582 INFO:teuthology.orchestra.run.smithi143.stdout: "whoami": 0, 2023-11-08T19:28:20.582 INFO:teuthology.orchestra.run.smithi143.stdout: "id": 6921, 2023-11-08T19:28:20.583 INFO:teuthology.orchestra.run.smithi143.stdout: "want_state": "up:active", 2023-11-08T19:28:20.583 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "up:active", 2023-11-08T19:28:20.583 INFO:teuthology.orchestra.run.smithi143.stdout: "fs_name": "cephfs", 2023-11-08T19:28:20.583 INFO:teuthology.orchestra.run.smithi143.stdout: "rank_uptime": 16.862309483000001, 2023-11-08T19:28:20.583 INFO:teuthology.orchestra.run.smithi143.stdout: "mdsmap_epoch": 55, 2023-11-08T19:28:20.584 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_epoch": 92, 2023-11-08T19:28:20.584 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_epoch_barrier": 92, 2023-11-08T19:28:20.584 INFO:teuthology.orchestra.run.smithi143.stdout: "uptime": 22.753284056999998 2023-11-08T19:28:20.584 INFO:teuthology.orchestra.run.smithi143.stdout:} 2023-11-08T19:28:20.586 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.557+0000 7f7143609700 1 -- 172.21.15.143:0/3528147075 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f712006af00 msgr2=0x7f712006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:20.586 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.557+0000 7f7143609700 1 --2- 172.21.15.143:0/3528147075 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f712006af00 0x7f712006d3c0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f713c1595c0 tx=0x7f7138008040 comp rx=0 tx=0).stop 2023-11-08T19:28:20.586 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.557+0000 7f7143609700 1 -- 172.21.15.143:0/3528147075 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f713c15de80 msgr2=0x7f713c153920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:20.586 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.557+0000 7f7143609700 1 --2- 172.21.15.143:0/3528147075 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f713c15de80 0x7f713c153920 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f713400cbe0 tx=0x7f713400a060 comp rx=0 tx=0).stop 2023-11-08T19:28:20.587 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.557+0000 7f7143609700 1 -- 172.21.15.143:0/3528147075 shutdown_connections 2023-11-08T19:28:20.587 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.557+0000 7f7143609700 1 --2- 172.21.15.143:0/3528147075 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f712006af00 0x7f712006d3c0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:20.587 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.557+0000 7f7143609700 1 --2- 172.21.15.143:0/3528147075 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f713c06c030 0x7f713c160250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:20.587 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.557+0000 7f7143609700 1 --2- 172.21.15.143:0/3528147075 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f713c15de80 0x7f713c153920 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:20.587 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.557+0000 7f7143609700 1 --2- 172.21.15.143:0/3528147075 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f713c06c730 0x7f713c15e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:20.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.557+0000 7f7143609700 1 -- 172.21.15.143:0/3528147075 >> 172.21.15.143:0/3528147075 conn(0x7f713c08b000 msgr2=0x7f713c0d0090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:20.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.557+0000 7f7143609700 1 -- 172.21.15.143:0/3528147075 shutdown_connections 2023-11-08T19:28:20.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.557+0000 7f7143609700 1 -- 172.21.15.143:0/3528147075 wait complete. 2023-11-08T19:28:20.588 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:28:20.589 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p d1/dir) 2023-11-08T19:28:20.612 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:28:20.612 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 1 d1) 2023-11-08T19:28:20.683 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:28:20.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.842+0000 7f04f7019700 1 Processor -- start 2023-11-08T19:28:20.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.842+0000 7f04f7019700 1 -- start start 2023-11-08T19:28:20.846 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.843+0000 7f04f7019700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f04f01618f0 0x7f04f006ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:20.847 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.843+0000 7f04f7019700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f04f006b150 0x7f04f00cb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:20.847 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.843+0000 7f04f7019700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04f0161df0 con 0x7f04f015dd00 2023-11-08T19:28:20.847 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.843+0000 7f04f7019700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04f006b600 con 0x7f04f01618f0 2023-11-08T19:28:20.847 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.843+0000 7f04f7019700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04f00cad50 con 0x7f04f006b150 2023-11-08T19:28:20.848 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.843+0000 7f04effff700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f04f01618f0 0x7f04f006ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:20.848 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.843+0000 7f04effff700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f04f01618f0 0x7f04f006ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54968/0 (socket says 172.21.15.143:54968) 2023-11-08T19:28:20.848 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.843+0000 7f04effff700 1 -- 172.21.15.143:0/292742099 learned_addr learned my addr 172.21.15.143:0/292742099 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:28:20.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.844+0000 7f04effff700 1 -- 172.21.15.143:0/292742099 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f04f006b150 msgr2=0x7f04f00cb1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:20.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.844+0000 7f04effff700 1 --2- 172.21.15.143:0/292742099 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f04f006b150 0x7f04f00cb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:20.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.844+0000 7f04effff700 1 -- 172.21.15.143:0/292742099 >> v1:172.21.15.174:6789/0 conn(0x7f04f015dd00 legacy=0x7f04f00cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:20.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.844+0000 7f04effff700 1 -- 172.21.15.143:0/292742099 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04f01620d0 con 0x7f04f01618f0 2023-11-08T19:28:20.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.844+0000 7f04effff700 1 --2- 172.21.15.143:0/292742099 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f04f01618f0 0x7f04f006ac10 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f04dc0098f0 tx=0x7f04dc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a4b02b4bc132d02 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:20.850 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.844+0000 7f04ef7fe700 1 -- 172.21.15.143:0/292742099 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04dc004230 con 0x7f04f01618f0 2023-11-08T19:28:20.850 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.844+0000 7f04ef7fe700 1 -- 172.21.15.143:0/292742099 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04dc004390 con 0x7f04f01618f0 2023-11-08T19:28:20.850 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.844+0000 7f04ef7fe700 1 -- 172.21.15.143:0/292742099 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04dc0036a0 con 0x7f04f01618f0 2023-11-08T19:28:20.850 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.845+0000 7f04f7019700 1 -- 172.21.15.143:0/292742099 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f04f01618f0 msgr2=0x7f04f006ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:20.850 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.845+0000 7f04f7019700 1 --2- 172.21.15.143:0/292742099 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f04f01618f0 0x7f04f006ac10 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f04dc0098f0 tx=0x7f04dc009c00 comp rx=0 tx=0).stop 2023-11-08T19:28:20.851 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.845+0000 7f04f7019700 1 -- 172.21.15.143:0/292742099 shutdown_connections 2023-11-08T19:28:20.851 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.845+0000 7f04f7019700 1 --2- 172.21.15.143:0/292742099 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f04f006b150 0x7f04f00cb1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:20.851 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.845+0000 7f04f7019700 1 --2- 172.21.15.143:0/292742099 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f04f01618f0 0x7f04f006ac10 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:20.851 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.845+0000 7f04f7019700 1 -- 172.21.15.143:0/292742099 >> 172.21.15.143:0/292742099 conn(0x7f04f0099ab0 msgr2=0x7f04f00d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:20.851 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.845+0000 7f04f7019700 1 -- 172.21.15.143:0/292742099 shutdown_connections 2023-11-08T19:28:20.852 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.845+0000 7f04f7019700 1 -- 172.21.15.143:0/292742099 wait complete. 2023-11-08T19:28:20.854 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.846+0000 7f04f7019700 1 Processor -- start 2023-11-08T19:28:20.854 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.846+0000 7f04f7019700 1 -- start start 2023-11-08T19:28:20.855 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.847+0000 7f04f7019700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f04f006b150 0x7f04f0145d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:20.855 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.847+0000 7f04f7019700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f04f015dd00 0x7f04f0156d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:20.855 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.847+0000 7f04f7019700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f04f01618f0 0x7f04f0157280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:20.855 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.847+0000 7f04f7019700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04f00c4600 con 0x7f04f006b150 2023-11-08T19:28:20.855 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.847+0000 7f04f7019700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04f00c42d0 con 0x7f04f015dd00 2023-11-08T19:28:20.856 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.847+0000 7f04f7019700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04f015e540 con 0x7f04f01618f0 2023-11-08T19:28:20.856 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.847+0000 7f04f4db5700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f04f006b150 0x7f04f0145d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:20.856 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.847+0000 7f04f4db5700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f04f006b150 0x7f04f0145d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54972/0 (socket says 172.21.15.143:54972) 2023-11-08T19:28:20.856 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.847+0000 7f04f4db5700 1 -- 172.21.15.143:0/503579364 learned_addr learned my addr 172.21.15.143:0/503579364 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:28:20.856 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.847+0000 7f04effff700 1 --2- 172.21.15.143:0/503579364 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f04f015dd00 0x7f04f0156d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:20.857 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.847+0000 7f04f55b6700 1 --2- 172.21.15.143:0/503579364 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f04f01618f0 0x7f04f0157280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:20.857 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.847+0000 7f04f4db5700 1 -- 172.21.15.143:0/503579364 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f04f01618f0 msgr2=0x7f04f0157280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:20.857 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.847+0000 7f04f4db5700 1 --2- 172.21.15.143:0/503579364 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f04f01618f0 0x7f04f0157280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:20.857 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.847+0000 7f04f4db5700 1 -- 172.21.15.143:0/503579364 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f04f015dd00 msgr2=0x7f04f0156d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:20.857 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.847+0000 7f04f4db5700 1 --2- 172.21.15.143:0/503579364 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f04f015dd00 0x7f04f0156d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:20.858 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.847+0000 7f04f4db5700 1 -- 172.21.15.143:0/503579364 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04dc009550 con 0x7f04f006b150 2023-11-08T19:28:20.858 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.847+0000 7f04f4db5700 1 --2- 172.21.15.143:0/503579364 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f04f006b150 0x7f04f0145d10 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f04e400c520 tx=0x7f04e400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:20.858 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.847+0000 7f04ed7fa700 1 -- 172.21.15.143:0/503579364 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04e40054e0 con 0x7f04f006b150 2023-11-08T19:28:20.858 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.847+0000 7f04ed7fa700 1 -- 172.21.15.143:0/503579364 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04e40036d0 con 0x7f04f006b150 2023-11-08T19:28:20.858 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.847+0000 7f04f7019700 1 -- 172.21.15.143:0/503579364 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04f015e820 con 0x7f04f006b150 2023-11-08T19:28:20.859 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.847+0000 7f04ed7fa700 1 -- 172.21.15.143:0/503579364 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04e4003a80 con 0x7f04f006b150 2023-11-08T19:28:20.859 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.848+0000 7f04f7019700 1 -- 172.21.15.143:0/503579364 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04f01530f0 con 0x7f04f006b150 2023-11-08T19:28:20.859 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.848+0000 7f04f7019700 1 -- 172.21.15.143:0/503579364 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04f00c4440 con 0x7f04f006b150 2023-11-08T19:28:20.859 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.849+0000 7f04ed7fa700 1 -- 172.21.15.143:0/503579364 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f04e4003be0 con 0x7f04f006b150 2023-11-08T19:28:20.859 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.849+0000 7f04ed7fa700 1 --2- 172.21.15.143:0/503579364 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f04d406af00 0x7f04d406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:20.860 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.849+0000 7f04ed7fa700 1 -- 172.21.15.143:0/503579364 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6772+0+0 (secure 0 0 0) 0x7f04e4088da0 con 0x7f04f006b150 2023-11-08T19:28:20.860 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.850+0000 7f04effff700 1 --2- 172.21.15.143:0/503579364 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f04d406af00 0x7f04d406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:20.860 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.850+0000 7f04effff700 1 --2- 172.21.15.143:0/503579364 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f04d406af00 0x7f04d406d3c0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f04dc009130 tx=0x7f04dc008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:20.860 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:20.854+0000 7f04ed7fa700 1 -- 172.21.15.143:0/503579364 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f04f00c4440 con 0x7f04f006b150 2023-11-08T19:28:21.088 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:21.087+0000 7f04f7019700 1 -- 172.21.15.143:0/503579364 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f04f00c4440 con 0x7f04f006b150 2023-11-08T19:28:21.089 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:21.087+0000 7f04ed7fa700 1 -- 172.21.15.143:0/503579364 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+4941 (secure 0 0 0) 0x7f04f00c4440 con 0x7f04f006b150 2023-11-08T19:28:21.089 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:28:21.089 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":55,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6906,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6834/3172664377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":3172664377},{"type":"v1","addr":"172.21.15.143:6834","nonce":3172664377}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47},{"gid":6909,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3953059900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":3953059900},{"type":"v1","addr":"172.21.15.143:6835","nonce":3953059900}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":46}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:03.092245+0000","modified":"2023-11-08T19:28:17.609590+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6921,"mds_1":6918},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6918":{"gid":6918,"name":"b","rank":1,"incarnation":54,"state":"up:active","state_seq":6,"addr":"172.21.15.174:6836/3601088618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3601088618},{"type":"v1","addr":"172.21.15.174:6836","nonce":3601088618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6921":{"gid":6921,"name":"d","rank":0,"incarnation":48,"state":"up:active","state_seq":3,"addr":"172.21.15.174:6837/2201447629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6835","nonce":2201447629},{"type":"v1","addr":"172.21.15.174:6837","nonce":2201447629}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:28:21.091 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:21.090+0000 7f04f7019700 1 -- 172.21.15.143:0/503579364 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f04d406af00 msgr2=0x7f04d406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:21.092 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:21.090+0000 7f04f7019700 1 --2- 172.21.15.143:0/503579364 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f04d406af00 0x7f04d406d3c0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f04dc009130 tx=0x7f04dc008030 comp rx=0 tx=0).stop 2023-11-08T19:28:21.092 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:21.090+0000 7f04f7019700 1 -- 172.21.15.143:0/503579364 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f04f006b150 msgr2=0x7f04f0145d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:21.092 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:21.090+0000 7f04f7019700 1 --2- 172.21.15.143:0/503579364 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f04f006b150 0x7f04f0145d10 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f04e400c520 tx=0x7f04e400c830 comp rx=0 tx=0).stop 2023-11-08T19:28:21.092 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:21.090+0000 7f04f7019700 1 -- 172.21.15.143:0/503579364 shutdown_connections 2023-11-08T19:28:21.093 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:21.090+0000 7f04f7019700 1 --2- 172.21.15.143:0/503579364 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f04d406af00 0x7f04d406d3c0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:21.093 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:21.090+0000 7f04f7019700 1 --2- 172.21.15.143:0/503579364 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f04f01618f0 0x7f04f0157280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:21.093 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:21.090+0000 7f04f7019700 1 --2- 172.21.15.143:0/503579364 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f04f006b150 0x7f04f0145d10 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:21.093 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:21.090+0000 7f04f7019700 1 --2- 172.21.15.143:0/503579364 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f04f015dd00 0x7f04f0156d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:21.093 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:21.090+0000 7f04f7019700 1 -- 172.21.15.143:0/503579364 >> 172.21.15.143:0/503579364 conn(0x7f04f0099ab0 msgr2=0x7f04f006e890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:21.094 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:21.090+0000 7f04f7019700 1 -- 172.21.15.143:0/503579364 shutdown_connections 2023-11-08T19:28:21.094 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:21.090+0000 7f04f7019700 1 -- 172.21.15.143:0/503579364 wait complete. 2023-11-08T19:28:21.094 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 55 2023-11-08T19:28:21.103 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json get subtrees 2023-11-08T19:28:21.254 INFO:teuthology.orchestra.run.smithi174.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2023-11-08T19:28:21.254 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2023-11-08T19:28:21.255 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1', 1)] 2023-11-08T19:28:23.256 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:28:23.389 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:23.388+0000 7f14f0f99700 1 Processor -- start 2023-11-08T19:28:23.390 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:23.388+0000 7f14f0f99700 1 -- start start 2023-11-08T19:28:23.392 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:23.389+0000 7f14f0f99700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f14ec15aa30 0x7f14ec0cf510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:23.392 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:23.389+0000 7f14f0f99700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f14ec0b2210 0x7f14ec0b26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:23.393 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:23.389+0000 7f14f0f99700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14ec0cf080 con 0x7f14ec0cbe20 2023-11-08T19:28:23.393 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:23.389+0000 7f14f0f99700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14ec0b2be0 con 0x7f14ec0b2210 2023-11-08T19:28:23.393 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:23.389+0000 7f14f0f99700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14ec0ce470 con 0x7f14ec15aa30 2023-11-08T19:28:23.393 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:23.389+0000 7f14e9d9b700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f14ec0b2210 0x7f14ec0b26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:23.394 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:23.389+0000 7f14ead9d700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f14ec0cbe20 0x7f14ec0cc220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:51134/0 (socket says 172.21.15.143:51134) 2023-11-08T19:28:23.394 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:23.389+0000 7f14e9d9b700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f14ec0b2210 0x7f14ec0b26a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54980/0 (socket says 172.21.15.143:54980) 2023-11-08T19:28:23.394 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:23.389+0000 7f14ead9d700 1 -- 172.21.15.143:0/2433267076 learned_addr learned my addr 172.21.15.143:0/2433267076 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:28:23.394 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:23.390+0000 7f14ea59c700 1 --2- 172.21.15.143:0/2433267076 >> v2:172.21.15.174:3300/0 conn(0x7f14ec15aa30 0x7f14ec0cf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:23.394 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:23.390+0000 7f14e9d9b700 1 -- 172.21.15.143:0/2433267076 >> v2:172.21.15.174:3300/0 conn(0x7f14ec15aa30 msgr2=0x7f14ec0cf510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:23.395 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:23.390+0000 7f14e9d9b700 1 --2- 172.21.15.143:0/2433267076 >> v2:172.21.15.174:3300/0 conn(0x7f14ec15aa30 0x7f14ec0cf510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:23.395 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:23.390+0000 7f14e9d9b700 1 -- 172.21.15.143:0/2433267076 >> v1:172.21.15.143:6789/0 conn(0x7f14ec0cbe20 legacy=0x7f14ec0cc220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:23.395 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:23.390+0000 7f14e959a700 1 -- 172.21.15.143:0/2433267076 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2700951036 0 0) 0x7f14ec0cf080 con 0x7f14ec0cbe20 2023-11-08T19:28:23.395 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:23.390+0000 7f14e9d9b700 1 -- 172.21.15.143:0/2433267076 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14ec0ce750 con 0x7f14ec0b2210 2023-11-08T19:28:23.395 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:23.390+0000 7f14e9d9b700 1 --2- 172.21.15.143:0/2433267076 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f14ec0b2210 0x7f14ec0b26a0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f14ec071c90 tx=0x7f14dc009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aab175afa6878f82 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:23.396 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:23.390+0000 7f14e959a700 1 -- 172.21.15.143:0/2433267076 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14dc011830 con 0x7f14ec0b2210 2023-11-08T19:28:23.396 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:23.390+0000 7f14e959a700 1 -- 172.21.15.143:0/2433267076 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14dc011990 con 0x7f14ec0b2210 2023-11-08T19:28:23.396 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:23.390+0000 7f14e959a700 1 -- 172.21.15.143:0/2433267076 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14dc011d20 con 0x7f14ec0b2210 2023-11-08T19:28:23.396 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:23.391+0000 7f14f0f99700 1 -- 172.21.15.143:0/2433267076 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f14ec0b2210 msgr2=0x7f14ec0b26a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:23.396 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:23.391+0000 7f14f0f99700 1 --2- 172.21.15.143:0/2433267076 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f14ec0b2210 0x7f14ec0b26a0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f14ec071c90 tx=0x7f14dc009b20 comp rx=0 tx=0).stop 2023-11-08T19:28:23.397 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:23.391+0000 7f14f0f99700 1 -- 172.21.15.143:0/2433267076 shutdown_connections 2023-11-08T19:28:23.397 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:23.391+0000 7f14f0f99700 1 --2- 172.21.15.143:0/2433267076 >> v2:172.21.15.174:3300/0 conn(0x7f14ec15aa30 0x7f14ec0cf510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:23.397 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:23.391+0000 7f14f0f99700 1 --2- 172.21.15.143:0/2433267076 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f14ec0b2210 0x7f14ec0b26a0 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:23.397 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:23.391+0000 7f14f0f99700 1 -- 172.21.15.143:0/2433267076 >> 172.21.15.143:0/2433267076 conn(0x7f14ec099ab0 msgr2=0x7f14ec0d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:23.397 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:23.391+0000 7f14f0f99700 1 -- 172.21.15.143:0/2433267076 shutdown_connections 2023-11-08T19:28:23.398 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:23.391+0000 7f14f0f99700 1 -- 172.21.15.143:0/2433267076 wait complete. 2023-11-08T19:28:23.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:23.392+0000 7f14f0f99700 1 Processor -- start 2023-11-08T19:28:23.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:23.393+0000 7f14f0f99700 1 -- start start 2023-11-08T19:28:23.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:23.393+0000 7f14f0f99700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f14ec0b2210 0x7f14ec0b5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:23.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:23.393+0000 7f14f0f99700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f14ec0cbe20 0x7f14ec0b36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:23.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:23.393+0000 7f14f0f99700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f14ec15aa30 0x7f14ec06db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:23.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:23.393+0000 7f14f0f99700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14ec0b61f0 con 0x7f14ec0b2210 2023-11-08T19:28:23.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:23.393+0000 7f14f0f99700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14ec138260 con 0x7f14ec15aa30 2023-11-08T19:28:23.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:23.393+0000 7f14f0f99700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14ec138540 con 0x7f14ec0cbe20 2023-11-08T19:28:23.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:23.394+0000 7f14ea59c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f14ec0b2210 0x7f14ec0b5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:23.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:23.394+0000 7f14ea59c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f14ec0b2210 0x7f14ec0b5cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54982/0 (socket says 172.21.15.143:54982) 2023-11-08T19:28:23.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:23.394+0000 7f14ea59c700 1 -- 172.21.15.143:0/3529766509 learned_addr learned my addr 172.21.15.143:0/3529766509 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:28:23.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:23.394+0000 7f14ead9d700 1 --2- 172.21.15.143:0/3529766509 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f14ec15aa30 0x7f14ec06db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:23.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:23.394+0000 7f14e9d9b700 1 --2- 172.21.15.143:0/3529766509 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f14ec0cbe20 0x7f14ec0b36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:23.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:23.394+0000 7f14ea59c700 1 -- 172.21.15.143:0/3529766509 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f14ec0cbe20 msgr2=0x7f14ec0b36c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:23.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:23.394+0000 7f14ea59c700 1 --2- 172.21.15.143:0/3529766509 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f14ec0cbe20 0x7f14ec0b36c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:23.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:23.394+0000 7f14ea59c700 1 -- 172.21.15.143:0/3529766509 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f14ec15aa30 msgr2=0x7f14ec06db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:23.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:23.394+0000 7f14ea59c700 1 --2- 172.21.15.143:0/3529766509 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f14ec15aa30 0x7f14ec06db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:23.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:23.394+0000 7f14ea59c700 1 -- 172.21.15.143:0/3529766509 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14dc0094a0 con 0x7f14ec0b2210 2023-11-08T19:28:23.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:23.394+0000 7f14e9d9b700 1 --2- 172.21.15.143:0/3529766509 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f14ec0cbe20 0x7f14ec0b36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:28:23.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:23.394+0000 7f14ea59c700 1 --2- 172.21.15.143:0/3529766509 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f14ec0b2210 0x7f14ec0b5cb0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f14e000d7d0 tx=0x7f14e000db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:23.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:23.394+0000 7f14daffd700 1 -- 172.21.15.143:0/3529766509 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14e00125f0 con 0x7f14ec0b2210 2023-11-08T19:28:23.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:23.394+0000 7f14daffd700 1 -- 172.21.15.143:0/3529766509 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14e0012750 con 0x7f14ec0b2210 2023-11-08T19:28:23.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:23.394+0000 7f14f0f99700 1 -- 172.21.15.143:0/3529766509 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14ec138820 con 0x7f14ec0b2210 2023-11-08T19:28:23.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:23.394+0000 7f14daffd700 1 -- 172.21.15.143:0/3529766509 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14e0012b00 con 0x7f14ec0b2210 2023-11-08T19:28:23.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:23.394+0000 7f14f0f99700 1 -- 172.21.15.143:0/3529766509 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14ec14b3d0 con 0x7f14ec0b2210 2023-11-08T19:28:23.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:23.395+0000 7f14f0f99700 1 -- 172.21.15.143:0/3529766509 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14ec138e70 con 0x7f14ec0b2210 2023-11-08T19:28:23.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:23.396+0000 7f14daffd700 1 -- 172.21.15.143:0/3529766509 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f14e0012c60 con 0x7f14ec0b2210 2023-11-08T19:28:23.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:23.396+0000 7f14daffd700 1 --2- 172.21.15.143:0/3529766509 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f14cc06af00 0x7f14cc06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:23.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:23.396+0000 7f14daffd700 1 -- 172.21.15.143:0/3529766509 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6772+0+0 (secure 0 0 0) 0x7f14e0089c00 con 0x7f14ec0b2210 2023-11-08T19:28:23.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:23.396+0000 7f14e9d9b700 1 --2- 172.21.15.143:0/3529766509 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f14cc06af00 0x7f14cc06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:23.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:23.397+0000 7f14e9d9b700 1 --2- 172.21.15.143:0/3529766509 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f14cc06af00 0x7f14cc06d3c0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f14ec14b7a0 tx=0x7f14dc008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:23.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:23.401+0000 7f14daffd700 1 -- 172.21.15.143:0/3529766509 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f14e00547e0 con 0x7f14ec0b2210 2023-11-08T19:28:23.631 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:23.630+0000 7f14f0f99700 1 -- 172.21.15.143:0/3529766509 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f14ec0b2ff0 con 0x7f14ec0b2210 2023-11-08T19:28:23.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:23.630+0000 7f14daffd700 1 -- 172.21.15.143:0/3529766509 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+4941 (secure 0 0 0) 0x7f14e0057e00 con 0x7f14ec0b2210 2023-11-08T19:28:23.632 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:28:23.632 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":55,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6906,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6834/3172664377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":3172664377},{"type":"v1","addr":"172.21.15.143:6834","nonce":3172664377}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47},{"gid":6909,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3953059900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":3953059900},{"type":"v1","addr":"172.21.15.143:6835","nonce":3953059900}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":46}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:03.092245+0000","modified":"2023-11-08T19:28:17.609590+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6921,"mds_1":6918},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6918":{"gid":6918,"name":"b","rank":1,"incarnation":54,"state":"up:active","state_seq":6,"addr":"172.21.15.174:6836/3601088618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3601088618},{"type":"v1","addr":"172.21.15.174:6836","nonce":3601088618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6921":{"gid":6921,"name":"d","rank":0,"incarnation":48,"state":"up:active","state_seq":3,"addr":"172.21.15.174:6837/2201447629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6835","nonce":2201447629},{"type":"v1","addr":"172.21.15.174:6837","nonce":2201447629}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:28:23.634 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:23.633+0000 7f14f0f99700 1 -- 172.21.15.143:0/3529766509 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f14cc06af00 msgr2=0x7f14cc06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:23.635 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:23.633+0000 7f14f0f99700 1 --2- 172.21.15.143:0/3529766509 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f14cc06af00 0x7f14cc06d3c0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f14ec14b7a0 tx=0x7f14dc008030 comp rx=0 tx=0).stop 2023-11-08T19:28:23.635 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:23.633+0000 7f14f0f99700 1 -- 172.21.15.143:0/3529766509 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f14ec0b2210 msgr2=0x7f14ec0b5cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:23.635 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:23.633+0000 7f14f0f99700 1 --2- 172.21.15.143:0/3529766509 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f14ec0b2210 0x7f14ec0b5cb0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f14e000d7d0 tx=0x7f14e000db90 comp rx=0 tx=0).stop 2023-11-08T19:28:23.635 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:23.633+0000 7f14f0f99700 1 -- 172.21.15.143:0/3529766509 shutdown_connections 2023-11-08T19:28:23.635 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:23.633+0000 7f14f0f99700 1 --2- 172.21.15.143:0/3529766509 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f14cc06af00 0x7f14cc06d3c0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:23.636 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:23.633+0000 7f14f0f99700 1 --2- 172.21.15.143:0/3529766509 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f14ec15aa30 0x7f14ec06db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:23.636 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:23.633+0000 7f14f0f99700 1 --2- 172.21.15.143:0/3529766509 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f14ec0b2210 0x7f14ec0b5cb0 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:23.636 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:23.633+0000 7f14f0f99700 1 --2- 172.21.15.143:0/3529766509 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f14ec0cbe20 0x7f14ec0b36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:23.636 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:23.633+0000 7f14f0f99700 1 -- 172.21.15.143:0/3529766509 >> 172.21.15.143:0/3529766509 conn(0x7f14ec099ab0 msgr2=0x7f14ec06ec80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:23.636 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:23.633+0000 7f14f0f99700 1 -- 172.21.15.143:0/3529766509 shutdown_connections 2023-11-08T19:28:23.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:23.633+0000 7f14f0f99700 1 -- 172.21.15.143:0/3529766509 wait complete. 2023-11-08T19:28:23.637 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 55 2023-11-08T19:28:23.647 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json get subtrees 2023-11-08T19:28:23.796 INFO:teuthology.orchestra.run.smithi174.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2023-11-08T19:28:23.797 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2023-11-08T19:28:23.797 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1', 1)] 2023-11-08T19:28:25.798 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:28:25.931 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:25.929+0000 7f0a5f62c700 1 Processor -- start 2023-11-08T19:28:25.931 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:25.929+0000 7f0a5f62c700 1 -- start start 2023-11-08T19:28:25.935 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:25.930+0000 7f0a5f62c700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0a5806c730 0x7f0a580cd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:25.935 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:25.930+0000 7f0a5f62c700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a580cd480 con 0x7f0a5806c030 2023-11-08T19:28:25.935 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:25.930+0000 7f0a5f62c700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a580cc610 con 0x7f0a5815de80 2023-11-08T19:28:25.936 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:25.930+0000 7f0a5f62c700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a580cc870 con 0x7f0a5806c730 2023-11-08T19:28:25.936 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:25.930+0000 7f0a5cbc7700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f0a5806c030 0x7f0a5806cb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:51146/0 (socket says 172.21.15.143:51146) 2023-11-08T19:28:25.936 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:25.930+0000 7f0a5cbc7700 1 -- 172.21.15.143:0/2166067460 learned_addr learned my addr 172.21.15.143:0/2166067460 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:28:25.936 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:25.930+0000 7f0a4ffff700 1 -- 172.21.15.143:0/2166067460 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 281207473 0 0) 0x7f0a580cd480 con 0x7f0a5806c030 2023-11-08T19:28:25.936 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:25.931+0000 7f0a4ffff700 1 -- 172.21.15.143:0/2166067460 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a440035b0 con 0x7f0a5806c030 2023-11-08T19:28:25.937 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:25.931+0000 7f0a5d3c8700 1 --2- 172.21.15.143:0/2166067460 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0a5806c730 0x7f0a580cd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:25.937 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:25.931+0000 7f0a4ffff700 1 -- 172.21.15.143:0/2166067460 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1684288281 0 0) 0x7f0a440035b0 con 0x7f0a5806c030 2023-11-08T19:28:25.937 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:25.931+0000 7f0a4ffff700 1 -- 172.21.15.143:0/2166067460 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0a5806c730 msgr2=0x7f0a580cd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:25.937 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:25.931+0000 7f0a4ffff700 1 --2- 172.21.15.143:0/2166067460 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0a5806c730 0x7f0a580cd910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:25.937 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:25.931+0000 7f0a4ffff700 1 -- 172.21.15.143:0/2166067460 >> v1:172.21.15.174:6789/0 conn(0x7f0a5815de80 legacy=0x7f0a580cbf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:25.938 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:25.931+0000 7f0a4ffff700 1 -- 172.21.15.143:0/2166067460 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a580ccb50 con 0x7f0a5806c030 2023-11-08T19:28:25.938 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:25.931+0000 7f0a4ffff700 1 -- 172.21.15.143:0/2166067460 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4147756303 0 0) 0x7f0a54001670 con 0x7f0a5806c030 2023-11-08T19:28:25.938 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:25.931+0000 7f0a4ffff700 1 -- 172.21.15.143:0/2166067460 >> v1:172.21.15.143:6789/0 conn(0x7f0a5806c030 legacy=0x7f0a5806cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:25.938 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:25.931+0000 7f0a4ffff700 1 --2- 172.21.15.143:0/2166067460 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0a5806c730 0x7f0a44003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:25.938 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:25.931+0000 7f0a4ffff700 1 --2- 172.21.15.143:0/2166067460 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0a44004060 0x7f0a44004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:25.939 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:25.931+0000 7f0a4ffff700 1 --2- 172.21.15.143:0/2166067460 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0a44004d90 0x7f0a44005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:25.939 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:25.931+0000 7f0a4ffff700 1 -- 172.21.15.143:0/2166067460 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a580cc870 con 0x7f0a44004060 2023-11-08T19:28:25.939 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:25.931+0000 7f0a4ffff700 1 -- 172.21.15.143:0/2166067460 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a4400bbd0 con 0x7f0a44004d90 2023-11-08T19:28:25.939 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:25.931+0000 7f0a4ffff700 1 -- 172.21.15.143:0/2166067460 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a4400bd90 con 0x7f0a5806c730 2023-11-08T19:28:25.939 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:25.931+0000 7f0a5cbc7700 1 --2- 172.21.15.143:0/2166067460 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0a44004060 0x7f0a44004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:25.940 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:25.931+0000 7f0a5dbc9700 1 --2- 172.21.15.143:0/2166067460 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0a44004d90 0x7f0a44005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:25.941 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:25.931+0000 7f0a5d3c8700 1 --2- 172.21.15.143:0/2166067460 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0a5806c730 0x7f0a44003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:25.941 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:25.931+0000 7f0a5cbc7700 1 -- 172.21.15.143:0/2166067460 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0a5806c730 msgr2=0x7f0a44003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:25.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:25.931+0000 7f0a5cbc7700 1 --2- 172.21.15.143:0/2166067460 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0a5806c730 0x7f0a44003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:25.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:25.931+0000 7f0a5cbc7700 1 -- 172.21.15.143:0/2166067460 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0a44004d90 msgr2=0x7f0a44005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:25.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:25.931+0000 7f0a5cbc7700 1 --2- 172.21.15.143:0/2166067460 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0a44004d90 0x7f0a44005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:25.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:25.931+0000 7f0a5cbc7700 1 -- 172.21.15.143:0/2166067460 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a440035b0 con 0x7f0a44004060 2023-11-08T19:28:25.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:25.931+0000 7f0a5d3c8700 1 --2- 172.21.15.143:0/2166067460 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0a5806c730 0x7f0a44003a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:28:25.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:25.931+0000 7f0a5cbc7700 1 --2- 172.21.15.143:0/2166067460 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0a44004060 0x7f0a44004850 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f0a5400c210 tx=0x7f0a44005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=290beb7ff6f54067 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:25.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:25.932+0000 7f0a4ffff700 1 -- 172.21.15.143:0/2166067460 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a54007c10 con 0x7f0a44004060 2023-11-08T19:28:25.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:25.932+0000 7f0a4ffff700 1 -- 172.21.15.143:0/2166067460 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a5400cdd0 con 0x7f0a44004060 2023-11-08T19:28:25.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:25.932+0000 7f0a4ffff700 1 -- 172.21.15.143:0/2166067460 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a54005ba0 con 0x7f0a44004060 2023-11-08T19:28:25.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:25.932+0000 7f0a5f62c700 1 -- 172.21.15.143:0/2166067460 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0a44004060 msgr2=0x7f0a44004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:25.944 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:25.932+0000 7f0a5f62c700 1 --2- 172.21.15.143:0/2166067460 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0a44004060 0x7f0a44004850 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f0a5400c210 tx=0x7f0a44005880 comp rx=0 tx=0).stop 2023-11-08T19:28:25.944 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:25.932+0000 7f0a5dbc9700 1 -- 172.21.15.143:0/2166067460 reap_dead start 2023-11-08T19:28:25.944 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:25.932+0000 7f0a5f62c700 1 -- 172.21.15.143:0/2166067460 shutdown_connections 2023-11-08T19:28:25.944 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:25.932+0000 7f0a5f62c700 1 -- 172.21.15.143:0/2166067460 >> 172.21.15.143:0/2166067460 conn(0x7f0a5808b000 msgr2=0x7f0a5809ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:25.944 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:25.932+0000 7f0a5f62c700 1 -- 172.21.15.143:0/2166067460 shutdown_connections 2023-11-08T19:28:25.945 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:25.932+0000 7f0a5f62c700 1 -- 172.21.15.143:0/2166067460 wait complete. 2023-11-08T19:28:25.945 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:25.933+0000 7f0a5f62c700 1 Processor -- start 2023-11-08T19:28:25.945 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:25.934+0000 7f0a5f62c700 1 -- start start 2023-11-08T19:28:25.948 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:25.934+0000 7f0a5f62c700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0a44004060 0x7f0a58156fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:25.948 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:25.934+0000 7f0a5f62c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0a580b44d0 0x7f0a580b4950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:25.948 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:25.934+0000 7f0a5f62c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0a580b57a0 0x7f0a580b5c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:25.948 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:25.934+0000 7f0a5f62c700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a580b6160 con 0x7f0a580b57a0 2023-11-08T19:28:25.949 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:25.934+0000 7f0a5f62c700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a58157500 con 0x7f0a580b44d0 2023-11-08T19:28:25.949 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:25.934+0000 7f0a5f62c700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a580b4f30 con 0x7f0a44004060 2023-11-08T19:28:25.949 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:25.934+0000 7f0a5dbc9700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0a580b57a0 0x7f0a580b5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:25.949 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:25.934+0000 7f0a5dbc9700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0a580b57a0 0x7f0a580b5c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55006/0 (socket says 172.21.15.143:55006) 2023-11-08T19:28:25.949 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:25.934+0000 7f0a5dbc9700 1 -- 172.21.15.143:0/3073054982 learned_addr learned my addr 172.21.15.143:0/3073054982 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:28:25.950 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:25.934+0000 7f0a5d3c8700 1 --2- 172.21.15.143:0/3073054982 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0a44004060 0x7f0a58156fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:25.950 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:25.935+0000 7f0a5cbc7700 1 --2- 172.21.15.143:0/3073054982 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0a580b44d0 0x7f0a580b4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:25.950 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:25.935+0000 7f0a5dbc9700 1 -- 172.21.15.143:0/3073054982 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0a44004060 msgr2=0x7f0a58156fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:25.950 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:25.935+0000 7f0a5dbc9700 1 --2- 172.21.15.143:0/3073054982 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0a44004060 0x7f0a58156fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:25.950 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:25.935+0000 7f0a5dbc9700 1 -- 172.21.15.143:0/3073054982 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0a580b44d0 msgr2=0x7f0a580b4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:25.951 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:25.935+0000 7f0a5dbc9700 1 --2- 172.21.15.143:0/3073054982 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0a580b44d0 0x7f0a580b4950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:25.951 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:25.935+0000 7f0a5dbc9700 1 -- 172.21.15.143:0/3073054982 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a580b51f0 con 0x7f0a580b57a0 2023-11-08T19:28:25.951 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:25.935+0000 7f0a5d3c8700 1 --2- 172.21.15.143:0/3073054982 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0a44004060 0x7f0a58156fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:28:25.951 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:25.935+0000 7f0a5dbc9700 1 --2- 172.21.15.143:0/3073054982 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0a580b57a0 0x7f0a580b5c20 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f0a58071bd0 tx=0x7f0a5000d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:25.951 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:25.935+0000 7f0a4dffb700 1 -- 172.21.15.143:0/3073054982 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a50003c10 con 0x7f0a580b57a0 2023-11-08T19:28:25.952 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:25.935+0000 7f0a4dffb700 1 -- 172.21.15.143:0/3073054982 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a50003d90 con 0x7f0a580b57a0 2023-11-08T19:28:25.952 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:25.935+0000 7f0a5f62c700 1 -- 172.21.15.143:0/3073054982 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a580b54d0 con 0x7f0a580b57a0 2023-11-08T19:28:25.952 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:25.935+0000 7f0a4dffb700 1 -- 172.21.15.143:0/3073054982 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a5000dbd0 con 0x7f0a580b57a0 2023-11-08T19:28:25.952 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:25.935+0000 7f0a5f62c700 1 -- 172.21.15.143:0/3073054982 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a581f4950 con 0x7f0a580b57a0 2023-11-08T19:28:25.952 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:25.936+0000 7f0a5f62c700 1 -- 172.21.15.143:0/3073054982 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a5804eee0 con 0x7f0a580b57a0 2023-11-08T19:28:25.952 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:25.936+0000 7f0a4dffb700 1 -- 172.21.15.143:0/3073054982 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0a5000dd30 con 0x7f0a580b57a0 2023-11-08T19:28:25.953 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:25.937+0000 7f0a4dffb700 1 --2- 172.21.15.143:0/3073054982 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f0a3c06af00 0x7f0a3c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:25.953 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:25.937+0000 7f0a4dffb700 1 -- 172.21.15.143:0/3073054982 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6772+0+0 (secure 0 0 0) 0x7f0a50089c60 con 0x7f0a580b57a0 2023-11-08T19:28:25.953 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:25.937+0000 7f0a5d3c8700 1 --2- 172.21.15.143:0/3073054982 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f0a3c06af00 0x7f0a3c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:25.953 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:25.937+0000 7f0a5d3c8700 1 --2- 172.21.15.143:0/3073054982 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f0a3c06af00 0x7f0a3c06d3c0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f0a48009970 tx=0x7f0a48008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:25.953 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:25.941+0000 7f0a4dffb700 1 -- 172.21.15.143:0/3073054982 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0a500548e0 con 0x7f0a580b57a0 2023-11-08T19:28:26.174 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:26.172+0000 7f0a5f62c700 1 -- 172.21.15.143:0/3073054982 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0a580ce490 con 0x7f0a580b57a0 2023-11-08T19:28:26.176 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:26.173+0000 7f0a4dffb700 1 -- 172.21.15.143:0/3073054982 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+4941 (secure 0 0 0) 0x7f0a50057f00 con 0x7f0a580b57a0 2023-11-08T19:28:26.176 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:28:26.177 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":55,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6906,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6834/3172664377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":3172664377},{"type":"v1","addr":"172.21.15.143:6834","nonce":3172664377}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47},{"gid":6909,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3953059900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":3953059900},{"type":"v1","addr":"172.21.15.143:6835","nonce":3953059900}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":46}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:03.092245+0000","modified":"2023-11-08T19:28:17.609590+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6921,"mds_1":6918},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6918":{"gid":6918,"name":"b","rank":1,"incarnation":54,"state":"up:active","state_seq":6,"addr":"172.21.15.174:6836/3601088618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3601088618},{"type":"v1","addr":"172.21.15.174:6836","nonce":3601088618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6921":{"gid":6921,"name":"d","rank":0,"incarnation":48,"state":"up:active","state_seq":3,"addr":"172.21.15.174:6837/2201447629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6835","nonce":2201447629},{"type":"v1","addr":"172.21.15.174:6837","nonce":2201447629}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:28:26.178 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:26.175+0000 7f0a5f62c700 1 -- 172.21.15.143:0/3073054982 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f0a3c06af00 msgr2=0x7f0a3c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:26.178 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:26.175+0000 7f0a5f62c700 1 --2- 172.21.15.143:0/3073054982 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f0a3c06af00 0x7f0a3c06d3c0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f0a48009970 tx=0x7f0a48008040 comp rx=0 tx=0).stop 2023-11-08T19:28:26.178 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:26.175+0000 7f0a5f62c700 1 -- 172.21.15.143:0/3073054982 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0a580b57a0 msgr2=0x7f0a580b5c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:26.179 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:26.175+0000 7f0a5f62c700 1 --2- 172.21.15.143:0/3073054982 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0a580b57a0 0x7f0a580b5c20 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f0a58071bd0 tx=0x7f0a5000d430 comp rx=0 tx=0).stop 2023-11-08T19:28:26.179 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:26.175+0000 7f0a5f62c700 1 -- 172.21.15.143:0/3073054982 shutdown_connections 2023-11-08T19:28:26.179 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:26.175+0000 7f0a5f62c700 1 --2- 172.21.15.143:0/3073054982 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f0a3c06af00 0x7f0a3c06d3c0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:26.179 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:26.175+0000 7f0a5f62c700 1 --2- 172.21.15.143:0/3073054982 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0a44004060 0x7f0a58156fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:26.179 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:26.175+0000 7f0a5f62c700 1 --2- 172.21.15.143:0/3073054982 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0a580b57a0 0x7f0a580b5c20 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:26.180 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:26.175+0000 7f0a5f62c700 1 --2- 172.21.15.143:0/3073054982 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0a580b44d0 0x7f0a580b4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:26.180 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:26.175+0000 7f0a5f62c700 1 -- 172.21.15.143:0/3073054982 >> 172.21.15.143:0/3073054982 conn(0x7f0a5808b000 msgr2=0x7f0a580d1870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:26.180 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:26.175+0000 7f0a5f62c700 1 -- 172.21.15.143:0/3073054982 shutdown_connections 2023-11-08T19:28:26.180 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:26.175+0000 7f0a5f62c700 1 -- 172.21.15.143:0/3073054982 wait complete. 2023-11-08T19:28:26.180 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 55 2023-11-08T19:28:26.189 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json get subtrees 2023-11-08T19:28:26.339 INFO:teuthology.orchestra.run.smithi174.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2023-11-08T19:28:26.340 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2023-11-08T19:28:26.341 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1', 1)] 2023-11-08T19:28:27.130 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T19:28:27.133 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T19:28:28.342 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:28:28.475 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:28.473+0000 7f8ff6135700 1 Processor -- start 2023-11-08T19:28:28.475 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:28.474+0000 7f8ff6135700 1 -- start start 2023-11-08T19:28:28.478 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:28.474+0000 7f8ff6135700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f8ff015a2d0 0x7f8ff006ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:28.478 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:28.474+0000 7f8ff6135700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8ff00ce500 0x7f8ff00ce9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:28.478 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:28.474+0000 7f8ff6135700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ff00cb5e0 con 0x7f8ff006c200 2023-11-08T19:28:28.478 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:28.474+0000 7f8ff6135700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ff00ceef0 con 0x7f8ff015a2d0 2023-11-08T19:28:28.478 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:28.474+0000 7f8ff6135700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff00cb8c0 con 0x7f8ff00ce500 2023-11-08T19:28:28.479 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:28.475+0000 7f8feeffd700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f8ff006c200 0x7f8ff00cab70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:51164/0 (socket says 172.21.15.143:51164) 2023-11-08T19:28:28.479 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:28.475+0000 7f8feeffd700 1 -- 172.21.15.143:0/3938338863 learned_addr learned my addr 172.21.15.143:0/3938338863 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:28:28.479 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:28.475+0000 7f8fef7fe700 1 --2- 172.21.15.143:0/3938338863 >> v2:172.21.15.174:3300/0 conn(0x7f8ff015a2d0 0x7f8ff006ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:28.479 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:28.475+0000 7f8feffff700 1 --2- 172.21.15.143:0/3938338863 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8ff00ce500 0x7f8ff00ce9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:28.479 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:28.475+0000 7f8fee7fc700 1 -- 172.21.15.143:0/3938338863 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1624298824 0 0) 0x7f8ff00cb5e0 con 0x7f8ff006c200 2023-11-08T19:28:28.480 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:28.475+0000 7f8fee7fc700 1 -- 172.21.15.143:0/3938338863 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8fdc0035b0 con 0x7f8ff006c200 2023-11-08T19:28:28.480 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:28.475+0000 7f8fef7fe700 1 -- 172.21.15.143:0/3938338863 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8ff00ce500 msgr2=0x7f8ff00ce9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:28.480 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:28.475+0000 7f8fef7fe700 1 --2- 172.21.15.143:0/3938338863 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8ff00ce500 0x7f8ff00ce9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:28.480 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:28.475+0000 7f8fef7fe700 1 -- 172.21.15.143:0/3938338863 >> v1:172.21.15.143:6789/0 conn(0x7f8ff006c200 legacy=0x7f8ff00cab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:28.480 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:28.475+0000 7f8fef7fe700 1 -- 172.21.15.143:0/3938338863 --> v2:172.21.15.174:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ff00cbba0 con 0x7f8ff015a2d0 2023-11-08T19:28:28.481 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:28.475+0000 7f8feffff700 1 --2- 172.21.15.143:0/3938338863 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8ff00ce500 0x7f8ff00ce9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T19:28:28.481 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:28.475+0000 7f8fef7fe700 1 --2- 172.21.15.143:0/3938338863 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8ff015a2d0 0x7f8ff006ac10 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f8fe0009840 tx=0x7f8fe0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d03fc4b8d4241cc server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:28.481 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:28.475+0000 7f8fee7fc700 1 -- 172.21.15.143:0/3938338863 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe0011830 con 0x7f8ff015a2d0 2023-11-08T19:28:28.481 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:28.475+0000 7f8fee7fc700 1 -- 172.21.15.143:0/3938338863 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fe0011990 con 0x7f8ff015a2d0 2023-11-08T19:28:28.481 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:28.475+0000 7f8fee7fc700 1 -- 172.21.15.143:0/3938338863 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe0011d00 con 0x7f8ff015a2d0 2023-11-08T19:28:28.482 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:28.476+0000 7f8ff6135700 1 -- 172.21.15.143:0/3938338863 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8ff015a2d0 msgr2=0x7f8ff006ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:28.482 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:28.476+0000 7f8ff6135700 1 --2- 172.21.15.143:0/3938338863 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8ff015a2d0 0x7f8ff006ac10 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f8fe0009840 tx=0x7f8fe0009b50 comp rx=0 tx=0).stop 2023-11-08T19:28:28.482 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:28.476+0000 7f8ff6135700 1 -- 172.21.15.143:0/3938338863 shutdown_connections 2023-11-08T19:28:28.482 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:28.476+0000 7f8ff6135700 1 --2- 172.21.15.143:0/3938338863 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8ff00ce500 0x7f8ff00ce9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:28.482 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:28.476+0000 7f8ff6135700 1 --2- 172.21.15.143:0/3938338863 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8ff015a2d0 0x7f8ff006ac10 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:28.483 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:28.476+0000 7f8ff6135700 1 -- 172.21.15.143:0/3938338863 >> 172.21.15.143:0/3938338863 conn(0x7f8ff00b1a20 msgr2=0x7f8ff00b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:28.483 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:28.476+0000 7f8ff6135700 1 -- 172.21.15.143:0/3938338863 shutdown_connections 2023-11-08T19:28:28.483 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:28.476+0000 7f8ff6135700 1 -- 172.21.15.143:0/3938338863 wait complete. 2023-11-08T19:28:28.486 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:28.477+0000 7f8ff6135700 1 Processor -- start 2023-11-08T19:28:28.486 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:28.477+0000 7f8ff6135700 1 -- start start 2023-11-08T19:28:28.486 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:28.478+0000 7f8ff6135700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8ff006c200 0x7f8ff01dfec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:28.486 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:28.478+0000 7f8ff6135700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8ff00ce500 0x7f8ff01582c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:28.487 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:28.478+0000 7f8ff6135700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8ff015a2d0 0x7f8ff0158800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:28.487 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:28.478+0000 7f8ff6135700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff01f5a80 con 0x7f8ff015a2d0 2023-11-08T19:28:28.487 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:28.478+0000 7f8ff6135700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff01f4a90 con 0x7f8ff00ce500 2023-11-08T19:28:28.487 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:28.478+0000 7f8ff6135700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff01f4ce0 con 0x7f8ff006c200 2023-11-08T19:28:28.487 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:28.478+0000 7f8feffff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8ff015a2d0 0x7f8ff0158800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:28.488 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:28.478+0000 7f8feffff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8ff015a2d0 0x7f8ff0158800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55020/0 (socket says 172.21.15.143:55020) 2023-11-08T19:28:28.488 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:28.478+0000 7f8feffff700 1 -- 172.21.15.143:0/410806766 learned_addr learned my addr 172.21.15.143:0/410806766 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:28:28.488 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:28.479+0000 7f8feeffd700 1 --2- 172.21.15.143:0/410806766 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8ff00ce500 0x7f8ff01582c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:28.488 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:28.479+0000 7f8feffff700 1 -- 172.21.15.143:0/410806766 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8ff006c200 msgr2=0x7f8ff01dfec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:28.488 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:28.479+0000 7f8feffff700 1 --2- 172.21.15.143:0/410806766 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8ff006c200 0x7f8ff01dfec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:28.489 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:28.479+0000 7f8feffff700 1 -- 172.21.15.143:0/410806766 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8ff00ce500 msgr2=0x7f8ff01582c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:28.489 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:28.479+0000 7f8feffff700 1 --2- 172.21.15.143:0/410806766 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8ff00ce500 0x7f8ff01582c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:28.489 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:28.479+0000 7f8feffff700 1 -- 172.21.15.143:0/410806766 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fe8009a70 con 0x7f8ff015a2d0 2023-11-08T19:28:28.489 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:28.479+0000 7f8feeffd700 1 --2- 172.21.15.143:0/410806766 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8ff00ce500 0x7f8ff01582c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:28:28.490 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:28.479+0000 7f8feffff700 1 --2- 172.21.15.143:0/410806766 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8ff015a2d0 0x7f8ff0158800 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f8fe800efc0 tx=0x7f8fe800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:28.490 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:28.479+0000 7f8fe7fff700 1 -- 172.21.15.143:0/410806766 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe8009d70 con 0x7f8ff015a2d0 2023-11-08T19:28:28.490 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:28.479+0000 7f8fe7fff700 1 -- 172.21.15.143:0/410806766 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fe800f040 con 0x7f8ff015a2d0 2023-11-08T19:28:28.490 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:28.479+0000 7f8ff6135700 1 -- 172.21.15.143:0/410806766 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fe00094a0 con 0x7f8ff015a2d0 2023-11-08T19:28:28.490 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:28.479+0000 7f8fe7fff700 1 -- 172.21.15.143:0/410806766 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe800ce00 con 0x7f8ff015a2d0 2023-11-08T19:28:28.491 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:28.479+0000 7f8ff6135700 1 -- 172.21.15.143:0/410806766 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ff01f52a0 con 0x7f8ff015a2d0 2023-11-08T19:28:28.491 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:28.480+0000 7f8ff6135700 1 -- 172.21.15.143:0/410806766 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ff004eee0 con 0x7f8ff015a2d0 2023-11-08T19:28:28.491 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:28.480+0000 7f8fe7fff700 1 -- 172.21.15.143:0/410806766 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8fe8014480 con 0x7f8ff015a2d0 2023-11-08T19:28:28.491 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:28.481+0000 7f8fe7fff700 1 --2- 172.21.15.143:0/410806766 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f8fd006af00 0x7f8fd006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:28.491 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:28.481+0000 7f8fe7fff700 1 -- 172.21.15.143:0/410806766 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6772+0+0 (secure 0 0 0) 0x7f8fe808a170 con 0x7f8ff015a2d0 2023-11-08T19:28:28.492 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:28.481+0000 7f8fef7fe700 1 --2- 172.21.15.143:0/410806766 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f8fd006af00 0x7f8fd006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:28.492 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:28.481+0000 7f8fef7fe700 1 --2- 172.21.15.143:0/410806766 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f8fd006af00 0x7f8fd006d3c0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f8ff0158ff0 tx=0x7f8fe0010000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:28.493 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:28.485+0000 7f8fe7fff700 1 -- 172.21.15.143:0/410806766 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8fe8054df0 con 0x7f8ff015a2d0 2023-11-08T19:28:28.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:28.719+0000 7f8ff6135700 1 -- 172.21.15.143:0/410806766 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8ff0072600 con 0x7f8ff015a2d0 2023-11-08T19:28:28.721 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:28.719+0000 7f8fe7fff700 1 -- 172.21.15.143:0/410806766 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+4941 (secure 0 0 0) 0x7f8fe8058410 con 0x7f8ff015a2d0 2023-11-08T19:28:28.721 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:28:28.721 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":55,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6906,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6834/3172664377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":3172664377},{"type":"v1","addr":"172.21.15.143:6834","nonce":3172664377}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47},{"gid":6909,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3953059900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":3953059900},{"type":"v1","addr":"172.21.15.143:6835","nonce":3953059900}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":46}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:03.092245+0000","modified":"2023-11-08T19:28:17.609590+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6921,"mds_1":6918},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6918":{"gid":6918,"name":"b","rank":1,"incarnation":54,"state":"up:active","state_seq":6,"addr":"172.21.15.174:6836/3601088618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3601088618},{"type":"v1","addr":"172.21.15.174:6836","nonce":3601088618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6921":{"gid":6921,"name":"d","rank":0,"incarnation":48,"state":"up:active","state_seq":3,"addr":"172.21.15.174:6837/2201447629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6835","nonce":2201447629},{"type":"v1","addr":"172.21.15.174:6837","nonce":2201447629}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:28:28.723 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:28.721+0000 7f8ff6135700 1 -- 172.21.15.143:0/410806766 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f8fd006af00 msgr2=0x7f8fd006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:28.723 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:28.721+0000 7f8ff6135700 1 --2- 172.21.15.143:0/410806766 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f8fd006af00 0x7f8fd006d3c0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f8ff0158ff0 tx=0x7f8fe0010000 comp rx=0 tx=0).stop 2023-11-08T19:28:28.723 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:28.721+0000 7f8ff6135700 1 -- 172.21.15.143:0/410806766 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8ff015a2d0 msgr2=0x7f8ff0158800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:28.724 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:28.721+0000 7f8ff6135700 1 --2- 172.21.15.143:0/410806766 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8ff015a2d0 0x7f8ff0158800 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f8fe800efc0 tx=0x7f8fe800c3a0 comp rx=0 tx=0).stop 2023-11-08T19:28:28.724 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:28.721+0000 7f8ff6135700 1 -- 172.21.15.143:0/410806766 shutdown_connections 2023-11-08T19:28:28.724 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:28.722+0000 7f8ff6135700 1 --2- 172.21.15.143:0/410806766 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f8fd006af00 0x7f8fd006d3c0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:28.724 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:28.722+0000 7f8ff6135700 1 --2- 172.21.15.143:0/410806766 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8ff006c200 0x7f8ff01dfec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:28.724 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:28.722+0000 7f8ff6135700 1 --2- 172.21.15.143:0/410806766 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8ff015a2d0 0x7f8ff0158800 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:28.725 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:28.722+0000 7f8ff6135700 1 --2- 172.21.15.143:0/410806766 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8ff00ce500 0x7f8ff01582c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:28.725 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:28.722+0000 7f8ff6135700 1 -- 172.21.15.143:0/410806766 >> 172.21.15.143:0/410806766 conn(0x7f8ff00b1a20 msgr2=0x7f8ff00cd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:28.725 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:28.722+0000 7f8ff6135700 1 -- 172.21.15.143:0/410806766 shutdown_connections 2023-11-08T19:28:28.725 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:28.722+0000 7f8ff6135700 1 -- 172.21.15.143:0/410806766 wait complete. 2023-11-08T19:28:28.726 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 55 2023-11-08T19:28:28.735 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json get subtrees 2023-11-08T19:28:28.891 INFO:teuthology.orchestra.run.smithi174.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"8","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1","dirfrag":"0x10000000000","snapid_first":2,"projected_version":"3","version":"3","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}}] 2023-11-08T19:28:28.892 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "8" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000000", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "3", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "3" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false } ] 2023-11-08T19:28:28.893 INFO:tasks.cephfs.cephfs_test_case:[('/d1', 1)] =?= [('/d1', 1)] 2023-11-08T19:28:28.893 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json dump snaps --server 2023-11-08T19:28:29.029 INFO:teuthology.orchestra.run.smithi174.stdout:{"last_snap":1,"last_created":1,"last_destroyed":1,"pending_noop":[],"snaps":[],"need_to_purge":{},"pending_update":[],"pending_destroy":[]} 2023-11-08T19:28:29.029 DEBUG:tasks.cephfs.filesystem:_json_asok output { "last_created": 1, "last_destroyed": 1, "last_snap": 1, "need_to_purge": {}, "pending_destroy": [], "pending_noop": [], "pending_update": [], "snaps": [] } 2023-11-08T19:28:29.030 INFO:tasks.cephfs.test_snapshots:testing snapserver mds_kill_mdstable_at=1 2023-11-08T19:28:29.030 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:28:29.151 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.150+0000 7fc220096700 1 Processor -- start 2023-11-08T19:28:29.152 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.150+0000 7fc220096700 1 -- start start 2023-11-08T19:28:29.153 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.151+0000 7fc220096700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fc218159870 0x7fc2180b2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:29.154 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.151+0000 7fc220096700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2180b2a60 con 0x7fc2180cc410 2023-11-08T19:28:29.154 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.151+0000 7fc220096700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2180b2d40 con 0x7fc21806ed60 2023-11-08T19:28:29.154 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.151+0000 7fc220096700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc21806b090 con 0x7fc218159870 2023-11-08T19:28:29.154 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.151+0000 7fc21e633700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fc2180cc410 0x7fc2180cc820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:51184/0 (socket says 172.21.15.143:51184) 2023-11-08T19:28:29.155 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.151+0000 7fc21e633700 1 -- 172.21.15.143:0/869346452 learned_addr learned my addr 172.21.15.143:0/869346452 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:28:29.155 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.151+0000 7fc21de32700 1 --2- 172.21.15.143:0/869346452 >> v2:172.21.15.174:3300/0 conn(0x7fc218159870 0x7fc2180b2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:29.155 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.151+0000 7fc21ce30700 1 -- 172.21.15.143:0/869346452 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4089550238 0 0) 0x7fc2180b2a60 con 0x7fc2180cc410 2023-11-08T19:28:29.155 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.151+0000 7fc21ce30700 1 -- 172.21.15.143:0/869346452 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc2040035b0 con 0x7fc2180cc410 2023-11-08T19:28:29.155 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.151+0000 7fc21de32700 1 -- 172.21.15.143:0/869346452 >> v1:172.21.15.174:6789/0 conn(0x7fc21806ed60 legacy=0x7fc21806b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:29.156 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.151+0000 7fc21de32700 1 -- 172.21.15.143:0/869346452 >> v1:172.21.15.143:6789/0 conn(0x7fc2180cc410 legacy=0x7fc2180cc820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:29.156 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.151+0000 7fc21ce30700 1 -- 172.21.15.143:0/869346452 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3096953619 0 0) 0x7fc2040035b0 con 0x7fc2180cc410 2023-11-08T19:28:29.156 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.151+0000 7fc21de32700 1 -- 172.21.15.143:0/869346452 --> v2:172.21.15.174:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2180b3020 con 0x7fc218159870 2023-11-08T19:28:29.156 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.152+0000 7fc21de32700 1 --2- 172.21.15.143:0/869346452 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc218159870 0x7fc2180b2210 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fc214009820 tx=0x7fc214009b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=22ff5113444acfb0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:29.156 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.152+0000 7fc21ce30700 1 -- 172.21.15.143:0/869346452 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2100019f0 con 0x7fc218159870 2023-11-08T19:28:29.156 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.152+0000 7fc21ce30700 1 -- 172.21.15.143:0/869346452 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc214005c30 con 0x7fc218159870 2023-11-08T19:28:29.157 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.152+0000 7fc21ce30700 1 -- 172.21.15.143:0/869346452 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc214004030 con 0x7fc218159870 2023-11-08T19:28:29.157 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.152+0000 7fc220096700 1 -- 172.21.15.143:0/869346452 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc218159870 msgr2=0x7fc2180b2210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:29.157 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.152+0000 7fc220096700 1 --2- 172.21.15.143:0/869346452 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc218159870 0x7fc2180b2210 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fc214009820 tx=0x7fc214009b30 comp rx=0 tx=0).stop 2023-11-08T19:28:29.157 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.152+0000 7fc220096700 1 -- 172.21.15.143:0/869346452 shutdown_connections 2023-11-08T19:28:29.157 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.152+0000 7fc220096700 1 --2- 172.21.15.143:0/869346452 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc218159870 0x7fc2180b2210 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:29.158 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.152+0000 7fc220096700 1 -- 172.21.15.143:0/869346452 >> 172.21.15.143:0/869346452 conn(0x7fc218099ab0 msgr2=0x7fc2180d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:29.158 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.152+0000 7fc220096700 1 -- 172.21.15.143:0/869346452 shutdown_connections 2023-11-08T19:28:29.158 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.152+0000 7fc220096700 1 -- 172.21.15.143:0/869346452 wait complete. 2023-11-08T19:28:29.161 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.153+0000 7fc220096700 1 Processor -- start 2023-11-08T19:28:29.161 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.154+0000 7fc220096700 1 -- start start 2023-11-08T19:28:29.161 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.154+0000 7fc220096700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc21806ed60 0x7fc2181f3400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:29.161 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.154+0000 7fc220096700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc2180cc410 0x7fc2180c9780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:29.162 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.154+0000 7fc220096700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc218159870 0x7fc2181f45c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:29.162 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.154+0000 7fc220096700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2180aea20 con 0x7fc218159870 2023-11-08T19:28:29.162 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.154+0000 7fc220096700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2180ad420 con 0x7fc21806ed60 2023-11-08T19:28:29.162 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.154+0000 7fc220096700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2181f4b00 con 0x7fc2180cc410 2023-11-08T19:28:29.163 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.154+0000 7fc21e633700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc218159870 0x7fc2181f45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:29.163 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.154+0000 7fc21e633700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc218159870 0x7fc2181f45c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55036/0 (socket says 172.21.15.143:55036) 2023-11-08T19:28:29.163 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.154+0000 7fc21e633700 1 -- 172.21.15.143:0/872985969 learned_addr learned my addr 172.21.15.143:0/872985969 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:28:29.163 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.155+0000 7fc21de32700 1 --2- 172.21.15.143:0/872985969 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc21806ed60 0x7fc2181f3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:29.163 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.155+0000 7fc21d631700 1 --2- 172.21.15.143:0/872985969 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc2180cc410 0x7fc2180c9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:29.164 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.155+0000 7fc21e633700 1 -- 172.21.15.143:0/872985969 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc2180cc410 msgr2=0x7fc2180c9780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:29.164 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.155+0000 7fc21e633700 1 --2- 172.21.15.143:0/872985969 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc2180cc410 0x7fc2180c9780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:29.164 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.155+0000 7fc21e633700 1 -- 172.21.15.143:0/872985969 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc21806ed60 msgr2=0x7fc2181f3400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:29.164 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.155+0000 7fc21e633700 1 --2- 172.21.15.143:0/872985969 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc21806ed60 0x7fc2181f3400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:29.164 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.155+0000 7fc21e633700 1 -- 172.21.15.143:0/872985969 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc214009430 con 0x7fc218159870 2023-11-08T19:28:29.165 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.155+0000 7fc21de32700 1 --2- 172.21.15.143:0/872985969 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc21806ed60 0x7fc2181f3400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:28:29.165 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.155+0000 7fc21e633700 1 --2- 172.21.15.143:0/872985969 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc218159870 0x7fc2181f45c0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fc21000cc70 tx=0x7fc21000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:29.165 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.155+0000 7fc20e7fc700 1 -- 172.21.15.143:0/872985969 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc21000ada0 con 0x7fc218159870 2023-11-08T19:28:29.165 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.155+0000 7fc20e7fc700 1 -- 172.21.15.143:0/872985969 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc21000ce70 con 0x7fc218159870 2023-11-08T19:28:29.165 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.155+0000 7fc220096700 1 -- 172.21.15.143:0/872985969 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2181f3a50 con 0x7fc218159870 2023-11-08T19:28:29.165 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.155+0000 7fc20e7fc700 1 -- 172.21.15.143:0/872985969 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2100072f0 con 0x7fc218159870 2023-11-08T19:28:29.166 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.155+0000 7fc220096700 1 -- 172.21.15.143:0/872985969 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2181f4010 con 0x7fc218159870 2023-11-08T19:28:29.166 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.156+0000 7fc220096700 1 -- 172.21.15.143:0/872985969 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2180ad5f0 con 0x7fc218159870 2023-11-08T19:28:29.166 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.156+0000 7fc20e7fc700 1 -- 172.21.15.143:0/872985969 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc210007450 con 0x7fc218159870 2023-11-08T19:28:29.166 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.157+0000 7fc20e7fc700 1 --2- 172.21.15.143:0/872985969 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc1fc06af00 0x7fc1fc06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:29.166 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.157+0000 7fc20e7fc700 1 -- 172.21.15.143:0/872985969 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6772+0+0 (secure 0 0 0) 0x7fc210089920 con 0x7fc218159870 2023-11-08T19:28:29.167 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.157+0000 7fc21de32700 1 --2- 172.21.15.143:0/872985969 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc1fc06af00 0x7fc1fc06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:29.167 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.157+0000 7fc21de32700 1 --2- 172.21.15.143:0/872985969 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc1fc06af00 0x7fc1fc06d3c0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fc214009fd0 tx=0x7fc214008070 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:29.167 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.161+0000 7fc20e7fc700 1 -- 172.21.15.143:0/872985969 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc2180ad5f0 con 0x7fc218159870 2023-11-08T19:28:29.395 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.393+0000 7fc220096700 1 -- 172.21.15.143:0/872985969 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc2180ad590 con 0x7fc218159870 2023-11-08T19:28:29.396 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.394+0000 7fc20e7fc700 1 -- 172.21.15.143:0/872985969 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+4942 (secure 0 0 0) 0x7fc210018090 con 0x7fc218159870 2023-11-08T19:28:29.396 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:28:29.396 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":56,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6906,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6834/3172664377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":3172664377},{"type":"v1","addr":"172.21.15.143:6834","nonce":3172664377}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47},{"gid":6909,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3953059900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":3953059900},{"type":"v1","addr":"172.21.15.143:6835","nonce":3953059900}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":46}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:03.092245+0000","modified":"2023-11-08T19:28:27.806643+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6921,"mds_1":6918},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6918":{"gid":6918,"name":"b","rank":1,"incarnation":54,"state":"up:active","state_seq":6,"addr":"172.21.15.174:6836/3601088618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3601088618},{"type":"v1","addr":"172.21.15.174:6836","nonce":3601088618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6921":{"gid":6921,"name":"d","rank":0,"incarnation":48,"state":"up:active","state_seq":3,"addr":"172.21.15.174:6837/2201447629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6835","nonce":2201447629},{"type":"v1","addr":"172.21.15.174:6837","nonce":2201447629}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:28:29.399 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.396+0000 7fc220096700 1 -- 172.21.15.143:0/872985969 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc1fc06af00 msgr2=0x7fc1fc06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:29.399 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.396+0000 7fc220096700 1 --2- 172.21.15.143:0/872985969 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc1fc06af00 0x7fc1fc06d3c0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fc214009fd0 tx=0x7fc214008070 comp rx=0 tx=0).stop 2023-11-08T19:28:29.399 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.396+0000 7fc220096700 1 -- 172.21.15.143:0/872985969 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc218159870 msgr2=0x7fc2181f45c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:29.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.396+0000 7fc220096700 1 --2- 172.21.15.143:0/872985969 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc218159870 0x7fc2181f45c0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fc21000cc70 tx=0x7fc21000a340 comp rx=0 tx=0).stop 2023-11-08T19:28:29.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.396+0000 7fc220096700 1 -- 172.21.15.143:0/872985969 shutdown_connections 2023-11-08T19:28:29.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.396+0000 7fc220096700 1 --2- 172.21.15.143:0/872985969 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc1fc06af00 0x7fc1fc06d3c0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:29.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.396+0000 7fc220096700 1 --2- 172.21.15.143:0/872985969 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc218159870 0x7fc2181f45c0 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:29.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.396+0000 7fc220096700 1 --2- 172.21.15.143:0/872985969 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc21806ed60 0x7fc2181f3400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:29.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.396+0000 7fc220096700 1 --2- 172.21.15.143:0/872985969 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc2180cc410 0x7fc2180c9780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:29.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.396+0000 7fc220096700 1 -- 172.21.15.143:0/872985969 >> 172.21.15.143:0/872985969 conn(0x7fc218099ab0 msgr2=0x7fc2180ce310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:29.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.397+0000 7fc220096700 1 -- 172.21.15.143:0/872985969 shutdown_connections 2023-11-08T19:28:29.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.397+0000 7fc220096700 1 -- 172.21.15.143:0/872985969 wait complete. 2023-11-08T19:28:29.404 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 56 2023-11-08T19:28:29.410 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds freeze 6:0 true 2023-11-08T19:28:29.533 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.531+0000 7f17b84f3700 1 Processor -- start 2023-11-08T19:28:29.533 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.531+0000 7f17b84f3700 1 -- start start 2023-11-08T19:28:29.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.532+0000 7f17b84f3700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f17b006b240 0x7f17b00caff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:29.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.532+0000 7f17b84f3700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17b0161890 con 0x7f17b00cb530 2023-11-08T19:28:29.540 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.532+0000 7f17b84f3700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17b006b740 con 0x7f17b015dba0 2023-11-08T19:28:29.540 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.532+0000 7f17b84f3700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17b006cd20 con 0x7f17b006b240 2023-11-08T19:28:29.540 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.532+0000 7f17b6a90700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f17b00cb530 0x7f17b00b22e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:51198/0 (socket says 172.21.15.143:51198) 2023-11-08T19:28:29.540 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.532+0000 7f17b6a90700 1 -- 172.21.15.143:0/3941068892 learned_addr learned my addr 172.21.15.143:0/3941068892 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:28:29.540 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.533+0000 7f17b5a8e700 1 --2- 172.21.15.143:0/3941068892 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f17b006b240 0x7f17b00caff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:29.541 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.533+0000 7f17b528d700 1 -- 172.21.15.143:0/3941068892 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1950597854 0 0) 0x7f17b0161890 con 0x7f17b00cb530 2023-11-08T19:28:29.541 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.533+0000 7f17b528d700 1 -- 172.21.15.143:0/3941068892 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f179c0035b0 con 0x7f17b00cb530 2023-11-08T19:28:29.541 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.533+0000 7f17b528d700 1 -- 172.21.15.143:0/3941068892 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3209681226 0 0) 0x7f179c0035b0 con 0x7f17b00cb530 2023-11-08T19:28:29.541 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.533+0000 7f17b528d700 1 -- 172.21.15.143:0/3941068892 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f17b006b240 msgr2=0x7f17b00caff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:29.541 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.533+0000 7f17b528d700 1 --2- 172.21.15.143:0/3941068892 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f17b006b240 0x7f17b00caff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:29.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.533+0000 7f17b528d700 1 -- 172.21.15.143:0/3941068892 >> v1:172.21.15.174:6789/0 conn(0x7f17b015dba0 legacy=0x7f17b006c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:29.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.533+0000 7f17b528d700 1 -- 172.21.15.143:0/3941068892 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17b006ba20 con 0x7f17b00cb530 2023-11-08T19:28:29.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.533+0000 7f17b528d700 1 -- 172.21.15.143:0/3941068892 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4147756303 0 0) 0x7f17a80019f0 con 0x7f17b00cb530 2023-11-08T19:28:29.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.533+0000 7f17b528d700 1 -- 172.21.15.143:0/3941068892 >> v1:172.21.15.143:6789/0 conn(0x7f17b00cb530 legacy=0x7f17b00b22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:29.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.533+0000 7f17b528d700 1 --2- 172.21.15.143:0/3941068892 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f179c003a30 0x7f179c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:29.543 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.533+0000 7f17b528d700 1 --2- 172.21.15.143:0/3941068892 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f179c0043e0 0x7f179c0048b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:29.543 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.533+0000 7f17b528d700 1 --2- 172.21.15.143:0/3941068892 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f179c0050b0 0x7f179c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:29.543 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.533+0000 7f17b528d700 1 -- 172.21.15.143:0/3941068892 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17b006cd20 con 0x7f179c0043e0 2023-11-08T19:28:29.543 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.533+0000 7f17b528d700 1 -- 172.21.15.143:0/3941068892 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f179c00bf80 con 0x7f179c003a30 2023-11-08T19:28:29.543 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.533+0000 7f17b528d700 1 -- 172.21.15.143:0/3941068892 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f179c00c140 con 0x7f179c0050b0 2023-11-08T19:28:29.544 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.533+0000 7f17b5a8e700 1 --2- 172.21.15.143:0/3941068892 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f17b006b240 0x7f17b00caff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T19:28:29.544 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.533+0000 7f17b628f700 1 --2- 172.21.15.143:0/3941068892 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f179c003a30 0x7f179c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:29.544 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.533+0000 7f17b6a90700 1 --2- 172.21.15.143:0/3941068892 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f179c0050b0 0x7f179c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:29.544 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.533+0000 7f17b5a8e700 1 --2- 172.21.15.143:0/3941068892 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f179c0043e0 0x7f179c0048b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:29.544 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.534+0000 7f17b5a8e700 1 -- 172.21.15.143:0/3941068892 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f179c0050b0 msgr2=0x7f179c00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:29.545 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.534+0000 7f17b5a8e700 1 --2- 172.21.15.143:0/3941068892 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f179c0050b0 0x7f179c00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:29.545 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.534+0000 7f17b5a8e700 1 -- 172.21.15.143:0/3941068892 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f179c003a30 msgr2=0x7f179c003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:29.545 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.534+0000 7f17b5a8e700 1 --2- 172.21.15.143:0/3941068892 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f179c003a30 0x7f179c003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:29.545 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.534+0000 7f17b5a8e700 1 -- 172.21.15.143:0/3941068892 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f179c0035b0 con 0x7f179c0043e0 2023-11-08T19:28:29.545 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.534+0000 7f17b5a8e700 1 --2- 172.21.15.143:0/3941068892 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f179c0043e0 0x7f179c0048b0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f17b00ce3e0 tx=0x7f179c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3282995bee3b1fb4 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:29.546 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.534+0000 7f17b528d700 1 -- 172.21.15.143:0/3941068892 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17ac004030 con 0x7f179c0043e0 2023-11-08T19:28:29.546 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.534+0000 7f17b528d700 1 -- 172.21.15.143:0/3941068892 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17ac004190 con 0x7f179c0043e0 2023-11-08T19:28:29.546 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.534+0000 7f17b528d700 1 -- 172.21.15.143:0/3941068892 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17ac004540 con 0x7f179c0043e0 2023-11-08T19:28:29.546 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.534+0000 7f17b84f3700 1 -- 172.21.15.143:0/3941068892 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f179c0043e0 msgr2=0x7f179c0048b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:29.546 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.534+0000 7f17b84f3700 1 --2- 172.21.15.143:0/3941068892 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f179c0043e0 0x7f179c0048b0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f17b00ce3e0 tx=0x7f179c005880 comp rx=0 tx=0).stop 2023-11-08T19:28:29.547 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.534+0000 7f17b6a90700 1 -- 172.21.15.143:0/3941068892 reap_dead start 2023-11-08T19:28:29.547 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.534+0000 7f17b84f3700 1 -- 172.21.15.143:0/3941068892 shutdown_connections 2023-11-08T19:28:29.547 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.534+0000 7f17b84f3700 1 -- 172.21.15.143:0/3941068892 >> 172.21.15.143:0/3941068892 conn(0x7f17b00cd2e0 msgr2=0x7f17b00cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:29.547 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.534+0000 7f17b84f3700 1 -- 172.21.15.143:0/3941068892 shutdown_connections 2023-11-08T19:28:29.547 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.534+0000 7f17b84f3700 1 -- 172.21.15.143:0/3941068892 wait complete. 2023-11-08T19:28:29.548 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.536+0000 7f17b84f3700 1 Processor -- start 2023-11-08T19:28:29.548 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.536+0000 7f17b84f3700 1 -- start start 2023-11-08T19:28:29.548 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.536+0000 7f17b84f3700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f179c0043e0 0x7f17b00ca160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:29.548 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.536+0000 7f17b84f3700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f17b0161340 0x7f17b0160ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:29.548 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.536+0000 7f17b84f3700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f17b00af0c0 0x7f17b014c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:29.549 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.536+0000 7f17b84f3700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17b0160640 con 0x7f17b0161340 2023-11-08T19:28:29.549 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.536+0000 7f17b84f3700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17b015fd10 con 0x7f17b00af0c0 2023-11-08T19:28:29.549 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.536+0000 7f17b84f3700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17b015fff0 con 0x7f179c0043e0 2023-11-08T19:28:29.549 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.536+0000 7f17b5a8e700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f17b0161340 0x7f17b0160ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:29.549 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.537+0000 7f17b5a8e700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f17b0161340 0x7f17b0160ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55054/0 (socket says 172.21.15.143:55054) 2023-11-08T19:28:29.550 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.537+0000 7f17b5a8e700 1 -- 172.21.15.143:0/1429508231 learned_addr learned my addr 172.21.15.143:0/1429508231 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:28:29.550 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.537+0000 7f17b6a90700 1 --2- 172.21.15.143:0/1429508231 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f17b00af0c0 0x7f17b014c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:29.550 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.537+0000 7f17b628f700 1 --2- 172.21.15.143:0/1429508231 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f179c0043e0 0x7f17b00ca160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:29.550 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.537+0000 7f17b5a8e700 1 -- 172.21.15.143:0/1429508231 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f179c0043e0 msgr2=0x7f17b00ca160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:29.550 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.537+0000 7f17b5a8e700 1 --2- 172.21.15.143:0/1429508231 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f179c0043e0 0x7f17b00ca160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:29.551 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.537+0000 7f17b5a8e700 1 -- 172.21.15.143:0/1429508231 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f17b00af0c0 msgr2=0x7f17b014c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:29.551 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.537+0000 7f17b5a8e700 1 --2- 172.21.15.143:0/1429508231 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f17b00af0c0 0x7f17b014c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:29.551 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.537+0000 7f17b5a8e700 1 -- 172.21.15.143:0/1429508231 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17b01602d0 con 0x7f17b0161340 2023-11-08T19:28:29.551 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.537+0000 7f17b628f700 1 --2- 172.21.15.143:0/1429508231 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f179c0043e0 0x7f17b00ca160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:28:29.552 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.537+0000 7f17b5a8e700 1 --2- 172.21.15.143:0/1429508231 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f17b0161340 0x7f17b0160ad0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f179c00bf50 tx=0x7f17ac0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:29.552 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.537+0000 7f17a6ffd700 1 -- 172.21.15.143:0/1429508231 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17ac004030 con 0x7f17b0161340 2023-11-08T19:28:29.552 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.537+0000 7f17a6ffd700 1 -- 172.21.15.143:0/1429508231 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17ac0038a0 con 0x7f17b0161340 2023-11-08T19:28:29.552 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.537+0000 7f17b84f3700 1 -- 172.21.15.143:0/1429508231 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17b01617c0 con 0x7f17b0161340 2023-11-08T19:28:29.552 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.537+0000 7f17a6ffd700 1 -- 172.21.15.143:0/1429508231 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17ac003c30 con 0x7f17b0161340 2023-11-08T19:28:29.553 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.537+0000 7f17b84f3700 1 -- 172.21.15.143:0/1429508231 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17b00ca790 con 0x7f17b0161340 2023-11-08T19:28:29.553 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.538+0000 7f17b84f3700 1 -- 172.21.15.143:0/1429508231 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17b01607f0 con 0x7f17b0161340 2023-11-08T19:28:29.554 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.538+0000 7f17a6ffd700 1 -- 172.21.15.143:0/1429508231 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f17ac013470 con 0x7f17b0161340 2023-11-08T19:28:29.554 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.539+0000 7f17a6ffd700 1 --2- 172.21.15.143:0/1429508231 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f179406af00 0x7f179406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:29.554 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.539+0000 7f17a6ffd700 1 -- 172.21.15.143:0/1429508231 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6772+0+0 (secure 0 0 0) 0x7f17ac08b0e0 con 0x7f17b0161340 2023-11-08T19:28:29.554 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.539+0000 7f17b628f700 1 --2- 172.21.15.143:0/1429508231 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f179406af00 0x7f179406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:29.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.539+0000 7f17b628f700 1 --2- 172.21.15.143:0/1429508231 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f179406af00 0x7f179406d3c0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f17a0007fd0 tx=0x7f17a0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:29.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.543+0000 7f17a6ffd700 1 -- 172.21.15.143:0/1429508231 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f17b01607f0 con 0x7f17b0161340 2023-11-08T19:28:29.816 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:29.814+0000 7f17b84f3700 1 -- 172.21.15.143:0/1429508231 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "mds freeze", "role_or_gid": "6:0", "val": "true"} v 0) v1 -- 0x7f17b01607f0 con 0x7f17b0161340 2023-11-08T19:28:30.816 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:30.814+0000 7f17a6ffd700 1 -- 172.21.15.143:0/1429508231 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "mds freeze", "role_or_gid": "6:0", "val": "true"}]=0 freezing mds.6921 v57) v1 ==== 112+0+0 (secure 0 0 0) 0x7f17b01607f0 con 0x7f17b0161340 2023-11-08T19:28:30.820 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:30.818+0000 7f17b84f3700 1 -- 172.21.15.143:0/1429508231 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f179406af00 msgr2=0x7f179406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:30.820 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:30.818+0000 7f17b84f3700 1 --2- 172.21.15.143:0/1429508231 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f179406af00 0x7f179406d3c0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f17a0007fd0 tx=0x7f17a0009040 comp rx=0 tx=0).stop 2023-11-08T19:28:30.820 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:30.818+0000 7f17b84f3700 1 -- 172.21.15.143:0/1429508231 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f17b0161340 msgr2=0x7f17b0160ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:30.821 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:30.818+0000 7f17b84f3700 1 --2- 172.21.15.143:0/1429508231 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f17b0161340 0x7f17b0160ad0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f179c00bf50 tx=0x7f17ac0047c0 comp rx=0 tx=0).stop 2023-11-08T19:28:30.821 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:30.818+0000 7f17b84f3700 1 -- 172.21.15.143:0/1429508231 shutdown_connections 2023-11-08T19:28:30.821 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:30.818+0000 7f17b84f3700 1 --2- 172.21.15.143:0/1429508231 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f179406af00 0x7f179406d3c0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:30.821 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:30.818+0000 7f17b84f3700 1 --2- 172.21.15.143:0/1429508231 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f179c0043e0 0x7f17b00ca160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:30.821 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:30.818+0000 7f17b84f3700 1 --2- 172.21.15.143:0/1429508231 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f17b00af0c0 0x7f17b014c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:30.822 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:30.818+0000 7f17b84f3700 1 --2- 172.21.15.143:0/1429508231 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f17b0161340 0x7f17b0160ad0 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:30.822 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:30.818+0000 7f17b84f3700 1 -- 172.21.15.143:0/1429508231 >> 172.21.15.143:0/1429508231 conn(0x7f17b00cd2e0 msgr2=0x7f17b015ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:30.822 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:30.818+0000 7f17b84f3700 1 -- 172.21.15.143:0/1429508231 shutdown_connections 2023-11-08T19:28:30.822 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:30.818+0000 7f17b84f3700 1 -- 172.21.15.143:0/1429508231 wait complete. 2023-11-08T19:28:30.823 INFO:teuthology.orchestra.run.smithi143.stderr:freezing mds.6921 2023-11-08T19:28:30.836 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json config set mds_kill_mdstable_at 1 2023-11-08T19:28:30.984 INFO:teuthology.orchestra.run.smithi174.stdout:{"success":"mds_kill_mdstable_at = '1' (not observed, change may require restart) "} 2023-11-08T19:28:30.984 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_kill_mdstable_at = '1' (not observed, change may require restart) " } 2023-11-08T19:28:30.985 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:28:30.985 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir d1/dir/.snap/s11) 2023-11-08T19:28:30.988 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mds_beacon_grace 2023-11-08T19:28:31.007 INFO:tasks.ceph.mds.d.smithi174.stderr:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el8/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: In function 'void MDSTableServer::handle_prepare(ceph::cref_t&)' thread 7f73faa75700 time 2023-11-08T19:28:31.007575+0000 2023-11-08T19:28:31.008 INFO:tasks.ceph.mds.d.smithi174.stderr:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el8/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: 61: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 1) 2023-11-08T19:28:31.008 INFO:tasks.ceph.mds.d.smithi174.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-08T19:28:31.009 INFO:tasks.ceph.mds.d.smithi174.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f740633df5b] 2023-11-08T19:28:31.009 INFO:tasks.ceph.mds.d.smithi174.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2ba121) [0x7f740633e121] 2023-11-08T19:28:31.009 INFO:tasks.ceph.mds.d.smithi174.stderr: 3: (MDSTableServer::handle_prepare(boost::intrusive_ptr const&)+0x3e0) [0x559c474cc340] 2023-11-08T19:28:31.009 INFO:tasks.ceph.mds.d.smithi174.stderr: 4: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xe9) [0x559c4718d099] 2023-11-08T19:28:31.010 INFO:tasks.ceph.mds.d.smithi174.stderr: 5: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x593) [0x559c471901c3] 2023-11-08T19:28:31.010 INFO:tasks.ceph.mds.d.smithi174.stderr: 6: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x559c471907bc] 2023-11-08T19:28:31.010 INFO:tasks.ceph.mds.d.smithi174.stderr: 7: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x559c4717a56f] 2023-11-08T19:28:31.010 INFO:tasks.ceph.mds.d.smithi174.stderr: 8: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f74065c59c8] 2023-11-08T19:28:31.010 INFO:tasks.ceph.mds.d.smithi174.stderr: 9: (DispatchQueue::entry()+0x51f) [0x7f74065c2b4f] 2023-11-08T19:28:31.011 INFO:tasks.ceph.mds.d.smithi174.stderr: 10: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f7406690351] 2023-11-08T19:28:31.011 INFO:tasks.ceph.mds.d.smithi174.stderr: 11: /lib64/libpthread.so.0(+0x81cf) [0x7f74050d61cf] 2023-11-08T19:28:31.011 INFO:tasks.ceph.mds.d.smithi174.stderr: 12: clone() 2023-11-08T19:28:31.011 INFO:tasks.ceph.mds.d.smithi174.stderr:*** Caught signal (Aborted) ** 2023-11-08T19:28:31.011 INFO:tasks.ceph.mds.d.smithi174.stderr: in thread 7f73faa75700 thread_name:ms_dispatch 2023-11-08T19:28:31.012 INFO:tasks.ceph.mds.d.smithi174.stderr:2023-11-08T19:28:31.007+0000 7f73faa75700 -1 /home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el8/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: In function 'void MDSTableServer::handle_prepare(ceph::cref_t&)' thread 7f73faa75700 time 2023-11-08T19:28:31.007575+0000 2023-11-08T19:28:31.012 INFO:tasks.ceph.mds.d.smithi174.stderr:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el8/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: 61: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 1) 2023-11-08T19:28:31.012 INFO:tasks.ceph.mds.d.smithi174.stderr: 2023-11-08T19:28:31.012 INFO:tasks.ceph.mds.d.smithi174.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-08T19:28:31.012 INFO:tasks.ceph.mds.d.smithi174.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f740633df5b] 2023-11-08T19:28:31.012 INFO:tasks.ceph.mds.d.smithi174.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2ba121) [0x7f740633e121] 2023-11-08T19:28:31.013 INFO:tasks.ceph.mds.d.smithi174.stderr: 3: (MDSTableServer::handle_prepare(boost::intrusive_ptr const&)+0x3e0) [0x559c474cc340] 2023-11-08T19:28:31.013 INFO:tasks.ceph.mds.d.smithi174.stderr: 4: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xe9) [0x559c4718d099] 2023-11-08T19:28:31.013 INFO:tasks.ceph.mds.d.smithi174.stderr: 5: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x593) [0x559c471901c3] 2023-11-08T19:28:31.013 INFO:tasks.ceph.mds.d.smithi174.stderr: 6: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x559c471907bc] 2023-11-08T19:28:31.013 INFO:tasks.ceph.mds.d.smithi174.stderr: 7: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x559c4717a56f] 2023-11-08T19:28:31.014 INFO:tasks.ceph.mds.d.smithi174.stderr: 8: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f74065c59c8] 2023-11-08T19:28:31.014 INFO:tasks.ceph.mds.d.smithi174.stderr: 9: (DispatchQueue::entry()+0x51f) [0x7f74065c2b4f] 2023-11-08T19:28:31.014 INFO:tasks.ceph.mds.d.smithi174.stderr: 10: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f7406690351] 2023-11-08T19:28:31.014 INFO:tasks.ceph.mds.d.smithi174.stderr: 11: /lib64/libpthread.so.0(+0x81cf) [0x7f74050d61cf] 2023-11-08T19:28:31.014 INFO:tasks.ceph.mds.d.smithi174.stderr: 12: clone() 2023-11-08T19:28:31.015 INFO:tasks.ceph.mds.d.smithi174.stderr: 2023-11-08T19:28:31.015 INFO:tasks.ceph.mds.d.smithi174.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-08T19:28:31.015 INFO:tasks.ceph.mds.d.smithi174.stderr: 1: /lib64/libpthread.so.0(+0x12ce0) [0x7f74050e0ce0] 2023-11-08T19:28:31.016 INFO:tasks.ceph.mds.d.smithi174.stderr: 2: gsignal() 2023-11-08T19:28:31.016 INFO:tasks.ceph.mds.d.smithi174.stderr: 3: abort() 2023-11-08T19:28:31.016 INFO:tasks.ceph.mds.d.smithi174.stderr: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f740633dfb5] 2023-11-08T19:28:31.016 INFO:tasks.ceph.mds.d.smithi174.stderr: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2ba121) [0x7f740633e121] 2023-11-08T19:28:31.016 INFO:tasks.ceph.mds.d.smithi174.stderr: 6: (MDSTableServer::handle_prepare(boost::intrusive_ptr const&)+0x3e0) [0x559c474cc340] 2023-11-08T19:28:31.017 INFO:tasks.ceph.mds.d.smithi174.stderr: 7: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xe9) [0x559c4718d099] 2023-11-08T19:28:31.017 INFO:tasks.ceph.mds.d.smithi174.stderr: 8: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x593) [0x559c471901c3] 2023-11-08T19:28:31.017 INFO:tasks.ceph.mds.d.smithi174.stderr: 9: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x559c471907bc] 2023-11-08T19:28:31.017 INFO:tasks.ceph.mds.d.smithi174.stderr: 10: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x559c4717a56f] 2023-11-08T19:28:31.017 INFO:tasks.ceph.mds.d.smithi174.stderr: 11: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f74065c59c8] 2023-11-08T19:28:31.018 INFO:tasks.ceph.mds.d.smithi174.stderr: 12: (DispatchQueue::entry()+0x51f) [0x7f74065c2b4f] 2023-11-08T19:28:31.018 INFO:tasks.ceph.mds.d.smithi174.stderr: 13: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f7406690351] 2023-11-08T19:28:31.018 INFO:tasks.ceph.mds.d.smithi174.stderr: 14: /lib64/libpthread.so.0(+0x81cf) [0x7f74050d61cf] 2023-11-08T19:28:31.018 INFO:tasks.ceph.mds.d.smithi174.stderr: 15: clone() 2023-11-08T19:28:31.018 INFO:tasks.ceph.mds.d.smithi174.stderr:2023-11-08T19:28:31.009+0000 7f73faa75700 -1 *** Caught signal (Aborted) ** 2023-11-08T19:28:31.019 INFO:tasks.ceph.mds.d.smithi174.stderr: in thread 7f73faa75700 thread_name:ms_dispatch 2023-11-08T19:28:31.019 INFO:tasks.ceph.mds.d.smithi174.stderr: 2023-11-08T19:28:31.019 INFO:tasks.ceph.mds.d.smithi174.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-08T19:28:31.019 INFO:tasks.ceph.mds.d.smithi174.stderr: 1: /lib64/libpthread.so.0(+0x12ce0) [0x7f74050e0ce0] 2023-11-08T19:28:31.021 INFO:tasks.ceph.mds.d.smithi174.stderr: 2: gsignal() 2023-11-08T19:28:31.021 INFO:tasks.ceph.mds.d.smithi174.stderr: 3: abort() 2023-11-08T19:28:31.021 INFO:tasks.ceph.mds.d.smithi174.stderr: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f740633dfb5] 2023-11-08T19:28:31.021 INFO:tasks.ceph.mds.d.smithi174.stderr: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2ba121) [0x7f740633e121] 2023-11-08T19:28:31.022 INFO:tasks.ceph.mds.d.smithi174.stderr: 6: (MDSTableServer::handle_prepare(boost::intrusive_ptr const&)+0x3e0) [0x559c474cc340] 2023-11-08T19:28:31.022 INFO:tasks.ceph.mds.d.smithi174.stderr: 7: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xe9) [0x559c4718d099] 2023-11-08T19:28:31.022 INFO:tasks.ceph.mds.d.smithi174.stderr: 8: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x593) [0x559c471901c3] 2023-11-08T19:28:31.022 INFO:tasks.ceph.mds.d.smithi174.stderr: 9: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x559c471907bc] 2023-11-08T19:28:31.022 INFO:tasks.ceph.mds.d.smithi174.stderr: 10: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x559c4717a56f] 2023-11-08T19:28:31.023 INFO:tasks.ceph.mds.d.smithi174.stderr: 11: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f74065c59c8] 2023-11-08T19:28:31.023 INFO:tasks.ceph.mds.d.smithi174.stderr: 12: (DispatchQueue::entry()+0x51f) [0x7f74065c2b4f] 2023-11-08T19:28:31.023 INFO:tasks.ceph.mds.d.smithi174.stderr: 13: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f7406690351] 2023-11-08T19:28:31.023 INFO:tasks.ceph.mds.d.smithi174.stderr: 14: /lib64/libpthread.so.0(+0x81cf) [0x7f74050d61cf] 2023-11-08T19:28:31.023 INFO:tasks.ceph.mds.d.smithi174.stderr: 15: clone() 2023-11-08T19:28:31.024 INFO:tasks.ceph.mds.d.smithi174.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-11-08T19:28:31.024 INFO:tasks.ceph.mds.d.smithi174.stderr: 2023-11-08T19:28:31.024 INFO:tasks.ceph.mds.d.smithi174.stderr: -4627> 2023-11-08T19:28:09.685+0000 7f73f8270700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-08T19:28:31.025 INFO:tasks.ceph.mds.d.smithi174.stderr: -1811> 2023-11-08T19:28:19.686+0000 7f73f8270700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-11-08T19:28:31.025 INFO:tasks.ceph.mds.d.smithi174.stderr: -1> 2023-11-08T19:28:31.007+0000 7f73faa75700 -1 /home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el8/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: In function 'void MDSTableServer::handle_prepare(ceph::cref_t&)' thread 7f73faa75700 time 2023-11-08T19:28:31.007575+0000 2023-11-08T19:28:31.025 INFO:tasks.ceph.mds.d.smithi174.stderr:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el8/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: 61: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 1) 2023-11-08T19:28:31.025 INFO:tasks.ceph.mds.d.smithi174.stderr: 2023-11-08T19:28:31.026 INFO:tasks.ceph.mds.d.smithi174.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-08T19:28:31.026 INFO:tasks.ceph.mds.d.smithi174.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f740633df5b] 2023-11-08T19:28:31.026 INFO:tasks.ceph.mds.d.smithi174.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2ba121) [0x7f740633e121] 2023-11-08T19:28:31.026 INFO:tasks.ceph.mds.d.smithi174.stderr: 3: (MDSTableServer::handle_prepare(boost::intrusive_ptr const&)+0x3e0) [0x559c474cc340] 2023-11-08T19:28:31.026 INFO:tasks.ceph.mds.d.smithi174.stderr: 4: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xe9) [0x559c4718d099] 2023-11-08T19:28:31.027 INFO:tasks.ceph.mds.d.smithi174.stderr: 5: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x593) [0x559c471901c3] 2023-11-08T19:28:31.027 INFO:tasks.ceph.mds.d.smithi174.stderr: 6: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x559c471907bc] 2023-11-08T19:28:31.027 INFO:tasks.ceph.mds.d.smithi174.stderr: 7: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x559c4717a56f] 2023-11-08T19:28:31.027 INFO:tasks.ceph.mds.d.smithi174.stderr: 8: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f74065c59c8] 2023-11-08T19:28:31.027 INFO:tasks.ceph.mds.d.smithi174.stderr: 9: (DispatchQueue::entry()+0x51f) [0x7f74065c2b4f] 2023-11-08T19:28:31.028 INFO:tasks.ceph.mds.d.smithi174.stderr: 10: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f7406690351] 2023-11-08T19:28:31.028 INFO:tasks.ceph.mds.d.smithi174.stderr: 11: /lib64/libpthread.so.0(+0x81cf) [0x7f74050d61cf] 2023-11-08T19:28:31.028 INFO:tasks.ceph.mds.d.smithi174.stderr: 12: clone() 2023-11-08T19:28:31.028 INFO:tasks.ceph.mds.d.smithi174.stderr: 2023-11-08T19:28:31.028 INFO:tasks.ceph.mds.d.smithi174.stderr: 0> 2023-11-08T19:28:31.009+0000 7f73faa75700 -1 *** Caught signal (Aborted) ** 2023-11-08T19:28:31.029 INFO:tasks.ceph.mds.d.smithi174.stderr: in thread 7f73faa75700 thread_name:ms_dispatch 2023-11-08T19:28:31.029 INFO:tasks.ceph.mds.d.smithi174.stderr: 2023-11-08T19:28:31.029 INFO:tasks.ceph.mds.d.smithi174.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-08T19:28:31.029 INFO:tasks.ceph.mds.d.smithi174.stderr: 1: /lib64/libpthread.so.0(+0x12ce0) [0x7f74050e0ce0] 2023-11-08T19:28:31.029 INFO:tasks.ceph.mds.d.smithi174.stderr: 2: gsignal() 2023-11-08T19:28:31.030 INFO:tasks.ceph.mds.d.smithi174.stderr: 3: abort() 2023-11-08T19:28:31.030 INFO:tasks.ceph.mds.d.smithi174.stderr: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f740633dfb5] 2023-11-08T19:28:31.030 INFO:tasks.ceph.mds.d.smithi174.stderr: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2ba121) [0x7f740633e121] 2023-11-08T19:28:31.030 INFO:tasks.ceph.mds.d.smithi174.stderr: 6: (MDSTableServer::handle_prepare(boost::intrusive_ptr const&)+0x3e0) [0x559c474cc340] 2023-11-08T19:28:31.030 INFO:tasks.ceph.mds.d.smithi174.stderr: 7: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xe9) [0x559c4718d099] 2023-11-08T19:28:31.031 INFO:tasks.ceph.mds.d.smithi174.stderr: 8: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x593) [0x559c471901c3] 2023-11-08T19:28:31.031 INFO:tasks.ceph.mds.d.smithi174.stderr: 9: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x559c471907bc] 2023-11-08T19:28:31.031 INFO:tasks.ceph.mds.d.smithi174.stderr: 10: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x559c4717a56f] 2023-11-08T19:28:31.031 INFO:tasks.ceph.mds.d.smithi174.stderr: 11: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f74065c59c8] 2023-11-08T19:28:31.031 INFO:tasks.ceph.mds.d.smithi174.stderr: 12: (DispatchQueue::entry()+0x51f) [0x7f74065c2b4f] 2023-11-08T19:28:31.031 INFO:tasks.ceph.mds.d.smithi174.stderr: 13: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f7406690351] 2023-11-08T19:28:31.032 INFO:tasks.ceph.mds.d.smithi174.stderr: 14: /lib64/libpthread.so.0(+0x81cf) [0x7f74050d61cf] 2023-11-08T19:28:31.032 INFO:tasks.ceph.mds.d.smithi174.stderr: 15: clone() 2023-11-08T19:28:31.032 INFO:tasks.ceph.mds.d.smithi174.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-11-08T19:28:31.032 INFO:tasks.ceph.mds.d.smithi174.stderr: 2023-11-08T19:28:31.033 INFO:tasks.ceph.mds.d.smithi174.stderr: -4627> 2023-11-08T19:28:09.685+0000 7f73f8270700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-08T19:28:31.033 INFO:tasks.ceph.mds.d.smithi174.stderr: -1811> 2023-11-08T19:28:19.686+0000 7f73f8270700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-11-08T19:28:31.034 INFO:tasks.ceph.mds.d.smithi174.stderr: -1> 2023-11-08T19:28:31.007+0000 7f73faa75700 -1 /home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el8/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: In function 'void MDSTableServer::handle_prepare(ceph::cref_t&)' thread 7f73faa75700 time 2023-11-08T19:28:31.007575+0000 2023-11-08T19:28:31.034 INFO:tasks.ceph.mds.d.smithi174.stderr:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el8/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: 61: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 1) 2023-11-08T19:28:31.034 INFO:tasks.ceph.mds.d.smithi174.stderr: 2023-11-08T19:28:31.035 INFO:tasks.ceph.mds.d.smithi174.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-08T19:28:31.035 INFO:tasks.ceph.mds.d.smithi174.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f740633df5b] 2023-11-08T19:28:31.035 INFO:tasks.ceph.mds.d.smithi174.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2ba121) [0x7f740633e121] 2023-11-08T19:28:31.035 INFO:tasks.ceph.mds.d.smithi174.stderr: 3: (MDSTableServer::handle_prepare(boost::intrusive_ptr const&)+0x3e0) [0x559c474cc340] 2023-11-08T19:28:31.035 INFO:tasks.ceph.mds.d.smithi174.stderr: 4: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xe9) [0x559c4718d099] 2023-11-08T19:28:31.035 INFO:tasks.ceph.mds.d.smithi174.stderr: 5: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x593) [0x559c471901c3] 2023-11-08T19:28:31.036 INFO:tasks.ceph.mds.d.smithi174.stderr: 6: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x559c471907bc] 2023-11-08T19:28:31.036 INFO:tasks.ceph.mds.d.smithi174.stderr: 7: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x559c4717a56f] 2023-11-08T19:28:31.036 INFO:tasks.ceph.mds.d.smithi174.stderr: 8: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f74065c59c8] 2023-11-08T19:28:31.036 INFO:tasks.ceph.mds.d.smithi174.stderr: 9: (DispatchQueue::entry()+0x51f) [0x7f74065c2b4f] 2023-11-08T19:28:31.036 INFO:tasks.ceph.mds.d.smithi174.stderr: 10: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f7406690351] 2023-11-08T19:28:31.037 INFO:tasks.ceph.mds.d.smithi174.stderr: 11: /lib64/libpthread.so.0(+0x81cf) [0x7f74050d61cf] 2023-11-08T19:28:31.037 INFO:tasks.ceph.mds.d.smithi174.stderr: 12: clone() 2023-11-08T19:28:31.037 INFO:tasks.ceph.mds.d.smithi174.stderr: 2023-11-08T19:28:31.037 INFO:tasks.ceph.mds.d.smithi174.stderr: 0> 2023-11-08T19:28:31.009+0000 7f73faa75700 -1 *** Caught signal (Aborted) ** 2023-11-08T19:28:31.037 INFO:tasks.ceph.mds.d.smithi174.stderr: in thread 7f73faa75700 thread_name:ms_dispatch 2023-11-08T19:28:31.038 INFO:tasks.ceph.mds.d.smithi174.stderr: 2023-11-08T19:28:31.038 INFO:tasks.ceph.mds.d.smithi174.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-08T19:28:31.038 INFO:tasks.ceph.mds.d.smithi174.stderr: 1: /lib64/libpthread.so.0(+0x12ce0) [0x7f74050e0ce0] 2023-11-08T19:28:31.038 INFO:tasks.ceph.mds.d.smithi174.stderr: 2: gsignal() 2023-11-08T19:28:31.038 INFO:tasks.ceph.mds.d.smithi174.stderr: 3: abort() 2023-11-08T19:28:31.039 INFO:tasks.ceph.mds.d.smithi174.stderr: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f740633dfb5] 2023-11-08T19:28:31.039 INFO:tasks.ceph.mds.d.smithi174.stderr: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2ba121) [0x7f740633e121] 2023-11-08T19:28:31.039 INFO:tasks.ceph.mds.d.smithi174.stderr: 6: (MDSTableServer::handle_prepare(boost::intrusive_ptr const&)+0x3e0) [0x559c474cc340] 2023-11-08T19:28:31.039 INFO:tasks.ceph.mds.d.smithi174.stderr: 7: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xe9) [0x559c4718d099] 2023-11-08T19:28:31.039 INFO:tasks.ceph.mds.d.smithi174.stderr: 8: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x593) [0x559c471901c3] 2023-11-08T19:28:31.040 INFO:tasks.ceph.mds.d.smithi174.stderr: 9: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x559c471907bc] 2023-11-08T19:28:31.040 INFO:tasks.ceph.mds.d.smithi174.stderr: 10: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x559c4717a56f] 2023-11-08T19:28:31.040 INFO:tasks.ceph.mds.d.smithi174.stderr: 11: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f74065c59c8] 2023-11-08T19:28:31.040 INFO:tasks.ceph.mds.d.smithi174.stderr: 12: (DispatchQueue::entry()+0x51f) [0x7f74065c2b4f] 2023-11-08T19:28:31.040 INFO:tasks.ceph.mds.d.smithi174.stderr: 13: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f7406690351] 2023-11-08T19:28:31.041 INFO:tasks.ceph.mds.d.smithi174.stderr: 14: /lib64/libpthread.so.0(+0x81cf) [0x7f74050d61cf] 2023-11-08T19:28:31.041 INFO:tasks.ceph.mds.d.smithi174.stderr: 15: clone() 2023-11-08T19:28:31.041 INFO:tasks.ceph.mds.d.smithi174.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-11-08T19:28:31.041 INFO:tasks.ceph.mds.d.smithi174.stderr: 2023-11-08T19:28:31.173 INFO:teuthology.orchestra.run.smithi143.stdout:{"mds_beacon_grace":"15.000000"} 2023-11-08T19:28:31.173 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_beacon_grace": "15.000000" } 2023-11-08T19:28:31.174 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:28:31.226 INFO:tasks.ceph.mds.d.smithi174.stderr:daemon-helper: command crashed with signal 6 2023-11-08T19:28:31.294 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:31.293+0000 7f00e1c30700 1 Processor -- start 2023-11-08T19:28:31.294 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:31.293+0000 7f00e1c30700 1 -- start start 2023-11-08T19:28:31.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:31.293+0000 7f00e1c30700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f00dc06b320 0x7f00dc168c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:31.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:31.293+0000 7f00e1c30700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00dc161df0 con 0x7f00dc15dd00 2023-11-08T19:28:31.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:31.294+0000 7f00e1c30700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00dc1620d0 con 0x7f00dc1618f0 2023-11-08T19:28:31.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:31.294+0000 7f00e1c30700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00dc0cad50 con 0x7f00dc06b320 2023-11-08T19:28:31.299 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:31.294+0000 7f00db7fe700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f00dc15dd00 0x7f00dc0cca70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:51214/0 (socket says 172.21.15.143:51214) 2023-11-08T19:28:31.299 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:31.294+0000 7f00db7fe700 1 -- 172.21.15.143:0/995240202 learned_addr learned my addr 172.21.15.143:0/995240202 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:28:31.299 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:31.294+0000 7f00da7fc700 1 -- 172.21.15.143:0/995240202 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3256408236 0 0) 0x7f00dc161df0 con 0x7f00dc15dd00 2023-11-08T19:28:31.299 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:31.294+0000 7f00da7fc700 1 -- 172.21.15.143:0/995240202 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00c40035b0 con 0x7f00dc15dd00 2023-11-08T19:28:31.300 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:31.294+0000 7f00dbfff700 1 --2- 172.21.15.143:0/995240202 >> v2:172.21.15.174:3300/0 conn(0x7f00dc06b320 0x7f00dc168c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:31.300 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:31.294+0000 7f00da7fc700 1 -- 172.21.15.143:0/995240202 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3879133952 0 0) 0x7f00c40035b0 con 0x7f00dc15dd00 2023-11-08T19:28:31.300 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:31.294+0000 7f00da7fc700 1 -- 172.21.15.143:0/995240202 >> v2:172.21.15.174:3300/0 conn(0x7f00dc06b320 msgr2=0x7f00dc168c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:31.300 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:31.294+0000 7f00da7fc700 1 --2- 172.21.15.143:0/995240202 >> v2:172.21.15.174:3300/0 conn(0x7f00dc06b320 0x7f00dc168c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:31.300 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:31.294+0000 7f00da7fc700 1 -- 172.21.15.143:0/995240202 >> v1:172.21.15.174:6789/0 conn(0x7f00dc1618f0 legacy=0x7f00dc06ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:31.301 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:31.294+0000 7f00da7fc700 1 -- 172.21.15.143:0/995240202 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00dc1623b0 con 0x7f00dc15dd00 2023-11-08T19:28:31.301 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:31.294+0000 7f00da7fc700 1 -- 172.21.15.143:0/995240202 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4147756303 0 0) 0x7f00cc0017f0 con 0x7f00dc15dd00 2023-11-08T19:28:31.301 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:31.294+0000 7f00da7fc700 1 -- 172.21.15.143:0/995240202 >> v1:172.21.15.143:6789/0 conn(0x7f00dc15dd00 legacy=0x7f00dc0cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:31.301 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:31.294+0000 7f00da7fc700 1 --2- 172.21.15.143:0/995240202 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f00c4003a00 0x7f00c4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:31.301 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:31.294+0000 7f00da7fc700 1 --2- 172.21.15.143:0/995240202 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f00c4004850 0x7f00c4004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:31.301 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:31.294+0000 7f00da7fc700 1 --2- 172.21.15.143:0/995240202 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f00c4005210 0x7f00c400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:31.302 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:31.294+0000 7f00da7fc700 1 -- 172.21.15.143:0/995240202 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00c400c080 con 0x7f00c4003a00 2023-11-08T19:28:31.302 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:31.294+0000 7f00da7fc700 1 -- 172.21.15.143:0/995240202 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00c400c1f0 con 0x7f00c4005210 2023-11-08T19:28:31.302 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:31.294+0000 7f00da7fc700 1 -- 172.21.15.143:0/995240202 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00c400c3b0 con 0x7f00c4004850 2023-11-08T19:28:31.302 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:31.294+0000 7f00db7fe700 1 --2- 172.21.15.143:0/995240202 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f00c4003a00 0x7f00c4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:31.303 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:31.294+0000 7f00dbfff700 1 --2- 172.21.15.143:0/995240202 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f00c4005210 0x7f00c400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:31.303 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:31.294+0000 7f00daffd700 1 --2- 172.21.15.143:0/995240202 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f00c4004850 0x7f00c4004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:31.303 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:31.295+0000 7f00db7fe700 1 -- 172.21.15.143:0/995240202 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f00c4004850 msgr2=0x7f00c4004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:31.303 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:31.295+0000 7f00db7fe700 1 --2- 172.21.15.143:0/995240202 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f00c4004850 0x7f00c4004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:31.303 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:31.295+0000 7f00db7fe700 1 -- 172.21.15.143:0/995240202 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f00c4005210 msgr2=0x7f00c400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:31.303 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:31.295+0000 7f00db7fe700 1 --2- 172.21.15.143:0/995240202 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f00c4005210 0x7f00c400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:31.304 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:31.295+0000 7f00dbfff700 1 -- 172.21.15.143:0/995240202 reap_dead start 2023-11-08T19:28:31.304 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:31.295+0000 7f00db7fe700 1 -- 172.21.15.143:0/995240202 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00c40035b0 con 0x7f00c4003a00 2023-11-08T19:28:31.304 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:31.295+0000 7f00db7fe700 1 --2- 172.21.15.143:0/995240202 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f00c4003a00 0x7f00c4003e20 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f00cc00c240 tx=0x7f00c4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25baa863ec2082e8 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:31.304 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:31.295+0000 7f00da7fc700 1 -- 172.21.15.143:0/995240202 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00cc00cd60 con 0x7f00c4003a00 2023-11-08T19:28:31.304 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:31.295+0000 7f00da7fc700 1 -- 172.21.15.143:0/995240202 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00cc007b80 con 0x7f00c4003a00 2023-11-08T19:28:31.305 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:31.295+0000 7f00da7fc700 1 -- 172.21.15.143:0/995240202 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00cc0072f0 con 0x7f00c4003a00 2023-11-08T19:28:31.305 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:31.295+0000 7f00e1c30700 1 -- 172.21.15.143:0/995240202 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f00c4003a00 msgr2=0x7f00c4003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:31.305 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:31.295+0000 7f00e1c30700 1 --2- 172.21.15.143:0/995240202 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f00c4003a00 0x7f00c4003e20 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f00cc00c240 tx=0x7f00c4005880 comp rx=0 tx=0).stop 2023-11-08T19:28:31.305 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:31.296+0000 7f00e1c30700 1 -- 172.21.15.143:0/995240202 shutdown_connections 2023-11-08T19:28:31.305 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:31.296+0000 7f00e1c30700 1 --2- 172.21.15.143:0/995240202 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f00c4003a00 0x7f00c4003e20 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:31.306 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:31.296+0000 7f00e1c30700 1 -- 172.21.15.143:0/995240202 >> 172.21.15.143:0/995240202 conn(0x7f00dc099ab0 msgr2=0x7f00dc0d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:31.306 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:31.296+0000 7f00e1c30700 1 -- 172.21.15.143:0/995240202 shutdown_connections 2023-11-08T19:28:31.306 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:31.296+0000 7f00e1c30700 1 -- 172.21.15.143:0/995240202 wait complete. 2023-11-08T19:28:31.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:31.297+0000 7f00e1c30700 1 Processor -- start 2023-11-08T19:28:31.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:31.297+0000 7f00e1c30700 1 -- start start 2023-11-08T19:28:31.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:31.298+0000 7f00e1c30700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f00c4003a00 0x7f00dc151c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:31.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:31.298+0000 7f00e1c30700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f00dc0c9e50 0x7f00dc0c6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:31.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:31.298+0000 7f00e1c30700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f00dc0c5bb0 0x7f00dc0c6640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:31.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:31.298+0000 7f00e1c30700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00dc0ca6a0 con 0x7f00dc0c9e50 2023-11-08T19:28:31.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:31.298+0000 7f00e1c30700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00dc0ca3c0 con 0x7f00dc0c5bb0 2023-11-08T19:28:31.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:31.298+0000 7f00e1c30700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00dc0b5000 con 0x7f00c4003a00 2023-11-08T19:28:31.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:31.298+0000 7f00daffd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f00dc0c9e50 0x7f00dc0c6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:31.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:31.298+0000 7f00daffd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f00dc0c9e50 0x7f00dc0c6100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55074/0 (socket says 172.21.15.143:55074) 2023-11-08T19:28:31.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:31.298+0000 7f00daffd700 1 -- 172.21.15.143:0/2109793834 learned_addr learned my addr 172.21.15.143:0/2109793834 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:28:31.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:31.298+0000 7f00db7fe700 1 --2- 172.21.15.143:0/2109793834 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f00c4003a00 0x7f00dc151c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:31.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:31.299+0000 7f00dbfff700 1 --2- 172.21.15.143:0/2109793834 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f00dc0c5bb0 0x7f00dc0c6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:31.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:31.299+0000 7f00daffd700 1 -- 172.21.15.143:0/2109793834 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f00c4003a00 msgr2=0x7f00dc151c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:31.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:31.299+0000 7f00daffd700 1 --2- 172.21.15.143:0/2109793834 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f00c4003a00 0x7f00dc151c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:31.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:31.299+0000 7f00daffd700 1 -- 172.21.15.143:0/2109793834 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f00dc0c5bb0 msgr2=0x7f00dc0c6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:31.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:31.299+0000 7f00daffd700 1 --2- 172.21.15.143:0/2109793834 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f00dc0c5bb0 0x7f00dc0c6640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:31.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:31.299+0000 7f00daffd700 1 -- 172.21.15.143:0/2109793834 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00dc15e260 con 0x7f00dc0c9e50 2023-11-08T19:28:31.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:31.299+0000 7f00dbfff700 1 --2- 172.21.15.143:0/2109793834 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f00dc0c5bb0 0x7f00dc0c6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:28:31.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:31.299+0000 7f00daffd700 1 --2- 172.21.15.143:0/2109793834 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f00dc0c9e50 0x7f00dc0c6100 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f00d000f820 tx=0x7f00d000fbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:31.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:31.299+0000 7f00c3fff700 1 -- 172.21.15.143:0/2109793834 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00d00052e0 con 0x7f00dc0c9e50 2023-11-08T19:28:31.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:31.299+0000 7f00c3fff700 1 -- 172.21.15.143:0/2109793834 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00d0005460 con 0x7f00dc0c9e50 2023-11-08T19:28:31.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:31.299+0000 7f00e1c30700 1 -- 172.21.15.143:0/2109793834 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00dc0b52e0 con 0x7f00dc0c9e50 2023-11-08T19:28:31.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:31.299+0000 7f00c3fff700 1 -- 172.21.15.143:0/2109793834 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00d00038c0 con 0x7f00dc0c9e50 2023-11-08T19:28:31.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:31.299+0000 7f00e1c30700 1 -- 172.21.15.143:0/2109793834 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00dc0c57f0 con 0x7f00dc0c9e50 2023-11-08T19:28:31.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:31.300+0000 7f00e1c30700 1 -- 172.21.15.143:0/2109793834 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00dc0b4e60 con 0x7f00dc0c9e50 2023-11-08T19:28:31.315 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:31.300+0000 7f00c3fff700 1 -- 172.21.15.143:0/2109793834 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f00d0003a20 con 0x7f00dc0c9e50 2023-11-08T19:28:31.315 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:31.301+0000 7f00c3fff700 1 --2- 172.21.15.143:0/2109793834 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f00c806af00 0x7f00c806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:31.315 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:31.301+0000 7f00c3fff700 1 -- 172.21.15.143:0/2109793834 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6772+0+0 (secure 0 0 0) 0x7f00d00897d0 con 0x7f00dc0c9e50 2023-11-08T19:28:31.315 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:31.301+0000 7f00db7fe700 1 --2- 172.21.15.143:0/2109793834 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f00c806af00 0x7f00c806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:31.315 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:31.301+0000 7f00db7fe700 1 --2- 172.21.15.143:0/2109793834 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f00c806af00 0x7f00c806d3c0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f00cc007d30 tx=0x7f00cc007000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:31.316 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:31.305+0000 7f00c3fff700 1 -- 172.21.15.143:0/2109793834 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f00dc0b4e60 con 0x7f00dc0c9e50 2023-11-08T19:28:31.557 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:31.555+0000 7f00e1c30700 1 -- 172.21.15.143:0/2109793834 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f00dc0b5650 con 0x7f00dc0c9e50 2023-11-08T19:28:31.557 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:31.556+0000 7f00c3fff700 1 -- 172.21.15.143:0/2109793834 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+4942 (secure 0 0 0) 0x7f00d00543b0 con 0x7f00dc0c9e50 2023-11-08T19:28:31.558 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:28:31.558 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":57,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6906,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6834/3172664377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":3172664377},{"type":"v1","addr":"172.21.15.143:6834","nonce":3172664377}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47},{"gid":6909,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3953059900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":3953059900},{"type":"v1","addr":"172.21.15.143:6835","nonce":3953059900}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":46}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:03.092245+0000","modified":"2023-11-08T19:28:29.816138+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6921,"mds_1":6918},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6918":{"gid":6918,"name":"b","rank":1,"incarnation":54,"state":"up:active","state_seq":6,"addr":"172.21.15.174:6836/3601088618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3601088618},{"type":"v1","addr":"172.21.15.174:6836","nonce":3601088618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6921":{"gid":6921,"name":"d","rank":0,"incarnation":48,"state":"up:active","state_seq":3,"addr":"172.21.15.174:6837/2201447629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6835","nonce":2201447629},{"type":"v1","addr":"172.21.15.174:6837","nonce":2201447629}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:28:31.559 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:31.558+0000 7f00e1c30700 1 -- 172.21.15.143:0/2109793834 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f00c806af00 msgr2=0x7f00c806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:31.560 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:31.558+0000 7f00e1c30700 1 --2- 172.21.15.143:0/2109793834 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f00c806af00 0x7f00c806d3c0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f00cc007d30 tx=0x7f00cc007000 comp rx=0 tx=0).stop 2023-11-08T19:28:31.560 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:31.558+0000 7f00e1c30700 1 -- 172.21.15.143:0/2109793834 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f00dc0c9e50 msgr2=0x7f00dc0c6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:31.560 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:31.558+0000 7f00e1c30700 1 --2- 172.21.15.143:0/2109793834 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f00dc0c9e50 0x7f00dc0c6100 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f00d000f820 tx=0x7f00d000fbe0 comp rx=0 tx=0).stop 2023-11-08T19:28:31.560 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:31.558+0000 7f00e1c30700 1 -- 172.21.15.143:0/2109793834 shutdown_connections 2023-11-08T19:28:31.561 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:31.558+0000 7f00e1c30700 1 --2- 172.21.15.143:0/2109793834 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f00c806af00 0x7f00c806d3c0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:31.561 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:31.558+0000 7f00e1c30700 1 --2- 172.21.15.143:0/2109793834 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f00c4003a00 0x7f00dc151c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:31.561 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:31.558+0000 7f00e1c30700 1 --2- 172.21.15.143:0/2109793834 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f00dc0c5bb0 0x7f00dc0c6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:31.561 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:31.558+0000 7f00e1c30700 1 --2- 172.21.15.143:0/2109793834 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f00dc0c9e50 0x7f00dc0c6100 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:31.561 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:31.558+0000 7f00e1c30700 1 -- 172.21.15.143:0/2109793834 >> 172.21.15.143:0/2109793834 conn(0x7f00dc099ab0 msgr2=0x7f00dc15bee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:31.561 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:31.558+0000 7f00e1c30700 1 -- 172.21.15.143:0/2109793834 shutdown_connections 2023-11-08T19:28:31.562 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:31.558+0000 7f00e1c30700 1 -- 172.21.15.143:0/2109793834 wait complete. 2023-11-08T19:28:31.562 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 57 2023-11-08T19:28:31.571 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2023-11-08T19:28:36.572 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:28:36.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:36.706+0000 7efc177cc700 1 Processor -- start 2023-11-08T19:28:36.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:36.706+0000 7efc177cc700 1 -- start start 2023-11-08T19:28:36.710 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:36.707+0000 7efc177cc700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7efc1006b320 0x7efc10168c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:36.710 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:36.707+0000 7efc177cc700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc10161df0 con 0x7efc1015dd00 2023-11-08T19:28:36.710 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:36.707+0000 7efc177cc700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc101620d0 con 0x7efc101618f0 2023-11-08T19:28:36.710 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:36.707+0000 7efc177cc700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc100cad50 con 0x7efc1006b320 2023-11-08T19:28:36.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:36.707+0000 7efc15568700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7efc1015dd00 0x7efc100cca70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:51244/0 (socket says 172.21.15.143:51244) 2023-11-08T19:28:36.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:36.707+0000 7efc15568700 1 -- 172.21.15.143:0/3843318224 learned_addr learned my addr 172.21.15.143:0/3843318224 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:28:36.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:36.707+0000 7efc15d69700 1 --2- 172.21.15.143:0/3843318224 >> v2:172.21.15.143:3300/0 conn(0x7efc1006b320 0x7efc10168c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:36.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:36.707+0000 7efc07fff700 1 -- 172.21.15.143:0/3843318224 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 57776925 0 0) 0x7efc10161df0 con 0x7efc1015dd00 2023-11-08T19:28:36.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:36.707+0000 7efc07fff700 1 -- 172.21.15.143:0/3843318224 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efbfc0035b0 con 0x7efc1015dd00 2023-11-08T19:28:36.713 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:36.707+0000 7efc15d69700 1 -- 172.21.15.143:0/3843318224 >> v1:172.21.15.174:6789/0 conn(0x7efc101618f0 legacy=0x7efc1006ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:36.713 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:36.707+0000 7efc15d69700 1 -- 172.21.15.143:0/3843318224 >> v1:172.21.15.143:6789/0 conn(0x7efc1015dd00 legacy=0x7efc100cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:36.713 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:36.707+0000 7efc15d69700 1 -- 172.21.15.143:0/3843318224 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc101623b0 con 0x7efc1006b320 2023-11-08T19:28:36.713 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:36.707+0000 7efc15d69700 1 --2- 172.21.15.143:0/3843318224 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efc1006b320 0x7efc10168c60 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7efc08009d90 tx=0x7efc0800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa0583fd9826f405 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:36.713 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:36.708+0000 7efc07fff700 1 -- 172.21.15.143:0/3843318224 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc0800e070 con 0x7efc1006b320 2023-11-08T19:28:36.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:36.708+0000 7efc07fff700 1 -- 172.21.15.143:0/3843318224 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc0800bb20 con 0x7efc1006b320 2023-11-08T19:28:36.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:36.708+0000 7efc07fff700 1 -- 172.21.15.143:0/3843318224 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc08012430 con 0x7efc1006b320 2023-11-08T19:28:36.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:36.708+0000 7efc177cc700 1 -- 172.21.15.143:0/3843318224 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efc1006b320 msgr2=0x7efc10168c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:36.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:36.708+0000 7efc177cc700 1 --2- 172.21.15.143:0/3843318224 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efc1006b320 0x7efc10168c60 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7efc08009d90 tx=0x7efc0800b330 comp rx=0 tx=0).stop 2023-11-08T19:28:36.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:36.708+0000 7efc177cc700 1 -- 172.21.15.143:0/3843318224 shutdown_connections 2023-11-08T19:28:36.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:36.708+0000 7efc177cc700 1 --2- 172.21.15.143:0/3843318224 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efc1006b320 0x7efc10168c60 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:36.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:36.708+0000 7efc177cc700 1 -- 172.21.15.143:0/3843318224 >> 172.21.15.143:0/3843318224 conn(0x7efc10099ab0 msgr2=0x7efc100d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:36.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:36.708+0000 7efc177cc700 1 -- 172.21.15.143:0/3843318224 shutdown_connections 2023-11-08T19:28:36.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:36.708+0000 7efc177cc700 1 -- 172.21.15.143:0/3843318224 wait complete. 2023-11-08T19:28:36.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:36.709+0000 7efc177cc700 1 Processor -- start 2023-11-08T19:28:36.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:36.710+0000 7efc177cc700 1 -- start start 2023-11-08T19:28:36.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:36.710+0000 7efc177cc700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efc1006b320 0x7efc10154d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:36.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:36.710+0000 7efc177cc700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efc1015dd00 0x7efc100b1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:36.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:36.710+0000 7efc177cc700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efc101618f0 0x7efc100af180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:36.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:36.710+0000 7efc177cc700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc100c4b60 con 0x7efc1006b320 2023-11-08T19:28:36.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:36.710+0000 7efc177cc700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc100b2420 con 0x7efc1015dd00 2023-11-08T19:28:36.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:36.710+0000 7efc177cc700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc100c4230 con 0x7efc101618f0 2023-11-08T19:28:36.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:36.710+0000 7efc15568700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efc1006b320 0x7efc10154d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:36.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:36.710+0000 7efc15568700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efc1006b320 0x7efc10154d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55096/0 (socket says 172.21.15.143:55096) 2023-11-08T19:28:36.721 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:36.710+0000 7efc15568700 1 -- 172.21.15.143:0/791674554 learned_addr learned my addr 172.21.15.143:0/791674554 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:28:36.721 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:36.711+0000 7efc15d69700 1 --2- 172.21.15.143:0/791674554 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efc101618f0 0x7efc100af180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:36.721 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:36.711+0000 7efc14d67700 1 --2- 172.21.15.143:0/791674554 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efc1015dd00 0x7efc100b1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:36.721 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:36.711+0000 7efc15568700 1 -- 172.21.15.143:0/791674554 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efc101618f0 msgr2=0x7efc100af180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:36.721 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:36.711+0000 7efc15568700 1 --2- 172.21.15.143:0/791674554 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efc101618f0 0x7efc100af180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:36.722 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:36.711+0000 7efc15568700 1 -- 172.21.15.143:0/791674554 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efc1015dd00 msgr2=0x7efc100b1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:36.722 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:36.711+0000 7efc15568700 1 --2- 172.21.15.143:0/791674554 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efc1015dd00 0x7efc100b1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:36.722 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:36.711+0000 7efc15568700 1 -- 172.21.15.143:0/791674554 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc08009a70 con 0x7efc1006b320 2023-11-08T19:28:36.722 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:36.711+0000 7efc15568700 1 --2- 172.21.15.143:0/791674554 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efc1006b320 0x7efc10154d70 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7efc0c001260 tx=0x7efc0c00ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:36.722 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:36.711+0000 7efc05ffb700 1 -- 172.21.15.143:0/791674554 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc0c0074e0 con 0x7efc1006b320 2023-11-08T19:28:36.723 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:36.711+0000 7efc05ffb700 1 -- 172.21.15.143:0/791674554 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc0c007640 con 0x7efc1006b320 2023-11-08T19:28:36.723 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:36.711+0000 7efc177cc700 1 -- 172.21.15.143:0/791674554 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc1015e260 con 0x7efc1006b320 2023-11-08T19:28:36.723 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:36.711+0000 7efc05ffb700 1 -- 172.21.15.143:0/791674554 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc0c005a50 con 0x7efc1006b320 2023-11-08T19:28:36.723 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:36.711+0000 7efc177cc700 1 -- 172.21.15.143:0/791674554 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc100c47f0 con 0x7efc1006b320 2023-11-08T19:28:36.723 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:36.712+0000 7efc177cc700 1 -- 172.21.15.143:0/791674554 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc1004eee0 con 0x7efc1006b320 2023-11-08T19:28:36.724 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:36.712+0000 7efc05ffb700 1 -- 172.21.15.143:0/791674554 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7efc0c005bb0 con 0x7efc1006b320 2023-11-08T19:28:36.724 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:36.713+0000 7efc05ffb700 1 --2- 172.21.15.143:0/791674554 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7efbf406af00 0x7efbf406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:36.724 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:36.713+0000 7efc05ffb700 1 -- 172.21.15.143:0/791674554 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6772+0+0 (secure 0 0 0) 0x7efc0c088f80 con 0x7efc1006b320 2023-11-08T19:28:36.724 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:36.713+0000 7efc14d67700 1 --2- 172.21.15.143:0/791674554 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7efbf406af00 0x7efbf406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:36.724 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:36.713+0000 7efc14d67700 1 --2- 172.21.15.143:0/791674554 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7efbf406af00 0x7efbf406d3c0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7efc100c71a0 tx=0x7efc00009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:36.725 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:36.717+0000 7efc05ffb700 1 -- 172.21.15.143:0/791674554 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7efc0c0bbca0 con 0x7efc1006b320 2023-11-08T19:28:36.826 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2023-11-08T19:28:36.949 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:36.948+0000 7efc177cc700 1 -- 172.21.15.143:0/791674554 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efc1006b920 con 0x7efc1006b320 2023-11-08T19:28:36.950 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:36.948+0000 7efc05ffb700 1 -- 172.21.15.143:0/791674554 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+4942 (secure 0 0 0) 0x7efc0c017020 con 0x7efc1006b320 2023-11-08T19:28:36.951 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:28:36.951 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":57,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6906,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6834/3172664377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":3172664377},{"type":"v1","addr":"172.21.15.143:6834","nonce":3172664377}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47},{"gid":6909,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3953059900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":3953059900},{"type":"v1","addr":"172.21.15.143:6835","nonce":3953059900}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":46}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:03.092245+0000","modified":"2023-11-08T19:28:29.816138+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6921,"mds_1":6918},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6918":{"gid":6918,"name":"b","rank":1,"incarnation":54,"state":"up:active","state_seq":6,"addr":"172.21.15.174:6836/3601088618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3601088618},{"type":"v1","addr":"172.21.15.174:6836","nonce":3601088618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6921":{"gid":6921,"name":"d","rank":0,"incarnation":48,"state":"up:active","state_seq":3,"addr":"172.21.15.174:6837/2201447629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6835","nonce":2201447629},{"type":"v1","addr":"172.21.15.174:6837","nonce":2201447629}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:28:36.953 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:36.950+0000 7efc177cc700 1 -- 172.21.15.143:0/791674554 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7efbf406af00 msgr2=0x7efbf406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:36.953 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:36.950+0000 7efc177cc700 1 --2- 172.21.15.143:0/791674554 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7efbf406af00 0x7efbf406d3c0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7efc100c71a0 tx=0x7efc00009040 comp rx=0 tx=0).stop 2023-11-08T19:28:36.953 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:36.950+0000 7efc177cc700 1 -- 172.21.15.143:0/791674554 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efc1006b320 msgr2=0x7efc10154d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:36.954 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:36.950+0000 7efc177cc700 1 --2- 172.21.15.143:0/791674554 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efc1006b320 0x7efc10154d70 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7efc0c001260 tx=0x7efc0c00ca90 comp rx=0 tx=0).stop 2023-11-08T19:28:36.954 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:36.951+0000 7efc177cc700 1 -- 172.21.15.143:0/791674554 shutdown_connections 2023-11-08T19:28:36.954 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:36.951+0000 7efc177cc700 1 --2- 172.21.15.143:0/791674554 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7efbf406af00 0x7efbf406d3c0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:36.954 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:36.951+0000 7efc177cc700 1 --2- 172.21.15.143:0/791674554 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efc101618f0 0x7efc100af180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:36.954 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:36.951+0000 7efc177cc700 1 --2- 172.21.15.143:0/791674554 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efc1006b320 0x7efc10154d70 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:36.955 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:36.951+0000 7efc177cc700 1 --2- 172.21.15.143:0/791674554 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efc1015dd00 0x7efc100b1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:36.955 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:36.951+0000 7efc177cc700 1 -- 172.21.15.143:0/791674554 >> 172.21.15.143:0/791674554 conn(0x7efc10099ab0 msgr2=0x7efc1006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:36.955 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:36.951+0000 7efc177cc700 1 -- 172.21.15.143:0/791674554 shutdown_connections 2023-11-08T19:28:36.955 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:36.951+0000 7efc177cc700 1 -- 172.21.15.143:0/791674554 wait complete. 2023-11-08T19:28:36.955 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 57 2023-11-08T19:28:36.964 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2023-11-08T19:28:41.966 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:28:42.099 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:42.098+0000 7fbfb9fa6700 1 Processor -- start 2023-11-08T19:28:42.100 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:42.098+0000 7fbfb9fa6700 1 -- start start 2023-11-08T19:28:42.102 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:42.099+0000 7fbfb9fa6700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbfb415aa30 0x7fbfb40cf510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:42.102 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:42.099+0000 7fbfb9fa6700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fbfb40cc530 0x7fbfb40b2690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:42.103 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:42.099+0000 7fbfb9fa6700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfb40cf080 con 0x7fbfb40b2210 2023-11-08T19:28:42.103 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:42.099+0000 7fbfb9fa6700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfb40b2bd0 con 0x7fbfb40cc530 2023-11-08T19:28:42.103 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:42.099+0000 7fbfb9fa6700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb40ce470 con 0x7fbfb415aa30 2023-11-08T19:28:42.103 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:42.099+0000 7fbfb3fff700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fbfb40cc530 0x7fbfb40b2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:42.103 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:42.100+0000 7fbfb3fff700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fbfb40cc530 0x7fbfb40b2690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55110/0 (socket says 172.21.15.143:55110) 2023-11-08T19:28:42.104 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:42.100+0000 7fbfb3fff700 1 -- 172.21.15.143:0/2823133190 learned_addr learned my addr 172.21.15.143:0/2823133190 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:28:42.104 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:42.100+0000 7fbfb37fe700 1 --2- 172.21.15.143:0/2823133190 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbfb415aa30 0x7fbfb40cf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:42.104 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:42.100+0000 7fbfb3fff700 1 -- 172.21.15.143:0/2823133190 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbfb415aa30 msgr2=0x7fbfb40cf510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:42.104 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:42.100+0000 7fbfb3fff700 1 --2- 172.21.15.143:0/2823133190 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbfb415aa30 0x7fbfb40cf510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:42.104 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:42.100+0000 7fbfb3fff700 1 -- 172.21.15.143:0/2823133190 >> v1:172.21.15.174:6789/0 conn(0x7fbfb40b2210 legacy=0x7fbfb40cbe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:42.105 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:42.100+0000 7fbfb3fff700 1 -- 172.21.15.143:0/2823133190 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfb40ce750 con 0x7fbfb40cc530 2023-11-08T19:28:42.105 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:42.100+0000 7fbfb37fe700 1 --2- 172.21.15.143:0/2823133190 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbfb415aa30 0x7fbfb40cf510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:28:42.105 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:42.100+0000 7fbfb3fff700 1 --2- 172.21.15.143:0/2823133190 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbfb40cc530 0x7fbfb40b2690 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7fbfac009e50 tx=0x7fbfac00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ad06b46a7261dbf server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:42.105 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:42.100+0000 7fbfb27fc700 1 -- 172.21.15.143:0/2823133190 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfac00e070 con 0x7fbfb40cc530 2023-11-08T19:28:42.105 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:42.100+0000 7fbfb27fc700 1 -- 172.21.15.143:0/2823133190 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfac003a20 con 0x7fbfb40cc530 2023-11-08T19:28:42.106 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:42.100+0000 7fbfb27fc700 1 -- 172.21.15.143:0/2823133190 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfac003dd0 con 0x7fbfb40cc530 2023-11-08T19:28:42.106 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:42.101+0000 7fbfb9fa6700 1 -- 172.21.15.143:0/2823133190 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbfb40cc530 msgr2=0x7fbfb40b2690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:42.106 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:42.101+0000 7fbfb9fa6700 1 --2- 172.21.15.143:0/2823133190 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbfb40cc530 0x7fbfb40b2690 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7fbfac009e50 tx=0x7fbfac00b330 comp rx=0 tx=0).stop 2023-11-08T19:28:42.106 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:42.101+0000 7fbfb9fa6700 1 -- 172.21.15.143:0/2823133190 shutdown_connections 2023-11-08T19:28:42.106 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:42.101+0000 7fbfb9fa6700 1 --2- 172.21.15.143:0/2823133190 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbfb40cc530 0x7fbfb40b2690 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:42.107 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:42.101+0000 7fbfb9fa6700 1 --2- 172.21.15.143:0/2823133190 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbfb415aa30 0x7fbfb40cf510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:42.107 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:42.101+0000 7fbfb9fa6700 1 -- 172.21.15.143:0/2823133190 >> 172.21.15.143:0/2823133190 conn(0x7fbfb4099ab0 msgr2=0x7fbfb40d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:42.107 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:42.101+0000 7fbfb9fa6700 1 -- 172.21.15.143:0/2823133190 shutdown_connections 2023-11-08T19:28:42.107 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:42.101+0000 7fbfb9fa6700 1 -- 172.21.15.143:0/2823133190 wait complete. 2023-11-08T19:28:42.110 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:42.102+0000 7fbfb9fa6700 1 Processor -- start 2023-11-08T19:28:42.110 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:42.102+0000 7fbfb9fa6700 1 -- start start 2023-11-08T19:28:42.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:42.103+0000 7fbfb9fa6700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbfb40b2210 0x7fbfb40ad380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:42.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:42.103+0000 7fbfb9fa6700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbfb40cc530 0x7fbfb4159cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:42.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:42.103+0000 7fbfb9fa6700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbfb415aa30 0x7fbfb40b33b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:42.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:42.103+0000 7fbfb9fa6700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb40ad8c0 con 0x7fbfb415aa30 2023-11-08T19:28:42.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:42.103+0000 7fbfb9fa6700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb40c9f70 con 0x7fbfb40b2210 2023-11-08T19:28:42.112 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:42.103+0000 7fbfb9fa6700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb40ca250 con 0x7fbfb40cc530 2023-11-08T19:28:42.112 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:42.103+0000 7fbfb3fff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbfb415aa30 0x7fbfb40b33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:42.112 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:42.103+0000 7fbfb3fff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbfb415aa30 0x7fbfb40b33b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55116/0 (socket says 172.21.15.143:55116) 2023-11-08T19:28:42.112 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:42.103+0000 7fbfb3fff700 1 -- 172.21.15.143:0/3402901262 learned_addr learned my addr 172.21.15.143:0/3402901262 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:28:42.112 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:42.103+0000 7fbfb37fe700 1 --2- 172.21.15.143:0/3402901262 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbfb40b2210 0x7fbfb40ad380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:42.113 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:42.103+0000 7fbfb2ffd700 1 --2- 172.21.15.143:0/3402901262 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbfb40cc530 0x7fbfb4159cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:42.113 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:42.103+0000 7fbfb3fff700 1 -- 172.21.15.143:0/3402901262 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbfb40cc530 msgr2=0x7fbfb4159cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:42.113 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:42.103+0000 7fbfb3fff700 1 --2- 172.21.15.143:0/3402901262 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbfb40cc530 0x7fbfb4159cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:42.113 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:42.103+0000 7fbfb3fff700 1 -- 172.21.15.143:0/3402901262 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbfb40b2210 msgr2=0x7fbfb40ad380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:42.113 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:42.103+0000 7fbfb3fff700 1 --2- 172.21.15.143:0/3402901262 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbfb40b2210 0x7fbfb40ad380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:42.114 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:42.103+0000 7fbfb3fff700 1 -- 172.21.15.143:0/3402901262 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfac009b30 con 0x7fbfb415aa30 2023-11-08T19:28:42.114 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:42.103+0000 7fbfb3fff700 1 --2- 172.21.15.143:0/3402901262 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbfb415aa30 0x7fbfb40b33b0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7fbfb406eb50 tx=0x7fbfac004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:42.114 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:42.104+0000 7fbf9bfff700 1 -- 172.21.15.143:0/3402901262 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfac004440 con 0x7fbfb415aa30 2023-11-08T19:28:42.114 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:42.104+0000 7fbf9bfff700 1 -- 172.21.15.143:0/3402901262 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfac0045a0 con 0x7fbfb415aa30 2023-11-08T19:28:42.114 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:42.104+0000 7fbfb9fa6700 1 -- 172.21.15.143:0/3402901262 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfb40adc90 con 0x7fbfb415aa30 2023-11-08T19:28:42.115 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:42.104+0000 7fbf9bfff700 1 -- 172.21.15.143:0/3402901262 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfac004950 con 0x7fbfb415aa30 2023-11-08T19:28:42.115 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:42.104+0000 7fbfb9fa6700 1 -- 172.21.15.143:0/3402901262 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfb415a1f0 con 0x7fbfb415aa30 2023-11-08T19:28:42.115 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:42.105+0000 7fbfb9fa6700 1 -- 172.21.15.143:0/3402901262 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfb40ae1b0 con 0x7fbfb415aa30 2023-11-08T19:28:42.115 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:42.105+0000 7fbf9bfff700 1 -- 172.21.15.143:0/3402901262 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbfac012430 con 0x7fbfb415aa30 2023-11-08T19:28:42.117 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:42.105+0000 7fbf9bfff700 1 --2- 172.21.15.143:0/3402901262 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fbfa006af00 0x7fbfa006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:42.117 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:42.106+0000 7fbf9bfff700 1 -- 172.21.15.143:0/3402901262 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6801+0+0 (secure 0 0 0) 0x7fbfac08aef0 con 0x7fbfb415aa30 2023-11-08T19:28:42.118 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:42.106+0000 7fbfb37fe700 1 --2- 172.21.15.143:0/3402901262 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fbfa006af00 0x7fbfa006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:42.118 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:42.106+0000 7fbfb37fe700 1 --2- 172.21.15.143:0/3402901262 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fbfa006af00 0x7fbfa006d3c0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fbfa800a970 tx=0x7fbfa8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:42.119 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:42.110+0000 7fbf9bfff700 1 -- 172.21.15.143:0/3402901262 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbfb40ae1b0 con 0x7fbfb415aa30 2023-11-08T19:28:42.341 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:42.340+0000 7fbfb9fa6700 1 -- 172.21.15.143:0/3402901262 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbfb40ae1b0 con 0x7fbfb415aa30 2023-11-08T19:28:42.342 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:42.340+0000 7fbf9bfff700 1 -- 172.21.15.143:0/3402901262 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+4942 (secure 0 0 0) 0x7fbfb40ae1b0 con 0x7fbfb415aa30 2023-11-08T19:28:42.343 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:28:42.343 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":57,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6906,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6834/3172664377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":3172664377},{"type":"v1","addr":"172.21.15.143:6834","nonce":3172664377}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47},{"gid":6909,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3953059900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":3953059900},{"type":"v1","addr":"172.21.15.143:6835","nonce":3953059900}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":46}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:03.092245+0000","modified":"2023-11-08T19:28:29.816138+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6921,"mds_1":6918},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6918":{"gid":6918,"name":"b","rank":1,"incarnation":54,"state":"up:active","state_seq":6,"addr":"172.21.15.174:6836/3601088618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3601088618},{"type":"v1","addr":"172.21.15.174:6836","nonce":3601088618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6921":{"gid":6921,"name":"d","rank":0,"incarnation":48,"state":"up:active","state_seq":3,"addr":"172.21.15.174:6837/2201447629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6835","nonce":2201447629},{"type":"v1","addr":"172.21.15.174:6837","nonce":2201447629}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:28:42.345 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:42.343+0000 7fbfb9fa6700 1 -- 172.21.15.143:0/3402901262 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fbfa006af00 msgr2=0x7fbfa006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:42.345 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:42.343+0000 7fbfb9fa6700 1 --2- 172.21.15.143:0/3402901262 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fbfa006af00 0x7fbfa006d3c0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fbfa800a970 tx=0x7fbfa8008040 comp rx=0 tx=0).stop 2023-11-08T19:28:42.345 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:42.343+0000 7fbfb9fa6700 1 -- 172.21.15.143:0/3402901262 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbfb415aa30 msgr2=0x7fbfb40b33b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:42.345 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:42.343+0000 7fbfb9fa6700 1 --2- 172.21.15.143:0/3402901262 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbfb415aa30 0x7fbfb40b33b0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7fbfb406eb50 tx=0x7fbfac004ad0 comp rx=0 tx=0).stop 2023-11-08T19:28:42.345 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:42.343+0000 7fbfb9fa6700 1 -- 172.21.15.143:0/3402901262 shutdown_connections 2023-11-08T19:28:42.346 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:42.343+0000 7fbfb9fa6700 1 --2- 172.21.15.143:0/3402901262 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fbfa006af00 0x7fbfa006d3c0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:42.346 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:42.343+0000 7fbfb9fa6700 1 --2- 172.21.15.143:0/3402901262 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbfb415aa30 0x7fbfb40b33b0 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:42.346 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:42.343+0000 7fbfb9fa6700 1 --2- 172.21.15.143:0/3402901262 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbfb40b2210 0x7fbfb40ad380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:42.346 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:42.343+0000 7fbfb9fa6700 1 --2- 172.21.15.143:0/3402901262 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbfb40cc530 0x7fbfb4159cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:42.347 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:42.343+0000 7fbfb9fa6700 1 -- 172.21.15.143:0/3402901262 >> 172.21.15.143:0/3402901262 conn(0x7fbfb4099ab0 msgr2=0x7fbfb40cc9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:42.348 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:42.343+0000 7fbfb9fa6700 1 -- 172.21.15.143:0/3402901262 shutdown_connections 2023-11-08T19:28:42.348 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:42.343+0000 7fbfb9fa6700 1 -- 172.21.15.143:0/3402901262 wait complete. 2023-11-08T19:28:42.349 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 57 2023-11-08T19:28:42.356 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2023-11-08T19:28:43.439 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~7s 2023-11-08T19:28:47.357 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:28:47.496 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:47.495+0000 7fd5784bd700 1 Processor -- start 2023-11-08T19:28:47.496 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:47.495+0000 7fd5784bd700 1 -- start start 2023-11-08T19:28:47.499 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:47.496+0000 7fd5784bd700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd57006dbb0 0x7fd57015fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:47.499 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:47.496+0000 7fd5784bd700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fd57006e9a0 0x7fd5700b3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:47.499 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:47.496+0000 7fd5784bd700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd57006bd50 con 0x7fd570072860 2023-11-08T19:28:47.499 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:47.496+0000 7fd5784bd700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5700cb1c0 con 0x7fd57006dbb0 2023-11-08T19:28:47.500 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:47.496+0000 7fd5784bd700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd57006bff0 con 0x7fd57006e9a0 2023-11-08T19:28:47.500 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:47.496+0000 7fd576259700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd57006dbb0 0x7fd57015fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:47.500 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:47.496+0000 7fd576259700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd57006dbb0 0x7fd57015fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55122/0 (socket says 172.21.15.143:55122) 2023-11-08T19:28:47.500 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:47.496+0000 7fd576259700 1 -- 172.21.15.143:0/3456165432 learned_addr learned my addr 172.21.15.143:0/3456165432 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:28:47.500 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:47.496+0000 7fd576a5a700 1 --2- 172.21.15.143:0/3456165432 >> v2:172.21.15.174:3300/0 conn(0x7fd57006e9a0 0x7fd5700b3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:47.501 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:47.496+0000 7fd576259700 1 -- 172.21.15.143:0/3456165432 >> v2:172.21.15.174:3300/0 conn(0x7fd57006e9a0 msgr2=0x7fd5700b3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:47.501 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:47.496+0000 7fd576259700 1 --2- 172.21.15.143:0/3456165432 >> v2:172.21.15.174:3300/0 conn(0x7fd57006e9a0 0x7fd5700b3240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:47.501 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:47.496+0000 7fd576259700 1 -- 172.21.15.143:0/3456165432 >> v1:172.21.15.174:6789/0 conn(0x7fd570072860 legacy=0x7fd57015ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:47.501 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:47.496+0000 7fd576259700 1 -- 172.21.15.143:0/3456165432 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd57006c2d0 con 0x7fd57006dbb0 2023-11-08T19:28:47.501 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:47.496+0000 7fd576a5a700 1 --2- 172.21.15.143:0/3456165432 >> v2:172.21.15.174:3300/0 conn(0x7fd57006e9a0 0x7fd5700b3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:28:47.502 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:47.496+0000 7fd576259700 1 --2- 172.21.15.143:0/3456165432 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd57006dbb0 0x7fd57015fa60 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7fd5700c38b0 tx=0x7fd56c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfbe8fc430aa924c server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:47.502 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:47.496+0000 7fd575257700 1 -- 172.21.15.143:0/3456165432 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd56c004230 con 0x7fd57006dbb0 2023-11-08T19:28:47.502 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:47.496+0000 7fd575257700 1 -- 172.21.15.143:0/3456165432 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd56c004390 con 0x7fd57006dbb0 2023-11-08T19:28:47.502 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:47.496+0000 7fd575257700 1 -- 172.21.15.143:0/3456165432 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd56c0036a0 con 0x7fd57006dbb0 2023-11-08T19:28:47.502 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:47.497+0000 7fd5784bd700 1 -- 172.21.15.143:0/3456165432 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd57006dbb0 msgr2=0x7fd57015fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:47.503 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:47.497+0000 7fd5784bd700 1 --2- 172.21.15.143:0/3456165432 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd57006dbb0 0x7fd57015fa60 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7fd5700c38b0 tx=0x7fd56c009bd0 comp rx=0 tx=0).stop 2023-11-08T19:28:47.503 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:47.497+0000 7fd5784bd700 1 -- 172.21.15.143:0/3456165432 shutdown_connections 2023-11-08T19:28:47.503 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:47.497+0000 7fd5784bd700 1 --2- 172.21.15.143:0/3456165432 >> v2:172.21.15.174:3300/0 conn(0x7fd57006e9a0 0x7fd5700b3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:47.503 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:47.497+0000 7fd5784bd700 1 --2- 172.21.15.143:0/3456165432 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd57006dbb0 0x7fd57015fa60 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:47.503 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:47.497+0000 7fd5784bd700 1 -- 172.21.15.143:0/3456165432 >> 172.21.15.143:0/3456165432 conn(0x7fd5700b1a20 msgr2=0x7fd5700b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:47.504 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:47.497+0000 7fd5784bd700 1 -- 172.21.15.143:0/3456165432 shutdown_connections 2023-11-08T19:28:47.504 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:47.497+0000 7fd5784bd700 1 -- 172.21.15.143:0/3456165432 wait complete. 2023-11-08T19:28:47.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:47.499+0000 7fd5784bd700 1 Processor -- start 2023-11-08T19:28:47.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:47.499+0000 7fd5784bd700 1 -- start start 2023-11-08T19:28:47.508 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:47.499+0000 7fd5784bd700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd57006dbb0 0x7fd5701566e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:47.508 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:47.499+0000 7fd5784bd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd57006e9a0 0x7fd570155b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:47.508 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:47.499+0000 7fd5784bd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd570072860 0x7fd57015b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:47.508 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:47.499+0000 7fd5784bd700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd57015f7a0 con 0x7fd570072860 2023-11-08T19:28:47.508 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:47.499+0000 7fd5784bd700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd57015ec20 con 0x7fd57006e9a0 2023-11-08T19:28:47.509 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:47.499+0000 7fd5784bd700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd57015ef00 con 0x7fd57006dbb0 2023-11-08T19:28:47.509 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:47.500+0000 7fd576a5a700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd570072860 0x7fd57015b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:47.509 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:47.500+0000 7fd576a5a700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd570072860 0x7fd57015b5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55132/0 (socket says 172.21.15.143:55132) 2023-11-08T19:28:47.509 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:47.500+0000 7fd576a5a700 1 -- 172.21.15.143:0/388865906 learned_addr learned my addr 172.21.15.143:0/388865906 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:28:47.509 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:47.500+0000 7fd576259700 1 --2- 172.21.15.143:0/388865906 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd57006dbb0 0x7fd5701566e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:47.510 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:47.500+0000 7fd575a58700 1 --2- 172.21.15.143:0/388865906 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd57006e9a0 0x7fd570155b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:47.510 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:47.500+0000 7fd576a5a700 1 -- 172.21.15.143:0/388865906 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd57006dbb0 msgr2=0x7fd5701566e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:47.510 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:47.500+0000 7fd576a5a700 1 --2- 172.21.15.143:0/388865906 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd57006dbb0 0x7fd5701566e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:47.510 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:47.500+0000 7fd576a5a700 1 -- 172.21.15.143:0/388865906 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd57006e9a0 msgr2=0x7fd570155b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:47.510 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:47.500+0000 7fd576a5a700 1 --2- 172.21.15.143:0/388865906 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd57006e9a0 0x7fd570155b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:47.511 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:47.500+0000 7fd576a5a700 1 -- 172.21.15.143:0/388865906 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd56c009550 con 0x7fd570072860 2023-11-08T19:28:47.511 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:47.500+0000 7fd576259700 1 --2- 172.21.15.143:0/388865906 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd57006dbb0 0x7fd5701566e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:28:47.511 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:47.500+0000 7fd575a58700 1 --2- 172.21.15.143:0/388865906 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd57006e9a0 0x7fd570155b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:28:47.511 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:47.500+0000 7fd576a5a700 1 --2- 172.21.15.143:0/388865906 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd570072860 0x7fd57015b5f0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fd56800cef0 tx=0x7fd56800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:47.511 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:47.500+0000 7fd566ffd700 1 -- 172.21.15.143:0/388865906 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd56800ae50 con 0x7fd570072860 2023-11-08T19:28:47.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:47.500+0000 7fd566ffd700 1 -- 172.21.15.143:0/388865906 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd568015440 con 0x7fd570072860 2023-11-08T19:28:47.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:47.500+0000 7fd5784bd700 1 -- 172.21.15.143:0/388865906 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd57015f1e0 con 0x7fd570072860 2023-11-08T19:28:47.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:47.500+0000 7fd566ffd700 1 -- 172.21.15.143:0/388865906 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5680157f0 con 0x7fd570072860 2023-11-08T19:28:47.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:47.500+0000 7fd5784bd700 1 -- 172.21.15.143:0/388865906 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd57015e7f0 con 0x7fd570072860 2023-11-08T19:28:47.513 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:47.501+0000 7fd5784bd700 1 -- 172.21.15.143:0/388865906 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5700c4260 con 0x7fd570072860 2023-11-08T19:28:47.513 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:47.502+0000 7fd566ffd700 1 -- 172.21.15.143:0/388865906 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd56800d430 con 0x7fd570072860 2023-11-08T19:28:47.513 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:47.502+0000 7fd566ffd700 1 --2- 172.21.15.143:0/388865906 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd55406af00 0x7fd55406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:47.513 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:47.502+0000 7fd566ffd700 1 -- 172.21.15.143:0/388865906 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6801+0+0 (secure 0 0 0) 0x7fd56808a090 con 0x7fd570072860 2023-11-08T19:28:47.513 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:47.502+0000 7fd576259700 1 --2- 172.21.15.143:0/388865906 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd55406af00 0x7fd55406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:47.514 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:47.503+0000 7fd576259700 1 --2- 172.21.15.143:0/388865906 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd55406af00 0x7fd55406d3c0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fd56c009c10 tx=0x7fd56c008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:47.514 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:47.507+0000 7fd566ffd700 1 -- 172.21.15.143:0/388865906 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd5700c4260 con 0x7fd570072860 2023-11-08T19:28:47.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:47.740+0000 7fd5784bd700 1 -- 172.21.15.143:0/388865906 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd5700c4260 con 0x7fd570072860 2023-11-08T19:28:47.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:47.741+0000 7fd566ffd700 1 -- 172.21.15.143:0/388865906 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+4990 (secure 0 0 0) 0x7fd5700c4260 con 0x7fd570072860 2023-11-08T19:28:47.743 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:28:47.743 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":58,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6906,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6834/3172664377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":3172664377},{"type":"v1","addr":"172.21.15.143:6834","nonce":3172664377}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47},{"gid":6909,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3953059900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":3953059900},{"type":"v1","addr":"172.21.15.143:6835","nonce":3953059900}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":46}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:03.092245+0000","modified":"2023-11-08T19:28:47.598672+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6921,"mds_1":6918},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6918":{"gid":6918,"name":"b","rank":1,"incarnation":54,"state":"up:active","state_seq":6,"addr":"172.21.15.174:6836/3601088618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3601088618},{"type":"v1","addr":"172.21.15.174:6836","nonce":3601088618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6921":{"gid":6921,"name":"d","rank":0,"incarnation":48,"state":"up:active","state_seq":3,"addr":"172.21.15.174:6837/2201447629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6835","nonce":2201447629},{"type":"v1","addr":"172.21.15.174:6837","nonce":2201447629}]},"join_fscid":-1,"laggy_since":"2023-11-08T19:28:47.598672+0000","export_targets":[1],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:28:47.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:47.743+0000 7fd5784bd700 1 -- 172.21.15.143:0/388865906 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd55406af00 msgr2=0x7fd55406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:47.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:47.743+0000 7fd5784bd700 1 --2- 172.21.15.143:0/388865906 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd55406af00 0x7fd55406d3c0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fd56c009c10 tx=0x7fd56c008030 comp rx=0 tx=0).stop 2023-11-08T19:28:47.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:47.743+0000 7fd5784bd700 1 -- 172.21.15.143:0/388865906 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd570072860 msgr2=0x7fd57015b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:47.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:47.743+0000 7fd5784bd700 1 --2- 172.21.15.143:0/388865906 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd570072860 0x7fd57015b5f0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fd56800cef0 tx=0x7fd56800a340 comp rx=0 tx=0).stop 2023-11-08T19:28:47.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:47.743+0000 7fd5784bd700 1 -- 172.21.15.143:0/388865906 shutdown_connections 2023-11-08T19:28:47.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:47.743+0000 7fd5784bd700 1 --2- 172.21.15.143:0/388865906 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd55406af00 0x7fd55406d3c0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:47.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:47.743+0000 7fd5784bd700 1 --2- 172.21.15.143:0/388865906 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd57006dbb0 0x7fd5701566e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:47.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:47.743+0000 7fd5784bd700 1 --2- 172.21.15.143:0/388865906 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd570072860 0x7fd57015b5f0 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:47.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:47.743+0000 7fd5784bd700 1 --2- 172.21.15.143:0/388865906 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd57006e9a0 0x7fd570155b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:47.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:47.743+0000 7fd5784bd700 1 -- 172.21.15.143:0/388865906 >> 172.21.15.143:0/388865906 conn(0x7fd5700b1a20 msgr2=0x7fd57015a940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:47.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:47.743+0000 7fd5784bd700 1 -- 172.21.15.143:0/388865906 shutdown_connections 2023-11-08T19:28:47.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:47.743+0000 7fd5784bd700 1 -- 172.21.15.143:0/388865906 wait complete. 2023-11-08T19:28:47.748 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 58 2023-11-08T19:28:47.756 DEBUG:tasks.ceph_test_case:wait_until_true: success in 15s and 0 retries 2023-11-08T19:28:47.757 DEBUG:teuthology.orchestra.run.smithi174:> sudo sysctl -n kernel.core_pattern 2023-11-08T19:28:47.789 INFO:teuthology.orchestra.run.smithi174.stdout:/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-08T19:28:47.790 INFO:tasks.cephfs.cephfs_test_case:Clearing core from directory: /home/ubuntu/cephtest/archive/coredump 2023-11-08T19:28:47.791 DEBUG:teuthology.orchestra.run.smithi174:> cd /home/ubuntu/cephtest/archive/coredump && sudo ls | sudo xargs file 2023-11-08T19:28:47.865 INFO:teuthology.orchestra.run.smithi174.stdout:1699471711.113265.core: ELF 64-bit LSB core file, x86-64, version 1 (SYSV), SVR4-style, from 'ceph-mds -f --cluster ceph -i d', real uid: 0, effective uid: 0, real gid: 0, effective gid: 0, execfn: '/bin/ceph-mds', platform: 'x86_64' 2023-11-08T19:28:47.867 INFO:tasks.cephfs.cephfs_test_case:Enumerated cores: ['1699471711.113265.core'] 2023-11-08T19:28:47.867 INFO:tasks.cephfs.cephfs_test_case:Found core file 1699471711.113265.core, deleting it 2023-11-08T19:28:47.868 DEBUG:teuthology.orchestra.run.smithi174:> cd /home/ubuntu/cephtest/archive/coredump && sudo rm -f 1699471711.113265.core 2023-11-08T19:28:47.940 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail 6:0 2023-11-08T19:28:48.064 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.061+0000 7f2ad35ab700 1 Processor -- start 2023-11-08T19:28:48.065 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.061+0000 7f2ad35ab700 1 -- start start 2023-11-08T19:28:48.065 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.062+0000 7f2ad35ab700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f2acc0b2920 0x7f2acc0cf510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:48.065 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.062+0000 7f2ad35ab700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2acc0cbe20 0x7f2acc0cc290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:48.065 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.062+0000 7f2ad35ab700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2acc0cf080 con 0x7f2acc15aa30 2023-11-08T19:28:48.066 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.062+0000 7f2ad35ab700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2acc0cc7d0 con 0x7f2acc0b2920 2023-11-08T19:28:48.066 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.062+0000 7f2ad35ab700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2acc0ce470 con 0x7f2acc0cbe20 2023-11-08T19:28:48.066 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.062+0000 7f2ad0b46700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f2acc0b2920 0x7f2acc0cf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:48.066 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.062+0000 7f2ad0b46700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f2acc0b2920 0x7f2acc0cf510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55138/0 (socket says 172.21.15.143:55138) 2023-11-08T19:28:48.066 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.062+0000 7f2ad0b46700 1 -- 172.21.15.143:0/3639486241 learned_addr learned my addr 172.21.15.143:0/3639486241 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:28:48.067 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.062+0000 7f2ad1b48700 1 --2- 172.21.15.143:0/3639486241 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2acc0cbe20 0x7f2acc0cc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:48.067 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.062+0000 7f2ad0b46700 1 -- 172.21.15.143:0/3639486241 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2acc0cbe20 msgr2=0x7f2acc0cc290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:48.067 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.062+0000 7f2ad0b46700 1 --2- 172.21.15.143:0/3639486241 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2acc0cbe20 0x7f2acc0cc290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:48.067 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.062+0000 7f2ad0b46700 1 -- 172.21.15.143:0/3639486241 >> v1:172.21.15.174:6789/0 conn(0x7f2acc15aa30 legacy=0x7f2acc0b2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:48.067 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.062+0000 7f2ad0b46700 1 -- 172.21.15.143:0/3639486241 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2acc0ce750 con 0x7f2acc0b2920 2023-11-08T19:28:48.068 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.062+0000 7f2ad1b48700 1 --2- 172.21.15.143:0/3639486241 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2acc0cbe20 0x7f2acc0cc290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:28:48.068 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.062+0000 7f2ad0b46700 1 --2- 172.21.15.143:0/3639486241 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2acc0b2920 0x7f2acc0cf510 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f2abc00bb90 tx=0x7f2abc00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d1ebbe9450c4486 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:48.068 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.062+0000 7f2ac3fff700 1 -- 172.21.15.143:0/3639486241 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2abc00dd50 con 0x7f2acc0b2920 2023-11-08T19:28:48.068 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.063+0000 7f2ac3fff700 1 -- 172.21.15.143:0/3639486241 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2abc010470 con 0x7f2acc0b2920 2023-11-08T19:28:48.068 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.063+0000 7f2ac3fff700 1 -- 172.21.15.143:0/3639486241 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2abc010800 con 0x7f2acc0b2920 2023-11-08T19:28:48.069 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.063+0000 7f2ad35ab700 1 -- 172.21.15.143:0/3639486241 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2acc0b2920 msgr2=0x7f2acc0cf510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:48.069 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.063+0000 7f2ad35ab700 1 --2- 172.21.15.143:0/3639486241 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2acc0b2920 0x7f2acc0cf510 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f2abc00bb90 tx=0x7f2abc00bea0 comp rx=0 tx=0).stop 2023-11-08T19:28:48.069 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.063+0000 7f2ad35ab700 1 -- 172.21.15.143:0/3639486241 shutdown_connections 2023-11-08T19:28:48.069 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.063+0000 7f2ad35ab700 1 --2- 172.21.15.143:0/3639486241 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2acc0cbe20 0x7f2acc0cc290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:48.069 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.063+0000 7f2ad35ab700 1 --2- 172.21.15.143:0/3639486241 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2acc0b2920 0x7f2acc0cf510 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:48.070 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.063+0000 7f2ad35ab700 1 -- 172.21.15.143:0/3639486241 >> 172.21.15.143:0/3639486241 conn(0x7f2acc099ab0 msgr2=0x7f2acc0d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:48.070 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.063+0000 7f2ad35ab700 1 -- 172.21.15.143:0/3639486241 shutdown_connections 2023-11-08T19:28:48.070 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.063+0000 7f2ad35ab700 1 -- 172.21.15.143:0/3639486241 wait complete. 2023-11-08T19:28:48.074 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.064+0000 7f2ad35ab700 1 Processor -- start 2023-11-08T19:28:48.074 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.064+0000 7f2ad35ab700 1 -- start start 2023-11-08T19:28:48.074 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.065+0000 7f2ad35ab700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2acc0b2920 0x7f2acc146d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:48.075 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.065+0000 7f2ad35ab700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2acc0cbe20 0x7f2acc1f28d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:48.075 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.065+0000 7f2ad35ab700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2acc15aa30 0x7f2acc1f2e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:48.075 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.065+0000 7f2ad35ab700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2acc15e0a0 con 0x7f2acc0cbe20 2023-11-08T19:28:48.076 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.065+0000 7f2ad35ab700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2acc15d490 con 0x7f2acc15aa30 2023-11-08T19:28:48.076 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.065+0000 7f2ad35ab700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2acc15d770 con 0x7f2acc0b2920 2023-11-08T19:28:48.077 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.065+0000 7f2ad0b46700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2acc0cbe20 0x7f2acc1f28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:48.077 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.065+0000 7f2ad0b46700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2acc0cbe20 0x7f2acc1f28d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55144/0 (socket says 172.21.15.143:55144) 2023-11-08T19:28:48.077 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.065+0000 7f2ad0b46700 1 -- 172.21.15.143:0/3099971686 learned_addr learned my addr 172.21.15.143:0/3099971686 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:28:48.077 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.065+0000 7f2ad1b48700 1 --2- 172.21.15.143:0/3099971686 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2acc15aa30 0x7f2acc1f2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:48.077 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.065+0000 7f2ad1347700 1 --2- 172.21.15.143:0/3099971686 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2acc0b2920 0x7f2acc146d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:48.078 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.065+0000 7f2ad0b46700 1 -- 172.21.15.143:0/3099971686 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2acc0b2920 msgr2=0x7f2acc146d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:48.078 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.065+0000 7f2ad0b46700 1 --2- 172.21.15.143:0/3099971686 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2acc0b2920 0x7f2acc146d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:48.078 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.065+0000 7f2ad0b46700 1 -- 172.21.15.143:0/3099971686 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2acc15aa30 msgr2=0x7f2acc1f2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:48.078 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.065+0000 7f2ad0b46700 1 --2- 172.21.15.143:0/3099971686 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2acc15aa30 0x7f2acc1f2e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:48.078 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.065+0000 7f2ad0b46700 1 -- 172.21.15.143:0/3099971686 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2abc00b7f0 con 0x7f2acc0cbe20 2023-11-08T19:28:48.079 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.065+0000 7f2ad1b48700 1 --2- 172.21.15.143:0/3099971686 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2acc15aa30 0x7f2acc1f2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:28:48.079 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.065+0000 7f2ad1347700 1 --2- 172.21.15.143:0/3099971686 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2acc0b2920 0x7f2acc146d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:28:48.079 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.066+0000 7f2ad0b46700 1 --2- 172.21.15.143:0/3099971686 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2acc0cbe20 0x7f2acc1f28d0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f2abc00bee0 tx=0x7f2abc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:48.079 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.066+0000 7f2ac1ffb700 1 -- 172.21.15.143:0/3099971686 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2abc01f070 con 0x7f2acc0cbe20 2023-11-08T19:28:48.079 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.066+0000 7f2ac1ffb700 1 -- 172.21.15.143:0/3099971686 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2abc0038b0 con 0x7f2acc0cbe20 2023-11-08T19:28:48.080 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.066+0000 7f2ac1ffb700 1 -- 172.21.15.143:0/3099971686 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2abc003c60 con 0x7f2acc0cbe20 2023-11-08T19:28:48.080 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.066+0000 7f2ad35ab700 1 -- 172.21.15.143:0/3099971686 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2acc15da50 con 0x7f2acc0cbe20 2023-11-08T19:28:48.080 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.066+0000 7f2ad35ab700 1 -- 172.21.15.143:0/3099971686 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2acc147260 con 0x7f2acc0cbe20 2023-11-08T19:28:48.080 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.067+0000 7f2ad35ab700 1 -- 172.21.15.143:0/3099971686 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2acc0c4230 con 0x7f2acc0cbe20 2023-11-08T19:28:48.080 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.067+0000 7f2ac1ffb700 1 -- 172.21.15.143:0/3099971686 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2abc019400 con 0x7f2acc0cbe20 2023-11-08T19:28:48.081 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.068+0000 7f2ac1ffb700 1 --2- 172.21.15.143:0/3099971686 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f2ab006af00 0x7f2ab006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:48.081 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.068+0000 7f2ac1ffb700 1 -- 172.21.15.143:0/3099971686 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6801+0+0 (secure 0 0 0) 0x7f2abc08d720 con 0x7f2acc0cbe20 2023-11-08T19:28:48.081 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.068+0000 7f2ad1347700 1 --2- 172.21.15.143:0/3099971686 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f2ab006af00 0x7f2ab006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:48.081 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.068+0000 7f2ad1347700 1 --2- 172.21.15.143:0/3099971686 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f2ab006af00 0x7f2ab006d3c0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f2ac8000fb0 tx=0x7f2ac8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:48.081 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.072+0000 7f2ac1ffb700 1 -- 172.21.15.143:0/3099971686 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2acc0c4230 con 0x7f2acc0cbe20 2023-11-08T19:28:48.342 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.341+0000 7f2ad35ab700 1 -- 172.21.15.143:0/3099971686 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "6:0"} v 0) v1 -- 0x7f2acc0c4230 con 0x7f2acc0cbe20 2023-11-08T19:28:48.344 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.343+0000 7f2ac1ffb700 1 -- 172.21.15.143:0/3099971686 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "6:0"}]=0 failed mds gid 6921 v59) v1 ==== 97+0+0 (secure 0 0 0) 0x7f2acc0c4230 con 0x7f2acc0cbe20 2023-11-08T19:28:48.347 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.345+0000 7f2ad35ab700 1 -- 172.21.15.143:0/3099971686 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f2ab006af00 msgr2=0x7f2ab006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:48.347 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.345+0000 7f2ad35ab700 1 --2- 172.21.15.143:0/3099971686 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f2ab006af00 0x7f2ab006d3c0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f2ac8000fb0 tx=0x7f2ac8009040 comp rx=0 tx=0).stop 2023-11-08T19:28:48.348 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.346+0000 7f2ad35ab700 1 -- 172.21.15.143:0/3099971686 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2acc0cbe20 msgr2=0x7f2acc1f28d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:48.348 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.346+0000 7f2ad35ab700 1 --2- 172.21.15.143:0/3099971686 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2acc0cbe20 0x7f2acc1f28d0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f2abc00bee0 tx=0x7f2abc004750 comp rx=0 tx=0).stop 2023-11-08T19:28:48.348 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.346+0000 7f2ad35ab700 1 -- 172.21.15.143:0/3099971686 shutdown_connections 2023-11-08T19:28:48.348 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.346+0000 7f2ad35ab700 1 --2- 172.21.15.143:0/3099971686 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f2ab006af00 0x7f2ab006d3c0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:48.348 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.346+0000 7f2ad35ab700 1 --2- 172.21.15.143:0/3099971686 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2acc0b2920 0x7f2acc146d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:48.349 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.346+0000 7f2ad35ab700 1 --2- 172.21.15.143:0/3099971686 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2acc15aa30 0x7f2acc1f2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:48.349 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.346+0000 7f2ad35ab700 1 --2- 172.21.15.143:0/3099971686 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2acc0cbe20 0x7f2acc1f28d0 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:48.349 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.346+0000 7f2ad35ab700 1 -- 172.21.15.143:0/3099971686 >> 172.21.15.143:0/3099971686 conn(0x7f2acc099ab0 msgr2=0x7f2acc0cda10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:48.349 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.346+0000 7f2ad35ab700 1 -- 172.21.15.143:0/3099971686 shutdown_connections 2023-11-08T19:28:48.349 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.346+0000 7f2ad35ab700 1 -- 172.21.15.143:0/3099971686 wait complete. 2023-11-08T19:28:48.350 INFO:teuthology.orchestra.run.smithi143.stderr:failed mds gid 6921 2023-11-08T19:28:48.360 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-08T19:28:48.360 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-08T19:28:48.361 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-08T19:28:48.361 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T19:28:48.362 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-08T19:28:48.362 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi174 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d' 2023-11-08T19:28:48.363 INFO:tasks.ceph.mds.d:Stopped 2023-11-08T19:28:48.363 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-08T19:28:48.366 INFO:tasks.ceph.mds.d:Started 2023-11-08T19:28:48.366 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:28:48.448 INFO:tasks.ceph.mds.d.smithi174.stdout:starting mds.d at 2023-11-08T19:28:48.487 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.485+0000 7fb61a3bd700 1 Processor -- start 2023-11-08T19:28:48.487 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.485+0000 7fb61a3bd700 1 -- start start 2023-11-08T19:28:48.489 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.486+0000 7fb61a3bd700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb61415dd00 0x7fb6140cca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:48.490 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.486+0000 7fb61a3bd700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb6141618f0 0x7fb61406ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:48.490 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.486+0000 7fb61a3bd700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6140ccfb0 con 0x7fb61406b150 2023-11-08T19:28:48.490 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.486+0000 7fb61a3bd700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb61406b600 con 0x7fb6141618f0 2023-11-08T19:28:48.490 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.486+0000 7fb61a3bd700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6140cad50 con 0x7fb61415dd00 2023-11-08T19:28:48.490 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.486+0000 7fb6137fe700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb6141618f0 0x7fb61406ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:48.491 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.486+0000 7fb61895a700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fb61406b150 0x7fb6140b2210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:51322/0 (socket says 172.21.15.143:51322) 2023-11-08T19:28:48.491 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.486+0000 7fb61895a700 1 -- 172.21.15.143:0/271453730 learned_addr learned my addr 172.21.15.143:0/271453730 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:28:48.491 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.486+0000 7fb613fff700 1 --2- 172.21.15.143:0/271453730 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb61415dd00 0x7fb6140cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:48.491 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.487+0000 7fb6137fe700 1 -- 172.21.15.143:0/271453730 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb61415dd00 msgr2=0x7fb6140cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:48.491 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.487+0000 7fb6137fe700 1 --2- 172.21.15.143:0/271453730 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb61415dd00 0x7fb6140cca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:48.492 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.487+0000 7fb6137fe700 1 -- 172.21.15.143:0/271453730 >> v1:172.21.15.143:6789/0 conn(0x7fb61406b150 legacy=0x7fb6140b2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:48.492 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.487+0000 7fb612ffd700 1 -- 172.21.15.143:0/271453730 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4143581376 0 0) 0x7fb6140ccfb0 con 0x7fb61406b150 2023-11-08T19:28:48.492 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.487+0000 7fb6137fe700 1 -- 172.21.15.143:0/271453730 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb614161df0 con 0x7fb6141618f0 2023-11-08T19:28:48.492 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.487+0000 7fb6137fe700 1 --2- 172.21.15.143:0/271453730 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb6141618f0 0x7fb61406ac10 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fb604009840 tx=0x7fb604009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b35be69b5f95009e server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:48.492 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.487+0000 7fb612ffd700 1 -- 172.21.15.143:0/271453730 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb604011830 con 0x7fb6141618f0 2023-11-08T19:28:48.493 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.487+0000 7fb612ffd700 1 -- 172.21.15.143:0/271453730 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb604011990 con 0x7fb6141618f0 2023-11-08T19:28:48.493 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.487+0000 7fb612ffd700 1 -- 172.21.15.143:0/271453730 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb604011d20 con 0x7fb6141618f0 2023-11-08T19:28:48.493 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.487+0000 7fb61a3bd700 1 -- 172.21.15.143:0/271453730 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb6141618f0 msgr2=0x7fb61406ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:48.493 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.487+0000 7fb61a3bd700 1 --2- 172.21.15.143:0/271453730 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb6141618f0 0x7fb61406ac10 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fb604009840 tx=0x7fb604009b50 comp rx=0 tx=0).stop 2023-11-08T19:28:48.493 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.487+0000 7fb61a3bd700 1 -- 172.21.15.143:0/271453730 shutdown_connections 2023-11-08T19:28:48.494 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.488+0000 7fb61a3bd700 1 --2- 172.21.15.143:0/271453730 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb61415dd00 0x7fb6140cca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:48.494 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.488+0000 7fb61a3bd700 1 --2- 172.21.15.143:0/271453730 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb6141618f0 0x7fb61406ac10 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:48.494 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.488+0000 7fb61a3bd700 1 -- 172.21.15.143:0/271453730 >> 172.21.15.143:0/271453730 conn(0x7fb614099ab0 msgr2=0x7fb6140d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:48.494 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.488+0000 7fb61a3bd700 1 -- 172.21.15.143:0/271453730 shutdown_connections 2023-11-08T19:28:48.494 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.488+0000 7fb61a3bd700 1 -- 172.21.15.143:0/271453730 wait complete. 2023-11-08T19:28:48.501 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.489+0000 7fb61a3bd700 1 Processor -- start 2023-11-08T19:28:48.501 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.489+0000 7fb61a3bd700 1 -- start start 2023-11-08T19:28:48.501 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.490+0000 7fb61a3bd700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb61406b150 0x7fb6140c5d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:48.502 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.490+0000 7fb61a3bd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb61415dd00 0x7fb614157370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:48.502 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.490+0000 7fb61a3bd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb6141618f0 0x7fb6140b33b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:48.502 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.490+0000 7fb61a3bd700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6141578b0 con 0x7fb61415dd00 2023-11-08T19:28:48.502 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.490+0000 7fb61a3bd700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6141582a0 con 0x7fb6141618f0 2023-11-08T19:28:48.503 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.490+0000 7fb61a3bd700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6140c46d0 con 0x7fb61406b150 2023-11-08T19:28:48.503 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.490+0000 7fb6137fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb61415dd00 0x7fb614157370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:48.503 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.490+0000 7fb6137fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb61415dd00 0x7fb614157370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55172/0 (socket says 172.21.15.143:55172) 2023-11-08T19:28:48.503 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.490+0000 7fb6137fe700 1 -- 172.21.15.143:0/1779096370 learned_addr learned my addr 172.21.15.143:0/1779096370 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:28:48.503 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.490+0000 7fb61895a700 1 --2- 172.21.15.143:0/1779096370 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb6141618f0 0x7fb6140b33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:48.503 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.490+0000 7fb613fff700 1 --2- 172.21.15.143:0/1779096370 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb61406b150 0x7fb6140c5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:48.504 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.490+0000 7fb6137fe700 1 -- 172.21.15.143:0/1779096370 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb61406b150 msgr2=0x7fb6140c5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:48.504 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.490+0000 7fb6137fe700 1 --2- 172.21.15.143:0/1779096370 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb61406b150 0x7fb6140c5d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:48.504 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.490+0000 7fb6137fe700 1 -- 172.21.15.143:0/1779096370 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb6141618f0 msgr2=0x7fb6140b33b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:48.504 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.490+0000 7fb6137fe700 1 --2- 172.21.15.143:0/1779096370 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb6141618f0 0x7fb6140b33b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:48.504 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.490+0000 7fb6137fe700 1 -- 172.21.15.143:0/1779096370 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6040094a0 con 0x7fb61415dd00 2023-11-08T19:28:48.505 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.490+0000 7fb6137fe700 1 --2- 172.21.15.143:0/1779096370 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb61415dd00 0x7fb614157370 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fb604009fd0 tx=0x7fb604004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:48.505 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.490+0000 7fb610ff9700 1 -- 172.21.15.143:0/1779096370 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6040036a0 con 0x7fb61415dd00 2023-11-08T19:28:48.505 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.490+0000 7fb610ff9700 1 -- 172.21.15.143:0/1779096370 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb60401e070 con 0x7fb61415dd00 2023-11-08T19:28:48.505 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.490+0000 7fb610ff9700 1 -- 172.21.15.143:0/1779096370 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb604003a60 con 0x7fb61415dd00 2023-11-08T19:28:48.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.490+0000 7fb61a3bd700 1 -- 172.21.15.143:0/1779096370 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6140c6470 con 0x7fb61415dd00 2023-11-08T19:28:48.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.490+0000 7fb61a3bd700 1 -- 172.21.15.143:0/1779096370 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6140c4360 con 0x7fb61415dd00 2023-11-08T19:28:48.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.491+0000 7fb61a3bd700 1 -- 172.21.15.143:0/1779096370 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6140aff60 con 0x7fb61415dd00 2023-11-08T19:28:48.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.492+0000 7fb610ff9700 1 -- 172.21.15.143:0/1779096370 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb604003bc0 con 0x7fb61415dd00 2023-11-08T19:28:48.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.492+0000 7fb610ff9700 1 --2- 172.21.15.143:0/1779096370 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fb60006af00 0x7fb60006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:48.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.492+0000 7fb610ff9700 1 -- 172.21.15.143:0/1779096370 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6887+0+0 (secure 0 0 0) 0x7fb60408a900 con 0x7fb61415dd00 2023-11-08T19:28:48.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.493+0000 7fb613fff700 1 --2- 172.21.15.143:0/1779096370 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fb60006af00 0x7fb60006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:48.508 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.493+0000 7fb613fff700 1 --2- 172.21.15.143:0/1779096370 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fb60006af00 0x7fb60006d3c0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fb608009bc0 tx=0x7fb6080091c0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:48.508 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.497+0000 7fb610ff9700 1 -- 172.21.15.143:0/1779096370 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb604055520 con 0x7fb61415dd00 2023-11-08T19:28:48.730 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.729+0000 7fb61a3bd700 1 -- 172.21.15.143:0/1779096370 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb6140c4840 con 0x7fb61415dd00 2023-11-08T19:28:48.731 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.730+0000 7fb610ff9700 1 -- 172.21.15.143:0/1779096370 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) v1 ==== 94+0+4148 (secure 0 0 0) 0x7fb604058b40 con 0x7fb61415dd00 2023-11-08T19:28:48.731 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:28:48.732 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":60,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6906,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6834/3172664377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":3172664377},{"type":"v1","addr":"172.21.15.143:6834","nonce":3172664377}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:03.092245+0000","modified":"2023-11-08T19:28:48.346449+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":95,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6909,"mds_1":6918},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6909":{"gid":6909,"name":"a","rank":0,"incarnation":60,"state":"up:replay","state_seq":1,"addr":"172.21.15.143:6835/3953059900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":3953059900},{"type":"v1","addr":"172.21.15.143:6835","nonce":3953059900}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6918":{"gid":6918,"name":"b","rank":1,"incarnation":54,"state":"up:active","state_seq":6,"addr":"172.21.15.174:6836/3601088618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3601088618},{"type":"v1","addr":"172.21.15.174:6836","nonce":3601088618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:28:48.733 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.732+0000 7fb61a3bd700 1 -- 172.21.15.143:0/1779096370 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fb60006af00 msgr2=0x7fb60006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:48.733 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.732+0000 7fb61a3bd700 1 --2- 172.21.15.143:0/1779096370 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fb60006af00 0x7fb60006d3c0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fb608009bc0 tx=0x7fb6080091c0 comp rx=0 tx=0).stop 2023-11-08T19:28:48.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.732+0000 7fb61a3bd700 1 -- 172.21.15.143:0/1779096370 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb61415dd00 msgr2=0x7fb614157370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:48.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.732+0000 7fb61a3bd700 1 --2- 172.21.15.143:0/1779096370 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb61415dd00 0x7fb614157370 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fb604009fd0 tx=0x7fb604004750 comp rx=0 tx=0).stop 2023-11-08T19:28:48.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.732+0000 7fb61a3bd700 1 -- 172.21.15.143:0/1779096370 shutdown_connections 2023-11-08T19:28:48.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.732+0000 7fb61a3bd700 1 --2- 172.21.15.143:0/1779096370 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fb60006af00 0x7fb60006d3c0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:48.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.732+0000 7fb61a3bd700 1 --2- 172.21.15.143:0/1779096370 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb61406b150 0x7fb6140c5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:48.735 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.732+0000 7fb61a3bd700 1 --2- 172.21.15.143:0/1779096370 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb6141618f0 0x7fb6140b33b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:48.735 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.732+0000 7fb61a3bd700 1 --2- 172.21.15.143:0/1779096370 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb61415dd00 0x7fb614157370 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:48.735 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.732+0000 7fb61a3bd700 1 -- 172.21.15.143:0/1779096370 >> 172.21.15.143:0/1779096370 conn(0x7fb614099ab0 msgr2=0x7fb6140d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:48.735 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.732+0000 7fb61a3bd700 1 -- 172.21.15.143:0/1779096370 shutdown_connections 2023-11-08T19:28:48.735 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:48.732+0000 7fb61a3bd700 1 -- 172.21.15.143:0/1779096370 wait complete. 2023-11-08T19:28:48.736 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 60 2023-11-08T19:28:48.745 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=30 retry_count=0)... 2023-11-08T19:28:50.151 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2023-11-08T19:28:53.686 INFO:tasks.ceph.mds.a.smithi143.stderr:2023-11-08T19:28:53.685+0000 7f5ba492b700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-08T19:28:53.687 INFO:tasks.ceph.mds.a.smithi143.stderr:2023-11-08T19:28:53.685+0000 7f5ba492b700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-11-08T19:28:53.747 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:28:53.880 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:53.878+0000 7f9301e43700 1 Processor -- start 2023-11-08T19:28:53.880 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:53.878+0000 7f9301e43700 1 -- start start 2023-11-08T19:28:53.882 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:53.879+0000 7f9301e43700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f92fc1618f0 0x7f92fc06ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:53.882 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:53.879+0000 7f9301e43700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f92fc06b150 0x7f92fc0cb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:53.883 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:53.879+0000 7f9301e43700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92fc161df0 con 0x7f92fc15dd00 2023-11-08T19:28:53.883 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:53.879+0000 7f9301e43700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92fc06b600 con 0x7f92fc06b150 2023-11-08T19:28:53.883 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:53.879+0000 7f9301e43700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92fc0cad50 con 0x7f92fc1618f0 2023-11-08T19:28:53.883 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:53.879+0000 7f92fbfff700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f92fc06b150 0x7f92fc0cb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:53.883 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:53.879+0000 7f92fb7fe700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f92fc15dd00 0x7f92fc0cca70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:51338/0 (socket says 172.21.15.143:51338) 2023-11-08T19:28:53.884 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:53.879+0000 7f92fb7fe700 1 -- 172.21.15.143:0/2710757847 learned_addr learned my addr 172.21.15.143:0/2710757847 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:28:53.884 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:53.880+0000 7f92faffd700 1 --2- 172.21.15.143:0/2710757847 >> v2:172.21.15.174:3300/0 conn(0x7f92fc1618f0 0x7f92fc06ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:53.884 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:53.880+0000 7f92fbfff700 1 -- 172.21.15.143:0/2710757847 >> v2:172.21.15.174:3300/0 conn(0x7f92fc1618f0 msgr2=0x7f92fc06ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:53.884 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:53.880+0000 7f92fbfff700 1 --2- 172.21.15.143:0/2710757847 >> v2:172.21.15.174:3300/0 conn(0x7f92fc1618f0 0x7f92fc06ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:53.885 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:53.880+0000 7f92fbfff700 1 -- 172.21.15.143:0/2710757847 >> v1:172.21.15.143:6789/0 conn(0x7f92fc15dd00 legacy=0x7f92fc0cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:53.885 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:53.880+0000 7f92fa7fc700 1 -- 172.21.15.143:0/2710757847 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1434093771 0 0) 0x7f92fc161df0 con 0x7f92fc15dd00 2023-11-08T19:28:53.885 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:53.880+0000 7f92fbfff700 1 -- 172.21.15.143:0/2710757847 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92fc1620d0 con 0x7f92fc06b150 2023-11-08T19:28:53.886 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:53.880+0000 7f92fbfff700 1 --2- 172.21.15.143:0/2710757847 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f92fc06b150 0x7f92fc0cb1a0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f92fc161850 tx=0x7f92f400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe1da99aa260ed0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:53.886 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:53.880+0000 7f92fa7fc700 1 -- 172.21.15.143:0/2710757847 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92f400e070 con 0x7f92fc06b150 2023-11-08T19:28:53.886 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:53.880+0000 7f92fa7fc700 1 -- 172.21.15.143:0/2710757847 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92f400bbd0 con 0x7f92fc06b150 2023-11-08T19:28:53.886 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:53.880+0000 7f92fa7fc700 1 -- 172.21.15.143:0/2710757847 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92f4012430 con 0x7f92fc06b150 2023-11-08T19:28:53.886 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:53.880+0000 7f9301e43700 1 -- 172.21.15.143:0/2710757847 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f92fc06b150 msgr2=0x7f92fc0cb1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:53.887 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:53.880+0000 7f9301e43700 1 --2- 172.21.15.143:0/2710757847 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f92fc06b150 0x7f92fc0cb1a0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f92fc161850 tx=0x7f92f400b330 comp rx=0 tx=0).stop 2023-11-08T19:28:53.887 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:53.881+0000 7f9301e43700 1 -- 172.21.15.143:0/2710757847 shutdown_connections 2023-11-08T19:28:53.887 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:53.881+0000 7f9301e43700 1 --2- 172.21.15.143:0/2710757847 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f92fc06b150 0x7f92fc0cb1a0 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:53.887 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:53.881+0000 7f9301e43700 1 --2- 172.21.15.143:0/2710757847 >> v2:172.21.15.174:3300/0 conn(0x7f92fc1618f0 0x7f92fc06ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:53.887 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:53.881+0000 7f9301e43700 1 -- 172.21.15.143:0/2710757847 >> 172.21.15.143:0/2710757847 conn(0x7f92fc099ab0 msgr2=0x7f92fc0d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:53.888 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:53.881+0000 7f9301e43700 1 -- 172.21.15.143:0/2710757847 shutdown_connections 2023-11-08T19:28:53.888 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:53.881+0000 7f9301e43700 1 -- 172.21.15.143:0/2710757847 wait complete. 2023-11-08T19:28:53.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:53.882+0000 7f9301e43700 1 Processor -- start 2023-11-08T19:28:53.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:53.882+0000 7f9301e43700 1 -- start start 2023-11-08T19:28:53.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:53.883+0000 7f9301e43700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f92fc06b150 0x7f92fc0c6d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:53.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:53.883+0000 7f9301e43700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f92fc15dd00 0x7f92fc0b6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:53.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:53.883+0000 7f9301e43700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f92fc1618f0 0x7f92fc151cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:53.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:53.883+0000 7f9301e43700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92fc0c4bf0 con 0x7f92fc06b150 2023-11-08T19:28:53.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:53.883+0000 7f9301e43700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92fc0b5880 con 0x7f92fc1618f0 2023-11-08T19:28:53.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:53.883+0000 7f9301e43700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92fc0c59b0 con 0x7f92fc15dd00 2023-11-08T19:28:53.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:53.883+0000 7f92fb7fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f92fc06b150 0x7f92fc0c6d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:53.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:53.883+0000 7f92fb7fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f92fc06b150 0x7f92fc0c6d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55190/0 (socket says 172.21.15.143:55190) 2023-11-08T19:28:53.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:53.883+0000 7f92fb7fe700 1 -- 172.21.15.143:0/1807145399 learned_addr learned my addr 172.21.15.143:0/1807145399 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:28:53.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:53.883+0000 7f92fbfff700 1 --2- 172.21.15.143:0/1807145399 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f92fc1618f0 0x7f92fc151cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:53.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:53.883+0000 7f92faffd700 1 --2- 172.21.15.143:0/1807145399 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f92fc15dd00 0x7f92fc0b6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:53.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:53.883+0000 7f92fb7fe700 1 -- 172.21.15.143:0/1807145399 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f92fc15dd00 msgr2=0x7f92fc0b6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:53.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:53.883+0000 7f92fb7fe700 1 --2- 172.21.15.143:0/1807145399 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f92fc15dd00 0x7f92fc0b6100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:53.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:53.883+0000 7f92fb7fe700 1 -- 172.21.15.143:0/1807145399 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f92fc1618f0 msgr2=0x7f92fc151cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:53.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:53.883+0000 7f92fb7fe700 1 --2- 172.21.15.143:0/1807145399 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f92fc1618f0 0x7f92fc151cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:53.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:53.883+0000 7f92fb7fe700 1 -- 172.21.15.143:0/1807145399 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92f4009a70 con 0x7f92fc06b150 2023-11-08T19:28:53.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:53.883+0000 7f92faffd700 1 --2- 172.21.15.143:0/1807145399 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f92fc15dd00 0x7f92fc0b6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:28:53.895 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:53.883+0000 7f92fb7fe700 1 --2- 172.21.15.143:0/1807145399 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f92fc06b150 0x7f92fc0c6d10 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f92e400c750 tx=0x7f92e400ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:53.895 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:53.883+0000 7f92f3fff700 1 -- 172.21.15.143:0/1807145399 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92e40074e0 con 0x7f92fc06b150 2023-11-08T19:28:53.895 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:53.883+0000 7f92f3fff700 1 -- 172.21.15.143:0/1807145399 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92e4007640 con 0x7f92fc06b150 2023-11-08T19:28:53.895 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:53.883+0000 7f9301e43700 1 -- 172.21.15.143:0/1807145399 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92fc1521f0 con 0x7f92fc06b150 2023-11-08T19:28:53.895 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:53.883+0000 7f92f3fff700 1 -- 172.21.15.143:0/1807145399 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92e4005a70 con 0x7f92fc06b150 2023-11-08T19:28:53.896 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:53.883+0000 7f9301e43700 1 -- 172.21.15.143:0/1807145399 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92fc0c4e30 con 0x7f92fc06b150 2023-11-08T19:28:53.896 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:53.884+0000 7f9301e43700 1 -- 172.21.15.143:0/1807145399 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92fc0aff60 con 0x7f92fc06b150 2023-11-08T19:28:53.896 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:53.885+0000 7f92f3fff700 1 -- 172.21.15.143:0/1807145399 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f92e4005bd0 con 0x7f92fc06b150 2023-11-08T19:28:53.896 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:53.885+0000 7f92f3fff700 1 --2- 172.21.15.143:0/1807145399 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f92dc06af00 0x7f92dc06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:53.897 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:53.885+0000 7f92f3fff700 1 -- 172.21.15.143:0/1807145399 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6887+0+0 (secure 0 0 0) 0x7f92e4088f60 con 0x7f92fc06b150 2023-11-08T19:28:53.897 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:53.885+0000 7f92faffd700 1 --2- 172.21.15.143:0/1807145399 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f92dc06af00 0x7f92dc06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:53.897 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:53.886+0000 7f92faffd700 1 --2- 172.21.15.143:0/1807145399 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f92dc06af00 0x7f92dc06d3c0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f92fc0c7250 tx=0x7f92ec008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:53.897 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:53.890+0000 7f92f3fff700 1 -- 172.21.15.143:0/1807145399 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f92e4053b70 con 0x7f92fc06b150 2023-11-08T19:28:54.121 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:54.120+0000 7f9301e43700 1 -- 172.21.15.143:0/1807145399 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f92fc04eee0 con 0x7f92fc06b150 2023-11-08T19:28:54.121 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:54.120+0000 7f92f3fff700 1 -- 172.21.15.143:0/1807145399 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f92e4017020 con 0x7f92fc06b150 2023-11-08T19:28:54.122 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:28:54.122 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":64,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5933,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/3918286710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6835","nonce":3918286710},{"type":"v1","addr":"172.21.15.174:6837","nonce":3918286710}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":6906,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6834/3172664377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":3172664377},{"type":"v1","addr":"172.21.15.143:6834","nonce":3172664377}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:03.092245+0000","modified":"2023-11-08T19:28:51.371927+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":95,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6909,"mds_1":6918},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6909":{"gid":6909,"name":"a","rank":0,"incarnation":60,"state":"up:active","state_seq":17,"addr":"172.21.15.143:6835/3953059900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":3953059900},{"type":"v1","addr":"172.21.15.143:6835","nonce":3953059900}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6918":{"gid":6918,"name":"b","rank":1,"incarnation":54,"state":"up:active","state_seq":6,"addr":"172.21.15.174:6836/3601088618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3601088618},{"type":"v1","addr":"172.21.15.174:6836","nonce":3601088618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:28:54.124 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:54.122+0000 7f9301e43700 1 -- 172.21.15.143:0/1807145399 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f92dc06af00 msgr2=0x7f92dc06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:54.124 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:54.122+0000 7f9301e43700 1 --2- 172.21.15.143:0/1807145399 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f92dc06af00 0x7f92dc06d3c0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f92fc0c7250 tx=0x7f92ec008040 comp rx=0 tx=0).stop 2023-11-08T19:28:54.124 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:54.122+0000 7f9301e43700 1 -- 172.21.15.143:0/1807145399 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f92fc06b150 msgr2=0x7f92fc0c6d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:54.124 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:54.122+0000 7f9301e43700 1 --2- 172.21.15.143:0/1807145399 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f92fc06b150 0x7f92fc0c6d10 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f92e400c750 tx=0x7f92e400ca60 comp rx=0 tx=0).stop 2023-11-08T19:28:54.125 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:54.122+0000 7f9301e43700 1 -- 172.21.15.143:0/1807145399 shutdown_connections 2023-11-08T19:28:54.125 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:54.122+0000 7f9301e43700 1 --2- 172.21.15.143:0/1807145399 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f92dc06af00 0x7f92dc06d3c0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:54.125 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:54.122+0000 7f9301e43700 1 --2- 172.21.15.143:0/1807145399 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f92fc1618f0 0x7f92fc151cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:54.125 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:54.122+0000 7f9301e43700 1 --2- 172.21.15.143:0/1807145399 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f92fc06b150 0x7f92fc0c6d10 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:54.125 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:54.122+0000 7f9301e43700 1 --2- 172.21.15.143:0/1807145399 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f92fc15dd00 0x7f92fc0b6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:54.126 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:54.122+0000 7f9301e43700 1 -- 172.21.15.143:0/1807145399 >> 172.21.15.143:0/1807145399 conn(0x7f92fc099ab0 msgr2=0x7f92fc0ae4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:54.126 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:54.122+0000 7f9301e43700 1 -- 172.21.15.143:0/1807145399 shutdown_connections 2023-11-08T19:28:54.126 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:54.122+0000 7f9301e43700 1 -- 172.21.15.143:0/1807145399 wait complete. 2023-11-08T19:28:54.127 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 64 2023-11-08T19:28:54.136 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2023-11-08T19:28:54.136 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:28:54.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:54.254+0000 7f6a7adb8700 1 Processor -- start 2023-11-08T19:28:54.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:54.254+0000 7f6a7adb8700 1 -- start start 2023-11-08T19:28:54.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:54.255+0000 7f6a7adb8700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6a7406c730 0x7f6a740cd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:54.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:54.255+0000 7f6a7adb8700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f6a7415de80 0x7f6a740cbf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:54.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:54.255+0000 7f6a7adb8700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a740cd480 con 0x7f6a7406c030 2023-11-08T19:28:54.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:54.255+0000 7f6a7adb8700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a740cc870 con 0x7f6a7415de80 2023-11-08T19:28:54.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:54.255+0000 7f6a7adb8700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a740ccb50 con 0x7f6a7406c730 2023-11-08T19:28:54.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:54.255+0000 7f6a79355700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f6a7415de80 0x7f6a740cbf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:54.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:54.255+0000 7f6a73fff700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f6a7406c030 0x7f6a7406cb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:51352/0 (socket says 172.21.15.143:51352) 2023-11-08T19:28:54.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:54.255+0000 7f6a73fff700 1 -- 172.21.15.143:0/1038878732 learned_addr learned my addr 172.21.15.143:0/1038878732 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:28:54.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:54.256+0000 7f6a78b54700 1 --2- 172.21.15.143:0/1038878732 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6a7406c730 0x7f6a740cd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:54.261 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:54.256+0000 7f6a79355700 1 -- 172.21.15.143:0/1038878732 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6a7406c730 msgr2=0x7f6a740cd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:54.261 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:54.256+0000 7f6a79355700 1 --2- 172.21.15.143:0/1038878732 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6a7406c730 0x7f6a740cd910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:54.261 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:54.256+0000 7f6a79355700 1 -- 172.21.15.143:0/1038878732 >> v1:172.21.15.143:6789/0 conn(0x7f6a7406c030 legacy=0x7f6a7406cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:54.261 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:54.256+0000 7f6a79355700 1 -- 172.21.15.143:0/1038878732 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a740cce30 con 0x7f6a7415de80 2023-11-08T19:28:54.261 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:54.256+0000 7f6a737fe700 1 -- 172.21.15.143:0/1038878732 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 289723617 0 0) 0x7f6a740cd480 con 0x7f6a7406c030 2023-11-08T19:28:54.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:54.256+0000 7f6a79355700 1 --2- 172.21.15.143:0/1038878732 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a7415de80 0x7f6a740cbf00 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f6a6c009d90 tx=0x7f6a6c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fef50e56993081cf server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:54.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:54.256+0000 7f6a737fe700 1 -- 172.21.15.143:0/1038878732 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a6c00e070 con 0x7f6a7415de80 2023-11-08T19:28:54.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:54.256+0000 7f6a737fe700 1 -- 172.21.15.143:0/1038878732 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a6c00bbd0 con 0x7f6a7415de80 2023-11-08T19:28:54.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:54.256+0000 7f6a737fe700 1 -- 172.21.15.143:0/1038878732 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a6c012430 con 0x7f6a7415de80 2023-11-08T19:28:54.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:54.256+0000 7f6a7adb8700 1 -- 172.21.15.143:0/1038878732 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a7415de80 msgr2=0x7f6a740cbf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:54.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:54.256+0000 7f6a7adb8700 1 --2- 172.21.15.143:0/1038878732 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a7415de80 0x7f6a740cbf00 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f6a6c009d90 tx=0x7f6a6c00b330 comp rx=0 tx=0).stop 2023-11-08T19:28:54.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:54.257+0000 7f6a7adb8700 1 -- 172.21.15.143:0/1038878732 shutdown_connections 2023-11-08T19:28:54.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:54.257+0000 7f6a7adb8700 1 --2- 172.21.15.143:0/1038878732 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a7415de80 0x7f6a740cbf00 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:54.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:54.257+0000 7f6a7adb8700 1 --2- 172.21.15.143:0/1038878732 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6a7406c730 0x7f6a740cd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:54.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:54.257+0000 7f6a7adb8700 1 -- 172.21.15.143:0/1038878732 >> 172.21.15.143:0/1038878732 conn(0x7f6a7408b000 msgr2=0x7f6a7409ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:54.264 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:54.257+0000 7f6a7adb8700 1 -- 172.21.15.143:0/1038878732 shutdown_connections 2023-11-08T19:28:54.264 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:54.257+0000 7f6a7adb8700 1 -- 172.21.15.143:0/1038878732 wait complete. 2023-11-08T19:28:54.264 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:54.258+0000 7f6a7adb8700 1 Processor -- start 2023-11-08T19:28:54.264 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:54.258+0000 7f6a7adb8700 1 -- start start 2023-11-08T19:28:54.268 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:54.259+0000 7f6a7adb8700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6a7406c030 0x7f6a7414fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:54.268 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:54.259+0000 7f6a7adb8700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a7406c730 0x7f6a7414f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:54.268 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:54.259+0000 7f6a7adb8700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6a7415de80 0x7f6a740b6e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:54.268 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:54.259+0000 7f6a7adb8700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a740b5d90 con 0x7f6a7406c730 2023-11-08T19:28:54.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:54.259+0000 7f6a7adb8700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a740b6070 con 0x7f6a7415de80 2023-11-08T19:28:54.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:54.259+0000 7f6a7adb8700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a740b6350 con 0x7f6a7406c030 2023-11-08T19:28:54.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:54.259+0000 7f6a73fff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a7406c730 0x7f6a7414f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:54.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:54.259+0000 7f6a73fff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a7406c730 0x7f6a7414f520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55206/0 (socket says 172.21.15.143:55206) 2023-11-08T19:28:54.270 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:54.259+0000 7f6a73fff700 1 -- 172.21.15.143:0/555816567 learned_addr learned my addr 172.21.15.143:0/555816567 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:28:54.270 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:54.259+0000 7f6a78b54700 1 --2- 172.21.15.143:0/555816567 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6a7406c030 0x7f6a7414fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:54.270 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:54.259+0000 7f6a79355700 1 --2- 172.21.15.143:0/555816567 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6a7415de80 0x7f6a740b6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:54.270 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:54.259+0000 7f6a73fff700 1 -- 172.21.15.143:0/555816567 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6a7406c030 msgr2=0x7f6a7414fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:54.270 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:54.259+0000 7f6a73fff700 1 --2- 172.21.15.143:0/555816567 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6a7406c030 0x7f6a7414fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:54.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:54.259+0000 7f6a73fff700 1 -- 172.21.15.143:0/555816567 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6a7415de80 msgr2=0x7f6a740b6e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:54.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:54.259+0000 7f6a73fff700 1 --2- 172.21.15.143:0/555816567 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6a7415de80 0x7f6a740b6e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:54.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:54.259+0000 7f6a73fff700 1 -- 172.21.15.143:0/555816567 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a6c009a70 con 0x7f6a7406c730 2023-11-08T19:28:54.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:54.259+0000 7f6a73fff700 1 --2- 172.21.15.143:0/555816567 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a7406c730 0x7f6a7414f520 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f6a6000c520 tx=0x7f6a6000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:54.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:54.259+0000 7f6a717fa700 1 -- 172.21.15.143:0/555816567 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a60004c80 con 0x7f6a7406c730 2023-11-08T19:28:54.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:54.259+0000 7f6a717fa700 1 -- 172.21.15.143:0/555816567 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a60004de0 con 0x7f6a7406c730 2023-11-08T19:28:54.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:54.259+0000 7f6a7adb8700 1 -- 172.21.15.143:0/555816567 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a740b6630 con 0x7f6a7406c730 2023-11-08T19:28:54.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:54.259+0000 7f6a717fa700 1 -- 172.21.15.143:0/555816567 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a600072d0 con 0x7f6a7406c730 2023-11-08T19:28:54.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:54.260+0000 7f6a7adb8700 1 -- 172.21.15.143:0/555816567 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a740b7370 con 0x7f6a7406c730 2023-11-08T19:28:54.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:54.261+0000 7f6a7adb8700 1 -- 172.21.15.143:0/555816567 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a740b6b50 con 0x7f6a7406c730 2023-11-08T19:28:54.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:54.261+0000 7f6a717fa700 1 -- 172.21.15.143:0/555816567 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6a60007430 con 0x7f6a7406c730 2023-11-08T19:28:54.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:54.261+0000 7f6a717fa700 1 --2- 172.21.15.143:0/555816567 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f6a5806af00 0x7f6a5806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:54.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:54.261+0000 7f6a717fa700 1 -- 172.21.15.143:0/555816567 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6887+0+0 (secure 0 0 0) 0x7f6a600895a0 con 0x7f6a7406c730 2023-11-08T19:28:54.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:54.262+0000 7f6a78b54700 1 --2- 172.21.15.143:0/555816567 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f6a5806af00 0x7f6a5806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:54.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:54.262+0000 7f6a78b54700 1 --2- 172.21.15.143:0/555816567 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f6a5806af00 0x7f6a5806d3c0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f6a74160d20 tx=0x7f6a68008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:54.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:54.266+0000 7f6a717fa700 1 -- 172.21.15.143:0/555816567 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6a740b6b50 con 0x7f6a7406c730 2023-11-08T19:28:54.501 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:54.500+0000 7f6a7adb8700 1 -- 172.21.15.143:0/555816567 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6a740b6b50 con 0x7f6a7406c730 2023-11-08T19:28:54.502 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:54.501+0000 7f6a717fa700 1 -- 172.21.15.143:0/555816567 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f6a740b6b50 con 0x7f6a7406c730 2023-11-08T19:28:54.502 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:28:54.503 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":64,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5933,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/3918286710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6835","nonce":3918286710},{"type":"v1","addr":"172.21.15.174:6837","nonce":3918286710}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":6906,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6834/3172664377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":3172664377},{"type":"v1","addr":"172.21.15.143:6834","nonce":3172664377}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:03.092245+0000","modified":"2023-11-08T19:28:51.371927+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":95,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6909,"mds_1":6918},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6909":{"gid":6909,"name":"a","rank":0,"incarnation":60,"state":"up:active","state_seq":17,"addr":"172.21.15.143:6835/3953059900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":3953059900},{"type":"v1","addr":"172.21.15.143:6835","nonce":3953059900}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6918":{"gid":6918,"name":"b","rank":1,"incarnation":54,"state":"up:active","state_seq":6,"addr":"172.21.15.174:6836/3601088618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3601088618},{"type":"v1","addr":"172.21.15.174:6836","nonce":3601088618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:28:54.504 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:54.503+0000 7f6a7adb8700 1 -- 172.21.15.143:0/555816567 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f6a5806af00 msgr2=0x7f6a5806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:54.505 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:54.503+0000 7f6a7adb8700 1 --2- 172.21.15.143:0/555816567 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f6a5806af00 0x7f6a5806d3c0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f6a74160d20 tx=0x7f6a68008040 comp rx=0 tx=0).stop 2023-11-08T19:28:54.505 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:54.503+0000 7f6a7adb8700 1 -- 172.21.15.143:0/555816567 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a7406c730 msgr2=0x7f6a7414f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:54.505 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:54.503+0000 7f6a7adb8700 1 --2- 172.21.15.143:0/555816567 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a7406c730 0x7f6a7414f520 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f6a6000c520 tx=0x7f6a6000c830 comp rx=0 tx=0).stop 2023-11-08T19:28:54.505 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:54.503+0000 7f6a7adb8700 1 -- 172.21.15.143:0/555816567 shutdown_connections 2023-11-08T19:28:54.505 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:54.503+0000 7f6a7adb8700 1 --2- 172.21.15.143:0/555816567 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f6a5806af00 0x7f6a5806d3c0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:54.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:54.503+0000 7f6a7adb8700 1 --2- 172.21.15.143:0/555816567 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6a7406c030 0x7f6a7414fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:54.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:54.503+0000 7f6a7adb8700 1 --2- 172.21.15.143:0/555816567 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6a7415de80 0x7f6a740b6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:54.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:54.503+0000 7f6a7adb8700 1 --2- 172.21.15.143:0/555816567 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a7406c730 0x7f6a7414f520 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:54.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:54.503+0000 7f6a7adb8700 1 -- 172.21.15.143:0/555816567 >> 172.21.15.143:0/555816567 conn(0x7f6a7408b000 msgr2=0x7f6a7406f310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:54.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:54.503+0000 7f6a7adb8700 1 -- 172.21.15.143:0/555816567 shutdown_connections 2023-11-08T19:28:54.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:54.503+0000 7f6a7adb8700 1 -- 172.21.15.143:0/555816567 wait complete. 2023-11-08T19:28:54.507 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 64 2023-11-08T19:28:54.918 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 64, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 2, 'explicitly_allowed_features': 2, 'created': '2023-11-08T19:28:03.092245+0000', 'modified': '2023-11-08T19:28:51.371927+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 95, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 2, 'in': [0, 1], 'up': {'mds_0': 6909, 'mds_1': 6918}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6909': {'gid': 6909, 'name': 'a', 'rank': 0, 'incarnation': 60, 'state': 'up:active', 'state_seq': 17, 'addr': '172.21.15.143:6835/3953059900', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.143:6833', 'nonce': 3953059900}, {'type': 'v1', 'addr': '172.21.15.143:6835', 'nonce': 3953059900}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_6918': {'gid': 6918, 'name': 'b', 'rank': 1, 'incarnation': 54, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.174:6836/3601088618', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.174:6834', 'nonce': 3601088618}, {'type': 'v1', 'addr': '172.21.15.174:6836', 'nonce': 3601088618}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-08T19:28:54.919 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 2/2 2023-11-08T19:28:54.920 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-11-08T19:28:55.054 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.045+0000 7f06e1d61700 1 Processor -- start 2023-11-08T19:28:55.055 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.045+0000 7f06e1d61700 1 -- start start 2023-11-08T19:28:55.055 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.046+0000 7f06e1d61700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f06dc15dd00 0x7f06dc0cca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:55.056 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.046+0000 7f06e1d61700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f06dc1618f0 0x7f06dc06ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:55.056 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.046+0000 7f06e1d61700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f06dc06b150 0x7f06dc0b2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:55.056 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.046+0000 7f06e1d61700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06dc06b600 con 0x7f06dc06b150 2023-11-08T19:28:55.057 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.046+0000 7f06e1d61700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06dc0cad50 con 0x7f06dc1618f0 2023-11-08T19:28:55.057 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.046+0000 7f06e1d61700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06dc0ccfb0 con 0x7f06dc15dd00 2023-11-08T19:28:55.057 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.046+0000 7f06dbfff700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f06dc06b150 0x7f06dc0b2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:55.057 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.046+0000 7f06dbfff700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f06dc06b150 0x7f06dc0b2210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55214/0 (socket says 172.21.15.143:55214) 2023-11-08T19:28:55.058 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.046+0000 7f06dbfff700 1 -- 172.21.15.143:0/3974454304 learned_addr learned my addr 172.21.15.143:0/3974454304 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:28:55.058 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.046+0000 7f06db7fe700 1 --2- 172.21.15.143:0/3974454304 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f06dc15dd00 0x7f06dc0cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:55.058 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.046+0000 7f06daffd700 1 --2- 172.21.15.143:0/3974454304 >> v2:172.21.15.174:3300/0 conn(0x7f06dc1618f0 0x7f06dc06ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:55.059 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.047+0000 7f06dbfff700 1 -- 172.21.15.143:0/3974454304 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f06dc15dd00 msgr2=0x7f06dc0cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:55.059 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.047+0000 7f06dbfff700 1 --2- 172.21.15.143:0/3974454304 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f06dc15dd00 0x7f06dc0cca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:55.059 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.047+0000 7f06dbfff700 1 -- 172.21.15.143:0/3974454304 >> v2:172.21.15.174:3300/0 conn(0x7f06dc1618f0 msgr2=0x7f06dc06ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:55.060 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.047+0000 7f06dbfff700 1 --2- 172.21.15.143:0/3974454304 >> v2:172.21.15.174:3300/0 conn(0x7f06dc1618f0 0x7f06dc06ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:55.060 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.047+0000 7f06dbfff700 1 -- 172.21.15.143:0/3974454304 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06dc161df0 con 0x7f06dc06b150 2023-11-08T19:28:55.060 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.047+0000 7f06dbfff700 1 --2- 172.21.15.143:0/3974454304 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f06dc06b150 0x7f06dc0b2210 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f06dc071c00 tx=0x7f06d400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f1e28678d665ea4 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:55.061 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.047+0000 7f06da7fc700 1 -- 172.21.15.143:0/3974454304 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06d400e070 con 0x7f06dc06b150 2023-11-08T19:28:55.061 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.047+0000 7f06da7fc700 1 -- 172.21.15.143:0/3974454304 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06d400b9c0 con 0x7f06dc06b150 2023-11-08T19:28:55.061 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.047+0000 7f06da7fc700 1 -- 172.21.15.143:0/3974454304 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06d400bd70 con 0x7f06dc06b150 2023-11-08T19:28:55.062 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.047+0000 7f06e1d61700 1 -- 172.21.15.143:0/3974454304 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f06dc06b150 msgr2=0x7f06dc0b2210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:55.062 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.047+0000 7f06e1d61700 1 --2- 172.21.15.143:0/3974454304 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f06dc06b150 0x7f06dc0b2210 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f06dc071c00 tx=0x7f06d400b330 comp rx=0 tx=0).stop 2023-11-08T19:28:55.063 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.047+0000 7f06e1d61700 1 -- 172.21.15.143:0/3974454304 shutdown_connections 2023-11-08T19:28:55.063 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.047+0000 7f06e1d61700 1 --2- 172.21.15.143:0/3974454304 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f06dc06b150 0x7f06dc0b2210 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:55.063 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.047+0000 7f06e1d61700 1 --2- 172.21.15.143:0/3974454304 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f06dc15dd00 0x7f06dc0cca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:55.064 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.047+0000 7f06e1d61700 1 --2- 172.21.15.143:0/3974454304 >> v2:172.21.15.174:3300/0 conn(0x7f06dc1618f0 0x7f06dc06ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:55.064 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.047+0000 7f06e1d61700 1 -- 172.21.15.143:0/3974454304 >> 172.21.15.143:0/3974454304 conn(0x7f06dc099ab0 msgr2=0x7f06dc0d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:55.065 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.047+0000 7f06e1d61700 1 -- 172.21.15.143:0/3974454304 shutdown_connections 2023-11-08T19:28:55.065 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.047+0000 7f06e1d61700 1 -- 172.21.15.143:0/3974454304 wait complete. 2023-11-08T19:28:55.065 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.049+0000 7f06e1d61700 1 Processor -- start 2023-11-08T19:28:55.066 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.049+0000 7f06e1d61700 1 -- start start 2023-11-08T19:28:55.066 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.050+0000 7f06e1d61700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f06dc06b150 0x7f06dc152100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:55.066 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.050+0000 7f06e1d61700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f06dc15dd00 0x7f06dc1f6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:55.066 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.050+0000 7f06e1d61700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f06dc1618f0 0x7f06dc0c9c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:55.067 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.050+0000 7f06e1d61700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06dc0c5c80 con 0x7f06dc06b150 2023-11-08T19:28:55.067 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.050+0000 7f06e1d61700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06dc0c5670 con 0x7f06dc1618f0 2023-11-08T19:28:55.067 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.050+0000 7f06e1d61700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06dc1f7de0 con 0x7f06dc15dd00 2023-11-08T19:28:55.068 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.050+0000 7f06db7fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f06dc06b150 0x7f06dc152100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:55.068 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.050+0000 7f06db7fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f06dc06b150 0x7f06dc152100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55216/0 (socket says 172.21.15.143:55216) 2023-11-08T19:28:55.068 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.050+0000 7f06db7fe700 1 -- 172.21.15.143:0/4034078012 learned_addr learned my addr 172.21.15.143:0/4034078012 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:28:55.068 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.050+0000 7f06dbfff700 1 --2- 172.21.15.143:0/4034078012 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f06dc1618f0 0x7f06dc0c9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:55.069 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.050+0000 7f06daffd700 1 --2- 172.21.15.143:0/4034078012 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f06dc15dd00 0x7f06dc1f6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:55.069 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.050+0000 7f06db7fe700 1 -- 172.21.15.143:0/4034078012 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f06dc15dd00 msgr2=0x7f06dc1f6eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:55.070 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.050+0000 7f06db7fe700 1 --2- 172.21.15.143:0/4034078012 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f06dc15dd00 0x7f06dc1f6eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:55.070 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.050+0000 7f06db7fe700 1 -- 172.21.15.143:0/4034078012 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f06dc1618f0 msgr2=0x7f06dc0c9c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:55.070 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.050+0000 7f06db7fe700 1 --2- 172.21.15.143:0/4034078012 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f06dc1618f0 0x7f06dc0c9c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:55.071 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.050+0000 7f06db7fe700 1 -- 172.21.15.143:0/4034078012 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06d4009b30 con 0x7f06dc06b150 2023-11-08T19:28:55.071 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.050+0000 7f06dbfff700 1 --2- 172.21.15.143:0/4034078012 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f06dc1618f0 0x7f06dc0c9c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:28:55.071 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.050+0000 7f06db7fe700 1 --2- 172.21.15.143:0/4034078012 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f06dc06b150 0x7f06dc152100 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f06c400e7e0 tx=0x7f06c400eba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:55.072 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.050+0000 7f06d3fff700 1 -- 172.21.15.143:0/4034078012 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06c4013610 con 0x7f06dc06b150 2023-11-08T19:28:55.072 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.050+0000 7f06d3fff700 1 -- 172.21.15.143:0/4034078012 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06c4013770 con 0x7f06dc06b150 2023-11-08T19:28:55.073 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.050+0000 7f06e1d61700 1 -- 172.21.15.143:0/4034078012 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06dc152640 con 0x7f06dc06b150 2023-11-08T19:28:55.073 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.050+0000 7f06d3fff700 1 -- 172.21.15.143:0/4034078012 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06c4013b00 con 0x7f06dc06b150 2023-11-08T19:28:55.073 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.050+0000 7f06e1d61700 1 -- 172.21.15.143:0/4034078012 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06dc1f7b00 con 0x7f06dc06b150 2023-11-08T19:28:55.092 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.054+0000 7f06edf5ab80 1 Processor -- start 2023-11-08T19:28:55.093 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.054+0000 7f06d3fff700 1 -- 172.21.15.143:0/4034078012 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f06c4013c60 con 0x7f06dc06b150 2023-11-08T19:28:55.094 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.054+0000 7f06edf5ab80 1 -- start start 2023-11-08T19:28:55.094 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.054+0000 7f06d3fff700 1 --2- 172.21.15.143:0/4034078012 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f06bc06af00 0x7f06bc06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:55.094 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.055+0000 7f06d3fff700 1 -- 172.21.15.143:0/4034078012 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6887+0+0 (secure 0 0 0) 0x7f06c4089a70 con 0x7f06dc06b150 2023-11-08T19:28:55.095 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.055+0000 7f06dbfff700 1 --2- 172.21.15.143:0/4034078012 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f06bc06af00 0x7f06bc06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:55.095 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.055+0000 7f06edf5ab80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560d62b176f0 0x560d62b47450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:55.095 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.055+0000 7f06edf5ab80 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x560d62b19c30 0x560d62b47990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:55.096 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.055+0000 7f06edf5ab80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560d62b17ad0 0x560d62b31f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:55.096 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.055+0000 7f06edf5ab80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d62a836e0 con 0x560d62b176f0 2023-11-08T19:28:55.096 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.055+0000 7f06edf5ab80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d62a773b0 con 0x560d62b17ad0 2023-11-08T19:28:55.097 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.055+0000 7f06edf5ab80 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x560d62a74cd0 con 0x560d62b19c30 2023-11-08T19:28:55.097 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.055+0000 7f06db7fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560d62b176f0 0x560d62b47450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:55.098 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.055+0000 7f06db7fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560d62b176f0 0x560d62b47450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55224/0 (socket says 172.21.15.143:55224) 2023-11-08T19:28:55.098 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.055+0000 7f06db7fe700 1 -- 172.21.15.143:0/2288721219 learned_addr learned my addr 172.21.15.143:0/2288721219 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:28:55.099 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.055+0000 7f06daffd700 1 --2- 172.21.15.143:0/2288721219 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x560d62b19c30 0x560d62b47990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:55.099 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.055+0000 7f06dbfff700 1 --2- 172.21.15.143:0/2288721219 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560d62b17ad0 0x560d62b31f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:55.099 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.056+0000 7f06db7fe700 1 -- 172.21.15.143:0/2288721219 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x560d62b19c30 msgr2=0x560d62b47990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:55.099 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.056+0000 7f06db7fe700 1 --2- 172.21.15.143:0/2288721219 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x560d62b19c30 0x560d62b47990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:55.099 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.056+0000 7f06dbfff700 1 --2- 172.21.15.143:0/4034078012 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f06bc06af00 0x7f06bc06d3c0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f06d400b370 tx=0x7f06d4009000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:55.099 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.056+0000 7f06db7fe700 1 -- 172.21.15.143:0/2288721219 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560d62b17ad0 msgr2=0x560d62b31f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:55.100 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.056+0000 7f06db7fe700 1 --2- 172.21.15.143:0/2288721219 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560d62b17ad0 0x560d62b31f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:55.100 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.056+0000 7f06db7fe700 1 -- 172.21.15.143:0/2288721219 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560d628ed5e0 con 0x560d62b176f0 2023-11-08T19:28:55.100 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.056+0000 7f06db7fe700 1 --2- 172.21.15.143:0/2288721219 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560d62b176f0 0x560d62b47450 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f06c4006ac0 tx=0x7f06c4058b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40b361c2fee25455 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:55.101 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.056+0000 7f06d17fa700 1 -- 172.21.15.143:0/2288721219 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06c405c480 con 0x560d62b176f0 2023-11-08T19:28:55.101 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.056+0000 7f06d17fa700 1 -- 172.21.15.143:0/2288721219 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06c4060630 con 0x560d62b176f0 2023-11-08T19:28:55.101 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.056+0000 7f06d17fa700 1 -- 172.21.15.143:0/2288721219 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06c405c0d0 con 0x560d62b176f0 2023-11-08T19:28:55.101 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.057+0000 7f06edf5ab80 1 -- 172.21.15.143:0/2288721219 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560d62b176f0 msgr2=0x560d62b47450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:55.102 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.057+0000 7f06edf5ab80 1 --2- 172.21.15.143:0/2288721219 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560d62b176f0 0x560d62b47450 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f06c4006ac0 tx=0x7f06c4058b00 comp rx=0 tx=0).stop 2023-11-08T19:28:55.102 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.057+0000 7f06edf5ab80 1 -- 172.21.15.143:0/2288721219 shutdown_connections 2023-11-08T19:28:55.102 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.057+0000 7f06edf5ab80 1 --2- 172.21.15.143:0/2288721219 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560d62b17ad0 0x560d62b31f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:55.102 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.057+0000 7f06edf5ab80 1 --2- 172.21.15.143:0/2288721219 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560d62b176f0 0x560d62b47450 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:55.103 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.057+0000 7f06edf5ab80 1 --2- 172.21.15.143:0/2288721219 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x560d62b19c30 0x560d62b47990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:55.103 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.057+0000 7f06edf5ab80 1 -- 172.21.15.143:0/2288721219 >> 172.21.15.143:0/2288721219 conn(0x560d62a92660 msgr2=0x560d62b1d3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:55.103 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.057+0000 7f06edf5ab80 1 -- 172.21.15.143:0/2288721219 shutdown_connections 2023-11-08T19:28:55.103 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.057+0000 7f06edf5ab80 1 -- 172.21.15.143:0/2288721219 wait complete. 2023-11-08T19:28:55.103 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.057+0000 7f06edf5ab80 1 -- start start 2023-11-08T19:28:55.104 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.057+0000 7f06edf5ab80 1 Processor -- start 2023-11-08T19:28:55.104 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.058+0000 7f06edf5ab80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560d62b176f0 0x560d62b5a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:55.104 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.058+0000 7f06edf5ab80 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x560d62b17ad0 0x560d62b5ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:55.104 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.058+0000 7f06edf5ab80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560d62b72580 0x560d62b5cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:55.105 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.058+0000 7f06edf5ab80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d62a705c0 con 0x560d62b72580 2023-11-08T19:28:55.105 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.058+0000 7f06edf5ab80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d62a0c0d0 con 0x560d62b176f0 2023-11-08T19:28:55.105 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.058+0000 7f06edf5ab80 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x560d62a0bd70 con 0x560d62b17ad0 2023-11-08T19:28:55.105 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.058+0000 7f06edf5ab80 10 client.0 _ll_register_callbacks cb 0x560d628687b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T19:28:55.105 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.058+0000 7f06dbfff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560d62b72580 0x560d62b5cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:55.106 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.058+0000 7f06dbfff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560d62b72580 0x560d62b5cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55234/0 (socket says 172.21.15.143:55234) 2023-11-08T19:28:55.106 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.058+0000 7f06dbfff700 1 -- 172.21.15.143:0/3930514069 learned_addr learned my addr 172.21.15.143:0/3930514069 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:28:55.106 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.059+0000 7f06daffd700 1 --2- 172.21.15.143:0/3930514069 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x560d62b17ad0 0x560d62b5ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:55.106 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.059+0000 7f06db7fe700 1 --2- 172.21.15.143:0/3930514069 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560d62b176f0 0x560d62b5a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:55.107 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.059+0000 7f06dbfff700 1 -- 172.21.15.143:0/3930514069 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x560d62b17ad0 msgr2=0x560d62b5ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:55.107 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.059+0000 7f06dbfff700 1 --2- 172.21.15.143:0/3930514069 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x560d62b17ad0 0x560d62b5ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:55.107 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.059+0000 7f06dbfff700 1 -- 172.21.15.143:0/3930514069 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560d62b176f0 msgr2=0x560d62b5a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:55.108 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.059+0000 7f06dbfff700 1 --2- 172.21.15.143:0/3930514069 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560d62b176f0 0x560d62b5a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:55.108 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.059+0000 7f06dbfff700 1 -- 172.21.15.143:0/3930514069 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06dc1f7b00 con 0x560d62b72580 2023-11-08T19:28:55.108 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.059+0000 7f06dbfff700 1 --2- 172.21.15.143:0/3930514069 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560d62b72580 0x560d62b5cf70 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f06d400b730 tx=0x7f06d4004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2696928e34ad2fd server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:55.108 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.059+0000 7f06ba7fc700 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2023-11-08T19:28:55.108 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.059+0000 7f06ba7fc700 1 -- 172.21.15.143:0/3930514069 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06d4027620 con 0x560d62b72580 2023-11-08T19:28:55.109 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.059+0000 7f06ba7fc700 1 -- 172.21.15.143:0/3930514069 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 6887+0+0 (secure 0 0 0) 0x7f06d4022030 con 0x560d62b72580 2023-11-08T19:28:55.109 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.059+0000 7f06edf5ab80 1 -- 172.21.15.143:0/3930514069 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560d62b0f880 con 0x560d62b72580 2023-11-08T19:28:55.109 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.059+0000 7f06ba7fc700 1 -- 172.21.15.143:0/3930514069 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) v2 ==== 24+0+0 (secure 0 0 0) 0x7f06d4004910 con 0x560d62b72580 2023-11-08T19:28:55.109 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.059+0000 7f06edf5ab80 10 client.7299 fetch_fsmap learned FSMap version 64 2023-11-08T19:28:55.109 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.059+0000 7f06edf5ab80 1 -- 172.21.15.143:0/3930514069 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=2+}) v3 -- 0x560d62899d90 con 0x560d62b72580 2023-11-08T19:28:55.110 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.059+0000 7f06ba7fc700 1 -- 172.21.15.143:0/3930514069 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560d62b0f880 con 0x560d62b72580 2023-11-08T19:28:55.110 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.059+0000 7f06ba7fc700 1 -- 172.21.15.143:0/3930514069 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 64) v1 ==== 3306+0+0 (secure 0 0 0) 0x7f06d4027780 con 0x560d62b72580 2023-11-08T19:28:55.110 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.059+0000 7f06edf5ab80 10 client.7299 fetch_fsmap finished waiting for FSMap version 64 2023-11-08T19:28:55.110 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.059+0000 7f06edf5ab80 10 client.7299 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-08T19:28:55.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.059+0000 7f06edf5ab80 20 client.7299 populate_metadata read hostname 'smithi143' 2023-11-08T19:28:55.112 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.059+0000 7f06edf5ab80 1 --2- 172.21.15.143:0/3930514069 >> [v2:172.21.15.143:6833/3953059900,v1:172.21.15.143:6835/3953059900] conn(0x560d62b74dd0 0x560d62b751b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:55.112 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.059+0000 7f06edf5ab80 4 client.7299 mds_command: new command op to 6909 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-08T19:28:55.112 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.059+0000 7f06edf5ab80 1 -- 172.21.15.143:0/3930514069 --> [v2:172.21.15.143:6833/3953059900,v1:172.21.15.143:6835/3953059900] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560d62aeba30 con 0x560d62b74dd0 2023-11-08T19:28:55.113 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.059+0000 7f06db7fe700 1 --2- 172.21.15.143:0/3930514069 >> [v2:172.21.15.143:6833/3953059900,v1:172.21.15.143:6835/3953059900] conn(0x560d62b74dd0 0x560d62b751b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:55.113 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.060+0000 7f06ba7fc700 10 client.7299 ms_handle_connect on v2:172.21.15.143:6833/3953059900 2023-11-08T19:28:55.113 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.060+0000 7f06db7fe700 1 --2- 172.21.15.143:0/3930514069 >> [v2:172.21.15.143:6833/3953059900,v1:172.21.15.143:6835/3953059900] conn(0x560d62b74dd0 0x560d62b751b0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92749d21021fb04f server_cookie=57738e29e927f671 in_seq=0 out_seq=0 2023-11-08T19:28:55.113 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.060+0000 7f06ba7fc700 1 -- 172.21.15.143:0/3930514069 <== mds.0 v2:172.21.15.143:6833/3953059900 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560d62aeba30 con 0x560d62b74dd0 2023-11-08T19:28:55.113 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.060+0000 7f06ba7fc700 10 client.7299 handle_command_reply: tid=0 2023-11-08T19:28:55.114 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.060+0000 7f06edf5ab80 1 client.7299 shutdown 2023-11-08T19:28:55.114 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.060+0000 7f06edf5ab80 1 -- 172.21.15.143:0/3930514069 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560d62b72580 msgr2=0x560d62b5cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:55.114 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.060+0000 7f06edf5ab80 1 --2- 172.21.15.143:0/3930514069 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560d62b72580 0x560d62b5cf70 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f06d400b730 tx=0x7f06d4004b40 comp rx=0 tx=0).stop 2023-11-08T19:28:55.114 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.060+0000 7f06edf5ab80 1 -- 172.21.15.143:0/3930514069 shutdown_connections 2023-11-08T19:28:55.114 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.060+0000 7f06edf5ab80 1 --2- 172.21.15.143:0/3930514069 >> [v2:172.21.15.143:6833/3953059900,v1:172.21.15.143:6835/3953059900] conn(0x560d62b74dd0 0x560d62b751b0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:55.115 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.060+0000 7f06edf5ab80 1 --2- 172.21.15.143:0/3930514069 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560d62b72580 0x560d62b5cf70 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:55.115 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.060+0000 7f06edf5ab80 1 --2- 172.21.15.143:0/3930514069 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560d62b176f0 0x560d62b5a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:55.115 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.060+0000 7f06edf5ab80 1 --2- 172.21.15.143:0/3930514069 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x560d62b17ad0 0x560d62b5ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:55.115 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.060+0000 7f06ba7fc700 0 client.7299 ms_handle_reset on v2:172.21.15.143:6833/3953059900 2023-11-08T19:28:55.115 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.060+0000 7f06edf5ab80 1 -- 172.21.15.143:0/3930514069 >> 172.21.15.143:0/3930514069 conn(0x560d62a92660 msgr2=0x560d62b47440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:55.116 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.061+0000 7f06edf5ab80 1 -- 172.21.15.143:0/3930514069 shutdown_connections 2023-11-08T19:28:55.116 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.061+0000 7f06edf5ab80 1 -- 172.21.15.143:0/3930514069 wait complete. 2023-11-08T19:28:55.116 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.061+0000 7f06edf5ab80 20 client.7299 trim_cache size 0 max 16384 2023-11-08T19:28:55.116 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.078+0000 7f06edf5ab80 1 Processor -- start 2023-11-08T19:28:55.116 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.078+0000 7f06edf5ab80 1 -- start start 2023-11-08T19:28:55.117 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.079+0000 7f06edf5ab80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560d62b176f0 0x560d62b5d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:55.117 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.079+0000 7f06edf5ab80 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x560d62b17ad0 0x560d62b17eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:55.117 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.079+0000 7f06edf5ab80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560d62b72580 0x560d62b70e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:55.117 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.079+0000 7f06edf5ab80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d628ef8d0 con 0x560d62b72580 2023-11-08T19:28:55.117 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.079+0000 7f06edf5ab80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d62b199f0 con 0x560d62b176f0 2023-11-08T19:28:55.117 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.079+0000 7f06edf5ab80 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x560d62b7cdf0 con 0x560d62b17ad0 2023-11-08T19:28:55.118 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.079+0000 7f06dbfff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560d62b72580 0x560d62b70e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:55.118 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.079+0000 7f06dbfff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560d62b72580 0x560d62b70e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55242/0 (socket says 172.21.15.143:55242) 2023-11-08T19:28:55.118 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.079+0000 7f06dbfff700 1 -- 172.21.15.143:0/2549487270 learned_addr learned my addr 172.21.15.143:0/2549487270 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:28:55.118 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.080+0000 7f06daffd700 1 --2- 172.21.15.143:0/2549487270 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x560d62b17ad0 0x560d62b17eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:55.118 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.080+0000 7f06db7fe700 1 --2- 172.21.15.143:0/2549487270 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560d62b176f0 0x560d62b5d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:55.119 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.080+0000 7f06dbfff700 1 -- 172.21.15.143:0/2549487270 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x560d62b17ad0 msgr2=0x560d62b17eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:55.119 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.080+0000 7f06dbfff700 1 --2- 172.21.15.143:0/2549487270 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x560d62b17ad0 0x560d62b17eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:55.119 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.080+0000 7f06dbfff700 1 -- 172.21.15.143:0/2549487270 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560d62b176f0 msgr2=0x560d62b5d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:55.119 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.080+0000 7f06dbfff700 1 --2- 172.21.15.143:0/2549487270 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560d62b176f0 0x560d62b5d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:55.120 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.080+0000 7f06dbfff700 1 -- 172.21.15.143:0/2549487270 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560d62882c10 con 0x560d62b72580 2023-11-08T19:28:55.120 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.080+0000 7f06dbfff700 1 --2- 172.21.15.143:0/2549487270 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560d62b72580 0x560d62b70e50 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f06d400d010 tx=0x7f06d4029070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e84d14b49809078 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:55.120 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.080+0000 7f06d1ffb700 1 -- 172.21.15.143:0/2549487270 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06d40070a0 con 0x560d62b72580 2023-11-08T19:28:55.120 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.080+0000 7f06d1ffb700 1 -- 172.21.15.143:0/2549487270 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06d400bd20 con 0x560d62b72580 2023-11-08T19:28:55.120 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.080+0000 7f06d1ffb700 1 -- 172.21.15.143:0/2549487270 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06d4027620 con 0x560d62b72580 2023-11-08T19:28:55.121 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.080+0000 7f06edf5ab80 1 -- 172.21.15.143:0/2549487270 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560d62b72580 msgr2=0x560d62b70e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:55.121 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.080+0000 7f06edf5ab80 1 --2- 172.21.15.143:0/2549487270 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560d62b72580 0x560d62b70e50 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f06d400d010 tx=0x7f06d4029070 comp rx=0 tx=0).stop 2023-11-08T19:28:55.121 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.081+0000 7f06edf5ab80 1 -- 172.21.15.143:0/2549487270 shutdown_connections 2023-11-08T19:28:55.121 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.081+0000 7f06edf5ab80 1 --2- 172.21.15.143:0/2549487270 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560d62b72580 0x560d62b70e50 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:55.122 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.081+0000 7f06edf5ab80 1 --2- 172.21.15.143:0/2549487270 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560d62b176f0 0x560d62b5d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:55.122 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.081+0000 7f06edf5ab80 1 --2- 172.21.15.143:0/2549487270 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x560d62b17ad0 0x560d62b17eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:55.122 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.081+0000 7f06edf5ab80 1 -- 172.21.15.143:0/2549487270 >> 172.21.15.143:0/2549487270 conn(0x560d62a92660 msgr2=0x560d62b5a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:55.122 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.081+0000 7f06edf5ab80 1 -- 172.21.15.143:0/2549487270 shutdown_connections 2023-11-08T19:28:55.123 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.081+0000 7f06edf5ab80 1 -- 172.21.15.143:0/2549487270 wait complete. 2023-11-08T19:28:55.123 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.081+0000 7f06edf5ab80 1 -- start start 2023-11-08T19:28:55.123 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.081+0000 7f06edf5ab80 1 Processor -- start 2023-11-08T19:28:55.123 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.082+0000 7f06edf5ab80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560d62b176f0 0x560d62b17eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:55.124 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.082+0000 7f06edf5ab80 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x560d62b17ad0 0x560d62b183f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:55.124 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.082+0000 7f06edf5ab80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560d62b72580 0x560d62b47440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:55.124 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.082+0000 7f06edf5ab80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d62bccd40 con 0x560d62b72580 2023-11-08T19:28:55.124 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.082+0000 7f06edf5ab80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d62b71780 con 0x560d62b176f0 2023-11-08T19:28:55.124 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.082+0000 7f06edf5ab80 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x560d62b5d230 con 0x560d62b17ad0 2023-11-08T19:28:55.125 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.082+0000 7f06edf5ab80 10 client.0 _ll_register_callbacks cb 0x7f06d4003eb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T19:28:55.125 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.082+0000 7f06dbfff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560d62b72580 0x560d62b47440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:55.125 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.082+0000 7f06dbfff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560d62b72580 0x560d62b47440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55248/0 (socket says 172.21.15.143:55248) 2023-11-08T19:28:55.125 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.082+0000 7f06dbfff700 1 -- 172.21.15.143:0/4275211558 learned_addr learned my addr 172.21.15.143:0/4275211558 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:28:55.125 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.082+0000 7f06daffd700 1 --2- 172.21.15.143:0/4275211558 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x560d62b17ad0 0x560d62b183f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:55.126 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.082+0000 7f06db7fe700 1 --2- 172.21.15.143:0/4275211558 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560d62b176f0 0x560d62b17eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:55.126 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.082+0000 7f06dbfff700 1 -- 172.21.15.143:0/4275211558 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x560d62b17ad0 msgr2=0x560d62b183f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:55.126 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.082+0000 7f06dbfff700 1 --2- 172.21.15.143:0/4275211558 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x560d62b17ad0 0x560d62b183f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:55.126 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.082+0000 7f06dbfff700 1 -- 172.21.15.143:0/4275211558 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560d62b176f0 msgr2=0x560d62b17eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:55.126 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.082+0000 7f06dbfff700 1 --2- 172.21.15.143:0/4275211558 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560d62b176f0 0x560d62b17eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:55.127 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.082+0000 7f06dbfff700 1 -- 172.21.15.143:0/4275211558 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560d62899d90 con 0x560d62b72580 2023-11-08T19:28:55.127 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.082+0000 7f06dbfff700 1 --2- 172.21.15.143:0/4275211558 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560d62b72580 0x560d62b47440 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f06d400b730 tx=0x7f06d4024000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=609056aa4613b66f server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:55.127 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.082+0000 7f06ba7fc700 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2023-11-08T19:28:55.127 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.082+0000 7f06ba7fc700 1 -- 172.21.15.143:0/4275211558 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06d400e070 con 0x560d62b72580 2023-11-08T19:28:55.127 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.083+0000 7f06edf5ab80 1 -- 172.21.15.143:0/4275211558 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f06c4060630 con 0x560d62b72580 2023-11-08T19:28:55.128 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.083+0000 7f06ba7fc700 1 -- 172.21.15.143:0/4275211558 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 6887+0+0 (secure 0 0 0) 0x7f06d4022040 con 0x560d62b72580 2023-11-08T19:28:55.128 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.083+0000 7f06ba7fc700 1 -- 172.21.15.143:0/4275211558 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) v2 ==== 24+0+0 (secure 0 0 0) 0x7f06d401c410 con 0x560d62b72580 2023-11-08T19:28:55.128 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.083+0000 7f06edf5ab80 10 client.7305 fetch_fsmap learned FSMap version 64 2023-11-08T19:28:55.128 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.083+0000 7f06edf5ab80 1 -- 172.21.15.143:0/4275211558 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=2+}) v3 -- 0x560d628c0200 con 0x560d62b72580 2023-11-08T19:28:55.128 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.083+0000 7f06ba7fc700 1 -- 172.21.15.143:0/4275211558 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06c4060630 con 0x560d62b72580 2023-11-08T19:28:55.129 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.083+0000 7f06ba7fc700 1 -- 172.21.15.143:0/4275211558 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 64) v1 ==== 3306+0+0 (secure 0 0 0) 0x7f06d401c570 con 0x560d62b72580 2023-11-08T19:28:55.129 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.083+0000 7f06edf5ab80 10 client.7305 fetch_fsmap finished waiting for FSMap version 64 2023-11-08T19:28:55.129 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.083+0000 7f06edf5ab80 10 client.7305 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-08T19:28:55.129 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.083+0000 7f06edf5ab80 20 client.7305 populate_metadata read hostname 'smithi143' 2023-11-08T19:28:55.129 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.083+0000 7f06edf5ab80 1 --2- 172.21.15.143:0/4275211558 >> [v2:172.21.15.143:6833/3953059900,v1:172.21.15.143:6835/3953059900] conn(0x560d62b47980 0x560d62b47d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:55.129 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.083+0000 7f06edf5ab80 4 client.7305 mds_command: new command op to 6909 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-08T19:28:55.130 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.083+0000 7f06edf5ab80 1 -- 172.21.15.143:0/4275211558 --> [v2:172.21.15.143:6833/3953059900,v1:172.21.15.143:6835/3953059900] -- command(tid 0: {"prefix": "status"}) v1 -- 0x560d62a695a0 con 0x560d62b47980 2023-11-08T19:28:55.130 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.083+0000 7f06db7fe700 1 --2- 172.21.15.143:0/4275211558 >> [v2:172.21.15.143:6833/3953059900,v1:172.21.15.143:6835/3953059900] conn(0x560d62b47980 0x560d62b47d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:55.130 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.083+0000 7f06ba7fc700 10 client.7305 ms_handle_connect on v2:172.21.15.143:6833/3953059900 2023-11-08T19:28:55.130 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.083+0000 7f06db7fe700 1 --2- 172.21.15.143:0/4275211558 >> [v2:172.21.15.143:6833/3953059900,v1:172.21.15.143:6835/3953059900] conn(0x560d62b47980 0x560d62b47d60 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=322ed848fa3896e0 server_cookie=a8013b3531a9f5d7 in_seq=0 out_seq=0 2023-11-08T19:28:55.130 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.083+0000 7f06ba7fc700 1 -- 172.21.15.143:0/4275211558 <== mds.0 v2:172.21.15.143:6833/3953059900 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x560d62a695a0 con 0x560d62b47980 2023-11-08T19:28:55.131 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.083+0000 7f06ba7fc700 10 client.7305 handle_command_reply: tid=0 2023-11-08T19:28:55.131 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.083+0000 7f06edf5ab80 1 client.7305 shutdown 2023-11-08T19:28:55.131 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.084+0000 7f06edf5ab80 1 -- 172.21.15.143:0/4275211558 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560d62b72580 msgr2=0x560d62b47440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:55.131 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.084+0000 7f06edf5ab80 1 --2- 172.21.15.143:0/4275211558 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560d62b72580 0x560d62b47440 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f06d400b730 tx=0x7f06d4024000 comp rx=0 tx=0).stop 2023-11-08T19:28:55.131 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.084+0000 7f06edf5ab80 1 -- 172.21.15.143:0/4275211558 shutdown_connections 2023-11-08T19:28:55.132 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.084+0000 7f06edf5ab80 1 --2- 172.21.15.143:0/4275211558 >> [v2:172.21.15.143:6833/3953059900,v1:172.21.15.143:6835/3953059900] conn(0x560d62b47980 0x560d62b47d60 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:55.132 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.084+0000 7f06edf5ab80 1 --2- 172.21.15.143:0/4275211558 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x560d62b72580 0x560d62b47440 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:55.132 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.084+0000 7f06edf5ab80 1 --2- 172.21.15.143:0/4275211558 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560d62b176f0 0x560d62b17eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:55.132 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.084+0000 7f06edf5ab80 1 --2- 172.21.15.143:0/4275211558 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x560d62b17ad0 0x560d62b183f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:55.132 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.084+0000 7f06ba7fc700 0 client.7305 ms_handle_reset on v2:172.21.15.143:6833/3953059900 2023-11-08T19:28:55.133 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.084+0000 7f06edf5ab80 1 -- 172.21.15.143:0/4275211558 >> 172.21.15.143:0/4275211558 conn(0x560d62a92660 msgr2=0x560d62b32dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:55.133 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.084+0000 7f06edf5ab80 1 -- 172.21.15.143:0/4275211558 shutdown_connections 2023-11-08T19:28:55.133 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.084+0000 7f06edf5ab80 1 -- 172.21.15.143:0/4275211558 wait complete. 2023-11-08T19:28:55.133 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.084+0000 7f06edf5ab80 20 client.7305 trim_cache size 0 max 16384 2023-11-08T19:28:55.134 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2023-11-08T19:28:55.134 INFO:teuthology.orchestra.run.smithi143.stdout: "cluster_fsid": "968483af-77fc-4068-ad7c-5c2a1a4613d7", 2023-11-08T19:28:55.134 INFO:teuthology.orchestra.run.smithi143.stdout: "whoami": 0, 2023-11-08T19:28:55.134 INFO:teuthology.orchestra.run.smithi143.stdout: "id": 6909, 2023-11-08T19:28:55.134 INFO:teuthology.orchestra.run.smithi143.stdout: "want_state": "up:active", 2023-11-08T19:28:55.135 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "up:active", 2023-11-08T19:28:55.135 INFO:teuthology.orchestra.run.smithi143.stdout: "fs_name": "cephfs", 2023-11-08T19:28:55.135 INFO:teuthology.orchestra.run.smithi143.stdout: "rank_uptime": 6.7292142789999998, 2023-11-08T19:28:55.135 INFO:teuthology.orchestra.run.smithi143.stdout: "mdsmap_epoch": 64, 2023-11-08T19:28:55.135 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_epoch": 95, 2023-11-08T19:28:55.136 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_epoch_barrier": 95, 2023-11-08T19:28:55.136 INFO:teuthology.orchestra.run.smithi143.stdout: "uptime": 57.295252327999997 2023-11-08T19:28:55.136 INFO:teuthology.orchestra.run.smithi143.stdout:} 2023-11-08T19:28:55.137 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.084+0000 7f06e1d61700 1 -- 172.21.15.143:0/4034078012 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f06bc06af00 msgr2=0x7f06bc06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:55.137 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.084+0000 7f06e1d61700 1 --2- 172.21.15.143:0/4034078012 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f06bc06af00 0x7f06bc06d3c0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f06d400b370 tx=0x7f06d4009000 comp rx=0 tx=0).stop 2023-11-08T19:28:55.138 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.084+0000 7f06e1d61700 1 -- 172.21.15.143:0/4034078012 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f06dc06b150 msgr2=0x7f06dc152100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:55.138 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.084+0000 7f06e1d61700 1 --2- 172.21.15.143:0/4034078012 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f06dc06b150 0x7f06dc152100 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f06c400e7e0 tx=0x7f06c400eba0 comp rx=0 tx=0).stop 2023-11-08T19:28:55.138 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.084+0000 7f06e1d61700 1 -- 172.21.15.143:0/4034078012 shutdown_connections 2023-11-08T19:28:55.138 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.084+0000 7f06e1d61700 1 --2- 172.21.15.143:0/4034078012 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f06bc06af00 0x7f06bc06d3c0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:55.139 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.084+0000 7f06e1d61700 1 --2- 172.21.15.143:0/4034078012 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f06dc1618f0 0x7f06dc0c9c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:55.139 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.084+0000 7f06e1d61700 1 --2- 172.21.15.143:0/4034078012 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f06dc06b150 0x7f06dc152100 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:55.139 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.084+0000 7f06e1d61700 1 --2- 172.21.15.143:0/4034078012 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f06dc15dd00 0x7f06dc1f6eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:55.139 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.084+0000 7f06e1d61700 1 -- 172.21.15.143:0/4034078012 >> 172.21.15.143:0/4034078012 conn(0x7f06dc099ab0 msgr2=0x7f06dc0ad5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:55.139 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.084+0000 7f06e1d61700 1 -- 172.21.15.143:0/4034078012 shutdown_connections 2023-11-08T19:28:55.139 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.084+0000 7f06e1d61700 1 -- 172.21.15.143:0/4034078012 wait complete. 2023-11-08T19:28:55.140 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-11-08T19:28:55.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.260+0000 7f6d158c0700 1 Processor -- start 2023-11-08T19:28:55.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.260+0000 7f6d158c0700 1 -- start start 2023-11-08T19:28:55.264 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.261+0000 7f6d158c0700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f6d1015dd00 0x7f6d100cca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:55.264 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.261+0000 7f6d158c0700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6d1006b320 0x7f6d100cb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:55.264 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.261+0000 7f6d158c0700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d100ccfb0 con 0x7f6d101618f0 2023-11-08T19:28:55.265 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.261+0000 7f6d158c0700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d100cad50 con 0x7f6d1015dd00 2023-11-08T19:28:55.265 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.261+0000 7f6d158c0700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d10161df0 con 0x7f6d1006b320 2023-11-08T19:28:55.265 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.261+0000 7f6d0effd700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f6d1015dd00 0x7f6d100cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:55.265 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.261+0000 7f6d0effd700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f6d1015dd00 0x7f6d100cca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55254/0 (socket says 172.21.15.143:55254) 2023-11-08T19:28:55.265 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.261+0000 7f6d0effd700 1 -- 172.21.15.143:0/3283124014 learned_addr learned my addr 172.21.15.143:0/3283124014 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:28:55.266 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.262+0000 7f6d0f7fe700 1 --2- 172.21.15.143:0/3283124014 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6d1006b320 0x7f6d100cb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:55.266 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.262+0000 7f6d0effd700 1 -- 172.21.15.143:0/3283124014 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6d1006b320 msgr2=0x7f6d100cb1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:55.266 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.262+0000 7f6d0effd700 1 --2- 172.21.15.143:0/3283124014 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6d1006b320 0x7f6d100cb1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:55.266 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.262+0000 7f6d0effd700 1 -- 172.21.15.143:0/3283124014 >> v1:172.21.15.174:6789/0 conn(0x7f6d101618f0 legacy=0x7f6d1006ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:55.266 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.262+0000 7f6d0effd700 1 -- 172.21.15.143:0/3283124014 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d101620d0 con 0x7f6d1015dd00 2023-11-08T19:28:55.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.262+0000 7f6d0effd700 1 --2- 172.21.15.143:0/3283124014 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d1015dd00 0x7f6d100cca70 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f6d04006fd0 tx=0x7f6d04009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff74d950a2ff1c96 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:55.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.262+0000 7f6d0dffb700 1 -- 172.21.15.143:0/3283124014 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d04005e30 con 0x7f6d1015dd00 2023-11-08T19:28:55.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.262+0000 7f6d0dffb700 1 -- 172.21.15.143:0/3283124014 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d04006030 con 0x7f6d1015dd00 2023-11-08T19:28:55.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.262+0000 7f6d0dffb700 1 -- 172.21.15.143:0/3283124014 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d040063c0 con 0x7f6d1015dd00 2023-11-08T19:28:55.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.262+0000 7f6d158c0700 1 -- 172.21.15.143:0/3283124014 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d1015dd00 msgr2=0x7f6d100cca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:55.268 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.262+0000 7f6d158c0700 1 --2- 172.21.15.143:0/3283124014 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d1015dd00 0x7f6d100cca70 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f6d04006fd0 tx=0x7f6d04009bd0 comp rx=0 tx=0).stop 2023-11-08T19:28:55.268 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.262+0000 7f6d158c0700 1 -- 172.21.15.143:0/3283124014 shutdown_connections 2023-11-08T19:28:55.268 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.262+0000 7f6d158c0700 1 --2- 172.21.15.143:0/3283124014 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6d1006b320 0x7f6d100cb1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:55.268 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.262+0000 7f6d158c0700 1 --2- 172.21.15.143:0/3283124014 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d1015dd00 0x7f6d100cca70 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:55.268 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.262+0000 7f6d158c0700 1 -- 172.21.15.143:0/3283124014 >> 172.21.15.143:0/3283124014 conn(0x7f6d10099ab0 msgr2=0x7f6d100d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:55.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.263+0000 7f6d158c0700 1 -- 172.21.15.143:0/3283124014 shutdown_connections 2023-11-08T19:28:55.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.263+0000 7f6d158c0700 1 -- 172.21.15.143:0/3283124014 wait complete. 2023-11-08T19:28:55.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.264+0000 7f6d158c0700 1 Processor -- start 2023-11-08T19:28:55.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.264+0000 7f6d158c0700 1 -- start start 2023-11-08T19:28:55.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.265+0000 7f6d158c0700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d1006b320 0x7f6d101f31f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:55.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.265+0000 7f6d158c0700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6d1015dd00 0x7f6d100c4230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:55.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.265+0000 7f6d158c0700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6d101618f0 0x7f6d100c77d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:55.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.265+0000 7f6d158c0700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d100c4bf0 con 0x7f6d1006b320 2023-11-08T19:28:55.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.265+0000 7f6d158c0700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d100c4ed0 con 0x7f6d1015dd00 2023-11-08T19:28:55.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.265+0000 7f6d158c0700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d100c51b0 con 0x7f6d101618f0 2023-11-08T19:28:55.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.265+0000 7f6d0effd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d1006b320 0x7f6d101f31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:55.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.265+0000 7f6d0effd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d1006b320 0x7f6d101f31f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55256/0 (socket says 172.21.15.143:55256) 2023-11-08T19:28:55.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.265+0000 7f6d0effd700 1 -- 172.21.15.143:0/2395482634 learned_addr learned my addr 172.21.15.143:0/2395482634 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:28:55.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.265+0000 7f6d0f7fe700 1 --2- 172.21.15.143:0/2395482634 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6d101618f0 0x7f6d100c77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:55.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.265+0000 7f6d0e7fc700 1 --2- 172.21.15.143:0/2395482634 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6d1015dd00 0x7f6d100c4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:55.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.265+0000 7f6d0effd700 1 -- 172.21.15.143:0/2395482634 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6d101618f0 msgr2=0x7f6d100c77d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:55.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.265+0000 7f6d0effd700 1 --2- 172.21.15.143:0/2395482634 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6d101618f0 0x7f6d100c77d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:55.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.265+0000 7f6d0effd700 1 -- 172.21.15.143:0/2395482634 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6d1015dd00 msgr2=0x7f6d100c4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:55.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.265+0000 7f6d0effd700 1 --2- 172.21.15.143:0/2395482634 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6d1015dd00 0x7f6d100c4230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:55.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.265+0000 7f6d0effd700 1 -- 172.21.15.143:0/2395482634 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d04009500 con 0x7f6d1006b320 2023-11-08T19:28:55.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.265+0000 7f6d0f7fe700 1 --2- 172.21.15.143:0/2395482634 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6d101618f0 0x7f6d100c77d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:28:55.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.265+0000 7f6d0effd700 1 --2- 172.21.15.143:0/2395482634 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d1006b320 0x7f6d101f31f0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f6d04000c00 tx=0x7f6d04004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:55.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.265+0000 7f6cf7fff700 1 -- 172.21.15.143:0/2395482634 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d04006950 con 0x7f6d1006b320 2023-11-08T19:28:55.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.265+0000 7f6cf7fff700 1 -- 172.21.15.143:0/2395482634 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d04020070 con 0x7f6d1006b320 2023-11-08T19:28:55.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.265+0000 7f6d158c0700 1 -- 172.21.15.143:0/2395482634 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d101f3af0 con 0x7f6d1006b320 2023-11-08T19:28:55.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.265+0000 7f6cf7fff700 1 -- 172.21.15.143:0/2395482634 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d04025650 con 0x7f6d1006b320 2023-11-08T19:28:55.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.265+0000 7f6d158c0700 1 -- 172.21.15.143:0/2395482634 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d100c93a0 con 0x7f6d1006b320 2023-11-08T19:28:55.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.267+0000 7f6cf7fff700 1 -- 172.21.15.143:0/2395482634 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6d04022030 con 0x7f6d1006b320 2023-11-08T19:28:55.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.267+0000 7f6cf7fff700 1 --2- 172.21.15.143:0/2395482634 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f6cfc06af00 0x7f6cfc06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:55.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.268+0000 7f6cf7fff700 1 -- 172.21.15.143:0/2395482634 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6887+0+0 (secure 0 0 0) 0x7f6d0408b100 con 0x7f6d1006b320 2023-11-08T19:28:55.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.268+0000 7f6d0e7fc700 1 --2- 172.21.15.143:0/2395482634 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f6cfc06af00 0x7f6cfc06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:55.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.268+0000 7f6d0e7fc700 1 --2- 172.21.15.143:0/2395482634 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f6cfc06af00 0x7f6cfc06d3c0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f6d100c8d70 tx=0x7f6d00008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:55.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.269+0000 7f6d21ab9b80 1 Processor -- start 2023-11-08T19:28:55.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.269+0000 7f6d21ab9b80 1 -- start start 2023-11-08T19:28:55.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.270+0000 7f6d21ab9b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f814a706f0 0x55f814aa0450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:55.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.270+0000 7f6d21ab9b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f814a72c30 0x55f814aa0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:55.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.270+0000 7f6d21ab9b80 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55f814a70ad0 0x55f814a8af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:55.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.270+0000 7f6d21ab9b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8149dc6e0 con 0x55f814a706f0 2023-11-08T19:28:55.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.270+0000 7f6d21ab9b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8149d03b0 con 0x55f814a72c30 2023-11-08T19:28:55.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.270+0000 7f6d21ab9b80 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f8149cdcd0 con 0x55f814a70ad0 2023-11-08T19:28:55.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.270+0000 7f6d0effd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f814a706f0 0x55f814aa0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:55.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.270+0000 7f6d0effd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f814a706f0 0x55f814aa0450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55264/0 (socket says 172.21.15.143:55264) 2023-11-08T19:28:55.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.270+0000 7f6d0effd700 1 -- 172.21.15.143:0/757790900 learned_addr learned my addr 172.21.15.143:0/757790900 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:28:55.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.270+0000 7f6d0f7fe700 1 --2- 172.21.15.143:0/757790900 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55f814a70ad0 0x55f814a8af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:55.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.270+0000 7f6d0e7fc700 1 --2- 172.21.15.143:0/757790900 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f814a72c30 0x55f814aa0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:55.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.271+0000 7f6d0effd700 1 -- 172.21.15.143:0/757790900 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55f814a70ad0 msgr2=0x55f814a8af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:55.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.271+0000 7f6d0effd700 1 --2- 172.21.15.143:0/757790900 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55f814a70ad0 0x55f814a8af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:55.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.271+0000 7f6d0effd700 1 -- 172.21.15.143:0/757790900 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f814a72c30 msgr2=0x55f814aa0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:55.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.271+0000 7f6d0effd700 1 --2- 172.21.15.143:0/757790900 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f814a72c30 0x55f814aa0990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:55.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.271+0000 7f6d0effd700 1 -- 172.21.15.143:0/757790900 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f8148465e0 con 0x55f814a706f0 2023-11-08T19:28:55.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.271+0000 7f6d0effd700 1 --2- 172.21.15.143:0/757790900 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f814a706f0 0x55f814aa0450 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f6d04061830 tx=0x7f6d04059ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c533b2de7585610 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:55.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.271+0000 7f6cf57fa700 1 -- 172.21.15.143:0/757790900 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d04061d60 con 0x55f814a706f0 2023-11-08T19:28:55.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.271+0000 7f6cf57fa700 1 -- 172.21.15.143:0/757790900 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d04084030 con 0x55f814a706f0 2023-11-08T19:28:55.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.271+0000 7f6cf57fa700 1 -- 172.21.15.143:0/757790900 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d040711c0 con 0x55f814a706f0 2023-11-08T19:28:55.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.272+0000 7f6d21ab9b80 1 -- 172.21.15.143:0/757790900 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f814a706f0 msgr2=0x55f814aa0450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:55.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.272+0000 7f6d21ab9b80 1 --2- 172.21.15.143:0/757790900 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f814a706f0 0x55f814aa0450 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f6d04061830 tx=0x7f6d04059ae0 comp rx=0 tx=0).stop 2023-11-08T19:28:55.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.272+0000 7f6d21ab9b80 1 -- 172.21.15.143:0/757790900 shutdown_connections 2023-11-08T19:28:55.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.272+0000 7f6d21ab9b80 1 --2- 172.21.15.143:0/757790900 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55f814a70ad0 0x55f814a8af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:55.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.272+0000 7f6d21ab9b80 1 --2- 172.21.15.143:0/757790900 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f814a706f0 0x55f814aa0450 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:55.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.272+0000 7f6d21ab9b80 1 --2- 172.21.15.143:0/757790900 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f814a72c30 0x55f814aa0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:55.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.272+0000 7f6d21ab9b80 1 -- 172.21.15.143:0/757790900 >> 172.21.15.143:0/757790900 conn(0x55f8149eb660 msgr2=0x55f814a763c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:55.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.272+0000 7f6d21ab9b80 1 -- 172.21.15.143:0/757790900 shutdown_connections 2023-11-08T19:28:55.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.272+0000 7f6d21ab9b80 1 -- 172.21.15.143:0/757790900 wait complete. 2023-11-08T19:28:55.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.272+0000 7f6d21ab9b80 1 -- start start 2023-11-08T19:28:55.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.272+0000 7f6d21ab9b80 1 Processor -- start 2023-11-08T19:28:55.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.273+0000 7f6d21ab9b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f814a706f0 0x55f814acac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:55.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.273+0000 7f6d21ab9b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f814a70ad0 0x55f814a8af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:55.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.273+0000 7f6d21ab9b80 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55f814ab5a30 0x55f814ab3d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:55.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.273+0000 7f6d21ab9b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8149c95c0 con 0x55f814a70ad0 2023-11-08T19:28:55.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.273+0000 7f6d21ab9b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8149650d0 con 0x55f814a706f0 2023-11-08T19:28:55.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.273+0000 7f6d21ab9b80 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f814964d70 con 0x55f814ab5a30 2023-11-08T19:28:55.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.273+0000 7f6d21ab9b80 10 client.0 _ll_register_callbacks cb 0x55f8147c17b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T19:28:55.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.273+0000 7f6d0e7fc700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f814a70ad0 0x55f814a8af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:55.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.273+0000 7f6d0e7fc700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f814a70ad0 0x55f814a8af50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55272/0 (socket says 172.21.15.143:55272) 2023-11-08T19:28:55.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.273+0000 7f6d0e7fc700 1 -- 172.21.15.143:0/3521343960 learned_addr learned my addr 172.21.15.143:0/3521343960 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:28:55.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.274+0000 7f6d0f7fe700 1 --2- 172.21.15.143:0/3521343960 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55f814ab5a30 0x55f814ab3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:55.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.274+0000 7f6d0e7fc700 1 -- 172.21.15.143:0/3521343960 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55f814ab5a30 msgr2=0x55f814ab3d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:55.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.274+0000 7f6d0e7fc700 1 --2- 172.21.15.143:0/3521343960 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55f814ab5a30 0x55f814ab3d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:55.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.274+0000 7f6d0e7fc700 1 -- 172.21.15.143:0/3521343960 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f814a706f0 msgr2=0x55f814acac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:55.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.274+0000 7f6d0effd700 1 --2- 172.21.15.143:0/3521343960 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f814a706f0 0x55f814acac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:55.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.274+0000 7f6d0e7fc700 1 --2- 172.21.15.143:0/3521343960 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f814a706f0 0x55f814acac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:55.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.274+0000 7f6d0e7fc700 1 -- 172.21.15.143:0/3521343960 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d100c93a0 con 0x55f814a70ad0 2023-11-08T19:28:55.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.274+0000 7f6d0e7fc700 1 --2- 172.21.15.143:0/3521343960 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f814a70ad0 0x55f814a8af50 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f6d00003730 tx=0x7f6d00004780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3855438c6a372932 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:55.293 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.274+0000 7f6cee7fc700 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2023-11-08T19:28:55.293 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.274+0000 7f6cee7fc700 1 -- 172.21.15.143:0/3521343960 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d00013760 con 0x55f814a70ad0 2023-11-08T19:28:55.293 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.274+0000 7f6cee7fc700 1 -- 172.21.15.143:0/3521343960 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 6887+0+0 (secure 0 0 0) 0x7f6d000138c0 con 0x55f814a70ad0 2023-11-08T19:28:55.293 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.274+0000 7f6d21ab9b80 1 -- 172.21.15.143:0/3521343960 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f814a68880 con 0x55f814a70ad0 2023-11-08T19:28:55.293 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.274+0000 7f6cee7fc700 1 -- 172.21.15.143:0/3521343960 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6d00027ea0 con 0x55f814a70ad0 2023-11-08T19:28:55.294 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.274+0000 7f6d21ab9b80 10 client.7317 fetch_fsmap learned FSMap version 64 2023-11-08T19:28:55.294 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.274+0000 7f6d21ab9b80 1 -- 172.21.15.143:0/3521343960 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=2+}) v3 -- 0x55f8147f2d90 con 0x55f814a70ad0 2023-11-08T19:28:55.294 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.274+0000 7f6cee7fc700 1 -- 172.21.15.143:0/3521343960 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f814a68880 con 0x55f814a70ad0 2023-11-08T19:28:55.294 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.275+0000 7f6cee7fc700 1 -- 172.21.15.143:0/3521343960 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 64) v1 ==== 3306+0+0 (secure 0 0 0) 0x7f6d00013ad0 con 0x55f814a70ad0 2023-11-08T19:28:55.294 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.275+0000 7f6d21ab9b80 10 client.7317 fetch_fsmap finished waiting for FSMap version 64 2023-11-08T19:28:55.295 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.275+0000 7f6d21ab9b80 10 client.7317 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-08T19:28:55.295 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.275+0000 7f6d21ab9b80 20 client.7317 populate_metadata read hostname 'smithi143' 2023-11-08T19:28:55.295 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.275+0000 7f6d21ab9b80 1 --2- 172.21.15.143:0/3521343960 >> [v2:172.21.15.174:6834/3601088618,v1:172.21.15.174:6836/3601088618] conn(0x55f814ad6400 0x55f814ad87f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:55.295 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.275+0000 7f6d21ab9b80 4 client.7317 mds_command: new command op to 6918 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-08T19:28:55.295 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.275+0000 7f6d21ab9b80 1 -- 172.21.15.143:0/3521343960 --> [v2:172.21.15.174:6834/3601088618,v1:172.21.15.174:6836/3601088618] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f814a44a30 con 0x55f814ad6400 2023-11-08T19:28:55.296 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.275+0000 7f6d0effd700 1 --2- 172.21.15.143:0/3521343960 >> [v2:172.21.15.174:6834/3601088618,v1:172.21.15.174:6836/3601088618] conn(0x55f814ad6400 0x55f814ad87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:55.296 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.275+0000 7f6d0effd700 1 --2- 172.21.15.143:0/3521343960 >> [v2:172.21.15.174:6834/3601088618,v1:172.21.15.174:6836/3601088618] conn(0x55f814ad6400 0x55f814ad87f0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e67c613c45b8c90 server_cookie=59aa62289b7cbd6d in_seq=0 out_seq=0 2023-11-08T19:28:55.296 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.275+0000 7f6cee7fc700 10 client.7317 ms_handle_connect on v2:172.21.15.174:6834/3601088618 2023-11-08T19:28:55.304 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.276+0000 7f6cee7fc700 1 -- 172.21.15.143:0/3521343960 <== mds.1 v2:172.21.15.174:6834/3601088618 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f814a44a30 con 0x55f814ad6400 2023-11-08T19:28:55.304 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.276+0000 7f6cee7fc700 10 client.7317 handle_command_reply: tid=0 2023-11-08T19:28:55.304 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.276+0000 7f6d21ab9b80 1 client.7317 shutdown 2023-11-08T19:28:55.304 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.276+0000 7f6d21ab9b80 1 -- 172.21.15.143:0/3521343960 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f814a70ad0 msgr2=0x55f814a8af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:55.305 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.276+0000 7f6d21ab9b80 1 --2- 172.21.15.143:0/3521343960 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f814a70ad0 0x55f814a8af50 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f6d00003730 tx=0x7f6d00004780 comp rx=0 tx=0).stop 2023-11-08T19:28:55.305 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.276+0000 7f6d21ab9b80 1 -- 172.21.15.143:0/3521343960 shutdown_connections 2023-11-08T19:28:55.305 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.276+0000 7f6d21ab9b80 1 --2- 172.21.15.143:0/3521343960 >> [v2:172.21.15.174:6834/3601088618,v1:172.21.15.174:6836/3601088618] conn(0x55f814ad6400 0x55f814ad87f0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:55.305 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.276+0000 7f6d21ab9b80 1 --2- 172.21.15.143:0/3521343960 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55f814ab5a30 0x55f814ab3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:55.305 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.276+0000 7f6d21ab9b80 1 --2- 172.21.15.143:0/3521343960 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f814a706f0 0x55f814acac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:55.306 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.276+0000 7f6cee7fc700 0 client.7317 ms_handle_reset on v2:172.21.15.174:6834/3601088618 2023-11-08T19:28:55.306 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.276+0000 7f6d21ab9b80 1 --2- 172.21.15.143:0/3521343960 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f814a70ad0 0x55f814a8af50 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:55.306 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.276+0000 7f6d21ab9b80 1 -- 172.21.15.143:0/3521343960 >> 172.21.15.143:0/3521343960 conn(0x55f8149eb660 msgr2=0x55f814a75bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:55.307 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.277+0000 7f6d21ab9b80 1 -- 172.21.15.143:0/3521343960 shutdown_connections 2023-11-08T19:28:55.307 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.277+0000 7f6d21ab9b80 1 -- 172.21.15.143:0/3521343960 wait complete. 2023-11-08T19:28:55.307 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.277+0000 7f6d21ab9b80 20 client.7317 trim_cache size 0 max 16384 2023-11-08T19:28:55.307 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.294+0000 7f6d21ab9b80 1 Processor -- start 2023-11-08T19:28:55.307 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.294+0000 7f6d21ab9b80 1 -- start start 2023-11-08T19:28:55.308 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.295+0000 7f6d21ab9b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f814a706f0 0x55f814a8af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:55.308 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.295+0000 7f6d21ab9b80 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55f814a70ad0 0x55f814ab2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:55.308 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.295+0000 7f6d21ab9b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f814ab5a30 0x55f814ab2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:55.308 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.295+0000 7f6d21ab9b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8148488d0 con 0x55f814a706f0 2023-11-08T19:28:55.308 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.295+0000 7f6d21ab9b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f814a74ff0 con 0x55f814ab5a30 2023-11-08T19:28:55.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.295+0000 7f6d21ab9b80 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f814a74dc0 con 0x55f814a70ad0 2023-11-08T19:28:55.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.295+0000 7f6d0effd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f814a706f0 0x55f814a8af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:55.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.295+0000 7f6d0effd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f814a706f0 0x55f814a8af90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55278/0 (socket says 172.21.15.143:55278) 2023-11-08T19:28:55.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.295+0000 7f6d0effd700 1 -- 172.21.15.143:0/929673400 learned_addr learned my addr 172.21.15.143:0/929673400 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:28:55.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.296+0000 7f6d0effd700 1 -- 172.21.15.143:0/929673400 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55f814a70ad0 msgr2=0x55f814ab2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:55.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.296+0000 7f6d0effd700 1 --2- 172.21.15.143:0/929673400 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55f814a70ad0 0x55f814ab2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:55.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.296+0000 7f6d0effd700 1 -- 172.21.15.143:0/929673400 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f814ab5a30 msgr2=0x55f814ab2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:55.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.296+0000 7f6d0effd700 1 --2- 172.21.15.143:0/929673400 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f814ab5a30 0x55f814ab2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:55.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.296+0000 7f6d0effd700 1 -- 172.21.15.143:0/929673400 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f8147dbc10 con 0x55f814a706f0 2023-11-08T19:28:55.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.296+0000 7f6d0effd700 1 --2- 172.21.15.143:0/929673400 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f814a706f0 0x55f814a8af90 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f6d04059b20 tx=0x7f6d04012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=375f406f3f826ad4 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:55.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.296+0000 7f6cf5ffb700 1 -- 172.21.15.143:0/929673400 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d04057e00 con 0x55f814a706f0 2023-11-08T19:28:55.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.296+0000 7f6cf5ffb700 1 -- 172.21.15.143:0/929673400 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d040782c0 con 0x55f814a706f0 2023-11-08T19:28:55.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.296+0000 7f6cf5ffb700 1 -- 172.21.15.143:0/929673400 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d04078430 con 0x55f814a706f0 2023-11-08T19:28:55.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.296+0000 7f6d21ab9b80 1 -- 172.21.15.143:0/929673400 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f814a706f0 msgr2=0x55f814a8af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:55.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.296+0000 7f6d21ab9b80 1 --2- 172.21.15.143:0/929673400 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f814a706f0 0x55f814a8af90 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f6d04059b20 tx=0x7f6d04012000 comp rx=0 tx=0).stop 2023-11-08T19:28:55.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.296+0000 7f6d21ab9b80 1 -- 172.21.15.143:0/929673400 shutdown_connections 2023-11-08T19:28:55.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.296+0000 7f6d21ab9b80 1 --2- 172.21.15.143:0/929673400 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f814ab5a30 0x55f814ab2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:55.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.296+0000 7f6d21ab9b80 1 --2- 172.21.15.143:0/929673400 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f814a706f0 0x55f814a8af90 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:55.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.296+0000 7f6d21ab9b80 1 --2- 172.21.15.143:0/929673400 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55f814a70ad0 0x55f814ab2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:55.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.296+0000 7f6d21ab9b80 1 -- 172.21.15.143:0/929673400 >> 172.21.15.143:0/929673400 conn(0x55f8149eb660 msgr2=0x55f814ada9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:55.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.297+0000 7f6d21ab9b80 1 -- 172.21.15.143:0/929673400 shutdown_connections 2023-11-08T19:28:55.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.297+0000 7f6d21ab9b80 1 -- 172.21.15.143:0/929673400 wait complete. 2023-11-08T19:28:55.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.297+0000 7f6d21ab9b80 1 -- start start 2023-11-08T19:28:55.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.297+0000 7f6d21ab9b80 1 Processor -- start 2023-11-08T19:28:55.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.298+0000 7f6d21ab9b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f814a706f0 0x55f814ab2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:55.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.298+0000 7f6d21ab9b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f814a70ad0 0x55f814ab2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:55.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.298+0000 7f6d21ab9b80 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55f814ab5a30 0x55f814b24d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:55.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.298+0000 7f6d21ab9b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f814acaea0 con 0x55f814a70ad0 2023-11-08T19:28:55.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.298+0000 7f6d21ab9b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f814aa1890 con 0x55f814a706f0 2023-11-08T19:28:55.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.298+0000 7f6d21ab9b80 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f814acd230 con 0x55f814ab5a30 2023-11-08T19:28:55.315 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.298+0000 7f6d21ab9b80 10 client.0 _ll_register_callbacks cb 0x7f6d000043f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T19:28:55.315 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.298+0000 7f6d0e7fc700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f814a70ad0 0x55f814ab2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:55.315 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.298+0000 7f6d0e7fc700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f814a70ad0 0x55f814ab2db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55286/0 (socket says 172.21.15.143:55286) 2023-11-08T19:28:55.315 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.298+0000 7f6d0e7fc700 1 -- 172.21.15.143:0/831288313 learned_addr learned my addr 172.21.15.143:0/831288313 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:28:55.315 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.298+0000 7f6d0effd700 1 --2- 172.21.15.143:0/831288313 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f814a706f0 0x55f814ab2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:55.316 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.298+0000 7f6d0f7fe700 1 --2- 172.21.15.143:0/831288313 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55f814ab5a30 0x55f814b24d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:55.316 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.298+0000 7f6d0e7fc700 1 -- 172.21.15.143:0/831288313 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55f814ab5a30 msgr2=0x55f814b24d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:55.316 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.298+0000 7f6d0e7fc700 1 --2- 172.21.15.143:0/831288313 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55f814ab5a30 0x55f814b24d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:55.316 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.298+0000 7f6d0e7fc700 1 -- 172.21.15.143:0/831288313 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f814a706f0 msgr2=0x55f814ab2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:55.316 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.298+0000 7f6d0e7fc700 1 --2- 172.21.15.143:0/831288313 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f814a706f0 0x55f814ab2870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:55.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.298+0000 7f6d0e7fc700 1 -- 172.21.15.143:0/831288313 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f8148465e0 con 0x55f814a70ad0 2023-11-08T19:28:55.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.298+0000 7f6d0e7fc700 1 --2- 172.21.15.143:0/831288313 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f814a70ad0 0x55f814ab2db0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f6d00003730 tx=0x7f6d00014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9c9f0940a7f1923 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:55.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.298+0000 7f6cee7fc700 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2023-11-08T19:28:55.318 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.298+0000 7f6cee7fc700 1 -- 172.21.15.143:0/831288313 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d0000a500 con 0x55f814a70ad0 2023-11-08T19:28:55.318 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.299+0000 7f6cee7fc700 1 -- 172.21.15.143:0/831288313 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 6887+0+0 (secure 0 0 0) 0x7f6d00003e80 con 0x55f814a70ad0 2023-11-08T19:28:55.318 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.299+0000 7f6d21ab9b80 1 -- 172.21.15.143:0/831288313 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6d04084030 con 0x55f814a70ad0 2023-11-08T19:28:55.318 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.299+0000 7f6cee7fc700 1 -- 172.21.15.143:0/831288313 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6d0001f480 con 0x55f814a70ad0 2023-11-08T19:28:55.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.299+0000 7f6d21ab9b80 10 client.7323 fetch_fsmap learned FSMap version 64 2023-11-08T19:28:55.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.299+0000 7f6d21ab9b80 1 -- 172.21.15.143:0/831288313 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=2+}) v3 -- 0x55f814819200 con 0x55f814a70ad0 2023-11-08T19:28:55.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.299+0000 7f6cee7fc700 1 -- 172.21.15.143:0/831288313 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d04084030 con 0x55f814a70ad0 2023-11-08T19:28:55.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.299+0000 7f6cee7fc700 1 -- 172.21.15.143:0/831288313 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 64) v1 ==== 3306+0+0 (secure 0 0 0) 0x7f6d0001f5e0 con 0x55f814a70ad0 2023-11-08T19:28:55.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.299+0000 7f6d21ab9b80 10 client.7323 fetch_fsmap finished waiting for FSMap version 64 2023-11-08T19:28:55.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.299+0000 7f6d21ab9b80 10 client.7323 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-08T19:28:55.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.299+0000 7f6d21ab9b80 20 client.7323 populate_metadata read hostname 'smithi143' 2023-11-08T19:28:55.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.299+0000 7f6d21ab9b80 1 --2- 172.21.15.143:0/831288313 >> [v2:172.21.15.174:6834/3601088618,v1:172.21.15.174:6836/3601088618] conn(0x55f814b252a0 0x55f814b25680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:55.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.299+0000 7f6d21ab9b80 4 client.7323 mds_command: new command op to 6918 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-08T19:28:55.321 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.299+0000 7f6d21ab9b80 1 -- 172.21.15.143:0/831288313 --> [v2:172.21.15.174:6834/3601088618,v1:172.21.15.174:6836/3601088618] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55f8149c25a0 con 0x55f814b252a0 2023-11-08T19:28:55.321 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.300+0000 7f6d0effd700 1 --2- 172.21.15.143:0/831288313 >> [v2:172.21.15.174:6834/3601088618,v1:172.21.15.174:6836/3601088618] conn(0x55f814b252a0 0x55f814b25680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:55.321 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.300+0000 7f6cee7fc700 10 client.7323 ms_handle_connect on v2:172.21.15.174:6834/3601088618 2023-11-08T19:28:55.321 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.300+0000 7f6d0effd700 1 --2- 172.21.15.143:0/831288313 >> [v2:172.21.15.174:6834/3601088618,v1:172.21.15.174:6836/3601088618] conn(0x55f814b252a0 0x55f814b25680 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3c0cf216fba0808b server_cookie=4bdf8ebe54ac12fe in_seq=0 out_seq=0 2023-11-08T19:28:55.321 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.300+0000 7f6cee7fc700 1 -- 172.21.15.143:0/831288313 <== mds.1 v2:172.21.15.174:6834/3601088618 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x55f8149c25a0 con 0x55f814b252a0 2023-11-08T19:28:55.322 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.300+0000 7f6cee7fc700 10 client.7323 handle_command_reply: tid=0 2023-11-08T19:28:55.322 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.300+0000 7f6d21ab9b80 1 client.7323 shutdown 2023-11-08T19:28:55.322 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.300+0000 7f6d21ab9b80 1 -- 172.21.15.143:0/831288313 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f814a70ad0 msgr2=0x55f814ab2db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:55.322 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.300+0000 7f6d21ab9b80 1 --2- 172.21.15.143:0/831288313 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f814a70ad0 0x55f814ab2db0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f6d00003730 tx=0x7f6d00014000 comp rx=0 tx=0).stop 2023-11-08T19:28:55.322 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.300+0000 7f6d21ab9b80 1 -- 172.21.15.143:0/831288313 shutdown_connections 2023-11-08T19:28:55.323 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.300+0000 7f6d21ab9b80 1 --2- 172.21.15.143:0/831288313 >> [v2:172.21.15.174:6834/3601088618,v1:172.21.15.174:6836/3601088618] conn(0x55f814b252a0 0x55f814b25680 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:55.323 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.300+0000 7f6d21ab9b80 1 --2- 172.21.15.143:0/831288313 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55f814ab5a30 0x55f814b24d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:55.323 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.300+0000 7f6d21ab9b80 1 --2- 172.21.15.143:0/831288313 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f814a706f0 0x55f814ab2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:55.323 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.300+0000 7f6cee7fc700 0 client.7323 ms_handle_reset on v2:172.21.15.174:6834/3601088618 2023-11-08T19:28:55.323 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.300+0000 7f6d21ab9b80 1 --2- 172.21.15.143:0/831288313 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f814a70ad0 0x55f814ab2db0 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:55.324 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.300+0000 7f6d21ab9b80 1 -- 172.21.15.143:0/831288313 >> 172.21.15.143:0/831288313 conn(0x55f8149eb660 msgr2=0x55f814a8ad60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:55.324 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.300+0000 7f6d21ab9b80 1 -- 172.21.15.143:0/831288313 shutdown_connections 2023-11-08T19:28:55.324 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.300+0000 7f6d21ab9b80 1 -- 172.21.15.143:0/831288313 wait complete. 2023-11-08T19:28:55.324 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.300+0000 7f6d21ab9b80 20 client.7323 trim_cache size 0 max 16384 2023-11-08T19:28:55.325 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2023-11-08T19:28:55.325 INFO:teuthology.orchestra.run.smithi143.stdout: "cluster_fsid": "968483af-77fc-4068-ad7c-5c2a1a4613d7", 2023-11-08T19:28:55.325 INFO:teuthology.orchestra.run.smithi143.stdout: "whoami": 1, 2023-11-08T19:28:55.325 INFO:teuthology.orchestra.run.smithi143.stdout: "id": 6918, 2023-11-08T19:28:55.326 INFO:teuthology.orchestra.run.smithi143.stdout: "want_state": "up:active", 2023-11-08T19:28:55.326 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "up:active", 2023-11-08T19:28:55.326 INFO:teuthology.orchestra.run.smithi143.stdout: "fs_name": "cephfs", 2023-11-08T19:28:55.326 INFO:teuthology.orchestra.run.smithi143.stdout: "rank_uptime": 37.70663768, 2023-11-08T19:28:55.326 INFO:teuthology.orchestra.run.smithi143.stdout: "mdsmap_epoch": 64, 2023-11-08T19:28:55.327 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_epoch": 95, 2023-11-08T19:28:55.327 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_epoch_barrier": 95, 2023-11-08T19:28:55.327 INFO:teuthology.orchestra.run.smithi143.stdout: "uptime": 57.497853057999997 2023-11-08T19:28:55.328 INFO:teuthology.orchestra.run.smithi143.stdout:} 2023-11-08T19:28:55.329 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.301+0000 7f6d158c0700 1 -- 172.21.15.143:0/2395482634 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f6cfc06af00 msgr2=0x7f6cfc06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:55.330 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.301+0000 7f6d158c0700 1 --2- 172.21.15.143:0/2395482634 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f6cfc06af00 0x7f6cfc06d3c0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f6d100c8d70 tx=0x7f6d00008040 comp rx=0 tx=0).stop 2023-11-08T19:28:55.330 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.301+0000 7f6d158c0700 1 -- 172.21.15.143:0/2395482634 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d1006b320 msgr2=0x7f6d101f31f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:55.330 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.301+0000 7f6d158c0700 1 --2- 172.21.15.143:0/2395482634 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d1006b320 0x7f6d101f31f0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f6d04000c00 tx=0x7f6d04004750 comp rx=0 tx=0).stop 2023-11-08T19:28:55.330 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.301+0000 7f6d158c0700 1 -- 172.21.15.143:0/2395482634 shutdown_connections 2023-11-08T19:28:55.330 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.301+0000 7f6d158c0700 1 --2- 172.21.15.143:0/2395482634 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f6cfc06af00 0x7f6cfc06d3c0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:55.331 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.301+0000 7f6d158c0700 1 --2- 172.21.15.143:0/2395482634 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6d101618f0 0x7f6d100c77d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:55.331 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.301+0000 7f6d158c0700 1 --2- 172.21.15.143:0/2395482634 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d1006b320 0x7f6d101f31f0 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:55.331 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.301+0000 7f6d158c0700 1 --2- 172.21.15.143:0/2395482634 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6d1015dd00 0x7f6d100c4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:55.331 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.301+0000 7f6d158c0700 1 -- 172.21.15.143:0/2395482634 >> 172.21.15.143:0/2395482634 conn(0x7f6d10099ab0 msgr2=0x7f6d1015cfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:55.331 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.301+0000 7f6d158c0700 1 -- 172.21.15.143:0/2395482634 shutdown_connections 2023-11-08T19:28:55.332 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.301+0000 7f6d158c0700 1 -- 172.21.15.143:0/2395482634 wait complete. 2023-11-08T19:28:55.332 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:28:55.456 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.452+0000 7f36cb3c2700 1 Processor -- start 2023-11-08T19:28:55.456 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.452+0000 7f36cb3c2700 1 -- start start 2023-11-08T19:28:55.456 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.453+0000 7f36cb3c2700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f36c406c730 0x7f36c40cd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:55.457 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.453+0000 7f36cb3c2700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f36c406c030 0x7f36c406cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:55.457 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.453+0000 7f36cb3c2700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36c40cd480 con 0x7f36c415de80 2023-11-08T19:28:55.457 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.453+0000 7f36cb3c2700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36c40cc610 con 0x7f36c406c030 2023-11-08T19:28:55.457 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.453+0000 7f36cb3c2700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36c406d050 con 0x7f36c406c730 2023-11-08T19:28:55.457 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.453+0000 7f36c895d700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f36c406c030 0x7f36c406cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:55.458 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.453+0000 7f36c895d700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f36c406c030 0x7f36c406cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55292/0 (socket says 172.21.15.143:55292) 2023-11-08T19:28:55.458 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.453+0000 7f36c895d700 1 -- 172.21.15.143:0/2163341238 learned_addr learned my addr 172.21.15.143:0/2163341238 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:28:55.458 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.453+0000 7f36c915e700 1 --2- 172.21.15.143:0/2163341238 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f36c406c730 0x7f36c40cd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:55.458 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.453+0000 7f36c895d700 1 -- 172.21.15.143:0/2163341238 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f36c406c730 msgr2=0x7f36c40cd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:55.459 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.453+0000 7f36c895d700 1 --2- 172.21.15.143:0/2163341238 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f36c406c730 0x7f36c40cd910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:55.459 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.453+0000 7f36c895d700 1 -- 172.21.15.143:0/2163341238 >> v1:172.21.15.174:6789/0 conn(0x7f36c415de80 legacy=0x7f36c40cbf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:55.459 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.453+0000 7f36c895d700 1 -- 172.21.15.143:0/2163341238 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36c40cc870 con 0x7f36c406c030 2023-11-08T19:28:55.460 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.453+0000 7f36c915e700 1 --2- 172.21.15.143:0/2163341238 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f36c406c730 0x7f36c40cd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:28:55.460 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.454+0000 7f36c895d700 1 --2- 172.21.15.143:0/2163341238 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f36c406c030 0x7f36c406cb10 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f36c00098f0 tx=0x7f36c0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8940519196998b server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:55.460 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.454+0000 7f36bbfff700 1 -- 172.21.15.143:0/2163341238 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36c0004230 con 0x7f36c406c030 2023-11-08T19:28:55.460 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.454+0000 7f36bbfff700 1 -- 172.21.15.143:0/2163341238 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36c0004390 con 0x7f36c406c030 2023-11-08T19:28:55.461 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.454+0000 7f36bbfff700 1 -- 172.21.15.143:0/2163341238 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36c00036a0 con 0x7f36c406c030 2023-11-08T19:28:55.461 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.454+0000 7f36cb3c2700 1 -- 172.21.15.143:0/2163341238 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f36c406c030 msgr2=0x7f36c406cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:55.461 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.454+0000 7f36cb3c2700 1 --2- 172.21.15.143:0/2163341238 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f36c406c030 0x7f36c406cb10 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f36c00098f0 tx=0x7f36c0009c00 comp rx=0 tx=0).stop 2023-11-08T19:28:55.461 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.454+0000 7f36cb3c2700 1 -- 172.21.15.143:0/2163341238 shutdown_connections 2023-11-08T19:28:55.461 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.454+0000 7f36cb3c2700 1 --2- 172.21.15.143:0/2163341238 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f36c406c730 0x7f36c40cd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:55.462 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.454+0000 7f36cb3c2700 1 --2- 172.21.15.143:0/2163341238 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f36c406c030 0x7f36c406cb10 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:55.462 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.454+0000 7f36cb3c2700 1 -- 172.21.15.143:0/2163341238 >> 172.21.15.143:0/2163341238 conn(0x7f36c408b000 msgr2=0x7f36c409ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:55.462 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.454+0000 7f36cb3c2700 1 -- 172.21.15.143:0/2163341238 shutdown_connections 2023-11-08T19:28:55.462 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.454+0000 7f36cb3c2700 1 -- 172.21.15.143:0/2163341238 wait complete. 2023-11-08T19:28:55.465 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.455+0000 7f36cb3c2700 1 Processor -- start 2023-11-08T19:28:55.465 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.456+0000 7f36cb3c2700 1 -- start start 2023-11-08T19:28:55.466 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.456+0000 7f36cb3c2700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f36c406c030 0x7f36c415e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:55.466 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.456+0000 7f36cb3c2700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f36c406c730 0x7f36c415eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:55.466 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.456+0000 7f36cb3c2700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f36c415de80 0x7f36c415f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:55.466 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.456+0000 7f36cb3c2700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36c415d4e0 con 0x7f36c406c030 2023-11-08T19:28:55.466 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.456+0000 7f36cb3c2700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36c40b5060 con 0x7f36c415de80 2023-11-08T19:28:55.467 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.456+0000 7f36cb3c2700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36c40b5340 con 0x7f36c406c730 2023-11-08T19:28:55.467 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.456+0000 7f36c915e700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f36c406c030 0x7f36c415e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:55.467 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.456+0000 7f36c915e700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f36c406c030 0x7f36c415e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55298/0 (socket says 172.21.15.143:55298) 2023-11-08T19:28:55.467 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.456+0000 7f36c915e700 1 -- 172.21.15.143:0/1911209107 learned_addr learned my addr 172.21.15.143:0/1911209107 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:28:55.468 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.456+0000 7f36c895d700 1 --2- 172.21.15.143:0/1911209107 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f36c406c730 0x7f36c415eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:55.468 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.457+0000 7f36c995f700 1 --2- 172.21.15.143:0/1911209107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f36c415de80 0x7f36c415f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:55.468 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.457+0000 7f36c915e700 1 -- 172.21.15.143:0/1911209107 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f36c406c730 msgr2=0x7f36c415eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:55.468 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.457+0000 7f36c915e700 1 --2- 172.21.15.143:0/1911209107 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f36c406c730 0x7f36c415eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:55.468 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.457+0000 7f36c915e700 1 -- 172.21.15.143:0/1911209107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f36c415de80 msgr2=0x7f36c415f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:55.469 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.457+0000 7f36c915e700 1 --2- 172.21.15.143:0/1911209107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f36c415de80 0x7f36c415f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:55.469 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.457+0000 7f36c915e700 1 -- 172.21.15.143:0/1911209107 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36c0009550 con 0x7f36c406c030 2023-11-08T19:28:55.469 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.457+0000 7f36c895d700 1 --2- 172.21.15.143:0/1911209107 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f36c406c730 0x7f36c415eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:28:55.469 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.457+0000 7f36c995f700 1 --2- 172.21.15.143:0/1911209107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f36c415de80 0x7f36c415f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:28:55.469 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.457+0000 7f36c915e700 1 --2- 172.21.15.143:0/1911209107 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f36c406c030 0x7f36c415e990 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f36b400c7f0 tx=0x7f36b400cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:55.469 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.457+0000 7f36b9ffb700 1 -- 172.21.15.143:0/1911209107 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36b4011800 con 0x7f36c406c030 2023-11-08T19:28:55.470 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.457+0000 7f36b9ffb700 1 -- 172.21.15.143:0/1911209107 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36b4011960 con 0x7f36c406c030 2023-11-08T19:28:55.470 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.457+0000 7f36cb3c2700 1 -- 172.21.15.143:0/1911209107 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36c40b5620 con 0x7f36c406c030 2023-11-08T19:28:55.470 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.457+0000 7f36b9ffb700 1 -- 172.21.15.143:0/1911209107 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36b4011d10 con 0x7f36c406c030 2023-11-08T19:28:55.471 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.457+0000 7f36cb3c2700 1 -- 172.21.15.143:0/1911209107 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36c415d100 con 0x7f36c406c030 2023-11-08T19:28:55.471 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.458+0000 7f36cb3c2700 1 -- 172.21.15.143:0/1911209107 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36c404eee0 con 0x7f36c406c030 2023-11-08T19:28:55.471 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.458+0000 7f36b9ffb700 1 -- 172.21.15.143:0/1911209107 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f36b4018360 con 0x7f36c406c030 2023-11-08T19:28:55.471 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.459+0000 7f36b9ffb700 1 --2- 172.21.15.143:0/1911209107 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f36a806af00 0x7f36a806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:55.471 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.459+0000 7f36b9ffb700 1 -- 172.21.15.143:0/1911209107 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6887+0+0 (secure 0 0 0) 0x7f36b4089a30 con 0x7f36c406c030 2023-11-08T19:28:55.471 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.459+0000 7f36c895d700 1 --2- 172.21.15.143:0/1911209107 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f36a806af00 0x7f36a806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:55.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.459+0000 7f36c895d700 1 --2- 172.21.15.143:0/1911209107 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f36a806af00 0x7f36a806d3c0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f36c406c410 tx=0x7f36c0008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:55.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.463+0000 7f36b9ffb700 1 -- 172.21.15.143:0/1911209107 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f36b4054640 con 0x7f36c406c030 2023-11-08T19:28:55.691 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.689+0000 7f36cb3c2700 1 -- 172.21.15.143:0/1911209107 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f36c40c8710 con 0x7f36c406c030 2023-11-08T19:28:55.692 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.689+0000 7f36b9ffb700 1 -- 172.21.15.143:0/1911209107 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f36b4057c60 con 0x7f36c406c030 2023-11-08T19:28:55.693 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:28:55.693 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":64,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5933,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/3918286710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6835","nonce":3918286710},{"type":"v1","addr":"172.21.15.174:6837","nonce":3918286710}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":6906,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6834/3172664377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":3172664377},{"type":"v1","addr":"172.21.15.143:6834","nonce":3172664377}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:03.092245+0000","modified":"2023-11-08T19:28:51.371927+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":95,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6909,"mds_1":6918},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6909":{"gid":6909,"name":"a","rank":0,"incarnation":60,"state":"up:active","state_seq":17,"addr":"172.21.15.143:6835/3953059900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":3953059900},{"type":"v1","addr":"172.21.15.143:6835","nonce":3953059900}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6918":{"gid":6918,"name":"b","rank":1,"incarnation":54,"state":"up:active","state_seq":6,"addr":"172.21.15.174:6836/3601088618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3601088618},{"type":"v1","addr":"172.21.15.174:6836","nonce":3601088618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:28:55.694 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.691+0000 7f36cb3c2700 1 -- 172.21.15.143:0/1911209107 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f36a806af00 msgr2=0x7f36a806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:55.695 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.691+0000 7f36cb3c2700 1 --2- 172.21.15.143:0/1911209107 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f36a806af00 0x7f36a806d3c0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f36c406c410 tx=0x7f36c0008030 comp rx=0 tx=0).stop 2023-11-08T19:28:55.695 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.691+0000 7f36cb3c2700 1 -- 172.21.15.143:0/1911209107 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f36c406c030 msgr2=0x7f36c415e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:55.695 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.691+0000 7f36cb3c2700 1 --2- 172.21.15.143:0/1911209107 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f36c406c030 0x7f36c415e990 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f36b400c7f0 tx=0x7f36b400cbb0 comp rx=0 tx=0).stop 2023-11-08T19:28:55.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.691+0000 7f36cb3c2700 1 -- 172.21.15.143:0/1911209107 shutdown_connections 2023-11-08T19:28:55.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.691+0000 7f36cb3c2700 1 --2- 172.21.15.143:0/1911209107 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f36a806af00 0x7f36a806d3c0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:55.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.691+0000 7f36cb3c2700 1 --2- 172.21.15.143:0/1911209107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f36c415de80 0x7f36c415f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:55.697 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.691+0000 7f36cb3c2700 1 --2- 172.21.15.143:0/1911209107 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f36c406c030 0x7f36c415e990 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:55.697 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.691+0000 7f36cb3c2700 1 --2- 172.21.15.143:0/1911209107 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f36c406c730 0x7f36c415eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:55.697 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.691+0000 7f36cb3c2700 1 -- 172.21.15.143:0/1911209107 >> 172.21.15.143:0/1911209107 conn(0x7f36c408b000 msgr2=0x7f36c40d19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:55.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.691+0000 7f36cb3c2700 1 -- 172.21.15.143:0/1911209107 shutdown_connections 2023-11-08T19:28:55.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.691+0000 7f36cb3c2700 1 -- 172.21.15.143:0/1911209107 wait complete. 2023-11-08T19:28:55.699 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 64 2023-11-08T19:28:55.706 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json dump snaps --server 2023-11-08T19:28:55.842 INFO:teuthology.orchestra.run.smithi143.stdout:{"last_snap":2,"last_created":2,"last_destroyed":1,"pending_noop":[],"snaps":[{"snapid":2,"ino":1099511627777,"stamp":"2023-11-08T19:28:31.007395+0000","name":"s11","metadata":{}}],"need_to_purge":{},"pending_update":[],"pending_destroy":[]} 2023-11-08T19:28:55.843 DEBUG:tasks.cephfs.filesystem:_json_asok output { "last_created": 2, "last_destroyed": 1, "last_snap": 2, "need_to_purge": {}, "pending_destroy": [], "pending_noop": [], "pending_update": [], "snaps": [ { "ino": 1099511627777, "metadata": {}, "name": "s11", "snapid": 2, "stamp": "2023-11-08T19:28:31.007395+0000" } ] } 2023-11-08T19:28:55.843 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-11-08T19:28:55.844 INFO:tasks.cephfs.test_snapshots:testing snapserver mds_kill_mdstable_at=2 2023-11-08T19:28:55.844 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:28:55.964 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.961+0000 7f5feb9c9700 1 Processor -- start 2023-11-08T19:28:55.964 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.962+0000 7f5feb9c9700 1 -- start start 2023-11-08T19:28:55.966 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.962+0000 7f5feb9c9700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5fe415ba30 0x7f5fe40b14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:55.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.962+0000 7f5feb9c9700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f5fe40d03a0 0x7f5fe40d1560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:55.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.962+0000 7f5feb9c9700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f5fe40d1aa0 0x7f5fe40ce2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:55.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.962+0000 7f5feb9c9700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fe40d1f10 con 0x7f5fe40d1aa0 2023-11-08T19:28:55.968 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.962+0000 7f5feb9c9700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fe40b1bc0 con 0x7f5fe40d03a0 2023-11-08T19:28:55.968 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.962+0000 7f5feb9c9700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe40b1ea0 con 0x7f5fe415ba30 2023-11-08T19:28:55.968 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.962+0000 7f5fe9f66700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f5fe40d1aa0 0x7f5fe40ce2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:55.968 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.963+0000 7f5fe9f66700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f5fe40d1aa0 0x7f5fe40ce2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55310/0 (socket says 172.21.15.143:55310) 2023-11-08T19:28:55.969 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.963+0000 7f5fe9f66700 1 -- 172.21.15.143:0/1969089515 learned_addr learned my addr 172.21.15.143:0/1969089515 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:28:55.969 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.963+0000 7f5fe9765700 1 --2- 172.21.15.143:0/1969089515 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5fe415ba30 0x7f5fe40b14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:55.969 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.963+0000 7f5fe8f64700 1 --2- 172.21.15.143:0/1969089515 >> v2:172.21.15.174:3300/0 conn(0x7f5fe40d03a0 0x7f5fe40d1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:55.969 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.963+0000 7f5fe9f66700 1 -- 172.21.15.143:0/1969089515 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5fe415ba30 msgr2=0x7f5fe40b14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:55.969 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.963+0000 7f5fe9f66700 1 --2- 172.21.15.143:0/1969089515 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5fe415ba30 0x7f5fe40b14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:55.970 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.963+0000 7f5fe9f66700 1 -- 172.21.15.143:0/1969089515 >> v2:172.21.15.174:3300/0 conn(0x7f5fe40d03a0 msgr2=0x7f5fe40d1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:55.970 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.963+0000 7f5fe9f66700 1 --2- 172.21.15.143:0/1969089515 >> v2:172.21.15.174:3300/0 conn(0x7f5fe40d03a0 0x7f5fe40d1560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:55.970 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.963+0000 7f5fe9f66700 1 -- 172.21.15.143:0/1969089515 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fe40b2180 con 0x7f5fe40d1aa0 2023-11-08T19:28:55.970 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.963+0000 7f5fe9f66700 1 --2- 172.21.15.143:0/1969089515 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5fe40d1aa0 0x7f5fe40ce2a0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f5fe40678e0 tx=0x7f5fdc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2353cd5c187dbd9 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:55.970 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.963+0000 7f5fdbfff700 1 -- 172.21.15.143:0/1969089515 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fdc00e070 con 0x7f5fe40d1aa0 2023-11-08T19:28:55.971 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.963+0000 7f5fdbfff700 1 -- 172.21.15.143:0/1969089515 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fdc00b910 con 0x7f5fe40d1aa0 2023-11-08T19:28:55.971 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.963+0000 7f5fdbfff700 1 -- 172.21.15.143:0/1969089515 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fdc00bcc0 con 0x7f5fe40d1aa0 2023-11-08T19:28:55.971 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.964+0000 7f5feb9c9700 1 -- 172.21.15.143:0/1969089515 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5fe40d1aa0 msgr2=0x7f5fe40ce2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:55.971 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.964+0000 7f5feb9c9700 1 --2- 172.21.15.143:0/1969089515 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5fe40d1aa0 0x7f5fe40ce2a0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f5fe40678e0 tx=0x7f5fdc00b330 comp rx=0 tx=0).stop 2023-11-08T19:28:55.971 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.964+0000 7f5feb9c9700 1 -- 172.21.15.143:0/1969089515 shutdown_connections 2023-11-08T19:28:55.972 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.964+0000 7f5feb9c9700 1 --2- 172.21.15.143:0/1969089515 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5fe40d1aa0 0x7f5fe40ce2a0 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:55.972 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.964+0000 7f5feb9c9700 1 --2- 172.21.15.143:0/1969089515 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5fe415ba30 0x7f5fe40b14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:55.972 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.964+0000 7f5feb9c9700 1 --2- 172.21.15.143:0/1969089515 >> v2:172.21.15.174:3300/0 conn(0x7f5fe40d03a0 0x7f5fe40d1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:55.972 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.964+0000 7f5feb9c9700 1 -- 172.21.15.143:0/1969089515 >> 172.21.15.143:0/1969089515 conn(0x7f5fe408b000 msgr2=0x7f5fe409ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:55.972 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.964+0000 7f5feb9c9700 1 -- 172.21.15.143:0/1969089515 shutdown_connections 2023-11-08T19:28:55.972 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.964+0000 7f5feb9c9700 1 -- 172.21.15.143:0/1969089515 wait complete. 2023-11-08T19:28:55.975 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.965+0000 7f5feb9c9700 1 Processor -- start 2023-11-08T19:28:55.975 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.965+0000 7f5feb9c9700 1 -- start start 2023-11-08T19:28:55.976 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.966+0000 7f5feb9c9700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5fe40d03a0 0x7f5fe4158060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:55.976 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.966+0000 7f5feb9c9700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5fe40d1aa0 0x7f5fe40c9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:55.976 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.966+0000 7f5feb9c9700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5fe415ba30 0x7f5fe4070360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:55.976 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.966+0000 7f5feb9c9700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe415c570 con 0x7f5fe40d1aa0 2023-11-08T19:28:55.977 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.966+0000 7f5feb9c9700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe40cd4f0 con 0x7f5fe415ba30 2023-11-08T19:28:55.977 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.966+0000 7f5feb9c9700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe415beb0 con 0x7f5fe40d03a0 2023-11-08T19:28:55.977 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.966+0000 7f5fe8f64700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5fe40d1aa0 0x7f5fe40c9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:55.977 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.966+0000 7f5fe8f64700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5fe40d1aa0 0x7f5fe40c9ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55314/0 (socket says 172.21.15.143:55314) 2023-11-08T19:28:55.977 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.966+0000 7f5fe8f64700 1 -- 172.21.15.143:0/2315753771 learned_addr learned my addr 172.21.15.143:0/2315753771 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:28:55.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.966+0000 7f5fe9f66700 1 --2- 172.21.15.143:0/2315753771 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5fe415ba30 0x7f5fe4070360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:55.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.966+0000 7f5fe8f64700 1 -- 172.21.15.143:0/2315753771 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5fe40d03a0 msgr2=0x7f5fe4158060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:55.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.966+0000 7f5fe9765700 1 --2- 172.21.15.143:0/2315753771 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5fe40d03a0 0x7f5fe4158060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:55.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.966+0000 7f5fe8f64700 1 --2- 172.21.15.143:0/2315753771 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5fe40d03a0 0x7f5fe4158060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:55.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.966+0000 7f5fe8f64700 1 -- 172.21.15.143:0/2315753771 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5fe415ba30 msgr2=0x7f5fe4070360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:55.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.966+0000 7f5fe8f64700 1 --2- 172.21.15.143:0/2315753771 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5fe415ba30 0x7f5fe4070360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:55.979 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.966+0000 7f5fe8f64700 1 -- 172.21.15.143:0/2315753771 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fdc009b30 con 0x7f5fe40d1aa0 2023-11-08T19:28:55.979 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.966+0000 7f5fe8f64700 1 --2- 172.21.15.143:0/2315753771 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5fe40d1aa0 0x7f5fe40c9ca0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f5fe000e930 tx=0x7f5fe000ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:55.979 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.966+0000 7f5fd9ffb700 1 -- 172.21.15.143:0/2315753771 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe0009690 con 0x7f5fe40d1aa0 2023-11-08T19:28:55.979 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.966+0000 7f5fd9ffb700 1 -- 172.21.15.143:0/2315753771 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fe000c850 con 0x7f5fe40d1aa0 2023-11-08T19:28:55.979 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.966+0000 7f5feb9c9700 1 -- 172.21.15.143:0/2315753771 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fe415c190 con 0x7f5fe40d1aa0 2023-11-08T19:28:55.980 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.966+0000 7f5fd9ffb700 1 -- 172.21.15.143:0/2315753771 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe000cbe0 con 0x7f5fe40d1aa0 2023-11-08T19:28:55.980 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.967+0000 7f5feb9c9700 1 -- 172.21.15.143:0/2315753771 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fe41585a0 con 0x7f5fe40d1aa0 2023-11-08T19:28:55.980 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.967+0000 7f5feb9c9700 1 -- 172.21.15.143:0/2315753771 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fe404eee0 con 0x7f5fe40d1aa0 2023-11-08T19:28:55.980 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.968+0000 7f5fd9ffb700 1 -- 172.21.15.143:0/2315753771 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5fe000cd40 con 0x7f5fe40d1aa0 2023-11-08T19:28:55.980 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.968+0000 7f5fd9ffb700 1 --2- 172.21.15.143:0/2315753771 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f5fc806af00 0x7f5fc806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:55.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.968+0000 7f5fd9ffb700 1 -- 172.21.15.143:0/2315753771 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6887+0+0 (secure 0 0 0) 0x7f5fe0089ae0 con 0x7f5fe40d1aa0 2023-11-08T19:28:55.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.969+0000 7f5fe9765700 1 --2- 172.21.15.143:0/2315753771 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f5fc806af00 0x7f5fc806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:55.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.969+0000 7f5fe9765700 1 --2- 172.21.15.143:0/2315753771 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f5fc806af00 0x7f5fc806d3c0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f5fe40c8e90 tx=0x7f5fd4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:55.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:55.973+0000 7f5fd9ffb700 1 -- 172.21.15.143:0/2315753771 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5fe00546f0 con 0x7f5fe40d1aa0 2023-11-08T19:28:56.214 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:56.211+0000 7f5feb9c9700 1 -- 172.21.15.143:0/2315753771 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5fe40b4130 con 0x7f5fe40d1aa0 2023-11-08T19:28:56.214 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:56.212+0000 7f5fd9ffb700 1 -- 172.21.15.143:0/2315753771 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f5fe0057d10 con 0x7f5fe40d1aa0 2023-11-08T19:28:56.215 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:28:56.215 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":64,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5933,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/3918286710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6835","nonce":3918286710},{"type":"v1","addr":"172.21.15.174:6837","nonce":3918286710}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":6906,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6834/3172664377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":3172664377},{"type":"v1","addr":"172.21.15.143:6834","nonce":3172664377}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:03.092245+0000","modified":"2023-11-08T19:28:51.371927+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":95,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6909,"mds_1":6918},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6909":{"gid":6909,"name":"a","rank":0,"incarnation":60,"state":"up:active","state_seq":17,"addr":"172.21.15.143:6835/3953059900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":3953059900},{"type":"v1","addr":"172.21.15.143:6835","nonce":3953059900}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6918":{"gid":6918,"name":"b","rank":1,"incarnation":54,"state":"up:active","state_seq":6,"addr":"172.21.15.174:6836/3601088618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3601088618},{"type":"v1","addr":"172.21.15.174:6836","nonce":3601088618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:28:56.217 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:56.214+0000 7f5feb9c9700 1 -- 172.21.15.143:0/2315753771 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f5fc806af00 msgr2=0x7f5fc806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:56.217 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:56.214+0000 7f5feb9c9700 1 --2- 172.21.15.143:0/2315753771 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f5fc806af00 0x7f5fc806d3c0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f5fe40c8e90 tx=0x7f5fd4008040 comp rx=0 tx=0).stop 2023-11-08T19:28:56.217 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:56.214+0000 7f5feb9c9700 1 -- 172.21.15.143:0/2315753771 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5fe40d1aa0 msgr2=0x7f5fe40c9ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:56.217 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:56.214+0000 7f5feb9c9700 1 --2- 172.21.15.143:0/2315753771 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5fe40d1aa0 0x7f5fe40c9ca0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f5fe000e930 tx=0x7f5fe000ecf0 comp rx=0 tx=0).stop 2023-11-08T19:28:56.218 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:56.214+0000 7f5feb9c9700 1 -- 172.21.15.143:0/2315753771 shutdown_connections 2023-11-08T19:28:56.218 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:56.214+0000 7f5feb9c9700 1 --2- 172.21.15.143:0/2315753771 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f5fc806af00 0x7f5fc806d3c0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:56.218 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:56.214+0000 7f5feb9c9700 1 --2- 172.21.15.143:0/2315753771 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5fe40d03a0 0x7f5fe4158060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:56.218 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:56.214+0000 7f5feb9c9700 1 --2- 172.21.15.143:0/2315753771 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5fe415ba30 0x7f5fe4070360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:56.218 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:56.214+0000 7f5feb9c9700 1 --2- 172.21.15.143:0/2315753771 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5fe40d1aa0 0x7f5fe40c9ca0 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:56.219 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:56.214+0000 7f5feb9c9700 1 -- 172.21.15.143:0/2315753771 >> 172.21.15.143:0/2315753771 conn(0x7f5fe408b000 msgr2=0x7f5fe415ad60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:56.219 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:56.214+0000 7f5feb9c9700 1 -- 172.21.15.143:0/2315753771 shutdown_connections 2023-11-08T19:28:56.219 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:56.214+0000 7f5feb9c9700 1 -- 172.21.15.143:0/2315753771 wait complete. 2023-11-08T19:28:56.219 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 64 2023-11-08T19:28:56.229 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds freeze 6:0 true 2023-11-08T19:28:56.362 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:56.360+0000 7f8d5c300700 1 Processor -- start 2023-11-08T19:28:56.363 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:56.360+0000 7f8d5c300700 1 -- start start 2023-11-08T19:28:56.365 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:56.361+0000 7f8d5c300700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f8d540cc8c0 0x7f8d540ccd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:56.365 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:56.361+0000 7f8d5c300700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f8d540c5880 0x7f8d540c5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:56.365 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:56.361+0000 7f8d5c300700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d540c61d0 con 0x7f8d54143a30 2023-11-08T19:28:56.366 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:56.361+0000 7f8d5c300700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d5415e110 con 0x7f8d540cc8c0 2023-11-08T19:28:56.366 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:56.361+0000 7f8d5c300700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d540cd290 con 0x7f8d540c5880 2023-11-08T19:28:56.366 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:56.361+0000 7f8d5989b700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f8d540cc8c0 0x7f8d540ccd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:56.366 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:56.361+0000 7f8d5a09c700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f8d54143a30 0x7f8d5406bfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:51474/0 (socket says 172.21.15.143:51474) 2023-11-08T19:28:56.366 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:56.361+0000 7f8d5a09c700 1 -- 172.21.15.143:0/4288768688 learned_addr learned my addr 172.21.15.143:0/4288768688 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:28:56.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:56.361+0000 7f8d5a89d700 1 --2- 172.21.15.143:0/4288768688 >> v2:172.21.15.174:3300/0 conn(0x7f8d540c5880 0x7f8d540c5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:56.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:56.361+0000 7f8d5989b700 1 -- 172.21.15.143:0/4288768688 >> v2:172.21.15.174:3300/0 conn(0x7f8d540c5880 msgr2=0x7f8d540c5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:56.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:56.361+0000 7f8d5989b700 1 --2- 172.21.15.143:0/4288768688 >> v2:172.21.15.174:3300/0 conn(0x7f8d540c5880 0x7f8d540c5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:56.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:56.361+0000 7f8d5989b700 1 -- 172.21.15.143:0/4288768688 >> v1:172.21.15.143:6789/0 conn(0x7f8d54143a30 legacy=0x7f8d5406bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:56.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:56.361+0000 7f8d5989b700 1 -- 172.21.15.143:0/4288768688 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d54143e10 con 0x7f8d540cc8c0 2023-11-08T19:28:56.368 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:56.361+0000 7f8d5909a700 1 -- 172.21.15.143:0/4288768688 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1325807333 0 0) 0x7f8d540c61d0 con 0x7f8d54143a30 2023-11-08T19:28:56.368 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:56.362+0000 7f8d5989b700 1 --2- 172.21.15.143:0/4288768688 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8d540cc8c0 0x7f8d540ccd50 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f8d540afb90 tx=0x7f8d50009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78eda2878ede15f4 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:56.368 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:56.362+0000 7f8d5909a700 1 -- 172.21.15.143:0/4288768688 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d50011830 con 0x7f8d540cc8c0 2023-11-08T19:28:56.368 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:56.362+0000 7f8d5909a700 1 -- 172.21.15.143:0/4288768688 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d50011990 con 0x7f8d540cc8c0 2023-11-08T19:28:56.368 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:56.362+0000 7f8d5909a700 1 -- 172.21.15.143:0/4288768688 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d50011d20 con 0x7f8d540cc8c0 2023-11-08T19:28:56.368 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:56.362+0000 7f8d5c300700 1 -- 172.21.15.143:0/4288768688 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8d540cc8c0 msgr2=0x7f8d540ccd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:56.369 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:56.362+0000 7f8d5c300700 1 --2- 172.21.15.143:0/4288768688 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8d540cc8c0 0x7f8d540ccd50 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f8d540afb90 tx=0x7f8d50009b50 comp rx=0 tx=0).stop 2023-11-08T19:28:56.369 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:56.362+0000 7f8d5c300700 1 -- 172.21.15.143:0/4288768688 shutdown_connections 2023-11-08T19:28:56.369 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:56.362+0000 7f8d5c300700 1 --2- 172.21.15.143:0/4288768688 >> v2:172.21.15.174:3300/0 conn(0x7f8d540c5880 0x7f8d540c5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:56.369 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:56.362+0000 7f8d5c300700 1 --2- 172.21.15.143:0/4288768688 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8d540cc8c0 0x7f8d540ccd50 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:56.370 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:56.362+0000 7f8d5c300700 1 -- 172.21.15.143:0/4288768688 >> 172.21.15.143:0/4288768688 conn(0x7f8d540b1c70 msgr2=0x7f8d540b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:56.370 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:56.362+0000 7f8d5c300700 1 -- 172.21.15.143:0/4288768688 shutdown_connections 2023-11-08T19:28:56.370 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:56.362+0000 7f8d5c300700 1 -- 172.21.15.143:0/4288768688 wait complete. 2023-11-08T19:28:56.374 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:56.364+0000 7f8d5c300700 1 Processor -- start 2023-11-08T19:28:56.374 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:56.364+0000 7f8d5c300700 1 -- start start 2023-11-08T19:28:56.375 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:56.364+0000 7f8d5c300700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8d540c5880 0x7f8d541378c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:56.375 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:56.364+0000 7f8d5c300700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8d540cc8c0 0x7f8d540afdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:56.375 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:56.364+0000 7f8d5c300700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8d54143a30 0x7f8d540b0330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:56.375 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:56.364+0000 7f8d5c300700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d540ad460 con 0x7f8d540cc8c0 2023-11-08T19:28:56.376 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:56.364+0000 7f8d5c300700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d540ac8e0 con 0x7f8d540c5880 2023-11-08T19:28:56.376 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:56.364+0000 7f8d5c300700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d540acb30 con 0x7f8d54143a30 2023-11-08T19:28:56.376 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:56.364+0000 7f8d5989b700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8d540cc8c0 0x7f8d540afdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:56.376 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:56.364+0000 7f8d5989b700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8d540cc8c0 0x7f8d540afdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55328/0 (socket says 172.21.15.143:55328) 2023-11-08T19:28:56.376 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:56.364+0000 7f8d5989b700 1 -- 172.21.15.143:0/630906316 learned_addr learned my addr 172.21.15.143:0/630906316 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:28:56.377 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:56.365+0000 7f8d5a09c700 1 --2- 172.21.15.143:0/630906316 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8d540c5880 0x7f8d541378c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:56.377 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:56.365+0000 7f8d5989b700 1 -- 172.21.15.143:0/630906316 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8d54143a30 msgr2=0x7f8d540b0330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:56.377 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:56.365+0000 7f8d5989b700 1 --2- 172.21.15.143:0/630906316 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8d54143a30 0x7f8d540b0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:56.377 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:56.365+0000 7f8d5989b700 1 -- 172.21.15.143:0/630906316 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8d540c5880 msgr2=0x7f8d541378c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:56.377 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:56.365+0000 7f8d5989b700 1 --2- 172.21.15.143:0/630906316 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8d540c5880 0x7f8d541378c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:56.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:56.365+0000 7f8d5989b700 1 -- 172.21.15.143:0/630906316 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d500094a0 con 0x7f8d540cc8c0 2023-11-08T19:28:56.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:56.365+0000 7f8d5989b700 1 --2- 172.21.15.143:0/630906316 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8d540cc8c0 0x7f8d540afdf0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f8d500097c0 tx=0x7f8d50004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:56.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:56.365+0000 7f8d4affd700 1 -- 172.21.15.143:0/630906316 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d500038a0 con 0x7f8d540cc8c0 2023-11-08T19:28:56.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:56.365+0000 7f8d4affd700 1 -- 172.21.15.143:0/630906316 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d5001f070 con 0x7f8d540cc8c0 2023-11-08T19:28:56.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:56.365+0000 7f8d5c300700 1 -- 172.21.15.143:0/630906316 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d540ace10 con 0x7f8d540cc8c0 2023-11-08T19:28:56.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:56.365+0000 7f8d4affd700 1 -- 172.21.15.143:0/630906316 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d50003c40 con 0x7f8d540cc8c0 2023-11-08T19:28:56.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:56.365+0000 7f8d5c300700 1 -- 172.21.15.143:0/630906316 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d54137e00 con 0x7f8d540cc8c0 2023-11-08T19:28:56.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:56.366+0000 7f8d5c300700 1 -- 172.21.15.143:0/630906316 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d540ad610 con 0x7f8d540cc8c0 2023-11-08T19:28:56.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:56.366+0000 7f8d4affd700 1 -- 172.21.15.143:0/630906316 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8d50026480 con 0x7f8d540cc8c0 2023-11-08T19:28:56.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:56.367+0000 7f8d4affd700 1 --2- 172.21.15.143:0/630906316 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f8d3806af00 0x7f8d3806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:56.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:56.367+0000 7f8d4affd700 1 -- 172.21.15.143:0/630906316 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6887+0+0 (secure 0 0 0) 0x7f8d50021070 con 0x7f8d540cc8c0 2023-11-08T19:28:56.380 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:56.367+0000 7f8d5a09c700 1 --2- 172.21.15.143:0/630906316 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f8d3806af00 0x7f8d3806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:56.380 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:56.367+0000 7f8d5a09c700 1 --2- 172.21.15.143:0/630906316 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f8d3806af00 0x7f8d3806d3c0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f8d44000f60 tx=0x7f8d44009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:56.380 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:56.371+0000 7f8d4affd700 1 -- 172.21.15.143:0/630906316 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8d540ad610 con 0x7f8d540cc8c0 2023-11-08T19:28:56.644 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:56.641+0000 7f8d5c300700 1 -- 172.21.15.143:0/630906316 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "mds freeze", "role_or_gid": "6:0", "val": "true"} v 0) v1 -- 0x7f8d540ad610 con 0x7f8d540cc8c0 2023-11-08T19:28:57.212 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T19:28:57.216 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T19:28:57.398 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:57.396+0000 7f8d4affd700 1 -- 172.21.15.143:0/630906316 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "mds freeze", "role_or_gid": "6:0", "val": "true"}]=0 freezing mds.6909 v65) v1 ==== 112+0+0 (secure 0 0 0) 0x7f8d540ad610 con 0x7f8d540cc8c0 2023-11-08T19:28:57.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:57.398+0000 7f8d5c300700 1 -- 172.21.15.143:0/630906316 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f8d3806af00 msgr2=0x7f8d3806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:57.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:57.398+0000 7f8d5c300700 1 --2- 172.21.15.143:0/630906316 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f8d3806af00 0x7f8d3806d3c0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f8d44000f60 tx=0x7f8d44009040 comp rx=0 tx=0).stop 2023-11-08T19:28:57.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:57.398+0000 7f8d5c300700 1 -- 172.21.15.143:0/630906316 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8d540cc8c0 msgr2=0x7f8d540afdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:57.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:57.398+0000 7f8d5c300700 1 --2- 172.21.15.143:0/630906316 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8d540cc8c0 0x7f8d540afdf0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f8d500097c0 tx=0x7f8d50004750 comp rx=0 tx=0).stop 2023-11-08T19:28:57.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:57.398+0000 7f8d5c300700 1 -- 172.21.15.143:0/630906316 shutdown_connections 2023-11-08T19:28:57.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:57.398+0000 7f8d5c300700 1 --2- 172.21.15.143:0/630906316 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f8d3806af00 0x7f8d3806d3c0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:57.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:57.398+0000 7f8d5c300700 1 --2- 172.21.15.143:0/630906316 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8d54143a30 0x7f8d540b0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:57.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:57.398+0000 7f8d5c300700 1 --2- 172.21.15.143:0/630906316 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8d540c5880 0x7f8d541378c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:57.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:57.398+0000 7f8d5c300700 1 --2- 172.21.15.143:0/630906316 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8d540cc8c0 0x7f8d540afdf0 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:57.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:57.398+0000 7f8d5c300700 1 -- 172.21.15.143:0/630906316 >> 172.21.15.143:0/630906316 conn(0x7f8d540b1c70 msgr2=0x7f8d54071800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:57.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:57.399+0000 7f8d5c300700 1 -- 172.21.15.143:0/630906316 shutdown_connections 2023-11-08T19:28:57.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:57.399+0000 7f8d5c300700 1 -- 172.21.15.143:0/630906316 wait complete. 2023-11-08T19:28:57.405 INFO:teuthology.orchestra.run.smithi143.stderr:freezing mds.6909 2023-11-08T19:28:57.416 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config set mds_kill_mdstable_at 2 2023-11-08T19:28:57.557 INFO:teuthology.orchestra.run.smithi143.stdout:{"success":"mds_kill_mdstable_at = '2' (not observed, change may require restart) "} 2023-11-08T19:28:57.558 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_kill_mdstable_at = '2' (not observed, change may require restart) " } 2023-11-08T19:28:57.558 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:28:57.559 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir d1/dir/.snap/s12) 2023-11-08T19:28:57.561 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mds_beacon_grace 2023-11-08T19:28:57.579 INFO:tasks.ceph.mds.a.smithi143.stderr:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el8/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: In function 'void MDSTableServer::_prepare_logged(ceph::cref_t&, version_t)' thread 7f5ba1124700 time 2023-11-08T19:28:57.579614+0000 2023-11-08T19:28:57.580 INFO:tasks.ceph.mds.a.smithi143.stderr:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el8/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: 77: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 2) 2023-11-08T19:28:57.581 INFO:tasks.ceph.mds.a.smithi143.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-08T19:28:57.581 INFO:tasks.ceph.mds.a.smithi143.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f5bb29f8f5b] 2023-11-08T19:28:57.582 INFO:tasks.ceph.mds.a.smithi143.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2ba121) [0x7f5bb29f9121] 2023-11-08T19:28:57.582 INFO:tasks.ceph.mds.a.smithi143.stderr: 3: (MDSTableServer::_prepare_logged(boost::intrusive_ptr const&, unsigned long)+0x6ad) [0x55649de86f5d] 2023-11-08T19:28:57.582 INFO:tasks.ceph.mds.a.smithi143.stderr: 4: (MDSContext::complete(int)+0x5f) [0x55649ded1ecf] 2023-11-08T19:28:57.582 INFO:tasks.ceph.mds.a.smithi143.stderr: 5: (MDSIOContextBase::complete(int)+0x574) [0x55649ded26b4] 2023-11-08T19:28:57.583 INFO:tasks.ceph.mds.a.smithi143.stderr: 6: (MDSLogContextBase::complete(int)+0x5b) [0x55649ded2a7b] 2023-11-08T19:28:57.583 INFO:tasks.ceph.mds.a.smithi143.stderr: 7: (Finisher::finisher_thread_entry()+0x19d) [0x7f5bb2a968cd] 2023-11-08T19:28:57.583 INFO:tasks.ceph.mds.a.smithi143.stderr: 8: /lib64/libpthread.so.0(+0x81cf) [0x7f5bb17911cf] 2023-11-08T19:28:57.583 INFO:tasks.ceph.mds.a.smithi143.stderr: 9: clone() 2023-11-08T19:28:57.583 INFO:tasks.ceph.mds.a.smithi143.stderr:*** Caught signal (Aborted) ** 2023-11-08T19:28:57.584 INFO:tasks.ceph.mds.a.smithi143.stderr: in thread 7f5ba1124700 thread_name:MR_Finisher 2023-11-08T19:28:57.584 INFO:tasks.ceph.mds.a.smithi143.stderr:2023-11-08T19:28:57.578+0000 7f5ba1124700 -1 /home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el8/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: In function 'void MDSTableServer::_prepare_logged(ceph::cref_t&, version_t)' thread 7f5ba1124700 time 2023-11-08T19:28:57.579614+0000 2023-11-08T19:28:57.584 INFO:tasks.ceph.mds.a.smithi143.stderr:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el8/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: 77: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 2) 2023-11-08T19:28:57.584 INFO:tasks.ceph.mds.a.smithi143.stderr: 2023-11-08T19:28:57.584 INFO:tasks.ceph.mds.a.smithi143.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-08T19:28:57.585 INFO:tasks.ceph.mds.a.smithi143.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f5bb29f8f5b] 2023-11-08T19:28:57.585 INFO:tasks.ceph.mds.a.smithi143.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2ba121) [0x7f5bb29f9121] 2023-11-08T19:28:57.585 INFO:tasks.ceph.mds.a.smithi143.stderr: 3: (MDSTableServer::_prepare_logged(boost::intrusive_ptr const&, unsigned long)+0x6ad) [0x55649de86f5d] 2023-11-08T19:28:57.585 INFO:tasks.ceph.mds.a.smithi143.stderr: 4: (MDSContext::complete(int)+0x5f) [0x55649ded1ecf] 2023-11-08T19:28:57.585 INFO:tasks.ceph.mds.a.smithi143.stderr: 5: (MDSIOContextBase::complete(int)+0x574) [0x55649ded26b4] 2023-11-08T19:28:57.585 INFO:tasks.ceph.mds.a.smithi143.stderr: 6: (MDSLogContextBase::complete(int)+0x5b) [0x55649ded2a7b] 2023-11-08T19:28:57.586 INFO:tasks.ceph.mds.a.smithi143.stderr: 7: (Finisher::finisher_thread_entry()+0x19d) [0x7f5bb2a968cd] 2023-11-08T19:28:57.586 INFO:tasks.ceph.mds.a.smithi143.stderr: 8: /lib64/libpthread.so.0(+0x81cf) [0x7f5bb17911cf] 2023-11-08T19:28:57.586 INFO:tasks.ceph.mds.a.smithi143.stderr: 9: clone() 2023-11-08T19:28:57.586 INFO:tasks.ceph.mds.a.smithi143.stderr: 2023-11-08T19:28:57.586 INFO:tasks.ceph.mds.a.smithi143.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-08T19:28:57.587 INFO:tasks.ceph.mds.a.smithi143.stderr: 1: /lib64/libpthread.so.0(+0x12ce0) [0x7f5bb179bce0] 2023-11-08T19:28:57.587 INFO:tasks.ceph.mds.a.smithi143.stderr: 2: gsignal() 2023-11-08T19:28:57.587 INFO:tasks.ceph.mds.a.smithi143.stderr: 3: abort() 2023-11-08T19:28:57.587 INFO:tasks.ceph.mds.a.smithi143.stderr: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f5bb29f8fb5] 2023-11-08T19:28:57.587 INFO:tasks.ceph.mds.a.smithi143.stderr: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2ba121) [0x7f5bb29f9121] 2023-11-08T19:28:57.588 INFO:tasks.ceph.mds.a.smithi143.stderr: 6: (MDSTableServer::_prepare_logged(boost::intrusive_ptr const&, unsigned long)+0x6ad) [0x55649de86f5d] 2023-11-08T19:28:57.588 INFO:tasks.ceph.mds.a.smithi143.stderr: 7: (MDSContext::complete(int)+0x5f) [0x55649ded1ecf] 2023-11-08T19:28:57.588 INFO:tasks.ceph.mds.a.smithi143.stderr: 8: (MDSIOContextBase::complete(int)+0x574) [0x55649ded26b4] 2023-11-08T19:28:57.588 INFO:tasks.ceph.mds.a.smithi143.stderr: 9: (MDSLogContextBase::complete(int)+0x5b) [0x55649ded2a7b] 2023-11-08T19:28:57.588 INFO:tasks.ceph.mds.a.smithi143.stderr: 10: (Finisher::finisher_thread_entry()+0x19d) [0x7f5bb2a968cd] 2023-11-08T19:28:57.589 INFO:tasks.ceph.mds.a.smithi143.stderr: 11: /lib64/libpthread.so.0(+0x81cf) [0x7f5bb17911cf] 2023-11-08T19:28:57.589 INFO:tasks.ceph.mds.a.smithi143.stderr: 12: clone() 2023-11-08T19:28:57.589 INFO:tasks.ceph.mds.a.smithi143.stderr:2023-11-08T19:28:57.579+0000 7f5ba1124700 -1 *** Caught signal (Aborted) ** 2023-11-08T19:28:57.589 INFO:tasks.ceph.mds.a.smithi143.stderr: in thread 7f5ba1124700 thread_name:MR_Finisher 2023-11-08T19:28:57.589 INFO:tasks.ceph.mds.a.smithi143.stderr: 2023-11-08T19:28:57.590 INFO:tasks.ceph.mds.a.smithi143.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-08T19:28:57.590 INFO:tasks.ceph.mds.a.smithi143.stderr: 1: /lib64/libpthread.so.0(+0x12ce0) [0x7f5bb179bce0] 2023-11-08T19:28:57.590 INFO:tasks.ceph.mds.a.smithi143.stderr: 2: gsignal() 2023-11-08T19:28:57.590 INFO:tasks.ceph.mds.a.smithi143.stderr: 3: abort() 2023-11-08T19:28:57.590 INFO:tasks.ceph.mds.a.smithi143.stderr: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f5bb29f8fb5] 2023-11-08T19:28:57.591 INFO:tasks.ceph.mds.a.smithi143.stderr: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2ba121) [0x7f5bb29f9121] 2023-11-08T19:28:57.591 INFO:tasks.ceph.mds.a.smithi143.stderr: 6: (MDSTableServer::_prepare_logged(boost::intrusive_ptr const&, unsigned long)+0x6ad) [0x55649de86f5d] 2023-11-08T19:28:57.591 INFO:tasks.ceph.mds.a.smithi143.stderr: 7: (MDSContext::complete(int)+0x5f) [0x55649ded1ecf] 2023-11-08T19:28:57.591 INFO:tasks.ceph.mds.a.smithi143.stderr: 8: (MDSIOContextBase::complete(int)+0x574) [0x55649ded26b4] 2023-11-08T19:28:57.591 INFO:tasks.ceph.mds.a.smithi143.stderr: 9: (MDSLogContextBase::complete(int)+0x5b) [0x55649ded2a7b] 2023-11-08T19:28:57.592 INFO:tasks.ceph.mds.a.smithi143.stderr: 10: (Finisher::finisher_thread_entry()+0x19d) [0x7f5bb2a968cd] 2023-11-08T19:28:57.592 INFO:tasks.ceph.mds.a.smithi143.stderr: 11: /lib64/libpthread.so.0(+0x81cf) [0x7f5bb17911cf] 2023-11-08T19:28:57.592 INFO:tasks.ceph.mds.a.smithi143.stderr: 12: clone() 2023-11-08T19:28:57.592 INFO:tasks.ceph.mds.a.smithi143.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-11-08T19:28:57.592 INFO:tasks.ceph.mds.a.smithi143.stderr: 2023-11-08T19:28:57.593 INFO:tasks.ceph.mds.a.smithi143.stderr: -277> 2023-11-08T19:28:53.685+0000 7f5ba492b700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-08T19:28:57.594 INFO:tasks.ceph.mds.a.smithi143.stderr: -275> 2023-11-08T19:28:53.685+0000 7f5ba492b700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-11-08T19:28:57.594 INFO:tasks.ceph.mds.a.smithi143.stderr: -1> 2023-11-08T19:28:57.578+0000 7f5ba1124700 -1 /home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el8/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: In function 'void MDSTableServer::_prepare_logged(ceph::cref_t&, version_t)' thread 7f5ba1124700 time 2023-11-08T19:28:57.579614+0000 2023-11-08T19:28:57.594 INFO:tasks.ceph.mds.a.smithi143.stderr:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el8/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: 77: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 2) 2023-11-08T19:28:57.594 INFO:tasks.ceph.mds.a.smithi143.stderr: 2023-11-08T19:28:57.594 INFO:tasks.ceph.mds.a.smithi143.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-08T19:28:57.595 INFO:tasks.ceph.mds.a.smithi143.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f5bb29f8f5b] 2023-11-08T19:28:57.595 INFO:tasks.ceph.mds.a.smithi143.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2ba121) [0x7f5bb29f9121] 2023-11-08T19:28:57.595 INFO:tasks.ceph.mds.a.smithi143.stderr: 3: (MDSTableServer::_prepare_logged(boost::intrusive_ptr const&, unsigned long)+0x6ad) [0x55649de86f5d] 2023-11-08T19:28:57.595 INFO:tasks.ceph.mds.a.smithi143.stderr: 4: (MDSContext::complete(int)+0x5f) [0x55649ded1ecf] 2023-11-08T19:28:57.595 INFO:tasks.ceph.mds.a.smithi143.stderr: 5: (MDSIOContextBase::complete(int)+0x574) [0x55649ded26b4] 2023-11-08T19:28:57.596 INFO:tasks.ceph.mds.a.smithi143.stderr: 6: (MDSLogContextBase::complete(int)+0x5b) [0x55649ded2a7b] 2023-11-08T19:28:57.596 INFO:tasks.ceph.mds.a.smithi143.stderr: 7: (Finisher::finisher_thread_entry()+0x19d) [0x7f5bb2a968cd] 2023-11-08T19:28:57.596 INFO:tasks.ceph.mds.a.smithi143.stderr: 8: /lib64/libpthread.so.0(+0x81cf) [0x7f5bb17911cf] 2023-11-08T19:28:57.596 INFO:tasks.ceph.mds.a.smithi143.stderr: 9: clone() 2023-11-08T19:28:57.596 INFO:tasks.ceph.mds.a.smithi143.stderr: 2023-11-08T19:28:57.597 INFO:tasks.ceph.mds.a.smithi143.stderr: 0> 2023-11-08T19:28:57.579+0000 7f5ba1124700 -1 *** Caught signal (Aborted) ** 2023-11-08T19:28:57.597 INFO:tasks.ceph.mds.a.smithi143.stderr: in thread 7f5ba1124700 thread_name:MR_Finisher 2023-11-08T19:28:57.597 INFO:tasks.ceph.mds.a.smithi143.stderr: 2023-11-08T19:28:57.597 INFO:tasks.ceph.mds.a.smithi143.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-08T19:28:57.597 INFO:tasks.ceph.mds.a.smithi143.stderr: 1: /lib64/libpthread.so.0(+0x12ce0) [0x7f5bb179bce0] 2023-11-08T19:28:57.597 INFO:tasks.ceph.mds.a.smithi143.stderr: 2: gsignal() 2023-11-08T19:28:57.598 INFO:tasks.ceph.mds.a.smithi143.stderr: 3: abort() 2023-11-08T19:28:57.598 INFO:tasks.ceph.mds.a.smithi143.stderr: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f5bb29f8fb5] 2023-11-08T19:28:57.598 INFO:tasks.ceph.mds.a.smithi143.stderr: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2ba121) [0x7f5bb29f9121] 2023-11-08T19:28:57.598 INFO:tasks.ceph.mds.a.smithi143.stderr: 6: (MDSTableServer::_prepare_logged(boost::intrusive_ptr const&, unsigned long)+0x6ad) [0x55649de86f5d] 2023-11-08T19:28:57.598 INFO:tasks.ceph.mds.a.smithi143.stderr: 7: (MDSContext::complete(int)+0x5f) [0x55649ded1ecf] 2023-11-08T19:28:57.599 INFO:tasks.ceph.mds.a.smithi143.stderr: 8: (MDSIOContextBase::complete(int)+0x574) [0x55649ded26b4] 2023-11-08T19:28:57.599 INFO:tasks.ceph.mds.a.smithi143.stderr: 9: (MDSLogContextBase::complete(int)+0x5b) [0x55649ded2a7b] 2023-11-08T19:28:57.599 INFO:tasks.ceph.mds.a.smithi143.stderr: 10: (Finisher::finisher_thread_entry()+0x19d) [0x7f5bb2a968cd] 2023-11-08T19:28:57.599 INFO:tasks.ceph.mds.a.smithi143.stderr: 11: /lib64/libpthread.so.0(+0x81cf) [0x7f5bb17911cf] 2023-11-08T19:28:57.599 INFO:tasks.ceph.mds.a.smithi143.stderr: 12: clone() 2023-11-08T19:28:57.600 INFO:tasks.ceph.mds.a.smithi143.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-11-08T19:28:57.600 INFO:tasks.ceph.mds.a.smithi143.stderr: 2023-11-08T19:28:57.600 INFO:tasks.ceph.mds.a.smithi143.stderr: -277> 2023-11-08T19:28:53.685+0000 7f5ba492b700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-08T19:28:57.600 INFO:tasks.ceph.mds.a.smithi143.stderr: -275> 2023-11-08T19:28:53.685+0000 7f5ba492b700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-11-08T19:28:57.600 INFO:tasks.ceph.mds.a.smithi143.stderr: -1> 2023-11-08T19:28:57.578+0000 7f5ba1124700 -1 /home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el8/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: In function 'void MDSTableServer::_prepare_logged(ceph::cref_t&, version_t)' thread 7f5ba1124700 time 2023-11-08T19:28:57.579614+0000 2023-11-08T19:28:57.601 INFO:tasks.ceph.mds.a.smithi143.stderr:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el8/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: 77: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 2) 2023-11-08T19:28:57.601 INFO:tasks.ceph.mds.a.smithi143.stderr: 2023-11-08T19:28:57.601 INFO:tasks.ceph.mds.a.smithi143.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-08T19:28:57.601 INFO:tasks.ceph.mds.a.smithi143.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f5bb29f8f5b] 2023-11-08T19:28:57.601 INFO:tasks.ceph.mds.a.smithi143.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2ba121) [0x7f5bb29f9121] 2023-11-08T19:28:57.602 INFO:tasks.ceph.mds.a.smithi143.stderr: 3: (MDSTableServer::_prepare_logged(boost::intrusive_ptr const&, unsigned long)+0x6ad) [0x55649de86f5d] 2023-11-08T19:28:57.602 INFO:tasks.ceph.mds.a.smithi143.stderr: 4: (MDSContext::complete(int)+0x5f) [0x55649ded1ecf] 2023-11-08T19:28:57.602 INFO:tasks.ceph.mds.a.smithi143.stderr: 5: (MDSIOContextBase::complete(int)+0x574) [0x55649ded26b4] 2023-11-08T19:28:57.602 INFO:tasks.ceph.mds.a.smithi143.stderr: 6: (MDSLogContextBase::complete(int)+0x5b) [0x55649ded2a7b] 2023-11-08T19:28:57.602 INFO:tasks.ceph.mds.a.smithi143.stderr: 7: (Finisher::finisher_thread_entry()+0x19d) [0x7f5bb2a968cd] 2023-11-08T19:28:57.603 INFO:tasks.ceph.mds.a.smithi143.stderr: 8: /lib64/libpthread.so.0(+0x81cf) [0x7f5bb17911cf] 2023-11-08T19:28:57.603 INFO:tasks.ceph.mds.a.smithi143.stderr: 9: clone() 2023-11-08T19:28:57.603 INFO:tasks.ceph.mds.a.smithi143.stderr: 2023-11-08T19:28:57.603 INFO:tasks.ceph.mds.a.smithi143.stderr: 0> 2023-11-08T19:28:57.579+0000 7f5ba1124700 -1 *** Caught signal (Aborted) ** 2023-11-08T19:28:57.603 INFO:tasks.ceph.mds.a.smithi143.stderr: in thread 7f5ba1124700 thread_name:MR_Finisher 2023-11-08T19:28:57.604 INFO:tasks.ceph.mds.a.smithi143.stderr: 2023-11-08T19:28:57.604 INFO:tasks.ceph.mds.a.smithi143.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-08T19:28:57.604 INFO:tasks.ceph.mds.a.smithi143.stderr: 1: /lib64/libpthread.so.0(+0x12ce0) [0x7f5bb179bce0] 2023-11-08T19:28:57.604 INFO:tasks.ceph.mds.a.smithi143.stderr: 2: gsignal() 2023-11-08T19:28:57.604 INFO:tasks.ceph.mds.a.smithi143.stderr: 3: abort() 2023-11-08T19:28:57.605 INFO:tasks.ceph.mds.a.smithi143.stderr: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f5bb29f8fb5] 2023-11-08T19:28:57.605 INFO:tasks.ceph.mds.a.smithi143.stderr: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2ba121) [0x7f5bb29f9121] 2023-11-08T19:28:57.605 INFO:tasks.ceph.mds.a.smithi143.stderr: 6: (MDSTableServer::_prepare_logged(boost::intrusive_ptr const&, unsigned long)+0x6ad) [0x55649de86f5d] 2023-11-08T19:28:57.605 INFO:tasks.ceph.mds.a.smithi143.stderr: 7: (MDSContext::complete(int)+0x5f) [0x55649ded1ecf] 2023-11-08T19:28:57.605 INFO:tasks.ceph.mds.a.smithi143.stderr: 8: (MDSIOContextBase::complete(int)+0x574) [0x55649ded26b4] 2023-11-08T19:28:57.605 INFO:tasks.ceph.mds.a.smithi143.stderr: 9: (MDSLogContextBase::complete(int)+0x5b) [0x55649ded2a7b] 2023-11-08T19:28:57.606 INFO:tasks.ceph.mds.a.smithi143.stderr: 10: (Finisher::finisher_thread_entry()+0x19d) [0x7f5bb2a968cd] 2023-11-08T19:28:57.606 INFO:tasks.ceph.mds.a.smithi143.stderr: 11: /lib64/libpthread.so.0(+0x81cf) [0x7f5bb17911cf] 2023-11-08T19:28:57.606 INFO:tasks.ceph.mds.a.smithi143.stderr: 12: clone() 2023-11-08T19:28:57.606 INFO:tasks.ceph.mds.a.smithi143.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-11-08T19:28:57.607 INFO:tasks.ceph.mds.a.smithi143.stderr: 2023-11-08T19:28:57.715 INFO:teuthology.orchestra.run.smithi143.stdout:{"mds_beacon_grace":"15.000000"} 2023-11-08T19:28:57.715 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_beacon_grace": "15.000000" } 2023-11-08T19:28:57.716 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:28:57.847 INFO:tasks.ceph.mds.a.smithi143.stderr:daemon-helper: command crashed with signal 6 2023-11-08T19:28:57.877 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:57.875+0000 7f7567d80700 1 Processor -- start 2023-11-08T19:28:57.877 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:57.875+0000 7f7567d80700 1 -- start start 2023-11-08T19:28:57.880 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:57.876+0000 7f7567d80700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f75600b2920 0x7f75600cf510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:57.880 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:57.876+0000 7f7567d80700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f75600cbe20 0x7f75600cc290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:57.880 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:57.876+0000 7f7567d80700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75600cf080 con 0x7f756015aa30 2023-11-08T19:28:57.881 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:57.876+0000 7f7567d80700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75600cc7d0 con 0x7f75600b2920 2023-11-08T19:28:57.881 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:57.876+0000 7f7567d80700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75600ce470 con 0x7f75600cbe20 2023-11-08T19:28:57.881 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:57.876+0000 7f756531b700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f75600b2920 0x7f75600cf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:57.882 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:57.876+0000 7f756531b700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f75600b2920 0x7f75600cf510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55336/0 (socket says 172.21.15.143:55336) 2023-11-08T19:28:57.882 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:57.876+0000 7f756531b700 1 -- 172.21.15.143:0/3982562265 learned_addr learned my addr 172.21.15.143:0/3982562265 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:28:57.882 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:57.876+0000 7f756531b700 1 -- 172.21.15.143:0/3982562265 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f75600cbe20 msgr2=0x7f75600cc290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:57.882 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:57.876+0000 7f756631d700 1 --2- 172.21.15.143:0/3982562265 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f75600cbe20 0x7f75600cc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:57.883 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:57.876+0000 7f756531b700 1 --2- 172.21.15.143:0/3982562265 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f75600cbe20 0x7f75600cc290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:57.883 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:57.876+0000 7f756531b700 1 -- 172.21.15.143:0/3982562265 >> v1:172.21.15.174:6789/0 conn(0x7f756015aa30 legacy=0x7f75600b2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:57.883 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:57.876+0000 7f756531b700 1 -- 172.21.15.143:0/3982562265 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75600ce750 con 0x7f75600b2920 2023-11-08T19:28:57.883 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:57.876+0000 7f756531b700 1 --2- 172.21.15.143:0/3982562265 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f75600b2920 0x7f75600cf510 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f755c00bb90 tx=0x7f755c00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=908a5b7894cf7a23 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:57.883 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:57.876+0000 7f7564b1a700 1 -- 172.21.15.143:0/3982562265 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f755c00dd10 con 0x7f75600b2920 2023-11-08T19:28:57.884 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:57.876+0000 7f7564b1a700 1 -- 172.21.15.143:0/3982562265 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f755c00de70 con 0x7f75600b2920 2023-11-08T19:28:57.884 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:57.876+0000 7f7564b1a700 1 -- 172.21.15.143:0/3982562265 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f755c004230 con 0x7f75600b2920 2023-11-08T19:28:57.884 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:57.877+0000 7f7567d80700 1 -- 172.21.15.143:0/3982562265 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f75600b2920 msgr2=0x7f75600cf510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:57.884 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:57.877+0000 7f7567d80700 1 --2- 172.21.15.143:0/3982562265 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f75600b2920 0x7f75600cf510 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f755c00bb90 tx=0x7f755c00bea0 comp rx=0 tx=0).stop 2023-11-08T19:28:57.884 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:57.877+0000 7f7567d80700 1 -- 172.21.15.143:0/3982562265 shutdown_connections 2023-11-08T19:28:57.885 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:57.877+0000 7f7567d80700 1 --2- 172.21.15.143:0/3982562265 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f75600cbe20 0x7f75600cc290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:57.885 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:57.877+0000 7f7567d80700 1 --2- 172.21.15.143:0/3982562265 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f75600b2920 0x7f75600cf510 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:57.885 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:57.877+0000 7f7567d80700 1 -- 172.21.15.143:0/3982562265 >> 172.21.15.143:0/3982562265 conn(0x7f7560099ab0 msgr2=0x7f75600d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:57.885 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:57.877+0000 7f7567d80700 1 -- 172.21.15.143:0/3982562265 shutdown_connections 2023-11-08T19:28:57.886 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:57.877+0000 7f7567d80700 1 -- 172.21.15.143:0/3982562265 wait complete. 2023-11-08T19:28:57.888 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:57.878+0000 7f7567d80700 1 Processor -- start 2023-11-08T19:28:57.889 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:57.878+0000 7f7567d80700 1 -- start start 2023-11-08T19:28:57.889 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:57.879+0000 7f7567d80700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f75600b2920 0x7f7560157b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:57.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:57.879+0000 7f7567d80700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f75600cbe20 0x7f756015d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:57.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:57.879+0000 7f7567d80700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f756015aa30 0x7f75600b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:57.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:57.879+0000 7f7567d80700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75600c8d70 con 0x7f75600cbe20 2023-11-08T19:28:57.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:57.879+0000 7f7567d80700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75600c9050 con 0x7f75600b2920 2023-11-08T19:28:57.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:57.879+0000 7f7567d80700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75600c9330 con 0x7f756015aa30 2023-11-08T19:28:57.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:57.879+0000 7f756531b700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f75600cbe20 0x7f756015d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:57.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:57.879+0000 7f756531b700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f75600cbe20 0x7f756015d2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55342/0 (socket says 172.21.15.143:55342) 2023-11-08T19:28:57.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:57.879+0000 7f756531b700 1 -- 172.21.15.143:0/2829140378 learned_addr learned my addr 172.21.15.143:0/2829140378 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:28:57.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:57.879+0000 7f756631d700 1 --2- 172.21.15.143:0/2829140378 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f756015aa30 0x7f75600b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:57.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:57.879+0000 7f7565b1c700 1 --2- 172.21.15.143:0/2829140378 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f75600b2920 0x7f7560157b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:57.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:57.879+0000 7f756531b700 1 -- 172.21.15.143:0/2829140378 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f756015aa30 msgr2=0x7f75600b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:57.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:57.879+0000 7f756531b700 1 --2- 172.21.15.143:0/2829140378 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f756015aa30 0x7f75600b1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:57.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:57.879+0000 7f756531b700 1 -- 172.21.15.143:0/2829140378 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f75600b2920 msgr2=0x7f7560157b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:57.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:57.879+0000 7f756531b700 1 --2- 172.21.15.143:0/2829140378 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f75600b2920 0x7f7560157b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:57.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:57.879+0000 7f756531b700 1 -- 172.21.15.143:0/2829140378 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f755c00b7f0 con 0x7f75600cbe20 2023-11-08T19:28:57.895 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:57.879+0000 7f756531b700 1 --2- 172.21.15.143:0/2829140378 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f75600cbe20 0x7f756015d2b0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f755c00b340 tx=0x7f755c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:57.895 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:57.880+0000 7f75567fc700 1 -- 172.21.15.143:0/2829140378 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f755c020070 con 0x7f75600cbe20 2023-11-08T19:28:57.895 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:57.880+0000 7f75567fc700 1 -- 172.21.15.143:0/2829140378 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f755c003a70 con 0x7f75600cbe20 2023-11-08T19:28:57.896 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:57.880+0000 7f75567fc700 1 -- 172.21.15.143:0/2829140378 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f755c003de0 con 0x7f75600cbe20 2023-11-08T19:28:57.896 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:57.880+0000 7f7567d80700 1 -- 172.21.15.143:0/2829140378 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75600c9610 con 0x7f75600cbe20 2023-11-08T19:28:57.896 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:57.880+0000 7f7567d80700 1 -- 172.21.15.143:0/2829140378 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7560158040 con 0x7f75600cbe20 2023-11-08T19:28:57.897 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:57.881+0000 7f7567d80700 1 -- 172.21.15.143:0/2829140378 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75600c9b30 con 0x7f75600cbe20 2023-11-08T19:28:57.897 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:57.881+0000 7f75567fc700 1 -- 172.21.15.143:0/2829140378 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f755c0193f0 con 0x7f75600cbe20 2023-11-08T19:28:57.897 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:57.881+0000 7f75567fc700 1 --2- 172.21.15.143:0/2829140378 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f754406af00 0x7f754406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:57.897 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:57.881+0000 7f75567fc700 1 -- 172.21.15.143:0/2829140378 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6887+0+0 (secure 0 0 0) 0x7f755c08cfd0 con 0x7f75600cbe20 2023-11-08T19:28:57.897 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:57.882+0000 7f7565b1c700 1 --2- 172.21.15.143:0/2829140378 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f754406af00 0x7f754406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:57.898 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:57.882+0000 7f7565b1c700 1 --2- 172.21.15.143:0/2829140378 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f754406af00 0x7f754406d3c0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f7550006fd0 tx=0x7f75500090e0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:57.898 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:57.886+0000 7f75567fc700 1 -- 172.21.15.143:0/2829140378 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f75600c9b30 con 0x7f75600cbe20 2023-11-08T19:28:58.122 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:58.120+0000 7f7567d80700 1 -- 172.21.15.143:0/2829140378 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f75600c9b30 con 0x7f75600cbe20 2023-11-08T19:28:58.123 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:58.120+0000 7f75567fc700 1 -- 172.21.15.143:0/2829140378 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f75600c9b30 con 0x7f75600cbe20 2023-11-08T19:28:58.123 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:28:58.123 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":65,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5933,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/3918286710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6835","nonce":3918286710},{"type":"v1","addr":"172.21.15.174:6837","nonce":3918286710}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":6906,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6834/3172664377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":3172664377},{"type":"v1","addr":"172.21.15.143:6834","nonce":3172664377}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:03.092245+0000","modified":"2023-11-08T19:28:56.643801+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":95,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6909,"mds_1":6918},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6909":{"gid":6909,"name":"a","rank":0,"incarnation":60,"state":"up:active","state_seq":17,"addr":"172.21.15.143:6835/3953059900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":3953059900},{"type":"v1","addr":"172.21.15.143:6835","nonce":3953059900}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6918":{"gid":6918,"name":"b","rank":1,"incarnation":54,"state":"up:active","state_seq":6,"addr":"172.21.15.174:6836/3601088618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3601088618},{"type":"v1","addr":"172.21.15.174:6836","nonce":3601088618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:28:58.125 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:58.122+0000 7f7567d80700 1 -- 172.21.15.143:0/2829140378 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f754406af00 msgr2=0x7f754406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:58.125 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:58.122+0000 7f7567d80700 1 --2- 172.21.15.143:0/2829140378 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f754406af00 0x7f754406d3c0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f7550006fd0 tx=0x7f75500090e0 comp rx=0 tx=0).stop 2023-11-08T19:28:58.126 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:58.122+0000 7f7567d80700 1 -- 172.21.15.143:0/2829140378 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f75600cbe20 msgr2=0x7f756015d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:58.126 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:58.122+0000 7f7567d80700 1 --2- 172.21.15.143:0/2829140378 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f75600cbe20 0x7f756015d2b0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f755c00b340 tx=0x7f755c004750 comp rx=0 tx=0).stop 2023-11-08T19:28:58.126 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:58.122+0000 7f7567d80700 1 -- 172.21.15.143:0/2829140378 shutdown_connections 2023-11-08T19:28:58.126 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:58.122+0000 7f7567d80700 1 --2- 172.21.15.143:0/2829140378 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f754406af00 0x7f754406d3c0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:58.127 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:58.122+0000 7f7567d80700 1 --2- 172.21.15.143:0/2829140378 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f756015aa30 0x7f75600b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:58.127 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:58.122+0000 7f7567d80700 1 --2- 172.21.15.143:0/2829140378 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f75600b2920 0x7f7560157b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:58.127 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:58.122+0000 7f7567d80700 1 --2- 172.21.15.143:0/2829140378 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f75600cbe20 0x7f756015d2b0 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:58.127 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:58.122+0000 7f7567d80700 1 -- 172.21.15.143:0/2829140378 >> 172.21.15.143:0/2829140378 conn(0x7f7560099ab0 msgr2=0x7f75600b01f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:58.127 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:58.122+0000 7f7567d80700 1 -- 172.21.15.143:0/2829140378 shutdown_connections 2023-11-08T19:28:58.127 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:28:58.122+0000 7f7567d80700 1 -- 172.21.15.143:0/2829140378 wait complete. 2023-11-08T19:28:58.128 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 65 2023-11-08T19:28:58.137 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2023-11-08T19:29:03.138 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:29:03.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:03.271+0000 7f3f89c8d700 1 Processor -- start 2023-11-08T19:29:03.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:03.271+0000 7f3f89c8d700 1 -- start start 2023-11-08T19:29:03.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:03.272+0000 7f3f89c8d700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f3f840b2510 0x7f3f840cb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:03.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:03.272+0000 7f3f89c8d700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f8406b820 con 0x7f3f8415aa20 2023-11-08T19:29:03.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:03.272+0000 7f3f89c8d700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f8406ac10 con 0x7f3f840cc4a0 2023-11-08T19:29:03.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:03.272+0000 7f3f89c8d700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f8406aef0 con 0x7f3f840b2510 2023-11-08T19:29:03.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:03.272+0000 7f3f82ffd700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f3f840b2510 0x7f3f840cb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:03.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:03.272+0000 7f3f837fe700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f3f8415aa20 0x7f3f840b2d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:51512/0 (socket says 172.21.15.143:51512) 2023-11-08T19:29:03.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:03.272+0000 7f3f837fe700 1 -- 172.21.15.143:0/3324702604 learned_addr learned my addr 172.21.15.143:0/3324702604 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:29:03.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:03.272+0000 7f3f82ffd700 1 -- 172.21.15.143:0/3324702604 >> v1:172.21.15.174:6789/0 conn(0x7f3f840cc4a0 legacy=0x7f3f8406bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:03.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:03.273+0000 7f3f82ffd700 1 -- 172.21.15.143:0/3324702604 >> v1:172.21.15.143:6789/0 conn(0x7f3f8415aa20 legacy=0x7f3f840b2d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:03.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:03.273+0000 7f3f82ffd700 1 -- 172.21.15.143:0/3324702604 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f8406b1d0 con 0x7f3f840b2510 2023-11-08T19:29:03.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:03.273+0000 7f3f82ffd700 1 --2- 172.21.15.143:0/3324702604 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3f840b2510 0x7f3f840cb4b0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f3f74009840 tx=0x7f3f74009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cfdd1f6e3d1ce71 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:03.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:03.273+0000 7f3f827fc700 1 -- 172.21.15.143:0/3324702604 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f74004230 con 0x7f3f840b2510 2023-11-08T19:29:03.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:03.273+0000 7f3f827fc700 1 -- 172.21.15.143:0/3324702604 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f74004390 con 0x7f3f840b2510 2023-11-08T19:29:03.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:03.273+0000 7f3f827fc700 1 -- 172.21.15.143:0/3324702604 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f740036a0 con 0x7f3f840b2510 2023-11-08T19:29:03.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:03.273+0000 7f3f89c8d700 1 -- 172.21.15.143:0/3324702604 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3f840b2510 msgr2=0x7f3f840cb4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:03.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:03.273+0000 7f3f89c8d700 1 --2- 172.21.15.143:0/3324702604 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3f840b2510 0x7f3f840cb4b0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f3f74009840 tx=0x7f3f74009b50 comp rx=0 tx=0).stop 2023-11-08T19:29:03.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:03.273+0000 7f3f89c8d700 1 -- 172.21.15.143:0/3324702604 shutdown_connections 2023-11-08T19:29:03.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:03.273+0000 7f3f89c8d700 1 --2- 172.21.15.143:0/3324702604 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3f840b2510 0x7f3f840cb4b0 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:03.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:03.273+0000 7f3f89c8d700 1 -- 172.21.15.143:0/3324702604 >> 172.21.15.143:0/3324702604 conn(0x7f3f840b1a20 msgr2=0x7f3f840b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:03.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:03.273+0000 7f3f89c8d700 1 -- 172.21.15.143:0/3324702604 shutdown_connections 2023-11-08T19:29:03.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:03.273+0000 7f3f89c8d700 1 -- 172.21.15.143:0/3324702604 wait complete. 2023-11-08T19:29:03.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:03.275+0000 7f3f89c8d700 1 Processor -- start 2023-11-08T19:29:03.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:03.275+0000 7f3f89c8d700 1 -- start start 2023-11-08T19:29:03.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:03.276+0000 7f3f89c8d700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3f840b2510 0x7f3f840c9ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:03.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:03.276+0000 7f3f89c8d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3f840cc4a0 0x7f3f84156ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:03.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:03.276+0000 7f3f89c8d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3f8415aa20 0x7f3f840c59d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:03.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:03.276+0000 7f3f89c8d700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f841573e0 con 0x7f3f8415aa20 2023-11-08T19:29:03.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:03.276+0000 7f3f89c8d700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f840c9040 con 0x7f3f840cc4a0 2023-11-08T19:29:03.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:03.276+0000 7f3f89c8d700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f840c7fa0 con 0x7f3f840b2510 2023-11-08T19:29:03.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:03.276+0000 7f3f83fff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3f8415aa20 0x7f3f840c59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:03.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:03.276+0000 7f3f83fff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3f8415aa20 0x7f3f840c59d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55368/0 (socket says 172.21.15.143:55368) 2023-11-08T19:29:03.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:03.276+0000 7f3f83fff700 1 -- 172.21.15.143:0/336979657 learned_addr learned my addr 172.21.15.143:0/336979657 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:29:03.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:03.276+0000 7f3f83fff700 1 -- 172.21.15.143:0/336979657 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3f840b2510 msgr2=0x7f3f840c9ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:03.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:03.276+0000 7f3f83fff700 1 --2- 172.21.15.143:0/336979657 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3f840b2510 0x7f3f840c9ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:03.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:03.276+0000 7f3f83fff700 1 -- 172.21.15.143:0/336979657 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3f840cc4a0 msgr2=0x7f3f84156ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:03.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:03.276+0000 7f3f83fff700 1 --2- 172.21.15.143:0/336979657 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3f840cc4a0 0x7f3f84156ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:03.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:03.276+0000 7f3f83fff700 1 -- 172.21.15.143:0/336979657 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f740094a0 con 0x7f3f8415aa20 2023-11-08T19:29:03.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:03.276+0000 7f3f83fff700 1 --2- 172.21.15.143:0/336979657 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3f8415aa20 0x7f3f840c59d0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c00cc40 tx=0x7f3f7c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:03.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:03.276+0000 7f3f7bfff700 1 -- 172.21.15.143:0/336979657 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f7c00aba0 con 0x7f3f8415aa20 2023-11-08T19:29:03.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:03.276+0000 7f3f7bfff700 1 -- 172.21.15.143:0/336979657 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f7c00ad00 con 0x7f3f8415aa20 2023-11-08T19:29:03.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:03.276+0000 7f3f89c8d700 1 -- 172.21.15.143:0/336979657 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f840c8280 con 0x7f3f8415aa20 2023-11-08T19:29:03.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:03.276+0000 7f3f7bfff700 1 -- 172.21.15.143:0/336979657 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f7c012660 con 0x7f3f8415aa20 2023-11-08T19:29:03.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:03.276+0000 7f3f89c8d700 1 -- 172.21.15.143:0/336979657 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f840c8840 con 0x7f3f8415aa20 2023-11-08T19:29:03.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:03.277+0000 7f3f89c8d700 1 -- 172.21.15.143:0/336979657 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f8404eee0 con 0x7f3f8415aa20 2023-11-08T19:29:03.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:03.278+0000 7f3f7bfff700 1 -- 172.21.15.143:0/336979657 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3f7c0127c0 con 0x7f3f8415aa20 2023-11-08T19:29:03.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:03.278+0000 7f3f7bfff700 1 --2- 172.21.15.143:0/336979657 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f3f6406af00 0x7f3f6406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:03.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:03.278+0000 7f3f7bfff700 1 -- 172.21.15.143:0/336979657 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6887+0+0 (secure 0 0 0) 0x7f3f7c08a1a0 con 0x7f3f8415aa20 2023-11-08T19:29:03.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:03.278+0000 7f3f837fe700 1 --2- 172.21.15.143:0/336979657 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f3f6406af00 0x7f3f6406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:03.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:03.279+0000 7f3f837fe700 1 --2- 172.21.15.143:0/336979657 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f3f6406af00 0x7f3f6406d3c0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f3f6c000f60 tx=0x7f3f6c007b10 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:03.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:03.283+0000 7f3f7bfff700 1 -- 172.21.15.143:0/336979657 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3f7c054e40 con 0x7f3f8415aa20 2023-11-08T19:29:03.496 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2023-11-08T19:29:03.516 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:03.514+0000 7f3f89c8d700 1 -- 172.21.15.143:0/336979657 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3f840b5de0 con 0x7f3f8415aa20 2023-11-08T19:29:03.516 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:03.515+0000 7f3f7bfff700 1 -- 172.21.15.143:0/336979657 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f3f7c016090 con 0x7f3f8415aa20 2023-11-08T19:29:03.517 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:29:03.517 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":65,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5933,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/3918286710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6835","nonce":3918286710},{"type":"v1","addr":"172.21.15.174:6837","nonce":3918286710}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":6906,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6834/3172664377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":3172664377},{"type":"v1","addr":"172.21.15.143:6834","nonce":3172664377}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:03.092245+0000","modified":"2023-11-08T19:28:56.643801+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":95,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6909,"mds_1":6918},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6909":{"gid":6909,"name":"a","rank":0,"incarnation":60,"state":"up:active","state_seq":17,"addr":"172.21.15.143:6835/3953059900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":3953059900},{"type":"v1","addr":"172.21.15.143:6835","nonce":3953059900}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6918":{"gid":6918,"name":"b","rank":1,"incarnation":54,"state":"up:active","state_seq":6,"addr":"172.21.15.174:6836/3601088618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3601088618},{"type":"v1","addr":"172.21.15.174:6836","nonce":3601088618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:29:03.519 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:03.517+0000 7f3f89c8d700 1 -- 172.21.15.143:0/336979657 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f3f6406af00 msgr2=0x7f3f6406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:03.519 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:03.517+0000 7f3f89c8d700 1 --2- 172.21.15.143:0/336979657 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f3f6406af00 0x7f3f6406d3c0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f3f6c000f60 tx=0x7f3f6c007b10 comp rx=0 tx=0).stop 2023-11-08T19:29:03.519 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:03.517+0000 7f3f89c8d700 1 -- 172.21.15.143:0/336979657 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3f8415aa20 msgr2=0x7f3f840c59d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:03.520 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:03.517+0000 7f3f89c8d700 1 --2- 172.21.15.143:0/336979657 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3f8415aa20 0x7f3f840c59d0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c00cc40 tx=0x7f3f7c00a060 comp rx=0 tx=0).stop 2023-11-08T19:29:03.520 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:03.517+0000 7f3f89c8d700 1 -- 172.21.15.143:0/336979657 shutdown_connections 2023-11-08T19:29:03.520 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:03.517+0000 7f3f89c8d700 1 --2- 172.21.15.143:0/336979657 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f3f6406af00 0x7f3f6406d3c0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:03.520 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:03.517+0000 7f3f89c8d700 1 --2- 172.21.15.143:0/336979657 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3f840b2510 0x7f3f840c9ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:03.520 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:03.517+0000 7f3f89c8d700 1 --2- 172.21.15.143:0/336979657 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3f8415aa20 0x7f3f840c59d0 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:03.521 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:03.517+0000 7f3f89c8d700 1 --2- 172.21.15.143:0/336979657 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3f840cc4a0 0x7f3f84156ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:03.521 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:03.517+0000 7f3f89c8d700 1 -- 172.21.15.143:0/336979657 >> 172.21.15.143:0/336979657 conn(0x7f3f840b1a20 msgr2=0x7f3f84157680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:03.521 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:03.517+0000 7f3f89c8d700 1 -- 172.21.15.143:0/336979657 shutdown_connections 2023-11-08T19:29:03.521 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:03.517+0000 7f3f89c8d700 1 -- 172.21.15.143:0/336979657 wait complete. 2023-11-08T19:29:03.522 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 65 2023-11-08T19:29:03.531 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2023-11-08T19:29:08.532 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:29:08.669 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:08.668+0000 7f5b9a8a5700 1 Processor -- start 2023-11-08T19:29:08.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:08.668+0000 7f5b9a8a5700 1 -- start start 2023-11-08T19:29:08.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:08.669+0000 7f5b9a8a5700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5b9406ed60 0x7f5b9406b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:08.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:08.669+0000 7f5b9a8a5700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b940b2a60 con 0x7f5b940cc410 2023-11-08T19:29:08.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:08.669+0000 7f5b9a8a5700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b940b2d40 con 0x7f5b94159870 2023-11-08T19:29:08.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:08.669+0000 7f5b9a8a5700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b9406b090 con 0x7f5b9406ed60 2023-11-08T19:29:08.674 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:08.669+0000 7f5b98e42700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f5b940cc410 0x7f5b940cc820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:51532/0 (socket says 172.21.15.143:51532) 2023-11-08T19:29:08.674 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:08.669+0000 7f5b98e42700 1 -- 172.21.15.143:0/2474622238 learned_addr learned my addr 172.21.15.143:0/2474622238 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:29:08.674 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:08.669+0000 7f5b92ffd700 1 -- 172.21.15.143:0/2474622238 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 251305655 0 0) 0x7f5b940b2a60 con 0x7f5b940cc410 2023-11-08T19:29:08.674 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:08.669+0000 7f5b937fe700 1 --2- 172.21.15.143:0/2474622238 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5b9406ed60 0x7f5b9406b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:08.674 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:08.669+0000 7f5b92ffd700 1 -- 172.21.15.143:0/2474622238 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b7c0035b0 con 0x7f5b940cc410 2023-11-08T19:29:08.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:08.669+0000 7f5b92ffd700 1 -- 172.21.15.143:0/2474622238 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1964332249 0 0) 0x7f5b7c0035b0 con 0x7f5b940cc410 2023-11-08T19:29:08.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:08.669+0000 7f5b92ffd700 1 -- 172.21.15.143:0/2474622238 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5b9406ed60 msgr2=0x7f5b9406b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:08.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:08.669+0000 7f5b92ffd700 1 --2- 172.21.15.143:0/2474622238 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5b9406ed60 0x7f5b9406b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:08.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:08.669+0000 7f5b92ffd700 1 -- 172.21.15.143:0/2474622238 >> v1:172.21.15.174:6789/0 conn(0x7f5b94159870 legacy=0x7f5b940b2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:08.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:08.669+0000 7f5b92ffd700 1 -- 172.21.15.143:0/2474622238 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b940b3020 con 0x7f5b940cc410 2023-11-08T19:29:08.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:08.669+0000 7f5b92ffd700 1 -- 172.21.15.143:0/2474622238 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4147756303 0 0) 0x7f5b8c0019f0 con 0x7f5b940cc410 2023-11-08T19:29:08.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:08.669+0000 7f5b92ffd700 1 -- 172.21.15.143:0/2474622238 >> v1:172.21.15.143:6789/0 conn(0x7f5b940cc410 legacy=0x7f5b940cc820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:08.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:08.669+0000 7f5b92ffd700 1 --2- 172.21.15.143:0/2474622238 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5b9406ed60 0x7f5b7c003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:08.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:08.669+0000 7f5b92ffd700 1 --2- 172.21.15.143:0/2474622238 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5b7c003f70 0x7f5b7c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:08.677 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:08.670+0000 7f5b92ffd700 1 --2- 172.21.15.143:0/2474622238 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5b7c004d90 0x7f5b7c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:08.677 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:08.670+0000 7f5b92ffd700 1 -- 172.21.15.143:0/2474622238 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b9406b090 con 0x7f5b7c004d90 2023-11-08T19:29:08.677 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:08.670+0000 7f5b92ffd700 1 -- 172.21.15.143:0/2474622238 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b7c00bbd0 con 0x7f5b7c003f70 2023-11-08T19:29:08.677 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:08.670+0000 7f5b92ffd700 1 -- 172.21.15.143:0/2474622238 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b7c00bd90 con 0x7f5b9406ed60 2023-11-08T19:29:08.677 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:08.670+0000 7f5b98e42700 1 --2- 172.21.15.143:0/2474622238 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5b7c004d90 0x7f5b7c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:08.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:08.670+0000 7f5b93fff700 1 --2- 172.21.15.143:0/2474622238 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5b9406ed60 0x7f5b7c003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:08.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:08.670+0000 7f5b937fe700 1 --2- 172.21.15.143:0/2474622238 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5b7c003f70 0x7f5b7c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:08.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:08.670+0000 7f5b98e42700 1 -- 172.21.15.143:0/2474622238 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5b9406ed60 msgr2=0x7f5b7c003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:08.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:08.670+0000 7f5b98e42700 1 --2- 172.21.15.143:0/2474622238 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5b9406ed60 0x7f5b7c003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:08.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:08.670+0000 7f5b98e42700 1 -- 172.21.15.143:0/2474622238 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5b7c003f70 msgr2=0x7f5b7c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:08.679 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:08.670+0000 7f5b98e42700 1 --2- 172.21.15.143:0/2474622238 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5b7c003f70 0x7f5b7c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:08.679 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:08.670+0000 7f5b98e42700 1 -- 172.21.15.143:0/2474622238 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b7c0035b0 con 0x7f5b7c004d90 2023-11-08T19:29:08.679 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:08.670+0000 7f5b98e42700 1 --2- 172.21.15.143:0/2474622238 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5b7c004d90 0x7f5b7c005210 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c00a830 tx=0x7f5b7c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c98c32ce008978c6 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:08.679 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:08.670+0000 7f5b92ffd700 1 -- 172.21.15.143:0/2474622238 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b8c012600 con 0x7f5b7c004d90 2023-11-08T19:29:08.679 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:08.670+0000 7f5b92ffd700 1 -- 172.21.15.143:0/2474622238 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b8c012780 con 0x7f5b7c004d90 2023-11-08T19:29:08.680 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:08.670+0000 7f5b92ffd700 1 -- 172.21.15.143:0/2474622238 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b8c012b30 con 0x7f5b7c004d90 2023-11-08T19:29:08.680 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:08.671+0000 7f5b9a8a5700 1 -- 172.21.15.143:0/2474622238 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5b7c004d90 msgr2=0x7f5b7c005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:08.680 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:08.671+0000 7f5b9a8a5700 1 --2- 172.21.15.143:0/2474622238 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5b7c004d90 0x7f5b7c005210 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c00a830 tx=0x7f5b7c005880 comp rx=0 tx=0).stop 2023-11-08T19:29:08.680 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:08.671+0000 7f5b98e42700 1 -- 172.21.15.143:0/2474622238 reap_dead start 2023-11-08T19:29:08.680 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:08.671+0000 7f5b9a8a5700 1 -- 172.21.15.143:0/2474622238 shutdown_connections 2023-11-08T19:29:08.681 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:08.671+0000 7f5b9a8a5700 1 -- 172.21.15.143:0/2474622238 >> 172.21.15.143:0/2474622238 conn(0x7f5b94099ab0 msgr2=0x7f5b940d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:08.681 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:08.671+0000 7f5b9a8a5700 1 -- 172.21.15.143:0/2474622238 shutdown_connections 2023-11-08T19:29:08.681 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:08.671+0000 7f5b9a8a5700 1 -- 172.21.15.143:0/2474622238 wait complete. 2023-11-08T19:29:08.684 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:08.672+0000 7f5b9a8a5700 1 Processor -- start 2023-11-08T19:29:08.684 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:08.672+0000 7f5b9a8a5700 1 -- start start 2023-11-08T19:29:08.684 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:08.673+0000 7f5b9a8a5700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5b9414efc0 0x7f5b941e56d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:08.684 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:08.673+0000 7f5b9a8a5700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5b941e5c10 0x7f5b9415bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:08.685 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:08.673+0000 7f5b9a8a5700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5b9415c1e0 0x7f5b940b00d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:08.685 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:08.673+0000 7f5b9a8a5700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b940c96b0 con 0x7f5b9414efc0 2023-11-08T19:29:08.685 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:08.673+0000 7f5b9a8a5700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b941e6090 con 0x7f5b941e5c10 2023-11-08T19:29:08.685 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:08.673+0000 7f5b9a8a5700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b940c87c0 con 0x7f5b9415c1e0 2023-11-08T19:29:08.685 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:08.673+0000 7f5b93fff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5b9414efc0 0x7f5b941e56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:08.686 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:08.673+0000 7f5b93fff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5b9414efc0 0x7f5b941e56d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55386/0 (socket says 172.21.15.143:55386) 2023-11-08T19:29:08.686 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:08.673+0000 7f5b93fff700 1 -- 172.21.15.143:0/259640160 learned_addr learned my addr 172.21.15.143:0/259640160 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:29:08.686 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:08.673+0000 7f5b937fe700 1 --2- 172.21.15.143:0/259640160 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5b941e5c10 0x7f5b9415bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:08.686 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:08.673+0000 7f5b98e42700 1 --2- 172.21.15.143:0/259640160 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5b9415c1e0 0x7f5b940b00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:08.687 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:08.673+0000 7f5b93fff700 1 -- 172.21.15.143:0/259640160 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5b9415c1e0 msgr2=0x7f5b940b00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:08.687 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:08.673+0000 7f5b93fff700 1 --2- 172.21.15.143:0/259640160 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5b9415c1e0 0x7f5b940b00d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:08.687 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:08.673+0000 7f5b93fff700 1 -- 172.21.15.143:0/259640160 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5b941e5c10 msgr2=0x7f5b9415bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:08.687 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:08.673+0000 7f5b93fff700 1 --2- 172.21.15.143:0/259640160 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5b941e5c10 0x7f5b9415bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:08.687 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:08.673+0000 7f5b93fff700 1 -- 172.21.15.143:0/259640160 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b940c9ad0 con 0x7f5b9414efc0 2023-11-08T19:29:08.688 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:08.673+0000 7f5b98e42700 1 --2- 172.21.15.143:0/259640160 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5b9415c1e0 0x7f5b940b00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:29:08.688 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:08.673+0000 7f5b93fff700 1 --2- 172.21.15.143:0/259640160 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5b9414efc0 0x7f5b941e56d0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f5b8400f890 tx=0x7f5b8400fc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:08.688 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:08.674+0000 7f5b90ff9700 1 -- 172.21.15.143:0/259640160 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b84001530 con 0x7f5b9414efc0 2023-11-08T19:29:08.688 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:08.674+0000 7f5b90ff9700 1 -- 172.21.15.143:0/259640160 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b8400fe90 con 0x7f5b9414efc0 2023-11-08T19:29:08.688 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:08.674+0000 7f5b9a8a5700 1 -- 172.21.15.143:0/259640160 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b9415c690 con 0x7f5b9414efc0 2023-11-08T19:29:08.688 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:08.674+0000 7f5b90ff9700 1 -- 172.21.15.143:0/259640160 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b84005500 con 0x7f5b9414efc0 2023-11-08T19:29:08.689 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:08.674+0000 7f5b9a8a5700 1 -- 172.21.15.143:0/259640160 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b940c8d10 con 0x7f5b9414efc0 2023-11-08T19:29:08.689 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:08.675+0000 7f5b9a8a5700 1 -- 172.21.15.143:0/259640160 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b940c97f0 con 0x7f5b9414efc0 2023-11-08T19:29:08.689 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:08.675+0000 7f5b90ff9700 1 -- 172.21.15.143:0/259640160 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5b840036d0 con 0x7f5b9414efc0 2023-11-08T19:29:08.689 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:08.675+0000 7f5b90ff9700 1 --2- 172.21.15.143:0/259640160 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f5b8006af00 0x7f5b8006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:08.689 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:08.676+0000 7f5b90ff9700 1 -- 172.21.15.143:0/259640160 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6887+0+0 (secure 0 0 0) 0x7f5b84089960 con 0x7f5b9414efc0 2023-11-08T19:29:08.690 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:08.676+0000 7f5b937fe700 1 --2- 172.21.15.143:0/259640160 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f5b8006af00 0x7f5b8006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:08.690 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:08.676+0000 7f5b937fe700 1 --2- 172.21.15.143:0/259640160 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f5b8006af00 0x7f5b8006d3c0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f5b940aff80 tx=0x7f5b88008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:08.690 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:08.680+0000 7f5b90ff9700 1 -- 172.21.15.143:0/259640160 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5b940c97f0 con 0x7f5b9414efc0 2023-11-08T19:29:08.910 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:08.909+0000 7f5b9a8a5700 1 -- 172.21.15.143:0/259640160 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5b940c97f0 con 0x7f5b9414efc0 2023-11-08T19:29:08.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:08.909+0000 7f5b90ff9700 1 -- 172.21.15.143:0/259640160 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f5b940c97f0 con 0x7f5b9414efc0 2023-11-08T19:29:08.911 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:29:08.912 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":65,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5933,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/3918286710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6835","nonce":3918286710},{"type":"v1","addr":"172.21.15.174:6837","nonce":3918286710}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":6906,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6834/3172664377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":3172664377},{"type":"v1","addr":"172.21.15.143:6834","nonce":3172664377}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:03.092245+0000","modified":"2023-11-08T19:28:56.643801+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":95,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6909,"mds_1":6918},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6909":{"gid":6909,"name":"a","rank":0,"incarnation":60,"state":"up:active","state_seq":17,"addr":"172.21.15.143:6835/3953059900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":3953059900},{"type":"v1","addr":"172.21.15.143:6835","nonce":3953059900}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6918":{"gid":6918,"name":"b","rank":1,"incarnation":54,"state":"up:active","state_seq":6,"addr":"172.21.15.174:6836/3601088618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3601088618},{"type":"v1","addr":"172.21.15.174:6836","nonce":3601088618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:29:08.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:08.912+0000 7f5b9a8a5700 1 -- 172.21.15.143:0/259640160 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f5b8006af00 msgr2=0x7f5b8006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:08.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:08.912+0000 7f5b9a8a5700 1 --2- 172.21.15.143:0/259640160 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f5b8006af00 0x7f5b8006d3c0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f5b940aff80 tx=0x7f5b88008040 comp rx=0 tx=0).stop 2023-11-08T19:29:08.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:08.912+0000 7f5b9a8a5700 1 -- 172.21.15.143:0/259640160 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5b9414efc0 msgr2=0x7f5b941e56d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:08.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:08.912+0000 7f5b9a8a5700 1 --2- 172.21.15.143:0/259640160 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5b9414efc0 0x7f5b941e56d0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f5b8400f890 tx=0x7f5b8400fc50 comp rx=0 tx=0).stop 2023-11-08T19:29:08.915 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:08.912+0000 7f5b9a8a5700 1 -- 172.21.15.143:0/259640160 shutdown_connections 2023-11-08T19:29:08.915 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:08.912+0000 7f5b9a8a5700 1 --2- 172.21.15.143:0/259640160 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f5b8006af00 0x7f5b8006d3c0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:08.915 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:08.912+0000 7f5b9a8a5700 1 --2- 172.21.15.143:0/259640160 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5b9415c1e0 0x7f5b940b00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:08.915 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:08.912+0000 7f5b9a8a5700 1 --2- 172.21.15.143:0/259640160 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5b9414efc0 0x7f5b941e56d0 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:08.915 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:08.912+0000 7f5b9a8a5700 1 --2- 172.21.15.143:0/259640160 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5b941e5c10 0x7f5b9415bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:08.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:08.912+0000 7f5b9a8a5700 1 -- 172.21.15.143:0/259640160 >> 172.21.15.143:0/259640160 conn(0x7f5b94099ab0 msgr2=0x7f5b940cce90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:08.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:08.912+0000 7f5b9a8a5700 1 -- 172.21.15.143:0/259640160 shutdown_connections 2023-11-08T19:29:08.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:08.912+0000 7f5b9a8a5700 1 -- 172.21.15.143:0/259640160 wait complete. 2023-11-08T19:29:08.916 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 65 2023-11-08T19:29:08.925 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2023-11-08T19:29:10.107 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~7s 2023-11-08T19:29:13.927 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:29:14.067 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.059+0000 7fa2b5b64700 1 Processor -- start 2023-11-08T19:29:14.067 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.059+0000 7fa2b5b64700 1 -- start start 2023-11-08T19:29:14.067 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.060+0000 7fa2b5b64700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa2b006c030 0x7fa2b00cd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:14.068 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.060+0000 7fa2b5b64700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2b00cd480 con 0x7fa2b015de80 2023-11-08T19:29:14.068 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.060+0000 7fa2b5b64700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2b00cc610 con 0x7fa2b006c730 2023-11-08T19:29:14.068 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.060+0000 7fa2b5b64700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b00cc870 con 0x7fa2b006c030 2023-11-08T19:29:14.068 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.060+0000 7fa2affff700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fa2b015de80 0x7fa2b00cbf00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:51552/0 (socket says 172.21.15.143:51552) 2023-11-08T19:29:14.068 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.060+0000 7fa2affff700 1 -- 172.21.15.143:0/3751177084 learned_addr learned my addr 172.21.15.143:0/3751177084 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:29:14.069 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.060+0000 7fa2aeffd700 1 --2- 172.21.15.143:0/3751177084 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa2b006c030 0x7fa2b00cd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:14.069 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.060+0000 7fa2ae7fc700 1 -- 172.21.15.143:0/3751177084 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2620951953 0 0) 0x7fa2b00cd480 con 0x7fa2b015de80 2023-11-08T19:29:14.069 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.060+0000 7fa2ae7fc700 1 -- 172.21.15.143:0/3751177084 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2980035b0 con 0x7fa2b015de80 2023-11-08T19:29:14.069 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.060+0000 7fa2ae7fc700 1 -- 172.21.15.143:0/3751177084 <== mon.1 v1:172.21.15.174:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2751057315 0 0) 0x7fa2b00cc610 con 0x7fa2b006c730 2023-11-08T19:29:14.069 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.060+0000 7fa2ae7fc700 1 -- 172.21.15.143:0/3751177084 --> v1:172.21.15.174:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2b00cd480 con 0x7fa2b006c730 2023-11-08T19:29:14.069 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.060+0000 7fa2ae7fc700 1 -- 172.21.15.143:0/3751177084 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 677577393 0 0) 0x7fa2980035b0 con 0x7fa2b015de80 2023-11-08T19:29:14.070 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.060+0000 7fa2ae7fc700 1 -- 172.21.15.143:0/3751177084 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa2b006c030 msgr2=0x7fa2b00cd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:14.070 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.060+0000 7fa2ae7fc700 1 --2- 172.21.15.143:0/3751177084 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa2b006c030 0x7fa2b00cd910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:14.070 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.060+0000 7fa2ae7fc700 1 -- 172.21.15.143:0/3751177084 >> v1:172.21.15.174:6789/0 conn(0x7fa2b006c730 legacy=0x7fa2b006cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:14.070 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.060+0000 7fa2ae7fc700 1 -- 172.21.15.143:0/3751177084 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2b00ccb50 con 0x7fa2b015de80 2023-11-08T19:29:14.071 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.060+0000 7fa2ae7fc700 1 -- 172.21.15.143:0/3751177084 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4147756303 0 0) 0x7fa2a80019f0 con 0x7fa2b015de80 2023-11-08T19:29:14.071 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.060+0000 7fa2ae7fc700 1 -- 172.21.15.143:0/3751177084 >> v1:172.21.15.143:6789/0 conn(0x7fa2b015de80 legacy=0x7fa2b00cbf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:14.071 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.060+0000 7fa2ae7fc700 1 --2- 172.21.15.143:0/3751177084 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa298003bc0 0x7fa298003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:14.071 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.060+0000 7fa2ae7fc700 1 --2- 172.21.15.143:0/3751177084 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa298004850 0x7fa298004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:14.072 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.060+0000 7fa2ae7fc700 1 --2- 172.21.15.143:0/3751177084 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa2b006c030 0x7fa298005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:14.072 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.060+0000 7fa2ae7fc700 1 -- 172.21.15.143:0/3751177084 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b00cc870 con 0x7fa298003bc0 2023-11-08T19:29:14.072 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.060+0000 7fa2ae7fc700 1 -- 172.21.15.143:0/3751177084 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa29800bac0 con 0x7fa298004850 2023-11-08T19:29:14.072 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.060+0000 7fa2ae7fc700 1 -- 172.21.15.143:0/3751177084 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa29800bc50 con 0x7fa2b006c030 2023-11-08T19:29:14.072 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.061+0000 7fa2af7fe700 1 --2- 172.21.15.143:0/3751177084 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa298003bc0 0x7fa298003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:14.072 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.061+0000 7fa2affff700 1 --2- 172.21.15.143:0/3751177084 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa2b006c030 0x7fa298005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:14.073 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.061+0000 7fa2aeffd700 1 --2- 172.21.15.143:0/3751177084 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa298004850 0x7fa298004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:14.073 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.061+0000 7fa2af7fe700 1 -- 172.21.15.143:0/3751177084 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa2b006c030 msgr2=0x7fa298005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:14.073 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.061+0000 7fa2af7fe700 1 --2- 172.21.15.143:0/3751177084 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa2b006c030 0x7fa298005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:14.073 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.061+0000 7fa2af7fe700 1 -- 172.21.15.143:0/3751177084 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa298004850 msgr2=0x7fa298004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:14.073 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.061+0000 7fa2af7fe700 1 --2- 172.21.15.143:0/3751177084 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa298004850 0x7fa298004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:14.074 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.061+0000 7fa2af7fe700 1 -- 172.21.15.143:0/3751177084 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2980035b0 con 0x7fa298003bc0 2023-11-08T19:29:14.074 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.061+0000 7fa2affff700 1 --2- 172.21.15.143:0/3751177084 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa2b006c030 0x7fa298005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:29:14.074 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.061+0000 7fa2af7fe700 1 --2- 172.21.15.143:0/3751177084 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa298003bc0 0x7fa298003fc0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fa2a0001940 tx=0x7fa298005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30aa2c5f25e625ad server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:14.074 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.061+0000 7fa2ae7fc700 1 -- 172.21.15.143:0/3751177084 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a0001430 con 0x7fa298003bc0 2023-11-08T19:29:14.075 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.061+0000 7fa2ae7fc700 1 -- 172.21.15.143:0/3751177084 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2a0001d70 con 0x7fa298003bc0 2023-11-08T19:29:14.075 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.061+0000 7fa2ae7fc700 1 -- 172.21.15.143:0/3751177084 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a0005510 con 0x7fa298003bc0 2023-11-08T19:29:14.075 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.062+0000 7fa2b5b64700 1 -- 172.21.15.143:0/3751177084 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa298003bc0 msgr2=0x7fa298003fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:14.075 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.062+0000 7fa2b5b64700 1 --2- 172.21.15.143:0/3751177084 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa298003bc0 0x7fa298003fc0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fa2a0001940 tx=0x7fa298005880 comp rx=0 tx=0).stop 2023-11-08T19:29:14.075 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.062+0000 7fa2affff700 1 -- 172.21.15.143:0/3751177084 reap_dead start 2023-11-08T19:29:14.076 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.062+0000 7fa2b5b64700 1 -- 172.21.15.143:0/3751177084 shutdown_connections 2023-11-08T19:29:14.076 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.062+0000 7fa2b5b64700 1 -- 172.21.15.143:0/3751177084 >> 172.21.15.143:0/3751177084 conn(0x7fa2b008b000 msgr2=0x7fa2b009ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:14.076 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.062+0000 7fa2b5b64700 1 -- 172.21.15.143:0/3751177084 shutdown_connections 2023-11-08T19:29:14.076 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.062+0000 7fa2b5b64700 1 -- 172.21.15.143:0/3751177084 wait complete. 2023-11-08T19:29:14.076 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.063+0000 7fa2b5b64700 1 Processor -- start 2023-11-08T19:29:14.077 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.063+0000 7fa2b5b64700 1 -- start start 2023-11-08T19:29:14.077 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.064+0000 7fa2b5b64700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa298003bc0 0x7fa2b01f4230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:14.077 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.064+0000 7fa2b5b64700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa2b00cb0c0 0x7fa2b00cdcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:14.077 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.064+0000 7fa2b5b64700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa2b00cb540 0x7fa2b00ce200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:14.077 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.064+0000 7fa2b5b64700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b00c98e0 con 0x7fa2b00cb540 2023-11-08T19:29:14.077 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.064+0000 7fa2b5b64700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b01f4770 con 0x7fa298003bc0 2023-11-08T19:29:14.078 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.064+0000 7fa2b5b64700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b00ca300 con 0x7fa2b00cb0c0 2023-11-08T19:29:14.078 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.064+0000 7fa2affff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa2b00cb540 0x7fa2b00ce200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:14.078 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.064+0000 7fa2affff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa2b00cb540 0x7fa2b00ce200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55410/0 (socket says 172.21.15.143:55410) 2023-11-08T19:29:14.078 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.064+0000 7fa2affff700 1 -- 172.21.15.143:0/3195917234 learned_addr learned my addr 172.21.15.143:0/3195917234 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:29:14.078 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.064+0000 7fa2aeffd700 1 --2- 172.21.15.143:0/3195917234 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa2b00cb0c0 0x7fa2b00cdcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:14.079 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.064+0000 7fa2af7fe700 1 --2- 172.21.15.143:0/3195917234 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa298003bc0 0x7fa2b01f4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:14.079 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.064+0000 7fa2affff700 1 -- 172.21.15.143:0/3195917234 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa2b00cb0c0 msgr2=0x7fa2b00cdcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:14.079 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.064+0000 7fa2affff700 1 --2- 172.21.15.143:0/3195917234 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa2b00cb0c0 0x7fa2b00cdcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:14.079 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.064+0000 7fa2affff700 1 -- 172.21.15.143:0/3195917234 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa298003bc0 msgr2=0x7fa2b01f4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:14.079 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.064+0000 7fa2affff700 1 --2- 172.21.15.143:0/3195917234 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa298003bc0 0x7fa2b01f4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:14.080 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.064+0000 7fa2affff700 1 -- 172.21.15.143:0/3195917234 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2b00ca5e0 con 0x7fa2b00cb540 2023-11-08T19:29:14.080 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.064+0000 7fa2affff700 1 --2- 172.21.15.143:0/3195917234 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa2b00cb540 0x7fa2b00ce200 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fa2b00c8930 tx=0x7fa2a8001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:14.080 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.065+0000 7fa297fff700 1 -- 172.21.15.143:0/3195917234 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a8017070 con 0x7fa2b00cb540 2023-11-08T19:29:14.080 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.065+0000 7fa297fff700 1 -- 172.21.15.143:0/3195917234 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2a800ec20 con 0x7fa2b00cb540 2023-11-08T19:29:14.080 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.065+0000 7fa2b5b64700 1 -- 172.21.15.143:0/3195917234 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2b00ca8c0 con 0x7fa2b00cb540 2023-11-08T19:29:14.081 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.065+0000 7fa297fff700 1 -- 172.21.15.143:0/3195917234 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a800edd0 con 0x7fa2b00cb540 2023-11-08T19:29:14.081 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.065+0000 7fa2b5b64700 1 -- 172.21.15.143:0/3195917234 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2b00b44d0 con 0x7fa2b00cb540 2023-11-08T19:29:14.081 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.066+0000 7fa2b5b64700 1 -- 172.21.15.143:0/3195917234 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2b00cade0 con 0x7fa2b00cb540 2023-11-08T19:29:14.082 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.066+0000 7fa297fff700 1 -- 172.21.15.143:0/3195917234 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa2a8013420 con 0x7fa2b00cb540 2023-11-08T19:29:14.082 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.066+0000 7fa297fff700 1 --2- 172.21.15.143:0/3195917234 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa29c06af00 0x7fa29c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:14.082 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.066+0000 7fa297fff700 1 -- 172.21.15.143:0/3195917234 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6887+0+0 (secure 0 0 0) 0x7fa2a8089720 con 0x7fa2b00cb540 2023-11-08T19:29:14.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.067+0000 7fa2af7fe700 1 --2- 172.21.15.143:0/3195917234 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa29c06af00 0x7fa29c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:14.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.067+0000 7fa2af7fe700 1 --2- 172.21.15.143:0/3195917234 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa29c06af00 0x7fa29c06d3c0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fa2a0005040 tx=0x7fa2a000a000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:14.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.071+0000 7fa297fff700 1 -- 172.21.15.143:0/3195917234 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa2b00cade0 con 0x7fa2b00cb540 2023-11-08T19:29:14.302 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.301+0000 7fa2b5b64700 1 -- 172.21.15.143:0/3195917234 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa2b00cade0 con 0x7fa2b00cb540 2023-11-08T19:29:14.303 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.301+0000 7fa297fff700 1 -- 172.21.15.143:0/3195917234 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) v1 ==== 94+0+4991 (secure 0 0 0) 0x7fa2b00cade0 con 0x7fa2b00cb540 2023-11-08T19:29:14.303 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:29:14.303 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":66,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5933,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/3918286710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6835","nonce":3918286710},{"type":"v1","addr":"172.21.15.174:6837","nonce":3918286710}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":6906,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6834/3172664377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":3172664377},{"type":"v1","addr":"172.21.15.143:6834","nonce":3172664377}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:03.092245+0000","modified":"2023-11-08T19:29:12.604102+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":95,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6909,"mds_1":6918},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6909":{"gid":6909,"name":"a","rank":0,"incarnation":60,"state":"up:active","state_seq":17,"addr":"172.21.15.143:6835/3953059900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":3953059900},{"type":"v1","addr":"172.21.15.143:6835","nonce":3953059900}]},"join_fscid":-1,"laggy_since":"2023-11-08T19:29:12.604102+0000","export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6918":{"gid":6918,"name":"b","rank":1,"incarnation":54,"state":"up:active","state_seq":6,"addr":"172.21.15.174:6836/3601088618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3601088618},{"type":"v1","addr":"172.21.15.174:6836","nonce":3601088618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:29:14.305 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.303+0000 7fa2b5b64700 1 -- 172.21.15.143:0/3195917234 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa29c06af00 msgr2=0x7fa29c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:14.305 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.303+0000 7fa2b5b64700 1 --2- 172.21.15.143:0/3195917234 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa29c06af00 0x7fa29c06d3c0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fa2a0005040 tx=0x7fa2a000a000 comp rx=0 tx=0).stop 2023-11-08T19:29:14.305 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.303+0000 7fa2b5b64700 1 -- 172.21.15.143:0/3195917234 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa2b00cb540 msgr2=0x7fa2b00ce200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:14.306 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.303+0000 7fa2b5b64700 1 --2- 172.21.15.143:0/3195917234 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa2b00cb540 0x7fa2b00ce200 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fa2b00c8930 tx=0x7fa2a8001ae0 comp rx=0 tx=0).stop 2023-11-08T19:29:14.306 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.303+0000 7fa2b5b64700 1 -- 172.21.15.143:0/3195917234 shutdown_connections 2023-11-08T19:29:14.306 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.303+0000 7fa2b5b64700 1 --2- 172.21.15.143:0/3195917234 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa29c06af00 0x7fa29c06d3c0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:14.307 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.303+0000 7fa2b5b64700 1 --2- 172.21.15.143:0/3195917234 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa2b00cb540 0x7fa2b00ce200 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:14.307 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.303+0000 7fa2b5b64700 1 --2- 172.21.15.143:0/3195917234 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa298003bc0 0x7fa2b01f4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:14.307 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.303+0000 7fa2b5b64700 1 --2- 172.21.15.143:0/3195917234 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa2b00cb0c0 0x7fa2b00cdcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:14.308 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.303+0000 7fa2b5b64700 1 -- 172.21.15.143:0/3195917234 >> 172.21.15.143:0/3195917234 conn(0x7fa2b008b000 msgr2=0x7fa2b0160480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:14.308 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.303+0000 7fa2b5b64700 1 -- 172.21.15.143:0/3195917234 shutdown_connections 2023-11-08T19:29:14.308 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.303+0000 7fa2b5b64700 1 -- 172.21.15.143:0/3195917234 wait complete. 2023-11-08T19:29:14.309 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 66 2023-11-08T19:29:14.317 DEBUG:tasks.ceph_test_case:wait_until_true: success in 15s and 0 retries 2023-11-08T19:29:14.318 DEBUG:teuthology.orchestra.run.smithi143:> sudo sysctl -n kernel.core_pattern 2023-11-08T19:29:14.344 INFO:teuthology.orchestra.run.smithi143.stdout:/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-08T19:29:14.345 INFO:tasks.cephfs.cephfs_test_case:Clearing core from directory: /home/ubuntu/cephtest/archive/coredump 2023-11-08T19:29:14.346 DEBUG:teuthology.orchestra.run.smithi143:> cd /home/ubuntu/cephtest/archive/coredump && sudo ls | sudo xargs file 2023-11-08T19:29:14.415 INFO:teuthology.orchestra.run.smithi143.stdout:1699471737.131231.core: ELF 64-bit LSB core file, x86-64, version 1 (SYSV), SVR4-style, from 'ceph-mds -f --cluster ceph -i a', real uid: 0, effective uid: 0, real gid: 0, effective gid: 0, execfn: '/bin/ceph-mds', platform: 'x86_64' 2023-11-08T19:29:14.417 INFO:tasks.cephfs.cephfs_test_case:Enumerated cores: ['1699471737.131231.core'] 2023-11-08T19:29:14.418 INFO:tasks.cephfs.cephfs_test_case:Found core file 1699471737.131231.core, deleting it 2023-11-08T19:29:14.418 DEBUG:teuthology.orchestra.run.smithi143:> cd /home/ubuntu/cephtest/archive/coredump && sudo rm -f 1699471737.131231.core 2023-11-08T19:29:14.486 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail 6:0 2023-11-08T19:29:14.657 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.655+0000 7f255773c700 1 Processor -- start 2023-11-08T19:29:14.657 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.655+0000 7f255773c700 1 -- start start 2023-11-08T19:29:14.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.656+0000 7f255773c700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f255015aa30 0x7f25500cf510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:14.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.656+0000 7f255773c700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f25500b2210 0x7f25500b26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:14.668 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.656+0000 7f255773c700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25500cf080 con 0x7f25500cbe20 2023-11-08T19:29:14.668 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.656+0000 7f255773c700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25500b2be0 con 0x7f25500b2210 2023-11-08T19:29:14.668 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.656+0000 7f255773c700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25500ce470 con 0x7f255015aa30 2023-11-08T19:29:14.669 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.656+0000 7f2554cd7700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f25500b2210 0x7f25500b26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:14.669 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.656+0000 7f2554cd7700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f25500b2210 0x7f25500b26a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55416/0 (socket says 172.21.15.143:55416) 2023-11-08T19:29:14.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.656+0000 7f2554cd7700 1 -- 172.21.15.143:0/2451488467 learned_addr learned my addr 172.21.15.143:0/2451488467 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:29:14.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.656+0000 7f25554d8700 1 --2- 172.21.15.143:0/2451488467 >> v2:172.21.15.174:3300/0 conn(0x7f255015aa30 0x7f25500cf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:14.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.656+0000 7f2554cd7700 1 -- 172.21.15.143:0/2451488467 >> v2:172.21.15.174:3300/0 conn(0x7f255015aa30 msgr2=0x7f25500cf510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:14.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.657+0000 7f2554cd7700 1 --2- 172.21.15.143:0/2451488467 >> v2:172.21.15.174:3300/0 conn(0x7f255015aa30 0x7f25500cf510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:14.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.657+0000 7f2554cd7700 1 -- 172.21.15.143:0/2451488467 >> v1:172.21.15.174:6789/0 conn(0x7f25500cbe20 legacy=0x7f25500cc220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:14.672 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.657+0000 7f2554cd7700 1 -- 172.21.15.143:0/2451488467 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25500ce750 con 0x7f25500b2210 2023-11-08T19:29:14.672 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.657+0000 7f2554cd7700 1 --2- 172.21.15.143:0/2451488467 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f25500b2210 0x7f25500b26a0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f2550071c90 tx=0x7f2544009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ba7c88efa21d66a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:14.672 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.657+0000 7f254ffff700 1 -- 172.21.15.143:0/2451488467 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2544004230 con 0x7f25500b2210 2023-11-08T19:29:14.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.657+0000 7f254ffff700 1 -- 172.21.15.143:0/2451488467 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2544004390 con 0x7f25500b2210 2023-11-08T19:29:14.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.657+0000 7f254ffff700 1 -- 172.21.15.143:0/2451488467 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25440036a0 con 0x7f25500b2210 2023-11-08T19:29:14.674 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.657+0000 7f255773c700 1 -- 172.21.15.143:0/2451488467 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f25500b2210 msgr2=0x7f25500b26a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:14.674 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.657+0000 7f255773c700 1 --2- 172.21.15.143:0/2451488467 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f25500b2210 0x7f25500b26a0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f2550071c90 tx=0x7f2544009bd0 comp rx=0 tx=0).stop 2023-11-08T19:29:14.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.657+0000 7f255773c700 1 -- 172.21.15.143:0/2451488467 shutdown_connections 2023-11-08T19:29:14.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.657+0000 7f255773c700 1 --2- 172.21.15.143:0/2451488467 >> v2:172.21.15.174:3300/0 conn(0x7f255015aa30 0x7f25500cf510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:14.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.657+0000 7f255773c700 1 --2- 172.21.15.143:0/2451488467 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f25500b2210 0x7f25500b26a0 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:14.677 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.657+0000 7f255773c700 1 -- 172.21.15.143:0/2451488467 >> 172.21.15.143:0/2451488467 conn(0x7f2550099ab0 msgr2=0x7f25500d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:14.677 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.658+0000 7f255773c700 1 -- 172.21.15.143:0/2451488467 shutdown_connections 2023-11-08T19:29:14.677 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.658+0000 7f255773c700 1 -- 172.21.15.143:0/2451488467 wait complete. 2023-11-08T19:29:14.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.659+0000 7f255773c700 1 Processor -- start 2023-11-08T19:29:14.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.659+0000 7f255773c700 1 -- start start 2023-11-08T19:29:14.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.660+0000 7f255773c700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f25500b2210 0x7f255014bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:14.679 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.660+0000 7f255773c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f25500cbe20 0x7f255014ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:14.679 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.660+0000 7f255773c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f255015aa30 0x7f255006e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:14.680 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.660+0000 7f255773c700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f255014f3a0 con 0x7f255015aa30 2023-11-08T19:29:14.680 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.660+0000 7f255773c700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25500b4f60 con 0x7f25500cbe20 2023-11-08T19:29:14.680 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.660+0000 7f255773c700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25500b5240 con 0x7f25500b2210 2023-11-08T19:29:14.681 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.660+0000 7f2555cd9700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f255015aa30 0x7f255006e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:14.681 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.660+0000 7f2555cd9700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f255015aa30 0x7f255006e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55426/0 (socket says 172.21.15.143:55426) 2023-11-08T19:29:14.681 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.660+0000 7f2555cd9700 1 -- 172.21.15.143:0/2245944260 learned_addr learned my addr 172.21.15.143:0/2245944260 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:29:14.682 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.660+0000 7f25554d8700 1 --2- 172.21.15.143:0/2245944260 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f25500b2210 0x7f255014bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:14.682 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.660+0000 7f2554cd7700 1 --2- 172.21.15.143:0/2245944260 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f25500cbe20 0x7f255014ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:14.683 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.660+0000 7f2555cd9700 1 -- 172.21.15.143:0/2245944260 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f25500b2210 msgr2=0x7f255014bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:14.683 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.660+0000 7f2555cd9700 1 --2- 172.21.15.143:0/2245944260 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f25500b2210 0x7f255014bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:14.684 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.660+0000 7f2555cd9700 1 -- 172.21.15.143:0/2245944260 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f25500cbe20 msgr2=0x7f255014ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:14.684 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.660+0000 7f2555cd9700 1 --2- 172.21.15.143:0/2245944260 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f25500cbe20 0x7f255014ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:14.684 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.660+0000 7f2555cd9700 1 -- 172.21.15.143:0/2245944260 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2544009550 con 0x7f255015aa30 2023-11-08T19:29:14.685 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.660+0000 7f25554d8700 1 --2- 172.21.15.143:0/2245944260 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f25500b2210 0x7f255014bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:29:14.685 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.661+0000 7f2555cd9700 1 --2- 172.21.15.143:0/2245944260 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f255015aa30 0x7f255006e730 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f254000c450 tx=0x7f254000c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:14.686 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.661+0000 7f254dffb700 1 -- 172.21.15.143:0/2245944260 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2540011790 con 0x7f255015aa30 2023-11-08T19:29:14.686 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.661+0000 7f254dffb700 1 -- 172.21.15.143:0/2245944260 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2540011910 con 0x7f255015aa30 2023-11-08T19:29:14.686 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.661+0000 7f255773c700 1 -- 172.21.15.143:0/2245944260 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25500b5520 con 0x7f255015aa30 2023-11-08T19:29:14.687 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.661+0000 7f254dffb700 1 -- 172.21.15.143:0/2245944260 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2540011cc0 con 0x7f255015aa30 2023-11-08T19:29:14.687 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.661+0000 7f255773c700 1 -- 172.21.15.143:0/2245944260 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25500b5ae0 con 0x7f255015aa30 2023-11-08T19:29:14.687 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.662+0000 7f255773c700 1 -- 172.21.15.143:0/2245944260 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25500b5cd0 con 0x7f255015aa30 2023-11-08T19:29:14.688 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.662+0000 7f254dffb700 1 -- 172.21.15.143:0/2245944260 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f254001c460 con 0x7f255015aa30 2023-11-08T19:29:14.688 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.663+0000 7f254dffb700 1 --2- 172.21.15.143:0/2245944260 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f253c06af00 0x7f253c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:14.689 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.663+0000 7f254dffb700 1 -- 172.21.15.143:0/2245944260 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6887+0+0 (secure 0 0 0) 0x7f2540089820 con 0x7f255015aa30 2023-11-08T19:29:14.689 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.663+0000 7f25554d8700 1 --2- 172.21.15.143:0/2245944260 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f253c06af00 0x7f253c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:14.689 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.663+0000 7f25554d8700 1 --2- 172.21.15.143:0/2245944260 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f253c06af00 0x7f253c06d3c0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f254800e390 tx=0x7f2548007600 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:14.690 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.667+0000 7f254dffb700 1 -- 172.21.15.143:0/2245944260 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2540054430 con 0x7f255015aa30 2023-11-08T19:29:14.940 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.939+0000 7f255773c700 1 -- 172.21.15.143:0/2245944260 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "6:0"} v 0) v1 -- 0x7f255004eee0 con 0x7f255015aa30 2023-11-08T19:29:14.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.941+0000 7f254dffb700 1 -- 172.21.15.143:0/2245944260 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "6:0"}]=0 failed mds gid 6909 v67) v1 ==== 97+0+0 (secure 0 0 0) 0x7f2540011e20 con 0x7f255015aa30 2023-11-08T19:29:14.944 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.943+0000 7f255773c700 1 -- 172.21.15.143:0/2245944260 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f253c06af00 msgr2=0x7f253c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:14.945 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.943+0000 7f255773c700 1 --2- 172.21.15.143:0/2245944260 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f253c06af00 0x7f253c06d3c0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f254800e390 tx=0x7f2548007600 comp rx=0 tx=0).stop 2023-11-08T19:29:14.945 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.943+0000 7f255773c700 1 -- 172.21.15.143:0/2245944260 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f255015aa30 msgr2=0x7f255006e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:14.945 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.943+0000 7f255773c700 1 --2- 172.21.15.143:0/2245944260 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f255015aa30 0x7f255006e730 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f254000c450 tx=0x7f254000c810 comp rx=0 tx=0).stop 2023-11-08T19:29:14.945 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.943+0000 7f255773c700 1 -- 172.21.15.143:0/2245944260 shutdown_connections 2023-11-08T19:29:14.946 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.943+0000 7f255773c700 1 --2- 172.21.15.143:0/2245944260 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f253c06af00 0x7f253c06d3c0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:14.946 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.943+0000 7f255773c700 1 --2- 172.21.15.143:0/2245944260 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f25500b2210 0x7f255014bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:14.946 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.943+0000 7f255773c700 1 --2- 172.21.15.143:0/2245944260 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f255015aa30 0x7f255006e730 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:14.946 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.943+0000 7f255773c700 1 --2- 172.21.15.143:0/2245944260 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f25500cbe20 0x7f255014ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:14.946 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.943+0000 7f255773c700 1 -- 172.21.15.143:0/2245944260 >> 172.21.15.143:0/2245944260 conn(0x7f2550099ab0 msgr2=0x7f25500cc200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:14.947 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.943+0000 7f255773c700 1 -- 172.21.15.143:0/2245944260 shutdown_connections 2023-11-08T19:29:14.947 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:14.943+0000 7f255773c700 1 -- 172.21.15.143:0/2245944260 wait complete. 2023-11-08T19:29:14.947 INFO:teuthology.orchestra.run.smithi143.stderr:failed mds gid 6909 2023-11-08T19:29:14.963 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-08T19:29:14.963 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-08T19:29:14.963 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-08T19:29:14.964 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T19:29:14.964 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-08T19:29:14.965 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi143 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a' 2023-11-08T19:29:14.965 INFO:tasks.ceph.mds.a:Stopped 2023-11-08T19:29:14.965 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-08T19:29:14.968 INFO:tasks.ceph.mds.a:Started 2023-11-08T19:29:14.968 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:29:15.044 INFO:tasks.ceph.mds.a.smithi143.stdout:starting mds.a at 2023-11-08T19:29:15.091 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:15.089+0000 7f45eb252700 1 Processor -- start 2023-11-08T19:29:15.092 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:15.089+0000 7f45eb252700 1 -- start start 2023-11-08T19:29:15.094 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:15.090+0000 7f45eb252700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f45e40b2920 0x7f45e40cf510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:15.094 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:15.090+0000 7f45eb252700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f45e40cbe20 0x7f45e40cc250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:15.095 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:15.090+0000 7f45eb252700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45e40cf080 con 0x7f45e415aa30 2023-11-08T19:29:15.095 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:15.090+0000 7f45eb252700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45e40ce470 con 0x7f45e40cbe20 2023-11-08T19:29:15.095 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:15.090+0000 7f45eb252700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45e40ce750 con 0x7f45e40b2920 2023-11-08T19:29:15.095 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:15.090+0000 7f45e97ef700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f45e40cbe20 0x7f45e40cc250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:15.095 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:15.090+0000 7f45e8fee700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f45e415aa30 0x7f45e40b2210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:51610/0 (socket says 172.21.15.143:51610) 2023-11-08T19:29:15.096 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:15.090+0000 7f45e8fee700 1 -- 172.21.15.143:0/912359173 learned_addr learned my addr 172.21.15.143:0/912359173 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:29:15.096 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:15.091+0000 7f45e3fff700 1 --2- 172.21.15.143:0/912359173 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f45e40b2920 0x7f45e40cf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:15.096 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:15.091+0000 7f45e97ef700 1 -- 172.21.15.143:0/912359173 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f45e40b2920 msgr2=0x7f45e40cf510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:15.096 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:15.091+0000 7f45e97ef700 1 --2- 172.21.15.143:0/912359173 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f45e40b2920 0x7f45e40cf510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:15.096 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:15.091+0000 7f45e97ef700 1 -- 172.21.15.143:0/912359173 >> v1:172.21.15.143:6789/0 conn(0x7f45e415aa30 legacy=0x7f45e40b2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:15.097 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:15.091+0000 7f45e37fe700 1 -- 172.21.15.143:0/912359173 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1944125786 0 0) 0x7f45e40cf080 con 0x7f45e415aa30 2023-11-08T19:29:15.097 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:15.091+0000 7f45e97ef700 1 -- 172.21.15.143:0/912359173 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45e40cea30 con 0x7f45e40cbe20 2023-11-08T19:29:15.097 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:15.091+0000 7f45e97ef700 1 --2- 172.21.15.143:0/912359173 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f45e40cbe20 0x7f45e40cc250 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f45dc009d90 tx=0x7f45dc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd96afb009793c3f server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:15.097 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:15.091+0000 7f45e37fe700 1 -- 172.21.15.143:0/912359173 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45dc00e070 con 0x7f45e40cbe20 2023-11-08T19:29:15.097 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:15.091+0000 7f45e37fe700 1 -- 172.21.15.143:0/912359173 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45dc00bbd0 con 0x7f45e40cbe20 2023-11-08T19:29:15.098 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:15.091+0000 7f45e37fe700 1 -- 172.21.15.143:0/912359173 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45dc012430 con 0x7f45e40cbe20 2023-11-08T19:29:15.098 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:15.091+0000 7f45eb252700 1 -- 172.21.15.143:0/912359173 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f45e40cbe20 msgr2=0x7f45e40cc250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:15.098 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:15.092+0000 7f45eb252700 1 --2- 172.21.15.143:0/912359173 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f45e40cbe20 0x7f45e40cc250 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f45dc009d90 tx=0x7f45dc00b330 comp rx=0 tx=0).stop 2023-11-08T19:29:15.098 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:15.092+0000 7f45eb252700 1 -- 172.21.15.143:0/912359173 shutdown_connections 2023-11-08T19:29:15.098 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:15.092+0000 7f45eb252700 1 --2- 172.21.15.143:0/912359173 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f45e40cbe20 0x7f45e40cc250 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:15.099 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:15.092+0000 7f45eb252700 1 --2- 172.21.15.143:0/912359173 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f45e40b2920 0x7f45e40cf510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:15.099 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:15.092+0000 7f45eb252700 1 -- 172.21.15.143:0/912359173 >> 172.21.15.143:0/912359173 conn(0x7f45e4099ab0 msgr2=0x7f45e40d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:15.099 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:15.092+0000 7f45eb252700 1 -- 172.21.15.143:0/912359173 shutdown_connections 2023-11-08T19:29:15.099 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:15.092+0000 7f45eb252700 1 -- 172.21.15.143:0/912359173 wait complete. 2023-11-08T19:29:15.099 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:15.093+0000 7f45eb252700 1 Processor -- start 2023-11-08T19:29:15.100 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:15.093+0000 7f45eb252700 1 -- start start 2023-11-08T19:29:15.103 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:15.094+0000 7f45eb252700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f45e40b2920 0x7f45e40c6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:15.103 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:15.094+0000 7f45eb252700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f45e40cbe20 0x7f45e40b55a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:15.103 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:15.094+0000 7f45eb252700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f45e415aa30 0x7f45e4153c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:15.103 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:15.094+0000 7f45eb252700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45e40c45f0 con 0x7f45e415aa30 2023-11-08T19:29:15.103 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:15.094+0000 7f45eb252700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45e40b5ae0 con 0x7f45e40b2920 2023-11-08T19:29:15.104 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:15.094+0000 7f45eb252700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45e40c4230 con 0x7f45e40cbe20 2023-11-08T19:29:15.104 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:15.094+0000 7f45e97ef700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f45e415aa30 0x7f45e4153c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:15.104 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:15.094+0000 7f45e97ef700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f45e415aa30 0x7f45e4153c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55466/0 (socket says 172.21.15.143:55466) 2023-11-08T19:29:15.104 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:15.094+0000 7f45e97ef700 1 -- 172.21.15.143:0/3997075360 learned_addr learned my addr 172.21.15.143:0/3997075360 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:29:15.104 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:15.094+0000 7f45e3fff700 1 --2- 172.21.15.143:0/3997075360 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f45e40cbe20 0x7f45e40b55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:15.105 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:15.094+0000 7f45e8fee700 1 --2- 172.21.15.143:0/3997075360 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f45e40b2920 0x7f45e40c6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:15.105 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:15.094+0000 7f45e97ef700 1 -- 172.21.15.143:0/3997075360 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f45e40cbe20 msgr2=0x7f45e40b55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:15.105 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:15.094+0000 7f45e97ef700 1 --2- 172.21.15.143:0/3997075360 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f45e40cbe20 0x7f45e40b55a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:15.105 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:15.094+0000 7f45e97ef700 1 -- 172.21.15.143:0/3997075360 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f45e40b2920 msgr2=0x7f45e40c6ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:15.105 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:15.094+0000 7f45e97ef700 1 --2- 172.21.15.143:0/3997075360 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f45e40b2920 0x7f45e40c6ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:15.106 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:15.094+0000 7f45e97ef700 1 -- 172.21.15.143:0/3997075360 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45dc009a70 con 0x7f45e415aa30 2023-11-08T19:29:15.106 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:15.094+0000 7f45e8fee700 1 --2- 172.21.15.143:0/3997075360 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f45e40b2920 0x7f45e40c6ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:29:15.106 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:15.094+0000 7f45e97ef700 1 --2- 172.21.15.143:0/3997075360 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f45e415aa30 0x7f45e4153c40 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f45e4070ae0 tx=0x7f45dc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:15.106 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:15.094+0000 7f45e17fa700 1 -- 172.21.15.143:0/3997075360 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45dc012430 con 0x7f45e415aa30 2023-11-08T19:29:15.106 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:15.094+0000 7f45e17fa700 1 -- 172.21.15.143:0/3997075360 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45dc020070 con 0x7f45e415aa30 2023-11-08T19:29:15.107 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:15.094+0000 7f45eb252700 1 -- 172.21.15.143:0/3997075360 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45e40b5d80 con 0x7f45e415aa30 2023-11-08T19:29:15.107 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:15.094+0000 7f45e17fa700 1 -- 172.21.15.143:0/3997075360 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45dc004280 con 0x7f45e415aa30 2023-11-08T19:29:15.107 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:15.094+0000 7f45eb252700 1 -- 172.21.15.143:0/3997075360 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45e40b6340 con 0x7f45e415aa30 2023-11-08T19:29:15.107 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:15.095+0000 7f45eb252700 1 -- 172.21.15.143:0/3997075360 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45e40c4730 con 0x7f45e415aa30 2023-11-08T19:29:15.107 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:15.096+0000 7f45e17fa700 1 -- 172.21.15.143:0/3997075360 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f45dc0043e0 con 0x7f45e415aa30 2023-11-08T19:29:15.108 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:15.096+0000 7f45e17fa700 1 --2- 172.21.15.143:0/3997075360 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f45c406af30 0x7f45c406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:15.108 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:15.096+0000 7f45e17fa700 1 -- 172.21.15.143:0/3997075360 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6973+0+0 (secure 0 0 0) 0x7f45dc022070 con 0x7f45e415aa30 2023-11-08T19:29:15.108 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:15.096+0000 7f45e8fee700 1 --2- 172.21.15.143:0/3997075360 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f45c406af30 0x7f45c406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:15.108 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:15.097+0000 7f45e8fee700 1 --2- 172.21.15.143:0/3997075360 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f45c406af30 0x7f45c406d3f0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f45d4001730 tx=0x7f45d4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:15.109 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:15.100+0000 7f45e17fa700 1 -- 172.21.15.143:0/3997075360 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f45e40c4730 con 0x7f45e415aa30 2023-11-08T19:29:15.336 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:15.335+0000 7f45eb252700 1 -- 172.21.15.143:0/3997075360 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f45e40c4730 con 0x7f45e415aa30 2023-11-08T19:29:15.337 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:15.336+0000 7f45e17fa700 1 -- 172.21.15.143:0/3997075360 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+4148 (secure 0 0 0) 0x7f45e40c4730 con 0x7f45e415aa30 2023-11-08T19:29:15.338 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:29:15.338 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":68,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5933,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/3918286710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6835","nonce":3918286710},{"type":"v1","addr":"172.21.15.174:6837","nonce":3918286710}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:03.092245+0000","modified":"2023-11-08T19:29:14.946927+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":96,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6906,"mds_1":6918},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6906":{"gid":6906,"name":"c","rank":0,"incarnation":68,"state":"up:replay","state_seq":1,"addr":"172.21.15.143:6834/3172664377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":3172664377},{"type":"v1","addr":"172.21.15.143:6834","nonce":3172664377}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6918":{"gid":6918,"name":"b","rank":1,"incarnation":54,"state":"up:active","state_seq":6,"addr":"172.21.15.174:6836/3601088618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3601088618},{"type":"v1","addr":"172.21.15.174:6836","nonce":3601088618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:29:15.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:15.338+0000 7f45eb252700 1 -- 172.21.15.143:0/3997075360 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f45c406af30 msgr2=0x7f45c406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:15.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:15.338+0000 7f45eb252700 1 --2- 172.21.15.143:0/3997075360 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f45c406af30 0x7f45c406d3f0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f45d4001730 tx=0x7f45d4009040 comp rx=0 tx=0).stop 2023-11-08T19:29:15.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:15.338+0000 7f45eb252700 1 -- 172.21.15.143:0/3997075360 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f45e415aa30 msgr2=0x7f45e4153c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:15.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:15.338+0000 7f45eb252700 1 --2- 172.21.15.143:0/3997075360 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f45e415aa30 0x7f45e4153c40 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f45e4070ae0 tx=0x7f45dc004ad0 comp rx=0 tx=0).stop 2023-11-08T19:29:15.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:15.338+0000 7f45eb252700 1 -- 172.21.15.143:0/3997075360 shutdown_connections 2023-11-08T19:29:15.341 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:15.338+0000 7f45eb252700 1 --2- 172.21.15.143:0/3997075360 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f45c406af30 0x7f45c406d3f0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:15.341 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:15.338+0000 7f45eb252700 1 --2- 172.21.15.143:0/3997075360 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f45e415aa30 0x7f45e4153c40 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:15.341 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:15.338+0000 7f45eb252700 1 --2- 172.21.15.143:0/3997075360 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f45e40b2920 0x7f45e40c6ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:15.341 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:15.338+0000 7f45eb252700 1 --2- 172.21.15.143:0/3997075360 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f45e40cbe20 0x7f45e40b55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:15.341 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:15.338+0000 7f45eb252700 1 -- 172.21.15.143:0/3997075360 >> 172.21.15.143:0/3997075360 conn(0x7f45e4099ab0 msgr2=0x7f45e40cd230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:15.342 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:15.338+0000 7f45eb252700 1 -- 172.21.15.143:0/3997075360 shutdown_connections 2023-11-08T19:29:15.342 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:15.338+0000 7f45eb252700 1 -- 172.21.15.143:0/3997075360 wait complete. 2023-11-08T19:29:15.342 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 68 2023-11-08T19:29:15.351 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=30 retry_count=0)... 2023-11-08T19:29:16.821 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a has been restored 2023-11-08T19:29:19.687 INFO:tasks.ceph.mds.c.smithi143.stderr:2023-11-08T19:29:19.686+0000 7fa9cae70700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-08T19:29:19.687 INFO:tasks.ceph.mds.c.smithi143.stderr:2023-11-08T19:29:19.686+0000 7fa9cae70700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-11-08T19:29:20.353 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:29:20.487 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.485+0000 7f34072c0700 1 Processor -- start 2023-11-08T19:29:20.487 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.485+0000 7f34072c0700 1 -- start start 2023-11-08T19:29:20.489 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.486+0000 7f34072c0700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f340015dd00 0x7f34000cca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:20.490 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.486+0000 7f34072c0700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f340006b320 0x7f34000cb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:20.490 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.486+0000 7f34072c0700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34000ccfb0 con 0x7f34001618f0 2023-11-08T19:29:20.490 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.486+0000 7f34072c0700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34000cad50 con 0x7f340015dd00 2023-11-08T19:29:20.490 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.486+0000 7f34072c0700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3400161df0 con 0x7f340006b320 2023-11-08T19:29:20.490 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.486+0000 7f340505c700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f340015dd00 0x7f34000cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:20.491 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.486+0000 7f340505c700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f340015dd00 0x7f34000cca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55478/0 (socket says 172.21.15.143:55478) 2023-11-08T19:29:20.491 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.486+0000 7f340505c700 1 -- 172.21.15.143:0/413201539 learned_addr learned my addr 172.21.15.143:0/413201539 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:29:20.491 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.487+0000 7f340585d700 1 --2- 172.21.15.143:0/413201539 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f340006b320 0x7f34000cb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:20.491 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.487+0000 7f340505c700 1 -- 172.21.15.143:0/413201539 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f340006b320 msgr2=0x7f34000cb1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:20.491 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.487+0000 7f340505c700 1 --2- 172.21.15.143:0/413201539 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f340006b320 0x7f34000cb1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:20.492 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.487+0000 7f340505c700 1 -- 172.21.15.143:0/413201539 >> v1:172.21.15.174:6789/0 conn(0x7f34001618f0 legacy=0x7f340006ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:20.492 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.487+0000 7f340505c700 1 -- 172.21.15.143:0/413201539 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34001620d0 con 0x7f340015dd00 2023-11-08T19:29:20.492 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.487+0000 7f340505c700 1 --2- 172.21.15.143:0/413201539 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f340015dd00 0x7f34000cca70 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f33f4006fd0 tx=0x7f33f4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5336c8bd16c1d97 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:20.492 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.487+0000 7f33f3fff700 1 -- 172.21.15.143:0/413201539 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f4005e30 con 0x7f340015dd00 2023-11-08T19:29:20.492 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.487+0000 7f33f3fff700 1 -- 172.21.15.143:0/413201539 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33f4004030 con 0x7f340015dd00 2023-11-08T19:29:20.493 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.487+0000 7f33f3fff700 1 -- 172.21.15.143:0/413201539 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f40043e0 con 0x7f340015dd00 2023-11-08T19:29:20.493 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.488+0000 7f34072c0700 1 -- 172.21.15.143:0/413201539 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f340015dd00 msgr2=0x7f34000cca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:20.493 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.488+0000 7f34072c0700 1 --2- 172.21.15.143:0/413201539 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f340015dd00 0x7f34000cca70 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f33f4006fd0 tx=0x7f33f4009bd0 comp rx=0 tx=0).stop 2023-11-08T19:29:20.493 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.488+0000 7f34072c0700 1 -- 172.21.15.143:0/413201539 shutdown_connections 2023-11-08T19:29:20.493 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.488+0000 7f34072c0700 1 --2- 172.21.15.143:0/413201539 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f340006b320 0x7f34000cb1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:20.493 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.488+0000 7f34072c0700 1 --2- 172.21.15.143:0/413201539 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f340015dd00 0x7f34000cca70 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:20.494 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.488+0000 7f34072c0700 1 -- 172.21.15.143:0/413201539 >> 172.21.15.143:0/413201539 conn(0x7f3400099ab0 msgr2=0x7f34000d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:20.494 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.488+0000 7f34072c0700 1 -- 172.21.15.143:0/413201539 shutdown_connections 2023-11-08T19:29:20.494 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.488+0000 7f34072c0700 1 -- 172.21.15.143:0/413201539 wait complete. 2023-11-08T19:29:20.497 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.489+0000 7f34072c0700 1 Processor -- start 2023-11-08T19:29:20.497 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.489+0000 7f34072c0700 1 -- start start 2023-11-08T19:29:20.497 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.490+0000 7f34072c0700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f340006b320 0x7f34000c6c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:20.498 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.490+0000 7f34072c0700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f340015dd00 0x7f3400147320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:20.498 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.490+0000 7f34072c0700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f34001618f0 0x7f3400147860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:20.498 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.490+0000 7f34072c0700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34000b2ad0 con 0x7f340015dd00 2023-11-08T19:29:20.498 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.490+0000 7f34072c0700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34000b2f60 con 0x7f34001618f0 2023-11-08T19:29:20.498 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.490+0000 7f34072c0700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34000b21e0 con 0x7f340006b320 2023-11-08T19:29:20.499 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.490+0000 7f340485b700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f340015dd00 0x7f3400147320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:20.499 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.490+0000 7f340485b700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f340015dd00 0x7f3400147320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55484/0 (socket says 172.21.15.143:55484) 2023-11-08T19:29:20.499 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.490+0000 7f340485b700 1 -- 172.21.15.143:0/2688592125 learned_addr learned my addr 172.21.15.143:0/2688592125 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:29:20.499 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.491+0000 7f340505c700 1 --2- 172.21.15.143:0/2688592125 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f340006b320 0x7f34000c6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:20.499 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.491+0000 7f340585d700 1 --2- 172.21.15.143:0/2688592125 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f34001618f0 0x7f3400147860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:20.500 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.491+0000 7f340485b700 1 -- 172.21.15.143:0/2688592125 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f340006b320 msgr2=0x7f34000c6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:20.500 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.491+0000 7f340485b700 1 --2- 172.21.15.143:0/2688592125 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f340006b320 0x7f34000c6c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:20.500 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.491+0000 7f340485b700 1 -- 172.21.15.143:0/2688592125 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f34001618f0 msgr2=0x7f3400147860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:20.500 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.491+0000 7f340485b700 1 --2- 172.21.15.143:0/2688592125 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f34001618f0 0x7f3400147860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:20.500 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.491+0000 7f340485b700 1 -- 172.21.15.143:0/2688592125 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33f4009500 con 0x7f340015dd00 2023-11-08T19:29:20.501 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.491+0000 7f340585d700 1 --2- 172.21.15.143:0/2688592125 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f34001618f0 0x7f3400147860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:29:20.501 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.491+0000 7f340485b700 1 --2- 172.21.15.143:0/2688592125 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f340015dd00 0x7f3400147320 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f33fc00c430 tx=0x7f33fc00c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:20.501 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.491+0000 7f33f1ffb700 1 -- 172.21.15.143:0/2688592125 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33fc0052d0 con 0x7f340015dd00 2023-11-08T19:29:20.501 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.491+0000 7f33f1ffb700 1 -- 172.21.15.143:0/2688592125 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33fc005450 con 0x7f340015dd00 2023-11-08T19:29:20.501 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.491+0000 7f34072c0700 1 -- 172.21.15.143:0/2688592125 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3400148130 con 0x7f340015dd00 2023-11-08T19:29:20.502 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.491+0000 7f33f1ffb700 1 -- 172.21.15.143:0/2688592125 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33fc006aa0 con 0x7f340015dd00 2023-11-08T19:29:20.502 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.491+0000 7f34072c0700 1 -- 172.21.15.143:0/2688592125 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34000b2760 con 0x7f340015dd00 2023-11-08T19:29:20.502 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.492+0000 7f34072c0700 1 -- 172.21.15.143:0/2688592125 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34000aff60 con 0x7f340015dd00 2023-11-08T19:29:20.502 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.493+0000 7f33f1ffb700 1 -- 172.21.15.143:0/2688592125 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f33fc003680 con 0x7f340015dd00 2023-11-08T19:29:20.502 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.493+0000 7f33f1ffb700 1 --2- 172.21.15.143:0/2688592125 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f33e406af00 0x7f33e406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:20.503 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.493+0000 7f33f1ffb700 1 -- 172.21.15.143:0/2688592125 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6973+0+0 (secure 0 0 0) 0x7f33fc088d10 con 0x7f340015dd00 2023-11-08T19:29:20.503 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.493+0000 7f340505c700 1 --2- 172.21.15.143:0/2688592125 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f33e406af00 0x7f33e406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:20.503 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.494+0000 7f340505c700 1 --2- 172.21.15.143:0/2688592125 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f33e406af00 0x7f33e406d3c0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f33f4009130 tx=0x7f33f4008070 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:20.504 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.497+0000 7f33f1ffb700 1 -- 172.21.15.143:0/2688592125 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f33fc0538d0 con 0x7f340015dd00 2023-11-08T19:29:20.729 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.728+0000 7f34072c0700 1 -- 172.21.15.143:0/2688592125 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f340004eee0 con 0x7f340015dd00 2023-11-08T19:29:20.730 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.728+0000 7f33f1ffb700 1 -- 172.21.15.143:0/2688592125 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f33fc088020 con 0x7f340015dd00 2023-11-08T19:29:20.730 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:29:20.730 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":72,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5933,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/3918286710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6835","nonce":3918286710},{"type":"v1","addr":"172.21.15.174:6837","nonce":3918286710}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":7386,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3664584522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":3664584522},{"type":"v1","addr":"172.21.15.143:6835","nonce":3664584522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:03.092245+0000","modified":"2023-11-08T19:29:17.961359+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":96,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6906,"mds_1":6918},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6906":{"gid":6906,"name":"c","rank":0,"incarnation":68,"state":"up:active","state_seq":24,"addr":"172.21.15.143:6834/3172664377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":3172664377},{"type":"v1","addr":"172.21.15.143:6834","nonce":3172664377}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6918":{"gid":6918,"name":"b","rank":1,"incarnation":54,"state":"up:active","state_seq":6,"addr":"172.21.15.174:6836/3601088618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3601088618},{"type":"v1","addr":"172.21.15.174:6836","nonce":3601088618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:29:20.732 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.730+0000 7f34072c0700 1 -- 172.21.15.143:0/2688592125 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f33e406af00 msgr2=0x7f33e406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:20.732 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.730+0000 7f34072c0700 1 --2- 172.21.15.143:0/2688592125 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f33e406af00 0x7f33e406d3c0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f33f4009130 tx=0x7f33f4008070 comp rx=0 tx=0).stop 2023-11-08T19:29:20.732 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.730+0000 7f34072c0700 1 -- 172.21.15.143:0/2688592125 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f340015dd00 msgr2=0x7f3400147320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:20.733 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.730+0000 7f34072c0700 1 --2- 172.21.15.143:0/2688592125 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f340015dd00 0x7f3400147320 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f33fc00c430 tx=0x7f33fc00c7f0 comp rx=0 tx=0).stop 2023-11-08T19:29:20.733 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.731+0000 7f34072c0700 1 -- 172.21.15.143:0/2688592125 shutdown_connections 2023-11-08T19:29:20.733 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.731+0000 7f34072c0700 1 --2- 172.21.15.143:0/2688592125 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f33e406af00 0x7f33e406d3c0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:20.733 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.731+0000 7f34072c0700 1 --2- 172.21.15.143:0/2688592125 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f340006b320 0x7f34000c6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:20.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.731+0000 7f34072c0700 1 --2- 172.21.15.143:0/2688592125 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f34001618f0 0x7f3400147860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:20.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.731+0000 7f34072c0700 1 --2- 172.21.15.143:0/2688592125 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f340015dd00 0x7f3400147320 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:20.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.731+0000 7f34072c0700 1 -- 172.21.15.143:0/2688592125 >> 172.21.15.143:0/2688592125 conn(0x7f3400099ab0 msgr2=0x7f34000cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:20.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.731+0000 7f34072c0700 1 -- 172.21.15.143:0/2688592125 shutdown_connections 2023-11-08T19:29:20.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.731+0000 7f34072c0700 1 -- 172.21.15.143:0/2688592125 wait complete. 2023-11-08T19:29:20.735 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 72 2023-11-08T19:29:20.744 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2023-11-08T19:29:20.745 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:29:20.864 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.863+0000 7f06261b4700 1 Processor -- start 2023-11-08T19:29:20.865 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.863+0000 7f06261b4700 1 -- start start 2023-11-08T19:29:20.867 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.864+0000 7f06261b4700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f062006b320 0x7f0620168c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:20.867 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.864+0000 7f06261b4700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0620161df0 con 0x7f06201618f0 2023-11-08T19:29:20.867 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.864+0000 7f06261b4700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06201620d0 con 0x7f062015dd00 2023-11-08T19:29:20.867 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.864+0000 7f06261b4700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06200cad50 con 0x7f062006b320 2023-11-08T19:29:20.868 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.864+0000 7f061effd700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f06201618f0 0x7f062006ac10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:51648/0 (socket says 172.21.15.143:51648) 2023-11-08T19:29:20.868 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.864+0000 7f061effd700 1 -- 172.21.15.143:0/1905385102 learned_addr learned my addr 172.21.15.143:0/1905385102 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:29:20.868 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.864+0000 7f061ffff700 1 --2- 172.21.15.143:0/1905385102 >> v2:172.21.15.143:3300/0 conn(0x7f062006b320 0x7f0620168c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:20.868 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.864+0000 7f061e7fc700 1 -- 172.21.15.143:0/1905385102 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3720989244 0 0) 0x7f0620161df0 con 0x7f06201618f0 2023-11-08T19:29:20.868 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.864+0000 7f061e7fc700 1 -- 172.21.15.143:0/1905385102 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f06080035b0 con 0x7f06201618f0 2023-11-08T19:29:20.869 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.864+0000 7f061ffff700 1 -- 172.21.15.143:0/1905385102 >> v1:172.21.15.174:6789/0 conn(0x7f062015dd00 legacy=0x7f06200cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:20.869 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.864+0000 7f061ffff700 1 -- 172.21.15.143:0/1905385102 >> v1:172.21.15.143:6789/0 conn(0x7f06201618f0 legacy=0x7f062006ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:20.869 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.864+0000 7f061ffff700 1 -- 172.21.15.143:0/1905385102 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06201623b0 con 0x7f062006b320 2023-11-08T19:29:20.869 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.864+0000 7f061ffff700 1 --2- 172.21.15.143:0/1905385102 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f062006b320 0x7f0620168c60 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f0618009d90 tx=0x7f061800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f24e10b2fd20376c server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:20.869 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.865+0000 7f061e7fc700 1 -- 172.21.15.143:0/1905385102 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f061800e070 con 0x7f062006b320 2023-11-08T19:29:20.870 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.865+0000 7f061e7fc700 1 -- 172.21.15.143:0/1905385102 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f061800bae0 con 0x7f062006b320 2023-11-08T19:29:20.870 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.865+0000 7f061e7fc700 1 -- 172.21.15.143:0/1905385102 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f061800be90 con 0x7f062006b320 2023-11-08T19:29:20.870 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.865+0000 7f06261b4700 1 -- 172.21.15.143:0/1905385102 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f062006b320 msgr2=0x7f0620168c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:20.870 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.865+0000 7f06261b4700 1 --2- 172.21.15.143:0/1905385102 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f062006b320 0x7f0620168c60 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f0618009d90 tx=0x7f061800b330 comp rx=0 tx=0).stop 2023-11-08T19:29:20.871 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.865+0000 7f06261b4700 1 -- 172.21.15.143:0/1905385102 shutdown_connections 2023-11-08T19:29:20.871 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.865+0000 7f06261b4700 1 --2- 172.21.15.143:0/1905385102 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f062006b320 0x7f0620168c60 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:20.871 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.865+0000 7f06261b4700 1 -- 172.21.15.143:0/1905385102 >> 172.21.15.143:0/1905385102 conn(0x7f0620099ab0 msgr2=0x7f06200d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:20.871 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.865+0000 7f06261b4700 1 -- 172.21.15.143:0/1905385102 shutdown_connections 2023-11-08T19:29:20.871 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.865+0000 7f06261b4700 1 -- 172.21.15.143:0/1905385102 wait complete. 2023-11-08T19:29:20.874 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.866+0000 7f06261b4700 1 Processor -- start 2023-11-08T19:29:20.874 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.867+0000 7f06261b4700 1 -- start start 2023-11-08T19:29:20.874 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.867+0000 7f06261b4700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f062006b320 0x7f06201f2df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:20.875 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.867+0000 7f06261b4700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f062015dd00 0x7f0620160190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:20.875 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.867+0000 7f06261b4700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f06201618f0 0x7f06200af140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:20.875 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.867+0000 7f06261b4700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06200c6c60 con 0x7f062006b320 2023-11-08T19:29:20.875 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.867+0000 7f06261b4700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06200c6050 con 0x7f062015dd00 2023-11-08T19:29:20.875 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.867+0000 7f06261b4700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06200c6330 con 0x7f06201618f0 2023-11-08T19:29:20.876 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.867+0000 7f061f7fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f062006b320 0x7f06201f2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:20.876 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.867+0000 7f061f7fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f062006b320 0x7f06201f2df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55500/0 (socket says 172.21.15.143:55500) 2023-11-08T19:29:20.876 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.867+0000 7f061f7fe700 1 -- 172.21.15.143:0/2366265739 learned_addr learned my addr 172.21.15.143:0/2366265739 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:29:20.876 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.867+0000 7f061effd700 1 --2- 172.21.15.143:0/2366265739 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f062015dd00 0x7f0620160190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:20.876 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.868+0000 7f061ffff700 1 --2- 172.21.15.143:0/2366265739 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f06201618f0 0x7f06200af140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:20.877 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.868+0000 7f061f7fe700 1 -- 172.21.15.143:0/2366265739 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f06201618f0 msgr2=0x7f06200af140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:20.877 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.868+0000 7f061f7fe700 1 --2- 172.21.15.143:0/2366265739 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f06201618f0 0x7f06200af140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:20.877 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.868+0000 7f061f7fe700 1 -- 172.21.15.143:0/2366265739 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f062015dd00 msgr2=0x7f0620160190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:20.877 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.868+0000 7f061f7fe700 1 --2- 172.21.15.143:0/2366265739 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f062015dd00 0x7f0620160190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:20.877 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.868+0000 7f061f7fe700 1 -- 172.21.15.143:0/2366265739 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0618009a70 con 0x7f062006b320 2023-11-08T19:29:20.878 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.868+0000 7f061ffff700 1 --2- 172.21.15.143:0/2366265739 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f06201618f0 0x7f06200af140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:29:20.878 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.868+0000 7f061f7fe700 1 --2- 172.21.15.143:0/2366265739 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f062006b320 0x7f06201f2df0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f061400c4b0 tx=0x7f061400c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:20.878 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.868+0000 7f0607fff700 1 -- 172.21.15.143:0/2366265739 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0614011790 con 0x7f062006b320 2023-11-08T19:29:20.878 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.868+0000 7f0607fff700 1 -- 172.21.15.143:0/2366265739 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06140118f0 con 0x7f062006b320 2023-11-08T19:29:20.878 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.868+0000 7f06261b4700 1 -- 172.21.15.143:0/2366265739 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f062015e260 con 0x7f062006b320 2023-11-08T19:29:20.879 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.868+0000 7f0607fff700 1 -- 172.21.15.143:0/2366265739 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0614011ca0 con 0x7f062006b320 2023-11-08T19:29:20.879 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.868+0000 7f06261b4700 1 -- 172.21.15.143:0/2366265739 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06200c68f0 con 0x7f062006b320 2023-11-08T19:29:20.879 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.869+0000 7f06261b4700 1 -- 172.21.15.143:0/2366265739 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06200ceec0 con 0x7f062006b320 2023-11-08T19:29:20.879 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.869+0000 7f0607fff700 1 -- 172.21.15.143:0/2366265739 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f061401c460 con 0x7f062006b320 2023-11-08T19:29:20.879 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.870+0000 7f0607fff700 1 --2- 172.21.15.143:0/2366265739 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f060c06af00 0x7f060c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:20.880 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.870+0000 7f0607fff700 1 -- 172.21.15.143:0/2366265739 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6973+0+0 (secure 0 0 0) 0x7f0614089890 con 0x7f062006b320 2023-11-08T19:29:20.880 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.870+0000 7f061effd700 1 --2- 172.21.15.143:0/2366265739 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f060c06af00 0x7f060c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:20.880 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.870+0000 7f061effd700 1 --2- 172.21.15.143:0/2366265739 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f060c06af00 0x7f060c06d3c0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f06200b6650 tx=0x7f0610009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:20.880 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:20.874+0000 7f0607fff700 1 -- 172.21.15.143:0/2366265739 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f06140544e0 con 0x7f062006b320 2023-11-08T19:29:21.106 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.105+0000 7f06261b4700 1 -- 172.21.15.143:0/2366265739 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f062004eee0 con 0x7f062006b320 2023-11-08T19:29:21.107 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.106+0000 7f0607fff700 1 -- 172.21.15.143:0/2366265739 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f0614057b00 con 0x7f062006b320 2023-11-08T19:29:21.107 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:29:21.108 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":72,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5933,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/3918286710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6835","nonce":3918286710},{"type":"v1","addr":"172.21.15.174:6837","nonce":3918286710}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":7386,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3664584522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":3664584522},{"type":"v1","addr":"172.21.15.143:6835","nonce":3664584522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:03.092245+0000","modified":"2023-11-08T19:29:17.961359+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":96,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6906,"mds_1":6918},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6906":{"gid":6906,"name":"c","rank":0,"incarnation":68,"state":"up:active","state_seq":24,"addr":"172.21.15.143:6834/3172664377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":3172664377},{"type":"v1","addr":"172.21.15.143:6834","nonce":3172664377}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6918":{"gid":6918,"name":"b","rank":1,"incarnation":54,"state":"up:active","state_seq":6,"addr":"172.21.15.174:6836/3601088618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3601088618},{"type":"v1","addr":"172.21.15.174:6836","nonce":3601088618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:29:21.109 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.108+0000 7f06261b4700 1 -- 172.21.15.143:0/2366265739 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f060c06af00 msgr2=0x7f060c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:21.109 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.108+0000 7f06261b4700 1 --2- 172.21.15.143:0/2366265739 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f060c06af00 0x7f060c06d3c0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f06200b6650 tx=0x7f0610009040 comp rx=0 tx=0).stop 2023-11-08T19:29:21.110 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.108+0000 7f06261b4700 1 -- 172.21.15.143:0/2366265739 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f062006b320 msgr2=0x7f06201f2df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:21.110 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.108+0000 7f06261b4700 1 --2- 172.21.15.143:0/2366265739 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f062006b320 0x7f06201f2df0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f061400c4b0 tx=0x7f061400c870 comp rx=0 tx=0).stop 2023-11-08T19:29:21.110 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.108+0000 7f06261b4700 1 -- 172.21.15.143:0/2366265739 shutdown_connections 2023-11-08T19:29:21.110 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.108+0000 7f06261b4700 1 --2- 172.21.15.143:0/2366265739 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f060c06af00 0x7f060c06d3c0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:21.110 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.108+0000 7f06261b4700 1 --2- 172.21.15.143:0/2366265739 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f06201618f0 0x7f06200af140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:21.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.108+0000 7f06261b4700 1 --2- 172.21.15.143:0/2366265739 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f062006b320 0x7f06201f2df0 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:21.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.108+0000 7f06261b4700 1 --2- 172.21.15.143:0/2366265739 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f062015dd00 0x7f0620160190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:21.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.108+0000 7f06261b4700 1 -- 172.21.15.143:0/2366265739 >> 172.21.15.143:0/2366265739 conn(0x7f0620099ab0 msgr2=0x7f062006e9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:21.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.108+0000 7f06261b4700 1 -- 172.21.15.143:0/2366265739 shutdown_connections 2023-11-08T19:29:21.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.108+0000 7f06261b4700 1 -- 172.21.15.143:0/2366265739 wait complete. 2023-11-08T19:29:21.112 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 72 2023-11-08T19:29:21.523 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 72, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 2, 'explicitly_allowed_features': 2, 'created': '2023-11-08T19:28:03.092245+0000', 'modified': '2023-11-08T19:29:17.961359+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 96, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 2, 'in': [0, 1], 'up': {'mds_0': 6906, 'mds_1': 6918}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6906': {'gid': 6906, 'name': 'c', 'rank': 0, 'incarnation': 68, 'state': 'up:active', 'state_seq': 24, 'addr': '172.21.15.143:6834/3172664377', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.143:6832', 'nonce': 3172664377}, {'type': 'v1', 'addr': '172.21.15.143:6834', 'nonce': 3172664377}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_6918': {'gid': 6918, 'name': 'b', 'rank': 1, 'incarnation': 54, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.174:6836/3601088618', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.174:6834', 'nonce': 3601088618}, {'type': 'v1', 'addr': '172.21.15.174:6836', 'nonce': 3601088618}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-08T19:29:21.524 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 2/2 2023-11-08T19:29:21.524 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-11-08T19:29:21.647 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.646+0000 7f30957d7700 1 Processor -- start 2023-11-08T19:29:21.648 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.646+0000 7f30957d7700 1 -- start start 2023-11-08T19:29:21.650 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.647+0000 7f30957d7700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f309015dc80 0x7f30900d1f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:21.651 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.647+0000 7f30957d7700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f30900d3690 0x7f30900b9690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:21.651 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.647+0000 7f30957d7700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f30900ce980 0x7f30900cee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:21.651 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.647+0000 7f30957d7700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30900cf350 con 0x7f30900d3690 2023-11-08T19:29:21.651 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.647+0000 7f30957d7700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30900b9bd0 con 0x7f309015dc80 2023-11-08T19:29:21.652 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.647+0000 7f30957d7700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30900d3200 con 0x7f30900ce980 2023-11-08T19:29:21.652 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.647+0000 7f308e7fc700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f30900d3690 0x7f30900b9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:21.652 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.647+0000 7f308e7fc700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f30900d3690 0x7f30900b9690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55506/0 (socket says 172.21.15.143:55506) 2023-11-08T19:29:21.653 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.647+0000 7f308e7fc700 1 -- 172.21.15.143:0/476674953 learned_addr learned my addr 172.21.15.143:0/476674953 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:29:21.653 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.647+0000 7f308f7fe700 1 --2- 172.21.15.143:0/476674953 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f30900ce980 0x7f30900cee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:21.653 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.647+0000 7f308effd700 1 --2- 172.21.15.143:0/476674953 >> v2:172.21.15.174:3300/0 conn(0x7f309015dc80 0x7f30900d1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:21.653 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.648+0000 7f308e7fc700 1 -- 172.21.15.143:0/476674953 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f30900ce980 msgr2=0x7f30900cee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:21.653 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.648+0000 7f308e7fc700 1 --2- 172.21.15.143:0/476674953 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f30900ce980 0x7f30900cee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:21.654 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.648+0000 7f308e7fc700 1 -- 172.21.15.143:0/476674953 >> v2:172.21.15.174:3300/0 conn(0x7f309015dc80 msgr2=0x7f30900d1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:21.654 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.648+0000 7f308e7fc700 1 --2- 172.21.15.143:0/476674953 >> v2:172.21.15.174:3300/0 conn(0x7f309015dc80 0x7f30900d1f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:21.654 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.648+0000 7f308e7fc700 1 -- 172.21.15.143:0/476674953 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f309015e060 con 0x7f30900d3690 2023-11-08T19:29:21.654 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.648+0000 7f308e7fc700 1 --2- 172.21.15.143:0/476674953 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f30900d3690 0x7f30900b9690 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f30900b6710 tx=0x7f3080009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2a45aed3e0d9c2e server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:21.654 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.648+0000 7f308dffb700 1 -- 172.21.15.143:0/476674953 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f308000b860 con 0x7f30900d3690 2023-11-08T19:29:21.655 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.648+0000 7f308dffb700 1 -- 172.21.15.143:0/476674953 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f308000b9c0 con 0x7f30900d3690 2023-11-08T19:29:21.655 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.648+0000 7f308dffb700 1 -- 172.21.15.143:0/476674953 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f308000bd50 con 0x7f30900d3690 2023-11-08T19:29:21.655 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.648+0000 7f30957d7700 1 -- 172.21.15.143:0/476674953 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f30900d3690 msgr2=0x7f30900b9690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:21.655 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.648+0000 7f30957d7700 1 --2- 172.21.15.143:0/476674953 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f30900d3690 0x7f30900b9690 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f30900b6710 tx=0x7f3080009c00 comp rx=0 tx=0).stop 2023-11-08T19:29:21.655 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.648+0000 7f30957d7700 1 -- 172.21.15.143:0/476674953 shutdown_connections 2023-11-08T19:29:21.656 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.648+0000 7f30957d7700 1 --2- 172.21.15.143:0/476674953 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f30900ce980 0x7f30900cee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:21.656 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.648+0000 7f30957d7700 1 --2- 172.21.15.143:0/476674953 >> v2:172.21.15.174:3300/0 conn(0x7f309015dc80 0x7f30900d1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:21.656 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.648+0000 7f30957d7700 1 --2- 172.21.15.143:0/476674953 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f30900d3690 0x7f30900b9690 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:21.656 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.648+0000 7f30957d7700 1 -- 172.21.15.143:0/476674953 >> 172.21.15.143:0/476674953 conn(0x7f30900668e0 msgr2=0x7f3090066cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:21.656 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.648+0000 7f30957d7700 1 -- 172.21.15.143:0/476674953 shutdown_connections 2023-11-08T19:29:21.656 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.648+0000 7f30957d7700 1 -- 172.21.15.143:0/476674953 wait complete. 2023-11-08T19:29:21.665 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.650+0000 7f30957d7700 1 Processor -- start 2023-11-08T19:29:21.666 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.650+0000 7f30957d7700 1 -- start start 2023-11-08T19:29:21.666 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.651+0000 7f30957d7700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f30900ce980 0x7f30901f2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:21.666 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.651+0000 7f30957d7700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f30900d3690 0x7f30900cd140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:21.666 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.651+0000 7f30957d7700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f309015dc80 0x7f309015bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:21.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.651+0000 7f30957d7700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30901f3400 con 0x7f30900d3690 2023-11-08T19:29:21.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.651+0000 7f30957d7700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30900cb110 con 0x7f309015dc80 2023-11-08T19:29:21.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.651+0000 7f30957d7700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30900cb3f0 con 0x7f30900ce980 2023-11-08T19:29:21.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.651+0000 7f308e7fc700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f30900d3690 0x7f30900cd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:21.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.651+0000 7f308e7fc700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f30900d3690 0x7f30900cd140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55512/0 (socket says 172.21.15.143:55512) 2023-11-08T19:29:21.668 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.651+0000 7f308e7fc700 1 -- 172.21.15.143:0/4150348622 learned_addr learned my addr 172.21.15.143:0/4150348622 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:29:21.668 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.651+0000 7f308effd700 1 --2- 172.21.15.143:0/4150348622 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f30900ce980 0x7f30901f2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:21.668 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.651+0000 7f308f7fe700 1 --2- 172.21.15.143:0/4150348622 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f309015dc80 0x7f309015bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:21.668 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.651+0000 7f308e7fc700 1 -- 172.21.15.143:0/4150348622 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f30900ce980 msgr2=0x7f30901f2ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:21.668 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.651+0000 7f308e7fc700 1 --2- 172.21.15.143:0/4150348622 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f30900ce980 0x7f30901f2ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:21.669 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.651+0000 7f308e7fc700 1 -- 172.21.15.143:0/4150348622 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f309015dc80 msgr2=0x7f309015bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:21.669 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.651+0000 7f308e7fc700 1 --2- 172.21.15.143:0/4150348622 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f309015dc80 0x7f309015bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:21.669 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.651+0000 7f308e7fc700 1 -- 172.21.15.143:0/4150348622 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3080009550 con 0x7f30900d3690 2023-11-08T19:29:21.669 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.651+0000 7f308effd700 1 --2- 172.21.15.143:0/4150348622 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f30900ce980 0x7f30901f2ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:29:21.669 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.651+0000 7f308f7fe700 1 --2- 172.21.15.143:0/4150348622 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f309015dc80 0x7f309015bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:29:21.669 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.651+0000 7f308e7fc700 1 --2- 172.21.15.143:0/4150348622 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f30900d3690 0x7f30900cd140 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f3080009c40 tx=0x7f3080004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:21.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.651+0000 7f30777fe700 1 -- 172.21.15.143:0/4150348622 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3080004380 con 0x7f30900d3690 2023-11-08T19:29:21.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.651+0000 7f30777fe700 1 -- 172.21.15.143:0/4150348622 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f308001f070 con 0x7f30900d3690 2023-11-08T19:29:21.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.651+0000 7f30957d7700 1 -- 172.21.15.143:0/4150348622 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30900cb6d0 con 0x7f30900d3690 2023-11-08T19:29:21.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.651+0000 7f30777fe700 1 -- 172.21.15.143:0/4150348622 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30800036a0 con 0x7f30900d3690 2023-11-08T19:29:21.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.651+0000 7f30957d7700 1 -- 172.21.15.143:0/4150348622 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30900cd680 con 0x7f30900d3690 2023-11-08T19:29:21.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.653+0000 7f30777fe700 1 -- 172.21.15.143:0/4150348622 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3080021030 con 0x7f30900d3690 2023-11-08T19:29:21.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.653+0000 7f30777fe700 1 --2- 172.21.15.143:0/4150348622 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f307c06af00 0x7f307c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:21.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.653+0000 7f30777fe700 1 -- 172.21.15.143:0/4150348622 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6973+0+0 (secure 0 0 0) 0x7f308008af60 con 0x7f30900d3690 2023-11-08T19:29:21.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.653+0000 7f308effd700 1 --2- 172.21.15.143:0/4150348622 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f307c06af00 0x7f307c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:21.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.654+0000 7f308effd700 1 --2- 172.21.15.143:0/4150348622 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f307c06af00 0x7f307c06d3c0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f3084009a00 tx=0x7f3084008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:21.672 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.655+0000 7f30a19d0b80 1 Processor -- start 2023-11-08T19:29:21.672 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.655+0000 7f30a19d0b80 1 -- start start 2023-11-08T19:29:21.672 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.656+0000 7f30a19d0b80 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x5574d0d486f0 0x5574d0d78450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:21.672 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.656+0000 7f30a19d0b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5574d0d4ac30 0x5574d0d78990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:21.672 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.656+0000 7f30a19d0b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5574d0d48ad0 0x5574d0d62f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:21.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.656+0000 7f30a19d0b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574d0cb46e0 con 0x5574d0d48ad0 2023-11-08T19:29:21.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.656+0000 7f30a19d0b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574d0ca83b0 con 0x5574d0d4ac30 2023-11-08T19:29:21.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.656+0000 7f30a19d0b80 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x5574d0ca5cd0 con 0x5574d0d486f0 2023-11-08T19:29:21.674 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.656+0000 7f308f7fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5574d0d48ad0 0x5574d0d62f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:21.674 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.656+0000 7f308f7fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5574d0d48ad0 0x5574d0d62f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55522/0 (socket says 172.21.15.143:55522) 2023-11-08T19:29:21.674 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.656+0000 7f308f7fe700 1 -- 172.21.15.143:0/4125402665 learned_addr learned my addr 172.21.15.143:0/4125402665 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:29:21.674 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.656+0000 7f308effd700 1 --2- 172.21.15.143:0/4125402665 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x5574d0d486f0 0x5574d0d78450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:21.674 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.656+0000 7f308e7fc700 1 --2- 172.21.15.143:0/4125402665 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5574d0d4ac30 0x5574d0d78990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:21.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.656+0000 7f308f7fe700 1 -- 172.21.15.143:0/4125402665 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x5574d0d486f0 msgr2=0x5574d0d78450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:21.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.656+0000 7f308f7fe700 1 --2- 172.21.15.143:0/4125402665 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x5574d0d486f0 0x5574d0d78450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:21.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.656+0000 7f308f7fe700 1 -- 172.21.15.143:0/4125402665 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5574d0d4ac30 msgr2=0x5574d0d78990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:21.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.656+0000 7f308f7fe700 1 --2- 172.21.15.143:0/4125402665 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5574d0d4ac30 0x5574d0d78990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:21.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.656+0000 7f308f7fe700 1 -- 172.21.15.143:0/4125402665 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5574d0b1e5e0 con 0x5574d0d48ad0 2023-11-08T19:29:21.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.656+0000 7f308e7fc700 1 --2- 172.21.15.143:0/4125402665 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5574d0d4ac30 0x5574d0d78990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:29:21.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.657+0000 7f308f7fe700 1 --2- 172.21.15.143:0/4125402665 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5574d0d48ad0 0x5574d0d62f90 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f3088009f60 tx=0x7f3088009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbc8a3cd73d44fb6 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:21.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.657+0000 7f3074ff9700 1 -- 172.21.15.143:0/4125402665 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3088003c20 con 0x5574d0d48ad0 2023-11-08T19:29:21.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.657+0000 7f3074ff9700 1 -- 172.21.15.143:0/4125402665 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f308800f050 con 0x5574d0d48ad0 2023-11-08T19:29:21.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.657+0000 7f3074ff9700 1 -- 172.21.15.143:0/4125402665 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3088004030 con 0x5574d0d48ad0 2023-11-08T19:29:21.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.657+0000 7f30a19d0b80 1 -- 172.21.15.143:0/4125402665 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5574d0d48ad0 msgr2=0x5574d0d62f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:21.677 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.657+0000 7f30a19d0b80 1 --2- 172.21.15.143:0/4125402665 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5574d0d48ad0 0x5574d0d62f90 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f3088009f60 tx=0x7f3088009f90 comp rx=0 tx=0).stop 2023-11-08T19:29:21.677 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.657+0000 7f30a19d0b80 1 -- 172.21.15.143:0/4125402665 shutdown_connections 2023-11-08T19:29:21.677 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.657+0000 7f30a19d0b80 1 --2- 172.21.15.143:0/4125402665 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x5574d0d486f0 0x5574d0d78450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:21.677 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.657+0000 7f30a19d0b80 1 --2- 172.21.15.143:0/4125402665 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5574d0d48ad0 0x5574d0d62f90 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:21.677 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.657+0000 7f30a19d0b80 1 --2- 172.21.15.143:0/4125402665 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5574d0d4ac30 0x5574d0d78990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:21.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.657+0000 7f30a19d0b80 1 -- 172.21.15.143:0/4125402665 >> 172.21.15.143:0/4125402665 conn(0x5574d0cc3660 msgr2=0x5574d0d4e3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:21.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.658+0000 7f30a19d0b80 1 -- 172.21.15.143:0/4125402665 shutdown_connections 2023-11-08T19:29:21.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.658+0000 7f30a19d0b80 1 -- 172.21.15.143:0/4125402665 wait complete. 2023-11-08T19:29:21.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.658+0000 7f30a19d0b80 1 -- start start 2023-11-08T19:29:21.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.658+0000 7f30a19d0b80 1 Processor -- start 2023-11-08T19:29:21.679 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.659+0000 7f30a19d0b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5574d0d486f0 0x5574d0d62270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:21.679 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.659+0000 7f30a19d0b80 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x5574d0d48ad0 0x5574d0d7b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:21.679 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.660+0000 7f30a19d0b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5574d0d7bc30 0x5574d0d8b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:21.679 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.660+0000 7f30a19d0b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574d0ca15c0 con 0x5574d0d486f0 2023-11-08T19:29:21.679 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.660+0000 7f308effd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5574d0d486f0 0x5574d0d62270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:21.680 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.660+0000 7f308effd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5574d0d486f0 0x5574d0d62270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55524/0 (socket says 172.21.15.143:55524) 2023-11-08T19:29:21.680 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.660+0000 7f308effd700 1 -- 172.21.15.143:0/1727242279 learned_addr learned my addr 172.21.15.143:0/1727242279 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:29:21.680 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.660+0000 7f30a19d0b80 1 -- 172.21.15.143:0/1727242279 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574d0c3d0d0 con 0x5574d0d7bc30 2023-11-08T19:29:21.680 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.660+0000 7f30a19d0b80 1 -- 172.21.15.143:0/1727242279 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x5574d0c3cd70 con 0x5574d0d48ad0 2023-11-08T19:29:21.680 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.660+0000 7f30a19d0b80 10 client.0 _ll_register_callbacks cb 0x5574d0a997b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T19:29:21.680 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.660+0000 7f308e7fc700 1 --2- 172.21.15.143:0/1727242279 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x5574d0d48ad0 0x5574d0d7b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:21.681 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.660+0000 7f308f7fe700 1 --2- 172.21.15.143:0/1727242279 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5574d0d7bc30 0x5574d0d8b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:21.681 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.660+0000 7f308effd700 1 -- 172.21.15.143:0/1727242279 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x5574d0d48ad0 msgr2=0x5574d0d7b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:21.681 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.660+0000 7f308effd700 1 --2- 172.21.15.143:0/1727242279 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x5574d0d48ad0 0x5574d0d7b6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:21.681 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.660+0000 7f308effd700 1 -- 172.21.15.143:0/1727242279 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5574d0d7bc30 msgr2=0x5574d0d8b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:21.681 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.660+0000 7f308effd700 1 --2- 172.21.15.143:0/1727242279 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5574d0d7bc30 0x5574d0d8b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:21.682 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.660+0000 7f308effd700 1 -- 172.21.15.143:0/1727242279 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3088009b60 con 0x5574d0d486f0 2023-11-08T19:29:21.682 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.660+0000 7f308effd700 1 --2- 172.21.15.143:0/1727242279 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5574d0d486f0 0x5574d0d62270 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f308400ffe0 tx=0x7f30840047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b3e9b134975e6dd server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:21.682 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.660+0000 7f306dffb700 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2023-11-08T19:29:21.682 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.660+0000 7f306dffb700 1 -- 172.21.15.143:0/1727242279 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3084004020 con 0x5574d0d486f0 2023-11-08T19:29:21.682 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.660+0000 7f306dffb700 1 -- 172.21.15.143:0/1727242279 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 6973+0+0 (secure 0 0 0) 0x7f3084004180 con 0x5574d0d486f0 2023-11-08T19:29:21.683 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.660+0000 7f30a19d0b80 1 -- 172.21.15.143:0/1727242279 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5574d0d40880 con 0x5574d0d486f0 2023-11-08T19:29:21.683 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.660+0000 7f306dffb700 1 -- 172.21.15.143:0/1727242279 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3084004340 con 0x5574d0d486f0 2023-11-08T19:29:21.683 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.660+0000 7f30a19d0b80 10 client.7422 fetch_fsmap learned FSMap version 72 2023-11-08T19:29:21.683 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.660+0000 7f30a19d0b80 1 -- 172.21.15.143:0/1727242279 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=2+}) v3 -- 0x5574d0acad90 con 0x5574d0d486f0 2023-11-08T19:29:21.684 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.661+0000 7f306dffb700 1 -- 172.21.15.143:0/1727242279 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5574d0d40880 con 0x5574d0d486f0 2023-11-08T19:29:21.684 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.661+0000 7f306dffb700 1 -- 172.21.15.143:0/1727242279 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 72) v1 ==== 3306+0+0 (secure 0 0 0) 0x7f30840036a0 con 0x5574d0d486f0 2023-11-08T19:29:21.684 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.661+0000 7f30a19d0b80 10 client.7422 fetch_fsmap finished waiting for FSMap version 72 2023-11-08T19:29:21.684 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.661+0000 7f30a19d0b80 10 client.7422 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-08T19:29:21.685 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.661+0000 7f30a19d0b80 20 client.7422 populate_metadata read hostname 'smithi143' 2023-11-08T19:29:21.685 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.661+0000 7f30a19d0b80 1 --2- 172.21.15.143:0/1727242279 >> [v2:172.21.15.143:6832/3172664377,v1:172.21.15.143:6834/3172664377] conn(0x5574d0d8d420 0x5574d0da28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:21.685 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.661+0000 7f30a19d0b80 4 client.7422 mds_command: new command op to 6906 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-08T19:29:21.685 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.661+0000 7f30a19d0b80 1 -- 172.21.15.143:0/1727242279 --> [v2:172.21.15.143:6832/3172664377,v1:172.21.15.143:6834/3172664377] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5574d0d1ca30 con 0x5574d0d8d420 2023-11-08T19:29:21.685 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.661+0000 7f308e7fc700 1 --2- 172.21.15.143:0/1727242279 >> [v2:172.21.15.143:6832/3172664377,v1:172.21.15.143:6834/3172664377] conn(0x5574d0d8d420 0x5574d0da28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:21.686 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.661+0000 7f306dffb700 10 client.7422 ms_handle_connect on v2:172.21.15.143:6832/3172664377 2023-11-08T19:29:21.686 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.661+0000 7f308e7fc700 1 --2- 172.21.15.143:0/1727242279 >> [v2:172.21.15.143:6832/3172664377,v1:172.21.15.143:6834/3172664377] conn(0x5574d0d8d420 0x5574d0da28f0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0acee258fddd059 server_cookie=4a071f470b15493 in_seq=0 out_seq=0 2023-11-08T19:29:21.686 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.662+0000 7f306dffb700 1 -- 172.21.15.143:0/1727242279 <== mds.0 v2:172.21.15.143:6832/3172664377 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5574d0d1ca30 con 0x5574d0d8d420 2023-11-08T19:29:21.686 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.662+0000 7f306dffb700 10 client.7422 handle_command_reply: tid=0 2023-11-08T19:29:21.686 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.662+0000 7f30a19d0b80 1 client.7422 shutdown 2023-11-08T19:29:21.687 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.662+0000 7f30a19d0b80 1 -- 172.21.15.143:0/1727242279 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5574d0d486f0 msgr2=0x5574d0d62270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:21.687 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.662+0000 7f30a19d0b80 1 --2- 172.21.15.143:0/1727242279 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5574d0d486f0 0x5574d0d62270 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f308400ffe0 tx=0x7f30840047c0 comp rx=0 tx=0).stop 2023-11-08T19:29:21.687 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.662+0000 7f30a19d0b80 1 -- 172.21.15.143:0/1727242279 shutdown_connections 2023-11-08T19:29:21.687 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.662+0000 7f30a19d0b80 1 --2- 172.21.15.143:0/1727242279 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5574d0d7bc30 0x5574d0d8b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:21.687 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.662+0000 7f30a19d0b80 1 --2- 172.21.15.143:0/1727242279 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5574d0d486f0 0x5574d0d62270 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:21.688 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.662+0000 7f30a19d0b80 1 --2- 172.21.15.143:0/1727242279 >> [v2:172.21.15.143:6832/3172664377,v1:172.21.15.143:6834/3172664377] conn(0x5574d0d8d420 0x5574d0da28f0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:21.688 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.662+0000 7f30a19d0b80 1 --2- 172.21.15.143:0/1727242279 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x5574d0d48ad0 0x5574d0d7b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:21.688 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.662+0000 7f306dffb700 0 client.7422 ms_handle_reset on v2:172.21.15.143:6832/3172664377 2023-11-08T19:29:21.689 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.662+0000 7f30a19d0b80 1 -- 172.21.15.143:0/1727242279 >> 172.21.15.143:0/1727242279 conn(0x5574d0cc3660 msgr2=0x5574d0d49020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:21.689 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.662+0000 7f30a19d0b80 1 -- 172.21.15.143:0/1727242279 shutdown_connections 2023-11-08T19:29:21.689 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.662+0000 7f30a19d0b80 1 -- 172.21.15.143:0/1727242279 wait complete. 2023-11-08T19:29:21.689 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.662+0000 7f30a19d0b80 20 client.7422 trim_cache size 0 max 16384 2023-11-08T19:29:21.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.680+0000 7f30a19d0b80 1 Processor -- start 2023-11-08T19:29:21.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.680+0000 7f30a19d0b80 1 -- start start 2023-11-08T19:29:21.697 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.681+0000 7f30a19d0b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5574d0d486f0 0x5574d0d632c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:21.697 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.681+0000 7f30a19d0b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5574d0d48ad0 0x5574d0d615b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:21.697 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.681+0000 7f30a19d0b80 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x5574d0d7bc30 0x5574d0d8c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:21.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.681+0000 7f30a19d0b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574d0b208d0 con 0x5574d0d486f0 2023-11-08T19:29:21.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.681+0000 7f30a19d0b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574d0d4aa10 con 0x5574d0d48ad0 2023-11-08T19:29:21.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.681+0000 7f30a19d0b80 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x5574d0d8e9f0 con 0x5574d0d7bc30 2023-11-08T19:29:21.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.681+0000 7f308effd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5574d0d486f0 0x5574d0d632c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:21.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.681+0000 7f308effd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5574d0d486f0 0x5574d0d632c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55532/0 (socket says 172.21.15.143:55532) 2023-11-08T19:29:21.699 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.681+0000 7f308effd700 1 -- 172.21.15.143:0/2526783491 learned_addr learned my addr 172.21.15.143:0/2526783491 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:29:21.699 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.681+0000 7f308effd700 1 -- 172.21.15.143:0/2526783491 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x5574d0d7bc30 msgr2=0x5574d0d8c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:21.699 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.681+0000 7f308effd700 1 --2- 172.21.15.143:0/2526783491 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x5574d0d7bc30 0x5574d0d8c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:21.699 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.681+0000 7f308effd700 1 -- 172.21.15.143:0/2526783491 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5574d0d48ad0 msgr2=0x5574d0d615b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:21.699 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.681+0000 7f308effd700 1 --2- 172.21.15.143:0/2526783491 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5574d0d48ad0 0x5574d0d615b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:21.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.681+0000 7f308effd700 1 -- 172.21.15.143:0/2526783491 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5574d0d49fc0 con 0x5574d0d486f0 2023-11-08T19:29:21.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.682+0000 7f308effd700 1 --2- 172.21.15.143:0/2526783491 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5574d0d486f0 0x5574d0d632c0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f3084004800 tx=0x7f308401b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cffe7969d7b3bc7b server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:21.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.682+0000 7f30757fa700 1 -- 172.21.15.143:0/2526783491 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3084009220 con 0x5574d0d486f0 2023-11-08T19:29:21.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.682+0000 7f30757fa700 1 -- 172.21.15.143:0/2526783491 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30840036a0 con 0x5574d0d486f0 2023-11-08T19:29:21.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.682+0000 7f30757fa700 1 -- 172.21.15.143:0/2526783491 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3084003d60 con 0x5574d0d486f0 2023-11-08T19:29:21.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.682+0000 7f30a19d0b80 1 -- 172.21.15.143:0/2526783491 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5574d0d486f0 msgr2=0x5574d0d632c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:21.701 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.682+0000 7f30a19d0b80 1 --2- 172.21.15.143:0/2526783491 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5574d0d486f0 0x5574d0d632c0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f3084004800 tx=0x7f308401b000 comp rx=0 tx=0).stop 2023-11-08T19:29:21.701 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.682+0000 7f30a19d0b80 1 -- 172.21.15.143:0/2526783491 shutdown_connections 2023-11-08T19:29:21.701 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.682+0000 7f30a19d0b80 1 --2- 172.21.15.143:0/2526783491 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x5574d0d7bc30 0x5574d0d8c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:21.701 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.682+0000 7f30a19d0b80 1 --2- 172.21.15.143:0/2526783491 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5574d0d486f0 0x5574d0d632c0 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:21.701 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.682+0000 7f30a19d0b80 1 --2- 172.21.15.143:0/2526783491 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5574d0d48ad0 0x5574d0d615b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:21.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.682+0000 7f30a19d0b80 1 -- 172.21.15.143:0/2526783491 >> 172.21.15.143:0/2526783491 conn(0x5574d0cc3660 msgr2=0x5574d0d758b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:21.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.682+0000 7f30a19d0b80 1 -- 172.21.15.143:0/2526783491 shutdown_connections 2023-11-08T19:29:21.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.682+0000 7f30a19d0b80 1 -- 172.21.15.143:0/2526783491 wait complete. 2023-11-08T19:29:21.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.683+0000 7f30a19d0b80 1 -- start start 2023-11-08T19:29:21.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.683+0000 7f30a19d0b80 1 Processor -- start 2023-11-08T19:29:21.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.684+0000 7f30a19d0b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5574d0d486f0 0x5574d0dffa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:21.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.684+0000 7f30a19d0b80 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x5574d0d48ad0 0x5574d0d60180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:21.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.684+0000 7f30a19d0b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5574d0d7bc30 0x5574d0d62080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:21.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.684+0000 7f30a19d0b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574d0d44d60 con 0x5574d0d7bc30 2023-11-08T19:29:21.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.684+0000 7f30a19d0b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574d0da4880 con 0x5574d0d486f0 2023-11-08T19:29:21.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.684+0000 7f30a19d0b80 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x5574d0d63950 con 0x5574d0d48ad0 2023-11-08T19:29:21.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.684+0000 7f30a19d0b80 10 client.0 _ll_register_callbacks cb 0x7f308400fc00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T19:29:21.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.684+0000 7f308f7fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5574d0d7bc30 0x5574d0d62080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:21.705 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.684+0000 7f308f7fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5574d0d7bc30 0x5574d0d62080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55542/0 (socket says 172.21.15.143:55542) 2023-11-08T19:29:21.705 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.684+0000 7f308f7fe700 1 -- 172.21.15.143:0/1949223237 learned_addr learned my addr 172.21.15.143:0/1949223237 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:29:21.705 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.684+0000 7f308e7fc700 1 --2- 172.21.15.143:0/1949223237 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x5574d0d48ad0 0x5574d0d60180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:21.705 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.685+0000 7f308effd700 1 --2- 172.21.15.143:0/1949223237 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5574d0d486f0 0x5574d0dffa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:21.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.685+0000 7f308f7fe700 1 -- 172.21.15.143:0/1949223237 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x5574d0d48ad0 msgr2=0x5574d0d60180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:21.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.685+0000 7f308f7fe700 1 --2- 172.21.15.143:0/1949223237 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x5574d0d48ad0 0x5574d0d60180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:21.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.685+0000 7f308f7fe700 1 -- 172.21.15.143:0/1949223237 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5574d0d486f0 msgr2=0x5574d0dffa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:21.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.685+0000 7f308f7fe700 1 --2- 172.21.15.143:0/1949223237 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5574d0d486f0 0x5574d0dffa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:21.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.685+0000 7f308f7fe700 1 -- 172.21.15.143:0/1949223237 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5574d0acad90 con 0x5574d0d7bc30 2023-11-08T19:29:21.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.685+0000 7f308e7fc700 1 --2- 172.21.15.143:0/1949223237 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x5574d0d48ad0 0x5574d0d60180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:29:21.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.685+0000 7f308effd700 1 --2- 172.21.15.143:0/1949223237 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5574d0d486f0 0x5574d0dffa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:29:21.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.685+0000 7f308f7fe700 1 --2- 172.21.15.143:0/1949223237 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5574d0d7bc30 0x5574d0d62080 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f3088000f80 tx=0x7f3088004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa4754d9590da63b server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:21.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.685+0000 7f306dffb700 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2023-11-08T19:29:21.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.685+0000 7f306dffb700 1 -- 172.21.15.143:0/1949223237 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3088004410 con 0x5574d0d7bc30 2023-11-08T19:29:21.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.685+0000 7f306dffb700 1 -- 172.21.15.143:0/1949223237 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 6973+0+0 (secure 0 0 0) 0x7f3088004570 con 0x5574d0d7bc30 2023-11-08T19:29:21.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.685+0000 7f30a19d0b80 1 -- 172.21.15.143:0/1949223237 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f308800f050 con 0x5574d0d7bc30 2023-11-08T19:29:21.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.685+0000 7f306dffb700 1 -- 172.21.15.143:0/1949223237 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3088004730 con 0x5574d0d7bc30 2023-11-08T19:29:21.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.685+0000 7f30a19d0b80 10 client.7428 fetch_fsmap learned FSMap version 72 2023-11-08T19:29:21.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.685+0000 7f30a19d0b80 1 -- 172.21.15.143:0/1949223237 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=2+}) v3 -- 0x5574d0ab3c10 con 0x5574d0d7bc30 2023-11-08T19:29:21.709 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.685+0000 7f306dffb700 1 -- 172.21.15.143:0/1949223237 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f308800f050 con 0x5574d0d7bc30 2023-11-08T19:29:21.709 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.685+0000 7f306dffb700 1 -- 172.21.15.143:0/1949223237 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 72) v1 ==== 3306+0+0 (secure 0 0 0) 0x7f30880283f0 con 0x5574d0d7bc30 2023-11-08T19:29:21.709 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.685+0000 7f30a19d0b80 10 client.7428 fetch_fsmap finished waiting for FSMap version 72 2023-11-08T19:29:21.709 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.685+0000 7f30a19d0b80 10 client.7428 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-08T19:29:21.709 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.685+0000 7f30a19d0b80 20 client.7428 populate_metadata read hostname 'smithi143' 2023-11-08T19:29:21.710 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.685+0000 7f30a19d0b80 1 --2- 172.21.15.143:0/1949223237 >> [v2:172.21.15.143:6832/3172664377,v1:172.21.15.143:6834/3172664377] conn(0x5574d0dac2c0 0x5574d0d625c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:21.710 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.685+0000 7f30a19d0b80 4 client.7428 mds_command: new command op to 6906 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-08T19:29:21.710 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.685+0000 7f30a19d0b80 1 -- 172.21.15.143:0/1949223237 --> [v2:172.21.15.143:6832/3172664377,v1:172.21.15.143:6834/3172664377] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5574d0c9a5a0 con 0x5574d0dac2c0 2023-11-08T19:29:21.710 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.685+0000 7f308effd700 1 --2- 172.21.15.143:0/1949223237 >> [v2:172.21.15.143:6832/3172664377,v1:172.21.15.143:6834/3172664377] conn(0x5574d0dac2c0 0x5574d0d625c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:21.710 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.686+0000 7f306dffb700 10 client.7428 ms_handle_connect on v2:172.21.15.143:6832/3172664377 2023-11-08T19:29:21.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.686+0000 7f308effd700 1 --2- 172.21.15.143:0/1949223237 >> [v2:172.21.15.143:6832/3172664377,v1:172.21.15.143:6834/3172664377] conn(0x5574d0dac2c0 0x5574d0d625c0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13cdd4db0d6bee45 server_cookie=12332dc673382333 in_seq=0 out_seq=0 2023-11-08T19:29:21.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.686+0000 7f306dffb700 1 -- 172.21.15.143:0/1949223237 <== mds.0 v2:172.21.15.143:6832/3172664377 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x5574d0c9a5a0 con 0x5574d0dac2c0 2023-11-08T19:29:21.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.686+0000 7f306dffb700 10 client.7428 handle_command_reply: tid=0 2023-11-08T19:29:21.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.686+0000 7f30a19d0b80 1 client.7428 shutdown 2023-11-08T19:29:21.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.686+0000 7f30a19d0b80 1 -- 172.21.15.143:0/1949223237 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5574d0d7bc30 msgr2=0x5574d0d62080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:21.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.686+0000 7f30a19d0b80 1 --2- 172.21.15.143:0/1949223237 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5574d0d7bc30 0x5574d0d62080 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f3088000f80 tx=0x7f3088004ad0 comp rx=0 tx=0).stop 2023-11-08T19:29:21.712 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.686+0000 7f30a19d0b80 1 -- 172.21.15.143:0/1949223237 shutdown_connections 2023-11-08T19:29:21.712 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.686+0000 7f30a19d0b80 1 --2- 172.21.15.143:0/1949223237 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5574d0d7bc30 0x5574d0d62080 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:21.712 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.686+0000 7f30a19d0b80 1 --2- 172.21.15.143:0/1949223237 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5574d0d486f0 0x5574d0dffa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:21.712 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.686+0000 7f30a19d0b80 1 --2- 172.21.15.143:0/1949223237 >> [v2:172.21.15.143:6832/3172664377,v1:172.21.15.143:6834/3172664377] conn(0x5574d0dac2c0 0x5574d0d625c0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:21.712 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.686+0000 7f30a19d0b80 1 --2- 172.21.15.143:0/1949223237 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x5574d0d48ad0 0x5574d0d60180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:21.713 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.686+0000 7f306dffb700 0 client.7428 ms_handle_reset on v2:172.21.15.143:6832/3172664377 2023-11-08T19:29:21.713 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.686+0000 7f30a19d0b80 1 -- 172.21.15.143:0/1949223237 >> 172.21.15.143:0/1949223237 conn(0x5574d0cc3660 msgr2=0x5574d0d8c240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:21.713 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.686+0000 7f30a19d0b80 1 -- 172.21.15.143:0/1949223237 shutdown_connections 2023-11-08T19:29:21.713 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.686+0000 7f30a19d0b80 1 -- 172.21.15.143:0/1949223237 wait complete. 2023-11-08T19:29:21.713 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.686+0000 7f30a19d0b80 20 client.7428 trim_cache size 0 max 16384 2023-11-08T19:29:21.714 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2023-11-08T19:29:21.714 INFO:teuthology.orchestra.run.smithi143.stdout: "cluster_fsid": "968483af-77fc-4068-ad7c-5c2a1a4613d7", 2023-11-08T19:29:21.714 INFO:teuthology.orchestra.run.smithi143.stdout: "whoami": 0, 2023-11-08T19:29:21.715 INFO:teuthology.orchestra.run.smithi143.stdout: "id": 6906, 2023-11-08T19:29:21.715 INFO:teuthology.orchestra.run.smithi143.stdout: "want_state": "up:active", 2023-11-08T19:29:21.715 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "up:active", 2023-11-08T19:29:21.716 INFO:teuthology.orchestra.run.smithi143.stdout: "fs_name": "cephfs", 2023-11-08T19:29:21.716 INFO:teuthology.orchestra.run.smithi143.stdout: "rank_uptime": 6.7368413599999997, 2023-11-08T19:29:21.716 INFO:teuthology.orchestra.run.smithi143.stdout: "mdsmap_epoch": 72, 2023-11-08T19:29:21.716 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_epoch": 96, 2023-11-08T19:29:21.716 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_epoch_barrier": 96, 2023-11-08T19:29:21.717 INFO:teuthology.orchestra.run.smithi143.stdout: "uptime": 83.897768021999994 2023-11-08T19:29:21.717 INFO:teuthology.orchestra.run.smithi143.stdout:} 2023-11-08T19:29:21.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.687+0000 7f30957d7700 1 -- 172.21.15.143:0/4150348622 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f307c06af00 msgr2=0x7f307c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:21.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.687+0000 7f30957d7700 1 --2- 172.21.15.143:0/4150348622 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f307c06af00 0x7f307c06d3c0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f3084009a00 tx=0x7f3084008040 comp rx=0 tx=0).stop 2023-11-08T19:29:21.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.687+0000 7f30957d7700 1 -- 172.21.15.143:0/4150348622 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f30900d3690 msgr2=0x7f30900cd140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:21.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.687+0000 7f30957d7700 1 --2- 172.21.15.143:0/4150348622 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f30900d3690 0x7f30900cd140 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f3080009c40 tx=0x7f3080004750 comp rx=0 tx=0).stop 2023-11-08T19:29:21.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.687+0000 7f30957d7700 1 -- 172.21.15.143:0/4150348622 shutdown_connections 2023-11-08T19:29:21.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.687+0000 7f30957d7700 1 --2- 172.21.15.143:0/4150348622 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f307c06af00 0x7f307c06d3c0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:21.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.687+0000 7f30957d7700 1 --2- 172.21.15.143:0/4150348622 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f30900ce980 0x7f30901f2ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:21.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.687+0000 7f30957d7700 1 --2- 172.21.15.143:0/4150348622 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f309015dc80 0x7f309015bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:21.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.687+0000 7f30957d7700 1 --2- 172.21.15.143:0/4150348622 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f30900d3690 0x7f30900cd140 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:21.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.687+0000 7f30957d7700 1 -- 172.21.15.143:0/4150348622 >> 172.21.15.143:0/4150348622 conn(0x7f30900668e0 msgr2=0x7f30900b8cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:21.721 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.687+0000 7f30957d7700 1 -- 172.21.15.143:0/4150348622 shutdown_connections 2023-11-08T19:29:21.721 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.687+0000 7f30957d7700 1 -- 172.21.15.143:0/4150348622 wait complete. 2023-11-08T19:29:21.721 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-11-08T19:29:21.842 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.841+0000 7f237950f700 1 Processor -- start 2023-11-08T19:29:21.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.841+0000 7f237950f700 1 -- start start 2023-11-08T19:29:21.846 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.842+0000 7f237950f700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f23740cf090 0x7f23740d1f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:21.847 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.842+0000 7f237950f700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23740d3200 con 0x7f237415dc80 2023-11-08T19:29:21.847 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.842+0000 7f237950f700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f237415e060 con 0x7f23740d3690 2023-11-08T19:29:21.847 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.842+0000 7f237950f700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23740d25f0 con 0x7f23740cf090 2023-11-08T19:29:21.847 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.842+0000 7f2372ffd700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f237415dc80 0x7f23740ce980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:51700/0 (socket says 172.21.15.143:51700) 2023-11-08T19:29:21.847 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.842+0000 7f2372ffd700 1 -- 172.21.15.143:0/2175329662 learned_addr learned my addr 172.21.15.143:0/2175329662 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:29:21.848 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.842+0000 7f23727fc700 1 --2- 172.21.15.143:0/2175329662 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f23740cf090 0x7f23740d1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:21.848 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.842+0000 7f2371ffb700 1 -- 172.21.15.143:0/2175329662 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2291902125 0 0) 0x7f23740d3200 con 0x7f237415dc80 2023-11-08T19:29:21.848 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.842+0000 7f2371ffb700 1 -- 172.21.15.143:0/2175329662 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f235c0035b0 con 0x7f237415dc80 2023-11-08T19:29:21.848 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.843+0000 7f2371ffb700 1 -- 172.21.15.143:0/2175329662 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3888789169 0 0) 0x7f235c0035b0 con 0x7f237415dc80 2023-11-08T19:29:21.848 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.843+0000 7f2371ffb700 1 -- 172.21.15.143:0/2175329662 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f23740cf090 msgr2=0x7f23740d1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:21.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.843+0000 7f2371ffb700 1 --2- 172.21.15.143:0/2175329662 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f23740cf090 0x7f23740d1f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:21.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.843+0000 7f2371ffb700 1 -- 172.21.15.143:0/2175329662 >> v1:172.21.15.174:6789/0 conn(0x7f23740d3690 legacy=0x7f23740b6ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:21.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.843+0000 7f2371ffb700 1 -- 172.21.15.143:0/2175329662 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23740d28d0 con 0x7f237415dc80 2023-11-08T19:29:21.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.843+0000 7f2371ffb700 1 -- 172.21.15.143:0/2175329662 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4147756303 0 0) 0x7f2368001670 con 0x7f237415dc80 2023-11-08T19:29:21.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.843+0000 7f2371ffb700 1 -- 172.21.15.143:0/2175329662 >> v1:172.21.15.143:6789/0 conn(0x7f237415dc80 legacy=0x7f23740ce980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:21.850 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.843+0000 7f2371ffb700 1 --2- 172.21.15.143:0/2175329662 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f235c003a30 0x7f235c003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:21.850 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.843+0000 7f2371ffb700 1 --2- 172.21.15.143:0/2175329662 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f23740cf090 0x7f235c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:21.850 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.843+0000 7f2371ffb700 1 --2- 172.21.15.143:0/2175329662 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f235c004d90 0x7f235c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:21.850 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.843+0000 7f2371ffb700 1 -- 172.21.15.143:0/2175329662 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23740d25f0 con 0x7f235c004d90 2023-11-08T19:29:21.851 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.843+0000 7f2371ffb700 1 -- 172.21.15.143:0/2175329662 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f235c00bb40 con 0x7f235c003a30 2023-11-08T19:29:21.851 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.843+0000 7f2371ffb700 1 -- 172.21.15.143:0/2175329662 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f235c00bd00 con 0x7f23740cf090 2023-11-08T19:29:21.851 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.843+0000 7f23737fe700 1 --2- 172.21.15.143:0/2175329662 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f235c004d90 0x7f235c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:21.851 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.843+0000 7f2372ffd700 1 --2- 172.21.15.143:0/2175329662 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f235c003a30 0x7f235c003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:21.851 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.843+0000 7f23727fc700 1 --2- 172.21.15.143:0/2175329662 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f23740cf090 0x7f235c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:21.851 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.843+0000 7f23737fe700 1 -- 172.21.15.143:0/2175329662 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f23740cf090 msgr2=0x7f235c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:21.852 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.843+0000 7f23737fe700 1 --2- 172.21.15.143:0/2175329662 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f23740cf090 0x7f235c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:21.852 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.843+0000 7f23737fe700 1 -- 172.21.15.143:0/2175329662 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f235c003a30 msgr2=0x7f235c003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:21.852 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.843+0000 7f23737fe700 1 --2- 172.21.15.143:0/2175329662 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f235c003a30 0x7f235c003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:21.852 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.843+0000 7f23737fe700 1 -- 172.21.15.143:0/2175329662 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f235c0035b0 con 0x7f235c004d90 2023-11-08T19:29:21.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.843+0000 7f23737fe700 1 --2- 172.21.15.143:0/2175329662 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f235c004d90 0x7f235c005210 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f23740675d0 tx=0x7f235c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28877d2f8c3a2b1a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:21.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.843+0000 7f2371ffb700 1 -- 172.21.15.143:0/2175329662 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f236c001db0 con 0x7f235c004d90 2023-11-08T19:29:21.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.843+0000 7f2371ffb700 1 -- 172.21.15.143:0/2175329662 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f236c00cc70 con 0x7f235c004d90 2023-11-08T19:29:21.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.843+0000 7f2371ffb700 1 -- 172.21.15.143:0/2175329662 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f236c003a00 con 0x7f235c004d90 2023-11-08T19:29:21.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.844+0000 7f237950f700 1 -- 172.21.15.143:0/2175329662 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f235c004d90 msgr2=0x7f235c005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:21.854 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.844+0000 7f237950f700 1 --2- 172.21.15.143:0/2175329662 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f235c004d90 0x7f235c005210 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f23740675d0 tx=0x7f235c005880 comp rx=0 tx=0).stop 2023-11-08T19:29:21.854 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.844+0000 7f23737fe700 1 -- 172.21.15.143:0/2175329662 reap_dead start 2023-11-08T19:29:21.854 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.844+0000 7f237950f700 1 -- 172.21.15.143:0/2175329662 shutdown_connections 2023-11-08T19:29:21.854 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.844+0000 7f237950f700 1 -- 172.21.15.143:0/2175329662 >> 172.21.15.143:0/2175329662 conn(0x7f23740668e0 msgr2=0x7f2374066cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:21.854 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.844+0000 7f237950f700 1 -- 172.21.15.143:0/2175329662 shutdown_connections 2023-11-08T19:29:21.855 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.844+0000 7f237950f700 1 -- 172.21.15.143:0/2175329662 wait complete. 2023-11-08T19:29:21.855 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.845+0000 7f237950f700 1 Processor -- start 2023-11-08T19:29:21.855 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.845+0000 7f237950f700 1 -- start start 2023-11-08T19:29:21.865 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.846+0000 7f237950f700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f237414f100 0x7f23740cd670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:21.865 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.846+0000 7f237950f700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f23741f49c0 0x7f2374159110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:21.865 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.846+0000 7f237950f700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f23741f44d0 0x7f2374159650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:21.866 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.846+0000 7f237950f700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23740cbb50 con 0x7f23741f49c0 2023-11-08T19:29:21.866 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.846+0000 7f237950f700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23740caf40 con 0x7f237414f100 2023-11-08T19:29:21.866 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.846+0000 7f237950f700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23740cb220 con 0x7f23741f44d0 2023-11-08T19:29:21.866 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.846+0000 7f23727fc700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f23741f49c0 0x7f2374159110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:21.866 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.846+0000 7f23727fc700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f23741f49c0 0x7f2374159110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55560/0 (socket says 172.21.15.143:55560) 2023-11-08T19:29:21.867 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.846+0000 7f23727fc700 1 -- 172.21.15.143:0/3344594586 learned_addr learned my addr 172.21.15.143:0/3344594586 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:29:21.867 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.846+0000 7f23737fe700 1 --2- 172.21.15.143:0/3344594586 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f23741f44d0 0x7f2374159650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:21.867 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.846+0000 7f2372ffd700 1 --2- 172.21.15.143:0/3344594586 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f237414f100 0x7f23740cd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:21.867 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.846+0000 7f23727fc700 1 -- 172.21.15.143:0/3344594586 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f23741f44d0 msgr2=0x7f2374159650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:21.867 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.846+0000 7f23727fc700 1 --2- 172.21.15.143:0/3344594586 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f23741f44d0 0x7f2374159650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:21.868 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.846+0000 7f23727fc700 1 -- 172.21.15.143:0/3344594586 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f237414f100 msgr2=0x7f23740cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:21.868 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.846+0000 7f23727fc700 1 --2- 172.21.15.143:0/3344594586 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f237414f100 0x7f23740cd670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:21.868 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.846+0000 7f23727fc700 1 -- 172.21.15.143:0/3344594586 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23740cb500 con 0x7f23741f49c0 2023-11-08T19:29:21.868 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.846+0000 7f23737fe700 1 --2- 172.21.15.143:0/3344594586 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f23741f44d0 0x7f2374159650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:29:21.868 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.847+0000 7f23727fc700 1 --2- 172.21.15.143:0/3344594586 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f23741f49c0 0x7f2374159110 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f2364011aa0 tx=0x7f2364011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:21.869 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.847+0000 7f235b7fe700 1 -- 172.21.15.143:0/3344594586 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f236400f980 con 0x7f23741f49c0 2023-11-08T19:29:21.869 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.847+0000 7f235b7fe700 1 -- 172.21.15.143:0/3344594586 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f236400fae0 con 0x7f23741f49c0 2023-11-08T19:29:21.869 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.847+0000 7f237950f700 1 -- 172.21.15.143:0/3344594586 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23740cb7e0 con 0x7f23741f49c0 2023-11-08T19:29:21.869 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.847+0000 7f235b7fe700 1 -- 172.21.15.143:0/3344594586 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f236400fe90 con 0x7f23741f49c0 2023-11-08T19:29:21.869 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.847+0000 7f237950f700 1 -- 172.21.15.143:0/3344594586 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f237414f4e0 con 0x7f23741f49c0 2023-11-08T19:29:21.870 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.851+0000 7f2385708b80 1 Processor -- start 2023-11-08T19:29:21.870 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.851+0000 7f2385708b80 1 -- start start 2023-11-08T19:29:21.870 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.851+0000 7f2385708b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a8ed1f96f0 0x55a8ed229450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:21.870 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.851+0000 7f2385708b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a8ed1fbc30 0x55a8ed229990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:21.870 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.851+0000 7f2385708b80 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55a8ed1f9ad0 0x55a8ed213f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:21.871 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.852+0000 7f2385708b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8ed1656e0 con 0x55a8ed1f96f0 2023-11-08T19:29:21.871 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.852+0000 7f2385708b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8ed1593b0 con 0x55a8ed1fbc30 2023-11-08T19:29:21.871 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.852+0000 7f2385708b80 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a8ed156cd0 con 0x55a8ed1f9ad0 2023-11-08T19:29:21.871 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.852+0000 7f23737fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a8ed1f96f0 0x55a8ed229450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:21.872 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.852+0000 7f23737fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a8ed1f96f0 0x55a8ed229450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55564/0 (socket says 172.21.15.143:55564) 2023-11-08T19:29:21.872 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.852+0000 7f23737fe700 1 -- 172.21.15.143:0/1272672035 learned_addr learned my addr 172.21.15.143:0/1272672035 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:29:21.872 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.852+0000 7f235b7fe700 1 -- 172.21.15.143:0/3344594586 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2364014440 con 0x7f23741f49c0 2023-11-08T19:29:21.872 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.852+0000 7f2372ffd700 1 --2- 172.21.15.143:0/1272672035 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a8ed1fbc30 0x55a8ed229990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:21.873 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.852+0000 7f23737fe700 1 -- 172.21.15.143:0/1272672035 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55a8ed1f9ad0 msgr2=0x55a8ed213f90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-08T19:29:21.873 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.852+0000 7f23737fe700 1 --2- 172.21.15.143:0/1272672035 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55a8ed1f9ad0 0x55a8ed213f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:21.873 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.852+0000 7f23737fe700 1 -- 172.21.15.143:0/1272672035 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a8ed1fbc30 msgr2=0x55a8ed229990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:21.873 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.852+0000 7f23737fe700 1 --2- 172.21.15.143:0/1272672035 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a8ed1fbc30 0x55a8ed229990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:21.873 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.852+0000 7f23737fe700 1 -- 172.21.15.143:0/1272672035 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a8ecfcf5e0 con 0x55a8ed1f96f0 2023-11-08T19:29:21.874 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.852+0000 7f23737fe700 1 --2- 172.21.15.143:0/1272672035 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a8ed1f96f0 0x55a8ed229450 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f236c015040 tx=0x7f236c011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9077e9e21f5aa10f server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:21.874 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.852+0000 7f235b7fe700 1 --2- 172.21.15.143:0/3344594586 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f236006af00 0x7f236006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:21.874 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.852+0000 7f2358ff9700 1 -- 172.21.15.143:0/1272672035 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f236c001cd0 con 0x55a8ed1f96f0 2023-11-08T19:29:21.874 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.852+0000 7f2358ff9700 1 -- 172.21.15.143:0/1272672035 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f236c021070 con 0x55a8ed1f96f0 2023-11-08T19:29:21.874 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.852+0000 7f2358ff9700 1 -- 172.21.15.143:0/1272672035 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f236c003a00 con 0x55a8ed1f96f0 2023-11-08T19:29:21.875 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.852+0000 7f235b7fe700 1 -- 172.21.15.143:0/3344594586 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6973+0+0 (secure 0 0 0) 0x7f236408a1e0 con 0x7f23741f49c0 2023-11-08T19:29:21.875 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.852+0000 7f2372ffd700 1 --2- 172.21.15.143:0/3344594586 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f236006af00 0x7f236006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:21.875 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.853+0000 7f2372ffd700 1 --2- 172.21.15.143:0/3344594586 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f236006af00 0x7f236006d3c0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x55a8ed200b10 tx=0x7f2368006040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:21.875 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.853+0000 7f2385708b80 1 -- 172.21.15.143:0/1272672035 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a8ed1f96f0 msgr2=0x55a8ed229450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:21.875 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.853+0000 7f2385708b80 1 --2- 172.21.15.143:0/1272672035 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a8ed1f96f0 0x55a8ed229450 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f236c015040 tx=0x7f236c011000 comp rx=0 tx=0).stop 2023-11-08T19:29:21.876 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.853+0000 7f2385708b80 1 -- 172.21.15.143:0/1272672035 shutdown_connections 2023-11-08T19:29:21.876 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.853+0000 7f2385708b80 1 --2- 172.21.15.143:0/1272672035 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55a8ed1f9ad0 0x55a8ed213f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:21.876 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.853+0000 7f2385708b80 1 --2- 172.21.15.143:0/1272672035 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a8ed1f96f0 0x55a8ed229450 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:21.876 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.853+0000 7f2385708b80 1 --2- 172.21.15.143:0/1272672035 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a8ed1fbc30 0x55a8ed229990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:21.876 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.853+0000 7f2385708b80 1 -- 172.21.15.143:0/1272672035 >> 172.21.15.143:0/1272672035 conn(0x55a8ed174660 msgr2=0x55a8ed1ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:21.877 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.853+0000 7f2385708b80 1 -- 172.21.15.143:0/1272672035 shutdown_connections 2023-11-08T19:29:21.877 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.853+0000 7f2385708b80 1 -- 172.21.15.143:0/1272672035 wait complete. 2023-11-08T19:29:21.877 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.854+0000 7f2385708b80 1 -- start start 2023-11-08T19:29:21.877 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.854+0000 7f2385708b80 1 Processor -- start 2023-11-08T19:29:21.877 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.854+0000 7f2385708b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a8ed1f96f0 0x55a8ed22d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:21.877 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.854+0000 7f2385708b80 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55a8ed1f9ad0 0x55a8ed2140e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:21.878 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.854+0000 7f2385708b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a8ed214880 0x55a8ed240830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:21.878 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.855+0000 7f2385708b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8ed1525c0 con 0x55a8ed214880 2023-11-08T19:29:21.878 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.855+0000 7f2385708b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8ed0ee0d0 con 0x55a8ed1f96f0 2023-11-08T19:29:21.878 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.855+0000 7f2385708b80 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a8ed0edd70 con 0x55a8ed1f9ad0 2023-11-08T19:29:21.878 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.855+0000 7f2385708b80 10 client.0 _ll_register_callbacks cb 0x55a8ecf4a7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T19:29:21.879 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.855+0000 7f23737fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a8ed214880 0x55a8ed240830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:21.879 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.855+0000 7f23737fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a8ed214880 0x55a8ed240830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55576/0 (socket says 172.21.15.143:55576) 2023-11-08T19:29:21.879 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.855+0000 7f23737fe700 1 -- 172.21.15.143:0/47395209 learned_addr learned my addr 172.21.15.143:0/47395209 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:29:21.879 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.855+0000 7f23727fc700 1 --2- 172.21.15.143:0/47395209 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55a8ed1f9ad0 0x55a8ed2140e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:21.879 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.855+0000 7f2372ffd700 1 --2- 172.21.15.143:0/47395209 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a8ed1f96f0 0x55a8ed22d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:21.880 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.855+0000 7f23737fe700 1 -- 172.21.15.143:0/47395209 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55a8ed1f9ad0 msgr2=0x55a8ed2140e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:21.880 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.855+0000 7f23737fe700 1 --2- 172.21.15.143:0/47395209 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55a8ed1f9ad0 0x55a8ed2140e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:21.880 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.855+0000 7f23737fe700 1 -- 172.21.15.143:0/47395209 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a8ed1f96f0 msgr2=0x55a8ed22d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:21.880 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.855+0000 7f23737fe700 1 --2- 172.21.15.143:0/47395209 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a8ed1f96f0 0x55a8ed22d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:21.881 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.855+0000 7f23737fe700 1 -- 172.21.15.143:0/47395209 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f235c0035b0 con 0x55a8ed214880 2023-11-08T19:29:21.881 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.855+0000 7f2372ffd700 1 --2- 172.21.15.143:0/47395209 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a8ed1f96f0 0x55a8ed22d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:29:21.881 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.855+0000 7f23737fe700 1 --2- 172.21.15.143:0/47395209 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a8ed214880 0x55a8ed240830 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f236c00c720 tx=0x7f236c001510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afdf21fe04e78f03 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:21.881 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.855+0000 7f2351ffb700 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2023-11-08T19:29:21.881 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.855+0000 7f2351ffb700 1 -- 172.21.15.143:0/47395209 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f236c003c00 con 0x55a8ed214880 2023-11-08T19:29:21.881 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.855+0000 7f2351ffb700 1 -- 172.21.15.143:0/47395209 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 6973+0+0 (secure 0 0 0) 0x7f236c023070 con 0x55a8ed214880 2023-11-08T19:29:21.882 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.855+0000 7f2385708b80 1 -- 172.21.15.143:0/47395209 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a8ed1f1880 con 0x55a8ed214880 2023-11-08T19:29:21.882 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.855+0000 7f2351ffb700 1 -- 172.21.15.143:0/47395209 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7f236c003d60 con 0x55a8ed214880 2023-11-08T19:29:21.882 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.855+0000 7f2385708b80 10 client.7440 fetch_fsmap learned FSMap version 72 2023-11-08T19:29:21.882 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.855+0000 7f2385708b80 1 -- 172.21.15.143:0/47395209 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=2+}) v3 -- 0x55a8ecf7bd90 con 0x55a8ed214880 2023-11-08T19:29:21.882 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.855+0000 7f2351ffb700 1 -- 172.21.15.143:0/47395209 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a8ed1f1880 con 0x55a8ed214880 2023-11-08T19:29:21.883 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.856+0000 7f2351ffb700 1 -- 172.21.15.143:0/47395209 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 72) v1 ==== 3306+0+0 (secure 0 0 0) 0x7f236c005070 con 0x55a8ed214880 2023-11-08T19:29:21.883 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.856+0000 7f2385708b80 10 client.7440 fetch_fsmap finished waiting for FSMap version 72 2023-11-08T19:29:21.883 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.856+0000 7f2385708b80 10 client.7440 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-08T19:29:21.883 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.856+0000 7f2385708b80 20 client.7440 populate_metadata read hostname 'smithi143' 2023-11-08T19:29:21.883 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.856+0000 7f2385708b80 1 --2- 172.21.15.143:0/47395209 >> [v2:172.21.15.174:6834/3601088618,v1:172.21.15.174:6836/3601088618] conn(0x55a8ed22ba80 0x55a8ed22be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:21.884 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.856+0000 7f2385708b80 4 client.7440 mds_command: new command op to 6918 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-08T19:29:21.884 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.856+0000 7f2385708b80 1 -- 172.21.15.143:0/47395209 --> [v2:172.21.15.174:6834/3601088618,v1:172.21.15.174:6836/3601088618] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a8ed1cda30 con 0x55a8ed22ba80 2023-11-08T19:29:21.884 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.856+0000 7f2372ffd700 1 --2- 172.21.15.143:0/47395209 >> [v2:172.21.15.174:6834/3601088618,v1:172.21.15.174:6836/3601088618] conn(0x55a8ed22ba80 0x55a8ed22be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:21.884 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.856+0000 7f2351ffb700 10 client.7440 ms_handle_connect on v2:172.21.15.174:6834/3601088618 2023-11-08T19:29:21.884 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.856+0000 7f2372ffd700 1 --2- 172.21.15.143:0/47395209 >> [v2:172.21.15.174:6834/3601088618,v1:172.21.15.174:6836/3601088618] conn(0x55a8ed22ba80 0x55a8ed22be60 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4fc0944b998ed8ba server_cookie=b3d6d56d6b537fd0 in_seq=0 out_seq=0 2023-11-08T19:29:21.885 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.857+0000 7f2351ffb700 1 -- 172.21.15.143:0/47395209 <== mds.1 v2:172.21.15.174:6834/3601088618 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a8ed1cda30 con 0x55a8ed22ba80 2023-11-08T19:29:21.885 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.857+0000 7f2351ffb700 10 client.7440 handle_command_reply: tid=0 2023-11-08T19:29:21.885 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.857+0000 7f2385708b80 1 client.7440 shutdown 2023-11-08T19:29:21.885 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.857+0000 7f2385708b80 1 -- 172.21.15.143:0/47395209 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a8ed214880 msgr2=0x55a8ed240830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:21.885 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.857+0000 7f2385708b80 1 --2- 172.21.15.143:0/47395209 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a8ed214880 0x55a8ed240830 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f236c00c720 tx=0x7f236c001510 comp rx=0 tx=0).stop 2023-11-08T19:29:21.886 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.857+0000 7f2385708b80 1 -- 172.21.15.143:0/47395209 shutdown_connections 2023-11-08T19:29:21.886 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.857+0000 7f2385708b80 1 --2- 172.21.15.143:0/47395209 >> [v2:172.21.15.174:6834/3601088618,v1:172.21.15.174:6836/3601088618] conn(0x55a8ed22ba80 0x55a8ed22be60 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:21.886 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.857+0000 7f2385708b80 1 --2- 172.21.15.143:0/47395209 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a8ed214880 0x55a8ed240830 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:21.886 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.857+0000 7f2351ffb700 0 client.7440 ms_handle_reset on v2:172.21.15.174:6834/3601088618 2023-11-08T19:29:21.886 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.857+0000 7f2385708b80 1 --2- 172.21.15.143:0/47395209 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a8ed1f96f0 0x55a8ed22d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:21.887 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.857+0000 7f2385708b80 1 --2- 172.21.15.143:0/47395209 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55a8ed1f9ad0 0x55a8ed2140e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:21.887 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.857+0000 7f2385708b80 1 -- 172.21.15.143:0/47395209 >> 172.21.15.143:0/47395209 conn(0x55a8ed174660 msgr2=0x55a8ed1fa020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:21.887 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.857+0000 7f2385708b80 1 -- 172.21.15.143:0/47395209 shutdown_connections 2023-11-08T19:29:21.887 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.857+0000 7f2385708b80 1 -- 172.21.15.143:0/47395209 wait complete. 2023-11-08T19:29:21.887 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.857+0000 7f2385708b80 20 client.7440 trim_cache size 0 max 16384 2023-11-08T19:29:21.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.875+0000 7f2385708b80 1 Processor -- start 2023-11-08T19:29:21.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.875+0000 7f2385708b80 1 -- start start 2023-11-08T19:29:21.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.876+0000 7f2385708b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a8ed1f96f0 0x55a8ed256420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:21.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.876+0000 7f2385708b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a8ed1f9ad0 0x55a8ed256960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:21.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.876+0000 7f2385708b80 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55a8ed214880 0x55a8ed246060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:21.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.876+0000 7f2385708b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8ecfd18d0 con 0x55a8ed1f9ad0 2023-11-08T19:29:21.895 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.876+0000 7f2385708b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8ed1fe6e0 con 0x55a8ed1f96f0 2023-11-08T19:29:21.895 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.876+0000 7f2385708b80 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a8ed226ad0 con 0x55a8ed214880 2023-11-08T19:29:21.895 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.876+0000 7f23727fc700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a8ed1f9ad0 0x55a8ed256960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:21.895 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.876+0000 7f23727fc700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a8ed1f9ad0 0x55a8ed256960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55582/0 (socket says 172.21.15.143:55582) 2023-11-08T19:29:21.895 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.876+0000 7f23727fc700 1 -- 172.21.15.143:0/1360031477 learned_addr learned my addr 172.21.15.143:0/1360031477 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:29:21.896 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.876+0000 7f23727fc700 1 -- 172.21.15.143:0/1360031477 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55a8ed214880 msgr2=0x55a8ed246060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:21.896 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.876+0000 7f23727fc700 1 --2- 172.21.15.143:0/1360031477 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55a8ed214880 0x55a8ed246060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:21.896 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.876+0000 7f23727fc700 1 -- 172.21.15.143:0/1360031477 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a8ed1f96f0 msgr2=0x55a8ed256420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:21.896 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.876+0000 7f23727fc700 1 --2- 172.21.15.143:0/1360031477 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a8ed1f96f0 0x55a8ed256420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:21.896 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.876+0000 7f23727fc700 1 -- 172.21.15.143:0/1360031477 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a8ecf64c10 con 0x55a8ed1f9ad0 2023-11-08T19:29:21.897 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.876+0000 7f23727fc700 1 --2- 172.21.15.143:0/1360031477 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a8ed1f9ad0 0x55a8ed256960 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f2364011a70 tx=0x7f2364009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9618fde62d63358 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:21.897 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.876+0000 7f23597fa700 1 -- 172.21.15.143:0/1360031477 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2364058c60 con 0x55a8ed1f9ad0 2023-11-08T19:29:21.897 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.876+0000 7f23597fa700 1 -- 172.21.15.143:0/1360031477 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f236405cd00 con 0x55a8ed1f9ad0 2023-11-08T19:29:21.897 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.876+0000 7f23597fa700 1 -- 172.21.15.143:0/1360031477 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2364060eb0 con 0x55a8ed1f9ad0 2023-11-08T19:29:21.897 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.877+0000 7f2385708b80 1 -- 172.21.15.143:0/1360031477 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a8ed1f9ad0 msgr2=0x55a8ed256960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:21.898 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.877+0000 7f2385708b80 1 --2- 172.21.15.143:0/1360031477 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a8ed1f9ad0 0x55a8ed256960 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f2364011a70 tx=0x7f2364009f90 comp rx=0 tx=0).stop 2023-11-08T19:29:21.898 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.877+0000 7f2385708b80 1 -- 172.21.15.143:0/1360031477 shutdown_connections 2023-11-08T19:29:21.898 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.877+0000 7f2385708b80 1 --2- 172.21.15.143:0/1360031477 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55a8ed214880 0x55a8ed246060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:21.898 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.877+0000 7f2385708b80 1 --2- 172.21.15.143:0/1360031477 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a8ed1f96f0 0x55a8ed256420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:21.898 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.877+0000 7f2385708b80 1 --2- 172.21.15.143:0/1360031477 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a8ed1f9ad0 0x55a8ed256960 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:21.899 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.877+0000 7f2385708b80 1 -- 172.21.15.143:0/1360031477 >> 172.21.15.143:0/1360031477 conn(0x55a8ed174660 msgr2=0x55a8ed1f57e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:21.899 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.877+0000 7f2385708b80 1 -- 172.21.15.143:0/1360031477 shutdown_connections 2023-11-08T19:29:21.899 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.877+0000 7f2385708b80 1 -- 172.21.15.143:0/1360031477 wait complete. 2023-11-08T19:29:21.899 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.877+0000 7f2385708b80 1 -- start start 2023-11-08T19:29:21.899 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.878+0000 7f2385708b80 1 Processor -- start 2023-11-08T19:29:21.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.879+0000 7f2385708b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a8ed1f96f0 0x55a8ed22be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:21.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.879+0000 7f2385708b80 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55a8ed1f9ad0 0x55a8ed22d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:21.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.879+0000 7f2385708b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a8ed214880 0x55a8ed255410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:21.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.879+0000 7f2385708b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8ed2566b0 con 0x55a8ed1f96f0 2023-11-08T19:29:21.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.879+0000 7f2385708b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8ed1fba50 con 0x55a8ed214880 2023-11-08T19:29:21.901 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.879+0000 7f2385708b80 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a8ed1fa910 con 0x55a8ed1f9ad0 2023-11-08T19:29:21.901 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.879+0000 7f2385708b80 10 client.0 _ll_register_callbacks cb 0x7f236c015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T19:29:21.901 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.879+0000 7f2372ffd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a8ed1f96f0 0x55a8ed22be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:21.901 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.879+0000 7f2372ffd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a8ed1f96f0 0x55a8ed22be60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55586/0 (socket says 172.21.15.143:55586) 2023-11-08T19:29:21.901 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.879+0000 7f2372ffd700 1 -- 172.21.15.143:0/2329449961 learned_addr learned my addr 172.21.15.143:0/2329449961 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:29:21.902 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.879+0000 7f23727fc700 1 --2- 172.21.15.143:0/2329449961 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55a8ed1f9ad0 0x55a8ed22d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:21.902 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.879+0000 7f23737fe700 1 --2- 172.21.15.143:0/2329449961 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a8ed214880 0x55a8ed255410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:21.902 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.879+0000 7f2372ffd700 1 -- 172.21.15.143:0/2329449961 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55a8ed1f9ad0 msgr2=0x55a8ed22d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:21.902 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.879+0000 7f2372ffd700 1 --2- 172.21.15.143:0/2329449961 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55a8ed1f9ad0 0x55a8ed22d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:21.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.879+0000 7f2372ffd700 1 -- 172.21.15.143:0/2329449961 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a8ed214880 msgr2=0x55a8ed255410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:21.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.879+0000 7f2372ffd700 1 --2- 172.21.15.143:0/2329449961 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a8ed214880 0x55a8ed255410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:21.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.879+0000 7f2372ffd700 1 -- 172.21.15.143:0/2329449961 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f237414f4e0 con 0x55a8ed1f96f0 2023-11-08T19:29:21.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.879+0000 7f2372ffd700 1 --2- 172.21.15.143:0/2329449961 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a8ed1f96f0 0x55a8ed22be60 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f237415c920 tx=0x7f2368009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52054151e5f3326f server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:21.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.879+0000 7f2351ffb700 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2023-11-08T19:29:21.904 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.879+0000 7f2351ffb700 1 -- 172.21.15.143:0/2329449961 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2368028900 con 0x55a8ed1f96f0 2023-11-08T19:29:21.904 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.879+0000 7f2351ffb700 1 -- 172.21.15.143:0/2329449961 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 6973+0+0 (secure 0 0 0) 0x7f236801f040 con 0x55a8ed1f96f0 2023-11-08T19:29:21.904 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.879+0000 7f2385708b80 1 -- 172.21.15.143:0/2329449961 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f236c021070 con 0x55a8ed1f96f0 2023-11-08T19:29:21.904 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.880+0000 7f2351ffb700 1 -- 172.21.15.143:0/2329449961 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2368016450 con 0x55a8ed1f96f0 2023-11-08T19:29:21.904 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.880+0000 7f2385708b80 10 client.7446 fetch_fsmap learned FSMap version 72 2023-11-08T19:29:21.905 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.880+0000 7f2385708b80 1 -- 172.21.15.143:0/2329449961 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=2+}) v3 -- 0x55a8ecfa2200 con 0x55a8ed1f96f0 2023-11-08T19:29:21.905 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.880+0000 7f2351ffb700 1 -- 172.21.15.143:0/2329449961 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f236c021070 con 0x55a8ed1f96f0 2023-11-08T19:29:21.905 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.880+0000 7f2351ffb700 1 -- 172.21.15.143:0/2329449961 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 72) v1 ==== 3306+0+0 (secure 0 0 0) 0x7f2368030030 con 0x55a8ed1f96f0 2023-11-08T19:29:21.905 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.880+0000 7f2385708b80 10 client.7446 fetch_fsmap finished waiting for FSMap version 72 2023-11-08T19:29:21.905 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.880+0000 7f2385708b80 10 client.7446 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-08T19:29:21.906 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.880+0000 7f2385708b80 20 client.7446 populate_metadata read hostname 'smithi143' 2023-11-08T19:29:21.906 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.880+0000 7f2385708b80 1 --2- 172.21.15.143:0/2329449961 >> [v2:172.21.15.174:6834/3601088618,v1:172.21.15.174:6836/3601088618] conn(0x55a8ed225860 0x55a8ed255aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:21.906 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.880+0000 7f2385708b80 4 client.7446 mds_command: new command op to 6918 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-08T19:29:21.906 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.880+0000 7f2385708b80 1 -- 172.21.15.143:0/2329449961 --> [v2:172.21.15.174:6834/3601088618,v1:172.21.15.174:6836/3601088618] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55a8ed14b5a0 con 0x55a8ed225860 2023-11-08T19:29:21.906 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.880+0000 7f23727fc700 1 --2- 172.21.15.143:0/2329449961 >> [v2:172.21.15.174:6834/3601088618,v1:172.21.15.174:6836/3601088618] conn(0x55a8ed225860 0x55a8ed255aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:21.907 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.880+0000 7f2351ffb700 10 client.7446 ms_handle_connect on v2:172.21.15.174:6834/3601088618 2023-11-08T19:29:21.907 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.880+0000 7f23727fc700 1 --2- 172.21.15.143:0/2329449961 >> [v2:172.21.15.174:6834/3601088618,v1:172.21.15.174:6836/3601088618] conn(0x55a8ed225860 0x55a8ed255aa0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3ccb9831c1bc561d server_cookie=7488df46b9d39f4e in_seq=0 out_seq=0 2023-11-08T19:29:21.907 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.881+0000 7f2351ffb700 1 -- 172.21.15.143:0/2329449961 <== mds.1 v2:172.21.15.174:6834/3601088618 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55a8ed14b5a0 con 0x55a8ed225860 2023-11-08T19:29:21.907 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.881+0000 7f2351ffb700 10 client.7446 handle_command_reply: tid=0 2023-11-08T19:29:21.907 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.881+0000 7f2385708b80 1 client.7446 shutdown 2023-11-08T19:29:21.908 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.881+0000 7f2385708b80 1 -- 172.21.15.143:0/2329449961 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a8ed1f96f0 msgr2=0x55a8ed22be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:21.908 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.881+0000 7f2385708b80 1 --2- 172.21.15.143:0/2329449961 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a8ed1f96f0 0x55a8ed22be60 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f237415c920 tx=0x7f2368009040 comp rx=0 tx=0).stop 2023-11-08T19:29:21.908 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.881+0000 7f2385708b80 1 -- 172.21.15.143:0/2329449961 shutdown_connections 2023-11-08T19:29:21.908 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.881+0000 7f2385708b80 1 --2- 172.21.15.143:0/2329449961 >> [v2:172.21.15.174:6834/3601088618,v1:172.21.15.174:6836/3601088618] conn(0x55a8ed225860 0x55a8ed255aa0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:21.908 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.881+0000 7f2385708b80 1 --2- 172.21.15.143:0/2329449961 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a8ed214880 0x55a8ed255410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:21.909 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.881+0000 7f2385708b80 1 --2- 172.21.15.143:0/2329449961 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a8ed1f96f0 0x55a8ed22be60 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:21.909 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.881+0000 7f2351ffb700 0 client.7446 ms_handle_reset on v2:172.21.15.174:6834/3601088618 2023-11-08T19:29:21.909 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.881+0000 7f2385708b80 1 --2- 172.21.15.143:0/2329449961 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55a8ed1f9ad0 0x55a8ed22d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:21.909 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.881+0000 7f2385708b80 1 -- 172.21.15.143:0/2329449961 >> 172.21.15.143:0/2329449961 conn(0x55a8ed174660 msgr2=0x55a8ed1fce00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:21.909 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.881+0000 7f2385708b80 1 -- 172.21.15.143:0/2329449961 shutdown_connections 2023-11-08T19:29:21.909 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.881+0000 7f2385708b80 1 -- 172.21.15.143:0/2329449961 wait complete. 2023-11-08T19:29:21.910 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.881+0000 7f2385708b80 20 client.7446 trim_cache size 0 max 16384 2023-11-08T19:29:21.910 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2023-11-08T19:29:21.910 INFO:teuthology.orchestra.run.smithi143.stdout: "cluster_fsid": "968483af-77fc-4068-ad7c-5c2a1a4613d7", 2023-11-08T19:29:21.910 INFO:teuthology.orchestra.run.smithi143.stdout: "whoami": 1, 2023-11-08T19:29:21.911 INFO:teuthology.orchestra.run.smithi143.stdout: "id": 6918, 2023-11-08T19:29:21.911 INFO:teuthology.orchestra.run.smithi143.stdout: "want_state": "up:active", 2023-11-08T19:29:21.911 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "up:active", 2023-11-08T19:29:21.911 INFO:teuthology.orchestra.run.smithi143.stdout: "fs_name": "cephfs", 2023-11-08T19:29:21.911 INFO:teuthology.orchestra.run.smithi143.stdout: "rank_uptime": 64.287137190999999, 2023-11-08T19:29:21.912 INFO:teuthology.orchestra.run.smithi143.stdout: "mdsmap_epoch": 72, 2023-11-08T19:29:21.912 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_epoch": 96, 2023-11-08T19:29:21.912 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_epoch_barrier": 96, 2023-11-08T19:29:21.912 INFO:teuthology.orchestra.run.smithi143.stdout: "uptime": 84.078354681999997 2023-11-08T19:29:21.912 INFO:teuthology.orchestra.run.smithi143.stdout:} 2023-11-08T19:29:21.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.881+0000 7f237950f700 1 -- 172.21.15.143:0/3344594586 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f236006af00 msgr2=0x7f236006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:21.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.881+0000 7f237950f700 1 --2- 172.21.15.143:0/3344594586 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f236006af00 0x7f236006d3c0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x55a8ed200b10 tx=0x7f2368006040 comp rx=0 tx=0).stop 2023-11-08T19:29:21.915 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.881+0000 7f237950f700 1 -- 172.21.15.143:0/3344594586 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f23741f49c0 msgr2=0x7f2374159110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:21.915 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.881+0000 7f237950f700 1 --2- 172.21.15.143:0/3344594586 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f23741f49c0 0x7f2374159110 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f2364011aa0 tx=0x7f2364011e60 comp rx=0 tx=0).stop 2023-11-08T19:29:21.915 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.881+0000 7f237950f700 1 -- 172.21.15.143:0/3344594586 shutdown_connections 2023-11-08T19:29:21.915 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.881+0000 7f237950f700 1 --2- 172.21.15.143:0/3344594586 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f236006af00 0x7f236006d3c0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:21.915 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.881+0000 7f237950f700 1 --2- 172.21.15.143:0/3344594586 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f23741f44d0 0x7f2374159650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:21.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.881+0000 7f237950f700 1 --2- 172.21.15.143:0/3344594586 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f237414f100 0x7f23740cd670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:21.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.881+0000 7f237950f700 1 --2- 172.21.15.143:0/3344594586 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f23741f49c0 0x7f2374159110 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:21.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.881+0000 7f237950f700 1 -- 172.21.15.143:0/3344594586 >> 172.21.15.143:0/3344594586 conn(0x7f23740668e0 msgr2=0x7f2374160a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:21.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.881+0000 7f237950f700 1 -- 172.21.15.143:0/3344594586 shutdown_connections 2023-11-08T19:29:21.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:21.881+0000 7f237950f700 1 -- 172.21.15.143:0/3344594586 wait complete. 2023-11-08T19:29:21.917 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:29:22.037 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.035+0000 7f2f59d17700 1 Processor -- start 2023-11-08T19:29:22.037 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.035+0000 7f2f59d17700 1 -- start start 2023-11-08T19:29:22.040 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.036+0000 7f2f59d17700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2f5415aa30 0x7f2f540cf510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:22.040 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.036+0000 7f2f59d17700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f2f540b2210 0x7f2f540b2690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:22.040 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.036+0000 7f2f59d17700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f540cf080 con 0x7f2f540cbe20 2023-11-08T19:29:22.040 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.036+0000 7f2f59d17700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f540b2bd0 con 0x7f2f540b2210 2023-11-08T19:29:22.040 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.036+0000 7f2f59d17700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f540ce470 con 0x7f2f5415aa30 2023-11-08T19:29:22.041 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.036+0000 7f2f52ffd700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f2f540b2210 0x7f2f540b2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:22.041 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.036+0000 7f2f52ffd700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f2f540b2210 0x7f2f540b2690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55596/0 (socket says 172.21.15.143:55596) 2023-11-08T19:29:22.041 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.036+0000 7f2f52ffd700 1 -- 172.21.15.143:0/662942960 learned_addr learned my addr 172.21.15.143:0/662942960 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:29:22.041 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.036+0000 7f2f537fe700 1 --2- 172.21.15.143:0/662942960 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2f5415aa30 0x7f2f540cf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:22.041 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.037+0000 7f2f52ffd700 1 -- 172.21.15.143:0/662942960 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2f5415aa30 msgr2=0x7f2f540cf510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:22.042 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.037+0000 7f2f52ffd700 1 --2- 172.21.15.143:0/662942960 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2f5415aa30 0x7f2f540cf510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:22.042 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.037+0000 7f2f52ffd700 1 -- 172.21.15.143:0/662942960 >> v1:172.21.15.174:6789/0 conn(0x7f2f540cbe20 legacy=0x7f2f540cc270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:22.042 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.037+0000 7f2f52ffd700 1 -- 172.21.15.143:0/662942960 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f540ce750 con 0x7f2f540b2210 2023-11-08T19:29:22.042 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.037+0000 7f2f52ffd700 1 --2- 172.21.15.143:0/662942960 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2f540b2210 0x7f2f540b2690 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f2f440098f0 tx=0x7f2f44009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8435f7a85556e1b8 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:22.042 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.037+0000 7f2f527fc700 1 -- 172.21.15.143:0/662942960 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f44011830 con 0x7f2f540b2210 2023-11-08T19:29:22.043 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.037+0000 7f2f527fc700 1 -- 172.21.15.143:0/662942960 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f44011990 con 0x7f2f540b2210 2023-11-08T19:29:22.043 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.037+0000 7f2f527fc700 1 -- 172.21.15.143:0/662942960 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f44011d20 con 0x7f2f540b2210 2023-11-08T19:29:22.043 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.037+0000 7f2f59d17700 1 -- 172.21.15.143:0/662942960 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2f540b2210 msgr2=0x7f2f540b2690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:22.043 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.037+0000 7f2f59d17700 1 --2- 172.21.15.143:0/662942960 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2f540b2210 0x7f2f540b2690 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f2f440098f0 tx=0x7f2f44009c00 comp rx=0 tx=0).stop 2023-11-08T19:29:22.043 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.037+0000 7f2f59d17700 1 -- 172.21.15.143:0/662942960 shutdown_connections 2023-11-08T19:29:22.044 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.037+0000 7f2f59d17700 1 --2- 172.21.15.143:0/662942960 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2f5415aa30 0x7f2f540cf510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:22.044 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.037+0000 7f2f59d17700 1 --2- 172.21.15.143:0/662942960 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2f540b2210 0x7f2f540b2690 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:22.044 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.037+0000 7f2f59d17700 1 -- 172.21.15.143:0/662942960 >> 172.21.15.143:0/662942960 conn(0x7f2f54099ab0 msgr2=0x7f2f540d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:22.044 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.037+0000 7f2f59d17700 1 -- 172.21.15.143:0/662942960 shutdown_connections 2023-11-08T19:29:22.044 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.038+0000 7f2f59d17700 1 -- 172.21.15.143:0/662942960 wait complete. 2023-11-08T19:29:22.045 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.039+0000 7f2f59d17700 1 Processor -- start 2023-11-08T19:29:22.048 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.039+0000 7f2f59d17700 1 -- start start 2023-11-08T19:29:22.048 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.040+0000 7f2f59d17700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2f540b2210 0x7f2f540c4230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:22.048 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.040+0000 7f2f59d17700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2f540cbe20 0x7f2f5415b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:22.048 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.040+0000 7f2f59d17700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2f5415aa30 0x7f2f5415b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:22.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.040+0000 7f2f59d17700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f540afa50 con 0x7f2f540b2210 2023-11-08T19:29:22.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.040+0000 7f2f59d17700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f540b4d20 con 0x7f2f540cbe20 2023-11-08T19:29:22.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.040+0000 7f2f59d17700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f540b5000 con 0x7f2f5415aa30 2023-11-08T19:29:22.050 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.040+0000 7f2f537fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2f540b2210 0x7f2f540c4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:22.050 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.040+0000 7f2f537fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2f540b2210 0x7f2f540c4230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55600/0 (socket says 172.21.15.143:55600) 2023-11-08T19:29:22.050 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.040+0000 7f2f537fe700 1 -- 172.21.15.143:0/353138188 learned_addr learned my addr 172.21.15.143:0/353138188 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:29:22.050 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.040+0000 7f2f53fff700 1 --2- 172.21.15.143:0/353138188 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2f5415aa30 0x7f2f5415b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:22.050 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.040+0000 7f2f52ffd700 1 --2- 172.21.15.143:0/353138188 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2f540cbe20 0x7f2f5415b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:22.051 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.040+0000 7f2f537fe700 1 -- 172.21.15.143:0/353138188 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2f5415aa30 msgr2=0x7f2f5415b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:22.051 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.040+0000 7f2f537fe700 1 --2- 172.21.15.143:0/353138188 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2f5415aa30 0x7f2f5415b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:22.051 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.040+0000 7f2f537fe700 1 -- 172.21.15.143:0/353138188 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2f540cbe20 msgr2=0x7f2f5415b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:22.051 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.040+0000 7f2f537fe700 1 --2- 172.21.15.143:0/353138188 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2f540cbe20 0x7f2f5415b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:22.052 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.040+0000 7f2f537fe700 1 -- 172.21.15.143:0/353138188 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f44009550 con 0x7f2f540b2210 2023-11-08T19:29:22.052 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.040+0000 7f2f52ffd700 1 --2- 172.21.15.143:0/353138188 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2f540cbe20 0x7f2f5415b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:29:22.052 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.040+0000 7f2f537fe700 1 --2- 172.21.15.143:0/353138188 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2f540b2210 0x7f2f540c4230 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f2f4800e900 tx=0x7f2f4800ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:22.052 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.040+0000 7f2f3bfff700 1 -- 172.21.15.143:0/353138188 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f48009650 con 0x7f2f540b2210 2023-11-08T19:29:22.052 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.040+0000 7f2f3bfff700 1 -- 172.21.15.143:0/353138188 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f4800ee50 con 0x7f2f540b2210 2023-11-08T19:29:22.053 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.040+0000 7f2f3bfff700 1 -- 172.21.15.143:0/353138188 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f4800ca50 con 0x7f2f540b2210 2023-11-08T19:29:22.053 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.040+0000 7f2f59d17700 1 -- 172.21.15.143:0/353138188 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f540b52e0 con 0x7f2f540b2210 2023-11-08T19:29:22.053 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.040+0000 7f2f59d17700 1 -- 172.21.15.143:0/353138188 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f540c4770 con 0x7f2f540b2210 2023-11-08T19:29:22.053 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.041+0000 7f2f59d17700 1 -- 172.21.15.143:0/353138188 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f5404eee0 con 0x7f2f540b2210 2023-11-08T19:29:22.053 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.042+0000 7f2f3bfff700 1 -- 172.21.15.143:0/353138188 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2f4800cbb0 con 0x7f2f540b2210 2023-11-08T19:29:22.054 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.042+0000 7f2f3bfff700 1 --2- 172.21.15.143:0/353138188 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f2f4006af00 0x7f2f4006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:22.054 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.042+0000 7f2f3bfff700 1 -- 172.21.15.143:0/353138188 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6973+0+0 (secure 0 0 0) 0x7f2f48089e60 con 0x7f2f540b2210 2023-11-08T19:29:22.054 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.042+0000 7f2f52ffd700 1 --2- 172.21.15.143:0/353138188 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f2f4006af00 0x7f2f4006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:22.054 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.043+0000 7f2f52ffd700 1 --2- 172.21.15.143:0/353138188 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f2f4006af00 0x7f2f4006d3c0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f2f44009c40 tx=0x7f2f44008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:22.054 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.046+0000 7f2f3bfff700 1 -- 172.21.15.143:0/353138188 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2f48054a20 con 0x7f2f540b2210 2023-11-08T19:29:22.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.276+0000 7f2f59d17700 1 -- 172.21.15.143:0/353138188 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2f540b5930 con 0x7f2f540b2210 2023-11-08T19:29:22.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.276+0000 7f2f3bfff700 1 -- 172.21.15.143:0/353138188 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f2f48021090 con 0x7f2f540b2210 2023-11-08T19:29:22.278 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:29:22.278 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":72,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5933,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/3918286710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6835","nonce":3918286710},{"type":"v1","addr":"172.21.15.174:6837","nonce":3918286710}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":7386,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3664584522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":3664584522},{"type":"v1","addr":"172.21.15.143:6835","nonce":3664584522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:03.092245+0000","modified":"2023-11-08T19:29:17.961359+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":96,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6906,"mds_1":6918},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6906":{"gid":6906,"name":"c","rank":0,"incarnation":68,"state":"up:active","state_seq":24,"addr":"172.21.15.143:6834/3172664377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":3172664377},{"type":"v1","addr":"172.21.15.143:6834","nonce":3172664377}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6918":{"gid":6918,"name":"b","rank":1,"incarnation":54,"state":"up:active","state_seq":6,"addr":"172.21.15.174:6836/3601088618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3601088618},{"type":"v1","addr":"172.21.15.174:6836","nonce":3601088618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:29:22.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.278+0000 7f2f59d17700 1 -- 172.21.15.143:0/353138188 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f2f4006af00 msgr2=0x7f2f4006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:22.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.278+0000 7f2f59d17700 1 --2- 172.21.15.143:0/353138188 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f2f4006af00 0x7f2f4006d3c0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f2f44009c40 tx=0x7f2f44008030 comp rx=0 tx=0).stop 2023-11-08T19:29:22.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.278+0000 7f2f59d17700 1 -- 172.21.15.143:0/353138188 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2f540b2210 msgr2=0x7f2f540c4230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:22.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.278+0000 7f2f59d17700 1 --2- 172.21.15.143:0/353138188 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2f540b2210 0x7f2f540c4230 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f2f4800e900 tx=0x7f2f4800ec10 comp rx=0 tx=0).stop 2023-11-08T19:29:22.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.278+0000 7f2f59d17700 1 -- 172.21.15.143:0/353138188 shutdown_connections 2023-11-08T19:29:22.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.278+0000 7f2f59d17700 1 --2- 172.21.15.143:0/353138188 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f2f4006af00 0x7f2f4006d3c0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:22.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.278+0000 7f2f59d17700 1 --2- 172.21.15.143:0/353138188 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2f5415aa30 0x7f2f5415b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:22.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.278+0000 7f2f59d17700 1 --2- 172.21.15.143:0/353138188 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2f540b2210 0x7f2f540c4230 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:22.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.278+0000 7f2f59d17700 1 --2- 172.21.15.143:0/353138188 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2f540cbe20 0x7f2f5415b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:22.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.278+0000 7f2f59d17700 1 -- 172.21.15.143:0/353138188 >> 172.21.15.143:0/353138188 conn(0x7f2f54099ab0 msgr2=0x7f2f540cf5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:22.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.279+0000 7f2f59d17700 1 -- 172.21.15.143:0/353138188 shutdown_connections 2023-11-08T19:29:22.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.279+0000 7f2f59d17700 1 -- 172.21.15.143:0/353138188 wait complete. 2023-11-08T19:29:22.283 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 72 2023-11-08T19:29:22.292 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json dump snaps --server 2023-11-08T19:29:22.428 INFO:teuthology.orchestra.run.smithi143.stdout:{"last_snap":3,"last_created":3,"last_destroyed":1,"pending_noop":[],"snaps":[{"snapid":2,"ino":1099511627777,"stamp":"2023-11-08T19:28:31.007395+0000","name":"s11","metadata":{}},{"snapid":3,"ino":1099511627777,"stamp":"2023-11-08T19:28:57.577300+0000","name":"s12","metadata":{}}],"need_to_purge":{},"pending_update":[],"pending_destroy":[]} 2023-11-08T19:29:22.429 DEBUG:tasks.cephfs.filesystem:_json_asok output { "last_created": 3, "last_destroyed": 1, "last_snap": 3, "need_to_purge": {}, "pending_destroy": [], "pending_noop": [], "pending_update": [], "snaps": [ { "ino": 1099511627777, "metadata": {}, "name": "s11", "snapid": 2, "stamp": "2023-11-08T19:28:31.007395+0000" }, { "ino": 1099511627777, "metadata": {}, "name": "s12", "snapid": 3, "stamp": "2023-11-08T19:28:57.577300+0000" } ] } 2023-11-08T19:29:22.429 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-11-08T19:29:22.429 INFO:tasks.cephfs.test_snapshots:testing snapserver mds_kill_mdstable_at=5 2023-11-08T19:29:22.430 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:29:22.551 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.549+0000 7f40466bd700 1 Processor -- start 2023-11-08T19:29:22.551 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.549+0000 7f40466bd700 1 -- start start 2023-11-08T19:29:22.554 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.550+0000 7f40466bd700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4040157e30 0x7f4040071aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:22.554 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.550+0000 7f40466bd700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f40400cde00 0x7f40400cd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:22.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.550+0000 7f40466bd700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f40400ce280 0x7f40400cd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:22.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.550+0000 7f40466bd700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4040071680 con 0x7f40400ce280 2023-11-08T19:29:22.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.550+0000 7f40466bd700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40400ceb30 con 0x7f40400cde00 2023-11-08T19:29:22.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.550+0000 7f40466bd700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4040070fe0 con 0x7f4040157e30 2023-11-08T19:29:22.556 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.550+0000 7f4044c5a700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f40400ce280 0x7f40400cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:22.556 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.550+0000 7f4044c5a700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f40400ce280 0x7f40400cd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55610/0 (socket says 172.21.15.143:55610) 2023-11-08T19:29:22.556 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.550+0000 7f4044c5a700 1 -- 172.21.15.143:0/930758714 learned_addr learned my addr 172.21.15.143:0/930758714 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:29:22.556 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.551+0000 7f403ffff700 1 --2- 172.21.15.143:0/930758714 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4040157e30 0x7f4040071aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:22.557 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.551+0000 7f403f7fe700 1 --2- 172.21.15.143:0/930758714 >> v2:172.21.15.174:3300/0 conn(0x7f40400cde00 0x7f40400cd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:22.557 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.551+0000 7f4044c5a700 1 -- 172.21.15.143:0/930758714 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4040157e30 msgr2=0x7f4040071aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:22.558 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.551+0000 7f4044c5a700 1 --2- 172.21.15.143:0/930758714 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4040157e30 0x7f4040071aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:22.558 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.551+0000 7f4044c5a700 1 -- 172.21.15.143:0/930758714 >> v2:172.21.15.174:3300/0 conn(0x7f40400cde00 msgr2=0x7f40400cd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:22.559 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.551+0000 7f4044c5a700 1 --2- 172.21.15.143:0/930758714 >> v2:172.21.15.174:3300/0 conn(0x7f40400cde00 0x7f40400cd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:22.559 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.551+0000 7f4044c5a700 1 -- 172.21.15.143:0/930758714 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40400ce850 con 0x7f40400ce280 2023-11-08T19:29:22.559 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.551+0000 7f403f7fe700 1 --2- 172.21.15.143:0/930758714 >> v2:172.21.15.174:3300/0 conn(0x7f40400cde00 0x7f40400cd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:29:22.560 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.551+0000 7f4044c5a700 1 --2- 172.21.15.143:0/930758714 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f40400ce280 0x7f40400cd700 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f404006f120 tx=0x7f403800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b51fd2d3a4289dd server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:22.560 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.551+0000 7f403effd700 1 -- 172.21.15.143:0/930758714 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f403800e070 con 0x7f40400ce280 2023-11-08T19:29:22.560 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.551+0000 7f403effd700 1 -- 172.21.15.143:0/930758714 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f403800b9c0 con 0x7f40400ce280 2023-11-08T19:29:22.561 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.551+0000 7f403effd700 1 -- 172.21.15.143:0/930758714 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f403800bd70 con 0x7f40400ce280 2023-11-08T19:29:22.561 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.552+0000 7f40466bd700 1 -- 172.21.15.143:0/930758714 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f40400ce280 msgr2=0x7f40400cd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:22.562 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.552+0000 7f40466bd700 1 --2- 172.21.15.143:0/930758714 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f40400ce280 0x7f40400cd700 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f404006f120 tx=0x7f403800b330 comp rx=0 tx=0).stop 2023-11-08T19:29:22.563 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.552+0000 7f40466bd700 1 -- 172.21.15.143:0/930758714 shutdown_connections 2023-11-08T19:29:22.563 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.552+0000 7f40466bd700 1 --2- 172.21.15.143:0/930758714 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f40400ce280 0x7f40400cd700 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:22.563 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.552+0000 7f40466bd700 1 --2- 172.21.15.143:0/930758714 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4040157e30 0x7f4040071aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:22.564 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.552+0000 7f40466bd700 1 --2- 172.21.15.143:0/930758714 >> v2:172.21.15.174:3300/0 conn(0x7f40400cde00 0x7f40400cd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:22.564 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.552+0000 7f40466bd700 1 -- 172.21.15.143:0/930758714 >> 172.21.15.143:0/930758714 conn(0x7f404006b4c0 msgr2=0x7f40400cf680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:22.564 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.552+0000 7f40466bd700 1 -- 172.21.15.143:0/930758714 shutdown_connections 2023-11-08T19:29:22.564 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.552+0000 7f40466bd700 1 -- 172.21.15.143:0/930758714 wait complete. 2023-11-08T19:29:22.564 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.553+0000 7f40466bd700 1 Processor -- start 2023-11-08T19:29:22.565 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.553+0000 7f40466bd700 1 -- start start 2023-11-08T19:29:22.568 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.554+0000 7f40466bd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f40400cde00 0x7f40400c9210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:22.568 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.554+0000 7f40466bd700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f40400ce280 0x7f40400c7220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:22.568 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.554+0000 7f40466bd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4040157e30 0x7f40401f77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:22.568 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.554+0000 7f40466bd700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4040149ef0 con 0x7f40400cde00 2023-11-08T19:29:22.569 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.554+0000 7f40466bd700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f404006e380 con 0x7f4040157e30 2023-11-08T19:29:22.569 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.554+0000 7f40466bd700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40400c7d90 con 0x7f40400ce280 2023-11-08T19:29:22.569 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.554+0000 7f403ffff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f40400cde00 0x7f40400c9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:22.569 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.554+0000 7f403ffff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f40400cde00 0x7f40400c9210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55614/0 (socket says 172.21.15.143:55614) 2023-11-08T19:29:22.570 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.554+0000 7f403ffff700 1 -- 172.21.15.143:0/2528354670 learned_addr learned my addr 172.21.15.143:0/2528354670 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:29:22.570 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.554+0000 7f403f7fe700 1 --2- 172.21.15.143:0/2528354670 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f40400ce280 0x7f40400c7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:22.570 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.554+0000 7f4044c5a700 1 --2- 172.21.15.143:0/2528354670 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4040157e30 0x7f40401f77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:22.571 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.554+0000 7f403ffff700 1 -- 172.21.15.143:0/2528354670 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f40400ce280 msgr2=0x7f40400c7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:22.571 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.554+0000 7f403ffff700 1 --2- 172.21.15.143:0/2528354670 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f40400ce280 0x7f40400c7220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:22.571 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.554+0000 7f403ffff700 1 -- 172.21.15.143:0/2528354670 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4040157e30 msgr2=0x7f40401f77a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:22.572 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.554+0000 7f403ffff700 1 --2- 172.21.15.143:0/2528354670 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4040157e30 0x7f40401f77a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:22.572 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.554+0000 7f403ffff700 1 -- 172.21.15.143:0/2528354670 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4038009b30 con 0x7f40400cde00 2023-11-08T19:29:22.573 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.554+0000 7f403ffff700 1 --2- 172.21.15.143:0/2528354670 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f40400cde00 0x7f40400c9210 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f403400c7f0 tx=0x7f403400cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:22.573 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.554+0000 7f403cff9700 1 -- 172.21.15.143:0/2528354670 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4034004220 con 0x7f40400cde00 2023-11-08T19:29:22.573 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.554+0000 7f403cff9700 1 -- 172.21.15.143:0/2528354670 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4034004380 con 0x7f40400cde00 2023-11-08T19:29:22.574 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.554+0000 7f40466bd700 1 -- 172.21.15.143:0/2528354670 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f404006d5e0 con 0x7f40400cde00 2023-11-08T19:29:22.574 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.555+0000 7f403cff9700 1 -- 172.21.15.143:0/2528354670 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40340036a0 con 0x7f40400cde00 2023-11-08T19:29:22.574 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.555+0000 7f40466bd700 1 -- 172.21.15.143:0/2528354670 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f404006dba0 con 0x7f40400cde00 2023-11-08T19:29:22.575 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.555+0000 7f40466bd700 1 -- 172.21.15.143:0/2528354670 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f404006e1f0 con 0x7f40400cde00 2023-11-08T19:29:22.575 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.556+0000 7f403cff9700 1 -- 172.21.15.143:0/2528354670 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4034003800 con 0x7f40400cde00 2023-11-08T19:29:22.576 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.556+0000 7f403cff9700 1 --2- 172.21.15.143:0/2528354670 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f402c06af00 0x7f402c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:22.576 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.556+0000 7f403cff9700 1 -- 172.21.15.143:0/2528354670 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6973+0+0 (secure 0 0 0) 0x7f40340894a0 con 0x7f40400cde00 2023-11-08T19:29:22.576 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.556+0000 7f403f7fe700 1 --2- 172.21.15.143:0/2528354670 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f402c06af00 0x7f402c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:22.577 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.557+0000 7f403f7fe700 1 --2- 172.21.15.143:0/2528354670 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f402c06af00 0x7f402c06d3c0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f404014bec0 tx=0x7f4030008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:22.577 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.561+0000 7f403cff9700 1 -- 172.21.15.143:0/2528354670 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f40340540f0 con 0x7f40400cde00 2023-11-08T19:29:22.799 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.798+0000 7f40466bd700 1 -- 172.21.15.143:0/2528354670 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f404004eee0 con 0x7f40400cde00 2023-11-08T19:29:22.800 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.798+0000 7f403cff9700 1 -- 172.21.15.143:0/2528354670 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f4034057710 con 0x7f40400cde00 2023-11-08T19:29:22.800 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:29:22.800 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":72,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5933,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/3918286710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6835","nonce":3918286710},{"type":"v1","addr":"172.21.15.174:6837","nonce":3918286710}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":7386,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3664584522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":3664584522},{"type":"v1","addr":"172.21.15.143:6835","nonce":3664584522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:03.092245+0000","modified":"2023-11-08T19:29:17.961359+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":96,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6906,"mds_1":6918},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6906":{"gid":6906,"name":"c","rank":0,"incarnation":68,"state":"up:active","state_seq":24,"addr":"172.21.15.143:6834/3172664377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":3172664377},{"type":"v1","addr":"172.21.15.143:6834","nonce":3172664377}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6918":{"gid":6918,"name":"b","rank":1,"incarnation":54,"state":"up:active","state_seq":6,"addr":"172.21.15.174:6836/3601088618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3601088618},{"type":"v1","addr":"172.21.15.174:6836","nonce":3601088618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:29:22.802 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.801+0000 7f40466bd700 1 -- 172.21.15.143:0/2528354670 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f402c06af00 msgr2=0x7f402c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:22.802 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.801+0000 7f40466bd700 1 --2- 172.21.15.143:0/2528354670 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f402c06af00 0x7f402c06d3c0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f404014bec0 tx=0x7f4030008040 comp rx=0 tx=0).stop 2023-11-08T19:29:22.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.801+0000 7f40466bd700 1 -- 172.21.15.143:0/2528354670 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f40400cde00 msgr2=0x7f40400c9210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:22.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.801+0000 7f40466bd700 1 --2- 172.21.15.143:0/2528354670 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f40400cde00 0x7f40400c9210 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f403400c7f0 tx=0x7f403400cb00 comp rx=0 tx=0).stop 2023-11-08T19:29:22.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.801+0000 7f40466bd700 1 -- 172.21.15.143:0/2528354670 shutdown_connections 2023-11-08T19:29:22.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.801+0000 7f40466bd700 1 --2- 172.21.15.143:0/2528354670 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f402c06af00 0x7f402c06d3c0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:22.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.801+0000 7f40466bd700 1 --2- 172.21.15.143:0/2528354670 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4040157e30 0x7f40401f77a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:22.804 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.801+0000 7f40466bd700 1 --2- 172.21.15.143:0/2528354670 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f40400cde00 0x7f40400c9210 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:22.804 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.801+0000 7f40466bd700 1 --2- 172.21.15.143:0/2528354670 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f40400ce280 0x7f40400c7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:22.804 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.801+0000 7f40466bd700 1 -- 172.21.15.143:0/2528354670 >> 172.21.15.143:0/2528354670 conn(0x7f404006b4c0 msgr2=0x7f404014c050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:22.804 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.801+0000 7f40466bd700 1 -- 172.21.15.143:0/2528354670 shutdown_connections 2023-11-08T19:29:22.804 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.801+0000 7f40466bd700 1 -- 172.21.15.143:0/2528354670 wait complete. 2023-11-08T19:29:22.805 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 72 2023-11-08T19:29:22.815 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds freeze 6:0 true 2023-11-08T19:29:22.937 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.935+0000 7efc97c98700 1 Processor -- start 2023-11-08T19:29:22.937 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.936+0000 7efc97c98700 1 -- start start 2023-11-08T19:29:22.940 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.936+0000 7efc97c98700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7efc9006c0c0 0x7efc900ccfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:22.940 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.936+0000 7efc97c98700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efc9015c9b0 0x7efc90071900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:22.940 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.936+0000 7efc97c98700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc900ced30 con 0x7efc9015ce30 2023-11-08T19:29:22.940 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.936+0000 7efc97c98700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc900cd4f0 con 0x7efc9006c0c0 2023-11-08T19:29:22.941 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.936+0000 7efc97c98700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc900cdf60 con 0x7efc9015c9b0 2023-11-08T19:29:22.941 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.937+0000 7efc95233700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efc9015c9b0 0x7efc90071900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:22.941 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.937+0000 7efc95a34700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7efc9006c0c0 0x7efc900ccfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:22.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.937+0000 7efc95a34700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7efc9006c0c0 0x7efc900ccfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.143:39120/0 (socket says 172.21.15.143:39120) 2023-11-08T19:29:22.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.937+0000 7efc95233700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efc9015c9b0 0x7efc90071900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.143:43260/0 (socket says 172.21.15.143:43260) 2023-11-08T19:29:22.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.937+0000 7efc95a34700 1 -- 172.21.15.143:0/3502531812 learned_addr learned my addr 172.21.15.143:0/3502531812 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:29:22.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.937+0000 7efc96235700 1 --1- 172.21.15.143:0/3502531812 >> v1:172.21.15.174:6789/0 conn(0x7efc9015ce30 0x7efc901522a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.174:6789/0 says I am v1:172.21.15.143:34480/0 (socket says 172.21.15.143:34480) 2023-11-08T19:29:22.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.937+0000 7efc95233700 1 -- 172.21.15.143:0/3502531812 >> v2:172.21.15.174:3300/0 conn(0x7efc9006c0c0 msgr2=0x7efc900ccfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:22.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.937+0000 7efc95233700 1 --2- 172.21.15.143:0/3502531812 >> v2:172.21.15.174:3300/0 conn(0x7efc9006c0c0 0x7efc900ccfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:22.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.937+0000 7efc95233700 1 -- 172.21.15.143:0/3502531812 >> v1:172.21.15.174:6789/0 conn(0x7efc9015ce30 legacy=0x7efc901522a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:22.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.937+0000 7efc95233700 1 -- 172.21.15.143:0/3502531812 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc900cea50 con 0x7efc9015c9b0 2023-11-08T19:29:22.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.937+0000 7efc95a34700 1 --2- 172.21.15.143:0/3502531812 >> v2:172.21.15.174:3300/0 conn(0x7efc9006c0c0 0x7efc900ccfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T19:29:22.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.937+0000 7efc95233700 1 --2- 172.21.15.143:0/3502531812 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efc9015c9b0 0x7efc90071900 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7efc8c009840 tx=0x7efc8c009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=209caaf518c56e92 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:22.944 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.937+0000 7efc94a32700 1 -- 172.21.15.143:0/3502531812 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc8c011830 con 0x7efc9015c9b0 2023-11-08T19:29:22.944 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.938+0000 7efc94a32700 1 -- 172.21.15.143:0/3502531812 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc8c011990 con 0x7efc9015c9b0 2023-11-08T19:29:22.944 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.938+0000 7efc94a32700 1 -- 172.21.15.143:0/3502531812 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc8c011d20 con 0x7efc9015c9b0 2023-11-08T19:29:22.944 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.938+0000 7efc97c98700 1 -- 172.21.15.143:0/3502531812 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efc9015c9b0 msgr2=0x7efc90071900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:22.945 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.938+0000 7efc97c98700 1 --2- 172.21.15.143:0/3502531812 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efc9015c9b0 0x7efc90071900 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7efc8c009840 tx=0x7efc8c009b50 comp rx=0 tx=0).stop 2023-11-08T19:29:22.946 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.938+0000 7efc97c98700 1 -- 172.21.15.143:0/3502531812 shutdown_connections 2023-11-08T19:29:22.946 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.938+0000 7efc97c98700 1 --2- 172.21.15.143:0/3502531812 >> v2:172.21.15.174:3300/0 conn(0x7efc9006c0c0 0x7efc900ccfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:22.947 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.938+0000 7efc97c98700 1 --2- 172.21.15.143:0/3502531812 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efc9015c9b0 0x7efc90071900 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:22.947 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.938+0000 7efc97c98700 1 -- 172.21.15.143:0/3502531812 >> 172.21.15.143:0/3502531812 conn(0x7efc900b1c70 msgr2=0x7efc900b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:22.947 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.938+0000 7efc97c98700 1 -- 172.21.15.143:0/3502531812 shutdown_connections 2023-11-08T19:29:22.948 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.938+0000 7efc97c98700 1 -- 172.21.15.143:0/3502531812 wait complete. 2023-11-08T19:29:22.950 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.939+0000 7efc97c98700 1 Processor -- start 2023-11-08T19:29:22.950 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.939+0000 7efc97c98700 1 -- start start 2023-11-08T19:29:22.951 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.940+0000 7efc97c98700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efc9006c0c0 0x7efc901f0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:22.951 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.940+0000 7efc97c98700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efc9015c9b0 0x7efc900c7840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:22.951 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.940+0000 7efc97c98700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efc9015ce30 0x7efc9015dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:22.951 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.940+0000 7efc97c98700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc900c7420 con 0x7efc9015ce30 2023-11-08T19:29:22.952 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.940+0000 7efc97c98700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc900c7110 con 0x7efc9015c9b0 2023-11-08T19:29:22.952 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.940+0000 7efc97c98700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc901f13e0 con 0x7efc9006c0c0 2023-11-08T19:29:22.952 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.940+0000 7efc96235700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efc9015ce30 0x7efc9015dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:22.952 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.940+0000 7efc96235700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efc9015ce30 0x7efc9015dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55630/0 (socket says 172.21.15.143:55630) 2023-11-08T19:29:22.952 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.940+0000 7efc96235700 1 -- 172.21.15.143:0/33020344 learned_addr learned my addr 172.21.15.143:0/33020344 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:29:22.953 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.940+0000 7efc95a34700 1 --2- 172.21.15.143:0/33020344 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efc9006c0c0 0x7efc901f0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:22.953 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.940+0000 7efc95233700 1 --2- 172.21.15.143:0/33020344 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efc9015c9b0 0x7efc900c7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:22.953 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.940+0000 7efc96235700 1 -- 172.21.15.143:0/33020344 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efc9006c0c0 msgr2=0x7efc901f0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:22.953 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.940+0000 7efc96235700 1 --2- 172.21.15.143:0/33020344 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efc9006c0c0 0x7efc901f0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:22.953 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.940+0000 7efc96235700 1 -- 172.21.15.143:0/33020344 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efc9015c9b0 msgr2=0x7efc900c7840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:22.954 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.940+0000 7efc96235700 1 --2- 172.21.15.143:0/33020344 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efc9015c9b0 0x7efc900c7840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:22.954 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.940+0000 7efc96235700 1 -- 172.21.15.143:0/33020344 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc80009550 con 0x7efc9015ce30 2023-11-08T19:29:22.954 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.941+0000 7efc96235700 1 --2- 172.21.15.143:0/33020344 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efc9015ce30 0x7efc9015dd40 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7efc8800cad0 tx=0x7efc8800ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:22.954 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.941+0000 7efc867fc700 1 -- 172.21.15.143:0/33020344 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc8800aac0 con 0x7efc9015ce30 2023-11-08T19:29:22.955 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.941+0000 7efc867fc700 1 -- 172.21.15.143:0/33020344 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc8800ac20 con 0x7efc9015ce30 2023-11-08T19:29:22.955 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.941+0000 7efc97c98700 1 -- 172.21.15.143:0/33020344 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc8c0094a0 con 0x7efc9015ce30 2023-11-08T19:29:22.955 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.941+0000 7efc867fc700 1 -- 172.21.15.143:0/33020344 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc88007070 con 0x7efc9015ce30 2023-11-08T19:29:22.955 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.941+0000 7efc97c98700 1 -- 172.21.15.143:0/33020344 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc901512c0 con 0x7efc9015ce30 2023-11-08T19:29:22.955 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.942+0000 7efc97c98700 1 -- 172.21.15.143:0/33020344 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc900c7560 con 0x7efc9015ce30 2023-11-08T19:29:22.956 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.942+0000 7efc867fc700 1 -- 172.21.15.143:0/33020344 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7efc880071d0 con 0x7efc9015ce30 2023-11-08T19:29:22.956 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.942+0000 7efc867fc700 1 --2- 172.21.15.143:0/33020344 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7efc7406af00 0x7efc7406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:22.956 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.943+0000 7efc867fc700 1 -- 172.21.15.143:0/33020344 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6973+0+0 (secure 0 0 0) 0x7efc880895d0 con 0x7efc9015ce30 2023-11-08T19:29:22.956 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.943+0000 7efc95a34700 1 --2- 172.21.15.143:0/33020344 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7efc7406af00 0x7efc7406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:22.956 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.943+0000 7efc95a34700 1 --2- 172.21.15.143:0/33020344 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7efc7406af00 0x7efc7406d3c0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7efc80009cf0 tx=0x7efc80008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:22.957 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:22.947+0000 7efc867fc700 1 -- 172.21.15.143:0/33020344 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7efc900c7560 con 0x7efc9015ce30 2023-11-08T19:29:23.224 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:23.222+0000 7efc97c98700 1 -- 172.21.15.143:0/33020344 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "mds freeze", "role_or_gid": "6:0", "val": "true"} v 0) v1 -- 0x7efc900c7560 con 0x7efc9015ce30 2023-11-08T19:29:23.654 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:23.653+0000 7efc867fc700 1 -- 172.21.15.143:0/33020344 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "mds freeze", "role_or_gid": "6:0", "val": "true"}]=0 freezing mds.6906 v73) v1 ==== 112+0+0 (secure 0 0 0) 0x7efc900c7560 con 0x7efc9015ce30 2023-11-08T19:29:23.657 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:23.655+0000 7efc97c98700 1 -- 172.21.15.143:0/33020344 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7efc7406af00 msgr2=0x7efc7406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:23.657 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:23.655+0000 7efc97c98700 1 --2- 172.21.15.143:0/33020344 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7efc7406af00 0x7efc7406d3c0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7efc80009cf0 tx=0x7efc80008040 comp rx=0 tx=0).stop 2023-11-08T19:29:23.658 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:23.655+0000 7efc97c98700 1 -- 172.21.15.143:0/33020344 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efc9015ce30 msgr2=0x7efc9015dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:23.658 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:23.655+0000 7efc97c98700 1 --2- 172.21.15.143:0/33020344 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efc9015ce30 0x7efc9015dd40 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7efc8800cad0 tx=0x7efc8800ce90 comp rx=0 tx=0).stop 2023-11-08T19:29:23.658 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:23.656+0000 7efc97c98700 1 -- 172.21.15.143:0/33020344 shutdown_connections 2023-11-08T19:29:23.658 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:23.656+0000 7efc97c98700 1 --2- 172.21.15.143:0/33020344 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7efc7406af00 0x7efc7406d3c0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:23.658 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:23.656+0000 7efc97c98700 1 --2- 172.21.15.143:0/33020344 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efc9006c0c0 0x7efc901f0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:23.659 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:23.656+0000 7efc97c98700 1 --2- 172.21.15.143:0/33020344 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efc9015ce30 0x7efc9015dd40 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:23.659 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:23.656+0000 7efc97c98700 1 --2- 172.21.15.143:0/33020344 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efc9015c9b0 0x7efc900c7840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:23.659 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:23.656+0000 7efc97c98700 1 -- 172.21.15.143:0/33020344 >> 172.21.15.143:0/33020344 conn(0x7efc900b1c70 msgr2=0x7efc9006b600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:23.659 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:23.656+0000 7efc97c98700 1 -- 172.21.15.143:0/33020344 shutdown_connections 2023-11-08T19:29:23.659 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:23.656+0000 7efc97c98700 1 -- 172.21.15.143:0/33020344 wait complete. 2023-11-08T19:29:23.660 INFO:teuthology.orchestra.run.smithi143.stderr:freezing mds.6906 2023-11-08T19:29:23.669 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json config set mds_kill_mdstable_at 5 2023-11-08T19:29:23.807 INFO:teuthology.orchestra.run.smithi143.stdout:{"success":"mds_kill_mdstable_at = '5' (not observed, change may require restart) "} 2023-11-08T19:29:23.808 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_kill_mdstable_at = '5' (not observed, change may require restart) " } 2023-11-08T19:29:23.808 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:29:23.809 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir d1/dir/.snap/s15) 2023-11-08T19:29:23.811 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mds_beacon_grace 2023-11-08T19:29:23.832 INFO:tasks.ceph.mds.c.smithi143.stderr:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el8/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: In function 'void MDSTableServer::handle_commit(ceph::cref_t&)' thread 7fa9cd675700 time 2023-11-08T19:29:23.832337+0000 2023-11-08T19:29:23.833 INFO:tasks.ceph.mds.c.smithi143.stderr:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el8/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: 145: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 5) 2023-11-08T19:29:23.834 INFO:tasks.ceph.mds.c.smithi143.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-08T19:29:23.834 INFO:tasks.ceph.mds.c.smithi143.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fa9d8f3df5b] 2023-11-08T19:29:23.834 INFO:tasks.ceph.mds.c.smithi143.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2ba121) [0x7fa9d8f3e121] 2023-11-08T19:29:23.834 INFO:tasks.ceph.mds.c.smithi143.stderr: 3: (MDSTableServer::handle_commit(boost::intrusive_ptr const&)+0x84c) [0x561abc0567ec] 2023-11-08T19:29:23.835 INFO:tasks.ceph.mds.c.smithi143.stderr: 4: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xe9) [0x561abbd15099] 2023-11-08T19:29:23.835 INFO:tasks.ceph.mds.c.smithi143.stderr: 5: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x593) [0x561abbd181c3] 2023-11-08T19:29:23.835 INFO:tasks.ceph.mds.c.smithi143.stderr: 6: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x561abbd187bc] 2023-11-08T19:29:23.835 INFO:tasks.ceph.mds.c.smithi143.stderr: 7: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x561abbd0256f] 2023-11-08T19:29:23.835 INFO:tasks.ceph.mds.c.smithi143.stderr: 8: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fa9d91c59c8] 2023-11-08T19:29:23.836 INFO:tasks.ceph.mds.c.smithi143.stderr: 9: (DispatchQueue::entry()+0x51f) [0x7fa9d91c2b4f] 2023-11-08T19:29:23.836 INFO:tasks.ceph.mds.c.smithi143.stderr: 10: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fa9d9290351] 2023-11-08T19:29:23.836 INFO:tasks.ceph.mds.c.smithi143.stderr: 11: /lib64/libpthread.so.0(+0x81cf) [0x7fa9d7cd61cf] 2023-11-08T19:29:23.836 INFO:tasks.ceph.mds.c.smithi143.stderr: 12: clone() 2023-11-08T19:29:23.836 INFO:tasks.ceph.mds.c.smithi143.stderr:2023-11-08T19:29:23.832+0000 7fa9cd675700 -1 /home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el8/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: In function 'void MDSTableServer::handle_commit(ceph::cref_t&)' thread 7fa9cd675700 time 2023-11-08T19:29:23.832337+0000 2023-11-08T19:29:23.837 INFO:tasks.ceph.mds.c.smithi143.stderr:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el8/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: 145: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 5) 2023-11-08T19:29:23.837 INFO:tasks.ceph.mds.c.smithi143.stderr: 2023-11-08T19:29:23.837 INFO:tasks.ceph.mds.c.smithi143.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-08T19:29:23.837 INFO:tasks.ceph.mds.c.smithi143.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fa9d8f3df5b] 2023-11-08T19:29:23.837 INFO:tasks.ceph.mds.c.smithi143.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2ba121) [0x7fa9d8f3e121] 2023-11-08T19:29:23.838 INFO:tasks.ceph.mds.c.smithi143.stderr: 3: (MDSTableServer::handle_commit(boost::intrusive_ptr const&)+0x84c) [0x561abc0567ec] 2023-11-08T19:29:23.838 INFO:tasks.ceph.mds.c.smithi143.stderr: 4: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xe9) [0x561abbd15099] 2023-11-08T19:29:23.838 INFO:tasks.ceph.mds.c.smithi143.stderr: 5: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x593) [0x561abbd181c3] 2023-11-08T19:29:23.838 INFO:tasks.ceph.mds.c.smithi143.stderr: 6: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x561abbd187bc] 2023-11-08T19:29:23.838 INFO:tasks.ceph.mds.c.smithi143.stderr: 7: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x561abbd0256f] 2023-11-08T19:29:23.839 INFO:tasks.ceph.mds.c.smithi143.stderr: 8: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fa9d91c59c8] 2023-11-08T19:29:23.839 INFO:tasks.ceph.mds.c.smithi143.stderr: 9: (DispatchQueue::entry()+0x51f) [0x7fa9d91c2b4f] 2023-11-08T19:29:23.839 INFO:tasks.ceph.mds.c.smithi143.stderr: 10: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fa9d9290351] 2023-11-08T19:29:23.839 INFO:tasks.ceph.mds.c.smithi143.stderr: 11: /lib64/libpthread.so.0(+0x81cf) [0x7fa9d7cd61cf] 2023-11-08T19:29:23.839 INFO:tasks.ceph.mds.c.smithi143.stderr: 12: clone() 2023-11-08T19:29:23.839 INFO:tasks.ceph.mds.c.smithi143.stderr: 2023-11-08T19:29:23.840 INFO:tasks.ceph.mds.c.smithi143.stderr:*** Caught signal (Aborted) ** 2023-11-08T19:29:23.840 INFO:tasks.ceph.mds.c.smithi143.stderr: in thread 7fa9cd675700 thread_name:ms_dispatch 2023-11-08T19:29:23.841 INFO:tasks.ceph.mds.c.smithi143.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-08T19:29:23.841 INFO:tasks.ceph.mds.c.smithi143.stderr: 1: /lib64/libpthread.so.0(+0x12ce0) [0x7fa9d7ce0ce0] 2023-11-08T19:29:23.841 INFO:tasks.ceph.mds.c.smithi143.stderr: 2: gsignal() 2023-11-08T19:29:23.841 INFO:tasks.ceph.mds.c.smithi143.stderr: 3: abort() 2023-11-08T19:29:23.842 INFO:tasks.ceph.mds.c.smithi143.stderr: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fa9d8f3dfb5] 2023-11-08T19:29:23.842 INFO:tasks.ceph.mds.c.smithi143.stderr: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2ba121) [0x7fa9d8f3e121] 2023-11-08T19:29:23.842 INFO:tasks.ceph.mds.c.smithi143.stderr: 6: (MDSTableServer::handle_commit(boost::intrusive_ptr const&)+0x84c) [0x561abc0567ec] 2023-11-08T19:29:23.842 INFO:tasks.ceph.mds.c.smithi143.stderr: 7: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xe9) [0x561abbd15099] 2023-11-08T19:29:23.842 INFO:tasks.ceph.mds.c.smithi143.stderr: 8: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x593) [0x561abbd181c3] 2023-11-08T19:29:23.843 INFO:tasks.ceph.mds.c.smithi143.stderr: 9: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x561abbd187bc] 2023-11-08T19:29:23.843 INFO:tasks.ceph.mds.c.smithi143.stderr: 10: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x561abbd0256f] 2023-11-08T19:29:23.843 INFO:tasks.ceph.mds.c.smithi143.stderr: 11: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fa9d91c59c8] 2023-11-08T19:29:23.843 INFO:tasks.ceph.mds.c.smithi143.stderr: 12: (DispatchQueue::entry()+0x51f) [0x7fa9d91c2b4f] 2023-11-08T19:29:23.843 INFO:tasks.ceph.mds.c.smithi143.stderr: 13: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fa9d9290351] 2023-11-08T19:29:23.844 INFO:tasks.ceph.mds.c.smithi143.stderr: 14: /lib64/libpthread.so.0(+0x81cf) [0x7fa9d7cd61cf] 2023-11-08T19:29:23.844 INFO:tasks.ceph.mds.c.smithi143.stderr: 15: clone() 2023-11-08T19:29:23.844 INFO:tasks.ceph.mds.c.smithi143.stderr:2023-11-08T19:29:23.833+0000 7fa9cd675700 -1 *** Caught signal (Aborted) ** 2023-11-08T19:29:23.844 INFO:tasks.ceph.mds.c.smithi143.stderr: in thread 7fa9cd675700 thread_name:ms_dispatch 2023-11-08T19:29:23.844 INFO:tasks.ceph.mds.c.smithi143.stderr: 2023-11-08T19:29:23.845 INFO:tasks.ceph.mds.c.smithi143.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-08T19:29:23.845 INFO:tasks.ceph.mds.c.smithi143.stderr: 1: /lib64/libpthread.so.0(+0x12ce0) [0x7fa9d7ce0ce0] 2023-11-08T19:29:23.845 INFO:tasks.ceph.mds.c.smithi143.stderr: 2: gsignal() 2023-11-08T19:29:23.845 INFO:tasks.ceph.mds.c.smithi143.stderr: 3: abort() 2023-11-08T19:29:23.845 INFO:tasks.ceph.mds.c.smithi143.stderr: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fa9d8f3dfb5] 2023-11-08T19:29:23.846 INFO:tasks.ceph.mds.c.smithi143.stderr: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2ba121) [0x7fa9d8f3e121] 2023-11-08T19:29:23.846 INFO:tasks.ceph.mds.c.smithi143.stderr: 6: (MDSTableServer::handle_commit(boost::intrusive_ptr const&)+0x84c) [0x561abc0567ec] 2023-11-08T19:29:23.846 INFO:tasks.ceph.mds.c.smithi143.stderr: 7: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xe9) [0x561abbd15099] 2023-11-08T19:29:23.846 INFO:tasks.ceph.mds.c.smithi143.stderr: 8: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x593) [0x561abbd181c3] 2023-11-08T19:29:23.846 INFO:tasks.ceph.mds.c.smithi143.stderr: 9: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x561abbd187bc] 2023-11-08T19:29:23.847 INFO:tasks.ceph.mds.c.smithi143.stderr: 10: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x561abbd0256f] 2023-11-08T19:29:23.847 INFO:tasks.ceph.mds.c.smithi143.stderr: 11: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fa9d91c59c8] 2023-11-08T19:29:23.847 INFO:tasks.ceph.mds.c.smithi143.stderr: 12: (DispatchQueue::entry()+0x51f) [0x7fa9d91c2b4f] 2023-11-08T19:29:23.847 INFO:tasks.ceph.mds.c.smithi143.stderr: 13: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fa9d9290351] 2023-11-08T19:29:23.847 INFO:tasks.ceph.mds.c.smithi143.stderr: 14: /lib64/libpthread.so.0(+0x81cf) [0x7fa9d7cd61cf] 2023-11-08T19:29:23.848 INFO:tasks.ceph.mds.c.smithi143.stderr: 15: clone() 2023-11-08T19:29:23.848 INFO:tasks.ceph.mds.c.smithi143.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-11-08T19:29:23.848 INFO:tasks.ceph.mds.c.smithi143.stderr: 2023-11-08T19:29:23.848 INFO:tasks.ceph.mds.c.smithi143.stderr: -538> 2023-11-08T19:29:19.686+0000 7fa9cae70700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-08T19:29:23.848 INFO:tasks.ceph.mds.c.smithi143.stderr: -536> 2023-11-08T19:29:19.686+0000 7fa9cae70700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-11-08T19:29:23.848 INFO:tasks.ceph.mds.c.smithi143.stderr: -1> 2023-11-08T19:29:23.832+0000 7fa9cd675700 -1 /home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el8/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: In function 'void MDSTableServer::handle_commit(ceph::cref_t&)' thread 7fa9cd675700 time 2023-11-08T19:29:23.832337+0000 2023-11-08T19:29:23.849 INFO:tasks.ceph.mds.c.smithi143.stderr:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el8/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: 145: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 5) 2023-11-08T19:29:23.849 INFO:tasks.ceph.mds.c.smithi143.stderr: 2023-11-08T19:29:23.849 INFO:tasks.ceph.mds.c.smithi143.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-08T19:29:23.849 INFO:tasks.ceph.mds.c.smithi143.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fa9d8f3df5b] 2023-11-08T19:29:23.849 INFO:tasks.ceph.mds.c.smithi143.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2ba121) [0x7fa9d8f3e121] 2023-11-08T19:29:23.850 INFO:tasks.ceph.mds.c.smithi143.stderr: 3: (MDSTableServer::handle_commit(boost::intrusive_ptr const&)+0x84c) [0x561abc0567ec] 2023-11-08T19:29:23.850 INFO:tasks.ceph.mds.c.smithi143.stderr: 4: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xe9) [0x561abbd15099] 2023-11-08T19:29:23.850 INFO:tasks.ceph.mds.c.smithi143.stderr: 5: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x593) [0x561abbd181c3] 2023-11-08T19:29:23.850 INFO:tasks.ceph.mds.c.smithi143.stderr: 6: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x561abbd187bc] 2023-11-08T19:29:23.850 INFO:tasks.ceph.mds.c.smithi143.stderr: 7: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x561abbd0256f] 2023-11-08T19:29:23.851 INFO:tasks.ceph.mds.c.smithi143.stderr: 8: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fa9d91c59c8] 2023-11-08T19:29:23.851 INFO:tasks.ceph.mds.c.smithi143.stderr: 9: (DispatchQueue::entry()+0x51f) [0x7fa9d91c2b4f] 2023-11-08T19:29:23.851 INFO:tasks.ceph.mds.c.smithi143.stderr: 10: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fa9d9290351] 2023-11-08T19:29:23.851 INFO:tasks.ceph.mds.c.smithi143.stderr: 11: /lib64/libpthread.so.0(+0x81cf) [0x7fa9d7cd61cf] 2023-11-08T19:29:23.851 INFO:tasks.ceph.mds.c.smithi143.stderr: 12: clone() 2023-11-08T19:29:23.852 INFO:tasks.ceph.mds.c.smithi143.stderr: 2023-11-08T19:29:23.852 INFO:tasks.ceph.mds.c.smithi143.stderr: 0> 2023-11-08T19:29:23.833+0000 7fa9cd675700 -1 *** Caught signal (Aborted) ** 2023-11-08T19:29:23.852 INFO:tasks.ceph.mds.c.smithi143.stderr: in thread 7fa9cd675700 thread_name:ms_dispatch 2023-11-08T19:29:23.852 INFO:tasks.ceph.mds.c.smithi143.stderr: 2023-11-08T19:29:23.852 INFO:tasks.ceph.mds.c.smithi143.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-08T19:29:23.852 INFO:tasks.ceph.mds.c.smithi143.stderr: 1: /lib64/libpthread.so.0(+0x12ce0) [0x7fa9d7ce0ce0] 2023-11-08T19:29:23.853 INFO:tasks.ceph.mds.c.smithi143.stderr: 2: gsignal() 2023-11-08T19:29:23.853 INFO:tasks.ceph.mds.c.smithi143.stderr: 3: abort() 2023-11-08T19:29:23.853 INFO:tasks.ceph.mds.c.smithi143.stderr: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fa9d8f3dfb5] 2023-11-08T19:29:23.853 INFO:tasks.ceph.mds.c.smithi143.stderr: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2ba121) [0x7fa9d8f3e121] 2023-11-08T19:29:23.853 INFO:tasks.ceph.mds.c.smithi143.stderr: 6: (MDSTableServer::handle_commit(boost::intrusive_ptr const&)+0x84c) [0x561abc0567ec] 2023-11-08T19:29:23.854 INFO:tasks.ceph.mds.c.smithi143.stderr: 7: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xe9) [0x561abbd15099] 2023-11-08T19:29:23.854 INFO:tasks.ceph.mds.c.smithi143.stderr: 8: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x593) [0x561abbd181c3] 2023-11-08T19:29:23.854 INFO:tasks.ceph.mds.c.smithi143.stderr: 9: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x561abbd187bc] 2023-11-08T19:29:23.854 INFO:tasks.ceph.mds.c.smithi143.stderr: 10: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x561abbd0256f] 2023-11-08T19:29:23.854 INFO:tasks.ceph.mds.c.smithi143.stderr: 11: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fa9d91c59c8] 2023-11-08T19:29:23.855 INFO:tasks.ceph.mds.c.smithi143.stderr: 12: (DispatchQueue::entry()+0x51f) [0x7fa9d91c2b4f] 2023-11-08T19:29:23.855 INFO:tasks.ceph.mds.c.smithi143.stderr: 13: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fa9d9290351] 2023-11-08T19:29:23.855 INFO:tasks.ceph.mds.c.smithi143.stderr: 14: /lib64/libpthread.so.0(+0x81cf) [0x7fa9d7cd61cf] 2023-11-08T19:29:23.855 INFO:tasks.ceph.mds.c.smithi143.stderr: 15: clone() 2023-11-08T19:29:23.855 INFO:tasks.ceph.mds.c.smithi143.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-11-08T19:29:23.856 INFO:tasks.ceph.mds.c.smithi143.stderr: 2023-11-08T19:29:23.856 INFO:tasks.ceph.mds.c.smithi143.stderr: -538> 2023-11-08T19:29:19.686+0000 7fa9cae70700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-08T19:29:23.857 INFO:tasks.ceph.mds.c.smithi143.stderr: -536> 2023-11-08T19:29:19.686+0000 7fa9cae70700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-11-08T19:29:23.857 INFO:tasks.ceph.mds.c.smithi143.stderr: -1> 2023-11-08T19:29:23.832+0000 7fa9cd675700 -1 /home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el8/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: In function 'void MDSTableServer::handle_commit(ceph::cref_t&)' thread 7fa9cd675700 time 2023-11-08T19:29:23.832337+0000 2023-11-08T19:29:23.857 INFO:tasks.ceph.mds.c.smithi143.stderr:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el8/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: 145: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 5) 2023-11-08T19:29:23.857 INFO:tasks.ceph.mds.c.smithi143.stderr: 2023-11-08T19:29:23.858 INFO:tasks.ceph.mds.c.smithi143.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-08T19:29:23.858 INFO:tasks.ceph.mds.c.smithi143.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fa9d8f3df5b] 2023-11-08T19:29:23.858 INFO:tasks.ceph.mds.c.smithi143.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2ba121) [0x7fa9d8f3e121] 2023-11-08T19:29:23.858 INFO:tasks.ceph.mds.c.smithi143.stderr: 3: (MDSTableServer::handle_commit(boost::intrusive_ptr const&)+0x84c) [0x561abc0567ec] 2023-11-08T19:29:23.858 INFO:tasks.ceph.mds.c.smithi143.stderr: 4: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xe9) [0x561abbd15099] 2023-11-08T19:29:23.859 INFO:tasks.ceph.mds.c.smithi143.stderr: 5: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x593) [0x561abbd181c3] 2023-11-08T19:29:23.859 INFO:tasks.ceph.mds.c.smithi143.stderr: 6: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x561abbd187bc] 2023-11-08T19:29:23.859 INFO:tasks.ceph.mds.c.smithi143.stderr: 7: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x561abbd0256f] 2023-11-08T19:29:23.859 INFO:tasks.ceph.mds.c.smithi143.stderr: 8: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fa9d91c59c8] 2023-11-08T19:29:23.859 INFO:tasks.ceph.mds.c.smithi143.stderr: 9: (DispatchQueue::entry()+0x51f) [0x7fa9d91c2b4f] 2023-11-08T19:29:23.859 INFO:tasks.ceph.mds.c.smithi143.stderr: 10: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fa9d9290351] 2023-11-08T19:29:23.860 INFO:tasks.ceph.mds.c.smithi143.stderr: 11: /lib64/libpthread.so.0(+0x81cf) [0x7fa9d7cd61cf] 2023-11-08T19:29:23.860 INFO:tasks.ceph.mds.c.smithi143.stderr: 12: clone() 2023-11-08T19:29:23.860 INFO:tasks.ceph.mds.c.smithi143.stderr: 2023-11-08T19:29:23.860 INFO:tasks.ceph.mds.c.smithi143.stderr: 0> 2023-11-08T19:29:23.833+0000 7fa9cd675700 -1 *** Caught signal (Aborted) ** 2023-11-08T19:29:23.860 INFO:tasks.ceph.mds.c.smithi143.stderr: in thread 7fa9cd675700 thread_name:ms_dispatch 2023-11-08T19:29:23.861 INFO:tasks.ceph.mds.c.smithi143.stderr: 2023-11-08T19:29:23.861 INFO:tasks.ceph.mds.c.smithi143.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-08T19:29:23.861 INFO:tasks.ceph.mds.c.smithi143.stderr: 1: /lib64/libpthread.so.0(+0x12ce0) [0x7fa9d7ce0ce0] 2023-11-08T19:29:23.861 INFO:tasks.ceph.mds.c.smithi143.stderr: 2: gsignal() 2023-11-08T19:29:23.861 INFO:tasks.ceph.mds.c.smithi143.stderr: 3: abort() 2023-11-08T19:29:23.862 INFO:tasks.ceph.mds.c.smithi143.stderr: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fa9d8f3dfb5] 2023-11-08T19:29:23.862 INFO:tasks.ceph.mds.c.smithi143.stderr: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2ba121) [0x7fa9d8f3e121] 2023-11-08T19:29:23.862 INFO:tasks.ceph.mds.c.smithi143.stderr: 6: (MDSTableServer::handle_commit(boost::intrusive_ptr const&)+0x84c) [0x561abc0567ec] 2023-11-08T19:29:23.862 INFO:tasks.ceph.mds.c.smithi143.stderr: 7: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xe9) [0x561abbd15099] 2023-11-08T19:29:23.862 INFO:tasks.ceph.mds.c.smithi143.stderr: 8: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x593) [0x561abbd181c3] 2023-11-08T19:29:23.863 INFO:tasks.ceph.mds.c.smithi143.stderr: 9: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x561abbd187bc] 2023-11-08T19:29:23.863 INFO:tasks.ceph.mds.c.smithi143.stderr: 10: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bf) [0x561abbd0256f] 2023-11-08T19:29:23.863 INFO:tasks.ceph.mds.c.smithi143.stderr: 11: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fa9d91c59c8] 2023-11-08T19:29:23.863 INFO:tasks.ceph.mds.c.smithi143.stderr: 12: (DispatchQueue::entry()+0x51f) [0x7fa9d91c2b4f] 2023-11-08T19:29:23.863 INFO:tasks.ceph.mds.c.smithi143.stderr: 13: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fa9d9290351] 2023-11-08T19:29:23.863 INFO:tasks.ceph.mds.c.smithi143.stderr: 14: /lib64/libpthread.so.0(+0x81cf) [0x7fa9d7cd61cf] 2023-11-08T19:29:23.864 INFO:tasks.ceph.mds.c.smithi143.stderr: 15: clone() 2023-11-08T19:29:23.864 INFO:tasks.ceph.mds.c.smithi143.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-11-08T19:29:23.864 INFO:tasks.ceph.mds.c.smithi143.stderr: 2023-11-08T19:29:23.973 INFO:teuthology.orchestra.run.smithi143.stdout:{"mds_beacon_grace":"15.000000"} 2023-11-08T19:29:23.974 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_beacon_grace": "15.000000" } 2023-11-08T19:29:23.974 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:29:24.083 INFO:tasks.ceph.mds.c.smithi143.stderr:daemon-helper: command crashed with signal 6 2023-11-08T19:29:24.096 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:24.094+0000 7f0edd683700 1 Processor -- start 2023-11-08T19:29:24.096 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:24.095+0000 7f0edd683700 1 -- start start 2023-11-08T19:29:24.099 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:24.095+0000 7f0edd683700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f0ed815ba30 0x7f0ed80b14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:24.099 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:24.095+0000 7f0edd683700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f0ed80d03a0 0x7f0ed80d1560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:24.099 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:24.095+0000 7f0edd683700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0ed80d1aa0 0x7f0ed80ce2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:24.100 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:24.095+0000 7f0edd683700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ed80d1f10 con 0x7f0ed80d03a0 2023-11-08T19:29:24.100 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:24.095+0000 7f0edd683700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ed80b1bc0 con 0x7f0ed815ba30 2023-11-08T19:29:24.100 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:24.095+0000 7f0edd683700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed80b1ea0 con 0x7f0ed80d1aa0 2023-11-08T19:29:24.100 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:24.095+0000 7f0ed67fc700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f0ed80d03a0 0x7f0ed80d1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:24.100 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:24.095+0000 7f0ed67fc700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f0ed80d03a0 0x7f0ed80d1560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55636/0 (socket says 172.21.15.143:55636) 2023-11-08T19:29:24.101 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:24.095+0000 7f0ed67fc700 1 -- 172.21.15.143:0/2940711520 learned_addr learned my addr 172.21.15.143:0/2940711520 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:29:24.101 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:24.096+0000 7f0ed77fe700 1 --2- 172.21.15.143:0/2940711520 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0ed80d1aa0 0x7f0ed80ce2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:24.101 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:24.096+0000 7f0ed6ffd700 1 --2- 172.21.15.143:0/2940711520 >> v2:172.21.15.174:3300/0 conn(0x7f0ed815ba30 0x7f0ed80b14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:24.101 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:24.096+0000 7f0ed67fc700 1 -- 172.21.15.143:0/2940711520 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0ed80d1aa0 msgr2=0x7f0ed80ce2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:24.101 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:24.096+0000 7f0ed67fc700 1 --2- 172.21.15.143:0/2940711520 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0ed80d1aa0 0x7f0ed80ce2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:24.102 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:24.096+0000 7f0ed67fc700 1 -- 172.21.15.143:0/2940711520 >> v2:172.21.15.174:3300/0 conn(0x7f0ed815ba30 msgr2=0x7f0ed80b14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:24.102 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:24.096+0000 7f0ed67fc700 1 --2- 172.21.15.143:0/2940711520 >> v2:172.21.15.174:3300/0 conn(0x7f0ed815ba30 0x7f0ed80b14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:24.102 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:24.096+0000 7f0ed67fc700 1 -- 172.21.15.143:0/2940711520 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ed80b2180 con 0x7f0ed80d03a0 2023-11-08T19:29:24.102 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:24.096+0000 7f0ed67fc700 1 --2- 172.21.15.143:0/2940711520 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0ed80d03a0 0x7f0ed80d1560 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f0ed80cf3d0 tx=0x7f0ec8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6467f6203901347c server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:24.102 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:24.096+0000 7f0ed5ffb700 1 -- 172.21.15.143:0/2940711520 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec8011830 con 0x7f0ed80d03a0 2023-11-08T19:29:24.103 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:24.096+0000 7f0ed5ffb700 1 -- 172.21.15.143:0/2940711520 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ec8011990 con 0x7f0ed80d03a0 2023-11-08T19:29:24.103 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:24.096+0000 7f0ed5ffb700 1 -- 172.21.15.143:0/2940711520 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec8011d20 con 0x7f0ed80d03a0 2023-11-08T19:29:24.103 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:24.097+0000 7f0edd683700 1 -- 172.21.15.143:0/2940711520 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0ed80d03a0 msgr2=0x7f0ed80d1560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:24.103 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:24.097+0000 7f0edd683700 1 --2- 172.21.15.143:0/2940711520 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0ed80d03a0 0x7f0ed80d1560 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f0ed80cf3d0 tx=0x7f0ec8009bd0 comp rx=0 tx=0).stop 2023-11-08T19:29:24.103 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:24.097+0000 7f0edd683700 1 -- 172.21.15.143:0/2940711520 shutdown_connections 2023-11-08T19:29:24.104 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:24.097+0000 7f0edd683700 1 --2- 172.21.15.143:0/2940711520 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0ed80d1aa0 0x7f0ed80ce2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:24.104 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:24.097+0000 7f0edd683700 1 --2- 172.21.15.143:0/2940711520 >> v2:172.21.15.174:3300/0 conn(0x7f0ed815ba30 0x7f0ed80b14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:24.104 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:24.097+0000 7f0edd683700 1 --2- 172.21.15.143:0/2940711520 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0ed80d03a0 0x7f0ed80d1560 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:24.104 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:24.097+0000 7f0edd683700 1 -- 172.21.15.143:0/2940711520 >> 172.21.15.143:0/2940711520 conn(0x7f0ed808b000 msgr2=0x7f0ed809ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:24.104 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:24.097+0000 7f0edd683700 1 -- 172.21.15.143:0/2940711520 shutdown_connections 2023-11-08T19:29:24.105 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:24.097+0000 7f0edd683700 1 -- 172.21.15.143:0/2940711520 wait complete. 2023-11-08T19:29:24.105 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:24.098+0000 7f0edd683700 1 Processor -- start 2023-11-08T19:29:24.105 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:24.098+0000 7f0edd683700 1 -- start start 2023-11-08T19:29:24.108 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:24.099+0000 7f0edd683700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0ed80d03a0 0x7f0ed81f0a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:24.108 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:24.099+0000 7f0edd683700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0ed80d1aa0 0x7f0ed80b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:24.108 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:24.099+0000 7f0edd683700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0ed815ba30 0x7f0ed815c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:24.109 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:24.099+0000 7f0edd683700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed80c9850 con 0x7f0ed80d1aa0 2023-11-08T19:29:24.109 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:24.099+0000 7f0edd683700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed80c8980 con 0x7f0ed80d03a0 2023-11-08T19:29:24.109 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:24.099+0000 7f0edd683700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed80b6bf0 con 0x7f0ed815ba30 2023-11-08T19:29:24.109 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:24.099+0000 7f0ed67fc700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0ed80d1aa0 0x7f0ed80b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:24.109 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:24.099+0000 7f0ed67fc700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0ed80d1aa0 0x7f0ed80b4eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55642/0 (socket says 172.21.15.143:55642) 2023-11-08T19:29:24.110 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:24.099+0000 7f0ed67fc700 1 -- 172.21.15.143:0/2712290416 learned_addr learned my addr 172.21.15.143:0/2712290416 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:29:24.110 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:24.099+0000 7f0ed77fe700 1 --2- 172.21.15.143:0/2712290416 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0ed815ba30 0x7f0ed815c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:24.110 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:24.099+0000 7f0ed6ffd700 1 --2- 172.21.15.143:0/2712290416 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0ed80d03a0 0x7f0ed81f0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:24.110 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:24.099+0000 7f0ed67fc700 1 -- 172.21.15.143:0/2712290416 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0ed815ba30 msgr2=0x7f0ed815c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:24.110 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:24.099+0000 7f0ed67fc700 1 --2- 172.21.15.143:0/2712290416 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0ed815ba30 0x7f0ed815c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:24.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:24.099+0000 7f0ed67fc700 1 -- 172.21.15.143:0/2712290416 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0ed80d03a0 msgr2=0x7f0ed81f0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:24.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:24.099+0000 7f0ed67fc700 1 --2- 172.21.15.143:0/2712290416 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0ed80d03a0 0x7f0ed81f0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:24.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:24.099+0000 7f0ed67fc700 1 -- 172.21.15.143:0/2712290416 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ec8009550 con 0x7f0ed80d1aa0 2023-11-08T19:29:24.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:24.099+0000 7f0ed77fe700 1 --2- 172.21.15.143:0/2712290416 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0ed815ba30 0x7f0ed815c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:29:24.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:24.099+0000 7f0ed67fc700 1 --2- 172.21.15.143:0/2712290416 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0ed80d1aa0 0x7f0ed80b4eb0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f0ec8000c00 tx=0x7f0ec8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:24.112 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:24.099+0000 7f0ebf7fe700 1 -- 172.21.15.143:0/2712290416 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec8004300 con 0x7f0ed80d1aa0 2023-11-08T19:29:24.112 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:24.100+0000 7f0ebf7fe700 1 -- 172.21.15.143:0/2712290416 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ec801f070 con 0x7f0ed80d1aa0 2023-11-08T19:29:24.112 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:24.100+0000 7f0edd683700 1 -- 172.21.15.143:0/2712290416 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ed80c8bd0 con 0x7f0ed80d1aa0 2023-11-08T19:29:24.112 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:24.100+0000 7f0ebf7fe700 1 -- 172.21.15.143:0/2712290416 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec80036a0 con 0x7f0ed80d1aa0 2023-11-08T19:29:24.112 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:24.100+0000 7f0edd683700 1 -- 172.21.15.143:0/2712290416 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ed80c9190 con 0x7f0ed80d1aa0 2023-11-08T19:29:24.112 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:24.100+0000 7f0edd683700 1 -- 172.21.15.143:0/2712290416 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ed804eee0 con 0x7f0ed80d1aa0 2023-11-08T19:29:24.113 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:24.101+0000 7f0ebf7fe700 1 -- 172.21.15.143:0/2712290416 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0ec8021030 con 0x7f0ed80d1aa0 2023-11-08T19:29:24.113 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:24.101+0000 7f0ebf7fe700 1 --2- 172.21.15.143:0/2712290416 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f0ec406af00 0x7f0ec406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:24.113 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:24.101+0000 7f0ebf7fe700 1 -- 172.21.15.143:0/2712290416 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6973+0+0 (secure 0 0 0) 0x7f0ec808ac90 con 0x7f0ed80d1aa0 2023-11-08T19:29:24.113 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:24.102+0000 7f0ed6ffd700 1 --2- 172.21.15.143:0/2712290416 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f0ec406af00 0x7f0ec406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:24.113 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:24.102+0000 7f0ed6ffd700 1 --2- 172.21.15.143:0/2712290416 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f0ec406af00 0x7f0ec406d3c0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc009ad0 tx=0x7f0ecc008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:24.114 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:24.106+0000 7f0ebf7fe700 1 -- 172.21.15.143:0/2712290416 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0ec80558e0 con 0x7f0ed80d1aa0 2023-11-08T19:29:24.338 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:24.337+0000 7f0edd683700 1 -- 172.21.15.143:0/2712290416 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0ed80cb790 con 0x7f0ed80d1aa0 2023-11-08T19:29:24.339 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:24.337+0000 7f0ebf7fe700 1 -- 172.21.15.143:0/2712290416 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f0ec8015020 con 0x7f0ed80d1aa0 2023-11-08T19:29:24.339 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:29:24.339 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":73,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5933,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/3918286710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6835","nonce":3918286710},{"type":"v1","addr":"172.21.15.174:6837","nonce":3918286710}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":7386,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3664584522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":3664584522},{"type":"v1","addr":"172.21.15.143:6835","nonce":3664584522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:03.092245+0000","modified":"2023-11-08T19:29:23.223822+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":96,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6906,"mds_1":6918},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6906":{"gid":6906,"name":"c","rank":0,"incarnation":68,"state":"up:active","state_seq":24,"addr":"172.21.15.143:6834/3172664377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":3172664377},{"type":"v1","addr":"172.21.15.143:6834","nonce":3172664377}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6918":{"gid":6918,"name":"b","rank":1,"incarnation":54,"state":"up:active","state_seq":6,"addr":"172.21.15.174:6836/3601088618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3601088618},{"type":"v1","addr":"172.21.15.174:6836","nonce":3601088618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:29:24.341 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:24.339+0000 7f0edd683700 1 -- 172.21.15.143:0/2712290416 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f0ec406af00 msgr2=0x7f0ec406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:24.341 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:24.339+0000 7f0edd683700 1 --2- 172.21.15.143:0/2712290416 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f0ec406af00 0x7f0ec406d3c0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc009ad0 tx=0x7f0ecc008040 comp rx=0 tx=0).stop 2023-11-08T19:29:24.341 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:24.339+0000 7f0edd683700 1 -- 172.21.15.143:0/2712290416 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0ed80d1aa0 msgr2=0x7f0ed80b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:24.342 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:24.339+0000 7f0edd683700 1 --2- 172.21.15.143:0/2712290416 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0ed80d1aa0 0x7f0ed80b4eb0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f0ec8000c00 tx=0x7f0ec8004750 comp rx=0 tx=0).stop 2023-11-08T19:29:24.342 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:24.339+0000 7f0edd683700 1 -- 172.21.15.143:0/2712290416 shutdown_connections 2023-11-08T19:29:24.342 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:24.339+0000 7f0edd683700 1 --2- 172.21.15.143:0/2712290416 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f0ec406af00 0x7f0ec406d3c0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:24.342 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:24.339+0000 7f0edd683700 1 --2- 172.21.15.143:0/2712290416 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0ed815ba30 0x7f0ed815c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:24.342 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:24.339+0000 7f0edd683700 1 --2- 172.21.15.143:0/2712290416 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0ed80d03a0 0x7f0ed81f0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:24.343 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:24.339+0000 7f0edd683700 1 --2- 172.21.15.143:0/2712290416 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0ed80d1aa0 0x7f0ed80b4eb0 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:24.343 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:24.339+0000 7f0edd683700 1 -- 172.21.15.143:0/2712290416 >> 172.21.15.143:0/2712290416 conn(0x7f0ed808b000 msgr2=0x7f0ed80ae480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:24.343 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:24.340+0000 7f0edd683700 1 -- 172.21.15.143:0/2712290416 shutdown_connections 2023-11-08T19:29:24.343 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:24.340+0000 7f0edd683700 1 -- 172.21.15.143:0/2712290416 wait complete. 2023-11-08T19:29:24.344 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 73 2023-11-08T19:29:24.353 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2023-11-08T19:29:27.483 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T19:29:27.486 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T19:29:29.355 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:29:29.488 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:29.487+0000 7f46e901d700 1 Processor -- start 2023-11-08T19:29:29.489 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:29.487+0000 7f46e901d700 1 -- start start 2023-11-08T19:29:29.493 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:29.488+0000 7f46e901d700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f46e40cc530 0x7f46e40cf510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:29.493 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:29.488+0000 7f46e901d700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46e40cf080 con 0x7f46e415aa30 2023-11-08T19:29:29.493 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:29.488+0000 7f46e901d700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46e40ce470 con 0x7f46e40b2920 2023-11-08T19:29:29.493 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:29.488+0000 7f46e901d700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46e40ce750 con 0x7f46e40cc530 2023-11-08T19:29:29.494 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:29.488+0000 7f46e2d9d700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f46e415aa30 0x7f46e40b2210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:51810/0 (socket says 172.21.15.143:51810) 2023-11-08T19:29:29.494 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:29.488+0000 7f46e2d9d700 1 -- 172.21.15.143:0/1874295986 learned_addr learned my addr 172.21.15.143:0/1874295986 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:29:29.494 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:29.488+0000 7f46e359e700 1 --2- 172.21.15.143:0/1874295986 >> v2:172.21.15.174:3300/0 conn(0x7f46e40cc530 0x7f46e40cf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:29.494 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:29.488+0000 7f46e1d9b700 1 -- 172.21.15.143:0/1874295986 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2000527657 0 0) 0x7f46e40cf080 con 0x7f46e415aa30 2023-11-08T19:29:29.495 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:29.488+0000 7f46e1d9b700 1 -- 172.21.15.143:0/1874295986 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f46cc0035b0 con 0x7f46e415aa30 2023-11-08T19:29:29.495 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:29.488+0000 7f46e1d9b700 1 -- 172.21.15.143:0/1874295986 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 653443072 0 0) 0x7f46cc0035b0 con 0x7f46e415aa30 2023-11-08T19:29:29.495 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:29.488+0000 7f46e1d9b700 1 -- 172.21.15.143:0/1874295986 >> v2:172.21.15.174:3300/0 conn(0x7f46e40cc530 msgr2=0x7f46e40cf510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:29.495 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:29.489+0000 7f46e1d9b700 1 --2- 172.21.15.143:0/1874295986 >> v2:172.21.15.174:3300/0 conn(0x7f46e40cc530 0x7f46e40cf510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:29.495 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:29.489+0000 7f46e1d9b700 1 -- 172.21.15.143:0/1874295986 >> v1:172.21.15.174:6789/0 conn(0x7f46e40b2920 legacy=0x7f46e40cbe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:29.496 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:29.489+0000 7f46e1d9b700 1 -- 172.21.15.143:0/1874295986 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46e40cea30 con 0x7f46e415aa30 2023-11-08T19:29:29.496 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:29.489+0000 7f46e1d9b700 1 -- 172.21.15.143:0/1874295986 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4147756303 0 0) 0x7f46d80017f0 con 0x7f46e415aa30 2023-11-08T19:29:29.496 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:29.489+0000 7f46e1d9b700 1 -- 172.21.15.143:0/1874295986 >> v1:172.21.15.143:6789/0 conn(0x7f46e415aa30 legacy=0x7f46e40b2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:29.496 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:29.489+0000 7f46e1d9b700 1 --2- 172.21.15.143:0/1874295986 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f46cc003a00 0x7f46cc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:29.497 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:29.489+0000 7f46e1d9b700 1 --2- 172.21.15.143:0/1874295986 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f46cc004400 0x7f46cc0048d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:29.497 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:29.489+0000 7f46e1d9b700 1 --2- 172.21.15.143:0/1874295986 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f46cc005170 0x7f46cc00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:29.497 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:29.489+0000 7f46e1d9b700 1 -- 172.21.15.143:0/1874295986 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46e40ce750 con 0x7f46cc004400 2023-11-08T19:29:29.497 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:29.489+0000 7f46e1d9b700 1 -- 172.21.15.143:0/1874295986 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46cc00c0b0 con 0x7f46cc003a00 2023-11-08T19:29:29.498 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:29.489+0000 7f46e1d9b700 1 -- 172.21.15.143:0/1874295986 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46cc00c270 con 0x7f46cc005170 2023-11-08T19:29:29.498 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:29.489+0000 7f46e259c700 1 --2- 172.21.15.143:0/1874295986 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f46cc004400 0x7f46cc0048d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:29.498 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:29.489+0000 7f46e2d9d700 1 --2- 172.21.15.143:0/1874295986 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f46cc003a00 0x7f46cc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:29.498 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:29.489+0000 7f46e359e700 1 --2- 172.21.15.143:0/1874295986 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f46cc005170 0x7f46cc00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:29.499 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:29.489+0000 7f46e259c700 1 -- 172.21.15.143:0/1874295986 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f46cc005170 msgr2=0x7f46cc00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:29.499 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:29.489+0000 7f46e259c700 1 --2- 172.21.15.143:0/1874295986 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f46cc005170 0x7f46cc00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:29.499 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:29.489+0000 7f46e259c700 1 -- 172.21.15.143:0/1874295986 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f46cc003a00 msgr2=0x7f46cc003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:29.499 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:29.489+0000 7f46e259c700 1 --2- 172.21.15.143:0/1874295986 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f46cc003a00 0x7f46cc003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:29.500 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:29.489+0000 7f46e359e700 1 -- 172.21.15.143:0/1874295986 reap_dead start 2023-11-08T19:29:29.500 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:29.489+0000 7f46e259c700 1 -- 172.21.15.143:0/1874295986 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46cc0035b0 con 0x7f46cc004400 2023-11-08T19:29:29.500 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:29.489+0000 7f46e259c700 1 --2- 172.21.15.143:0/1874295986 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f46cc004400 0x7f46cc0048d0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f46d40018b0 tx=0x7f46cc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebcff5ee14f9a4d1 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:29.500 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:29.489+0000 7f46e1d9b700 1 -- 172.21.15.143:0/1874295986 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46d4001370 con 0x7f46cc004400 2023-11-08T19:29:29.500 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:29.489+0000 7f46e1d9b700 1 -- 172.21.15.143:0/1874295986 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46d40052d0 con 0x7f46cc004400 2023-11-08T19:29:29.501 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:29.489+0000 7f46e1d9b700 1 -- 172.21.15.143:0/1874295986 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46d4003680 con 0x7f46cc004400 2023-11-08T19:29:29.501 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:29.490+0000 7f46e901d700 1 -- 172.21.15.143:0/1874295986 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f46cc004400 msgr2=0x7f46cc0048d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:29.501 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:29.490+0000 7f46e901d700 1 --2- 172.21.15.143:0/1874295986 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f46cc004400 0x7f46cc0048d0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f46d40018b0 tx=0x7f46cc005880 comp rx=0 tx=0).stop 2023-11-08T19:29:29.501 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:29.490+0000 7f46e901d700 1 -- 172.21.15.143:0/1874295986 shutdown_connections 2023-11-08T19:29:29.502 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:29.490+0000 7f46e901d700 1 --2- 172.21.15.143:0/1874295986 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f46cc004400 0x7f46cc0048d0 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:29.502 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:29.490+0000 7f46e901d700 1 -- 172.21.15.143:0/1874295986 >> 172.21.15.143:0/1874295986 conn(0x7f46e4099ab0 msgr2=0x7f46e40d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:29.502 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:29.490+0000 7f46e901d700 1 -- 172.21.15.143:0/1874295986 shutdown_connections 2023-11-08T19:29:29.502 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:29.490+0000 7f46e901d700 1 -- 172.21.15.143:0/1874295986 wait complete. 2023-11-08T19:29:29.503 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:29.492+0000 7f46e901d700 1 Processor -- start 2023-11-08T19:29:29.503 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:29.492+0000 7f46e901d700 1 -- start start 2023-11-08T19:29:29.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:29.492+0000 7f46e901d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f46cc004400 0x7f46e40c4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:29.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:29.492+0000 7f46e901d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f46e414fdb0 0x7f46e40c6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:29.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:29.492+0000 7f46e901d700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f46e415bbc0 0x7f46e40b5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:29.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:29.492+0000 7f46e901d700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46e40c6a80 con 0x7f46cc004400 2023-11-08T19:29:29.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:29.492+0000 7f46e901d700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46e4150230 con 0x7f46e414fdb0 2023-11-08T19:29:29.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:29.492+0000 7f46e901d700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46e40b3550 con 0x7f46e415bbc0 2023-11-08T19:29:29.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:29.492+0000 7f46e2d9d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f46cc004400 0x7f46e40c4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:29.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:29.493+0000 7f46e2d9d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f46cc004400 0x7f46e40c4f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55666/0 (socket says 172.21.15.143:55666) 2023-11-08T19:29:29.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:29.493+0000 7f46e2d9d700 1 -- 172.21.15.143:0/2561063260 learned_addr learned my addr 172.21.15.143:0/2561063260 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:29:29.508 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:29.493+0000 7f46e259c700 1 --2- 172.21.15.143:0/2561063260 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f46e414fdb0 0x7f46e40c6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:29.508 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:29.493+0000 7f46e359e700 1 --2- 172.21.15.143:0/2561063260 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f46e415bbc0 0x7f46e40b5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:29.508 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:29.493+0000 7f46e2d9d700 1 -- 172.21.15.143:0/2561063260 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f46e415bbc0 msgr2=0x7f46e40b5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:29.508 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:29.493+0000 7f46e2d9d700 1 --2- 172.21.15.143:0/2561063260 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f46e415bbc0 0x7f46e40b5730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:29.509 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:29.493+0000 7f46e2d9d700 1 -- 172.21.15.143:0/2561063260 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f46e414fdb0 msgr2=0x7f46e40c6230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:29.509 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:29.493+0000 7f46e2d9d700 1 --2- 172.21.15.143:0/2561063260 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f46e414fdb0 0x7f46e40c6230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:29.509 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:29.493+0000 7f46e2d9d700 1 -- 172.21.15.143:0/2561063260 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46e40b3830 con 0x7f46cc004400 2023-11-08T19:29:29.509 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:29.493+0000 7f46e259c700 1 --2- 172.21.15.143:0/2561063260 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f46e414fdb0 0x7f46e40c6230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:29:29.509 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:29.493+0000 7f46e2d9d700 1 --2- 172.21.15.143:0/2561063260 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f46cc004400 0x7f46e40c4f00 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f46d8001920 tx=0x7f46d8011da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:29.510 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:29.493+0000 7f46cb7fe700 1 -- 172.21.15.143:0/2561063260 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46d80074e0 con 0x7f46cc004400 2023-11-08T19:29:29.510 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:29.493+0000 7f46cb7fe700 1 -- 172.21.15.143:0/2561063260 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46d8007640 con 0x7f46cc004400 2023-11-08T19:29:29.510 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:29.493+0000 7f46e901d700 1 -- 172.21.15.143:0/2561063260 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46e40b3b10 con 0x7f46cc004400 2023-11-08T19:29:29.510 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:29.493+0000 7f46cb7fe700 1 -- 172.21.15.143:0/2561063260 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46d8005a70 con 0x7f46cc004400 2023-11-08T19:29:29.511 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:29.493+0000 7f46e901d700 1 -- 172.21.15.143:0/2561063260 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46e40c6770 con 0x7f46cc004400 2023-11-08T19:29:29.511 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:29.494+0000 7f46e901d700 1 -- 172.21.15.143:0/2561063260 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46e404eee0 con 0x7f46cc004400 2023-11-08T19:29:29.511 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:29.495+0000 7f46cb7fe700 1 -- 172.21.15.143:0/2561063260 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f46d8018020 con 0x7f46cc004400 2023-11-08T19:29:29.511 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:29.495+0000 7f46cb7fe700 1 --2- 172.21.15.143:0/2561063260 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f46d006af00 0x7f46d006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:29.511 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:29.495+0000 7f46cb7fe700 1 -- 172.21.15.143:0/2561063260 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6973+0+0 (secure 0 0 0) 0x7f46d808b7a0 con 0x7f46cc004400 2023-11-08T19:29:29.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:29.495+0000 7f46e259c700 1 --2- 172.21.15.143:0/2561063260 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f46d006af00 0x7f46d006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:29.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:29.496+0000 7f46e259c700 1 --2- 172.21.15.143:0/2561063260 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f46d006af00 0x7f46d006d3c0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f46d40018b0 tx=0x7f46d4007f90 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:29.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:29.499+0000 7f46cb7fe700 1 -- 172.21.15.143:0/2561063260 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f46d80563f0 con 0x7f46cc004400 2023-11-08T19:29:29.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:29.732+0000 7f46e901d700 1 -- 172.21.15.143:0/2561063260 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f46e40b2ff0 con 0x7f46cc004400 2023-11-08T19:29:29.735 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:29.733+0000 7f46cb7fe700 1 -- 172.21.15.143:0/2561063260 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f46d8059a10 con 0x7f46cc004400 2023-11-08T19:29:29.735 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:29:29.736 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":73,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5933,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/3918286710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6835","nonce":3918286710},{"type":"v1","addr":"172.21.15.174:6837","nonce":3918286710}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":7386,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3664584522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":3664584522},{"type":"v1","addr":"172.21.15.143:6835","nonce":3664584522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:03.092245+0000","modified":"2023-11-08T19:29:23.223822+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":96,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6906,"mds_1":6918},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6906":{"gid":6906,"name":"c","rank":0,"incarnation":68,"state":"up:active","state_seq":24,"addr":"172.21.15.143:6834/3172664377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":3172664377},{"type":"v1","addr":"172.21.15.143:6834","nonce":3172664377}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6918":{"gid":6918,"name":"b","rank":1,"incarnation":54,"state":"up:active","state_seq":6,"addr":"172.21.15.174:6836/3601088618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3601088618},{"type":"v1","addr":"172.21.15.174:6836","nonce":3601088618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:29:29.737 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:29.735+0000 7f46e901d700 1 -- 172.21.15.143:0/2561063260 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f46d006af00 msgr2=0x7f46d006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:29.737 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:29.735+0000 7f46e901d700 1 --2- 172.21.15.143:0/2561063260 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f46d006af00 0x7f46d006d3c0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f46d40018b0 tx=0x7f46d4007f90 comp rx=0 tx=0).stop 2023-11-08T19:29:29.737 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:29.735+0000 7f46e901d700 1 -- 172.21.15.143:0/2561063260 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f46cc004400 msgr2=0x7f46e40c4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:29.738 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:29.735+0000 7f46e901d700 1 --2- 172.21.15.143:0/2561063260 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f46cc004400 0x7f46e40c4f00 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f46d8001920 tx=0x7f46d8011da0 comp rx=0 tx=0).stop 2023-11-08T19:29:29.738 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:29.735+0000 7f46e901d700 1 -- 172.21.15.143:0/2561063260 shutdown_connections 2023-11-08T19:29:29.738 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:29.735+0000 7f46e901d700 1 --2- 172.21.15.143:0/2561063260 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f46d006af00 0x7f46d006d3c0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:29.738 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:29.735+0000 7f46e901d700 1 --2- 172.21.15.143:0/2561063260 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f46e415bbc0 0x7f46e40b5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:29.738 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:29.735+0000 7f46e901d700 1 --2- 172.21.15.143:0/2561063260 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f46cc004400 0x7f46e40c4f00 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:29.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:29.735+0000 7f46e901d700 1 --2- 172.21.15.143:0/2561063260 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f46e414fdb0 0x7f46e40c6230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:29.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:29.735+0000 7f46e901d700 1 -- 172.21.15.143:0/2561063260 >> 172.21.15.143:0/2561063260 conn(0x7f46e4099ab0 msgr2=0x7f46e40cd8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:29.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:29.735+0000 7f46e901d700 1 -- 172.21.15.143:0/2561063260 shutdown_connections 2023-11-08T19:29:29.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:29.735+0000 7f46e901d700 1 -- 172.21.15.143:0/2561063260 wait complete. 2023-11-08T19:29:29.740 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 73 2023-11-08T19:29:29.749 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2023-11-08T19:29:30.142 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~0s 2023-11-08T19:29:34.751 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:29:34.884 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:34.882+0000 7f83c2f2d700 1 Processor -- start 2023-11-08T19:29:34.885 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:34.882+0000 7f83c2f2d700 1 -- start start 2023-11-08T19:29:34.887 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:34.882+0000 7f83c2f2d700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f83bc15dd00 0x7f83bc0cca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:34.887 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:34.882+0000 7f83c2f2d700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f83bc1618f0 0x7f83bc06ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:34.887 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:34.883+0000 7f83c2f2d700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83bc0ccfb0 con 0x7f83bc06b150 2023-11-08T19:29:34.887 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:34.883+0000 7f83c2f2d700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83bc06b600 con 0x7f83bc1618f0 2023-11-08T19:29:34.887 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:34.883+0000 7f83c2f2d700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83bc0cad50 con 0x7f83bc15dd00 2023-11-08T19:29:34.888 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:34.883+0000 7f83c0cc9700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f83bc15dd00 0x7f83bc0cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:34.888 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:34.883+0000 7f83bbfff700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f83bc1618f0 0x7f83bc06ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:34.888 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:34.883+0000 7f83bbfff700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f83bc1618f0 0x7f83bc06ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.143:39176/0 (socket says 172.21.15.143:39176) 2023-11-08T19:29:34.888 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:34.883+0000 7f83c0cc9700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f83bc15dd00 0x7f83bc0cca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.143:43316/0 (socket says 172.21.15.143:43316) 2023-11-08T19:29:34.888 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:34.883+0000 7f83bbfff700 1 -- 172.21.15.143:0/3239903458 learned_addr learned my addr 172.21.15.143:0/3239903458 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:29:34.889 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:34.883+0000 7f83c14ca700 1 --1- 172.21.15.143:0/3239903458 >> v1:172.21.15.174:6789/0 conn(0x7f83bc06b150 0x7f83bc0b2210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.174:6789/0 says I am v1:172.21.15.143:34536/0 (socket says 172.21.15.143:34536) 2023-11-08T19:29:34.889 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:34.883+0000 7f83bbfff700 1 -- 172.21.15.143:0/3239903458 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f83bc15dd00 msgr2=0x7f83bc0cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:34.889 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:34.883+0000 7f83bbfff700 1 --2- 172.21.15.143:0/3239903458 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f83bc15dd00 0x7f83bc0cca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:34.889 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:34.883+0000 7f83bbfff700 1 -- 172.21.15.143:0/3239903458 >> v1:172.21.15.174:6789/0 conn(0x7f83bc06b150 legacy=0x7f83bc0b2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:34.889 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:34.883+0000 7f83bbfff700 1 -- 172.21.15.143:0/3239903458 --> v2:172.21.15.174:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83bc161df0 con 0x7f83bc1618f0 2023-11-08T19:29:34.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:34.883+0000 7f83c0cc9700 1 --2- 172.21.15.143:0/3239903458 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f83bc15dd00 0x7f83bc0cca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T19:29:34.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:34.883+0000 7f83bbfff700 1 --2- 172.21.15.143:0/3239903458 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f83bc1618f0 0x7f83bc06ac10 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f83a80098f0 tx=0x7f83a8009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f99a7656b1388c51 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:34.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:34.884+0000 7f83bb7fe700 1 -- 172.21.15.143:0/3239903458 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83a8011830 con 0x7f83bc1618f0 2023-11-08T19:29:34.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:34.884+0000 7f83bb7fe700 1 -- 172.21.15.143:0/3239903458 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83a8011990 con 0x7f83bc1618f0 2023-11-08T19:29:34.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:34.884+0000 7f83bb7fe700 1 -- 172.21.15.143:0/3239903458 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83a8011d20 con 0x7f83bc1618f0 2023-11-08T19:29:34.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:34.884+0000 7f83c2f2d700 1 -- 172.21.15.143:0/3239903458 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f83bc1618f0 msgr2=0x7f83bc06ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:34.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:34.884+0000 7f83c2f2d700 1 --2- 172.21.15.143:0/3239903458 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f83bc1618f0 0x7f83bc06ac10 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f83a80098f0 tx=0x7f83a8009c00 comp rx=0 tx=0).stop 2023-11-08T19:29:34.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:34.884+0000 7f83c2f2d700 1 -- 172.21.15.143:0/3239903458 shutdown_connections 2023-11-08T19:29:34.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:34.884+0000 7f83c2f2d700 1 --2- 172.21.15.143:0/3239903458 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f83bc15dd00 0x7f83bc0cca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:34.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:34.884+0000 7f83c2f2d700 1 --2- 172.21.15.143:0/3239903458 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f83bc1618f0 0x7f83bc06ac10 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:34.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:34.884+0000 7f83c2f2d700 1 -- 172.21.15.143:0/3239903458 >> 172.21.15.143:0/3239903458 conn(0x7f83bc099ab0 msgr2=0x7f83bc0d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:34.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:34.884+0000 7f83c2f2d700 1 -- 172.21.15.143:0/3239903458 shutdown_connections 2023-11-08T19:29:34.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:34.884+0000 7f83c2f2d700 1 -- 172.21.15.143:0/3239903458 wait complete. 2023-11-08T19:29:34.895 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:34.885+0000 7f83c2f2d700 1 Processor -- start 2023-11-08T19:29:34.895 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:34.885+0000 7f83c2f2d700 1 -- start start 2023-11-08T19:29:34.895 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:34.886+0000 7f83c2f2d700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f83bc06b150 0x7f83bc151aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:34.895 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:34.886+0000 7f83c2f2d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f83bc15dd00 0x7f83bc0c87d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:34.896 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:34.886+0000 7f83c2f2d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f83bc1618f0 0x7f83bc0c8d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:34.896 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:34.886+0000 7f83c2f2d700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83bc151fe0 con 0x7f83bc1618f0 2023-11-08T19:29:34.896 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:34.886+0000 7f83c2f2d700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83bc0c9250 con 0x7f83bc15dd00 2023-11-08T19:29:34.896 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:34.886+0000 7f83c2f2d700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83bc0b0270 con 0x7f83bc06b150 2023-11-08T19:29:34.896 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:34.886+0000 7f83c14ca700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f83bc1618f0 0x7f83bc0c8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:34.897 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:34.886+0000 7f83c14ca700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f83bc1618f0 0x7f83bc0c8d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55686/0 (socket says 172.21.15.143:55686) 2023-11-08T19:29:34.897 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:34.886+0000 7f83c14ca700 1 -- 172.21.15.143:0/4238391947 learned_addr learned my addr 172.21.15.143:0/4238391947 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:29:34.897 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:34.887+0000 7f83c0cc9700 1 --2- 172.21.15.143:0/4238391947 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f83bc06b150 0x7f83bc151aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:34.897 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:34.887+0000 7f83bbfff700 1 --2- 172.21.15.143:0/4238391947 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f83bc15dd00 0x7f83bc0c87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:34.897 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:34.887+0000 7f83c14ca700 1 -- 172.21.15.143:0/4238391947 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f83bc06b150 msgr2=0x7f83bc151aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:34.897 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:34.887+0000 7f83c14ca700 1 --2- 172.21.15.143:0/4238391947 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f83bc06b150 0x7f83bc151aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:34.898 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:34.887+0000 7f83c14ca700 1 -- 172.21.15.143:0/4238391947 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f83bc15dd00 msgr2=0x7f83bc0c87d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:34.898 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:34.887+0000 7f83c14ca700 1 --2- 172.21.15.143:0/4238391947 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f83bc15dd00 0x7f83bc0c87d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:34.898 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:34.887+0000 7f83c14ca700 1 -- 172.21.15.143:0/4238391947 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83b0009430 con 0x7f83bc1618f0 2023-11-08T19:29:34.898 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:34.887+0000 7f83c14ca700 1 --2- 172.21.15.143:0/4238391947 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f83bc1618f0 0x7f83bc0c8d10 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f83b400cc00 tx=0x7f83b400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:34.899 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:34.887+0000 7f83b97fa700 1 -- 172.21.15.143:0/4238391947 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83b400aba0 con 0x7f83bc1618f0 2023-11-08T19:29:34.899 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:34.887+0000 7f83b97fa700 1 -- 172.21.15.143:0/4238391947 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83b400ad00 con 0x7f83bc1618f0 2023-11-08T19:29:34.899 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:34.887+0000 7f83c2f2d700 1 -- 172.21.15.143:0/4238391947 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83a8009550 con 0x7f83bc1618f0 2023-11-08T19:29:34.899 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:34.887+0000 7f83b97fa700 1 -- 172.21.15.143:0/4238391947 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83b4007270 con 0x7f83bc1618f0 2023-11-08T19:29:34.899 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:34.887+0000 7f83c2f2d700 1 -- 172.21.15.143:0/4238391947 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83bc0b0830 con 0x7f83bc1618f0 2023-11-08T19:29:34.899 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:34.888+0000 7f83c2f2d700 1 -- 172.21.15.143:0/4238391947 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83bc04eee0 con 0x7f83bc1618f0 2023-11-08T19:29:34.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:34.889+0000 7f83b97fa700 1 -- 172.21.15.143:0/4238391947 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f83b40073d0 con 0x7f83bc1618f0 2023-11-08T19:29:34.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:34.889+0000 7f83b97fa700 1 --2- 172.21.15.143:0/4238391947 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f83a006af00 0x7f83a006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:34.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:34.889+0000 7f83b97fa700 1 -- 172.21.15.143:0/4238391947 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6973+0+0 (secure 0 0 0) 0x7f83b40897c0 con 0x7f83bc1618f0 2023-11-08T19:29:34.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:34.889+0000 7f83c0cc9700 1 --2- 172.21.15.143:0/4238391947 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f83a006af00 0x7f83a006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:34.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:34.890+0000 7f83c0cc9700 1 --2- 172.21.15.143:0/4238391947 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f83a006af00 0x7f83a006d3c0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f83b0009ce0 tx=0x7f83b0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:34.901 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:34.893+0000 7f83b97fa700 1 -- 172.21.15.143:0/4238391947 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f83b40544c0 con 0x7f83bc1618f0 2023-11-08T19:29:35.129 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:35.126+0000 7f83c2f2d700 1 -- 172.21.15.143:0/4238391947 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f83bc06b920 con 0x7f83bc1618f0 2023-11-08T19:29:35.129 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:35.127+0000 7f83b97fa700 1 -- 172.21.15.143:0/4238391947 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f83b4057ae0 con 0x7f83bc1618f0 2023-11-08T19:29:35.130 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:29:35.130 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":73,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5933,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/3918286710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6835","nonce":3918286710},{"type":"v1","addr":"172.21.15.174:6837","nonce":3918286710}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":7386,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3664584522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":3664584522},{"type":"v1","addr":"172.21.15.143:6835","nonce":3664584522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:03.092245+0000","modified":"2023-11-08T19:29:23.223822+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":96,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6906,"mds_1":6918},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6906":{"gid":6906,"name":"c","rank":0,"incarnation":68,"state":"up:active","state_seq":24,"addr":"172.21.15.143:6834/3172664377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":3172664377},{"type":"v1","addr":"172.21.15.143:6834","nonce":3172664377}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6918":{"gid":6918,"name":"b","rank":1,"incarnation":54,"state":"up:active","state_seq":6,"addr":"172.21.15.174:6836/3601088618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3601088618},{"type":"v1","addr":"172.21.15.174:6836","nonce":3601088618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:29:35.133 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:35.129+0000 7f83c2f2d700 1 -- 172.21.15.143:0/4238391947 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f83a006af00 msgr2=0x7f83a006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:35.133 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:35.129+0000 7f83c2f2d700 1 --2- 172.21.15.143:0/4238391947 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f83a006af00 0x7f83a006d3c0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f83b0009ce0 tx=0x7f83b0008040 comp rx=0 tx=0).stop 2023-11-08T19:29:35.133 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:35.129+0000 7f83c2f2d700 1 -- 172.21.15.143:0/4238391947 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f83bc1618f0 msgr2=0x7f83bc0c8d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:35.134 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:35.129+0000 7f83c2f2d700 1 --2- 172.21.15.143:0/4238391947 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f83bc1618f0 0x7f83bc0c8d10 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f83b400cc00 tx=0x7f83b400a060 comp rx=0 tx=0).stop 2023-11-08T19:29:35.134 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:35.129+0000 7f83c2f2d700 1 -- 172.21.15.143:0/4238391947 shutdown_connections 2023-11-08T19:29:35.134 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:35.129+0000 7f83c2f2d700 1 --2- 172.21.15.143:0/4238391947 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f83a006af00 0x7f83a006d3c0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:35.135 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:35.129+0000 7f83c2f2d700 1 --2- 172.21.15.143:0/4238391947 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f83bc06b150 0x7f83bc151aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:35.135 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:35.129+0000 7f83c2f2d700 1 --2- 172.21.15.143:0/4238391947 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f83bc1618f0 0x7f83bc0c8d10 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:35.135 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:35.129+0000 7f83c2f2d700 1 --2- 172.21.15.143:0/4238391947 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f83bc15dd00 0x7f83bc0c87d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:35.135 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:35.129+0000 7f83c2f2d700 1 -- 172.21.15.143:0/4238391947 >> 172.21.15.143:0/4238391947 conn(0x7f83bc099ab0 msgr2=0x7f83bc0ad380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:35.136 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:35.130+0000 7f83c2f2d700 1 -- 172.21.15.143:0/4238391947 shutdown_connections 2023-11-08T19:29:35.136 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:35.130+0000 7f83c2f2d700 1 -- 172.21.15.143:0/4238391947 wait complete. 2023-11-08T19:29:35.136 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 73 2023-11-08T19:29:35.144 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2023-11-08T19:29:36.753 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~7s 2023-11-08T19:29:40.145 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:29:40.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.279+0000 7ffa6c636700 1 Processor -- start 2023-11-08T19:29:40.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.279+0000 7ffa6c636700 1 -- start start 2023-11-08T19:29:40.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.280+0000 7ffa6c636700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ffa640b2510 0x7ffa640cb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:40.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.280+0000 7ffa6c636700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7ffa640cc4a0 0x7ffa6406bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:40.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.280+0000 7ffa6c636700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa6406b820 con 0x7ffa6415aa20 2023-11-08T19:29:40.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.280+0000 7ffa6c636700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa6406ac10 con 0x7ffa640cc4a0 2023-11-08T19:29:40.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.280+0000 7ffa6c636700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa6406aef0 con 0x7ffa640b2510 2023-11-08T19:29:40.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.280+0000 7ffa6a3d2700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7ffa6415aa20 0x7ffa640b2d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:51844/0 (socket says 172.21.15.143:51844) 2023-11-08T19:29:40.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.280+0000 7ffa6a3d2700 1 -- 172.21.15.143:0/1118776666 learned_addr learned my addr 172.21.15.143:0/1118776666 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:29:40.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.281+0000 7ffa6abd3700 1 --2- 172.21.15.143:0/1118776666 >> v2:172.21.15.174:3300/0 conn(0x7ffa640cc4a0 0x7ffa6406bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:40.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.281+0000 7ffa69bd1700 1 --2- 172.21.15.143:0/1118776666 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ffa640b2510 0x7ffa640cb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:40.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.281+0000 7ffa693d0700 1 -- 172.21.15.143:0/1118776666 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 180816625 0 0) 0x7ffa6406b820 con 0x7ffa6415aa20 2023-11-08T19:29:40.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.281+0000 7ffa693d0700 1 -- 172.21.15.143:0/1118776666 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa500035b0 con 0x7ffa6415aa20 2023-11-08T19:29:40.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.281+0000 7ffa69bd1700 1 -- 172.21.15.143:0/1118776666 >> v2:172.21.15.174:3300/0 conn(0x7ffa640cc4a0 msgr2=0x7ffa6406bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:40.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.281+0000 7ffa69bd1700 1 --2- 172.21.15.143:0/1118776666 >> v2:172.21.15.174:3300/0 conn(0x7ffa640cc4a0 0x7ffa6406bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:40.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.281+0000 7ffa69bd1700 1 -- 172.21.15.143:0/1118776666 >> v1:172.21.15.143:6789/0 conn(0x7ffa6415aa20 legacy=0x7ffa640b2d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:40.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.281+0000 7ffa69bd1700 1 -- 172.21.15.143:0/1118776666 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa6406b1d0 con 0x7ffa640b2510 2023-11-08T19:29:40.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.281+0000 7ffa69bd1700 1 --2- 172.21.15.143:0/1118776666 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ffa640b2510 0x7ffa640cb4b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ffa60009840 tx=0x7ffa60009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2da3fc161ba3dad server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:40.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.281+0000 7ffa693d0700 1 -- 172.21.15.143:0/1118776666 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa60011830 con 0x7ffa640b2510 2023-11-08T19:29:40.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.281+0000 7ffa693d0700 1 -- 172.21.15.143:0/1118776666 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa60011990 con 0x7ffa640b2510 2023-11-08T19:29:40.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.281+0000 7ffa693d0700 1 -- 172.21.15.143:0/1118776666 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa60011d20 con 0x7ffa640b2510 2023-11-08T19:29:40.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.282+0000 7ffa6c636700 1 -- 172.21.15.143:0/1118776666 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ffa640b2510 msgr2=0x7ffa640cb4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:40.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.282+0000 7ffa6c636700 1 --2- 172.21.15.143:0/1118776666 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ffa640b2510 0x7ffa640cb4b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ffa60009840 tx=0x7ffa60009b50 comp rx=0 tx=0).stop 2023-11-08T19:29:40.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.282+0000 7ffa6c636700 1 -- 172.21.15.143:0/1118776666 shutdown_connections 2023-11-08T19:29:40.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.282+0000 7ffa6c636700 1 --2- 172.21.15.143:0/1118776666 >> v2:172.21.15.174:3300/0 conn(0x7ffa640cc4a0 0x7ffa6406bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:40.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.282+0000 7ffa6c636700 1 --2- 172.21.15.143:0/1118776666 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ffa640b2510 0x7ffa640cb4b0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:40.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.282+0000 7ffa6c636700 1 -- 172.21.15.143:0/1118776666 >> 172.21.15.143:0/1118776666 conn(0x7ffa640b1a20 msgr2=0x7ffa640b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:40.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.282+0000 7ffa6c636700 1 -- 172.21.15.143:0/1118776666 shutdown_connections 2023-11-08T19:29:40.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.282+0000 7ffa6c636700 1 -- 172.21.15.143:0/1118776666 wait complete. 2023-11-08T19:29:40.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.283+0000 7ffa6c636700 1 Processor -- start 2023-11-08T19:29:40.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.283+0000 7ffa6c636700 1 -- start start 2023-11-08T19:29:40.294 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.284+0000 7ffa6c636700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffa640b2510 0x7ffa641efcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:40.295 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.284+0000 7ffa6c636700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ffa640cc4a0 0x7ffa64158130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:40.295 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.284+0000 7ffa6c636700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffa6415aa20 0x7ffa64158670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:40.295 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.284+0000 7ffa6c636700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa64142f60 con 0x7ffa6415aa20 2023-11-08T19:29:40.295 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.284+0000 7ffa6c636700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa641f0210 con 0x7ffa640b2510 2023-11-08T19:29:40.296 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.284+0000 7ffa6c636700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa6415cd40 con 0x7ffa640cc4a0 2023-11-08T19:29:40.296 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.284+0000 7ffa6abd3700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffa6415aa20 0x7ffa64158670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:40.296 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.284+0000 7ffa6abd3700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffa6415aa20 0x7ffa64158670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55700/0 (socket says 172.21.15.143:55700) 2023-11-08T19:29:40.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.284+0000 7ffa6abd3700 1 -- 172.21.15.143:0/359222112 learned_addr learned my addr 172.21.15.143:0/359222112 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:29:40.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.284+0000 7ffa69bd1700 1 --2- 172.21.15.143:0/359222112 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ffa640cc4a0 0x7ffa64158130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:40.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.284+0000 7ffa6a3d2700 1 --2- 172.21.15.143:0/359222112 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffa640b2510 0x7ffa641efcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:40.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.285+0000 7ffa6abd3700 1 -- 172.21.15.143:0/359222112 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ffa640cc4a0 msgr2=0x7ffa64158130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:40.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.285+0000 7ffa6abd3700 1 --2- 172.21.15.143:0/359222112 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ffa640cc4a0 0x7ffa64158130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:40.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.285+0000 7ffa6abd3700 1 -- 172.21.15.143:0/359222112 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffa640b2510 msgr2=0x7ffa641efcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:40.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.285+0000 7ffa6abd3700 1 --2- 172.21.15.143:0/359222112 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffa640b2510 0x7ffa641efcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:40.299 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.285+0000 7ffa6abd3700 1 -- 172.21.15.143:0/359222112 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa600094a0 con 0x7ffa6415aa20 2023-11-08T19:29:40.299 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.285+0000 7ffa6a3d2700 1 --2- 172.21.15.143:0/359222112 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffa640b2510 0x7ffa641efcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:29:40.299 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.285+0000 7ffa6abd3700 1 --2- 172.21.15.143:0/359222112 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffa6415aa20 0x7ffa64158670 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7ffa5c00efc0 tx=0x7ffa5c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:40.299 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.285+0000 7ffa5affd700 1 -- 172.21.15.143:0/359222112 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa5c00cd40 con 0x7ffa6415aa20 2023-11-08T19:29:40.300 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.285+0000 7ffa5affd700 1 -- 172.21.15.143:0/359222112 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa5c00f040 con 0x7ffa6415aa20 2023-11-08T19:29:40.300 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.285+0000 7ffa6c636700 1 -- 172.21.15.143:0/359222112 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa641455d0 con 0x7ffa6415aa20 2023-11-08T19:29:40.300 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.285+0000 7ffa5affd700 1 -- 172.21.15.143:0/359222112 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa5c0146c0 con 0x7ffa6415aa20 2023-11-08T19:29:40.300 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.285+0000 7ffa6c636700 1 -- 172.21.15.143:0/359222112 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa64145b90 con 0x7ffa6415aa20 2023-11-08T19:29:40.300 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.286+0000 7ffa6c636700 1 -- 172.21.15.143:0/359222112 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa6415cb80 con 0x7ffa6415aa20 2023-11-08T19:29:40.301 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.286+0000 7ffa5affd700 1 -- 172.21.15.143:0/359222112 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ffa5c014820 con 0x7ffa6415aa20 2023-11-08T19:29:40.301 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.287+0000 7ffa5affd700 1 --2- 172.21.15.143:0/359222112 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7ffa4806af00 0x7ffa4806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:40.301 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.287+0000 7ffa5affd700 1 -- 172.21.15.143:0/359222112 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6973+0+0 (secure 0 0 0) 0x7ffa5c08a670 con 0x7ffa6415aa20 2023-11-08T19:29:40.301 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.287+0000 7ffa6a3d2700 1 --2- 172.21.15.143:0/359222112 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7ffa4806af00 0x7ffa4806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:40.301 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.287+0000 7ffa6a3d2700 1 --2- 172.21.15.143:0/359222112 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7ffa4806af00 0x7ffa4806d3c0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7ffa540011e0 tx=0x7ffa54009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:40.302 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.291+0000 7ffa5affd700 1 -- 172.21.15.143:0/359222112 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ffa6415cb80 con 0x7ffa6415aa20 2023-11-08T19:29:40.526 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.524+0000 7ffa6c636700 1 -- 172.21.15.143:0/359222112 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffa6415cb80 con 0x7ffa6415aa20 2023-11-08T19:29:40.526 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.525+0000 7ffa5affd700 1 -- 172.21.15.143:0/359222112 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+4991 (secure 0 0 0) 0x7ffa6415cb80 con 0x7ffa6415aa20 2023-11-08T19:29:40.527 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:29:40.527 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":74,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5933,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/3918286710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6835","nonce":3918286710},{"type":"v1","addr":"172.21.15.174:6837","nonce":3918286710}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":7386,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/3664584522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":3664584522},{"type":"v1","addr":"172.21.15.143:6835","nonce":3664584522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:03.092245+0000","modified":"2023-11-08T19:29:37.609130+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":96,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6906,"mds_1":6918},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6906":{"gid":6906,"name":"c","rank":0,"incarnation":68,"state":"up:active","state_seq":24,"addr":"172.21.15.143:6834/3172664377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":3172664377},{"type":"v1","addr":"172.21.15.143:6834","nonce":3172664377}]},"join_fscid":-1,"laggy_since":"2023-11-08T19:29:37.609129+0000","export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6918":{"gid":6918,"name":"b","rank":1,"incarnation":54,"state":"up:active","state_seq":6,"addr":"172.21.15.174:6836/3601088618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3601088618},{"type":"v1","addr":"172.21.15.174:6836","nonce":3601088618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:29:40.528 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.527+0000 7ffa6c636700 1 -- 172.21.15.143:0/359222112 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7ffa4806af00 msgr2=0x7ffa4806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:40.529 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.527+0000 7ffa6c636700 1 --2- 172.21.15.143:0/359222112 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7ffa4806af00 0x7ffa4806d3c0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7ffa540011e0 tx=0x7ffa54009040 comp rx=0 tx=0).stop 2023-11-08T19:29:40.529 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.527+0000 7ffa6c636700 1 -- 172.21.15.143:0/359222112 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffa6415aa20 msgr2=0x7ffa64158670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:40.529 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.527+0000 7ffa6c636700 1 --2- 172.21.15.143:0/359222112 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffa6415aa20 0x7ffa64158670 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7ffa5c00efc0 tx=0x7ffa5c00c3a0 comp rx=0 tx=0).stop 2023-11-08T19:29:40.529 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.527+0000 7ffa6c636700 1 -- 172.21.15.143:0/359222112 shutdown_connections 2023-11-08T19:29:40.529 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.527+0000 7ffa6c636700 1 --2- 172.21.15.143:0/359222112 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7ffa4806af00 0x7ffa4806d3c0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:40.530 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.527+0000 7ffa6c636700 1 --2- 172.21.15.143:0/359222112 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffa6415aa20 0x7ffa64158670 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:40.530 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.527+0000 7ffa6c636700 1 --2- 172.21.15.143:0/359222112 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffa640b2510 0x7ffa641efcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:40.530 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.527+0000 7ffa6c636700 1 --2- 172.21.15.143:0/359222112 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ffa640cc4a0 0x7ffa64158130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:40.530 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.527+0000 7ffa6c636700 1 -- 172.21.15.143:0/359222112 >> 172.21.15.143:0/359222112 conn(0x7ffa640b1a20 msgr2=0x7ffa64156890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:40.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.527+0000 7ffa6c636700 1 -- 172.21.15.143:0/359222112 shutdown_connections 2023-11-08T19:29:40.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.527+0000 7ffa6c636700 1 -- 172.21.15.143:0/359222112 wait complete. 2023-11-08T19:29:40.531 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 74 2023-11-08T19:29:40.541 DEBUG:tasks.ceph_test_case:wait_until_true: success in 15s and 0 retries 2023-11-08T19:29:40.541 DEBUG:teuthology.orchestra.run.smithi143:> sudo sysctl -n kernel.core_pattern 2023-11-08T19:29:40.567 INFO:teuthology.orchestra.run.smithi143.stdout:/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-08T19:29:40.568 INFO:tasks.cephfs.cephfs_test_case:Clearing core from directory: /home/ubuntu/cephtest/archive/coredump 2023-11-08T19:29:40.568 DEBUG:teuthology.orchestra.run.smithi143:> cd /home/ubuntu/cephtest/archive/coredump && sudo ls | sudo xargs file 2023-11-08T19:29:40.637 INFO:teuthology.orchestra.run.smithi143.stdout:1699471763.131232.core: ELF 64-bit LSB core file, x86-64, version 1 (SYSV), SVR4-style, from 'ceph-mds -f --cluster ceph -i c', real uid: 0, effective uid: 0, real gid: 0, effective gid: 0, execfn: '/bin/ceph-mds', platform: 'x86_64' 2023-11-08T19:29:40.639 INFO:tasks.cephfs.cephfs_test_case:Enumerated cores: ['1699471763.131232.core'] 2023-11-08T19:29:40.639 INFO:tasks.cephfs.cephfs_test_case:Found core file 1699471763.131232.core, deleting it 2023-11-08T19:29:40.639 DEBUG:teuthology.orchestra.run.smithi143:> cd /home/ubuntu/cephtest/archive/coredump && sudo rm -f 1699471763.131232.core 2023-11-08T19:29:40.707 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail 6:0 2023-11-08T19:29:40.868 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.867+0000 7f28ffac4700 1 Processor -- start 2023-11-08T19:29:40.869 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.867+0000 7f28ffac4700 1 -- start start 2023-11-08T19:29:40.871 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.868+0000 7f28ffac4700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f28f815dd00 0x7f28f80cca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:40.871 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.868+0000 7f28ffac4700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f28f81618f0 0x7f28f806ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:40.871 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.868+0000 7f28ffac4700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f28f806b150 0x7f28f80b2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:40.872 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.868+0000 7f28ffac4700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28f806b600 con 0x7f28f81618f0 2023-11-08T19:29:40.872 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.868+0000 7f28ffac4700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28f80cad50 con 0x7f28f806b150 2023-11-08T19:29:40.872 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.868+0000 7f28ffac4700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28f80ccfb0 con 0x7f28f815dd00 2023-11-08T19:29:40.872 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.868+0000 7f28fd05f700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f28f81618f0 0x7f28f806ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:40.873 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.868+0000 7f28fd05f700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f28f81618f0 0x7f28f806ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55704/0 (socket says 172.21.15.143:55704) 2023-11-08T19:29:40.873 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.868+0000 7f28fd05f700 1 -- 172.21.15.143:0/755826268 learned_addr learned my addr 172.21.15.143:0/755826268 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:29:40.873 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.868+0000 7f28fe061700 1 --2- 172.21.15.143:0/755826268 >> v2:172.21.15.174:3300/0 conn(0x7f28f806b150 0x7f28f80b2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:40.873 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.868+0000 7f28fd860700 1 --2- 172.21.15.143:0/755826268 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f28f815dd00 0x7f28f80cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:40.873 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.869+0000 7f28fd05f700 1 -- 172.21.15.143:0/755826268 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f28f815dd00 msgr2=0x7f28f80cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:40.873 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.869+0000 7f28fd05f700 1 --2- 172.21.15.143:0/755826268 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f28f815dd00 0x7f28f80cca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:40.874 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.869+0000 7f28fd05f700 1 -- 172.21.15.143:0/755826268 >> v2:172.21.15.174:3300/0 conn(0x7f28f806b150 msgr2=0x7f28f80b2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:40.874 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.869+0000 7f28fd05f700 1 --2- 172.21.15.143:0/755826268 >> v2:172.21.15.174:3300/0 conn(0x7f28f806b150 0x7f28f80b2210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:40.874 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.869+0000 7f28fd05f700 1 -- 172.21.15.143:0/755826268 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28f8161df0 con 0x7f28f81618f0 2023-11-08T19:29:40.874 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.869+0000 7f28fe061700 1 --2- 172.21.15.143:0/755826268 >> v2:172.21.15.174:3300/0 conn(0x7f28f806b150 0x7f28f80b2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:29:40.875 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.869+0000 7f28fd860700 1 --2- 172.21.15.143:0/755826268 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f28f815dd00 0x7f28f80cca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:29:40.875 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.869+0000 7f28fd05f700 1 --2- 172.21.15.143:0/755826268 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f28f81618f0 0x7f28f806ac10 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f28f80708b0 tx=0x7f28f4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe86a5ac79b70094 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:40.875 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.869+0000 7f28fc85e700 1 -- 172.21.15.143:0/755826268 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f400b870 con 0x7f28f81618f0 2023-11-08T19:29:40.875 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.869+0000 7f28fc85e700 1 -- 172.21.15.143:0/755826268 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28f400b9d0 con 0x7f28f81618f0 2023-11-08T19:29:40.875 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.869+0000 7f28fc85e700 1 -- 172.21.15.143:0/755826268 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f400bd60 con 0x7f28f81618f0 2023-11-08T19:29:40.875 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.869+0000 7f28ffac4700 1 -- 172.21.15.143:0/755826268 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f28f81618f0 msgr2=0x7f28f806ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:40.876 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.869+0000 7f28ffac4700 1 --2- 172.21.15.143:0/755826268 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f28f81618f0 0x7f28f806ac10 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f28f80708b0 tx=0x7f28f4009bd0 comp rx=0 tx=0).stop 2023-11-08T19:29:40.876 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.869+0000 7f28ffac4700 1 -- 172.21.15.143:0/755826268 shutdown_connections 2023-11-08T19:29:40.876 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.869+0000 7f28ffac4700 1 --2- 172.21.15.143:0/755826268 >> v2:172.21.15.174:3300/0 conn(0x7f28f806b150 0x7f28f80b2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:40.876 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.869+0000 7f28ffac4700 1 --2- 172.21.15.143:0/755826268 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f28f815dd00 0x7f28f80cca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:40.876 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.869+0000 7f28ffac4700 1 --2- 172.21.15.143:0/755826268 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f28f81618f0 0x7f28f806ac10 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:40.877 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.869+0000 7f28ffac4700 1 -- 172.21.15.143:0/755826268 >> 172.21.15.143:0/755826268 conn(0x7f28f8099ab0 msgr2=0x7f28f80d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:40.877 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.870+0000 7f28ffac4700 1 -- 172.21.15.143:0/755826268 shutdown_connections 2023-11-08T19:29:40.877 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.870+0000 7f28ffac4700 1 -- 172.21.15.143:0/755826268 wait complete. 2023-11-08T19:29:40.880 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.871+0000 7f28ffac4700 1 Processor -- start 2023-11-08T19:29:40.880 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.871+0000 7f28ffac4700 1 -- start start 2023-11-08T19:29:40.880 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.872+0000 7f28ffac4700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f28f806b150 0x7f28f8154820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:40.880 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.872+0000 7f28ffac4700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f28f815dd00 0x7f28f81f3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:40.880 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.872+0000 7f28ffac4700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f28f81618f0 0x7f28f81f4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:40.881 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.872+0000 7f28ffac4700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28f80b1490 con 0x7f28f81618f0 2023-11-08T19:29:40.881 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.872+0000 7f28ffac4700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28f80b5310 con 0x7f28f815dd00 2023-11-08T19:29:40.881 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.872+0000 7f28ffac4700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28f80b55f0 con 0x7f28f806b150 2023-11-08T19:29:40.881 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.872+0000 7f28fe061700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f28f81618f0 0x7f28f81f4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:40.881 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.872+0000 7f28fe061700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f28f81618f0 0x7f28f81f4140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55714/0 (socket says 172.21.15.143:55714) 2023-11-08T19:29:40.882 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.872+0000 7f28fe061700 1 -- 172.21.15.143:0/2624176922 learned_addr learned my addr 172.21.15.143:0/2624176922 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:29:40.882 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.872+0000 7f28fd860700 1 --2- 172.21.15.143:0/2624176922 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f28f806b150 0x7f28f8154820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:40.882 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.872+0000 7f28fd05f700 1 --2- 172.21.15.143:0/2624176922 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f28f815dd00 0x7f28f81f3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:40.882 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.872+0000 7f28fe061700 1 -- 172.21.15.143:0/2624176922 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f28f806b150 msgr2=0x7f28f8154820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:40.882 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.872+0000 7f28fe061700 1 --2- 172.21.15.143:0/2624176922 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f28f806b150 0x7f28f8154820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:40.883 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.872+0000 7f28fe061700 1 -- 172.21.15.143:0/2624176922 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f28f815dd00 msgr2=0x7f28f81f3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:40.883 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.872+0000 7f28fe061700 1 --2- 172.21.15.143:0/2624176922 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f28f815dd00 0x7f28f81f3c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:40.883 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.872+0000 7f28fe061700 1 -- 172.21.15.143:0/2624176922 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28f4009550 con 0x7f28f81618f0 2023-11-08T19:29:40.883 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.872+0000 7f28fd05f700 1 --2- 172.21.15.143:0/2624176922 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f28f815dd00 0x7f28f81f3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:29:40.883 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.873+0000 7f28fe061700 1 --2- 172.21.15.143:0/2624176922 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f28f81618f0 0x7f28f81f4140 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f28f000cd10 tx=0x7f28f000a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:40.884 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.873+0000 7f28ee7fc700 1 -- 172.21.15.143:0/2624176922 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f000ce60 con 0x7f28f81618f0 2023-11-08T19:29:40.884 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.873+0000 7f28ee7fc700 1 -- 172.21.15.143:0/2624176922 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28f000f480 con 0x7f28f81618f0 2023-11-08T19:29:40.884 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.873+0000 7f28ffac4700 1 -- 172.21.15.143:0/2624176922 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28f80b58d0 con 0x7f28f81618f0 2023-11-08T19:29:40.885 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.873+0000 7f28ee7fc700 1 -- 172.21.15.143:0/2624176922 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f000f810 con 0x7f28f81618f0 2023-11-08T19:29:40.885 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.873+0000 7f28ffac4700 1 -- 172.21.15.143:0/2624176922 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28f8154d90 con 0x7f28f81618f0 2023-11-08T19:29:40.886 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.874+0000 7f28ffac4700 1 -- 172.21.15.143:0/2624176922 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28f804eee0 con 0x7f28f81618f0 2023-11-08T19:29:40.886 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.874+0000 7f28ee7fc700 1 -- 172.21.15.143:0/2624176922 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f28f000b040 con 0x7f28f81618f0 2023-11-08T19:29:40.886 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.875+0000 7f28ee7fc700 1 --2- 172.21.15.143:0/2624176922 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f28dc06af00 0x7f28dc06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:40.886 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.875+0000 7f28ee7fc700 1 -- 172.21.15.143:0/2624176922 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6973+0+0 (secure 0 0 0) 0x7f28f008a220 con 0x7f28f81618f0 2023-11-08T19:29:40.886 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.875+0000 7f28fd860700 1 --2- 172.21.15.143:0/2624176922 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f28dc06af00 0x7f28dc06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:40.887 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.875+0000 7f28fd860700 1 --2- 172.21.15.143:0/2624176922 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f28dc06af00 0x7f28dc06d3c0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f28e800abe0 tx=0x7f28e8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:40.887 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:40.879+0000 7f28ee7fc700 1 -- 172.21.15.143:0/2624176922 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f28f0054de0 con 0x7f28f81618f0 2023-11-08T19:29:41.150 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:41.149+0000 7f28ffac4700 1 -- 172.21.15.143:0/2624176922 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "6:0"} v 0) v1 -- 0x7f28f806b920 con 0x7f28f81618f0 2023-11-08T19:29:41.152 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:41.151+0000 7f28ee7fc700 1 -- 172.21.15.143:0/2624176922 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "6:0"}]=0 failed mds gid 6906 v75) v1 ==== 97+0+0 (secure 0 0 0) 0x7f28f0058400 con 0x7f28f81618f0 2023-11-08T19:29:41.156 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:41.154+0000 7f28ffac4700 1 -- 172.21.15.143:0/2624176922 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f28dc06af00 msgr2=0x7f28dc06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:41.156 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:41.154+0000 7f28ffac4700 1 --2- 172.21.15.143:0/2624176922 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f28dc06af00 0x7f28dc06d3c0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f28e800abe0 tx=0x7f28e8008040 comp rx=0 tx=0).stop 2023-11-08T19:29:41.156 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:41.154+0000 7f28ffac4700 1 -- 172.21.15.143:0/2624176922 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f28f81618f0 msgr2=0x7f28f81f4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:41.157 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:41.154+0000 7f28ffac4700 1 --2- 172.21.15.143:0/2624176922 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f28f81618f0 0x7f28f81f4140 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f28f000cd10 tx=0x7f28f000a560 comp rx=0 tx=0).stop 2023-11-08T19:29:41.157 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:41.154+0000 7f28ffac4700 1 -- 172.21.15.143:0/2624176922 shutdown_connections 2023-11-08T19:29:41.157 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:41.154+0000 7f28ffac4700 1 --2- 172.21.15.143:0/2624176922 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f28dc06af00 0x7f28dc06d3c0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:41.157 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:41.154+0000 7f28ffac4700 1 --2- 172.21.15.143:0/2624176922 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f28f806b150 0x7f28f8154820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:41.157 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:41.154+0000 7f28ffac4700 1 --2- 172.21.15.143:0/2624176922 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f28f81618f0 0x7f28f81f4140 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:41.158 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:41.154+0000 7f28ffac4700 1 --2- 172.21.15.143:0/2624176922 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f28f815dd00 0x7f28f81f3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:41.158 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:41.154+0000 7f28ffac4700 1 -- 172.21.15.143:0/2624176922 >> 172.21.15.143:0/2624176922 conn(0x7f28f8099ab0 msgr2=0x7f28f80ae2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:41.158 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:41.154+0000 7f28ffac4700 1 -- 172.21.15.143:0/2624176922 shutdown_connections 2023-11-08T19:29:41.159 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:41.156+0000 7f28ffac4700 1 -- 172.21.15.143:0/2624176922 wait complete. 2023-11-08T19:29:41.159 INFO:teuthology.orchestra.run.smithi143.stderr:failed mds gid 6906 2023-11-08T19:29:41.173 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-08T19:29:41.173 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-08T19:29:41.174 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-08T19:29:41.174 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T19:29:41.175 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-08T19:29:41.175 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi143 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c' 2023-11-08T19:29:41.176 INFO:tasks.ceph.mds.c:Stopped 2023-11-08T19:29:41.176 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-08T19:29:41.178 INFO:tasks.ceph.mds.c:Started 2023-11-08T19:29:41.179 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:29:41.257 INFO:tasks.ceph.mds.c.smithi143.stdout:starting mds.c at 2023-11-08T19:29:41.301 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:41.300+0000 7ffaef7cc700 1 Processor -- start 2023-11-08T19:29:41.302 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:41.300+0000 7ffaef7cc700 1 -- start start 2023-11-08T19:29:41.308 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:41.301+0000 7ffaef7cc700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7ffae806ed60 0x7ffae806b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:41.308 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:41.301+0000 7ffaef7cc700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ffae80cc410 0x7ffae80cc860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:41.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:41.301+0000 7ffaef7cc700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffae80b2a60 con 0x7ffae8159870 2023-11-08T19:29:41.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:41.301+0000 7ffaef7cc700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffae806b090 con 0x7ffae806ed60 2023-11-08T19:29:41.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:41.301+0000 7ffaef7cc700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffae80b2d40 con 0x7ffae80cc410 2023-11-08T19:29:41.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:41.301+0000 7ffaed568700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7ffae8159870 0x7ffae80b2210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:51902/0 (socket says 172.21.15.143:51902) 2023-11-08T19:29:41.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:41.301+0000 7ffaed568700 1 -- 172.21.15.143:0/2760417979 learned_addr learned my addr 172.21.15.143:0/2760417979 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:29:41.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:41.302+0000 7ffaedd69700 1 --2- 172.21.15.143:0/2760417979 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ffae80cc410 0x7ffae80cc860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:41.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:41.302+0000 7ffaecd67700 1 --2- 172.21.15.143:0/2760417979 >> v2:172.21.15.174:3300/0 conn(0x7ffae806ed60 0x7ffae806b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:41.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:41.302+0000 7ffadffff700 1 -- 172.21.15.143:0/2760417979 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1367575707 0 0) 0x7ffae80b2a60 con 0x7ffae8159870 2023-11-08T19:29:41.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:41.302+0000 7ffadffff700 1 -- 172.21.15.143:0/2760417979 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffad40035b0 con 0x7ffae8159870 2023-11-08T19:29:41.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:41.302+0000 7ffadffff700 1 -- 172.21.15.143:0/2760417979 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 139861087 0 0) 0x7ffad40035b0 con 0x7ffae8159870 2023-11-08T19:29:41.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:41.302+0000 7ffadffff700 1 -- 172.21.15.143:0/2760417979 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ffae80cc410 msgr2=0x7ffae80cc860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:41.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:41.302+0000 7ffadffff700 1 --2- 172.21.15.143:0/2760417979 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ffae80cc410 0x7ffae80cc860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:41.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:41.302+0000 7ffadffff700 1 -- 172.21.15.143:0/2760417979 >> v2:172.21.15.174:3300/0 conn(0x7ffae806ed60 msgr2=0x7ffae806b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:41.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:41.302+0000 7ffadffff700 1 --2- 172.21.15.143:0/2760417979 >> v2:172.21.15.174:3300/0 conn(0x7ffae806ed60 0x7ffae806b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:41.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:41.302+0000 7ffadffff700 1 -- 172.21.15.143:0/2760417979 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffae80b3020 con 0x7ffae8159870 2023-11-08T19:29:41.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:41.302+0000 7ffadffff700 1 -- 172.21.15.143:0/2760417979 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4147756303 0 0) 0x7ffad80017f0 con 0x7ffae8159870 2023-11-08T19:29:41.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:41.302+0000 7ffadffff700 1 -- 172.21.15.143:0/2760417979 >> v1:172.21.15.143:6789/0 conn(0x7ffae8159870 legacy=0x7ffae80b2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:41.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:41.302+0000 7ffadffff700 1 --2- 172.21.15.143:0/2760417979 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffad4003a00 0x7ffad4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:41.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:41.302+0000 7ffadffff700 1 --2- 172.21.15.143:0/2760417979 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffad4004850 0x7ffad4004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:41.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:41.302+0000 7ffadffff700 1 --2- 172.21.15.143:0/2760417979 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ffae80cc410 0x7ffad4005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:41.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:41.302+0000 7ffadffff700 1 -- 172.21.15.143:0/2760417979 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffae806b090 con 0x7ffad4003a00 2023-11-08T19:29:41.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:41.302+0000 7ffadffff700 1 -- 172.21.15.143:0/2760417979 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffae80b2d40 con 0x7ffad4004850 2023-11-08T19:29:41.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:41.302+0000 7ffadffff700 1 -- 172.21.15.143:0/2760417979 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffad400ba90 con 0x7ffae80cc410 2023-11-08T19:29:41.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:41.302+0000 7ffaecd67700 1 --2- 172.21.15.143:0/2760417979 >> v2:172.21.15.174:3300/0 conn(0x7ffae806ed60 0x7ffae806b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T19:29:41.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:41.302+0000 7ffaed568700 1 --2- 172.21.15.143:0/2760417979 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffad4003a00 0x7ffad4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:41.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:41.302+0000 7ffaedd69700 1 --2- 172.21.15.143:0/2760417979 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ffae80cc410 0x7ffad4005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:41.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:41.302+0000 7ffaecd67700 1 --2- 172.21.15.143:0/2760417979 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffad4004850 0x7ffad4004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:41.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:41.302+0000 7ffaed568700 1 -- 172.21.15.143:0/2760417979 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ffae80cc410 msgr2=0x7ffad4005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:41.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:41.302+0000 7ffaed568700 1 --2- 172.21.15.143:0/2760417979 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ffae80cc410 0x7ffad4005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:41.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:41.302+0000 7ffaed568700 1 -- 172.21.15.143:0/2760417979 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffad4004850 msgr2=0x7ffad4004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:41.315 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:41.302+0000 7ffaed568700 1 --2- 172.21.15.143:0/2760417979 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffad4004850 0x7ffad4004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:41.315 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:41.302+0000 7ffaed568700 1 -- 172.21.15.143:0/2760417979 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffad40035b0 con 0x7ffad4003a00 2023-11-08T19:29:41.315 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:41.302+0000 7ffaedd69700 1 --2- 172.21.15.143:0/2760417979 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ffae80cc410 0x7ffad4005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:29:41.315 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:41.303+0000 7ffaed568700 1 --2- 172.21.15.143:0/2760417979 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffad4003a00 0x7ffad4003e20 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7ffad800c320 tx=0x7ffad4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0e175fd608f4458 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:41.315 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:41.303+0000 7ffadffff700 1 -- 172.21.15.143:0/2760417979 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffad800ce60 con 0x7ffad4003a00 2023-11-08T19:29:41.316 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:41.303+0000 7ffadffff700 1 -- 172.21.15.143:0/2760417979 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffad8007bf0 con 0x7ffad4003a00 2023-11-08T19:29:41.316 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:41.303+0000 7ffadffff700 1 -- 172.21.15.143:0/2760417979 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffad8007310 con 0x7ffad4003a00 2023-11-08T19:29:41.316 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:41.303+0000 7ffaef7cc700 1 -- 172.21.15.143:0/2760417979 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffad4003a00 msgr2=0x7ffad4003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:41.316 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:41.303+0000 7ffaef7cc700 1 --2- 172.21.15.143:0/2760417979 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffad4003a00 0x7ffad4003e20 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7ffad800c320 tx=0x7ffad4005880 comp rx=0 tx=0).stop 2023-11-08T19:29:41.316 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:41.303+0000 7ffaedd69700 1 -- 172.21.15.143:0/2760417979 reap_dead start 2023-11-08T19:29:41.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:41.303+0000 7ffaef7cc700 1 -- 172.21.15.143:0/2760417979 shutdown_connections 2023-11-08T19:29:41.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:41.303+0000 7ffaef7cc700 1 -- 172.21.15.143:0/2760417979 >> 172.21.15.143:0/2760417979 conn(0x7ffae8099ab0 msgr2=0x7ffae80d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:41.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:41.303+0000 7ffaef7cc700 1 -- 172.21.15.143:0/2760417979 shutdown_connections 2023-11-08T19:29:41.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:41.303+0000 7ffaef7cc700 1 -- 172.21.15.143:0/2760417979 wait complete. 2023-11-08T19:29:41.318 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:41.305+0000 7ffaef7cc700 1 Processor -- start 2023-11-08T19:29:41.318 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:41.305+0000 7ffaef7cc700 1 -- start start 2023-11-08T19:29:41.318 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:41.305+0000 7ffaef7cc700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffad4003a00 0x7ffae81577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:41.318 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:41.305+0000 7ffaef7cc700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ffae8157d30 0x7ffae8159c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:41.318 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:41.305+0000 7ffaef7cc700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffae815a190 0x7ffae815a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:41.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:41.305+0000 7ffaef7cc700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffae81581b0 con 0x7ffae815a190 2023-11-08T19:29:41.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:41.305+0000 7ffaef7cc700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffae814e7c0 con 0x7ffad4003a00 2023-11-08T19:29:41.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:41.305+0000 7ffaef7cc700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffae80ae340 con 0x7ffae8157d30 2023-11-08T19:29:41.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:41.305+0000 7ffaedd69700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffae815a190 0x7ffae815a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:41.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:41.305+0000 7ffaedd69700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffae815a190 0x7ffae815a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55764/0 (socket says 172.21.15.143:55764) 2023-11-08T19:29:41.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:41.305+0000 7ffaedd69700 1 -- 172.21.15.143:0/222476998 learned_addr learned my addr 172.21.15.143:0/222476998 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:29:41.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:41.306+0000 7ffaed568700 1 --2- 172.21.15.143:0/222476998 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffad4003a00 0x7ffae81577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:41.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:41.306+0000 7ffaecd67700 1 --2- 172.21.15.143:0/222476998 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ffae8157d30 0x7ffae8159c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:41.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:41.306+0000 7ffaedd69700 1 -- 172.21.15.143:0/222476998 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ffae8157d30 msgr2=0x7ffae8159c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:41.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:41.306+0000 7ffaedd69700 1 --2- 172.21.15.143:0/222476998 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ffae8157d30 0x7ffae8159c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:41.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:41.306+0000 7ffaedd69700 1 -- 172.21.15.143:0/222476998 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffad4003a00 msgr2=0x7ffae81577f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:41.321 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:41.306+0000 7ffaedd69700 1 --2- 172.21.15.143:0/222476998 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffad4003a00 0x7ffae81577f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:41.321 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:41.306+0000 7ffaedd69700 1 -- 172.21.15.143:0/222476998 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffae814dbb0 con 0x7ffae815a190 2023-11-08T19:29:41.321 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:41.306+0000 7ffaecd67700 1 --2- 172.21.15.143:0/222476998 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ffae8157d30 0x7ffae8159c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:29:41.321 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:41.306+0000 7ffaedd69700 1 --2- 172.21.15.143:0/222476998 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffae815a190 0x7ffae815a610 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7ffae80cace0 tx=0x7ffae00104a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:41.321 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:41.306+0000 7ffaddffb700 1 -- 172.21.15.143:0/222476998 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffae001b070 con 0x7ffae815a190 2023-11-08T19:29:41.322 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:41.306+0000 7ffaddffb700 1 -- 172.21.15.143:0/222476998 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffae000f040 con 0x7ffae815a190 2023-11-08T19:29:41.322 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:41.306+0000 7ffaddffb700 1 -- 172.21.15.143:0/222476998 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffae000e050 con 0x7ffae815a190 2023-11-08T19:29:41.322 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:41.306+0000 7ffaef7cc700 1 -- 172.21.15.143:0/222476998 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffae814de90 con 0x7ffae815a190 2023-11-08T19:29:41.322 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:41.306+0000 7ffaef7cc700 1 -- 172.21.15.143:0/222476998 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffae814e450 con 0x7ffae815a190 2023-11-08T19:29:41.322 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:41.307+0000 7ffaef7cc700 1 -- 172.21.15.143:0/222476998 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffae804eee0 con 0x7ffae815a190 2023-11-08T19:29:41.323 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:41.308+0000 7ffaddffb700 1 -- 172.21.15.143:0/222476998 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ffae0015440 con 0x7ffae815a190 2023-11-08T19:29:41.324 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:41.308+0000 7ffaddffb700 1 --2- 172.21.15.143:0/222476998 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7ffacc06af00 0x7ffacc06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:41.324 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:41.308+0000 7ffaddffb700 1 -- 172.21.15.143:0/222476998 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 7059+0+0 (secure 0 0 0) 0x7ffae0089c20 con 0x7ffae815a190 2023-11-08T19:29:41.324 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:41.308+0000 7ffaed568700 1 --2- 172.21.15.143:0/222476998 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7ffacc06af00 0x7ffacc06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:41.324 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:41.309+0000 7ffaed568700 1 --2- 172.21.15.143:0/222476998 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7ffacc06af00 0x7ffacc06d3c0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7ffad8000ef0 tx=0x7ffad800a000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:41.325 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:41.313+0000 7ffaddffb700 1 -- 172.21.15.143:0/222476998 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ffae0054780 con 0x7ffae815a190 2023-11-08T19:29:41.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:41.540+0000 7ffaef7cc700 1 -- 172.21.15.143:0/222476998 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffae815d380 con 0x7ffae815a190 2023-11-08T19:29:41.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:41.540+0000 7ffaddffb700 1 -- 172.21.15.143:0/222476998 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) v1 ==== 94+0+4148 (secure 0 0 0) 0x7ffae0057da0 con 0x7ffae815a190 2023-11-08T19:29:41.543 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:29:41.543 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":76,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5933,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/3918286710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6835","nonce":3918286710},{"type":"v1","addr":"172.21.15.174:6837","nonce":3918286710}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:03.092245+0000","modified":"2023-11-08T19:29:41.154874+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":97,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7386,"mds_1":6918},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6918":{"gid":6918,"name":"b","rank":1,"incarnation":54,"state":"up:active","state_seq":6,"addr":"172.21.15.174:6836/3601088618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3601088618},{"type":"v1","addr":"172.21.15.174:6836","nonce":3601088618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7386":{"gid":7386,"name":"a","rank":0,"incarnation":76,"state":"up:replay","state_seq":1,"addr":"172.21.15.143:6835/3664584522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":3664584522},{"type":"v1","addr":"172.21.15.143:6835","nonce":3664584522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:29:41.544 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:41.542+0000 7ffaef7cc700 1 -- 172.21.15.143:0/222476998 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7ffacc06af00 msgr2=0x7ffacc06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:41.544 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:41.542+0000 7ffaef7cc700 1 --2- 172.21.15.143:0/222476998 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7ffacc06af00 0x7ffacc06d3c0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7ffad8000ef0 tx=0x7ffad800a000 comp rx=0 tx=0).stop 2023-11-08T19:29:41.545 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:41.542+0000 7ffaef7cc700 1 -- 172.21.15.143:0/222476998 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffae815a190 msgr2=0x7ffae815a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:41.545 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:41.542+0000 7ffaef7cc700 1 --2- 172.21.15.143:0/222476998 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffae815a190 0x7ffae815a610 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7ffae80cace0 tx=0x7ffae00104a0 comp rx=0 tx=0).stop 2023-11-08T19:29:41.545 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:41.542+0000 7ffaef7cc700 1 -- 172.21.15.143:0/222476998 shutdown_connections 2023-11-08T19:29:41.545 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:41.542+0000 7ffaef7cc700 1 --2- 172.21.15.143:0/222476998 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7ffacc06af00 0x7ffacc06d3c0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:41.546 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:41.542+0000 7ffaef7cc700 1 --2- 172.21.15.143:0/222476998 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffae815a190 0x7ffae815a610 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:41.546 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:41.542+0000 7ffaef7cc700 1 --2- 172.21.15.143:0/222476998 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffad4003a00 0x7ffae81577f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:41.546 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:41.542+0000 7ffaef7cc700 1 --2- 172.21.15.143:0/222476998 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ffae8157d30 0x7ffae8159c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:41.546 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:41.542+0000 7ffaef7cc700 1 -- 172.21.15.143:0/222476998 >> 172.21.15.143:0/222476998 conn(0x7ffae8099ab0 msgr2=0x7ffae80af880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:41.546 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:41.542+0000 7ffaef7cc700 1 -- 172.21.15.143:0/222476998 shutdown_connections 2023-11-08T19:29:41.547 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:41.542+0000 7ffaef7cc700 1 -- 172.21.15.143:0/222476998 wait complete. 2023-11-08T19:29:41.547 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 76 2023-11-08T19:29:41.556 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=30 retry_count=0)... 2023-11-08T19:29:43.467 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c has been restored 2023-11-08T19:29:45.958 INFO:tasks.ceph.mds.a.smithi143.stderr:2023-11-08T19:29:45.957+0000 7f98bc9d7700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-08T19:29:45.958 INFO:tasks.ceph.mds.a.smithi143.stderr:2023-11-08T19:29:45.957+0000 7f98bc9d7700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-11-08T19:29:46.558 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:29:46.692 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:46.691+0000 7ff78ed80700 1 Processor -- start 2023-11-08T19:29:46.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:46.691+0000 7ff78ed80700 1 -- start start 2023-11-08T19:29:46.695 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:46.692+0000 7ff78ed80700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7ff7880b2920 0x7ff7880cf510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:46.695 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:46.692+0000 7ff78ed80700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff7880cbe20 0x7ff7880cc290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:46.695 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:46.692+0000 7ff78ed80700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7880cf080 con 0x7ff78815aa30 2023-11-08T19:29:46.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:46.692+0000 7ff78ed80700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7880cc7d0 con 0x7ff7880b2920 2023-11-08T19:29:46.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:46.692+0000 7ff78ed80700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7880ce470 con 0x7ff7880cbe20 2023-11-08T19:29:46.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:46.692+0000 7ff78d31d700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff7880cbe20 0x7ff7880cc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:46.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:46.692+0000 7ff787fff700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7ff7880b2920 0x7ff7880cf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:46.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:46.692+0000 7ff78d31d700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff7880cbe20 0x7ff7880cc290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.143:43418/0 (socket says 172.21.15.143:43418) 2023-11-08T19:29:46.697 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:46.692+0000 7ff78d31d700 1 -- 172.21.15.143:0/3710814323 learned_addr learned my addr 172.21.15.143:0/3710814323 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:29:46.697 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:46.693+0000 7ff78d31d700 1 -- 172.21.15.143:0/3710814323 >> v2:172.21.15.174:3300/0 conn(0x7ff7880b2920 msgr2=0x7ff7880cf510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:46.697 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:46.693+0000 7ff78d31d700 1 --2- 172.21.15.143:0/3710814323 >> v2:172.21.15.174:3300/0 conn(0x7ff7880b2920 0x7ff7880cf510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:46.697 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:46.693+0000 7ff78d31d700 1 -- 172.21.15.143:0/3710814323 >> v1:172.21.15.174:6789/0 conn(0x7ff78815aa30 legacy=0x7ff7880b2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:46.697 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:46.693+0000 7ff78d31d700 1 -- 172.21.15.143:0/3710814323 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7880ce750 con 0x7ff7880cbe20 2023-11-08T19:29:46.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:46.693+0000 7ff787fff700 1 --2- 172.21.15.143:0/3710814323 >> v2:172.21.15.174:3300/0 conn(0x7ff7880b2920 0x7ff7880cf510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T19:29:46.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:46.693+0000 7ff78d31d700 1 --2- 172.21.15.143:0/3710814323 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff7880cbe20 0x7ff7880cc290 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff780009e50 tx=0x7ff78000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b43e2a97e9e9be4 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:46.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:46.693+0000 7ff7877fe700 1 -- 172.21.15.143:0/3710814323 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff78000e070 con 0x7ff7880cbe20 2023-11-08T19:29:46.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:46.693+0000 7ff7877fe700 1 -- 172.21.15.143:0/3710814323 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff78000b910 con 0x7ff7880cbe20 2023-11-08T19:29:46.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:46.693+0000 7ff7877fe700 1 -- 172.21.15.143:0/3710814323 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff78000bcc0 con 0x7ff7880cbe20 2023-11-08T19:29:46.699 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:46.693+0000 7ff78ed80700 1 -- 172.21.15.143:0/3710814323 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff7880cbe20 msgr2=0x7ff7880cc290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:46.699 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:46.693+0000 7ff78ed80700 1 --2- 172.21.15.143:0/3710814323 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff7880cbe20 0x7ff7880cc290 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff780009e50 tx=0x7ff78000b330 comp rx=0 tx=0).stop 2023-11-08T19:29:46.699 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:46.694+0000 7ff78ed80700 1 -- 172.21.15.143:0/3710814323 shutdown_connections 2023-11-08T19:29:46.699 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:46.694+0000 7ff78ed80700 1 --2- 172.21.15.143:0/3710814323 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff7880cbe20 0x7ff7880cc290 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:46.699 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:46.694+0000 7ff78ed80700 1 --2- 172.21.15.143:0/3710814323 >> v2:172.21.15.174:3300/0 conn(0x7ff7880b2920 0x7ff7880cf510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:46.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:46.694+0000 7ff78ed80700 1 -- 172.21.15.143:0/3710814323 >> 172.21.15.143:0/3710814323 conn(0x7ff788099ab0 msgr2=0x7ff7880d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:46.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:46.694+0000 7ff78ed80700 1 -- 172.21.15.143:0/3710814323 shutdown_connections 2023-11-08T19:29:46.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:46.694+0000 7ff78ed80700 1 -- 172.21.15.143:0/3710814323 wait complete. 2023-11-08T19:29:46.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:46.695+0000 7ff78ed80700 1 Processor -- start 2023-11-08T19:29:46.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:46.695+0000 7ff78ed80700 1 -- start start 2023-11-08T19:29:46.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:46.696+0000 7ff78ed80700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff7880b2920 0x7ff7881eebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:46.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:46.696+0000 7ff78ed80700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff7880cbe20 0x7ff78815b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:46.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:46.696+0000 7ff78ed80700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff78815aa30 0x7ff78815b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:46.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:46.696+0000 7ff78ed80700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff78815bf30 con 0x7ff78815aa30 2023-11-08T19:29:46.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:46.696+0000 7ff78ed80700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7880b62e0 con 0x7ff7880cbe20 2023-11-08T19:29:46.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:46.696+0000 7ff78ed80700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff78815bc50 con 0x7ff7880b2920 2023-11-08T19:29:46.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:46.696+0000 7ff78d31d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff78815aa30 0x7ff78815b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:46.705 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:46.696+0000 7ff78d31d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff78815aa30 0x7ff78815b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55786/0 (socket says 172.21.15.143:55786) 2023-11-08T19:29:46.705 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:46.696+0000 7ff78d31d700 1 -- 172.21.15.143:0/196530242 learned_addr learned my addr 172.21.15.143:0/196530242 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:29:46.705 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:46.696+0000 7ff78cb1c700 1 --2- 172.21.15.143:0/196530242 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff7880b2920 0x7ff7881eebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:46.705 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:46.696+0000 7ff787fff700 1 --2- 172.21.15.143:0/196530242 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff7880cbe20 0x7ff78815b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:46.705 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:46.696+0000 7ff78d31d700 1 -- 172.21.15.143:0/196530242 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff7880b2920 msgr2=0x7ff7881eebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:46.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:46.696+0000 7ff78d31d700 1 --2- 172.21.15.143:0/196530242 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff7880b2920 0x7ff7881eebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:46.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:46.696+0000 7ff78d31d700 1 -- 172.21.15.143:0/196530242 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff7880cbe20 msgr2=0x7ff78815b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:46.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:46.696+0000 7ff78d31d700 1 --2- 172.21.15.143:0/196530242 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff7880cbe20 0x7ff78815b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:46.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:46.696+0000 7ff78d31d700 1 -- 172.21.15.143:0/196530242 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff77400b740 con 0x7ff78815aa30 2023-11-08T19:29:46.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:46.697+0000 7ff78d31d700 1 --2- 172.21.15.143:0/196530242 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff78815aa30 0x7ff78815b620 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7ff7880b2f40 tx=0x7ff780004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:46.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:46.697+0000 7ff7857fa700 1 -- 172.21.15.143:0/196530242 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff780004800 con 0x7ff78815aa30 2023-11-08T19:29:46.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:46.697+0000 7ff7857fa700 1 -- 172.21.15.143:0/196530242 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff780004960 con 0x7ff78815aa30 2023-11-08T19:29:46.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:46.697+0000 7ff78ed80700 1 -- 172.21.15.143:0/196530242 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff780009b30 con 0x7ff78815aa30 2023-11-08T19:29:46.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:46.697+0000 7ff7857fa700 1 -- 172.21.15.143:0/196530242 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff78001c680 con 0x7ff78815aa30 2023-11-08T19:29:46.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:46.697+0000 7ff78ed80700 1 -- 172.21.15.143:0/196530242 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7880c5080 con 0x7ff78815aa30 2023-11-08T19:29:46.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:46.698+0000 7ff78ed80700 1 -- 172.21.15.143:0/196530242 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7880c4c00 con 0x7ff78815aa30 2023-11-08T19:29:46.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:46.698+0000 7ff7857fa700 1 -- 172.21.15.143:0/196530242 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff78001c7e0 con 0x7ff78815aa30 2023-11-08T19:29:46.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:46.698+0000 7ff7857fa700 1 --2- 172.21.15.143:0/196530242 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7ff76c06af00 0x7ff76c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:46.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:46.699+0000 7ff7857fa700 1 -- 172.21.15.143:0/196530242 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 7059+0+0 (secure 0 0 0) 0x7ff78008b740 con 0x7ff78815aa30 2023-11-08T19:29:46.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:46.699+0000 7ff78cb1c700 1 --2- 172.21.15.143:0/196530242 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7ff76c06af00 0x7ff76c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:46.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:46.699+0000 7ff78cb1c700 1 --2- 172.21.15.143:0/196530242 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7ff76c06af00 0x7ff76c06d3c0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7ff7880af8a0 tx=0x7ff77c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:46.709 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:46.703+0000 7ff7857fa700 1 -- 172.21.15.143:0/196530242 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff7880c4c00 con 0x7ff78815aa30 2023-11-08T19:29:46.936 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:46.934+0000 7ff78ed80700 1 -- 172.21.15.143:0/196530242 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff7880c4c00 con 0x7ff78815aa30 2023-11-08T19:29:46.936 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:46.935+0000 7ff7857fa700 1 -- 172.21.15.143:0/196530242 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+4943 (secure 0 0 0) 0x7ff7880c4c00 con 0x7ff78815aa30 2023-11-08T19:29:46.937 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:29:46.937 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":80,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5933,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/3918286710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6835","nonce":3918286710},{"type":"v1","addr":"172.21.15.174:6837","nonce":3918286710}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":7494,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6834/1535103237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":1535103237},{"type":"v1","addr":"172.21.15.143:6834","nonce":1535103237}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:03.092245+0000","modified":"2023-11-08T19:29:44.172957+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":97,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7386,"mds_1":6918},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6918":{"gid":6918,"name":"b","rank":1,"incarnation":54,"state":"up:active","state_seq":6,"addr":"172.21.15.174:6836/3601088618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3601088618},{"type":"v1","addr":"172.21.15.174:6836","nonce":3601088618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7386":{"gid":7386,"name":"a","rank":0,"incarnation":76,"state":"up:active","state_seq":11,"addr":"172.21.15.143:6835/3664584522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":3664584522},{"type":"v1","addr":"172.21.15.143:6835","nonce":3664584522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:29:46.938 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:46.937+0000 7ff78ed80700 1 -- 172.21.15.143:0/196530242 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7ff76c06af00 msgr2=0x7ff76c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:46.939 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:46.937+0000 7ff78ed80700 1 --2- 172.21.15.143:0/196530242 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7ff76c06af00 0x7ff76c06d3c0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7ff7880af8a0 tx=0x7ff77c009040 comp rx=0 tx=0).stop 2023-11-08T19:29:46.939 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:46.937+0000 7ff78ed80700 1 -- 172.21.15.143:0/196530242 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff78815aa30 msgr2=0x7ff78815b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:46.939 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:46.937+0000 7ff78ed80700 1 --2- 172.21.15.143:0/196530242 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff78815aa30 0x7ff78815b620 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7ff7880b2f40 tx=0x7ff780004ad0 comp rx=0 tx=0).stop 2023-11-08T19:29:46.939 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:46.937+0000 7ff78ed80700 1 -- 172.21.15.143:0/196530242 shutdown_connections 2023-11-08T19:29:46.939 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:46.937+0000 7ff78ed80700 1 --2- 172.21.15.143:0/196530242 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7ff76c06af00 0x7ff76c06d3c0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:46.940 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:46.937+0000 7ff78ed80700 1 --2- 172.21.15.143:0/196530242 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff7880b2920 0x7ff7881eebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:46.940 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:46.937+0000 7ff78ed80700 1 --2- 172.21.15.143:0/196530242 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff78815aa30 0x7ff78815b620 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:46.940 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:46.937+0000 7ff78ed80700 1 --2- 172.21.15.143:0/196530242 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff7880cbe20 0x7ff78815b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:46.940 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:46.937+0000 7ff78ed80700 1 -- 172.21.15.143:0/196530242 >> 172.21.15.143:0/196530242 conn(0x7ff788099ab0 msgr2=0x7ff7880cf510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:46.940 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:46.937+0000 7ff78ed80700 1 -- 172.21.15.143:0/196530242 shutdown_connections 2023-11-08T19:29:46.941 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:46.937+0000 7ff78ed80700 1 -- 172.21.15.143:0/196530242 wait complete. 2023-11-08T19:29:46.941 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 80 2023-11-08T19:29:46.950 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2023-11-08T19:29:46.951 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:29:47.071 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.070+0000 7f093e9bf700 1 Processor -- start 2023-11-08T19:29:47.071 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.070+0000 7f093e9bf700 1 -- start start 2023-11-08T19:29:47.074 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.071+0000 7f093e9bf700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f09380b2920 0x7f09380cf510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:47.074 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.071+0000 7f093e9bf700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f09380cbe20 0x7f09380cc290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:47.074 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.071+0000 7f093e9bf700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09380cf080 con 0x7f093815aa30 2023-11-08T19:29:47.074 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.071+0000 7f093e9bf700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09380cc7d0 con 0x7f09380b2920 2023-11-08T19:29:47.075 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.071+0000 7f093e9bf700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09380ce470 con 0x7f09380cbe20 2023-11-08T19:29:47.075 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.071+0000 7f0937fff700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f093815aa30 0x7f09380b2210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:51944/0 (socket says 172.21.15.143:51944) 2023-11-08T19:29:47.075 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.071+0000 7f0937fff700 1 -- 172.21.15.143:0/2067871482 learned_addr learned my addr 172.21.15.143:0/2067871482 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:29:47.075 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.071+0000 7f093cf5c700 1 --2- 172.21.15.143:0/2067871482 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f09380cbe20 0x7f09380cc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:47.075 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.071+0000 7f09377fe700 1 --2- 172.21.15.143:0/2067871482 >> v2:172.21.15.174:3300/0 conn(0x7f09380b2920 0x7f09380cf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:47.076 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.071+0000 7f0936ffd700 1 -- 172.21.15.143:0/2067871482 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2428272102 0 0) 0x7f09380cf080 con 0x7f093815aa30 2023-11-08T19:29:47.077 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.071+0000 7f0936ffd700 1 -- 172.21.15.143:0/2067871482 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f09240035b0 con 0x7f093815aa30 2023-11-08T19:29:47.077 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.071+0000 7f0936ffd700 1 -- 172.21.15.143:0/2067871482 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4140987092 0 0) 0x7f09240035b0 con 0x7f093815aa30 2023-11-08T19:29:47.077 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.071+0000 7f093cf5c700 1 -- 172.21.15.143:0/2067871482 >> v2:172.21.15.174:3300/0 conn(0x7f09380b2920 msgr2=0x7f09380cf510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:47.077 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.071+0000 7f093cf5c700 1 --2- 172.21.15.143:0/2067871482 >> v2:172.21.15.174:3300/0 conn(0x7f09380b2920 0x7f09380cf510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:47.077 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.071+0000 7f093cf5c700 1 -- 172.21.15.143:0/2067871482 >> v1:172.21.15.143:6789/0 conn(0x7f093815aa30 legacy=0x7f09380b2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:47.078 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.071+0000 7f093cf5c700 1 -- 172.21.15.143:0/2067871482 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09380ce750 con 0x7f09380cbe20 2023-11-08T19:29:47.078 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.072+0000 7f093cf5c700 1 --2- 172.21.15.143:0/2067871482 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f09380cbe20 0x7f09380cc290 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f0930009d90 tx=0x7f093000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=473148a6cc6d8340 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:47.078 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.072+0000 7f0936ffd700 1 -- 172.21.15.143:0/2067871482 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f093000e070 con 0x7f09380cbe20 2023-11-08T19:29:47.078 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.072+0000 7f0936ffd700 1 -- 172.21.15.143:0/2067871482 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0930003a20 con 0x7f09380cbe20 2023-11-08T19:29:47.078 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.072+0000 7f0936ffd700 1 -- 172.21.15.143:0/2067871482 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09280017f0 con 0x7f09380cbe20 2023-11-08T19:29:47.079 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.072+0000 7f093e9bf700 1 -- 172.21.15.143:0/2067871482 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f09380cbe20 msgr2=0x7f09380cc290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:47.079 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.072+0000 7f093e9bf700 1 --2- 172.21.15.143:0/2067871482 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f09380cbe20 0x7f09380cc290 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f0930009d90 tx=0x7f093000b330 comp rx=0 tx=0).stop 2023-11-08T19:29:47.079 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.072+0000 7f093e9bf700 1 -- 172.21.15.143:0/2067871482 shutdown_connections 2023-11-08T19:29:47.079 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.072+0000 7f093e9bf700 1 --2- 172.21.15.143:0/2067871482 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f09380cbe20 0x7f09380cc290 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:47.079 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.072+0000 7f093e9bf700 1 --2- 172.21.15.143:0/2067871482 >> v2:172.21.15.174:3300/0 conn(0x7f09380b2920 0x7f09380cf510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:47.080 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.072+0000 7f093e9bf700 1 -- 172.21.15.143:0/2067871482 >> 172.21.15.143:0/2067871482 conn(0x7f0938099ab0 msgr2=0x7f09380d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:47.080 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.072+0000 7f093e9bf700 1 -- 172.21.15.143:0/2067871482 shutdown_connections 2023-11-08T19:29:47.080 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.072+0000 7f093e9bf700 1 -- 172.21.15.143:0/2067871482 wait complete. 2023-11-08T19:29:47.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.073+0000 7f093e9bf700 1 Processor -- start 2023-11-08T19:29:47.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.074+0000 7f093e9bf700 1 -- start start 2023-11-08T19:29:47.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.074+0000 7f093e9bf700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f09380b2920 0x7f09380c9d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:47.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.074+0000 7f093e9bf700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f09380cbe20 0x7f09380c7840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:47.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.074+0000 7f093e9bf700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f093815aa30 0x7f093815da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:47.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.074+0000 7f093e9bf700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09380ca3d0 con 0x7f09380b2920 2023-11-08T19:29:47.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.074+0000 7f093e9bf700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09380ca6b0 con 0x7f09380cbe20 2023-11-08T19:29:47.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.074+0000 7f093e9bf700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f093815b400 con 0x7f093815aa30 2023-11-08T19:29:47.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.074+0000 7f0937fff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f09380b2920 0x7f09380c9d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:47.085 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.074+0000 7f0937fff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f09380b2920 0x7f09380c9d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55796/0 (socket says 172.21.15.143:55796) 2023-11-08T19:29:47.085 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.074+0000 7f0937fff700 1 -- 172.21.15.143:0/4039199686 learned_addr learned my addr 172.21.15.143:0/4039199686 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:29:47.085 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.075+0000 7f093cf5c700 1 --2- 172.21.15.143:0/4039199686 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f093815aa30 0x7f093815da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:47.085 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.075+0000 7f0937fff700 1 -- 172.21.15.143:0/4039199686 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f093815aa30 msgr2=0x7f093815da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:47.085 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.075+0000 7f0937fff700 1 --2- 172.21.15.143:0/4039199686 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f093815aa30 0x7f093815da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:47.086 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.075+0000 7f0937fff700 1 -- 172.21.15.143:0/4039199686 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f09380cbe20 msgr2=0x7f09380c7840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:47.086 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.075+0000 7f09377fe700 1 --2- 172.21.15.143:0/4039199686 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f09380cbe20 0x7f09380c7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:47.086 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.075+0000 7f0937fff700 1 --2- 172.21.15.143:0/4039199686 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f09380cbe20 0x7f09380c7840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:47.086 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.075+0000 7f0937fff700 1 -- 172.21.15.143:0/4039199686 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0930009a70 con 0x7f09380b2920 2023-11-08T19:29:47.086 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.075+0000 7f0937fff700 1 --2- 172.21.15.143:0/4039199686 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f09380b2920 0x7f09380c9d60 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f092800c6d0 tx=0x7f092800ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:47.087 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.075+0000 7f0934ff9700 1 -- 172.21.15.143:0/4039199686 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09280074e0 con 0x7f09380b2920 2023-11-08T19:29:47.087 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.075+0000 7f0934ff9700 1 -- 172.21.15.143:0/4039199686 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0928007640 con 0x7f09380b2920 2023-11-08T19:29:47.087 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.075+0000 7f0934ff9700 1 -- 172.21.15.143:0/4039199686 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0928005a70 con 0x7f09380b2920 2023-11-08T19:29:47.087 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.075+0000 7f093e9bf700 1 -- 172.21.15.143:0/4039199686 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f093815b6e0 con 0x7f09380b2920 2023-11-08T19:29:47.088 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.075+0000 7f093e9bf700 1 -- 172.21.15.143:0/4039199686 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09380c7d80 con 0x7f09380b2920 2023-11-08T19:29:47.089 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.076+0000 7f093e9bf700 1 -- 172.21.15.143:0/4039199686 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f093815bc00 con 0x7f09380b2920 2023-11-08T19:29:47.089 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.076+0000 7f0934ff9700 1 -- 172.21.15.143:0/4039199686 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0928005bd0 con 0x7f09380b2920 2023-11-08T19:29:47.090 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.077+0000 7f0934ff9700 1 --2- 172.21.15.143:0/4039199686 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f091806af00 0x7f091806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:47.090 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.077+0000 7f0934ff9700 1 -- 172.21.15.143:0/4039199686 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 7059+0+0 (secure 0 0 0) 0x7f0928089050 con 0x7f09380b2920 2023-11-08T19:29:47.091 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.077+0000 7f09377fe700 1 --2- 172.21.15.143:0/4039199686 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f091806af00 0x7f091806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:47.091 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.077+0000 7f09377fe700 1 --2- 172.21.15.143:0/4039199686 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f091806af00 0x7f091806d3c0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f0938159050 tx=0x7f0920008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:47.091 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.081+0000 7f0934ff9700 1 -- 172.21.15.143:0/4039199686 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f093815bc00 con 0x7f09380b2920 2023-11-08T19:29:47.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.313+0000 7f093e9bf700 1 -- 172.21.15.143:0/4039199686 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f093804eee0 con 0x7f09380b2920 2023-11-08T19:29:47.315 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.313+0000 7f0934ff9700 1 -- 172.21.15.143:0/4039199686 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f0928017090 con 0x7f09380b2920 2023-11-08T19:29:47.315 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:29:47.316 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":80,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5933,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/3918286710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6835","nonce":3918286710},{"type":"v1","addr":"172.21.15.174:6837","nonce":3918286710}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":7494,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6834/1535103237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":1535103237},{"type":"v1","addr":"172.21.15.143:6834","nonce":1535103237}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:03.092245+0000","modified":"2023-11-08T19:29:44.172957+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":97,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7386,"mds_1":6918},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6918":{"gid":6918,"name":"b","rank":1,"incarnation":54,"state":"up:active","state_seq":6,"addr":"172.21.15.174:6836/3601088618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3601088618},{"type":"v1","addr":"172.21.15.174:6836","nonce":3601088618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7386":{"gid":7386,"name":"a","rank":0,"incarnation":76,"state":"up:active","state_seq":11,"addr":"172.21.15.143:6835/3664584522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":3664584522},{"type":"v1","addr":"172.21.15.143:6835","nonce":3664584522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:29:47.318 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.316+0000 7f093e9bf700 1 -- 172.21.15.143:0/4039199686 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f091806af00 msgr2=0x7f091806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:47.318 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.316+0000 7f093e9bf700 1 --2- 172.21.15.143:0/4039199686 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f091806af00 0x7f091806d3c0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f0938159050 tx=0x7f0920008040 comp rx=0 tx=0).stop 2023-11-08T19:29:47.318 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.316+0000 7f093e9bf700 1 -- 172.21.15.143:0/4039199686 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f09380b2920 msgr2=0x7f09380c9d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:47.318 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.316+0000 7f093e9bf700 1 --2- 172.21.15.143:0/4039199686 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f09380b2920 0x7f09380c9d60 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f092800c6d0 tx=0x7f092800ca90 comp rx=0 tx=0).stop 2023-11-08T19:29:47.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.316+0000 7f093e9bf700 1 -- 172.21.15.143:0/4039199686 shutdown_connections 2023-11-08T19:29:47.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.316+0000 7f093e9bf700 1 --2- 172.21.15.143:0/4039199686 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f091806af00 0x7f091806d3c0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:47.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.316+0000 7f093e9bf700 1 --2- 172.21.15.143:0/4039199686 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f093815aa30 0x7f093815da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:47.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.316+0000 7f093e9bf700 1 --2- 172.21.15.143:0/4039199686 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f09380b2920 0x7f09380c9d60 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:47.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.316+0000 7f093e9bf700 1 --2- 172.21.15.143:0/4039199686 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f09380cbe20 0x7f09380c7840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:47.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.316+0000 7f093e9bf700 1 -- 172.21.15.143:0/4039199686 >> 172.21.15.143:0/4039199686 conn(0x7f0938099ab0 msgr2=0x7f09380af490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:47.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.316+0000 7f093e9bf700 1 -- 172.21.15.143:0/4039199686 shutdown_connections 2023-11-08T19:29:47.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.316+0000 7f093e9bf700 1 -- 172.21.15.143:0/4039199686 wait complete. 2023-11-08T19:29:47.321 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 80 2023-11-08T19:29:47.732 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 80, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 2, 'explicitly_allowed_features': 2, 'created': '2023-11-08T19:28:03.092245+0000', 'modified': '2023-11-08T19:29:44.172957+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 97, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 2, 'in': [0, 1], 'up': {'mds_0': 7386, 'mds_1': 6918}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6918': {'gid': 6918, 'name': 'b', 'rank': 1, 'incarnation': 54, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.174:6836/3601088618', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.174:6834', 'nonce': 3601088618}, {'type': 'v1', 'addr': '172.21.15.174:6836', 'nonce': 3601088618}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_7386': {'gid': 7386, 'name': 'a', 'rank': 0, 'incarnation': 76, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.143:6835/3664584522', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.143:6833', 'nonce': 3664584522}, {'type': 'v1', 'addr': '172.21.15.143:6835', 'nonce': 3664584522}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-08T19:29:47.732 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 2/2 2023-11-08T19:29:47.733 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-11-08T19:29:47.855 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.853+0000 7f40e2627700 1 Processor -- start 2023-11-08T19:29:47.855 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.854+0000 7f40e2627700 1 -- start start 2023-11-08T19:29:47.857 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.855+0000 7f40e2627700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f40dc15ba30 0x7f40dc0b14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:47.858 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.855+0000 7f40e2627700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f40dc0d1c70 0x7f40dc0cc5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:47.858 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.855+0000 7f40e2627700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40dc0b1bc0 con 0x7f40dc0d03a0 2023-11-08T19:29:47.858 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.855+0000 7f40e2627700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40dc0b1ea0 con 0x7f40dc15ba30 2023-11-08T19:29:47.858 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.855+0000 7f40e2627700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40dc0b2180 con 0x7f40dc0d1c70 2023-11-08T19:29:47.858 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.855+0000 7f40dbfff700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f40dc15ba30 0x7f40dc0b14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:47.859 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.855+0000 7f40db7fe700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f40dc0d03a0 0x7f40dc0d1560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:51960/0 (socket says 172.21.15.143:51960) 2023-11-08T19:29:47.859 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.855+0000 7f40db7fe700 1 -- 172.21.15.143:0/4232437812 learned_addr learned my addr 172.21.15.143:0/4232437812 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:29:47.859 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.855+0000 7f40dbfff700 1 -- 172.21.15.143:0/4232437812 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f40dc0d1c70 msgr2=0x7f40dc0cc5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:47.859 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.855+0000 7f40dbfff700 1 --2- 172.21.15.143:0/4232437812 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f40dc0d1c70 0x7f40dc0cc5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:47.859 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.855+0000 7f40dbfff700 1 -- 172.21.15.143:0/4232437812 >> v1:172.21.15.143:6789/0 conn(0x7f40dc0d03a0 legacy=0x7f40dc0d1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:47.860 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.855+0000 7f40dbfff700 1 -- 172.21.15.143:0/4232437812 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40dc0b2460 con 0x7f40dc15ba30 2023-11-08T19:29:47.860 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.855+0000 7f40daffd700 1 -- 172.21.15.143:0/4232437812 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 211538770 0 0) 0x7f40dc0b1bc0 con 0x7f40dc0d03a0 2023-11-08T19:29:47.860 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.855+0000 7f40dbfff700 1 --2- 172.21.15.143:0/4232437812 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f40dc15ba30 0x7f40dc0b14c0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f40c4009840 tx=0x7f40c4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37f4004088a9d0d6 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:47.860 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.855+0000 7f40daffd700 1 -- 172.21.15.143:0/4232437812 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40c4011830 con 0x7f40dc15ba30 2023-11-08T19:29:47.860 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.855+0000 7f40daffd700 1 -- 172.21.15.143:0/4232437812 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40c4011990 con 0x7f40dc15ba30 2023-11-08T19:29:47.861 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.855+0000 7f40daffd700 1 -- 172.21.15.143:0/4232437812 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40c4011d20 con 0x7f40dc15ba30 2023-11-08T19:29:47.861 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.856+0000 7f40e2627700 1 -- 172.21.15.143:0/4232437812 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f40dc15ba30 msgr2=0x7f40dc0b14c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:47.861 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.856+0000 7f40e2627700 1 --2- 172.21.15.143:0/4232437812 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f40dc15ba30 0x7f40dc0b14c0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f40c4009840 tx=0x7f40c4009b50 comp rx=0 tx=0).stop 2023-11-08T19:29:47.861 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.856+0000 7f40e2627700 1 -- 172.21.15.143:0/4232437812 shutdown_connections 2023-11-08T19:29:47.861 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.856+0000 7f40e2627700 1 --2- 172.21.15.143:0/4232437812 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f40dc0d1c70 0x7f40dc0cc5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:47.862 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.856+0000 7f40e2627700 1 --2- 172.21.15.143:0/4232437812 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f40dc15ba30 0x7f40dc0b14c0 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:47.862 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.856+0000 7f40e2627700 1 -- 172.21.15.143:0/4232437812 >> 172.21.15.143:0/4232437812 conn(0x7f40dc08b000 msgr2=0x7f40dc09ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:47.862 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.856+0000 7f40e2627700 1 -- 172.21.15.143:0/4232437812 shutdown_connections 2023-11-08T19:29:47.862 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.856+0000 7f40e2627700 1 -- 172.21.15.143:0/4232437812 wait complete. 2023-11-08T19:29:47.871 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.857+0000 7f40e2627700 1 Processor -- start 2023-11-08T19:29:47.871 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.857+0000 7f40e2627700 1 -- start start 2023-11-08T19:29:47.871 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.858+0000 7f40e2627700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f40dc0d03a0 0x7f40dc152310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:47.871 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.858+0000 7f40e2627700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f40dc0d1c70 0x7f40dc1590a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:47.872 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.858+0000 7f40e2627700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f40dc15ba30 0x7f40dc1595e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:47.872 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.858+0000 7f40e2627700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40dc0b4970 con 0x7f40dc0d1c70 2023-11-08T19:29:47.872 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.858+0000 7f40e2627700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40dc15efc0 con 0x7f40dc15ba30 2023-11-08T19:29:47.872 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.858+0000 7f40e2627700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40dc15ea00 con 0x7f40dc0d03a0 2023-11-08T19:29:47.872 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.858+0000 7f40db7fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f40dc0d1c70 0x7f40dc1590a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:47.873 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.858+0000 7f40db7fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f40dc0d1c70 0x7f40dc1590a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55812/0 (socket says 172.21.15.143:55812) 2023-11-08T19:29:47.873 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.858+0000 7f40db7fe700 1 -- 172.21.15.143:0/3257767738 learned_addr learned my addr 172.21.15.143:0/3257767738 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:29:47.873 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.858+0000 7f40dbfff700 1 --2- 172.21.15.143:0/3257767738 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f40dc0d03a0 0x7f40dc152310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:47.873 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.858+0000 7f40e0bc4700 1 --2- 172.21.15.143:0/3257767738 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f40dc15ba30 0x7f40dc1595e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:47.873 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.859+0000 7f40db7fe700 1 -- 172.21.15.143:0/3257767738 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f40dc0d03a0 msgr2=0x7f40dc152310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:47.874 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.859+0000 7f40db7fe700 1 --2- 172.21.15.143:0/3257767738 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f40dc0d03a0 0x7f40dc152310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:47.874 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.859+0000 7f40db7fe700 1 -- 172.21.15.143:0/3257767738 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f40dc15ba30 msgr2=0x7f40dc1595e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:47.874 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.859+0000 7f40db7fe700 1 --2- 172.21.15.143:0/3257767738 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f40dc15ba30 0x7f40dc1595e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:47.874 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.859+0000 7f40db7fe700 1 -- 172.21.15.143:0/3257767738 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40c40094a0 con 0x7f40dc0d1c70 2023-11-08T19:29:47.874 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.859+0000 7f40db7fe700 1 --2- 172.21.15.143:0/3257767738 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f40dc0d1c70 0x7f40dc1590a0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f40cc0011e0 tx=0x7f40cc00cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:47.875 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.859+0000 7f40d8ff9700 1 -- 172.21.15.143:0/3257767738 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40cc004c80 con 0x7f40dc0d1c70 2023-11-08T19:29:47.875 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.859+0000 7f40d8ff9700 1 -- 172.21.15.143:0/3257767738 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40cc004e00 con 0x7f40dc0d1c70 2023-11-08T19:29:47.875 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.859+0000 7f40e2627700 1 -- 172.21.15.143:0/3257767738 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40dc15ece0 con 0x7f40dc0d1c70 2023-11-08T19:29:47.875 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.859+0000 7f40d8ff9700 1 -- 172.21.15.143:0/3257767738 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40cc007310 con 0x7f40dc0d1c70 2023-11-08T19:29:47.875 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.859+0000 7f40e2627700 1 -- 172.21.15.143:0/3257767738 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40dc15dca0 con 0x7f40dc0d1c70 2023-11-08T19:29:47.876 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.860+0000 7f40d8ff9700 1 -- 172.21.15.143:0/3257767738 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f40cc007470 con 0x7f40dc0d1c70 2023-11-08T19:29:47.876 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.861+0000 7f40d8ff9700 1 --2- 172.21.15.143:0/3257767738 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f40bc06af00 0x7f40bc06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:47.876 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.861+0000 7f40d8ff9700 1 -- 172.21.15.143:0/3257767738 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 7059+0+0 (secure 0 0 0) 0x7f40cc089840 con 0x7f40dc0d1c70 2023-11-08T19:29:47.876 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.861+0000 7f40dbfff700 1 --2- 172.21.15.143:0/3257767738 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f40bc06af00 0x7f40bc06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:47.876 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.861+0000 7f40dbfff700 1 --2- 172.21.15.143:0/3257767738 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f40bc06af00 0x7f40bc06d3c0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f40c40090b0 tx=0x7f40c401c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:47.876 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.863+0000 7f40ee820b80 1 Processor -- start 2023-11-08T19:29:47.877 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.863+0000 7f40ee820b80 1 -- start start 2023-11-08T19:29:47.877 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.863+0000 7f40ee820b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556f236d4b30 0x556f236b7450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:47.877 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.863+0000 7f40ee820b80 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x556f236d55d0 0x556f236b7990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:47.877 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.863+0000 7f40ee820b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f2368b450 0x556f236874c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:47.877 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.863+0000 7f40ee820b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f235f36e0 con 0x556f236d4b30 2023-11-08T19:29:47.878 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.863+0000 7f40ee820b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f235e73b0 con 0x556f2368b450 2023-11-08T19:29:47.878 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.863+0000 7f40ee820b80 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x556f235e4cd0 con 0x556f236d55d0 2023-11-08T19:29:47.878 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.864+0000 7f40dbfff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556f236d4b30 0x556f236b7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:47.878 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.864+0000 7f40dbfff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556f236d4b30 0x556f236b7450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55818/0 (socket says 172.21.15.143:55818) 2023-11-08T19:29:47.878 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.864+0000 7f40dbfff700 1 -- 172.21.15.143:0/2540274932 learned_addr learned my addr 172.21.15.143:0/2540274932 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:29:47.879 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.864+0000 7f40db7fe700 1 --2- 172.21.15.143:0/2540274932 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x556f236d55d0 0x556f236b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:47.879 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.864+0000 7f40e0bc4700 1 --2- 172.21.15.143:0/2540274932 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f2368b450 0x556f236874c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:47.879 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.864+0000 7f40dbfff700 1 -- 172.21.15.143:0/2540274932 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x556f236d55d0 msgr2=0x556f236b7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:47.879 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.864+0000 7f40dbfff700 1 --2- 172.21.15.143:0/2540274932 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x556f236d55d0 0x556f236b7990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:47.879 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.864+0000 7f40dbfff700 1 -- 172.21.15.143:0/2540274932 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f2368b450 msgr2=0x556f236874c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:47.880 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.864+0000 7f40dbfff700 1 --2- 172.21.15.143:0/2540274932 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f2368b450 0x556f236874c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:47.880 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.864+0000 7f40dbfff700 1 -- 172.21.15.143:0/2540274932 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556f2345d5e0 con 0x556f236d4b30 2023-11-08T19:29:47.880 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.864+0000 7f40dbfff700 1 --2- 172.21.15.143:0/2540274932 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556f236d4b30 0x556f236b7450 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f40c4011860 tx=0x7f40c4010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6a15945db566f5a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:47.880 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.864+0000 7f40d1ffb700 1 -- 172.21.15.143:0/2540274932 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40c402a070 con 0x556f236d4b30 2023-11-08T19:29:47.880 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.864+0000 7f40d1ffb700 1 -- 172.21.15.143:0/2540274932 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40c4003720 con 0x556f236d4b30 2023-11-08T19:29:47.881 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.864+0000 7f40d1ffb700 1 -- 172.21.15.143:0/2540274932 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40c4003890 con 0x556f236d4b30 2023-11-08T19:29:47.881 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.865+0000 7f40ee820b80 1 -- 172.21.15.143:0/2540274932 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556f236d4b30 msgr2=0x556f236b7450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:47.881 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.865+0000 7f40ee820b80 1 --2- 172.21.15.143:0/2540274932 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556f236d4b30 0x556f236b7450 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f40c4011860 tx=0x7f40c4010000 comp rx=0 tx=0).stop 2023-11-08T19:29:47.881 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.865+0000 7f40ee820b80 1 -- 172.21.15.143:0/2540274932 shutdown_connections 2023-11-08T19:29:47.881 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.865+0000 7f40ee820b80 1 --2- 172.21.15.143:0/2540274932 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f2368b450 0x556f236874c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:47.882 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.865+0000 7f40ee820b80 1 --2- 172.21.15.143:0/2540274932 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556f236d4b30 0x556f236b7450 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:47.882 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.865+0000 7f40ee820b80 1 --2- 172.21.15.143:0/2540274932 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x556f236d55d0 0x556f236b7990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:47.882 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.865+0000 7f40ee820b80 1 -- 172.21.15.143:0/2540274932 >> 172.21.15.143:0/2540274932 conn(0x556f23602660 msgr2=0x556f2368d3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:47.882 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.865+0000 7f40ee820b80 1 -- 172.21.15.143:0/2540274932 shutdown_connections 2023-11-08T19:29:47.882 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.865+0000 7f40ee820b80 1 -- 172.21.15.143:0/2540274932 wait complete. 2023-11-08T19:29:47.883 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.865+0000 7f40ee820b80 1 -- start start 2023-11-08T19:29:47.883 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.866+0000 7f40ee820b80 1 Processor -- start 2023-11-08T19:29:47.883 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.866+0000 7f40ee820b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556f2368b450 0x556f236a1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:47.883 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.866+0000 7f40ee820b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f236d4b30 0x556f236ba5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:47.883 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.866+0000 7f40ee820b80 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x556f236d55d0 0x556f236d59b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:47.883 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.866+0000 7f40ee820b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f235e05c0 con 0x556f2368b450 2023-11-08T19:29:47.884 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.866+0000 7f40ee820b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f2357c0d0 con 0x556f236d4b30 2023-11-08T19:29:47.884 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.866+0000 7f40ee820b80 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x556f2357bd70 con 0x556f236d55d0 2023-11-08T19:29:47.884 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.866+0000 7f40ee820b80 10 client.0 _ll_register_callbacks cb 0x556f233d87b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T19:29:47.884 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.866+0000 7f40dbfff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556f2368b450 0x556f236a1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:47.884 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.866+0000 7f40dbfff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556f2368b450 0x556f236a1d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55824/0 (socket says 172.21.15.143:55824) 2023-11-08T19:29:47.885 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.866+0000 7f40dbfff700 1 -- 172.21.15.143:0/265743143 learned_addr learned my addr 172.21.15.143:0/265743143 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:29:47.885 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.867+0000 7f40e0bc4700 1 --2- 172.21.15.143:0/265743143 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x556f236d55d0 0x556f236d59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:47.885 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.867+0000 7f40db7fe700 1 --2- 172.21.15.143:0/265743143 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f236d4b30 0x556f236ba5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:47.885 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.867+0000 7f40dbfff700 1 -- 172.21.15.143:0/265743143 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x556f236d55d0 msgr2=0x556f236d59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:47.885 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.867+0000 7f40dbfff700 1 --2- 172.21.15.143:0/265743143 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x556f236d55d0 0x556f236d59b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:47.886 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.867+0000 7f40dbfff700 1 -- 172.21.15.143:0/265743143 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f236d4b30 msgr2=0x556f236ba5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:47.886 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.867+0000 7f40dbfff700 1 --2- 172.21.15.143:0/265743143 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f236d4b30 0x556f236ba5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:47.886 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.867+0000 7f40dbfff700 1 -- 172.21.15.143:0/265743143 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40dc0b2460 con 0x556f2368b450 2023-11-08T19:29:47.886 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.867+0000 7f40e0bc4700 1 --2- 172.21.15.143:0/265743143 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x556f236d55d0 0x556f236d59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:29:47.886 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.867+0000 7f40dbfff700 1 --2- 172.21.15.143:0/265743143 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556f2368b450 0x556f236a1d30 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f40dc08c000 tx=0x7f40c40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e75111b859840698 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:47.887 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.867+0000 7f40baffd700 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2023-11-08T19:29:47.887 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.867+0000 7f40baffd700 1 -- 172.21.15.143:0/265743143 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40c402a040 con 0x556f2368b450 2023-11-08T19:29:47.887 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.867+0000 7f40baffd700 1 -- 172.21.15.143:0/265743143 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 7059+0+0 (secure 0 0 0) 0x7f40c402f670 con 0x556f2368b450 2023-11-08T19:29:47.887 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.867+0000 7f40ee820b80 1 -- 172.21.15.143:0/265743143 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556f2367f880 con 0x556f2368b450 2023-11-08T19:29:47.887 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.867+0000 7f40baffd700 1 -- 172.21.15.143:0/265743143 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7f40c402f830 con 0x556f2368b450 2023-11-08T19:29:47.888 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.867+0000 7f40ee820b80 10 client.7524 fetch_fsmap learned FSMap version 80 2023-11-08T19:29:47.888 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.867+0000 7f40ee820b80 1 -- 172.21.15.143:0/265743143 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=2+}) v3 -- 0x556f23409d90 con 0x556f2368b450 2023-11-08T19:29:47.888 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.867+0000 7f40baffd700 1 -- 172.21.15.143:0/265743143 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556f2367f880 con 0x556f2368b450 2023-11-08T19:29:47.888 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.867+0000 7f40baffd700 1 -- 172.21.15.143:0/265743143 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 80) v1 ==== 3306+0+0 (secure 0 0 0) 0x7f40c4036070 con 0x556f2368b450 2023-11-08T19:29:47.888 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.867+0000 7f40ee820b80 10 client.7524 fetch_fsmap finished waiting for FSMap version 80 2023-11-08T19:29:47.889 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.867+0000 7f40ee820b80 10 client.7524 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-08T19:29:47.889 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.867+0000 7f40ee820b80 20 client.7524 populate_metadata read hostname 'smithi143' 2023-11-08T19:29:47.889 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.867+0000 7f40ee820b80 1 --2- 172.21.15.143:0/265743143 >> [v2:172.21.15.174:6834/3601088618,v1:172.21.15.174:6836/3601088618] conn(0x556f236e96c0 0x556f236e9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:47.889 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.867+0000 7f40ee820b80 4 client.7524 mds_command: new command op to 6918 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-08T19:29:47.889 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.868+0000 7f40ee820b80 1 -- 172.21.15.143:0/265743143 --> [v2:172.21.15.174:6834/3601088618,v1:172.21.15.174:6836/3601088618] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556f2365ba30 con 0x556f236e96c0 2023-11-08T19:29:47.889 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.868+0000 7f40db7fe700 1 --2- 172.21.15.143:0/265743143 >> [v2:172.21.15.174:6834/3601088618,v1:172.21.15.174:6836/3601088618] conn(0x556f236e96c0 0x556f236e9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:47.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.868+0000 7f40baffd700 10 client.7524 ms_handle_connect on v2:172.21.15.174:6834/3601088618 2023-11-08T19:29:47.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.868+0000 7f40db7fe700 1 --2- 172.21.15.143:0/265743143 >> [v2:172.21.15.174:6834/3601088618,v1:172.21.15.174:6836/3601088618] conn(0x556f236e96c0 0x556f236e9aa0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=944cef5b3cbb669d server_cookie=3a307d17a20bc71 in_seq=0 out_seq=0 2023-11-08T19:29:47.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.869+0000 7f40baffd700 1 -- 172.21.15.143:0/265743143 <== mds.1 v2:172.21.15.174:6834/3601088618 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556f2365ba30 con 0x556f236e96c0 2023-11-08T19:29:47.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.869+0000 7f40baffd700 10 client.7524 handle_command_reply: tid=0 2023-11-08T19:29:47.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.869+0000 7f40ee820b80 1 client.7524 shutdown 2023-11-08T19:29:47.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.869+0000 7f40ee820b80 1 -- 172.21.15.143:0/265743143 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556f2368b450 msgr2=0x556f236a1d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:47.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.869+0000 7f40ee820b80 1 --2- 172.21.15.143:0/265743143 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556f2368b450 0x556f236a1d30 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f40dc08c000 tx=0x7f40c40047c0 comp rx=0 tx=0).stop 2023-11-08T19:29:47.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.869+0000 7f40ee820b80 1 -- 172.21.15.143:0/265743143 shutdown_connections 2023-11-08T19:29:47.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.869+0000 7f40ee820b80 1 --2- 172.21.15.143:0/265743143 >> [v2:172.21.15.174:6834/3601088618,v1:172.21.15.174:6836/3601088618] conn(0x556f236e96c0 0x556f236e9aa0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:47.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.869+0000 7f40ee820b80 1 --2- 172.21.15.143:0/265743143 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x556f236d55d0 0x556f236d59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:47.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.869+0000 7f40ee820b80 1 --2- 172.21.15.143:0/265743143 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556f2368b450 0x556f236a1d30 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:47.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.869+0000 7f40baffd700 0 client.7524 ms_handle_reset on v2:172.21.15.174:6834/3601088618 2023-11-08T19:29:47.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.869+0000 7f40ee820b80 1 --2- 172.21.15.143:0/265743143 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f236d4b30 0x556f236ba5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:47.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.869+0000 7f40ee820b80 1 -- 172.21.15.143:0/265743143 >> 172.21.15.143:0/265743143 conn(0x556f23602660 msgr2=0x556f236a29c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:47.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.869+0000 7f40ee820b80 1 -- 172.21.15.143:0/265743143 shutdown_connections 2023-11-08T19:29:47.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.869+0000 7f40ee820b80 1 -- 172.21.15.143:0/265743143 wait complete. 2023-11-08T19:29:47.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.869+0000 7f40ee820b80 20 client.7524 trim_cache size 0 max 16384 2023-11-08T19:29:47.899 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.887+0000 7f40ee820b80 1 Processor -- start 2023-11-08T19:29:47.899 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.887+0000 7f40ee820b80 1 -- start start 2023-11-08T19:29:47.899 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.888+0000 7f40ee820b80 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x556f2368b450 0x556f23689960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:47.899 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.888+0000 7f40ee820b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f236d4b30 0x556f236cc9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:47.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.888+0000 7f40ee820b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556f236d55d0 0x556f236ccf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:47.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.888+0000 7f40ee820b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f2345f8d0 con 0x556f236d55d0 2023-11-08T19:29:47.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.888+0000 7f40ee820b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f2368ae70 con 0x556f236d4b30 2023-11-08T19:29:47.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.888+0000 7f40ee820b80 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x556f236c9da0 con 0x556f2368b450 2023-11-08T19:29:47.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.888+0000 7f40e0bc4700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556f236d55d0 0x556f236ccf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:47.901 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.888+0000 7f40e0bc4700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556f236d55d0 0x556f236ccf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55836/0 (socket says 172.21.15.143:55836) 2023-11-08T19:29:47.901 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.888+0000 7f40e0bc4700 1 -- 172.21.15.143:0/2785066020 learned_addr learned my addr 172.21.15.143:0/2785066020 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:29:47.901 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.888+0000 7f40dbfff700 1 --2- 172.21.15.143:0/2785066020 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x556f2368b450 0x556f23689960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:47.901 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.888+0000 7f40db7fe700 1 --2- 172.21.15.143:0/2785066020 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f236d4b30 0x556f236cc9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:47.901 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.888+0000 7f40e0bc4700 1 -- 172.21.15.143:0/2785066020 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x556f2368b450 msgr2=0x556f23689960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:47.902 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.888+0000 7f40e0bc4700 1 --2- 172.21.15.143:0/2785066020 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x556f2368b450 0x556f23689960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:47.902 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.888+0000 7f40e0bc4700 1 -- 172.21.15.143:0/2785066020 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f236d4b30 msgr2=0x556f236cc9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:47.902 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.888+0000 7f40e0bc4700 1 --2- 172.21.15.143:0/2785066020 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f236d4b30 0x556f236cc9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:47.902 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.888+0000 7f40e0bc4700 1 -- 172.21.15.143:0/2785066020 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556f233f2c10 con 0x556f236d55d0 2023-11-08T19:29:47.902 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.888+0000 7f40db7fe700 1 --2- 172.21.15.143:0/2785066020 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f236d4b30 0x556f236cc9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:29:47.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.888+0000 7f40e0bc4700 1 --2- 172.21.15.143:0/2785066020 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556f236d55d0 0x556f236ccf10 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x556f236884c0 tx=0x7f40d4009d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f6b9330016c7a88 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:47.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.888+0000 7f40d27fc700 1 -- 172.21.15.143:0/2785066020 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40d400acd0 con 0x556f236d55d0 2023-11-08T19:29:47.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.888+0000 7f40d27fc700 1 -- 172.21.15.143:0/2785066020 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40d4012040 con 0x556f236d55d0 2023-11-08T19:29:47.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.888+0000 7f40d27fc700 1 -- 172.21.15.143:0/2785066020 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40d4016630 con 0x556f236d55d0 2023-11-08T19:29:47.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.889+0000 7f40ee820b80 1 -- 172.21.15.143:0/2785066020 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556f236d55d0 msgr2=0x556f236ccf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:47.904 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.889+0000 7f40ee820b80 1 --2- 172.21.15.143:0/2785066020 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556f236d55d0 0x556f236ccf10 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x556f236884c0 tx=0x7f40d4009d80 comp rx=0 tx=0).stop 2023-11-08T19:29:47.904 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.889+0000 7f40ee820b80 1 -- 172.21.15.143:0/2785066020 shutdown_connections 2023-11-08T19:29:47.904 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.889+0000 7f40ee820b80 1 --2- 172.21.15.143:0/2785066020 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x556f2368b450 0x556f23689960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:47.904 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.889+0000 7f40ee820b80 1 --2- 172.21.15.143:0/2785066020 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556f236d55d0 0x556f236ccf10 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:47.904 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.889+0000 7f40ee820b80 1 --2- 172.21.15.143:0/2785066020 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f236d4b30 0x556f236cc9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:47.905 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.889+0000 7f40ee820b80 1 -- 172.21.15.143:0/2785066020 >> 172.21.15.143:0/2785066020 conn(0x556f23602660 msgr2=0x556f236a1200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:47.905 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.889+0000 7f40ee820b80 1 -- 172.21.15.143:0/2785066020 shutdown_connections 2023-11-08T19:29:47.905 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.889+0000 7f40ee820b80 1 -- 172.21.15.143:0/2785066020 wait complete. 2023-11-08T19:29:47.905 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.889+0000 7f40ee820b80 1 -- start start 2023-11-08T19:29:47.905 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.890+0000 7f40ee820b80 1 Processor -- start 2023-11-08T19:29:47.905 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.890+0000 7f40ee820b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556f2368b450 0x556f23728890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:47.906 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.890+0000 7f40ee820b80 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x556f236d4b30 0x556f236884f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:47.906 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.890+0000 7f40ee820b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f236d55d0 0x556f23688a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:47.906 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.890+0000 7f40ee820b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f236c86a0 con 0x556f2368b450 2023-11-08T19:29:47.906 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.890+0000 7f40ee820b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f236a1e60 con 0x556f236d55d0 2023-11-08T19:29:47.906 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.890+0000 7f40ee820b80 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x556f23687ce0 con 0x556f236d4b30 2023-11-08T19:29:47.907 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.890+0000 7f40ee820b80 10 client.0 _ll_register_callbacks cb 0x7f40c4032070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T19:29:47.907 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.891+0000 7f40dbfff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556f2368b450 0x556f23728890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:47.907 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.891+0000 7f40dbfff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556f2368b450 0x556f23728890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55838/0 (socket says 172.21.15.143:55838) 2023-11-08T19:29:47.907 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.891+0000 7f40dbfff700 1 -- 172.21.15.143:0/1343177194 learned_addr learned my addr 172.21.15.143:0/1343177194 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:29:47.907 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.891+0000 7f40db7fe700 1 --2- 172.21.15.143:0/1343177194 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x556f236d4b30 0x556f236884f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:47.908 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.891+0000 7f40e0bc4700 1 --2- 172.21.15.143:0/1343177194 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f236d55d0 0x556f23688a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:47.908 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.891+0000 7f40dbfff700 1 -- 172.21.15.143:0/1343177194 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x556f236d4b30 msgr2=0x556f236884f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:47.908 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.891+0000 7f40dbfff700 1 --2- 172.21.15.143:0/1343177194 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x556f236d4b30 0x556f236884f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:47.908 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.891+0000 7f40dbfff700 1 -- 172.21.15.143:0/1343177194 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f236d55d0 msgr2=0x556f23688a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:47.908 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.891+0000 7f40dbfff700 1 --2- 172.21.15.143:0/1343177194 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f236d55d0 0x556f23688a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:47.909 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.891+0000 7f40dbfff700 1 -- 172.21.15.143:0/1343177194 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40d40099e0 con 0x556f2368b450 2023-11-08T19:29:47.909 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.891+0000 7f40e0bc4700 1 --2- 172.21.15.143:0/1343177194 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f236d55d0 0x556f23688a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:29:47.909 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.891+0000 7f40dbfff700 1 --2- 172.21.15.143:0/1343177194 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556f2368b450 0x556f23728890 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f40dc08c000 tx=0x7f40c4008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97885a15938571b1 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:47.909 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.891+0000 7f40baffd700 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2023-11-08T19:29:47.909 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.891+0000 7f40baffd700 1 -- 172.21.15.143:0/1343177194 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40c402a040 con 0x556f2368b450 2023-11-08T19:29:47.910 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.891+0000 7f40baffd700 1 -- 172.21.15.143:0/1343177194 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 7059+0+0 (secure 0 0 0) 0x7f40c403f710 con 0x556f2368b450 2023-11-08T19:29:47.910 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.891+0000 7f40ee820b80 1 -- 172.21.15.143:0/1343177194 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f40c4003720 con 0x556f2368b450 2023-11-08T19:29:47.910 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.891+0000 7f40baffd700 1 -- 172.21.15.143:0/1343177194 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7f40c402fa30 con 0x556f2368b450 2023-11-08T19:29:47.910 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.891+0000 7f40ee820b80 10 client.7530 fetch_fsmap learned FSMap version 80 2023-11-08T19:29:47.910 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.891+0000 7f40ee820b80 1 -- 172.21.15.143:0/1343177194 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=2+}) v3 -- 0x556f23430200 con 0x556f2368b450 2023-11-08T19:29:47.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.891+0000 7f40baffd700 1 -- 172.21.15.143:0/1343177194 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40c4003720 con 0x556f2368b450 2023-11-08T19:29:47.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.891+0000 7f40baffd700 1 -- 172.21.15.143:0/1343177194 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 80) v1 ==== 3306+0+0 (secure 0 0 0) 0x7f40c40043d0 con 0x556f2368b450 2023-11-08T19:29:47.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.892+0000 7f40ee820b80 10 client.7530 fetch_fsmap finished waiting for FSMap version 80 2023-11-08T19:29:47.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.892+0000 7f40ee820b80 10 client.7530 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-08T19:29:47.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.892+0000 7f40ee820b80 20 client.7530 populate_metadata read hostname 'smithi143' 2023-11-08T19:29:47.913 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.892+0000 7f40ee820b80 1 --2- 172.21.15.143:0/1343177194 >> [v2:172.21.15.174:6834/3601088618,v1:172.21.15.174:6836/3601088618] conn(0x556f23742a10 0x556f23742df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:47.913 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.892+0000 7f40ee820b80 4 client.7530 mds_command: new command op to 6918 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-08T19:29:47.913 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.892+0000 7f40ee820b80 1 -- 172.21.15.143:0/1343177194 --> [v2:172.21.15.174:6834/3601088618,v1:172.21.15.174:6836/3601088618] -- command(tid 0: {"prefix": "status"}) v1 -- 0x556f235d95a0 con 0x556f23742a10 2023-11-08T19:29:47.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.892+0000 7f40db7fe700 1 --2- 172.21.15.143:0/1343177194 >> [v2:172.21.15.174:6834/3601088618,v1:172.21.15.174:6836/3601088618] conn(0x556f23742a10 0x556f23742df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:47.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.892+0000 7f40baffd700 10 client.7530 ms_handle_connect on v2:172.21.15.174:6834/3601088618 2023-11-08T19:29:47.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.892+0000 7f40db7fe700 1 --2- 172.21.15.143:0/1343177194 >> [v2:172.21.15.174:6834/3601088618,v1:172.21.15.174:6836/3601088618] conn(0x556f23742a10 0x556f23742df0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d3023d49a405bcad server_cookie=385036dc53436e4d in_seq=0 out_seq=0 2023-11-08T19:29:47.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.892+0000 7f40baffd700 1 -- 172.21.15.143:0/1343177194 <== mds.1 v2:172.21.15.174:6834/3601088618 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x556f235d95a0 con 0x556f23742a10 2023-11-08T19:29:47.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.892+0000 7f40baffd700 10 client.7530 handle_command_reply: tid=0 2023-11-08T19:29:47.915 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.892+0000 7f40ee820b80 1 client.7530 shutdown 2023-11-08T19:29:47.915 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.892+0000 7f40ee820b80 1 -- 172.21.15.143:0/1343177194 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556f2368b450 msgr2=0x556f23728890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:47.915 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.892+0000 7f40ee820b80 1 --2- 172.21.15.143:0/1343177194 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556f2368b450 0x556f23728890 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f40dc08c000 tx=0x7f40c4008030 comp rx=0 tx=0).stop 2023-11-08T19:29:47.915 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.892+0000 7f40ee820b80 1 -- 172.21.15.143:0/1343177194 shutdown_connections 2023-11-08T19:29:47.915 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.892+0000 7f40ee820b80 1 --2- 172.21.15.143:0/1343177194 >> [v2:172.21.15.174:6834/3601088618,v1:172.21.15.174:6836/3601088618] conn(0x556f23742a10 0x556f23742df0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:47.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.892+0000 7f40ee820b80 1 --2- 172.21.15.143:0/1343177194 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f236d55d0 0x556f23688a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:47.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.892+0000 7f40ee820b80 1 --2- 172.21.15.143:0/1343177194 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556f2368b450 0x556f23728890 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:47.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.892+0000 7f40baffd700 0 client.7530 ms_handle_reset on v2:172.21.15.174:6834/3601088618 2023-11-08T19:29:47.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.892+0000 7f40ee820b80 1 --2- 172.21.15.143:0/1343177194 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x556f236d4b30 0x556f236884f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:47.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.892+0000 7f40ee820b80 1 -- 172.21.15.143:0/1343177194 >> 172.21.15.143:0/1343177194 conn(0x556f23602660 msgr2=0x556f236ccee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:47.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.893+0000 7f40ee820b80 1 -- 172.21.15.143:0/1343177194 shutdown_connections 2023-11-08T19:29:47.917 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.893+0000 7f40ee820b80 1 -- 172.21.15.143:0/1343177194 wait complete. 2023-11-08T19:29:47.917 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.893+0000 7f40ee820b80 20 client.7530 trim_cache size 0 max 16384 2023-11-08T19:29:47.917 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2023-11-08T19:29:47.918 INFO:teuthology.orchestra.run.smithi143.stdout: "cluster_fsid": "968483af-77fc-4068-ad7c-5c2a1a4613d7", 2023-11-08T19:29:47.918 INFO:teuthology.orchestra.run.smithi143.stdout: "whoami": 1, 2023-11-08T19:29:47.918 INFO:teuthology.orchestra.run.smithi143.stdout: "id": 6918, 2023-11-08T19:29:47.918 INFO:teuthology.orchestra.run.smithi143.stdout: "want_state": "up:active", 2023-11-08T19:29:47.918 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "up:active", 2023-11-08T19:29:47.919 INFO:teuthology.orchestra.run.smithi143.stdout: "fs_name": "cephfs", 2023-11-08T19:29:47.919 INFO:teuthology.orchestra.run.smithi143.stdout: "rank_uptime": 90.298822889999997, 2023-11-08T19:29:47.919 INFO:teuthology.orchestra.run.smithi143.stdout: "mdsmap_epoch": 80, 2023-11-08T19:29:47.919 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_epoch": 97, 2023-11-08T19:29:47.919 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_epoch_barrier": 97, 2023-11-08T19:29:47.920 INFO:teuthology.orchestra.run.smithi143.stdout: "uptime": 110.090038814 2023-11-08T19:29:47.920 INFO:teuthology.orchestra.run.smithi143.stdout:} 2023-11-08T19:29:47.921 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.893+0000 7f40e2627700 1 -- 172.21.15.143:0/3257767738 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f40bc06af00 msgr2=0x7f40bc06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:47.921 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.893+0000 7f40e2627700 1 --2- 172.21.15.143:0/3257767738 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f40bc06af00 0x7f40bc06d3c0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f40c40090b0 tx=0x7f40c401c040 comp rx=0 tx=0).stop 2023-11-08T19:29:47.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.893+0000 7f40e2627700 1 -- 172.21.15.143:0/3257767738 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f40dc0d1c70 msgr2=0x7f40dc1590a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:47.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.893+0000 7f40e2627700 1 --2- 172.21.15.143:0/3257767738 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f40dc0d1c70 0x7f40dc1590a0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f40cc0011e0 tx=0x7f40cc00cae0 comp rx=0 tx=0).stop 2023-11-08T19:29:47.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.893+0000 7f40e2627700 1 -- 172.21.15.143:0/3257767738 shutdown_connections 2023-11-08T19:29:47.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.893+0000 7f40e2627700 1 --2- 172.21.15.143:0/3257767738 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f40bc06af00 0x7f40bc06d3c0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:47.923 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.893+0000 7f40e2627700 1 --2- 172.21.15.143:0/3257767738 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f40dc0d03a0 0x7f40dc152310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:47.923 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.893+0000 7f40e2627700 1 --2- 172.21.15.143:0/3257767738 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f40dc15ba30 0x7f40dc1595e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:47.923 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.893+0000 7f40e2627700 1 --2- 172.21.15.143:0/3257767738 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f40dc0d1c70 0x7f40dc1590a0 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:47.923 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.893+0000 7f40e2627700 1 -- 172.21.15.143:0/3257767738 >> 172.21.15.143:0/3257767738 conn(0x7f40dc08b000 msgr2=0x7f40dc0ce900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:47.923 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.893+0000 7f40e2627700 1 -- 172.21.15.143:0/3257767738 shutdown_connections 2023-11-08T19:29:47.924 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:47.893+0000 7f40e2627700 1 -- 172.21.15.143:0/3257767738 wait complete. 2023-11-08T19:29:47.924 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-11-08T19:29:48.045 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.044+0000 7faf6bc30700 1 Processor -- start 2023-11-08T19:29:48.045 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.044+0000 7faf6bc30700 1 -- start start 2023-11-08T19:29:48.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.044+0000 7faf6bc30700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7faf6415a9a0 0x7faf6406e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:48.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.045+0000 7faf6bc30700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf640cbe70 con 0x7faf640b1810 2023-11-08T19:29:48.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.045+0000 7faf6bc30700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf640cb260 con 0x7faf640cadc0 2023-11-08T19:29:48.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.045+0000 7faf6bc30700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf6406e890 con 0x7faf6415a9a0 2023-11-08T19:29:48.050 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.045+0000 7faf6a1cd700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7faf640b1810 0x7faf6406cc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:52004/0 (socket says 172.21.15.143:52004) 2023-11-08T19:29:48.050 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.045+0000 7faf6a1cd700 1 -- 172.21.15.143:0/2801462634 learned_addr learned my addr 172.21.15.143:0/2801462634 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:29:48.050 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.045+0000 7faf699cc700 1 --2- 172.21.15.143:0/2801462634 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7faf6415a9a0 0x7faf6406e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:48.050 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.045+0000 7faf689ca700 1 -- 172.21.15.143:0/2801462634 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1416892732 0 0) 0x7faf640cbe70 con 0x7faf640b1810 2023-11-08T19:29:48.051 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.045+0000 7faf689ca700 1 -- 172.21.15.143:0/2801462634 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf500035b0 con 0x7faf640b1810 2023-11-08T19:29:48.051 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.045+0000 7faf689ca700 1 -- 172.21.15.143:0/2801462634 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1919480937 0 0) 0x7faf500035b0 con 0x7faf640b1810 2023-11-08T19:29:48.051 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.045+0000 7faf689ca700 1 -- 172.21.15.143:0/2801462634 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7faf6415a9a0 msgr2=0x7faf6406e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:48.051 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.045+0000 7faf689ca700 1 --2- 172.21.15.143:0/2801462634 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7faf6415a9a0 0x7faf6406e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:48.052 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.045+0000 7faf689ca700 1 -- 172.21.15.143:0/2801462634 >> v1:172.21.15.174:6789/0 conn(0x7faf640cadc0 legacy=0x7faf640b1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:48.052 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.045+0000 7faf689ca700 1 -- 172.21.15.143:0/2801462634 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf640cb540 con 0x7faf640b1810 2023-11-08T19:29:48.052 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.045+0000 7faf689ca700 1 -- 172.21.15.143:0/2801462634 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4147756303 0 0) 0x7faf5c0019f0 con 0x7faf640b1810 2023-11-08T19:29:48.052 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.045+0000 7faf689ca700 1 -- 172.21.15.143:0/2801462634 >> v1:172.21.15.143:6789/0 conn(0x7faf640b1810 legacy=0x7faf6406cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:48.052 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.045+0000 7faf689ca700 1 --2- 172.21.15.143:0/2801462634 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faf50003a30 0x7faf50003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:48.053 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.045+0000 7faf689ca700 1 --2- 172.21.15.143:0/2801462634 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7faf6415a9a0 0x7faf50004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:48.053 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.045+0000 7faf689ca700 1 --2- 172.21.15.143:0/2801462634 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faf50004d90 0x7faf50005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:48.053 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.045+0000 7faf689ca700 1 -- 172.21.15.143:0/2801462634 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf6406e890 con 0x7faf50003a30 2023-11-08T19:29:48.053 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.045+0000 7faf689ca700 1 -- 172.21.15.143:0/2801462634 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf5000bb40 con 0x7faf50004d90 2023-11-08T19:29:48.053 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.045+0000 7faf689ca700 1 -- 172.21.15.143:0/2801462634 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf5000df10 con 0x7faf6415a9a0 2023-11-08T19:29:48.054 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.045+0000 7faf699cc700 1 --2- 172.21.15.143:0/2801462634 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faf50003a30 0x7faf50003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:48.054 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.045+0000 7faf6a1cd700 1 --2- 172.21.15.143:0/2801462634 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faf50004d90 0x7faf50005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:48.054 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.045+0000 7faf691cb700 1 --2- 172.21.15.143:0/2801462634 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7faf6415a9a0 0x7faf50004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:48.054 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.046+0000 7faf699cc700 1 -- 172.21.15.143:0/2801462634 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7faf6415a9a0 msgr2=0x7faf50004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:48.054 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.046+0000 7faf699cc700 1 --2- 172.21.15.143:0/2801462634 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7faf6415a9a0 0x7faf50004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:48.055 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.046+0000 7faf699cc700 1 -- 172.21.15.143:0/2801462634 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faf50004d90 msgr2=0x7faf50005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:48.055 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.046+0000 7faf699cc700 1 --2- 172.21.15.143:0/2801462634 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faf50004d90 0x7faf50005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:48.055 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.046+0000 7faf699cc700 1 -- 172.21.15.143:0/2801462634 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf500035b0 con 0x7faf50003a30 2023-11-08T19:29:48.055 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.046+0000 7faf699cc700 1 --2- 172.21.15.143:0/2801462634 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faf50003a30 0x7faf50003e50 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7faf60009b60 tx=0x7faf50005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40352fec46e6e6ad server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:48.055 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.046+0000 7faf689ca700 1 -- 172.21.15.143:0/2801462634 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf6000e6c0 con 0x7faf50003a30 2023-11-08T19:29:48.056 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.046+0000 7faf689ca700 1 -- 172.21.15.143:0/2801462634 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf6000e820 con 0x7faf50003a30 2023-11-08T19:29:48.056 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.046+0000 7faf689ca700 1 -- 172.21.15.143:0/2801462634 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf6000eb90 con 0x7faf50003a30 2023-11-08T19:29:48.056 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.047+0000 7faf6bc30700 1 -- 172.21.15.143:0/2801462634 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faf50003a30 msgr2=0x7faf50003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:48.056 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.047+0000 7faf6bc30700 1 --2- 172.21.15.143:0/2801462634 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faf50003a30 0x7faf50003e50 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7faf60009b60 tx=0x7faf50005880 comp rx=0 tx=0).stop 2023-11-08T19:29:48.056 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.047+0000 7faf6a1cd700 1 -- 172.21.15.143:0/2801462634 reap_dead start 2023-11-08T19:29:48.057 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.047+0000 7faf6bc30700 1 -- 172.21.15.143:0/2801462634 shutdown_connections 2023-11-08T19:29:48.057 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.047+0000 7faf6bc30700 1 -- 172.21.15.143:0/2801462634 >> 172.21.15.143:0/2801462634 conn(0x7faf6406b4c0 msgr2=0x7faf640cf680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:48.057 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.047+0000 7faf6bc30700 1 -- 172.21.15.143:0/2801462634 shutdown_connections 2023-11-08T19:29:48.057 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.047+0000 7faf6bc30700 1 -- 172.21.15.143:0/2801462634 wait complete. 2023-11-08T19:29:48.066 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.048+0000 7faf6bc30700 1 Processor -- start 2023-11-08T19:29:48.066 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.048+0000 7faf6bc30700 1 -- start start 2023-11-08T19:29:48.066 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.049+0000 7faf6bc30700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faf50003a30 0x7faf6414e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:48.066 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.049+0000 7faf6bc30700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7faf641545a0 0x7faf64154a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:48.067 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.049+0000 7faf6bc30700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faf6415c040 0x7faf6415c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:48.067 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.049+0000 7faf6bc30700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf6415ca30 con 0x7faf6415c040 2023-11-08T19:29:48.067 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.049+0000 7faf6bc30700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf64152940 con 0x7faf50003a30 2023-11-08T19:29:48.067 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.049+0000 7faf6bc30700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf64155010 con 0x7faf641545a0 2023-11-08T19:29:48.068 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.049+0000 7faf6a1cd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faf6415c040 0x7faf6415c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:48.068 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.049+0000 7faf6a1cd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faf6415c040 0x7faf6415c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55862/0 (socket says 172.21.15.143:55862) 2023-11-08T19:29:48.068 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.049+0000 7faf6a1cd700 1 -- 172.21.15.143:0/1390032910 learned_addr learned my addr 172.21.15.143:0/1390032910 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:29:48.068 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.049+0000 7faf699cc700 1 --2- 172.21.15.143:0/1390032910 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faf50003a30 0x7faf6414e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:48.068 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.049+0000 7faf691cb700 1 --2- 172.21.15.143:0/1390032910 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7faf641545a0 0x7faf64154a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:48.068 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.049+0000 7faf6a1cd700 1 -- 172.21.15.143:0/1390032910 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7faf641545a0 msgr2=0x7faf64154a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:48.069 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.049+0000 7faf6a1cd700 1 --2- 172.21.15.143:0/1390032910 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7faf641545a0 0x7faf64154a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:48.069 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.049+0000 7faf6a1cd700 1 -- 172.21.15.143:0/1390032910 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faf50003a30 msgr2=0x7faf6414e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:48.069 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.049+0000 7faf6a1cd700 1 --2- 172.21.15.143:0/1390032910 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faf50003a30 0x7faf6414e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:48.069 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.049+0000 7faf6a1cd700 1 -- 172.21.15.143:0/1390032910 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf641f3f20 con 0x7faf6415c040 2023-11-08T19:29:48.069 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.049+0000 7faf6a1cd700 1 --2- 172.21.15.143:0/1390032910 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faf6415c040 0x7faf6415c4f0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7faf640cab80 tx=0x7faf5c00a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:48.070 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.049+0000 7faf5a7fc700 1 -- 172.21.15.143:0/1390032910 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf5c00acb0 con 0x7faf6415c040 2023-11-08T19:29:48.070 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.050+0000 7faf5a7fc700 1 -- 172.21.15.143:0/1390032910 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf5c00ae10 con 0x7faf6415c040 2023-11-08T19:29:48.070 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.050+0000 7faf6bc30700 1 -- 172.21.15.143:0/1390032910 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf641f4200 con 0x7faf6415c040 2023-11-08T19:29:48.070 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.050+0000 7faf5a7fc700 1 -- 172.21.15.143:0/1390032910 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf5c013630 con 0x7faf6415c040 2023-11-08T19:29:48.071 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.050+0000 7faf6bc30700 1 -- 172.21.15.143:0/1390032910 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf641f3bf0 con 0x7faf6415c040 2023-11-08T19:29:48.071 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.051+0000 7faf5a7fc700 1 -- 172.21.15.143:0/1390032910 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7faf5c013790 con 0x7faf6415c040 2023-11-08T19:29:48.071 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.051+0000 7faf5a7fc700 1 --2- 172.21.15.143:0/1390032910 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7faf4806af00 0x7faf4806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:48.071 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.052+0000 7faf5a7fc700 1 -- 172.21.15.143:0/1390032910 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 7059+0+0 (secure 0 0 0) 0x7faf5c08a350 con 0x7faf6415c040 2023-11-08T19:29:48.071 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.052+0000 7faf699cc700 1 --2- 172.21.15.143:0/1390032910 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7faf4806af00 0x7faf4806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:48.072 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.052+0000 7faf699cc700 1 --2- 172.21.15.143:0/1390032910 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7faf4806af00 0x7faf4806d3c0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7faf600090b0 tx=0x7faf6001f040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:48.072 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.053+0000 7faf77e29b80 1 Processor -- start 2023-11-08T19:29:48.072 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.053+0000 7faf77e29b80 1 -- start start 2023-11-08T19:29:48.072 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.054+0000 7faf77e29b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5631272c36f0 0x5631272f3450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:48.072 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.054+0000 7faf77e29b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5631272c5c30 0x5631272f3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:48.073 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.054+0000 7faf77e29b80 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x5631272c3ad0 0x5631272ddf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:48.073 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.054+0000 7faf77e29b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x56312722f6e0 con 0x5631272c5c30 2023-11-08T19:29:48.073 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.054+0000 7faf77e29b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631272233b0 con 0x5631272c36f0 2023-11-08T19:29:48.073 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.054+0000 7faf77e29b80 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x563127220cd0 con 0x5631272c3ad0 2023-11-08T19:29:48.073 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.054+0000 7faf691cb700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5631272c5c30 0x5631272f3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:48.074 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.054+0000 7faf691cb700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5631272c5c30 0x5631272f3990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55868/0 (socket says 172.21.15.143:55868) 2023-11-08T19:29:48.074 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.054+0000 7faf691cb700 1 -- 172.21.15.143:0/3045998245 learned_addr learned my addr 172.21.15.143:0/3045998245 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:29:48.074 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.054+0000 7faf699cc700 1 --2- 172.21.15.143:0/3045998245 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5631272c36f0 0x5631272f3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:48.074 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.054+0000 7faf6a1cd700 1 --2- 172.21.15.143:0/3045998245 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x5631272c3ad0 0x5631272ddf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:48.074 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.054+0000 7faf691cb700 1 -- 172.21.15.143:0/3045998245 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x5631272c3ad0 msgr2=0x5631272ddf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:48.075 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.054+0000 7faf691cb700 1 --2- 172.21.15.143:0/3045998245 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x5631272c3ad0 0x5631272ddf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:48.075 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.054+0000 7faf691cb700 1 -- 172.21.15.143:0/3045998245 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5631272c36f0 msgr2=0x5631272f3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:48.075 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.054+0000 7faf691cb700 1 --2- 172.21.15.143:0/3045998245 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5631272c36f0 0x5631272f3450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:48.075 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.054+0000 7faf691cb700 1 -- 172.21.15.143:0/3045998245 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5631270995e0 con 0x5631272c5c30 2023-11-08T19:29:48.075 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.055+0000 7faf691cb700 1 --2- 172.21.15.143:0/3045998245 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5631272c5c30 0x5631272f3990 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7faf54001770 tx=0x7faf54007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=678cdb0702cd977c server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:48.076 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.055+0000 7faf477fe700 1 -- 172.21.15.143:0/3045998245 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf540053a0 con 0x5631272c5c30 2023-11-08T19:29:48.076 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.055+0000 7faf477fe700 1 -- 172.21.15.143:0/3045998245 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf54005500 con 0x5631272c5c30 2023-11-08T19:29:48.076 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.055+0000 7faf477fe700 1 -- 172.21.15.143:0/3045998245 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf54003880 con 0x5631272c5c30 2023-11-08T19:29:48.076 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.055+0000 7faf77e29b80 1 -- 172.21.15.143:0/3045998245 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5631272c5c30 msgr2=0x5631272f3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:48.076 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.055+0000 7faf77e29b80 1 --2- 172.21.15.143:0/3045998245 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5631272c5c30 0x5631272f3990 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7faf54001770 tx=0x7faf54007d10 comp rx=0 tx=0).stop 2023-11-08T19:29:48.077 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.055+0000 7faf77e29b80 1 -- 172.21.15.143:0/3045998245 shutdown_connections 2023-11-08T19:29:48.077 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.055+0000 7faf77e29b80 1 --2- 172.21.15.143:0/3045998245 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x5631272c3ad0 0x5631272ddf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:48.077 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.055+0000 7faf77e29b80 1 --2- 172.21.15.143:0/3045998245 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5631272c36f0 0x5631272f3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:48.077 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.055+0000 7faf77e29b80 1 --2- 172.21.15.143:0/3045998245 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5631272c5c30 0x5631272f3990 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:48.077 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.055+0000 7faf77e29b80 1 -- 172.21.15.143:0/3045998245 >> 172.21.15.143:0/3045998245 conn(0x56312723e660 msgr2=0x5631272c93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:48.077 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.055+0000 7faf77e29b80 1 -- 172.21.15.143:0/3045998245 shutdown_connections 2023-11-08T19:29:48.078 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.056+0000 7faf77e29b80 1 -- 172.21.15.143:0/3045998245 wait complete. 2023-11-08T19:29:48.078 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.056+0000 7faf77e29b80 1 -- start start 2023-11-08T19:29:48.078 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.056+0000 7faf77e29b80 1 Processor -- start 2023-11-08T19:29:48.078 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.057+0000 7faf77e29b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5631272c36f0 0x56312731d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:48.079 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.057+0000 7faf77e29b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5631272c3ad0 0x5631272f62f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:48.079 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.057+0000 7faf77e29b80 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x5631272c4020 0x5631272c44a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:48.079 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.057+0000 7faf77e29b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x56312721c5c0 con 0x5631272c3ad0 2023-11-08T19:29:48.079 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.057+0000 7faf77e29b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631271b80d0 con 0x5631272c36f0 2023-11-08T19:29:48.079 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.057+0000 7faf77e29b80 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x5631271b7d70 con 0x5631272c4020 2023-11-08T19:29:48.079 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.057+0000 7faf77e29b80 10 client.0 _ll_register_callbacks cb 0x5631270147b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T19:29:48.080 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.057+0000 7faf691cb700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5631272c3ad0 0x5631272f62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:48.080 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.057+0000 7faf691cb700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5631272c3ad0 0x5631272f62f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55876/0 (socket says 172.21.15.143:55876) 2023-11-08T19:29:48.080 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.057+0000 7faf691cb700 1 -- 172.21.15.143:0/1546948523 learned_addr learned my addr 172.21.15.143:0/1546948523 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:29:48.080 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.057+0000 7faf6a1cd700 1 --2- 172.21.15.143:0/1546948523 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x5631272c4020 0x5631272c44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:48.080 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.057+0000 7faf699cc700 1 --2- 172.21.15.143:0/1546948523 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5631272c36f0 0x56312731d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:48.081 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.057+0000 7faf691cb700 1 -- 172.21.15.143:0/1546948523 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x5631272c4020 msgr2=0x5631272c44a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:48.081 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.057+0000 7faf691cb700 1 --2- 172.21.15.143:0/1546948523 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x5631272c4020 0x5631272c44a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:48.081 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.057+0000 7faf691cb700 1 -- 172.21.15.143:0/1546948523 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5631272c36f0 msgr2=0x56312731d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:48.081 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.057+0000 7faf691cb700 1 --2- 172.21.15.143:0/1546948523 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5631272c36f0 0x56312731d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:48.081 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.058+0000 7faf691cb700 1 -- 172.21.15.143:0/1546948523 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf640cb260 con 0x5631272c3ad0 2023-11-08T19:29:48.082 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.058+0000 7faf699cc700 1 --2- 172.21.15.143:0/1546948523 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5631272c36f0 0x56312731d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:29:48.082 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.058+0000 7faf691cb700 1 --2- 172.21.15.143:0/1546948523 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5631272c3ad0 0x5631272f62f0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7faf54000c00 tx=0x7faf54001190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db7691e71cf31506 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:48.082 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.058+0000 7faf44ff9700 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2023-11-08T19:29:48.082 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.058+0000 7faf44ff9700 1 -- 172.21.15.143:0/1546948523 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf540053a0 con 0x5631272c3ad0 2023-11-08T19:29:48.082 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.058+0000 7faf44ff9700 1 -- 172.21.15.143:0/1546948523 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 7059+0+0 (secure 0 0 0) 0x7faf54012950 con 0x5631272c3ad0 2023-11-08T19:29:48.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.058+0000 7faf77e29b80 1 -- 172.21.15.143:0/1546948523 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5631272bb880 con 0x5631272c3ad0 2023-11-08T19:29:48.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.058+0000 7faf44ff9700 1 -- 172.21.15.143:0/1546948523 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7faf54012b10 con 0x5631272c3ad0 2023-11-08T19:29:48.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.058+0000 7faf77e29b80 10 client.7542 fetch_fsmap learned FSMap version 80 2023-11-08T19:29:48.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.058+0000 7faf77e29b80 1 -- 172.21.15.143:0/1546948523 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=2+}) v3 -- 0x563127045d90 con 0x5631272c3ad0 2023-11-08T19:29:48.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.058+0000 7faf44ff9700 1 -- 172.21.15.143:0/1546948523 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5631272bb880 con 0x5631272c3ad0 2023-11-08T19:29:48.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.058+0000 7faf44ff9700 1 -- 172.21.15.143:0/1546948523 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 80) v1 ==== 3306+0+0 (secure 0 0 0) 0x7faf54012c70 con 0x5631272c3ad0 2023-11-08T19:29:48.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.058+0000 7faf77e29b80 10 client.7542 fetch_fsmap finished waiting for FSMap version 80 2023-11-08T19:29:48.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.058+0000 7faf77e29b80 10 client.7542 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-08T19:29:48.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.058+0000 7faf77e29b80 20 client.7542 populate_metadata read hostname 'smithi143' 2023-11-08T19:29:48.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.058+0000 7faf77e29b80 1 --2- 172.21.15.143:0/1546948523 >> [v2:172.21.15.143:6833/3664584522,v1:172.21.15.143:6835/3664584522] conn(0x563127321870 0x5631272de250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:48.085 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.058+0000 7faf77e29b80 4 client.7542 mds_command: new command op to 7386 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-08T19:29:48.085 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.058+0000 7faf77e29b80 1 -- 172.21.15.143:0/1546948523 --> [v2:172.21.15.143:6833/3664584522,v1:172.21.15.143:6835/3664584522] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563127297a30 con 0x563127321870 2023-11-08T19:29:48.085 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.058+0000 7faf699cc700 1 --2- 172.21.15.143:0/1546948523 >> [v2:172.21.15.143:6833/3664584522,v1:172.21.15.143:6835/3664584522] conn(0x563127321870 0x5631272de250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:48.085 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.059+0000 7faf44ff9700 10 client.7542 ms_handle_connect on v2:172.21.15.143:6833/3664584522 2023-11-08T19:29:48.085 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.059+0000 7faf699cc700 1 --2- 172.21.15.143:0/1546948523 >> [v2:172.21.15.143:6833/3664584522,v1:172.21.15.143:6835/3664584522] conn(0x563127321870 0x5631272de250 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f73adb6c154f80f3 server_cookie=20a92838007e842 in_seq=0 out_seq=0 2023-11-08T19:29:48.086 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.059+0000 7faf44ff9700 1 -- 172.21.15.143:0/1546948523 <== mds.0 v2:172.21.15.143:6833/3664584522 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563127297a30 con 0x563127321870 2023-11-08T19:29:48.086 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.059+0000 7faf44ff9700 10 client.7542 handle_command_reply: tid=0 2023-11-08T19:29:48.086 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.059+0000 7faf77e29b80 1 client.7542 shutdown 2023-11-08T19:29:48.086 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.059+0000 7faf77e29b80 1 -- 172.21.15.143:0/1546948523 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5631272c3ad0 msgr2=0x5631272f62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:48.086 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.059+0000 7faf77e29b80 1 --2- 172.21.15.143:0/1546948523 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5631272c3ad0 0x5631272f62f0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7faf54000c00 tx=0x7faf54001190 comp rx=0 tx=0).stop 2023-11-08T19:29:48.087 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.060+0000 7faf77e29b80 1 -- 172.21.15.143:0/1546948523 shutdown_connections 2023-11-08T19:29:48.087 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.060+0000 7faf77e29b80 1 --2- 172.21.15.143:0/1546948523 >> [v2:172.21.15.143:6833/3664584522,v1:172.21.15.143:6835/3664584522] conn(0x563127321870 0x5631272de250 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:48.087 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.060+0000 7faf77e29b80 1 --2- 172.21.15.143:0/1546948523 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x5631272c4020 0x5631272c44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:48.087 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.060+0000 7faf77e29b80 1 --2- 172.21.15.143:0/1546948523 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5631272c36f0 0x56312731d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:48.087 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.060+0000 7faf44ff9700 0 client.7542 ms_handle_reset on v2:172.21.15.143:6833/3664584522 2023-11-08T19:29:48.088 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.060+0000 7faf77e29b80 1 --2- 172.21.15.143:0/1546948523 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5631272c3ad0 0x5631272f62f0 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:48.088 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.060+0000 7faf77e29b80 1 -- 172.21.15.143:0/1546948523 >> 172.21.15.143:0/1546948523 conn(0x56312723e660 msgr2=0x5631272c68a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:48.088 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.060+0000 7faf77e29b80 1 -- 172.21.15.143:0/1546948523 shutdown_connections 2023-11-08T19:29:48.088 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.060+0000 7faf77e29b80 1 -- 172.21.15.143:0/1546948523 wait complete. 2023-11-08T19:29:48.088 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.060+0000 7faf77e29b80 20 client.7542 trim_cache size 0 max 16384 2023-11-08T19:29:48.094 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.078+0000 7faf77e29b80 1 Processor -- start 2023-11-08T19:29:48.094 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.078+0000 7faf77e29b80 1 -- start start 2023-11-08T19:29:48.095 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.079+0000 7faf77e29b80 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x5631272c36f0 0x5631272c29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:48.095 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.079+0000 7faf77e29b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5631272c3ad0 0x5631272c4690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:48.095 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.079+0000 7faf77e29b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5631272c4020 0x56312730d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:48.095 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.079+0000 7faf77e29b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x56312709b8d0 con 0x5631272c4020 2023-11-08T19:29:48.095 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.079+0000 7faf77e29b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x563127330d10 con 0x5631272c3ad0 2023-11-08T19:29:48.096 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.079+0000 7faf77e29b80 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x5631272ddce0 con 0x5631272c36f0 2023-11-08T19:29:48.096 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.079+0000 7faf6a1cd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5631272c4020 0x56312730d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:48.096 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.079+0000 7faf6a1cd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5631272c4020 0x56312730d840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55884/0 (socket says 172.21.15.143:55884) 2023-11-08T19:29:48.096 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.079+0000 7faf6a1cd700 1 -- 172.21.15.143:0/1351486901 learned_addr learned my addr 172.21.15.143:0/1351486901 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:29:48.096 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.079+0000 7faf699cc700 1 --2- 172.21.15.143:0/1351486901 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x5631272c36f0 0x5631272c29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:48.097 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.079+0000 7faf691cb700 1 --2- 172.21.15.143:0/1351486901 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5631272c3ad0 0x5631272c4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:48.097 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.079+0000 7faf6a1cd700 1 -- 172.21.15.143:0/1351486901 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x5631272c36f0 msgr2=0x5631272c29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:48.097 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.079+0000 7faf6a1cd700 1 --2- 172.21.15.143:0/1351486901 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x5631272c36f0 0x5631272c29d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:48.097 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.079+0000 7faf6a1cd700 1 -- 172.21.15.143:0/1351486901 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5631272c3ad0 msgr2=0x5631272c4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:48.097 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.079+0000 7faf6a1cd700 1 --2- 172.21.15.143:0/1351486901 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5631272c3ad0 0x5631272c4690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:48.098 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.079+0000 7faf6a1cd700 1 -- 172.21.15.143:0/1351486901 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56312702ec10 con 0x5631272c4020 2023-11-08T19:29:48.098 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.079+0000 7faf699cc700 1 --2- 172.21.15.143:0/1351486901 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x5631272c36f0 0x5631272c29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:29:48.098 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.079+0000 7faf6a1cd700 1 --2- 172.21.15.143:0/1351486901 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5631272c4020 0x56312730d840 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7faf5c058b00 tx=0x7faf5c009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c037000034e955a7 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:48.098 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.079+0000 7faf47fff700 1 -- 172.21.15.143:0/1351486901 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf5c059ea0 con 0x5631272c4020 2023-11-08T19:29:48.098 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.079+0000 7faf47fff700 1 -- 172.21.15.143:0/1351486901 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf5c05ccc0 con 0x5631272c4020 2023-11-08T19:29:48.099 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.079+0000 7faf47fff700 1 -- 172.21.15.143:0/1351486901 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf5c060e70 con 0x5631272c4020 2023-11-08T19:29:48.099 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.080+0000 7faf77e29b80 1 -- 172.21.15.143:0/1351486901 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5631272c4020 msgr2=0x56312730d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:48.099 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.080+0000 7faf77e29b80 1 --2- 172.21.15.143:0/1351486901 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5631272c4020 0x56312730d840 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7faf5c058b00 tx=0x7faf5c009040 comp rx=0 tx=0).stop 2023-11-08T19:29:48.099 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.080+0000 7faf77e29b80 1 -- 172.21.15.143:0/1351486901 shutdown_connections 2023-11-08T19:29:48.099 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.080+0000 7faf77e29b80 1 --2- 172.21.15.143:0/1351486901 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x5631272c36f0 0x5631272c29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:48.100 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.080+0000 7faf77e29b80 1 --2- 172.21.15.143:0/1351486901 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5631272c4020 0x56312730d840 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:48.100 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.080+0000 7faf77e29b80 1 --2- 172.21.15.143:0/1351486901 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5631272c3ad0 0x5631272c4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:48.100 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.080+0000 7faf77e29b80 1 -- 172.21.15.143:0/1351486901 >> 172.21.15.143:0/1351486901 conn(0x56312723e660 msgr2=0x56312731d230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:48.100 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.080+0000 7faf77e29b80 1 -- 172.21.15.143:0/1351486901 shutdown_connections 2023-11-08T19:29:48.100 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.080+0000 7faf77e29b80 1 -- 172.21.15.143:0/1351486901 wait complete. 2023-11-08T19:29:48.101 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.080+0000 7faf77e29b80 1 -- start start 2023-11-08T19:29:48.101 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.081+0000 7faf77e29b80 1 Processor -- start 2023-11-08T19:29:48.101 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.081+0000 7faf77e29b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5631272c36f0 0x5631272dd770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:48.101 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.081+0000 7faf77e29b80 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x5631272c3ad0 0x56312736e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:48.101 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.081+0000 7faf77e29b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5631272c4020 0x563127377a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:48.102 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.081+0000 7faf77e29b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631272dd1b0 con 0x5631272c4020 2023-11-08T19:29:48.102 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.081+0000 7faf77e29b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x563127378490 con 0x5631272c36f0 2023-11-08T19:29:48.102 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.081+0000 7faf77e29b80 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x5631272dd320 con 0x5631272c3ad0 2023-11-08T19:29:48.102 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.081+0000 7faf77e29b80 10 client.0 _ll_register_callbacks cb 0x7faf540039b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T19:29:48.102 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.082+0000 7faf6a1cd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5631272c4020 0x563127377a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:48.103 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.082+0000 7faf6a1cd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5631272c4020 0x563127377a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55890/0 (socket says 172.21.15.143:55890) 2023-11-08T19:29:48.103 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.082+0000 7faf6a1cd700 1 -- 172.21.15.143:0/965514642 learned_addr learned my addr 172.21.15.143:0/965514642 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:29:48.103 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.082+0000 7faf691cb700 1 --2- 172.21.15.143:0/965514642 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x5631272c3ad0 0x56312736e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:48.103 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.082+0000 7faf699cc700 1 --2- 172.21.15.143:0/965514642 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5631272c36f0 0x5631272dd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:48.103 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.082+0000 7faf6a1cd700 1 -- 172.21.15.143:0/965514642 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x5631272c3ad0 msgr2=0x56312736e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:48.104 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.082+0000 7faf6a1cd700 1 --2- 172.21.15.143:0/965514642 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x5631272c3ad0 0x56312736e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:48.104 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.082+0000 7faf6a1cd700 1 -- 172.21.15.143:0/965514642 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5631272c36f0 msgr2=0x5631272dd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:48.104 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.082+0000 7faf6a1cd700 1 --2- 172.21.15.143:0/965514642 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5631272c36f0 0x5631272dd770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:48.104 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.082+0000 7faf6a1cd700 1 -- 172.21.15.143:0/965514642 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf641f3bf0 con 0x5631272c4020 2023-11-08T19:29:48.104 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.082+0000 7faf6a1cd700 1 --2- 172.21.15.143:0/965514642 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5631272c4020 0x563127377a10 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7faf5c058870 tx=0x7faf5c000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f73098ffd25b09b1 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:48.105 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.082+0000 7faf44ff9700 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2023-11-08T19:29:48.105 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.082+0000 7faf44ff9700 1 -- 172.21.15.143:0/965514642 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf5c0112b0 con 0x5631272c4020 2023-11-08T19:29:48.105 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.082+0000 7faf44ff9700 1 -- 172.21.15.143:0/965514642 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 7059+0+0 (secure 0 0 0) 0x7faf5c011410 con 0x5631272c4020 2023-11-08T19:29:48.105 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.082+0000 7faf77e29b80 1 -- 172.21.15.143:0/965514642 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7faf54005500 con 0x5631272c4020 2023-11-08T19:29:48.105 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.082+0000 7faf44ff9700 1 -- 172.21.15.143:0/965514642 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7faf5c058dd0 con 0x5631272c4020 2023-11-08T19:29:48.106 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.082+0000 7faf77e29b80 10 client.7548 fetch_fsmap learned FSMap version 80 2023-11-08T19:29:48.106 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.082+0000 7faf77e29b80 1 -- 172.21.15.143:0/965514642 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=2+}) v3 -- 0x56312706c200 con 0x5631272c4020 2023-11-08T19:29:48.106 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.082+0000 7faf44ff9700 1 -- 172.21.15.143:0/965514642 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf5c081080 con 0x5631272c4020 2023-11-08T19:29:48.106 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.082+0000 7faf44ff9700 1 -- 172.21.15.143:0/965514642 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 80) v1 ==== 3306+0+0 (secure 0 0 0) 0x7faf5c084070 con 0x5631272c4020 2023-11-08T19:29:48.106 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.082+0000 7faf77e29b80 10 client.7548 fetch_fsmap finished waiting for FSMap version 80 2023-11-08T19:29:48.107 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.082+0000 7faf77e29b80 10 client.7548 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-08T19:29:48.107 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.082+0000 7faf77e29b80 20 client.7548 populate_metadata read hostname 'smithi143' 2023-11-08T19:29:48.107 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.082+0000 7faf77e29b80 1 --2- 172.21.15.143:0/965514642 >> [v2:172.21.15.143:6833/3664584522,v1:172.21.15.143:6835/3664584522] conn(0x5631272ddf40 0x5631272de320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:48.107 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.082+0000 7faf77e29b80 4 client.7548 mds_command: new command op to 7386 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-08T19:29:48.107 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.082+0000 7faf77e29b80 1 -- 172.21.15.143:0/965514642 --> [v2:172.21.15.143:6833/3664584522,v1:172.21.15.143:6835/3664584522] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5631272155a0 con 0x5631272ddf40 2023-11-08T19:29:48.108 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.083+0000 7faf699cc700 1 --2- 172.21.15.143:0/965514642 >> [v2:172.21.15.143:6833/3664584522,v1:172.21.15.143:6835/3664584522] conn(0x5631272ddf40 0x5631272de320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:48.108 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.083+0000 7faf44ff9700 10 client.7548 ms_handle_connect on v2:172.21.15.143:6833/3664584522 2023-11-08T19:29:48.108 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.083+0000 7faf699cc700 1 --2- 172.21.15.143:0/965514642 >> [v2:172.21.15.143:6833/3664584522,v1:172.21.15.143:6835/3664584522] conn(0x5631272ddf40 0x5631272de320 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89325dacee80f348 server_cookie=134dbda55c84d7cb in_seq=0 out_seq=0 2023-11-08T19:29:48.108 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.083+0000 7faf44ff9700 1 -- 172.21.15.143:0/965514642 <== mds.0 v2:172.21.15.143:6833/3664584522 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x5631272155a0 con 0x5631272ddf40 2023-11-08T19:29:48.108 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.083+0000 7faf44ff9700 10 client.7548 handle_command_reply: tid=0 2023-11-08T19:29:48.108 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.083+0000 7faf77e29b80 1 client.7548 shutdown 2023-11-08T19:29:48.109 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.083+0000 7faf77e29b80 1 -- 172.21.15.143:0/965514642 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5631272c4020 msgr2=0x563127377a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:48.109 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.083+0000 7faf77e29b80 1 --2- 172.21.15.143:0/965514642 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5631272c4020 0x563127377a10 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7faf5c058870 tx=0x7faf5c000f40 comp rx=0 tx=0).stop 2023-11-08T19:29:48.109 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.083+0000 7faf77e29b80 1 -- 172.21.15.143:0/965514642 shutdown_connections 2023-11-08T19:29:48.109 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.083+0000 7faf77e29b80 1 --2- 172.21.15.143:0/965514642 >> [v2:172.21.15.143:6833/3664584522,v1:172.21.15.143:6835/3664584522] conn(0x5631272ddf40 0x5631272de320 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:48.109 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.083+0000 7faf77e29b80 1 --2- 172.21.15.143:0/965514642 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5631272c4020 0x563127377a10 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:48.110 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.083+0000 7faf77e29b80 1 --2- 172.21.15.143:0/965514642 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5631272c36f0 0x5631272dd770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:48.110 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.083+0000 7faf44ff9700 0 client.7548 ms_handle_reset on v2:172.21.15.143:6833/3664584522 2023-11-08T19:29:48.110 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.083+0000 7faf77e29b80 1 --2- 172.21.15.143:0/965514642 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x5631272c3ad0 0x56312736e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:48.110 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.083+0000 7faf77e29b80 1 -- 172.21.15.143:0/965514642 >> 172.21.15.143:0/965514642 conn(0x56312723e660 msgr2=0x5631272bf7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:48.110 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.083+0000 7faf77e29b80 1 -- 172.21.15.143:0/965514642 shutdown_connections 2023-11-08T19:29:48.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.083+0000 7faf77e29b80 1 -- 172.21.15.143:0/965514642 wait complete. 2023-11-08T19:29:48.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.084+0000 7faf77e29b80 20 client.7548 trim_cache size 0 max 16384 2023-11-08T19:29:48.111 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2023-11-08T19:29:48.111 INFO:teuthology.orchestra.run.smithi143.stdout: "cluster_fsid": "968483af-77fc-4068-ad7c-5c2a1a4613d7", 2023-11-08T19:29:48.112 INFO:teuthology.orchestra.run.smithi143.stdout: "whoami": 0, 2023-11-08T19:29:48.112 INFO:teuthology.orchestra.run.smithi143.stdout: "id": 7386, 2023-11-08T19:29:48.112 INFO:teuthology.orchestra.run.smithi143.stdout: "want_state": "up:active", 2023-11-08T19:29:48.112 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "up:active", 2023-11-08T19:29:48.112 INFO:teuthology.orchestra.run.smithi143.stdout: "fs_name": "cephfs", 2023-11-08T19:29:48.113 INFO:teuthology.orchestra.run.smithi143.stdout: "rank_uptime": 6.9232409370000001, 2023-11-08T19:29:48.113 INFO:teuthology.orchestra.run.smithi143.stdout: "mdsmap_epoch": 80, 2023-11-08T19:29:48.113 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_epoch": 97, 2023-11-08T19:29:48.113 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_epoch_barrier": 97, 2023-11-08T19:29:48.113 INFO:teuthology.orchestra.run.smithi143.stdout: "uptime": 33.038963334000002 2023-11-08T19:29:48.114 INFO:teuthology.orchestra.run.smithi143.stdout:} 2023-11-08T19:29:48.115 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.084+0000 7faf6bc30700 1 -- 172.21.15.143:0/1390032910 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7faf4806af00 msgr2=0x7faf4806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:48.116 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.084+0000 7faf6bc30700 1 --2- 172.21.15.143:0/1390032910 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7faf4806af00 0x7faf4806d3c0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7faf600090b0 tx=0x7faf6001f040 comp rx=0 tx=0).stop 2023-11-08T19:29:48.116 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.084+0000 7faf6bc30700 1 -- 172.21.15.143:0/1390032910 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faf6415c040 msgr2=0x7faf6415c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:48.116 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.084+0000 7faf6bc30700 1 --2- 172.21.15.143:0/1390032910 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faf6415c040 0x7faf6415c4f0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7faf640cab80 tx=0x7faf5c00a5f0 comp rx=0 tx=0).stop 2023-11-08T19:29:48.116 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.084+0000 7faf6bc30700 1 -- 172.21.15.143:0/1390032910 shutdown_connections 2023-11-08T19:29:48.116 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.084+0000 7faf6bc30700 1 --2- 172.21.15.143:0/1390032910 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7faf4806af00 0x7faf4806d3c0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:48.117 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.084+0000 7faf6bc30700 1 --2- 172.21.15.143:0/1390032910 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faf6415c040 0x7faf6415c4f0 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:48.117 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.084+0000 7faf6bc30700 1 --2- 172.21.15.143:0/1390032910 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faf50003a30 0x7faf6414e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:48.117 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.084+0000 7faf6bc30700 1 --2- 172.21.15.143:0/1390032910 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7faf641545a0 0x7faf64154a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:48.117 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.084+0000 7faf6bc30700 1 -- 172.21.15.143:0/1390032910 >> 172.21.15.143:0/1390032910 conn(0x7faf6406b4c0 msgr2=0x7faf6406cac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:48.117 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.084+0000 7faf6bc30700 1 -- 172.21.15.143:0/1390032910 shutdown_connections 2023-11-08T19:29:48.118 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.084+0000 7faf6bc30700 1 -- 172.21.15.143:0/1390032910 wait complete. 2023-11-08T19:29:48.118 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:29:48.240 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.239+0000 7f08da9a2700 1 Processor -- start 2023-11-08T19:29:48.240 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.239+0000 7f08da9a2700 1 -- start start 2023-11-08T19:29:48.243 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.239+0000 7f08da9a2700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f08d415dd00 0x7f08d40cca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:48.243 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.239+0000 7f08da9a2700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f08d41618f0 0x7f08d406ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:48.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.239+0000 7f08da9a2700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08d40ccfb0 con 0x7f08d406b150 2023-11-08T19:29:48.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.239+0000 7f08da9a2700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08d406b600 con 0x7f08d415dd00 2023-11-08T19:29:48.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.239+0000 7f08da9a2700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08d40cad50 con 0x7f08d41618f0 2023-11-08T19:29:48.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.239+0000 7f08d3fff700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f08d415dd00 0x7f08d40cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:48.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.240+0000 7f08d3fff700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f08d415dd00 0x7f08d40cca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55894/0 (socket says 172.21.15.143:55894) 2023-11-08T19:29:48.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.240+0000 7f08d3fff700 1 -- 172.21.15.143:0/2264812363 learned_addr learned my addr 172.21.15.143:0/2264812363 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:29:48.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.240+0000 7f08d37fe700 1 --2- 172.21.15.143:0/2264812363 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f08d41618f0 0x7f08d406ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:48.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.240+0000 7f08d3fff700 1 -- 172.21.15.143:0/2264812363 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f08d41618f0 msgr2=0x7f08d406ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:48.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.240+0000 7f08d3fff700 1 --2- 172.21.15.143:0/2264812363 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f08d41618f0 0x7f08d406ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:48.246 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.240+0000 7f08d3fff700 1 -- 172.21.15.143:0/2264812363 >> v1:172.21.15.174:6789/0 conn(0x7f08d406b150 legacy=0x7f08d40b2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:48.246 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.240+0000 7f08d3fff700 1 -- 172.21.15.143:0/2264812363 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08d4161df0 con 0x7f08d415dd00 2023-11-08T19:29:48.246 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.240+0000 7f08d3fff700 1 --2- 172.21.15.143:0/2264812363 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f08d415dd00 0x7f08d40cca70 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f08c4006fd0 tx=0x7f08c4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6307b1ae1027141f server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:48.246 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.240+0000 7f08d2ffd700 1 -- 172.21.15.143:0/2264812363 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c4005e30 con 0x7f08d415dd00 2023-11-08T19:29:48.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.240+0000 7f08d2ffd700 1 -- 172.21.15.143:0/2264812363 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08c4006030 con 0x7f08d415dd00 2023-11-08T19:29:48.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.240+0000 7f08d2ffd700 1 -- 172.21.15.143:0/2264812363 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c40063c0 con 0x7f08d415dd00 2023-11-08T19:29:48.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.241+0000 7f08da9a2700 1 -- 172.21.15.143:0/2264812363 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f08d415dd00 msgr2=0x7f08d40cca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:48.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.241+0000 7f08da9a2700 1 --2- 172.21.15.143:0/2264812363 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f08d415dd00 0x7f08d40cca70 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f08c4006fd0 tx=0x7f08c4009bd0 comp rx=0 tx=0).stop 2023-11-08T19:29:48.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.241+0000 7f08da9a2700 1 -- 172.21.15.143:0/2264812363 shutdown_connections 2023-11-08T19:29:48.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.241+0000 7f08da9a2700 1 --2- 172.21.15.143:0/2264812363 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f08d415dd00 0x7f08d40cca70 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:48.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.241+0000 7f08da9a2700 1 --2- 172.21.15.143:0/2264812363 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f08d41618f0 0x7f08d406ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:48.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.241+0000 7f08da9a2700 1 -- 172.21.15.143:0/2264812363 >> 172.21.15.143:0/2264812363 conn(0x7f08d4099ab0 msgr2=0x7f08d40d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:48.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.241+0000 7f08da9a2700 1 -- 172.21.15.143:0/2264812363 shutdown_connections 2023-11-08T19:29:48.249 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.241+0000 7f08da9a2700 1 -- 172.21.15.143:0/2264812363 wait complete. 2023-11-08T19:29:48.249 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.242+0000 7f08da9a2700 1 Processor -- start 2023-11-08T19:29:48.249 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.242+0000 7f08da9a2700 1 -- start start 2023-11-08T19:29:48.253 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.243+0000 7f08da9a2700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f08d406b150 0x7f08d4151f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:48.253 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.243+0000 7f08da9a2700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f08d415dd00 0x7f08d40b44e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:48.253 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.243+0000 7f08da9a2700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f08d41618f0 0x7f08d4160420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:48.254 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.243+0000 7f08da9a2700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08d40b34d0 con 0x7f08d415dd00 2023-11-08T19:29:48.254 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.243+0000 7f08da9a2700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08d40b3720 con 0x7f08d406b150 2023-11-08T19:29:48.254 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.243+0000 7f08da9a2700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08d40b3a00 con 0x7f08d41618f0 2023-11-08T19:29:48.254 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.243+0000 7f08d37fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f08d415dd00 0x7f08d40b44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:48.254 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.243+0000 7f08d37fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f08d415dd00 0x7f08d40b44e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55902/0 (socket says 172.21.15.143:55902) 2023-11-08T19:29:48.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.243+0000 7f08d37fe700 1 -- 172.21.15.143:0/2027763604 learned_addr learned my addr 172.21.15.143:0/2027763604 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:29:48.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.243+0000 7f08d3fff700 1 --2- 172.21.15.143:0/2027763604 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f08d406b150 0x7f08d4151f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:48.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.243+0000 7f08d8f3f700 1 --2- 172.21.15.143:0/2027763604 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f08d41618f0 0x7f08d4160420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:48.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.243+0000 7f08d37fe700 1 -- 172.21.15.143:0/2027763604 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f08d41618f0 msgr2=0x7f08d4160420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:48.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.243+0000 7f08d37fe700 1 --2- 172.21.15.143:0/2027763604 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f08d41618f0 0x7f08d4160420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:48.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.243+0000 7f08d37fe700 1 -- 172.21.15.143:0/2027763604 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f08d406b150 msgr2=0x7f08d4151f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:48.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.243+0000 7f08d37fe700 1 --2- 172.21.15.143:0/2027763604 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f08d406b150 0x7f08d4151f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:48.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.243+0000 7f08d37fe700 1 -- 172.21.15.143:0/2027763604 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08c4009500 con 0x7f08d415dd00 2023-11-08T19:29:48.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.243+0000 7f08d3fff700 1 --2- 172.21.15.143:0/2027763604 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f08d406b150 0x7f08d4151f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:29:48.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.243+0000 7f08d37fe700 1 --2- 172.21.15.143:0/2027763604 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f08d415dd00 0x7f08d40b44e0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f08bc00d800 tx=0x7f08bc00db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:48.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.243+0000 7f08d0ff9700 1 -- 172.21.15.143:0/2027763604 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08bc0125f0 con 0x7f08d415dd00 2023-11-08T19:29:48.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.243+0000 7f08d0ff9700 1 -- 172.21.15.143:0/2027763604 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08bc012750 con 0x7f08d415dd00 2023-11-08T19:29:48.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.243+0000 7f08da9a2700 1 -- 172.21.15.143:0/2027763604 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08d40b3ce0 con 0x7f08d415dd00 2023-11-08T19:29:48.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.243+0000 7f08d0ff9700 1 -- 172.21.15.143:0/2027763604 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08bc012b00 con 0x7f08d415dd00 2023-11-08T19:29:48.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.244+0000 7f08da9a2700 1 -- 172.21.15.143:0/2027763604 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08d41524a0 con 0x7f08d415dd00 2023-11-08T19:29:48.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.244+0000 7f08da9a2700 1 -- 172.21.15.143:0/2027763604 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08d40aff60 con 0x7f08d415dd00 2023-11-08T19:29:48.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.245+0000 7f08d0ff9700 1 -- 172.21.15.143:0/2027763604 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f08bc012c60 con 0x7f08d415dd00 2023-11-08T19:29:48.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.245+0000 7f08d0ff9700 1 --2- 172.21.15.143:0/2027763604 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f08b406af00 0x7f08b406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:48.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.245+0000 7f08d0ff9700 1 -- 172.21.15.143:0/2027763604 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 7059+0+0 (secure 0 0 0) 0x7f08bc089db0 con 0x7f08d415dd00 2023-11-08T19:29:48.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.245+0000 7f08d3fff700 1 --2- 172.21.15.143:0/2027763604 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f08b406af00 0x7f08b406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:48.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.246+0000 7f08d3fff700 1 --2- 172.21.15.143:0/2027763604 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f08b406af00 0x7f08b406d3c0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f08c4013040 tx=0x7f08c4008070 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:48.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.250+0000 7f08d0ff9700 1 -- 172.21.15.143:0/2027763604 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f08bc0549a0 con 0x7f08d415dd00 2023-11-08T19:29:48.479 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.478+0000 7f08da9a2700 1 -- 172.21.15.143:0/2027763604 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f08d404eee0 con 0x7f08d415dd00 2023-11-08T19:29:48.480 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.478+0000 7f08d0ff9700 1 -- 172.21.15.143:0/2027763604 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f08bc057fc0 con 0x7f08d415dd00 2023-11-08T19:29:48.480 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:29:48.480 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":80,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5933,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/3918286710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6835","nonce":3918286710},{"type":"v1","addr":"172.21.15.174:6837","nonce":3918286710}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":7494,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6834/1535103237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":1535103237},{"type":"v1","addr":"172.21.15.143:6834","nonce":1535103237}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:03.092245+0000","modified":"2023-11-08T19:29:44.172957+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":97,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7386,"mds_1":6918},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6918":{"gid":6918,"name":"b","rank":1,"incarnation":54,"state":"up:active","state_seq":6,"addr":"172.21.15.174:6836/3601088618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3601088618},{"type":"v1","addr":"172.21.15.174:6836","nonce":3601088618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7386":{"gid":7386,"name":"a","rank":0,"incarnation":76,"state":"up:active","state_seq":11,"addr":"172.21.15.143:6835/3664584522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":3664584522},{"type":"v1","addr":"172.21.15.143:6835","nonce":3664584522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:29:48.482 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.480+0000 7f08da9a2700 1 -- 172.21.15.143:0/2027763604 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f08b406af00 msgr2=0x7f08b406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:48.482 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.480+0000 7f08da9a2700 1 --2- 172.21.15.143:0/2027763604 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f08b406af00 0x7f08b406d3c0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f08c4013040 tx=0x7f08c4008070 comp rx=0 tx=0).stop 2023-11-08T19:29:48.483 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.480+0000 7f08da9a2700 1 -- 172.21.15.143:0/2027763604 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f08d415dd00 msgr2=0x7f08d40b44e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:48.483 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.480+0000 7f08da9a2700 1 --2- 172.21.15.143:0/2027763604 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f08d415dd00 0x7f08d40b44e0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f08bc00d800 tx=0x7f08bc00db10 comp rx=0 tx=0).stop 2023-11-08T19:29:48.483 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.481+0000 7f08da9a2700 1 -- 172.21.15.143:0/2027763604 shutdown_connections 2023-11-08T19:29:48.483 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.481+0000 7f08da9a2700 1 --2- 172.21.15.143:0/2027763604 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f08b406af00 0x7f08b406d3c0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:48.483 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.481+0000 7f08da9a2700 1 --2- 172.21.15.143:0/2027763604 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f08d41618f0 0x7f08d4160420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:48.484 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.481+0000 7f08da9a2700 1 --2- 172.21.15.143:0/2027763604 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f08d406b150 0x7f08d4151f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:48.484 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.481+0000 7f08da9a2700 1 --2- 172.21.15.143:0/2027763604 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f08d415dd00 0x7f08d40b44e0 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:48.484 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.481+0000 7f08da9a2700 1 -- 172.21.15.143:0/2027763604 >> 172.21.15.143:0/2027763604 conn(0x7f08d4099ab0 msgr2=0x7f08d40ad380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:48.484 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.481+0000 7f08da9a2700 1 -- 172.21.15.143:0/2027763604 shutdown_connections 2023-11-08T19:29:48.484 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.481+0000 7f08da9a2700 1 -- 172.21.15.143:0/2027763604 wait complete. 2023-11-08T19:29:48.485 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 80 2023-11-08T19:29:48.494 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json dump snaps --server 2023-11-08T19:29:48.631 INFO:teuthology.orchestra.run.smithi143.stdout:{"last_snap":4,"last_created":4,"last_destroyed":1,"pending_noop":[],"snaps":[{"snapid":2,"ino":1099511627777,"stamp":"2023-11-08T19:28:31.007395+0000","name":"s11","metadata":{}},{"snapid":3,"ino":1099511627777,"stamp":"2023-11-08T19:28:57.577300+0000","name":"s12","metadata":{}},{"snapid":4,"ino":1099511627777,"stamp":"2023-11-08T19:29:23.827405+0000","name":"s15","metadata":{}}],"need_to_purge":{},"pending_update":[],"pending_destroy":[]} 2023-11-08T19:29:48.632 DEBUG:tasks.cephfs.filesystem:_json_asok output { "last_created": 4, "last_destroyed": 1, "last_snap": 4, "need_to_purge": {}, "pending_destroy": [], "pending_noop": [], "pending_update": [], "snaps": [ { "ino": 1099511627777, "metadata": {}, "name": "s11", "snapid": 2, "stamp": "2023-11-08T19:28:31.007395+0000" }, { "ino": 1099511627777, "metadata": {}, "name": "s12", "snapid": 3, "stamp": "2023-11-08T19:28:57.577300+0000" }, { "ino": 1099511627777, "metadata": {}, "name": "s15", "snapid": 4, "stamp": "2023-11-08T19:29:23.827405+0000" } ] } 2023-11-08T19:29:48.632 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-11-08T19:29:48.633 INFO:tasks.cephfs.test_snapshots:testing snapserver mds_kill_mdstable_at=6 2023-11-08T19:29:48.633 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:29:48.753 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.752+0000 7fcd74aa4700 1 Processor -- start 2023-11-08T19:29:48.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.752+0000 7fcd74aa4700 1 -- start start 2023-11-08T19:29:48.758 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.753+0000 7fcd74aa4700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcd7006b320 0x7fcd70168c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:48.758 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.753+0000 7fcd74aa4700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd70161df0 con 0x7fcd701618f0 2023-11-08T19:29:48.758 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.753+0000 7fcd74aa4700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd701620d0 con 0x7fcd7015dd00 2023-11-08T19:29:48.758 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.753+0000 7fcd74aa4700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd700cad50 con 0x7fcd7006b320 2023-11-08T19:29:48.759 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.753+0000 7fcd6dd9b700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fcd701618f0 0x7fcd7006ac10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:52062/0 (socket says 172.21.15.143:52062) 2023-11-08T19:29:48.759 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.753+0000 7fcd6dd9b700 1 -- 172.21.15.143:0/914984565 learned_addr learned my addr 172.21.15.143:0/914984565 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:29:48.759 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.753+0000 7fcd6d59a700 1 -- 172.21.15.143:0/914984565 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2245492445 0 0) 0x7fcd70161df0 con 0x7fcd701618f0 2023-11-08T19:29:48.759 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.753+0000 7fcd6d59a700 1 -- 172.21.15.143:0/914984565 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd580035b0 con 0x7fcd701618f0 2023-11-08T19:29:48.759 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.753+0000 7fcd6ed9d700 1 --2- 172.21.15.143:0/914984565 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcd7006b320 0x7fcd70168c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:48.760 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.753+0000 7fcd6d59a700 1 -- 172.21.15.143:0/914984565 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2263876195 0 0) 0x7fcd580035b0 con 0x7fcd701618f0 2023-11-08T19:29:48.760 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.754+0000 7fcd6d59a700 1 -- 172.21.15.143:0/914984565 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcd7006b320 msgr2=0x7fcd70168c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:48.760 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.754+0000 7fcd6d59a700 1 --2- 172.21.15.143:0/914984565 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcd7006b320 0x7fcd70168c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:48.760 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.754+0000 7fcd6d59a700 1 -- 172.21.15.143:0/914984565 >> v1:172.21.15.174:6789/0 conn(0x7fcd7015dd00 legacy=0x7fcd700cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:48.760 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.754+0000 7fcd6d59a700 1 -- 172.21.15.143:0/914984565 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd701623b0 con 0x7fcd701618f0 2023-11-08T19:29:48.761 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.754+0000 7fcd6d59a700 1 -- 172.21.15.143:0/914984565 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4147756303 0 0) 0x7fcd60001670 con 0x7fcd701618f0 2023-11-08T19:29:48.761 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.754+0000 7fcd6d59a700 1 -- 172.21.15.143:0/914984565 >> v1:172.21.15.143:6789/0 conn(0x7fcd701618f0 legacy=0x7fcd7006ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:48.761 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.754+0000 7fcd6d59a700 1 --2- 172.21.15.143:0/914984565 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcd58003a30 0x7fcd58003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:48.761 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.754+0000 7fcd6d59a700 1 --2- 172.21.15.143:0/914984565 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcd7006b320 0x7fcd58004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:48.762 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.754+0000 7fcd6d59a700 1 --2- 172.21.15.143:0/914984565 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcd58004d90 0x7fcd58005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:48.762 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.754+0000 7fcd6d59a700 1 -- 172.21.15.143:0/914984565 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd5800bac0 con 0x7fcd58004d90 2023-11-08T19:29:48.762 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.754+0000 7fcd6d59a700 1 -- 172.21.15.143:0/914984565 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd5800bc30 con 0x7fcd58003a30 2023-11-08T19:29:48.762 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.754+0000 7fcd6d59a700 1 -- 172.21.15.143:0/914984565 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd5800bdf0 con 0x7fcd7006b320 2023-11-08T19:29:48.762 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.754+0000 7fcd6ed9d700 1 --2- 172.21.15.143:0/914984565 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcd58004d90 0x7fcd58005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:48.763 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.754+0000 7fcd6dd9b700 1 --2- 172.21.15.143:0/914984565 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcd7006b320 0x7fcd58004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:48.763 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.754+0000 7fcd6e59c700 1 --2- 172.21.15.143:0/914984565 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcd58003a30 0x7fcd58003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:48.763 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.754+0000 7fcd6ed9d700 1 -- 172.21.15.143:0/914984565 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcd7006b320 msgr2=0x7fcd58004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:48.763 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.754+0000 7fcd6ed9d700 1 --2- 172.21.15.143:0/914984565 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcd7006b320 0x7fcd58004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:48.763 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.754+0000 7fcd6ed9d700 1 -- 172.21.15.143:0/914984565 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcd58003a30 msgr2=0x7fcd58003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:48.763 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.754+0000 7fcd6ed9d700 1 --2- 172.21.15.143:0/914984565 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcd58003a30 0x7fcd58003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:48.764 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.754+0000 7fcd6ed9d700 1 -- 172.21.15.143:0/914984565 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd580035b0 con 0x7fcd58004d90 2023-11-08T19:29:48.764 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.754+0000 7fcd6ed9d700 1 --2- 172.21.15.143:0/914984565 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcd58004d90 0x7fcd58005210 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fcd68009f90 tx=0x7fcd58005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5c08c229161f7b0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:48.764 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.754+0000 7fcd6d59a700 1 -- 172.21.15.143:0/914984565 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd6800ec30 con 0x7fcd58004d90 2023-11-08T19:29:48.764 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.754+0000 7fcd6d59a700 1 -- 172.21.15.143:0/914984565 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd6800f040 con 0x7fcd58004d90 2023-11-08T19:29:48.764 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.754+0000 7fcd6d59a700 1 -- 172.21.15.143:0/914984565 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd68014480 con 0x7fcd58004d90 2023-11-08T19:29:48.765 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.755+0000 7fcd74aa4700 1 -- 172.21.15.143:0/914984565 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcd58004d90 msgr2=0x7fcd58005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:48.765 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.755+0000 7fcd74aa4700 1 --2- 172.21.15.143:0/914984565 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcd58004d90 0x7fcd58005210 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fcd68009f90 tx=0x7fcd58005880 comp rx=0 tx=0).stop 2023-11-08T19:29:48.765 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.755+0000 7fcd6ed9d700 1 -- 172.21.15.143:0/914984565 reap_dead start 2023-11-08T19:29:48.765 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.755+0000 7fcd74aa4700 1 -- 172.21.15.143:0/914984565 shutdown_connections 2023-11-08T19:29:48.765 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.755+0000 7fcd74aa4700 1 -- 172.21.15.143:0/914984565 >> 172.21.15.143:0/914984565 conn(0x7fcd70099ab0 msgr2=0x7fcd700d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:48.765 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.755+0000 7fcd74aa4700 1 -- 172.21.15.143:0/914984565 shutdown_connections 2023-11-08T19:29:48.766 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.755+0000 7fcd74aa4700 1 -- 172.21.15.143:0/914984565 wait complete. 2023-11-08T19:29:48.766 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.756+0000 7fcd74aa4700 1 Processor -- start 2023-11-08T19:29:48.766 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.756+0000 7fcd74aa4700 1 -- start start 2023-11-08T19:29:48.769 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.757+0000 7fcd74aa4700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcd58004d90 0x7fcd700c77d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:48.769 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.757+0000 7fcd74aa4700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcd700c6520 0x7fcd700c9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:48.769 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.757+0000 7fcd74aa4700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcd701f3de0 0x7fcd701f4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:48.769 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.757+0000 7fcd74aa4700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd700c8650 con 0x7fcd700c6520 2023-11-08T19:29:48.770 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.757+0000 7fcd74aa4700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd700c69a0 con 0x7fcd701f3de0 2023-11-08T19:29:48.770 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.757+0000 7fcd74aa4700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd701f36b0 con 0x7fcd58004d90 2023-11-08T19:29:48.770 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.757+0000 7fcd6dd9b700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcd700c6520 0x7fcd700c9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:48.770 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.757+0000 7fcd6dd9b700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcd700c6520 0x7fcd700c9fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55924/0 (socket says 172.21.15.143:55924) 2023-11-08T19:29:48.770 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.757+0000 7fcd6dd9b700 1 -- 172.21.15.143:0/3139262061 learned_addr learned my addr 172.21.15.143:0/3139262061 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:29:48.771 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.757+0000 7fcd6e59c700 1 --2- 172.21.15.143:0/3139262061 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcd58004d90 0x7fcd700c77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:48.771 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.757+0000 7fcd6ed9d700 1 --2- 172.21.15.143:0/3139262061 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcd701f3de0 0x7fcd701f4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:48.771 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.757+0000 7fcd6dd9b700 1 -- 172.21.15.143:0/3139262061 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcd58004d90 msgr2=0x7fcd700c77d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:48.771 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.757+0000 7fcd6dd9b700 1 --2- 172.21.15.143:0/3139262061 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcd58004d90 0x7fcd700c77d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:48.771 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.757+0000 7fcd6dd9b700 1 -- 172.21.15.143:0/3139262061 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcd701f3de0 msgr2=0x7fcd701f4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:48.772 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.757+0000 7fcd6dd9b700 1 --2- 172.21.15.143:0/3139262061 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcd701f3de0 0x7fcd701f4260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:48.772 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.757+0000 7fcd6dd9b700 1 -- 172.21.15.143:0/3139262061 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd7015e260 con 0x7fcd700c6520 2023-11-08T19:29:48.772 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.758+0000 7fcd6ed9d700 1 --2- 172.21.15.143:0/3139262061 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcd701f3de0 0x7fcd701f4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:29:48.772 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.758+0000 7fcd6dd9b700 1 --2- 172.21.15.143:0/3139262061 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcd700c6520 0x7fcd700c9fd0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fcd6000eaf0 tx=0x7fcd6000ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:48.772 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.758+0000 7fcd5effd700 1 -- 172.21.15.143:0/3139262061 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd60005e00 con 0x7fcd700c6520 2023-11-08T19:29:48.773 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.758+0000 7fcd5effd700 1 -- 172.21.15.143:0/3139262061 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd6000c7e0 con 0x7fcd700c6520 2023-11-08T19:29:48.773 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.758+0000 7fcd74aa4700 1 -- 172.21.15.143:0/3139262061 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd701555b0 con 0x7fcd700c6520 2023-11-08T19:29:48.773 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.758+0000 7fcd5effd700 1 -- 172.21.15.143:0/3139262061 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd6000cb70 con 0x7fcd700c6520 2023-11-08T19:29:48.773 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.758+0000 7fcd74aa4700 1 -- 172.21.15.143:0/3139262061 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd701f3950 con 0x7fcd700c6520 2023-11-08T19:29:48.773 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.759+0000 7fcd74aa4700 1 -- 172.21.15.143:0/3139262061 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd701f3ae0 con 0x7fcd700c6520 2023-11-08T19:29:48.774 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.759+0000 7fcd5effd700 1 -- 172.21.15.143:0/3139262061 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fcd6000ccd0 con 0x7fcd700c6520 2023-11-08T19:29:48.774 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.760+0000 7fcd5effd700 1 --2- 172.21.15.143:0/3139262061 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fcd5006af00 0x7fcd5006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:48.774 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.760+0000 7fcd5effd700 1 -- 172.21.15.143:0/3139262061 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 7059+0+0 (secure 0 0 0) 0x7fcd60089740 con 0x7fcd700c6520 2023-11-08T19:29:48.774 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.760+0000 7fcd6e59c700 1 --2- 172.21.15.143:0/3139262061 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fcd5006af00 0x7fcd5006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:48.775 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.760+0000 7fcd6e59c700 1 --2- 172.21.15.143:0/3139262061 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fcd5006af00 0x7fcd5006d3c0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fcd640017e0 tx=0x7fcd64009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:48.775 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.764+0000 7fcd5effd700 1 -- 172.21.15.143:0/3139262061 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcd600542a0 con 0x7fcd700c6520 2023-11-08T19:29:48.997 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.995+0000 7fcd74aa4700 1 -- 172.21.15.143:0/3139262061 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcd701555b0 con 0x7fcd700c6520 2023-11-08T19:29:48.997 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.996+0000 7fcd5effd700 1 -- 172.21.15.143:0/3139262061 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fcd701555b0 con 0x7fcd700c6520 2023-11-08T19:29:48.998 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:29:48.998 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":80,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5933,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/3918286710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6835","nonce":3918286710},{"type":"v1","addr":"172.21.15.174:6837","nonce":3918286710}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":7494,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6834/1535103237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":1535103237},{"type":"v1","addr":"172.21.15.143:6834","nonce":1535103237}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:03.092245+0000","modified":"2023-11-08T19:29:44.172957+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":97,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7386,"mds_1":6918},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6918":{"gid":6918,"name":"b","rank":1,"incarnation":54,"state":"up:active","state_seq":6,"addr":"172.21.15.174:6836/3601088618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3601088618},{"type":"v1","addr":"172.21.15.174:6836","nonce":3601088618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7386":{"gid":7386,"name":"a","rank":0,"incarnation":76,"state":"up:active","state_seq":11,"addr":"172.21.15.143:6835/3664584522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":3664584522},{"type":"v1","addr":"172.21.15.143:6835","nonce":3664584522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:29:48.999 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.998+0000 7fcd74aa4700 1 -- 172.21.15.143:0/3139262061 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fcd5006af00 msgr2=0x7fcd5006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:49.000 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.998+0000 7fcd74aa4700 1 --2- 172.21.15.143:0/3139262061 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fcd5006af00 0x7fcd5006d3c0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fcd640017e0 tx=0x7fcd64009040 comp rx=0 tx=0).stop 2023-11-08T19:29:49.000 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.998+0000 7fcd74aa4700 1 -- 172.21.15.143:0/3139262061 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcd700c6520 msgr2=0x7fcd700c9fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:49.000 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.998+0000 7fcd74aa4700 1 --2- 172.21.15.143:0/3139262061 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcd700c6520 0x7fcd700c9fd0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fcd6000eaf0 tx=0x7fcd6000ee00 comp rx=0 tx=0).stop 2023-11-08T19:29:49.000 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.998+0000 7fcd74aa4700 1 -- 172.21.15.143:0/3139262061 shutdown_connections 2023-11-08T19:29:49.001 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.998+0000 7fcd74aa4700 1 --2- 172.21.15.143:0/3139262061 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fcd5006af00 0x7fcd5006d3c0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:49.001 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.998+0000 7fcd74aa4700 1 --2- 172.21.15.143:0/3139262061 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcd58004d90 0x7fcd700c77d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:49.001 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.998+0000 7fcd74aa4700 1 --2- 172.21.15.143:0/3139262061 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcd701f3de0 0x7fcd701f4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:49.001 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.998+0000 7fcd74aa4700 1 --2- 172.21.15.143:0/3139262061 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcd700c6520 0x7fcd700c9fd0 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:49.001 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.998+0000 7fcd74aa4700 1 -- 172.21.15.143:0/3139262061 >> 172.21.15.143:0/3139262061 conn(0x7fcd70099ab0 msgr2=0x7fcd700ad380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:49.001 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.998+0000 7fcd74aa4700 1 -- 172.21.15.143:0/3139262061 shutdown_connections 2023-11-08T19:29:49.002 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:48.998+0000 7fcd74aa4700 1 -- 172.21.15.143:0/3139262061 wait complete. 2023-11-08T19:29:49.002 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 80 2023-11-08T19:29:49.011 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds freeze 6:0 true 2023-11-08T19:29:49.133 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:49.132+0000 7fc8a58a3700 1 Processor -- start 2023-11-08T19:29:49.134 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:49.132+0000 7fc8a58a3700 1 -- start start 2023-11-08T19:29:49.137 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:49.133+0000 7fc8a58a3700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc8a00b5630 0x7fc8a00b5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:49.137 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:49.133+0000 7fc8a58a3700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc8a00d1bd0 0x7fc8a00d1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:49.137 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:49.133+0000 7fc8a58a3700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8a00b9b00 con 0x7fc8a015dca0 2023-11-08T19:29:49.137 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:49.133+0000 7fc8a58a3700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8a00b6020 con 0x7fc8a00b5630 2023-11-08T19:29:49.137 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:49.133+0000 7fc8a58a3700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a00b3360 con 0x7fc8a00d1bd0 2023-11-08T19:29:49.138 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:49.133+0000 7fc89e7fc700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc8a00b5630 0x7fc8a00b5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:49.138 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:49.133+0000 7fc89effd700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fc8a015dca0 0x7fc8a00b93f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:52082/0 (socket says 172.21.15.143:52082) 2023-11-08T19:29:49.138 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:49.133+0000 7fc89effd700 1 -- 172.21.15.143:0/1533811592 learned_addr learned my addr 172.21.15.143:0/1533811592 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:29:49.138 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:49.133+0000 7fc89f7fe700 1 --2- 172.21.15.143:0/1533811592 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc8a00d1bd0 0x7fc8a00d1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:49.139 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:49.134+0000 7fc89e7fc700 1 -- 172.21.15.143:0/1533811592 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc8a00d1bd0 msgr2=0x7fc8a00d1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:49.139 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:49.134+0000 7fc89e7fc700 1 --2- 172.21.15.143:0/1533811592 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc8a00d1bd0 0x7fc8a00d1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:49.139 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:49.134+0000 7fc89e7fc700 1 -- 172.21.15.143:0/1533811592 >> v1:172.21.15.143:6789/0 conn(0x7fc8a015dca0 legacy=0x7fc8a00b93f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:49.139 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:49.134+0000 7fc89e7fc700 1 -- 172.21.15.143:0/1533811592 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8a00b8f50 con 0x7fc8a00b5630 2023-11-08T19:29:49.139 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:49.134+0000 7fc89dffb700 1 -- 172.21.15.143:0/1533811592 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3040247250 0 0) 0x7fc8a00b9b00 con 0x7fc8a015dca0 2023-11-08T19:29:49.140 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:49.134+0000 7fc89e7fc700 1 --2- 172.21.15.143:0/1533811592 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc8a00b5630 0x7fc8a00b5ae0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7fc890009870 tx=0x7fc890009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a4f8c724fde93d5 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:49.140 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:49.134+0000 7fc89dffb700 1 -- 172.21.15.143:0/1533811592 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc890011830 con 0x7fc8a00b5630 2023-11-08T19:29:49.140 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:49.134+0000 7fc89dffb700 1 -- 172.21.15.143:0/1533811592 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc890011990 con 0x7fc8a00b5630 2023-11-08T19:29:49.140 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:49.134+0000 7fc89dffb700 1 -- 172.21.15.143:0/1533811592 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc890011d20 con 0x7fc8a00b5630 2023-11-08T19:29:49.140 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:49.134+0000 7fc8a58a3700 1 -- 172.21.15.143:0/1533811592 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc8a00b5630 msgr2=0x7fc8a00b5ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:49.141 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:49.134+0000 7fc8a58a3700 1 --2- 172.21.15.143:0/1533811592 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc8a00b5630 0x7fc8a00b5ae0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7fc890009870 tx=0x7fc890009b80 comp rx=0 tx=0).stop 2023-11-08T19:29:49.141 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:49.134+0000 7fc8a58a3700 1 -- 172.21.15.143:0/1533811592 shutdown_connections 2023-11-08T19:29:49.141 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:49.134+0000 7fc8a58a3700 1 --2- 172.21.15.143:0/1533811592 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc8a00d1bd0 0x7fc8a00d1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:49.141 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:49.134+0000 7fc8a58a3700 1 --2- 172.21.15.143:0/1533811592 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc8a00b5630 0x7fc8a00b5ae0 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:49.142 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:49.134+0000 7fc8a58a3700 1 -- 172.21.15.143:0/1533811592 >> 172.21.15.143:0/1533811592 conn(0x7fc8a0066d00 msgr2=0x7fc8a0067110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:49.142 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:49.134+0000 7fc8a58a3700 1 -- 172.21.15.143:0/1533811592 shutdown_connections 2023-11-08T19:29:49.142 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:49.135+0000 7fc8a58a3700 1 -- 172.21.15.143:0/1533811592 wait complete. 2023-11-08T19:29:49.142 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:49.136+0000 7fc8a58a3700 1 Processor -- start 2023-11-08T19:29:49.144 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:49.136+0000 7fc8a58a3700 1 -- start start 2023-11-08T19:29:49.145 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:49.136+0000 7fc8a58a3700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc8a00b5630 0x7fc8a015ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:49.145 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:49.137+0000 7fc8a58a3700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc8a00d1bd0 0x7fc8a00cd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:49.145 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:49.137+0000 7fc8a58a3700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc8a015dca0 0x7fc8a0160630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:49.145 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:49.137+0000 7fc8a58a3700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a01f7180 con 0x7fc8a00b5630 2023-11-08T19:29:49.146 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:49.137+0000 7fc8a58a3700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a01f7460 con 0x7fc8a00d1bd0 2023-11-08T19:29:49.146 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:49.137+0000 7fc8a58a3700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a01f7740 con 0x7fc8a015dca0 2023-11-08T19:29:49.146 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:49.137+0000 7fc89effd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc8a00b5630 0x7fc8a015ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:49.146 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:49.137+0000 7fc89effd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc8a00b5630 0x7fc8a015ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55934/0 (socket says 172.21.15.143:55934) 2023-11-08T19:29:49.146 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:49.137+0000 7fc89effd700 1 -- 172.21.15.143:0/2918836056 learned_addr learned my addr 172.21.15.143:0/2918836056 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:29:49.147 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:49.137+0000 7fc89effd700 1 -- 172.21.15.143:0/2918836056 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc8a015dca0 msgr2=0x7fc8a0160630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:49.147 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:49.137+0000 7fc89e7fc700 1 --2- 172.21.15.143:0/2918836056 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc8a00d1bd0 0x7fc8a00cd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:49.147 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:49.137+0000 7fc89f7fe700 1 --2- 172.21.15.143:0/2918836056 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc8a015dca0 0x7fc8a0160630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:49.147 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:49.137+0000 7fc89effd700 1 --2- 172.21.15.143:0/2918836056 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc8a015dca0 0x7fc8a0160630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:49.147 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:49.137+0000 7fc89effd700 1 -- 172.21.15.143:0/2918836056 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc8a00d1bd0 msgr2=0x7fc8a00cd3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:49.148 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:49.137+0000 7fc89effd700 1 --2- 172.21.15.143:0/2918836056 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc8a00d1bd0 0x7fc8a00cd3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:49.148 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:49.137+0000 7fc89effd700 1 -- 172.21.15.143:0/2918836056 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8900094a0 con 0x7fc8a00b5630 2023-11-08T19:29:49.148 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:49.137+0000 7fc89effd700 1 --2- 172.21.15.143:0/2918836056 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc8a00b5630 0x7fc8a015ade0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7fc89400c550 tx=0x7fc89400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:49.148 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:49.137+0000 7fc887fff700 1 -- 172.21.15.143:0/2918836056 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc894004c80 con 0x7fc8a00b5630 2023-11-08T19:29:49.148 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:49.137+0000 7fc887fff700 1 -- 172.21.15.143:0/2918836056 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc894004e00 con 0x7fc8a00b5630 2023-11-08T19:29:49.148 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:49.137+0000 7fc8a58a3700 1 -- 172.21.15.143:0/2918836056 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8a01f7a20 con 0x7fc8a00b5630 2023-11-08T19:29:49.149 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:49.137+0000 7fc887fff700 1 -- 172.21.15.143:0/2918836056 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc894007310 con 0x7fc8a00b5630 2023-11-08T19:29:49.149 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:49.137+0000 7fc8a58a3700 1 -- 172.21.15.143:0/2918836056 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8a015b7a0 con 0x7fc8a00b5630 2023-11-08T19:29:49.149 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:49.138+0000 7fc8a58a3700 1 -- 172.21.15.143:0/2918836056 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8a01f8070 con 0x7fc8a00b5630 2023-11-08T19:29:49.149 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:49.139+0000 7fc887fff700 1 -- 172.21.15.143:0/2918836056 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc894007470 con 0x7fc8a00b5630 2023-11-08T19:29:49.149 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:49.139+0000 7fc887fff700 1 --2- 172.21.15.143:0/2918836056 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc88c06af00 0x7fc88c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:49.149 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:49.139+0000 7fc887fff700 1 -- 172.21.15.143:0/2918836056 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 7059+0+0 (secure 0 0 0) 0x7fc894089820 con 0x7fc8a00b5630 2023-11-08T19:29:49.150 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:49.139+0000 7fc89e7fc700 1 --2- 172.21.15.143:0/2918836056 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc88c06af00 0x7fc88c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:49.150 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:49.140+0000 7fc89e7fc700 1 --2- 172.21.15.143:0/2918836056 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc88c06af00 0x7fc88c06d3c0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fc8a01f7040 tx=0x7fc890008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:49.150 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:49.143+0000 7fc887fff700 1 -- 172.21.15.143:0/2918836056 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc8a01f8070 con 0x7fc8a00b5630 2023-11-08T19:29:49.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:49.415+0000 7fc8a58a3700 1 -- 172.21.15.143:0/2918836056 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "mds freeze", "role_or_gid": "6:0", "val": "true"} v 0) v1 -- 0x7fc8a01f8070 con 0x7fc8a00b5630 2023-11-08T19:29:49.659 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:49.658+0000 7fc887fff700 1 -- 172.21.15.143:0/2918836056 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "mds freeze", "role_or_gid": "6:0", "val": "true"}]=0 freezing mds.7386 v81) v1 ==== 112+0+0 (secure 0 0 0) 0x7fc8a01f8070 con 0x7fc8a00b5630 2023-11-08T19:29:49.663 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:49.661+0000 7fc8a58a3700 1 -- 172.21.15.143:0/2918836056 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc88c06af00 msgr2=0x7fc88c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:49.663 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:49.661+0000 7fc8a58a3700 1 --2- 172.21.15.143:0/2918836056 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc88c06af00 0x7fc88c06d3c0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fc8a01f7040 tx=0x7fc890008030 comp rx=0 tx=0).stop 2023-11-08T19:29:49.664 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:49.661+0000 7fc8a58a3700 1 -- 172.21.15.143:0/2918836056 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc8a00b5630 msgr2=0x7fc8a015ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:49.664 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:49.661+0000 7fc8a58a3700 1 --2- 172.21.15.143:0/2918836056 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc8a00b5630 0x7fc8a015ade0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7fc89400c550 tx=0x7fc89400c910 comp rx=0 tx=0).stop 2023-11-08T19:29:49.664 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:49.661+0000 7fc8a58a3700 1 -- 172.21.15.143:0/2918836056 shutdown_connections 2023-11-08T19:29:49.664 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:49.661+0000 7fc8a58a3700 1 --2- 172.21.15.143:0/2918836056 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc88c06af00 0x7fc88c06d3c0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:49.664 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:49.661+0000 7fc8a58a3700 1 --2- 172.21.15.143:0/2918836056 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc8a015dca0 0x7fc8a0160630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:49.665 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:49.661+0000 7fc8a58a3700 1 --2- 172.21.15.143:0/2918836056 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc8a00b5630 0x7fc8a015ade0 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:49.665 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:49.661+0000 7fc8a58a3700 1 --2- 172.21.15.143:0/2918836056 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc8a00d1bd0 0x7fc8a00cd3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:49.665 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:49.661+0000 7fc8a58a3700 1 -- 172.21.15.143:0/2918836056 >> 172.21.15.143:0/2918836056 conn(0x7fc8a0066d00 msgr2=0x7fc8a00670e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:49.665 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:49.661+0000 7fc8a58a3700 1 -- 172.21.15.143:0/2918836056 shutdown_connections 2023-11-08T19:29:49.665 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:49.661+0000 7fc8a58a3700 1 -- 172.21.15.143:0/2918836056 wait complete. 2023-11-08T19:29:49.666 INFO:teuthology.orchestra.run.smithi143.stderr:freezing mds.7386 2023-11-08T19:29:49.675 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config set mds_kill_mdstable_at 6 2023-11-08T19:29:49.810 INFO:teuthology.orchestra.run.smithi143.stdout:{"success":"mds_kill_mdstable_at = '6' (not observed, change may require restart) "} 2023-11-08T19:29:49.810 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_kill_mdstable_at = '6' (not observed, change may require restart) " } 2023-11-08T19:29:49.811 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:29:49.811 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir d1/dir/.snap/s16) 2023-11-08T19:29:49.814 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mds_beacon_grace 2023-11-08T19:29:49.984 INFO:teuthology.orchestra.run.smithi143.stdout:{"mds_beacon_grace":"15.000000"} 2023-11-08T19:29:49.985 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_beacon_grace": "15.000000" } 2023-11-08T19:29:49.985 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:29:50.051 INFO:tasks.ceph.mds.a.smithi143.stderr:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el8/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: In function 'void MDSTableServer::_commit_logged(ceph::cref_t&)' thread 7f98b91d0700 time 2023-11-08T19:29:50.051347+0000 2023-11-08T19:29:50.052 INFO:tasks.ceph.mds.a.smithi143.stderr:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el8/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: 171: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 6) 2023-11-08T19:29:50.053 INFO:tasks.ceph.mds.a.smithi143.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-08T19:29:50.054 INFO:tasks.ceph.mds.a.smithi143.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f98caaa4f5b] 2023-11-08T19:29:50.054 INFO:tasks.ceph.mds.a.smithi143.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2ba121) [0x7f98caaa5121] 2023-11-08T19:29:50.055 INFO:tasks.ceph.mds.a.smithi143.stderr: 3: (MDSTableServer::_commit_logged(boost::intrusive_ptr const&)+0x2ab) [0x55ed92ae2f2b] 2023-11-08T19:29:50.055 INFO:tasks.ceph.mds.a.smithi143.stderr: 4: (MDSContext::complete(int)+0x5f) [0x55ed92b2fecf] 2023-11-08T19:29:50.055 INFO:tasks.ceph.mds.a.smithi143.stderr: 5: (MDSIOContextBase::complete(int)+0x574) [0x55ed92b306b4] 2023-11-08T19:29:50.056 INFO:tasks.ceph.mds.a.smithi143.stderr: 6: (MDSLogContextBase::complete(int)+0x5b) [0x55ed92b30a7b] 2023-11-08T19:29:50.056 INFO:tasks.ceph.mds.a.smithi143.stderr: 7: (Finisher::finisher_thread_entry()+0x19d) [0x7f98cab428cd] 2023-11-08T19:29:50.057 INFO:tasks.ceph.mds.a.smithi143.stderr: 8: /lib64/libpthread.so.0(+0x81cf) [0x7f98c983d1cf] 2023-11-08T19:29:50.057 INFO:tasks.ceph.mds.a.smithi143.stderr: 9: clone() 2023-11-08T19:29:50.057 INFO:tasks.ceph.mds.a.smithi143.stderr:2023-11-08T19:29:50.051+0000 7f98b91d0700 -1 /home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el8/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: In function 'void MDSTableServer::_commit_logged(ceph::cref_t&)' thread 7f98b91d0700 time 2023-11-08T19:29:50.051347+0000 2023-11-08T19:29:50.058 INFO:tasks.ceph.mds.a.smithi143.stderr:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el8/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: 171: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 6) 2023-11-08T19:29:50.058 INFO:tasks.ceph.mds.a.smithi143.stderr: 2023-11-08T19:29:50.058 INFO:tasks.ceph.mds.a.smithi143.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-08T19:29:50.059 INFO:tasks.ceph.mds.a.smithi143.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f98caaa4f5b] 2023-11-08T19:29:50.059 INFO:tasks.ceph.mds.a.smithi143.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2ba121) [0x7f98caaa5121] 2023-11-08T19:29:50.059 INFO:tasks.ceph.mds.a.smithi143.stderr: 3: (MDSTableServer::_commit_logged(boost::intrusive_ptr const&)+0x2ab) [0x55ed92ae2f2b] 2023-11-08T19:29:50.059 INFO:tasks.ceph.mds.a.smithi143.stderr: 4: (MDSContext::complete(int)+0x5f) [0x55ed92b2fecf] 2023-11-08T19:29:50.060 INFO:tasks.ceph.mds.a.smithi143.stderr: 5: (MDSIOContextBase::complete(int)+0x574) [0x55ed92b306b4] 2023-11-08T19:29:50.060 INFO:tasks.ceph.mds.a.smithi143.stderr: 6: (MDSLogContextBase::complete(int)+0x5b) [0x55ed92b30a7b] 2023-11-08T19:29:50.060 INFO:tasks.ceph.mds.a.smithi143.stderr: 7: (Finisher::finisher_thread_entry()+0x19d) [0x7f98cab428cd] 2023-11-08T19:29:50.060 INFO:tasks.ceph.mds.a.smithi143.stderr: 8: /lib64/libpthread.so.0(+0x81cf) [0x7f98c983d1cf] 2023-11-08T19:29:50.060 INFO:tasks.ceph.mds.a.smithi143.stderr: 9: clone() 2023-11-08T19:29:50.061 INFO:tasks.ceph.mds.a.smithi143.stderr: 2023-11-08T19:29:50.061 INFO:tasks.ceph.mds.a.smithi143.stderr:*** Caught signal (Aborted) ** 2023-11-08T19:29:50.061 INFO:tasks.ceph.mds.a.smithi143.stderr: in thread 7f98b91d0700 thread_name:MR_Finisher 2023-11-08T19:29:50.061 INFO:tasks.ceph.mds.a.smithi143.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-08T19:29:50.061 INFO:tasks.ceph.mds.a.smithi143.stderr: 1: /lib64/libpthread.so.0(+0x12ce0) [0x7f98c9847ce0] 2023-11-08T19:29:50.061 INFO:tasks.ceph.mds.a.smithi143.stderr: 2: gsignal() 2023-11-08T19:29:50.062 INFO:tasks.ceph.mds.a.smithi143.stderr: 3: abort() 2023-11-08T19:29:50.062 INFO:tasks.ceph.mds.a.smithi143.stderr: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f98caaa4fb5] 2023-11-08T19:29:50.062 INFO:tasks.ceph.mds.a.smithi143.stderr: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2ba121) [0x7f98caaa5121] 2023-11-08T19:29:50.062 INFO:tasks.ceph.mds.a.smithi143.stderr: 6: (MDSTableServer::_commit_logged(boost::intrusive_ptr const&)+0x2ab) [0x55ed92ae2f2b] 2023-11-08T19:29:50.063 INFO:tasks.ceph.mds.a.smithi143.stderr: 7: (MDSContext::complete(int)+0x5f) [0x55ed92b2fecf] 2023-11-08T19:29:50.063 INFO:tasks.ceph.mds.a.smithi143.stderr: 8: (MDSIOContextBase::complete(int)+0x574) [0x55ed92b306b4] 2023-11-08T19:29:50.063 INFO:tasks.ceph.mds.a.smithi143.stderr: 9: (MDSLogContextBase::complete(int)+0x5b) [0x55ed92b30a7b] 2023-11-08T19:29:50.063 INFO:tasks.ceph.mds.a.smithi143.stderr: 10: (Finisher::finisher_thread_entry()+0x19d) [0x7f98cab428cd] 2023-11-08T19:29:50.063 INFO:tasks.ceph.mds.a.smithi143.stderr: 11: /lib64/libpthread.so.0(+0x81cf) [0x7f98c983d1cf] 2023-11-08T19:29:50.064 INFO:tasks.ceph.mds.a.smithi143.stderr: 12: clone() 2023-11-08T19:29:50.064 INFO:tasks.ceph.mds.a.smithi143.stderr:2023-11-08T19:29:50.052+0000 7f98b91d0700 -1 *** Caught signal (Aborted) ** 2023-11-08T19:29:50.064 INFO:tasks.ceph.mds.a.smithi143.stderr: in thread 7f98b91d0700 thread_name:MR_Finisher 2023-11-08T19:29:50.064 INFO:tasks.ceph.mds.a.smithi143.stderr: 2023-11-08T19:29:50.065 INFO:tasks.ceph.mds.a.smithi143.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-08T19:29:50.065 INFO:tasks.ceph.mds.a.smithi143.stderr: 1: /lib64/libpthread.so.0(+0x12ce0) [0x7f98c9847ce0] 2023-11-08T19:29:50.065 INFO:tasks.ceph.mds.a.smithi143.stderr: 2: gsignal() 2023-11-08T19:29:50.065 INFO:tasks.ceph.mds.a.smithi143.stderr: 3: abort() 2023-11-08T19:29:50.065 INFO:tasks.ceph.mds.a.smithi143.stderr: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f98caaa4fb5] 2023-11-08T19:29:50.066 INFO:tasks.ceph.mds.a.smithi143.stderr: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2ba121) [0x7f98caaa5121] 2023-11-08T19:29:50.066 INFO:tasks.ceph.mds.a.smithi143.stderr: 6: (MDSTableServer::_commit_logged(boost::intrusive_ptr const&)+0x2ab) [0x55ed92ae2f2b] 2023-11-08T19:29:50.066 INFO:tasks.ceph.mds.a.smithi143.stderr: 7: (MDSContext::complete(int)+0x5f) [0x55ed92b2fecf] 2023-11-08T19:29:50.066 INFO:tasks.ceph.mds.a.smithi143.stderr: 8: (MDSIOContextBase::complete(int)+0x574) [0x55ed92b306b4] 2023-11-08T19:29:50.066 INFO:tasks.ceph.mds.a.smithi143.stderr: 9: (MDSLogContextBase::complete(int)+0x5b) [0x55ed92b30a7b] 2023-11-08T19:29:50.067 INFO:tasks.ceph.mds.a.smithi143.stderr: 10: (Finisher::finisher_thread_entry()+0x19d) [0x7f98cab428cd] 2023-11-08T19:29:50.067 INFO:tasks.ceph.mds.a.smithi143.stderr: 11: /lib64/libpthread.so.0(+0x81cf) [0x7f98c983d1cf] 2023-11-08T19:29:50.067 INFO:tasks.ceph.mds.a.smithi143.stderr: 12: clone() 2023-11-08T19:29:50.067 INFO:tasks.ceph.mds.a.smithi143.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-11-08T19:29:50.067 INFO:tasks.ceph.mds.a.smithi143.stderr: 2023-11-08T19:29:50.069 INFO:tasks.ceph.mds.a.smithi143.stderr: -421> 2023-11-08T19:29:45.957+0000 7f98bc9d7700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-08T19:29:50.069 INFO:tasks.ceph.mds.a.smithi143.stderr: -419> 2023-11-08T19:29:45.957+0000 7f98bc9d7700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-11-08T19:29:50.069 INFO:tasks.ceph.mds.a.smithi143.stderr: -1> 2023-11-08T19:29:50.051+0000 7f98b91d0700 -1 /home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el8/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: In function 'void MDSTableServer::_commit_logged(ceph::cref_t&)' thread 7f98b91d0700 time 2023-11-08T19:29:50.051347+0000 2023-11-08T19:29:50.069 INFO:tasks.ceph.mds.a.smithi143.stderr:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el8/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: 171: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 6) 2023-11-08T19:29:50.070 INFO:tasks.ceph.mds.a.smithi143.stderr: 2023-11-08T19:29:50.070 INFO:tasks.ceph.mds.a.smithi143.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-08T19:29:50.070 INFO:tasks.ceph.mds.a.smithi143.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f98caaa4f5b] 2023-11-08T19:29:50.071 INFO:tasks.ceph.mds.a.smithi143.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2ba121) [0x7f98caaa5121] 2023-11-08T19:29:50.071 INFO:tasks.ceph.mds.a.smithi143.stderr: 3: (MDSTableServer::_commit_logged(boost::intrusive_ptr const&)+0x2ab) [0x55ed92ae2f2b] 2023-11-08T19:29:50.071 INFO:tasks.ceph.mds.a.smithi143.stderr: 4: (MDSContext::complete(int)+0x5f) [0x55ed92b2fecf] 2023-11-08T19:29:50.071 INFO:tasks.ceph.mds.a.smithi143.stderr: 5: (MDSIOContextBase::complete(int)+0x574) [0x55ed92b306b4] 2023-11-08T19:29:50.071 INFO:tasks.ceph.mds.a.smithi143.stderr: 6: (MDSLogContextBase::complete(int)+0x5b) [0x55ed92b30a7b] 2023-11-08T19:29:50.072 INFO:tasks.ceph.mds.a.smithi143.stderr: 7: (Finisher::finisher_thread_entry()+0x19d) [0x7f98cab428cd] 2023-11-08T19:29:50.072 INFO:tasks.ceph.mds.a.smithi143.stderr: 8: /lib64/libpthread.so.0(+0x81cf) [0x7f98c983d1cf] 2023-11-08T19:29:50.072 INFO:tasks.ceph.mds.a.smithi143.stderr: 9: clone() 2023-11-08T19:29:50.072 INFO:tasks.ceph.mds.a.smithi143.stderr: 2023-11-08T19:29:50.072 INFO:tasks.ceph.mds.a.smithi143.stderr: 0> 2023-11-08T19:29:50.052+0000 7f98b91d0700 -1 *** Caught signal (Aborted) ** 2023-11-08T19:29:50.073 INFO:tasks.ceph.mds.a.smithi143.stderr: in thread 7f98b91d0700 thread_name:MR_Finisher 2023-11-08T19:29:50.073 INFO:tasks.ceph.mds.a.smithi143.stderr: 2023-11-08T19:29:50.073 INFO:tasks.ceph.mds.a.smithi143.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-08T19:29:50.073 INFO:tasks.ceph.mds.a.smithi143.stderr: 1: /lib64/libpthread.so.0(+0x12ce0) [0x7f98c9847ce0] 2023-11-08T19:29:50.073 INFO:tasks.ceph.mds.a.smithi143.stderr: 2: gsignal() 2023-11-08T19:29:50.073 INFO:tasks.ceph.mds.a.smithi143.stderr: 3: abort() 2023-11-08T19:29:50.074 INFO:tasks.ceph.mds.a.smithi143.stderr: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f98caaa4fb5] 2023-11-08T19:29:50.074 INFO:tasks.ceph.mds.a.smithi143.stderr: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2ba121) [0x7f98caaa5121] 2023-11-08T19:29:50.074 INFO:tasks.ceph.mds.a.smithi143.stderr: 6: (MDSTableServer::_commit_logged(boost::intrusive_ptr const&)+0x2ab) [0x55ed92ae2f2b] 2023-11-08T19:29:50.074 INFO:tasks.ceph.mds.a.smithi143.stderr: 7: (MDSContext::complete(int)+0x5f) [0x55ed92b2fecf] 2023-11-08T19:29:50.075 INFO:tasks.ceph.mds.a.smithi143.stderr: 8: (MDSIOContextBase::complete(int)+0x574) [0x55ed92b306b4] 2023-11-08T19:29:50.075 INFO:tasks.ceph.mds.a.smithi143.stderr: 9: (MDSLogContextBase::complete(int)+0x5b) [0x55ed92b30a7b] 2023-11-08T19:29:50.075 INFO:tasks.ceph.mds.a.smithi143.stderr: 10: (Finisher::finisher_thread_entry()+0x19d) [0x7f98cab428cd] 2023-11-08T19:29:50.075 INFO:tasks.ceph.mds.a.smithi143.stderr: 11: /lib64/libpthread.so.0(+0x81cf) [0x7f98c983d1cf] 2023-11-08T19:29:50.076 INFO:tasks.ceph.mds.a.smithi143.stderr: 12: clone() 2023-11-08T19:29:50.076 INFO:tasks.ceph.mds.a.smithi143.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-11-08T19:29:50.076 INFO:tasks.ceph.mds.a.smithi143.stderr: 2023-11-08T19:29:50.076 INFO:tasks.ceph.mds.a.smithi143.stderr: -421> 2023-11-08T19:29:45.957+0000 7f98bc9d7700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-08T19:29:50.076 INFO:tasks.ceph.mds.a.smithi143.stderr: -419> 2023-11-08T19:29:45.957+0000 7f98bc9d7700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-11-08T19:29:50.076 INFO:tasks.ceph.mds.a.smithi143.stderr: -1> 2023-11-08T19:29:50.051+0000 7f98b91d0700 -1 /home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el8/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: In function 'void MDSTableServer::_commit_logged(ceph::cref_t&)' thread 7f98b91d0700 time 2023-11-08T19:29:50.051347+0000 2023-11-08T19:29:50.077 INFO:tasks.ceph.mds.a.smithi143.stderr:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el8/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: 171: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 6) 2023-11-08T19:29:50.077 INFO:tasks.ceph.mds.a.smithi143.stderr: 2023-11-08T19:29:50.077 INFO:tasks.ceph.mds.a.smithi143.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-08T19:29:50.077 INFO:tasks.ceph.mds.a.smithi143.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f98caaa4f5b] 2023-11-08T19:29:50.077 INFO:tasks.ceph.mds.a.smithi143.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2ba121) [0x7f98caaa5121] 2023-11-08T19:29:50.078 INFO:tasks.ceph.mds.a.smithi143.stderr: 3: (MDSTableServer::_commit_logged(boost::intrusive_ptr const&)+0x2ab) [0x55ed92ae2f2b] 2023-11-08T19:29:50.078 INFO:tasks.ceph.mds.a.smithi143.stderr: 4: (MDSContext::complete(int)+0x5f) [0x55ed92b2fecf] 2023-11-08T19:29:50.078 INFO:tasks.ceph.mds.a.smithi143.stderr: 5: (MDSIOContextBase::complete(int)+0x574) [0x55ed92b306b4] 2023-11-08T19:29:50.078 INFO:tasks.ceph.mds.a.smithi143.stderr: 6: (MDSLogContextBase::complete(int)+0x5b) [0x55ed92b30a7b] 2023-11-08T19:29:50.079 INFO:tasks.ceph.mds.a.smithi143.stderr: 7: (Finisher::finisher_thread_entry()+0x19d) [0x7f98cab428cd] 2023-11-08T19:29:50.079 INFO:tasks.ceph.mds.a.smithi143.stderr: 8: /lib64/libpthread.so.0(+0x81cf) [0x7f98c983d1cf] 2023-11-08T19:29:50.079 INFO:tasks.ceph.mds.a.smithi143.stderr: 9: clone() 2023-11-08T19:29:50.079 INFO:tasks.ceph.mds.a.smithi143.stderr: 2023-11-08T19:29:50.079 INFO:tasks.ceph.mds.a.smithi143.stderr: 0> 2023-11-08T19:29:50.052+0000 7f98b91d0700 -1 *** Caught signal (Aborted) ** 2023-11-08T19:29:50.080 INFO:tasks.ceph.mds.a.smithi143.stderr: in thread 7f98b91d0700 thread_name:MR_Finisher 2023-11-08T19:29:50.080 INFO:tasks.ceph.mds.a.smithi143.stderr: 2023-11-08T19:29:50.080 INFO:tasks.ceph.mds.a.smithi143.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-08T19:29:50.080 INFO:tasks.ceph.mds.a.smithi143.stderr: 1: /lib64/libpthread.so.0(+0x12ce0) [0x7f98c9847ce0] 2023-11-08T19:29:50.081 INFO:tasks.ceph.mds.a.smithi143.stderr: 2: gsignal() 2023-11-08T19:29:50.081 INFO:tasks.ceph.mds.a.smithi143.stderr: 3: abort() 2023-11-08T19:29:50.081 INFO:tasks.ceph.mds.a.smithi143.stderr: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f98caaa4fb5] 2023-11-08T19:29:50.082 INFO:tasks.ceph.mds.a.smithi143.stderr: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2ba121) [0x7f98caaa5121] 2023-11-08T19:29:50.082 INFO:tasks.ceph.mds.a.smithi143.stderr: 6: (MDSTableServer::_commit_logged(boost::intrusive_ptr const&)+0x2ab) [0x55ed92ae2f2b] 2023-11-08T19:29:50.082 INFO:tasks.ceph.mds.a.smithi143.stderr: 7: (MDSContext::complete(int)+0x5f) [0x55ed92b2fecf] 2023-11-08T19:29:50.083 INFO:tasks.ceph.mds.a.smithi143.stderr: 8: (MDSIOContextBase::complete(int)+0x574) [0x55ed92b306b4] 2023-11-08T19:29:50.083 INFO:tasks.ceph.mds.a.smithi143.stderr: 9: (MDSLogContextBase::complete(int)+0x5b) [0x55ed92b30a7b] 2023-11-08T19:29:50.083 INFO:tasks.ceph.mds.a.smithi143.stderr: 10: (Finisher::finisher_thread_entry()+0x19d) [0x7f98cab428cd] 2023-11-08T19:29:50.083 INFO:tasks.ceph.mds.a.smithi143.stderr: 11: /lib64/libpthread.so.0(+0x81cf) [0x7f98c983d1cf] 2023-11-08T19:29:50.083 INFO:tasks.ceph.mds.a.smithi143.stderr: 12: clone() 2023-11-08T19:29:50.084 INFO:tasks.ceph.mds.a.smithi143.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-11-08T19:29:50.084 INFO:tasks.ceph.mds.a.smithi143.stderr: 2023-11-08T19:29:50.115 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:50.114+0000 7f0805960700 1 Processor -- start 2023-11-08T19:29:50.116 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:50.114+0000 7f0805960700 1 -- start start 2023-11-08T19:29:50.118 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:50.114+0000 7f0805960700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f080015ba30 0x7f08000b14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:50.119 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:50.114+0000 7f0805960700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f08000d03a0 0x7f08000d1560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:50.119 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:50.114+0000 7f0805960700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f08000d1aa0 0x7f08000ce2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:50.119 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:50.114+0000 7f0805960700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08000d1f10 con 0x7f080015ba30 2023-11-08T19:29:50.119 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:50.114+0000 7f0805960700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08000b1bc0 con 0x7f08000d1aa0 2023-11-08T19:29:50.120 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:50.115+0000 7f0805960700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08000b1ea0 con 0x7f08000d03a0 2023-11-08T19:29:50.120 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:50.115+0000 7f07feffd700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f080015ba30 0x7f08000b14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:50.120 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:50.115+0000 7f07feffd700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f080015ba30 0x7f08000b14c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55942/0 (socket says 172.21.15.143:55942) 2023-11-08T19:29:50.120 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:50.115+0000 7f07feffd700 1 -- 172.21.15.143:0/1263365278 learned_addr learned my addr 172.21.15.143:0/1263365278 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:29:50.120 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:50.115+0000 7f07fe7fc700 1 --2- 172.21.15.143:0/1263365278 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f08000d03a0 0x7f08000d1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:50.121 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:50.115+0000 7f07ff7fe700 1 --2- 172.21.15.143:0/1263365278 >> v2:172.21.15.174:3300/0 conn(0x7f08000d1aa0 0x7f08000ce2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:50.121 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:50.115+0000 7f07feffd700 1 -- 172.21.15.143:0/1263365278 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f08000d03a0 msgr2=0x7f08000d1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:50.121 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:50.115+0000 7f07feffd700 1 --2- 172.21.15.143:0/1263365278 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f08000d03a0 0x7f08000d1560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:50.121 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:50.115+0000 7f07feffd700 1 -- 172.21.15.143:0/1263365278 >> v2:172.21.15.174:3300/0 conn(0x7f08000d1aa0 msgr2=0x7f08000ce2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:50.121 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:50.115+0000 7f07feffd700 1 --2- 172.21.15.143:0/1263365278 >> v2:172.21.15.174:3300/0 conn(0x7f08000d1aa0 0x7f08000ce2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:50.122 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:50.115+0000 7f07feffd700 1 -- 172.21.15.143:0/1263365278 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08000b2180 con 0x7f080015ba30 2023-11-08T19:29:50.122 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:50.115+0000 7f07feffd700 1 --2- 172.21.15.143:0/1263365278 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f080015ba30 0x7f08000b14c0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f08000b1060 tx=0x7f07f0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1f1832065ed0125 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:50.123 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:50.115+0000 7f07fdffb700 1 -- 172.21.15.143:0/1263365278 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07f0011830 con 0x7f080015ba30 2023-11-08T19:29:50.123 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:50.115+0000 7f07fdffb700 1 -- 172.21.15.143:0/1263365278 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07f0011990 con 0x7f080015ba30 2023-11-08T19:29:50.123 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:50.115+0000 7f07fdffb700 1 -- 172.21.15.143:0/1263365278 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07f0011d20 con 0x7f080015ba30 2023-11-08T19:29:50.123 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:50.116+0000 7f0805960700 1 -- 172.21.15.143:0/1263365278 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f080015ba30 msgr2=0x7f08000b14c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:50.124 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:50.116+0000 7f0805960700 1 --2- 172.21.15.143:0/1263365278 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f080015ba30 0x7f08000b14c0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f08000b1060 tx=0x7f07f0009bd0 comp rx=0 tx=0).stop 2023-11-08T19:29:50.124 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:50.116+0000 7f0805960700 1 -- 172.21.15.143:0/1263365278 shutdown_connections 2023-11-08T19:29:50.124 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:50.116+0000 7f0805960700 1 --2- 172.21.15.143:0/1263365278 >> v2:172.21.15.174:3300/0 conn(0x7f08000d1aa0 0x7f08000ce2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:50.124 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:50.116+0000 7f0805960700 1 --2- 172.21.15.143:0/1263365278 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f080015ba30 0x7f08000b14c0 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:50.124 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:50.116+0000 7f0805960700 1 --2- 172.21.15.143:0/1263365278 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f08000d03a0 0x7f08000d1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:50.125 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:50.116+0000 7f0805960700 1 -- 172.21.15.143:0/1263365278 >> 172.21.15.143:0/1263365278 conn(0x7f080008b000 msgr2=0x7f080009ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:50.125 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:50.116+0000 7f0805960700 1 -- 172.21.15.143:0/1263365278 shutdown_connections 2023-11-08T19:29:50.125 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:50.116+0000 7f0805960700 1 -- 172.21.15.143:0/1263365278 wait complete. 2023-11-08T19:29:50.125 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:50.117+0000 7f0805960700 1 Processor -- start 2023-11-08T19:29:50.125 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:50.117+0000 7f0805960700 1 -- start start 2023-11-08T19:29:50.128 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:50.118+0000 7f0805960700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f08000d03a0 0x7f08001f35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:50.128 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:50.118+0000 7f0805960700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f08000d1aa0 0x7f08000c6660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:50.128 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:50.118+0000 7f0805960700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f080015ba30 0x7f08000b6950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:50.129 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:50.118+0000 7f0805960700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08001f45f0 con 0x7f08000d1aa0 2023-11-08T19:29:50.129 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:50.118+0000 7f0805960700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08001f3b20 con 0x7f080015ba30 2023-11-08T19:29:50.129 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:50.118+0000 7f0805960700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08000c5600 con 0x7f08000d03a0 2023-11-08T19:29:50.129 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:50.118+0000 7f07fe7fc700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f08000d1aa0 0x7f08000c6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:50.130 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:50.118+0000 7f07fe7fc700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f08000d1aa0 0x7f08000c6660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55948/0 (socket says 172.21.15.143:55948) 2023-11-08T19:29:50.130 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:50.118+0000 7f07fe7fc700 1 -- 172.21.15.143:0/2957416781 learned_addr learned my addr 172.21.15.143:0/2957416781 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:29:50.130 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:50.118+0000 7f07feffd700 1 --2- 172.21.15.143:0/2957416781 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f08000d03a0 0x7f08001f35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:50.131 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:50.118+0000 7f07ff7fe700 1 --2- 172.21.15.143:0/2957416781 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f080015ba30 0x7f08000b6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:50.131 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:50.119+0000 7f07fe7fc700 1 -- 172.21.15.143:0/2957416781 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f08000d03a0 msgr2=0x7f08001f35e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:50.131 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:50.119+0000 7f07fe7fc700 1 --2- 172.21.15.143:0/2957416781 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f08000d03a0 0x7f08001f35e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:50.131 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:50.119+0000 7f07fe7fc700 1 -- 172.21.15.143:0/2957416781 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f080015ba30 msgr2=0x7f08000b6950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:50.131 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:50.119+0000 7f07fe7fc700 1 --2- 172.21.15.143:0/2957416781 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f080015ba30 0x7f08000b6950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:50.132 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:50.119+0000 7f07fe7fc700 1 -- 172.21.15.143:0/2957416781 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07f0009550 con 0x7f08000d1aa0 2023-11-08T19:29:50.132 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:50.119+0000 7f07feffd700 1 --2- 172.21.15.143:0/2957416781 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f08000d03a0 0x7f08001f35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:29:50.132 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:50.119+0000 7f07ff7fe700 1 --2- 172.21.15.143:0/2957416781 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f080015ba30 0x7f08000b6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:29:50.133 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:50.119+0000 7f07fe7fc700 1 --2- 172.21.15.143:0/2957416781 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f08000d1aa0 0x7f08000c6660 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f07f400e930 tx=0x7f07f400ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:50.133 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:50.119+0000 7f07e7fff700 1 -- 172.21.15.143:0/2957416781 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07f4009690 con 0x7f08000d1aa0 2023-11-08T19:29:50.133 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:50.119+0000 7f07e7fff700 1 -- 172.21.15.143:0/2957416781 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07f400c850 con 0x7f08000d1aa0 2023-11-08T19:29:50.133 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:50.119+0000 7f07e7fff700 1 -- 172.21.15.143:0/2957416781 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07f400cbe0 con 0x7f08000d1aa0 2023-11-08T19:29:50.133 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:50.119+0000 7f0805960700 1 -- 172.21.15.143:0/2957416781 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08001f3d50 con 0x7f08000d1aa0 2023-11-08T19:29:50.134 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:50.119+0000 7f0805960700 1 -- 172.21.15.143:0/2957416781 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08001f4310 con 0x7f08000d1aa0 2023-11-08T19:29:50.134 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:50.120+0000 7f0805960700 1 -- 172.21.15.143:0/2957416781 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f080004eee0 con 0x7f08000d1aa0 2023-11-08T19:29:50.134 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:50.120+0000 7f07e7fff700 1 -- 172.21.15.143:0/2957416781 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f07f400cd40 con 0x7f08000d1aa0 2023-11-08T19:29:50.134 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:50.121+0000 7f07e7fff700 1 --2- 172.21.15.143:0/2957416781 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f07ec06af00 0x7f07ec06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:50.135 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:50.121+0000 7f07e7fff700 1 -- 172.21.15.143:0/2957416781 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 7059+0+0 (secure 0 0 0) 0x7f07f4089bd0 con 0x7f08000d1aa0 2023-11-08T19:29:50.135 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:50.121+0000 7f07feffd700 1 --2- 172.21.15.143:0/2957416781 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f07ec06af00 0x7f07ec06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:50.135 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:50.121+0000 7f07feffd700 1 --2- 172.21.15.143:0/2957416781 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f07ec06af00 0x7f07ec06d3c0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f07f0014040 tx=0x7f07f0008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:50.135 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:50.125+0000 7f07e7fff700 1 -- 172.21.15.143:0/2957416781 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f07f40547c0 con 0x7f08000d1aa0 2023-11-08T19:29:50.273 INFO:tasks.ceph.mds.a.smithi143.stderr:daemon-helper: command crashed with signal 6 2023-11-08T19:29:50.361 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:50.359+0000 7f0805960700 1 -- 172.21.15.143:0/2957416781 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f08000c5190 con 0x7f08000d1aa0 2023-11-08T19:29:50.361 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:50.360+0000 7f07e7fff700 1 -- 172.21.15.143:0/2957416781 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f07f4057de0 con 0x7f08000d1aa0 2023-11-08T19:29:50.362 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:29:50.362 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":81,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5933,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/3918286710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6835","nonce":3918286710},{"type":"v1","addr":"172.21.15.174:6837","nonce":3918286710}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":7494,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6834/1535103237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":1535103237},{"type":"v1","addr":"172.21.15.143:6834","nonce":1535103237}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:03.092245+0000","modified":"2023-11-08T19:29:49.416779+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":97,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7386,"mds_1":6918},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6918":{"gid":6918,"name":"b","rank":1,"incarnation":54,"state":"up:active","state_seq":6,"addr":"172.21.15.174:6836/3601088618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3601088618},{"type":"v1","addr":"172.21.15.174:6836","nonce":3601088618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7386":{"gid":7386,"name":"a","rank":0,"incarnation":76,"state":"up:active","state_seq":11,"addr":"172.21.15.143:6835/3664584522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":3664584522},{"type":"v1","addr":"172.21.15.143:6835","nonce":3664584522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:29:50.364 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:50.362+0000 7f0805960700 1 -- 172.21.15.143:0/2957416781 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f07ec06af00 msgr2=0x7f07ec06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:50.364 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:50.362+0000 7f0805960700 1 --2- 172.21.15.143:0/2957416781 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f07ec06af00 0x7f07ec06d3c0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f07f0014040 tx=0x7f07f0008030 comp rx=0 tx=0).stop 2023-11-08T19:29:50.364 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:50.362+0000 7f0805960700 1 -- 172.21.15.143:0/2957416781 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f08000d1aa0 msgr2=0x7f08000c6660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:50.365 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:50.362+0000 7f0805960700 1 --2- 172.21.15.143:0/2957416781 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f08000d1aa0 0x7f08000c6660 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f07f400e930 tx=0x7f07f400ecf0 comp rx=0 tx=0).stop 2023-11-08T19:29:50.365 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:50.362+0000 7f0805960700 1 -- 172.21.15.143:0/2957416781 shutdown_connections 2023-11-08T19:29:50.365 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:50.362+0000 7f0805960700 1 --2- 172.21.15.143:0/2957416781 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f07ec06af00 0x7f07ec06d3c0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:50.365 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:50.362+0000 7f0805960700 1 --2- 172.21.15.143:0/2957416781 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f08000d03a0 0x7f08001f35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:50.365 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:50.362+0000 7f0805960700 1 --2- 172.21.15.143:0/2957416781 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f080015ba30 0x7f08000b6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:50.365 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:50.362+0000 7f0805960700 1 --2- 172.21.15.143:0/2957416781 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f08000d1aa0 0x7f08000c6660 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:50.366 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:50.362+0000 7f0805960700 1 -- 172.21.15.143:0/2957416781 >> 172.21.15.143:0/2957416781 conn(0x7f080008b000 msgr2=0x7f08000cea40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:50.366 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:50.362+0000 7f0805960700 1 -- 172.21.15.143:0/2957416781 shutdown_connections 2023-11-08T19:29:50.366 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:50.362+0000 7f0805960700 1 -- 172.21.15.143:0/2957416781 wait complete. 2023-11-08T19:29:50.367 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 81 2023-11-08T19:29:50.376 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2023-11-08T19:29:55.377 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:29:55.509 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:55.504+0000 7f5bc310d700 1 Processor -- start 2023-11-08T19:29:55.509 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:55.504+0000 7f5bc310d700 1 -- start start 2023-11-08T19:29:55.510 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:55.504+0000 7f5bc310d700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f5bbc15dd00 0x7f5bbc0cca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:55.510 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:55.504+0000 7f5bc310d700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5bbc06b320 0x7f5bbc0cb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:55.510 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:55.504+0000 7f5bc310d700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bbc0ccfb0 con 0x7f5bbc1618f0 2023-11-08T19:29:55.510 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:55.504+0000 7f5bc310d700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bbc0cad50 con 0x7f5bbc15dd00 2023-11-08T19:29:55.511 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:55.504+0000 7f5bc310d700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bbc161df0 con 0x7f5bbc06b320 2023-11-08T19:29:55.511 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:55.505+0000 7f5bbbfff700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f5bbc1618f0 0x7f5bbc06ac10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:52120/0 (socket says 172.21.15.143:52120) 2023-11-08T19:29:55.511 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:55.505+0000 7f5bbbfff700 1 -- 172.21.15.143:0/3042242224 learned_addr learned my addr 172.21.15.143:0/3042242224 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:29:55.511 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:55.505+0000 7f5bbb7fe700 1 -- 172.21.15.143:0/3042242224 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1224210637 0 0) 0x7f5bbc0ccfb0 con 0x7f5bbc1618f0 2023-11-08T19:29:55.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:55.505+0000 7f5bbb7fe700 1 -- 172.21.15.143:0/3042242224 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5bac0035b0 con 0x7f5bbc1618f0 2023-11-08T19:29:55.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:55.505+0000 7f5bc16aa700 1 --2- 172.21.15.143:0/3042242224 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5bbc06b320 0x7f5bbc0cb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:55.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:55.505+0000 7f5bc0ea9700 1 --2- 172.21.15.143:0/3042242224 >> v2:172.21.15.174:3300/0 conn(0x7f5bbc15dd00 0x7f5bbc0cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:55.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:55.505+0000 7f5bbb7fe700 1 -- 172.21.15.143:0/3042242224 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 536097598 0 0) 0x7f5bac0035b0 con 0x7f5bbc1618f0 2023-11-08T19:29:55.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:55.505+0000 7f5bbb7fe700 1 -- 172.21.15.143:0/3042242224 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5bbc06b320 msgr2=0x7f5bbc0cb1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:55.513 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:55.505+0000 7f5bbb7fe700 1 --2- 172.21.15.143:0/3042242224 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5bbc06b320 0x7f5bbc0cb1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:55.513 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:55.505+0000 7f5bbb7fe700 1 -- 172.21.15.143:0/3042242224 >> v2:172.21.15.174:3300/0 conn(0x7f5bbc15dd00 msgr2=0x7f5bbc0cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:55.513 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:55.505+0000 7f5bbb7fe700 1 --2- 172.21.15.143:0/3042242224 >> v2:172.21.15.174:3300/0 conn(0x7f5bbc15dd00 0x7f5bbc0cca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:55.513 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:55.505+0000 7f5bbb7fe700 1 -- 172.21.15.143:0/3042242224 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bbc1620d0 con 0x7f5bbc1618f0 2023-11-08T19:29:55.513 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:55.505+0000 7f5bbb7fe700 1 -- 172.21.15.143:0/3042242224 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4147756303 0 0) 0x7f5ba8001670 con 0x7f5bbc1618f0 2023-11-08T19:29:55.514 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:55.505+0000 7f5bbb7fe700 1 -- 172.21.15.143:0/3042242224 >> v1:172.21.15.143:6789/0 conn(0x7f5bbc1618f0 legacy=0x7f5bbc06ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:55.514 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:55.505+0000 7f5bbb7fe700 1 --2- 172.21.15.143:0/3042242224 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5bac003a00 0x7f5bac003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:55.514 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:55.505+0000 7f5bbb7fe700 1 --2- 172.21.15.143:0/3042242224 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5bac004850 0x7f5bac004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:55.514 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:55.505+0000 7f5bbb7fe700 1 --2- 172.21.15.143:0/3042242224 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5bbc06b320 0x7f5bac005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:55.514 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:55.505+0000 7f5bbb7fe700 1 -- 172.21.15.143:0/3042242224 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bbc161df0 con 0x7f5bac003a00 2023-11-08T19:29:55.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:55.505+0000 7f5bbb7fe700 1 -- 172.21.15.143:0/3042242224 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bac00ba10 con 0x7f5bac004850 2023-11-08T19:29:55.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:55.505+0000 7f5bbb7fe700 1 -- 172.21.15.143:0/3042242224 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bac00bb50 con 0x7f5bbc06b320 2023-11-08T19:29:55.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:55.505+0000 7f5bc0ea9700 1 --2- 172.21.15.143:0/3042242224 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5bac003a00 0x7f5bac003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:55.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:55.505+0000 7f5bbbfff700 1 --2- 172.21.15.143:0/3042242224 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5bac004850 0x7f5bac004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:55.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:55.505+0000 7f5bc16aa700 1 --2- 172.21.15.143:0/3042242224 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5bbc06b320 0x7f5bac005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:55.516 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:55.506+0000 7f5bc0ea9700 1 -- 172.21.15.143:0/3042242224 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5bbc06b320 msgr2=0x7f5bac005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:55.516 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:55.506+0000 7f5bc0ea9700 1 --2- 172.21.15.143:0/3042242224 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5bbc06b320 0x7f5bac005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:55.516 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:55.506+0000 7f5bc0ea9700 1 -- 172.21.15.143:0/3042242224 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5bac004850 msgr2=0x7f5bac004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:55.516 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:55.506+0000 7f5bc0ea9700 1 --2- 172.21.15.143:0/3042242224 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5bac004850 0x7f5bac004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:55.516 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:55.506+0000 7f5bc0ea9700 1 -- 172.21.15.143:0/3042242224 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bac0035b0 con 0x7f5bac003a00 2023-11-08T19:29:55.517 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:55.506+0000 7f5bbbfff700 1 --2- 172.21.15.143:0/3042242224 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5bac004850 0x7f5bac004cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:29:55.517 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:55.506+0000 7f5bc0ea9700 1 --2- 172.21.15.143:0/3042242224 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5bac003a00 0x7f5bac003e20 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f5bb0009b40 tx=0x7f5bac005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac4d9d760605a40a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:55.517 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:55.506+0000 7f5bbb7fe700 1 -- 172.21.15.143:0/3042242224 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb0005a20 con 0x7f5bac003a00 2023-11-08T19:29:55.517 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:55.506+0000 7f5bbb7fe700 1 -- 172.21.15.143:0/3042242224 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bb0005b80 con 0x7f5bac003a00 2023-11-08T19:29:55.517 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:55.506+0000 7f5bbb7fe700 1 -- 172.21.15.143:0/3042242224 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb000e6c0 con 0x7f5bac003a00 2023-11-08T19:29:55.518 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:55.506+0000 7f5bc310d700 1 -- 172.21.15.143:0/3042242224 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5bac003a00 msgr2=0x7f5bac003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:55.518 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:55.506+0000 7f5bc310d700 1 --2- 172.21.15.143:0/3042242224 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5bac003a00 0x7f5bac003e20 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f5bb0009b40 tx=0x7f5bac005880 comp rx=0 tx=0).stop 2023-11-08T19:29:55.518 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:55.506+0000 7f5bc16aa700 1 -- 172.21.15.143:0/3042242224 reap_dead start 2023-11-08T19:29:55.518 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:55.506+0000 7f5bc310d700 1 -- 172.21.15.143:0/3042242224 shutdown_connections 2023-11-08T19:29:55.518 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:55.506+0000 7f5bc310d700 1 -- 172.21.15.143:0/3042242224 >> 172.21.15.143:0/3042242224 conn(0x7f5bbc099ab0 msgr2=0x7f5bbc0d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:55.519 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:55.506+0000 7f5bc310d700 1 -- 172.21.15.143:0/3042242224 shutdown_connections 2023-11-08T19:29:55.519 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:55.507+0000 7f5bc310d700 1 -- 172.21.15.143:0/3042242224 wait complete. 2023-11-08T19:29:55.519 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:55.508+0000 7f5bc310d700 1 Processor -- start 2023-11-08T19:29:55.519 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:55.508+0000 7f5bc310d700 1 -- start start 2023-11-08T19:29:55.522 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:55.509+0000 7f5bc310d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5bac003a00 0x7f5bbc0b33b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:55.522 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:55.509+0000 7f5bc310d700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5bbc15acb0 0x7f5bbc0ae230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:55.523 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:55.509+0000 7f5bc310d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5bbc0ae770 0x7f5bbc0c4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:55.523 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:55.509+0000 7f5bc310d700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bbc0aec20 con 0x7f5bac003a00 2023-11-08T19:29:55.523 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:55.509+0000 7f5bc310d700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bbc0b38f0 con 0x7f5bbc0ae770 2023-11-08T19:29:55.523 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:55.509+0000 7f5bc310d700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bbc0c89f0 con 0x7f5bbc15acb0 2023-11-08T19:29:55.523 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:55.509+0000 7f5bc0ea9700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5bac003a00 0x7f5bbc0b33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:55.524 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:55.509+0000 7f5bc0ea9700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5bac003a00 0x7f5bbc0b33b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55976/0 (socket says 172.21.15.143:55976) 2023-11-08T19:29:55.524 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:55.509+0000 7f5bc0ea9700 1 -- 172.21.15.143:0/4202616479 learned_addr learned my addr 172.21.15.143:0/4202616479 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:29:55.524 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:55.509+0000 7f5bbbfff700 1 --2- 172.21.15.143:0/4202616479 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5bbc15acb0 0x7f5bbc0ae230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:55.524 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:55.509+0000 7f5bc16aa700 1 --2- 172.21.15.143:0/4202616479 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5bbc0ae770 0x7f5bbc0c4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:55.525 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:55.509+0000 7f5bc0ea9700 1 -- 172.21.15.143:0/4202616479 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5bbc15acb0 msgr2=0x7f5bbc0ae230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:55.525 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:55.509+0000 7f5bc0ea9700 1 --2- 172.21.15.143:0/4202616479 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5bbc15acb0 0x7f5bbc0ae230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:55.525 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:55.509+0000 7f5bc0ea9700 1 -- 172.21.15.143:0/4202616479 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5bbc0ae770 msgr2=0x7f5bbc0c4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:55.525 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:55.509+0000 7f5bc0ea9700 1 --2- 172.21.15.143:0/4202616479 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5bbc0ae770 0x7f5bbc0c4d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:55.525 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:55.509+0000 7f5bc0ea9700 1 -- 172.21.15.143:0/4202616479 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bbc0c8c40 con 0x7f5bac003a00 2023-11-08T19:29:55.526 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:55.509+0000 7f5bc16aa700 1 --2- 172.21.15.143:0/4202616479 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5bbc0ae770 0x7f5bbc0c4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:29:55.526 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:55.509+0000 7f5bc0ea9700 1 --2- 172.21.15.143:0/4202616479 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5bac003a00 0x7f5bbc0b33b0 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f5bb0000c00 tx=0x7f5bb00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:55.526 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:55.509+0000 7f5bb97fa700 1 -- 172.21.15.143:0/4202616479 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb0020070 con 0x7f5bac003a00 2023-11-08T19:29:55.526 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:55.509+0000 7f5bb97fa700 1 -- 172.21.15.143:0/4202616479 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bb0005e30 con 0x7f5bac003a00 2023-11-08T19:29:55.526 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:55.509+0000 7f5bb97fa700 1 -- 172.21.15.143:0/4202616479 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb0004460 con 0x7f5bac003a00 2023-11-08T19:29:55.527 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:55.509+0000 7f5bc310d700 1 -- 172.21.15.143:0/4202616479 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bbc0c8f20 con 0x7f5bac003a00 2023-11-08T19:29:55.527 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:55.509+0000 7f5bc310d700 1 -- 172.21.15.143:0/4202616479 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bbc0c94e0 con 0x7f5bac003a00 2023-11-08T19:29:55.527 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:55.510+0000 7f5bc310d700 1 -- 172.21.15.143:0/4202616479 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bbc0c9670 con 0x7f5bac003a00 2023-11-08T19:29:55.527 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:55.511+0000 7f5bb97fa700 1 -- 172.21.15.143:0/4202616479 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5bb00036a0 con 0x7f5bac003a00 2023-11-08T19:29:55.527 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:55.511+0000 7f5bb97fa700 1 --2- 172.21.15.143:0/4202616479 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f5ba006af00 0x7f5ba006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:55.528 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:55.511+0000 7f5bb97fa700 1 -- 172.21.15.143:0/4202616479 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 7059+0+0 (secure 0 0 0) 0x7f5bb008ac20 con 0x7f5bac003a00 2023-11-08T19:29:55.528 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:55.511+0000 7f5bbbfff700 1 --2- 172.21.15.143:0/4202616479 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f5ba006af00 0x7f5ba006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:55.528 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:55.512+0000 7f5bbbfff700 1 --2- 172.21.15.143:0/4202616479 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f5ba006af00 0x7f5ba006d3c0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc0aee10 tx=0x7f5ba8007040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:55.528 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:55.516+0000 7f5bb97fa700 1 -- 172.21.15.143:0/4202616479 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5bb0055810 con 0x7f5bac003a00 2023-11-08T19:29:55.750 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:55.749+0000 7f5bc310d700 1 -- 172.21.15.143:0/4202616479 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5bbc0aeb80 con 0x7f5bac003a00 2023-11-08T19:29:55.751 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:55.749+0000 7f5bb97fa700 1 -- 172.21.15.143:0/4202616479 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f5bb0058e30 con 0x7f5bac003a00 2023-11-08T19:29:55.751 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:29:55.751 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":81,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5933,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/3918286710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6835","nonce":3918286710},{"type":"v1","addr":"172.21.15.174:6837","nonce":3918286710}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":7494,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6834/1535103237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":1535103237},{"type":"v1","addr":"172.21.15.143:6834","nonce":1535103237}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:03.092245+0000","modified":"2023-11-08T19:29:49.416779+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":97,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7386,"mds_1":6918},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6918":{"gid":6918,"name":"b","rank":1,"incarnation":54,"state":"up:active","state_seq":6,"addr":"172.21.15.174:6836/3601088618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3601088618},{"type":"v1","addr":"172.21.15.174:6836","nonce":3601088618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7386":{"gid":7386,"name":"a","rank":0,"incarnation":76,"state":"up:active","state_seq":11,"addr":"172.21.15.143:6835/3664584522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":3664584522},{"type":"v1","addr":"172.21.15.143:6835","nonce":3664584522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:29:55.753 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:55.751+0000 7f5bc310d700 1 -- 172.21.15.143:0/4202616479 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f5ba006af00 msgr2=0x7f5ba006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:55.753 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:55.751+0000 7f5bc310d700 1 --2- 172.21.15.143:0/4202616479 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f5ba006af00 0x7f5ba006d3c0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc0aee10 tx=0x7f5ba8007040 comp rx=0 tx=0).stop 2023-11-08T19:29:55.753 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:55.751+0000 7f5bc310d700 1 -- 172.21.15.143:0/4202616479 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5bac003a00 msgr2=0x7f5bbc0b33b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:55.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:55.751+0000 7f5bc310d700 1 --2- 172.21.15.143:0/4202616479 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5bac003a00 0x7f5bbc0b33b0 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f5bb0000c00 tx=0x7f5bb00047c0 comp rx=0 tx=0).stop 2023-11-08T19:29:55.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:55.751+0000 7f5bc310d700 1 -- 172.21.15.143:0/4202616479 shutdown_connections 2023-11-08T19:29:55.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:55.751+0000 7f5bc310d700 1 --2- 172.21.15.143:0/4202616479 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f5ba006af00 0x7f5ba006d3c0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:55.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:55.751+0000 7f5bc310d700 1 --2- 172.21.15.143:0/4202616479 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5bbc0ae770 0x7f5bbc0c4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:55.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:55.751+0000 7f5bc310d700 1 --2- 172.21.15.143:0/4202616479 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5bac003a00 0x7f5bbc0b33b0 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:55.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:55.751+0000 7f5bc310d700 1 --2- 172.21.15.143:0/4202616479 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5bbc15acb0 0x7f5bbc0ae230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:55.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:55.751+0000 7f5bc310d700 1 -- 172.21.15.143:0/4202616479 >> 172.21.15.143:0/4202616479 conn(0x7f5bbc099ab0 msgr2=0x7f5bbc0ad410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:55.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:55.751+0000 7f5bc310d700 1 -- 172.21.15.143:0/4202616479 shutdown_connections 2023-11-08T19:29:55.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:29:55.752+0000 7f5bc310d700 1 -- 172.21.15.143:0/4202616479 wait complete. 2023-11-08T19:29:55.756 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 81 2023-11-08T19:29:55.765 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2023-11-08T19:29:56.795 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2023-11-08T19:29:57.547 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T19:29:57.550 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T19:30:00.766 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:30:00.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:00.901+0000 7f5cdc597700 1 Processor -- start 2023-11-08T19:30:00.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:00.901+0000 7f5cdc597700 1 -- start start 2023-11-08T19:30:00.905 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:00.902+0000 7f5cdc597700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f5cd4157e30 0x7f5cd4071aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:00.906 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:00.902+0000 7f5cdc597700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cd40ceb30 con 0x7f5cd40cd8d0 2023-11-08T19:30:00.906 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:00.902+0000 7f5cdc597700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cd4070fe0 con 0x7f5cd40cde00 2023-11-08T19:30:00.906 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:00.902+0000 7f5cdc597700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cd4071680 con 0x7f5cd4157e30 2023-11-08T19:30:00.906 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:00.902+0000 7f5cda333700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f5cd4157e30 0x7f5cd4071aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:00.906 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:00.902+0000 7f5cdab34700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f5cd40cd8d0 0x7f5cd40ca590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:52144/0 (socket says 172.21.15.143:52144) 2023-11-08T19:30:00.907 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:00.902+0000 7f5cda333700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f5cd4157e30 0x7f5cd4071aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55986/0 (socket says 172.21.15.143:55986) 2023-11-08T19:30:00.907 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:00.902+0000 7f5cdab34700 1 -- 172.21.15.143:0/1979264145 learned_addr learned my addr 172.21.15.143:0/1979264145 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:30:00.907 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:00.903+0000 7f5cda333700 1 -- 172.21.15.143:0/1979264145 >> v1:172.21.15.174:6789/0 conn(0x7f5cd40cde00 legacy=0x7f5cd40cd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:00.907 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:00.903+0000 7f5cda333700 1 -- 172.21.15.143:0/1979264145 >> v1:172.21.15.143:6789/0 conn(0x7f5cd40cd8d0 legacy=0x7f5cd40ca590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:00.907 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:00.903+0000 7f5cd9331700 1 -- 172.21.15.143:0/1979264145 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1433266241 0 0) 0x7f5cd40ceb30 con 0x7f5cd40cd8d0 2023-11-08T19:30:00.908 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:00.903+0000 7f5cda333700 1 -- 172.21.15.143:0/1979264145 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cd40ce850 con 0x7f5cd4157e30 2023-11-08T19:30:00.908 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:00.903+0000 7f5cda333700 1 --2- 172.21.15.143:0/1979264145 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5cd4157e30 0x7f5cd4071aa0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f5cc4009840 tx=0x7f5cc4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af5ab7371bfcdbf5 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:00.908 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:00.903+0000 7f5cd9331700 1 -- 172.21.15.143:0/1979264145 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cc4011830 con 0x7f5cd4157e30 2023-11-08T19:30:00.908 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:00.903+0000 7f5cd9331700 1 -- 172.21.15.143:0/1979264145 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cc4011990 con 0x7f5cd4157e30 2023-11-08T19:30:00.908 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:00.903+0000 7f5cd9331700 1 -- 172.21.15.143:0/1979264145 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cc4011d20 con 0x7f5cd4157e30 2023-11-08T19:30:00.909 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:00.903+0000 7f5cdc597700 1 -- 172.21.15.143:0/1979264145 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5cd4157e30 msgr2=0x7f5cd4071aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:00.909 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:00.903+0000 7f5cdc597700 1 --2- 172.21.15.143:0/1979264145 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5cd4157e30 0x7f5cd4071aa0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f5cc4009840 tx=0x7f5cc4009b50 comp rx=0 tx=0).stop 2023-11-08T19:30:00.909 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:00.904+0000 7f5cdc597700 1 -- 172.21.15.143:0/1979264145 shutdown_connections 2023-11-08T19:30:00.909 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:00.904+0000 7f5cdc597700 1 --2- 172.21.15.143:0/1979264145 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5cd4157e30 0x7f5cd4071aa0 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:00.909 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:00.904+0000 7f5cdc597700 1 -- 172.21.15.143:0/1979264145 >> 172.21.15.143:0/1979264145 conn(0x7f5cd406b4c0 msgr2=0x7f5cd40cf680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:00.910 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:00.904+0000 7f5cdc597700 1 -- 172.21.15.143:0/1979264145 shutdown_connections 2023-11-08T19:30:00.910 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:00.904+0000 7f5cdc597700 1 -- 172.21.15.143:0/1979264145 wait complete. 2023-11-08T19:30:00.913 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:00.905+0000 7f5cdc597700 1 Processor -- start 2023-11-08T19:30:00.913 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:00.905+0000 7f5cdc597700 1 -- start start 2023-11-08T19:30:00.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:00.906+0000 7f5cdc597700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5cd40cd8d0 0x7f5cd40b49b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:00.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:00.906+0000 7f5cdc597700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5cd40cde00 0x7f5cd406e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:00.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:00.906+0000 7f5cdc597700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5cd4157e30 0x7f5cd40b0400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:00.915 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:00.906+0000 7f5cdc597700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd4147b90 con 0x7f5cd4157e30 2023-11-08T19:30:00.915 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:00.906+0000 7f5cdc597700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd406e9d0 con 0x7f5cd40cd8d0 2023-11-08T19:30:00.915 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:00.906+0000 7f5cdc597700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd414bbb0 con 0x7f5cd40cde00 2023-11-08T19:30:00.915 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:00.906+0000 7f5cdab34700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5cd4157e30 0x7f5cd40b0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:00.915 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:00.906+0000 7f5cdab34700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5cd4157e30 0x7f5cd40b0400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55996/0 (socket says 172.21.15.143:55996) 2023-11-08T19:30:00.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:00.906+0000 7f5cdab34700 1 -- 172.21.15.143:0/3131396310 learned_addr learned my addr 172.21.15.143:0/3131396310 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:30:00.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:00.906+0000 7f5cda333700 1 --2- 172.21.15.143:0/3131396310 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5cd40cd8d0 0x7f5cd40b49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:00.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:00.906+0000 7f5cd9b32700 1 --2- 172.21.15.143:0/3131396310 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5cd40cde00 0x7f5cd406e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:00.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:00.906+0000 7f5cdab34700 1 -- 172.21.15.143:0/3131396310 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5cd40cde00 msgr2=0x7f5cd406e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:00.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:00.906+0000 7f5cdab34700 1 --2- 172.21.15.143:0/3131396310 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5cd40cde00 0x7f5cd406e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:00.917 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:00.906+0000 7f5cdab34700 1 -- 172.21.15.143:0/3131396310 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5cd40cd8d0 msgr2=0x7f5cd40b49b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:00.917 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:00.906+0000 7f5cdab34700 1 --2- 172.21.15.143:0/3131396310 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5cd40cd8d0 0x7f5cd40b49b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:00.917 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:00.906+0000 7f5cdab34700 1 -- 172.21.15.143:0/3131396310 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cc40094a0 con 0x7f5cd4157e30 2023-11-08T19:30:00.917 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:00.906+0000 7f5cda333700 1 --2- 172.21.15.143:0/3131396310 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5cd40cd8d0 0x7f5cd40b49b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:30:00.918 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:00.906+0000 7f5cd9b32700 1 --2- 172.21.15.143:0/3131396310 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5cd40cde00 0x7f5cd406e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:30:00.918 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:00.906+0000 7f5cdab34700 1 --2- 172.21.15.143:0/3131396310 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5cd4157e30 0x7f5cd40b0400 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f5ccc00cca0 tx=0x7f5ccc00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:00.918 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:00.906+0000 7f5ccaffd700 1 -- 172.21.15.143:0/3131396310 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ccc00cea0 con 0x7f5cd4157e30 2023-11-08T19:30:00.918 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:00.906+0000 7f5ccaffd700 1 -- 172.21.15.143:0/3131396310 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ccc00ad60 con 0x7f5cd4157e30 2023-11-08T19:30:00.919 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:00.906+0000 7f5cdc597700 1 -- 172.21.15.143:0/3131396310 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cd414be20 con 0x7f5cd4157e30 2023-11-08T19:30:00.919 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:00.906+0000 7f5ccaffd700 1 -- 172.21.15.143:0/3131396310 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ccc007310 con 0x7f5cd4157e30 2023-11-08T19:30:00.919 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:00.906+0000 7f5cdc597700 1 -- 172.21.15.143:0/3131396310 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cd414c3e0 con 0x7f5cd4157e30 2023-11-08T19:30:00.919 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:00.907+0000 7f5cdc597700 1 -- 172.21.15.143:0/3131396310 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cd404eee0 con 0x7f5cd4157e30 2023-11-08T19:30:00.919 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:00.908+0000 7f5ccaffd700 1 -- 172.21.15.143:0/3131396310 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5ccc007470 con 0x7f5cd4157e30 2023-11-08T19:30:00.919 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:00.908+0000 7f5ccaffd700 1 --2- 172.21.15.143:0/3131396310 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f5cb806af00 0x7f5cb806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:00.920 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:00.908+0000 7f5ccaffd700 1 -- 172.21.15.143:0/3131396310 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 7059+0+0 (secure 0 0 0) 0x7f5ccc089970 con 0x7f5cd4157e30 2023-11-08T19:30:00.920 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:00.908+0000 7f5cda333700 1 --2- 172.21.15.143:0/3131396310 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f5cb806af00 0x7f5cb806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:00.920 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:00.909+0000 7f5cda333700 1 --2- 172.21.15.143:0/3131396310 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f5cb806af00 0x7f5cb806d3c0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f5cd40cb4a0 tx=0x7f5cc401c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:00.920 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:00.912+0000 7f5ccaffd700 1 -- 172.21.15.143:0/3131396310 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5ccc054560 con 0x7f5cd4157e30 2023-11-08T19:30:01.147 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:01.146+0000 7f5cdc597700 1 -- 172.21.15.143:0/3131396310 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5cd40b5600 con 0x7f5cd4157e30 2023-11-08T19:30:01.147 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:01.146+0000 7f5ccaffd700 1 -- 172.21.15.143:0/3131396310 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f5ccc018090 con 0x7f5cd4157e30 2023-11-08T19:30:01.148 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:30:01.148 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":81,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5933,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/3918286710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6835","nonce":3918286710},{"type":"v1","addr":"172.21.15.174:6837","nonce":3918286710}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":7494,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6834/1535103237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":1535103237},{"type":"v1","addr":"172.21.15.143:6834","nonce":1535103237}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:03.092245+0000","modified":"2023-11-08T19:29:49.416779+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":97,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7386,"mds_1":6918},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6918":{"gid":6918,"name":"b","rank":1,"incarnation":54,"state":"up:active","state_seq":6,"addr":"172.21.15.174:6836/3601088618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3601088618},{"type":"v1","addr":"172.21.15.174:6836","nonce":3601088618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7386":{"gid":7386,"name":"a","rank":0,"incarnation":76,"state":"up:active","state_seq":11,"addr":"172.21.15.143:6835/3664584522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":3664584522},{"type":"v1","addr":"172.21.15.143:6835","nonce":3664584522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:30:01.150 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:01.148+0000 7f5cdc597700 1 -- 172.21.15.143:0/3131396310 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f5cb806af00 msgr2=0x7f5cb806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:01.151 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:01.148+0000 7f5cdc597700 1 --2- 172.21.15.143:0/3131396310 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f5cb806af00 0x7f5cb806d3c0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f5cd40cb4a0 tx=0x7f5cc401c040 comp rx=0 tx=0).stop 2023-11-08T19:30:01.151 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:01.148+0000 7f5cdc597700 1 -- 172.21.15.143:0/3131396310 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5cd4157e30 msgr2=0x7f5cd40b0400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:01.151 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:01.148+0000 7f5cdc597700 1 --2- 172.21.15.143:0/3131396310 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5cd4157e30 0x7f5cd40b0400 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f5ccc00cca0 tx=0x7f5ccc00a340 comp rx=0 tx=0).stop 2023-11-08T19:30:01.151 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:01.148+0000 7f5cdc597700 1 -- 172.21.15.143:0/3131396310 shutdown_connections 2023-11-08T19:30:01.152 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:01.148+0000 7f5cdc597700 1 --2- 172.21.15.143:0/3131396310 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f5cb806af00 0x7f5cb806d3c0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:01.152 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:01.148+0000 7f5cdc597700 1 --2- 172.21.15.143:0/3131396310 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5cd4157e30 0x7f5cd40b0400 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:01.152 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:01.148+0000 7f5cdc597700 1 --2- 172.21.15.143:0/3131396310 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5cd40cd8d0 0x7f5cd40b49b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:01.152 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:01.148+0000 7f5cdc597700 1 --2- 172.21.15.143:0/3131396310 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5cd40cde00 0x7f5cd406e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:01.153 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:01.149+0000 7f5cdc597700 1 -- 172.21.15.143:0/3131396310 >> 172.21.15.143:0/3131396310 conn(0x7f5cd406b4c0 msgr2=0x7f5cd4151b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:01.153 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:01.149+0000 7f5cdc597700 1 -- 172.21.15.143:0/3131396310 shutdown_connections 2023-11-08T19:30:01.153 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:01.149+0000 7f5cdc597700 1 -- 172.21.15.143:0/3131396310 wait complete. 2023-11-08T19:30:01.154 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 81 2023-11-08T19:30:01.162 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2023-11-08T19:30:03.405 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~7s 2023-11-08T19:30:06.164 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:30:06.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:06.296+0000 7fb710e68700 1 Processor -- start 2023-11-08T19:30:06.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:06.296+0000 7fb710e68700 1 -- start start 2023-11-08T19:30:06.300 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:06.297+0000 7fb710e68700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb70c0b2920 0x7fb70c0cf510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:06.301 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:06.297+0000 7fb710e68700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb70c0cbe20 0x7fb70c0cc250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:06.301 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:06.297+0000 7fb710e68700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb70c0cf080 con 0x7fb70c15aa30 2023-11-08T19:30:06.301 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:06.297+0000 7fb710e68700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb70c0ce470 con 0x7fb70c0cbe20 2023-11-08T19:30:06.301 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:06.297+0000 7fb710e68700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb70c0ce750 con 0x7fb70c0b2920 2023-11-08T19:30:06.302 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:06.298+0000 7fb70ad9d700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb70c0cbe20 0x7fb70c0cc250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:06.302 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:06.298+0000 7fb70ad9d700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb70c0cbe20 0x7fb70c0cc250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56006/0 (socket says 172.21.15.143:56006) 2023-11-08T19:30:06.302 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:06.298+0000 7fb70ad9d700 1 -- 172.21.15.143:0/1782902621 learned_addr learned my addr 172.21.15.143:0/1782902621 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:30:06.302 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:06.298+0000 7fb709d9b700 1 --2- 172.21.15.143:0/1782902621 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb70c0b2920 0x7fb70c0cf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:06.302 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:06.298+0000 7fb70ad9d700 1 -- 172.21.15.143:0/1782902621 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb70c0b2920 msgr2=0x7fb70c0cf510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:06.303 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:06.298+0000 7fb70ad9d700 1 --2- 172.21.15.143:0/1782902621 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb70c0b2920 0x7fb70c0cf510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:06.303 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:06.298+0000 7fb70ad9d700 1 -- 172.21.15.143:0/1782902621 >> v1:172.21.15.174:6789/0 conn(0x7fb70c15aa30 legacy=0x7fb70c0b2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:06.303 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:06.298+0000 7fb70ad9d700 1 -- 172.21.15.143:0/1782902621 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb70c0cea30 con 0x7fb70c0cbe20 2023-11-08T19:30:06.303 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:06.298+0000 7fb70ad9d700 1 --2- 172.21.15.143:0/1782902621 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb70c0cbe20 0x7fb70c0cc250 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7fb704009e50 tx=0x7fb70400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=798fcfafbadaddd0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:06.303 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:06.298+0000 7fb70959a700 1 -- 172.21.15.143:0/1782902621 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb70400e070 con 0x7fb70c0cbe20 2023-11-08T19:30:06.304 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:06.298+0000 7fb70959a700 1 -- 172.21.15.143:0/1782902621 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb70400b9c0 con 0x7fb70c0cbe20 2023-11-08T19:30:06.304 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:06.298+0000 7fb70959a700 1 -- 172.21.15.143:0/1782902621 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb70400bd70 con 0x7fb70c0cbe20 2023-11-08T19:30:06.304 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:06.299+0000 7fb710e68700 1 -- 172.21.15.143:0/1782902621 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb70c0cbe20 msgr2=0x7fb70c0cc250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:06.305 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:06.299+0000 7fb710e68700 1 --2- 172.21.15.143:0/1782902621 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb70c0cbe20 0x7fb70c0cc250 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7fb704009e50 tx=0x7fb70400b330 comp rx=0 tx=0).stop 2023-11-08T19:30:06.305 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:06.299+0000 7fb710e68700 1 -- 172.21.15.143:0/1782902621 shutdown_connections 2023-11-08T19:30:06.305 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:06.299+0000 7fb710e68700 1 --2- 172.21.15.143:0/1782902621 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb70c0cbe20 0x7fb70c0cc250 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:06.305 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:06.299+0000 7fb710e68700 1 --2- 172.21.15.143:0/1782902621 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb70c0b2920 0x7fb70c0cf510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:06.306 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:06.299+0000 7fb710e68700 1 -- 172.21.15.143:0/1782902621 >> 172.21.15.143:0/1782902621 conn(0x7fb70c099ab0 msgr2=0x7fb70c0d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:06.306 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:06.299+0000 7fb710e68700 1 -- 172.21.15.143:0/1782902621 shutdown_connections 2023-11-08T19:30:06.306 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:06.299+0000 7fb710e68700 1 -- 172.21.15.143:0/1782902621 wait complete. 2023-11-08T19:30:06.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:06.300+0000 7fb710e68700 1 Processor -- start 2023-11-08T19:30:06.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:06.300+0000 7fb710e68700 1 -- start start 2023-11-08T19:30:06.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:06.301+0000 7fb710e68700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb70c0b2920 0x7fb70c15c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:06.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:06.301+0000 7fb710e68700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb70c0cbe20 0x7fb70c0c5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:06.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:06.301+0000 7fb710e68700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb70c15aa30 0x7fb70c15b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:06.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:06.301+0000 7fb710e68700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb70c1f1490 con 0x7fb70c0cbe20 2023-11-08T19:30:06.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:06.301+0000 7fb710e68700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb70c1f4ed0 con 0x7fb70c0b2920 2023-11-08T19:30:06.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:06.301+0000 7fb710e68700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb70c1f51b0 con 0x7fb70c15aa30 2023-11-08T19:30:06.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:06.301+0000 7fb709d9b700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb70c0cbe20 0x7fb70c0c5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:06.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:06.301+0000 7fb709d9b700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb70c0cbe20 0x7fb70c0c5ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56010/0 (socket says 172.21.15.143:56010) 2023-11-08T19:30:06.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:06.301+0000 7fb709d9b700 1 -- 172.21.15.143:0/153363996 learned_addr learned my addr 172.21.15.143:0/153363996 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:30:06.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:06.301+0000 7fb70a59c700 1 --2- 172.21.15.143:0/153363996 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb70c0b2920 0x7fb70c15c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:06.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:06.301+0000 7fb70ad9d700 1 --2- 172.21.15.143:0/153363996 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb70c15aa30 0x7fb70c15b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:06.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:06.301+0000 7fb709d9b700 1 -- 172.21.15.143:0/153363996 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb70c15aa30 msgr2=0x7fb70c15b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:06.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:06.302+0000 7fb709d9b700 1 --2- 172.21.15.143:0/153363996 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb70c15aa30 0x7fb70c15b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:06.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:06.302+0000 7fb709d9b700 1 -- 172.21.15.143:0/153363996 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb70c0b2920 msgr2=0x7fb70c15c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:06.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:06.302+0000 7fb709d9b700 1 --2- 172.21.15.143:0/153363996 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb70c0b2920 0x7fb70c15c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:06.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:06.302+0000 7fb709d9b700 1 -- 172.21.15.143:0/153363996 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb704009b30 con 0x7fb70c0cbe20 2023-11-08T19:30:06.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:06.302+0000 7fb70a59c700 1 --2- 172.21.15.143:0/153363996 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb70c0b2920 0x7fb70c15c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:30:06.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:06.302+0000 7fb70ad9d700 1 --2- 172.21.15.143:0/153363996 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb70c15aa30 0x7fb70c15b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:30:06.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:06.302+0000 7fb709d9b700 1 --2- 172.21.15.143:0/153363996 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb70c0cbe20 0x7fb70c0c5ef0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc00e810 tx=0x7fb6fc00ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:06.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:06.302+0000 7fb702ffd700 1 -- 172.21.15.143:0/153363996 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6fc013610 con 0x7fb70c0cbe20 2023-11-08T19:30:06.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:06.302+0000 7fb702ffd700 1 -- 172.21.15.143:0/153363996 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6fc013770 con 0x7fb70c0cbe20 2023-11-08T19:30:06.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:06.302+0000 7fb710e68700 1 -- 172.21.15.143:0/153363996 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb70c1f5490 con 0x7fb70c0cbe20 2023-11-08T19:30:06.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:06.302+0000 7fb702ffd700 1 -- 172.21.15.143:0/153363996 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6fc013b20 con 0x7fb70c0cbe20 2023-11-08T19:30:06.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:06.302+0000 7fb710e68700 1 -- 172.21.15.143:0/153363996 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb70c15cd50 con 0x7fb70c0cbe20 2023-11-08T19:30:06.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:06.303+0000 7fb710e68700 1 -- 172.21.15.143:0/153363996 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb70c04eee0 con 0x7fb70c0cbe20 2023-11-08T19:30:06.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:06.303+0000 7fb702ffd700 1 -- 172.21.15.143:0/153363996 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb6fc013c80 con 0x7fb70c0cbe20 2023-11-08T19:30:06.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:06.304+0000 7fb702ffd700 1 --2- 172.21.15.143:0/153363996 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fb6ec06af00 0x7fb6ec06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:06.315 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:06.304+0000 7fb702ffd700 1 -- 172.21.15.143:0/153363996 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 7059+0+0 (secure 0 0 0) 0x7fb6fc089990 con 0x7fb70c0cbe20 2023-11-08T19:30:06.315 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:06.304+0000 7fb70a59c700 1 --2- 172.21.15.143:0/153363996 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fb6ec06af00 0x7fb6ec06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:06.315 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:06.305+0000 7fb70a59c700 1 --2- 172.21.15.143:0/153363996 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fb6ec06af00 0x7fb6ec06d3c0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fb6f4000fb0 tx=0x7fb6f4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:06.316 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:06.308+0000 7fb702ffd700 1 -- 172.21.15.143:0/153363996 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb6fc0543d0 con 0x7fb70c0cbe20 2023-11-08T19:30:06.540 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:06.538+0000 7fb710e68700 1 -- 172.21.15.143:0/153363996 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb70c0b2ff0 con 0x7fb70c0cbe20 2023-11-08T19:30:06.540 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:06.539+0000 7fb702ffd700 1 -- 172.21.15.143:0/153363996 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fb6fc0579f0 con 0x7fb70c0cbe20 2023-11-08T19:30:06.541 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:30:06.541 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":81,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5933,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/3918286710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6835","nonce":3918286710},{"type":"v1","addr":"172.21.15.174:6837","nonce":3918286710}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":7494,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6834/1535103237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":1535103237},{"type":"v1","addr":"172.21.15.143:6834","nonce":1535103237}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:03.092245+0000","modified":"2023-11-08T19:29:49.416779+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":97,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7386,"mds_1":6918},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6918":{"gid":6918,"name":"b","rank":1,"incarnation":54,"state":"up:active","state_seq":6,"addr":"172.21.15.174:6836/3601088618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3601088618},{"type":"v1","addr":"172.21.15.174:6836","nonce":3601088618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7386":{"gid":7386,"name":"a","rank":0,"incarnation":76,"state":"up:active","state_seq":11,"addr":"172.21.15.143:6835/3664584522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":3664584522},{"type":"v1","addr":"172.21.15.143:6835","nonce":3664584522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:30:06.543 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:06.541+0000 7fb710e68700 1 -- 172.21.15.143:0/153363996 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fb6ec06af00 msgr2=0x7fb6ec06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:06.543 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:06.541+0000 7fb710e68700 1 --2- 172.21.15.143:0/153363996 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fb6ec06af00 0x7fb6ec06d3c0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fb6f4000fb0 tx=0x7fb6f4009040 comp rx=0 tx=0).stop 2023-11-08T19:30:06.543 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:06.541+0000 7fb710e68700 1 -- 172.21.15.143:0/153363996 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb70c0cbe20 msgr2=0x7fb70c0c5ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:06.543 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:06.541+0000 7fb710e68700 1 --2- 172.21.15.143:0/153363996 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb70c0cbe20 0x7fb70c0c5ef0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc00e810 tx=0x7fb6fc00ebd0 comp rx=0 tx=0).stop 2023-11-08T19:30:06.544 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:06.541+0000 7fb710e68700 1 -- 172.21.15.143:0/153363996 shutdown_connections 2023-11-08T19:30:06.544 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:06.541+0000 7fb710e68700 1 --2- 172.21.15.143:0/153363996 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fb6ec06af00 0x7fb6ec06d3c0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:06.544 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:06.541+0000 7fb710e68700 1 --2- 172.21.15.143:0/153363996 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb70c15aa30 0x7fb70c15b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:06.544 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:06.541+0000 7fb710e68700 1 --2- 172.21.15.143:0/153363996 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb70c0b2920 0x7fb70c15c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:06.544 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:06.541+0000 7fb710e68700 1 --2- 172.21.15.143:0/153363996 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb70c0cbe20 0x7fb70c0c5ef0 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:06.545 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:06.541+0000 7fb710e68700 1 -- 172.21.15.143:0/153363996 >> 172.21.15.143:0/153363996 conn(0x7fb70c099ab0 msgr2=0x7fb70c0b01f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:06.545 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:06.541+0000 7fb710e68700 1 -- 172.21.15.143:0/153363996 shutdown_connections 2023-11-08T19:30:06.545 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:06.541+0000 7fb710e68700 1 -- 172.21.15.143:0/153363996 wait complete. 2023-11-08T19:30:06.545 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 81 2023-11-08T19:30:06.555 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2023-11-08T19:30:10.017 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~13s 2023-11-08T19:30:11.556 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:30:11.689 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:11.688+0000 7f7799fc9700 1 Processor -- start 2023-11-08T19:30:11.689 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:11.688+0000 7f7799fc9700 1 -- start start 2023-11-08T19:30:11.692 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:11.688+0000 7f7799fc9700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f7794159870 0x7f77940b2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:11.692 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:11.688+0000 7f7799fc9700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f77940cc410 0x7f77940cc860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:11.692 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:11.688+0000 7f7799fc9700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77940b2a60 con 0x7f779406ed60 2023-11-08T19:30:11.692 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:11.688+0000 7f7799fc9700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f779406b090 con 0x7f7794159870 2023-11-08T19:30:11.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:11.688+0000 7f7799fc9700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f779406bbc0 con 0x7f77940cc410 2023-11-08T19:30:11.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:11.689+0000 7f77937fe700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f7794159870 0x7f77940b2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:11.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:11.689+0000 7f7793fff700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f77940cc410 0x7f77940cc860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:11.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:11.689+0000 7f7792ffd700 1 --1- >> v1:172.21.15.174:6789/0 conn(0x7f779406ed60 0x7f779406b4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.174:6789/0 says I am v1:172.21.15.143:34874/0 (socket says 172.21.15.143:34874) 2023-11-08T19:30:11.694 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:11.689+0000 7f7792ffd700 1 -- 172.21.15.143:0/1325627373 learned_addr learned my addr 172.21.15.143:0/1325627373 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:30:11.694 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:11.689+0000 7f77937fe700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f7794159870 0x7f77940b2210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.143:39518/0 (socket says 172.21.15.143:39518) 2023-11-08T19:30:11.694 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:11.689+0000 7f77937fe700 1 -- 172.21.15.143:0/1325627373 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f77940cc410 msgr2=0x7f77940cc860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:11.694 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:11.689+0000 7f77937fe700 1 --2- 172.21.15.143:0/1325627373 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f77940cc410 0x7f77940cc860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:11.697 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:11.689+0000 7f77937fe700 1 -- 172.21.15.143:0/1325627373 >> v1:172.21.15.174:6789/0 conn(0x7f779406ed60 legacy=0x7f779406b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:11.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:11.689+0000 7f77927fc700 1 -- 172.21.15.143:0/1325627373 <== mon.1 v1:172.21.15.174:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2115564199 0 0) 0x7f77940b2a60 con 0x7f779406ed60 2023-11-08T19:30:11.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:11.689+0000 7f77937fe700 1 -- 172.21.15.143:0/1325627373 --> v2:172.21.15.174:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77940b2d40 con 0x7f7794159870 2023-11-08T19:30:11.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:11.689+0000 7f77937fe700 1 --2- 172.21.15.143:0/1325627373 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7794159870 0x7f77940b2210 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f7784009820 tx=0x7f7784009b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=156da0fd4b13a01 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:11.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:11.690+0000 7f77927fc700 1 -- 172.21.15.143:0/1325627373 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7784005e30 con 0x7f7794159870 2023-11-08T19:30:11.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:11.690+0000 7f77927fc700 1 -- 172.21.15.143:0/1325627373 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7784006030 con 0x7f7794159870 2023-11-08T19:30:11.699 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:11.690+0000 7f77927fc700 1 -- 172.21.15.143:0/1325627373 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77840063c0 con 0x7f7794159870 2023-11-08T19:30:11.699 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:11.690+0000 7f7799fc9700 1 -- 172.21.15.143:0/1325627373 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7794159870 msgr2=0x7f77940b2210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:11.699 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:11.690+0000 7f7799fc9700 1 --2- 172.21.15.143:0/1325627373 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7794159870 0x7f77940b2210 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f7784009820 tx=0x7f7784009b30 comp rx=0 tx=0).stop 2023-11-08T19:30:11.699 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:11.690+0000 7f7799fc9700 1 -- 172.21.15.143:0/1325627373 shutdown_connections 2023-11-08T19:30:11.699 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:11.690+0000 7f7799fc9700 1 --2- 172.21.15.143:0/1325627373 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f77940cc410 0x7f77940cc860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:11.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:11.690+0000 7f7799fc9700 1 --2- 172.21.15.143:0/1325627373 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7794159870 0x7f77940b2210 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:11.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:11.690+0000 7f7799fc9700 1 -- 172.21.15.143:0/1325627373 >> 172.21.15.143:0/1325627373 conn(0x7f7794099ab0 msgr2=0x7f77940d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:11.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:11.690+0000 7f7799fc9700 1 -- 172.21.15.143:0/1325627373 shutdown_connections 2023-11-08T19:30:11.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:11.690+0000 7f7799fc9700 1 -- 172.21.15.143:0/1325627373 wait complete. 2023-11-08T19:30:11.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:11.691+0000 7f7799fc9700 1 Processor -- start 2023-11-08T19:30:11.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:11.692+0000 7f7799fc9700 1 -- start start 2023-11-08T19:30:11.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:11.692+0000 7f7799fc9700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f779406ed60 0x7f77940cfe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:11.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:11.692+0000 7f7799fc9700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f77940cc410 0x7f77940af040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:11.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:11.692+0000 7f7799fc9700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7794159870 0x7f77940cd3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:11.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:11.692+0000 7f7799fc9700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77940b5cd0 con 0x7f7794159870 2023-11-08T19:30:11.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:11.692+0000 7f7799fc9700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f779415c870 con 0x7f779406ed60 2023-11-08T19:30:11.705 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:11.692+0000 7f7799fc9700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77940b5330 con 0x7f77940cc410 2023-11-08T19:30:11.705 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:11.692+0000 7f7793fff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7794159870 0x7f77940cd3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:11.705 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:11.692+0000 7f7793fff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7794159870 0x7f77940cd3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56026/0 (socket says 172.21.15.143:56026) 2023-11-08T19:30:11.705 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:11.692+0000 7f7793fff700 1 -- 172.21.15.143:0/1568022271 learned_addr learned my addr 172.21.15.143:0/1568022271 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:30:11.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:11.692+0000 7f7792ffd700 1 --2- 172.21.15.143:0/1568022271 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f77940cc410 0x7f77940af040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:11.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:11.692+0000 7f77937fe700 1 --2- 172.21.15.143:0/1568022271 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f779406ed60 0x7f77940cfe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:11.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:11.693+0000 7f7793fff700 1 -- 172.21.15.143:0/1568022271 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f77940cc410 msgr2=0x7f77940af040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:11.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:11.693+0000 7f7793fff700 1 --2- 172.21.15.143:0/1568022271 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f77940cc410 0x7f77940af040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:11.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:11.693+0000 7f7793fff700 1 -- 172.21.15.143:0/1568022271 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f779406ed60 msgr2=0x7f77940cfe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:11.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:11.693+0000 7f7793fff700 1 --2- 172.21.15.143:0/1568022271 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f779406ed60 0x7f77940cfe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:11.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:11.693+0000 7f7793fff700 1 -- 172.21.15.143:0/1568022271 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7784009430 con 0x7f7794159870 2023-11-08T19:30:11.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:11.693+0000 7f7793fff700 1 --2- 172.21.15.143:0/1568022271 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7794159870 0x7f77940cd3e0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f778c00efc0 tx=0x7f778c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:11.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:11.693+0000 7f777bfff700 1 -- 172.21.15.143:0/1568022271 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f778c009ea0 con 0x7f7794159870 2023-11-08T19:30:11.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:11.693+0000 7f777bfff700 1 -- 172.21.15.143:0/1568022271 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f778c00f040 con 0x7f7794159870 2023-11-08T19:30:11.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:11.693+0000 7f7799fc9700 1 -- 172.21.15.143:0/1568022271 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77940b5610 con 0x7f7794159870 2023-11-08T19:30:11.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:11.693+0000 7f777bfff700 1 -- 172.21.15.143:0/1568022271 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f778c00ced0 con 0x7f7794159870 2023-11-08T19:30:11.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:11.693+0000 7f7799fc9700 1 -- 172.21.15.143:0/1568022271 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77940b60f0 con 0x7f7794159870 2023-11-08T19:30:11.709 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:11.694+0000 7f7799fc9700 1 -- 172.21.15.143:0/1568022271 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77940b5e10 con 0x7f7794159870 2023-11-08T19:30:11.709 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:11.695+0000 7f777bfff700 1 -- 172.21.15.143:0/1568022271 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f778c014480 con 0x7f7794159870 2023-11-08T19:30:11.709 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:11.695+0000 7f777bfff700 1 --2- 172.21.15.143:0/1568022271 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f778006af00 0x7f778006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:11.709 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:11.695+0000 7f777bfff700 1 -- 172.21.15.143:0/1568022271 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 7059+0+0 (secure 0 0 0) 0x7f778c08a580 con 0x7f7794159870 2023-11-08T19:30:11.709 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:11.695+0000 7f77937fe700 1 --2- 172.21.15.143:0/1568022271 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f778006af00 0x7f778006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:11.709 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:11.696+0000 7f77937fe700 1 --2- 172.21.15.143:0/1568022271 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f778006af00 0x7f778006d3c0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f7784000c00 tx=0x7f7784008070 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:11.710 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:11.699+0000 7f777bfff700 1 -- 172.21.15.143:0/1568022271 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f77940b5e10 con 0x7f7794159870 2023-11-08T19:30:11.937 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:11.934+0000 7f7799fc9700 1 -- 172.21.15.143:0/1568022271 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f77940b5e10 con 0x7f7794159870 2023-11-08T19:30:11.937 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:11.935+0000 7f777bfff700 1 -- 172.21.15.143:0/1568022271 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+4991 (secure 0 0 0) 0x7f77940b5e10 con 0x7f7794159870 2023-11-08T19:30:11.938 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:30:11.938 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":82,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5933,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/3918286710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6835","nonce":3918286710},{"type":"v1","addr":"172.21.15.174:6837","nonce":3918286710}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":7494,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6834/1535103237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":1535103237},{"type":"v1","addr":"172.21.15.143:6834","nonce":1535103237}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:03.092245+0000","modified":"2023-11-08T19:30:07.615282+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":97,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7386,"mds_1":6918},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6918":{"gid":6918,"name":"b","rank":1,"incarnation":54,"state":"up:active","state_seq":6,"addr":"172.21.15.174:6836/3601088618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3601088618},{"type":"v1","addr":"172.21.15.174:6836","nonce":3601088618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7386":{"gid":7386,"name":"a","rank":0,"incarnation":76,"state":"up:active","state_seq":11,"addr":"172.21.15.143:6835/3664584522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":3664584522},{"type":"v1","addr":"172.21.15.143:6835","nonce":3664584522}]},"join_fscid":-1,"laggy_since":"2023-11-08T19:30:07.615282+0000","export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:30:11.939 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:11.937+0000 7f7799fc9700 1 -- 172.21.15.143:0/1568022271 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f778006af00 msgr2=0x7f778006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:11.940 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:11.937+0000 7f7799fc9700 1 --2- 172.21.15.143:0/1568022271 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f778006af00 0x7f778006d3c0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f7784000c00 tx=0x7f7784008070 comp rx=0 tx=0).stop 2023-11-08T19:30:11.940 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:11.937+0000 7f7799fc9700 1 -- 172.21.15.143:0/1568022271 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7794159870 msgr2=0x7f77940cd3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:11.940 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:11.937+0000 7f7799fc9700 1 --2- 172.21.15.143:0/1568022271 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7794159870 0x7f77940cd3e0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f778c00efc0 tx=0x7f778c00c3a0 comp rx=0 tx=0).stop 2023-11-08T19:30:11.940 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:11.937+0000 7f7799fc9700 1 -- 172.21.15.143:0/1568022271 shutdown_connections 2023-11-08T19:30:11.941 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:11.937+0000 7f7799fc9700 1 --2- 172.21.15.143:0/1568022271 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f778006af00 0x7f778006d3c0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:11.941 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:11.937+0000 7f7799fc9700 1 --2- 172.21.15.143:0/1568022271 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7794159870 0x7f77940cd3e0 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:11.941 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:11.937+0000 7f7799fc9700 1 --2- 172.21.15.143:0/1568022271 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f779406ed60 0x7f77940cfe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:11.941 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:11.937+0000 7f7799fc9700 1 --2- 172.21.15.143:0/1568022271 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f77940cc410 0x7f77940af040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:11.941 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:11.937+0000 7f7799fc9700 1 -- 172.21.15.143:0/1568022271 >> 172.21.15.143:0/1568022271 conn(0x7f7794099ab0 msgr2=0x7f77941579e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:11.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:11.937+0000 7f7799fc9700 1 -- 172.21.15.143:0/1568022271 shutdown_connections 2023-11-08T19:30:11.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:11.937+0000 7f7799fc9700 1 -- 172.21.15.143:0/1568022271 wait complete. 2023-11-08T19:30:11.942 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 82 2023-11-08T19:30:11.952 DEBUG:tasks.ceph_test_case:wait_until_true: success in 20s and 0 retries 2023-11-08T19:30:11.952 DEBUG:teuthology.orchestra.run.smithi143:> sudo sysctl -n kernel.core_pattern 2023-11-08T19:30:11.978 INFO:teuthology.orchestra.run.smithi143.stdout:/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-08T19:30:11.980 INFO:tasks.cephfs.cephfs_test_case:Clearing core from directory: /home/ubuntu/cephtest/archive/coredump 2023-11-08T19:30:11.980 DEBUG:teuthology.orchestra.run.smithi143:> cd /home/ubuntu/cephtest/archive/coredump && sudo ls | sudo xargs file 2023-11-08T19:30:12.049 INFO:teuthology.orchestra.run.smithi143.stdout:1699471790.134676.core: ELF 64-bit LSB core file, x86-64, version 1 (SYSV), SVR4-style, from 'ceph-mds -f --cluster ceph -i a', real uid: 0, effective uid: 0, real gid: 0, effective gid: 0, execfn: '/bin/ceph-mds', platform: 'x86_64' 2023-11-08T19:30:12.051 INFO:tasks.cephfs.cephfs_test_case:Enumerated cores: ['1699471790.134676.core'] 2023-11-08T19:30:12.051 INFO:tasks.cephfs.cephfs_test_case:Found core file 1699471790.134676.core, deleting it 2023-11-08T19:30:12.052 DEBUG:teuthology.orchestra.run.smithi143:> cd /home/ubuntu/cephtest/archive/coredump && sudo rm -f 1699471790.134676.core 2023-11-08T19:30:12.120 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail 6:0 2023-11-08T19:30:12.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.279+0000 7f782f21f700 1 Processor -- start 2023-11-08T19:30:12.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.279+0000 7f782f21f700 1 -- start start 2023-11-08T19:30:12.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.280+0000 7f782f21f700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f78280b2920 0x7f78280cf510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:12.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.280+0000 7f782f21f700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f78280cbe20 0x7f78280cc250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:12.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.280+0000 7f782f21f700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78280cf080 con 0x7f782815aa30 2023-11-08T19:30:12.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.280+0000 7f782f21f700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78280ce470 con 0x7f78280cbe20 2023-11-08T19:30:12.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.280+0000 7f782f21f700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78280ce750 con 0x7f78280b2920 2023-11-08T19:30:12.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.280+0000 7f782cfbb700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f782815aa30 0x7f78280b2210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:52186/0 (socket says 172.21.15.143:52186) 2023-11-08T19:30:12.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.280+0000 7f782cfbb700 1 -- 172.21.15.143:0/1929568115 learned_addr learned my addr 172.21.15.143:0/1929568115 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:30:12.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.280+0000 7f7827fff700 1 --2- 172.21.15.143:0/1929568115 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f78280b2920 0x7f78280cf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:12.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.280+0000 7f782d7bc700 1 --2- 172.21.15.143:0/1929568115 >> v2:172.21.15.174:3300/0 conn(0x7f78280cbe20 0x7f78280cc250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:12.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.280+0000 7f78277fe700 1 -- 172.21.15.143:0/1929568115 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2210404012 0 0) 0x7f78280cf080 con 0x7f782815aa30 2023-11-08T19:30:12.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.280+0000 7f78277fe700 1 -- 172.21.15.143:0/1929568115 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f78180035b0 con 0x7f782815aa30 2023-11-08T19:30:12.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.280+0000 7f78277fe700 1 -- 172.21.15.143:0/1929568115 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2730581512 0 0) 0x7f78180035b0 con 0x7f782815aa30 2023-11-08T19:30:12.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.280+0000 7f78277fe700 1 -- 172.21.15.143:0/1929568115 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f78280b2920 msgr2=0x7f78280cf510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:12.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.280+0000 7f78277fe700 1 --2- 172.21.15.143:0/1929568115 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f78280b2920 0x7f78280cf510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:12.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.280+0000 7f78277fe700 1 -- 172.21.15.143:0/1929568115 >> v2:172.21.15.174:3300/0 conn(0x7f78280cbe20 msgr2=0x7f78280cc250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:12.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.280+0000 7f78277fe700 1 --2- 172.21.15.143:0/1929568115 >> v2:172.21.15.174:3300/0 conn(0x7f78280cbe20 0x7f78280cc250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:12.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.280+0000 7f78277fe700 1 -- 172.21.15.143:0/1929568115 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78280cea30 con 0x7f782815aa30 2023-11-08T19:30:12.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.280+0000 7f78277fe700 1 -- 172.21.15.143:0/1929568115 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4147756303 0 0) 0x7f781c0017f0 con 0x7f782815aa30 2023-11-08T19:30:12.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.280+0000 7f78277fe700 1 -- 172.21.15.143:0/1929568115 >> v1:172.21.15.143:6789/0 conn(0x7f782815aa30 legacy=0x7f78280b2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:12.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.280+0000 7f78277fe700 1 --2- 172.21.15.143:0/1929568115 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7818003a00 0x7f7818003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:12.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.280+0000 7f78277fe700 1 --2- 172.21.15.143:0/1929568115 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7818004850 0x7f7818004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:12.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.280+0000 7f78277fe700 1 --2- 172.21.15.143:0/1929568115 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f78280b2920 0x7f7818005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:12.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.280+0000 7f78277fe700 1 -- 172.21.15.143:0/1929568115 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78280ce470 con 0x7f7818003a00 2023-11-08T19:30:12.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.280+0000 7f78277fe700 1 -- 172.21.15.143:0/1929568115 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78280ce750 con 0x7f7818004850 2023-11-08T19:30:12.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.280+0000 7f78277fe700 1 -- 172.21.15.143:0/1929568115 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f781800ba10 con 0x7f78280b2920 2023-11-08T19:30:12.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.281+0000 7f782cfbb700 1 --2- 172.21.15.143:0/1929568115 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7818003a00 0x7f7818003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:12.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.281+0000 7f7827fff700 1 --2- 172.21.15.143:0/1929568115 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7818004850 0x7f7818004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:12.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.281+0000 7f782d7bc700 1 --2- 172.21.15.143:0/1929568115 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f78280b2920 0x7f7818005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:12.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.281+0000 7f782cfbb700 1 -- 172.21.15.143:0/1929568115 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f78280b2920 msgr2=0x7f7818005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:12.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.281+0000 7f782cfbb700 1 --2- 172.21.15.143:0/1929568115 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f78280b2920 0x7f7818005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:12.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.281+0000 7f782cfbb700 1 -- 172.21.15.143:0/1929568115 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7818004850 msgr2=0x7f7818004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:12.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.281+0000 7f782cfbb700 1 --2- 172.21.15.143:0/1929568115 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7818004850 0x7f7818004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:12.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.281+0000 7f782cfbb700 1 -- 172.21.15.143:0/1929568115 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78180035b0 con 0x7f7818003a00 2023-11-08T19:30:12.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.281+0000 7f7827fff700 1 --2- 172.21.15.143:0/1929568115 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7818004850 0x7f7818004cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:30:12.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.281+0000 7f782cfbb700 1 --2- 172.21.15.143:0/1929568115 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7818003a00 0x7f7818003e20 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f781c00c240 tx=0x7f7818005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98e8553a96c7facc server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:12.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.281+0000 7f78277fe700 1 -- 172.21.15.143:0/1929568115 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f781c00cde0 con 0x7f7818003a00 2023-11-08T19:30:12.293 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.281+0000 7f78277fe700 1 -- 172.21.15.143:0/1929568115 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f781c007bd0 con 0x7f7818003a00 2023-11-08T19:30:12.293 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.281+0000 7f78277fe700 1 -- 172.21.15.143:0/1929568115 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f781c007310 con 0x7f7818003a00 2023-11-08T19:30:12.293 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.282+0000 7f782f21f700 1 -- 172.21.15.143:0/1929568115 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7818003a00 msgr2=0x7f7818003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:12.293 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.282+0000 7f782f21f700 1 --2- 172.21.15.143:0/1929568115 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7818003a00 0x7f7818003e20 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f781c00c240 tx=0x7f7818005880 comp rx=0 tx=0).stop 2023-11-08T19:30:12.293 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.282+0000 7f782d7bc700 1 -- 172.21.15.143:0/1929568115 reap_dead start 2023-11-08T19:30:12.294 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.282+0000 7f782f21f700 1 -- 172.21.15.143:0/1929568115 shutdown_connections 2023-11-08T19:30:12.294 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.282+0000 7f782f21f700 1 -- 172.21.15.143:0/1929568115 >> 172.21.15.143:0/1929568115 conn(0x7f7828099ab0 msgr2=0x7f78280d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:12.294 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.282+0000 7f782f21f700 1 -- 172.21.15.143:0/1929568115 shutdown_connections 2023-11-08T19:30:12.294 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.282+0000 7f782f21f700 1 -- 172.21.15.143:0/1929568115 wait complete. 2023-11-08T19:30:12.294 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.283+0000 7f782f21f700 1 Processor -- start 2023-11-08T19:30:12.295 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.283+0000 7f782f21f700 1 -- start start 2023-11-08T19:30:12.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.284+0000 7f782f21f700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7818003a00 0x7f78280b5d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:12.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.284+0000 7f782f21f700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f78280c6d40 0x7f782814fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:12.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.284+0000 7f782f21f700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f782815d190 0x7f782815d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:12.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.284+0000 7f782f21f700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f782815db50 con 0x7f78280c6d40 2023-11-08T19:30:12.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.284+0000 7f782f21f700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78280b33b0 con 0x7f7818003a00 2023-11-08T19:30:12.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.284+0000 7f782f21f700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78280c5360 con 0x7f782815d190 2023-11-08T19:30:12.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.284+0000 7f7827fff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f78280c6d40 0x7f782814fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:12.299 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.284+0000 7f7827fff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f78280c6d40 0x7f782814fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56046/0 (socket says 172.21.15.143:56046) 2023-11-08T19:30:12.299 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.284+0000 7f7827fff700 1 -- 172.21.15.143:0/1976070191 learned_addr learned my addr 172.21.15.143:0/1976070191 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:30:12.299 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.284+0000 7f782cfbb700 1 --2- 172.21.15.143:0/1976070191 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7818003a00 0x7f78280b5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:12.299 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.284+0000 7f782d7bc700 1 --2- 172.21.15.143:0/1976070191 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f782815d190 0x7f782815d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:12.299 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.284+0000 7f7827fff700 1 -- 172.21.15.143:0/1976070191 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f782815d190 msgr2=0x7f782815d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:12.300 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.284+0000 7f7827fff700 1 --2- 172.21.15.143:0/1976070191 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f782815d190 0x7f782815d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:12.300 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.284+0000 7f7827fff700 1 -- 172.21.15.143:0/1976070191 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7818003a00 msgr2=0x7f78280b5d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:12.300 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.284+0000 7f7827fff700 1 --2- 172.21.15.143:0/1976070191 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7818003a00 0x7f78280b5d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:12.300 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.284+0000 7f7827fff700 1 -- 172.21.15.143:0/1976070191 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f782815bae0 con 0x7f78280c6d40 2023-11-08T19:30:12.300 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.284+0000 7f7827fff700 1 --2- 172.21.15.143:0/1976070191 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f78280c6d40 0x7f782814fcb0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f7814010850 tx=0x7f7814010b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:12.301 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.284+0000 7f78257fa700 1 -- 172.21.15.143:0/1976070191 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7814004020 con 0x7f78280c6d40 2023-11-08T19:30:12.301 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.284+0000 7f78257fa700 1 -- 172.21.15.143:0/1976070191 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7814004180 con 0x7f78280c6d40 2023-11-08T19:30:12.301 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.284+0000 7f782f21f700 1 -- 172.21.15.143:0/1976070191 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f782815bdc0 con 0x7f78280c6d40 2023-11-08T19:30:12.301 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.284+0000 7f78257fa700 1 -- 172.21.15.143:0/1976070191 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7814004510 con 0x7f78280c6d40 2023-11-08T19:30:12.301 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.285+0000 7f782f21f700 1 -- 172.21.15.143:0/1976070191 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f782815c380 con 0x7f78280c6d40 2023-11-08T19:30:12.302 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.285+0000 7f782f21f700 1 -- 172.21.15.143:0/1976070191 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78280c54d0 con 0x7f78280c6d40 2023-11-08T19:30:12.302 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.286+0000 7f78257fa700 1 -- 172.21.15.143:0/1976070191 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f78140036a0 con 0x7f78280c6d40 2023-11-08T19:30:12.302 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.286+0000 7f78257fa700 1 --2- 172.21.15.143:0/1976070191 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f780c06af00 0x7f780c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:12.302 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.286+0000 7f78257fa700 1 -- 172.21.15.143:0/1976070191 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 7059+0+0 (secure 0 0 0) 0x7f7814089c60 con 0x7f78280c6d40 2023-11-08T19:30:12.302 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.287+0000 7f782cfbb700 1 --2- 172.21.15.143:0/1976070191 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f780c06af00 0x7f780c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:12.303 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.287+0000 7f782cfbb700 1 --2- 172.21.15.143:0/1976070191 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f780c06af00 0x7f780c06d3c0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f781c000ef0 tx=0x7f781c00a000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:12.303 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.291+0000 7f78257fa700 1 -- 172.21.15.143:0/1976070191 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f78280c54d0 con 0x7f78280c6d40 2023-11-08T19:30:12.566 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.563+0000 7f782f21f700 1 -- 172.21.15.143:0/1976070191 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "6:0"} v 0) v1 -- 0x7f78280c5690 con 0x7f78280c6d40 2023-11-08T19:30:12.567 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.565+0000 7f78257fa700 1 -- 172.21.15.143:0/1976070191 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "6:0"}]=0 failed mds gid 7386 v83) v1 ==== 97+0+0 (secure 0 0 0) 0x7f7814054850 con 0x7f78280c6d40 2023-11-08T19:30:12.570 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.567+0000 7f782f21f700 1 -- 172.21.15.143:0/1976070191 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f780c06af00 msgr2=0x7f780c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:12.570 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.567+0000 7f782f21f700 1 --2- 172.21.15.143:0/1976070191 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f780c06af00 0x7f780c06d3c0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f781c000ef0 tx=0x7f781c00a000 comp rx=0 tx=0).stop 2023-11-08T19:30:12.571 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.567+0000 7f782f21f700 1 -- 172.21.15.143:0/1976070191 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f78280c6d40 msgr2=0x7f782814fcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:12.571 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.567+0000 7f782f21f700 1 --2- 172.21.15.143:0/1976070191 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f78280c6d40 0x7f782814fcb0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f7814010850 tx=0x7f7814010b60 comp rx=0 tx=0).stop 2023-11-08T19:30:12.571 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.568+0000 7f782f21f700 1 -- 172.21.15.143:0/1976070191 shutdown_connections 2023-11-08T19:30:12.571 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.568+0000 7f782f21f700 1 --2- 172.21.15.143:0/1976070191 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f780c06af00 0x7f780c06d3c0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:12.572 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.568+0000 7f782f21f700 1 --2- 172.21.15.143:0/1976070191 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f782815d190 0x7f782815d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:12.572 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.568+0000 7f782f21f700 1 --2- 172.21.15.143:0/1976070191 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7818003a00 0x7f78280b5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:12.572 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.568+0000 7f782f21f700 1 --2- 172.21.15.143:0/1976070191 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f78280c6d40 0x7f782814fcb0 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:12.572 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.568+0000 7f782f21f700 1 -- 172.21.15.143:0/1976070191 >> 172.21.15.143:0/1976070191 conn(0x7f7828099ab0 msgr2=0x7f782806f3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:12.573 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.568+0000 7f782f21f700 1 -- 172.21.15.143:0/1976070191 shutdown_connections 2023-11-08T19:30:12.573 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.568+0000 7f782f21f700 1 -- 172.21.15.143:0/1976070191 wait complete. 2023-11-08T19:30:12.573 INFO:teuthology.orchestra.run.smithi143.stderr:failed mds gid 7386 2023-11-08T19:30:12.585 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-08T19:30:12.586 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-08T19:30:12.586 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-08T19:30:12.587 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T19:30:12.587 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-08T19:30:12.588 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi143 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a' 2023-11-08T19:30:12.588 INFO:tasks.ceph.mds.a:Stopped 2023-11-08T19:30:12.589 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-08T19:30:12.591 INFO:tasks.ceph.mds.a:Started 2023-11-08T19:30:12.591 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:30:12.670 INFO:tasks.ceph.mds.a.smithi143.stdout:starting mds.a at 2023-11-08T19:30:12.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.714+0000 7f1f41158700 1 Processor -- start 2023-11-08T19:30:12.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.714+0000 7f1f41158700 1 -- start start 2023-11-08T19:30:12.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.715+0000 7f1f41158700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1f3c1618f0 0x7f1f3c06ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:12.721 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.715+0000 7f1f41158700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f3c161df0 con 0x7f1f3c06b150 2023-11-08T19:30:12.721 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.715+0000 7f1f41158700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f3c1620d0 con 0x7f1f3c15dd00 2023-11-08T19:30:12.721 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.715+0000 7f1f41158700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f3c06b600 con 0x7f1f3c1618f0 2023-11-08T19:30:12.721 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.715+0000 7f1f3b59e700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f1f3c06b150 0x7f1f3c0cb1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:52242/0 (socket says 172.21.15.143:52242) 2023-11-08T19:30:12.721 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.715+0000 7f1f3b59e700 1 -- 172.21.15.143:0/1912841213 learned_addr learned my addr 172.21.15.143:0/1912841213 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:30:12.722 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.715+0000 7f1f3a59c700 1 --2- 172.21.15.143:0/1912841213 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1f3c1618f0 0x7f1f3c06ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:12.722 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.715+0000 7f1f39d9b700 1 -- 172.21.15.143:0/1912841213 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 236890574 0 0) 0x7f1f3c161df0 con 0x7f1f3c06b150 2023-11-08T19:30:12.722 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.715+0000 7f1f39d9b700 1 -- 172.21.15.143:0/1912841213 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f240035b0 con 0x7f1f3c06b150 2023-11-08T19:30:12.722 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.715+0000 7f1f39d9b700 1 -- 172.21.15.143:0/1912841213 <== mon.1 v1:172.21.15.174:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1754932463 0 0) 0x7f1f3c1620d0 con 0x7f1f3c15dd00 2023-11-08T19:30:12.722 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.715+0000 7f1f39d9b700 1 -- 172.21.15.143:0/1912841213 --> v1:172.21.15.174:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f3c161df0 con 0x7f1f3c15dd00 2023-11-08T19:30:12.723 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.715+0000 7f1f3a59c700 1 -- 172.21.15.143:0/1912841213 >> v1:172.21.15.174:6789/0 conn(0x7f1f3c15dd00 legacy=0x7f1f3c0cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:12.723 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.715+0000 7f1f3a59c700 1 -- 172.21.15.143:0/1912841213 >> v1:172.21.15.143:6789/0 conn(0x7f1f3c06b150 legacy=0x7f1f3c0cb1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:12.723 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.715+0000 7f1f39d9b700 1 -- 172.21.15.143:0/1912841213 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3761575000 0 0) 0x7f1f240035b0 con 0x7f1f3c06b150 2023-11-08T19:30:12.723 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.715+0000 7f1f3a59c700 1 -- 172.21.15.143:0/1912841213 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f3c1623b0 con 0x7f1f3c1618f0 2023-11-08T19:30:12.723 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.716+0000 7f1f3a59c700 1 --2- 172.21.15.143:0/1912841213 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1f3c1618f0 0x7f1f3c06ac10 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1f30009870 tx=0x7f1f30009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b4e4d5f963aebd3 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:12.724 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.716+0000 7f1f39d9b700 1 -- 172.21.15.143:0/1912841213 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f340019f0 con 0x7f1f3c1618f0 2023-11-08T19:30:12.724 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.716+0000 7f1f39d9b700 1 -- 172.21.15.143:0/1912841213 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f30004030 con 0x7f1f3c1618f0 2023-11-08T19:30:12.724 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.716+0000 7f1f39d9b700 1 -- 172.21.15.143:0/1912841213 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f300043a0 con 0x7f1f3c1618f0 2023-11-08T19:30:12.724 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.716+0000 7f1f41158700 1 -- 172.21.15.143:0/1912841213 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1f3c1618f0 msgr2=0x7f1f3c06ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:12.724 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.716+0000 7f1f41158700 1 --2- 172.21.15.143:0/1912841213 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1f3c1618f0 0x7f1f3c06ac10 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1f30009870 tx=0x7f1f30009b80 comp rx=0 tx=0).stop 2023-11-08T19:30:12.724 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.716+0000 7f1f41158700 1 -- 172.21.15.143:0/1912841213 shutdown_connections 2023-11-08T19:30:12.725 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.716+0000 7f1f41158700 1 --2- 172.21.15.143:0/1912841213 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1f3c1618f0 0x7f1f3c06ac10 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:12.725 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.716+0000 7f1f41158700 1 -- 172.21.15.143:0/1912841213 >> 172.21.15.143:0/1912841213 conn(0x7f1f3c099ab0 msgr2=0x7f1f3c0d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:12.725 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.716+0000 7f1f41158700 1 -- 172.21.15.143:0/1912841213 shutdown_connections 2023-11-08T19:30:12.725 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.716+0000 7f1f41158700 1 -- 172.21.15.143:0/1912841213 wait complete. 2023-11-08T19:30:12.725 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.718+0000 7f1f41158700 1 Processor -- start 2023-11-08T19:30:12.726 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.718+0000 7f1f41158700 1 -- start start 2023-11-08T19:30:12.728 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.718+0000 7f1f41158700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1f3c06b150 0x7f1f3c0b0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:12.729 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.718+0000 7f1f41158700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1f3c15dd00 0x7f1f3c1573b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:12.729 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.718+0000 7f1f41158700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1f3c1618f0 0x7f1f3c0af080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:12.729 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.718+0000 7f1f41158700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f3c1578f0 con 0x7f1f3c1618f0 2023-11-08T19:30:12.729 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.718+0000 7f1f41158700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f3c0b10e0 con 0x7f1f3c15dd00 2023-11-08T19:30:12.730 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.718+0000 7f1f41158700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f3c0b13c0 con 0x7f1f3c06b150 2023-11-08T19:30:12.730 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.719+0000 7f1f3b59e700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1f3c1618f0 0x7f1f3c0af080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:12.730 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.719+0000 7f1f3b59e700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1f3c1618f0 0x7f1f3c0af080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56094/0 (socket says 172.21.15.143:56094) 2023-11-08T19:30:12.730 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.719+0000 7f1f3b59e700 1 -- 172.21.15.143:0/1600185933 learned_addr learned my addr 172.21.15.143:0/1600185933 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:30:12.730 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.719+0000 7f1f3ad9d700 1 --2- 172.21.15.143:0/1600185933 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1f3c06b150 0x7f1f3c0b0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:12.730 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.719+0000 7f1f3a59c700 1 --2- 172.21.15.143:0/1600185933 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1f3c15dd00 0x7f1f3c1573b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:12.731 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.719+0000 7f1f3b59e700 1 -- 172.21.15.143:0/1600185933 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1f3c06b150 msgr2=0x7f1f3c0b0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:12.731 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.719+0000 7f1f3b59e700 1 --2- 172.21.15.143:0/1600185933 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1f3c06b150 0x7f1f3c0b0aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:12.731 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.719+0000 7f1f3b59e700 1 -- 172.21.15.143:0/1600185933 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1f3c15dd00 msgr2=0x7f1f3c1573b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:12.731 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.719+0000 7f1f3b59e700 1 --2- 172.21.15.143:0/1600185933 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1f3c15dd00 0x7f1f3c1573b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:12.731 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.719+0000 7f1f3b59e700 1 -- 172.21.15.143:0/1600185933 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f300094a0 con 0x7f1f3c1618f0 2023-11-08T19:30:12.732 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.719+0000 7f1f3b59e700 1 --2- 172.21.15.143:0/1600185933 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1f3c1618f0 0x7f1f3c0af080 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f1f3400cc50 tx=0x7f1f3400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:12.732 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.719+0000 7f1f237fe700 1 -- 172.21.15.143:0/1600185933 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f3400ade0 con 0x7f1f3c1618f0 2023-11-08T19:30:12.732 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.719+0000 7f1f237fe700 1 -- 172.21.15.143:0/1600185933 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f3400ce50 con 0x7f1f3c1618f0 2023-11-08T19:30:12.732 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.719+0000 7f1f41158700 1 -- 172.21.15.143:0/1600185933 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f3c15e260 con 0x7f1f3c1618f0 2023-11-08T19:30:12.732 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.719+0000 7f1f237fe700 1 -- 172.21.15.143:0/1600185933 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f34007310 con 0x7f1f3c1618f0 2023-11-08T19:30:12.733 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.719+0000 7f1f41158700 1 -- 172.21.15.143:0/1600185933 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f3c0b1980 con 0x7f1f3c1618f0 2023-11-08T19:30:12.733 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.720+0000 7f1f41158700 1 -- 172.21.15.143:0/1600185933 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f3c0b1fd0 con 0x7f1f3c1618f0 2023-11-08T19:30:12.733 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.721+0000 7f1f237fe700 1 -- 172.21.15.143:0/1600185933 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1f34007470 con 0x7f1f3c1618f0 2023-11-08T19:30:12.733 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.721+0000 7f1f237fe700 1 --2- 172.21.15.143:0/1600185933 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f1f2806af00 0x7f1f2806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:12.733 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.721+0000 7f1f237fe700 1 -- 172.21.15.143:0/1600185933 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 7145+0+0 (secure 0 0 0) 0x7f1f340897b0 con 0x7f1f3c1618f0 2023-11-08T19:30:12.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.721+0000 7f1f3ad9d700 1 --2- 172.21.15.143:0/1600185933 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f1f2806af00 0x7f1f2806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:12.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.722+0000 7f1f3ad9d700 1 --2- 172.21.15.143:0/1600185933 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f1f2806af00 0x7f1f2806d3c0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c001230 tx=0x7f1f2c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:12.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.725+0000 7f1f237fe700 1 -- 172.21.15.143:0/1600185933 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1f340542c0 con 0x7f1f3c1618f0 2023-11-08T19:30:12.958 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.955+0000 7f1f41158700 1 -- 172.21.15.143:0/1600185933 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1f3c04eee0 con 0x7f1f3c1618f0 2023-11-08T19:30:12.958 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.956+0000 7f1f237fe700 1 -- 172.21.15.143:0/1600185933 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 84 v84) v1 ==== 94+0+4148 (secure 0 0 0) 0x7f1f340578e0 con 0x7f1f3c1618f0 2023-11-08T19:30:12.959 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:30:12.959 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":84,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5933,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/3918286710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6835","nonce":3918286710},{"type":"v1","addr":"172.21.15.174:6837","nonce":3918286710}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:03.092245+0000","modified":"2023-11-08T19:30:12.572467+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":98,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7494,"mds_1":6918},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6918":{"gid":6918,"name":"b","rank":1,"incarnation":54,"state":"up:active","state_seq":6,"addr":"172.21.15.174:6836/3601088618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3601088618},{"type":"v1","addr":"172.21.15.174:6836","nonce":3601088618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7494":{"gid":7494,"name":"c","rank":0,"incarnation":84,"state":"up:replay","state_seq":1,"addr":"172.21.15.143:6834/1535103237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":1535103237},{"type":"v1","addr":"172.21.15.143:6834","nonce":1535103237}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:30:12.960 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.958+0000 7f1f41158700 1 -- 172.21.15.143:0/1600185933 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f1f2806af00 msgr2=0x7f1f2806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:12.961 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.958+0000 7f1f41158700 1 --2- 172.21.15.143:0/1600185933 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f1f2806af00 0x7f1f2806d3c0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c001230 tx=0x7f1f2c009040 comp rx=0 tx=0).stop 2023-11-08T19:30:12.961 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.958+0000 7f1f41158700 1 -- 172.21.15.143:0/1600185933 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1f3c1618f0 msgr2=0x7f1f3c0af080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:12.961 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.958+0000 7f1f41158700 1 --2- 172.21.15.143:0/1600185933 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1f3c1618f0 0x7f1f3c0af080 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f1f3400cc50 tx=0x7f1f3400a340 comp rx=0 tx=0).stop 2023-11-08T19:30:12.961 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.958+0000 7f1f41158700 1 -- 172.21.15.143:0/1600185933 shutdown_connections 2023-11-08T19:30:12.961 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.958+0000 7f1f41158700 1 --2- 172.21.15.143:0/1600185933 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f1f2806af00 0x7f1f2806d3c0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:12.962 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.958+0000 7f1f41158700 1 --2- 172.21.15.143:0/1600185933 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1f3c06b150 0x7f1f3c0b0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:12.962 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.958+0000 7f1f41158700 1 --2- 172.21.15.143:0/1600185933 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1f3c1618f0 0x7f1f3c0af080 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:12.962 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.958+0000 7f1f41158700 1 --2- 172.21.15.143:0/1600185933 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1f3c15dd00 0x7f1f3c1573b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:12.962 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.958+0000 7f1f41158700 1 -- 172.21.15.143:0/1600185933 >> 172.21.15.143:0/1600185933 conn(0x7f1f3c099ab0 msgr2=0x7f1f3c0cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:12.962 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.958+0000 7f1f41158700 1 -- 172.21.15.143:0/1600185933 shutdown_connections 2023-11-08T19:30:12.963 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:12.958+0000 7f1f41158700 1 -- 172.21.15.143:0/1600185933 wait complete. 2023-11-08T19:30:12.963 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 84 2023-11-08T19:30:12.972 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=30 retry_count=0)... 2023-11-08T19:30:16.730 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a has been restored 2023-11-08T19:30:17.170 INFO:tasks.ceph.mds.c.smithi143.stderr:2023-11-08T19:30:17.168+0000 7f2b9e2d4700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-08T19:30:17.170 INFO:tasks.ceph.mds.c.smithi143.stderr:2023-11-08T19:30:17.168+0000 7f2b9e2d4700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-11-08T19:30:17.974 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:30:18.107 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.106+0000 7f7f79fbf700 1 Processor -- start 2023-11-08T19:30:18.108 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.106+0000 7f7f79fbf700 1 -- start start 2023-11-08T19:30:18.113 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.107+0000 7f7f79fbf700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f7f7415ba30 0x7f7f740b14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:18.113 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.107+0000 7f7f79fbf700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7f740d1c70 0x7f7f740cc5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:18.113 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.107+0000 7f7f79fbf700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f740b1bc0 con 0x7f7f740d03a0 2023-11-08T19:30:18.113 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.107+0000 7f7f79fbf700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f740b1ea0 con 0x7f7f7415ba30 2023-11-08T19:30:18.114 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.107+0000 7f7f79fbf700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f740b2180 con 0x7f7f740d1c70 2023-11-08T19:30:18.114 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.107+0000 7f7f737fe700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f7f7415ba30 0x7f7f740b14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:18.114 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.107+0000 7f7f737fe700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f7f7415ba30 0x7f7f740b14c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56106/0 (socket says 172.21.15.143:56106) 2023-11-08T19:30:18.115 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.107+0000 7f7f737fe700 1 -- 172.21.15.143:0/2117574455 learned_addr learned my addr 172.21.15.143:0/2117574455 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:30:18.115 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.107+0000 7f7f73fff700 1 --2- 172.21.15.143:0/2117574455 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7f740d1c70 0x7f7f740cc5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:18.115 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.107+0000 7f7f737fe700 1 -- 172.21.15.143:0/2117574455 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7f740d1c70 msgr2=0x7f7f740cc5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:18.115 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.107+0000 7f7f737fe700 1 --2- 172.21.15.143:0/2117574455 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7f740d1c70 0x7f7f740cc5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:18.115 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.107+0000 7f7f737fe700 1 -- 172.21.15.143:0/2117574455 >> v1:172.21.15.174:6789/0 conn(0x7f7f740d03a0 legacy=0x7f7f740d1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:18.116 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.107+0000 7f7f737fe700 1 -- 172.21.15.143:0/2117574455 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f740b2460 con 0x7f7f7415ba30 2023-11-08T19:30:18.116 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.107+0000 7f7f73fff700 1 --2- 172.21.15.143:0/2117574455 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7f740d1c70 0x7f7f740cc5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:30:18.116 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.107+0000 7f7f737fe700 1 --2- 172.21.15.143:0/2117574455 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f7415ba30 0x7f7f740b14c0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f7f680098f0 tx=0x7f7f68009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92d01386ea261c29 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:18.116 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.108+0000 7f7f727fc700 1 -- 172.21.15.143:0/2117574455 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f68004230 con 0x7f7f7415ba30 2023-11-08T19:30:18.117 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.108+0000 7f7f727fc700 1 -- 172.21.15.143:0/2117574455 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f68004390 con 0x7f7f7415ba30 2023-11-08T19:30:18.117 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.108+0000 7f7f727fc700 1 -- 172.21.15.143:0/2117574455 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f680036a0 con 0x7f7f7415ba30 2023-11-08T19:30:18.117 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.108+0000 7f7f79fbf700 1 -- 172.21.15.143:0/2117574455 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f7415ba30 msgr2=0x7f7f740b14c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:18.117 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.108+0000 7f7f79fbf700 1 --2- 172.21.15.143:0/2117574455 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f7415ba30 0x7f7f740b14c0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f7f680098f0 tx=0x7f7f68009c00 comp rx=0 tx=0).stop 2023-11-08T19:30:18.117 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.108+0000 7f7f79fbf700 1 -- 172.21.15.143:0/2117574455 shutdown_connections 2023-11-08T19:30:18.118 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.108+0000 7f7f79fbf700 1 --2- 172.21.15.143:0/2117574455 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7f740d1c70 0x7f7f740cc5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:18.118 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.108+0000 7f7f79fbf700 1 --2- 172.21.15.143:0/2117574455 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f7415ba30 0x7f7f740b14c0 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:18.118 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.108+0000 7f7f79fbf700 1 -- 172.21.15.143:0/2117574455 >> 172.21.15.143:0/2117574455 conn(0x7f7f7408b000 msgr2=0x7f7f7409ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:18.118 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.108+0000 7f7f79fbf700 1 -- 172.21.15.143:0/2117574455 shutdown_connections 2023-11-08T19:30:18.118 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.108+0000 7f7f79fbf700 1 -- 172.21.15.143:0/2117574455 wait complete. 2023-11-08T19:30:18.119 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.109+0000 7f7f79fbf700 1 Processor -- start 2023-11-08T19:30:18.119 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.110+0000 7f7f79fbf700 1 -- start start 2023-11-08T19:30:18.119 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.110+0000 7f7f79fbf700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f740d03a0 0x7f7f74157730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:18.119 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.110+0000 7f7f79fbf700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7f740d1c70 0x7f7f740c7600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:18.119 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.110+0000 7f7f79fbf700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7f7415ba30 0x7f7f7415c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:18.120 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.110+0000 7f7f79fbf700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f740c7170 con 0x7f7f740d03a0 2023-11-08T19:30:18.120 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.110+0000 7f7f79fbf700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f740c6560 con 0x7f7f7415ba30 2023-11-08T19:30:18.120 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.110+0000 7f7f79fbf700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f740c6840 con 0x7f7f740d1c70 2023-11-08T19:30:18.120 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.110+0000 7f7f737fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f740d03a0 0x7f7f74157730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:18.120 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.110+0000 7f7f737fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f740d03a0 0x7f7f74157730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56112/0 (socket says 172.21.15.143:56112) 2023-11-08T19:30:18.121 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.111+0000 7f7f737fe700 1 -- 172.21.15.143:0/17538631 learned_addr learned my addr 172.21.15.143:0/17538631 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:30:18.121 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.111+0000 7f7f737fe700 1 -- 172.21.15.143:0/17538631 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7f740d1c70 msgr2=0x7f7f740c7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:18.121 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.111+0000 7f7f737fe700 1 --2- 172.21.15.143:0/17538631 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7f740d1c70 0x7f7f740c7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:18.121 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.111+0000 7f7f737fe700 1 -- 172.21.15.143:0/17538631 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7f7415ba30 msgr2=0x7f7f7415c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:18.121 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.111+0000 7f7f737fe700 1 --2- 172.21.15.143:0/17538631 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7f7415ba30 0x7f7f7415c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:18.122 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.111+0000 7f7f737fe700 1 -- 172.21.15.143:0/17538631 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f68009550 con 0x7f7f740d03a0 2023-11-08T19:30:18.122 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.111+0000 7f7f737fe700 1 --2- 172.21.15.143:0/17538631 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f740d03a0 0x7f7f74157730 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f7f68009130 tx=0x7f7f68004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:18.122 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.111+0000 7f7f5bfff700 1 -- 172.21.15.143:0/17538631 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f6800b860 con 0x7f7f740d03a0 2023-11-08T19:30:18.122 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.111+0000 7f7f5bfff700 1 -- 172.21.15.143:0/17538631 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f6800b9c0 con 0x7f7f740d03a0 2023-11-08T19:30:18.123 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.111+0000 7f7f79fbf700 1 -- 172.21.15.143:0/17538631 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f740c6b20 con 0x7f7f740d03a0 2023-11-08T19:30:18.123 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.111+0000 7f7f5bfff700 1 -- 172.21.15.143:0/17538631 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f6800bd70 con 0x7f7f740d03a0 2023-11-08T19:30:18.123 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.111+0000 7f7f79fbf700 1 -- 172.21.15.143:0/17538631 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f74157c70 con 0x7f7f740d03a0 2023-11-08T19:30:18.125 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.112+0000 7f7f79fbf700 1 -- 172.21.15.143:0/17538631 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f740d0800 con 0x7f7f740d03a0 2023-11-08T19:30:18.125 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.113+0000 7f7f5bfff700 1 -- 172.21.15.143:0/17538631 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7f6801b440 con 0x7f7f740d03a0 2023-11-08T19:30:18.125 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.113+0000 7f7f5bfff700 1 --2- 172.21.15.143:0/17538631 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f7f6006af00 0x7f7f6006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:18.125 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.113+0000 7f7f5bfff700 1 -- 172.21.15.143:0/17538631 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 7145+0+0 (secure 0 0 0) 0x7f7f6808b530 con 0x7f7f740d03a0 2023-11-08T19:30:18.126 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.113+0000 7f7f72ffd700 1 --2- 172.21.15.143:0/17538631 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f7f6006af00 0x7f7f6006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:18.126 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.114+0000 7f7f72ffd700 1 --2- 172.21.15.143:0/17538631 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f7f6006af00 0x7f7f6006d3c0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f7f7406bfb0 tx=0x7f7f64007a40 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:18.126 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.117+0000 7f7f5bfff700 1 -- 172.21.15.143:0/17538631 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7f740d0800 con 0x7f7f740d03a0 2023-11-08T19:30:18.349 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.348+0000 7f7f79fbf700 1 -- 172.21.15.143:0/17538631 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7f740d0800 con 0x7f7f740d03a0 2023-11-08T19:30:18.350 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.348+0000 7f7f5bfff700 1 -- 172.21.15.143:0/17538631 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 88 v88) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f7f680560d0 con 0x7f7f740d03a0 2023-11-08T19:30:18.350 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:30:18.350 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":88,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5933,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/3918286710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6835","nonce":3918286710},{"type":"v1","addr":"172.21.15.174:6837","nonce":3918286710}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":6161,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/2129643800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":2129643800},{"type":"v1","addr":"172.21.15.143:6835","nonce":2129643800}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":88,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:03.092245+0000","modified":"2023-11-08T19:30:15.587914+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":98,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7494,"mds_1":6918},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6918":{"gid":6918,"name":"b","rank":1,"incarnation":54,"state":"up:active","state_seq":6,"addr":"172.21.15.174:6836/3601088618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3601088618},{"type":"v1","addr":"172.21.15.174:6836","nonce":3601088618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7494":{"gid":7494,"name":"c","rank":0,"incarnation":84,"state":"up:active","state_seq":12,"addr":"172.21.15.143:6834/1535103237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":1535103237},{"type":"v1","addr":"172.21.15.143:6834","nonce":1535103237}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:30:18.352 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.350+0000 7f7f79fbf700 1 -- 172.21.15.143:0/17538631 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f7f6006af00 msgr2=0x7f7f6006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:18.352 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.350+0000 7f7f79fbf700 1 --2- 172.21.15.143:0/17538631 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f7f6006af00 0x7f7f6006d3c0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f7f7406bfb0 tx=0x7f7f64007a40 comp rx=0 tx=0).stop 2023-11-08T19:30:18.352 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.350+0000 7f7f79fbf700 1 -- 172.21.15.143:0/17538631 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f740d03a0 msgr2=0x7f7f74157730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:18.353 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.350+0000 7f7f79fbf700 1 --2- 172.21.15.143:0/17538631 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f740d03a0 0x7f7f74157730 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f7f68009130 tx=0x7f7f68004750 comp rx=0 tx=0).stop 2023-11-08T19:30:18.353 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.350+0000 7f7f79fbf700 1 -- 172.21.15.143:0/17538631 shutdown_connections 2023-11-08T19:30:18.353 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.350+0000 7f7f79fbf700 1 --2- 172.21.15.143:0/17538631 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f7f6006af00 0x7f7f6006d3c0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:18.353 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.350+0000 7f7f79fbf700 1 --2- 172.21.15.143:0/17538631 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7f7415ba30 0x7f7f7415c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:18.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.350+0000 7f7f79fbf700 1 --2- 172.21.15.143:0/17538631 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f740d03a0 0x7f7f74157730 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:18.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.350+0000 7f7f79fbf700 1 --2- 172.21.15.143:0/17538631 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7f740d1c70 0x7f7f740c7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:18.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.350+0000 7f7f79fbf700 1 -- 172.21.15.143:0/17538631 >> 172.21.15.143:0/17538631 conn(0x7f7f7408b000 msgr2=0x7f7f740ceb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:18.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.350+0000 7f7f79fbf700 1 -- 172.21.15.143:0/17538631 shutdown_connections 2023-11-08T19:30:18.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.350+0000 7f7f79fbf700 1 -- 172.21.15.143:0/17538631 wait complete. 2023-11-08T19:30:18.355 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 88 2023-11-08T19:30:18.364 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2023-11-08T19:30:18.364 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:30:18.484 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.483+0000 7fa107b12700 1 Processor -- start 2023-11-08T19:30:18.484 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.483+0000 7fa107b12700 1 -- start start 2023-11-08T19:30:18.486 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.484+0000 7fa107b12700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa1000cc530 0x7fa1000cf510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:18.487 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.484+0000 7fa107b12700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1000cf080 con 0x7fa10015aa30 2023-11-08T19:30:18.487 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.484+0000 7fa107b12700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1000ce470 con 0x7fa1000b2920 2023-11-08T19:30:18.487 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.484+0000 7fa107b12700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1000ce750 con 0x7fa1000cc530 2023-11-08T19:30:18.487 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.484+0000 7fa1058ae700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fa10015aa30 0x7fa1000b2210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:52274/0 (socket says 172.21.15.143:52274) 2023-11-08T19:30:18.488 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.484+0000 7fa1058ae700 1 -- 172.21.15.143:0/2935827788 learned_addr learned my addr 172.21.15.143:0/2935827788 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:30:18.488 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.484+0000 7fa1060af700 1 --2- 172.21.15.143:0/2935827788 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa1000cc530 0x7fa1000cf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:18.488 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.484+0000 7fa1048ac700 1 -- 172.21.15.143:0/2935827788 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2343042385 0 0) 0x7fa1000cf080 con 0x7fa10015aa30 2023-11-08T19:30:18.488 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.484+0000 7fa1048ac700 1 -- 172.21.15.143:0/2935827788 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa0ec0035b0 con 0x7fa10015aa30 2023-11-08T19:30:18.488 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.484+0000 7fa1048ac700 1 -- 172.21.15.143:0/2935827788 <== mon.1 v1:172.21.15.174:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1363833641 0 0) 0x7fa1000ce470 con 0x7fa1000b2920 2023-11-08T19:30:18.489 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.484+0000 7fa1048ac700 1 -- 172.21.15.143:0/2935827788 --> v1:172.21.15.174:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa1000cf080 con 0x7fa1000b2920 2023-11-08T19:30:18.489 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.484+0000 7fa1048ac700 1 -- 172.21.15.143:0/2935827788 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3971097638 0 0) 0x7fa0ec0035b0 con 0x7fa10015aa30 2023-11-08T19:30:18.489 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.484+0000 7fa1060af700 1 -- 172.21.15.143:0/2935827788 >> v1:172.21.15.174:6789/0 conn(0x7fa1000b2920 legacy=0x7fa1000cbe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:18.489 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.484+0000 7fa1060af700 1 -- 172.21.15.143:0/2935827788 >> v1:172.21.15.143:6789/0 conn(0x7fa10015aa30 legacy=0x7fa1000b2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:18.489 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.484+0000 7fa1060af700 1 -- 172.21.15.143:0/2935827788 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1000cea30 con 0x7fa1000cc530 2023-11-08T19:30:18.489 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.484+0000 7fa1060af700 1 --2- 172.21.15.143:0/2935827788 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa1000cc530 0x7fa1000cf510 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa0f8009dc0 tx=0x7fa0f800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6afd81922696e031 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:18.490 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.485+0000 7fa1048ac700 1 -- 172.21.15.143:0/2935827788 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0f800e070 con 0x7fa1000cc530 2023-11-08T19:30:18.490 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.485+0000 7fa1048ac700 1 -- 172.21.15.143:0/2935827788 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0f8003a20 con 0x7fa1000cc530 2023-11-08T19:30:18.490 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.485+0000 7fa1048ac700 1 -- 172.21.15.143:0/2935827788 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0f00017f0 con 0x7fa1000cc530 2023-11-08T19:30:18.490 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.485+0000 7fa107b12700 1 -- 172.21.15.143:0/2935827788 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa1000cc530 msgr2=0x7fa1000cf510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:18.490 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.485+0000 7fa107b12700 1 --2- 172.21.15.143:0/2935827788 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa1000cc530 0x7fa1000cf510 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa0f8009dc0 tx=0x7fa0f800b330 comp rx=0 tx=0).stop 2023-11-08T19:30:18.491 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.485+0000 7fa107b12700 1 -- 172.21.15.143:0/2935827788 shutdown_connections 2023-11-08T19:30:18.491 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.485+0000 7fa107b12700 1 --2- 172.21.15.143:0/2935827788 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa1000cc530 0x7fa1000cf510 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:18.491 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.485+0000 7fa107b12700 1 -- 172.21.15.143:0/2935827788 >> 172.21.15.143:0/2935827788 conn(0x7fa100099ab0 msgr2=0x7fa1000d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:18.491 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.485+0000 7fa107b12700 1 -- 172.21.15.143:0/2935827788 shutdown_connections 2023-11-08T19:30:18.491 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.485+0000 7fa107b12700 1 -- 172.21.15.143:0/2935827788 wait complete. 2023-11-08T19:30:18.494 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.486+0000 7fa107b12700 1 Processor -- start 2023-11-08T19:30:18.494 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.486+0000 7fa107b12700 1 -- start start 2023-11-08T19:30:18.494 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.487+0000 7fa107b12700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa1000b2920 0x7fa1001ec980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:18.495 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.487+0000 7fa107b12700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa1000cc530 0x7fa1000ae890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:18.495 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.487+0000 7fa107b12700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa10015aa30 0x7fa1000aedd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:18.495 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.487+0000 7fa107b12700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1000c5ed0 con 0x7fa1000cc530 2023-11-08T19:30:18.495 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.487+0000 7fa107b12700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1001ef470 con 0x7fa10015aa30 2023-11-08T19:30:18.495 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.487+0000 7fa107b12700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1001ecec0 con 0x7fa1000b2920 2023-11-08T19:30:18.496 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.487+0000 7fa1050ad700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa1000cc530 0x7fa1000ae890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:18.496 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.487+0000 7fa1050ad700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa1000cc530 0x7fa1000ae890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56126/0 (socket says 172.21.15.143:56126) 2023-11-08T19:30:18.496 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.487+0000 7fa1050ad700 1 -- 172.21.15.143:0/701780201 learned_addr learned my addr 172.21.15.143:0/701780201 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:30:18.496 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.487+0000 7fa1058ae700 1 --2- 172.21.15.143:0/701780201 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa1000b2920 0x7fa1001ec980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:18.496 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.487+0000 7fa1060af700 1 --2- 172.21.15.143:0/701780201 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa10015aa30 0x7fa1000aedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:18.497 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.487+0000 7fa1050ad700 1 -- 172.21.15.143:0/701780201 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa1000b2920 msgr2=0x7fa1001ec980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:18.497 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.487+0000 7fa1050ad700 1 --2- 172.21.15.143:0/701780201 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa1000b2920 0x7fa1001ec980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:18.497 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.487+0000 7fa1050ad700 1 -- 172.21.15.143:0/701780201 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa10015aa30 msgr2=0x7fa1000aedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:18.497 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.487+0000 7fa1050ad700 1 --2- 172.21.15.143:0/701780201 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa10015aa30 0x7fa1000aedd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:18.497 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.487+0000 7fa1050ad700 1 -- 172.21.15.143:0/701780201 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0f8009a70 con 0x7fa1000cc530 2023-11-08T19:30:18.497 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.488+0000 7fa1050ad700 1 --2- 172.21.15.143:0/701780201 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa1000cc530 0x7fa1000ae890 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc00c550 tx=0x7fa0fc00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:18.498 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.488+0000 7fa0f67fc700 1 -- 172.21.15.143:0/701780201 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0fc0052d0 con 0x7fa1000cc530 2023-11-08T19:30:18.498 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.488+0000 7fa0f67fc700 1 -- 172.21.15.143:0/701780201 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0fc005430 con 0x7fa1000cc530 2023-11-08T19:30:18.498 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.488+0000 7fa107b12700 1 -- 172.21.15.143:0/701780201 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1000c9180 con 0x7fa1000cc530 2023-11-08T19:30:18.498 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.488+0000 7fa0f67fc700 1 -- 172.21.15.143:0/701780201 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0fc0038c0 con 0x7fa1000cc530 2023-11-08T19:30:18.498 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.488+0000 7fa107b12700 1 -- 172.21.15.143:0/701780201 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1001ef680 con 0x7fa1000cc530 2023-11-08T19:30:18.499 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.489+0000 7fa107b12700 1 -- 172.21.15.143:0/701780201 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa10004eee0 con 0x7fa1000cc530 2023-11-08T19:30:18.499 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.489+0000 7fa0f67fc700 1 -- 172.21.15.143:0/701780201 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa0fc003a20 con 0x7fa1000cc530 2023-11-08T19:30:18.499 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.489+0000 7fa0f67fc700 1 --2- 172.21.15.143:0/701780201 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa0e406af00 0x7fa0e406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:18.499 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.490+0000 7fa0f67fc700 1 -- 172.21.15.143:0/701780201 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 7145+0+0 (secure 0 0 0) 0x7fa0fc0890e0 con 0x7fa1000cc530 2023-11-08T19:30:18.499 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.490+0000 7fa1058ae700 1 --2- 172.21.15.143:0/701780201 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa0e406af00 0x7fa0e406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:18.500 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.490+0000 7fa1058ae700 1 --2- 172.21.15.143:0/701780201 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa0e406af00 0x7fa0e406d3c0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fa0f0001360 tx=0x7fa0f0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:18.500 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.494+0000 7fa0f67fc700 1 -- 172.21.15.143:0/701780201 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa0fc053bf0 con 0x7fa1000cc530 2023-11-08T19:30:18.730 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.729+0000 7fa107b12700 1 -- 172.21.15.143:0/701780201 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa10015ae10 con 0x7fa1000cc530 2023-11-08T19:30:18.731 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.729+0000 7fa0f67fc700 1 -- 172.21.15.143:0/701780201 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 88 v88) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fa0fc057210 con 0x7fa1000cc530 2023-11-08T19:30:18.731 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:30:18.731 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":88,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5933,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/3918286710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6835","nonce":3918286710},{"type":"v1","addr":"172.21.15.174:6837","nonce":3918286710}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":6161,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/2129643800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":2129643800},{"type":"v1","addr":"172.21.15.143:6835","nonce":2129643800}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":88,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:03.092245+0000","modified":"2023-11-08T19:30:15.587914+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":98,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7494,"mds_1":6918},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6918":{"gid":6918,"name":"b","rank":1,"incarnation":54,"state":"up:active","state_seq":6,"addr":"172.21.15.174:6836/3601088618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3601088618},{"type":"v1","addr":"172.21.15.174:6836","nonce":3601088618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7494":{"gid":7494,"name":"c","rank":0,"incarnation":84,"state":"up:active","state_seq":12,"addr":"172.21.15.143:6834/1535103237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":1535103237},{"type":"v1","addr":"172.21.15.143:6834","nonce":1535103237}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:30:18.733 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.732+0000 7fa107b12700 1 -- 172.21.15.143:0/701780201 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa0e406af00 msgr2=0x7fa0e406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:18.733 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.732+0000 7fa107b12700 1 --2- 172.21.15.143:0/701780201 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa0e406af00 0x7fa0e406d3c0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fa0f0001360 tx=0x7fa0f0009040 comp rx=0 tx=0).stop 2023-11-08T19:30:18.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.732+0000 7fa107b12700 1 -- 172.21.15.143:0/701780201 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa1000cc530 msgr2=0x7fa1000ae890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:18.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.732+0000 7fa107b12700 1 --2- 172.21.15.143:0/701780201 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa1000cc530 0x7fa1000ae890 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc00c550 tx=0x7fa0fc00c910 comp rx=0 tx=0).stop 2023-11-08T19:30:18.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.732+0000 7fa107b12700 1 -- 172.21.15.143:0/701780201 shutdown_connections 2023-11-08T19:30:18.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.732+0000 7fa107b12700 1 --2- 172.21.15.143:0/701780201 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa0e406af00 0x7fa0e406d3c0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:18.735 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.732+0000 7fa107b12700 1 --2- 172.21.15.143:0/701780201 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa1000b2920 0x7fa1001ec980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:18.735 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.732+0000 7fa107b12700 1 --2- 172.21.15.143:0/701780201 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa10015aa30 0x7fa1000aedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:18.735 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.732+0000 7fa107b12700 1 --2- 172.21.15.143:0/701780201 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa1000cc530 0x7fa1000ae890 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:18.735 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.732+0000 7fa107b12700 1 -- 172.21.15.143:0/701780201 >> 172.21.15.143:0/701780201 conn(0x7fa100099ab0 msgr2=0x7fa1000b0a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:18.735 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.732+0000 7fa107b12700 1 -- 172.21.15.143:0/701780201 shutdown_connections 2023-11-08T19:30:18.735 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:18.732+0000 7fa107b12700 1 -- 172.21.15.143:0/701780201 wait complete. 2023-11-08T19:30:18.736 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 88 2023-11-08T19:30:19.148 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 88, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 2, 'explicitly_allowed_features': 2, 'created': '2023-11-08T19:28:03.092245+0000', 'modified': '2023-11-08T19:30:15.587914+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 98, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 2, 'in': [0, 1], 'up': {'mds_0': 7494, 'mds_1': 6918}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6918': {'gid': 6918, 'name': 'b', 'rank': 1, 'incarnation': 54, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.174:6836/3601088618', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.174:6834', 'nonce': 3601088618}, {'type': 'v1', 'addr': '172.21.15.174:6836', 'nonce': 3601088618}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_7494': {'gid': 7494, 'name': 'c', 'rank': 0, 'incarnation': 84, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.143:6834/1535103237', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.143:6832', 'nonce': 1535103237}, {'type': 'v1', 'addr': '172.21.15.143:6834', 'nonce': 1535103237}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-08T19:30:19.148 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 2/2 2023-11-08T19:30:19.149 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-11-08T19:30:19.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.272+0000 7f07a72cb700 1 Processor -- start 2023-11-08T19:30:19.274 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.272+0000 7f07a72cb700 1 -- start start 2023-11-08T19:30:19.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.273+0000 7f07a72cb700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f07a006c730 0x7f07a00cd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:19.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.273+0000 7f07a72cb700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07a00cd480 con 0x7f07a015de80 2023-11-08T19:30:19.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.273+0000 7f07a72cb700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07a00cc610 con 0x7f07a006c030 2023-11-08T19:30:19.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.273+0000 7f07a72cb700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07a00cc870 con 0x7f07a006c730 2023-11-08T19:30:19.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.273+0000 7f07a5868700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f07a015de80 0x7f07a00cbf00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:52290/0 (socket says 172.21.15.143:52290) 2023-11-08T19:30:19.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.273+0000 7f07a5868700 1 -- 172.21.15.143:0/3582568339 learned_addr learned my addr 172.21.15.143:0/3582568339 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:30:19.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.273+0000 7f0797fff700 1 -- 172.21.15.143:0/3582568339 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3199718081 0 0) 0x7f07a00cd480 con 0x7f07a015de80 2023-11-08T19:30:19.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.273+0000 7f0797fff700 1 -- 172.21.15.143:0/3582568339 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f078c0035b0 con 0x7f07a015de80 2023-11-08T19:30:19.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.273+0000 7f07a5067700 1 --2- 172.21.15.143:0/3582568339 >> v2:172.21.15.174:3300/0 conn(0x7f07a006c730 0x7f07a00cd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:19.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.274+0000 7f0797fff700 1 -- 172.21.15.143:0/3582568339 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3234464473 0 0) 0x7f078c0035b0 con 0x7f07a015de80 2023-11-08T19:30:19.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.274+0000 7f0797fff700 1 -- 172.21.15.143:0/3582568339 >> v2:172.21.15.174:3300/0 conn(0x7f07a006c730 msgr2=0x7f07a00cd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:19.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.274+0000 7f0797fff700 1 --2- 172.21.15.143:0/3582568339 >> v2:172.21.15.174:3300/0 conn(0x7f07a006c730 0x7f07a00cd910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:19.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.274+0000 7f0797fff700 1 -- 172.21.15.143:0/3582568339 >> v1:172.21.15.174:6789/0 conn(0x7f07a006c030 legacy=0x7f07a006cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:19.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.274+0000 7f0797fff700 1 -- 172.21.15.143:0/3582568339 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07a00ccb50 con 0x7f07a015de80 2023-11-08T19:30:19.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.274+0000 7f0797fff700 1 -- 172.21.15.143:0/3582568339 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4147756303 0 0) 0x7f07980019f0 con 0x7f07a015de80 2023-11-08T19:30:19.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.274+0000 7f0797fff700 1 -- 172.21.15.143:0/3582568339 >> v1:172.21.15.143:6789/0 conn(0x7f07a015de80 legacy=0x7f07a00cbf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:19.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.274+0000 7f0797fff700 1 --2- 172.21.15.143:0/3582568339 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f078c003a00 0x7f078c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:19.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.274+0000 7f0797fff700 1 --2- 172.21.15.143:0/3582568339 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f078c004850 0x7f078c004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:19.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.274+0000 7f0797fff700 1 --2- 172.21.15.143:0/3582568339 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f078c005210 0x7f078c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:19.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.274+0000 7f0797fff700 1 -- 172.21.15.143:0/3582568339 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07a00cc870 con 0x7f078c003a00 2023-11-08T19:30:19.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.274+0000 7f0797fff700 1 -- 172.21.15.143:0/3582568339 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f078c00c0b0 con 0x7f078c005210 2023-11-08T19:30:19.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.274+0000 7f0797fff700 1 -- 172.21.15.143:0/3582568339 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f078c00c270 con 0x7f078c004850 2023-11-08T19:30:19.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.274+0000 7f07a5067700 1 --2- 172.21.15.143:0/3582568339 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f078c003a00 0x7f078c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:19.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.274+0000 7f07a5868700 1 --2- 172.21.15.143:0/3582568339 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f078c005210 0x7f078c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:19.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.274+0000 7f07a4866700 1 --2- 172.21.15.143:0/3582568339 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f078c004850 0x7f078c004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:19.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.274+0000 7f07a5067700 1 -- 172.21.15.143:0/3582568339 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f078c004850 msgr2=0x7f078c004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:19.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.274+0000 7f07a5067700 1 --2- 172.21.15.143:0/3582568339 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f078c004850 0x7f078c004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:19.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.274+0000 7f07a5067700 1 -- 172.21.15.143:0/3582568339 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f078c005210 msgr2=0x7f078c00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:19.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.274+0000 7f07a5067700 1 --2- 172.21.15.143:0/3582568339 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f078c005210 0x7f078c00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:19.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.274+0000 7f07a5868700 1 -- 172.21.15.143:0/3582568339 reap_dead start 2023-11-08T19:30:19.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.274+0000 7f07a5067700 1 -- 172.21.15.143:0/3582568339 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f078c0035b0 con 0x7f078c003a00 2023-11-08T19:30:19.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.274+0000 7f07a5067700 1 --2- 172.21.15.143:0/3582568339 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f078c003a00 0x7f078c003e20 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f0790009b60 tx=0x7f078c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=877e9cb5f4ccc523 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:19.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.275+0000 7f0797fff700 1 -- 172.21.15.143:0/3582568339 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f079000e6c0 con 0x7f078c003a00 2023-11-08T19:30:19.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.275+0000 7f0797fff700 1 -- 172.21.15.143:0/3582568339 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f079000e820 con 0x7f078c003a00 2023-11-08T19:30:19.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.275+0000 7f0797fff700 1 -- 172.21.15.143:0/3582568339 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f079000eb90 con 0x7f078c003a00 2023-11-08T19:30:19.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.275+0000 7f07a72cb700 1 -- 172.21.15.143:0/3582568339 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f078c003a00 msgr2=0x7f078c003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:19.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.275+0000 7f07a72cb700 1 --2- 172.21.15.143:0/3582568339 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f078c003a00 0x7f078c003e20 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f0790009b60 tx=0x7f078c005880 comp rx=0 tx=0).stop 2023-11-08T19:30:19.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.275+0000 7f07a72cb700 1 -- 172.21.15.143:0/3582568339 shutdown_connections 2023-11-08T19:30:19.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.275+0000 7f07a72cb700 1 --2- 172.21.15.143:0/3582568339 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f078c003a00 0x7f078c003e20 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:19.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.275+0000 7f07a72cb700 1 -- 172.21.15.143:0/3582568339 >> 172.21.15.143:0/3582568339 conn(0x7f07a008b000 msgr2=0x7f07a009ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:19.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.275+0000 7f07a72cb700 1 -- 172.21.15.143:0/3582568339 shutdown_connections 2023-11-08T19:30:19.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.275+0000 7f07a72cb700 1 -- 172.21.15.143:0/3582568339 wait complete. 2023-11-08T19:30:19.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.276+0000 7f07a72cb700 1 Processor -- start 2023-11-08T19:30:19.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.276+0000 7f07a72cb700 1 -- start start 2023-11-08T19:30:19.296 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.277+0000 7f07a72cb700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f078c003a00 0x7f07a00c6980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:19.296 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.277+0000 7f07a72cb700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f07a00c53e0 0x7f07a014fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:19.296 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.277+0000 7f07a72cb700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f07a00c73c0 0x7f07a00d1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:19.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.277+0000 7f07a72cb700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07a00c6560 con 0x7f078c003a00 2023-11-08T19:30:19.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.277+0000 7f07a72cb700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07a00c58e0 con 0x7f07a00c53e0 2023-11-08T19:30:19.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.277+0000 7f07a72cb700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07a00c5bc0 con 0x7f07a00c73c0 2023-11-08T19:30:19.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.277+0000 7f07a5067700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f078c003a00 0x7f07a00c6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:19.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.277+0000 7f07a5067700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f078c003a00 0x7f07a00c6980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56146/0 (socket says 172.21.15.143:56146) 2023-11-08T19:30:19.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.277+0000 7f07a5067700 1 -- 172.21.15.143:0/66523571 learned_addr learned my addr 172.21.15.143:0/66523571 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:30:19.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.278+0000 7f07a5868700 1 --2- 172.21.15.143:0/66523571 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f07a00c73c0 0x7f07a00d1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:19.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.278+0000 7f07a5067700 1 -- 172.21.15.143:0/66523571 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f07a00c73c0 msgr2=0x7f07a00d1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:19.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.278+0000 7f07a5067700 1 --2- 172.21.15.143:0/66523571 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f07a00c73c0 0x7f07a00d1840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:19.299 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.278+0000 7f07a5067700 1 -- 172.21.15.143:0/66523571 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f07a00c53e0 msgr2=0x7f07a014fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:19.299 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.278+0000 7f07a4866700 1 --2- 172.21.15.143:0/66523571 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f07a00c53e0 0x7f07a014fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:19.299 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.278+0000 7f07a5067700 1 --2- 172.21.15.143:0/66523571 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f07a00c53e0 0x7f07a014fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:19.299 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.278+0000 7f07a5067700 1 -- 172.21.15.143:0/66523571 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07a00c5ea0 con 0x7f078c003a00 2023-11-08T19:30:19.299 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.278+0000 7f07a5067700 1 --2- 172.21.15.143:0/66523571 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f078c003a00 0x7f07a00c6980 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f078c00c080 tx=0x7f07900047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:19.300 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.278+0000 7f0795ffb700 1 -- 172.21.15.143:0/66523571 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f079000e6c0 con 0x7f078c003a00 2023-11-08T19:30:19.300 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.278+0000 7f0795ffb700 1 -- 172.21.15.143:0/66523571 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0790003ad0 con 0x7f078c003a00 2023-11-08T19:30:19.300 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.278+0000 7f0795ffb700 1 -- 172.21.15.143:0/66523571 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0790003e80 con 0x7f078c003a00 2023-11-08T19:30:19.300 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.278+0000 7f07a72cb700 1 -- 172.21.15.143:0/66523571 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07a00c6180 con 0x7f078c003a00 2023-11-08T19:30:19.300 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.278+0000 7f07a72cb700 1 -- 172.21.15.143:0/66523571 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07a00c7870 con 0x7f078c003a00 2023-11-08T19:30:19.300 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.279+0000 7f0795ffb700 1 -- 172.21.15.143:0/66523571 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0790013470 con 0x7f078c003a00 2023-11-08T19:30:19.301 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.280+0000 7f0795ffb700 1 --2- 172.21.15.143:0/66523571 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f078406af00 0x7f078406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:19.301 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.280+0000 7f0795ffb700 1 -- 172.21.15.143:0/66523571 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 7145+0+0 (secure 0 0 0) 0x7f079008b210 con 0x7f078c003a00 2023-11-08T19:30:19.301 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.280+0000 7f07a4866700 1 --2- 172.21.15.143:0/66523571 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f078406af00 0x7f078406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:19.301 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.281+0000 7f07a4866700 1 --2- 172.21.15.143:0/66523571 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f078406af00 0x7f078406d3c0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f079c007fd0 tx=0x7f079c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:19.301 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.282+0000 7f07b34c4b80 1 Processor -- start 2023-11-08T19:30:19.302 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.282+0000 7f07b34c4b80 1 -- start start 2023-11-08T19:30:19.302 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.283+0000 7f07b34c4b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e4cef486f0 0x55e4cef78450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:19.302 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.283+0000 7f07b34c4b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e4cef4ac30 0x55e4cef78990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:19.302 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.283+0000 7f07b34c4b80 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55e4cef48ad0 0x55e4cef62f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:19.302 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.283+0000 7f07b34c4b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4ceeb46e0 con 0x55e4cef4ac30 2023-11-08T19:30:19.303 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.283+0000 7f07b34c4b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4ceea83b0 con 0x55e4cef486f0 2023-11-08T19:30:19.303 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.283+0000 7f07b34c4b80 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e4ceea5cd0 con 0x55e4cef48ad0 2023-11-08T19:30:19.303 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.283+0000 7f07a4866700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e4cef4ac30 0x55e4cef78990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:19.303 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.283+0000 7f07a4866700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e4cef4ac30 0x55e4cef78990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56152/0 (socket says 172.21.15.143:56152) 2023-11-08T19:30:19.303 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.283+0000 7f07a4866700 1 -- 172.21.15.143:0/3605084804 learned_addr learned my addr 172.21.15.143:0/3605084804 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:30:19.304 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.283+0000 7f07a5868700 1 --2- 172.21.15.143:0/3605084804 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55e4cef48ad0 0x55e4cef62f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:19.304 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.283+0000 7f07a5067700 1 --2- 172.21.15.143:0/3605084804 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e4cef486f0 0x55e4cef78450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:19.304 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.283+0000 7f07a4866700 1 -- 172.21.15.143:0/3605084804 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55e4cef48ad0 msgr2=0x55e4cef62f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:19.304 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.283+0000 7f07a4866700 1 --2- 172.21.15.143:0/3605084804 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55e4cef48ad0 0x55e4cef62f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:19.304 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.283+0000 7f07a4866700 1 -- 172.21.15.143:0/3605084804 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e4cef486f0 msgr2=0x55e4cef78450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:19.305 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.283+0000 7f07a4866700 1 --2- 172.21.15.143:0/3605084804 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e4cef486f0 0x55e4cef78450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:19.305 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.283+0000 7f07a4866700 1 -- 172.21.15.143:0/3605084804 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e4ced1e5e0 con 0x55e4cef4ac30 2023-11-08T19:30:19.305 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.283+0000 7f07a4866700 1 --2- 172.21.15.143:0/3605084804 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e4cef4ac30 0x55e4cef78990 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f079c0014b0 tx=0x7f079c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f41499e498b8360d server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:19.305 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.284+0000 7f0782ffd700 1 -- 172.21.15.143:0/3605084804 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f079c014400 con 0x55e4cef4ac30 2023-11-08T19:30:19.305 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.284+0000 7f0782ffd700 1 -- 172.21.15.143:0/3605084804 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f079c014560 con 0x55e4cef4ac30 2023-11-08T19:30:19.306 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.284+0000 7f0782ffd700 1 -- 172.21.15.143:0/3605084804 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f079c0148d0 con 0x55e4cef4ac30 2023-11-08T19:30:19.306 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.284+0000 7f07b34c4b80 1 -- 172.21.15.143:0/3605084804 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e4cef4ac30 msgr2=0x55e4cef78990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:19.306 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.284+0000 7f07b34c4b80 1 --2- 172.21.15.143:0/3605084804 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e4cef4ac30 0x55e4cef78990 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f079c0014b0 tx=0x7f079c013000 comp rx=0 tx=0).stop 2023-11-08T19:30:19.306 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.284+0000 7f07b34c4b80 1 -- 172.21.15.143:0/3605084804 shutdown_connections 2023-11-08T19:30:19.306 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.284+0000 7f07b34c4b80 1 --2- 172.21.15.143:0/3605084804 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55e4cef48ad0 0x55e4cef62f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:19.307 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.284+0000 7f07b34c4b80 1 --2- 172.21.15.143:0/3605084804 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e4cef486f0 0x55e4cef78450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:19.307 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.284+0000 7f07b34c4b80 1 --2- 172.21.15.143:0/3605084804 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e4cef4ac30 0x55e4cef78990 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:19.307 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.284+0000 7f07b34c4b80 1 -- 172.21.15.143:0/3605084804 >> 172.21.15.143:0/3605084804 conn(0x55e4ceec3660 msgr2=0x55e4cef4e3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:19.308 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.284+0000 7f07b34c4b80 1 -- 172.21.15.143:0/3605084804 shutdown_connections 2023-11-08T19:30:19.308 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.284+0000 7f07b34c4b80 1 -- 172.21.15.143:0/3605084804 wait complete. 2023-11-08T19:30:19.308 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.285+0000 7f07b34c4b80 1 -- start start 2023-11-08T19:30:19.308 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.285+0000 7f07b34c4b80 1 Processor -- start 2023-11-08T19:30:19.308 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.286+0000 7f07b34c4b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e4cef486f0 0x55e4cef7a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:19.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.286+0000 7f07b34c4b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e4cef48ad0 0x55e4cef8ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:19.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.286+0000 7f07b34c4b80 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55e4cef8cfb0 0x55e4cef49020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:19.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.286+0000 7f07b34c4b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4ceea15c0 con 0x55e4cef486f0 2023-11-08T19:30:19.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.286+0000 7f07b34c4b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4cee3d0d0 con 0x55e4cef48ad0 2023-11-08T19:30:19.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.286+0000 7f07b34c4b80 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e4cee3cd70 con 0x55e4cef8cfb0 2023-11-08T19:30:19.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.286+0000 7f07b34c4b80 10 client.0 _ll_register_callbacks cb 0x55e4cec997b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T19:30:19.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.286+0000 7f07a5067700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e4cef486f0 0x55e4cef7a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:19.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.286+0000 7f07a5067700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e4cef486f0 0x55e4cef7a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56158/0 (socket says 172.21.15.143:56158) 2023-11-08T19:30:19.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.286+0000 7f07a5067700 1 -- 172.21.15.143:0/988530849 learned_addr learned my addr 172.21.15.143:0/988530849 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:30:19.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.286+0000 7f07a4866700 1 --2- 172.21.15.143:0/988530849 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e4cef48ad0 0x55e4cef8ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:19.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.286+0000 7f07a5868700 1 --2- 172.21.15.143:0/988530849 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55e4cef8cfb0 0x55e4cef49020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:19.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.286+0000 7f07a5067700 1 -- 172.21.15.143:0/988530849 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55e4cef8cfb0 msgr2=0x55e4cef49020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:19.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.286+0000 7f07a5067700 1 --2- 172.21.15.143:0/988530849 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55e4cef8cfb0 0x55e4cef49020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:19.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.286+0000 7f07a5067700 1 -- 172.21.15.143:0/988530849 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e4cef48ad0 msgr2=0x55e4cef8ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:19.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.286+0000 7f07a5067700 1 --2- 172.21.15.143:0/988530849 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e4cef48ad0 0x55e4cef8ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:19.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.286+0000 7f07a5067700 1 -- 172.21.15.143:0/988530849 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f079c00f040 con 0x55e4cef486f0 2023-11-08T19:30:19.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.286+0000 7f07a5067700 1 --2- 172.21.15.143:0/988530849 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e4cef486f0 0x55e4cef7a640 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f0790059770 tx=0x7f07900596a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d004ee9aa7a040c server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:19.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.286+0000 7f077bfff700 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2023-11-08T19:30:19.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.287+0000 7f077bfff700 1 -- 172.21.15.143:0/988530849 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f079005a170 con 0x55e4cef486f0 2023-11-08T19:30:19.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.287+0000 7f077bfff700 1 -- 172.21.15.143:0/988530849 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 7145+0+0 (secure 0 0 0) 0x7f0790059a70 con 0x55e4cef486f0 2023-11-08T19:30:19.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.287+0000 7f07b34c4b80 1 -- 172.21.15.143:0/988530849 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e4cef40880 con 0x55e4cef486f0 2023-11-08T19:30:19.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.287+0000 7f077bfff700 1 -- 172.21.15.143:0/988530849 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7f079005dbc0 con 0x55e4cef486f0 2023-11-08T19:30:19.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.287+0000 7f07b34c4b80 10 client.7641 fetch_fsmap learned FSMap version 88 2023-11-08T19:30:19.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.287+0000 7f07b34c4b80 1 -- 172.21.15.143:0/988530849 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=88,monmap=2+}) v3 -- 0x55e4ceccad90 con 0x55e4cef486f0 2023-11-08T19:30:19.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.287+0000 7f077bfff700 1 -- 172.21.15.143:0/988530849 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0790075070 con 0x55e4cef486f0 2023-11-08T19:30:19.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.287+0000 7f077bfff700 1 -- 172.21.15.143:0/988530849 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 88) v1 ==== 3306+0+0 (secure 0 0 0) 0x7f0790071180 con 0x55e4cef486f0 2023-11-08T19:30:19.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.287+0000 7f07b34c4b80 10 client.7641 fetch_fsmap finished waiting for FSMap version 88 2023-11-08T19:30:19.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.287+0000 7f07b34c4b80 10 client.7641 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-08T19:30:19.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.287+0000 7f07b34c4b80 20 client.7641 populate_metadata read hostname 'smithi143' 2023-11-08T19:30:19.315 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.287+0000 7f07b34c4b80 1 --2- 172.21.15.143:0/988530849 >> [v2:172.21.15.174:6834/3601088618,v1:172.21.15.174:6836/3601088618] conn(0x55e4cef639f0 0x55e4cefa1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:19.315 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.287+0000 7f07b34c4b80 4 client.7641 mds_command: new command op to 6918 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-08T19:30:19.315 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.287+0000 7f07b34c4b80 1 -- 172.21.15.143:0/988530849 --> [v2:172.21.15.174:6834/3601088618,v1:172.21.15.174:6836/3601088618] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e4cef1ca30 con 0x55e4cef639f0 2023-11-08T19:30:19.315 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.288+0000 7f07a4866700 1 --2- 172.21.15.143:0/988530849 >> [v2:172.21.15.174:6834/3601088618,v1:172.21.15.174:6836/3601088618] conn(0x55e4cef639f0 0x55e4cefa1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:19.315 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.288+0000 7f077bfff700 10 client.7641 ms_handle_connect on v2:172.21.15.174:6834/3601088618 2023-11-08T19:30:19.316 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.288+0000 7f07a4866700 1 --2- 172.21.15.143:0/988530849 >> [v2:172.21.15.174:6834/3601088618,v1:172.21.15.174:6836/3601088618] conn(0x55e4cef639f0 0x55e4cefa1bc0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=241716a23f421925 server_cookie=d3becb9ad75a957e in_seq=0 out_seq=0 2023-11-08T19:30:19.316 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.289+0000 7f077bfff700 1 -- 172.21.15.143:0/988530849 <== mds.1 v2:172.21.15.174:6834/3601088618 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e4cef1ca30 con 0x55e4cef639f0 2023-11-08T19:30:19.316 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.289+0000 7f077bfff700 10 client.7641 handle_command_reply: tid=0 2023-11-08T19:30:19.316 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.289+0000 7f07b34c4b80 1 client.7641 shutdown 2023-11-08T19:30:19.316 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.289+0000 7f07b34c4b80 1 -- 172.21.15.143:0/988530849 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e4cef486f0 msgr2=0x55e4cef7a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:19.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.289+0000 7f07b34c4b80 1 --2- 172.21.15.143:0/988530849 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e4cef486f0 0x55e4cef7a640 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f0790059770 tx=0x7f07900596a0 comp rx=0 tx=0).stop 2023-11-08T19:30:19.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.289+0000 7f07b34c4b80 1 -- 172.21.15.143:0/988530849 shutdown_connections 2023-11-08T19:30:19.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.289+0000 7f07b34c4b80 1 --2- 172.21.15.143:0/988530849 >> [v2:172.21.15.174:6834/3601088618,v1:172.21.15.174:6836/3601088618] conn(0x55e4cef639f0 0x55e4cefa1bc0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:19.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.289+0000 7f07b34c4b80 1 --2- 172.21.15.143:0/988530849 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55e4cef8cfb0 0x55e4cef49020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:19.318 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.289+0000 7f077bfff700 0 client.7641 ms_handle_reset on v2:172.21.15.174:6834/3601088618 2023-11-08T19:30:19.318 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.289+0000 7f07b34c4b80 1 --2- 172.21.15.143:0/988530849 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e4cef486f0 0x55e4cef7a640 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:19.318 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.289+0000 7f07b34c4b80 1 --2- 172.21.15.143:0/988530849 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e4cef48ad0 0x55e4cef8ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:19.318 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.289+0000 7f07b34c4b80 1 -- 172.21.15.143:0/988530849 >> 172.21.15.143:0/988530849 conn(0x55e4ceec3660 msgr2=0x55e4cef4b8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:19.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.289+0000 7f07b34c4b80 1 -- 172.21.15.143:0/988530849 shutdown_connections 2023-11-08T19:30:19.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.289+0000 7f07b34c4b80 1 -- 172.21.15.143:0/988530849 wait complete. 2023-11-08T19:30:19.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.289+0000 7f07b34c4b80 20 client.7641 trim_cache size 0 max 16384 2023-11-08T19:30:19.325 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.307+0000 7f07b34c4b80 1 Processor -- start 2023-11-08T19:30:19.325 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.307+0000 7f07b34c4b80 1 -- start start 2023-11-08T19:30:19.325 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.308+0000 7f07b34c4b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e4cef486f0 0x55e4cef4c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:19.325 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.308+0000 7f07b34c4b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e4cef48ad0 0x55e4cef78440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:19.325 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.308+0000 7f07b34c4b80 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55e4cef8cfb0 0x55e4cef78980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:19.326 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.308+0000 7f07b34c4b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4ced208d0 con 0x55e4cef48ad0 2023-11-08T19:30:19.326 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.308+0000 7f07b34c4b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4cef2a540 con 0x55e4cef486f0 2023-11-08T19:30:19.326 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.308+0000 7f07b34c4b80 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e4cef4cad0 con 0x55e4cef8cfb0 2023-11-08T19:30:19.326 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.308+0000 7f07a4866700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e4cef48ad0 0x55e4cef78440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:19.327 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.308+0000 7f07a4866700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e4cef48ad0 0x55e4cef78440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56168/0 (socket says 172.21.15.143:56168) 2023-11-08T19:30:19.327 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.308+0000 7f07a4866700 1 -- 172.21.15.143:0/1413036933 learned_addr learned my addr 172.21.15.143:0/1413036933 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:30:19.327 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.308+0000 7f07a5067700 1 --2- 172.21.15.143:0/1413036933 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e4cef486f0 0x55e4cef4c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:19.327 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.308+0000 7f07a4866700 1 -- 172.21.15.143:0/1413036933 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55e4cef8cfb0 msgr2=0x55e4cef78980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:19.327 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.308+0000 7f07a5868700 1 --2- 172.21.15.143:0/1413036933 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55e4cef8cfb0 0x55e4cef78980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:19.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.308+0000 7f07a4866700 1 --2- 172.21.15.143:0/1413036933 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55e4cef8cfb0 0x55e4cef78980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:19.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.308+0000 7f07a4866700 1 -- 172.21.15.143:0/1413036933 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e4cef486f0 msgr2=0x55e4cef4c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:19.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.308+0000 7f07a4866700 1 --2- 172.21.15.143:0/1413036933 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e4cef486f0 0x55e4cef4c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:19.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.308+0000 7f07a4866700 1 -- 172.21.15.143:0/1413036933 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e4cef7c110 con 0x55e4cef48ad0 2023-11-08T19:30:19.329 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.308+0000 7f07a5067700 1 --2- 172.21.15.143:0/1413036933 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e4cef486f0 0x55e4cef4c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:30:19.329 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.308+0000 7f07a5868700 1 --2- 172.21.15.143:0/1413036933 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55e4cef8cfb0 0x55e4cef78980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:30:19.329 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.308+0000 7f07a4866700 1 --2- 172.21.15.143:0/1413036933 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e4cef48ad0 0x55e4cef78440 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f079c009010 tx=0x7f079c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f09790e91b06f4c server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:19.329 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.308+0000 7f07837fe700 1 -- 172.21.15.143:0/1413036933 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f079c014400 con 0x55e4cef48ad0 2023-11-08T19:30:19.329 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.308+0000 7f07837fe700 1 -- 172.21.15.143:0/1413036933 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f079c006a60 con 0x55e4cef48ad0 2023-11-08T19:30:19.330 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.308+0000 7f07837fe700 1 -- 172.21.15.143:0/1413036933 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f079c006dd0 con 0x55e4cef48ad0 2023-11-08T19:30:19.330 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.309+0000 7f07b34c4b80 1 -- 172.21.15.143:0/1413036933 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e4cef48ad0 msgr2=0x55e4cef78440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:19.330 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.309+0000 7f07b34c4b80 1 --2- 172.21.15.143:0/1413036933 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e4cef48ad0 0x55e4cef78440 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f079c009010 tx=0x7f079c015000 comp rx=0 tx=0).stop 2023-11-08T19:30:19.330 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.309+0000 7f07b34c4b80 1 -- 172.21.15.143:0/1413036933 shutdown_connections 2023-11-08T19:30:19.330 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.309+0000 7f07b34c4b80 1 --2- 172.21.15.143:0/1413036933 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55e4cef8cfb0 0x55e4cef78980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:19.331 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.309+0000 7f07b34c4b80 1 --2- 172.21.15.143:0/1413036933 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e4cef486f0 0x55e4cef4c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:19.331 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.309+0000 7f07b34c4b80 1 --2- 172.21.15.143:0/1413036933 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e4cef48ad0 0x55e4cef78440 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:19.331 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.309+0000 7f07b34c4b80 1 -- 172.21.15.143:0/1413036933 >> 172.21.15.143:0/1413036933 conn(0x55e4ceec3660 msgr2=0x55e4cef447e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:19.331 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.309+0000 7f07b34c4b80 1 -- 172.21.15.143:0/1413036933 shutdown_connections 2023-11-08T19:30:19.331 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.309+0000 7f07b34c4b80 1 -- 172.21.15.143:0/1413036933 wait complete. 2023-11-08T19:30:19.332 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.310+0000 7f07b34c4b80 1 -- start start 2023-11-08T19:30:19.332 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.310+0000 7f07b34c4b80 1 Processor -- start 2023-11-08T19:30:19.332 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.311+0000 7f07b34c4b80 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55e4cef486f0 0x55e4cef78440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:19.332 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.311+0000 7f07b34c4b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e4cef48ad0 0x55e4cef78980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:19.332 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.311+0000 7f07b34c4b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e4cef8cfb0 0x55e4cef795c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:19.333 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.311+0000 7f07b34c4b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4cef93050 con 0x55e4cef48ad0 2023-11-08T19:30:19.333 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.311+0000 7f07b34c4b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4cefb6940 con 0x55e4cef8cfb0 2023-11-08T19:30:19.333 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.311+0000 7f07b34c4b80 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e4cef62350 con 0x55e4cef486f0 2023-11-08T19:30:19.333 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.311+0000 7f07b34c4b80 10 client.0 _ll_register_callbacks cb 0x7f0790059930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T19:30:19.334 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.311+0000 7f07a4866700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e4cef48ad0 0x55e4cef78980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:19.334 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.311+0000 7f07a4866700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e4cef48ad0 0x55e4cef78980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56174/0 (socket says 172.21.15.143:56174) 2023-11-08T19:30:19.334 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.311+0000 7f07a4866700 1 -- 172.21.15.143:0/3123454994 learned_addr learned my addr 172.21.15.143:0/3123454994 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:30:19.334 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.311+0000 7f07a5067700 1 --2- 172.21.15.143:0/3123454994 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55e4cef486f0 0x55e4cef78440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:19.334 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.311+0000 7f07a5868700 1 --2- 172.21.15.143:0/3123454994 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e4cef8cfb0 0x55e4cef795c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:19.335 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.311+0000 7f07a4866700 1 -- 172.21.15.143:0/3123454994 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55e4cef486f0 msgr2=0x55e4cef78440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:19.335 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.311+0000 7f07a4866700 1 --2- 172.21.15.143:0/3123454994 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55e4cef486f0 0x55e4cef78440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:19.335 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.311+0000 7f07a4866700 1 -- 172.21.15.143:0/3123454994 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e4cef8cfb0 msgr2=0x55e4cef795c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:19.335 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.311+0000 7f07a4866700 1 --2- 172.21.15.143:0/3123454994 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e4cef8cfb0 0x55e4cef795c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:19.336 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.311+0000 7f07a4866700 1 -- 172.21.15.143:0/3123454994 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e4ced1e5e0 con 0x55e4cef48ad0 2023-11-08T19:30:19.336 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.312+0000 7f07a4866700 1 --2- 172.21.15.143:0/3123454994 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e4cef48ad0 0x55e4cef78980 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f079c020000 tx=0x7f079c001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6c2d70e0862234c server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:19.336 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.312+0000 7f077bfff700 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2023-11-08T19:30:19.336 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.312+0000 7f077bfff700 1 -- 172.21.15.143:0/3123454994 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f079c0146d0 con 0x55e4cef48ad0 2023-11-08T19:30:19.336 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.312+0000 7f077bfff700 1 -- 172.21.15.143:0/3123454994 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 7145+0+0 (secure 0 0 0) 0x7f079c01c050 con 0x55e4cef48ad0 2023-11-08T19:30:19.337 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.312+0000 7f07b34c4b80 1 -- 172.21.15.143:0/3123454994 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f079c014560 con 0x55e4cef48ad0 2023-11-08T19:30:19.337 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.312+0000 7f077bfff700 1 -- 172.21.15.143:0/3123454994 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7f079c014830 con 0x55e4cef48ad0 2023-11-08T19:30:19.337 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.312+0000 7f07b34c4b80 10 client.7647 fetch_fsmap learned FSMap version 88 2023-11-08T19:30:19.337 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.312+0000 7f07b34c4b80 1 -- 172.21.15.143:0/3123454994 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=88,monmap=2+}) v3 -- 0x55e4cecb3c10 con 0x55e4cef48ad0 2023-11-08T19:30:19.337 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.312+0000 7f077bfff700 1 -- 172.21.15.143:0/3123454994 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f079c014560 con 0x55e4cef48ad0 2023-11-08T19:30:19.338 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.312+0000 7f077bfff700 1 -- 172.21.15.143:0/3123454994 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 88) v1 ==== 3306+0+0 (secure 0 0 0) 0x7f079c014990 con 0x55e4cef48ad0 2023-11-08T19:30:19.338 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.312+0000 7f07b34c4b80 10 client.7647 fetch_fsmap finished waiting for FSMap version 88 2023-11-08T19:30:19.338 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.312+0000 7f07b34c4b80 10 client.7647 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-08T19:30:19.338 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.312+0000 7f07b34c4b80 20 client.7647 populate_metadata read hostname 'smithi143' 2023-11-08T19:30:19.339 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.312+0000 7f07b34c4b80 1 --2- 172.21.15.143:0/3123454994 >> [v2:172.21.15.174:6834/3601088618,v1:172.21.15.174:6836/3601088618] conn(0x55e4cef4c590 0x55e4cef79b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:19.339 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.312+0000 7f07b34c4b80 4 client.7647 mds_command: new command op to 6918 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-08T19:30:19.339 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.312+0000 7f07b34c4b80 1 -- 172.21.15.143:0/3123454994 --> [v2:172.21.15.174:6834/3601088618,v1:172.21.15.174:6836/3601088618] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55e4cee9a5a0 con 0x55e4cef4c590 2023-11-08T19:30:19.339 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.312+0000 7f07a5067700 1 --2- 172.21.15.143:0/3123454994 >> [v2:172.21.15.174:6834/3601088618,v1:172.21.15.174:6836/3601088618] conn(0x55e4cef4c590 0x55e4cef79b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:19.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.313+0000 7f07a5067700 1 --2- 172.21.15.143:0/3123454994 >> [v2:172.21.15.174:6834/3601088618,v1:172.21.15.174:6836/3601088618] conn(0x55e4cef4c590 0x55e4cef79b00 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=97d148247a6bef52 server_cookie=9ce90c0a9c8988ad in_seq=0 out_seq=0 2023-11-08T19:30:19.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.313+0000 7f077bfff700 10 client.7647 ms_handle_connect on v2:172.21.15.174:6834/3601088618 2023-11-08T19:30:19.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.313+0000 7f077bfff700 1 -- 172.21.15.143:0/3123454994 <== mds.1 v2:172.21.15.174:6834/3601088618 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x55e4cee9a5a0 con 0x55e4cef4c590 2023-11-08T19:30:19.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.313+0000 7f077bfff700 10 client.7647 handle_command_reply: tid=0 2023-11-08T19:30:19.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.313+0000 7f07b34c4b80 1 client.7647 shutdown 2023-11-08T19:30:19.341 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.313+0000 7f07b34c4b80 1 -- 172.21.15.143:0/3123454994 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e4cef48ad0 msgr2=0x55e4cef78980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:19.341 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.313+0000 7f07b34c4b80 1 --2- 172.21.15.143:0/3123454994 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e4cef48ad0 0x55e4cef78980 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f079c020000 tx=0x7f079c001120 comp rx=0 tx=0).stop 2023-11-08T19:30:19.341 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.313+0000 7f07b34c4b80 1 -- 172.21.15.143:0/3123454994 shutdown_connections 2023-11-08T19:30:19.341 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.313+0000 7f07b34c4b80 1 --2- 172.21.15.143:0/3123454994 >> [v2:172.21.15.174:6834/3601088618,v1:172.21.15.174:6836/3601088618] conn(0x55e4cef4c590 0x55e4cef79b00 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:19.341 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.313+0000 7f07b34c4b80 1 --2- 172.21.15.143:0/3123454994 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55e4cef486f0 0x55e4cef78440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:19.342 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.313+0000 7f077bfff700 0 client.7647 ms_handle_reset on v2:172.21.15.174:6834/3601088618 2023-11-08T19:30:19.342 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.313+0000 7f07b34c4b80 1 --2- 172.21.15.143:0/3123454994 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e4cef8cfb0 0x55e4cef795c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:19.342 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.313+0000 7f07b34c4b80 1 --2- 172.21.15.143:0/3123454994 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e4cef48ad0 0x55e4cef78980 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:19.342 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.313+0000 7f07b34c4b80 1 -- 172.21.15.143:0/3123454994 >> 172.21.15.143:0/3123454994 conn(0x55e4ceec3660 msgr2=0x55e4cef447e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:19.343 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.313+0000 7f07b34c4b80 1 -- 172.21.15.143:0/3123454994 shutdown_connections 2023-11-08T19:30:19.343 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.313+0000 7f07b34c4b80 1 -- 172.21.15.143:0/3123454994 wait complete. 2023-11-08T19:30:19.343 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.313+0000 7f07b34c4b80 20 client.7647 trim_cache size 0 max 16384 2023-11-08T19:30:19.344 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2023-11-08T19:30:19.344 INFO:teuthology.orchestra.run.smithi143.stdout: "cluster_fsid": "968483af-77fc-4068-ad7c-5c2a1a4613d7", 2023-11-08T19:30:19.344 INFO:teuthology.orchestra.run.smithi143.stdout: "whoami": 1, 2023-11-08T19:30:19.344 INFO:teuthology.orchestra.run.smithi143.stdout: "id": 6918, 2023-11-08T19:30:19.344 INFO:teuthology.orchestra.run.smithi143.stdout: "want_state": "up:active", 2023-11-08T19:30:19.345 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "up:active", 2023-11-08T19:30:19.345 INFO:teuthology.orchestra.run.smithi143.stdout: "fs_name": "cephfs", 2023-11-08T19:30:19.345 INFO:teuthology.orchestra.run.smithi143.stdout: "rank_uptime": 121.719531046, 2023-11-08T19:30:19.345 INFO:teuthology.orchestra.run.smithi143.stdout: "mdsmap_epoch": 88, 2023-11-08T19:30:19.345 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_epoch": 98, 2023-11-08T19:30:19.346 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_epoch_barrier": 98, 2023-11-08T19:30:19.346 INFO:teuthology.orchestra.run.smithi143.stdout: "uptime": 141.51074720400001 2023-11-08T19:30:19.346 INFO:teuthology.orchestra.run.smithi143.stdout:} 2023-11-08T19:30:19.348 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.314+0000 7f07a72cb700 1 -- 172.21.15.143:0/66523571 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f078406af00 msgr2=0x7f078406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:19.348 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.314+0000 7f07a72cb700 1 --2- 172.21.15.143:0/66523571 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f078406af00 0x7f078406d3c0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f079c007fd0 tx=0x7f079c009040 comp rx=0 tx=0).stop 2023-11-08T19:30:19.348 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.314+0000 7f07a72cb700 1 -- 172.21.15.143:0/66523571 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f078c003a00 msgr2=0x7f07a00c6980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:19.348 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.314+0000 7f07a72cb700 1 --2- 172.21.15.143:0/66523571 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f078c003a00 0x7f07a00c6980 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f078c00c080 tx=0x7f07900047c0 comp rx=0 tx=0).stop 2023-11-08T19:30:19.348 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.314+0000 7f07a72cb700 1 -- 172.21.15.143:0/66523571 shutdown_connections 2023-11-08T19:30:19.349 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.314+0000 7f07a72cb700 1 --2- 172.21.15.143:0/66523571 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f078406af00 0x7f078406d3c0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:19.349 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.314+0000 7f07a72cb700 1 --2- 172.21.15.143:0/66523571 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f07a00c73c0 0x7f07a00d1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:19.349 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.314+0000 7f07a72cb700 1 --2- 172.21.15.143:0/66523571 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f078c003a00 0x7f07a00c6980 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:19.349 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.314+0000 7f07a72cb700 1 --2- 172.21.15.143:0/66523571 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f07a00c53e0 0x7f07a014fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:19.349 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.314+0000 7f07a72cb700 1 -- 172.21.15.143:0/66523571 >> 172.21.15.143:0/66523571 conn(0x7f07a008b000 msgr2=0x7f07a00d0420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:19.350 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.314+0000 7f07a72cb700 1 -- 172.21.15.143:0/66523571 shutdown_connections 2023-11-08T19:30:19.350 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.314+0000 7f07a72cb700 1 -- 172.21.15.143:0/66523571 wait complete. 2023-11-08T19:30:19.350 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-11-08T19:30:19.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.471+0000 7fd2b385e700 1 Processor -- start 2023-11-08T19:30:19.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.471+0000 7fd2b385e700 1 -- start start 2023-11-08T19:30:19.474 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.472+0000 7fd2b385e700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd2ac0d3690 0x7fd2ac0d1f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:19.474 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.472+0000 7fd2b385e700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2ac0d3200 con 0x7fd2ac0cf090 2023-11-08T19:30:19.475 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.472+0000 7fd2b385e700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2ac15e060 con 0x7fd2ac15dc80 2023-11-08T19:30:19.475 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.472+0000 7fd2b385e700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2ac0d25f0 con 0x7fd2ac0d3690 2023-11-08T19:30:19.475 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.472+0000 7fd2b0df9700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fd2ac0cf090 0x7fd2ac0b6ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:52336/0 (socket says 172.21.15.143:52336) 2023-11-08T19:30:19.475 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.472+0000 7fd2b0df9700 1 -- 172.21.15.143:0/2861720875 learned_addr learned my addr 172.21.15.143:0/2861720875 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:30:19.475 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.472+0000 7fd2b1dfb700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd2ac0d3690 0x7fd2ac0d1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:19.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.472+0000 7fd2a3fff700 1 -- 172.21.15.143:0/2861720875 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2750979990 0 0) 0x7fd2ac0d3200 con 0x7fd2ac0cf090 2023-11-08T19:30:19.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.472+0000 7fd2a3fff700 1 -- 172.21.15.143:0/2861720875 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2980035b0 con 0x7fd2ac0cf090 2023-11-08T19:30:19.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.472+0000 7fd2b1dfb700 1 -- 172.21.15.143:0/2861720875 >> v1:172.21.15.174:6789/0 conn(0x7fd2ac15dc80 legacy=0x7fd2ac0ce980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:19.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.472+0000 7fd2b1dfb700 1 -- 172.21.15.143:0/2861720875 >> v1:172.21.15.143:6789/0 conn(0x7fd2ac0cf090 legacy=0x7fd2ac0b6ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:19.477 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.472+0000 7fd2b1dfb700 1 -- 172.21.15.143:0/2861720875 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2ac0d28d0 con 0x7fd2ac0d3690 2023-11-08T19:30:19.477 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.472+0000 7fd2b1dfb700 1 --2- 172.21.15.143:0/2861720875 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd2ac0d3690 0x7fd2ac0d1f80 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7fd2a4009d90 tx=0x7fd2a400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6036890fb7eb4bc server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:19.477 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.472+0000 7fd2a3fff700 1 -- 172.21.15.143:0/2861720875 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a400e070 con 0x7fd2ac0d3690 2023-11-08T19:30:19.477 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.472+0000 7fd2a3fff700 1 -- 172.21.15.143:0/2861720875 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2a400bb20 con 0x7fd2ac0d3690 2023-11-08T19:30:19.477 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.472+0000 7fd2a3fff700 1 -- 172.21.15.143:0/2861720875 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a4012430 con 0x7fd2ac0d3690 2023-11-08T19:30:19.478 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.473+0000 7fd2b385e700 1 -- 172.21.15.143:0/2861720875 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd2ac0d3690 msgr2=0x7fd2ac0d1f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:19.478 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.473+0000 7fd2b385e700 1 --2- 172.21.15.143:0/2861720875 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd2ac0d3690 0x7fd2ac0d1f80 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7fd2a4009d90 tx=0x7fd2a400b330 comp rx=0 tx=0).stop 2023-11-08T19:30:19.478 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.473+0000 7fd2b385e700 1 -- 172.21.15.143:0/2861720875 shutdown_connections 2023-11-08T19:30:19.478 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.473+0000 7fd2b385e700 1 --2- 172.21.15.143:0/2861720875 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd2ac0d3690 0x7fd2ac0d1f80 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:19.478 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.473+0000 7fd2b385e700 1 -- 172.21.15.143:0/2861720875 >> 172.21.15.143:0/2861720875 conn(0x7fd2ac0668e0 msgr2=0x7fd2ac066cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:19.479 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.473+0000 7fd2b385e700 1 -- 172.21.15.143:0/2861720875 shutdown_connections 2023-11-08T19:30:19.479 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.473+0000 7fd2b385e700 1 -- 172.21.15.143:0/2861720875 wait complete. 2023-11-08T19:30:19.484 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.474+0000 7fd2b385e700 1 Processor -- start 2023-11-08T19:30:19.484 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.474+0000 7fd2b385e700 1 -- start start 2023-11-08T19:30:19.485 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.475+0000 7fd2b385e700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd2ac0cf090 0x7fd2ac0c9590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:19.485 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.475+0000 7fd2b385e700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd2ac0d3690 0x7fd2ac0cd970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:19.485 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.475+0000 7fd2b385e700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd2ac15dc80 0x7fd2ac15be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:19.485 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.475+0000 7fd2b385e700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2ac1f3930 con 0x7fd2ac0d3690 2023-11-08T19:30:19.485 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.475+0000 7fd2b385e700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2ac0c87d0 con 0x7fd2ac15dc80 2023-11-08T19:30:19.486 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.475+0000 7fd2b385e700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2ac0c8ab0 con 0x7fd2ac0cf090 2023-11-08T19:30:19.486 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.475+0000 7fd2b0df9700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd2ac0d3690 0x7fd2ac0cd970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:19.486 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.475+0000 7fd2b0df9700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd2ac0d3690 0x7fd2ac0cd970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56188/0 (socket says 172.21.15.143:56188) 2023-11-08T19:30:19.486 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.475+0000 7fd2b0df9700 1 -- 172.21.15.143:0/543303646 learned_addr learned my addr 172.21.15.143:0/543303646 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:30:19.486 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.475+0000 7fd2b15fa700 1 --2- 172.21.15.143:0/543303646 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd2ac0cf090 0x7fd2ac0c9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:19.487 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.475+0000 7fd2b1dfb700 1 --2- 172.21.15.143:0/543303646 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd2ac15dc80 0x7fd2ac15be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:19.487 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.475+0000 7fd2b0df9700 1 -- 172.21.15.143:0/543303646 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd2ac0cf090 msgr2=0x7fd2ac0c9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:19.487 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.475+0000 7fd2b0df9700 1 --2- 172.21.15.143:0/543303646 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd2ac0cf090 0x7fd2ac0c9590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:19.487 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.475+0000 7fd2b0df9700 1 -- 172.21.15.143:0/543303646 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd2ac15dc80 msgr2=0x7fd2ac15be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:19.487 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.475+0000 7fd2b0df9700 1 --2- 172.21.15.143:0/543303646 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd2ac15dc80 0x7fd2ac15be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:19.488 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.475+0000 7fd2b0df9700 1 -- 172.21.15.143:0/543303646 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2a4009a70 con 0x7fd2ac0d3690 2023-11-08T19:30:19.488 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.475+0000 7fd2b0df9700 1 --2- 172.21.15.143:0/543303646 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd2ac0d3690 0x7fd2ac0cd970 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fd2a800c700 tx=0x7fd2a800cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:19.488 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.476+0000 7fd2a1ffb700 1 -- 172.21.15.143:0/543303646 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a8005d70 con 0x7fd2ac0d3690 2023-11-08T19:30:19.488 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.476+0000 7fd2a1ffb700 1 -- 172.21.15.143:0/543303646 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2a80070d0 con 0x7fd2ac0d3690 2023-11-08T19:30:19.489 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.476+0000 7fd2b385e700 1 -- 172.21.15.143:0/543303646 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2ac0c8d90 con 0x7fd2ac0d3690 2023-11-08T19:30:19.489 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.476+0000 7fd2a1ffb700 1 -- 172.21.15.143:0/543303646 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a8007460 con 0x7fd2ac0d3690 2023-11-08T19:30:19.489 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.476+0000 7fd2b385e700 1 -- 172.21.15.143:0/543303646 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2ac14ad70 con 0x7fd2ac0d3690 2023-11-08T19:30:19.489 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.477+0000 7fd2a1ffb700 1 -- 172.21.15.143:0/543303646 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd2a80075c0 con 0x7fd2ac0d3690 2023-11-08T19:30:19.489 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.478+0000 7fd2a1ffb700 1 --2- 172.21.15.143:0/543303646 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd29006af00 0x7fd29006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:19.490 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.478+0000 7fd2a1ffb700 1 -- 172.21.15.143:0/543303646 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 7145+0+0 (secure 0 0 0) 0x7fd2a80891f0 con 0x7fd2ac0d3690 2023-11-08T19:30:19.490 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.478+0000 7fd2b15fa700 1 --2- 172.21.15.143:0/543303646 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd29006af00 0x7fd29006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:19.490 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.478+0000 7fd2b15fa700 1 --2- 172.21.15.143:0/543303646 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd29006af00 0x7fd29006d3c0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fd29c000f60 tx=0x7fd29c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:19.490 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.480+0000 7fd2bfa57b80 1 Processor -- start 2023-11-08T19:30:19.490 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.480+0000 7fd2bfa57b80 1 -- start start 2023-11-08T19:30:19.490 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.480+0000 7fd2bfa57b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5566357876f0 0x5566357b7450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:19.491 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.481+0000 7fd2bfa57b80 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x556635789c30 0x5566357b7990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:19.491 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.481+0000 7fd2bfa57b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556635787ad0 0x5566357a1f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:19.491 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.481+0000 7fd2bfa57b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566356f36e0 con 0x5566357876f0 2023-11-08T19:30:19.491 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.481+0000 7fd2bfa57b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566356e73b0 con 0x556635787ad0 2023-11-08T19:30:19.491 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.481+0000 7fd2bfa57b80 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x5566356e4cd0 con 0x556635789c30 2023-11-08T19:30:19.492 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.481+0000 7fd2b15fa700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5566357876f0 0x5566357b7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:19.492 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.481+0000 7fd2b15fa700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5566357876f0 0x5566357b7450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56194/0 (socket says 172.21.15.143:56194) 2023-11-08T19:30:19.492 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.481+0000 7fd2b15fa700 1 -- 172.21.15.143:0/28973852 learned_addr learned my addr 172.21.15.143:0/28973852 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:30:19.492 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.481+0000 7fd2b0df9700 1 --2- 172.21.15.143:0/28973852 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x556635789c30 0x5566357b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:19.492 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.481+0000 7fd2b1dfb700 1 --2- 172.21.15.143:0/28973852 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556635787ad0 0x5566357a1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:19.493 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.481+0000 7fd2b15fa700 1 -- 172.21.15.143:0/28973852 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x556635789c30 msgr2=0x5566357b7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:19.493 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.481+0000 7fd2b15fa700 1 --2- 172.21.15.143:0/28973852 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x556635789c30 0x5566357b7990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:19.493 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.481+0000 7fd2b15fa700 1 -- 172.21.15.143:0/28973852 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556635787ad0 msgr2=0x5566357a1f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:19.493 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.481+0000 7fd2b15fa700 1 --2- 172.21.15.143:0/28973852 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556635787ad0 0x5566357a1f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:19.493 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.481+0000 7fd2b15fa700 1 -- 172.21.15.143:0/28973852 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55663555d5e0 con 0x5566357876f0 2023-11-08T19:30:19.494 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.481+0000 7fd2b1dfb700 1 --2- 172.21.15.143:0/28973852 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556635787ad0 0x5566357a1f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:30:19.494 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.481+0000 7fd2b15fa700 1 --2- 172.21.15.143:0/28973852 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5566357876f0 0x5566357b7450 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fd29c0053e0 tx=0x7fd29c011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7aa95cc09f6d58db server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:19.494 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.481+0000 7fd296ffd700 1 -- 172.21.15.143:0/28973852 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd29c005410 con 0x5566357876f0 2023-11-08T19:30:19.494 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.481+0000 7fd296ffd700 1 -- 172.21.15.143:0/28973852 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd29c021450 con 0x5566357876f0 2023-11-08T19:30:19.494 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.481+0000 7fd296ffd700 1 -- 172.21.15.143:0/28973852 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd29c0217c0 con 0x5566357876f0 2023-11-08T19:30:19.495 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.482+0000 7fd2bfa57b80 1 -- 172.21.15.143:0/28973852 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5566357876f0 msgr2=0x5566357b7450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:19.495 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.482+0000 7fd2bfa57b80 1 --2- 172.21.15.143:0/28973852 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5566357876f0 0x5566357b7450 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fd29c0053e0 tx=0x7fd29c011000 comp rx=0 tx=0).stop 2023-11-08T19:30:19.495 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.482+0000 7fd2bfa57b80 1 -- 172.21.15.143:0/28973852 shutdown_connections 2023-11-08T19:30:19.495 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.482+0000 7fd2bfa57b80 1 --2- 172.21.15.143:0/28973852 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556635787ad0 0x5566357a1f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:19.495 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.482+0000 7fd2bfa57b80 1 --2- 172.21.15.143:0/28973852 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5566357876f0 0x5566357b7450 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:19.496 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.482+0000 7fd2bfa57b80 1 --2- 172.21.15.143:0/28973852 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x556635789c30 0x5566357b7990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:19.496 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.482+0000 7fd2bfa57b80 1 -- 172.21.15.143:0/28973852 >> 172.21.15.143:0/28973852 conn(0x556635702660 msgr2=0x55663578d3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:19.496 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.482+0000 7fd2bfa57b80 1 -- 172.21.15.143:0/28973852 shutdown_connections 2023-11-08T19:30:19.496 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.482+0000 7fd2bfa57b80 1 -- 172.21.15.143:0/28973852 wait complete. 2023-11-08T19:30:19.496 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.482+0000 7fd2bfa57b80 1 -- start start 2023-11-08T19:30:19.496 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.483+0000 7fd2bfa57b80 1 Processor -- start 2023-11-08T19:30:19.501 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.483+0000 7fd2bfa57b80 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x5566357876f0 0x5566357cb470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:19.501 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.483+0000 7fd2bfa57b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556635787ad0 0x5566357b6da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:19.501 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.483+0000 7fd2bfa57b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566357e4920 0x5566357b72e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:19.501 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.483+0000 7fd2bfa57b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566356e05c0 con 0x556635787ad0 2023-11-08T19:30:19.502 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.483+0000 7fd2bfa57b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55663567c0d0 con 0x5566357e4920 2023-11-08T19:30:19.502 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.483+0000 7fd2bfa57b80 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x55663567bd70 con 0x5566357876f0 2023-11-08T19:30:19.502 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.484+0000 7fd2bfa57b80 10 client.0 _ll_register_callbacks cb 0x5566354d87b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T19:30:19.502 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.484+0000 7fd2b0df9700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556635787ad0 0x5566357b6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:19.502 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.484+0000 7fd2b0df9700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556635787ad0 0x5566357b6da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56202/0 (socket says 172.21.15.143:56202) 2023-11-08T19:30:19.503 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.484+0000 7fd2b0df9700 1 -- 172.21.15.143:0/43134112 learned_addr learned my addr 172.21.15.143:0/43134112 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:30:19.503 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.484+0000 7fd2b15fa700 1 --2- 172.21.15.143:0/43134112 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x5566357876f0 0x5566357cb470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:19.503 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.484+0000 7fd2b1dfb700 1 --2- 172.21.15.143:0/43134112 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566357e4920 0x5566357b72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:19.503 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.484+0000 7fd2b0df9700 1 -- 172.21.15.143:0/43134112 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x5566357876f0 msgr2=0x5566357cb470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:19.503 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.484+0000 7fd2b0df9700 1 --2- 172.21.15.143:0/43134112 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x5566357876f0 0x5566357cb470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:19.504 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.484+0000 7fd2b0df9700 1 -- 172.21.15.143:0/43134112 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566357e4920 msgr2=0x5566357b72e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:19.504 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.484+0000 7fd2b0df9700 1 --2- 172.21.15.143:0/43134112 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566357e4920 0x5566357b72e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:19.504 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.484+0000 7fd2b0df9700 1 -- 172.21.15.143:0/43134112 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd29c00f040 con 0x556635787ad0 2023-11-08T19:30:19.504 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.484+0000 7fd2b1dfb700 1 --2- 172.21.15.143:0/43134112 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566357e4920 0x5566357b72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:30:19.504 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.484+0000 7fd2b0df9700 1 --2- 172.21.15.143:0/43134112 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556635787ad0 0x5566357b6da0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7fd2a8087170 tx=0x7fd2a8057e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baaee883778d4d31 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:19.504 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.484+0000 7fd28bfff700 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2023-11-08T19:30:19.505 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.484+0000 7fd28bfff700 1 -- 172.21.15.143:0/43134112 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a8057930 con 0x556635787ad0 2023-11-08T19:30:19.505 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.484+0000 7fd28bfff700 1 -- 172.21.15.143:0/43134112 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 7145+0+0 (secure 0 0 0) 0x7fd2a805ba80 con 0x556635787ad0 2023-11-08T19:30:19.505 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.484+0000 7fd2bfa57b80 1 -- 172.21.15.143:0/43134112 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55663577f880 con 0x556635787ad0 2023-11-08T19:30:19.505 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.484+0000 7fd28bfff700 1 -- 172.21.15.143:0/43134112 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd2a805fc30 con 0x556635787ad0 2023-11-08T19:30:19.505 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.484+0000 7fd2bfa57b80 10 client.7662 fetch_fsmap learned FSMap version 88 2023-11-08T19:30:19.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.484+0000 7fd2bfa57b80 1 -- 172.21.15.143:0/43134112 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=88,monmap=2+}) v3 -- 0x556635509d90 con 0x556635787ad0 2023-11-08T19:30:19.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.484+0000 7fd28bfff700 1 -- 172.21.15.143:0/43134112 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55663577f880 con 0x556635787ad0 2023-11-08T19:30:19.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.484+0000 7fd28bfff700 1 -- 172.21.15.143:0/43134112 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 88) v1 ==== 3306+0+0 (secure 0 0 0) 0x7fd2a80760c0 con 0x556635787ad0 2023-11-08T19:30:19.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.484+0000 7fd2bfa57b80 10 client.7662 fetch_fsmap finished waiting for FSMap version 88 2023-11-08T19:30:19.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.485+0000 7fd2bfa57b80 10 client.7662 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-08T19:30:19.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.485+0000 7fd2bfa57b80 20 client.7662 populate_metadata read hostname 'smithi143' 2023-11-08T19:30:19.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.485+0000 7fd2bfa57b80 1 --2- 172.21.15.143:0/43134112 >> [v2:172.21.15.143:6832/1535103237,v1:172.21.15.143:6834/1535103237] conn(0x5566357ca100 0x5566357e1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:19.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.485+0000 7fd2bfa57b80 4 client.7662 mds_command: new command op to 7494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-08T19:30:19.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.485+0000 7fd2bfa57b80 1 -- 172.21.15.143:0/43134112 --> [v2:172.21.15.143:6832/1535103237,v1:172.21.15.143:6834/1535103237] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55663575ba30 con 0x5566357ca100 2023-11-08T19:30:19.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.485+0000 7fd2b15fa700 1 --2- 172.21.15.143:0/43134112 >> [v2:172.21.15.143:6832/1535103237,v1:172.21.15.143:6834/1535103237] conn(0x5566357ca100 0x5566357e1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:19.508 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.485+0000 7fd28bfff700 10 client.7662 ms_handle_connect on v2:172.21.15.143:6832/1535103237 2023-11-08T19:30:19.508 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.485+0000 7fd2b15fa700 1 --2- 172.21.15.143:0/43134112 >> [v2:172.21.15.143:6832/1535103237,v1:172.21.15.143:6834/1535103237] conn(0x5566357ca100 0x5566357e1ae0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36bf919ddae4ad21 server_cookie=822429849fb51ed6 in_seq=0 out_seq=0 2023-11-08T19:30:19.508 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.486+0000 7fd28bfff700 1 -- 172.21.15.143:0/43134112 <== mds.0 v2:172.21.15.143:6832/1535103237 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55663575ba30 con 0x5566357ca100 2023-11-08T19:30:19.508 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.486+0000 7fd28bfff700 10 client.7662 handle_command_reply: tid=0 2023-11-08T19:30:19.508 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.486+0000 7fd2bfa57b80 1 client.7662 shutdown 2023-11-08T19:30:19.508 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.486+0000 7fd2bfa57b80 1 -- 172.21.15.143:0/43134112 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556635787ad0 msgr2=0x5566357b6da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:19.509 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.486+0000 7fd2bfa57b80 1 --2- 172.21.15.143:0/43134112 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556635787ad0 0x5566357b6da0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7fd2a8087170 tx=0x7fd2a8057e70 comp rx=0 tx=0).stop 2023-11-08T19:30:19.509 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.486+0000 7fd2bfa57b80 1 -- 172.21.15.143:0/43134112 shutdown_connections 2023-11-08T19:30:19.509 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.486+0000 7fd2bfa57b80 1 --2- 172.21.15.143:0/43134112 >> [v2:172.21.15.143:6832/1535103237,v1:172.21.15.143:6834/1535103237] conn(0x5566357ca100 0x5566357e1ae0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:19.509 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.486+0000 7fd2bfa57b80 1 --2- 172.21.15.143:0/43134112 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x5566357876f0 0x5566357cb470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:19.509 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.486+0000 7fd28bfff700 0 client.7662 ms_handle_reset on v2:172.21.15.143:6832/1535103237 2023-11-08T19:30:19.510 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.486+0000 7fd2bfa57b80 1 --2- 172.21.15.143:0/43134112 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566357e4920 0x5566357b72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:19.510 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.486+0000 7fd2bfa57b80 1 --2- 172.21.15.143:0/43134112 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556635787ad0 0x5566357b6da0 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:19.510 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.486+0000 7fd2bfa57b80 1 -- 172.21.15.143:0/43134112 >> 172.21.15.143:0/43134112 conn(0x556635702660 msgr2=0x5566357a2e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:19.510 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.486+0000 7fd2bfa57b80 1 -- 172.21.15.143:0/43134112 shutdown_connections 2023-11-08T19:30:19.510 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.486+0000 7fd2bfa57b80 1 -- 172.21.15.143:0/43134112 wait complete. 2023-11-08T19:30:19.511 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.486+0000 7fd2bfa57b80 20 client.7662 trim_cache size 0 max 16384 2023-11-08T19:30:19.517 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.504+0000 7fd2bfa57b80 1 Processor -- start 2023-11-08T19:30:19.517 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.504+0000 7fd2bfa57b80 1 -- start start 2023-11-08T19:30:19.517 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.505+0000 7fd2bfa57b80 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x5566357876f0 0x5566357ba3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:19.517 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.505+0000 7fd2bfa57b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556635787ad0 0x5566357ba910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:19.518 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.505+0000 7fd2bfa57b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5566357e4920 0x5566357ead30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:19.518 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.505+0000 7fd2bfa57b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x55663555f8d0 con 0x5566357e4920 2023-11-08T19:30:19.518 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.505+0000 7fd2bfa57b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566357b8630 con 0x556635787ad0 2023-11-08T19:30:19.518 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.505+0000 7fd2bfa57b80 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x5566357b89c0 con 0x5566357876f0 2023-11-08T19:30:19.518 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.505+0000 7fd2b1dfb700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5566357e4920 0x5566357ead30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:19.519 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.505+0000 7fd2b1dfb700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5566357e4920 0x5566357ead30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56212/0 (socket says 172.21.15.143:56212) 2023-11-08T19:30:19.519 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.505+0000 7fd2b1dfb700 1 -- 172.21.15.143:0/3950835893 learned_addr learned my addr 172.21.15.143:0/3950835893 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:30:19.519 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.505+0000 7fd2b15fa700 1 --2- 172.21.15.143:0/3950835893 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x5566357876f0 0x5566357ba3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:19.519 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.505+0000 7fd2b0df9700 1 --2- 172.21.15.143:0/3950835893 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556635787ad0 0x5566357ba910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:19.519 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.505+0000 7fd2b1dfb700 1 -- 172.21.15.143:0/3950835893 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x5566357876f0 msgr2=0x5566357ba3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:19.520 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.505+0000 7fd2b1dfb700 1 --2- 172.21.15.143:0/3950835893 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x5566357876f0 0x5566357ba3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:19.520 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.505+0000 7fd2b1dfb700 1 -- 172.21.15.143:0/3950835893 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556635787ad0 msgr2=0x5566357ba910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:19.520 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.505+0000 7fd2b1dfb700 1 --2- 172.21.15.143:0/3950835893 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556635787ad0 0x5566357ba910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:19.520 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.505+0000 7fd2b1dfb700 1 -- 172.21.15.143:0/3950835893 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5566354f2c10 con 0x5566357e4920 2023-11-08T19:30:19.520 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.505+0000 7fd2b0df9700 1 --2- 172.21.15.143:0/3950835893 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556635787ad0 0x5566357ba910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:30:19.520 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.505+0000 7fd2b1dfb700 1 --2- 172.21.15.143:0/3950835893 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5566357e4920 0x5566357ead30 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac0b67a0 tx=0x7fd2a4010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f86fa9f474430f25 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:19.521 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.505+0000 7fd2977fe700 1 -- 172.21.15.143:0/3950835893 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a4003ce0 con 0x5566357e4920 2023-11-08T19:30:19.521 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.505+0000 7fd2977fe700 1 -- 172.21.15.143:0/3950835893 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2a4003e40 con 0x5566357e4920 2023-11-08T19:30:19.521 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.505+0000 7fd2977fe700 1 -- 172.21.15.143:0/3950835893 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a4012800 con 0x5566357e4920 2023-11-08T19:30:19.521 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.506+0000 7fd2bfa57b80 1 -- 172.21.15.143:0/3950835893 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5566357e4920 msgr2=0x5566357ead30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:19.521 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.506+0000 7fd2bfa57b80 1 --2- 172.21.15.143:0/3950835893 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5566357e4920 0x5566357ead30 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac0b67a0 tx=0x7fd2a4010000 comp rx=0 tx=0).stop 2023-11-08T19:30:19.522 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.506+0000 7fd2bfa57b80 1 -- 172.21.15.143:0/3950835893 shutdown_connections 2023-11-08T19:30:19.522 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.506+0000 7fd2bfa57b80 1 --2- 172.21.15.143:0/3950835893 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x5566357876f0 0x5566357ba3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:19.522 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.506+0000 7fd2bfa57b80 1 --2- 172.21.15.143:0/3950835893 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5566357e4920 0x5566357ead30 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:19.522 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.506+0000 7fd2bfa57b80 1 --2- 172.21.15.143:0/3950835893 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556635787ad0 0x5566357ba910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:19.522 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.506+0000 7fd2bfa57b80 1 -- 172.21.15.143:0/3950835893 >> 172.21.15.143:0/3950835893 conn(0x556635702660 msgr2=0x55663578d390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:19.523 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.506+0000 7fd2bfa57b80 1 -- 172.21.15.143:0/3950835893 shutdown_connections 2023-11-08T19:30:19.523 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.506+0000 7fd2bfa57b80 1 -- 172.21.15.143:0/3950835893 wait complete. 2023-11-08T19:30:19.523 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.507+0000 7fd2bfa57b80 1 -- start start 2023-11-08T19:30:19.523 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.507+0000 7fd2bfa57b80 1 Processor -- start 2023-11-08T19:30:19.523 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.507+0000 7fd2bfa57b80 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x5566357876f0 0x5566357c89b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:19.524 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.508+0000 7fd2bfa57b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556635787ad0 0x5566357ca4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:19.524 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.508+0000 7fd2bfa57b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566357e4920 0x5566357caa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:19.524 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.508+0000 7fd2bfa57b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566357b55c0 con 0x556635787ad0 2023-11-08T19:30:19.524 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.508+0000 7fd2bfa57b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566357c8ef0 con 0x5566357e4920 2023-11-08T19:30:19.524 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.508+0000 7fd2bfa57b80 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x5566357f4430 con 0x5566357876f0 2023-11-08T19:30:19.525 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.508+0000 7fd2bfa57b80 10 client.0 _ll_register_callbacks cb 0x7fd2a8057470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T19:30:19.525 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.508+0000 7fd2b0df9700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556635787ad0 0x5566357ca4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:19.525 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.508+0000 7fd2b0df9700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556635787ad0 0x5566357ca4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56216/0 (socket says 172.21.15.143:56216) 2023-11-08T19:30:19.525 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.508+0000 7fd2b0df9700 1 -- 172.21.15.143:0/3967573819 learned_addr learned my addr 172.21.15.143:0/3967573819 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:30:19.525 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.508+0000 7fd2b15fa700 1 --2- 172.21.15.143:0/3967573819 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x5566357876f0 0x5566357c89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:19.525 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.508+0000 7fd2b1dfb700 1 --2- 172.21.15.143:0/3967573819 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566357e4920 0x5566357caa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:19.526 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.508+0000 7fd2b0df9700 1 -- 172.21.15.143:0/3967573819 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x5566357876f0 msgr2=0x5566357c89b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:19.526 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.508+0000 7fd2b0df9700 1 --2- 172.21.15.143:0/3967573819 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x5566357876f0 0x5566357c89b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:19.526 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.508+0000 7fd2b0df9700 1 -- 172.21.15.143:0/3967573819 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566357e4920 msgr2=0x5566357caa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:19.526 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.508+0000 7fd2b0df9700 1 --2- 172.21.15.143:0/3967573819 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566357e4920 0x5566357caa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:19.526 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.508+0000 7fd2b0df9700 1 -- 172.21.15.143:0/3967573819 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2ac0d28d0 con 0x556635787ad0 2023-11-08T19:30:19.527 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.508+0000 7fd2b0df9700 1 --2- 172.21.15.143:0/3967573819 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556635787ad0 0x5566357ca4e0 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7fd2a80578c0 tx=0x7fd2a8001150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24a889aad72ab3af server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:19.527 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.508+0000 7fd28bfff700 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2023-11-08T19:30:19.527 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.508+0000 7fd28bfff700 1 -- 172.21.15.143:0/3967573819 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a8083020 con 0x556635787ad0 2023-11-08T19:30:19.527 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.508+0000 7fd28bfff700 1 -- 172.21.15.143:0/3967573819 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 7145+0+0 (secure 0 0 0) 0x7fd2a805ba80 con 0x556635787ad0 2023-11-08T19:30:19.527 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.508+0000 7fd2bfa57b80 1 -- 172.21.15.143:0/3967573819 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd29c021450 con 0x556635787ad0 2023-11-08T19:30:19.528 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.508+0000 7fd28bfff700 1 -- 172.21.15.143:0/3967573819 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd2a800e040 con 0x556635787ad0 2023-11-08T19:30:19.528 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.508+0000 7fd2bfa57b80 10 client.7668 fetch_fsmap learned FSMap version 88 2023-11-08T19:30:19.528 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.508+0000 7fd2bfa57b80 1 -- 172.21.15.143:0/3967573819 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=88,monmap=2+}) v3 -- 0x556635530200 con 0x556635787ad0 2023-11-08T19:30:19.528 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.508+0000 7fd28bfff700 1 -- 172.21.15.143:0/3967573819 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd29c021450 con 0x556635787ad0 2023-11-08T19:30:19.528 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.509+0000 7fd28bfff700 1 -- 172.21.15.143:0/3967573819 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 88) v1 ==== 3306+0+0 (secure 0 0 0) 0x7fd2a8053720 con 0x556635787ad0 2023-11-08T19:30:19.529 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.509+0000 7fd2bfa57b80 10 client.7668 fetch_fsmap finished waiting for FSMap version 88 2023-11-08T19:30:19.529 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.509+0000 7fd2bfa57b80 10 client.7668 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-08T19:30:19.529 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.509+0000 7fd2bfa57b80 20 client.7668 populate_metadata read hostname 'smithi143' 2023-11-08T19:30:19.529 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.509+0000 7fd2bfa57b80 1 --2- 172.21.15.143:0/3967573819 >> [v2:172.21.15.143:6832/1535103237,v1:172.21.15.143:6834/1535103237] conn(0x55663583b450 0x55663583b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:19.529 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.509+0000 7fd2bfa57b80 4 client.7668 mds_command: new command op to 7494 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-08T19:30:19.529 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.509+0000 7fd2bfa57b80 1 -- 172.21.15.143:0/3967573819 --> [v2:172.21.15.143:6832/1535103237,v1:172.21.15.143:6834/1535103237] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5566356d95a0 con 0x55663583b450 2023-11-08T19:30:19.530 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.509+0000 7fd2b15fa700 1 --2- 172.21.15.143:0/3967573819 >> [v2:172.21.15.143:6832/1535103237,v1:172.21.15.143:6834/1535103237] conn(0x55663583b450 0x55663583b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:19.530 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.509+0000 7fd28bfff700 10 client.7668 ms_handle_connect on v2:172.21.15.143:6832/1535103237 2023-11-08T19:30:19.530 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.509+0000 7fd2b15fa700 1 --2- 172.21.15.143:0/3967573819 >> [v2:172.21.15.143:6832/1535103237,v1:172.21.15.143:6834/1535103237] conn(0x55663583b450 0x55663583b830 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=804080ca948f2922 server_cookie=e353d3509b610216 in_seq=0 out_seq=0 2023-11-08T19:30:19.530 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.509+0000 7fd28bfff700 1 -- 172.21.15.143:0/3967573819 <== mds.0 v2:172.21.15.143:6832/1535103237 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x5566356d95a0 con 0x55663583b450 2023-11-08T19:30:19.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.509+0000 7fd28bfff700 10 client.7668 handle_command_reply: tid=0 2023-11-08T19:30:19.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.509+0000 7fd2bfa57b80 1 client.7668 shutdown 2023-11-08T19:30:19.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.509+0000 7fd2bfa57b80 1 -- 172.21.15.143:0/3967573819 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556635787ad0 msgr2=0x5566357ca4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:19.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.509+0000 7fd2bfa57b80 1 --2- 172.21.15.143:0/3967573819 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556635787ad0 0x5566357ca4e0 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7fd2a80578c0 tx=0x7fd2a8001150 comp rx=0 tx=0).stop 2023-11-08T19:30:19.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.509+0000 7fd2bfa57b80 1 -- 172.21.15.143:0/3967573819 shutdown_connections 2023-11-08T19:30:19.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.509+0000 7fd2bfa57b80 1 --2- 172.21.15.143:0/3967573819 >> [v2:172.21.15.143:6832/1535103237,v1:172.21.15.143:6834/1535103237] conn(0x55663583b450 0x55663583b830 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:19.532 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.509+0000 7fd2bfa57b80 1 --2- 172.21.15.143:0/3967573819 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x5566357876f0 0x5566357c89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:19.532 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.509+0000 7fd28bfff700 0 client.7668 ms_handle_reset on v2:172.21.15.143:6832/1535103237 2023-11-08T19:30:19.532 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.509+0000 7fd2bfa57b80 1 --2- 172.21.15.143:0/3967573819 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566357e4920 0x5566357caa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:19.532 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.509+0000 7fd2bfa57b80 1 --2- 172.21.15.143:0/3967573819 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556635787ad0 0x5566357ca4e0 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:19.532 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.509+0000 7fd2bfa57b80 1 -- 172.21.15.143:0/3967573819 >> 172.21.15.143:0/3967573819 conn(0x556635702660 msgr2=0x55663578d390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:19.533 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.510+0000 7fd2bfa57b80 1 -- 172.21.15.143:0/3967573819 shutdown_connections 2023-11-08T19:30:19.533 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.510+0000 7fd2bfa57b80 1 -- 172.21.15.143:0/3967573819 wait complete. 2023-11-08T19:30:19.533 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.510+0000 7fd2bfa57b80 20 client.7668 trim_cache size 0 max 16384 2023-11-08T19:30:19.533 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2023-11-08T19:30:19.534 INFO:teuthology.orchestra.run.smithi143.stdout: "cluster_fsid": "968483af-77fc-4068-ad7c-5c2a1a4613d7", 2023-11-08T19:30:19.534 INFO:teuthology.orchestra.run.smithi143.stdout: "whoami": 0, 2023-11-08T19:30:19.534 INFO:teuthology.orchestra.run.smithi143.stdout: "id": 7494, 2023-11-08T19:30:19.534 INFO:teuthology.orchestra.run.smithi143.stdout: "want_state": "up:active", 2023-11-08T19:30:19.535 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "up:active", 2023-11-08T19:30:19.535 INFO:teuthology.orchestra.run.smithi143.stdout: "fs_name": "cephfs", 2023-11-08T19:30:19.535 INFO:teuthology.orchestra.run.smithi143.stdout: "rank_uptime": 6.9346530849999999, 2023-11-08T19:30:19.535 INFO:teuthology.orchestra.run.smithi143.stdout: "mdsmap_epoch": 88, 2023-11-08T19:30:19.535 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_epoch": 98, 2023-11-08T19:30:19.535 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_epoch_barrier": 98, 2023-11-08T19:30:19.536 INFO:teuthology.orchestra.run.smithi143.stdout: "uptime": 38.252622877999997 2023-11-08T19:30:19.536 INFO:teuthology.orchestra.run.smithi143.stdout:} 2023-11-08T19:30:19.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.510+0000 7fd2b385e700 1 -- 172.21.15.143:0/543303646 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd29006af00 msgr2=0x7fd29006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:19.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.510+0000 7fd2b385e700 1 --2- 172.21.15.143:0/543303646 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd29006af00 0x7fd29006d3c0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fd29c000f60 tx=0x7fd29c009040 comp rx=0 tx=0).stop 2023-11-08T19:30:19.538 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.510+0000 7fd2b385e700 1 -- 172.21.15.143:0/543303646 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd2ac0d3690 msgr2=0x7fd2ac0cd970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:19.538 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.510+0000 7fd2b385e700 1 --2- 172.21.15.143:0/543303646 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd2ac0d3690 0x7fd2ac0cd970 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fd2a800c700 tx=0x7fd2a800cac0 comp rx=0 tx=0).stop 2023-11-08T19:30:19.538 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.510+0000 7fd2b385e700 1 -- 172.21.15.143:0/543303646 shutdown_connections 2023-11-08T19:30:19.538 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.510+0000 7fd2b385e700 1 --2- 172.21.15.143:0/543303646 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd29006af00 0x7fd29006d3c0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:19.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.510+0000 7fd2b385e700 1 --2- 172.21.15.143:0/543303646 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd2ac0cf090 0x7fd2ac0c9590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:19.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.510+0000 7fd2b385e700 1 --2- 172.21.15.143:0/543303646 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd2ac15dc80 0x7fd2ac15be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:19.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.510+0000 7fd2b385e700 1 --2- 172.21.15.143:0/543303646 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd2ac0d3690 0x7fd2ac0cd970 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:19.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.510+0000 7fd2b385e700 1 -- 172.21.15.143:0/543303646 >> 172.21.15.143:0/543303646 conn(0x7fd2ac0668e0 msgr2=0x7fd2ac15cc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:19.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.510+0000 7fd2b385e700 1 -- 172.21.15.143:0/543303646 shutdown_connections 2023-11-08T19:30:19.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.510+0000 7fd2b385e700 1 -- 172.21.15.143:0/543303646 wait complete. 2023-11-08T19:30:19.540 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:30:19.660 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.659+0000 7f1250905700 1 Processor -- start 2023-11-08T19:30:19.661 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.659+0000 7f1250905700 1 -- start start 2023-11-08T19:30:19.663 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.660+0000 7f1250905700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f124c072860 0x7f124c160170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:19.663 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.660+0000 7f1250905700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f124c06e9a0 0x7f124c0b3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:19.664 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.660+0000 7f1250905700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f124c06bd50 con 0x7f124c06dbb0 2023-11-08T19:30:19.664 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.660+0000 7f1250905700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f124c0cb1c0 con 0x7f124c06e9a0 2023-11-08T19:30:19.664 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.660+0000 7f1250905700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f124c06bff0 con 0x7f124c072860 2023-11-08T19:30:19.664 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.660+0000 7f124ad9d700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f124c06e9a0 0x7f124c0b3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:19.665 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.660+0000 7f124ad9d700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f124c06e9a0 0x7f124c0b3240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56224/0 (socket says 172.21.15.143:56224) 2023-11-08T19:30:19.665 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.660+0000 7f124ad9d700 1 -- 172.21.15.143:0/2823777888 learned_addr learned my addr 172.21.15.143:0/2823777888 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:30:19.665 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.661+0000 7f1249d9b700 1 --2- 172.21.15.143:0/2823777888 >> v2:172.21.15.174:3300/0 conn(0x7f124c072860 0x7f124c160170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:19.665 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.661+0000 7f124ad9d700 1 -- 172.21.15.143:0/2823777888 >> v2:172.21.15.174:3300/0 conn(0x7f124c072860 msgr2=0x7f124c160170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:19.665 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.661+0000 7f124ad9d700 1 --2- 172.21.15.143:0/2823777888 >> v2:172.21.15.174:3300/0 conn(0x7f124c072860 0x7f124c160170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:19.665 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.661+0000 7f124ad9d700 1 -- 172.21.15.143:0/2823777888 >> v1:172.21.15.174:6789/0 conn(0x7f124c06dbb0 legacy=0x7f124c15fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:19.666 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.661+0000 7f124ad9d700 1 -- 172.21.15.143:0/2823777888 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f124c06c2d0 con 0x7f124c06e9a0 2023-11-08T19:30:19.666 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.661+0000 7f124ad9d700 1 --2- 172.21.15.143:0/2823777888 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f124c06e9a0 0x7f124c0b3240 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f124c0ca270 tx=0x7f124400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d093bc476410699b server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:19.666 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.661+0000 7f124959a700 1 -- 172.21.15.143:0/2823777888 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f124400e070 con 0x7f124c06e9a0 2023-11-08T19:30:19.666 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.661+0000 7f124959a700 1 -- 172.21.15.143:0/2823777888 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f124400b9c0 con 0x7f124c06e9a0 2023-11-08T19:30:19.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.661+0000 7f124959a700 1 -- 172.21.15.143:0/2823777888 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f124400bd70 con 0x7f124c06e9a0 2023-11-08T19:30:19.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.661+0000 7f1250905700 1 -- 172.21.15.143:0/2823777888 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f124c06e9a0 msgr2=0x7f124c0b3240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:19.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.661+0000 7f1250905700 1 --2- 172.21.15.143:0/2823777888 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f124c06e9a0 0x7f124c0b3240 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f124c0ca270 tx=0x7f124400b330 comp rx=0 tx=0).stop 2023-11-08T19:30:19.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.662+0000 7f1250905700 1 -- 172.21.15.143:0/2823777888 shutdown_connections 2023-11-08T19:30:19.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.662+0000 7f1250905700 1 --2- 172.21.15.143:0/2823777888 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f124c06e9a0 0x7f124c0b3240 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:19.668 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.662+0000 7f1250905700 1 --2- 172.21.15.143:0/2823777888 >> v2:172.21.15.174:3300/0 conn(0x7f124c072860 0x7f124c160170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:19.668 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.662+0000 7f1250905700 1 -- 172.21.15.143:0/2823777888 >> 172.21.15.143:0/2823777888 conn(0x7f124c0b1a20 msgr2=0x7f124c0b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:19.668 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.662+0000 7f1250905700 1 -- 172.21.15.143:0/2823777888 shutdown_connections 2023-11-08T19:30:19.668 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.662+0000 7f1250905700 1 -- 172.21.15.143:0/2823777888 wait complete. 2023-11-08T19:30:19.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.663+0000 7f1250905700 1 Processor -- start 2023-11-08T19:30:19.672 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.663+0000 7f1250905700 1 -- start start 2023-11-08T19:30:19.672 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.664+0000 7f1250905700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f124c06dbb0 0x7f124c0cf1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:19.672 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.664+0000 7f1250905700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f124c06e9a0 0x7f124c0c3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:19.672 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.664+0000 7f1250905700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f124c072860 0x7f124c0cd9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:19.672 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.664+0000 7f1250905700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f124c0ae400 con 0x7f124c06dbb0 2023-11-08T19:30:19.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.664+0000 7f1250905700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f124c0ae0d0 con 0x7f124c06e9a0 2023-11-08T19:30:19.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.664+0000 7f1250905700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f124c0cdf30 con 0x7f124c072860 2023-11-08T19:30:19.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.664+0000 7f124a59c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f124c06dbb0 0x7f124c0cf1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:19.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.664+0000 7f124a59c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f124c06dbb0 0x7f124c0cf1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56228/0 (socket says 172.21.15.143:56228) 2023-11-08T19:30:19.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.664+0000 7f124a59c700 1 -- 172.21.15.143:0/3345332647 learned_addr learned my addr 172.21.15.143:0/3345332647 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:30:19.674 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.664+0000 7f1249d9b700 1 --2- 172.21.15.143:0/3345332647 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f124c06e9a0 0x7f124c0c3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:19.674 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.664+0000 7f124ad9d700 1 --2- 172.21.15.143:0/3345332647 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f124c072860 0x7f124c0cd9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:19.674 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.664+0000 7f124a59c700 1 -- 172.21.15.143:0/3345332647 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f124c072860 msgr2=0x7f124c0cd9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:19.674 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.664+0000 7f124a59c700 1 --2- 172.21.15.143:0/3345332647 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f124c072860 0x7f124c0cd9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:19.674 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.664+0000 7f124a59c700 1 -- 172.21.15.143:0/3345332647 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f124c06e9a0 msgr2=0x7f124c0c3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:19.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.664+0000 7f124a59c700 1 --2- 172.21.15.143:0/3345332647 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f124c06e9a0 0x7f124c0c3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:19.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.664+0000 7f124a59c700 1 -- 172.21.15.143:0/3345332647 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1244009b30 con 0x7f124c06dbb0 2023-11-08T19:30:19.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.664+0000 7f1249d9b700 1 --2- 172.21.15.143:0/3345332647 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f124c06e9a0 0x7f124c0c3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:30:19.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.664+0000 7f124a59c700 1 --2- 172.21.15.143:0/3345332647 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f124c06dbb0 0x7f124c0cf1d0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f124000c4e0 tx=0x7f124000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:19.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.664+0000 7f123affd700 1 -- 172.21.15.143:0/3345332647 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12400052d0 con 0x7f124c06dbb0 2023-11-08T19:30:19.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.664+0000 7f123affd700 1 -- 172.21.15.143:0/3345332647 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1240005430 con 0x7f124c06dbb0 2023-11-08T19:30:19.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.664+0000 7f1250905700 1 -- 172.21.15.143:0/3345332647 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f124c0ce1d0 con 0x7f124c06dbb0 2023-11-08T19:30:19.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.664+0000 7f123affd700 1 -- 172.21.15.143:0/3345332647 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12400038c0 con 0x7f124c06dbb0 2023-11-08T19:30:19.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.664+0000 7f1250905700 1 -- 172.21.15.143:0/3345332647 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f124c0ce790 con 0x7f124c06dbb0 2023-11-08T19:30:19.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.665+0000 7f1250905700 1 -- 172.21.15.143:0/3345332647 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f124c0ce9b0 con 0x7f124c06dbb0 2023-11-08T19:30:19.677 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.666+0000 7f123affd700 1 -- 172.21.15.143:0/3345332647 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1240003a20 con 0x7f124c06dbb0 2023-11-08T19:30:19.677 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.666+0000 7f123affd700 1 --2- 172.21.15.143:0/3345332647 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f122c06af00 0x7f122c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:19.677 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.666+0000 7f123affd700 1 -- 172.21.15.143:0/3345332647 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 7145+0+0 (secure 0 0 0) 0x7f1240089050 con 0x7f124c06dbb0 2023-11-08T19:30:19.677 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.666+0000 7f1249d9b700 1 --2- 172.21.15.143:0/3345332647 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f122c06af00 0x7f122c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:19.677 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.667+0000 7f1249d9b700 1 --2- 172.21.15.143:0/3345332647 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f122c06af00 0x7f122c06d3c0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f124c0ced20 tx=0x7f123c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:19.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.670+0000 7f123affd700 1 -- 172.21.15.143:0/3345332647 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1240053b60 con 0x7f124c06dbb0 2023-11-08T19:30:19.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.909+0000 7f1250905700 1 -- 172.21.15.143:0/3345332647 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f124c04eee0 con 0x7f124c06dbb0 2023-11-08T19:30:19.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.910+0000 7f123affd700 1 -- 172.21.15.143:0/3345332647 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 88 v88) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f1240057180 con 0x7f124c06dbb0 2023-11-08T19:30:19.911 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:30:19.912 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":88,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5933,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/3918286710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6835","nonce":3918286710},{"type":"v1","addr":"172.21.15.174:6837","nonce":3918286710}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":6161,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/2129643800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":2129643800},{"type":"v1","addr":"172.21.15.143:6835","nonce":2129643800}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":88,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:03.092245+0000","modified":"2023-11-08T19:30:15.587914+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":98,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7494,"mds_1":6918},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6918":{"gid":6918,"name":"b","rank":1,"incarnation":54,"state":"up:active","state_seq":6,"addr":"172.21.15.174:6836/3601088618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3601088618},{"type":"v1","addr":"172.21.15.174:6836","nonce":3601088618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7494":{"gid":7494,"name":"c","rank":0,"incarnation":84,"state":"up:active","state_seq":12,"addr":"172.21.15.143:6834/1535103237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":1535103237},{"type":"v1","addr":"172.21.15.143:6834","nonce":1535103237}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:30:19.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.912+0000 7f1250905700 1 -- 172.21.15.143:0/3345332647 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f122c06af00 msgr2=0x7f122c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:19.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.912+0000 7f1250905700 1 --2- 172.21.15.143:0/3345332647 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f122c06af00 0x7f122c06d3c0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f124c0ced20 tx=0x7f123c008040 comp rx=0 tx=0).stop 2023-11-08T19:30:19.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.912+0000 7f1250905700 1 -- 172.21.15.143:0/3345332647 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f124c06dbb0 msgr2=0x7f124c0cf1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:19.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.912+0000 7f1250905700 1 --2- 172.21.15.143:0/3345332647 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f124c06dbb0 0x7f124c0cf1d0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f124000c4e0 tx=0x7f124000c8a0 comp rx=0 tx=0).stop 2023-11-08T19:30:19.915 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.912+0000 7f1250905700 1 -- 172.21.15.143:0/3345332647 shutdown_connections 2023-11-08T19:30:19.915 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.912+0000 7f1250905700 1 --2- 172.21.15.143:0/3345332647 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f122c06af00 0x7f122c06d3c0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:19.915 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.912+0000 7f1250905700 1 --2- 172.21.15.143:0/3345332647 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f124c072860 0x7f124c0cd9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:19.915 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.912+0000 7f1250905700 1 --2- 172.21.15.143:0/3345332647 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f124c06dbb0 0x7f124c0cf1d0 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:19.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.912+0000 7f1250905700 1 --2- 172.21.15.143:0/3345332647 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f124c06e9a0 0x7f124c0c3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:19.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.912+0000 7f1250905700 1 -- 172.21.15.143:0/3345332647 >> 172.21.15.143:0/3345332647 conn(0x7f124c0b1a20 msgr2=0x7f124c0c9080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:19.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.912+0000 7f1250905700 1 -- 172.21.15.143:0/3345332647 shutdown_connections 2023-11-08T19:30:19.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:19.912+0000 7f1250905700 1 -- 172.21.15.143:0/3345332647 wait complete. 2023-11-08T19:30:19.917 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 88 2023-11-08T19:30:19.926 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json dump snaps --server 2023-11-08T19:30:20.062 INFO:teuthology.orchestra.run.smithi143.stdout:{"last_snap":5,"last_created":5,"last_destroyed":1,"pending_noop":[],"snaps":[{"snapid":2,"ino":1099511627777,"stamp":"2023-11-08T19:28:31.007395+0000","name":"s11","metadata":{}},{"snapid":3,"ino":1099511627777,"stamp":"2023-11-08T19:28:57.577300+0000","name":"s12","metadata":{}},{"snapid":4,"ino":1099511627777,"stamp":"2023-11-08T19:29:23.827405+0000","name":"s15","metadata":{}},{"snapid":5,"ino":1099511627777,"stamp":"2023-11-08T19:29:49.829697+0000","name":"s16","metadata":{}}],"need_to_purge":{},"pending_update":[],"pending_destroy":[]} 2023-11-08T19:30:20.063 DEBUG:tasks.cephfs.filesystem:_json_asok output { "last_created": 5, "last_destroyed": 1, "last_snap": 5, "need_to_purge": {}, "pending_destroy": [], "pending_noop": [], "pending_update": [], "snaps": [ { "ino": 1099511627777, "metadata": {}, "name": "s11", "snapid": 2, "stamp": "2023-11-08T19:28:31.007395+0000" }, { "ino": 1099511627777, "metadata": {}, "name": "s12", "snapid": 3, "stamp": "2023-11-08T19:28:57.577300+0000" }, { "ino": 1099511627777, "metadata": {}, "name": "s15", "snapid": 4, "stamp": "2023-11-08T19:29:23.827405+0000" }, { "ino": 1099511627777, "metadata": {}, "name": "s16", "snapid": 5, "stamp": "2023-11-08T19:29:49.829697+0000" } ] } 2023-11-08T19:30:20.063 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-11-08T19:30:20.064 DEBUG:teuthology.orchestra.run.smithi143:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-08T19:30:20.067 DEBUG:teuthology.orchestra.run.smithi174:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-08T19:30:20.110 INFO:teuthology.orchestra.run.smithi143.stdout:[global] 2023-11-08T19:30:20.111 INFO:teuthology.orchestra.run.smithi143.stdout: chdir = "" 2023-11-08T19:30:20.111 INFO:teuthology.orchestra.run.smithi143.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-08T19:30:20.111 INFO:teuthology.orchestra.run.smithi143.stdout: auth supported = cephx 2023-11-08T19:30:20.111 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:30:20.112 INFO:teuthology.orchestra.run.smithi143.stdout: filestore xattr use omap = true 2023-11-08T19:30:20.112 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:30:20.112 INFO:teuthology.orchestra.run.smithi143.stdout: mon clock drift allowed = 1.000 2023-11-08T19:30:20.112 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:30:20.112 INFO:teuthology.orchestra.run.smithi143.stdout: osd crush chooseleaf type = 0 2023-11-08T19:30:20.113 INFO:teuthology.orchestra.run.smithi143.stdout: auth debug = true 2023-11-08T19:30:20.113 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:30:20.113 INFO:teuthology.orchestra.run.smithi143.stdout: ms die on old message = true 2023-11-08T19:30:20.113 INFO:teuthology.orchestra.run.smithi143.stdout: ms die on bug = true 2023-11-08T19:30:20.113 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:30:20.114 INFO:teuthology.orchestra.run.smithi143.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-08T19:30:20.114 INFO:teuthology.orchestra.run.smithi143.stdout: mon pg warn max object skew = 0 2023-11-08T19:30:20.114 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:30:20.114 INFO:teuthology.orchestra.run.smithi143.stdout: # disable pg_autoscaler by default for new pools 2023-11-08T19:30:20.114 INFO:teuthology.orchestra.run.smithi143.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-08T19:30:20.115 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:30:20.115 INFO:teuthology.orchestra.run.smithi143.stdout: osd pool default size = 2 2023-11-08T19:30:20.115 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:30:20.116 INFO:teuthology.orchestra.run.smithi143.stdout: mon osd allow primary affinity = true 2023-11-08T19:30:20.116 INFO:teuthology.orchestra.run.smithi143.stdout: mon osd allow pg remap = true 2023-11-08T19:30:20.116 INFO:teuthology.orchestra.run.smithi143.stdout: mon warn on legacy crush tunables = false 2023-11-08T19:30:20.116 INFO:teuthology.orchestra.run.smithi143.stdout: mon warn on crush straw calc version zero = false 2023-11-08T19:30:20.116 INFO:teuthology.orchestra.run.smithi143.stdout: mon warn on no sortbitwise = false 2023-11-08T19:30:20.117 INFO:teuthology.orchestra.run.smithi143.stdout: mon warn on osd down out interval zero = false 2023-11-08T19:30:20.117 INFO:teuthology.orchestra.run.smithi143.stdout: mon warn on too few osds = false 2023-11-08T19:30:20.117 INFO:teuthology.orchestra.run.smithi143.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-08T19:30:20.117 INFO:teuthology.orchestra.run.smithi143.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-08T19:30:20.117 INFO:teuthology.orchestra.run.smithi143.stdout: mon_allow_pool_size_one = true 2023-11-08T19:30:20.117 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:30:20.118 INFO:teuthology.orchestra.run.smithi143.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-11-08T19:30:20.118 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:30:20.118 INFO:teuthology.orchestra.run.smithi143.stdout: osd default data pool replay window = 5 2023-11-08T19:30:20.118 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:30:20.118 INFO:teuthology.orchestra.run.smithi143.stdout: mon allow pool delete = true 2023-11-08T19:30:20.119 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:30:20.119 INFO:teuthology.orchestra.run.smithi143.stdout: mon cluster log file level = debug 2023-11-08T19:30:20.119 INFO:teuthology.orchestra.run.smithi143.stdout: debug asserts on shutdown = true 2023-11-08T19:30:20.119 INFO:teuthology.orchestra.run.smithi143.stdout: mon health detail to clog = false 2023-11-08T19:30:20.119 INFO:teuthology.orchestra.run.smithi143.stdout: mon host = "172.21.15.143,172.21.15.174,[v2:172.21.15.174:3301,v1:172.21.15.174:6790]" 2023-11-08T19:30:20.120 INFO:teuthology.orchestra.run.smithi143.stdout: ms die on skipped message = False 2023-11-08T19:30:20.120 INFO:teuthology.orchestra.run.smithi143.stdout: fsid = 968483af-77fc-4068-ad7c-5c2a1a4613d7 2023-11-08T19:30:20.120 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:30:20.120 INFO:teuthology.orchestra.run.smithi143.stdout:[osd] 2023-11-08T19:30:20.120 INFO:teuthology.orchestra.run.smithi143.stdout: osd journal size = 100 2023-11-08T19:30:20.121 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:30:20.121 INFO:teuthology.orchestra.run.smithi143.stdout: osd scrub load threshold = 5.0 2023-11-08T19:30:20.121 INFO:teuthology.orchestra.run.smithi143.stdout: osd scrub max interval = 600 2023-11-08T19:30:20.121 INFO:teuthology.orchestra.run.smithi143.stdout: osd mclock profile = high_recovery_ops 2023-11-08T19:30:20.121 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:30:20.121 INFO:teuthology.orchestra.run.smithi143.stdout: osd recover clone overlap = true 2023-11-08T19:30:20.122 INFO:teuthology.orchestra.run.smithi143.stdout: osd recovery max chunk = 1048576 2023-11-08T19:30:20.122 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:30:20.122 INFO:teuthology.orchestra.run.smithi143.stdout: osd debug shutdown = true 2023-11-08T19:30:20.122 INFO:teuthology.orchestra.run.smithi143.stdout: osd debug op order = true 2023-11-08T19:30:20.122 INFO:teuthology.orchestra.run.smithi143.stdout: osd debug verify stray on activate = true 2023-11-08T19:30:20.123 INFO:teuthology.orchestra.run.smithi143.stdout: osd debug trim objects = true 2023-11-08T19:30:20.123 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:30:20.123 INFO:teuthology.orchestra.run.smithi143.stdout: osd open classes on start = true 2023-11-08T19:30:20.123 INFO:teuthology.orchestra.run.smithi143.stdout: osd debug pg log writeout = true 2023-11-08T19:30:20.123 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:30:20.124 INFO:teuthology.orchestra.run.smithi143.stdout: osd deep scrub update digest min age = 30 2023-11-08T19:30:20.124 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:30:20.124 INFO:teuthology.orchestra.run.smithi143.stdout: osd map max advance = 10 2023-11-08T19:30:20.124 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:30:20.124 INFO:teuthology.orchestra.run.smithi143.stdout: journal zero on create = true 2023-11-08T19:30:20.124 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:30:20.125 INFO:teuthology.orchestra.run.smithi143.stdout: filestore ondisk finisher threads = 3 2023-11-08T19:30:20.125 INFO:teuthology.orchestra.run.smithi143.stdout: filestore apply finisher threads = 3 2023-11-08T19:30:20.125 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:30:20.125 INFO:teuthology.orchestra.run.smithi143.stdout: bdev debug aio = true 2023-11-08T19:30:20.125 INFO:teuthology.orchestra.run.smithi143.stdout: osd debug misdirected ops = true 2023-11-08T19:30:20.126 INFO:teuthology.orchestra.run.smithi143.stdout: bdev async discard = True 2023-11-08T19:30:20.126 INFO:teuthology.orchestra.run.smithi143.stdout: bdev enable discard = True 2023-11-08T19:30:20.126 INFO:teuthology.orchestra.run.smithi143.stdout: bluestore allocator = bitmap 2023-11-08T19:30:20.127 INFO:teuthology.orchestra.run.smithi143.stdout: bluestore block size = 96636764160 2023-11-08T19:30:20.127 INFO:teuthology.orchestra.run.smithi143.stdout: bluestore fsck on mount = True 2023-11-08T19:30:20.127 INFO:teuthology.orchestra.run.smithi143.stdout: debug bluefs = 1/20 2023-11-08T19:30:20.127 INFO:teuthology.orchestra.run.smithi143.stdout: debug bluestore = 1/20 2023-11-08T19:30:20.127 INFO:teuthology.orchestra.run.smithi143.stdout: debug ms = 1 2023-11-08T19:30:20.128 INFO:teuthology.orchestra.run.smithi143.stdout: debug osd = 20 2023-11-08T19:30:20.128 INFO:teuthology.orchestra.run.smithi143.stdout: debug rocksdb = 4/10 2023-11-08T19:30:20.128 INFO:teuthology.orchestra.run.smithi143.stdout: mon osd backfillfull_ratio = 0.85 2023-11-08T19:30:20.128 INFO:teuthology.orchestra.run.smithi143.stdout: mon osd full ratio = 0.9 2023-11-08T19:30:20.128 INFO:teuthology.orchestra.run.smithi143.stdout: mon osd nearfull ratio = 0.8 2023-11-08T19:30:20.129 INFO:teuthology.orchestra.run.smithi143.stdout: osd failsafe full ratio = 0.95 2023-11-08T19:30:20.129 INFO:teuthology.orchestra.run.smithi143.stdout: osd objectstore = bluestore 2023-11-08T19:30:20.129 INFO:teuthology.orchestra.run.smithi143.stdout: osd op complaint time = 180 2023-11-08T19:30:20.129 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:30:20.129 INFO:teuthology.orchestra.run.smithi143.stdout:[mgr] 2023-11-08T19:30:20.130 INFO:teuthology.orchestra.run.smithi143.stdout: debug ms = 1 2023-11-08T19:30:20.130 INFO:teuthology.orchestra.run.smithi143.stdout: debug mgr = 20 2023-11-08T19:30:20.130 INFO:teuthology.orchestra.run.smithi143.stdout: debug mon = 20 2023-11-08T19:30:20.130 INFO:teuthology.orchestra.run.smithi143.stdout: debug auth = 20 2023-11-08T19:30:20.130 INFO:teuthology.orchestra.run.smithi143.stdout: mon reweight min pgs per osd = 4 2023-11-08T19:30:20.130 INFO:teuthology.orchestra.run.smithi143.stdout: mon reweight min bytes per osd = 10 2023-11-08T19:30:20.131 INFO:teuthology.orchestra.run.smithi143.stdout: mgr/telemetry/nag = false 2023-11-08T19:30:20.131 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:30:20.131 INFO:teuthology.orchestra.run.smithi143.stdout:[mon] 2023-11-08T19:30:20.131 INFO:teuthology.orchestra.run.smithi143.stdout: debug ms = 1 2023-11-08T19:30:20.131 INFO:teuthology.orchestra.run.smithi143.stdout: debug mon = 20 2023-11-08T19:30:20.132 INFO:teuthology.orchestra.run.smithi143.stdout: debug paxos = 20 2023-11-08T19:30:20.132 INFO:teuthology.orchestra.run.smithi143.stdout: debug auth = 20 2023-11-08T19:30:20.132 INFO:teuthology.orchestra.run.smithi143.stdout: mon data avail warn = 5 2023-11-08T19:30:20.132 INFO:teuthology.orchestra.run.smithi143.stdout: mon mgr mkfs grace = 240 2023-11-08T19:30:20.132 INFO:teuthology.orchestra.run.smithi143.stdout: mon reweight min pgs per osd = 4 2023-11-08T19:30:20.133 INFO:teuthology.orchestra.run.smithi143.stdout: mon osd reporter subtree level = osd 2023-11-08T19:30:20.133 INFO:teuthology.orchestra.run.smithi143.stdout: mon osd prime pg temp = true 2023-11-08T19:30:20.133 INFO:teuthology.orchestra.run.smithi143.stdout: mon reweight min bytes per osd = 10 2023-11-08T19:30:20.133 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:30:20.133 INFO:teuthology.orchestra.run.smithi143.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-08T19:30:20.133 INFO:teuthology.orchestra.run.smithi143.stdout: auth mon ticket ttl = 660 # 11m 2023-11-08T19:30:20.134 INFO:teuthology.orchestra.run.smithi143.stdout: auth service ticket ttl = 240 # 4m 2023-11-08T19:30:20.134 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:30:20.134 INFO:teuthology.orchestra.run.smithi143.stdout: # don't complain about insecure global_id in the test suite 2023-11-08T19:30:20.134 INFO:teuthology.orchestra.run.smithi143.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-08T19:30:20.134 INFO:teuthology.orchestra.run.smithi143.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-08T19:30:20.135 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:30:20.135 INFO:teuthology.orchestra.run.smithi143.stdout: # 1m isn't quite enough 2023-11-08T19:30:20.135 INFO:teuthology.orchestra.run.smithi143.stdout: mon_down_mkfs_grace = 2m 2023-11-08T19:30:20.135 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:30:20.135 INFO:teuthology.orchestra.run.smithi143.stdout: mon_warn_on_filestore_osds = false 2023-11-08T19:30:20.136 INFO:teuthology.orchestra.run.smithi143.stdout: mon op complaint time = 120 2023-11-08T19:30:20.136 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:30:20.136 INFO:teuthology.orchestra.run.smithi143.stdout:[client] 2023-11-08T19:30:20.136 INFO:teuthology.orchestra.run.smithi143.stdout: rgw cache enabled = true 2023-11-08T19:30:20.136 INFO:teuthology.orchestra.run.smithi143.stdout: rgw enable ops log = true 2023-11-08T19:30:20.137 INFO:teuthology.orchestra.run.smithi143.stdout: rgw enable usage log = true 2023-11-08T19:30:20.137 INFO:teuthology.orchestra.run.smithi143.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-08T19:30:20.137 INFO:teuthology.orchestra.run.smithi143.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-08T19:30:20.137 INFO:teuthology.orchestra.run.smithi143.stdout: client mount timeout = 600 2023-11-08T19:30:20.137 INFO:teuthology.orchestra.run.smithi143.stdout: debug client = 20 2023-11-08T19:30:20.137 INFO:teuthology.orchestra.run.smithi143.stdout: debug ms = 1 2023-11-08T19:30:20.138 INFO:teuthology.orchestra.run.smithi143.stdout: pid file = "" 2023-11-08T19:30:20.138 INFO:teuthology.orchestra.run.smithi143.stdout: rados mon op timeout = 900 2023-11-08T19:30:20.138 INFO:teuthology.orchestra.run.smithi143.stdout: rados osd op timeout = 900 2023-11-08T19:30:20.138 INFO:teuthology.orchestra.run.smithi143.stdout:[mon.a] 2023-11-08T19:30:20.138 INFO:teuthology.orchestra.run.smithi143.stdout:[mon.b] 2023-11-08T19:30:20.139 INFO:teuthology.orchestra.run.smithi143.stdout:[mon.c] 2023-11-08T19:30:20.139 INFO:teuthology.orchestra.run.smithi143.stdout:[mds.a] 2023-11-08T19:30:20.139 INFO:teuthology.orchestra.run.smithi143.stdout:[mds.c] 2023-11-08T19:30:20.139 INFO:teuthology.orchestra.run.smithi143.stdout:[mds.b] 2023-11-08T19:30:20.139 INFO:teuthology.orchestra.run.smithi143.stdout:[mds.d] 2023-11-08T19:30:20.140 INFO:teuthology.orchestra.run.smithi143.stdout:[mds] 2023-11-08T19:30:20.140 INFO:teuthology.orchestra.run.smithi143.stdout: debug mds = 20 2023-11-08T19:30:20.140 INFO:teuthology.orchestra.run.smithi143.stdout: debug mds balancer = 20 2023-11-08T19:30:20.140 INFO:teuthology.orchestra.run.smithi143.stdout: debug ms = 1 2023-11-08T19:30:20.140 INFO:teuthology.orchestra.run.smithi143.stdout: mds debug frag = True 2023-11-08T19:30:20.141 INFO:teuthology.orchestra.run.smithi143.stdout: mds debug scatterstat = True 2023-11-08T19:30:20.141 INFO:teuthology.orchestra.run.smithi143.stdout: mds op complaint time = 180 2023-11-08T19:30:20.141 INFO:teuthology.orchestra.run.smithi143.stdout: mds verify scatter = True 2023-11-08T19:30:20.141 INFO:teuthology.orchestra.run.smithi143.stdout: osd op complaint time = 180 2023-11-08T19:30:20.141 INFO:teuthology.orchestra.run.smithi143.stdout: rados mon op timeout = 900 2023-11-08T19:30:20.141 INFO:teuthology.orchestra.run.smithi143.stdout: rados osd op timeout = 900 2023-11-08T19:30:20.142 INFO:teuthology.orchestra.run.smithi143.stdout: mds_reconnect_timeout = 5 2023-11-08T19:30:20.143 INFO:teuthology.orchestra.run.smithi174.stdout:[global] 2023-11-08T19:30:20.144 INFO:teuthology.orchestra.run.smithi174.stdout: chdir = "" 2023-11-08T19:30:20.144 INFO:teuthology.orchestra.run.smithi174.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-08T19:30:20.144 INFO:teuthology.orchestra.run.smithi174.stdout: auth supported = cephx 2023-11-08T19:30:20.144 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-08T19:30:20.144 INFO:teuthology.orchestra.run.smithi174.stdout: filestore xattr use omap = true 2023-11-08T19:30:20.145 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-08T19:30:20.145 INFO:teuthology.orchestra.run.smithi174.stdout: mon clock drift allowed = 1.000 2023-11-08T19:30:20.145 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-08T19:30:20.145 INFO:teuthology.orchestra.run.smithi174.stdout: osd crush chooseleaf type = 0 2023-11-08T19:30:20.145 INFO:teuthology.orchestra.run.smithi174.stdout: auth debug = true 2023-11-08T19:30:20.146 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-08T19:30:20.146 INFO:teuthology.orchestra.run.smithi174.stdout: ms die on old message = true 2023-11-08T19:30:20.146 INFO:teuthology.orchestra.run.smithi174.stdout: ms die on bug = true 2023-11-08T19:30:20.146 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-08T19:30:20.146 INFO:teuthology.orchestra.run.smithi174.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-08T19:30:20.147 INFO:teuthology.orchestra.run.smithi174.stdout: mon pg warn max object skew = 0 2023-11-08T19:30:20.147 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-08T19:30:20.147 INFO:teuthology.orchestra.run.smithi174.stdout: # disable pg_autoscaler by default for new pools 2023-11-08T19:30:20.147 INFO:teuthology.orchestra.run.smithi174.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-08T19:30:20.147 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-08T19:30:20.147 INFO:teuthology.orchestra.run.smithi174.stdout: osd pool default size = 2 2023-11-08T19:30:20.148 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-08T19:30:20.148 INFO:teuthology.orchestra.run.smithi174.stdout: mon osd allow primary affinity = true 2023-11-08T19:30:20.148 INFO:teuthology.orchestra.run.smithi174.stdout: mon osd allow pg remap = true 2023-11-08T19:30:20.148 INFO:teuthology.orchestra.run.smithi174.stdout: mon warn on legacy crush tunables = false 2023-11-08T19:30:20.148 INFO:teuthology.orchestra.run.smithi174.stdout: mon warn on crush straw calc version zero = false 2023-11-08T19:30:20.149 INFO:teuthology.orchestra.run.smithi174.stdout: mon warn on no sortbitwise = false 2023-11-08T19:30:20.149 INFO:teuthology.orchestra.run.smithi174.stdout: mon warn on osd down out interval zero = false 2023-11-08T19:30:20.149 INFO:teuthology.orchestra.run.smithi174.stdout: mon warn on too few osds = false 2023-11-08T19:30:20.149 INFO:teuthology.orchestra.run.smithi174.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-08T19:30:20.150 INFO:teuthology.orchestra.run.smithi174.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-08T19:30:20.150 INFO:teuthology.orchestra.run.smithi174.stdout: mon_allow_pool_size_one = true 2023-11-08T19:30:20.150 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-08T19:30:20.150 INFO:teuthology.orchestra.run.smithi174.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-11-08T19:30:20.150 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-08T19:30:20.151 INFO:teuthology.orchestra.run.smithi174.stdout: osd default data pool replay window = 5 2023-11-08T19:30:20.151 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-08T19:30:20.151 INFO:teuthology.orchestra.run.smithi174.stdout: mon allow pool delete = true 2023-11-08T19:30:20.151 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-08T19:30:20.151 INFO:teuthology.orchestra.run.smithi174.stdout: mon cluster log file level = debug 2023-11-08T19:30:20.152 INFO:teuthology.orchestra.run.smithi174.stdout: debug asserts on shutdown = true 2023-11-08T19:30:20.152 INFO:teuthology.orchestra.run.smithi174.stdout: mon health detail to clog = false 2023-11-08T19:30:20.152 INFO:teuthology.orchestra.run.smithi174.stdout: mon host = "172.21.15.143,172.21.15.174,[v2:172.21.15.174:3301,v1:172.21.15.174:6790]" 2023-11-08T19:30:20.152 INFO:teuthology.orchestra.run.smithi174.stdout: ms die on skipped message = False 2023-11-08T19:30:20.152 INFO:teuthology.orchestra.run.smithi174.stdout: fsid = 968483af-77fc-4068-ad7c-5c2a1a4613d7 2023-11-08T19:30:20.152 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-08T19:30:20.153 INFO:teuthology.orchestra.run.smithi174.stdout:[osd] 2023-11-08T19:30:20.153 INFO:teuthology.orchestra.run.smithi174.stdout: osd journal size = 100 2023-11-08T19:30:20.153 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-08T19:30:20.153 INFO:teuthology.orchestra.run.smithi174.stdout: osd scrub load threshold = 5.0 2023-11-08T19:30:20.153 INFO:teuthology.orchestra.run.smithi174.stdout: osd scrub max interval = 600 2023-11-08T19:30:20.154 INFO:teuthology.orchestra.run.smithi174.stdout: osd mclock profile = high_recovery_ops 2023-11-08T19:30:20.154 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-08T19:30:20.154 INFO:teuthology.orchestra.run.smithi174.stdout: osd recover clone overlap = true 2023-11-08T19:30:20.154 INFO:teuthology.orchestra.run.smithi174.stdout: osd recovery max chunk = 1048576 2023-11-08T19:30:20.154 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-08T19:30:20.155 INFO:teuthology.orchestra.run.smithi174.stdout: osd debug shutdown = true 2023-11-08T19:30:20.155 INFO:teuthology.orchestra.run.smithi174.stdout: osd debug op order = true 2023-11-08T19:30:20.155 INFO:teuthology.orchestra.run.smithi174.stdout: osd debug verify stray on activate = true 2023-11-08T19:30:20.155 INFO:teuthology.orchestra.run.smithi174.stdout: osd debug trim objects = true 2023-11-08T19:30:20.155 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-08T19:30:20.155 INFO:teuthology.orchestra.run.smithi174.stdout: osd open classes on start = true 2023-11-08T19:30:20.156 INFO:teuthology.orchestra.run.smithi174.stdout: osd debug pg log writeout = true 2023-11-08T19:30:20.156 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-08T19:30:20.156 INFO:teuthology.orchestra.run.smithi174.stdout: osd deep scrub update digest min age = 30 2023-11-08T19:30:20.156 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-08T19:30:20.156 INFO:teuthology.orchestra.run.smithi174.stdout: osd map max advance = 10 2023-11-08T19:30:20.157 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-08T19:30:20.157 INFO:teuthology.orchestra.run.smithi174.stdout: journal zero on create = true 2023-11-08T19:30:20.157 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-08T19:30:20.157 INFO:teuthology.orchestra.run.smithi174.stdout: filestore ondisk finisher threads = 3 2023-11-08T19:30:20.157 INFO:teuthology.orchestra.run.smithi174.stdout: filestore apply finisher threads = 3 2023-11-08T19:30:20.157 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-08T19:30:20.158 INFO:teuthology.orchestra.run.smithi174.stdout: bdev debug aio = true 2023-11-08T19:30:20.158 INFO:teuthology.orchestra.run.smithi174.stdout: osd debug misdirected ops = true 2023-11-08T19:30:20.158 INFO:teuthology.orchestra.run.smithi174.stdout: bdev async discard = True 2023-11-08T19:30:20.158 INFO:teuthology.orchestra.run.smithi174.stdout: bdev enable discard = True 2023-11-08T19:30:20.158 INFO:teuthology.orchestra.run.smithi174.stdout: bluestore allocator = bitmap 2023-11-08T19:30:20.159 INFO:teuthology.orchestra.run.smithi174.stdout: bluestore block size = 96636764160 2023-11-08T19:30:20.159 INFO:teuthology.orchestra.run.smithi174.stdout: bluestore fsck on mount = True 2023-11-08T19:30:20.159 INFO:teuthology.orchestra.run.smithi174.stdout: debug bluefs = 1/20 2023-11-08T19:30:20.159 INFO:teuthology.orchestra.run.smithi174.stdout: debug bluestore = 1/20 2023-11-08T19:30:20.159 INFO:teuthology.orchestra.run.smithi174.stdout: debug ms = 1 2023-11-08T19:30:20.160 INFO:teuthology.orchestra.run.smithi174.stdout: debug osd = 20 2023-11-08T19:30:20.160 INFO:teuthology.orchestra.run.smithi174.stdout: debug rocksdb = 4/10 2023-11-08T19:30:20.160 INFO:teuthology.orchestra.run.smithi174.stdout: mon osd backfillfull_ratio = 0.85 2023-11-08T19:30:20.160 INFO:teuthology.orchestra.run.smithi174.stdout: mon osd full ratio = 0.9 2023-11-08T19:30:20.160 INFO:teuthology.orchestra.run.smithi174.stdout: mon osd nearfull ratio = 0.8 2023-11-08T19:30:20.160 INFO:teuthology.orchestra.run.smithi174.stdout: osd failsafe full ratio = 0.95 2023-11-08T19:30:20.161 INFO:teuthology.orchestra.run.smithi174.stdout: osd objectstore = bluestore 2023-11-08T19:30:20.161 INFO:teuthology.orchestra.run.smithi174.stdout: osd op complaint time = 180 2023-11-08T19:30:20.161 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-08T19:30:20.161 INFO:teuthology.orchestra.run.smithi174.stdout:[mgr] 2023-11-08T19:30:20.161 INFO:teuthology.orchestra.run.smithi174.stdout: debug ms = 1 2023-11-08T19:30:20.162 INFO:teuthology.orchestra.run.smithi174.stdout: debug mgr = 20 2023-11-08T19:30:20.162 INFO:teuthology.orchestra.run.smithi174.stdout: debug mon = 20 2023-11-08T19:30:20.162 INFO:teuthology.orchestra.run.smithi174.stdout: debug auth = 20 2023-11-08T19:30:20.162 INFO:teuthology.orchestra.run.smithi174.stdout: mon reweight min pgs per osd = 4 2023-11-08T19:30:20.162 INFO:teuthology.orchestra.run.smithi174.stdout: mon reweight min bytes per osd = 10 2023-11-08T19:30:20.163 INFO:teuthology.orchestra.run.smithi174.stdout: mgr/telemetry/nag = false 2023-11-08T19:30:20.163 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-08T19:30:20.163 INFO:teuthology.orchestra.run.smithi174.stdout:[mon] 2023-11-08T19:30:20.163 INFO:teuthology.orchestra.run.smithi174.stdout: debug ms = 1 2023-11-08T19:30:20.163 INFO:teuthology.orchestra.run.smithi174.stdout: debug mon = 20 2023-11-08T19:30:20.163 INFO:teuthology.orchestra.run.smithi174.stdout: debug paxos = 20 2023-11-08T19:30:20.164 INFO:teuthology.orchestra.run.smithi174.stdout: debug auth = 20 2023-11-08T19:30:20.164 INFO:teuthology.orchestra.run.smithi174.stdout: mon data avail warn = 5 2023-11-08T19:30:20.164 INFO:teuthology.orchestra.run.smithi174.stdout: mon mgr mkfs grace = 240 2023-11-08T19:30:20.164 INFO:teuthology.orchestra.run.smithi174.stdout: mon reweight min pgs per osd = 4 2023-11-08T19:30:20.164 INFO:teuthology.orchestra.run.smithi174.stdout: mon osd reporter subtree level = osd 2023-11-08T19:30:20.165 INFO:teuthology.orchestra.run.smithi174.stdout: mon osd prime pg temp = true 2023-11-08T19:30:20.165 INFO:teuthology.orchestra.run.smithi174.stdout: mon reweight min bytes per osd = 10 2023-11-08T19:30:20.165 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-08T19:30:20.165 INFO:teuthology.orchestra.run.smithi174.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-08T19:30:20.165 INFO:teuthology.orchestra.run.smithi174.stdout: auth mon ticket ttl = 660 # 11m 2023-11-08T19:30:20.165 INFO:teuthology.orchestra.run.smithi174.stdout: auth service ticket ttl = 240 # 4m 2023-11-08T19:30:20.166 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-08T19:30:20.166 INFO:teuthology.orchestra.run.smithi174.stdout: # don't complain about insecure global_id in the test suite 2023-11-08T19:30:20.166 INFO:teuthology.orchestra.run.smithi174.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-08T19:30:20.166 INFO:teuthology.orchestra.run.smithi174.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-08T19:30:20.166 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-08T19:30:20.167 INFO:teuthology.orchestra.run.smithi174.stdout: # 1m isn't quite enough 2023-11-08T19:30:20.167 INFO:teuthology.orchestra.run.smithi174.stdout: mon_down_mkfs_grace = 2m 2023-11-08T19:30:20.167 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-08T19:30:20.167 INFO:teuthology.orchestra.run.smithi174.stdout: mon_warn_on_filestore_osds = false 2023-11-08T19:30:20.167 INFO:teuthology.orchestra.run.smithi174.stdout: mon op complaint time = 120 2023-11-08T19:30:20.168 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-08T19:30:20.168 INFO:teuthology.orchestra.run.smithi174.stdout:[client] 2023-11-08T19:30:20.168 INFO:teuthology.orchestra.run.smithi174.stdout: rgw cache enabled = true 2023-11-08T19:30:20.168 INFO:teuthology.orchestra.run.smithi174.stdout: rgw enable ops log = true 2023-11-08T19:30:20.168 INFO:teuthology.orchestra.run.smithi174.stdout: rgw enable usage log = true 2023-11-08T19:30:20.168 INFO:teuthology.orchestra.run.smithi174.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-08T19:30:20.169 INFO:teuthology.orchestra.run.smithi174.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-08T19:30:20.169 INFO:teuthology.orchestra.run.smithi174.stdout: client mount timeout = 600 2023-11-08T19:30:20.169 INFO:teuthology.orchestra.run.smithi174.stdout: debug client = 20 2023-11-08T19:30:20.169 INFO:teuthology.orchestra.run.smithi174.stdout: debug ms = 1 2023-11-08T19:30:20.169 INFO:teuthology.orchestra.run.smithi174.stdout: pid file = "" 2023-11-08T19:30:20.170 INFO:teuthology.orchestra.run.smithi174.stdout: rados mon op timeout = 900 2023-11-08T19:30:20.170 INFO:teuthology.orchestra.run.smithi174.stdout: rados osd op timeout = 900 2023-11-08T19:30:20.170 INFO:teuthology.orchestra.run.smithi174.stdout:[mon.a] 2023-11-08T19:30:20.170 INFO:teuthology.orchestra.run.smithi174.stdout:[mon.b] 2023-11-08T19:30:20.170 INFO:teuthology.orchestra.run.smithi174.stdout:[mon.c] 2023-11-08T19:30:20.170 INFO:teuthology.orchestra.run.smithi174.stdout:[mds.a] 2023-11-08T19:30:20.171 INFO:teuthology.orchestra.run.smithi174.stdout:[mds.c] 2023-11-08T19:30:20.171 INFO:teuthology.orchestra.run.smithi174.stdout:[mds.b] 2023-11-08T19:30:20.171 INFO:teuthology.orchestra.run.smithi174.stdout:[mds.d] 2023-11-08T19:30:20.171 INFO:teuthology.orchestra.run.smithi174.stdout:[mds] 2023-11-08T19:30:20.171 INFO:teuthology.orchestra.run.smithi174.stdout: debug mds = 20 2023-11-08T19:30:20.172 INFO:teuthology.orchestra.run.smithi174.stdout: debug mds balancer = 20 2023-11-08T19:30:20.172 INFO:teuthology.orchestra.run.smithi174.stdout: debug ms = 1 2023-11-08T19:30:20.172 INFO:teuthology.orchestra.run.smithi174.stdout: mds debug frag = True 2023-11-08T19:30:20.172 INFO:teuthology.orchestra.run.smithi174.stdout: mds debug scatterstat = True 2023-11-08T19:30:20.172 INFO:teuthology.orchestra.run.smithi174.stdout: mds op complaint time = 180 2023-11-08T19:30:20.172 INFO:teuthology.orchestra.run.smithi174.stdout: mds verify scatter = True 2023-11-08T19:30:20.173 INFO:teuthology.orchestra.run.smithi174.stdout: osd op complaint time = 180 2023-11-08T19:30:20.173 INFO:teuthology.orchestra.run.smithi174.stdout: rados mon op timeout = 900 2023-11-08T19:30:20.173 INFO:teuthology.orchestra.run.smithi174.stdout: rados osd op timeout = 900 2023-11-08T19:30:20.173 INFO:teuthology.orchestra.run.smithi174.stdout: mds_reconnect_timeout = 5 2023-11-08T19:30:20.174 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:30:20.174 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir d1/dir/.snap/*) 2023-11-08T19:30:20.223 INFO:tasks.cephfs.test_snapshots:testing snapserver mds_kill_mdstable_at=2, also kill snapclient 2023-11-08T19:30:20.224 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:30:20.385 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.383+0000 7fd9e26c8700 1 Processor -- start 2023-11-08T19:30:20.385 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.383+0000 7fd9e26c8700 1 -- start start 2023-11-08T19:30:20.388 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.384+0000 7fd9e26c8700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd9dc15a8f0 0x7fd9dc15f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:20.388 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.384+0000 7fd9e26c8700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd9dc0ccea0 0x7fd9dc0cd310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:20.388 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.384+0000 7fd9e26c8700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9dc06f780 con 0x7fd9dc15acd0 2023-11-08T19:30:20.388 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.384+0000 7fd9e26c8700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9dc06e8e0 con 0x7fd9dc0ccea0 2023-11-08T19:30:20.388 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.384+0000 7fd9e26c8700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9dc0cd850 con 0x7fd9dc15a8f0 2023-11-08T19:30:20.389 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.384+0000 7fd9e0c65700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd9dc0ccea0 0x7fd9dc0cd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:20.389 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.384+0000 7fd9e0c65700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd9dc0ccea0 0x7fd9dc0cd310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56240/0 (socket says 172.21.15.143:56240) 2023-11-08T19:30:20.389 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.384+0000 7fd9e0c65700 1 -- 172.21.15.143:0/2396192135 learned_addr learned my addr 172.21.15.143:0/2396192135 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:30:20.390 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.385+0000 7fd9dbfff700 1 --2- 172.21.15.143:0/2396192135 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd9dc15a8f0 0x7fd9dc15f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:20.390 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.385+0000 7fd9e0c65700 1 -- 172.21.15.143:0/2396192135 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd9dc15a8f0 msgr2=0x7fd9dc15f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:20.390 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.385+0000 7fd9e0c65700 1 --2- 172.21.15.143:0/2396192135 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd9dc15a8f0 0x7fd9dc15f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:20.390 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.385+0000 7fd9e0c65700 1 -- 172.21.15.143:0/2396192135 >> v1:172.21.15.174:6789/0 conn(0x7fd9dc15acd0 legacy=0x7fd9dc06e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:20.390 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.385+0000 7fd9e0c65700 1 -- 172.21.15.143:0/2396192135 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9dc06eb70 con 0x7fd9dc0ccea0 2023-11-08T19:30:20.391 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.385+0000 7fd9dbfff700 1 --2- 172.21.15.143:0/2396192135 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd9dc15a8f0 0x7fd9dc15f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:30:20.391 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.385+0000 7fd9e0c65700 1 --2- 172.21.15.143:0/2396192135 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd9dc0ccea0 0x7fd9dc0cd310 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7fd9d400d150 tx=0x7fd9d400d460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=983d04143abc1321 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:20.391 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.385+0000 7fd9daffd700 1 -- 172.21.15.143:0/2396192135 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d4010070 con 0x7fd9dc0ccea0 2023-11-08T19:30:20.391 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.385+0000 7fd9daffd700 1 -- 172.21.15.143:0/2396192135 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9d400dd80 con 0x7fd9dc0ccea0 2023-11-08T19:30:20.391 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.385+0000 7fd9daffd700 1 -- 172.21.15.143:0/2396192135 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d400b310 con 0x7fd9dc0ccea0 2023-11-08T19:30:20.392 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.385+0000 7fd9e26c8700 1 -- 172.21.15.143:0/2396192135 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd9dc0ccea0 msgr2=0x7fd9dc0cd310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:20.392 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.385+0000 7fd9e26c8700 1 --2- 172.21.15.143:0/2396192135 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd9dc0ccea0 0x7fd9dc0cd310 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7fd9d400d150 tx=0x7fd9d400d460 comp rx=0 tx=0).stop 2023-11-08T19:30:20.392 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.385+0000 7fd9e26c8700 1 -- 172.21.15.143:0/2396192135 shutdown_connections 2023-11-08T19:30:20.392 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.385+0000 7fd9e26c8700 1 --2- 172.21.15.143:0/2396192135 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd9dc0ccea0 0x7fd9dc0cd310 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:20.392 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.385+0000 7fd9e26c8700 1 --2- 172.21.15.143:0/2396192135 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd9dc15a8f0 0x7fd9dc15f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:20.393 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.386+0000 7fd9e26c8700 1 -- 172.21.15.143:0/2396192135 >> 172.21.15.143:0/2396192135 conn(0x7fd9dc0b23d0 msgr2=0x7fd9dc0b27b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:20.393 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.386+0000 7fd9e26c8700 1 -- 172.21.15.143:0/2396192135 shutdown_connections 2023-11-08T19:30:20.393 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.386+0000 7fd9e26c8700 1 -- 172.21.15.143:0/2396192135 wait complete. 2023-11-08T19:30:20.393 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.387+0000 7fd9e26c8700 1 Processor -- start 2023-11-08T19:30:20.396 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.387+0000 7fd9e26c8700 1 -- start start 2023-11-08T19:30:20.396 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.388+0000 7fd9e26c8700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd9dc0ccea0 0x7fd9dc1489b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:20.397 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.388+0000 7fd9e26c8700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd9dc15a8f0 0x7fd9dc0cb310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:20.397 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.388+0000 7fd9e26c8700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd9dc15acd0 0x7fd9dc147ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:20.397 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.388+0000 7fd9e26c8700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9dc0cb850 con 0x7fd9dc15a8f0 2023-11-08T19:30:20.397 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.388+0000 7fd9e26c8700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9dc0c8fe0 con 0x7fd9dc0ccea0 2023-11-08T19:30:20.397 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.388+0000 7fd9e26c8700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9dc0b3370 con 0x7fd9dc15acd0 2023-11-08T19:30:20.398 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.388+0000 7fd9db7fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd9dc15a8f0 0x7fd9dc0cb310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:20.398 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.388+0000 7fd9db7fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd9dc15a8f0 0x7fd9dc0cb310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56244/0 (socket says 172.21.15.143:56244) 2023-11-08T19:30:20.398 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.388+0000 7fd9db7fe700 1 -- 172.21.15.143:0/3499120691 learned_addr learned my addr 172.21.15.143:0/3499120691 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:30:20.398 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.388+0000 7fd9dbfff700 1 --2- 172.21.15.143:0/3499120691 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd9dc0ccea0 0x7fd9dc1489b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:20.398 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.388+0000 7fd9e0c65700 1 --2- 172.21.15.143:0/3499120691 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd9dc15acd0 0x7fd9dc147ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:20.399 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.388+0000 7fd9db7fe700 1 -- 172.21.15.143:0/3499120691 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd9dc15acd0 msgr2=0x7fd9dc147ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:20.399 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.388+0000 7fd9db7fe700 1 --2- 172.21.15.143:0/3499120691 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd9dc15acd0 0x7fd9dc147ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:20.399 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.388+0000 7fd9db7fe700 1 -- 172.21.15.143:0/3499120691 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd9dc0ccea0 msgr2=0x7fd9dc1489b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:20.399 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.388+0000 7fd9db7fe700 1 --2- 172.21.15.143:0/3499120691 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd9dc0ccea0 0x7fd9dc1489b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:20.399 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.388+0000 7fd9db7fe700 1 -- 172.21.15.143:0/3499120691 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9d400bd40 con 0x7fd9dc15a8f0 2023-11-08T19:30:20.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.388+0000 7fd9e0c65700 1 --2- 172.21.15.143:0/3499120691 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd9dc15acd0 0x7fd9dc147ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:30:20.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.388+0000 7fd9db7fe700 1 --2- 172.21.15.143:0/3499120691 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd9dc15a8f0 0x7fd9dc0cb310 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7fd9d000b450 tx=0x7fd9d000b760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:20.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.388+0000 7fd9d8ff9700 1 -- 172.21.15.143:0/3499120691 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d0011830 con 0x7fd9dc15a8f0 2023-11-08T19:30:20.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.388+0000 7fd9d8ff9700 1 -- 172.21.15.143:0/3499120691 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9d0011990 con 0x7fd9dc15a8f0 2023-11-08T19:30:20.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.388+0000 7fd9e26c8700 1 -- 172.21.15.143:0/3499120691 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9dc0b3650 con 0x7fd9dc15a8f0 2023-11-08T19:30:20.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.388+0000 7fd9d8ff9700 1 -- 172.21.15.143:0/3499120691 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d0011d40 con 0x7fd9dc15a8f0 2023-11-08T19:30:20.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.388+0000 7fd9e26c8700 1 -- 172.21.15.143:0/3499120691 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9dc0b3c10 con 0x7fd9dc15a8f0 2023-11-08T19:30:20.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.389+0000 7fd9e26c8700 1 -- 172.21.15.143:0/3499120691 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9dc04f730 con 0x7fd9dc15a8f0 2023-11-08T19:30:20.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.390+0000 7fd9d8ff9700 1 -- 172.21.15.143:0/3499120691 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd9d001a400 con 0x7fd9dc15a8f0 2023-11-08T19:30:20.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.390+0000 7fd9d8ff9700 1 --2- 172.21.15.143:0/3499120691 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd9c806af00 0x7fd9c806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:20.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.390+0000 7fd9d8ff9700 1 -- 172.21.15.143:0/3499120691 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 7145+0+0 (secure 0 0 0) 0x7fd9d0089810 con 0x7fd9dc15a8f0 2023-11-08T19:30:20.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.390+0000 7fd9dbfff700 1 --2- 172.21.15.143:0/3499120691 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd9c806af00 0x7fd9c806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:20.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.391+0000 7fd9dbfff700 1 --2- 172.21.15.143:0/3499120691 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd9c806af00 0x7fd9c806d3c0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc006fd0 tx=0x7fd9cc008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:20.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.395+0000 7fd9d8ff9700 1 -- 172.21.15.143:0/3499120691 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd9d0054280 con 0x7fd9dc15a8f0 2023-11-08T19:30:20.628 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.627+0000 7fd9e26c8700 1 -- 172.21.15.143:0/3499120691 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd9dc0cec90 con 0x7fd9dc15a8f0 2023-11-08T19:30:20.629 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.628+0000 7fd9d8ff9700 1 -- 172.21.15.143:0/3499120691 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 88 v88) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fd9d00578a0 con 0x7fd9dc15a8f0 2023-11-08T19:30:20.629 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:30:20.630 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":88,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5933,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/3918286710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6835","nonce":3918286710},{"type":"v1","addr":"172.21.15.174:6837","nonce":3918286710}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":6161,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/2129643800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":2129643800},{"type":"v1","addr":"172.21.15.143:6835","nonce":2129643800}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":88,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:03.092245+0000","modified":"2023-11-08T19:30:15.587914+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":98,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7494,"mds_1":6918},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6918":{"gid":6918,"name":"b","rank":1,"incarnation":54,"state":"up:active","state_seq":6,"addr":"172.21.15.174:6836/3601088618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3601088618},{"type":"v1","addr":"172.21.15.174:6836","nonce":3601088618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7494":{"gid":7494,"name":"c","rank":0,"incarnation":84,"state":"up:active","state_seq":12,"addr":"172.21.15.143:6834/1535103237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":1535103237},{"type":"v1","addr":"172.21.15.143:6834","nonce":1535103237}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:30:20.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.630+0000 7fd9e26c8700 1 -- 172.21.15.143:0/3499120691 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd9c806af00 msgr2=0x7fd9c806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:20.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.630+0000 7fd9e26c8700 1 --2- 172.21.15.143:0/3499120691 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd9c806af00 0x7fd9c806d3c0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc006fd0 tx=0x7fd9cc008040 comp rx=0 tx=0).stop 2023-11-08T19:30:20.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.630+0000 7fd9e26c8700 1 -- 172.21.15.143:0/3499120691 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd9dc15a8f0 msgr2=0x7fd9dc0cb310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:20.633 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.630+0000 7fd9e26c8700 1 --2- 172.21.15.143:0/3499120691 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd9dc15a8f0 0x7fd9dc0cb310 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7fd9d000b450 tx=0x7fd9d000b760 comp rx=0 tx=0).stop 2023-11-08T19:30:20.633 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.630+0000 7fd9e26c8700 1 -- 172.21.15.143:0/3499120691 shutdown_connections 2023-11-08T19:30:20.633 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.630+0000 7fd9e26c8700 1 --2- 172.21.15.143:0/3499120691 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd9c806af00 0x7fd9c806d3c0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:20.634 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.630+0000 7fd9e26c8700 1 --2- 172.21.15.143:0/3499120691 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd9dc15acd0 0x7fd9dc147ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:20.634 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.630+0000 7fd9e26c8700 1 --2- 172.21.15.143:0/3499120691 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd9dc0ccea0 0x7fd9dc1489b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:20.634 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.630+0000 7fd9e26c8700 1 --2- 172.21.15.143:0/3499120691 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd9dc15a8f0 0x7fd9dc0cb310 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:20.634 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.630+0000 7fd9e26c8700 1 -- 172.21.15.143:0/3499120691 >> 172.21.15.143:0/3499120691 conn(0x7fd9dc0b23d0 msgr2=0x7fd9dc06abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:20.635 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.630+0000 7fd9e26c8700 1 -- 172.21.15.143:0/3499120691 shutdown_connections 2023-11-08T19:30:20.635 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.630+0000 7fd9e26c8700 1 -- 172.21.15.143:0/3499120691 wait complete. 2023-11-08T19:30:20.635 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 88 2023-11-08T19:30:20.644 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json dump snaps --server 2023-11-08T19:30:20.779 INFO:teuthology.orchestra.run.smithi143.stdout:{"last_snap":9,"last_created":5,"last_destroyed":8,"pending_noop":[],"snaps":[{"snapid":5,"ino":1099511627777,"stamp":"2023-11-08T19:29:49.829697+0000","name":"s16","metadata":{}}],"need_to_purge":{"16":[2,3,4,6,7,8]},"pending_update":[],"pending_destroy":[{"version":16,"removed_snap":5,"seq":9}]} 2023-11-08T19:30:20.780 DEBUG:tasks.cephfs.filesystem:_json_asok output { "last_created": 5, "last_destroyed": 8, "last_snap": 9, "need_to_purge": { "16": [ 2, 3, 4, 6, 7, 8 ] }, "pending_destroy": [ { "removed_snap": 5, "seq": 9, "version": 16 } ], "pending_noop": [], "pending_update": [], "snaps": [ { "ino": 1099511627777, "metadata": {}, "name": "s16", "snapid": 5, "stamp": "2023-11-08T19:29:49.829697+0000" } ] } 2023-11-08T19:30:20.781 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds freeze 6:0 true 2023-11-08T19:30:20.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.901+0000 7f5eb17b9700 1 Processor -- start 2023-11-08T19:30:20.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.901+0000 7f5eb17b9700 1 -- start start 2023-11-08T19:30:20.910 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.902+0000 7f5eb17b9700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f5eac06abe0 0x7f5eac0b2770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:20.910 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.902+0000 7f5eb17b9700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5eac06e0f0 0x7f5eac06e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:20.910 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.902+0000 7f5eb17b9700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5eac0724b0 con 0x7f5eac06afc0 2023-11-08T19:30:20.910 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.902+0000 7f5eb17b9700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5eac072a70 con 0x7f5eac06abe0 2023-11-08T19:30:20.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.902+0000 7f5eb17b9700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5eac06eac0 con 0x7f5eac06e0f0 2023-11-08T19:30:20.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.902+0000 7f5eaa7fc700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f5eac06afc0 0x7f5eac0cbe30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:52404/0 (socket says 172.21.15.143:52404) 2023-11-08T19:30:20.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.902+0000 7f5eaa7fc700 1 -- 172.21.15.143:0/4056742462 learned_addr learned my addr 172.21.15.143:0/4056742462 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:30:20.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.902+0000 7f5ea9ffb700 1 -- 172.21.15.143:0/4056742462 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 284602660 0 0) 0x7f5eac0724b0 con 0x7f5eac06afc0 2023-11-08T19:30:20.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.902+0000 7f5ea9ffb700 1 -- 172.21.15.143:0/4056742462 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e940035b0 con 0x7f5eac06afc0 2023-11-08T19:30:20.912 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.902+0000 7f5eab7fe700 1 --2- 172.21.15.143:0/4056742462 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5eac06e0f0 0x7f5eac06e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:20.912 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.902+0000 7f5eaaffd700 1 --2- 172.21.15.143:0/4056742462 >> v2:172.21.15.174:3300/0 conn(0x7f5eac06abe0 0x7f5eac0b2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:20.912 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.903+0000 7f5ea9ffb700 1 -- 172.21.15.143:0/4056742462 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3477317328 0 0) 0x7f5e940035b0 con 0x7f5eac06afc0 2023-11-08T19:30:20.912 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.903+0000 7f5ea9ffb700 1 -- 172.21.15.143:0/4056742462 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5eac06e0f0 msgr2=0x7f5eac06e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:20.912 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.903+0000 7f5ea9ffb700 1 --2- 172.21.15.143:0/4056742462 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5eac06e0f0 0x7f5eac06e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:20.913 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.903+0000 7f5ea9ffb700 1 -- 172.21.15.143:0/4056742462 >> v2:172.21.15.174:3300/0 conn(0x7f5eac06abe0 msgr2=0x7f5eac0b2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:20.913 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.903+0000 7f5ea9ffb700 1 --2- 172.21.15.143:0/4056742462 >> v2:172.21.15.174:3300/0 conn(0x7f5eac06abe0 0x7f5eac0b2770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:20.913 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.903+0000 7f5ea9ffb700 1 -- 172.21.15.143:0/4056742462 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5eac072790 con 0x7f5eac06afc0 2023-11-08T19:30:20.913 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.903+0000 7f5ea9ffb700 1 -- 172.21.15.143:0/4056742462 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4147756303 0 0) 0x7f5e9c001670 con 0x7f5eac06afc0 2023-11-08T19:30:20.913 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.903+0000 7f5ea9ffb700 1 -- 172.21.15.143:0/4056742462 >> v1:172.21.15.143:6789/0 conn(0x7f5eac06afc0 legacy=0x7f5eac0cbe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:20.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.903+0000 7f5ea9ffb700 1 --2- 172.21.15.143:0/4056742462 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5e94003a00 0x7f5e94003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:20.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.903+0000 7f5ea9ffb700 1 --2- 172.21.15.143:0/4056742462 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5eac06e0f0 0x7f5e94004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:20.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.903+0000 7f5ea9ffb700 1 --2- 172.21.15.143:0/4056742462 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5e94004d90 0x7f5e94005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:20.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.903+0000 7f5ea9ffb700 1 -- 172.21.15.143:0/4056742462 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eac072a70 con 0x7f5e94003a00 2023-11-08T19:30:20.915 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.903+0000 7f5ea9ffb700 1 -- 172.21.15.143:0/4056742462 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e9400ba10 con 0x7f5e94004d90 2023-11-08T19:30:20.915 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.903+0000 7f5ea9ffb700 1 -- 172.21.15.143:0/4056742462 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e9400bb50 con 0x7f5eac06e0f0 2023-11-08T19:30:20.915 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.903+0000 7f5eaaffd700 1 --2- 172.21.15.143:0/4056742462 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5e94003a00 0x7f5e94003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:20.915 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.903+0000 7f5eab7fe700 1 --2- 172.21.15.143:0/4056742462 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5e94004d90 0x7f5e94005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:20.915 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.903+0000 7f5eaa7fc700 1 --2- 172.21.15.143:0/4056742462 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5eac06e0f0 0x7f5e94004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:20.915 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.903+0000 7f5eaaffd700 1 -- 172.21.15.143:0/4056742462 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5eac06e0f0 msgr2=0x7f5e94004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:20.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.903+0000 7f5eaaffd700 1 --2- 172.21.15.143:0/4056742462 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5eac06e0f0 0x7f5e94004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:20.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.903+0000 7f5eaaffd700 1 -- 172.21.15.143:0/4056742462 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5e94004d90 msgr2=0x7f5e94005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:20.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.903+0000 7f5eaaffd700 1 --2- 172.21.15.143:0/4056742462 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5e94004d90 0x7f5e94005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:20.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.903+0000 7f5eaaffd700 1 -- 172.21.15.143:0/4056742462 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e940035b0 con 0x7f5e94003a00 2023-11-08T19:30:20.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.903+0000 7f5eaaffd700 1 --2- 172.21.15.143:0/4056742462 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5e94003a00 0x7f5e94003e20 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f5ea0009b90 tx=0x7f5e94005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73c5071f499aec9a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:20.917 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.904+0000 7f5ea9ffb700 1 -- 172.21.15.143:0/4056742462 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea000e6c0 con 0x7f5e94003a00 2023-11-08T19:30:20.917 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.904+0000 7f5ea9ffb700 1 -- 172.21.15.143:0/4056742462 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ea000e820 con 0x7f5e94003a00 2023-11-08T19:30:20.917 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.904+0000 7f5ea9ffb700 1 -- 172.21.15.143:0/4056742462 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea000eb90 con 0x7f5e94003a00 2023-11-08T19:30:20.917 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.904+0000 7f5eb17b9700 1 -- 172.21.15.143:0/4056742462 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5e94003a00 msgr2=0x7f5e94003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:20.917 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.904+0000 7f5eb17b9700 1 --2- 172.21.15.143:0/4056742462 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5e94003a00 0x7f5e94003e20 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f5ea0009b90 tx=0x7f5e94005880 comp rx=0 tx=0).stop 2023-11-08T19:30:20.918 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.904+0000 7f5eab7fe700 1 -- 172.21.15.143:0/4056742462 reap_dead start 2023-11-08T19:30:20.918 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.904+0000 7f5eb17b9700 1 -- 172.21.15.143:0/4056742462 shutdown_connections 2023-11-08T19:30:20.918 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.904+0000 7f5eb17b9700 1 -- 172.21.15.143:0/4056742462 >> 172.21.15.143:0/4056742462 conn(0x7f5eac06bcf0 msgr2=0x7f5eac06c100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:20.918 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.904+0000 7f5eb17b9700 1 -- 172.21.15.143:0/4056742462 shutdown_connections 2023-11-08T19:30:20.918 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.904+0000 7f5eb17b9700 1 -- 172.21.15.143:0/4056742462 wait complete. 2023-11-08T19:30:20.919 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.906+0000 7f5eb17b9700 1 Processor -- start 2023-11-08T19:30:20.919 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.906+0000 7f5eb17b9700 1 -- start start 2023-11-08T19:30:20.919 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.906+0000 7f5eb17b9700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5e94003a00 0x7f5eac151ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:20.919 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.906+0000 7f5eb17b9700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5eac0b3480 0x7f5eac15f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:20.919 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.906+0000 7f5eb17b9700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5eac132cc0 0x7f5eac133140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:20.920 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.906+0000 7f5eb17b9700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eac14ef10 con 0x7f5eac132cc0 2023-11-08T19:30:20.920 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.906+0000 7f5eb17b9700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eac133680 con 0x7f5eac0b3480 2023-11-08T19:30:20.920 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.906+0000 7f5eb17b9700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5eac160830 con 0x7f5e94003a00 2023-11-08T19:30:20.920 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.906+0000 7f5eab7fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5eac132cc0 0x7f5eac133140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:20.920 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.906+0000 7f5eab7fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5eac132cc0 0x7f5eac133140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56264/0 (socket says 172.21.15.143:56264) 2023-11-08T19:30:20.921 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.906+0000 7f5eab7fe700 1 -- 172.21.15.143:0/3968777583 learned_addr learned my addr 172.21.15.143:0/3968777583 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:30:20.921 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.907+0000 7f5eaa7fc700 1 --2- 172.21.15.143:0/3968777583 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5eac0b3480 0x7f5eac15f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:20.921 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.907+0000 7f5eaaffd700 1 --2- 172.21.15.143:0/3968777583 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5e94003a00 0x7f5eac151ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:20.921 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.907+0000 7f5eab7fe700 1 -- 172.21.15.143:0/3968777583 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5e94003a00 msgr2=0x7f5eac151ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:20.921 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.907+0000 7f5eab7fe700 1 --2- 172.21.15.143:0/3968777583 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5e94003a00 0x7f5eac151ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:20.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.907+0000 7f5eab7fe700 1 -- 172.21.15.143:0/3968777583 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5eac0b3480 msgr2=0x7f5eac15f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:20.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.907+0000 7f5eab7fe700 1 --2- 172.21.15.143:0/3968777583 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5eac0b3480 0x7f5eac15f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:20.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.907+0000 7f5eab7fe700 1 -- 172.21.15.143:0/3968777583 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5eac160270 con 0x7f5eac132cc0 2023-11-08T19:30:20.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.907+0000 7f5eaa7fc700 1 --2- 172.21.15.143:0/3968777583 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5eac0b3480 0x7f5eac15f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:30:20.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.907+0000 7f5eab7fe700 1 --2- 172.21.15.143:0/3968777583 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5eac132cc0 0x7f5eac133140 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f5eac06e4d0 tx=0x7f5ea4011ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:20.923 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.907+0000 7f5e937fe700 1 -- 172.21.15.143:0/3968777583 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea400e040 con 0x7f5eac132cc0 2023-11-08T19:30:20.923 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.907+0000 7f5e937fe700 1 -- 172.21.15.143:0/3968777583 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ea400f040 con 0x7f5eac132cc0 2023-11-08T19:30:20.923 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.907+0000 7f5eb17b9700 1 -- 172.21.15.143:0/3968777583 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5eac160550 con 0x7f5eac132cc0 2023-11-08T19:30:20.923 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.907+0000 7f5e937fe700 1 -- 172.21.15.143:0/3968777583 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea400fca0 con 0x7f5eac132cc0 2023-11-08T19:30:20.923 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.907+0000 7f5eb17b9700 1 -- 172.21.15.143:0/3968777583 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5eac131040 con 0x7f5eac132cc0 2023-11-08T19:30:20.924 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.908+0000 7f5eb17b9700 1 -- 172.21.15.143:0/3968777583 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5eac04f730 con 0x7f5eac132cc0 2023-11-08T19:30:20.924 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.908+0000 7f5e937fe700 1 -- 172.21.15.143:0/3968777583 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5ea4015430 con 0x7f5eac132cc0 2023-11-08T19:30:20.924 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.909+0000 7f5e937fe700 1 --2- 172.21.15.143:0/3968777583 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f5e9806af00 0x7f5e9806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:20.924 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.909+0000 7f5e937fe700 1 -- 172.21.15.143:0/3968777583 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 7145+0+0 (secure 0 0 0) 0x7f5ea408a300 con 0x7f5eac132cc0 2023-11-08T19:30:20.924 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.909+0000 7f5eaaffd700 1 --2- 172.21.15.143:0/3968777583 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f5e9806af00 0x7f5e9806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:20.925 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.909+0000 7f5eaaffd700 1 --2- 172.21.15.143:0/3968777583 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f5e9806af00 0x7f5e9806d3c0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f5ea0009a80 tx=0x7f5ea0008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:20.925 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:20.913+0000 7f5e937fe700 1 -- 172.21.15.143:0/3968777583 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5ea4054f50 con 0x7f5eac132cc0 2023-11-08T19:30:21.183 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:21.181+0000 7f5eb17b9700 1 -- 172.21.15.143:0/3968777583 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "mds freeze", "role_or_gid": "6:0", "val": "true"} v 0) v1 -- 0x7f5eac0ca1d0 con 0x7f5eac132cc0 2023-11-08T19:30:21.634 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:21.633+0000 7f5e937fe700 1 -- 172.21.15.143:0/3968777583 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "mds freeze", "role_or_gid": "6:0", "val": "true"}]=0 freezing mds.7494 v89) v1 ==== 112+0+0 (secure 0 0 0) 0x7f5ea4058570 con 0x7f5eac132cc0 2023-11-08T19:30:21.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:21.635+0000 7f5eb17b9700 1 -- 172.21.15.143:0/3968777583 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f5e9806af00 msgr2=0x7f5e9806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:21.638 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:21.635+0000 7f5eb17b9700 1 --2- 172.21.15.143:0/3968777583 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f5e9806af00 0x7f5e9806d3c0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f5ea0009a80 tx=0x7f5ea0008030 comp rx=0 tx=0).stop 2023-11-08T19:30:21.638 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:21.636+0000 7f5eb17b9700 1 -- 172.21.15.143:0/3968777583 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5eac132cc0 msgr2=0x7f5eac133140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:21.638 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:21.636+0000 7f5eb17b9700 1 --2- 172.21.15.143:0/3968777583 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5eac132cc0 0x7f5eac133140 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f5eac06e4d0 tx=0x7f5ea4011ed0 comp rx=0 tx=0).stop 2023-11-08T19:30:21.638 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:21.636+0000 7f5eb17b9700 1 -- 172.21.15.143:0/3968777583 shutdown_connections 2023-11-08T19:30:21.638 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:21.636+0000 7f5eb17b9700 1 --2- 172.21.15.143:0/3968777583 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f5e9806af00 0x7f5e9806d3c0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:21.639 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:21.636+0000 7f5eb17b9700 1 --2- 172.21.15.143:0/3968777583 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5e94003a00 0x7f5eac151ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:21.639 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:21.636+0000 7f5eb17b9700 1 --2- 172.21.15.143:0/3968777583 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5eac132cc0 0x7f5eac133140 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:21.639 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:21.636+0000 7f5eb17b9700 1 --2- 172.21.15.143:0/3968777583 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5eac0b3480 0x7f5eac15f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:21.639 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:21.636+0000 7f5eb17b9700 1 -- 172.21.15.143:0/3968777583 >> 172.21.15.143:0/3968777583 conn(0x7f5eac06bcf0 msgr2=0x7f5eac0ccf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:21.639 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:21.636+0000 7f5eb17b9700 1 -- 172.21.15.143:0/3968777583 shutdown_connections 2023-11-08T19:30:21.640 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:21.636+0000 7f5eb17b9700 1 -- 172.21.15.143:0/3968777583 wait complete. 2023-11-08T19:30:21.640 INFO:teuthology.orchestra.run.smithi143.stderr:freezing mds.7494 2023-11-08T19:30:21.651 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds freeze 6:1 true 2023-11-08T19:30:21.787 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:21.785+0000 7fcc93bac700 1 Processor -- start 2023-11-08T19:30:21.787 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:21.786+0000 7fcc93bac700 1 -- start start 2023-11-08T19:30:21.789 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:21.786+0000 7fcc93bac700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fcc8c072460 0x7fcc8c0cd960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:21.790 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:21.786+0000 7fcc93bac700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc8c071fd0 con 0x7fcc8c0cc720 2023-11-08T19:30:21.790 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:21.786+0000 7fcc93bac700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc8c06d800 con 0x7fcc8c15aab0 2023-11-08T19:30:21.790 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:21.786+0000 7fcc93bac700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc8c0716e0 con 0x7fcc8c072460 2023-11-08T19:30:21.790 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:21.786+0000 7fcc92149700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fcc8c072460 0x7fcc8c0cd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:21.790 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:21.786+0000 7fcc91147700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fcc8c0cc720 0x7fcc8c0b4870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:52424/0 (socket says 172.21.15.143:52424) 2023-11-08T19:30:21.791 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:21.786+0000 7fcc91147700 1 -- 172.21.15.143:0/3555893023 learned_addr learned my addr 172.21.15.143:0/3555893023 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:30:21.791 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:21.787+0000 7fcc92149700 1 -- 172.21.15.143:0/3555893023 >> v1:172.21.15.174:6789/0 conn(0x7fcc8c15aab0 legacy=0x7fcc8c0cc010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:21.791 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:21.787+0000 7fcc92149700 1 -- 172.21.15.143:0/3555893023 >> v1:172.21.15.143:6789/0 conn(0x7fcc8c0cc720 legacy=0x7fcc8c0b4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:21.791 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:21.787+0000 7fcc92149700 1 -- 172.21.15.143:0/3555893023 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc8c06daa0 con 0x7fcc8c072460 2023-11-08T19:30:21.791 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:21.787+0000 7fcc90946700 1 -- 172.21.15.143:0/3555893023 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3801748048 0 0) 0x7fcc8c071fd0 con 0x7fcc8c0cc720 2023-11-08T19:30:21.792 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:21.787+0000 7fcc92149700 1 --2- 172.21.15.143:0/3555893023 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcc8c072460 0x7fcc8c0cd960 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7fcc84009d90 tx=0x7fcc8400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c9c43547ba07565 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:21.792 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:21.787+0000 7fcc90946700 1 -- 172.21.15.143:0/3555893023 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc8400e070 con 0x7fcc8c072460 2023-11-08T19:30:21.792 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:21.787+0000 7fcc90946700 1 -- 172.21.15.143:0/3555893023 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc8400bbd0 con 0x7fcc8c072460 2023-11-08T19:30:21.792 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:21.787+0000 7fcc90946700 1 -- 172.21.15.143:0/3555893023 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc84012430 con 0x7fcc8c072460 2023-11-08T19:30:21.792 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:21.788+0000 7fcc93bac700 1 -- 172.21.15.143:0/3555893023 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcc8c072460 msgr2=0x7fcc8c0cd960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:21.793 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:21.788+0000 7fcc93bac700 1 --2- 172.21.15.143:0/3555893023 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcc8c072460 0x7fcc8c0cd960 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7fcc84009d90 tx=0x7fcc8400b330 comp rx=0 tx=0).stop 2023-11-08T19:30:21.793 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:21.788+0000 7fcc93bac700 1 -- 172.21.15.143:0/3555893023 shutdown_connections 2023-11-08T19:30:21.793 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:21.788+0000 7fcc93bac700 1 --2- 172.21.15.143:0/3555893023 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcc8c072460 0x7fcc8c0cd960 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:21.793 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:21.788+0000 7fcc93bac700 1 -- 172.21.15.143:0/3555893023 >> 172.21.15.143:0/3555893023 conn(0x7fcc8c06bcf0 msgr2=0x7fcc8c06c100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:21.793 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:21.788+0000 7fcc93bac700 1 -- 172.21.15.143:0/3555893023 shutdown_connections 2023-11-08T19:30:21.794 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:21.788+0000 7fcc93bac700 1 -- 172.21.15.143:0/3555893023 wait complete. 2023-11-08T19:30:21.797 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:21.789+0000 7fcc93bac700 1 Processor -- start 2023-11-08T19:30:21.797 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:21.789+0000 7fcc93bac700 1 -- start start 2023-11-08T19:30:21.797 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:21.790+0000 7fcc93bac700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcc8c072460 0x7fcc8c14b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:21.797 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:21.790+0000 7fcc93bac700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcc8c0cc720 0x7fcc8c06abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:21.797 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:21.790+0000 7fcc93bac700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcc8c15aab0 0x7fcc8c06b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:21.798 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:21.790+0000 7fcc93bac700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc8c158250 con 0x7fcc8c072460 2023-11-08T19:30:21.798 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:21.790+0000 7fcc93bac700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc8c158770 con 0x7fcc8c15aab0 2023-11-08T19:30:21.798 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:21.790+0000 7fcc93bac700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc8c158a50 con 0x7fcc8c0cc720 2023-11-08T19:30:21.798 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:21.790+0000 7fcc91948700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcc8c072460 0x7fcc8c14b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:21.799 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:21.790+0000 7fcc91948700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcc8c072460 0x7fcc8c14b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56274/0 (socket says 172.21.15.143:56274) 2023-11-08T19:30:21.799 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:21.790+0000 7fcc91948700 1 -- 172.21.15.143:0/50094691 learned_addr learned my addr 172.21.15.143:0/50094691 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:30:21.799 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:21.790+0000 7fcc92149700 1 --2- 172.21.15.143:0/50094691 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcc8c15aab0 0x7fcc8c06b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:21.799 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:21.790+0000 7fcc91147700 1 --2- 172.21.15.143:0/50094691 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcc8c0cc720 0x7fcc8c06abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:21.799 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:21.790+0000 7fcc91948700 1 -- 172.21.15.143:0/50094691 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcc8c0cc720 msgr2=0x7fcc8c06abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:21.800 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:21.790+0000 7fcc91948700 1 --2- 172.21.15.143:0/50094691 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcc8c0cc720 0x7fcc8c06abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:21.800 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:21.790+0000 7fcc91948700 1 -- 172.21.15.143:0/50094691 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcc8c15aab0 msgr2=0x7fcc8c06b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:21.800 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:21.790+0000 7fcc91948700 1 --2- 172.21.15.143:0/50094691 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcc8c15aab0 0x7fcc8c06b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:21.800 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:21.790+0000 7fcc91948700 1 -- 172.21.15.143:0/50094691 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc84009a70 con 0x7fcc8c072460 2023-11-08T19:30:21.800 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:21.790+0000 7fcc92149700 1 --2- 172.21.15.143:0/50094691 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcc8c15aab0 0x7fcc8c06b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:30:21.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:21.790+0000 7fcc91948700 1 --2- 172.21.15.143:0/50094691 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcc8c072460 0x7fcc8c14b140 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7fcc8000c450 tx=0x7fcc8000c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:21.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:21.790+0000 7fcc7e7fc700 1 -- 172.21.15.143:0/50094691 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc800052d0 con 0x7fcc8c072460 2023-11-08T19:30:21.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:21.790+0000 7fcc7e7fc700 1 -- 172.21.15.143:0/50094691 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc80005430 con 0x7fcc8c072460 2023-11-08T19:30:21.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:21.790+0000 7fcc93bac700 1 -- 172.21.15.143:0/50094691 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc8c158d30 con 0x7fcc8c072460 2023-11-08T19:30:21.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:21.790+0000 7fcc7e7fc700 1 -- 172.21.15.143:0/50094691 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc800038c0 con 0x7fcc8c072460 2023-11-08T19:30:21.802 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:21.790+0000 7fcc93bac700 1 -- 172.21.15.143:0/50094691 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc8c14b680 con 0x7fcc8c072460 2023-11-08T19:30:21.802 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:21.791+0000 7fcc93bac700 1 -- 172.21.15.143:0/50094691 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc8c04f730 con 0x7fcc8c072460 2023-11-08T19:30:21.802 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:21.792+0000 7fcc7e7fc700 1 -- 172.21.15.143:0/50094691 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fcc80003a20 con 0x7fcc8c072460 2023-11-08T19:30:21.802 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:21.792+0000 7fcc7e7fc700 1 --2- 172.21.15.143:0/50094691 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fcc7006af00 0x7fcc7006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:21.802 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:21.792+0000 7fcc7e7fc700 1 -- 172.21.15.143:0/50094691 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 7233+0+0 (secure 0 0 0) 0x7fcc80088ea0 con 0x7fcc8c072460 2023-11-08T19:30:21.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:21.793+0000 7fcc91147700 1 --2- 172.21.15.143:0/50094691 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fcc7006af00 0x7fcc7006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:21.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:21.793+0000 7fcc91147700 1 --2- 172.21.15.143:0/50094691 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fcc7006af00 0x7fcc7006d3c0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c0a9bf0 tx=0x7fcc88009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:21.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:21.797+0000 7fcc7e7fc700 1 -- 172.21.15.143:0/50094691 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcc800539e0 con 0x7fcc8c072460 2023-11-08T19:30:22.070 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:22.068+0000 7fcc93bac700 1 -- 172.21.15.143:0/50094691 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "mds freeze", "role_or_gid": "6:1", "val": "true"} v 0) v1 -- 0x7fcc8c0cb910 con 0x7fcc8c072460 2023-11-08T19:30:22.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:22.641+0000 7fcc7e7fc700 1 -- 172.21.15.143:0/50094691 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "mds freeze", "role_or_gid": "6:1", "val": "true"}]=0 freezing mds.6918 v90) v1 ==== 112+0+0 (secure 0 0 0) 0x7fcc80057000 con 0x7fcc8c072460 2023-11-08T19:30:22.647 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:22.645+0000 7fcc93bac700 1 -- 172.21.15.143:0/50094691 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fcc7006af00 msgr2=0x7fcc7006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:22.647 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:22.645+0000 7fcc93bac700 1 --2- 172.21.15.143:0/50094691 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fcc7006af00 0x7fcc7006d3c0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c0a9bf0 tx=0x7fcc88009040 comp rx=0 tx=0).stop 2023-11-08T19:30:22.647 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:22.645+0000 7fcc93bac700 1 -- 172.21.15.143:0/50094691 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcc8c072460 msgr2=0x7fcc8c14b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:22.648 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:22.645+0000 7fcc93bac700 1 --2- 172.21.15.143:0/50094691 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcc8c072460 0x7fcc8c14b140 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7fcc8000c450 tx=0x7fcc8000c810 comp rx=0 tx=0).stop 2023-11-08T19:30:22.648 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:22.645+0000 7fcc93bac700 1 -- 172.21.15.143:0/50094691 shutdown_connections 2023-11-08T19:30:22.648 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:22.645+0000 7fcc93bac700 1 --2- 172.21.15.143:0/50094691 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fcc7006af00 0x7fcc7006d3c0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:22.648 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:22.645+0000 7fcc93bac700 1 --2- 172.21.15.143:0/50094691 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcc8c15aab0 0x7fcc8c06b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:22.648 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:22.645+0000 7fcc93bac700 1 --2- 172.21.15.143:0/50094691 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcc8c072460 0x7fcc8c14b140 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:22.649 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:22.645+0000 7fcc93bac700 1 --2- 172.21.15.143:0/50094691 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcc8c0cc720 0x7fcc8c06abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:22.649 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:22.645+0000 7fcc93bac700 1 -- 172.21.15.143:0/50094691 >> 172.21.15.143:0/50094691 conn(0x7fcc8c06bcf0 msgr2=0x7fcc8c0cf730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:22.649 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:22.645+0000 7fcc93bac700 1 -- 172.21.15.143:0/50094691 shutdown_connections 2023-11-08T19:30:22.649 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:22.645+0000 7fcc93bac700 1 -- 172.21.15.143:0/50094691 wait complete. 2023-11-08T19:30:22.650 INFO:teuthology.orchestra.run.smithi143.stderr:freezing mds.6918 2023-11-08T19:30:22.660 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json config set mds_kill_mdstable_at 2 2023-11-08T19:30:22.802 INFO:teuthology.orchestra.run.smithi143.stdout:{"success":"mds_kill_mdstable_at = '2' (not observed, change may require restart) "} 2023-11-08T19:30:22.803 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_kill_mdstable_at = '2' (not observed, change may require restart) " } 2023-11-08T19:30:22.803 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:30:22.804 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir d1/dir/.snap/s22) 2023-11-08T19:30:22.807 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mds_beacon_grace 2023-11-08T19:30:22.823 INFO:tasks.ceph.mds.c.smithi143.stderr:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el8/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: In function 'void MDSTableServer::_prepare_logged(ceph::cref_t&, version_t)' thread 7f2b9aacd700 time 2023-11-08T19:30:22.823177+0000 2023-11-08T19:30:22.823 INFO:tasks.ceph.mds.c.smithi143.stderr:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el8/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: 77: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 2) 2023-11-08T19:30:22.825 INFO:tasks.ceph.mds.c.smithi143.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-08T19:30:22.825 INFO:tasks.ceph.mds.c.smithi143.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f2bac3a1f5b] 2023-11-08T19:30:22.825 INFO:tasks.ceph.mds.c.smithi143.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2ba121) [0x7f2bac3a2121] 2023-11-08T19:30:22.825 INFO:tasks.ceph.mds.c.smithi143.stderr: 3: (MDSTableServer::_prepare_logged(boost::intrusive_ptr const&, unsigned long)+0x6ad) [0x564177b35f5d] 2023-11-08T19:30:22.826 INFO:tasks.ceph.mds.c.smithi143.stderr: 4: (MDSContext::complete(int)+0x5f) [0x564177b80ecf] 2023-11-08T19:30:22.826 INFO:tasks.ceph.mds.c.smithi143.stderr: 5: (MDSIOContextBase::complete(int)+0x574) [0x564177b816b4] 2023-11-08T19:30:22.826 INFO:tasks.ceph.mds.c.smithi143.stderr: 6: (MDSLogContextBase::complete(int)+0x5b) [0x564177b81a7b] 2023-11-08T19:30:22.826 INFO:tasks.ceph.mds.c.smithi143.stderr: 7: (Finisher::finisher_thread_entry()+0x19d) [0x7f2bac43f8cd] 2023-11-08T19:30:22.826 INFO:tasks.ceph.mds.c.smithi143.stderr: 8: /lib64/libpthread.so.0(+0x81cf) [0x7f2bab13a1cf] 2023-11-08T19:30:22.827 INFO:tasks.ceph.mds.c.smithi143.stderr: 9: clone() 2023-11-08T19:30:22.827 INFO:tasks.ceph.mds.c.smithi143.stderr:2023-11-08T19:30:22.823+0000 7f2b9aacd700 -1 /home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el8/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: In function 'void MDSTableServer::_prepare_logged(ceph::cref_t&, version_t)' thread 7f2b9aacd700 time 2023-11-08T19:30:22.823177+0000 2023-11-08T19:30:22.827 INFO:tasks.ceph.mds.c.smithi143.stderr:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el8/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: 77: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 2) 2023-11-08T19:30:22.827 INFO:tasks.ceph.mds.c.smithi143.stderr: 2023-11-08T19:30:22.827 INFO:tasks.ceph.mds.c.smithi143.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-08T19:30:22.828 INFO:tasks.ceph.mds.c.smithi143.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f2bac3a1f5b] 2023-11-08T19:30:22.828 INFO:tasks.ceph.mds.c.smithi143.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2ba121) [0x7f2bac3a2121] 2023-11-08T19:30:22.828 INFO:tasks.ceph.mds.c.smithi143.stderr: 3: (MDSTableServer::_prepare_logged(boost::intrusive_ptr const&, unsigned long)+0x6ad) [0x564177b35f5d] 2023-11-08T19:30:22.828 INFO:tasks.ceph.mds.c.smithi143.stderr: 4: (MDSContext::complete(int)+0x5f) [0x564177b80ecf] 2023-11-08T19:30:22.829 INFO:tasks.ceph.mds.c.smithi143.stderr: 5: (MDSIOContextBase::complete(int)+0x574) [0x564177b816b4] 2023-11-08T19:30:22.829 INFO:tasks.ceph.mds.c.smithi143.stderr: 6: (MDSLogContextBase::complete(int)+0x5b) [0x564177b81a7b] 2023-11-08T19:30:22.829 INFO:tasks.ceph.mds.c.smithi143.stderr: 7: (Finisher::finisher_thread_entry()+0x19d) [0x7f2bac43f8cd] 2023-11-08T19:30:22.829 INFO:tasks.ceph.mds.c.smithi143.stderr: 8: /lib64/libpthread.so.0(+0x81cf) [0x7f2bab13a1cf] 2023-11-08T19:30:22.829 INFO:tasks.ceph.mds.c.smithi143.stderr: 9: clone() 2023-11-08T19:30:22.830 INFO:tasks.ceph.mds.c.smithi143.stderr: 2023-11-08T19:30:22.830 INFO:tasks.ceph.mds.c.smithi143.stderr:*** Caught signal (Aborted) ** 2023-11-08T19:30:22.830 INFO:tasks.ceph.mds.c.smithi143.stderr: in thread 7f2b9aacd700 thread_name:MR_Finisher 2023-11-08T19:30:22.830 INFO:tasks.ceph.mds.c.smithi143.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-08T19:30:22.831 INFO:tasks.ceph.mds.c.smithi143.stderr: 1: /lib64/libpthread.so.0(+0x12ce0) [0x7f2bab144ce0] 2023-11-08T19:30:22.831 INFO:tasks.ceph.mds.c.smithi143.stderr: 2: gsignal() 2023-11-08T19:30:22.831 INFO:tasks.ceph.mds.c.smithi143.stderr: 3: abort() 2023-11-08T19:30:22.831 INFO:tasks.ceph.mds.c.smithi143.stderr: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f2bac3a1fb5] 2023-11-08T19:30:22.831 INFO:tasks.ceph.mds.c.smithi143.stderr: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2ba121) [0x7f2bac3a2121] 2023-11-08T19:30:22.832 INFO:tasks.ceph.mds.c.smithi143.stderr: 6: (MDSTableServer::_prepare_logged(boost::intrusive_ptr const&, unsigned long)+0x6ad) [0x564177b35f5d] 2023-11-08T19:30:22.832 INFO:tasks.ceph.mds.c.smithi143.stderr: 7: (MDSContext::complete(int)+0x5f) [0x564177b80ecf] 2023-11-08T19:30:22.832 INFO:tasks.ceph.mds.c.smithi143.stderr: 8: (MDSIOContextBase::complete(int)+0x574) [0x564177b816b4] 2023-11-08T19:30:22.832 INFO:tasks.ceph.mds.c.smithi143.stderr: 9: (MDSLogContextBase::complete(int)+0x5b) [0x564177b81a7b] 2023-11-08T19:30:22.832 INFO:tasks.ceph.mds.c.smithi143.stderr: 10: (Finisher::finisher_thread_entry()+0x19d) [0x7f2bac43f8cd] 2023-11-08T19:30:22.833 INFO:tasks.ceph.mds.c.smithi143.stderr: 11: /lib64/libpthread.so.0(+0x81cf) [0x7f2bab13a1cf] 2023-11-08T19:30:22.833 INFO:tasks.ceph.mds.c.smithi143.stderr: 12: clone() 2023-11-08T19:30:22.833 INFO:tasks.ceph.mds.c.smithi143.stderr:2023-11-08T19:30:22.824+0000 7f2b9aacd700 -1 *** Caught signal (Aborted) ** 2023-11-08T19:30:22.833 INFO:tasks.ceph.mds.c.smithi143.stderr: in thread 7f2b9aacd700 thread_name:MR_Finisher 2023-11-08T19:30:22.834 INFO:tasks.ceph.mds.c.smithi143.stderr: 2023-11-08T19:30:22.834 INFO:tasks.ceph.mds.c.smithi143.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-08T19:30:22.834 INFO:tasks.ceph.mds.c.smithi143.stderr: 1: /lib64/libpthread.so.0(+0x12ce0) [0x7f2bab144ce0] 2023-11-08T19:30:22.834 INFO:tasks.ceph.mds.c.smithi143.stderr: 2: gsignal() 2023-11-08T19:30:22.834 INFO:tasks.ceph.mds.c.smithi143.stderr: 3: abort() 2023-11-08T19:30:22.835 INFO:tasks.ceph.mds.c.smithi143.stderr: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f2bac3a1fb5] 2023-11-08T19:30:22.835 INFO:tasks.ceph.mds.c.smithi143.stderr: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2ba121) [0x7f2bac3a2121] 2023-11-08T19:30:22.835 INFO:tasks.ceph.mds.c.smithi143.stderr: 6: (MDSTableServer::_prepare_logged(boost::intrusive_ptr const&, unsigned long)+0x6ad) [0x564177b35f5d] 2023-11-08T19:30:22.835 INFO:tasks.ceph.mds.c.smithi143.stderr: 7: (MDSContext::complete(int)+0x5f) [0x564177b80ecf] 2023-11-08T19:30:22.836 INFO:tasks.ceph.mds.c.smithi143.stderr: 8: (MDSIOContextBase::complete(int)+0x574) [0x564177b816b4] 2023-11-08T19:30:22.836 INFO:tasks.ceph.mds.c.smithi143.stderr: 9: (MDSLogContextBase::complete(int)+0x5b) [0x564177b81a7b] 2023-11-08T19:30:22.836 INFO:tasks.ceph.mds.c.smithi143.stderr: 10: (Finisher::finisher_thread_entry()+0x19d) [0x7f2bac43f8cd] 2023-11-08T19:30:22.836 INFO:tasks.ceph.mds.c.smithi143.stderr: 11: /lib64/libpthread.so.0(+0x81cf) [0x7f2bab13a1cf] 2023-11-08T19:30:22.836 INFO:tasks.ceph.mds.c.smithi143.stderr: 12: clone() 2023-11-08T19:30:22.836 INFO:tasks.ceph.mds.c.smithi143.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-11-08T19:30:22.837 INFO:tasks.ceph.mds.c.smithi143.stderr: 2023-11-08T19:30:22.838 INFO:tasks.ceph.mds.c.smithi143.stderr: -847> 2023-11-08T19:30:17.168+0000 7f2b9e2d4700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-08T19:30:22.838 INFO:tasks.ceph.mds.c.smithi143.stderr: -845> 2023-11-08T19:30:17.168+0000 7f2b9e2d4700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-11-08T19:30:22.838 INFO:tasks.ceph.mds.c.smithi143.stderr: -1> 2023-11-08T19:30:22.823+0000 7f2b9aacd700 -1 /home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el8/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: In function 'void MDSTableServer::_prepare_logged(ceph::cref_t&, version_t)' thread 7f2b9aacd700 time 2023-11-08T19:30:22.823177+0000 2023-11-08T19:30:22.838 INFO:tasks.ceph.mds.c.smithi143.stderr:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el8/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: 77: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 2) 2023-11-08T19:30:22.838 INFO:tasks.ceph.mds.c.smithi143.stderr: 2023-11-08T19:30:22.839 INFO:tasks.ceph.mds.c.smithi143.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-08T19:30:22.839 INFO:tasks.ceph.mds.c.smithi143.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f2bac3a1f5b] 2023-11-08T19:30:22.839 INFO:tasks.ceph.mds.c.smithi143.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2ba121) [0x7f2bac3a2121] 2023-11-08T19:30:22.839 INFO:tasks.ceph.mds.c.smithi143.stderr: 3: (MDSTableServer::_prepare_logged(boost::intrusive_ptr const&, unsigned long)+0x6ad) [0x564177b35f5d] 2023-11-08T19:30:22.839 INFO:tasks.ceph.mds.c.smithi143.stderr: 4: (MDSContext::complete(int)+0x5f) [0x564177b80ecf] 2023-11-08T19:30:22.840 INFO:tasks.ceph.mds.c.smithi143.stderr: 5: (MDSIOContextBase::complete(int)+0x574) [0x564177b816b4] 2023-11-08T19:30:22.840 INFO:tasks.ceph.mds.c.smithi143.stderr: 6: (MDSLogContextBase::complete(int)+0x5b) [0x564177b81a7b] 2023-11-08T19:30:22.840 INFO:tasks.ceph.mds.c.smithi143.stderr: 7: (Finisher::finisher_thread_entry()+0x19d) [0x7f2bac43f8cd] 2023-11-08T19:30:22.840 INFO:tasks.ceph.mds.c.smithi143.stderr: 8: /lib64/libpthread.so.0(+0x81cf) [0x7f2bab13a1cf] 2023-11-08T19:30:22.840 INFO:tasks.ceph.mds.c.smithi143.stderr: 9: clone() 2023-11-08T19:30:22.841 INFO:tasks.ceph.mds.c.smithi143.stderr: 2023-11-08T19:30:22.841 INFO:tasks.ceph.mds.c.smithi143.stderr: 0> 2023-11-08T19:30:22.824+0000 7f2b9aacd700 -1 *** Caught signal (Aborted) ** 2023-11-08T19:30:22.841 INFO:tasks.ceph.mds.c.smithi143.stderr: in thread 7f2b9aacd700 thread_name:MR_Finisher 2023-11-08T19:30:22.841 INFO:tasks.ceph.mds.c.smithi143.stderr: 2023-11-08T19:30:22.841 INFO:tasks.ceph.mds.c.smithi143.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-08T19:30:22.842 INFO:tasks.ceph.mds.c.smithi143.stderr: 1: /lib64/libpthread.so.0(+0x12ce0) [0x7f2bab144ce0] 2023-11-08T19:30:22.842 INFO:tasks.ceph.mds.c.smithi143.stderr: 2: gsignal() 2023-11-08T19:30:22.842 INFO:tasks.ceph.mds.c.smithi143.stderr: 3: abort() 2023-11-08T19:30:22.842 INFO:tasks.ceph.mds.c.smithi143.stderr: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f2bac3a1fb5] 2023-11-08T19:30:22.842 INFO:tasks.ceph.mds.c.smithi143.stderr: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2ba121) [0x7f2bac3a2121] 2023-11-08T19:30:22.843 INFO:tasks.ceph.mds.c.smithi143.stderr: 6: (MDSTableServer::_prepare_logged(boost::intrusive_ptr const&, unsigned long)+0x6ad) [0x564177b35f5d] 2023-11-08T19:30:22.843 INFO:tasks.ceph.mds.c.smithi143.stderr: 7: (MDSContext::complete(int)+0x5f) [0x564177b80ecf] 2023-11-08T19:30:22.843 INFO:tasks.ceph.mds.c.smithi143.stderr: 8: (MDSIOContextBase::complete(int)+0x574) [0x564177b816b4] 2023-11-08T19:30:22.843 INFO:tasks.ceph.mds.c.smithi143.stderr: 9: (MDSLogContextBase::complete(int)+0x5b) [0x564177b81a7b] 2023-11-08T19:30:22.843 INFO:tasks.ceph.mds.c.smithi143.stderr: 10: (Finisher::finisher_thread_entry()+0x19d) [0x7f2bac43f8cd] 2023-11-08T19:30:22.844 INFO:tasks.ceph.mds.c.smithi143.stderr: 11: /lib64/libpthread.so.0(+0x81cf) [0x7f2bab13a1cf] 2023-11-08T19:30:22.844 INFO:tasks.ceph.mds.c.smithi143.stderr: 12: clone() 2023-11-08T19:30:22.844 INFO:tasks.ceph.mds.c.smithi143.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-11-08T19:30:22.844 INFO:tasks.ceph.mds.c.smithi143.stderr: 2023-11-08T19:30:22.844 INFO:tasks.ceph.mds.c.smithi143.stderr: -847> 2023-11-08T19:30:17.168+0000 7f2b9e2d4700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-08T19:30:22.845 INFO:tasks.ceph.mds.c.smithi143.stderr: -845> 2023-11-08T19:30:17.168+0000 7f2b9e2d4700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-11-08T19:30:22.845 INFO:tasks.ceph.mds.c.smithi143.stderr: -1> 2023-11-08T19:30:22.823+0000 7f2b9aacd700 -1 /home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el8/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: In function 'void MDSTableServer::_prepare_logged(ceph::cref_t&, version_t)' thread 7f2b9aacd700 time 2023-11-08T19:30:22.823177+0000 2023-11-08T19:30:22.845 INFO:tasks.ceph.mds.c.smithi143.stderr:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el8/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: 77: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 2) 2023-11-08T19:30:22.845 INFO:tasks.ceph.mds.c.smithi143.stderr: 2023-11-08T19:30:22.845 INFO:tasks.ceph.mds.c.smithi143.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-08T19:30:22.846 INFO:tasks.ceph.mds.c.smithi143.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f2bac3a1f5b] 2023-11-08T19:30:22.846 INFO:tasks.ceph.mds.c.smithi143.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2ba121) [0x7f2bac3a2121] 2023-11-08T19:30:22.846 INFO:tasks.ceph.mds.c.smithi143.stderr: 3: (MDSTableServer::_prepare_logged(boost::intrusive_ptr const&, unsigned long)+0x6ad) [0x564177b35f5d] 2023-11-08T19:30:22.846 INFO:tasks.ceph.mds.c.smithi143.stderr: 4: (MDSContext::complete(int)+0x5f) [0x564177b80ecf] 2023-11-08T19:30:22.847 INFO:tasks.ceph.mds.c.smithi143.stderr: 5: (MDSIOContextBase::complete(int)+0x574) [0x564177b816b4] 2023-11-08T19:30:22.847 INFO:tasks.ceph.mds.c.smithi143.stderr: 6: (MDSLogContextBase::complete(int)+0x5b) [0x564177b81a7b] 2023-11-08T19:30:22.847 INFO:tasks.ceph.mds.c.smithi143.stderr: 7: (Finisher::finisher_thread_entry()+0x19d) [0x7f2bac43f8cd] 2023-11-08T19:30:22.847 INFO:tasks.ceph.mds.c.smithi143.stderr: 8: /lib64/libpthread.so.0(+0x81cf) [0x7f2bab13a1cf] 2023-11-08T19:30:22.847 INFO:tasks.ceph.mds.c.smithi143.stderr: 9: clone() 2023-11-08T19:30:22.848 INFO:tasks.ceph.mds.c.smithi143.stderr: 2023-11-08T19:30:22.848 INFO:tasks.ceph.mds.c.smithi143.stderr: 0> 2023-11-08T19:30:22.824+0000 7f2b9aacd700 -1 *** Caught signal (Aborted) ** 2023-11-08T19:30:22.848 INFO:tasks.ceph.mds.c.smithi143.stderr: in thread 7f2b9aacd700 thread_name:MR_Finisher 2023-11-08T19:30:22.848 INFO:tasks.ceph.mds.c.smithi143.stderr: 2023-11-08T19:30:22.848 INFO:tasks.ceph.mds.c.smithi143.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-08T19:30:22.848 INFO:tasks.ceph.mds.c.smithi143.stderr: 1: /lib64/libpthread.so.0(+0x12ce0) [0x7f2bab144ce0] 2023-11-08T19:30:22.849 INFO:tasks.ceph.mds.c.smithi143.stderr: 2: gsignal() 2023-11-08T19:30:22.849 INFO:tasks.ceph.mds.c.smithi143.stderr: 3: abort() 2023-11-08T19:30:22.849 INFO:tasks.ceph.mds.c.smithi143.stderr: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f2bac3a1fb5] 2023-11-08T19:30:22.849 INFO:tasks.ceph.mds.c.smithi143.stderr: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2ba121) [0x7f2bac3a2121] 2023-11-08T19:30:22.849 INFO:tasks.ceph.mds.c.smithi143.stderr: 6: (MDSTableServer::_prepare_logged(boost::intrusive_ptr const&, unsigned long)+0x6ad) [0x564177b35f5d] 2023-11-08T19:30:22.850 INFO:tasks.ceph.mds.c.smithi143.stderr: 7: (MDSContext::complete(int)+0x5f) [0x564177b80ecf] 2023-11-08T19:30:22.850 INFO:tasks.ceph.mds.c.smithi143.stderr: 8: (MDSIOContextBase::complete(int)+0x574) [0x564177b816b4] 2023-11-08T19:30:22.850 INFO:tasks.ceph.mds.c.smithi143.stderr: 9: (MDSLogContextBase::complete(int)+0x5b) [0x564177b81a7b] 2023-11-08T19:30:22.850 INFO:tasks.ceph.mds.c.smithi143.stderr: 10: (Finisher::finisher_thread_entry()+0x19d) [0x7f2bac43f8cd] 2023-11-08T19:30:22.850 INFO:tasks.ceph.mds.c.smithi143.stderr: 11: /lib64/libpthread.so.0(+0x81cf) [0x7f2bab13a1cf] 2023-11-08T19:30:22.851 INFO:tasks.ceph.mds.c.smithi143.stderr: 12: clone() 2023-11-08T19:30:22.851 INFO:tasks.ceph.mds.c.smithi143.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-11-08T19:30:22.851 INFO:tasks.ceph.mds.c.smithi143.stderr: 2023-11-08T19:30:22.960 INFO:teuthology.orchestra.run.smithi143.stdout:{"mds_beacon_grace":"15.000000"} 2023-11-08T19:30:22.960 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_beacon_grace": "15.000000" } 2023-11-08T19:30:22.961 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:30:23.094 INFO:tasks.ceph.mds.c.smithi143.stderr:daemon-helper: command crashed with signal 6 2023-11-08T19:30:23.122 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:23.121+0000 7f26f9930700 1 Processor -- start 2023-11-08T19:30:23.123 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:23.121+0000 7f26f9930700 1 -- start start 2023-11-08T19:30:23.125 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:23.122+0000 7f26f9930700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f26f415dda0 0x7f26f40b95e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:23.125 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:23.122+0000 7f26f9930700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f26f40d3390 0x7f26f40d1b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:23.126 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:23.122+0000 7f26f9930700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26f40d3810 con 0x7f26f40ce860 2023-11-08T19:30:23.126 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:23.122+0000 7f26f9930700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26f415e180 con 0x7f26f415dda0 2023-11-08T19:30:23.126 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:23.122+0000 7f26f9930700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26f40d22f0 con 0x7f26f40d3390 2023-11-08T19:30:23.126 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:23.122+0000 7f26f2ffd700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f26f415dda0 0x7f26f40b95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:23.126 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:23.122+0000 7f26f2ffd700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f26f415dda0 0x7f26f40b95e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56282/0 (socket says 172.21.15.143:56282) 2023-11-08T19:30:23.127 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:23.122+0000 7f26f2ffd700 1 -- 172.21.15.143:0/2325176590 learned_addr learned my addr 172.21.15.143:0/2325176590 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:30:23.127 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:23.122+0000 7f26f2ffd700 1 -- 172.21.15.143:0/2325176590 >> v2:172.21.15.174:3300/0 conn(0x7f26f40d3390 msgr2=0x7f26f40d1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:23.127 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:23.122+0000 7f26f2ffd700 1 --2- 172.21.15.143:0/2325176590 >> v2:172.21.15.174:3300/0 conn(0x7f26f40d3390 0x7f26f40d1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:23.127 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:23.122+0000 7f26f2ffd700 1 -- 172.21.15.143:0/2325176590 >> v1:172.21.15.174:6789/0 conn(0x7f26f40ce860 legacy=0x7f26f40b5510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:23.128 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:23.122+0000 7f26f2ffd700 1 -- 172.21.15.143:0/2325176590 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26f40d20c0 con 0x7f26f415dda0 2023-11-08T19:30:23.128 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:23.122+0000 7f26f2ffd700 1 --2- 172.21.15.143:0/2325176590 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f26f415dda0 0x7f26f40b95e0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f26f4068160 tx=0x7f26e8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7097e8804819860 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:23.128 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:23.123+0000 7f26f1ffb700 1 -- 172.21.15.143:0/2325176590 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26e8004230 con 0x7f26f415dda0 2023-11-08T19:30:23.128 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:23.123+0000 7f26f1ffb700 1 -- 172.21.15.143:0/2325176590 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26e8004390 con 0x7f26f415dda0 2023-11-08T19:30:23.128 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:23.123+0000 7f26f1ffb700 1 -- 172.21.15.143:0/2325176590 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26e80036a0 con 0x7f26f415dda0 2023-11-08T19:30:23.129 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:23.123+0000 7f26f9930700 1 -- 172.21.15.143:0/2325176590 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f26f415dda0 msgr2=0x7f26f40b95e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:23.129 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:23.123+0000 7f26f9930700 1 --2- 172.21.15.143:0/2325176590 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f26f415dda0 0x7f26f40b95e0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f26f4068160 tx=0x7f26e8009bd0 comp rx=0 tx=0).stop 2023-11-08T19:30:23.129 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:23.123+0000 7f26f9930700 1 -- 172.21.15.143:0/2325176590 shutdown_connections 2023-11-08T19:30:23.129 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:23.123+0000 7f26f9930700 1 --2- 172.21.15.143:0/2325176590 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f26f415dda0 0x7f26f40b95e0 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:23.129 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:23.123+0000 7f26f9930700 1 --2- 172.21.15.143:0/2325176590 >> v2:172.21.15.174:3300/0 conn(0x7f26f40d3390 0x7f26f40d1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:23.130 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:23.123+0000 7f26f9930700 1 -- 172.21.15.143:0/2325176590 >> 172.21.15.143:0/2325176590 conn(0x7f26f406b000 msgr2=0x7f26f406b410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:23.130 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:23.123+0000 7f26f9930700 1 -- 172.21.15.143:0/2325176590 shutdown_connections 2023-11-08T19:30:23.130 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:23.123+0000 7f26f9930700 1 -- 172.21.15.143:0/2325176590 wait complete. 2023-11-08T19:30:23.132 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:23.124+0000 7f26f9930700 1 Processor -- start 2023-11-08T19:30:23.133 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:23.125+0000 7f26f9930700 1 -- start start 2023-11-08T19:30:23.133 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:23.125+0000 7f26f9930700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f26f40ce860 0x7f26f4146b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:23.133 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:23.125+0000 7f26f9930700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f26f40d3390 0x7f26f40cab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:23.133 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:23.125+0000 7f26f9930700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f26f415dda0 0x7f26f4145ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:23.134 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:23.125+0000 7f26f9930700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26f40cb320 con 0x7f26f415dda0 2023-11-08T19:30:23.134 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:23.125+0000 7f26f9930700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26f40cb0a0 con 0x7f26f40d3390 2023-11-08T19:30:23.134 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:23.125+0000 7f26f9930700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26f40b46c0 con 0x7f26f40ce860 2023-11-08T19:30:23.134 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:23.125+0000 7f26f37fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f26f415dda0 0x7f26f4145ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:23.135 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:23.125+0000 7f26f37fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f26f415dda0 0x7f26f4145ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56292/0 (socket says 172.21.15.143:56292) 2023-11-08T19:30:23.135 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:23.126+0000 7f26f37fe700 1 -- 172.21.15.143:0/47407160 learned_addr learned my addr 172.21.15.143:0/47407160 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:30:23.135 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:23.126+0000 7f26f2ffd700 1 --2- 172.21.15.143:0/47407160 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f26f40ce860 0x7f26f4146b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:23.135 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:23.126+0000 7f26f37fe700 1 -- 172.21.15.143:0/47407160 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f26f40ce860 msgr2=0x7f26f4146b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:23.135 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:23.126+0000 7f26f37fe700 1 --2- 172.21.15.143:0/47407160 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f26f40ce860 0x7f26f4146b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:23.136 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:23.126+0000 7f26f27fc700 1 --2- 172.21.15.143:0/47407160 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f26f40d3390 0x7f26f40cab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:23.136 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:23.126+0000 7f26f37fe700 1 -- 172.21.15.143:0/47407160 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f26f40d3390 msgr2=0x7f26f40cab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:23.136 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:23.126+0000 7f26f37fe700 1 --2- 172.21.15.143:0/47407160 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f26f40d3390 0x7f26f40cab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:23.136 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:23.126+0000 7f26f37fe700 1 -- 172.21.15.143:0/47407160 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26e8009550 con 0x7f26f415dda0 2023-11-08T19:30:23.136 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:23.126+0000 7f26f2ffd700 1 --2- 172.21.15.143:0/47407160 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f26f40ce860 0x7f26f4146b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:30:23.137 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:23.126+0000 7f26f37fe700 1 --2- 172.21.15.143:0/47407160 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f26f415dda0 0x7f26f4145ef0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f26ec00b990 tx=0x7f26ec00bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:23.137 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:23.126+0000 7f26dbfff700 1 -- 172.21.15.143:0/47407160 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26ec00c9d0 con 0x7f26f415dda0 2023-11-08T19:30:23.137 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:23.126+0000 7f26dbfff700 1 -- 172.21.15.143:0/47407160 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26ec00cb30 con 0x7f26f415dda0 2023-11-08T19:30:23.137 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:23.126+0000 7f26f9930700 1 -- 172.21.15.143:0/47407160 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26f40b49a0 con 0x7f26f415dda0 2023-11-08T19:30:23.137 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:23.126+0000 7f26dbfff700 1 -- 172.21.15.143:0/47407160 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26ec014440 con 0x7f26f415dda0 2023-11-08T19:30:23.138 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:23.126+0000 7f26f9930700 1 -- 172.21.15.143:0/47407160 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26f40b4f60 con 0x7f26f415dda0 2023-11-08T19:30:23.138 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:23.127+0000 7f26f9930700 1 -- 172.21.15.143:0/47407160 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26f40b9120 con 0x7f26f415dda0 2023-11-08T19:30:23.138 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:23.127+0000 7f26dbfff700 1 -- 172.21.15.143:0/47407160 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f26ec006d50 con 0x7f26f415dda0 2023-11-08T19:30:23.138 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:23.128+0000 7f26dbfff700 1 --2- 172.21.15.143:0/47407160 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f26e006af00 0x7f26e006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:23.139 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:23.128+0000 7f26dbfff700 1 -- 172.21.15.143:0/47407160 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 7217+0+0 (secure 0 0 0) 0x7f26ec089610 con 0x7f26f415dda0 2023-11-08T19:30:23.139 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:23.128+0000 7f26f2ffd700 1 --2- 172.21.15.143:0/47407160 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f26e006af00 0x7f26e006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:23.139 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:23.128+0000 7f26f2ffd700 1 --2- 172.21.15.143:0/47407160 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f26e006af00 0x7f26e006d3c0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f26e8009c10 tx=0x7f26e8008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:23.140 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:23.132+0000 7f26dbfff700 1 -- 172.21.15.143:0/47407160 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f26ec054160 con 0x7f26f415dda0 2023-11-08T19:30:23.362 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:23.361+0000 7f26f9930700 1 -- 172.21.15.143:0/47407160 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f26f404f730 con 0x7f26f415dda0 2023-11-08T19:30:23.363 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:23.361+0000 7f26dbfff700 1 -- 172.21.15.143:0/47407160 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f26ec057780 con 0x7f26f415dda0 2023-11-08T19:30:23.363 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:30:23.363 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":90,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5933,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/3918286710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6835","nonce":3918286710},{"type":"v1","addr":"172.21.15.174:6837","nonce":3918286710}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":6161,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/2129643800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":2129643800},{"type":"v1","addr":"172.21.15.143:6835","nonce":2129643800}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:03.092245+0000","modified":"2023-11-08T19:30:22.070052+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":98,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7494,"mds_1":6918},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6918":{"gid":6918,"name":"b","rank":1,"incarnation":54,"state":"up:active","state_seq":6,"addr":"172.21.15.174:6836/3601088618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3601088618},{"type":"v1","addr":"172.21.15.174:6836","nonce":3601088618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7494":{"gid":7494,"name":"c","rank":0,"incarnation":84,"state":"up:active","state_seq":12,"addr":"172.21.15.143:6834/1535103237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":1535103237},{"type":"v1","addr":"172.21.15.143:6834","nonce":1535103237}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:30:23.365 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:23.363+0000 7f26f9930700 1 -- 172.21.15.143:0/47407160 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f26e006af00 msgr2=0x7f26e006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:23.365 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:23.363+0000 7f26f9930700 1 --2- 172.21.15.143:0/47407160 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f26e006af00 0x7f26e006d3c0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f26e8009c10 tx=0x7f26e8008030 comp rx=0 tx=0).stop 2023-11-08T19:30:23.365 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:23.363+0000 7f26f9930700 1 -- 172.21.15.143:0/47407160 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f26f415dda0 msgr2=0x7f26f4145ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:23.366 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:23.363+0000 7f26f9930700 1 --2- 172.21.15.143:0/47407160 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f26f415dda0 0x7f26f4145ef0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f26ec00b990 tx=0x7f26ec00bd50 comp rx=0 tx=0).stop 2023-11-08T19:30:23.366 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:23.363+0000 7f26f9930700 1 -- 172.21.15.143:0/47407160 shutdown_connections 2023-11-08T19:30:23.366 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:23.363+0000 7f26f9930700 1 --2- 172.21.15.143:0/47407160 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f26e006af00 0x7f26e006d3c0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:23.366 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:23.363+0000 7f26f9930700 1 --2- 172.21.15.143:0/47407160 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f26f40ce860 0x7f26f4146b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:23.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:23.363+0000 7f26f9930700 1 --2- 172.21.15.143:0/47407160 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f26f415dda0 0x7f26f4145ef0 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:23.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:23.363+0000 7f26f9930700 1 --2- 172.21.15.143:0/47407160 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f26f40d3390 0x7f26f40cab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:23.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:23.363+0000 7f26f9930700 1 -- 172.21.15.143:0/47407160 >> 172.21.15.143:0/47407160 conn(0x7f26f406b000 msgr2=0x7f26f40d1440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:23.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:23.364+0000 7f26f9930700 1 -- 172.21.15.143:0/47407160 shutdown_connections 2023-11-08T19:30:23.368 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:23.364+0000 7f26f9930700 1 -- 172.21.15.143:0/47407160 wait complete. 2023-11-08T19:30:23.368 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 90 2023-11-08T19:30:23.377 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2023-11-08T19:30:27.644 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T19:30:27.650 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T19:30:28.379 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:30:28.513 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:28.511+0000 7fdd326ff700 1 Processor -- start 2023-11-08T19:30:28.513 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:28.511+0000 7fdd326ff700 1 -- start start 2023-11-08T19:30:28.516 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:28.512+0000 7fdd326ff700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fdd2c15bb20 0x7fdd2c06d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:28.516 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:28.512+0000 7fdd326ff700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdd2c06f240 0x7fdd2c06abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:28.516 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:28.512+0000 7fdd326ff700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fdd2c06b120 0x7fdd2c0ccb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:28.516 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:28.512+0000 7fdd326ff700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd2c06b5b0 con 0x7fdd2c15bb20 2023-11-08T19:30:28.517 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:28.512+0000 7fdd326ff700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd2c06dc90 con 0x7fdd2c06b120 2023-11-08T19:30:28.517 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:28.512+0000 7fdd326ff700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd2c06f7b0 con 0x7fdd2c06f240 2023-11-08T19:30:28.517 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:28.512+0000 7fdd2bfff700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fdd2c15bb20 0x7fdd2c06d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:28.518 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:28.512+0000 7fdd2bfff700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fdd2c15bb20 0x7fdd2c06d750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56302/0 (socket says 172.21.15.143:56302) 2023-11-08T19:30:28.518 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:28.512+0000 7fdd2bfff700 1 -- 172.21.15.143:0/4212611203 learned_addr learned my addr 172.21.15.143:0/4212611203 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:30:28.518 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:28.512+0000 7fdd2b7fe700 1 --2- 172.21.15.143:0/4212611203 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdd2c06f240 0x7fdd2c06abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:28.518 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:28.512+0000 7fdd30c9c700 1 --2- 172.21.15.143:0/4212611203 >> v2:172.21.15.174:3300/0 conn(0x7fdd2c06b120 0x7fdd2c0ccb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:28.518 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:28.513+0000 7fdd2bfff700 1 -- 172.21.15.143:0/4212611203 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdd2c06f240 msgr2=0x7fdd2c06abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:28.519 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:28.513+0000 7fdd2bfff700 1 --2- 172.21.15.143:0/4212611203 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdd2c06f240 0x7fdd2c06abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:28.519 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:28.513+0000 7fdd2bfff700 1 -- 172.21.15.143:0/4212611203 >> v2:172.21.15.174:3300/0 conn(0x7fdd2c06b120 msgr2=0x7fdd2c0ccb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:28.519 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:28.513+0000 7fdd2bfff700 1 --2- 172.21.15.143:0/4212611203 >> v2:172.21.15.174:3300/0 conn(0x7fdd2c06b120 0x7fdd2c0ccb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:28.519 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:28.513+0000 7fdd2bfff700 1 -- 172.21.15.143:0/4212611203 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd2c06fa90 con 0x7fdd2c15bb20 2023-11-08T19:30:28.520 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:28.513+0000 7fdd30c9c700 1 --2- 172.21.15.143:0/4212611203 >> v2:172.21.15.174:3300/0 conn(0x7fdd2c06b120 0x7fdd2c0ccb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:30:28.520 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:28.513+0000 7fdd2b7fe700 1 --2- 172.21.15.143:0/4212611203 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdd2c06f240 0x7fdd2c06abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:30:28.520 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:28.513+0000 7fdd2bfff700 1 --2- 172.21.15.143:0/4212611203 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd2c15bb20 0x7fdd2c06d750 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c0b6190 tx=0x7fdd1c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b3e0bc9dff98d0d server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:28.520 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:28.513+0000 7fdd2affd700 1 -- 172.21.15.143:0/4212611203 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd1c00b870 con 0x7fdd2c15bb20 2023-11-08T19:30:28.520 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:28.513+0000 7fdd2affd700 1 -- 172.21.15.143:0/4212611203 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd1c00b9d0 con 0x7fdd2c15bb20 2023-11-08T19:30:28.520 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:28.513+0000 7fdd2affd700 1 -- 172.21.15.143:0/4212611203 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd1c00bd60 con 0x7fdd2c15bb20 2023-11-08T19:30:28.521 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:28.513+0000 7fdd326ff700 1 -- 172.21.15.143:0/4212611203 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd2c15bb20 msgr2=0x7fdd2c06d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:28.521 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:28.513+0000 7fdd326ff700 1 --2- 172.21.15.143:0/4212611203 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd2c15bb20 0x7fdd2c06d750 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c0b6190 tx=0x7fdd1c009bd0 comp rx=0 tx=0).stop 2023-11-08T19:30:28.521 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:28.513+0000 7fdd326ff700 1 -- 172.21.15.143:0/4212611203 shutdown_connections 2023-11-08T19:30:28.521 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:28.513+0000 7fdd326ff700 1 --2- 172.21.15.143:0/4212611203 >> v2:172.21.15.174:3300/0 conn(0x7fdd2c06b120 0x7fdd2c0ccb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:28.521 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:28.513+0000 7fdd326ff700 1 --2- 172.21.15.143:0/4212611203 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd2c15bb20 0x7fdd2c06d750 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:28.522 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:28.513+0000 7fdd326ff700 1 --2- 172.21.15.143:0/4212611203 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdd2c06f240 0x7fdd2c06abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:28.522 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:28.513+0000 7fdd326ff700 1 -- 172.21.15.143:0/4212611203 >> 172.21.15.143:0/4212611203 conn(0x7fdd2c0b25b0 msgr2=0x7fdd2c0b2990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:28.522 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:28.514+0000 7fdd326ff700 1 -- 172.21.15.143:0/4212611203 shutdown_connections 2023-11-08T19:30:28.522 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:28.514+0000 7fdd326ff700 1 -- 172.21.15.143:0/4212611203 wait complete. 2023-11-08T19:30:28.522 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:28.515+0000 7fdd326ff700 1 Processor -- start 2023-11-08T19:30:28.523 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:28.515+0000 7fdd326ff700 1 -- start start 2023-11-08T19:30:28.525 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:28.516+0000 7fdd326ff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdd2c06b120 0x7fdd2c152b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:28.526 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:28.516+0000 7fdd326ff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd2c06f240 0x7fdd2c15e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:28.526 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:28.516+0000 7fdd326ff700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdd2c15bb20 0x7fdd2c15ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:28.526 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:28.516+0000 7fdd326ff700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd2c15f0c0 con 0x7fdd2c06f240 2023-11-08T19:30:28.526 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:28.516+0000 7fdd326ff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd2c0c83a0 con 0x7fdd2c06b120 2023-11-08T19:30:28.526 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:28.516+0000 7fdd326ff700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd2c0c7a70 con 0x7fdd2c15bb20 2023-11-08T19:30:28.527 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:28.516+0000 7fdd2b7fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd2c06f240 0x7fdd2c15e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:28.527 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:28.516+0000 7fdd2b7fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd2c06f240 0x7fdd2c15e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56310/0 (socket says 172.21.15.143:56310) 2023-11-08T19:30:28.527 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:28.516+0000 7fdd2b7fe700 1 -- 172.21.15.143:0/2128460575 learned_addr learned my addr 172.21.15.143:0/2128460575 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:30:28.527 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:28.516+0000 7fdd2bfff700 1 --2- 172.21.15.143:0/2128460575 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdd2c06b120 0x7fdd2c152b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:28.527 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:28.516+0000 7fdd30c9c700 1 --2- 172.21.15.143:0/2128460575 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdd2c15bb20 0x7fdd2c15ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:28.528 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:28.516+0000 7fdd2b7fe700 1 -- 172.21.15.143:0/2128460575 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdd2c15bb20 msgr2=0x7fdd2c15ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:28.528 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:28.516+0000 7fdd2b7fe700 1 --2- 172.21.15.143:0/2128460575 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdd2c15bb20 0x7fdd2c15ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:28.528 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:28.516+0000 7fdd2b7fe700 1 -- 172.21.15.143:0/2128460575 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdd2c06b120 msgr2=0x7fdd2c152b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:28.528 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:28.516+0000 7fdd2b7fe700 1 --2- 172.21.15.143:0/2128460575 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdd2c06b120 0x7fdd2c152b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:28.528 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:28.516+0000 7fdd2b7fe700 1 -- 172.21.15.143:0/2128460575 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd1c009550 con 0x7fdd2c06f240 2023-11-08T19:30:28.529 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:28.516+0000 7fdd2bfff700 1 --2- 172.21.15.143:0/2128460575 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdd2c06b120 0x7fdd2c152b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:30:28.529 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:28.516+0000 7fdd2b7fe700 1 --2- 172.21.15.143:0/2128460575 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd2c06f240 0x7fdd2c15e540 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7fdd1400e810 tx=0x7fdd1400ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:28.529 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:28.516+0000 7fdd28ff9700 1 -- 172.21.15.143:0/2128460575 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd14013610 con 0x7fdd2c06f240 2023-11-08T19:30:28.529 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:28.516+0000 7fdd28ff9700 1 -- 172.21.15.143:0/2128460575 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd14013770 con 0x7fdd2c06f240 2023-11-08T19:30:28.530 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:28.516+0000 7fdd28ff9700 1 -- 172.21.15.143:0/2128460575 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd14013b20 con 0x7fdd2c06f240 2023-11-08T19:30:28.530 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:28.516+0000 7fdd326ff700 1 -- 172.21.15.143:0/2128460575 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd2c0c7d50 con 0x7fdd2c06f240 2023-11-08T19:30:28.530 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:28.516+0000 7fdd326ff700 1 -- 172.21.15.143:0/2128460575 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd2c15a030 con 0x7fdd2c06f240 2023-11-08T19:30:28.530 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:28.517+0000 7fdd326ff700 1 -- 172.21.15.143:0/2128460575 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd2c0b2bc0 con 0x7fdd2c06f240 2023-11-08T19:30:28.530 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:28.518+0000 7fdd28ff9700 1 -- 172.21.15.143:0/2128460575 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fdd14013c80 con 0x7fdd2c06f240 2023-11-08T19:30:28.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:28.518+0000 7fdd28ff9700 1 --2- 172.21.15.143:0/2128460575 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fdd0c06af00 0x7fdd0c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:28.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:28.518+0000 7fdd28ff9700 1 -- 172.21.15.143:0/2128460575 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 7173+0+0 (secure 0 0 0) 0x7fdd140897b0 con 0x7fdd2c06f240 2023-11-08T19:30:28.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:28.518+0000 7fdd2bfff700 1 --2- 172.21.15.143:0/2128460575 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fdd0c06af00 0x7fdd0c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:28.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:28.519+0000 7fdd2bfff700 1 --2- 172.21.15.143:0/2128460575 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fdd0c06af00 0x7fdd0c06d3c0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c000c00 tx=0x7fdd1c008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:28.532 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:28.523+0000 7fdd28ff9700 1 -- 172.21.15.143:0/2128460575 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdd14054330 con 0x7fdd2c06f240 2023-11-08T19:30:28.760 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:28.759+0000 7fdd326ff700 1 -- 172.21.15.143:0/2128460575 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdd2c0c8510 con 0x7fdd2c06f240 2023-11-08T19:30:28.760 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:28.759+0000 7fdd28ff9700 1 -- 172.21.15.143:0/2128460575 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fdd14057950 con 0x7fdd2c06f240 2023-11-08T19:30:28.761 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:30:28.761 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":90,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5933,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/3918286710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6835","nonce":3918286710},{"type":"v1","addr":"172.21.15.174:6837","nonce":3918286710}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":6161,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/2129643800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":2129643800},{"type":"v1","addr":"172.21.15.143:6835","nonce":2129643800}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:03.092245+0000","modified":"2023-11-08T19:30:22.070052+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":98,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7494,"mds_1":6918},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6918":{"gid":6918,"name":"b","rank":1,"incarnation":54,"state":"up:active","state_seq":6,"addr":"172.21.15.174:6836/3601088618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3601088618},{"type":"v1","addr":"172.21.15.174:6836","nonce":3601088618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7494":{"gid":7494,"name":"c","rank":0,"incarnation":84,"state":"up:active","state_seq":12,"addr":"172.21.15.143:6834/1535103237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":1535103237},{"type":"v1","addr":"172.21.15.143:6834","nonce":1535103237}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:30:28.763 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:28.761+0000 7fdd326ff700 1 -- 172.21.15.143:0/2128460575 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fdd0c06af00 msgr2=0x7fdd0c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:28.763 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:28.761+0000 7fdd326ff700 1 --2- 172.21.15.143:0/2128460575 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fdd0c06af00 0x7fdd0c06d3c0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c000c00 tx=0x7fdd1c008030 comp rx=0 tx=0).stop 2023-11-08T19:30:28.764 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:28.761+0000 7fdd326ff700 1 -- 172.21.15.143:0/2128460575 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd2c06f240 msgr2=0x7fdd2c15e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:28.764 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:28.761+0000 7fdd326ff700 1 --2- 172.21.15.143:0/2128460575 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd2c06f240 0x7fdd2c15e540 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7fdd1400e810 tx=0x7fdd1400ebd0 comp rx=0 tx=0).stop 2023-11-08T19:30:28.764 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:28.761+0000 7fdd326ff700 1 -- 172.21.15.143:0/2128460575 shutdown_connections 2023-11-08T19:30:28.764 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:28.761+0000 7fdd326ff700 1 --2- 172.21.15.143:0/2128460575 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fdd0c06af00 0x7fdd0c06d3c0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:28.764 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:28.761+0000 7fdd326ff700 1 --2- 172.21.15.143:0/2128460575 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdd2c15bb20 0x7fdd2c15ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:28.765 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:28.761+0000 7fdd326ff700 1 --2- 172.21.15.143:0/2128460575 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdd2c06b120 0x7fdd2c152b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:28.765 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:28.761+0000 7fdd326ff700 1 --2- 172.21.15.143:0/2128460575 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd2c06f240 0x7fdd2c15e540 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:28.765 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:28.761+0000 7fdd326ff700 1 -- 172.21.15.143:0/2128460575 >> 172.21.15.143:0/2128460575 conn(0x7fdd2c0b25b0 msgr2=0x7fdd2c0ad8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:28.765 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:28.762+0000 7fdd326ff700 1 -- 172.21.15.143:0/2128460575 shutdown_connections 2023-11-08T19:30:28.765 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:28.762+0000 7fdd326ff700 1 -- 172.21.15.143:0/2128460575 wait complete. 2023-11-08T19:30:28.766 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 90 2023-11-08T19:30:28.775 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2023-11-08T19:30:30.066 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~0s 2023-11-08T19:30:33.777 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:30:33.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:33.910+0000 7fa550b30700 1 Processor -- start 2023-11-08T19:30:33.912 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:33.910+0000 7fa550b30700 1 -- start start 2023-11-08T19:30:33.918 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:33.911+0000 7fa550b30700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fa54c0cef70 0x7fa54c0b95e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:33.918 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:33.911+0000 7fa550b30700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa54c0d3390 0x7fa54c0d1b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:33.919 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:33.911+0000 7fa550b30700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa54c0d20c0 con 0x7fa54c15dda0 2023-11-08T19:30:33.919 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:33.911+0000 7fa550b30700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa54c0d3800 con 0x7fa54c0d3390 2023-11-08T19:30:33.919 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:33.911+0000 7fa550b30700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa54c15e180 con 0x7fa54c0cef70 2023-11-08T19:30:33.919 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:33.911+0000 7fa54ad9d700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa54c0d3390 0x7fa54c0d1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:33.919 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:33.911+0000 7fa54a59c700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fa54c15dda0 0x7fa54c0ce860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:52474/0 (socket says 172.21.15.143:52474) 2023-11-08T19:30:33.920 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:33.911+0000 7fa54a59c700 1 -- 172.21.15.143:0/1325167889 learned_addr learned my addr 172.21.15.143:0/1325167889 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:30:33.920 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:33.911+0000 7fa549d9b700 1 --2- 172.21.15.143:0/1325167889 >> v2:172.21.15.174:3300/0 conn(0x7fa54c0cef70 0x7fa54c0b95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:33.920 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:33.911+0000 7fa54ad9d700 1 -- 172.21.15.143:0/1325167889 >> v2:172.21.15.174:3300/0 conn(0x7fa54c0cef70 msgr2=0x7fa54c0b95e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:33.920 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:33.911+0000 7fa54ad9d700 1 --2- 172.21.15.143:0/1325167889 >> v2:172.21.15.174:3300/0 conn(0x7fa54c0cef70 0x7fa54c0b95e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:33.920 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:33.911+0000 7fa54ad9d700 1 -- 172.21.15.143:0/1325167889 >> v1:172.21.15.143:6789/0 conn(0x7fa54c15dda0 legacy=0x7fa54c0ce860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:33.921 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:33.911+0000 7fa54ad9d700 1 -- 172.21.15.143:0/1325167889 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa54c0d22f0 con 0x7fa54c0d3390 2023-11-08T19:30:33.921 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:33.911+0000 7fa54959a700 1 -- 172.21.15.143:0/1325167889 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 573720540 0 0) 0x7fa54c0d20c0 con 0x7fa54c15dda0 2023-11-08T19:30:33.921 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:33.911+0000 7fa549d9b700 1 --2- 172.21.15.143:0/1325167889 >> v2:172.21.15.174:3300/0 conn(0x7fa54c0cef70 0x7fa54c0b95e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:30:33.921 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:33.911+0000 7fa54ad9d700 1 --2- 172.21.15.143:0/1325167889 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa54c0d3390 0x7fa54c0d1b80 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7fa54c068160 tx=0x7fa54400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4521b66a3c52be0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:33.921 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:33.911+0000 7fa54959a700 1 -- 172.21.15.143:0/1325167889 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa54400e070 con 0x7fa54c0d3390 2023-11-08T19:30:33.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:33.911+0000 7fa54959a700 1 -- 172.21.15.143:0/1325167889 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa54400bbd0 con 0x7fa54c0d3390 2023-11-08T19:30:33.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:33.911+0000 7fa54959a700 1 -- 172.21.15.143:0/1325167889 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa544012430 con 0x7fa54c0d3390 2023-11-08T19:30:33.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:33.912+0000 7fa550b30700 1 -- 172.21.15.143:0/1325167889 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa54c0d3390 msgr2=0x7fa54c0d1b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:33.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:33.912+0000 7fa550b30700 1 --2- 172.21.15.143:0/1325167889 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa54c0d3390 0x7fa54c0d1b80 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7fa54c068160 tx=0x7fa54400b330 comp rx=0 tx=0).stop 2023-11-08T19:30:33.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:33.912+0000 7fa550b30700 1 -- 172.21.15.143:0/1325167889 shutdown_connections 2023-11-08T19:30:33.923 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:33.912+0000 7fa550b30700 1 --2- 172.21.15.143:0/1325167889 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa54c0d3390 0x7fa54c0d1b80 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:33.923 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:33.912+0000 7fa550b30700 1 --2- 172.21.15.143:0/1325167889 >> v2:172.21.15.174:3300/0 conn(0x7fa54c0cef70 0x7fa54c0b95e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:33.923 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:33.912+0000 7fa550b30700 1 -- 172.21.15.143:0/1325167889 >> 172.21.15.143:0/1325167889 conn(0x7fa54c06b000 msgr2=0x7fa54c06b410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:33.923 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:33.912+0000 7fa550b30700 1 -- 172.21.15.143:0/1325167889 shutdown_connections 2023-11-08T19:30:33.923 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:33.912+0000 7fa550b30700 1 -- 172.21.15.143:0/1325167889 wait complete. 2023-11-08T19:30:33.924 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:33.913+0000 7fa550b30700 1 Processor -- start 2023-11-08T19:30:33.924 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:33.913+0000 7fa550b30700 1 -- start start 2023-11-08T19:30:33.924 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:33.914+0000 7fa550b30700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa54c0cef70 0x7fa54c0b4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:33.924 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:33.914+0000 7fa550b30700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa54c0d3390 0x7fa54c1f2df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:33.924 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:33.914+0000 7fa550b30700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa54c15dda0 0x7fa54c0b80d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:33.924 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:33.914+0000 7fa550b30700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa54c0b6410 con 0x7fa54c0cef70 2023-11-08T19:30:33.925 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:33.914+0000 7fa550b30700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa54c0b5f80 con 0x7fa54c15dda0 2023-11-08T19:30:33.925 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:33.914+0000 7fa550b30700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa54c146680 con 0x7fa54c0d3390 2023-11-08T19:30:33.925 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:33.914+0000 7fa54a59c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa54c0cef70 0x7fa54c0b4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:33.925 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:33.914+0000 7fa54a59c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa54c0cef70 0x7fa54c0b4ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56326/0 (socket says 172.21.15.143:56326) 2023-11-08T19:30:33.925 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:33.914+0000 7fa54a59c700 1 -- 172.21.15.143:0/2521798231 learned_addr learned my addr 172.21.15.143:0/2521798231 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:30:33.926 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:33.914+0000 7fa549d9b700 1 --2- 172.21.15.143:0/2521798231 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa54c0d3390 0x7fa54c1f2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:33.926 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:33.915+0000 7fa54ad9d700 1 --2- 172.21.15.143:0/2521798231 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa54c15dda0 0x7fa54c0b80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:33.926 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:33.915+0000 7fa54a59c700 1 -- 172.21.15.143:0/2521798231 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa54c0d3390 msgr2=0x7fa54c1f2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:33.927 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:33.915+0000 7fa54a59c700 1 --2- 172.21.15.143:0/2521798231 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa54c0d3390 0x7fa54c1f2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:33.927 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:33.915+0000 7fa54a59c700 1 -- 172.21.15.143:0/2521798231 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa54c15dda0 msgr2=0x7fa54c0b80d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:33.927 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:33.915+0000 7fa54a59c700 1 --2- 172.21.15.143:0/2521798231 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa54c15dda0 0x7fa54c0b80d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:33.927 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:33.915+0000 7fa54a59c700 1 -- 172.21.15.143:0/2521798231 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa544009a70 con 0x7fa54c0cef70 2023-11-08T19:30:33.927 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:33.915+0000 7fa549d9b700 1 --2- 172.21.15.143:0/2521798231 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa54c0d3390 0x7fa54c1f2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:30:33.928 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:33.915+0000 7fa54a59c700 1 --2- 172.21.15.143:0/2521798231 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa54c0cef70 0x7fa54c0b4ad0 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7fa54000c580 tx=0x7fa54000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:33.928 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:33.915+0000 7fa53affd700 1 -- 172.21.15.143:0/2521798231 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa540004c80 con 0x7fa54c0cef70 2023-11-08T19:30:33.928 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:33.915+0000 7fa53affd700 1 -- 172.21.15.143:0/2521798231 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa540004e00 con 0x7fa54c0cef70 2023-11-08T19:30:33.928 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:33.915+0000 7fa550b30700 1 -- 172.21.15.143:0/2521798231 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa54c0b5370 con 0x7fa54c0cef70 2023-11-08T19:30:33.928 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:33.915+0000 7fa53affd700 1 -- 172.21.15.143:0/2521798231 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa540007310 con 0x7fa54c0cef70 2023-11-08T19:30:33.928 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:33.915+0000 7fa550b30700 1 -- 172.21.15.143:0/2521798231 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa54c0b5930 con 0x7fa54c0cef70 2023-11-08T19:30:33.929 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:33.916+0000 7fa550b30700 1 -- 172.21.15.143:0/2521798231 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa54c0b6260 con 0x7fa54c0cef70 2023-11-08T19:30:33.929 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:33.917+0000 7fa53affd700 1 -- 172.21.15.143:0/2521798231 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa540007470 con 0x7fa54c0cef70 2023-11-08T19:30:33.929 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:33.917+0000 7fa53affd700 1 --2- 172.21.15.143:0/2521798231 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa52c06af00 0x7fa52c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:33.929 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:33.917+0000 7fa53affd700 1 -- 172.21.15.143:0/2521798231 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 7173+0+0 (secure 0 0 0) 0x7fa5400898e0 con 0x7fa54c0cef70 2023-11-08T19:30:33.929 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:33.917+0000 7fa549d9b700 1 --2- 172.21.15.143:0/2521798231 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa52c06af00 0x7fa52c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:33.930 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:33.918+0000 7fa549d9b700 1 --2- 172.21.15.143:0/2521798231 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa52c06af00 0x7fa52c06d3c0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fa54c1f6ad0 tx=0x7fa53c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:33.930 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:33.921+0000 7fa53affd700 1 -- 172.21.15.143:0/2521798231 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa5400543d0 con 0x7fa54c0cef70 2023-11-08T19:30:34.152 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:34.151+0000 7fa550b30700 1 -- 172.21.15.143:0/2521798231 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa54c04f730 con 0x7fa54c0cef70 2023-11-08T19:30:34.152 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:34.151+0000 7fa53affd700 1 -- 172.21.15.143:0/2521798231 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fa5400579f0 con 0x7fa54c0cef70 2023-11-08T19:30:34.153 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:30:34.153 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":90,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5933,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/3918286710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6835","nonce":3918286710},{"type":"v1","addr":"172.21.15.174:6837","nonce":3918286710}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":6161,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/2129643800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":2129643800},{"type":"v1","addr":"172.21.15.143:6835","nonce":2129643800}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:03.092245+0000","modified":"2023-11-08T19:30:22.070052+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":98,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7494,"mds_1":6918},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6918":{"gid":6918,"name":"b","rank":1,"incarnation":54,"state":"up:active","state_seq":6,"addr":"172.21.15.174:6836/3601088618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3601088618},{"type":"v1","addr":"172.21.15.174:6836","nonce":3601088618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7494":{"gid":7494,"name":"c","rank":0,"incarnation":84,"state":"up:active","state_seq":12,"addr":"172.21.15.143:6834/1535103237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":1535103237},{"type":"v1","addr":"172.21.15.143:6834","nonce":1535103237}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:30:34.155 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:34.153+0000 7fa550b30700 1 -- 172.21.15.143:0/2521798231 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa52c06af00 msgr2=0x7fa52c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:34.155 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:34.153+0000 7fa550b30700 1 --2- 172.21.15.143:0/2521798231 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa52c06af00 0x7fa52c06d3c0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fa54c1f6ad0 tx=0x7fa53c008040 comp rx=0 tx=0).stop 2023-11-08T19:30:34.155 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:34.153+0000 7fa550b30700 1 -- 172.21.15.143:0/2521798231 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa54c0cef70 msgr2=0x7fa54c0b4ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:34.155 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:34.153+0000 7fa550b30700 1 --2- 172.21.15.143:0/2521798231 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa54c0cef70 0x7fa54c0b4ad0 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7fa54000c580 tx=0x7fa54000c940 comp rx=0 tx=0).stop 2023-11-08T19:30:34.156 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:34.153+0000 7fa550b30700 1 -- 172.21.15.143:0/2521798231 shutdown_connections 2023-11-08T19:30:34.156 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:34.153+0000 7fa550b30700 1 --2- 172.21.15.143:0/2521798231 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa52c06af00 0x7fa52c06d3c0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:34.156 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:34.153+0000 7fa550b30700 1 --2- 172.21.15.143:0/2521798231 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa54c15dda0 0x7fa54c0b80d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:34.156 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:34.153+0000 7fa550b30700 1 --2- 172.21.15.143:0/2521798231 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa54c0cef70 0x7fa54c0b4ad0 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:34.156 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:34.153+0000 7fa550b30700 1 --2- 172.21.15.143:0/2521798231 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa54c0d3390 0x7fa54c1f2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:34.157 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:34.153+0000 7fa550b30700 1 -- 172.21.15.143:0/2521798231 >> 172.21.15.143:0/2521798231 conn(0x7fa54c06b000 msgr2=0x7fa54c15cf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:34.157 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:34.153+0000 7fa550b30700 1 -- 172.21.15.143:0/2521798231 shutdown_connections 2023-11-08T19:30:34.157 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:34.153+0000 7fa550b30700 1 -- 172.21.15.143:0/2521798231 wait complete. 2023-11-08T19:30:34.157 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 90 2023-11-08T19:30:34.167 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2023-11-08T19:30:36.678 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~7s 2023-11-08T19:30:39.168 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:30:39.301 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.300+0000 7fa2586d8700 1 Processor -- start 2023-11-08T19:30:39.301 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.300+0000 7fa2586d8700 1 -- start start 2023-11-08T19:30:39.305 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.301+0000 7fa2586d8700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fa25006ce10 0x7fa2500ceb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:39.305 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.301+0000 7fa2586d8700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa2500b2f30 0x7fa25015d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:39.306 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.301+0000 7fa2586d8700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2500d0000 con 0x7fa2500b2b50 2023-11-08T19:30:39.306 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.301+0000 7fa2586d8700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2500d02e0 con 0x7fa25006ce10 2023-11-08T19:30:39.306 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.301+0000 7fa2586d8700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2500d05c0 con 0x7fa2500b2f30 2023-11-08T19:30:39.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.301+0000 7fa256474700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fa2500b2b50 0x7fa25006c700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:52490/0 (socket says 172.21.15.143:52490) 2023-11-08T19:30:39.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.301+0000 7fa256474700 1 -- 172.21.15.143:0/358812959 learned_addr learned my addr 172.21.15.143:0/358812959 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:30:39.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.301+0000 7fa255c73700 1 --2- 172.21.15.143:0/358812959 >> v2:172.21.15.174:3300/0 conn(0x7fa25006ce10 0x7fa2500ceb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:39.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.301+0000 7fa256c75700 1 --2- 172.21.15.143:0/358812959 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa2500b2f30 0x7fa25015d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:39.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.301+0000 7fa255472700 1 -- 172.21.15.143:0/358812959 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1660480984 0 0) 0x7fa2500d0000 con 0x7fa2500b2b50 2023-11-08T19:30:39.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.301+0000 7fa255472700 1 -- 172.21.15.143:0/358812959 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa23c0035b0 con 0x7fa2500b2b50 2023-11-08T19:30:39.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.301+0000 7fa255472700 1 -- 172.21.15.143:0/358812959 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1269999262 0 0) 0x7fa23c0035b0 con 0x7fa2500b2b50 2023-11-08T19:30:39.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.301+0000 7fa255472700 1 -- 172.21.15.143:0/358812959 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa2500b2f30 msgr2=0x7fa25015d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:39.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.301+0000 7fa255472700 1 --2- 172.21.15.143:0/358812959 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa2500b2f30 0x7fa25015d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:39.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.301+0000 7fa255472700 1 -- 172.21.15.143:0/358812959 >> v2:172.21.15.174:3300/0 conn(0x7fa25006ce10 msgr2=0x7fa2500ceb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:39.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.301+0000 7fa255472700 1 --2- 172.21.15.143:0/358812959 >> v2:172.21.15.174:3300/0 conn(0x7fa25006ce10 0x7fa2500ceb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:39.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.301+0000 7fa255472700 1 -- 172.21.15.143:0/358812959 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2500d08a0 con 0x7fa2500b2b50 2023-11-08T19:30:39.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.301+0000 7fa255472700 1 -- 172.21.15.143:0/358812959 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4147756303 0 0) 0x7fa24c0017f0 con 0x7fa2500b2b50 2023-11-08T19:30:39.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.301+0000 7fa255472700 1 -- 172.21.15.143:0/358812959 >> v1:172.21.15.143:6789/0 conn(0x7fa2500b2b50 legacy=0x7fa25006c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:39.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.301+0000 7fa255472700 1 --2- 172.21.15.143:0/358812959 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa23c003a00 0x7fa23c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:39.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.301+0000 7fa255472700 1 --2- 172.21.15.143:0/358812959 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa23c004850 0x7fa23c004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:39.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.301+0000 7fa255472700 1 --2- 172.21.15.143:0/358812959 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa23c005210 0x7fa23c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:39.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.301+0000 7fa255472700 1 -- 172.21.15.143:0/358812959 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2500d02e0 con 0x7fa23c003a00 2023-11-08T19:30:39.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.301+0000 7fa255472700 1 -- 172.21.15.143:0/358812959 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2500d05c0 con 0x7fa23c005210 2023-11-08T19:30:39.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.301+0000 7fa255472700 1 -- 172.21.15.143:0/358812959 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa23c00bf50 con 0x7fa23c004850 2023-11-08T19:30:39.315 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.301+0000 7fa256c75700 1 --2- 172.21.15.143:0/358812959 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa2500b2f30 0x7fa25015d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T19:30:39.315 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.301+0000 7fa256474700 1 --2- 172.21.15.143:0/358812959 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa23c003a00 0x7fa23c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:39.315 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.302+0000 7fa255c73700 1 --2- 172.21.15.143:0/358812959 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa23c004850 0x7fa23c004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:39.315 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.302+0000 7fa256c75700 1 --2- 172.21.15.143:0/358812959 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa23c005210 0x7fa23c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:39.316 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.302+0000 7fa256474700 1 -- 172.21.15.143:0/358812959 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa23c004850 msgr2=0x7fa23c004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:39.316 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.302+0000 7fa256474700 1 --2- 172.21.15.143:0/358812959 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa23c004850 0x7fa23c004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:39.316 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.302+0000 7fa256474700 1 -- 172.21.15.143:0/358812959 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa23c005210 msgr2=0x7fa23c00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:39.316 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.302+0000 7fa256474700 1 --2- 172.21.15.143:0/358812959 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa23c005210 0x7fa23c00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:39.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.302+0000 7fa256474700 1 -- 172.21.15.143:0/358812959 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa23c0035b0 con 0x7fa23c003a00 2023-11-08T19:30:39.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.302+0000 7fa256474700 1 --2- 172.21.15.143:0/358812959 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa23c003a00 0x7fa23c003e20 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7fa24c00c320 tx=0x7fa23c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6bb90cc0d53abb8 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:39.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.302+0000 7fa255472700 1 -- 172.21.15.143:0/358812959 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa24c00ce60 con 0x7fa23c003a00 2023-11-08T19:30:39.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.302+0000 7fa255472700 1 -- 172.21.15.143:0/358812959 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa24c007bf0 con 0x7fa23c003a00 2023-11-08T19:30:39.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.302+0000 7fa255472700 1 -- 172.21.15.143:0/358812959 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa24c0072f0 con 0x7fa23c003a00 2023-11-08T19:30:39.318 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.303+0000 7fa2586d8700 1 -- 172.21.15.143:0/358812959 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa23c003a00 msgr2=0x7fa23c003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:39.318 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.303+0000 7fa2586d8700 1 --2- 172.21.15.143:0/358812959 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa23c003a00 0x7fa23c003e20 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7fa24c00c320 tx=0x7fa23c005880 comp rx=0 tx=0).stop 2023-11-08T19:30:39.318 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.303+0000 7fa256c75700 1 -- 172.21.15.143:0/358812959 reap_dead start 2023-11-08T19:30:39.318 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.303+0000 7fa2586d8700 1 -- 172.21.15.143:0/358812959 shutdown_connections 2023-11-08T19:30:39.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.303+0000 7fa2586d8700 1 -- 172.21.15.143:0/358812959 >> 172.21.15.143:0/358812959 conn(0x7fa25009aa70 msgr2=0x7fa2500d18f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:39.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.303+0000 7fa2586d8700 1 -- 172.21.15.143:0/358812959 shutdown_connections 2023-11-08T19:30:39.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.303+0000 7fa2586d8700 1 -- 172.21.15.143:0/358812959 wait complete. 2023-11-08T19:30:39.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.304+0000 7fa2586d8700 1 Processor -- start 2023-11-08T19:30:39.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.304+0000 7fa2586d8700 1 -- start start 2023-11-08T19:30:39.323 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.305+0000 7fa2586d8700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa23c003a00 0x7fa25015c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:39.323 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.305+0000 7fa2586d8700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa2501613b0 0x7fa2501478a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:39.323 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.305+0000 7fa2586d8700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa25015a6b0 0x7fa25015cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:39.324 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.305+0000 7fa2586d8700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2501596a0 con 0x7fa2501613b0 2023-11-08T19:30:39.324 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.305+0000 7fa2586d8700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa25015a220 con 0x7fa25015a6b0 2023-11-08T19:30:39.324 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.305+0000 7fa2586d8700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa250147de0 con 0x7fa23c003a00 2023-11-08T19:30:39.324 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.305+0000 7fa255c73700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa2501613b0 0x7fa2501478a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:39.324 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.305+0000 7fa255c73700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa2501613b0 0x7fa2501478a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56350/0 (socket says 172.21.15.143:56350) 2023-11-08T19:30:39.325 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.305+0000 7fa255c73700 1 -- 172.21.15.143:0/1438983831 learned_addr learned my addr 172.21.15.143:0/1438983831 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:30:39.325 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.305+0000 7fa256474700 1 --2- 172.21.15.143:0/1438983831 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa23c003a00 0x7fa25015c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:39.325 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.305+0000 7fa256c75700 1 --2- 172.21.15.143:0/1438983831 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa25015a6b0 0x7fa25015cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:39.325 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.305+0000 7fa255c73700 1 -- 172.21.15.143:0/1438983831 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa23c003a00 msgr2=0x7fa25015c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:39.326 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.305+0000 7fa255c73700 1 --2- 172.21.15.143:0/1438983831 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa23c003a00 0x7fa25015c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:39.326 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.305+0000 7fa255c73700 1 -- 172.21.15.143:0/1438983831 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa25015a6b0 msgr2=0x7fa25015cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:39.326 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.305+0000 7fa255c73700 1 --2- 172.21.15.143:0/1438983831 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa25015a6b0 0x7fa25015cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:39.326 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.305+0000 7fa255c73700 1 -- 172.21.15.143:0/1438983831 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2501598f0 con 0x7fa2501613b0 2023-11-08T19:30:39.326 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.305+0000 7fa255c73700 1 --2- 172.21.15.143:0/1438983831 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa2501613b0 0x7fa2501478a0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7fa2400119a0 tx=0x7fa240011cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:39.327 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.305+0000 7fa246ffd700 1 -- 172.21.15.143:0/1438983831 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa24000f780 con 0x7fa2501613b0 2023-11-08T19:30:39.327 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.305+0000 7fa246ffd700 1 -- 172.21.15.143:0/1438983831 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa24000f8e0 con 0x7fa2501613b0 2023-11-08T19:30:39.327 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.305+0000 7fa2586d8700 1 -- 172.21.15.143:0/1438983831 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa250159bd0 con 0x7fa2501613b0 2023-11-08T19:30:39.327 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.305+0000 7fa246ffd700 1 -- 172.21.15.143:0/1438983831 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa24000fc70 con 0x7fa2501613b0 2023-11-08T19:30:39.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.305+0000 7fa2586d8700 1 -- 172.21.15.143:0/1438983831 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2501481f0 con 0x7fa2501613b0 2023-11-08T19:30:39.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.306+0000 7fa2586d8700 1 -- 172.21.15.143:0/1438983831 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa25015a3d0 con 0x7fa2501613b0 2023-11-08T19:30:39.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.307+0000 7fa246ffd700 1 -- 172.21.15.143:0/1438983831 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa24001d400 con 0x7fa2501613b0 2023-11-08T19:30:39.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.307+0000 7fa246ffd700 1 --2- 172.21.15.143:0/1438983831 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa23406af00 0x7fa23406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:39.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.307+0000 7fa246ffd700 1 -- 172.21.15.143:0/1438983831 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 7173+0+0 (secure 0 0 0) 0x7fa24008a070 con 0x7fa2501613b0 2023-11-08T19:30:39.329 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.307+0000 7fa256474700 1 --2- 172.21.15.143:0/1438983831 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa23406af00 0x7fa23406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:39.329 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.308+0000 7fa256474700 1 --2- 172.21.15.143:0/1438983831 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa23406af00 0x7fa23406d3c0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fa24c000ef0 tx=0x7fa24c00a000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:39.329 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.312+0000 7fa246ffd700 1 -- 172.21.15.143:0/1438983831 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa25015a3d0 con 0x7fa2501613b0 2023-11-08T19:30:39.547 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.545+0000 7fa2586d8700 1 -- 172.21.15.143:0/1438983831 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa25015a390 con 0x7fa2501613b0 2023-11-08T19:30:39.547 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.545+0000 7fa246ffd700 1 -- 172.21.15.143:0/1438983831 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+4991 (secure 0 0 0) 0x7fa240054b60 con 0x7fa2501613b0 2023-11-08T19:30:39.548 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:30:39.548 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":91,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5933,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/3918286710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6835","nonce":3918286710},{"type":"v1","addr":"172.21.15.174:6837","nonce":3918286710}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":6161,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/2129643800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":2129643800},{"type":"v1","addr":"172.21.15.143:6835","nonce":2129643800}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:03.092245+0000","modified":"2023-11-08T19:30:37.621320+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":98,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7494,"mds_1":6918},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6918":{"gid":6918,"name":"b","rank":1,"incarnation":54,"state":"up:active","state_seq":6,"addr":"172.21.15.174:6836/3601088618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3601088618},{"type":"v1","addr":"172.21.15.174:6836","nonce":3601088618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7494":{"gid":7494,"name":"c","rank":0,"incarnation":84,"state":"up:active","state_seq":12,"addr":"172.21.15.143:6834/1535103237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":1535103237},{"type":"v1","addr":"172.21.15.143:6834","nonce":1535103237}]},"join_fscid":-1,"laggy_since":"2023-11-08T19:30:37.621320+0000","export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:30:39.550 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.547+0000 7fa2586d8700 1 -- 172.21.15.143:0/1438983831 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa23406af00 msgr2=0x7fa23406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:39.550 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.547+0000 7fa2586d8700 1 --2- 172.21.15.143:0/1438983831 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa23406af00 0x7fa23406d3c0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fa24c000ef0 tx=0x7fa24c00a000 comp rx=0 tx=0).stop 2023-11-08T19:30:39.550 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.547+0000 7fa2586d8700 1 -- 172.21.15.143:0/1438983831 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa2501613b0 msgr2=0x7fa2501478a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:39.550 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.547+0000 7fa2586d8700 1 --2- 172.21.15.143:0/1438983831 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa2501613b0 0x7fa2501478a0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7fa2400119a0 tx=0x7fa240011cb0 comp rx=0 tx=0).stop 2023-11-08T19:30:39.551 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.547+0000 7fa2586d8700 1 -- 172.21.15.143:0/1438983831 shutdown_connections 2023-11-08T19:30:39.551 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.547+0000 7fa2586d8700 1 --2- 172.21.15.143:0/1438983831 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa23406af00 0x7fa23406d3c0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:39.551 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.547+0000 7fa2586d8700 1 --2- 172.21.15.143:0/1438983831 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa23c003a00 0x7fa25015c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:39.551 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.547+0000 7fa2586d8700 1 --2- 172.21.15.143:0/1438983831 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa25015a6b0 0x7fa25015cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:39.552 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.547+0000 7fa2586d8700 1 --2- 172.21.15.143:0/1438983831 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa2501613b0 0x7fa2501478a0 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:39.552 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.547+0000 7fa2586d8700 1 -- 172.21.15.143:0/1438983831 >> 172.21.15.143:0/1438983831 conn(0x7fa25009aa70 msgr2=0x7fa2500b33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:39.552 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.547+0000 7fa2586d8700 1 -- 172.21.15.143:0/1438983831 shutdown_connections 2023-11-08T19:30:39.552 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.547+0000 7fa2586d8700 1 -- 172.21.15.143:0/1438983831 wait complete. 2023-11-08T19:30:39.552 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 91 2023-11-08T19:30:39.561 DEBUG:tasks.ceph_test_case:wait_until_true: success in 15s and 0 retries 2023-11-08T19:30:39.562 DEBUG:teuthology.orchestra.run.smithi143:> sudo sysctl -n kernel.core_pattern 2023-11-08T19:30:39.588 INFO:teuthology.orchestra.run.smithi143.stdout:/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-08T19:30:39.590 INFO:tasks.cephfs.cephfs_test_case:Clearing core from directory: /home/ubuntu/cephtest/archive/coredump 2023-11-08T19:30:39.590 DEBUG:teuthology.orchestra.run.smithi143:> cd /home/ubuntu/cephtest/archive/coredump && sudo ls | sudo xargs file 2023-11-08T19:30:39.659 INFO:teuthology.orchestra.run.smithi143.stdout:1699471822.135557.core: ELF 64-bit LSB core file, x86-64, version 1 (SYSV), SVR4-style, from 'ceph-mds -f --cluster ceph -i c', real uid: 0, effective uid: 0, real gid: 0, effective gid: 0, execfn: '/bin/ceph-mds', platform: 'x86_64' 2023-11-08T19:30:39.661 INFO:tasks.cephfs.cephfs_test_case:Enumerated cores: ['1699471822.135557.core'] 2023-11-08T19:30:39.661 INFO:tasks.cephfs.cephfs_test_case:Found core file 1699471822.135557.core, deleting it 2023-11-08T19:30:39.662 DEBUG:teuthology.orchestra.run.smithi143:> cd /home/ubuntu/cephtest/archive/coredump && sudo rm -f 1699471822.135557.core 2023-11-08T19:30:39.730 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:30:39.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.891+0000 7f3b17c11700 1 Processor -- start 2023-11-08T19:30:39.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.891+0000 7f3b17c11700 1 -- start start 2023-11-08T19:30:39.897 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.892+0000 7f3b17c11700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3b100b2f30 0x7f3b100ceb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:39.897 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.892+0000 7f3b17c11700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b100d0000 con 0x7f3b100b2b50 2023-11-08T19:30:39.897 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.892+0000 7f3b17c11700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b100d02e0 con 0x7f3b1006ce10 2023-11-08T19:30:39.897 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.892+0000 7f3b17c11700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b100d05c0 con 0x7f3b100b2f30 2023-11-08T19:30:39.897 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.892+0000 7f3b159ad700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f3b100b2b50 0x7f3b1006c700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:52510/0 (socket says 172.21.15.143:52510) 2023-11-08T19:30:39.898 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.892+0000 7f3b159ad700 1 -- 172.21.15.143:0/2517383245 learned_addr learned my addr 172.21.15.143:0/2517383245 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:30:39.898 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.892+0000 7f3b161ae700 1 --2- 172.21.15.143:0/2517383245 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3b100b2f30 0x7f3b100ceb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:39.898 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.892+0000 7f3b149ab700 1 -- 172.21.15.143:0/2517383245 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3226278121 0 0) 0x7f3b100d0000 con 0x7f3b100b2b50 2023-11-08T19:30:39.898 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.892+0000 7f3b149ab700 1 -- 172.21.15.143:0/2517383245 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3afc0035b0 con 0x7f3b100b2b50 2023-11-08T19:30:39.898 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.893+0000 7f3b149ab700 1 -- 172.21.15.143:0/2517383245 <== mon.1 v1:172.21.15.174:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1319392451 0 0) 0x7f3b100d02e0 con 0x7f3b1006ce10 2023-11-08T19:30:39.899 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.893+0000 7f3b149ab700 1 -- 172.21.15.143:0/2517383245 --> v1:172.21.15.174:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3b100d0000 con 0x7f3b1006ce10 2023-11-08T19:30:39.899 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.893+0000 7f3b149ab700 1 -- 172.21.15.143:0/2517383245 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3992303826 0 0) 0x7f3afc0035b0 con 0x7f3b100b2b50 2023-11-08T19:30:39.899 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.893+0000 7f3b149ab700 1 -- 172.21.15.143:0/2517383245 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3b100b2f30 msgr2=0x7f3b100ceb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:39.899 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.893+0000 7f3b149ab700 1 --2- 172.21.15.143:0/2517383245 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3b100b2f30 0x7f3b100ceb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:39.899 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.893+0000 7f3b149ab700 1 -- 172.21.15.143:0/2517383245 >> v1:172.21.15.174:6789/0 conn(0x7f3b1006ce10 legacy=0x7f3b1015d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:39.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.893+0000 7f3b149ab700 1 -- 172.21.15.143:0/2517383245 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b100d08a0 con 0x7f3b100b2b50 2023-11-08T19:30:39.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.893+0000 7f3b149ab700 1 -- 172.21.15.143:0/2517383245 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4147756303 0 0) 0x7f3b000017f0 con 0x7f3b100b2b50 2023-11-08T19:30:39.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.893+0000 7f3b149ab700 1 -- 172.21.15.143:0/2517383245 >> v1:172.21.15.143:6789/0 conn(0x7f3b100b2b50 legacy=0x7f3b1006c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:39.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.893+0000 7f3b149ab700 1 --2- 172.21.15.143:0/2517383245 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3afc003bc0 0x7f3afc003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:39.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.893+0000 7f3b149ab700 1 --2- 172.21.15.143:0/2517383245 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3afc004850 0x7f3afc004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:39.901 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.893+0000 7f3b149ab700 1 --2- 172.21.15.143:0/2517383245 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3afc005210 0x7f3afc00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:39.901 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.893+0000 7f3b149ab700 1 -- 172.21.15.143:0/2517383245 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b100d05c0 con 0x7f3afc005210 2023-11-08T19:30:39.901 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.893+0000 7f3b149ab700 1 -- 172.21.15.143:0/2517383245 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3afc00bf80 con 0x7f3afc003bc0 2023-11-08T19:30:39.901 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.893+0000 7f3b149ab700 1 -- 172.21.15.143:0/2517383245 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3afc00c140 con 0x7f3afc004850 2023-11-08T19:30:39.901 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.893+0000 7f3b161ae700 1 --2- 172.21.15.143:0/2517383245 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3b100b2f30 0x7f3b100ceb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T19:30:39.902 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.893+0000 7f3b161ae700 1 --2- 172.21.15.143:0/2517383245 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3afc005210 0x7f3afc00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:39.902 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.893+0000 7f3b159ad700 1 --2- 172.21.15.143:0/2517383245 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3afc003bc0 0x7f3afc003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:39.902 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.893+0000 7f3b151ac700 1 --2- 172.21.15.143:0/2517383245 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3afc004850 0x7f3afc004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:39.902 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.893+0000 7f3b161ae700 1 -- 172.21.15.143:0/2517383245 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3afc004850 msgr2=0x7f3afc004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:39.902 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.893+0000 7f3b161ae700 1 --2- 172.21.15.143:0/2517383245 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3afc004850 0x7f3afc004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:39.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.893+0000 7f3b161ae700 1 -- 172.21.15.143:0/2517383245 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3afc003bc0 msgr2=0x7f3afc003fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:39.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.893+0000 7f3b161ae700 1 --2- 172.21.15.143:0/2517383245 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3afc003bc0 0x7f3afc003fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:39.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.893+0000 7f3b161ae700 1 -- 172.21.15.143:0/2517383245 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3afc0035b0 con 0x7f3afc005210 2023-11-08T19:30:39.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.893+0000 7f3b161ae700 1 --2- 172.21.15.143:0/2517383245 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3afc005210 0x7f3afc00ba10 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f3b100b3310 tx=0x7f3afc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6dd44b376206ebd server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:39.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.893+0000 7f3b149ab700 1 -- 172.21.15.143:0/2517383245 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b08003c20 con 0x7f3afc005210 2023-11-08T19:30:39.904 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.893+0000 7f3b149ab700 1 -- 172.21.15.143:0/2517383245 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b0800f040 con 0x7f3afc005210 2023-11-08T19:30:39.904 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.894+0000 7f3b149ab700 1 -- 172.21.15.143:0/2517383245 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b0800ea60 con 0x7f3afc005210 2023-11-08T19:30:39.904 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.894+0000 7f3b17c11700 1 -- 172.21.15.143:0/2517383245 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3afc005210 msgr2=0x7f3afc00ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:39.904 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.894+0000 7f3b17c11700 1 --2- 172.21.15.143:0/2517383245 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3afc005210 0x7f3afc00ba10 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f3b100b3310 tx=0x7f3afc005880 comp rx=0 tx=0).stop 2023-11-08T19:30:39.904 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.894+0000 7f3b161ae700 1 -- 172.21.15.143:0/2517383245 reap_dead start 2023-11-08T19:30:39.905 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.894+0000 7f3b17c11700 1 -- 172.21.15.143:0/2517383245 shutdown_connections 2023-11-08T19:30:39.905 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.894+0000 7f3b17c11700 1 -- 172.21.15.143:0/2517383245 >> 172.21.15.143:0/2517383245 conn(0x7f3b1009aa70 msgr2=0x7f3b100d18f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:39.905 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.894+0000 7f3b17c11700 1 -- 172.21.15.143:0/2517383245 shutdown_connections 2023-11-08T19:30:39.905 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.894+0000 7f3b17c11700 1 -- 172.21.15.143:0/2517383245 wait complete. 2023-11-08T19:30:39.905 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.895+0000 7f3b17c11700 1 Processor -- start 2023-11-08T19:30:39.905 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.895+0000 7f3b17c11700 1 -- start start 2023-11-08T19:30:39.908 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.896+0000 7f3b17c11700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3b101f3750 0x7f3b101deba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:39.908 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.896+0000 7f3b17c11700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3b101df0e0 0x7f3b101dd060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:39.909 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.896+0000 7f3b17c11700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3b101dd5a0 0x7f3b101e4440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:39.909 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.896+0000 7f3b17c11700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b100c68e0 con 0x7f3b101df0e0 2023-11-08T19:30:39.909 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.896+0000 7f3b17c11700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b101d63d0 con 0x7f3b101dd5a0 2023-11-08T19:30:39.909 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.896+0000 7f3b17c11700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b101de710 con 0x7f3b101f3750 2023-11-08T19:30:39.909 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.896+0000 7f3b151ac700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3b101df0e0 0x7f3b101dd060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:39.910 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.896+0000 7f3b151ac700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3b101df0e0 0x7f3b101dd060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56370/0 (socket says 172.21.15.143:56370) 2023-11-08T19:30:39.910 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.896+0000 7f3b151ac700 1 -- 172.21.15.143:0/843899525 learned_addr learned my addr 172.21.15.143:0/843899525 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:30:39.910 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.896+0000 7f3b159ad700 1 --2- 172.21.15.143:0/843899525 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3b101f3750 0x7f3b101deba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:39.910 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.896+0000 7f3b161ae700 1 --2- 172.21.15.143:0/843899525 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3b101dd5a0 0x7f3b101e4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:39.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.896+0000 7f3b151ac700 1 -- 172.21.15.143:0/843899525 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3b101f3750 msgr2=0x7f3b101deba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:39.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.896+0000 7f3b151ac700 1 --2- 172.21.15.143:0/843899525 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3b101f3750 0x7f3b101deba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:39.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.896+0000 7f3b151ac700 1 -- 172.21.15.143:0/843899525 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3b101dd5a0 msgr2=0x7f3b101e4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:39.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.896+0000 7f3b151ac700 1 --2- 172.21.15.143:0/843899525 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3b101dd5a0 0x7f3b101e4440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:39.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.896+0000 7f3b151ac700 1 -- 172.21.15.143:0/843899525 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b101ddb00 con 0x7f3b101df0e0 2023-11-08T19:30:39.912 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.896+0000 7f3b161ae700 1 --2- 172.21.15.143:0/843899525 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3b101dd5a0 0x7f3b101e4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:30:39.912 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.897+0000 7f3b151ac700 1 --2- 172.21.15.143:0/843899525 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3b101df0e0 0x7f3b101dd060 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f3b0c00f7f0 tx=0x7f3b0c00fb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:39.912 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.897+0000 7f3b067fc700 1 -- 172.21.15.143:0/843899525 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b0c0052e0 con 0x7f3b101df0e0 2023-11-08T19:30:39.912 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.897+0000 7f3b067fc700 1 -- 172.21.15.143:0/843899525 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b0c005440 con 0x7f3b101df0e0 2023-11-08T19:30:39.912 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.897+0000 7f3b17c11700 1 -- 172.21.15.143:0/843899525 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b101ddde0 con 0x7f3b101df0e0 2023-11-08T19:30:39.913 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.897+0000 7f3b067fc700 1 -- 172.21.15.143:0/843899525 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b0c0038c0 con 0x7f3b101df0e0 2023-11-08T19:30:39.913 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.897+0000 7f3b17c11700 1 -- 172.21.15.143:0/843899525 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b101de3a0 con 0x7f3b101df0e0 2023-11-08T19:30:39.913 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.898+0000 7f3b17c11700 1 -- 172.21.15.143:0/843899525 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b1004f730 con 0x7f3b101df0e0 2023-11-08T19:30:39.913 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.898+0000 7f3b067fc700 1 -- 172.21.15.143:0/843899525 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3b0c003a20 con 0x7f3b101df0e0 2023-11-08T19:30:39.913 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.898+0000 7f3b067fc700 1 --2- 172.21.15.143:0/843899525 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f3af406af00 0x7f3af406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:39.913 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.899+0000 7f3b067fc700 1 -- 172.21.15.143:0/843899525 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 7173+0+0 (secure 0 0 0) 0x7f3b0c089a80 con 0x7f3b101df0e0 2023-11-08T19:30:39.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.899+0000 7f3b159ad700 1 --2- 172.21.15.143:0/843899525 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f3af406af00 0x7f3af406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:39.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.899+0000 7f3b159ad700 1 --2- 172.21.15.143:0/843899525 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f3af406af00 0x7f3af406d3c0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f3b000011d0 tx=0x7f3b00009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:39.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:39.903+0000 7f3b067fc700 1 -- 172.21.15.143:0/843899525 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3b0c054570 con 0x7f3b101df0e0 2023-11-08T19:30:40.134 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.133+0000 7f3b17c11700 1 -- 172.21.15.143:0/843899525 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3b101de850 con 0x7f3b101df0e0 2023-11-08T19:30:40.135 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.133+0000 7f3b067fc700 1 -- 172.21.15.143:0/843899525 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+4991 (secure 0 0 0) 0x7f3b0c057b90 con 0x7f3b101df0e0 2023-11-08T19:30:40.135 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:30:40.135 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":91,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5933,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/3918286710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6835","nonce":3918286710},{"type":"v1","addr":"172.21.15.174:6837","nonce":3918286710}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":6161,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/2129643800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":2129643800},{"type":"v1","addr":"172.21.15.143:6835","nonce":2129643800}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:03.092245+0000","modified":"2023-11-08T19:30:37.621320+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":98,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7494,"mds_1":6918},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6918":{"gid":6918,"name":"b","rank":1,"incarnation":54,"state":"up:active","state_seq":6,"addr":"172.21.15.174:6836/3601088618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3601088618},{"type":"v1","addr":"172.21.15.174:6836","nonce":3601088618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7494":{"gid":7494,"name":"c","rank":0,"incarnation":84,"state":"up:active","state_seq":12,"addr":"172.21.15.143:6834/1535103237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":1535103237},{"type":"v1","addr":"172.21.15.143:6834","nonce":1535103237}]},"join_fscid":-1,"laggy_since":"2023-11-08T19:30:37.621320+0000","export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:30:40.137 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.135+0000 7f3b17c11700 1 -- 172.21.15.143:0/843899525 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f3af406af00 msgr2=0x7f3af406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:40.137 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.135+0000 7f3b17c11700 1 --2- 172.21.15.143:0/843899525 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f3af406af00 0x7f3af406d3c0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f3b000011d0 tx=0x7f3b00009040 comp rx=0 tx=0).stop 2023-11-08T19:30:40.138 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.135+0000 7f3b17c11700 1 -- 172.21.15.143:0/843899525 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3b101df0e0 msgr2=0x7f3b101dd060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:40.138 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.135+0000 7f3b17c11700 1 --2- 172.21.15.143:0/843899525 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3b101df0e0 0x7f3b101dd060 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f3b0c00f7f0 tx=0x7f3b0c00fb00 comp rx=0 tx=0).stop 2023-11-08T19:30:40.138 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.136+0000 7f3b17c11700 1 -- 172.21.15.143:0/843899525 shutdown_connections 2023-11-08T19:30:40.138 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.136+0000 7f3b17c11700 1 --2- 172.21.15.143:0/843899525 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f3af406af00 0x7f3af406d3c0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:40.138 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.136+0000 7f3b17c11700 1 --2- 172.21.15.143:0/843899525 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3b101f3750 0x7f3b101deba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:40.139 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.136+0000 7f3b17c11700 1 --2- 172.21.15.143:0/843899525 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3b101dd5a0 0x7f3b101e4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:40.139 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.136+0000 7f3b17c11700 1 --2- 172.21.15.143:0/843899525 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3b101df0e0 0x7f3b101dd060 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:40.139 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.136+0000 7f3b17c11700 1 -- 172.21.15.143:0/843899525 >> 172.21.15.143:0/843899525 conn(0x7f3b1009aa70 msgr2=0x7f3b101df620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:40.139 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.136+0000 7f3b17c11700 1 -- 172.21.15.143:0/843899525 shutdown_connections 2023-11-08T19:30:40.139 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.136+0000 7f3b17c11700 1 -- 172.21.15.143:0/843899525 wait complete. 2023-11-08T19:30:40.140 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 91 2023-11-08T19:30:40.149 INFO:tasks.ceph.mds.b:Sent signal 9 2023-11-08T19:30:40.150 INFO:tasks.cephfs.mount:Suspending the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-08T19:30:40.151 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:30:40.151 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip link set brx.0 down 2023-11-08T19:30:40.190 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail 6:0 2023-11-08T19:30:40.351 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.350+0000 7f858c792700 1 Processor -- start 2023-11-08T19:30:40.352 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.350+0000 7f858c792700 1 -- start start 2023-11-08T19:30:40.352 INFO:tasks.ceph.mds.b.smithi174.stderr:daemon-helper: command crashed with signal 9 2023-11-08T19:30:40.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.351+0000 7f858c792700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f858415eac0 0x7f858406abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:40.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.351+0000 7f858c792700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f858415ef70 0x7f858406b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:40.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.351+0000 7f858c792700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85840cb640 con 0x7f858415b900 2023-11-08T19:30:40.356 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.351+0000 7f858c792700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8584070120 con 0x7f858415eac0 2023-11-08T19:30:40.356 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.351+0000 7f858c792700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85840d0500 con 0x7f858415ef70 2023-11-08T19:30:40.356 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.351+0000 7f858a52e700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f858415b900 0x7f85840ce3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:52532/0 (socket says 172.21.15.143:52532) 2023-11-08T19:30:40.356 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.351+0000 7f858a52e700 1 -- 172.21.15.143:0/1689641720 learned_addr learned my addr 172.21.15.143:0/1689641720 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:30:40.356 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.351+0000 7f858ad2f700 1 --2- 172.21.15.143:0/1689641720 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f858415ef70 0x7f858406b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:40.356 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.351+0000 7f8589d2d700 1 --2- 172.21.15.143:0/1689641720 >> v2:172.21.15.174:3300/0 conn(0x7f858415eac0 0x7f858406abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:40.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.351+0000 7f858952c700 1 -- 172.21.15.143:0/1689641720 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 852156953 0 0) 0x7f85840cb640 con 0x7f858415b900 2023-11-08T19:30:40.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.351+0000 7f858952c700 1 -- 172.21.15.143:0/1689641720 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f85700035b0 con 0x7f858415b900 2023-11-08T19:30:40.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.351+0000 7f8589d2d700 1 -- 172.21.15.143:0/1689641720 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f858415ef70 msgr2=0x7f858406b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:40.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.351+0000 7f8589d2d700 1 --2- 172.21.15.143:0/1689641720 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f858415ef70 0x7f858406b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:40.358 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.351+0000 7f8589d2d700 1 -- 172.21.15.143:0/1689641720 >> v1:172.21.15.143:6789/0 conn(0x7f858415b900 legacy=0x7f85840ce3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:40.358 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.351+0000 7f8589d2d700 1 -- 172.21.15.143:0/1689641720 --> v2:172.21.15.174:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f858406fe40 con 0x7f858415eac0 2023-11-08T19:30:40.358 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.351+0000 7f858ad2f700 1 --2- 172.21.15.143:0/1689641720 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f858415ef70 0x7f858406b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T19:30:40.358 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.352+0000 7f8589d2d700 1 --2- 172.21.15.143:0/1689641720 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f858415eac0 0x7f858406abe0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f8574009840 tx=0x7f8574009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e8ea78014f32f456 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:40.358 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.352+0000 7f858952c700 1 -- 172.21.15.143:0/1689641720 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8574011830 con 0x7f858415eac0 2023-11-08T19:30:40.358 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.352+0000 7f858952c700 1 -- 172.21.15.143:0/1689641720 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8574011990 con 0x7f858415eac0 2023-11-08T19:30:40.359 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.352+0000 7f858952c700 1 -- 172.21.15.143:0/1689641720 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8574011d00 con 0x7f858415eac0 2023-11-08T19:30:40.359 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.352+0000 7f858c792700 1 -- 172.21.15.143:0/1689641720 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f858415eac0 msgr2=0x7f858406abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:40.359 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.352+0000 7f858c792700 1 --2- 172.21.15.143:0/1689641720 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f858415eac0 0x7f858406abe0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f8574009840 tx=0x7f8574009b50 comp rx=0 tx=0).stop 2023-11-08T19:30:40.359 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.352+0000 7f858c792700 1 -- 172.21.15.143:0/1689641720 shutdown_connections 2023-11-08T19:30:40.359 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.352+0000 7f858c792700 1 --2- 172.21.15.143:0/1689641720 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f858415ef70 0x7f858406b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:40.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.352+0000 7f858c792700 1 --2- 172.21.15.143:0/1689641720 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f858415eac0 0x7f858406abe0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:40.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.352+0000 7f858c792700 1 -- 172.21.15.143:0/1689641720 >> 172.21.15.143:0/1689641720 conn(0x7f85840b23d0 msgr2=0x7f85840b27b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:40.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.353+0000 7f858c792700 1 -- 172.21.15.143:0/1689641720 shutdown_connections 2023-11-08T19:30:40.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.353+0000 7f858c792700 1 -- 172.21.15.143:0/1689641720 wait complete. 2023-11-08T19:30:40.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.354+0000 7f858c792700 1 Processor -- start 2023-11-08T19:30:40.361 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.354+0000 7f858c792700 1 -- start start 2023-11-08T19:30:40.364 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.354+0000 7f858c792700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f858415b900 0x7f85841f6e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:40.364 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.354+0000 7f858c792700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f858415eac0 0x7f85840c9a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:40.365 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.354+0000 7f858c792700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f858415ef70 0x7f85841f3170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:40.365 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.354+0000 7f858c792700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85840c9640 con 0x7f858415b900 2023-11-08T19:30:40.365 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.354+0000 7f858c792700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85841f7d30 con 0x7f858415eac0 2023-11-08T19:30:40.365 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.354+0000 7f858c792700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85841f7490 con 0x7f858415ef70 2023-11-08T19:30:40.365 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.355+0000 7f858a52e700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f858415b900 0x7f85841f6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:40.366 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.355+0000 7f858a52e700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f858415b900 0x7f85841f6e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56384/0 (socket says 172.21.15.143:56384) 2023-11-08T19:30:40.366 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.355+0000 7f858a52e700 1 -- 172.21.15.143:0/3605017084 learned_addr learned my addr 172.21.15.143:0/3605017084 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:30:40.366 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.355+0000 7f858ad2f700 1 --2- 172.21.15.143:0/3605017084 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f858415ef70 0x7f85841f3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:40.366 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.355+0000 7f8589d2d700 1 --2- 172.21.15.143:0/3605017084 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f858415eac0 0x7f85840c9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:40.366 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.355+0000 7f858a52e700 1 -- 172.21.15.143:0/3605017084 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f858415ef70 msgr2=0x7f85841f3170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:40.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.355+0000 7f858a52e700 1 --2- 172.21.15.143:0/3605017084 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f858415ef70 0x7f85841f3170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:40.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.355+0000 7f858a52e700 1 -- 172.21.15.143:0/3605017084 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f858415eac0 msgr2=0x7f85840c9a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:40.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.355+0000 7f858a52e700 1 --2- 172.21.15.143:0/3605017084 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f858415eac0 0x7f85840c9a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:40.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.355+0000 7f858a52e700 1 -- 172.21.15.143:0/3605017084 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f857c009a70 con 0x7f858415b900 2023-11-08T19:30:40.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.355+0000 7f8589d2d700 1 --2- 172.21.15.143:0/3605017084 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f858415eac0 0x7f85840c9a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:30:40.368 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.355+0000 7f858a52e700 1 --2- 172.21.15.143:0/3605017084 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f858415b900 0x7f85841f6e00 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f858000c420 tx=0x7f858000c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:40.368 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.355+0000 7f857affd700 1 -- 172.21.15.143:0/3605017084 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8580005d70 con 0x7f858415b900 2023-11-08T19:30:40.368 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.355+0000 7f857affd700 1 -- 172.21.15.143:0/3605017084 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85800070d0 con 0x7f858415b900 2023-11-08T19:30:40.368 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.355+0000 7f858c792700 1 -- 172.21.15.143:0/3605017084 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85740094a0 con 0x7f858415b900 2023-11-08T19:30:40.368 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.355+0000 7f857affd700 1 -- 172.21.15.143:0/3605017084 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8580007440 con 0x7f858415b900 2023-11-08T19:30:40.369 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.355+0000 7f858c792700 1 -- 172.21.15.143:0/3605017084 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85841f7a50 con 0x7f858415b900 2023-11-08T19:30:40.369 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.356+0000 7f858c792700 1 -- 172.21.15.143:0/3605017084 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f858406f990 con 0x7f858415b900 2023-11-08T19:30:40.369 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.357+0000 7f857affd700 1 -- 172.21.15.143:0/3605017084 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f85800075a0 con 0x7f858415b900 2023-11-08T19:30:40.369 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.357+0000 7f857affd700 1 --2- 172.21.15.143:0/3605017084 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f856806af00 0x7f856806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:40.369 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.357+0000 7f857affd700 1 -- 172.21.15.143:0/3605017084 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 7173+0+0 (secure 0 0 0) 0x7f8580088e70 con 0x7f858415b900 2023-11-08T19:30:40.370 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.357+0000 7f8589d2d700 1 --2- 172.21.15.143:0/3605017084 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f856806af00 0x7f856806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:40.370 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.358+0000 7f8589d2d700 1 --2- 172.21.15.143:0/3605017084 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f856806af00 0x7f856806d3c0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f8574009fd0 tx=0x7f857401c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:40.370 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.361+0000 7f857affd700 1 -- 172.21.15.143:0/3605017084 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8580053aa0 con 0x7f858415b900 2023-11-08T19:30:40.630 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.629+0000 7f858c792700 1 -- 172.21.15.143:0/3605017084 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "6:0"} v 0) v1 -- 0x7f858404f730 con 0x7f858415b900 2023-11-08T19:30:40.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.631+0000 7f857affd700 1 -- 172.21.15.143:0/3605017084 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "6:0"}]=0 failed mds gid 7494 v92) v1 ==== 97+0+0 (secure 0 0 0) 0x7f8580017020 con 0x7f858415b900 2023-11-08T19:30:40.635 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.633+0000 7f858c792700 1 -- 172.21.15.143:0/3605017084 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f856806af00 msgr2=0x7f856806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:40.636 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.633+0000 7f858c792700 1 --2- 172.21.15.143:0/3605017084 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f856806af00 0x7f856806d3c0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f8574009fd0 tx=0x7f857401c040 comp rx=0 tx=0).stop 2023-11-08T19:30:40.636 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.633+0000 7f858c792700 1 -- 172.21.15.143:0/3605017084 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f858415b900 msgr2=0x7f85841f6e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:40.636 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.633+0000 7f858c792700 1 --2- 172.21.15.143:0/3605017084 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f858415b900 0x7f85841f6e00 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f858000c420 tx=0x7f858000c7e0 comp rx=0 tx=0).stop 2023-11-08T19:30:40.636 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.634+0000 7f858c792700 1 -- 172.21.15.143:0/3605017084 shutdown_connections 2023-11-08T19:30:40.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.634+0000 7f858c792700 1 --2- 172.21.15.143:0/3605017084 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f856806af00 0x7f856806d3c0 secure :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f8574009fd0 tx=0x7f857401c040 comp rx=0 tx=0).stop 2023-11-08T19:30:40.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.634+0000 7f858c792700 1 --2- 172.21.15.143:0/3605017084 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f858415ef70 0x7f85841f3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:40.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.634+0000 7f858c792700 1 --2- 172.21.15.143:0/3605017084 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f858415b900 0x7f85841f6e00 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:40.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.634+0000 7f858c792700 1 --2- 172.21.15.143:0/3605017084 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f858415eac0 0x7f85840c9a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:40.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.634+0000 7f858c792700 1 -- 172.21.15.143:0/3605017084 >> 172.21.15.143:0/3605017084 conn(0x7f85840b23d0 msgr2=0x7f85840b0240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:40.638 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.634+0000 7f858c792700 1 -- 172.21.15.143:0/3605017084 shutdown_connections 2023-11-08T19:30:40.638 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.634+0000 7f858c792700 1 -- 172.21.15.143:0/3605017084 wait complete. 2023-11-08T19:30:40.638 INFO:teuthology.orchestra.run.smithi143.stderr:failed mds gid 7494 2023-11-08T19:30:40.650 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-08T19:30:40.650 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-08T19:30:40.651 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-08T19:30:40.651 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T19:30:40.652 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-08T19:30:40.652 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi143 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c' 2023-11-08T19:30:40.653 INFO:tasks.ceph.mds.c:Stopped 2023-11-08T19:30:40.653 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-08T19:30:40.655 INFO:tasks.ceph.mds.c:Started 2023-11-08T19:30:40.656 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:30:40.733 INFO:tasks.ceph.mds.c.smithi143.stdout:starting mds.c at 2023-11-08T19:30:40.780 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.777+0000 7f2a56b85700 1 Processor -- start 2023-11-08T19:30:40.781 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.778+0000 7f2a56b85700 1 -- start start 2023-11-08T19:30:40.781 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.778+0000 7f2a56b85700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f2a501612f0 0x7f2a500ccac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:40.781 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.778+0000 7f2a56b85700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f2a5006b2f0 0x7f2a5006de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:40.781 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.778+0000 7f2a56b85700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a5006d9f0 con 0x7f2a50162200 2023-11-08T19:30:40.782 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.778+0000 7f2a56b85700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a500cb6f0 con 0x7f2a5006b2f0 2023-11-08T19:30:40.782 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.778+0000 7f2a56b85700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a5006d750 con 0x7f2a501612f0 2023-11-08T19:30:40.782 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.778+0000 7f2a55122700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f2a5006b2f0 0x7f2a5006de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:40.782 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.778+0000 7f2a55122700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f2a5006b2f0 0x7f2a5006de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56422/0 (socket says 172.21.15.143:56422) 2023-11-08T19:30:40.782 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.778+0000 7f2a55122700 1 -- 172.21.15.143:0/2101635539 learned_addr learned my addr 172.21.15.143:0/2101635539 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:30:40.783 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.779+0000 7f2a54921700 1 --2- 172.21.15.143:0/2101635539 >> v2:172.21.15.174:3300/0 conn(0x7f2a501612f0 0x7f2a500ccac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:40.783 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.779+0000 7f2a55122700 1 -- 172.21.15.143:0/2101635539 >> v2:172.21.15.174:3300/0 conn(0x7f2a501612f0 msgr2=0x7f2a500ccac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:40.783 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.779+0000 7f2a55122700 1 --2- 172.21.15.143:0/2101635539 >> v2:172.21.15.174:3300/0 conn(0x7f2a501612f0 0x7f2a500ccac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:40.783 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.779+0000 7f2a55122700 1 -- 172.21.15.143:0/2101635539 >> v1:172.21.15.174:6789/0 conn(0x7f2a50162200 legacy=0x7f2a5006abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:40.783 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.779+0000 7f2a55122700 1 -- 172.21.15.143:0/2101635539 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a5006fdd0 con 0x7f2a5006b2f0 2023-11-08T19:30:40.784 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.779+0000 7f2a54921700 1 --2- 172.21.15.143:0/2101635539 >> v2:172.21.15.174:3300/0 conn(0x7f2a501612f0 0x7f2a500ccac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:30:40.784 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.779+0000 7f2a55122700 1 --2- 172.21.15.143:0/2101635539 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2a5006b2f0 0x7f2a5006de80 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f2a500d18b0 tx=0x7f2a4800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ccb1870803763ad server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:40.784 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.779+0000 7f2a4f7fe700 1 -- 172.21.15.143:0/2101635539 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a4800e070 con 0x7f2a5006b2f0 2023-11-08T19:30:40.784 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.779+0000 7f2a4f7fe700 1 -- 172.21.15.143:0/2101635539 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a48003a20 con 0x7f2a5006b2f0 2023-11-08T19:30:40.784 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.779+0000 7f2a4f7fe700 1 -- 172.21.15.143:0/2101635539 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a48003dd0 con 0x7f2a5006b2f0 2023-11-08T19:30:40.787 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.780+0000 7f2a56b85700 1 -- 172.21.15.143:0/2101635539 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2a5006b2f0 msgr2=0x7f2a5006de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:40.788 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.780+0000 7f2a56b85700 1 --2- 172.21.15.143:0/2101635539 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2a5006b2f0 0x7f2a5006de80 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f2a500d18b0 tx=0x7f2a4800b330 comp rx=0 tx=0).stop 2023-11-08T19:30:40.788 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.780+0000 7f2a56b85700 1 -- 172.21.15.143:0/2101635539 shutdown_connections 2023-11-08T19:30:40.788 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.780+0000 7f2a56b85700 1 --2- 172.21.15.143:0/2101635539 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2a5006b2f0 0x7f2a5006de80 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:40.788 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.780+0000 7f2a56b85700 1 --2- 172.21.15.143:0/2101635539 >> v2:172.21.15.174:3300/0 conn(0x7f2a501612f0 0x7f2a500ccac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:40.789 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.780+0000 7f2a56b85700 1 -- 172.21.15.143:0/2101635539 >> 172.21.15.143:0/2101635539 conn(0x7f2a500b23d0 msgr2=0x7f2a500b27b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:40.789 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.780+0000 7f2a56b85700 1 -- 172.21.15.143:0/2101635539 shutdown_connections 2023-11-08T19:30:40.789 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.780+0000 7f2a56b85700 1 -- 172.21.15.143:0/2101635539 wait complete. 2023-11-08T19:30:40.789 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.781+0000 7f2a56b85700 1 Processor -- start 2023-11-08T19:30:40.789 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.781+0000 7f2a56b85700 1 -- start start 2023-11-08T19:30:40.790 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.782+0000 7f2a56b85700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2a5006b2f0 0x7f2a500b6050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:40.790 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.782+0000 7f2a56b85700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2a501612f0 0x7f2a500ca1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:40.790 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.782+0000 7f2a56b85700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a50162200 0x7f2a5014fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:40.790 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.782+0000 7f2a56b85700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a500c9100 con 0x7f2a5006b2f0 2023-11-08T19:30:40.790 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.782+0000 7f2a56b85700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a500c93e0 con 0x7f2a50162200 2023-11-08T19:30:40.791 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.782+0000 7f2a56b85700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a500c96c0 con 0x7f2a501612f0 2023-11-08T19:30:40.791 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.782+0000 7f2a54921700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2a5006b2f0 0x7f2a500b6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:40.791 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.782+0000 7f2a54921700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2a5006b2f0 0x7f2a500b6050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56424/0 (socket says 172.21.15.143:56424) 2023-11-08T19:30:40.791 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.782+0000 7f2a54921700 1 -- 172.21.15.143:0/533751672 learned_addr learned my addr 172.21.15.143:0/533751672 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:30:40.791 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.782+0000 7f2a55122700 1 --2- 172.21.15.143:0/533751672 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a50162200 0x7f2a5014fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:40.792 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.782+0000 7f2a4ffff700 1 --2- 172.21.15.143:0/533751672 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2a501612f0 0x7f2a500ca1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:40.792 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.782+0000 7f2a54921700 1 -- 172.21.15.143:0/533751672 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2a501612f0 msgr2=0x7f2a500ca1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:40.792 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.782+0000 7f2a54921700 1 --2- 172.21.15.143:0/533751672 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2a501612f0 0x7f2a500ca1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:40.792 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.782+0000 7f2a54921700 1 -- 172.21.15.143:0/533751672 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a50162200 msgr2=0x7f2a5014fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:40.792 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.782+0000 7f2a54921700 1 --2- 172.21.15.143:0/533751672 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a50162200 0x7f2a5014fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:40.792 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.782+0000 7f2a54921700 1 -- 172.21.15.143:0/533751672 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a48009b30 con 0x7f2a5006b2f0 2023-11-08T19:30:40.793 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.782+0000 7f2a54921700 1 --2- 172.21.15.143:0/533751672 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2a5006b2f0 0x7f2a500b6050 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f2a4400c750 tx=0x7f2a4400ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:40.793 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.782+0000 7f2a4d7fa700 1 -- 172.21.15.143:0/533751672 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a44011800 con 0x7f2a5006b2f0 2023-11-08T19:30:40.793 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.782+0000 7f2a4d7fa700 1 -- 172.21.15.143:0/533751672 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a44011960 con 0x7f2a5006b2f0 2023-11-08T19:30:40.793 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.783+0000 7f2a56b85700 1 -- 172.21.15.143:0/533751672 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a500c99a0 con 0x7f2a5006b2f0 2023-11-08T19:30:40.793 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.783+0000 7f2a4d7fa700 1 -- 172.21.15.143:0/533751672 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a44011cd0 con 0x7f2a5006b2f0 2023-11-08T19:30:40.794 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.783+0000 7f2a56b85700 1 -- 172.21.15.143:0/533751672 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a5015a5b0 con 0x7f2a5006b2f0 2023-11-08T19:30:40.794 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.783+0000 7f2a56b85700 1 -- 172.21.15.143:0/533751672 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a5004f730 con 0x7f2a5006b2f0 2023-11-08T19:30:40.794 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.784+0000 7f2a4d7fa700 1 -- 172.21.15.143:0/533751672 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2a44023480 con 0x7f2a5006b2f0 2023-11-08T19:30:40.794 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.785+0000 7f2a4d7fa700 1 --2- 172.21.15.143:0/533751672 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f2a3406af00 0x7f2a3406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:40.794 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.785+0000 7f2a4d7fa700 1 -- 172.21.15.143:0/533751672 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 7231+0+0 (secure 0 0 0) 0x7f2a44089d50 con 0x7f2a5006b2f0 2023-11-08T19:30:40.795 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.785+0000 7f2a4ffff700 1 --2- 172.21.15.143:0/533751672 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f2a3406af00 0x7f2a3406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:40.795 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.785+0000 7f2a4ffff700 1 --2- 172.21.15.143:0/533751672 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f2a3406af00 0x7f2a3406d3c0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f2a500b2b90 tx=0x7f2a3c0090e0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:40.795 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:40.788+0000 7f2a4d7fa700 1 -- 172.21.15.143:0/533751672 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2a44054980 con 0x7f2a5006b2f0 2023-11-08T19:30:41.018 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:41.017+0000 7f2a56b85700 1 -- 172.21.15.143:0/533751672 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2a50072300 con 0x7f2a5006b2f0 2023-11-08T19:30:41.019 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:41.018+0000 7f2a4d7fa700 1 -- 172.21.15.143:0/533751672 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 93 v93) v1 ==== 94+0+4149 (secure 0 0 0) 0x7f2a4401e020 con 0x7f2a5006b2f0 2023-11-08T19:30:41.019 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:30:41.020 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":93,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5933,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/3918286710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6835","nonce":3918286710},{"type":"v1","addr":"172.21.15.174:6837","nonce":3918286710}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61}],"filesystems":[{"mdsmap":{"epoch":93,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:03.092245+0000","modified":"2023-11-08T19:30:40.635633+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":102,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6161,"mds_1":6918},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6161":{"gid":6161,"name":"a","rank":0,"incarnation":93,"state":"up:replay","state_seq":1,"addr":"172.21.15.143:6835/2129643800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":2129643800},{"type":"v1","addr":"172.21.15.143:6835","nonce":2129643800}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6918":{"gid":6918,"name":"b","rank":1,"incarnation":54,"state":"up:active","state_seq":6,"addr":"172.21.15.174:6836/3601088618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3601088618},{"type":"v1","addr":"172.21.15.174:6836","nonce":3601088618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:30:41.021 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:41.020+0000 7f2a56b85700 1 -- 172.21.15.143:0/533751672 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f2a3406af00 msgr2=0x7f2a3406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:41.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:41.020+0000 7f2a56b85700 1 --2- 172.21.15.143:0/533751672 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f2a3406af00 0x7f2a3406d3c0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f2a500b2b90 tx=0x7f2a3c0090e0 comp rx=0 tx=0).stop 2023-11-08T19:30:41.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:41.020+0000 7f2a56b85700 1 -- 172.21.15.143:0/533751672 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2a5006b2f0 msgr2=0x7f2a500b6050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:41.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:41.020+0000 7f2a56b85700 1 --2- 172.21.15.143:0/533751672 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2a5006b2f0 0x7f2a500b6050 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f2a4400c750 tx=0x7f2a4400ca60 comp rx=0 tx=0).stop 2023-11-08T19:30:41.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:41.020+0000 7f2a56b85700 1 -- 172.21.15.143:0/533751672 shutdown_connections 2023-11-08T19:30:41.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:41.020+0000 7f2a56b85700 1 --2- 172.21.15.143:0/533751672 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f2a3406af00 0x7f2a3406d3c0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:41.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:41.020+0000 7f2a56b85700 1 --2- 172.21.15.143:0/533751672 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a50162200 0x7f2a5014fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:41.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:41.020+0000 7f2a56b85700 1 --2- 172.21.15.143:0/533751672 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2a5006b2f0 0x7f2a500b6050 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:41.024 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:41.020+0000 7f2a56b85700 1 --2- 172.21.15.143:0/533751672 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2a501612f0 0x7f2a500ca1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:41.024 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:41.020+0000 7f2a56b85700 1 -- 172.21.15.143:0/533751672 >> 172.21.15.143:0/533751672 conn(0x7f2a500b23d0 msgr2=0x7f2a5006e710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:41.024 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:41.020+0000 7f2a56b85700 1 -- 172.21.15.143:0/533751672 shutdown_connections 2023-11-08T19:30:41.024 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:41.020+0000 7f2a56b85700 1 -- 172.21.15.143:0/533751672 wait complete. 2023-11-08T19:30:41.025 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 93 2023-11-08T19:30:41.033 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=30 retry_count=0)... 2023-11-08T19:30:43.291 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2023-11-08T19:30:43.292 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c has been restored 2023-11-08T19:30:46.035 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:30:46.168 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.167+0000 7fed8467b700 1 Processor -- start 2023-11-08T19:30:46.169 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.167+0000 7fed8467b700 1 -- start start 2023-11-08T19:30:46.171 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.168+0000 7fed8467b700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fed7c15e030 0x7fed7c0b3c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:46.171 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.168+0000 7fed8467b700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fed7c06b2f0 0x7fed7c06e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:46.172 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.168+0000 7fed8467b700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed7c15e410 con 0x7fed7c0b4160 2023-11-08T19:30:46.172 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.168+0000 7fed8467b700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed7c06def0 con 0x7fed7c15e030 2023-11-08T19:30:46.172 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.168+0000 7fed8467b700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed7c072410 con 0x7fed7c06b2f0 2023-11-08T19:30:46.172 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.168+0000 7fed82417700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fed7c15e030 0x7fed7c0b3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:46.172 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.168+0000 7fed81c16700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fed7c0b4160 0x7fed7c06abe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:52602/0 (socket says 172.21.15.143:52602) 2023-11-08T19:30:46.173 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.168+0000 7fed82417700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fed7c15e030 0x7fed7c0b3c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56448/0 (socket says 172.21.15.143:56448) 2023-11-08T19:30:46.173 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.168+0000 7fed81c16700 1 -- 172.21.15.143:0/674233625 learned_addr learned my addr 172.21.15.143:0/674233625 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:30:46.173 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.168+0000 7fed82c18700 1 --2- 172.21.15.143:0/674233625 >> v2:172.21.15.174:3300/0 conn(0x7fed7c06b2f0 0x7fed7c06e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:46.173 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.168+0000 7fed82417700 1 -- 172.21.15.143:0/674233625 >> v2:172.21.15.174:3300/0 conn(0x7fed7c06b2f0 msgr2=0x7fed7c06e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:46.173 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.168+0000 7fed82417700 1 --2- 172.21.15.143:0/674233625 >> v2:172.21.15.174:3300/0 conn(0x7fed7c06b2f0 0x7fed7c06e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:46.174 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.168+0000 7fed82417700 1 -- 172.21.15.143:0/674233625 >> v1:172.21.15.143:6789/0 conn(0x7fed7c0b4160 legacy=0x7fed7c06abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:46.174 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.168+0000 7fed82417700 1 -- 172.21.15.143:0/674233625 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed7c0726f0 con 0x7fed7c15e030 2023-11-08T19:30:46.174 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.168+0000 7fed81415700 1 -- 172.21.15.143:0/674233625 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3984684116 0 0) 0x7fed7c15e410 con 0x7fed7c0b4160 2023-11-08T19:30:46.174 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.168+0000 7fed82417700 1 --2- 172.21.15.143:0/674233625 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fed7c15e030 0x7fed7c0b3c20 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7fed7c0c7890 tx=0x7fed6c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da8b2febe5c50ee1 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:46.174 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.169+0000 7fed81415700 1 -- 172.21.15.143:0/674233625 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed6c011830 con 0x7fed7c15e030 2023-11-08T19:30:46.175 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.169+0000 7fed81415700 1 -- 172.21.15.143:0/674233625 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed6c011990 con 0x7fed7c15e030 2023-11-08T19:30:46.175 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.169+0000 7fed81415700 1 -- 172.21.15.143:0/674233625 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed6c011d20 con 0x7fed7c15e030 2023-11-08T19:30:46.175 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.169+0000 7fed8467b700 1 -- 172.21.15.143:0/674233625 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fed7c15e030 msgr2=0x7fed7c0b3c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:46.175 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.169+0000 7fed8467b700 1 --2- 172.21.15.143:0/674233625 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fed7c15e030 0x7fed7c0b3c20 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7fed7c0c7890 tx=0x7fed6c009b20 comp rx=0 tx=0).stop 2023-11-08T19:30:46.175 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.169+0000 7fed8467b700 1 -- 172.21.15.143:0/674233625 shutdown_connections 2023-11-08T19:30:46.176 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.169+0000 7fed8467b700 1 --2- 172.21.15.143:0/674233625 >> v2:172.21.15.174:3300/0 conn(0x7fed7c06b2f0 0x7fed7c06e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:46.176 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.169+0000 7fed8467b700 1 --2- 172.21.15.143:0/674233625 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fed7c15e030 0x7fed7c0b3c20 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:46.176 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.169+0000 7fed8467b700 1 -- 172.21.15.143:0/674233625 >> 172.21.15.143:0/674233625 conn(0x7fed7c0d0830 msgr2=0x7fed7c0d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:46.176 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.169+0000 7fed8467b700 1 -- 172.21.15.143:0/674233625 shutdown_connections 2023-11-08T19:30:46.176 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.169+0000 7fed8467b700 1 -- 172.21.15.143:0/674233625 wait complete. 2023-11-08T19:30:46.179 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.170+0000 7fed8467b700 1 Processor -- start 2023-11-08T19:30:46.180 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.170+0000 7fed8467b700 1 -- start start 2023-11-08T19:30:46.180 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.171+0000 7fed8467b700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fed7c06b2f0 0x7fed7c160010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:46.180 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.171+0000 7fed8467b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fed7c0b4160 0x7fed7c0ce3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:46.181 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.171+0000 7fed8467b700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fed7c15e030 0x7fed7c0ce920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:46.181 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.171+0000 7fed8467b700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed7c15f300 con 0x7fed7c15e030 2023-11-08T19:30:46.181 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.171+0000 7fed8467b700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed7c15e410 con 0x7fed7c0b4160 2023-11-08T19:30:46.181 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.171+0000 7fed8467b700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed7c15e6f0 con 0x7fed7c06b2f0 2023-11-08T19:30:46.182 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.171+0000 7fed82c18700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fed7c15e030 0x7fed7c0ce920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:46.182 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.171+0000 7fed82c18700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fed7c15e030 0x7fed7c0ce920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56458/0 (socket says 172.21.15.143:56458) 2023-11-08T19:30:46.182 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.171+0000 7fed82c18700 1 -- 172.21.15.143:0/255057452 learned_addr learned my addr 172.21.15.143:0/255057452 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:30:46.182 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.172+0000 7fed82417700 1 --2- 172.21.15.143:0/255057452 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fed7c06b2f0 0x7fed7c160010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:46.183 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.172+0000 7fed81c16700 1 --2- 172.21.15.143:0/255057452 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fed7c0b4160 0x7fed7c0ce3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:46.183 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.172+0000 7fed82c18700 1 -- 172.21.15.143:0/255057452 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fed7c06b2f0 msgr2=0x7fed7c160010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:46.183 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.172+0000 7fed82c18700 1 --2- 172.21.15.143:0/255057452 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fed7c06b2f0 0x7fed7c160010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:46.183 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.172+0000 7fed82c18700 1 -- 172.21.15.143:0/255057452 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fed7c0b4160 msgr2=0x7fed7c0ce3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:46.183 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.172+0000 7fed82c18700 1 --2- 172.21.15.143:0/255057452 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fed7c0b4160 0x7fed7c0ce3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:46.184 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.172+0000 7fed82c18700 1 -- 172.21.15.143:0/255057452 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed6c0094a0 con 0x7fed7c15e030 2023-11-08T19:30:46.184 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.172+0000 7fed81c16700 1 --2- 172.21.15.143:0/255057452 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fed7c0b4160 0x7fed7c0ce3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:30:46.184 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.172+0000 7fed82c18700 1 --2- 172.21.15.143:0/255057452 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fed7c15e030 0x7fed7c0ce920 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7fed74007de0 tx=0x7fed7400e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:46.184 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.172+0000 7fed72ffd700 1 -- 172.21.15.143:0/255057452 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed7400ede0 con 0x7fed7c15e030 2023-11-08T19:30:46.184 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.172+0000 7fed72ffd700 1 -- 172.21.15.143:0/255057452 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed74014420 con 0x7fed7c15e030 2023-11-08T19:30:46.184 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.172+0000 7fed8467b700 1 -- 172.21.15.143:0/255057452 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed7c15e9d0 con 0x7fed7c15e030 2023-11-08T19:30:46.185 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.172+0000 7fed72ffd700 1 -- 172.21.15.143:0/255057452 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed740147d0 con 0x7fed7c15e030 2023-11-08T19:30:46.185 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.172+0000 7fed8467b700 1 -- 172.21.15.143:0/255057452 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed7c15ef90 con 0x7fed7c15e030 2023-11-08T19:30:46.185 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.173+0000 7fed8467b700 1 -- 172.21.15.143:0/255057452 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed7c0c4360 con 0x7fed7c15e030 2023-11-08T19:30:46.185 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.173+0000 7fed72ffd700 1 -- 172.21.15.143:0/255057452 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fed74014930 con 0x7fed7c15e030 2023-11-08T19:30:46.185 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.174+0000 7fed72ffd700 1 --2- 172.21.15.143:0/255057452 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fed6006af00 0x7fed6006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:46.185 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.174+0000 7fed72ffd700 1 -- 172.21.15.143:0/255057452 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 7231+0+0 (secure 0 0 0) 0x7fed7408a390 con 0x7fed7c15e030 2023-11-08T19:30:46.186 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.174+0000 7fed82417700 1 --2- 172.21.15.143:0/255057452 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fed6006af00 0x7fed6006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:46.186 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.174+0000 7fed82417700 1 --2- 172.21.15.143:0/255057452 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fed6006af00 0x7fed6006d3c0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fed6c0090b0 tx=0x7fed6c01c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:46.186 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.178+0000 7fed72ffd700 1 -- 172.21.15.143:0/255057452 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fed7c0c4360 con 0x7fed7c15e030 2023-11-08T19:30:46.410 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.409+0000 7fed8467b700 1 -- 172.21.15.143:0/255057452 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fed7c0c4360 con 0x7fed7c15e030 2023-11-08T19:30:46.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.409+0000 7fed72ffd700 1 -- 172.21.15.143:0/255057452 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) v1 ==== 94+0+4944 (secure 0 0 0) 0x7fed7c0c4360 con 0x7fed7c15e030 2023-11-08T19:30:46.411 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:30:46.411 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":94,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5933,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/3918286710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6835","nonce":3918286710},{"type":"v1","addr":"172.21.15.174:6837","nonce":3918286710}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":7743,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6834/2489252684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":2489252684},{"type":"v1","addr":"172.21.15.143:6834","nonce":2489252684}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:03.092245+0000","modified":"2023-11-08T19:30:40.652694+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":102,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6161,"mds_1":6918},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6161":{"gid":6161,"name":"a","rank":0,"incarnation":93,"state":"up:resolve","state_seq":8,"addr":"172.21.15.143:6835/2129643800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":2129643800},{"type":"v1","addr":"172.21.15.143:6835","nonce":2129643800}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6918":{"gid":6918,"name":"b","rank":1,"incarnation":54,"state":"up:active","state_seq":6,"addr":"172.21.15.174:6836/3601088618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3601088618},{"type":"v1","addr":"172.21.15.174:6836","nonce":3601088618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:30:46.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.411+0000 7fed8467b700 1 -- 172.21.15.143:0/255057452 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fed6006af00 msgr2=0x7fed6006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:46.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.411+0000 7fed8467b700 1 --2- 172.21.15.143:0/255057452 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fed6006af00 0x7fed6006d3c0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fed6c0090b0 tx=0x7fed6c01c040 comp rx=0 tx=0).stop 2023-11-08T19:30:46.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.411+0000 7fed8467b700 1 -- 172.21.15.143:0/255057452 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fed7c15e030 msgr2=0x7fed7c0ce920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:46.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.411+0000 7fed8467b700 1 --2- 172.21.15.143:0/255057452 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fed7c15e030 0x7fed7c0ce920 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7fed74007de0 tx=0x7fed7400e3c0 comp rx=0 tx=0).stop 2023-11-08T19:30:46.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.412+0000 7fed8467b700 1 -- 172.21.15.143:0/255057452 shutdown_connections 2023-11-08T19:30:46.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.412+0000 7fed8467b700 1 --2- 172.21.15.143:0/255057452 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fed6006af00 0x7fed6006d3c0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:46.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.412+0000 7fed8467b700 1 --2- 172.21.15.143:0/255057452 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fed7c06b2f0 0x7fed7c160010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:46.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.412+0000 7fed8467b700 1 --2- 172.21.15.143:0/255057452 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fed7c15e030 0x7fed7c0ce920 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:46.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.412+0000 7fed8467b700 1 --2- 172.21.15.143:0/255057452 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fed7c0b4160 0x7fed7c0ce3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:46.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.412+0000 7fed8467b700 1 -- 172.21.15.143:0/255057452 >> 172.21.15.143:0/255057452 conn(0x7fed7c0d0830 msgr2=0x7fed7c15d260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:46.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.412+0000 7fed8467b700 1 -- 172.21.15.143:0/255057452 shutdown_connections 2023-11-08T19:30:46.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.412+0000 7fed8467b700 1 -- 172.21.15.143:0/255057452 wait complete. 2023-11-08T19:30:46.416 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 94 2023-11-08T19:30:46.425 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2023-11-08T19:30:46.425 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:30:46.551 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.550+0000 7f7f4bd37700 1 Processor -- start 2023-11-08T19:30:46.551 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.550+0000 7f7f4bd37700 1 -- start start 2023-11-08T19:30:46.554 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.551+0000 7f7f4bd37700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f7f4415b900 0x7f7f440ce3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:46.554 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.551+0000 7f7f4bd37700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f7f4415eac0 0x7f7f4406abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:46.554 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.551+0000 7f7f4bd37700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f440ce910 con 0x7f7f4415ef70 2023-11-08T19:30:46.554 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.551+0000 7f7f4bd37700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f44070120 con 0x7f7f4415b900 2023-11-08T19:30:46.554 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.551+0000 7f7f4bd37700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f440cb640 con 0x7f7f4415eac0 2023-11-08T19:30:46.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.551+0000 7f7f49ad3700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f7f4415b900 0x7f7f440ce3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:46.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.551+0000 7f7f4a2d4700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f7f4415ef70 0x7f7f44161ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:52620/0 (socket says 172.21.15.143:52620) 2023-11-08T19:30:46.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.551+0000 7f7f49ad3700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f7f4415b900 0x7f7f440ce3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56462/0 (socket says 172.21.15.143:56462) 2023-11-08T19:30:46.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.551+0000 7f7f4a2d4700 1 -- 172.21.15.143:0/2115040157 learned_addr learned my addr 172.21.15.143:0/2115040157 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:30:46.556 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.551+0000 7f7f492d2700 1 --2- 172.21.15.143:0/2115040157 >> v2:172.21.15.174:3300/0 conn(0x7f7f4415eac0 0x7f7f4406abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:46.556 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.551+0000 7f7f49ad3700 1 -- 172.21.15.143:0/2115040157 >> v2:172.21.15.174:3300/0 conn(0x7f7f4415eac0 msgr2=0x7f7f4406abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:46.556 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.551+0000 7f7f49ad3700 1 --2- 172.21.15.143:0/2115040157 >> v2:172.21.15.174:3300/0 conn(0x7f7f4415eac0 0x7f7f4406abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:46.556 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.551+0000 7f7f49ad3700 1 -- 172.21.15.143:0/2115040157 >> v1:172.21.15.143:6789/0 conn(0x7f7f4415ef70 legacy=0x7f7f44161ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:46.556 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.551+0000 7f7f48ad1700 1 -- 172.21.15.143:0/2115040157 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1237007365 0 0) 0x7f7f440ce910 con 0x7f7f4415ef70 2023-11-08T19:30:46.556 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.551+0000 7f7f49ad3700 1 -- 172.21.15.143:0/2115040157 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f440d0500 con 0x7f7f4415b900 2023-11-08T19:30:46.557 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.551+0000 7f7f492d2700 1 --2- 172.21.15.143:0/2115040157 >> v2:172.21.15.174:3300/0 conn(0x7f7f4415eac0 0x7f7f4406abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:30:46.557 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.551+0000 7f7f49ad3700 1 --2- 172.21.15.143:0/2115040157 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f4415b900 0x7f7f440ce3d0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f7f440b6990 tx=0x7f7f3400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29f3f9ccf95aca97 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:46.557 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.551+0000 7f7f48ad1700 1 -- 172.21.15.143:0/2115040157 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f3400e070 con 0x7f7f4415b900 2023-11-08T19:30:46.557 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.551+0000 7f7f48ad1700 1 -- 172.21.15.143:0/2115040157 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f3400bbd0 con 0x7f7f4415b900 2023-11-08T19:30:46.557 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.551+0000 7f7f48ad1700 1 -- 172.21.15.143:0/2115040157 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f34012430 con 0x7f7f4415b900 2023-11-08T19:30:46.558 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.552+0000 7f7f4bd37700 1 -- 172.21.15.143:0/2115040157 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f4415b900 msgr2=0x7f7f440ce3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:46.558 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.552+0000 7f7f4bd37700 1 --2- 172.21.15.143:0/2115040157 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f4415b900 0x7f7f440ce3d0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f7f440b6990 tx=0x7f7f3400b330 comp rx=0 tx=0).stop 2023-11-08T19:30:46.558 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.552+0000 7f7f4bd37700 1 -- 172.21.15.143:0/2115040157 shutdown_connections 2023-11-08T19:30:46.558 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.552+0000 7f7f4bd37700 1 --2- 172.21.15.143:0/2115040157 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f4415b900 0x7f7f440ce3d0 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:46.558 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.552+0000 7f7f4bd37700 1 --2- 172.21.15.143:0/2115040157 >> v2:172.21.15.174:3300/0 conn(0x7f7f4415eac0 0x7f7f4406abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:46.559 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.552+0000 7f7f4bd37700 1 -- 172.21.15.143:0/2115040157 >> 172.21.15.143:0/2115040157 conn(0x7f7f440b23d0 msgr2=0x7f7f440b27b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:46.559 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.552+0000 7f7f4bd37700 1 -- 172.21.15.143:0/2115040157 shutdown_connections 2023-11-08T19:30:46.559 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.552+0000 7f7f4bd37700 1 -- 172.21.15.143:0/2115040157 wait complete. 2023-11-08T19:30:46.562 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.553+0000 7f7f4bd37700 1 Processor -- start 2023-11-08T19:30:46.562 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.553+0000 7f7f4bd37700 1 -- start start 2023-11-08T19:30:46.562 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.554+0000 7f7f4bd37700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f4415b900 0x7f7f4415a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:46.563 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.554+0000 7f7f4bd37700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7f4415eac0 0x7f7f440b3420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:46.563 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.554+0000 7f7f4bd37700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7f4415ef70 0x7f7f440ad8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:46.563 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.554+0000 7f7f4bd37700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f4415a600 con 0x7f7f4415b900 2023-11-08T19:30:46.563 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.554+0000 7f7f4bd37700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f440ae900 con 0x7f7f4415ef70 2023-11-08T19:30:46.563 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.554+0000 7f7f4bd37700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f440ae340 con 0x7f7f4415eac0 2023-11-08T19:30:46.564 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.554+0000 7f7f49ad3700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f4415b900 0x7f7f4415a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:46.564 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.554+0000 7f7f49ad3700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f4415b900 0x7f7f4415a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56468/0 (socket says 172.21.15.143:56468) 2023-11-08T19:30:46.564 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.554+0000 7f7f49ad3700 1 -- 172.21.15.143:0/1566300357 learned_addr learned my addr 172.21.15.143:0/1566300357 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:30:46.564 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.554+0000 7f7f4a2d4700 1 --2- 172.21.15.143:0/1566300357 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7f4415ef70 0x7f7f440ad8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:46.564 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.554+0000 7f7f492d2700 1 --2- 172.21.15.143:0/1566300357 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7f4415eac0 0x7f7f440b3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:46.564 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.554+0000 7f7f49ad3700 1 -- 172.21.15.143:0/1566300357 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7f4415eac0 msgr2=0x7f7f440b3420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:46.565 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.554+0000 7f7f49ad3700 1 --2- 172.21.15.143:0/1566300357 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7f4415eac0 0x7f7f440b3420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:46.565 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.554+0000 7f7f49ad3700 1 -- 172.21.15.143:0/1566300357 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7f4415ef70 msgr2=0x7f7f440ad8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:46.565 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.554+0000 7f7f49ad3700 1 --2- 172.21.15.143:0/1566300357 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7f4415ef70 0x7f7f440ad8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:46.565 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.554+0000 7f7f49ad3700 1 -- 172.21.15.143:0/1566300357 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f34009a70 con 0x7f7f4415b900 2023-11-08T19:30:46.565 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.554+0000 7f7f4a2d4700 1 --2- 172.21.15.143:0/1566300357 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7f4415ef70 0x7f7f440ad8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:30:46.566 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.555+0000 7f7f49ad3700 1 --2- 172.21.15.143:0/1566300357 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f4415b900 0x7f7f4415a0c0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f7f34014040 tx=0x7f7f34004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:46.566 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.555+0000 7f7f427fc700 1 -- 172.21.15.143:0/1566300357 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f34012430 con 0x7f7f4415b900 2023-11-08T19:30:46.566 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.555+0000 7f7f427fc700 1 -- 172.21.15.143:0/1566300357 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f34020070 con 0x7f7f4415b900 2023-11-08T19:30:46.566 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.555+0000 7f7f4bd37700 1 -- 172.21.15.143:0/1566300357 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f440ae620 con 0x7f7f4415b900 2023-11-08T19:30:46.566 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.555+0000 7f7f427fc700 1 -- 172.21.15.143:0/1566300357 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f34004260 con 0x7f7f4415b900 2023-11-08T19:30:46.567 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.555+0000 7f7f4bd37700 1 -- 172.21.15.143:0/1566300357 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f440af0f0 con 0x7f7f4415b900 2023-11-08T19:30:46.567 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.556+0000 7f7f4bd37700 1 -- 172.21.15.143:0/1566300357 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f440716b0 con 0x7f7f4415b900 2023-11-08T19:30:46.567 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.556+0000 7f7f427fc700 1 -- 172.21.15.143:0/1566300357 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7f340043c0 con 0x7f7f4415b900 2023-11-08T19:30:46.567 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.556+0000 7f7f427fc700 1 --2- 172.21.15.143:0/1566300357 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f7f3006af00 0x7f7f3006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:46.567 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.556+0000 7f7f427fc700 1 -- 172.21.15.143:0/1566300357 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 7231+0+0 (secure 0 0 0) 0x7f7f34022070 con 0x7f7f4415b900 2023-11-08T19:30:46.567 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.557+0000 7f7f492d2700 1 --2- 172.21.15.143:0/1566300357 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f7f3006af00 0x7f7f3006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:46.568 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.557+0000 7f7f492d2700 1 --2- 172.21.15.143:0/1566300357 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f7f3006af00 0x7f7f3006d3c0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f7f38006fd0 tx=0x7f7f38008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:46.568 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.561+0000 7f7f427fc700 1 -- 172.21.15.143:0/1566300357 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7f34056dc0 con 0x7f7f4415b900 2023-11-08T19:30:46.799 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.797+0000 7f7f4bd37700 1 -- 172.21.15.143:0/1566300357 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7f4404f730 con 0x7f7f4415b900 2023-11-08T19:30:46.799 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.798+0000 7f7f427fc700 1 -- 172.21.15.143:0/1566300357 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) v1 ==== 94+0+4944 (secure 0 0 0) 0x7f7f3405a3e0 con 0x7f7f4415b900 2023-11-08T19:30:46.800 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:30:46.800 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":94,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5933,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/3918286710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6835","nonce":3918286710},{"type":"v1","addr":"172.21.15.174:6837","nonce":3918286710}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":7743,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6834/2489252684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":2489252684},{"type":"v1","addr":"172.21.15.143:6834","nonce":2489252684}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:03.092245+0000","modified":"2023-11-08T19:30:40.652694+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":102,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6161,"mds_1":6918},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6161":{"gid":6161,"name":"a","rank":0,"incarnation":93,"state":"up:resolve","state_seq":8,"addr":"172.21.15.143:6835/2129643800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":2129643800},{"type":"v1","addr":"172.21.15.143:6835","nonce":2129643800}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6918":{"gid":6918,"name":"b","rank":1,"incarnation":54,"state":"up:active","state_seq":6,"addr":"172.21.15.174:6836/3601088618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3601088618},{"type":"v1","addr":"172.21.15.174:6836","nonce":3601088618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:30:46.802 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.800+0000 7f7f4bd37700 1 -- 172.21.15.143:0/1566300357 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f7f3006af00 msgr2=0x7f7f3006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:46.802 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.800+0000 7f7f4bd37700 1 --2- 172.21.15.143:0/1566300357 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f7f3006af00 0x7f7f3006d3c0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f7f38006fd0 tx=0x7f7f38008040 comp rx=0 tx=0).stop 2023-11-08T19:30:46.802 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.800+0000 7f7f4bd37700 1 -- 172.21.15.143:0/1566300357 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f4415b900 msgr2=0x7f7f4415a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:46.802 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.800+0000 7f7f4bd37700 1 --2- 172.21.15.143:0/1566300357 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f4415b900 0x7f7f4415a0c0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f7f34014040 tx=0x7f7f34004ad0 comp rx=0 tx=0).stop 2023-11-08T19:30:46.802 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.800+0000 7f7f4bd37700 1 -- 172.21.15.143:0/1566300357 shutdown_connections 2023-11-08T19:30:46.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.800+0000 7f7f4bd37700 1 --2- 172.21.15.143:0/1566300357 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f7f3006af00 0x7f7f3006d3c0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:46.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.800+0000 7f7f4bd37700 1 --2- 172.21.15.143:0/1566300357 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7f4415ef70 0x7f7f440ad8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:46.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.800+0000 7f7f4bd37700 1 --2- 172.21.15.143:0/1566300357 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f4415b900 0x7f7f4415a0c0 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:46.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.800+0000 7f7f4bd37700 1 --2- 172.21.15.143:0/1566300357 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7f4415eac0 0x7f7f440b3420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:46.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.800+0000 7f7f4bd37700 1 -- 172.21.15.143:0/1566300357 >> 172.21.15.143:0/1566300357 conn(0x7f7f440b23d0 msgr2=0x7f7f440cca80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:46.804 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.800+0000 7f7f4bd37700 1 -- 172.21.15.143:0/1566300357 shutdown_connections 2023-11-08T19:30:46.804 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.800+0000 7f7f4bd37700 1 -- 172.21.15.143:0/1566300357 wait complete. 2023-11-08T19:30:46.804 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 94 2023-11-08T19:30:46.813 DEBUG:tasks.cephfs.filesystem:Looked up MDS state for mds.0: up:resolve 2023-11-08T19:30:46.814 DEBUG:tasks.cephfs.filesystem:reached state 'up:resolve' in 0.3885037899017334s 2023-11-08T19:30:46.814 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:30:46.934 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.933+0000 7fc119c33700 1 Processor -- start 2023-11-08T19:30:46.935 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.933+0000 7fc119c33700 1 -- start start 2023-11-08T19:30:46.937 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.934+0000 7fc119c33700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc11415bb20 0x7fc1140ccb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:46.937 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.934+0000 7fc119c33700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc11406d750 0x7fc11406abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:46.938 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.934+0000 7fc119c33700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc11406dbd0 con 0x7fc11406b120 2023-11-08T19:30:46.938 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.934+0000 7fc119c33700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc11406fef0 con 0x7fc11415bb20 2023-11-08T19:30:46.938 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.934+0000 7fc119c33700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc11406de50 con 0x7fc11406d750 2023-11-08T19:30:46.938 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.934+0000 7fc1137fe700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc11415bb20 0x7fc1140ccb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:46.939 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.934+0000 7fc113fff700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fc11406b120 0x7fc1140cd1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:52634/0 (socket says 172.21.15.143:52634) 2023-11-08T19:30:46.939 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.934+0000 7fc113fff700 1 -- 172.21.15.143:0/958388855 learned_addr learned my addr 172.21.15.143:0/958388855 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:30:46.939 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.934+0000 7fc112ffd700 1 --2- 172.21.15.143:0/958388855 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc11406d750 0x7fc11406abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:46.939 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.934+0000 7fc1137fe700 1 -- 172.21.15.143:0/958388855 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc11406d750 msgr2=0x7fc11406abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:46.939 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.934+0000 7fc1137fe700 1 --2- 172.21.15.143:0/958388855 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc11406d750 0x7fc11406abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:46.940 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.934+0000 7fc1137fe700 1 -- 172.21.15.143:0/958388855 >> v1:172.21.15.143:6789/0 conn(0x7fc11406b120 legacy=0x7fc1140cd1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:46.940 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.934+0000 7fc1137fe700 1 -- 172.21.15.143:0/958388855 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc11406f550 con 0x7fc11415bb20 2023-11-08T19:30:46.940 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.934+0000 7fc1127fc700 1 -- 172.21.15.143:0/958388855 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 536069868 0 0) 0x7fc11406dbd0 con 0x7fc11406b120 2023-11-08T19:30:46.940 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.934+0000 7fc112ffd700 1 --2- 172.21.15.143:0/958388855 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc11406d750 0x7fc11406abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:30:46.940 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.934+0000 7fc1137fe700 1 --2- 172.21.15.143:0/958388855 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc11415bb20 0x7fc1140ccb60 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7fc108009840 tx=0x7fc108009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7afcd341ed319d7a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:46.941 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.934+0000 7fc1127fc700 1 -- 172.21.15.143:0/958388855 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc108011830 con 0x7fc11415bb20 2023-11-08T19:30:46.941 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.935+0000 7fc1127fc700 1 -- 172.21.15.143:0/958388855 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc108011990 con 0x7fc11415bb20 2023-11-08T19:30:46.941 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.935+0000 7fc1127fc700 1 -- 172.21.15.143:0/958388855 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc108011d20 con 0x7fc11415bb20 2023-11-08T19:30:46.941 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.935+0000 7fc119c33700 1 -- 172.21.15.143:0/958388855 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc11415bb20 msgr2=0x7fc1140ccb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:46.941 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.935+0000 7fc119c33700 1 --2- 172.21.15.143:0/958388855 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc11415bb20 0x7fc1140ccb60 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7fc108009840 tx=0x7fc108009b50 comp rx=0 tx=0).stop 2023-11-08T19:30:46.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.935+0000 7fc119c33700 1 -- 172.21.15.143:0/958388855 shutdown_connections 2023-11-08T19:30:46.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.935+0000 7fc119c33700 1 --2- 172.21.15.143:0/958388855 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc11415bb20 0x7fc1140ccb60 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:46.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.935+0000 7fc119c33700 1 --2- 172.21.15.143:0/958388855 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc11406d750 0x7fc11406abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:46.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.935+0000 7fc119c33700 1 -- 172.21.15.143:0/958388855 >> 172.21.15.143:0/958388855 conn(0x7fc1140b23d0 msgr2=0x7fc1140b27b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:46.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.935+0000 7fc119c33700 1 -- 172.21.15.143:0/958388855 shutdown_connections 2023-11-08T19:30:46.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.935+0000 7fc119c33700 1 -- 172.21.15.143:0/958388855 wait complete. 2023-11-08T19:30:46.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.936+0000 7fc119c33700 1 Processor -- start 2023-11-08T19:30:46.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.936+0000 7fc119c33700 1 -- start start 2023-11-08T19:30:46.946 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.937+0000 7fc119c33700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc11406b120 0x7fc1140ca220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:46.946 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.937+0000 7fc119c33700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc11406d750 0x7fc1140b18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:46.946 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.937+0000 7fc119c33700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc11415bb20 0x7fc114159ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:46.946 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.937+0000 7fc119c33700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1140b1740 con 0x7fc11406d750 2023-11-08T19:30:46.947 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.937+0000 7fc119c33700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1140b1e20 con 0x7fc11406b120 2023-11-08T19:30:46.947 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.937+0000 7fc119c33700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc11415d0f0 con 0x7fc11415bb20 2023-11-08T19:30:46.947 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.937+0000 7fc112ffd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc11406d750 0x7fc1140b18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:46.947 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.937+0000 7fc112ffd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc11406d750 0x7fc1140b18e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56484/0 (socket says 172.21.15.143:56484) 2023-11-08T19:30:46.947 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.937+0000 7fc112ffd700 1 -- 172.21.15.143:0/3752802401 learned_addr learned my addr 172.21.15.143:0/3752802401 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:30:46.948 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.937+0000 7fc1137fe700 1 --2- 172.21.15.143:0/3752802401 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc11406b120 0x7fc1140ca220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:46.948 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.937+0000 7fc112ffd700 1 -- 172.21.15.143:0/3752802401 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc11415bb20 msgr2=0x7fc114159ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:46.948 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.937+0000 7fc113fff700 1 --2- 172.21.15.143:0/3752802401 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc11415bb20 0x7fc114159ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:46.948 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.937+0000 7fc112ffd700 1 --2- 172.21.15.143:0/3752802401 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc11415bb20 0x7fc114159ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:46.948 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.937+0000 7fc112ffd700 1 -- 172.21.15.143:0/3752802401 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc11406b120 msgr2=0x7fc1140ca220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:46.949 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.937+0000 7fc112ffd700 1 --2- 172.21.15.143:0/3752802401 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc11406b120 0x7fc1140ca220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:46.949 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.937+0000 7fc112ffd700 1 -- 172.21.15.143:0/3752802401 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1080094a0 con 0x7fc11406d750 2023-11-08T19:30:46.949 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.937+0000 7fc113fff700 1 --2- 172.21.15.143:0/3752802401 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc11415bb20 0x7fc114159ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:30:46.949 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.937+0000 7fc1137fe700 1 --2- 172.21.15.143:0/3752802401 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc11406b120 0x7fc1140ca220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:30:46.949 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.938+0000 7fc112ffd700 1 --2- 172.21.15.143:0/3752802401 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc11406d750 0x7fc1140b18e0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7fc10400c780 tx=0x7fc10400cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:46.949 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.938+0000 7fc0fbfff700 1 -- 172.21.15.143:0/3752802401 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc104011800 con 0x7fc11406d750 2023-11-08T19:30:46.950 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.938+0000 7fc0fbfff700 1 -- 172.21.15.143:0/3752802401 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc104011960 con 0x7fc11406d750 2023-11-08T19:30:46.950 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.938+0000 7fc119c33700 1 -- 172.21.15.143:0/3752802401 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc11415d3d0 con 0x7fc11406d750 2023-11-08T19:30:46.950 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.938+0000 7fc0fbfff700 1 -- 172.21.15.143:0/3752802401 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc104011cd0 con 0x7fc11406d750 2023-11-08T19:30:46.950 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.938+0000 7fc119c33700 1 -- 172.21.15.143:0/3752802401 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc11415d990 con 0x7fc11406d750 2023-11-08T19:30:46.950 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.939+0000 7fc119c33700 1 -- 172.21.15.143:0/3752802401 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc11404f730 con 0x7fc11406d750 2023-11-08T19:30:46.951 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.939+0000 7fc0fbfff700 1 -- 172.21.15.143:0/3752802401 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc104023480 con 0x7fc11406d750 2023-11-08T19:30:46.951 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.940+0000 7fc0fbfff700 1 --2- 172.21.15.143:0/3752802401 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc10006af00 0x7fc10006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:46.951 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.940+0000 7fc0fbfff700 1 -- 172.21.15.143:0/3752802401 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 7231+0+0 (secure 0 0 0) 0x7fc104089f70 con 0x7fc11406d750 2023-11-08T19:30:46.951 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.940+0000 7fc1137fe700 1 --2- 172.21.15.143:0/3752802401 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc10006af00 0x7fc10006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:46.951 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.940+0000 7fc1137fe700 1 --2- 172.21.15.143:0/3752802401 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc10006af00 0x7fc10006d3c0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fc1080090b0 tx=0x7fc10801c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:46.952 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:46.944+0000 7fc0fbfff700 1 -- 172.21.15.143:0/3752802401 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc104054ac0 con 0x7fc11406d750 2023-11-08T19:30:47.178 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.177+0000 7fc119c33700 1 -- 172.21.15.143:0/3752802401 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc1140cdb50 con 0x7fc11406d750 2023-11-08T19:30:47.179 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.177+0000 7fc0fbfff700 1 -- 172.21.15.143:0/3752802401 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) v1 ==== 94+0+4944 (secure 0 0 0) 0x7fc10401e090 con 0x7fc11406d750 2023-11-08T19:30:47.179 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:30:47.179 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":94,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5933,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/3918286710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6835","nonce":3918286710},{"type":"v1","addr":"172.21.15.174:6837","nonce":3918286710}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":7743,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6834/2489252684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":2489252684},{"type":"v1","addr":"172.21.15.143:6834","nonce":2489252684}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:03.092245+0000","modified":"2023-11-08T19:30:40.652694+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":102,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6161,"mds_1":6918},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6161":{"gid":6161,"name":"a","rank":0,"incarnation":93,"state":"up:resolve","state_seq":8,"addr":"172.21.15.143:6835/2129643800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":2129643800},{"type":"v1","addr":"172.21.15.143:6835","nonce":2129643800}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6918":{"gid":6918,"name":"b","rank":1,"incarnation":54,"state":"up:active","state_seq":6,"addr":"172.21.15.174:6836/3601088618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3601088618},{"type":"v1","addr":"172.21.15.174:6836","nonce":3601088618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:30:47.181 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.179+0000 7fc119c33700 1 -- 172.21.15.143:0/3752802401 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc10006af00 msgr2=0x7fc10006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:47.182 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.179+0000 7fc119c33700 1 --2- 172.21.15.143:0/3752802401 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc10006af00 0x7fc10006d3c0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fc1080090b0 tx=0x7fc10801c040 comp rx=0 tx=0).stop 2023-11-08T19:30:47.182 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.179+0000 7fc119c33700 1 -- 172.21.15.143:0/3752802401 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc11406d750 msgr2=0x7fc1140b18e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:47.182 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.179+0000 7fc119c33700 1 --2- 172.21.15.143:0/3752802401 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc11406d750 0x7fc1140b18e0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7fc10400c780 tx=0x7fc10400cb40 comp rx=0 tx=0).stop 2023-11-08T19:30:47.182 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.179+0000 7fc119c33700 1 -- 172.21.15.143:0/3752802401 shutdown_connections 2023-11-08T19:30:47.183 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.179+0000 7fc119c33700 1 --2- 172.21.15.143:0/3752802401 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc10006af00 0x7fc10006d3c0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:47.183 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.179+0000 7fc119c33700 1 --2- 172.21.15.143:0/3752802401 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc11415bb20 0x7fc114159ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:47.183 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.179+0000 7fc119c33700 1 --2- 172.21.15.143:0/3752802401 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc11406b120 0x7fc1140ca220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:47.183 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.179+0000 7fc119c33700 1 --2- 172.21.15.143:0/3752802401 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc11406d750 0x7fc1140b18e0 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:47.183 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.179+0000 7fc119c33700 1 -- 172.21.15.143:0/3752802401 >> 172.21.15.143:0/3752802401 conn(0x7fc1140b23d0 msgr2=0x7fc1140ce560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:47.184 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.180+0000 7fc119c33700 1 -- 172.21.15.143:0/3752802401 shutdown_connections 2023-11-08T19:30:47.184 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.180+0000 7fc119c33700 1 -- 172.21.15.143:0/3752802401 wait complete. 2023-11-08T19:30:47.184 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 94 2023-11-08T19:30:47.193 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json dump snaps --server 2023-11-08T19:30:47.329 INFO:teuthology.orchestra.run.smithi143.stdout:{"last_snap":10,"last_created":5,"last_destroyed":9,"pending_noop":[],"snaps":[],"need_to_purge":{},"pending_update":[{"version":21,"snapinfo":{"snapid":10,"ino":1099511627777,"stamp":"2023-11-08T19:30:22.821247+0000","name":"s22","metadata":{}}}],"pending_destroy":[]} 2023-11-08T19:30:47.330 DEBUG:tasks.cephfs.filesystem:_json_asok output { "last_created": 5, "last_destroyed": 9, "last_snap": 10, "need_to_purge": {}, "pending_destroy": [], "pending_noop": [], "pending_update": [ { "snapInfinityo": { "ino": 1099511627777, "metadata": {}, "name": "s22", "snapid": 10, "stamp": "2023-11-08T19:30:22.821247+0000" }, "version": 21 } ], "snaps": [] } 2023-11-08T19:30:47.330 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail 6:1 2023-11-08T19:30:47.452 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.450+0000 7fe63ca80700 1 Processor -- start 2023-11-08T19:30:47.452 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.450+0000 7fe63ca80700 1 -- start start 2023-11-08T19:30:47.455 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.451+0000 7fe63ca80700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe63815eac0 0x7fe63806abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:47.455 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.451+0000 7fe63ca80700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe63815ef70 0x7fe63806b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:47.455 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.451+0000 7fe63ca80700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6380cb640 con 0x7fe63815b900 2023-11-08T19:30:47.455 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.451+0000 7fe63ca80700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe638070120 con 0x7fe63815eac0 2023-11-08T19:30:47.456 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.451+0000 7fe63ca80700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6380d0500 con 0x7fe63815ef70 2023-11-08T19:30:47.456 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.452+0000 7fe635d9b700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe63815eac0 0x7fe63806abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:47.456 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.452+0000 7fe635d9b700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe63815eac0 0x7fe63806abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56494/0 (socket says 172.21.15.143:56494) 2023-11-08T19:30:47.456 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.452+0000 7fe635d9b700 1 -- 172.21.15.143:0/3835474519 learned_addr learned my addr 172.21.15.143:0/3835474519 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:30:47.456 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.452+0000 7fe636d9d700 1 --2- 172.21.15.143:0/3835474519 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe63815ef70 0x7fe63806b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:47.457 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.452+0000 7fe635d9b700 1 -- 172.21.15.143:0/3835474519 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe63815ef70 msgr2=0x7fe63806b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:47.457 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.452+0000 7fe635d9b700 1 --2- 172.21.15.143:0/3835474519 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe63815ef70 0x7fe63806b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:47.457 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.452+0000 7fe635d9b700 1 -- 172.21.15.143:0/3835474519 >> v1:172.21.15.174:6789/0 conn(0x7fe63815b900 legacy=0x7fe6380ce3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:47.457 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.452+0000 7fe635d9b700 1 -- 172.21.15.143:0/3835474519 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe63806fe40 con 0x7fe63815eac0 2023-11-08T19:30:47.457 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.452+0000 7fe635d9b700 1 --2- 172.21.15.143:0/3835474519 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe63815eac0 0x7fe63806abe0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7fe6280098f0 tx=0x7fe628009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a494f97bced3d8f server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:47.458 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.452+0000 7fe63559a700 1 -- 172.21.15.143:0/3835474519 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe628004230 con 0x7fe63815eac0 2023-11-08T19:30:47.458 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.452+0000 7fe63559a700 1 -- 172.21.15.143:0/3835474519 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe628004390 con 0x7fe63815eac0 2023-11-08T19:30:47.458 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.452+0000 7fe63559a700 1 -- 172.21.15.143:0/3835474519 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6280036a0 con 0x7fe63815eac0 2023-11-08T19:30:47.458 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.453+0000 7fe63ca80700 1 -- 172.21.15.143:0/3835474519 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe63815eac0 msgr2=0x7fe63806abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:47.459 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.453+0000 7fe63ca80700 1 --2- 172.21.15.143:0/3835474519 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe63815eac0 0x7fe63806abe0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7fe6280098f0 tx=0x7fe628009c00 comp rx=0 tx=0).stop 2023-11-08T19:30:47.459 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.453+0000 7fe63ca80700 1 -- 172.21.15.143:0/3835474519 shutdown_connections 2023-11-08T19:30:47.459 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.453+0000 7fe63ca80700 1 --2- 172.21.15.143:0/3835474519 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe63815ef70 0x7fe63806b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:47.459 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.453+0000 7fe63ca80700 1 --2- 172.21.15.143:0/3835474519 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe63815eac0 0x7fe63806abe0 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:47.459 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.453+0000 7fe63ca80700 1 -- 172.21.15.143:0/3835474519 >> 172.21.15.143:0/3835474519 conn(0x7fe6380b23d0 msgr2=0x7fe6380b27b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:47.459 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.453+0000 7fe63ca80700 1 -- 172.21.15.143:0/3835474519 shutdown_connections 2023-11-08T19:30:47.460 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.453+0000 7fe63ca80700 1 -- 172.21.15.143:0/3835474519 wait complete. 2023-11-08T19:30:47.462 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.454+0000 7fe63ca80700 1 Processor -- start 2023-11-08T19:30:47.463 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.454+0000 7fe63ca80700 1 -- start start 2023-11-08T19:30:47.463 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.455+0000 7fe63ca80700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe63815b900 0x7fe63815ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:47.463 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.455+0000 7fe63ca80700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe63815eac0 0x7fe6381f4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:47.463 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.455+0000 7fe63ca80700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe63815ef70 0x7fe6380c8c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:47.464 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.455+0000 7fe63ca80700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6380c8810 con 0x7fe63815b900 2023-11-08T19:30:47.464 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.455+0000 7fe63ca80700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6380c7b90 con 0x7fe63815ef70 2023-11-08T19:30:47.464 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.455+0000 7fe63ca80700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6380c7e70 con 0x7fe63815eac0 2023-11-08T19:30:47.464 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.455+0000 7fe63659c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe63815b900 0x7fe63815ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:47.464 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.455+0000 7fe63659c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe63815b900 0x7fe63815ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56498/0 (socket says 172.21.15.143:56498) 2023-11-08T19:30:47.465 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.455+0000 7fe63659c700 1 -- 172.21.15.143:0/2981086450 learned_addr learned my addr 172.21.15.143:0/2981086450 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:30:47.465 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.455+0000 7fe635d9b700 1 --2- 172.21.15.143:0/2981086450 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe63815eac0 0x7fe6381f4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:47.465 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.455+0000 7fe636d9d700 1 --2- 172.21.15.143:0/2981086450 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe63815ef70 0x7fe6380c8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:47.465 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.455+0000 7fe63659c700 1 -- 172.21.15.143:0/2981086450 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe63815eac0 msgr2=0x7fe6381f4cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:47.465 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.455+0000 7fe63659c700 1 --2- 172.21.15.143:0/2981086450 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe63815eac0 0x7fe6381f4cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:47.466 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.455+0000 7fe63659c700 1 -- 172.21.15.143:0/2981086450 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe63815ef70 msgr2=0x7fe6380c8c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:47.466 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.455+0000 7fe63659c700 1 --2- 172.21.15.143:0/2981086450 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe63815ef70 0x7fe6380c8c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:47.466 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.455+0000 7fe63659c700 1 -- 172.21.15.143:0/2981086450 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe628009550 con 0x7fe63815b900 2023-11-08T19:30:47.466 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.455+0000 7fe63659c700 1 --2- 172.21.15.143:0/2981086450 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe63815b900 0x7fe63815ae20 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7fe62c00b4a0 tx=0x7fe62c00b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:47.466 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.456+0000 7fe626ffd700 1 -- 172.21.15.143:0/2981086450 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe62c011830 con 0x7fe63815b900 2023-11-08T19:30:47.467 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.456+0000 7fe626ffd700 1 -- 172.21.15.143:0/2981086450 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe62c011990 con 0x7fe63815b900 2023-11-08T19:30:47.467 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.456+0000 7fe626ffd700 1 -- 172.21.15.143:0/2981086450 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe62c011d40 con 0x7fe63815b900 2023-11-08T19:30:47.467 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.456+0000 7fe63ca80700 1 -- 172.21.15.143:0/2981086450 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6380c8150 con 0x7fe63815b900 2023-11-08T19:30:47.467 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.456+0000 7fe63ca80700 1 -- 172.21.15.143:0/2981086450 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6381f5210 con 0x7fe63815b900 2023-11-08T19:30:47.467 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.457+0000 7fe63ca80700 1 -- 172.21.15.143:0/2981086450 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe63804f730 con 0x7fe63815b900 2023-11-08T19:30:47.468 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.457+0000 7fe626ffd700 1 -- 172.21.15.143:0/2981086450 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe62c01a400 con 0x7fe63815b900 2023-11-08T19:30:47.468 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.457+0000 7fe626ffd700 1 --2- 172.21.15.143:0/2981086450 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fe61806af00 0x7fe61806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:47.468 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.458+0000 7fe626ffd700 1 -- 172.21.15.143:0/2981086450 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 7231+0+0 (secure 0 0 0) 0x7fe62c0898d0 con 0x7fe63815b900 2023-11-08T19:30:47.468 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.458+0000 7fe635d9b700 1 --2- 172.21.15.143:0/2981086450 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fe61806af00 0x7fe61806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:47.468 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.458+0000 7fe635d9b700 1 --2- 172.21.15.143:0/2981086450 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fe61806af00 0x7fe61806d3c0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fe628009130 tx=0x7fe628008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:47.469 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.462+0000 7fe626ffd700 1 -- 172.21.15.143:0/2981086450 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe62c054390 con 0x7fe63815b900 2023-11-08T19:30:47.736 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.735+0000 7fe63ca80700 1 -- 172.21.15.143:0/2981086450 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "6:1"} v 0) v1 -- 0x7fe638072300 con 0x7fe63815b900 2023-11-08T19:30:47.738 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.737+0000 7fe626ffd700 1 -- 172.21.15.143:0/2981086450 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "6:1"}]=0 failed mds gid 6918 v95) v1 ==== 97+0+0 (secure 0 0 0) 0x7fe62c0579b0 con 0x7fe63815b900 2023-11-08T19:30:47.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.739+0000 7fe63ca80700 1 -- 172.21.15.143:0/2981086450 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fe61806af00 msgr2=0x7fe61806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:47.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.739+0000 7fe63ca80700 1 --2- 172.21.15.143:0/2981086450 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fe61806af00 0x7fe61806d3c0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fe628009130 tx=0x7fe628008030 comp rx=0 tx=0).stop 2023-11-08T19:30:47.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.739+0000 7fe63ca80700 1 -- 172.21.15.143:0/2981086450 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe63815b900 msgr2=0x7fe63815ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:47.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.739+0000 7fe63ca80700 1 --2- 172.21.15.143:0/2981086450 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe63815b900 0x7fe63815ae20 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7fe62c00b4a0 tx=0x7fe62c00b860 comp rx=0 tx=0).stop 2023-11-08T19:30:47.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.740+0000 7fe63ca80700 1 -- 172.21.15.143:0/2981086450 shutdown_connections 2023-11-08T19:30:47.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.740+0000 7fe63ca80700 1 --2- 172.21.15.143:0/2981086450 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fe61806af00 0x7fe61806d3c0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:47.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.740+0000 7fe63ca80700 1 --2- 172.21.15.143:0/2981086450 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe63815ef70 0x7fe6380c8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:47.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.740+0000 7fe63ca80700 1 --2- 172.21.15.143:0/2981086450 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe63815b900 0x7fe63815ae20 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:47.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.740+0000 7fe63ca80700 1 --2- 172.21.15.143:0/2981086450 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe63815eac0 0x7fe6381f4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:47.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.740+0000 7fe63ca80700 1 -- 172.21.15.143:0/2981086450 >> 172.21.15.143:0/2981086450 conn(0x7fe6380b23d0 msgr2=0x7fe6380716b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:47.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.740+0000 7fe63ca80700 1 -- 172.21.15.143:0/2981086450 shutdown_connections 2023-11-08T19:30:47.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.740+0000 7fe63ca80700 1 -- 172.21.15.143:0/2981086450 wait complete. 2023-11-08T19:30:47.744 INFO:teuthology.orchestra.run.smithi143.stderr:failed mds gid 6918 2023-11-08T19:30:47.755 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-08T19:30:47.756 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-08T19:30:47.756 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-08T19:30:47.757 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T19:30:47.757 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-08T19:30:47.757 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi174 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b' 2023-11-08T19:30:47.758 INFO:tasks.ceph.mds.b:Stopped 2023-11-08T19:30:47.758 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-08T19:30:47.761 INFO:tasks.ceph.mds.b:Started 2023-11-08T19:30:47.761 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:30:47.846 INFO:tasks.ceph.mds.b.smithi174.stdout:starting mds.b at 2023-11-08T19:30:47.883 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.882+0000 7fabecbb6700 1 Processor -- start 2023-11-08T19:30:47.883 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.882+0000 7fabecbb6700 1 -- start start 2023-11-08T19:30:47.886 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.882+0000 7fabecbb6700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fabe806bbf0 0x7fabe806bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:47.886 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.882+0000 7fabecbb6700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fabe80b3840 0x7fabe81617c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:47.886 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.883+0000 7fabecbb6700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabe81586c0 con 0x7fabe8066b00 2023-11-08T19:30:47.886 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.883+0000 7fabecbb6700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabe806c510 con 0x7fabe806bbf0 2023-11-08T19:30:47.886 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.883+0000 7fabecbb6700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabe815e370 con 0x7fabe80b3840 2023-11-08T19:30:47.887 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.883+0000 7fabe659c700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fabe806bbf0 0x7fabe806bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:47.887 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.883+0000 7fabe6d9d700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fabe8066b00 0x7fabe8161d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:52674/0 (socket says 172.21.15.143:52674) 2023-11-08T19:30:47.887 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.883+0000 7fabe6d9d700 1 -- 172.21.15.143:0/1366256255 learned_addr learned my addr 172.21.15.143:0/1366256255 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:30:47.887 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.883+0000 7fabe5d9b700 1 --2- 172.21.15.143:0/1366256255 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fabe80b3840 0x7fabe81617c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:47.888 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.883+0000 7fabe559a700 1 -- 172.21.15.143:0/1366256255 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1657652897 0 0) 0x7fabe81586c0 con 0x7fabe8066b00 2023-11-08T19:30:47.888 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.883+0000 7fabe559a700 1 -- 172.21.15.143:0/1366256255 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fabd40035b0 con 0x7fabe8066b00 2023-11-08T19:30:47.888 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.883+0000 7fabe659c700 1 -- 172.21.15.143:0/1366256255 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fabe80b3840 msgr2=0x7fabe81617c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:47.888 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.883+0000 7fabe659c700 1 --2- 172.21.15.143:0/1366256255 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fabe80b3840 0x7fabe81617c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:47.888 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.883+0000 7fabe659c700 1 -- 172.21.15.143:0/1366256255 >> v1:172.21.15.143:6789/0 conn(0x7fabe8066b00 legacy=0x7fabe8161d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:47.889 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.883+0000 7fabe659c700 1 -- 172.21.15.143:0/1366256255 --> v2:172.21.15.174:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabe815e650 con 0x7fabe806bbf0 2023-11-08T19:30:47.889 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.883+0000 7fabe659c700 1 --2- 172.21.15.143:0/1366256255 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fabe806bbf0 0x7fabe806bfd0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fabd0009840 tx=0x7fabd0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bfa63996f587bc6c server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:47.889 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.883+0000 7fabe559a700 1 -- 172.21.15.143:0/1366256255 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd0011830 con 0x7fabe806bbf0 2023-11-08T19:30:47.889 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.883+0000 7fabe559a700 1 -- 172.21.15.143:0/1366256255 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabd0011990 con 0x7fabe806bbf0 2023-11-08T19:30:47.889 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.883+0000 7fabe559a700 1 -- 172.21.15.143:0/1366256255 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd0011d20 con 0x7fabe806bbf0 2023-11-08T19:30:47.889 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.884+0000 7fabecbb6700 1 -- 172.21.15.143:0/1366256255 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fabe806bbf0 msgr2=0x7fabe806bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:47.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.884+0000 7fabecbb6700 1 --2- 172.21.15.143:0/1366256255 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fabe806bbf0 0x7fabe806bfd0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fabd0009840 tx=0x7fabd0009b50 comp rx=0 tx=0).stop 2023-11-08T19:30:47.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.884+0000 7fabecbb6700 1 -- 172.21.15.143:0/1366256255 shutdown_connections 2023-11-08T19:30:47.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.884+0000 7fabecbb6700 1 --2- 172.21.15.143:0/1366256255 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fabe806bbf0 0x7fabe806bfd0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:47.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.884+0000 7fabecbb6700 1 --2- 172.21.15.143:0/1366256255 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fabe80b3840 0x7fabe81617c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:47.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.884+0000 7fabecbb6700 1 -- 172.21.15.143:0/1366256255 >> 172.21.15.143:0/1366256255 conn(0x7fabe806e260 msgr2=0x7fabe806e670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:47.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.884+0000 7fabecbb6700 1 -- 172.21.15.143:0/1366256255 shutdown_connections 2023-11-08T19:30:47.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.884+0000 7fabecbb6700 1 -- 172.21.15.143:0/1366256255 wait complete. 2023-11-08T19:30:47.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.886+0000 7fabecbb6700 1 Processor -- start 2023-11-08T19:30:47.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.886+0000 7fabecbb6700 1 -- start start 2023-11-08T19:30:47.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.887+0000 7fabecbb6700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fabe8066b00 0x7fabe81561e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:47.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.887+0000 7fabecbb6700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fabe806bbf0 0x7fabe8156720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:47.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.887+0000 7fabecbb6700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fabe8146780 0x7fabe8146c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:47.895 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.887+0000 7fabecbb6700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabe8144430 con 0x7fabe8146780 2023-11-08T19:30:47.895 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.887+0000 7fabecbb6700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabe8157240 con 0x7fabe806bbf0 2023-11-08T19:30:47.895 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.887+0000 7fabecbb6700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabe8156f60 con 0x7fabe8066b00 2023-11-08T19:30:47.895 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.887+0000 7fabe6d9d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fabe8146780 0x7fabe8146c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:47.895 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.887+0000 7fabe6d9d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fabe8146780 0x7fabe8146c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56526/0 (socket says 172.21.15.143:56526) 2023-11-08T19:30:47.896 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.887+0000 7fabe6d9d700 1 -- 172.21.15.143:0/2061168942 learned_addr learned my addr 172.21.15.143:0/2061168942 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:30:47.896 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.887+0000 7fabe5d9b700 1 --2- 172.21.15.143:0/2061168942 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fabe806bbf0 0x7fabe8156720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:47.896 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.887+0000 7fabe659c700 1 --2- 172.21.15.143:0/2061168942 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fabe8066b00 0x7fabe81561e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:47.896 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.887+0000 7fabe6d9d700 1 -- 172.21.15.143:0/2061168942 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fabe8066b00 msgr2=0x7fabe81561e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:47.896 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.887+0000 7fabe6d9d700 1 --2- 172.21.15.143:0/2061168942 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fabe8066b00 0x7fabe81561e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:47.897 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.887+0000 7fabe6d9d700 1 -- 172.21.15.143:0/2061168942 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fabe806bbf0 msgr2=0x7fabe8156720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:47.897 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.887+0000 7fabe6d9d700 1 --2- 172.21.15.143:0/2061168942 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fabe806bbf0 0x7fabe8156720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:47.897 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.887+0000 7fabe6d9d700 1 -- 172.21.15.143:0/2061168942 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabd00094a0 con 0x7fabe8146780 2023-11-08T19:30:47.897 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.887+0000 7fabe6d9d700 1 --2- 172.21.15.143:0/2061168942 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fabe8146780 0x7fabe8146c30 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7fabe000cb40 tx=0x7fabe000ce50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:47.897 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.887+0000 7fabdeffd700 1 -- 172.21.15.143:0/2061168942 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabe000a9e0 con 0x7fabe8146780 2023-11-08T19:30:47.898 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.887+0000 7fabdeffd700 1 -- 172.21.15.143:0/2061168942 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabe000ab40 con 0x7fabe8146780 2023-11-08T19:30:47.898 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.887+0000 7fabecbb6700 1 -- 172.21.15.143:0/2061168942 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabe8144860 con 0x7fabe8146780 2023-11-08T19:30:47.898 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.887+0000 7fabdeffd700 1 -- 172.21.15.143:0/2061168942 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabe0012400 con 0x7fabe8146780 2023-11-08T19:30:47.898 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.887+0000 7fabecbb6700 1 -- 172.21.15.143:0/2061168942 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabe8144e20 con 0x7fabe8146780 2023-11-08T19:30:47.898 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.888+0000 7fabecbb6700 1 -- 172.21.15.143:0/2061168942 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabe8145040 con 0x7fabe8146780 2023-11-08T19:30:47.899 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.889+0000 7fabdeffd700 1 -- 172.21.15.143:0/2061168942 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fabe001b3b0 con 0x7fabe8146780 2023-11-08T19:30:47.899 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.889+0000 7fabdeffd700 1 --2- 172.21.15.143:0/2061168942 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fabc806af00 0x7fabc806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:47.899 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.889+0000 7fabdeffd700 1 -- 172.21.15.143:0/2061168942 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 7317+0+0 (secure 0 0 0) 0x7fabe00896c0 con 0x7fabe8146780 2023-11-08T19:30:47.899 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.889+0000 7fabe659c700 1 --2- 172.21.15.143:0/2061168942 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fabc806af00 0x7fabc806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:47.899 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.890+0000 7fabe659c700 1 --2- 172.21.15.143:0/2061168942 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fabc806af00 0x7fabc806d3c0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fabd0000c00 tx=0x7fabd0008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:47.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:47.894+0000 7fabdeffd700 1 -- 172.21.15.143:0/2061168942 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fabe0054260 con 0x7fabe8146780 2023-11-08T19:30:48.127 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:48.126+0000 7fabecbb6700 1 -- 172.21.15.143:0/2061168942 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fabe804f730 con 0x7fabe8146780 2023-11-08T19:30:48.128 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:48.126+0000 7fabdeffd700 1 -- 172.21.15.143:0/2061168942 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) v1 ==== 94+0+4150 (secure 0 0 0) 0x7fabe0057880 con 0x7fabe8146780 2023-11-08T19:30:48.128 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:30:48.128 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":96,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5933,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/3918286710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6835","nonce":3918286710},{"type":"v1","addr":"172.21.15.174:6837","nonce":3918286710}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:03.092245+0000","modified":"2023-11-08T19:30:47.741127+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":103,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6161,"mds_1":7743},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6161":{"gid":6161,"name":"a","rank":0,"incarnation":93,"state":"up:resolve","state_seq":8,"addr":"172.21.15.143:6835/2129643800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":2129643800},{"type":"v1","addr":"172.21.15.143:6835","nonce":2129643800}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7743":{"gid":7743,"name":"c","rank":1,"incarnation":96,"state":"up:replay","state_seq":1,"addr":"172.21.15.143:6834/2489252684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":2489252684},{"type":"v1","addr":"172.21.15.143:6834","nonce":2489252684}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:30:48.130 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:48.128+0000 7fabecbb6700 1 -- 172.21.15.143:0/2061168942 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fabc806af00 msgr2=0x7fabc806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:48.130 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:48.128+0000 7fabecbb6700 1 --2- 172.21.15.143:0/2061168942 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fabc806af00 0x7fabc806d3c0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fabd0000c00 tx=0x7fabd0008030 comp rx=0 tx=0).stop 2023-11-08T19:30:48.131 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:48.128+0000 7fabecbb6700 1 -- 172.21.15.143:0/2061168942 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fabe8146780 msgr2=0x7fabe8146c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:48.131 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:48.128+0000 7fabecbb6700 1 --2- 172.21.15.143:0/2061168942 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fabe8146780 0x7fabe8146c30 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7fabe000cb40 tx=0x7fabe000ce50 comp rx=0 tx=0).stop 2023-11-08T19:30:48.131 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:48.128+0000 7fabecbb6700 1 -- 172.21.15.143:0/2061168942 shutdown_connections 2023-11-08T19:30:48.131 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:48.128+0000 7fabecbb6700 1 --2- 172.21.15.143:0/2061168942 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fabc806af00 0x7fabc806d3c0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:48.131 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:48.128+0000 7fabecbb6700 1 --2- 172.21.15.143:0/2061168942 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fabe8066b00 0x7fabe81561e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:48.132 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:48.128+0000 7fabecbb6700 1 --2- 172.21.15.143:0/2061168942 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fabe8146780 0x7fabe8146c30 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:48.132 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:48.128+0000 7fabecbb6700 1 --2- 172.21.15.143:0/2061168942 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fabe806bbf0 0x7fabe8156720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:48.132 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:48.128+0000 7fabecbb6700 1 -- 172.21.15.143:0/2061168942 >> 172.21.15.143:0/2061168942 conn(0x7fabe806e260 msgr2=0x7fabe8070e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:48.132 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:48.129+0000 7fabecbb6700 1 -- 172.21.15.143:0/2061168942 shutdown_connections 2023-11-08T19:30:48.132 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:48.129+0000 7fabecbb6700 1 -- 172.21.15.143:0/2061168942 wait complete. 2023-11-08T19:30:48.133 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 96 2023-11-08T19:30:48.142 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=30 retry_count=0)... 2023-11-08T19:30:50.004 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2023-11-08T19:30:53.144 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:30:53.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.276+0000 7f576edef700 1 Processor -- start 2023-11-08T19:30:53.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.276+0000 7f576edef700 1 -- start start 2023-11-08T19:30:53.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.277+0000 7f576edef700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f57680cef70 0x7f57680b95e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:53.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.277+0000 7f576edef700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f57680d3390 0x7f57680d1b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:53.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.277+0000 7f576edef700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57680d20c0 con 0x7f576815dda0 2023-11-08T19:30:53.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.277+0000 7f576edef700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57680d3800 con 0x7f57680d3390 2023-11-08T19:30:53.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.277+0000 7f576edef700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f576815e180 con 0x7f57680cef70 2023-11-08T19:30:53.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.277+0000 7f576d38c700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f57680d3390 0x7f57680d1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:53.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.277+0000 7f576cb8b700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f576815dda0 0x7f57680ce860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:52690/0 (socket says 172.21.15.143:52690) 2023-11-08T19:30:53.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.277+0000 7f576cb8b700 1 -- 172.21.15.143:0/1086398021 learned_addr learned my addr 172.21.15.143:0/1086398021 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:30:53.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.277+0000 7f5767fff700 1 --2- 172.21.15.143:0/1086398021 >> v2:172.21.15.174:3300/0 conn(0x7f57680cef70 0x7f57680b95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:53.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.277+0000 7f576d38c700 1 -- 172.21.15.143:0/1086398021 >> v2:172.21.15.174:3300/0 conn(0x7f57680cef70 msgr2=0x7f57680b95e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:53.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.277+0000 7f576d38c700 1 --2- 172.21.15.143:0/1086398021 >> v2:172.21.15.174:3300/0 conn(0x7f57680cef70 0x7f57680b95e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:53.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.277+0000 7f576d38c700 1 -- 172.21.15.143:0/1086398021 >> v1:172.21.15.143:6789/0 conn(0x7f576815dda0 legacy=0x7f57680ce860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:53.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.277+0000 7f57677fe700 1 -- 172.21.15.143:0/1086398021 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 184969325 0 0) 0x7f57680d20c0 con 0x7f576815dda0 2023-11-08T19:30:53.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.277+0000 7f576d38c700 1 -- 172.21.15.143:0/1086398021 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57680d22f0 con 0x7f57680d3390 2023-11-08T19:30:53.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.277+0000 7f5767fff700 1 --2- 172.21.15.143:0/1086398021 >> v2:172.21.15.174:3300/0 conn(0x7f57680cef70 0x7f57680b95e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:30:53.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.277+0000 7f576d38c700 1 --2- 172.21.15.143:0/1086398021 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f57680d3390 0x7f57680d1b80 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f5768068160 tx=0x7f576000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7604a2f97cbe89ec server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:53.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.277+0000 7f57677fe700 1 -- 172.21.15.143:0/1086398021 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f576000e070 con 0x7f57680d3390 2023-11-08T19:30:53.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.277+0000 7f57677fe700 1 -- 172.21.15.143:0/1086398021 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f576000bbd0 con 0x7f57680d3390 2023-11-08T19:30:53.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.277+0000 7f57677fe700 1 -- 172.21.15.143:0/1086398021 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5760012430 con 0x7f57680d3390 2023-11-08T19:30:53.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.278+0000 7f576edef700 1 -- 172.21.15.143:0/1086398021 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f57680d3390 msgr2=0x7f57680d1b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:53.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.278+0000 7f576edef700 1 --2- 172.21.15.143:0/1086398021 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f57680d3390 0x7f57680d1b80 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f5768068160 tx=0x7f576000b330 comp rx=0 tx=0).stop 2023-11-08T19:30:53.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.278+0000 7f576edef700 1 -- 172.21.15.143:0/1086398021 shutdown_connections 2023-11-08T19:30:53.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.278+0000 7f576edef700 1 --2- 172.21.15.143:0/1086398021 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f57680d3390 0x7f57680d1b80 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:53.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.278+0000 7f576edef700 1 --2- 172.21.15.143:0/1086398021 >> v2:172.21.15.174:3300/0 conn(0x7f57680cef70 0x7f57680b95e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:53.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.278+0000 7f576edef700 1 -- 172.21.15.143:0/1086398021 >> 172.21.15.143:0/1086398021 conn(0x7f576806b000 msgr2=0x7f576806b410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:53.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.278+0000 7f576edef700 1 -- 172.21.15.143:0/1086398021 shutdown_connections 2023-11-08T19:30:53.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.278+0000 7f576edef700 1 -- 172.21.15.143:0/1086398021 wait complete. 2023-11-08T19:30:53.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.279+0000 7f576edef700 1 Processor -- start 2023-11-08T19:30:53.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.279+0000 7f576edef700 1 -- start start 2023-11-08T19:30:53.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.280+0000 7f576edef700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f57680cef70 0x7f57680b4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:53.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.280+0000 7f576edef700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f57680d3390 0x7f57681f2df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:53.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.280+0000 7f576edef700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f576815dda0 0x7f57680b80d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:53.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.280+0000 7f576edef700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57680b6410 con 0x7f576815dda0 2023-11-08T19:30:53.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.280+0000 7f576edef700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57680b5f80 con 0x7f57680d3390 2023-11-08T19:30:53.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.280+0000 7f576edef700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5768146680 con 0x7f57680cef70 2023-11-08T19:30:53.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.280+0000 7f576d38c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f576815dda0 0x7f57680b80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:53.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.280+0000 7f576d38c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f576815dda0 0x7f57680b80d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56546/0 (socket says 172.21.15.143:56546) 2023-11-08T19:30:53.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.280+0000 7f576d38c700 1 -- 172.21.15.143:0/811948394 learned_addr learned my addr 172.21.15.143:0/811948394 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:30:53.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.281+0000 7f5767fff700 1 --2- 172.21.15.143:0/811948394 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f57680d3390 0x7f57681f2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:53.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.281+0000 7f576cb8b700 1 --2- 172.21.15.143:0/811948394 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f57680cef70 0x7f57680b4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:53.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.281+0000 7f576d38c700 1 -- 172.21.15.143:0/811948394 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f57680cef70 msgr2=0x7f57680b4ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:53.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.281+0000 7f576d38c700 1 --2- 172.21.15.143:0/811948394 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f57680cef70 0x7f57680b4ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:53.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.281+0000 7f576d38c700 1 -- 172.21.15.143:0/811948394 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f57680d3390 msgr2=0x7f57681f2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:53.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.281+0000 7f576d38c700 1 --2- 172.21.15.143:0/811948394 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f57680d3390 0x7f57681f2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:53.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.281+0000 7f576d38c700 1 -- 172.21.15.143:0/811948394 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5760009a70 con 0x7f576815dda0 2023-11-08T19:30:53.293 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.281+0000 7f5767fff700 1 --2- 172.21.15.143:0/811948394 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f57680d3390 0x7f57681f2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:30:53.293 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.281+0000 7f576d38c700 1 --2- 172.21.15.143:0/811948394 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f576815dda0 0x7f57680b80d0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f576000b730 tx=0x7f5760004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:53.293 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.281+0000 7f57657fa700 1 -- 172.21.15.143:0/811948394 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5760012430 con 0x7f576815dda0 2023-11-08T19:30:53.293 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.281+0000 7f57657fa700 1 -- 172.21.15.143:0/811948394 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5760020070 con 0x7f576815dda0 2023-11-08T19:30:53.293 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.281+0000 7f576edef700 1 -- 172.21.15.143:0/811948394 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57680b5370 con 0x7f576815dda0 2023-11-08T19:30:53.294 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.281+0000 7f57657fa700 1 -- 172.21.15.143:0/811948394 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5760004280 con 0x7f576815dda0 2023-11-08T19:30:53.294 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.281+0000 7f576edef700 1 -- 172.21.15.143:0/811948394 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57680b5930 con 0x7f576815dda0 2023-11-08T19:30:53.294 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.282+0000 7f576edef700 1 -- 172.21.15.143:0/811948394 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f576804f730 con 0x7f576815dda0 2023-11-08T19:30:53.294 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.282+0000 7f57657fa700 1 -- 172.21.15.143:0/811948394 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f57600043e0 con 0x7f576815dda0 2023-11-08T19:30:53.294 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.283+0000 7f57657fa700 1 --2- 172.21.15.143:0/811948394 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f574c06af00 0x7f574c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:53.295 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.283+0000 7f57657fa700 1 -- 172.21.15.143:0/811948394 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 7317+0+0 (secure 0 0 0) 0x7f5760022070 con 0x7f576815dda0 2023-11-08T19:30:53.295 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.283+0000 7f576cb8b700 1 --2- 172.21.15.143:0/811948394 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f574c06af00 0x7f574c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:53.295 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.283+0000 7f576cb8b700 1 --2- 172.21.15.143:0/811948394 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f574c06af00 0x7f574c06d3c0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f575c000f60 tx=0x7f575c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:53.295 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.287+0000 7f57657fa700 1 -- 172.21.15.143:0/811948394 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f57600568c0 con 0x7f576815dda0 2023-11-08T19:30:53.519 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.518+0000 7f576edef700 1 -- 172.21.15.143:0/811948394 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f57680b60c0 con 0x7f576815dda0 2023-11-08T19:30:53.520 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.518+0000 7f57657fa700 1 -- 172.21.15.143:0/811948394 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 98 v98) v1 ==== 94+0+4950 (secure 0 0 0) 0x7f576001ae60 con 0x7f576815dda0 2023-11-08T19:30:53.520 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:30:53.520 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":98,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5933,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/3918286710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6835","nonce":3918286710},{"type":"v1","addr":"172.21.15.174:6837","nonce":3918286710}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":6245,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6836/2689884850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2689884850},{"type":"v1","addr":"172.21.15.174:6836","nonce":2689884850}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":97}],"filesystems":[{"mdsmap":{"epoch":98,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:03.092245+0000","modified":"2023-11-08T19:30:48.753530+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":103,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6161,"mds_1":7743},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6161":{"gid":6161,"name":"a","rank":0,"incarnation":93,"state":"up:reconnect","state_seq":11,"addr":"172.21.15.143:6835/2129643800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":2129643800},{"type":"v1","addr":"172.21.15.143:6835","nonce":2129643800}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7743":{"gid":7743,"name":"c","rank":1,"incarnation":96,"state":"up:reconnect","state_seq":4,"addr":"172.21.15.143:6834/2489252684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":2489252684},{"type":"v1","addr":"172.21.15.143:6834","nonce":2489252684}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:30:53.522 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.520+0000 7f576edef700 1 -- 172.21.15.143:0/811948394 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f574c06af00 msgr2=0x7f574c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:53.522 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.520+0000 7f576edef700 1 --2- 172.21.15.143:0/811948394 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f574c06af00 0x7f574c06d3c0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f575c000f60 tx=0x7f575c009040 comp rx=0 tx=0).stop 2023-11-08T19:30:53.522 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.520+0000 7f576edef700 1 -- 172.21.15.143:0/811948394 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f576815dda0 msgr2=0x7f57680b80d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:53.523 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.520+0000 7f576edef700 1 --2- 172.21.15.143:0/811948394 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f576815dda0 0x7f57680b80d0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f576000b730 tx=0x7f5760004ad0 comp rx=0 tx=0).stop 2023-11-08T19:30:53.523 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.520+0000 7f576edef700 1 -- 172.21.15.143:0/811948394 shutdown_connections 2023-11-08T19:30:53.523 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.520+0000 7f576edef700 1 --2- 172.21.15.143:0/811948394 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f574c06af00 0x7f574c06d3c0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:53.523 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.520+0000 7f576edef700 1 --2- 172.21.15.143:0/811948394 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f57680cef70 0x7f57680b4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:53.523 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.520+0000 7f576edef700 1 --2- 172.21.15.143:0/811948394 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f576815dda0 0x7f57680b80d0 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:53.524 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.520+0000 7f576edef700 1 --2- 172.21.15.143:0/811948394 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f57680d3390 0x7f57681f2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:53.524 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.520+0000 7f576edef700 1 -- 172.21.15.143:0/811948394 >> 172.21.15.143:0/811948394 conn(0x7f576806b000 msgr2=0x7f576815cf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:53.524 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.520+0000 7f576edef700 1 -- 172.21.15.143:0/811948394 shutdown_connections 2023-11-08T19:30:53.524 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.521+0000 7f576edef700 1 -- 172.21.15.143:0/811948394 wait complete. 2023-11-08T19:30:53.525 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 98 2023-11-08T19:30:53.534 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2023-11-08T19:30:53.535 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:30:53.656 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.655+0000 7fa6becef700 1 Processor -- start 2023-11-08T19:30:53.656 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.655+0000 7fa6becef700 1 -- start start 2023-11-08T19:30:53.659 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.655+0000 7fa6becef700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa6b815dc40 0x7fa6b806abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:53.659 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.655+0000 7fa6becef700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fa6b80b1e60 0x7fa6b806b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:53.659 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.655+0000 7fa6becef700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6b80ce990 con 0x7fa6b80b1760 2023-11-08T19:30:53.659 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.656+0000 7fa6becef700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6b80cebc0 con 0x7fa6b815dc40 2023-11-08T19:30:53.659 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.656+0000 7fa6becef700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6b80ceea0 con 0x7fa6b80b1e60 2023-11-08T19:30:53.660 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.656+0000 7fa6bca8b700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa6b815dc40 0x7fa6b806abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:53.660 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.656+0000 7fa6bca8b700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa6b815dc40 0x7fa6b806abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56550/0 (socket says 172.21.15.143:56550) 2023-11-08T19:30:53.660 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.656+0000 7fa6bca8b700 1 -- 172.21.15.143:0/2498529528 learned_addr learned my addr 172.21.15.143:0/2498529528 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:30:53.660 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.656+0000 7fa6b7fff700 1 --2- 172.21.15.143:0/2498529528 >> v2:172.21.15.174:3300/0 conn(0x7fa6b80b1e60 0x7fa6b806b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:53.661 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.656+0000 7fa6bca8b700 1 -- 172.21.15.143:0/2498529528 >> v2:172.21.15.174:3300/0 conn(0x7fa6b80b1e60 msgr2=0x7fa6b806b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:53.661 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.656+0000 7fa6bca8b700 1 --2- 172.21.15.143:0/2498529528 >> v2:172.21.15.174:3300/0 conn(0x7fa6b80b1e60 0x7fa6b806b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:53.661 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.656+0000 7fa6bca8b700 1 -- 172.21.15.143:0/2498529528 >> v1:172.21.15.174:6789/0 conn(0x7fa6b80b1760 legacy=0x7fa6b806d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:53.661 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.656+0000 7fa6bca8b700 1 -- 172.21.15.143:0/2498529528 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6b80cf180 con 0x7fa6b815dc40 2023-11-08T19:30:53.661 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.656+0000 7fa6bca8b700 1 --2- 172.21.15.143:0/2498529528 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6b815dc40 0x7fa6b806abe0 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7fa6b80d1930 tx=0x7fa6ac009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f585f21f03a1a7f1 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:53.662 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.656+0000 7fa6b77fe700 1 -- 172.21.15.143:0/2498529528 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6ac004230 con 0x7fa6b815dc40 2023-11-08T19:30:53.662 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.656+0000 7fa6b77fe700 1 -- 172.21.15.143:0/2498529528 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6ac004390 con 0x7fa6b815dc40 2023-11-08T19:30:53.662 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.656+0000 7fa6b77fe700 1 -- 172.21.15.143:0/2498529528 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6ac0036a0 con 0x7fa6b815dc40 2023-11-08T19:30:53.662 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.657+0000 7fa6becef700 1 -- 172.21.15.143:0/2498529528 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6b815dc40 msgr2=0x7fa6b806abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:53.662 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.657+0000 7fa6becef700 1 --2- 172.21.15.143:0/2498529528 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6b815dc40 0x7fa6b806abe0 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7fa6b80d1930 tx=0x7fa6ac009bd0 comp rx=0 tx=0).stop 2023-11-08T19:30:53.663 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.657+0000 7fa6becef700 1 -- 172.21.15.143:0/2498529528 shutdown_connections 2023-11-08T19:30:53.663 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.657+0000 7fa6becef700 1 --2- 172.21.15.143:0/2498529528 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6b815dc40 0x7fa6b806abe0 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:53.663 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.657+0000 7fa6becef700 1 --2- 172.21.15.143:0/2498529528 >> v2:172.21.15.174:3300/0 conn(0x7fa6b80b1e60 0x7fa6b806b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:53.663 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.657+0000 7fa6becef700 1 -- 172.21.15.143:0/2498529528 >> 172.21.15.143:0/2498529528 conn(0x7fa6b8066f10 msgr2=0x7fa6b806bcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:53.663 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.657+0000 7fa6becef700 1 -- 172.21.15.143:0/2498529528 shutdown_connections 2023-11-08T19:30:53.664 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.657+0000 7fa6becef700 1 -- 172.21.15.143:0/2498529528 wait complete. 2023-11-08T19:30:53.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.658+0000 7fa6becef700 1 Processor -- start 2023-11-08T19:30:53.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.658+0000 7fa6becef700 1 -- start start 2023-11-08T19:30:53.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.659+0000 7fa6becef700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6b80b1760 0x7fa6b80b0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:53.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.659+0000 7fa6becef700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa6b80b1e60 0x7fa6b8158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:53.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.659+0000 7fa6becef700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa6b815dc40 0x7fa6b8159530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:53.668 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.659+0000 7fa6becef700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6b8147de0 con 0x7fa6b80b1760 2023-11-08T19:30:53.668 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.659+0000 7fa6becef700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6b80b14e0 con 0x7fa6b815dc40 2023-11-08T19:30:53.668 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.659+0000 7fa6becef700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6b814a1a0 con 0x7fa6b80b1e60 2023-11-08T19:30:53.668 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.659+0000 7fa6bca8b700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6b80b1760 0x7fa6b80b0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:53.668 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.659+0000 7fa6bca8b700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6b80b1760 0x7fa6b80b0fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56556/0 (socket says 172.21.15.143:56556) 2023-11-08T19:30:53.669 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.659+0000 7fa6bca8b700 1 -- 172.21.15.143:0/343074750 learned_addr learned my addr 172.21.15.143:0/343074750 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:30:53.669 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.659+0000 7fa6bd28c700 1 --2- 172.21.15.143:0/343074750 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa6b815dc40 0x7fa6b8159530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:53.669 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.659+0000 7fa6b7fff700 1 --2- 172.21.15.143:0/343074750 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa6b80b1e60 0x7fa6b8158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:53.669 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.659+0000 7fa6bca8b700 1 -- 172.21.15.143:0/343074750 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa6b80b1e60 msgr2=0x7fa6b8158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:53.669 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.659+0000 7fa6bca8b700 1 --2- 172.21.15.143:0/343074750 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa6b80b1e60 0x7fa6b8158ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:53.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.659+0000 7fa6bca8b700 1 -- 172.21.15.143:0/343074750 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa6b815dc40 msgr2=0x7fa6b8159530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:53.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.659+0000 7fa6bca8b700 1 --2- 172.21.15.143:0/343074750 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa6b815dc40 0x7fa6b8159530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:53.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.659+0000 7fa6bca8b700 1 -- 172.21.15.143:0/343074750 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6ac009550 con 0x7fa6b80b1760 2023-11-08T19:30:53.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.659+0000 7fa6bd28c700 1 --2- 172.21.15.143:0/343074750 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa6b815dc40 0x7fa6b8159530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:30:53.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.659+0000 7fa6bca8b700 1 --2- 172.21.15.143:0/343074750 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6b80b1760 0x7fa6b80b0fa0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac009130 tx=0x7fa6ac004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:53.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.660+0000 7fa6b57fa700 1 -- 172.21.15.143:0/343074750 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6ac011630 con 0x7fa6b80b1760 2023-11-08T19:30:53.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.660+0000 7fa6b57fa700 1 -- 172.21.15.143:0/343074750 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6ac011790 con 0x7fa6b80b1760 2023-11-08T19:30:53.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.660+0000 7fa6b57fa700 1 -- 172.21.15.143:0/343074750 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6ac011b40 con 0x7fa6b80b1760 2023-11-08T19:30:53.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.660+0000 7fa6becef700 1 -- 172.21.15.143:0/343074750 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6b814a480 con 0x7fa6b80b1760 2023-11-08T19:30:53.672 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.660+0000 7fa6becef700 1 -- 172.21.15.143:0/343074750 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6b814aa40 con 0x7fa6b80b1760 2023-11-08T19:30:53.672 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.661+0000 7fa6becef700 1 -- 172.21.15.143:0/343074750 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6b804f730 con 0x7fa6b80b1760 2023-11-08T19:30:53.672 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.661+0000 7fa6b57fa700 1 -- 172.21.15.143:0/343074750 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa6ac011ca0 con 0x7fa6b80b1760 2023-11-08T19:30:53.672 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.661+0000 7fa6b57fa700 1 --2- 172.21.15.143:0/343074750 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa69c06af00 0x7fa69c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:53.672 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.661+0000 7fa6b57fa700 1 -- 172.21.15.143:0/343074750 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 7317+0+0 (secure 0 0 0) 0x7fa6ac08b150 con 0x7fa6b80b1760 2023-11-08T19:30:53.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.661+0000 7fa6b7fff700 1 --2- 172.21.15.143:0/343074750 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa69c06af00 0x7fa69c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:53.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.662+0000 7fa6b7fff700 1 --2- 172.21.15.143:0/343074750 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa69c06af00 0x7fa69c06d3c0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fa6b80a94f0 tx=0x7fa6a4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:53.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.666+0000 7fa6b57fa700 1 -- 172.21.15.143:0/343074750 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa6ac055cf0 con 0x7fa6b80b1760 2023-11-08T19:30:53.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.902+0000 7fa6becef700 1 -- 172.21.15.143:0/343074750 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa6b815a3e0 con 0x7fa6b80b1760 2023-11-08T19:30:53.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.902+0000 7fa6b57fa700 1 -- 172.21.15.143:0/343074750 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 98 v98) v1 ==== 94+0+4950 (secure 0 0 0) 0x7fa6ac01b440 con 0x7fa6b80b1760 2023-11-08T19:30:53.904 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:30:53.904 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":98,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5933,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/3918286710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6835","nonce":3918286710},{"type":"v1","addr":"172.21.15.174:6837","nonce":3918286710}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":6245,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6836/2689884850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2689884850},{"type":"v1","addr":"172.21.15.174:6836","nonce":2689884850}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":97}],"filesystems":[{"mdsmap":{"epoch":98,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:03.092245+0000","modified":"2023-11-08T19:30:48.753530+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":103,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6161,"mds_1":7743},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6161":{"gid":6161,"name":"a","rank":0,"incarnation":93,"state":"up:reconnect","state_seq":11,"addr":"172.21.15.143:6835/2129643800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":2129643800},{"type":"v1","addr":"172.21.15.143:6835","nonce":2129643800}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7743":{"gid":7743,"name":"c","rank":1,"incarnation":96,"state":"up:reconnect","state_seq":4,"addr":"172.21.15.143:6834/2489252684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":2489252684},{"type":"v1","addr":"172.21.15.143:6834","nonce":2489252684}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:30:53.906 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.904+0000 7fa6becef700 1 -- 172.21.15.143:0/343074750 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa69c06af00 msgr2=0x7fa69c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:53.906 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.904+0000 7fa6becef700 1 --2- 172.21.15.143:0/343074750 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa69c06af00 0x7fa69c06d3c0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fa6b80a94f0 tx=0x7fa6a4008040 comp rx=0 tx=0).stop 2023-11-08T19:30:53.907 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.904+0000 7fa6becef700 1 -- 172.21.15.143:0/343074750 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6b80b1760 msgr2=0x7fa6b80b0fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:53.907 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.904+0000 7fa6becef700 1 --2- 172.21.15.143:0/343074750 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6b80b1760 0x7fa6b80b0fa0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac009130 tx=0x7fa6ac004750 comp rx=0 tx=0).stop 2023-11-08T19:30:53.907 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.904+0000 7fa6becef700 1 -- 172.21.15.143:0/343074750 shutdown_connections 2023-11-08T19:30:53.907 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.904+0000 7fa6becef700 1 --2- 172.21.15.143:0/343074750 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa69c06af00 0x7fa69c06d3c0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:53.907 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.904+0000 7fa6becef700 1 --2- 172.21.15.143:0/343074750 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa6b815dc40 0x7fa6b8159530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:53.908 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.904+0000 7fa6becef700 1 --2- 172.21.15.143:0/343074750 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6b80b1760 0x7fa6b80b0fa0 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:53.908 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.904+0000 7fa6becef700 1 --2- 172.21.15.143:0/343074750 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa6b80b1e60 0x7fa6b8158ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:53.908 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.905+0000 7fa6becef700 1 -- 172.21.15.143:0/343074750 >> 172.21.15.143:0/343074750 conn(0x7fa6b8066f10 msgr2=0x7fa6b80b2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:53.908 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.905+0000 7fa6becef700 1 -- 172.21.15.143:0/343074750 shutdown_connections 2023-11-08T19:30:53.909 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:53.905+0000 7fa6becef700 1 -- 172.21.15.143:0/343074750 wait complete. 2023-11-08T19:30:53.909 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 98 2023-11-08T19:30:53.918 DEBUG:tasks.cephfs.filesystem:Looked up MDS state for mds.0: up:reconnect 2023-11-08T19:30:54.919 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:30:55.053 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:55.051+0000 7ff5810e5700 1 Processor -- start 2023-11-08T19:30:55.054 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:55.051+0000 7ff5810e5700 1 -- start start 2023-11-08T19:30:55.056 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:55.052+0000 7ff5810e5700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7ff57c0d0100 0x7ff57c15f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:55.056 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:55.052+0000 7ff5810e5700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff57c06f780 con 0x7ff57c15acd0 2023-11-08T19:30:55.056 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:55.052+0000 7ff5810e5700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff57c06e8e0 con 0x7ff57c15a8f0 2023-11-08T19:30:55.057 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:55.052+0000 7ff5810e5700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff57c06eb70 con 0x7ff57c0d0100 2023-11-08T19:30:55.057 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:55.052+0000 7ff57a59c700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7ff57c15acd0 0x7ff57c0ccea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:52720/0 (socket says 172.21.15.143:52720) 2023-11-08T19:30:55.057 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:55.052+0000 7ff57a59c700 1 -- 172.21.15.143:0/3306610620 learned_addr learned my addr 172.21.15.143:0/3306610620 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:30:55.057 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:55.053+0000 7ff57b59e700 1 --2- 172.21.15.143:0/3306610620 >> v2:172.21.15.174:3300/0 conn(0x7ff57c0d0100 0x7ff57c15f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:55.058 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:55.053+0000 7ff579d9b700 1 -- 172.21.15.143:0/3306610620 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3257862020 0 0) 0x7ff57c06f780 con 0x7ff57c15acd0 2023-11-08T19:30:55.058 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:55.053+0000 7ff579d9b700 1 -- 172.21.15.143:0/3306610620 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff5680035b0 con 0x7ff57c15acd0 2023-11-08T19:30:55.058 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:55.053+0000 7ff57b59e700 1 -- 172.21.15.143:0/3306610620 >> v1:172.21.15.174:6789/0 conn(0x7ff57c15a8f0 legacy=0x7ff57c06e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:55.058 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:55.053+0000 7ff579d9b700 1 -- 172.21.15.143:0/3306610620 <== mon.1 v1:172.21.15.174:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3375968130 0 0) 0x7ff57c06e8e0 con 0x7ff57c15a8f0 2023-11-08T19:30:55.059 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:55.053+0000 7ff57b59e700 1 -- 172.21.15.143:0/3306610620 >> v1:172.21.15.143:6789/0 conn(0x7ff57c15acd0 legacy=0x7ff57c0ccea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:55.059 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:55.053+0000 7ff57b59e700 1 -- 172.21.15.143:0/3306610620 --> v2:172.21.15.174:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff57c06ee50 con 0x7ff57c0d0100 2023-11-08T19:30:55.059 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:55.053+0000 7ff57b59e700 1 --2- 172.21.15.143:0/3306610620 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff57c0d0100 0x7ff57c15f080 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7ff574009d90 tx=0x7ff57400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ff94499ec295706 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:55.059 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:55.053+0000 7ff579d9b700 1 -- 172.21.15.143:0/3306610620 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff57400e070 con 0x7ff57c0d0100 2023-11-08T19:30:55.059 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:55.053+0000 7ff579d9b700 1 -- 172.21.15.143:0/3306610620 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff57400bb20 con 0x7ff57c0d0100 2023-11-08T19:30:55.060 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:55.053+0000 7ff579d9b700 1 -- 172.21.15.143:0/3306610620 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff57400bed0 con 0x7ff57c0d0100 2023-11-08T19:30:55.060 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:55.054+0000 7ff5810e5700 1 -- 172.21.15.143:0/3306610620 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff57c0d0100 msgr2=0x7ff57c15f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:55.060 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:55.054+0000 7ff5810e5700 1 --2- 172.21.15.143:0/3306610620 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff57c0d0100 0x7ff57c15f080 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7ff574009d90 tx=0x7ff57400b330 comp rx=0 tx=0).stop 2023-11-08T19:30:55.060 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:55.054+0000 7ff5810e5700 1 -- 172.21.15.143:0/3306610620 shutdown_connections 2023-11-08T19:30:55.060 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:55.054+0000 7ff5810e5700 1 --2- 172.21.15.143:0/3306610620 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff57c0d0100 0x7ff57c15f080 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:55.061 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:55.054+0000 7ff5810e5700 1 -- 172.21.15.143:0/3306610620 >> 172.21.15.143:0/3306610620 conn(0x7ff57c0b23d0 msgr2=0x7ff57c0b27b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:55.061 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:55.054+0000 7ff5810e5700 1 -- 172.21.15.143:0/3306610620 shutdown_connections 2023-11-08T19:30:55.061 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:55.054+0000 7ff5810e5700 1 -- 172.21.15.143:0/3306610620 wait complete. 2023-11-08T19:30:55.064 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:55.056+0000 7ff5810e5700 1 Processor -- start 2023-11-08T19:30:55.065 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:55.056+0000 7ff5810e5700 1 -- start start 2023-11-08T19:30:55.065 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:55.056+0000 7ff5810e5700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff57c0d0100 0x7ff57c06b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:55.065 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:55.056+0000 7ff5810e5700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff57c15a8f0 0x7ff57c151fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:55.065 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:55.056+0000 7ff5810e5700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff57c15acd0 0x7ff57c1f1220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:55.066 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:55.056+0000 7ff5810e5700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff57c152510 con 0x7ff57c0d0100 2023-11-08T19:30:55.066 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:55.056+0000 7ff5810e5700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff57c1f1b30 con 0x7ff57c15acd0 2023-11-08T19:30:55.066 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:55.056+0000 7ff5810e5700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff57c1f1e10 con 0x7ff57c15a8f0 2023-11-08T19:30:55.066 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:55.056+0000 7ff57ad9d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff57c0d0100 0x7ff57c06b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:55.067 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:55.056+0000 7ff57ad9d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff57c0d0100 0x7ff57c06b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56570/0 (socket says 172.21.15.143:56570) 2023-11-08T19:30:55.067 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:55.056+0000 7ff57ad9d700 1 -- 172.21.15.143:0/1212337743 learned_addr learned my addr 172.21.15.143:0/1212337743 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:30:55.067 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:55.057+0000 7ff57b59e700 1 --2- 172.21.15.143:0/1212337743 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff57c15acd0 0x7ff57c1f1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:55.067 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:55.057+0000 7ff57a59c700 1 --2- 172.21.15.143:0/1212337743 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff57c15a8f0 0x7ff57c151fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:55.067 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:55.057+0000 7ff57ad9d700 1 -- 172.21.15.143:0/1212337743 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff57c15a8f0 msgr2=0x7ff57c151fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:55.068 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:55.057+0000 7ff57ad9d700 1 --2- 172.21.15.143:0/1212337743 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff57c15a8f0 0x7ff57c151fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:55.068 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:55.057+0000 7ff57ad9d700 1 -- 172.21.15.143:0/1212337743 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff57c15acd0 msgr2=0x7ff57c1f1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:55.068 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:55.057+0000 7ff57ad9d700 1 --2- 172.21.15.143:0/1212337743 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff57c15acd0 0x7ff57c1f1220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:55.068 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:55.057+0000 7ff57ad9d700 1 -- 172.21.15.143:0/1212337743 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff574009a70 con 0x7ff57c0d0100 2023-11-08T19:30:55.069 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:55.057+0000 7ff57a59c700 1 --2- 172.21.15.143:0/1212337743 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff57c15a8f0 0x7ff57c151fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:30:55.069 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:55.057+0000 7ff57ad9d700 1 --2- 172.21.15.143:0/1212337743 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff57c0d0100 0x7ff57c06b780 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7ff5640011e0 tx=0x7ff56400ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:55.069 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:55.057+0000 7ff5737fe700 1 -- 172.21.15.143:0/1212337743 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5640054e0 con 0x7ff57c0d0100 2023-11-08T19:30:55.069 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:55.057+0000 7ff5737fe700 1 -- 172.21.15.143:0/1212337743 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff564003680 con 0x7ff57c0d0100 2023-11-08T19:30:55.069 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:55.057+0000 7ff5810e5700 1 -- 172.21.15.143:0/1212337743 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff57c1f20f0 con 0x7ff57c0d0100 2023-11-08T19:30:55.070 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:55.057+0000 7ff5737fe700 1 -- 172.21.15.143:0/1212337743 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff564003a30 con 0x7ff57c0d0100 2023-11-08T19:30:55.070 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:55.057+0000 7ff5810e5700 1 -- 172.21.15.143:0/1212337743 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff57c0c9810 con 0x7ff57c0d0100 2023-11-08T19:30:55.070 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:55.058+0000 7ff5810e5700 1 -- 172.21.15.143:0/1212337743 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff57c0c99d0 con 0x7ff57c0d0100 2023-11-08T19:30:55.070 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:55.058+0000 7ff5737fe700 1 -- 172.21.15.143:0/1212337743 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff564003b90 con 0x7ff57c0d0100 2023-11-08T19:30:55.071 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:55.059+0000 7ff5737fe700 1 --2- 172.21.15.143:0/1212337743 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7ff55c06af00 0x7ff55c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:55.071 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:55.059+0000 7ff5737fe700 1 -- 172.21.15.143:0/1212337743 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 7317+0+0 (secure 0 0 0) 0x7ff5640890c0 con 0x7ff57c0d0100 2023-11-08T19:30:55.071 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:55.059+0000 7ff57a59c700 1 --2- 172.21.15.143:0/1212337743 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7ff55c06af00 0x7ff55c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:55.071 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:55.060+0000 7ff57a59c700 1 --2- 172.21.15.143:0/1212337743 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7ff55c06af00 0x7ff55c06d3c0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7ff57c06bcc0 tx=0x7ff56c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:55.072 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:55.063+0000 7ff5737fe700 1 -- 172.21.15.143:0/1212337743 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff564053b20 con 0x7ff57c0d0100 2023-11-08T19:30:55.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:55.295+0000 7ff5810e5700 1 -- 172.21.15.143:0/1212337743 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff57c04f730 con 0x7ff57c0d0100 2023-11-08T19:30:55.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:55.296+0000 7ff5737fe700 1 -- 172.21.15.143:0/1212337743 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 98 v98) v1 ==== 94+0+4950 (secure 0 0 0) 0x7ff564017090 con 0x7ff57c0d0100 2023-11-08T19:30:55.298 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:30:55.298 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":98,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5933,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/3918286710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6835","nonce":3918286710},{"type":"v1","addr":"172.21.15.174:6837","nonce":3918286710}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":6245,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6836/2689884850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2689884850},{"type":"v1","addr":"172.21.15.174:6836","nonce":2689884850}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":97}],"filesystems":[{"mdsmap":{"epoch":98,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:03.092245+0000","modified":"2023-11-08T19:30:48.753530+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":103,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6161,"mds_1":7743},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6161":{"gid":6161,"name":"a","rank":0,"incarnation":93,"state":"up:reconnect","state_seq":11,"addr":"172.21.15.143:6835/2129643800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":2129643800},{"type":"v1","addr":"172.21.15.143:6835","nonce":2129643800}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7743":{"gid":7743,"name":"c","rank":1,"incarnation":96,"state":"up:reconnect","state_seq":4,"addr":"172.21.15.143:6834/2489252684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":2489252684},{"type":"v1","addr":"172.21.15.143:6834","nonce":2489252684}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:30:55.299 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:55.298+0000 7ff5810e5700 1 -- 172.21.15.143:0/1212337743 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7ff55c06af00 msgr2=0x7ff55c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:55.300 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:55.298+0000 7ff5810e5700 1 --2- 172.21.15.143:0/1212337743 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7ff55c06af00 0x7ff55c06d3c0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7ff57c06bcc0 tx=0x7ff56c009040 comp rx=0 tx=0).stop 2023-11-08T19:30:55.300 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:55.298+0000 7ff5810e5700 1 -- 172.21.15.143:0/1212337743 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff57c0d0100 msgr2=0x7ff57c06b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:55.300 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:55.298+0000 7ff5810e5700 1 --2- 172.21.15.143:0/1212337743 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff57c0d0100 0x7ff57c06b780 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7ff5640011e0 tx=0x7ff56400ca30 comp rx=0 tx=0).stop 2023-11-08T19:30:55.300 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:55.298+0000 7ff5810e5700 1 -- 172.21.15.143:0/1212337743 shutdown_connections 2023-11-08T19:30:55.301 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:55.298+0000 7ff5810e5700 1 --2- 172.21.15.143:0/1212337743 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7ff55c06af00 0x7ff55c06d3c0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:55.301 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:55.298+0000 7ff5810e5700 1 --2- 172.21.15.143:0/1212337743 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff57c15acd0 0x7ff57c1f1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:55.301 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:55.298+0000 7ff5810e5700 1 --2- 172.21.15.143:0/1212337743 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff57c0d0100 0x7ff57c06b780 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:55.301 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:55.298+0000 7ff5810e5700 1 --2- 172.21.15.143:0/1212337743 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff57c15a8f0 0x7ff57c151fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:55.301 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:55.298+0000 7ff5810e5700 1 -- 172.21.15.143:0/1212337743 >> 172.21.15.143:0/1212337743 conn(0x7ff57c0b23d0 msgr2=0x7ff57c15d470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:55.302 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:55.298+0000 7ff5810e5700 1 -- 172.21.15.143:0/1212337743 shutdown_connections 2023-11-08T19:30:55.302 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:55.298+0000 7ff5810e5700 1 -- 172.21.15.143:0/1212337743 wait complete. 2023-11-08T19:30:55.302 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 98 2023-11-08T19:30:55.311 DEBUG:tasks.cephfs.filesystem:Looked up MDS state for mds.0: up:reconnect 2023-11-08T19:30:56.312 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:30:56.444 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:56.442+0000 7f2495254700 1 Processor -- start 2023-11-08T19:30:56.444 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:56.442+0000 7f2495254700 1 -- start start 2023-11-08T19:30:56.452 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:56.443+0000 7f2495254700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f249015a8f0 0x7f249015f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:56.453 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:56.443+0000 7f2495254700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f24900ccea0 0x7f24900cd310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:56.453 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:56.443+0000 7f2495254700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f249006f780 con 0x7f249015acd0 2023-11-08T19:30:56.453 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:56.443+0000 7f2495254700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f249006e8e0 con 0x7f24900ccea0 2023-11-08T19:30:56.453 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:56.443+0000 7f2495254700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24900cd850 con 0x7f249015a8f0 2023-11-08T19:30:56.454 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:56.443+0000 7f248e59c700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f249015acd0 0x7f249006e1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:52734/0 (socket says 172.21.15.143:52734) 2023-11-08T19:30:56.454 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:56.443+0000 7f248e59c700 1 -- 172.21.15.143:0/585009638 learned_addr learned my addr 172.21.15.143:0/585009638 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:30:56.454 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:56.443+0000 7f248dd9b700 1 -- 172.21.15.143:0/585009638 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 637323100 0 0) 0x7f249006f780 con 0x7f249015acd0 2023-11-08T19:30:56.454 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:56.443+0000 7f248f59e700 1 --2- 172.21.15.143:0/585009638 >> v2:172.21.15.174:3300/0 conn(0x7f24900ccea0 0x7f24900cd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:56.455 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:56.443+0000 7f248ed9d700 1 --2- 172.21.15.143:0/585009638 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f249015a8f0 0x7f249015f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:56.455 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:56.443+0000 7f248dd9b700 1 -- 172.21.15.143:0/585009638 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f24780035b0 con 0x7f249015acd0 2023-11-08T19:30:56.455 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:56.444+0000 7f248dd9b700 1 -- 172.21.15.143:0/585009638 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 858057988 0 0) 0x7f24780035b0 con 0x7f249015acd0 2023-11-08T19:30:56.455 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:56.444+0000 7f248dd9b700 1 -- 172.21.15.143:0/585009638 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f249015a8f0 msgr2=0x7f249015f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:56.456 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:56.444+0000 7f248dd9b700 1 --2- 172.21.15.143:0/585009638 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f249015a8f0 0x7f249015f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:56.456 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:56.444+0000 7f248dd9b700 1 -- 172.21.15.143:0/585009638 >> v2:172.21.15.174:3300/0 conn(0x7f24900ccea0 msgr2=0x7f24900cd310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:56.456 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:56.444+0000 7f248dd9b700 1 --2- 172.21.15.143:0/585009638 >> v2:172.21.15.174:3300/0 conn(0x7f24900ccea0 0x7f24900cd310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:56.456 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:56.444+0000 7f248dd9b700 1 -- 172.21.15.143:0/585009638 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f249006eb70 con 0x7f249015acd0 2023-11-08T19:30:56.456 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:56.444+0000 7f248dd9b700 1 -- 172.21.15.143:0/585009638 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4147756303 0 0) 0x7f2480001670 con 0x7f249015acd0 2023-11-08T19:30:56.457 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:56.444+0000 7f248dd9b700 1 -- 172.21.15.143:0/585009638 >> v1:172.21.15.143:6789/0 conn(0x7f249015acd0 legacy=0x7f249006e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:56.457 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:56.444+0000 7f248dd9b700 1 --2- 172.21.15.143:0/585009638 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2478003a00 0x7f2478003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:56.457 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:56.444+0000 7f248dd9b700 1 --2- 172.21.15.143:0/585009638 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f249015a8f0 0x7f2478004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:56.457 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:56.444+0000 7f248dd9b700 1 --2- 172.21.15.143:0/585009638 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2478004d90 0x7f2478005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:56.458 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:56.444+0000 7f248dd9b700 1 -- 172.21.15.143:0/585009638 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f249006e8e0 con 0x7f2478003a00 2023-11-08T19:30:56.458 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:56.444+0000 7f248dd9b700 1 -- 172.21.15.143:0/585009638 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24900cd850 con 0x7f2478004d90 2023-11-08T19:30:56.458 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:56.444+0000 7f248dd9b700 1 -- 172.21.15.143:0/585009638 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f247800ba10 con 0x7f249015a8f0 2023-11-08T19:30:56.458 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:56.444+0000 7f248ed9d700 1 --2- 172.21.15.143:0/585009638 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2478003a00 0x7f2478003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:56.458 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:56.444+0000 7f248f59e700 1 --2- 172.21.15.143:0/585009638 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2478004d90 0x7f2478005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:56.459 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:56.444+0000 7f248e59c700 1 --2- 172.21.15.143:0/585009638 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f249015a8f0 0x7f2478004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:56.459 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:56.444+0000 7f248ed9d700 1 -- 172.21.15.143:0/585009638 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f249015a8f0 msgr2=0x7f2478004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:56.459 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:56.444+0000 7f248ed9d700 1 --2- 172.21.15.143:0/585009638 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f249015a8f0 0x7f2478004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:56.459 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:56.444+0000 7f248ed9d700 1 -- 172.21.15.143:0/585009638 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2478004d90 msgr2=0x7f2478005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:56.460 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:56.444+0000 7f248ed9d700 1 --2- 172.21.15.143:0/585009638 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2478004d90 0x7f2478005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:56.460 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:56.444+0000 7f248ed9d700 1 -- 172.21.15.143:0/585009638 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24780035b0 con 0x7f2478003a00 2023-11-08T19:30:56.460 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:56.444+0000 7f248ed9d700 1 --2- 172.21.15.143:0/585009638 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2478003a00 0x7f2478003e20 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f2484009b60 tx=0x7f2478005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b30b535b0cf06e26 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:56.460 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:56.444+0000 7f248dd9b700 1 -- 172.21.15.143:0/585009638 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f248400e6c0 con 0x7f2478003a00 2023-11-08T19:30:56.461 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:56.444+0000 7f248dd9b700 1 -- 172.21.15.143:0/585009638 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f248400e820 con 0x7f2478003a00 2023-11-08T19:30:56.461 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:56.444+0000 7f248dd9b700 1 -- 172.21.15.143:0/585009638 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f248400eb90 con 0x7f2478003a00 2023-11-08T19:30:56.461 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:56.445+0000 7f2495254700 1 -- 172.21.15.143:0/585009638 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2478003a00 msgr2=0x7f2478003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:56.461 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:56.445+0000 7f2495254700 1 --2- 172.21.15.143:0/585009638 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2478003a00 0x7f2478003e20 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f2484009b60 tx=0x7f2478005880 comp rx=0 tx=0).stop 2023-11-08T19:30:56.461 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:56.445+0000 7f248f59e700 1 -- 172.21.15.143:0/585009638 reap_dead start 2023-11-08T19:30:56.462 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:56.445+0000 7f2495254700 1 -- 172.21.15.143:0/585009638 shutdown_connections 2023-11-08T19:30:56.462 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:56.445+0000 7f2495254700 1 -- 172.21.15.143:0/585009638 >> 172.21.15.143:0/585009638 conn(0x7f24900b23d0 msgr2=0x7f24900b27b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:56.462 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:56.445+0000 7f2495254700 1 -- 172.21.15.143:0/585009638 shutdown_connections 2023-11-08T19:30:56.462 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:56.445+0000 7f2495254700 1 -- 172.21.15.143:0/585009638 wait complete. 2023-11-08T19:30:56.463 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:56.447+0000 7f2495254700 1 Processor -- start 2023-11-08T19:30:56.463 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:56.447+0000 7f2495254700 1 -- start start 2023-11-08T19:30:56.463 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:56.447+0000 7f2495254700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2478003a00 0x7f24900cf7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:56.463 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:56.447+0000 7f2495254700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2490159e40 0x7f24900c9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:56.463 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:56.447+0000 7f2495254700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f249015d300 0x7f249015d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:56.464 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:56.447+0000 7f2495254700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24900af930 con 0x7f2478003a00 2023-11-08T19:30:56.464 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:56.447+0000 7f2495254700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f249015dcf0 con 0x7f2490159e40 2023-11-08T19:30:56.464 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:56.447+0000 7f2495254700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24900aedd0 con 0x7f249015d300 2023-11-08T19:30:56.464 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:56.448+0000 7f248ed9d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2478003a00 0x7f24900cf7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:56.464 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:56.448+0000 7f248ed9d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2478003a00 0x7f24900cf7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56592/0 (socket says 172.21.15.143:56592) 2023-11-08T19:30:56.465 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:56.448+0000 7f248ed9d700 1 -- 172.21.15.143:0/1019105160 learned_addr learned my addr 172.21.15.143:0/1019105160 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:30:56.465 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:56.448+0000 7f248ed9d700 1 -- 172.21.15.143:0/1019105160 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f249015d300 msgr2=0x7f249015d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:56.465 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:56.448+0000 7f248ed9d700 1 --2- 172.21.15.143:0/1019105160 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f249015d300 0x7f249015d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:56.465 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:56.448+0000 7f248ed9d700 1 -- 172.21.15.143:0/1019105160 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2490159e40 msgr2=0x7f24900c9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:56.466 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:56.448+0000 7f248ed9d700 1 --2- 172.21.15.143:0/1019105160 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2490159e40 0x7f24900c9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:56.466 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:56.448+0000 7f248ed9d700 1 -- 172.21.15.143:0/1019105160 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24900af090 con 0x7f2478003a00 2023-11-08T19:30:56.466 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:56.448+0000 7f248ed9d700 1 --2- 172.21.15.143:0/1019105160 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2478003a00 0x7f24900cf7f0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f2478004770 tx=0x7f24840047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:56.466 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:56.448+0000 7f24777fe700 1 -- 172.21.15.143:0/1019105160 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f248400e6c0 con 0x7f2478003a00 2023-11-08T19:30:56.466 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:56.448+0000 7f24777fe700 1 -- 172.21.15.143:0/1019105160 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2484003ab0 con 0x7f2478003a00 2023-11-08T19:30:56.467 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:56.448+0000 7f2495254700 1 -- 172.21.15.143:0/1019105160 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24900af370 con 0x7f2478003a00 2023-11-08T19:30:56.467 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:56.448+0000 7f24777fe700 1 -- 172.21.15.143:0/1019105160 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2484003e60 con 0x7f2478003a00 2023-11-08T19:30:56.467 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:56.448+0000 7f2495254700 1 -- 172.21.15.143:0/1019105160 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24900ada90 con 0x7f2478003a00 2023-11-08T19:30:56.467 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:56.449+0000 7f2495254700 1 -- 172.21.15.143:0/1019105160 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24900ad8f0 con 0x7f2478003a00 2023-11-08T19:30:56.468 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:56.450+0000 7f24777fe700 1 -- 172.21.15.143:0/1019105160 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2484013470 con 0x7f2478003a00 2023-11-08T19:30:56.468 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:56.450+0000 7f24777fe700 1 --2- 172.21.15.143:0/1019105160 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f247c06aef0 0x7f247c06d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:56.468 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:56.450+0000 7f24777fe700 1 -- 172.21.15.143:0/1019105160 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 7360+0+0 (secure 0 0 0) 0x7f248408b4c0 con 0x7f2478003a00 2023-11-08T19:30:56.468 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:56.450+0000 7f248e59c700 1 --2- 172.21.15.143:0/1019105160 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f247c06aef0 0x7f247c06d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:56.469 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:56.451+0000 7f248e59c700 1 --2- 172.21.15.143:0/1019105160 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f247c06aef0 0x7f247c06d3b0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f2480000fb0 tx=0x7f2480001c80 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:56.469 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:56.454+0000 7f24777fe700 1 -- 172.21.15.143:0/1019105160 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f24900ad8f0 con 0x7f2478003a00 2023-11-08T19:30:56.689 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:56.687+0000 7f2495254700 1 -- 172.21.15.143:0/1019105160 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f24900ad8f0 con 0x7f2478003a00 2023-11-08T19:30:56.689 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:56.688+0000 7f24777fe700 1 -- 172.21.15.143:0/1019105160 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 98 v98) v1 ==== 94+0+4950 (secure 0 0 0) 0x7f24900ad8f0 con 0x7f2478003a00 2023-11-08T19:30:56.690 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:30:56.690 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":98,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5933,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/3918286710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6835","nonce":3918286710},{"type":"v1","addr":"172.21.15.174:6837","nonce":3918286710}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":6245,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6836/2689884850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2689884850},{"type":"v1","addr":"172.21.15.174:6836","nonce":2689884850}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":97}],"filesystems":[{"mdsmap":{"epoch":98,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:03.092245+0000","modified":"2023-11-08T19:30:48.753530+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":103,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6161,"mds_1":7743},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6161":{"gid":6161,"name":"a","rank":0,"incarnation":93,"state":"up:reconnect","state_seq":11,"addr":"172.21.15.143:6835/2129643800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":2129643800},{"type":"v1","addr":"172.21.15.143:6835","nonce":2129643800}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7743":{"gid":7743,"name":"c","rank":1,"incarnation":96,"state":"up:reconnect","state_seq":4,"addr":"172.21.15.143:6834/2489252684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":2489252684},{"type":"v1","addr":"172.21.15.143:6834","nonce":2489252684}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:30:56.691 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:56.690+0000 7f2495254700 1 -- 172.21.15.143:0/1019105160 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f247c06aef0 msgr2=0x7f247c06d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:56.692 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:56.690+0000 7f2495254700 1 --2- 172.21.15.143:0/1019105160 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f247c06aef0 0x7f247c06d3b0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f2480000fb0 tx=0x7f2480001c80 comp rx=0 tx=0).stop 2023-11-08T19:30:56.692 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:56.690+0000 7f2495254700 1 -- 172.21.15.143:0/1019105160 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2478003a00 msgr2=0x7f24900cf7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:56.692 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:56.690+0000 7f2495254700 1 --2- 172.21.15.143:0/1019105160 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2478003a00 0x7f24900cf7f0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f2478004770 tx=0x7f24840047c0 comp rx=0 tx=0).stop 2023-11-08T19:30:56.692 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:56.690+0000 7f2495254700 1 -- 172.21.15.143:0/1019105160 shutdown_connections 2023-11-08T19:30:56.692 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:56.690+0000 7f2495254700 1 --2- 172.21.15.143:0/1019105160 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f247c06aef0 0x7f247c06d3b0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:56.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:56.690+0000 7f2495254700 1 --2- 172.21.15.143:0/1019105160 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f249015d300 0x7f249015d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:56.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:56.690+0000 7f2495254700 1 --2- 172.21.15.143:0/1019105160 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2478003a00 0x7f24900cf7f0 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:56.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:56.690+0000 7f2495254700 1 --2- 172.21.15.143:0/1019105160 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2490159e40 0x7f24900c9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:56.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:56.690+0000 7f2495254700 1 -- 172.21.15.143:0/1019105160 >> 172.21.15.143:0/1019105160 conn(0x7f24900b23d0 msgr2=0x7f24900b5380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:56.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:56.690+0000 7f2495254700 1 -- 172.21.15.143:0/1019105160 shutdown_connections 2023-11-08T19:30:56.694 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:56.690+0000 7f2495254700 1 -- 172.21.15.143:0/1019105160 wait complete. 2023-11-08T19:30:56.694 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 98 2023-11-08T19:30:56.703 DEBUG:tasks.cephfs.filesystem:Looked up MDS state for mds.0: up:reconnect 2023-11-08T19:30:57.705 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:30:57.709 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T19:30:57.752 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T19:30:57.848 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:57.846+0000 7f5c87e73700 1 Processor -- start 2023-11-08T19:30:57.848 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:57.846+0000 7f5c87e73700 1 -- start start 2023-11-08T19:30:57.851 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:57.847+0000 7f5c87e73700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5c8006ce10 0x7f5c800ceb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:57.851 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:57.847+0000 7f5c87e73700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f5c800b2f30 0x7f5c8015d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:57.851 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:57.847+0000 7f5c87e73700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c800d0000 con 0x7f5c800b2b50 2023-11-08T19:30:57.851 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:57.847+0000 7f5c87e73700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c800d02e0 con 0x7f5c800b2f30 2023-11-08T19:30:57.851 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:57.847+0000 7f5c87e73700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c800d05c0 con 0x7f5c8006ce10 2023-11-08T19:30:57.852 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:57.847+0000 7f5c86410700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f5c800b2f30 0x7f5c8015d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:57.852 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:57.847+0000 7f5c85c0f700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f5c800b2b50 0x7f5c8006c700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:52760/0 (socket says 172.21.15.143:52760) 2023-11-08T19:30:57.852 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:57.847+0000 7f5c85c0f700 1 -- 172.21.15.143:0/1520178303 learned_addr learned my addr 172.21.15.143:0/1520178303 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:30:57.852 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:57.847+0000 7f5c86410700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f5c800b2f30 0x7f5c8015d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56610/0 (socket says 172.21.15.143:56610) 2023-11-08T19:30:57.852 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:57.847+0000 7f5c8540e700 1 --2- 172.21.15.143:0/1520178303 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5c8006ce10 0x7f5c800ceb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:57.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:57.847+0000 7f5c86410700 1 -- 172.21.15.143:0/1520178303 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5c8006ce10 msgr2=0x7f5c800ceb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:57.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:57.847+0000 7f5c86410700 1 --2- 172.21.15.143:0/1520178303 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5c8006ce10 0x7f5c800ceb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:57.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:57.847+0000 7f5c86410700 1 -- 172.21.15.143:0/1520178303 >> v1:172.21.15.143:6789/0 conn(0x7f5c800b2b50 legacy=0x7f5c8006c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:57.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:57.847+0000 7f5c86410700 1 -- 172.21.15.143:0/1520178303 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c800d08a0 con 0x7f5c800b2f30 2023-11-08T19:30:57.854 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:57.847+0000 7f5c84c0d700 1 -- 172.21.15.143:0/1520178303 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1163440947 0 0) 0x7f5c800d0000 con 0x7f5c800b2b50 2023-11-08T19:30:57.854 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:57.847+0000 7f5c86410700 1 --2- 172.21.15.143:0/1520178303 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c800b2f30 0x7f5c8015d560 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f5c78009d90 tx=0x7f5c7800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b396654cdf5274a9 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:57.854 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:57.848+0000 7f5c84c0d700 1 -- 172.21.15.143:0/1520178303 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c7800e070 con 0x7f5c800b2f30 2023-11-08T19:30:57.854 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:57.848+0000 7f5c84c0d700 1 -- 172.21.15.143:0/1520178303 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c7800bb20 con 0x7f5c800b2f30 2023-11-08T19:30:57.854 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:57.848+0000 7f5c84c0d700 1 -- 172.21.15.143:0/1520178303 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c78012430 con 0x7f5c800b2f30 2023-11-08T19:30:57.855 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:57.848+0000 7f5c87e73700 1 -- 172.21.15.143:0/1520178303 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c800b2f30 msgr2=0x7f5c8015d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:57.855 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:57.848+0000 7f5c87e73700 1 --2- 172.21.15.143:0/1520178303 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c800b2f30 0x7f5c8015d560 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f5c78009d90 tx=0x7f5c7800b330 comp rx=0 tx=0).stop 2023-11-08T19:30:57.855 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:57.848+0000 7f5c87e73700 1 -- 172.21.15.143:0/1520178303 shutdown_connections 2023-11-08T19:30:57.855 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:57.848+0000 7f5c87e73700 1 --2- 172.21.15.143:0/1520178303 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c800b2f30 0x7f5c8015d560 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:57.856 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:57.848+0000 7f5c87e73700 1 --2- 172.21.15.143:0/1520178303 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5c8006ce10 0x7f5c800ceb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:57.856 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:57.848+0000 7f5c87e73700 1 -- 172.21.15.143:0/1520178303 >> 172.21.15.143:0/1520178303 conn(0x7f5c8009aa70 msgr2=0x7f5c800d18f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:57.856 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:57.848+0000 7f5c87e73700 1 -- 172.21.15.143:0/1520178303 shutdown_connections 2023-11-08T19:30:57.856 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:57.848+0000 7f5c87e73700 1 -- 172.21.15.143:0/1520178303 wait complete. 2023-11-08T19:30:57.856 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:57.849+0000 7f5c87e73700 1 Processor -- start 2023-11-08T19:30:57.857 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:57.850+0000 7f5c87e73700 1 -- start start 2023-11-08T19:30:57.860 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:57.850+0000 7f5c87e73700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c8006ce10 0x7f5c801d8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:57.860 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:57.850+0000 7f5c87e73700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5c800b2b50 0x7f5c801daf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:57.860 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:57.850+0000 7f5c87e73700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5c800b2f30 0x7f5c801df120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:57.860 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:57.850+0000 7f5c87e73700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c801daa70 con 0x7f5c8006ce10 2023-11-08T19:30:57.860 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:57.850+0000 7f5c87e73700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c801dce50 con 0x7f5c800b2b50 2023-11-08T19:30:57.861 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:57.850+0000 7f5c87e73700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c800704c0 con 0x7f5c800b2f30 2023-11-08T19:30:57.861 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:57.850+0000 7f5c85c0f700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c8006ce10 0x7f5c801d8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:57.861 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:57.850+0000 7f5c85c0f700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c8006ce10 0x7f5c801d8ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56612/0 (socket says 172.21.15.143:56612) 2023-11-08T19:30:57.861 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:57.850+0000 7f5c85c0f700 1 -- 172.21.15.143:0/4077173623 learned_addr learned my addr 172.21.15.143:0/4077173623 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:30:57.861 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:57.851+0000 7f5c86410700 1 --2- 172.21.15.143:0/4077173623 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5c800b2f30 0x7f5c801df120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:57.862 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:57.851+0000 7f5c8540e700 1 --2- 172.21.15.143:0/4077173623 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5c800b2b50 0x7f5c801daf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:57.862 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:57.851+0000 7f5c85c0f700 1 -- 172.21.15.143:0/4077173623 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5c800b2f30 msgr2=0x7f5c801df120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:57.862 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:57.851+0000 7f5c85c0f700 1 --2- 172.21.15.143:0/4077173623 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5c800b2f30 0x7f5c801df120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:57.862 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:57.851+0000 7f5c85c0f700 1 -- 172.21.15.143:0/4077173623 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5c800b2b50 msgr2=0x7f5c801daf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:57.862 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:57.851+0000 7f5c85c0f700 1 --2- 172.21.15.143:0/4077173623 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5c800b2b50 0x7f5c801daf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:57.863 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:57.851+0000 7f5c85c0f700 1 -- 172.21.15.143:0/4077173623 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c78009a70 con 0x7f5c8006ce10 2023-11-08T19:30:57.863 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:57.851+0000 7f5c85c0f700 1 --2- 172.21.15.143:0/4077173623 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c8006ce10 0x7f5c801d8ce0 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c0010e0 tx=0x7f5c7c00cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:57.863 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:57.851+0000 7f5c767fc700 1 -- 172.21.15.143:0/4077173623 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c7c0074e0 con 0x7f5c8006ce10 2023-11-08T19:30:57.863 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:57.851+0000 7f5c767fc700 1 -- 172.21.15.143:0/4077173623 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c7c007640 con 0x7f5c8006ce10 2023-11-08T19:30:57.864 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:57.851+0000 7f5c87e73700 1 -- 172.21.15.143:0/4077173623 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c801da420 con 0x7f5c8006ce10 2023-11-08T19:30:57.864 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:57.851+0000 7f5c767fc700 1 -- 172.21.15.143:0/4077173623 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c7c005a70 con 0x7f5c8006ce10 2023-11-08T19:30:57.864 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:57.851+0000 7f5c87e73700 1 -- 172.21.15.143:0/4077173623 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c800707a0 con 0x7f5c8006ce10 2023-11-08T19:30:57.864 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:57.852+0000 7f5c87e73700 1 -- 172.21.15.143:0/4077173623 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c801dad50 con 0x7f5c8006ce10 2023-11-08T19:30:57.864 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:57.853+0000 7f5c767fc700 1 -- 172.21.15.143:0/4077173623 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5c7c005bd0 con 0x7f5c8006ce10 2023-11-08T19:30:57.864 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:57.853+0000 7f5c767fc700 1 --2- 172.21.15.143:0/4077173623 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f5c6406af00 0x7f5c6406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:57.865 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:57.853+0000 7f5c767fc700 1 -- 172.21.15.143:0/4077173623 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 7360+0+0 (secure 0 0 0) 0x7f5c7c089620 con 0x7f5c8006ce10 2023-11-08T19:30:57.865 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:57.853+0000 7f5c8540e700 1 --2- 172.21.15.143:0/4077173623 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f5c6406af00 0x7f5c6406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:57.865 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:57.854+0000 7f5c8540e700 1 --2- 172.21.15.143:0/4077173623 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f5c6406af00 0x7f5c6406d3c0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f5c801e4da0 tx=0x7f5c70008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:57.865 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:57.857+0000 7f5c767fc700 1 -- 172.21.15.143:0/4077173623 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5c7c054060 con 0x7f5c8006ce10 2023-11-08T19:30:58.092 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.091+0000 7f5c87e73700 1 -- 172.21.15.143:0/4077173623 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5c8004f730 con 0x7f5c8006ce10 2023-11-08T19:30:58.093 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.091+0000 7f5c767fc700 1 -- 172.21.15.143:0/4077173623 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 100 v100) v1 ==== 95+0+4946 (secure 0 0 0) 0x7f5c7c017070 con 0x7f5c8006ce10 2023-11-08T19:30:58.093 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:30:58.094 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":100,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5933,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/3918286710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6835","nonce":3918286710},{"type":"v1","addr":"172.21.15.174:6837","nonce":3918286710}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":6245,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6836/2689884850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2689884850},{"type":"v1","addr":"172.21.15.174:6836","nonce":2689884850}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":97}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:03.092245+0000","modified":"2023-11-08T19:30:56.788736+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":103,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6161,"mds_1":7743},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6161":{"gid":6161,"name":"a","rank":0,"incarnation":93,"state":"up:active","state_seq":14,"addr":"172.21.15.143:6835/2129643800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":2129643800},{"type":"v1","addr":"172.21.15.143:6835","nonce":2129643800}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7743":{"gid":7743,"name":"c","rank":1,"incarnation":96,"state":"up:active","state_seq":7,"addr":"172.21.15.143:6834/2489252684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":2489252684},{"type":"v1","addr":"172.21.15.143:6834","nonce":2489252684}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:30:58.095 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.093+0000 7f5c87e73700 1 -- 172.21.15.143:0/4077173623 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f5c6406af00 msgr2=0x7f5c6406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:58.095 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.093+0000 7f5c87e73700 1 --2- 172.21.15.143:0/4077173623 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f5c6406af00 0x7f5c6406d3c0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f5c801e4da0 tx=0x7f5c70008040 comp rx=0 tx=0).stop 2023-11-08T19:30:58.096 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.093+0000 7f5c87e73700 1 -- 172.21.15.143:0/4077173623 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c8006ce10 msgr2=0x7f5c801d8ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:58.096 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.093+0000 7f5c87e73700 1 --2- 172.21.15.143:0/4077173623 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c8006ce10 0x7f5c801d8ce0 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c0010e0 tx=0x7f5c7c00cae0 comp rx=0 tx=0).stop 2023-11-08T19:30:58.096 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.093+0000 7f5c87e73700 1 -- 172.21.15.143:0/4077173623 shutdown_connections 2023-11-08T19:30:58.096 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.093+0000 7f5c87e73700 1 --2- 172.21.15.143:0/4077173623 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f5c6406af00 0x7f5c6406d3c0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:58.096 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.093+0000 7f5c87e73700 1 --2- 172.21.15.143:0/4077173623 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5c800b2f30 0x7f5c801df120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:58.097 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.093+0000 7f5c87e73700 1 --2- 172.21.15.143:0/4077173623 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c8006ce10 0x7f5c801d8ce0 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:58.097 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.093+0000 7f5c87e73700 1 --2- 172.21.15.143:0/4077173623 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5c800b2b50 0x7f5c801daf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:58.097 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.093+0000 7f5c87e73700 1 -- 172.21.15.143:0/4077173623 >> 172.21.15.143:0/4077173623 conn(0x7f5c8009aa70 msgr2=0x7f5c801dbdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:58.097 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.093+0000 7f5c87e73700 1 -- 172.21.15.143:0/4077173623 shutdown_connections 2023-11-08T19:30:58.097 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.093+0000 7f5c87e73700 1 -- 172.21.15.143:0/4077173623 wait complete. 2023-11-08T19:30:58.098 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 100 2023-11-08T19:30:58.106 DEBUG:tasks.cephfs.filesystem:Looked up MDS state for mds.0: up:active 2023-11-08T19:30:58.107 DEBUG:tasks.cephfs.filesystem:reached state 'up:active' in 4.57208251953125s 2023-11-08T19:30:58.107 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:30:58.266 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.264+0000 7fee585d4700 1 Processor -- start 2023-11-08T19:30:58.266 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.264+0000 7fee585d4700 1 -- start start 2023-11-08T19:30:58.268 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.265+0000 7fee585d4700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fee5015acd0 0x7fee5015f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:58.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.265+0000 7fee585d4700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee5006f780 con 0x7fee500ccea0 2023-11-08T19:30:58.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.265+0000 7fee585d4700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee5006e8e0 con 0x7fee5015a8f0 2023-11-08T19:30:58.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.265+0000 7fee585d4700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee5006eb70 con 0x7fee5015acd0 2023-11-08T19:30:58.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.265+0000 7fee55b6f700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fee5015acd0 0x7fee5015f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:58.270 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.266+0000 7fee56b71700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fee500ccea0 0x7fee500cd2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:52778/0 (socket says 172.21.15.143:52778) 2023-11-08T19:30:58.270 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.266+0000 7fee55b6f700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fee5015acd0 0x7fee5015f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56622/0 (socket says 172.21.15.143:56622) 2023-11-08T19:30:58.270 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.266+0000 7fee56b71700 1 -- 172.21.15.143:0/783582075 learned_addr learned my addr 172.21.15.143:0/783582075 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:30:58.270 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.266+0000 7fee55b6f700 1 -- 172.21.15.143:0/783582075 >> v1:172.21.15.174:6789/0 conn(0x7fee5015a8f0 legacy=0x7fee5006e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:58.270 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.266+0000 7fee55b6f700 1 -- 172.21.15.143:0/783582075 >> v1:172.21.15.143:6789/0 conn(0x7fee500ccea0 legacy=0x7fee500cd2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:58.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.266+0000 7fee5536e700 1 -- 172.21.15.143:0/783582075 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1319934585 0 0) 0x7fee5006f780 con 0x7fee500ccea0 2023-11-08T19:30:58.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.266+0000 7fee55b6f700 1 -- 172.21.15.143:0/783582075 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee5006ee50 con 0x7fee5015acd0 2023-11-08T19:30:58.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.266+0000 7fee55b6f700 1 --2- 172.21.15.143:0/783582075 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fee5015acd0 0x7fee5015f080 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7fee4c009840 tx=0x7fee4c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5abc7c18ae009e7 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:58.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.266+0000 7fee5536e700 1 -- 172.21.15.143:0/783582075 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee4c004230 con 0x7fee5015acd0 2023-11-08T19:30:58.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.266+0000 7fee5536e700 1 -- 172.21.15.143:0/783582075 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee4c004390 con 0x7fee5015acd0 2023-11-08T19:30:58.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.266+0000 7fee5536e700 1 -- 172.21.15.143:0/783582075 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee4c0036a0 con 0x7fee5015acd0 2023-11-08T19:30:58.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.267+0000 7fee585d4700 1 -- 172.21.15.143:0/783582075 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fee5015acd0 msgr2=0x7fee5015f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:58.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.267+0000 7fee585d4700 1 --2- 172.21.15.143:0/783582075 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fee5015acd0 0x7fee5015f080 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7fee4c009840 tx=0x7fee4c009b50 comp rx=0 tx=0).stop 2023-11-08T19:30:58.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.267+0000 7fee585d4700 1 -- 172.21.15.143:0/783582075 shutdown_connections 2023-11-08T19:30:58.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.267+0000 7fee585d4700 1 --2- 172.21.15.143:0/783582075 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fee5015acd0 0x7fee5015f080 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:58.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.267+0000 7fee585d4700 1 -- 172.21.15.143:0/783582075 >> 172.21.15.143:0/783582075 conn(0x7fee500b23d0 msgr2=0x7fee500b27b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:58.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.267+0000 7fee585d4700 1 -- 172.21.15.143:0/783582075 shutdown_connections 2023-11-08T19:30:58.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.267+0000 7fee585d4700 1 -- 172.21.15.143:0/783582075 wait complete. 2023-11-08T19:30:58.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.268+0000 7fee585d4700 1 Processor -- start 2023-11-08T19:30:58.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.268+0000 7fee585d4700 1 -- start start 2023-11-08T19:30:58.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.269+0000 7fee585d4700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fee500ccea0 0x7fee50153650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:58.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.269+0000 7fee585d4700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fee5015a8f0 0x7fee50154b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:58.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.269+0000 7fee585d4700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fee5015acd0 0x7fee500d07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:58.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.269+0000 7fee585d4700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee50155a80 con 0x7fee5015a8f0 2023-11-08T19:30:58.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.269+0000 7fee585d4700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee501554c0 con 0x7fee500ccea0 2023-11-08T19:30:58.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.269+0000 7fee585d4700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee501557a0 con 0x7fee5015acd0 2023-11-08T19:30:58.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.269+0000 7fee55b6f700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fee5015a8f0 0x7fee50154b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:58.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.269+0000 7fee55b6f700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fee5015a8f0 0x7fee50154b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56628/0 (socket says 172.21.15.143:56628) 2023-11-08T19:30:58.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.269+0000 7fee55b6f700 1 -- 172.21.15.143:0/1482997394 learned_addr learned my addr 172.21.15.143:0/1482997394 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:30:58.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.269+0000 7fee56370700 1 --2- 172.21.15.143:0/1482997394 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fee500ccea0 0x7fee50153650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:58.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.269+0000 7fee56b71700 1 --2- 172.21.15.143:0/1482997394 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fee5015acd0 0x7fee500d07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:58.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.269+0000 7fee55b6f700 1 -- 172.21.15.143:0/1482997394 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fee5015acd0 msgr2=0x7fee500d07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:58.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.269+0000 7fee55b6f700 1 --2- 172.21.15.143:0/1482997394 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fee5015acd0 0x7fee500d07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:58.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.269+0000 7fee55b6f700 1 -- 172.21.15.143:0/1482997394 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fee500ccea0 msgr2=0x7fee50153650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:58.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.269+0000 7fee55b6f700 1 --2- 172.21.15.143:0/1482997394 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fee500ccea0 0x7fee50153650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:58.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.269+0000 7fee55b6f700 1 -- 172.21.15.143:0/1482997394 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee4c0094a0 con 0x7fee5015a8f0 2023-11-08T19:30:58.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.269+0000 7fee55b6f700 1 --2- 172.21.15.143:0/1482997394 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fee5015a8f0 0x7fee50154b50 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7fee4c0090b0 tx=0x7fee4c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:58.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.269+0000 7fee46ffd700 1 -- 172.21.15.143:0/1482997394 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee4c011630 con 0x7fee5015a8f0 2023-11-08T19:30:58.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.269+0000 7fee46ffd700 1 -- 172.21.15.143:0/1482997394 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee4c011790 con 0x7fee5015a8f0 2023-11-08T19:30:58.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.269+0000 7fee585d4700 1 -- 172.21.15.143:0/1482997394 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee501e4eb0 con 0x7fee5015a8f0 2023-11-08T19:30:58.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.269+0000 7fee46ffd700 1 -- 172.21.15.143:0/1482997394 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee4c011b40 con 0x7fee5015a8f0 2023-11-08T19:30:58.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.270+0000 7fee585d4700 1 -- 172.21.15.143:0/1482997394 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee50153b90 con 0x7fee5015a8f0 2023-11-08T19:30:58.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.270+0000 7fee585d4700 1 -- 172.21.15.143:0/1482997394 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee501e4d10 con 0x7fee5015a8f0 2023-11-08T19:30:58.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.271+0000 7fee46ffd700 1 -- 172.21.15.143:0/1482997394 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fee4c011ca0 con 0x7fee5015a8f0 2023-11-08T19:30:58.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.271+0000 7fee46ffd700 1 --2- 172.21.15.143:0/1482997394 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fee3406af00 0x7fee3406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:58.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.271+0000 7fee46ffd700 1 -- 172.21.15.143:0/1482997394 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 7360+0+0 (secure 0 0 0) 0x7fee4c08b530 con 0x7fee5015a8f0 2023-11-08T19:30:58.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.271+0000 7fee56370700 1 --2- 172.21.15.143:0/1482997394 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fee3406af00 0x7fee3406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:58.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.272+0000 7fee56370700 1 --2- 172.21.15.143:0/1482997394 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fee3406af00 0x7fee3406d3c0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fee40007fd0 tx=0x7fee40007ec0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:58.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.276+0000 7fee46ffd700 1 -- 172.21.15.143:0/1482997394 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fee501e4d10 con 0x7fee5015a8f0 2023-11-08T19:30:58.508 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.506+0000 7fee585d4700 1 -- 172.21.15.143:0/1482997394 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fee501e53d0 con 0x7fee5015a8f0 2023-11-08T19:30:58.508 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.507+0000 7fee46ffd700 1 -- 172.21.15.143:0/1482997394 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 100 v100) v1 ==== 95+0+4946 (secure 0 0 0) 0x7fee501e53d0 con 0x7fee5015a8f0 2023-11-08T19:30:58.509 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:30:58.509 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":100,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5933,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/3918286710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6835","nonce":3918286710},{"type":"v1","addr":"172.21.15.174:6837","nonce":3918286710}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":6245,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6836/2689884850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2689884850},{"type":"v1","addr":"172.21.15.174:6836","nonce":2689884850}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":97}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:03.092245+0000","modified":"2023-11-08T19:30:56.788736+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":103,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6161,"mds_1":7743},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6161":{"gid":6161,"name":"a","rank":0,"incarnation":93,"state":"up:active","state_seq":14,"addr":"172.21.15.143:6835/2129643800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":2129643800},{"type":"v1","addr":"172.21.15.143:6835","nonce":2129643800}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7743":{"gid":7743,"name":"c","rank":1,"incarnation":96,"state":"up:active","state_seq":7,"addr":"172.21.15.143:6834/2489252684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":2489252684},{"type":"v1","addr":"172.21.15.143:6834","nonce":2489252684}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:30:58.510 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.509+0000 7fee585d4700 1 -- 172.21.15.143:0/1482997394 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fee3406af00 msgr2=0x7fee3406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:58.511 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.509+0000 7fee585d4700 1 --2- 172.21.15.143:0/1482997394 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fee3406af00 0x7fee3406d3c0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fee40007fd0 tx=0x7fee40007ec0 comp rx=0 tx=0).stop 2023-11-08T19:30:58.511 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.509+0000 7fee585d4700 1 -- 172.21.15.143:0/1482997394 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fee5015a8f0 msgr2=0x7fee50154b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:58.511 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.509+0000 7fee585d4700 1 --2- 172.21.15.143:0/1482997394 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fee5015a8f0 0x7fee50154b50 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7fee4c0090b0 tx=0x7fee4c0047c0 comp rx=0 tx=0).stop 2023-11-08T19:30:58.511 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.509+0000 7fee585d4700 1 -- 172.21.15.143:0/1482997394 shutdown_connections 2023-11-08T19:30:58.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.509+0000 7fee585d4700 1 --2- 172.21.15.143:0/1482997394 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fee3406af00 0x7fee3406d3c0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:58.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.509+0000 7fee585d4700 1 --2- 172.21.15.143:0/1482997394 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fee5015acd0 0x7fee500d07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:58.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.509+0000 7fee585d4700 1 --2- 172.21.15.143:0/1482997394 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fee500ccea0 0x7fee50153650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:58.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.509+0000 7fee585d4700 1 --2- 172.21.15.143:0/1482997394 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fee5015a8f0 0x7fee50154b50 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:58.513 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.509+0000 7fee585d4700 1 -- 172.21.15.143:0/1482997394 >> 172.21.15.143:0/1482997394 conn(0x7fee500b23d0 msgr2=0x7fee500b6100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:58.513 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.509+0000 7fee585d4700 1 -- 172.21.15.143:0/1482997394 shutdown_connections 2023-11-08T19:30:58.513 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.509+0000 7fee585d4700 1 -- 172.21.15.143:0/1482997394 wait complete. 2023-11-08T19:30:58.513 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 100 2023-11-08T19:30:58.522 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json dump snaps --server 2023-11-08T19:30:58.589 INFO:tasks.ceph.mds.a.smithi143.stderr:2023-11-08T19:30:58.587+0000 7f257a6f9700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-08T19:30:58.589 INFO:tasks.ceph.mds.a.smithi143.stderr:2023-11-08T19:30:58.587+0000 7f257a6f9700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-11-08T19:30:58.659 INFO:teuthology.orchestra.run.smithi143.stdout:{"last_snap":10,"last_created":5,"last_destroyed":9,"pending_noop":[],"snaps":[],"need_to_purge":{},"pending_update":[],"pending_destroy":[]} 2023-11-08T19:30:58.660 DEBUG:tasks.cephfs.filesystem:_json_asok output { "last_created": 5, "last_destroyed": 9, "last_snap": 10, "need_to_purge": {}, "pending_destroy": [], "pending_noop": [], "pending_update": [], "snaps": [] } 2023-11-08T19:30:58.660 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-11-08T19:30:58.661 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:30:58.783 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.782+0000 7fe69b5d8700 1 Processor -- start 2023-11-08T19:30:58.784 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.782+0000 7fe69b5d8700 1 -- start start 2023-11-08T19:30:58.786 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.783+0000 7fe69b5d8700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fe6940d23f0 0x7fe69415dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:58.786 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.783+0000 7fe69b5d8700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe69406cdf0 0x7fe6940b2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:58.786 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.783+0000 7fe69b5d8700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6940d00e0 con 0x7fe69415db00 2023-11-08T19:30:58.787 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.783+0000 7fe69b5d8700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6940d03c0 con 0x7fe69406cdf0 2023-11-08T19:30:58.787 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.783+0000 7fe69b5d8700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6940d06a0 con 0x7fe6940d23f0 2023-11-08T19:30:58.787 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.783+0000 7fe699b75700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe69406cdf0 0x7fe6940b2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:58.787 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.783+0000 7fe699374700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fe69415db00 0x7fe69406c6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:52788/0 (socket says 172.21.15.143:52788) 2023-11-08T19:30:58.787 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.783+0000 7fe699374700 1 -- 172.21.15.143:0/2696658243 learned_addr learned my addr 172.21.15.143:0/2696658243 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:30:58.788 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.783+0000 7fe698b73700 1 --2- 172.21.15.143:0/2696658243 >> v2:172.21.15.174:3300/0 conn(0x7fe6940d23f0 0x7fe69415dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:58.788 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.783+0000 7fe699b75700 1 -- 172.21.15.143:0/2696658243 >> v2:172.21.15.174:3300/0 conn(0x7fe6940d23f0 msgr2=0x7fe69415dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:58.788 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.783+0000 7fe699b75700 1 --2- 172.21.15.143:0/2696658243 >> v2:172.21.15.174:3300/0 conn(0x7fe6940d23f0 0x7fe69415dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:58.788 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.783+0000 7fe699b75700 1 -- 172.21.15.143:0/2696658243 >> v1:172.21.15.143:6789/0 conn(0x7fe69415db00 legacy=0x7fe69406c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:58.788 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.783+0000 7fe699b75700 1 -- 172.21.15.143:0/2696658243 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6940d0980 con 0x7fe69406cdf0 2023-11-08T19:30:58.789 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.783+0000 7fe68bfff700 1 -- 172.21.15.143:0/2696658243 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3390706863 0 0) 0x7fe6940d00e0 con 0x7fe69415db00 2023-11-08T19:30:58.789 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.783+0000 7fe699b75700 1 --2- 172.21.15.143:0/2696658243 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe69406cdf0 0x7fe6940b2500 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7fe694071e80 tx=0x7fe68c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2900322ef3bd3c0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:58.789 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.784+0000 7fe68bfff700 1 -- 172.21.15.143:0/2696658243 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe68c00e070 con 0x7fe69406cdf0 2023-11-08T19:30:58.789 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.784+0000 7fe68bfff700 1 -- 172.21.15.143:0/2696658243 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe68c00bbd0 con 0x7fe69406cdf0 2023-11-08T19:30:58.789 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.784+0000 7fe68bfff700 1 -- 172.21.15.143:0/2696658243 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe68c012430 con 0x7fe69406cdf0 2023-11-08T19:30:58.789 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.784+0000 7fe69b5d8700 1 -- 172.21.15.143:0/2696658243 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe69406cdf0 msgr2=0x7fe6940b2500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:58.790 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.784+0000 7fe69b5d8700 1 --2- 172.21.15.143:0/2696658243 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe69406cdf0 0x7fe6940b2500 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7fe694071e80 tx=0x7fe68c00b330 comp rx=0 tx=0).stop 2023-11-08T19:30:58.790 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.784+0000 7fe69b5d8700 1 -- 172.21.15.143:0/2696658243 shutdown_connections 2023-11-08T19:30:58.790 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.784+0000 7fe69b5d8700 1 --2- 172.21.15.143:0/2696658243 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe69406cdf0 0x7fe6940b2500 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:58.790 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.784+0000 7fe69b5d8700 1 --2- 172.21.15.143:0/2696658243 >> v2:172.21.15.174:3300/0 conn(0x7fe6940d23f0 0x7fe69415dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:58.791 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.784+0000 7fe69b5d8700 1 -- 172.21.15.143:0/2696658243 >> 172.21.15.143:0/2696658243 conn(0x7fe6940d18d0 msgr2=0x7fe6940d1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:58.791 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.784+0000 7fe69b5d8700 1 -- 172.21.15.143:0/2696658243 shutdown_connections 2023-11-08T19:30:58.791 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.784+0000 7fe69b5d8700 1 -- 172.21.15.143:0/2696658243 wait complete. 2023-11-08T19:30:58.794 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.785+0000 7fe69b5d8700 1 Processor -- start 2023-11-08T19:30:58.794 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.785+0000 7fe69b5d8700 1 -- start start 2023-11-08T19:30:58.794 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.786+0000 7fe69b5d8700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe69406cdf0 0x7fe69414fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:58.794 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.786+0000 7fe69b5d8700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe6940d23f0 0x7fe6940cb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:58.795 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.786+0000 7fe69b5d8700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe69415db00 0x7fe6940cbb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:58.795 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.786+0000 7fe69b5d8700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6940ca9b0 con 0x7fe69406cdf0 2023-11-08T19:30:58.795 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.786+0000 7fe69b5d8700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe694150300 con 0x7fe69415db00 2023-11-08T19:30:58.795 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.786+0000 7fe69b5d8700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6940cadd0 con 0x7fe6940d23f0 2023-11-08T19:30:58.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.786+0000 7fe699374700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe69406cdf0 0x7fe69414fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:58.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.786+0000 7fe699374700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe69406cdf0 0x7fe69414fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56640/0 (socket says 172.21.15.143:56640) 2023-11-08T19:30:58.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.786+0000 7fe699374700 1 -- 172.21.15.143:0/1088913002 learned_addr learned my addr 172.21.15.143:0/1088913002 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:30:58.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.787+0000 7fe699b75700 1 --2- 172.21.15.143:0/1088913002 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe69415db00 0x7fe6940cbb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:58.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.787+0000 7fe698b73700 1 --2- 172.21.15.143:0/1088913002 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe6940d23f0 0x7fe6940cb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:58.797 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.787+0000 7fe699374700 1 -- 172.21.15.143:0/1088913002 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe6940d23f0 msgr2=0x7fe6940cb5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:58.797 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.787+0000 7fe699374700 1 --2- 172.21.15.143:0/1088913002 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe6940d23f0 0x7fe6940cb5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:58.797 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.787+0000 7fe699374700 1 -- 172.21.15.143:0/1088913002 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe69415db00 msgr2=0x7fe6940cbb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:58.797 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.787+0000 7fe699374700 1 --2- 172.21.15.143:0/1088913002 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe69415db00 0x7fe6940cbb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:58.797 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.787+0000 7fe699374700 1 -- 172.21.15.143:0/1088913002 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe68c009a70 con 0x7fe69406cdf0 2023-11-08T19:30:58.798 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.787+0000 7fe698b73700 1 --2- 172.21.15.143:0/1088913002 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe6940d23f0 0x7fe6940cb5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:30:58.798 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.787+0000 7fe699b75700 1 --2- 172.21.15.143:0/1088913002 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe69415db00 0x7fe6940cbb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:30:58.798 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.787+0000 7fe699374700 1 --2- 172.21.15.143:0/1088913002 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe69406cdf0 0x7fe69414fdc0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7fe69000c550 tx=0x7fe69000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:58.798 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.787+0000 7fe689ffb700 1 -- 172.21.15.143:0/1088913002 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe690004c80 con 0x7fe69406cdf0 2023-11-08T19:30:58.798 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.787+0000 7fe69b5d8700 1 -- 172.21.15.143:0/1088913002 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6940c9d30 con 0x7fe69406cdf0 2023-11-08T19:30:58.799 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.787+0000 7fe689ffb700 1 -- 172.21.15.143:0/1088913002 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe690004e00 con 0x7fe69406cdf0 2023-11-08T19:30:58.799 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.787+0000 7fe689ffb700 1 -- 172.21.15.143:0/1088913002 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe690007310 con 0x7fe69406cdf0 2023-11-08T19:30:58.799 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.787+0000 7fe69b5d8700 1 -- 172.21.15.143:0/1088913002 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6940ca2f0 con 0x7fe69406cdf0 2023-11-08T19:30:58.799 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.788+0000 7fe69b5d8700 1 -- 172.21.15.143:0/1088913002 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6940cac20 con 0x7fe69406cdf0 2023-11-08T19:30:58.799 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.788+0000 7fe689ffb700 1 -- 172.21.15.143:0/1088913002 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe690007470 con 0x7fe69406cdf0 2023-11-08T19:30:58.800 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.789+0000 7fe689ffb700 1 --2- 172.21.15.143:0/1088913002 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fe67806af30 0x7fe67806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:58.800 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.789+0000 7fe698b73700 1 --2- 172.21.15.143:0/1088913002 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fe67806af30 0x7fe67806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:58.800 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.790+0000 7fe698b73700 1 --2- 172.21.15.143:0/1088913002 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fe67806af30 0x7fe67806d3f0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fe69409eae0 tx=0x7fe684008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:58.800 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.790+0000 7fe689ffb700 1 -- 172.21.15.143:0/1088913002 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 7360+0+0 (secure 0 0 0) 0x7fe690058790 con 0x7fe69406cdf0 2023-11-08T19:30:58.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:58.795+0000 7fe689ffb700 1 -- 172.21.15.143:0/1088913002 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe6900543d0 con 0x7fe69406cdf0 2023-11-08T19:30:59.026 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:59.025+0000 7fe69b5d8700 1 -- 172.21.15.143:0/1088913002 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe69404f730 con 0x7fe69406cdf0 2023-11-08T19:30:59.027 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:59.025+0000 7fe689ffb700 1 -- 172.21.15.143:0/1088913002 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 100 v100) v1 ==== 95+0+4946 (secure 0 0 0) 0x7fe6900579f0 con 0x7fe69406cdf0 2023-11-08T19:30:59.027 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:30:59.027 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":100,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5933,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/3918286710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6835","nonce":3918286710},{"type":"v1","addr":"172.21.15.174:6837","nonce":3918286710}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":6245,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6836/2689884850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2689884850},{"type":"v1","addr":"172.21.15.174:6836","nonce":2689884850}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":97}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:03.092245+0000","modified":"2023-11-08T19:30:56.788736+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":103,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6161,"mds_1":7743},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6161":{"gid":6161,"name":"a","rank":0,"incarnation":93,"state":"up:active","state_seq":14,"addr":"172.21.15.143:6835/2129643800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":2129643800},{"type":"v1","addr":"172.21.15.143:6835","nonce":2129643800}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7743":{"gid":7743,"name":"c","rank":1,"incarnation":96,"state":"up:active","state_seq":7,"addr":"172.21.15.143:6834/2489252684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":2489252684},{"type":"v1","addr":"172.21.15.143:6834","nonce":2489252684}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:30:59.029 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:59.027+0000 7fe69b5d8700 1 -- 172.21.15.143:0/1088913002 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fe67806af30 msgr2=0x7fe67806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:59.029 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:59.027+0000 7fe69b5d8700 1 --2- 172.21.15.143:0/1088913002 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fe67806af30 0x7fe67806d3f0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fe69409eae0 tx=0x7fe684008040 comp rx=0 tx=0).stop 2023-11-08T19:30:59.029 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:59.027+0000 7fe69b5d8700 1 -- 172.21.15.143:0/1088913002 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe69406cdf0 msgr2=0x7fe69414fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:59.029 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:59.027+0000 7fe69b5d8700 1 --2- 172.21.15.143:0/1088913002 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe69406cdf0 0x7fe69414fdc0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7fe69000c550 tx=0x7fe69000c910 comp rx=0 tx=0).stop 2023-11-08T19:30:59.030 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:59.027+0000 7fe69b5d8700 1 -- 172.21.15.143:0/1088913002 shutdown_connections 2023-11-08T19:30:59.030 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:59.027+0000 7fe69b5d8700 1 --2- 172.21.15.143:0/1088913002 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fe67806af30 0x7fe67806d3f0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:59.030 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:59.027+0000 7fe69b5d8700 1 --2- 172.21.15.143:0/1088913002 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe69415db00 0x7fe6940cbb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:59.030 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:59.027+0000 7fe69b5d8700 1 --2- 172.21.15.143:0/1088913002 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe69406cdf0 0x7fe69414fdc0 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:59.030 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:59.027+0000 7fe69b5d8700 1 --2- 172.21.15.143:0/1088913002 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe6940d23f0 0x7fe6940cb5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:59.031 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:59.027+0000 7fe69b5d8700 1 -- 172.21.15.143:0/1088913002 >> 172.21.15.143:0/1088913002 conn(0x7fe6940d18d0 msgr2=0x7fe6940d1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:59.031 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:59.027+0000 7fe69b5d8700 1 -- 172.21.15.143:0/1088913002 shutdown_connections 2023-11-08T19:30:59.031 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:30:59.027+0000 7fe69b5d8700 1 -- 172.21.15.143:0/1088913002 wait complete. 2023-11-08T19:30:59.031 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 100 2023-11-08T19:30:59.041 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json dump snaps --server 2023-11-08T19:30:59.178 INFO:teuthology.orchestra.run.smithi143.stdout:{"last_snap":10,"last_created":5,"last_destroyed":9,"pending_noop":[],"snaps":[],"need_to_purge":{},"pending_update":[],"pending_destroy":[]} 2023-11-08T19:30:59.178 DEBUG:tasks.cephfs.filesystem:_json_asok output { "last_created": 5, "last_destroyed": 9, "last_snap": 10, "need_to_purge": {}, "pending_destroy": [], "pending_noop": [], "pending_update": [], "snaps": [] } 2023-11-08T19:30:59.179 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-08T19:30:59.179 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:30:59.180 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip link set brx.0 up 2023-11-08T19:31:02.917 INFO:teuthology.orchestra.run.smithi143.stderr:mkdir: cannot create directory ‘d1/dir/.snap/s22’: Permission denied 2023-11-08T19:31:02.918 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-08T19:31:02.919 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-08T19:31:02.919 DEBUG:teuthology.orchestra.run.smithi143:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-08T19:31:02.973 DEBUG:teuthology.orchestra.run.smithi174:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-08T19:31:03.019 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-08T19:31:03.020 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-08T19:31:03.020 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:31:03.038 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-08T19:31:03.038 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:31:03.095 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:31:03.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.254+0000 7fd44709a700 1 Processor -- start 2023-11-08T19:31:03.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.254+0000 7fd44709a700 1 -- start start 2023-11-08T19:31:03.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.255+0000 7fd44709a700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd4400ccf30 0x7fd44015dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:03.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.255+0000 7fd44709a700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4400d00e0 con 0x7fd4400d23f0 2023-11-08T19:31:03.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.255+0000 7fd44709a700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4400d03c0 con 0x7fd44015db00 2023-11-08T19:31:03.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.255+0000 7fd44709a700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4400d06a0 con 0x7fd4400ccf30 2023-11-08T19:31:03.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.255+0000 7fd445637700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd4400ccf30 0x7fd44015dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:03.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.255+0000 7fd43ffff700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fd4400d23f0 0x7fd4400b2500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:52816/0 (socket says 172.21.15.143:52816) 2023-11-08T19:31:03.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.255+0000 7fd43ffff700 1 -- 172.21.15.143:0/1789753506 learned_addr learned my addr 172.21.15.143:0/1789753506 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:31:03.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.255+0000 7fd445637700 1 -- 172.21.15.143:0/1789753506 >> v1:172.21.15.174:6789/0 conn(0x7fd44015db00 legacy=0x7fd44006c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:03.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.255+0000 7fd445637700 1 -- 172.21.15.143:0/1789753506 >> v1:172.21.15.143:6789/0 conn(0x7fd4400d23f0 legacy=0x7fd4400b2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:03.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.255+0000 7fd445637700 1 -- 172.21.15.143:0/1789753506 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4400d0980 con 0x7fd4400ccf30 2023-11-08T19:31:03.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.255+0000 7fd43f7fe700 1 -- 172.21.15.143:0/1789753506 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3040009654 0 0) 0x7fd4400d00e0 con 0x7fd4400d23f0 2023-11-08T19:31:03.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.255+0000 7fd445637700 1 --2- 172.21.15.143:0/1789753506 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd4400ccf30 0x7fd44015dee0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7fd438009d90 tx=0x7fd43800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8af2eaf3f875209a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:03.261 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.256+0000 7fd43f7fe700 1 -- 172.21.15.143:0/1789753506 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd43800e070 con 0x7fd4400ccf30 2023-11-08T19:31:03.261 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.256+0000 7fd43f7fe700 1 -- 172.21.15.143:0/1789753506 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd43800bbd0 con 0x7fd4400ccf30 2023-11-08T19:31:03.261 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.256+0000 7fd43f7fe700 1 -- 172.21.15.143:0/1789753506 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd438012430 con 0x7fd4400ccf30 2023-11-08T19:31:03.261 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.256+0000 7fd44709a700 1 -- 172.21.15.143:0/1789753506 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd4400ccf30 msgr2=0x7fd44015dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:03.261 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.256+0000 7fd44709a700 1 --2- 172.21.15.143:0/1789753506 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd4400ccf30 0x7fd44015dee0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7fd438009d90 tx=0x7fd43800b330 comp rx=0 tx=0).stop 2023-11-08T19:31:03.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.256+0000 7fd44709a700 1 -- 172.21.15.143:0/1789753506 shutdown_connections 2023-11-08T19:31:03.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.256+0000 7fd44709a700 1 --2- 172.21.15.143:0/1789753506 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd4400ccf30 0x7fd44015dee0 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:03.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.256+0000 7fd44709a700 1 -- 172.21.15.143:0/1789753506 >> 172.21.15.143:0/1789753506 conn(0x7fd4400d18d0 msgr2=0x7fd4400d1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:03.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.256+0000 7fd44709a700 1 -- 172.21.15.143:0/1789753506 shutdown_connections 2023-11-08T19:31:03.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.256+0000 7fd44709a700 1 -- 172.21.15.143:0/1789753506 wait complete. 2023-11-08T19:31:03.265 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.258+0000 7fd44709a700 1 Processor -- start 2023-11-08T19:31:03.265 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.258+0000 7fd44709a700 1 -- start start 2023-11-08T19:31:03.266 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.259+0000 7fd44709a700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd4400ccf30 0x7fd4400ca370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:03.266 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.259+0000 7fd44709a700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd4400d23f0 0x7fd44015cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:03.266 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.259+0000 7fd44709a700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd44015db00 0x7fd44014ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:03.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.259+0000 7fd44709a700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd44014b1b0 con 0x7fd44015db00 2023-11-08T19:31:03.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.259+0000 7fd44709a700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4400caa80 con 0x7fd4400d23f0 2023-11-08T19:31:03.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.259+0000 7fd44709a700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4400cad60 con 0x7fd4400ccf30 2023-11-08T19:31:03.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.259+0000 7fd445637700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd44015db00 0x7fd44014ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:03.268 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.259+0000 7fd445637700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd44015db00 0x7fd44014ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56670/0 (socket says 172.21.15.143:56670) 2023-11-08T19:31:03.268 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.259+0000 7fd445637700 1 -- 172.21.15.143:0/2757378397 learned_addr learned my addr 172.21.15.143:0/2757378397 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:31:03.268 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.259+0000 7fd43ffff700 1 --2- 172.21.15.143:0/2757378397 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd4400d23f0 0x7fd44015cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:03.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.259+0000 7fd445637700 1 -- 172.21.15.143:0/2757378397 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd4400ccf30 msgr2=0x7fd4400ca370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:03.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.259+0000 7fd444e36700 1 --2- 172.21.15.143:0/2757378397 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd4400ccf30 0x7fd4400ca370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:03.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.259+0000 7fd445637700 1 --2- 172.21.15.143:0/2757378397 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd4400ccf30 0x7fd4400ca370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:03.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.259+0000 7fd445637700 1 -- 172.21.15.143:0/2757378397 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd4400d23f0 msgr2=0x7fd44015cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:03.270 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.259+0000 7fd445637700 1 --2- 172.21.15.143:0/2757378397 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd4400d23f0 0x7fd44015cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:03.270 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.259+0000 7fd445637700 1 -- 172.21.15.143:0/2757378397 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd438009a70 con 0x7fd44015db00 2023-11-08T19:31:03.270 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.259+0000 7fd43ffff700 1 --2- 172.21.15.143:0/2757378397 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd4400d23f0 0x7fd44015cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:31:03.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.259+0000 7fd445637700 1 --2- 172.21.15.143:0/2757378397 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd44015db00 0x7fd44014ac70 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7fd438014040 tx=0x7fd438004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:03.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.259+0000 7fd43d7fa700 1 -- 172.21.15.143:0/2757378397 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd43801b810 con 0x7fd44015db00 2023-11-08T19:31:03.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.259+0000 7fd43d7fa700 1 -- 172.21.15.143:0/2757378397 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd438020070 con 0x7fd44015db00 2023-11-08T19:31:03.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.259+0000 7fd44709a700 1 -- 172.21.15.143:0/2757378397 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4400cb040 con 0x7fd44015db00 2023-11-08T19:31:03.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.259+0000 7fd43d7fa700 1 -- 172.21.15.143:0/2757378397 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd43801bbb0 con 0x7fd44015db00 2023-11-08T19:31:03.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.259+0000 7fd44709a700 1 -- 172.21.15.143:0/2757378397 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4400cb600 con 0x7fd44015db00 2023-11-08T19:31:03.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.260+0000 7fd44709a700 1 -- 172.21.15.143:0/2757378397 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd44004f730 con 0x7fd44015db00 2023-11-08T19:31:03.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.261+0000 7fd43d7fa700 1 -- 172.21.15.143:0/2757378397 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd43801bd10 con 0x7fd44015db00 2023-11-08T19:31:03.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.261+0000 7fd43d7fa700 1 --2- 172.21.15.143:0/2757378397 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd42406af00 0x7fd42406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:03.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.261+0000 7fd43d7fa700 1 -- 172.21.15.143:0/2757378397 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 7360+0+0 (secure 0 0 0) 0x7fd438022070 con 0x7fd44015db00 2023-11-08T19:31:03.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.261+0000 7fd444e36700 1 --2- 172.21.15.143:0/2757378397 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd42406af00 0x7fd42406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:03.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.262+0000 7fd444e36700 1 --2- 172.21.15.143:0/2757378397 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd42406af00 0x7fd42406d3c0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fd434000f60 tx=0x7fd434009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:03.274 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.266+0000 7fd43d7fa700 1 -- 172.21.15.143:0/2757378397 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd43805a4d0 con 0x7fd44015db00 2023-11-08T19:31:03.457 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.456+0000 7fd44709a700 1 -- 172.21.15.143:0/2757378397 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd44006b550 con 0x7fd44015db00 2023-11-08T19:31:03.459 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.457+0000 7fd43d7fa700 1 -- 172.21.15.143:0/2757378397 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v104) v1 ==== 74+0+22721 (secure 0 0 0) 0x7fd43805a060 con 0x7fd44015db00 2023-11-08T19:31:03.459 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:31:03.459 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":104,"fsid":"968483af-77fc-4068-ad7c-5c2a1a4613d7","created":"2023-11-08T19:24:32.826484+0000","modified":"2023-11-08T19:30:55.774106+0000","last_up_change":"2023-11-08T19:24:43.573880+0000","last_in_change":"2023-11-08T19:24:35.314525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:44.551530+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"snap-test-pool","create_time":"2023-11-08T19:26:02.874384+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":36,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"test_data_pool","create_time":"2023-11-08T19:27:05.601949+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":66,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_metadata_pool","create_time":"2023-11-08T19:27:06.536369+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-11-08T19:28:01.533566+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-11-08T19:28:02.053445+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":9,"snap_epoch":100,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bae98ec1-d94e-403d-a220-a522a53ea6f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6809","nonce":1487830566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6811","nonce":1487830566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6815","nonce":1487830566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6813","nonce":1487830566}]},"public_addr":"172.21.15.143:6809/1487830566","cluster_addr":"172.21.15.143:6811/1487830566","heartbeat_back_addr":"172.21.15.143:6815/1487830566","heartbeat_front_addr":"172.21.15.143:6813/1487830566","state":["exists","up"]},{"osd":1,"uuid":"5cb6538a-1ccb-4322-8633-09f66af0b7e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6801","nonce":787341397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6803","nonce":787341397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6807","nonce":787341397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6805","nonce":787341397}]},"public_addr":"172.21.15.143:6801/787341397","cluster_addr":"172.21.15.143:6803/787341397","heartbeat_back_addr":"172.21.15.143:6807/787341397","heartbeat_front_addr":"172.21.15.143:6805/787341397","state":["exists","up"]},{"osd":2,"uuid":"cdbf522c-60ef-4d60-9653-8dcb529c101a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6822","nonce":2301325597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6826","nonce":2301325597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6831","nonce":2301325597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6829","nonce":2301325597}]},"public_addr":"172.21.15.143:6822/2301325597","cluster_addr":"172.21.15.143:6826/2301325597","heartbeat_back_addr":"172.21.15.143:6831/2301325597","heartbeat_front_addr":"172.21.15.143:6829/2301325597","state":["exists","up"]},{"osd":3,"uuid":"4e2783cf-ebb9-4dd8-8245-bc1d17ce4325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6817","nonce":2021504428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6819","nonce":2021504428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6827","nonce":2021504428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6823","nonce":2021504428}]},"public_addr":"172.21.15.143:6817/2021504428","cluster_addr":"172.21.15.143:6819/2021504428","heartbeat_back_addr":"172.21.15.143:6827/2021504428","heartbeat_front_addr":"172.21.15.143:6823/2021504428","state":["exists","up"]},{"osd":4,"uuid":"691256fa-9b77-49b9-a346-7ae1adf0ca80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6801","nonce":1993380595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6803","nonce":1993380595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6807","nonce":1993380595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6805","nonce":1993380595}]},"public_addr":"172.21.15.174:6801/1993380595","cluster_addr":"172.21.15.174:6803/1993380595","heartbeat_back_addr":"172.21.15.174:6807/1993380595","heartbeat_front_addr":"172.21.15.174:6805/1993380595","state":["exists","up"]},{"osd":5,"uuid":"756fc352-48c9-46ce-b52e-0541fe6f8525","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6809","nonce":1777923303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6811","nonce":1777923303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6815","nonce":1777923303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6813","nonce":1777923303}]},"public_addr":"172.21.15.174:6809/1777923303","cluster_addr":"172.21.15.174:6811/1777923303","heartbeat_back_addr":"172.21.15.174:6815/1777923303","heartbeat_front_addr":"172.21.15.174:6813/1777923303","state":["exists","up"]},{"osd":6,"uuid":"fab2ac2b-ed97-4cde-9825-b480eabf8b8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6825","nonce":2755130316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6827","nonce":2755130316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6831","nonce":2755130316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6829","nonce":2755130316}]},"public_addr":"172.21.15.174:6825/2755130316","cluster_addr":"172.21.15.174:6827/2755130316","heartbeat_back_addr":"172.21.15.174:6831/2755130316","heartbeat_front_addr":"172.21.15.174:6829/2755130316","state":["exists","up"]},{"osd":7,"uuid":"3de9b797-2ab6-4d41-a863-7e7d638fd703","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6817","nonce":2847141411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6819","nonce":2847141411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6823","nonce":2847141411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6821","nonce":2847141411}]},"public_addr":"172.21.15.174:6817/2847141411","cluster_addr":"172.21.15.174:6819/2847141411","heartbeat_back_addr":"172.21.15.174:6823/2847141411","heartbeat_front_addr":"172.21.15.174:6821/2847141411","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.477076+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.496890+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.539305+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.454673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.538881+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.521294+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.520712+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.540404+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.35","mappings":[{"from":5,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.174:6834/3601088618":"2023-11-09T19:30:47.736745+0000","172.21.15.143:6834/1535103237":"2023-11-09T19:30:40.630538+0000","172.21.15.143:6832/1535103237":"2023-11-09T19:30:40.630538+0000","172.21.15.143:0/2918728496":"2023-11-08T20:30:55.736484+0000","172.21.15.143:6835/3664584522":"2023-11-09T19:30:12.566205+0000","172.21.15.143:6833/3664584522":"2023-11-09T19:30:12.566205+0000","172.21.15.143:6832/3172664377":"2023-11-09T19:29:41.150862+0000","172.21.15.174:6836/3601088618":"2023-11-09T19:30:47.736745+0000","172.21.15.174:6837/2201447629":"2023-11-09T19:28:48.342407+0000","172.21.15.174:6835/2201447629":"2023-11-09T19:28:48.342407+0000","172.21.15.143:6833/3953059900":"2023-11-09T19:29:14.940251+0000","172.21.15.143:6835/3953059900":"2023-11-09T19:29:14.940251+0000","172.21.15.143:6834/3172664377":"2023-11-09T19:29:41.150862+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:31:03.461 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.460+0000 7fd44709a700 1 -- 172.21.15.143:0/2757378397 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd42406af00 msgr2=0x7fd42406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:03.462 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.460+0000 7fd44709a700 1 --2- 172.21.15.143:0/2757378397 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd42406af00 0x7fd42406d3c0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fd434000f60 tx=0x7fd434009040 comp rx=0 tx=0).stop 2023-11-08T19:31:03.462 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.460+0000 7fd44709a700 1 -- 172.21.15.143:0/2757378397 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd44015db00 msgr2=0x7fd44014ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:03.462 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.460+0000 7fd44709a700 1 --2- 172.21.15.143:0/2757378397 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd44015db00 0x7fd44014ac70 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7fd438014040 tx=0x7fd438004ad0 comp rx=0 tx=0).stop 2023-11-08T19:31:03.462 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.460+0000 7fd44709a700 1 -- 172.21.15.143:0/2757378397 shutdown_connections 2023-11-08T19:31:03.462 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.460+0000 7fd44709a700 1 --2- 172.21.15.143:0/2757378397 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd42406af00 0x7fd42406d3c0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:03.463 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.460+0000 7fd44709a700 1 --2- 172.21.15.143:0/2757378397 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd4400ccf30 0x7fd4400ca370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:03.463 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.460+0000 7fd44709a700 1 --2- 172.21.15.143:0/2757378397 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd44015db00 0x7fd44014ac70 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:03.463 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.460+0000 7fd44709a700 1 --2- 172.21.15.143:0/2757378397 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd4400d23f0 0x7fd44015cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:03.463 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.460+0000 7fd44709a700 1 -- 172.21.15.143:0/2757378397 >> 172.21.15.143:0/2757378397 conn(0x7fd4400d18d0 msgr2=0x7fd4400d1080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:03.463 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.460+0000 7fd44709a700 1 -- 172.21.15.143:0/2757378397 shutdown_connections 2023-11-08T19:31:03.464 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.460+0000 7fd44709a700 1 -- 172.21.15.143:0/2757378397 wait complete. 2023-11-08T19:31:03.474 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T19:24:44.551530+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'snap-test-pool', 'create_time': '2023-11-08T19:26:02.874384+0000', 'flags': 16385, 'flags_names': 'hashpspool,pool_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '36', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'pool', 'snap_seq': 2, 'snap_epoch': 36, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'test_data_pool', 'create_time': '2023-11-08T19:27:05.601949+0000', 'flags': 16385, 'flags_names': 'hashpspool,pool_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'pool', 'snap_seq': 2, 'snap_epoch': 66, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'test_metadata_pool', 'create_time': '2023-11-08T19:27:06.536369+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-08T19:28:01.533566+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2023-11-08T19:28:02.053445+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '100', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 9, 'snap_epoch': 100, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-08T19:31:03.475 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T19:31:03.599 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.598+0000 7fbff2d8c700 1 Processor -- start 2023-11-08T19:31:03.599 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.598+0000 7fbff2d8c700 1 -- start start 2023-11-08T19:31:03.601 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.598+0000 7fbff2d8c700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fbfec072110 0x7fbfec0b4990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:03.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.598+0000 7fbff2d8c700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fbfec0b4ed0 0x7fbfec0b5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:03.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.598+0000 7fbff2d8c700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfec071650 con 0x7fbfec15a800 2023-11-08T19:31:03.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.598+0000 7fbff2d8c700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfec0b2f80 con 0x7fbfec0b4ed0 2023-11-08T19:31:03.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.598+0000 7fbff2d8c700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfec0b3260 con 0x7fbfec072110 2023-11-08T19:31:03.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.599+0000 7fbff1329700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fbfec0b4ed0 0x7fbfec0b5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:03.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.599+0000 7fbff1329700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fbfec0b4ed0 0x7fbfec0b5340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56678/0 (socket says 172.21.15.143:56678) 2023-11-08T19:31:03.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.599+0000 7fbff1329700 1 -- 172.21.15.143:0/3470078590 learned_addr learned my addr 172.21.15.143:0/3470078590 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:31:03.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.599+0000 7fbfebfff700 1 --2- 172.21.15.143:0/3470078590 >> v2:172.21.15.174:3300/0 conn(0x7fbfec072110 0x7fbfec0b4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:03.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.599+0000 7fbff1329700 1 -- 172.21.15.143:0/3470078590 >> v2:172.21.15.174:3300/0 conn(0x7fbfec072110 msgr2=0x7fbfec0b4990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:03.604 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.599+0000 7fbff1329700 1 --2- 172.21.15.143:0/3470078590 >> v2:172.21.15.174:3300/0 conn(0x7fbfec072110 0x7fbfec0b4990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:03.604 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.599+0000 7fbff1329700 1 -- 172.21.15.143:0/3470078590 >> v1:172.21.15.174:6789/0 conn(0x7fbfec15a800 legacy=0x7fbfec0b2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:03.604 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.599+0000 7fbff1329700 1 -- 172.21.15.143:0/3470078590 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfec071910 con 0x7fbfec0b4ed0 2023-11-08T19:31:03.604 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.599+0000 7fbff1329700 1 --2- 172.21.15.143:0/3470078590 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbfec0b4ed0 0x7fbfec0b5340 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7fbfec06c9d0 tx=0x7fbfe400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbeaa627a4ddf1a2 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:03.604 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.599+0000 7fbfeb7fe700 1 -- 172.21.15.143:0/3470078590 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe400e070 con 0x7fbfec0b4ed0 2023-11-08T19:31:03.604 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.599+0000 7fbfeb7fe700 1 -- 172.21.15.143:0/3470078590 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfe400b910 con 0x7fbfec0b4ed0 2023-11-08T19:31:03.605 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.599+0000 7fbfeb7fe700 1 -- 172.21.15.143:0/3470078590 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe400bcc0 con 0x7fbfec0b4ed0 2023-11-08T19:31:03.605 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.600+0000 7fbff2d8c700 1 -- 172.21.15.143:0/3470078590 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbfec0b4ed0 msgr2=0x7fbfec0b5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:03.605 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.600+0000 7fbff2d8c700 1 --2- 172.21.15.143:0/3470078590 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbfec0b4ed0 0x7fbfec0b5340 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7fbfec06c9d0 tx=0x7fbfe400b330 comp rx=0 tx=0).stop 2023-11-08T19:31:03.605 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.600+0000 7fbff2d8c700 1 -- 172.21.15.143:0/3470078590 shutdown_connections 2023-11-08T19:31:03.605 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.600+0000 7fbff2d8c700 1 --2- 172.21.15.143:0/3470078590 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbfec0b4ed0 0x7fbfec0b5340 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:03.606 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.600+0000 7fbff2d8c700 1 --2- 172.21.15.143:0/3470078590 >> v2:172.21.15.174:3300/0 conn(0x7fbfec072110 0x7fbfec0b4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:03.606 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.600+0000 7fbff2d8c700 1 -- 172.21.15.143:0/3470078590 >> 172.21.15.143:0/3470078590 conn(0x7fbfec06bcf0 msgr2=0x7fbfec06c100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:03.606 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.600+0000 7fbff2d8c700 1 -- 172.21.15.143:0/3470078590 shutdown_connections 2023-11-08T19:31:03.606 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.600+0000 7fbff2d8c700 1 -- 172.21.15.143:0/3470078590 wait complete. 2023-11-08T19:31:03.609 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.601+0000 7fbff2d8c700 1 Processor -- start 2023-11-08T19:31:03.609 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.601+0000 7fbff2d8c700 1 -- start start 2023-11-08T19:31:03.610 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.602+0000 7fbff2d8c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbfec072110 0x7fbfec153aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:03.610 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.602+0000 7fbff2d8c700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbfec0b4ed0 0x7fbfec152300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:03.610 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.602+0000 7fbff2d8c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbfec15a800 0x7fbfec143fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:03.610 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.602+0000 7fbff2d8c700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfec15d500 con 0x7fbfec072110 2023-11-08T19:31:03.610 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.602+0000 7fbff2d8c700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfec152a00 con 0x7fbfec15a800 2023-11-08T19:31:03.611 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.602+0000 7fbff2d8c700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfec152ce0 con 0x7fbfec0b4ed0 2023-11-08T19:31:03.611 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.602+0000 7fbff0b28700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbfec072110 0x7fbfec153aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:03.611 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.602+0000 7fbff0b28700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbfec072110 0x7fbfec153aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56680/0 (socket says 172.21.15.143:56680) 2023-11-08T19:31:03.611 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.602+0000 7fbff0b28700 1 -- 172.21.15.143:0/989049570 learned_addr learned my addr 172.21.15.143:0/989049570 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:31:03.611 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.602+0000 7fbff1329700 1 --2- 172.21.15.143:0/989049570 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbfec15a800 0x7fbfec143fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:03.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.602+0000 7fbfebfff700 1 --2- 172.21.15.143:0/989049570 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbfec0b4ed0 0x7fbfec152300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:03.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.602+0000 7fbff0b28700 1 -- 172.21.15.143:0/989049570 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbfec0b4ed0 msgr2=0x7fbfec152300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:03.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.602+0000 7fbff0b28700 1 --2- 172.21.15.143:0/989049570 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbfec0b4ed0 0x7fbfec152300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:03.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.602+0000 7fbff0b28700 1 -- 172.21.15.143:0/989049570 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbfec15a800 msgr2=0x7fbfec143fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:03.613 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.602+0000 7fbff0b28700 1 --2- 172.21.15.143:0/989049570 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbfec15a800 0x7fbfec143fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:03.613 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.602+0000 7fbff0b28700 1 -- 172.21.15.143:0/989049570 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfe4009b30 con 0x7fbfec072110 2023-11-08T19:31:03.613 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.602+0000 7fbfebfff700 1 --2- 172.21.15.143:0/989049570 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbfec0b4ed0 0x7fbfec152300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:31:03.613 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.603+0000 7fbff0b28700 1 --2- 172.21.15.143:0/989049570 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbfec072110 0x7fbfec153aa0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7fbfe000c450 tx=0x7fbfe000c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:03.613 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.603+0000 7fbfe97fa700 1 -- 172.21.15.143:0/989049570 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe00052d0 con 0x7fbfec072110 2023-11-08T19:31:03.614 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.603+0000 7fbfe97fa700 1 -- 172.21.15.143:0/989049570 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfe0005430 con 0x7fbfec072110 2023-11-08T19:31:03.614 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.603+0000 7fbff2d8c700 1 -- 172.21.15.143:0/989049570 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfec152fc0 con 0x7fbfec072110 2023-11-08T19:31:03.614 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.603+0000 7fbfe97fa700 1 -- 172.21.15.143:0/989049570 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe00038c0 con 0x7fbfec072110 2023-11-08T19:31:03.614 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.603+0000 7fbff2d8c700 1 -- 172.21.15.143:0/989049570 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfec153580 con 0x7fbfec072110 2023-11-08T19:31:03.614 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.604+0000 7fbff2d8c700 1 -- 172.21.15.143:0/989049570 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfec04f730 con 0x7fbfec072110 2023-11-08T19:31:03.614 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.604+0000 7fbfe97fa700 1 -- 172.21.15.143:0/989049570 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbfe0003a20 con 0x7fbfec072110 2023-11-08T19:31:03.615 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.604+0000 7fbfe97fa700 1 --2- 172.21.15.143:0/989049570 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fbfd006af00 0x7fbfd006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:03.615 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.605+0000 7fbfe97fa700 1 -- 172.21.15.143:0/989049570 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 7360+0+0 (secure 0 0 0) 0x7fbfe0088ff0 con 0x7fbfec072110 2023-11-08T19:31:03.615 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.605+0000 7fbfebfff700 1 --2- 172.21.15.143:0/989049570 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fbfd006af00 0x7fbfd006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:03.615 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.605+0000 7fbfebfff700 1 --2- 172.21.15.143:0/989049570 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fbfd006af00 0x7fbfd006d3c0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fbfec06e5d0 tx=0x7fbfd8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:03.616 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.609+0000 7fbfe97fa700 1 -- 172.21.15.143:0/989049570 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbfe0053a30 con 0x7fbfec072110 2023-11-08T19:31:03.800 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.799+0000 7fbff2d8c700 1 -- 172.21.15.143:0/989049570 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbfec0b4b80 con 0x7fbfec072110 2023-11-08T19:31:03.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.799+0000 7fbfe97fa700 1 -- 172.21.15.143:0/989049570 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v104) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbfe0017020 con 0x7fbfec072110 2023-11-08T19:31:03.801 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2023-11-08T19:31:03.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.801+0000 7fbff2d8c700 1 -- 172.21.15.143:0/989049570 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fbfd006af00 msgr2=0x7fbfd006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:03.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.801+0000 7fbff2d8c700 1 --2- 172.21.15.143:0/989049570 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fbfd006af00 0x7fbfd006d3c0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fbfec06e5d0 tx=0x7fbfd8008040 comp rx=0 tx=0).stop 2023-11-08T19:31:03.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.801+0000 7fbff2d8c700 1 -- 172.21.15.143:0/989049570 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbfec072110 msgr2=0x7fbfec153aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:03.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.801+0000 7fbff2d8c700 1 --2- 172.21.15.143:0/989049570 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbfec072110 0x7fbfec153aa0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7fbfe000c450 tx=0x7fbfe000c760 comp rx=0 tx=0).stop 2023-11-08T19:31:03.804 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.801+0000 7fbff2d8c700 1 -- 172.21.15.143:0/989049570 shutdown_connections 2023-11-08T19:31:03.804 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.801+0000 7fbff2d8c700 1 --2- 172.21.15.143:0/989049570 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fbfd006af00 0x7fbfd006d3c0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:03.804 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.801+0000 7fbff2d8c700 1 --2- 172.21.15.143:0/989049570 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbfec15a800 0x7fbfec143fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:03.804 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.801+0000 7fbff2d8c700 1 --2- 172.21.15.143:0/989049570 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbfec072110 0x7fbfec153aa0 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:03.804 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.801+0000 7fbff2d8c700 1 --2- 172.21.15.143:0/989049570 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbfec0b4ed0 0x7fbfec152300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:03.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.801+0000 7fbff2d8c700 1 -- 172.21.15.143:0/989049570 >> 172.21.15.143:0/989049570 conn(0x7fbfec06bcf0 msgr2=0x7fbfec0cf980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:03.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.801+0000 7fbff2d8c700 1 -- 172.21.15.143:0/989049570 shutdown_connections 2023-11-08T19:31:03.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.801+0000 7fbff2d8c700 1 -- 172.21.15.143:0/989049570 wait complete. 2023-11-08T19:31:03.815 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get snap-test-pool pg_num 2023-11-08T19:31:03.939 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.937+0000 7f68590e7700 1 Processor -- start 2023-11-08T19:31:03.939 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.937+0000 7f68590e7700 1 -- start start 2023-11-08T19:31:03.947 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.938+0000 7f68590e7700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f68540cd130 0x7f68540cd560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:03.947 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.938+0000 7f68590e7700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68540b68c0 con 0x7f68540b64e0 2023-11-08T19:31:03.948 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.938+0000 7f68590e7700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f685415c550 con 0x7f68540724d0 2023-11-08T19:31:03.948 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.938+0000 7f68590e7700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68540b45f0 con 0x7f68540cd130 2023-11-08T19:31:03.948 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.938+0000 7f6852d9d700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f68540b64e0 0x7f685415c9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:52842/0 (socket says 172.21.15.143:52842) 2023-11-08T19:31:03.949 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.938+0000 7f6852d9d700 1 -- 172.21.15.143:0/1270171760 learned_addr learned my addr 172.21.15.143:0/1270171760 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:31:03.949 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.939+0000 7f685359e700 1 --2- 172.21.15.143:0/1270171760 >> v2:172.21.15.174:3300/0 conn(0x7f68540cd130 0x7f68540cd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:03.949 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.939+0000 7f6851d9b700 1 -- 172.21.15.143:0/1270171760 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1195017417 0 0) 0x7f6848001110 con 0x7f68540b64e0 2023-11-08T19:31:03.950 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.939+0000 7f6851d9b700 1 -- 172.21.15.143:0/1270171760 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f683c0035b0 con 0x7f68540b64e0 2023-11-08T19:31:03.950 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.939+0000 7f6851d9b700 1 -- 172.21.15.143:0/1270171760 <== mon.1 v1:172.21.15.174:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4156744762 0 0) 0x7f685415c550 con 0x7f68540724d0 2023-11-08T19:31:03.950 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.939+0000 7f6851d9b700 1 -- 172.21.15.143:0/1270171760 --> v1:172.21.15.174:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6848001110 con 0x7f68540724d0 2023-11-08T19:31:03.950 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.939+0000 7f6851d9b700 1 -- 172.21.15.143:0/1270171760 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1344687473 0 0) 0x7f683c0035b0 con 0x7f68540b64e0 2023-11-08T19:31:03.951 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.939+0000 7f6851d9b700 1 -- 172.21.15.143:0/1270171760 >> v2:172.21.15.174:3300/0 conn(0x7f68540cd130 msgr2=0x7f68540cd560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:03.951 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.939+0000 7f6851d9b700 1 --2- 172.21.15.143:0/1270171760 >> v2:172.21.15.174:3300/0 conn(0x7f68540cd130 0x7f68540cd560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:03.951 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.939+0000 7f6851d9b700 1 -- 172.21.15.143:0/1270171760 >> v1:172.21.15.174:6789/0 conn(0x7f68540724d0 legacy=0x7f68540b3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:03.952 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.939+0000 7f6851d9b700 1 -- 172.21.15.143:0/1270171760 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f685415bf80 con 0x7f68540b64e0 2023-11-08T19:31:03.952 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.939+0000 7f6851d9b700 1 -- 172.21.15.143:0/1270171760 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4147756303 0 0) 0x7f6848001800 con 0x7f68540b64e0 2023-11-08T19:31:03.952 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.939+0000 7f6851d9b700 1 -- 172.21.15.143:0/1270171760 >> v1:172.21.15.143:6789/0 conn(0x7f68540b64e0 legacy=0x7f685415c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:03.953 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.939+0000 7f6851d9b700 1 --2- 172.21.15.143:0/1270171760 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f683c003be0 0x7f683c003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:03.953 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.939+0000 7f6851d9b700 1 --2- 172.21.15.143:0/1270171760 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f683c004850 0x7f683c004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:03.953 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.939+0000 7f6851d9b700 1 --2- 172.21.15.143:0/1270171760 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f683c005210 0x7f683c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:03.953 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.939+0000 7f6851d9b700 1 -- 172.21.15.143:0/1270171760 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68540b45f0 con 0x7f683c003be0 2023-11-08T19:31:03.954 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.939+0000 7f6851d9b700 1 -- 172.21.15.143:0/1270171760 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f683c00c0e0 con 0x7f683c005210 2023-11-08T19:31:03.954 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.939+0000 7f6851d9b700 1 -- 172.21.15.143:0/1270171760 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f683c00c2a0 con 0x7f683c004850 2023-11-08T19:31:03.954 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.939+0000 7f6852d9d700 1 --2- 172.21.15.143:0/1270171760 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f683c003be0 0x7f683c003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:03.954 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.939+0000 7f685359e700 1 --2- 172.21.15.143:0/1270171760 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f683c005210 0x7f683c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:03.955 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.939+0000 7f685259c700 1 --2- 172.21.15.143:0/1270171760 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f683c004850 0x7f683c004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:03.955 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.939+0000 7f6852d9d700 1 -- 172.21.15.143:0/1270171760 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f683c004850 msgr2=0x7f683c004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:03.955 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.939+0000 7f6852d9d700 1 --2- 172.21.15.143:0/1270171760 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f683c004850 0x7f683c004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:03.955 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.939+0000 7f6852d9d700 1 -- 172.21.15.143:0/1270171760 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f683c005210 msgr2=0x7f683c00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:03.956 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.939+0000 7f6852d9d700 1 --2- 172.21.15.143:0/1270171760 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f683c005210 0x7f683c00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:03.957 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.939+0000 7f685359e700 1 -- 172.21.15.143:0/1270171760 reap_dead start 2023-11-08T19:31:03.957 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.939+0000 7f6852d9d700 1 -- 172.21.15.143:0/1270171760 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f683c0035b0 con 0x7f683c003be0 2023-11-08T19:31:03.957 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.939+0000 7f685259c700 1 --2- 172.21.15.143:0/1270171760 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f683c004850 0x7f683c004cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:31:03.957 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.940+0000 7f6852d9d700 1 --2- 172.21.15.143:0/1270171760 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f683c003be0 0x7f683c003fe0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f684800c110 tx=0x7f683c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63caebb2af4627b4 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:03.958 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.940+0000 7f6851d9b700 1 -- 172.21.15.143:0/1270171760 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f684800ccb0 con 0x7f683c003be0 2023-11-08T19:31:03.958 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.940+0000 7f6851d9b700 1 -- 172.21.15.143:0/1270171760 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f684800ce10 con 0x7f683c003be0 2023-11-08T19:31:03.958 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.940+0000 7f6851d9b700 1 -- 172.21.15.143:0/1270171760 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6848005ba0 con 0x7f683c003be0 2023-11-08T19:31:03.959 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.940+0000 7f68590e7700 1 -- 172.21.15.143:0/1270171760 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f683c003be0 msgr2=0x7f683c003fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:03.959 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.940+0000 7f68590e7700 1 --2- 172.21.15.143:0/1270171760 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f683c003be0 0x7f683c003fe0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f684800c110 tx=0x7f683c005880 comp rx=0 tx=0).stop 2023-11-08T19:31:03.959 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.940+0000 7f68590e7700 1 -- 172.21.15.143:0/1270171760 shutdown_connections 2023-11-08T19:31:03.959 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.940+0000 7f68590e7700 1 --2- 172.21.15.143:0/1270171760 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f683c003be0 0x7f683c003fe0 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:03.960 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.940+0000 7f68590e7700 1 -- 172.21.15.143:0/1270171760 >> 172.21.15.143:0/1270171760 conn(0x7f685406bcf0 msgr2=0x7f685406c100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:03.960 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.940+0000 7f68590e7700 1 -- 172.21.15.143:0/1270171760 shutdown_connections 2023-11-08T19:31:03.960 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.940+0000 7f68590e7700 1 -- 172.21.15.143:0/1270171760 wait complete. 2023-11-08T19:31:03.960 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.942+0000 7f68590e7700 1 Processor -- start 2023-11-08T19:31:03.961 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.942+0000 7f68590e7700 1 -- start start 2023-11-08T19:31:03.961 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.942+0000 7f68590e7700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f683c003be0 0x7f6854140350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:03.961 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.942+0000 7f68590e7700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f68541e62a0 0x7f68541e4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:03.962 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.943+0000 7f68590e7700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6854140e60 0x7f6854141310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:03.962 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.943+0000 7f68590e7700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68541e5440 con 0x7f683c003be0 2023-11-08T19:31:03.962 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.943+0000 7f68590e7700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68541e6740 con 0x7f68541e62a0 2023-11-08T19:31:03.962 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.943+0000 7f68590e7700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68541e6a20 con 0x7f6854140e60 2023-11-08T19:31:03.963 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.943+0000 7f6852d9d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f683c003be0 0x7f6854140350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:03.963 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.943+0000 7f6852d9d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f683c003be0 0x7f6854140350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56698/0 (socket says 172.21.15.143:56698) 2023-11-08T19:31:03.963 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.943+0000 7f6852d9d700 1 -- 172.21.15.143:0/3058447257 learned_addr learned my addr 172.21.15.143:0/3058447257 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:31:03.963 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.943+0000 7f6852d9d700 1 -- 172.21.15.143:0/3058447257 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6854140e60 msgr2=0x7f6854141310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:03.963 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.943+0000 7f685359e700 1 --2- 172.21.15.143:0/3058447257 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6854140e60 0x7f6854141310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:03.964 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.943+0000 7f6852d9d700 1 --2- 172.21.15.143:0/3058447257 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6854140e60 0x7f6854141310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:03.964 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.943+0000 7f685259c700 1 --2- 172.21.15.143:0/3058447257 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f68541e62a0 0x7f68541e4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:03.964 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.943+0000 7f6852d9d700 1 -- 172.21.15.143:0/3058447257 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f68541e62a0 msgr2=0x7f68541e4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:03.964 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.943+0000 7f6852d9d700 1 --2- 172.21.15.143:0/3058447257 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f68541e62a0 0x7f68541e4f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:03.965 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.943+0000 7f6852d9d700 1 -- 172.21.15.143:0/3058447257 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68541e6d00 con 0x7f683c003be0 2023-11-08T19:31:03.965 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.943+0000 7f6852d9d700 1 --2- 172.21.15.143:0/3058447257 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f683c003be0 0x7f6854140350 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f683c0047d0 tx=0x7f6848001ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:03.965 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.943+0000 7f683b7fe700 1 -- 172.21.15.143:0/3058447257 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68480070d0 con 0x7f683c003be0 2023-11-08T19:31:03.965 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.943+0000 7f683b7fe700 1 -- 172.21.15.143:0/3058447257 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6848007230 con 0x7f683c003be0 2023-11-08T19:31:03.965 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.943+0000 7f683b7fe700 1 -- 172.21.15.143:0/3058447257 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68480075c0 con 0x7f683c003be0 2023-11-08T19:31:03.966 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.943+0000 7f68590e7700 1 -- 172.21.15.143:0/3058447257 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68541e6fe0 con 0x7f683c003be0 2023-11-08T19:31:03.966 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.943+0000 7f68590e7700 1 -- 172.21.15.143:0/3058447257 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6854141a30 con 0x7f683c003be0 2023-11-08T19:31:03.966 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.944+0000 7f68590e7700 1 -- 172.21.15.143:0/3058447257 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6854140cb0 con 0x7f683c003be0 2023-11-08T19:31:03.966 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.945+0000 7f683b7fe700 1 -- 172.21.15.143:0/3058447257 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6848007720 con 0x7f683c003be0 2023-11-08T19:31:03.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.945+0000 7f683b7fe700 1 --2- 172.21.15.143:0/3058447257 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f684006af30 0x7f684006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:03.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.945+0000 7f683b7fe700 1 -- 172.21.15.143:0/3058447257 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 7360+0+0 (secure 0 0 0) 0x7f684808b090 con 0x7f683c003be0 2023-11-08T19:31:03.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.946+0000 7f685259c700 1 --2- 172.21.15.143:0/3058447257 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f684006af30 0x7f684006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:03.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.946+0000 7f685259c700 1 --2- 172.21.15.143:0/3058447257 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f684006af30 0x7f684006d3f0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f6854143b50 tx=0x7f6844009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:03.968 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:03.950+0000 7f683b7fe700 1 -- 172.21.15.143:0/3058447257 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6848055c80 con 0x7f683c003be0 2023-11-08T19:31:04.141 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.139+0000 7f68590e7700 1 -- 172.21.15.143:0/3058447257 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"} v 0) v1 -- 0x7f68540b68c0 con 0x7f683c003be0 2023-11-08T19:31:04.141 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.140+0000 7f683b7fe700 1 -- 172.21.15.143:0/3058447257 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"}]=0 v104) v1 ==== 103+0+11 (secure 0 0 0) 0x7f68540b68c0 con 0x7f683c003be0 2023-11-08T19:31:04.141 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2023-11-08T19:31:04.143 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.142+0000 7f68590e7700 1 -- 172.21.15.143:0/3058447257 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f684006af30 msgr2=0x7f684006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:04.143 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.142+0000 7f68590e7700 1 --2- 172.21.15.143:0/3058447257 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f684006af30 0x7f684006d3f0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f6854143b50 tx=0x7f6844009040 comp rx=0 tx=0).stop 2023-11-08T19:31:04.144 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.142+0000 7f68590e7700 1 -- 172.21.15.143:0/3058447257 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f683c003be0 msgr2=0x7f6854140350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:04.144 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.142+0000 7f68590e7700 1 --2- 172.21.15.143:0/3058447257 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f683c003be0 0x7f6854140350 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f683c0047d0 tx=0x7f6848001ac0 comp rx=0 tx=0).stop 2023-11-08T19:31:04.144 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.142+0000 7f68590e7700 1 -- 172.21.15.143:0/3058447257 shutdown_connections 2023-11-08T19:31:04.144 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.142+0000 7f68590e7700 1 --2- 172.21.15.143:0/3058447257 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f684006af30 0x7f684006d3f0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:04.145 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.142+0000 7f68590e7700 1 --2- 172.21.15.143:0/3058447257 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6854140e60 0x7f6854141310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:04.145 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.142+0000 7f68590e7700 1 --2- 172.21.15.143:0/3058447257 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f683c003be0 0x7f6854140350 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:04.145 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.142+0000 7f68590e7700 1 --2- 172.21.15.143:0/3058447257 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f68541e62a0 0x7f68541e4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:04.145 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.142+0000 7f68590e7700 1 -- 172.21.15.143:0/3058447257 >> 172.21.15.143:0/3058447257 conn(0x7f685406bcf0 msgr2=0x7f685406c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:04.146 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.142+0000 7f68590e7700 1 -- 172.21.15.143:0/3058447257 shutdown_connections 2023-11-08T19:31:04.146 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.142+0000 7f68590e7700 1 -- 172.21.15.143:0/3058447257 wait complete. 2023-11-08T19:31:04.155 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get test_data_pool pg_num 2023-11-08T19:31:04.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.277+0000 7f7141bd6700 1 Processor -- start 2023-11-08T19:31:04.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.277+0000 7f7141bd6700 1 -- start start 2023-11-08T19:31:04.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.278+0000 7f7141bd6700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f713c0cf7d0 0x7f713c0cfbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:04.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.278+0000 7f7141bd6700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f713c0b0840 0x7f713c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:04.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.278+0000 7f7141bd6700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f713c0d0930 con 0x7f713c0d00f0 2023-11-08T19:31:04.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.278+0000 7f7141bd6700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f713c0d0650 con 0x7f713c0b0840 2023-11-08T19:31:04.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.278+0000 7f7141bd6700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f713c0b1d50 con 0x7f713c0cf7d0 2023-11-08T19:31:04.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.278+0000 7f713affd700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f713c0d00f0 0x7f713c0ae5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:52860/0 (socket says 172.21.15.143:52860) 2023-11-08T19:31:04.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.278+0000 7f713affd700 1 -- 172.21.15.143:0/3669765200 learned_addr learned my addr 172.21.15.143:0/3669765200 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:31:04.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.279+0000 7f713b7fe700 1 --2- 172.21.15.143:0/3669765200 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f713c0cf7d0 0x7f713c0cfbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:04.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.279+0000 7f713bfff700 1 --2- 172.21.15.143:0/3669765200 >> v2:172.21.15.174:3300/0 conn(0x7f713c0b0840 0x7f713c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:04.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.279+0000 7f713a7fc700 1 -- 172.21.15.143:0/3669765200 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2617665854 0 0) 0x7f713c0d0930 con 0x7f713c0d00f0 2023-11-08T19:31:04.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.279+0000 7f713a7fc700 1 -- 172.21.15.143:0/3669765200 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f71280035b0 con 0x7f713c0d00f0 2023-11-08T19:31:04.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.279+0000 7f713a7fc700 1 -- 172.21.15.143:0/3669765200 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2226248615 0 0) 0x7f71280035b0 con 0x7f713c0d00f0 2023-11-08T19:31:04.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.279+0000 7f713a7fc700 1 -- 172.21.15.143:0/3669765200 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f713c0cf7d0 msgr2=0x7f713c0cfbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:04.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.279+0000 7f713a7fc700 1 --2- 172.21.15.143:0/3669765200 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f713c0cf7d0 0x7f713c0cfbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:04.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.279+0000 7f713a7fc700 1 -- 172.21.15.143:0/3669765200 >> v2:172.21.15.174:3300/0 conn(0x7f713c0b0840 msgr2=0x7f713c06d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:04.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.279+0000 7f713a7fc700 1 --2- 172.21.15.143:0/3669765200 >> v2:172.21.15.174:3300/0 conn(0x7f713c0b0840 0x7f713c06d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:04.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.279+0000 7f713a7fc700 1 -- 172.21.15.143:0/3669765200 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f713c0b2030 con 0x7f713c0d00f0 2023-11-08T19:31:04.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.279+0000 7f713a7fc700 1 -- 172.21.15.143:0/3669765200 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4147756303 0 0) 0x7f712c001670 con 0x7f713c0d00f0 2023-11-08T19:31:04.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.279+0000 7f713a7fc700 1 -- 172.21.15.143:0/3669765200 >> v1:172.21.15.143:6789/0 conn(0x7f713c0d00f0 legacy=0x7f713c0ae5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:04.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.279+0000 7f713a7fc700 1 --2- 172.21.15.143:0/3669765200 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7128003a00 0x7f7128003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:04.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.279+0000 7f713a7fc700 1 --2- 172.21.15.143:0/3669765200 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7128004850 0x7f7128004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:04.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.279+0000 7f713a7fc700 1 --2- 172.21.15.143:0/3669765200 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7128005210 0x7f712800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:04.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.279+0000 7f713a7fc700 1 -- 172.21.15.143:0/3669765200 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f713c0d0650 con 0x7f7128005210 2023-11-08T19:31:04.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.279+0000 7f713a7fc700 1 -- 172.21.15.143:0/3669765200 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f713c0b1d50 con 0x7f7128003a00 2023-11-08T19:31:04.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.279+0000 7f713a7fc700 1 -- 172.21.15.143:0/3669765200 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f712800bf50 con 0x7f7128004850 2023-11-08T19:31:04.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.279+0000 7f713bfff700 1 --2- 172.21.15.143:0/3669765200 >> v2:172.21.15.174:3300/0 conn(0x7f713c0b0840 0x7f713c06d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T19:31:04.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.279+0000 7f713b7fe700 1 --2- 172.21.15.143:0/3669765200 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f713c0cf7d0 0x7f713c0cfbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T19:31:04.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.279+0000 7f713bfff700 1 --2- 172.21.15.143:0/3669765200 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7128005210 0x7f712800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:04.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.279+0000 7f713affd700 1 --2- 172.21.15.143:0/3669765200 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7128004850 0x7f7128004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:04.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.279+0000 7f713b7fe700 1 --2- 172.21.15.143:0/3669765200 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7128003a00 0x7f7128003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:04.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.280+0000 7f713bfff700 1 -- 172.21.15.143:0/3669765200 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7128004850 msgr2=0x7f7128004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:04.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.280+0000 7f713bfff700 1 --2- 172.21.15.143:0/3669765200 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7128004850 0x7f7128004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:04.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.280+0000 7f713bfff700 1 -- 172.21.15.143:0/3669765200 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7128003a00 msgr2=0x7f7128003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:04.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.280+0000 7f713bfff700 1 --2- 172.21.15.143:0/3669765200 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7128003a00 0x7f7128003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:04.293 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.280+0000 7f713bfff700 1 -- 172.21.15.143:0/3669765200 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71280035b0 con 0x7f7128005210 2023-11-08T19:31:04.293 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.280+0000 7f713b7fe700 1 --2- 172.21.15.143:0/3669765200 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7128003a00 0x7f7128003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:31:04.293 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.280+0000 7f713bfff700 1 --2- 172.21.15.143:0/3669765200 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7128005210 0x7f712800ba10 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f713400e380 tx=0x7f7128005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51f50ee497769637 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:04.293 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.280+0000 7f713a7fc700 1 -- 172.21.15.143:0/3669765200 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f713400ee40 con 0x7f7128005210 2023-11-08T19:31:04.293 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.280+0000 7f713a7fc700 1 -- 172.21.15.143:0/3669765200 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f713400f040 con 0x7f7128005210 2023-11-08T19:31:04.294 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.280+0000 7f713a7fc700 1 -- 172.21.15.143:0/3669765200 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7134003c40 con 0x7f7128005210 2023-11-08T19:31:04.294 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.280+0000 7f7141bd6700 1 -- 172.21.15.143:0/3669765200 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7128005210 msgr2=0x7f712800ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:04.294 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.280+0000 7f7141bd6700 1 --2- 172.21.15.143:0/3669765200 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7128005210 0x7f712800ba10 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f713400e380 tx=0x7f7128005880 comp rx=0 tx=0).stop 2023-11-08T19:31:04.294 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.280+0000 7f713bfff700 1 -- 172.21.15.143:0/3669765200 reap_dead start 2023-11-08T19:31:04.294 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.280+0000 7f7141bd6700 1 -- 172.21.15.143:0/3669765200 shutdown_connections 2023-11-08T19:31:04.295 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.280+0000 7f7141bd6700 1 -- 172.21.15.143:0/3669765200 >> 172.21.15.143:0/3669765200 conn(0x7f713c0d18d0 msgr2=0x7f713c0d1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:04.295 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.280+0000 7f7141bd6700 1 -- 172.21.15.143:0/3669765200 shutdown_connections 2023-11-08T19:31:04.295 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.280+0000 7f7141bd6700 1 -- 172.21.15.143:0/3669765200 wait complete. 2023-11-08T19:31:04.295 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.282+0000 7f7141bd6700 1 Processor -- start 2023-11-08T19:31:04.295 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.282+0000 7f7141bd6700 1 -- start start 2023-11-08T19:31:04.296 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.282+0000 7f7141bd6700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f713c1f4000 0x7f713c15f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:04.296 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.282+0000 7f7141bd6700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f713c1f43e0 0x7f713c15fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:04.296 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.282+0000 7f7141bd6700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f713c0ca160 0x7f713c0ca610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:04.296 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.282+0000 7f7141bd6700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f713c0cae90 con 0x7f713c1f4000 2023-11-08T19:31:04.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.283+0000 7f7141bd6700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f713c0cb170 con 0x7f713c1f43e0 2023-11-08T19:31:04.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.283+0000 7f7141bd6700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f713c0cb450 con 0x7f713c0ca160 2023-11-08T19:31:04.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.283+0000 7f713b7fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f713c1f4000 0x7f713c15f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:04.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.283+0000 7f713b7fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f713c1f4000 0x7f713c15f900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56718/0 (socket says 172.21.15.143:56718) 2023-11-08T19:31:04.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.283+0000 7f713b7fe700 1 -- 172.21.15.143:0/2751523504 learned_addr learned my addr 172.21.15.143:0/2751523504 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:31:04.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.283+0000 7f713b7fe700 1 -- 172.21.15.143:0/2751523504 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f713c0ca160 msgr2=0x7f713c0ca610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:04.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.283+0000 7f713b7fe700 1 --2- 172.21.15.143:0/2751523504 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f713c0ca160 0x7f713c0ca610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:04.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.283+0000 7f713b7fe700 1 -- 172.21.15.143:0/2751523504 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f713c1f43e0 msgr2=0x7f713c15fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:04.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.283+0000 7f713b7fe700 1 --2- 172.21.15.143:0/2751523504 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f713c1f43e0 0x7f713c15fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:04.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.283+0000 7f713b7fe700 1 -- 172.21.15.143:0/2751523504 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f713c0c54a0 con 0x7f713c1f4000 2023-11-08T19:31:04.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.283+0000 7f713b7fe700 1 --2- 172.21.15.143:0/2751523504 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f713c1f4000 0x7f713c15f900 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f71240109e0 tx=0x7f7124010da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:04.299 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.283+0000 7f7133fff700 1 -- 172.21.15.143:0/2751523504 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f712400e7f0 con 0x7f713c1f4000 2023-11-08T19:31:04.299 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.283+0000 7f7133fff700 1 -- 172.21.15.143:0/2751523504 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f712400e950 con 0x7f713c1f4000 2023-11-08T19:31:04.299 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.283+0000 7f7141bd6700 1 -- 172.21.15.143:0/2751523504 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f713c0cb730 con 0x7f713c1f4000 2023-11-08T19:31:04.299 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.283+0000 7f7133fff700 1 -- 172.21.15.143:0/2751523504 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f712400ecc0 con 0x7f713c1f4000 2023-11-08T19:31:04.299 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.283+0000 7f7141bd6700 1 -- 172.21.15.143:0/2751523504 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f713c0c89a0 con 0x7f713c1f4000 2023-11-08T19:31:04.300 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.284+0000 7f7141bd6700 1 -- 172.21.15.143:0/2751523504 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f713c0c52f0 con 0x7f713c1f4000 2023-11-08T19:31:04.301 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.285+0000 7f7133fff700 1 -- 172.21.15.143:0/2751523504 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7124016460 con 0x7f713c1f4000 2023-11-08T19:31:04.301 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.285+0000 7f7133fff700 1 --2- 172.21.15.143:0/2751523504 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f711c06af00 0x7f711c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:04.301 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.285+0000 7f7133fff700 1 -- 172.21.15.143:0/2751523504 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 7360+0+0 (secure 0 0 0) 0x7f712408a050 con 0x7f713c1f4000 2023-11-08T19:31:04.301 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.285+0000 7f713affd700 1 --2- 172.21.15.143:0/2751523504 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f711c06af00 0x7f711c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:04.301 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.286+0000 7f713affd700 1 --2- 172.21.15.143:0/2751523504 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f711c06af00 0x7f711c06d3c0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f712c001050 tx=0x7f712c000bc0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:04.302 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.290+0000 7f7133fff700 1 -- 172.21.15.143:0/2751523504 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7124054b20 con 0x7f713c1f4000 2023-11-08T19:31:04.478 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.477+0000 7f7141bd6700 1 -- 172.21.15.143:0/2751523504 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "test_data_pool", "var": "pg_num"} v 0) v1 -- 0x7f713c04f730 con 0x7f713c1f4000 2023-11-08T19:31:04.479 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.477+0000 7f7133fff700 1 -- 172.21.15.143:0/2751523504 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "test_data_pool", "var": "pg_num"}]=0 v104) v1 ==== 103+0+11 (secure 0 0 0) 0x7f7124058140 con 0x7f713c1f4000 2023-11-08T19:31:04.479 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2023-11-08T19:31:04.481 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.479+0000 7f7141bd6700 1 -- 172.21.15.143:0/2751523504 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f711c06af00 msgr2=0x7f711c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:04.481 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.479+0000 7f7141bd6700 1 --2- 172.21.15.143:0/2751523504 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f711c06af00 0x7f711c06d3c0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f712c001050 tx=0x7f712c000bc0 comp rx=0 tx=0).stop 2023-11-08T19:31:04.482 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.479+0000 7f7141bd6700 1 -- 172.21.15.143:0/2751523504 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f713c1f4000 msgr2=0x7f713c15f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:04.482 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.479+0000 7f7141bd6700 1 --2- 172.21.15.143:0/2751523504 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f713c1f4000 0x7f713c15f900 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f71240109e0 tx=0x7f7124010da0 comp rx=0 tx=0).stop 2023-11-08T19:31:04.482 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.479+0000 7f7141bd6700 1 -- 172.21.15.143:0/2751523504 shutdown_connections 2023-11-08T19:31:04.482 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.479+0000 7f7141bd6700 1 --2- 172.21.15.143:0/2751523504 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f711c06af00 0x7f711c06d3c0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:04.483 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.479+0000 7f7141bd6700 1 --2- 172.21.15.143:0/2751523504 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f713c0ca160 0x7f713c0ca610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:04.483 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.479+0000 7f7141bd6700 1 --2- 172.21.15.143:0/2751523504 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f713c1f4000 0x7f713c15f900 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:04.483 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.479+0000 7f7141bd6700 1 --2- 172.21.15.143:0/2751523504 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f713c1f43e0 0x7f713c15fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:04.483 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.479+0000 7f7141bd6700 1 -- 172.21.15.143:0/2751523504 >> 172.21.15.143:0/2751523504 conn(0x7f713c0d18d0 msgr2=0x7f713c0b0c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:04.483 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.479+0000 7f7141bd6700 1 -- 172.21.15.143:0/2751523504 shutdown_connections 2023-11-08T19:31:04.484 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.479+0000 7f7141bd6700 1 -- 172.21.15.143:0/2751523504 wait complete. 2023-11-08T19:31:04.493 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get test_metadata_pool pg_num 2023-11-08T19:31:04.616 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.615+0000 7f123fcb9700 1 Processor -- start 2023-11-08T19:31:04.616 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.615+0000 7f123fcb9700 1 -- start start 2023-11-08T19:31:04.618 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.616+0000 7f123fcb9700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f123815a800 0x7f1238072110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:04.619 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.616+0000 7f123fcb9700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f12380b2770 0x7f12380b4990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:04.619 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.616+0000 7f123fcb9700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1238071650 con 0x7f12380b4ed0 2023-11-08T19:31:04.619 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.616+0000 7f123fcb9700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12380b2f80 con 0x7f123815a800 2023-11-08T19:31:04.619 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.616+0000 7f123fcb9700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12380b3260 con 0x7f12380b2770 2023-11-08T19:31:04.619 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.616+0000 7f123da55700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f123815a800 0x7f1238072110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:04.620 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.616+0000 7f123e256700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f12380b4ed0 0x7f12380cad60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:52884/0 (socket says 172.21.15.143:52884) 2023-11-08T19:31:04.620 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.616+0000 7f123da55700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f123815a800 0x7f1238072110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56726/0 (socket says 172.21.15.143:56726) 2023-11-08T19:31:04.620 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.616+0000 7f123e256700 1 -- 172.21.15.143:0/1075385153 learned_addr learned my addr 172.21.15.143:0/1075385153 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:31:04.620 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.616+0000 7f123d254700 1 --2- 172.21.15.143:0/1075385153 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f12380b2770 0x7f12380b4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:04.620 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.616+0000 7f123da55700 1 -- 172.21.15.143:0/1075385153 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f12380b2770 msgr2=0x7f12380b4990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:04.621 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.616+0000 7f123da55700 1 --2- 172.21.15.143:0/1075385153 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f12380b2770 0x7f12380b4990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:04.621 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.616+0000 7f123da55700 1 -- 172.21.15.143:0/1075385153 >> v1:172.21.15.143:6789/0 conn(0x7f12380b4ed0 legacy=0x7f12380cad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:04.621 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.616+0000 7f123da55700 1 -- 172.21.15.143:0/1075385153 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1238071910 con 0x7f123815a800 2023-11-08T19:31:04.621 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.616+0000 7f123ca53700 1 -- 172.21.15.143:0/1075385153 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 735568913 0 0) 0x7f1238071650 con 0x7f12380b4ed0 2023-11-08T19:31:04.621 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.616+0000 7f123d254700 1 --2- 172.21.15.143:0/1075385153 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f12380b2770 0x7f12380b4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:31:04.622 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.616+0000 7f123da55700 1 --2- 172.21.15.143:0/1075385153 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f123815a800 0x7f1238072110 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f123400bae0 tx=0x7f123400bdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58224fb208b48074 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:04.622 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.616+0000 7f123ca53700 1 -- 172.21.15.143:0/1075385153 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f123400dd30 con 0x7f123815a800 2023-11-08T19:31:04.622 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.616+0000 7f123ca53700 1 -- 172.21.15.143:0/1075385153 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f123400de90 con 0x7f123815a800 2023-11-08T19:31:04.622 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.616+0000 7f123ca53700 1 -- 172.21.15.143:0/1075385153 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1234010690 con 0x7f123815a800 2023-11-08T19:31:04.623 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.617+0000 7f123fcb9700 1 -- 172.21.15.143:0/1075385153 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f123815a800 msgr2=0x7f1238072110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:04.623 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.617+0000 7f123fcb9700 1 --2- 172.21.15.143:0/1075385153 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f123815a800 0x7f1238072110 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f123400bae0 tx=0x7f123400bdf0 comp rx=0 tx=0).stop 2023-11-08T19:31:04.623 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.617+0000 7f123fcb9700 1 -- 172.21.15.143:0/1075385153 shutdown_connections 2023-11-08T19:31:04.623 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.617+0000 7f123fcb9700 1 --2- 172.21.15.143:0/1075385153 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f123815a800 0x7f1238072110 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:04.623 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.617+0000 7f123fcb9700 1 --2- 172.21.15.143:0/1075385153 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f12380b2770 0x7f12380b4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:04.623 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.617+0000 7f123fcb9700 1 -- 172.21.15.143:0/1075385153 >> 172.21.15.143:0/1075385153 conn(0x7f123806bcf0 msgr2=0x7f123806c100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:04.624 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.617+0000 7f123fcb9700 1 -- 172.21.15.143:0/1075385153 shutdown_connections 2023-11-08T19:31:04.624 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.617+0000 7f123fcb9700 1 -- 172.21.15.143:0/1075385153 wait complete. 2023-11-08T19:31:04.626 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.618+0000 7f123fcb9700 1 Processor -- start 2023-11-08T19:31:04.626 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.618+0000 7f123fcb9700 1 -- start start 2023-11-08T19:31:04.627 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.619+0000 7f123fcb9700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f12380b2770 0x7f12381481d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:04.627 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.619+0000 7f123fcb9700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f12380b4ed0 0x7f12381542c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:04.627 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.619+0000 7f123fcb9700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f123815a800 0x7f1238154800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:04.627 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.619+0000 7f123fcb9700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f123815c920 con 0x7f123815a800 2023-11-08T19:31:04.628 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.619+0000 7f123fcb9700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1238148710 con 0x7f12380b4ed0 2023-11-08T19:31:04.628 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.619+0000 7f123fcb9700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f123815bca0 con 0x7f12380b2770 2023-11-08T19:31:04.628 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.619+0000 7f123e256700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f123815a800 0x7f1238154800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:04.628 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.619+0000 7f123e256700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f123815a800 0x7f1238154800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56734/0 (socket says 172.21.15.143:56734) 2023-11-08T19:31:04.628 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.619+0000 7f123e256700 1 -- 172.21.15.143:0/1036564966 learned_addr learned my addr 172.21.15.143:0/1036564966 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:31:04.629 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.619+0000 7f123da55700 1 --2- 172.21.15.143:0/1036564966 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f12380b2770 0x7f12381481d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:04.629 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.619+0000 7f123d254700 1 --2- 172.21.15.143:0/1036564966 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f12380b4ed0 0x7f12381542c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:04.629 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.619+0000 7f123e256700 1 -- 172.21.15.143:0/1036564966 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f12380b2770 msgr2=0x7f12381481d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:04.629 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.619+0000 7f123e256700 1 --2- 172.21.15.143:0/1036564966 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f12380b2770 0x7f12381481d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:04.629 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.619+0000 7f123e256700 1 -- 172.21.15.143:0/1036564966 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f12380b4ed0 msgr2=0x7f12381542c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:04.630 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.619+0000 7f123e256700 1 --2- 172.21.15.143:0/1036564966 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f12380b4ed0 0x7f12381542c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:04.630 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.619+0000 7f123e256700 1 -- 172.21.15.143:0/1036564966 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f123400b740 con 0x7f123815a800 2023-11-08T19:31:04.630 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.620+0000 7f123e256700 1 --2- 172.21.15.143:0/1036564966 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f123815a800 0x7f1238154800 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f123000cca0 tx=0x7f123000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:04.630 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.620+0000 7f122e7fc700 1 -- 172.21.15.143:0/1036564966 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f123000ac20 con 0x7f123815a800 2023-11-08T19:31:04.631 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.620+0000 7f122e7fc700 1 -- 172.21.15.143:0/1036564966 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f123000ad80 con 0x7f123815a800 2023-11-08T19:31:04.631 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.620+0000 7f122e7fc700 1 -- 172.21.15.143:0/1036564966 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1230007310 con 0x7f123815a800 2023-11-08T19:31:04.631 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.620+0000 7f123fcb9700 1 -- 172.21.15.143:0/1036564966 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f123815bf80 con 0x7f123815a800 2023-11-08T19:31:04.631 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.620+0000 7f123fcb9700 1 -- 172.21.15.143:0/1036564966 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f123815c540 con 0x7f123815a800 2023-11-08T19:31:04.631 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.621+0000 7f123fcb9700 1 -- 172.21.15.143:0/1036564966 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f123804f730 con 0x7f123815a800 2023-11-08T19:31:04.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.622+0000 7f122e7fc700 1 -- 172.21.15.143:0/1036564966 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1230007470 con 0x7f123815a800 2023-11-08T19:31:04.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.622+0000 7f122e7fc700 1 --2- 172.21.15.143:0/1036564966 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f121c06af00 0x7f121c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:04.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.622+0000 7f122e7fc700 1 -- 172.21.15.143:0/1036564966 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 7360+0+0 (secure 0 0 0) 0x7f1230089840 con 0x7f123815a800 2023-11-08T19:31:04.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.622+0000 7f123da55700 1 --2- 172.21.15.143:0/1036564966 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f121c06af00 0x7f121c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:04.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.623+0000 7f123da55700 1 --2- 172.21.15.143:0/1036564966 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f121c06af00 0x7f121c06d3c0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f123400b2c0 tx=0x7f1234008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:04.633 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.626+0000 7f122e7fc700 1 -- 172.21.15.143:0/1036564966 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1230054280 con 0x7f123815a800 2023-11-08T19:31:04.819 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.818+0000 7f123fcb9700 1 -- 172.21.15.143:0/1036564966 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "test_metadata_pool", "var": "pg_num"} v 0) v1 -- 0x7f12380b39f0 con 0x7f123815a800 2023-11-08T19:31:04.819 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.818+0000 7f122e7fc700 1 -- 172.21.15.143:0/1036564966 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "test_metadata_pool", "var": "pg_num"}]=0 v104) v1 ==== 107+0+11 (secure 0 0 0) 0x7f12300180b0 con 0x7f123815a800 2023-11-08T19:31:04.820 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2023-11-08T19:31:04.822 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.820+0000 7f123fcb9700 1 -- 172.21.15.143:0/1036564966 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f121c06af00 msgr2=0x7f121c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:04.822 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.820+0000 7f123fcb9700 1 --2- 172.21.15.143:0/1036564966 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f121c06af00 0x7f121c06d3c0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f123400b2c0 tx=0x7f1234008040 comp rx=0 tx=0).stop 2023-11-08T19:31:04.822 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.820+0000 7f123fcb9700 1 -- 172.21.15.143:0/1036564966 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f123815a800 msgr2=0x7f1238154800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:04.822 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.820+0000 7f123fcb9700 1 --2- 172.21.15.143:0/1036564966 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f123815a800 0x7f1238154800 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f123000cca0 tx=0x7f123000a060 comp rx=0 tx=0).stop 2023-11-08T19:31:04.823 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.820+0000 7f123fcb9700 1 -- 172.21.15.143:0/1036564966 shutdown_connections 2023-11-08T19:31:04.823 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.820+0000 7f123fcb9700 1 --2- 172.21.15.143:0/1036564966 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f121c06af00 0x7f121c06d3c0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:04.823 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.820+0000 7f123fcb9700 1 --2- 172.21.15.143:0/1036564966 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f12380b2770 0x7f12381481d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:04.823 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.820+0000 7f123fcb9700 1 --2- 172.21.15.143:0/1036564966 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f123815a800 0x7f1238154800 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:04.823 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.820+0000 7f123fcb9700 1 --2- 172.21.15.143:0/1036564966 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f12380b4ed0 0x7f12381542c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:04.824 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.820+0000 7f123fcb9700 1 -- 172.21.15.143:0/1036564966 >> 172.21.15.143:0/1036564966 conn(0x7f123806bcf0 msgr2=0x7f12380b5310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:04.824 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.820+0000 7f123fcb9700 1 -- 172.21.15.143:0/1036564966 shutdown_connections 2023-11-08T19:31:04.824 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.820+0000 7f123fcb9700 1 -- 172.21.15.143:0/1036564966 wait complete. 2023-11-08T19:31:04.834 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-08T19:31:04.957 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.956+0000 7f8c29105700 1 Processor -- start 2023-11-08T19:31:04.958 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.956+0000 7f8c29105700 1 -- start start 2023-11-08T19:31:04.960 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.957+0000 7f8c29105700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f8c240cd8c0 0x7f8c2406dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:04.960 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.957+0000 7f8c29105700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f8c241617c0 0x7f8c2415ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:04.960 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.957+0000 7f8c29105700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c2415e2e0 con 0x7f8c2406e180 2023-11-08T19:31:04.961 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.957+0000 7f8c29105700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c240cae60 con 0x7f8c241617c0 2023-11-08T19:31:04.961 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.957+0000 7f8c29105700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c24161ff0 con 0x7f8c240cd8c0 2023-11-08T19:31:04.961 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.957+0000 7f8c2259c700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f8c241617c0 0x7f8c2415ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:04.961 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.957+0000 7f8c2359e700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f8c2406e180 0x7f8c24071650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:52894/0 (socket says 172.21.15.143:52894) 2023-11-08T19:31:04.962 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.957+0000 7f8c2359e700 1 -- 172.21.15.143:0/746116088 learned_addr learned my addr 172.21.15.143:0/746116088 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:31:04.962 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.957+0000 7f8c22d9d700 1 --2- 172.21.15.143:0/746116088 >> v2:172.21.15.174:3300/0 conn(0x7f8c240cd8c0 0x7f8c2406dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:04.962 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.957+0000 7f8c2259c700 1 -- 172.21.15.143:0/746116088 >> v2:172.21.15.174:3300/0 conn(0x7f8c240cd8c0 msgr2=0x7f8c2406dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:04.963 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.957+0000 7f8c2259c700 1 --2- 172.21.15.143:0/746116088 >> v2:172.21.15.174:3300/0 conn(0x7f8c240cd8c0 0x7f8c2406dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:04.963 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.957+0000 7f8c2259c700 1 -- 172.21.15.143:0/746116088 >> v1:172.21.15.143:6789/0 conn(0x7f8c2406e180 legacy=0x7f8c24071650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:04.963 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.957+0000 7f8c2259c700 1 -- 172.21.15.143:0/746116088 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c241622d0 con 0x7f8c241617c0 2023-11-08T19:31:04.963 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.958+0000 7f8c2259c700 1 --2- 172.21.15.143:0/746116088 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8c241617c0 0x7f8c2415ea70 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f8c240c9e80 tx=0x7f8c14009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a7f7a14a453cef8 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:04.963 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.958+0000 7f8c21d9b700 1 -- 172.21.15.143:0/746116088 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c14011830 con 0x7f8c241617c0 2023-11-08T19:31:04.964 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.958+0000 7f8c21d9b700 1 -- 172.21.15.143:0/746116088 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c14011990 con 0x7f8c241617c0 2023-11-08T19:31:04.964 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.958+0000 7f8c21d9b700 1 -- 172.21.15.143:0/746116088 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c14011d20 con 0x7f8c241617c0 2023-11-08T19:31:04.964 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.958+0000 7f8c29105700 1 -- 172.21.15.143:0/746116088 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8c241617c0 msgr2=0x7f8c2415ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:04.964 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.958+0000 7f8c29105700 1 --2- 172.21.15.143:0/746116088 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8c241617c0 0x7f8c2415ea70 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f8c240c9e80 tx=0x7f8c14009b20 comp rx=0 tx=0).stop 2023-11-08T19:31:04.965 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.958+0000 7f8c29105700 1 -- 172.21.15.143:0/746116088 shutdown_connections 2023-11-08T19:31:04.965 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.958+0000 7f8c29105700 1 --2- 172.21.15.143:0/746116088 >> v2:172.21.15.174:3300/0 conn(0x7f8c240cd8c0 0x7f8c2406dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:04.965 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.958+0000 7f8c29105700 1 --2- 172.21.15.143:0/746116088 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8c241617c0 0x7f8c2415ea70 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:04.965 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.958+0000 7f8c29105700 1 -- 172.21.15.143:0/746116088 >> 172.21.15.143:0/746116088 conn(0x7f8c2406bcf0 msgr2=0x7f8c2406c100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:04.966 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.958+0000 7f8c29105700 1 -- 172.21.15.143:0/746116088 shutdown_connections 2023-11-08T19:31:04.966 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.958+0000 7f8c29105700 1 -- 172.21.15.143:0/746116088 wait complete. 2023-11-08T19:31:04.969 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.960+0000 7f8c29105700 1 Processor -- start 2023-11-08T19:31:04.969 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.960+0000 7f8c29105700 1 -- start start 2023-11-08T19:31:04.969 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.960+0000 7f8c29105700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8c2406e180 0x7f8c2414b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:04.969 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.960+0000 7f8c29105700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c240cd8c0 0x7f8c24156710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:04.970 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.961+0000 7f8c29105700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8c241617c0 0x7f8c24156c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:04.970 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.961+0000 7f8c29105700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c241602d0 con 0x7f8c241617c0 2023-11-08T19:31:04.970 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.961+0000 7f8c29105700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c24159760 con 0x7f8c240cd8c0 2023-11-08T19:31:04.970 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.961+0000 7f8c29105700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c24159a40 con 0x7f8c2406e180 2023-11-08T19:31:04.971 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.961+0000 7f8c2359e700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8c241617c0 0x7f8c24156c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:04.971 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.961+0000 7f8c2359e700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8c241617c0 0x7f8c24156c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56746/0 (socket says 172.21.15.143:56746) 2023-11-08T19:31:04.971 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.961+0000 7f8c2359e700 1 -- 172.21.15.143:0/1576377418 learned_addr learned my addr 172.21.15.143:0/1576377418 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:31:04.971 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.961+0000 7f8c2259c700 1 --2- 172.21.15.143:0/1576377418 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c240cd8c0 0x7f8c24156710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:04.971 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.961+0000 7f8c22d9d700 1 --2- 172.21.15.143:0/1576377418 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8c2406e180 0x7f8c2414b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:04.972 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.961+0000 7f8c2359e700 1 -- 172.21.15.143:0/1576377418 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8c2406e180 msgr2=0x7f8c2414b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:04.972 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.961+0000 7f8c2359e700 1 --2- 172.21.15.143:0/1576377418 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8c2406e180 0x7f8c2414b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:04.972 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.961+0000 7f8c2359e700 1 -- 172.21.15.143:0/1576377418 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c240cd8c0 msgr2=0x7f8c24156710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:04.972 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.961+0000 7f8c2359e700 1 --2- 172.21.15.143:0/1576377418 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c240cd8c0 0x7f8c24156710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:04.972 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.961+0000 7f8c2359e700 1 -- 172.21.15.143:0/1576377418 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c140094a0 con 0x7f8c241617c0 2023-11-08T19:31:04.973 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.961+0000 7f8c2359e700 1 --2- 172.21.15.143:0/1576377418 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8c241617c0 0x7f8c24156c50 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c00cbc0 tx=0x7f8c1c00ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:04.973 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.961+0000 7f8c0b7fe700 1 -- 172.21.15.143:0/1576377418 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c1c00a9e0 con 0x7f8c241617c0 2023-11-08T19:31:04.973 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.961+0000 7f8c0b7fe700 1 -- 172.21.15.143:0/1576377418 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c1c00ab40 con 0x7f8c241617c0 2023-11-08T19:31:04.973 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.961+0000 7f8c0b7fe700 1 -- 172.21.15.143:0/1576377418 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c1c012400 con 0x7f8c241617c0 2023-11-08T19:31:04.973 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.961+0000 7f8c29105700 1 -- 172.21.15.143:0/1576377418 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c24159d20 con 0x7f8c241617c0 2023-11-08T19:31:04.974 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.961+0000 7f8c29105700 1 -- 172.21.15.143:0/1576377418 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c2414b800 con 0x7f8c241617c0 2023-11-08T19:31:04.974 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.962+0000 7f8c29105700 1 -- 172.21.15.143:0/1576377418 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c2415a240 con 0x7f8c241617c0 2023-11-08T19:31:04.974 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.963+0000 7f8c0b7fe700 1 -- 172.21.15.143:0/1576377418 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8c1c01b3b0 con 0x7f8c241617c0 2023-11-08T19:31:04.974 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.963+0000 7f8c0b7fe700 1 --2- 172.21.15.143:0/1576377418 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f8c1006af00 0x7f8c1006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:04.974 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.963+0000 7f8c0b7fe700 1 -- 172.21.15.143:0/1576377418 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 7360+0+0 (secure 0 0 0) 0x7f8c1c089750 con 0x7f8c241617c0 2023-11-08T19:31:04.975 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.963+0000 7f8c22d9d700 1 --2- 172.21.15.143:0/1576377418 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f8c1006af00 0x7f8c1006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:04.975 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.964+0000 7f8c22d9d700 1 --2- 172.21.15.143:0/1576377418 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f8c1006af00 0x7f8c1006d3c0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f8c18006b20 tx=0x7f8c18008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:04.975 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:04.968+0000 7f8c0b7fe700 1 -- 172.21.15.143:0/1576377418 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8c2415a240 con 0x7f8c241617c0 2023-11-08T19:31:05.159 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.158+0000 7f8c29105700 1 -- 172.21.15.143:0/1576377418 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8c2415a240 con 0x7f8c241617c0 2023-11-08T19:31:05.159 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.158+0000 7f8c0b7fe700 1 -- 172.21.15.143:0/1576377418 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v104) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8c2415a240 con 0x7f8c241617c0 2023-11-08T19:31:05.160 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2023-11-08T19:31:05.162 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.160+0000 7f8c29105700 1 -- 172.21.15.143:0/1576377418 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f8c1006af00 msgr2=0x7f8c1006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:05.162 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.160+0000 7f8c29105700 1 --2- 172.21.15.143:0/1576377418 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f8c1006af00 0x7f8c1006d3c0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f8c18006b20 tx=0x7f8c18008040 comp rx=0 tx=0).stop 2023-11-08T19:31:05.162 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.160+0000 7f8c29105700 1 -- 172.21.15.143:0/1576377418 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8c241617c0 msgr2=0x7f8c24156c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:05.162 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.160+0000 7f8c29105700 1 --2- 172.21.15.143:0/1576377418 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8c241617c0 0x7f8c24156c50 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c00cbc0 tx=0x7f8c1c00ced0 comp rx=0 tx=0).stop 2023-11-08T19:31:05.163 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.160+0000 7f8c29105700 1 -- 172.21.15.143:0/1576377418 shutdown_connections 2023-11-08T19:31:05.163 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.160+0000 7f8c29105700 1 --2- 172.21.15.143:0/1576377418 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f8c1006af00 0x7f8c1006d3c0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:05.163 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.160+0000 7f8c29105700 1 --2- 172.21.15.143:0/1576377418 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8c2406e180 0x7f8c2414b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:05.163 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.160+0000 7f8c29105700 1 --2- 172.21.15.143:0/1576377418 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8c241617c0 0x7f8c24156c50 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:05.163 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.160+0000 7f8c29105700 1 --2- 172.21.15.143:0/1576377418 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c240cd8c0 0x7f8c24156710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:05.163 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.160+0000 7f8c29105700 1 -- 172.21.15.143:0/1576377418 >> 172.21.15.143:0/1576377418 conn(0x7f8c2406bcf0 msgr2=0x7f8c240d07a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:05.164 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.160+0000 7f8c29105700 1 -- 172.21.15.143:0/1576377418 shutdown_connections 2023-11-08T19:31:05.164 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.160+0000 7f8c29105700 1 -- 172.21.15.143:0/1576377418 wait complete. 2023-11-08T19:31:05.173 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-08T19:31:05.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.296+0000 7f777c383700 1 Processor -- start 2023-11-08T19:31:05.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.296+0000 7f777c383700 1 -- start start 2023-11-08T19:31:05.301 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.297+0000 7f777c383700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f77740cd8c0 0x7f777406dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:05.301 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.297+0000 7f777c383700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f77741617c0 0x7f777415ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:05.301 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.297+0000 7f777c383700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f777406e180 0x7f777406e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:05.301 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.297+0000 7f777c383700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77740cae60 con 0x7f77741617c0 2023-11-08T19:31:05.301 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.297+0000 7f777c383700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f777406eb50 con 0x7f777406e180 2023-11-08T19:31:05.302 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.297+0000 7f777c383700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f777415e2e0 con 0x7f77740cd8c0 2023-11-08T19:31:05.302 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.298+0000 7f777991e700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f77741617c0 0x7f777415ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:05.302 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.298+0000 7f777991e700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f77741617c0 0x7f777415ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56750/0 (socket says 172.21.15.143:56750) 2023-11-08T19:31:05.302 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.298+0000 7f777991e700 1 -- 172.21.15.143:0/1723528440 learned_addr learned my addr 172.21.15.143:0/1723528440 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:31:05.302 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.298+0000 7f777a11f700 1 --2- 172.21.15.143:0/1723528440 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f77740cd8c0 0x7f777406dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:05.303 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.298+0000 7f777a920700 1 --2- 172.21.15.143:0/1723528440 >> v2:172.21.15.174:3300/0 conn(0x7f777406e180 0x7f777406e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:05.303 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.298+0000 7f777991e700 1 -- 172.21.15.143:0/1723528440 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f77740cd8c0 msgr2=0x7f777406dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:05.303 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.298+0000 7f777991e700 1 --2- 172.21.15.143:0/1723528440 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f77740cd8c0 0x7f777406dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:05.303 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.298+0000 7f777991e700 1 -- 172.21.15.143:0/1723528440 >> v2:172.21.15.174:3300/0 conn(0x7f777406e180 msgr2=0x7f777406e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:05.303 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.298+0000 7f777991e700 1 --2- 172.21.15.143:0/1723528440 >> v2:172.21.15.174:3300/0 conn(0x7f777406e180 0x7f777406e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:05.304 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.298+0000 7f777991e700 1 -- 172.21.15.143:0/1723528440 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7774161ff0 con 0x7f77741617c0 2023-11-08T19:31:05.304 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.298+0000 7f777a920700 1 --2- 172.21.15.143:0/1723528440 >> v2:172.21.15.174:3300/0 conn(0x7f777406e180 0x7f777406e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:31:05.304 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.298+0000 7f777a11f700 1 --2- 172.21.15.143:0/1723528440 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f77740cd8c0 0x7f777406dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:31:05.304 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.298+0000 7f777991e700 1 --2- 172.21.15.143:0/1723528440 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f77741617c0 0x7f777415ea70 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f7774161700 tx=0x7f7770009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=537be33187135d0a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:05.304 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.298+0000 7f777911d700 1 -- 172.21.15.143:0/1723528440 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f777000b870 con 0x7f77741617c0 2023-11-08T19:31:05.305 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.298+0000 7f777911d700 1 -- 172.21.15.143:0/1723528440 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f777000b9d0 con 0x7f77741617c0 2023-11-08T19:31:05.305 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.298+0000 7f777911d700 1 -- 172.21.15.143:0/1723528440 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f777000bd60 con 0x7f77741617c0 2023-11-08T19:31:05.305 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.299+0000 7f777c383700 1 -- 172.21.15.143:0/1723528440 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f77741617c0 msgr2=0x7f777415ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:05.305 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.299+0000 7f777c383700 1 --2- 172.21.15.143:0/1723528440 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f77741617c0 0x7f777415ea70 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f7774161700 tx=0x7f7770009bd0 comp rx=0 tx=0).stop 2023-11-08T19:31:05.305 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.299+0000 7f777c383700 1 -- 172.21.15.143:0/1723528440 shutdown_connections 2023-11-08T19:31:05.306 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.299+0000 7f777c383700 1 --2- 172.21.15.143:0/1723528440 >> v2:172.21.15.174:3300/0 conn(0x7f777406e180 0x7f777406e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:05.306 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.299+0000 7f777c383700 1 --2- 172.21.15.143:0/1723528440 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f77740cd8c0 0x7f777406dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:05.306 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.299+0000 7f777c383700 1 --2- 172.21.15.143:0/1723528440 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f77741617c0 0x7f777415ea70 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:05.306 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.299+0000 7f777c383700 1 -- 172.21.15.143:0/1723528440 >> 172.21.15.143:0/1723528440 conn(0x7f777406bcf0 msgr2=0x7f777406c100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:05.306 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.299+0000 7f777c383700 1 -- 172.21.15.143:0/1723528440 shutdown_connections 2023-11-08T19:31:05.307 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.299+0000 7f777c383700 1 -- 172.21.15.143:0/1723528440 wait complete. 2023-11-08T19:31:05.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.300+0000 7f777c383700 1 Processor -- start 2023-11-08T19:31:05.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.300+0000 7f777c383700 1 -- start start 2023-11-08T19:31:05.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.301+0000 7f777c383700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f777406e180 0x7f77741566b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:05.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.301+0000 7f777c383700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f77740cd8c0 0x7f77741f72f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:05.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.301+0000 7f777c383700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f77741617c0 0x7f77741f3430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:05.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.301+0000 7f777c383700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7774160020 con 0x7f777406e180 2023-11-08T19:31:05.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.301+0000 7f777c383700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77741f6e60 con 0x7f77740cd8c0 2023-11-08T19:31:05.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.301+0000 7f777c383700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77741f3d20 con 0x7f77741617c0 2023-11-08T19:31:05.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.301+0000 7f777a11f700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f777406e180 0x7f77741566b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:05.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.301+0000 7f777a11f700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f777406e180 0x7f77741566b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56754/0 (socket says 172.21.15.143:56754) 2023-11-08T19:31:05.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.301+0000 7f777a11f700 1 -- 172.21.15.143:0/1422600055 learned_addr learned my addr 172.21.15.143:0/1422600055 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:31:05.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.301+0000 7f777991e700 1 --2- 172.21.15.143:0/1422600055 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f77740cd8c0 0x7f77741f72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:05.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.301+0000 7f777a920700 1 --2- 172.21.15.143:0/1422600055 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f77741617c0 0x7f77741f3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:05.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.301+0000 7f777a11f700 1 -- 172.21.15.143:0/1422600055 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f77741617c0 msgr2=0x7f77741f3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:05.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.301+0000 7f777a11f700 1 --2- 172.21.15.143:0/1422600055 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f77741617c0 0x7f77741f3430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:05.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.301+0000 7f777a11f700 1 -- 172.21.15.143:0/1422600055 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f77740cd8c0 msgr2=0x7f77741f72f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:05.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.301+0000 7f777a11f700 1 --2- 172.21.15.143:0/1422600055 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f77740cd8c0 0x7f77741f72f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:05.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.301+0000 7f777a11f700 1 -- 172.21.15.143:0/1422600055 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7770009550 con 0x7f777406e180 2023-11-08T19:31:05.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.301+0000 7f777991e700 1 --2- 172.21.15.143:0/1422600055 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f77740cd8c0 0x7f77741f72f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:31:05.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.301+0000 7f777a11f700 1 --2- 172.21.15.143:0/1422600055 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f777406e180 0x7f77741566b0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f776800e810 tx=0x7f776800ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:05.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.301+0000 7f7766ffd700 1 -- 172.21.15.143:0/1422600055 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7768004230 con 0x7f777406e180 2023-11-08T19:31:05.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.301+0000 7f7766ffd700 1 -- 172.21.15.143:0/1422600055 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7768004390 con 0x7f777406e180 2023-11-08T19:31:05.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.301+0000 7f777c383700 1 -- 172.21.15.143:0/1422600055 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77741f4000 con 0x7f777406e180 2023-11-08T19:31:05.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.301+0000 7f7766ffd700 1 -- 172.21.15.143:0/1422600055 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77680036a0 con 0x7f777406e180 2023-11-08T19:31:05.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.301+0000 7f777c383700 1 -- 172.21.15.143:0/1422600055 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77741f42e0 con 0x7f777406e180 2023-11-08T19:31:05.315 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.302+0000 7f777c383700 1 -- 172.21.15.143:0/1422600055 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77741f7140 con 0x7f777406e180 2023-11-08T19:31:05.315 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.303+0000 7f7766ffd700 1 -- 172.21.15.143:0/1422600055 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7768003800 con 0x7f777406e180 2023-11-08T19:31:05.315 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.303+0000 7f7766ffd700 1 --2- 172.21.15.143:0/1422600055 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f775806af00 0x7f775806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:05.315 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.303+0000 7f7766ffd700 1 -- 172.21.15.143:0/1422600055 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 7360+0+0 (secure 0 0 0) 0x7f7768089730 con 0x7f777406e180 2023-11-08T19:31:05.315 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.303+0000 7f777991e700 1 --2- 172.21.15.143:0/1422600055 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f775806af00 0x7f775806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:05.316 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.304+0000 7f777991e700 1 --2- 172.21.15.143:0/1422600055 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f775806af00 0x7f775806d3c0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f7770000c00 tx=0x7f7770008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:05.316 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.307+0000 7f7766ffd700 1 -- 172.21.15.143:0/1422600055 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f77680540e0 con 0x7f777406e180 2023-11-08T19:31:05.498 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.496+0000 7f777c383700 1 -- 172.21.15.143:0/1422600055 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f777404f730 con 0x7f777406e180 2023-11-08T19:31:05.499 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.496+0000 7f7766ffd700 1 -- 172.21.15.143:0/1422600055 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v104) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7768057700 con 0x7f777406e180 2023-11-08T19:31:05.499 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 64 2023-11-08T19:31:05.500 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.498+0000 7f777c383700 1 -- 172.21.15.143:0/1422600055 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f775806af00 msgr2=0x7f775806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:05.501 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.498+0000 7f777c383700 1 --2- 172.21.15.143:0/1422600055 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f775806af00 0x7f775806d3c0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f7770000c00 tx=0x7f7770008030 comp rx=0 tx=0).stop 2023-11-08T19:31:05.501 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.498+0000 7f777c383700 1 -- 172.21.15.143:0/1422600055 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f777406e180 msgr2=0x7f77741566b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:05.501 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.498+0000 7f777c383700 1 --2- 172.21.15.143:0/1422600055 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f777406e180 0x7f77741566b0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f776800e810 tx=0x7f776800ebd0 comp rx=0 tx=0).stop 2023-11-08T19:31:05.501 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.498+0000 7f777c383700 1 -- 172.21.15.143:0/1422600055 shutdown_connections 2023-11-08T19:31:05.501 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.498+0000 7f777c383700 1 --2- 172.21.15.143:0/1422600055 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f775806af00 0x7f775806d3c0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:05.502 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.498+0000 7f777c383700 1 --2- 172.21.15.143:0/1422600055 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f77741617c0 0x7f77741f3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:05.502 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.498+0000 7f777c383700 1 --2- 172.21.15.143:0/1422600055 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f777406e180 0x7f77741566b0 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:05.502 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.498+0000 7f777c383700 1 --2- 172.21.15.143:0/1422600055 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f77740cd8c0 0x7f77741f72f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:05.502 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.498+0000 7f777c383700 1 -- 172.21.15.143:0/1422600055 >> 172.21.15.143:0/1422600055 conn(0x7f777406bcf0 msgr2=0x7f777406c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:05.503 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.498+0000 7f777c383700 1 -- 172.21.15.143:0/1422600055 shutdown_connections 2023-11-08T19:31:05.503 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.498+0000 7f777c383700 1 -- 172.21.15.143:0/1422600055 wait complete. 2023-11-08T19:31:05.512 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-08T19:31:05.633 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.632+0000 7f2e9e185700 1 Processor -- start 2023-11-08T19:31:05.634 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.632+0000 7f2e9e185700 1 -- start start 2023-11-08T19:31:05.636 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.633+0000 7f2e9e185700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2e9815b900 0x7f2e980ce3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:05.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.633+0000 7f2e9e185700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f2e9815eac0 0x7f2e9806abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:05.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.633+0000 7f2e9e185700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e980ce910 con 0x7f2e9815ef40 2023-11-08T19:31:05.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.633+0000 7f2e9e185700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e98070120 con 0x7f2e9815eac0 2023-11-08T19:31:05.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.633+0000 7f2e9e185700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e980cb640 con 0x7f2e9815b900 2023-11-08T19:31:05.638 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.633+0000 7f2e96ffd700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f2e9815eac0 0x7f2e9806abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:05.638 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.633+0000 7f2e96ffd700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f2e9815eac0 0x7f2e9806abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56764/0 (socket says 172.21.15.143:56764) 2023-11-08T19:31:05.638 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.633+0000 7f2e96ffd700 1 -- 172.21.15.143:0/2469694456 learned_addr learned my addr 172.21.15.143:0/2469694456 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:31:05.638 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.633+0000 7f2e977fe700 1 --2- 172.21.15.143:0/2469694456 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2e9815b900 0x7f2e980ce3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:05.638 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.633+0000 7f2e96ffd700 1 -- 172.21.15.143:0/2469694456 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2e9815b900 msgr2=0x7f2e980ce3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:05.639 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.633+0000 7f2e96ffd700 1 --2- 172.21.15.143:0/2469694456 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2e9815b900 0x7f2e980ce3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:05.639 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.633+0000 7f2e96ffd700 1 -- 172.21.15.143:0/2469694456 >> v1:172.21.15.174:6789/0 conn(0x7f2e9815ef40 legacy=0x7f2e98161ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:05.639 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.633+0000 7f2e96ffd700 1 -- 172.21.15.143:0/2469694456 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e980d0500 con 0x7f2e9815eac0 2023-11-08T19:31:05.639 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.633+0000 7f2e977fe700 1 --2- 172.21.15.143:0/2469694456 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2e9815b900 0x7f2e980ce3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:31:05.639 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.634+0000 7f2e96ffd700 1 --2- 172.21.15.143:0/2469694456 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2e9815eac0 0x7f2e9806abe0 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f2e880098f0 tx=0x7f2e88009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b52821ec66f89ae0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:05.640 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.634+0000 7f2e967fc700 1 -- 172.21.15.143:0/2469694456 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e88004230 con 0x7f2e9815eac0 2023-11-08T19:31:05.640 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.634+0000 7f2e967fc700 1 -- 172.21.15.143:0/2469694456 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e88004390 con 0x7f2e9815eac0 2023-11-08T19:31:05.640 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.634+0000 7f2e967fc700 1 -- 172.21.15.143:0/2469694456 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e880036a0 con 0x7f2e9815eac0 2023-11-08T19:31:05.640 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.634+0000 7f2e9e185700 1 -- 172.21.15.143:0/2469694456 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2e9815eac0 msgr2=0x7f2e9806abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:05.640 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.634+0000 7f2e9e185700 1 --2- 172.21.15.143:0/2469694456 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2e9815eac0 0x7f2e9806abe0 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f2e880098f0 tx=0x7f2e88009c00 comp rx=0 tx=0).stop 2023-11-08T19:31:05.641 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.634+0000 7f2e9e185700 1 -- 172.21.15.143:0/2469694456 shutdown_connections 2023-11-08T19:31:05.641 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.634+0000 7f2e9e185700 1 --2- 172.21.15.143:0/2469694456 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2e9815b900 0x7f2e980ce3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:05.641 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.634+0000 7f2e9e185700 1 --2- 172.21.15.143:0/2469694456 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2e9815eac0 0x7f2e9806abe0 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:05.641 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.634+0000 7f2e9e185700 1 -- 172.21.15.143:0/2469694456 >> 172.21.15.143:0/2469694456 conn(0x7f2e980b23d0 msgr2=0x7f2e980b27b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:05.641 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.634+0000 7f2e9e185700 1 -- 172.21.15.143:0/2469694456 shutdown_connections 2023-11-08T19:31:05.642 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.634+0000 7f2e9e185700 1 -- 172.21.15.143:0/2469694456 wait complete. 2023-11-08T19:31:05.642 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.635+0000 7f2e9e185700 1 Processor -- start 2023-11-08T19:31:05.645 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.636+0000 7f2e9e185700 1 -- start start 2023-11-08T19:31:05.645 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.636+0000 7f2e9e185700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2e9815b900 0x7f2e9814fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:05.645 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.636+0000 7f2e9e185700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2e9815eac0 0x7f2e980ca260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:05.646 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.636+0000 7f2e9e185700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2e9815ef40 0x7f2e981f3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:05.646 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.636+0000 7f2e9e185700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e98150260 con 0x7f2e9815ef40 2023-11-08T19:31:05.646 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.636+0000 7f2e9e185700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e980aea80 con 0x7f2e9815b900 2023-11-08T19:31:05.646 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.636+0000 7f2e9e185700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e980c78d0 con 0x7f2e9815eac0 2023-11-08T19:31:05.646 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.636+0000 7f2e97fff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2e9815ef40 0x7f2e981f3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:05.647 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.636+0000 7f2e97fff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2e9815ef40 0x7f2e981f3a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56772/0 (socket says 172.21.15.143:56772) 2023-11-08T19:31:05.647 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.636+0000 7f2e97fff700 1 -- 172.21.15.143:0/1225553460 learned_addr learned my addr 172.21.15.143:0/1225553460 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:31:05.647 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.637+0000 7f2e96ffd700 1 --2- 172.21.15.143:0/1225553460 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2e9815eac0 0x7f2e980ca260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:05.647 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.637+0000 7f2e977fe700 1 --2- 172.21.15.143:0/1225553460 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2e9815b900 0x7f2e9814fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:05.647 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.637+0000 7f2e97fff700 1 -- 172.21.15.143:0/1225553460 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2e9815eac0 msgr2=0x7f2e980ca260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:05.648 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.637+0000 7f2e97fff700 1 --2- 172.21.15.143:0/1225553460 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2e9815eac0 0x7f2e980ca260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:05.648 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.637+0000 7f2e97fff700 1 -- 172.21.15.143:0/1225553460 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2e9815b900 msgr2=0x7f2e9814fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:05.648 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.637+0000 7f2e97fff700 1 --2- 172.21.15.143:0/1225553460 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2e9815b900 0x7f2e9814fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:05.648 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.637+0000 7f2e97fff700 1 -- 172.21.15.143:0/1225553460 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e88009550 con 0x7f2e9815ef40 2023-11-08T19:31:05.648 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.637+0000 7f2e977fe700 1 --2- 172.21.15.143:0/1225553460 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2e9815b900 0x7f2e9814fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:31:05.649 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.637+0000 7f2e96ffd700 1 --2- 172.21.15.143:0/1225553460 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2e9815eac0 0x7f2e980ca260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:31:05.649 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.637+0000 7f2e97fff700 1 --2- 172.21.15.143:0/1225553460 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2e9815ef40 0x7f2e981f3a60 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f2e9000cba0 tx=0x7f2e9000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:05.649 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.637+0000 7f2e7ffff700 1 -- 172.21.15.143:0/1225553460 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e9000ab80 con 0x7f2e9815ef40 2023-11-08T19:31:05.649 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.637+0000 7f2e7ffff700 1 -- 172.21.15.143:0/1225553460 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e9000ace0 con 0x7f2e9815ef40 2023-11-08T19:31:05.649 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.637+0000 7f2e9e185700 1 -- 172.21.15.143:0/1225553460 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e980c7b90 con 0x7f2e9815ef40 2023-11-08T19:31:05.650 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.637+0000 7f2e7ffff700 1 -- 172.21.15.143:0/1225553460 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e90012660 con 0x7f2e9815ef40 2023-11-08T19:31:05.650 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.637+0000 7f2e9e185700 1 -- 172.21.15.143:0/1225553460 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e980aecd0 con 0x7f2e9815ef40 2023-11-08T19:31:05.650 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.638+0000 7f2e9e185700 1 -- 172.21.15.143:0/1225553460 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e9804f730 con 0x7f2e9815ef40 2023-11-08T19:31:05.650 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.639+0000 7f2e7ffff700 1 -- 172.21.15.143:0/1225553460 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2e900127c0 con 0x7f2e9815ef40 2023-11-08T19:31:05.650 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.639+0000 7f2e7ffff700 1 --2- 172.21.15.143:0/1225553460 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f2e8406af00 0x7f2e8406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:05.651 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.639+0000 7f2e7ffff700 1 -- 172.21.15.143:0/1225553460 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 7360+0+0 (secure 0 0 0) 0x7f2e9008a3d0 con 0x7f2e9815ef40 2023-11-08T19:31:05.651 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.639+0000 7f2e977fe700 1 --2- 172.21.15.143:0/1225553460 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f2e8406af00 0x7f2e8406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:05.651 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.640+0000 7f2e977fe700 1 --2- 172.21.15.143:0/1225553460 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f2e8406af00 0x7f2e8406d3c0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f2e8c00aa20 tx=0x7f2e8c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:05.651 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.643+0000 7f2e7ffff700 1 -- 172.21.15.143:0/1225553460 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2e90054ea0 con 0x7f2e9815ef40 2023-11-08T19:31:05.852 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.851+0000 7f2e9e185700 1 -- 172.21.15.143:0/1225553460 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f2e98072300 con 0x7f2e9815ef40 2023-11-08T19:31:05.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.852+0000 7f2e7ffff700 1 -- 172.21.15.143:0/1225553460 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 13 entries v104) v1 ==== 81+0+803 (secure 0 0 0) 0x7f2e90016080 con 0x7f2e9815ef40 2023-11-08T19:31:05.853 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.15.174:6834/3601088618 2023-11-09T19:30:47.736745+0000 2023-11-08T19:31:05.854 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.15.143:6834/1535103237 2023-11-09T19:30:40.630538+0000 2023-11-08T19:31:05.854 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.15.143:6832/1535103237 2023-11-09T19:30:40.630538+0000 2023-11-08T19:31:05.854 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.15.143:0/2918728496 2023-11-08T20:30:55.736484+0000 2023-11-08T19:31:05.854 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.15.143:6835/3664584522 2023-11-09T19:30:12.566205+0000 2023-11-08T19:31:05.855 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.15.143:6833/3664584522 2023-11-09T19:30:12.566205+0000 2023-11-08T19:31:05.855 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.15.143:6832/3172664377 2023-11-09T19:29:41.150862+0000 2023-11-08T19:31:05.855 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.15.174:6836/3601088618 2023-11-09T19:30:47.736745+0000 2023-11-08T19:31:05.855 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.15.174:6837/2201447629 2023-11-09T19:28:48.342407+0000 2023-11-08T19:31:05.855 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.15.174:6835/2201447629 2023-11-09T19:28:48.342407+0000 2023-11-08T19:31:05.856 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.15.143:6833/3953059900 2023-11-09T19:29:14.940251+0000 2023-11-08T19:31:05.856 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.15.143:6835/3953059900 2023-11-09T19:29:14.940251+0000 2023-11-08T19:31:05.856 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.15.143:6834/3172664377 2023-11-09T19:29:41.150862+0000 2023-11-08T19:31:05.857 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.854+0000 7f2e9e185700 1 -- 172.21.15.143:0/1225553460 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f2e8406af00 msgr2=0x7f2e8406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:05.857 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.854+0000 7f2e9e185700 1 --2- 172.21.15.143:0/1225553460 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f2e8406af00 0x7f2e8406d3c0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f2e8c00aa20 tx=0x7f2e8c008040 comp rx=0 tx=0).stop 2023-11-08T19:31:05.858 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.854+0000 7f2e9e185700 1 -- 172.21.15.143:0/1225553460 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2e9815ef40 msgr2=0x7f2e981f3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:05.858 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.854+0000 7f2e9e185700 1 --2- 172.21.15.143:0/1225553460 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2e9815ef40 0x7f2e981f3a60 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f2e9000cba0 tx=0x7f2e9000cf60 comp rx=0 tx=0).stop 2023-11-08T19:31:05.859 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.854+0000 7f2e9e185700 1 -- 172.21.15.143:0/1225553460 shutdown_connections 2023-11-08T19:31:05.859 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.854+0000 7f2e9e185700 1 --2- 172.21.15.143:0/1225553460 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f2e8406af00 0x7f2e8406d3c0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:05.859 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.854+0000 7f2e9e185700 1 --2- 172.21.15.143:0/1225553460 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2e9815ef40 0x7f2e981f3a60 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:05.860 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.854+0000 7f2e9e185700 1 --2- 172.21.15.143:0/1225553460 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2e9815b900 0x7f2e9814fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:05.860 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.854+0000 7f2e9e185700 1 --2- 172.21.15.143:0/1225553460 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2e9815eac0 0x7f2e980ca260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:05.860 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.854+0000 7f2e9e185700 1 -- 172.21.15.143:0/1225553460 >> 172.21.15.143:0/1225553460 conn(0x7f2e980b23d0 msgr2=0x7f2e981606b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:05.860 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.854+0000 7f2e9e185700 1 -- 172.21.15.143:0/1225553460 shutdown_connections 2023-11-08T19:31:05.861 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:05.854+0000 7f2e9e185700 1 -- 172.21.15.143:0/1225553460 wait complete. 2023-11-08T19:31:05.861 INFO:teuthology.orchestra.run.smithi143.stderr:listed 13 entries 2023-11-08T19:31:05.867 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-08T19:31:05.868 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:31:05.868 DEBUG:teuthology.orchestra.run.smithi143:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-11-08T19:31:05.904 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi143.front.sepia.ceph.com 2023-11-08T19:31:05.905 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:31:05.905 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-08T19:31:05.962 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2023-11-08T19:31:05.963 DEBUG:teuthology.orchestra.run.smithi174:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:31:05.984 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284224k,nr_inodes=4071056,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=31,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19549 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260544k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-08T19:31:05.985 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-08T19:31:05.985 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:31:06.019 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284264k,nr_inodes=4071066,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=44,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20643 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-08T19:31:06.020 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2023-11-08T19:31:06.020 DEBUG:teuthology.orchestra.run.smithi174:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:31:06.044 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284224k,nr_inodes=4071056,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=31,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19549 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260544k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-08T19:31:06.045 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:31:06.179 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.178+0000 7f8a5fed8700 1 Processor -- start 2023-11-08T19:31:06.180 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.178+0000 7f8a5fed8700 1 -- start start 2023-11-08T19:31:06.185 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.179+0000 7f8a5fed8700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8a5815bdc0 0x7f8a580ae4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:06.185 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.179+0000 7f8a5fed8700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f8a580ccf70 0x7f8a580cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:06.186 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.179+0000 7f8a5fed8700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a580b1250 con 0x7f8a580cfe30 2023-11-08T19:31:06.186 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.179+0000 7f8a5fed8700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a580b20b0 con 0x7f8a580ccf70 2023-11-08T19:31:06.186 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.179+0000 7f8a5fed8700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a580b14a0 con 0x7f8a5815bdc0 2023-11-08T19:31:06.186 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.179+0000 7f8a5e475700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f8a580cfe30 0x7f8a580d02c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:52934/0 (socket says 172.21.15.143:52934) 2023-11-08T19:31:06.186 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.179+0000 7f8a5e475700 1 -- 172.21.15.143:0/1330672844 learned_addr learned my addr 172.21.15.143:0/1330672844 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:31:06.187 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.179+0000 7f8a5d473700 1 --2- 172.21.15.143:0/1330672844 >> v2:172.21.15.174:3300/0 conn(0x7f8a580ccf70 0x7f8a580cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:06.187 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.179+0000 7f8a5dc74700 1 --2- 172.21.15.143:0/1330672844 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8a5815bdc0 0x7f8a580ae4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:06.187 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.179+0000 7f8a5cc72700 1 -- 172.21.15.143:0/1330672844 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2758218347 0 0) 0x7f8a580b1250 con 0x7f8a580cfe30 2023-11-08T19:31:06.187 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.179+0000 7f8a5cc72700 1 -- 172.21.15.143:0/1330672844 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8a440035b0 con 0x7f8a580cfe30 2023-11-08T19:31:06.187 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.179+0000 7f8a5cc72700 1 -- 172.21.15.143:0/1330672844 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1834940307 0 0) 0x7f8a440035b0 con 0x7f8a580cfe30 2023-11-08T19:31:06.188 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.179+0000 7f8a5cc72700 1 -- 172.21.15.143:0/1330672844 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8a5815bdc0 msgr2=0x7f8a580ae4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:06.188 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.179+0000 7f8a5cc72700 1 --2- 172.21.15.143:0/1330672844 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8a5815bdc0 0x7f8a580ae4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:06.188 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.179+0000 7f8a5cc72700 1 -- 172.21.15.143:0/1330672844 >> v2:172.21.15.174:3300/0 conn(0x7f8a580ccf70 msgr2=0x7f8a580cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:06.188 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.179+0000 7f8a5cc72700 1 --2- 172.21.15.143:0/1330672844 >> v2:172.21.15.174:3300/0 conn(0x7f8a580ccf70 0x7f8a580cd3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:06.188 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.179+0000 7f8a5cc72700 1 -- 172.21.15.143:0/1330672844 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a580b1780 con 0x7f8a580cfe30 2023-11-08T19:31:06.189 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.179+0000 7f8a5cc72700 1 -- 172.21.15.143:0/1330672844 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4147756303 0 0) 0x7f8a500019f0 con 0x7f8a580cfe30 2023-11-08T19:31:06.189 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.179+0000 7f8a5cc72700 1 -- 172.21.15.143:0/1330672844 >> v1:172.21.15.143:6789/0 conn(0x7f8a580cfe30 legacy=0x7f8a580d02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:06.189 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.180+0000 7f8a5cc72700 1 --2- 172.21.15.143:0/1330672844 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8a5815bdc0 0x7f8a44003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:06.189 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.180+0000 7f8a5cc72700 1 --2- 172.21.15.143:0/1330672844 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a44003f80 0x7f8a44004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:06.189 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.180+0000 7f8a5cc72700 1 --2- 172.21.15.143:0/1330672844 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8a44004d90 0x7f8a44005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:06.190 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.180+0000 7f8a5cc72700 1 -- 172.21.15.143:0/1330672844 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a580b20b0 con 0x7f8a44003f80 2023-11-08T19:31:06.190 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.180+0000 7f8a5cc72700 1 -- 172.21.15.143:0/1330672844 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a580b14a0 con 0x7f8a44004d90 2023-11-08T19:31:06.190 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.180+0000 7f8a5cc72700 1 -- 172.21.15.143:0/1330672844 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a4400ba10 con 0x7f8a5815bdc0 2023-11-08T19:31:06.190 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.180+0000 7f8a5d473700 1 --2- 172.21.15.143:0/1330672844 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a44003f80 0x7f8a44004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:06.191 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.180+0000 7f8a5e475700 1 --2- 172.21.15.143:0/1330672844 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8a44004d90 0x7f8a44005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:06.191 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.180+0000 7f8a5dc74700 1 --2- 172.21.15.143:0/1330672844 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8a5815bdc0 0x7f8a44003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:06.191 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.180+0000 7f8a5d473700 1 -- 172.21.15.143:0/1330672844 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8a5815bdc0 msgr2=0x7f8a44003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:06.191 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.180+0000 7f8a5d473700 1 --2- 172.21.15.143:0/1330672844 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8a5815bdc0 0x7f8a44003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:06.191 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.180+0000 7f8a5d473700 1 -- 172.21.15.143:0/1330672844 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8a44004d90 msgr2=0x7f8a44005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:06.192 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.180+0000 7f8a5d473700 1 --2- 172.21.15.143:0/1330672844 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8a44004d90 0x7f8a44005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:06.192 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.180+0000 7f8a5d473700 1 -- 172.21.15.143:0/1330672844 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a440035b0 con 0x7f8a44003f80 2023-11-08T19:31:06.192 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.180+0000 7f8a5d473700 1 --2- 172.21.15.143:0/1330672844 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a44003f80 0x7f8a44004850 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f8a48009b60 tx=0x7f8a44005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c712d4582e1dc620 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:06.192 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.180+0000 7f8a5cc72700 1 -- 172.21.15.143:0/1330672844 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a4800e660 con 0x7f8a44003f80 2023-11-08T19:31:06.192 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.180+0000 7f8a5cc72700 1 -- 172.21.15.143:0/1330672844 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a4800e7c0 con 0x7f8a44003f80 2023-11-08T19:31:06.193 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.180+0000 7f8a5cc72700 1 -- 172.21.15.143:0/1330672844 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a4800eb30 con 0x7f8a44003f80 2023-11-08T19:31:06.193 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.181+0000 7f8a5fed8700 1 -- 172.21.15.143:0/1330672844 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a44003f80 msgr2=0x7f8a44004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:06.193 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.181+0000 7f8a5fed8700 1 --2- 172.21.15.143:0/1330672844 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a44003f80 0x7f8a44004850 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f8a48009b60 tx=0x7f8a44005880 comp rx=0 tx=0).stop 2023-11-08T19:31:06.193 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.181+0000 7f8a5e475700 1 -- 172.21.15.143:0/1330672844 reap_dead start 2023-11-08T19:31:06.193 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.181+0000 7f8a5fed8700 1 -- 172.21.15.143:0/1330672844 shutdown_connections 2023-11-08T19:31:06.194 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.181+0000 7f8a5fed8700 1 -- 172.21.15.143:0/1330672844 >> 172.21.15.143:0/1330672844 conn(0x7f8a5809aa70 msgr2=0x7f8a580d18f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:06.194 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.181+0000 7f8a5fed8700 1 -- 172.21.15.143:0/1330672844 shutdown_connections 2023-11-08T19:31:06.194 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.181+0000 7f8a5fed8700 1 -- 172.21.15.143:0/1330672844 wait complete. 2023-11-08T19:31:06.194 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.182+0000 7f8a5fed8700 1 Processor -- start 2023-11-08T19:31:06.194 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.182+0000 7f8a5fed8700 1 -- start start 2023-11-08T19:31:06.195 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.183+0000 7f8a5fed8700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8a44003f80 0x7f8a58156e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:06.195 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.183+0000 7f8a5fed8700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8a58157590 0x7f8a5806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:06.195 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.183+0000 7f8a5fed8700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a58157a10 0x7f8a5806ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:06.195 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.183+0000 7f8a5fed8700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a580c6fa0 con 0x7f8a58157a10 2023-11-08T19:31:06.195 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.183+0000 7f8a5fed8700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a581573b0 con 0x7f8a44003f80 2023-11-08T19:31:06.196 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.183+0000 7f8a5fed8700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a5806d9e0 con 0x7f8a58157590 2023-11-08T19:31:06.196 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.183+0000 7f8a5e475700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a58157a10 0x7f8a5806ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:06.196 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.183+0000 7f8a5e475700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a58157a10 0x7f8a5806ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56792/0 (socket says 172.21.15.143:56792) 2023-11-08T19:31:06.196 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.183+0000 7f8a5e475700 1 -- 172.21.15.143:0/1719427874 learned_addr learned my addr 172.21.15.143:0/1719427874 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:31:06.197 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.183+0000 7f8a5d473700 1 --2- 172.21.15.143:0/1719427874 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8a58157590 0x7f8a5806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:06.197 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.183+0000 7f8a5dc74700 1 --2- 172.21.15.143:0/1719427874 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8a44003f80 0x7f8a58156e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:06.197 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.183+0000 7f8a5e475700 1 -- 172.21.15.143:0/1719427874 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8a58157590 msgr2=0x7f8a5806e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:06.197 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.183+0000 7f8a5e475700 1 --2- 172.21.15.143:0/1719427874 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8a58157590 0x7f8a5806e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:06.197 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.183+0000 7f8a5e475700 1 -- 172.21.15.143:0/1719427874 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8a44003f80 msgr2=0x7f8a58156e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:06.198 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.183+0000 7f8a5e475700 1 --2- 172.21.15.143:0/1719427874 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8a44003f80 0x7f8a58156e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:06.198 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.183+0000 7f8a5e475700 1 -- 172.21.15.143:0/1719427874 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a5806dcc0 con 0x7f8a58157a10 2023-11-08T19:31:06.198 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.183+0000 7f8a5e475700 1 --2- 172.21.15.143:0/1719427874 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a58157a10 0x7f8a5806ef30 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f8a50000f80 tx=0x7f8a5000a640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:06.198 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.183+0000 7f8a4e7fc700 1 -- 172.21.15.143:0/1719427874 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a5000adb0 con 0x7f8a58157a10 2023-11-08T19:31:06.199 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.183+0000 7f8a4e7fc700 1 -- 172.21.15.143:0/1719427874 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a500133f0 con 0x7f8a58157a10 2023-11-08T19:31:06.199 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.184+0000 7f8a5fed8700 1 -- 172.21.15.143:0/1719427874 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a580c68e0 con 0x7f8a58157a10 2023-11-08T19:31:06.199 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.184+0000 7f8a4e7fc700 1 -- 172.21.15.143:0/1719427874 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a500137a0 con 0x7f8a58157a10 2023-11-08T19:31:06.199 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.184+0000 7f8a5fed8700 1 -- 172.21.15.143:0/1719427874 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a580c73c0 con 0x7f8a58157a10 2023-11-08T19:31:06.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.185+0000 7f8a5fed8700 1 -- 172.21.15.143:0/1719427874 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a580c6740 con 0x7f8a58157a10 2023-11-08T19:31:06.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.185+0000 7f8a4e7fc700 1 -- 172.21.15.143:0/1719427874 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8a50013900 con 0x7f8a58157a10 2023-11-08T19:31:06.201 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.185+0000 7f8a4e7fc700 1 --2- 172.21.15.143:0/1719427874 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f8a3c06af00 0x7f8a3c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:06.201 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.185+0000 7f8a4e7fc700 1 -- 172.21.15.143:0/1719427874 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 7360+0+0 (secure 0 0 0) 0x7f8a5008a660 con 0x7f8a58157a10 2023-11-08T19:31:06.201 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.186+0000 7f8a5dc74700 1 --2- 172.21.15.143:0/1719427874 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f8a3c06af00 0x7f8a3c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:06.201 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.186+0000 7f8a5dc74700 1 --2- 172.21.15.143:0/1719427874 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f8a3c06af00 0x7f8a3c06d3c0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f8a54009b60 tx=0x7f8a540091c0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:06.202 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.190+0000 7f8a4e7fc700 1 -- 172.21.15.143:0/1719427874 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8a580c6740 con 0x7f8a58157a10 2023-11-08T19:31:06.420 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.419+0000 7f8a5fed8700 1 -- 172.21.15.143:0/1719427874 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8a580c70e0 con 0x7f8a58157a10 2023-11-08T19:31:06.421 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.419+0000 7f8a4e7fc700 1 -- 172.21.15.143:0/1719427874 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 100 v100) v1 ==== 95+0+4946 (secure 0 0 0) 0x7f8a580c70e0 con 0x7f8a58157a10 2023-11-08T19:31:06.421 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:31:06.421 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":100,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5933,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/3918286710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6835","nonce":3918286710},{"type":"v1","addr":"172.21.15.174:6837","nonce":3918286710}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":6245,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6836/2689884850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2689884850},{"type":"v1","addr":"172.21.15.174:6836","nonce":2689884850}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":97}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:03.092245+0000","modified":"2023-11-08T19:30:56.788736+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":103,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6161,"mds_1":7743},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6161":{"gid":6161,"name":"a","rank":0,"incarnation":93,"state":"up:active","state_seq":14,"addr":"172.21.15.143:6835/2129643800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":2129643800},{"type":"v1","addr":"172.21.15.143:6835","nonce":2129643800}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7743":{"gid":7743,"name":"c","rank":1,"incarnation":96,"state":"up:active","state_seq":7,"addr":"172.21.15.143:6834/2489252684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":2489252684},{"type":"v1","addr":"172.21.15.143:6834","nonce":2489252684}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:31:06.423 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.421+0000 7f8a5fed8700 1 -- 172.21.15.143:0/1719427874 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f8a3c06af00 msgr2=0x7f8a3c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:06.423 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.421+0000 7f8a5fed8700 1 --2- 172.21.15.143:0/1719427874 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f8a3c06af00 0x7f8a3c06d3c0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f8a54009b60 tx=0x7f8a540091c0 comp rx=0 tx=0).stop 2023-11-08T19:31:06.423 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.421+0000 7f8a5fed8700 1 -- 172.21.15.143:0/1719427874 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a58157a10 msgr2=0x7f8a5806ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:06.423 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.421+0000 7f8a5fed8700 1 --2- 172.21.15.143:0/1719427874 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a58157a10 0x7f8a5806ef30 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f8a50000f80 tx=0x7f8a5000a640 comp rx=0 tx=0).stop 2023-11-08T19:31:06.423 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.421+0000 7f8a5fed8700 1 -- 172.21.15.143:0/1719427874 shutdown_connections 2023-11-08T19:31:06.424 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.421+0000 7f8a5fed8700 1 --2- 172.21.15.143:0/1719427874 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f8a3c06af00 0x7f8a3c06d3c0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:06.424 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.421+0000 7f8a5fed8700 1 --2- 172.21.15.143:0/1719427874 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a58157a10 0x7f8a5806ef30 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:06.424 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.421+0000 7f8a5fed8700 1 --2- 172.21.15.143:0/1719427874 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8a44003f80 0x7f8a58156e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:06.424 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.421+0000 7f8a5fed8700 1 --2- 172.21.15.143:0/1719427874 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8a58157590 0x7f8a5806e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:06.424 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.421+0000 7f8a5fed8700 1 -- 172.21.15.143:0/1719427874 >> 172.21.15.143:0/1719427874 conn(0x7f8a5809aa70 msgr2=0x7f8a580aef40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:06.425 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.421+0000 7f8a5fed8700 1 -- 172.21.15.143:0/1719427874 shutdown_connections 2023-11-08T19:31:06.425 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.421+0000 7f8a5fed8700 1 -- 172.21.15.143:0/1719427874 wait complete. 2023-11-08T19:31:06.425 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 100 2023-11-08T19:31:06.438 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:31:06.557 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.556+0000 7fb5c0d87700 1 Processor -- start 2023-11-08T19:31:06.557 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.556+0000 7fb5c0d87700 1 -- start start 2023-11-08T19:31:06.560 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.557+0000 7fb5c0d87700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb5bc06f370 0x7fb5bc06bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:06.560 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.557+0000 7fb5c0d87700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5bc071b40 con 0x7fb5bc15abf0 2023-11-08T19:31:06.560 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.557+0000 7fb5c0d87700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5bc0b50b0 con 0x7fb5bc0b4100 2023-11-08T19:31:06.560 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.557+0000 7fb5c0d87700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5bc15afd0 con 0x7fb5bc06f370 2023-11-08T19:31:06.561 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.557+0000 7fb5ba59c700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fb5bc15abf0 0x7fb5bc071fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:52950/0 (socket says 172.21.15.143:52950) 2023-11-08T19:31:06.561 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.557+0000 7fb5ba59c700 1 -- 172.21.15.143:0/3525384084 learned_addr learned my addr 172.21.15.143:0/3525384084 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:31:06.561 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.557+0000 7fb5b9d9b700 1 --2- 172.21.15.143:0/3525384084 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb5bc06f370 0x7fb5bc06bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:06.561 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.557+0000 7fb5b959a700 1 -- 172.21.15.143:0/3525384084 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2083233359 0 0) 0x7fb5bc071b40 con 0x7fb5bc15abf0 2023-11-08T19:31:06.561 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.557+0000 7fb5b959a700 1 -- 172.21.15.143:0/3525384084 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb5a40035b0 con 0x7fb5bc15abf0 2023-11-08T19:31:06.562 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.557+0000 7fb5b959a700 1 -- 172.21.15.143:0/3525384084 <== mon.1 v1:172.21.15.174:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 134861161 0 0) 0x7fb5bc0b50b0 con 0x7fb5bc0b4100 2023-11-08T19:31:06.562 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.557+0000 7fb5b959a700 1 -- 172.21.15.143:0/3525384084 --> v1:172.21.15.174:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb5bc071b40 con 0x7fb5bc0b4100 2023-11-08T19:31:06.562 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.557+0000 7fb5b959a700 1 -- 172.21.15.143:0/3525384084 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 188715732 0 0) 0x7fb5a40035b0 con 0x7fb5bc15abf0 2023-11-08T19:31:06.562 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.557+0000 7fb5b9d9b700 1 -- 172.21.15.143:0/3525384084 >> v1:172.21.15.174:6789/0 conn(0x7fb5bc0b4100 legacy=0x7fb5bc0cd2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:06.562 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.557+0000 7fb5b9d9b700 1 -- 172.21.15.143:0/3525384084 >> v1:172.21.15.143:6789/0 conn(0x7fb5bc15abf0 legacy=0x7fb5bc071fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:06.563 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.557+0000 7fb5b9d9b700 1 -- 172.21.15.143:0/3525384084 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5bc0726e0 con 0x7fb5bc06f370 2023-11-08T19:31:06.563 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.558+0000 7fb5b9d9b700 1 --2- 172.21.15.143:0/3525384084 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb5bc06f370 0x7fb5bc06bbf0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb5b0009870 tx=0x7fb5b0009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96db69b9e289240d server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:06.563 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.558+0000 7fb5b959a700 1 -- 172.21.15.143:0/3525384084 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5ac001670 con 0x7fb5bc06f370 2023-11-08T19:31:06.563 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.558+0000 7fb5b959a700 1 -- 172.21.15.143:0/3525384084 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5b0004030 con 0x7fb5bc06f370 2023-11-08T19:31:06.563 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.558+0000 7fb5b959a700 1 -- 172.21.15.143:0/3525384084 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5b00043c0 con 0x7fb5bc06f370 2023-11-08T19:31:06.564 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.558+0000 7fb5c0d87700 1 -- 172.21.15.143:0/3525384084 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb5bc06f370 msgr2=0x7fb5bc06bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:06.564 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.558+0000 7fb5c0d87700 1 --2- 172.21.15.143:0/3525384084 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb5bc06f370 0x7fb5bc06bbf0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb5b0009870 tx=0x7fb5b0009b80 comp rx=0 tx=0).stop 2023-11-08T19:31:06.564 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.558+0000 7fb5c0d87700 1 -- 172.21.15.143:0/3525384084 shutdown_connections 2023-11-08T19:31:06.564 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.558+0000 7fb5c0d87700 1 --2- 172.21.15.143:0/3525384084 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb5bc06f370 0x7fb5bc06bbf0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:06.564 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.558+0000 7fb5c0d87700 1 -- 172.21.15.143:0/3525384084 >> 172.21.15.143:0/3525384084 conn(0x7fb5bc06e260 msgr2=0x7fb5bc06e670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:06.564 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.558+0000 7fb5c0d87700 1 -- 172.21.15.143:0/3525384084 shutdown_connections 2023-11-08T19:31:06.565 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.558+0000 7fb5c0d87700 1 -- 172.21.15.143:0/3525384084 wait complete. 2023-11-08T19:31:06.567 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.560+0000 7fb5c0d87700 1 Processor -- start 2023-11-08T19:31:06.568 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.560+0000 7fb5c0d87700 1 -- start start 2023-11-08T19:31:06.568 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.560+0000 7fb5c0d87700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb5bc06f370 0x7fb5bc0af020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:06.568 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.560+0000 7fb5c0d87700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb5bc0b4100 0x7fb5bc1e4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:06.568 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.560+0000 7fb5c0d87700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb5bc15abf0 0x7fb5bc1f6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:06.568 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.560+0000 7fb5c0d87700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5bc0aa740 con 0x7fb5bc06f370 2023-11-08T19:31:06.569 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.560+0000 7fb5c0d87700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5bc0aa430 con 0x7fb5bc15abf0 2023-11-08T19:31:06.569 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.560+0000 7fb5c0d87700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5bc1e5150 con 0x7fb5bc0b4100 2023-11-08T19:31:06.569 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.560+0000 7fb5ba59c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb5bc06f370 0x7fb5bc0af020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:06.569 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.561+0000 7fb5ba59c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb5bc06f370 0x7fb5bc0af020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56802/0 (socket says 172.21.15.143:56802) 2023-11-08T19:31:06.569 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.561+0000 7fb5ba59c700 1 -- 172.21.15.143:0/3545944353 learned_addr learned my addr 172.21.15.143:0/3545944353 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:31:06.570 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.561+0000 7fb5ba59c700 1 -- 172.21.15.143:0/3545944353 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb5bc0b4100 msgr2=0x7fb5bc1e4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:06.570 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.561+0000 7fb5b9d9b700 1 --2- 172.21.15.143:0/3545944353 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb5bc0b4100 0x7fb5bc1e4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:06.570 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.561+0000 7fb5bad9d700 1 --2- 172.21.15.143:0/3545944353 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb5bc15abf0 0x7fb5bc1f6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:06.570 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.561+0000 7fb5ba59c700 1 --2- 172.21.15.143:0/3545944353 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb5bc0b4100 0x7fb5bc1e4c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:06.571 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.561+0000 7fb5ba59c700 1 -- 172.21.15.143:0/3545944353 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb5bc15abf0 msgr2=0x7fb5bc1f6fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:06.571 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.561+0000 7fb5ba59c700 1 --2- 172.21.15.143:0/3545944353 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb5bc15abf0 0x7fb5bc1f6fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:06.571 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.561+0000 7fb5ba59c700 1 -- 172.21.15.143:0/3545944353 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5b00094a0 con 0x7fb5bc06f370 2023-11-08T19:31:06.571 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.561+0000 7fb5ba59c700 1 --2- 172.21.15.143:0/3545944353 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb5bc06f370 0x7fb5bc0af020 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac00c500 tx=0x7fb5ac00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:06.571 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.561+0000 7fb5aaffd700 1 -- 172.21.15.143:0/3545944353 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5ac004c80 con 0x7fb5bc06f370 2023-11-08T19:31:06.571 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.561+0000 7fb5aaffd700 1 -- 172.21.15.143:0/3545944353 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5ac004e00 con 0x7fb5bc06f370 2023-11-08T19:31:06.572 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.561+0000 7fb5c0d87700 1 -- 172.21.15.143:0/3545944353 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5bc1e5410 con 0x7fb5bc06f370 2023-11-08T19:31:06.572 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.561+0000 7fb5aaffd700 1 -- 172.21.15.143:0/3545944353 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5ac007310 con 0x7fb5bc06f370 2023-11-08T19:31:06.572 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.561+0000 7fb5c0d87700 1 -- 172.21.15.143:0/3545944353 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5bc1e59d0 con 0x7fb5bc06f370 2023-11-08T19:31:06.572 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.562+0000 7fb5c0d87700 1 -- 172.21.15.143:0/3545944353 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5bc04f730 con 0x7fb5bc06f370 2023-11-08T19:31:06.572 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.562+0000 7fb5aaffd700 1 -- 172.21.15.143:0/3545944353 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb5ac007470 con 0x7fb5bc06f370 2023-11-08T19:31:06.573 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.563+0000 7fb5aaffd700 1 --2- 172.21.15.143:0/3545944353 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fb59c06af00 0x7fb59c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:06.573 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.563+0000 7fb5aaffd700 1 -- 172.21.15.143:0/3545944353 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 7360+0+0 (secure 0 0 0) 0x7fb5ac089980 con 0x7fb5bc06f370 2023-11-08T19:31:06.573 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.563+0000 7fb5b9d9b700 1 --2- 172.21.15.143:0/3545944353 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fb59c06af00 0x7fb59c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:06.573 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.563+0000 7fb5b9d9b700 1 --2- 172.21.15.143:0/3545944353 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fb59c06af00 0x7fb59c06d3c0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc06f7b0 tx=0x7fb5b0008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:06.574 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.567+0000 7fb5aaffd700 1 -- 172.21.15.143:0/3545944353 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb5ac054450 con 0x7fb5bc06f370 2023-11-08T19:31:06.764 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.763+0000 7fb5c0d87700 1 -- 172.21.15.143:0/3545944353 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb5bc06dd60 con 0x7fb5bc06f370 2023-11-08T19:31:06.766 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.764+0000 7fb5aaffd700 1 -- 172.21.15.143:0/3545944353 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v104) v1 ==== 74+0+22721 (secure 0 0 0) 0x7fb5ac057a70 con 0x7fb5bc06f370 2023-11-08T19:31:06.766 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:31:06.767 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":104,"fsid":"968483af-77fc-4068-ad7c-5c2a1a4613d7","created":"2023-11-08T19:24:32.826484+0000","modified":"2023-11-08T19:30:55.774106+0000","last_up_change":"2023-11-08T19:24:43.573880+0000","last_in_change":"2023-11-08T19:24:35.314525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:44.551530+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"snap-test-pool","create_time":"2023-11-08T19:26:02.874384+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":36,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"test_data_pool","create_time":"2023-11-08T19:27:05.601949+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":66,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_metadata_pool","create_time":"2023-11-08T19:27:06.536369+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-11-08T19:28:01.533566+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-11-08T19:28:02.053445+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":9,"snap_epoch":100,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bae98ec1-d94e-403d-a220-a522a53ea6f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6809","nonce":1487830566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6811","nonce":1487830566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6815","nonce":1487830566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6813","nonce":1487830566}]},"public_addr":"172.21.15.143:6809/1487830566","cluster_addr":"172.21.15.143:6811/1487830566","heartbeat_back_addr":"172.21.15.143:6815/1487830566","heartbeat_front_addr":"172.21.15.143:6813/1487830566","state":["exists","up"]},{"osd":1,"uuid":"5cb6538a-1ccb-4322-8633-09f66af0b7e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6801","nonce":787341397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6803","nonce":787341397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6807","nonce":787341397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6805","nonce":787341397}]},"public_addr":"172.21.15.143:6801/787341397","cluster_addr":"172.21.15.143:6803/787341397","heartbeat_back_addr":"172.21.15.143:6807/787341397","heartbeat_front_addr":"172.21.15.143:6805/787341397","state":["exists","up"]},{"osd":2,"uuid":"cdbf522c-60ef-4d60-9653-8dcb529c101a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6822","nonce":2301325597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6826","nonce":2301325597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6831","nonce":2301325597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6829","nonce":2301325597}]},"public_addr":"172.21.15.143:6822/2301325597","cluster_addr":"172.21.15.143:6826/2301325597","heartbeat_back_addr":"172.21.15.143:6831/2301325597","heartbeat_front_addr":"172.21.15.143:6829/2301325597","state":["exists","up"]},{"osd":3,"uuid":"4e2783cf-ebb9-4dd8-8245-bc1d17ce4325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6817","nonce":2021504428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6819","nonce":2021504428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6827","nonce":2021504428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6823","nonce":2021504428}]},"public_addr":"172.21.15.143:6817/2021504428","cluster_addr":"172.21.15.143:6819/2021504428","heartbeat_back_addr":"172.21.15.143:6827/2021504428","heartbeat_front_addr":"172.21.15.143:6823/2021504428","state":["exists","up"]},{"osd":4,"uuid":"691256fa-9b77-49b9-a346-7ae1adf0ca80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6801","nonce":1993380595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6803","nonce":1993380595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6807","nonce":1993380595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6805","nonce":1993380595}]},"public_addr":"172.21.15.174:6801/1993380595","cluster_addr":"172.21.15.174:6803/1993380595","heartbeat_back_addr":"172.21.15.174:6807/1993380595","heartbeat_front_addr":"172.21.15.174:6805/1993380595","state":["exists","up"]},{"osd":5,"uuid":"756fc352-48c9-46ce-b52e-0541fe6f8525","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6809","nonce":1777923303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6811","nonce":1777923303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6815","nonce":1777923303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6813","nonce":1777923303}]},"public_addr":"172.21.15.174:6809/1777923303","cluster_addr":"172.21.15.174:6811/1777923303","heartbeat_back_addr":"172.21.15.174:6815/1777923303","heartbeat_front_addr":"172.21.15.174:6813/1777923303","state":["exists","up"]},{"osd":6,"uuid":"fab2ac2b-ed97-4cde-9825-b480eabf8b8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6825","nonce":2755130316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6827","nonce":2755130316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6831","nonce":2755130316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6829","nonce":2755130316}]},"public_addr":"172.21.15.174:6825/2755130316","cluster_addr":"172.21.15.174:6827/2755130316","heartbeat_back_addr":"172.21.15.174:6831/2755130316","heartbeat_front_addr":"172.21.15.174:6829/2755130316","state":["exists","up"]},{"osd":7,"uuid":"3de9b797-2ab6-4d41-a863-7e7d638fd703","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6817","nonce":2847141411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6819","nonce":2847141411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6823","nonce":2847141411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6821","nonce":2847141411}]},"public_addr":"172.21.15.174:6817/2847141411","cluster_addr":"172.21.15.174:6819/2847141411","heartbeat_back_addr":"172.21.15.174:6823/2847141411","heartbeat_front_addr":"172.21.15.174:6821/2847141411","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.477076+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.496890+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.539305+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.454673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.538881+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.521294+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.520712+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.540404+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.35","mappings":[{"from":5,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.174:6834/3601088618":"2023-11-09T19:30:47.736745+0000","172.21.15.143:6834/1535103237":"2023-11-09T19:30:40.630538+0000","172.21.15.143:6832/1535103237":"2023-11-09T19:30:40.630538+0000","172.21.15.143:0/2918728496":"2023-11-08T20:30:55.736484+0000","172.21.15.143:6835/3664584522":"2023-11-09T19:30:12.566205+0000","172.21.15.143:6833/3664584522":"2023-11-09T19:30:12.566205+0000","172.21.15.143:6832/3172664377":"2023-11-09T19:29:41.150862+0000","172.21.15.174:6836/3601088618":"2023-11-09T19:30:47.736745+0000","172.21.15.174:6837/2201447629":"2023-11-09T19:28:48.342407+0000","172.21.15.174:6835/2201447629":"2023-11-09T19:28:48.342407+0000","172.21.15.143:6833/3953059900":"2023-11-09T19:29:14.940251+0000","172.21.15.143:6835/3953059900":"2023-11-09T19:29:14.940251+0000","172.21.15.143:6834/3172664377":"2023-11-09T19:29:41.150862+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:31:06.768 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.767+0000 7fb5c0d87700 1 -- 172.21.15.143:0/3545944353 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fb59c06af00 msgr2=0x7fb59c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:06.769 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.767+0000 7fb5c0d87700 1 --2- 172.21.15.143:0/3545944353 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fb59c06af00 0x7fb59c06d3c0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc06f7b0 tx=0x7fb5b0008030 comp rx=0 tx=0).stop 2023-11-08T19:31:06.769 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.767+0000 7fb5c0d87700 1 -- 172.21.15.143:0/3545944353 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb5bc06f370 msgr2=0x7fb5bc0af020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:06.769 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.767+0000 7fb5c0d87700 1 --2- 172.21.15.143:0/3545944353 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb5bc06f370 0x7fb5bc0af020 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac00c500 tx=0x7fb5ac00c8c0 comp rx=0 tx=0).stop 2023-11-08T19:31:06.769 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.767+0000 7fb5c0d87700 1 -- 172.21.15.143:0/3545944353 shutdown_connections 2023-11-08T19:31:06.769 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.767+0000 7fb5c0d87700 1 --2- 172.21.15.143:0/3545944353 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fb59c06af00 0x7fb59c06d3c0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:06.770 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.767+0000 7fb5c0d87700 1 --2- 172.21.15.143:0/3545944353 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb5bc15abf0 0x7fb5bc1f6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:06.770 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.767+0000 7fb5c0d87700 1 --2- 172.21.15.143:0/3545944353 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb5bc06f370 0x7fb5bc0af020 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:06.770 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.767+0000 7fb5c0d87700 1 --2- 172.21.15.143:0/3545944353 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb5bc0b4100 0x7fb5bc1e4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:06.770 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.767+0000 7fb5c0d87700 1 -- 172.21.15.143:0/3545944353 >> 172.21.15.143:0/3545944353 conn(0x7fb5bc06e260 msgr2=0x7fb5bc0cd340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:06.771 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.767+0000 7fb5c0d87700 1 -- 172.21.15.143:0/3545944353 shutdown_connections 2023-11-08T19:31:06.771 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.767+0000 7fb5c0d87700 1 -- 172.21.15.143:0/3545944353 wait complete. 2023-11-08T19:31:06.780 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T19:24:44.551530+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'snap-test-pool', 'create_time': '2023-11-08T19:26:02.874384+0000', 'flags': 16385, 'flags_names': 'hashpspool,pool_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '36', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'pool', 'snap_seq': 2, 'snap_epoch': 36, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'test_data_pool', 'create_time': '2023-11-08T19:27:05.601949+0000', 'flags': 16385, 'flags_names': 'hashpspool,pool_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'pool', 'snap_seq': 2, 'snap_epoch': 66, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'test_metadata_pool', 'create_time': '2023-11-08T19:27:06.536369+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-08T19:28:01.533566+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2023-11-08T19:28:02.053445+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '100', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 9, 'snap_epoch': 100, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-08T19:31:06.781 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T19:31:06.906 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.905+0000 7f6c71286700 1 Processor -- start 2023-11-08T19:31:06.906 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.905+0000 7f6c71286700 1 -- start start 2023-11-08T19:31:06.908 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.906+0000 7f6c71286700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f6c6c0d3510 0x7f6c6c1687c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:06.909 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.906+0000 7f6c71286700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c6c0d2240 con 0x7f6c6c0d1e00 2023-11-08T19:31:06.909 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.906+0000 7f6c71286700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c6c0d2470 con 0x7f6c6c0ce7f0 2023-11-08T19:31:06.909 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.906+0000 7f6c71286700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c6c0d30f0 con 0x7f6c6c0d3510 2023-11-08T19:31:06.909 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.906+0000 7f6c6b7fe700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f6c6c0d3510 0x7f6c6c1687c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:06.909 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.906+0000 7f6c6a7fc700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f6c6c0d1e00 0x7f6c6c0cebd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:52964/0 (socket says 172.21.15.143:52964) 2023-11-08T19:31:06.910 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.906+0000 7f6c6a7fc700 1 -- 172.21.15.143:0/1252358970 learned_addr learned my addr 172.21.15.143:0/1252358970 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:31:06.910 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.906+0000 7f6c6b7fe700 1 -- 172.21.15.143:0/1252358970 >> v1:172.21.15.174:6789/0 conn(0x7f6c6c0ce7f0 legacy=0x7f6c6c0b9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:06.910 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.906+0000 7f6c6b7fe700 1 -- 172.21.15.143:0/1252358970 >> v1:172.21.15.143:6789/0 conn(0x7f6c6c0d1e00 legacy=0x7f6c6c0cebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:06.910 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.906+0000 7f6c6b7fe700 1 -- 172.21.15.143:0/1252358970 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c6c0d2750 con 0x7f6c6c0d3510 2023-11-08T19:31:06.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.906+0000 7f6c69ffb700 1 -- 172.21.15.143:0/1252358970 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1894851759 0 0) 0x7f6c6c0d2240 con 0x7f6c6c0d1e00 2023-11-08T19:31:06.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.906+0000 7f6c6b7fe700 1 --2- 172.21.15.143:0/1252358970 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6c6c0d3510 0x7f6c6c1687c0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f6c64009d90 tx=0x7f6c6400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2c74c9769d112fd server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:06.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.906+0000 7f6c69ffb700 1 -- 172.21.15.143:0/1252358970 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c6400e070 con 0x7f6c6c0d3510 2023-11-08T19:31:06.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.906+0000 7f6c69ffb700 1 -- 172.21.15.143:0/1252358970 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c6400bb90 con 0x7f6c6c0d3510 2023-11-08T19:31:06.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.906+0000 7f6c69ffb700 1 -- 172.21.15.143:0/1252358970 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c64003a20 con 0x7f6c6c0d3510 2023-11-08T19:31:06.912 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.907+0000 7f6c71286700 1 -- 172.21.15.143:0/1252358970 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6c6c0d3510 msgr2=0x7f6c6c1687c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:06.912 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.907+0000 7f6c71286700 1 --2- 172.21.15.143:0/1252358970 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6c6c0d3510 0x7f6c6c1687c0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f6c64009d90 tx=0x7f6c6400b330 comp rx=0 tx=0).stop 2023-11-08T19:31:06.912 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.907+0000 7f6c71286700 1 -- 172.21.15.143:0/1252358970 shutdown_connections 2023-11-08T19:31:06.912 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.907+0000 7f6c71286700 1 --2- 172.21.15.143:0/1252358970 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6c6c0d3510 0x7f6c6c1687c0 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:06.912 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.907+0000 7f6c71286700 1 -- 172.21.15.143:0/1252358970 >> 172.21.15.143:0/1252358970 conn(0x7f6c6c0d05a0 msgr2=0x7f6c6c0d09b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:06.913 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.907+0000 7f6c71286700 1 -- 172.21.15.143:0/1252358970 shutdown_connections 2023-11-08T19:31:06.913 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.907+0000 7f6c71286700 1 -- 172.21.15.143:0/1252358970 wait complete. 2023-11-08T19:31:06.915 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.908+0000 7f6c71286700 1 Processor -- start 2023-11-08T19:31:06.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.908+0000 7f6c71286700 1 -- start start 2023-11-08T19:31:06.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.909+0000 7f6c71286700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6c6c0ce7f0 0x7f6c6c0ca010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:06.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.909+0000 7f6c71286700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6c6c0d1e00 0x7f6c6c156f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:06.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.909+0000 7f6c71286700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6c6c0d3510 0x7f6c6c0cbbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:06.917 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.909+0000 7f6c71286700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c6c0c8d20 con 0x7f6c6c0ce7f0 2023-11-08T19:31:06.917 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.909+0000 7f6c71286700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c6c0c9b80 con 0x7f6c6c0d3510 2023-11-08T19:31:06.917 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.909+0000 7f6c71286700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c6c0cdcf0 con 0x7f6c6c0d1e00 2023-11-08T19:31:06.917 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.909+0000 7f6c6affd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6c6c0ce7f0 0x7f6c6c0ca010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:06.917 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.909+0000 7f6c6affd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6c6c0ce7f0 0x7f6c6c0ca010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56818/0 (socket says 172.21.15.143:56818) 2023-11-08T19:31:06.918 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.909+0000 7f6c6affd700 1 -- 172.21.15.143:0/590739749 learned_addr learned my addr 172.21.15.143:0/590739749 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:31:06.918 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.909+0000 7f6c6a7fc700 1 --2- 172.21.15.143:0/590739749 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6c6c0d1e00 0x7f6c6c156f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:06.918 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.909+0000 7f6c6affd700 1 -- 172.21.15.143:0/590739749 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6c6c0d1e00 msgr2=0x7f6c6c156f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:06.918 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.909+0000 7f6c6affd700 1 --2- 172.21.15.143:0/590739749 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6c6c0d1e00 0x7f6c6c156f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:06.919 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.909+0000 7f6c6affd700 1 -- 172.21.15.143:0/590739749 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6c6c0d3510 msgr2=0x7f6c6c0cbbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:06.919 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.909+0000 7f6c6affd700 1 --2- 172.21.15.143:0/590739749 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6c6c0d3510 0x7f6c6c0cbbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:06.919 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.909+0000 7f6c6affd700 1 -- 172.21.15.143:0/590739749 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c64009a70 con 0x7f6c6c0ce7f0 2023-11-08T19:31:06.919 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.910+0000 7f6c6affd700 1 --2- 172.21.15.143:0/590739749 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6c6c0ce7f0 0x7f6c6c0ca010 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f6c5c00b4a0 tx=0x7f6c5c00b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:06.919 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.910+0000 7f6c537fe700 1 -- 172.21.15.143:0/590739749 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c5c011830 con 0x7f6c6c0ce7f0 2023-11-08T19:31:06.920 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.910+0000 7f6c537fe700 1 -- 172.21.15.143:0/590739749 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c5c0119b0 con 0x7f6c6c0ce7f0 2023-11-08T19:31:06.920 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.910+0000 7f6c71286700 1 -- 172.21.15.143:0/590739749 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c6c1574c0 con 0x7f6c6c0ce7f0 2023-11-08T19:31:06.920 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.910+0000 7f6c537fe700 1 -- 172.21.15.143:0/590739749 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c5c011d60 con 0x7f6c6c0ce7f0 2023-11-08T19:31:06.920 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.910+0000 7f6c71286700 1 -- 172.21.15.143:0/590739749 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c6c0c9250 con 0x7f6c6c0ce7f0 2023-11-08T19:31:06.921 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.911+0000 7f6c71286700 1 -- 172.21.15.143:0/590739749 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c6c0c9e60 con 0x7f6c6c0ce7f0 2023-11-08T19:31:06.921 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.911+0000 7f6c537fe700 1 -- 172.21.15.143:0/590739749 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6c5c01a400 con 0x7f6c6c0ce7f0 2023-11-08T19:31:06.921 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.911+0000 7f6c537fe700 1 --2- 172.21.15.143:0/590739749 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f6c5806af00 0x7f6c5806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:06.921 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.912+0000 7f6c537fe700 1 -- 172.21.15.143:0/590739749 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 7360+0+0 (secure 0 0 0) 0x7f6c5c089cf0 con 0x7f6c6c0ce7f0 2023-11-08T19:31:06.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.912+0000 7f6c6a7fc700 1 --2- 172.21.15.143:0/590739749 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f6c5806af00 0x7f6c5806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:06.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.912+0000 7f6c6a7fc700 1 --2- 172.21.15.143:0/590739749 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f6c5806af00 0x7f6c5806d3c0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f6c60000f60 tx=0x7f6c60009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:06.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:06.916+0000 7f6c537fe700 1 -- 172.21.15.143:0/590739749 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6c5c08e070 con 0x7f6c6c0ce7f0 2023-11-08T19:31:07.105 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.104+0000 7f6c71286700 1 -- 172.21.15.143:0/590739749 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6c6c04f730 con 0x7f6c6c0ce7f0 2023-11-08T19:31:07.106 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.104+0000 7f6c537fe700 1 -- 172.21.15.143:0/590739749 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v104) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6c5c0150a0 con 0x7f6c6c0ce7f0 2023-11-08T19:31:07.106 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2023-11-08T19:31:07.108 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.106+0000 7f6c71286700 1 -- 172.21.15.143:0/590739749 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f6c5806af00 msgr2=0x7f6c5806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:07.108 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.106+0000 7f6c71286700 1 --2- 172.21.15.143:0/590739749 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f6c5806af00 0x7f6c5806d3c0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f6c60000f60 tx=0x7f6c60009040 comp rx=0 tx=0).stop 2023-11-08T19:31:07.109 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.106+0000 7f6c71286700 1 -- 172.21.15.143:0/590739749 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6c6c0ce7f0 msgr2=0x7f6c6c0ca010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:07.109 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.106+0000 7f6c71286700 1 --2- 172.21.15.143:0/590739749 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6c6c0ce7f0 0x7f6c6c0ca010 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f6c5c00b4a0 tx=0x7f6c5c00b860 comp rx=0 tx=0).stop 2023-11-08T19:31:07.109 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.106+0000 7f6c71286700 1 -- 172.21.15.143:0/590739749 shutdown_connections 2023-11-08T19:31:07.109 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.106+0000 7f6c71286700 1 --2- 172.21.15.143:0/590739749 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f6c5806af00 0x7f6c5806d3c0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:07.109 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.106+0000 7f6c71286700 1 --2- 172.21.15.143:0/590739749 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6c6c0d3510 0x7f6c6c0cbbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:07.110 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.106+0000 7f6c71286700 1 --2- 172.21.15.143:0/590739749 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6c6c0ce7f0 0x7f6c6c0ca010 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:07.110 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.106+0000 7f6c71286700 1 --2- 172.21.15.143:0/590739749 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6c6c0d1e00 0x7f6c6c156f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:07.110 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.106+0000 7f6c71286700 1 -- 172.21.15.143:0/590739749 >> 172.21.15.143:0/590739749 conn(0x7f6c6c0d05a0 msgr2=0x7f6c6c0b8aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:07.110 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.106+0000 7f6c71286700 1 -- 172.21.15.143:0/590739749 shutdown_connections 2023-11-08T19:31:07.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.106+0000 7f6c71286700 1 -- 172.21.15.143:0/590739749 wait complete. 2023-11-08T19:31:07.120 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get snap-test-pool pg_num 2023-11-08T19:31:07.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.244+0000 7f7612e84700 1 Processor -- start 2023-11-08T19:31:07.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.244+0000 7f7612e84700 1 -- start start 2023-11-08T19:31:07.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.245+0000 7f7612e84700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f760c15f4b0 0x7f760c0cfed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:07.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.245+0000 7f7612e84700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f760c0ce9c0 0x7f760c0d0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:07.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.245+0000 7f7612e84700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f760c0b2c20 con 0x7f760c0c56b0 2023-11-08T19:31:07.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.245+0000 7f7612e84700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f760c0d2800 con 0x7f760c0ce9c0 2023-11-08T19:31:07.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.245+0000 7f7612e84700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f760c0b30b0 con 0x7f760c15f4b0 2023-11-08T19:31:07.249 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.245+0000 7f760bfff700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f760c0ce9c0 0x7f760c0d0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:07.249 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.245+0000 7f7611421700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f760c0c56b0 0x7f760c0c5b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:52982/0 (socket says 172.21.15.143:52982) 2023-11-08T19:31:07.249 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.245+0000 7f7611421700 1 -- 172.21.15.143:0/238760646 learned_addr learned my addr 172.21.15.143:0/238760646 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:31:07.249 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.245+0000 7f760bfff700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f760c0ce9c0 0x7f760c0d0410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56826/0 (socket says 172.21.15.143:56826) 2023-11-08T19:31:07.250 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.245+0000 7f7610c20700 1 --2- 172.21.15.143:0/238760646 >> v2:172.21.15.174:3300/0 conn(0x7f760c15f4b0 0x7f760c0cfed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:07.250 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.245+0000 7f760bfff700 1 -- 172.21.15.143:0/238760646 >> v2:172.21.15.174:3300/0 conn(0x7f760c15f4b0 msgr2=0x7f760c0cfed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:07.250 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.245+0000 7f760bfff700 1 --2- 172.21.15.143:0/238760646 >> v2:172.21.15.174:3300/0 conn(0x7f760c15f4b0 0x7f760c0cfed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:07.250 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.245+0000 7f760bfff700 1 -- 172.21.15.143:0/238760646 >> v1:172.21.15.143:6789/0 conn(0x7f760c0c56b0 legacy=0x7f760c0c5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:07.250 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.245+0000 7f760bfff700 1 -- 172.21.15.143:0/238760646 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f760c0d24b0 con 0x7f760c0ce9c0 2023-11-08T19:31:07.251 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.245+0000 7f7610c20700 1 --2- 172.21.15.143:0/238760646 >> v2:172.21.15.174:3300/0 conn(0x7f760c15f4b0 0x7f760c0cfed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:31:07.251 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.245+0000 7f760bfff700 1 --2- 172.21.15.143:0/238760646 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f760c0ce9c0 0x7f760c0d0410 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f760c0712e0 tx=0x7f75f8009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b7aef1559e28126 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:07.251 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.245+0000 7f760b7fe700 1 -- 172.21.15.143:0/238760646 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75f8011830 con 0x7f760c0ce9c0 2023-11-08T19:31:07.251 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.245+0000 7f760b7fe700 1 -- 172.21.15.143:0/238760646 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75f8011990 con 0x7f760c0ce9c0 2023-11-08T19:31:07.251 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.245+0000 7f760b7fe700 1 -- 172.21.15.143:0/238760646 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75f8011d20 con 0x7f760c0ce9c0 2023-11-08T19:31:07.251 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.246+0000 7f7612e84700 1 -- 172.21.15.143:0/238760646 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f760c0ce9c0 msgr2=0x7f760c0d0410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:07.252 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.246+0000 7f7612e84700 1 --2- 172.21.15.143:0/238760646 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f760c0ce9c0 0x7f760c0d0410 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f760c0712e0 tx=0x7f75f8009b20 comp rx=0 tx=0).stop 2023-11-08T19:31:07.253 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.246+0000 7f7612e84700 1 -- 172.21.15.143:0/238760646 shutdown_connections 2023-11-08T19:31:07.254 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.246+0000 7f7612e84700 1 --2- 172.21.15.143:0/238760646 >> v2:172.21.15.174:3300/0 conn(0x7f760c15f4b0 0x7f760c0cfed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:07.254 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.246+0000 7f7612e84700 1 --2- 172.21.15.143:0/238760646 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f760c0ce9c0 0x7f760c0d0410 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:07.254 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.246+0000 7f7612e84700 1 -- 172.21.15.143:0/238760646 >> 172.21.15.143:0/238760646 conn(0x7f760c0d18f0 msgr2=0x7f760c0d1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:07.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.246+0000 7f7612e84700 1 -- 172.21.15.143:0/238760646 shutdown_connections 2023-11-08T19:31:07.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.246+0000 7f7612e84700 1 -- 172.21.15.143:0/238760646 wait complete. 2023-11-08T19:31:07.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.247+0000 7f7612e84700 1 Processor -- start 2023-11-08T19:31:07.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.247+0000 7f7612e84700 1 -- start start 2023-11-08T19:31:07.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.248+0000 7f7612e84700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f760c0c56b0 0x7f760c0b1e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:07.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.248+0000 7f7612e84700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f760c0ce9c0 0x7f760c14d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:07.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.248+0000 7f7612e84700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f760c15f4b0 0x7f760c14ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:07.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.248+0000 7f7612e84700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f760c0cb370 con 0x7f760c0ce9c0 2023-11-08T19:31:07.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.248+0000 7f7612e84700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f760c14b910 con 0x7f760c0c56b0 2023-11-08T19:31:07.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.248+0000 7f7612e84700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f760c14bbf0 con 0x7f760c15f4b0 2023-11-08T19:31:07.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.248+0000 7f760bfff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f760c0ce9c0 0x7f760c14d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:07.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.248+0000 7f760bfff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f760c0ce9c0 0x7f760c14d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56832/0 (socket says 172.21.15.143:56832) 2023-11-08T19:31:07.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.248+0000 7f760bfff700 1 -- 172.21.15.143:0/2726981601 learned_addr learned my addr 172.21.15.143:0/2726981601 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:31:07.261 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.248+0000 7f7611421700 1 --2- 172.21.15.143:0/2726981601 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f760c15f4b0 0x7f760c14ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:07.261 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.248+0000 7f760bfff700 1 -- 172.21.15.143:0/2726981601 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f760c15f4b0 msgr2=0x7f760c14ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:07.261 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.248+0000 7f760bfff700 1 --2- 172.21.15.143:0/2726981601 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f760c15f4b0 0x7f760c14ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:07.261 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.248+0000 7f760bfff700 1 -- 172.21.15.143:0/2726981601 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f760c0c56b0 msgr2=0x7f760c0b1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:07.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.248+0000 7f760bfff700 1 --2- 172.21.15.143:0/2726981601 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f760c0c56b0 0x7f760c0b1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:07.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.248+0000 7f760bfff700 1 -- 172.21.15.143:0/2726981601 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75f80094a0 con 0x7f760c0ce9c0 2023-11-08T19:31:07.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.248+0000 7f760bfff700 1 --2- 172.21.15.143:0/2726981601 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f760c0ce9c0 0x7f760c14d870 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f75f8009b60 tx=0x7f75f8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:07.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.249+0000 7f76097fa700 1 -- 172.21.15.143:0/2726981601 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75f80038a0 con 0x7f760c0ce9c0 2023-11-08T19:31:07.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.249+0000 7f76097fa700 1 -- 172.21.15.143:0/2726981601 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75f801e070 con 0x7f760c0ce9c0 2023-11-08T19:31:07.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.249+0000 7f7612e84700 1 -- 172.21.15.143:0/2726981601 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f760c14bed0 con 0x7f760c0ce9c0 2023-11-08T19:31:07.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.249+0000 7f76097fa700 1 -- 172.21.15.143:0/2726981601 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75f8003c20 con 0x7f760c0ce9c0 2023-11-08T19:31:07.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.249+0000 7f7612e84700 1 -- 172.21.15.143:0/2726981601 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f760c15ae40 con 0x7f760c0ce9c0 2023-11-08T19:31:07.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.250+0000 7f7612e84700 1 -- 172.21.15.143:0/2726981601 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f760c14bd30 con 0x7f760c0ce9c0 2023-11-08T19:31:07.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.250+0000 7f76097fa700 1 -- 172.21.15.143:0/2726981601 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f75f802d450 con 0x7f760c0ce9c0 2023-11-08T19:31:07.264 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.250+0000 7f76097fa700 1 --2- 172.21.15.143:0/2726981601 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f75f006af00 0x7f75f006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:07.264 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.251+0000 7f76097fa700 1 -- 172.21.15.143:0/2726981601 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 7360+0+0 (secure 0 0 0) 0x7f75f808acd0 con 0x7f760c0ce9c0 2023-11-08T19:31:07.264 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.251+0000 7f7610c20700 1 --2- 172.21.15.143:0/2726981601 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f75f006af00 0x7f75f006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:07.264 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.251+0000 7f7610c20700 1 --2- 172.21.15.143:0/2726981601 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f75f006af00 0x7f75f006d3c0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f7600006f30 tx=0x7f7600008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:07.264 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.255+0000 7f76097fa700 1 -- 172.21.15.143:0/2726981601 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f760c14bd30 con 0x7f760c0ce9c0 2023-11-08T19:31:07.445 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.443+0000 7f7612e84700 1 -- 172.21.15.143:0/2726981601 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"} v 0) v1 -- 0x7f760c14c3f0 con 0x7f760c0ce9c0 2023-11-08T19:31:07.445 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.444+0000 7f76097fa700 1 -- 172.21.15.143:0/2726981601 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"}]=0 v104) v1 ==== 103+0+11 (secure 0 0 0) 0x7f760c14c3f0 con 0x7f760c0ce9c0 2023-11-08T19:31:07.446 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2023-11-08T19:31:07.448 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.446+0000 7f7612e84700 1 -- 172.21.15.143:0/2726981601 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f75f006af00 msgr2=0x7f75f006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:07.448 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.446+0000 7f7612e84700 1 --2- 172.21.15.143:0/2726981601 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f75f006af00 0x7f75f006d3c0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f7600006f30 tx=0x7f7600008040 comp rx=0 tx=0).stop 2023-11-08T19:31:07.448 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.446+0000 7f7612e84700 1 -- 172.21.15.143:0/2726981601 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f760c0ce9c0 msgr2=0x7f760c14d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:07.448 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.446+0000 7f7612e84700 1 --2- 172.21.15.143:0/2726981601 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f760c0ce9c0 0x7f760c14d870 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f75f8009b60 tx=0x7f75f8004750 comp rx=0 tx=0).stop 2023-11-08T19:31:07.449 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.446+0000 7f7612e84700 1 -- 172.21.15.143:0/2726981601 shutdown_connections 2023-11-08T19:31:07.449 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.446+0000 7f7612e84700 1 --2- 172.21.15.143:0/2726981601 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f75f006af00 0x7f75f006d3c0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:07.449 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.446+0000 7f7612e84700 1 --2- 172.21.15.143:0/2726981601 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f760c15f4b0 0x7f760c14ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:07.449 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.446+0000 7f7612e84700 1 --2- 172.21.15.143:0/2726981601 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f760c0c56b0 0x7f760c0b1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:07.450 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.446+0000 7f7612e84700 1 --2- 172.21.15.143:0/2726981601 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f760c0ce9c0 0x7f760c14d870 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:07.450 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.446+0000 7f7612e84700 1 -- 172.21.15.143:0/2726981601 >> 172.21.15.143:0/2726981601 conn(0x7f760c0d18f0 msgr2=0x7f760c0c7aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:07.450 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.446+0000 7f7612e84700 1 -- 172.21.15.143:0/2726981601 shutdown_connections 2023-11-08T19:31:07.450 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.446+0000 7f7612e84700 1 -- 172.21.15.143:0/2726981601 wait complete. 2023-11-08T19:31:07.459 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get test_data_pool pg_num 2023-11-08T19:31:07.583 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.581+0000 7f12a8bd1700 1 Processor -- start 2023-11-08T19:31:07.583 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.581+0000 7f12a8bd1700 1 -- start start 2023-11-08T19:31:07.585 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.582+0000 7f12a8bd1700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f12a4072110 0x7f12a40b4990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:07.585 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.582+0000 7f12a8bd1700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12a4071650 con 0x7f12a40b4ed0 2023-11-08T19:31:07.586 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.582+0000 7f12a8bd1700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12a40b2f80 con 0x7f12a415a800 2023-11-08T19:31:07.586 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.582+0000 7f12a8bd1700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12a40b3260 con 0x7f12a4072110 2023-11-08T19:31:07.586 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.583+0000 7f12a1d9b700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f12a4072110 0x7f12a40b4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:07.586 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.583+0000 7f12a2d9d700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f12a40b4ed0 0x7f12a40cca40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:52996/0 (socket says 172.21.15.143:52996) 2023-11-08T19:31:07.586 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.583+0000 7f12a2d9d700 1 -- 172.21.15.143:0/1144300578 learned_addr learned my addr 172.21.15.143:0/1144300578 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:31:07.587 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.583+0000 7f12a1d9b700 1 -- 172.21.15.143:0/1144300578 >> v1:172.21.15.174:6789/0 conn(0x7f12a415a800 legacy=0x7f12a40b2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:07.587 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.583+0000 7f12a1d9b700 1 -- 172.21.15.143:0/1144300578 >> v1:172.21.15.143:6789/0 conn(0x7f12a40b4ed0 legacy=0x7f12a40cca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:07.587 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.583+0000 7f12a1d9b700 1 -- 172.21.15.143:0/1144300578 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12a4071910 con 0x7f12a4072110 2023-11-08T19:31:07.587 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.583+0000 7f12a159a700 1 -- 172.21.15.143:0/1144300578 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 433709866 0 0) 0x7f12a4071650 con 0x7f12a40b4ed0 2023-11-08T19:31:07.587 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.583+0000 7f12a1d9b700 1 --2- 172.21.15.143:0/1144300578 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f12a4072110 0x7f12a40b4990 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f1294009840 tx=0x7f1294009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98f01b28934ffba0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:07.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.583+0000 7f12a159a700 1 -- 172.21.15.143:0/1144300578 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1294011830 con 0x7f12a4072110 2023-11-08T19:31:07.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.583+0000 7f12a159a700 1 -- 172.21.15.143:0/1144300578 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1294011990 con 0x7f12a4072110 2023-11-08T19:31:07.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.583+0000 7f12a159a700 1 -- 172.21.15.143:0/1144300578 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1294011d20 con 0x7f12a4072110 2023-11-08T19:31:07.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.584+0000 7f12a8bd1700 1 -- 172.21.15.143:0/1144300578 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f12a4072110 msgr2=0x7f12a40b4990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:07.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.584+0000 7f12a8bd1700 1 --2- 172.21.15.143:0/1144300578 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f12a4072110 0x7f12a40b4990 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f1294009840 tx=0x7f1294009b50 comp rx=0 tx=0).stop 2023-11-08T19:31:07.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.584+0000 7f12a8bd1700 1 -- 172.21.15.143:0/1144300578 shutdown_connections 2023-11-08T19:31:07.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.584+0000 7f12a8bd1700 1 --2- 172.21.15.143:0/1144300578 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f12a4072110 0x7f12a40b4990 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:07.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.584+0000 7f12a8bd1700 1 -- 172.21.15.143:0/1144300578 >> 172.21.15.143:0/1144300578 conn(0x7f12a406bcf0 msgr2=0x7f12a406c100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:07.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.584+0000 7f12a8bd1700 1 -- 172.21.15.143:0/1144300578 shutdown_connections 2023-11-08T19:31:07.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.584+0000 7f12a8bd1700 1 -- 172.21.15.143:0/1144300578 wait complete. 2023-11-08T19:31:07.592 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.585+0000 7f12a8bd1700 1 Processor -- start 2023-11-08T19:31:07.592 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.585+0000 7f12a8bd1700 1 -- start start 2023-11-08T19:31:07.593 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.586+0000 7f12a8bd1700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f12a4072110 0x7f12a414dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:07.593 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.586+0000 7f12a8bd1700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f12a40b4ed0 0x7f12a41f76b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:07.593 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.586+0000 7f12a8bd1700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f12a415a800 0x7f12a41f36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:07.593 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.586+0000 7f12a8bd1700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12a41f46c0 con 0x7f12a4072110 2023-11-08T19:31:07.593 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.586+0000 7f12a8bd1700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12a41edb70 con 0x7f12a40b4ed0 2023-11-08T19:31:07.594 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.586+0000 7f12a8bd1700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12a41f7bf0 con 0x7f12a415a800 2023-11-08T19:31:07.594 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.586+0000 7f12a259c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f12a4072110 0x7f12a414dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:07.594 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.586+0000 7f12a259c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f12a4072110 0x7f12a414dc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56844/0 (socket says 172.21.15.143:56844) 2023-11-08T19:31:07.594 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.586+0000 7f12a259c700 1 -- 172.21.15.143:0/2945487574 learned_addr learned my addr 172.21.15.143:0/2945487574 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:31:07.594 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.586+0000 7f12a1d9b700 1 --2- 172.21.15.143:0/2945487574 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f12a40b4ed0 0x7f12a41f76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:07.595 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.586+0000 7f12a2d9d700 1 --2- 172.21.15.143:0/2945487574 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f12a415a800 0x7f12a41f36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:07.595 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.586+0000 7f12a259c700 1 -- 172.21.15.143:0/2945487574 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f12a415a800 msgr2=0x7f12a41f36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:07.595 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.586+0000 7f12a259c700 1 --2- 172.21.15.143:0/2945487574 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f12a415a800 0x7f12a41f36d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:07.595 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.586+0000 7f12a259c700 1 -- 172.21.15.143:0/2945487574 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f12a40b4ed0 msgr2=0x7f12a41f76b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:07.596 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.586+0000 7f12a259c700 1 --2- 172.21.15.143:0/2945487574 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f12a40b4ed0 0x7f12a41f76b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:07.596 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.586+0000 7f12a259c700 1 -- 172.21.15.143:0/2945487574 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12940094a0 con 0x7f12a4072110 2023-11-08T19:31:07.596 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.586+0000 7f12a259c700 1 --2- 172.21.15.143:0/2945487574 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f12a4072110 0x7f12a414dc20 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f129800c4e0 tx=0x7f129800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:07.596 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.586+0000 7f1292ffd700 1 -- 172.21.15.143:0/2945487574 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12980052d0 con 0x7f12a4072110 2023-11-08T19:31:07.596 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.586+0000 7f1292ffd700 1 -- 172.21.15.143:0/2945487574 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1298005430 con 0x7f12a4072110 2023-11-08T19:31:07.596 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.586+0000 7f12a8bd1700 1 -- 172.21.15.143:0/2945487574 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12a41f4100 con 0x7f12a4072110 2023-11-08T19:31:07.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.586+0000 7f1292ffd700 1 -- 172.21.15.143:0/2945487574 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12980038c0 con 0x7f12a4072110 2023-11-08T19:31:07.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.586+0000 7f12a8bd1700 1 -- 172.21.15.143:0/2945487574 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12a41f6eb0 con 0x7f12a4072110 2023-11-08T19:31:07.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.587+0000 7f12a8bd1700 1 -- 172.21.15.143:0/2945487574 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12a41f6d10 con 0x7f12a4072110 2023-11-08T19:31:07.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.588+0000 7f1292ffd700 1 -- 172.21.15.143:0/2945487574 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1298003a20 con 0x7f12a4072110 2023-11-08T19:31:07.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.588+0000 7f1292ffd700 1 --2- 172.21.15.143:0/2945487574 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f128406af00 0x7f128406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:07.598 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.588+0000 7f1292ffd700 1 -- 172.21.15.143:0/2945487574 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 7360+0+0 (secure 0 0 0) 0x7f1298089090 con 0x7f12a4072110 2023-11-08T19:31:07.598 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.588+0000 7f12a1d9b700 1 --2- 172.21.15.143:0/2945487574 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f128406af00 0x7f128406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:07.598 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.589+0000 7f12a1d9b700 1 --2- 172.21.15.143:0/2945487574 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f128406af00 0x7f128406d3c0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f12a4156b30 tx=0x7f129401c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:07.599 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.593+0000 7f1292ffd700 1 -- 172.21.15.143:0/2945487574 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f12a41f6d10 con 0x7f12a4072110 2023-11-08T19:31:07.790 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.788+0000 7f12a8bd1700 1 -- 172.21.15.143:0/2945487574 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "test_data_pool", "var": "pg_num"} v 0) v1 -- 0x7f12a41f6d10 con 0x7f12a4072110 2023-11-08T19:31:07.790 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.789+0000 7f1292ffd700 1 -- 172.21.15.143:0/2945487574 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "test_data_pool", "var": "pg_num"}]=0 v104) v1 ==== 103+0+11 (secure 0 0 0) 0x7f1298017020 con 0x7f12a4072110 2023-11-08T19:31:07.791 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2023-11-08T19:31:07.792 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.791+0000 7f12a8bd1700 1 -- 172.21.15.143:0/2945487574 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f128406af00 msgr2=0x7f128406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:07.793 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.791+0000 7f12a8bd1700 1 --2- 172.21.15.143:0/2945487574 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f128406af00 0x7f128406d3c0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f12a4156b30 tx=0x7f129401c040 comp rx=0 tx=0).stop 2023-11-08T19:31:07.793 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.791+0000 7f12a8bd1700 1 -- 172.21.15.143:0/2945487574 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f12a4072110 msgr2=0x7f12a414dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:07.793 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.791+0000 7f12a8bd1700 1 --2- 172.21.15.143:0/2945487574 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f12a4072110 0x7f12a414dc20 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f129800c4e0 tx=0x7f129800c8a0 comp rx=0 tx=0).stop 2023-11-08T19:31:07.793 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.791+0000 7f12a8bd1700 1 -- 172.21.15.143:0/2945487574 shutdown_connections 2023-11-08T19:31:07.793 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.791+0000 7f12a8bd1700 1 --2- 172.21.15.143:0/2945487574 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f128406af00 0x7f128406d3c0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:07.794 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.791+0000 7f12a8bd1700 1 --2- 172.21.15.143:0/2945487574 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f12a415a800 0x7f12a41f36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:07.794 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.791+0000 7f12a8bd1700 1 --2- 172.21.15.143:0/2945487574 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f12a4072110 0x7f12a414dc20 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:07.794 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.791+0000 7f12a8bd1700 1 --2- 172.21.15.143:0/2945487574 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f12a40b4ed0 0x7f12a41f76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:07.794 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.791+0000 7f12a8bd1700 1 -- 172.21.15.143:0/2945487574 >> 172.21.15.143:0/2945487574 conn(0x7f12a406bcf0 msgr2=0x7f12a40cad40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:07.794 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.791+0000 7f12a8bd1700 1 -- 172.21.15.143:0/2945487574 shutdown_connections 2023-11-08T19:31:07.795 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.791+0000 7f12a8bd1700 1 -- 172.21.15.143:0/2945487574 wait complete. 2023-11-08T19:31:07.804 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get test_metadata_pool pg_num 2023-11-08T19:31:07.928 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.927+0000 7fd2517db700 1 Processor -- start 2023-11-08T19:31:07.929 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.927+0000 7fd2517db700 1 -- start start 2023-11-08T19:31:07.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.928+0000 7fd2517db700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fd24c0ce7f0 0x7fd24c0d1e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:07.933 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.928+0000 7fd2517db700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd24c0d3970 con 0x7fd24c0d3510 2023-11-08T19:31:07.933 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.928+0000 7fd2517db700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd24c0d2470 con 0x7fd24c0cebd0 2023-11-08T19:31:07.933 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.928+0000 7fd2517db700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd24c0d30f0 con 0x7fd24c0ce7f0 2023-11-08T19:31:07.933 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.928+0000 7fd24a7fc700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fd24c0d3510 0x7fd24c0b9590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:53006/0 (socket says 172.21.15.143:53006) 2023-11-08T19:31:07.933 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.928+0000 7fd24a7fc700 1 -- 172.21.15.143:0/2635989922 learned_addr learned my addr 172.21.15.143:0/2635989922 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:31:07.934 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.928+0000 7fd24affd700 1 --2- 172.21.15.143:0/2635989922 >> v2:172.21.15.174:3300/0 conn(0x7fd24c0ce7f0 0x7fd24c0d1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:07.934 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.928+0000 7fd249ffb700 1 -- 172.21.15.143:0/2635989922 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3648420665 0 0) 0x7fd24c0d3970 con 0x7fd24c0d3510 2023-11-08T19:31:07.934 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.928+0000 7fd249ffb700 1 -- 172.21.15.143:0/2635989922 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2340035b0 con 0x7fd24c0d3510 2023-11-08T19:31:07.934 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.928+0000 7fd249ffb700 1 -- 172.21.15.143:0/2635989922 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1108461986 0 0) 0x7fd2340035b0 con 0x7fd24c0d3510 2023-11-08T19:31:07.935 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.928+0000 7fd249ffb700 1 -- 172.21.15.143:0/2635989922 >> v2:172.21.15.174:3300/0 conn(0x7fd24c0ce7f0 msgr2=0x7fd24c0d1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:07.935 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.928+0000 7fd249ffb700 1 --2- 172.21.15.143:0/2635989922 >> v2:172.21.15.174:3300/0 conn(0x7fd24c0ce7f0 0x7fd24c0d1e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:07.935 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.928+0000 7fd249ffb700 1 -- 172.21.15.143:0/2635989922 >> v1:172.21.15.174:6789/0 conn(0x7fd24c0cebd0 legacy=0x7fd24c0b6c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:07.935 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.928+0000 7fd249ffb700 1 -- 172.21.15.143:0/2635989922 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd24c0d2750 con 0x7fd24c0d3510 2023-11-08T19:31:07.935 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.928+0000 7fd249ffb700 1 -- 172.21.15.143:0/2635989922 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4147756303 0 0) 0x7fd23c001670 con 0x7fd24c0d3510 2023-11-08T19:31:07.936 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.929+0000 7fd249ffb700 1 -- 172.21.15.143:0/2635989922 >> v1:172.21.15.143:6789/0 conn(0x7fd24c0d3510 legacy=0x7fd24c0b9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:07.936 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.929+0000 7fd249ffb700 1 --2- 172.21.15.143:0/2635989922 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd234003a00 0x7fd234003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:07.936 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.929+0000 7fd249ffb700 1 --2- 172.21.15.143:0/2635989922 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd234004850 0x7fd234004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:07.936 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.929+0000 7fd249ffb700 1 --2- 172.21.15.143:0/2635989922 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd234005210 0x7fd23400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:07.936 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.929+0000 7fd249ffb700 1 -- 172.21.15.143:0/2635989922 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd24c0d30f0 con 0x7fd234003a00 2023-11-08T19:31:07.937 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.929+0000 7fd249ffb700 1 -- 172.21.15.143:0/2635989922 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd23400c1b0 con 0x7fd234005210 2023-11-08T19:31:07.937 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.929+0000 7fd249ffb700 1 -- 172.21.15.143:0/2635989922 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd23400c370 con 0x7fd234004850 2023-11-08T19:31:07.937 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.929+0000 7fd24affd700 1 --2- 172.21.15.143:0/2635989922 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd234003a00 0x7fd234003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:07.937 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.929+0000 7fd24a7fc700 1 --2- 172.21.15.143:0/2635989922 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd234004850 0x7fd234004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:07.937 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.929+0000 7fd24b7fe700 1 --2- 172.21.15.143:0/2635989922 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd234005210 0x7fd23400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:07.937 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.929+0000 7fd24affd700 1 -- 172.21.15.143:0/2635989922 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd234004850 msgr2=0x7fd234004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:07.938 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.929+0000 7fd24affd700 1 --2- 172.21.15.143:0/2635989922 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd234004850 0x7fd234004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:07.938 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.929+0000 7fd24affd700 1 -- 172.21.15.143:0/2635989922 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd234005210 msgr2=0x7fd23400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:07.938 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.929+0000 7fd24affd700 1 --2- 172.21.15.143:0/2635989922 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd234005210 0x7fd23400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:07.938 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.929+0000 7fd24affd700 1 -- 172.21.15.143:0/2635989922 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2340035b0 con 0x7fd234003a00 2023-11-08T19:31:07.939 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.929+0000 7fd24b7fe700 1 --2- 172.21.15.143:0/2635989922 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd234005210 0x7fd23400ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:31:07.939 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.929+0000 7fd24b7fe700 1 -- 172.21.15.143:0/2635989922 reap_dead start 2023-11-08T19:31:07.939 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.929+0000 7fd24affd700 1 --2- 172.21.15.143:0/2635989922 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd234003a00 0x7fd234003e20 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7fd240009b60 tx=0x7fd234005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fceddaae53ac5785 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:07.939 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.929+0000 7fd249ffb700 1 -- 172.21.15.143:0/2635989922 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd24000e6c0 con 0x7fd234003a00 2023-11-08T19:31:07.939 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.929+0000 7fd249ffb700 1 -- 172.21.15.143:0/2635989922 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd24000e820 con 0x7fd234003a00 2023-11-08T19:31:07.940 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.929+0000 7fd249ffb700 1 -- 172.21.15.143:0/2635989922 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd24000eb90 con 0x7fd234003a00 2023-11-08T19:31:07.940 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.930+0000 7fd2517db700 1 -- 172.21.15.143:0/2635989922 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd234003a00 msgr2=0x7fd234003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:07.940 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.930+0000 7fd2517db700 1 --2- 172.21.15.143:0/2635989922 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd234003a00 0x7fd234003e20 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7fd240009b60 tx=0x7fd234005880 comp rx=0 tx=0).stop 2023-11-08T19:31:07.940 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.930+0000 7fd2517db700 1 -- 172.21.15.143:0/2635989922 shutdown_connections 2023-11-08T19:31:07.940 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.930+0000 7fd2517db700 1 --2- 172.21.15.143:0/2635989922 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd234003a00 0x7fd234003e20 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:07.941 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.930+0000 7fd2517db700 1 -- 172.21.15.143:0/2635989922 >> 172.21.15.143:0/2635989922 conn(0x7fd24c0d05a0 msgr2=0x7fd24c0d09b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:07.941 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.930+0000 7fd2517db700 1 -- 172.21.15.143:0/2635989922 shutdown_connections 2023-11-08T19:31:07.941 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.930+0000 7fd2517db700 1 -- 172.21.15.143:0/2635989922 wait complete. 2023-11-08T19:31:07.944 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.931+0000 7fd2517db700 1 Processor -- start 2023-11-08T19:31:07.944 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.932+0000 7fd2517db700 1 -- start start 2023-11-08T19:31:07.944 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.932+0000 7fd2517db700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd234003a00 0x7fd24c15e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:07.944 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.932+0000 7fd2517db700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd24c15f640 0x7fd24c152e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:07.945 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.932+0000 7fd2517db700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd24c0cc020 0x7fd24c0b73e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:07.945 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.932+0000 7fd2517db700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd24c15ef20 con 0x7fd24c15f640 2023-11-08T19:31:07.945 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.932+0000 7fd2517db700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd24c0caf80 con 0x7fd234003a00 2023-11-08T19:31:07.945 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.932+0000 7fd2517db700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd24c0cb260 con 0x7fd24c0cc020 2023-11-08T19:31:07.945 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.933+0000 7fd24a7fc700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd24c15f640 0x7fd24c152e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:07.946 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.933+0000 7fd24a7fc700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd24c15f640 0x7fd24c152e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56864/0 (socket says 172.21.15.143:56864) 2023-11-08T19:31:07.946 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.933+0000 7fd24a7fc700 1 -- 172.21.15.143:0/1727564399 learned_addr learned my addr 172.21.15.143:0/1727564399 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:31:07.946 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.933+0000 7fd24b7fe700 1 --2- 172.21.15.143:0/1727564399 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd24c0cc020 0x7fd24c0b73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:07.946 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.933+0000 7fd24affd700 1 --2- 172.21.15.143:0/1727564399 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd234003a00 0x7fd24c15e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:07.946 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.933+0000 7fd24a7fc700 1 -- 172.21.15.143:0/1727564399 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd24c0cc020 msgr2=0x7fd24c0b73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:07.947 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.933+0000 7fd24a7fc700 1 --2- 172.21.15.143:0/1727564399 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd24c0cc020 0x7fd24c0b73e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:07.947 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.933+0000 7fd24a7fc700 1 -- 172.21.15.143:0/1727564399 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd234003a00 msgr2=0x7fd24c15e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:07.947 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.933+0000 7fd24a7fc700 1 --2- 172.21.15.143:0/1727564399 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd234003a00 0x7fd24c15e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:07.947 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.933+0000 7fd24a7fc700 1 -- 172.21.15.143:0/1727564399 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd24c0cb540 con 0x7fd24c15f640 2023-11-08T19:31:07.947 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.933+0000 7fd24affd700 1 --2- 172.21.15.143:0/1727564399 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd234003a00 0x7fd24c15e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:31:07.948 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.933+0000 7fd24b7fe700 1 --2- 172.21.15.143:0/1727564399 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd24c0cc020 0x7fd24c0b73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:31:07.948 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.933+0000 7fd24a7fc700 1 --2- 172.21.15.143:0/1727564399 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd24c15f640 0x7fd24c152e80 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7fd23c007290 tx=0x7fd23c007650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:07.948 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.933+0000 7fd2337fe700 1 -- 172.21.15.143:0/1727564399 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd23c007830 con 0x7fd24c15f640 2023-11-08T19:31:07.948 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.933+0000 7fd2337fe700 1 -- 172.21.15.143:0/1727564399 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd23c00f8d0 con 0x7fd24c15f640 2023-11-08T19:31:07.948 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.933+0000 7fd2517db700 1 -- 172.21.15.143:0/1727564399 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd24c0cb820 con 0x7fd24c15f640 2023-11-08T19:31:07.949 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.933+0000 7fd2337fe700 1 -- 172.21.15.143:0/1727564399 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd23c00fc60 con 0x7fd24c15f640 2023-11-08T19:31:07.949 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.933+0000 7fd2517db700 1 -- 172.21.15.143:0/1727564399 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd24c1533c0 con 0x7fd24c15f640 2023-11-08T19:31:07.949 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.934+0000 7fd2517db700 1 -- 172.21.15.143:0/1727564399 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd24c0cbd40 con 0x7fd24c15f640 2023-11-08T19:31:07.949 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.935+0000 7fd2337fe700 1 -- 172.21.15.143:0/1727564399 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd23c01d460 con 0x7fd24c15f640 2023-11-08T19:31:07.949 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.935+0000 7fd2337fe700 1 --2- 172.21.15.143:0/1727564399 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd23806af00 0x7fd23806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:07.950 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.935+0000 7fd2337fe700 1 -- 172.21.15.143:0/1727564399 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 7360+0+0 (secure 0 0 0) 0x7fd23c089d40 con 0x7fd24c15f640 2023-11-08T19:31:07.950 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.935+0000 7fd24affd700 1 --2- 172.21.15.143:0/1727564399 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd23806af00 0x7fd23806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:07.950 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.936+0000 7fd24affd700 1 --2- 172.21.15.143:0/1727564399 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd23806af00 0x7fd23806d3c0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fd2400090b0 tx=0x7fd240008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:07.950 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:07.940+0000 7fd2337fe700 1 -- 172.21.15.143:0/1727564399 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd24c0cbd40 con 0x7fd24c15f640 2023-11-08T19:31:08.131 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.129+0000 7fd2517db700 1 -- 172.21.15.143:0/1727564399 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "test_metadata_pool", "var": "pg_num"} v 0) v1 -- 0x7fd24c0cbd40 con 0x7fd24c15f640 2023-11-08T19:31:08.131 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.129+0000 7fd2337fe700 1 -- 172.21.15.143:0/1727564399 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "test_metadata_pool", "var": "pg_num"}]=0 v104) v1 ==== 107+0+11 (secure 0 0 0) 0x7fd24c0cbd40 con 0x7fd24c15f640 2023-11-08T19:31:08.132 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2023-11-08T19:31:08.133 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.131+0000 7fd2517db700 1 -- 172.21.15.143:0/1727564399 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd23806af00 msgr2=0x7fd23806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:08.134 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.131+0000 7fd2517db700 1 --2- 172.21.15.143:0/1727564399 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd23806af00 0x7fd23806d3c0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fd2400090b0 tx=0x7fd240008030 comp rx=0 tx=0).stop 2023-11-08T19:31:08.134 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.131+0000 7fd2517db700 1 -- 172.21.15.143:0/1727564399 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd24c15f640 msgr2=0x7fd24c152e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:08.134 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.131+0000 7fd2517db700 1 --2- 172.21.15.143:0/1727564399 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd24c15f640 0x7fd24c152e80 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7fd23c007290 tx=0x7fd23c007650 comp rx=0 tx=0).stop 2023-11-08T19:31:08.135 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.131+0000 7fd2517db700 1 -- 172.21.15.143:0/1727564399 shutdown_connections 2023-11-08T19:31:08.135 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.131+0000 7fd2517db700 1 --2- 172.21.15.143:0/1727564399 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd23806af00 0x7fd23806d3c0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:08.135 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.131+0000 7fd2517db700 1 --2- 172.21.15.143:0/1727564399 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd24c0cc020 0x7fd24c0b73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:08.136 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.131+0000 7fd2517db700 1 --2- 172.21.15.143:0/1727564399 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd234003a00 0x7fd24c15e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:08.136 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.131+0000 7fd2517db700 1 --2- 172.21.15.143:0/1727564399 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd24c15f640 0x7fd24c152e80 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:08.136 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.131+0000 7fd2517db700 1 -- 172.21.15.143:0/1727564399 >> 172.21.15.143:0/1727564399 conn(0x7fd24c0d05a0 msgr2=0x7fd24c0b3e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:08.136 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.131+0000 7fd2517db700 1 -- 172.21.15.143:0/1727564399 shutdown_connections 2023-11-08T19:31:08.137 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.131+0000 7fd2517db700 1 -- 172.21.15.143:0/1727564399 wait complete. 2023-11-08T19:31:08.145 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-08T19:31:08.270 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.269+0000 7f9862665700 1 Processor -- start 2023-11-08T19:31:08.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.269+0000 7f9862665700 1 -- start start 2023-11-08T19:31:08.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.270+0000 7f9862665700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f985c072110 0x7f985c0b4990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:08.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.270+0000 7f9862665700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f985c0b4ed0 0x7f985c0b5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:08.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.270+0000 7f9862665700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f985c071650 con 0x7f985c15a800 2023-11-08T19:31:08.274 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.270+0000 7f9862665700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f985c0b2f80 con 0x7f985c072110 2023-11-08T19:31:08.274 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.270+0000 7f9862665700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f985c0b3260 con 0x7f985c0b4ed0 2023-11-08T19:31:08.274 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.270+0000 7f985bfff700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f985c15a800 0x7f985c0b2770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:53024/0 (socket says 172.21.15.143:53024) 2023-11-08T19:31:08.274 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.270+0000 7f985bfff700 1 -- 172.21.15.143:0/2031100131 learned_addr learned my addr 172.21.15.143:0/2031100131 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:31:08.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.270+0000 7f985b7fe700 1 --2- 172.21.15.143:0/2031100131 >> v2:172.21.15.174:3300/0 conn(0x7f985c072110 0x7f985c0b4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:08.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.270+0000 7f9860c02700 1 --2- 172.21.15.143:0/2031100131 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f985c0b4ed0 0x7f985c0b5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:08.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.270+0000 7f985affd700 1 -- 172.21.15.143:0/2031100131 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2623048702 0 0) 0x7f985c071650 con 0x7f985c15a800 2023-11-08T19:31:08.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.270+0000 7f985affd700 1 -- 172.21.15.143:0/2031100131 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f98440035b0 con 0x7f985c15a800 2023-11-08T19:31:08.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.270+0000 7f9860c02700 1 -- 172.21.15.143:0/2031100131 >> v2:172.21.15.174:3300/0 conn(0x7f985c072110 msgr2=0x7f985c0b4990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:08.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.270+0000 7f9860c02700 1 --2- 172.21.15.143:0/2031100131 >> v2:172.21.15.174:3300/0 conn(0x7f985c072110 0x7f985c0b4990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:08.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.270+0000 7f9860c02700 1 -- 172.21.15.143:0/2031100131 >> v1:172.21.15.143:6789/0 conn(0x7f985c15a800 legacy=0x7f985c0b2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:08.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.270+0000 7f9860c02700 1 -- 172.21.15.143:0/2031100131 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f985c071910 con 0x7f985c0b4ed0 2023-11-08T19:31:08.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.270+0000 7f985b7fe700 1 --2- 172.21.15.143:0/2031100131 >> v2:172.21.15.174:3300/0 conn(0x7f985c072110 0x7f985c0b4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T19:31:08.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.271+0000 7f9860c02700 1 --2- 172.21.15.143:0/2031100131 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f985c0b4ed0 0x7f985c0b5340 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9854009d90 tx=0x7f985400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12226644d8487b8 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:08.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.271+0000 7f985affd700 1 -- 172.21.15.143:0/2031100131 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f985400e070 con 0x7f985c0b4ed0 2023-11-08T19:31:08.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.271+0000 7f985affd700 1 -- 172.21.15.143:0/2031100131 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f985400bb20 con 0x7f985c0b4ed0 2023-11-08T19:31:08.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.271+0000 7f985affd700 1 -- 172.21.15.143:0/2031100131 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9854012430 con 0x7f985c0b4ed0 2023-11-08T19:31:08.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.271+0000 7f9862665700 1 -- 172.21.15.143:0/2031100131 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f985c0b4ed0 msgr2=0x7f985c0b5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:08.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.271+0000 7f9862665700 1 --2- 172.21.15.143:0/2031100131 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f985c0b4ed0 0x7f985c0b5340 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9854009d90 tx=0x7f985400b330 comp rx=0 tx=0).stop 2023-11-08T19:31:08.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.271+0000 7f9862665700 1 -- 172.21.15.143:0/2031100131 shutdown_connections 2023-11-08T19:31:08.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.271+0000 7f9862665700 1 --2- 172.21.15.143:0/2031100131 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f985c0b4ed0 0x7f985c0b5340 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:08.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.271+0000 7f9862665700 1 --2- 172.21.15.143:0/2031100131 >> v2:172.21.15.174:3300/0 conn(0x7f985c072110 0x7f985c0b4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:08.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.271+0000 7f9862665700 1 -- 172.21.15.143:0/2031100131 >> 172.21.15.143:0/2031100131 conn(0x7f985c06bcf0 msgr2=0x7f985c06c100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:08.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.271+0000 7f9862665700 1 -- 172.21.15.143:0/2031100131 shutdown_connections 2023-11-08T19:31:08.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.272+0000 7f9862665700 1 -- 172.21.15.143:0/2031100131 wait complete. 2023-11-08T19:31:08.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.273+0000 7f9862665700 1 Processor -- start 2023-11-08T19:31:08.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.273+0000 7f9862665700 1 -- start start 2023-11-08T19:31:08.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.274+0000 7f9862665700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f985c072110 0x7f985c1f0720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:08.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.274+0000 7f9862665700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f985c0b4ed0 0x7f985c0c5f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:08.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.274+0000 7f9862665700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f985c15a800 0x7f985c0ce870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:08.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.274+0000 7f9862665700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f985c1f1fe0 con 0x7f985c072110 2023-11-08T19:31:08.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.274+0000 7f9862665700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f985c1f0c60 con 0x7f985c15a800 2023-11-08T19:31:08.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.274+0000 7f9862665700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f985c1f2230 con 0x7f985c0b4ed0 2023-11-08T19:31:08.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.274+0000 7f985bfff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f985c072110 0x7f985c1f0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:08.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.274+0000 7f985bfff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f985c072110 0x7f985c1f0720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56876/0 (socket says 172.21.15.143:56876) 2023-11-08T19:31:08.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.274+0000 7f985bfff700 1 -- 172.21.15.143:0/3495702293 learned_addr learned my addr 172.21.15.143:0/3495702293 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:31:08.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.274+0000 7f985b7fe700 1 --2- 172.21.15.143:0/3495702293 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f985c0b4ed0 0x7f985c0c5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:08.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.274+0000 7f9860c02700 1 --2- 172.21.15.143:0/3495702293 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f985c15a800 0x7f985c0ce870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:08.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.274+0000 7f985bfff700 1 -- 172.21.15.143:0/3495702293 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f985c0b4ed0 msgr2=0x7f985c0c5f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:08.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.274+0000 7f985bfff700 1 --2- 172.21.15.143:0/3495702293 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f985c0b4ed0 0x7f985c0c5f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:08.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.274+0000 7f985bfff700 1 -- 172.21.15.143:0/3495702293 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f985c15a800 msgr2=0x7f985c0ce870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:08.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.274+0000 7f985bfff700 1 --2- 172.21.15.143:0/3495702293 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f985c15a800 0x7f985c0ce870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:08.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.274+0000 7f985bfff700 1 -- 172.21.15.143:0/3495702293 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f984c0094a0 con 0x7f985c072110 2023-11-08T19:31:08.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.274+0000 7f9860c02700 1 --2- 172.21.15.143:0/3495702293 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f985c15a800 0x7f985c0ce870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:31:08.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.274+0000 7f985b7fe700 1 --2- 172.21.15.143:0/3495702293 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f985c0b4ed0 0x7f985c0c5f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:31:08.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.274+0000 7f985bfff700 1 --2- 172.21.15.143:0/3495702293 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f985c072110 0x7f985c1f0720 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f9850001eb0 tx=0x7f985000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:08.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.274+0000 7f9858ff9700 1 -- 172.21.15.143:0/3495702293 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9850005d70 con 0x7f985c072110 2023-11-08T19:31:08.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.274+0000 7f9858ff9700 1 -- 172.21.15.143:0/3495702293 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98500070d0 con 0x7f985c072110 2023-11-08T19:31:08.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.274+0000 7f9862665700 1 -- 172.21.15.143:0/3495702293 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9854009a70 con 0x7f985c072110 2023-11-08T19:31:08.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.274+0000 7f9858ff9700 1 -- 172.21.15.143:0/3495702293 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9850007460 con 0x7f985c072110 2023-11-08T19:31:08.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.274+0000 7f9862665700 1 -- 172.21.15.143:0/3495702293 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f985c1f27f0 con 0x7f985c072110 2023-11-08T19:31:08.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.275+0000 7f9862665700 1 -- 172.21.15.143:0/3495702293 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f985c04f730 con 0x7f985c072110 2023-11-08T19:31:08.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.276+0000 7f9858ff9700 1 -- 172.21.15.143:0/3495702293 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f98500075c0 con 0x7f985c072110 2023-11-08T19:31:08.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.276+0000 7f9858ff9700 1 --2- 172.21.15.143:0/3495702293 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f984806af00 0x7f984806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:08.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.277+0000 7f9858ff9700 1 -- 172.21.15.143:0/3495702293 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 7360+0+0 (secure 0 0 0) 0x7f9850089470 con 0x7f985c072110 2023-11-08T19:31:08.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.277+0000 7f985b7fe700 1 --2- 172.21.15.143:0/3495702293 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f984806af00 0x7f984806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:08.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.277+0000 7f985b7fe700 1 --2- 172.21.15.143:0/3495702293 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f984806af00 0x7f984806d3c0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f985c0ca650 tx=0x7f984c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:08.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.281+0000 7f9858ff9700 1 -- 172.21.15.143:0/3495702293 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9850054040 con 0x7f985c072110 2023-11-08T19:31:08.468 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.467+0000 7f9862665700 1 -- 172.21.15.143:0/3495702293 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f985c074820 con 0x7f985c072110 2023-11-08T19:31:08.469 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.467+0000 7f9858ff9700 1 -- 172.21.15.143:0/3495702293 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v104) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9850057660 con 0x7f985c072110 2023-11-08T19:31:08.469 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2023-11-08T19:31:08.471 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.469+0000 7f9862665700 1 -- 172.21.15.143:0/3495702293 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f984806af00 msgr2=0x7f984806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:08.471 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.469+0000 7f9862665700 1 --2- 172.21.15.143:0/3495702293 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f984806af00 0x7f984806d3c0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f985c0ca650 tx=0x7f984c008040 comp rx=0 tx=0).stop 2023-11-08T19:31:08.471 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.469+0000 7f9862665700 1 -- 172.21.15.143:0/3495702293 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f985c072110 msgr2=0x7f985c1f0720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:08.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.469+0000 7f9862665700 1 --2- 172.21.15.143:0/3495702293 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f985c072110 0x7f985c1f0720 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f9850001eb0 tx=0x7f985000c860 comp rx=0 tx=0).stop 2023-11-08T19:31:08.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.469+0000 7f9862665700 1 -- 172.21.15.143:0/3495702293 shutdown_connections 2023-11-08T19:31:08.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.469+0000 7f9862665700 1 --2- 172.21.15.143:0/3495702293 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f984806af00 0x7f984806d3c0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:08.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.469+0000 7f9862665700 1 --2- 172.21.15.143:0/3495702293 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f985c15a800 0x7f985c0ce870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:08.473 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.469+0000 7f9862665700 1 --2- 172.21.15.143:0/3495702293 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f985c072110 0x7f985c1f0720 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:08.473 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.469+0000 7f9862665700 1 --2- 172.21.15.143:0/3495702293 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f985c0b4ed0 0x7f985c0c5f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:08.473 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.469+0000 7f9862665700 1 -- 172.21.15.143:0/3495702293 >> 172.21.15.143:0/3495702293 conn(0x7f985c06bcf0 msgr2=0x7f985c0cf580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:08.473 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.469+0000 7f9862665700 1 -- 172.21.15.143:0/3495702293 shutdown_connections 2023-11-08T19:31:08.473 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.469+0000 7f9862665700 1 -- 172.21.15.143:0/3495702293 wait complete. 2023-11-08T19:31:08.483 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-08T19:31:08.606 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.604+0000 7f3e97e97700 1 Processor -- start 2023-11-08T19:31:08.606 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.605+0000 7f3e97e97700 1 -- start start 2023-11-08T19:31:08.608 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.605+0000 7f3e97e97700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3e900c6680 0x7f3e90166b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:08.608 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.605+0000 7f3e97e97700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e900d0440 con 0x7f3e9015be10 2023-11-08T19:31:08.608 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.605+0000 7f3e97e97700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e900ccef0 con 0x7f3e9015c1f0 2023-11-08T19:31:08.609 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.605+0000 7f3e97e97700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e900cd1d0 con 0x7f3e900c6680 2023-11-08T19:31:08.609 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.605+0000 7f3e95c33700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f3e9015be10 0x7f3e900cf620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:53038/0 (socket says 172.21.15.143:53038) 2023-11-08T19:31:08.609 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.605+0000 7f3e95c33700 1 -- 172.21.15.143:0/2236523664 learned_addr learned my addr 172.21.15.143:0/2236523664 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:31:08.609 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.606+0000 7f3e96434700 1 --2- 172.21.15.143:0/2236523664 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3e900c6680 0x7f3e90166b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:08.609 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.606+0000 7f3e94c31700 1 -- 172.21.15.143:0/2236523664 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3843000291 0 0) 0x7f3e900d0440 con 0x7f3e9015be10 2023-11-08T19:31:08.610 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.606+0000 7f3e94c31700 1 -- 172.21.15.143:0/2236523664 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e7c0035b0 con 0x7f3e9015be10 2023-11-08T19:31:08.610 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.606+0000 7f3e94c31700 1 -- 172.21.15.143:0/2236523664 <== mon.1 v1:172.21.15.174:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2059761176 0 0) 0x7f3e900ccef0 con 0x7f3e9015c1f0 2023-11-08T19:31:08.610 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.606+0000 7f3e94c31700 1 -- 172.21.15.143:0/2236523664 --> v1:172.21.15.174:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e900d0440 con 0x7f3e9015c1f0 2023-11-08T19:31:08.610 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.606+0000 7f3e96434700 1 -- 172.21.15.143:0/2236523664 >> v1:172.21.15.174:6789/0 conn(0x7f3e9015c1f0 legacy=0x7f3e900cfd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:08.611 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.606+0000 7f3e96434700 1 -- 172.21.15.143:0/2236523664 >> v1:172.21.15.143:6789/0 conn(0x7f3e9015be10 legacy=0x7f3e900cf620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:08.611 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.606+0000 7f3e96434700 1 -- 172.21.15.143:0/2236523664 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e900cd4b0 con 0x7f3e900c6680 2023-11-08T19:31:08.611 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.606+0000 7f3e96434700 1 --2- 172.21.15.143:0/2236523664 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3e900c6680 0x7f3e90166b50 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3e88009dc0 tx=0x7f3e8800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8c4adbaadc405d6 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:08.611 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.606+0000 7f3e94c31700 1 -- 172.21.15.143:0/2236523664 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e8800e070 con 0x7f3e900c6680 2023-11-08T19:31:08.611 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.606+0000 7f3e94c31700 1 -- 172.21.15.143:0/2236523664 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e8800bb20 con 0x7f3e900c6680 2023-11-08T19:31:08.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.606+0000 7f3e94c31700 1 -- 172.21.15.143:0/2236523664 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e88012430 con 0x7f3e900c6680 2023-11-08T19:31:08.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.607+0000 7f3e97e97700 1 -- 172.21.15.143:0/2236523664 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3e900c6680 msgr2=0x7f3e90166b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:08.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.607+0000 7f3e97e97700 1 --2- 172.21.15.143:0/2236523664 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3e900c6680 0x7f3e90166b50 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3e88009dc0 tx=0x7f3e8800b330 comp rx=0 tx=0).stop 2023-11-08T19:31:08.615 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.607+0000 7f3e97e97700 1 -- 172.21.15.143:0/2236523664 shutdown_connections 2023-11-08T19:31:08.615 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.607+0000 7f3e97e97700 1 --2- 172.21.15.143:0/2236523664 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3e900c6680 0x7f3e90166b50 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:08.615 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.607+0000 7f3e97e97700 1 -- 172.21.15.143:0/2236523664 >> 172.21.15.143:0/2236523664 conn(0x7f3e900d18d0 msgr2=0x7f3e900d1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:08.616 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.607+0000 7f3e97e97700 1 -- 172.21.15.143:0/2236523664 shutdown_connections 2023-11-08T19:31:08.616 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.607+0000 7f3e97e97700 1 -- 172.21.15.143:0/2236523664 wait complete. 2023-11-08T19:31:08.616 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.609+0000 7f3e97e97700 1 Processor -- start 2023-11-08T19:31:08.616 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.609+0000 7f3e97e97700 1 -- start start 2023-11-08T19:31:08.616 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.609+0000 7f3e97e97700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3e900c6680 0x7f3e9015c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:08.617 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.609+0000 7f3e97e97700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e9015be10 0x7f3e9015cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:08.617 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.609+0000 7f3e97e97700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3e9015c1f0 0x7f3e9015e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:08.617 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.609+0000 7f3e97e97700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e900c94d0 con 0x7f3e9015be10 2023-11-08T19:31:08.617 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.609+0000 7f3e97e97700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e900c96e0 con 0x7f3e900c6680 2023-11-08T19:31:08.617 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.609+0000 7f3e97e97700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e900c99c0 con 0x7f3e9015c1f0 2023-11-08T19:31:08.618 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.610+0000 7f3e95432700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e9015be10 0x7f3e9015cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:08.618 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.610+0000 7f3e95432700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e9015be10 0x7f3e9015cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56892/0 (socket says 172.21.15.143:56892) 2023-11-08T19:31:08.618 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.610+0000 7f3e95432700 1 -- 172.21.15.143:0/2528625528 learned_addr learned my addr 172.21.15.143:0/2528625528 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:31:08.618 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.610+0000 7f3e96434700 1 --2- 172.21.15.143:0/2528625528 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3e9015c1f0 0x7f3e9015e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:08.618 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.610+0000 7f3e95c33700 1 --2- 172.21.15.143:0/2528625528 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3e900c6680 0x7f3e9015c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:08.619 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.610+0000 7f3e95432700 1 -- 172.21.15.143:0/2528625528 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3e9015c1f0 msgr2=0x7f3e9015e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:08.619 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.610+0000 7f3e95432700 1 --2- 172.21.15.143:0/2528625528 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3e9015c1f0 0x7f3e9015e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:08.619 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.610+0000 7f3e95432700 1 -- 172.21.15.143:0/2528625528 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3e900c6680 msgr2=0x7f3e9015c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:08.619 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.610+0000 7f3e95432700 1 --2- 172.21.15.143:0/2528625528 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3e900c6680 0x7f3e9015c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:08.619 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.610+0000 7f3e95432700 1 -- 172.21.15.143:0/2528625528 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e88009a70 con 0x7f3e9015be10 2023-11-08T19:31:08.620 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.610+0000 7f3e95432700 1 --2- 172.21.15.143:0/2528625528 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e9015be10 0x7f3e9015cb10 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c00c520 tx=0x7f3e8c00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:08.620 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.610+0000 7f3e867fc700 1 -- 172.21.15.143:0/2528625528 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e8c0052d0 con 0x7f3e9015be10 2023-11-08T19:31:08.620 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.610+0000 7f3e867fc700 1 -- 172.21.15.143:0/2528625528 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e8c005430 con 0x7f3e9015be10 2023-11-08T19:31:08.620 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.610+0000 7f3e867fc700 1 -- 172.21.15.143:0/2528625528 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e8c0038c0 con 0x7f3e9015be10 2023-11-08T19:31:08.620 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.610+0000 7f3e97e97700 1 -- 172.21.15.143:0/2528625528 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e900c9ca0 con 0x7f3e9015be10 2023-11-08T19:31:08.621 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.610+0000 7f3e97e97700 1 -- 172.21.15.143:0/2528625528 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e900ca260 con 0x7f3e9015be10 2023-11-08T19:31:08.621 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.611+0000 7f3e97e97700 1 -- 172.21.15.143:0/2528625528 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e9004f730 con 0x7f3e9015be10 2023-11-08T19:31:08.621 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.612+0000 7f3e867fc700 1 -- 172.21.15.143:0/2528625528 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3e8c003a20 con 0x7f3e9015be10 2023-11-08T19:31:08.621 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.612+0000 7f3e867fc700 1 --2- 172.21.15.143:0/2528625528 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f3e7406af00 0x7f3e7406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:08.621 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.612+0000 7f3e867fc700 1 -- 172.21.15.143:0/2528625528 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 7360+0+0 (secure 0 0 0) 0x7f3e8c089090 con 0x7f3e9015be10 2023-11-08T19:31:08.622 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.612+0000 7f3e95c33700 1 --2- 172.21.15.143:0/2528625528 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f3e7406af00 0x7f3e7406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:08.622 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.613+0000 7f3e95c33700 1 --2- 172.21.15.143:0/2528625528 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f3e7406af00 0x7f3e7406d3c0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f3e80001000 tx=0x7f3e80007c70 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:08.622 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.616+0000 7f3e867fc700 1 -- 172.21.15.143:0/2528625528 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3e8c053ad0 con 0x7f3e9015be10 2023-11-08T19:31:08.808 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.807+0000 7f3e97e97700 1 -- 172.21.15.143:0/2528625528 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3e900cee80 con 0x7f3e9015be10 2023-11-08T19:31:08.809 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.807+0000 7f3e867fc700 1 -- 172.21.15.143:0/2528625528 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v104) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3e8c017020 con 0x7f3e9015be10 2023-11-08T19:31:08.809 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 64 2023-11-08T19:31:08.811 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.809+0000 7f3e97e97700 1 -- 172.21.15.143:0/2528625528 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f3e7406af00 msgr2=0x7f3e7406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:08.811 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.810+0000 7f3e97e97700 1 --2- 172.21.15.143:0/2528625528 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f3e7406af00 0x7f3e7406d3c0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f3e80001000 tx=0x7f3e80007c70 comp rx=0 tx=0).stop 2023-11-08T19:31:08.812 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.810+0000 7f3e97e97700 1 -- 172.21.15.143:0/2528625528 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e9015be10 msgr2=0x7f3e9015cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:08.812 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.810+0000 7f3e97e97700 1 --2- 172.21.15.143:0/2528625528 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e9015be10 0x7f3e9015cb10 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c00c520 tx=0x7f3e8c00c830 comp rx=0 tx=0).stop 2023-11-08T19:31:08.812 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.810+0000 7f3e97e97700 1 -- 172.21.15.143:0/2528625528 shutdown_connections 2023-11-08T19:31:08.812 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.810+0000 7f3e97e97700 1 --2- 172.21.15.143:0/2528625528 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f3e7406af00 0x7f3e7406d3c0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:08.812 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.810+0000 7f3e97e97700 1 --2- 172.21.15.143:0/2528625528 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3e9015c1f0 0x7f3e9015e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:08.813 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.810+0000 7f3e97e97700 1 --2- 172.21.15.143:0/2528625528 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3e900c6680 0x7f3e9015c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:08.813 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.810+0000 7f3e97e97700 1 --2- 172.21.15.143:0/2528625528 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e9015be10 0x7f3e9015cb10 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:08.813 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.810+0000 7f3e97e97700 1 -- 172.21.15.143:0/2528625528 >> 172.21.15.143:0/2528625528 conn(0x7f3e900d18d0 msgr2=0x7f3e900c6b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:08.813 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.810+0000 7f3e97e97700 1 -- 172.21.15.143:0/2528625528 shutdown_connections 2023-11-08T19:31:08.813 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.810+0000 7f3e97e97700 1 -- 172.21.15.143:0/2528625528 wait complete. 2023-11-08T19:31:08.823 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:31:08.946 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.942+0000 7f02bece3700 1 Processor -- start 2023-11-08T19:31:08.947 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.942+0000 7f02bece3700 1 -- start start 2023-11-08T19:31:08.947 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.943+0000 7f02bece3700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f02b815de00 0x7f02b8162200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:08.947 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.943+0000 7f02bece3700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f02b806f2e0 0x7f02b806b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:08.947 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.943+0000 7f02bece3700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02b8162740 con 0x7f02b806abe0 2023-11-08T19:31:08.948 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.943+0000 7f02bece3700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02b806b5a0 con 0x7f02b806f2e0 2023-11-08T19:31:08.948 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.943+0000 7f02bece3700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02b80cb230 con 0x7f02b815de00 2023-11-08T19:31:08.948 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.943+0000 7f02bd280700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f02b806f2e0 0x7f02b806b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:08.948 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.943+0000 7f02bd280700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f02b806f2e0 0x7f02b806b060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56900/0 (socket says 172.21.15.143:56900) 2023-11-08T19:31:08.948 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.943+0000 7f02bd280700 1 -- 172.21.15.143:0/1018364158 learned_addr learned my addr 172.21.15.143:0/1018364158 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:31:08.949 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.943+0000 7f02bca7f700 1 --2- 172.21.15.143:0/1018364158 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f02b815de00 0x7f02b8162200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:08.949 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.944+0000 7f02bd280700 1 -- 172.21.15.143:0/1018364158 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f02b815de00 msgr2=0x7f02b8162200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:08.949 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.944+0000 7f02bd280700 1 --2- 172.21.15.143:0/1018364158 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f02b815de00 0x7f02b8162200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:08.949 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.944+0000 7f02bd280700 1 -- 172.21.15.143:0/1018364158 >> v1:172.21.15.174:6789/0 conn(0x7f02b806abe0 legacy=0x7f02b80cd210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:08.949 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.944+0000 7f02bd280700 1 -- 172.21.15.143:0/1018364158 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02b80cb510 con 0x7f02b806f2e0 2023-11-08T19:31:08.950 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.944+0000 7f02bd280700 1 --2- 172.21.15.143:0/1018364158 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f02b806f2e0 0x7f02b806b060 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f02b0009e50 tx=0x7f02b000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=756482305dae342e server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:08.950 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.944+0000 7f02b77fe700 1 -- 172.21.15.143:0/1018364158 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b000e070 con 0x7f02b806f2e0 2023-11-08T19:31:08.950 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.944+0000 7f02b77fe700 1 -- 172.21.15.143:0/1018364158 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02b000b9c0 con 0x7f02b806f2e0 2023-11-08T19:31:08.950 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.944+0000 7f02b77fe700 1 -- 172.21.15.143:0/1018364158 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b000bd70 con 0x7f02b806f2e0 2023-11-08T19:31:08.950 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.944+0000 7f02bece3700 1 -- 172.21.15.143:0/1018364158 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f02b806f2e0 msgr2=0x7f02b806b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:08.951 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.944+0000 7f02bece3700 1 --2- 172.21.15.143:0/1018364158 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f02b806f2e0 0x7f02b806b060 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f02b0009e50 tx=0x7f02b000b330 comp rx=0 tx=0).stop 2023-11-08T19:31:08.951 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.945+0000 7f02bece3700 1 -- 172.21.15.143:0/1018364158 shutdown_connections 2023-11-08T19:31:08.951 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.945+0000 7f02bece3700 1 --2- 172.21.15.143:0/1018364158 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f02b806f2e0 0x7f02b806b060 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:08.951 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.945+0000 7f02bece3700 1 --2- 172.21.15.143:0/1018364158 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f02b815de00 0x7f02b8162200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:08.951 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.945+0000 7f02bece3700 1 -- 172.21.15.143:0/1018364158 >> 172.21.15.143:0/1018364158 conn(0x7f02b80b23d0 msgr2=0x7f02b80b27b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:08.952 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.945+0000 7f02bece3700 1 -- 172.21.15.143:0/1018364158 shutdown_connections 2023-11-08T19:31:08.952 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.945+0000 7f02bece3700 1 -- 172.21.15.143:0/1018364158 wait complete. 2023-11-08T19:31:08.955 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.946+0000 7f02bece3700 1 Processor -- start 2023-11-08T19:31:08.955 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.946+0000 7f02bece3700 1 -- start start 2023-11-08T19:31:08.955 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.947+0000 7f02bece3700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f02b806abe0 0x7f02b815fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:08.955 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.947+0000 7f02bece3700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f02b806f2e0 0x7f02b814bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:08.955 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.947+0000 7f02bece3700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f02b815de00 0x7f02b80c8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:08.956 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.947+0000 7f02bece3700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02b814d340 con 0x7f02b806abe0 2023-11-08T19:31:08.956 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.947+0000 7f02bece3700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02b814c6c0 con 0x7f02b815de00 2023-11-08T19:31:08.956 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.947+0000 7f02bece3700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02b814c9a0 con 0x7f02b806f2e0 2023-11-08T19:31:08.956 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.947+0000 7f02bca7f700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f02b806abe0 0x7f02b815fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:08.956 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.947+0000 7f02bca7f700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f02b806abe0 0x7f02b815fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56902/0 (socket says 172.21.15.143:56902) 2023-11-08T19:31:08.957 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.947+0000 7f02bca7f700 1 -- 172.21.15.143:0/823310650 learned_addr learned my addr 172.21.15.143:0/823310650 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:31:08.957 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.947+0000 7f02b7fff700 1 --2- 172.21.15.143:0/823310650 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f02b806f2e0 0x7f02b814bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:08.957 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.947+0000 7f02bd280700 1 --2- 172.21.15.143:0/823310650 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f02b815de00 0x7f02b80c8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:08.957 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.947+0000 7f02bca7f700 1 -- 172.21.15.143:0/823310650 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f02b806f2e0 msgr2=0x7f02b814bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:08.957 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.947+0000 7f02bca7f700 1 --2- 172.21.15.143:0/823310650 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f02b806f2e0 0x7f02b814bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:08.958 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.947+0000 7f02bca7f700 1 -- 172.21.15.143:0/823310650 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f02b815de00 msgr2=0x7f02b80c8ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:08.958 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.947+0000 7f02bca7f700 1 --2- 172.21.15.143:0/823310650 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f02b815de00 0x7f02b80c8ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:08.958 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.947+0000 7f02bca7f700 1 -- 172.21.15.143:0/823310650 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02b0009b30 con 0x7f02b806abe0 2023-11-08T19:31:08.958 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.947+0000 7f02b7fff700 1 --2- 172.21.15.143:0/823310650 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f02b806f2e0 0x7f02b814bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:31:08.958 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.947+0000 7f02bca7f700 1 --2- 172.21.15.143:0/823310650 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f02b806abe0 0x7f02b815fcf0 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f02ac00d800 tx=0x7f02ac00db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:08.959 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.947+0000 7f02b57fa700 1 -- 172.21.15.143:0/823310650 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02ac0125f0 con 0x7f02b806abe0 2023-11-08T19:31:08.959 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.947+0000 7f02b57fa700 1 -- 172.21.15.143:0/823310650 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02ac012750 con 0x7f02b806abe0 2023-11-08T19:31:08.959 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.947+0000 7f02b57fa700 1 -- 172.21.15.143:0/823310650 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02ac012b00 con 0x7f02b806abe0 2023-11-08T19:31:08.959 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.947+0000 7f02bece3700 1 -- 172.21.15.143:0/823310650 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02b814cc80 con 0x7f02b806abe0 2023-11-08T19:31:08.959 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.948+0000 7f02bece3700 1 -- 172.21.15.143:0/823310650 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02b814c100 con 0x7f02b806abe0 2023-11-08T19:31:08.960 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.948+0000 7f02bece3700 1 -- 172.21.15.143:0/823310650 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02b804f730 con 0x7f02b806abe0 2023-11-08T19:31:08.960 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.949+0000 7f02b57fa700 1 -- 172.21.15.143:0/823310650 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f02ac012c60 con 0x7f02b806abe0 2023-11-08T19:31:08.960 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.949+0000 7f02b57fa700 1 --2- 172.21.15.143:0/823310650 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f029c06af00 0x7f029c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:08.960 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.949+0000 7f02b57fa700 1 -- 172.21.15.143:0/823310650 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 7360+0+0 (secure 0 0 0) 0x7f02ac089d00 con 0x7f02b806abe0 2023-11-08T19:31:08.961 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.949+0000 7f02b7fff700 1 --2- 172.21.15.143:0/823310650 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f029c06af00 0x7f029c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:08.961 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.950+0000 7f02b7fff700 1 --2- 172.21.15.143:0/823310650 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f029c06af00 0x7f029c06d3c0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f02a4000f60 tx=0x7f02a4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:08.961 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:08.954+0000 7f02b57fa700 1 -- 172.21.15.143:0/823310650 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f02ac0547d0 con 0x7f02b806abe0 2023-11-08T19:31:09.186 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.185+0000 7f02bece3700 1 -- 172.21.15.143:0/823310650 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f02b8072300 con 0x7f02b806abe0 2023-11-08T19:31:09.187 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.185+0000 7f02b57fa700 1 -- 172.21.15.143:0/823310650 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 100 v100) v1 ==== 95+0+4946 (secure 0 0 0) 0x7f02ac057df0 con 0x7f02b806abe0 2023-11-08T19:31:09.187 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:31:09.187 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":100,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5933,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/3918286710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6835","nonce":3918286710},{"type":"v1","addr":"172.21.15.174:6837","nonce":3918286710}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":6245,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6836/2689884850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2689884850},{"type":"v1","addr":"172.21.15.174:6836","nonce":2689884850}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":97}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:03.092245+0000","modified":"2023-11-08T19:30:56.788736+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":103,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6161,"mds_1":7743},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6161":{"gid":6161,"name":"a","rank":0,"incarnation":93,"state":"up:active","state_seq":14,"addr":"172.21.15.143:6835/2129643800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":2129643800},{"type":"v1","addr":"172.21.15.143:6835","nonce":2129643800}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7743":{"gid":7743,"name":"c","rank":1,"incarnation":96,"state":"up:active","state_seq":7,"addr":"172.21.15.143:6834/2489252684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":2489252684},{"type":"v1","addr":"172.21.15.143:6834","nonce":2489252684}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:31:09.189 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.187+0000 7f02bece3700 1 -- 172.21.15.143:0/823310650 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f029c06af00 msgr2=0x7f029c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:09.189 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.187+0000 7f02bece3700 1 --2- 172.21.15.143:0/823310650 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f029c06af00 0x7f029c06d3c0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f02a4000f60 tx=0x7f02a4009040 comp rx=0 tx=0).stop 2023-11-08T19:31:09.189 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.187+0000 7f02bece3700 1 -- 172.21.15.143:0/823310650 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f02b806abe0 msgr2=0x7f02b815fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:09.190 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.187+0000 7f02bece3700 1 --2- 172.21.15.143:0/823310650 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f02b806abe0 0x7f02b815fcf0 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f02ac00d800 tx=0x7f02ac00db10 comp rx=0 tx=0).stop 2023-11-08T19:31:09.190 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.187+0000 7f02bece3700 1 -- 172.21.15.143:0/823310650 shutdown_connections 2023-11-08T19:31:09.190 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.187+0000 7f02bece3700 1 --2- 172.21.15.143:0/823310650 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f029c06af00 0x7f029c06d3c0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:09.190 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.187+0000 7f02bece3700 1 --2- 172.21.15.143:0/823310650 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f02b815de00 0x7f02b80c8ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:09.190 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.187+0000 7f02bece3700 1 --2- 172.21.15.143:0/823310650 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f02b806abe0 0x7f02b815fcf0 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:09.191 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.187+0000 7f02bece3700 1 --2- 172.21.15.143:0/823310650 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f02b806f2e0 0x7f02b814bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:09.191 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.187+0000 7f02bece3700 1 -- 172.21.15.143:0/823310650 >> 172.21.15.143:0/823310650 conn(0x7f02b80b23d0 msgr2=0x7f02b815d0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:09.191 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.187+0000 7f02bece3700 1 -- 172.21.15.143:0/823310650 shutdown_connections 2023-11-08T19:31:09.191 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.187+0000 7f02bece3700 1 -- 172.21.15.143:0/823310650 wait complete. 2023-11-08T19:31:09.192 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 100 2023-11-08T19:31:09.201 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:31:09.322 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.320+0000 7f3133d4d700 1 Processor -- start 2023-11-08T19:31:09.322 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.320+0000 7f3133d4d700 1 -- start start 2023-11-08T19:31:09.324 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.321+0000 7f3133d4d700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f312c0b2770 0x7f312c0b2b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:09.325 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.321+0000 7f3133d4d700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f312c06de80 0x7f312c1616a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:09.325 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.321+0000 7f3133d4d700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f312c1625d0 con 0x7f312c0722d0 2023-11-08T19:31:09.325 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.321+0000 7f3133d4d700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f312c15e470 con 0x7f312c0b2770 2023-11-08T19:31:09.325 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.321+0000 7f3133d4d700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f312c071e40 con 0x7f312c06de80 2023-11-08T19:31:09.325 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.321+0000 7f3131ae9700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f312c0b2770 0x7f312c0b2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:09.326 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.322+0000 7f3131ae9700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f312c0b2770 0x7f312c0b2b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56910/0 (socket says 172.21.15.143:56910) 2023-11-08T19:31:09.326 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.322+0000 7f3131ae9700 1 -- 172.21.15.143:0/1911648979 learned_addr learned my addr 172.21.15.143:0/1911648979 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:31:09.326 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.322+0000 7f31322ea700 1 --2- 172.21.15.143:0/1911648979 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f312c06de80 0x7f312c1616a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:09.326 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.322+0000 7f3131ae9700 1 -- 172.21.15.143:0/1911648979 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f312c06de80 msgr2=0x7f312c1616a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:09.326 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.322+0000 7f3131ae9700 1 --2- 172.21.15.143:0/1911648979 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f312c06de80 0x7f312c1616a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:09.327 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.322+0000 7f3131ae9700 1 -- 172.21.15.143:0/1911648979 >> v1:172.21.15.174:6789/0 conn(0x7f312c0722d0 legacy=0x7f312c06d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:09.327 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.322+0000 7f3131ae9700 1 -- 172.21.15.143:0/1911648979 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f312c0b3090 con 0x7f312c0b2770 2023-11-08T19:31:09.327 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.322+0000 7f31322ea700 1 --2- 172.21.15.143:0/1911648979 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f312c06de80 0x7f312c1616a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:31:09.327 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.322+0000 7f3131ae9700 1 --2- 172.21.15.143:0/1911648979 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f312c0b2770 0x7f312c0b2b50 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f311c0098f0 tx=0x7f311c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bc4ef4f0d635a71 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:09.327 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.322+0000 7f3130ae7700 1 -- 172.21.15.143:0/1911648979 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f311c004230 con 0x7f312c0b2770 2023-11-08T19:31:09.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.322+0000 7f3130ae7700 1 -- 172.21.15.143:0/1911648979 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f311c004390 con 0x7f312c0b2770 2023-11-08T19:31:09.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.322+0000 7f3130ae7700 1 -- 172.21.15.143:0/1911648979 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f311c0036a0 con 0x7f312c0b2770 2023-11-08T19:31:09.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.323+0000 7f3133d4d700 1 -- 172.21.15.143:0/1911648979 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f312c0b2770 msgr2=0x7f312c0b2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:09.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.323+0000 7f3133d4d700 1 --2- 172.21.15.143:0/1911648979 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f312c0b2770 0x7f312c0b2b50 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f311c0098f0 tx=0x7f311c009c00 comp rx=0 tx=0).stop 2023-11-08T19:31:09.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.323+0000 7f3133d4d700 1 -- 172.21.15.143:0/1911648979 shutdown_connections 2023-11-08T19:31:09.329 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.323+0000 7f3133d4d700 1 --2- 172.21.15.143:0/1911648979 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f312c06de80 0x7f312c1616a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:09.329 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.323+0000 7f3133d4d700 1 --2- 172.21.15.143:0/1911648979 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f312c0b2770 0x7f312c0b2b50 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:09.329 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.323+0000 7f3133d4d700 1 -- 172.21.15.143:0/1911648979 >> 172.21.15.143:0/1911648979 conn(0x7f312c06bcf0 msgr2=0x7f312c06c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:09.329 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.323+0000 7f3133d4d700 1 -- 172.21.15.143:0/1911648979 shutdown_connections 2023-11-08T19:31:09.329 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.323+0000 7f3133d4d700 1 -- 172.21.15.143:0/1911648979 wait complete. 2023-11-08T19:31:09.332 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.324+0000 7f3133d4d700 1 Processor -- start 2023-11-08T19:31:09.332 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.324+0000 7f3133d4d700 1 -- start start 2023-11-08T19:31:09.333 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.325+0000 7f3133d4d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f312c06de80 0x7f312c153d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:09.333 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.325+0000 7f3133d4d700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f312c0722d0 0x7f312c0d07a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:09.333 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.325+0000 7f3133d4d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f312c0b2770 0x7f312c0d0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:09.333 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.325+0000 7f3133d4d700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f312c0b45a0 con 0x7f312c06de80 2023-11-08T19:31:09.333 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.325+0000 7f3133d4d700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f312c0b3920 con 0x7f312c0b2770 2023-11-08T19:31:09.334 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.325+0000 7f3133d4d700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f312c0b3c00 con 0x7f312c0722d0 2023-11-08T19:31:09.334 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.325+0000 7f3131ae9700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f312c06de80 0x7f312c153d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:09.334 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.325+0000 7f3131ae9700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f312c06de80 0x7f312c153d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56916/0 (socket says 172.21.15.143:56916) 2023-11-08T19:31:09.334 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.325+0000 7f3131ae9700 1 -- 172.21.15.143:0/3080104330 learned_addr learned my addr 172.21.15.143:0/3080104330 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:31:09.334 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.325+0000 7f31312e8700 1 --2- 172.21.15.143:0/3080104330 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f312c0722d0 0x7f312c0d07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:09.335 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.325+0000 7f31322ea700 1 --2- 172.21.15.143:0/3080104330 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f312c0b2770 0x7f312c0d0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:09.335 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.325+0000 7f3131ae9700 1 -- 172.21.15.143:0/3080104330 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f312c0722d0 msgr2=0x7f312c0d07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:09.335 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.325+0000 7f3131ae9700 1 --2- 172.21.15.143:0/3080104330 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f312c0722d0 0x7f312c0d07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:09.335 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.325+0000 7f3131ae9700 1 -- 172.21.15.143:0/3080104330 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f312c0b2770 msgr2=0x7f312c0d0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:09.335 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.325+0000 7f3131ae9700 1 --2- 172.21.15.143:0/3080104330 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f312c0b2770 0x7f312c0d0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:09.336 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.325+0000 7f3131ae9700 1 -- 172.21.15.143:0/3080104330 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f311c009550 con 0x7f312c06de80 2023-11-08T19:31:09.336 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.325+0000 7f31322ea700 1 --2- 172.21.15.143:0/3080104330 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f312c0b2770 0x7f312c0d0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:31:09.336 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.325+0000 7f31312e8700 1 --2- 172.21.15.143:0/3080104330 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f312c0722d0 0x7f312c0d07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:31:09.336 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.325+0000 7f3131ae9700 1 --2- 172.21.15.143:0/3080104330 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f312c06de80 0x7f312c153d80 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f311c009840 tx=0x7f311c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:09.336 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.325+0000 7f31227fc700 1 -- 172.21.15.143:0/3080104330 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f311c00b660 con 0x7f312c06de80 2023-11-08T19:31:09.337 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.325+0000 7f31227fc700 1 -- 172.21.15.143:0/3080104330 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f311c00b7c0 con 0x7f312c06de80 2023-11-08T19:31:09.337 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.325+0000 7f3133d4d700 1 -- 172.21.15.143:0/3080104330 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f312c0b3ee0 con 0x7f312c06de80 2023-11-08T19:31:09.337 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.325+0000 7f31227fc700 1 -- 172.21.15.143:0/3080104330 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f311c00bb70 con 0x7f312c06de80 2023-11-08T19:31:09.337 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.325+0000 7f3133d4d700 1 -- 172.21.15.143:0/3080104330 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f312c0b3560 con 0x7f312c06de80 2023-11-08T19:31:09.337 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.326+0000 7f3133d4d700 1 -- 172.21.15.143:0/3080104330 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f312c0b36f0 con 0x7f312c06de80 2023-11-08T19:31:09.337 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.327+0000 7f31227fc700 1 -- 172.21.15.143:0/3080104330 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f311c00bcd0 con 0x7f312c06de80 2023-11-08T19:31:09.338 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.327+0000 7f31227fc700 1 --2- 172.21.15.143:0/3080104330 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f311006af00 0x7f311006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:09.338 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.327+0000 7f31227fc700 1 -- 172.21.15.143:0/3080104330 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 7360+0+0 (secure 0 0 0) 0x7f311c08b4a0 con 0x7f312c06de80 2023-11-08T19:31:09.338 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.327+0000 7f31312e8700 1 --2- 172.21.15.143:0/3080104330 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f311006af00 0x7f311006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:09.338 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.328+0000 7f31312e8700 1 --2- 172.21.15.143:0/3080104330 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f311006af00 0x7f311006d3c0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f312c15d900 tx=0x7f3128009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:09.339 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.331+0000 7f31227fc700 1 -- 172.21.15.143:0/3080104330 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f311c055f70 con 0x7f312c06de80 2023-11-08T19:31:09.523 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.522+0000 7f3133d4d700 1 -- 172.21.15.143:0/3080104330 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f312c04f730 con 0x7f312c06de80 2023-11-08T19:31:09.524 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.523+0000 7f31227fc700 1 -- 172.21.15.143:0/3080104330 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v104) v1 ==== 74+0+22721 (secure 0 0 0) 0x7f311c059590 con 0x7f312c06de80 2023-11-08T19:31:09.525 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:31:09.525 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":104,"fsid":"968483af-77fc-4068-ad7c-5c2a1a4613d7","created":"2023-11-08T19:24:32.826484+0000","modified":"2023-11-08T19:30:55.774106+0000","last_up_change":"2023-11-08T19:24:43.573880+0000","last_in_change":"2023-11-08T19:24:35.314525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:44.551530+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"snap-test-pool","create_time":"2023-11-08T19:26:02.874384+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":36,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"test_data_pool","create_time":"2023-11-08T19:27:05.601949+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":66,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_metadata_pool","create_time":"2023-11-08T19:27:06.536369+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-11-08T19:28:01.533566+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-11-08T19:28:02.053445+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":9,"snap_epoch":100,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bae98ec1-d94e-403d-a220-a522a53ea6f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6809","nonce":1487830566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6811","nonce":1487830566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6815","nonce":1487830566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6813","nonce":1487830566}]},"public_addr":"172.21.15.143:6809/1487830566","cluster_addr":"172.21.15.143:6811/1487830566","heartbeat_back_addr":"172.21.15.143:6815/1487830566","heartbeat_front_addr":"172.21.15.143:6813/1487830566","state":["exists","up"]},{"osd":1,"uuid":"5cb6538a-1ccb-4322-8633-09f66af0b7e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6801","nonce":787341397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6803","nonce":787341397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6807","nonce":787341397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6805","nonce":787341397}]},"public_addr":"172.21.15.143:6801/787341397","cluster_addr":"172.21.15.143:6803/787341397","heartbeat_back_addr":"172.21.15.143:6807/787341397","heartbeat_front_addr":"172.21.15.143:6805/787341397","state":["exists","up"]},{"osd":2,"uuid":"cdbf522c-60ef-4d60-9653-8dcb529c101a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6822","nonce":2301325597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6826","nonce":2301325597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6831","nonce":2301325597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6829","nonce":2301325597}]},"public_addr":"172.21.15.143:6822/2301325597","cluster_addr":"172.21.15.143:6826/2301325597","heartbeat_back_addr":"172.21.15.143:6831/2301325597","heartbeat_front_addr":"172.21.15.143:6829/2301325597","state":["exists","up"]},{"osd":3,"uuid":"4e2783cf-ebb9-4dd8-8245-bc1d17ce4325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6817","nonce":2021504428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6819","nonce":2021504428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6827","nonce":2021504428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6823","nonce":2021504428}]},"public_addr":"172.21.15.143:6817/2021504428","cluster_addr":"172.21.15.143:6819/2021504428","heartbeat_back_addr":"172.21.15.143:6827/2021504428","heartbeat_front_addr":"172.21.15.143:6823/2021504428","state":["exists","up"]},{"osd":4,"uuid":"691256fa-9b77-49b9-a346-7ae1adf0ca80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6801","nonce":1993380595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6803","nonce":1993380595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6807","nonce":1993380595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6805","nonce":1993380595}]},"public_addr":"172.21.15.174:6801/1993380595","cluster_addr":"172.21.15.174:6803/1993380595","heartbeat_back_addr":"172.21.15.174:6807/1993380595","heartbeat_front_addr":"172.21.15.174:6805/1993380595","state":["exists","up"]},{"osd":5,"uuid":"756fc352-48c9-46ce-b52e-0541fe6f8525","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6809","nonce":1777923303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6811","nonce":1777923303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6815","nonce":1777923303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6813","nonce":1777923303}]},"public_addr":"172.21.15.174:6809/1777923303","cluster_addr":"172.21.15.174:6811/1777923303","heartbeat_back_addr":"172.21.15.174:6815/1777923303","heartbeat_front_addr":"172.21.15.174:6813/1777923303","state":["exists","up"]},{"osd":6,"uuid":"fab2ac2b-ed97-4cde-9825-b480eabf8b8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6825","nonce":2755130316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6827","nonce":2755130316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6831","nonce":2755130316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6829","nonce":2755130316}]},"public_addr":"172.21.15.174:6825/2755130316","cluster_addr":"172.21.15.174:6827/2755130316","heartbeat_back_addr":"172.21.15.174:6831/2755130316","heartbeat_front_addr":"172.21.15.174:6829/2755130316","state":["exists","up"]},{"osd":7,"uuid":"3de9b797-2ab6-4d41-a863-7e7d638fd703","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6817","nonce":2847141411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6819","nonce":2847141411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6823","nonce":2847141411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6821","nonce":2847141411}]},"public_addr":"172.21.15.174:6817/2847141411","cluster_addr":"172.21.15.174:6819/2847141411","heartbeat_back_addr":"172.21.15.174:6823/2847141411","heartbeat_front_addr":"172.21.15.174:6821/2847141411","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.477076+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.496890+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.539305+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.454673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.538881+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.521294+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.520712+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.540404+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.35","mappings":[{"from":5,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.174:6834/3601088618":"2023-11-09T19:30:47.736745+0000","172.21.15.143:6834/1535103237":"2023-11-09T19:30:40.630538+0000","172.21.15.143:6832/1535103237":"2023-11-09T19:30:40.630538+0000","172.21.15.143:0/2918728496":"2023-11-08T20:30:55.736484+0000","172.21.15.143:6835/3664584522":"2023-11-09T19:30:12.566205+0000","172.21.15.143:6833/3664584522":"2023-11-09T19:30:12.566205+0000","172.21.15.143:6832/3172664377":"2023-11-09T19:29:41.150862+0000","172.21.15.174:6836/3601088618":"2023-11-09T19:30:47.736745+0000","172.21.15.174:6837/2201447629":"2023-11-09T19:28:48.342407+0000","172.21.15.174:6835/2201447629":"2023-11-09T19:28:48.342407+0000","172.21.15.143:6833/3953059900":"2023-11-09T19:29:14.940251+0000","172.21.15.143:6835/3953059900":"2023-11-09T19:29:14.940251+0000","172.21.15.143:6834/3172664377":"2023-11-09T19:29:41.150862+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:31:09.527 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.525+0000 7f3133d4d700 1 -- 172.21.15.143:0/3080104330 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f311006af00 msgr2=0x7f311006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:09.527 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.525+0000 7f3133d4d700 1 --2- 172.21.15.143:0/3080104330 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f311006af00 0x7f311006d3c0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f312c15d900 tx=0x7f3128009040 comp rx=0 tx=0).stop 2023-11-08T19:31:09.527 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.525+0000 7f3133d4d700 1 -- 172.21.15.143:0/3080104330 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f312c06de80 msgr2=0x7f312c153d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:09.527 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.525+0000 7f3133d4d700 1 --2- 172.21.15.143:0/3080104330 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f312c06de80 0x7f312c153d80 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f311c009840 tx=0x7f311c004750 comp rx=0 tx=0).stop 2023-11-08T19:31:09.528 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.525+0000 7f3133d4d700 1 -- 172.21.15.143:0/3080104330 shutdown_connections 2023-11-08T19:31:09.528 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.525+0000 7f3133d4d700 1 --2- 172.21.15.143:0/3080104330 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f311006af00 0x7f311006d3c0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:09.528 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.525+0000 7f3133d4d700 1 --2- 172.21.15.143:0/3080104330 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f312c0b2770 0x7f312c0d0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:09.528 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.525+0000 7f3133d4d700 1 --2- 172.21.15.143:0/3080104330 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f312c06de80 0x7f312c153d80 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:09.528 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.525+0000 7f3133d4d700 1 --2- 172.21.15.143:0/3080104330 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f312c0722d0 0x7f312c0d07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:09.529 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.525+0000 7f3133d4d700 1 -- 172.21.15.143:0/3080104330 >> 172.21.15.143:0/3080104330 conn(0x7f312c06bcf0 msgr2=0x7f312c15e5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:09.529 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.525+0000 7f3133d4d700 1 -- 172.21.15.143:0/3080104330 shutdown_connections 2023-11-08T19:31:09.529 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.525+0000 7f3133d4d700 1 -- 172.21.15.143:0/3080104330 wait complete. 2023-11-08T19:31:09.539 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-08T19:31:09.540 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:31:09.659 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.658+0000 7f7d6c1d4700 1 Processor -- start 2023-11-08T19:31:09.660 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.658+0000 7f7d6c1d4700 1 -- start start 2023-11-08T19:31:09.662 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.659+0000 7f7d6c1d4700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f7d6415a8f0 0x7f7d6415f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:09.663 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.659+0000 7f7d6c1d4700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7d640ccea0 0x7f7d640cd2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:09.663 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.659+0000 7f7d6c1d4700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d6406f780 con 0x7f7d6415acd0 2023-11-08T19:31:09.663 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.659+0000 7f7d6c1d4700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d6406e8e0 con 0x7f7d6415a8f0 2023-11-08T19:31:09.663 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.659+0000 7f7d6c1d4700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d6406eb70 con 0x7f7d640ccea0 2023-11-08T19:31:09.664 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.659+0000 7f7d6976f700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f7d6415acd0 0x7f7d6406e1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:53080/0 (socket says 172.21.15.143:53080) 2023-11-08T19:31:09.664 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.659+0000 7f7d6976f700 1 -- 172.21.15.143:0/220320817 learned_addr learned my addr 172.21.15.143:0/220320817 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:31:09.664 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.659+0000 7f7d6a771700 1 --2- 172.21.15.143:0/220320817 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7d640ccea0 0x7f7d640cd2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:09.664 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.659+0000 7f7d69f70700 1 --2- 172.21.15.143:0/220320817 >> v2:172.21.15.174:3300/0 conn(0x7f7d6415a8f0 0x7f7d6415f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:09.664 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.659+0000 7f7d68f6e700 1 -- 172.21.15.143:0/220320817 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3558623641 0 0) 0x7f7d6406f780 con 0x7f7d6415acd0 2023-11-08T19:31:09.665 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.659+0000 7f7d68f6e700 1 -- 172.21.15.143:0/220320817 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d500035b0 con 0x7f7d6415acd0 2023-11-08T19:31:09.665 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.660+0000 7f7d68f6e700 1 -- 172.21.15.143:0/220320817 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1681012546 0 0) 0x7f7d500035b0 con 0x7f7d6415acd0 2023-11-08T19:31:09.665 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.660+0000 7f7d6a771700 1 -- 172.21.15.143:0/220320817 >> v2:172.21.15.174:3300/0 conn(0x7f7d6415a8f0 msgr2=0x7f7d6415f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:09.665 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.660+0000 7f7d6a771700 1 --2- 172.21.15.143:0/220320817 >> v2:172.21.15.174:3300/0 conn(0x7f7d6415a8f0 0x7f7d6415f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:09.665 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.660+0000 7f7d6a771700 1 -- 172.21.15.143:0/220320817 >> v1:172.21.15.143:6789/0 conn(0x7f7d6415acd0 legacy=0x7f7d6406e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:09.666 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.660+0000 7f7d6a771700 1 -- 172.21.15.143:0/220320817 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d6406ee50 con 0x7f7d640ccea0 2023-11-08T19:31:09.666 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.660+0000 7f7d69f70700 1 --2- 172.21.15.143:0/220320817 >> v2:172.21.15.174:3300/0 conn(0x7f7d6415a8f0 0x7f7d6415f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T19:31:09.666 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.660+0000 7f7d6a771700 1 --2- 172.21.15.143:0/220320817 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7d640ccea0 0x7f7d640cd2d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c009d90 tx=0x7f7d5c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b03d33e8435d1fe server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:09.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.660+0000 7f7d68f6e700 1 -- 172.21.15.143:0/220320817 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d5c00e070 con 0x7f7d640ccea0 2023-11-08T19:31:09.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.660+0000 7f7d68f6e700 1 -- 172.21.15.143:0/220320817 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d5c003a20 con 0x7f7d640ccea0 2023-11-08T19:31:09.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.660+0000 7f7d68f6e700 1 -- 172.21.15.143:0/220320817 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d54001670 con 0x7f7d640ccea0 2023-11-08T19:31:09.668 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.661+0000 7f7d6c1d4700 1 -- 172.21.15.143:0/220320817 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7d640ccea0 msgr2=0x7f7d640cd2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:09.668 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.661+0000 7f7d6c1d4700 1 --2- 172.21.15.143:0/220320817 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7d640ccea0 0x7f7d640cd2d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c009d90 tx=0x7f7d5c00b330 comp rx=0 tx=0).stop 2023-11-08T19:31:09.668 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.661+0000 7f7d6c1d4700 1 -- 172.21.15.143:0/220320817 shutdown_connections 2023-11-08T19:31:09.668 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.661+0000 7f7d6c1d4700 1 --2- 172.21.15.143:0/220320817 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7d640ccea0 0x7f7d640cd2d0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:09.668 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.661+0000 7f7d6c1d4700 1 --2- 172.21.15.143:0/220320817 >> v2:172.21.15.174:3300/0 conn(0x7f7d6415a8f0 0x7f7d6415f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:09.669 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.661+0000 7f7d6c1d4700 1 -- 172.21.15.143:0/220320817 >> 172.21.15.143:0/220320817 conn(0x7f7d640b23d0 msgr2=0x7f7d640b27b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:09.669 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.661+0000 7f7d6c1d4700 1 -- 172.21.15.143:0/220320817 shutdown_connections 2023-11-08T19:31:09.669 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.661+0000 7f7d6c1d4700 1 -- 172.21.15.143:0/220320817 wait complete. 2023-11-08T19:31:09.672 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.662+0000 7f7d6c1d4700 1 Processor -- start 2023-11-08T19:31:09.672 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.662+0000 7f7d6c1d4700 1 -- start start 2023-11-08T19:31:09.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.663+0000 7f7d6c1d4700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7d640ccea0 0x7f7d64156d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:09.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.663+0000 7f7d6c1d4700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7d6415a8f0 0x7f7d641e6920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:09.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.663+0000 7f7d6c1d4700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7d6415acd0 0x7f7d641e6e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:09.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.663+0000 7f7d6c1d4700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d640c8660 con 0x7f7d6415a8f0 2023-11-08T19:31:09.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.663+0000 7f7d6c1d4700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d641e74e0 con 0x7f7d6415acd0 2023-11-08T19:31:09.674 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.663+0000 7f7d6c1d4700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d641e77c0 con 0x7f7d640ccea0 2023-11-08T19:31:09.674 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.663+0000 7f7d6976f700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7d6415a8f0 0x7f7d641e6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:09.674 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.663+0000 7f7d6976f700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7d6415a8f0 0x7f7d641e6920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56932/0 (socket says 172.21.15.143:56932) 2023-11-08T19:31:09.674 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.663+0000 7f7d6976f700 1 -- 172.21.15.143:0/2919514184 learned_addr learned my addr 172.21.15.143:0/2919514184 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:31:09.674 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.664+0000 7f7d69f70700 1 --2- 172.21.15.143:0/2919514184 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7d640ccea0 0x7f7d64156d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:09.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.664+0000 7f7d6a771700 1 --2- 172.21.15.143:0/2919514184 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7d6415acd0 0x7f7d641e6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:09.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.664+0000 7f7d6976f700 1 -- 172.21.15.143:0/2919514184 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7d640ccea0 msgr2=0x7f7d64156d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:09.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.664+0000 7f7d6976f700 1 --2- 172.21.15.143:0/2919514184 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7d640ccea0 0x7f7d64156d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:09.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.664+0000 7f7d6976f700 1 -- 172.21.15.143:0/2919514184 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7d6415acd0 msgr2=0x7f7d641e6e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:09.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.664+0000 7f7d6976f700 1 --2- 172.21.15.143:0/2919514184 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7d6415acd0 0x7f7d641e6e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:09.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.664+0000 7f7d6976f700 1 -- 172.21.15.143:0/2919514184 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d600094a0 con 0x7f7d6415a8f0 2023-11-08T19:31:09.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.664+0000 7f7d69f70700 1 --2- 172.21.15.143:0/2919514184 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7d640ccea0 0x7f7d64156d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:31:09.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.664+0000 7f7d6976f700 1 --2- 172.21.15.143:0/2919514184 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7d6415a8f0 0x7f7d641e6920 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f7d5400c750 tx=0x7f7d5400cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:09.676 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.664+0000 7f7d5a7fc700 1 -- 172.21.15.143:0/2919514184 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d54004c80 con 0x7f7d6415a8f0 2023-11-08T19:31:09.677 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.664+0000 7f7d5a7fc700 1 -- 172.21.15.143:0/2919514184 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d54004de0 con 0x7f7d6415a8f0 2023-11-08T19:31:09.677 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.664+0000 7f7d6c1d4700 1 -- 172.21.15.143:0/2919514184 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d5c009a70 con 0x7f7d6415a8f0 2023-11-08T19:31:09.677 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.664+0000 7f7d5a7fc700 1 -- 172.21.15.143:0/2919514184 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d540072f0 con 0x7f7d6415a8f0 2023-11-08T19:31:09.677 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.664+0000 7f7d6c1d4700 1 -- 172.21.15.143:0/2919514184 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d640c82f0 con 0x7f7d6415a8f0 2023-11-08T19:31:09.677 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.665+0000 7f7d6c1d4700 1 -- 172.21.15.143:0/2919514184 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d640c8810 con 0x7f7d6415a8f0 2023-11-08T19:31:09.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.665+0000 7f7d5a7fc700 1 -- 172.21.15.143:0/2919514184 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7d54007450 con 0x7f7d6415a8f0 2023-11-08T19:31:09.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.666+0000 7f7d5a7fc700 1 --2- 172.21.15.143:0/2919514184 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f7d4806af00 0x7f7d4806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:09.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.666+0000 7f7d5a7fc700 1 -- 172.21.15.143:0/2919514184 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 7360+0+0 (secure 0 0 0) 0x7f7d540899d0 con 0x7f7d6415a8f0 2023-11-08T19:31:09.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.666+0000 7f7d69f70700 1 --2- 172.21.15.143:0/2919514184 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f7d4806af00 0x7f7d4806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:09.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.666+0000 7f7d69f70700 1 --2- 172.21.15.143:0/2919514184 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f7d4806af00 0x7f7d4806d3c0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f7d640c9f60 tx=0x7f7d60008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:09.679 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.670+0000 7f7d5a7fc700 1 -- 172.21.15.143:0/2919514184 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7d640c8810 con 0x7f7d6415a8f0 2023-11-08T19:31:09.902 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.901+0000 7f7d6c1d4700 1 -- 172.21.15.143:0/2919514184 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7d6404f730 con 0x7f7d6415a8f0 2023-11-08T19:31:09.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.902+0000 7f7d5a7fc700 1 -- 172.21.15.143:0/2919514184 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 100 v100) v1 ==== 95+0+4946 (secure 0 0 0) 0x7f7d540544a0 con 0x7f7d6415a8f0 2023-11-08T19:31:09.904 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:31:09.904 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":100,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5933,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/3918286710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6835","nonce":3918286710},{"type":"v1","addr":"172.21.15.174:6837","nonce":3918286710}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":6245,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6836/2689884850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2689884850},{"type":"v1","addr":"172.21.15.174:6836","nonce":2689884850}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":97}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:03.092245+0000","modified":"2023-11-08T19:30:56.788736+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":103,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6161,"mds_1":7743},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6161":{"gid":6161,"name":"a","rank":0,"incarnation":93,"state":"up:active","state_seq":14,"addr":"172.21.15.143:6835/2129643800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":2129643800},{"type":"v1","addr":"172.21.15.143:6835","nonce":2129643800}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7743":{"gid":7743,"name":"c","rank":1,"incarnation":96,"state":"up:active","state_seq":7,"addr":"172.21.15.143:6834/2489252684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":2489252684},{"type":"v1","addr":"172.21.15.143:6834","nonce":2489252684}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:31:09.906 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.904+0000 7f7d6c1d4700 1 -- 172.21.15.143:0/2919514184 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f7d4806af00 msgr2=0x7f7d4806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:09.906 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.904+0000 7f7d6c1d4700 1 --2- 172.21.15.143:0/2919514184 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f7d4806af00 0x7f7d4806d3c0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f7d640c9f60 tx=0x7f7d60008040 comp rx=0 tx=0).stop 2023-11-08T19:31:09.906 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.904+0000 7f7d6c1d4700 1 -- 172.21.15.143:0/2919514184 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7d6415a8f0 msgr2=0x7f7d641e6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:09.906 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.904+0000 7f7d6c1d4700 1 --2- 172.21.15.143:0/2919514184 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7d6415a8f0 0x7f7d641e6920 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f7d5400c750 tx=0x7f7d5400cb10 comp rx=0 tx=0).stop 2023-11-08T19:31:09.907 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.904+0000 7f7d6c1d4700 1 -- 172.21.15.143:0/2919514184 shutdown_connections 2023-11-08T19:31:09.907 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.904+0000 7f7d6c1d4700 1 --2- 172.21.15.143:0/2919514184 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f7d4806af00 0x7f7d4806d3c0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:09.907 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.904+0000 7f7d6c1d4700 1 --2- 172.21.15.143:0/2919514184 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7d640ccea0 0x7f7d64156d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:09.907 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.904+0000 7f7d6c1d4700 1 --2- 172.21.15.143:0/2919514184 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7d6415acd0 0x7f7d641e6e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:09.908 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.904+0000 7f7d6c1d4700 1 --2- 172.21.15.143:0/2919514184 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7d6415a8f0 0x7f7d641e6920 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:09.908 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.904+0000 7f7d6c1d4700 1 -- 172.21.15.143:0/2919514184 >> 172.21.15.143:0/2919514184 conn(0x7f7d640b23d0 msgr2=0x7f7d640b3370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:09.908 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.904+0000 7f7d6c1d4700 1 -- 172.21.15.143:0/2919514184 shutdown_connections 2023-11-08T19:31:09.908 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:09.904+0000 7f7d6c1d4700 1 -- 172.21.15.143:0/2919514184 wait complete. 2023-11-08T19:31:09.909 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 100 2023-11-08T19:31:09.917 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:31:10.037 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.035+0000 7f590b81d700 1 Processor -- start 2023-11-08T19:31:10.037 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.036+0000 7f590b81d700 1 -- start start 2023-11-08T19:31:10.040 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.036+0000 7f590b81d700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f590415b900 0x7f59040ce3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:10.040 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.036+0000 7f590b81d700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f590415eac0 0x7f590406abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:10.040 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.036+0000 7f590b81d700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59040ce910 con 0x7f590415ef40 2023-11-08T19:31:10.040 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.036+0000 7f590b81d700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5904070120 con 0x7f590415b900 2023-11-08T19:31:10.041 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.037+0000 7f590b81d700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59040cb640 con 0x7f590415eac0 2023-11-08T19:31:10.041 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.037+0000 7f59095b9700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f590415b900 0x7f59040ce3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:10.041 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.037+0000 7f59095b9700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f590415b900 0x7f59040ce3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56938/0 (socket says 172.21.15.143:56938) 2023-11-08T19:31:10.041 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.037+0000 7f59095b9700 1 -- 172.21.15.143:0/268150909 learned_addr learned my addr 172.21.15.143:0/268150909 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:31:10.041 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.037+0000 7f5909dba700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f590415ef40 0x7f5904161ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:53096/0 (socket says 172.21.15.143:53096) 2023-11-08T19:31:10.042 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.037+0000 7f5908db8700 1 --2- 172.21.15.143:0/268150909 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f590415eac0 0x7f590406abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:10.042 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.037+0000 7f59095b9700 1 -- 172.21.15.143:0/268150909 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f590415eac0 msgr2=0x7f590406abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:10.042 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.037+0000 7f59095b9700 1 --2- 172.21.15.143:0/268150909 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f590415eac0 0x7f590406abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:10.042 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.037+0000 7f59095b9700 1 -- 172.21.15.143:0/268150909 >> v1:172.21.15.143:6789/0 conn(0x7f590415ef40 legacy=0x7f5904161ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:10.042 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.037+0000 7f59095b9700 1 -- 172.21.15.143:0/268150909 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59040d0500 con 0x7f590415b900 2023-11-08T19:31:10.043 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.037+0000 7f58fbfff700 1 -- 172.21.15.143:0/268150909 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2373895415 0 0) 0x7f59040ce910 con 0x7f590415ef40 2023-11-08T19:31:10.043 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.037+0000 7f59095b9700 1 --2- 172.21.15.143:0/268150909 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f590415b900 0x7f59040ce3d0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f58f40098f0 tx=0x7f58f4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81d26b5533c1d647 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:10.043 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.037+0000 7f58fbfff700 1 -- 172.21.15.143:0/268150909 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f400b8a0 con 0x7f590415b900 2023-11-08T19:31:10.043 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.037+0000 7f58fbfff700 1 -- 172.21.15.143:0/268150909 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58f400ba00 con 0x7f590415b900 2023-11-08T19:31:10.043 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.037+0000 7f58fbfff700 1 -- 172.21.15.143:0/268150909 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f400bd90 con 0x7f590415b900 2023-11-08T19:31:10.044 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.038+0000 7f590b81d700 1 -- 172.21.15.143:0/268150909 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f590415b900 msgr2=0x7f59040ce3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:10.044 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.038+0000 7f590b81d700 1 --2- 172.21.15.143:0/268150909 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f590415b900 0x7f59040ce3d0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f58f40098f0 tx=0x7f58f4009c00 comp rx=0 tx=0).stop 2023-11-08T19:31:10.044 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.038+0000 7f590b81d700 1 -- 172.21.15.143:0/268150909 shutdown_connections 2023-11-08T19:31:10.044 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.038+0000 7f590b81d700 1 --2- 172.21.15.143:0/268150909 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f590415b900 0x7f59040ce3d0 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:10.044 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.038+0000 7f590b81d700 1 --2- 172.21.15.143:0/268150909 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f590415eac0 0x7f590406abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:10.045 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.038+0000 7f590b81d700 1 -- 172.21.15.143:0/268150909 >> 172.21.15.143:0/268150909 conn(0x7f59040b23d0 msgr2=0x7f59040b27b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:10.045 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.038+0000 7f590b81d700 1 -- 172.21.15.143:0/268150909 shutdown_connections 2023-11-08T19:31:10.045 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.038+0000 7f590b81d700 1 -- 172.21.15.143:0/268150909 wait complete. 2023-11-08T19:31:10.048 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.039+0000 7f590b81d700 1 Processor -- start 2023-11-08T19:31:10.048 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.039+0000 7f590b81d700 1 -- start start 2023-11-08T19:31:10.048 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.040+0000 7f590b81d700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f590415b900 0x7f59040ae330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:10.048 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.040+0000 7f590b81d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f590415eac0 0x7f590406ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:10.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.040+0000 7f590b81d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f590415ef40 0x7f59040c8e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:10.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.040+0000 7f590b81d700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59040c8ac0 con 0x7f590415eac0 2023-11-08T19:31:10.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.040+0000 7f590b81d700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f590414a8c0 con 0x7f590415ef40 2023-11-08T19:31:10.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.040+0000 7f590b81d700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59040c7e40 con 0x7f590415b900 2023-11-08T19:31:10.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.040+0000 7f5908db8700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f590415eac0 0x7f590406ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:10.050 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.040+0000 7f5908db8700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f590415eac0 0x7f590406ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56946/0 (socket says 172.21.15.143:56946) 2023-11-08T19:31:10.050 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.040+0000 7f5908db8700 1 -- 172.21.15.143:0/3083320119 learned_addr learned my addr 172.21.15.143:0/3083320119 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:31:10.050 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.040+0000 7f59095b9700 1 --2- 172.21.15.143:0/3083320119 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f590415b900 0x7f59040ae330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:10.050 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.040+0000 7f5909dba700 1 --2- 172.21.15.143:0/3083320119 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f590415ef40 0x7f59040c8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:10.051 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.040+0000 7f5908db8700 1 -- 172.21.15.143:0/3083320119 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f590415b900 msgr2=0x7f59040ae330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:10.051 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.040+0000 7f5908db8700 1 --2- 172.21.15.143:0/3083320119 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f590415b900 0x7f59040ae330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:10.051 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.040+0000 7f5908db8700 1 -- 172.21.15.143:0/3083320119 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f590415ef40 msgr2=0x7f59040c8e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:10.051 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.040+0000 7f5908db8700 1 --2- 172.21.15.143:0/3083320119 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f590415ef40 0x7f59040c8e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:10.051 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.040+0000 7f5908db8700 1 -- 172.21.15.143:0/3083320119 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58f4009550 con 0x7f590415eac0 2023-11-08T19:31:10.052 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.040+0000 7f5909dba700 1 --2- 172.21.15.143:0/3083320119 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f590415ef40 0x7f59040c8e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:31:10.052 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.040+0000 7f5908db8700 1 --2- 172.21.15.143:0/3083320119 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f590415eac0 0x7f590406ea10 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f590000e840 tx=0x7f590000ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:10.052 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.040+0000 7f58f9ffb700 1 -- 172.21.15.143:0/3083320119 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5900013610 con 0x7f590415eac0 2023-11-08T19:31:10.052 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.040+0000 7f58f9ffb700 1 -- 172.21.15.143:0/3083320119 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5900013770 con 0x7f590415eac0 2023-11-08T19:31:10.052 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.041+0000 7f590b81d700 1 -- 172.21.15.143:0/3083320119 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59040c8120 con 0x7f590415eac0 2023-11-08T19:31:10.052 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.041+0000 7f58f9ffb700 1 -- 172.21.15.143:0/3083320119 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5900013b20 con 0x7f590415eac0 2023-11-08T19:31:10.053 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.041+0000 7f590b81d700 1 -- 172.21.15.143:0/3083320119 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59040c86e0 con 0x7f590415eac0 2023-11-08T19:31:10.053 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.042+0000 7f590b81d700 1 -- 172.21.15.143:0/3083320119 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f590404f730 con 0x7f590415eac0 2023-11-08T19:31:10.053 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.042+0000 7f58f9ffb700 1 -- 172.21.15.143:0/3083320119 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5900013c80 con 0x7f590415eac0 2023-11-08T19:31:10.053 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.043+0000 7f58f9ffb700 1 --2- 172.21.15.143:0/3083320119 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f58e806af00 0x7f58e806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:10.053 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.043+0000 7f58f9ffb700 1 -- 172.21.15.143:0/3083320119 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 7360+0+0 (secure 0 0 0) 0x7f59000898b0 con 0x7f590415eac0 2023-11-08T19:31:10.054 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.043+0000 7f59095b9700 1 --2- 172.21.15.143:0/3083320119 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f58e806af00 0x7f58e806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:10.054 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.043+0000 7f59095b9700 1 --2- 172.21.15.143:0/3083320119 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f58e806af00 0x7f58e806d3c0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f58f4014040 tx=0x7f58f4008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:10.054 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.047+0000 7f58f9ffb700 1 -- 172.21.15.143:0/3083320119 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f59000542f0 con 0x7f590415eac0 2023-11-08T19:31:10.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.276+0000 7f590b81d700 1 -- 172.21.15.143:0/3083320119 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5904072300 con 0x7f590415eac0 2023-11-08T19:31:10.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.276+0000 7f58f9ffb700 1 -- 172.21.15.143:0/3083320119 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 100 v100) v1 ==== 95+0+4946 (secure 0 0 0) 0x7f5900057910 con 0x7f590415eac0 2023-11-08T19:31:10.278 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:31:10.278 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":100,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5933,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/3918286710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6835","nonce":3918286710},{"type":"v1","addr":"172.21.15.174:6837","nonce":3918286710}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":6245,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6836/2689884850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2689884850},{"type":"v1","addr":"172.21.15.174:6836","nonce":2689884850}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":97}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:03.092245+0000","modified":"2023-11-08T19:30:56.788736+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":103,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6161,"mds_1":7743},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6161":{"gid":6161,"name":"a","rank":0,"incarnation":93,"state":"up:active","state_seq":14,"addr":"172.21.15.143:6835/2129643800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":2129643800},{"type":"v1","addr":"172.21.15.143:6835","nonce":2129643800}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7743":{"gid":7743,"name":"c","rank":1,"incarnation":96,"state":"up:active","state_seq":7,"addr":"172.21.15.143:6834/2489252684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":2489252684},{"type":"v1","addr":"172.21.15.143:6834","nonce":2489252684}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:31:10.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.279+0000 7f590b81d700 1 -- 172.21.15.143:0/3083320119 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f58e806af00 msgr2=0x7f58e806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:10.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.279+0000 7f590b81d700 1 --2- 172.21.15.143:0/3083320119 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f58e806af00 0x7f58e806d3c0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f58f4014040 tx=0x7f58f4008030 comp rx=0 tx=0).stop 2023-11-08T19:31:10.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.279+0000 7f590b81d700 1 -- 172.21.15.143:0/3083320119 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f590415eac0 msgr2=0x7f590406ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:10.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.279+0000 7f590b81d700 1 --2- 172.21.15.143:0/3083320119 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f590415eac0 0x7f590406ea10 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f590000e840 tx=0x7f590000ec00 comp rx=0 tx=0).stop 2023-11-08T19:31:10.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.279+0000 7f590b81d700 1 -- 172.21.15.143:0/3083320119 shutdown_connections 2023-11-08T19:31:10.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.279+0000 7f590b81d700 1 --2- 172.21.15.143:0/3083320119 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f58e806af00 0x7f58e806d3c0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:10.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.279+0000 7f590b81d700 1 --2- 172.21.15.143:0/3083320119 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f590415b900 0x7f59040ae330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:10.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.279+0000 7f590b81d700 1 --2- 172.21.15.143:0/3083320119 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f590415ef40 0x7f59040c8e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:10.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.279+0000 7f590b81d700 1 --2- 172.21.15.143:0/3083320119 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f590415eac0 0x7f590406ea10 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:10.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.279+0000 7f590b81d700 1 -- 172.21.15.143:0/3083320119 >> 172.21.15.143:0/3083320119 conn(0x7f59040b23d0 msgr2=0x7f59040b09b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:10.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.279+0000 7f590b81d700 1 -- 172.21.15.143:0/3083320119 shutdown_connections 2023-11-08T19:31:10.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.279+0000 7f590b81d700 1 -- 172.21.15.143:0/3083320119 wait complete. 2023-11-08T19:31:10.283 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 100 2023-11-08T19:31:10.292 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:31:10.412 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.411+0000 7fa03fff8700 1 Processor -- start 2023-11-08T19:31:10.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.411+0000 7fa03fff8700 1 -- start start 2023-11-08T19:31:10.417 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.412+0000 7fa03fff8700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fa0380b2f30 0x7fa0380ceb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:10.417 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.412+0000 7fa03fff8700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0380d0000 con 0x7fa0380b2b50 2023-11-08T19:31:10.417 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.412+0000 7fa03fff8700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0380d02e0 con 0x7fa03806ce10 2023-11-08T19:31:10.417 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.412+0000 7fa03fff8700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0380d05c0 con 0x7fa0380b2f30 2023-11-08T19:31:10.417 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.412+0000 7fa03dd94700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fa0380b2b50 0x7fa03806c700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:53106/0 (socket says 172.21.15.143:53106) 2023-11-08T19:31:10.418 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.412+0000 7fa03dd94700 1 -- 172.21.15.143:0/3276330648 learned_addr learned my addr 172.21.15.143:0/3276330648 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:31:10.418 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.412+0000 7fa03e595700 1 --2- 172.21.15.143:0/3276330648 >> v2:172.21.15.174:3300/0 conn(0x7fa0380b2f30 0x7fa0380ceb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:10.418 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.412+0000 7fa03cd92700 1 -- 172.21.15.143:0/3276330648 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2415154123 0 0) 0x7fa0380d0000 con 0x7fa0380b2b50 2023-11-08T19:31:10.418 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.412+0000 7fa03cd92700 1 -- 172.21.15.143:0/3276330648 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa0240035b0 con 0x7fa0380b2b50 2023-11-08T19:31:10.418 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.412+0000 7fa03cd92700 1 -- 172.21.15.143:0/3276330648 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3130030293 0 0) 0x7fa0240035b0 con 0x7fa0380b2b50 2023-11-08T19:31:10.419 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.412+0000 7fa03cd92700 1 -- 172.21.15.143:0/3276330648 >> v2:172.21.15.174:3300/0 conn(0x7fa0380b2f30 msgr2=0x7fa0380ceb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:10.419 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.412+0000 7fa03cd92700 1 --2- 172.21.15.143:0/3276330648 >> v2:172.21.15.174:3300/0 conn(0x7fa0380b2f30 0x7fa0380ceb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:10.419 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.413+0000 7fa03cd92700 1 -- 172.21.15.143:0/3276330648 >> v1:172.21.15.174:6789/0 conn(0x7fa03806ce10 legacy=0x7fa03815d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:10.419 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.413+0000 7fa03cd92700 1 -- 172.21.15.143:0/3276330648 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0380d08a0 con 0x7fa0380b2b50 2023-11-08T19:31:10.419 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.413+0000 7fa03cd92700 1 -- 172.21.15.143:0/3276330648 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4147756303 0 0) 0x7fa0280017f0 con 0x7fa0380b2b50 2023-11-08T19:31:10.420 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.413+0000 7fa03cd92700 1 -- 172.21.15.143:0/3276330648 >> v1:172.21.15.143:6789/0 conn(0x7fa0380b2b50 legacy=0x7fa03806c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:10.420 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.413+0000 7fa03cd92700 1 --2- 172.21.15.143:0/3276330648 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa024003a00 0x7fa024003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:10.420 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.413+0000 7fa03cd92700 1 --2- 172.21.15.143:0/3276330648 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa024004850 0x7fa024004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:10.420 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.413+0000 7fa03cd92700 1 --2- 172.21.15.143:0/3276330648 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa024005210 0x7fa02400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:10.420 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.413+0000 7fa03cd92700 1 -- 172.21.15.143:0/3276330648 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0380d05c0 con 0x7fa024005210 2023-11-08T19:31:10.420 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.413+0000 7fa03cd92700 1 -- 172.21.15.143:0/3276330648 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa02400c0c0 con 0x7fa024004850 2023-11-08T19:31:10.421 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.413+0000 7fa03cd92700 1 -- 172.21.15.143:0/3276330648 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa02400c280 con 0x7fa024003a00 2023-11-08T19:31:10.421 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.413+0000 7fa03e595700 1 --2- 172.21.15.143:0/3276330648 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa024005210 0x7fa02400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:10.421 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.413+0000 7fa03dd94700 1 --2- 172.21.15.143:0/3276330648 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa024003a00 0x7fa024003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:10.421 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.413+0000 7fa03d593700 1 --2- 172.21.15.143:0/3276330648 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa024004850 0x7fa024004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:10.421 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.413+0000 7fa03e595700 1 -- 172.21.15.143:0/3276330648 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa024003a00 msgr2=0x7fa024003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:10.422 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.413+0000 7fa03e595700 1 --2- 172.21.15.143:0/3276330648 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa024003a00 0x7fa024003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:10.422 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.413+0000 7fa03e595700 1 -- 172.21.15.143:0/3276330648 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa024004850 msgr2=0x7fa024004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:10.422 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.413+0000 7fa03e595700 1 --2- 172.21.15.143:0/3276330648 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa024004850 0x7fa024004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:10.422 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.413+0000 7fa03e595700 1 -- 172.21.15.143:0/3276330648 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0240035b0 con 0x7fa024005210 2023-11-08T19:31:10.422 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.413+0000 7fa03d593700 1 --2- 172.21.15.143:0/3276330648 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa024004850 0x7fa024004cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:31:10.423 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.413+0000 7fa03e595700 1 -- 172.21.15.143:0/3276330648 reap_dead start 2023-11-08T19:31:10.423 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.413+0000 7fa03e595700 1 --2- 172.21.15.143:0/3276330648 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa024005210 0x7fa02400ba10 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7fa030009fc0 tx=0x7fa024005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24c7e31ceb012d1 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:10.423 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.413+0000 7fa03cd92700 1 -- 172.21.15.143:0/3276330648 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa030003a20 con 0x7fa024005210 2023-11-08T19:31:10.423 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.413+0000 7fa03cd92700 1 -- 172.21.15.143:0/3276330648 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa03000f040 con 0x7fa024005210 2023-11-08T19:31:10.423 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.413+0000 7fa03cd92700 1 -- 172.21.15.143:0/3276330648 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa030003da0 con 0x7fa024005210 2023-11-08T19:31:10.423 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.414+0000 7fa03fff8700 1 -- 172.21.15.143:0/3276330648 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa024005210 msgr2=0x7fa02400ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:10.424 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.414+0000 7fa03fff8700 1 --2- 172.21.15.143:0/3276330648 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa024005210 0x7fa02400ba10 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7fa030009fc0 tx=0x7fa024005880 comp rx=0 tx=0).stop 2023-11-08T19:31:10.424 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.414+0000 7fa03fff8700 1 -- 172.21.15.143:0/3276330648 shutdown_connections 2023-11-08T19:31:10.424 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.414+0000 7fa03fff8700 1 --2- 172.21.15.143:0/3276330648 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa024005210 0x7fa02400ba10 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:10.424 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.414+0000 7fa03fff8700 1 -- 172.21.15.143:0/3276330648 >> 172.21.15.143:0/3276330648 conn(0x7fa03809aa70 msgr2=0x7fa0380d18f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:10.424 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.414+0000 7fa03fff8700 1 -- 172.21.15.143:0/3276330648 shutdown_connections 2023-11-08T19:31:10.425 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.414+0000 7fa03fff8700 1 -- 172.21.15.143:0/3276330648 wait complete. 2023-11-08T19:31:10.425 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.415+0000 7fa03fff8700 1 Processor -- start 2023-11-08T19:31:10.425 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.415+0000 7fa03fff8700 1 -- start start 2023-11-08T19:31:10.427 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.416+0000 7fa03fff8700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa024005210 0x7fa0381f37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:10.428 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.416+0000 7fa03fff8700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa0380c8360 0x7fa03815c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:10.428 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.416+0000 7fa03fff8700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa03806fcc0 0x7fa0380b17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:10.428 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.416+0000 7fa03fff8700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa03815b250 con 0x7fa0380c8360 2023-11-08T19:31:10.428 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.416+0000 7fa03fff8700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0380c80f0 con 0x7fa024005210 2023-11-08T19:31:10.428 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.416+0000 7fa03fff8700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0380c87c0 con 0x7fa03806fcc0 2023-11-08T19:31:10.429 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.416+0000 7fa03d593700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa0380c8360 0x7fa03815c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:10.429 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.416+0000 7fa03d593700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa0380c8360 0x7fa03815c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56966/0 (socket says 172.21.15.143:56966) 2023-11-08T19:31:10.429 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.416+0000 7fa03d593700 1 -- 172.21.15.143:0/1214234851 learned_addr learned my addr 172.21.15.143:0/1214234851 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:31:10.429 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.416+0000 7fa03e595700 1 --2- 172.21.15.143:0/1214234851 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa03806fcc0 0x7fa0380b17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:10.429 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.416+0000 7fa03dd94700 1 --2- 172.21.15.143:0/1214234851 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa024005210 0x7fa0381f37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:10.430 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.416+0000 7fa03d593700 1 -- 172.21.15.143:0/1214234851 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa03806fcc0 msgr2=0x7fa0380b17d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:10.430 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.416+0000 7fa03d593700 1 --2- 172.21.15.143:0/1214234851 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa03806fcc0 0x7fa0380b17d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:10.430 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.416+0000 7fa03d593700 1 -- 172.21.15.143:0/1214234851 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa024005210 msgr2=0x7fa0381f37a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:10.430 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.416+0000 7fa03d593700 1 --2- 172.21.15.143:0/1214234851 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa024005210 0x7fa0381f37a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:10.430 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.416+0000 7fa03d593700 1 -- 172.21.15.143:0/1214234851 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa03815cb70 con 0x7fa0380c8360 2023-11-08T19:31:10.431 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.416+0000 7fa03dd94700 1 --2- 172.21.15.143:0/1214234851 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa024005210 0x7fa0381f37a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:31:10.431 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.416+0000 7fa03d593700 1 --2- 172.21.15.143:0/1214234851 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa0380c8360 0x7fa03815c630 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7fa03400e580 tx=0x7fa03400e940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:10.431 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.416+0000 7fa02e7fc700 1 -- 172.21.15.143:0/1214234851 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0340052e0 con 0x7fa0380c8360 2023-11-08T19:31:10.431 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.417+0000 7fa02e7fc700 1 -- 172.21.15.143:0/1214234851 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa034005440 con 0x7fa0380c8360 2023-11-08T19:31:10.431 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.417+0000 7fa03fff8700 1 -- 172.21.15.143:0/1214234851 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0381f3e30 con 0x7fa0380c8360 2023-11-08T19:31:10.432 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.417+0000 7fa02e7fc700 1 -- 172.21.15.143:0/1214234851 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0340038c0 con 0x7fa0380c8360 2023-11-08T19:31:10.432 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.417+0000 7fa03fff8700 1 -- 172.21.15.143:0/1214234851 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0381f43f0 con 0x7fa0380c8360 2023-11-08T19:31:10.432 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.418+0000 7fa03fff8700 1 -- 172.21.15.143:0/1214234851 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa03804f730 con 0x7fa0380c8360 2023-11-08T19:31:10.432 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.418+0000 7fa02e7fc700 1 -- 172.21.15.143:0/1214234851 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa034003a20 con 0x7fa0380c8360 2023-11-08T19:31:10.432 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.418+0000 7fa02e7fc700 1 --2- 172.21.15.143:0/1214234851 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa01c06af00 0x7fa01c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:10.433 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.419+0000 7fa02e7fc700 1 -- 172.21.15.143:0/1214234851 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 7360+0+0 (secure 0 0 0) 0x7fa034089220 con 0x7fa0380c8360 2023-11-08T19:31:10.433 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.419+0000 7fa03dd94700 1 --2- 172.21.15.143:0/1214234851 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa01c06af00 0x7fa01c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:10.433 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.419+0000 7fa03dd94700 1 --2- 172.21.15.143:0/1214234851 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa01c06af00 0x7fa01c06d3c0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fa028000ef0 tx=0x7fa028009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:10.433 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.423+0000 7fa02e7fc700 1 -- 172.21.15.143:0/1214234851 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa034053da0 con 0x7fa0380c8360 2023-11-08T19:31:10.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.611+0000 7fa03fff8700 1 -- 172.21.15.143:0/1214234851 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa0380d2000 con 0x7fa0380c8360 2023-11-08T19:31:10.613 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.612+0000 7fa02e7fc700 1 -- 172.21.15.143:0/1214234851 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v104) v1 ==== 74+0+22721 (secure 0 0 0) 0x7fa0340573c0 con 0x7fa0380c8360 2023-11-08T19:31:10.614 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:31:10.614 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":104,"fsid":"968483af-77fc-4068-ad7c-5c2a1a4613d7","created":"2023-11-08T19:24:32.826484+0000","modified":"2023-11-08T19:30:55.774106+0000","last_up_change":"2023-11-08T19:24:43.573880+0000","last_in_change":"2023-11-08T19:24:35.314525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:44.551530+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"snap-test-pool","create_time":"2023-11-08T19:26:02.874384+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":36,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"test_data_pool","create_time":"2023-11-08T19:27:05.601949+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":66,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_metadata_pool","create_time":"2023-11-08T19:27:06.536369+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-11-08T19:28:01.533566+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-11-08T19:28:02.053445+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":9,"snap_epoch":100,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bae98ec1-d94e-403d-a220-a522a53ea6f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6809","nonce":1487830566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6811","nonce":1487830566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6815","nonce":1487830566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6813","nonce":1487830566}]},"public_addr":"172.21.15.143:6809/1487830566","cluster_addr":"172.21.15.143:6811/1487830566","heartbeat_back_addr":"172.21.15.143:6815/1487830566","heartbeat_front_addr":"172.21.15.143:6813/1487830566","state":["exists","up"]},{"osd":1,"uuid":"5cb6538a-1ccb-4322-8633-09f66af0b7e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6801","nonce":787341397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6803","nonce":787341397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6807","nonce":787341397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6805","nonce":787341397}]},"public_addr":"172.21.15.143:6801/787341397","cluster_addr":"172.21.15.143:6803/787341397","heartbeat_back_addr":"172.21.15.143:6807/787341397","heartbeat_front_addr":"172.21.15.143:6805/787341397","state":["exists","up"]},{"osd":2,"uuid":"cdbf522c-60ef-4d60-9653-8dcb529c101a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6822","nonce":2301325597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6826","nonce":2301325597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6831","nonce":2301325597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6829","nonce":2301325597}]},"public_addr":"172.21.15.143:6822/2301325597","cluster_addr":"172.21.15.143:6826/2301325597","heartbeat_back_addr":"172.21.15.143:6831/2301325597","heartbeat_front_addr":"172.21.15.143:6829/2301325597","state":["exists","up"]},{"osd":3,"uuid":"4e2783cf-ebb9-4dd8-8245-bc1d17ce4325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6817","nonce":2021504428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6819","nonce":2021504428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6827","nonce":2021504428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6823","nonce":2021504428}]},"public_addr":"172.21.15.143:6817/2021504428","cluster_addr":"172.21.15.143:6819/2021504428","heartbeat_back_addr":"172.21.15.143:6827/2021504428","heartbeat_front_addr":"172.21.15.143:6823/2021504428","state":["exists","up"]},{"osd":4,"uuid":"691256fa-9b77-49b9-a346-7ae1adf0ca80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6801","nonce":1993380595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6803","nonce":1993380595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6807","nonce":1993380595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6805","nonce":1993380595}]},"public_addr":"172.21.15.174:6801/1993380595","cluster_addr":"172.21.15.174:6803/1993380595","heartbeat_back_addr":"172.21.15.174:6807/1993380595","heartbeat_front_addr":"172.21.15.174:6805/1993380595","state":["exists","up"]},{"osd":5,"uuid":"756fc352-48c9-46ce-b52e-0541fe6f8525","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6809","nonce":1777923303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6811","nonce":1777923303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6815","nonce":1777923303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6813","nonce":1777923303}]},"public_addr":"172.21.15.174:6809/1777923303","cluster_addr":"172.21.15.174:6811/1777923303","heartbeat_back_addr":"172.21.15.174:6815/1777923303","heartbeat_front_addr":"172.21.15.174:6813/1777923303","state":["exists","up"]},{"osd":6,"uuid":"fab2ac2b-ed97-4cde-9825-b480eabf8b8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6825","nonce":2755130316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6827","nonce":2755130316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6831","nonce":2755130316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6829","nonce":2755130316}]},"public_addr":"172.21.15.174:6825/2755130316","cluster_addr":"172.21.15.174:6827/2755130316","heartbeat_back_addr":"172.21.15.174:6831/2755130316","heartbeat_front_addr":"172.21.15.174:6829/2755130316","state":["exists","up"]},{"osd":7,"uuid":"3de9b797-2ab6-4d41-a863-7e7d638fd703","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6817","nonce":2847141411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6819","nonce":2847141411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6823","nonce":2847141411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6821","nonce":2847141411}]},"public_addr":"172.21.15.174:6817/2847141411","cluster_addr":"172.21.15.174:6819/2847141411","heartbeat_back_addr":"172.21.15.174:6823/2847141411","heartbeat_front_addr":"172.21.15.174:6821/2847141411","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.477076+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.496890+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.539305+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.454673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.538881+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.521294+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.520712+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.540404+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.35","mappings":[{"from":5,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.174:6834/3601088618":"2023-11-09T19:30:47.736745+0000","172.21.15.143:6834/1535103237":"2023-11-09T19:30:40.630538+0000","172.21.15.143:6832/1535103237":"2023-11-09T19:30:40.630538+0000","172.21.15.143:0/2918728496":"2023-11-08T20:30:55.736484+0000","172.21.15.143:6835/3664584522":"2023-11-09T19:30:12.566205+0000","172.21.15.143:6833/3664584522":"2023-11-09T19:30:12.566205+0000","172.21.15.143:6832/3172664377":"2023-11-09T19:29:41.150862+0000","172.21.15.174:6836/3601088618":"2023-11-09T19:30:47.736745+0000","172.21.15.174:6837/2201447629":"2023-11-09T19:28:48.342407+0000","172.21.15.174:6835/2201447629":"2023-11-09T19:28:48.342407+0000","172.21.15.143:6833/3953059900":"2023-11-09T19:29:14.940251+0000","172.21.15.143:6835/3953059900":"2023-11-09T19:29:14.940251+0000","172.21.15.143:6834/3172664377":"2023-11-09T19:29:41.150862+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:31:10.616 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.615+0000 7fa03fff8700 1 -- 172.21.15.143:0/1214234851 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa01c06af00 msgr2=0x7fa01c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:10.616 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.615+0000 7fa03fff8700 1 --2- 172.21.15.143:0/1214234851 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa01c06af00 0x7fa01c06d3c0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fa028000ef0 tx=0x7fa028009040 comp rx=0 tx=0).stop 2023-11-08T19:31:10.616 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.615+0000 7fa03fff8700 1 -- 172.21.15.143:0/1214234851 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa0380c8360 msgr2=0x7fa03815c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:10.617 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.615+0000 7fa03fff8700 1 --2- 172.21.15.143:0/1214234851 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa0380c8360 0x7fa03815c630 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7fa03400e580 tx=0x7fa03400e940 comp rx=0 tx=0).stop 2023-11-08T19:31:10.617 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.615+0000 7fa03fff8700 1 -- 172.21.15.143:0/1214234851 shutdown_connections 2023-11-08T19:31:10.617 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.615+0000 7fa03fff8700 1 --2- 172.21.15.143:0/1214234851 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa01c06af00 0x7fa01c06d3c0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:10.617 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.615+0000 7fa03fff8700 1 --2- 172.21.15.143:0/1214234851 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa03806fcc0 0x7fa0380b17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:10.618 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.615+0000 7fa03fff8700 1 --2- 172.21.15.143:0/1214234851 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa024005210 0x7fa0381f37a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:10.618 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.615+0000 7fa03fff8700 1 --2- 172.21.15.143:0/1214234851 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa0380c8360 0x7fa03815c630 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:10.618 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.615+0000 7fa03fff8700 1 -- 172.21.15.143:0/1214234851 >> 172.21.15.143:0/1214234851 conn(0x7fa03809aa70 msgr2=0x7fa03806e9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:10.618 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.615+0000 7fa03fff8700 1 -- 172.21.15.143:0/1214234851 shutdown_connections 2023-11-08T19:31:10.618 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.615+0000 7fa03fff8700 1 -- 172.21.15.143:0/1214234851 wait complete. 2023-11-08T19:31:10.628 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-08T19:31:10.750 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.749+0000 7f7a742da700 1 Processor -- start 2023-11-08T19:31:10.751 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.749+0000 7f7a742da700 1 -- start start 2023-11-08T19:31:10.757 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.750+0000 7f7a742da700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7a6c0d00e0 0x7f7a6c15f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:10.757 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.750+0000 7f7a742da700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a6c06f760 con 0x7f7a6c15acd0 2023-11-08T19:31:10.757 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.750+0000 7f7a742da700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a6c06e8e0 con 0x7f7a6c15a8f0 2023-11-08T19:31:10.757 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.750+0000 7f7a742da700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a6c06eb50 con 0x7f7a6c0d00e0 2023-11-08T19:31:10.758 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.750+0000 7f7a71875700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f7a6c15acd0 0x7f7a6c0ccea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:53128/0 (socket says 172.21.15.143:53128) 2023-11-08T19:31:10.758 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.750+0000 7f7a71875700 1 -- 172.21.15.143:0/888766910 learned_addr learned my addr 172.21.15.143:0/888766910 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:31:10.758 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.750+0000 7f7a72877700 1 --2- 172.21.15.143:0/888766910 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7a6c0d00e0 0x7f7a6c15f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:10.758 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.750+0000 7f7a71074700 1 -- 172.21.15.143:0/888766910 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2362937809 0 0) 0x7f7a6c06f760 con 0x7f7a6c15acd0 2023-11-08T19:31:10.759 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.751+0000 7f7a71074700 1 -- 172.21.15.143:0/888766910 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a580035b0 con 0x7f7a6c15acd0 2023-11-08T19:31:10.759 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.751+0000 7f7a71074700 1 -- 172.21.15.143:0/888766910 <== mon.1 v1:172.21.15.174:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2990802520 0 0) 0x7f7a6c06e8e0 con 0x7f7a6c15a8f0 2023-11-08T19:31:10.759 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.751+0000 7f7a71074700 1 -- 172.21.15.143:0/888766910 --> v1:172.21.15.174:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a6c06f760 con 0x7f7a6c15a8f0 2023-11-08T19:31:10.759 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.751+0000 7f7a71074700 1 -- 172.21.15.143:0/888766910 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4159942708 0 0) 0x7f7a580035b0 con 0x7f7a6c15acd0 2023-11-08T19:31:10.759 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.751+0000 7f7a71074700 1 -- 172.21.15.143:0/888766910 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7a6c0d00e0 msgr2=0x7f7a6c15f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:10.759 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.751+0000 7f7a71074700 1 --2- 172.21.15.143:0/888766910 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7a6c0d00e0 0x7f7a6c15f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:10.760 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.751+0000 7f7a71074700 1 -- 172.21.15.143:0/888766910 >> v1:172.21.15.174:6789/0 conn(0x7f7a6c15a8f0 legacy=0x7f7a6c06e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:10.760 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.751+0000 7f7a71074700 1 -- 172.21.15.143:0/888766910 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a6c06ee30 con 0x7f7a6c15acd0 2023-11-08T19:31:10.760 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.751+0000 7f7a71074700 1 -- 172.21.15.143:0/888766910 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4147756303 0 0) 0x7f7a68001670 con 0x7f7a6c15acd0 2023-11-08T19:31:10.760 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.751+0000 7f7a71074700 1 -- 172.21.15.143:0/888766910 >> v1:172.21.15.143:6789/0 conn(0x7f7a6c15acd0 legacy=0x7f7a6c0ccea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:10.760 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.751+0000 7f7a71074700 1 --2- 172.21.15.143:0/888766910 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7a6c0d00e0 0x7f7a58003bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:10.761 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.751+0000 7f7a71074700 1 --2- 172.21.15.143:0/888766910 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a58004170 0x7f7a58004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:10.761 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.751+0000 7f7a71074700 1 --2- 172.21.15.143:0/888766910 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7a58004d90 0x7f7a58005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:10.761 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.751+0000 7f7a71074700 1 -- 172.21.15.143:0/888766910 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a6c06eb50 con 0x7f7a58004170 2023-11-08T19:31:10.761 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.751+0000 7f7a71074700 1 -- 172.21.15.143:0/888766910 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a5800bb50 con 0x7f7a58004d90 2023-11-08T19:31:10.761 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.751+0000 7f7a71074700 1 -- 172.21.15.143:0/888766910 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a5800bd10 con 0x7f7a6c0d00e0 2023-11-08T19:31:10.761 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.751+0000 7f7a71875700 1 --2- 172.21.15.143:0/888766910 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a58004170 0x7f7a58004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:10.762 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.751+0000 7f7a72076700 1 --2- 172.21.15.143:0/888766910 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7a6c0d00e0 0x7f7a58003bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:10.762 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.751+0000 7f7a72877700 1 --2- 172.21.15.143:0/888766910 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7a58004d90 0x7f7a58005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:10.762 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.751+0000 7f7a71875700 1 -- 172.21.15.143:0/888766910 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7a6c0d00e0 msgr2=0x7f7a58003bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:10.762 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.751+0000 7f7a71875700 1 --2- 172.21.15.143:0/888766910 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7a6c0d00e0 0x7f7a58003bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:10.762 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.751+0000 7f7a71875700 1 -- 172.21.15.143:0/888766910 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7a58004d90 msgr2=0x7f7a58005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:10.763 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.751+0000 7f7a71875700 1 --2- 172.21.15.143:0/888766910 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7a58004d90 0x7f7a58005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:10.763 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.751+0000 7f7a71875700 1 -- 172.21.15.143:0/888766910 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a580035b0 con 0x7f7a58004170 2023-11-08T19:31:10.763 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.751+0000 7f7a71875700 1 --2- 172.21.15.143:0/888766910 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a58004170 0x7f7a58004850 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f7a6800c0e0 tx=0x7f7a58005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=799a97a8c24d16d5 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:10.763 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.752+0000 7f7a71074700 1 -- 172.21.15.143:0/888766910 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a6800cbc0 con 0x7f7a58004170 2023-11-08T19:31:10.763 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.752+0000 7f7a71074700 1 -- 172.21.15.143:0/888766910 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a6800cd20 con 0x7f7a58004170 2023-11-08T19:31:10.764 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.752+0000 7f7a71074700 1 -- 172.21.15.143:0/888766910 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a68005ba0 con 0x7f7a58004170 2023-11-08T19:31:10.764 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.752+0000 7f7a742da700 1 -- 172.21.15.143:0/888766910 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a58004170 msgr2=0x7f7a58004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:10.764 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.752+0000 7f7a742da700 1 --2- 172.21.15.143:0/888766910 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a58004170 0x7f7a58004850 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f7a6800c0e0 tx=0x7f7a58005880 comp rx=0 tx=0).stop 2023-11-08T19:31:10.764 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.752+0000 7f7a72877700 1 -- 172.21.15.143:0/888766910 reap_dead start 2023-11-08T19:31:10.764 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.752+0000 7f7a742da700 1 -- 172.21.15.143:0/888766910 shutdown_connections 2023-11-08T19:31:10.765 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.752+0000 7f7a742da700 1 -- 172.21.15.143:0/888766910 >> 172.21.15.143:0/888766910 conn(0x7f7a6c0b25b0 msgr2=0x7f7a6c0b2990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:10.765 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.752+0000 7f7a742da700 1 -- 172.21.15.143:0/888766910 shutdown_connections 2023-11-08T19:31:10.765 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.752+0000 7f7a742da700 1 -- 172.21.15.143:0/888766910 wait complete. 2023-11-08T19:31:10.765 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.753+0000 7f7a742da700 1 Processor -- start 2023-11-08T19:31:10.765 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.753+0000 7f7a742da700 1 -- start start 2023-11-08T19:31:10.765 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.754+0000 7f7a742da700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7a58004170 0x7f7a6c159690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:10.766 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.754+0000 7f7a742da700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a6c0cb510 0x7f7a6c0a94e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:10.766 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.754+0000 7f7a742da700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7a6c0a9a20 0x7f7a6c0b55a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:10.766 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.754+0000 7f7a742da700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a6c0c9e90 con 0x7f7a6c0cb510 2023-11-08T19:31:10.766 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.754+0000 7f7a742da700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a6c0ca190 con 0x7f7a58004170 2023-11-08T19:31:10.766 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.754+0000 7f7a742da700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a6c0ca470 con 0x7f7a6c0a9a20 2023-11-08T19:31:10.766 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.754+0000 7f7a71875700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a6c0cb510 0x7f7a6c0a94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:10.767 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.754+0000 7f7a71875700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a6c0cb510 0x7f7a6c0a94e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56986/0 (socket says 172.21.15.143:56986) 2023-11-08T19:31:10.767 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.754+0000 7f7a71875700 1 -- 172.21.15.143:0/3608231772 learned_addr learned my addr 172.21.15.143:0/3608231772 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:31:10.767 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.754+0000 7f7a72877700 1 --2- 172.21.15.143:0/3608231772 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7a6c0a9a20 0x7f7a6c0b55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:10.767 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.754+0000 7f7a72076700 1 --2- 172.21.15.143:0/3608231772 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7a58004170 0x7f7a6c159690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:10.767 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.755+0000 7f7a71875700 1 -- 172.21.15.143:0/3608231772 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7a6c0a9a20 msgr2=0x7f7a6c0b55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:10.768 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.755+0000 7f7a71875700 1 --2- 172.21.15.143:0/3608231772 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7a6c0a9a20 0x7f7a6c0b55a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:10.768 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.755+0000 7f7a71875700 1 -- 172.21.15.143:0/3608231772 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7a58004170 msgr2=0x7f7a6c159690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:10.768 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.755+0000 7f7a71875700 1 --2- 172.21.15.143:0/3608231772 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7a58004170 0x7f7a6c159690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:10.768 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.755+0000 7f7a71875700 1 -- 172.21.15.143:0/3608231772 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a6c0ca750 con 0x7f7a6c0cb510 2023-11-08T19:31:10.768 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.755+0000 7f7a72877700 1 --2- 172.21.15.143:0/3608231772 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7a6c0a9a20 0x7f7a6c0b55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:31:10.768 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.755+0000 7f7a71875700 1 --2- 172.21.15.143:0/3608231772 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a6c0cb510 0x7f7a6c0a94e0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f7a68001eb0 tx=0x7f7a68001930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:10.769 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.755+0000 7f7a62ffd700 1 -- 172.21.15.143:0/3608231772 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a680072d0 con 0x7f7a6c0cb510 2023-11-08T19:31:10.769 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.755+0000 7f7a62ffd700 1 -- 172.21.15.143:0/3608231772 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a68007430 con 0x7f7a6c0cb510 2023-11-08T19:31:10.769 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.755+0000 7f7a742da700 1 -- 172.21.15.143:0/3608231772 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a6c0caa30 con 0x7f7a6c0cb510 2023-11-08T19:31:10.769 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.755+0000 7f7a62ffd700 1 -- 172.21.15.143:0/3608231772 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a680077e0 con 0x7f7a6c0cb510 2023-11-08T19:31:10.769 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.755+0000 7f7a742da700 1 -- 172.21.15.143:0/3608231772 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a6c0caff0 con 0x7f7a6c0cb510 2023-11-08T19:31:10.770 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.756+0000 7f7a742da700 1 -- 172.21.15.143:0/3608231772 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a6c0cb1b0 con 0x7f7a6c0cb510 2023-11-08T19:31:10.771 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.756+0000 7f7a62ffd700 1 -- 172.21.15.143:0/3608231772 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7a68019030 con 0x7f7a6c0cb510 2023-11-08T19:31:10.771 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.757+0000 7f7a62ffd700 1 --2- 172.21.15.143:0/3608231772 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f7a5006af00 0x7f7a5006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:10.771 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.757+0000 7f7a62ffd700 1 -- 172.21.15.143:0/3608231772 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 7360+0+0 (secure 0 0 0) 0x7f7a6808b040 con 0x7f7a6c0cb510 2023-11-08T19:31:10.771 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.757+0000 7f7a72076700 1 --2- 172.21.15.143:0/3608231772 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f7a5006af00 0x7f7a5006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:10.771 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.757+0000 7f7a72076700 1 --2- 172.21.15.143:0/3608231772 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f7a5006af00 0x7f7a5006d3c0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c007fd0 tx=0x7f7a5c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:10.772 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.761+0000 7f7a62ffd700 1 -- 172.21.15.143:0/3608231772 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7a68055c30 con 0x7f7a6c0cb510 2023-11-08T19:31:10.998 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.996+0000 7f7a742da700 1 -- 172.21.15.143:0/3608231772 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f7a6c04f730 con 0x7f7a6c0cb510 2023-11-08T19:31:10.998 INFO:tasks.ceph.mon.a.smithi143.stderr:2023-11-08T19:31:10.997+0000 7f65b584e700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-08T19:31:10.999 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:10.998+0000 7f7a62ffd700 1 -- 172.21.15.143:0/3608231772 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v101) v1 ==== 161+0+0 (secure 0 0 0) 0x7f7a68059250 con 0x7f7a6c0cb510 2023-11-08T19:31:11.002 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:11.001+0000 7f7a742da700 1 -- 172.21.15.143:0/3608231772 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f7a5006af00 msgr2=0x7f7a5006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:11.002 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:11.001+0000 7f7a742da700 1 --2- 172.21.15.143:0/3608231772 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f7a5006af00 0x7f7a5006d3c0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c007fd0 tx=0x7f7a5c009040 comp rx=0 tx=0).stop 2023-11-08T19:31:11.003 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:11.001+0000 7f7a742da700 1 -- 172.21.15.143:0/3608231772 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a6c0cb510 msgr2=0x7f7a6c0a94e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:11.003 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:11.001+0000 7f7a742da700 1 --2- 172.21.15.143:0/3608231772 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a6c0cb510 0x7f7a6c0a94e0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f7a68001eb0 tx=0x7f7a68001930 comp rx=0 tx=0).stop 2023-11-08T19:31:11.005 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:11.004+0000 7f7a742da700 1 -- 172.21.15.143:0/3608231772 shutdown_connections 2023-11-08T19:31:11.006 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:11.004+0000 7f7a742da700 1 --2- 172.21.15.143:0/3608231772 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f7a5006af00 0x7f7a5006d3c0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:11.006 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:11.004+0000 7f7a742da700 1 --2- 172.21.15.143:0/3608231772 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7a6c0a9a20 0x7f7a6c0b55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:11.006 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:11.004+0000 7f7a742da700 1 --2- 172.21.15.143:0/3608231772 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7a58004170 0x7f7a6c159690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:11.006 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:11.004+0000 7f7a742da700 1 --2- 172.21.15.143:0/3608231772 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a6c0cb510 0x7f7a6c0a94e0 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:11.006 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:11.004+0000 7f7a742da700 1 -- 172.21.15.143:0/3608231772 >> 172.21.15.143:0/3608231772 conn(0x7f7a6c0b25b0 msgr2=0x7f7a6c15b0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:11.007 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:11.004+0000 7f7a742da700 1 -- 172.21.15.143:0/3608231772 shutdown_connections 2023-11-08T19:31:11.007 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:11.004+0000 7f7a742da700 1 -- 172.21.15.143:0/3608231772 wait complete. 2023-11-08T19:31:11.007 INFO:teuthology.orchestra.run.smithi143.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-08T19:31:11.008 INFO:tasks.ceph.mds.a.smithi143.stderr: -636> 2023-11-08T19:30:58.587+0000 7f257a6f9700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-08T19:31:11.008 INFO:tasks.ceph.mds.a.smithi143.stderr: -634> 2023-11-08T19:30:58.587+0000 7f257a6f9700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-11-08T19:31:11.023 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-08T19:31:11.029 INFO:tasks.ceph.mds.c.smithi143.stdout:starting mds.c at 2023-11-08T19:31:11.036 INFO:tasks.ceph.mds.a.smithi143.stdout:starting mds.a at 2023-11-08T19:31:11.154 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:11.147+0000 7fbd5ab95700 1 Processor -- start 2023-11-08T19:31:11.155 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:11.147+0000 7fbd5ab95700 1 -- start start 2023-11-08T19:31:11.155 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:11.148+0000 7fbd5ab95700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fbd5406abe0 0x7fbd540b2770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:11.155 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:11.148+0000 7fbd5ab95700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fbd5406e0f0 0x7fbd5406e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:11.155 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:11.148+0000 7fbd5ab95700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd540724b0 con 0x7fbd5406afc0 2023-11-08T19:31:11.155 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:11.148+0000 7fbd5ab95700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd54072a70 con 0x7fbd5406abe0 2023-11-08T19:31:11.156 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:11.148+0000 7fbd5ab95700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd5406eac0 con 0x7fbd5406e0f0 2023-11-08T19:31:11.156 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:11.148+0000 7fbd58931700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fbd5406abe0 0x7fbd540b2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:11.156 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:11.148+0000 7fbd53fff700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fbd5406afc0 0x7fbd540cbe30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:53172/0 (socket says 172.21.15.143:53172) 2023-11-08T19:31:11.156 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:11.148+0000 7fbd58931700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fbd5406abe0 0x7fbd540b2770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57016/0 (socket says 172.21.15.143:57016) 2023-11-08T19:31:11.156 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:11.148+0000 7fbd53fff700 1 -- 172.21.15.143:0/123882678 learned_addr learned my addr 172.21.15.143:0/123882678 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:31:11.157 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:11.148+0000 7fbd59132700 1 --2- 172.21.15.143:0/123882678 >> v2:172.21.15.174:3300/0 conn(0x7fbd5406e0f0 0x7fbd5406e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:11.157 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:11.148+0000 7fbd58931700 1 -- 172.21.15.143:0/123882678 >> v2:172.21.15.174:3300/0 conn(0x7fbd5406e0f0 msgr2=0x7fbd5406e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:11.157 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:11.148+0000 7fbd58931700 1 --2- 172.21.15.143:0/123882678 >> v2:172.21.15.174:3300/0 conn(0x7fbd5406e0f0 0x7fbd5406e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:11.157 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:11.148+0000 7fbd58931700 1 -- 172.21.15.143:0/123882678 >> v1:172.21.15.143:6789/0 conn(0x7fbd5406afc0 legacy=0x7fbd540cbe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:11.157 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:11.148+0000 7fbd58931700 1 -- 172.21.15.143:0/123882678 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd54072790 con 0x7fbd5406abe0 2023-11-08T19:31:11.158 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:11.148+0000 7fbd537fe700 1 -- 172.21.15.143:0/123882678 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4196679434 0 0) 0x7fbd540724b0 con 0x7fbd5406afc0 2023-11-08T19:31:11.158 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:11.148+0000 7fbd59132700 1 --2- 172.21.15.143:0/123882678 >> v2:172.21.15.174:3300/0 conn(0x7fbd5406e0f0 0x7fbd5406e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:31:11.158 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:11.148+0000 7fbd58931700 1 --2- 172.21.15.143:0/123882678 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbd5406abe0 0x7fbd540b2770 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7fbd54162820 tx=0x7fbd48009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f0e9b9b73dbfc25 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:11.158 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:11.148+0000 7fbd537fe700 1 -- 172.21.15.143:0/123882678 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd48011830 con 0x7fbd5406abe0 2023-11-08T19:31:11.158 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:11.148+0000 7fbd537fe700 1 -- 172.21.15.143:0/123882678 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd48011990 con 0x7fbd5406abe0 2023-11-08T19:31:11.159 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:11.148+0000 7fbd537fe700 1 -- 172.21.15.143:0/123882678 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd48011d20 con 0x7fbd5406abe0 2023-11-08T19:31:11.159 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:11.149+0000 7fbd5ab95700 1 -- 172.21.15.143:0/123882678 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbd5406abe0 msgr2=0x7fbd540b2770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:11.159 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:11.149+0000 7fbd5ab95700 1 --2- 172.21.15.143:0/123882678 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbd5406abe0 0x7fbd540b2770 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7fbd54162820 tx=0x7fbd48009b20 comp rx=0 tx=0).stop 2023-11-08T19:31:11.159 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:11.149+0000 7fbd5ab95700 1 -- 172.21.15.143:0/123882678 shutdown_connections 2023-11-08T19:31:11.159 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:11.149+0000 7fbd5ab95700 1 --2- 172.21.15.143:0/123882678 >> v2:172.21.15.174:3300/0 conn(0x7fbd5406e0f0 0x7fbd5406e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:11.160 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:11.149+0000 7fbd5ab95700 1 --2- 172.21.15.143:0/123882678 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbd5406abe0 0x7fbd540b2770 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:11.160 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:11.149+0000 7fbd5ab95700 1 -- 172.21.15.143:0/123882678 >> 172.21.15.143:0/123882678 conn(0x7fbd5406bcf0 msgr2=0x7fbd5406c100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:11.160 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:11.149+0000 7fbd5ab95700 1 -- 172.21.15.143:0/123882678 shutdown_connections 2023-11-08T19:31:11.160 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:11.149+0000 7fbd5ab95700 1 -- 172.21.15.143:0/123882678 wait complete. 2023-11-08T19:31:11.161 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:11.150+0000 7fbd5ab95700 1 Processor -- start 2023-11-08T19:31:11.161 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:11.150+0000 7fbd5ab95700 1 -- start start 2023-11-08T19:31:11.161 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:11.151+0000 7fbd5ab95700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbd5406abe0 0x7fbd5415d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:11.161 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:11.151+0000 7fbd5ab95700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbd5406afc0 0x7fbd541f2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:11.161 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:11.151+0000 7fbd5ab95700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbd5406e0f0 0x7fbd54160070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:11.162 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:11.151+0000 7fbd5ab95700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd54155ea0 con 0x7fbd5406afc0 2023-11-08T19:31:11.162 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:11.151+0000 7fbd5ab95700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd54155360 con 0x7fbd5406abe0 2023-11-08T19:31:11.162 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:11.151+0000 7fbd5ab95700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd54155600 con 0x7fbd5406e0f0 2023-11-08T19:31:11.162 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:11.151+0000 7fbd53fff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbd5406afc0 0x7fbd541f2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:11.162 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:11.151+0000 7fbd53fff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbd5406afc0 0x7fbd541f2b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57024/0 (socket says 172.21.15.143:57024) 2023-11-08T19:31:11.163 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:11.151+0000 7fbd53fff700 1 -- 172.21.15.143:0/3660103 learned_addr learned my addr 172.21.15.143:0/3660103 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:31:11.163 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:11.151+0000 7fbd59132700 1 --2- 172.21.15.143:0/3660103 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbd5406e0f0 0x7fbd54160070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:11.163 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:11.151+0000 7fbd53fff700 1 -- 172.21.15.143:0/3660103 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbd5406e0f0 msgr2=0x7fbd54160070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:11.163 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:11.151+0000 7fbd53fff700 1 --2- 172.21.15.143:0/3660103 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbd5406e0f0 0x7fbd54160070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:11.163 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:11.151+0000 7fbd53fff700 1 -- 172.21.15.143:0/3660103 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbd5406abe0 msgr2=0x7fbd5415d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:11.163 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:11.151+0000 7fbd53fff700 1 --2- 172.21.15.143:0/3660103 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbd5406abe0 0x7fbd5415d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:11.164 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:11.151+0000 7fbd53fff700 1 -- 172.21.15.143:0/3660103 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd480094a0 con 0x7fbd5406afc0 2023-11-08T19:31:11.164 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:11.152+0000 7fbd53fff700 1 --2- 172.21.15.143:0/3660103 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbd5406afc0 0x7fbd541f2b80 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7fbd400011e0 tx=0x7fbd4000cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:11.164 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:11.152+0000 7fbd517fa700 1 -- 172.21.15.143:0/3660103 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd40004c80 con 0x7fbd5406afc0 2023-11-08T19:31:11.164 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:11.152+0000 7fbd517fa700 1 -- 172.21.15.143:0/3660103 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd40004e00 con 0x7fbd5406afc0 2023-11-08T19:31:11.164 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:11.152+0000 7fbd5ab95700 1 -- 172.21.15.143:0/3660103 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd541558e0 con 0x7fbd5406afc0 2023-11-08T19:31:11.165 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:11.152+0000 7fbd517fa700 1 -- 172.21.15.143:0/3660103 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd40007310 con 0x7fbd5406afc0 2023-11-08T19:31:11.165 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:11.152+0000 7fbd5ab95700 1 -- 172.21.15.143:0/3660103 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd54152d20 con 0x7fbd5406afc0 2023-11-08T19:31:11.165 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:11.153+0000 7fbd5ab95700 1 -- 172.21.15.143:0/3660103 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd5404f730 con 0x7fbd5406afc0 2023-11-08T19:31:11.165 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:11.153+0000 7fbd517fa700 1 -- 172.21.15.143:0/3660103 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbd40007470 con 0x7fbd5406afc0 2023-11-08T19:31:11.166 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:11.154+0000 7fbd517fa700 1 --2- 172.21.15.143:0/3660103 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fbd3806af00 0x7fbd3806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:11.166 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:11.154+0000 7fbd517fa700 1 -- 172.21.15.143:0/3660103 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 7532+0+0 (secure 0 0 0) 0x7fbd40089b20 con 0x7fbd5406afc0 2023-11-08T19:31:11.166 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:11.154+0000 7fbd58931700 1 --2- 172.21.15.143:0/3660103 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fbd3806af00 0x7fbd3806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:11.167 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:11.154+0000 7fbd58931700 1 --2- 172.21.15.143:0/3660103 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fbd3806af00 0x7fbd3806d3c0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fbd48000c00 tx=0x7fbd48010000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:11.167 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:11.158+0000 7fbd517fa700 1 -- 172.21.15.143:0/3660103 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbd400544a0 con 0x7fbd5406afc0 2023-11-08T19:31:11.389 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:11.387+0000 7fbd5ab95700 1 -- 172.21.15.143:0/3660103 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fbd54150a10 con 0x7fbd5406afc0 2023-11-08T19:31:12.001 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:12.000+0000 7fbd517fa700 1 -- 172.21.15.143:0/3660103 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v102) v1 ==== 104+0+0 (secure 0 0 0) 0x7fbd40057ac0 con 0x7fbd5406afc0 2023-11-08T19:31:12.004 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:12.002+0000 7fbd5ab95700 1 -- 172.21.15.143:0/3660103 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fbd3806af00 msgr2=0x7fbd3806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:12.004 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:12.002+0000 7fbd5ab95700 1 --2- 172.21.15.143:0/3660103 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fbd3806af00 0x7fbd3806d3c0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fbd48000c00 tx=0x7fbd48010000 comp rx=0 tx=0).stop 2023-11-08T19:31:12.004 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:12.002+0000 7fbd5ab95700 1 -- 172.21.15.143:0/3660103 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbd5406afc0 msgr2=0x7fbd541f2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:12.004 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:12.002+0000 7fbd5ab95700 1 --2- 172.21.15.143:0/3660103 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbd5406afc0 0x7fbd541f2b80 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7fbd400011e0 tx=0x7fbd4000cae0 comp rx=0 tx=0).stop 2023-11-08T19:31:12.005 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:12.003+0000 7fbd5ab95700 1 -- 172.21.15.143:0/3660103 shutdown_connections 2023-11-08T19:31:12.005 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:12.003+0000 7fbd5ab95700 1 --2- 172.21.15.143:0/3660103 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fbd3806af00 0x7fbd3806d3c0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:12.005 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:12.003+0000 7fbd5ab95700 1 --2- 172.21.15.143:0/3660103 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbd5406e0f0 0x7fbd54160070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:12.005 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:12.003+0000 7fbd5ab95700 1 --2- 172.21.15.143:0/3660103 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbd5406abe0 0x7fbd5415d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:12.005 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:12.003+0000 7fbd5ab95700 1 --2- 172.21.15.143:0/3660103 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbd5406afc0 0x7fbd541f2b80 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:12.006 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:12.003+0000 7fbd5ab95700 1 -- 172.21.15.143:0/3660103 >> 172.21.15.143:0/3660103 conn(0x7fbd5406bcf0 msgr2=0x7fbd540b5670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:12.006 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:12.003+0000 7fbd5ab95700 1 -- 172.21.15.143:0/3660103 shutdown_connections 2023-11-08T19:31:12.006 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:12.003+0000 7fbd5ab95700 1 -- 172.21.15.143:0/3660103 wait complete. 2023-11-08T19:31:12.019 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-08T19:31:12.187 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:12.186+0000 7f1ed137a700 1 Processor -- start 2023-11-08T19:31:12.188 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:12.186+0000 7f1ed137a700 1 -- start start 2023-11-08T19:31:12.191 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:12.187+0000 7f1ed137a700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f1ecc0b7250 0x7f1ecc0d2460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:12.191 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:12.187+0000 7f1ed137a700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ecc0b4830 con 0x7f1ecc0d0630 2023-11-08T19:31:12.192 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:12.187+0000 7f1ed137a700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ecc0b4ad0 con 0x7f1ecc0cf720 2023-11-08T19:31:12.192 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:12.187+0000 7f1ed137a700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ecc0b4db0 con 0x7f1ecc0b7250 2023-11-08T19:31:12.192 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:12.187+0000 7f1ecb7fe700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f1ecc0d0630 0x7f1ecc0b55b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:53192/0 (socket says 172.21.15.143:53192) 2023-11-08T19:31:12.192 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:12.187+0000 7f1ecb7fe700 1 -- 172.21.15.143:0/467068545 learned_addr learned my addr 172.21.15.143:0/467068545 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:31:12.193 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:12.187+0000 7f1ecaffd700 1 --2- 172.21.15.143:0/467068545 >> v2:172.21.15.174:3300/0 conn(0x7f1ecc0b7250 0x7f1ecc0d2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:12.193 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:12.187+0000 7f1ec9ffb700 1 -- 172.21.15.143:0/467068545 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1728500523 0 0) 0x7f1ecc0b4830 con 0x7f1ecc0d0630 2023-11-08T19:31:12.193 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:12.187+0000 7f1ec9ffb700 1 -- 172.21.15.143:0/467068545 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1eb40035b0 con 0x7f1ecc0d0630 2023-11-08T19:31:12.193 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:12.187+0000 7f1ec9ffb700 1 -- 172.21.15.143:0/467068545 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1621615986 0 0) 0x7f1eb40035b0 con 0x7f1ecc0d0630 2023-11-08T19:31:12.193 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:12.187+0000 7f1ec9ffb700 1 -- 172.21.15.143:0/467068545 >> v2:172.21.15.174:3300/0 conn(0x7f1ecc0b7250 msgr2=0x7f1ecc0d2460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:12.194 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:12.187+0000 7f1ec9ffb700 1 --2- 172.21.15.143:0/467068545 >> v2:172.21.15.174:3300/0 conn(0x7f1ecc0b7250 0x7f1ecc0d2460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:12.194 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:12.187+0000 7f1ec9ffb700 1 -- 172.21.15.143:0/467068545 >> v1:172.21.15.174:6789/0 conn(0x7f1ecc0cf720 legacy=0x7f1ecc0cff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:12.194 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:12.187+0000 7f1ec9ffb700 1 -- 172.21.15.143:0/467068545 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ecc0b5090 con 0x7f1ecc0d0630 2023-11-08T19:31:12.194 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:12.187+0000 7f1ec9ffb700 1 -- 172.21.15.143:0/467068545 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4147756303 0 0) 0x7f1ec40019f0 con 0x7f1ecc0d0630 2023-11-08T19:31:12.194 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:12.187+0000 7f1ec9ffb700 1 -- 172.21.15.143:0/467068545 >> v1:172.21.15.143:6789/0 conn(0x7f1ecc0d0630 legacy=0x7f1ecc0b55b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:12.195 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:12.187+0000 7f1ec9ffb700 1 --2- 172.21.15.143:0/467068545 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1eb4003a00 0x7f1eb4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:12.195 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:12.187+0000 7f1ec9ffb700 1 --2- 172.21.15.143:0/467068545 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1eb4004850 0x7f1eb4004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:12.195 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:12.187+0000 7f1ec9ffb700 1 --2- 172.21.15.143:0/467068545 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1eb4005210 0x7f1eb400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:12.195 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:12.187+0000 7f1ec9ffb700 1 -- 172.21.15.143:0/467068545 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ecc0b4db0 con 0x7f1eb4005210 2023-11-08T19:31:12.195 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:12.187+0000 7f1ec9ffb700 1 -- 172.21.15.143:0/467068545 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb400c0b0 con 0x7f1eb4004850 2023-11-08T19:31:12.196 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:12.187+0000 7f1ec9ffb700 1 -- 172.21.15.143:0/467068545 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb400c270 con 0x7f1eb4003a00 2023-11-08T19:31:12.196 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:12.187+0000 7f1ecb7fe700 1 --2- 172.21.15.143:0/467068545 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1eb4005210 0x7f1eb400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:12.196 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:12.188+0000 7f1eca7fc700 1 --2- 172.21.15.143:0/467068545 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1eb4004850 0x7f1eb4004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:12.196 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:12.188+0000 7f1ecaffd700 1 --2- 172.21.15.143:0/467068545 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1eb4003a00 0x7f1eb4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:12.196 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:12.188+0000 7f1ecb7fe700 1 -- 172.21.15.143:0/467068545 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1eb4003a00 msgr2=0x7f1eb4003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:12.197 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:12.188+0000 7f1ecb7fe700 1 --2- 172.21.15.143:0/467068545 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1eb4003a00 0x7f1eb4003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:12.197 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:12.188+0000 7f1ecb7fe700 1 -- 172.21.15.143:0/467068545 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1eb4004850 msgr2=0x7f1eb4004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:12.197 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:12.188+0000 7f1ecb7fe700 1 --2- 172.21.15.143:0/467068545 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1eb4004850 0x7f1eb4004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:12.197 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:12.188+0000 7f1ecb7fe700 1 -- 172.21.15.143:0/467068545 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1eb40035b0 con 0x7f1eb4005210 2023-11-08T19:31:12.197 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:12.188+0000 7f1ecb7fe700 1 -- 172.21.15.143:0/467068545 reap_dead start 2023-11-08T19:31:12.198 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:12.188+0000 7f1ecb7fe700 1 --2- 172.21.15.143:0/467068545 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1eb4005210 0x7f1eb400ba10 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f1ec400a830 tx=0x7f1eb4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff6be6420c086edc server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:12.198 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:12.188+0000 7f1ec9ffb700 1 -- 172.21.15.143:0/467068545 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ec400a2f0 con 0x7f1eb4005210 2023-11-08T19:31:12.198 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:12.188+0000 7f1ec9ffb700 1 -- 172.21.15.143:0/467068545 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ec4012620 con 0x7f1eb4005210 2023-11-08T19:31:12.198 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:12.188+0000 7f1ec9ffb700 1 -- 172.21.15.143:0/467068545 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ec40129d0 con 0x7f1eb4005210 2023-11-08T19:31:12.198 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:12.189+0000 7f1ed137a700 1 -- 172.21.15.143:0/467068545 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1eb4005210 msgr2=0x7f1eb400ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:12.199 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:12.189+0000 7f1ed137a700 1 --2- 172.21.15.143:0/467068545 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1eb4005210 0x7f1eb400ba10 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f1ec400a830 tx=0x7f1eb4005880 comp rx=0 tx=0).stop 2023-11-08T19:31:12.199 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:12.189+0000 7f1ed137a700 1 -- 172.21.15.143:0/467068545 shutdown_connections 2023-11-08T19:31:12.199 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:12.189+0000 7f1ed137a700 1 --2- 172.21.15.143:0/467068545 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1eb4005210 0x7f1eb400ba10 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:12.199 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:12.189+0000 7f1ed137a700 1 -- 172.21.15.143:0/467068545 >> 172.21.15.143:0/467068545 conn(0x7f1ecc0d1940 msgr2=0x7f1ecc0d1d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:12.199 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:12.189+0000 7f1ed137a700 1 -- 172.21.15.143:0/467068545 shutdown_connections 2023-11-08T19:31:12.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:12.189+0000 7f1ed137a700 1 -- 172.21.15.143:0/467068545 wait complete. 2023-11-08T19:31:12.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:12.190+0000 7f1ed137a700 1 Processor -- start 2023-11-08T19:31:12.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:12.190+0000 7f1ed137a700 1 -- start start 2023-11-08T19:31:12.203 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:12.191+0000 7f1ed137a700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1eb4005210 0x7f1ecc1f62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:12.203 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:12.191+0000 7f1ed137a700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1ecc1f4590 0x7f1ecc0b2940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:12.203 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:12.191+0000 7f1ed137a700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1ecc0b2e80 0x7f1ecc1f33d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:12.203 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:12.191+0000 7f1ed137a700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ecc0b3330 con 0x7f1ecc0b2e80 2023-11-08T19:31:12.204 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:12.191+0000 7f1ed137a700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ecc06c3b0 con 0x7f1eb4005210 2023-11-08T19:31:12.204 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:12.191+0000 7f1ed137a700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ecc1f4170 con 0x7f1ecc1f4590 2023-11-08T19:31:12.204 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:12.191+0000 7f1ecb7fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1ecc0b2e80 0x7f1ecc1f33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:12.204 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:12.191+0000 7f1ecb7fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1ecc0b2e80 0x7f1ecc1f33d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57050/0 (socket says 172.21.15.143:57050) 2023-11-08T19:31:12.204 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:12.191+0000 7f1ecb7fe700 1 -- 172.21.15.143:0/3980180268 learned_addr learned my addr 172.21.15.143:0/3980180268 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:31:12.205 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:12.191+0000 7f1eca7fc700 1 --2- 172.21.15.143:0/3980180268 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1ecc1f4590 0x7f1ecc0b2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:12.205 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:12.191+0000 7f1ecaffd700 1 --2- 172.21.15.143:0/3980180268 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1eb4005210 0x7f1ecc1f62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:12.205 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:12.191+0000 7f1ecb7fe700 1 -- 172.21.15.143:0/3980180268 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1ecc1f4590 msgr2=0x7f1ecc0b2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:12.205 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:12.191+0000 7f1ecb7fe700 1 --2- 172.21.15.143:0/3980180268 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1ecc1f4590 0x7f1ecc0b2940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:12.205 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:12.191+0000 7f1ecb7fe700 1 -- 172.21.15.143:0/3980180268 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1eb4005210 msgr2=0x7f1ecc1f62e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:12.206 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:12.191+0000 7f1ecb7fe700 1 --2- 172.21.15.143:0/3980180268 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1eb4005210 0x7f1ecc1f62e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:12.206 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:12.191+0000 7f1ecb7fe700 1 -- 172.21.15.143:0/3980180268 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ecc06c670 con 0x7f1ecc0b2e80 2023-11-08T19:31:12.206 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:12.191+0000 7f1ecb7fe700 1 --2- 172.21.15.143:0/3980180268 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1ecc0b2e80 0x7f1ecc1f33d0 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f1ecc0c89a0 tx=0x7f1ec4001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:12.206 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:12.191+0000 7f1eb37fe700 1 -- 172.21.15.143:0/3980180268 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ec4001250 con 0x7f1ecc0b2e80 2023-11-08T19:31:12.206 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:12.191+0000 7f1eb37fe700 1 -- 172.21.15.143:0/3980180268 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ec40017b0 con 0x7f1ecc0b2e80 2023-11-08T19:31:12.207 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:12.191+0000 7f1ed137a700 1 -- 172.21.15.143:0/3980180268 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ecc06c950 con 0x7f1ecc0b2e80 2023-11-08T19:31:12.207 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:12.191+0000 7f1eb37fe700 1 -- 172.21.15.143:0/3980180268 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ec401d420 con 0x7f1ecc0b2e80 2023-11-08T19:31:12.207 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:12.192+0000 7f1ed137a700 1 -- 172.21.15.143:0/3980180268 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ecc1f3910 con 0x7f1ecc0b2e80 2023-11-08T19:31:12.207 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:12.193+0000 7f1ed137a700 1 -- 172.21.15.143:0/3980180268 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ecc0cb880 con 0x7f1ecc0b2e80 2023-11-08T19:31:12.207 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:12.193+0000 7f1eb37fe700 1 -- 172.21.15.143:0/3980180268 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1ec401d7b0 con 0x7f1ecc0b2e80 2023-11-08T19:31:12.208 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:12.193+0000 7f1eb37fe700 1 --2- 172.21.15.143:0/3980180268 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f1eb806af00 0x7f1eb806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:12.208 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:12.193+0000 7f1eb37fe700 1 -- 172.21.15.143:0/3980180268 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 7532+0+0 (secure 0 0 0) 0x7f1ec408be70 con 0x7f1ecc0b2e80 2023-11-08T19:31:12.208 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:12.193+0000 7f1ecaffd700 1 --2- 172.21.15.143:0/3980180268 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f1eb806af00 0x7f1eb806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:12.208 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:12.194+0000 7f1ecaffd700 1 --2- 172.21.15.143:0/3980180268 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f1eb806af00 0x7f1eb806d3c0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc009be0 tx=0x7f1ebc009210 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:12.208 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:12.197+0000 7f1eb37fe700 1 -- 172.21.15.143:0/3980180268 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1ec4056810 con 0x7f1ecc0b2e80 2023-11-08T19:31:12.386 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:12.385+0000 7f1ed137a700 1 -- 172.21.15.143:0/3980180268 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f1ecc06cb70 con 0x7f1ecc0b2e80 2023-11-08T19:31:13.005 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:13.004+0000 7f1eb37fe700 1 -- 172.21.15.143:0/3980180268 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v106) v1 ==== 181+0+0 (secure 0 0 0) 0x7f1ec4059e30 con 0x7f1ecc0b2e80 2023-11-08T19:31:13.009 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:13.006+0000 7f1ed137a700 1 -- 172.21.15.143:0/3980180268 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f1eb806af00 msgr2=0x7f1eb806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:13.009 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:13.006+0000 7f1ed137a700 1 --2- 172.21.15.143:0/3980180268 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f1eb806af00 0x7f1eb806d3c0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc009be0 tx=0x7f1ebc009210 comp rx=0 tx=0).stop 2023-11-08T19:31:13.010 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:13.006+0000 7f1ed137a700 1 -- 172.21.15.143:0/3980180268 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1ecc0b2e80 msgr2=0x7f1ecc1f33d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:13.010 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:13.006+0000 7f1ed137a700 1 --2- 172.21.15.143:0/3980180268 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1ecc0b2e80 0x7f1ecc1f33d0 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f1ecc0c89a0 tx=0x7f1ec4001d80 comp rx=0 tx=0).stop 2023-11-08T19:31:13.010 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:13.007+0000 7f1ed137a700 1 -- 172.21.15.143:0/3980180268 shutdown_connections 2023-11-08T19:31:13.011 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:13.007+0000 7f1ed137a700 1 --2- 172.21.15.143:0/3980180268 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f1eb806af00 0x7f1eb806d3c0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:13.011 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:13.007+0000 7f1ed137a700 1 --2- 172.21.15.143:0/3980180268 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1ecc0b2e80 0x7f1ecc1f33d0 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:13.011 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:13.007+0000 7f1ed137a700 1 --2- 172.21.15.143:0/3980180268 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1eb4005210 0x7f1ecc1f62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:13.012 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:13.007+0000 7f1ed137a700 1 --2- 172.21.15.143:0/3980180268 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1ecc1f4590 0x7f1ecc0b2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:13.012 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:13.007+0000 7f1ed137a700 1 -- 172.21.15.143:0/3980180268 >> 172.21.15.143:0/3980180268 conn(0x7f1ecc0d1940 msgr2=0x7f1ecc15eda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:13.012 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:13.007+0000 7f1ed137a700 1 -- 172.21.15.143:0/3980180268 shutdown_connections 2023-11-08T19:31:13.013 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:13.007+0000 7f1ed137a700 1 -- 172.21.15.143:0/3980180268 wait complete. 2023-11-08T19:31:13.013 INFO:teuthology.orchestra.run.smithi143.stderr:pool 'cephfs_metadata' removed 2023-11-08T19:31:13.024 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-08T19:31:13.154 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:13.152+0000 7f4ac34b3700 1 Processor -- start 2023-11-08T19:31:13.154 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:13.153+0000 7f4ac34b3700 1 -- start start 2023-11-08T19:31:13.157 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:13.153+0000 7f4ac34b3700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f4abc0b5590 0x7f4abc0b59d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:13.157 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:13.153+0000 7f4ac34b3700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4abc0d1f10 0x7f4abc0d23a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:13.157 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:13.153+0000 7f4ac34b3700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4abc0b5f10 con 0x7f4abc15de20 2023-11-08T19:31:13.157 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:13.153+0000 7f4ac34b3700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4abc15e200 con 0x7f4abc0b5590 2023-11-08T19:31:13.157 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:13.153+0000 7f4ac34b3700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4abc1627a0 con 0x7f4abc0d1f10 2023-11-08T19:31:13.158 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:13.154+0000 7f4ac0a4e700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f4abc0b5590 0x7f4abc0b59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:13.158 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:13.154+0000 7f4ac124f700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f4abc15de20 0x7f4abc0b9370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:53208/0 (socket says 172.21.15.143:53208) 2023-11-08T19:31:13.158 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:13.154+0000 7f4ac124f700 1 -- 172.21.15.143:0/2839226921 learned_addr learned my addr 172.21.15.143:0/2839226921 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:31:13.158 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:13.154+0000 7f4ac0a4e700 1 -- 172.21.15.143:0/2839226921 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4abc0d1f10 msgr2=0x7f4abc0d23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:13.159 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:13.154+0000 7f4ac0a4e700 1 --2- 172.21.15.143:0/2839226921 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4abc0d1f10 0x7f4abc0d23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:13.159 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:13.154+0000 7f4ac0a4e700 1 -- 172.21.15.143:0/2839226921 >> v1:172.21.15.143:6789/0 conn(0x7f4abc15de20 legacy=0x7f4abc0b9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:13.159 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:13.154+0000 7f4ac0a4e700 1 -- 172.21.15.143:0/2839226921 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4abc0b9a80 con 0x7f4abc0b5590 2023-11-08T19:31:13.159 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:13.154+0000 7f4ac0a4e700 1 --2- 172.21.15.143:0/2839226921 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4abc0b5590 0x7f4abc0b59d0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f4aac009840 tx=0x7f4aac009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbf0a00bf5773570 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:13.159 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:13.154+0000 7f4ab3fff700 1 -- 172.21.15.143:0/2839226921 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aac011830 con 0x7f4abc0b5590 2023-11-08T19:31:13.160 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:13.154+0000 7f4ab3fff700 1 -- 172.21.15.143:0/2839226921 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4aac011990 con 0x7f4abc0b5590 2023-11-08T19:31:13.160 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:13.154+0000 7f4ab3fff700 1 -- 172.21.15.143:0/2839226921 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aac011d20 con 0x7f4abc0b5590 2023-11-08T19:31:13.160 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:13.155+0000 7f4ac34b3700 1 -- 172.21.15.143:0/2839226921 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4abc0b5590 msgr2=0x7f4abc0b59d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:13.160 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:13.155+0000 7f4ac34b3700 1 --2- 172.21.15.143:0/2839226921 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4abc0b5590 0x7f4abc0b59d0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f4aac009840 tx=0x7f4aac009b50 comp rx=0 tx=0).stop 2023-11-08T19:31:13.161 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:13.155+0000 7f4ac34b3700 1 -- 172.21.15.143:0/2839226921 shutdown_connections 2023-11-08T19:31:13.161 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:13.155+0000 7f4ac34b3700 1 --2- 172.21.15.143:0/2839226921 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4abc0d1f10 0x7f4abc0d23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:13.161 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:13.155+0000 7f4ac34b3700 1 --2- 172.21.15.143:0/2839226921 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4abc0b5590 0x7f4abc0b59d0 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:13.161 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:13.155+0000 7f4ac34b3700 1 -- 172.21.15.143:0/2839226921 >> 172.21.15.143:0/2839226921 conn(0x7f4abc0d0590 msgr2=0x7f4abc0d09a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:13.161 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:13.155+0000 7f4ac34b3700 1 -- 172.21.15.143:0/2839226921 shutdown_connections 2023-11-08T19:31:13.162 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:13.155+0000 7f4ac34b3700 1 -- 172.21.15.143:0/2839226921 wait complete. 2023-11-08T19:31:13.165 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:13.156+0000 7f4ac34b3700 1 Processor -- start 2023-11-08T19:31:13.165 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:13.156+0000 7f4ac34b3700 1 -- start start 2023-11-08T19:31:13.165 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:13.157+0000 7f4ac34b3700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4abc0b5590 0x7f4abc1549e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:13.166 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:13.157+0000 7f4ac34b3700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4abc0d1f10 0x7f4abc0cc3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:13.166 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:13.157+0000 7f4ac34b3700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4abc15de20 0x7f4abc15e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:13.166 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:13.157+0000 7f4ac34b3700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4abc0c8480 con 0x7f4abc15de20 2023-11-08T19:31:13.166 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:13.157+0000 7f4ac34b3700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4abc0c7590 con 0x7f4abc0d1f10 2023-11-08T19:31:13.166 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:13.157+0000 7f4ac34b3700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4abc0c7870 con 0x7f4abc0b5590 2023-11-08T19:31:13.167 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:13.157+0000 7f4ac1a50700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4abc15de20 0x7f4abc15e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:13.167 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:13.157+0000 7f4ac1a50700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4abc15de20 0x7f4abc15e4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57064/0 (socket says 172.21.15.143:57064) 2023-11-08T19:31:13.167 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:13.157+0000 7f4ac1a50700 1 -- 172.21.15.143:0/3016346648 learned_addr learned my addr 172.21.15.143:0/3016346648 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:31:13.167 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:13.157+0000 7f4ac124f700 1 --2- 172.21.15.143:0/3016346648 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4abc0b5590 0x7f4abc1549e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:13.167 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:13.157+0000 7f4ac0a4e700 1 --2- 172.21.15.143:0/3016346648 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4abc0d1f10 0x7f4abc0cc3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:13.168 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:13.157+0000 7f4ac1a50700 1 -- 172.21.15.143:0/3016346648 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4abc0b5590 msgr2=0x7f4abc1549e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:13.168 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:13.157+0000 7f4ac1a50700 1 --2- 172.21.15.143:0/3016346648 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4abc0b5590 0x7f4abc1549e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:13.168 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:13.157+0000 7f4ac1a50700 1 -- 172.21.15.143:0/3016346648 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4abc0d1f10 msgr2=0x7f4abc0cc3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:13.168 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:13.157+0000 7f4ac1a50700 1 --2- 172.21.15.143:0/3016346648 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4abc0d1f10 0x7f4abc0cc3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:13.168 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:13.157+0000 7f4ac1a50700 1 -- 172.21.15.143:0/3016346648 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4aac0094a0 con 0x7f4abc15de20 2023-11-08T19:31:13.169 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:13.157+0000 7f4ac0a4e700 1 --2- 172.21.15.143:0/3016346648 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4abc0d1f10 0x7f4abc0cc3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:31:13.169 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:13.157+0000 7f4ac1a50700 1 --2- 172.21.15.143:0/3016346648 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4abc15de20 0x7f4abc15e4e0 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f4ab400b970 tx=0x7f4ab400bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:13.169 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:13.157+0000 7f4ab1ffb700 1 -- 172.21.15.143:0/3016346648 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ab400c9b0 con 0x7f4abc15de20 2023-11-08T19:31:13.169 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:13.157+0000 7f4ab1ffb700 1 -- 172.21.15.143:0/3016346648 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ab400cb30 con 0x7f4abc15de20 2023-11-08T19:31:13.169 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:13.157+0000 7f4ac34b3700 1 -- 172.21.15.143:0/3016346648 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4abc0c7b50 con 0x7f4abc15de20 2023-11-08T19:31:13.170 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:13.157+0000 7f4ab1ffb700 1 -- 172.21.15.143:0/3016346648 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ab4014440 con 0x7f4abc15de20 2023-11-08T19:31:13.170 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:13.158+0000 7f4ac34b3700 1 -- 172.21.15.143:0/3016346648 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4abc0c8110 con 0x7f4abc15de20 2023-11-08T19:31:13.170 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:13.159+0000 7f4ac34b3700 1 -- 172.21.15.143:0/3016346648 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4abc04f730 con 0x7f4abc15de20 2023-11-08T19:31:13.170 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:13.159+0000 7f4ab1ffb700 1 -- 172.21.15.143:0/3016346648 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4ab401d3b0 con 0x7f4abc15de20 2023-11-08T19:31:13.171 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:13.159+0000 7f4ab1ffb700 1 --2- 172.21.15.143:0/3016346648 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f4aa006af00 0x7f4aa006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:13.171 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:13.159+0000 7f4ab1ffb700 1 -- 172.21.15.143:0/3016346648 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 7094+0+0 (secure 0 0 0) 0x7f4ab4006e40 con 0x7f4abc15de20 2023-11-08T19:31:13.171 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:13.160+0000 7f4ac124f700 1 --2- 172.21.15.143:0/3016346648 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f4aa006af00 0x7f4aa006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:13.172 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:13.160+0000 7f4ac124f700 1 --2- 172.21.15.143:0/3016346648 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f4aa006af00 0x7f4aa006d3c0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f4ab8005910 tx=0x7f4ab8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:13.172 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:13.164+0000 7f4ab1ffb700 1 -- 172.21.15.143:0/3016346648 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4ab4054470 con 0x7f4abc15de20 2023-11-08T19:31:13.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:13.352+0000 7f4ac34b3700 1 -- 172.21.15.143:0/3016346648 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f4abc0d1790 con 0x7f4abc15de20 2023-11-08T19:31:14.007 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.005+0000 7f4ab1ffb700 1 -- 172.21.15.143:0/3016346648 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v107) v1 ==== 169+0+0 (secure 0 0 0) 0x7f4ab4057a90 con 0x7f4abc15de20 2023-11-08T19:31:14.009 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.008+0000 7f4ac34b3700 1 -- 172.21.15.143:0/3016346648 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f4aa006af00 msgr2=0x7f4aa006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:14.010 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.008+0000 7f4ac34b3700 1 --2- 172.21.15.143:0/3016346648 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f4aa006af00 0x7f4aa006d3c0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f4ab8005910 tx=0x7f4ab8009040 comp rx=0 tx=0).stop 2023-11-08T19:31:14.010 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.008+0000 7f4ac34b3700 1 -- 172.21.15.143:0/3016346648 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4abc15de20 msgr2=0x7f4abc15e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:14.010 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.008+0000 7f4ac34b3700 1 --2- 172.21.15.143:0/3016346648 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4abc15de20 0x7f4abc15e4e0 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f4ab400b970 tx=0x7f4ab400bd30 comp rx=0 tx=0).stop 2023-11-08T19:31:14.014 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.008+0000 7f4ac34b3700 1 -- 172.21.15.143:0/3016346648 shutdown_connections 2023-11-08T19:31:14.014 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.008+0000 7f4ac34b3700 1 --2- 172.21.15.143:0/3016346648 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f4aa006af00 0x7f4aa006d3c0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:14.014 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.008+0000 7f4ac34b3700 1 --2- 172.21.15.143:0/3016346648 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4abc0b5590 0x7f4abc1549e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:14.014 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.008+0000 7f4ac34b3700 1 --2- 172.21.15.143:0/3016346648 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4abc15de20 0x7f4abc15e4e0 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:14.014 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.008+0000 7f4ac34b3700 1 --2- 172.21.15.143:0/3016346648 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4abc0d1f10 0x7f4abc0cc3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:14.015 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.008+0000 7f4ac34b3700 1 -- 172.21.15.143:0/3016346648 >> 172.21.15.143:0/3016346648 conn(0x7f4abc0d0590 msgr2=0x7f4abc0d23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:14.015 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.008+0000 7f4ac34b3700 1 -- 172.21.15.143:0/3016346648 shutdown_connections 2023-11-08T19:31:14.015 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.008+0000 7f4ac34b3700 1 -- 172.21.15.143:0/3016346648 wait complete. 2023-11-08T19:31:14.016 INFO:teuthology.orchestra.run.smithi143.stderr:pool 'cephfs_data' removed 2023-11-08T19:31:14.023 DEBUG:teuthology.orchestra.run.smithi143:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-08T19:31:14.025 DEBUG:teuthology.orchestra.run.smithi174:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-08T19:31:14.069 INFO:teuthology.orchestra.run.smithi143.stdout:[global] 2023-11-08T19:31:14.069 INFO:teuthology.orchestra.run.smithi143.stdout: chdir = "" 2023-11-08T19:31:14.069 INFO:teuthology.orchestra.run.smithi143.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-08T19:31:14.070 INFO:teuthology.orchestra.run.smithi143.stdout: auth supported = cephx 2023-11-08T19:31:14.070 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:31:14.070 INFO:teuthology.orchestra.run.smithi143.stdout: filestore xattr use omap = true 2023-11-08T19:31:14.070 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:31:14.070 INFO:teuthology.orchestra.run.smithi143.stdout: mon clock drift allowed = 1.000 2023-11-08T19:31:14.071 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:31:14.071 INFO:teuthology.orchestra.run.smithi143.stdout: osd crush chooseleaf type = 0 2023-11-08T19:31:14.071 INFO:teuthology.orchestra.run.smithi143.stdout: auth debug = true 2023-11-08T19:31:14.071 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:31:14.071 INFO:teuthology.orchestra.run.smithi143.stdout: ms die on old message = true 2023-11-08T19:31:14.072 INFO:teuthology.orchestra.run.smithi143.stdout: ms die on bug = true 2023-11-08T19:31:14.072 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:31:14.072 INFO:teuthology.orchestra.run.smithi143.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-08T19:31:14.072 INFO:teuthology.orchestra.run.smithi143.stdout: mon pg warn max object skew = 0 2023-11-08T19:31:14.072 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:31:14.073 INFO:teuthology.orchestra.run.smithi143.stdout: # disable pg_autoscaler by default for new pools 2023-11-08T19:31:14.073 INFO:teuthology.orchestra.run.smithi143.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-08T19:31:14.073 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:31:14.073 INFO:teuthology.orchestra.run.smithi143.stdout: osd pool default size = 2 2023-11-08T19:31:14.073 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:31:14.073 INFO:teuthology.orchestra.run.smithi143.stdout: mon osd allow primary affinity = true 2023-11-08T19:31:14.074 INFO:teuthology.orchestra.run.smithi143.stdout: mon osd allow pg remap = true 2023-11-08T19:31:14.074 INFO:teuthology.orchestra.run.smithi143.stdout: mon warn on legacy crush tunables = false 2023-11-08T19:31:14.074 INFO:teuthology.orchestra.run.smithi143.stdout: mon warn on crush straw calc version zero = false 2023-11-08T19:31:14.074 INFO:teuthology.orchestra.run.smithi143.stdout: mon warn on no sortbitwise = false 2023-11-08T19:31:14.074 INFO:teuthology.orchestra.run.smithi143.stdout: mon warn on osd down out interval zero = false 2023-11-08T19:31:14.075 INFO:teuthology.orchestra.run.smithi143.stdout: mon warn on too few osds = false 2023-11-08T19:31:14.075 INFO:teuthology.orchestra.run.smithi143.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-08T19:31:14.075 INFO:teuthology.orchestra.run.smithi143.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-08T19:31:14.075 INFO:teuthology.orchestra.run.smithi143.stdout: mon_allow_pool_size_one = true 2023-11-08T19:31:14.075 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:31:14.075 INFO:teuthology.orchestra.run.smithi143.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-11-08T19:31:14.076 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:31:14.076 INFO:teuthology.orchestra.run.smithi143.stdout: osd default data pool replay window = 5 2023-11-08T19:31:14.076 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:31:14.076 INFO:teuthology.orchestra.run.smithi143.stdout: mon allow pool delete = true 2023-11-08T19:31:14.076 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:31:14.077 INFO:teuthology.orchestra.run.smithi143.stdout: mon cluster log file level = debug 2023-11-08T19:31:14.077 INFO:teuthology.orchestra.run.smithi143.stdout: debug asserts on shutdown = true 2023-11-08T19:31:14.077 INFO:teuthology.orchestra.run.smithi143.stdout: mon health detail to clog = false 2023-11-08T19:31:14.077 INFO:teuthology.orchestra.run.smithi143.stdout: mon host = "172.21.15.143,172.21.15.174,[v2:172.21.15.174:3301,v1:172.21.15.174:6790]" 2023-11-08T19:31:14.077 INFO:teuthology.orchestra.run.smithi143.stdout: ms die on skipped message = False 2023-11-08T19:31:14.077 INFO:teuthology.orchestra.run.smithi143.stdout: fsid = 968483af-77fc-4068-ad7c-5c2a1a4613d7 2023-11-08T19:31:14.078 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:31:14.078 INFO:teuthology.orchestra.run.smithi143.stdout:[osd] 2023-11-08T19:31:14.078 INFO:teuthology.orchestra.run.smithi143.stdout: osd journal size = 100 2023-11-08T19:31:14.078 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:31:14.078 INFO:teuthology.orchestra.run.smithi143.stdout: osd scrub load threshold = 5.0 2023-11-08T19:31:14.079 INFO:teuthology.orchestra.run.smithi143.stdout: osd scrub max interval = 600 2023-11-08T19:31:14.079 INFO:teuthology.orchestra.run.smithi143.stdout: osd mclock profile = high_recovery_ops 2023-11-08T19:31:14.079 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:31:14.079 INFO:teuthology.orchestra.run.smithi143.stdout: osd recover clone overlap = true 2023-11-08T19:31:14.079 INFO:teuthology.orchestra.run.smithi143.stdout: osd recovery max chunk = 1048576 2023-11-08T19:31:14.080 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:31:14.080 INFO:teuthology.orchestra.run.smithi143.stdout: osd debug shutdown = true 2023-11-08T19:31:14.080 INFO:teuthology.orchestra.run.smithi143.stdout: osd debug op order = true 2023-11-08T19:31:14.080 INFO:teuthology.orchestra.run.smithi143.stdout: osd debug verify stray on activate = true 2023-11-08T19:31:14.080 INFO:teuthology.orchestra.run.smithi143.stdout: osd debug trim objects = true 2023-11-08T19:31:14.080 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:31:14.081 INFO:teuthology.orchestra.run.smithi143.stdout: osd open classes on start = true 2023-11-08T19:31:14.081 INFO:teuthology.orchestra.run.smithi143.stdout: osd debug pg log writeout = true 2023-11-08T19:31:14.081 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:31:14.081 INFO:teuthology.orchestra.run.smithi143.stdout: osd deep scrub update digest min age = 30 2023-11-08T19:31:14.081 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:31:14.082 INFO:teuthology.orchestra.run.smithi143.stdout: osd map max advance = 10 2023-11-08T19:31:14.082 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:31:14.082 INFO:teuthology.orchestra.run.smithi143.stdout: journal zero on create = true 2023-11-08T19:31:14.082 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:31:14.082 INFO:teuthology.orchestra.run.smithi143.stdout: filestore ondisk finisher threads = 3 2023-11-08T19:31:14.083 INFO:teuthology.orchestra.run.smithi143.stdout: filestore apply finisher threads = 3 2023-11-08T19:31:14.083 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:31:14.083 INFO:teuthology.orchestra.run.smithi143.stdout: bdev debug aio = true 2023-11-08T19:31:14.083 INFO:teuthology.orchestra.run.smithi143.stdout: osd debug misdirected ops = true 2023-11-08T19:31:14.083 INFO:teuthology.orchestra.run.smithi143.stdout: bdev async discard = True 2023-11-08T19:31:14.084 INFO:teuthology.orchestra.run.smithi143.stdout: bdev enable discard = True 2023-11-08T19:31:14.084 INFO:teuthology.orchestra.run.smithi143.stdout: bluestore allocator = bitmap 2023-11-08T19:31:14.084 INFO:teuthology.orchestra.run.smithi143.stdout: bluestore block size = 96636764160 2023-11-08T19:31:14.084 INFO:teuthology.orchestra.run.smithi143.stdout: bluestore fsck on mount = True 2023-11-08T19:31:14.084 INFO:teuthology.orchestra.run.smithi143.stdout: debug bluefs = 1/20 2023-11-08T19:31:14.085 INFO:teuthology.orchestra.run.smithi143.stdout: debug bluestore = 1/20 2023-11-08T19:31:14.085 INFO:teuthology.orchestra.run.smithi143.stdout: debug ms = 1 2023-11-08T19:31:14.085 INFO:teuthology.orchestra.run.smithi143.stdout: debug osd = 20 2023-11-08T19:31:14.085 INFO:teuthology.orchestra.run.smithi143.stdout: debug rocksdb = 4/10 2023-11-08T19:31:14.085 INFO:teuthology.orchestra.run.smithi143.stdout: mon osd backfillfull_ratio = 0.85 2023-11-08T19:31:14.085 INFO:teuthology.orchestra.run.smithi143.stdout: mon osd full ratio = 0.9 2023-11-08T19:31:14.086 INFO:teuthology.orchestra.run.smithi143.stdout: mon osd nearfull ratio = 0.8 2023-11-08T19:31:14.086 INFO:teuthology.orchestra.run.smithi143.stdout: osd failsafe full ratio = 0.95 2023-11-08T19:31:14.086 INFO:teuthology.orchestra.run.smithi143.stdout: osd objectstore = bluestore 2023-11-08T19:31:14.086 INFO:teuthology.orchestra.run.smithi143.stdout: osd op complaint time = 180 2023-11-08T19:31:14.086 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:31:14.087 INFO:teuthology.orchestra.run.smithi143.stdout:[mgr] 2023-11-08T19:31:14.087 INFO:teuthology.orchestra.run.smithi143.stdout: debug ms = 1 2023-11-08T19:31:14.087 INFO:teuthology.orchestra.run.smithi143.stdout: debug mgr = 20 2023-11-08T19:31:14.087 INFO:teuthology.orchestra.run.smithi143.stdout: debug mon = 20 2023-11-08T19:31:14.087 INFO:teuthology.orchestra.run.smithi143.stdout: debug auth = 20 2023-11-08T19:31:14.087 INFO:teuthology.orchestra.run.smithi143.stdout: mon reweight min pgs per osd = 4 2023-11-08T19:31:14.088 INFO:teuthology.orchestra.run.smithi143.stdout: mon reweight min bytes per osd = 10 2023-11-08T19:31:14.088 INFO:teuthology.orchestra.run.smithi143.stdout: mgr/telemetry/nag = false 2023-11-08T19:31:14.088 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:31:14.088 INFO:teuthology.orchestra.run.smithi143.stdout:[mon] 2023-11-08T19:31:14.088 INFO:teuthology.orchestra.run.smithi143.stdout: debug ms = 1 2023-11-08T19:31:14.089 INFO:teuthology.orchestra.run.smithi143.stdout: debug mon = 20 2023-11-08T19:31:14.089 INFO:teuthology.orchestra.run.smithi143.stdout: debug paxos = 20 2023-11-08T19:31:14.089 INFO:teuthology.orchestra.run.smithi143.stdout: debug auth = 20 2023-11-08T19:31:14.089 INFO:teuthology.orchestra.run.smithi143.stdout: mon data avail warn = 5 2023-11-08T19:31:14.089 INFO:teuthology.orchestra.run.smithi143.stdout: mon mgr mkfs grace = 240 2023-11-08T19:31:14.090 INFO:teuthology.orchestra.run.smithi143.stdout: mon reweight min pgs per osd = 4 2023-11-08T19:31:14.090 INFO:teuthology.orchestra.run.smithi143.stdout: mon osd reporter subtree level = osd 2023-11-08T19:31:14.090 INFO:teuthology.orchestra.run.smithi143.stdout: mon osd prime pg temp = true 2023-11-08T19:31:14.090 INFO:teuthology.orchestra.run.smithi143.stdout: mon reweight min bytes per osd = 10 2023-11-08T19:31:14.090 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:31:14.091 INFO:teuthology.orchestra.run.smithi143.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-08T19:31:14.091 INFO:teuthology.orchestra.run.smithi143.stdout: auth mon ticket ttl = 660 # 11m 2023-11-08T19:31:14.091 INFO:teuthology.orchestra.run.smithi143.stdout: auth service ticket ttl = 240 # 4m 2023-11-08T19:31:14.091 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:31:14.091 INFO:teuthology.orchestra.run.smithi143.stdout: # don't complain about insecure global_id in the test suite 2023-11-08T19:31:14.091 INFO:teuthology.orchestra.run.smithi143.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-08T19:31:14.092 INFO:teuthology.orchestra.run.smithi143.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-08T19:31:14.092 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:31:14.092 INFO:teuthology.orchestra.run.smithi143.stdout: # 1m isn't quite enough 2023-11-08T19:31:14.092 INFO:teuthology.orchestra.run.smithi143.stdout: mon_down_mkfs_grace = 2m 2023-11-08T19:31:14.092 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:31:14.093 INFO:teuthology.orchestra.run.smithi143.stdout: mon_warn_on_filestore_osds = false 2023-11-08T19:31:14.093 INFO:teuthology.orchestra.run.smithi143.stdout: mon op complaint time = 120 2023-11-08T19:31:14.093 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:31:14.093 INFO:teuthology.orchestra.run.smithi143.stdout:[client] 2023-11-08T19:31:14.093 INFO:teuthology.orchestra.run.smithi143.stdout: rgw cache enabled = true 2023-11-08T19:31:14.093 INFO:teuthology.orchestra.run.smithi143.stdout: rgw enable ops log = true 2023-11-08T19:31:14.094 INFO:teuthology.orchestra.run.smithi143.stdout: rgw enable usage log = true 2023-11-08T19:31:14.094 INFO:teuthology.orchestra.run.smithi143.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-08T19:31:14.094 INFO:teuthology.orchestra.run.smithi143.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-08T19:31:14.094 INFO:teuthology.orchestra.run.smithi143.stdout: client mount timeout = 600 2023-11-08T19:31:14.094 INFO:teuthology.orchestra.run.smithi143.stdout: debug client = 20 2023-11-08T19:31:14.095 INFO:teuthology.orchestra.run.smithi143.stdout: debug ms = 1 2023-11-08T19:31:14.095 INFO:teuthology.orchestra.run.smithi143.stdout: pid file = "" 2023-11-08T19:31:14.095 INFO:teuthology.orchestra.run.smithi143.stdout: rados mon op timeout = 900 2023-11-08T19:31:14.095 INFO:teuthology.orchestra.run.smithi143.stdout: rados osd op timeout = 900 2023-11-08T19:31:14.095 INFO:teuthology.orchestra.run.smithi143.stdout:[mon.a] 2023-11-08T19:31:14.096 INFO:teuthology.orchestra.run.smithi143.stdout:[mon.b] 2023-11-08T19:31:14.096 INFO:teuthology.orchestra.run.smithi143.stdout:[mon.c] 2023-11-08T19:31:14.096 INFO:teuthology.orchestra.run.smithi143.stdout:[mds.a] 2023-11-08T19:31:14.096 INFO:teuthology.orchestra.run.smithi143.stdout:[mds.c] 2023-11-08T19:31:14.096 INFO:teuthology.orchestra.run.smithi143.stdout:[mds.b] 2023-11-08T19:31:14.097 INFO:teuthology.orchestra.run.smithi143.stdout:[mds.d] 2023-11-08T19:31:14.097 INFO:teuthology.orchestra.run.smithi143.stdout:[mds] 2023-11-08T19:31:14.097 INFO:teuthology.orchestra.run.smithi143.stdout: debug mds = 20 2023-11-08T19:31:14.097 INFO:teuthology.orchestra.run.smithi143.stdout: debug mds balancer = 20 2023-11-08T19:31:14.097 INFO:teuthology.orchestra.run.smithi143.stdout: debug ms = 1 2023-11-08T19:31:14.097 INFO:teuthology.orchestra.run.smithi143.stdout: mds debug frag = True 2023-11-08T19:31:14.098 INFO:teuthology.orchestra.run.smithi143.stdout: mds debug scatterstat = True 2023-11-08T19:31:14.098 INFO:teuthology.orchestra.run.smithi143.stdout: mds op complaint time = 180 2023-11-08T19:31:14.098 INFO:teuthology.orchestra.run.smithi143.stdout: mds verify scatter = True 2023-11-08T19:31:14.098 INFO:teuthology.orchestra.run.smithi143.stdout: osd op complaint time = 180 2023-11-08T19:31:14.098 INFO:teuthology.orchestra.run.smithi143.stdout: rados mon op timeout = 900 2023-11-08T19:31:14.099 INFO:teuthology.orchestra.run.smithi143.stdout: rados osd op timeout = 900 2023-11-08T19:31:14.100 INFO:teuthology.orchestra.run.smithi174.stdout:[global] 2023-11-08T19:31:14.100 INFO:teuthology.orchestra.run.smithi174.stdout: chdir = "" 2023-11-08T19:31:14.101 INFO:teuthology.orchestra.run.smithi174.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-08T19:31:14.101 INFO:teuthology.orchestra.run.smithi174.stdout: auth supported = cephx 2023-11-08T19:31:14.101 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-08T19:31:14.101 INFO:teuthology.orchestra.run.smithi174.stdout: filestore xattr use omap = true 2023-11-08T19:31:14.101 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-08T19:31:14.102 INFO:teuthology.orchestra.run.smithi174.stdout: mon clock drift allowed = 1.000 2023-11-08T19:31:14.102 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-08T19:31:14.102 INFO:teuthology.orchestra.run.smithi174.stdout: osd crush chooseleaf type = 0 2023-11-08T19:31:14.102 INFO:teuthology.orchestra.run.smithi174.stdout: auth debug = true 2023-11-08T19:31:14.102 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-08T19:31:14.103 INFO:teuthology.orchestra.run.smithi174.stdout: ms die on old message = true 2023-11-08T19:31:14.103 INFO:teuthology.orchestra.run.smithi174.stdout: ms die on bug = true 2023-11-08T19:31:14.103 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-08T19:31:14.103 INFO:teuthology.orchestra.run.smithi174.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-08T19:31:14.103 INFO:teuthology.orchestra.run.smithi174.stdout: mon pg warn max object skew = 0 2023-11-08T19:31:14.104 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-08T19:31:14.104 INFO:teuthology.orchestra.run.smithi174.stdout: # disable pg_autoscaler by default for new pools 2023-11-08T19:31:14.104 INFO:teuthology.orchestra.run.smithi174.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-08T19:31:14.104 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-08T19:31:14.104 INFO:teuthology.orchestra.run.smithi174.stdout: osd pool default size = 2 2023-11-08T19:31:14.104 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-08T19:31:14.105 INFO:teuthology.orchestra.run.smithi174.stdout: mon osd allow primary affinity = true 2023-11-08T19:31:14.105 INFO:teuthology.orchestra.run.smithi174.stdout: mon osd allow pg remap = true 2023-11-08T19:31:14.105 INFO:teuthology.orchestra.run.smithi174.stdout: mon warn on legacy crush tunables = false 2023-11-08T19:31:14.105 INFO:teuthology.orchestra.run.smithi174.stdout: mon warn on crush straw calc version zero = false 2023-11-08T19:31:14.105 INFO:teuthology.orchestra.run.smithi174.stdout: mon warn on no sortbitwise = false 2023-11-08T19:31:14.106 INFO:teuthology.orchestra.run.smithi174.stdout: mon warn on osd down out interval zero = false 2023-11-08T19:31:14.106 INFO:teuthology.orchestra.run.smithi174.stdout: mon warn on too few osds = false 2023-11-08T19:31:14.106 INFO:teuthology.orchestra.run.smithi174.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-08T19:31:14.106 INFO:teuthology.orchestra.run.smithi174.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-08T19:31:14.106 INFO:teuthology.orchestra.run.smithi174.stdout: mon_allow_pool_size_one = true 2023-11-08T19:31:14.107 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-08T19:31:14.107 INFO:teuthology.orchestra.run.smithi174.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-11-08T19:31:14.107 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-08T19:31:14.107 INFO:teuthology.orchestra.run.smithi174.stdout: osd default data pool replay window = 5 2023-11-08T19:31:14.107 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-08T19:31:14.108 INFO:teuthology.orchestra.run.smithi174.stdout: mon allow pool delete = true 2023-11-08T19:31:14.108 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-08T19:31:14.108 INFO:teuthology.orchestra.run.smithi174.stdout: mon cluster log file level = debug 2023-11-08T19:31:14.108 INFO:teuthology.orchestra.run.smithi174.stdout: debug asserts on shutdown = true 2023-11-08T19:31:14.108 INFO:teuthology.orchestra.run.smithi174.stdout: mon health detail to clog = false 2023-11-08T19:31:14.109 INFO:teuthology.orchestra.run.smithi174.stdout: mon host = "172.21.15.143,172.21.15.174,[v2:172.21.15.174:3301,v1:172.21.15.174:6790]" 2023-11-08T19:31:14.109 INFO:teuthology.orchestra.run.smithi174.stdout: ms die on skipped message = False 2023-11-08T19:31:14.109 INFO:teuthology.orchestra.run.smithi174.stdout: fsid = 968483af-77fc-4068-ad7c-5c2a1a4613d7 2023-11-08T19:31:14.109 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-08T19:31:14.109 INFO:teuthology.orchestra.run.smithi174.stdout:[osd] 2023-11-08T19:31:14.109 INFO:teuthology.orchestra.run.smithi174.stdout: osd journal size = 100 2023-11-08T19:31:14.110 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-08T19:31:14.110 INFO:teuthology.orchestra.run.smithi174.stdout: osd scrub load threshold = 5.0 2023-11-08T19:31:14.110 INFO:teuthology.orchestra.run.smithi174.stdout: osd scrub max interval = 600 2023-11-08T19:31:14.110 INFO:teuthology.orchestra.run.smithi174.stdout: osd mclock profile = high_recovery_ops 2023-11-08T19:31:14.110 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-08T19:31:14.111 INFO:teuthology.orchestra.run.smithi174.stdout: osd recover clone overlap = true 2023-11-08T19:31:14.111 INFO:teuthology.orchestra.run.smithi174.stdout: osd recovery max chunk = 1048576 2023-11-08T19:31:14.111 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-08T19:31:14.111 INFO:teuthology.orchestra.run.smithi174.stdout: osd debug shutdown = true 2023-11-08T19:31:14.111 INFO:teuthology.orchestra.run.smithi174.stdout: osd debug op order = true 2023-11-08T19:31:14.112 INFO:teuthology.orchestra.run.smithi174.stdout: osd debug verify stray on activate = true 2023-11-08T19:31:14.112 INFO:teuthology.orchestra.run.smithi174.stdout: osd debug trim objects = true 2023-11-08T19:31:14.112 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-08T19:31:14.112 INFO:teuthology.orchestra.run.smithi174.stdout: osd open classes on start = true 2023-11-08T19:31:14.112 INFO:teuthology.orchestra.run.smithi174.stdout: osd debug pg log writeout = true 2023-11-08T19:31:14.113 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-08T19:31:14.113 INFO:teuthology.orchestra.run.smithi174.stdout: osd deep scrub update digest min age = 30 2023-11-08T19:31:14.113 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-08T19:31:14.113 INFO:teuthology.orchestra.run.smithi174.stdout: osd map max advance = 10 2023-11-08T19:31:14.113 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-08T19:31:14.114 INFO:teuthology.orchestra.run.smithi174.stdout: journal zero on create = true 2023-11-08T19:31:14.114 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-08T19:31:14.114 INFO:teuthology.orchestra.run.smithi174.stdout: filestore ondisk finisher threads = 3 2023-11-08T19:31:14.114 INFO:teuthology.orchestra.run.smithi174.stdout: filestore apply finisher threads = 3 2023-11-08T19:31:14.114 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-08T19:31:14.115 INFO:teuthology.orchestra.run.smithi174.stdout: bdev debug aio = true 2023-11-08T19:31:14.115 INFO:teuthology.orchestra.run.smithi174.stdout: osd debug misdirected ops = true 2023-11-08T19:31:14.115 INFO:teuthology.orchestra.run.smithi174.stdout: bdev async discard = True 2023-11-08T19:31:14.115 INFO:teuthology.orchestra.run.smithi174.stdout: bdev enable discard = True 2023-11-08T19:31:14.115 INFO:teuthology.orchestra.run.smithi174.stdout: bluestore allocator = bitmap 2023-11-08T19:31:14.115 INFO:teuthology.orchestra.run.smithi174.stdout: bluestore block size = 96636764160 2023-11-08T19:31:14.116 INFO:teuthology.orchestra.run.smithi174.stdout: bluestore fsck on mount = True 2023-11-08T19:31:14.116 INFO:teuthology.orchestra.run.smithi174.stdout: debug bluefs = 1/20 2023-11-08T19:31:14.116 INFO:teuthology.orchestra.run.smithi174.stdout: debug bluestore = 1/20 2023-11-08T19:31:14.116 INFO:teuthology.orchestra.run.smithi174.stdout: debug ms = 1 2023-11-08T19:31:14.116 INFO:teuthology.orchestra.run.smithi174.stdout: debug osd = 20 2023-11-08T19:31:14.117 INFO:teuthology.orchestra.run.smithi174.stdout: debug rocksdb = 4/10 2023-11-08T19:31:14.117 INFO:teuthology.orchestra.run.smithi174.stdout: mon osd backfillfull_ratio = 0.85 2023-11-08T19:31:14.117 INFO:teuthology.orchestra.run.smithi174.stdout: mon osd full ratio = 0.9 2023-11-08T19:31:14.117 INFO:teuthology.orchestra.run.smithi174.stdout: mon osd nearfull ratio = 0.8 2023-11-08T19:31:14.117 INFO:teuthology.orchestra.run.smithi174.stdout: osd failsafe full ratio = 0.95 2023-11-08T19:31:14.118 INFO:teuthology.orchestra.run.smithi174.stdout: osd objectstore = bluestore 2023-11-08T19:31:14.118 INFO:teuthology.orchestra.run.smithi174.stdout: osd op complaint time = 180 2023-11-08T19:31:14.118 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-08T19:31:14.118 INFO:teuthology.orchestra.run.smithi174.stdout:[mgr] 2023-11-08T19:31:14.118 INFO:teuthology.orchestra.run.smithi174.stdout: debug ms = 1 2023-11-08T19:31:14.119 INFO:teuthology.orchestra.run.smithi174.stdout: debug mgr = 20 2023-11-08T19:31:14.119 INFO:teuthology.orchestra.run.smithi174.stdout: debug mon = 20 2023-11-08T19:31:14.119 INFO:teuthology.orchestra.run.smithi174.stdout: debug auth = 20 2023-11-08T19:31:14.119 INFO:teuthology.orchestra.run.smithi174.stdout: mon reweight min pgs per osd = 4 2023-11-08T19:31:14.119 INFO:teuthology.orchestra.run.smithi174.stdout: mon reweight min bytes per osd = 10 2023-11-08T19:31:14.119 INFO:teuthology.orchestra.run.smithi174.stdout: mgr/telemetry/nag = false 2023-11-08T19:31:14.120 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-08T19:31:14.120 INFO:teuthology.orchestra.run.smithi174.stdout:[mon] 2023-11-08T19:31:14.120 INFO:teuthology.orchestra.run.smithi174.stdout: debug ms = 1 2023-11-08T19:31:14.120 INFO:teuthology.orchestra.run.smithi174.stdout: debug mon = 20 2023-11-08T19:31:14.120 INFO:teuthology.orchestra.run.smithi174.stdout: debug paxos = 20 2023-11-08T19:31:14.121 INFO:teuthology.orchestra.run.smithi174.stdout: debug auth = 20 2023-11-08T19:31:14.121 INFO:teuthology.orchestra.run.smithi174.stdout: mon data avail warn = 5 2023-11-08T19:31:14.121 INFO:teuthology.orchestra.run.smithi174.stdout: mon mgr mkfs grace = 240 2023-11-08T19:31:14.121 INFO:teuthology.orchestra.run.smithi174.stdout: mon reweight min pgs per osd = 4 2023-11-08T19:31:14.121 INFO:teuthology.orchestra.run.smithi174.stdout: mon osd reporter subtree level = osd 2023-11-08T19:31:14.122 INFO:teuthology.orchestra.run.smithi174.stdout: mon osd prime pg temp = true 2023-11-08T19:31:14.122 INFO:teuthology.orchestra.run.smithi174.stdout: mon reweight min bytes per osd = 10 2023-11-08T19:31:14.122 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-08T19:31:14.122 INFO:teuthology.orchestra.run.smithi174.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-08T19:31:14.122 INFO:teuthology.orchestra.run.smithi174.stdout: auth mon ticket ttl = 660 # 11m 2023-11-08T19:31:14.123 INFO:teuthology.orchestra.run.smithi174.stdout: auth service ticket ttl = 240 # 4m 2023-11-08T19:31:14.123 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-08T19:31:14.123 INFO:teuthology.orchestra.run.smithi174.stdout: # don't complain about insecure global_id in the test suite 2023-11-08T19:31:14.123 INFO:teuthology.orchestra.run.smithi174.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-08T19:31:14.123 INFO:teuthology.orchestra.run.smithi174.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-08T19:31:14.123 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-08T19:31:14.124 INFO:teuthology.orchestra.run.smithi174.stdout: # 1m isn't quite enough 2023-11-08T19:31:14.124 INFO:teuthology.orchestra.run.smithi174.stdout: mon_down_mkfs_grace = 2m 2023-11-08T19:31:14.124 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-08T19:31:14.124 INFO:teuthology.orchestra.run.smithi174.stdout: mon_warn_on_filestore_osds = false 2023-11-08T19:31:14.124 INFO:teuthology.orchestra.run.smithi174.stdout: mon op complaint time = 120 2023-11-08T19:31:14.125 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-08T19:31:14.125 INFO:teuthology.orchestra.run.smithi174.stdout:[client] 2023-11-08T19:31:14.125 INFO:teuthology.orchestra.run.smithi174.stdout: rgw cache enabled = true 2023-11-08T19:31:14.125 INFO:teuthology.orchestra.run.smithi174.stdout: rgw enable ops log = true 2023-11-08T19:31:14.125 INFO:teuthology.orchestra.run.smithi174.stdout: rgw enable usage log = true 2023-11-08T19:31:14.125 INFO:teuthology.orchestra.run.smithi174.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-08T19:31:14.126 INFO:teuthology.orchestra.run.smithi174.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-08T19:31:14.126 INFO:teuthology.orchestra.run.smithi174.stdout: client mount timeout = 600 2023-11-08T19:31:14.126 INFO:teuthology.orchestra.run.smithi174.stdout: debug client = 20 2023-11-08T19:31:14.126 INFO:teuthology.orchestra.run.smithi174.stdout: debug ms = 1 2023-11-08T19:31:14.126 INFO:teuthology.orchestra.run.smithi174.stdout: pid file = "" 2023-11-08T19:31:14.127 INFO:teuthology.orchestra.run.smithi174.stdout: rados mon op timeout = 900 2023-11-08T19:31:14.127 INFO:teuthology.orchestra.run.smithi174.stdout: rados osd op timeout = 900 2023-11-08T19:31:14.127 INFO:teuthology.orchestra.run.smithi174.stdout:[mon.a] 2023-11-08T19:31:14.127 INFO:teuthology.orchestra.run.smithi174.stdout:[mon.b] 2023-11-08T19:31:14.127 INFO:teuthology.orchestra.run.smithi174.stdout:[mon.c] 2023-11-08T19:31:14.128 INFO:teuthology.orchestra.run.smithi174.stdout:[mds.a] 2023-11-08T19:31:14.128 INFO:teuthology.orchestra.run.smithi174.stdout:[mds.c] 2023-11-08T19:31:14.128 INFO:teuthology.orchestra.run.smithi174.stdout:[mds.b] 2023-11-08T19:31:14.128 INFO:teuthology.orchestra.run.smithi174.stdout:[mds.d] 2023-11-08T19:31:14.128 INFO:teuthology.orchestra.run.smithi174.stdout:[mds] 2023-11-08T19:31:14.128 INFO:teuthology.orchestra.run.smithi174.stdout: debug mds = 20 2023-11-08T19:31:14.129 INFO:teuthology.orchestra.run.smithi174.stdout: debug mds balancer = 20 2023-11-08T19:31:14.129 INFO:teuthology.orchestra.run.smithi174.stdout: debug ms = 1 2023-11-08T19:31:14.129 INFO:teuthology.orchestra.run.smithi174.stdout: mds debug frag = True 2023-11-08T19:31:14.129 INFO:teuthology.orchestra.run.smithi174.stdout: mds debug scatterstat = True 2023-11-08T19:31:14.129 INFO:teuthology.orchestra.run.smithi174.stdout: mds op complaint time = 180 2023-11-08T19:31:14.130 INFO:teuthology.orchestra.run.smithi174.stdout: mds verify scatter = True 2023-11-08T19:31:14.130 INFO:teuthology.orchestra.run.smithi174.stdout: osd op complaint time = 180 2023-11-08T19:31:14.130 INFO:teuthology.orchestra.run.smithi174.stdout: rados mon op timeout = 900 2023-11-08T19:31:14.130 INFO:teuthology.orchestra.run.smithi174.stdout: rados osd op timeout = 900 2023-11-08T19:31:14.131 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-08T19:31:14.270 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.269+0000 7f7870868700 1 Processor -- start 2023-11-08T19:31:14.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.269+0000 7f7870868700 1 -- start start 2023-11-08T19:31:14.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.270+0000 7f7870868700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f786c15dba0 0x7f786c06c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:14.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.270+0000 7f7870868700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f786c0b22e0 0x7f786c0b2700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:14.274 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.270+0000 7f7870868700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f786c161890 con 0x7f786c06b240 2023-11-08T19:31:14.274 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.270+0000 7f7870868700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f786c0b2c40 con 0x7f786c0b22e0 2023-11-08T19:31:14.274 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.270+0000 7f7870868700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f786c06b740 con 0x7f786c15dba0 2023-11-08T19:31:14.274 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.270+0000 7f786ad9d700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f786c0b22e0 0x7f786c0b2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:14.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.270+0000 7f786ad9d700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f786c0b22e0 0x7f786c0b2700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57072/0 (socket says 172.21.15.143:57072) 2023-11-08T19:31:14.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.270+0000 7f786ad9d700 1 -- 172.21.15.143:0/3351305416 learned_addr learned my addr 172.21.15.143:0/3351305416 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:31:14.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.270+0000 7f786a59c700 1 --2- 172.21.15.143:0/3351305416 >> v2:172.21.15.174:3300/0 conn(0x7f786c15dba0 0x7f786c06c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:14.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.270+0000 7f786ad9d700 1 -- 172.21.15.143:0/3351305416 >> v2:172.21.15.174:3300/0 conn(0x7f786c15dba0 msgr2=0x7f786c06c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:14.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.270+0000 7f786ad9d700 1 --2- 172.21.15.143:0/3351305416 >> v2:172.21.15.174:3300/0 conn(0x7f786c15dba0 0x7f786c06c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:14.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.270+0000 7f786ad9d700 1 -- 172.21.15.143:0/3351305416 >> v1:172.21.15.174:6789/0 conn(0x7f786c06b240 legacy=0x7f786c0caff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:14.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.270+0000 7f786ad9d700 1 -- 172.21.15.143:0/3351305416 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f786c06ba20 con 0x7f786c0b22e0 2023-11-08T19:31:14.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.270+0000 7f786ad9d700 1 --2- 172.21.15.143:0/3351305416 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f786c0b22e0 0x7f786c0b2700 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f786c067460 tx=0x7f786400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16f10eabfc309ae7 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:14.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.270+0000 7f786959a700 1 -- 172.21.15.143:0/3351305416 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f786400e070 con 0x7f786c0b22e0 2023-11-08T19:31:14.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.270+0000 7f786959a700 1 -- 172.21.15.143:0/3351305416 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7864003a20 con 0x7f786c0b22e0 2023-11-08T19:31:14.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.270+0000 7f786959a700 1 -- 172.21.15.143:0/3351305416 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7864003dd0 con 0x7f786c0b22e0 2023-11-08T19:31:14.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.271+0000 7f7870868700 1 -- 172.21.15.143:0/3351305416 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f786c0b22e0 msgr2=0x7f786c0b2700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:14.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.271+0000 7f7870868700 1 --2- 172.21.15.143:0/3351305416 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f786c0b22e0 0x7f786c0b2700 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f786c067460 tx=0x7f786400b330 comp rx=0 tx=0).stop 2023-11-08T19:31:14.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.271+0000 7f7870868700 1 -- 172.21.15.143:0/3351305416 shutdown_connections 2023-11-08T19:31:14.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.271+0000 7f7870868700 1 --2- 172.21.15.143:0/3351305416 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f786c0b22e0 0x7f786c0b2700 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:14.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.271+0000 7f7870868700 1 --2- 172.21.15.143:0/3351305416 >> v2:172.21.15.174:3300/0 conn(0x7f786c15dba0 0x7f786c06c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:14.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.271+0000 7f7870868700 1 -- 172.21.15.143:0/3351305416 >> 172.21.15.143:0/3351305416 conn(0x7f786c0cd2e0 msgr2=0x7f786c0cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:14.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.271+0000 7f7870868700 1 -- 172.21.15.143:0/3351305416 shutdown_connections 2023-11-08T19:31:14.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.271+0000 7f7870868700 1 -- 172.21.15.143:0/3351305416 wait complete. 2023-11-08T19:31:14.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.273+0000 7f7870868700 1 Processor -- start 2023-11-08T19:31:14.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.273+0000 7f7870868700 1 -- start start 2023-11-08T19:31:14.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.274+0000 7f7870868700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f786c06b240 0x7f786c0ad500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:14.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.274+0000 7f7870868700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f786c0b22e0 0x7f786c0add00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:14.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.274+0000 7f7870868700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f786c15dba0 0x7f786c158f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:14.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.274+0000 7f7870868700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f786c0c8370 con 0x7f786c15dba0 2023-11-08T19:31:14.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.274+0000 7f7870868700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f786c0ae240 con 0x7f786c0b22e0 2023-11-08T19:31:14.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.274+0000 7f7870868700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f786c159450 con 0x7f786c06b240 2023-11-08T19:31:14.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.274+0000 7f786ad9d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f786c15dba0 0x7f786c158f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:14.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.274+0000 7f786ad9d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f786c15dba0 0x7f786c158f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57078/0 (socket says 172.21.15.143:57078) 2023-11-08T19:31:14.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.274+0000 7f786ad9d700 1 -- 172.21.15.143:0/2880640175 learned_addr learned my addr 172.21.15.143:0/2880640175 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:31:14.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.274+0000 7f786a59c700 1 --2- 172.21.15.143:0/2880640175 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f786c06b240 0x7f786c0ad500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:14.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.274+0000 7f7869d9b700 1 --2- 172.21.15.143:0/2880640175 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f786c0b22e0 0x7f786c0add00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:14.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.274+0000 7f786ad9d700 1 -- 172.21.15.143:0/2880640175 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f786c06b240 msgr2=0x7f786c0ad500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:14.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.274+0000 7f786ad9d700 1 --2- 172.21.15.143:0/2880640175 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f786c06b240 0x7f786c0ad500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:14.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.274+0000 7f786ad9d700 1 -- 172.21.15.143:0/2880640175 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f786c0b22e0 msgr2=0x7f786c0add00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:14.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.274+0000 7f786ad9d700 1 --2- 172.21.15.143:0/2880640175 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f786c0b22e0 0x7f786c0add00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:14.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.274+0000 7f786ad9d700 1 -- 172.21.15.143:0/2880640175 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7864009b30 con 0x7f786c15dba0 2023-11-08T19:31:14.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.274+0000 7f786ad9d700 1 --2- 172.21.15.143:0/2880640175 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f786c15dba0 0x7f786c158f10 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f7864015040 tx=0x7f7864004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:14.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.274+0000 7f785affd700 1 -- 172.21.15.143:0/2880640175 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7864004440 con 0x7f786c15dba0 2023-11-08T19:31:14.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.274+0000 7f785affd700 1 -- 172.21.15.143:0/2880640175 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78640092e0 con 0x7f786c15dba0 2023-11-08T19:31:14.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.274+0000 7f7870868700 1 -- 172.21.15.143:0/2880640175 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f786c0c79d0 con 0x7f786c15dba0 2023-11-08T19:31:14.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.274+0000 7f785affd700 1 -- 172.21.15.143:0/2880640175 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7864004800 con 0x7f786c15dba0 2023-11-08T19:31:14.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.275+0000 7f7870868700 1 -- 172.21.15.143:0/2880640175 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f786c0c7f90 con 0x7f786c15dba0 2023-11-08T19:31:14.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.276+0000 7f7870868700 1 -- 172.21.15.143:0/2880640175 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f786c0c84b0 con 0x7f786c15dba0 2023-11-08T19:31:14.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.276+0000 7f785affd700 1 -- 172.21.15.143:0/2880640175 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f786401c440 con 0x7f786c15dba0 2023-11-08T19:31:14.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.276+0000 7f785affd700 1 --2- 172.21.15.143:0/2880640175 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f784c06af00 0x7f784c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:14.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.276+0000 7f785affd700 1 -- 172.21.15.143:0/2880640175 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6651+0+0 (secure 0 0 0) 0x7f786408ab00 con 0x7f786c15dba0 2023-11-08T19:31:14.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.277+0000 7f786a59c700 1 --2- 172.21.15.143:0/2880640175 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f784c06af00 0x7f784c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:14.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.277+0000 7f786a59c700 1 --2- 172.21.15.143:0/2880640175 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f784c06af00 0x7f784c06d3c0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f785c006b20 tx=0x7f785c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:14.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.281+0000 7f785affd700 1 -- 172.21.15.143:0/2880640175 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f786c0c84b0 con 0x7f786c15dba0 2023-11-08T19:31:14.470 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.468+0000 7f7870868700 1 -- 172.21.15.143:0/2880640175 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f786c0c84b0 con 0x7f786c15dba0 2023-11-08T19:31:14.473 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.472+0000 7f785affd700 1 -- 172.21.15.143:0/2880640175 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v20) v1 ==== 104+0+0 (secure 0 0 0) 0x7f786c0c84b0 con 0x7f786c15dba0 2023-11-08T19:31:14.475 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.474+0000 7f7870868700 1 -- 172.21.15.143:0/2880640175 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f784c06af00 msgr2=0x7f784c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:14.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.474+0000 7f7870868700 1 --2- 172.21.15.143:0/2880640175 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f784c06af00 0x7f784c06d3c0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f785c006b20 tx=0x7f785c008040 comp rx=0 tx=0).stop 2023-11-08T19:31:14.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.474+0000 7f7870868700 1 -- 172.21.15.143:0/2880640175 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f786c15dba0 msgr2=0x7f786c158f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:14.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.474+0000 7f7870868700 1 --2- 172.21.15.143:0/2880640175 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f786c15dba0 0x7f786c158f10 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f7864015040 tx=0x7f7864004ad0 comp rx=0 tx=0).stop 2023-11-08T19:31:14.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.474+0000 7f7870868700 1 -- 172.21.15.143:0/2880640175 shutdown_connections 2023-11-08T19:31:14.477 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.474+0000 7f7870868700 1 --2- 172.21.15.143:0/2880640175 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f784c06af00 0x7f784c06d3c0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:14.477 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.474+0000 7f7870868700 1 --2- 172.21.15.143:0/2880640175 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f786c06b240 0x7f786c0ad500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:14.477 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.474+0000 7f7870868700 1 --2- 172.21.15.143:0/2880640175 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f786c15dba0 0x7f786c158f10 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:14.477 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.474+0000 7f7870868700 1 --2- 172.21.15.143:0/2880640175 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f786c0b22e0 0x7f786c0add00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:14.477 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.474+0000 7f7870868700 1 -- 172.21.15.143:0/2880640175 >> 172.21.15.143:0/2880640175 conn(0x7f786c0cd2e0 msgr2=0x7f786c0cc090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:14.478 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.474+0000 7f7870868700 1 -- 172.21.15.143:0/2880640175 shutdown_connections 2023-11-08T19:31:14.478 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.474+0000 7f7870868700 1 -- 172.21.15.143:0/2880640175 wait complete. 2023-11-08T19:31:14.487 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snapshots.TestSnapshots.test_kill_mdstable' 2023-11-08T19:31:14.608 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.607+0000 7f9a49841700 1 Processor -- start 2023-11-08T19:31:14.609 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.607+0000 7f9a49841700 1 -- start start 2023-11-08T19:31:14.611 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.608+0000 7f9a49841700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f9a4415dc80 0x7f9a440d1f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:14.611 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.608+0000 7f9a49841700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9a440d3690 0x7f9a440b9690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:14.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.608+0000 7f9a49841700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a440b7cf0 con 0x7f9a440ce980 2023-11-08T19:31:14.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.608+0000 7f9a49841700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a440b9bd0 con 0x7f9a4415dc80 2023-11-08T19:31:14.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.608+0000 7f9a49841700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a440d3200 con 0x7f9a440d3690 2023-11-08T19:31:14.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.608+0000 7f9a427fc700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9a440d3690 0x7f9a440b9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:14.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.608+0000 7f9a42ffd700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f9a4415dc80 0x7f9a440d1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:14.613 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.608+0000 7f9a42ffd700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f9a4415dc80 0x7f9a440d1f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.143:40580/0 (socket says 172.21.15.143:40580) 2023-11-08T19:31:14.613 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.608+0000 7f9a427fc700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9a440d3690 0x7f9a440b9690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.143:44720/0 (socket says 172.21.15.143:44720) 2023-11-08T19:31:14.613 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.608+0000 7f9a42ffd700 1 -- 172.21.15.143:0/2379349616 learned_addr learned my addr 172.21.15.143:0/2379349616 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:31:14.613 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.608+0000 7f9a437fe700 1 --1- 172.21.15.143:0/2379349616 >> v1:172.21.15.174:6789/0 conn(0x7f9a440ce980 0x7f9a440b5630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.174:6789/0 says I am v1:172.21.15.143:35942/0 (socket says 172.21.15.143:35942) 2023-11-08T19:31:14.613 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.609+0000 7f9a427fc700 1 -- 172.21.15.143:0/2379349616 >> v2:172.21.15.174:3300/0 conn(0x7f9a4415dc80 msgr2=0x7f9a440d1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:14.614 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.609+0000 7f9a427fc700 1 --2- 172.21.15.143:0/2379349616 >> v2:172.21.15.174:3300/0 conn(0x7f9a4415dc80 0x7f9a440d1f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:14.614 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.609+0000 7f9a427fc700 1 -- 172.21.15.143:0/2379349616 >> v1:172.21.15.174:6789/0 conn(0x7f9a440ce980 legacy=0x7f9a440b5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:14.614 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.609+0000 7f9a427fc700 1 -- 172.21.15.143:0/2379349616 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a4415e060 con 0x7f9a440d3690 2023-11-08T19:31:14.614 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.609+0000 7f9a41ffb700 1 -- 172.21.15.143:0/2379349616 <== mon.1 v1:172.21.15.174:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 220325891 0 0) 0x7f9a3c001490 con 0x7f9a440ce980 2023-11-08T19:31:14.614 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.609+0000 7f9a42ffd700 1 --2- 172.21.15.143:0/2379349616 >> v2:172.21.15.174:3300/0 conn(0x7f9a4415dc80 0x7f9a440d1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T19:31:14.615 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.609+0000 7f9a427fc700 1 --2- 172.21.15.143:0/2379349616 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9a440d3690 0x7f9a440b9690 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f9a34009840 tx=0x7f9a34009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eea7c23cd4e9f5c server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:14.615 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.609+0000 7f9a41ffb700 1 -- 172.21.15.143:0/2379349616 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a34011830 con 0x7f9a440d3690 2023-11-08T19:31:14.615 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.609+0000 7f9a41ffb700 1 -- 172.21.15.143:0/2379349616 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a34011990 con 0x7f9a440d3690 2023-11-08T19:31:14.615 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.609+0000 7f9a41ffb700 1 -- 172.21.15.143:0/2379349616 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a34011d20 con 0x7f9a440d3690 2023-11-08T19:31:14.616 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.610+0000 7f9a49841700 1 -- 172.21.15.143:0/2379349616 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9a440d3690 msgr2=0x7f9a440b9690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:14.616 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.610+0000 7f9a49841700 1 --2- 172.21.15.143:0/2379349616 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9a440d3690 0x7f9a440b9690 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f9a34009840 tx=0x7f9a34009b50 comp rx=0 tx=0).stop 2023-11-08T19:31:14.616 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.610+0000 7f9a49841700 1 -- 172.21.15.143:0/2379349616 shutdown_connections 2023-11-08T19:31:14.616 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.610+0000 7f9a49841700 1 --2- 172.21.15.143:0/2379349616 >> v2:172.21.15.174:3300/0 conn(0x7f9a4415dc80 0x7f9a440d1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:14.616 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.610+0000 7f9a49841700 1 --2- 172.21.15.143:0/2379349616 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9a440d3690 0x7f9a440b9690 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:14.617 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.610+0000 7f9a49841700 1 -- 172.21.15.143:0/2379349616 >> 172.21.15.143:0/2379349616 conn(0x7f9a440668e0 msgr2=0x7f9a44066cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:14.617 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.610+0000 7f9a49841700 1 -- 172.21.15.143:0/2379349616 shutdown_connections 2023-11-08T19:31:14.617 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.610+0000 7f9a49841700 1 -- 172.21.15.143:0/2379349616 wait complete. 2023-11-08T19:31:14.620 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.611+0000 7f9a49841700 1 Processor -- start 2023-11-08T19:31:14.620 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.611+0000 7f9a49841700 1 -- start start 2023-11-08T19:31:14.620 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.612+0000 7f9a49841700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9a440ce980 0x7f9a441f6c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:14.620 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.612+0000 7f9a49841700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9a440d3690 0x7f9a441f7140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:14.620 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.612+0000 7f9a49841700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9a4415dc80 0x7f9a44145680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:14.621 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.612+0000 7f9a49841700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a4415fc90 con 0x7f9a440ce980 2023-11-08T19:31:14.621 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.612+0000 7f9a49841700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a441f34c0 con 0x7f9a4415dc80 2023-11-08T19:31:14.621 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.612+0000 7f9a49841700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a4415ff30 con 0x7f9a440d3690 2023-11-08T19:31:14.621 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.612+0000 7f9a42ffd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9a440ce980 0x7f9a441f6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:14.622 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.612+0000 7f9a42ffd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9a440ce980 0x7f9a441f6c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57088/0 (socket says 172.21.15.143:57088) 2023-11-08T19:31:14.622 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.612+0000 7f9a42ffd700 1 -- 172.21.15.143:0/2030429954 learned_addr learned my addr 172.21.15.143:0/2030429954 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:31:14.622 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.613+0000 7f9a427fc700 1 --2- 172.21.15.143:0/2030429954 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9a440d3690 0x7f9a441f7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:14.622 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.613+0000 7f9a42ffd700 1 -- 172.21.15.143:0/2030429954 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9a440d3690 msgr2=0x7f9a441f7140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:14.623 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.613+0000 7f9a42ffd700 1 --2- 172.21.15.143:0/2030429954 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9a440d3690 0x7f9a441f7140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:14.623 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.613+0000 7f9a42ffd700 1 -- 172.21.15.143:0/2030429954 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9a4415dc80 msgr2=0x7f9a44145680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:14.623 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.613+0000 7f9a42ffd700 1 --2- 172.21.15.143:0/2030429954 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9a4415dc80 0x7f9a44145680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:14.623 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.613+0000 7f9a42ffd700 1 -- 172.21.15.143:0/2030429954 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a38009550 con 0x7f9a440ce980 2023-11-08T19:31:14.623 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.613+0000 7f9a42ffd700 1 --2- 172.21.15.143:0/2030429954 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9a440ce980 0x7f9a441f6c00 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f9a3800e9b0 tx=0x7f9a3800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:14.623 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.613+0000 7f9a2bfff700 1 -- 172.21.15.143:0/2030429954 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a3800c850 con 0x7f9a440ce980 2023-11-08T19:31:14.624 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.613+0000 7f9a2bfff700 1 -- 172.21.15.143:0/2030429954 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a3800c9b0 con 0x7f9a440ce980 2023-11-08T19:31:14.624 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.613+0000 7f9a49841700 1 -- 172.21.15.143:0/2030429954 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a340094a0 con 0x7f9a440ce980 2023-11-08T19:31:14.624 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.613+0000 7f9a2bfff700 1 -- 172.21.15.143:0/2030429954 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a3800cd20 con 0x7f9a440ce980 2023-11-08T19:31:14.624 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.613+0000 7f9a49841700 1 -- 172.21.15.143:0/2030429954 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a441604f0 con 0x7f9a440ce980 2023-11-08T19:31:14.624 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.614+0000 7f9a49841700 1 -- 172.21.15.143:0/2030429954 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a440caee0 con 0x7f9a440ce980 2023-11-08T19:31:14.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.614+0000 7f9a2bfff700 1 -- 172.21.15.143:0/2030429954 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9a38013470 con 0x7f9a440ce980 2023-11-08T19:31:14.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.615+0000 7f9a2bfff700 1 --2- 172.21.15.143:0/2030429954 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f9a3006af00 0x7f9a3006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:14.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.615+0000 7f9a2bfff700 1 -- 172.21.15.143:0/2030429954 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6651+0+0 (secure 0 0 0) 0x7f9a38089b00 con 0x7f9a440ce980 2023-11-08T19:31:14.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.615+0000 7f9a427fc700 1 --2- 172.21.15.143:0/2030429954 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f9a3006af00 0x7f9a3006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:14.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.615+0000 7f9a427fc700 1 --2- 172.21.15.143:0/2030429954 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f9a3006af00 0x7f9a3006d3c0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f9a440d3a90 tx=0x7f9a34008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:14.626 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.619+0000 7f9a2bfff700 1 -- 172.21.15.143:0/2030429954 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9a38054800 con 0x7f9a440ce980 2023-11-08T19:31:14.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:14.891+0000 7f9a49841700 1 -- 172.21.15.143:0/2030429954 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snapshots.TestSnapshots.test_kill_mdstable"]} v 0) v1 -- 0x7f9a4404eee0 con 0x7f9a440ce980 2023-11-08T19:31:15.008 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:15.007+0000 7f9a2bfff700 1 -- 172.21.15.143:0/2030429954 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snapshots.TestSnapshots.test_kill_mdstable"]}]=0 v348) v1 ==== 139+0+0 (secure 0 0 0) 0x7f9a380170d0 con 0x7f9a440ce980 2023-11-08T19:31:15.011 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:15.009+0000 7f9a49841700 1 -- 172.21.15.143:0/2030429954 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f9a3006af00 msgr2=0x7f9a3006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:15.011 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:15.009+0000 7f9a49841700 1 --2- 172.21.15.143:0/2030429954 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f9a3006af00 0x7f9a3006d3c0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f9a440d3a90 tx=0x7f9a34008030 comp rx=0 tx=0).stop 2023-11-08T19:31:15.011 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:15.009+0000 7f9a49841700 1 -- 172.21.15.143:0/2030429954 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9a440ce980 msgr2=0x7f9a441f6c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:15.011 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:15.009+0000 7f9a49841700 1 --2- 172.21.15.143:0/2030429954 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9a440ce980 0x7f9a441f6c00 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f9a3800e9b0 tx=0x7f9a3800ed70 comp rx=0 tx=0).stop 2023-11-08T19:31:15.012 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:15.009+0000 7f9a49841700 1 -- 172.21.15.143:0/2030429954 shutdown_connections 2023-11-08T19:31:15.012 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:15.009+0000 7f9a49841700 1 --2- 172.21.15.143:0/2030429954 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f9a3006af00 0x7f9a3006d3c0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:15.012 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:15.009+0000 7f9a49841700 1 --2- 172.21.15.143:0/2030429954 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9a4415dc80 0x7f9a44145680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:15.012 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:15.009+0000 7f9a49841700 1 --2- 172.21.15.143:0/2030429954 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9a440ce980 0x7f9a441f6c00 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:15.012 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:15.009+0000 7f9a49841700 1 --2- 172.21.15.143:0/2030429954 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9a440d3690 0x7f9a441f7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:15.013 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:15.009+0000 7f9a49841700 1 -- 172.21.15.143:0/2030429954 >> 172.21.15.143:0/2030429954 conn(0x7f9a440668e0 msgr2=0x7f9a440d42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:15.013 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:15.009+0000 7f9a49841700 1 -- 172.21.15.143:0/2030429954 shutdown_connections 2023-11-08T19:31:15.013 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:15.009+0000 7f9a49841700 1 -- 172.21.15.143:0/2030429954 wait complete. 2023-11-08T19:31:15.023 INFO:tasks.cephfs_test_runner:check snaptable transcation ... ERROR 2023-11-08T19:31:15.024 INFO:tasks.cephfs_test_runner: 2023-11-08T19:31:15.024 INFO:tasks.cephfs_test_runner:====================================================================== 2023-11-08T19:31:15.024 INFO:tasks.cephfs_test_runner:ERROR: test_kill_mdstable (tasks.cephfs.test_snapshots.TestSnapshots) 2023-11-08T19:31:15.024 INFO:tasks.cephfs_test_runner:check snaptable transcation 2023-11-08T19:31:15.025 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-11-08T19:31:15.025 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-11-08T19:31:15.025 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_batrick_ceph_e797689ab88984658d73bbc928b2ed06c676d14a/qa/tasks/cephfs/test_snapshots.py", line 149, in test_kill_mdstable 2023-11-08T19:31:15.025 INFO:tasks.cephfs_test_runner: proc.wait() 2023-11-08T19:31:15.026 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 161, in wait 2023-11-08T19:31:15.026 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2023-11-08T19:31:15.026 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 181, in _raise_for_status 2023-11-08T19:31:15.026 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2023-11-08T19:31:15.027 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi143 with status 1: '(cd /home/ubuntu/cephtest/mnt.0 && exec mkdir d1/dir/.snap/s22)' 2023-11-08T19:31:15.027 INFO:tasks.cephfs_test_runner: 2023-11-08T19:31:15.027 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-11-08T19:31:15.028 INFO:tasks.cephfs_test_runner:Ran 5 tests in 351.022s 2023-11-08T19:31:15.028 INFO:tasks.cephfs_test_runner: 2023-11-08T19:31:15.028 INFO:tasks.cephfs_test_runner:FAILED (errors=1) 2023-11-08T19:31:15.028 INFO:tasks.cephfs_test_runner: 2023-11-08T19:31:15.029 INFO:tasks.cephfs_test_runner:====================================================================== 2023-11-08T19:31:15.029 INFO:tasks.cephfs_test_runner:ERROR: test_kill_mdstable (tasks.cephfs.test_snapshots.TestSnapshots) 2023-11-08T19:31:15.029 INFO:tasks.cephfs_test_runner:check snaptable transcation 2023-11-08T19:31:15.029 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-11-08T19:31:15.030 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-11-08T19:31:15.030 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_batrick_ceph_e797689ab88984658d73bbc928b2ed06c676d14a/qa/tasks/cephfs/test_snapshots.py", line 149, in test_kill_mdstable 2023-11-08T19:31:15.030 INFO:tasks.cephfs_test_runner: proc.wait() 2023-11-08T19:31:15.031 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 161, in wait 2023-11-08T19:31:15.031 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2023-11-08T19:31:15.031 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 181, in _raise_for_status 2023-11-08T19:31:15.031 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2023-11-08T19:31:15.032 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi143 with status 1: '(cd /home/ubuntu/cephtest/mnt.0 && exec mkdir d1/dir/.snap/s22)' 2023-11-08T19:31:15.032 INFO:tasks.cephfs_test_runner: 2023-11-08T19:31:15.032 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_batrick_ceph_e797689ab88984658d73bbc928b2ed06c676d14a/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_kill_mdstable (tasks.cephfs.test_snapshots.TestSnapshots) 2023-11-08T19:31:15.107 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=a1594914561d41c4bf17575f28574757 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_batrick_ceph_e797689ab88984658d73bbc928b2ed06c676d14a/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_kill_mdstable (tasks.cephfs.test_snapshots.TestSnapshots) 2023-11-08T19:31:15.110 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2023-11-08T19:31:15.134 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2023-11-08T19:31:15.144 INFO:tasks.kclient:Unmounting kernel clients... 2023-11-08T19:31:15.144 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-08T19:31:15.145 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:31:15.166 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284264k,nr_inodes=4071066,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=44,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20643 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-08T19:31:15.167 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-11-08T19:31:15.167 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:31:15.224 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284264k,nr_inodes=4071066,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=44,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20643 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-08T19:31:15.225 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2023-11-08T19:31:15.225 DEBUG:teuthology.orchestra.run.smithi174:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:31:15.248 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284224k,nr_inodes=4071056,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=31,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19549 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260544k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-08T19:31:15.248 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2023-11-08T19:31:15.249 DEBUG:teuthology.orchestra.run.smithi174:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:31:15.308 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284224k,nr_inodes=4071056,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=31,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19549 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260544k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-08T19:31:15.309 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:31:15.310 DEBUG:teuthology.orchestra.run.smithi143:> ip netns list 2023-11-08T19:31:15.327 INFO:teuthology.orchestra.run.smithi143.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-08T19:31:15.327 INFO:teuthology.orchestra.run.smithi143.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-08T19:31:15.327 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:31:15.328 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-08T19:31:15.405 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:31:15.406 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-08T19:31:15.492 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:31:15.493 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip link delete ceph-brx 2023-11-08T19:31:15.625 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:31:15.626 DEBUG:teuthology.orchestra.run.smithi143:> ip netns list 2023-11-08T19:31:15.644 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:31:15.644 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip link delete ceph-brx 2023-11-08T19:31:15.714 INFO:teuthology.orchestra.run.smithi143.stderr:Cannot find device "ceph-brx" 2023-11-08T19:31:15.715 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-08T19:31:15.716 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:31:15.716 DEBUG:teuthology.orchestra.run.smithi174:> ip netns list 2023-11-08T19:31:15.737 INFO:teuthology.orchestra.run.smithi174.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-11-08T19:31:15.737 INFO:teuthology.orchestra.run.smithi174.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-08T19:31:15.738 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:31:15.738 DEBUG:teuthology.orchestra.run.smithi174:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-08T19:31:15.819 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:31:15.820 DEBUG:teuthology.orchestra.run.smithi174:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-08T19:31:15.910 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:31:15.910 DEBUG:teuthology.orchestra.run.smithi174:> sudo ip link delete ceph-brx 2023-11-08T19:31:16.001 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:31:16.001 DEBUG:teuthology.orchestra.run.smithi174:> ip netns list 2023-11-08T19:31:16.019 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:31:16.020 DEBUG:teuthology.orchestra.run.smithi174:> sudo ip link delete ceph-brx 2023-11-08T19:31:16.091 INFO:teuthology.orchestra.run.smithi174.stderr:Cannot find device "ceph-brx" 2023-11-08T19:31:16.091 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-08T19:31:16.092 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-11-08T19:31:16.101 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-11-08T19:31:16.102 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-08T19:31:16.226 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.225+0000 7fadc6def700 1 Processor -- start 2023-11-08T19:31:16.227 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.225+0000 7fadc6def700 1 -- start start 2023-11-08T19:31:16.229 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.226+0000 7fadc6def700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fadc015f510 0x7fadc00b24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:16.229 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.226+0000 7fadc6def700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fadc00b1100 0x7fadc00b1530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:16.230 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.226+0000 7fadc6def700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadc00b1a70 con 0x7fadc015ba40 2023-11-08T19:31:16.230 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.226+0000 7fadc6def700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadc015f970 con 0x7fadc015f510 2023-11-08T19:31:16.230 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.226+0000 7fadc6def700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadc015be20 con 0x7fadc00b1100 2023-11-08T19:31:16.230 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.226+0000 7fadbffff700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fadc015f510 0x7fadc00b24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:16.231 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.226+0000 7fadc4b8b700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fadc015ba40 0x7fadc006e470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:53250/0 (socket says 172.21.15.143:53250) 2023-11-08T19:31:16.231 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.226+0000 7fadc4b8b700 1 -- 172.21.15.143:0/126693272 learned_addr learned my addr 172.21.15.143:0/126693272 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:31:16.231 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.226+0000 7fadc538c700 1 --2- 172.21.15.143:0/126693272 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fadc00b1100 0x7fadc00b1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:16.231 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.226+0000 7fadbffff700 1 -- 172.21.15.143:0/126693272 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fadc00b1100 msgr2=0x7fadc00b1530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:16.232 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.226+0000 7fadbffff700 1 --2- 172.21.15.143:0/126693272 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fadc00b1100 0x7fadc00b1530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:16.232 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.226+0000 7fadbffff700 1 -- 172.21.15.143:0/126693272 >> v1:172.21.15.143:6789/0 conn(0x7fadc015ba40 legacy=0x7fadc006e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:16.232 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.226+0000 7fadbffff700 1 -- 172.21.15.143:0/126693272 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadc0070f00 con 0x7fadc015f510 2023-11-08T19:31:16.233 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.226+0000 7fadbf7fe700 1 -- 172.21.15.143:0/126693272 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 686450099 0 0) 0x7fadc00b1a70 con 0x7fadc015ba40 2023-11-08T19:31:16.233 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.226+0000 7fadc538c700 1 --2- 172.21.15.143:0/126693272 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fadc00b1100 0x7fadc00b1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:31:16.233 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.226+0000 7fadbffff700 1 --2- 172.21.15.143:0/126693272 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fadc015f510 0x7fadc00b24d0 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7fadac009840 tx=0x7fadac009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1f341b42af381e0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:16.233 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.226+0000 7fadbf7fe700 1 -- 172.21.15.143:0/126693272 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadac011830 con 0x7fadc015f510 2023-11-08T19:31:16.234 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.227+0000 7fadbf7fe700 1 -- 172.21.15.143:0/126693272 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadac011990 con 0x7fadc015f510 2023-11-08T19:31:16.234 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.227+0000 7fadbf7fe700 1 -- 172.21.15.143:0/126693272 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadac011d20 con 0x7fadc015f510 2023-11-08T19:31:16.234 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.227+0000 7fadc6def700 1 -- 172.21.15.143:0/126693272 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fadc015f510 msgr2=0x7fadc00b24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:16.234 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.227+0000 7fadc6def700 1 --2- 172.21.15.143:0/126693272 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fadc015f510 0x7fadc00b24d0 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7fadac009840 tx=0x7fadac009b50 comp rx=0 tx=0).stop 2023-11-08T19:31:16.235 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.227+0000 7fadc6def700 1 -- 172.21.15.143:0/126693272 shutdown_connections 2023-11-08T19:31:16.235 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.227+0000 7fadc6def700 1 --2- 172.21.15.143:0/126693272 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fadc00b1100 0x7fadc00b1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:16.235 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.227+0000 7fadc6def700 1 --2- 172.21.15.143:0/126693272 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fadc015f510 0x7fadc00b24d0 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:16.236 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.227+0000 7fadc6def700 1 -- 172.21.15.143:0/126693272 >> 172.21.15.143:0/126693272 conn(0x7fadc006b4c0 msgr2=0x7fadc00cf680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:16.236 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.227+0000 7fadc6def700 1 -- 172.21.15.143:0/126693272 shutdown_connections 2023-11-08T19:31:16.236 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.227+0000 7fadc6def700 1 -- 172.21.15.143:0/126693272 wait complete. 2023-11-08T19:31:16.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.228+0000 7fadc6def700 1 Processor -- start 2023-11-08T19:31:16.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.228+0000 7fadc6def700 1 -- start start 2023-11-08T19:31:16.240 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.229+0000 7fadc6def700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fadc00b1100 0x7fadc01f44f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:16.240 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.229+0000 7fadc6def700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fadc015ba40 0x7fadc01f1570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:16.240 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.229+0000 7fadc6def700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fadc015f510 0x7fadc00c7490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:16.240 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.229+0000 7fadc6def700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadc015d3f0 con 0x7fadc00b1100 2023-11-08T19:31:16.241 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.229+0000 7fadc6def700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadc00c7d90 con 0x7fadc015ba40 2023-11-08T19:31:16.241 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.229+0000 7fadc6def700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadc015d640 con 0x7fadc015f510 2023-11-08T19:31:16.241 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.229+0000 7fadc4b8b700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fadc00b1100 0x7fadc01f44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:16.241 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.229+0000 7fadc4b8b700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fadc00b1100 0x7fadc01f44f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57102/0 (socket says 172.21.15.143:57102) 2023-11-08T19:31:16.242 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.229+0000 7fadc4b8b700 1 -- 172.21.15.143:0/3836766866 learned_addr learned my addr 172.21.15.143:0/3836766866 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:31:16.242 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.229+0000 7fadc538c700 1 --2- 172.21.15.143:0/3836766866 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fadc015f510 0x7fadc00c7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:16.242 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.230+0000 7fadbffff700 1 --2- 172.21.15.143:0/3836766866 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fadc015ba40 0x7fadc01f1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:16.243 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.230+0000 7fadc4b8b700 1 -- 172.21.15.143:0/3836766866 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fadc015f510 msgr2=0x7fadc00c7490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:16.243 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.230+0000 7fadc4b8b700 1 --2- 172.21.15.143:0/3836766866 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fadc015f510 0x7fadc00c7490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:16.243 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.230+0000 7fadc4b8b700 1 -- 172.21.15.143:0/3836766866 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fadc015ba40 msgr2=0x7fadc01f1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:16.243 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.230+0000 7fadc4b8b700 1 --2- 172.21.15.143:0/3836766866 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fadc015ba40 0x7fadc01f1570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:16.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.230+0000 7fadc4b8b700 1 -- 172.21.15.143:0/3836766866 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadac0094a0 con 0x7fadc00b1100 2023-11-08T19:31:16.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.230+0000 7fadbffff700 1 --2- 172.21.15.143:0/3836766866 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fadc015ba40 0x7fadc01f1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:31:16.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.230+0000 7fadc4b8b700 1 --2- 172.21.15.143:0/3836766866 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fadc00b1100 0x7fadc01f44f0 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7fadb40011e0 tx=0x7fadb400ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:16.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.230+0000 7fadbd7fa700 1 -- 172.21.15.143:0/3836766866 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadb4004c80 con 0x7fadc00b1100 2023-11-08T19:31:16.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.230+0000 7fadbd7fa700 1 -- 172.21.15.143:0/3836766866 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadb4004de0 con 0x7fadc00b1100 2023-11-08T19:31:16.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.230+0000 7fadc6def700 1 -- 172.21.15.143:0/3836766866 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadc015d920 con 0x7fadc00b1100 2023-11-08T19:31:16.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.230+0000 7fadbd7fa700 1 -- 172.21.15.143:0/3836766866 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadb4007310 con 0x7fadc00b1100 2023-11-08T19:31:16.246 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.230+0000 7fadc6def700 1 -- 172.21.15.143:0/3836766866 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadc00c79d0 con 0x7fadc00b1100 2023-11-08T19:31:16.246 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.231+0000 7fadc6def700 1 -- 172.21.15.143:0/3836766866 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadc015de40 con 0x7fadc00b1100 2023-11-08T19:31:16.246 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.231+0000 7fadbd7fa700 1 -- 172.21.15.143:0/3836766866 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fadb4007470 con 0x7fadc00b1100 2023-11-08T19:31:16.246 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.232+0000 7fadbd7fa700 1 --2- 172.21.15.143:0/3836766866 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fada406af00 0x7fada406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:16.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.232+0000 7fadbd7fa700 1 -- 172.21.15.143:0/3836766866 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6651+0+0 (secure 0 0 0) 0x7fadb40895d0 con 0x7fadc00b1100 2023-11-08T19:31:16.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.232+0000 7fadbffff700 1 --2- 172.21.15.143:0/3836766866 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fada406af00 0x7fada406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:16.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.232+0000 7fadbffff700 1 --2- 172.21.15.143:0/3836766866 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fada406af00 0x7fada406d3c0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fadc00b4c40 tx=0x7fadac008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:16.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.236+0000 7fadbd7fa700 1 -- 172.21.15.143:0/3836766866 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fadc015de40 con 0x7fadc00b1100 2023-11-08T19:31:16.423 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.422+0000 7fadc6def700 1 -- 172.21.15.143:0/3836766866 --> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fadc006e9f0 con 0x7fada406af00 2023-11-08T19:31:16.432 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.431+0000 7fadbd7fa700 1 -- 172.21.15.143:0/3836766866 <== mgr.4099 v2:172.21.15.174:6832/4037280178 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+293885 (secure 0 0 0) 0x7fadc006e9f0 con 0x7fada406af00 2023-11-08T19:31:16.436 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:31:16.439 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.433+0000 7fadc6def700 1 -- 172.21.15.143:0/3836766866 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fada406af00 msgr2=0x7fada406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:16.439 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.433+0000 7fadc6def700 1 --2- 172.21.15.143:0/3836766866 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fada406af00 0x7fada406d3c0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fadc00b4c40 tx=0x7fadac008030 comp rx=0 tx=0).stop 2023-11-08T19:31:16.440 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.433+0000 7fadc6def700 1 -- 172.21.15.143:0/3836766866 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fadc00b1100 msgr2=0x7fadc01f44f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:16.440 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.433+0000 7fadc6def700 1 --2- 172.21.15.143:0/3836766866 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fadc00b1100 0x7fadc01f44f0 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7fadb40011e0 tx=0x7fadb400ca30 comp rx=0 tx=0).stop 2023-11-08T19:31:16.441 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.434+0000 7fadc6def700 1 -- 172.21.15.143:0/3836766866 shutdown_connections 2023-11-08T19:31:16.441 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.434+0000 7fadc6def700 1 --2- 172.21.15.143:0/3836766866 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fada406af00 0x7fada406d3c0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:16.441 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.434+0000 7fadc6def700 1 --2- 172.21.15.143:0/3836766866 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fadc015f510 0x7fadc00c7490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:16.442 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.434+0000 7fadc6def700 1 --2- 172.21.15.143:0/3836766866 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fadc00b1100 0x7fadc01f44f0 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:16.442 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.434+0000 7fadc6def700 1 --2- 172.21.15.143:0/3836766866 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fadc015ba40 0x7fadc01f1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:16.442 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.434+0000 7fadc6def700 1 -- 172.21.15.143:0/3836766866 >> 172.21.15.143:0/3836766866 conn(0x7fadc006b4c0 msgr2=0x7fadc015ad50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:16.443 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.434+0000 7fadc6def700 1 -- 172.21.15.143:0/3836766866 shutdown_connections 2023-11-08T19:31:16.443 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.434+0000 7fadc6def700 1 -- 172.21.15.143:0/3836766866 wait complete. 2023-11-08T19:31:16.443 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2023-11-08T19:31:16.447 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":295,"stamp":"2023-11-08T19:31:14.623401+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":386,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":244264,"kb_used_data":11468,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754730456,"statfs":{"total":773094113280,"available":772843986944,"internally_reserved":0,"allocated":11743232,"data_stored":6374972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-67825,"num_objects":-41,"num_object_clones":0,"num_object_copies":-82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-41,"num_whiteouts":0,"num_read":-469,"num_read_kb":-599,"num_write":-178,"num_write_kb":-142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001769"},"pg_stats":[{"pgid":"12.d","version":"0'0","reported_seq":50,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T19:30:47.742203+0000","last_change":"2023-11-08T19:27:08.190826+0000","last_active":"2023-11-08T19:30:47.742203+0000","last_peered":"2023-11-08T19:30:47.742203+0000","last_clean":"2023-11-08T19:30:47.742203+0000","last_became_active":"2023-11-08T19:27:08.190440+0000","last_became_peered":"2023-11-08T19:27:08.190440+0000","last_unstale":"2023-11-08T19:30:47.742203+0000","last_undegraded":"2023-11-08T19:30:47.742203+0000","last_fullsized":"2023-11-08T19:30:47.742203+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:42:48.376462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":252,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T19:30:49.624415+0000","last_change":"2023-11-08T19:24:45.888122+0000","last_active":"2023-11-08T19:30:49.624415+0000","last_peered":"2023-11-08T19:30:49.624415+0000","last_clean":"2023-11-08T19:30:49.624415+0000","last_became_active":"2023-11-08T19:24:45.887048+0000","last_became_peered":"2023-11-08T19:24:45.887048+0000","last_unstale":"2023-11-08T19:30:49.624415+0000","last_undegraded":"2023-11-08T19:30:49.624415+0000","last_fullsized":"2023-11-08T19:30:49.624415+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:44.575133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:44.575133+0000","last_clean_scrub_stamp":"2023-11-08T19:24:44.575133+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:19:20.202062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":78,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.668985+0000","last_change":"2023-11-08T19:26:06.542044+0000","last_active":"2023-11-08T19:31:11.668985+0000","last_peered":"2023-11-08T19:31:11.668985+0000","last_clean":"2023-11-08T19:31:11.668985+0000","last_became_active":"2023-11-08T19:26:04.603056+0000","last_became_peered":"2023-11-08T19:26:04.603056+0000","last_unstale":"2023-11-08T19:31:11.668985+0000","last_undegraded":"2023-11-08T19:31:11.668985+0000","last_fullsized":"2023-11-08T19:31:11.668985+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:41:50.771457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000517414,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":57,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.667621+0000","last_change":"2023-11-08T19:27:09.496842+0000","last_active":"2023-11-08T19:31:11.667621+0000","last_peered":"2023-11-08T19:31:11.667621+0000","last_clean":"2023-11-08T19:31:11.667621+0000","last_became_active":"2023-11-08T19:27:07.185776+0000","last_became_peered":"2023-11-08T19:27:07.185776+0000","last_unstale":"2023-11-08T19:31:11.667621+0000","last_undegraded":"2023-11-08T19:31:11.667621+0000","last_fullsized":"2023-11-08T19:31:11.667621+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:21:13.923970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031747200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":50,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.668213+0000","last_change":"2023-11-08T19:27:08.188241+0000","last_active":"2023-11-08T19:31:11.668213+0000","last_peered":"2023-11-08T19:31:11.668213+0000","last_clean":"2023-11-08T19:31:11.668213+0000","last_became_active":"2023-11-08T19:27:08.187863+0000","last_became_peered":"2023-11-08T19:27:08.187863+0000","last_unstale":"2023-11-08T19:31:11.668213+0000","last_undegraded":"2023-11-08T19:31:11.668213+0000","last_fullsized":"2023-11-08T19:31:11.668213+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:45:22.084015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":76,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.667763+0000","last_change":"2023-11-08T19:26:06.265419+0000","last_active":"2023-11-08T19:31:11.667763+0000","last_peered":"2023-11-08T19:31:11.667763+0000","last_clean":"2023-11-08T19:31:11.667763+0000","last_became_active":"2023-11-08T19:26:04.605333+0000","last_became_peered":"2023-11-08T19:26:04.605333+0000","last_unstale":"2023-11-08T19:31:11.667763+0000","last_undegraded":"2023-11-08T19:31:11.667763+0000","last_fullsized":"2023-11-08T19:31:11.667763+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:01:56.991346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064606400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":57,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T19:30:47.742946+0000","last_change":"2023-11-08T19:27:09.182113+0000","last_active":"2023-11-08T19:30:47.742946+0000","last_peered":"2023-11-08T19:30:47.742946+0000","last_clean":"2023-11-08T19:30:47.742946+0000","last_became_active":"2023-11-08T19:27:07.189939+0000","last_became_peered":"2023-11-08T19:27:07.189939+0000","last_unstale":"2023-11-08T19:30:47.742946+0000","last_undegraded":"2023-11-08T19:30:47.742946+0000","last_fullsized":"2023-11-08T19:30:47.742946+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:41:08.822366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036507500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":46,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.444341+0000","last_change":"2023-11-08T19:27:08.187127+0000","last_active":"2023-11-08T19:31:11.444341+0000","last_peered":"2023-11-08T19:31:11.444341+0000","last_clean":"2023-11-08T19:31:11.444341+0000","last_became_active":"2023-11-08T19:27:08.182648+0000","last_became_peered":"2023-11-08T19:27:08.182648+0000","last_unstale":"2023-11-08T19:31:11.444341+0000","last_undegraded":"2023-11-08T19:31:11.444341+0000","last_fullsized":"2023-11-08T19:31:11.444341+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:50:12.101878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":86,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T19:30:47.741732+0000","last_change":"2023-11-08T19:26:05.609570+0000","last_active":"2023-11-08T19:30:47.741732+0000","last_peered":"2023-11-08T19:30:47.741732+0000","last_clean":"2023-11-08T19:30:47.741732+0000","last_became_active":"2023-11-08T19:26:04.608379+0000","last_became_peered":"2023-11-08T19:26:04.608379+0000","last_unstale":"2023-11-08T19:30:47.741732+0000","last_undegraded":"2023-11-08T19:30:47.741732+0000","last_fullsized":"2023-11-08T19:30:47.741732+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:40:08.986649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000529657,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":53,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.668577+0000","last_change":"2023-11-08T19:27:09.211991+0000","last_active":"2023-11-08T19:31:11.668577+0000","last_peered":"2023-11-08T19:31:11.668577+0000","last_clean":"2023-11-08T19:31:11.668577+0000","last_became_active":"2023-11-08T19:27:07.195231+0000","last_became_peered":"2023-11-08T19:27:07.195231+0000","last_unstale":"2023-11-08T19:31:11.668577+0000","last_undegraded":"2023-11-08T19:31:11.668577+0000","last_fullsized":"2023-11-08T19:31:11.668577+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:25:14.487524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000441221,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":46,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.444418+0000","last_change":"2023-11-08T19:27:08.195364+0000","last_active":"2023-11-08T19:31:11.444418+0000","last_peered":"2023-11-08T19:31:11.444418+0000","last_clean":"2023-11-08T19:31:11.444418+0000","last_became_active":"2023-11-08T19:27:08.195074+0000","last_became_peered":"2023-11-08T19:27:08.195074+0000","last_unstale":"2023-11-08T19:31:11.444418+0000","last_undegraded":"2023-11-08T19:31:11.444418+0000","last_fullsized":"2023-11-08T19:31:11.444418+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:02:57.552873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":76,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.002147+0000","last_change":"2023-11-08T19:26:06.394300+0000","last_active":"2023-11-08T19:31:11.002147+0000","last_peered":"2023-11-08T19:31:11.002147+0000","last_clean":"2023-11-08T19:31:11.002147+0000","last_became_active":"2023-11-08T19:26:04.601685+0000","last_became_peered":"2023-11-08T19:26:04.601685+0000","last_unstale":"2023-11-08T19:31:11.002147+0000","last_undegraded":"2023-11-08T19:31:11.002147+0000","last_fullsized":"2023-11-08T19:31:11.002147+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:51:06.626474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061044099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":53,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.444658+0000","last_change":"2023-11-08T19:27:09.178466+0000","last_active":"2023-11-08T19:31:11.444658+0000","last_peered":"2023-11-08T19:31:11.444658+0000","last_clean":"2023-11-08T19:31:11.444658+0000","last_became_active":"2023-11-08T19:27:07.189386+0000","last_became_peered":"2023-11-08T19:27:07.189386+0000","last_unstale":"2023-11-08T19:31:11.444658+0000","last_undegraded":"2023-11-08T19:31:11.444658+0000","last_fullsized":"2023-11-08T19:31:11.444658+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:48:51.139820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049390100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":50,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.666965+0000","last_change":"2023-11-08T19:27:08.189452+0000","last_active":"2023-11-08T19:31:11.666965+0000","last_peered":"2023-11-08T19:31:11.666965+0000","last_clean":"2023-11-08T19:31:11.666965+0000","last_became_active":"2023-11-08T19:27:08.189088+0000","last_became_peered":"2023-11-08T19:27:08.189088+0000","last_unstale":"2023-11-08T19:31:11.666965+0000","last_undegraded":"2023-11-08T19:31:11.666965+0000","last_fullsized":"2023-11-08T19:31:11.666965+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:25:18.963220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":77,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.666780+0000","last_change":"2023-11-08T19:26:06.296634+0000","last_active":"2023-11-08T19:31:11.666780+0000","last_peered":"2023-11-08T19:31:11.666780+0000","last_clean":"2023-11-08T19:31:11.666780+0000","last_became_active":"2023-11-08T19:26:04.603088+0000","last_became_peered":"2023-11-08T19:26:04.603088+0000","last_unstale":"2023-11-08T19:31:11.666780+0000","last_undegraded":"2023-11-08T19:31:11.666780+0000","last_fullsized":"2023-11-08T19:31:11.666780+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:01:56.991346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038180899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":53,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.666847+0000","last_change":"2023-11-08T19:27:09.431220+0000","last_active":"2023-11-08T19:31:11.666847+0000","last_peered":"2023-11-08T19:31:11.666847+0000","last_clean":"2023-11-08T19:31:11.666847+0000","last_became_active":"2023-11-08T19:27:07.191589+0000","last_became_peered":"2023-11-08T19:27:07.191589+0000","last_unstale":"2023-11-08T19:31:11.666847+0000","last_undegraded":"2023-11-08T19:31:11.666847+0000","last_fullsized":"2023-11-08T19:31:11.666847+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:18:19.964487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000729527,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":46,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.002470+0000","last_change":"2023-11-08T19:27:08.193337+0000","last_active":"2023-11-08T19:31:11.002470+0000","last_peered":"2023-11-08T19:31:11.002470+0000","last_clean":"2023-11-08T19:31:11.002470+0000","last_became_active":"2023-11-08T19:27:08.192997+0000","last_became_peered":"2023-11-08T19:27:08.192997+0000","last_unstale":"2023-11-08T19:31:11.002470+0000","last_undegraded":"2023-11-08T19:31:11.002470+0000","last_fullsized":"2023-11-08T19:31:11.002470+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:45:22.084015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":82,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.668119+0000","last_change":"2023-11-08T19:26:06.424511+0000","last_active":"2023-11-08T19:31:11.668119+0000","last_peered":"2023-11-08T19:31:11.668119+0000","last_clean":"2023-11-08T19:31:11.668119+0000","last_became_active":"2023-11-08T19:26:04.598973+0000","last_became_peered":"2023-11-08T19:26:04.598973+0000","last_unstale":"2023-11-08T19:31:11.668119+0000","last_undegraded":"2023-11-08T19:31:11.668119+0000","last_fullsized":"2023-11-08T19:31:11.668119+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:41:53.497456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031755900000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":57,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.667045+0000","last_change":"2023-11-08T19:27:09.497024+0000","last_active":"2023-11-08T19:31:11.667045+0000","last_peered":"2023-11-08T19:31:11.667045+0000","last_clean":"2023-11-08T19:31:11.667045+0000","last_became_active":"2023-11-08T19:27:07.190132+0000","last_became_peered":"2023-11-08T19:27:07.190132+0000","last_unstale":"2023-11-08T19:31:11.667045+0000","last_undegraded":"2023-11-08T19:31:11.667045+0000","last_fullsized":"2023-11-08T19:31:11.667045+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:36:34.881217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054126000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":46,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.002863+0000","last_change":"2023-11-08T19:27:08.194196+0000","last_active":"2023-11-08T19:31:11.002863+0000","last_peered":"2023-11-08T19:31:11.002863+0000","last_clean":"2023-11-08T19:31:11.002863+0000","last_became_active":"2023-11-08T19:27:08.193912+0000","last_became_peered":"2023-11-08T19:27:08.193912+0000","last_unstale":"2023-11-08T19:31:11.002863+0000","last_undegraded":"2023-11-08T19:31:11.002863+0000","last_fullsized":"2023-11-08T19:31:11.002863+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:25:18.963220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":77,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.667020+0000","last_change":"2023-11-08T19:26:06.296937+0000","last_active":"2023-11-08T19:31:11.667020+0000","last_peered":"2023-11-08T19:31:11.667020+0000","last_clean":"2023-11-08T19:31:11.667020+0000","last_became_active":"2023-11-08T19:26:04.599999+0000","last_became_peered":"2023-11-08T19:26:04.599999+0000","last_unstale":"2023-11-08T19:31:11.667020+0000","last_undegraded":"2023-11-08T19:31:11.667020+0000","last_fullsized":"2023-11-08T19:31:11.667020+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:50:43.827570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025605700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":57,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.667208+0000","last_change":"2023-11-08T19:27:09.497040+0000","last_active":"2023-11-08T19:31:11.667208+0000","last_peered":"2023-11-08T19:31:11.667208+0000","last_clean":"2023-11-08T19:31:11.667208+0000","last_became_active":"2023-11-08T19:27:07.189135+0000","last_became_peered":"2023-11-08T19:27:07.189135+0000","last_unstale":"2023-11-08T19:31:11.667208+0000","last_undegraded":"2023-11-08T19:31:11.667208+0000","last_fullsized":"2023-11-08T19:31:11.667208+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:12:29.400972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051794299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":50,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T19:30:47.742598+0000","last_change":"2023-11-08T19:27:08.187053+0000","last_active":"2023-11-08T19:30:47.742598+0000","last_peered":"2023-11-08T19:30:47.742598+0000","last_clean":"2023-11-08T19:30:47.742598+0000","last_became_active":"2023-11-08T19:27:08.186643+0000","last_became_peered":"2023-11-08T19:27:08.186643+0000","last_unstale":"2023-11-08T19:30:47.742598+0000","last_undegraded":"2023-11-08T19:30:47.742598+0000","last_fullsized":"2023-11-08T19:30:47.742598+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:35:30.073885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":86,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T19:30:47.742555+0000","last_change":"2023-11-08T19:26:05.609824+0000","last_active":"2023-11-08T19:30:47.742555+0000","last_peered":"2023-11-08T19:30:47.742555+0000","last_clean":"2023-11-08T19:30:47.742555+0000","last_became_active":"2023-11-08T19:26:04.603320+0000","last_became_peered":"2023-11-08T19:26:04.603320+0000","last_unstale":"2023-11-08T19:30:47.742555+0000","last_undegraded":"2023-11-08T19:30:47.742555+0000","last_fullsized":"2023-11-08T19:30:47.742555+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:05:45.976615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063225,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":57,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.668286+0000","last_change":"2023-11-08T19:27:09.496655+0000","last_active":"2023-11-08T19:31:11.668286+0000","last_peered":"2023-11-08T19:31:11.668286+0000","last_clean":"2023-11-08T19:31:11.668286+0000","last_became_active":"2023-11-08T19:27:07.189895+0000","last_became_peered":"2023-11-08T19:27:07.189895+0000","last_unstale":"2023-11-08T19:31:11.668286+0000","last_undegraded":"2023-11-08T19:31:11.668286+0000","last_fullsized":"2023-11-08T19:31:11.668286+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:41:08.822366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045654199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":78,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.667623+0000","last_change":"2023-11-08T19:26:06.541360+0000","last_active":"2023-11-08T19:31:11.667623+0000","last_peered":"2023-11-08T19:31:11.667623+0000","last_clean":"2023-11-08T19:31:11.667623+0000","last_became_active":"2023-11-08T19:26:04.603725+0000","last_became_peered":"2023-11-08T19:26:04.603725+0000","last_unstale":"2023-11-08T19:31:11.667623+0000","last_undegraded":"2023-11-08T19:31:11.667623+0000","last_fullsized":"2023-11-08T19:31:11.667623+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:28:48.576742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053365400000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":53,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.669050+0000","last_change":"2023-11-08T19:27:09.283599+0000","last_active":"2023-11-08T19:31:11.669050+0000","last_peered":"2023-11-08T19:31:11.669050+0000","last_clean":"2023-11-08T19:31:11.669050+0000","last_became_active":"2023-11-08T19:27:07.189595+0000","last_became_peered":"2023-11-08T19:27:07.189595+0000","last_unstale":"2023-11-08T19:31:11.669050+0000","last_undegraded":"2023-11-08T19:31:11.669050+0000","last_fullsized":"2023-11-08T19:31:11.669050+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:48:20.082682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000323479,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":47,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.667698+0000","last_change":"2023-11-08T19:27:08.185395+0000","last_active":"2023-11-08T19:31:11.667698+0000","last_peered":"2023-11-08T19:31:11.667698+0000","last_clean":"2023-11-08T19:31:11.667698+0000","last_became_active":"2023-11-08T19:27:08.184765+0000","last_became_peered":"2023-11-08T19:27:08.184765+0000","last_unstale":"2023-11-08T19:31:11.667698+0000","last_undegraded":"2023-11-08T19:31:11.667698+0000","last_fullsized":"2023-11-08T19:31:11.667698+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:08:51.864418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":82,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.667206+0000","last_change":"2023-11-08T19:26:06.424236+0000","last_active":"2023-11-08T19:31:11.667206+0000","last_peered":"2023-11-08T19:31:11.667206+0000","last_clean":"2023-11-08T19:31:11.667206+0000","last_became_active":"2023-11-08T19:26:04.600115+0000","last_became_peered":"2023-11-08T19:26:04.600115+0000","last_unstale":"2023-11-08T19:31:11.667206+0000","last_undegraded":"2023-11-08T19:31:11.667206+0000","last_fullsized":"2023-11-08T19:31:11.667206+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:18:01.876605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065803900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":53,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.002693+0000","last_change":"2023-11-08T19:27:09.331899+0000","last_active":"2023-11-08T19:31:11.002693+0000","last_peered":"2023-11-08T19:31:11.002693+0000","last_clean":"2023-11-08T19:31:11.002693+0000","last_became_active":"2023-11-08T19:27:07.191460+0000","last_became_peered":"2023-11-08T19:27:07.191460+0000","last_unstale":"2023-11-08T19:31:11.002693+0000","last_undegraded":"2023-11-08T19:31:11.002693+0000","last_fullsized":"2023-11-08T19:31:11.002693+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:51:29.247104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044907500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":50,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T19:30:47.742222+0000","last_change":"2023-11-08T19:27:08.191802+0000","last_active":"2023-11-08T19:30:47.742222+0000","last_peered":"2023-11-08T19:30:47.742222+0000","last_clean":"2023-11-08T19:30:47.742222+0000","last_became_active":"2023-11-08T19:27:08.191520+0000","last_became_peered":"2023-11-08T19:27:08.191520+0000","last_unstale":"2023-11-08T19:30:47.742222+0000","last_undegraded":"2023-11-08T19:30:47.742222+0000","last_fullsized":"2023-11-08T19:30:47.742222+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:49:16.458634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":86,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T19:30:47.741071+0000","last_change":"2023-11-08T19:26:05.608892+0000","last_active":"2023-11-08T19:30:47.741071+0000","last_peered":"2023-11-08T19:30:47.741071+0000","last_clean":"2023-11-08T19:30:47.741071+0000","last_became_active":"2023-11-08T19:26:04.604519+0000","last_became_peered":"2023-11-08T19:26:04.604519+0000","last_unstale":"2023-11-08T19:30:47.741071+0000","last_undegraded":"2023-11-08T19:30:47.741071+0000","last_fullsized":"2023-11-08T19:30:47.741071+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:11:26.508634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051248099999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":57,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T19:30:47.742386+0000","last_change":"2023-11-08T19:27:09.181431+0000","last_active":"2023-11-08T19:30:47.742386+0000","last_peered":"2023-11-08T19:30:47.742386+0000","last_clean":"2023-11-08T19:30:47.742386+0000","last_became_active":"2023-11-08T19:27:07.190800+0000","last_became_peered":"2023-11-08T19:27:07.190800+0000","last_unstale":"2023-11-08T19:30:47.742386+0000","last_undegraded":"2023-11-08T19:30:47.742386+0000","last_fullsized":"2023-11-08T19:30:47.742386+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:46:30.452678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656099999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":46,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.667947+0000","last_change":"2023-11-08T19:27:08.187983+0000","last_active":"2023-11-08T19:31:11.667947+0000","last_peered":"2023-11-08T19:31:11.667947+0000","last_clean":"2023-11-08T19:31:11.667947+0000","last_became_active":"2023-11-08T19:27:08.187322+0000","last_became_peered":"2023-11-08T19:27:08.187322+0000","last_unstale":"2023-11-08T19:31:11.667947+0000","last_undegraded":"2023-11-08T19:31:11.667947+0000","last_fullsized":"2023-11-08T19:31:11.667947+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:33:07.647519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":76,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.002194+0000","last_change":"2023-11-08T19:26:06.394075+0000","last_active":"2023-11-08T19:31:11.002194+0000","last_peered":"2023-11-08T19:31:11.002194+0000","last_clean":"2023-11-08T19:31:11.002194+0000","last_became_active":"2023-11-08T19:26:04.606967+0000","last_became_peered":"2023-11-08T19:26:04.606967+0000","last_unstale":"2023-11-08T19:31:11.002194+0000","last_undegraded":"2023-11-08T19:31:11.002194+0000","last_fullsized":"2023-11-08T19:31:11.002194+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:19:56.286835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040971999999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":57,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T19:30:47.741945+0000","last_change":"2023-11-08T19:27:09.180722+0000","last_active":"2023-11-08T19:30:47.741945+0000","last_peered":"2023-11-08T19:30:47.741945+0000","last_clean":"2023-11-08T19:30:47.741945+0000","last_became_active":"2023-11-08T19:27:07.186977+0000","last_became_peered":"2023-11-08T19:27:07.186977+0000","last_unstale":"2023-11-08T19:30:47.741945+0000","last_undegraded":"2023-11-08T19:30:47.741945+0000","last_fullsized":"2023-11-08T19:30:47.741945+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:17:59.966463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057675100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":46,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.002184+0000","last_change":"2023-11-08T19:27:08.193963+0000","last_active":"2023-11-08T19:31:11.002184+0000","last_peered":"2023-11-08T19:31:11.002184+0000","last_clean":"2023-11-08T19:31:11.002184+0000","last_became_active":"2023-11-08T19:27:08.193098+0000","last_became_peered":"2023-11-08T19:27:08.193098+0000","last_unstale":"2023-11-08T19:31:11.002184+0000","last_undegraded":"2023-11-08T19:31:11.002184+0000","last_fullsized":"2023-11-08T19:31:11.002184+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:41:09.827370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":76,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.668693+0000","last_change":"2023-11-08T19:26:06.265686+0000","last_active":"2023-11-08T19:31:11.668693+0000","last_peered":"2023-11-08T19:31:11.668693+0000","last_clean":"2023-11-08T19:31:11.668693+0000","last_became_active":"2023-11-08T19:26:04.602111+0000","last_became_peered":"2023-11-08T19:26:04.602111+0000","last_unstale":"2023-11-08T19:31:11.668693+0000","last_undegraded":"2023-11-08T19:31:11.668693+0000","last_fullsized":"2023-11-08T19:31:11.668693+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:40:11.164951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056197899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":57,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T19:30:47.742552+0000","last_change":"2023-11-08T19:27:09.181333+0000","last_active":"2023-11-08T19:30:47.742552+0000","last_peered":"2023-11-08T19:30:47.742552+0000","last_clean":"2023-11-08T19:30:47.742552+0000","last_became_active":"2023-11-08T19:27:07.188657+0000","last_became_peered":"2023-11-08T19:27:07.188657+0000","last_unstale":"2023-11-08T19:30:47.742552+0000","last_undegraded":"2023-11-08T19:30:47.742552+0000","last_fullsized":"2023-11-08T19:30:47.742552+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:25:56.788615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001227393,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":50,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T19:30:47.742754+0000","last_change":"2023-11-08T19:27:08.188533+0000","last_active":"2023-11-08T19:30:47.742754+0000","last_peered":"2023-11-08T19:30:47.742754+0000","last_clean":"2023-11-08T19:30:47.742754+0000","last_became_active":"2023-11-08T19:27:08.188240+0000","last_became_peered":"2023-11-08T19:27:08.188240+0000","last_unstale":"2023-11-08T19:30:47.742754+0000","last_undegraded":"2023-11-08T19:30:47.742754+0000","last_fullsized":"2023-11-08T19:30:47.742754+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:55:37.653521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":50,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T19:30:47.743578+0000","last_change":"2023-11-08T19:27:08.188458+0000","last_active":"2023-11-08T19:30:47.743578+0000","last_peered":"2023-11-08T19:30:47.743578+0000","last_clean":"2023-11-08T19:30:47.743578+0000","last_became_active":"2023-11-08T19:27:08.187880+0000","last_became_peered":"2023-11-08T19:27:08.187880+0000","last_unstale":"2023-11-08T19:30:47.743578+0000","last_undegraded":"2023-11-08T19:30:47.743578+0000","last_fullsized":"2023-11-08T19:30:47.743578+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:25:28.618214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":77,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.444854+0000","last_change":"2023-11-08T19:26:05.606898+0000","last_active":"2023-11-08T19:31:11.444854+0000","last_peered":"2023-11-08T19:31:11.444854+0000","last_clean":"2023-11-08T19:31:11.444854+0000","last_became_active":"2023-11-08T19:26:04.603955+0000","last_became_peered":"2023-11-08T19:26:04.603955+0000","last_unstale":"2023-11-08T19:31:11.444854+0000","last_undegraded":"2023-11-08T19:31:11.444854+0000","last_fullsized":"2023-11-08T19:31:11.444854+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:19:11.589559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036473499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":53,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.444503+0000","last_change":"2023-11-08T19:27:09.178770+0000","last_active":"2023-11-08T19:31:11.444503+0000","last_peered":"2023-11-08T19:31:11.444503+0000","last_clean":"2023-11-08T19:31:11.444503+0000","last_became_active":"2023-11-08T19:27:07.194992+0000","last_became_peered":"2023-11-08T19:27:07.194992+0000","last_unstale":"2023-11-08T19:31:11.444503+0000","last_undegraded":"2023-11-08T19:31:11.444503+0000","last_fullsized":"2023-11-08T19:31:11.444503+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:35:34.026910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000770285,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":46,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.667108+0000","last_change":"2023-11-08T19:27:08.193953+0000","last_active":"2023-11-08T19:31:11.667108+0000","last_peered":"2023-11-08T19:31:11.667108+0000","last_clean":"2023-11-08T19:31:11.667108+0000","last_became_active":"2023-11-08T19:27:08.193653+0000","last_became_peered":"2023-11-08T19:27:08.193653+0000","last_unstale":"2023-11-08T19:31:11.667108+0000","last_undegraded":"2023-11-08T19:31:11.667108+0000","last_fullsized":"2023-11-08T19:31:11.667108+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:35:30.073885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":82,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.667882+0000","last_change":"2023-11-08T19:26:06.424464+0000","last_active":"2023-11-08T19:31:11.667882+0000","last_peered":"2023-11-08T19:31:11.667882+0000","last_clean":"2023-11-08T19:31:11.667882+0000","last_became_active":"2023-11-08T19:26:04.603144+0000","last_became_peered":"2023-11-08T19:26:04.603144+0000","last_unstale":"2023-11-08T19:31:11.667882+0000","last_undegraded":"2023-11-08T19:31:11.667882+0000","last_fullsized":"2023-11-08T19:31:11.667882+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:42:55.269365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054090799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":57,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T19:30:47.742114+0000","last_change":"2023-11-08T19:27:09.180479+0000","last_active":"2023-11-08T19:30:47.742114+0000","last_peered":"2023-11-08T19:30:47.742114+0000","last_clean":"2023-11-08T19:30:47.742114+0000","last_became_active":"2023-11-08T19:27:07.190438+0000","last_became_peered":"2023-11-08T19:27:07.190438+0000","last_unstale":"2023-11-08T19:30:47.742114+0000","last_undegraded":"2023-11-08T19:30:47.742114+0000","last_fullsized":"2023-11-08T19:30:47.742114+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:25:14.487524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050893300000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":46,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.667409+0000","last_change":"2023-11-08T19:27:08.190530+0000","last_active":"2023-11-08T19:31:11.667409+0000","last_peered":"2023-11-08T19:31:11.667409+0000","last_clean":"2023-11-08T19:31:11.667409+0000","last_became_active":"2023-11-08T19:27:08.190203+0000","last_became_peered":"2023-11-08T19:27:08.190203+0000","last_unstale":"2023-11-08T19:31:11.667409+0000","last_undegraded":"2023-11-08T19:31:11.667409+0000","last_fullsized":"2023-11-08T19:31:11.667409+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:26:32.777602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":78,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.668651+0000","last_change":"2023-11-08T19:26:06.541623+0000","last_active":"2023-11-08T19:31:11.668651+0000","last_peered":"2023-11-08T19:31:11.668651+0000","last_clean":"2023-11-08T19:31:11.668651+0000","last_became_active":"2023-11-08T19:26:04.601961+0000","last_became_peered":"2023-11-08T19:26:04.601961+0000","last_unstale":"2023-11-08T19:31:11.668651+0000","last_undegraded":"2023-11-08T19:31:11.668651+0000","last_fullsized":"2023-11-08T19:31:11.668651+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:11:26.508634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000466386,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":54,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.666591+0000","last_change":"2023-11-08T19:27:09.431231+0000","last_active":"2023-11-08T19:31:11.666591+0000","last_peered":"2023-11-08T19:31:11.666591+0000","last_clean":"2023-11-08T19:31:11.666591+0000","last_became_active":"2023-11-08T19:27:07.191583+0000","last_became_peered":"2023-11-08T19:27:07.191583+0000","last_unstale":"2023-11-08T19:31:11.666591+0000","last_undegraded":"2023-11-08T19:31:11.666591+0000","last_fullsized":"2023-11-08T19:31:11.666591+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:58:48.357266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093695200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":46,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.666559+0000","last_change":"2023-11-08T19:27:08.187900+0000","last_active":"2023-11-08T19:31:11.666559+0000","last_peered":"2023-11-08T19:31:11.666559+0000","last_clean":"2023-11-08T19:31:11.666559+0000","last_became_active":"2023-11-08T19:27:08.187626+0000","last_became_peered":"2023-11-08T19:27:08.187626+0000","last_unstale":"2023-11-08T19:31:11.666559+0000","last_undegraded":"2023-11-08T19:31:11.666559+0000","last_fullsized":"2023-11-08T19:31:11.666559+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:00:55.878397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":86,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T19:30:47.741631+0000","last_change":"2023-11-08T19:26:05.608980+0000","last_active":"2023-11-08T19:30:47.741631+0000","last_peered":"2023-11-08T19:30:47.741631+0000","last_clean":"2023-11-08T19:30:47.741631+0000","last_became_active":"2023-11-08T19:26:04.606386+0000","last_became_peered":"2023-11-08T19:26:04.606386+0000","last_unstale":"2023-11-08T19:30:47.741631+0000","last_undegraded":"2023-11-08T19:30:47.741631+0000","last_fullsized":"2023-11-08T19:30:47.741631+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:41:50.771457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037950599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":53,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.444816+0000","last_change":"2023-11-08T19:27:09.179037+0000","last_active":"2023-11-08T19:31:11.444816+0000","last_peered":"2023-11-08T19:31:11.444816+0000","last_clean":"2023-11-08T19:31:11.444816+0000","last_became_active":"2023-11-08T19:27:07.195033+0000","last_became_peered":"2023-11-08T19:27:07.195033+0000","last_unstale":"2023-11-08T19:31:11.444816+0000","last_undegraded":"2023-11-08T19:31:11.444816+0000","last_fullsized":"2023-11-08T19:31:11.444816+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:19:08.253483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041824800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":46,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.002110+0000","last_change":"2023-11-08T19:27:08.193761+0000","last_active":"2023-11-08T19:31:11.002110+0000","last_peered":"2023-11-08T19:31:11.002110+0000","last_clean":"2023-11-08T19:31:11.002110+0000","last_became_active":"2023-11-08T19:27:08.193017+0000","last_became_peered":"2023-11-08T19:27:08.193017+0000","last_unstale":"2023-11-08T19:31:11.002110+0000","last_undegraded":"2023-11-08T19:31:11.002110+0000","last_fullsized":"2023-11-08T19:31:11.002110+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:18:00.971466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":82,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.667284+0000","last_change":"2023-11-08T19:26:06.424073+0000","last_active":"2023-11-08T19:31:11.667284+0000","last_peered":"2023-11-08T19:31:11.667284+0000","last_clean":"2023-11-08T19:31:11.667284+0000","last_became_active":"2023-11-08T19:26:04.599046+0000","last_became_peered":"2023-11-08T19:26:04.599046+0000","last_unstale":"2023-11-08T19:31:11.667284+0000","last_undegraded":"2023-11-08T19:31:11.667284+0000","last_fullsized":"2023-11-08T19:31:11.667284+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:04:43.562566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035092199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":58,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-11-08T19:30:55.779019+0000","last_change":"2023-11-08T19:27:09.181716+0000","last_active":"2023-11-08T19:30:55.779019+0000","last_peered":"2023-11-08T19:30:55.779019+0000","last_clean":"2023-11-08T19:30:55.779019+0000","last_became_active":"2023-11-08T19:27:07.187077+0000","last_became_peered":"2023-11-08T19:27:07.187077+0000","last_unstale":"2023-11-08T19:30:55.779019+0000","last_undegraded":"2023-11-08T19:30:55.779019+0000","last_fullsized":"2023-11-08T19:30:55.779019+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:51:29.247104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056673599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":46,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.667869+0000","last_change":"2023-11-08T19:27:08.188143+0000","last_active":"2023-11-08T19:31:11.667869+0000","last_peered":"2023-11-08T19:31:11.667869+0000","last_clean":"2023-11-08T19:31:11.667869+0000","last_became_active":"2023-11-08T19:27:08.187714+0000","last_became_peered":"2023-11-08T19:27:08.187714+0000","last_unstale":"2023-11-08T19:31:11.667869+0000","last_undegraded":"2023-11-08T19:31:11.667869+0000","last_fullsized":"2023-11-08T19:31:11.667869+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:02:10.863403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":76,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.002535+0000","last_change":"2023-11-08T19:26:06.394320+0000","last_active":"2023-11-08T19:31:11.002535+0000","last_peered":"2023-11-08T19:31:11.002535+0000","last_clean":"2023-11-08T19:31:11.002535+0000","last_became_active":"2023-11-08T19:26:04.603212+0000","last_became_peered":"2023-11-08T19:26:04.603212+0000","last_unstale":"2023-11-08T19:31:11.002535+0000","last_undegraded":"2023-11-08T19:31:11.002535+0000","last_fullsized":"2023-11-08T19:31:11.002535+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:25:29.947079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054695399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":57,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T19:30:47.742297+0000","last_change":"2023-11-08T19:27:09.181613+0000","last_active":"2023-11-08T19:30:47.742297+0000","last_peered":"2023-11-08T19:30:47.742297+0000","last_clean":"2023-11-08T19:30:47.742297+0000","last_became_active":"2023-11-08T19:27:07.190897+0000","last_became_peered":"2023-11-08T19:27:07.190897+0000","last_unstale":"2023-11-08T19:30:47.742297+0000","last_undegraded":"2023-11-08T19:30:47.742297+0000","last_fullsized":"2023-11-08T19:30:47.742297+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:27:57.388125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011172459999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":46,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.668020+0000","last_change":"2023-11-08T19:27:08.188668+0000","last_active":"2023-11-08T19:31:11.668020+0000","last_peered":"2023-11-08T19:31:11.668020+0000","last_clean":"2023-11-08T19:31:11.668020+0000","last_became_active":"2023-11-08T19:27:08.188441+0000","last_became_peered":"2023-11-08T19:27:08.188441+0000","last_unstale":"2023-11-08T19:31:11.668020+0000","last_undegraded":"2023-11-08T19:31:11.668020+0000","last_fullsized":"2023-11-08T19:31:11.668020+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:45:44.379160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":82,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.667134+0000","last_change":"2023-11-08T19:26:06.423754+0000","last_active":"2023-11-08T19:31:11.667134+0000","last_peered":"2023-11-08T19:31:11.667134+0000","last_clean":"2023-11-08T19:31:11.667134+0000","last_became_active":"2023-11-08T19:26:04.599322+0000","last_became_peered":"2023-11-08T19:26:04.599322+0000","last_unstale":"2023-11-08T19:31:11.667134+0000","last_undegraded":"2023-11-08T19:31:11.667134+0000","last_fullsized":"2023-11-08T19:31:11.667134+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:13:08.689128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042670000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":57,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T19:30:47.743024+0000","last_change":"2023-11-08T19:27:09.181656+0000","last_active":"2023-11-08T19:30:47.743024+0000","last_peered":"2023-11-08T19:30:47.743024+0000","last_clean":"2023-11-08T19:30:47.743024+0000","last_became_active":"2023-11-08T19:27:07.187337+0000","last_became_peered":"2023-11-08T19:27:07.187337+0000","last_unstale":"2023-11-08T19:30:47.743024+0000","last_undegraded":"2023-11-08T19:30:47.743024+0000","last_fullsized":"2023-11-08T19:30:47.743024+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:14:17.605057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000346967,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":46,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.002332+0000","last_change":"2023-11-08T19:27:08.194065+0000","last_active":"2023-11-08T19:31:11.002332+0000","last_peered":"2023-11-08T19:31:11.002332+0000","last_clean":"2023-11-08T19:31:11.002332+0000","last_became_active":"2023-11-08T19:27:08.193532+0000","last_became_peered":"2023-11-08T19:27:08.193532+0000","last_unstale":"2023-11-08T19:31:11.002332+0000","last_undegraded":"2023-11-08T19:31:11.002332+0000","last_fullsized":"2023-11-08T19:31:11.002332+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:27:58.393129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":76,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.002769+0000","last_change":"2023-11-08T19:26:06.394488+0000","last_active":"2023-11-08T19:31:11.002769+0000","last_peered":"2023-11-08T19:31:11.002769+0000","last_clean":"2023-11-08T19:31:11.002769+0000","last_became_active":"2023-11-08T19:26:04.606805+0000","last_became_peered":"2023-11-08T19:26:04.606805+0000","last_unstale":"2023-11-08T19:31:11.002769+0000","last_undegraded":"2023-11-08T19:31:11.002769+0000","last_fullsized":"2023-11-08T19:31:11.002769+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:25:20.857133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045317000000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":53,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.668089+0000","last_change":"2023-11-08T19:27:09.211647+0000","last_active":"2023-11-08T19:31:11.668089+0000","last_peered":"2023-11-08T19:31:11.668089+0000","last_clean":"2023-11-08T19:31:11.668089+0000","last_became_active":"2023-11-08T19:27:07.190309+0000","last_became_peered":"2023-11-08T19:27:07.190309+0000","last_unstale":"2023-11-08T19:31:11.668089+0000","last_undegraded":"2023-11-08T19:31:11.668089+0000","last_fullsized":"2023-11-08T19:31:11.668089+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:56:04.292154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047615199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":46,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.002763+0000","last_change":"2023-11-08T19:27:08.194884+0000","last_active":"2023-11-08T19:31:11.002763+0000","last_peered":"2023-11-08T19:31:11.002763+0000","last_clean":"2023-11-08T19:31:11.002763+0000","last_became_active":"2023-11-08T19:27:08.193241+0000","last_became_peered":"2023-11-08T19:27:08.193241+0000","last_unstale":"2023-11-08T19:31:11.002763+0000","last_undegraded":"2023-11-08T19:31:11.002763+0000","last_fullsized":"2023-11-08T19:31:11.002763+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:46:31.457682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":76,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.668753+0000","last_change":"2023-11-08T19:26:06.265907+0000","last_active":"2023-11-08T19:31:11.668753+0000","last_peered":"2023-11-08T19:31:11.668753+0000","last_clean":"2023-11-08T19:31:11.668753+0000","last_became_active":"2023-11-08T19:26:04.604772+0000","last_became_peered":"2023-11-08T19:26:04.604772+0000","last_unstale":"2023-11-08T19:31:11.668753+0000","last_undegraded":"2023-11-08T19:31:11.668753+0000","last_fullsized":"2023-11-08T19:31:11.668753+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:44:16.139794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066875300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":53,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.667612+0000","last_change":"2023-11-08T19:27:09.282699+0000","last_active":"2023-11-08T19:31:11.667612+0000","last_peered":"2023-11-08T19:31:11.667612+0000","last_clean":"2023-11-08T19:31:11.667612+0000","last_became_active":"2023-11-08T19:27:07.191212+0000","last_became_peered":"2023-11-08T19:27:07.191212+0000","last_unstale":"2023-11-08T19:31:11.667612+0000","last_undegraded":"2023-11-08T19:31:11.667612+0000","last_fullsized":"2023-11-08T19:31:11.667612+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:35:14.715110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046640800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":47,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.668470+0000","last_change":"2023-11-08T19:27:08.187644+0000","last_active":"2023-11-08T19:31:11.668470+0000","last_peered":"2023-11-08T19:31:11.668470+0000","last_clean":"2023-11-08T19:31:11.668470+0000","last_became_active":"2023-11-08T19:27:08.187376+0000","last_became_peered":"2023-11-08T19:27:08.187376+0000","last_unstale":"2023-11-08T19:31:11.668470+0000","last_undegraded":"2023-11-08T19:31:11.668470+0000","last_fullsized":"2023-11-08T19:31:11.668470+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:22:40.300185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":77,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.444647+0000","last_change":"2023-11-08T19:26:05.606946+0000","last_active":"2023-11-08T19:31:11.444647+0000","last_peered":"2023-11-08T19:31:11.444647+0000","last_clean":"2023-11-08T19:31:11.444647+0000","last_became_active":"2023-11-08T19:26:04.598775+0000","last_became_peered":"2023-11-08T19:26:04.598775+0000","last_unstale":"2023-11-08T19:31:11.444647+0000","last_undegraded":"2023-11-08T19:31:11.444647+0000","last_fullsized":"2023-11-08T19:31:11.444647+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:20:56.898927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049870399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":53,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.002400+0000","last_change":"2023-11-08T19:27:09.331646+0000","last_active":"2023-11-08T19:31:11.002400+0000","last_peered":"2023-11-08T19:31:11.002400+0000","last_clean":"2023-11-08T19:31:11.002400+0000","last_became_active":"2023-11-08T19:27:07.190619+0000","last_became_peered":"2023-11-08T19:27:07.190619+0000","last_unstale":"2023-11-08T19:31:11.002400+0000","last_undegraded":"2023-11-08T19:31:11.002400+0000","last_fullsized":"2023-11-08T19:31:11.002400+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:42:47.371458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065918400000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":50,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T19:30:47.742380+0000","last_change":"2023-11-08T19:27:08.183798+0000","last_active":"2023-11-08T19:30:47.742380+0000","last_peered":"2023-11-08T19:30:47.742380+0000","last_clean":"2023-11-08T19:30:47.742380+0000","last_became_active":"2023-11-08T19:27:08.183537+0000","last_became_peered":"2023-11-08T19:27:08.183537+0000","last_unstale":"2023-11-08T19:30:47.742380+0000","last_undegraded":"2023-11-08T19:30:47.742380+0000","last_fullsized":"2023-11-08T19:30:47.742380+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:50:39.026265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":86,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T19:30:54.625364+0000","last_change":"2023-11-08T19:26:05.610012+0000","last_active":"2023-11-08T19:30:54.625364+0000","last_peered":"2023-11-08T19:30:54.625364+0000","last_clean":"2023-11-08T19:30:54.625364+0000","last_became_active":"2023-11-08T19:26:04.603209+0000","last_became_peered":"2023-11-08T19:26:04.603209+0000","last_unstale":"2023-11-08T19:30:54.625364+0000","last_undegraded":"2023-11-08T19:30:54.625364+0000","last_fullsized":"2023-11-08T19:30:54.625364+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:35:02.306997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053335100000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":53,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.444512+0000","last_change":"2023-11-08T19:27:09.178405+0000","last_active":"2023-11-08T19:31:11.444512+0000","last_peered":"2023-11-08T19:31:11.444512+0000","last_clean":"2023-11-08T19:31:11.444512+0000","last_became_active":"2023-11-08T19:27:07.188169+0000","last_became_peered":"2023-11-08T19:27:07.188169+0000","last_unstale":"2023-11-08T19:31:11.444512+0000","last_undegraded":"2023-11-08T19:31:11.444512+0000","last_fullsized":"2023-11-08T19:31:11.444512+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:39:25.855467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056209700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":46,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.002018+0000","last_change":"2023-11-08T19:27:08.193508+0000","last_active":"2023-11-08T19:31:11.002018+0000","last_peered":"2023-11-08T19:31:11.002018+0000","last_clean":"2023-11-08T19:31:11.002018+0000","last_became_active":"2023-11-08T19:27:08.193037+0000","last_became_peered":"2023-11-08T19:27:08.193037+0000","last_unstale":"2023-11-08T19:31:11.002018+0000","last_undegraded":"2023-11-08T19:31:11.002018+0000","last_fullsized":"2023-11-08T19:31:11.002018+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:29:50.067887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":76,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.667822+0000","last_change":"2023-11-08T19:26:06.265318+0000","last_active":"2023-11-08T19:31:11.667822+0000","last_peered":"2023-11-08T19:31:11.667822+0000","last_clean":"2023-11-08T19:31:11.667822+0000","last_became_active":"2023-11-08T19:26:04.603409+0000","last_became_peered":"2023-11-08T19:26:04.603409+0000","last_unstale":"2023-11-08T19:31:11.667822+0000","last_undegraded":"2023-11-08T19:31:11.667822+0000","last_fullsized":"2023-11-08T19:31:11.667822+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:27:24.933111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065684300000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":57,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T19:30:47.742892+0000","last_change":"2023-11-08T19:27:09.182282+0000","last_active":"2023-11-08T19:30:47.742892+0000","last_peered":"2023-11-08T19:30:47.742892+0000","last_clean":"2023-11-08T19:30:47.742892+0000","last_became_active":"2023-11-08T19:27:07.186944+0000","last_became_peered":"2023-11-08T19:27:07.186944+0000","last_unstale":"2023-11-08T19:30:47.742892+0000","last_undegraded":"2023-11-08T19:30:47.742892+0000","last_fullsized":"2023-11-08T19:30:47.742892+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:45:21.079011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00028665599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":46,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.667282+0000","last_change":"2023-11-08T19:27:08.193990+0000","last_active":"2023-11-08T19:31:11.667282+0000","last_peered":"2023-11-08T19:31:11.667282+0000","last_clean":"2023-11-08T19:31:11.667282+0000","last_became_active":"2023-11-08T19:27:08.193624+0000","last_became_peered":"2023-11-08T19:27:08.193624+0000","last_unstale":"2023-11-08T19:31:11.667282+0000","last_undegraded":"2023-11-08T19:31:11.667282+0000","last_fullsized":"2023-11-08T19:31:11.667282+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:46:41.321120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":82,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.667129+0000","last_change":"2023-11-08T19:26:06.423863+0000","last_active":"2023-11-08T19:31:11.667129+0000","last_peered":"2023-11-08T19:31:11.667129+0000","last_clean":"2023-11-08T19:31:11.667129+0000","last_became_active":"2023-11-08T19:26:04.608639+0000","last_became_peered":"2023-11-08T19:26:04.608639+0000","last_unstale":"2023-11-08T19:31:11.667129+0000","last_undegraded":"2023-11-08T19:31:11.667129+0000","last_fullsized":"2023-11-08T19:31:11.667129+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:04:43.631102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044384999999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":53,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.668714+0000","last_change":"2023-11-08T19:27:09.283232+0000","last_active":"2023-11-08T19:31:11.668714+0000","last_peered":"2023-11-08T19:31:11.668714+0000","last_clean":"2023-11-08T19:31:11.668714+0000","last_became_active":"2023-11-08T19:27:07.190931+0000","last_became_peered":"2023-11-08T19:27:07.190931+0000","last_unstale":"2023-11-08T19:31:11.668714+0000","last_undegraded":"2023-11-08T19:31:11.668714+0000","last_fullsized":"2023-11-08T19:31:11.668714+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:43:28.225967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034146799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":50,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.667555+0000","last_change":"2023-11-08T19:27:08.187252+0000","last_active":"2023-11-08T19:31:11.667555+0000","last_peered":"2023-11-08T19:31:11.667555+0000","last_clean":"2023-11-08T19:31:11.667555+0000","last_became_active":"2023-11-08T19:27:08.186952+0000","last_became_peered":"2023-11-08T19:27:08.186952+0000","last_unstale":"2023-11-08T19:31:11.667555+0000","last_undegraded":"2023-11-08T19:31:11.667555+0000","last_fullsized":"2023-11-08T19:31:11.667555+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:12:52.022374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":76,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.002192+0000","last_change":"2023-11-08T19:26:06.394081+0000","last_active":"2023-11-08T19:31:11.002192+0000","last_peered":"2023-11-08T19:31:11.002192+0000","last_clean":"2023-11-08T19:31:11.002192+0000","last_became_active":"2023-11-08T19:26:04.602137+0000","last_became_peered":"2023-11-08T19:26:04.602137+0000","last_unstale":"2023-11-08T19:31:11.002192+0000","last_undegraded":"2023-11-08T19:31:11.002192+0000","last_fullsized":"2023-11-08T19:31:11.002192+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:05:46.951663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056998200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":53,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.444723+0000","last_change":"2023-11-08T19:27:09.178816+0000","last_active":"2023-11-08T19:31:11.444723+0000","last_peered":"2023-11-08T19:31:11.444723+0000","last_clean":"2023-11-08T19:31:11.444723+0000","last_became_active":"2023-11-08T19:27:07.192001+0000","last_became_peered":"2023-11-08T19:27:07.192001+0000","last_unstale":"2023-11-08T19:31:11.444723+0000","last_undegraded":"2023-11-08T19:31:11.444723+0000","last_fullsized":"2023-11-08T19:31:11.444723+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:04:12.481212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045359000000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":46,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.666798+0000","last_change":"2023-11-08T19:27:08.193836+0000","last_active":"2023-11-08T19:31:11.666798+0000","last_peered":"2023-11-08T19:31:11.666798+0000","last_clean":"2023-11-08T19:31:11.666798+0000","last_became_active":"2023-11-08T19:27:08.193458+0000","last_became_peered":"2023-11-08T19:27:08.193458+0000","last_unstale":"2023-11-08T19:31:11.666798+0000","last_undegraded":"2023-11-08T19:31:11.666798+0000","last_fullsized":"2023-11-08T19:31:11.666798+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:14:18.610061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":78,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.667284+0000","last_change":"2023-11-08T19:26:06.541338+0000","last_active":"2023-11-08T19:31:11.667284+0000","last_peered":"2023-11-08T19:31:11.667284+0000","last_clean":"2023-11-08T19:31:11.667284+0000","last_became_active":"2023-11-08T19:26:04.602183+0000","last_became_peered":"2023-11-08T19:26:04.602183+0000","last_unstale":"2023-11-08T19:31:11.667284+0000","last_undegraded":"2023-11-08T19:31:11.667284+0000","last_fullsized":"2023-11-08T19:31:11.667284+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:06:47.446686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052029099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":57,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T19:30:47.742943+0000","last_change":"2023-11-08T19:27:09.181622+0000","last_active":"2023-11-08T19:30:47.742943+0000","last_peered":"2023-11-08T19:30:47.742943+0000","last_clean":"2023-11-08T19:30:47.742943+0000","last_became_active":"2023-11-08T19:27:07.184507+0000","last_became_peered":"2023-11-08T19:27:07.184507+0000","last_unstale":"2023-11-08T19:30:47.742943+0000","last_undegraded":"2023-11-08T19:30:47.742943+0000","last_fullsized":"2023-11-08T19:30:47.742943+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:58:48.357266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051924700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":46,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.668291+0000","last_change":"2023-11-08T19:27:08.194263+0000","last_active":"2023-11-08T19:31:11.668291+0000","last_peered":"2023-11-08T19:31:11.668291+0000","last_clean":"2023-11-08T19:31:11.668291+0000","last_became_active":"2023-11-08T19:27:08.193938+0000","last_became_peered":"2023-11-08T19:27:08.193938+0000","last_unstale":"2023-11-08T19:31:11.668291+0000","last_undegraded":"2023-11-08T19:31:11.668291+0000","last_fullsized":"2023-11-08T19:31:11.668291+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:40:49.009717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":76,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.668798+0000","last_change":"2023-11-08T19:26:06.265690+0000","last_active":"2023-11-08T19:31:11.668798+0000","last_peered":"2023-11-08T19:31:11.668798+0000","last_clean":"2023-11-08T19:31:11.668798+0000","last_became_active":"2023-11-08T19:26:04.603397+0000","last_became_peered":"2023-11-08T19:26:04.603397+0000","last_unstale":"2023-11-08T19:31:11.668798+0000","last_undegraded":"2023-11-08T19:31:11.668798+0000","last_fullsized":"2023-11-08T19:31:11.668798+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:50:43.827570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053361499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":53,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.668234+0000","last_change":"2023-11-08T19:27:09.282750+0000","last_active":"2023-11-08T19:31:11.668234+0000","last_peered":"2023-11-08T19:31:11.668234+0000","last_clean":"2023-11-08T19:31:11.668234+0000","last_became_active":"2023-11-08T19:27:07.189943+0000","last_became_peered":"2023-11-08T19:27:07.189943+0000","last_unstale":"2023-11-08T19:31:11.668234+0000","last_undegraded":"2023-11-08T19:31:11.668234+0000","last_fullsized":"2023-11-08T19:31:11.668234+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:31:16.081058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000398725,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":50,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T19:30:47.743464+0000","last_change":"2023-11-08T19:27:08.188147+0000","last_active":"2023-11-08T19:30:47.743464+0000","last_peered":"2023-11-08T19:30:47.743464+0000","last_clean":"2023-11-08T19:30:47.743464+0000","last_became_active":"2023-11-08T19:27:08.187487+0000","last_became_peered":"2023-11-08T19:27:08.187487+0000","last_unstale":"2023-11-08T19:30:47.743464+0000","last_undegraded":"2023-11-08T19:30:47.743464+0000","last_fullsized":"2023-11-08T19:30:47.743464+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:22:40.300185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":82,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.668250+0000","last_change":"2023-11-08T19:26:06.425133+0000","last_active":"2023-11-08T19:31:11.668250+0000","last_peered":"2023-11-08T19:31:11.668250+0000","last_clean":"2023-11-08T19:31:11.668250+0000","last_became_active":"2023-11-08T19:26:04.601103+0000","last_became_peered":"2023-11-08T19:26:04.601103+0000","last_unstale":"2023-11-08T19:31:11.668250+0000","last_undegraded":"2023-11-08T19:31:11.668250+0000","last_fullsized":"2023-11-08T19:31:11.668250+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:31:14.042561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000229244,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":57,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T19:30:47.741439+0000","last_change":"2023-11-08T19:27:09.179852+0000","last_active":"2023-11-08T19:30:47.741439+0000","last_peered":"2023-11-08T19:30:47.741439+0000","last_clean":"2023-11-08T19:30:47.741439+0000","last_became_active":"2023-11-08T19:27:07.191926+0000","last_became_peered":"2023-11-08T19:27:07.191926+0000","last_unstale":"2023-11-08T19:30:47.741439+0000","last_undegraded":"2023-11-08T19:30:47.741439+0000","last_fullsized":"2023-11-08T19:30:47.741439+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:20:38.080129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055457100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":50,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T19:30:47.743303+0000","last_change":"2023-11-08T19:27:08.186054+0000","last_active":"2023-11-08T19:30:47.743303+0000","last_peered":"2023-11-08T19:30:47.743303+0000","last_clean":"2023-11-08T19:30:47.743303+0000","last_became_active":"2023-11-08T19:27:08.185581+0000","last_became_peered":"2023-11-08T19:27:08.185581+0000","last_unstale":"2023-11-08T19:30:47.743303+0000","last_undegraded":"2023-11-08T19:30:47.743303+0000","last_fullsized":"2023-11-08T19:30:47.743303+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:00:55.878397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":86,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-11-08T19:30:55.778968+0000","last_change":"2023-11-08T19:26:05.609625+0000","last_active":"2023-11-08T19:30:55.778968+0000","last_peered":"2023-11-08T19:30:55.778968+0000","last_clean":"2023-11-08T19:30:55.778968+0000","last_became_active":"2023-11-08T19:26:04.606312+0000","last_became_peered":"2023-11-08T19:26:04.606312+0000","last_unstale":"2023-11-08T19:30:55.778968+0000","last_undegraded":"2023-11-08T19:30:55.778968+0000","last_fullsized":"2023-11-08T19:30:55.778968+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:54:42.032901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000505935,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":53,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.444366+0000","last_change":"2023-11-08T19:27:09.177985+0000","last_active":"2023-11-08T19:31:11.444366+0000","last_peered":"2023-11-08T19:31:11.444366+0000","last_clean":"2023-11-08T19:31:11.444366+0000","last_became_active":"2023-11-08T19:27:07.193254+0000","last_became_peered":"2023-11-08T19:27:07.193254+0000","last_unstale":"2023-11-08T19:31:11.444366+0000","last_undegraded":"2023-11-08T19:31:11.444366+0000","last_fullsized":"2023-11-08T19:31:11.444366+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:23:38.662629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041830799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":50,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T19:30:47.742560+0000","last_change":"2023-11-08T19:27:08.188860+0000","last_active":"2023-11-08T19:30:47.742560+0000","last_peered":"2023-11-08T19:30:47.742560+0000","last_clean":"2023-11-08T19:30:47.742560+0000","last_became_active":"2023-11-08T19:27:08.188560+0000","last_became_peered":"2023-11-08T19:27:08.188560+0000","last_unstale":"2023-11-08T19:30:47.742560+0000","last_undegraded":"2023-11-08T19:30:47.742560+0000","last_fullsized":"2023-11-08T19:30:47.742560+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:18:20.969491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":77,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.667266+0000","last_change":"2023-11-08T19:26:06.297255+0000","last_active":"2023-11-08T19:31:11.667266+0000","last_peered":"2023-11-08T19:31:11.667266+0000","last_clean":"2023-11-08T19:31:11.667266+0000","last_became_active":"2023-11-08T19:26:04.603979+0000","last_became_peered":"2023-11-08T19:26:04.603979+0000","last_unstale":"2023-11-08T19:31:11.667266+0000","last_undegraded":"2023-11-08T19:31:11.667266+0000","last_fullsized":"2023-11-08T19:31:11.667266+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:40:11.164951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079032099999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":53,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.002043+0000","last_change":"2023-11-08T19:27:09.331990+0000","last_active":"2023-11-08T19:31:11.002043+0000","last_peered":"2023-11-08T19:31:11.002043+0000","last_clean":"2023-11-08T19:31:11.002043+0000","last_became_active":"2023-11-08T19:27:07.185478+0000","last_became_peered":"2023-11-08T19:27:07.185478+0000","last_unstale":"2023-11-08T19:31:11.002043+0000","last_undegraded":"2023-11-08T19:31:11.002043+0000","last_fullsized":"2023-11-08T19:31:11.002043+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:20:38.080129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010661850000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574929,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33052,"kb_used_data":1872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338788,"statfs":{"total":96636764160,"available":96602918912,"internally_reserved":0,"allocated":1916928,"data_stored":1239855,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:30:56 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.54400000000000004,"15min":0.51100000000000001},"min":{"1min":0.45300000000000001,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.71899999999999997,"5min":0.78600000000000003,"15min":1.0149999999999999},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.53500000000000003,"15min":0.52100000000000002},"min":{"1min":0.42099999999999999,"5min":0.35099999999999998,"15min":0.32300000000000001},"max":{"1min":0.79300000000000004,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.61399999999999999}]},{"osd":1,"last update":"Wed Nov 8 19:30:56 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.59799999999999998,"15min":0.57599999999999996},"min":{"1min":0.47699999999999998,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.73499999999999999,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.623,"5min":0.59599999999999997,"15min":0.56699999999999995},"min":{"1min":0.505,"5min":0.40600000000000003,"15min":0.33900000000000002},"max":{"1min":0.79500000000000004,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.52400000000000002}]},{"osd":2,"last update":"Wed Nov 8 19:30:56 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.54400000000000004,"15min":0.505},"min":{"1min":0.372,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.67600000000000005,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.57399999999999995,"15min":0.55600000000000005},"min":{"1min":0.45900000000000002,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.78400000000000003,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.53600000000000003}]},{"osd":3,"last update":"Wed Nov 8 19:30:56 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.57499999999999996,"15min":0.56399999999999995},"min":{"1min":0.42199999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.77300000000000002,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.495},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.63800000000000001,"15min":0.59299999999999997},"min":{"1min":0.47399999999999998,"5min":0.437,"15min":0.40100000000000002},"max":{"1min":0.81399999999999995,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.63200000000000001}]},{"osd":4,"last update":"Wed Nov 8 19:30:56 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.501,"15min":0.505},"min":{"1min":0.32400000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.753,"5min":0.753,"15min":0.78600000000000003},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.53400000000000003,"15min":0.53700000000000003},"min":{"1min":0.35199999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.71799999999999997,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.39100000000000001}]},{"osd":5,"last update":"Wed Nov 8 19:30:56 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.56699999999999995,"15min":0.57499999999999996},"min":{"1min":0.35099999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.77300000000000002,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.627,"15min":0.63700000000000001},"min":{"1min":0.377,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.83899999999999997,"5min":0.89900000000000002,"15min":0.94299999999999995},"last":0.56599999999999995}]},{"osd":6,"last update":"Wed Nov 8 19:30:56 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.57999999999999996,"15min":0.59699999999999998},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.97299999999999998},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.59699999999999998,"15min":0.60499999999999998},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.78900000000000003,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.60199999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574929,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32520,"kb_used_data":1344,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339320,"statfs":{"total":96636764160,"available":96603463680,"internally_reserved":0,"allocated":1376256,"data_stored":690996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:30:57 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.55900000000000005,"15min":1.1459999999999999},"min":{"1min":0.41999999999999998,"5min":0.33200000000000002,"15min":0.27000000000000002},"max":{"1min":0.84599999999999997,"5min":0.91700000000000004,"15min":21.359999999999999},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.56899999999999995,"15min":0.81599999999999995},"min":{"1min":0.39700000000000002,"5min":0.378,"15min":0.30399999999999999},"max":{"1min":0.68700000000000006,"5min":0.90300000000000002,"15min":10.329000000000001},"last":0.42999999999999999}]},{"osd":1,"last update":"Wed Nov 8 19:30:57 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.53800000000000003,"15min":0.59499999999999997},"min":{"1min":0.41199999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.81399999999999995,"5min":0.86299999999999999,"15min":3.4239999999999999},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.60499999999999998,"15min":0.63400000000000001},"min":{"1min":0.35899999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":2.9260000000000002,"5min":2.9260000000000002,"15min":3.4809999999999999},"last":0.41199999999999998}]},{"osd":2,"last update":"Wed Nov 8 19:30:57 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.58599999999999997,"15min":0.83399999999999996},"min":{"1min":0.51800000000000002,"5min":0.38500000000000001,"15min":0.34399999999999997},"max":{"1min":0.84199999999999997,"5min":0.96399999999999997,"15min":10.398},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.56799999999999995,"15min":0.66200000000000003},"min":{"1min":0.41799999999999998,"5min":0.318,"15min":0.318},"max":{"1min":0.77500000000000002,"5min":0.89100000000000001,"15min":5.0170000000000003},"last":0.47399999999999998}]},{"osd":3,"last update":"Wed Nov 8 19:30:57 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.63100000000000001,"15min":0.66100000000000003},"min":{"1min":0.46200000000000002,"5min":0.34599999999999997,"15min":0.317},"max":{"1min":3.0089999999999999,"5min":3.0089999999999999,"15min":3.5710000000000002},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.59699999999999998,"15min":0.85999999999999999},"min":{"1min":0.39000000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.81100000000000005,"5min":0.93500000000000005,"15min":10.368},"last":0.51600000000000001}]},{"osd":4,"last update":"Wed Nov 8 19:30:57 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54400000000000004,"15min":0.52000000000000002},"min":{"1min":0.34699999999999998,"5min":0.33000000000000002,"15min":0.29199999999999998},"max":{"1min":0.79100000000000004,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.5},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.55900000000000005,"15min":0.55800000000000005},"min":{"1min":0.45000000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.73799999999999999,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.60999999999999999}]},{"osd":5,"last update":"Wed Nov 8 19:30:57 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.55000000000000004,"15min":0.53200000000000003},"min":{"1min":0.42699999999999999,"5min":0.34599999999999997,"15min":0.307},"max":{"1min":0.70499999999999996,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.60399999999999998,"15min":0.58599999999999997},"min":{"1min":0.41199999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.54400000000000004}]},{"osd":7,"last update":"Wed Nov 8 19:30:57 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.60999999999999999,"15min":0.58299999999999996},"min":{"1min":0.39700000000000002,"5min":0.39500000000000002,"15min":0.34699999999999998},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.60599999999999998,"15min":0.59199999999999997},"min":{"1min":0.46300000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.82199999999999995,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.59999999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574930,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32456,"kb_used_data":1276,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339384,"statfs":{"total":96636764160,"available":96603529216,"internally_reserved":0,"allocated":1306624,"data_stored":639107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:30:55 2023","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.35499999999999998,"15min":0.374},"min":{"1min":0.222,"5min":0.219,"15min":0.219},"max":{"1min":0.437,"5min":0.69699999999999995,"15min":1.389},"last":0.28299999999999997},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.36399999999999999,"15min":0.61699999999999999},"min":{"1min":0.255,"5min":0.23000000000000001,"15min":0.21199999999999999},"max":{"1min":0.52900000000000003,"5min":0.622,"15min":9.1370000000000005},"last":0.36399999999999999}]},{"osd":2,"last update":"Wed Nov 8 19:30:55 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.35899999999999999,"15min":0.36499999999999999},"min":{"1min":0.248,"5min":0.216,"15min":0.216},"max":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":1.2709999999999999},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.39200000000000002,"15min":0.41899999999999998},"min":{"1min":0.27700000000000002,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.53400000000000003,"5min":0.63600000000000001,"15min":1.425},"last":0.41799999999999998}]},{"osd":3,"last update":"Wed Nov 8 19:30:55 2023","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.40899999999999997,"15min":0.74399999999999999},"min":{"1min":0.29599999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.54600000000000004,"5min":0.63800000000000001,"15min":11.957000000000001},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.39100000000000001,"15min":0.91800000000000004},"min":{"1min":0.27900000000000003,"5min":0.252,"15min":0.252},"max":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":18.039000000000001},"last":0.25}]},{"osd":4,"last update":"Wed Nov 8 19:30:55 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59299999999999997,"15min":0.751},"min":{"1min":0.35199999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.81100000000000005,"5min":1.4370000000000001,"15min":6.181},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.60999999999999999,"15min":0.63},"min":{"1min":0.41399999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.83899999999999997,"5min":1.411,"15min":1.488},"last":0.57699999999999996}]},{"osd":5,"last update":"Wed Nov 8 19:30:55 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61199999999999999,"15min":0.86099999999999999},"min":{"1min":0.375,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.85399999999999998,"5min":1.421,"15min":9.2420000000000009},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60599999999999998,"15min":0.78900000000000003},"min":{"1min":0.42499999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.80100000000000005,"5min":1.2969999999999999,"15min":6.2110000000000003},"last":0.63}]},{"osd":6,"last update":"Wed Nov 8 19:30:55 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.63500000000000001,"15min":0.86899999999999999},"min":{"1min":0.40799999999999997,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.78700000000000003,"5min":1.4530000000000001,"15min":9.1880000000000006},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63500000000000001,"15min":0.65600000000000003},"min":{"1min":0.44500000000000001,"5min":0.38,"15min":0.38},"max":{"1min":0.86399999999999999,"5min":1.272,"15min":1.462},"last":0.65600000000000003}]},{"osd":7,"last update":"Wed Nov 8 19:30:55 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.625,"15min":0.78800000000000003},"min":{"1min":0.41499999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0309999999999999,"5min":1.3979999999999999,"15min":6.1479999999999997},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.64600000000000002,"15min":0.90000000000000002},"min":{"1min":0.51300000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.97499999999999998,"5min":1.353,"15min":9.2170000000000005},"last":0.64500000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574930,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33076,"kb_used_data":1896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338764,"statfs":{"total":96636764160,"available":96602894336,"internally_reserved":0,"allocated":1941504,"data_stored":1263942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Nov 8 19:31:01 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.33300000000000002,"15min":0.45900000000000002},"min":{"1min":0.20899999999999999,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.51000000000000001,"5min":0.55300000000000005,"15min":4.1200000000000001},"last":0.376},{"interface":"front","average":{"1min":0.439,"5min":0.371,"15min":0.91900000000000004},"min":{"1min":0.22800000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.62,"5min":0.62,"15min":16.940999999999999},"last":0.34300000000000003}]},{"osd":2,"last update":"Wed Nov 8 19:31:01 2023","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.35299999999999998,"15min":0.38700000000000001},"min":{"1min":0.25,"5min":0.22,"15min":0.22},"max":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":1.1899999999999999},"last":0.26700000000000002},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.39100000000000001,"15min":0.60499999999999998},"min":{"1min":0.32400000000000001,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":7.0209999999999999},"last":0.29099999999999998}]},{"osd":3,"last update":"Wed Nov 8 19:31:01 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.38400000000000001,"15min":0.51200000000000001},"min":{"1min":0.29899999999999999,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":4.0940000000000003},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.39500000000000002,"15min":0.433},"min":{"1min":0.32500000000000001,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.61899999999999999,"5min":0.62,"15min":1.2549999999999999},"last":0.312}]},{"osd":4,"last update":"Wed Nov 8 19:31:01 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.58899999999999997,"15min":0.71699999999999997},"min":{"1min":0.42799999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":4.0049999999999999},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.58399999999999996,"15min":0.623},"min":{"1min":0.442,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.746,"5min":0.79200000000000004,"15min":1.2270000000000001},"last":0.54000000000000004}]},{"osd":5,"last update":"Wed Nov 8 19:31:01 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.59699999999999998,"15min":0.72799999999999998},"min":{"1min":0.47199999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":4.3090000000000002},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.61199999999999999,"15min":0.72399999999999998},"min":{"1min":0.436,"5min":0.373,"15min":0.373},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":4.0709999999999997},"last":0.56999999999999995}]},{"osd":6,"last update":"Wed Nov 8 19:31:01 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.621,"15min":0.65800000000000003},"min":{"1min":0.40999999999999998,"5min":0.373,"15min":0.373},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":1.2849999999999999},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.64900000000000002,"15min":0.78500000000000003},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":4.2560000000000002},"last":0.64100000000000001}]},{"osd":7,"last update":"Wed Nov 8 19:31:01 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.625,"15min":0.74299999999999999},"min":{"1min":0.47099999999999997,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":4.2850000000000001},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.64700000000000002,"15min":0.77000000000000002},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.002,"5min":1.002,"15min":4.0460000000000003},"last":0.63}]}]},{"osd":2,"up_from":13,"seq":55834574930,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28284,"kb_used_data":1264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343556,"statfs":{"total":96636764160,"available":96607801344,"internally_reserved":0,"allocated":1294336,"data_stored":631401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:30:53 2023","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.35099999999999998,"15min":0.33800000000000002},"min":{"1min":0.23599999999999999,"5min":0.182,"15min":0.182},"max":{"1min":0.629,"5min":0.629,"15min":0.629},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.35899999999999999,"15min":0.55900000000000005},"min":{"1min":0.26800000000000002,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.56499999999999995,"5min":0.68000000000000005,"15min":6.0999999999999996},"last":0.34899999999999998}]},{"osd":1,"last update":"Wed Nov 8 19:30:53 2023","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.35599999999999998,"15min":0.34599999999999997},"min":{"1min":0.247,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.38800000000000001,"15min":0.39000000000000001},"min":{"1min":0.30299999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.63100000000000001,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.41099999999999998}]},{"osd":3,"last update":"Wed Nov 8 19:30:53 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.40600000000000003,"15min":0.39200000000000002},"min":{"1min":0.28199999999999997,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.60999999999999999,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.40000000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"min":{"1min":0.28799999999999998,"5min":0.217,"15min":0.217},"max":{"1min":0.621,"5min":0.65900000000000003,"15min":0.70299999999999996},"last":0.32900000000000001}]},{"osd":4,"last update":"Wed Nov 8 19:30:53 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56000000000000005,"15min":0.57599999999999996},"min":{"1min":0.41699999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.58099999999999996,"15min":0.58299999999999996},"min":{"1min":0.40500000000000003,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.872,"5min":0.872,"15min":0.90400000000000003},"last":0.59099999999999997}]},{"osd":5,"last update":"Wed Nov 8 19:30:53 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61799999999999999,"15min":0.59399999999999997},"min":{"1min":0.40799999999999997,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.622,"5min":0.60599999999999998,"15min":0.61099999999999999},"min":{"1min":0.48799999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.60099999999999998}]},{"osd":6,"last update":"Wed Nov 8 19:30:53 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.59399999999999997,"15min":0.59799999999999998},"min":{"1min":0.432,"5min":0.379,"15min":0.371},"max":{"1min":0.81000000000000005,"5min":0.83599999999999997,"15min":0.84799999999999998},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.59999999999999998,"15min":0.58599999999999997},"min":{"1min":0.42799999999999999,"5min":0.40300000000000002,"15min":0.36599999999999999},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.53400000000000003}]},{"osd":7,"last update":"Wed Nov 8 19:30:53 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.60899999999999999,"15min":0.58899999999999997},"min":{"1min":0.46200000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.61699999999999999,"15min":0.60899999999999999},"min":{"1min":0.41699999999999998,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.79600000000000004,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.623}]}]},{"osd":3,"up_from":13,"seq":55834574931,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28296,"kb_used_data":1276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343544,"statfs":{"total":96636764160,"available":96607789056,"internally_reserved":0,"allocated":1306624,"data_stored":639107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:30:59 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.33500000000000002,"15min":0.34000000000000002},"min":{"1min":0.254,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.48599999999999999,"5min":0.49099999999999999,"15min":0.626},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.378,"5min":0.372,"15min":0.35599999999999998},"min":{"1min":0.26700000000000002,"5min":0.23699999999999999,"15min":0.23300000000000001},"max":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.60799999999999998},"last":0.46500000000000002}]},{"osd":1,"last update":"Wed Nov 8 19:30:59 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.35999999999999999,"15min":0.35599999999999998},"min":{"1min":0.245,"5min":0.24099999999999999,"15min":0.23999999999999999},"max":{"1min":0.53300000000000003,"5min":0.61699999999999999,"15min":0.61699999999999999},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.40100000000000002,"15min":0.39000000000000001},"min":{"1min":0.30099999999999999,"5min":0.27900000000000003,"15min":0.27700000000000002},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.82099999999999995},"last":0.496}]},{"osd":2,"last update":"Wed Nov 8 19:30:59 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.40999999999999998,"15min":0.38300000000000001},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28799999999999998},"max":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58499999999999996},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.41999999999999998,"15min":0.40899999999999997},"min":{"1min":0.30499999999999999,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.59799999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.55900000000000005}]},{"osd":4,"last update":"Wed Nov 8 19:30:59 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.63,"15min":0.60799999999999998},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.85699999999999998,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63700000000000001,"15min":0.61499999999999999},"min":{"1min":0.38600000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.89600000000000002,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.67100000000000004}]},{"osd":5,"last update":"Wed Nov 8 19:30:59 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59899999999999998,"15min":0.58299999999999996},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.83099999999999996,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.623,"5min":0.622,"15min":0.59199999999999997},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.879,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.60399999999999998}]},{"osd":6,"last update":"Wed Nov 8 19:30:59 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.61399999999999999,"15min":0.59099999999999997},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41399999999999998},"max":{"1min":0.83899999999999997,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63300000000000001,"15min":0.61299999999999999},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.90500000000000003},"last":0.79700000000000004}]},{"osd":7,"last update":"Wed Nov 8 19:30:59 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.626,"15min":0.59999999999999998},"min":{"1min":0.41899999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":1.0009999999999999},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.63900000000000001,"15min":0.60999999999999999},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.85099999999999998,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.63800000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574930,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28296,"kb_used_data":1276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343544,"statfs":{"total":96636764160,"available":96607789056,"internally_reserved":0,"allocated":1306624,"data_stored":639107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:30:58 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54700000000000004,"15min":0.69899999999999995},"min":{"1min":0.311,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.71499999999999997,"5min":0.84799999999999998,"15min":6.5970000000000004},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.53500000000000003,"15min":0.56299999999999994},"min":{"1min":0.36299999999999999,"5min":0.32200000000000001,"15min":0.317},"max":{"1min":0.73099999999999998,"5min":0.75900000000000001,"15min":2.5030000000000001},"last":0.47099999999999997}]},{"osd":1,"last update":"Wed Nov 8 19:30:58 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.55400000000000005,"15min":0.51500000000000001},"min":{"1min":0.41699999999999998,"5min":0.32900000000000001,"15min":0.24299999999999999},"max":{"1min":0.72799999999999998,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.55900000000000005,"15min":0.52500000000000002},"min":{"1min":0.49299999999999999,"5min":0.312,"15min":0.312},"max":{"1min":0.72799999999999998,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.64500000000000002}]},{"osd":2,"last update":"Wed Nov 8 19:30:58 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55300000000000005},"min":{"1min":0.505,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.64400000000000002,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.58799999999999997,"15min":0.76000000000000001},"min":{"1min":0.50700000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.70999999999999996,"5min":0.871,"15min":6.6920000000000002},"last":0.44500000000000001}]},{"osd":3,"last update":"Wed Nov 8 19:30:58 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.56499999999999995,"15min":0.58099999999999996},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.69799999999999995,"5min":0.79800000000000004,"15min":1.0700000000000001},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.61799999999999999,"15min":0.61299999999999999},"min":{"1min":0.47799999999999998,"5min":0.41699999999999998,"15min":0.34599999999999997},"max":{"1min":0.70399999999999996,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.60199999999999998}]},{"osd":5,"last update":"Wed Nov 8 19:30:58 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.501,"15min":0.53100000000000003},"min":{"1min":0.39100000000000001,"5min":0.29599999999999999,"15min":0.28100000000000003},"max":{"1min":0.66100000000000003,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.51700000000000002,"15min":0.57099999999999995},"min":{"1min":0.377,"5min":0.32600000000000001,"15min":0.30299999999999999},"max":{"1min":0.65700000000000003,"5min":0.876,"15min":1.1060000000000001},"last":0.501}]},{"osd":6,"last update":"Wed Nov 8 19:30:58 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.54700000000000004,"15min":0.57199999999999995},"min":{"1min":0.35199999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.71899999999999997,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.60299999999999998,"15min":0.60999999999999999},"min":{"1min":0.373,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.74099999999999999,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.61899999999999999}]},{"osd":7,"last update":"Wed Nov 8 19:30:58 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.55800000000000005,"15min":0.60999999999999999},"min":{"1min":0.41599999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.68000000000000005,"5min":0.84499999999999997,"15min":1.141},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.59199999999999997,"15min":0.63},"min":{"1min":0.41399999999999998,"5min":0.38700000000000001,"15min":0.33000000000000002},"max":{"1min":0.76400000000000001,"5min":0.86399999999999999,"15min":0.86799999999999999},"last":0.63600000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574930,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28284,"kb_used_data":1264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343556,"statfs":{"total":96636764160,"available":96607801344,"internally_reserved":0,"allocated":1294336,"data_stored":631457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:30:51 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.57199999999999995,"15min":0.53900000000000003},"min":{"1min":0.35199999999999998,"5min":0.307,"15min":0.307},"max":{"1min":3.9689999999999999,"5min":3.9689999999999999,"15min":3.9689999999999999},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.54800000000000004,"15min":0.52200000000000002},"min":{"1min":0.41699999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.68899999999999995,"5min":0.746,"15min":0.746},"last":0.53400000000000003}]},{"osd":1,"last update":"Wed Nov 8 19:30:51 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.55300000000000005,"15min":0.53300000000000003},"min":{"1min":0.42399999999999999,"5min":0.37,"15min":0.37},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.54900000000000004,"15min":0.54100000000000004},"min":{"1min":0.44400000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.753,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.51400000000000001}]},{"osd":2,"last update":"Wed Nov 8 19:30:51 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.56399999999999995,"15min":0.54400000000000004},"min":{"1min":0.40000000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.749,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.56499999999999995,"15min":0.57299999999999995},"min":{"1min":0.46400000000000002,"5min":0.316,"15min":0.316},"max":{"1min":0.746,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.56499999999999995}]},{"osd":3,"last update":"Wed Nov 8 19:30:51 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.58499999999999996,"15min":0.56200000000000006},"min":{"1min":0.39600000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.73699999999999999,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.57999999999999996,"15min":0.59599999999999997},"min":{"1min":0.39900000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.79000000000000004,"5min":0.87,"15min":0.87},"last":0.61799999999999999}]},{"osd":4,"last update":"Wed Nov 8 19:30:51 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.495,"15min":0.54000000000000004},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.71899999999999997,"5min":0.80500000000000005,"15min":0.86799999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.53900000000000003,"15min":0.56100000000000005},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.751,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.71499999999999997}]},{"osd":6,"last update":"Wed Nov 8 19:30:51 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.56100000000000005,"15min":0.59599999999999997},"min":{"1min":0.39700000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.73599999999999999,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59199999999999997,"15min":0.60899999999999999},"min":{"1min":0.41399999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.76400000000000001,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.68500000000000005}]},{"osd":7,"last update":"Wed Nov 8 19:30:51 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.57099999999999995,"15min":0.59899999999999998},"min":{"1min":0.38200000000000001,"5min":0.371,"15min":0.36899999999999999},"max":{"1min":0.74099999999999999,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.746},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58999999999999997,"15min":0.63},"min":{"1min":0.433,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.91900000000000004},"last":0.70499999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-08T19:31:16.455 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-08T19:31:16.607 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.606+0000 7f0095526700 1 Processor -- start 2023-11-08T19:31:16.608 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.606+0000 7f0095526700 1 -- start start 2023-11-08T19:31:16.610 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.607+0000 7f0095526700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f009015dc80 0x7f00900d1f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:16.610 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.607+0000 7f0095526700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f00900d3690 0x7f00900b9690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:16.611 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.607+0000 7f0095526700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f00900ce980 0x7f00900cee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:16.611 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.607+0000 7f0095526700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00900cf370 con 0x7f00900d3690 2023-11-08T19:31:16.611 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.607+0000 7f0095526700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00900b9bd0 con 0x7f00900ce980 2023-11-08T19:31:16.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.607+0000 7f0095526700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00900d3200 con 0x7f009015dc80 2023-11-08T19:31:16.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.607+0000 7f008e7fc700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f00900d3690 0x7f00900b9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:16.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.607+0000 7f008e7fc700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f00900d3690 0x7f00900b9690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57112/0 (socket says 172.21.15.143:57112) 2023-11-08T19:31:16.613 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.607+0000 7f008e7fc700 1 -- 172.21.15.143:0/3666530644 learned_addr learned my addr 172.21.15.143:0/3666530644 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:31:16.613 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.607+0000 7f008effd700 1 --2- 172.21.15.143:0/3666530644 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f009015dc80 0x7f00900d1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:16.613 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.607+0000 7f008f7fe700 1 --2- 172.21.15.143:0/3666530644 >> v2:172.21.15.174:3300/0 conn(0x7f00900ce980 0x7f00900cee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:16.614 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.607+0000 7f008f7fe700 1 -- 172.21.15.143:0/3666530644 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f009015dc80 msgr2=0x7f00900d1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:16.614 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.607+0000 7f008f7fe700 1 --2- 172.21.15.143:0/3666530644 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f009015dc80 0x7f00900d1f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:16.614 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.607+0000 7f008f7fe700 1 -- 172.21.15.143:0/3666530644 >> v2:172.21.15.143:3300/0 conn(0x7f00900d3690 msgr2=0x7f00900b9690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:16.614 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.607+0000 7f008f7fe700 1 --2- 172.21.15.143:0/3666530644 >> v2:172.21.15.143:3300/0 conn(0x7f00900d3690 0x7f00900b9690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:16.615 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.607+0000 7f008f7fe700 1 -- 172.21.15.143:0/3666530644 --> v2:172.21.15.174:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f009015e060 con 0x7f00900ce980 2023-11-08T19:31:16.615 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.607+0000 7f008effd700 1 --2- 172.21.15.143:0/3666530644 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f009015dc80 0x7f00900d1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T19:31:16.615 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.608+0000 7f008f7fe700 1 --2- 172.21.15.143:0/3666530644 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f00900ce980 0x7f00900cee30 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f00900ce190 tx=0x7f0088014ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb1a24c95665820a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:16.616 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.608+0000 7f008dffb700 1 -- 172.21.15.143:0/3666530644 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0088014e70 con 0x7f00900ce980 2023-11-08T19:31:16.616 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.608+0000 7f008dffb700 1 -- 172.21.15.143:0/3666530644 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f008800fd50 con 0x7f00900ce980 2023-11-08T19:31:16.616 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.608+0000 7f008dffb700 1 -- 172.21.15.143:0/3666530644 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0088013b20 con 0x7f00900ce980 2023-11-08T19:31:16.617 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.609+0000 7f0095526700 1 -- 172.21.15.143:0/3666530644 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f00900ce980 msgr2=0x7f00900cee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:16.617 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.609+0000 7f0095526700 1 --2- 172.21.15.143:0/3666530644 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f00900ce980 0x7f00900cee30 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f00900ce190 tx=0x7f0088014ce0 comp rx=0 tx=0).stop 2023-11-08T19:31:16.617 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.609+0000 7f0095526700 1 -- 172.21.15.143:0/3666530644 shutdown_connections 2023-11-08T19:31:16.618 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.609+0000 7f0095526700 1 --2- 172.21.15.143:0/3666530644 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f00900ce980 0x7f00900cee30 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:16.618 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.609+0000 7f0095526700 1 --2- 172.21.15.143:0/3666530644 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f009015dc80 0x7f00900d1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:16.619 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.609+0000 7f0095526700 1 --2- 172.21.15.143:0/3666530644 >> v2:172.21.15.143:3300/0 conn(0x7f00900d3690 0x7f00900b9690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:16.619 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.609+0000 7f0095526700 1 -- 172.21.15.143:0/3666530644 >> 172.21.15.143:0/3666530644 conn(0x7f00900668e0 msgr2=0x7f0090066cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:16.621 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.609+0000 7f0095526700 1 -- 172.21.15.143:0/3666530644 shutdown_connections 2023-11-08T19:31:16.621 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.609+0000 7f0095526700 1 -- 172.21.15.143:0/3666530644 wait complete. 2023-11-08T19:31:16.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.610+0000 7f0095526700 1 Processor -- start 2023-11-08T19:31:16.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.610+0000 7f0095526700 1 -- start start 2023-11-08T19:31:16.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.611+0000 7f0095526700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f00900ce980 0x7f009015e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:16.626 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.611+0000 7f0095526700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f00900d3690 0x7f00900c76e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:16.626 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.611+0000 7f0095526700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f009015dc80 0x7f009014b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:16.626 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.611+0000 7f0095526700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f009014b9e0 con 0x7f00900ce980 2023-11-08T19:31:16.626 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.611+0000 7f0095526700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f009015e860 con 0x7f00900d3690 2023-11-08T19:31:16.627 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.611+0000 7f0095526700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0090160a20 con 0x7f009015dc80 2023-11-08T19:31:16.627 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.611+0000 7f008effd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f00900ce980 0x7f009015e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:16.627 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.611+0000 7f008effd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f00900ce980 0x7f009015e280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57116/0 (socket says 172.21.15.143:57116) 2023-11-08T19:31:16.628 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.611+0000 7f008effd700 1 -- 172.21.15.143:0/2713351700 learned_addr learned my addr 172.21.15.143:0/2713351700 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:31:16.628 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.611+0000 7f008f7fe700 1 --2- 172.21.15.143:0/2713351700 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f009015dc80 0x7f009014b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:16.628 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.611+0000 7f008e7fc700 1 --2- 172.21.15.143:0/2713351700 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f00900d3690 0x7f00900c76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:16.628 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.611+0000 7f008effd700 1 -- 172.21.15.143:0/2713351700 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f009015dc80 msgr2=0x7f009014b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:16.629 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.611+0000 7f008effd700 1 --2- 172.21.15.143:0/2713351700 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f009015dc80 0x7f009014b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:16.629 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.611+0000 7f008effd700 1 -- 172.21.15.143:0/2713351700 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f00900d3690 msgr2=0x7f00900c76e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:16.629 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.611+0000 7f008effd700 1 --2- 172.21.15.143:0/2713351700 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f00900d3690 0x7f00900c76e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:16.629 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.611+0000 7f008effd700 1 -- 172.21.15.143:0/2713351700 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00800094a0 con 0x7f00900ce980 2023-11-08T19:31:16.630 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.611+0000 7f008e7fc700 1 --2- 172.21.15.143:0/2713351700 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f00900d3690 0x7f00900c76e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:31:16.630 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.611+0000 7f008effd700 1 --2- 172.21.15.143:0/2713351700 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f00900ce980 0x7f009015e280 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f008000ec10 tx=0x7f008000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:16.630 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.612+0000 7f00777fe700 1 -- 172.21.15.143:0/2713351700 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f008000c850 con 0x7f00900ce980 2023-11-08T19:31:16.631 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.612+0000 7f00777fe700 1 -- 172.21.15.143:0/2713351700 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f008000c9b0 con 0x7f00900ce980 2023-11-08T19:31:16.631 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.612+0000 7f0095526700 1 -- 172.21.15.143:0/2713351700 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0088014660 con 0x7f00900ce980 2023-11-08T19:31:16.631 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.612+0000 7f00777fe700 1 -- 172.21.15.143:0/2713351700 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f008000cd40 con 0x7f00900ce980 2023-11-08T19:31:16.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.612+0000 7f0095526700 1 -- 172.21.15.143:0/2713351700 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0090160fa0 con 0x7f00900ce980 2023-11-08T19:31:16.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.613+0000 7f0095526700 1 -- 172.21.15.143:0/2713351700 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f009004eee0 con 0x7f00900ce980 2023-11-08T19:31:16.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.613+0000 7f00777fe700 1 -- 172.21.15.143:0/2713351700 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0080013470 con 0x7f00900ce980 2023-11-08T19:31:16.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.613+0000 7f00777fe700 1 --2- 172.21.15.143:0/2713351700 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f007c06af30 0x7f007c06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:16.633 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.613+0000 7f00777fe700 1 -- 172.21.15.143:0/2713351700 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6651+0+0 (secure 0 0 0) 0x7f0080089a60 con 0x7f00900ce980 2023-11-08T19:31:16.633 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.614+0000 7f008e7fc700 1 --2- 172.21.15.143:0/2713351700 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f007c06af30 0x7f007c06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:16.633 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.614+0000 7f008e7fc700 1 --2- 172.21.15.143:0/2713351700 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f007c06af30 0x7f007c06d3f0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f00900672f0 tx=0x7f0084008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:16.633 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.618+0000 7f00777fe700 1 -- 172.21.15.143:0/2713351700 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f00800546d0 con 0x7f00900ce980 2023-11-08T19:31:16.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.804+0000 7f0095526700 1 -- 172.21.15.143:0/2713351700 --> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f00900d28d0 con 0x7f007c06af30 2023-11-08T19:31:16.813 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.812+0000 7f00777fe700 1 -- 172.21.15.143:0/2713351700 <== mgr.4099 v2:172.21.15.174:6832/4037280178 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+293897 (secure 0 0 0) 0x7f0084004030 con 0x7f007c06af30 2023-11-08T19:31:16.817 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:31:16.819 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.815+0000 7f0095526700 1 -- 172.21.15.143:0/2713351700 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f007c06af30 msgr2=0x7f007c06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:16.820 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.815+0000 7f0095526700 1 --2- 172.21.15.143:0/2713351700 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f007c06af30 0x7f007c06d3f0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f00900672f0 tx=0x7f0084008040 comp rx=0 tx=0).stop 2023-11-08T19:31:16.820 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.815+0000 7f0095526700 1 -- 172.21.15.143:0/2713351700 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f00900ce980 msgr2=0x7f009015e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:16.820 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.815+0000 7f0095526700 1 --2- 172.21.15.143:0/2713351700 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f00900ce980 0x7f009015e280 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f008000ec10 tx=0x7f008000ef20 comp rx=0 tx=0).stop 2023-11-08T19:31:16.821 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.815+0000 7f0095526700 1 -- 172.21.15.143:0/2713351700 shutdown_connections 2023-11-08T19:31:16.821 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.815+0000 7f0095526700 1 --2- 172.21.15.143:0/2713351700 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f007c06af30 0x7f007c06d3f0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:16.821 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.815+0000 7f0095526700 1 --2- 172.21.15.143:0/2713351700 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f009015dc80 0x7f009014b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:16.821 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.815+0000 7f0095526700 1 --2- 172.21.15.143:0/2713351700 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f00900ce980 0x7f009015e280 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:16.822 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.815+0000 7f0095526700 1 --2- 172.21.15.143:0/2713351700 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f00900d3690 0x7f00900c76e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:16.822 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.815+0000 7f0095526700 1 -- 172.21.15.143:0/2713351700 >> 172.21.15.143:0/2713351700 conn(0x7f00900668e0 msgr2=0x7f0090159db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:16.822 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.815+0000 7f0095526700 1 -- 172.21.15.143:0/2713351700 shutdown_connections 2023-11-08T19:31:16.822 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.816+0000 7f0095526700 1 -- 172.21.15.143:0/2713351700 wait complete. 2023-11-08T19:31:16.823 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2023-11-08T19:31:16.829 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":296,"stamp":"2023-11-08T19:31:16.623813+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":371,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":244204,"kb_used_data":11400,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754730516,"statfs":{"total":773094113280,"available":772844048384,"internally_reserved":0,"allocated":11673600,"data_stored":6319993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-68399,"num_objects":-41,"num_object_clones":0,"num_object_copies":-82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-41,"num_whiteouts":0,"num_read":-469,"num_read_kb":-599,"num_write":-179,"num_write_kb":-143,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001715"},"pg_stats":[{"pgid":"12.d","version":"0'0","reported_seq":53,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.009381+0000","last_change":"2023-11-08T19:27:08.190826+0000","last_active":"2023-11-08T19:31:13.009381+0000","last_peered":"2023-11-08T19:31:13.009381+0000","last_clean":"2023-11-08T19:31:13.009381+0000","last_became_active":"2023-11-08T19:27:08.190440+0000","last_became_peered":"2023-11-08T19:27:08.190440+0000","last_unstale":"2023-11-08T19:31:13.009381+0000","last_undegraded":"2023-11-08T19:31:13.009381+0000","last_fullsized":"2023-11-08T19:31:13.009381+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:42:48.376462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":255,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.008215+0000","last_change":"2023-11-08T19:24:45.888122+0000","last_active":"2023-11-08T19:31:13.008215+0000","last_peered":"2023-11-08T19:31:13.008215+0000","last_clean":"2023-11-08T19:31:13.008215+0000","last_became_active":"2023-11-08T19:24:45.887048+0000","last_became_peered":"2023-11-08T19:24:45.887048+0000","last_unstale":"2023-11-08T19:31:13.008215+0000","last_undegraded":"2023-11-08T19:31:13.008215+0000","last_fullsized":"2023-11-08T19:31:13.008215+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:44.575133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:44.575133+0000","last_clean_scrub_stamp":"2023-11-08T19:24:44.575133+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:19:20.202062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":78,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.668985+0000","last_change":"2023-11-08T19:26:06.542044+0000","last_active":"2023-11-08T19:31:11.668985+0000","last_peered":"2023-11-08T19:31:11.668985+0000","last_clean":"2023-11-08T19:31:11.668985+0000","last_became_active":"2023-11-08T19:26:04.603056+0000","last_became_peered":"2023-11-08T19:26:04.603056+0000","last_unstale":"2023-11-08T19:31:11.668985+0000","last_undegraded":"2023-11-08T19:31:11.668985+0000","last_fullsized":"2023-11-08T19:31:11.668985+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:41:50.771457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000517414,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":57,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.667621+0000","last_change":"2023-11-08T19:27:09.496842+0000","last_active":"2023-11-08T19:31:11.667621+0000","last_peered":"2023-11-08T19:31:11.667621+0000","last_clean":"2023-11-08T19:31:11.667621+0000","last_became_active":"2023-11-08T19:27:07.185776+0000","last_became_peered":"2023-11-08T19:27:07.185776+0000","last_unstale":"2023-11-08T19:31:11.667621+0000","last_undegraded":"2023-11-08T19:31:11.667621+0000","last_fullsized":"2023-11-08T19:31:11.667621+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:21:13.923970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031747200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":50,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.668213+0000","last_change":"2023-11-08T19:27:08.188241+0000","last_active":"2023-11-08T19:31:11.668213+0000","last_peered":"2023-11-08T19:31:11.668213+0000","last_clean":"2023-11-08T19:31:11.668213+0000","last_became_active":"2023-11-08T19:27:08.187863+0000","last_became_peered":"2023-11-08T19:27:08.187863+0000","last_unstale":"2023-11-08T19:31:11.668213+0000","last_undegraded":"2023-11-08T19:31:11.668213+0000","last_fullsized":"2023-11-08T19:31:11.668213+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:45:22.084015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":76,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.667763+0000","last_change":"2023-11-08T19:26:06.265419+0000","last_active":"2023-11-08T19:31:11.667763+0000","last_peered":"2023-11-08T19:31:11.667763+0000","last_clean":"2023-11-08T19:31:11.667763+0000","last_became_active":"2023-11-08T19:26:04.605333+0000","last_became_peered":"2023-11-08T19:26:04.605333+0000","last_unstale":"2023-11-08T19:31:11.667763+0000","last_undegraded":"2023-11-08T19:31:11.667763+0000","last_fullsized":"2023-11-08T19:31:11.667763+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:01:56.991346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064606400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":60,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.010017+0000","last_change":"2023-11-08T19:27:09.182113+0000","last_active":"2023-11-08T19:31:13.010017+0000","last_peered":"2023-11-08T19:31:13.010017+0000","last_clean":"2023-11-08T19:31:13.010017+0000","last_became_active":"2023-11-08T19:27:07.189939+0000","last_became_peered":"2023-11-08T19:27:07.189939+0000","last_unstale":"2023-11-08T19:31:13.010017+0000","last_undegraded":"2023-11-08T19:31:13.010017+0000","last_fullsized":"2023-11-08T19:31:13.010017+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:41:08.822366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036507500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":46,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.444341+0000","last_change":"2023-11-08T19:27:08.187127+0000","last_active":"2023-11-08T19:31:11.444341+0000","last_peered":"2023-11-08T19:31:11.444341+0000","last_clean":"2023-11-08T19:31:11.444341+0000","last_became_active":"2023-11-08T19:27:08.182648+0000","last_became_peered":"2023-11-08T19:27:08.182648+0000","last_unstale":"2023-11-08T19:31:11.444341+0000","last_undegraded":"2023-11-08T19:31:11.444341+0000","last_fullsized":"2023-11-08T19:31:11.444341+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:50:12.101878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":89,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.009586+0000","last_change":"2023-11-08T19:26:05.609570+0000","last_active":"2023-11-08T19:31:13.009586+0000","last_peered":"2023-11-08T19:31:13.009586+0000","last_clean":"2023-11-08T19:31:13.009586+0000","last_became_active":"2023-11-08T19:26:04.608379+0000","last_became_peered":"2023-11-08T19:26:04.608379+0000","last_unstale":"2023-11-08T19:31:13.009586+0000","last_undegraded":"2023-11-08T19:31:13.009586+0000","last_fullsized":"2023-11-08T19:31:13.009586+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:40:08.986649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000529657,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":53,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.668577+0000","last_change":"2023-11-08T19:27:09.211991+0000","last_active":"2023-11-08T19:31:11.668577+0000","last_peered":"2023-11-08T19:31:11.668577+0000","last_clean":"2023-11-08T19:31:11.668577+0000","last_became_active":"2023-11-08T19:27:07.195231+0000","last_became_peered":"2023-11-08T19:27:07.195231+0000","last_unstale":"2023-11-08T19:31:11.668577+0000","last_undegraded":"2023-11-08T19:31:11.668577+0000","last_fullsized":"2023-11-08T19:31:11.668577+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:25:14.487524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000441221,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":46,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.444418+0000","last_change":"2023-11-08T19:27:08.195364+0000","last_active":"2023-11-08T19:31:11.444418+0000","last_peered":"2023-11-08T19:31:11.444418+0000","last_clean":"2023-11-08T19:31:11.444418+0000","last_became_active":"2023-11-08T19:27:08.195074+0000","last_became_peered":"2023-11-08T19:27:08.195074+0000","last_unstale":"2023-11-08T19:31:11.444418+0000","last_undegraded":"2023-11-08T19:31:11.444418+0000","last_fullsized":"2023-11-08T19:31:11.444418+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:02:57.552873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":76,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.002147+0000","last_change":"2023-11-08T19:26:06.394300+0000","last_active":"2023-11-08T19:31:11.002147+0000","last_peered":"2023-11-08T19:31:11.002147+0000","last_clean":"2023-11-08T19:31:11.002147+0000","last_became_active":"2023-11-08T19:26:04.601685+0000","last_became_peered":"2023-11-08T19:26:04.601685+0000","last_unstale":"2023-11-08T19:31:11.002147+0000","last_undegraded":"2023-11-08T19:31:11.002147+0000","last_fullsized":"2023-11-08T19:31:11.002147+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:51:06.626474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061044099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":53,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.444658+0000","last_change":"2023-11-08T19:27:09.178466+0000","last_active":"2023-11-08T19:31:11.444658+0000","last_peered":"2023-11-08T19:31:11.444658+0000","last_clean":"2023-11-08T19:31:11.444658+0000","last_became_active":"2023-11-08T19:27:07.189386+0000","last_became_peered":"2023-11-08T19:27:07.189386+0000","last_unstale":"2023-11-08T19:31:11.444658+0000","last_undegraded":"2023-11-08T19:31:11.444658+0000","last_fullsized":"2023-11-08T19:31:11.444658+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:48:51.139820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049390100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":50,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.666965+0000","last_change":"2023-11-08T19:27:08.189452+0000","last_active":"2023-11-08T19:31:11.666965+0000","last_peered":"2023-11-08T19:31:11.666965+0000","last_clean":"2023-11-08T19:31:11.666965+0000","last_became_active":"2023-11-08T19:27:08.189088+0000","last_became_peered":"2023-11-08T19:27:08.189088+0000","last_unstale":"2023-11-08T19:31:11.666965+0000","last_undegraded":"2023-11-08T19:31:11.666965+0000","last_fullsized":"2023-11-08T19:31:11.666965+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:25:18.963220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":77,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.666780+0000","last_change":"2023-11-08T19:26:06.296634+0000","last_active":"2023-11-08T19:31:11.666780+0000","last_peered":"2023-11-08T19:31:11.666780+0000","last_clean":"2023-11-08T19:31:11.666780+0000","last_became_active":"2023-11-08T19:26:04.603088+0000","last_became_peered":"2023-11-08T19:26:04.603088+0000","last_unstale":"2023-11-08T19:31:11.666780+0000","last_undegraded":"2023-11-08T19:31:11.666780+0000","last_fullsized":"2023-11-08T19:31:11.666780+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:01:56.991346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038180899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":53,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.666847+0000","last_change":"2023-11-08T19:27:09.431220+0000","last_active":"2023-11-08T19:31:11.666847+0000","last_peered":"2023-11-08T19:31:11.666847+0000","last_clean":"2023-11-08T19:31:11.666847+0000","last_became_active":"2023-11-08T19:27:07.191589+0000","last_became_peered":"2023-11-08T19:27:07.191589+0000","last_unstale":"2023-11-08T19:31:11.666847+0000","last_undegraded":"2023-11-08T19:31:11.666847+0000","last_fullsized":"2023-11-08T19:31:11.666847+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:18:19.964487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000729527,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":46,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.002470+0000","last_change":"2023-11-08T19:27:08.193337+0000","last_active":"2023-11-08T19:31:11.002470+0000","last_peered":"2023-11-08T19:31:11.002470+0000","last_clean":"2023-11-08T19:31:11.002470+0000","last_became_active":"2023-11-08T19:27:08.192997+0000","last_became_peered":"2023-11-08T19:27:08.192997+0000","last_unstale":"2023-11-08T19:31:11.002470+0000","last_undegraded":"2023-11-08T19:31:11.002470+0000","last_fullsized":"2023-11-08T19:31:11.002470+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:45:22.084015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":82,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.668119+0000","last_change":"2023-11-08T19:26:06.424511+0000","last_active":"2023-11-08T19:31:11.668119+0000","last_peered":"2023-11-08T19:31:11.668119+0000","last_clean":"2023-11-08T19:31:11.668119+0000","last_became_active":"2023-11-08T19:26:04.598973+0000","last_became_peered":"2023-11-08T19:26:04.598973+0000","last_unstale":"2023-11-08T19:31:11.668119+0000","last_undegraded":"2023-11-08T19:31:11.668119+0000","last_fullsized":"2023-11-08T19:31:11.668119+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:41:53.497456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031755900000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":57,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.667045+0000","last_change":"2023-11-08T19:27:09.497024+0000","last_active":"2023-11-08T19:31:11.667045+0000","last_peered":"2023-11-08T19:31:11.667045+0000","last_clean":"2023-11-08T19:31:11.667045+0000","last_became_active":"2023-11-08T19:27:07.190132+0000","last_became_peered":"2023-11-08T19:27:07.190132+0000","last_unstale":"2023-11-08T19:31:11.667045+0000","last_undegraded":"2023-11-08T19:31:11.667045+0000","last_fullsized":"2023-11-08T19:31:11.667045+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:36:34.881217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054126000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":46,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.002863+0000","last_change":"2023-11-08T19:27:08.194196+0000","last_active":"2023-11-08T19:31:11.002863+0000","last_peered":"2023-11-08T19:31:11.002863+0000","last_clean":"2023-11-08T19:31:11.002863+0000","last_became_active":"2023-11-08T19:27:08.193912+0000","last_became_peered":"2023-11-08T19:27:08.193912+0000","last_unstale":"2023-11-08T19:31:11.002863+0000","last_undegraded":"2023-11-08T19:31:11.002863+0000","last_fullsized":"2023-11-08T19:31:11.002863+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:25:18.963220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":77,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.667020+0000","last_change":"2023-11-08T19:26:06.296937+0000","last_active":"2023-11-08T19:31:11.667020+0000","last_peered":"2023-11-08T19:31:11.667020+0000","last_clean":"2023-11-08T19:31:11.667020+0000","last_became_active":"2023-11-08T19:26:04.599999+0000","last_became_peered":"2023-11-08T19:26:04.599999+0000","last_unstale":"2023-11-08T19:31:11.667020+0000","last_undegraded":"2023-11-08T19:31:11.667020+0000","last_fullsized":"2023-11-08T19:31:11.667020+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:50:43.827570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025605700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":57,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.667208+0000","last_change":"2023-11-08T19:27:09.497040+0000","last_active":"2023-11-08T19:31:11.667208+0000","last_peered":"2023-11-08T19:31:11.667208+0000","last_clean":"2023-11-08T19:31:11.667208+0000","last_became_active":"2023-11-08T19:27:07.189135+0000","last_became_peered":"2023-11-08T19:27:07.189135+0000","last_unstale":"2023-11-08T19:31:11.667208+0000","last_undegraded":"2023-11-08T19:31:11.667208+0000","last_fullsized":"2023-11-08T19:31:11.667208+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:12:29.400972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051794299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":53,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.011810+0000","last_change":"2023-11-08T19:27:08.187053+0000","last_active":"2023-11-08T19:31:13.011810+0000","last_peered":"2023-11-08T19:31:13.011810+0000","last_clean":"2023-11-08T19:31:13.011810+0000","last_became_active":"2023-11-08T19:27:08.186643+0000","last_became_peered":"2023-11-08T19:27:08.186643+0000","last_unstale":"2023-11-08T19:31:13.011810+0000","last_undegraded":"2023-11-08T19:31:13.011810+0000","last_fullsized":"2023-11-08T19:31:13.011810+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:35:30.073885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":89,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.010538+0000","last_change":"2023-11-08T19:26:05.609824+0000","last_active":"2023-11-08T19:31:13.010538+0000","last_peered":"2023-11-08T19:31:13.010538+0000","last_clean":"2023-11-08T19:31:13.010538+0000","last_became_active":"2023-11-08T19:26:04.603320+0000","last_became_peered":"2023-11-08T19:26:04.603320+0000","last_unstale":"2023-11-08T19:31:13.010538+0000","last_undegraded":"2023-11-08T19:31:13.010538+0000","last_fullsized":"2023-11-08T19:31:13.010538+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:05:45.976615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063225,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":57,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.668286+0000","last_change":"2023-11-08T19:27:09.496655+0000","last_active":"2023-11-08T19:31:11.668286+0000","last_peered":"2023-11-08T19:31:11.668286+0000","last_clean":"2023-11-08T19:31:11.668286+0000","last_became_active":"2023-11-08T19:27:07.189895+0000","last_became_peered":"2023-11-08T19:27:07.189895+0000","last_unstale":"2023-11-08T19:31:11.668286+0000","last_undegraded":"2023-11-08T19:31:11.668286+0000","last_fullsized":"2023-11-08T19:31:11.668286+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:41:08.822366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045654199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":78,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.667623+0000","last_change":"2023-11-08T19:26:06.541360+0000","last_active":"2023-11-08T19:31:11.667623+0000","last_peered":"2023-11-08T19:31:11.667623+0000","last_clean":"2023-11-08T19:31:11.667623+0000","last_became_active":"2023-11-08T19:26:04.603725+0000","last_became_peered":"2023-11-08T19:26:04.603725+0000","last_unstale":"2023-11-08T19:31:11.667623+0000","last_undegraded":"2023-11-08T19:31:11.667623+0000","last_fullsized":"2023-11-08T19:31:11.667623+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:28:48.576742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053365400000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":53,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.669050+0000","last_change":"2023-11-08T19:27:09.283599+0000","last_active":"2023-11-08T19:31:11.669050+0000","last_peered":"2023-11-08T19:31:11.669050+0000","last_clean":"2023-11-08T19:31:11.669050+0000","last_became_active":"2023-11-08T19:27:07.189595+0000","last_became_peered":"2023-11-08T19:27:07.189595+0000","last_unstale":"2023-11-08T19:31:11.669050+0000","last_undegraded":"2023-11-08T19:31:11.669050+0000","last_fullsized":"2023-11-08T19:31:11.669050+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:48:20.082682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000323479,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":47,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.667698+0000","last_change":"2023-11-08T19:27:08.185395+0000","last_active":"2023-11-08T19:31:11.667698+0000","last_peered":"2023-11-08T19:31:11.667698+0000","last_clean":"2023-11-08T19:31:11.667698+0000","last_became_active":"2023-11-08T19:27:08.184765+0000","last_became_peered":"2023-11-08T19:27:08.184765+0000","last_unstale":"2023-11-08T19:31:11.667698+0000","last_undegraded":"2023-11-08T19:31:11.667698+0000","last_fullsized":"2023-11-08T19:31:11.667698+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:08:51.864418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":82,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.667206+0000","last_change":"2023-11-08T19:26:06.424236+0000","last_active":"2023-11-08T19:31:11.667206+0000","last_peered":"2023-11-08T19:31:11.667206+0000","last_clean":"2023-11-08T19:31:11.667206+0000","last_became_active":"2023-11-08T19:26:04.600115+0000","last_became_peered":"2023-11-08T19:26:04.600115+0000","last_unstale":"2023-11-08T19:31:11.667206+0000","last_undegraded":"2023-11-08T19:31:11.667206+0000","last_fullsized":"2023-11-08T19:31:11.667206+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:18:01.876605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065803900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":53,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.002693+0000","last_change":"2023-11-08T19:27:09.331899+0000","last_active":"2023-11-08T19:31:11.002693+0000","last_peered":"2023-11-08T19:31:11.002693+0000","last_clean":"2023-11-08T19:31:11.002693+0000","last_became_active":"2023-11-08T19:27:07.191460+0000","last_became_peered":"2023-11-08T19:27:07.191460+0000","last_unstale":"2023-11-08T19:31:11.002693+0000","last_undegraded":"2023-11-08T19:31:11.002693+0000","last_fullsized":"2023-11-08T19:31:11.002693+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:51:29.247104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044907500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":53,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.009947+0000","last_change":"2023-11-08T19:27:08.191802+0000","last_active":"2023-11-08T19:31:13.009947+0000","last_peered":"2023-11-08T19:31:13.009947+0000","last_clean":"2023-11-08T19:31:13.009947+0000","last_became_active":"2023-11-08T19:27:08.191520+0000","last_became_peered":"2023-11-08T19:27:08.191520+0000","last_unstale":"2023-11-08T19:31:13.009947+0000","last_undegraded":"2023-11-08T19:31:13.009947+0000","last_fullsized":"2023-11-08T19:31:13.009947+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:49:16.458634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":89,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.008132+0000","last_change":"2023-11-08T19:26:05.608892+0000","last_active":"2023-11-08T19:31:13.008132+0000","last_peered":"2023-11-08T19:31:13.008132+0000","last_clean":"2023-11-08T19:31:13.008132+0000","last_became_active":"2023-11-08T19:26:04.604519+0000","last_became_peered":"2023-11-08T19:26:04.604519+0000","last_unstale":"2023-11-08T19:31:13.008132+0000","last_undegraded":"2023-11-08T19:31:13.008132+0000","last_fullsized":"2023-11-08T19:31:13.008132+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:11:26.508634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051248099999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":60,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.010356+0000","last_change":"2023-11-08T19:27:09.181431+0000","last_active":"2023-11-08T19:31:13.010356+0000","last_peered":"2023-11-08T19:31:13.010356+0000","last_clean":"2023-11-08T19:31:13.010356+0000","last_became_active":"2023-11-08T19:27:07.190800+0000","last_became_peered":"2023-11-08T19:27:07.190800+0000","last_unstale":"2023-11-08T19:31:13.010356+0000","last_undegraded":"2023-11-08T19:31:13.010356+0000","last_fullsized":"2023-11-08T19:31:13.010356+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:46:30.452678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656099999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":46,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.667947+0000","last_change":"2023-11-08T19:27:08.187983+0000","last_active":"2023-11-08T19:31:11.667947+0000","last_peered":"2023-11-08T19:31:11.667947+0000","last_clean":"2023-11-08T19:31:11.667947+0000","last_became_active":"2023-11-08T19:27:08.187322+0000","last_became_peered":"2023-11-08T19:27:08.187322+0000","last_unstale":"2023-11-08T19:31:11.667947+0000","last_undegraded":"2023-11-08T19:31:11.667947+0000","last_fullsized":"2023-11-08T19:31:11.667947+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:33:07.647519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":76,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.002194+0000","last_change":"2023-11-08T19:26:06.394075+0000","last_active":"2023-11-08T19:31:11.002194+0000","last_peered":"2023-11-08T19:31:11.002194+0000","last_clean":"2023-11-08T19:31:11.002194+0000","last_became_active":"2023-11-08T19:26:04.606967+0000","last_became_peered":"2023-11-08T19:26:04.606967+0000","last_unstale":"2023-11-08T19:31:11.002194+0000","last_undegraded":"2023-11-08T19:31:11.002194+0000","last_fullsized":"2023-11-08T19:31:11.002194+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:19:56.286835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040971999999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":60,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.008770+0000","last_change":"2023-11-08T19:27:09.180722+0000","last_active":"2023-11-08T19:31:13.008770+0000","last_peered":"2023-11-08T19:31:13.008770+0000","last_clean":"2023-11-08T19:31:13.008770+0000","last_became_active":"2023-11-08T19:27:07.186977+0000","last_became_peered":"2023-11-08T19:27:07.186977+0000","last_unstale":"2023-11-08T19:31:13.008770+0000","last_undegraded":"2023-11-08T19:31:13.008770+0000","last_fullsized":"2023-11-08T19:31:13.008770+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:17:59.966463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057675100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":46,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.002184+0000","last_change":"2023-11-08T19:27:08.193963+0000","last_active":"2023-11-08T19:31:11.002184+0000","last_peered":"2023-11-08T19:31:11.002184+0000","last_clean":"2023-11-08T19:31:11.002184+0000","last_became_active":"2023-11-08T19:27:08.193098+0000","last_became_peered":"2023-11-08T19:27:08.193098+0000","last_unstale":"2023-11-08T19:31:11.002184+0000","last_undegraded":"2023-11-08T19:31:11.002184+0000","last_fullsized":"2023-11-08T19:31:11.002184+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:41:09.827370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":76,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.668693+0000","last_change":"2023-11-08T19:26:06.265686+0000","last_active":"2023-11-08T19:31:11.668693+0000","last_peered":"2023-11-08T19:31:11.668693+0000","last_clean":"2023-11-08T19:31:11.668693+0000","last_became_active":"2023-11-08T19:26:04.602111+0000","last_became_peered":"2023-11-08T19:26:04.602111+0000","last_unstale":"2023-11-08T19:31:11.668693+0000","last_undegraded":"2023-11-08T19:31:11.668693+0000","last_fullsized":"2023-11-08T19:31:11.668693+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:40:11.164951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056197899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":60,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.008458+0000","last_change":"2023-11-08T19:27:09.181333+0000","last_active":"2023-11-08T19:31:13.008458+0000","last_peered":"2023-11-08T19:31:13.008458+0000","last_clean":"2023-11-08T19:31:13.008458+0000","last_became_active":"2023-11-08T19:27:07.188657+0000","last_became_peered":"2023-11-08T19:27:07.188657+0000","last_unstale":"2023-11-08T19:31:13.008458+0000","last_undegraded":"2023-11-08T19:31:13.008458+0000","last_fullsized":"2023-11-08T19:31:13.008458+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:25:56.788615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001227393,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":53,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.009645+0000","last_change":"2023-11-08T19:27:08.188533+0000","last_active":"2023-11-08T19:31:13.009645+0000","last_peered":"2023-11-08T19:31:13.009645+0000","last_clean":"2023-11-08T19:31:13.009645+0000","last_became_active":"2023-11-08T19:27:08.188240+0000","last_became_peered":"2023-11-08T19:27:08.188240+0000","last_unstale":"2023-11-08T19:31:13.009645+0000","last_undegraded":"2023-11-08T19:31:13.009645+0000","last_fullsized":"2023-11-08T19:31:13.009645+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:55:37.653521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":53,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.010070+0000","last_change":"2023-11-08T19:27:08.188458+0000","last_active":"2023-11-08T19:31:13.010070+0000","last_peered":"2023-11-08T19:31:13.010070+0000","last_clean":"2023-11-08T19:31:13.010070+0000","last_became_active":"2023-11-08T19:27:08.187880+0000","last_became_peered":"2023-11-08T19:27:08.187880+0000","last_unstale":"2023-11-08T19:31:13.010070+0000","last_undegraded":"2023-11-08T19:31:13.010070+0000","last_fullsized":"2023-11-08T19:31:13.010070+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:25:28.618214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":77,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.444854+0000","last_change":"2023-11-08T19:26:05.606898+0000","last_active":"2023-11-08T19:31:11.444854+0000","last_peered":"2023-11-08T19:31:11.444854+0000","last_clean":"2023-11-08T19:31:11.444854+0000","last_became_active":"2023-11-08T19:26:04.603955+0000","last_became_peered":"2023-11-08T19:26:04.603955+0000","last_unstale":"2023-11-08T19:31:11.444854+0000","last_undegraded":"2023-11-08T19:31:11.444854+0000","last_fullsized":"2023-11-08T19:31:11.444854+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:19:11.589559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036473499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":53,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.444503+0000","last_change":"2023-11-08T19:27:09.178770+0000","last_active":"2023-11-08T19:31:11.444503+0000","last_peered":"2023-11-08T19:31:11.444503+0000","last_clean":"2023-11-08T19:31:11.444503+0000","last_became_active":"2023-11-08T19:27:07.194992+0000","last_became_peered":"2023-11-08T19:27:07.194992+0000","last_unstale":"2023-11-08T19:31:11.444503+0000","last_undegraded":"2023-11-08T19:31:11.444503+0000","last_fullsized":"2023-11-08T19:31:11.444503+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:35:34.026910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000770285,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":46,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.667108+0000","last_change":"2023-11-08T19:27:08.193953+0000","last_active":"2023-11-08T19:31:11.667108+0000","last_peered":"2023-11-08T19:31:11.667108+0000","last_clean":"2023-11-08T19:31:11.667108+0000","last_became_active":"2023-11-08T19:27:08.193653+0000","last_became_peered":"2023-11-08T19:27:08.193653+0000","last_unstale":"2023-11-08T19:31:11.667108+0000","last_undegraded":"2023-11-08T19:31:11.667108+0000","last_fullsized":"2023-11-08T19:31:11.667108+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:35:30.073885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":82,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.667882+0000","last_change":"2023-11-08T19:26:06.424464+0000","last_active":"2023-11-08T19:31:11.667882+0000","last_peered":"2023-11-08T19:31:11.667882+0000","last_clean":"2023-11-08T19:31:11.667882+0000","last_became_active":"2023-11-08T19:26:04.603144+0000","last_became_peered":"2023-11-08T19:26:04.603144+0000","last_unstale":"2023-11-08T19:31:11.667882+0000","last_undegraded":"2023-11-08T19:31:11.667882+0000","last_fullsized":"2023-11-08T19:31:11.667882+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:42:55.269365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054090799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":60,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.009866+0000","last_change":"2023-11-08T19:27:09.180479+0000","last_active":"2023-11-08T19:31:13.009866+0000","last_peered":"2023-11-08T19:31:13.009866+0000","last_clean":"2023-11-08T19:31:13.009866+0000","last_became_active":"2023-11-08T19:27:07.190438+0000","last_became_peered":"2023-11-08T19:27:07.190438+0000","last_unstale":"2023-11-08T19:31:13.009866+0000","last_undegraded":"2023-11-08T19:31:13.009866+0000","last_fullsized":"2023-11-08T19:31:13.009866+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:25:14.487524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050893300000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":46,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.667409+0000","last_change":"2023-11-08T19:27:08.190530+0000","last_active":"2023-11-08T19:31:11.667409+0000","last_peered":"2023-11-08T19:31:11.667409+0000","last_clean":"2023-11-08T19:31:11.667409+0000","last_became_active":"2023-11-08T19:27:08.190203+0000","last_became_peered":"2023-11-08T19:27:08.190203+0000","last_unstale":"2023-11-08T19:31:11.667409+0000","last_undegraded":"2023-11-08T19:31:11.667409+0000","last_fullsized":"2023-11-08T19:31:11.667409+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:26:32.777602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":78,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.668651+0000","last_change":"2023-11-08T19:26:06.541623+0000","last_active":"2023-11-08T19:31:11.668651+0000","last_peered":"2023-11-08T19:31:11.668651+0000","last_clean":"2023-11-08T19:31:11.668651+0000","last_became_active":"2023-11-08T19:26:04.601961+0000","last_became_peered":"2023-11-08T19:26:04.601961+0000","last_unstale":"2023-11-08T19:31:11.668651+0000","last_undegraded":"2023-11-08T19:31:11.668651+0000","last_fullsized":"2023-11-08T19:31:11.668651+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:11:26.508634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000466386,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":54,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.666591+0000","last_change":"2023-11-08T19:27:09.431231+0000","last_active":"2023-11-08T19:31:11.666591+0000","last_peered":"2023-11-08T19:31:11.666591+0000","last_clean":"2023-11-08T19:31:11.666591+0000","last_became_active":"2023-11-08T19:27:07.191583+0000","last_became_peered":"2023-11-08T19:27:07.191583+0000","last_unstale":"2023-11-08T19:31:11.666591+0000","last_undegraded":"2023-11-08T19:31:11.666591+0000","last_fullsized":"2023-11-08T19:31:11.666591+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:58:48.357266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093695200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":46,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.666559+0000","last_change":"2023-11-08T19:27:08.187900+0000","last_active":"2023-11-08T19:31:11.666559+0000","last_peered":"2023-11-08T19:31:11.666559+0000","last_clean":"2023-11-08T19:31:11.666559+0000","last_became_active":"2023-11-08T19:27:08.187626+0000","last_became_peered":"2023-11-08T19:27:08.187626+0000","last_unstale":"2023-11-08T19:31:11.666559+0000","last_undegraded":"2023-11-08T19:31:11.666559+0000","last_fullsized":"2023-11-08T19:31:11.666559+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:00:55.878397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":89,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.009597+0000","last_change":"2023-11-08T19:26:05.608980+0000","last_active":"2023-11-08T19:31:13.009597+0000","last_peered":"2023-11-08T19:31:13.009597+0000","last_clean":"2023-11-08T19:31:13.009597+0000","last_became_active":"2023-11-08T19:26:04.606386+0000","last_became_peered":"2023-11-08T19:26:04.606386+0000","last_unstale":"2023-11-08T19:31:13.009597+0000","last_undegraded":"2023-11-08T19:31:13.009597+0000","last_fullsized":"2023-11-08T19:31:13.009597+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:41:50.771457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037950599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":53,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.444816+0000","last_change":"2023-11-08T19:27:09.179037+0000","last_active":"2023-11-08T19:31:11.444816+0000","last_peered":"2023-11-08T19:31:11.444816+0000","last_clean":"2023-11-08T19:31:11.444816+0000","last_became_active":"2023-11-08T19:27:07.195033+0000","last_became_peered":"2023-11-08T19:27:07.195033+0000","last_unstale":"2023-11-08T19:31:11.444816+0000","last_undegraded":"2023-11-08T19:31:11.444816+0000","last_fullsized":"2023-11-08T19:31:11.444816+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:19:08.253483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041824800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":46,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.002110+0000","last_change":"2023-11-08T19:27:08.193761+0000","last_active":"2023-11-08T19:31:11.002110+0000","last_peered":"2023-11-08T19:31:11.002110+0000","last_clean":"2023-11-08T19:31:11.002110+0000","last_became_active":"2023-11-08T19:27:08.193017+0000","last_became_peered":"2023-11-08T19:27:08.193017+0000","last_unstale":"2023-11-08T19:31:11.002110+0000","last_undegraded":"2023-11-08T19:31:11.002110+0000","last_fullsized":"2023-11-08T19:31:11.002110+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:18:00.971466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":82,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.667284+0000","last_change":"2023-11-08T19:26:06.424073+0000","last_active":"2023-11-08T19:31:11.667284+0000","last_peered":"2023-11-08T19:31:11.667284+0000","last_clean":"2023-11-08T19:31:11.667284+0000","last_became_active":"2023-11-08T19:26:04.599046+0000","last_became_peered":"2023-11-08T19:26:04.599046+0000","last_unstale":"2023-11-08T19:31:11.667284+0000","last_undegraded":"2023-11-08T19:31:11.667284+0000","last_fullsized":"2023-11-08T19:31:11.667284+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:04:43.562566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035092199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":61,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.009142+0000","last_change":"2023-11-08T19:27:09.181716+0000","last_active":"2023-11-08T19:31:13.009142+0000","last_peered":"2023-11-08T19:31:13.009142+0000","last_clean":"2023-11-08T19:31:13.009142+0000","last_became_active":"2023-11-08T19:27:07.187077+0000","last_became_peered":"2023-11-08T19:27:07.187077+0000","last_unstale":"2023-11-08T19:31:13.009142+0000","last_undegraded":"2023-11-08T19:31:13.009142+0000","last_fullsized":"2023-11-08T19:31:13.009142+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:51:29.247104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056673599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":46,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.667869+0000","last_change":"2023-11-08T19:27:08.188143+0000","last_active":"2023-11-08T19:31:11.667869+0000","last_peered":"2023-11-08T19:31:11.667869+0000","last_clean":"2023-11-08T19:31:11.667869+0000","last_became_active":"2023-11-08T19:27:08.187714+0000","last_became_peered":"2023-11-08T19:27:08.187714+0000","last_unstale":"2023-11-08T19:31:11.667869+0000","last_undegraded":"2023-11-08T19:31:11.667869+0000","last_fullsized":"2023-11-08T19:31:11.667869+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:02:10.863403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":76,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.002535+0000","last_change":"2023-11-08T19:26:06.394320+0000","last_active":"2023-11-08T19:31:11.002535+0000","last_peered":"2023-11-08T19:31:11.002535+0000","last_clean":"2023-11-08T19:31:11.002535+0000","last_became_active":"2023-11-08T19:26:04.603212+0000","last_became_peered":"2023-11-08T19:26:04.603212+0000","last_unstale":"2023-11-08T19:31:11.002535+0000","last_undegraded":"2023-11-08T19:31:11.002535+0000","last_fullsized":"2023-11-08T19:31:11.002535+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:25:29.947079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054695399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":60,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.009834+0000","last_change":"2023-11-08T19:27:09.181613+0000","last_active":"2023-11-08T19:31:13.009834+0000","last_peered":"2023-11-08T19:31:13.009834+0000","last_clean":"2023-11-08T19:31:13.009834+0000","last_became_active":"2023-11-08T19:27:07.190897+0000","last_became_peered":"2023-11-08T19:27:07.190897+0000","last_unstale":"2023-11-08T19:31:13.009834+0000","last_undegraded":"2023-11-08T19:31:13.009834+0000","last_fullsized":"2023-11-08T19:31:13.009834+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:27:57.388125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011172459999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":46,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.668020+0000","last_change":"2023-11-08T19:27:08.188668+0000","last_active":"2023-11-08T19:31:11.668020+0000","last_peered":"2023-11-08T19:31:11.668020+0000","last_clean":"2023-11-08T19:31:11.668020+0000","last_became_active":"2023-11-08T19:27:08.188441+0000","last_became_peered":"2023-11-08T19:27:08.188441+0000","last_unstale":"2023-11-08T19:31:11.668020+0000","last_undegraded":"2023-11-08T19:31:11.668020+0000","last_fullsized":"2023-11-08T19:31:11.668020+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:45:44.379160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":82,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.667134+0000","last_change":"2023-11-08T19:26:06.423754+0000","last_active":"2023-11-08T19:31:11.667134+0000","last_peered":"2023-11-08T19:31:11.667134+0000","last_clean":"2023-11-08T19:31:11.667134+0000","last_became_active":"2023-11-08T19:26:04.599322+0000","last_became_peered":"2023-11-08T19:26:04.599322+0000","last_unstale":"2023-11-08T19:31:11.667134+0000","last_undegraded":"2023-11-08T19:31:11.667134+0000","last_fullsized":"2023-11-08T19:31:11.667134+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:13:08.689128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042670000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":60,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.010577+0000","last_change":"2023-11-08T19:27:09.181656+0000","last_active":"2023-11-08T19:31:13.010577+0000","last_peered":"2023-11-08T19:31:13.010577+0000","last_clean":"2023-11-08T19:31:13.010577+0000","last_became_active":"2023-11-08T19:27:07.187337+0000","last_became_peered":"2023-11-08T19:27:07.187337+0000","last_unstale":"2023-11-08T19:31:13.010577+0000","last_undegraded":"2023-11-08T19:31:13.010577+0000","last_fullsized":"2023-11-08T19:31:13.010577+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:14:17.605057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000346967,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":46,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.002332+0000","last_change":"2023-11-08T19:27:08.194065+0000","last_active":"2023-11-08T19:31:11.002332+0000","last_peered":"2023-11-08T19:31:11.002332+0000","last_clean":"2023-11-08T19:31:11.002332+0000","last_became_active":"2023-11-08T19:27:08.193532+0000","last_became_peered":"2023-11-08T19:27:08.193532+0000","last_unstale":"2023-11-08T19:31:11.002332+0000","last_undegraded":"2023-11-08T19:31:11.002332+0000","last_fullsized":"2023-11-08T19:31:11.002332+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:27:58.393129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":76,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.002769+0000","last_change":"2023-11-08T19:26:06.394488+0000","last_active":"2023-11-08T19:31:11.002769+0000","last_peered":"2023-11-08T19:31:11.002769+0000","last_clean":"2023-11-08T19:31:11.002769+0000","last_became_active":"2023-11-08T19:26:04.606805+0000","last_became_peered":"2023-11-08T19:26:04.606805+0000","last_unstale":"2023-11-08T19:31:11.002769+0000","last_undegraded":"2023-11-08T19:31:11.002769+0000","last_fullsized":"2023-11-08T19:31:11.002769+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:25:20.857133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045317000000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":53,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.668089+0000","last_change":"2023-11-08T19:27:09.211647+0000","last_active":"2023-11-08T19:31:11.668089+0000","last_peered":"2023-11-08T19:31:11.668089+0000","last_clean":"2023-11-08T19:31:11.668089+0000","last_became_active":"2023-11-08T19:27:07.190309+0000","last_became_peered":"2023-11-08T19:27:07.190309+0000","last_unstale":"2023-11-08T19:31:11.668089+0000","last_undegraded":"2023-11-08T19:31:11.668089+0000","last_fullsized":"2023-11-08T19:31:11.668089+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:56:04.292154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047615199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":46,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.002763+0000","last_change":"2023-11-08T19:27:08.194884+0000","last_active":"2023-11-08T19:31:11.002763+0000","last_peered":"2023-11-08T19:31:11.002763+0000","last_clean":"2023-11-08T19:31:11.002763+0000","last_became_active":"2023-11-08T19:27:08.193241+0000","last_became_peered":"2023-11-08T19:27:08.193241+0000","last_unstale":"2023-11-08T19:31:11.002763+0000","last_undegraded":"2023-11-08T19:31:11.002763+0000","last_fullsized":"2023-11-08T19:31:11.002763+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:46:31.457682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":76,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.668753+0000","last_change":"2023-11-08T19:26:06.265907+0000","last_active":"2023-11-08T19:31:11.668753+0000","last_peered":"2023-11-08T19:31:11.668753+0000","last_clean":"2023-11-08T19:31:11.668753+0000","last_became_active":"2023-11-08T19:26:04.604772+0000","last_became_peered":"2023-11-08T19:26:04.604772+0000","last_unstale":"2023-11-08T19:31:11.668753+0000","last_undegraded":"2023-11-08T19:31:11.668753+0000","last_fullsized":"2023-11-08T19:31:11.668753+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:44:16.139794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066875300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":53,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.667612+0000","last_change":"2023-11-08T19:27:09.282699+0000","last_active":"2023-11-08T19:31:11.667612+0000","last_peered":"2023-11-08T19:31:11.667612+0000","last_clean":"2023-11-08T19:31:11.667612+0000","last_became_active":"2023-11-08T19:27:07.191212+0000","last_became_peered":"2023-11-08T19:27:07.191212+0000","last_unstale":"2023-11-08T19:31:11.667612+0000","last_undegraded":"2023-11-08T19:31:11.667612+0000","last_fullsized":"2023-11-08T19:31:11.667612+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:35:14.715110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046640800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":47,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.668470+0000","last_change":"2023-11-08T19:27:08.187644+0000","last_active":"2023-11-08T19:31:11.668470+0000","last_peered":"2023-11-08T19:31:11.668470+0000","last_clean":"2023-11-08T19:31:11.668470+0000","last_became_active":"2023-11-08T19:27:08.187376+0000","last_became_peered":"2023-11-08T19:27:08.187376+0000","last_unstale":"2023-11-08T19:31:11.668470+0000","last_undegraded":"2023-11-08T19:31:11.668470+0000","last_fullsized":"2023-11-08T19:31:11.668470+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:22:40.300185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":77,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.444647+0000","last_change":"2023-11-08T19:26:05.606946+0000","last_active":"2023-11-08T19:31:11.444647+0000","last_peered":"2023-11-08T19:31:11.444647+0000","last_clean":"2023-11-08T19:31:11.444647+0000","last_became_active":"2023-11-08T19:26:04.598775+0000","last_became_peered":"2023-11-08T19:26:04.598775+0000","last_unstale":"2023-11-08T19:31:11.444647+0000","last_undegraded":"2023-11-08T19:31:11.444647+0000","last_fullsized":"2023-11-08T19:31:11.444647+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:20:56.898927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049870399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":53,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.002400+0000","last_change":"2023-11-08T19:27:09.331646+0000","last_active":"2023-11-08T19:31:11.002400+0000","last_peered":"2023-11-08T19:31:11.002400+0000","last_clean":"2023-11-08T19:31:11.002400+0000","last_became_active":"2023-11-08T19:27:07.190619+0000","last_became_peered":"2023-11-08T19:27:07.190619+0000","last_unstale":"2023-11-08T19:31:11.002400+0000","last_undegraded":"2023-11-08T19:31:11.002400+0000","last_fullsized":"2023-11-08T19:31:11.002400+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:42:47.371458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065918400000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":53,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.011570+0000","last_change":"2023-11-08T19:27:08.183798+0000","last_active":"2023-11-08T19:31:13.011570+0000","last_peered":"2023-11-08T19:31:13.011570+0000","last_clean":"2023-11-08T19:31:13.011570+0000","last_became_active":"2023-11-08T19:27:08.183537+0000","last_became_peered":"2023-11-08T19:27:08.183537+0000","last_unstale":"2023-11-08T19:31:13.011570+0000","last_undegraded":"2023-11-08T19:31:13.011570+0000","last_fullsized":"2023-11-08T19:31:13.011570+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:50:39.026265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":89,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.011263+0000","last_change":"2023-11-08T19:26:05.610012+0000","last_active":"2023-11-08T19:31:13.011263+0000","last_peered":"2023-11-08T19:31:13.011263+0000","last_clean":"2023-11-08T19:31:13.011263+0000","last_became_active":"2023-11-08T19:26:04.603209+0000","last_became_peered":"2023-11-08T19:26:04.603209+0000","last_unstale":"2023-11-08T19:31:13.011263+0000","last_undegraded":"2023-11-08T19:31:13.011263+0000","last_fullsized":"2023-11-08T19:31:13.011263+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:35:02.306997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053335100000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":53,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.444512+0000","last_change":"2023-11-08T19:27:09.178405+0000","last_active":"2023-11-08T19:31:11.444512+0000","last_peered":"2023-11-08T19:31:11.444512+0000","last_clean":"2023-11-08T19:31:11.444512+0000","last_became_active":"2023-11-08T19:27:07.188169+0000","last_became_peered":"2023-11-08T19:27:07.188169+0000","last_unstale":"2023-11-08T19:31:11.444512+0000","last_undegraded":"2023-11-08T19:31:11.444512+0000","last_fullsized":"2023-11-08T19:31:11.444512+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:39:25.855467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056209700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":46,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.002018+0000","last_change":"2023-11-08T19:27:08.193508+0000","last_active":"2023-11-08T19:31:11.002018+0000","last_peered":"2023-11-08T19:31:11.002018+0000","last_clean":"2023-11-08T19:31:11.002018+0000","last_became_active":"2023-11-08T19:27:08.193037+0000","last_became_peered":"2023-11-08T19:27:08.193037+0000","last_unstale":"2023-11-08T19:31:11.002018+0000","last_undegraded":"2023-11-08T19:31:11.002018+0000","last_fullsized":"2023-11-08T19:31:11.002018+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:29:50.067887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":76,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.667822+0000","last_change":"2023-11-08T19:26:06.265318+0000","last_active":"2023-11-08T19:31:11.667822+0000","last_peered":"2023-11-08T19:31:11.667822+0000","last_clean":"2023-11-08T19:31:11.667822+0000","last_became_active":"2023-11-08T19:26:04.603409+0000","last_became_peered":"2023-11-08T19:26:04.603409+0000","last_unstale":"2023-11-08T19:31:11.667822+0000","last_undegraded":"2023-11-08T19:31:11.667822+0000","last_fullsized":"2023-11-08T19:31:11.667822+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:27:24.933111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065684300000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":60,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.010096+0000","last_change":"2023-11-08T19:27:09.182282+0000","last_active":"2023-11-08T19:31:13.010096+0000","last_peered":"2023-11-08T19:31:13.010096+0000","last_clean":"2023-11-08T19:31:13.010096+0000","last_became_active":"2023-11-08T19:27:07.186944+0000","last_became_peered":"2023-11-08T19:27:07.186944+0000","last_unstale":"2023-11-08T19:31:13.010096+0000","last_undegraded":"2023-11-08T19:31:13.010096+0000","last_fullsized":"2023-11-08T19:31:13.010096+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:45:21.079011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00028665599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":46,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.667282+0000","last_change":"2023-11-08T19:27:08.193990+0000","last_active":"2023-11-08T19:31:11.667282+0000","last_peered":"2023-11-08T19:31:11.667282+0000","last_clean":"2023-11-08T19:31:11.667282+0000","last_became_active":"2023-11-08T19:27:08.193624+0000","last_became_peered":"2023-11-08T19:27:08.193624+0000","last_unstale":"2023-11-08T19:31:11.667282+0000","last_undegraded":"2023-11-08T19:31:11.667282+0000","last_fullsized":"2023-11-08T19:31:11.667282+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:46:41.321120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":82,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.667129+0000","last_change":"2023-11-08T19:26:06.423863+0000","last_active":"2023-11-08T19:31:11.667129+0000","last_peered":"2023-11-08T19:31:11.667129+0000","last_clean":"2023-11-08T19:31:11.667129+0000","last_became_active":"2023-11-08T19:26:04.608639+0000","last_became_peered":"2023-11-08T19:26:04.608639+0000","last_unstale":"2023-11-08T19:31:11.667129+0000","last_undegraded":"2023-11-08T19:31:11.667129+0000","last_fullsized":"2023-11-08T19:31:11.667129+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:04:43.631102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044384999999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":53,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.668714+0000","last_change":"2023-11-08T19:27:09.283232+0000","last_active":"2023-11-08T19:31:11.668714+0000","last_peered":"2023-11-08T19:31:11.668714+0000","last_clean":"2023-11-08T19:31:11.668714+0000","last_became_active":"2023-11-08T19:27:07.190931+0000","last_became_peered":"2023-11-08T19:27:07.190931+0000","last_unstale":"2023-11-08T19:31:11.668714+0000","last_undegraded":"2023-11-08T19:31:11.668714+0000","last_fullsized":"2023-11-08T19:31:11.668714+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:43:28.225967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034146799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":50,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.667555+0000","last_change":"2023-11-08T19:27:08.187252+0000","last_active":"2023-11-08T19:31:11.667555+0000","last_peered":"2023-11-08T19:31:11.667555+0000","last_clean":"2023-11-08T19:31:11.667555+0000","last_became_active":"2023-11-08T19:27:08.186952+0000","last_became_peered":"2023-11-08T19:27:08.186952+0000","last_unstale":"2023-11-08T19:31:11.667555+0000","last_undegraded":"2023-11-08T19:31:11.667555+0000","last_fullsized":"2023-11-08T19:31:11.667555+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:12:52.022374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":76,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.002192+0000","last_change":"2023-11-08T19:26:06.394081+0000","last_active":"2023-11-08T19:31:11.002192+0000","last_peered":"2023-11-08T19:31:11.002192+0000","last_clean":"2023-11-08T19:31:11.002192+0000","last_became_active":"2023-11-08T19:26:04.602137+0000","last_became_peered":"2023-11-08T19:26:04.602137+0000","last_unstale":"2023-11-08T19:31:11.002192+0000","last_undegraded":"2023-11-08T19:31:11.002192+0000","last_fullsized":"2023-11-08T19:31:11.002192+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:05:46.951663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056998200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":53,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.444723+0000","last_change":"2023-11-08T19:27:09.178816+0000","last_active":"2023-11-08T19:31:11.444723+0000","last_peered":"2023-11-08T19:31:11.444723+0000","last_clean":"2023-11-08T19:31:11.444723+0000","last_became_active":"2023-11-08T19:27:07.192001+0000","last_became_peered":"2023-11-08T19:27:07.192001+0000","last_unstale":"2023-11-08T19:31:11.444723+0000","last_undegraded":"2023-11-08T19:31:11.444723+0000","last_fullsized":"2023-11-08T19:31:11.444723+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:04:12.481212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045359000000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":46,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.666798+0000","last_change":"2023-11-08T19:27:08.193836+0000","last_active":"2023-11-08T19:31:11.666798+0000","last_peered":"2023-11-08T19:31:11.666798+0000","last_clean":"2023-11-08T19:31:11.666798+0000","last_became_active":"2023-11-08T19:27:08.193458+0000","last_became_peered":"2023-11-08T19:27:08.193458+0000","last_unstale":"2023-11-08T19:31:11.666798+0000","last_undegraded":"2023-11-08T19:31:11.666798+0000","last_fullsized":"2023-11-08T19:31:11.666798+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:14:18.610061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":78,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.667284+0000","last_change":"2023-11-08T19:26:06.541338+0000","last_active":"2023-11-08T19:31:11.667284+0000","last_peered":"2023-11-08T19:31:11.667284+0000","last_clean":"2023-11-08T19:31:11.667284+0000","last_became_active":"2023-11-08T19:26:04.602183+0000","last_became_peered":"2023-11-08T19:26:04.602183+0000","last_unstale":"2023-11-08T19:31:11.667284+0000","last_undegraded":"2023-11-08T19:31:11.667284+0000","last_fullsized":"2023-11-08T19:31:11.667284+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:06:47.446686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052029099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":60,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.010485+0000","last_change":"2023-11-08T19:27:09.181622+0000","last_active":"2023-11-08T19:31:13.010485+0000","last_peered":"2023-11-08T19:31:13.010485+0000","last_clean":"2023-11-08T19:31:13.010485+0000","last_became_active":"2023-11-08T19:27:07.184507+0000","last_became_peered":"2023-11-08T19:27:07.184507+0000","last_unstale":"2023-11-08T19:31:13.010485+0000","last_undegraded":"2023-11-08T19:31:13.010485+0000","last_fullsized":"2023-11-08T19:31:13.010485+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:58:48.357266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051924700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":46,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.668291+0000","last_change":"2023-11-08T19:27:08.194263+0000","last_active":"2023-11-08T19:31:11.668291+0000","last_peered":"2023-11-08T19:31:11.668291+0000","last_clean":"2023-11-08T19:31:11.668291+0000","last_became_active":"2023-11-08T19:27:08.193938+0000","last_became_peered":"2023-11-08T19:27:08.193938+0000","last_unstale":"2023-11-08T19:31:11.668291+0000","last_undegraded":"2023-11-08T19:31:11.668291+0000","last_fullsized":"2023-11-08T19:31:11.668291+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:40:49.009717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":76,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.668798+0000","last_change":"2023-11-08T19:26:06.265690+0000","last_active":"2023-11-08T19:31:11.668798+0000","last_peered":"2023-11-08T19:31:11.668798+0000","last_clean":"2023-11-08T19:31:11.668798+0000","last_became_active":"2023-11-08T19:26:04.603397+0000","last_became_peered":"2023-11-08T19:26:04.603397+0000","last_unstale":"2023-11-08T19:31:11.668798+0000","last_undegraded":"2023-11-08T19:31:11.668798+0000","last_fullsized":"2023-11-08T19:31:11.668798+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:50:43.827570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053361499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":53,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.668234+0000","last_change":"2023-11-08T19:27:09.282750+0000","last_active":"2023-11-08T19:31:11.668234+0000","last_peered":"2023-11-08T19:31:11.668234+0000","last_clean":"2023-11-08T19:31:11.668234+0000","last_became_active":"2023-11-08T19:27:07.189943+0000","last_became_peered":"2023-11-08T19:27:07.189943+0000","last_unstale":"2023-11-08T19:31:11.668234+0000","last_undegraded":"2023-11-08T19:31:11.668234+0000","last_fullsized":"2023-11-08T19:31:11.668234+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:31:16.081058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000398725,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":53,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.009718+0000","last_change":"2023-11-08T19:27:08.188147+0000","last_active":"2023-11-08T19:31:13.009718+0000","last_peered":"2023-11-08T19:31:13.009718+0000","last_clean":"2023-11-08T19:31:13.009718+0000","last_became_active":"2023-11-08T19:27:08.187487+0000","last_became_peered":"2023-11-08T19:27:08.187487+0000","last_unstale":"2023-11-08T19:31:13.009718+0000","last_undegraded":"2023-11-08T19:31:13.009718+0000","last_fullsized":"2023-11-08T19:31:13.009718+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:22:40.300185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":82,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.668250+0000","last_change":"2023-11-08T19:26:06.425133+0000","last_active":"2023-11-08T19:31:11.668250+0000","last_peered":"2023-11-08T19:31:11.668250+0000","last_clean":"2023-11-08T19:31:11.668250+0000","last_became_active":"2023-11-08T19:26:04.601103+0000","last_became_peered":"2023-11-08T19:26:04.601103+0000","last_unstale":"2023-11-08T19:31:11.668250+0000","last_undegraded":"2023-11-08T19:31:11.668250+0000","last_fullsized":"2023-11-08T19:31:11.668250+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:31:14.042561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000229244,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":60,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.009084+0000","last_change":"2023-11-08T19:27:09.179852+0000","last_active":"2023-11-08T19:31:13.009084+0000","last_peered":"2023-11-08T19:31:13.009084+0000","last_clean":"2023-11-08T19:31:13.009084+0000","last_became_active":"2023-11-08T19:27:07.191926+0000","last_became_peered":"2023-11-08T19:27:07.191926+0000","last_unstale":"2023-11-08T19:31:13.009084+0000","last_undegraded":"2023-11-08T19:31:13.009084+0000","last_fullsized":"2023-11-08T19:31:13.009084+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:20:38.080129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055457100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":53,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.010318+0000","last_change":"2023-11-08T19:27:08.186054+0000","last_active":"2023-11-08T19:31:13.010318+0000","last_peered":"2023-11-08T19:31:13.010318+0000","last_clean":"2023-11-08T19:31:13.010318+0000","last_became_active":"2023-11-08T19:27:08.185581+0000","last_became_peered":"2023-11-08T19:27:08.185581+0000","last_unstale":"2023-11-08T19:31:13.010318+0000","last_undegraded":"2023-11-08T19:31:13.010318+0000","last_fullsized":"2023-11-08T19:31:13.010318+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:00:55.878397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":89,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.008858+0000","last_change":"2023-11-08T19:26:05.609625+0000","last_active":"2023-11-08T19:31:13.008858+0000","last_peered":"2023-11-08T19:31:13.008858+0000","last_clean":"2023-11-08T19:31:13.008858+0000","last_became_active":"2023-11-08T19:26:04.606312+0000","last_became_peered":"2023-11-08T19:26:04.606312+0000","last_unstale":"2023-11-08T19:31:13.008858+0000","last_undegraded":"2023-11-08T19:31:13.008858+0000","last_fullsized":"2023-11-08T19:31:13.008858+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:54:42.032901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000505935,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":53,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.444366+0000","last_change":"2023-11-08T19:27:09.177985+0000","last_active":"2023-11-08T19:31:11.444366+0000","last_peered":"2023-11-08T19:31:11.444366+0000","last_clean":"2023-11-08T19:31:11.444366+0000","last_became_active":"2023-11-08T19:27:07.193254+0000","last_became_peered":"2023-11-08T19:27:07.193254+0000","last_unstale":"2023-11-08T19:31:11.444366+0000","last_undegraded":"2023-11-08T19:31:11.444366+0000","last_fullsized":"2023-11-08T19:31:11.444366+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:23:38.662629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041830799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":53,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.011772+0000","last_change":"2023-11-08T19:27:08.188860+0000","last_active":"2023-11-08T19:31:13.011772+0000","last_peered":"2023-11-08T19:31:13.011772+0000","last_clean":"2023-11-08T19:31:13.011772+0000","last_became_active":"2023-11-08T19:27:08.188560+0000","last_became_peered":"2023-11-08T19:27:08.188560+0000","last_unstale":"2023-11-08T19:31:13.011772+0000","last_undegraded":"2023-11-08T19:31:13.011772+0000","last_fullsized":"2023-11-08T19:31:13.011772+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:18:20.969491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":77,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.667266+0000","last_change":"2023-11-08T19:26:06.297255+0000","last_active":"2023-11-08T19:31:11.667266+0000","last_peered":"2023-11-08T19:31:11.667266+0000","last_clean":"2023-11-08T19:31:11.667266+0000","last_became_active":"2023-11-08T19:26:04.603979+0000","last_became_peered":"2023-11-08T19:26:04.603979+0000","last_unstale":"2023-11-08T19:31:11.667266+0000","last_undegraded":"2023-11-08T19:31:11.667266+0000","last_fullsized":"2023-11-08T19:31:11.667266+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:40:11.164951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079032099999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":53,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.002043+0000","last_change":"2023-11-08T19:27:09.331990+0000","last_active":"2023-11-08T19:31:11.002043+0000","last_peered":"2023-11-08T19:31:11.002043+0000","last_clean":"2023-11-08T19:31:11.002043+0000","last_became_active":"2023-11-08T19:27:07.185478+0000","last_became_peered":"2023-11-08T19:27:07.185478+0000","last_unstale":"2023-11-08T19:31:11.002043+0000","last_undegraded":"2023-11-08T19:31:11.002043+0000","last_fullsized":"2023-11-08T19:31:11.002043+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:20:38.080129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010661850000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574930,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33036,"kb_used_data":1856,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338804,"statfs":{"total":96636764160,"available":96602935296,"internally_reserved":0,"allocated":1900544,"data_stored":1229475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:30:56 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.54400000000000004,"15min":0.51100000000000001},"min":{"1min":0.45300000000000001,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.71899999999999997,"5min":0.78600000000000003,"15min":1.0149999999999999},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.53500000000000003,"15min":0.52100000000000002},"min":{"1min":0.42099999999999999,"5min":0.35099999999999998,"15min":0.32300000000000001},"max":{"1min":0.79300000000000004,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.56000000000000005}]},{"osd":1,"last update":"Wed Nov 8 19:30:56 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.59799999999999998,"15min":0.57599999999999996},"min":{"1min":0.47699999999999998,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.73499999999999999,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.623,"5min":0.59599999999999997,"15min":0.56699999999999995},"min":{"1min":0.505,"5min":0.40600000000000003,"15min":0.33900000000000002},"max":{"1min":0.79500000000000004,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.45900000000000002}]},{"osd":2,"last update":"Wed Nov 8 19:30:56 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.54400000000000004,"15min":0.505},"min":{"1min":0.372,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.67600000000000005,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.57399999999999995,"15min":0.55600000000000005},"min":{"1min":0.45900000000000002,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.78400000000000003,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.40000000000000002}]},{"osd":3,"last update":"Wed Nov 8 19:30:56 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.57499999999999996,"15min":0.56399999999999995},"min":{"1min":0.42199999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.77300000000000002,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.498},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.63800000000000001,"15min":0.59299999999999997},"min":{"1min":0.47399999999999998,"5min":0.437,"15min":0.40100000000000002},"max":{"1min":0.81399999999999995,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.53900000000000003}]},{"osd":4,"last update":"Wed Nov 8 19:30:56 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.501,"15min":0.505},"min":{"1min":0.32400000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.753,"5min":0.753,"15min":0.78600000000000003},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.53400000000000003,"15min":0.53700000000000003},"min":{"1min":0.35199999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.71799999999999997,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.58099999999999996}]},{"osd":5,"last update":"Wed Nov 8 19:30:56 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.56699999999999995,"15min":0.57499999999999996},"min":{"1min":0.35099999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.77300000000000002,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.624},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.627,"15min":0.63700000000000001},"min":{"1min":0.377,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.83899999999999997,"5min":0.89900000000000002,"15min":0.94299999999999995},"last":0.60699999999999998}]},{"osd":6,"last update":"Wed Nov 8 19:30:56 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.57999999999999996,"15min":0.59699999999999998},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.97299999999999998},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.59699999999999998,"15min":0.60499999999999998},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.78900000000000003,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.52800000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574930,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32476,"kb_used_data":1292,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339364,"statfs":{"total":96636764160,"available":96603508736,"internally_reserved":0,"allocated":1323008,"data_stored":646397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:30:57 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.55900000000000005,"15min":1.1459999999999999},"min":{"1min":0.41999999999999998,"5min":0.33200000000000002,"15min":0.27000000000000002},"max":{"1min":0.84599999999999997,"5min":0.91700000000000004,"15min":21.359999999999999},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.56899999999999995,"15min":0.81599999999999995},"min":{"1min":0.39700000000000002,"5min":0.378,"15min":0.30399999999999999},"max":{"1min":0.68700000000000006,"5min":0.90300000000000002,"15min":10.329000000000001},"last":0.67200000000000004}]},{"osd":1,"last update":"Wed Nov 8 19:30:57 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.53800000000000003,"15min":0.59499999999999997},"min":{"1min":0.41199999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.81399999999999995,"5min":0.86299999999999999,"15min":3.4239999999999999},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.60499999999999998,"15min":0.63400000000000001},"min":{"1min":0.35899999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":2.9260000000000002,"5min":2.9260000000000002,"15min":3.4809999999999999},"last":0.65500000000000003}]},{"osd":2,"last update":"Wed Nov 8 19:30:57 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.58599999999999997,"15min":0.83399999999999996},"min":{"1min":0.51800000000000002,"5min":0.38500000000000001,"15min":0.34399999999999997},"max":{"1min":0.84199999999999997,"5min":0.96399999999999997,"15min":10.398},"last":0.623},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.56799999999999995,"15min":0.66200000000000003},"min":{"1min":0.41799999999999998,"5min":0.318,"15min":0.318},"max":{"1min":0.77500000000000002,"5min":0.89100000000000001,"15min":5.0170000000000003},"last":0.63500000000000001}]},{"osd":3,"last update":"Wed Nov 8 19:30:57 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.63100000000000001,"15min":0.66100000000000003},"min":{"1min":0.46200000000000002,"5min":0.34599999999999997,"15min":0.317},"max":{"1min":3.0089999999999999,"5min":3.0089999999999999,"15min":3.5710000000000002},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.59699999999999998,"15min":0.85999999999999999},"min":{"1min":0.39000000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.81100000000000005,"5min":0.93500000000000005,"15min":10.368},"last":0.75900000000000001}]},{"osd":4,"last update":"Wed Nov 8 19:30:57 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54400000000000004,"15min":0.52000000000000002},"min":{"1min":0.34699999999999998,"5min":0.33000000000000002,"15min":0.29199999999999998},"max":{"1min":0.79100000000000004,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.55900000000000005,"15min":0.55800000000000005},"min":{"1min":0.45000000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.73799999999999999,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.46700000000000003}]},{"osd":5,"last update":"Wed Nov 8 19:30:57 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.55000000000000004,"15min":0.53200000000000003},"min":{"1min":0.42699999999999999,"5min":0.34599999999999997,"15min":0.307},"max":{"1min":0.70499999999999996,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.60399999999999998,"15min":0.58599999999999997},"min":{"1min":0.41199999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.60099999999999998}]},{"osd":7,"last update":"Wed Nov 8 19:30:57 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.60999999999999999,"15min":0.58299999999999996},"min":{"1min":0.39700000000000002,"5min":0.39500000000000002,"15min":0.34699999999999998},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.60599999999999998,"15min":0.59199999999999997},"min":{"1min":0.46300000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.82199999999999995,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.74199999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574930,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32456,"kb_used_data":1276,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339384,"statfs":{"total":96636764160,"available":96603529216,"internally_reserved":0,"allocated":1306624,"data_stored":639107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:30:55 2023","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.35499999999999998,"15min":0.374},"min":{"1min":0.222,"5min":0.219,"15min":0.219},"max":{"1min":0.437,"5min":0.69699999999999995,"15min":1.389},"last":0.28299999999999997},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.36399999999999999,"15min":0.61699999999999999},"min":{"1min":0.255,"5min":0.23000000000000001,"15min":0.21199999999999999},"max":{"1min":0.52900000000000003,"5min":0.622,"15min":9.1370000000000005},"last":0.36399999999999999}]},{"osd":2,"last update":"Wed Nov 8 19:30:55 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.35899999999999999,"15min":0.36499999999999999},"min":{"1min":0.248,"5min":0.216,"15min":0.216},"max":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":1.2709999999999999},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.39200000000000002,"15min":0.41899999999999998},"min":{"1min":0.27700000000000002,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.53400000000000003,"5min":0.63600000000000001,"15min":1.425},"last":0.41799999999999998}]},{"osd":3,"last update":"Wed Nov 8 19:30:55 2023","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.40899999999999997,"15min":0.74399999999999999},"min":{"1min":0.29599999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.54600000000000004,"5min":0.63800000000000001,"15min":11.957000000000001},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.39100000000000001,"15min":0.91800000000000004},"min":{"1min":0.27900000000000003,"5min":0.252,"15min":0.252},"max":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":18.039000000000001},"last":0.25}]},{"osd":4,"last update":"Wed Nov 8 19:30:55 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59299999999999997,"15min":0.751},"min":{"1min":0.35199999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.81100000000000005,"5min":1.4370000000000001,"15min":6.181},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.60999999999999999,"15min":0.63},"min":{"1min":0.41399999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.83899999999999997,"5min":1.411,"15min":1.488},"last":0.57699999999999996}]},{"osd":5,"last update":"Wed Nov 8 19:30:55 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61199999999999999,"15min":0.86099999999999999},"min":{"1min":0.375,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.85399999999999998,"5min":1.421,"15min":9.2420000000000009},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60599999999999998,"15min":0.78900000000000003},"min":{"1min":0.42499999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.80100000000000005,"5min":1.2969999999999999,"15min":6.2110000000000003},"last":0.63}]},{"osd":6,"last update":"Wed Nov 8 19:30:55 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.63500000000000001,"15min":0.86899999999999999},"min":{"1min":0.40799999999999997,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.78700000000000003,"5min":1.4530000000000001,"15min":9.1880000000000006},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63500000000000001,"15min":0.65600000000000003},"min":{"1min":0.44500000000000001,"5min":0.38,"15min":0.38},"max":{"1min":0.86399999999999999,"5min":1.272,"15min":1.462},"last":0.65600000000000003}]},{"osd":7,"last update":"Wed Nov 8 19:30:55 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.625,"15min":0.78800000000000003},"min":{"1min":0.41499999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0309999999999999,"5min":1.3979999999999999,"15min":6.1479999999999997},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.64600000000000002,"15min":0.90000000000000002},"min":{"1min":0.51300000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.97499999999999998,"5min":1.353,"15min":9.2170000000000005},"last":0.64500000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574930,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33076,"kb_used_data":1896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338764,"statfs":{"total":96636764160,"available":96602894336,"internally_reserved":0,"allocated":1941504,"data_stored":1263942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Nov 8 19:31:01 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.33300000000000002,"15min":0.45900000000000002},"min":{"1min":0.20899999999999999,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.51000000000000001,"5min":0.55300000000000005,"15min":4.1200000000000001},"last":0.376},{"interface":"front","average":{"1min":0.439,"5min":0.371,"15min":0.91900000000000004},"min":{"1min":0.22800000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.62,"5min":0.62,"15min":16.940999999999999},"last":0.34300000000000003}]},{"osd":2,"last update":"Wed Nov 8 19:31:01 2023","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.35299999999999998,"15min":0.38700000000000001},"min":{"1min":0.25,"5min":0.22,"15min":0.22},"max":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":1.1899999999999999},"last":0.26700000000000002},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.39100000000000001,"15min":0.60499999999999998},"min":{"1min":0.32400000000000001,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":7.0209999999999999},"last":0.29099999999999998}]},{"osd":3,"last update":"Wed Nov 8 19:31:01 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.38400000000000001,"15min":0.51200000000000001},"min":{"1min":0.29899999999999999,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":4.0940000000000003},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.39500000000000002,"15min":0.433},"min":{"1min":0.32500000000000001,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.61899999999999999,"5min":0.62,"15min":1.2549999999999999},"last":0.312}]},{"osd":4,"last update":"Wed Nov 8 19:31:01 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.58899999999999997,"15min":0.71699999999999997},"min":{"1min":0.42799999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":4.0049999999999999},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.58399999999999996,"15min":0.623},"min":{"1min":0.442,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.746,"5min":0.79200000000000004,"15min":1.2270000000000001},"last":0.54000000000000004}]},{"osd":5,"last update":"Wed Nov 8 19:31:01 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.59699999999999998,"15min":0.72799999999999998},"min":{"1min":0.47199999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":4.3090000000000002},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.61199999999999999,"15min":0.72399999999999998},"min":{"1min":0.436,"5min":0.373,"15min":0.373},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":4.0709999999999997},"last":0.56999999999999995}]},{"osd":6,"last update":"Wed Nov 8 19:31:01 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.621,"15min":0.65800000000000003},"min":{"1min":0.40999999999999998,"5min":0.373,"15min":0.373},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":1.2849999999999999},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.64900000000000002,"15min":0.78500000000000003},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":4.2560000000000002},"last":0.64100000000000001}]},{"osd":7,"last update":"Wed Nov 8 19:31:01 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.625,"15min":0.74299999999999999},"min":{"1min":0.47099999999999997,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":4.2850000000000001},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.64700000000000002,"15min":0.77000000000000002},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.002,"5min":1.002,"15min":4.0460000000000003},"last":0.63}]}]},{"osd":2,"up_from":13,"seq":55834574930,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28284,"kb_used_data":1264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343556,"statfs":{"total":96636764160,"available":96607801344,"internally_reserved":0,"allocated":1294336,"data_stored":631401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:30:53 2023","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.35099999999999998,"15min":0.33800000000000002},"min":{"1min":0.23599999999999999,"5min":0.182,"15min":0.182},"max":{"1min":0.629,"5min":0.629,"15min":0.629},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.35899999999999999,"15min":0.55900000000000005},"min":{"1min":0.26800000000000002,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.56499999999999995,"5min":0.68000000000000005,"15min":6.0999999999999996},"last":0.34899999999999998}]},{"osd":1,"last update":"Wed Nov 8 19:30:53 2023","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.35599999999999998,"15min":0.34599999999999997},"min":{"1min":0.247,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.38800000000000001,"15min":0.39000000000000001},"min":{"1min":0.30299999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.63100000000000001,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.41099999999999998}]},{"osd":3,"last update":"Wed Nov 8 19:30:53 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.40600000000000003,"15min":0.39200000000000002},"min":{"1min":0.28199999999999997,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.60999999999999999,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.40000000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"min":{"1min":0.28799999999999998,"5min":0.217,"15min":0.217},"max":{"1min":0.621,"5min":0.65900000000000003,"15min":0.70299999999999996},"last":0.32900000000000001}]},{"osd":4,"last update":"Wed Nov 8 19:30:53 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56000000000000005,"15min":0.57599999999999996},"min":{"1min":0.41699999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.58099999999999996,"15min":0.58299999999999996},"min":{"1min":0.40500000000000003,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.872,"5min":0.872,"15min":0.90400000000000003},"last":0.59099999999999997}]},{"osd":5,"last update":"Wed Nov 8 19:30:53 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61799999999999999,"15min":0.59399999999999997},"min":{"1min":0.40799999999999997,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.622,"5min":0.60599999999999998,"15min":0.61099999999999999},"min":{"1min":0.48799999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.60099999999999998}]},{"osd":6,"last update":"Wed Nov 8 19:30:53 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.59399999999999997,"15min":0.59799999999999998},"min":{"1min":0.432,"5min":0.379,"15min":0.371},"max":{"1min":0.81000000000000005,"5min":0.83599999999999997,"15min":0.84799999999999998},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.59999999999999998,"15min":0.58599999999999997},"min":{"1min":0.42799999999999999,"5min":0.40300000000000002,"15min":0.36599999999999999},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.53400000000000003}]},{"osd":7,"last update":"Wed Nov 8 19:30:53 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.60899999999999999,"15min":0.58899999999999997},"min":{"1min":0.46200000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.61699999999999999,"15min":0.60899999999999999},"min":{"1min":0.41699999999999998,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.79600000000000004,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.623}]}]},{"osd":3,"up_from":13,"seq":55834574931,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28296,"kb_used_data":1276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343544,"statfs":{"total":96636764160,"available":96607789056,"internally_reserved":0,"allocated":1306624,"data_stored":639107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:30:59 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.33500000000000002,"15min":0.34000000000000002},"min":{"1min":0.254,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.48599999999999999,"5min":0.49099999999999999,"15min":0.626},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.378,"5min":0.372,"15min":0.35599999999999998},"min":{"1min":0.26700000000000002,"5min":0.23699999999999999,"15min":0.23300000000000001},"max":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.60799999999999998},"last":0.46500000000000002}]},{"osd":1,"last update":"Wed Nov 8 19:30:59 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.35999999999999999,"15min":0.35599999999999998},"min":{"1min":0.245,"5min":0.24099999999999999,"15min":0.23999999999999999},"max":{"1min":0.53300000000000003,"5min":0.61699999999999999,"15min":0.61699999999999999},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.40100000000000002,"15min":0.39000000000000001},"min":{"1min":0.30099999999999999,"5min":0.27900000000000003,"15min":0.27700000000000002},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.82099999999999995},"last":0.496}]},{"osd":2,"last update":"Wed Nov 8 19:30:59 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.40999999999999998,"15min":0.38300000000000001},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28799999999999998},"max":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58499999999999996},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.41999999999999998,"15min":0.40899999999999997},"min":{"1min":0.30499999999999999,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.59799999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.55900000000000005}]},{"osd":4,"last update":"Wed Nov 8 19:30:59 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.63,"15min":0.60799999999999998},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.85699999999999998,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63700000000000001,"15min":0.61499999999999999},"min":{"1min":0.38600000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.89600000000000002,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.67100000000000004}]},{"osd":5,"last update":"Wed Nov 8 19:30:59 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59899999999999998,"15min":0.58299999999999996},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.83099999999999996,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.623,"5min":0.622,"15min":0.59199999999999997},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.879,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.60399999999999998}]},{"osd":6,"last update":"Wed Nov 8 19:30:59 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.61399999999999999,"15min":0.59099999999999997},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41399999999999998},"max":{"1min":0.83899999999999997,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63300000000000001,"15min":0.61299999999999999},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.90500000000000003},"last":0.79700000000000004}]},{"osd":7,"last update":"Wed Nov 8 19:30:59 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.626,"15min":0.59999999999999998},"min":{"1min":0.41899999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":1.0009999999999999},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.63900000000000001,"15min":0.60999999999999999},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.85099999999999998,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.63800000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574930,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28296,"kb_used_data":1276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343544,"statfs":{"total":96636764160,"available":96607789056,"internally_reserved":0,"allocated":1306624,"data_stored":639107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:30:58 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54700000000000004,"15min":0.69899999999999995},"min":{"1min":0.311,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.71499999999999997,"5min":0.84799999999999998,"15min":6.5970000000000004},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.53500000000000003,"15min":0.56299999999999994},"min":{"1min":0.36299999999999999,"5min":0.32200000000000001,"15min":0.317},"max":{"1min":0.73099999999999998,"5min":0.75900000000000001,"15min":2.5030000000000001},"last":0.47099999999999997}]},{"osd":1,"last update":"Wed Nov 8 19:30:58 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.55400000000000005,"15min":0.51500000000000001},"min":{"1min":0.41699999999999998,"5min":0.32900000000000001,"15min":0.24299999999999999},"max":{"1min":0.72799999999999998,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.55900000000000005,"15min":0.52500000000000002},"min":{"1min":0.49299999999999999,"5min":0.312,"15min":0.312},"max":{"1min":0.72799999999999998,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.64500000000000002}]},{"osd":2,"last update":"Wed Nov 8 19:30:58 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55300000000000005},"min":{"1min":0.505,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.64400000000000002,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.58799999999999997,"15min":0.76000000000000001},"min":{"1min":0.50700000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.70999999999999996,"5min":0.871,"15min":6.6920000000000002},"last":0.44500000000000001}]},{"osd":3,"last update":"Wed Nov 8 19:30:58 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.56499999999999995,"15min":0.58099999999999996},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.69799999999999995,"5min":0.79800000000000004,"15min":1.0700000000000001},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.61799999999999999,"15min":0.61299999999999999},"min":{"1min":0.47799999999999998,"5min":0.41699999999999998,"15min":0.34599999999999997},"max":{"1min":0.70399999999999996,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.60199999999999998}]},{"osd":5,"last update":"Wed Nov 8 19:30:58 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.501,"15min":0.53100000000000003},"min":{"1min":0.39100000000000001,"5min":0.29599999999999999,"15min":0.28100000000000003},"max":{"1min":0.66100000000000003,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.51700000000000002,"15min":0.57099999999999995},"min":{"1min":0.377,"5min":0.32600000000000001,"15min":0.30299999999999999},"max":{"1min":0.65700000000000003,"5min":0.876,"15min":1.1060000000000001},"last":0.501}]},{"osd":6,"last update":"Wed Nov 8 19:30:58 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.54700000000000004,"15min":0.57199999999999995},"min":{"1min":0.35199999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.71899999999999997,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.60299999999999998,"15min":0.60999999999999999},"min":{"1min":0.373,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.74099999999999999,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.61899999999999999}]},{"osd":7,"last update":"Wed Nov 8 19:30:58 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.55800000000000005,"15min":0.60999999999999999},"min":{"1min":0.41599999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.68000000000000005,"5min":0.84499999999999997,"15min":1.141},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.59199999999999997,"15min":0.63},"min":{"1min":0.41399999999999998,"5min":0.38700000000000001,"15min":0.33000000000000002},"max":{"1min":0.76400000000000001,"5min":0.86399999999999999,"15min":0.86799999999999999},"last":0.63600000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574930,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28284,"kb_used_data":1264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343556,"statfs":{"total":96636764160,"available":96607801344,"internally_reserved":0,"allocated":1294336,"data_stored":631457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:30:51 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.57199999999999995,"15min":0.53900000000000003},"min":{"1min":0.35199999999999998,"5min":0.307,"15min":0.307},"max":{"1min":3.9689999999999999,"5min":3.9689999999999999,"15min":3.9689999999999999},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.54800000000000004,"15min":0.52200000000000002},"min":{"1min":0.41699999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.68899999999999995,"5min":0.746,"15min":0.746},"last":0.53400000000000003}]},{"osd":1,"last update":"Wed Nov 8 19:30:51 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.55300000000000005,"15min":0.53300000000000003},"min":{"1min":0.42399999999999999,"5min":0.37,"15min":0.37},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.54900000000000004,"15min":0.54100000000000004},"min":{"1min":0.44400000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.753,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.51400000000000001}]},{"osd":2,"last update":"Wed Nov 8 19:30:51 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.56399999999999995,"15min":0.54400000000000004},"min":{"1min":0.40000000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.749,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.56499999999999995,"15min":0.57299999999999995},"min":{"1min":0.46400000000000002,"5min":0.316,"15min":0.316},"max":{"1min":0.746,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.56499999999999995}]},{"osd":3,"last update":"Wed Nov 8 19:30:51 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.58499999999999996,"15min":0.56200000000000006},"min":{"1min":0.39600000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.73699999999999999,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.57999999999999996,"15min":0.59599999999999997},"min":{"1min":0.39900000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.79000000000000004,"5min":0.87,"15min":0.87},"last":0.61799999999999999}]},{"osd":4,"last update":"Wed Nov 8 19:30:51 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.495,"15min":0.54000000000000004},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.71899999999999997,"5min":0.80500000000000005,"15min":0.86799999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.53900000000000003,"15min":0.56100000000000005},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.751,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.71499999999999997}]},{"osd":6,"last update":"Wed Nov 8 19:30:51 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.56100000000000005,"15min":0.59599999999999997},"min":{"1min":0.39700000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.73599999999999999,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59199999999999997,"15min":0.60899999999999999},"min":{"1min":0.41399999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.76400000000000001,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.68500000000000005}]},{"osd":7,"last update":"Wed Nov 8 19:30:51 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.57099999999999995,"15min":0.59899999999999998},"min":{"1min":0.38200000000000001,"5min":0.371,"15min":0.36899999999999999},"max":{"1min":0.74099999999999999,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.746},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58999999999999997,"15min":0.63},"min":{"1min":0.433,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.91900000000000004},"last":0.70499999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-08T19:31:16.835 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-11-08T19:31:16.837 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-08T19:31:16.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.987+0000 7f3e1536c700 1 Processor -- start 2023-11-08T19:31:16.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.987+0000 7f3e1536c700 1 -- start start 2023-11-08T19:31:16.991 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.988+0000 7f3e1536c700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f3e101618f0 0x7f3e1006ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:16.992 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.988+0000 7f3e1536c700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f3e1006b150 0x7f3e100cb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:16.992 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.988+0000 7f3e1536c700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e10161df0 con 0x7f3e1015dd00 2023-11-08T19:31:16.992 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.988+0000 7f3e1536c700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e1006b600 con 0x7f3e101618f0 2023-11-08T19:31:16.992 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.988+0000 7f3e1536c700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e100cad50 con 0x7f3e1006b150 2023-11-08T19:31:16.993 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.988+0000 7f3e0e7fc700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f3e101618f0 0x7f3e1006ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:16.993 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.988+0000 7f3e0effd700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f3e1015dd00 0x7f3e100cca70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:53278/0 (socket says 172.21.15.143:53278) 2023-11-08T19:31:16.994 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.988+0000 7f3e0effd700 1 -- 172.21.15.143:0/1783229919 learned_addr learned my addr 172.21.15.143:0/1783229919 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:31:16.994 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.988+0000 7f3e0f7fe700 1 --2- 172.21.15.143:0/1783229919 >> v2:172.21.15.174:3300/0 conn(0x7f3e1006b150 0x7f3e100cb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:16.994 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.988+0000 7f3e0e7fc700 1 -- 172.21.15.143:0/1783229919 >> v2:172.21.15.174:3300/0 conn(0x7f3e1006b150 msgr2=0x7f3e100cb1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:16.994 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.988+0000 7f3e0e7fc700 1 --2- 172.21.15.143:0/1783229919 >> v2:172.21.15.174:3300/0 conn(0x7f3e1006b150 0x7f3e100cb1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:16.995 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.988+0000 7f3e0e7fc700 1 -- 172.21.15.143:0/1783229919 >> v1:172.21.15.143:6789/0 conn(0x7f3e1015dd00 legacy=0x7f3e100cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:16.995 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.988+0000 7f3e0e7fc700 1 -- 172.21.15.143:0/1783229919 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e101620d0 con 0x7f3e101618f0 2023-11-08T19:31:16.995 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.989+0000 7f3e0e7fc700 1 --2- 172.21.15.143:0/1783229919 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e101618f0 0x7f3e1006ac10 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f3e10099f70 tx=0x7f3e00009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ebb3e44d2fe022e server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:16.996 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.989+0000 7f3e0dffb700 1 -- 172.21.15.143:0/1783229919 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e00011830 con 0x7f3e101618f0 2023-11-08T19:31:16.996 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.989+0000 7f3e0dffb700 1 -- 172.21.15.143:0/1783229919 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e00011990 con 0x7f3e101618f0 2023-11-08T19:31:16.996 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.989+0000 7f3e0dffb700 1 -- 172.21.15.143:0/1783229919 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e00011d20 con 0x7f3e101618f0 2023-11-08T19:31:16.996 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.989+0000 7f3e1536c700 1 -- 172.21.15.143:0/1783229919 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e101618f0 msgr2=0x7f3e1006ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:16.997 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.989+0000 7f3e1536c700 1 --2- 172.21.15.143:0/1783229919 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e101618f0 0x7f3e1006ac10 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f3e10099f70 tx=0x7f3e00009b20 comp rx=0 tx=0).stop 2023-11-08T19:31:16.997 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.989+0000 7f3e1536c700 1 -- 172.21.15.143:0/1783229919 shutdown_connections 2023-11-08T19:31:16.997 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.989+0000 7f3e1536c700 1 --2- 172.21.15.143:0/1783229919 >> v2:172.21.15.174:3300/0 conn(0x7f3e1006b150 0x7f3e100cb1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:16.997 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.989+0000 7f3e1536c700 1 --2- 172.21.15.143:0/1783229919 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e101618f0 0x7f3e1006ac10 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:16.998 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.989+0000 7f3e1536c700 1 -- 172.21.15.143:0/1783229919 >> 172.21.15.143:0/1783229919 conn(0x7f3e10099ab0 msgr2=0x7f3e100d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:16.998 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.989+0000 7f3e1536c700 1 -- 172.21.15.143:0/1783229919 shutdown_connections 2023-11-08T19:31:16.998 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.989+0000 7f3e1536c700 1 -- 172.21.15.143:0/1783229919 wait complete. 2023-11-08T19:31:17.001 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.991+0000 7f3e1536c700 1 Processor -- start 2023-11-08T19:31:17.001 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.991+0000 7f3e1536c700 1 -- start start 2023-11-08T19:31:17.002 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.991+0000 7f3e1536c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e1006b150 0x7f3e100b1140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:17.002 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.991+0000 7f3e1536c700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3e1015dd00 0x7f3e100c77d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:17.002 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.991+0000 7f3e1536c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3e101618f0 0x7f3e10152120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:17.002 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.991+0000 7f3e1536c700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e10152660 con 0x7f3e1006b150 2023-11-08T19:31:17.003 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.991+0000 7f3e1536c700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e100c9650 con 0x7f3e101618f0 2023-11-08T19:31:17.003 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.991+0000 7f3e1536c700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e100b3a70 con 0x7f3e1015dd00 2023-11-08T19:31:17.003 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.991+0000 7f3e0effd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e1006b150 0x7f3e100b1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:17.004 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.991+0000 7f3e0effd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e1006b150 0x7f3e100b1140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57130/0 (socket says 172.21.15.143:57130) 2023-11-08T19:31:17.004 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.991+0000 7f3e0effd700 1 -- 172.21.15.143:0/3770802358 learned_addr learned my addr 172.21.15.143:0/3770802358 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:31:17.004 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.992+0000 7f3e0effd700 1 -- 172.21.15.143:0/3770802358 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3e1015dd00 msgr2=0x7f3e100c77d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:17.005 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.992+0000 7f3e0effd700 1 --2- 172.21.15.143:0/3770802358 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3e1015dd00 0x7f3e100c77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:17.005 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.992+0000 7f3e0effd700 1 -- 172.21.15.143:0/3770802358 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3e101618f0 msgr2=0x7f3e10152120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:17.005 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.992+0000 7f3e0effd700 1 --2- 172.21.15.143:0/3770802358 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3e101618f0 0x7f3e10152120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:17.006 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.992+0000 7f3e0effd700 1 -- 172.21.15.143:0/3770802358 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e000094a0 con 0x7f3e1006b150 2023-11-08T19:31:17.006 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.992+0000 7f3e0effd700 1 --2- 172.21.15.143:0/3770802358 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e1006b150 0x7f3e100b1140 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f3e0400c730 tx=0x7f3e0400caf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:17.006 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.992+0000 7f3df77fe700 1 -- 172.21.15.143:0/3770802358 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e040074e0 con 0x7f3e1006b150 2023-11-08T19:31:17.006 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.992+0000 7f3df77fe700 1 -- 172.21.15.143:0/3770802358 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e04007640 con 0x7f3e1006b150 2023-11-08T19:31:17.007 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.992+0000 7f3df77fe700 1 -- 172.21.15.143:0/3770802358 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e04005a30 con 0x7f3e1006b150 2023-11-08T19:31:17.007 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.992+0000 7f3e1536c700 1 -- 172.21.15.143:0/3770802358 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e100b5d30 con 0x7f3e1006b150 2023-11-08T19:31:17.007 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.992+0000 7f3e1536c700 1 -- 172.21.15.143:0/3770802358 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e100b62f0 con 0x7f3e1006b150 2023-11-08T19:31:17.008 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.993+0000 7f3e1536c700 1 -- 172.21.15.143:0/3770802358 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e100aff60 con 0x7f3e1006b150 2023-11-08T19:31:17.008 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.993+0000 7f3df77fe700 1 -- 172.21.15.143:0/3770802358 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3e04005b90 con 0x7f3e1006b150 2023-11-08T19:31:17.008 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.994+0000 7f3df77fe700 1 --2- 172.21.15.143:0/3770802358 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f3dfc06af00 0x7f3dfc06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:17.008 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.994+0000 7f3df77fe700 1 -- 172.21.15.143:0/3770802358 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6651+0+0 (secure 0 0 0) 0x7f3e04088ce0 con 0x7f3e1006b150 2023-11-08T19:31:17.009 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.994+0000 7f3e0e7fc700 1 --2- 172.21.15.143:0/3770802358 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f3dfc06af00 0x7f3dfc06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:17.009 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.994+0000 7f3e0e7fc700 1 --2- 172.21.15.143:0/3770802358 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f3dfc06af00 0x7f3dfc06d3c0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f3e100b3260 tx=0x7f3e00010000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:17.009 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:16.998+0000 7f3df77fe700 1 -- 172.21.15.143:0/3770802358 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3e04053b20 con 0x7f3e1006b150 2023-11-08T19:31:17.186 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:17.185+0000 7f3e1536c700 1 -- 172.21.15.143:0/3770802358 --> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3e100ce5f0 con 0x7f3dfc06af00 2023-11-08T19:31:17.195 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:17.194+0000 7f3df77fe700 1 -- 172.21.15.143:0/3770802358 <== mgr.4099 v2:172.21.15.174:6832/4037280178 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+293897 (secure 0 0 0) 0x7f3e100ce5f0 con 0x7f3dfc06af00 2023-11-08T19:31:17.198 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:31:17.201 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:17.197+0000 7f3e1536c700 1 -- 172.21.15.143:0/3770802358 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f3dfc06af00 msgr2=0x7f3dfc06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:17.201 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:17.197+0000 7f3e1536c700 1 --2- 172.21.15.143:0/3770802358 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f3dfc06af00 0x7f3dfc06d3c0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f3e100b3260 tx=0x7f3e00010000 comp rx=0 tx=0).stop 2023-11-08T19:31:17.202 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:17.197+0000 7f3e1536c700 1 -- 172.21.15.143:0/3770802358 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e1006b150 msgr2=0x7f3e100b1140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:17.202 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:17.197+0000 7f3e1536c700 1 --2- 172.21.15.143:0/3770802358 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e1006b150 0x7f3e100b1140 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f3e0400c730 tx=0x7f3e0400caf0 comp rx=0 tx=0).stop 2023-11-08T19:31:17.202 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:17.197+0000 7f3e1536c700 1 -- 172.21.15.143:0/3770802358 shutdown_connections 2023-11-08T19:31:17.202 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:17.197+0000 7f3e1536c700 1 --2- 172.21.15.143:0/3770802358 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f3dfc06af00 0x7f3dfc06d3c0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:17.203 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:17.197+0000 7f3e1536c700 1 --2- 172.21.15.143:0/3770802358 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3e101618f0 0x7f3e10152120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:17.203 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:17.197+0000 7f3e1536c700 1 --2- 172.21.15.143:0/3770802358 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e1006b150 0x7f3e100b1140 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:17.203 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:17.197+0000 7f3e1536c700 1 --2- 172.21.15.143:0/3770802358 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3e1015dd00 0x7f3e100c77d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:17.204 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:17.197+0000 7f3e1536c700 1 -- 172.21.15.143:0/3770802358 >> 172.21.15.143:0/3770802358 conn(0x7f3e10099ab0 msgr2=0x7f3e100ad430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:17.204 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:17.197+0000 7f3e1536c700 1 -- 172.21.15.143:0/3770802358 shutdown_connections 2023-11-08T19:31:17.204 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:17.197+0000 7f3e1536c700 1 -- 172.21.15.143:0/3770802358 wait complete. 2023-11-08T19:31:17.204 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2023-11-08T19:31:17.210 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":296,"stamp":"2023-11-08T19:31:16.623813+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":371,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":244204,"kb_used_data":11400,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754730516,"statfs":{"total":773094113280,"available":772844048384,"internally_reserved":0,"allocated":11673600,"data_stored":6319993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-68399,"num_objects":-41,"num_object_clones":0,"num_object_copies":-82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-41,"num_whiteouts":0,"num_read":-469,"num_read_kb":-599,"num_write":-179,"num_write_kb":-143,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001715"},"pg_stats":[{"pgid":"12.d","version":"0'0","reported_seq":53,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.009381+0000","last_change":"2023-11-08T19:27:08.190826+0000","last_active":"2023-11-08T19:31:13.009381+0000","last_peered":"2023-11-08T19:31:13.009381+0000","last_clean":"2023-11-08T19:31:13.009381+0000","last_became_active":"2023-11-08T19:27:08.190440+0000","last_became_peered":"2023-11-08T19:27:08.190440+0000","last_unstale":"2023-11-08T19:31:13.009381+0000","last_undegraded":"2023-11-08T19:31:13.009381+0000","last_fullsized":"2023-11-08T19:31:13.009381+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:42:48.376462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":255,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.008215+0000","last_change":"2023-11-08T19:24:45.888122+0000","last_active":"2023-11-08T19:31:13.008215+0000","last_peered":"2023-11-08T19:31:13.008215+0000","last_clean":"2023-11-08T19:31:13.008215+0000","last_became_active":"2023-11-08T19:24:45.887048+0000","last_became_peered":"2023-11-08T19:24:45.887048+0000","last_unstale":"2023-11-08T19:31:13.008215+0000","last_undegraded":"2023-11-08T19:31:13.008215+0000","last_fullsized":"2023-11-08T19:31:13.008215+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:44.575133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:44.575133+0000","last_clean_scrub_stamp":"2023-11-08T19:24:44.575133+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:19:20.202062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":78,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.668985+0000","last_change":"2023-11-08T19:26:06.542044+0000","last_active":"2023-11-08T19:31:11.668985+0000","last_peered":"2023-11-08T19:31:11.668985+0000","last_clean":"2023-11-08T19:31:11.668985+0000","last_became_active":"2023-11-08T19:26:04.603056+0000","last_became_peered":"2023-11-08T19:26:04.603056+0000","last_unstale":"2023-11-08T19:31:11.668985+0000","last_undegraded":"2023-11-08T19:31:11.668985+0000","last_fullsized":"2023-11-08T19:31:11.668985+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:41:50.771457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000517414,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":57,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.667621+0000","last_change":"2023-11-08T19:27:09.496842+0000","last_active":"2023-11-08T19:31:11.667621+0000","last_peered":"2023-11-08T19:31:11.667621+0000","last_clean":"2023-11-08T19:31:11.667621+0000","last_became_active":"2023-11-08T19:27:07.185776+0000","last_became_peered":"2023-11-08T19:27:07.185776+0000","last_unstale":"2023-11-08T19:31:11.667621+0000","last_undegraded":"2023-11-08T19:31:11.667621+0000","last_fullsized":"2023-11-08T19:31:11.667621+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:21:13.923970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031747200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":50,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.668213+0000","last_change":"2023-11-08T19:27:08.188241+0000","last_active":"2023-11-08T19:31:11.668213+0000","last_peered":"2023-11-08T19:31:11.668213+0000","last_clean":"2023-11-08T19:31:11.668213+0000","last_became_active":"2023-11-08T19:27:08.187863+0000","last_became_peered":"2023-11-08T19:27:08.187863+0000","last_unstale":"2023-11-08T19:31:11.668213+0000","last_undegraded":"2023-11-08T19:31:11.668213+0000","last_fullsized":"2023-11-08T19:31:11.668213+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:45:22.084015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":76,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.667763+0000","last_change":"2023-11-08T19:26:06.265419+0000","last_active":"2023-11-08T19:31:11.667763+0000","last_peered":"2023-11-08T19:31:11.667763+0000","last_clean":"2023-11-08T19:31:11.667763+0000","last_became_active":"2023-11-08T19:26:04.605333+0000","last_became_peered":"2023-11-08T19:26:04.605333+0000","last_unstale":"2023-11-08T19:31:11.667763+0000","last_undegraded":"2023-11-08T19:31:11.667763+0000","last_fullsized":"2023-11-08T19:31:11.667763+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:01:56.991346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064606400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":60,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.010017+0000","last_change":"2023-11-08T19:27:09.182113+0000","last_active":"2023-11-08T19:31:13.010017+0000","last_peered":"2023-11-08T19:31:13.010017+0000","last_clean":"2023-11-08T19:31:13.010017+0000","last_became_active":"2023-11-08T19:27:07.189939+0000","last_became_peered":"2023-11-08T19:27:07.189939+0000","last_unstale":"2023-11-08T19:31:13.010017+0000","last_undegraded":"2023-11-08T19:31:13.010017+0000","last_fullsized":"2023-11-08T19:31:13.010017+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:41:08.822366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036507500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":46,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.444341+0000","last_change":"2023-11-08T19:27:08.187127+0000","last_active":"2023-11-08T19:31:11.444341+0000","last_peered":"2023-11-08T19:31:11.444341+0000","last_clean":"2023-11-08T19:31:11.444341+0000","last_became_active":"2023-11-08T19:27:08.182648+0000","last_became_peered":"2023-11-08T19:27:08.182648+0000","last_unstale":"2023-11-08T19:31:11.444341+0000","last_undegraded":"2023-11-08T19:31:11.444341+0000","last_fullsized":"2023-11-08T19:31:11.444341+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:50:12.101878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":89,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.009586+0000","last_change":"2023-11-08T19:26:05.609570+0000","last_active":"2023-11-08T19:31:13.009586+0000","last_peered":"2023-11-08T19:31:13.009586+0000","last_clean":"2023-11-08T19:31:13.009586+0000","last_became_active":"2023-11-08T19:26:04.608379+0000","last_became_peered":"2023-11-08T19:26:04.608379+0000","last_unstale":"2023-11-08T19:31:13.009586+0000","last_undegraded":"2023-11-08T19:31:13.009586+0000","last_fullsized":"2023-11-08T19:31:13.009586+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:40:08.986649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000529657,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":53,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.668577+0000","last_change":"2023-11-08T19:27:09.211991+0000","last_active":"2023-11-08T19:31:11.668577+0000","last_peered":"2023-11-08T19:31:11.668577+0000","last_clean":"2023-11-08T19:31:11.668577+0000","last_became_active":"2023-11-08T19:27:07.195231+0000","last_became_peered":"2023-11-08T19:27:07.195231+0000","last_unstale":"2023-11-08T19:31:11.668577+0000","last_undegraded":"2023-11-08T19:31:11.668577+0000","last_fullsized":"2023-11-08T19:31:11.668577+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:25:14.487524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000441221,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":46,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.444418+0000","last_change":"2023-11-08T19:27:08.195364+0000","last_active":"2023-11-08T19:31:11.444418+0000","last_peered":"2023-11-08T19:31:11.444418+0000","last_clean":"2023-11-08T19:31:11.444418+0000","last_became_active":"2023-11-08T19:27:08.195074+0000","last_became_peered":"2023-11-08T19:27:08.195074+0000","last_unstale":"2023-11-08T19:31:11.444418+0000","last_undegraded":"2023-11-08T19:31:11.444418+0000","last_fullsized":"2023-11-08T19:31:11.444418+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:02:57.552873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":76,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.002147+0000","last_change":"2023-11-08T19:26:06.394300+0000","last_active":"2023-11-08T19:31:11.002147+0000","last_peered":"2023-11-08T19:31:11.002147+0000","last_clean":"2023-11-08T19:31:11.002147+0000","last_became_active":"2023-11-08T19:26:04.601685+0000","last_became_peered":"2023-11-08T19:26:04.601685+0000","last_unstale":"2023-11-08T19:31:11.002147+0000","last_undegraded":"2023-11-08T19:31:11.002147+0000","last_fullsized":"2023-11-08T19:31:11.002147+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:51:06.626474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061044099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":53,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.444658+0000","last_change":"2023-11-08T19:27:09.178466+0000","last_active":"2023-11-08T19:31:11.444658+0000","last_peered":"2023-11-08T19:31:11.444658+0000","last_clean":"2023-11-08T19:31:11.444658+0000","last_became_active":"2023-11-08T19:27:07.189386+0000","last_became_peered":"2023-11-08T19:27:07.189386+0000","last_unstale":"2023-11-08T19:31:11.444658+0000","last_undegraded":"2023-11-08T19:31:11.444658+0000","last_fullsized":"2023-11-08T19:31:11.444658+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:48:51.139820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049390100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":50,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.666965+0000","last_change":"2023-11-08T19:27:08.189452+0000","last_active":"2023-11-08T19:31:11.666965+0000","last_peered":"2023-11-08T19:31:11.666965+0000","last_clean":"2023-11-08T19:31:11.666965+0000","last_became_active":"2023-11-08T19:27:08.189088+0000","last_became_peered":"2023-11-08T19:27:08.189088+0000","last_unstale":"2023-11-08T19:31:11.666965+0000","last_undegraded":"2023-11-08T19:31:11.666965+0000","last_fullsized":"2023-11-08T19:31:11.666965+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:25:18.963220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":77,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.666780+0000","last_change":"2023-11-08T19:26:06.296634+0000","last_active":"2023-11-08T19:31:11.666780+0000","last_peered":"2023-11-08T19:31:11.666780+0000","last_clean":"2023-11-08T19:31:11.666780+0000","last_became_active":"2023-11-08T19:26:04.603088+0000","last_became_peered":"2023-11-08T19:26:04.603088+0000","last_unstale":"2023-11-08T19:31:11.666780+0000","last_undegraded":"2023-11-08T19:31:11.666780+0000","last_fullsized":"2023-11-08T19:31:11.666780+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:01:56.991346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038180899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":53,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.666847+0000","last_change":"2023-11-08T19:27:09.431220+0000","last_active":"2023-11-08T19:31:11.666847+0000","last_peered":"2023-11-08T19:31:11.666847+0000","last_clean":"2023-11-08T19:31:11.666847+0000","last_became_active":"2023-11-08T19:27:07.191589+0000","last_became_peered":"2023-11-08T19:27:07.191589+0000","last_unstale":"2023-11-08T19:31:11.666847+0000","last_undegraded":"2023-11-08T19:31:11.666847+0000","last_fullsized":"2023-11-08T19:31:11.666847+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:18:19.964487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000729527,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":46,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.002470+0000","last_change":"2023-11-08T19:27:08.193337+0000","last_active":"2023-11-08T19:31:11.002470+0000","last_peered":"2023-11-08T19:31:11.002470+0000","last_clean":"2023-11-08T19:31:11.002470+0000","last_became_active":"2023-11-08T19:27:08.192997+0000","last_became_peered":"2023-11-08T19:27:08.192997+0000","last_unstale":"2023-11-08T19:31:11.002470+0000","last_undegraded":"2023-11-08T19:31:11.002470+0000","last_fullsized":"2023-11-08T19:31:11.002470+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:45:22.084015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":82,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.668119+0000","last_change":"2023-11-08T19:26:06.424511+0000","last_active":"2023-11-08T19:31:11.668119+0000","last_peered":"2023-11-08T19:31:11.668119+0000","last_clean":"2023-11-08T19:31:11.668119+0000","last_became_active":"2023-11-08T19:26:04.598973+0000","last_became_peered":"2023-11-08T19:26:04.598973+0000","last_unstale":"2023-11-08T19:31:11.668119+0000","last_undegraded":"2023-11-08T19:31:11.668119+0000","last_fullsized":"2023-11-08T19:31:11.668119+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:41:53.497456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031755900000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":57,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.667045+0000","last_change":"2023-11-08T19:27:09.497024+0000","last_active":"2023-11-08T19:31:11.667045+0000","last_peered":"2023-11-08T19:31:11.667045+0000","last_clean":"2023-11-08T19:31:11.667045+0000","last_became_active":"2023-11-08T19:27:07.190132+0000","last_became_peered":"2023-11-08T19:27:07.190132+0000","last_unstale":"2023-11-08T19:31:11.667045+0000","last_undegraded":"2023-11-08T19:31:11.667045+0000","last_fullsized":"2023-11-08T19:31:11.667045+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:36:34.881217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054126000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":46,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.002863+0000","last_change":"2023-11-08T19:27:08.194196+0000","last_active":"2023-11-08T19:31:11.002863+0000","last_peered":"2023-11-08T19:31:11.002863+0000","last_clean":"2023-11-08T19:31:11.002863+0000","last_became_active":"2023-11-08T19:27:08.193912+0000","last_became_peered":"2023-11-08T19:27:08.193912+0000","last_unstale":"2023-11-08T19:31:11.002863+0000","last_undegraded":"2023-11-08T19:31:11.002863+0000","last_fullsized":"2023-11-08T19:31:11.002863+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:25:18.963220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":77,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.667020+0000","last_change":"2023-11-08T19:26:06.296937+0000","last_active":"2023-11-08T19:31:11.667020+0000","last_peered":"2023-11-08T19:31:11.667020+0000","last_clean":"2023-11-08T19:31:11.667020+0000","last_became_active":"2023-11-08T19:26:04.599999+0000","last_became_peered":"2023-11-08T19:26:04.599999+0000","last_unstale":"2023-11-08T19:31:11.667020+0000","last_undegraded":"2023-11-08T19:31:11.667020+0000","last_fullsized":"2023-11-08T19:31:11.667020+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:50:43.827570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025605700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":57,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.667208+0000","last_change":"2023-11-08T19:27:09.497040+0000","last_active":"2023-11-08T19:31:11.667208+0000","last_peered":"2023-11-08T19:31:11.667208+0000","last_clean":"2023-11-08T19:31:11.667208+0000","last_became_active":"2023-11-08T19:27:07.189135+0000","last_became_peered":"2023-11-08T19:27:07.189135+0000","last_unstale":"2023-11-08T19:31:11.667208+0000","last_undegraded":"2023-11-08T19:31:11.667208+0000","last_fullsized":"2023-11-08T19:31:11.667208+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:12:29.400972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051794299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":53,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.011810+0000","last_change":"2023-11-08T19:27:08.187053+0000","last_active":"2023-11-08T19:31:13.011810+0000","last_peered":"2023-11-08T19:31:13.011810+0000","last_clean":"2023-11-08T19:31:13.011810+0000","last_became_active":"2023-11-08T19:27:08.186643+0000","last_became_peered":"2023-11-08T19:27:08.186643+0000","last_unstale":"2023-11-08T19:31:13.011810+0000","last_undegraded":"2023-11-08T19:31:13.011810+0000","last_fullsized":"2023-11-08T19:31:13.011810+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:35:30.073885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":89,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.010538+0000","last_change":"2023-11-08T19:26:05.609824+0000","last_active":"2023-11-08T19:31:13.010538+0000","last_peered":"2023-11-08T19:31:13.010538+0000","last_clean":"2023-11-08T19:31:13.010538+0000","last_became_active":"2023-11-08T19:26:04.603320+0000","last_became_peered":"2023-11-08T19:26:04.603320+0000","last_unstale":"2023-11-08T19:31:13.010538+0000","last_undegraded":"2023-11-08T19:31:13.010538+0000","last_fullsized":"2023-11-08T19:31:13.010538+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:05:45.976615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063225,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":57,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.668286+0000","last_change":"2023-11-08T19:27:09.496655+0000","last_active":"2023-11-08T19:31:11.668286+0000","last_peered":"2023-11-08T19:31:11.668286+0000","last_clean":"2023-11-08T19:31:11.668286+0000","last_became_active":"2023-11-08T19:27:07.189895+0000","last_became_peered":"2023-11-08T19:27:07.189895+0000","last_unstale":"2023-11-08T19:31:11.668286+0000","last_undegraded":"2023-11-08T19:31:11.668286+0000","last_fullsized":"2023-11-08T19:31:11.668286+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:41:08.822366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045654199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":78,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.667623+0000","last_change":"2023-11-08T19:26:06.541360+0000","last_active":"2023-11-08T19:31:11.667623+0000","last_peered":"2023-11-08T19:31:11.667623+0000","last_clean":"2023-11-08T19:31:11.667623+0000","last_became_active":"2023-11-08T19:26:04.603725+0000","last_became_peered":"2023-11-08T19:26:04.603725+0000","last_unstale":"2023-11-08T19:31:11.667623+0000","last_undegraded":"2023-11-08T19:31:11.667623+0000","last_fullsized":"2023-11-08T19:31:11.667623+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:28:48.576742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053365400000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":53,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.669050+0000","last_change":"2023-11-08T19:27:09.283599+0000","last_active":"2023-11-08T19:31:11.669050+0000","last_peered":"2023-11-08T19:31:11.669050+0000","last_clean":"2023-11-08T19:31:11.669050+0000","last_became_active":"2023-11-08T19:27:07.189595+0000","last_became_peered":"2023-11-08T19:27:07.189595+0000","last_unstale":"2023-11-08T19:31:11.669050+0000","last_undegraded":"2023-11-08T19:31:11.669050+0000","last_fullsized":"2023-11-08T19:31:11.669050+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:48:20.082682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000323479,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":47,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.667698+0000","last_change":"2023-11-08T19:27:08.185395+0000","last_active":"2023-11-08T19:31:11.667698+0000","last_peered":"2023-11-08T19:31:11.667698+0000","last_clean":"2023-11-08T19:31:11.667698+0000","last_became_active":"2023-11-08T19:27:08.184765+0000","last_became_peered":"2023-11-08T19:27:08.184765+0000","last_unstale":"2023-11-08T19:31:11.667698+0000","last_undegraded":"2023-11-08T19:31:11.667698+0000","last_fullsized":"2023-11-08T19:31:11.667698+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:08:51.864418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":82,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.667206+0000","last_change":"2023-11-08T19:26:06.424236+0000","last_active":"2023-11-08T19:31:11.667206+0000","last_peered":"2023-11-08T19:31:11.667206+0000","last_clean":"2023-11-08T19:31:11.667206+0000","last_became_active":"2023-11-08T19:26:04.600115+0000","last_became_peered":"2023-11-08T19:26:04.600115+0000","last_unstale":"2023-11-08T19:31:11.667206+0000","last_undegraded":"2023-11-08T19:31:11.667206+0000","last_fullsized":"2023-11-08T19:31:11.667206+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:18:01.876605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065803900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":53,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.002693+0000","last_change":"2023-11-08T19:27:09.331899+0000","last_active":"2023-11-08T19:31:11.002693+0000","last_peered":"2023-11-08T19:31:11.002693+0000","last_clean":"2023-11-08T19:31:11.002693+0000","last_became_active":"2023-11-08T19:27:07.191460+0000","last_became_peered":"2023-11-08T19:27:07.191460+0000","last_unstale":"2023-11-08T19:31:11.002693+0000","last_undegraded":"2023-11-08T19:31:11.002693+0000","last_fullsized":"2023-11-08T19:31:11.002693+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:51:29.247104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044907500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":53,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.009947+0000","last_change":"2023-11-08T19:27:08.191802+0000","last_active":"2023-11-08T19:31:13.009947+0000","last_peered":"2023-11-08T19:31:13.009947+0000","last_clean":"2023-11-08T19:31:13.009947+0000","last_became_active":"2023-11-08T19:27:08.191520+0000","last_became_peered":"2023-11-08T19:27:08.191520+0000","last_unstale":"2023-11-08T19:31:13.009947+0000","last_undegraded":"2023-11-08T19:31:13.009947+0000","last_fullsized":"2023-11-08T19:31:13.009947+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:49:16.458634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":89,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.008132+0000","last_change":"2023-11-08T19:26:05.608892+0000","last_active":"2023-11-08T19:31:13.008132+0000","last_peered":"2023-11-08T19:31:13.008132+0000","last_clean":"2023-11-08T19:31:13.008132+0000","last_became_active":"2023-11-08T19:26:04.604519+0000","last_became_peered":"2023-11-08T19:26:04.604519+0000","last_unstale":"2023-11-08T19:31:13.008132+0000","last_undegraded":"2023-11-08T19:31:13.008132+0000","last_fullsized":"2023-11-08T19:31:13.008132+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:11:26.508634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051248099999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":60,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.010356+0000","last_change":"2023-11-08T19:27:09.181431+0000","last_active":"2023-11-08T19:31:13.010356+0000","last_peered":"2023-11-08T19:31:13.010356+0000","last_clean":"2023-11-08T19:31:13.010356+0000","last_became_active":"2023-11-08T19:27:07.190800+0000","last_became_peered":"2023-11-08T19:27:07.190800+0000","last_unstale":"2023-11-08T19:31:13.010356+0000","last_undegraded":"2023-11-08T19:31:13.010356+0000","last_fullsized":"2023-11-08T19:31:13.010356+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:46:30.452678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656099999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":46,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.667947+0000","last_change":"2023-11-08T19:27:08.187983+0000","last_active":"2023-11-08T19:31:11.667947+0000","last_peered":"2023-11-08T19:31:11.667947+0000","last_clean":"2023-11-08T19:31:11.667947+0000","last_became_active":"2023-11-08T19:27:08.187322+0000","last_became_peered":"2023-11-08T19:27:08.187322+0000","last_unstale":"2023-11-08T19:31:11.667947+0000","last_undegraded":"2023-11-08T19:31:11.667947+0000","last_fullsized":"2023-11-08T19:31:11.667947+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:33:07.647519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":76,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.002194+0000","last_change":"2023-11-08T19:26:06.394075+0000","last_active":"2023-11-08T19:31:11.002194+0000","last_peered":"2023-11-08T19:31:11.002194+0000","last_clean":"2023-11-08T19:31:11.002194+0000","last_became_active":"2023-11-08T19:26:04.606967+0000","last_became_peered":"2023-11-08T19:26:04.606967+0000","last_unstale":"2023-11-08T19:31:11.002194+0000","last_undegraded":"2023-11-08T19:31:11.002194+0000","last_fullsized":"2023-11-08T19:31:11.002194+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:19:56.286835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040971999999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":60,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.008770+0000","last_change":"2023-11-08T19:27:09.180722+0000","last_active":"2023-11-08T19:31:13.008770+0000","last_peered":"2023-11-08T19:31:13.008770+0000","last_clean":"2023-11-08T19:31:13.008770+0000","last_became_active":"2023-11-08T19:27:07.186977+0000","last_became_peered":"2023-11-08T19:27:07.186977+0000","last_unstale":"2023-11-08T19:31:13.008770+0000","last_undegraded":"2023-11-08T19:31:13.008770+0000","last_fullsized":"2023-11-08T19:31:13.008770+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:17:59.966463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057675100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":46,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.002184+0000","last_change":"2023-11-08T19:27:08.193963+0000","last_active":"2023-11-08T19:31:11.002184+0000","last_peered":"2023-11-08T19:31:11.002184+0000","last_clean":"2023-11-08T19:31:11.002184+0000","last_became_active":"2023-11-08T19:27:08.193098+0000","last_became_peered":"2023-11-08T19:27:08.193098+0000","last_unstale":"2023-11-08T19:31:11.002184+0000","last_undegraded":"2023-11-08T19:31:11.002184+0000","last_fullsized":"2023-11-08T19:31:11.002184+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:41:09.827370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":76,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.668693+0000","last_change":"2023-11-08T19:26:06.265686+0000","last_active":"2023-11-08T19:31:11.668693+0000","last_peered":"2023-11-08T19:31:11.668693+0000","last_clean":"2023-11-08T19:31:11.668693+0000","last_became_active":"2023-11-08T19:26:04.602111+0000","last_became_peered":"2023-11-08T19:26:04.602111+0000","last_unstale":"2023-11-08T19:31:11.668693+0000","last_undegraded":"2023-11-08T19:31:11.668693+0000","last_fullsized":"2023-11-08T19:31:11.668693+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:40:11.164951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056197899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":60,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.008458+0000","last_change":"2023-11-08T19:27:09.181333+0000","last_active":"2023-11-08T19:31:13.008458+0000","last_peered":"2023-11-08T19:31:13.008458+0000","last_clean":"2023-11-08T19:31:13.008458+0000","last_became_active":"2023-11-08T19:27:07.188657+0000","last_became_peered":"2023-11-08T19:27:07.188657+0000","last_unstale":"2023-11-08T19:31:13.008458+0000","last_undegraded":"2023-11-08T19:31:13.008458+0000","last_fullsized":"2023-11-08T19:31:13.008458+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:25:56.788615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001227393,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":53,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.009645+0000","last_change":"2023-11-08T19:27:08.188533+0000","last_active":"2023-11-08T19:31:13.009645+0000","last_peered":"2023-11-08T19:31:13.009645+0000","last_clean":"2023-11-08T19:31:13.009645+0000","last_became_active":"2023-11-08T19:27:08.188240+0000","last_became_peered":"2023-11-08T19:27:08.188240+0000","last_unstale":"2023-11-08T19:31:13.009645+0000","last_undegraded":"2023-11-08T19:31:13.009645+0000","last_fullsized":"2023-11-08T19:31:13.009645+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:55:37.653521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":53,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.010070+0000","last_change":"2023-11-08T19:27:08.188458+0000","last_active":"2023-11-08T19:31:13.010070+0000","last_peered":"2023-11-08T19:31:13.010070+0000","last_clean":"2023-11-08T19:31:13.010070+0000","last_became_active":"2023-11-08T19:27:08.187880+0000","last_became_peered":"2023-11-08T19:27:08.187880+0000","last_unstale":"2023-11-08T19:31:13.010070+0000","last_undegraded":"2023-11-08T19:31:13.010070+0000","last_fullsized":"2023-11-08T19:31:13.010070+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:25:28.618214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":77,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.444854+0000","last_change":"2023-11-08T19:26:05.606898+0000","last_active":"2023-11-08T19:31:11.444854+0000","last_peered":"2023-11-08T19:31:11.444854+0000","last_clean":"2023-11-08T19:31:11.444854+0000","last_became_active":"2023-11-08T19:26:04.603955+0000","last_became_peered":"2023-11-08T19:26:04.603955+0000","last_unstale":"2023-11-08T19:31:11.444854+0000","last_undegraded":"2023-11-08T19:31:11.444854+0000","last_fullsized":"2023-11-08T19:31:11.444854+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:19:11.589559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036473499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":53,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.444503+0000","last_change":"2023-11-08T19:27:09.178770+0000","last_active":"2023-11-08T19:31:11.444503+0000","last_peered":"2023-11-08T19:31:11.444503+0000","last_clean":"2023-11-08T19:31:11.444503+0000","last_became_active":"2023-11-08T19:27:07.194992+0000","last_became_peered":"2023-11-08T19:27:07.194992+0000","last_unstale":"2023-11-08T19:31:11.444503+0000","last_undegraded":"2023-11-08T19:31:11.444503+0000","last_fullsized":"2023-11-08T19:31:11.444503+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:35:34.026910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000770285,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":46,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.667108+0000","last_change":"2023-11-08T19:27:08.193953+0000","last_active":"2023-11-08T19:31:11.667108+0000","last_peered":"2023-11-08T19:31:11.667108+0000","last_clean":"2023-11-08T19:31:11.667108+0000","last_became_active":"2023-11-08T19:27:08.193653+0000","last_became_peered":"2023-11-08T19:27:08.193653+0000","last_unstale":"2023-11-08T19:31:11.667108+0000","last_undegraded":"2023-11-08T19:31:11.667108+0000","last_fullsized":"2023-11-08T19:31:11.667108+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:35:30.073885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":82,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.667882+0000","last_change":"2023-11-08T19:26:06.424464+0000","last_active":"2023-11-08T19:31:11.667882+0000","last_peered":"2023-11-08T19:31:11.667882+0000","last_clean":"2023-11-08T19:31:11.667882+0000","last_became_active":"2023-11-08T19:26:04.603144+0000","last_became_peered":"2023-11-08T19:26:04.603144+0000","last_unstale":"2023-11-08T19:31:11.667882+0000","last_undegraded":"2023-11-08T19:31:11.667882+0000","last_fullsized":"2023-11-08T19:31:11.667882+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:42:55.269365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054090799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":60,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.009866+0000","last_change":"2023-11-08T19:27:09.180479+0000","last_active":"2023-11-08T19:31:13.009866+0000","last_peered":"2023-11-08T19:31:13.009866+0000","last_clean":"2023-11-08T19:31:13.009866+0000","last_became_active":"2023-11-08T19:27:07.190438+0000","last_became_peered":"2023-11-08T19:27:07.190438+0000","last_unstale":"2023-11-08T19:31:13.009866+0000","last_undegraded":"2023-11-08T19:31:13.009866+0000","last_fullsized":"2023-11-08T19:31:13.009866+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:25:14.487524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050893300000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":46,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.667409+0000","last_change":"2023-11-08T19:27:08.190530+0000","last_active":"2023-11-08T19:31:11.667409+0000","last_peered":"2023-11-08T19:31:11.667409+0000","last_clean":"2023-11-08T19:31:11.667409+0000","last_became_active":"2023-11-08T19:27:08.190203+0000","last_became_peered":"2023-11-08T19:27:08.190203+0000","last_unstale":"2023-11-08T19:31:11.667409+0000","last_undegraded":"2023-11-08T19:31:11.667409+0000","last_fullsized":"2023-11-08T19:31:11.667409+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:26:32.777602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":78,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.668651+0000","last_change":"2023-11-08T19:26:06.541623+0000","last_active":"2023-11-08T19:31:11.668651+0000","last_peered":"2023-11-08T19:31:11.668651+0000","last_clean":"2023-11-08T19:31:11.668651+0000","last_became_active":"2023-11-08T19:26:04.601961+0000","last_became_peered":"2023-11-08T19:26:04.601961+0000","last_unstale":"2023-11-08T19:31:11.668651+0000","last_undegraded":"2023-11-08T19:31:11.668651+0000","last_fullsized":"2023-11-08T19:31:11.668651+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:11:26.508634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000466386,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":54,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.666591+0000","last_change":"2023-11-08T19:27:09.431231+0000","last_active":"2023-11-08T19:31:11.666591+0000","last_peered":"2023-11-08T19:31:11.666591+0000","last_clean":"2023-11-08T19:31:11.666591+0000","last_became_active":"2023-11-08T19:27:07.191583+0000","last_became_peered":"2023-11-08T19:27:07.191583+0000","last_unstale":"2023-11-08T19:31:11.666591+0000","last_undegraded":"2023-11-08T19:31:11.666591+0000","last_fullsized":"2023-11-08T19:31:11.666591+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:58:48.357266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093695200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":46,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.666559+0000","last_change":"2023-11-08T19:27:08.187900+0000","last_active":"2023-11-08T19:31:11.666559+0000","last_peered":"2023-11-08T19:31:11.666559+0000","last_clean":"2023-11-08T19:31:11.666559+0000","last_became_active":"2023-11-08T19:27:08.187626+0000","last_became_peered":"2023-11-08T19:27:08.187626+0000","last_unstale":"2023-11-08T19:31:11.666559+0000","last_undegraded":"2023-11-08T19:31:11.666559+0000","last_fullsized":"2023-11-08T19:31:11.666559+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:00:55.878397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":89,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.009597+0000","last_change":"2023-11-08T19:26:05.608980+0000","last_active":"2023-11-08T19:31:13.009597+0000","last_peered":"2023-11-08T19:31:13.009597+0000","last_clean":"2023-11-08T19:31:13.009597+0000","last_became_active":"2023-11-08T19:26:04.606386+0000","last_became_peered":"2023-11-08T19:26:04.606386+0000","last_unstale":"2023-11-08T19:31:13.009597+0000","last_undegraded":"2023-11-08T19:31:13.009597+0000","last_fullsized":"2023-11-08T19:31:13.009597+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:41:50.771457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037950599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":53,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.444816+0000","last_change":"2023-11-08T19:27:09.179037+0000","last_active":"2023-11-08T19:31:11.444816+0000","last_peered":"2023-11-08T19:31:11.444816+0000","last_clean":"2023-11-08T19:31:11.444816+0000","last_became_active":"2023-11-08T19:27:07.195033+0000","last_became_peered":"2023-11-08T19:27:07.195033+0000","last_unstale":"2023-11-08T19:31:11.444816+0000","last_undegraded":"2023-11-08T19:31:11.444816+0000","last_fullsized":"2023-11-08T19:31:11.444816+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:19:08.253483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041824800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":46,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.002110+0000","last_change":"2023-11-08T19:27:08.193761+0000","last_active":"2023-11-08T19:31:11.002110+0000","last_peered":"2023-11-08T19:31:11.002110+0000","last_clean":"2023-11-08T19:31:11.002110+0000","last_became_active":"2023-11-08T19:27:08.193017+0000","last_became_peered":"2023-11-08T19:27:08.193017+0000","last_unstale":"2023-11-08T19:31:11.002110+0000","last_undegraded":"2023-11-08T19:31:11.002110+0000","last_fullsized":"2023-11-08T19:31:11.002110+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:18:00.971466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":82,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.667284+0000","last_change":"2023-11-08T19:26:06.424073+0000","last_active":"2023-11-08T19:31:11.667284+0000","last_peered":"2023-11-08T19:31:11.667284+0000","last_clean":"2023-11-08T19:31:11.667284+0000","last_became_active":"2023-11-08T19:26:04.599046+0000","last_became_peered":"2023-11-08T19:26:04.599046+0000","last_unstale":"2023-11-08T19:31:11.667284+0000","last_undegraded":"2023-11-08T19:31:11.667284+0000","last_fullsized":"2023-11-08T19:31:11.667284+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:04:43.562566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035092199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":61,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.009142+0000","last_change":"2023-11-08T19:27:09.181716+0000","last_active":"2023-11-08T19:31:13.009142+0000","last_peered":"2023-11-08T19:31:13.009142+0000","last_clean":"2023-11-08T19:31:13.009142+0000","last_became_active":"2023-11-08T19:27:07.187077+0000","last_became_peered":"2023-11-08T19:27:07.187077+0000","last_unstale":"2023-11-08T19:31:13.009142+0000","last_undegraded":"2023-11-08T19:31:13.009142+0000","last_fullsized":"2023-11-08T19:31:13.009142+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:51:29.247104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056673599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":46,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.667869+0000","last_change":"2023-11-08T19:27:08.188143+0000","last_active":"2023-11-08T19:31:11.667869+0000","last_peered":"2023-11-08T19:31:11.667869+0000","last_clean":"2023-11-08T19:31:11.667869+0000","last_became_active":"2023-11-08T19:27:08.187714+0000","last_became_peered":"2023-11-08T19:27:08.187714+0000","last_unstale":"2023-11-08T19:31:11.667869+0000","last_undegraded":"2023-11-08T19:31:11.667869+0000","last_fullsized":"2023-11-08T19:31:11.667869+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:02:10.863403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":76,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.002535+0000","last_change":"2023-11-08T19:26:06.394320+0000","last_active":"2023-11-08T19:31:11.002535+0000","last_peered":"2023-11-08T19:31:11.002535+0000","last_clean":"2023-11-08T19:31:11.002535+0000","last_became_active":"2023-11-08T19:26:04.603212+0000","last_became_peered":"2023-11-08T19:26:04.603212+0000","last_unstale":"2023-11-08T19:31:11.002535+0000","last_undegraded":"2023-11-08T19:31:11.002535+0000","last_fullsized":"2023-11-08T19:31:11.002535+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:25:29.947079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054695399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":60,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.009834+0000","last_change":"2023-11-08T19:27:09.181613+0000","last_active":"2023-11-08T19:31:13.009834+0000","last_peered":"2023-11-08T19:31:13.009834+0000","last_clean":"2023-11-08T19:31:13.009834+0000","last_became_active":"2023-11-08T19:27:07.190897+0000","last_became_peered":"2023-11-08T19:27:07.190897+0000","last_unstale":"2023-11-08T19:31:13.009834+0000","last_undegraded":"2023-11-08T19:31:13.009834+0000","last_fullsized":"2023-11-08T19:31:13.009834+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:27:57.388125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011172459999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":46,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.668020+0000","last_change":"2023-11-08T19:27:08.188668+0000","last_active":"2023-11-08T19:31:11.668020+0000","last_peered":"2023-11-08T19:31:11.668020+0000","last_clean":"2023-11-08T19:31:11.668020+0000","last_became_active":"2023-11-08T19:27:08.188441+0000","last_became_peered":"2023-11-08T19:27:08.188441+0000","last_unstale":"2023-11-08T19:31:11.668020+0000","last_undegraded":"2023-11-08T19:31:11.668020+0000","last_fullsized":"2023-11-08T19:31:11.668020+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:45:44.379160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":82,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.667134+0000","last_change":"2023-11-08T19:26:06.423754+0000","last_active":"2023-11-08T19:31:11.667134+0000","last_peered":"2023-11-08T19:31:11.667134+0000","last_clean":"2023-11-08T19:31:11.667134+0000","last_became_active":"2023-11-08T19:26:04.599322+0000","last_became_peered":"2023-11-08T19:26:04.599322+0000","last_unstale":"2023-11-08T19:31:11.667134+0000","last_undegraded":"2023-11-08T19:31:11.667134+0000","last_fullsized":"2023-11-08T19:31:11.667134+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:13:08.689128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042670000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":60,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.010577+0000","last_change":"2023-11-08T19:27:09.181656+0000","last_active":"2023-11-08T19:31:13.010577+0000","last_peered":"2023-11-08T19:31:13.010577+0000","last_clean":"2023-11-08T19:31:13.010577+0000","last_became_active":"2023-11-08T19:27:07.187337+0000","last_became_peered":"2023-11-08T19:27:07.187337+0000","last_unstale":"2023-11-08T19:31:13.010577+0000","last_undegraded":"2023-11-08T19:31:13.010577+0000","last_fullsized":"2023-11-08T19:31:13.010577+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:14:17.605057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000346967,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":46,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.002332+0000","last_change":"2023-11-08T19:27:08.194065+0000","last_active":"2023-11-08T19:31:11.002332+0000","last_peered":"2023-11-08T19:31:11.002332+0000","last_clean":"2023-11-08T19:31:11.002332+0000","last_became_active":"2023-11-08T19:27:08.193532+0000","last_became_peered":"2023-11-08T19:27:08.193532+0000","last_unstale":"2023-11-08T19:31:11.002332+0000","last_undegraded":"2023-11-08T19:31:11.002332+0000","last_fullsized":"2023-11-08T19:31:11.002332+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:27:58.393129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":76,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.002769+0000","last_change":"2023-11-08T19:26:06.394488+0000","last_active":"2023-11-08T19:31:11.002769+0000","last_peered":"2023-11-08T19:31:11.002769+0000","last_clean":"2023-11-08T19:31:11.002769+0000","last_became_active":"2023-11-08T19:26:04.606805+0000","last_became_peered":"2023-11-08T19:26:04.606805+0000","last_unstale":"2023-11-08T19:31:11.002769+0000","last_undegraded":"2023-11-08T19:31:11.002769+0000","last_fullsized":"2023-11-08T19:31:11.002769+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:25:20.857133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045317000000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":53,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.668089+0000","last_change":"2023-11-08T19:27:09.211647+0000","last_active":"2023-11-08T19:31:11.668089+0000","last_peered":"2023-11-08T19:31:11.668089+0000","last_clean":"2023-11-08T19:31:11.668089+0000","last_became_active":"2023-11-08T19:27:07.190309+0000","last_became_peered":"2023-11-08T19:27:07.190309+0000","last_unstale":"2023-11-08T19:31:11.668089+0000","last_undegraded":"2023-11-08T19:31:11.668089+0000","last_fullsized":"2023-11-08T19:31:11.668089+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:56:04.292154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047615199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":46,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.002763+0000","last_change":"2023-11-08T19:27:08.194884+0000","last_active":"2023-11-08T19:31:11.002763+0000","last_peered":"2023-11-08T19:31:11.002763+0000","last_clean":"2023-11-08T19:31:11.002763+0000","last_became_active":"2023-11-08T19:27:08.193241+0000","last_became_peered":"2023-11-08T19:27:08.193241+0000","last_unstale":"2023-11-08T19:31:11.002763+0000","last_undegraded":"2023-11-08T19:31:11.002763+0000","last_fullsized":"2023-11-08T19:31:11.002763+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:46:31.457682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":76,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.668753+0000","last_change":"2023-11-08T19:26:06.265907+0000","last_active":"2023-11-08T19:31:11.668753+0000","last_peered":"2023-11-08T19:31:11.668753+0000","last_clean":"2023-11-08T19:31:11.668753+0000","last_became_active":"2023-11-08T19:26:04.604772+0000","last_became_peered":"2023-11-08T19:26:04.604772+0000","last_unstale":"2023-11-08T19:31:11.668753+0000","last_undegraded":"2023-11-08T19:31:11.668753+0000","last_fullsized":"2023-11-08T19:31:11.668753+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:44:16.139794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066875300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":53,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.667612+0000","last_change":"2023-11-08T19:27:09.282699+0000","last_active":"2023-11-08T19:31:11.667612+0000","last_peered":"2023-11-08T19:31:11.667612+0000","last_clean":"2023-11-08T19:31:11.667612+0000","last_became_active":"2023-11-08T19:27:07.191212+0000","last_became_peered":"2023-11-08T19:27:07.191212+0000","last_unstale":"2023-11-08T19:31:11.667612+0000","last_undegraded":"2023-11-08T19:31:11.667612+0000","last_fullsized":"2023-11-08T19:31:11.667612+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:35:14.715110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046640800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":47,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.668470+0000","last_change":"2023-11-08T19:27:08.187644+0000","last_active":"2023-11-08T19:31:11.668470+0000","last_peered":"2023-11-08T19:31:11.668470+0000","last_clean":"2023-11-08T19:31:11.668470+0000","last_became_active":"2023-11-08T19:27:08.187376+0000","last_became_peered":"2023-11-08T19:27:08.187376+0000","last_unstale":"2023-11-08T19:31:11.668470+0000","last_undegraded":"2023-11-08T19:31:11.668470+0000","last_fullsized":"2023-11-08T19:31:11.668470+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:22:40.300185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":77,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.444647+0000","last_change":"2023-11-08T19:26:05.606946+0000","last_active":"2023-11-08T19:31:11.444647+0000","last_peered":"2023-11-08T19:31:11.444647+0000","last_clean":"2023-11-08T19:31:11.444647+0000","last_became_active":"2023-11-08T19:26:04.598775+0000","last_became_peered":"2023-11-08T19:26:04.598775+0000","last_unstale":"2023-11-08T19:31:11.444647+0000","last_undegraded":"2023-11-08T19:31:11.444647+0000","last_fullsized":"2023-11-08T19:31:11.444647+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:20:56.898927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049870399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":53,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.002400+0000","last_change":"2023-11-08T19:27:09.331646+0000","last_active":"2023-11-08T19:31:11.002400+0000","last_peered":"2023-11-08T19:31:11.002400+0000","last_clean":"2023-11-08T19:31:11.002400+0000","last_became_active":"2023-11-08T19:27:07.190619+0000","last_became_peered":"2023-11-08T19:27:07.190619+0000","last_unstale":"2023-11-08T19:31:11.002400+0000","last_undegraded":"2023-11-08T19:31:11.002400+0000","last_fullsized":"2023-11-08T19:31:11.002400+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:42:47.371458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065918400000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":53,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.011570+0000","last_change":"2023-11-08T19:27:08.183798+0000","last_active":"2023-11-08T19:31:13.011570+0000","last_peered":"2023-11-08T19:31:13.011570+0000","last_clean":"2023-11-08T19:31:13.011570+0000","last_became_active":"2023-11-08T19:27:08.183537+0000","last_became_peered":"2023-11-08T19:27:08.183537+0000","last_unstale":"2023-11-08T19:31:13.011570+0000","last_undegraded":"2023-11-08T19:31:13.011570+0000","last_fullsized":"2023-11-08T19:31:13.011570+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:50:39.026265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":89,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.011263+0000","last_change":"2023-11-08T19:26:05.610012+0000","last_active":"2023-11-08T19:31:13.011263+0000","last_peered":"2023-11-08T19:31:13.011263+0000","last_clean":"2023-11-08T19:31:13.011263+0000","last_became_active":"2023-11-08T19:26:04.603209+0000","last_became_peered":"2023-11-08T19:26:04.603209+0000","last_unstale":"2023-11-08T19:31:13.011263+0000","last_undegraded":"2023-11-08T19:31:13.011263+0000","last_fullsized":"2023-11-08T19:31:13.011263+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:35:02.306997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053335100000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":53,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.444512+0000","last_change":"2023-11-08T19:27:09.178405+0000","last_active":"2023-11-08T19:31:11.444512+0000","last_peered":"2023-11-08T19:31:11.444512+0000","last_clean":"2023-11-08T19:31:11.444512+0000","last_became_active":"2023-11-08T19:27:07.188169+0000","last_became_peered":"2023-11-08T19:27:07.188169+0000","last_unstale":"2023-11-08T19:31:11.444512+0000","last_undegraded":"2023-11-08T19:31:11.444512+0000","last_fullsized":"2023-11-08T19:31:11.444512+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:39:25.855467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056209700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":46,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.002018+0000","last_change":"2023-11-08T19:27:08.193508+0000","last_active":"2023-11-08T19:31:11.002018+0000","last_peered":"2023-11-08T19:31:11.002018+0000","last_clean":"2023-11-08T19:31:11.002018+0000","last_became_active":"2023-11-08T19:27:08.193037+0000","last_became_peered":"2023-11-08T19:27:08.193037+0000","last_unstale":"2023-11-08T19:31:11.002018+0000","last_undegraded":"2023-11-08T19:31:11.002018+0000","last_fullsized":"2023-11-08T19:31:11.002018+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:29:50.067887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":76,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.667822+0000","last_change":"2023-11-08T19:26:06.265318+0000","last_active":"2023-11-08T19:31:11.667822+0000","last_peered":"2023-11-08T19:31:11.667822+0000","last_clean":"2023-11-08T19:31:11.667822+0000","last_became_active":"2023-11-08T19:26:04.603409+0000","last_became_peered":"2023-11-08T19:26:04.603409+0000","last_unstale":"2023-11-08T19:31:11.667822+0000","last_undegraded":"2023-11-08T19:31:11.667822+0000","last_fullsized":"2023-11-08T19:31:11.667822+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:27:24.933111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065684300000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":60,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.010096+0000","last_change":"2023-11-08T19:27:09.182282+0000","last_active":"2023-11-08T19:31:13.010096+0000","last_peered":"2023-11-08T19:31:13.010096+0000","last_clean":"2023-11-08T19:31:13.010096+0000","last_became_active":"2023-11-08T19:27:07.186944+0000","last_became_peered":"2023-11-08T19:27:07.186944+0000","last_unstale":"2023-11-08T19:31:13.010096+0000","last_undegraded":"2023-11-08T19:31:13.010096+0000","last_fullsized":"2023-11-08T19:31:13.010096+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:45:21.079011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00028665599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":46,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.667282+0000","last_change":"2023-11-08T19:27:08.193990+0000","last_active":"2023-11-08T19:31:11.667282+0000","last_peered":"2023-11-08T19:31:11.667282+0000","last_clean":"2023-11-08T19:31:11.667282+0000","last_became_active":"2023-11-08T19:27:08.193624+0000","last_became_peered":"2023-11-08T19:27:08.193624+0000","last_unstale":"2023-11-08T19:31:11.667282+0000","last_undegraded":"2023-11-08T19:31:11.667282+0000","last_fullsized":"2023-11-08T19:31:11.667282+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:46:41.321120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":82,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.667129+0000","last_change":"2023-11-08T19:26:06.423863+0000","last_active":"2023-11-08T19:31:11.667129+0000","last_peered":"2023-11-08T19:31:11.667129+0000","last_clean":"2023-11-08T19:31:11.667129+0000","last_became_active":"2023-11-08T19:26:04.608639+0000","last_became_peered":"2023-11-08T19:26:04.608639+0000","last_unstale":"2023-11-08T19:31:11.667129+0000","last_undegraded":"2023-11-08T19:31:11.667129+0000","last_fullsized":"2023-11-08T19:31:11.667129+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:04:43.631102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044384999999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":53,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.668714+0000","last_change":"2023-11-08T19:27:09.283232+0000","last_active":"2023-11-08T19:31:11.668714+0000","last_peered":"2023-11-08T19:31:11.668714+0000","last_clean":"2023-11-08T19:31:11.668714+0000","last_became_active":"2023-11-08T19:27:07.190931+0000","last_became_peered":"2023-11-08T19:27:07.190931+0000","last_unstale":"2023-11-08T19:31:11.668714+0000","last_undegraded":"2023-11-08T19:31:11.668714+0000","last_fullsized":"2023-11-08T19:31:11.668714+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:43:28.225967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034146799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":50,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.667555+0000","last_change":"2023-11-08T19:27:08.187252+0000","last_active":"2023-11-08T19:31:11.667555+0000","last_peered":"2023-11-08T19:31:11.667555+0000","last_clean":"2023-11-08T19:31:11.667555+0000","last_became_active":"2023-11-08T19:27:08.186952+0000","last_became_peered":"2023-11-08T19:27:08.186952+0000","last_unstale":"2023-11-08T19:31:11.667555+0000","last_undegraded":"2023-11-08T19:31:11.667555+0000","last_fullsized":"2023-11-08T19:31:11.667555+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:12:52.022374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":76,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.002192+0000","last_change":"2023-11-08T19:26:06.394081+0000","last_active":"2023-11-08T19:31:11.002192+0000","last_peered":"2023-11-08T19:31:11.002192+0000","last_clean":"2023-11-08T19:31:11.002192+0000","last_became_active":"2023-11-08T19:26:04.602137+0000","last_became_peered":"2023-11-08T19:26:04.602137+0000","last_unstale":"2023-11-08T19:31:11.002192+0000","last_undegraded":"2023-11-08T19:31:11.002192+0000","last_fullsized":"2023-11-08T19:31:11.002192+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:05:46.951663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056998200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":53,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.444723+0000","last_change":"2023-11-08T19:27:09.178816+0000","last_active":"2023-11-08T19:31:11.444723+0000","last_peered":"2023-11-08T19:31:11.444723+0000","last_clean":"2023-11-08T19:31:11.444723+0000","last_became_active":"2023-11-08T19:27:07.192001+0000","last_became_peered":"2023-11-08T19:27:07.192001+0000","last_unstale":"2023-11-08T19:31:11.444723+0000","last_undegraded":"2023-11-08T19:31:11.444723+0000","last_fullsized":"2023-11-08T19:31:11.444723+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:04:12.481212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045359000000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":46,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.666798+0000","last_change":"2023-11-08T19:27:08.193836+0000","last_active":"2023-11-08T19:31:11.666798+0000","last_peered":"2023-11-08T19:31:11.666798+0000","last_clean":"2023-11-08T19:31:11.666798+0000","last_became_active":"2023-11-08T19:27:08.193458+0000","last_became_peered":"2023-11-08T19:27:08.193458+0000","last_unstale":"2023-11-08T19:31:11.666798+0000","last_undegraded":"2023-11-08T19:31:11.666798+0000","last_fullsized":"2023-11-08T19:31:11.666798+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:14:18.610061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":78,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.667284+0000","last_change":"2023-11-08T19:26:06.541338+0000","last_active":"2023-11-08T19:31:11.667284+0000","last_peered":"2023-11-08T19:31:11.667284+0000","last_clean":"2023-11-08T19:31:11.667284+0000","last_became_active":"2023-11-08T19:26:04.602183+0000","last_became_peered":"2023-11-08T19:26:04.602183+0000","last_unstale":"2023-11-08T19:31:11.667284+0000","last_undegraded":"2023-11-08T19:31:11.667284+0000","last_fullsized":"2023-11-08T19:31:11.667284+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:06:47.446686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052029099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":60,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.010485+0000","last_change":"2023-11-08T19:27:09.181622+0000","last_active":"2023-11-08T19:31:13.010485+0000","last_peered":"2023-11-08T19:31:13.010485+0000","last_clean":"2023-11-08T19:31:13.010485+0000","last_became_active":"2023-11-08T19:27:07.184507+0000","last_became_peered":"2023-11-08T19:27:07.184507+0000","last_unstale":"2023-11-08T19:31:13.010485+0000","last_undegraded":"2023-11-08T19:31:13.010485+0000","last_fullsized":"2023-11-08T19:31:13.010485+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:58:48.357266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051924700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":46,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.668291+0000","last_change":"2023-11-08T19:27:08.194263+0000","last_active":"2023-11-08T19:31:11.668291+0000","last_peered":"2023-11-08T19:31:11.668291+0000","last_clean":"2023-11-08T19:31:11.668291+0000","last_became_active":"2023-11-08T19:27:08.193938+0000","last_became_peered":"2023-11-08T19:27:08.193938+0000","last_unstale":"2023-11-08T19:31:11.668291+0000","last_undegraded":"2023-11-08T19:31:11.668291+0000","last_fullsized":"2023-11-08T19:31:11.668291+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:40:49.009717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":76,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.668798+0000","last_change":"2023-11-08T19:26:06.265690+0000","last_active":"2023-11-08T19:31:11.668798+0000","last_peered":"2023-11-08T19:31:11.668798+0000","last_clean":"2023-11-08T19:31:11.668798+0000","last_became_active":"2023-11-08T19:26:04.603397+0000","last_became_peered":"2023-11-08T19:26:04.603397+0000","last_unstale":"2023-11-08T19:31:11.668798+0000","last_undegraded":"2023-11-08T19:31:11.668798+0000","last_fullsized":"2023-11-08T19:31:11.668798+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:50:43.827570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053361499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":53,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.668234+0000","last_change":"2023-11-08T19:27:09.282750+0000","last_active":"2023-11-08T19:31:11.668234+0000","last_peered":"2023-11-08T19:31:11.668234+0000","last_clean":"2023-11-08T19:31:11.668234+0000","last_became_active":"2023-11-08T19:27:07.189943+0000","last_became_peered":"2023-11-08T19:27:07.189943+0000","last_unstale":"2023-11-08T19:31:11.668234+0000","last_undegraded":"2023-11-08T19:31:11.668234+0000","last_fullsized":"2023-11-08T19:31:11.668234+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:31:16.081058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000398725,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":53,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.009718+0000","last_change":"2023-11-08T19:27:08.188147+0000","last_active":"2023-11-08T19:31:13.009718+0000","last_peered":"2023-11-08T19:31:13.009718+0000","last_clean":"2023-11-08T19:31:13.009718+0000","last_became_active":"2023-11-08T19:27:08.187487+0000","last_became_peered":"2023-11-08T19:27:08.187487+0000","last_unstale":"2023-11-08T19:31:13.009718+0000","last_undegraded":"2023-11-08T19:31:13.009718+0000","last_fullsized":"2023-11-08T19:31:13.009718+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:22:40.300185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":82,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.668250+0000","last_change":"2023-11-08T19:26:06.425133+0000","last_active":"2023-11-08T19:31:11.668250+0000","last_peered":"2023-11-08T19:31:11.668250+0000","last_clean":"2023-11-08T19:31:11.668250+0000","last_became_active":"2023-11-08T19:26:04.601103+0000","last_became_peered":"2023-11-08T19:26:04.601103+0000","last_unstale":"2023-11-08T19:31:11.668250+0000","last_undegraded":"2023-11-08T19:31:11.668250+0000","last_fullsized":"2023-11-08T19:31:11.668250+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:31:14.042561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000229244,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":60,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.009084+0000","last_change":"2023-11-08T19:27:09.179852+0000","last_active":"2023-11-08T19:31:13.009084+0000","last_peered":"2023-11-08T19:31:13.009084+0000","last_clean":"2023-11-08T19:31:13.009084+0000","last_became_active":"2023-11-08T19:27:07.191926+0000","last_became_peered":"2023-11-08T19:27:07.191926+0000","last_unstale":"2023-11-08T19:31:13.009084+0000","last_undegraded":"2023-11-08T19:31:13.009084+0000","last_fullsized":"2023-11-08T19:31:13.009084+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:20:38.080129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055457100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":53,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.010318+0000","last_change":"2023-11-08T19:27:08.186054+0000","last_active":"2023-11-08T19:31:13.010318+0000","last_peered":"2023-11-08T19:31:13.010318+0000","last_clean":"2023-11-08T19:31:13.010318+0000","last_became_active":"2023-11-08T19:27:08.185581+0000","last_became_peered":"2023-11-08T19:27:08.185581+0000","last_unstale":"2023-11-08T19:31:13.010318+0000","last_undegraded":"2023-11-08T19:31:13.010318+0000","last_fullsized":"2023-11-08T19:31:13.010318+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:00:55.878397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":89,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.008858+0000","last_change":"2023-11-08T19:26:05.609625+0000","last_active":"2023-11-08T19:31:13.008858+0000","last_peered":"2023-11-08T19:31:13.008858+0000","last_clean":"2023-11-08T19:31:13.008858+0000","last_became_active":"2023-11-08T19:26:04.606312+0000","last_became_peered":"2023-11-08T19:26:04.606312+0000","last_unstale":"2023-11-08T19:31:13.008858+0000","last_undegraded":"2023-11-08T19:31:13.008858+0000","last_fullsized":"2023-11-08T19:31:13.008858+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:54:42.032901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000505935,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":53,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.444366+0000","last_change":"2023-11-08T19:27:09.177985+0000","last_active":"2023-11-08T19:31:11.444366+0000","last_peered":"2023-11-08T19:31:11.444366+0000","last_clean":"2023-11-08T19:31:11.444366+0000","last_became_active":"2023-11-08T19:27:07.193254+0000","last_became_peered":"2023-11-08T19:27:07.193254+0000","last_unstale":"2023-11-08T19:31:11.444366+0000","last_undegraded":"2023-11-08T19:31:11.444366+0000","last_fullsized":"2023-11-08T19:31:11.444366+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:23:38.662629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041830799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":53,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.011772+0000","last_change":"2023-11-08T19:27:08.188860+0000","last_active":"2023-11-08T19:31:13.011772+0000","last_peered":"2023-11-08T19:31:13.011772+0000","last_clean":"2023-11-08T19:31:13.011772+0000","last_became_active":"2023-11-08T19:27:08.188560+0000","last_became_peered":"2023-11-08T19:27:08.188560+0000","last_unstale":"2023-11-08T19:31:13.011772+0000","last_undegraded":"2023-11-08T19:31:13.011772+0000","last_fullsized":"2023-11-08T19:31:13.011772+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:18:20.969491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":77,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.667266+0000","last_change":"2023-11-08T19:26:06.297255+0000","last_active":"2023-11-08T19:31:11.667266+0000","last_peered":"2023-11-08T19:31:11.667266+0000","last_clean":"2023-11-08T19:31:11.667266+0000","last_became_active":"2023-11-08T19:26:04.603979+0000","last_became_peered":"2023-11-08T19:26:04.603979+0000","last_unstale":"2023-11-08T19:31:11.667266+0000","last_undegraded":"2023-11-08T19:31:11.667266+0000","last_fullsized":"2023-11-08T19:31:11.667266+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:40:11.164951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079032099999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":53,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-08T19:31:11.002043+0000","last_change":"2023-11-08T19:27:09.331990+0000","last_active":"2023-11-08T19:31:11.002043+0000","last_peered":"2023-11-08T19:31:11.002043+0000","last_clean":"2023-11-08T19:31:11.002043+0000","last_became_active":"2023-11-08T19:27:07.185478+0000","last_became_peered":"2023-11-08T19:27:07.185478+0000","last_unstale":"2023-11-08T19:31:11.002043+0000","last_undegraded":"2023-11-08T19:31:11.002043+0000","last_fullsized":"2023-11-08T19:31:11.002043+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:20:38.080129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010661850000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574930,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33036,"kb_used_data":1856,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338804,"statfs":{"total":96636764160,"available":96602935296,"internally_reserved":0,"allocated":1900544,"data_stored":1229475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:30:56 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.54400000000000004,"15min":0.51100000000000001},"min":{"1min":0.45300000000000001,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.71899999999999997,"5min":0.78600000000000003,"15min":1.0149999999999999},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.53500000000000003,"15min":0.52100000000000002},"min":{"1min":0.42099999999999999,"5min":0.35099999999999998,"15min":0.32300000000000001},"max":{"1min":0.79300000000000004,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.56000000000000005}]},{"osd":1,"last update":"Wed Nov 8 19:30:56 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.59799999999999998,"15min":0.57599999999999996},"min":{"1min":0.47699999999999998,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.73499999999999999,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.623,"5min":0.59599999999999997,"15min":0.56699999999999995},"min":{"1min":0.505,"5min":0.40600000000000003,"15min":0.33900000000000002},"max":{"1min":0.79500000000000004,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.45900000000000002}]},{"osd":2,"last update":"Wed Nov 8 19:30:56 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.54400000000000004,"15min":0.505},"min":{"1min":0.372,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.67600000000000005,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.57399999999999995,"15min":0.55600000000000005},"min":{"1min":0.45900000000000002,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.78400000000000003,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.40000000000000002}]},{"osd":3,"last update":"Wed Nov 8 19:30:56 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.57499999999999996,"15min":0.56399999999999995},"min":{"1min":0.42199999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.77300000000000002,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.498},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.63800000000000001,"15min":0.59299999999999997},"min":{"1min":0.47399999999999998,"5min":0.437,"15min":0.40100000000000002},"max":{"1min":0.81399999999999995,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.53900000000000003}]},{"osd":4,"last update":"Wed Nov 8 19:30:56 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.501,"15min":0.505},"min":{"1min":0.32400000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.753,"5min":0.753,"15min":0.78600000000000003},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.53400000000000003,"15min":0.53700000000000003},"min":{"1min":0.35199999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.71799999999999997,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.58099999999999996}]},{"osd":5,"last update":"Wed Nov 8 19:30:56 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.56699999999999995,"15min":0.57499999999999996},"min":{"1min":0.35099999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.77300000000000002,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.624},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.627,"15min":0.63700000000000001},"min":{"1min":0.377,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.83899999999999997,"5min":0.89900000000000002,"15min":0.94299999999999995},"last":0.60699999999999998}]},{"osd":6,"last update":"Wed Nov 8 19:30:56 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.57999999999999996,"15min":0.59699999999999998},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.97299999999999998},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.59699999999999998,"15min":0.60499999999999998},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.78900000000000003,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.52800000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574930,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32476,"kb_used_data":1292,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339364,"statfs":{"total":96636764160,"available":96603508736,"internally_reserved":0,"allocated":1323008,"data_stored":646397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:30:57 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.55900000000000005,"15min":1.1459999999999999},"min":{"1min":0.41999999999999998,"5min":0.33200000000000002,"15min":0.27000000000000002},"max":{"1min":0.84599999999999997,"5min":0.91700000000000004,"15min":21.359999999999999},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.56899999999999995,"15min":0.81599999999999995},"min":{"1min":0.39700000000000002,"5min":0.378,"15min":0.30399999999999999},"max":{"1min":0.68700000000000006,"5min":0.90300000000000002,"15min":10.329000000000001},"last":0.67200000000000004}]},{"osd":1,"last update":"Wed Nov 8 19:30:57 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.53800000000000003,"15min":0.59499999999999997},"min":{"1min":0.41199999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.81399999999999995,"5min":0.86299999999999999,"15min":3.4239999999999999},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.60499999999999998,"15min":0.63400000000000001},"min":{"1min":0.35899999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":2.9260000000000002,"5min":2.9260000000000002,"15min":3.4809999999999999},"last":0.65500000000000003}]},{"osd":2,"last update":"Wed Nov 8 19:30:57 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.58599999999999997,"15min":0.83399999999999996},"min":{"1min":0.51800000000000002,"5min":0.38500000000000001,"15min":0.34399999999999997},"max":{"1min":0.84199999999999997,"5min":0.96399999999999997,"15min":10.398},"last":0.623},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.56799999999999995,"15min":0.66200000000000003},"min":{"1min":0.41799999999999998,"5min":0.318,"15min":0.318},"max":{"1min":0.77500000000000002,"5min":0.89100000000000001,"15min":5.0170000000000003},"last":0.63500000000000001}]},{"osd":3,"last update":"Wed Nov 8 19:30:57 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.63100000000000001,"15min":0.66100000000000003},"min":{"1min":0.46200000000000002,"5min":0.34599999999999997,"15min":0.317},"max":{"1min":3.0089999999999999,"5min":3.0089999999999999,"15min":3.5710000000000002},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.59699999999999998,"15min":0.85999999999999999},"min":{"1min":0.39000000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.81100000000000005,"5min":0.93500000000000005,"15min":10.368},"last":0.75900000000000001}]},{"osd":4,"last update":"Wed Nov 8 19:30:57 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54400000000000004,"15min":0.52000000000000002},"min":{"1min":0.34699999999999998,"5min":0.33000000000000002,"15min":0.29199999999999998},"max":{"1min":0.79100000000000004,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.55900000000000005,"15min":0.55800000000000005},"min":{"1min":0.45000000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.73799999999999999,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.46700000000000003}]},{"osd":5,"last update":"Wed Nov 8 19:30:57 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.55000000000000004,"15min":0.53200000000000003},"min":{"1min":0.42699999999999999,"5min":0.34599999999999997,"15min":0.307},"max":{"1min":0.70499999999999996,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.60399999999999998,"15min":0.58599999999999997},"min":{"1min":0.41199999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.60099999999999998}]},{"osd":7,"last update":"Wed Nov 8 19:30:57 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.60999999999999999,"15min":0.58299999999999996},"min":{"1min":0.39700000000000002,"5min":0.39500000000000002,"15min":0.34699999999999998},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.60599999999999998,"15min":0.59199999999999997},"min":{"1min":0.46300000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.82199999999999995,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.74199999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574930,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32456,"kb_used_data":1276,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339384,"statfs":{"total":96636764160,"available":96603529216,"internally_reserved":0,"allocated":1306624,"data_stored":639107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:30:55 2023","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.35499999999999998,"15min":0.374},"min":{"1min":0.222,"5min":0.219,"15min":0.219},"max":{"1min":0.437,"5min":0.69699999999999995,"15min":1.389},"last":0.28299999999999997},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.36399999999999999,"15min":0.61699999999999999},"min":{"1min":0.255,"5min":0.23000000000000001,"15min":0.21199999999999999},"max":{"1min":0.52900000000000003,"5min":0.622,"15min":9.1370000000000005},"last":0.36399999999999999}]},{"osd":2,"last update":"Wed Nov 8 19:30:55 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.35899999999999999,"15min":0.36499999999999999},"min":{"1min":0.248,"5min":0.216,"15min":0.216},"max":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":1.2709999999999999},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.39200000000000002,"15min":0.41899999999999998},"min":{"1min":0.27700000000000002,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.53400000000000003,"5min":0.63600000000000001,"15min":1.425},"last":0.41799999999999998}]},{"osd":3,"last update":"Wed Nov 8 19:30:55 2023","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.40899999999999997,"15min":0.74399999999999999},"min":{"1min":0.29599999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.54600000000000004,"5min":0.63800000000000001,"15min":11.957000000000001},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.39100000000000001,"15min":0.91800000000000004},"min":{"1min":0.27900000000000003,"5min":0.252,"15min":0.252},"max":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":18.039000000000001},"last":0.25}]},{"osd":4,"last update":"Wed Nov 8 19:30:55 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59299999999999997,"15min":0.751},"min":{"1min":0.35199999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.81100000000000005,"5min":1.4370000000000001,"15min":6.181},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.60999999999999999,"15min":0.63},"min":{"1min":0.41399999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.83899999999999997,"5min":1.411,"15min":1.488},"last":0.57699999999999996}]},{"osd":5,"last update":"Wed Nov 8 19:30:55 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61199999999999999,"15min":0.86099999999999999},"min":{"1min":0.375,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.85399999999999998,"5min":1.421,"15min":9.2420000000000009},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60599999999999998,"15min":0.78900000000000003},"min":{"1min":0.42499999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.80100000000000005,"5min":1.2969999999999999,"15min":6.2110000000000003},"last":0.63}]},{"osd":6,"last update":"Wed Nov 8 19:30:55 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.63500000000000001,"15min":0.86899999999999999},"min":{"1min":0.40799999999999997,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.78700000000000003,"5min":1.4530000000000001,"15min":9.1880000000000006},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63500000000000001,"15min":0.65600000000000003},"min":{"1min":0.44500000000000001,"5min":0.38,"15min":0.38},"max":{"1min":0.86399999999999999,"5min":1.272,"15min":1.462},"last":0.65600000000000003}]},{"osd":7,"last update":"Wed Nov 8 19:30:55 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.625,"15min":0.78800000000000003},"min":{"1min":0.41499999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0309999999999999,"5min":1.3979999999999999,"15min":6.1479999999999997},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.64600000000000002,"15min":0.90000000000000002},"min":{"1min":0.51300000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.97499999999999998,"5min":1.353,"15min":9.2170000000000005},"last":0.64500000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574930,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33076,"kb_used_data":1896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338764,"statfs":{"total":96636764160,"available":96602894336,"internally_reserved":0,"allocated":1941504,"data_stored":1263942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Nov 8 19:31:01 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.33300000000000002,"15min":0.45900000000000002},"min":{"1min":0.20899999999999999,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.51000000000000001,"5min":0.55300000000000005,"15min":4.1200000000000001},"last":0.376},{"interface":"front","average":{"1min":0.439,"5min":0.371,"15min":0.91900000000000004},"min":{"1min":0.22800000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.62,"5min":0.62,"15min":16.940999999999999},"last":0.34300000000000003}]},{"osd":2,"last update":"Wed Nov 8 19:31:01 2023","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.35299999999999998,"15min":0.38700000000000001},"min":{"1min":0.25,"5min":0.22,"15min":0.22},"max":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":1.1899999999999999},"last":0.26700000000000002},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.39100000000000001,"15min":0.60499999999999998},"min":{"1min":0.32400000000000001,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":7.0209999999999999},"last":0.29099999999999998}]},{"osd":3,"last update":"Wed Nov 8 19:31:01 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.38400000000000001,"15min":0.51200000000000001},"min":{"1min":0.29899999999999999,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":4.0940000000000003},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.39500000000000002,"15min":0.433},"min":{"1min":0.32500000000000001,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.61899999999999999,"5min":0.62,"15min":1.2549999999999999},"last":0.312}]},{"osd":4,"last update":"Wed Nov 8 19:31:01 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.58899999999999997,"15min":0.71699999999999997},"min":{"1min":0.42799999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":4.0049999999999999},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.58399999999999996,"15min":0.623},"min":{"1min":0.442,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.746,"5min":0.79200000000000004,"15min":1.2270000000000001},"last":0.54000000000000004}]},{"osd":5,"last update":"Wed Nov 8 19:31:01 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.59699999999999998,"15min":0.72799999999999998},"min":{"1min":0.47199999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":4.3090000000000002},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.61199999999999999,"15min":0.72399999999999998},"min":{"1min":0.436,"5min":0.373,"15min":0.373},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":4.0709999999999997},"last":0.56999999999999995}]},{"osd":6,"last update":"Wed Nov 8 19:31:01 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.621,"15min":0.65800000000000003},"min":{"1min":0.40999999999999998,"5min":0.373,"15min":0.373},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":1.2849999999999999},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.64900000000000002,"15min":0.78500000000000003},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":4.2560000000000002},"last":0.64100000000000001}]},{"osd":7,"last update":"Wed Nov 8 19:31:01 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.625,"15min":0.74299999999999999},"min":{"1min":0.47099999999999997,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":4.2850000000000001},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.64700000000000002,"15min":0.77000000000000002},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.002,"5min":1.002,"15min":4.0460000000000003},"last":0.63}]}]},{"osd":2,"up_from":13,"seq":55834574930,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28284,"kb_used_data":1264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343556,"statfs":{"total":96636764160,"available":96607801344,"internally_reserved":0,"allocated":1294336,"data_stored":631401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:30:53 2023","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.35099999999999998,"15min":0.33800000000000002},"min":{"1min":0.23599999999999999,"5min":0.182,"15min":0.182},"max":{"1min":0.629,"5min":0.629,"15min":0.629},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.35899999999999999,"15min":0.55900000000000005},"min":{"1min":0.26800000000000002,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.56499999999999995,"5min":0.68000000000000005,"15min":6.0999999999999996},"last":0.34899999999999998}]},{"osd":1,"last update":"Wed Nov 8 19:30:53 2023","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.35599999999999998,"15min":0.34599999999999997},"min":{"1min":0.247,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.38800000000000001,"15min":0.39000000000000001},"min":{"1min":0.30299999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.63100000000000001,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.41099999999999998}]},{"osd":3,"last update":"Wed Nov 8 19:30:53 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.40600000000000003,"15min":0.39200000000000002},"min":{"1min":0.28199999999999997,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.60999999999999999,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.40000000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"min":{"1min":0.28799999999999998,"5min":0.217,"15min":0.217},"max":{"1min":0.621,"5min":0.65900000000000003,"15min":0.70299999999999996},"last":0.32900000000000001}]},{"osd":4,"last update":"Wed Nov 8 19:30:53 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56000000000000005,"15min":0.57599999999999996},"min":{"1min":0.41699999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.58099999999999996,"15min":0.58299999999999996},"min":{"1min":0.40500000000000003,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.872,"5min":0.872,"15min":0.90400000000000003},"last":0.59099999999999997}]},{"osd":5,"last update":"Wed Nov 8 19:30:53 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61799999999999999,"15min":0.59399999999999997},"min":{"1min":0.40799999999999997,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.622,"5min":0.60599999999999998,"15min":0.61099999999999999},"min":{"1min":0.48799999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.60099999999999998}]},{"osd":6,"last update":"Wed Nov 8 19:30:53 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.59399999999999997,"15min":0.59799999999999998},"min":{"1min":0.432,"5min":0.379,"15min":0.371},"max":{"1min":0.81000000000000005,"5min":0.83599999999999997,"15min":0.84799999999999998},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.59999999999999998,"15min":0.58599999999999997},"min":{"1min":0.42799999999999999,"5min":0.40300000000000002,"15min":0.36599999999999999},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.53400000000000003}]},{"osd":7,"last update":"Wed Nov 8 19:30:53 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.60899999999999999,"15min":0.58899999999999997},"min":{"1min":0.46200000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.61699999999999999,"15min":0.60899999999999999},"min":{"1min":0.41699999999999998,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.79600000000000004,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.623}]}]},{"osd":3,"up_from":13,"seq":55834574931,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28296,"kb_used_data":1276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343544,"statfs":{"total":96636764160,"available":96607789056,"internally_reserved":0,"allocated":1306624,"data_stored":639107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:30:59 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.33500000000000002,"15min":0.34000000000000002},"min":{"1min":0.254,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.48599999999999999,"5min":0.49099999999999999,"15min":0.626},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.378,"5min":0.372,"15min":0.35599999999999998},"min":{"1min":0.26700000000000002,"5min":0.23699999999999999,"15min":0.23300000000000001},"max":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.60799999999999998},"last":0.46500000000000002}]},{"osd":1,"last update":"Wed Nov 8 19:30:59 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.35999999999999999,"15min":0.35599999999999998},"min":{"1min":0.245,"5min":0.24099999999999999,"15min":0.23999999999999999},"max":{"1min":0.53300000000000003,"5min":0.61699999999999999,"15min":0.61699999999999999},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.40100000000000002,"15min":0.39000000000000001},"min":{"1min":0.30099999999999999,"5min":0.27900000000000003,"15min":0.27700000000000002},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.82099999999999995},"last":0.496}]},{"osd":2,"last update":"Wed Nov 8 19:30:59 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.40999999999999998,"15min":0.38300000000000001},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28799999999999998},"max":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58499999999999996},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.41999999999999998,"15min":0.40899999999999997},"min":{"1min":0.30499999999999999,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.59799999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.55900000000000005}]},{"osd":4,"last update":"Wed Nov 8 19:30:59 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.63,"15min":0.60799999999999998},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.85699999999999998,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63700000000000001,"15min":0.61499999999999999},"min":{"1min":0.38600000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.89600000000000002,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.67100000000000004}]},{"osd":5,"last update":"Wed Nov 8 19:30:59 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59899999999999998,"15min":0.58299999999999996},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.83099999999999996,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.623,"5min":0.622,"15min":0.59199999999999997},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.879,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.60399999999999998}]},{"osd":6,"last update":"Wed Nov 8 19:30:59 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.61399999999999999,"15min":0.59099999999999997},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41399999999999998},"max":{"1min":0.83899999999999997,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63300000000000001,"15min":0.61299999999999999},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.90500000000000003},"last":0.79700000000000004}]},{"osd":7,"last update":"Wed Nov 8 19:30:59 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.626,"15min":0.59999999999999998},"min":{"1min":0.41899999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":1.0009999999999999},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.63900000000000001,"15min":0.60999999999999999},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.85099999999999998,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.63800000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574930,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28296,"kb_used_data":1276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343544,"statfs":{"total":96636764160,"available":96607789056,"internally_reserved":0,"allocated":1306624,"data_stored":639107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:30:58 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54700000000000004,"15min":0.69899999999999995},"min":{"1min":0.311,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.71499999999999997,"5min":0.84799999999999998,"15min":6.5970000000000004},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.53500000000000003,"15min":0.56299999999999994},"min":{"1min":0.36299999999999999,"5min":0.32200000000000001,"15min":0.317},"max":{"1min":0.73099999999999998,"5min":0.75900000000000001,"15min":2.5030000000000001},"last":0.47099999999999997}]},{"osd":1,"last update":"Wed Nov 8 19:30:58 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.55400000000000005,"15min":0.51500000000000001},"min":{"1min":0.41699999999999998,"5min":0.32900000000000001,"15min":0.24299999999999999},"max":{"1min":0.72799999999999998,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.55900000000000005,"15min":0.52500000000000002},"min":{"1min":0.49299999999999999,"5min":0.312,"15min":0.312},"max":{"1min":0.72799999999999998,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.64500000000000002}]},{"osd":2,"last update":"Wed Nov 8 19:30:58 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55300000000000005},"min":{"1min":0.505,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.64400000000000002,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.58799999999999997,"15min":0.76000000000000001},"min":{"1min":0.50700000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.70999999999999996,"5min":0.871,"15min":6.6920000000000002},"last":0.44500000000000001}]},{"osd":3,"last update":"Wed Nov 8 19:30:58 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.56499999999999995,"15min":0.58099999999999996},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.69799999999999995,"5min":0.79800000000000004,"15min":1.0700000000000001},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.61799999999999999,"15min":0.61299999999999999},"min":{"1min":0.47799999999999998,"5min":0.41699999999999998,"15min":0.34599999999999997},"max":{"1min":0.70399999999999996,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.60199999999999998}]},{"osd":5,"last update":"Wed Nov 8 19:30:58 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.501,"15min":0.53100000000000003},"min":{"1min":0.39100000000000001,"5min":0.29599999999999999,"15min":0.28100000000000003},"max":{"1min":0.66100000000000003,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.51700000000000002,"15min":0.57099999999999995},"min":{"1min":0.377,"5min":0.32600000000000001,"15min":0.30299999999999999},"max":{"1min":0.65700000000000003,"5min":0.876,"15min":1.1060000000000001},"last":0.501}]},{"osd":6,"last update":"Wed Nov 8 19:30:58 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.54700000000000004,"15min":0.57199999999999995},"min":{"1min":0.35199999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.71899999999999997,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.60299999999999998,"15min":0.60999999999999999},"min":{"1min":0.373,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.74099999999999999,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.61899999999999999}]},{"osd":7,"last update":"Wed Nov 8 19:30:58 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.55800000000000005,"15min":0.60999999999999999},"min":{"1min":0.41599999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.68000000000000005,"5min":0.84499999999999997,"15min":1.141},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.59199999999999997,"15min":0.63},"min":{"1min":0.41399999999999998,"5min":0.38700000000000001,"15min":0.33000000000000002},"max":{"1min":0.76400000000000001,"5min":0.86399999999999999,"15min":0.86799999999999999},"last":0.63600000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574930,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28284,"kb_used_data":1264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343556,"statfs":{"total":96636764160,"available":96607801344,"internally_reserved":0,"allocated":1294336,"data_stored":631457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:30:51 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.57199999999999995,"15min":0.53900000000000003},"min":{"1min":0.35199999999999998,"5min":0.307,"15min":0.307},"max":{"1min":3.9689999999999999,"5min":3.9689999999999999,"15min":3.9689999999999999},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.54800000000000004,"15min":0.52200000000000002},"min":{"1min":0.41699999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.68899999999999995,"5min":0.746,"15min":0.746},"last":0.53400000000000003}]},{"osd":1,"last update":"Wed Nov 8 19:30:51 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.55300000000000005,"15min":0.53300000000000003},"min":{"1min":0.42399999999999999,"5min":0.37,"15min":0.37},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.54900000000000004,"15min":0.54100000000000004},"min":{"1min":0.44400000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.753,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.51400000000000001}]},{"osd":2,"last update":"Wed Nov 8 19:30:51 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.56399999999999995,"15min":0.54400000000000004},"min":{"1min":0.40000000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.749,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.56499999999999995,"15min":0.57299999999999995},"min":{"1min":0.46400000000000002,"5min":0.316,"15min":0.316},"max":{"1min":0.746,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.56499999999999995}]},{"osd":3,"last update":"Wed Nov 8 19:30:51 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.58499999999999996,"15min":0.56200000000000006},"min":{"1min":0.39600000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.73699999999999999,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.57999999999999996,"15min":0.59599999999999997},"min":{"1min":0.39900000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.79000000000000004,"5min":0.87,"15min":0.87},"last":0.61799999999999999}]},{"osd":4,"last update":"Wed Nov 8 19:30:51 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.495,"15min":0.54000000000000004},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.71899999999999997,"5min":0.80500000000000005,"15min":0.86799999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.53900000000000003,"15min":0.56100000000000005},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.751,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.71499999999999997}]},{"osd":6,"last update":"Wed Nov 8 19:30:51 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.56100000000000005,"15min":0.59599999999999997},"min":{"1min":0.39700000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.73599999999999999,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59199999999999997,"15min":0.60899999999999999},"min":{"1min":0.41399999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.76400000000000001,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.68500000000000005}]},{"osd":7,"last update":"Wed Nov 8 19:30:51 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.57099999999999995,"15min":0.59899999999999998},"min":{"1min":0.38200000000000001,"5min":0.371,"15min":0.36899999999999999},"max":{"1min":0.74099999999999999,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.746},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58999999999999997,"15min":0.63},"min":{"1min":0.433,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.91900000000000004},"last":0.70499999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-08T19:31:17.216 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:31:17.371 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:17.369+0000 7f1d0506d700 1 Processor -- start 2023-11-08T19:31:17.371 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:17.369+0000 7f1d0506d700 1 -- start start 2023-11-08T19:31:17.374 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:17.370+0000 7f1d0506d700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f1d0015aa30 0x7f1d000cf510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:17.374 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:17.370+0000 7f1d0506d700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1d000b2210 0x7f1d000b2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:17.375 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:17.370+0000 7f1d0506d700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f1d000cbe20 0x7f1d000cc270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:17.375 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:17.370+0000 7f1d0506d700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d000b2bb0 con 0x7f1d0015aa30 2023-11-08T19:31:17.375 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:17.370+0000 7f1d0506d700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d000cf080 con 0x7f1d000cbe20 2023-11-08T19:31:17.376 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:17.370+0000 7f1d0506d700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d000ce470 con 0x7f1d000b2210 2023-11-08T19:31:17.376 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:17.370+0000 7f1cfed9d700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f1d0015aa30 0x7f1d000cf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:17.376 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:17.370+0000 7f1cfed9d700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f1d0015aa30 0x7f1d000cf510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57138/0 (socket says 172.21.15.143:57138) 2023-11-08T19:31:17.377 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:17.370+0000 7f1cfed9d700 1 -- 172.21.15.143:0/146826956 learned_addr learned my addr 172.21.15.143:0/146826956 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:31:17.377 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:17.370+0000 7f1cff59e700 1 --2- 172.21.15.143:0/146826956 >> v2:172.21.15.174:3300/0 conn(0x7f1d000cbe20 0x7f1d000cc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:17.377 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:17.370+0000 7f1cfe59c700 1 --2- 172.21.15.143:0/146826956 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1d000b2210 0x7f1d000b2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:17.377 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:17.371+0000 7f1cfed9d700 1 -- 172.21.15.143:0/146826956 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1d000b2210 msgr2=0x7f1d000b2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:17.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:17.371+0000 7f1cfed9d700 1 --2- 172.21.15.143:0/146826956 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1d000b2210 0x7f1d000b2670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:17.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:17.371+0000 7f1cfed9d700 1 -- 172.21.15.143:0/146826956 >> v2:172.21.15.174:3300/0 conn(0x7f1d000cbe20 msgr2=0x7f1d000cc270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:17.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:17.371+0000 7f1cfed9d700 1 --2- 172.21.15.143:0/146826956 >> v2:172.21.15.174:3300/0 conn(0x7f1d000cbe20 0x7f1d000cc270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:17.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:17.371+0000 7f1cfed9d700 1 -- 172.21.15.143:0/146826956 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d000ce750 con 0x7f1d0015aa30 2023-11-08T19:31:17.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:17.371+0000 7f1cff59e700 1 --2- 172.21.15.143:0/146826956 >> v2:172.21.15.174:3300/0 conn(0x7f1d000cbe20 0x7f1d000cc270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:31:17.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:17.371+0000 7f1cfe59c700 1 --2- 172.21.15.143:0/146826956 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1d000b2210 0x7f1d000b2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:31:17.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:17.371+0000 7f1cfed9d700 1 --2- 172.21.15.143:0/146826956 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1d0015aa30 0x7f1d000cf510 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f1d000ccf00 tx=0x7f1cf4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd241137a14077fc server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:17.380 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:17.371+0000 7f1cfdd9b700 1 -- 172.21.15.143:0/146826956 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf4005e30 con 0x7f1d0015aa30 2023-11-08T19:31:17.380 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:17.371+0000 7f1cfdd9b700 1 -- 172.21.15.143:0/146826956 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cf4004030 con 0x7f1d0015aa30 2023-11-08T19:31:17.380 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:17.371+0000 7f1cfdd9b700 1 -- 172.21.15.143:0/146826956 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf40043e0 con 0x7f1d0015aa30 2023-11-08T19:31:17.380 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:17.371+0000 7f1d0506d700 1 -- 172.21.15.143:0/146826956 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1d0015aa30 msgr2=0x7f1d000cf510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:17.381 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:17.371+0000 7f1d0506d700 1 --2- 172.21.15.143:0/146826956 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1d0015aa30 0x7f1d000cf510 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f1d000ccf00 tx=0x7f1cf4009bd0 comp rx=0 tx=0).stop 2023-11-08T19:31:17.381 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:17.371+0000 7f1d0506d700 1 -- 172.21.15.143:0/146826956 shutdown_connections 2023-11-08T19:31:17.381 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:17.371+0000 7f1d0506d700 1 --2- 172.21.15.143:0/146826956 >> v2:172.21.15.174:3300/0 conn(0x7f1d000cbe20 0x7f1d000cc270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:17.381 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:17.371+0000 7f1d0506d700 1 --2- 172.21.15.143:0/146826956 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1d0015aa30 0x7f1d000cf510 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:17.382 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:17.371+0000 7f1d0506d700 1 --2- 172.21.15.143:0/146826956 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1d000b2210 0x7f1d000b2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:17.382 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:17.371+0000 7f1d0506d700 1 -- 172.21.15.143:0/146826956 >> 172.21.15.143:0/146826956 conn(0x7f1d00099ab0 msgr2=0x7f1d000d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:17.382 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:17.372+0000 7f1d0506d700 1 -- 172.21.15.143:0/146826956 shutdown_connections 2023-11-08T19:31:17.383 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:17.372+0000 7f1d0506d700 1 -- 172.21.15.143:0/146826956 wait complete. 2023-11-08T19:31:17.383 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:17.373+0000 7f1d0506d700 1 Processor -- start 2023-11-08T19:31:17.383 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:17.373+0000 7f1d0506d700 1 -- start start 2023-11-08T19:31:17.385 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:17.374+0000 7f1d0506d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1d000b2210 0x7f1d0015ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:17.386 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:17.374+0000 7f1d0506d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1d000cbe20 0x7f1d0006ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:17.386 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:17.374+0000 7f1d0506d700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1d0015aa30 0x7f1d0006f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:17.386 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:17.374+0000 7f1d0506d700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d000c5a50 con 0x7f1d000b2210 2023-11-08T19:31:17.387 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:17.374+0000 7f1d0506d700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d000b4580 con 0x7f1d000cbe20 2023-11-08T19:31:17.387 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:17.374+0000 7f1d0506d700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d000c5140 con 0x7f1d0015aa30 2023-11-08T19:31:17.387 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:17.374+0000 7f1cfed9d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1d000b2210 0x7f1d0015ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:17.387 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:17.374+0000 7f1cfed9d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1d000b2210 0x7f1d0015ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57144/0 (socket says 172.21.15.143:57144) 2023-11-08T19:31:17.388 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:17.374+0000 7f1cfed9d700 1 -- 172.21.15.143:0/1802260077 learned_addr learned my addr 172.21.15.143:0/1802260077 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:31:17.388 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:17.374+0000 7f1cfe59c700 1 --2- 172.21.15.143:0/1802260077 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1d000cbe20 0x7f1d0006ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:17.388 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:17.374+0000 7f1cff59e700 1 --2- 172.21.15.143:0/1802260077 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1d0015aa30 0x7f1d0006f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:17.388 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:17.374+0000 7f1cfed9d700 1 -- 172.21.15.143:0/1802260077 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1d0015aa30 msgr2=0x7f1d0006f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:17.389 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:17.374+0000 7f1cfed9d700 1 --2- 172.21.15.143:0/1802260077 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1d0015aa30 0x7f1d0006f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:17.389 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:17.374+0000 7f1cfed9d700 1 -- 172.21.15.143:0/1802260077 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1d000cbe20 msgr2=0x7f1d0006ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:17.389 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:17.374+0000 7f1cfed9d700 1 --2- 172.21.15.143:0/1802260077 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1d000cbe20 0x7f1d0006ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:17.389 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:17.374+0000 7f1cfed9d700 1 -- 172.21.15.143:0/1802260077 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cf4009500 con 0x7f1d000b2210 2023-11-08T19:31:17.390 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:17.374+0000 7f1cfed9d700 1 --2- 172.21.15.143:0/1802260077 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1d000b2210 0x7f1d0015ae10 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f1cf4009130 tx=0x7f1cf4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:17.390 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:17.374+0000 7f1ce77fe700 1 -- 172.21.15.143:0/1802260077 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf4003c70 con 0x7f1d000b2210 2023-11-08T19:31:17.390 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:17.374+0000 7f1ce77fe700 1 -- 172.21.15.143:0/1802260077 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cf4003dd0 con 0x7f1d000b2210 2023-11-08T19:31:17.391 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:17.375+0000 7f1ce77fe700 1 -- 172.21.15.143:0/1802260077 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf4011660 con 0x7f1d000b2210 2023-11-08T19:31:17.391 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:17.375+0000 7f1d0506d700 1 -- 172.21.15.143:0/1802260077 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d000c5400 con 0x7f1d000b2210 2023-11-08T19:31:17.391 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:17.375+0000 7f1d0506d700 1 -- 172.21.15.143:0/1802260077 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d0015b350 con 0x7f1d000b2210 2023-11-08T19:31:17.391 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:17.375+0000 7f1d0506d700 1 -- 172.21.15.143:0/1802260077 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d0004eee0 con 0x7f1d000b2210 2023-11-08T19:31:17.392 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:17.376+0000 7f1ce77fe700 1 -- 172.21.15.143:0/1802260077 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1cf402f480 con 0x7f1d000b2210 2023-11-08T19:31:17.392 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:17.377+0000 7f1ce77fe700 1 --2- 172.21.15.143:0/1802260077 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f1cec06af00 0x7f1cec06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:17.392 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:17.377+0000 7f1ce77fe700 1 -- 172.21.15.143:0/1802260077 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6651+0+0 (secure 0 0 0) 0x7f1cf408adc0 con 0x7f1d000b2210 2023-11-08T19:31:17.392 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:17.377+0000 7f1cfe59c700 1 --2- 172.21.15.143:0/1802260077 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f1cec06af00 0x7f1cec06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:17.393 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:17.377+0000 7f1cfe59c700 1 --2- 172.21.15.143:0/1802260077 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f1cec06af00 0x7f1cec06d3c0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f1d000b3560 tx=0x7f1cf0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:17.393 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:17.380+0000 7f1ce77fe700 1 -- 172.21.15.143:0/1802260077 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1cf4055c30 con 0x7f1d000b2210 2023-11-08T19:31:17.569 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:17.568+0000 7f1d0506d700 1 -- 172.21.15.143:0/1802260077 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1d000cc780 con 0x7f1d000b2210 2023-11-08T19:31:17.571 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:17.570+0000 7f1ce77fe700 1 -- 172.21.15.143:0/1802260077 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v107) v1 ==== 74+0+19269 (secure 0 0 0) 0x7f1cf4059250 con 0x7f1d000b2210 2023-11-08T19:31:17.571 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:31:17.572 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":107,"fsid":"968483af-77fc-4068-ad7c-5c2a1a4613d7","created":"2023-11-08T19:24:32.826484+0000","modified":"2023-11-08T19:31:14.005275+0000","last_up_change":"2023-11-08T19:24:43.573880+0000","last_in_change":"2023-11-08T19:24:35.314525+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:44.551530+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"snap-test-pool","create_time":"2023-11-08T19:26:02.874384+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":36,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"test_data_pool","create_time":"2023-11-08T19:27:05.601949+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":66,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_metadata_pool","create_time":"2023-11-08T19:27:06.536369+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bae98ec1-d94e-403d-a220-a522a53ea6f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6809","nonce":1487830566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6811","nonce":1487830566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6815","nonce":1487830566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1487830566},{"type":"v1","addr":"172.21.15.143:6813","nonce":1487830566}]},"public_addr":"172.21.15.143:6809/1487830566","cluster_addr":"172.21.15.143:6811/1487830566","heartbeat_back_addr":"172.21.15.143:6815/1487830566","heartbeat_front_addr":"172.21.15.143:6813/1487830566","state":["exists","up"]},{"osd":1,"uuid":"5cb6538a-1ccb-4322-8633-09f66af0b7e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6801","nonce":787341397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6803","nonce":787341397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6807","nonce":787341397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":787341397},{"type":"v1","addr":"172.21.15.143:6805","nonce":787341397}]},"public_addr":"172.21.15.143:6801/787341397","cluster_addr":"172.21.15.143:6803/787341397","heartbeat_back_addr":"172.21.15.143:6807/787341397","heartbeat_front_addr":"172.21.15.143:6805/787341397","state":["exists","up"]},{"osd":2,"uuid":"cdbf522c-60ef-4d60-9653-8dcb529c101a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6822","nonce":2301325597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6826","nonce":2301325597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6831","nonce":2301325597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":2301325597},{"type":"v1","addr":"172.21.15.143:6829","nonce":2301325597}]},"public_addr":"172.21.15.143:6822/2301325597","cluster_addr":"172.21.15.143:6826/2301325597","heartbeat_back_addr":"172.21.15.143:6831/2301325597","heartbeat_front_addr":"172.21.15.143:6829/2301325597","state":["exists","up"]},{"osd":3,"uuid":"4e2783cf-ebb9-4dd8-8245-bc1d17ce4325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6817","nonce":2021504428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6819","nonce":2021504428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6825","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6827","nonce":2021504428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6821","nonce":2021504428},{"type":"v1","addr":"172.21.15.143:6823","nonce":2021504428}]},"public_addr":"172.21.15.143:6817/2021504428","cluster_addr":"172.21.15.143:6819/2021504428","heartbeat_back_addr":"172.21.15.143:6827/2021504428","heartbeat_front_addr":"172.21.15.143:6823/2021504428","state":["exists","up"]},{"osd":4,"uuid":"691256fa-9b77-49b9-a346-7ae1adf0ca80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6801","nonce":1993380595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6803","nonce":1993380595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6807","nonce":1993380595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":1993380595},{"type":"v1","addr":"172.21.15.174:6805","nonce":1993380595}]},"public_addr":"172.21.15.174:6801/1993380595","cluster_addr":"172.21.15.174:6803/1993380595","heartbeat_back_addr":"172.21.15.174:6807/1993380595","heartbeat_front_addr":"172.21.15.174:6805/1993380595","state":["exists","up"]},{"osd":5,"uuid":"756fc352-48c9-46ce-b52e-0541fe6f8525","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6809","nonce":1777923303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6811","nonce":1777923303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6815","nonce":1777923303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":1777923303},{"type":"v1","addr":"172.21.15.174:6813","nonce":1777923303}]},"public_addr":"172.21.15.174:6809/1777923303","cluster_addr":"172.21.15.174:6811/1777923303","heartbeat_back_addr":"172.21.15.174:6815/1777923303","heartbeat_front_addr":"172.21.15.174:6813/1777923303","state":["exists","up"]},{"osd":6,"uuid":"fab2ac2b-ed97-4cde-9825-b480eabf8b8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6825","nonce":2755130316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6827","nonce":2755130316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6831","nonce":2755130316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":2755130316},{"type":"v1","addr":"172.21.15.174:6829","nonce":2755130316}]},"public_addr":"172.21.15.174:6825/2755130316","cluster_addr":"172.21.15.174:6827/2755130316","heartbeat_back_addr":"172.21.15.174:6831/2755130316","heartbeat_front_addr":"172.21.15.174:6829/2755130316","state":["exists","up"]},{"osd":7,"uuid":"3de9b797-2ab6-4d41-a863-7e7d638fd703","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6817","nonce":2847141411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6819","nonce":2847141411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6823","nonce":2847141411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2847141411},{"type":"v1","addr":"172.21.15.174:6821","nonce":2847141411}]},"public_addr":"172.21.15.174:6817/2847141411","cluster_addr":"172.21.15.174:6819/2847141411","heartbeat_back_addr":"172.21.15.174:6823/2847141411","heartbeat_front_addr":"172.21.15.174:6821/2847141411","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.477076+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.496890+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.539305+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.454673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.538881+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.521294+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.520712+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:40.540404+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.143:6834/2489252684":"2023-11-09T19:31:10.998094+0000","172.21.15.143:6833/2129643800":"2023-11-09T19:31:10.998078+0000","172.21.15.143:6835/2129643800":"2023-11-09T19:31:10.998078+0000","172.21.15.143:6835/3953059900":"2023-11-09T19:29:14.940251+0000","172.21.15.143:6833/3953059900":"2023-11-09T19:29:14.940251+0000","172.21.15.174:6837/2201447629":"2023-11-09T19:28:48.342407+0000","172.21.15.174:6835/2201447629":"2023-11-09T19:28:48.342407+0000","172.21.15.174:6836/3601088618":"2023-11-09T19:30:47.736745+0000","172.21.15.143:6832/2489252684":"2023-11-09T19:31:10.998094+0000","172.21.15.143:6834/3172664377":"2023-11-09T19:29:41.150862+0000","172.21.15.143:6832/3172664377":"2023-11-09T19:29:41.150862+0000","172.21.15.143:6833/3664584522":"2023-11-09T19:30:12.566205+0000","172.21.15.143:6835/3664584522":"2023-11-09T19:30:12.566205+0000","172.21.15.143:0/2918728496":"2023-11-08T20:30:55.736484+0000","172.21.15.143:6832/1535103237":"2023-11-09T19:30:40.630538+0000","172.21.15.143:6834/1535103237":"2023-11-09T19:30:40.630538+0000","172.21.15.174:6834/3601088618":"2023-11-09T19:30:47.736745+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:31:17.573 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:17.572+0000 7f1d0506d700 1 -- 172.21.15.143:0/1802260077 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f1cec06af00 msgr2=0x7f1cec06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:17.573 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:17.572+0000 7f1d0506d700 1 --2- 172.21.15.143:0/1802260077 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f1cec06af00 0x7f1cec06d3c0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f1d000b3560 tx=0x7f1cf0008040 comp rx=0 tx=0).stop 2023-11-08T19:31:17.574 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:17.572+0000 7f1d0506d700 1 -- 172.21.15.143:0/1802260077 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1d000b2210 msgr2=0x7f1d0015ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:17.574 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:17.572+0000 7f1d0506d700 1 --2- 172.21.15.143:0/1802260077 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1d000b2210 0x7f1d0015ae10 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f1cf4009130 tx=0x7f1cf4004750 comp rx=0 tx=0).stop 2023-11-08T19:31:17.574 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:17.572+0000 7f1d0506d700 1 -- 172.21.15.143:0/1802260077 shutdown_connections 2023-11-08T19:31:17.575 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:17.572+0000 7f1d0506d700 1 --2- 172.21.15.143:0/1802260077 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f1cec06af00 0x7f1cec06d3c0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:17.575 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:17.572+0000 7f1d0506d700 1 --2- 172.21.15.143:0/1802260077 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1d0015aa30 0x7f1d0006f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:17.575 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:17.572+0000 7f1d0506d700 1 --2- 172.21.15.143:0/1802260077 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1d000b2210 0x7f1d0015ae10 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:17.576 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:17.572+0000 7f1d0506d700 1 --2- 172.21.15.143:0/1802260077 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1d000cbe20 0x7f1d0006ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:17.576 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:17.572+0000 7f1d0506d700 1 -- 172.21.15.143:0/1802260077 >> 172.21.15.143:0/1802260077 conn(0x7f1d00099ab0 msgr2=0x7f1d00159cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:17.576 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:17.572+0000 7f1d0506d700 1 -- 172.21.15.143:0/1802260077 shutdown_connections 2023-11-08T19:31:17.576 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:17.572+0000 7f1d0506d700 1 -- 172.21.15.143:0/1802260077 wait complete. 2023-11-08T19:31:18.586 INFO:tasks.ceph:Scrubbing osd.0 2023-11-08T19:31:18.587 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2023-11-08T19:31:18.726 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.724+0000 7f117b8b9700 1 Processor -- start 2023-11-08T19:31:18.726 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.724+0000 7f117b8b9700 1 -- start start 2023-11-08T19:31:18.731 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.725+0000 7f117b8b9700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f117415bb20 0x7f11740b2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:18.732 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.725+0000 7f117b8b9700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11740d0420 con 0x7f11740d1d80 2023-11-08T19:31:18.732 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.725+0000 7f117b8b9700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11740cf810 con 0x7f11740b1730 2023-11-08T19:31:18.732 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.725+0000 7f117b8b9700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11740cf600 con 0x7f117415bb20 2023-11-08T19:31:18.732 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.725+0000 7f1179e56700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f11740d1d80 0x7f11740cc4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:53310/0 (socket says 172.21.15.143:53310) 2023-11-08T19:31:18.733 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.725+0000 7f1179e56700 1 -- 172.21.15.143:0/1849479800 learned_addr learned my addr 172.21.15.143:0/1849479800 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:31:18.733 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.725+0000 7f1179655700 1 --2- 172.21.15.143:0/1849479800 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f117415bb20 0x7f11740b2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:18.733 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.725+0000 7f116bfff700 1 -- 172.21.15.143:0/1849479800 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 252781749 0 0) 0x7f11740d0420 con 0x7f11740d1d80 2023-11-08T19:31:18.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.725+0000 7f116bfff700 1 -- 172.21.15.143:0/1849479800 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f11600035b0 con 0x7f11740d1d80 2023-11-08T19:31:18.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.725+0000 7f116bfff700 1 -- 172.21.15.143:0/1849479800 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1448483207 0 0) 0x7f11600035b0 con 0x7f11740d1d80 2023-11-08T19:31:18.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.725+0000 7f116bfff700 1 -- 172.21.15.143:0/1849479800 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f117415bb20 msgr2=0x7f11740b2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:18.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.725+0000 7f116bfff700 1 --2- 172.21.15.143:0/1849479800 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f117415bb20 0x7f11740b2670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:18.735 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.725+0000 7f116bfff700 1 -- 172.21.15.143:0/1849479800 >> v1:172.21.15.174:6789/0 conn(0x7f11740b1730 legacy=0x7f11740d1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:18.735 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.725+0000 7f116bfff700 1 -- 172.21.15.143:0/1849479800 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11740cfaf0 con 0x7f11740d1d80 2023-11-08T19:31:18.735 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.726+0000 7f116bfff700 1 -- 172.21.15.143:0/1849479800 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4147756303 0 0) 0x7f116c0019f0 con 0x7f11740d1d80 2023-11-08T19:31:18.736 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.726+0000 7f116bfff700 1 -- 172.21.15.143:0/1849479800 >> v1:172.21.15.143:6789/0 conn(0x7f11740d1d80 legacy=0x7f11740cc4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:18.736 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.726+0000 7f116bfff700 1 --2- 172.21.15.143:0/1849479800 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1160003a30 0x7f1160003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:18.736 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.726+0000 7f116bfff700 1 --2- 172.21.15.143:0/1849479800 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f11600043e0 0x7f11600048b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:18.736 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.726+0000 7f116bfff700 1 --2- 172.21.15.143:0/1849479800 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f117415bb20 0x7f1160005110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:18.737 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.726+0000 7f116bfff700 1 -- 172.21.15.143:0/1849479800 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11740cf600 con 0x7f1160003a30 2023-11-08T19:31:18.737 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.726+0000 7f116bfff700 1 -- 172.21.15.143:0/1849479800 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f116000ba10 con 0x7f11600043e0 2023-11-08T19:31:18.737 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.726+0000 7f116bfff700 1 -- 172.21.15.143:0/1849479800 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f116000bbd0 con 0x7f117415bb20 2023-11-08T19:31:18.738 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.726+0000 7f1179655700 1 --2- 172.21.15.143:0/1849479800 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1160003a30 0x7f1160003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:18.738 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.726+0000 7f1178e54700 1 --2- 172.21.15.143:0/1849479800 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f11600043e0 0x7f11600048b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:18.738 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.726+0000 7f1179e56700 1 --2- 172.21.15.143:0/1849479800 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f117415bb20 0x7f1160005110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:18.738 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.726+0000 7f1179655700 1 -- 172.21.15.143:0/1849479800 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f117415bb20 msgr2=0x7f1160005110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:18.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.726+0000 7f1179655700 1 --2- 172.21.15.143:0/1849479800 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f117415bb20 0x7f1160005110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:18.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.726+0000 7f1179655700 1 -- 172.21.15.143:0/1849479800 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f11600043e0 msgr2=0x7f11600048b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:18.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.726+0000 7f1179655700 1 --2- 172.21.15.143:0/1849479800 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f11600043e0 0x7f11600048b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:18.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.726+0000 7f1179655700 1 -- 172.21.15.143:0/1849479800 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11600035b0 con 0x7f1160003a30 2023-11-08T19:31:18.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.726+0000 7f1179e56700 1 --2- 172.21.15.143:0/1849479800 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f117415bb20 0x7f1160005110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:31:18.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.726+0000 7f1179655700 1 --2- 172.21.15.143:0/1849479800 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1160003a30 0x7f1160003e10 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f1164009b60 tx=0x7f1160005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73622344926bfde6 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:18.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.726+0000 7f116bfff700 1 -- 172.21.15.143:0/1849479800 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f116400e6c0 con 0x7f1160003a30 2023-11-08T19:31:18.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.726+0000 7f116bfff700 1 -- 172.21.15.143:0/1849479800 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f116400e820 con 0x7f1160003a30 2023-11-08T19:31:18.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.726+0000 7f116bfff700 1 -- 172.21.15.143:0/1849479800 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f116400eb90 con 0x7f1160003a30 2023-11-08T19:31:18.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.727+0000 7f117b8b9700 1 -- 172.21.15.143:0/1849479800 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1160003a30 msgr2=0x7f1160003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:18.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.727+0000 7f117b8b9700 1 --2- 172.21.15.143:0/1849479800 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1160003a30 0x7f1160003e10 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f1164009b60 tx=0x7f1160005880 comp rx=0 tx=0).stop 2023-11-08T19:31:18.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.727+0000 7f1179e56700 1 -- 172.21.15.143:0/1849479800 reap_dead start 2023-11-08T19:31:18.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.727+0000 7f117b8b9700 1 -- 172.21.15.143:0/1849479800 shutdown_connections 2023-11-08T19:31:18.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.727+0000 7f117b8b9700 1 -- 172.21.15.143:0/1849479800 >> 172.21.15.143:0/1849479800 conn(0x7f117408b000 msgr2=0x7f117409ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:18.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.727+0000 7f117b8b9700 1 -- 172.21.15.143:0/1849479800 shutdown_connections 2023-11-08T19:31:18.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.727+0000 7f117b8b9700 1 -- 172.21.15.143:0/1849479800 wait complete. 2023-11-08T19:31:18.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.728+0000 7f117b8b9700 1 Processor -- start 2023-11-08T19:31:18.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.728+0000 7f117b8b9700 1 -- start start 2023-11-08T19:31:18.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.729+0000 7f117b8b9700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1160003a30 0x7f11740b5e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:18.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.729+0000 7f117b8b9700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f11741e5d30 0x7f11740b1c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:18.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.729+0000 7f117b8b9700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f11740b2180 0x7f11741530b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:18.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.729+0000 7f117b8b9700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11740b4ac0 con 0x7f11741e5d30 2023-11-08T19:31:18.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.729+0000 7f117b8b9700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11741543f0 con 0x7f11740b2180 2023-11-08T19:31:18.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.729+0000 7f117b8b9700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11740b2600 con 0x7f1160003a30 2023-11-08T19:31:18.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.729+0000 7f1178e54700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f11741e5d30 0x7f11740b1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:18.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.729+0000 7f1178e54700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f11741e5d30 0x7f11740b1c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57166/0 (socket says 172.21.15.143:57166) 2023-11-08T19:31:18.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.729+0000 7f1178e54700 1 -- 172.21.15.143:0/3422511490 learned_addr learned my addr 172.21.15.143:0/3422511490 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:31:18.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.729+0000 7f1179655700 1 --2- 172.21.15.143:0/3422511490 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1160003a30 0x7f11740b5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:18.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.729+0000 7f1178e54700 1 -- 172.21.15.143:0/3422511490 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1160003a30 msgr2=0x7f11740b5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:18.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.729+0000 7f1178e54700 1 --2- 172.21.15.143:0/3422511490 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1160003a30 0x7f11740b5e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:18.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.729+0000 7f1179e56700 1 --2- 172.21.15.143:0/3422511490 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f11740b2180 0x7f11741530b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:18.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.729+0000 7f1178e54700 1 -- 172.21.15.143:0/3422511490 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f11740b2180 msgr2=0x7f11741530b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:18.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.729+0000 7f1178e54700 1 --2- 172.21.15.143:0/3422511490 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f11740b2180 0x7f11741530b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:18.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.729+0000 7f1178e54700 1 -- 172.21.15.143:0/3422511490 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1174153e30 con 0x7f11741e5d30 2023-11-08T19:31:18.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.729+0000 7f1178e54700 1 --2- 172.21.15.143:0/3422511490 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f11741e5d30 0x7f11740b1c40 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f117000f840 tx=0x7f117000fc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:18.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.730+0000 7f1169ffb700 1 -- 172.21.15.143:0/3422511490 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11700052e0 con 0x7f11741e5d30 2023-11-08T19:31:18.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.730+0000 7f1169ffb700 1 -- 172.21.15.143:0/3422511490 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1170005440 con 0x7f11741e5d30 2023-11-08T19:31:18.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.730+0000 7f1169ffb700 1 -- 172.21.15.143:0/3422511490 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11700038c0 con 0x7f11741e5d30 2023-11-08T19:31:18.750 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.730+0000 7f117b8b9700 1 -- 172.21.15.143:0/3422511490 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1174154110 con 0x7f11741e5d30 2023-11-08T19:31:18.750 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.730+0000 7f117b8b9700 1 -- 172.21.15.143:0/3422511490 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11741e3610 con 0x7f11741e5d30 2023-11-08T19:31:18.752 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.731+0000 7f117b8b9700 1 -- 172.21.15.143:0/3422511490 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f11740d00b0 con 0x7f11741e5d30 2023-11-08T19:31:18.752 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.731+0000 7f1169ffb700 1 -- 172.21.15.143:0/3422511490 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1170003a20 con 0x7f11741e5d30 2023-11-08T19:31:18.753 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.731+0000 7f1169ffb700 1 --2- 172.21.15.143:0/3422511490 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f115806af00 0x7f115806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:18.753 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.732+0000 7f1169ffb700 1 -- 172.21.15.143:0/3422511490 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6651+0+0 (secure 0 0 0) 0x7f1170089840 con 0x7f11741e5d30 2023-11-08T19:31:18.753 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.732+0000 7f1179655700 1 --2- 172.21.15.143:0/3422511490 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f115806af00 0x7f115806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:18.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.732+0000 7f1169ffb700 1 --2- 172.21.15.143:0/3422511490 >> [v2:172.21.15.143:6808/1487830566,v1:172.21.15.143:6809/1487830566] conn(0x7f1158073a10 0x7f1158075e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:18.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.732+0000 7f1169ffb700 1 -- 172.21.15.143:0/3422511490 --> [v2:172.21.15.143:6808/1487830566,v1:172.21.15.143:6809/1487830566] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1158076500 con 0x7f1158073a10 2023-11-08T19:31:18.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.732+0000 7f1169ffb700 1 -- 172.21.15.143:0/3422511490 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_get_version_reply(handle=1 version=107) v2 ==== 24+0+0 (secure 0 0 0) 0x7f117008e2c0 con 0x7f11741e5d30 2023-11-08T19:31:18.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.732+0000 7f1179e56700 1 --2- 172.21.15.143:0/3422511490 >> [v2:172.21.15.143:6808/1487830566,v1:172.21.15.143:6809/1487830566] conn(0x7f1158073a10 0x7f1158075e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:18.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.732+0000 7f1179e56700 1 --2- 172.21.15.143:0/3422511490 >> [v2:172.21.15.143:6808/1487830566,v1:172.21.15.143:6809/1487830566] conn(0x7f1158073a10 0x7f1158075e30 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:18.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.732+0000 7f1179655700 1 --2- 172.21.15.143:0/3422511490 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f115806af00 0x7f115806d3c0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f11640090b0 tx=0x7f1164008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:18.756 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.733+0000 7f1169ffb700 1 -- 172.21.15.143:0/3422511490 <== osd.0 v2:172.21.15.143:6808/1487830566 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26745 (crc 0 0 0) 0x7f1158076500 con 0x7f1158073a10 2023-11-08T19:31:18.756 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.747+0000 7f117b8b9700 1 -- 172.21.15.143:0/3422511490 --> [v2:172.21.15.143:6808/1487830566,v1:172.21.15.143:6809/1487830566] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f11740cfdd0 con 0x7f1158073a10 2023-11-08T19:31:18.756 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.748+0000 7f1169ffb700 1 -- 172.21.15.143:0/3422511490 <== osd.0 v2:172.21.15.143:6808/1487830566 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f116c0019b0 con 0x7f1158073a10 2023-11-08T19:31:18.757 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2023-11-08T19:31:18.757 INFO:teuthology.orchestra.run.smithi143.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-08T19:31:18.757 INFO:teuthology.orchestra.run.smithi143.stdout:} 2023-11-08T19:31:18.758 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.748+0000 7f117b8b9700 1 -- 172.21.15.143:0/3422511490 >> [v2:172.21.15.143:6808/1487830566,v1:172.21.15.143:6809/1487830566] conn(0x7f1158073a10 msgr2=0x7f1158075e30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:18.759 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.748+0000 7f117b8b9700 1 --2- 172.21.15.143:0/3422511490 >> [v2:172.21.15.143:6808/1487830566,v1:172.21.15.143:6809/1487830566] conn(0x7f1158073a10 0x7f1158075e30 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:18.759 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.748+0000 7f117b8b9700 1 -- 172.21.15.143:0/3422511490 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f115806af00 msgr2=0x7f115806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:18.759 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.748+0000 7f117b8b9700 1 --2- 172.21.15.143:0/3422511490 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f115806af00 0x7f115806d3c0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f11640090b0 tx=0x7f1164008030 comp rx=0 tx=0).stop 2023-11-08T19:31:18.760 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.748+0000 7f117b8b9700 1 -- 172.21.15.143:0/3422511490 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f11741e5d30 msgr2=0x7f11740b1c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:18.760 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.748+0000 7f117b8b9700 1 --2- 172.21.15.143:0/3422511490 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f11741e5d30 0x7f11740b1c40 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f117000f840 tx=0x7f117000fc00 comp rx=0 tx=0).stop 2023-11-08T19:31:18.760 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.748+0000 7f1179e56700 1 -- 172.21.15.143:0/3422511490 reap_dead start 2023-11-08T19:31:18.760 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.748+0000 7f117b8b9700 1 -- 172.21.15.143:0/3422511490 shutdown_connections 2023-11-08T19:31:18.761 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.748+0000 7f117b8b9700 1 -- 172.21.15.143:0/3422511490 >> 172.21.15.143:0/3422511490 conn(0x7f117408b000 msgr2=0x7f11740b2f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:18.761 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.748+0000 7f117b8b9700 1 -- 172.21.15.143:0/3422511490 shutdown_connections 2023-11-08T19:31:18.761 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.748+0000 7f117b8b9700 1 -- 172.21.15.143:0/3422511490 wait complete. 2023-11-08T19:31:18.762 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2023-11-08T19:31:18.884 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.883+0000 7f908e962700 1 Processor -- start 2023-11-08T19:31:18.885 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.883+0000 7f908e962700 1 -- start start 2023-11-08T19:31:18.887 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.884+0000 7f908e962700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f908806dbb0 0x7f90880ca470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:18.887 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.884+0000 7f908e962700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f90880668e0 0x7f90880ca9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:18.887 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.884+0000 7f908e962700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90880ce7f0 con 0x7f908806df90 2023-11-08T19:31:18.888 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.884+0000 7f908e962700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9088066db0 con 0x7f90880668e0 2023-11-08T19:31:18.888 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.884+0000 7f908e962700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90880af5e0 con 0x7f908806dbb0 2023-11-08T19:31:18.888 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.884+0000 7f9087fff700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f908806dbb0 0x7f90880ca470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:18.889 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.884+0000 7f90877fe700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f90880668e0 0x7f90880ca9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:18.889 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.884+0000 7f908ceff700 1 --1- >> v1:172.21.15.174:6789/0 conn(0x7f908806df90 0x7f90881621f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.174:6789/0 says I am v1:172.21.15.143:36034/0 (socket says 172.21.15.143:36034) 2023-11-08T19:31:18.889 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.884+0000 7f908ceff700 1 -- 172.21.15.143:0/3261017547 learned_addr learned my addr 172.21.15.143:0/3261017547 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:31:18.889 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.884+0000 7f9087fff700 1 --2- 172.21.15.143:0/3261017547 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f908806dbb0 0x7f90880ca470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.143:44812/0 (socket says 172.21.15.143:44812) 2023-11-08T19:31:18.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.884+0000 7f90877fe700 1 -- 172.21.15.143:0/3261017547 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f908806dbb0 msgr2=0x7f90880ca470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:18.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.884+0000 7f90877fe700 1 --2- 172.21.15.143:0/3261017547 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f908806dbb0 0x7f90880ca470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:18.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.884+0000 7f90877fe700 1 -- 172.21.15.143:0/3261017547 >> v1:172.21.15.174:6789/0 conn(0x7f908806df90 legacy=0x7f90881621f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:18.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.884+0000 7f90877fe700 1 -- 172.21.15.143:0/3261017547 --> v2:172.21.15.174:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90880cb520 con 0x7f90880668e0 2023-11-08T19:31:18.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.884+0000 7f9086ffd700 1 -- 172.21.15.143:0/3261017547 <== mon.1 v1:172.21.15.174:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 636430947 0 0) 0x7f90880ce7f0 con 0x7f908806df90 2023-11-08T19:31:18.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.884+0000 7f90877fe700 1 --2- 172.21.15.143:0/3261017547 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f90880668e0 0x7f90880ca9b0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f907c009840 tx=0x7f907c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4fc31b6f65715525 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:18.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.885+0000 7f9086ffd700 1 -- 172.21.15.143:0/3261017547 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f907c011830 con 0x7f90880668e0 2023-11-08T19:31:18.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.885+0000 7f9086ffd700 1 -- 172.21.15.143:0/3261017547 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f907c011990 con 0x7f90880668e0 2023-11-08T19:31:18.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.885+0000 7f9086ffd700 1 -- 172.21.15.143:0/3261017547 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f907c011d20 con 0x7f90880668e0 2023-11-08T19:31:18.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.885+0000 7f908e962700 1 -- 172.21.15.143:0/3261017547 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f90880668e0 msgr2=0x7f90880ca9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:18.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.885+0000 7f908e962700 1 --2- 172.21.15.143:0/3261017547 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f90880668e0 0x7f90880ca9b0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f907c009840 tx=0x7f907c009b50 comp rx=0 tx=0).stop 2023-11-08T19:31:18.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.885+0000 7f908e962700 1 -- 172.21.15.143:0/3261017547 shutdown_connections 2023-11-08T19:31:18.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.885+0000 7f908e962700 1 --2- 172.21.15.143:0/3261017547 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f908806dbb0 0x7f90880ca470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:18.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.885+0000 7f908e962700 1 --2- 172.21.15.143:0/3261017547 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f90880668e0 0x7f90880ca9b0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:18.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.885+0000 7f908e962700 1 -- 172.21.15.143:0/3261017547 >> 172.21.15.143:0/3261017547 conn(0x7f90880b1a20 msgr2=0x7f90880b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:18.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.885+0000 7f908e962700 1 -- 172.21.15.143:0/3261017547 shutdown_connections 2023-11-08T19:31:18.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.885+0000 7f908e962700 1 -- 172.21.15.143:0/3261017547 wait complete. 2023-11-08T19:31:18.897 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.886+0000 7f908e962700 1 Processor -- start 2023-11-08T19:31:18.897 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.887+0000 7f908e962700 1 -- start start 2023-11-08T19:31:18.898 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.887+0000 7f908e962700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f90880668e0 0x7f908815dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:18.898 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.887+0000 7f908e962700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f908806dbb0 0x7f90880c6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:18.898 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.887+0000 7f908e962700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f908806df90 0x7f9088160fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:18.899 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.887+0000 7f908e962700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90881f3c10 con 0x7f908806df90 2023-11-08T19:31:18.899 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.887+0000 7f908e962700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90881f3ef0 con 0x7f908806dbb0 2023-11-08T19:31:18.899 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.887+0000 7f908e962700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90881f41d0 con 0x7f90880668e0 2023-11-08T19:31:18.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.887+0000 7f908ceff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f908806df90 0x7f9088160fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:18.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.887+0000 7f908ceff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f908806df90 0x7f9088160fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57184/0 (socket says 172.21.15.143:57184) 2023-11-08T19:31:18.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.887+0000 7f908ceff700 1 -- 172.21.15.143:0/2137507859 learned_addr learned my addr 172.21.15.143:0/2137507859 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:31:18.901 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.888+0000 7f9087fff700 1 --2- 172.21.15.143:0/2137507859 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f90880668e0 0x7f908815dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:18.901 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.888+0000 7f90877fe700 1 --2- 172.21.15.143:0/2137507859 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f908806dbb0 0x7f90880c6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:18.901 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.888+0000 7f908ceff700 1 -- 172.21.15.143:0/2137507859 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f90880668e0 msgr2=0x7f908815dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:18.902 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.888+0000 7f908ceff700 1 --2- 172.21.15.143:0/2137507859 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f90880668e0 0x7f908815dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:18.902 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.888+0000 7f908ceff700 1 -- 172.21.15.143:0/2137507859 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f908806dbb0 msgr2=0x7f90880c6430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:18.902 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.888+0000 7f908ceff700 1 --2- 172.21.15.143:0/2137507859 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f908806dbb0 0x7f90880c6430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:18.902 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.888+0000 7f908ceff700 1 -- 172.21.15.143:0/2137507859 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f907c0094a0 con 0x7f908806df90 2023-11-08T19:31:18.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.888+0000 7f9087fff700 1 --2- 172.21.15.143:0/2137507859 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f90880668e0 0x7f908815dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:31:18.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.888+0000 7f90877fe700 1 --2- 172.21.15.143:0/2137507859 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f908806dbb0 0x7f90880c6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:31:18.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.888+0000 7f908ceff700 1 --2- 172.21.15.143:0/2137507859 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f908806df90 0x7f9088160fe0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f908000cb60 tx=0x7f908000cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:18.904 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.888+0000 7f9084ff9700 1 -- 172.21.15.143:0/2137507859 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f908000ab50 con 0x7f908806df90 2023-11-08T19:31:18.904 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.888+0000 7f9084ff9700 1 -- 172.21.15.143:0/2137507859 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f908000acb0 con 0x7f908806df90 2023-11-08T19:31:18.904 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.888+0000 7f9084ff9700 1 -- 172.21.15.143:0/2137507859 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9080007310 con 0x7f908806df90 2023-11-08T19:31:18.904 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.888+0000 7f908e962700 1 -- 172.21.15.143:0/2137507859 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90881f44b0 con 0x7f908806df90 2023-11-08T19:31:18.905 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.888+0000 7f908e962700 1 -- 172.21.15.143:0/2137507859 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90881f7e40 con 0x7f908806df90 2023-11-08T19:31:18.905 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.889+0000 7f908e962700 1 -- 172.21.15.143:0/2137507859 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f908806d1a0 con 0x7f908806df90 2023-11-08T19:31:18.905 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.889+0000 7f9084ff9700 1 -- 172.21.15.143:0/2137507859 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9080007470 con 0x7f908806df90 2023-11-08T19:31:18.905 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.890+0000 7f9084ff9700 1 --2- 172.21.15.143:0/2137507859 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f907406af00 0x7f907406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:18.906 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.890+0000 7f9084ff9700 1 -- 172.21.15.143:0/2137507859 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6651+0+0 (secure 0 0 0) 0x7f90800894a0 con 0x7f908806df90 2023-11-08T19:31:18.906 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.890+0000 7f9087fff700 1 --2- 172.21.15.143:0/2137507859 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f907406af00 0x7f907406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:18.907 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.890+0000 7f9087fff700 1 --2- 172.21.15.143:0/2137507859 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f907406af00 0x7f907406d3c0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f9078009c30 tx=0x7f9078008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:18.907 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:18.894+0000 7f9084ff9700 1 -- 172.21.15.143:0/2137507859 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9080054230 con 0x7f908806df90 2023-11-08T19:31:19.088 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.086+0000 7f908e962700 1 -- 172.21.15.143:0/2137507859 --> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f90881f4060 con 0x7f907406af00 2023-11-08T19:31:19.088 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.087+0000 7f9084ff9700 1 -- 172.21.15.143:0/2137507859 <== mgr.4099 v2:172.21.15.174:6832/4037280178 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f90881f4060 con 0x7f907406af00 2023-11-08T19:31:19.091 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.089+0000 7f908e962700 1 -- 172.21.15.143:0/2137507859 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f907406af00 msgr2=0x7f907406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:19.092 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.089+0000 7f908e962700 1 --2- 172.21.15.143:0/2137507859 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f907406af00 0x7f907406d3c0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f9078009c30 tx=0x7f9078008040 comp rx=0 tx=0).stop 2023-11-08T19:31:19.092 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.089+0000 7f908e962700 1 -- 172.21.15.143:0/2137507859 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f908806df90 msgr2=0x7f9088160fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:19.092 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.089+0000 7f908e962700 1 --2- 172.21.15.143:0/2137507859 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f908806df90 0x7f9088160fe0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f908000cb60 tx=0x7f908000cf20 comp rx=0 tx=0).stop 2023-11-08T19:31:19.092 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.089+0000 7f908e962700 1 -- 172.21.15.143:0/2137507859 shutdown_connections 2023-11-08T19:31:19.093 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.089+0000 7f908e962700 1 --2- 172.21.15.143:0/2137507859 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f907406af00 0x7f907406d3c0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:19.093 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.089+0000 7f908e962700 1 --2- 172.21.15.143:0/2137507859 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f90880668e0 0x7f908815dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:19.093 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.089+0000 7f908e962700 1 --2- 172.21.15.143:0/2137507859 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f908806df90 0x7f9088160fe0 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:19.093 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.089+0000 7f908e962700 1 --2- 172.21.15.143:0/2137507859 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f908806dbb0 0x7f90880c6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:19.094 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.089+0000 7f908e962700 1 -- 172.21.15.143:0/2137507859 >> 172.21.15.143:0/2137507859 conn(0x7f90880b1a20 msgr2=0x7f908815a900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:19.094 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.089+0000 7f908e962700 1 -- 172.21.15.143:0/2137507859 shutdown_connections 2023-11-08T19:31:19.094 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.089+0000 7f908e962700 1 -- 172.21.15.143:0/2137507859 wait complete. 2023-11-08T19:31:19.095 INFO:teuthology.orchestra.run.smithi143.stderr:instructed osd(s) 0 to deep-scrub 2023-11-08T19:31:19.103 INFO:tasks.ceph:Scrubbing osd.1 2023-11-08T19:31:19.103 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2023-11-08T19:31:19.229 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.227+0000 7fcb52828700 1 Processor -- start 2023-11-08T19:31:19.229 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.227+0000 7fcb52828700 1 -- start start 2023-11-08T19:31:19.232 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.228+0000 7fcb52828700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcb4c15dba0 0x7fcb4c06c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:19.232 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.228+0000 7fcb52828700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fcb4c06b240 0x7fcb4c0caff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:19.233 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.228+0000 7fcb52828700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fcb4c0cb530 0x7fcb4c0b22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:19.233 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.228+0000 7fcb52828700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb4c161890 con 0x7fcb4c0cb530 2023-11-08T19:31:19.233 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.228+0000 7fcb52828700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb4c06b740 con 0x7fcb4c06b240 2023-11-08T19:31:19.234 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.228+0000 7fcb52828700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb4c06ba20 con 0x7fcb4c15dba0 2023-11-08T19:31:19.234 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.228+0000 7fcb50dc5700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fcb4c0cb530 0x7fcb4c0b22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:19.234 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.228+0000 7fcb50dc5700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fcb4c0cb530 0x7fcb4c0b22e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57188/0 (socket says 172.21.15.143:57188) 2023-11-08T19:31:19.235 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.228+0000 7fcb50dc5700 1 -- 172.21.15.143:0/1879367774 learned_addr learned my addr 172.21.15.143:0/1879367774 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:31:19.235 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.228+0000 7fcb4bfff700 1 --2- 172.21.15.143:0/1879367774 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcb4c15dba0 0x7fcb4c06c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:19.235 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.228+0000 7fcb4b7fe700 1 --2- 172.21.15.143:0/1879367774 >> v2:172.21.15.174:3300/0 conn(0x7fcb4c06b240 0x7fcb4c0caff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:19.235 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.229+0000 7fcb50dc5700 1 -- 172.21.15.143:0/1879367774 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcb4c15dba0 msgr2=0x7fcb4c06c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:19.236 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.229+0000 7fcb50dc5700 1 --2- 172.21.15.143:0/1879367774 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcb4c15dba0 0x7fcb4c06c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:19.236 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.229+0000 7fcb50dc5700 1 -- 172.21.15.143:0/1879367774 >> v2:172.21.15.174:3300/0 conn(0x7fcb4c06b240 msgr2=0x7fcb4c0caff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:19.236 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.229+0000 7fcb50dc5700 1 --2- 172.21.15.143:0/1879367774 >> v2:172.21.15.174:3300/0 conn(0x7fcb4c06b240 0x7fcb4c0caff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:19.236 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.229+0000 7fcb50dc5700 1 -- 172.21.15.143:0/1879367774 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb4c06bd00 con 0x7fcb4c0cb530 2023-11-08T19:31:19.237 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.229+0000 7fcb4bfff700 1 --2- 172.21.15.143:0/1879367774 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcb4c15dba0 0x7fcb4c06c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:31:19.237 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.229+0000 7fcb50dc5700 1 --2- 172.21.15.143:0/1879367774 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcb4c0cb530 0x7fcb4c0b22e0 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c067370 tx=0x7fcb4400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e07946f0f275dc8 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:19.237 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.229+0000 7fcb4affd700 1 -- 172.21.15.143:0/1879367774 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb4400e070 con 0x7fcb4c0cb530 2023-11-08T19:31:19.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.229+0000 7fcb4affd700 1 -- 172.21.15.143:0/1879367774 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb4400b9c0 con 0x7fcb4c0cb530 2023-11-08T19:31:19.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.229+0000 7fcb4affd700 1 -- 172.21.15.143:0/1879367774 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb4400bd70 con 0x7fcb4c0cb530 2023-11-08T19:31:19.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.229+0000 7fcb52828700 1 -- 172.21.15.143:0/1879367774 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcb4c0cb530 msgr2=0x7fcb4c0b22e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:19.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.229+0000 7fcb52828700 1 --2- 172.21.15.143:0/1879367774 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcb4c0cb530 0x7fcb4c0b22e0 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c067370 tx=0x7fcb4400b330 comp rx=0 tx=0).stop 2023-11-08T19:31:19.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.229+0000 7fcb52828700 1 -- 172.21.15.143:0/1879367774 shutdown_connections 2023-11-08T19:31:19.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.229+0000 7fcb52828700 1 --2- 172.21.15.143:0/1879367774 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcb4c0cb530 0x7fcb4c0b22e0 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:19.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.229+0000 7fcb52828700 1 --2- 172.21.15.143:0/1879367774 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcb4c15dba0 0x7fcb4c06c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:19.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.229+0000 7fcb52828700 1 --2- 172.21.15.143:0/1879367774 >> v2:172.21.15.174:3300/0 conn(0x7fcb4c06b240 0x7fcb4c0caff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:19.240 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.229+0000 7fcb52828700 1 -- 172.21.15.143:0/1879367774 >> 172.21.15.143:0/1879367774 conn(0x7fcb4c0cd2e0 msgr2=0x7fcb4c0cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:19.240 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.229+0000 7fcb52828700 1 -- 172.21.15.143:0/1879367774 shutdown_connections 2023-11-08T19:31:19.240 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.230+0000 7fcb52828700 1 -- 172.21.15.143:0/1879367774 wait complete. 2023-11-08T19:31:19.241 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.231+0000 7fcb52828700 1 Processor -- start 2023-11-08T19:31:19.241 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.231+0000 7fcb52828700 1 -- start start 2023-11-08T19:31:19.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.232+0000 7fcb52828700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcb4c06b240 0x7fcb4c1519c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:19.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.232+0000 7fcb52828700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcb4c0cb530 0x7fcb4c0ca560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:19.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.232+0000 7fcb52828700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcb4c15dba0 0x7fcb4c160110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:19.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.232+0000 7fcb52828700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb4c15f070 con 0x7fcb4c06b240 2023-11-08T19:31:19.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.232+0000 7fcb52828700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb4c15f350 con 0x7fcb4c0cb530 2023-11-08T19:31:19.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.232+0000 7fcb52828700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb4c15f630 con 0x7fcb4c15dba0 2023-11-08T19:31:19.246 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.232+0000 7fcb4bfff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcb4c06b240 0x7fcb4c1519c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:19.246 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.232+0000 7fcb4bfff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcb4c06b240 0x7fcb4c1519c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57192/0 (socket says 172.21.15.143:57192) 2023-11-08T19:31:19.246 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.232+0000 7fcb4bfff700 1 -- 172.21.15.143:0/2945135097 learned_addr learned my addr 172.21.15.143:0/2945135097 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:31:19.246 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.232+0000 7fcb50dc5700 1 --2- 172.21.15.143:0/2945135097 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcb4c15dba0 0x7fcb4c160110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:19.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.232+0000 7fcb4b7fe700 1 --2- 172.21.15.143:0/2945135097 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcb4c0cb530 0x7fcb4c0ca560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:19.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.232+0000 7fcb4bfff700 1 -- 172.21.15.143:0/2945135097 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcb4c15dba0 msgr2=0x7fcb4c160110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:19.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.232+0000 7fcb4bfff700 1 --2- 172.21.15.143:0/2945135097 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcb4c15dba0 0x7fcb4c160110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:19.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.232+0000 7fcb4bfff700 1 -- 172.21.15.143:0/2945135097 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcb4c0cb530 msgr2=0x7fcb4c0ca560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:19.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.232+0000 7fcb4bfff700 1 --2- 172.21.15.143:0/2945135097 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcb4c0cb530 0x7fcb4c0ca560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:19.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.232+0000 7fcb4bfff700 1 -- 172.21.15.143:0/2945135097 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb44009b30 con 0x7fcb4c06b240 2023-11-08T19:31:19.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.232+0000 7fcb4bfff700 1 --2- 172.21.15.143:0/2945135097 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcb4c06b240 0x7fcb4c1519c0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7fcb3400e810 tx=0x7fcb3400eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:19.249 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.232+0000 7fcb48ff9700 1 -- 172.21.15.143:0/2945135097 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb3400c020 con 0x7fcb4c06b240 2023-11-08T19:31:19.249 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.232+0000 7fcb48ff9700 1 -- 172.21.15.143:0/2945135097 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb34004250 con 0x7fcb4c06b240 2023-11-08T19:31:19.249 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.232+0000 7fcb52828700 1 -- 172.21.15.143:0/2945135097 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb4c15f910 con 0x7fcb4c06b240 2023-11-08T19:31:19.249 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.232+0000 7fcb48ff9700 1 -- 172.21.15.143:0/2945135097 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb340036a0 con 0x7fcb4c06b240 2023-11-08T19:31:19.250 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.232+0000 7fcb52828700 1 -- 172.21.15.143:0/2945135097 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb4c0caaa0 con 0x7fcb4c06b240 2023-11-08T19:31:19.250 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.233+0000 7fcb52828700 1 -- 172.21.15.143:0/2945135097 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fcb4c06bfe0 con 0x7fcb4c06b240 2023-11-08T19:31:19.250 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.234+0000 7fcb48ff9700 1 -- 172.21.15.143:0/2945135097 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fcb34003800 con 0x7fcb4c06b240 2023-11-08T19:31:19.250 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.234+0000 7fcb48ff9700 1 --2- 172.21.15.143:0/2945135097 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fcb2c06af00 0x7fcb2c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:19.251 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.234+0000 7fcb48ff9700 1 -- 172.21.15.143:0/2945135097 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6651+0+0 (secure 0 0 0) 0x7fcb340892d0 con 0x7fcb4c06b240 2023-11-08T19:31:19.251 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.234+0000 7fcb48ff9700 1 --2- 172.21.15.143:0/2945135097 >> [v2:172.21.15.143:6800/787341397,v1:172.21.15.143:6801/787341397] conn(0x7fcb2c073a10 0x7fcb2c075e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:19.251 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.234+0000 7fcb48ff9700 1 -- 172.21.15.143:0/2945135097 --> [v2:172.21.15.143:6800/787341397,v1:172.21.15.143:6801/787341397] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcb2c076500 con 0x7fcb2c073a10 2023-11-08T19:31:19.252 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.234+0000 7fcb48ff9700 1 -- 172.21.15.143:0/2945135097 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_get_version_reply(handle=1 version=107) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcb3408b990 con 0x7fcb4c06b240 2023-11-08T19:31:19.252 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.234+0000 7fcb4b7fe700 1 --2- 172.21.15.143:0/2945135097 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fcb2c06af00 0x7fcb2c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:19.252 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.234+0000 7fcb50dc5700 1 --2- 172.21.15.143:0/2945135097 >> [v2:172.21.15.143:6800/787341397,v1:172.21.15.143:6801/787341397] conn(0x7fcb2c073a10 0x7fcb2c075e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:19.252 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.235+0000 7fcb50dc5700 1 --2- 172.21.15.143:0/2945135097 >> [v2:172.21.15.143:6800/787341397,v1:172.21.15.143:6801/787341397] conn(0x7fcb2c073a10 0x7fcb2c075e30 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:19.253 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.235+0000 7fcb4b7fe700 1 --2- 172.21.15.143:0/2945135097 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fcb2c06af00 0x7fcb2c06d3c0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c009950 tx=0x7fcb3c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:19.253 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.235+0000 7fcb48ff9700 1 -- 172.21.15.143:0/2945135097 <== osd.1 v2:172.21.15.143:6800/787341397 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26745 (crc 0 0 0) 0x7fcb2c076500 con 0x7fcb2c073a10 2023-11-08T19:31:19.254 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.250+0000 7fcb52828700 1 -- 172.21.15.143:0/2945135097 --> [v2:172.21.15.143:6800/787341397,v1:172.21.15.143:6801/787341397] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fcb4c0cacc0 con 0x7fcb2c073a10 2023-11-08T19:31:19.254 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.250+0000 7fcb48ff9700 1 -- 172.21.15.143:0/2945135097 <== osd.1 v2:172.21.15.143:6800/787341397 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fcb4c0cacc0 con 0x7fcb2c073a10 2023-11-08T19:31:19.255 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2023-11-08T19:31:19.255 INFO:teuthology.orchestra.run.smithi143.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-08T19:31:19.255 INFO:teuthology.orchestra.run.smithi143.stdout:} 2023-11-08T19:31:19.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.251+0000 7fcb52828700 1 -- 172.21.15.143:0/2945135097 >> [v2:172.21.15.143:6800/787341397,v1:172.21.15.143:6801/787341397] conn(0x7fcb2c073a10 msgr2=0x7fcb2c075e30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:19.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.251+0000 7fcb52828700 1 --2- 172.21.15.143:0/2945135097 >> [v2:172.21.15.143:6800/787341397,v1:172.21.15.143:6801/787341397] conn(0x7fcb2c073a10 0x7fcb2c075e30 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:19.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.251+0000 7fcb52828700 1 -- 172.21.15.143:0/2945135097 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fcb2c06af00 msgr2=0x7fcb2c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:19.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.251+0000 7fcb52828700 1 --2- 172.21.15.143:0/2945135097 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fcb2c06af00 0x7fcb2c06d3c0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c009950 tx=0x7fcb3c008040 comp rx=0 tx=0).stop 2023-11-08T19:31:19.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.251+0000 7fcb52828700 1 -- 172.21.15.143:0/2945135097 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcb4c06b240 msgr2=0x7fcb4c1519c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:19.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.251+0000 7fcb52828700 1 --2- 172.21.15.143:0/2945135097 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcb4c06b240 0x7fcb4c1519c0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7fcb3400e810 tx=0x7fcb3400eb20 comp rx=0 tx=0).stop 2023-11-08T19:31:19.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.251+0000 7fcb50dc5700 1 -- 172.21.15.143:0/2945135097 reap_dead start 2023-11-08T19:31:19.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.251+0000 7fcb52828700 1 -- 172.21.15.143:0/2945135097 shutdown_connections 2023-11-08T19:31:19.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.251+0000 7fcb52828700 1 -- 172.21.15.143:0/2945135097 >> 172.21.15.143:0/2945135097 conn(0x7fcb4c0cd2e0 msgr2=0x7fcb4c0c5fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:19.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.251+0000 7fcb52828700 1 -- 172.21.15.143:0/2945135097 shutdown_connections 2023-11-08T19:31:19.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.251+0000 7fcb52828700 1 -- 172.21.15.143:0/2945135097 wait complete. 2023-11-08T19:31:19.264 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2023-11-08T19:31:19.385 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.384+0000 7fa6b5121700 1 Processor -- start 2023-11-08T19:31:19.385 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.384+0000 7fa6b5121700 1 -- start start 2023-11-08T19:31:19.388 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.384+0000 7fa6b5121700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa6b015dd00 0x7fa6b00cca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:19.388 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.384+0000 7fa6b5121700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fa6b01618f0 0x7fa6b006ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:19.388 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.384+0000 7fa6b5121700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa6b006b150 0x7fa6b00b2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:19.389 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.384+0000 7fa6b5121700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6b006b600 con 0x7fa6b006b150 2023-11-08T19:31:19.389 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.384+0000 7fa6b5121700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6b00cad50 con 0x7fa6b01618f0 2023-11-08T19:31:19.389 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.384+0000 7fa6b5121700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6b00ccfb0 con 0x7fa6b015dd00 2023-11-08T19:31:19.389 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.384+0000 7fa6af59e700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa6b006b150 0x7fa6b00b2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:19.390 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.385+0000 7fa6af59e700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa6b006b150 0x7fa6b00b2210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57204/0 (socket says 172.21.15.143:57204) 2023-11-08T19:31:19.390 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.385+0000 7fa6af59e700 1 -- 172.21.15.143:0/1771841516 learned_addr learned my addr 172.21.15.143:0/1771841516 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:31:19.390 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.385+0000 7fa6aed9d700 1 --2- 172.21.15.143:0/1771841516 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa6b015dd00 0x7fa6b00cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:19.391 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.385+0000 7fa6af59e700 1 -- 172.21.15.143:0/1771841516 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa6b015dd00 msgr2=0x7fa6b00cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:19.391 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.385+0000 7fa6af59e700 1 --2- 172.21.15.143:0/1771841516 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa6b015dd00 0x7fa6b00cca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:19.391 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.385+0000 7fa6ae59c700 1 --2- 172.21.15.143:0/1771841516 >> v2:172.21.15.174:3300/0 conn(0x7fa6b01618f0 0x7fa6b006ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:19.392 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.385+0000 7fa6af59e700 1 -- 172.21.15.143:0/1771841516 >> v2:172.21.15.174:3300/0 conn(0x7fa6b01618f0 msgr2=0x7fa6b006ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:19.392 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.385+0000 7fa6af59e700 1 --2- 172.21.15.143:0/1771841516 >> v2:172.21.15.174:3300/0 conn(0x7fa6b01618f0 0x7fa6b006ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:19.392 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.385+0000 7fa6af59e700 1 -- 172.21.15.143:0/1771841516 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6b0161df0 con 0x7fa6b006b150 2023-11-08T19:31:19.392 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.385+0000 7fa6af59e700 1 --2- 172.21.15.143:0/1771841516 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6b006b150 0x7fa6b00b2210 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7fa6b0071c00 tx=0x7fa6a800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcdf69502b7c15d2 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:19.393 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.385+0000 7fa6add9b700 1 -- 172.21.15.143:0/1771841516 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6a800e070 con 0x7fa6b006b150 2023-11-08T19:31:19.393 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.385+0000 7fa6add9b700 1 -- 172.21.15.143:0/1771841516 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6a800b9c0 con 0x7fa6b006b150 2023-11-08T19:31:19.393 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.385+0000 7fa6add9b700 1 -- 172.21.15.143:0/1771841516 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6a800bd70 con 0x7fa6b006b150 2023-11-08T19:31:19.393 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.386+0000 7fa6b5121700 1 -- 172.21.15.143:0/1771841516 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6b006b150 msgr2=0x7fa6b00b2210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:19.394 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.386+0000 7fa6b5121700 1 --2- 172.21.15.143:0/1771841516 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6b006b150 0x7fa6b00b2210 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7fa6b0071c00 tx=0x7fa6a800b330 comp rx=0 tx=0).stop 2023-11-08T19:31:19.394 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.386+0000 7fa6b5121700 1 -- 172.21.15.143:0/1771841516 shutdown_connections 2023-11-08T19:31:19.394 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.386+0000 7fa6b5121700 1 --2- 172.21.15.143:0/1771841516 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6b006b150 0x7fa6b00b2210 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:19.395 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.386+0000 7fa6b5121700 1 --2- 172.21.15.143:0/1771841516 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa6b015dd00 0x7fa6b00cca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:19.395 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.386+0000 7fa6b5121700 1 --2- 172.21.15.143:0/1771841516 >> v2:172.21.15.174:3300/0 conn(0x7fa6b01618f0 0x7fa6b006ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:19.395 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.386+0000 7fa6b5121700 1 -- 172.21.15.143:0/1771841516 >> 172.21.15.143:0/1771841516 conn(0x7fa6b0099ab0 msgr2=0x7fa6b00d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:19.395 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.386+0000 7fa6b5121700 1 -- 172.21.15.143:0/1771841516 shutdown_connections 2023-11-08T19:31:19.396 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.386+0000 7fa6b5121700 1 -- 172.21.15.143:0/1771841516 wait complete. 2023-11-08T19:31:19.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.388+0000 7fa6b5121700 1 Processor -- start 2023-11-08T19:31:19.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.388+0000 7fa6b5121700 1 -- start start 2023-11-08T19:31:19.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.388+0000 7fa6b5121700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6b006b150 0x7fa6b0152100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:19.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.388+0000 7fa6b5121700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa6b015dd00 0x7fa6b01f6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:19.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.389+0000 7fa6b5121700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa6b01618f0 0x7fa6b00c9c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:19.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.389+0000 7fa6b5121700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6b00c5c80 con 0x7fa6b006b150 2023-11-08T19:31:19.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.389+0000 7fa6b5121700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6b00c5670 con 0x7fa6b015dd00 2023-11-08T19:31:19.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.389+0000 7fa6b5121700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6b01f7de0 con 0x7fa6b01618f0 2023-11-08T19:31:19.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.389+0000 7fa6aed9d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6b006b150 0x7fa6b0152100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:19.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.389+0000 7fa6aed9d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6b006b150 0x7fa6b0152100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57206/0 (socket says 172.21.15.143:57206) 2023-11-08T19:31:19.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.389+0000 7fa6aed9d700 1 -- 172.21.15.143:0/2666393631 learned_addr learned my addr 172.21.15.143:0/2666393631 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:31:19.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.389+0000 7fa6af59e700 1 --2- 172.21.15.143:0/2666393631 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa6b01618f0 0x7fa6b00c9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:19.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.389+0000 7fa6ae59c700 1 --2- 172.21.15.143:0/2666393631 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa6b015dd00 0x7fa6b01f6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:19.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.389+0000 7fa6aed9d700 1 -- 172.21.15.143:0/2666393631 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa6b01618f0 msgr2=0x7fa6b00c9c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:19.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.389+0000 7fa6aed9d700 1 --2- 172.21.15.143:0/2666393631 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa6b01618f0 0x7fa6b00c9c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:19.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.389+0000 7fa6aed9d700 1 -- 172.21.15.143:0/2666393631 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa6b015dd00 msgr2=0x7fa6b01f6eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:19.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.389+0000 7fa6aed9d700 1 --2- 172.21.15.143:0/2666393631 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa6b015dd00 0x7fa6b01f6eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:19.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.389+0000 7fa6aed9d700 1 -- 172.21.15.143:0/2666393631 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6a8009b30 con 0x7fa6b006b150 2023-11-08T19:31:19.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.389+0000 7fa6aed9d700 1 --2- 172.21.15.143:0/2666393631 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6b006b150 0x7fa6b0152100 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7fa6a000c7f0 tx=0x7fa6a000cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:19.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.389+0000 7fa6977fe700 1 -- 172.21.15.143:0/2666393631 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6a0011800 con 0x7fa6b006b150 2023-11-08T19:31:19.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.389+0000 7fa6977fe700 1 -- 172.21.15.143:0/2666393631 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6a0011960 con 0x7fa6b006b150 2023-11-08T19:31:19.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.389+0000 7fa6b5121700 1 -- 172.21.15.143:0/2666393631 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6b0152640 con 0x7fa6b006b150 2023-11-08T19:31:19.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.389+0000 7fa6977fe700 1 -- 172.21.15.143:0/2666393631 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6a0011d10 con 0x7fa6b006b150 2023-11-08T19:31:19.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.389+0000 7fa6b5121700 1 -- 172.21.15.143:0/2666393631 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6b01f7b00 con 0x7fa6b006b150 2023-11-08T19:31:19.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.390+0000 7fa6b5121700 1 -- 172.21.15.143:0/2666393631 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6b00aff60 con 0x7fa6b006b150 2023-11-08T19:31:19.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.391+0000 7fa6977fe700 1 -- 172.21.15.143:0/2666393631 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa6a0018360 con 0x7fa6b006b150 2023-11-08T19:31:19.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.391+0000 7fa6977fe700 1 --2- 172.21.15.143:0/2666393631 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa69c06af00 0x7fa69c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:19.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.391+0000 7fa6977fe700 1 -- 172.21.15.143:0/2666393631 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6651+0+0 (secure 0 0 0) 0x7fa6a00898b0 con 0x7fa6b006b150 2023-11-08T19:31:19.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.391+0000 7fa6ae59c700 1 --2- 172.21.15.143:0/2666393631 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa69c06af00 0x7fa69c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:19.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.392+0000 7fa6ae59c700 1 --2- 172.21.15.143:0/2666393631 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa69c06af00 0x7fa69c06d3c0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fa6b0161820 tx=0x7fa6a4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:19.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.395+0000 7fa6977fe700 1 -- 172.21.15.143:0/2666393631 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa6a00545c0 con 0x7fa6b006b150 2023-11-08T19:31:19.586 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.584+0000 7fa6b5121700 1 -- 172.21.15.143:0/2666393631 --> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7fa6b00cf810 con 0x7fa69c06af00 2023-11-08T19:31:19.586 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.585+0000 7fa6977fe700 1 -- 172.21.15.143:0/2666393631 <== mgr.4099 v2:172.21.15.174:6832/4037280178 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fa6b00cf810 con 0x7fa69c06af00 2023-11-08T19:31:19.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.587+0000 7fa6b5121700 1 -- 172.21.15.143:0/2666393631 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa69c06af00 msgr2=0x7fa69c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:19.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.587+0000 7fa6b5121700 1 --2- 172.21.15.143:0/2666393631 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa69c06af00 0x7fa69c06d3c0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fa6b0161820 tx=0x7fa6a4008040 comp rx=0 tx=0).stop 2023-11-08T19:31:19.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.587+0000 7fa6b5121700 1 -- 172.21.15.143:0/2666393631 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6b006b150 msgr2=0x7fa6b0152100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:19.590 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.587+0000 7fa6b5121700 1 --2- 172.21.15.143:0/2666393631 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6b006b150 0x7fa6b0152100 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7fa6a000c7f0 tx=0x7fa6a000cbb0 comp rx=0 tx=0).stop 2023-11-08T19:31:19.590 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.587+0000 7fa6b5121700 1 -- 172.21.15.143:0/2666393631 shutdown_connections 2023-11-08T19:31:19.590 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.587+0000 7fa6b5121700 1 --2- 172.21.15.143:0/2666393631 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa69c06af00 0x7fa69c06d3c0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:19.591 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.587+0000 7fa6b5121700 1 --2- 172.21.15.143:0/2666393631 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa6b01618f0 0x7fa6b00c9c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:19.591 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.587+0000 7fa6b5121700 1 --2- 172.21.15.143:0/2666393631 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6b006b150 0x7fa6b0152100 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:19.591 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.587+0000 7fa6b5121700 1 --2- 172.21.15.143:0/2666393631 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa6b015dd00 0x7fa6b01f6eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:19.591 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.587+0000 7fa6b5121700 1 -- 172.21.15.143:0/2666393631 >> 172.21.15.143:0/2666393631 conn(0x7fa6b0099ab0 msgr2=0x7fa6b00ad5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:19.592 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.587+0000 7fa6b5121700 1 -- 172.21.15.143:0/2666393631 shutdown_connections 2023-11-08T19:31:19.592 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.587+0000 7fa6b5121700 1 -- 172.21.15.143:0/2666393631 wait complete. 2023-11-08T19:31:19.592 INFO:teuthology.orchestra.run.smithi143.stderr:instructed osd(s) 1 to deep-scrub 2023-11-08T19:31:19.601 INFO:tasks.ceph:Scrubbing osd.2 2023-11-08T19:31:19.601 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2023-11-08T19:31:19.726 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.725+0000 7efbf6bfd700 1 Processor -- start 2023-11-08T19:31:19.726 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.725+0000 7efbf6bfd700 1 -- start start 2023-11-08T19:31:19.729 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.725+0000 7efbf6bfd700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7efbf015dba0 0x7efbf006c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:19.729 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.725+0000 7efbf6bfd700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7efbf006b240 0x7efbf00caff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:19.729 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.725+0000 7efbf6bfd700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efbf00cb530 0x7efbf00b22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:19.730 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.725+0000 7efbf6bfd700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efbf0161890 con 0x7efbf015dba0 2023-11-08T19:31:19.730 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.725+0000 7efbf6bfd700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efbf006b740 con 0x7efbf006b240 2023-11-08T19:31:19.730 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.725+0000 7efbf6bfd700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efbf006ba20 con 0x7efbf00cb530 2023-11-08T19:31:19.730 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.726+0000 7efbf4999700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7efbf015dba0 0x7efbf006c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:19.731 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.726+0000 7efbf4999700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7efbf015dba0 0x7efbf006c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57214/0 (socket says 172.21.15.143:57214) 2023-11-08T19:31:19.731 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.726+0000 7efbf4999700 1 -- 172.21.15.143:0/221357429 learned_addr learned my addr 172.21.15.143:0/221357429 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:31:19.731 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.726+0000 7efbeffff700 1 --2- 172.21.15.143:0/221357429 >> v2:172.21.15.174:3300/0 conn(0x7efbf006b240 0x7efbf00caff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:19.731 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.726+0000 7efbf519a700 1 --2- 172.21.15.143:0/221357429 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efbf00cb530 0x7efbf00b22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:19.732 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.726+0000 7efbf4999700 1 -- 172.21.15.143:0/221357429 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efbf00cb530 msgr2=0x7efbf00b22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:19.732 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.726+0000 7efbf4999700 1 --2- 172.21.15.143:0/221357429 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efbf00cb530 0x7efbf00b22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:19.732 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.726+0000 7efbf4999700 1 -- 172.21.15.143:0/221357429 >> v2:172.21.15.174:3300/0 conn(0x7efbf006b240 msgr2=0x7efbf00caff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:19.733 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.726+0000 7efbf4999700 1 --2- 172.21.15.143:0/221357429 >> v2:172.21.15.174:3300/0 conn(0x7efbf006b240 0x7efbf00caff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:19.733 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.726+0000 7efbf4999700 1 -- 172.21.15.143:0/221357429 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efbf006bd00 con 0x7efbf015dba0 2023-11-08T19:31:19.733 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.726+0000 7efbeffff700 1 --2- 172.21.15.143:0/221357429 >> v2:172.21.15.174:3300/0 conn(0x7efbf006b240 0x7efbf00caff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:31:19.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.726+0000 7efbf4999700 1 --2- 172.21.15.143:0/221357429 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efbf015dba0 0x7efbf006c610 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7efbf00d0810 tx=0x7efbe4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93666566017ee439 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:19.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.726+0000 7efbef7fe700 1 -- 172.21.15.143:0/221357429 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbe4011830 con 0x7efbf015dba0 2023-11-08T19:31:19.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.726+0000 7efbef7fe700 1 -- 172.21.15.143:0/221357429 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efbe4011990 con 0x7efbf015dba0 2023-11-08T19:31:19.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.726+0000 7efbef7fe700 1 -- 172.21.15.143:0/221357429 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbe4011d20 con 0x7efbf015dba0 2023-11-08T19:31:19.735 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.727+0000 7efbf6bfd700 1 -- 172.21.15.143:0/221357429 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efbf015dba0 msgr2=0x7efbf006c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:19.735 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.727+0000 7efbf6bfd700 1 --2- 172.21.15.143:0/221357429 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efbf015dba0 0x7efbf006c610 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7efbf00d0810 tx=0x7efbe4009bd0 comp rx=0 tx=0).stop 2023-11-08T19:31:19.735 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.727+0000 7efbf6bfd700 1 -- 172.21.15.143:0/221357429 shutdown_connections 2023-11-08T19:31:19.736 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.727+0000 7efbf6bfd700 1 --2- 172.21.15.143:0/221357429 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efbf00cb530 0x7efbf00b22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:19.736 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.727+0000 7efbf6bfd700 1 --2- 172.21.15.143:0/221357429 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efbf015dba0 0x7efbf006c610 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:19.736 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.727+0000 7efbf6bfd700 1 --2- 172.21.15.143:0/221357429 >> v2:172.21.15.174:3300/0 conn(0x7efbf006b240 0x7efbf00caff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:19.736 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.727+0000 7efbf6bfd700 1 -- 172.21.15.143:0/221357429 >> 172.21.15.143:0/221357429 conn(0x7efbf00cd2e0 msgr2=0x7efbf00cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:19.737 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.727+0000 7efbf6bfd700 1 -- 172.21.15.143:0/221357429 shutdown_connections 2023-11-08T19:31:19.737 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.727+0000 7efbf6bfd700 1 -- 172.21.15.143:0/221357429 wait complete. 2023-11-08T19:31:19.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.728+0000 7efbf6bfd700 1 Processor -- start 2023-11-08T19:31:19.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.728+0000 7efbf6bfd700 1 -- start start 2023-11-08T19:31:19.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.729+0000 7efbf6bfd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efbf006b240 0x7efbf0151be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:19.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.729+0000 7efbf6bfd700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efbf00cb530 0x7efbf00b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:19.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.729+0000 7efbf6bfd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efbf015dba0 0x7efbf0072030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:19.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.729+0000 7efbf6bfd700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efbf0152120 con 0x7efbf015dba0 2023-11-08T19:31:19.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.729+0000 7efbf6bfd700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efbf015e7e0 con 0x7efbf006b240 2023-11-08T19:31:19.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.729+0000 7efbf6bfd700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efbf015eac0 con 0x7efbf00cb530 2023-11-08T19:31:19.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.729+0000 7efbf519a700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efbf015dba0 0x7efbf0072030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:19.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.729+0000 7efbf519a700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efbf015dba0 0x7efbf0072030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57222/0 (socket says 172.21.15.143:57222) 2023-11-08T19:31:19.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.729+0000 7efbf519a700 1 -- 172.21.15.143:0/3515301902 learned_addr learned my addr 172.21.15.143:0/3515301902 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:31:19.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.729+0000 7efbeffff700 1 --2- 172.21.15.143:0/3515301902 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efbf00cb530 0x7efbf00b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:19.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.729+0000 7efbf4999700 1 --2- 172.21.15.143:0/3515301902 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efbf006b240 0x7efbf0151be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:19.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.729+0000 7efbf519a700 1 -- 172.21.15.143:0/3515301902 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efbf00cb530 msgr2=0x7efbf00b3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:19.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.729+0000 7efbf519a700 1 --2- 172.21.15.143:0/3515301902 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efbf00cb530 0x7efbf00b3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:19.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.729+0000 7efbf519a700 1 -- 172.21.15.143:0/3515301902 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efbf006b240 msgr2=0x7efbf0151be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:19.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.729+0000 7efbf519a700 1 --2- 172.21.15.143:0/3515301902 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efbf006b240 0x7efbf0151be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:19.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.729+0000 7efbf519a700 1 -- 172.21.15.143:0/3515301902 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efbe4009550 con 0x7efbf015dba0 2023-11-08T19:31:19.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.730+0000 7efbf519a700 1 --2- 172.21.15.143:0/3515301902 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efbf015dba0 0x7efbf0072030 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7efbe800efc0 tx=0x7efbe800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:19.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.730+0000 7efbed7fa700 1 -- 172.21.15.143:0/3515301902 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbe8009ea0 con 0x7efbf015dba0 2023-11-08T19:31:19.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.730+0000 7efbed7fa700 1 -- 172.21.15.143:0/3515301902 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efbe800f040 con 0x7efbf015dba0 2023-11-08T19:31:19.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.730+0000 7efbf6bfd700 1 -- 172.21.15.143:0/3515301902 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efbf015eda0 con 0x7efbf015dba0 2023-11-08T19:31:19.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.730+0000 7efbed7fa700 1 -- 172.21.15.143:0/3515301902 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbe800ced0 con 0x7efbf015dba0 2023-11-08T19:31:19.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.730+0000 7efbf6bfd700 1 -- 172.21.15.143:0/3515301902 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efbf015f360 con 0x7efbf015dba0 2023-11-08T19:31:19.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.730+0000 7efbf6bfd700 1 -- 172.21.15.143:0/3515301902 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7efbf006bfe0 con 0x7efbf015dba0 2023-11-08T19:31:19.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.731+0000 7efbed7fa700 1 -- 172.21.15.143:0/3515301902 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7efbe8014480 con 0x7efbf015dba0 2023-11-08T19:31:19.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.732+0000 7efbed7fa700 1 --2- 172.21.15.143:0/3515301902 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7efbd406af00 0x7efbd406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:19.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.732+0000 7efbed7fa700 1 -- 172.21.15.143:0/3515301902 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6651+0+0 (secure 0 0 0) 0x7efbe808a930 con 0x7efbf015dba0 2023-11-08T19:31:19.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.732+0000 7efbed7fa700 1 --2- 172.21.15.143:0/3515301902 >> [v2:172.21.15.143:6820/2301325597,v1:172.21.15.143:6822/2301325597] conn(0x7efbd40738a0 0x7efbd4075cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:19.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.732+0000 7efbed7fa700 1 -- 172.21.15.143:0/3515301902 --> [v2:172.21.15.143:6820/2301325597,v1:172.21.15.143:6822/2301325597] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7efbd4076390 con 0x7efbd40738a0 2023-11-08T19:31:19.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.732+0000 7efbed7fa700 1 -- 172.21.15.143:0/3515301902 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_get_version_reply(handle=1 version=107) v2 ==== 24+0+0 (secure 0 0 0) 0x7efbe808ad80 con 0x7efbf015dba0 2023-11-08T19:31:19.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.732+0000 7efbf4999700 1 --2- 172.21.15.143:0/3515301902 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7efbd406af00 0x7efbd406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:19.750 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.732+0000 7efbeffff700 1 --2- 172.21.15.143:0/3515301902 >> [v2:172.21.15.143:6820/2301325597,v1:172.21.15.143:6822/2301325597] conn(0x7efbd40738a0 0x7efbd4075cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:19.750 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.732+0000 7efbeffff700 1 --2- 172.21.15.143:0/3515301902 >> [v2:172.21.15.143:6820/2301325597,v1:172.21.15.143:6822/2301325597] conn(0x7efbd40738a0 0x7efbd4075cc0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:19.750 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.732+0000 7efbf4999700 1 --2- 172.21.15.143:0/3515301902 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7efbd406af00 0x7efbd406d3c0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7efbe4014040 tx=0x7efbe4008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:19.751 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.733+0000 7efbed7fa700 1 -- 172.21.15.143:0/3515301902 <== osd.2 v2:172.21.15.143:6820/2301325597 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26745 (crc 0 0 0) 0x7efbd4076390 con 0x7efbd40738a0 2023-11-08T19:31:19.753 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.747+0000 7efbf6bfd700 1 -- 172.21.15.143:0/3515301902 --> [v2:172.21.15.143:6820/2301325597,v1:172.21.15.143:6822/2301325597] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7efbf015f580 con 0x7efbd40738a0 2023-11-08T19:31:19.753 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.748+0000 7efbed7fa700 1 -- 172.21.15.143:0/3515301902 <== osd.2 v2:172.21.15.143:6820/2301325597 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7efbf015f580 con 0x7efbd40738a0 2023-11-08T19:31:19.754 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2023-11-08T19:31:19.754 INFO:teuthology.orchestra.run.smithi143.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-08T19:31:19.755 INFO:teuthology.orchestra.run.smithi143.stdout:} 2023-11-08T19:31:19.756 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.748+0000 7efbf6bfd700 1 -- 172.21.15.143:0/3515301902 >> [v2:172.21.15.143:6820/2301325597,v1:172.21.15.143:6822/2301325597] conn(0x7efbd40738a0 msgr2=0x7efbd4075cc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:19.756 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.748+0000 7efbf6bfd700 1 --2- 172.21.15.143:0/3515301902 >> [v2:172.21.15.143:6820/2301325597,v1:172.21.15.143:6822/2301325597] conn(0x7efbd40738a0 0x7efbd4075cc0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:19.756 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.748+0000 7efbf6bfd700 1 -- 172.21.15.143:0/3515301902 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7efbd406af00 msgr2=0x7efbd406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:19.757 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.748+0000 7efbf6bfd700 1 --2- 172.21.15.143:0/3515301902 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7efbd406af00 0x7efbd406d3c0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7efbe4014040 tx=0x7efbe4008030 comp rx=0 tx=0).stop 2023-11-08T19:31:19.757 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.748+0000 7efbf6bfd700 1 -- 172.21.15.143:0/3515301902 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efbf015dba0 msgr2=0x7efbf0072030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:19.757 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.748+0000 7efbf6bfd700 1 --2- 172.21.15.143:0/3515301902 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efbf015dba0 0x7efbf0072030 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7efbe800efc0 tx=0x7efbe800c3a0 comp rx=0 tx=0).stop 2023-11-08T19:31:19.758 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.748+0000 7efbf519a700 1 -- 172.21.15.143:0/3515301902 reap_dead start 2023-11-08T19:31:19.758 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.748+0000 7efbf6bfd700 1 -- 172.21.15.143:0/3515301902 shutdown_connections 2023-11-08T19:31:19.759 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.748+0000 7efbf6bfd700 1 -- 172.21.15.143:0/3515301902 >> 172.21.15.143:0/3515301902 conn(0x7efbf00cd2e0 msgr2=0x7efbf00ce970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:19.759 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.748+0000 7efbf6bfd700 1 -- 172.21.15.143:0/3515301902 shutdown_connections 2023-11-08T19:31:19.759 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.748+0000 7efbf6bfd700 1 -- 172.21.15.143:0/3515301902 wait complete. 2023-11-08T19:31:19.761 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2023-11-08T19:31:19.882 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.881+0000 7f6e53e22700 1 Processor -- start 2023-11-08T19:31:19.883 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.881+0000 7f6e53e22700 1 -- start start 2023-11-08T19:31:19.885 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.882+0000 7f6e53e22700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f6e4c15dd00 0x7f6e4c0cca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:19.886 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.882+0000 7f6e53e22700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6e4c1618f0 0x7f6e4c06ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:19.886 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.882+0000 7f6e53e22700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e4c0ccfb0 con 0x7f6e4c06b150 2023-11-08T19:31:19.886 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.882+0000 7f6e53e22700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e4c06b600 con 0x7f6e4c15dd00 2023-11-08T19:31:19.887 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.882+0000 7f6e53e22700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4c0cad50 con 0x7f6e4c1618f0 2023-11-08T19:31:19.887 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.882+0000 7f6e51bbe700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f6e4c15dd00 0x7f6e4c0cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:19.887 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.882+0000 7f6e523bf700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f6e4c06b150 0x7f6e4c0b2210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:53384/0 (socket says 172.21.15.143:53384) 2023-11-08T19:31:19.888 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.882+0000 7f6e523bf700 1 -- 172.21.15.143:0/3204534037 learned_addr learned my addr 172.21.15.143:0/3204534037 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:31:19.888 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.882+0000 7f6e513bd700 1 --2- 172.21.15.143:0/3204534037 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6e4c1618f0 0x7f6e4c06ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:19.888 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.883+0000 7f6e51bbe700 1 -- 172.21.15.143:0/3204534037 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6e4c1618f0 msgr2=0x7f6e4c06ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:19.888 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.883+0000 7f6e51bbe700 1 --2- 172.21.15.143:0/3204534037 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6e4c1618f0 0x7f6e4c06ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:19.889 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.883+0000 7f6e51bbe700 1 -- 172.21.15.143:0/3204534037 >> v1:172.21.15.143:6789/0 conn(0x7f6e4c06b150 legacy=0x7f6e4c0b2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:19.889 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.883+0000 7f6e51bbe700 1 -- 172.21.15.143:0/3204534037 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e4c161df0 con 0x7f6e4c15dd00 2023-11-08T19:31:19.889 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.883+0000 7f6e50bbc700 1 -- 172.21.15.143:0/3204534037 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4047511992 0 0) 0x7f6e4c0ccfb0 con 0x7f6e4c06b150 2023-11-08T19:31:19.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.883+0000 7f6e51bbe700 1 --2- 172.21.15.143:0/3204534037 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6e4c15dd00 0x7f6e4c0cca70 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c009820 tx=0x7f6e3c009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27e491f0fec12ee2 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:19.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.883+0000 7f6e50bbc700 1 -- 172.21.15.143:0/3204534037 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e3c005e30 con 0x7f6e4c15dd00 2023-11-08T19:31:19.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.883+0000 7f6e50bbc700 1 -- 172.21.15.143:0/3204534037 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e3c006030 con 0x7f6e4c15dd00 2023-11-08T19:31:19.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.883+0000 7f6e50bbc700 1 -- 172.21.15.143:0/3204534037 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e3c0063e0 con 0x7f6e4c15dd00 2023-11-08T19:31:19.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.883+0000 7f6e53e22700 1 -- 172.21.15.143:0/3204534037 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6e4c15dd00 msgr2=0x7f6e4c0cca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:19.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.883+0000 7f6e53e22700 1 --2- 172.21.15.143:0/3204534037 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6e4c15dd00 0x7f6e4c0cca70 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c009820 tx=0x7f6e3c009b30 comp rx=0 tx=0).stop 2023-11-08T19:31:19.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.884+0000 7f6e53e22700 1 -- 172.21.15.143:0/3204534037 shutdown_connections 2023-11-08T19:31:19.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.884+0000 7f6e53e22700 1 --2- 172.21.15.143:0/3204534037 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6e4c15dd00 0x7f6e4c0cca70 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:19.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.884+0000 7f6e53e22700 1 --2- 172.21.15.143:0/3204534037 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6e4c1618f0 0x7f6e4c06ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:19.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.884+0000 7f6e53e22700 1 -- 172.21.15.143:0/3204534037 >> 172.21.15.143:0/3204534037 conn(0x7f6e4c099ab0 msgr2=0x7f6e4c0d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:19.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.884+0000 7f6e53e22700 1 -- 172.21.15.143:0/3204534037 shutdown_connections 2023-11-08T19:31:19.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.884+0000 7f6e53e22700 1 -- 172.21.15.143:0/3204534037 wait complete. 2023-11-08T19:31:19.896 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.885+0000 7f6e53e22700 1 Processor -- start 2023-11-08T19:31:19.896 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.885+0000 7f6e53e22700 1 -- start start 2023-11-08T19:31:19.897 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.886+0000 7f6e53e22700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6e4c06b150 0x7f6e4c0c9840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:19.897 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.886+0000 7f6e53e22700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6e4c15dd00 0x7f6e4c0b5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:19.897 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.886+0000 7f6e53e22700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6e4c1618f0 0x7f6e4c1f30b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:19.898 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.886+0000 7f6e53e22700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4c0c9d80 con 0x7f6e4c1618f0 2023-11-08T19:31:19.898 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.886+0000 7f6e53e22700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4c0b6230 con 0x7f6e4c06b150 2023-11-08T19:31:19.898 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.886+0000 7f6e53e22700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4c0b50b0 con 0x7f6e4c15dd00 2023-11-08T19:31:19.898 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.886+0000 7f6e523bf700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6e4c1618f0 0x7f6e4c1f30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:19.899 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.886+0000 7f6e523bf700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6e4c1618f0 0x7f6e4c1f30b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57234/0 (socket says 172.21.15.143:57234) 2023-11-08T19:31:19.899 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.886+0000 7f6e523bf700 1 -- 172.21.15.143:0/2530284842 learned_addr learned my addr 172.21.15.143:0/2530284842 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:31:19.899 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.886+0000 7f6e513bd700 1 --2- 172.21.15.143:0/2530284842 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6e4c15dd00 0x7f6e4c0b5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:19.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.886+0000 7f6e51bbe700 1 --2- 172.21.15.143:0/2530284842 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6e4c06b150 0x7f6e4c0c9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:19.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.886+0000 7f6e523bf700 1 -- 172.21.15.143:0/2530284842 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6e4c15dd00 msgr2=0x7f6e4c0b5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:19.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.887+0000 7f6e523bf700 1 --2- 172.21.15.143:0/2530284842 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6e4c15dd00 0x7f6e4c0b5cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:19.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.887+0000 7f6e523bf700 1 -- 172.21.15.143:0/2530284842 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6e4c06b150 msgr2=0x7f6e4c0c9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:19.901 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.887+0000 7f6e523bf700 1 --2- 172.21.15.143:0/2530284842 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6e4c06b150 0x7f6e4c0c9840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:19.901 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.887+0000 7f6e523bf700 1 -- 172.21.15.143:0/2530284842 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e3c009430 con 0x7f6e4c1618f0 2023-11-08T19:31:19.901 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.887+0000 7f6e51bbe700 1 --2- 172.21.15.143:0/2530284842 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6e4c06b150 0x7f6e4c0c9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:31:19.901 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.887+0000 7f6e523bf700 1 --2- 172.21.15.143:0/2530284842 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6e4c1618f0 0x7f6e4c1f30b0 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f6e4400cc70 tx=0x7f6e4400cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:19.902 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.887+0000 7f6e427fc700 1 -- 172.21.15.143:0/2530284842 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e4400ab70 con 0x7f6e4c1618f0 2023-11-08T19:31:19.902 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.887+0000 7f6e427fc700 1 -- 172.21.15.143:0/2530284842 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e4400acf0 con 0x7f6e4c1618f0 2023-11-08T19:31:19.902 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.887+0000 7f6e53e22700 1 -- 172.21.15.143:0/2530284842 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e4c1f35f0 con 0x7f6e4c1618f0 2023-11-08T19:31:19.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.887+0000 7f6e427fc700 1 -- 172.21.15.143:0/2530284842 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e44007310 con 0x7f6e4c1618f0 2023-11-08T19:31:19.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.887+0000 7f6e53e22700 1 -- 172.21.15.143:0/2530284842 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e4c0b4a60 con 0x7f6e4c1618f0 2023-11-08T19:31:19.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.888+0000 7f6e53e22700 1 -- 172.21.15.143:0/2530284842 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e4c0aff60 con 0x7f6e4c1618f0 2023-11-08T19:31:19.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.888+0000 7f6e427fc700 1 -- 172.21.15.143:0/2530284842 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6e44007470 con 0x7f6e4c1618f0 2023-11-08T19:31:19.904 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.889+0000 7f6e427fc700 1 --2- 172.21.15.143:0/2530284842 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f6e3006af00 0x7f6e3006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:19.904 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.889+0000 7f6e427fc700 1 -- 172.21.15.143:0/2530284842 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6651+0+0 (secure 0 0 0) 0x7f6e440897e0 con 0x7f6e4c1618f0 2023-11-08T19:31:19.904 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.889+0000 7f6e51bbe700 1 --2- 172.21.15.143:0/2530284842 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f6e3006af00 0x7f6e3006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:19.905 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.889+0000 7f6e51bbe700 1 --2- 172.21.15.143:0/2530284842 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f6e3006af00 0x7f6e3006d3c0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c013040 tx=0x7f6e3c008070 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:19.905 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:19.893+0000 7f6e427fc700 1 -- 172.21.15.143:0/2530284842 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6e440544e0 con 0x7f6e4c1618f0 2023-11-08T19:31:20.085 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.084+0000 7f6e53e22700 1 -- 172.21.15.143:0/2530284842 --> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f6e4c1623b0 con 0x7f6e3006af00 2023-11-08T19:31:20.086 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.084+0000 7f6e427fc700 1 -- 172.21.15.143:0/2530284842 <== mgr.4099 v2:172.21.15.174:6832/4037280178 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f6e3c005cb0 con 0x7f6e3006af00 2023-11-08T19:31:20.088 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.086+0000 7f6e53e22700 1 -- 172.21.15.143:0/2530284842 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f6e3006af00 msgr2=0x7f6e3006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:20.088 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.086+0000 7f6e53e22700 1 --2- 172.21.15.143:0/2530284842 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f6e3006af00 0x7f6e3006d3c0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c013040 tx=0x7f6e3c008070 comp rx=0 tx=0).stop 2023-11-08T19:31:20.089 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.086+0000 7f6e53e22700 1 -- 172.21.15.143:0/2530284842 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6e4c1618f0 msgr2=0x7f6e4c1f30b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:20.089 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.086+0000 7f6e53e22700 1 --2- 172.21.15.143:0/2530284842 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6e4c1618f0 0x7f6e4c1f30b0 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f6e4400cc70 tx=0x7f6e4400cf80 comp rx=0 tx=0).stop 2023-11-08T19:31:20.089 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.087+0000 7f6e53e22700 1 -- 172.21.15.143:0/2530284842 shutdown_connections 2023-11-08T19:31:20.090 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.087+0000 7f6e53e22700 1 --2- 172.21.15.143:0/2530284842 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f6e3006af00 0x7f6e3006d3c0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:20.090 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.087+0000 7f6e53e22700 1 --2- 172.21.15.143:0/2530284842 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6e4c1618f0 0x7f6e4c1f30b0 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:20.090 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.087+0000 7f6e53e22700 1 --2- 172.21.15.143:0/2530284842 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6e4c06b150 0x7f6e4c0c9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:20.091 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.087+0000 7f6e53e22700 1 --2- 172.21.15.143:0/2530284842 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6e4c15dd00 0x7f6e4c0b5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:20.091 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.087+0000 7f6e53e22700 1 -- 172.21.15.143:0/2530284842 >> 172.21.15.143:0/2530284842 conn(0x7f6e4c099ab0 msgr2=0x7f6e4c06f230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:20.091 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.087+0000 7f6e53e22700 1 -- 172.21.15.143:0/2530284842 shutdown_connections 2023-11-08T19:31:20.091 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.087+0000 7f6e53e22700 1 -- 172.21.15.143:0/2530284842 wait complete. 2023-11-08T19:31:20.092 INFO:teuthology.orchestra.run.smithi143.stderr:instructed osd(s) 2 to deep-scrub 2023-11-08T19:31:20.100 INFO:tasks.ceph:Scrubbing osd.3 2023-11-08T19:31:20.100 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2023-11-08T19:31:20.224 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.223+0000 7fc4a1ed2700 1 Processor -- start 2023-11-08T19:31:20.225 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.223+0000 7fc4a1ed2700 1 -- start start 2023-11-08T19:31:20.228 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.224+0000 7fc4a1ed2700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc49c15bb20 0x7fc49c0b2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:20.228 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.224+0000 7fc4a1ed2700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc49c0b1730 0x7fc49c0d1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:20.228 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.224+0000 7fc4a1ed2700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fc49c0d1bb0 0x7fc49c0ce420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:20.229 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.224+0000 7fc4a1ed2700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc49c0cf600 con 0x7fc49c15bb20 2023-11-08T19:31:20.230 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.224+0000 7fc4a1ed2700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc49c0d0420 con 0x7fc49c0d1bb0 2023-11-08T19:31:20.230 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.224+0000 7fc4a1ed2700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc49c0d2040 con 0x7fc49c0b1730 2023-11-08T19:31:20.230 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.224+0000 7fc49b7fe700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc49c15bb20 0x7fc49c0b2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:20.231 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.224+0000 7fc49b7fe700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc49c15bb20 0x7fc49c0b2670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57240/0 (socket says 172.21.15.143:57240) 2023-11-08T19:31:20.231 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.224+0000 7fc49b7fe700 1 -- 172.21.15.143:0/1510492919 learned_addr learned my addr 172.21.15.143:0/1510492919 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:31:20.231 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.224+0000 7fc49affd700 1 --2- 172.21.15.143:0/1510492919 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc49c0b1730 0x7fc49c0d1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:20.232 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.224+0000 7fc49bfff700 1 --2- 172.21.15.143:0/1510492919 >> v2:172.21.15.174:3300/0 conn(0x7fc49c0d1bb0 0x7fc49c0ce420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:20.232 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.224+0000 7fc49b7fe700 1 -- 172.21.15.143:0/1510492919 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc49c0b1730 msgr2=0x7fc49c0d1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:20.232 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.224+0000 7fc49b7fe700 1 --2- 172.21.15.143:0/1510492919 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc49c0b1730 0x7fc49c0d1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:20.232 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.224+0000 7fc49b7fe700 1 -- 172.21.15.143:0/1510492919 >> v2:172.21.15.174:3300/0 conn(0x7fc49c0d1bb0 msgr2=0x7fc49c0ce420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:20.233 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.224+0000 7fc49b7fe700 1 --2- 172.21.15.143:0/1510492919 >> v2:172.21.15.174:3300/0 conn(0x7fc49c0d1bb0 0x7fc49c0ce420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:20.233 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.224+0000 7fc49b7fe700 1 -- 172.21.15.143:0/1510492919 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc49c0cf810 con 0x7fc49c15bb20 2023-11-08T19:31:20.234 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.224+0000 7fc49bfff700 1 --2- 172.21.15.143:0/1510492919 >> v2:172.21.15.174:3300/0 conn(0x7fc49c0d1bb0 0x7fc49c0ce420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:31:20.234 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.225+0000 7fc49b7fe700 1 --2- 172.21.15.143:0/1510492919 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc49c15bb20 0x7fc49c0b2670 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7fc49c0cbfc0 tx=0x7fc490009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10e901251ef096b1 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:20.234 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.225+0000 7fc49a7fc700 1 -- 172.21.15.143:0/1510492919 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc490011830 con 0x7fc49c15bb20 2023-11-08T19:31:20.235 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.225+0000 7fc49a7fc700 1 -- 172.21.15.143:0/1510492919 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc490011990 con 0x7fc49c15bb20 2023-11-08T19:31:20.235 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.225+0000 7fc49a7fc700 1 -- 172.21.15.143:0/1510492919 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc490011d20 con 0x7fc49c15bb20 2023-11-08T19:31:20.235 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.225+0000 7fc4a1ed2700 1 -- 172.21.15.143:0/1510492919 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc49c15bb20 msgr2=0x7fc49c0b2670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:20.235 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.225+0000 7fc4a1ed2700 1 --2- 172.21.15.143:0/1510492919 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc49c15bb20 0x7fc49c0b2670 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7fc49c0cbfc0 tx=0x7fc490009bd0 comp rx=0 tx=0).stop 2023-11-08T19:31:20.236 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.225+0000 7fc4a1ed2700 1 -- 172.21.15.143:0/1510492919 shutdown_connections 2023-11-08T19:31:20.236 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.225+0000 7fc4a1ed2700 1 --2- 172.21.15.143:0/1510492919 >> v2:172.21.15.174:3300/0 conn(0x7fc49c0d1bb0 0x7fc49c0ce420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:20.236 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.225+0000 7fc4a1ed2700 1 --2- 172.21.15.143:0/1510492919 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc49c15bb20 0x7fc49c0b2670 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:20.237 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.225+0000 7fc4a1ed2700 1 --2- 172.21.15.143:0/1510492919 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc49c0b1730 0x7fc49c0d1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:20.237 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.225+0000 7fc4a1ed2700 1 -- 172.21.15.143:0/1510492919 >> 172.21.15.143:0/1510492919 conn(0x7fc49c08b000 msgr2=0x7fc49c09ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:20.237 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.225+0000 7fc4a1ed2700 1 -- 172.21.15.143:0/1510492919 shutdown_connections 2023-11-08T19:31:20.237 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.225+0000 7fc4a1ed2700 1 -- 172.21.15.143:0/1510492919 wait complete. 2023-11-08T19:31:20.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.226+0000 7fc4a1ed2700 1 Processor -- start 2023-11-08T19:31:20.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.227+0000 7fc4a1ed2700 1 -- start start 2023-11-08T19:31:20.241 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.227+0000 7fc4a1ed2700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc49c0b1730 0x7fc49c1f3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:20.241 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.227+0000 7fc4a1ed2700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc49c0d1bb0 0x7fc49c0c6d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:20.242 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.227+0000 7fc4a1ed2700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc49c15bb20 0x7fc49c0b4650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:20.242 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.227+0000 7fc4a1ed2700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc49c0cc800 con 0x7fc49c15bb20 2023-11-08T19:31:20.242 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.227+0000 7fc4a1ed2700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc49c0ccae0 con 0x7fc49c0d1bb0 2023-11-08T19:31:20.243 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.227+0000 7fc4a1ed2700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc49c0c6570 con 0x7fc49c0b1730 2023-11-08T19:31:20.243 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.227+0000 7fc49bfff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc49c15bb20 0x7fc49c0b4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:20.243 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.227+0000 7fc49bfff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc49c15bb20 0x7fc49c0b4650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57248/0 (socket says 172.21.15.143:57248) 2023-11-08T19:31:20.243 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.227+0000 7fc49bfff700 1 -- 172.21.15.143:0/2482140460 learned_addr learned my addr 172.21.15.143:0/2482140460 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:31:20.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.228+0000 7fc49b7fe700 1 --2- 172.21.15.143:0/2482140460 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc49c0b1730 0x7fc49c1f3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:20.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.228+0000 7fc49affd700 1 --2- 172.21.15.143:0/2482140460 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc49c0d1bb0 0x7fc49c0c6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:20.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.228+0000 7fc49bfff700 1 -- 172.21.15.143:0/2482140460 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc49c0b1730 msgr2=0x7fc49c1f3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:20.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.228+0000 7fc49bfff700 1 --2- 172.21.15.143:0/2482140460 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc49c0b1730 0x7fc49c1f3670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:20.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.228+0000 7fc49bfff700 1 -- 172.21.15.143:0/2482140460 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc49c0d1bb0 msgr2=0x7fc49c0c6d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:20.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.228+0000 7fc49bfff700 1 --2- 172.21.15.143:0/2482140460 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc49c0d1bb0 0x7fc49c0c6d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:20.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.228+0000 7fc49bfff700 1 -- 172.21.15.143:0/2482140460 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc490009550 con 0x7fc49c15bb20 2023-11-08T19:31:20.246 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.228+0000 7fc49bfff700 1 --2- 172.21.15.143:0/2482140460 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc49c15bb20 0x7fc49c0b4650 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7fc494007ef0 tx=0x7fc49400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:20.246 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.228+0000 7fc483fff700 1 -- 172.21.15.143:0/2482140460 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc49400dd80 con 0x7fc49c15bb20 2023-11-08T19:31:20.246 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.228+0000 7fc483fff700 1 -- 172.21.15.143:0/2482140460 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc49400f040 con 0x7fc49c15bb20 2023-11-08T19:31:20.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.228+0000 7fc4a1ed2700 1 -- 172.21.15.143:0/2482140460 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc49c0c6850 con 0x7fc49c15bb20 2023-11-08T19:31:20.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.228+0000 7fc483fff700 1 -- 172.21.15.143:0/2482140460 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc494003c60 con 0x7fc49c15bb20 2023-11-08T19:31:20.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.228+0000 7fc4a1ed2700 1 -- 172.21.15.143:0/2482140460 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc49c0c7490 con 0x7fc49c15bb20 2023-11-08T19:31:20.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.229+0000 7fc4a1ed2700 1 -- 172.21.15.143:0/2482140460 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc49c0cfdd0 con 0x7fc49c15bb20 2023-11-08T19:31:20.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.229+0000 7fc483fff700 1 -- 172.21.15.143:0/2482140460 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc494004030 con 0x7fc49c15bb20 2023-11-08T19:31:20.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.230+0000 7fc483fff700 1 --2- 172.21.15.143:0/2482140460 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc48806af00 0x7fc48806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:20.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.230+0000 7fc483fff700 1 -- 172.21.15.143:0/2482140460 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6651+0+0 (secure 0 0 0) 0x7fc4940897c0 con 0x7fc49c15bb20 2023-11-08T19:31:20.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.230+0000 7fc483fff700 1 --2- 172.21.15.143:0/2482140460 >> [v2:172.21.15.143:6816/2021504428,v1:172.21.15.143:6817/2021504428] conn(0x7fc488073a10 0x7fc488075e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:20.249 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.230+0000 7fc483fff700 1 -- 172.21.15.143:0/2482140460 --> [v2:172.21.15.143:6816/2021504428,v1:172.21.15.143:6817/2021504428] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc488076500 con 0x7fc488073a10 2023-11-08T19:31:20.249 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.230+0000 7fc483fff700 1 -- 172.21.15.143:0/2482140460 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_get_version_reply(handle=1 version=107) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc49408be80 con 0x7fc49c15bb20 2023-11-08T19:31:20.249 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.230+0000 7fc49b7fe700 1 --2- 172.21.15.143:0/2482140460 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc48806af00 0x7fc48806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:20.250 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.230+0000 7fc49affd700 1 --2- 172.21.15.143:0/2482140460 >> [v2:172.21.15.143:6816/2021504428,v1:172.21.15.143:6817/2021504428] conn(0x7fc488073a10 0x7fc488075e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:20.250 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.230+0000 7fc49affd700 1 --2- 172.21.15.143:0/2482140460 >> [v2:172.21.15.143:6816/2021504428,v1:172.21.15.143:6817/2021504428] conn(0x7fc488073a10 0x7fc488075e30 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:20.250 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.231+0000 7fc49b7fe700 1 --2- 172.21.15.143:0/2482140460 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc48806af00 0x7fc48806d3c0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fc490009c10 tx=0x7fc490008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:20.250 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.231+0000 7fc483fff700 1 -- 172.21.15.143:0/2482140460 <== osd.3 v2:172.21.15.143:6816/2021504428 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26745 (crc 0 0 0) 0x7fc488076500 con 0x7fc488073a10 2023-11-08T19:31:20.251 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.246+0000 7fc4a1ed2700 1 -- 172.21.15.143:0/2482140460 --> [v2:172.21.15.143:6816/2021504428,v1:172.21.15.143:6817/2021504428] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fc49c0cfaf0 con 0x7fc488073a10 2023-11-08T19:31:20.251 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.246+0000 7fc483fff700 1 -- 172.21.15.143:0/2482140460 <== osd.3 v2:172.21.15.143:6816/2021504428 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fc48c009b50 con 0x7fc488073a10 2023-11-08T19:31:20.252 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2023-11-08T19:31:20.252 INFO:teuthology.orchestra.run.smithi143.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-08T19:31:20.253 INFO:teuthology.orchestra.run.smithi143.stdout:} 2023-11-08T19:31:20.254 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.246+0000 7fc4a1ed2700 1 -- 172.21.15.143:0/2482140460 >> [v2:172.21.15.143:6816/2021504428,v1:172.21.15.143:6817/2021504428] conn(0x7fc488073a10 msgr2=0x7fc488075e30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:20.254 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.246+0000 7fc4a1ed2700 1 --2- 172.21.15.143:0/2482140460 >> [v2:172.21.15.143:6816/2021504428,v1:172.21.15.143:6817/2021504428] conn(0x7fc488073a10 0x7fc488075e30 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:20.254 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.247+0000 7fc4a1ed2700 1 -- 172.21.15.143:0/2482140460 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc48806af00 msgr2=0x7fc48806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:20.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.247+0000 7fc4a1ed2700 1 --2- 172.21.15.143:0/2482140460 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc48806af00 0x7fc48806d3c0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fc490009c10 tx=0x7fc490008030 comp rx=0 tx=0).stop 2023-11-08T19:31:20.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.247+0000 7fc4a1ed2700 1 -- 172.21.15.143:0/2482140460 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc49c15bb20 msgr2=0x7fc49c0b4650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:20.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.247+0000 7fc4a1ed2700 1 --2- 172.21.15.143:0/2482140460 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc49c15bb20 0x7fc49c0b4650 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7fc494007ef0 tx=0x7fc49400d3b0 comp rx=0 tx=0).stop 2023-11-08T19:31:20.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.247+0000 7fc49bfff700 1 -- 172.21.15.143:0/2482140460 reap_dead start 2023-11-08T19:31:20.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.247+0000 7fc4a1ed2700 1 -- 172.21.15.143:0/2482140460 shutdown_connections 2023-11-08T19:31:20.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.247+0000 7fc4a1ed2700 1 -- 172.21.15.143:0/2482140460 >> 172.21.15.143:0/2482140460 conn(0x7fc49c08b000 msgr2=0x7fc49c0cab10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:20.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.247+0000 7fc4a1ed2700 1 -- 172.21.15.143:0/2482140460 shutdown_connections 2023-11-08T19:31:20.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.247+0000 7fc4a1ed2700 1 -- 172.21.15.143:0/2482140460 wait complete. 2023-11-08T19:31:20.260 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2023-11-08T19:31:20.382 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.379+0000 7f994f6c3700 1 Processor -- start 2023-11-08T19:31:20.383 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.379+0000 7f994f6c3700 1 -- start start 2023-11-08T19:31:20.384 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.380+0000 7f994f6c3700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f994806dbb0 0x7f99480ca470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:20.384 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.380+0000 7f994f6c3700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f99480668e0 0x7f99480ca9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:20.385 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.380+0000 7f994f6c3700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99480ce7f0 con 0x7f994806df90 2023-11-08T19:31:20.385 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.380+0000 7f994f6c3700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9948066db0 con 0x7f99480668e0 2023-11-08T19:31:20.386 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.380+0000 7f994f6c3700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99480af5e0 con 0x7f994806dbb0 2023-11-08T19:31:20.392 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.380+0000 7f994cc5e700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f99480668e0 0x7f99480ca9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:20.392 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.380+0000 7f994cc5e700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f99480668e0 0x7f99480ca9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57254/0 (socket says 172.21.15.143:57254) 2023-11-08T19:31:20.393 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.380+0000 7f994cc5e700 1 -- 172.21.15.143:0/555593244 learned_addr learned my addr 172.21.15.143:0/555593244 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:31:20.393 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.380+0000 7f994d45f700 1 --2- 172.21.15.143:0/555593244 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f994806dbb0 0x7f99480ca470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:20.393 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.381+0000 7f994cc5e700 1 -- 172.21.15.143:0/555593244 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f994806dbb0 msgr2=0x7f99480ca470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:20.394 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.381+0000 7f994cc5e700 1 --2- 172.21.15.143:0/555593244 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f994806dbb0 0x7f99480ca470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:20.394 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.381+0000 7f994cc5e700 1 -- 172.21.15.143:0/555593244 >> v1:172.21.15.174:6789/0 conn(0x7f994806df90 legacy=0x7f99481621f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:20.395 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.381+0000 7f994cc5e700 1 -- 172.21.15.143:0/555593244 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99480cb520 con 0x7f99480668e0 2023-11-08T19:31:20.395 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.381+0000 7f994cc5e700 1 --2- 172.21.15.143:0/555593244 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f99480668e0 0x7f99480ca9b0 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f99440098f0 tx=0x7f9944009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb0494e66788d1bf server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:20.395 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.381+0000 7f993ffff700 1 -- 172.21.15.143:0/555593244 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f994400b8a0 con 0x7f99480668e0 2023-11-08T19:31:20.395 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.381+0000 7f993ffff700 1 -- 172.21.15.143:0/555593244 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f994400ba00 con 0x7f99480668e0 2023-11-08T19:31:20.396 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.381+0000 7f993ffff700 1 -- 172.21.15.143:0/555593244 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f994400bd90 con 0x7f99480668e0 2023-11-08T19:31:20.396 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.381+0000 7f994f6c3700 1 -- 172.21.15.143:0/555593244 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f99480668e0 msgr2=0x7f99480ca9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:20.397 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.381+0000 7f994f6c3700 1 --2- 172.21.15.143:0/555593244 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f99480668e0 0x7f99480ca9b0 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f99440098f0 tx=0x7f9944009c00 comp rx=0 tx=0).stop 2023-11-08T19:31:20.398 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.382+0000 7f994f6c3700 1 -- 172.21.15.143:0/555593244 shutdown_connections 2023-11-08T19:31:20.398 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.382+0000 7f994f6c3700 1 --2- 172.21.15.143:0/555593244 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f994806dbb0 0x7f99480ca470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:20.398 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.382+0000 7f994f6c3700 1 --2- 172.21.15.143:0/555593244 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f99480668e0 0x7f99480ca9b0 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:20.398 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.382+0000 7f994f6c3700 1 -- 172.21.15.143:0/555593244 >> 172.21.15.143:0/555593244 conn(0x7f99480b1a20 msgr2=0x7f99480b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:20.399 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.382+0000 7f994f6c3700 1 -- 172.21.15.143:0/555593244 shutdown_connections 2023-11-08T19:31:20.399 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.382+0000 7f994f6c3700 1 -- 172.21.15.143:0/555593244 wait complete. 2023-11-08T19:31:20.399 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.383+0000 7f994f6c3700 1 Processor -- start 2023-11-08T19:31:20.399 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.383+0000 7f994f6c3700 1 -- start start 2023-11-08T19:31:20.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.384+0000 7f994f6c3700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f99480668e0 0x7f99480b3490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:20.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.384+0000 7f994f6c3700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f994806dbb0 0x7f994806ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:20.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.384+0000 7f994f6c3700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f994806df90 0x7f994806b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:20.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.384+0000 7f994f6c3700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f994815fe00 con 0x7f994806df90 2023-11-08T19:31:20.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.384+0000 7f994f6c3700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99480b54e0 con 0x7f994806dbb0 2023-11-08T19:31:20.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.384+0000 7f994f6c3700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99480b39d0 con 0x7f99480668e0 2023-11-08T19:31:20.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.384+0000 7f994dc60700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f994806df90 0x7f994806b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:20.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.384+0000 7f994dc60700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f994806df90 0x7f994806b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57262/0 (socket says 172.21.15.143:57262) 2023-11-08T19:31:20.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.384+0000 7f994dc60700 1 -- 172.21.15.143:0/2929118522 learned_addr learned my addr 172.21.15.143:0/2929118522 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:31:20.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.384+0000 7f994cc5e700 1 --2- 172.21.15.143:0/2929118522 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f994806dbb0 0x7f994806ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:20.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.384+0000 7f994d45f700 1 --2- 172.21.15.143:0/2929118522 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f99480668e0 0x7f99480b3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:20.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.384+0000 7f994dc60700 1 -- 172.21.15.143:0/2929118522 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f99480668e0 msgr2=0x7f99480b3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:20.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.384+0000 7f994dc60700 1 --2- 172.21.15.143:0/2929118522 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f99480668e0 0x7f99480b3490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:20.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.384+0000 7f994dc60700 1 -- 172.21.15.143:0/2929118522 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f994806dbb0 msgr2=0x7f994806ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:20.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.384+0000 7f994dc60700 1 --2- 172.21.15.143:0/2929118522 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f994806dbb0 0x7f994806ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:20.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.384+0000 7f994dc60700 1 -- 172.21.15.143:0/2929118522 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9944009550 con 0x7f994806df90 2023-11-08T19:31:20.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.384+0000 7f994dc60700 1 --2- 172.21.15.143:0/2929118522 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f994806df90 0x7f994806b380 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f994000cb50 tx=0x7f994000cf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:20.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.384+0000 7f993dffb700 1 -- 172.21.15.143:0/2929118522 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9940003c00 con 0x7f994806df90 2023-11-08T19:31:20.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.384+0000 7f993dffb700 1 -- 172.21.15.143:0/2929118522 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9940003d60 con 0x7f994806df90 2023-11-08T19:31:20.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.384+0000 7f994f6c3700 1 -- 172.21.15.143:0/2929118522 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99480b48d0 con 0x7f994806df90 2023-11-08T19:31:20.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.384+0000 7f993dffb700 1 -- 172.21.15.143:0/2929118522 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f994000ab70 con 0x7f994806df90 2023-11-08T19:31:20.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.385+0000 7f994f6c3700 1 -- 172.21.15.143:0/2929118522 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99480b4e90 con 0x7f994806df90 2023-11-08T19:31:20.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.385+0000 7f994f6c3700 1 -- 172.21.15.143:0/2929118522 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f994806d1a0 con 0x7f994806df90 2023-11-08T19:31:20.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.386+0000 7f993dffb700 1 -- 172.21.15.143:0/2929118522 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f994000acd0 con 0x7f994806df90 2023-11-08T19:31:20.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.386+0000 7f993dffb700 1 --2- 172.21.15.143:0/2929118522 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f992c06af00 0x7f992c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:20.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.386+0000 7f993dffb700 1 -- 172.21.15.143:0/2929118522 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6651+0+0 (secure 0 0 0) 0x7f9940089950 con 0x7f994806df90 2023-11-08T19:31:20.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.386+0000 7f994d45f700 1 --2- 172.21.15.143:0/2929118522 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f992c06af00 0x7f992c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:20.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.387+0000 7f994d45f700 1 --2- 172.21.15.143:0/2929118522 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f992c06af00 0x7f992c06d3c0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f99480b5c30 tx=0x7f9938008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:20.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.390+0000 7f993dffb700 1 -- 172.21.15.143:0/2929118522 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f99400546e0 con 0x7f994806df90 2023-11-08T19:31:20.583 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.582+0000 7f994f6c3700 1 -- 172.21.15.143:0/2929118522 --> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7f9948066fc0 con 0x7f992c06af00 2023-11-08T19:31:20.584 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.582+0000 7f993dffb700 1 -- 172.21.15.143:0/2929118522 <== mgr.4099 v2:172.21.15.174:6832/4037280178 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f9938009dd0 con 0x7f992c06af00 2023-11-08T19:31:20.587 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.585+0000 7f994f6c3700 1 -- 172.21.15.143:0/2929118522 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f992c06af00 msgr2=0x7f992c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:20.587 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.585+0000 7f994f6c3700 1 --2- 172.21.15.143:0/2929118522 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f992c06af00 0x7f992c06d3c0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f99480b5c30 tx=0x7f9938008040 comp rx=0 tx=0).stop 2023-11-08T19:31:20.587 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.585+0000 7f994f6c3700 1 -- 172.21.15.143:0/2929118522 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f994806df90 msgr2=0x7f994806b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:20.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.585+0000 7f994f6c3700 1 --2- 172.21.15.143:0/2929118522 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f994806df90 0x7f994806b380 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f994000cb50 tx=0x7f994000cf10 comp rx=0 tx=0).stop 2023-11-08T19:31:20.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.585+0000 7f994f6c3700 1 -- 172.21.15.143:0/2929118522 shutdown_connections 2023-11-08T19:31:20.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.585+0000 7f994f6c3700 1 --2- 172.21.15.143:0/2929118522 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f992c06af00 0x7f992c06d3c0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:20.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.585+0000 7f994f6c3700 1 --2- 172.21.15.143:0/2929118522 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f99480668e0 0x7f99480b3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:20.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.585+0000 7f994f6c3700 1 --2- 172.21.15.143:0/2929118522 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f994806df90 0x7f994806b380 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:20.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.585+0000 7f994f6c3700 1 --2- 172.21.15.143:0/2929118522 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f994806dbb0 0x7f994806ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:20.590 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.585+0000 7f994f6c3700 1 -- 172.21.15.143:0/2929118522 >> 172.21.15.143:0/2929118522 conn(0x7f99480b1a20 msgr2=0x7f9948161a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:20.590 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.585+0000 7f994f6c3700 1 -- 172.21.15.143:0/2929118522 shutdown_connections 2023-11-08T19:31:20.590 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.585+0000 7f994f6c3700 1 -- 172.21.15.143:0/2929118522 wait complete. 2023-11-08T19:31:20.591 INFO:teuthology.orchestra.run.smithi143.stderr:instructed osd(s) 3 to deep-scrub 2023-11-08T19:31:20.598 INFO:tasks.ceph:Scrubbing osd.4 2023-11-08T19:31:20.599 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2023-11-08T19:31:20.723 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.722+0000 7fa0d4b94700 1 Processor -- start 2023-11-08T19:31:20.724 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.722+0000 7fa0d4b94700 1 -- start start 2023-11-08T19:31:20.726 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.723+0000 7fa0d4b94700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fa0d015dba0 0x7fa0d006c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:20.726 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.723+0000 7fa0d4b94700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa0d00b22e0 0x7fa0d00b2750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:20.726 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.723+0000 7fa0d4b94700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0d0161890 con 0x7fa0d006b240 2023-11-08T19:31:20.727 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.723+0000 7fa0d4b94700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0d00b2c90 con 0x7fa0d015dba0 2023-11-08T19:31:20.727 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.723+0000 7fa0d4b94700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0d006b740 con 0x7fa0d00b22e0 2023-11-08T19:31:20.727 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.723+0000 7fa0ced9d700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa0d00b22e0 0x7fa0d00b2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:20.727 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.723+0000 7fa0ce59c700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fa0d015dba0 0x7fa0d006c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:20.728 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.723+0000 7fa0ced9d700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa0d00b22e0 0x7fa0d00b2750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.143:44908/0 (socket says 172.21.15.143:44908) 2023-11-08T19:31:20.728 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.723+0000 7fa0ce59c700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fa0d015dba0 0x7fa0d006c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.143:40764/0 (socket says 172.21.15.143:40764) 2023-11-08T19:31:20.728 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.723+0000 7fa0ced9d700 1 -- 172.21.15.143:0/3461258247 learned_addr learned my addr 172.21.15.143:0/3461258247 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:31:20.729 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.723+0000 7fa0ced9d700 1 -- 172.21.15.143:0/3461258247 >> v2:172.21.15.174:3300/0 conn(0x7fa0d015dba0 msgr2=0x7fa0d006c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:20.729 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.723+0000 7fa0ced9d700 1 --2- 172.21.15.143:0/3461258247 >> v2:172.21.15.174:3300/0 conn(0x7fa0d015dba0 0x7fa0d006c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:20.729 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.723+0000 7fa0ced9d700 1 -- 172.21.15.143:0/3461258247 >> v1:172.21.15.174:6789/0 conn(0x7fa0d006b240 legacy=0x7fa0d00caff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:20.730 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.723+0000 7fa0ced9d700 1 -- 172.21.15.143:0/3461258247 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0d006ba20 con 0x7fa0d00b22e0 2023-11-08T19:31:20.730 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.724+0000 7fa0ced9d700 1 --2- 172.21.15.143:0/3461258247 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa0d00b22e0 0x7fa0d00b2750 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa0c8009e50 tx=0x7fa0c800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c4e173dfc77a378b server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:20.730 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.724+0000 7fa0cd59a700 1 -- 172.21.15.143:0/3461258247 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0c800e070 con 0x7fa0d00b22e0 2023-11-08T19:31:20.730 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.724+0000 7fa0cd59a700 1 -- 172.21.15.143:0/3461258247 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0c800b910 con 0x7fa0d00b22e0 2023-11-08T19:31:20.731 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.724+0000 7fa0cd59a700 1 -- 172.21.15.143:0/3461258247 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0c800bcc0 con 0x7fa0d00b22e0 2023-11-08T19:31:20.731 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.724+0000 7fa0d4b94700 1 -- 172.21.15.143:0/3461258247 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa0d00b22e0 msgr2=0x7fa0d00b2750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:20.731 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.724+0000 7fa0d4b94700 1 --2- 172.21.15.143:0/3461258247 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa0d00b22e0 0x7fa0d00b2750 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa0c8009e50 tx=0x7fa0c800b330 comp rx=0 tx=0).stop 2023-11-08T19:31:20.731 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.724+0000 7fa0d4b94700 1 -- 172.21.15.143:0/3461258247 shutdown_connections 2023-11-08T19:31:20.732 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.724+0000 7fa0d4b94700 1 --2- 172.21.15.143:0/3461258247 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa0d00b22e0 0x7fa0d00b2750 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:20.732 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.724+0000 7fa0d4b94700 1 --2- 172.21.15.143:0/3461258247 >> v2:172.21.15.174:3300/0 conn(0x7fa0d015dba0 0x7fa0d006c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:20.733 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.724+0000 7fa0d4b94700 1 -- 172.21.15.143:0/3461258247 >> 172.21.15.143:0/3461258247 conn(0x7fa0d00cd2e0 msgr2=0x7fa0d00cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:20.733 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.724+0000 7fa0d4b94700 1 -- 172.21.15.143:0/3461258247 shutdown_connections 2023-11-08T19:31:20.733 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.724+0000 7fa0d4b94700 1 -- 172.21.15.143:0/3461258247 wait complete. 2023-11-08T19:31:20.737 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.726+0000 7fa0d4b94700 1 Processor -- start 2023-11-08T19:31:20.737 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.726+0000 7fa0d4b94700 1 -- start start 2023-11-08T19:31:20.737 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.726+0000 7fa0d4b94700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa0d006b240 0x7fa0d015bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:20.737 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.727+0000 7fa0d4b94700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa0d00b22e0 0x7fa0d00ca620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:20.738 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.727+0000 7fa0d4b94700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa0d015dba0 0x7fa0d00c7b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:20.738 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.727+0000 7fa0d4b94700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0d00c5220 con 0x7fa0d00b22e0 2023-11-08T19:31:20.738 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.727+0000 7fa0d4b94700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0d00c4390 con 0x7fa0d006b240 2023-11-08T19:31:20.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.727+0000 7fa0d4b94700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0d015c010 con 0x7fa0d015dba0 2023-11-08T19:31:20.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.727+0000 7fa0cdd9b700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa0d00b22e0 0x7fa0d00ca620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:20.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.727+0000 7fa0cdd9b700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa0d00b22e0 0x7fa0d00ca620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57274/0 (socket says 172.21.15.143:57274) 2023-11-08T19:31:20.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.727+0000 7fa0cdd9b700 1 -- 172.21.15.143:0/3512024065 learned_addr learned my addr 172.21.15.143:0/3512024065 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:31:20.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.727+0000 7fa0ced9d700 1 --2- 172.21.15.143:0/3512024065 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa0d015dba0 0x7fa0d00c7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:20.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.727+0000 7fa0ce59c700 1 --2- 172.21.15.143:0/3512024065 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa0d006b240 0x7fa0d015bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:20.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.727+0000 7fa0cdd9b700 1 -- 172.21.15.143:0/3512024065 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa0d015dba0 msgr2=0x7fa0d00c7b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:20.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.727+0000 7fa0cdd9b700 1 --2- 172.21.15.143:0/3512024065 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa0d015dba0 0x7fa0d00c7b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:20.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.727+0000 7fa0cdd9b700 1 -- 172.21.15.143:0/3512024065 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa0d006b240 msgr2=0x7fa0d015bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:20.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.727+0000 7fa0cdd9b700 1 --2- 172.21.15.143:0/3512024065 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa0d006b240 0x7fa0d015bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:20.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.727+0000 7fa0cdd9b700 1 -- 172.21.15.143:0/3512024065 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0c8009b30 con 0x7fa0d00b22e0 2023-11-08T19:31:20.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.727+0000 7fa0cdd9b700 1 --2- 172.21.15.143:0/3512024065 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa0d00b22e0 0x7fa0d00ca620 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7fa0c400c490 tx=0x7fa0c400c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:20.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.727+0000 7fa0beffd700 1 -- 172.21.15.143:0/3512024065 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0c400cea0 con 0x7fa0d00b22e0 2023-11-08T19:31:20.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.727+0000 7fa0beffd700 1 -- 172.21.15.143:0/3512024065 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0c40052f0 con 0x7fa0d00b22e0 2023-11-08T19:31:20.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.727+0000 7fa0d4b94700 1 -- 172.21.15.143:0/3512024065 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0d00c45a0 con 0x7fa0d00b22e0 2023-11-08T19:31:20.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.727+0000 7fa0beffd700 1 -- 172.21.15.143:0/3512024065 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0c4006860 con 0x7fa0d00b22e0 2023-11-08T19:31:20.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.727+0000 7fa0d4b94700 1 -- 172.21.15.143:0/3512024065 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0d00c4b60 con 0x7fa0d00b22e0 2023-11-08T19:31:20.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.728+0000 7fa0d4b94700 1 -- 172.21.15.143:0/3512024065 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa0d006bfe0 con 0x7fa0d00b22e0 2023-11-08T19:31:20.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.729+0000 7fa0beffd700 1 -- 172.21.15.143:0/3512024065 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa0c4003680 con 0x7fa0d00b22e0 2023-11-08T19:31:20.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.729+0000 7fa0beffd700 1 --2- 172.21.15.143:0/3512024065 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa0b006af00 0x7fa0b006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:20.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.729+0000 7fa0beffd700 1 -- 172.21.15.143:0/3512024065 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6651+0+0 (secure 0 0 0) 0x7fa0c4088b00 con 0x7fa0d00b22e0 2023-11-08T19:31:20.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.729+0000 7fa0beffd700 1 --2- 172.21.15.143:0/3512024065 >> [v2:172.21.15.174:6800/1993380595,v1:172.21.15.174:6801/1993380595] conn(0x7fa0b0073a10 0x7fa0b0075e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:20.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.729+0000 7fa0beffd700 1 -- 172.21.15.143:0/3512024065 --> [v2:172.21.15.174:6800/1993380595,v1:172.21.15.174:6801/1993380595] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa0b0076500 con 0x7fa0b0073a10 2023-11-08T19:31:20.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.729+0000 7fa0beffd700 1 -- 172.21.15.143:0/3512024065 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_get_version_reply(handle=1 version=107) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa0c408d2c0 con 0x7fa0d00b22e0 2023-11-08T19:31:20.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.729+0000 7fa0ce59c700 1 --2- 172.21.15.143:0/3512024065 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa0b006af00 0x7fa0b006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:20.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.730+0000 7fa0ced9d700 1 --2- 172.21.15.143:0/3512024065 >> [v2:172.21.15.174:6800/1993380595,v1:172.21.15.174:6801/1993380595] conn(0x7fa0b0073a10 0x7fa0b0075e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:20.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.730+0000 7fa0ce59c700 1 --2- 172.21.15.143:0/3512024065 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa0b006af00 0x7fa0b006d3c0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fa0c0009b40 tx=0x7fa0c0009210 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:20.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.730+0000 7fa0ced9d700 1 --2- 172.21.15.143:0/3512024065 >> [v2:172.21.15.174:6800/1993380595,v1:172.21.15.174:6801/1993380595] conn(0x7fa0b0073a10 0x7fa0b0075e30 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:20.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.731+0000 7fa0beffd700 1 -- 172.21.15.143:0/3512024065 <== osd.4 v2:172.21.15.174:6800/1993380595 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26745 (crc 0 0 0) 0x7fa0b0076500 con 0x7fa0b0073a10 2023-11-08T19:31:20.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.746+0000 7fa0d4b94700 1 -- 172.21.15.143:0/3512024065 --> [v2:172.21.15.174:6800/1993380595,v1:172.21.15.174:6801/1993380595] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fa0d006bd00 con 0x7fa0b0073a10 2023-11-08T19:31:20.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.746+0000 7fa0beffd700 1 -- 172.21.15.143:0/3512024065 <== osd.4 v2:172.21.15.174:6800/1993380595 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fa0c80092e0 con 0x7fa0b0073a10 2023-11-08T19:31:20.748 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2023-11-08T19:31:20.749 INFO:teuthology.orchestra.run.smithi143.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-08T19:31:20.749 INFO:teuthology.orchestra.run.smithi143.stdout:} 2023-11-08T19:31:20.750 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.746+0000 7fa0d4b94700 1 -- 172.21.15.143:0/3512024065 >> [v2:172.21.15.174:6800/1993380595,v1:172.21.15.174:6801/1993380595] conn(0x7fa0b0073a10 msgr2=0x7fa0b0075e30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:20.750 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.746+0000 7fa0d4b94700 1 --2- 172.21.15.143:0/3512024065 >> [v2:172.21.15.174:6800/1993380595,v1:172.21.15.174:6801/1993380595] conn(0x7fa0b0073a10 0x7fa0b0075e30 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:20.751 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.746+0000 7fa0d4b94700 1 -- 172.21.15.143:0/3512024065 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa0b006af00 msgr2=0x7fa0b006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:20.751 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.746+0000 7fa0d4b94700 1 --2- 172.21.15.143:0/3512024065 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fa0b006af00 0x7fa0b006d3c0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fa0c0009b40 tx=0x7fa0c0009210 comp rx=0 tx=0).stop 2023-11-08T19:31:20.751 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.746+0000 7fa0d4b94700 1 -- 172.21.15.143:0/3512024065 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa0d00b22e0 msgr2=0x7fa0d00ca620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:20.751 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.746+0000 7fa0d4b94700 1 --2- 172.21.15.143:0/3512024065 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa0d00b22e0 0x7fa0d00ca620 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7fa0c400c490 tx=0x7fa0c400c7a0 comp rx=0 tx=0).stop 2023-11-08T19:31:20.752 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.746+0000 7fa0ced9d700 1 -- 172.21.15.143:0/3512024065 reap_dead start 2023-11-08T19:31:20.752 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.746+0000 7fa0d4b94700 1 -- 172.21.15.143:0/3512024065 shutdown_connections 2023-11-08T19:31:20.752 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.746+0000 7fa0d4b94700 1 -- 172.21.15.143:0/3512024065 >> 172.21.15.143:0/3512024065 conn(0x7fa0d00cd2e0 msgr2=0x7fa0d00cc500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:20.753 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.747+0000 7fa0d4b94700 1 -- 172.21.15.143:0/3512024065 shutdown_connections 2023-11-08T19:31:20.753 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.747+0000 7fa0d4b94700 1 -- 172.21.15.143:0/3512024065 wait complete. 2023-11-08T19:31:20.759 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2023-11-08T19:31:20.881 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.879+0000 7f86f596c700 1 Processor -- start 2023-11-08T19:31:20.881 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.879+0000 7f86f596c700 1 -- start start 2023-11-08T19:31:20.888 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.880+0000 7f86f596c700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f86f006b320 0x7f86f0168c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:20.888 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.880+0000 7f86f596c700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86f0161df0 con 0x7f86f01618f0 2023-11-08T19:31:20.888 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.880+0000 7f86f596c700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86f01620d0 con 0x7f86f015dd00 2023-11-08T19:31:20.889 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.880+0000 7f86f596c700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86f00cad50 con 0x7f86f006b320 2023-11-08T19:31:20.889 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.880+0000 7f86ee7fc700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f86f01618f0 0x7f86f006ac10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:53438/0 (socket says 172.21.15.143:53438) 2023-11-08T19:31:20.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.880+0000 7f86ee7fc700 1 -- 172.21.15.143:0/168474173 learned_addr learned my addr 172.21.15.143:0/168474173 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:31:20.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.880+0000 7f86ef7fe700 1 --2- 172.21.15.143:0/168474173 >> v2:172.21.15.174:3300/0 conn(0x7f86f006b320 0x7f86f0168c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:20.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.880+0000 7f86edffb700 1 -- 172.21.15.143:0/168474173 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1669445447 0 0) 0x7f86f0161df0 con 0x7f86f01618f0 2023-11-08T19:31:20.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.881+0000 7f86edffb700 1 -- 172.21.15.143:0/168474173 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86dc0035b0 con 0x7f86f01618f0 2023-11-08T19:31:20.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.881+0000 7f86edffb700 1 -- 172.21.15.143:0/168474173 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2755962346 0 0) 0x7f86dc0035b0 con 0x7f86f01618f0 2023-11-08T19:31:20.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.881+0000 7f86edffb700 1 -- 172.21.15.143:0/168474173 >> v2:172.21.15.174:3300/0 conn(0x7f86f006b320 msgr2=0x7f86f0168c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:20.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.881+0000 7f86edffb700 1 --2- 172.21.15.143:0/168474173 >> v2:172.21.15.174:3300/0 conn(0x7f86f006b320 0x7f86f0168c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:20.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.881+0000 7f86edffb700 1 -- 172.21.15.143:0/168474173 >> v1:172.21.15.174:6789/0 conn(0x7f86f015dd00 legacy=0x7f86f00cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:20.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.881+0000 7f86edffb700 1 -- 172.21.15.143:0/168474173 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86f01623b0 con 0x7f86f01618f0 2023-11-08T19:31:20.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.881+0000 7f86edffb700 1 -- 172.21.15.143:0/168474173 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4147756303 0 0) 0x7f86d8001670 con 0x7f86f01618f0 2023-11-08T19:31:20.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.881+0000 7f86edffb700 1 -- 172.21.15.143:0/168474173 >> v1:172.21.15.143:6789/0 conn(0x7f86f01618f0 legacy=0x7f86f006ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:20.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.881+0000 7f86edffb700 1 --2- 172.21.15.143:0/168474173 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86dc003a00 0x7f86dc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:20.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.881+0000 7f86edffb700 1 --2- 172.21.15.143:0/168474173 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f86dc004400 0x7f86dc0048d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:20.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.881+0000 7f86edffb700 1 --2- 172.21.15.143:0/168474173 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f86dc0050d0 0x7f86dc00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:20.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.881+0000 7f86edffb700 1 -- 172.21.15.143:0/168474173 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86dc00c000 con 0x7f86dc003a00 2023-11-08T19:31:20.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.881+0000 7f86edffb700 1 -- 172.21.15.143:0/168474173 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86dc00c170 con 0x7f86dc004400 2023-11-08T19:31:20.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.881+0000 7f86edffb700 1 -- 172.21.15.143:0/168474173 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86dc00c330 con 0x7f86dc0050d0 2023-11-08T19:31:20.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.881+0000 7f86eeffd700 1 --2- 172.21.15.143:0/168474173 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86dc003a00 0x7f86dc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:20.895 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.881+0000 7f86ee7fc700 1 --2- 172.21.15.143:0/168474173 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f86dc004400 0x7f86dc0048d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:20.895 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.881+0000 7f86ef7fe700 1 --2- 172.21.15.143:0/168474173 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f86dc0050d0 0x7f86dc00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:20.895 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.881+0000 7f86eeffd700 1 -- 172.21.15.143:0/168474173 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f86dc0050d0 msgr2=0x7f86dc00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:20.896 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.881+0000 7f86eeffd700 1 --2- 172.21.15.143:0/168474173 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f86dc0050d0 0x7f86dc00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:20.896 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.881+0000 7f86eeffd700 1 -- 172.21.15.143:0/168474173 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f86dc004400 msgr2=0x7f86dc0048d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:20.896 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.881+0000 7f86eeffd700 1 --2- 172.21.15.143:0/168474173 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f86dc004400 0x7f86dc0048d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:20.897 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.881+0000 7f86eeffd700 1 -- 172.21.15.143:0/168474173 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86dc0035b0 con 0x7f86dc003a00 2023-11-08T19:31:20.897 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.881+0000 7f86ef7fe700 1 --2- 172.21.15.143:0/168474173 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f86dc0050d0 0x7f86dc00ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:31:20.897 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.881+0000 7f86ef7fe700 1 -- 172.21.15.143:0/168474173 reap_dead start 2023-11-08T19:31:20.897 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.881+0000 7f86eeffd700 1 --2- 172.21.15.143:0/168474173 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86dc003a00 0x7f86dc003de0 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f86e0001930 tx=0x7f86dc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=371b41f660a2f094 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:20.898 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.882+0000 7f86edffb700 1 -- 172.21.15.143:0/168474173 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e00013a0 con 0x7f86dc003a00 2023-11-08T19:31:20.898 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.882+0000 7f86edffb700 1 -- 172.21.15.143:0/168474173 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86e00052f0 con 0x7f86dc003a00 2023-11-08T19:31:20.898 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.882+0000 7f86edffb700 1 -- 172.21.15.143:0/168474173 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e00036d0 con 0x7f86dc003a00 2023-11-08T19:31:20.899 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.882+0000 7f86f596c700 1 -- 172.21.15.143:0/168474173 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86dc003a00 msgr2=0x7f86dc003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:20.899 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.882+0000 7f86f596c700 1 --2- 172.21.15.143:0/168474173 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86dc003a00 0x7f86dc003de0 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f86e0001930 tx=0x7f86dc005880 comp rx=0 tx=0).stop 2023-11-08T19:31:20.899 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.882+0000 7f86f596c700 1 -- 172.21.15.143:0/168474173 shutdown_connections 2023-11-08T19:31:20.899 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.882+0000 7f86f596c700 1 --2- 172.21.15.143:0/168474173 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86dc003a00 0x7f86dc003de0 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:20.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.882+0000 7f86f596c700 1 -- 172.21.15.143:0/168474173 >> 172.21.15.143:0/168474173 conn(0x7f86f0099ab0 msgr2=0x7f86f00d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:20.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.882+0000 7f86f596c700 1 -- 172.21.15.143:0/168474173 shutdown_connections 2023-11-08T19:31:20.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.882+0000 7f86f596c700 1 -- 172.21.15.143:0/168474173 wait complete. 2023-11-08T19:31:20.901 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.883+0000 7f86f596c700 1 Processor -- start 2023-11-08T19:31:20.901 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.884+0000 7f86f596c700 1 -- start start 2023-11-08T19:31:20.901 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.884+0000 7f86f596c700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f86dc003a00 0x7f86f0151c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:20.901 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.884+0000 7f86f596c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f86f00b0b80 0x7f86f00cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:20.902 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.884+0000 7f86f596c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86f00d00c0 0x7f86f00c6480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:20.902 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.884+0000 7f86f596c700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86f01f7740 con 0x7f86f00d00c0 2023-11-08T19:31:20.902 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.884+0000 7f86f596c700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86f00d0570 con 0x7f86f00b0b80 2023-11-08T19:31:20.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.884+0000 7f86f596c700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86f01f6ac0 con 0x7f86dc003a00 2023-11-08T19:31:20.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.884+0000 7f86ef7fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86f00d00c0 0x7f86f00c6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:20.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.884+0000 7f86ef7fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86f00d00c0 0x7f86f00c6480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57298/0 (socket says 172.21.15.143:57298) 2023-11-08T19:31:20.904 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.884+0000 7f86ef7fe700 1 -- 172.21.15.143:0/33503858 learned_addr learned my addr 172.21.15.143:0/33503858 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:31:20.904 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.885+0000 7f86eeffd700 1 --2- 172.21.15.143:0/33503858 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f86dc003a00 0x7f86f0151c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:20.904 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.885+0000 7f86ee7fc700 1 --2- 172.21.15.143:0/33503858 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f86f00b0b80 0x7f86f00cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:20.904 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.885+0000 7f86ef7fe700 1 -- 172.21.15.143:0/33503858 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f86dc003a00 msgr2=0x7f86f0151c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:20.905 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.885+0000 7f86ef7fe700 1 --2- 172.21.15.143:0/33503858 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f86dc003a00 0x7f86f0151c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:20.905 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.885+0000 7f86ef7fe700 1 -- 172.21.15.143:0/33503858 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f86f00b0b80 msgr2=0x7f86f00cfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:20.905 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.885+0000 7f86ef7fe700 1 --2- 172.21.15.143:0/33503858 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f86f00b0b80 0x7f86f00cfb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:20.905 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.885+0000 7f86ef7fe700 1 -- 172.21.15.143:0/33503858 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86f015e260 con 0x7f86f00d00c0 2023-11-08T19:31:20.906 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.885+0000 7f86ef7fe700 1 --2- 172.21.15.143:0/33503858 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86f00d00c0 0x7f86f00c6480 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f86f00708d0 tx=0x7f86e80114b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:20.906 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.885+0000 7f86e7fff700 1 -- 172.21.15.143:0/33503858 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e800e030 con 0x7f86f00d00c0 2023-11-08T19:31:20.907 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.885+0000 7f86e7fff700 1 -- 172.21.15.143:0/33503858 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86e8011c90 con 0x7f86f00d00c0 2023-11-08T19:31:20.907 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.885+0000 7f86f596c700 1 -- 172.21.15.143:0/33503858 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86f01f6da0 con 0x7f86f00d00c0 2023-11-08T19:31:20.907 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.885+0000 7f86e7fff700 1 -- 172.21.15.143:0/33503858 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e8016470 con 0x7f86f00d00c0 2023-11-08T19:31:20.908 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.885+0000 7f86f596c700 1 -- 172.21.15.143:0/33503858 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86f01f7360 con 0x7f86f00d00c0 2023-11-08T19:31:20.908 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.886+0000 7f86f596c700 1 -- 172.21.15.143:0/33503858 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86f01f7880 con 0x7f86f00d00c0 2023-11-08T19:31:20.908 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.886+0000 7f86e7fff700 1 -- 172.21.15.143:0/33503858 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f86e80165d0 con 0x7f86f00d00c0 2023-11-08T19:31:20.908 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.887+0000 7f86e7fff700 1 --2- 172.21.15.143:0/33503858 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f86d006af00 0x7f86d006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:20.909 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.887+0000 7f86e7fff700 1 -- 172.21.15.143:0/33503858 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6651+0+0 (secure 0 0 0) 0x7f86e800f070 con 0x7f86f00d00c0 2023-11-08T19:31:20.909 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.887+0000 7f86eeffd700 1 --2- 172.21.15.143:0/33503858 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f86d006af00 0x7f86d006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:20.910 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.887+0000 7f86eeffd700 1 --2- 172.21.15.143:0/33503858 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f86d006af00 0x7f86d006d3c0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f86e0015040 tx=0x7f86e0005000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:20.910 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:20.891+0000 7f86e7fff700 1 -- 172.21.15.143:0/33503858 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f86f01f7880 con 0x7f86f00d00c0 2023-11-08T19:31:21.082 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.080+0000 7f86f596c700 1 -- 172.21.15.143:0/33503858 --> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f86f015e0e0 con 0x7f86d006af00 2023-11-08T19:31:21.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.081+0000 7f86e7fff700 1 -- 172.21.15.143:0/33503858 <== mgr.4099 v2:172.21.15.174:6832/4037280178 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f86e0001320 con 0x7f86d006af00 2023-11-08T19:31:21.085 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.084+0000 7f86f596c700 1 -- 172.21.15.143:0/33503858 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f86d006af00 msgr2=0x7f86d006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:21.086 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.084+0000 7f86f596c700 1 --2- 172.21.15.143:0/33503858 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f86d006af00 0x7f86d006d3c0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f86e0015040 tx=0x7f86e0005000 comp rx=0 tx=0).stop 2023-11-08T19:31:21.086 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.084+0000 7f86f596c700 1 -- 172.21.15.143:0/33503858 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86f00d00c0 msgr2=0x7f86f00c6480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:21.086 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.084+0000 7f86f596c700 1 --2- 172.21.15.143:0/33503858 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86f00d00c0 0x7f86f00c6480 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f86f00708d0 tx=0x7f86e80114b0 comp rx=0 tx=0).stop 2023-11-08T19:31:21.087 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.084+0000 7f86f596c700 1 -- 172.21.15.143:0/33503858 shutdown_connections 2023-11-08T19:31:21.087 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.084+0000 7f86f596c700 1 --2- 172.21.15.143:0/33503858 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f86d006af00 0x7f86d006d3c0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:21.087 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.084+0000 7f86f596c700 1 --2- 172.21.15.143:0/33503858 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f86dc003a00 0x7f86f0151c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:21.087 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.084+0000 7f86f596c700 1 --2- 172.21.15.143:0/33503858 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86f00d00c0 0x7f86f00c6480 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:21.088 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.084+0000 7f86f596c700 1 --2- 172.21.15.143:0/33503858 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f86f00b0b80 0x7f86f00cfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:21.088 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.084+0000 7f86f596c700 1 -- 172.21.15.143:0/33503858 >> 172.21.15.143:0/33503858 conn(0x7f86f0099ab0 msgr2=0x7f86f00af6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:21.088 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.084+0000 7f86f596c700 1 -- 172.21.15.143:0/33503858 shutdown_connections 2023-11-08T19:31:21.088 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.084+0000 7f86f596c700 1 -- 172.21.15.143:0/33503858 wait complete. 2023-11-08T19:31:21.089 INFO:teuthology.orchestra.run.smithi143.stderr:instructed osd(s) 4 to deep-scrub 2023-11-08T19:31:21.097 INFO:tasks.ceph:Scrubbing osd.5 2023-11-08T19:31:21.098 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2023-11-08T19:31:21.223 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.222+0000 7fd968264700 1 Processor -- start 2023-11-08T19:31:21.224 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.222+0000 7fd968264700 1 -- start start 2023-11-08T19:31:21.226 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.223+0000 7fd968264700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fd9600d1d80 0x7fd9600b2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:21.227 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.223+0000 7fd968264700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9600d0420 con 0x7fd9600b1730 2023-11-08T19:31:21.227 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.223+0000 7fd968264700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9600cf810 con 0x7fd96015bb20 2023-11-08T19:31:21.227 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.223+0000 7fd968264700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9600cf600 con 0x7fd9600d1d80 2023-11-08T19:31:21.228 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.223+0000 7fd966801700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fd9600b1730 0x7fd9600cc4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:53460/0 (socket says 172.21.15.143:53460) 2023-11-08T19:31:21.228 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.223+0000 7fd966801700 1 -- 172.21.15.143:0/3008896566 learned_addr learned my addr 172.21.15.143:0/3008896566 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:31:21.228 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.223+0000 7fd9657ff700 1 --2- 172.21.15.143:0/3008896566 >> v2:172.21.15.174:3300/0 conn(0x7fd9600d1d80 0x7fd9600b2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:21.228 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.223+0000 7fd964ffe700 1 -- 172.21.15.143:0/3008896566 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3285214381 0 0) 0x7fd9600d0420 con 0x7fd9600b1730 2023-11-08T19:31:21.229 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.223+0000 7fd964ffe700 1 -- 172.21.15.143:0/3008896566 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd94c0035b0 con 0x7fd9600b1730 2023-11-08T19:31:21.229 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.223+0000 7fd9657ff700 1 -- 172.21.15.143:0/3008896566 >> v1:172.21.15.174:6789/0 conn(0x7fd96015bb20 legacy=0x7fd9600d1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:21.229 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.223+0000 7fd964ffe700 1 -- 172.21.15.143:0/3008896566 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1514479412 0 0) 0x7fd94c0035b0 con 0x7fd9600b1730 2023-11-08T19:31:21.230 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.223+0000 7fd9657ff700 1 -- 172.21.15.143:0/3008896566 >> v1:172.21.15.143:6789/0 conn(0x7fd9600b1730 legacy=0x7fd9600cc4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:21.230 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.223+0000 7fd9657ff700 1 -- 172.21.15.143:0/3008896566 --> v2:172.21.15.174:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9600cfaf0 con 0x7fd9600d1d80 2023-11-08T19:31:21.231 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.224+0000 7fd9657ff700 1 --2- 172.21.15.143:0/3008896566 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd9600d1d80 0x7fd9600b2670 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fd950009840 tx=0x7fd950009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51e6166d69fddec2 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:21.232 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.224+0000 7fd964ffe700 1 -- 172.21.15.143:0/3008896566 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9580019f0 con 0x7fd9600d1d80 2023-11-08T19:31:21.232 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.224+0000 7fd964ffe700 1 -- 172.21.15.143:0/3008896566 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd950004030 con 0x7fd9600d1d80 2023-11-08T19:31:21.233 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.224+0000 7fd964ffe700 1 -- 172.21.15.143:0/3008896566 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9500043c0 con 0x7fd9600d1d80 2023-11-08T19:31:21.234 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.225+0000 7fd968264700 1 -- 172.21.15.143:0/3008896566 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd9600d1d80 msgr2=0x7fd9600b2670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:21.234 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.225+0000 7fd968264700 1 --2- 172.21.15.143:0/3008896566 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd9600d1d80 0x7fd9600b2670 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fd950009840 tx=0x7fd950009b50 comp rx=0 tx=0).stop 2023-11-08T19:31:21.235 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.225+0000 7fd968264700 1 -- 172.21.15.143:0/3008896566 shutdown_connections 2023-11-08T19:31:21.235 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.225+0000 7fd968264700 1 --2- 172.21.15.143:0/3008896566 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd9600d1d80 0x7fd9600b2670 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:21.236 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.225+0000 7fd968264700 1 -- 172.21.15.143:0/3008896566 >> 172.21.15.143:0/3008896566 conn(0x7fd96008b000 msgr2=0x7fd96009ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:21.236 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.225+0000 7fd968264700 1 -- 172.21.15.143:0/3008896566 shutdown_connections 2023-11-08T19:31:21.237 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.225+0000 7fd968264700 1 -- 172.21.15.143:0/3008896566 wait complete. 2023-11-08T19:31:21.243 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.226+0000 7fd968264700 1 Processor -- start 2023-11-08T19:31:21.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.226+0000 7fd968264700 1 -- start start 2023-11-08T19:31:21.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.227+0000 7fd968264700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd9600b1730 0x7fd96015a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:21.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.227+0000 7fd968264700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd9600d1d80 0x7fd96006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:21.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.227+0000 7fd968264700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd96015bb20 0x7fd96006d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:21.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.227+0000 7fd968264700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9601f46d0 con 0x7fd9600d1d80 2023-11-08T19:31:21.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.227+0000 7fd968264700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9601f3a50 con 0x7fd9600b1730 2023-11-08T19:31:21.246 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.227+0000 7fd968264700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9601f3d30 con 0x7fd96015bb20 2023-11-08T19:31:21.246 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.227+0000 7fd9657ff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd9600d1d80 0x7fd96006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:21.246 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.227+0000 7fd9657ff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd9600d1d80 0x7fd96006d390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57310/0 (socket says 172.21.15.143:57310) 2023-11-08T19:31:21.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.227+0000 7fd9657ff700 1 -- 172.21.15.143:0/2441034345 learned_addr learned my addr 172.21.15.143:0/2441034345 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:31:21.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.227+0000 7fd9657ff700 1 -- 172.21.15.143:0/2441034345 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd96015bb20 msgr2=0x7fd96006d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:21.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.227+0000 7fd966801700 1 --2- 172.21.15.143:0/2441034345 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd96015bb20 0x7fd96006d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:21.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.227+0000 7fd966000700 1 --2- 172.21.15.143:0/2441034345 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd9600b1730 0x7fd96015a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:21.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.227+0000 7fd9657ff700 1 --2- 172.21.15.143:0/2441034345 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd96015bb20 0x7fd96006d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:21.249 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.227+0000 7fd9657ff700 1 -- 172.21.15.143:0/2441034345 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd9600b1730 msgr2=0x7fd96015a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:21.249 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.227+0000 7fd9657ff700 1 --2- 172.21.15.143:0/2441034345 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd9600b1730 0x7fd96015a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:21.249 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.227+0000 7fd9657ff700 1 -- 172.21.15.143:0/2441034345 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9500094a0 con 0x7fd9600d1d80 2023-11-08T19:31:21.249 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.227+0000 7fd966801700 1 --2- 172.21.15.143:0/2441034345 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd96015bb20 0x7fd96006d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:31:21.250 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.227+0000 7fd966000700 1 --2- 172.21.15.143:0/2441034345 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd9600b1730 0x7fd96015a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:31:21.250 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.227+0000 7fd9657ff700 1 --2- 172.21.15.143:0/2441034345 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd9600d1d80 0x7fd96006d390 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7fd9600cf5b0 tx=0x7fd950004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:21.250 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.227+0000 7fd9567fc700 1 -- 172.21.15.143:0/2441034345 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd950010790 con 0x7fd9600d1d80 2023-11-08T19:31:21.251 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.227+0000 7fd9567fc700 1 -- 172.21.15.143:0/2441034345 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9500108f0 con 0x7fd9600d1d80 2023-11-08T19:31:21.251 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.227+0000 7fd968264700 1 -- 172.21.15.143:0/2441034345 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9601f4010 con 0x7fd9600d1d80 2023-11-08T19:31:21.252 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.227+0000 7fd9567fc700 1 -- 172.21.15.143:0/2441034345 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd950010ca0 con 0x7fd9600d1d80 2023-11-08T19:31:21.252 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.227+0000 7fd968264700 1 -- 172.21.15.143:0/2441034345 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd96015ade0 con 0x7fd9600d1d80 2023-11-08T19:31:21.253 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.228+0000 7fd968264700 1 -- 172.21.15.143:0/2441034345 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd9600d00b0 con 0x7fd9600d1d80 2023-11-08T19:31:21.253 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.229+0000 7fd9567fc700 1 -- 172.21.15.143:0/2441034345 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd950025460 con 0x7fd9600d1d80 2023-11-08T19:31:21.254 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.229+0000 7fd9567fc700 1 --2- 172.21.15.143:0/2441034345 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd94406af00 0x7fd94406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:21.254 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.229+0000 7fd9567fc700 1 -- 172.21.15.143:0/2441034345 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6651+0+0 (secure 0 0 0) 0x7fd95008b360 con 0x7fd9600d1d80 2023-11-08T19:31:21.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.229+0000 7fd9567fc700 1 --2- 172.21.15.143:0/2441034345 >> [v2:172.21.15.174:6808/1777923303,v1:172.21.15.174:6809/1777923303] conn(0x7fd9440738e0 0x7fd944075d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:21.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.229+0000 7fd966000700 1 --2- 172.21.15.143:0/2441034345 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd94406af00 0x7fd94406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:21.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.229+0000 7fd9567fc700 1 -- 172.21.15.143:0/2441034345 --> [v2:172.21.15.174:6808/1777923303,v1:172.21.15.174:6809/1777923303] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd9440763d0 con 0x7fd9440738e0 2023-11-08T19:31:21.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.229+0000 7fd9567fc700 1 -- 172.21.15.143:0/2441034345 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_get_version_reply(handle=1 version=107) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd95008da20 con 0x7fd9600d1d80 2023-11-08T19:31:21.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.229+0000 7fd966801700 1 --2- 172.21.15.143:0/2441034345 >> [v2:172.21.15.174:6808/1777923303,v1:172.21.15.174:6809/1777923303] conn(0x7fd9440738e0 0x7fd944075d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:21.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.230+0000 7fd966000700 1 --2- 172.21.15.143:0/2441034345 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd94406af00 0x7fd94406d3c0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fd95c001800 tx=0x7fd95c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:21.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.230+0000 7fd966801700 1 --2- 172.21.15.143:0/2441034345 >> [v2:172.21.15.174:6808/1777923303,v1:172.21.15.174:6809/1777923303] conn(0x7fd9440738e0 0x7fd944075d00 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:21.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.231+0000 7fd9567fc700 1 -- 172.21.15.143:0/2441034345 <== osd.5 v2:172.21.15.174:6808/1777923303 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26745 (crc 0 0 0) 0x7fd9440763d0 con 0x7fd9440738e0 2023-11-08T19:31:21.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.246+0000 7fd968264700 1 -- 172.21.15.143:0/2441034345 --> [v2:172.21.15.174:6808/1777923303,v1:172.21.15.174:6809/1777923303] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fd9600cfdd0 con 0x7fd9440738e0 2023-11-08T19:31:21.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.246+0000 7fd9567fc700 1 -- 172.21.15.143:0/2441034345 <== osd.5 v2:172.21.15.174:6808/1777923303 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fd9580070f0 con 0x7fd9440738e0 2023-11-08T19:31:21.261 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2023-11-08T19:31:21.261 INFO:teuthology.orchestra.run.smithi143.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-08T19:31:21.262 INFO:teuthology.orchestra.run.smithi143.stdout:} 2023-11-08T19:31:21.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.246+0000 7fd968264700 1 -- 172.21.15.143:0/2441034345 >> [v2:172.21.15.174:6808/1777923303,v1:172.21.15.174:6809/1777923303] conn(0x7fd9440738e0 msgr2=0x7fd944075d00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:21.264 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.246+0000 7fd968264700 1 --2- 172.21.15.143:0/2441034345 >> [v2:172.21.15.174:6808/1777923303,v1:172.21.15.174:6809/1777923303] conn(0x7fd9440738e0 0x7fd944075d00 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:21.264 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.246+0000 7fd968264700 1 -- 172.21.15.143:0/2441034345 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd94406af00 msgr2=0x7fd94406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:21.264 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.246+0000 7fd968264700 1 --2- 172.21.15.143:0/2441034345 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fd94406af00 0x7fd94406d3c0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fd95c001800 tx=0x7fd95c009040 comp rx=0 tx=0).stop 2023-11-08T19:31:21.265 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.246+0000 7fd968264700 1 -- 172.21.15.143:0/2441034345 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd9600d1d80 msgr2=0x7fd96006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:21.265 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.246+0000 7fd968264700 1 --2- 172.21.15.143:0/2441034345 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd9600d1d80 0x7fd96006d390 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7fd9600cf5b0 tx=0x7fd950004750 comp rx=0 tx=0).stop 2023-11-08T19:31:21.265 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.246+0000 7fd966801700 1 -- 172.21.15.143:0/2441034345 reap_dead start 2023-11-08T19:31:21.266 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.247+0000 7fd968264700 1 -- 172.21.15.143:0/2441034345 shutdown_connections 2023-11-08T19:31:21.266 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.247+0000 7fd968264700 1 -- 172.21.15.143:0/2441034345 >> 172.21.15.143:0/2441034345 conn(0x7fd96008b000 msgr2=0x7fd9600b34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:21.266 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.247+0000 7fd968264700 1 -- 172.21.15.143:0/2441034345 shutdown_connections 2023-11-08T19:31:21.266 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.247+0000 7fd968264700 1 -- 172.21.15.143:0/2441034345 wait complete. 2023-11-08T19:31:21.267 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2023-11-08T19:31:21.388 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.386+0000 7f36d39e2700 1 Processor -- start 2023-11-08T19:31:21.388 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.387+0000 7f36d39e2700 1 -- start start 2023-11-08T19:31:21.391 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.387+0000 7f36d39e2700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f36cc15dd00 0x7f36cc0cca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:21.391 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.387+0000 7f36d39e2700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f36cc06b320 0x7f36cc0cb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:21.391 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.387+0000 7f36d39e2700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36cc0ccfb0 con 0x7f36cc1618f0 2023-11-08T19:31:21.392 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.387+0000 7f36d39e2700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36cc0cad50 con 0x7f36cc06b320 2023-11-08T19:31:21.392 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.387+0000 7f36d39e2700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36cc161df0 con 0x7f36cc15dd00 2023-11-08T19:31:21.392 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.387+0000 7f36d1f7f700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f36cc06b320 0x7f36cc0cb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:21.393 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.388+0000 7f36d1f7f700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f36cc06b320 0x7f36cc0cb1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57320/0 (socket says 172.21.15.143:57320) 2023-11-08T19:31:21.393 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.388+0000 7f36d1f7f700 1 -- 172.21.15.143:0/3076595681 learned_addr learned my addr 172.21.15.143:0/3076595681 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:31:21.393 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.388+0000 7f36d177e700 1 --2- 172.21.15.143:0/3076595681 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f36cc15dd00 0x7f36cc0cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:21.394 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.388+0000 7f36d1f7f700 1 -- 172.21.15.143:0/3076595681 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f36cc15dd00 msgr2=0x7f36cc0cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:21.394 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.388+0000 7f36d1f7f700 1 --2- 172.21.15.143:0/3076595681 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f36cc15dd00 0x7f36cc0cca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:21.395 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.388+0000 7f36d1f7f700 1 -- 172.21.15.143:0/3076595681 >> v1:172.21.15.143:6789/0 conn(0x7f36cc1618f0 legacy=0x7f36cc06ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:21.395 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.388+0000 7f36d1f7f700 1 -- 172.21.15.143:0/3076595681 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36cc1620d0 con 0x7f36cc06b320 2023-11-08T19:31:21.396 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.388+0000 7f36c3fff700 1 -- 172.21.15.143:0/3076595681 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1937198433 0 0) 0x7f36cc0ccfb0 con 0x7f36cc1618f0 2023-11-08T19:31:21.396 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.388+0000 7f36d1f7f700 1 --2- 172.21.15.143:0/3076595681 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f36cc06b320 0x7f36cc0cb1a0 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f36c4009e50 tx=0x7f36c400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f33ff2b289b14abf server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:21.397 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.388+0000 7f36c3fff700 1 -- 172.21.15.143:0/3076595681 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36c400e070 con 0x7f36cc06b320 2023-11-08T19:31:21.397 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.388+0000 7f36c3fff700 1 -- 172.21.15.143:0/3076595681 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36c400b910 con 0x7f36cc06b320 2023-11-08T19:31:21.398 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.388+0000 7f36c3fff700 1 -- 172.21.15.143:0/3076595681 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36c400bcc0 con 0x7f36cc06b320 2023-11-08T19:31:21.398 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.389+0000 7f36d39e2700 1 -- 172.21.15.143:0/3076595681 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f36cc06b320 msgr2=0x7f36cc0cb1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:21.399 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.389+0000 7f36d39e2700 1 --2- 172.21.15.143:0/3076595681 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f36cc06b320 0x7f36cc0cb1a0 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f36c4009e50 tx=0x7f36c400b330 comp rx=0 tx=0).stop 2023-11-08T19:31:21.399 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.389+0000 7f36d39e2700 1 -- 172.21.15.143:0/3076595681 shutdown_connections 2023-11-08T19:31:21.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.389+0000 7f36d39e2700 1 --2- 172.21.15.143:0/3076595681 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f36cc06b320 0x7f36cc0cb1a0 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:21.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.389+0000 7f36d39e2700 1 --2- 172.21.15.143:0/3076595681 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f36cc15dd00 0x7f36cc0cca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:21.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.389+0000 7f36d39e2700 1 -- 172.21.15.143:0/3076595681 >> 172.21.15.143:0/3076595681 conn(0x7f36cc099ab0 msgr2=0x7f36cc0d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:21.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.389+0000 7f36d39e2700 1 -- 172.21.15.143:0/3076595681 shutdown_connections 2023-11-08T19:31:21.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.389+0000 7f36d39e2700 1 -- 172.21.15.143:0/3076595681 wait complete. 2023-11-08T19:31:21.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.390+0000 7f36d39e2700 1 Processor -- start 2023-11-08T19:31:21.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.390+0000 7f36d39e2700 1 -- start start 2023-11-08T19:31:21.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.391+0000 7f36d39e2700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f36cc06b320 0x7f36cc0b2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:21.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.391+0000 7f36d39e2700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f36cc15dd00 0x7f36cc158160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:21.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.391+0000 7f36d39e2700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f36cc1618f0 0x7f36cc0c5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:21.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.391+0000 7f36d39e2700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36cc1586a0 con 0x7f36cc1618f0 2023-11-08T19:31:21.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.391+0000 7f36d39e2700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36cc0c6120 con 0x7f36cc15dd00 2023-11-08T19:31:21.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.391+0000 7f36d39e2700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36cc0c63c0 con 0x7f36cc06b320 2023-11-08T19:31:21.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.391+0000 7f36d1f7f700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f36cc1618f0 0x7f36cc0c5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:21.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.391+0000 7f36d1f7f700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f36cc1618f0 0x7f36cc0c5be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57326/0 (socket says 172.21.15.143:57326) 2023-11-08T19:31:21.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.391+0000 7f36d1f7f700 1 -- 172.21.15.143:0/1537958025 learned_addr learned my addr 172.21.15.143:0/1537958025 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:31:21.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.391+0000 7f36d0f7d700 1 --2- 172.21.15.143:0/1537958025 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f36cc15dd00 0x7f36cc158160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:21.409 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.392+0000 7f36d177e700 1 --2- 172.21.15.143:0/1537958025 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f36cc06b320 0x7f36cc0b2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:21.409 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.392+0000 7f36d1f7f700 1 -- 172.21.15.143:0/1537958025 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f36cc06b320 msgr2=0x7f36cc0b2600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:21.409 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.392+0000 7f36d1f7f700 1 --2- 172.21.15.143:0/1537958025 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f36cc06b320 0x7f36cc0b2600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:21.409 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.392+0000 7f36d1f7f700 1 -- 172.21.15.143:0/1537958025 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f36cc15dd00 msgr2=0x7f36cc158160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:21.410 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.392+0000 7f36d1f7f700 1 --2- 172.21.15.143:0/1537958025 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f36cc15dd00 0x7f36cc158160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:21.410 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.392+0000 7f36d1f7f700 1 -- 172.21.15.143:0/1537958025 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36c4009b30 con 0x7f36cc1618f0 2023-11-08T19:31:21.410 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.392+0000 7f36d177e700 1 --2- 172.21.15.143:0/1537958025 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f36cc06b320 0x7f36cc0b2600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:31:21.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.392+0000 7f36d0f7d700 1 --2- 172.21.15.143:0/1537958025 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f36cc15dd00 0x7f36cc158160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:31:21.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.392+0000 7f36d1f7f700 1 --2- 172.21.15.143:0/1537958025 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f36cc1618f0 0x7f36cc0c5be0 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f36c400b680 tx=0x7f36c4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:21.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.392+0000 7f36c1ffb700 1 -- 172.21.15.143:0/1537958025 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36c40099b0 con 0x7f36cc1618f0 2023-11-08T19:31:21.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.392+0000 7f36c1ffb700 1 -- 172.21.15.143:0/1537958025 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36c40092e0 con 0x7f36cc1618f0 2023-11-08T19:31:21.412 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.392+0000 7f36d39e2700 1 -- 172.21.15.143:0/1537958025 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36cc0c66a0 con 0x7f36cc1618f0 2023-11-08T19:31:21.412 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.392+0000 7f36c1ffb700 1 -- 172.21.15.143:0/1537958025 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36c401c660 con 0x7f36cc1618f0 2023-11-08T19:31:21.412 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.392+0000 7f36d39e2700 1 -- 172.21.15.143:0/1537958025 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36cc0c45b0 con 0x7f36cc1618f0 2023-11-08T19:31:21.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.393+0000 7f36d39e2700 1 -- 172.21.15.143:0/1537958025 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36cc0aff60 con 0x7f36cc1618f0 2023-11-08T19:31:21.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.394+0000 7f36c1ffb700 1 -- 172.21.15.143:0/1537958025 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f36c401c7c0 con 0x7f36cc1618f0 2023-11-08T19:31:21.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.394+0000 7f36c1ffb700 1 --2- 172.21.15.143:0/1537958025 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f36b006af00 0x7f36b006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:21.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.394+0000 7f36c1ffb700 1 -- 172.21.15.143:0/1537958025 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6651+0+0 (secure 0 0 0) 0x7f36c408b340 con 0x7f36cc1618f0 2023-11-08T19:31:21.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.394+0000 7f36d177e700 1 --2- 172.21.15.143:0/1537958025 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f36b006af00 0x7f36b006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:21.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.395+0000 7f36d177e700 1 --2- 172.21.15.143:0/1537958025 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f36b006af00 0x7f36b006d3c0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f36bc009c10 tx=0x7f36bc008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:21.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.399+0000 7f36c1ffb700 1 -- 172.21.15.143:0/1537958025 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f36c40560d0 con 0x7f36cc1618f0 2023-11-08T19:31:21.586 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.585+0000 7f36d39e2700 1 -- 172.21.15.143:0/1537958025 --> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7f36cc162690 con 0x7f36b006af00 2023-11-08T19:31:21.587 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.586+0000 7f36c1ffb700 1 -- 172.21.15.143:0/1537958025 <== mgr.4099 v2:172.21.15.174:6832/4037280178 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f36bc004030 con 0x7f36b006af00 2023-11-08T19:31:21.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.588+0000 7f36d39e2700 1 -- 172.21.15.143:0/1537958025 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f36b006af00 msgr2=0x7f36b006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:21.590 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.588+0000 7f36d39e2700 1 --2- 172.21.15.143:0/1537958025 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f36b006af00 0x7f36b006d3c0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f36bc009c10 tx=0x7f36bc008040 comp rx=0 tx=0).stop 2023-11-08T19:31:21.590 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.588+0000 7f36d39e2700 1 -- 172.21.15.143:0/1537958025 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f36cc1618f0 msgr2=0x7f36cc0c5be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:21.590 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.588+0000 7f36d39e2700 1 --2- 172.21.15.143:0/1537958025 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f36cc1618f0 0x7f36cc0c5be0 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f36c400b680 tx=0x7f36c4004ad0 comp rx=0 tx=0).stop 2023-11-08T19:31:21.591 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.588+0000 7f36d39e2700 1 -- 172.21.15.143:0/1537958025 shutdown_connections 2023-11-08T19:31:21.591 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.588+0000 7f36d39e2700 1 --2- 172.21.15.143:0/1537958025 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f36b006af00 0x7f36b006d3c0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:21.591 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.588+0000 7f36d39e2700 1 --2- 172.21.15.143:0/1537958025 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f36cc06b320 0x7f36cc0b2600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:21.592 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.588+0000 7f36d39e2700 1 --2- 172.21.15.143:0/1537958025 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f36cc1618f0 0x7f36cc0c5be0 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:21.592 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.588+0000 7f36d39e2700 1 --2- 172.21.15.143:0/1537958025 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f36cc15dd00 0x7f36cc158160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:21.592 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.588+0000 7f36d39e2700 1 -- 172.21.15.143:0/1537958025 >> 172.21.15.143:0/1537958025 conn(0x7f36cc099ab0 msgr2=0x7f36cc0ad380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:21.593 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.588+0000 7f36d39e2700 1 -- 172.21.15.143:0/1537958025 shutdown_connections 2023-11-08T19:31:21.593 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.588+0000 7f36d39e2700 1 -- 172.21.15.143:0/1537958025 wait complete. 2023-11-08T19:31:21.593 INFO:teuthology.orchestra.run.smithi143.stderr:instructed osd(s) 5 to deep-scrub 2023-11-08T19:31:21.601 INFO:tasks.ceph:Scrubbing osd.6 2023-11-08T19:31:21.602 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2023-11-08T19:31:21.726 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.724+0000 7f9e7830f700 1 Processor -- start 2023-11-08T19:31:21.726 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.724+0000 7f9e7830f700 1 -- start start 2023-11-08T19:31:21.729 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.725+0000 7f9e7830f700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f9e7006b240 0x7f9e700caff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:21.729 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.725+0000 7f9e7830f700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9e700cb530 0x7f9e700b22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:21.729 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.725+0000 7f9e7830f700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e70161890 con 0x7f9e7015dba0 2023-11-08T19:31:21.730 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.725+0000 7f9e7830f700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e7006cd20 con 0x7f9e7006b240 2023-11-08T19:31:21.730 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.725+0000 7f9e7830f700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e7006b740 con 0x7f9e700cb530 2023-11-08T19:31:21.730 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.725+0000 7f9e758aa700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f9e7006b240 0x7f9e700caff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:21.731 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.725+0000 7f9e758aa700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f9e7006b240 0x7f9e700caff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57332/0 (socket says 172.21.15.143:57332) 2023-11-08T19:31:21.731 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.725+0000 7f9e758aa700 1 -- 172.21.15.143:0/455411861 learned_addr learned my addr 172.21.15.143:0/455411861 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:31:21.731 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.726+0000 7f9e768ac700 1 --2- 172.21.15.143:0/455411861 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9e700cb530 0x7f9e700b22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:21.732 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.726+0000 7f9e758aa700 1 -- 172.21.15.143:0/455411861 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9e700cb530 msgr2=0x7f9e700b22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:21.732 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.726+0000 7f9e758aa700 1 --2- 172.21.15.143:0/455411861 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9e700cb530 0x7f9e700b22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:21.732 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.726+0000 7f9e758aa700 1 -- 172.21.15.143:0/455411861 >> v1:172.21.15.174:6789/0 conn(0x7f9e7015dba0 legacy=0x7f9e7006c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:21.733 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.726+0000 7f9e758aa700 1 -- 172.21.15.143:0/455411861 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e7006ba20 con 0x7f9e7006b240 2023-11-08T19:31:21.733 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.726+0000 7f9e758aa700 1 --2- 172.21.15.143:0/455411861 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9e7006b240 0x7f9e700caff0 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c0098f0 tx=0x7f9e6c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=865efe6973a7dc5f server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:21.733 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.726+0000 7f9e750a9700 1 -- 172.21.15.143:0/455411861 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e6c011830 con 0x7f9e7006b240 2023-11-08T19:31:21.733 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.726+0000 7f9e750a9700 1 -- 172.21.15.143:0/455411861 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e6c011990 con 0x7f9e7006b240 2023-11-08T19:31:21.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.726+0000 7f9e750a9700 1 -- 172.21.15.143:0/455411861 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e6c011d20 con 0x7f9e7006b240 2023-11-08T19:31:21.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.727+0000 7f9e7830f700 1 -- 172.21.15.143:0/455411861 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9e7006b240 msgr2=0x7f9e700caff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:21.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.727+0000 7f9e7830f700 1 --2- 172.21.15.143:0/455411861 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9e7006b240 0x7f9e700caff0 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c0098f0 tx=0x7f9e6c009c00 comp rx=0 tx=0).stop 2023-11-08T19:31:21.735 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.727+0000 7f9e7830f700 1 -- 172.21.15.143:0/455411861 shutdown_connections 2023-11-08T19:31:21.735 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.727+0000 7f9e7830f700 1 --2- 172.21.15.143:0/455411861 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9e700cb530 0x7f9e700b22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:21.735 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.727+0000 7f9e7830f700 1 --2- 172.21.15.143:0/455411861 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9e7006b240 0x7f9e700caff0 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:21.735 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.727+0000 7f9e7830f700 1 -- 172.21.15.143:0/455411861 >> 172.21.15.143:0/455411861 conn(0x7f9e700cd2e0 msgr2=0x7f9e700cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:21.736 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.727+0000 7f9e7830f700 1 -- 172.21.15.143:0/455411861 shutdown_connections 2023-11-08T19:31:21.736 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.727+0000 7f9e7830f700 1 -- 172.21.15.143:0/455411861 wait complete. 2023-11-08T19:31:21.736 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.728+0000 7f9e7830f700 1 Processor -- start 2023-11-08T19:31:21.737 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.728+0000 7f9e7830f700 1 -- start start 2023-11-08T19:31:21.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.729+0000 7f9e7830f700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9e7006b240 0x7f9e700adde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:21.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.729+0000 7f9e7830f700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9e700cb530 0x7f9e700c6f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:21.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.729+0000 7f9e7830f700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9e7015dba0 0x7f9e700ad420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:21.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.729+0000 7f9e7830f700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e700af510 con 0x7f9e7015dba0 2023-11-08T19:31:21.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.729+0000 7f9e7830f700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e700c74c0 con 0x7f9e7006b240 2023-11-08T19:31:21.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.729+0000 7f9e7830f700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e700ad960 con 0x7f9e700cb530 2023-11-08T19:31:21.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.729+0000 7f9e768ac700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9e7015dba0 0x7f9e700ad420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:21.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.729+0000 7f9e768ac700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9e7015dba0 0x7f9e700ad420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57340/0 (socket says 172.21.15.143:57340) 2023-11-08T19:31:21.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.729+0000 7f9e768ac700 1 -- 172.21.15.143:0/1206124049 learned_addr learned my addr 172.21.15.143:0/1206124049 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:31:21.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.729+0000 7f9e758aa700 1 --2- 172.21.15.143:0/1206124049 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9e700cb530 0x7f9e700c6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:21.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.729+0000 7f9e760ab700 1 --2- 172.21.15.143:0/1206124049 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9e7006b240 0x7f9e700adde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:21.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.729+0000 7f9e768ac700 1 -- 172.21.15.143:0/1206124049 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9e700cb530 msgr2=0x7f9e700c6f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:21.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.729+0000 7f9e768ac700 1 --2- 172.21.15.143:0/1206124049 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9e700cb530 0x7f9e700c6f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:21.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.729+0000 7f9e768ac700 1 -- 172.21.15.143:0/1206124049 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9e7006b240 msgr2=0x7f9e700adde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:21.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.729+0000 7f9e768ac700 1 --2- 172.21.15.143:0/1206124049 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9e7006b240 0x7f9e700adde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:21.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.729+0000 7f9e768ac700 1 -- 172.21.15.143:0/1206124049 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e6c009550 con 0x7f9e7015dba0 2023-11-08T19:31:21.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.729+0000 7f9e768ac700 1 --2- 172.21.15.143:0/1206124049 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9e7015dba0 0x7f9e700ad420 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f9e6800efc0 tx=0x7f9e6800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:21.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.729+0000 7f9e66ffd700 1 -- 172.21.15.143:0/1206124049 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e68009ea0 con 0x7f9e7015dba0 2023-11-08T19:31:21.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.729+0000 7f9e66ffd700 1 -- 172.21.15.143:0/1206124049 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e6800f040 con 0x7f9e7015dba0 2023-11-08T19:31:21.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.729+0000 7f9e7830f700 1 -- 172.21.15.143:0/1206124049 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e700ae470 con 0x7f9e7015dba0 2023-11-08T19:31:21.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.729+0000 7f9e66ffd700 1 -- 172.21.15.143:0/1206124049 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e6800ced0 con 0x7f9e7015dba0 2023-11-08T19:31:21.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.729+0000 7f9e7830f700 1 -- 172.21.15.143:0/1206124049 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e700aea30 con 0x7f9e7015dba0 2023-11-08T19:31:21.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.730+0000 7f9e7830f700 1 -- 172.21.15.143:0/1206124049 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9e7006bfe0 con 0x7f9e7015dba0 2023-11-08T19:31:21.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.731+0000 7f9e66ffd700 1 -- 172.21.15.143:0/1206124049 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9e68014480 con 0x7f9e7015dba0 2023-11-08T19:31:21.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.731+0000 7f9e66ffd700 1 --2- 172.21.15.143:0/1206124049 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f9e5406af00 0x7f9e5406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:21.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.731+0000 7f9e66ffd700 1 -- 172.21.15.143:0/1206124049 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6651+0+0 (secure 0 0 0) 0x7f9e6808a150 con 0x7f9e7015dba0 2023-11-08T19:31:21.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.731+0000 7f9e66ffd700 1 --2- 172.21.15.143:0/1206124049 >> [v2:172.21.15.174:6824/2755130316,v1:172.21.15.174:6825/2755130316] conn(0x7f9e540738a0 0x7f9e54075cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:21.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.731+0000 7f9e66ffd700 1 -- 172.21.15.143:0/1206124049 --> [v2:172.21.15.174:6824/2755130316,v1:172.21.15.174:6825/2755130316] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9e54076390 con 0x7f9e540738a0 2023-11-08T19:31:21.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.731+0000 7f9e66ffd700 1 -- 172.21.15.143:0/1206124049 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_get_version_reply(handle=1 version=107) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9e6808c810 con 0x7f9e7015dba0 2023-11-08T19:31:21.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.731+0000 7f9e760ab700 1 --2- 172.21.15.143:0/1206124049 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f9e5406af00 0x7f9e5406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:21.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.731+0000 7f9e758aa700 1 --2- 172.21.15.143:0/1206124049 >> [v2:172.21.15.174:6824/2755130316,v1:172.21.15.174:6825/2755130316] conn(0x7f9e540738a0 0x7f9e54075cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:21.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.732+0000 7f9e760ab700 1 --2- 172.21.15.143:0/1206124049 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f9e5406af00 0x7f9e5406d3c0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f9e700caad0 tx=0x7f9e60009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:21.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.732+0000 7f9e758aa700 1 --2- 172.21.15.143:0/1206124049 >> [v2:172.21.15.174:6824/2755130316,v1:172.21.15.174:6825/2755130316] conn(0x7f9e540738a0 0x7f9e54075cc0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:21.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.733+0000 7f9e66ffd700 1 -- 172.21.15.143:0/1206124049 <== osd.6 v2:172.21.15.174:6824/2755130316 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26745 (crc 0 0 0) 0x7f9e54076390 con 0x7f9e540738a0 2023-11-08T19:31:21.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.747+0000 7f9e7830f700 1 -- 172.21.15.143:0/1206124049 --> [v2:172.21.15.174:6824/2755130316,v1:172.21.15.174:6825/2755130316] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f9e7006bd00 con 0x7f9e540738a0 2023-11-08T19:31:21.750 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.748+0000 7f9e66ffd700 1 -- 172.21.15.143:0/1206124049 <== osd.6 v2:172.21.15.174:6824/2755130316 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f9e6c011b00 con 0x7f9e540738a0 2023-11-08T19:31:21.750 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2023-11-08T19:31:21.751 INFO:teuthology.orchestra.run.smithi143.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-08T19:31:21.751 INFO:teuthology.orchestra.run.smithi143.stdout:} 2023-11-08T19:31:21.752 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.748+0000 7f9e7830f700 1 -- 172.21.15.143:0/1206124049 >> [v2:172.21.15.174:6824/2755130316,v1:172.21.15.174:6825/2755130316] conn(0x7f9e540738a0 msgr2=0x7f9e54075cc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:21.752 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.748+0000 7f9e7830f700 1 --2- 172.21.15.143:0/1206124049 >> [v2:172.21.15.174:6824/2755130316,v1:172.21.15.174:6825/2755130316] conn(0x7f9e540738a0 0x7f9e54075cc0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:21.753 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.748+0000 7f9e7830f700 1 -- 172.21.15.143:0/1206124049 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f9e5406af00 msgr2=0x7f9e5406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:21.753 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.748+0000 7f9e7830f700 1 --2- 172.21.15.143:0/1206124049 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f9e5406af00 0x7f9e5406d3c0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f9e700caad0 tx=0x7f9e60009040 comp rx=0 tx=0).stop 2023-11-08T19:31:21.753 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.748+0000 7f9e7830f700 1 -- 172.21.15.143:0/1206124049 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9e7015dba0 msgr2=0x7f9e700ad420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:21.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.748+0000 7f9e7830f700 1 --2- 172.21.15.143:0/1206124049 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9e7015dba0 0x7f9e700ad420 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f9e6800efc0 tx=0x7f9e6800c3a0 comp rx=0 tx=0).stop 2023-11-08T19:31:21.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.748+0000 7f9e768ac700 1 -- 172.21.15.143:0/1206124049 reap_dead start 2023-11-08T19:31:21.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.748+0000 7f9e7830f700 1 -- 172.21.15.143:0/1206124049 shutdown_connections 2023-11-08T19:31:21.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.748+0000 7f9e7830f700 1 -- 172.21.15.143:0/1206124049 >> 172.21.15.143:0/1206124049 conn(0x7f9e700cd2e0 msgr2=0x7f9e7015ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:21.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.749+0000 7f9e7830f700 1 -- 172.21.15.143:0/1206124049 shutdown_connections 2023-11-08T19:31:21.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.749+0000 7f9e7830f700 1 -- 172.21.15.143:0/1206124049 wait complete. 2023-11-08T19:31:21.761 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2023-11-08T19:31:21.882 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.881+0000 7fc75e041700 1 Processor -- start 2023-11-08T19:31:21.883 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.881+0000 7fc75e041700 1 -- start start 2023-11-08T19:31:21.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.882+0000 7fc75e041700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc75806b320 0x7fc758168c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:21.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.882+0000 7fc75e041700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc758161df0 con 0x7fc75815dd00 2023-11-08T19:31:21.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.882+0000 7fc75e041700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7581620d0 con 0x7fc7581618f0 2023-11-08T19:31:21.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.882+0000 7fc75e041700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7580cad50 con 0x7fc75806b320 2023-11-08T19:31:21.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.882+0000 7fc7577fe700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fc75815dd00 0x7fc7580cca70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:53500/0 (socket says 172.21.15.143:53500) 2023-11-08T19:31:21.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.882+0000 7fc7577fe700 1 -- 172.21.15.143:0/3873801291 learned_addr learned my addr 172.21.15.143:0/3873801291 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:31:21.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.882+0000 7fc757fff700 1 --2- 172.21.15.143:0/3873801291 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc75806b320 0x7fc758168c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:21.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.882+0000 7fc7567fc700 1 -- 172.21.15.143:0/3873801291 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3186482361 0 0) 0x7fc758161df0 con 0x7fc75815dd00 2023-11-08T19:31:21.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.882+0000 7fc7567fc700 1 -- 172.21.15.143:0/3873801291 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc7400035b0 con 0x7fc75815dd00 2023-11-08T19:31:21.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.882+0000 7fc7567fc700 1 -- 172.21.15.143:0/3873801291 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3890245553 0 0) 0x7fc7400035b0 con 0x7fc75815dd00 2023-11-08T19:31:21.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.882+0000 7fc7567fc700 1 -- 172.21.15.143:0/3873801291 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc75806b320 msgr2=0x7fc758168c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:21.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.882+0000 7fc7567fc700 1 --2- 172.21.15.143:0/3873801291 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc75806b320 0x7fc758168c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:21.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.882+0000 7fc7567fc700 1 -- 172.21.15.143:0/3873801291 >> v1:172.21.15.174:6789/0 conn(0x7fc7581618f0 legacy=0x7fc75806ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:21.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.882+0000 7fc7567fc700 1 -- 172.21.15.143:0/3873801291 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7581623b0 con 0x7fc75815dd00 2023-11-08T19:31:21.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.882+0000 7fc7567fc700 1 -- 172.21.15.143:0/3873801291 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4147756303 0 0) 0x7fc74c0017f0 con 0x7fc75815dd00 2023-11-08T19:31:21.895 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.882+0000 7fc7567fc700 1 -- 172.21.15.143:0/3873801291 >> v1:172.21.15.143:6789/0 conn(0x7fc75815dd00 legacy=0x7fc7580cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:21.895 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.882+0000 7fc7567fc700 1 --2- 172.21.15.143:0/3873801291 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc740003a30 0x7fc740003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:21.895 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.883+0000 7fc7567fc700 1 --2- 172.21.15.143:0/3873801291 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc75806b320 0x7fc740004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:21.896 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.883+0000 7fc7567fc700 1 --2- 172.21.15.143:0/3873801291 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc740004d90 0x7fc740005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:21.896 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.883+0000 7fc7567fc700 1 -- 172.21.15.143:0/3873801291 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc74000bac0 con 0x7fc740004d90 2023-11-08T19:31:21.897 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.883+0000 7fc7567fc700 1 -- 172.21.15.143:0/3873801291 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc740014440 con 0x7fc740003a30 2023-11-08T19:31:21.897 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.883+0000 7fc7567fc700 1 -- 172.21.15.143:0/3873801291 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc740014600 con 0x7fc75806b320 2023-11-08T19:31:21.897 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.883+0000 7fc757fff700 1 --2- 172.21.15.143:0/3873801291 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc740004d90 0x7fc740005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:21.898 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.883+0000 7fc756ffd700 1 --2- 172.21.15.143:0/3873801291 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc75806b320 0x7fc740004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:21.899 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.883+0000 7fc7577fe700 1 --2- 172.21.15.143:0/3873801291 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc740003a30 0x7fc740003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:21.899 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.883+0000 7fc757fff700 1 -- 172.21.15.143:0/3873801291 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc75806b320 msgr2=0x7fc740004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:21.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.883+0000 7fc757fff700 1 --2- 172.21.15.143:0/3873801291 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc75806b320 0x7fc740004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:21.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.883+0000 7fc757fff700 1 -- 172.21.15.143:0/3873801291 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc740003a30 msgr2=0x7fc740003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:21.901 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.883+0000 7fc757fff700 1 --2- 172.21.15.143:0/3873801291 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc740003a30 0x7fc740003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:21.901 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.883+0000 7fc757fff700 1 -- 172.21.15.143:0/3873801291 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7400035b0 con 0x7fc740004d90 2023-11-08T19:31:21.902 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.883+0000 7fc757fff700 1 --2- 172.21.15.143:0/3873801291 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc740004d90 0x7fc740005210 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7fc750009f90 tx=0x7fc740005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=227b46fc0cf61f83 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:21.902 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.883+0000 7fc7567fc700 1 -- 172.21.15.143:0/3873801291 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc75000ec30 con 0x7fc740004d90 2023-11-08T19:31:21.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.883+0000 7fc7567fc700 1 -- 172.21.15.143:0/3873801291 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc75000f040 con 0x7fc740004d90 2023-11-08T19:31:21.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.883+0000 7fc7567fc700 1 -- 172.21.15.143:0/3873801291 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc750014480 con 0x7fc740004d90 2023-11-08T19:31:21.904 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.884+0000 7fc75e041700 1 -- 172.21.15.143:0/3873801291 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc740004d90 msgr2=0x7fc740005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:21.904 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.884+0000 7fc75e041700 1 --2- 172.21.15.143:0/3873801291 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc740004d90 0x7fc740005210 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7fc750009f90 tx=0x7fc740005880 comp rx=0 tx=0).stop 2023-11-08T19:31:21.905 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.884+0000 7fc757fff700 1 -- 172.21.15.143:0/3873801291 reap_dead start 2023-11-08T19:31:21.905 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.884+0000 7fc75e041700 1 -- 172.21.15.143:0/3873801291 shutdown_connections 2023-11-08T19:31:21.906 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.884+0000 7fc75e041700 1 -- 172.21.15.143:0/3873801291 >> 172.21.15.143:0/3873801291 conn(0x7fc758099ab0 msgr2=0x7fc7580d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:21.906 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.884+0000 7fc75e041700 1 -- 172.21.15.143:0/3873801291 shutdown_connections 2023-11-08T19:31:21.906 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.884+0000 7fc75e041700 1 -- 172.21.15.143:0/3873801291 wait complete. 2023-11-08T19:31:21.906 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.885+0000 7fc75e041700 1 Processor -- start 2023-11-08T19:31:21.907 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.885+0000 7fc75e041700 1 -- start start 2023-11-08T19:31:21.907 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.886+0000 7fc75e041700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc758155cf0 0x7fc7580b5e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:21.907 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.886+0000 7fc75e041700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc7580b00f0 0x7fc7580ae9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:21.908 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.886+0000 7fc75e041700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc7580b0570 0x7fc7580aef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:21.908 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.886+0000 7fc75e041700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7580b3a10 con 0x7fc7580b0570 2023-11-08T19:31:21.908 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.886+0000 7fc75e041700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7580c5900 con 0x7fc758155cf0 2023-11-08T19:31:21.908 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.886+0000 7fc75e041700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7580c5360 con 0x7fc7580b00f0 2023-11-08T19:31:21.909 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.886+0000 7fc757fff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc7580b0570 0x7fc7580aef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:21.909 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.886+0000 7fc757fff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc7580b0570 0x7fc7580aef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57358/0 (socket says 172.21.15.143:57358) 2023-11-08T19:31:21.909 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.886+0000 7fc757fff700 1 -- 172.21.15.143:0/1055408573 learned_addr learned my addr 172.21.15.143:0/1055408573 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:31:21.910 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.886+0000 7fc7577fe700 1 --2- 172.21.15.143:0/1055408573 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc758155cf0 0x7fc7580b5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:21.910 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.886+0000 7fc756ffd700 1 --2- 172.21.15.143:0/1055408573 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc7580b00f0 0x7fc7580ae9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:21.910 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.886+0000 7fc757fff700 1 -- 172.21.15.143:0/1055408573 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc7580b00f0 msgr2=0x7fc7580ae9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:21.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.886+0000 7fc757fff700 1 --2- 172.21.15.143:0/1055408573 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc7580b00f0 0x7fc7580ae9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:21.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.886+0000 7fc757fff700 1 -- 172.21.15.143:0/1055408573 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc758155cf0 msgr2=0x7fc7580b5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:21.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.886+0000 7fc757fff700 1 --2- 172.21.15.143:0/1055408573 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc758155cf0 0x7fc7580b5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:21.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.886+0000 7fc757fff700 1 -- 172.21.15.143:0/1055408573 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc75815e260 con 0x7fc7580b0570 2023-11-08T19:31:21.912 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.886+0000 7fc7577fe700 1 --2- 172.21.15.143:0/1055408573 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc758155cf0 0x7fc7580b5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:31:21.912 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.886+0000 7fc756ffd700 1 --2- 172.21.15.143:0/1055408573 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc7580b00f0 0x7fc7580ae9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:31:21.912 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.887+0000 7fc757fff700 1 --2- 172.21.15.143:0/1055408573 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc7580b0570 0x7fc7580aef20 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7fc74000bc00 tx=0x7fc750009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:21.913 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.887+0000 7fc73ffff700 1 -- 172.21.15.143:0/1055408573 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc750007000 con 0x7fc7580b0570 2023-11-08T19:31:21.913 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.887+0000 7fc73ffff700 1 -- 172.21.15.143:0/1055408573 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc750009410 con 0x7fc7580b0570 2023-11-08T19:31:21.913 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.887+0000 7fc73ffff700 1 -- 172.21.15.143:0/1055408573 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc75000ee40 con 0x7fc7580b0570 2023-11-08T19:31:21.913 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.887+0000 7fc75e041700 1 -- 172.21.15.143:0/1055408573 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7580b63d0 con 0x7fc7580b0570 2023-11-08T19:31:21.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.887+0000 7fc75e041700 1 -- 172.21.15.143:0/1055408573 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7580c5620 con 0x7fc7580b0570 2023-11-08T19:31:21.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.888+0000 7fc75e041700 1 -- 172.21.15.143:0/1055408573 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc75804eee0 con 0x7fc7580b0570 2023-11-08T19:31:21.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.888+0000 7fc73ffff700 1 -- 172.21.15.143:0/1055408573 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc75001d420 con 0x7fc7580b0570 2023-11-08T19:31:21.915 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.888+0000 7fc73ffff700 1 --2- 172.21.15.143:0/1055408573 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc74406af00 0x7fc74406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:21.915 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.889+0000 7fc73ffff700 1 -- 172.21.15.143:0/1055408573 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6651+0+0 (secure 0 0 0) 0x7fc75008acc0 con 0x7fc7580b0570 2023-11-08T19:31:21.915 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.889+0000 7fc7577fe700 1 --2- 172.21.15.143:0/1055408573 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc74406af00 0x7fc74406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:21.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.889+0000 7fc7577fe700 1 --2- 172.21.15.143:0/1055408573 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc74406af00 0x7fc74406d3c0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fc74c000fd0 tx=0x7fc74c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:21.917 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:21.893+0000 7fc73ffff700 1 -- 172.21.15.143:0/1055408573 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc750055b70 con 0x7fc7580b0570 2023-11-08T19:31:22.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.082+0000 7fc75e041700 1 -- 172.21.15.143:0/1055408573 --> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7fc75815e0e0 con 0x7fc74406af00 2023-11-08T19:31:22.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.083+0000 7fc73ffff700 1 -- 172.21.15.143:0/1055408573 <== mgr.4099 v2:172.21.15.174:6832/4037280178 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fc74c007e40 con 0x7fc74406af00 2023-11-08T19:31:22.087 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.085+0000 7fc75e041700 1 -- 172.21.15.143:0/1055408573 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc74406af00 msgr2=0x7fc74406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:22.087 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.085+0000 7fc75e041700 1 --2- 172.21.15.143:0/1055408573 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc74406af00 0x7fc74406d3c0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fc74c000fd0 tx=0x7fc74c009040 comp rx=0 tx=0).stop 2023-11-08T19:31:22.087 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.085+0000 7fc75e041700 1 -- 172.21.15.143:0/1055408573 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc7580b0570 msgr2=0x7fc7580aef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:22.088 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.085+0000 7fc75e041700 1 --2- 172.21.15.143:0/1055408573 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc7580b0570 0x7fc7580aef20 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7fc74000bc00 tx=0x7fc750009750 comp rx=0 tx=0).stop 2023-11-08T19:31:22.088 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.085+0000 7fc75e041700 1 -- 172.21.15.143:0/1055408573 shutdown_connections 2023-11-08T19:31:22.089 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.085+0000 7fc75e041700 1 --2- 172.21.15.143:0/1055408573 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fc74406af00 0x7fc74406d3c0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:22.089 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.085+0000 7fc75e041700 1 --2- 172.21.15.143:0/1055408573 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc7580b0570 0x7fc7580aef20 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:22.090 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.085+0000 7fc75e041700 1 --2- 172.21.15.143:0/1055408573 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc758155cf0 0x7fc7580b5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:22.090 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.085+0000 7fc75e041700 1 --2- 172.21.15.143:0/1055408573 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc7580b00f0 0x7fc7580ae9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:22.090 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.085+0000 7fc75e041700 1 -- 172.21.15.143:0/1055408573 >> 172.21.15.143:0/1055408573 conn(0x7fc758099ab0 msgr2=0x7fc7580cdb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:22.091 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.085+0000 7fc75e041700 1 -- 172.21.15.143:0/1055408573 shutdown_connections 2023-11-08T19:31:22.091 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.085+0000 7fc75e041700 1 -- 172.21.15.143:0/1055408573 wait complete. 2023-11-08T19:31:22.091 INFO:teuthology.orchestra.run.smithi143.stderr:instructed osd(s) 6 to deep-scrub 2023-11-08T19:31:22.099 INFO:tasks.ceph:Scrubbing osd.7 2023-11-08T19:31:22.100 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2023-11-08T19:31:22.224 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.222+0000 7f7907f81700 1 Processor -- start 2023-11-08T19:31:22.224 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.223+0000 7f7907f81700 1 -- start start 2023-11-08T19:31:22.227 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.223+0000 7f7907f81700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f790015dba0 0x7f790006c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:22.227 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.223+0000 7f7907f81700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f790006b240 0x7f79000caff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:22.228 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.223+0000 7f7907f81700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f79000cb530 0x7f79000b22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:22.228 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.223+0000 7f7907f81700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7900161890 con 0x7f790015dba0 2023-11-08T19:31:22.228 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.223+0000 7f7907f81700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f790006b740 con 0x7f790006b240 2023-11-08T19:31:22.229 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.223+0000 7f7907f81700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f790006ba20 con 0x7f79000cb530 2023-11-08T19:31:22.229 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.224+0000 7f7905d1d700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f790015dba0 0x7f790006c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:22.229 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.224+0000 7f7905d1d700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f790015dba0 0x7f790006c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57364/0 (socket says 172.21.15.143:57364) 2023-11-08T19:31:22.229 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.224+0000 7f7905d1d700 1 -- 172.21.15.143:0/1672622934 learned_addr learned my addr 172.21.15.143:0/1672622934 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:31:22.230 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.224+0000 7f790651e700 1 --2- 172.21.15.143:0/1672622934 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f79000cb530 0x7f79000b22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:22.230 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.224+0000 7f790551c700 1 --2- 172.21.15.143:0/1672622934 >> v2:172.21.15.174:3300/0 conn(0x7f790006b240 0x7f79000caff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:22.230 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.224+0000 7f7905d1d700 1 -- 172.21.15.143:0/1672622934 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f79000cb530 msgr2=0x7f79000b22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:22.231 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.224+0000 7f7905d1d700 1 --2- 172.21.15.143:0/1672622934 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f79000cb530 0x7f79000b22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:22.231 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.224+0000 7f7905d1d700 1 -- 172.21.15.143:0/1672622934 >> v2:172.21.15.174:3300/0 conn(0x7f790006b240 msgr2=0x7f79000caff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:22.231 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.224+0000 7f7905d1d700 1 --2- 172.21.15.143:0/1672622934 >> v2:172.21.15.174:3300/0 conn(0x7f790006b240 0x7f79000caff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:22.232 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.224+0000 7f7905d1d700 1 -- 172.21.15.143:0/1672622934 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f790006bd00 con 0x7f790015dba0 2023-11-08T19:31:22.232 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.224+0000 7f790651e700 1 --2- 172.21.15.143:0/1672622934 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f79000cb530 0x7f79000b22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:31:22.232 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.224+0000 7f7905d1d700 1 --2- 172.21.15.143:0/1672622934 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f790015dba0 0x7f790006c610 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f79000d0810 tx=0x7f78f0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=203696b4bee69204 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:22.233 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.224+0000 7f7904d1b700 1 -- 172.21.15.143:0/1672622934 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78f000b870 con 0x7f790015dba0 2023-11-08T19:31:22.233 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.224+0000 7f7904d1b700 1 -- 172.21.15.143:0/1672622934 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78f000b9d0 con 0x7f790015dba0 2023-11-08T19:31:22.233 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.224+0000 7f7904d1b700 1 -- 172.21.15.143:0/1672622934 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78f000bd60 con 0x7f790015dba0 2023-11-08T19:31:22.234 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.225+0000 7f7907f81700 1 -- 172.21.15.143:0/1672622934 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f790015dba0 msgr2=0x7f790006c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:22.234 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.225+0000 7f7907f81700 1 --2- 172.21.15.143:0/1672622934 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f790015dba0 0x7f790006c610 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f79000d0810 tx=0x7f78f0009bd0 comp rx=0 tx=0).stop 2023-11-08T19:31:22.234 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.225+0000 7f7907f81700 1 -- 172.21.15.143:0/1672622934 shutdown_connections 2023-11-08T19:31:22.235 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.225+0000 7f7907f81700 1 --2- 172.21.15.143:0/1672622934 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f79000cb530 0x7f79000b22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:22.235 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.225+0000 7f7907f81700 1 --2- 172.21.15.143:0/1672622934 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f790015dba0 0x7f790006c610 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:22.235 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.225+0000 7f7907f81700 1 --2- 172.21.15.143:0/1672622934 >> v2:172.21.15.174:3300/0 conn(0x7f790006b240 0x7f79000caff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:22.236 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.225+0000 7f7907f81700 1 -- 172.21.15.143:0/1672622934 >> 172.21.15.143:0/1672622934 conn(0x7f79000cd2e0 msgr2=0x7f79000cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:22.236 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.225+0000 7f7907f81700 1 -- 172.21.15.143:0/1672622934 shutdown_connections 2023-11-08T19:31:22.236 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.225+0000 7f7907f81700 1 -- 172.21.15.143:0/1672622934 wait complete. 2023-11-08T19:31:22.237 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.226+0000 7f7907f81700 1 Processor -- start 2023-11-08T19:31:22.237 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.226+0000 7f7907f81700 1 -- start start 2023-11-08T19:31:22.240 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.227+0000 7f7907f81700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f790006b240 0x7f79001611f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:22.240 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.227+0000 7f7907f81700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f79000cb530 0x7f79000c8de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:22.241 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.227+0000 7f7907f81700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f790015dba0 0x7f790015e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:22.241 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.227+0000 7f7907f81700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7900161730 con 0x7f79000cb530 2023-11-08T19:31:22.241 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.227+0000 7f7907f81700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79000ca8b0 con 0x7f790015dba0 2023-11-08T19:31:22.242 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.227+0000 7f7907f81700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79000c9810 con 0x7f790006b240 2023-11-08T19:31:22.242 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.227+0000 7f790551c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f79000cb530 0x7f79000c8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:22.242 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.227+0000 7f790551c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f79000cb530 0x7f79000c8de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57372/0 (socket says 172.21.15.143:57372) 2023-11-08T19:31:22.243 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.227+0000 7f790551c700 1 -- 172.21.15.143:0/1251582743 learned_addr learned my addr 172.21.15.143:0/1251582743 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:31:22.243 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.227+0000 7f7905d1d700 1 --2- 172.21.15.143:0/1251582743 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f790006b240 0x7f79001611f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:22.243 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.227+0000 7f790651e700 1 --2- 172.21.15.143:0/1251582743 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f790015dba0 0x7f790015e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:22.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.227+0000 7f790551c700 1 -- 172.21.15.143:0/1251582743 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f790006b240 msgr2=0x7f79001611f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:22.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.227+0000 7f790551c700 1 --2- 172.21.15.143:0/1251582743 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f790006b240 0x7f79001611f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:22.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.227+0000 7f790551c700 1 -- 172.21.15.143:0/1251582743 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f790015dba0 msgr2=0x7f790015e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:22.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.227+0000 7f790551c700 1 --2- 172.21.15.143:0/1251582743 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f790015dba0 0x7f790015e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:22.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.227+0000 7f790551c700 1 -- 172.21.15.143:0/1251582743 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78f0009550 con 0x7f79000cb530 2023-11-08T19:31:22.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.227+0000 7f790551c700 1 --2- 172.21.15.143:0/1251582743 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f79000cb530 0x7f79000c8de0 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f78fc00c820 tx=0x7f78fc00cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:22.246 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.227+0000 7f78f67fc700 1 -- 172.21.15.143:0/1251582743 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78fc004220 con 0x7f79000cb530 2023-11-08T19:31:22.246 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.227+0000 7f78f67fc700 1 -- 172.21.15.143:0/1251582743 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78fc004380 con 0x7f79000cb530 2023-11-08T19:31:22.246 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.227+0000 7f7907f81700 1 -- 172.21.15.143:0/1251582743 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79000c9af0 con 0x7f79000cb530 2023-11-08T19:31:22.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.227+0000 7f78f67fc700 1 -- 172.21.15.143:0/1251582743 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78fc0036a0 con 0x7f79000cb530 2023-11-08T19:31:22.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.227+0000 7f7907f81700 1 -- 172.21.15.143:0/1251582743 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79000ca0b0 con 0x7f79000cb530 2023-11-08T19:31:22.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.228+0000 7f7907f81700 1 -- 172.21.15.143:0/1251582743 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f790006bfe0 con 0x7f79000cb530 2023-11-08T19:31:22.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.229+0000 7f78f67fc700 1 -- 172.21.15.143:0/1251582743 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f78fc003820 con 0x7f79000cb530 2023-11-08T19:31:22.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.229+0000 7f78f67fc700 1 --2- 172.21.15.143:0/1251582743 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f78e406af00 0x7f78e406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:22.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.229+0000 7f78f67fc700 1 -- 172.21.15.143:0/1251582743 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6651+0+0 (secure 0 0 0) 0x7f78fc089480 con 0x7f79000cb530 2023-11-08T19:31:22.249 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.229+0000 7f78f67fc700 1 --2- 172.21.15.143:0/1251582743 >> [v2:172.21.15.174:6816/2847141411,v1:172.21.15.174:6817/2847141411] conn(0x7f78e4073a10 0x7f78e4075e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:22.249 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.229+0000 7f78f67fc700 1 -- 172.21.15.143:0/1251582743 --> [v2:172.21.15.174:6816/2847141411,v1:172.21.15.174:6817/2847141411] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f78e4076500 con 0x7f78e4073a10 2023-11-08T19:31:22.249 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.229+0000 7f78f67fc700 1 -- 172.21.15.143:0/1251582743 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_get_version_reply(handle=1 version=107) v2 ==== 24+0+0 (secure 0 0 0) 0x7f78fc08ba60 con 0x7f79000cb530 2023-11-08T19:31:22.250 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.229+0000 7f7905d1d700 1 --2- 172.21.15.143:0/1251582743 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f78e406af00 0x7f78e406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:22.250 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.230+0000 7f790651e700 1 --2- 172.21.15.143:0/1251582743 >> [v2:172.21.15.174:6816/2847141411,v1:172.21.15.174:6817/2847141411] conn(0x7f78e4073a10 0x7f78e4075e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:22.250 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.230+0000 7f790651e700 1 --2- 172.21.15.143:0/1251582743 >> [v2:172.21.15.174:6816/2847141411,v1:172.21.15.174:6817/2847141411] conn(0x7f78e4073a10 0x7f78e4075e30 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:22.251 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.230+0000 7f7905d1d700 1 --2- 172.21.15.143:0/1251582743 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f78e406af00 0x7f78e406d3c0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f78f0014040 tx=0x7f78f0008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:22.251 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.231+0000 7f78f67fc700 1 -- 172.21.15.143:0/1251582743 <== osd.7 v2:172.21.15.174:6816/2847141411 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26745 (crc 0 0 0) 0x7f78e4076500 con 0x7f78e4073a10 2023-11-08T19:31:22.252 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.245+0000 7f7907f81700 1 -- 172.21.15.143:0/1251582743 --> [v2:172.21.15.174:6816/2847141411,v1:172.21.15.174:6817/2847141411] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f79000cfc50 con 0x7f78e4073a10 2023-11-08T19:31:22.252 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.246+0000 7f78f67fc700 1 -- 172.21.15.143:0/1251582743 <== osd.7 v2:172.21.15.174:6816/2847141411 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f79000cfc50 con 0x7f78e4073a10 2023-11-08T19:31:22.253 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2023-11-08T19:31:22.253 INFO:teuthology.orchestra.run.smithi143.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-08T19:31:22.254 INFO:teuthology.orchestra.run.smithi143.stdout:} 2023-11-08T19:31:22.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.246+0000 7f7907f81700 1 -- 172.21.15.143:0/1251582743 >> [v2:172.21.15.174:6816/2847141411,v1:172.21.15.174:6817/2847141411] conn(0x7f78e4073a10 msgr2=0x7f78e4075e30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:22.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.246+0000 7f7907f81700 1 --2- 172.21.15.143:0/1251582743 >> [v2:172.21.15.174:6816/2847141411,v1:172.21.15.174:6817/2847141411] conn(0x7f78e4073a10 0x7f78e4075e30 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:22.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.246+0000 7f7907f81700 1 -- 172.21.15.143:0/1251582743 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f78e406af00 msgr2=0x7f78e406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:22.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.246+0000 7f7907f81700 1 --2- 172.21.15.143:0/1251582743 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f78e406af00 0x7f78e406d3c0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f78f0014040 tx=0x7f78f0008030 comp rx=0 tx=0).stop 2023-11-08T19:31:22.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.246+0000 7f7907f81700 1 -- 172.21.15.143:0/1251582743 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f79000cb530 msgr2=0x7f79000c8de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:22.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.246+0000 7f7907f81700 1 --2- 172.21.15.143:0/1251582743 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f79000cb530 0x7f79000c8de0 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f78fc00c820 tx=0x7f78fc00cb30 comp rx=0 tx=0).stop 2023-11-08T19:31:22.256 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.246+0000 7f790651e700 1 -- 172.21.15.143:0/1251582743 reap_dead start 2023-11-08T19:31:22.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.246+0000 7f7907f81700 1 -- 172.21.15.143:0/1251582743 shutdown_connections 2023-11-08T19:31:22.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.246+0000 7f7907f81700 1 -- 172.21.15.143:0/1251582743 >> 172.21.15.143:0/1251582743 conn(0x7f79000cd2e0 msgr2=0x7f790015cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:22.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.246+0000 7f7907f81700 1 -- 172.21.15.143:0/1251582743 shutdown_connections 2023-11-08T19:31:22.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.247+0000 7f7907f81700 1 -- 172.21.15.143:0/1251582743 wait complete. 2023-11-08T19:31:22.259 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2023-11-08T19:31:22.380 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.379+0000 7fcff7488700 1 Processor -- start 2023-11-08T19:31:22.381 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.379+0000 7fcff7488700 1 -- start start 2023-11-08T19:31:22.384 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.380+0000 7fcff7488700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fcff015dd00 0x7fcff00cca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:22.384 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.380+0000 7fcff7488700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcff006b320 0x7fcff00cb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:22.384 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.380+0000 7fcff7488700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcff00ccfb0 con 0x7fcff01618f0 2023-11-08T19:31:22.385 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.380+0000 7fcff7488700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcff00cad50 con 0x7fcff015dd00 2023-11-08T19:31:22.385 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.380+0000 7fcff7488700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcff0161df0 con 0x7fcff006b320 2023-11-08T19:31:22.386 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.380+0000 7fcff5224700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fcff015dd00 0x7fcff00cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:22.387 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.380+0000 7fcff5224700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fcff015dd00 0x7fcff00cca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57380/0 (socket says 172.21.15.143:57380) 2023-11-08T19:31:22.387 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.380+0000 7fcff5224700 1 -- 172.21.15.143:0/116791420 learned_addr learned my addr 172.21.15.143:0/116791420 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:31:22.387 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.380+0000 7fcff4a23700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fcff01618f0 0x7fcff006ac10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:53534/0 (socket says 172.21.15.143:53534) 2023-11-08T19:31:22.388 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.380+0000 7fcff5a25700 1 --2- 172.21.15.143:0/116791420 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcff006b320 0x7fcff00cb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:22.388 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.380+0000 7fcff5224700 1 -- 172.21.15.143:0/116791420 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcff006b320 msgr2=0x7fcff00cb1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:22.388 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.380+0000 7fcff5224700 1 --2- 172.21.15.143:0/116791420 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcff006b320 0x7fcff00cb1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:22.389 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.380+0000 7fcff5224700 1 -- 172.21.15.143:0/116791420 >> v1:172.21.15.143:6789/0 conn(0x7fcff01618f0 legacy=0x7fcff006ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:22.389 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.380+0000 7fcff5224700 1 -- 172.21.15.143:0/116791420 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcff01620d0 con 0x7fcff015dd00 2023-11-08T19:31:22.390 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.380+0000 7fcff5224700 1 --2- 172.21.15.143:0/116791420 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcff015dd00 0x7fcff00cca70 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7fcfec006fd0 tx=0x7fcfec009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10294d674e8ca8a6 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:22.390 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.380+0000 7fcfe7fff700 1 -- 172.21.15.143:0/116791420 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfec005e30 con 0x7fcff015dd00 2023-11-08T19:31:22.391 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.381+0000 7fcfe7fff700 1 -- 172.21.15.143:0/116791420 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfec004030 con 0x7fcff015dd00 2023-11-08T19:31:22.391 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.381+0000 7fcfe7fff700 1 -- 172.21.15.143:0/116791420 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfec0043e0 con 0x7fcff015dd00 2023-11-08T19:31:22.391 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.381+0000 7fcff7488700 1 -- 172.21.15.143:0/116791420 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcff015dd00 msgr2=0x7fcff00cca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:22.393 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.381+0000 7fcff7488700 1 --2- 172.21.15.143:0/116791420 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcff015dd00 0x7fcff00cca70 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7fcfec006fd0 tx=0x7fcfec009bd0 comp rx=0 tx=0).stop 2023-11-08T19:31:22.394 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.381+0000 7fcff7488700 1 -- 172.21.15.143:0/116791420 shutdown_connections 2023-11-08T19:31:22.394 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.381+0000 7fcff7488700 1 --2- 172.21.15.143:0/116791420 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcff006b320 0x7fcff00cb1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:22.395 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.381+0000 7fcff7488700 1 --2- 172.21.15.143:0/116791420 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcff015dd00 0x7fcff00cca70 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:22.395 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.381+0000 7fcff7488700 1 -- 172.21.15.143:0/116791420 >> 172.21.15.143:0/116791420 conn(0x7fcff0099ab0 msgr2=0x7fcff00d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:22.396 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.381+0000 7fcff7488700 1 -- 172.21.15.143:0/116791420 shutdown_connections 2023-11-08T19:31:22.396 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.381+0000 7fcff7488700 1 -- 172.21.15.143:0/116791420 wait complete. 2023-11-08T19:31:22.397 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.382+0000 7fcff7488700 1 Processor -- start 2023-11-08T19:31:22.397 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.382+0000 7fcff7488700 1 -- start start 2023-11-08T19:31:22.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.383+0000 7fcff7488700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcff006b320 0x7fcff00b0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:22.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.383+0000 7fcff7488700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcff015dd00 0x7fcff00b2560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:22.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.383+0000 7fcff7488700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcff01618f0 0x7fcff00b4070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:22.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.383+0000 7fcff7488700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcff00b14e0 con 0x7fcff01618f0 2023-11-08T19:31:22.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.383+0000 7fcff7488700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcff00b2aa0 con 0x7fcff015dd00 2023-11-08T19:31:22.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.383+0000 7fcff7488700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcff00b45b0 con 0x7fcff006b320 2023-11-08T19:31:22.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.383+0000 7fcff5a25700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcff01618f0 0x7fcff00b4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:22.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.383+0000 7fcff5a25700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcff01618f0 0x7fcff00b4070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57388/0 (socket says 172.21.15.143:57388) 2023-11-08T19:31:22.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.383+0000 7fcff5a25700 1 -- 172.21.15.143:0/3552103592 learned_addr learned my addr 172.21.15.143:0/3552103592 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:31:22.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.383+0000 7fcff5224700 1 --2- 172.21.15.143:0/3552103592 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcff006b320 0x7fcff00b0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:22.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.383+0000 7fcff4a23700 1 --2- 172.21.15.143:0/3552103592 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcff015dd00 0x7fcff00b2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:22.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.383+0000 7fcff5a25700 1 -- 172.21.15.143:0/3552103592 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcff006b320 msgr2=0x7fcff00b0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:22.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.383+0000 7fcff5a25700 1 --2- 172.21.15.143:0/3552103592 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcff006b320 0x7fcff00b0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:22.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.383+0000 7fcff5a25700 1 -- 172.21.15.143:0/3552103592 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcff015dd00 msgr2=0x7fcff00b2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:22.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.383+0000 7fcff5a25700 1 --2- 172.21.15.143:0/3552103592 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcff015dd00 0x7fcff00b2560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:22.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.383+0000 7fcff5a25700 1 -- 172.21.15.143:0/3552103592 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfec009500 con 0x7fcff01618f0 2023-11-08T19:31:22.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.383+0000 7fcff4a23700 1 --2- 172.21.15.143:0/3552103592 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcff015dd00 0x7fcff00b2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:31:22.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.384+0000 7fcff5a25700 1 --2- 172.21.15.143:0/3552103592 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcff01618f0 0x7fcff00b4070 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7fcfe8007e30 tx=0x7fcfe800e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:22.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.384+0000 7fcfe5ffb700 1 -- 172.21.15.143:0/3552103592 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfe800eea0 con 0x7fcff01618f0 2023-11-08T19:31:22.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.384+0000 7fcfe5ffb700 1 -- 172.21.15.143:0/3552103592 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfe8014420 con 0x7fcff01618f0 2023-11-08T19:31:22.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.384+0000 7fcff7488700 1 -- 172.21.15.143:0/3552103592 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcff00b1200 con 0x7fcff01618f0 2023-11-08T19:31:22.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.384+0000 7fcfe5ffb700 1 -- 172.21.15.143:0/3552103592 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfe80147d0 con 0x7fcff01618f0 2023-11-08T19:31:22.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.384+0000 7fcff7488700 1 -- 172.21.15.143:0/3552103592 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcff00c46b0 con 0x7fcff01618f0 2023-11-08T19:31:22.409 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.385+0000 7fcff7488700 1 -- 172.21.15.143:0/3552103592 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcff00aff60 con 0x7fcff01618f0 2023-11-08T19:31:22.409 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.385+0000 7fcfe5ffb700 1 -- 172.21.15.143:0/3552103592 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fcfe8014930 con 0x7fcff01618f0 2023-11-08T19:31:22.409 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.386+0000 7fcfe5ffb700 1 --2- 172.21.15.143:0/3552103592 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fcfd406af00 0x7fcfd406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:22.409 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.386+0000 7fcfe5ffb700 1 -- 172.21.15.143:0/3552103592 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6651+0+0 (secure 0 0 0) 0x7fcfe808a1c0 con 0x7fcff01618f0 2023-11-08T19:31:22.410 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.386+0000 7fcff5224700 1 --2- 172.21.15.143:0/3552103592 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fcfd406af00 0x7fcfd406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:22.410 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.386+0000 7fcff5224700 1 --2- 172.21.15.143:0/3552103592 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fcfd406af00 0x7fcfd406d3c0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fcfec000c00 tx=0x7fcfec008070 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:22.410 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.390+0000 7fcfe5ffb700 1 -- 172.21.15.143:0/3552103592 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcfe8054e30 con 0x7fcff01618f0 2023-11-08T19:31:22.582 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.581+0000 7fcff7488700 1 -- 172.21.15.143:0/3552103592 --> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7fcff0162690 con 0x7fcfd406af00 2023-11-08T19:31:22.584 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.581+0000 7fcfe5ffb700 1 -- 172.21.15.143:0/3552103592 <== mgr.4099 v2:172.21.15.174:6832/4037280178 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fcfec006ca0 con 0x7fcfd406af00 2023-11-08T19:31:22.585 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.584+0000 7fcff7488700 1 -- 172.21.15.143:0/3552103592 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fcfd406af00 msgr2=0x7fcfd406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:22.586 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.584+0000 7fcff7488700 1 --2- 172.21.15.143:0/3552103592 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fcfd406af00 0x7fcfd406d3c0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fcfec000c00 tx=0x7fcfec008070 comp rx=0 tx=0).stop 2023-11-08T19:31:22.586 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.584+0000 7fcff7488700 1 -- 172.21.15.143:0/3552103592 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcff01618f0 msgr2=0x7fcff00b4070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:22.587 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.584+0000 7fcff7488700 1 --2- 172.21.15.143:0/3552103592 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcff01618f0 0x7fcff00b4070 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7fcfe8007e30 tx=0x7fcfe800e3c0 comp rx=0 tx=0).stop 2023-11-08T19:31:22.587 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.584+0000 7fcff7488700 1 -- 172.21.15.143:0/3552103592 shutdown_connections 2023-11-08T19:31:22.587 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.584+0000 7fcff7488700 1 --2- 172.21.15.143:0/3552103592 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7fcfd406af00 0x7fcfd406d3c0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:22.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.584+0000 7fcff7488700 1 --2- 172.21.15.143:0/3552103592 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcff006b320 0x7fcff00b0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:22.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.584+0000 7fcff7488700 1 --2- 172.21.15.143:0/3552103592 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcff01618f0 0x7fcff00b4070 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:22.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.584+0000 7fcff7488700 1 --2- 172.21.15.143:0/3552103592 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcff015dd00 0x7fcff00b2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:22.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.584+0000 7fcff7488700 1 -- 172.21.15.143:0/3552103592 >> 172.21.15.143:0/3552103592 conn(0x7fcff0099ab0 msgr2=0x7fcff00ad380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:22.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.584+0000 7fcff7488700 1 -- 172.21.15.143:0/3552103592 shutdown_connections 2023-11-08T19:31:22.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.584+0000 7fcff7488700 1 -- 172.21.15.143:0/3552103592 wait complete. 2023-11-08T19:31:22.590 INFO:teuthology.orchestra.run.smithi143.stderr:instructed osd(s) 7 to deep-scrub 2023-11-08T19:31:22.597 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-08T19:31:22.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.718+0000 7f3a1e097700 1 Processor -- start 2023-11-08T19:31:22.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.718+0000 7f3a1e097700 1 -- start start 2023-11-08T19:31:22.722 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.718+0000 7f3a1e097700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f3a180d1c70 0x7f3a180b14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:22.722 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.718+0000 7f3a1e097700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f3a180d03a0 0x7f3a180cc5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:22.723 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.718+0000 7f3a1e097700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a180b1bc0 con 0x7f3a1815ba30 2023-11-08T19:31:22.723 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.718+0000 7f3a1e097700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a180b1ea0 con 0x7f3a180d1c70 2023-11-08T19:31:22.723 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.718+0000 7f3a1e097700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a180b2180 con 0x7f3a180d03a0 2023-11-08T19:31:22.724 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.719+0000 7f3a16ffd700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f3a180d1c70 0x7f3a180b14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:22.724 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.719+0000 7f3a16ffd700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f3a180d1c70 0x7f3a180b14c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57394/0 (socket says 172.21.15.143:57394) 2023-11-08T19:31:22.724 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.719+0000 7f3a16ffd700 1 -- 172.21.15.143:0/3726428536 learned_addr learned my addr 172.21.15.143:0/3726428536 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:31:22.725 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.719+0000 7f3a17fff700 1 --2- 172.21.15.143:0/3726428536 >> v2:172.21.15.174:3300/0 conn(0x7f3a180d03a0 0x7f3a180cc5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:22.725 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.719+0000 7f3a16ffd700 1 -- 172.21.15.143:0/3726428536 >> v2:172.21.15.174:3300/0 conn(0x7f3a180d03a0 msgr2=0x7f3a180cc5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:22.725 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.719+0000 7f3a16ffd700 1 --2- 172.21.15.143:0/3726428536 >> v2:172.21.15.174:3300/0 conn(0x7f3a180d03a0 0x7f3a180cc5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:22.725 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.719+0000 7f3a16ffd700 1 -- 172.21.15.143:0/3726428536 >> v1:172.21.15.174:6789/0 conn(0x7f3a1815ba30 legacy=0x7f3a180d1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:22.726 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.719+0000 7f3a16ffd700 1 -- 172.21.15.143:0/3726428536 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a180b2460 con 0x7f3a180d1c70 2023-11-08T19:31:22.726 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.719+0000 7f3a16ffd700 1 --2- 172.21.15.143:0/3726428536 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3a180d1c70 0x7f3a180b14c0 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f3a180d2520 tx=0x7f3a0c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6853b16cfc35698 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:22.726 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.719+0000 7f3a167fc700 1 -- 172.21.15.143:0/3726428536 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a0c004230 con 0x7f3a180d1c70 2023-11-08T19:31:22.727 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.719+0000 7f3a167fc700 1 -- 172.21.15.143:0/3726428536 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a0c004390 con 0x7f3a180d1c70 2023-11-08T19:31:22.727 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.719+0000 7f3a167fc700 1 -- 172.21.15.143:0/3726428536 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a0c0036a0 con 0x7f3a180d1c70 2023-11-08T19:31:22.727 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.720+0000 7f3a1e097700 1 -- 172.21.15.143:0/3726428536 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3a180d1c70 msgr2=0x7f3a180b14c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:22.727 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.720+0000 7f3a1e097700 1 --2- 172.21.15.143:0/3726428536 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3a180d1c70 0x7f3a180b14c0 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f3a180d2520 tx=0x7f3a0c009bd0 comp rx=0 tx=0).stop 2023-11-08T19:31:22.728 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.720+0000 7f3a1e097700 1 -- 172.21.15.143:0/3726428536 shutdown_connections 2023-11-08T19:31:22.728 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.720+0000 7f3a1e097700 1 --2- 172.21.15.143:0/3726428536 >> v2:172.21.15.174:3300/0 conn(0x7f3a180d03a0 0x7f3a180cc5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:22.729 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.720+0000 7f3a1e097700 1 --2- 172.21.15.143:0/3726428536 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3a180d1c70 0x7f3a180b14c0 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:22.729 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.720+0000 7f3a1e097700 1 -- 172.21.15.143:0/3726428536 >> 172.21.15.143:0/3726428536 conn(0x7f3a1808b000 msgr2=0x7f3a1809ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:22.729 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.720+0000 7f3a1e097700 1 -- 172.21.15.143:0/3726428536 shutdown_connections 2023-11-08T19:31:22.730 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.720+0000 7f3a1e097700 1 -- 172.21.15.143:0/3726428536 wait complete. 2023-11-08T19:31:22.730 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.721+0000 7f3a1e097700 1 Processor -- start 2023-11-08T19:31:22.730 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.721+0000 7f3a1e097700 1 -- start start 2023-11-08T19:31:22.733 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.722+0000 7f3a1e097700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3a180d03a0 0x7f3a18157f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:22.733 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.722+0000 7f3a1e097700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a180d1c70 0x7f3a180b4f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:22.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.722+0000 7f3a1e097700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3a1815ba30 0x7f3a180b5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:22.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.722+0000 7f3a1e097700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a180b4770 con 0x7f3a180d03a0 2023-11-08T19:31:22.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.722+0000 7f3a1e097700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a180c9cb0 con 0x7f3a180d1c70 2023-11-08T19:31:22.735 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.722+0000 7f3a1e097700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a180c99d0 con 0x7f3a1815ba30 2023-11-08T19:31:22.735 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.722+0000 7f3a177fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3a180d03a0 0x7f3a18157f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:22.735 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.722+0000 7f3a177fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3a180d03a0 0x7f3a18157f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57398/0 (socket says 172.21.15.143:57398) 2023-11-08T19:31:22.735 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.722+0000 7f3a177fe700 1 -- 172.21.15.143:0/2822287464 learned_addr learned my addr 172.21.15.143:0/2822287464 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:31:22.736 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.722+0000 7f3a16ffd700 1 --2- 172.21.15.143:0/2822287464 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a180d1c70 0x7f3a180b4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:22.736 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.722+0000 7f3a17fff700 1 --2- 172.21.15.143:0/2822287464 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3a1815ba30 0x7f3a180b5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:22.736 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.722+0000 7f3a177fe700 1 -- 172.21.15.143:0/2822287464 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3a1815ba30 msgr2=0x7f3a180b5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:22.736 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.722+0000 7f3a177fe700 1 --2- 172.21.15.143:0/2822287464 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3a1815ba30 0x7f3a180b5460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:22.737 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.722+0000 7f3a177fe700 1 -- 172.21.15.143:0/2822287464 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a180d1c70 msgr2=0x7f3a180b4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:22.737 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.722+0000 7f3a177fe700 1 --2- 172.21.15.143:0/2822287464 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a180d1c70 0x7f3a180b4f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:22.737 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.722+0000 7f3a177fe700 1 -- 172.21.15.143:0/2822287464 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a0c009550 con 0x7f3a180d03a0 2023-11-08T19:31:22.738 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.722+0000 7f3a177fe700 1 --2- 172.21.15.143:0/2822287464 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3a180d03a0 0x7f3a18157f40 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f3a0800b470 tx=0x7f3a0800b780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:22.738 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.723+0000 7f39fffff700 1 -- 172.21.15.143:0/2822287464 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a08011830 con 0x7f3a180d03a0 2023-11-08T19:31:22.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.723+0000 7f39fffff700 1 -- 172.21.15.143:0/2822287464 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a080119b0 con 0x7f3a180d03a0 2023-11-08T19:31:22.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.723+0000 7f39fffff700 1 -- 172.21.15.143:0/2822287464 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a08011d60 con 0x7f3a180d03a0 2023-11-08T19:31:22.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.723+0000 7f3a1e097700 1 -- 172.21.15.143:0/2822287464 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a180ca990 con 0x7f3a180d03a0 2023-11-08T19:31:22.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.723+0000 7f3a1e097700 1 -- 172.21.15.143:0/2822287464 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a180caf50 con 0x7f3a180d03a0 2023-11-08T19:31:22.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.723+0000 7f3a1e097700 1 -- 172.21.15.143:0/2822287464 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a1804eee0 con 0x7f3a180d03a0 2023-11-08T19:31:22.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.724+0000 7f39fffff700 1 -- 172.21.15.143:0/2822287464 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3a0801a400 con 0x7f3a180d03a0 2023-11-08T19:31:22.740 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.724+0000 7f39fffff700 1 --2- 172.21.15.143:0/2822287464 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f3a0406af00 0x7f3a0406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:22.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.724+0000 7f39fffff700 1 -- 172.21.15.143:0/2822287464 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6651+0+0 (secure 0 0 0) 0x7f3a08089870 con 0x7f3a180d03a0 2023-11-08T19:31:22.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.725+0000 7f3a16ffd700 1 --2- 172.21.15.143:0/2822287464 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f3a0406af00 0x7f3a0406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:22.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.725+0000 7f3a16ffd700 1 --2- 172.21.15.143:0/2822287464 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f3a0406af00 0x7f3a0406d3c0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c009130 tx=0x7f3a0c008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:22.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.729+0000 7f39fffff700 1 -- 172.21.15.143:0/2822287464 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3a08054570 con 0x7f3a180d03a0 2023-11-08T19:31:22.917 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.915+0000 7f3a1e097700 1 -- 172.21.15.143:0/2822287464 --> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3a180b0d30 con 0x7f3a0406af00 2023-11-08T19:31:22.925 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.924+0000 7f39fffff700 1 -- 172.21.15.143:0/2822287464 <== mgr.4099 v2:172.21.15.174:6832/4037280178 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+293795 (secure 0 0 0) 0x7f3a180b0d30 con 0x7f3a0406af00 2023-11-08T19:31:22.929 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:31:22.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.927+0000 7f3a1e097700 1 -- 172.21.15.143:0/2822287464 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f3a0406af00 msgr2=0x7f3a0406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:22.933 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.927+0000 7f3a1e097700 1 --2- 172.21.15.143:0/2822287464 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f3a0406af00 0x7f3a0406d3c0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c009130 tx=0x7f3a0c008030 comp rx=0 tx=0).stop 2023-11-08T19:31:22.933 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.927+0000 7f3a1e097700 1 -- 172.21.15.143:0/2822287464 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3a180d03a0 msgr2=0x7f3a18157f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:22.933 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.927+0000 7f3a1e097700 1 --2- 172.21.15.143:0/2822287464 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3a180d03a0 0x7f3a18157f40 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f3a0800b470 tx=0x7f3a0800b780 comp rx=0 tx=0).stop 2023-11-08T19:31:22.933 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.927+0000 7f3a1e097700 1 -- 172.21.15.143:0/2822287464 shutdown_connections 2023-11-08T19:31:22.934 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.927+0000 7f3a1e097700 1 --2- 172.21.15.143:0/2822287464 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f3a0406af00 0x7f3a0406d3c0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:22.934 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.927+0000 7f3a1e097700 1 --2- 172.21.15.143:0/2822287464 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3a1815ba30 0x7f3a180b5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:22.935 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.927+0000 7f3a1e097700 1 --2- 172.21.15.143:0/2822287464 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3a180d03a0 0x7f3a18157f40 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:22.935 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.927+0000 7f3a1e097700 1 --2- 172.21.15.143:0/2822287464 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a180d1c70 0x7f3a180b4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:22.935 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.927+0000 7f3a1e097700 1 -- 172.21.15.143:0/2822287464 >> 172.21.15.143:0/2822287464 conn(0x7f3a1808b000 msgr2=0x7f3a180ceb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:22.936 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.927+0000 7f3a1e097700 1 -- 172.21.15.143:0/2822287464 shutdown_connections 2023-11-08T19:31:22.936 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:22.927+0000 7f3a1e097700 1 -- 172.21.15.143:0/2822287464 wait complete. 2023-11-08T19:31:22.936 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2023-11-08T19:31:22.940 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":299,"stamp":"2023-11-08T19:31:22.625555+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":244328,"kb_used_data":11496,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754730392,"statfs":{"total":773094113280,"available":772843921408,"internally_reserved":0,"allocated":11771904,"data_stored":6351912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.619505"},"pg_stats":[{"pgid":"12.d","version":"0'0","reported_seq":53,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.009381+0000","last_change":"2023-11-08T19:27:08.190826+0000","last_active":"2023-11-08T19:31:13.009381+0000","last_peered":"2023-11-08T19:31:13.009381+0000","last_clean":"2023-11-08T19:31:13.009381+0000","last_became_active":"2023-11-08T19:27:08.190440+0000","last_became_peered":"2023-11-08T19:27:08.190440+0000","last_unstale":"2023-11-08T19:31:13.009381+0000","last_undegraded":"2023-11-08T19:31:13.009381+0000","last_fullsized":"2023-11-08T19:31:13.009381+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:42:48.376462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":255,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.008215+0000","last_change":"2023-11-08T19:24:45.888122+0000","last_active":"2023-11-08T19:31:13.008215+0000","last_peered":"2023-11-08T19:31:13.008215+0000","last_clean":"2023-11-08T19:31:13.008215+0000","last_became_active":"2023-11-08T19:24:45.887048+0000","last_became_peered":"2023-11-08T19:24:45.887048+0000","last_unstale":"2023-11-08T19:31:13.008215+0000","last_undegraded":"2023-11-08T19:31:13.008215+0000","last_fullsized":"2023-11-08T19:31:13.008215+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:44.575133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:44.575133+0000","last_clean_scrub_stamp":"2023-11-08T19:24:44.575133+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:19:20.202062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":79,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.400314+0000","last_change":"2023-11-08T19:26:06.542044+0000","last_active":"2023-11-08T19:31:13.400314+0000","last_peered":"2023-11-08T19:31:13.400314+0000","last_clean":"2023-11-08T19:31:13.400314+0000","last_became_active":"2023-11-08T19:26:04.603056+0000","last_became_peered":"2023-11-08T19:26:04.603056+0000","last_unstale":"2023-11-08T19:31:13.400314+0000","last_undegraded":"2023-11-08T19:31:13.400314+0000","last_fullsized":"2023-11-08T19:31:13.400314+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:41:50.771457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000517414,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":58,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.400998+0000","last_change":"2023-11-08T19:27:09.496842+0000","last_active":"2023-11-08T19:31:13.400998+0000","last_peered":"2023-11-08T19:31:13.400998+0000","last_clean":"2023-11-08T19:31:13.400998+0000","last_became_active":"2023-11-08T19:27:07.185776+0000","last_became_peered":"2023-11-08T19:27:07.185776+0000","last_unstale":"2023-11-08T19:31:13.400998+0000","last_undegraded":"2023-11-08T19:31:13.400998+0000","last_fullsized":"2023-11-08T19:31:13.400998+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:21:13.923970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031747200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":51,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.398503+0000","last_change":"2023-11-08T19:27:08.188241+0000","last_active":"2023-11-08T19:31:13.398503+0000","last_peered":"2023-11-08T19:31:13.398503+0000","last_clean":"2023-11-08T19:31:13.398503+0000","last_became_active":"2023-11-08T19:27:08.187863+0000","last_became_peered":"2023-11-08T19:27:08.187863+0000","last_unstale":"2023-11-08T19:31:13.398503+0000","last_undegraded":"2023-11-08T19:31:13.398503+0000","last_fullsized":"2023-11-08T19:31:13.398503+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:45:22.084015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":77,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.114067+0000","last_change":"2023-11-08T19:26:06.265419+0000","last_active":"2023-11-08T19:31:13.114067+0000","last_peered":"2023-11-08T19:31:13.114067+0000","last_clean":"2023-11-08T19:31:13.114067+0000","last_became_active":"2023-11-08T19:26:04.605333+0000","last_became_peered":"2023-11-08T19:26:04.605333+0000","last_unstale":"2023-11-08T19:31:13.114067+0000","last_undegraded":"2023-11-08T19:31:13.114067+0000","last_fullsized":"2023-11-08T19:31:13.114067+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:01:56.991346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064606400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":60,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.010017+0000","last_change":"2023-11-08T19:27:09.182113+0000","last_active":"2023-11-08T19:31:13.010017+0000","last_peered":"2023-11-08T19:31:13.010017+0000","last_clean":"2023-11-08T19:31:13.010017+0000","last_became_active":"2023-11-08T19:27:07.189939+0000","last_became_peered":"2023-11-08T19:27:07.189939+0000","last_unstale":"2023-11-08T19:31:13.010017+0000","last_undegraded":"2023-11-08T19:31:13.010017+0000","last_fullsized":"2023-11-08T19:31:13.010017+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:41:08.822366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036507500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":47,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.144692+0000","last_change":"2023-11-08T19:27:08.187127+0000","last_active":"2023-11-08T19:31:13.144692+0000","last_peered":"2023-11-08T19:31:13.144692+0000","last_clean":"2023-11-08T19:31:13.144692+0000","last_became_active":"2023-11-08T19:27:08.182648+0000","last_became_peered":"2023-11-08T19:27:08.182648+0000","last_unstale":"2023-11-08T19:31:13.144692+0000","last_undegraded":"2023-11-08T19:31:13.144692+0000","last_fullsized":"2023-11-08T19:31:13.144692+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:50:12.101878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":89,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.009586+0000","last_change":"2023-11-08T19:26:05.609570+0000","last_active":"2023-11-08T19:31:13.009586+0000","last_peered":"2023-11-08T19:31:13.009586+0000","last_clean":"2023-11-08T19:31:13.009586+0000","last_became_active":"2023-11-08T19:26:04.608379+0000","last_became_peered":"2023-11-08T19:26:04.608379+0000","last_unstale":"2023-11-08T19:31:13.009586+0000","last_undegraded":"2023-11-08T19:31:13.009586+0000","last_fullsized":"2023-11-08T19:31:13.009586+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:40:08.986649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000529657,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":54,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.114895+0000","last_change":"2023-11-08T19:27:09.211991+0000","last_active":"2023-11-08T19:31:13.114895+0000","last_peered":"2023-11-08T19:31:13.114895+0000","last_clean":"2023-11-08T19:31:13.114895+0000","last_became_active":"2023-11-08T19:27:07.195231+0000","last_became_peered":"2023-11-08T19:27:07.195231+0000","last_unstale":"2023-11-08T19:31:13.114895+0000","last_undegraded":"2023-11-08T19:31:13.114895+0000","last_fullsized":"2023-11-08T19:31:13.114895+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:25:14.487524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000441221,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":47,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.145241+0000","last_change":"2023-11-08T19:27:08.195364+0000","last_active":"2023-11-08T19:31:13.145241+0000","last_peered":"2023-11-08T19:31:13.145241+0000","last_clean":"2023-11-08T19:31:13.145241+0000","last_became_active":"2023-11-08T19:27:08.195074+0000","last_became_peered":"2023-11-08T19:27:08.195074+0000","last_unstale":"2023-11-08T19:31:13.145241+0000","last_undegraded":"2023-11-08T19:31:13.145241+0000","last_fullsized":"2023-11-08T19:31:13.145241+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:02:57.552873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":77,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.398778+0000","last_change":"2023-11-08T19:26:06.394300+0000","last_active":"2023-11-08T19:31:13.398778+0000","last_peered":"2023-11-08T19:31:13.398778+0000","last_clean":"2023-11-08T19:31:13.398778+0000","last_became_active":"2023-11-08T19:26:04.601685+0000","last_became_peered":"2023-11-08T19:26:04.601685+0000","last_unstale":"2023-11-08T19:31:13.398778+0000","last_undegraded":"2023-11-08T19:31:13.398778+0000","last_fullsized":"2023-11-08T19:31:13.398778+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:51:06.626474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061044099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":54,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.145608+0000","last_change":"2023-11-08T19:27:09.178466+0000","last_active":"2023-11-08T19:31:13.145608+0000","last_peered":"2023-11-08T19:31:13.145608+0000","last_clean":"2023-11-08T19:31:13.145608+0000","last_became_active":"2023-11-08T19:27:07.189386+0000","last_became_peered":"2023-11-08T19:27:07.189386+0000","last_unstale":"2023-11-08T19:31:13.145608+0000","last_undegraded":"2023-11-08T19:31:13.145608+0000","last_fullsized":"2023-11-08T19:31:13.145608+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:48:51.139820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049390100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":51,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.400192+0000","last_change":"2023-11-08T19:27:08.189452+0000","last_active":"2023-11-08T19:31:13.400192+0000","last_peered":"2023-11-08T19:31:13.400192+0000","last_clean":"2023-11-08T19:31:13.400192+0000","last_became_active":"2023-11-08T19:27:08.189088+0000","last_became_peered":"2023-11-08T19:27:08.189088+0000","last_unstale":"2023-11-08T19:31:13.400192+0000","last_undegraded":"2023-11-08T19:31:13.400192+0000","last_fullsized":"2023-11-08T19:31:13.400192+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:25:18.963220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":78,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.010072+0000","last_change":"2023-11-08T19:26:06.296634+0000","last_active":"2023-11-08T19:31:13.010072+0000","last_peered":"2023-11-08T19:31:13.010072+0000","last_clean":"2023-11-08T19:31:13.010072+0000","last_became_active":"2023-11-08T19:26:04.603088+0000","last_became_peered":"2023-11-08T19:26:04.603088+0000","last_unstale":"2023-11-08T19:31:13.010072+0000","last_undegraded":"2023-11-08T19:31:13.010072+0000","last_fullsized":"2023-11-08T19:31:13.010072+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:01:56.991346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038180899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":54,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.011098+0000","last_change":"2023-11-08T19:27:09.431220+0000","last_active":"2023-11-08T19:31:13.011098+0000","last_peered":"2023-11-08T19:31:13.011098+0000","last_clean":"2023-11-08T19:31:13.011098+0000","last_became_active":"2023-11-08T19:27:07.191589+0000","last_became_peered":"2023-11-08T19:27:07.191589+0000","last_unstale":"2023-11-08T19:31:13.011098+0000","last_undegraded":"2023-11-08T19:31:13.011098+0000","last_fullsized":"2023-11-08T19:31:13.011098+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:18:19.964487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000729527,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":47,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.399851+0000","last_change":"2023-11-08T19:27:08.193337+0000","last_active":"2023-11-08T19:31:13.399851+0000","last_peered":"2023-11-08T19:31:13.399851+0000","last_clean":"2023-11-08T19:31:13.399851+0000","last_became_active":"2023-11-08T19:27:08.192997+0000","last_became_peered":"2023-11-08T19:27:08.192997+0000","last_unstale":"2023-11-08T19:31:13.399851+0000","last_undegraded":"2023-11-08T19:31:13.399851+0000","last_fullsized":"2023-11-08T19:31:13.399851+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:45:22.084015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":83,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.399877+0000","last_change":"2023-11-08T19:26:06.424511+0000","last_active":"2023-11-08T19:31:13.399877+0000","last_peered":"2023-11-08T19:31:13.399877+0000","last_clean":"2023-11-08T19:31:13.399877+0000","last_became_active":"2023-11-08T19:26:04.598973+0000","last_became_peered":"2023-11-08T19:26:04.598973+0000","last_unstale":"2023-11-08T19:31:13.399877+0000","last_undegraded":"2023-11-08T19:31:13.399877+0000","last_fullsized":"2023-11-08T19:31:13.399877+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:41:53.497456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031755900000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":58,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.400325+0000","last_change":"2023-11-08T19:27:09.497024+0000","last_active":"2023-11-08T19:31:13.400325+0000","last_peered":"2023-11-08T19:31:13.400325+0000","last_clean":"2023-11-08T19:31:13.400325+0000","last_became_active":"2023-11-08T19:27:07.190132+0000","last_became_peered":"2023-11-08T19:27:07.190132+0000","last_unstale":"2023-11-08T19:31:13.400325+0000","last_undegraded":"2023-11-08T19:31:13.400325+0000","last_fullsized":"2023-11-08T19:31:13.400325+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:36:34.881217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054126000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":47,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.400395+0000","last_change":"2023-11-08T19:27:08.194196+0000","last_active":"2023-11-08T19:31:13.400395+0000","last_peered":"2023-11-08T19:31:13.400395+0000","last_clean":"2023-11-08T19:31:13.400395+0000","last_became_active":"2023-11-08T19:27:08.193912+0000","last_became_peered":"2023-11-08T19:27:08.193912+0000","last_unstale":"2023-11-08T19:31:13.400395+0000","last_undegraded":"2023-11-08T19:31:13.400395+0000","last_fullsized":"2023-11-08T19:31:13.400395+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:25:18.963220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":78,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.011538+0000","last_change":"2023-11-08T19:26:06.296937+0000","last_active":"2023-11-08T19:31:13.011538+0000","last_peered":"2023-11-08T19:31:13.011538+0000","last_clean":"2023-11-08T19:31:13.011538+0000","last_became_active":"2023-11-08T19:26:04.599999+0000","last_became_peered":"2023-11-08T19:26:04.599999+0000","last_unstale":"2023-11-08T19:31:13.011538+0000","last_undegraded":"2023-11-08T19:31:13.011538+0000","last_fullsized":"2023-11-08T19:31:13.011538+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:50:43.827570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025605700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":58,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.399556+0000","last_change":"2023-11-08T19:27:09.497040+0000","last_active":"2023-11-08T19:31:13.399556+0000","last_peered":"2023-11-08T19:31:13.399556+0000","last_clean":"2023-11-08T19:31:13.399556+0000","last_became_active":"2023-11-08T19:27:07.189135+0000","last_became_peered":"2023-11-08T19:27:07.189135+0000","last_unstale":"2023-11-08T19:31:13.399556+0000","last_undegraded":"2023-11-08T19:31:13.399556+0000","last_fullsized":"2023-11-08T19:31:13.399556+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:12:29.400972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051794299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":53,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.011810+0000","last_change":"2023-11-08T19:27:08.187053+0000","last_active":"2023-11-08T19:31:13.011810+0000","last_peered":"2023-11-08T19:31:13.011810+0000","last_clean":"2023-11-08T19:31:13.011810+0000","last_became_active":"2023-11-08T19:27:08.186643+0000","last_became_peered":"2023-11-08T19:27:08.186643+0000","last_unstale":"2023-11-08T19:31:13.011810+0000","last_undegraded":"2023-11-08T19:31:13.011810+0000","last_fullsized":"2023-11-08T19:31:13.011810+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:35:30.073885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":89,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.010538+0000","last_change":"2023-11-08T19:26:05.609824+0000","last_active":"2023-11-08T19:31:13.010538+0000","last_peered":"2023-11-08T19:31:13.010538+0000","last_clean":"2023-11-08T19:31:13.010538+0000","last_became_active":"2023-11-08T19:26:04.603320+0000","last_became_peered":"2023-11-08T19:26:04.603320+0000","last_unstale":"2023-11-08T19:31:13.010538+0000","last_undegraded":"2023-11-08T19:31:13.010538+0000","last_fullsized":"2023-11-08T19:31:13.010538+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:05:45.976615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063225,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":58,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.398705+0000","last_change":"2023-11-08T19:27:09.496655+0000","last_active":"2023-11-08T19:31:13.398705+0000","last_peered":"2023-11-08T19:31:13.398705+0000","last_clean":"2023-11-08T19:31:13.398705+0000","last_became_active":"2023-11-08T19:27:07.189895+0000","last_became_peered":"2023-11-08T19:27:07.189895+0000","last_unstale":"2023-11-08T19:31:13.398705+0000","last_undegraded":"2023-11-08T19:31:13.398705+0000","last_fullsized":"2023-11-08T19:31:13.398705+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:41:08.822366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045654199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":79,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.399241+0000","last_change":"2023-11-08T19:26:06.541360+0000","last_active":"2023-11-08T19:31:13.399241+0000","last_peered":"2023-11-08T19:31:13.399241+0000","last_clean":"2023-11-08T19:31:13.399241+0000","last_became_active":"2023-11-08T19:26:04.603725+0000","last_became_peered":"2023-11-08T19:26:04.603725+0000","last_unstale":"2023-11-08T19:31:13.399241+0000","last_undegraded":"2023-11-08T19:31:13.399241+0000","last_fullsized":"2023-11-08T19:31:13.399241+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:28:48.576742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053365400000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":54,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.400452+0000","last_change":"2023-11-08T19:27:09.283599+0000","last_active":"2023-11-08T19:31:13.400452+0000","last_peered":"2023-11-08T19:31:13.400452+0000","last_clean":"2023-11-08T19:31:13.400452+0000","last_became_active":"2023-11-08T19:27:07.189595+0000","last_became_peered":"2023-11-08T19:27:07.189595+0000","last_unstale":"2023-11-08T19:31:13.400452+0000","last_undegraded":"2023-11-08T19:31:13.400452+0000","last_fullsized":"2023-11-08T19:31:13.400452+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:48:20.082682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000323479,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":48,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.114231+0000","last_change":"2023-11-08T19:27:08.185395+0000","last_active":"2023-11-08T19:31:13.114231+0000","last_peered":"2023-11-08T19:31:13.114231+0000","last_clean":"2023-11-08T19:31:13.114231+0000","last_became_active":"2023-11-08T19:27:08.184765+0000","last_became_peered":"2023-11-08T19:27:08.184765+0000","last_unstale":"2023-11-08T19:31:13.114231+0000","last_undegraded":"2023-11-08T19:31:13.114231+0000","last_fullsized":"2023-11-08T19:31:13.114231+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:08:51.864418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":83,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.400473+0000","last_change":"2023-11-08T19:26:06.424236+0000","last_active":"2023-11-08T19:31:13.400473+0000","last_peered":"2023-11-08T19:31:13.400473+0000","last_clean":"2023-11-08T19:31:13.400473+0000","last_became_active":"2023-11-08T19:26:04.600115+0000","last_became_peered":"2023-11-08T19:26:04.600115+0000","last_unstale":"2023-11-08T19:31:13.400473+0000","last_undegraded":"2023-11-08T19:31:13.400473+0000","last_fullsized":"2023-11-08T19:31:13.400473+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:18:01.876605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065803900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":54,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.400237+0000","last_change":"2023-11-08T19:27:09.331899+0000","last_active":"2023-11-08T19:31:13.400237+0000","last_peered":"2023-11-08T19:31:13.400237+0000","last_clean":"2023-11-08T19:31:13.400237+0000","last_became_active":"2023-11-08T19:27:07.191460+0000","last_became_peered":"2023-11-08T19:27:07.191460+0000","last_unstale":"2023-11-08T19:31:13.400237+0000","last_undegraded":"2023-11-08T19:31:13.400237+0000","last_fullsized":"2023-11-08T19:31:13.400237+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:51:29.247104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044907500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":54,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:14.014499+0000","last_change":"2023-11-08T19:27:08.191802+0000","last_active":"2023-11-08T19:31:14.014499+0000","last_peered":"2023-11-08T19:31:14.014499+0000","last_clean":"2023-11-08T19:31:14.014499+0000","last_became_active":"2023-11-08T19:27:08.191520+0000","last_became_peered":"2023-11-08T19:27:08.191520+0000","last_unstale":"2023-11-08T19:31:14.014499+0000","last_undegraded":"2023-11-08T19:31:14.014499+0000","last_fullsized":"2023-11-08T19:31:14.014499+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:49:16.458634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":89,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.008132+0000","last_change":"2023-11-08T19:26:05.608892+0000","last_active":"2023-11-08T19:31:13.008132+0000","last_peered":"2023-11-08T19:31:13.008132+0000","last_clean":"2023-11-08T19:31:13.008132+0000","last_became_active":"2023-11-08T19:26:04.604519+0000","last_became_peered":"2023-11-08T19:26:04.604519+0000","last_unstale":"2023-11-08T19:31:13.008132+0000","last_undegraded":"2023-11-08T19:31:13.008132+0000","last_fullsized":"2023-11-08T19:31:13.008132+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:11:26.508634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051248099999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":61,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:14.017094+0000","last_change":"2023-11-08T19:27:09.181431+0000","last_active":"2023-11-08T19:31:14.017094+0000","last_peered":"2023-11-08T19:31:14.017094+0000","last_clean":"2023-11-08T19:31:14.017094+0000","last_became_active":"2023-11-08T19:27:07.190800+0000","last_became_peered":"2023-11-08T19:27:07.190800+0000","last_unstale":"2023-11-08T19:31:14.017094+0000","last_undegraded":"2023-11-08T19:31:14.017094+0000","last_fullsized":"2023-11-08T19:31:14.017094+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:46:30.452678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656099999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":48,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:14.012585+0000","last_change":"2023-11-08T19:27:08.187983+0000","last_active":"2023-11-08T19:31:14.012585+0000","last_peered":"2023-11-08T19:31:14.012585+0000","last_clean":"2023-11-08T19:31:14.012585+0000","last_became_active":"2023-11-08T19:27:08.187322+0000","last_became_peered":"2023-11-08T19:27:08.187322+0000","last_unstale":"2023-11-08T19:31:14.012585+0000","last_undegraded":"2023-11-08T19:31:14.012585+0000","last_fullsized":"2023-11-08T19:31:14.012585+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:33:07.647519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":77,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.398936+0000","last_change":"2023-11-08T19:26:06.394075+0000","last_active":"2023-11-08T19:31:13.398936+0000","last_peered":"2023-11-08T19:31:13.398936+0000","last_clean":"2023-11-08T19:31:13.398936+0000","last_became_active":"2023-11-08T19:26:04.606967+0000","last_became_peered":"2023-11-08T19:26:04.606967+0000","last_unstale":"2023-11-08T19:31:13.398936+0000","last_undegraded":"2023-11-08T19:31:13.398936+0000","last_fullsized":"2023-11-08T19:31:13.398936+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:19:56.286835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040971999999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":61,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:14.017157+0000","last_change":"2023-11-08T19:27:09.180722+0000","last_active":"2023-11-08T19:31:14.017157+0000","last_peered":"2023-11-08T19:31:14.017157+0000","last_clean":"2023-11-08T19:31:14.017157+0000","last_became_active":"2023-11-08T19:27:07.186977+0000","last_became_peered":"2023-11-08T19:27:07.186977+0000","last_unstale":"2023-11-08T19:31:14.017157+0000","last_undegraded":"2023-11-08T19:31:14.017157+0000","last_fullsized":"2023-11-08T19:31:14.017157+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:17:59.966463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057675100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":48,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:14.219501+0000","last_change":"2023-11-08T19:27:08.193963+0000","last_active":"2023-11-08T19:31:14.219501+0000","last_peered":"2023-11-08T19:31:14.219501+0000","last_clean":"2023-11-08T19:31:14.219501+0000","last_became_active":"2023-11-08T19:27:08.193098+0000","last_became_peered":"2023-11-08T19:27:08.193098+0000","last_unstale":"2023-11-08T19:31:14.219501+0000","last_undegraded":"2023-11-08T19:31:14.219501+0000","last_fullsized":"2023-11-08T19:31:14.219501+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:41:09.827370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":77,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.115178+0000","last_change":"2023-11-08T19:26:06.265686+0000","last_active":"2023-11-08T19:31:13.115178+0000","last_peered":"2023-11-08T19:31:13.115178+0000","last_clean":"2023-11-08T19:31:13.115178+0000","last_became_active":"2023-11-08T19:26:04.602111+0000","last_became_peered":"2023-11-08T19:26:04.602111+0000","last_unstale":"2023-11-08T19:31:13.115178+0000","last_undegraded":"2023-11-08T19:31:13.115178+0000","last_fullsized":"2023-11-08T19:31:13.115178+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:40:11.164951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056197899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":61,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:14.013585+0000","last_change":"2023-11-08T19:27:09.181333+0000","last_active":"2023-11-08T19:31:14.013585+0000","last_peered":"2023-11-08T19:31:14.013585+0000","last_clean":"2023-11-08T19:31:14.013585+0000","last_became_active":"2023-11-08T19:27:07.188657+0000","last_became_peered":"2023-11-08T19:27:07.188657+0000","last_unstale":"2023-11-08T19:31:14.013585+0000","last_undegraded":"2023-11-08T19:31:14.013585+0000","last_fullsized":"2023-11-08T19:31:14.013585+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:25:56.788615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001227393,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":54,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:14.015025+0000","last_change":"2023-11-08T19:27:08.188533+0000","last_active":"2023-11-08T19:31:14.015025+0000","last_peered":"2023-11-08T19:31:14.015025+0000","last_clean":"2023-11-08T19:31:14.015025+0000","last_became_active":"2023-11-08T19:27:08.188240+0000","last_became_peered":"2023-11-08T19:27:08.188240+0000","last_unstale":"2023-11-08T19:31:14.015025+0000","last_undegraded":"2023-11-08T19:31:14.015025+0000","last_fullsized":"2023-11-08T19:31:14.015025+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:55:37.653521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":54,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:14.016242+0000","last_change":"2023-11-08T19:27:08.188458+0000","last_active":"2023-11-08T19:31:14.016242+0000","last_peered":"2023-11-08T19:31:14.016242+0000","last_clean":"2023-11-08T19:31:14.016242+0000","last_became_active":"2023-11-08T19:27:08.187880+0000","last_became_peered":"2023-11-08T19:27:08.187880+0000","last_unstale":"2023-11-08T19:31:14.016242+0000","last_undegraded":"2023-11-08T19:31:14.016242+0000","last_fullsized":"2023-11-08T19:31:14.016242+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:25:28.618214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":78,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.145899+0000","last_change":"2023-11-08T19:26:05.606898+0000","last_active":"2023-11-08T19:31:13.145899+0000","last_peered":"2023-11-08T19:31:13.145899+0000","last_clean":"2023-11-08T19:31:13.145899+0000","last_became_active":"2023-11-08T19:26:04.603955+0000","last_became_peered":"2023-11-08T19:26:04.603955+0000","last_unstale":"2023-11-08T19:31:13.145899+0000","last_undegraded":"2023-11-08T19:31:13.145899+0000","last_fullsized":"2023-11-08T19:31:13.145899+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:19:11.589559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036473499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":55,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:14.215612+0000","last_change":"2023-11-08T19:27:09.178770+0000","last_active":"2023-11-08T19:31:14.215612+0000","last_peered":"2023-11-08T19:31:14.215612+0000","last_clean":"2023-11-08T19:31:14.215612+0000","last_became_active":"2023-11-08T19:27:07.194992+0000","last_became_peered":"2023-11-08T19:27:07.194992+0000","last_unstale":"2023-11-08T19:31:14.215612+0000","last_undegraded":"2023-11-08T19:31:14.215612+0000","last_fullsized":"2023-11-08T19:31:14.215612+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:35:34.026910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000770285,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":47,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.011204+0000","last_change":"2023-11-08T19:27:08.193953+0000","last_active":"2023-11-08T19:31:13.011204+0000","last_peered":"2023-11-08T19:31:13.011204+0000","last_clean":"2023-11-08T19:31:13.011204+0000","last_became_active":"2023-11-08T19:27:08.193653+0000","last_became_peered":"2023-11-08T19:27:08.193653+0000","last_unstale":"2023-11-08T19:31:13.011204+0000","last_undegraded":"2023-11-08T19:31:13.011204+0000","last_fullsized":"2023-11-08T19:31:13.011204+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:35:30.073885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":83,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.400621+0000","last_change":"2023-11-08T19:26:06.424464+0000","last_active":"2023-11-08T19:31:13.400621+0000","last_peered":"2023-11-08T19:31:13.400621+0000","last_clean":"2023-11-08T19:31:13.400621+0000","last_became_active":"2023-11-08T19:26:04.603144+0000","last_became_peered":"2023-11-08T19:26:04.603144+0000","last_unstale":"2023-11-08T19:31:13.400621+0000","last_undegraded":"2023-11-08T19:31:13.400621+0000","last_fullsized":"2023-11-08T19:31:13.400621+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:42:55.269365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054090799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":61,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:14.014321+0000","last_change":"2023-11-08T19:27:09.180479+0000","last_active":"2023-11-08T19:31:14.014321+0000","last_peered":"2023-11-08T19:31:14.014321+0000","last_clean":"2023-11-08T19:31:14.014321+0000","last_became_active":"2023-11-08T19:27:07.190438+0000","last_became_peered":"2023-11-08T19:27:07.190438+0000","last_unstale":"2023-11-08T19:31:14.014321+0000","last_undegraded":"2023-11-08T19:31:14.014321+0000","last_fullsized":"2023-11-08T19:31:14.014321+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:25:14.487524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050893300000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":47,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.010828+0000","last_change":"2023-11-08T19:27:08.190530+0000","last_active":"2023-11-08T19:31:13.010828+0000","last_peered":"2023-11-08T19:31:13.010828+0000","last_clean":"2023-11-08T19:31:13.010828+0000","last_became_active":"2023-11-08T19:27:08.190203+0000","last_became_peered":"2023-11-08T19:27:08.190203+0000","last_unstale":"2023-11-08T19:31:13.010828+0000","last_undegraded":"2023-11-08T19:31:13.010828+0000","last_fullsized":"2023-11-08T19:31:13.010828+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:26:32.777602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":79,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.399746+0000","last_change":"2023-11-08T19:26:06.541623+0000","last_active":"2023-11-08T19:31:13.399746+0000","last_peered":"2023-11-08T19:31:13.399746+0000","last_clean":"2023-11-08T19:31:13.399746+0000","last_became_active":"2023-11-08T19:26:04.601961+0000","last_became_peered":"2023-11-08T19:26:04.601961+0000","last_unstale":"2023-11-08T19:31:13.399746+0000","last_undegraded":"2023-11-08T19:31:13.399746+0000","last_fullsized":"2023-11-08T19:31:13.399746+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:11:26.508634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000466386,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":55,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.009949+0000","last_change":"2023-11-08T19:27:09.431231+0000","last_active":"2023-11-08T19:31:13.009949+0000","last_peered":"2023-11-08T19:31:13.009949+0000","last_clean":"2023-11-08T19:31:13.009949+0000","last_became_active":"2023-11-08T19:27:07.191583+0000","last_became_peered":"2023-11-08T19:27:07.191583+0000","last_unstale":"2023-11-08T19:31:13.009949+0000","last_undegraded":"2023-11-08T19:31:13.009949+0000","last_fullsized":"2023-11-08T19:31:13.009949+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:58:48.357266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093695200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":47,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.010061+0000","last_change":"2023-11-08T19:27:08.187900+0000","last_active":"2023-11-08T19:31:13.010061+0000","last_peered":"2023-11-08T19:31:13.010061+0000","last_clean":"2023-11-08T19:31:13.010061+0000","last_became_active":"2023-11-08T19:27:08.187626+0000","last_became_peered":"2023-11-08T19:27:08.187626+0000","last_unstale":"2023-11-08T19:31:13.010061+0000","last_undegraded":"2023-11-08T19:31:13.010061+0000","last_fullsized":"2023-11-08T19:31:13.010061+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:00:55.878397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":89,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.009597+0000","last_change":"2023-11-08T19:26:05.608980+0000","last_active":"2023-11-08T19:31:13.009597+0000","last_peered":"2023-11-08T19:31:13.009597+0000","last_clean":"2023-11-08T19:31:13.009597+0000","last_became_active":"2023-11-08T19:26:04.606386+0000","last_became_peered":"2023-11-08T19:26:04.606386+0000","last_unstale":"2023-11-08T19:31:13.009597+0000","last_undegraded":"2023-11-08T19:31:13.009597+0000","last_fullsized":"2023-11-08T19:31:13.009597+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:41:50.771457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037950599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":55,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:14.220577+0000","last_change":"2023-11-08T19:27:09.179037+0000","last_active":"2023-11-08T19:31:14.220577+0000","last_peered":"2023-11-08T19:31:14.220577+0000","last_clean":"2023-11-08T19:31:14.220577+0000","last_became_active":"2023-11-08T19:27:07.195033+0000","last_became_peered":"2023-11-08T19:27:07.195033+0000","last_unstale":"2023-11-08T19:31:14.220577+0000","last_undegraded":"2023-11-08T19:31:14.220577+0000","last_fullsized":"2023-11-08T19:31:14.220577+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:19:08.253483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041824800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":47,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.399945+0000","last_change":"2023-11-08T19:27:08.193761+0000","last_active":"2023-11-08T19:31:13.399945+0000","last_peered":"2023-11-08T19:31:13.399945+0000","last_clean":"2023-11-08T19:31:13.399945+0000","last_became_active":"2023-11-08T19:27:08.193017+0000","last_became_peered":"2023-11-08T19:27:08.193017+0000","last_unstale":"2023-11-08T19:31:13.399945+0000","last_undegraded":"2023-11-08T19:31:13.399945+0000","last_fullsized":"2023-11-08T19:31:13.399945+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:18:00.971466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":83,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.400557+0000","last_change":"2023-11-08T19:26:06.424073+0000","last_active":"2023-11-08T19:31:13.400557+0000","last_peered":"2023-11-08T19:31:13.400557+0000","last_clean":"2023-11-08T19:31:13.400557+0000","last_became_active":"2023-11-08T19:26:04.599046+0000","last_became_peered":"2023-11-08T19:26:04.599046+0000","last_unstale":"2023-11-08T19:31:13.400557+0000","last_undegraded":"2023-11-08T19:31:13.400557+0000","last_fullsized":"2023-11-08T19:31:13.400557+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:04:43.562566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035092199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":61,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.009142+0000","last_change":"2023-11-08T19:27:09.181716+0000","last_active":"2023-11-08T19:31:13.009142+0000","last_peered":"2023-11-08T19:31:13.009142+0000","last_clean":"2023-11-08T19:31:13.009142+0000","last_became_active":"2023-11-08T19:27:07.187077+0000","last_became_peered":"2023-11-08T19:27:07.187077+0000","last_unstale":"2023-11-08T19:31:13.009142+0000","last_undegraded":"2023-11-08T19:31:13.009142+0000","last_fullsized":"2023-11-08T19:31:13.009142+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:51:29.247104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056673599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":47,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.114391+0000","last_change":"2023-11-08T19:27:08.188143+0000","last_active":"2023-11-08T19:31:13.114391+0000","last_peered":"2023-11-08T19:31:13.114391+0000","last_clean":"2023-11-08T19:31:13.114391+0000","last_became_active":"2023-11-08T19:27:08.187714+0000","last_became_peered":"2023-11-08T19:27:08.187714+0000","last_unstale":"2023-11-08T19:31:13.114391+0000","last_undegraded":"2023-11-08T19:31:13.114391+0000","last_fullsized":"2023-11-08T19:31:13.114391+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:02:10.863403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":77,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.398755+0000","last_change":"2023-11-08T19:26:06.394320+0000","last_active":"2023-11-08T19:31:13.398755+0000","last_peered":"2023-11-08T19:31:13.398755+0000","last_clean":"2023-11-08T19:31:13.398755+0000","last_became_active":"2023-11-08T19:26:04.603212+0000","last_became_peered":"2023-11-08T19:26:04.603212+0000","last_unstale":"2023-11-08T19:31:13.398755+0000","last_undegraded":"2023-11-08T19:31:13.398755+0000","last_fullsized":"2023-11-08T19:31:13.398755+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:25:29.947079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054695399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":60,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.009834+0000","last_change":"2023-11-08T19:27:09.181613+0000","last_active":"2023-11-08T19:31:13.009834+0000","last_peered":"2023-11-08T19:31:13.009834+0000","last_clean":"2023-11-08T19:31:13.009834+0000","last_became_active":"2023-11-08T19:27:07.190897+0000","last_became_peered":"2023-11-08T19:27:07.190897+0000","last_unstale":"2023-11-08T19:31:13.009834+0000","last_undegraded":"2023-11-08T19:31:13.009834+0000","last_fullsized":"2023-11-08T19:31:13.009834+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:27:57.388125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011172459999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":47,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.114574+0000","last_change":"2023-11-08T19:27:08.188668+0000","last_active":"2023-11-08T19:31:13.114574+0000","last_peered":"2023-11-08T19:31:13.114574+0000","last_clean":"2023-11-08T19:31:13.114574+0000","last_became_active":"2023-11-08T19:27:08.188441+0000","last_became_peered":"2023-11-08T19:27:08.188441+0000","last_unstale":"2023-11-08T19:31:13.114574+0000","last_undegraded":"2023-11-08T19:31:13.114574+0000","last_fullsized":"2023-11-08T19:31:13.114574+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:45:44.379160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":83,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.399480+0000","last_change":"2023-11-08T19:26:06.423754+0000","last_active":"2023-11-08T19:31:13.399480+0000","last_peered":"2023-11-08T19:31:13.399480+0000","last_clean":"2023-11-08T19:31:13.399480+0000","last_became_active":"2023-11-08T19:26:04.599322+0000","last_became_peered":"2023-11-08T19:26:04.599322+0000","last_unstale":"2023-11-08T19:31:13.399480+0000","last_undegraded":"2023-11-08T19:31:13.399480+0000","last_fullsized":"2023-11-08T19:31:13.399480+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:13:08.689128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042670000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":60,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.010577+0000","last_change":"2023-11-08T19:27:09.181656+0000","last_active":"2023-11-08T19:31:13.010577+0000","last_peered":"2023-11-08T19:31:13.010577+0000","last_clean":"2023-11-08T19:31:13.010577+0000","last_became_active":"2023-11-08T19:27:07.187337+0000","last_became_peered":"2023-11-08T19:27:07.187337+0000","last_unstale":"2023-11-08T19:31:13.010577+0000","last_undegraded":"2023-11-08T19:31:13.010577+0000","last_fullsized":"2023-11-08T19:31:13.010577+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:14:17.605057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000346967,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":47,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.399669+0000","last_change":"2023-11-08T19:27:08.194065+0000","last_active":"2023-11-08T19:31:13.399669+0000","last_peered":"2023-11-08T19:31:13.399669+0000","last_clean":"2023-11-08T19:31:13.399669+0000","last_became_active":"2023-11-08T19:27:08.193532+0000","last_became_peered":"2023-11-08T19:27:08.193532+0000","last_unstale":"2023-11-08T19:31:13.399669+0000","last_undegraded":"2023-11-08T19:31:13.399669+0000","last_fullsized":"2023-11-08T19:31:13.399669+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:27:58.393129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":77,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.400310+0000","last_change":"2023-11-08T19:26:06.394488+0000","last_active":"2023-11-08T19:31:13.400310+0000","last_peered":"2023-11-08T19:31:13.400310+0000","last_clean":"2023-11-08T19:31:13.400310+0000","last_became_active":"2023-11-08T19:26:04.606805+0000","last_became_peered":"2023-11-08T19:26:04.606805+0000","last_unstale":"2023-11-08T19:31:13.400310+0000","last_undegraded":"2023-11-08T19:31:13.400310+0000","last_fullsized":"2023-11-08T19:31:13.400310+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:25:20.857133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045317000000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":54,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.114849+0000","last_change":"2023-11-08T19:27:09.211647+0000","last_active":"2023-11-08T19:31:13.114849+0000","last_peered":"2023-11-08T19:31:13.114849+0000","last_clean":"2023-11-08T19:31:13.114849+0000","last_became_active":"2023-11-08T19:27:07.190309+0000","last_became_peered":"2023-11-08T19:27:07.190309+0000","last_unstale":"2023-11-08T19:31:13.114849+0000","last_undegraded":"2023-11-08T19:31:13.114849+0000","last_fullsized":"2023-11-08T19:31:13.114849+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:56:04.292154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047615199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":47,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.400512+0000","last_change":"2023-11-08T19:27:08.194884+0000","last_active":"2023-11-08T19:31:13.400512+0000","last_peered":"2023-11-08T19:31:13.400512+0000","last_clean":"2023-11-08T19:31:13.400512+0000","last_became_active":"2023-11-08T19:27:08.193241+0000","last_became_peered":"2023-11-08T19:27:08.193241+0000","last_unstale":"2023-11-08T19:31:13.400512+0000","last_undegraded":"2023-11-08T19:31:13.400512+0000","last_fullsized":"2023-11-08T19:31:13.400512+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:46:31.457682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":77,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.115596+0000","last_change":"2023-11-08T19:26:06.265907+0000","last_active":"2023-11-08T19:31:13.115596+0000","last_peered":"2023-11-08T19:31:13.115596+0000","last_clean":"2023-11-08T19:31:13.115596+0000","last_became_active":"2023-11-08T19:26:04.604772+0000","last_became_peered":"2023-11-08T19:26:04.604772+0000","last_unstale":"2023-11-08T19:31:13.115596+0000","last_undegraded":"2023-11-08T19:31:13.115596+0000","last_fullsized":"2023-11-08T19:31:13.115596+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:44:16.139794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066875300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":54,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.399622+0000","last_change":"2023-11-08T19:27:09.282699+0000","last_active":"2023-11-08T19:31:13.399622+0000","last_peered":"2023-11-08T19:31:13.399622+0000","last_clean":"2023-11-08T19:31:13.399622+0000","last_became_active":"2023-11-08T19:27:07.191212+0000","last_became_peered":"2023-11-08T19:27:07.191212+0000","last_unstale":"2023-11-08T19:31:13.399622+0000","last_undegraded":"2023-11-08T19:31:13.399622+0000","last_fullsized":"2023-11-08T19:31:13.399622+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:35:14.715110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046640800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":48,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.114710+0000","last_change":"2023-11-08T19:27:08.187644+0000","last_active":"2023-11-08T19:31:13.114710+0000","last_peered":"2023-11-08T19:31:13.114710+0000","last_clean":"2023-11-08T19:31:13.114710+0000","last_became_active":"2023-11-08T19:27:08.187376+0000","last_became_peered":"2023-11-08T19:27:08.187376+0000","last_unstale":"2023-11-08T19:31:13.114710+0000","last_undegraded":"2023-11-08T19:31:13.114710+0000","last_fullsized":"2023-11-08T19:31:13.114710+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:22:40.300185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":78,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.145602+0000","last_change":"2023-11-08T19:26:05.606946+0000","last_active":"2023-11-08T19:31:13.145602+0000","last_peered":"2023-11-08T19:31:13.145602+0000","last_clean":"2023-11-08T19:31:13.145602+0000","last_became_active":"2023-11-08T19:26:04.598775+0000","last_became_peered":"2023-11-08T19:26:04.598775+0000","last_unstale":"2023-11-08T19:31:13.145602+0000","last_undegraded":"2023-11-08T19:31:13.145602+0000","last_fullsized":"2023-11-08T19:31:13.145602+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:20:56.898927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049870399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":54,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.399068+0000","last_change":"2023-11-08T19:27:09.331646+0000","last_active":"2023-11-08T19:31:13.399068+0000","last_peered":"2023-11-08T19:31:13.399068+0000","last_clean":"2023-11-08T19:31:13.399068+0000","last_became_active":"2023-11-08T19:27:07.190619+0000","last_became_peered":"2023-11-08T19:27:07.190619+0000","last_unstale":"2023-11-08T19:31:13.399068+0000","last_undegraded":"2023-11-08T19:31:13.399068+0000","last_fullsized":"2023-11-08T19:31:13.399068+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:42:47.371458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065918400000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":54,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:14.013671+0000","last_change":"2023-11-08T19:27:08.183798+0000","last_active":"2023-11-08T19:31:14.013671+0000","last_peered":"2023-11-08T19:31:14.013671+0000","last_clean":"2023-11-08T19:31:14.013671+0000","last_became_active":"2023-11-08T19:27:08.183537+0000","last_became_peered":"2023-11-08T19:27:08.183537+0000","last_unstale":"2023-11-08T19:31:14.013671+0000","last_undegraded":"2023-11-08T19:31:14.013671+0000","last_fullsized":"2023-11-08T19:31:14.013671+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:50:39.026265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":89,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.011263+0000","last_change":"2023-11-08T19:26:05.610012+0000","last_active":"2023-11-08T19:31:13.011263+0000","last_peered":"2023-11-08T19:31:13.011263+0000","last_clean":"2023-11-08T19:31:13.011263+0000","last_became_active":"2023-11-08T19:26:04.603209+0000","last_became_peered":"2023-11-08T19:26:04.603209+0000","last_unstale":"2023-11-08T19:31:13.011263+0000","last_undegraded":"2023-11-08T19:31:13.011263+0000","last_fullsized":"2023-11-08T19:31:13.011263+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:35:02.306997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053335100000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":54,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.145463+0000","last_change":"2023-11-08T19:27:09.178405+0000","last_active":"2023-11-08T19:31:13.145463+0000","last_peered":"2023-11-08T19:31:13.145463+0000","last_clean":"2023-11-08T19:31:13.145463+0000","last_became_active":"2023-11-08T19:27:07.188169+0000","last_became_peered":"2023-11-08T19:27:07.188169+0000","last_unstale":"2023-11-08T19:31:13.145463+0000","last_undegraded":"2023-11-08T19:31:13.145463+0000","last_fullsized":"2023-11-08T19:31:13.145463+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:39:25.855467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056209700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":47,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.398571+0000","last_change":"2023-11-08T19:27:08.193508+0000","last_active":"2023-11-08T19:31:13.398571+0000","last_peered":"2023-11-08T19:31:13.398571+0000","last_clean":"2023-11-08T19:31:13.398571+0000","last_became_active":"2023-11-08T19:27:08.193037+0000","last_became_peered":"2023-11-08T19:27:08.193037+0000","last_unstale":"2023-11-08T19:31:13.398571+0000","last_undegraded":"2023-11-08T19:31:13.398571+0000","last_fullsized":"2023-11-08T19:31:13.398571+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:29:50.067887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":77,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.114176+0000","last_change":"2023-11-08T19:26:06.265318+0000","last_active":"2023-11-08T19:31:13.114176+0000","last_peered":"2023-11-08T19:31:13.114176+0000","last_clean":"2023-11-08T19:31:13.114176+0000","last_became_active":"2023-11-08T19:26:04.603409+0000","last_became_peered":"2023-11-08T19:26:04.603409+0000","last_unstale":"2023-11-08T19:31:13.114176+0000","last_undegraded":"2023-11-08T19:31:13.114176+0000","last_fullsized":"2023-11-08T19:31:13.114176+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:27:24.933111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065684300000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":60,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.010096+0000","last_change":"2023-11-08T19:27:09.182282+0000","last_active":"2023-11-08T19:31:13.010096+0000","last_peered":"2023-11-08T19:31:13.010096+0000","last_clean":"2023-11-08T19:31:13.010096+0000","last_became_active":"2023-11-08T19:27:07.186944+0000","last_became_peered":"2023-11-08T19:27:07.186944+0000","last_unstale":"2023-11-08T19:31:13.010096+0000","last_undegraded":"2023-11-08T19:31:13.010096+0000","last_fullsized":"2023-11-08T19:31:13.010096+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:45:21.079011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00028665599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":47,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.398807+0000","last_change":"2023-11-08T19:27:08.193990+0000","last_active":"2023-11-08T19:31:13.398807+0000","last_peered":"2023-11-08T19:31:13.398807+0000","last_clean":"2023-11-08T19:31:13.398807+0000","last_became_active":"2023-11-08T19:27:08.193624+0000","last_became_peered":"2023-11-08T19:27:08.193624+0000","last_unstale":"2023-11-08T19:31:13.398807+0000","last_undegraded":"2023-11-08T19:31:13.398807+0000","last_fullsized":"2023-11-08T19:31:13.398807+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:46:41.321120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":83,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.400400+0000","last_change":"2023-11-08T19:26:06.423863+0000","last_active":"2023-11-08T19:31:13.400400+0000","last_peered":"2023-11-08T19:31:13.400400+0000","last_clean":"2023-11-08T19:31:13.400400+0000","last_became_active":"2023-11-08T19:26:04.608639+0000","last_became_peered":"2023-11-08T19:26:04.608639+0000","last_unstale":"2023-11-08T19:31:13.400400+0000","last_undegraded":"2023-11-08T19:31:13.400400+0000","last_fullsized":"2023-11-08T19:31:13.400400+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:04:43.631102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044384999999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":54,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.399997+0000","last_change":"2023-11-08T19:27:09.283232+0000","last_active":"2023-11-08T19:31:13.399997+0000","last_peered":"2023-11-08T19:31:13.399997+0000","last_clean":"2023-11-08T19:31:13.399997+0000","last_became_active":"2023-11-08T19:27:07.190931+0000","last_became_peered":"2023-11-08T19:27:07.190931+0000","last_unstale":"2023-11-08T19:31:13.399997+0000","last_undegraded":"2023-11-08T19:31:13.399997+0000","last_fullsized":"2023-11-08T19:31:13.399997+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:43:28.225967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034146799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":51,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.400915+0000","last_change":"2023-11-08T19:27:08.187252+0000","last_active":"2023-11-08T19:31:13.400915+0000","last_peered":"2023-11-08T19:31:13.400915+0000","last_clean":"2023-11-08T19:31:13.400915+0000","last_became_active":"2023-11-08T19:27:08.186952+0000","last_became_peered":"2023-11-08T19:27:08.186952+0000","last_unstale":"2023-11-08T19:31:13.400915+0000","last_undegraded":"2023-11-08T19:31:13.400915+0000","last_fullsized":"2023-11-08T19:31:13.400915+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:12:52.022374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":77,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.399941+0000","last_change":"2023-11-08T19:26:06.394081+0000","last_active":"2023-11-08T19:31:13.399941+0000","last_peered":"2023-11-08T19:31:13.399941+0000","last_clean":"2023-11-08T19:31:13.399941+0000","last_became_active":"2023-11-08T19:26:04.602137+0000","last_became_peered":"2023-11-08T19:26:04.602137+0000","last_unstale":"2023-11-08T19:31:13.399941+0000","last_undegraded":"2023-11-08T19:31:13.399941+0000","last_fullsized":"2023-11-08T19:31:13.399941+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:05:46.951663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056998200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":54,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.146173+0000","last_change":"2023-11-08T19:27:09.178816+0000","last_active":"2023-11-08T19:31:13.146173+0000","last_peered":"2023-11-08T19:31:13.146173+0000","last_clean":"2023-11-08T19:31:13.146173+0000","last_became_active":"2023-11-08T19:27:07.192001+0000","last_became_peered":"2023-11-08T19:27:07.192001+0000","last_unstale":"2023-11-08T19:31:13.146173+0000","last_undegraded":"2023-11-08T19:31:13.146173+0000","last_fullsized":"2023-11-08T19:31:13.146173+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:04:12.481212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045359000000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":47,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.010479+0000","last_change":"2023-11-08T19:27:08.193836+0000","last_active":"2023-11-08T19:31:13.010479+0000","last_peered":"2023-11-08T19:31:13.010479+0000","last_clean":"2023-11-08T19:31:13.010479+0000","last_became_active":"2023-11-08T19:27:08.193458+0000","last_became_peered":"2023-11-08T19:27:08.193458+0000","last_unstale":"2023-11-08T19:31:13.010479+0000","last_undegraded":"2023-11-08T19:31:13.010479+0000","last_fullsized":"2023-11-08T19:31:13.010479+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:14:18.610061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":79,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.398714+0000","last_change":"2023-11-08T19:26:06.541338+0000","last_active":"2023-11-08T19:31:13.398714+0000","last_peered":"2023-11-08T19:31:13.398714+0000","last_clean":"2023-11-08T19:31:13.398714+0000","last_became_active":"2023-11-08T19:26:04.602183+0000","last_became_peered":"2023-11-08T19:26:04.602183+0000","last_unstale":"2023-11-08T19:31:13.398714+0000","last_undegraded":"2023-11-08T19:31:13.398714+0000","last_fullsized":"2023-11-08T19:31:13.398714+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:06:47.446686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052029099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":60,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.010485+0000","last_change":"2023-11-08T19:27:09.181622+0000","last_active":"2023-11-08T19:31:13.010485+0000","last_peered":"2023-11-08T19:31:13.010485+0000","last_clean":"2023-11-08T19:31:13.010485+0000","last_became_active":"2023-11-08T19:27:07.184507+0000","last_became_peered":"2023-11-08T19:27:07.184507+0000","last_unstale":"2023-11-08T19:31:13.010485+0000","last_undegraded":"2023-11-08T19:31:13.010485+0000","last_fullsized":"2023-11-08T19:31:13.010485+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:58:48.357266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051924700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":47,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.399167+0000","last_change":"2023-11-08T19:27:08.194263+0000","last_active":"2023-11-08T19:31:13.399167+0000","last_peered":"2023-11-08T19:31:13.399167+0000","last_clean":"2023-11-08T19:31:13.399167+0000","last_became_active":"2023-11-08T19:27:08.193938+0000","last_became_peered":"2023-11-08T19:27:08.193938+0000","last_unstale":"2023-11-08T19:31:13.399167+0000","last_undegraded":"2023-11-08T19:31:13.399167+0000","last_fullsized":"2023-11-08T19:31:13.399167+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:40:49.009717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":77,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.115375+0000","last_change":"2023-11-08T19:26:06.265690+0000","last_active":"2023-11-08T19:31:13.115375+0000","last_peered":"2023-11-08T19:31:13.115375+0000","last_clean":"2023-11-08T19:31:13.115375+0000","last_became_active":"2023-11-08T19:26:04.603397+0000","last_became_peered":"2023-11-08T19:26:04.603397+0000","last_unstale":"2023-11-08T19:31:13.115375+0000","last_undegraded":"2023-11-08T19:31:13.115375+0000","last_fullsized":"2023-11-08T19:31:13.115375+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:50:43.827570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053361499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":54,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.399190+0000","last_change":"2023-11-08T19:27:09.282750+0000","last_active":"2023-11-08T19:31:13.399190+0000","last_peered":"2023-11-08T19:31:13.399190+0000","last_clean":"2023-11-08T19:31:13.399190+0000","last_became_active":"2023-11-08T19:27:07.189943+0000","last_became_peered":"2023-11-08T19:27:07.189943+0000","last_unstale":"2023-11-08T19:31:13.399190+0000","last_undegraded":"2023-11-08T19:31:13.399190+0000","last_fullsized":"2023-11-08T19:31:13.399190+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:31:16.081058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000398725,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":53,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.009718+0000","last_change":"2023-11-08T19:27:08.188147+0000","last_active":"2023-11-08T19:31:13.009718+0000","last_peered":"2023-11-08T19:31:13.009718+0000","last_clean":"2023-11-08T19:31:13.009718+0000","last_became_active":"2023-11-08T19:27:08.187487+0000","last_became_peered":"2023-11-08T19:27:08.187487+0000","last_unstale":"2023-11-08T19:31:13.009718+0000","last_undegraded":"2023-11-08T19:31:13.009718+0000","last_fullsized":"2023-11-08T19:31:13.009718+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:22:40.300185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":83,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.401216+0000","last_change":"2023-11-08T19:26:06.425133+0000","last_active":"2023-11-08T19:31:13.401216+0000","last_peered":"2023-11-08T19:31:13.401216+0000","last_clean":"2023-11-08T19:31:13.401216+0000","last_became_active":"2023-11-08T19:26:04.601103+0000","last_became_peered":"2023-11-08T19:26:04.601103+0000","last_unstale":"2023-11-08T19:31:13.401216+0000","last_undegraded":"2023-11-08T19:31:13.401216+0000","last_fullsized":"2023-11-08T19:31:13.401216+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:31:14.042561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000229244,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":60,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.009084+0000","last_change":"2023-11-08T19:27:09.179852+0000","last_active":"2023-11-08T19:31:13.009084+0000","last_peered":"2023-11-08T19:31:13.009084+0000","last_clean":"2023-11-08T19:31:13.009084+0000","last_became_active":"2023-11-08T19:27:07.191926+0000","last_became_peered":"2023-11-08T19:27:07.191926+0000","last_unstale":"2023-11-08T19:31:13.009084+0000","last_undegraded":"2023-11-08T19:31:13.009084+0000","last_fullsized":"2023-11-08T19:31:13.009084+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:20:38.080129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055457100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":53,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.010318+0000","last_change":"2023-11-08T19:27:08.186054+0000","last_active":"2023-11-08T19:31:13.010318+0000","last_peered":"2023-11-08T19:31:13.010318+0000","last_clean":"2023-11-08T19:31:13.010318+0000","last_became_active":"2023-11-08T19:27:08.185581+0000","last_became_peered":"2023-11-08T19:27:08.185581+0000","last_unstale":"2023-11-08T19:31:13.010318+0000","last_undegraded":"2023-11-08T19:31:13.010318+0000","last_fullsized":"2023-11-08T19:31:13.010318+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:00:55.878397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":89,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.008858+0000","last_change":"2023-11-08T19:26:05.609625+0000","last_active":"2023-11-08T19:31:13.008858+0000","last_peered":"2023-11-08T19:31:13.008858+0000","last_clean":"2023-11-08T19:31:13.008858+0000","last_became_active":"2023-11-08T19:26:04.606312+0000","last_became_peered":"2023-11-08T19:26:04.606312+0000","last_unstale":"2023-11-08T19:31:13.008858+0000","last_undegraded":"2023-11-08T19:31:13.008858+0000","last_fullsized":"2023-11-08T19:31:13.008858+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:54:42.032901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000505935,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":54,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.144846+0000","last_change":"2023-11-08T19:27:09.177985+0000","last_active":"2023-11-08T19:31:13.144846+0000","last_peered":"2023-11-08T19:31:13.144846+0000","last_clean":"2023-11-08T19:31:13.144846+0000","last_became_active":"2023-11-08T19:27:07.193254+0000","last_became_peered":"2023-11-08T19:27:07.193254+0000","last_unstale":"2023-11-08T19:31:13.144846+0000","last_undegraded":"2023-11-08T19:31:13.144846+0000","last_fullsized":"2023-11-08T19:31:13.144846+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:23:38.662629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041830799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":53,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.011772+0000","last_change":"2023-11-08T19:27:08.188860+0000","last_active":"2023-11-08T19:31:13.011772+0000","last_peered":"2023-11-08T19:31:13.011772+0000","last_clean":"2023-11-08T19:31:13.011772+0000","last_became_active":"2023-11-08T19:27:08.188560+0000","last_became_peered":"2023-11-08T19:27:08.188560+0000","last_unstale":"2023-11-08T19:31:13.011772+0000","last_undegraded":"2023-11-08T19:31:13.011772+0000","last_fullsized":"2023-11-08T19:31:13.011772+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:18:20.969491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":78,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.010963+0000","last_change":"2023-11-08T19:26:06.297255+0000","last_active":"2023-11-08T19:31:13.010963+0000","last_peered":"2023-11-08T19:31:13.010963+0000","last_clean":"2023-11-08T19:31:13.010963+0000","last_became_active":"2023-11-08T19:26:04.603979+0000","last_became_peered":"2023-11-08T19:26:04.603979+0000","last_unstale":"2023-11-08T19:31:13.010963+0000","last_undegraded":"2023-11-08T19:31:13.010963+0000","last_fullsized":"2023-11-08T19:31:13.010963+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:40:11.164951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079032099999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":54,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-11-08T19:31:13.398610+0000","last_change":"2023-11-08T19:27:09.331990+0000","last_active":"2023-11-08T19:31:13.398610+0000","last_peered":"2023-11-08T19:31:13.398610+0000","last_clean":"2023-11-08T19:31:13.398610+0000","last_became_active":"2023-11-08T19:27:07.185478+0000","last_became_peered":"2023-11-08T19:27:07.185478+0000","last_unstale":"2023-11-08T19:31:13.398610+0000","last_undegraded":"2023-11-08T19:31:13.398610+0000","last_fullsized":"2023-11-08T19:31:13.398610+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:20:38.080129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010661850000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574931,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33056,"kb_used_data":1872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338784,"statfs":{"total":96636764160,"available":96602914816,"internally_reserved":0,"allocated":1916928,"data_stored":1236765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:30:56 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.54400000000000004,"15min":0.51100000000000001},"min":{"1min":0.45300000000000001,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.71899999999999997,"5min":0.78600000000000003,"15min":1.0149999999999999},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.53500000000000003,"15min":0.52100000000000002},"min":{"1min":0.42099999999999999,"5min":0.35099999999999998,"15min":0.32300000000000001},"max":{"1min":0.79300000000000004,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.63500000000000001}]},{"osd":1,"last update":"Wed Nov 8 19:30:56 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.59799999999999998,"15min":0.57599999999999996},"min":{"1min":0.47699999999999998,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.73499999999999999,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.623,"5min":0.59599999999999997,"15min":0.56699999999999995},"min":{"1min":0.505,"5min":0.40600000000000003,"15min":0.33900000000000002},"max":{"1min":0.79500000000000004,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.45000000000000001}]},{"osd":2,"last update":"Wed Nov 8 19:30:56 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.54400000000000004,"15min":0.505},"min":{"1min":0.372,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.67600000000000005,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.57399999999999995,"15min":0.55600000000000005},"min":{"1min":0.45900000000000002,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.78400000000000003,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.50600000000000001}]},{"osd":3,"last update":"Wed Nov 8 19:30:56 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.57499999999999996,"15min":0.56399999999999995},"min":{"1min":0.42199999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.77300000000000002,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.63800000000000001,"15min":0.59299999999999997},"min":{"1min":0.47399999999999998,"5min":0.437,"15min":0.40100000000000002},"max":{"1min":0.81399999999999995,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.61499999999999999}]},{"osd":4,"last update":"Wed Nov 8 19:30:56 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.501,"15min":0.505},"min":{"1min":0.32400000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.753,"5min":0.753,"15min":0.78600000000000003},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.53400000000000003,"15min":0.53700000000000003},"min":{"1min":0.35199999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.71799999999999997,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.48999999999999999}]},{"osd":5,"last update":"Wed Nov 8 19:30:56 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.56699999999999995,"15min":0.57499999999999996},"min":{"1min":0.35099999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.77300000000000002,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.627,"15min":0.63700000000000001},"min":{"1min":0.377,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.83899999999999997,"5min":0.89900000000000002,"15min":0.94299999999999995},"last":0.68500000000000005}]},{"osd":6,"last update":"Wed Nov 8 19:30:56 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.57999999999999996,"15min":0.59699999999999998},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.97299999999999998},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.59699999999999998,"15min":0.60499999999999998},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.78900000000000003,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.57399999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574931,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32476,"kb_used_data":1292,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339364,"statfs":{"total":96636764160,"available":96603508736,"internally_reserved":0,"allocated":1323008,"data_stored":646397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:30:57 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.55900000000000005,"15min":1.1459999999999999},"min":{"1min":0.41999999999999998,"5min":0.33200000000000002,"15min":0.27000000000000002},"max":{"1min":0.84599999999999997,"5min":0.91700000000000004,"15min":21.359999999999999},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.56899999999999995,"15min":0.81599999999999995},"min":{"1min":0.39700000000000002,"5min":0.378,"15min":0.30399999999999999},"max":{"1min":0.68700000000000006,"5min":0.90300000000000002,"15min":10.329000000000001},"last":0.432}]},{"osd":1,"last update":"Wed Nov 8 19:30:57 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.53800000000000003,"15min":0.59499999999999997},"min":{"1min":0.41199999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.81399999999999995,"5min":0.86299999999999999,"15min":3.4239999999999999},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.60499999999999998,"15min":0.63400000000000001},"min":{"1min":0.35899999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":2.9260000000000002,"5min":2.9260000000000002,"15min":3.4809999999999999},"last":0.48999999999999999}]},{"osd":2,"last update":"Wed Nov 8 19:30:57 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.58599999999999997,"15min":0.83399999999999996},"min":{"1min":0.51800000000000002,"5min":0.38500000000000001,"15min":0.34399999999999997},"max":{"1min":0.84199999999999997,"5min":0.96399999999999997,"15min":10.398},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.56799999999999995,"15min":0.66200000000000003},"min":{"1min":0.41799999999999998,"5min":0.318,"15min":0.318},"max":{"1min":0.77500000000000002,"5min":0.89100000000000001,"15min":5.0170000000000003},"last":0.57899999999999996}]},{"osd":3,"last update":"Wed Nov 8 19:30:57 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.63100000000000001,"15min":0.66100000000000003},"min":{"1min":0.46200000000000002,"5min":0.34599999999999997,"15min":0.317},"max":{"1min":3.0089999999999999,"5min":3.0089999999999999,"15min":3.5710000000000002},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.59699999999999998,"15min":0.85999999999999999},"min":{"1min":0.39000000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.81100000000000005,"5min":0.93500000000000005,"15min":10.368},"last":0.63800000000000001}]},{"osd":4,"last update":"Wed Nov 8 19:30:57 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54400000000000004,"15min":0.52000000000000002},"min":{"1min":0.34699999999999998,"5min":0.33000000000000002,"15min":0.29199999999999998},"max":{"1min":0.79100000000000004,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.55900000000000005,"15min":0.55800000000000005},"min":{"1min":0.45000000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.73799999999999999,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.748}]},{"osd":5,"last update":"Wed Nov 8 19:30:57 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.55000000000000004,"15min":0.53200000000000003},"min":{"1min":0.42699999999999999,"5min":0.34599999999999997,"15min":0.307},"max":{"1min":0.70499999999999996,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.60399999999999998,"15min":0.58599999999999997},"min":{"1min":0.41199999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.68899999999999995}]},{"osd":7,"last update":"Wed Nov 8 19:30:57 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.60999999999999999,"15min":0.58299999999999996},"min":{"1min":0.39700000000000002,"5min":0.39500000000000002,"15min":0.34699999999999998},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.60599999999999998,"15min":0.59199999999999997},"min":{"1min":0.46300000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.82199999999999995,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.75800000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574931,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32476,"kb_used_data":1292,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339364,"statfs":{"total":96636764160,"available":96603508736,"internally_reserved":0,"allocated":1323008,"data_stored":646397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:30:55 2023","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.35499999999999998,"15min":0.374},"min":{"1min":0.222,"5min":0.219,"15min":0.219},"max":{"1min":0.437,"5min":0.69699999999999995,"15min":1.389},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.36399999999999999,"15min":0.61699999999999999},"min":{"1min":0.255,"5min":0.23000000000000001,"15min":0.21199999999999999},"max":{"1min":0.52900000000000003,"5min":0.622,"15min":9.1370000000000005},"last":0.39000000000000001}]},{"osd":2,"last update":"Wed Nov 8 19:30:55 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.35899999999999999,"15min":0.36499999999999999},"min":{"1min":0.248,"5min":0.216,"15min":0.216},"max":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":1.2709999999999999},"last":0.372},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.39200000000000002,"15min":0.41899999999999998},"min":{"1min":0.27700000000000002,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.53400000000000003,"5min":0.63600000000000001,"15min":1.425},"last":0.35399999999999998}]},{"osd":3,"last update":"Wed Nov 8 19:30:55 2023","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.40899999999999997,"15min":0.74399999999999999},"min":{"1min":0.29599999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.54600000000000004,"5min":0.63800000000000001,"15min":11.957000000000001},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.39100000000000001,"15min":0.91800000000000004},"min":{"1min":0.27900000000000003,"5min":0.252,"15min":0.252},"max":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":18.039000000000001},"last":0.33500000000000002}]},{"osd":4,"last update":"Wed Nov 8 19:30:55 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59299999999999997,"15min":0.751},"min":{"1min":0.35199999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.81100000000000005,"5min":1.4370000000000001,"15min":6.181},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.60999999999999999,"15min":0.63},"min":{"1min":0.41399999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.83899999999999997,"5min":1.411,"15min":1.488},"last":0.439}]},{"osd":5,"last update":"Wed Nov 8 19:30:55 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61199999999999999,"15min":0.86099999999999999},"min":{"1min":0.375,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.85399999999999998,"5min":1.421,"15min":9.2420000000000009},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60599999999999998,"15min":0.78900000000000003},"min":{"1min":0.42499999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.80100000000000005,"5min":1.2969999999999999,"15min":6.2110000000000003},"last":0.44800000000000001}]},{"osd":6,"last update":"Wed Nov 8 19:30:55 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.63500000000000001,"15min":0.86899999999999999},"min":{"1min":0.40799999999999997,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.78700000000000003,"5min":1.4530000000000001,"15min":9.1880000000000006},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63500000000000001,"15min":0.65600000000000003},"min":{"1min":0.44500000000000001,"5min":0.38,"15min":0.38},"max":{"1min":0.86399999999999999,"5min":1.272,"15min":1.462},"last":0.48799999999999999}]},{"osd":7,"last update":"Wed Nov 8 19:30:55 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.625,"15min":0.78800000000000003},"min":{"1min":0.41499999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0309999999999999,"5min":1.3979999999999999,"15min":6.1479999999999997},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.64600000000000002,"15min":0.90000000000000002},"min":{"1min":0.51300000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.97499999999999998,"5min":1.353,"15min":9.2170000000000005},"last":0.499}]}]},{"osd":0,"up_from":13,"seq":55834574931,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33056,"kb_used_data":1872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338784,"statfs":{"total":96636764160,"available":96602914816,"internally_reserved":0,"allocated":1916928,"data_stored":1236765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Nov 8 19:31:01 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.33300000000000002,"15min":0.45900000000000002},"min":{"1min":0.20899999999999999,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.51000000000000001,"5min":0.55300000000000005,"15min":4.1200000000000001},"last":0.25},{"interface":"front","average":{"1min":0.439,"5min":0.371,"15min":0.91900000000000004},"min":{"1min":0.22800000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.62,"5min":0.62,"15min":16.940999999999999},"last":0.35499999999999998}]},{"osd":2,"last update":"Wed Nov 8 19:31:01 2023","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.35299999999999998,"15min":0.38700000000000001},"min":{"1min":0.25,"5min":0.22,"15min":0.22},"max":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":1.1899999999999999},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.39100000000000001,"15min":0.60499999999999998},"min":{"1min":0.32400000000000001,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":7.0209999999999999},"last":0.28199999999999997}]},{"osd":3,"last update":"Wed Nov 8 19:31:01 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.38400000000000001,"15min":0.51200000000000001},"min":{"1min":0.29899999999999999,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":4.0940000000000003},"last":0.29799999999999999},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.39500000000000002,"15min":0.433},"min":{"1min":0.32500000000000001,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.61899999999999999,"5min":0.62,"15min":1.2549999999999999},"last":0.32600000000000001}]},{"osd":4,"last update":"Wed Nov 8 19:31:01 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.58899999999999997,"15min":0.71699999999999997},"min":{"1min":0.42799999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":4.0049999999999999},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.58399999999999996,"15min":0.623},"min":{"1min":0.442,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.746,"5min":0.79200000000000004,"15min":1.2270000000000001},"last":0.379}]},{"osd":5,"last update":"Wed Nov 8 19:31:01 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.59699999999999998,"15min":0.72799999999999998},"min":{"1min":0.47199999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":4.3090000000000002},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.61199999999999999,"15min":0.72399999999999998},"min":{"1min":0.436,"5min":0.373,"15min":0.373},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":4.0709999999999997},"last":0.435}]},{"osd":6,"last update":"Wed Nov 8 19:31:01 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.621,"15min":0.65800000000000003},"min":{"1min":0.40999999999999998,"5min":0.373,"15min":0.373},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":1.2849999999999999},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.64900000000000002,"15min":0.78500000000000003},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":4.2560000000000002},"last":0.45900000000000002}]},{"osd":7,"last update":"Wed Nov 8 19:31:01 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.625,"15min":0.74299999999999999},"min":{"1min":0.47099999999999997,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":4.2850000000000001},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.64700000000000002,"15min":0.77000000000000002},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.002,"5min":1.002,"15min":4.0460000000000003},"last":0.47299999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574931,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28316,"kb_used_data":1292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343524,"statfs":{"total":96636764160,"available":96607768576,"internally_reserved":0,"allocated":1323008,"data_stored":646397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:30:53 2023","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.35099999999999998,"15min":0.33800000000000002},"min":{"1min":0.23599999999999999,"5min":0.182,"15min":0.182},"max":{"1min":0.629,"5min":0.629,"15min":0.629},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.35899999999999999,"15min":0.55900000000000005},"min":{"1min":0.26800000000000002,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.56499999999999995,"5min":0.68000000000000005,"15min":6.0999999999999996},"last":0.375}]},{"osd":1,"last update":"Wed Nov 8 19:30:53 2023","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.35599999999999998,"15min":0.34599999999999997},"min":{"1min":0.247,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.38800000000000001,"15min":0.39000000000000001},"min":{"1min":0.30299999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.63100000000000001,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.34499999999999997}]},{"osd":3,"last update":"Wed Nov 8 19:30:53 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.40600000000000003,"15min":0.39200000000000002},"min":{"1min":0.28199999999999997,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.60999999999999999,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.40000000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"min":{"1min":0.28799999999999998,"5min":0.217,"15min":0.217},"max":{"1min":0.621,"5min":0.65900000000000003,"15min":0.70299999999999996},"last":0.41999999999999998}]},{"osd":4,"last update":"Wed Nov 8 19:30:53 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56000000000000005,"15min":0.57599999999999996},"min":{"1min":0.41699999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.502},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.58099999999999996,"15min":0.58299999999999996},"min":{"1min":0.40500000000000003,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.872,"5min":0.872,"15min":0.90400000000000003},"last":0.54300000000000004}]},{"osd":5,"last update":"Wed Nov 8 19:30:53 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61799999999999999,"15min":0.59399999999999997},"min":{"1min":0.40799999999999997,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.622,"5min":0.60599999999999998,"15min":0.61099999999999999},"min":{"1min":0.48799999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.57799999999999996}]},{"osd":6,"last update":"Wed Nov 8 19:30:53 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.59399999999999997,"15min":0.59799999999999998},"min":{"1min":0.432,"5min":0.379,"15min":0.371},"max":{"1min":0.81000000000000005,"5min":0.83599999999999997,"15min":0.84799999999999998},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.59999999999999998,"15min":0.58599999999999997},"min":{"1min":0.42799999999999999,"5min":0.40300000000000002,"15min":0.36599999999999999},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.48099999999999998}]},{"osd":7,"last update":"Wed Nov 8 19:30:53 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.60899999999999999,"15min":0.58899999999999997},"min":{"1min":0.46200000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.61699999999999999,"15min":0.60899999999999999},"min":{"1min":0.41699999999999998,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.79600000000000004,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.56699999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574932,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28316,"kb_used_data":1292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343524,"statfs":{"total":96636764160,"available":96607768576,"internally_reserved":0,"allocated":1323008,"data_stored":646397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:30:59 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.33500000000000002,"15min":0.34000000000000002},"min":{"1min":0.254,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.48599999999999999,"5min":0.49099999999999999,"15min":0.626},"last":0.28599999999999998},{"interface":"front","average":{"1min":0.378,"5min":0.372,"15min":0.35599999999999998},"min":{"1min":0.26700000000000002,"5min":0.23699999999999999,"15min":0.23300000000000001},"max":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.60799999999999998},"last":0.35499999999999998}]},{"osd":1,"last update":"Wed Nov 8 19:30:59 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.35999999999999999,"15min":0.35599999999999998},"min":{"1min":0.245,"5min":0.24099999999999999,"15min":0.23999999999999999},"max":{"1min":0.53300000000000003,"5min":0.61699999999999999,"15min":0.61699999999999999},"last":0.317},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.40100000000000002,"15min":0.39000000000000001},"min":{"1min":0.30099999999999999,"5min":0.27900000000000003,"15min":0.27700000000000002},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.82099999999999995},"last":0.33100000000000002}]},{"osd":2,"last update":"Wed Nov 8 19:30:59 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.40999999999999998,"15min":0.38300000000000001},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28799999999999998},"max":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58499999999999996},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.41999999999999998,"15min":0.40899999999999997},"min":{"1min":0.30499999999999999,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.59799999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.435}]},{"osd":4,"last update":"Wed Nov 8 19:30:59 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.63,"15min":0.60799999999999998},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.85699999999999998,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63700000000000001,"15min":0.61499999999999999},"min":{"1min":0.38600000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.89600000000000002,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.58899999999999997}]},{"osd":5,"last update":"Wed Nov 8 19:30:59 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59899999999999998,"15min":0.58299999999999996},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.83099999999999996,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.623,"5min":0.622,"15min":0.59199999999999997},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.879,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.49399999999999999}]},{"osd":6,"last update":"Wed Nov 8 19:30:59 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.61399999999999999,"15min":0.59099999999999997},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41399999999999998},"max":{"1min":0.83899999999999997,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.621},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63300000000000001,"15min":0.61299999999999999},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.90500000000000003},"last":0.63100000000000001}]},{"osd":7,"last update":"Wed Nov 8 19:30:59 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.626,"15min":0.59999999999999998},"min":{"1min":0.41899999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":1.0009999999999999},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.63900000000000001,"15min":0.60999999999999999},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.85099999999999998,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.64600000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574931,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28316,"kb_used_data":1292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343524,"statfs":{"total":96636764160,"available":96607768576,"internally_reserved":0,"allocated":1323008,"data_stored":646397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:30:58 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54700000000000004,"15min":0.69899999999999995},"min":{"1min":0.311,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.71499999999999997,"5min":0.84799999999999998,"15min":6.5970000000000004},"last":0.44},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.53500000000000003,"15min":0.56299999999999994},"min":{"1min":0.36299999999999999,"5min":0.32200000000000001,"15min":0.317},"max":{"1min":0.73099999999999998,"5min":0.75900000000000001,"15min":2.5030000000000001},"last":0.64100000000000001}]},{"osd":1,"last update":"Wed Nov 8 19:30:58 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.55400000000000005,"15min":0.51500000000000001},"min":{"1min":0.41699999999999998,"5min":0.32900000000000001,"15min":0.24299999999999999},"max":{"1min":0.72799999999999998,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.55900000000000005,"15min":0.52500000000000002},"min":{"1min":0.49299999999999999,"5min":0.312,"15min":0.312},"max":{"1min":0.72799999999999998,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.53900000000000003}]},{"osd":2,"last update":"Wed Nov 8 19:30:58 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55300000000000005},"min":{"1min":0.505,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.64400000000000002,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.58799999999999997,"15min":0.76000000000000001},"min":{"1min":0.50700000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.70999999999999996,"5min":0.871,"15min":6.6920000000000002},"last":0.48999999999999999}]},{"osd":3,"last update":"Wed Nov 8 19:30:58 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.56499999999999995,"15min":0.58099999999999996},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.69799999999999995,"5min":0.79800000000000004,"15min":1.0700000000000001},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.61799999999999999,"15min":0.61299999999999999},"min":{"1min":0.47799999999999998,"5min":0.41699999999999998,"15min":0.34599999999999997},"max":{"1min":0.70399999999999996,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.50900000000000001}]},{"osd":5,"last update":"Wed Nov 8 19:30:58 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.501,"15min":0.53100000000000003},"min":{"1min":0.39100000000000001,"5min":0.29599999999999999,"15min":0.28100000000000003},"max":{"1min":0.66100000000000003,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.621},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.51700000000000002,"15min":0.57099999999999995},"min":{"1min":0.377,"5min":0.32600000000000001,"15min":0.30299999999999999},"max":{"1min":0.65700000000000003,"5min":0.876,"15min":1.1060000000000001},"last":0.56799999999999995}]},{"osd":6,"last update":"Wed Nov 8 19:30:58 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.54700000000000004,"15min":0.57199999999999995},"min":{"1min":0.35199999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.71899999999999997,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.60299999999999998,"15min":0.60999999999999999},"min":{"1min":0.373,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.74099999999999999,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.65800000000000003}]},{"osd":7,"last update":"Wed Nov 8 19:30:58 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.55800000000000005,"15min":0.60999999999999999},"min":{"1min":0.41599999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.68000000000000005,"5min":0.84499999999999997,"15min":1.141},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.59199999999999997,"15min":0.63},"min":{"1min":0.41399999999999998,"5min":0.38700000000000001,"15min":0.33000000000000002},"max":{"1min":0.76400000000000001,"5min":0.86399999999999999,"15min":0.86799999999999999},"last":0.71499999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574931,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28316,"kb_used_data":1292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343524,"statfs":{"total":96636764160,"available":96607768576,"internally_reserved":0,"allocated":1323008,"data_stored":646397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:30:51 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.57199999999999995,"15min":0.53900000000000003},"min":{"1min":0.35199999999999998,"5min":0.307,"15min":0.307},"max":{"1min":3.9689999999999999,"5min":3.9689999999999999,"15min":3.9689999999999999},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.54800000000000004,"15min":0.52200000000000002},"min":{"1min":0.41699999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.68899999999999995,"5min":0.746,"15min":0.746},"last":0.67200000000000004}]},{"osd":1,"last update":"Wed Nov 8 19:30:51 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.55300000000000005,"15min":0.53300000000000003},"min":{"1min":0.42399999999999999,"5min":0.37,"15min":0.37},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.54900000000000004,"15min":0.54100000000000004},"min":{"1min":0.44400000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.753,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.57899999999999996}]},{"osd":2,"last update":"Wed Nov 8 19:30:51 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.56399999999999995,"15min":0.54400000000000004},"min":{"1min":0.40000000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.749,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.621},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.56499999999999995,"15min":0.57299999999999995},"min":{"1min":0.46400000000000002,"5min":0.316,"15min":0.316},"max":{"1min":0.746,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.64400000000000002}]},{"osd":3,"last update":"Wed Nov 8 19:30:51 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.58499999999999996,"15min":0.56200000000000006},"min":{"1min":0.39600000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.73699999999999999,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.57999999999999996,"15min":0.59599999999999997},"min":{"1min":0.39900000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.79000000000000004,"5min":0.87,"15min":0.87},"last":0.60499999999999998}]},{"osd":4,"last update":"Wed Nov 8 19:30:51 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.495,"15min":0.54000000000000004},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.71899999999999997,"5min":0.80500000000000005,"15min":0.86799999999999999},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.53900000000000003,"15min":0.56100000000000005},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.751,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.41399999999999998}]},{"osd":6,"last update":"Wed Nov 8 19:30:51 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.56100000000000005,"15min":0.59599999999999997},"min":{"1min":0.39700000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.73599999999999999,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59199999999999997,"15min":0.60899999999999999},"min":{"1min":0.41399999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.76400000000000001,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.52400000000000002}]},{"osd":7,"last update":"Wed Nov 8 19:30:51 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.57099999999999995,"15min":0.59899999999999998},"min":{"1min":0.38200000000000001,"5min":0.371,"15min":0.36899999999999999},"max":{"1min":0.74099999999999999,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58999999999999997,"15min":0.63},"min":{"1min":0.433,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.91900000000000004},"last":0.496}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-08T19:31:22.951 INFO:tasks.ceph:pgid 12.d last_scrub_stamp 2023-11-08T19:27:07.167285+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=7, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:22.952 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-11-08T19:24:44.575133+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=24, tm_sec=44, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:22.952 INFO:tasks.ceph:pgid 6.7 last_scrub_stamp 2023-11-08T19:26:03.578599+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=26, tm_sec=3, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:22.953 INFO:tasks.ceph:pgid 11.a last_scrub_stamp 2023-11-08T19:27:06.162282+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=6, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:22.954 INFO:tasks.ceph:pgid 12.8 last_scrub_stamp 2023-11-08T19:27:07.167285+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=7, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:22.954 INFO:tasks.ceph:pgid 6.2 last_scrub_stamp 2023-11-08T19:26:03.578599+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=26, tm_sec=3, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:22.955 INFO:tasks.ceph:pgid 11.f last_scrub_stamp 2023-11-08T19:27:06.162282+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=6, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:22.955 INFO:tasks.ceph:pgid 12.9 last_scrub_stamp 2023-11-08T19:27:07.167285+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=7, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:22.956 INFO:tasks.ceph:pgid 6.3 last_scrub_stamp 2023-11-08T19:26:03.578599+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=26, tm_sec=3, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:22.956 INFO:tasks.ceph:pgid 11.e last_scrub_stamp 2023-11-08T19:27:06.162282+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=6, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:22.957 INFO:tasks.ceph:pgid 12.a last_scrub_stamp 2023-11-08T19:27:07.167285+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=7, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:22.958 INFO:tasks.ceph:pgid 6.0 last_scrub_stamp 2023-11-08T19:26:03.578599+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=26, tm_sec=3, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:22.959 INFO:tasks.ceph:pgid 11.d last_scrub_stamp 2023-11-08T19:27:06.162282+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=6, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:22.960 INFO:tasks.ceph:pgid 12.b last_scrub_stamp 2023-11-08T19:27:07.167285+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=7, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:22.960 INFO:tasks.ceph:pgid 6.1 last_scrub_stamp 2023-11-08T19:26:03.578599+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=26, tm_sec=3, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:22.961 INFO:tasks.ceph:pgid 11.c last_scrub_stamp 2023-11-08T19:27:06.162282+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=6, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:22.962 INFO:tasks.ceph:pgid 12.c last_scrub_stamp 2023-11-08T19:27:07.167285+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=7, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:22.962 INFO:tasks.ceph:pgid 6.6 last_scrub_stamp 2023-11-08T19:26:03.578599+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=26, tm_sec=3, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:22.963 INFO:tasks.ceph:pgid 11.b last_scrub_stamp 2023-11-08T19:27:06.162282+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=6, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:22.964 INFO:tasks.ceph:pgid 12.e last_scrub_stamp 2023-11-08T19:27:07.167285+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=7, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:22.964 INFO:tasks.ceph:pgid 6.4 last_scrub_stamp 2023-11-08T19:26:03.578599+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=26, tm_sec=3, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:22.965 INFO:tasks.ceph:pgid 11.9 last_scrub_stamp 2023-11-08T19:27:06.162282+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=6, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:22.966 INFO:tasks.ceph:pgid 12.f last_scrub_stamp 2023-11-08T19:27:07.167285+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=7, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:22.967 INFO:tasks.ceph:pgid 6.5 last_scrub_stamp 2023-11-08T19:26:03.578599+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=26, tm_sec=3, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:22.967 INFO:tasks.ceph:pgid 11.8 last_scrub_stamp 2023-11-08T19:27:06.162282+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=6, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:22.968 INFO:tasks.ceph:pgid 6.a last_scrub_stamp 2023-11-08T19:26:03.578599+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=26, tm_sec=3, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:22.968 INFO:tasks.ceph:pgid 11.7 last_scrub_stamp 2023-11-08T19:27:06.162282+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=6, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:22.968 INFO:tasks.ceph:pgid 12.0 last_scrub_stamp 2023-11-08T19:27:07.167285+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=7, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:22.969 INFO:tasks.ceph:pgid 6.b last_scrub_stamp 2023-11-08T19:26:03.578599+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=26, tm_sec=3, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:22.969 INFO:tasks.ceph:pgid 11.6 last_scrub_stamp 2023-11-08T19:27:06.162282+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=6, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:22.969 INFO:tasks.ceph:pgid 12.1 last_scrub_stamp 2023-11-08T19:27:07.167285+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=7, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:22.970 INFO:tasks.ceph:pgid 6.8 last_scrub_stamp 2023-11-08T19:26:03.578599+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=26, tm_sec=3, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:22.970 INFO:tasks.ceph:pgid 11.5 last_scrub_stamp 2023-11-08T19:27:06.162282+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=6, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:22.971 INFO:tasks.ceph:pgid 12.2 last_scrub_stamp 2023-11-08T19:27:07.167285+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=7, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:22.971 INFO:tasks.ceph:pgid 6.9 last_scrub_stamp 2023-11-08T19:26:03.578599+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=26, tm_sec=3, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:22.971 INFO:tasks.ceph:pgid 11.4 last_scrub_stamp 2023-11-08T19:27:06.162282+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=6, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:22.972 INFO:tasks.ceph:pgid 12.3 last_scrub_stamp 2023-11-08T19:27:07.167285+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=7, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:22.972 INFO:tasks.ceph:pgid 6.e last_scrub_stamp 2023-11-08T19:26:03.578599+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=26, tm_sec=3, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:22.973 INFO:tasks.ceph:pgid 11.3 last_scrub_stamp 2023-11-08T19:27:06.162282+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=6, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:22.973 INFO:tasks.ceph:pgid 12.4 last_scrub_stamp 2023-11-08T19:27:07.167285+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=7, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:22.973 INFO:tasks.ceph:pgid 12.5 last_scrub_stamp 2023-11-08T19:27:07.167285+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=7, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:22.974 INFO:tasks.ceph:pgid 6.f last_scrub_stamp 2023-11-08T19:26:03.578599+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=26, tm_sec=3, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:22.974 INFO:tasks.ceph:pgid 11.2 last_scrub_stamp 2023-11-08T19:27:06.162282+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=6, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:22.975 INFO:tasks.ceph:pgid 12.6 last_scrub_stamp 2023-11-08T19:27:07.167285+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=7, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:22.975 INFO:tasks.ceph:pgid 6.c last_scrub_stamp 2023-11-08T19:26:03.578599+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=26, tm_sec=3, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:22.975 INFO:tasks.ceph:pgid 11.1 last_scrub_stamp 2023-11-08T19:27:06.162282+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=6, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:22.976 INFO:tasks.ceph:pgid 12.7 last_scrub_stamp 2023-11-08T19:27:07.167285+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=7, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:22.976 INFO:tasks.ceph:pgid 6.d last_scrub_stamp 2023-11-08T19:26:03.578599+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=26, tm_sec=3, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:22.977 INFO:tasks.ceph:pgid 11.0 last_scrub_stamp 2023-11-08T19:27:06.162282+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=6, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:22.977 INFO:tasks.ceph:pgid 12.18 last_scrub_stamp 2023-11-08T19:27:07.167285+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=7, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:22.977 INFO:tasks.ceph:pgid 6.12 last_scrub_stamp 2023-11-08T19:26:03.578599+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=26, tm_sec=3, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:22.978 INFO:tasks.ceph:pgid 11.1f last_scrub_stamp 2023-11-08T19:27:06.162282+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=6, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:22.978 INFO:tasks.ceph:pgid 12.19 last_scrub_stamp 2023-11-08T19:27:07.167285+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=7, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:22.978 INFO:tasks.ceph:pgid 6.13 last_scrub_stamp 2023-11-08T19:26:03.578599+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=26, tm_sec=3, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:22.979 INFO:tasks.ceph:pgid 11.1e last_scrub_stamp 2023-11-08T19:27:06.162282+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=6, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:22.979 INFO:tasks.ceph:pgid 12.1a last_scrub_stamp 2023-11-08T19:27:07.167285+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=7, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:22.979 INFO:tasks.ceph:pgid 6.10 last_scrub_stamp 2023-11-08T19:26:03.578599+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=26, tm_sec=3, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:22.980 INFO:tasks.ceph:pgid 11.1d last_scrub_stamp 2023-11-08T19:27:06.162282+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=6, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:22.980 INFO:tasks.ceph:pgid 12.1b last_scrub_stamp 2023-11-08T19:27:07.167285+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=7, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:22.981 INFO:tasks.ceph:pgid 6.11 last_scrub_stamp 2023-11-08T19:26:03.578599+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=26, tm_sec=3, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:22.981 INFO:tasks.ceph:pgid 11.1c last_scrub_stamp 2023-11-08T19:27:06.162282+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=6, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:22.981 INFO:tasks.ceph:pgid 12.1c last_scrub_stamp 2023-11-08T19:27:07.167285+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=7, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:22.982 INFO:tasks.ceph:pgid 6.16 last_scrub_stamp 2023-11-08T19:26:03.578599+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=26, tm_sec=3, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:22.982 INFO:tasks.ceph:pgid 11.1b last_scrub_stamp 2023-11-08T19:27:06.162282+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=6, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:22.982 INFO:tasks.ceph:pgid 12.1d last_scrub_stamp 2023-11-08T19:27:07.167285+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=7, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:22.983 INFO:tasks.ceph:pgid 6.17 last_scrub_stamp 2023-11-08T19:26:03.578599+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=26, tm_sec=3, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:22.983 INFO:tasks.ceph:pgid 11.1a last_scrub_stamp 2023-11-08T19:27:06.162282+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=6, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:22.984 INFO:tasks.ceph:pgid 12.1e last_scrub_stamp 2023-11-08T19:27:07.167285+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=7, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:22.984 INFO:tasks.ceph:pgid 6.14 last_scrub_stamp 2023-11-08T19:26:03.578599+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=26, tm_sec=3, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:22.984 INFO:tasks.ceph:pgid 11.19 last_scrub_stamp 2023-11-08T19:27:06.162282+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=6, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:22.985 INFO:tasks.ceph:pgid 12.1f last_scrub_stamp 2023-11-08T19:27:07.167285+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=7, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:22.985 INFO:tasks.ceph:pgid 6.15 last_scrub_stamp 2023-11-08T19:26:03.578599+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=26, tm_sec=3, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:22.985 INFO:tasks.ceph:pgid 11.18 last_scrub_stamp 2023-11-08T19:27:06.162282+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=6, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:22.986 INFO:tasks.ceph:pgid 12.10 last_scrub_stamp 2023-11-08T19:27:07.167285+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=7, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:22.986 INFO:tasks.ceph:pgid 6.1a last_scrub_stamp 2023-11-08T19:26:03.578599+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=26, tm_sec=3, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:22.987 INFO:tasks.ceph:pgid 11.17 last_scrub_stamp 2023-11-08T19:27:06.162282+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=6, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:22.987 INFO:tasks.ceph:pgid 12.11 last_scrub_stamp 2023-11-08T19:27:07.167285+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=7, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:22.987 INFO:tasks.ceph:pgid 6.1b last_scrub_stamp 2023-11-08T19:26:03.578599+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=26, tm_sec=3, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:22.988 INFO:tasks.ceph:pgid 11.16 last_scrub_stamp 2023-11-08T19:27:06.162282+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=6, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:22.988 INFO:tasks.ceph:pgid 12.12 last_scrub_stamp 2023-11-08T19:27:07.167285+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=7, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:22.989 INFO:tasks.ceph:pgid 6.18 last_scrub_stamp 2023-11-08T19:26:03.578599+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=26, tm_sec=3, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:22.989 INFO:tasks.ceph:pgid 11.15 last_scrub_stamp 2023-11-08T19:27:06.162282+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=6, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:22.989 INFO:tasks.ceph:pgid 12.13 last_scrub_stamp 2023-11-08T19:27:07.167285+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=7, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:22.990 INFO:tasks.ceph:pgid 6.19 last_scrub_stamp 2023-11-08T19:26:03.578599+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=26, tm_sec=3, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:22.990 INFO:tasks.ceph:pgid 11.14 last_scrub_stamp 2023-11-08T19:27:06.162282+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=6, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:22.991 INFO:tasks.ceph:pgid 12.14 last_scrub_stamp 2023-11-08T19:27:07.167285+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=7, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:22.991 INFO:tasks.ceph:pgid 6.1e last_scrub_stamp 2023-11-08T19:26:03.578599+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=26, tm_sec=3, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:22.991 INFO:tasks.ceph:pgid 11.13 last_scrub_stamp 2023-11-08T19:27:06.162282+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=6, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:22.992 INFO:tasks.ceph:pgid 12.15 last_scrub_stamp 2023-11-08T19:27:07.167285+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=7, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:22.992 INFO:tasks.ceph:pgid 6.1f last_scrub_stamp 2023-11-08T19:26:03.578599+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=26, tm_sec=3, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:22.992 INFO:tasks.ceph:pgid 11.12 last_scrub_stamp 2023-11-08T19:27:06.162282+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=6, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:22.993 INFO:tasks.ceph:pgid 12.16 last_scrub_stamp 2023-11-08T19:27:07.167285+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=7, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:22.993 INFO:tasks.ceph:pgid 6.1c last_scrub_stamp 2023-11-08T19:26:03.578599+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=26, tm_sec=3, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:22.994 INFO:tasks.ceph:pgid 11.11 last_scrub_stamp 2023-11-08T19:27:06.162282+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=6, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:22.994 INFO:tasks.ceph:pgid 12.17 last_scrub_stamp 2023-11-08T19:27:07.167285+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=7, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:22.994 INFO:tasks.ceph:pgid 6.1d last_scrub_stamp 2023-11-08T19:26:03.578599+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=26, tm_sec=3, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:22.995 INFO:tasks.ceph:pgid 11.10 last_scrub_stamp 2023-11-08T19:27:06.162282+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=6, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:22.995 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-11-08T19:31:27.814 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T19:31:27.818 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T19:31:42.996 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-08T19:31:43.133 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:43.132+0000 7f3e5a1cb700 1 Processor -- start 2023-11-08T19:31:43.134 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:43.132+0000 7f3e5a1cb700 1 -- start start 2023-11-08T19:31:43.141 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:43.132+0000 7f3e5a1cb700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f3e541618f0 0x7f3e5406ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:43.141 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:43.132+0000 7f3e5a1cb700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e54161df0 con 0x7f3e5415dd00 2023-11-08T19:31:43.142 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:43.132+0000 7f3e5a1cb700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e541620d0 con 0x7f3e5406b150 2023-11-08T19:31:43.142 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:43.132+0000 7f3e5a1cb700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e5406b600 con 0x7f3e541618f0 2023-11-08T19:31:43.142 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:43.132+0000 7f3e537fe700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f3e5415dd00 0x7f3e540cca70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:53560/0 (socket says 172.21.15.143:53560) 2023-11-08T19:31:43.143 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:43.133+0000 7f3e537fe700 1 -- 172.21.15.143:0/3114045121 learned_addr learned my addr 172.21.15.143:0/3114045121 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:31:43.143 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:43.133+0000 7f3e527fc700 1 -- 172.21.15.143:0/3114045121 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2130416452 0 0) 0x7f3e54161df0 con 0x7f3e5415dd00 2023-11-08T19:31:43.143 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:43.133+0000 7f3e527fc700 1 -- 172.21.15.143:0/3114045121 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e400035b0 con 0x7f3e5415dd00 2023-11-08T19:31:43.144 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:43.133+0000 7f3e52ffd700 1 --2- 172.21.15.143:0/3114045121 >> v2:172.21.15.174:3300/0 conn(0x7f3e541618f0 0x7f3e5406ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:43.144 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:43.133+0000 7f3e527fc700 1 -- 172.21.15.143:0/3114045121 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 216033269 0 0) 0x7f3e400035b0 con 0x7f3e5415dd00 2023-11-08T19:31:43.144 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:43.133+0000 7f3e527fc700 1 -- 172.21.15.143:0/3114045121 >> v2:172.21.15.174:3300/0 conn(0x7f3e541618f0 msgr2=0x7f3e5406ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:43.145 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:43.133+0000 7f3e527fc700 1 --2- 172.21.15.143:0/3114045121 >> v2:172.21.15.174:3300/0 conn(0x7f3e541618f0 0x7f3e5406ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:43.145 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:43.133+0000 7f3e527fc700 1 -- 172.21.15.143:0/3114045121 >> v1:172.21.15.174:6789/0 conn(0x7f3e5406b150 legacy=0x7f3e540cb1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:43.145 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:43.133+0000 7f3e527fc700 1 -- 172.21.15.143:0/3114045121 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e541623b0 con 0x7f3e5415dd00 2023-11-08T19:31:43.146 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:43.133+0000 7f3e527fc700 1 -- 172.21.15.143:0/3114045121 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4147756303 0 0) 0x7f3e440017f0 con 0x7f3e5415dd00 2023-11-08T19:31:43.146 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:43.133+0000 7f3e527fc700 1 -- 172.21.15.143:0/3114045121 >> v1:172.21.15.143:6789/0 conn(0x7f3e5415dd00 legacy=0x7f3e540cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:43.146 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:43.133+0000 7f3e527fc700 1 --2- 172.21.15.143:0/3114045121 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3e40003a00 0x7f3e40003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:43.147 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:43.133+0000 7f3e527fc700 1 --2- 172.21.15.143:0/3114045121 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e40004850 0x7f3e40004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:43.147 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:43.133+0000 7f3e527fc700 1 --2- 172.21.15.143:0/3114045121 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3e40005210 0x7f3e4000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:43.147 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:43.133+0000 7f3e527fc700 1 -- 172.21.15.143:0/3114045121 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e5406b600 con 0x7f3e40004850 2023-11-08T19:31:43.148 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:43.133+0000 7f3e527fc700 1 -- 172.21.15.143:0/3114045121 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e4000c0b0 con 0x7f3e40005210 2023-11-08T19:31:43.148 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:43.133+0000 7f3e527fc700 1 -- 172.21.15.143:0/3114045121 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e4000c270 con 0x7f3e40003a00 2023-11-08T19:31:43.148 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:43.133+0000 7f3e52ffd700 1 --2- 172.21.15.143:0/3114045121 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e40004850 0x7f3e40004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:43.149 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:43.133+0000 7f3e53fff700 1 --2- 172.21.15.143:0/3114045121 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3e40005210 0x7f3e4000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:43.149 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:43.133+0000 7f3e537fe700 1 --2- 172.21.15.143:0/3114045121 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3e40003a00 0x7f3e40003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:43.149 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:43.134+0000 7f3e52ffd700 1 -- 172.21.15.143:0/3114045121 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3e40003a00 msgr2=0x7f3e40003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:43.149 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:43.134+0000 7f3e52ffd700 1 --2- 172.21.15.143:0/3114045121 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3e40003a00 0x7f3e40003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:43.150 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:43.134+0000 7f3e52ffd700 1 -- 172.21.15.143:0/3114045121 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3e40005210 msgr2=0x7f3e4000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:43.150 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:43.134+0000 7f3e52ffd700 1 --2- 172.21.15.143:0/3114045121 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3e40005210 0x7f3e4000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:43.150 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:43.134+0000 7f3e53fff700 1 -- 172.21.15.143:0/3114045121 reap_dead start 2023-11-08T19:31:43.151 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:43.134+0000 7f3e52ffd700 1 -- 172.21.15.143:0/3114045121 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e400035b0 con 0x7f3e40004850 2023-11-08T19:31:43.151 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:43.134+0000 7f3e52ffd700 1 --2- 172.21.15.143:0/3114045121 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e40004850 0x7f3e40004cd0 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c009b60 tx=0x7f3e40005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a17a849a5f7b20 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:43.151 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:43.134+0000 7f3e527fc700 1 -- 172.21.15.143:0/3114045121 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e3c00e6c0 con 0x7f3e40004850 2023-11-08T19:31:43.151 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:43.134+0000 7f3e527fc700 1 -- 172.21.15.143:0/3114045121 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e3c00e820 con 0x7f3e40004850 2023-11-08T19:31:43.152 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:43.134+0000 7f3e527fc700 1 -- 172.21.15.143:0/3114045121 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e3c00eb90 con 0x7f3e40004850 2023-11-08T19:31:43.152 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:43.134+0000 7f3e5a1cb700 1 -- 172.21.15.143:0/3114045121 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e40004850 msgr2=0x7f3e40004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:43.152 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:43.134+0000 7f3e5a1cb700 1 --2- 172.21.15.143:0/3114045121 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e40004850 0x7f3e40004cd0 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c009b60 tx=0x7f3e40005880 comp rx=0 tx=0).stop 2023-11-08T19:31:43.153 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:43.134+0000 7f3e5a1cb700 1 -- 172.21.15.143:0/3114045121 shutdown_connections 2023-11-08T19:31:43.153 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:43.134+0000 7f3e5a1cb700 1 --2- 172.21.15.143:0/3114045121 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e40004850 0x7f3e40004cd0 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:43.153 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:43.134+0000 7f3e5a1cb700 1 -- 172.21.15.143:0/3114045121 >> 172.21.15.143:0/3114045121 conn(0x7f3e54099ab0 msgr2=0x7f3e540d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:43.153 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:43.135+0000 7f3e5a1cb700 1 -- 172.21.15.143:0/3114045121 shutdown_connections 2023-11-08T19:31:43.154 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:43.135+0000 7f3e5a1cb700 1 -- 172.21.15.143:0/3114045121 wait complete. 2023-11-08T19:31:43.154 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:43.136+0000 7f3e5a1cb700 1 Processor -- start 2023-11-08T19:31:43.154 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:43.136+0000 7f3e5a1cb700 1 -- start start 2023-11-08T19:31:43.155 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:43.137+0000 7f3e5a1cb700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3e40004850 0x7f3e541545b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:43.155 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:43.137+0000 7f3e5a1cb700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3e541f4020 0x7f3e541f4480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:43.155 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:43.137+0000 7f3e5a1cb700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e541f49c0 0x7f3e540b1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:43.155 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:43.137+0000 7f3e5a1cb700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e5415f7d0 con 0x7f3e541f49c0 2023-11-08T19:31:43.156 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:43.137+0000 7f3e5a1cb700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e5415eea0 con 0x7f3e40004850 2023-11-08T19:31:43.156 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:43.137+0000 7f3e5a1cb700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e5415f180 con 0x7f3e541f4020 2023-11-08T19:31:43.156 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:43.137+0000 7f3e53fff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e541f49c0 0x7f3e540b1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:43.157 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:43.137+0000 7f3e53fff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e541f49c0 0x7f3e540b1720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57420/0 (socket says 172.21.15.143:57420) 2023-11-08T19:31:43.157 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:43.137+0000 7f3e53fff700 1 -- 172.21.15.143:0/59061431 learned_addr learned my addr 172.21.15.143:0/59061431 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:31:43.157 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:43.137+0000 7f3e537fe700 1 --2- 172.21.15.143:0/59061431 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3e40004850 0x7f3e541545b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:43.157 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:43.137+0000 7f3e52ffd700 1 --2- 172.21.15.143:0/59061431 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3e541f4020 0x7f3e541f4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:43.158 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:43.137+0000 7f3e53fff700 1 -- 172.21.15.143:0/59061431 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3e541f4020 msgr2=0x7f3e541f4480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:43.158 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:43.137+0000 7f3e53fff700 1 --2- 172.21.15.143:0/59061431 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3e541f4020 0x7f3e541f4480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:43.158 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:43.137+0000 7f3e53fff700 1 -- 172.21.15.143:0/59061431 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3e40004850 msgr2=0x7f3e541545b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:43.159 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:43.137+0000 7f3e53fff700 1 --2- 172.21.15.143:0/59061431 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3e40004850 0x7f3e541545b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:43.159 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:43.137+0000 7f3e53fff700 1 -- 172.21.15.143:0/59061431 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e5415e260 con 0x7f3e541f49c0 2023-11-08T19:31:43.159 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:43.137+0000 7f3e53fff700 1 --2- 172.21.15.143:0/59061431 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e541f49c0 0x7f3e540b1720 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f3e540d0760 tx=0x7f3e4c00fe70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:43.160 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:43.137+0000 7f3e4bfff700 1 -- 172.21.15.143:0/59061431 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e4c003cd0 con 0x7f3e541f49c0 2023-11-08T19:31:43.160 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:43.137+0000 7f3e4bfff700 1 -- 172.21.15.143:0/59061431 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e4c003e30 con 0x7f3e541f49c0 2023-11-08T19:31:43.160 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:43.137+0000 7f3e4bfff700 1 -- 172.21.15.143:0/59061431 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e4c005310 con 0x7f3e541f49c0 2023-11-08T19:31:43.161 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:43.137+0000 7f3e5a1cb700 1 -- 172.21.15.143:0/59061431 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e5415f460 con 0x7f3e541f49c0 2023-11-08T19:31:43.161 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:43.137+0000 7f3e5a1cb700 1 -- 172.21.15.143:0/59061431 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e541f5130 con 0x7f3e541f49c0 2023-11-08T19:31:43.161 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:43.138+0000 7f3e5a1cb700 1 -- 172.21.15.143:0/59061431 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e540b1540 con 0x7f3e541f49c0 2023-11-08T19:31:43.162 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:43.139+0000 7f3e4bfff700 1 -- 172.21.15.143:0/59061431 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3e4c005470 con 0x7f3e541f49c0 2023-11-08T19:31:43.162 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:43.139+0000 7f3e4bfff700 1 --2- 172.21.15.143:0/59061431 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f3e3406af00 0x7f3e3406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:43.162 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:43.139+0000 7f3e4bfff700 1 -- 172.21.15.143:0/59061431 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6651+0+0 (secure 0 0 0) 0x7f3e4c089990 con 0x7f3e541f49c0 2023-11-08T19:31:43.163 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:43.139+0000 7f3e537fe700 1 --2- 172.21.15.143:0/59061431 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f3e3406af00 0x7f3e3406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:43.163 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:43.140+0000 7f3e537fe700 1 --2- 172.21.15.143:0/59061431 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f3e3406af00 0x7f3e3406d3c0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f3e44000ef0 tx=0x7f3e44009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:43.164 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:43.144+0000 7f3e4bfff700 1 -- 172.21.15.143:0/59061431 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3e4c054690 con 0x7f3e541f49c0 2023-11-08T19:31:43.329 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:43.328+0000 7f3e5a1cb700 1 -- 172.21.15.143:0/59061431 --> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3e5415e0e0 con 0x7f3e3406af00 2023-11-08T19:31:43.338 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:43.337+0000 7f3e4bfff700 1 -- 172.21.15.143:0/59061431 <== mgr.4099 v2:172.21.15.174:6832/4037280178 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+292624 (secure 0 0 0) 0x7f3e440127e0 con 0x7f3e3406af00 2023-11-08T19:31:43.343 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:31:43.345 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:43.340+0000 7f3e5a1cb700 1 -- 172.21.15.143:0/59061431 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f3e3406af00 msgr2=0x7f3e3406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:43.346 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:43.340+0000 7f3e5a1cb700 1 --2- 172.21.15.143:0/59061431 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f3e3406af00 0x7f3e3406d3c0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f3e44000ef0 tx=0x7f3e44009040 comp rx=0 tx=0).stop 2023-11-08T19:31:43.346 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:43.340+0000 7f3e5a1cb700 1 -- 172.21.15.143:0/59061431 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e541f49c0 msgr2=0x7f3e540b1720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:43.347 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:43.340+0000 7f3e5a1cb700 1 --2- 172.21.15.143:0/59061431 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e541f49c0 0x7f3e540b1720 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f3e540d0760 tx=0x7f3e4c00fe70 comp rx=0 tx=0).stop 2023-11-08T19:31:43.347 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:43.340+0000 7f3e5a1cb700 1 -- 172.21.15.143:0/59061431 shutdown_connections 2023-11-08T19:31:43.347 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:43.340+0000 7f3e5a1cb700 1 --2- 172.21.15.143:0/59061431 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f3e3406af00 0x7f3e3406d3c0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:43.348 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:43.340+0000 7f3e5a1cb700 1 --2- 172.21.15.143:0/59061431 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e541f49c0 0x7f3e540b1720 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:43.348 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:43.340+0000 7f3e5a1cb700 1 --2- 172.21.15.143:0/59061431 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3e40004850 0x7f3e541545b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:43.348 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:43.340+0000 7f3e5a1cb700 1 --2- 172.21.15.143:0/59061431 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3e541f4020 0x7f3e541f4480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:43.349 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:43.340+0000 7f3e5a1cb700 1 -- 172.21.15.143:0/59061431 >> 172.21.15.143:0/59061431 conn(0x7f3e54099ab0 msgr2=0x7f3e540cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:43.349 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:43.340+0000 7f3e5a1cb700 1 -- 172.21.15.143:0/59061431 shutdown_connections 2023-11-08T19:31:43.349 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:31:43.340+0000 7f3e5a1cb700 1 -- 172.21.15.143:0/59061431 wait complete. 2023-11-08T19:31:43.350 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2023-11-08T19:31:43.353 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":309,"stamp":"2023-11-08T19:31:42.631503+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":244264,"kb_used_data":11496,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754730456,"statfs":{"total":773094113280,"available":772843986944,"internally_reserved":0,"allocated":11771904,"data_stored":6351912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003616"},"pg_stats":[{"pgid":"12.d","version":"0'0","reported_seq":55,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:22.583090+0000","last_change":"2023-11-08T19:27:08.190826+0000","last_active":"2023-11-08T19:31:22.583090+0000","last_peered":"2023-11-08T19:31:22.583090+0000","last_clean":"2023-11-08T19:31:22.583090+0000","last_became_active":"2023-11-08T19:27:08.190440+0000","last_became_peered":"2023-11-08T19:27:08.190440+0000","last_unstale":"2023-11-08T19:31:22.583090+0000","last_undegraded":"2023-11-08T19:31:22.583090+0000","last_fullsized":"2023-11-08T19:31:22.583090+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"107'119","reported_seq":266,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:27.048714+0000","last_change":"2023-11-08T19:31:27.048761+0000","last_active":"2023-11-08T19:31:27.048714+0000","last_peered":"2023-11-08T19:31:27.048714+0000","last_clean":"2023-11-08T19:31:27.048714+0000","last_became_active":"2023-11-08T19:24:45.887048+0000","last_became_peered":"2023-11-08T19:24:45.887048+0000","last_unstale":"2023-11-08T19:31:27.048714+0000","last_undegraded":"2023-11-08T19:31:27.048714+0000","last_fullsized":"2023-11-08T19:31:27.048714+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"107'119","last_scrub_stamp":"2023-11-08T19:31:27.048604+0000","last_deep_scrub":"107'119","last_deep_scrub_stamp":"2023-11-08T19:31:27.048604+0000","last_clean_scrub_stamp":"2023-11-08T19:31:27.048604+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:16:40.175526+0000","scrub_duration":0.0086873920000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":88,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:32.804593+0000","last_change":"2023-11-08T19:31:32.804615+0000","last_active":"2023-11-08T19:31:32.804593+0000","last_peered":"2023-11-08T19:31:32.804593+0000","last_clean":"2023-11-08T19:31:32.804593+0000","last_became_active":"2023-11-08T19:26:04.603056+0000","last_became_peered":"2023-11-08T19:26:04.603056+0000","last_unstale":"2023-11-08T19:31:32.804593+0000","last_undegraded":"2023-11-08T19:31:32.804593+0000","last_fullsized":"2023-11-08T19:31:32.804593+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:32.804495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:32.804495+0000","last_clean_scrub_stamp":"2023-11-08T19:31:32.804495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:16:22.607276+0000","scrub_duration":0.0015962439999999999,"objects_trimmed":0,"snaptrim_duration":0.000517414,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":60,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:20.584175+0000","last_change":"2023-11-08T19:27:09.496842+0000","last_active":"2023-11-08T19:31:20.584175+0000","last_peered":"2023-11-08T19:31:20.584175+0000","last_clean":"2023-11-08T19:31:20.584175+0000","last_became_active":"2023-11-08T19:27:07.185776+0000","last_became_peered":"2023-11-08T19:27:07.185776+0000","last_unstale":"2023-11-08T19:31:20.584175+0000","last_undegraded":"2023-11-08T19:31:20.584175+0000","last_fullsized":"2023-11-08T19:31:20.584175+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031747200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":53,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:20.584206+0000","last_change":"2023-11-08T19:27:08.188241+0000","last_active":"2023-11-08T19:31:20.584206+0000","last_peered":"2023-11-08T19:31:20.584206+0000","last_clean":"2023-11-08T19:31:20.584206+0000","last_became_active":"2023-11-08T19:27:08.187863+0000","last_became_peered":"2023-11-08T19:27:08.187863+0000","last_unstale":"2023-11-08T19:31:20.584206+0000","last_undegraded":"2023-11-08T19:31:20.584206+0000","last_fullsized":"2023-11-08T19:31:20.584206+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":78,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:14.012288+0000","last_change":"2023-11-08T19:26:06.265419+0000","last_active":"2023-11-08T19:31:14.012288+0000","last_peered":"2023-11-08T19:31:14.012288+0000","last_clean":"2023-11-08T19:31:14.012288+0000","last_became_active":"2023-11-08T19:26:04.605333+0000","last_became_peered":"2023-11-08T19:26:04.605333+0000","last_unstale":"2023-11-08T19:31:14.012288+0000","last_undegraded":"2023-11-08T19:31:14.012288+0000","last_fullsized":"2023-11-08T19:31:14.012288+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064606400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":62,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:22.583048+0000","last_change":"2023-11-08T19:27:09.182113+0000","last_active":"2023-11-08T19:31:22.583048+0000","last_peered":"2023-11-08T19:31:22.583048+0000","last_clean":"2023-11-08T19:31:22.583048+0000","last_became_active":"2023-11-08T19:27:07.189939+0000","last_became_peered":"2023-11-08T19:27:07.189939+0000","last_unstale":"2023-11-08T19:31:22.583048+0000","last_undegraded":"2023-11-08T19:31:22.583048+0000","last_fullsized":"2023-11-08T19:31:22.583048+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036507500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":49,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:20.086140+0000","last_change":"2023-11-08T19:27:08.187127+0000","last_active":"2023-11-08T19:31:20.086140+0000","last_peered":"2023-11-08T19:31:20.086140+0000","last_clean":"2023-11-08T19:31:20.086140+0000","last_became_active":"2023-11-08T19:27:08.182648+0000","last_became_peered":"2023-11-08T19:27:08.182648+0000","last_unstale":"2023-11-08T19:31:20.086140+0000","last_undegraded":"2023-11-08T19:31:20.086140+0000","last_fullsized":"2023-11-08T19:31:20.086140+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":90,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:14.020006+0000","last_change":"2023-11-08T19:26:05.609570+0000","last_active":"2023-11-08T19:31:14.020006+0000","last_peered":"2023-11-08T19:31:14.020006+0000","last_clean":"2023-11-08T19:31:14.020006+0000","last_became_active":"2023-11-08T19:26:04.608379+0000","last_became_peered":"2023-11-08T19:26:04.608379+0000","last_unstale":"2023-11-08T19:31:14.020006+0000","last_undegraded":"2023-11-08T19:31:14.020006+0000","last_fullsized":"2023-11-08T19:31:14.020006+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000529657,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":56,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:21.082859+0000","last_change":"2023-11-08T19:27:09.211991+0000","last_active":"2023-11-08T19:31:21.082859+0000","last_peered":"2023-11-08T19:31:21.082859+0000","last_clean":"2023-11-08T19:31:21.082859+0000","last_became_active":"2023-11-08T19:27:07.195231+0000","last_became_peered":"2023-11-08T19:27:07.195231+0000","last_unstale":"2023-11-08T19:31:21.082859+0000","last_undegraded":"2023-11-08T19:31:21.082859+0000","last_fullsized":"2023-11-08T19:31:21.082859+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000441221,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":49,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:20.086078+0000","last_change":"2023-11-08T19:27:08.195364+0000","last_active":"2023-11-08T19:31:20.086078+0000","last_peered":"2023-11-08T19:31:20.086078+0000","last_clean":"2023-11-08T19:31:20.086078+0000","last_became_active":"2023-11-08T19:27:08.195074+0000","last_became_peered":"2023-11-08T19:27:08.195074+0000","last_unstale":"2023-11-08T19:31:20.086078+0000","last_undegraded":"2023-11-08T19:31:20.086078+0000","last_fullsized":"2023-11-08T19:31:20.086078+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":86,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:29.665197+0000","last_change":"2023-11-08T19:31:29.665227+0000","last_active":"2023-11-08T19:31:29.665197+0000","last_peered":"2023-11-08T19:31:29.665197+0000","last_clean":"2023-11-08T19:31:29.665197+0000","last_became_active":"2023-11-08T19:26:04.601685+0000","last_became_peered":"2023-11-08T19:26:04.601685+0000","last_unstale":"2023-11-08T19:31:29.665197+0000","last_undegraded":"2023-11-08T19:31:29.665197+0000","last_fullsized":"2023-11-08T19:31:29.665197+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:29.665096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:29.665096+0000","last_clean_scrub_stamp":"2023-11-08T19:31:29.665096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:57:56.435973+0000","scrub_duration":0.001580171,"objects_trimmed":0,"snaptrim_duration":0.00061044099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":64,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:34.809538+0000","last_change":"2023-11-08T19:31:34.809538+0000","last_active":"2023-11-08T19:31:34.809538+0000","last_peered":"2023-11-08T19:31:34.809538+0000","last_clean":"2023-11-08T19:31:34.809538+0000","last_became_active":"2023-11-08T19:27:07.189386+0000","last_became_peered":"2023-11-08T19:27:07.189386+0000","last_unstale":"2023-11-08T19:31:34.809538+0000","last_undegraded":"2023-11-08T19:31:34.809538+0000","last_fullsized":"2023-11-08T19:31:34.809538+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:34.809370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:34.809370+0000","last_clean_scrub_stamp":"2023-11-08T19:31:34.809370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:04:11.405772+0000","scrub_duration":0.0021291940000000001,"objects_trimmed":0,"snaptrim_duration":0.00049390100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":53,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:20.584235+0000","last_change":"2023-11-08T19:27:08.189452+0000","last_active":"2023-11-08T19:31:20.584235+0000","last_peered":"2023-11-08T19:31:20.584235+0000","last_clean":"2023-11-08T19:31:20.584235+0000","last_became_active":"2023-11-08T19:27:08.189088+0000","last_became_peered":"2023-11-08T19:27:08.189088+0000","last_unstale":"2023-11-08T19:31:20.584235+0000","last_undegraded":"2023-11-08T19:31:20.584235+0000","last_fullsized":"2023-11-08T19:31:20.584235+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":87,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:21.041246+0000","last_change":"2023-11-08T19:31:21.041265+0000","last_active":"2023-11-08T19:31:21.041246+0000","last_peered":"2023-11-08T19:31:21.041246+0000","last_clean":"2023-11-08T19:31:21.041246+0000","last_became_active":"2023-11-08T19:26:04.603088+0000","last_became_peered":"2023-11-08T19:26:04.603088+0000","last_unstale":"2023-11-08T19:31:21.041246+0000","last_undegraded":"2023-11-08T19:31:21.041246+0000","last_fullsized":"2023-11-08T19:31:21.041246+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:21.041166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:21.041166+0000","last_clean_scrub_stamp":"2023-11-08T19:31:21.041166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:25:28.147920+0000","scrub_duration":0.001667876,"objects_trimmed":0,"snaptrim_duration":0.00038180899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":64,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:28.086650+0000","last_change":"2023-11-08T19:31:28.086650+0000","last_active":"2023-11-08T19:31:28.086650+0000","last_peered":"2023-11-08T19:31:28.086650+0000","last_clean":"2023-11-08T19:31:28.086650+0000","last_became_active":"2023-11-08T19:27:07.191589+0000","last_became_peered":"2023-11-08T19:27:07.191589+0000","last_unstale":"2023-11-08T19:31:28.086650+0000","last_undegraded":"2023-11-08T19:31:28.086650+0000","last_fullsized":"2023-11-08T19:31:28.086650+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:28.086495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:28.086495+0000","last_clean_scrub_stamp":"2023-11-08T19:31:28.086495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:17:02.712148+0000","scrub_duration":0.0016141980000000001,"objects_trimmed":0,"snaptrim_duration":0.000729527,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":49,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:19.088760+0000","last_change":"2023-11-08T19:27:08.193337+0000","last_active":"2023-11-08T19:31:19.088760+0000","last_peered":"2023-11-08T19:31:19.088760+0000","last_clean":"2023-11-08T19:31:19.088760+0000","last_became_active":"2023-11-08T19:27:08.192997+0000","last_became_peered":"2023-11-08T19:27:08.192997+0000","last_unstale":"2023-11-08T19:31:19.088760+0000","last_undegraded":"2023-11-08T19:31:19.088760+0000","last_fullsized":"2023-11-08T19:31:19.088760+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":84,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:14.218333+0000","last_change":"2023-11-08T19:26:06.424511+0000","last_active":"2023-11-08T19:31:14.218333+0000","last_peered":"2023-11-08T19:31:14.218333+0000","last_clean":"2023-11-08T19:31:14.218333+0000","last_became_active":"2023-11-08T19:26:04.598973+0000","last_became_peered":"2023-11-08T19:26:04.598973+0000","last_unstale":"2023-11-08T19:31:14.218333+0000","last_undegraded":"2023-11-08T19:31:14.218333+0000","last_fullsized":"2023-11-08T19:31:14.218333+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031755900000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":60,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:20.584212+0000","last_change":"2023-11-08T19:27:09.497024+0000","last_active":"2023-11-08T19:31:20.584212+0000","last_peered":"2023-11-08T19:31:20.584212+0000","last_clean":"2023-11-08T19:31:20.584212+0000","last_became_active":"2023-11-08T19:27:07.190132+0000","last_became_peered":"2023-11-08T19:27:07.190132+0000","last_unstale":"2023-11-08T19:31:20.584212+0000","last_undegraded":"2023-11-08T19:31:20.584212+0000","last_fullsized":"2023-11-08T19:31:20.584212+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054126000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":49,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:19.088768+0000","last_change":"2023-11-08T19:27:08.194196+0000","last_active":"2023-11-08T19:31:19.088768+0000","last_peered":"2023-11-08T19:31:19.088768+0000","last_clean":"2023-11-08T19:31:19.088768+0000","last_became_active":"2023-11-08T19:27:08.193912+0000","last_became_peered":"2023-11-08T19:27:08.193912+0000","last_unstale":"2023-11-08T19:31:19.088768+0000","last_undegraded":"2023-11-08T19:31:19.088768+0000","last_fullsized":"2023-11-08T19:31:19.088768+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":87,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:22.069061+0000","last_change":"2023-11-08T19:31:22.069101+0000","last_active":"2023-11-08T19:31:22.069061+0000","last_peered":"2023-11-08T19:31:22.069061+0000","last_clean":"2023-11-08T19:31:22.069061+0000","last_became_active":"2023-11-08T19:26:04.599999+0000","last_became_peered":"2023-11-08T19:26:04.599999+0000","last_unstale":"2023-11-08T19:31:22.069061+0000","last_undegraded":"2023-11-08T19:31:22.069061+0000","last_fullsized":"2023-11-08T19:31:22.069061+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:22.068925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:22.068925+0000","last_clean_scrub_stamp":"2023-11-08T19:31:22.068925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:59:28.785563+0000","scrub_duration":0.0018102260000000001,"objects_trimmed":0,"snaptrim_duration":0.00025605700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":60,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:20.584140+0000","last_change":"2023-11-08T19:27:09.497040+0000","last_active":"2023-11-08T19:31:20.584140+0000","last_peered":"2023-11-08T19:31:20.584140+0000","last_clean":"2023-11-08T19:31:20.584140+0000","last_became_active":"2023-11-08T19:27:07.189135+0000","last_became_peered":"2023-11-08T19:27:07.189135+0000","last_unstale":"2023-11-08T19:31:20.584140+0000","last_undegraded":"2023-11-08T19:31:20.584140+0000","last_fullsized":"2023-11-08T19:31:20.584140+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051794299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":63,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:34.286026+0000","last_change":"2023-11-08T19:31:34.286026+0000","last_active":"2023-11-08T19:31:34.286026+0000","last_peered":"2023-11-08T19:31:34.286026+0000","last_clean":"2023-11-08T19:31:34.286026+0000","last_became_active":"2023-11-08T19:27:08.186643+0000","last_became_peered":"2023-11-08T19:27:08.186643+0000","last_unstale":"2023-11-08T19:31:34.286026+0000","last_undegraded":"2023-11-08T19:31:34.286026+0000","last_fullsized":"2023-11-08T19:31:34.286026+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:34.285872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:34.285872+0000","last_clean_scrub_stamp":"2023-11-08T19:31:34.285872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:22:31.137393+0000","scrub_duration":0.0015767699999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":90,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:14.017179+0000","last_change":"2023-11-08T19:26:05.609824+0000","last_active":"2023-11-08T19:31:14.017179+0000","last_peered":"2023-11-08T19:31:14.017179+0000","last_clean":"2023-11-08T19:31:14.017179+0000","last_became_active":"2023-11-08T19:26:04.603320+0000","last_became_peered":"2023-11-08T19:26:04.603320+0000","last_unstale":"2023-11-08T19:31:14.017179+0000","last_undegraded":"2023-11-08T19:31:14.017179+0000","last_fullsized":"2023-11-08T19:31:14.017179+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063225,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":60,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:20.584194+0000","last_change":"2023-11-08T19:27:09.496655+0000","last_active":"2023-11-08T19:31:20.584194+0000","last_peered":"2023-11-08T19:31:20.584194+0000","last_clean":"2023-11-08T19:31:20.584194+0000","last_became_active":"2023-11-08T19:27:07.189895+0000","last_became_peered":"2023-11-08T19:27:07.189895+0000","last_unstale":"2023-11-08T19:31:20.584194+0000","last_undegraded":"2023-11-08T19:31:20.584194+0000","last_fullsized":"2023-11-08T19:31:20.584194+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045654199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":88,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:24.725788+0000","last_change":"2023-11-08T19:31:24.725829+0000","last_active":"2023-11-08T19:31:24.725788+0000","last_peered":"2023-11-08T19:31:24.725788+0000","last_clean":"2023-11-08T19:31:24.725788+0000","last_became_active":"2023-11-08T19:26:04.603725+0000","last_became_peered":"2023-11-08T19:26:04.603725+0000","last_unstale":"2023-11-08T19:31:24.725788+0000","last_undegraded":"2023-11-08T19:31:24.725788+0000","last_fullsized":"2023-11-08T19:31:24.725788+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:24.725673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:24.725673+0000","last_clean_scrub_stamp":"2023-11-08T19:31:24.725673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:15:54.517242+0000","scrub_duration":0.00144779,"objects_trimmed":0,"snaptrim_duration":0.00053365400000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":64,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:35.727927+0000","last_change":"2023-11-08T19:31:35.727950+0000","last_active":"2023-11-08T19:31:35.727927+0000","last_peered":"2023-11-08T19:31:35.727927+0000","last_clean":"2023-11-08T19:31:35.727927+0000","last_became_active":"2023-11-08T19:27:07.189595+0000","last_became_peered":"2023-11-08T19:27:07.189595+0000","last_unstale":"2023-11-08T19:31:35.727927+0000","last_undegraded":"2023-11-08T19:31:35.727927+0000","last_fullsized":"2023-11-08T19:31:35.727927+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:35.727812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:35.727812+0000","last_clean_scrub_stamp":"2023-11-08T19:31:35.727812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:00:59.786393+0000","scrub_duration":0.0019661430000000001,"objects_trimmed":0,"snaptrim_duration":0.000323479,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":49,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:14.011635+0000","last_change":"2023-11-08T19:27:08.185395+0000","last_active":"2023-11-08T19:31:14.011635+0000","last_peered":"2023-11-08T19:31:14.011635+0000","last_clean":"2023-11-08T19:31:14.011635+0000","last_became_active":"2023-11-08T19:27:08.184765+0000","last_became_peered":"2023-11-08T19:27:08.184765+0000","last_unstale":"2023-11-08T19:31:14.011635+0000","last_undegraded":"2023-11-08T19:31:14.011635+0000","last_fullsized":"2023-11-08T19:31:14.011635+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":92,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:32.544440+0000","last_change":"2023-11-08T19:31:32.544462+0000","last_active":"2023-11-08T19:31:32.544440+0000","last_peered":"2023-11-08T19:31:32.544440+0000","last_clean":"2023-11-08T19:31:32.544440+0000","last_became_active":"2023-11-08T19:26:04.600115+0000","last_became_peered":"2023-11-08T19:26:04.600115+0000","last_unstale":"2023-11-08T19:31:32.544440+0000","last_undegraded":"2023-11-08T19:31:32.544440+0000","last_fullsized":"2023-11-08T19:31:32.544440+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:32.544303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:32.544303+0000","last_clean_scrub_stamp":"2023-11-08T19:31:32.544303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:35:56.347935+0000","scrub_duration":0.002175448,"objects_trimmed":0,"snaptrim_duration":0.00065803900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":56,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:19.088716+0000","last_change":"2023-11-08T19:27:09.331899+0000","last_active":"2023-11-08T19:31:19.088716+0000","last_peered":"2023-11-08T19:31:19.088716+0000","last_clean":"2023-11-08T19:31:19.088716+0000","last_became_active":"2023-11-08T19:27:07.191460+0000","last_became_peered":"2023-11-08T19:27:07.191460+0000","last_unstale":"2023-11-08T19:31:19.088716+0000","last_undegraded":"2023-11-08T19:31:19.088716+0000","last_fullsized":"2023-11-08T19:31:19.088716+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044907500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":55,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:19.628625+0000","last_change":"2023-11-08T19:27:08.191802+0000","last_active":"2023-11-08T19:31:19.628625+0000","last_peered":"2023-11-08T19:31:19.628625+0000","last_clean":"2023-11-08T19:31:19.628625+0000","last_became_active":"2023-11-08T19:27:08.191520+0000","last_became_peered":"2023-11-08T19:27:08.191520+0000","last_unstale":"2023-11-08T19:31:19.628625+0000","last_undegraded":"2023-11-08T19:31:19.628625+0000","last_fullsized":"2023-11-08T19:31:19.628625+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":98,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:29.015607+0000","last_change":"2023-11-08T19:31:29.015630+0000","last_active":"2023-11-08T19:31:29.015607+0000","last_peered":"2023-11-08T19:31:29.015607+0000","last_clean":"2023-11-08T19:31:29.015607+0000","last_became_active":"2023-11-08T19:26:04.604519+0000","last_became_peered":"2023-11-08T19:26:04.604519+0000","last_unstale":"2023-11-08T19:31:29.015607+0000","last_undegraded":"2023-11-08T19:31:29.015607+0000","last_fullsized":"2023-11-08T19:31:29.015607+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:29.015490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:29.015490+0000","last_clean_scrub_stamp":"2023-11-08T19:31:29.015490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:35:52.819122+0000","scrub_duration":0.0020686609999999999,"objects_trimmed":0,"snaptrim_duration":0.00051248099999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":62,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:19.628717+0000","last_change":"2023-11-08T19:27:09.181431+0000","last_active":"2023-11-08T19:31:19.628717+0000","last_peered":"2023-11-08T19:31:19.628717+0000","last_clean":"2023-11-08T19:31:19.628717+0000","last_became_active":"2023-11-08T19:27:07.190800+0000","last_became_peered":"2023-11-08T19:27:07.190800+0000","last_unstale":"2023-11-08T19:31:19.628717+0000","last_undegraded":"2023-11-08T19:31:19.628717+0000","last_fullsized":"2023-11-08T19:31:19.628717+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656099999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":49,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:18.639811+0000","last_change":"2023-11-08T19:27:08.187983+0000","last_active":"2023-11-08T19:31:18.639811+0000","last_peered":"2023-11-08T19:31:18.639811+0000","last_clean":"2023-11-08T19:31:18.639811+0000","last_became_active":"2023-11-08T19:27:08.187322+0000","last_became_peered":"2023-11-08T19:27:08.187322+0000","last_unstale":"2023-11-08T19:31:18.639811+0000","last_undegraded":"2023-11-08T19:31:18.639811+0000","last_fullsized":"2023-11-08T19:31:18.639811+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":86,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:22.798579+0000","last_change":"2023-11-08T19:31:22.798598+0000","last_active":"2023-11-08T19:31:22.798579+0000","last_peered":"2023-11-08T19:31:22.798579+0000","last_clean":"2023-11-08T19:31:22.798579+0000","last_became_active":"2023-11-08T19:26:04.606967+0000","last_became_peered":"2023-11-08T19:26:04.606967+0000","last_unstale":"2023-11-08T19:31:22.798579+0000","last_undegraded":"2023-11-08T19:31:22.798579+0000","last_fullsized":"2023-11-08T19:31:22.798579+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:22.798496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:22.798496+0000","last_clean_scrub_stamp":"2023-11-08T19:31:22.798496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:28:48.162603+0000","scrub_duration":0.002120331,"objects_trimmed":0,"snaptrim_duration":0.00040971999999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":62,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:19.628658+0000","last_change":"2023-11-08T19:27:09.180722+0000","last_active":"2023-11-08T19:31:19.628658+0000","last_peered":"2023-11-08T19:31:19.628658+0000","last_clean":"2023-11-08T19:31:19.628658+0000","last_became_active":"2023-11-08T19:27:07.186977+0000","last_became_peered":"2023-11-08T19:27:07.186977+0000","last_unstale":"2023-11-08T19:31:19.628658+0000","last_undegraded":"2023-11-08T19:31:19.628658+0000","last_fullsized":"2023-11-08T19:31:19.628658+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057675100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":49,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:18.636423+0000","last_change":"2023-11-08T19:27:08.193963+0000","last_active":"2023-11-08T19:31:18.636423+0000","last_peered":"2023-11-08T19:31:18.636423+0000","last_clean":"2023-11-08T19:31:18.636423+0000","last_became_active":"2023-11-08T19:27:08.193098+0000","last_became_peered":"2023-11-08T19:27:08.193098+0000","last_unstale":"2023-11-08T19:31:18.636423+0000","last_undegraded":"2023-11-08T19:31:18.636423+0000","last_fullsized":"2023-11-08T19:31:18.636423+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":86,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:33.077049+0000","last_change":"2023-11-08T19:31:33.077074+0000","last_active":"2023-11-08T19:31:33.077049+0000","last_peered":"2023-11-08T19:31:33.077049+0000","last_clean":"2023-11-08T19:31:33.077049+0000","last_became_active":"2023-11-08T19:26:04.602111+0000","last_became_peered":"2023-11-08T19:26:04.602111+0000","last_unstale":"2023-11-08T19:31:33.077049+0000","last_undegraded":"2023-11-08T19:31:33.077049+0000","last_fullsized":"2023-11-08T19:31:33.077049+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:33.076926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:33.076926+0000","last_clean_scrub_stamp":"2023-11-08T19:31:33.076926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:09:46.810104+0000","scrub_duration":0.0022106909999999999,"objects_trimmed":0,"snaptrim_duration":0.00056197899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":70,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:22.258974+0000","last_change":"2023-11-08T19:31:22.259009+0000","last_active":"2023-11-08T19:31:22.258974+0000","last_peered":"2023-11-08T19:31:22.258974+0000","last_clean":"2023-11-08T19:31:22.258974+0000","last_became_active":"2023-11-08T19:27:07.188657+0000","last_became_peered":"2023-11-08T19:27:07.188657+0000","last_unstale":"2023-11-08T19:31:22.258974+0000","last_undegraded":"2023-11-08T19:31:22.258974+0000","last_fullsized":"2023-11-08T19:31:22.258974+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:22.258851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:22.258851+0000","last_clean_scrub_stamp":"2023-11-08T19:31:22.258851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:41:50.529438+0000","scrub_duration":0.001495632,"objects_trimmed":0,"snaptrim_duration":0.001227393,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":63,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:31.191488+0000","last_change":"2023-11-08T19:31:31.191513+0000","last_active":"2023-11-08T19:31:31.191488+0000","last_peered":"2023-11-08T19:31:31.191488+0000","last_clean":"2023-11-08T19:31:31.191488+0000","last_became_active":"2023-11-08T19:27:08.188240+0000","last_became_peered":"2023-11-08T19:27:08.188240+0000","last_unstale":"2023-11-08T19:31:31.191488+0000","last_undegraded":"2023-11-08T19:31:31.191488+0000","last_fullsized":"2023-11-08T19:31:31.191488+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:31.191371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:31.191371+0000","last_clean_scrub_stamp":"2023-11-08T19:31:31.191371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:35:54.995003+0000","scrub_duration":0.001875533,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":63,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:39.374627+0000","last_change":"2023-11-08T19:31:39.374649+0000","last_active":"2023-11-08T19:31:39.374627+0000","last_peered":"2023-11-08T19:31:39.374627+0000","last_clean":"2023-11-08T19:31:39.374627+0000","last_became_active":"2023-11-08T19:27:08.187880+0000","last_became_peered":"2023-11-08T19:27:08.187880+0000","last_unstale":"2023-11-08T19:31:39.374627+0000","last_undegraded":"2023-11-08T19:31:39.374627+0000","last_fullsized":"2023-11-08T19:31:39.374627+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:39.374517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:39.374517+0000","last_clean_scrub_stamp":"2023-11-08T19:31:39.374517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:00:01.282223+0000","scrub_duration":0.0021558079999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":87,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:27.808067+0000","last_change":"2023-11-08T19:31:27.808093+0000","last_active":"2023-11-08T19:31:27.808067+0000","last_peered":"2023-11-08T19:31:27.808067+0000","last_clean":"2023-11-08T19:31:27.808067+0000","last_became_active":"2023-11-08T19:26:04.603955+0000","last_became_peered":"2023-11-08T19:26:04.603955+0000","last_unstale":"2023-11-08T19:31:27.808067+0000","last_undegraded":"2023-11-08T19:31:27.808067+0000","last_fullsized":"2023-11-08T19:31:27.808067+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:27.807939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:27.807939+0000","last_clean_scrub_stamp":"2023-11-08T19:31:27.807939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:20:39.173012+0000","scrub_duration":0.0016595620000000001,"objects_trimmed":0,"snaptrim_duration":0.00036473499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":56,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:18.637055+0000","last_change":"2023-11-08T19:27:09.178770+0000","last_active":"2023-11-08T19:31:18.637055+0000","last_peered":"2023-11-08T19:31:18.637055+0000","last_clean":"2023-11-08T19:31:18.637055+0000","last_became_active":"2023-11-08T19:27:07.194992+0000","last_became_peered":"2023-11-08T19:27:07.194992+0000","last_unstale":"2023-11-08T19:31:18.637055+0000","last_undegraded":"2023-11-08T19:31:18.637055+0000","last_fullsized":"2023-11-08T19:31:18.637055+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000770285,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":49,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:19.586621+0000","last_change":"2023-11-08T19:27:08.193953+0000","last_active":"2023-11-08T19:31:19.586621+0000","last_peered":"2023-11-08T19:31:19.586621+0000","last_clean":"2023-11-08T19:31:19.586621+0000","last_became_active":"2023-11-08T19:27:08.193653+0000","last_became_peered":"2023-11-08T19:27:08.193653+0000","last_unstale":"2023-11-08T19:31:19.586621+0000","last_undegraded":"2023-11-08T19:31:19.586621+0000","last_fullsized":"2023-11-08T19:31:19.586621+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":92,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:33.590064+0000","last_change":"2023-11-08T19:31:33.590087+0000","last_active":"2023-11-08T19:31:33.590064+0000","last_peered":"2023-11-08T19:31:33.590064+0000","last_clean":"2023-11-08T19:31:33.590064+0000","last_became_active":"2023-11-08T19:26:04.603144+0000","last_became_peered":"2023-11-08T19:26:04.603144+0000","last_unstale":"2023-11-08T19:31:33.590064+0000","last_undegraded":"2023-11-08T19:31:33.590064+0000","last_fullsized":"2023-11-08T19:31:33.590064+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:33.589962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:33.589962+0000","last_clean_scrub_stamp":"2023-11-08T19:31:33.589962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:22:30.441483+0000","scrub_duration":0.0016553749999999999,"objects_trimmed":0,"snaptrim_duration":0.00054090799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":70,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:26.196071+0000","last_change":"2023-11-08T19:31:26.196111+0000","last_active":"2023-11-08T19:31:26.196071+0000","last_peered":"2023-11-08T19:31:26.196071+0000","last_clean":"2023-11-08T19:31:26.196071+0000","last_became_active":"2023-11-08T19:27:07.190438+0000","last_became_peered":"2023-11-08T19:27:07.190438+0000","last_unstale":"2023-11-08T19:31:26.196071+0000","last_undegraded":"2023-11-08T19:31:26.196071+0000","last_fullsized":"2023-11-08T19:31:26.196071+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:26.195915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:26.195915+0000","last_clean_scrub_stamp":"2023-11-08T19:31:26.195915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:09:39.929093+0000","scrub_duration":0.0021248539999999998,"objects_trimmed":0,"snaptrim_duration":0.00050893300000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":49,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:19.586689+0000","last_change":"2023-11-08T19:27:08.190530+0000","last_active":"2023-11-08T19:31:19.586689+0000","last_peered":"2023-11-08T19:31:19.586689+0000","last_clean":"2023-11-08T19:31:19.586689+0000","last_became_active":"2023-11-08T19:27:08.190203+0000","last_became_peered":"2023-11-08T19:27:08.190203+0000","last_unstale":"2023-11-08T19:31:19.586689+0000","last_undegraded":"2023-11-08T19:31:19.586689+0000","last_fullsized":"2023-11-08T19:31:19.586689+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":88,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:26.732249+0000","last_change":"2023-11-08T19:31:26.732275+0000","last_active":"2023-11-08T19:31:26.732249+0000","last_peered":"2023-11-08T19:31:26.732249+0000","last_clean":"2023-11-08T19:31:26.732249+0000","last_became_active":"2023-11-08T19:26:04.601961+0000","last_became_peered":"2023-11-08T19:26:04.601961+0000","last_unstale":"2023-11-08T19:31:26.732249+0000","last_undegraded":"2023-11-08T19:31:26.732249+0000","last_fullsized":"2023-11-08T19:31:26.732249+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:26.732141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:26.732141+0000","last_clean_scrub_stamp":"2023-11-08T19:31:26.732141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:25:33.838895+0000","scrub_duration":0.0020533159999999999,"objects_trimmed":0,"snaptrim_duration":0.000466386,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":64,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:26.031920+0000","last_change":"2023-11-08T19:31:26.031963+0000","last_active":"2023-11-08T19:31:26.031920+0000","last_peered":"2023-11-08T19:31:26.031920+0000","last_clean":"2023-11-08T19:31:26.031920+0000","last_became_active":"2023-11-08T19:27:07.191583+0000","last_became_peered":"2023-11-08T19:27:07.191583+0000","last_unstale":"2023-11-08T19:31:26.031920+0000","last_undegraded":"2023-11-08T19:31:26.031920+0000","last_fullsized":"2023-11-08T19:31:26.031920+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:26.031797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:26.031797+0000","last_clean_scrub_stamp":"2023-11-08T19:31:26.031797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:09:39.764974+0000","scrub_duration":0.0015837099999999999,"objects_trimmed":0,"snaptrim_duration":0.00093695200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":56,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:29.050883+0000","last_change":"2023-11-08T19:31:29.050922+0000","last_active":"2023-11-08T19:31:29.050883+0000","last_peered":"2023-11-08T19:31:29.050883+0000","last_clean":"2023-11-08T19:31:29.050883+0000","last_became_active":"2023-11-08T19:27:08.187626+0000","last_became_peered":"2023-11-08T19:27:08.187626+0000","last_unstale":"2023-11-08T19:31:29.050883+0000","last_undegraded":"2023-11-08T19:31:29.050883+0000","last_fullsized":"2023-11-08T19:31:29.050883+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:29.050774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:29.050774+0000","last_clean_scrub_stamp":"2023-11-08T19:31:29.050774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:58:25.326897+0000","scrub_duration":0.001829125,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":90,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:14.017427+0000","last_change":"2023-11-08T19:26:05.608980+0000","last_active":"2023-11-08T19:31:14.017427+0000","last_peered":"2023-11-08T19:31:14.017427+0000","last_clean":"2023-11-08T19:31:14.017427+0000","last_became_active":"2023-11-08T19:26:04.606386+0000","last_became_peered":"2023-11-08T19:26:04.606386+0000","last_unstale":"2023-11-08T19:31:14.017427+0000","last_undegraded":"2023-11-08T19:31:14.017427+0000","last_fullsized":"2023-11-08T19:31:14.017427+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037950599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":64,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:36.795896+0000","last_change":"2023-11-08T19:31:36.795921+0000","last_active":"2023-11-08T19:31:36.795896+0000","last_peered":"2023-11-08T19:31:36.795896+0000","last_clean":"2023-11-08T19:31:36.795896+0000","last_became_active":"2023-11-08T19:27:07.195033+0000","last_became_peered":"2023-11-08T19:27:07.195033+0000","last_unstale":"2023-11-08T19:31:36.795896+0000","last_undegraded":"2023-11-08T19:31:36.795896+0000","last_fullsized":"2023-11-08T19:31:36.795896+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:36.795786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:36.795786+0000","last_clean_scrub_stamp":"2023-11-08T19:31:36.795786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:34:05.177568+0000","scrub_duration":0.0014630890000000001,"objects_trimmed":0,"snaptrim_duration":0.00041824800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":48,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:14.216620+0000","last_change":"2023-11-08T19:27:08.193761+0000","last_active":"2023-11-08T19:31:14.216620+0000","last_peered":"2023-11-08T19:31:14.216620+0000","last_clean":"2023-11-08T19:31:14.216620+0000","last_became_active":"2023-11-08T19:27:08.193017+0000","last_became_peered":"2023-11-08T19:27:08.193017+0000","last_unstale":"2023-11-08T19:31:14.216620+0000","last_undegraded":"2023-11-08T19:31:14.216620+0000","last_fullsized":"2023-11-08T19:31:14.216620+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":92,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:37.663409+0000","last_change":"2023-11-08T19:31:37.663435+0000","last_active":"2023-11-08T19:31:37.663409+0000","last_peered":"2023-11-08T19:31:37.663409+0000","last_clean":"2023-11-08T19:31:37.663409+0000","last_became_active":"2023-11-08T19:26:04.599046+0000","last_became_peered":"2023-11-08T19:26:04.599046+0000","last_unstale":"2023-11-08T19:31:37.663409+0000","last_undegraded":"2023-11-08T19:31:37.663409+0000","last_fullsized":"2023-11-08T19:31:37.663409+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:37.663280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:37.663280+0000","last_clean_scrub_stamp":"2023-11-08T19:31:37.663280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:20:56.015718+0000","scrub_duration":0.0015101190000000001,"objects_trimmed":0,"snaptrim_duration":0.00035092199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":62,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:14.017969+0000","last_change":"2023-11-08T19:27:09.181716+0000","last_active":"2023-11-08T19:31:14.017969+0000","last_peered":"2023-11-08T19:31:14.017969+0000","last_clean":"2023-11-08T19:31:14.017969+0000","last_became_active":"2023-11-08T19:27:07.187077+0000","last_became_peered":"2023-11-08T19:27:07.187077+0000","last_unstale":"2023-11-08T19:31:14.017969+0000","last_undegraded":"2023-11-08T19:31:14.017969+0000","last_fullsized":"2023-11-08T19:31:14.017969+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056673599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":48,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:14.012488+0000","last_change":"2023-11-08T19:27:08.188143+0000","last_active":"2023-11-08T19:31:14.012488+0000","last_peered":"2023-11-08T19:31:14.012488+0000","last_clean":"2023-11-08T19:31:14.012488+0000","last_became_active":"2023-11-08T19:27:08.187714+0000","last_became_peered":"2023-11-08T19:27:08.187714+0000","last_unstale":"2023-11-08T19:31:14.012488+0000","last_undegraded":"2023-11-08T19:31:14.012488+0000","last_fullsized":"2023-11-08T19:31:14.012488+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":86,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:35.500759+0000","last_change":"2023-11-08T19:31:35.500784+0000","last_active":"2023-11-08T19:31:35.500759+0000","last_peered":"2023-11-08T19:31:35.500759+0000","last_clean":"2023-11-08T19:31:35.500759+0000","last_became_active":"2023-11-08T19:26:04.603212+0000","last_became_peered":"2023-11-08T19:26:04.603212+0000","last_unstale":"2023-11-08T19:31:35.500759+0000","last_undegraded":"2023-11-08T19:31:35.500759+0000","last_fullsized":"2023-11-08T19:31:35.500759+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:35.500649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:35.500649+0000","last_clean_scrub_stamp":"2023-11-08T19:31:35.500649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:15:40.433937+0000","scrub_duration":0.0018424209999999999,"objects_trimmed":0,"snaptrim_duration":0.00054695399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":61,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:14.016997+0000","last_change":"2023-11-08T19:27:09.181613+0000","last_active":"2023-11-08T19:31:14.016997+0000","last_peered":"2023-11-08T19:31:14.016997+0000","last_clean":"2023-11-08T19:31:14.016997+0000","last_became_active":"2023-11-08T19:27:07.190897+0000","last_became_peered":"2023-11-08T19:27:07.190897+0000","last_unstale":"2023-11-08T19:31:14.016997+0000","last_undegraded":"2023-11-08T19:31:14.016997+0000","last_fullsized":"2023-11-08T19:31:14.016997+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011172459999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":48,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:14.013099+0000","last_change":"2023-11-08T19:27:08.188668+0000","last_active":"2023-11-08T19:31:14.013099+0000","last_peered":"2023-11-08T19:31:14.013099+0000","last_clean":"2023-11-08T19:31:14.013099+0000","last_became_active":"2023-11-08T19:27:08.188441+0000","last_became_peered":"2023-11-08T19:27:08.188441+0000","last_unstale":"2023-11-08T19:31:14.013099+0000","last_undegraded":"2023-11-08T19:31:14.013099+0000","last_fullsized":"2023-11-08T19:31:14.013099+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":92,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:24.529044+0000","last_change":"2023-11-08T19:31:24.529066+0000","last_active":"2023-11-08T19:31:24.529044+0000","last_peered":"2023-11-08T19:31:24.529044+0000","last_clean":"2023-11-08T19:31:24.529044+0000","last_became_active":"2023-11-08T19:26:04.599322+0000","last_became_peered":"2023-11-08T19:26:04.599322+0000","last_unstale":"2023-11-08T19:31:24.529044+0000","last_undegraded":"2023-11-08T19:31:24.529044+0000","last_fullsized":"2023-11-08T19:31:24.529044+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:24.528935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:24.528935+0000","last_clean_scrub_stamp":"2023-11-08T19:31:24.528935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:09:38.262113+0000","scrub_duration":0.001486217,"objects_trimmed":0,"snaptrim_duration":0.00042670000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":69,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:23.244792+0000","last_change":"2023-11-08T19:31:23.244841+0000","last_active":"2023-11-08T19:31:23.244792+0000","last_peered":"2023-11-08T19:31:23.244792+0000","last_clean":"2023-11-08T19:31:23.244792+0000","last_became_active":"2023-11-08T19:27:07.187337+0000","last_became_peered":"2023-11-08T19:27:07.187337+0000","last_unstale":"2023-11-08T19:31:23.244792+0000","last_undegraded":"2023-11-08T19:31:23.244792+0000","last_fullsized":"2023-11-08T19:31:23.244792+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:23.244666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:23.244666+0000","last_clean_scrub_stamp":"2023-11-08T19:31:23.244666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:40:55.948223+0000","scrub_duration":0.001423278,"objects_trimmed":0,"snaptrim_duration":0.000346967,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":48,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:14.222866+0000","last_change":"2023-11-08T19:27:08.194065+0000","last_active":"2023-11-08T19:31:14.222866+0000","last_peered":"2023-11-08T19:31:14.222866+0000","last_clean":"2023-11-08T19:31:14.222866+0000","last_became_active":"2023-11-08T19:27:08.193532+0000","last_became_peered":"2023-11-08T19:27:08.193532+0000","last_unstale":"2023-11-08T19:31:14.222866+0000","last_undegraded":"2023-11-08T19:31:14.222866+0000","last_fullsized":"2023-11-08T19:31:14.222866+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":86,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:25.726576+0000","last_change":"2023-11-08T19:31:25.726612+0000","last_active":"2023-11-08T19:31:25.726576+0000","last_peered":"2023-11-08T19:31:25.726576+0000","last_clean":"2023-11-08T19:31:25.726576+0000","last_became_active":"2023-11-08T19:26:04.606805+0000","last_became_peered":"2023-11-08T19:26:04.606805+0000","last_unstale":"2023-11-08T19:31:25.726576+0000","last_undegraded":"2023-11-08T19:31:25.726576+0000","last_fullsized":"2023-11-08T19:31:25.726576+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:25.726413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:25.726413+0000","last_clean_scrub_stamp":"2023-11-08T19:31:25.726413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:51:20.104960+0000","scrub_duration":0.002105923,"objects_trimmed":0,"snaptrim_duration":0.00045317000000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":55,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:14.013142+0000","last_change":"2023-11-08T19:27:09.211647+0000","last_active":"2023-11-08T19:31:14.013142+0000","last_peered":"2023-11-08T19:31:14.013142+0000","last_clean":"2023-11-08T19:31:14.013142+0000","last_became_active":"2023-11-08T19:27:07.190309+0000","last_became_peered":"2023-11-08T19:27:07.190309+0000","last_unstale":"2023-11-08T19:31:14.013142+0000","last_undegraded":"2023-11-08T19:31:14.013142+0000","last_fullsized":"2023-11-08T19:31:14.013142+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047615199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":48,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:14.222389+0000","last_change":"2023-11-08T19:27:08.194884+0000","last_active":"2023-11-08T19:31:14.222389+0000","last_peered":"2023-11-08T19:31:14.222389+0000","last_clean":"2023-11-08T19:31:14.222389+0000","last_became_active":"2023-11-08T19:27:08.193241+0000","last_became_peered":"2023-11-08T19:27:08.193241+0000","last_unstale":"2023-11-08T19:31:14.222389+0000","last_undegraded":"2023-11-08T19:31:14.222389+0000","last_fullsized":"2023-11-08T19:31:14.222389+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":78,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:14.012269+0000","last_change":"2023-11-08T19:26:06.265907+0000","last_active":"2023-11-08T19:31:14.012269+0000","last_peered":"2023-11-08T19:31:14.012269+0000","last_clean":"2023-11-08T19:31:14.012269+0000","last_became_active":"2023-11-08T19:26:04.604772+0000","last_became_peered":"2023-11-08T19:26:04.604772+0000","last_unstale":"2023-11-08T19:31:14.012269+0000","last_undegraded":"2023-11-08T19:31:14.012269+0000","last_fullsized":"2023-11-08T19:31:14.012269+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066875300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":55,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:14.214114+0000","last_change":"2023-11-08T19:27:09.282699+0000","last_active":"2023-11-08T19:31:14.214114+0000","last_peered":"2023-11-08T19:31:14.214114+0000","last_clean":"2023-11-08T19:31:14.214114+0000","last_became_active":"2023-11-08T19:27:07.191212+0000","last_became_peered":"2023-11-08T19:27:07.191212+0000","last_unstale":"2023-11-08T19:31:14.214114+0000","last_undegraded":"2023-11-08T19:31:14.214114+0000","last_fullsized":"2023-11-08T19:31:14.214114+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046640800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":49,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:14.013501+0000","last_change":"2023-11-08T19:27:08.187644+0000","last_active":"2023-11-08T19:31:14.013501+0000","last_peered":"2023-11-08T19:31:14.013501+0000","last_clean":"2023-11-08T19:31:14.013501+0000","last_became_active":"2023-11-08T19:27:08.187376+0000","last_became_peered":"2023-11-08T19:27:08.187376+0000","last_unstale":"2023-11-08T19:31:14.013501+0000","last_undegraded":"2023-11-08T19:31:14.013501+0000","last_fullsized":"2023-11-08T19:31:14.013501+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":87,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:26.809222+0000","last_change":"2023-11-08T19:31:26.809256+0000","last_active":"2023-11-08T19:31:26.809222+0000","last_peered":"2023-11-08T19:31:26.809222+0000","last_clean":"2023-11-08T19:31:26.809222+0000","last_became_active":"2023-11-08T19:26:04.598775+0000","last_became_peered":"2023-11-08T19:26:04.598775+0000","last_unstale":"2023-11-08T19:31:26.809222+0000","last_undegraded":"2023-11-08T19:31:26.809222+0000","last_fullsized":"2023-11-08T19:31:26.809222+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:26.809102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:26.809102+0000","last_clean_scrub_stamp":"2023-11-08T19:31:26.809102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:19:38.763559+0000","scrub_duration":0.002091,"objects_trimmed":0,"snaptrim_duration":0.00049870399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":55,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:14.224065+0000","last_change":"2023-11-08T19:27:09.331646+0000","last_active":"2023-11-08T19:31:14.224065+0000","last_peered":"2023-11-08T19:31:14.224065+0000","last_clean":"2023-11-08T19:31:14.224065+0000","last_became_active":"2023-11-08T19:27:07.190619+0000","last_became_peered":"2023-11-08T19:27:07.190619+0000","last_unstale":"2023-11-08T19:31:14.224065+0000","last_undegraded":"2023-11-08T19:31:14.224065+0000","last_fullsized":"2023-11-08T19:31:14.224065+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065918400000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":63,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:38.329109+0000","last_change":"2023-11-08T19:31:38.329133+0000","last_active":"2023-11-08T19:31:38.329109+0000","last_peered":"2023-11-08T19:31:38.329109+0000","last_clean":"2023-11-08T19:31:38.329109+0000","last_became_active":"2023-11-08T19:27:08.183537+0000","last_became_peered":"2023-11-08T19:27:08.183537+0000","last_unstale":"2023-11-08T19:31:38.329109+0000","last_undegraded":"2023-11-08T19:31:38.329109+0000","last_fullsized":"2023-11-08T19:31:38.329109+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:38.328998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:38.328998+0000","last_clean_scrub_stamp":"2023-11-08T19:31:38.328998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:34:44.791135+0000","scrub_duration":0.0020231419999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":90,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:14.017693+0000","last_change":"2023-11-08T19:26:05.610012+0000","last_active":"2023-11-08T19:31:14.017693+0000","last_peered":"2023-11-08T19:31:14.017693+0000","last_clean":"2023-11-08T19:31:14.017693+0000","last_became_active":"2023-11-08T19:26:04.603209+0000","last_became_peered":"2023-11-08T19:26:04.603209+0000","last_unstale":"2023-11-08T19:31:14.017693+0000","last_undegraded":"2023-11-08T19:31:14.017693+0000","last_fullsized":"2023-11-08T19:31:14.017693+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053335100000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":63,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:32.803351+0000","last_change":"2023-11-08T19:31:32.803377+0000","last_active":"2023-11-08T19:31:32.803351+0000","last_peered":"2023-11-08T19:31:32.803351+0000","last_clean":"2023-11-08T19:31:32.803351+0000","last_became_active":"2023-11-08T19:27:07.188169+0000","last_became_peered":"2023-11-08T19:27:07.188169+0000","last_unstale":"2023-11-08T19:31:32.803351+0000","last_undegraded":"2023-11-08T19:31:32.803351+0000","last_fullsized":"2023-11-08T19:31:32.803351+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:32.803247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:32.803247+0000","last_clean_scrub_stamp":"2023-11-08T19:31:32.803247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:30:32.589037+0000","scrub_duration":0.001747384,"objects_trimmed":0,"snaptrim_duration":0.00056209700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":48,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:14.216640+0000","last_change":"2023-11-08T19:27:08.193508+0000","last_active":"2023-11-08T19:31:14.216640+0000","last_peered":"2023-11-08T19:31:14.216640+0000","last_clean":"2023-11-08T19:31:14.216640+0000","last_became_active":"2023-11-08T19:27:08.193037+0000","last_became_peered":"2023-11-08T19:27:08.193037+0000","last_unstale":"2023-11-08T19:31:14.216640+0000","last_undegraded":"2023-11-08T19:31:14.216640+0000","last_fullsized":"2023-11-08T19:31:14.216640+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":86,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:30.100037+0000","last_change":"2023-11-08T19:31:30.100059+0000","last_active":"2023-11-08T19:31:30.100037+0000","last_peered":"2023-11-08T19:31:30.100037+0000","last_clean":"2023-11-08T19:31:30.100037+0000","last_became_active":"2023-11-08T19:26:04.603409+0000","last_became_peered":"2023-11-08T19:26:04.603409+0000","last_unstale":"2023-11-08T19:31:30.100037+0000","last_undegraded":"2023-11-08T19:31:30.100037+0000","last_fullsized":"2023-11-08T19:31:30.100037+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:30.099915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:30.099915+0000","last_clean_scrub_stamp":"2023-11-08T19:31:30.099915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:03:25.984959+0000","scrub_duration":0.001620628,"objects_trimmed":0,"snaptrim_duration":0.00065684300000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":61,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:14.018120+0000","last_change":"2023-11-08T19:27:09.182282+0000","last_active":"2023-11-08T19:31:14.018120+0000","last_peered":"2023-11-08T19:31:14.018120+0000","last_clean":"2023-11-08T19:31:14.018120+0000","last_became_active":"2023-11-08T19:27:07.186944+0000","last_became_peered":"2023-11-08T19:27:07.186944+0000","last_unstale":"2023-11-08T19:31:14.018120+0000","last_undegraded":"2023-11-08T19:31:14.018120+0000","last_fullsized":"2023-11-08T19:31:14.018120+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00028665599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":48,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:14.213515+0000","last_change":"2023-11-08T19:27:08.193990+0000","last_active":"2023-11-08T19:31:14.213515+0000","last_peered":"2023-11-08T19:31:14.213515+0000","last_clean":"2023-11-08T19:31:14.213515+0000","last_became_active":"2023-11-08T19:27:08.193624+0000","last_became_peered":"2023-11-08T19:27:08.193624+0000","last_unstale":"2023-11-08T19:31:14.213515+0000","last_undegraded":"2023-11-08T19:31:14.213515+0000","last_fullsized":"2023-11-08T19:31:14.213515+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":92,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:30.601612+0000","last_change":"2023-11-08T19:31:30.601633+0000","last_active":"2023-11-08T19:31:30.601612+0000","last_peered":"2023-11-08T19:31:30.601612+0000","last_clean":"2023-11-08T19:31:30.601612+0000","last_became_active":"2023-11-08T19:26:04.608639+0000","last_became_peered":"2023-11-08T19:26:04.608639+0000","last_unstale":"2023-11-08T19:31:30.601612+0000","last_undegraded":"2023-11-08T19:31:30.601612+0000","last_fullsized":"2023-11-08T19:31:30.601612+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:30.601479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:30.601479+0000","last_clean_scrub_stamp":"2023-11-08T19:31:30.601479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:16:43.728400+0000","scrub_duration":0.0016375770000000001,"objects_trimmed":0,"snaptrim_duration":0.00044384999999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":55,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:14.214750+0000","last_change":"2023-11-08T19:27:09.283232+0000","last_active":"2023-11-08T19:31:14.214750+0000","last_peered":"2023-11-08T19:31:14.214750+0000","last_clean":"2023-11-08T19:31:14.214750+0000","last_became_active":"2023-11-08T19:27:07.190931+0000","last_became_peered":"2023-11-08T19:27:07.190931+0000","last_unstale":"2023-11-08T19:31:14.214750+0000","last_undegraded":"2023-11-08T19:31:14.214750+0000","last_fullsized":"2023-11-08T19:31:14.214750+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034146799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":52,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:14.215293+0000","last_change":"2023-11-08T19:27:08.187252+0000","last_active":"2023-11-08T19:31:14.215293+0000","last_peered":"2023-11-08T19:31:14.215293+0000","last_clean":"2023-11-08T19:31:14.215293+0000","last_became_active":"2023-11-08T19:27:08.186952+0000","last_became_peered":"2023-11-08T19:27:08.186952+0000","last_unstale":"2023-11-08T19:31:14.215293+0000","last_undegraded":"2023-11-08T19:31:14.215293+0000","last_fullsized":"2023-11-08T19:31:14.215293+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":86,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:32.594224+0000","last_change":"2023-11-08T19:31:32.594249+0000","last_active":"2023-11-08T19:31:32.594224+0000","last_peered":"2023-11-08T19:31:32.594224+0000","last_clean":"2023-11-08T19:31:32.594224+0000","last_became_active":"2023-11-08T19:26:04.602137+0000","last_became_peered":"2023-11-08T19:26:04.602137+0000","last_unstale":"2023-11-08T19:31:32.594224+0000","last_undegraded":"2023-11-08T19:31:32.594224+0000","last_fullsized":"2023-11-08T19:31:32.594224+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:32.594119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:32.594119+0000","last_clean_scrub_stamp":"2023-11-08T19:31:32.594119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:04:34.319826+0000","scrub_duration":0.00148785,"objects_trimmed":0,"snaptrim_duration":0.00056998200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":63,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:35.819610+0000","last_change":"2023-11-08T19:31:35.819632+0000","last_active":"2023-11-08T19:31:35.819610+0000","last_peered":"2023-11-08T19:31:35.819610+0000","last_clean":"2023-11-08T19:31:35.819610+0000","last_became_active":"2023-11-08T19:27:07.192001+0000","last_became_peered":"2023-11-08T19:27:07.192001+0000","last_unstale":"2023-11-08T19:31:35.819610+0000","last_undegraded":"2023-11-08T19:31:35.819610+0000","last_fullsized":"2023-11-08T19:31:35.819610+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:35.819499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:35.819499+0000","last_clean_scrub_stamp":"2023-11-08T19:31:35.819499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:56:02.315632+0000","scrub_duration":0.0014740599999999999,"objects_trimmed":0,"snaptrim_duration":0.00045359000000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":56,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:30.097280+0000","last_change":"2023-11-08T19:31:30.097308+0000","last_active":"2023-11-08T19:31:30.097280+0000","last_peered":"2023-11-08T19:31:30.097280+0000","last_clean":"2023-11-08T19:31:30.097280+0000","last_became_active":"2023-11-08T19:27:08.193458+0000","last_became_peered":"2023-11-08T19:27:08.193458+0000","last_unstale":"2023-11-08T19:31:30.097280+0000","last_undegraded":"2023-11-08T19:31:30.097280+0000","last_fullsized":"2023-11-08T19:31:30.097280+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:30.097159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:30.097159+0000","last_clean_scrub_stamp":"2023-11-08T19:31:30.097159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:12:53.982878+0000","scrub_duration":0.001628696,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":88,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:23.750030+0000","last_change":"2023-11-08T19:31:23.750053+0000","last_active":"2023-11-08T19:31:23.750030+0000","last_peered":"2023-11-08T19:31:23.750030+0000","last_clean":"2023-11-08T19:31:23.750030+0000","last_became_active":"2023-11-08T19:26:04.602183+0000","last_became_peered":"2023-11-08T19:26:04.602183+0000","last_unstale":"2023-11-08T19:31:23.750030+0000","last_undegraded":"2023-11-08T19:31:23.750030+0000","last_fullsized":"2023-11-08T19:31:23.750030+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:23.749930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:23.749930+0000","last_clean_scrub_stamp":"2023-11-08T19:31:23.749930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:48:22.893896+0000","scrub_duration":0.001725201,"objects_trimmed":0,"snaptrim_duration":0.00052029099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":69,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:29.200104+0000","last_change":"2023-11-08T19:31:29.200138+0000","last_active":"2023-11-08T19:31:29.200104+0000","last_peered":"2023-11-08T19:31:29.200104+0000","last_clean":"2023-11-08T19:31:29.200104+0000","last_became_active":"2023-11-08T19:27:07.184507+0000","last_became_peered":"2023-11-08T19:27:07.184507+0000","last_unstale":"2023-11-08T19:31:29.200104+0000","last_undegraded":"2023-11-08T19:31:29.200104+0000","last_fullsized":"2023-11-08T19:31:29.200104+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:29.199968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:29.199968+0000","last_clean_scrub_stamp":"2023-11-08T19:31:29.199968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:17:03.825621+0000","scrub_duration":0.001394225,"objects_trimmed":0,"snaptrim_duration":0.00051924700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":48,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:14.213911+0000","last_change":"2023-11-08T19:27:08.194263+0000","last_active":"2023-11-08T19:31:14.213911+0000","last_peered":"2023-11-08T19:31:14.213911+0000","last_clean":"2023-11-08T19:31:14.213911+0000","last_became_active":"2023-11-08T19:27:08.193938+0000","last_became_peered":"2023-11-08T19:27:08.193938+0000","last_unstale":"2023-11-08T19:31:14.213911+0000","last_undegraded":"2023-11-08T19:31:14.213911+0000","last_fullsized":"2023-11-08T19:31:14.213911+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":86,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:35.112226+0000","last_change":"2023-11-08T19:31:35.112252+0000","last_active":"2023-11-08T19:31:35.112226+0000","last_peered":"2023-11-08T19:31:35.112226+0000","last_clean":"2023-11-08T19:31:35.112226+0000","last_became_active":"2023-11-08T19:26:04.603397+0000","last_became_peered":"2023-11-08T19:26:04.603397+0000","last_unstale":"2023-11-08T19:31:35.112226+0000","last_undegraded":"2023-11-08T19:31:35.112226+0000","last_fullsized":"2023-11-08T19:31:35.112226+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:35.112102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:35.112102+0000","last_clean_scrub_stamp":"2023-11-08T19:31:35.112102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:17:09.737756+0000","scrub_duration":0.0016909259999999999,"objects_trimmed":0,"snaptrim_duration":0.00053361499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":63,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:34.769713+0000","last_change":"2023-11-08T19:31:34.769739+0000","last_active":"2023-11-08T19:31:34.769713+0000","last_peered":"2023-11-08T19:31:34.769713+0000","last_clean":"2023-11-08T19:31:34.769713+0000","last_became_active":"2023-11-08T19:27:07.189943+0000","last_became_peered":"2023-11-08T19:27:07.189943+0000","last_unstale":"2023-11-08T19:31:34.769713+0000","last_undegraded":"2023-11-08T19:31:34.769713+0000","last_fullsized":"2023-11-08T19:31:34.769713+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:34.769604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:34.769604+0000","last_clean_scrub_stamp":"2023-11-08T19:31:34.769604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:51:01.063515+0000","scrub_duration":0.001529461,"objects_trimmed":0,"snaptrim_duration":0.000398725,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":62,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:35.319818+0000","last_change":"2023-11-08T19:31:35.319840+0000","last_active":"2023-11-08T19:31:35.319818+0000","last_peered":"2023-11-08T19:31:35.319818+0000","last_clean":"2023-11-08T19:31:35.319818+0000","last_became_active":"2023-11-08T19:27:08.187487+0000","last_became_peered":"2023-11-08T19:27:08.187487+0000","last_unstale":"2023-11-08T19:31:35.319818+0000","last_undegraded":"2023-11-08T19:31:35.319818+0000","last_fullsized":"2023-11-08T19:31:35.319818+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:35.319727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:35.319727+0000","last_clean_scrub_stamp":"2023-11-08T19:31:35.319727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:41:21.686280+0000","scrub_duration":0.001956157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":84,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:14.218861+0000","last_change":"2023-11-08T19:26:06.425133+0000","last_active":"2023-11-08T19:31:14.218861+0000","last_peered":"2023-11-08T19:31:14.218861+0000","last_clean":"2023-11-08T19:31:14.218861+0000","last_became_active":"2023-11-08T19:26:04.601103+0000","last_became_peered":"2023-11-08T19:26:04.601103+0000","last_unstale":"2023-11-08T19:31:14.218861+0000","last_undegraded":"2023-11-08T19:31:14.218861+0000","last_fullsized":"2023-11-08T19:31:14.218861+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000229244,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":61,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:14.016853+0000","last_change":"2023-11-08T19:27:09.179852+0000","last_active":"2023-11-08T19:31:14.016853+0000","last_peered":"2023-11-08T19:31:14.016853+0000","last_clean":"2023-11-08T19:31:14.016853+0000","last_became_active":"2023-11-08T19:27:07.191926+0000","last_became_peered":"2023-11-08T19:27:07.191926+0000","last_unstale":"2023-11-08T19:31:14.016853+0000","last_undegraded":"2023-11-08T19:31:14.016853+0000","last_fullsized":"2023-11-08T19:31:14.016853+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055457100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":54,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:14.015100+0000","last_change":"2023-11-08T19:27:08.186054+0000","last_active":"2023-11-08T19:31:14.015100+0000","last_peered":"2023-11-08T19:31:14.015100+0000","last_clean":"2023-11-08T19:31:14.015100+0000","last_became_active":"2023-11-08T19:27:08.185581+0000","last_became_peered":"2023-11-08T19:27:08.185581+0000","last_unstale":"2023-11-08T19:31:14.015100+0000","last_undegraded":"2023-11-08T19:31:14.015100+0000","last_fullsized":"2023-11-08T19:31:14.015100+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":90,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:14.017266+0000","last_change":"2023-11-08T19:26:05.609625+0000","last_active":"2023-11-08T19:31:14.017266+0000","last_peered":"2023-11-08T19:31:14.017266+0000","last_clean":"2023-11-08T19:31:14.017266+0000","last_became_active":"2023-11-08T19:26:04.606312+0000","last_became_peered":"2023-11-08T19:26:04.606312+0000","last_unstale":"2023-11-08T19:31:14.017266+0000","last_undegraded":"2023-11-08T19:31:14.017266+0000","last_fullsized":"2023-11-08T19:31:14.017266+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:03.578599+0000","last_clean_scrub_stamp":"2023-11-08T19:26:03.578599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000505935,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":63,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:30.876367+0000","last_change":"2023-11-08T19:31:30.876396+0000","last_active":"2023-11-08T19:31:30.876367+0000","last_peered":"2023-11-08T19:31:30.876367+0000","last_clean":"2023-11-08T19:31:30.876367+0000","last_became_active":"2023-11-08T19:27:07.193254+0000","last_became_peered":"2023-11-08T19:27:07.193254+0000","last_unstale":"2023-11-08T19:31:30.876367+0000","last_undegraded":"2023-11-08T19:31:30.876367+0000","last_fullsized":"2023-11-08T19:31:30.876367+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:30.876249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:30.876249+0000","last_clean_scrub_stamp":"2023-11-08T19:31:30.876249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:32:42.716359+0000","scrub_duration":0.0014983080000000001,"objects_trimmed":0,"snaptrim_duration":0.00041830799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":62,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:36.343730+0000","last_change":"2023-11-08T19:31:36.343755+0000","last_active":"2023-11-08T19:31:36.343730+0000","last_peered":"2023-11-08T19:31:36.343730+0000","last_clean":"2023-11-08T19:31:36.343730+0000","last_became_active":"2023-11-08T19:27:08.188560+0000","last_became_peered":"2023-11-08T19:27:08.188560+0000","last_unstale":"2023-11-08T19:31:36.343730+0000","last_undegraded":"2023-11-08T19:31:36.343730+0000","last_fullsized":"2023-11-08T19:31:36.343730+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:36.343622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:36.343622+0000","last_clean_scrub_stamp":"2023-11-08T19:31:36.343622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:29:56.719829+0000","scrub_duration":0.002226525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":87,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:23.098586+0000","last_change":"2023-11-08T19:31:23.098607+0000","last_active":"2023-11-08T19:31:23.098586+0000","last_peered":"2023-11-08T19:31:23.098586+0000","last_clean":"2023-11-08T19:31:23.098586+0000","last_became_active":"2023-11-08T19:26:04.603979+0000","last_became_peered":"2023-11-08T19:26:04.603979+0000","last_unstale":"2023-11-08T19:31:23.098586+0000","last_undegraded":"2023-11-08T19:31:23.098586+0000","last_fullsized":"2023-11-08T19:31:23.098586+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:23.098455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:23.098455+0000","last_clean_scrub_stamp":"2023-11-08T19:31:23.098455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:03:18.983499+0000","scrub_duration":0.0017647400000000001,"objects_trimmed":0,"snaptrim_duration":0.00079032099999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":63,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:37.483304+0000","last_change":"2023-11-08T19:31:37.483328+0000","last_active":"2023-11-08T19:31:37.483304+0000","last_peered":"2023-11-08T19:31:37.483304+0000","last_clean":"2023-11-08T19:31:37.483304+0000","last_became_active":"2023-11-08T19:27:07.185478+0000","last_became_peered":"2023-11-08T19:27:07.185478+0000","last_unstale":"2023-11-08T19:31:37.483304+0000","last_undegraded":"2023-11-08T19:31:37.483304+0000","last_fullsized":"2023-11-08T19:31:37.483304+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:37.483195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:37.483195+0000","last_clean_scrub_stamp":"2023-11-08T19:31:37.483195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:53:43.068960+0000","scrub_duration":0.001795891,"objects_trimmed":0,"snaptrim_duration":0.0010661850000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574935,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33048,"kb_used_data":1872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338792,"statfs":{"total":96636764160,"available":96602923008,"internally_reserved":0,"allocated":1916928,"data_stored":1236765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:30:56 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.54400000000000004,"15min":0.51100000000000001},"min":{"1min":0.45300000000000001,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.71899999999999997,"5min":0.78600000000000003,"15min":1.0149999999999999},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.53500000000000003,"15min":0.52100000000000002},"min":{"1min":0.42099999999999999,"5min":0.35099999999999998,"15min":0.32300000000000001},"max":{"1min":0.79300000000000004,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.68000000000000005}]},{"osd":1,"last update":"Wed Nov 8 19:30:56 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.59799999999999998,"15min":0.57599999999999996},"min":{"1min":0.47699999999999998,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.73499999999999999,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.623,"5min":0.59599999999999997,"15min":0.56699999999999995},"min":{"1min":0.505,"5min":0.40600000000000003,"15min":0.33900000000000002},"max":{"1min":0.79500000000000004,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.72699999999999998}]},{"osd":2,"last update":"Wed Nov 8 19:30:56 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.54400000000000004,"15min":0.505},"min":{"1min":0.372,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.67600000000000005,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.57399999999999995,"15min":0.55600000000000005},"min":{"1min":0.45900000000000002,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.78400000000000003,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.73999999999999999}]},{"osd":3,"last update":"Wed Nov 8 19:30:56 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.57499999999999996,"15min":0.56399999999999995},"min":{"1min":0.42199999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.77300000000000002,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.63800000000000001,"15min":0.59299999999999997},"min":{"1min":0.47399999999999998,"5min":0.437,"15min":0.40100000000000002},"max":{"1min":0.81399999999999995,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.65200000000000002}]},{"osd":4,"last update":"Wed Nov 8 19:30:56 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.501,"15min":0.505},"min":{"1min":0.32400000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.753,"5min":0.753,"15min":0.78600000000000003},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.53400000000000003,"15min":0.53700000000000003},"min":{"1min":0.35199999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.71799999999999997,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.51000000000000001}]},{"osd":5,"last update":"Wed Nov 8 19:30:56 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.56699999999999995,"15min":0.57499999999999996},"min":{"1min":0.35099999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.77300000000000002,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.627,"15min":0.63700000000000001},"min":{"1min":0.377,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.83899999999999997,"5min":0.89900000000000002,"15min":0.94299999999999995},"last":0.63600000000000001}]},{"osd":6,"last update":"Wed Nov 8 19:30:56 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.57999999999999996,"15min":0.59699999999999998},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.97299999999999998},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.59699999999999998,"15min":0.60499999999999998},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.78900000000000003,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.61699999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574935,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32468,"kb_used_data":1292,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339372,"statfs":{"total":96636764160,"available":96603516928,"internally_reserved":0,"allocated":1323008,"data_stored":646397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:30:57 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.55900000000000005,"15min":1.1459999999999999},"min":{"1min":0.41999999999999998,"5min":0.33200000000000002,"15min":0.27000000000000002},"max":{"1min":0.84599999999999997,"5min":0.91700000000000004,"15min":21.359999999999999},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.56899999999999995,"15min":0.81599999999999995},"min":{"1min":0.39700000000000002,"5min":0.378,"15min":0.30399999999999999},"max":{"1min":0.68700000000000006,"5min":0.90300000000000002,"15min":10.329000000000001},"last":0.58099999999999996}]},{"osd":1,"last update":"Wed Nov 8 19:30:57 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.53800000000000003,"15min":0.59499999999999997},"min":{"1min":0.41199999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.81399999999999995,"5min":0.86299999999999999,"15min":3.4239999999999999},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.60499999999999998,"15min":0.63400000000000001},"min":{"1min":0.35899999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":2.9260000000000002,"5min":2.9260000000000002,"15min":3.4809999999999999},"last":0.59499999999999997}]},{"osd":2,"last update":"Wed Nov 8 19:30:57 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.58599999999999997,"15min":0.83399999999999996},"min":{"1min":0.51800000000000002,"5min":0.38500000000000001,"15min":0.34399999999999997},"max":{"1min":0.84199999999999997,"5min":0.96399999999999997,"15min":10.398},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.56799999999999995,"15min":0.66200000000000003},"min":{"1min":0.41799999999999998,"5min":0.318,"15min":0.318},"max":{"1min":0.77500000000000002,"5min":0.89100000000000001,"15min":5.0170000000000003},"last":0.629}]},{"osd":3,"last update":"Wed Nov 8 19:30:57 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.63100000000000001,"15min":0.66100000000000003},"min":{"1min":0.46200000000000002,"5min":0.34599999999999997,"15min":0.317},"max":{"1min":3.0089999999999999,"5min":3.0089999999999999,"15min":3.5710000000000002},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.59699999999999998,"15min":0.85999999999999999},"min":{"1min":0.39000000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.81100000000000005,"5min":0.93500000000000005,"15min":10.368},"last":0.64700000000000002}]},{"osd":4,"last update":"Wed Nov 8 19:30:57 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54400000000000004,"15min":0.52000000000000002},"min":{"1min":0.34699999999999998,"5min":0.33000000000000002,"15min":0.29199999999999998},"max":{"1min":0.79100000000000004,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.55900000000000005,"15min":0.55800000000000005},"min":{"1min":0.45000000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.73799999999999999,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.435}]},{"osd":5,"last update":"Wed Nov 8 19:30:57 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.55000000000000004,"15min":0.53200000000000003},"min":{"1min":0.42699999999999999,"5min":0.34599999999999997,"15min":0.307},"max":{"1min":0.70499999999999996,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.60399999999999998,"15min":0.58599999999999997},"min":{"1min":0.41199999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.51100000000000001}]},{"osd":7,"last update":"Wed Nov 8 19:30:57 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.60999999999999999,"15min":0.58299999999999996},"min":{"1min":0.39700000000000002,"5min":0.39500000000000002,"15min":0.34699999999999998},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.60599999999999998,"15min":0.59199999999999997},"min":{"1min":0.46300000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.82199999999999995,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.41399999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574935,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32468,"kb_used_data":1292,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339372,"statfs":{"total":96636764160,"available":96603516928,"internally_reserved":0,"allocated":1323008,"data_stored":646397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:30:55 2023","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.35499999999999998,"15min":0.374},"min":{"1min":0.222,"5min":0.219,"15min":0.219},"max":{"1min":0.437,"5min":0.69699999999999995,"15min":1.389},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.36399999999999999,"15min":0.61699999999999999},"min":{"1min":0.255,"5min":0.23000000000000001,"15min":0.21199999999999999},"max":{"1min":0.52900000000000003,"5min":0.622,"15min":9.1370000000000005},"last":0.40999999999999998}]},{"osd":2,"last update":"Wed Nov 8 19:30:55 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.35899999999999999,"15min":0.36499999999999999},"min":{"1min":0.248,"5min":0.216,"15min":0.216},"max":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":1.2709999999999999},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.39200000000000002,"15min":0.41899999999999998},"min":{"1min":0.27700000000000002,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.53400000000000003,"5min":0.63600000000000001,"15min":1.425},"last":0.46999999999999997}]},{"osd":3,"last update":"Wed Nov 8 19:30:55 2023","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.40899999999999997,"15min":0.74399999999999999},"min":{"1min":0.29599999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.54600000000000004,"5min":0.63800000000000001,"15min":11.957000000000001},"last":0.495},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.39100000000000001,"15min":0.91800000000000004},"min":{"1min":0.27900000000000003,"5min":0.252,"15min":0.252},"max":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":18.039000000000001},"last":0.45100000000000001}]},{"osd":4,"last update":"Wed Nov 8 19:30:55 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59299999999999997,"15min":0.751},"min":{"1min":0.35199999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.81100000000000005,"5min":1.4370000000000001,"15min":6.181},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.60999999999999999,"15min":0.63},"min":{"1min":0.41399999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.83899999999999997,"5min":1.411,"15min":1.488},"last":0.80200000000000005}]},{"osd":5,"last update":"Wed Nov 8 19:30:55 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61199999999999999,"15min":0.86099999999999999},"min":{"1min":0.375,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.85399999999999998,"5min":1.421,"15min":9.2420000000000009},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60599999999999998,"15min":0.78900000000000003},"min":{"1min":0.42499999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.80100000000000005,"5min":1.2969999999999999,"15min":6.2110000000000003},"last":0.64000000000000001}]},{"osd":6,"last update":"Wed Nov 8 19:30:55 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.63500000000000001,"15min":0.86899999999999999},"min":{"1min":0.40799999999999997,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.78700000000000003,"5min":1.4530000000000001,"15min":9.1880000000000006},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63500000000000001,"15min":0.65600000000000003},"min":{"1min":0.44500000000000001,"5min":0.38,"15min":0.38},"max":{"1min":0.86399999999999999,"5min":1.272,"15min":1.462},"last":0.627}]},{"osd":7,"last update":"Wed Nov 8 19:30:55 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.625,"15min":0.78800000000000003},"min":{"1min":0.41499999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0309999999999999,"5min":1.3979999999999999,"15min":6.1479999999999997},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.64600000000000002,"15min":0.90000000000000002},"min":{"1min":0.51300000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.97499999999999998,"5min":1.353,"15min":9.2170000000000005},"last":0.61399999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574935,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33048,"kb_used_data":1872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338792,"statfs":{"total":96636764160,"available":96602923008,"internally_reserved":0,"allocated":1916928,"data_stored":1236765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Nov 8 19:31:01 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.33300000000000002,"15min":0.45900000000000002},"min":{"1min":0.20899999999999999,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.51000000000000001,"5min":0.55300000000000005,"15min":4.1200000000000001},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.439,"5min":0.371,"15min":0.91900000000000004},"min":{"1min":0.22800000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.62,"5min":0.62,"15min":16.940999999999999},"last":0.40699999999999997}]},{"osd":2,"last update":"Wed Nov 8 19:31:01 2023","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.35299999999999998,"15min":0.38700000000000001},"min":{"1min":0.25,"5min":0.22,"15min":0.22},"max":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":1.1899999999999999},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.39100000000000001,"15min":0.60499999999999998},"min":{"1min":0.32400000000000001,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":7.0209999999999999},"last":0.46100000000000002}]},{"osd":3,"last update":"Wed Nov 8 19:31:01 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.38400000000000001,"15min":0.51200000000000001},"min":{"1min":0.29899999999999999,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":4.0940000000000003},"last":0.44},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.39500000000000002,"15min":0.433},"min":{"1min":0.32500000000000001,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.61899999999999999,"5min":0.62,"15min":1.2549999999999999},"last":0.47499999999999998}]},{"osd":4,"last update":"Wed Nov 8 19:31:01 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.58899999999999997,"15min":0.71699999999999997},"min":{"1min":0.42799999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":4.0049999999999999},"last":0.629},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.58399999999999996,"15min":0.623},"min":{"1min":0.442,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.746,"5min":0.79200000000000004,"15min":1.2270000000000001},"last":0.66300000000000003}]},{"osd":5,"last update":"Wed Nov 8 19:31:01 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.59699999999999998,"15min":0.72799999999999998},"min":{"1min":0.47199999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":4.3090000000000002},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.61199999999999999,"15min":0.72399999999999998},"min":{"1min":0.436,"5min":0.373,"15min":0.373},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":4.0709999999999997},"last":0.68000000000000005}]},{"osd":6,"last update":"Wed Nov 8 19:31:01 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.621,"15min":0.65800000000000003},"min":{"1min":0.40999999999999998,"5min":0.373,"15min":0.373},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":1.2849999999999999},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.64900000000000002,"15min":0.78500000000000003},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":4.2560000000000002},"last":0.69699999999999995}]},{"osd":7,"last update":"Wed Nov 8 19:31:01 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.625,"15min":0.74299999999999999},"min":{"1min":0.47099999999999997,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":4.2850000000000001},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.64700000000000002,"15min":0.77000000000000002},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.002,"5min":1.002,"15min":4.0460000000000003},"last":0.71399999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574935,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28308,"kb_used_data":1292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343532,"statfs":{"total":96636764160,"available":96607776768,"internally_reserved":0,"allocated":1323008,"data_stored":646397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:30:53 2023","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.35099999999999998,"15min":0.33800000000000002},"min":{"1min":0.23599999999999999,"5min":0.182,"15min":0.182},"max":{"1min":0.629,"5min":0.629,"15min":0.629},"last":0.28499999999999998},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.35899999999999999,"15min":0.55900000000000005},"min":{"1min":0.26800000000000002,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.56499999999999995,"5min":0.68000000000000005,"15min":6.0999999999999996},"last":0.35999999999999999}]},{"osd":1,"last update":"Wed Nov 8 19:30:53 2023","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.35599999999999998,"15min":0.34599999999999997},"min":{"1min":0.247,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.38800000000000001,"15min":0.39000000000000001},"min":{"1min":0.30299999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.63100000000000001,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.41799999999999998}]},{"osd":3,"last update":"Wed Nov 8 19:30:53 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.40600000000000003,"15min":0.39200000000000002},"min":{"1min":0.28199999999999997,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.60999999999999999,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.40000000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"min":{"1min":0.28799999999999998,"5min":0.217,"15min":0.217},"max":{"1min":0.621,"5min":0.65900000000000003,"15min":0.70299999999999996},"last":0.33800000000000002}]},{"osd":4,"last update":"Wed Nov 8 19:30:53 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56000000000000005,"15min":0.57599999999999996},"min":{"1min":0.41699999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.58099999999999996,"15min":0.58299999999999996},"min":{"1min":0.40500000000000003,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.872,"5min":0.872,"15min":0.90400000000000003},"last":0.73799999999999999}]},{"osd":5,"last update":"Wed Nov 8 19:30:53 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61799999999999999,"15min":0.59399999999999997},"min":{"1min":0.40799999999999997,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.622,"5min":0.60599999999999998,"15min":0.61099999999999999},"min":{"1min":0.48799999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.75800000000000001}]},{"osd":6,"last update":"Wed Nov 8 19:30:53 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.59399999999999997,"15min":0.59799999999999998},"min":{"1min":0.432,"5min":0.379,"15min":0.371},"max":{"1min":0.81000000000000005,"5min":0.83599999999999997,"15min":0.84799999999999998},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.59999999999999998,"15min":0.58599999999999997},"min":{"1min":0.42799999999999999,"5min":0.40300000000000002,"15min":0.36599999999999999},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.69699999999999995}]},{"osd":7,"last update":"Wed Nov 8 19:30:53 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.60899999999999999,"15min":0.58899999999999997},"min":{"1min":0.46200000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.61699999999999999,"15min":0.60899999999999999},"min":{"1min":0.41699999999999998,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.79600000000000004,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.72499999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574936,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28308,"kb_used_data":1292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343532,"statfs":{"total":96636764160,"available":96607776768,"internally_reserved":0,"allocated":1323008,"data_stored":646397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:30:59 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.33500000000000002,"15min":0.34000000000000002},"min":{"1min":0.254,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.48599999999999999,"5min":0.49099999999999999,"15min":0.626},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.378,"5min":0.372,"15min":0.35599999999999998},"min":{"1min":0.26700000000000002,"5min":0.23699999999999999,"15min":0.23300000000000001},"max":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.60799999999999998},"last":0.35299999999999998}]},{"osd":1,"last update":"Wed Nov 8 19:30:59 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.35999999999999999,"15min":0.35599999999999998},"min":{"1min":0.245,"5min":0.24099999999999999,"15min":0.23999999999999999},"max":{"1min":0.53300000000000003,"5min":0.61699999999999999,"15min":0.61699999999999999},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.40100000000000002,"15min":0.39000000000000001},"min":{"1min":0.30099999999999999,"5min":0.27900000000000003,"15min":0.27700000000000002},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.82099999999999995},"last":0.379}]},{"osd":2,"last update":"Wed Nov 8 19:30:59 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.40999999999999998,"15min":0.38300000000000001},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28799999999999998},"max":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58499999999999996},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.41999999999999998,"15min":0.40899999999999997},"min":{"1min":0.30499999999999999,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.59799999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.50900000000000001}]},{"osd":4,"last update":"Wed Nov 8 19:30:59 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.63,"15min":0.60799999999999998},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.85699999999999998,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63700000000000001,"15min":0.61499999999999999},"min":{"1min":0.38600000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.89600000000000002,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.57199999999999995}]},{"osd":5,"last update":"Wed Nov 8 19:30:59 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59899999999999998,"15min":0.58299999999999996},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.83099999999999996,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.623,"5min":0.622,"15min":0.59199999999999997},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.879,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.495}]},{"osd":6,"last update":"Wed Nov 8 19:30:59 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.61399999999999999,"15min":0.59099999999999997},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41399999999999998},"max":{"1min":0.83899999999999997,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63300000000000001,"15min":0.61299999999999999},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.90500000000000003},"last":0.55800000000000005}]},{"osd":7,"last update":"Wed Nov 8 19:30:59 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.626,"15min":0.59999999999999998},"min":{"1min":0.41899999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":1.0009999999999999},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.63900000000000001,"15min":0.60999999999999999},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.85099999999999998,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.60999999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574935,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28308,"kb_used_data":1292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343532,"statfs":{"total":96636764160,"available":96607776768,"internally_reserved":0,"allocated":1323008,"data_stored":646397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:30:58 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54700000000000004,"15min":0.69899999999999995},"min":{"1min":0.311,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.71499999999999997,"5min":0.84799999999999998,"15min":6.5970000000000004},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.53500000000000003,"15min":0.56299999999999994},"min":{"1min":0.36299999999999999,"5min":0.32200000000000001,"15min":0.317},"max":{"1min":0.73099999999999998,"5min":0.75900000000000001,"15min":2.5030000000000001},"last":0.70499999999999996}]},{"osd":1,"last update":"Wed Nov 8 19:30:58 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.55400000000000005,"15min":0.51500000000000001},"min":{"1min":0.41699999999999998,"5min":0.32900000000000001,"15min":0.24299999999999999},"max":{"1min":0.72799999999999998,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.55900000000000005,"15min":0.52500000000000002},"min":{"1min":0.49299999999999999,"5min":0.312,"15min":0.312},"max":{"1min":0.72799999999999998,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.75800000000000001}]},{"osd":2,"last update":"Wed Nov 8 19:30:58 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55300000000000005},"min":{"1min":0.505,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.64400000000000002,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.58799999999999997,"15min":0.76000000000000001},"min":{"1min":0.50700000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.70999999999999996,"5min":0.871,"15min":6.6920000000000002},"last":0.67300000000000004}]},{"osd":3,"last update":"Wed Nov 8 19:30:58 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.56499999999999995,"15min":0.58099999999999996},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.69799999999999995,"5min":0.79800000000000004,"15min":1.0700000000000001},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.61799999999999999,"15min":0.61299999999999999},"min":{"1min":0.47799999999999998,"5min":0.41699999999999998,"15min":0.34599999999999997},"max":{"1min":0.70399999999999996,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.746}]},{"osd":5,"last update":"Wed Nov 8 19:30:58 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.501,"15min":0.53100000000000003},"min":{"1min":0.39100000000000001,"5min":0.29599999999999999,"15min":0.28100000000000003},"max":{"1min":0.66100000000000003,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.51700000000000002,"15min":0.57099999999999995},"min":{"1min":0.377,"5min":0.32600000000000001,"15min":0.30299999999999999},"max":{"1min":0.65700000000000003,"5min":0.876,"15min":1.1060000000000001},"last":0.48399999999999999}]},{"osd":6,"last update":"Wed Nov 8 19:30:58 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.54700000000000004,"15min":0.57199999999999995},"min":{"1min":0.35199999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.71899999999999997,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.60299999999999998,"15min":0.60999999999999999},"min":{"1min":0.373,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.74099999999999999,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.60199999999999998}]},{"osd":7,"last update":"Wed Nov 8 19:30:58 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.55800000000000005,"15min":0.60999999999999999},"min":{"1min":0.41599999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.68000000000000005,"5min":0.84499999999999997,"15min":1.141},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.59199999999999997,"15min":0.63},"min":{"1min":0.41399999999999998,"5min":0.38700000000000001,"15min":0.33000000000000002},"max":{"1min":0.76400000000000001,"5min":0.86399999999999999,"15min":0.86799999999999999},"last":0.57899999999999996}]}]},{"osd":5,"up_from":13,"seq":55834574935,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28308,"kb_used_data":1292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343532,"statfs":{"total":96636764160,"available":96607776768,"internally_reserved":0,"allocated":1323008,"data_stored":646397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:30:51 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.57199999999999995,"15min":0.53900000000000003},"min":{"1min":0.35199999999999998,"5min":0.307,"15min":0.307},"max":{"1min":3.9689999999999999,"5min":3.9689999999999999,"15min":3.9689999999999999},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.54800000000000004,"15min":0.52200000000000002},"min":{"1min":0.41699999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.68899999999999995,"5min":0.746,"15min":0.746},"last":0.628}]},{"osd":1,"last update":"Wed Nov 8 19:30:51 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.55300000000000005,"15min":0.53300000000000003},"min":{"1min":0.42399999999999999,"5min":0.37,"15min":0.37},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.54900000000000004,"15min":0.54100000000000004},"min":{"1min":0.44400000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.753,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.61599999999999999}]},{"osd":2,"last update":"Wed Nov 8 19:30:51 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.56399999999999995,"15min":0.54400000000000004},"min":{"1min":0.40000000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.749,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.56499999999999995,"15min":0.57299999999999995},"min":{"1min":0.46400000000000002,"5min":0.316,"15min":0.316},"max":{"1min":0.746,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.56000000000000005}]},{"osd":3,"last update":"Wed Nov 8 19:30:51 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.58499999999999996,"15min":0.56200000000000006},"min":{"1min":0.39600000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.73699999999999999,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.57999999999999996,"15min":0.59599999999999997},"min":{"1min":0.39900000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.79000000000000004,"5min":0.87,"15min":0.87},"last":0.60299999999999998}]},{"osd":4,"last update":"Wed Nov 8 19:30:51 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.495,"15min":0.54000000000000004},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.71899999999999997,"5min":0.80500000000000005,"15min":0.86799999999999999},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.53900000000000003,"15min":0.56100000000000005},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.751,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.43099999999999999}]},{"osd":6,"last update":"Wed Nov 8 19:30:51 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.56100000000000005,"15min":0.59599999999999997},"min":{"1min":0.39700000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.73599999999999999,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59199999999999997,"15min":0.60899999999999999},"min":{"1min":0.41399999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.76400000000000001,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.53800000000000003}]},{"osd":7,"last update":"Wed Nov 8 19:30:51 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.57099999999999995,"15min":0.59899999999999998},"min":{"1min":0.38200000000000001,"5min":0.371,"15min":0.36899999999999999},"max":{"1min":0.74099999999999999,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58999999999999997,"15min":0.63},"min":{"1min":0.433,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.91900000000000004},"last":0.51200000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-08T19:31:43.360 INFO:tasks.ceph:pgid 12.d last_scrub_stamp 2023-11-08T19:27:07.167285+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=7, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:43.361 INFO:tasks.ceph:pgid 11.a last_scrub_stamp 2023-11-08T19:27:06.162282+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=6, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:43.361 INFO:tasks.ceph:pgid 12.8 last_scrub_stamp 2023-11-08T19:27:07.167285+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=7, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:43.362 INFO:tasks.ceph:pgid 6.2 last_scrub_stamp 2023-11-08T19:26:03.578599+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=26, tm_sec=3, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:43.362 INFO:tasks.ceph:pgid 11.f last_scrub_stamp 2023-11-08T19:27:06.162282+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=6, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:43.362 INFO:tasks.ceph:pgid 12.9 last_scrub_stamp 2023-11-08T19:27:07.167285+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=7, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:43.363 INFO:tasks.ceph:pgid 6.3 last_scrub_stamp 2023-11-08T19:26:03.578599+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=26, tm_sec=3, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:43.363 INFO:tasks.ceph:pgid 11.e last_scrub_stamp 2023-11-08T19:27:06.162282+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=6, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:43.364 INFO:tasks.ceph:pgid 12.a last_scrub_stamp 2023-11-08T19:27:07.167285+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=7, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:43.364 INFO:tasks.ceph:pgid 12.b last_scrub_stamp 2023-11-08T19:27:07.167285+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=7, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:43.365 INFO:tasks.ceph:pgid 12.c last_scrub_stamp 2023-11-08T19:27:07.167285+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=7, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:43.365 INFO:tasks.ceph:pgid 6.6 last_scrub_stamp 2023-11-08T19:26:03.578599+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=26, tm_sec=3, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:43.366 INFO:tasks.ceph:pgid 11.b last_scrub_stamp 2023-11-08T19:27:06.162282+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=6, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:43.366 INFO:tasks.ceph:pgid 12.e last_scrub_stamp 2023-11-08T19:27:07.167285+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=7, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:43.367 INFO:tasks.ceph:pgid 11.9 last_scrub_stamp 2023-11-08T19:27:06.162282+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=6, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:43.367 INFO:tasks.ceph:pgid 6.5 last_scrub_stamp 2023-11-08T19:26:03.578599+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=26, tm_sec=3, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:43.367 INFO:tasks.ceph:pgid 11.8 last_scrub_stamp 2023-11-08T19:27:06.162282+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=6, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:43.368 INFO:tasks.ceph:pgid 12.0 last_scrub_stamp 2023-11-08T19:27:07.167285+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=7, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:43.368 INFO:tasks.ceph:pgid 11.6 last_scrub_stamp 2023-11-08T19:27:06.162282+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=6, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:43.369 INFO:tasks.ceph:pgid 12.1 last_scrub_stamp 2023-11-08T19:27:07.167285+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=7, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:43.369 INFO:tasks.ceph:pgid 11.5 last_scrub_stamp 2023-11-08T19:27:06.162282+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=6, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:43.369 INFO:tasks.ceph:pgid 12.2 last_scrub_stamp 2023-11-08T19:27:07.167285+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=7, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:43.370 INFO:tasks.ceph:pgid 11.4 last_scrub_stamp 2023-11-08T19:27:06.162282+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=6, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:43.370 INFO:tasks.ceph:pgid 12.3 last_scrub_stamp 2023-11-08T19:27:07.167285+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=7, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:43.371 INFO:tasks.ceph:pgid 11.2 last_scrub_stamp 2023-11-08T19:27:06.162282+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=6, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:43.371 INFO:tasks.ceph:pgid 12.6 last_scrub_stamp 2023-11-08T19:27:07.167285+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=7, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:43.372 INFO:tasks.ceph:pgid 12.7 last_scrub_stamp 2023-11-08T19:27:07.167285+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=7, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:43.372 INFO:tasks.ceph:pgid 6.12 last_scrub_stamp 2023-11-08T19:26:03.578599+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=26, tm_sec=3, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:43.372 INFO:tasks.ceph:pgid 12.19 last_scrub_stamp 2023-11-08T19:27:07.167285+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=7, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:43.373 INFO:tasks.ceph:pgid 11.1e last_scrub_stamp 2023-11-08T19:27:06.162282+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=6, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:43.373 INFO:tasks.ceph:pgid 12.1a last_scrub_stamp 2023-11-08T19:27:07.167285+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=7, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:43.374 INFO:tasks.ceph:pgid 11.1d last_scrub_stamp 2023-11-08T19:27:06.162282+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=6, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:43.374 INFO:tasks.ceph:pgid 12.1b last_scrub_stamp 2023-11-08T19:27:07.167285+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=7, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:43.375 INFO:tasks.ceph:pgid 12.1c last_scrub_stamp 2023-11-08T19:27:07.167285+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=7, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:43.375 INFO:tasks.ceph:pgid 11.1b last_scrub_stamp 2023-11-08T19:27:06.162282+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=6, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:43.375 INFO:tasks.ceph:pgid 12.1d last_scrub_stamp 2023-11-08T19:27:07.167285+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=7, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:43.376 INFO:tasks.ceph:pgid 6.17 last_scrub_stamp 2023-11-08T19:26:03.578599+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=26, tm_sec=3, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:43.376 INFO:tasks.ceph:pgid 11.1a last_scrub_stamp 2023-11-08T19:27:06.162282+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=6, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:43.376 INFO:tasks.ceph:pgid 12.1e last_scrub_stamp 2023-11-08T19:27:07.167285+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=7, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:43.377 INFO:tasks.ceph:pgid 11.19 last_scrub_stamp 2023-11-08T19:27:06.162282+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=6, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:43.378 INFO:tasks.ceph:pgid 6.15 last_scrub_stamp 2023-11-08T19:26:03.578599+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=26, tm_sec=3, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:43.378 INFO:tasks.ceph:pgid 12.10 last_scrub_stamp 2023-11-08T19:27:07.167285+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=7, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:43.378 INFO:tasks.ceph:pgid 11.17 last_scrub_stamp 2023-11-08T19:27:06.162282+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=6, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:43.379 INFO:tasks.ceph:pgid 12.11 last_scrub_stamp 2023-11-08T19:27:07.167285+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=7, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:43.379 INFO:tasks.ceph:pgid 11.16 last_scrub_stamp 2023-11-08T19:27:06.162282+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=6, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:43.380 INFO:tasks.ceph:pgid 12.12 last_scrub_stamp 2023-11-08T19:27:07.167285+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=7, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:43.380 INFO:tasks.ceph:pgid 12.14 last_scrub_stamp 2023-11-08T19:27:07.167285+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=7, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:43.381 INFO:tasks.ceph:pgid 6.1f last_scrub_stamp 2023-11-08T19:26:03.578599+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=26, tm_sec=3, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:43.381 INFO:tasks.ceph:pgid 11.12 last_scrub_stamp 2023-11-08T19:27:06.162282+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=6, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:43.382 INFO:tasks.ceph:pgid 12.16 last_scrub_stamp 2023-11-08T19:27:07.167285+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=7, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:43.382 INFO:tasks.ceph:pgid 6.1c last_scrub_stamp 2023-11-08T19:26:03.578599+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=26, tm_sec=3, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:31:43.382 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-11-08T19:31:57.879 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T19:31:57.882 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T19:32:03.383 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-08T19:32:03.517 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:03.515+0000 7f930c1d6700 1 Processor -- start 2023-11-08T19:32:03.517 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:03.516+0000 7f930c1d6700 1 -- start start 2023-11-08T19:32:03.520 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:03.516+0000 7f930c1d6700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f930415dd00 0x7f93040cca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:03.520 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:03.516+0000 7f930c1d6700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f93041618f0 0x7f930406ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:03.521 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:03.516+0000 7f930c1d6700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f930406b150 0x7f93040b2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:03.521 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:03.516+0000 7f930c1d6700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f930406b600 con 0x7f930415dd00 2023-11-08T19:32:03.521 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:03.516+0000 7f930c1d6700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93040cad50 con 0x7f930406b150 2023-11-08T19:32:03.522 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:03.516+0000 7f930c1d6700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93040ccfb0 con 0x7f93041618f0 2023-11-08T19:32:03.522 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:03.517+0000 7f9309f72700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f930415dd00 0x7f93040cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:03.522 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:03.517+0000 7f9309f72700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f930415dd00 0x7f93040cca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57424/0 (socket says 172.21.15.143:57424) 2023-11-08T19:32:03.523 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:03.517+0000 7f9309f72700 1 -- 172.21.15.143:0/745358603 learned_addr learned my addr 172.21.15.143:0/745358603 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:32:03.523 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:03.517+0000 7f9309771700 1 --2- 172.21.15.143:0/745358603 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f93041618f0 0x7f930406ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:03.523 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:03.517+0000 7f930a773700 1 --2- 172.21.15.143:0/745358603 >> v2:172.21.15.174:3300/0 conn(0x7f930406b150 0x7f93040b2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:03.524 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:03.517+0000 7f9309f72700 1 -- 172.21.15.143:0/745358603 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f93041618f0 msgr2=0x7f930406ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:32:03.524 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:03.517+0000 7f9309f72700 1 --2- 172.21.15.143:0/745358603 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f93041618f0 0x7f930406ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:03.525 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:03.517+0000 7f9309f72700 1 -- 172.21.15.143:0/745358603 >> v2:172.21.15.174:3300/0 conn(0x7f930406b150 msgr2=0x7f93040b2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:32:03.525 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:03.517+0000 7f9309f72700 1 --2- 172.21.15.143:0/745358603 >> v2:172.21.15.174:3300/0 conn(0x7f930406b150 0x7f93040b2210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:03.525 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:03.517+0000 7f9309f72700 1 -- 172.21.15.143:0/745358603 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9304161df0 con 0x7f930415dd00 2023-11-08T19:32:03.525 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:03.517+0000 7f9309f72700 1 --2- 172.21.15.143:0/745358603 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f930415dd00 0x7f93040cca70 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f93040708b0 tx=0x7f92f4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1deb0cca6d7db260 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:03.526 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:03.517+0000 7f9308f70700 1 -- 172.21.15.143:0/745358603 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92f4005e30 con 0x7f930415dd00 2023-11-08T19:32:03.526 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:03.517+0000 7f9308f70700 1 -- 172.21.15.143:0/745358603 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92f4004030 con 0x7f930415dd00 2023-11-08T19:32:03.526 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:03.517+0000 7f9308f70700 1 -- 172.21.15.143:0/745358603 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92f40043e0 con 0x7f930415dd00 2023-11-08T19:32:03.527 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:03.518+0000 7f930c1d6700 1 -- 172.21.15.143:0/745358603 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f930415dd00 msgr2=0x7f93040cca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:03.527 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:03.518+0000 7f930c1d6700 1 --2- 172.21.15.143:0/745358603 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f930415dd00 0x7f93040cca70 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f93040708b0 tx=0x7f92f4009bd0 comp rx=0 tx=0).stop 2023-11-08T19:32:03.528 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:03.518+0000 7f930c1d6700 1 -- 172.21.15.143:0/745358603 shutdown_connections 2023-11-08T19:32:03.528 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:03.518+0000 7f930c1d6700 1 --2- 172.21.15.143:0/745358603 >> v2:172.21.15.174:3300/0 conn(0x7f930406b150 0x7f93040b2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:03.528 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:03.518+0000 7f930c1d6700 1 --2- 172.21.15.143:0/745358603 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f930415dd00 0x7f93040cca70 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:03.528 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:03.518+0000 7f930c1d6700 1 --2- 172.21.15.143:0/745358603 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f93041618f0 0x7f930406ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:03.529 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:03.518+0000 7f930c1d6700 1 -- 172.21.15.143:0/745358603 >> 172.21.15.143:0/745358603 conn(0x7f9304099ab0 msgr2=0x7f93040d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:32:03.529 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:03.518+0000 7f930c1d6700 1 -- 172.21.15.143:0/745358603 shutdown_connections 2023-11-08T19:32:03.530 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:03.518+0000 7f930c1d6700 1 -- 172.21.15.143:0/745358603 wait complete. 2023-11-08T19:32:03.530 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:03.519+0000 7f930c1d6700 1 Processor -- start 2023-11-08T19:32:03.530 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:03.519+0000 7f930c1d6700 1 -- start start 2023-11-08T19:32:03.533 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:03.520+0000 7f930c1d6700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f930406b150 0x7f93041f2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:03.534 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:03.520+0000 7f930c1d6700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f930415dd00 0x7f93040c62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:03.534 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:03.520+0000 7f930c1d6700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f93041618f0 0x7f930415bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:03.535 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:03.520+0000 7f930c1d6700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93040b44a0 con 0x7f93041618f0 2023-11-08T19:32:03.535 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:03.520+0000 7f930c1d6700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93040b4780 con 0x7f930415dd00 2023-11-08T19:32:03.535 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:03.520+0000 7f930c1d6700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93040b4a60 con 0x7f930406b150 2023-11-08T19:32:03.536 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:03.520+0000 7f930a773700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f93041618f0 0x7f930415bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:03.536 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:03.520+0000 7f930a773700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f93041618f0 0x7f930415bec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57434/0 (socket says 172.21.15.143:57434) 2023-11-08T19:32:03.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:03.520+0000 7f930a773700 1 -- 172.21.15.143:0/3884197742 learned_addr learned my addr 172.21.15.143:0/3884197742 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:32:03.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:03.520+0000 7f9309f72700 1 --2- 172.21.15.143:0/3884197742 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f930406b150 0x7f93041f2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:03.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:03.520+0000 7f9309771700 1 --2- 172.21.15.143:0/3884197742 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f930415dd00 0x7f93040c62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:03.538 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:03.520+0000 7f930a773700 1 -- 172.21.15.143:0/3884197742 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f930406b150 msgr2=0x7f93041f2fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:03.538 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:03.520+0000 7f930a773700 1 --2- 172.21.15.143:0/3884197742 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f930406b150 0x7f93041f2fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:03.538 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:03.520+0000 7f930a773700 1 -- 172.21.15.143:0/3884197742 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f930415dd00 msgr2=0x7f93040c62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:03.538 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:03.520+0000 7f930a773700 1 --2- 172.21.15.143:0/3884197742 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f930415dd00 0x7f93040c62a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:03.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:03.520+0000 7f930a773700 1 -- 172.21.15.143:0/3884197742 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92f4009500 con 0x7f93041618f0 2023-11-08T19:32:03.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:03.520+0000 7f9309771700 1 --2- 172.21.15.143:0/3884197742 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f930415dd00 0x7f93040c62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:32:03.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:03.520+0000 7f930a773700 1 --2- 172.21.15.143:0/3884197742 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f93041618f0 0x7f930415bec0 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f92fc00cb80 tx=0x7f92fc00cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:03.540 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:03.520+0000 7f92fa7fc700 1 -- 172.21.15.143:0/3884197742 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92fc00a8e0 con 0x7f93041618f0 2023-11-08T19:32:03.540 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:03.520+0000 7f92fa7fc700 1 -- 172.21.15.143:0/3884197742 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92fc00aa40 con 0x7f93041618f0 2023-11-08T19:32:03.540 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:03.520+0000 7f930c1d6700 1 -- 172.21.15.143:0/3884197742 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93040b4d40 con 0x7f93041618f0 2023-11-08T19:32:03.541 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:03.521+0000 7f92fa7fc700 1 -- 172.21.15.143:0/3884197742 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92fc00ae30 con 0x7f93041618f0 2023-11-08T19:32:03.541 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:03.521+0000 7f930c1d6700 1 -- 172.21.15.143:0/3884197742 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93040c6d40 con 0x7f93041618f0 2023-11-08T19:32:03.541 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:03.522+0000 7f930c1d6700 1 -- 172.21.15.143:0/3884197742 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93040b4f60 con 0x7f93041618f0 2023-11-08T19:32:03.541 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:03.522+0000 7f92fa7fc700 1 -- 172.21.15.143:0/3884197742 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f92fc003a20 con 0x7f93041618f0 2023-11-08T19:32:03.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:03.522+0000 7f92fa7fc700 1 --2- 172.21.15.143:0/3884197742 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f92e806af00 0x7f92e806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:03.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:03.522+0000 7f92fa7fc700 1 -- 172.21.15.143:0/3884197742 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6651+0+0 (secure 0 0 0) 0x7f92fc0899c0 con 0x7f93041618f0 2023-11-08T19:32:03.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:03.523+0000 7f9309f72700 1 --2- 172.21.15.143:0/3884197742 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f92e806af00 0x7f92e806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:03.543 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:03.523+0000 7f9309f72700 1 --2- 172.21.15.143:0/3884197742 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f92e806af00 0x7f92e806d3c0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f93040708b0 tx=0x7f92f4008070 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:03.543 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:03.527+0000 7f92fa7fc700 1 -- 172.21.15.143:0/3884197742 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f92fc0546c0 con 0x7f93041618f0 2023-11-08T19:32:03.713 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:03.712+0000 7f930c1d6700 1 -- 172.21.15.143:0/3884197742 --> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f93041623b0 con 0x7f92e806af00 2023-11-08T19:32:03.722 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:03.721+0000 7f92fa7fc700 1 -- 172.21.15.143:0/3884197742 <== mgr.4099 v2:172.21.15.174:6832/4037280178 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+294623 (secure 0 0 0) 0x7f92f4004300 con 0x7f92e806af00 2023-11-08T19:32:03.725 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:32:03.728 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:03.723+0000 7f930c1d6700 1 -- 172.21.15.143:0/3884197742 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f92e806af00 msgr2=0x7f92e806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:03.728 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:03.723+0000 7f930c1d6700 1 --2- 172.21.15.143:0/3884197742 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f92e806af00 0x7f92e806d3c0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f93040708b0 tx=0x7f92f4008070 comp rx=0 tx=0).stop 2023-11-08T19:32:03.729 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:03.723+0000 7f930c1d6700 1 -- 172.21.15.143:0/3884197742 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f93041618f0 msgr2=0x7f930415bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:03.729 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:03.723+0000 7f930c1d6700 1 --2- 172.21.15.143:0/3884197742 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f93041618f0 0x7f930415bec0 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f92fc00cb80 tx=0x7f92fc00cf40 comp rx=0 tx=0).stop 2023-11-08T19:32:03.729 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:03.723+0000 7f930c1d6700 1 -- 172.21.15.143:0/3884197742 shutdown_connections 2023-11-08T19:32:03.730 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:03.723+0000 7f930c1d6700 1 --2- 172.21.15.143:0/3884197742 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f92e806af00 0x7f92e806d3c0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:03.730 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:03.723+0000 7f930c1d6700 1 --2- 172.21.15.143:0/3884197742 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f930406b150 0x7f93041f2fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:03.730 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:03.723+0000 7f930c1d6700 1 --2- 172.21.15.143:0/3884197742 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f93041618f0 0x7f930415bec0 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:03.731 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:03.723+0000 7f930c1d6700 1 --2- 172.21.15.143:0/3884197742 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f930415dd00 0x7f93040c62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:03.731 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:03.723+0000 7f930c1d6700 1 -- 172.21.15.143:0/3884197742 >> 172.21.15.143:0/3884197742 conn(0x7f9304099ab0 msgr2=0x7f93040cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:32:03.731 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:03.724+0000 7f930c1d6700 1 -- 172.21.15.143:0/3884197742 shutdown_connections 2023-11-08T19:32:03.731 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:03.724+0000 7f930c1d6700 1 -- 172.21.15.143:0/3884197742 wait complete. 2023-11-08T19:32:03.732 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2023-11-08T19:32:03.737 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":319,"stamp":"2023-11-08T19:32:02.636852+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":244264,"kb_used_data":11496,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754730456,"statfs":{"total":773094113280,"available":772843986944,"internally_reserved":0,"allocated":11771904,"data_stored":6351912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003430"},"pg_stats":[{"pgid":"12.d","version":"0'0","reported_seq":55,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:22.583090+0000","last_change":"2023-11-08T19:27:08.190826+0000","last_active":"2023-11-08T19:31:22.583090+0000","last_peered":"2023-11-08T19:31:22.583090+0000","last_clean":"2023-11-08T19:31:22.583090+0000","last_became_active":"2023-11-08T19:27:08.190440+0000","last_became_peered":"2023-11-08T19:27:08.190440+0000","last_unstale":"2023-11-08T19:31:22.583090+0000","last_undegraded":"2023-11-08T19:31:22.583090+0000","last_fullsized":"2023-11-08T19:31:22.583090+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"107'119","reported_seq":266,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:27.048714+0000","last_change":"2023-11-08T19:31:27.048761+0000","last_active":"2023-11-08T19:31:27.048714+0000","last_peered":"2023-11-08T19:31:27.048714+0000","last_clean":"2023-11-08T19:31:27.048714+0000","last_became_active":"2023-11-08T19:24:45.887048+0000","last_became_peered":"2023-11-08T19:24:45.887048+0000","last_unstale":"2023-11-08T19:31:27.048714+0000","last_undegraded":"2023-11-08T19:31:27.048714+0000","last_fullsized":"2023-11-08T19:31:27.048714+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"107'119","last_scrub_stamp":"2023-11-08T19:31:27.048604+0000","last_deep_scrub":"107'119","last_deep_scrub_stamp":"2023-11-08T19:31:27.048604+0000","last_clean_scrub_stamp":"2023-11-08T19:31:27.048604+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:16:40.175526+0000","scrub_duration":0.0086873920000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":88,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:32.804593+0000","last_change":"2023-11-08T19:31:32.804615+0000","last_active":"2023-11-08T19:31:32.804593+0000","last_peered":"2023-11-08T19:31:32.804593+0000","last_clean":"2023-11-08T19:31:32.804593+0000","last_became_active":"2023-11-08T19:26:04.603056+0000","last_became_peered":"2023-11-08T19:26:04.603056+0000","last_unstale":"2023-11-08T19:31:32.804593+0000","last_undegraded":"2023-11-08T19:31:32.804593+0000","last_fullsized":"2023-11-08T19:31:32.804593+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:32.804495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:32.804495+0000","last_clean_scrub_stamp":"2023-11-08T19:31:32.804495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:16:22.607276+0000","scrub_duration":0.0015962439999999999,"objects_trimmed":0,"snaptrim_duration":0.000517414,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":68,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:45.765293+0000","last_change":"2023-11-08T19:31:45.765332+0000","last_active":"2023-11-08T19:31:45.765293+0000","last_peered":"2023-11-08T19:31:45.765293+0000","last_clean":"2023-11-08T19:31:45.765293+0000","last_became_active":"2023-11-08T19:27:07.185776+0000","last_became_peered":"2023-11-08T19:27:07.185776+0000","last_unstale":"2023-11-08T19:31:45.765293+0000","last_undegraded":"2023-11-08T19:31:45.765293+0000","last_fullsized":"2023-11-08T19:31:45.765293+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:45.765147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:45.765147+0000","last_clean_scrub_stamp":"2023-11-08T19:31:45.765147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:48:06.344962+0000","scrub_duration":0.0020421279999999998,"objects_trimmed":0,"snaptrim_duration":0.00031747200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":61,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:51.824231+0000","last_change":"2023-11-08T19:31:51.824255+0000","last_active":"2023-11-08T19:31:51.824231+0000","last_peered":"2023-11-08T19:31:51.824231+0000","last_clean":"2023-11-08T19:31:51.824231+0000","last_became_active":"2023-11-08T19:27:08.187863+0000","last_became_peered":"2023-11-08T19:27:08.187863+0000","last_unstale":"2023-11-08T19:31:51.824231+0000","last_undegraded":"2023-11-08T19:31:51.824231+0000","last_fullsized":"2023-11-08T19:31:51.824231+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:51.824119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:51.824119+0000","last_clean_scrub_stamp":"2023-11-08T19:31:51.824119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:42:55.031701+0000","scrub_duration":0.0016592600000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":86,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:42.107550+0000","last_change":"2023-11-08T19:31:42.107596+0000","last_active":"2023-11-08T19:31:42.107550+0000","last_peered":"2023-11-08T19:31:42.107550+0000","last_clean":"2023-11-08T19:31:42.107550+0000","last_became_active":"2023-11-08T19:26:04.605333+0000","last_became_peered":"2023-11-08T19:26:04.605333+0000","last_unstale":"2023-11-08T19:31:42.107550+0000","last_undegraded":"2023-11-08T19:31:42.107550+0000","last_fullsized":"2023-11-08T19:31:42.107550+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:42.107419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:42.107419+0000","last_clean_scrub_stamp":"2023-11-08T19:31:42.107419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:13:05.993137+0000","scrub_duration":0.002168569,"objects_trimmed":0,"snaptrim_duration":0.00064606400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":62,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:22.583048+0000","last_change":"2023-11-08T19:27:09.182113+0000","last_active":"2023-11-08T19:31:22.583048+0000","last_peered":"2023-11-08T19:31:22.583048+0000","last_clean":"2023-11-08T19:31:22.583048+0000","last_became_active":"2023-11-08T19:27:07.189939+0000","last_became_peered":"2023-11-08T19:27:07.189939+0000","last_unstale":"2023-11-08T19:31:22.583048+0000","last_undegraded":"2023-11-08T19:31:22.583048+0000","last_fullsized":"2023-11-08T19:31:22.583048+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036507500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":57,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:41.865868+0000","last_change":"2023-11-08T19:31:41.865899+0000","last_active":"2023-11-08T19:31:41.865868+0000","last_peered":"2023-11-08T19:31:41.865868+0000","last_clean":"2023-11-08T19:31:41.865868+0000","last_became_active":"2023-11-08T19:27:08.182648+0000","last_became_peered":"2023-11-08T19:27:08.182648+0000","last_unstale":"2023-11-08T19:31:41.865868+0000","last_undegraded":"2023-11-08T19:31:41.865868+0000","last_fullsized":"2023-11-08T19:31:41.865868+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:41.865752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:41.865752+0000","last_clean_scrub_stamp":"2023-11-08T19:31:41.865752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:05:41.475657+0000","scrub_duration":0.0018669470000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":98,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:58.949917+0000","last_change":"2023-11-08T19:31:58.949943+0000","last_active":"2023-11-08T19:31:58.949917+0000","last_peered":"2023-11-08T19:31:58.949917+0000","last_clean":"2023-11-08T19:31:58.949917+0000","last_became_active":"2023-11-08T19:26:04.608379+0000","last_became_peered":"2023-11-08T19:26:04.608379+0000","last_unstale":"2023-11-08T19:31:58.949917+0000","last_undegraded":"2023-11-08T19:31:58.949917+0000","last_fullsized":"2023-11-08T19:31:58.949917+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:58.949790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:58.949790+0000","last_clean_scrub_stamp":"2023-11-08T19:31:58.949790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:04:49.781093+0000","scrub_duration":0.001585752,"objects_trimmed":0,"snaptrim_duration":0.000529657,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":64,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:54.122758+0000","last_change":"2023-11-08T19:31:54.122795+0000","last_active":"2023-11-08T19:31:54.122758+0000","last_peered":"2023-11-08T19:31:54.122758+0000","last_clean":"2023-11-08T19:31:54.122758+0000","last_became_active":"2023-11-08T19:27:07.195231+0000","last_became_peered":"2023-11-08T19:27:07.195231+0000","last_unstale":"2023-11-08T19:31:54.122758+0000","last_undegraded":"2023-11-08T19:31:54.122758+0000","last_fullsized":"2023-11-08T19:31:54.122758+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:54.122605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:54.122605+0000","last_clean_scrub_stamp":"2023-11-08T19:31:54.122605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:43:15.004601+0000","scrub_duration":0.002189346,"objects_trimmed":0,"snaptrim_duration":0.000441221,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":57,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:48.764425+0000","last_change":"2023-11-08T19:31:48.764447+0000","last_active":"2023-11-08T19:31:48.764425+0000","last_peered":"2023-11-08T19:31:48.764425+0000","last_clean":"2023-11-08T19:31:48.764425+0000","last_became_active":"2023-11-08T19:27:08.195074+0000","last_became_peered":"2023-11-08T19:27:08.195074+0000","last_unstale":"2023-11-08T19:31:48.764425+0000","last_undegraded":"2023-11-08T19:31:48.764425+0000","last_fullsized":"2023-11-08T19:31:48.764425+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:48.764327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:48.764327+0000","last_clean_scrub_stamp":"2023-11-08T19:31:48.764327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:30:53.197298+0000","scrub_duration":0.001484452,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":86,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:29.665197+0000","last_change":"2023-11-08T19:31:29.665227+0000","last_active":"2023-11-08T19:31:29.665197+0000","last_peered":"2023-11-08T19:31:29.665197+0000","last_clean":"2023-11-08T19:31:29.665197+0000","last_became_active":"2023-11-08T19:26:04.601685+0000","last_became_peered":"2023-11-08T19:26:04.601685+0000","last_unstale":"2023-11-08T19:31:29.665197+0000","last_undegraded":"2023-11-08T19:31:29.665197+0000","last_fullsized":"2023-11-08T19:31:29.665197+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:29.665096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:29.665096+0000","last_clean_scrub_stamp":"2023-11-08T19:31:29.665096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:57:56.435973+0000","scrub_duration":0.001580171,"objects_trimmed":0,"snaptrim_duration":0.00061044099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":64,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:34.809538+0000","last_change":"2023-11-08T19:31:34.809538+0000","last_active":"2023-11-08T19:31:34.809538+0000","last_peered":"2023-11-08T19:31:34.809538+0000","last_clean":"2023-11-08T19:31:34.809538+0000","last_became_active":"2023-11-08T19:27:07.189386+0000","last_became_peered":"2023-11-08T19:27:07.189386+0000","last_unstale":"2023-11-08T19:31:34.809538+0000","last_undegraded":"2023-11-08T19:31:34.809538+0000","last_fullsized":"2023-11-08T19:31:34.809538+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:34.809370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:34.809370+0000","last_clean_scrub_stamp":"2023-11-08T19:31:34.809370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:04:11.405772+0000","scrub_duration":0.0021291940000000001,"objects_trimmed":0,"snaptrim_duration":0.00049390100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":61,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:57.832690+0000","last_change":"2023-11-08T19:31:57.832728+0000","last_active":"2023-11-08T19:31:57.832690+0000","last_peered":"2023-11-08T19:31:57.832690+0000","last_clean":"2023-11-08T19:31:57.832690+0000","last_became_active":"2023-11-08T19:27:08.189088+0000","last_became_peered":"2023-11-08T19:27:08.189088+0000","last_unstale":"2023-11-08T19:31:57.832690+0000","last_undegraded":"2023-11-08T19:31:57.832690+0000","last_fullsized":"2023-11-08T19:31:57.832690+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:57.832572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:57.832572+0000","last_clean_scrub_stamp":"2023-11-08T19:31:57.832572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:00:29.635279+0000","scrub_duration":0.0017369670000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":87,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:21.041246+0000","last_change":"2023-11-08T19:31:21.041265+0000","last_active":"2023-11-08T19:31:21.041246+0000","last_peered":"2023-11-08T19:31:21.041246+0000","last_clean":"2023-11-08T19:31:21.041246+0000","last_became_active":"2023-11-08T19:26:04.603088+0000","last_became_peered":"2023-11-08T19:26:04.603088+0000","last_unstale":"2023-11-08T19:31:21.041246+0000","last_undegraded":"2023-11-08T19:31:21.041246+0000","last_fullsized":"2023-11-08T19:31:21.041246+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:21.041166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:21.041166+0000","last_clean_scrub_stamp":"2023-11-08T19:31:21.041166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:25:28.147920+0000","scrub_duration":0.001667876,"objects_trimmed":0,"snaptrim_duration":0.00038180899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":64,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:28.086650+0000","last_change":"2023-11-08T19:31:28.086650+0000","last_active":"2023-11-08T19:31:28.086650+0000","last_peered":"2023-11-08T19:31:28.086650+0000","last_clean":"2023-11-08T19:31:28.086650+0000","last_became_active":"2023-11-08T19:27:07.191589+0000","last_became_peered":"2023-11-08T19:27:07.191589+0000","last_unstale":"2023-11-08T19:31:28.086650+0000","last_undegraded":"2023-11-08T19:31:28.086650+0000","last_fullsized":"2023-11-08T19:31:28.086650+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:28.086495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:28.086495+0000","last_clean_scrub_stamp":"2023-11-08T19:31:28.086495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:17:02.712148+0000","scrub_duration":0.0016141980000000001,"objects_trimmed":0,"snaptrim_duration":0.000729527,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":57,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:53.564766+0000","last_change":"2023-11-08T19:31:53.564800+0000","last_active":"2023-11-08T19:31:53.564766+0000","last_peered":"2023-11-08T19:31:53.564766+0000","last_clean":"2023-11-08T19:31:53.564766+0000","last_became_active":"2023-11-08T19:27:08.192997+0000","last_became_peered":"2023-11-08T19:27:08.192997+0000","last_unstale":"2023-11-08T19:31:53.564766+0000","last_undegraded":"2023-11-08T19:31:53.564766+0000","last_fullsized":"2023-11-08T19:31:53.564766+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:53.564655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:53.564655+0000","last_clean_scrub_stamp":"2023-11-08T19:31:53.564655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:17:37.295865+0000","scrub_duration":0.0019884429999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":92,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:43.790033+0000","last_change":"2023-11-08T19:31:43.790051+0000","last_active":"2023-11-08T19:31:43.790033+0000","last_peered":"2023-11-08T19:31:43.790033+0000","last_clean":"2023-11-08T19:31:43.790033+0000","last_became_active":"2023-11-08T19:26:04.598973+0000","last_became_peered":"2023-11-08T19:26:04.598973+0000","last_unstale":"2023-11-08T19:31:43.790033+0000","last_undegraded":"2023-11-08T19:31:43.790033+0000","last_fullsized":"2023-11-08T19:31:43.790033+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:43.789955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:43.789955+0000","last_clean_scrub_stamp":"2023-11-08T19:31:43.789955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:00:05.697661+0000","scrub_duration":0.0016943609999999999,"objects_trimmed":0,"snaptrim_duration":0.00031755900000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":68,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:47.752890+0000","last_change":"2023-11-08T19:31:47.752912+0000","last_active":"2023-11-08T19:31:47.752890+0000","last_peered":"2023-11-08T19:31:47.752890+0000","last_clean":"2023-11-08T19:31:47.752890+0000","last_became_active":"2023-11-08T19:27:07.190132+0000","last_became_peered":"2023-11-08T19:27:07.190132+0000","last_unstale":"2023-11-08T19:31:47.752890+0000","last_undegraded":"2023-11-08T19:31:47.752890+0000","last_fullsized":"2023-11-08T19:31:47.752890+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:47.752785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:47.752785+0000","last_clean_scrub_stamp":"2023-11-08T19:31:47.752785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:31:24.364068+0000","scrub_duration":0.0014687420000000001,"objects_trimmed":0,"snaptrim_duration":0.00054126000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":57,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:45.481002+0000","last_change":"2023-11-08T19:31:45.481039+0000","last_active":"2023-11-08T19:31:45.481002+0000","last_peered":"2023-11-08T19:31:45.481002+0000","last_clean":"2023-11-08T19:31:45.481002+0000","last_became_active":"2023-11-08T19:27:08.193912+0000","last_became_peered":"2023-11-08T19:27:08.193912+0000","last_unstale":"2023-11-08T19:31:45.481002+0000","last_undegraded":"2023-11-08T19:31:45.481002+0000","last_fullsized":"2023-11-08T19:31:45.481002+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:45.480846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:45.480846+0000","last_clean_scrub_stamp":"2023-11-08T19:31:45.480846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:27:16.952754+0000","scrub_duration":0.0021767169999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":87,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:22.069061+0000","last_change":"2023-11-08T19:31:22.069101+0000","last_active":"2023-11-08T19:31:22.069061+0000","last_peered":"2023-11-08T19:31:22.069061+0000","last_clean":"2023-11-08T19:31:22.069061+0000","last_became_active":"2023-11-08T19:26:04.599999+0000","last_became_peered":"2023-11-08T19:26:04.599999+0000","last_unstale":"2023-11-08T19:31:22.069061+0000","last_undegraded":"2023-11-08T19:31:22.069061+0000","last_fullsized":"2023-11-08T19:31:22.069061+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:22.068925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:22.068925+0000","last_clean_scrub_stamp":"2023-11-08T19:31:22.068925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:59:28.785563+0000","scrub_duration":0.0018102260000000001,"objects_trimmed":0,"snaptrim_duration":0.00025605700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":68,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:46.723527+0000","last_change":"2023-11-08T19:31:46.723563+0000","last_active":"2023-11-08T19:31:46.723527+0000","last_peered":"2023-11-08T19:31:46.723527+0000","last_clean":"2023-11-08T19:31:46.723527+0000","last_became_active":"2023-11-08T19:27:07.189135+0000","last_became_peered":"2023-11-08T19:27:07.189135+0000","last_unstale":"2023-11-08T19:31:46.723527+0000","last_undegraded":"2023-11-08T19:31:46.723527+0000","last_fullsized":"2023-11-08T19:31:46.723527+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:46.723408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:46.723408+0000","last_clean_scrub_stamp":"2023-11-08T19:31:46.723408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:20:55.064105+0000","scrub_duration":0.002142251,"objects_trimmed":0,"snaptrim_duration":0.00051794299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":63,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:34.286026+0000","last_change":"2023-11-08T19:31:34.286026+0000","last_active":"2023-11-08T19:31:34.286026+0000","last_peered":"2023-11-08T19:31:34.286026+0000","last_clean":"2023-11-08T19:31:34.286026+0000","last_became_active":"2023-11-08T19:27:08.186643+0000","last_became_peered":"2023-11-08T19:27:08.186643+0000","last_unstale":"2023-11-08T19:31:34.286026+0000","last_undegraded":"2023-11-08T19:31:34.286026+0000","last_fullsized":"2023-11-08T19:31:34.286026+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:34.285872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:34.285872+0000","last_clean_scrub_stamp":"2023-11-08T19:31:34.285872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:22:31.137393+0000","scrub_duration":0.0015767699999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":98,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:44.925918+0000","last_change":"2023-11-08T19:31:44.925942+0000","last_active":"2023-11-08T19:31:44.925918+0000","last_peered":"2023-11-08T19:31:44.925918+0000","last_clean":"2023-11-08T19:31:44.925918+0000","last_became_active":"2023-11-08T19:26:04.603320+0000","last_became_peered":"2023-11-08T19:26:04.603320+0000","last_unstale":"2023-11-08T19:31:44.925918+0000","last_undegraded":"2023-11-08T19:31:44.925918+0000","last_fullsized":"2023-11-08T19:31:44.925918+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:44.925801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:44.925801+0000","last_clean_scrub_stamp":"2023-11-08T19:31:44.925801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:15:37.041939+0000","scrub_duration":0.0014464549999999999,"objects_trimmed":0,"snaptrim_duration":0.00063225,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":68,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:48.744345+0000","last_change":"2023-11-08T19:31:48.744367+0000","last_active":"2023-11-08T19:31:48.744345+0000","last_peered":"2023-11-08T19:31:48.744345+0000","last_clean":"2023-11-08T19:31:48.744345+0000","last_became_active":"2023-11-08T19:27:07.189895+0000","last_became_peered":"2023-11-08T19:27:07.189895+0000","last_unstale":"2023-11-08T19:31:48.744345+0000","last_undegraded":"2023-11-08T19:31:48.744345+0000","last_fullsized":"2023-11-08T19:31:48.744345+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:48.744245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:48.744245+0000","last_clean_scrub_stamp":"2023-11-08T19:31:48.744245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:40:58.059066+0000","scrub_duration":0.0018029750000000001,"objects_trimmed":0,"snaptrim_duration":0.00045654199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":88,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:24.725788+0000","last_change":"2023-11-08T19:31:24.725829+0000","last_active":"2023-11-08T19:31:24.725788+0000","last_peered":"2023-11-08T19:31:24.725788+0000","last_clean":"2023-11-08T19:31:24.725788+0000","last_became_active":"2023-11-08T19:26:04.603725+0000","last_became_peered":"2023-11-08T19:26:04.603725+0000","last_unstale":"2023-11-08T19:31:24.725788+0000","last_undegraded":"2023-11-08T19:31:24.725788+0000","last_fullsized":"2023-11-08T19:31:24.725788+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:24.725673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:24.725673+0000","last_clean_scrub_stamp":"2023-11-08T19:31:24.725673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:15:54.517242+0000","scrub_duration":0.00144779,"objects_trimmed":0,"snaptrim_duration":0.00053365400000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":64,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:35.727927+0000","last_change":"2023-11-08T19:31:35.727950+0000","last_active":"2023-11-08T19:31:35.727927+0000","last_peered":"2023-11-08T19:31:35.727927+0000","last_clean":"2023-11-08T19:31:35.727927+0000","last_became_active":"2023-11-08T19:27:07.189595+0000","last_became_peered":"2023-11-08T19:27:07.189595+0000","last_unstale":"2023-11-08T19:31:35.727927+0000","last_undegraded":"2023-11-08T19:31:35.727927+0000","last_fullsized":"2023-11-08T19:31:35.727927+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:35.727812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:35.727812+0000","last_clean_scrub_stamp":"2023-11-08T19:31:35.727812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:00:59.786393+0000","scrub_duration":0.0019661430000000001,"objects_trimmed":0,"snaptrim_duration":0.000323479,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":57,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:55.095220+0000","last_change":"2023-11-08T19:31:55.095261+0000","last_active":"2023-11-08T19:31:55.095220+0000","last_peered":"2023-11-08T19:31:55.095220+0000","last_clean":"2023-11-08T19:31:55.095220+0000","last_became_active":"2023-11-08T19:27:08.184765+0000","last_became_peered":"2023-11-08T19:27:08.184765+0000","last_unstale":"2023-11-08T19:31:55.095220+0000","last_undegraded":"2023-11-08T19:31:55.095220+0000","last_fullsized":"2023-11-08T19:31:55.095220+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:55.095069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:55.095069+0000","last_clean_scrub_stamp":"2023-11-08T19:31:55.095069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:54:29.082954+0000","scrub_duration":0.001731813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":92,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:32.544440+0000","last_change":"2023-11-08T19:31:32.544462+0000","last_active":"2023-11-08T19:31:32.544440+0000","last_peered":"2023-11-08T19:31:32.544440+0000","last_clean":"2023-11-08T19:31:32.544440+0000","last_became_active":"2023-11-08T19:26:04.600115+0000","last_became_peered":"2023-11-08T19:26:04.600115+0000","last_unstale":"2023-11-08T19:31:32.544440+0000","last_undegraded":"2023-11-08T19:31:32.544440+0000","last_fullsized":"2023-11-08T19:31:32.544440+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:32.544303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:32.544303+0000","last_clean_scrub_stamp":"2023-11-08T19:31:32.544303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:35:56.347935+0000","scrub_duration":0.002175448,"objects_trimmed":0,"snaptrim_duration":0.00065803900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":64,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:43.455210+0000","last_change":"2023-11-08T19:31:43.455229+0000","last_active":"2023-11-08T19:31:43.455210+0000","last_peered":"2023-11-08T19:31:43.455210+0000","last_clean":"2023-11-08T19:31:43.455210+0000","last_became_active":"2023-11-08T19:27:07.191460+0000","last_became_peered":"2023-11-08T19:27:07.191460+0000","last_unstale":"2023-11-08T19:31:43.455210+0000","last_undegraded":"2023-11-08T19:31:43.455210+0000","last_fullsized":"2023-11-08T19:31:43.455210+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:43.455118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:43.455118+0000","last_clean_scrub_stamp":"2023-11-08T19:31:43.455118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:48:33.020893+0000","scrub_duration":0.001194278,"objects_trimmed":0,"snaptrim_duration":0.00044907500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":63,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:43.455214+0000","last_change":"2023-11-08T19:31:43.455237+0000","last_active":"2023-11-08T19:31:43.455214+0000","last_peered":"2023-11-08T19:31:43.455214+0000","last_clean":"2023-11-08T19:31:43.455214+0000","last_became_active":"2023-11-08T19:27:08.191520+0000","last_became_peered":"2023-11-08T19:27:08.191520+0000","last_unstale":"2023-11-08T19:31:43.455214+0000","last_undegraded":"2023-11-08T19:31:43.455214+0000","last_fullsized":"2023-11-08T19:31:43.455214+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:43.455101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:43.455101+0000","last_clean_scrub_stamp":"2023-11-08T19:31:43.455101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:19:44.483623+0000","scrub_duration":0.001510302,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":98,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:29.015607+0000","last_change":"2023-11-08T19:31:29.015630+0000","last_active":"2023-11-08T19:31:29.015607+0000","last_peered":"2023-11-08T19:31:29.015607+0000","last_clean":"2023-11-08T19:31:29.015607+0000","last_became_active":"2023-11-08T19:26:04.604519+0000","last_became_peered":"2023-11-08T19:26:04.604519+0000","last_unstale":"2023-11-08T19:31:29.015607+0000","last_undegraded":"2023-11-08T19:31:29.015607+0000","last_fullsized":"2023-11-08T19:31:29.015607+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:29.015490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:29.015490+0000","last_clean_scrub_stamp":"2023-11-08T19:31:29.015490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:35:52.819122+0000","scrub_duration":0.0020686609999999999,"objects_trimmed":0,"snaptrim_duration":0.00051248099999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":62,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:19.628717+0000","last_change":"2023-11-08T19:27:09.181431+0000","last_active":"2023-11-08T19:31:19.628717+0000","last_peered":"2023-11-08T19:31:19.628717+0000","last_clean":"2023-11-08T19:31:19.628717+0000","last_became_active":"2023-11-08T19:27:07.190800+0000","last_became_peered":"2023-11-08T19:27:07.190800+0000","last_unstale":"2023-11-08T19:31:19.628717+0000","last_undegraded":"2023-11-08T19:31:19.628717+0000","last_fullsized":"2023-11-08T19:31:19.628717+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656099999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":49,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:18.639811+0000","last_change":"2023-11-08T19:27:08.187983+0000","last_active":"2023-11-08T19:31:18.639811+0000","last_peered":"2023-11-08T19:31:18.639811+0000","last_clean":"2023-11-08T19:31:18.639811+0000","last_became_active":"2023-11-08T19:27:08.187322+0000","last_became_peered":"2023-11-08T19:27:08.187322+0000","last_unstale":"2023-11-08T19:31:18.639811+0000","last_undegraded":"2023-11-08T19:31:18.639811+0000","last_fullsized":"2023-11-08T19:31:18.639811+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":86,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:22.798579+0000","last_change":"2023-11-08T19:31:22.798598+0000","last_active":"2023-11-08T19:31:22.798579+0000","last_peered":"2023-11-08T19:31:22.798579+0000","last_clean":"2023-11-08T19:31:22.798579+0000","last_became_active":"2023-11-08T19:26:04.606967+0000","last_became_peered":"2023-11-08T19:26:04.606967+0000","last_unstale":"2023-11-08T19:31:22.798579+0000","last_undegraded":"2023-11-08T19:31:22.798579+0000","last_fullsized":"2023-11-08T19:31:22.798579+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:22.798496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:22.798496+0000","last_clean_scrub_stamp":"2023-11-08T19:31:22.798496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:28:48.162603+0000","scrub_duration":0.002120331,"objects_trimmed":0,"snaptrim_duration":0.00040971999999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":62,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:19.628658+0000","last_change":"2023-11-08T19:27:09.180722+0000","last_active":"2023-11-08T19:31:19.628658+0000","last_peered":"2023-11-08T19:31:19.628658+0000","last_clean":"2023-11-08T19:31:19.628658+0000","last_became_active":"2023-11-08T19:27:07.186977+0000","last_became_peered":"2023-11-08T19:27:07.186977+0000","last_unstale":"2023-11-08T19:31:19.628658+0000","last_undegraded":"2023-11-08T19:31:19.628658+0000","last_fullsized":"2023-11-08T19:31:19.628658+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057675100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":57,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:58.458746+0000","last_change":"2023-11-08T19:31:58.458785+0000","last_active":"2023-11-08T19:31:58.458746+0000","last_peered":"2023-11-08T19:31:58.458746+0000","last_clean":"2023-11-08T19:31:58.458746+0000","last_became_active":"2023-11-08T19:27:08.193098+0000","last_became_peered":"2023-11-08T19:27:08.193098+0000","last_unstale":"2023-11-08T19:31:58.458746+0000","last_undegraded":"2023-11-08T19:31:58.458746+0000","last_fullsized":"2023-11-08T19:31:58.458746+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:58.458626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:58.458626+0000","last_clean_scrub_stamp":"2023-11-08T19:31:58.458626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:26:45.661191+0000","scrub_duration":0.0023236260000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":86,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:33.077049+0000","last_change":"2023-11-08T19:31:33.077074+0000","last_active":"2023-11-08T19:31:33.077049+0000","last_peered":"2023-11-08T19:31:33.077049+0000","last_clean":"2023-11-08T19:31:33.077049+0000","last_became_active":"2023-11-08T19:26:04.602111+0000","last_became_peered":"2023-11-08T19:26:04.602111+0000","last_unstale":"2023-11-08T19:31:33.077049+0000","last_undegraded":"2023-11-08T19:31:33.077049+0000","last_fullsized":"2023-11-08T19:31:33.077049+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:33.076926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:33.076926+0000","last_clean_scrub_stamp":"2023-11-08T19:31:33.076926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:09:46.810104+0000","scrub_duration":0.0022106909999999999,"objects_trimmed":0,"snaptrim_duration":0.00056197899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":70,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:22.258974+0000","last_change":"2023-11-08T19:31:22.259009+0000","last_active":"2023-11-08T19:31:22.258974+0000","last_peered":"2023-11-08T19:31:22.258974+0000","last_clean":"2023-11-08T19:31:22.258974+0000","last_became_active":"2023-11-08T19:27:07.188657+0000","last_became_peered":"2023-11-08T19:27:07.188657+0000","last_unstale":"2023-11-08T19:31:22.258974+0000","last_undegraded":"2023-11-08T19:31:22.258974+0000","last_fullsized":"2023-11-08T19:31:22.258974+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:22.258851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:22.258851+0000","last_clean_scrub_stamp":"2023-11-08T19:31:22.258851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:41:50.529438+0000","scrub_duration":0.001495632,"objects_trimmed":0,"snaptrim_duration":0.001227393,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":63,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:31.191488+0000","last_change":"2023-11-08T19:31:31.191513+0000","last_active":"2023-11-08T19:31:31.191488+0000","last_peered":"2023-11-08T19:31:31.191488+0000","last_clean":"2023-11-08T19:31:31.191488+0000","last_became_active":"2023-11-08T19:27:08.188240+0000","last_became_peered":"2023-11-08T19:27:08.188240+0000","last_unstale":"2023-11-08T19:31:31.191488+0000","last_undegraded":"2023-11-08T19:31:31.191488+0000","last_fullsized":"2023-11-08T19:31:31.191488+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:31.191371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:31.191371+0000","last_clean_scrub_stamp":"2023-11-08T19:31:31.191371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:35:54.995003+0000","scrub_duration":0.001875533,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":63,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:39.374627+0000","last_change":"2023-11-08T19:31:39.374649+0000","last_active":"2023-11-08T19:31:39.374627+0000","last_peered":"2023-11-08T19:31:39.374627+0000","last_clean":"2023-11-08T19:31:39.374627+0000","last_became_active":"2023-11-08T19:27:08.187880+0000","last_became_peered":"2023-11-08T19:27:08.187880+0000","last_unstale":"2023-11-08T19:31:39.374627+0000","last_undegraded":"2023-11-08T19:31:39.374627+0000","last_fullsized":"2023-11-08T19:31:39.374627+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:39.374517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:39.374517+0000","last_clean_scrub_stamp":"2023-11-08T19:31:39.374517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:00:01.282223+0000","scrub_duration":0.0021558079999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":87,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:27.808067+0000","last_change":"2023-11-08T19:31:27.808093+0000","last_active":"2023-11-08T19:31:27.808067+0000","last_peered":"2023-11-08T19:31:27.808067+0000","last_clean":"2023-11-08T19:31:27.808067+0000","last_became_active":"2023-11-08T19:26:04.603955+0000","last_became_peered":"2023-11-08T19:26:04.603955+0000","last_unstale":"2023-11-08T19:31:27.808067+0000","last_undegraded":"2023-11-08T19:31:27.808067+0000","last_fullsized":"2023-11-08T19:31:27.808067+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:27.807939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:27.807939+0000","last_clean_scrub_stamp":"2023-11-08T19:31:27.807939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:20:39.173012+0000","scrub_duration":0.0016595620000000001,"objects_trimmed":0,"snaptrim_duration":0.00036473499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":64,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:39.814570+0000","last_change":"2023-11-08T19:31:39.814593+0000","last_active":"2023-11-08T19:31:39.814570+0000","last_peered":"2023-11-08T19:31:39.814570+0000","last_clean":"2023-11-08T19:31:39.814570+0000","last_became_active":"2023-11-08T19:27:07.194992+0000","last_became_peered":"2023-11-08T19:27:07.194992+0000","last_unstale":"2023-11-08T19:31:39.814570+0000","last_undegraded":"2023-11-08T19:31:39.814570+0000","last_fullsized":"2023-11-08T19:31:39.814570+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:39.814469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:39.814469+0000","last_clean_scrub_stamp":"2023-11-08T19:31:39.814469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:41:17.129654+0000","scrub_duration":0.001378043,"objects_trimmed":0,"snaptrim_duration":0.000770285,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":57,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:48.263043+0000","last_change":"2023-11-08T19:31:48.263067+0000","last_active":"2023-11-08T19:31:48.263043+0000","last_peered":"2023-11-08T19:31:48.263043+0000","last_clean":"2023-11-08T19:31:48.263043+0000","last_became_active":"2023-11-08T19:27:08.193653+0000","last_became_peered":"2023-11-08T19:27:08.193653+0000","last_unstale":"2023-11-08T19:31:48.263043+0000","last_undegraded":"2023-11-08T19:31:48.263043+0000","last_fullsized":"2023-11-08T19:31:48.263043+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:48.262912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:48.262912+0000","last_clean_scrub_stamp":"2023-11-08T19:31:48.262912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:43:09.144908+0000","scrub_duration":0.0015807149999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":92,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:33.590064+0000","last_change":"2023-11-08T19:31:33.590087+0000","last_active":"2023-11-08T19:31:33.590064+0000","last_peered":"2023-11-08T19:31:33.590064+0000","last_clean":"2023-11-08T19:31:33.590064+0000","last_became_active":"2023-11-08T19:26:04.603144+0000","last_became_peered":"2023-11-08T19:26:04.603144+0000","last_unstale":"2023-11-08T19:31:33.590064+0000","last_undegraded":"2023-11-08T19:31:33.590064+0000","last_fullsized":"2023-11-08T19:31:33.590064+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:33.589962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:33.589962+0000","last_clean_scrub_stamp":"2023-11-08T19:31:33.589962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:22:30.441483+0000","scrub_duration":0.0016553749999999999,"objects_trimmed":0,"snaptrim_duration":0.00054090799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":70,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:26.196071+0000","last_change":"2023-11-08T19:31:26.196111+0000","last_active":"2023-11-08T19:31:26.196071+0000","last_peered":"2023-11-08T19:31:26.196071+0000","last_clean":"2023-11-08T19:31:26.196071+0000","last_became_active":"2023-11-08T19:27:07.190438+0000","last_became_peered":"2023-11-08T19:27:07.190438+0000","last_unstale":"2023-11-08T19:31:26.196071+0000","last_undegraded":"2023-11-08T19:31:26.196071+0000","last_fullsized":"2023-11-08T19:31:26.196071+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:26.195915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:26.195915+0000","last_clean_scrub_stamp":"2023-11-08T19:31:26.195915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:09:39.929093+0000","scrub_duration":0.0021248539999999998,"objects_trimmed":0,"snaptrim_duration":0.00050893300000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":57,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:46.249163+0000","last_change":"2023-11-08T19:31:46.249191+0000","last_active":"2023-11-08T19:31:46.249163+0000","last_peered":"2023-11-08T19:31:46.249163+0000","last_clean":"2023-11-08T19:31:46.249163+0000","last_became_active":"2023-11-08T19:27:08.190203+0000","last_became_peered":"2023-11-08T19:27:08.190203+0000","last_unstale":"2023-11-08T19:31:46.249163+0000","last_undegraded":"2023-11-08T19:31:46.249163+0000","last_fullsized":"2023-11-08T19:31:46.249163+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:46.249056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:46.249056+0000","last_clean_scrub_stamp":"2023-11-08T19:31:46.249056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:30:06.625263+0000","scrub_duration":0.0018596820000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":88,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:26.732249+0000","last_change":"2023-11-08T19:31:26.732275+0000","last_active":"2023-11-08T19:31:26.732249+0000","last_peered":"2023-11-08T19:31:26.732249+0000","last_clean":"2023-11-08T19:31:26.732249+0000","last_became_active":"2023-11-08T19:26:04.601961+0000","last_became_peered":"2023-11-08T19:26:04.601961+0000","last_unstale":"2023-11-08T19:31:26.732249+0000","last_undegraded":"2023-11-08T19:31:26.732249+0000","last_fullsized":"2023-11-08T19:31:26.732249+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:26.732141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:26.732141+0000","last_clean_scrub_stamp":"2023-11-08T19:31:26.732141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:25:33.838895+0000","scrub_duration":0.0020533159999999999,"objects_trimmed":0,"snaptrim_duration":0.000466386,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":64,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:26.031920+0000","last_change":"2023-11-08T19:31:26.031963+0000","last_active":"2023-11-08T19:31:26.031920+0000","last_peered":"2023-11-08T19:31:26.031920+0000","last_clean":"2023-11-08T19:31:26.031920+0000","last_became_active":"2023-11-08T19:27:07.191583+0000","last_became_peered":"2023-11-08T19:27:07.191583+0000","last_unstale":"2023-11-08T19:31:26.031920+0000","last_undegraded":"2023-11-08T19:31:26.031920+0000","last_fullsized":"2023-11-08T19:31:26.031920+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:26.031797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:26.031797+0000","last_clean_scrub_stamp":"2023-11-08T19:31:26.031797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:09:39.764974+0000","scrub_duration":0.0015837099999999999,"objects_trimmed":0,"snaptrim_duration":0.00093695200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":57,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:29.050922+0000","last_change":"2023-11-08T19:31:29.050922+0000","last_active":"2023-11-08T19:31:29.050922+0000","last_peered":"2023-11-08T19:31:29.050922+0000","last_clean":"2023-11-08T19:31:29.050922+0000","last_became_active":"2023-11-08T19:27:08.187626+0000","last_became_peered":"2023-11-08T19:27:08.187626+0000","last_unstale":"2023-11-08T19:31:29.050922+0000","last_undegraded":"2023-11-08T19:31:29.050922+0000","last_fullsized":"2023-11-08T19:31:29.050922+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:29.050774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:29.050774+0000","last_clean_scrub_stamp":"2023-11-08T19:31:29.050774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:58:25.326897+0000","scrub_duration":0.001829125,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":98,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:51.959971+0000","last_change":"2023-11-08T19:31:51.959999+0000","last_active":"2023-11-08T19:31:51.959971+0000","last_peered":"2023-11-08T19:31:51.959971+0000","last_clean":"2023-11-08T19:31:51.959971+0000","last_became_active":"2023-11-08T19:26:04.606386+0000","last_became_peered":"2023-11-08T19:26:04.606386+0000","last_unstale":"2023-11-08T19:31:51.959971+0000","last_undegraded":"2023-11-08T19:31:51.959971+0000","last_fullsized":"2023-11-08T19:31:51.959971+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:51.959862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:51.959862+0000","last_clean_scrub_stamp":"2023-11-08T19:31:51.959862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:54:20.974301+0000","scrub_duration":0.001831734,"objects_trimmed":0,"snaptrim_duration":0.00037950599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":64,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:36.795896+0000","last_change":"2023-11-08T19:31:36.795921+0000","last_active":"2023-11-08T19:31:36.795896+0000","last_peered":"2023-11-08T19:31:36.795896+0000","last_clean":"2023-11-08T19:31:36.795896+0000","last_became_active":"2023-11-08T19:27:07.195033+0000","last_became_peered":"2023-11-08T19:27:07.195033+0000","last_unstale":"2023-11-08T19:31:36.795896+0000","last_undegraded":"2023-11-08T19:31:36.795896+0000","last_fullsized":"2023-11-08T19:31:36.795896+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:36.795786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:36.795786+0000","last_clean_scrub_stamp":"2023-11-08T19:31:36.795786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:34:05.177568+0000","scrub_duration":0.0014630890000000001,"objects_trimmed":0,"snaptrim_duration":0.00041824800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":57,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:46.454024+0000","last_change":"2023-11-08T19:31:46.454024+0000","last_active":"2023-11-08T19:31:46.454024+0000","last_peered":"2023-11-08T19:31:46.454024+0000","last_clean":"2023-11-08T19:31:46.454024+0000","last_became_active":"2023-11-08T19:27:08.193017+0000","last_became_peered":"2023-11-08T19:27:08.193017+0000","last_unstale":"2023-11-08T19:31:46.454024+0000","last_undegraded":"2023-11-08T19:31:46.454024+0000","last_fullsized":"2023-11-08T19:31:46.454024+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:46.453891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:46.453891+0000","last_clean_scrub_stamp":"2023-11-08T19:31:46.453891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:04:39.925346+0000","scrub_duration":0.0018276740000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":92,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:37.663409+0000","last_change":"2023-11-08T19:31:37.663435+0000","last_active":"2023-11-08T19:31:37.663409+0000","last_peered":"2023-11-08T19:31:37.663409+0000","last_clean":"2023-11-08T19:31:37.663409+0000","last_became_active":"2023-11-08T19:26:04.599046+0000","last_became_peered":"2023-11-08T19:26:04.599046+0000","last_unstale":"2023-11-08T19:31:37.663409+0000","last_undegraded":"2023-11-08T19:31:37.663409+0000","last_fullsized":"2023-11-08T19:31:37.663409+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:37.663280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:37.663280+0000","last_clean_scrub_stamp":"2023-11-08T19:31:37.663280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:20:56.015718+0000","scrub_duration":0.0015101190000000001,"objects_trimmed":0,"snaptrim_duration":0.00035092199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":62,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:14.017969+0000","last_change":"2023-11-08T19:27:09.181716+0000","last_active":"2023-11-08T19:31:14.017969+0000","last_peered":"2023-11-08T19:31:14.017969+0000","last_clean":"2023-11-08T19:31:14.017969+0000","last_became_active":"2023-11-08T19:27:07.187077+0000","last_became_peered":"2023-11-08T19:27:07.187077+0000","last_unstale":"2023-11-08T19:31:14.017969+0000","last_undegraded":"2023-11-08T19:31:14.017969+0000","last_fullsized":"2023-11-08T19:31:14.017969+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056673599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":48,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:14.012488+0000","last_change":"2023-11-08T19:27:08.188143+0000","last_active":"2023-11-08T19:31:14.012488+0000","last_peered":"2023-11-08T19:31:14.012488+0000","last_clean":"2023-11-08T19:31:14.012488+0000","last_became_active":"2023-11-08T19:27:08.187714+0000","last_became_peered":"2023-11-08T19:27:08.187714+0000","last_unstale":"2023-11-08T19:31:14.012488+0000","last_undegraded":"2023-11-08T19:31:14.012488+0000","last_fullsized":"2023-11-08T19:31:14.012488+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":86,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:35.500759+0000","last_change":"2023-11-08T19:31:35.500784+0000","last_active":"2023-11-08T19:31:35.500759+0000","last_peered":"2023-11-08T19:31:35.500759+0000","last_clean":"2023-11-08T19:31:35.500759+0000","last_became_active":"2023-11-08T19:26:04.603212+0000","last_became_peered":"2023-11-08T19:26:04.603212+0000","last_unstale":"2023-11-08T19:31:35.500759+0000","last_undegraded":"2023-11-08T19:31:35.500759+0000","last_fullsized":"2023-11-08T19:31:35.500759+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:35.500649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:35.500649+0000","last_clean_scrub_stamp":"2023-11-08T19:31:35.500649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:15:40.433937+0000","scrub_duration":0.0018424209999999999,"objects_trimmed":0,"snaptrim_duration":0.00054695399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":61,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:14.016997+0000","last_change":"2023-11-08T19:27:09.181613+0000","last_active":"2023-11-08T19:31:14.016997+0000","last_peered":"2023-11-08T19:31:14.016997+0000","last_clean":"2023-11-08T19:31:14.016997+0000","last_became_active":"2023-11-08T19:27:07.190897+0000","last_became_peered":"2023-11-08T19:27:07.190897+0000","last_unstale":"2023-11-08T19:31:14.016997+0000","last_undegraded":"2023-11-08T19:31:14.016997+0000","last_fullsized":"2023-11-08T19:31:14.016997+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011172459999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":48,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:14.013099+0000","last_change":"2023-11-08T19:27:08.188668+0000","last_active":"2023-11-08T19:31:14.013099+0000","last_peered":"2023-11-08T19:31:14.013099+0000","last_clean":"2023-11-08T19:31:14.013099+0000","last_became_active":"2023-11-08T19:27:08.188441+0000","last_became_peered":"2023-11-08T19:27:08.188441+0000","last_unstale":"2023-11-08T19:31:14.013099+0000","last_undegraded":"2023-11-08T19:31:14.013099+0000","last_fullsized":"2023-11-08T19:31:14.013099+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":92,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:24.529044+0000","last_change":"2023-11-08T19:31:24.529066+0000","last_active":"2023-11-08T19:31:24.529044+0000","last_peered":"2023-11-08T19:31:24.529044+0000","last_clean":"2023-11-08T19:31:24.529044+0000","last_became_active":"2023-11-08T19:26:04.599322+0000","last_became_peered":"2023-11-08T19:26:04.599322+0000","last_unstale":"2023-11-08T19:31:24.529044+0000","last_undegraded":"2023-11-08T19:31:24.529044+0000","last_fullsized":"2023-11-08T19:31:24.529044+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:24.528935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:24.528935+0000","last_clean_scrub_stamp":"2023-11-08T19:31:24.528935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:09:38.262113+0000","scrub_duration":0.001486217,"objects_trimmed":0,"snaptrim_duration":0.00042670000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":69,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:23.244792+0000","last_change":"2023-11-08T19:31:23.244841+0000","last_active":"2023-11-08T19:31:23.244792+0000","last_peered":"2023-11-08T19:31:23.244792+0000","last_clean":"2023-11-08T19:31:23.244792+0000","last_became_active":"2023-11-08T19:27:07.187337+0000","last_became_peered":"2023-11-08T19:27:07.187337+0000","last_unstale":"2023-11-08T19:31:23.244792+0000","last_undegraded":"2023-11-08T19:31:23.244792+0000","last_fullsized":"2023-11-08T19:31:23.244792+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:23.244666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:23.244666+0000","last_clean_scrub_stamp":"2023-11-08T19:31:23.244666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:40:55.948223+0000","scrub_duration":0.001423278,"objects_trimmed":0,"snaptrim_duration":0.000346967,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":56,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:54.564944+0000","last_change":"2023-11-08T19:31:54.564968+0000","last_active":"2023-11-08T19:31:54.564944+0000","last_peered":"2023-11-08T19:31:54.564944+0000","last_clean":"2023-11-08T19:31:54.564944+0000","last_became_active":"2023-11-08T19:27:08.193532+0000","last_became_peered":"2023-11-08T19:27:08.193532+0000","last_unstale":"2023-11-08T19:31:54.564944+0000","last_undegraded":"2023-11-08T19:31:54.564944+0000","last_fullsized":"2023-11-08T19:31:54.564944+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:54.564832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:54.564832+0000","last_clean_scrub_stamp":"2023-11-08T19:31:54.564832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:16:11.657792+0000","scrub_duration":0.001445223,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":86,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:25.726576+0000","last_change":"2023-11-08T19:31:25.726612+0000","last_active":"2023-11-08T19:31:25.726576+0000","last_peered":"2023-11-08T19:31:25.726576+0000","last_clean":"2023-11-08T19:31:25.726576+0000","last_became_active":"2023-11-08T19:26:04.606805+0000","last_became_peered":"2023-11-08T19:26:04.606805+0000","last_unstale":"2023-11-08T19:31:25.726576+0000","last_undegraded":"2023-11-08T19:31:25.726576+0000","last_fullsized":"2023-11-08T19:31:25.726576+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:25.726413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:25.726413+0000","last_clean_scrub_stamp":"2023-11-08T19:31:25.726413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:51:20.104960+0000","scrub_duration":0.002105923,"objects_trimmed":0,"snaptrim_duration":0.00045317000000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":63,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:51.112509+0000","last_change":"2023-11-08T19:31:51.112535+0000","last_active":"2023-11-08T19:31:51.112509+0000","last_peered":"2023-11-08T19:31:51.112509+0000","last_clean":"2023-11-08T19:31:51.112509+0000","last_became_active":"2023-11-08T19:27:07.190309+0000","last_became_peered":"2023-11-08T19:27:07.190309+0000","last_unstale":"2023-11-08T19:31:51.112509+0000","last_undegraded":"2023-11-08T19:31:51.112509+0000","last_fullsized":"2023-11-08T19:31:51.112509+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:51.112386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:51.112386+0000","last_clean_scrub_stamp":"2023-11-08T19:31:51.112386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:30:11.488593+0000","scrub_duration":0.0015743689999999999,"objects_trimmed":0,"snaptrim_duration":0.00047615199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":56,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:49.431129+0000","last_change":"2023-11-08T19:31:49.431152+0000","last_active":"2023-11-08T19:31:49.431129+0000","last_peered":"2023-11-08T19:31:49.431129+0000","last_clean":"2023-11-08T19:31:49.431129+0000","last_became_active":"2023-11-08T19:27:08.193241+0000","last_became_peered":"2023-11-08T19:27:08.193241+0000","last_unstale":"2023-11-08T19:31:49.431129+0000","last_undegraded":"2023-11-08T19:31:49.431129+0000","last_fullsized":"2023-11-08T19:31:49.431129+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:49.431025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:49.431025+0000","last_clean_scrub_stamp":"2023-11-08T19:31:49.431025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:29:51.038404+0000","scrub_duration":0.0016466090000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":86,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:43.118702+0000","last_change":"2023-11-08T19:31:43.118741+0000","last_active":"2023-11-08T19:31:43.118702+0000","last_peered":"2023-11-08T19:31:43.118702+0000","last_clean":"2023-11-08T19:31:43.118702+0000","last_became_active":"2023-11-08T19:26:04.604772+0000","last_became_peered":"2023-11-08T19:26:04.604772+0000","last_unstale":"2023-11-08T19:31:43.118702+0000","last_undegraded":"2023-11-08T19:31:43.118702+0000","last_fullsized":"2023-11-08T19:31:43.118702+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:43.118568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:43.118568+0000","last_clean_scrub_stamp":"2023-11-08T19:31:43.118568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:10:29.603174+0000","scrub_duration":0.0017187769999999999,"objects_trimmed":0,"snaptrim_duration":0.00066875300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":63,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:39.612694+0000","last_change":"2023-11-08T19:31:39.612728+0000","last_active":"2023-11-08T19:31:39.612694+0000","last_peered":"2023-11-08T19:31:39.612694+0000","last_clean":"2023-11-08T19:31:39.612694+0000","last_became_active":"2023-11-08T19:27:07.191212+0000","last_became_peered":"2023-11-08T19:27:07.191212+0000","last_unstale":"2023-11-08T19:31:39.612694+0000","last_undegraded":"2023-11-08T19:31:39.612694+0000","last_fullsized":"2023-11-08T19:31:39.612694+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:39.612544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:39.612544+0000","last_clean_scrub_stamp":"2023-11-08T19:31:39.612544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:17:14.238197+0000","scrub_duration":0.0019683529999999999,"objects_trimmed":0,"snaptrim_duration":0.00046640800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":49,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:14.013501+0000","last_change":"2023-11-08T19:27:08.187644+0000","last_active":"2023-11-08T19:31:14.013501+0000","last_peered":"2023-11-08T19:31:14.013501+0000","last_clean":"2023-11-08T19:31:14.013501+0000","last_became_active":"2023-11-08T19:27:08.187376+0000","last_became_peered":"2023-11-08T19:27:08.187376+0000","last_unstale":"2023-11-08T19:31:14.013501+0000","last_undegraded":"2023-11-08T19:31:14.013501+0000","last_fullsized":"2023-11-08T19:31:14.013501+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":87,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:26.809222+0000","last_change":"2023-11-08T19:31:26.809256+0000","last_active":"2023-11-08T19:31:26.809222+0000","last_peered":"2023-11-08T19:31:26.809222+0000","last_clean":"2023-11-08T19:31:26.809222+0000","last_became_active":"2023-11-08T19:26:04.598775+0000","last_became_peered":"2023-11-08T19:26:04.598775+0000","last_unstale":"2023-11-08T19:31:26.809222+0000","last_undegraded":"2023-11-08T19:31:26.809222+0000","last_fullsized":"2023-11-08T19:31:26.809222+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:26.809102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:26.809102+0000","last_clean_scrub_stamp":"2023-11-08T19:31:26.809102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:19:38.763559+0000","scrub_duration":0.002091,"objects_trimmed":0,"snaptrim_duration":0.00049870399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":64,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:39.485120+0000","last_change":"2023-11-08T19:31:39.485120+0000","last_active":"2023-11-08T19:31:39.485120+0000","last_peered":"2023-11-08T19:31:39.485120+0000","last_clean":"2023-11-08T19:31:39.485120+0000","last_became_active":"2023-11-08T19:27:07.190619+0000","last_became_peered":"2023-11-08T19:27:07.190619+0000","last_unstale":"2023-11-08T19:31:39.485120+0000","last_undegraded":"2023-11-08T19:31:39.485120+0000","last_fullsized":"2023-11-08T19:31:39.485120+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:39.484993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:39.484993+0000","last_clean_scrub_stamp":"2023-11-08T19:31:39.484993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:09:29.056050+0000","scrub_duration":0.001731263,"objects_trimmed":0,"snaptrim_duration":0.00065918400000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":63,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:38.329109+0000","last_change":"2023-11-08T19:31:38.329133+0000","last_active":"2023-11-08T19:31:38.329109+0000","last_peered":"2023-11-08T19:31:38.329109+0000","last_clean":"2023-11-08T19:31:38.329109+0000","last_became_active":"2023-11-08T19:27:08.183537+0000","last_became_peered":"2023-11-08T19:27:08.183537+0000","last_unstale":"2023-11-08T19:31:38.329109+0000","last_undegraded":"2023-11-08T19:31:38.329109+0000","last_fullsized":"2023-11-08T19:31:38.329109+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:38.328998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:38.328998+0000","last_clean_scrub_stamp":"2023-11-08T19:31:38.328998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:34:44.791135+0000","scrub_duration":0.0020231419999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":98,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:40.913956+0000","last_change":"2023-11-08T19:31:40.913984+0000","last_active":"2023-11-08T19:31:40.913956+0000","last_peered":"2023-11-08T19:31:40.913956+0000","last_clean":"2023-11-08T19:31:40.913956+0000","last_became_active":"2023-11-08T19:26:04.603209+0000","last_became_peered":"2023-11-08T19:26:04.603209+0000","last_unstale":"2023-11-08T19:31:40.913956+0000","last_undegraded":"2023-11-08T19:31:40.913956+0000","last_fullsized":"2023-11-08T19:31:40.913956+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:40.913792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:40.913792+0000","last_clean_scrub_stamp":"2023-11-08T19:31:40.913792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:20:59.266230+0000","scrub_duration":0.001580529,"objects_trimmed":0,"snaptrim_duration":0.00053335100000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":64,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:32.803377+0000","last_change":"2023-11-08T19:31:32.803377+0000","last_active":"2023-11-08T19:31:32.803377+0000","last_peered":"2023-11-08T19:31:32.803377+0000","last_clean":"2023-11-08T19:31:32.803377+0000","last_became_active":"2023-11-08T19:27:07.188169+0000","last_became_peered":"2023-11-08T19:27:07.188169+0000","last_unstale":"2023-11-08T19:31:32.803377+0000","last_undegraded":"2023-11-08T19:31:32.803377+0000","last_fullsized":"2023-11-08T19:31:32.803377+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:32.803247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:32.803247+0000","last_clean_scrub_stamp":"2023-11-08T19:31:32.803247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:30:32.589037+0000","scrub_duration":0.001747384,"objects_trimmed":0,"snaptrim_duration":0.00056209700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":57,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:44.503224+0000","last_change":"2023-11-08T19:31:44.503224+0000","last_active":"2023-11-08T19:31:44.503224+0000","last_peered":"2023-11-08T19:31:44.503224+0000","last_clean":"2023-11-08T19:31:44.503224+0000","last_became_active":"2023-11-08T19:27:08.193037+0000","last_became_peered":"2023-11-08T19:27:08.193037+0000","last_unstale":"2023-11-08T19:31:44.503224+0000","last_undegraded":"2023-11-08T19:31:44.503224+0000","last_fullsized":"2023-11-08T19:31:44.503224+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:44.503078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:44.503078+0000","last_clean_scrub_stamp":"2023-11-08T19:31:44.503078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:44:27.056698+0000","scrub_duration":0.0016849320000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":86,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:30.100037+0000","last_change":"2023-11-08T19:31:30.100059+0000","last_active":"2023-11-08T19:31:30.100037+0000","last_peered":"2023-11-08T19:31:30.100037+0000","last_clean":"2023-11-08T19:31:30.100037+0000","last_became_active":"2023-11-08T19:26:04.603409+0000","last_became_peered":"2023-11-08T19:26:04.603409+0000","last_unstale":"2023-11-08T19:31:30.100037+0000","last_undegraded":"2023-11-08T19:31:30.100037+0000","last_fullsized":"2023-11-08T19:31:30.100037+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:30.099915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:30.099915+0000","last_clean_scrub_stamp":"2023-11-08T19:31:30.099915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:03:25.984959+0000","scrub_duration":0.001620628,"objects_trimmed":0,"snaptrim_duration":0.00065684300000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":62,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:22.583044+0000","last_change":"2023-11-08T19:27:09.182282+0000","last_active":"2023-11-08T19:31:22.583044+0000","last_peered":"2023-11-08T19:31:22.583044+0000","last_clean":"2023-11-08T19:31:22.583044+0000","last_became_active":"2023-11-08T19:27:07.186944+0000","last_became_peered":"2023-11-08T19:27:07.186944+0000","last_unstale":"2023-11-08T19:31:22.583044+0000","last_undegraded":"2023-11-08T19:31:22.583044+0000","last_fullsized":"2023-11-08T19:31:22.583044+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00028665599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":49,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:21.587395+0000","last_change":"2023-11-08T19:27:08.193990+0000","last_active":"2023-11-08T19:31:21.587395+0000","last_peered":"2023-11-08T19:31:21.587395+0000","last_clean":"2023-11-08T19:31:21.587395+0000","last_became_active":"2023-11-08T19:27:08.193624+0000","last_became_peered":"2023-11-08T19:27:08.193624+0000","last_unstale":"2023-11-08T19:31:21.587395+0000","last_undegraded":"2023-11-08T19:31:21.587395+0000","last_fullsized":"2023-11-08T19:31:21.587395+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":92,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:30.601612+0000","last_change":"2023-11-08T19:31:30.601633+0000","last_active":"2023-11-08T19:31:30.601612+0000","last_peered":"2023-11-08T19:31:30.601612+0000","last_clean":"2023-11-08T19:31:30.601612+0000","last_became_active":"2023-11-08T19:26:04.608639+0000","last_became_peered":"2023-11-08T19:26:04.608639+0000","last_unstale":"2023-11-08T19:31:30.601612+0000","last_undegraded":"2023-11-08T19:31:30.601612+0000","last_fullsized":"2023-11-08T19:31:30.601612+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:30.601479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:30.601479+0000","last_clean_scrub_stamp":"2023-11-08T19:31:30.601479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:16:43.728400+0000","scrub_duration":0.0016375770000000001,"objects_trimmed":0,"snaptrim_duration":0.00044384999999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":64,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:50.637123+0000","last_change":"2023-11-08T19:31:50.637123+0000","last_active":"2023-11-08T19:31:50.637123+0000","last_peered":"2023-11-08T19:31:50.637123+0000","last_clean":"2023-11-08T19:31:50.637123+0000","last_became_active":"2023-11-08T19:27:07.190931+0000","last_became_peered":"2023-11-08T19:27:07.190931+0000","last_unstale":"2023-11-08T19:31:50.637123+0000","last_undegraded":"2023-11-08T19:31:50.637123+0000","last_fullsized":"2023-11-08T19:31:50.637123+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:50.636977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:50.636977+0000","last_clean_scrub_stamp":"2023-11-08T19:31:50.636977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:59:11.822136+0000","scrub_duration":0.001523333,"objects_trimmed":0,"snaptrim_duration":0.00034146799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":61,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:53.801652+0000","last_change":"2023-11-08T19:31:53.801674+0000","last_active":"2023-11-08T19:31:53.801652+0000","last_peered":"2023-11-08T19:31:53.801652+0000","last_clean":"2023-11-08T19:31:53.801652+0000","last_became_active":"2023-11-08T19:27:08.186952+0000","last_became_peered":"2023-11-08T19:27:08.186952+0000","last_unstale":"2023-11-08T19:31:53.801652+0000","last_undegraded":"2023-11-08T19:31:53.801652+0000","last_fullsized":"2023-11-08T19:31:53.801652+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:53.801547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:53.801547+0000","last_clean_scrub_stamp":"2023-11-08T19:31:53.801547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:56:01.800602+0000","scrub_duration":0.001479352,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":86,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:32.594224+0000","last_change":"2023-11-08T19:31:32.594249+0000","last_active":"2023-11-08T19:31:32.594224+0000","last_peered":"2023-11-08T19:31:32.594224+0000","last_clean":"2023-11-08T19:31:32.594224+0000","last_became_active":"2023-11-08T19:26:04.602137+0000","last_became_peered":"2023-11-08T19:26:04.602137+0000","last_unstale":"2023-11-08T19:31:32.594224+0000","last_undegraded":"2023-11-08T19:31:32.594224+0000","last_fullsized":"2023-11-08T19:31:32.594224+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:32.594119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:32.594119+0000","last_clean_scrub_stamp":"2023-11-08T19:31:32.594119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:04:34.319826+0000","scrub_duration":0.00148785,"objects_trimmed":0,"snaptrim_duration":0.00056998200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":64,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:35.819632+0000","last_change":"2023-11-08T19:31:35.819632+0000","last_active":"2023-11-08T19:31:35.819632+0000","last_peered":"2023-11-08T19:31:35.819632+0000","last_clean":"2023-11-08T19:31:35.819632+0000","last_became_active":"2023-11-08T19:27:07.192001+0000","last_became_peered":"2023-11-08T19:27:07.192001+0000","last_unstale":"2023-11-08T19:31:35.819632+0000","last_undegraded":"2023-11-08T19:31:35.819632+0000","last_fullsized":"2023-11-08T19:31:35.819632+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:35.819499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:35.819499+0000","last_clean_scrub_stamp":"2023-11-08T19:31:35.819499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:56:02.315632+0000","scrub_duration":0.0014740599999999999,"objects_trimmed":0,"snaptrim_duration":0.00045359000000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":57,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:30.097308+0000","last_change":"2023-11-08T19:31:30.097308+0000","last_active":"2023-11-08T19:31:30.097308+0000","last_peered":"2023-11-08T19:31:30.097308+0000","last_clean":"2023-11-08T19:31:30.097308+0000","last_became_active":"2023-11-08T19:27:08.193458+0000","last_became_peered":"2023-11-08T19:27:08.193458+0000","last_unstale":"2023-11-08T19:31:30.097308+0000","last_undegraded":"2023-11-08T19:31:30.097308+0000","last_fullsized":"2023-11-08T19:31:30.097308+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:30.097159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:30.097159+0000","last_clean_scrub_stamp":"2023-11-08T19:31:30.097159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:12:53.982878+0000","scrub_duration":0.001628696,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":88,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:23.750030+0000","last_change":"2023-11-08T19:31:23.750053+0000","last_active":"2023-11-08T19:31:23.750030+0000","last_peered":"2023-11-08T19:31:23.750030+0000","last_clean":"2023-11-08T19:31:23.750030+0000","last_became_active":"2023-11-08T19:26:04.602183+0000","last_became_peered":"2023-11-08T19:26:04.602183+0000","last_unstale":"2023-11-08T19:31:23.750030+0000","last_undegraded":"2023-11-08T19:31:23.750030+0000","last_fullsized":"2023-11-08T19:31:23.750030+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:23.749930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:23.749930+0000","last_clean_scrub_stamp":"2023-11-08T19:31:23.749930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:48:22.893896+0000","scrub_duration":0.001725201,"objects_trimmed":0,"snaptrim_duration":0.00052029099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":70,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:29.200138+0000","last_change":"2023-11-08T19:31:29.200138+0000","last_active":"2023-11-08T19:31:29.200138+0000","last_peered":"2023-11-08T19:31:29.200138+0000","last_clean":"2023-11-08T19:31:29.200138+0000","last_became_active":"2023-11-08T19:27:07.184507+0000","last_became_peered":"2023-11-08T19:27:07.184507+0000","last_unstale":"2023-11-08T19:31:29.200138+0000","last_undegraded":"2023-11-08T19:31:29.200138+0000","last_fullsized":"2023-11-08T19:31:29.200138+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:29.199968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:29.199968+0000","last_clean_scrub_stamp":"2023-11-08T19:31:29.199968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:17:03.825621+0000","scrub_duration":0.001394225,"objects_trimmed":0,"snaptrim_duration":0.00051924700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":49,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:21.587424+0000","last_change":"2023-11-08T19:27:08.194263+0000","last_active":"2023-11-08T19:31:21.587424+0000","last_peered":"2023-11-08T19:31:21.587424+0000","last_clean":"2023-11-08T19:31:21.587424+0000","last_became_active":"2023-11-08T19:27:08.193938+0000","last_became_peered":"2023-11-08T19:27:08.193938+0000","last_unstale":"2023-11-08T19:31:21.587424+0000","last_undegraded":"2023-11-08T19:31:21.587424+0000","last_fullsized":"2023-11-08T19:31:21.587424+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:07.167285+0000","last_clean_scrub_stamp":"2023-11-08T19:27:07.167285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":86,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:35.112226+0000","last_change":"2023-11-08T19:31:35.112252+0000","last_active":"2023-11-08T19:31:35.112226+0000","last_peered":"2023-11-08T19:31:35.112226+0000","last_clean":"2023-11-08T19:31:35.112226+0000","last_became_active":"2023-11-08T19:26:04.603397+0000","last_became_peered":"2023-11-08T19:26:04.603397+0000","last_unstale":"2023-11-08T19:31:35.112226+0000","last_undegraded":"2023-11-08T19:31:35.112226+0000","last_fullsized":"2023-11-08T19:31:35.112226+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:35.112102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:35.112102+0000","last_clean_scrub_stamp":"2023-11-08T19:31:35.112102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:17:09.737756+0000","scrub_duration":0.0016909259999999999,"objects_trimmed":0,"snaptrim_duration":0.00053361499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":64,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:34.769739+0000","last_change":"2023-11-08T19:31:34.769739+0000","last_active":"2023-11-08T19:31:34.769739+0000","last_peered":"2023-11-08T19:31:34.769739+0000","last_clean":"2023-11-08T19:31:34.769739+0000","last_became_active":"2023-11-08T19:27:07.189943+0000","last_became_peered":"2023-11-08T19:27:07.189943+0000","last_unstale":"2023-11-08T19:31:34.769739+0000","last_undegraded":"2023-11-08T19:31:34.769739+0000","last_fullsized":"2023-11-08T19:31:34.769739+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:34.769604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:34.769604+0000","last_clean_scrub_stamp":"2023-11-08T19:31:34.769604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:51:01.063515+0000","scrub_duration":0.001529461,"objects_trimmed":0,"snaptrim_duration":0.000398725,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":63,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:35.319840+0000","last_change":"2023-11-08T19:31:35.319840+0000","last_active":"2023-11-08T19:31:35.319840+0000","last_peered":"2023-11-08T19:31:35.319840+0000","last_clean":"2023-11-08T19:31:35.319840+0000","last_became_active":"2023-11-08T19:27:08.187487+0000","last_became_peered":"2023-11-08T19:27:08.187487+0000","last_unstale":"2023-11-08T19:31:35.319840+0000","last_undegraded":"2023-11-08T19:31:35.319840+0000","last_fullsized":"2023-11-08T19:31:35.319840+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:35.319727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:35.319727+0000","last_clean_scrub_stamp":"2023-11-08T19:31:35.319727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:41:21.686280+0000","scrub_duration":0.001956157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":92,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:39.721489+0000","last_change":"2023-11-08T19:31:39.721512+0000","last_active":"2023-11-08T19:31:39.721489+0000","last_peered":"2023-11-08T19:31:39.721489+0000","last_clean":"2023-11-08T19:31:39.721489+0000","last_became_active":"2023-11-08T19:26:04.601103+0000","last_became_peered":"2023-11-08T19:26:04.601103+0000","last_unstale":"2023-11-08T19:31:39.721489+0000","last_undegraded":"2023-11-08T19:31:39.721489+0000","last_fullsized":"2023-11-08T19:31:39.721489+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:39.721350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:39.721350+0000","last_clean_scrub_stamp":"2023-11-08T19:31:39.721350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:30:00.097557+0000","scrub_duration":0.001887569,"objects_trimmed":0,"snaptrim_duration":0.000229244,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":62,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:22.583058+0000","last_change":"2023-11-08T19:27:09.179852+0000","last_active":"2023-11-08T19:31:22.583058+0000","last_peered":"2023-11-08T19:31:22.583058+0000","last_clean":"2023-11-08T19:31:22.583058+0000","last_became_active":"2023-11-08T19:27:07.191926+0000","last_became_peered":"2023-11-08T19:27:07.191926+0000","last_unstale":"2023-11-08T19:31:22.583058+0000","last_undegraded":"2023-11-08T19:31:22.583058+0000","last_fullsized":"2023-11-08T19:31:22.583058+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:06.162282+0000","last_clean_scrub_stamp":"2023-11-08T19:27:06.162282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055457100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":63,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:40.373522+0000","last_change":"2023-11-08T19:31:40.373522+0000","last_active":"2023-11-08T19:31:40.373522+0000","last_peered":"2023-11-08T19:31:40.373522+0000","last_clean":"2023-11-08T19:31:40.373522+0000","last_became_active":"2023-11-08T19:27:08.185581+0000","last_became_peered":"2023-11-08T19:27:08.185581+0000","last_unstale":"2023-11-08T19:31:40.373522+0000","last_undegraded":"2023-11-08T19:31:40.373522+0000","last_fullsized":"2023-11-08T19:31:40.373522+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:40.373357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:40.373357+0000","last_clean_scrub_stamp":"2023-11-08T19:31:40.373357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:48:00.953173+0000","scrub_duration":0.0015121480000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":98,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:46.864396+0000","last_change":"2023-11-08T19:31:46.864422+0000","last_active":"2023-11-08T19:31:46.864396+0000","last_peered":"2023-11-08T19:31:46.864396+0000","last_clean":"2023-11-08T19:31:46.864396+0000","last_became_active":"2023-11-08T19:26:04.606312+0000","last_became_peered":"2023-11-08T19:26:04.606312+0000","last_unstale":"2023-11-08T19:31:46.864396+0000","last_undegraded":"2023-11-08T19:31:46.864396+0000","last_fullsized":"2023-11-08T19:31:46.864396+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:46.864278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:46.864278+0000","last_clean_scrub_stamp":"2023-11-08T19:31:46.864278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:43:07.746274+0000","scrub_duration":0.0014882389999999999,"objects_trimmed":0,"snaptrim_duration":0.000505935,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":64,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:30.876396+0000","last_change":"2023-11-08T19:31:30.876396+0000","last_active":"2023-11-08T19:31:30.876396+0000","last_peered":"2023-11-08T19:31:30.876396+0000","last_clean":"2023-11-08T19:31:30.876396+0000","last_became_active":"2023-11-08T19:27:07.193254+0000","last_became_peered":"2023-11-08T19:27:07.193254+0000","last_unstale":"2023-11-08T19:31:30.876396+0000","last_undegraded":"2023-11-08T19:31:30.876396+0000","last_fullsized":"2023-11-08T19:31:30.876396+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:30.876249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:30.876249+0000","last_clean_scrub_stamp":"2023-11-08T19:31:30.876249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:32:42.716359+0000","scrub_duration":0.0014983080000000001,"objects_trimmed":0,"snaptrim_duration":0.00041830799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":63,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:36.343755+0000","last_change":"2023-11-08T19:31:36.343755+0000","last_active":"2023-11-08T19:31:36.343755+0000","last_peered":"2023-11-08T19:31:36.343755+0000","last_clean":"2023-11-08T19:31:36.343755+0000","last_became_active":"2023-11-08T19:27:08.188560+0000","last_became_peered":"2023-11-08T19:27:08.188560+0000","last_unstale":"2023-11-08T19:31:36.343755+0000","last_undegraded":"2023-11-08T19:31:36.343755+0000","last_fullsized":"2023-11-08T19:31:36.343755+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:36.343622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:36.343622+0000","last_clean_scrub_stamp":"2023-11-08T19:31:36.343622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:29:56.719829+0000","scrub_duration":0.002226525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":87,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:23.098586+0000","last_change":"2023-11-08T19:31:23.098607+0000","last_active":"2023-11-08T19:31:23.098586+0000","last_peered":"2023-11-08T19:31:23.098586+0000","last_clean":"2023-11-08T19:31:23.098586+0000","last_became_active":"2023-11-08T19:26:04.603979+0000","last_became_peered":"2023-11-08T19:26:04.603979+0000","last_unstale":"2023-11-08T19:31:23.098586+0000","last_undegraded":"2023-11-08T19:31:23.098586+0000","last_fullsized":"2023-11-08T19:31:23.098586+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:23.098455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:23.098455+0000","last_clean_scrub_stamp":"2023-11-08T19:31:23.098455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:03:18.983499+0000","scrub_duration":0.0017647400000000001,"objects_trimmed":0,"snaptrim_duration":0.00079032099999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":64,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:37.483328+0000","last_change":"2023-11-08T19:31:37.483328+0000","last_active":"2023-11-08T19:31:37.483328+0000","last_peered":"2023-11-08T19:31:37.483328+0000","last_clean":"2023-11-08T19:31:37.483328+0000","last_became_active":"2023-11-08T19:27:07.185478+0000","last_became_peered":"2023-11-08T19:27:07.185478+0000","last_unstale":"2023-11-08T19:31:37.483328+0000","last_undegraded":"2023-11-08T19:31:37.483328+0000","last_fullsized":"2023-11-08T19:31:37.483328+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:37.483195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:37.483195+0000","last_clean_scrub_stamp":"2023-11-08T19:31:37.483195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:53:43.068960+0000","scrub_duration":0.001795891,"objects_trimmed":0,"snaptrim_duration":0.0010661850000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574939,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33048,"kb_used_data":1872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338792,"statfs":{"total":96636764160,"available":96602923008,"internally_reserved":0,"allocated":1916928,"data_stored":1236765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.54200000000000004,"15min":0.51300000000000001},"min":{"1min":0.314,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.70399999999999996,"5min":0.76600000000000001,"15min":1.0149999999999999},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.53800000000000003,"15min":0.52300000000000002},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.32300000000000001},"max":{"1min":0.70099999999999996,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.71899999999999997}]},{"osd":1,"last update":"Wed Nov 8 19:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60099999999999998,"15min":0.57799999999999996},"min":{"1min":0.40100000000000002,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.78700000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.626},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.59499999999999997,"15min":0.56899999999999995},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.33900000000000002},"max":{"1min":0.76000000000000001,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.67000000000000004}]},{"osd":2,"last update":"Wed Nov 8 19:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.54200000000000004,"15min":0.51000000000000001},"min":{"1min":0.39200000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.66700000000000004,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56999999999999995,"15min":0.55700000000000005},"min":{"1min":0.35999999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.80400000000000005,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.68899999999999995}]},{"osd":3,"last update":"Wed Nov 8 19:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.57999999999999996,"15min":0.56599999999999995},"min":{"1min":0.46500000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.72199999999999998,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.625,"5min":0.63100000000000001,"15min":0.59699999999999998},"min":{"1min":0.505,"5min":0.437,"15min":0.40100000000000002},"max":{"1min":0.752,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.65900000000000003}]},{"osd":4,"last update":"Wed Nov 8 19:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.48499999999999999,"15min":0.502},"min":{"1min":0.33600000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.65700000000000003,"5min":0.753,"15min":0.78600000000000003},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.52700000000000002,"15min":0.53700000000000003},"min":{"1min":0.38300000000000001,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.73499999999999999,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.59299999999999997}]},{"osd":5,"last update":"Wed Nov 8 19:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.55200000000000005,"15min":0.57199999999999995},"min":{"1min":0.36199999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.71799999999999997,"5min":0.78500000000000003,"15min":0.82499999999999996},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.61499999999999999,"15min":0.63400000000000001},"min":{"1min":0.432,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.75600000000000001,"5min":0.89900000000000002,"15min":0.94299999999999995},"last":0.60999999999999999}]},{"osd":6,"last update":"Wed Nov 8 19:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.59499999999999997},"min":{"1min":0.42999999999999999,"5min":0.377,"15min":0.377},"max":{"1min":0.76200000000000001,"5min":0.86199999999999999,"15min":0.97299999999999998},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58399999999999996,"15min":0.60299999999999998},"min":{"1min":0.375,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.77300000000000002,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.64200000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574939,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32468,"kb_used_data":1292,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339372,"statfs":{"total":96636764160,"available":96603516928,"internally_reserved":0,"allocated":1323008,"data_stored":646397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.57099999999999995,"15min":1.0900000000000001},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.27000000000000002},"max":{"1min":0.78200000000000003,"5min":0.84599999999999997,"15min":21.359999999999999},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57499999999999996,"15min":0.79200000000000004},"min":{"1min":0.42999999999999999,"5min":0.378,"15min":0.30399999999999999},"max":{"1min":0.83599999999999997,"5min":0.85699999999999998,"15min":10.329000000000001},"last":0.5}]},{"osd":1,"last update":"Wed Nov 8 19:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.55000000000000004,"15min":0.59199999999999997},"min":{"1min":0.41499999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.72399999999999998,"5min":0.81399999999999995,"15min":3.4239999999999999},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.61199999999999999,"15min":0.63},"min":{"1min":0.41199999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.77900000000000003,"5min":2.9260000000000002,"15min":3.4809999999999999},"last":0.58999999999999997}]},{"osd":2,"last update":"Wed Nov 8 19:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.59399999999999997,"15min":0.81000000000000005},"min":{"1min":0.49399999999999999,"5min":0.40000000000000002,"15min":0.34399999999999997},"max":{"1min":0.79400000000000004,"5min":0.84199999999999997,"15min":10.398},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.57499999999999996,"15min":0.65400000000000003},"min":{"1min":0.46300000000000002,"5min":0.377,"15min":0.318},"max":{"1min":0.67700000000000005,"5min":0.83599999999999997,"15min":5.0170000000000003},"last":0.57099999999999995}]},{"osd":3,"last update":"Wed Nov 8 19:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.63700000000000001,"15min":0.65600000000000003},"min":{"1min":0.45600000000000002,"5min":0.35999999999999999,"15min":0.317},"max":{"1min":0.81599999999999995,"5min":3.0089999999999999,"15min":3.5710000000000002},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.60899999999999999,"15min":0.83699999999999997},"min":{"1min":0.435,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.86099999999999999,"5min":0.86799999999999999,"15min":10.368},"last":0.65600000000000003}]},{"osd":4,"last update":"Wed Nov 8 19:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52900000000000003,"15min":0.52000000000000002},"min":{"1min":0.371,"5min":0.33000000000000002,"15min":0.29199999999999998},"max":{"1min":0.72799999999999998,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.55300000000000005,"15min":0.55900000000000005},"min":{"1min":0.38700000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.76300000000000001,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.79900000000000004}]},{"osd":5,"last update":"Wed Nov 8 19:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53700000000000003,"15min":0.53300000000000003},"min":{"1min":0.36699999999999999,"5min":0.34599999999999997,"15min":0.307},"max":{"1min":0.80100000000000005,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.59099999999999997,"15min":0.58599999999999997},"min":{"1min":0.41099999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.84899999999999998,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.72399999999999998}]},{"osd":7,"last update":"Wed Nov 8 19:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.59799999999999998,"15min":0.58399999999999996},"min":{"1min":0.373,"5min":0.373,"15min":0.34699999999999998},"max":{"1min":0.81899999999999995,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.59799999999999998,"15min":0.59399999999999997},"min":{"1min":0.41399999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.83699999999999997,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.74099999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574939,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32468,"kb_used_data":1292,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339372,"statfs":{"total":96636764160,"available":96603516928,"internally_reserved":0,"allocated":1323008,"data_stored":646397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:31:55 2023","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.35399999999999998,"15min":0.372},"min":{"1min":0.27400000000000002,"5min":0.222,"15min":0.219},"max":{"1min":0.45200000000000001,"5min":0.69699999999999995,"15min":1.389},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.38,"15min":0.59599999999999997},"min":{"1min":0.20599999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":1.5109999999999999,"5min":1.5109999999999999,"15min":9.1370000000000005},"last":0.5}]},{"osd":2,"last update":"Wed Nov 8 19:31:55 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.36799999999999999,"15min":0.36499999999999999},"min":{"1min":0.27900000000000003,"5min":0.216,"15min":0.216},"max":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":1.2709999999999999},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.39200000000000002,"15min":0.41599999999999998},"min":{"1min":0.24199999999999999,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.47899999999999998,"5min":0.63600000000000001,"15min":1.425},"last":0.47899999999999998}]},{"osd":3,"last update":"Wed Nov 8 19:31:55 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.41899999999999998,"15min":0.71199999999999997},"min":{"1min":0.29699999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.59799999999999998,"5min":0.63800000000000001,"15min":11.957000000000001},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.40000000000000002,"15min":0.86699999999999999},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":1.5600000000000001,"5min":1.5600000000000001,"15min":18.039000000000001},"last":0.61599999999999999}]},{"osd":4,"last update":"Wed Nov 8 19:31:55 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.59199999999999997,"15min":0.73499999999999999},"min":{"1min":0.40100000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.84899999999999998,"5min":0.94999999999999996,"15min":6.181},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59799999999999998,"15min":0.627},"min":{"1min":0.40100000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.88,"5min":0.996,"15min":1.488},"last":0.88}]},{"osd":5,"last update":"Wed Nov 8 19:31:55 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.60399999999999998,"15min":0.83699999999999997},"min":{"1min":0.41499999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.601,"5min":1.601,"15min":9.2420000000000009},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59699999999999998,"15min":0.77000000000000002},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.82599999999999996,"5min":0.878,"15min":6.2110000000000003},"last":0.82599999999999996}]},{"osd":6,"last update":"Wed Nov 8 19:31:55 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.623,"15min":0.84599999999999997},"min":{"1min":0.442,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.665,"5min":1.665,"15min":9.1880000000000006},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.625,"15min":0.65300000000000002},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.90500000000000003,"5min":0.98399999999999999,"15min":1.462},"last":0.84099999999999997}]},{"osd":7,"last update":"Wed Nov 8 19:31:55 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.61299999999999999,"15min":0.77000000000000002},"min":{"1min":0.40899999999999997,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.79200000000000004,"5min":1.0309999999999999,"15min":6.1479999999999997},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.64400000000000002,"15min":0.876},"min":{"1min":0.47499999999999998,"5min":0.40200000000000002,"15min":0.36099999999999999},"max":{"1min":1.7909999999999999,"5min":1.7909999999999999,"15min":9.2170000000000005},"last":0.86199999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574939,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33048,"kb_used_data":1872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338792,"statfs":{"total":96636764160,"available":96602923008,"internally_reserved":0,"allocated":1916928,"data_stored":1236765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Nov 8 19:31:01 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.33300000000000002,"15min":0.45900000000000002},"min":{"1min":0.20899999999999999,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.51000000000000001,"5min":0.55300000000000005,"15min":4.1200000000000001},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.439,"5min":0.371,"15min":0.91900000000000004},"min":{"1min":0.22800000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.62,"5min":0.62,"15min":16.940999999999999},"last":0.45800000000000002}]},{"osd":2,"last update":"Wed Nov 8 19:31:01 2023","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.35299999999999998,"15min":0.38700000000000001},"min":{"1min":0.25,"5min":0.22,"15min":0.22},"max":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":1.1899999999999999},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.39100000000000001,"15min":0.60499999999999998},"min":{"1min":0.32400000000000001,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":7.0209999999999999},"last":0.40500000000000003}]},{"osd":3,"last update":"Wed Nov 8 19:31:01 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.38400000000000001,"15min":0.51200000000000001},"min":{"1min":0.29899999999999999,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":4.0940000000000003},"last":0.435},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.39500000000000002,"15min":0.433},"min":{"1min":0.32500000000000001,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.61899999999999999,"5min":0.62,"15min":1.2549999999999999},"last":0.47399999999999998}]},{"osd":4,"last update":"Wed Nov 8 19:31:01 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.58899999999999997,"15min":0.71699999999999997},"min":{"1min":0.42799999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":4.0049999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.58399999999999996,"15min":0.623},"min":{"1min":0.442,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.746,"5min":0.79200000000000004,"15min":1.2270000000000001},"last":0.746}]},{"osd":5,"last update":"Wed Nov 8 19:31:01 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.59699999999999998,"15min":0.72799999999999998},"min":{"1min":0.47199999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":4.3090000000000002},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.61199999999999999,"15min":0.72399999999999998},"min":{"1min":0.436,"5min":0.373,"15min":0.373},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":4.0709999999999997},"last":0.76200000000000001}]},{"osd":6,"last update":"Wed Nov 8 19:31:01 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.621,"15min":0.65800000000000003},"min":{"1min":0.40999999999999998,"5min":0.373,"15min":0.373},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":1.2849999999999999},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.64900000000000002,"15min":0.78500000000000003},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":4.2560000000000002},"last":0.78100000000000003}]},{"osd":7,"last update":"Wed Nov 8 19:31:01 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.625,"15min":0.74299999999999999},"min":{"1min":0.47099999999999997,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":4.2850000000000001},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.64700000000000002,"15min":0.77000000000000002},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.002,"5min":1.002,"15min":4.0460000000000003},"last":0.55800000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574939,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28308,"kb_used_data":1292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343532,"statfs":{"total":96636764160,"available":96607776768,"internally_reserved":0,"allocated":1323008,"data_stored":646397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.35499999999999998,"15min":0.34000000000000002},"min":{"1min":0.251,"5min":0.182,"15min":0.182},"max":{"1min":0.48499999999999999,"5min":0.629,"15min":0.629},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.375,"15min":0.54100000000000004},"min":{"1min":0.28100000000000003,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.52000000000000002,"5min":0.68000000000000005,"15min":6.0999999999999996},"last":0.39000000000000001}]},{"osd":1,"last update":"Wed Nov 8 19:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.36199999999999999,"15min":0.34799999999999998},"min":{"1min":0.24399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.59499999999999997,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.39400000000000002,"15min":0.39000000000000001},"min":{"1min":0.28000000000000003,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.56599999999999995,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.47199999999999998}]},{"osd":3,"last update":"Wed Nov 8 19:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.41599999999999998,"15min":0.39400000000000002},"min":{"1min":0.30399999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.56899999999999995,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.441},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.40500000000000003,"15min":0.39700000000000002},"min":{"1min":0.28100000000000003,"5min":0.217,"15min":0.217},"max":{"1min":0.54400000000000004,"5min":0.65900000000000003,"15min":0.70299999999999996},"last":0.36799999999999999}]},{"osd":4,"last update":"Wed Nov 8 19:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.56499999999999995,"15min":0.57599999999999996},"min":{"1min":0.432,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.81499999999999995,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.58899999999999997,"15min":0.58499999999999996},"min":{"1min":0.43099999999999999,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.74199999999999999,"5min":0.872,"15min":0.90400000000000003},"last":0.50700000000000001}]},{"osd":5,"last update":"Wed Nov 8 19:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.627,"15min":0.59799999999999998},"min":{"1min":0.49399999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.85999999999999999,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.622},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.61499999999999999,"15min":0.61299999999999999},"min":{"1min":0.439,"5min":0.38,"15min":0.38},"max":{"1min":0.92100000000000004,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.59899999999999998}]},{"osd":6,"last update":"Wed Nov 8 19:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.59799999999999998,"15min":0.59899999999999998},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.621,"5min":0.60899999999999999,"15min":0.58899999999999997},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.36599999999999999},"max":{"1min":0.872,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.57799999999999996}]},{"osd":7,"last update":"Wed Nov 8 19:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.61499999999999999,"15min":0.59199999999999997},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.85799999999999998,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.626,"5min":0.624,"15min":0.60999999999999999},"min":{"1min":0.45800000000000002,"5min":0.41699999999999998,"15min":0.38400000000000001},"max":{"1min":0.85999999999999999,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.52200000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574940,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28308,"kb_used_data":1292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343532,"statfs":{"total":96636764160,"available":96607776768,"internally_reserved":0,"allocated":1323008,"data_stored":646397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:30:59 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.33500000000000002,"15min":0.34000000000000002},"min":{"1min":0.254,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.48599999999999999,"5min":0.49099999999999999,"15min":0.626},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.378,"5min":0.372,"15min":0.35599999999999998},"min":{"1min":0.26700000000000002,"5min":0.23699999999999999,"15min":0.23300000000000001},"max":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.60799999999999998},"last":0.32400000000000001}]},{"osd":1,"last update":"Wed Nov 8 19:30:59 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.35999999999999999,"15min":0.35599999999999998},"min":{"1min":0.245,"5min":0.24099999999999999,"15min":0.23999999999999999},"max":{"1min":0.53300000000000003,"5min":0.61699999999999999,"15min":0.61699999999999999},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.40100000000000002,"15min":0.39000000000000001},"min":{"1min":0.30099999999999999,"5min":0.27900000000000003,"15min":0.27700000000000002},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.82099999999999995},"last":0.38}]},{"osd":2,"last update":"Wed Nov 8 19:30:59 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.40999999999999998,"15min":0.38300000000000001},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28799999999999998},"max":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58499999999999996},"last":0.443},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.41999999999999998,"15min":0.40899999999999997},"min":{"1min":0.30499999999999999,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.59799999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.42499999999999999}]},{"osd":4,"last update":"Wed Nov 8 19:30:59 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.63,"15min":0.60799999999999998},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.85699999999999998,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63700000000000001,"15min":0.61499999999999999},"min":{"1min":0.38600000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.89600000000000002,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.67800000000000005}]},{"osd":5,"last update":"Wed Nov 8 19:30:59 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59899999999999998,"15min":0.58299999999999996},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.83099999999999996,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.623,"5min":0.622,"15min":0.59199999999999997},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.879,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.65500000000000003}]},{"osd":6,"last update":"Wed Nov 8 19:30:59 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.61399999999999999,"15min":0.59099999999999997},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41399999999999998},"max":{"1min":0.83899999999999997,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63300000000000001,"15min":0.61299999999999999},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.90500000000000003},"last":0.59399999999999997}]},{"osd":7,"last update":"Wed Nov 8 19:30:59 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.626,"15min":0.59999999999999998},"min":{"1min":0.41899999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":1.0009999999999999},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.63900000000000001,"15min":0.60999999999999999},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.85099999999999998,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.63400000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574939,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28308,"kb_used_data":1292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343532,"statfs":{"total":96636764160,"available":96607776768,"internally_reserved":0,"allocated":1323008,"data_stored":646397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:30:58 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54700000000000004,"15min":0.69899999999999995},"min":{"1min":0.311,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.71499999999999997,"5min":0.84799999999999998,"15min":6.5970000000000004},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.53500000000000003,"15min":0.56299999999999994},"min":{"1min":0.36299999999999999,"5min":0.32200000000000001,"15min":0.317},"max":{"1min":0.73099999999999998,"5min":0.75900000000000001,"15min":2.5030000000000001},"last":0.63}]},{"osd":1,"last update":"Wed Nov 8 19:30:58 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.55400000000000005,"15min":0.51500000000000001},"min":{"1min":0.41699999999999998,"5min":0.32900000000000001,"15min":0.24299999999999999},"max":{"1min":0.72799999999999998,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.55900000000000005,"15min":0.52500000000000002},"min":{"1min":0.49299999999999999,"5min":0.312,"15min":0.312},"max":{"1min":0.72799999999999998,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.60399999999999998}]},{"osd":2,"last update":"Wed Nov 8 19:30:58 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55300000000000005},"min":{"1min":0.505,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.64400000000000002,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.58799999999999997,"15min":0.76000000000000001},"min":{"1min":0.50700000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.70999999999999996,"5min":0.871,"15min":6.6920000000000002},"last":0.64400000000000002}]},{"osd":3,"last update":"Wed Nov 8 19:30:58 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.56499999999999995,"15min":0.58099999999999996},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.69799999999999995,"5min":0.79800000000000004,"15min":1.0700000000000001},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.61799999999999999,"15min":0.61299999999999999},"min":{"1min":0.47799999999999998,"5min":0.41699999999999998,"15min":0.34599999999999997},"max":{"1min":0.70399999999999996,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.61699999999999999}]},{"osd":5,"last update":"Wed Nov 8 19:30:58 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.501,"15min":0.53100000000000003},"min":{"1min":0.39100000000000001,"5min":0.29599999999999999,"15min":0.28100000000000003},"max":{"1min":0.66100000000000003,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.51700000000000002,"15min":0.57099999999999995},"min":{"1min":0.377,"5min":0.32600000000000001,"15min":0.30299999999999999},"max":{"1min":0.65700000000000003,"5min":0.876,"15min":1.1060000000000001},"last":0.48899999999999999}]},{"osd":6,"last update":"Wed Nov 8 19:30:58 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.54700000000000004,"15min":0.57199999999999995},"min":{"1min":0.35199999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.71899999999999997,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.60299999999999998,"15min":0.60999999999999999},"min":{"1min":0.373,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.74099999999999999,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.56200000000000006}]},{"osd":7,"last update":"Wed Nov 8 19:30:58 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.55800000000000005,"15min":0.60999999999999999},"min":{"1min":0.41599999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.68000000000000005,"5min":0.84499999999999997,"15min":1.141},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.59199999999999997,"15min":0.63},"min":{"1min":0.41399999999999998,"5min":0.38700000000000001,"15min":0.33000000000000002},"max":{"1min":0.76400000000000001,"5min":0.86399999999999999,"15min":0.86799999999999999},"last":0.57699999999999996}]}]},{"osd":5,"up_from":13,"seq":55834574939,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28308,"kb_used_data":1292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343532,"statfs":{"total":96636764160,"available":96607776768,"internally_reserved":0,"allocated":1323008,"data_stored":646397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:31:52 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.58599999999999997,"15min":0.54200000000000004},"min":{"1min":0.35499999999999998,"5min":0.31900000000000001,"15min":0.307},"max":{"1min":0.76900000000000002,"5min":3.9689999999999999,"15min":3.9689999999999999},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55500000000000005,"15min":0.52500000000000002},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.67200000000000004,"5min":0.746,"15min":0.746},"last":0.56899999999999995}]},{"osd":1,"last update":"Wed Nov 8 19:31:52 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.56100000000000005,"15min":0.53600000000000003},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.72599999999999998,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.54700000000000004,"15min":0.54000000000000004},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.66500000000000004,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.52300000000000002}]},{"osd":2,"last update":"Wed Nov 8 19:31:52 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.56999999999999995,"15min":0.54800000000000004},"min":{"1min":0.43099999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.76400000000000001,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.39500000000000002,"5min":0.316,"15min":0.316},"max":{"1min":0.71599999999999997,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.50800000000000001}]},{"osd":3,"last update":"Wed Nov 8 19:31:52 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59899999999999998,"15min":0.56499999999999995},"min":{"1min":0.46000000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.80000000000000004,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58999999999999997,"15min":0.59499999999999997},"min":{"1min":0.38600000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.78700000000000003,"5min":0.87,"15min":0.87},"last":0.59299999999999997}]},{"osd":4,"last update":"Wed Nov 8 19:31:52 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.48499999999999999,"15min":0.53300000000000003},"min":{"1min":0.34699999999999998,"5min":0.315,"15min":0.315},"max":{"1min":0.74099999999999999,"5min":0.80500000000000005,"15min":0.86799999999999999},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.53000000000000003,"15min":0.55800000000000005},"min":{"1min":0.36199999999999999,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.48999999999999999}]},{"osd":6,"last update":"Wed Nov 8 19:31:52 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.54800000000000004,"15min":0.58999999999999997},"min":{"1min":0.35799999999999998,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.70299999999999996,"5min":0.755,"15min":0.77800000000000002},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.58399999999999996,"15min":0.60399999999999998},"min":{"1min":0.40200000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.71599999999999997,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.36199999999999999}]},{"osd":7,"last update":"Wed Nov 8 19:31:52 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.57099999999999995,"15min":0.59599999999999997},"min":{"1min":0.37,"5min":0.37,"15min":0.36899999999999999},"max":{"1min":0.75900000000000001,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.58699999999999997,"15min":0.626},"min":{"1min":0.433,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.73499999999999999,"5min":0.81299999999999994,"15min":0.91900000000000004},"last":0.40899999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-08T19:32:03.746 INFO:tasks.ceph:pgid 12.d last_scrub_stamp 2023-11-08T19:27:07.167285+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=7, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:03.747 INFO:tasks.ceph:pgid 11.f last_scrub_stamp 2023-11-08T19:27:06.162282+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=6, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:03.748 INFO:tasks.ceph:pgid 11.5 last_scrub_stamp 2023-11-08T19:27:06.162282+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=6, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:03.749 INFO:tasks.ceph:pgid 12.2 last_scrub_stamp 2023-11-08T19:27:07.167285+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=7, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:03.750 INFO:tasks.ceph:pgid 11.4 last_scrub_stamp 2023-11-08T19:27:06.162282+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=6, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:03.751 INFO:tasks.ceph:pgid 11.1e last_scrub_stamp 2023-11-08T19:27:06.162282+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=6, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:03.751 INFO:tasks.ceph:pgid 12.1a last_scrub_stamp 2023-11-08T19:27:07.167285+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=7, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:03.752 INFO:tasks.ceph:pgid 11.1d last_scrub_stamp 2023-11-08T19:27:06.162282+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=6, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:03.752 INFO:tasks.ceph:pgid 12.1b last_scrub_stamp 2023-11-08T19:27:07.167285+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=7, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:03.753 INFO:tasks.ceph:pgid 12.1e last_scrub_stamp 2023-11-08T19:27:07.167285+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=7, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:03.753 INFO:tasks.ceph:pgid 11.17 last_scrub_stamp 2023-11-08T19:27:06.162282+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=6, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:03.754 INFO:tasks.ceph:pgid 12.11 last_scrub_stamp 2023-11-08T19:27:07.167285+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=7, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:03.754 INFO:tasks.ceph:pgid 12.14 last_scrub_stamp 2023-11-08T19:27:07.167285+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=7, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:03.755 INFO:tasks.ceph:pgid 11.12 last_scrub_stamp 2023-11-08T19:27:06.162282+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=6, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=31, tm_sec=17, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:03.755 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-11-08T19:32:23.756 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-08T19:32:23.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:23.888+0000 7f745c412700 1 Processor -- start 2023-11-08T19:32:23.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:23.888+0000 7f745c412700 1 -- start start 2023-11-08T19:32:23.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:23.889+0000 7f745c412700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7454159680 0x7f74540cc7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:23.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:23.889+0000 7f745c412700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f74540b7200 0x7f74540cf3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:23.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:23.889+0000 7f745c412700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f745415ce10 con 0x7f74540b3260 2023-11-08T19:32:23.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:23.889+0000 7f745c412700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f745415c290 con 0x7f74540b7200 2023-11-08T19:32:23.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:23.889+0000 7f745c412700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f745415c570 con 0x7f7454159680 2023-11-08T19:32:23.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:23.890+0000 7f745a9af700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f74540b3260 0x7f745415f870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:53596/0 (socket says 172.21.15.143:53596) 2023-11-08T19:32:23.895 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:23.890+0000 7f745a9af700 1 -- 172.21.15.143:0/3317024464 learned_addr learned my addr 172.21.15.143:0/3317024464 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-11-08T19:32:23.895 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:23.890+0000 7f745a1ae700 1 --2- 172.21.15.143:0/3317024464 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7454159680 0x7f74540cc7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:23.895 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:23.890+0000 7f74599ad700 1 --2- 172.21.15.143:0/3317024464 >> v2:172.21.15.174:3300/0 conn(0x7f74540b7200 0x7f74540cf3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:23.896 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:23.890+0000 7f74591ac700 1 -- 172.21.15.143:0/3317024464 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1514213744 0 0) 0x7f745415ce10 con 0x7f74540b3260 2023-11-08T19:32:23.896 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:23.890+0000 7f74591ac700 1 -- 172.21.15.143:0/3317024464 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f74400035b0 con 0x7f74540b3260 2023-11-08T19:32:23.896 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:23.890+0000 7f74591ac700 1 -- 172.21.15.143:0/3317024464 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1663282631 0 0) 0x7f74400035b0 con 0x7f74540b3260 2023-11-08T19:32:23.896 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:23.890+0000 7f745a1ae700 1 -- 172.21.15.143:0/3317024464 >> v2:172.21.15.174:3300/0 conn(0x7f74540b7200 msgr2=0x7f74540cf3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:32:23.897 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:23.890+0000 7f745a1ae700 1 --2- 172.21.15.143:0/3317024464 >> v2:172.21.15.174:3300/0 conn(0x7f74540b7200 0x7f74540cf3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:23.897 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:23.890+0000 7f745a1ae700 1 -- 172.21.15.143:0/3317024464 >> v1:172.21.15.143:6789/0 conn(0x7f74540b3260 legacy=0x7f745415f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:23.897 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:23.890+0000 7f745a1ae700 1 -- 172.21.15.143:0/3317024464 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f745415c850 con 0x7f7454159680 2023-11-08T19:32:23.898 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:23.890+0000 7f74599ad700 1 --2- 172.21.15.143:0/3317024464 >> v2:172.21.15.174:3300/0 conn(0x7f74540b7200 0x7f74540cf3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T19:32:23.898 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:23.890+0000 7f745a1ae700 1 --2- 172.21.15.143:0/3317024464 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7454159680 0x7f74540cc7d0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7444009840 tx=0x7f7444009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c1dfcc3190389eb server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:23.898 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:23.891+0000 7f74591ac700 1 -- 172.21.15.143:0/3317024464 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f744c0019f0 con 0x7f7454159680 2023-11-08T19:32:23.899 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:23.891+0000 7f74591ac700 1 -- 172.21.15.143:0/3317024464 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7444004030 con 0x7f7454159680 2023-11-08T19:32:23.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:23.891+0000 7f74591ac700 1 -- 172.21.15.143:0/3317024464 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74440043c0 con 0x7f7454159680 2023-11-08T19:32:23.904 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:23.891+0000 7f745c412700 1 -- 172.21.15.143:0/3317024464 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7454159680 msgr2=0x7f74540cc7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:23.904 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:23.891+0000 7f745c412700 1 --2- 172.21.15.143:0/3317024464 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7454159680 0x7f74540cc7d0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7444009840 tx=0x7f7444009b50 comp rx=0 tx=0).stop 2023-11-08T19:32:23.905 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:23.891+0000 7f745c412700 1 -- 172.21.15.143:0/3317024464 shutdown_connections 2023-11-08T19:32:23.905 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:23.891+0000 7f745c412700 1 --2- 172.21.15.143:0/3317024464 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7454159680 0x7f74540cc7d0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:23.906 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:23.891+0000 7f745c412700 1 --2- 172.21.15.143:0/3317024464 >> v2:172.21.15.174:3300/0 conn(0x7f74540b7200 0x7f74540cf3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:23.906 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:23.891+0000 7f745c412700 1 -- 172.21.15.143:0/3317024464 >> 172.21.15.143:0/3317024464 conn(0x7f74540ce260 msgr2=0x7f74540ce670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:32:23.906 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:23.891+0000 7f745c412700 1 -- 172.21.15.143:0/3317024464 shutdown_connections 2023-11-08T19:32:23.907 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:23.891+0000 7f745c412700 1 -- 172.21.15.143:0/3317024464 wait complete. 2023-11-08T19:32:23.912 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:23.893+0000 7f745c412700 1 Processor -- start 2023-11-08T19:32:23.912 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:23.893+0000 7f745c412700 1 -- start start 2023-11-08T19:32:23.912 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:23.894+0000 7f745c412700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f74540b3260 0x7f74540b5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:23.913 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:23.894+0000 7f745c412700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f74540b7200 0x7f74540b2450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:23.913 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:23.894+0000 7f745c412700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7454159680 0x7f74540b2990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:23.913 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:23.894+0000 7f745c412700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74540b2280 con 0x7f7454159680 2023-11-08T19:32:23.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:23.894+0000 7f745c412700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74540b0f90 con 0x7f74540b3260 2023-11-08T19:32:23.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:23.894+0000 7f745c412700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74540b11e0 con 0x7f74540b7200 2023-11-08T19:32:23.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:23.894+0000 7f745a9af700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7454159680 0x7f74540b2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:23.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:23.894+0000 7f745a9af700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7454159680 0x7f74540b2990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57446/0 (socket says 172.21.15.143:57446) 2023-11-08T19:32:23.915 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:23.894+0000 7f745a9af700 1 -- 172.21.15.143:0/3875531139 learned_addr learned my addr 172.21.15.143:0/3875531139 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:32:23.915 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:23.894+0000 7f745a1ae700 1 --2- 172.21.15.143:0/3875531139 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f74540b3260 0x7f74540b5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:23.915 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:23.894+0000 7f74599ad700 1 --2- 172.21.15.143:0/3875531139 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f74540b7200 0x7f74540b2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:23.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:23.894+0000 7f745a9af700 1 -- 172.21.15.143:0/3875531139 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f74540b7200 msgr2=0x7f74540b2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:23.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:23.894+0000 7f745a9af700 1 --2- 172.21.15.143:0/3875531139 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f74540b7200 0x7f74540b2450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:23.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:23.894+0000 7f745a9af700 1 -- 172.21.15.143:0/3875531139 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f74540b3260 msgr2=0x7f74540b5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:23.917 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:23.894+0000 7f745a9af700 1 --2- 172.21.15.143:0/3875531139 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f74540b3260 0x7f74540b5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:23.917 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:23.894+0000 7f745a9af700 1 -- 172.21.15.143:0/3875531139 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74500094a0 con 0x7f7454159680 2023-11-08T19:32:23.917 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:23.894+0000 7f745a9af700 1 --2- 172.21.15.143:0/3875531139 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7454159680 0x7f74540b2990 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f744c00cc00 tx=0x7f744c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:23.917 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:23.894+0000 7f744affd700 1 -- 172.21.15.143:0/3875531139 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f744c00ab80 con 0x7f7454159680 2023-11-08T19:32:23.918 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:23.895+0000 7f744affd700 1 -- 172.21.15.143:0/3875531139 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f744c00ad00 con 0x7f7454159680 2023-11-08T19:32:23.918 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:23.895+0000 7f745c412700 1 -- 172.21.15.143:0/3875531139 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74440094a0 con 0x7f7454159680 2023-11-08T19:32:23.918 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:23.895+0000 7f744affd700 1 -- 172.21.15.143:0/3875531139 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f744c007310 con 0x7f7454159680 2023-11-08T19:32:23.919 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:23.895+0000 7f745c412700 1 -- 172.21.15.143:0/3875531139 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74540b17a0 con 0x7f7454159680 2023-11-08T19:32:23.919 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:23.895+0000 7f745c412700 1 -- 172.21.15.143:0/3875531139 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f745404eee0 con 0x7f7454159680 2023-11-08T19:32:23.919 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:23.896+0000 7f744affd700 1 -- 172.21.15.143:0/3875531139 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f744c007470 con 0x7f7454159680 2023-11-08T19:32:23.920 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:23.896+0000 7f744affd700 1 --2- 172.21.15.143:0/3875531139 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f743806af00 0x7f743806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:23.920 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:23.897+0000 7f744affd700 1 -- 172.21.15.143:0/3875531139 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6651+0+0 (secure 0 0 0) 0x7f744c0897e0 con 0x7f7454159680 2023-11-08T19:32:23.920 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:23.897+0000 7f745a1ae700 1 --2- 172.21.15.143:0/3875531139 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f743806af00 0x7f743806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:23.921 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:23.897+0000 7f745a1ae700 1 --2- 172.21.15.143:0/3875531139 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f743806af00 0x7f743806d3c0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f74440090b0 tx=0x7f7444008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:23.921 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:23.901+0000 7f744affd700 1 -- 172.21.15.143:0/3875531139 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f744c054570 con 0x7f7454159680 2023-11-08T19:32:24.086 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.085+0000 7f745c412700 1 -- 172.21.15.143:0/3875531139 --> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f745415cb30 con 0x7f743806af00 2023-11-08T19:32:24.094 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.093+0000 7f744affd700 1 -- 172.21.15.143:0/3875531139 <== mgr.4099 v2:172.21.15.174:6832/4037280178 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+295494 (secure 0 0 0) 0x7f74440044a0 con 0x7f743806af00 2023-11-08T19:32:24.099 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:32:24.102 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.097+0000 7f745c412700 1 -- 172.21.15.143:0/3875531139 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f743806af00 msgr2=0x7f743806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:24.102 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.097+0000 7f745c412700 1 --2- 172.21.15.143:0/3875531139 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f743806af00 0x7f743806d3c0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f74440090b0 tx=0x7f7444008030 comp rx=0 tx=0).stop 2023-11-08T19:32:24.106 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.097+0000 7f745c412700 1 -- 172.21.15.143:0/3875531139 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7454159680 msgr2=0x7f74540b2990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:24.106 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.097+0000 7f745c412700 1 --2- 172.21.15.143:0/3875531139 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7454159680 0x7f74540b2990 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f744c00cc00 tx=0x7f744c00a060 comp rx=0 tx=0).stop 2023-11-08T19:32:24.107 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.097+0000 7f745c412700 1 -- 172.21.15.143:0/3875531139 shutdown_connections 2023-11-08T19:32:24.107 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.097+0000 7f745c412700 1 --2- 172.21.15.143:0/3875531139 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f743806af00 0x7f743806d3c0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:24.107 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.097+0000 7f745c412700 1 --2- 172.21.15.143:0/3875531139 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7454159680 0x7f74540b2990 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:24.107 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.097+0000 7f745c412700 1 --2- 172.21.15.143:0/3875531139 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f74540b3260 0x7f74540b5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:24.108 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.097+0000 7f745c412700 1 --2- 172.21.15.143:0/3875531139 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f74540b7200 0x7f74540b2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:24.108 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.097+0000 7f745c412700 1 -- 172.21.15.143:0/3875531139 >> 172.21.15.143:0/3875531139 conn(0x7f74540ce260 msgr2=0x7f745415f730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:32:24.108 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.097+0000 7f745c412700 1 -- 172.21.15.143:0/3875531139 shutdown_connections 2023-11-08T19:32:24.109 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.097+0000 7f745c412700 1 -- 172.21.15.143:0/3875531139 wait complete. 2023-11-08T19:32:24.109 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2023-11-08T19:32:24.111 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":329,"stamp":"2023-11-08T19:32:22.643194+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":244264,"kb_used_data":11496,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754730456,"statfs":{"total":773094113280,"available":772843986944,"internally_reserved":0,"allocated":11771904,"data_stored":6351912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004032"},"pg_stats":[{"pgid":"12.d","version":"0'0","reported_seq":63,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:13.914217+0000","last_change":"2023-11-08T19:32:13.914258+0000","last_active":"2023-11-08T19:32:13.914217+0000","last_peered":"2023-11-08T19:32:13.914217+0000","last_clean":"2023-11-08T19:32:13.914217+0000","last_became_active":"2023-11-08T19:27:08.190440+0000","last_became_peered":"2023-11-08T19:27:08.190440+0000","last_unstale":"2023-11-08T19:32:13.914217+0000","last_undegraded":"2023-11-08T19:32:13.914217+0000","last_fullsized":"2023-11-08T19:32:13.914217+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:13.914088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:13.914088+0000","last_clean_scrub_stamp":"2023-11-08T19:32:13.914088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:42:58.194651+0000","scrub_duration":0.00220586,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"107'119","reported_seq":266,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:27.048714+0000","last_change":"2023-11-08T19:31:27.048761+0000","last_active":"2023-11-08T19:31:27.048714+0000","last_peered":"2023-11-08T19:31:27.048714+0000","last_clean":"2023-11-08T19:31:27.048714+0000","last_became_active":"2023-11-08T19:24:45.887048+0000","last_became_peered":"2023-11-08T19:24:45.887048+0000","last_unstale":"2023-11-08T19:31:27.048714+0000","last_undegraded":"2023-11-08T19:31:27.048714+0000","last_fullsized":"2023-11-08T19:31:27.048714+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"107'119","last_scrub_stamp":"2023-11-08T19:31:27.048604+0000","last_deep_scrub":"107'119","last_deep_scrub_stamp":"2023-11-08T19:31:27.048604+0000","last_clean_scrub_stamp":"2023-11-08T19:31:27.048604+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:16:40.175526+0000","scrub_duration":0.0086873920000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":88,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:32.804593+0000","last_change":"2023-11-08T19:31:32.804615+0000","last_active":"2023-11-08T19:31:32.804593+0000","last_peered":"2023-11-08T19:31:32.804593+0000","last_clean":"2023-11-08T19:31:32.804593+0000","last_became_active":"2023-11-08T19:26:04.603056+0000","last_became_peered":"2023-11-08T19:26:04.603056+0000","last_unstale":"2023-11-08T19:31:32.804593+0000","last_undegraded":"2023-11-08T19:31:32.804593+0000","last_fullsized":"2023-11-08T19:31:32.804593+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:32.804495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:32.804495+0000","last_clean_scrub_stamp":"2023-11-08T19:31:32.804495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:16:22.607276+0000","scrub_duration":0.0015962439999999999,"objects_trimmed":0,"snaptrim_duration":0.000517414,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":68,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:45.765293+0000","last_change":"2023-11-08T19:31:45.765332+0000","last_active":"2023-11-08T19:31:45.765293+0000","last_peered":"2023-11-08T19:31:45.765293+0000","last_clean":"2023-11-08T19:31:45.765293+0000","last_became_active":"2023-11-08T19:27:07.185776+0000","last_became_peered":"2023-11-08T19:27:07.185776+0000","last_unstale":"2023-11-08T19:31:45.765293+0000","last_undegraded":"2023-11-08T19:31:45.765293+0000","last_fullsized":"2023-11-08T19:31:45.765293+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:45.765147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:45.765147+0000","last_clean_scrub_stamp":"2023-11-08T19:31:45.765147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:48:06.344962+0000","scrub_duration":0.0020421279999999998,"objects_trimmed":0,"snaptrim_duration":0.00031747200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":61,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:51.824231+0000","last_change":"2023-11-08T19:31:51.824255+0000","last_active":"2023-11-08T19:31:51.824231+0000","last_peered":"2023-11-08T19:31:51.824231+0000","last_clean":"2023-11-08T19:31:51.824231+0000","last_became_active":"2023-11-08T19:27:08.187863+0000","last_became_peered":"2023-11-08T19:27:08.187863+0000","last_unstale":"2023-11-08T19:31:51.824231+0000","last_undegraded":"2023-11-08T19:31:51.824231+0000","last_fullsized":"2023-11-08T19:31:51.824231+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:51.824119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:51.824119+0000","last_clean_scrub_stamp":"2023-11-08T19:31:51.824119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:42:55.031701+0000","scrub_duration":0.0016592600000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":87,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:42.107596+0000","last_change":"2023-11-08T19:31:42.107596+0000","last_active":"2023-11-08T19:31:42.107596+0000","last_peered":"2023-11-08T19:31:42.107596+0000","last_clean":"2023-11-08T19:31:42.107596+0000","last_became_active":"2023-11-08T19:26:04.605333+0000","last_became_peered":"2023-11-08T19:26:04.605333+0000","last_unstale":"2023-11-08T19:31:42.107596+0000","last_undegraded":"2023-11-08T19:31:42.107596+0000","last_fullsized":"2023-11-08T19:31:42.107596+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:42.107419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:42.107419+0000","last_clean_scrub_stamp":"2023-11-08T19:31:42.107419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:13:05.993137+0000","scrub_duration":0.002168569,"objects_trimmed":0,"snaptrim_duration":0.00064606400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":70,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:04.978201+0000","last_change":"2023-11-08T19:32:04.978225+0000","last_active":"2023-11-08T19:32:04.978201+0000","last_peered":"2023-11-08T19:32:04.978201+0000","last_clean":"2023-11-08T19:32:04.978201+0000","last_became_active":"2023-11-08T19:27:07.189939+0000","last_became_peered":"2023-11-08T19:27:07.189939+0000","last_unstale":"2023-11-08T19:32:04.978201+0000","last_undegraded":"2023-11-08T19:32:04.978201+0000","last_fullsized":"2023-11-08T19:32:04.978201+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:04.978079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:04.978079+0000","last_clean_scrub_stamp":"2023-11-08T19:32:04.978079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:00:36.780786+0000","scrub_duration":0.001943944,"objects_trimmed":0,"snaptrim_duration":0.00036507500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":57,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:41.865868+0000","last_change":"2023-11-08T19:31:41.865899+0000","last_active":"2023-11-08T19:31:41.865868+0000","last_peered":"2023-11-08T19:31:41.865868+0000","last_clean":"2023-11-08T19:31:41.865868+0000","last_became_active":"2023-11-08T19:27:08.182648+0000","last_became_peered":"2023-11-08T19:27:08.182648+0000","last_unstale":"2023-11-08T19:31:41.865868+0000","last_undegraded":"2023-11-08T19:31:41.865868+0000","last_fullsized":"2023-11-08T19:31:41.865868+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:41.865752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:41.865752+0000","last_clean_scrub_stamp":"2023-11-08T19:31:41.865752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:05:41.475657+0000","scrub_duration":0.0018669470000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":99,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:58.949943+0000","last_change":"2023-11-08T19:31:58.949943+0000","last_active":"2023-11-08T19:31:58.949943+0000","last_peered":"2023-11-08T19:31:58.949943+0000","last_clean":"2023-11-08T19:31:58.949943+0000","last_became_active":"2023-11-08T19:26:04.608379+0000","last_became_peered":"2023-11-08T19:26:04.608379+0000","last_unstale":"2023-11-08T19:31:58.949943+0000","last_undegraded":"2023-11-08T19:31:58.949943+0000","last_fullsized":"2023-11-08T19:31:58.949943+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:58.949790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:58.949790+0000","last_clean_scrub_stamp":"2023-11-08T19:31:58.949790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:04:49.781093+0000","scrub_duration":0.001585752,"objects_trimmed":0,"snaptrim_duration":0.000529657,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":64,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:54.122758+0000","last_change":"2023-11-08T19:31:54.122795+0000","last_active":"2023-11-08T19:31:54.122758+0000","last_peered":"2023-11-08T19:31:54.122758+0000","last_clean":"2023-11-08T19:31:54.122758+0000","last_became_active":"2023-11-08T19:27:07.195231+0000","last_became_peered":"2023-11-08T19:27:07.195231+0000","last_unstale":"2023-11-08T19:31:54.122758+0000","last_undegraded":"2023-11-08T19:31:54.122758+0000","last_fullsized":"2023-11-08T19:31:54.122758+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:54.122605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:54.122605+0000","last_clean_scrub_stamp":"2023-11-08T19:31:54.122605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:43:15.004601+0000","scrub_duration":0.002189346,"objects_trimmed":0,"snaptrim_duration":0.000441221,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":57,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:48.764425+0000","last_change":"2023-11-08T19:31:48.764447+0000","last_active":"2023-11-08T19:31:48.764425+0000","last_peered":"2023-11-08T19:31:48.764425+0000","last_clean":"2023-11-08T19:31:48.764425+0000","last_became_active":"2023-11-08T19:27:08.195074+0000","last_became_peered":"2023-11-08T19:27:08.195074+0000","last_unstale":"2023-11-08T19:31:48.764425+0000","last_undegraded":"2023-11-08T19:31:48.764425+0000","last_fullsized":"2023-11-08T19:31:48.764425+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:48.764327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:48.764327+0000","last_clean_scrub_stamp":"2023-11-08T19:31:48.764327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:30:53.197298+0000","scrub_duration":0.001484452,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":87,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:29.665227+0000","last_change":"2023-11-08T19:31:29.665227+0000","last_active":"2023-11-08T19:31:29.665227+0000","last_peered":"2023-11-08T19:31:29.665227+0000","last_clean":"2023-11-08T19:31:29.665227+0000","last_became_active":"2023-11-08T19:26:04.601685+0000","last_became_peered":"2023-11-08T19:26:04.601685+0000","last_unstale":"2023-11-08T19:31:29.665227+0000","last_undegraded":"2023-11-08T19:31:29.665227+0000","last_fullsized":"2023-11-08T19:31:29.665227+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:29.665096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:29.665096+0000","last_clean_scrub_stamp":"2023-11-08T19:31:29.665096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:57:56.435973+0000","scrub_duration":0.001580171,"objects_trimmed":0,"snaptrim_duration":0.00061044099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":64,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:34.809538+0000","last_change":"2023-11-08T19:31:34.809538+0000","last_active":"2023-11-08T19:31:34.809538+0000","last_peered":"2023-11-08T19:31:34.809538+0000","last_clean":"2023-11-08T19:31:34.809538+0000","last_became_active":"2023-11-08T19:27:07.189386+0000","last_became_peered":"2023-11-08T19:27:07.189386+0000","last_unstale":"2023-11-08T19:31:34.809538+0000","last_undegraded":"2023-11-08T19:31:34.809538+0000","last_fullsized":"2023-11-08T19:31:34.809538+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:34.809370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:34.809370+0000","last_clean_scrub_stamp":"2023-11-08T19:31:34.809370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:04:11.405772+0000","scrub_duration":0.0021291940000000001,"objects_trimmed":0,"snaptrim_duration":0.00049390100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":61,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:57.832690+0000","last_change":"2023-11-08T19:31:57.832728+0000","last_active":"2023-11-08T19:31:57.832690+0000","last_peered":"2023-11-08T19:31:57.832690+0000","last_clean":"2023-11-08T19:31:57.832690+0000","last_became_active":"2023-11-08T19:27:08.189088+0000","last_became_peered":"2023-11-08T19:27:08.189088+0000","last_unstale":"2023-11-08T19:31:57.832690+0000","last_undegraded":"2023-11-08T19:31:57.832690+0000","last_fullsized":"2023-11-08T19:31:57.832690+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:57.832572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:57.832572+0000","last_clean_scrub_stamp":"2023-11-08T19:31:57.832572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:00:29.635279+0000","scrub_duration":0.0017369670000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":87,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:21.041246+0000","last_change":"2023-11-08T19:31:21.041265+0000","last_active":"2023-11-08T19:31:21.041246+0000","last_peered":"2023-11-08T19:31:21.041246+0000","last_clean":"2023-11-08T19:31:21.041246+0000","last_became_active":"2023-11-08T19:26:04.603088+0000","last_became_peered":"2023-11-08T19:26:04.603088+0000","last_unstale":"2023-11-08T19:31:21.041246+0000","last_undegraded":"2023-11-08T19:31:21.041246+0000","last_fullsized":"2023-11-08T19:31:21.041246+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:21.041166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:21.041166+0000","last_clean_scrub_stamp":"2023-11-08T19:31:21.041166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:25:28.147920+0000","scrub_duration":0.001667876,"objects_trimmed":0,"snaptrim_duration":0.00038180899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":64,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:28.086650+0000","last_change":"2023-11-08T19:31:28.086650+0000","last_active":"2023-11-08T19:31:28.086650+0000","last_peered":"2023-11-08T19:31:28.086650+0000","last_clean":"2023-11-08T19:31:28.086650+0000","last_became_active":"2023-11-08T19:27:07.191589+0000","last_became_peered":"2023-11-08T19:27:07.191589+0000","last_unstale":"2023-11-08T19:31:28.086650+0000","last_undegraded":"2023-11-08T19:31:28.086650+0000","last_fullsized":"2023-11-08T19:31:28.086650+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:28.086495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:28.086495+0000","last_clean_scrub_stamp":"2023-11-08T19:31:28.086495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:17:02.712148+0000","scrub_duration":0.0016141980000000001,"objects_trimmed":0,"snaptrim_duration":0.000729527,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":57,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:53.564766+0000","last_change":"2023-11-08T19:31:53.564800+0000","last_active":"2023-11-08T19:31:53.564766+0000","last_peered":"2023-11-08T19:31:53.564766+0000","last_clean":"2023-11-08T19:31:53.564766+0000","last_became_active":"2023-11-08T19:27:08.192997+0000","last_became_peered":"2023-11-08T19:27:08.192997+0000","last_unstale":"2023-11-08T19:31:53.564766+0000","last_undegraded":"2023-11-08T19:31:53.564766+0000","last_fullsized":"2023-11-08T19:31:53.564766+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:53.564655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:53.564655+0000","last_clean_scrub_stamp":"2023-11-08T19:31:53.564655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:17:37.295865+0000","scrub_duration":0.0019884429999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":92,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:43.790033+0000","last_change":"2023-11-08T19:31:43.790051+0000","last_active":"2023-11-08T19:31:43.790033+0000","last_peered":"2023-11-08T19:31:43.790033+0000","last_clean":"2023-11-08T19:31:43.790033+0000","last_became_active":"2023-11-08T19:26:04.598973+0000","last_became_peered":"2023-11-08T19:26:04.598973+0000","last_unstale":"2023-11-08T19:31:43.790033+0000","last_undegraded":"2023-11-08T19:31:43.790033+0000","last_fullsized":"2023-11-08T19:31:43.790033+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:43.789955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:43.789955+0000","last_clean_scrub_stamp":"2023-11-08T19:31:43.789955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:00:05.697661+0000","scrub_duration":0.0016943609999999999,"objects_trimmed":0,"snaptrim_duration":0.00031755900000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":68,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:47.752890+0000","last_change":"2023-11-08T19:31:47.752912+0000","last_active":"2023-11-08T19:31:47.752890+0000","last_peered":"2023-11-08T19:31:47.752890+0000","last_clean":"2023-11-08T19:31:47.752890+0000","last_became_active":"2023-11-08T19:27:07.190132+0000","last_became_peered":"2023-11-08T19:27:07.190132+0000","last_unstale":"2023-11-08T19:31:47.752890+0000","last_undegraded":"2023-11-08T19:31:47.752890+0000","last_fullsized":"2023-11-08T19:31:47.752890+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:47.752785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:47.752785+0000","last_clean_scrub_stamp":"2023-11-08T19:31:47.752785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:31:24.364068+0000","scrub_duration":0.0014687420000000001,"objects_trimmed":0,"snaptrim_duration":0.00054126000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":57,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:45.481002+0000","last_change":"2023-11-08T19:31:45.481039+0000","last_active":"2023-11-08T19:31:45.481002+0000","last_peered":"2023-11-08T19:31:45.481002+0000","last_clean":"2023-11-08T19:31:45.481002+0000","last_became_active":"2023-11-08T19:27:08.193912+0000","last_became_peered":"2023-11-08T19:27:08.193912+0000","last_unstale":"2023-11-08T19:31:45.481002+0000","last_undegraded":"2023-11-08T19:31:45.481002+0000","last_fullsized":"2023-11-08T19:31:45.481002+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:45.480846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:45.480846+0000","last_clean_scrub_stamp":"2023-11-08T19:31:45.480846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:27:16.952754+0000","scrub_duration":0.0021767169999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":87,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:22.069061+0000","last_change":"2023-11-08T19:31:22.069101+0000","last_active":"2023-11-08T19:31:22.069061+0000","last_peered":"2023-11-08T19:31:22.069061+0000","last_clean":"2023-11-08T19:31:22.069061+0000","last_became_active":"2023-11-08T19:26:04.599999+0000","last_became_peered":"2023-11-08T19:26:04.599999+0000","last_unstale":"2023-11-08T19:31:22.069061+0000","last_undegraded":"2023-11-08T19:31:22.069061+0000","last_fullsized":"2023-11-08T19:31:22.069061+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:22.068925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:22.068925+0000","last_clean_scrub_stamp":"2023-11-08T19:31:22.068925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:59:28.785563+0000","scrub_duration":0.0018102260000000001,"objects_trimmed":0,"snaptrim_duration":0.00025605700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":68,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:46.723527+0000","last_change":"2023-11-08T19:31:46.723563+0000","last_active":"2023-11-08T19:31:46.723527+0000","last_peered":"2023-11-08T19:31:46.723527+0000","last_clean":"2023-11-08T19:31:46.723527+0000","last_became_active":"2023-11-08T19:27:07.189135+0000","last_became_peered":"2023-11-08T19:27:07.189135+0000","last_unstale":"2023-11-08T19:31:46.723527+0000","last_undegraded":"2023-11-08T19:31:46.723527+0000","last_fullsized":"2023-11-08T19:31:46.723527+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:46.723408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:46.723408+0000","last_clean_scrub_stamp":"2023-11-08T19:31:46.723408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:20:55.064105+0000","scrub_duration":0.002142251,"objects_trimmed":0,"snaptrim_duration":0.00051794299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":63,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:34.286026+0000","last_change":"2023-11-08T19:31:34.286026+0000","last_active":"2023-11-08T19:31:34.286026+0000","last_peered":"2023-11-08T19:31:34.286026+0000","last_clean":"2023-11-08T19:31:34.286026+0000","last_became_active":"2023-11-08T19:27:08.186643+0000","last_became_peered":"2023-11-08T19:27:08.186643+0000","last_unstale":"2023-11-08T19:31:34.286026+0000","last_undegraded":"2023-11-08T19:31:34.286026+0000","last_fullsized":"2023-11-08T19:31:34.286026+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:34.285872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:34.285872+0000","last_clean_scrub_stamp":"2023-11-08T19:31:34.285872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:22:31.137393+0000","scrub_duration":0.0015767699999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":99,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:44.925942+0000","last_change":"2023-11-08T19:31:44.925942+0000","last_active":"2023-11-08T19:31:44.925942+0000","last_peered":"2023-11-08T19:31:44.925942+0000","last_clean":"2023-11-08T19:31:44.925942+0000","last_became_active":"2023-11-08T19:26:04.603320+0000","last_became_peered":"2023-11-08T19:26:04.603320+0000","last_unstale":"2023-11-08T19:31:44.925942+0000","last_undegraded":"2023-11-08T19:31:44.925942+0000","last_fullsized":"2023-11-08T19:31:44.925942+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:44.925801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:44.925801+0000","last_clean_scrub_stamp":"2023-11-08T19:31:44.925801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:15:37.041939+0000","scrub_duration":0.0014464549999999999,"objects_trimmed":0,"snaptrim_duration":0.00063225,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":68,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:48.744345+0000","last_change":"2023-11-08T19:31:48.744367+0000","last_active":"2023-11-08T19:31:48.744345+0000","last_peered":"2023-11-08T19:31:48.744345+0000","last_clean":"2023-11-08T19:31:48.744345+0000","last_became_active":"2023-11-08T19:27:07.189895+0000","last_became_peered":"2023-11-08T19:27:07.189895+0000","last_unstale":"2023-11-08T19:31:48.744345+0000","last_undegraded":"2023-11-08T19:31:48.744345+0000","last_fullsized":"2023-11-08T19:31:48.744345+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:48.744245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:48.744245+0000","last_clean_scrub_stamp":"2023-11-08T19:31:48.744245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:40:58.059066+0000","scrub_duration":0.0018029750000000001,"objects_trimmed":0,"snaptrim_duration":0.00045654199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":88,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:24.725788+0000","last_change":"2023-11-08T19:31:24.725829+0000","last_active":"2023-11-08T19:31:24.725788+0000","last_peered":"2023-11-08T19:31:24.725788+0000","last_clean":"2023-11-08T19:31:24.725788+0000","last_became_active":"2023-11-08T19:26:04.603725+0000","last_became_peered":"2023-11-08T19:26:04.603725+0000","last_unstale":"2023-11-08T19:31:24.725788+0000","last_undegraded":"2023-11-08T19:31:24.725788+0000","last_fullsized":"2023-11-08T19:31:24.725788+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:24.725673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:24.725673+0000","last_clean_scrub_stamp":"2023-11-08T19:31:24.725673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:15:54.517242+0000","scrub_duration":0.00144779,"objects_trimmed":0,"snaptrim_duration":0.00053365400000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":64,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:35.727927+0000","last_change":"2023-11-08T19:31:35.727950+0000","last_active":"2023-11-08T19:31:35.727927+0000","last_peered":"2023-11-08T19:31:35.727927+0000","last_clean":"2023-11-08T19:31:35.727927+0000","last_became_active":"2023-11-08T19:27:07.189595+0000","last_became_peered":"2023-11-08T19:27:07.189595+0000","last_unstale":"2023-11-08T19:31:35.727927+0000","last_undegraded":"2023-11-08T19:31:35.727927+0000","last_fullsized":"2023-11-08T19:31:35.727927+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:35.727812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:35.727812+0000","last_clean_scrub_stamp":"2023-11-08T19:31:35.727812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:00:59.786393+0000","scrub_duration":0.0019661430000000001,"objects_trimmed":0,"snaptrim_duration":0.000323479,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":57,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:55.095220+0000","last_change":"2023-11-08T19:31:55.095261+0000","last_active":"2023-11-08T19:31:55.095220+0000","last_peered":"2023-11-08T19:31:55.095220+0000","last_clean":"2023-11-08T19:31:55.095220+0000","last_became_active":"2023-11-08T19:27:08.184765+0000","last_became_peered":"2023-11-08T19:27:08.184765+0000","last_unstale":"2023-11-08T19:31:55.095220+0000","last_undegraded":"2023-11-08T19:31:55.095220+0000","last_fullsized":"2023-11-08T19:31:55.095220+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:55.095069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:55.095069+0000","last_clean_scrub_stamp":"2023-11-08T19:31:55.095069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:54:29.082954+0000","scrub_duration":0.001731813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":92,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:32.544440+0000","last_change":"2023-11-08T19:31:32.544462+0000","last_active":"2023-11-08T19:31:32.544440+0000","last_peered":"2023-11-08T19:31:32.544440+0000","last_clean":"2023-11-08T19:31:32.544440+0000","last_became_active":"2023-11-08T19:26:04.600115+0000","last_became_peered":"2023-11-08T19:26:04.600115+0000","last_unstale":"2023-11-08T19:31:32.544440+0000","last_undegraded":"2023-11-08T19:31:32.544440+0000","last_fullsized":"2023-11-08T19:31:32.544440+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:32.544303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:32.544303+0000","last_clean_scrub_stamp":"2023-11-08T19:31:32.544303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:35:56.347935+0000","scrub_duration":0.002175448,"objects_trimmed":0,"snaptrim_duration":0.00065803900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":64,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:43.455210+0000","last_change":"2023-11-08T19:31:43.455229+0000","last_active":"2023-11-08T19:31:43.455210+0000","last_peered":"2023-11-08T19:31:43.455210+0000","last_clean":"2023-11-08T19:31:43.455210+0000","last_became_active":"2023-11-08T19:27:07.191460+0000","last_became_peered":"2023-11-08T19:27:07.191460+0000","last_unstale":"2023-11-08T19:31:43.455210+0000","last_undegraded":"2023-11-08T19:31:43.455210+0000","last_fullsized":"2023-11-08T19:31:43.455210+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:43.455118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:43.455118+0000","last_clean_scrub_stamp":"2023-11-08T19:31:43.455118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:48:33.020893+0000","scrub_duration":0.001194278,"objects_trimmed":0,"snaptrim_duration":0.00044907500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":63,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:43.455214+0000","last_change":"2023-11-08T19:31:43.455237+0000","last_active":"2023-11-08T19:31:43.455214+0000","last_peered":"2023-11-08T19:31:43.455214+0000","last_clean":"2023-11-08T19:31:43.455214+0000","last_became_active":"2023-11-08T19:27:08.191520+0000","last_became_peered":"2023-11-08T19:27:08.191520+0000","last_unstale":"2023-11-08T19:31:43.455214+0000","last_undegraded":"2023-11-08T19:31:43.455214+0000","last_fullsized":"2023-11-08T19:31:43.455214+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:43.455101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:43.455101+0000","last_clean_scrub_stamp":"2023-11-08T19:31:43.455101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:19:44.483623+0000","scrub_duration":0.001510302,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":98,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:29.015607+0000","last_change":"2023-11-08T19:31:29.015630+0000","last_active":"2023-11-08T19:31:29.015607+0000","last_peered":"2023-11-08T19:31:29.015607+0000","last_clean":"2023-11-08T19:31:29.015607+0000","last_became_active":"2023-11-08T19:26:04.604519+0000","last_became_peered":"2023-11-08T19:26:04.604519+0000","last_unstale":"2023-11-08T19:31:29.015607+0000","last_undegraded":"2023-11-08T19:31:29.015607+0000","last_fullsized":"2023-11-08T19:31:29.015607+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:29.015490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:29.015490+0000","last_clean_scrub_stamp":"2023-11-08T19:31:29.015490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:35:52.819122+0000","scrub_duration":0.0020686609999999999,"objects_trimmed":0,"snaptrim_duration":0.00051248099999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":70,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:12.926935+0000","last_change":"2023-11-08T19:32:12.926958+0000","last_active":"2023-11-08T19:32:12.926935+0000","last_peered":"2023-11-08T19:32:12.926935+0000","last_clean":"2023-11-08T19:32:12.926935+0000","last_became_active":"2023-11-08T19:27:07.190800+0000","last_became_peered":"2023-11-08T19:27:07.190800+0000","last_unstale":"2023-11-08T19:32:12.926935+0000","last_undegraded":"2023-11-08T19:32:12.926935+0000","last_fullsized":"2023-11-08T19:32:12.926935+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:12.926825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:12.926825+0000","last_clean_scrub_stamp":"2023-11-08T19:32:12.926825+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:44:36.831180+0000","scrub_duration":0.001530757,"objects_trimmed":0,"snaptrim_duration":0.00069656099999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":57,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:00.185407+0000","last_change":"2023-11-08T19:32:00.185432+0000","last_active":"2023-11-08T19:32:00.185407+0000","last_peered":"2023-11-08T19:32:00.185407+0000","last_clean":"2023-11-08T19:32:00.185407+0000","last_became_active":"2023-11-08T19:27:08.187322+0000","last_became_peered":"2023-11-08T19:27:08.187322+0000","last_unstale":"2023-11-08T19:32:00.185407+0000","last_undegraded":"2023-11-08T19:32:00.185407+0000","last_fullsized":"2023-11-08T19:32:00.185407+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:00.185294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:00.185294+0000","last_clean_scrub_stamp":"2023-11-08T19:32:00.185294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:21:08.525991+0000","scrub_duration":0.0017404250000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":86,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:22.798579+0000","last_change":"2023-11-08T19:31:22.798598+0000","last_active":"2023-11-08T19:31:22.798579+0000","last_peered":"2023-11-08T19:31:22.798579+0000","last_clean":"2023-11-08T19:31:22.798579+0000","last_became_active":"2023-11-08T19:26:04.606967+0000","last_became_peered":"2023-11-08T19:26:04.606967+0000","last_unstale":"2023-11-08T19:31:22.798579+0000","last_undegraded":"2023-11-08T19:31:22.798579+0000","last_fullsized":"2023-11-08T19:31:22.798579+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:22.798496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:22.798496+0000","last_clean_scrub_stamp":"2023-11-08T19:31:22.798496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:28:48.162603+0000","scrub_duration":0.002120331,"objects_trimmed":0,"snaptrim_duration":0.00040971999999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":70,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:00.939224+0000","last_change":"2023-11-08T19:32:00.939249+0000","last_active":"2023-11-08T19:32:00.939224+0000","last_peered":"2023-11-08T19:32:00.939224+0000","last_clean":"2023-11-08T19:32:00.939224+0000","last_became_active":"2023-11-08T19:27:07.186977+0000","last_became_peered":"2023-11-08T19:27:07.186977+0000","last_unstale":"2023-11-08T19:32:00.939224+0000","last_undegraded":"2023-11-08T19:32:00.939224+0000","last_fullsized":"2023-11-08T19:32:00.939224+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:00.939097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:00.939097+0000","last_clean_scrub_stamp":"2023-11-08T19:32:00.939097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:43:04.146678+0000","scrub_duration":0.001633142,"objects_trimmed":0,"snaptrim_duration":0.00057675100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":57,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:58.458746+0000","last_change":"2023-11-08T19:31:58.458785+0000","last_active":"2023-11-08T19:31:58.458746+0000","last_peered":"2023-11-08T19:31:58.458746+0000","last_clean":"2023-11-08T19:31:58.458746+0000","last_became_active":"2023-11-08T19:27:08.193098+0000","last_became_peered":"2023-11-08T19:27:08.193098+0000","last_unstale":"2023-11-08T19:31:58.458746+0000","last_undegraded":"2023-11-08T19:31:58.458746+0000","last_fullsized":"2023-11-08T19:31:58.458746+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:58.458626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:58.458626+0000","last_clean_scrub_stamp":"2023-11-08T19:31:58.458626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:26:45.661191+0000","scrub_duration":0.0023236260000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":86,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:33.077049+0000","last_change":"2023-11-08T19:31:33.077074+0000","last_active":"2023-11-08T19:31:33.077049+0000","last_peered":"2023-11-08T19:31:33.077049+0000","last_clean":"2023-11-08T19:31:33.077049+0000","last_became_active":"2023-11-08T19:26:04.602111+0000","last_became_peered":"2023-11-08T19:26:04.602111+0000","last_unstale":"2023-11-08T19:31:33.077049+0000","last_undegraded":"2023-11-08T19:31:33.077049+0000","last_fullsized":"2023-11-08T19:31:33.077049+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:33.076926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:33.076926+0000","last_clean_scrub_stamp":"2023-11-08T19:31:33.076926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:09:46.810104+0000","scrub_duration":0.0022106909999999999,"objects_trimmed":0,"snaptrim_duration":0.00056197899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":70,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:22.258974+0000","last_change":"2023-11-08T19:31:22.259009+0000","last_active":"2023-11-08T19:31:22.258974+0000","last_peered":"2023-11-08T19:31:22.258974+0000","last_clean":"2023-11-08T19:31:22.258974+0000","last_became_active":"2023-11-08T19:27:07.188657+0000","last_became_peered":"2023-11-08T19:27:07.188657+0000","last_unstale":"2023-11-08T19:31:22.258974+0000","last_undegraded":"2023-11-08T19:31:22.258974+0000","last_fullsized":"2023-11-08T19:31:22.258974+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:22.258851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:22.258851+0000","last_clean_scrub_stamp":"2023-11-08T19:31:22.258851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:41:50.529438+0000","scrub_duration":0.001495632,"objects_trimmed":0,"snaptrim_duration":0.001227393,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":63,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:31.191488+0000","last_change":"2023-11-08T19:31:31.191513+0000","last_active":"2023-11-08T19:31:31.191488+0000","last_peered":"2023-11-08T19:31:31.191488+0000","last_clean":"2023-11-08T19:31:31.191488+0000","last_became_active":"2023-11-08T19:27:08.188240+0000","last_became_peered":"2023-11-08T19:27:08.188240+0000","last_unstale":"2023-11-08T19:31:31.191488+0000","last_undegraded":"2023-11-08T19:31:31.191488+0000","last_fullsized":"2023-11-08T19:31:31.191488+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:31.191371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:31.191371+0000","last_clean_scrub_stamp":"2023-11-08T19:31:31.191371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:35:54.995003+0000","scrub_duration":0.001875533,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":63,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:39.374627+0000","last_change":"2023-11-08T19:31:39.374649+0000","last_active":"2023-11-08T19:31:39.374627+0000","last_peered":"2023-11-08T19:31:39.374627+0000","last_clean":"2023-11-08T19:31:39.374627+0000","last_became_active":"2023-11-08T19:27:08.187880+0000","last_became_peered":"2023-11-08T19:27:08.187880+0000","last_unstale":"2023-11-08T19:31:39.374627+0000","last_undegraded":"2023-11-08T19:31:39.374627+0000","last_fullsized":"2023-11-08T19:31:39.374627+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:39.374517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:39.374517+0000","last_clean_scrub_stamp":"2023-11-08T19:31:39.374517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:00:01.282223+0000","scrub_duration":0.0021558079999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":87,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:27.808067+0000","last_change":"2023-11-08T19:31:27.808093+0000","last_active":"2023-11-08T19:31:27.808067+0000","last_peered":"2023-11-08T19:31:27.808067+0000","last_clean":"2023-11-08T19:31:27.808067+0000","last_became_active":"2023-11-08T19:26:04.603955+0000","last_became_peered":"2023-11-08T19:26:04.603955+0000","last_unstale":"2023-11-08T19:31:27.808067+0000","last_undegraded":"2023-11-08T19:31:27.808067+0000","last_fullsized":"2023-11-08T19:31:27.808067+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:27.807939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:27.807939+0000","last_clean_scrub_stamp":"2023-11-08T19:31:27.807939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:20:39.173012+0000","scrub_duration":0.0016595620000000001,"objects_trimmed":0,"snaptrim_duration":0.00036473499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":64,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:39.814570+0000","last_change":"2023-11-08T19:31:39.814593+0000","last_active":"2023-11-08T19:31:39.814570+0000","last_peered":"2023-11-08T19:31:39.814570+0000","last_clean":"2023-11-08T19:31:39.814570+0000","last_became_active":"2023-11-08T19:27:07.194992+0000","last_became_peered":"2023-11-08T19:27:07.194992+0000","last_unstale":"2023-11-08T19:31:39.814570+0000","last_undegraded":"2023-11-08T19:31:39.814570+0000","last_fullsized":"2023-11-08T19:31:39.814570+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:39.814469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:39.814469+0000","last_clean_scrub_stamp":"2023-11-08T19:31:39.814469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:41:17.129654+0000","scrub_duration":0.001378043,"objects_trimmed":0,"snaptrim_duration":0.000770285,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":57,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:48.263043+0000","last_change":"2023-11-08T19:31:48.263067+0000","last_active":"2023-11-08T19:31:48.263043+0000","last_peered":"2023-11-08T19:31:48.263043+0000","last_clean":"2023-11-08T19:31:48.263043+0000","last_became_active":"2023-11-08T19:27:08.193653+0000","last_became_peered":"2023-11-08T19:27:08.193653+0000","last_unstale":"2023-11-08T19:31:48.263043+0000","last_undegraded":"2023-11-08T19:31:48.263043+0000","last_fullsized":"2023-11-08T19:31:48.263043+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:48.262912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:48.262912+0000","last_clean_scrub_stamp":"2023-11-08T19:31:48.262912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:43:09.144908+0000","scrub_duration":0.0015807149999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":92,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:33.590064+0000","last_change":"2023-11-08T19:31:33.590087+0000","last_active":"2023-11-08T19:31:33.590064+0000","last_peered":"2023-11-08T19:31:33.590064+0000","last_clean":"2023-11-08T19:31:33.590064+0000","last_became_active":"2023-11-08T19:26:04.603144+0000","last_became_peered":"2023-11-08T19:26:04.603144+0000","last_unstale":"2023-11-08T19:31:33.590064+0000","last_undegraded":"2023-11-08T19:31:33.590064+0000","last_fullsized":"2023-11-08T19:31:33.590064+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:33.589962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:33.589962+0000","last_clean_scrub_stamp":"2023-11-08T19:31:33.589962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:22:30.441483+0000","scrub_duration":0.0016553749999999999,"objects_trimmed":0,"snaptrim_duration":0.00054090799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":70,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:26.196071+0000","last_change":"2023-11-08T19:31:26.196111+0000","last_active":"2023-11-08T19:31:26.196071+0000","last_peered":"2023-11-08T19:31:26.196071+0000","last_clean":"2023-11-08T19:31:26.196071+0000","last_became_active":"2023-11-08T19:27:07.190438+0000","last_became_peered":"2023-11-08T19:27:07.190438+0000","last_unstale":"2023-11-08T19:31:26.196071+0000","last_undegraded":"2023-11-08T19:31:26.196071+0000","last_fullsized":"2023-11-08T19:31:26.196071+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:26.195915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:26.195915+0000","last_clean_scrub_stamp":"2023-11-08T19:31:26.195915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:09:39.929093+0000","scrub_duration":0.0021248539999999998,"objects_trimmed":0,"snaptrim_duration":0.00050893300000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":57,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:46.249163+0000","last_change":"2023-11-08T19:31:46.249191+0000","last_active":"2023-11-08T19:31:46.249163+0000","last_peered":"2023-11-08T19:31:46.249163+0000","last_clean":"2023-11-08T19:31:46.249163+0000","last_became_active":"2023-11-08T19:27:08.190203+0000","last_became_peered":"2023-11-08T19:27:08.190203+0000","last_unstale":"2023-11-08T19:31:46.249163+0000","last_undegraded":"2023-11-08T19:31:46.249163+0000","last_fullsized":"2023-11-08T19:31:46.249163+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:46.249056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:46.249056+0000","last_clean_scrub_stamp":"2023-11-08T19:31:46.249056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:30:06.625263+0000","scrub_duration":0.0018596820000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":88,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:26.732249+0000","last_change":"2023-11-08T19:31:26.732275+0000","last_active":"2023-11-08T19:31:26.732249+0000","last_peered":"2023-11-08T19:31:26.732249+0000","last_clean":"2023-11-08T19:31:26.732249+0000","last_became_active":"2023-11-08T19:26:04.601961+0000","last_became_peered":"2023-11-08T19:26:04.601961+0000","last_unstale":"2023-11-08T19:31:26.732249+0000","last_undegraded":"2023-11-08T19:31:26.732249+0000","last_fullsized":"2023-11-08T19:31:26.732249+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:26.732141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:26.732141+0000","last_clean_scrub_stamp":"2023-11-08T19:31:26.732141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:25:33.838895+0000","scrub_duration":0.0020533159999999999,"objects_trimmed":0,"snaptrim_duration":0.000466386,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":64,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:26.031920+0000","last_change":"2023-11-08T19:31:26.031963+0000","last_active":"2023-11-08T19:31:26.031920+0000","last_peered":"2023-11-08T19:31:26.031920+0000","last_clean":"2023-11-08T19:31:26.031920+0000","last_became_active":"2023-11-08T19:27:07.191583+0000","last_became_peered":"2023-11-08T19:27:07.191583+0000","last_unstale":"2023-11-08T19:31:26.031920+0000","last_undegraded":"2023-11-08T19:31:26.031920+0000","last_fullsized":"2023-11-08T19:31:26.031920+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:26.031797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:26.031797+0000","last_clean_scrub_stamp":"2023-11-08T19:31:26.031797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:09:39.764974+0000","scrub_duration":0.0015837099999999999,"objects_trimmed":0,"snaptrim_duration":0.00093695200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":57,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:29.050922+0000","last_change":"2023-11-08T19:31:29.050922+0000","last_active":"2023-11-08T19:31:29.050922+0000","last_peered":"2023-11-08T19:31:29.050922+0000","last_clean":"2023-11-08T19:31:29.050922+0000","last_became_active":"2023-11-08T19:27:08.187626+0000","last_became_peered":"2023-11-08T19:27:08.187626+0000","last_unstale":"2023-11-08T19:31:29.050922+0000","last_undegraded":"2023-11-08T19:31:29.050922+0000","last_fullsized":"2023-11-08T19:31:29.050922+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:29.050774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:29.050774+0000","last_clean_scrub_stamp":"2023-11-08T19:31:29.050774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:58:25.326897+0000","scrub_duration":0.001829125,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":98,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:51.959971+0000","last_change":"2023-11-08T19:31:51.959999+0000","last_active":"2023-11-08T19:31:51.959971+0000","last_peered":"2023-11-08T19:31:51.959971+0000","last_clean":"2023-11-08T19:31:51.959971+0000","last_became_active":"2023-11-08T19:26:04.606386+0000","last_became_peered":"2023-11-08T19:26:04.606386+0000","last_unstale":"2023-11-08T19:31:51.959971+0000","last_undegraded":"2023-11-08T19:31:51.959971+0000","last_fullsized":"2023-11-08T19:31:51.959971+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:51.959862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:51.959862+0000","last_clean_scrub_stamp":"2023-11-08T19:31:51.959862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:54:20.974301+0000","scrub_duration":0.001831734,"objects_trimmed":0,"snaptrim_duration":0.00037950599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":64,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:36.795896+0000","last_change":"2023-11-08T19:31:36.795921+0000","last_active":"2023-11-08T19:31:36.795896+0000","last_peered":"2023-11-08T19:31:36.795896+0000","last_clean":"2023-11-08T19:31:36.795896+0000","last_became_active":"2023-11-08T19:27:07.195033+0000","last_became_peered":"2023-11-08T19:27:07.195033+0000","last_unstale":"2023-11-08T19:31:36.795896+0000","last_undegraded":"2023-11-08T19:31:36.795896+0000","last_fullsized":"2023-11-08T19:31:36.795896+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:36.795786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:36.795786+0000","last_clean_scrub_stamp":"2023-11-08T19:31:36.795786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:34:05.177568+0000","scrub_duration":0.0014630890000000001,"objects_trimmed":0,"snaptrim_duration":0.00041824800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":57,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:46.454024+0000","last_change":"2023-11-08T19:31:46.454024+0000","last_active":"2023-11-08T19:31:46.454024+0000","last_peered":"2023-11-08T19:31:46.454024+0000","last_clean":"2023-11-08T19:31:46.454024+0000","last_became_active":"2023-11-08T19:27:08.193017+0000","last_became_peered":"2023-11-08T19:27:08.193017+0000","last_unstale":"2023-11-08T19:31:46.454024+0000","last_undegraded":"2023-11-08T19:31:46.454024+0000","last_fullsized":"2023-11-08T19:31:46.454024+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:46.453891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:46.453891+0000","last_clean_scrub_stamp":"2023-11-08T19:31:46.453891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:04:39.925346+0000","scrub_duration":0.0018276740000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":92,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:37.663409+0000","last_change":"2023-11-08T19:31:37.663435+0000","last_active":"2023-11-08T19:31:37.663409+0000","last_peered":"2023-11-08T19:31:37.663409+0000","last_clean":"2023-11-08T19:31:37.663409+0000","last_became_active":"2023-11-08T19:26:04.599046+0000","last_became_peered":"2023-11-08T19:26:04.599046+0000","last_unstale":"2023-11-08T19:31:37.663409+0000","last_undegraded":"2023-11-08T19:31:37.663409+0000","last_fullsized":"2023-11-08T19:31:37.663409+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:37.663280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:37.663280+0000","last_clean_scrub_stamp":"2023-11-08T19:31:37.663280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:20:56.015718+0000","scrub_duration":0.0015101190000000001,"objects_trimmed":0,"snaptrim_duration":0.00035092199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":70,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:01.936867+0000","last_change":"2023-11-08T19:32:01.936889+0000","last_active":"2023-11-08T19:32:01.936867+0000","last_peered":"2023-11-08T19:32:01.936867+0000","last_clean":"2023-11-08T19:32:01.936867+0000","last_became_active":"2023-11-08T19:27:07.187077+0000","last_became_peered":"2023-11-08T19:27:07.187077+0000","last_unstale":"2023-11-08T19:32:01.936867+0000","last_undegraded":"2023-11-08T19:32:01.936867+0000","last_fullsized":"2023-11-08T19:32:01.936867+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:01.936768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:01.936768+0000","last_clean_scrub_stamp":"2023-11-08T19:32:01.936768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:02:32.795159+0000","scrub_duration":0.0014581959999999999,"objects_trimmed":0,"snaptrim_duration":0.00056673599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":57,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:04.144448+0000","last_change":"2023-11-08T19:32:04.144448+0000","last_active":"2023-11-08T19:32:04.144448+0000","last_peered":"2023-11-08T19:32:04.144448+0000","last_clean":"2023-11-08T19:32:04.144448+0000","last_became_active":"2023-11-08T19:27:08.187714+0000","last_became_peered":"2023-11-08T19:27:08.187714+0000","last_unstale":"2023-11-08T19:32:04.144448+0000","last_undegraded":"2023-11-08T19:32:04.144448+0000","last_fullsized":"2023-11-08T19:32:04.144448+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:04.144315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:04.144315+0000","last_clean_scrub_stamp":"2023-11-08T19:32:04.144315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:43:07.351897+0000","scrub_duration":0.0016533069999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":86,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:35.500759+0000","last_change":"2023-11-08T19:31:35.500784+0000","last_active":"2023-11-08T19:31:35.500759+0000","last_peered":"2023-11-08T19:31:35.500759+0000","last_clean":"2023-11-08T19:31:35.500759+0000","last_became_active":"2023-11-08T19:26:04.603212+0000","last_became_peered":"2023-11-08T19:26:04.603212+0000","last_unstale":"2023-11-08T19:31:35.500759+0000","last_undegraded":"2023-11-08T19:31:35.500759+0000","last_fullsized":"2023-11-08T19:31:35.500759+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:35.500649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:35.500649+0000","last_clean_scrub_stamp":"2023-11-08T19:31:35.500649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:15:40.433937+0000","scrub_duration":0.0018424209999999999,"objects_trimmed":0,"snaptrim_duration":0.00054695399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":70,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:11.953371+0000","last_change":"2023-11-08T19:32:11.953411+0000","last_active":"2023-11-08T19:32:11.953371+0000","last_peered":"2023-11-08T19:32:11.953371+0000","last_clean":"2023-11-08T19:32:11.953371+0000","last_became_active":"2023-11-08T19:27:07.190897+0000","last_became_peered":"2023-11-08T19:27:07.190897+0000","last_unstale":"2023-11-08T19:32:11.953371+0000","last_undegraded":"2023-11-08T19:32:11.953371+0000","last_fullsized":"2023-11-08T19:32:11.953371+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:11.953246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:11.953246+0000","last_clean_scrub_stamp":"2023-11-08T19:32:11.953246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:28:04.941093+0000","scrub_duration":0.0021790490000000002,"objects_trimmed":0,"snaptrim_duration":0.0011172459999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":57,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:01.165256+0000","last_change":"2023-11-08T19:32:01.165256+0000","last_active":"2023-11-08T19:32:01.165256+0000","last_peered":"2023-11-08T19:32:01.165256+0000","last_clean":"2023-11-08T19:32:01.165256+0000","last_became_active":"2023-11-08T19:27:08.188441+0000","last_became_peered":"2023-11-08T19:27:08.188441+0000","last_unstale":"2023-11-08T19:32:01.165256+0000","last_undegraded":"2023-11-08T19:32:01.165256+0000","last_fullsized":"2023-11-08T19:32:01.165256+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:01.165078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:01.165078+0000","last_clean_scrub_stamp":"2023-11-08T19:32:01.165078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:31:37.776361+0000","scrub_duration":0.002016772,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":92,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:24.529044+0000","last_change":"2023-11-08T19:31:24.529066+0000","last_active":"2023-11-08T19:31:24.529044+0000","last_peered":"2023-11-08T19:31:24.529044+0000","last_clean":"2023-11-08T19:31:24.529044+0000","last_became_active":"2023-11-08T19:26:04.599322+0000","last_became_peered":"2023-11-08T19:26:04.599322+0000","last_unstale":"2023-11-08T19:31:24.529044+0000","last_undegraded":"2023-11-08T19:31:24.529044+0000","last_fullsized":"2023-11-08T19:31:24.529044+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:24.528935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:24.528935+0000","last_clean_scrub_stamp":"2023-11-08T19:31:24.528935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:09:38.262113+0000","scrub_duration":0.001486217,"objects_trimmed":0,"snaptrim_duration":0.00042670000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":70,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:23.244841+0000","last_change":"2023-11-08T19:31:23.244841+0000","last_active":"2023-11-08T19:31:23.244841+0000","last_peered":"2023-11-08T19:31:23.244841+0000","last_clean":"2023-11-08T19:31:23.244841+0000","last_became_active":"2023-11-08T19:27:07.187337+0000","last_became_peered":"2023-11-08T19:27:07.187337+0000","last_unstale":"2023-11-08T19:31:23.244841+0000","last_undegraded":"2023-11-08T19:31:23.244841+0000","last_fullsized":"2023-11-08T19:31:23.244841+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:23.244666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:23.244666+0000","last_clean_scrub_stamp":"2023-11-08T19:31:23.244666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:40:55.948223+0000","scrub_duration":0.001423278,"objects_trimmed":0,"snaptrim_duration":0.000346967,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":57,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:54.564968+0000","last_change":"2023-11-08T19:31:54.564968+0000","last_active":"2023-11-08T19:31:54.564968+0000","last_peered":"2023-11-08T19:31:54.564968+0000","last_clean":"2023-11-08T19:31:54.564968+0000","last_became_active":"2023-11-08T19:27:08.193532+0000","last_became_peered":"2023-11-08T19:27:08.193532+0000","last_unstale":"2023-11-08T19:31:54.564968+0000","last_undegraded":"2023-11-08T19:31:54.564968+0000","last_fullsized":"2023-11-08T19:31:54.564968+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:54.564832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:54.564832+0000","last_clean_scrub_stamp":"2023-11-08T19:31:54.564832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:16:11.657792+0000","scrub_duration":0.001445223,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":86,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:25.726576+0000","last_change":"2023-11-08T19:31:25.726612+0000","last_active":"2023-11-08T19:31:25.726576+0000","last_peered":"2023-11-08T19:31:25.726576+0000","last_clean":"2023-11-08T19:31:25.726576+0000","last_became_active":"2023-11-08T19:26:04.606805+0000","last_became_peered":"2023-11-08T19:26:04.606805+0000","last_unstale":"2023-11-08T19:31:25.726576+0000","last_undegraded":"2023-11-08T19:31:25.726576+0000","last_fullsized":"2023-11-08T19:31:25.726576+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:25.726413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:25.726413+0000","last_clean_scrub_stamp":"2023-11-08T19:31:25.726413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:51:20.104960+0000","scrub_duration":0.002105923,"objects_trimmed":0,"snaptrim_duration":0.00045317000000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":64,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:51.112535+0000","last_change":"2023-11-08T19:31:51.112535+0000","last_active":"2023-11-08T19:31:51.112535+0000","last_peered":"2023-11-08T19:31:51.112535+0000","last_clean":"2023-11-08T19:31:51.112535+0000","last_became_active":"2023-11-08T19:27:07.190309+0000","last_became_peered":"2023-11-08T19:27:07.190309+0000","last_unstale":"2023-11-08T19:31:51.112535+0000","last_undegraded":"2023-11-08T19:31:51.112535+0000","last_fullsized":"2023-11-08T19:31:51.112535+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:51.112386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:51.112386+0000","last_clean_scrub_stamp":"2023-11-08T19:31:51.112386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:30:11.488593+0000","scrub_duration":0.0015743689999999999,"objects_trimmed":0,"snaptrim_duration":0.00047615199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":57,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:49.431152+0000","last_change":"2023-11-08T19:31:49.431152+0000","last_active":"2023-11-08T19:31:49.431152+0000","last_peered":"2023-11-08T19:31:49.431152+0000","last_clean":"2023-11-08T19:31:49.431152+0000","last_became_active":"2023-11-08T19:27:08.193241+0000","last_became_peered":"2023-11-08T19:27:08.193241+0000","last_unstale":"2023-11-08T19:31:49.431152+0000","last_undegraded":"2023-11-08T19:31:49.431152+0000","last_fullsized":"2023-11-08T19:31:49.431152+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:49.431025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:49.431025+0000","last_clean_scrub_stamp":"2023-11-08T19:31:49.431025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:29:51.038404+0000","scrub_duration":0.0016466090000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":86,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:43.118702+0000","last_change":"2023-11-08T19:31:43.118741+0000","last_active":"2023-11-08T19:31:43.118702+0000","last_peered":"2023-11-08T19:31:43.118702+0000","last_clean":"2023-11-08T19:31:43.118702+0000","last_became_active":"2023-11-08T19:26:04.604772+0000","last_became_peered":"2023-11-08T19:26:04.604772+0000","last_unstale":"2023-11-08T19:31:43.118702+0000","last_undegraded":"2023-11-08T19:31:43.118702+0000","last_fullsized":"2023-11-08T19:31:43.118702+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:43.118568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:43.118568+0000","last_clean_scrub_stamp":"2023-11-08T19:31:43.118568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:10:29.603174+0000","scrub_duration":0.0017187769999999999,"objects_trimmed":0,"snaptrim_duration":0.00066875300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":64,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:39.612728+0000","last_change":"2023-11-08T19:31:39.612728+0000","last_active":"2023-11-08T19:31:39.612728+0000","last_peered":"2023-11-08T19:31:39.612728+0000","last_clean":"2023-11-08T19:31:39.612728+0000","last_became_active":"2023-11-08T19:27:07.191212+0000","last_became_peered":"2023-11-08T19:27:07.191212+0000","last_unstale":"2023-11-08T19:31:39.612728+0000","last_undegraded":"2023-11-08T19:31:39.612728+0000","last_fullsized":"2023-11-08T19:31:39.612728+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:39.612544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:39.612544+0000","last_clean_scrub_stamp":"2023-11-08T19:31:39.612544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:17:14.238197+0000","scrub_duration":0.0019683529999999999,"objects_trimmed":0,"snaptrim_duration":0.00046640800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":57,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:02.153713+0000","last_change":"2023-11-08T19:32:02.153740+0000","last_active":"2023-11-08T19:32:02.153713+0000","last_peered":"2023-11-08T19:32:02.153713+0000","last_clean":"2023-11-08T19:32:02.153713+0000","last_became_active":"2023-11-08T19:27:08.187376+0000","last_became_peered":"2023-11-08T19:27:08.187376+0000","last_unstale":"2023-11-08T19:32:02.153713+0000","last_undegraded":"2023-11-08T19:32:02.153713+0000","last_fullsized":"2023-11-08T19:32:02.153713+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:02.153601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:02.153601+0000","last_clean_scrub_stamp":"2023-11-08T19:32:02.153601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:41:11.468421+0000","scrub_duration":0.0018033540000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":87,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:26.809222+0000","last_change":"2023-11-08T19:31:26.809256+0000","last_active":"2023-11-08T19:31:26.809222+0000","last_peered":"2023-11-08T19:31:26.809222+0000","last_clean":"2023-11-08T19:31:26.809222+0000","last_became_active":"2023-11-08T19:26:04.598775+0000","last_became_peered":"2023-11-08T19:26:04.598775+0000","last_unstale":"2023-11-08T19:31:26.809222+0000","last_undegraded":"2023-11-08T19:31:26.809222+0000","last_fullsized":"2023-11-08T19:31:26.809222+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:26.809102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:26.809102+0000","last_clean_scrub_stamp":"2023-11-08T19:31:26.809102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:19:38.763559+0000","scrub_duration":0.002091,"objects_trimmed":0,"snaptrim_duration":0.00049870399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":64,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:39.485120+0000","last_change":"2023-11-08T19:31:39.485120+0000","last_active":"2023-11-08T19:31:39.485120+0000","last_peered":"2023-11-08T19:31:39.485120+0000","last_clean":"2023-11-08T19:31:39.485120+0000","last_became_active":"2023-11-08T19:27:07.190619+0000","last_became_peered":"2023-11-08T19:27:07.190619+0000","last_unstale":"2023-11-08T19:31:39.485120+0000","last_undegraded":"2023-11-08T19:31:39.485120+0000","last_fullsized":"2023-11-08T19:31:39.485120+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:39.484993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:39.484993+0000","last_clean_scrub_stamp":"2023-11-08T19:31:39.484993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:09:29.056050+0000","scrub_duration":0.001731263,"objects_trimmed":0,"snaptrim_duration":0.00065918400000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":63,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:38.329109+0000","last_change":"2023-11-08T19:31:38.329133+0000","last_active":"2023-11-08T19:31:38.329109+0000","last_peered":"2023-11-08T19:31:38.329109+0000","last_clean":"2023-11-08T19:31:38.329109+0000","last_became_active":"2023-11-08T19:27:08.183537+0000","last_became_peered":"2023-11-08T19:27:08.183537+0000","last_unstale":"2023-11-08T19:31:38.329109+0000","last_undegraded":"2023-11-08T19:31:38.329109+0000","last_fullsized":"2023-11-08T19:31:38.329109+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:38.328998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:38.328998+0000","last_clean_scrub_stamp":"2023-11-08T19:31:38.328998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:34:44.791135+0000","scrub_duration":0.0020231419999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":98,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:40.913956+0000","last_change":"2023-11-08T19:31:40.913984+0000","last_active":"2023-11-08T19:31:40.913956+0000","last_peered":"2023-11-08T19:31:40.913956+0000","last_clean":"2023-11-08T19:31:40.913956+0000","last_became_active":"2023-11-08T19:26:04.603209+0000","last_became_peered":"2023-11-08T19:26:04.603209+0000","last_unstale":"2023-11-08T19:31:40.913956+0000","last_undegraded":"2023-11-08T19:31:40.913956+0000","last_fullsized":"2023-11-08T19:31:40.913956+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:40.913792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:40.913792+0000","last_clean_scrub_stamp":"2023-11-08T19:31:40.913792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:20:59.266230+0000","scrub_duration":0.001580529,"objects_trimmed":0,"snaptrim_duration":0.00053335100000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":64,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:32.803377+0000","last_change":"2023-11-08T19:31:32.803377+0000","last_active":"2023-11-08T19:31:32.803377+0000","last_peered":"2023-11-08T19:31:32.803377+0000","last_clean":"2023-11-08T19:31:32.803377+0000","last_became_active":"2023-11-08T19:27:07.188169+0000","last_became_peered":"2023-11-08T19:27:07.188169+0000","last_unstale":"2023-11-08T19:31:32.803377+0000","last_undegraded":"2023-11-08T19:31:32.803377+0000","last_fullsized":"2023-11-08T19:31:32.803377+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:32.803247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:32.803247+0000","last_clean_scrub_stamp":"2023-11-08T19:31:32.803247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:30:32.589037+0000","scrub_duration":0.001747384,"objects_trimmed":0,"snaptrim_duration":0.00056209700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":57,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:44.503224+0000","last_change":"2023-11-08T19:31:44.503224+0000","last_active":"2023-11-08T19:31:44.503224+0000","last_peered":"2023-11-08T19:31:44.503224+0000","last_clean":"2023-11-08T19:31:44.503224+0000","last_became_active":"2023-11-08T19:27:08.193037+0000","last_became_peered":"2023-11-08T19:27:08.193037+0000","last_unstale":"2023-11-08T19:31:44.503224+0000","last_undegraded":"2023-11-08T19:31:44.503224+0000","last_fullsized":"2023-11-08T19:31:44.503224+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:44.503078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:44.503078+0000","last_clean_scrub_stamp":"2023-11-08T19:31:44.503078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:44:27.056698+0000","scrub_duration":0.0016849320000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":86,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:30.100037+0000","last_change":"2023-11-08T19:31:30.100059+0000","last_active":"2023-11-08T19:31:30.100037+0000","last_peered":"2023-11-08T19:31:30.100037+0000","last_clean":"2023-11-08T19:31:30.100037+0000","last_became_active":"2023-11-08T19:26:04.603409+0000","last_became_peered":"2023-11-08T19:26:04.603409+0000","last_unstale":"2023-11-08T19:31:30.100037+0000","last_undegraded":"2023-11-08T19:31:30.100037+0000","last_fullsized":"2023-11-08T19:31:30.100037+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:30.099915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:30.099915+0000","last_clean_scrub_stamp":"2023-11-08T19:31:30.099915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:03:25.984959+0000","scrub_duration":0.001620628,"objects_trimmed":0,"snaptrim_duration":0.00065684300000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":70,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:03.973223+0000","last_change":"2023-11-08T19:32:03.973245+0000","last_active":"2023-11-08T19:32:03.973223+0000","last_peered":"2023-11-08T19:32:03.973223+0000","last_clean":"2023-11-08T19:32:03.973223+0000","last_became_active":"2023-11-08T19:27:07.186944+0000","last_became_peered":"2023-11-08T19:27:07.186944+0000","last_unstale":"2023-11-08T19:32:03.973223+0000","last_undegraded":"2023-11-08T19:32:03.973223+0000","last_fullsized":"2023-11-08T19:32:03.973223+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:03.973101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:03.973101+0000","last_clean_scrub_stamp":"2023-11-08T19:32:03.973101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:47:47.958394+0000","scrub_duration":0.001429186,"objects_trimmed":0,"snaptrim_duration":0.00028665599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":57,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:03.742695+0000","last_change":"2023-11-08T19:32:03.742724+0000","last_active":"2023-11-08T19:32:03.742695+0000","last_peered":"2023-11-08T19:32:03.742695+0000","last_clean":"2023-11-08T19:32:03.742695+0000","last_became_active":"2023-11-08T19:27:08.193624+0000","last_became_peered":"2023-11-08T19:27:08.193624+0000","last_unstale":"2023-11-08T19:32:03.742695+0000","last_undegraded":"2023-11-08T19:32:03.742695+0000","last_fullsized":"2023-11-08T19:32:03.742695+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:03.742562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:03.742562+0000","last_clean_scrub_stamp":"2023-11-08T19:32:03.742562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:43:24.624558+0000","scrub_duration":0.0016943609999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":92,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:30.601612+0000","last_change":"2023-11-08T19:31:30.601633+0000","last_active":"2023-11-08T19:31:30.601612+0000","last_peered":"2023-11-08T19:31:30.601612+0000","last_clean":"2023-11-08T19:31:30.601612+0000","last_became_active":"2023-11-08T19:26:04.608639+0000","last_became_peered":"2023-11-08T19:26:04.608639+0000","last_unstale":"2023-11-08T19:31:30.601612+0000","last_undegraded":"2023-11-08T19:31:30.601612+0000","last_fullsized":"2023-11-08T19:31:30.601612+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:30.601479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:30.601479+0000","last_clean_scrub_stamp":"2023-11-08T19:31:30.601479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:16:43.728400+0000","scrub_duration":0.0016375770000000001,"objects_trimmed":0,"snaptrim_duration":0.00044384999999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":64,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:50.637123+0000","last_change":"2023-11-08T19:31:50.637123+0000","last_active":"2023-11-08T19:31:50.637123+0000","last_peered":"2023-11-08T19:31:50.637123+0000","last_clean":"2023-11-08T19:31:50.637123+0000","last_became_active":"2023-11-08T19:27:07.190931+0000","last_became_peered":"2023-11-08T19:27:07.190931+0000","last_unstale":"2023-11-08T19:31:50.637123+0000","last_undegraded":"2023-11-08T19:31:50.637123+0000","last_fullsized":"2023-11-08T19:31:50.637123+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:50.636977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:50.636977+0000","last_clean_scrub_stamp":"2023-11-08T19:31:50.636977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:59:11.822136+0000","scrub_duration":0.001523333,"objects_trimmed":0,"snaptrim_duration":0.00034146799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":61,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:53.801652+0000","last_change":"2023-11-08T19:31:53.801674+0000","last_active":"2023-11-08T19:31:53.801652+0000","last_peered":"2023-11-08T19:31:53.801652+0000","last_clean":"2023-11-08T19:31:53.801652+0000","last_became_active":"2023-11-08T19:27:08.186952+0000","last_became_peered":"2023-11-08T19:27:08.186952+0000","last_unstale":"2023-11-08T19:31:53.801652+0000","last_undegraded":"2023-11-08T19:31:53.801652+0000","last_fullsized":"2023-11-08T19:31:53.801652+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:53.801547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:53.801547+0000","last_clean_scrub_stamp":"2023-11-08T19:31:53.801547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:56:01.800602+0000","scrub_duration":0.001479352,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":86,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:32.594224+0000","last_change":"2023-11-08T19:31:32.594249+0000","last_active":"2023-11-08T19:31:32.594224+0000","last_peered":"2023-11-08T19:31:32.594224+0000","last_clean":"2023-11-08T19:31:32.594224+0000","last_became_active":"2023-11-08T19:26:04.602137+0000","last_became_peered":"2023-11-08T19:26:04.602137+0000","last_unstale":"2023-11-08T19:31:32.594224+0000","last_undegraded":"2023-11-08T19:31:32.594224+0000","last_fullsized":"2023-11-08T19:31:32.594224+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:32.594119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:32.594119+0000","last_clean_scrub_stamp":"2023-11-08T19:31:32.594119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:04:34.319826+0000","scrub_duration":0.00148785,"objects_trimmed":0,"snaptrim_duration":0.00056998200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":64,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:35.819632+0000","last_change":"2023-11-08T19:31:35.819632+0000","last_active":"2023-11-08T19:31:35.819632+0000","last_peered":"2023-11-08T19:31:35.819632+0000","last_clean":"2023-11-08T19:31:35.819632+0000","last_became_active":"2023-11-08T19:27:07.192001+0000","last_became_peered":"2023-11-08T19:27:07.192001+0000","last_unstale":"2023-11-08T19:31:35.819632+0000","last_undegraded":"2023-11-08T19:31:35.819632+0000","last_fullsized":"2023-11-08T19:31:35.819632+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:35.819499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:35.819499+0000","last_clean_scrub_stamp":"2023-11-08T19:31:35.819499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:56:02.315632+0000","scrub_duration":0.0014740599999999999,"objects_trimmed":0,"snaptrim_duration":0.00045359000000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":57,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:30.097308+0000","last_change":"2023-11-08T19:31:30.097308+0000","last_active":"2023-11-08T19:31:30.097308+0000","last_peered":"2023-11-08T19:31:30.097308+0000","last_clean":"2023-11-08T19:31:30.097308+0000","last_became_active":"2023-11-08T19:27:08.193458+0000","last_became_peered":"2023-11-08T19:27:08.193458+0000","last_unstale":"2023-11-08T19:31:30.097308+0000","last_undegraded":"2023-11-08T19:31:30.097308+0000","last_fullsized":"2023-11-08T19:31:30.097308+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:30.097159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:30.097159+0000","last_clean_scrub_stamp":"2023-11-08T19:31:30.097159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:12:53.982878+0000","scrub_duration":0.001628696,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":88,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:23.750030+0000","last_change":"2023-11-08T19:31:23.750053+0000","last_active":"2023-11-08T19:31:23.750030+0000","last_peered":"2023-11-08T19:31:23.750030+0000","last_clean":"2023-11-08T19:31:23.750030+0000","last_became_active":"2023-11-08T19:26:04.602183+0000","last_became_peered":"2023-11-08T19:26:04.602183+0000","last_unstale":"2023-11-08T19:31:23.750030+0000","last_undegraded":"2023-11-08T19:31:23.750030+0000","last_fullsized":"2023-11-08T19:31:23.750030+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:23.749930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:23.749930+0000","last_clean_scrub_stamp":"2023-11-08T19:31:23.749930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:48:22.893896+0000","scrub_duration":0.001725201,"objects_trimmed":0,"snaptrim_duration":0.00052029099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":70,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:29.200138+0000","last_change":"2023-11-08T19:31:29.200138+0000","last_active":"2023-11-08T19:31:29.200138+0000","last_peered":"2023-11-08T19:31:29.200138+0000","last_clean":"2023-11-08T19:31:29.200138+0000","last_became_active":"2023-11-08T19:27:07.184507+0000","last_became_peered":"2023-11-08T19:27:07.184507+0000","last_unstale":"2023-11-08T19:31:29.200138+0000","last_undegraded":"2023-11-08T19:31:29.200138+0000","last_fullsized":"2023-11-08T19:31:29.200138+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:29.199968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:29.199968+0000","last_clean_scrub_stamp":"2023-11-08T19:31:29.199968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:17:03.825621+0000","scrub_duration":0.001394225,"objects_trimmed":0,"snaptrim_duration":0.00051924700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":57,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:01.755886+0000","last_change":"2023-11-08T19:32:01.755923+0000","last_active":"2023-11-08T19:32:01.755886+0000","last_peered":"2023-11-08T19:32:01.755886+0000","last_clean":"2023-11-08T19:32:01.755886+0000","last_became_active":"2023-11-08T19:27:08.193938+0000","last_became_peered":"2023-11-08T19:27:08.193938+0000","last_unstale":"2023-11-08T19:32:01.755886+0000","last_undegraded":"2023-11-08T19:32:01.755886+0000","last_fullsized":"2023-11-08T19:32:01.755886+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:01.755735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:01.755735+0000","last_clean_scrub_stamp":"2023-11-08T19:32:01.755735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:30:22.131943+0000","scrub_duration":0.0020522639999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":87,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:35.112252+0000","last_change":"2023-11-08T19:31:35.112252+0000","last_active":"2023-11-08T19:31:35.112252+0000","last_peered":"2023-11-08T19:31:35.112252+0000","last_clean":"2023-11-08T19:31:35.112252+0000","last_became_active":"2023-11-08T19:26:04.603397+0000","last_became_peered":"2023-11-08T19:26:04.603397+0000","last_unstale":"2023-11-08T19:31:35.112252+0000","last_undegraded":"2023-11-08T19:31:35.112252+0000","last_fullsized":"2023-11-08T19:31:35.112252+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:35.112102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:35.112102+0000","last_clean_scrub_stamp":"2023-11-08T19:31:35.112102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:17:09.737756+0000","scrub_duration":0.0016909259999999999,"objects_trimmed":0,"snaptrim_duration":0.00053361499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":64,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:34.769739+0000","last_change":"2023-11-08T19:31:34.769739+0000","last_active":"2023-11-08T19:31:34.769739+0000","last_peered":"2023-11-08T19:31:34.769739+0000","last_clean":"2023-11-08T19:31:34.769739+0000","last_became_active":"2023-11-08T19:27:07.189943+0000","last_became_peered":"2023-11-08T19:27:07.189943+0000","last_unstale":"2023-11-08T19:31:34.769739+0000","last_undegraded":"2023-11-08T19:31:34.769739+0000","last_fullsized":"2023-11-08T19:31:34.769739+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:34.769604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:34.769604+0000","last_clean_scrub_stamp":"2023-11-08T19:31:34.769604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:51:01.063515+0000","scrub_duration":0.001529461,"objects_trimmed":0,"snaptrim_duration":0.000398725,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":63,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:35.319840+0000","last_change":"2023-11-08T19:31:35.319840+0000","last_active":"2023-11-08T19:31:35.319840+0000","last_peered":"2023-11-08T19:31:35.319840+0000","last_clean":"2023-11-08T19:31:35.319840+0000","last_became_active":"2023-11-08T19:27:08.187487+0000","last_became_peered":"2023-11-08T19:27:08.187487+0000","last_unstale":"2023-11-08T19:31:35.319840+0000","last_undegraded":"2023-11-08T19:31:35.319840+0000","last_fullsized":"2023-11-08T19:31:35.319840+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:35.319727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:35.319727+0000","last_clean_scrub_stamp":"2023-11-08T19:31:35.319727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:41:21.686280+0000","scrub_duration":0.001956157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":93,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:39.721512+0000","last_change":"2023-11-08T19:31:39.721512+0000","last_active":"2023-11-08T19:31:39.721512+0000","last_peered":"2023-11-08T19:31:39.721512+0000","last_clean":"2023-11-08T19:31:39.721512+0000","last_became_active":"2023-11-08T19:26:04.601103+0000","last_became_peered":"2023-11-08T19:26:04.601103+0000","last_unstale":"2023-11-08T19:31:39.721512+0000","last_undegraded":"2023-11-08T19:31:39.721512+0000","last_fullsized":"2023-11-08T19:31:39.721512+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:39.721350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:39.721350+0000","last_clean_scrub_stamp":"2023-11-08T19:31:39.721350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:30:00.097557+0000","scrub_duration":0.001887569,"objects_trimmed":0,"snaptrim_duration":0.000229244,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":70,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:59.904958+0000","last_change":"2023-11-08T19:31:59.904991+0000","last_active":"2023-11-08T19:31:59.904958+0000","last_peered":"2023-11-08T19:31:59.904958+0000","last_clean":"2023-11-08T19:31:59.904958+0000","last_became_active":"2023-11-08T19:27:07.191926+0000","last_became_peered":"2023-11-08T19:27:07.191926+0000","last_unstale":"2023-11-08T19:31:59.904958+0000","last_undegraded":"2023-11-08T19:31:59.904958+0000","last_fullsized":"2023-11-08T19:31:59.904958+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:59.904842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:59.904842+0000","last_clean_scrub_stamp":"2023-11-08T19:31:59.904842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:41:09.219663+0000","scrub_duration":0.00190055,"objects_trimmed":0,"snaptrim_duration":0.00055457100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":63,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:40.373522+0000","last_change":"2023-11-08T19:31:40.373522+0000","last_active":"2023-11-08T19:31:40.373522+0000","last_peered":"2023-11-08T19:31:40.373522+0000","last_clean":"2023-11-08T19:31:40.373522+0000","last_became_active":"2023-11-08T19:27:08.185581+0000","last_became_peered":"2023-11-08T19:27:08.185581+0000","last_unstale":"2023-11-08T19:31:40.373522+0000","last_undegraded":"2023-11-08T19:31:40.373522+0000","last_fullsized":"2023-11-08T19:31:40.373522+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:40.373357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:40.373357+0000","last_clean_scrub_stamp":"2023-11-08T19:31:40.373357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:48:00.953173+0000","scrub_duration":0.0015121480000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":98,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:46.864396+0000","last_change":"2023-11-08T19:31:46.864422+0000","last_active":"2023-11-08T19:31:46.864396+0000","last_peered":"2023-11-08T19:31:46.864396+0000","last_clean":"2023-11-08T19:31:46.864396+0000","last_became_active":"2023-11-08T19:26:04.606312+0000","last_became_peered":"2023-11-08T19:26:04.606312+0000","last_unstale":"2023-11-08T19:31:46.864396+0000","last_undegraded":"2023-11-08T19:31:46.864396+0000","last_fullsized":"2023-11-08T19:31:46.864396+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:46.864278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:46.864278+0000","last_clean_scrub_stamp":"2023-11-08T19:31:46.864278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:43:07.746274+0000","scrub_duration":0.0014882389999999999,"objects_trimmed":0,"snaptrim_duration":0.000505935,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":64,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:30.876396+0000","last_change":"2023-11-08T19:31:30.876396+0000","last_active":"2023-11-08T19:31:30.876396+0000","last_peered":"2023-11-08T19:31:30.876396+0000","last_clean":"2023-11-08T19:31:30.876396+0000","last_became_active":"2023-11-08T19:27:07.193254+0000","last_became_peered":"2023-11-08T19:27:07.193254+0000","last_unstale":"2023-11-08T19:31:30.876396+0000","last_undegraded":"2023-11-08T19:31:30.876396+0000","last_fullsized":"2023-11-08T19:31:30.876396+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:30.876249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:30.876249+0000","last_clean_scrub_stamp":"2023-11-08T19:31:30.876249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:32:42.716359+0000","scrub_duration":0.0014983080000000001,"objects_trimmed":0,"snaptrim_duration":0.00041830799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":63,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:36.343755+0000","last_change":"2023-11-08T19:31:36.343755+0000","last_active":"2023-11-08T19:31:36.343755+0000","last_peered":"2023-11-08T19:31:36.343755+0000","last_clean":"2023-11-08T19:31:36.343755+0000","last_became_active":"2023-11-08T19:27:08.188560+0000","last_became_peered":"2023-11-08T19:27:08.188560+0000","last_unstale":"2023-11-08T19:31:36.343755+0000","last_undegraded":"2023-11-08T19:31:36.343755+0000","last_fullsized":"2023-11-08T19:31:36.343755+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:36.343622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:36.343622+0000","last_clean_scrub_stamp":"2023-11-08T19:31:36.343622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:29:56.719829+0000","scrub_duration":0.002226525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":87,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:23.098586+0000","last_change":"2023-11-08T19:31:23.098607+0000","last_active":"2023-11-08T19:31:23.098586+0000","last_peered":"2023-11-08T19:31:23.098586+0000","last_clean":"2023-11-08T19:31:23.098586+0000","last_became_active":"2023-11-08T19:26:04.603979+0000","last_became_peered":"2023-11-08T19:26:04.603979+0000","last_unstale":"2023-11-08T19:31:23.098586+0000","last_undegraded":"2023-11-08T19:31:23.098586+0000","last_fullsized":"2023-11-08T19:31:23.098586+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:23.098455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:23.098455+0000","last_clean_scrub_stamp":"2023-11-08T19:31:23.098455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:03:18.983499+0000","scrub_duration":0.0017647400000000001,"objects_trimmed":0,"snaptrim_duration":0.00079032099999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":64,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:31:37.483328+0000","last_change":"2023-11-08T19:31:37.483328+0000","last_active":"2023-11-08T19:31:37.483328+0000","last_peered":"2023-11-08T19:31:37.483328+0000","last_clean":"2023-11-08T19:31:37.483328+0000","last_became_active":"2023-11-08T19:27:07.185478+0000","last_became_peered":"2023-11-08T19:27:07.185478+0000","last_unstale":"2023-11-08T19:31:37.483328+0000","last_undegraded":"2023-11-08T19:31:37.483328+0000","last_fullsized":"2023-11-08T19:31:37.483328+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:31:37.483195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:31:37.483195+0000","last_clean_scrub_stamp":"2023-11-08T19:31:37.483195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:53:43.068960+0000","scrub_duration":0.001795891,"objects_trimmed":0,"snaptrim_duration":0.0010661850000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574943,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33048,"kb_used_data":1872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338792,"statfs":{"total":96636764160,"available":96602923008,"internally_reserved":0,"allocated":1916928,"data_stored":1236765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.54200000000000004,"15min":0.51300000000000001},"min":{"1min":0.314,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.70399999999999996,"5min":0.76600000000000001,"15min":1.0149999999999999},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.53800000000000003,"15min":0.52300000000000002},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.32300000000000001},"max":{"1min":0.70099999999999996,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.63400000000000001}]},{"osd":1,"last update":"Wed Nov 8 19:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60099999999999998,"15min":0.57799999999999996},"min":{"1min":0.40100000000000002,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.78700000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.59499999999999997,"15min":0.56899999999999995},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.33900000000000002},"max":{"1min":0.76000000000000001,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.66100000000000003}]},{"osd":2,"last update":"Wed Nov 8 19:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.54200000000000004,"15min":0.51000000000000001},"min":{"1min":0.39200000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.66700000000000004,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56999999999999995,"15min":0.55700000000000005},"min":{"1min":0.35999999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.80400000000000005,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.61699999999999999}]},{"osd":3,"last update":"Wed Nov 8 19:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.57999999999999996,"15min":0.56599999999999995},"min":{"1min":0.46500000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.72199999999999998,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.625,"5min":0.63100000000000001,"15min":0.59699999999999998},"min":{"1min":0.505,"5min":0.437,"15min":0.40100000000000002},"max":{"1min":0.752,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.70099999999999996}]},{"osd":4,"last update":"Wed Nov 8 19:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.48499999999999999,"15min":0.502},"min":{"1min":0.33600000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.65700000000000003,"5min":0.753,"15min":0.78600000000000003},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.52700000000000002,"15min":0.53700000000000003},"min":{"1min":0.38300000000000001,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.73499999999999999,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.48799999999999999}]},{"osd":5,"last update":"Wed Nov 8 19:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.55200000000000005,"15min":0.57199999999999995},"min":{"1min":0.36199999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.71799999999999997,"5min":0.78500000000000003,"15min":0.82499999999999996},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.61499999999999999,"15min":0.63400000000000001},"min":{"1min":0.432,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.75600000000000001,"5min":0.89900000000000002,"15min":0.94299999999999995},"last":0.56000000000000005}]},{"osd":6,"last update":"Wed Nov 8 19:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.59499999999999997},"min":{"1min":0.42999999999999999,"5min":0.377,"15min":0.377},"max":{"1min":0.76200000000000001,"5min":0.86199999999999999,"15min":0.97299999999999998},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58399999999999996,"15min":0.60299999999999998},"min":{"1min":0.375,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.77300000000000002,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.57799999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574943,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32468,"kb_used_data":1292,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339372,"statfs":{"total":96636764160,"available":96603516928,"internally_reserved":0,"allocated":1323008,"data_stored":646397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.57099999999999995,"15min":1.0900000000000001},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.27000000000000002},"max":{"1min":0.78200000000000003,"5min":0.84599999999999997,"15min":21.359999999999999},"last":0.499},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57499999999999996,"15min":0.79200000000000004},"min":{"1min":0.42999999999999999,"5min":0.378,"15min":0.30399999999999999},"max":{"1min":0.83599999999999997,"5min":0.85699999999999998,"15min":10.329000000000001},"last":0.59899999999999998}]},{"osd":1,"last update":"Wed Nov 8 19:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.55000000000000004,"15min":0.59199999999999997},"min":{"1min":0.41499999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.72399999999999998,"5min":0.81399999999999995,"15min":3.4239999999999999},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.61199999999999999,"15min":0.63},"min":{"1min":0.41199999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.77900000000000003,"5min":2.9260000000000002,"15min":3.4809999999999999},"last":0.625}]},{"osd":2,"last update":"Wed Nov 8 19:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.59399999999999997,"15min":0.81000000000000005},"min":{"1min":0.49399999999999999,"5min":0.40000000000000002,"15min":0.34399999999999997},"max":{"1min":0.79400000000000004,"5min":0.84199999999999997,"15min":10.398},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.57499999999999996,"15min":0.65400000000000003},"min":{"1min":0.46300000000000002,"5min":0.377,"15min":0.318},"max":{"1min":0.67700000000000005,"5min":0.83599999999999997,"15min":5.0170000000000003},"last":0.57899999999999996}]},{"osd":3,"last update":"Wed Nov 8 19:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.63700000000000001,"15min":0.65600000000000003},"min":{"1min":0.45600000000000002,"5min":0.35999999999999999,"15min":0.317},"max":{"1min":0.81599999999999995,"5min":3.0089999999999999,"15min":3.5710000000000002},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.60899999999999999,"15min":0.83699999999999997},"min":{"1min":0.435,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.86099999999999999,"5min":0.86799999999999999,"15min":10.368},"last":0.68100000000000005}]},{"osd":4,"last update":"Wed Nov 8 19:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52900000000000003,"15min":0.52000000000000002},"min":{"1min":0.371,"5min":0.33000000000000002,"15min":0.29199999999999998},"max":{"1min":0.72799999999999998,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.434},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.55300000000000005,"15min":0.55900000000000005},"min":{"1min":0.38700000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.76300000000000001,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.54300000000000004}]},{"osd":5,"last update":"Wed Nov 8 19:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53700000000000003,"15min":0.53300000000000003},"min":{"1min":0.36699999999999999,"5min":0.34599999999999997,"15min":0.307},"max":{"1min":0.80100000000000005,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.59099999999999997,"15min":0.58599999999999997},"min":{"1min":0.41099999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.84899999999999998,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.65500000000000003}]},{"osd":7,"last update":"Wed Nov 8 19:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.59799999999999998,"15min":0.58399999999999996},"min":{"1min":0.373,"5min":0.373,"15min":0.34699999999999998},"max":{"1min":0.81899999999999995,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.59799999999999998,"15min":0.59399999999999997},"min":{"1min":0.41399999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.83699999999999997,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.61399999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574943,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32468,"kb_used_data":1292,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339372,"statfs":{"total":96636764160,"available":96603516928,"internally_reserved":0,"allocated":1323008,"data_stored":646397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:31:55 2023","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.35399999999999998,"15min":0.372},"min":{"1min":0.27400000000000002,"5min":0.222,"15min":0.219},"max":{"1min":0.45200000000000001,"5min":0.69699999999999995,"15min":1.389},"last":0.314},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.38,"15min":0.59599999999999997},"min":{"1min":0.20599999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":1.5109999999999999,"5min":1.5109999999999999,"15min":9.1370000000000005},"last":0.436}]},{"osd":2,"last update":"Wed Nov 8 19:31:55 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.36799999999999999,"15min":0.36499999999999999},"min":{"1min":0.27900000000000003,"5min":0.216,"15min":0.216},"max":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":1.2709999999999999},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.39200000000000002,"15min":0.41599999999999998},"min":{"1min":0.24199999999999999,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.47899999999999998,"5min":0.63600000000000001,"15min":1.425},"last":0.379}]},{"osd":3,"last update":"Wed Nov 8 19:31:55 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.41899999999999998,"15min":0.71199999999999997},"min":{"1min":0.29699999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.59799999999999998,"5min":0.63800000000000001,"15min":11.957000000000001},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.40000000000000002,"15min":0.86699999999999999},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":1.5600000000000001,"5min":1.5600000000000001,"15min":18.039000000000001},"last":0.47999999999999998}]},{"osd":4,"last update":"Wed Nov 8 19:31:55 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.59199999999999997,"15min":0.73499999999999999},"min":{"1min":0.40100000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.84899999999999998,"5min":0.94999999999999996,"15min":6.181},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59799999999999998,"15min":0.627},"min":{"1min":0.40100000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.88,"5min":0.996,"15min":1.488},"last":0.65100000000000002}]},{"osd":5,"last update":"Wed Nov 8 19:31:55 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.60399999999999998,"15min":0.83699999999999997},"min":{"1min":0.41499999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.601,"5min":1.601,"15min":9.2420000000000009},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59699999999999998,"15min":0.77000000000000002},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.82599999999999996,"5min":0.878,"15min":6.2110000000000003},"last":0.66600000000000004}]},{"osd":6,"last update":"Wed Nov 8 19:31:55 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.623,"15min":0.84599999999999997},"min":{"1min":0.442,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.665,"5min":1.665,"15min":9.1880000000000006},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.625,"15min":0.65300000000000002},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.90500000000000003,"5min":0.98399999999999999,"15min":1.462},"last":0.58399999999999996}]},{"osd":7,"last update":"Wed Nov 8 19:31:55 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.61299999999999999,"15min":0.77000000000000002},"min":{"1min":0.40899999999999997,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.79200000000000004,"5min":1.0309999999999999,"15min":6.1479999999999997},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.64400000000000002,"15min":0.876},"min":{"1min":0.47499999999999998,"5min":0.40200000000000002,"15min":0.36099999999999999},"max":{"1min":1.7909999999999999,"5min":1.7909999999999999,"15min":9.2170000000000005},"last":0.55000000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574943,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33048,"kb_used_data":1872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338792,"statfs":{"total":96636764160,"available":96602923008,"internally_reserved":0,"allocated":1916928,"data_stored":1236765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Nov 8 19:32:04 2023","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.34000000000000002,"15min":0.44800000000000001},"min":{"1min":0.24099999999999999,"5min":0.20899999999999999,"15min":0.19800000000000001},"max":{"1min":0.54300000000000004,"5min":0.55300000000000005,"15min":4.1200000000000001},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.371,"15min":0.86299999999999999},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.45800000000000002,"5min":0.62,"15min":16.940999999999999},"last":0.30599999999999999}]},{"osd":2,"last update":"Wed Nov 8 19:32:04 2023","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.35699999999999998,"15min":0.38400000000000001},"min":{"1min":0.223,"5min":0.22,"15min":0.22},"max":{"1min":0.59999999999999998,"5min":0.60699999999999998,"15min":1.1899999999999999},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.39500000000000002,"15min":0.58399999999999996},"min":{"1min":0.28199999999999997,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.56599999999999995,"5min":0.78900000000000003,"15min":7.0209999999999999},"last":0.44400000000000001}]},{"osd":3,"last update":"Wed Nov 8 19:32:04 2023","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.38400000000000001,"15min":0.499},"min":{"1min":0.26500000000000001,"5min":0.25,"15min":0.23200000000000001},"max":{"1min":0.54400000000000004,"5min":0.60299999999999998,"15min":4.0940000000000003},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.40000000000000002,"15min":0.42999999999999999},"min":{"1min":0.28399999999999997,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":1.2549999999999999},"last":0.42699999999999999}]},{"osd":4,"last update":"Wed Nov 8 19:32:04 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.59099999999999997,"15min":0.70199999999999996},"min":{"1min":0.438,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.752,"5min":0.92200000000000004,"15min":4.0049999999999999},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.58499999999999996,"15min":0.62},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":1.2270000000000001},"last":0.54800000000000004}]},{"osd":5,"last update":"Wed Nov 8 19:32:04 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.59299999999999997,"15min":0.71299999999999997},"min":{"1min":0.41599999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.71899999999999997,"5min":1.0149999999999999,"15min":4.3090000000000002},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.62,"15min":0.71499999999999997},"min":{"1min":0.435,"5min":0.373,"15min":0.373},"max":{"1min":0.86899999999999999,"5min":0.96899999999999997,"15min":4.0709999999999997},"last":0.59399999999999997}]},{"osd":6,"last update":"Wed Nov 8 19:32:04 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.625,"15min":0.65500000000000003},"min":{"1min":0.42499999999999999,"5min":0.373,"15min":0.373},"max":{"1min":0.76700000000000002,"5min":0.93999999999999995,"15min":1.2849999999999999},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.65000000000000002,"15min":0.77000000000000002},"min":{"1min":0.45900000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.91800000000000004,"5min":0.95199999999999996,"15min":4.2560000000000002},"last":0.57099999999999995}]},{"osd":7,"last update":"Wed Nov 8 19:32:04 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.63100000000000001,"15min":0.73299999999999998},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.83399999999999996,"5min":0.90400000000000003,"15min":4.2850000000000001},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.65400000000000003,"15min":0.75900000000000001},"min":{"1min":0.47299999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.90500000000000003,"5min":1.002,"15min":4.0460000000000003},"last":0.61899999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574943,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28308,"kb_used_data":1292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343532,"statfs":{"total":96636764160,"available":96607776768,"internally_reserved":0,"allocated":1323008,"data_stored":646397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.35499999999999998,"15min":0.34000000000000002},"min":{"1min":0.251,"5min":0.182,"15min":0.182},"max":{"1min":0.48499999999999999,"5min":0.629,"15min":0.629},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.375,"15min":0.54100000000000004},"min":{"1min":0.28100000000000003,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.52000000000000002,"5min":0.68000000000000005,"15min":6.0999999999999996},"last":0.44800000000000001}]},{"osd":1,"last update":"Wed Nov 8 19:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.36199999999999999,"15min":0.34799999999999998},"min":{"1min":0.24399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.59499999999999997,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.39400000000000002,"15min":0.39000000000000001},"min":{"1min":0.28000000000000003,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.56599999999999995,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.50900000000000001}]},{"osd":3,"last update":"Wed Nov 8 19:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.41599999999999998,"15min":0.39400000000000002},"min":{"1min":0.30399999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.56899999999999995,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.40500000000000003,"15min":0.39700000000000002},"min":{"1min":0.28100000000000003,"5min":0.217,"15min":0.217},"max":{"1min":0.54400000000000004,"5min":0.65900000000000003,"15min":0.70299999999999996},"last":0.432}]},{"osd":4,"last update":"Wed Nov 8 19:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.56499999999999995,"15min":0.57599999999999996},"min":{"1min":0.432,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.81499999999999995,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.497},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.58899999999999997,"15min":0.58499999999999996},"min":{"1min":0.43099999999999999,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.74199999999999999,"5min":0.872,"15min":0.90400000000000003},"last":0.61499999999999999}]},{"osd":5,"last update":"Wed Nov 8 19:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.627,"15min":0.59799999999999998},"min":{"1min":0.49399999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.85999999999999999,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.61499999999999999,"15min":0.61299999999999999},"min":{"1min":0.439,"5min":0.38,"15min":0.38},"max":{"1min":0.92100000000000004,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.58799999999999997}]},{"osd":6,"last update":"Wed Nov 8 19:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.59799999999999998,"15min":0.59899999999999998},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.621,"5min":0.60899999999999999,"15min":0.58899999999999997},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.36599999999999999},"max":{"1min":0.872,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.56999999999999995}]},{"osd":7,"last update":"Wed Nov 8 19:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.61499999999999999,"15min":0.59199999999999997},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.85799999999999998,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.626,"5min":0.624,"15min":0.60999999999999999},"min":{"1min":0.45800000000000002,"5min":0.41699999999999998,"15min":0.38400000000000001},"max":{"1min":0.85999999999999999,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.52500000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574944,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28308,"kb_used_data":1292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343532,"statfs":{"total":96636764160,"available":96607776768,"internally_reserved":0,"allocated":1323008,"data_stored":646397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:31:59 2023","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"min":{"1min":0.249,"5min":0.224,"15min":0.20300000000000001},"max":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.378,"15min":0.35899999999999999},"min":{"1min":0.26400000000000001,"5min":0.23699999999999999,"15min":0.23300000000000001},"max":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.60799999999999998},"last":0.36699999999999999}]},{"osd":1,"last update":"Wed Nov 8 19:31:59 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.36699999999999999,"15min":0.35799999999999998},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.54100000000000004,"5min":0.61699999999999999,"15min":0.61699999999999999},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.40500000000000003,"15min":0.39100000000000001},"min":{"1min":0.28199999999999997,"5min":0.27900000000000003,"15min":0.27700000000000002},"max":{"1min":0.58799999999999997,"5min":0.64800000000000002,"15min":0.82099999999999995},"last":0.45100000000000001}]},{"osd":2,"last update":"Wed Nov 8 19:31:59 2023","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.41799999999999998,"15min":0.38700000000000001},"min":{"1min":0.30299999999999999,"5min":0.28999999999999998,"15min":0.28799999999999998},"max":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.42799999999999999,"15min":0.41199999999999998},"min":{"1min":0.315,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.50600000000000001}]},{"osd":4,"last update":"Wed Nov 8 19:31:59 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.626,"15min":0.60899999999999999},"min":{"1min":0.45100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64100000000000001,"15min":0.61799999999999999},"min":{"1min":0.46700000000000003,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.95899999999999996,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.65100000000000002}]},{"osd":5,"last update":"Wed Nov 8 19:31:59 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.59699999999999998,"15min":0.58499999999999996},"min":{"1min":0.44,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.61699999999999999,"15min":0.59299999999999997},"min":{"1min":0.38700000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.84799999999999998,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.67200000000000004}]},{"osd":6,"last update":"Wed Nov 8 19:31:59 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.61399999999999999,"15min":0.59299999999999997},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.627,"15min":0.61399999999999999},"min":{"1min":0.5,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.90500000000000003},"last":0.626}]},{"osd":7,"last update":"Wed Nov 8 19:31:59 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.621,"15min":0.60099999999999998},"min":{"1min":0.41299999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.82999999999999996,"5min":0.92900000000000005,"15min":1.0009999999999999},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64700000000000002,"15min":0.61299999999999999},"min":{"1min":0.45300000000000001,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.61399999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574943,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28308,"kb_used_data":1292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343532,"statfs":{"total":96636764160,"available":96607776768,"internally_reserved":0,"allocated":1323008,"data_stored":646397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:32:00 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.55000000000000004,"15min":0.68500000000000005},"min":{"1min":0.44,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.78000000000000003,"5min":0.84799999999999998,"15min":6.5970000000000004},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.54900000000000004,"15min":0.56399999999999995},"min":{"1min":0.39500000000000002,"5min":0.32200000000000001,"15min":0.317},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":2.5030000000000001},"last":0.65600000000000003}]},{"osd":1,"last update":"Wed Nov 8 19:32:00 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.55700000000000005,"15min":0.52000000000000002},"min":{"1min":0.42499999999999999,"5min":0.32900000000000001,"15min":0.24299999999999999},"max":{"1min":0.80500000000000005,"5min":0.86699999999999999,"15min":0.89100000000000001},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.62,"5min":0.57299999999999995,"15min":0.53200000000000003},"min":{"1min":0.38200000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.77200000000000002,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.82499999999999996}]},{"osd":2,"last update":"Wed Nov 8 19:32:00 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.56399999999999995,"15min":0.55700000000000005},"min":{"1min":0.48299999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.77900000000000003,"5min":0.80600000000000005,"15min":0.81200000000000006},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58099999999999996,"15min":0.74199999999999999},"min":{"1min":0.38100000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.78400000000000003,"5min":0.86199999999999999,"15min":6.6920000000000002},"last":0.72799999999999998}]},{"osd":3,"last update":"Wed Nov 8 19:32:00 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.57099999999999995,"15min":0.58299999999999996},"min":{"1min":0.41099999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":1.0700000000000001},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.622,"15min":0.61499999999999999},"min":{"1min":0.50900000000000001,"5min":0.41699999999999998,"15min":0.34599999999999997},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.91700000000000004},"last":0.875}]},{"osd":5,"last update":"Wed Nov 8 19:32:00 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.48399999999999999,"15min":0.52600000000000002},"min":{"1min":0.311,"5min":0.29599999999999999,"15min":0.28100000000000003},"max":{"1min":0.74299999999999999,"5min":0.81599999999999995,"15min":0.83799999999999997},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.50700000000000001,"15min":0.56499999999999995},"min":{"1min":0.34999999999999998,"5min":0.32600000000000001,"15min":0.30299999999999999},"max":{"1min":0.66600000000000004,"5min":0.745,"15min":1.1060000000000001},"last":0.81000000000000005}]},{"osd":6,"last update":"Wed Nov 8 19:32:00 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.53400000000000003,"15min":0.56799999999999995},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.68200000000000005,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.59199999999999997,"15min":0.60799999999999998},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.79200000000000004,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.76100000000000001}]},{"osd":7,"last update":"Wed Nov 8 19:32:00 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.55400000000000005,"15min":0.60499999999999998},"min":{"1min":0.40300000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.72999999999999998,"5min":0.84499999999999997,"15min":1.141},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.58599999999999997,"15min":0.624},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.33000000000000002},"max":{"1min":0.81999999999999995,"5min":0.86399999999999999,"15min":0.86799999999999999},"last":0.79500000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574943,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28308,"kb_used_data":1292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343532,"statfs":{"total":96636764160,"available":96607776768,"internally_reserved":0,"allocated":1323008,"data_stored":646397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:31:52 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.58599999999999997,"15min":0.54200000000000004},"min":{"1min":0.35499999999999998,"5min":0.31900000000000001,"15min":0.307},"max":{"1min":0.76900000000000002,"5min":3.9689999999999999,"15min":3.9689999999999999},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55500000000000005,"15min":0.52500000000000002},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.67200000000000004,"5min":0.746,"15min":0.746},"last":0.54900000000000004}]},{"osd":1,"last update":"Wed Nov 8 19:31:52 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.56100000000000005,"15min":0.53600000000000003},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.72599999999999998,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.54700000000000004,"15min":0.54000000000000004},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.66500000000000004,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.63500000000000001}]},{"osd":2,"last update":"Wed Nov 8 19:31:52 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.56999999999999995,"15min":0.54800000000000004},"min":{"1min":0.43099999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.76400000000000001,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.39500000000000002,"5min":0.316,"15min":0.316},"max":{"1min":0.71599999999999997,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.66300000000000003}]},{"osd":3,"last update":"Wed Nov 8 19:31:52 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59899999999999998,"15min":0.56499999999999995},"min":{"1min":0.46000000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.80000000000000004,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58999999999999997,"15min":0.59499999999999997},"min":{"1min":0.38600000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.78700000000000003,"5min":0.87,"15min":0.87},"last":0.70599999999999996}]},{"osd":4,"last update":"Wed Nov 8 19:31:52 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.48499999999999999,"15min":0.53300000000000003},"min":{"1min":0.34699999999999998,"5min":0.315,"15min":0.315},"max":{"1min":0.74099999999999999,"5min":0.80500000000000005,"15min":0.86799999999999999},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.53000000000000003,"15min":0.55800000000000005},"min":{"1min":0.36199999999999999,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.41699999999999998}]},{"osd":6,"last update":"Wed Nov 8 19:31:52 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.54800000000000004,"15min":0.58999999999999997},"min":{"1min":0.35799999999999998,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.70299999999999996,"5min":0.755,"15min":0.77800000000000002},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.58399999999999996,"15min":0.60399999999999998},"min":{"1min":0.40200000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.71599999999999997,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.44400000000000001}]},{"osd":7,"last update":"Wed Nov 8 19:31:52 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.57099999999999995,"15min":0.59599999999999997},"min":{"1min":0.37,"5min":0.37,"15min":0.36899999999999999},"max":{"1min":0.75900000000000001,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.58699999999999997,"15min":0.626},"min":{"1min":0.433,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.73499999999999999,"5min":0.81299999999999994,"15min":0.91900000000000004},"last":0.56100000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-08T19:32:24.120 DEBUG:teuthology.orchestra.run.smithi143:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2023-11-08T19:32:24.270 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.269+0000 7f331985b700 1 Processor -- start 2023-11-08T19:32:24.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.269+0000 7f331985b700 1 -- start start 2023-11-08T19:32:24.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.270+0000 7f331985b700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f331415dca0 0x7f33140b93f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:24.274 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.270+0000 7f331985b700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f33140b5ab0 0x7f33140b7280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:24.274 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.270+0000 7f331985b700 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33140b8f50 con 0x7f33140b5630 2023-11-08T19:32:24.274 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.270+0000 7f331985b700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33140b3360 con 0x7f33140b5ab0 2023-11-08T19:32:24.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.270+0000 7f331985b700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33140b7eb0 con 0x7f331415dca0 2023-11-08T19:32:24.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.270+0000 7f3312ffd700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f331415dca0 0x7f33140b93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:24.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.270+0000 7f33137fe700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f33140b5ab0 0x7f33140b7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:24.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.270+0000 7f33137fe700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f33140b5ab0 0x7f33140b7280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.143:40954/0 (socket says 172.21.15.143:40954) 2023-11-08T19:32:24.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.270+0000 7f33137fe700 1 -- 172.21.15.143:0/477303615 learned_addr learned my addr 172.21.15.143:0/477303615 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:32:24.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.270+0000 7f3312ffd700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f331415dca0 0x7f33140b93f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.143:45090/0 (socket says 172.21.15.143:45090) 2023-11-08T19:32:24.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.270+0000 7f33127fc700 1 --1- >> v1:172.21.15.174:6789/0 conn(0x7f33140b5630 0x7f33140b6b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.174:6789/0 says I am v1:172.21.15.143:36310/0 (socket says 172.21.15.143:36310) 2023-11-08T19:32:24.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.270+0000 7f3312ffd700 1 -- 172.21.15.143:0/477303615 >> v2:172.21.15.174:3300/0 conn(0x7f33140b5ab0 msgr2=0x7f33140b7280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:32:24.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.270+0000 7f3312ffd700 1 --2- 172.21.15.143:0/477303615 >> v2:172.21.15.174:3300/0 conn(0x7f33140b5ab0 0x7f33140b7280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:24.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.270+0000 7f3312ffd700 1 -- 172.21.15.143:0/477303615 >> v1:172.21.15.174:6789/0 conn(0x7f33140b5630 legacy=0x7f33140b6b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:24.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.270+0000 7f3312ffd700 1 -- 172.21.15.143:0/477303615 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33140b8190 con 0x7f331415dca0 2023-11-08T19:32:24.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.270+0000 7f33137fe700 1 --2- 172.21.15.143:0/477303615 >> v2:172.21.15.174:3300/0 conn(0x7f33140b5ab0 0x7f33140b7280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T19:32:24.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.271+0000 7f3312ffd700 1 --2- 172.21.15.143:0/477303615 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f331415dca0 0x7f33140b93f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3308009840 tx=0x7f3308009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac8777e331848ec7 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:24.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.271+0000 7f3311ffb700 1 -- 172.21.15.143:0/477303615 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3308011830 con 0x7f331415dca0 2023-11-08T19:32:24.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.271+0000 7f3311ffb700 1 -- 172.21.15.143:0/477303615 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3308011990 con 0x7f331415dca0 2023-11-08T19:32:24.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.271+0000 7f3311ffb700 1 -- 172.21.15.143:0/477303615 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3308011d20 con 0x7f331415dca0 2023-11-08T19:32:24.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.272+0000 7f331985b700 1 -- 172.21.15.143:0/477303615 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f331415dca0 msgr2=0x7f33140b93f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:24.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.272+0000 7f331985b700 1 --2- 172.21.15.143:0/477303615 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f331415dca0 0x7f33140b93f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3308009840 tx=0x7f3308009b50 comp rx=0 tx=0).stop 2023-11-08T19:32:24.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.272+0000 7f331985b700 1 -- 172.21.15.143:0/477303615 shutdown_connections 2023-11-08T19:32:24.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.272+0000 7f331985b700 1 --2- 172.21.15.143:0/477303615 >> v2:172.21.15.174:3300/0 conn(0x7f33140b5ab0 0x7f33140b7280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:24.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.272+0000 7f331985b700 1 --2- 172.21.15.143:0/477303615 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f331415dca0 0x7f33140b93f0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:24.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.272+0000 7f331985b700 1 -- 172.21.15.143:0/477303615 >> 172.21.15.143:0/477303615 conn(0x7f3314066d00 msgr2=0x7f3314067110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:32:24.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.272+0000 7f331985b700 1 -- 172.21.15.143:0/477303615 shutdown_connections 2023-11-08T19:32:24.281 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.272+0000 7f331985b700 1 -- 172.21.15.143:0/477303615 wait complete. 2023-11-08T19:32:24.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.273+0000 7f331985b700 1 Processor -- start 2023-11-08T19:32:24.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.273+0000 7f331985b700 1 -- start start 2023-11-08T19:32:24.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.274+0000 7f331985b700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f33140b5630 0x7f33140cac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:24.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.274+0000 7f331985b700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f33140b5ab0 0x7f33140cb1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:24.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.274+0000 7f331985b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f331415dca0 0x7f33140ccf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:24.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.274+0000 7f331985b700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33140cbb40 con 0x7f33140b5630 2023-11-08T19:32:24.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.274+0000 7f331985b700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33140c8000 con 0x7f331415dca0 2023-11-08T19:32:24.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.274+0000 7f331985b700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3314161080 con 0x7f33140b5ab0 2023-11-08T19:32:24.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.274+0000 7f3312ffd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f33140b5630 0x7f33140cac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:24.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.274+0000 7f3312ffd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f33140b5630 0x7f33140cac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57458/0 (socket says 172.21.15.143:57458) 2023-11-08T19:32:24.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.274+0000 7f3312ffd700 1 -- 172.21.15.143:0/3101863035 learned_addr learned my addr 172.21.15.143:0/3101863035 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:32:24.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.274+0000 7f33127fc700 1 --2- 172.21.15.143:0/3101863035 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f33140b5ab0 0x7f33140cb1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:24.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.274+0000 7f33137fe700 1 --2- 172.21.15.143:0/3101863035 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f331415dca0 0x7f33140ccf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:24.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.274+0000 7f3312ffd700 1 -- 172.21.15.143:0/3101863035 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f33140b5ab0 msgr2=0x7f33140cb1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:24.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.274+0000 7f3312ffd700 1 --2- 172.21.15.143:0/3101863035 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f33140b5ab0 0x7f33140cb1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:24.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.274+0000 7f3312ffd700 1 -- 172.21.15.143:0/3101863035 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f331415dca0 msgr2=0x7f33140ccf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:24.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.274+0000 7f3312ffd700 1 --2- 172.21.15.143:0/3101863035 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f331415dca0 0x7f33140ccf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:24.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.274+0000 7f3312ffd700 1 -- 172.21.15.143:0/3101863035 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f330c009b30 con 0x7f33140b5630 2023-11-08T19:32:24.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.274+0000 7f3312ffd700 1 --2- 172.21.15.143:0/3101863035 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f33140b5630 0x7f33140cac80 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f3308014040 tx=0x7f3308004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:24.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.274+0000 7f32fbfff700 1 -- 172.21.15.143:0/3101863035 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33080036a0 con 0x7f33140b5630 2023-11-08T19:32:24.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.274+0000 7f32fbfff700 1 -- 172.21.15.143:0/3101863035 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f330801f070 con 0x7f33140b5630 2023-11-08T19:32:24.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.274+0000 7f331985b700 1 -- 172.21.15.143:0/3101863035 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33080094a0 con 0x7f33140b5630 2023-11-08T19:32:24.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.274+0000 7f32fbfff700 1 -- 172.21.15.143:0/3101863035 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3308003a20 con 0x7f33140b5630 2023-11-08T19:32:24.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.275+0000 7f331985b700 1 -- 172.21.15.143:0/3101863035 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33140c8250 con 0x7f33140b5630 2023-11-08T19:32:24.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.276+0000 7f331985b700 1 -- 172.21.15.143:0/3101863035 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f331404eee0 con 0x7f33140b5630 2023-11-08T19:32:24.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.276+0000 7f32fbfff700 1 -- 172.21.15.143:0/3101863035 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3308003b80 con 0x7f33140b5630 2023-11-08T19:32:24.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.276+0000 7f32fbfff700 1 --2- 172.21.15.143:0/3101863035 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f330006af00 0x7f330006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:24.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.276+0000 7f32fbfff700 1 -- 172.21.15.143:0/3101863035 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6651+0+0 (secure 0 0 0) 0x7f3308021070 con 0x7f33140b5630 2023-11-08T19:32:24.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.276+0000 7f33127fc700 1 --2- 172.21.15.143:0/3101863035 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f330006af00 0x7f330006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:24.293 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.277+0000 7f33127fc700 1 --2- 172.21.15.143:0/3101863035 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f330006af00 0x7f330006d3c0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f3314151b20 tx=0x7f3304007ed0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:24.293 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.281+0000 7f32fbfff700 1 -- 172.21.15.143:0/3101863035 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3308055e00 con 0x7f33140b5630 2023-11-08T19:32:24.469 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.467+0000 7f331985b700 1 -- 172.21.15.143:0/3101863035 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f33140b3a90 con 0x7f33140b5630 2023-11-08T19:32:24.472 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.471+0000 7f32fbfff700 1 -- 172.21.15.143:0/3101863035 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v21)=0 v21) v1 ==== 123+0+0 (secure 0 0 0) 0x7f33080150c0 con 0x7f33140b5630 2023-11-08T19:32:24.473 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.472+0000 7f32fbfff700 1 -- 172.21.15.143:0/3101863035 <== mon.0 v2:172.21.15.143:3300/0 8 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f3308055e00 con 0x7f33140b5630 2023-11-08T19:32:24.475 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.473+0000 7f331985b700 1 -- 172.21.15.143:0/3101863035 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f330006af00 msgr2=0x7f330006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:24.475 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.473+0000 7f331985b700 1 --2- 172.21.15.143:0/3101863035 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f330006af00 0x7f330006d3c0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f3314151b20 tx=0x7f3304007ed0 comp rx=0 tx=0).stop 2023-11-08T19:32:24.475 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.473+0000 7f331985b700 1 -- 172.21.15.143:0/3101863035 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f33140b5630 msgr2=0x7f33140cac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:24.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.473+0000 7f331985b700 1 --2- 172.21.15.143:0/3101863035 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f33140b5630 0x7f33140cac80 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f3308014040 tx=0x7f3308004750 comp rx=0 tx=0).stop 2023-11-08T19:32:24.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.473+0000 7f331985b700 1 -- 172.21.15.143:0/3101863035 shutdown_connections 2023-11-08T19:32:24.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.473+0000 7f331985b700 1 --2- 172.21.15.143:0/3101863035 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f330006af00 0x7f330006d3c0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:24.477 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.473+0000 7f331985b700 1 --2- 172.21.15.143:0/3101863035 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f331415dca0 0x7f33140ccf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:24.477 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.473+0000 7f331985b700 1 --2- 172.21.15.143:0/3101863035 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f33140b5630 0x7f33140cac80 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:24.477 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.473+0000 7f331985b700 1 --2- 172.21.15.143:0/3101863035 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f33140b5ab0 0x7f33140cb1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:24.478 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.473+0000 7f331985b700 1 -- 172.21.15.143:0/3101863035 >> 172.21.15.143:0/3101863035 conn(0x7f3314066d00 msgr2=0x7f3314153280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:32:24.478 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.473+0000 7f331985b700 1 -- 172.21.15.143:0/3101863035 shutdown_connections 2023-11-08T19:32:24.478 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.473+0000 7f331985b700 1 -- 172.21.15.143:0/3101863035 wait complete. 2023-11-08T19:32:24.486 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-08T19:32:24.487 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:32:24.607 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.605+0000 7f13df36f700 1 Processor -- start 2023-11-08T19:32:24.607 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.606+0000 7f13df36f700 1 -- start start 2023-11-08T19:32:24.610 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.606+0000 7f13df36f700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f13d815aa30 0x7f13d80cf510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:24.610 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.606+0000 7f13df36f700 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f13d80b2210 0x7f13d80b2690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:24.610 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.606+0000 7f13df36f700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f13d80cbe20 0x7f13d80cc270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:24.611 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.606+0000 7f13df36f700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13d80b2bd0 con 0x7f13d80cbe20 2023-11-08T19:32:24.611 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.606+0000 7f13df36f700 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13d80cf080 con 0x7f13d80b2210 2023-11-08T19:32:24.611 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.606+0000 7f13df36f700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13d80ce470 con 0x7f13d815aa30 2023-11-08T19:32:24.611 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.607+0000 7f13dd90c700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f13d80cbe20 0x7f13d80cc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:24.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.607+0000 7f13dd90c700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f13d80cbe20 0x7f13d80cc270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57470/0 (socket says 172.21.15.143:57470) 2023-11-08T19:32:24.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.607+0000 7f13dd90c700 1 -- 172.21.15.143:0/3874355635 learned_addr learned my addr 172.21.15.143:0/3874355635 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:32:24.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.607+0000 7f13dc90a700 1 --2- 172.21.15.143:0/3874355635 >> v2:172.21.15.174:3300/0 conn(0x7f13d80b2210 0x7f13d80b2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:24.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.607+0000 7f13dd10b700 1 --2- 172.21.15.143:0/3874355635 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f13d815aa30 0x7f13d80cf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:24.613 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.607+0000 7f13dd90c700 1 -- 172.21.15.143:0/3874355635 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f13d815aa30 msgr2=0x7f13d80cf510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:32:24.613 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.607+0000 7f13dd90c700 1 --2- 172.21.15.143:0/3874355635 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f13d815aa30 0x7f13d80cf510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:24.613 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.607+0000 7f13dd90c700 1 -- 172.21.15.143:0/3874355635 >> v2:172.21.15.174:3300/0 conn(0x7f13d80b2210 msgr2=0x7f13d80b2690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:32:24.614 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.607+0000 7f13dd90c700 1 --2- 172.21.15.143:0/3874355635 >> v2:172.21.15.174:3300/0 conn(0x7f13d80b2210 0x7f13d80b2690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:24.614 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.607+0000 7f13dd90c700 1 -- 172.21.15.143:0/3874355635 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13d80ce750 con 0x7f13d80cbe20 2023-11-08T19:32:24.614 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.607+0000 7f13dd90c700 1 --2- 172.21.15.143:0/3874355635 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f13d80cbe20 0x7f13d80cc270 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f13d809a1e0 tx=0x7f13d000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a3e34600ac9f8f9 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:24.615 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.607+0000 7f13cffff700 1 -- 172.21.15.143:0/3874355635 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13d000e070 con 0x7f13d80cbe20 2023-11-08T19:32:24.615 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.607+0000 7f13cffff700 1 -- 172.21.15.143:0/3874355635 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f13d000b9c0 con 0x7f13d80cbe20 2023-11-08T19:32:24.615 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.607+0000 7f13cffff700 1 -- 172.21.15.143:0/3874355635 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13d000bd90 con 0x7f13d80cbe20 2023-11-08T19:32:24.616 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.608+0000 7f13df36f700 1 -- 172.21.15.143:0/3874355635 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f13d80cbe20 msgr2=0x7f13d80cc270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:24.616 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.608+0000 7f13df36f700 1 --2- 172.21.15.143:0/3874355635 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f13d80cbe20 0x7f13d80cc270 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f13d809a1e0 tx=0x7f13d000b330 comp rx=0 tx=0).stop 2023-11-08T19:32:24.616 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.608+0000 7f13df36f700 1 -- 172.21.15.143:0/3874355635 shutdown_connections 2023-11-08T19:32:24.616 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.608+0000 7f13df36f700 1 --2- 172.21.15.143:0/3874355635 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f13d80cbe20 0x7f13d80cc270 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:24.617 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.608+0000 7f13df36f700 1 --2- 172.21.15.143:0/3874355635 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f13d815aa30 0x7f13d80cf510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:24.617 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.608+0000 7f13df36f700 1 --2- 172.21.15.143:0/3874355635 >> v2:172.21.15.174:3300/0 conn(0x7f13d80b2210 0x7f13d80b2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:24.617 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.608+0000 7f13df36f700 1 -- 172.21.15.143:0/3874355635 >> 172.21.15.143:0/3874355635 conn(0x7f13d8099ab0 msgr2=0x7f13d80d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:32:24.617 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.608+0000 7f13df36f700 1 -- 172.21.15.143:0/3874355635 shutdown_connections 2023-11-08T19:32:24.618 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.608+0000 7f13df36f700 1 -- 172.21.15.143:0/3874355635 wait complete. 2023-11-08T19:32:24.621 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.609+0000 7f13df36f700 1 Processor -- start 2023-11-08T19:32:24.622 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.609+0000 7f13df36f700 1 -- start start 2023-11-08T19:32:24.622 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.610+0000 7f13df36f700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f13d80b2210 0x7f13d815cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:24.622 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.610+0000 7f13df36f700 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f13d80cbe20 0x7f13d815ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:24.623 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.610+0000 7f13df36f700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f13d815aa30 0x7f13d81538c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:24.623 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.610+0000 7f13df36f700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13d815b7f0 con 0x7f13d80b2210 2023-11-08T19:32:24.623 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.610+0000 7f13df36f700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13d815bad0 con 0x7f13d815aa30 2023-11-08T19:32:24.623 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.610+0000 7f13df36f700 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13d815bdb0 con 0x7f13d80cbe20 2023-11-08T19:32:24.624 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.610+0000 7f13dd10b700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f13d80b2210 0x7f13d815cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:24.624 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.610+0000 7f13dd10b700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f13d80b2210 0x7f13d815cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57472/0 (socket says 172.21.15.143:57472) 2023-11-08T19:32:24.624 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.610+0000 7f13dd10b700 1 -- 172.21.15.143:0/3324815192 learned_addr learned my addr 172.21.15.143:0/3324815192 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-11-08T19:32:24.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.610+0000 7f13dd90c700 1 --2- 172.21.15.143:0/3324815192 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f13d815aa30 0x7f13d81538c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:24.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.610+0000 7f13dc90a700 1 --2- 172.21.15.143:0/3324815192 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f13d80cbe20 0x7f13d815ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:24.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.610+0000 7f13dd10b700 1 -- 172.21.15.143:0/3324815192 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f13d80cbe20 msgr2=0x7f13d815ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:24.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.610+0000 7f13dd10b700 1 --2- 172.21.15.143:0/3324815192 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f13d80cbe20 0x7f13d815ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:24.626 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.610+0000 7f13dd10b700 1 -- 172.21.15.143:0/3324815192 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f13d815aa30 msgr2=0x7f13d81538c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:24.626 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.610+0000 7f13dd10b700 1 --2- 172.21.15.143:0/3324815192 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f13d815aa30 0x7f13d81538c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:24.626 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.610+0000 7f13dd10b700 1 -- 172.21.15.143:0/3324815192 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13d0009b30 con 0x7f13d80b2210 2023-11-08T19:32:24.627 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.610+0000 7f13dc90a700 1 --2- 172.21.15.143:0/3324815192 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f13d80cbe20 0x7f13d815ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:32:24.627 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.610+0000 7f13dd90c700 1 --2- 172.21.15.143:0/3324815192 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f13d815aa30 0x7f13d81538c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:32:24.627 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.610+0000 7f13dd10b700 1 --2- 172.21.15.143:0/3324815192 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f13d80b2210 0x7f13d815cfc0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f13c800e730 tx=0x7f13c800eaf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:24.627 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.611+0000 7f13cdffb700 1 -- 172.21.15.143:0/3324815192 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13c8013610 con 0x7f13d80b2210 2023-11-08T19:32:24.628 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.611+0000 7f13cdffb700 1 -- 172.21.15.143:0/3324815192 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f13c8013770 con 0x7f13d80b2210 2023-11-08T19:32:24.628 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.611+0000 7f13cdffb700 1 -- 172.21.15.143:0/3324815192 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13c8013b40 con 0x7f13d80b2210 2023-11-08T19:32:24.628 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.611+0000 7f13df36f700 1 -- 172.21.15.143:0/3324815192 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13d815c090 con 0x7f13d80b2210 2023-11-08T19:32:24.629 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.611+0000 7f13df36f700 1 -- 172.21.15.143:0/3324815192 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13d815d500 con 0x7f13d80b2210 2023-11-08T19:32:24.629 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.612+0000 7f13df36f700 1 -- 172.21.15.143:0/3324815192 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13d806efe0 con 0x7f13d80b2210 2023-11-08T19:32:24.629 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.612+0000 7f13cdffb700 1 -- 172.21.15.143:0/3324815192 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f13c8013ca0 con 0x7f13d80b2210 2023-11-08T19:32:24.630 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.612+0000 7f13cdffb700 1 --2- 172.21.15.143:0/3324815192 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f13bc06af00 0x7f13bc06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:24.630 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.613+0000 7f13cdffb700 1 -- 172.21.15.143:0/3324815192 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6651+0+0 (secure 0 0 0) 0x7f13c8089460 con 0x7f13d80b2210 2023-11-08T19:32:24.630 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.613+0000 7f13dc90a700 1 --2- 172.21.15.143:0/3324815192 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f13bc06af00 0x7f13bc06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:24.630 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.613+0000 7f13dc90a700 1 --2- 172.21.15.143:0/3324815192 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f13bc06af00 0x7f13bc06d3c0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f13d80b1e80 tx=0x7f13d4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:24.631 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.617+0000 7f13cdffb700 1 -- 172.21.15.143:0/3324815192 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f13c8054030 con 0x7f13d80b2210 2023-11-08T19:32:24.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.848+0000 7f13df36f700 1 -- 172.21.15.143:0/3324815192 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f13d804eee0 con 0x7f13d80b2210 2023-11-08T19:32:24.850 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.848+0000 7f13cdffb700 1 -- 172.21.15.143:0/3324815192 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 102 v102) v1 ==== 95+0+3714 (secure 0 0 0) 0x7f13c8021ea0 con 0x7f13d80b2210 2023-11-08T19:32:24.851 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-11-08T19:32:24.851 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":102,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5933,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/3918286710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6835","nonce":3918286710},{"type":"v1","addr":"172.21.15.174:6837","nonce":3918286710}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":6245,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6836/2689884850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2689884850},{"type":"v1","addr":"172.21.15.174:6836","nonce":2689884850}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":97},{"gid":8004,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6833/3843079270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":3843079270},{"type":"v1","addr":"172.21.15.143:6833","nonce":3843079270}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102},{"gid":8013,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/2248567243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":2248567243},{"type":"v1","addr":"172.21.15.143:6835","nonce":2248567243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102}],"filesystems":[]} 2023-11-08T19:32:24.852 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.850+0000 7f13df36f700 1 -- 172.21.15.143:0/3324815192 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f13bc06af00 msgr2=0x7f13bc06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:24.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.850+0000 7f13df36f700 1 --2- 172.21.15.143:0/3324815192 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f13bc06af00 0x7f13bc06d3c0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f13d80b1e80 tx=0x7f13d4008040 comp rx=0 tx=0).stop 2023-11-08T19:32:24.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.850+0000 7f13df36f700 1 -- 172.21.15.143:0/3324815192 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f13d80b2210 msgr2=0x7f13d815cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:24.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.850+0000 7f13df36f700 1 --2- 172.21.15.143:0/3324815192 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f13d80b2210 0x7f13d815cfc0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f13c800e730 tx=0x7f13c800eaf0 comp rx=0 tx=0).stop 2023-11-08T19:32:24.854 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.850+0000 7f13df36f700 1 -- 172.21.15.143:0/3324815192 shutdown_connections 2023-11-08T19:32:24.854 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.850+0000 7f13df36f700 1 --2- 172.21.15.143:0/3324815192 >> [v2:172.21.15.174:6832/4037280178,v1:172.21.15.174:6833/4037280178] conn(0x7f13bc06af00 0x7f13bc06d3c0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:24.854 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.850+0000 7f13df36f700 1 --2- 172.21.15.143:0/3324815192 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f13d815aa30 0x7f13d81538c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:24.855 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.850+0000 7f13df36f700 1 --2- 172.21.15.143:0/3324815192 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f13d80b2210 0x7f13d815cfc0 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:24.855 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.850+0000 7f13df36f700 1 --2- 172.21.15.143:0/3324815192 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f13d80cbe20 0x7f13d815ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:24.855 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.850+0000 7f13df36f700 1 -- 172.21.15.143:0/3324815192 >> 172.21.15.143:0/3324815192 conn(0x7f13d8099ab0 msgr2=0x7f13d80af770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:32:24.856 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.850+0000 7f13df36f700 1 -- 172.21.15.143:0/3324815192 shutdown_connections 2023-11-08T19:32:24.856 INFO:teuthology.orchestra.run.smithi143.stderr:2023-11-08T19:32:24.850+0000 7f13df36f700 1 -- 172.21.15.143:0/3324815192 wait complete. 2023-11-08T19:32:24.857 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 102 2023-11-08T19:32:24.864 DEBUG:tasks.cephfs.filesystem:already dead... 2023-11-08T19:32:24.864 INFO:teuthology.misc:Shutting down mds daemons... 2023-11-08T19:32:24.865 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-08T19:32:24.865 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T19:32:24.873 INFO:tasks.ceph.mds.a:Stopped 2023-11-08T19:32:24.874 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-08T19:32:24.874 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T19:32:24.922 INFO:tasks.ceph.mds.c:Stopped 2023-11-08T19:32:24.927 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-08T19:32:24.928 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T19:32:24.938 INFO:tasks.ceph.mds.b:Stopped 2023-11-08T19:32:24.938 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-08T19:32:24.939 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T19:32:24.989 INFO:tasks.ceph.mds.d:Stopped 2023-11-08T19:32:24.990 INFO:teuthology.misc:Shutting down osd daemons... 2023-11-08T19:32:24.990 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-11-08T19:32:24.991 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T19:32:25.025 INFO:tasks.ceph.osd.0:Stopped 2023-11-08T19:32:25.026 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-11-08T19:32:25.026 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T19:32:25.110 INFO:tasks.ceph.osd.1:Stopped 2023-11-08T19:32:25.111 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-11-08T19:32:25.111 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T19:32:25.154 INFO:tasks.ceph.osd.2:Stopped 2023-11-08T19:32:25.154 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-11-08T19:32:25.155 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T19:32:25.197 INFO:tasks.ceph.osd.3:Stopped 2023-11-08T19:32:25.198 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2023-11-08T19:32:25.198 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T19:32:25.231 INFO:tasks.ceph.osd.4:Stopped 2023-11-08T19:32:25.232 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2023-11-08T19:32:25.232 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T19:32:25.315 INFO:tasks.ceph.osd.5:Stopped 2023-11-08T19:32:25.316 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2023-11-08T19:32:25.316 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T19:32:25.366 INFO:tasks.ceph.osd.6:Stopped 2023-11-08T19:32:25.367 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2023-11-08T19:32:25.367 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T19:32:25.403 INFO:tasks.ceph.osd.7:Stopped 2023-11-08T19:32:25.403 INFO:teuthology.misc:Shutting down mgr daemons... 2023-11-08T19:32:25.404 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-11-08T19:32:25.404 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T19:32:25.427 INFO:tasks.ceph.mgr.y:Stopped 2023-11-08T19:32:25.427 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-11-08T19:32:25.428 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T19:32:25.470 INFO:tasks.ceph.mgr.x:Stopped 2023-11-08T19:32:25.471 INFO:teuthology.misc:Shutting down mon daemons... 2023-11-08T19:32:25.472 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-11-08T19:32:25.472 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T19:32:25.482 INFO:tasks.ceph.mon.a:Stopped 2023-11-08T19:32:25.483 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-11-08T19:32:25.483 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T19:32:25.522 INFO:tasks.ceph.mon.b:Stopped 2023-11-08T19:32:25.523 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-11-08T19:32:25.523 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T19:32:25.573 INFO:tasks.ceph.mon.c:Stopped 2023-11-08T19:32:25.574 INFO:tasks.ceph:Checking cluster log for badness... 2023-11-08T19:32:25.575 DEBUG:teuthology.orchestra.run.smithi143:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'evicting unresponsive client' | egrep -v RECENT_CRASH | head -n 1 2023-11-08T19:32:25.606 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi143.front.sepia.ceph.com 2023-11-08T19:32:25.607 DEBUG:teuthology.orchestra.run.smithi143:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-11-08T19:32:25.832 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi143.front.sepia.ceph.com 2023-11-08T19:32:25.832 DEBUG:teuthology.orchestra.run.smithi143:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-11-08T19:32:25.910 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi143.front.sepia.ceph.com 2023-11-08T19:32:25.911 DEBUG:teuthology.orchestra.run.smithi143:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-11-08T19:32:25.986 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi143.front.sepia.ceph.com 2023-11-08T19:32:25.987 DEBUG:teuthology.orchestra.run.smithi143:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-11-08T19:32:26.063 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi174.front.sepia.ceph.com 2023-11-08T19:32:26.064 DEBUG:teuthology.orchestra.run.smithi174:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2023-11-08T19:32:26.241 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi174.front.sepia.ceph.com 2023-11-08T19:32:26.241 DEBUG:teuthology.orchestra.run.smithi174:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2023-11-08T19:32:26.321 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi174.front.sepia.ceph.com 2023-11-08T19:32:26.321 DEBUG:teuthology.orchestra.run.smithi174:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2023-11-08T19:32:26.401 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi174.front.sepia.ceph.com 2023-11-08T19:32:26.402 DEBUG:teuthology.orchestra.run.smithi174:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2023-11-08T19:32:26.498 INFO:tasks.ceph:Archiving mon data... 2023-11-08T19:32:26.499 DEBUG:teuthology.misc:Transferring archived files from smithi143:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/pdonnell-2023-11-08_19:01:12-fs:functional-main-distro-default-smithi/7452105/data/mon.a.tgz 2023-11-08T19:32:26.500 DEBUG:teuthology.orchestra.run.smithi143:> mktemp 2023-11-08T19:32:26.518 INFO:teuthology.orchestra.run.smithi143.stdout:/tmp/tmp.mdPICbpK0L 2023-11-08T19:32:26.519 DEBUG:teuthology.orchestra.run.smithi143:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.mdPICbpK0L 2023-11-08T19:32:26.684 DEBUG:teuthology.orchestra.run.smithi143:> sudo chmod 0666 /tmp/tmp.mdPICbpK0L 2023-11-08T19:32:26.770 DEBUG:teuthology.orchestra.remote:smithi143:/tmp/tmp.mdPICbpK0L is 469KB 2023-11-08T19:32:26.882 DEBUG:teuthology.orchestra.run.smithi143:> rm -fr /tmp/tmp.mdPICbpK0L 2023-11-08T19:32:26.899 DEBUG:teuthology.misc:Transferring archived files from smithi174:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/pdonnell-2023-11-08_19:01:12-fs:functional-main-distro-default-smithi/7452105/data/mon.b.tgz 2023-11-08T19:32:26.899 DEBUG:teuthology.orchestra.run.smithi174:> mktemp 2023-11-08T19:32:26.917 INFO:teuthology.orchestra.run.smithi174.stdout:/tmp/tmp.2IexSVr8Wq 2023-11-08T19:32:26.918 DEBUG:teuthology.orchestra.run.smithi174:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.2IexSVr8Wq 2023-11-08T19:32:27.090 DEBUG:teuthology.orchestra.run.smithi174:> sudo chmod 0666 /tmp/tmp.2IexSVr8Wq 2023-11-08T19:32:27.177 DEBUG:teuthology.orchestra.remote:smithi174:/tmp/tmp.2IexSVr8Wq is 480KB 2023-11-08T19:32:27.331 DEBUG:teuthology.orchestra.run.smithi174:> rm -fr /tmp/tmp.2IexSVr8Wq 2023-11-08T19:32:27.349 DEBUG:teuthology.misc:Transferring archived files from smithi174:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/pdonnell-2023-11-08_19:01:12-fs:functional-main-distro-default-smithi/7452105/data/mon.c.tgz 2023-11-08T19:32:27.350 DEBUG:teuthology.orchestra.run.smithi174:> mktemp 2023-11-08T19:32:27.404 INFO:teuthology.orchestra.run.smithi174.stdout:/tmp/tmp.YOd6irJeO1 2023-11-08T19:32:27.405 DEBUG:teuthology.orchestra.run.smithi174:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.YOd6irJeO1 2023-11-08T19:32:27.561 DEBUG:teuthology.orchestra.run.smithi174:> sudo chmod 0666 /tmp/tmp.YOd6irJeO1 2023-11-08T19:32:27.649 DEBUG:teuthology.orchestra.remote:smithi174:/tmp/tmp.YOd6irJeO1 is 480KB 2023-11-08T19:32:27.748 DEBUG:teuthology.orchestra.run.smithi174:> rm -fr /tmp/tmp.YOd6irJeO1 2023-11-08T19:32:27.765 INFO:tasks.ceph:Cleaning ceph cluster... 2023-11-08T19:32:27.765 DEBUG:teuthology.orchestra.run.smithi143:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-11-08T19:32:27.768 DEBUG:teuthology.orchestra.run.smithi174:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-11-08T19:32:27.830 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-11-08T19:32:27.831 DEBUG:teuthology.orchestra.run.smithi143:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-11-08T19:32:27.839 DEBUG:teuthology.orchestra.run.smithi174:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-11-08T19:32:27.880 INFO:teuthology.orchestra.run.smithi143.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-11-08T19:32:27.912 INFO:teuthology.orchestra.run.smithi174.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-11-08T19:32:27.918 INFO:tasks.ceph:Archiving crash dumps... 2023-11-08T19:32:27.939 DEBUG:teuthology.misc:Transferring archived files from smithi143:/var/lib/ceph/crash to /home/teuthworker/archive/pdonnell-2023-11-08_19:01:12-fs:functional-main-distro-default-smithi/7452105/remote/smithi143/crash 2023-11-08T19:32:27.941 DEBUG:teuthology.orchestra.run.smithi143:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-11-08T19:32:27.951 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T19:32:27.991 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T19:32:28.027 DEBUG:teuthology.misc:Transferring archived files from smithi174:/var/lib/ceph/crash to /home/teuthworker/archive/pdonnell-2023-11-08_19:01:12-fs:functional-main-distro-default-smithi/7452105/remote/smithi174/crash 2023-11-08T19:32:28.028 DEBUG:teuthology.orchestra.run.smithi174:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-11-08T19:32:28.092 INFO:tasks.ceph:Shutting down logrotate 2023-11-08T19:32:28.092 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T19:32:28.095 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T19:32:28.174 DEBUG:teuthology.orchestra.run.smithi143:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-11-08T19:32:28.202 DEBUG:teuthology.orchestra.run.smithi174:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-11-08T19:32:28.229 INFO:tasks.ceph:Compressing logs... 2023-11-08T19:32:28.229 DEBUG:teuthology.orchestra.run.smithi143:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-11-08T19:32:28.245 DEBUG:teuthology.orchestra.run.smithi174:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-11-08T19:32:28.271 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111118.log 2023-11-08T19:32:28.272 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.132731.log 2023-11-08T19:32:28.304 INFO:teuthology.orchestra.run.smithi174.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.111762.log 2023-11-08T19:32:28.305 INFO:teuthology.orchestra.run.smithi174.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2023-11-08T19:32:28.305 INFO:teuthology.orchestra.run.smithi174.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.111147.log 2023-11-08T19:32:28.305 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/ceph-client.3.111762.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.111762.log.gz 2023-11-08T19:32:28.306 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104128.log 2023-11-08T19:32:28.306 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/ceph-client.1.111147.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-11-08T19:32:28.306 INFO:teuthology.orchestra.run.smithi174.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.111147.log.gz 2023-11-08T19:32:28.306 INFO:teuthology.orchestra.run.smithi174.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104036.log 2023-11-08T19:32:28.307 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/ceph-client.admin.104128.log: /var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103990.log 2023-11-08T19:32:28.307 INFO:teuthology.orchestra.run.smithi174.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.104128.log.gz 2023-11-08T19:32:28.307 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/ceph-client.admin.104036.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2023-11-08T19:32:28.308 INFO:teuthology.orchestra.run.smithi174.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.104036.log.gz 2023-11-08T19:32:28.308 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/ceph-client.admin.103990.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2023-11-08T19:32:28.308 INFO:teuthology.orchestra.run.smithi174.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104316.log 2023-11-08T19:32:28.308 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/ceph-osd.6.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.103990.log.gz 2023-11-08T19:32:28.309 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/ceph-mds.d.log: /var/log/ceph/ceph-client.admin.104316.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.104316.log.gz 2023-11-08T19:32:28.309 INFO:teuthology.orchestra.run.smithi174.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-11-08T19:32:28.309 INFO:teuthology.orchestra.run.smithi174.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104218.log 2023-11-08T19:32:28.309 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/ceph.audit.log: 92.3% -- replaced with /var/log/ceph/ceph.log.gz 2023-11-08T19:32:28.310 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/ceph-client.admin.104218.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.104218.log.gz 2023-11-08T19:32:28.310 INFO:teuthology.orchestra.run.smithi174.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2023-11-08T19:32:28.310 INFO:teuthology.orchestra.run.smithi174.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2023-11-08T19:32:28.311 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/ceph-osd.4.log: /var/log/ceph/ceph-mgr.x.log: 93.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-11-08T19:32:28.314 INFO:teuthology.orchestra.run.smithi174.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.111155.log 2023-11-08T19:32:28.314 INFO:teuthology.orchestra.run.smithi174.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104081.log 2023-11-08T19:32:28.315 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/ceph-client.1.111155.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.111155.log.gz 2023-11-08T19:32:28.315 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/ceph-client.admin.104081.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.104081.log.gz 2023-11-08T19:32:28.315 INFO:teuthology.orchestra.run.smithi174.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2023-11-08T19:32:28.315 INFO:teuthology.orchestra.run.smithi174.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2023-11-08T19:32:28.326 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/ceph-mon.c.log: /var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.111766.log 2023-11-08T19:32:28.327 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/ceph-client.3.111766.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.111766.log.gz 2023-11-08T19:32:28.327 INFO:teuthology.orchestra.run.smithi174.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2023-11-08T19:32:28.337 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104173.log 2023-11-08T19:32:28.340 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.111118.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111117.log 2023-11-08T19:32:28.341 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.0.132731.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.111118.log.gz 2023-11-08T19:32:28.341 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126997.log 2023-11-08T19:32:28.341 INFO:teuthology.orchestra.run.smithi143.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.132731.log.gz 2023-11-08T19:32:28.342 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139930.log 2023-11-08T19:32:28.342 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.111117.log: /var/log/ceph/ceph-client.admin.126997.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120252.log 2023-11-08T19:32:28.342 INFO:teuthology.orchestra.run.smithi143.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.111117.log.gz 2023-11-08T19:32:28.342 INFO:teuthology.orchestra.run.smithi143.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.126997.log.gz 2023-11-08T19:32:28.343 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.139930.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115713.log 2023-11-08T19:32:28.343 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.120252.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.139930.log.gz 2023-11-08T19:32:28.343 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122536.log 2023-11-08T19:32:28.343 INFO:teuthology.orchestra.run.smithi143.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.120252.log.gz 2023-11-08T19:32:28.344 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.115713.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114967.log 2023-11-08T19:32:28.344 INFO:teuthology.orchestra.run.smithi143.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.115713.log.gz 2023-11-08T19:32:28.344 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.122536.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137631.log 2023-11-08T19:32:28.345 INFO:teuthology.orchestra.run.smithi143.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.122536.log.gz 2023-11-08T19:32:28.345 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.114967.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137032.log 2023-11-08T19:32:28.345 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.137631.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.114967.log.gz 2023-11-08T19:32:28.345 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114506.log 2023-11-08T19:32:28.346 INFO:teuthology.orchestra.run.smithi143.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.137631.log.gz 2023-11-08T19:32:28.346 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.137032.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115667.log 2023-11-08T19:32:28.346 INFO:teuthology.orchestra.run.smithi143.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.137032.log.gz 2023-11-08T19:32:28.346 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.114506.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112479.log 2023-11-08T19:32:28.347 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.115667.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116088.log 2023-11-08T19:32:28.347 INFO:teuthology.orchestra.run.smithi143.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.114506.log.gz 2023-11-08T19:32:28.347 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.112479.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127136.log 2023-11-08T19:32:28.348 INFO:teuthology.orchestra.run.smithi143.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.115667.log.gz 2023-11-08T19:32:28.348 INFO:teuthology.orchestra.run.smithi143.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.112479.log.gz 2023-11-08T19:32:28.348 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.116088.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133854.log 2023-11-08T19:32:28.348 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.127136.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.116088.log.gz 2023-11-08T19:32:28.349 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.113391.log 2023-11-08T19:32:28.349 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.133854.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.127136.log.gz 2023-11-08T19:32:28.349 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138381.log 2023-11-08T19:32:28.349 INFO:teuthology.orchestra.run.smithi143.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.133854.log.gz 2023-11-08T19:32:28.350 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130129.log 2023-11-08T19:32:28.350 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.0.113391.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.113391.log.gz 2023-11-08T19:32:28.350 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.138381.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117096.log 2023-11-08T19:32:28.351 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.130129.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.138381.log.gz 2023-11-08T19:32:28.351 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114552.log 2023-11-08T19:32:28.351 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.117096.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.130129.log.gz 2023-11-08T19:32:28.351 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130773.log 2023-11-08T19:32:28.352 INFO:teuthology.orchestra.run.smithi143.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.117096.log.gz 2023-11-08T19:32:28.352 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.114552.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116866.log 2023-11-08T19:32:28.352 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.130773.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.114552.log.gz 2023-11-08T19:32:28.352 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133045.log 2023-11-08T19:32:28.353 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.116866.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.130773.log.gz 2023-11-08T19:32:28.353 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123118.log 2023-11-08T19:32:28.353 INFO:teuthology.orchestra.run.smithi143.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.116866.log.gz 2023-11-08T19:32:28.353 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.133045.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129991.log 2023-11-08T19:32:28.354 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.123118.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125074.log 2023-11-08T19:32:28.354 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.129991.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.123118.log.gz 2023-11-08T19:32:28.354 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137213.log 2023-11-08T19:32:28.355 INFO:teuthology.orchestra.run.smithi143.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.133045.log.gz 2023-11-08T19:32:28.355 INFO:teuthology.orchestra.run.smithi143.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.129991.log.gz 2023-11-08T19:32:28.355 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.125074.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113621.log 2023-11-08T19:32:28.355 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.137213.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138611.log 2023-11-08T19:32:28.356 INFO:teuthology.orchestra.run.smithi143.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.125074.log.gz 2023-11-08T19:32:28.356 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.113621.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128687.log 2023-11-08T19:32:28.356 INFO:teuthology.orchestra.run.smithi143.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.137213.log.gz 2023-11-08T19:32:28.356 INFO:teuthology.orchestra.run.smithi143.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.113621.log.gz 2023-11-08T19:32:28.357 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.138611.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110710.log 2023-11-08T19:32:28.357 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.128687.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.138611.log.gz 2023-11-08T19:32:28.357 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130175.log 2023-11-08T19:32:28.357 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.110710.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.128687.log.gz 2023-11-08T19:32:28.358 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122429.log 2023-11-08T19:32:28.358 INFO:teuthology.orchestra.run.smithi143.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.110710.log.gz 2023-11-08T19:32:28.358 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.130175.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140758.log 2023-11-08T19:32:28.359 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.122429.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.130175.log.gz 2023-11-08T19:32:28.359 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128549.log 2023-11-08T19:32:28.359 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.140758.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129945.log 2023-11-08T19:32:28.359 INFO:teuthology.orchestra.run.smithi143.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.122429.log.gz 2023-11-08T19:32:28.360 INFO:teuthology.orchestra.run.smithi143.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.140758.log.gz 2023-11-08T19:32:28.360 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.128549.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132056.log 2023-11-08T19:32:28.360 INFO:teuthology.orchestra.run.smithi143.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.128549.log.gz 2023-11-08T19:32:28.360 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.129945.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135091.log 2023-11-08T19:32:28.361 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.132056.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119529.log 2023-11-08T19:32:28.361 INFO:teuthology.orchestra.run.smithi143.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.129945.log.gz 2023-11-08T19:32:28.361 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.135091.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110610.log 2023-11-08T19:32:28.361 INFO:teuthology.orchestra.run.smithi143.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.132056.log.gz 2023-11-08T19:32:28.362 INFO:teuthology.orchestra.run.smithi143.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.135091.log.gz 2023-11-08T19:32:28.362 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.119529.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2023-11-08T19:32:28.362 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.110610.log: 89.4%gzip -- replaced with /var/log/ceph/ceph-client.admin.119529.log.gz-5 2023-11-08T19:32:28.363 INFO:teuthology.orchestra.run.smithi143.stderr:--verbose -- /var/log/ceph/ceph-client.admin.121768.log 2023-11-08T19:32:28.363 INFO:teuthology.orchestra.run.smithi143.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.110610.log.gz 2023-11-08T19:32:28.363 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127947.log 2023-11-08T19:32:28.363 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.121768.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120188.log 2023-11-08T19:32:28.364 INFO:teuthology.orchestra.run.smithi143.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.121768.log.gz 2023-11-08T19:32:28.364 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.127947.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140344.log 2023-11-08T19:32:28.364 INFO:teuthology.orchestra.run.smithi143.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.127947.log.gz 2023-11-08T19:32:28.364 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137988.log 2023-11-08T19:32:28.365 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.120188.log: /var/log/ceph/ceph-client.admin.140344.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111482.log 2023-11-08T19:32:28.365 INFO:teuthology.orchestra.run.smithi143.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.120188.log.gz 2023-11-08T19:32:28.365 INFO:teuthology.orchestra.run.smithi143.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.140344.log.gz 2023-11-08T19:32:28.366 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.137988.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138427.log 2023-11-08T19:32:28.366 INFO:teuthology.orchestra.run.smithi143.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.137988.log.gz 2023-11-08T19:32:28.366 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127458.log 2023-11-08T19:32:28.366 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.111482.log: /var/log/ceph/ceph-client.admin.138427.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139707.log 2023-11-08T19:32:28.367 INFO:teuthology.orchestra.run.smithi143.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.111482.log.gz 2023-11-08T19:32:28.367 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.127458.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.138427.log.gz 2023-11-08T19:32:28.367 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117278.log 2023-11-08T19:32:28.368 INFO:teuthology.orchestra.run.smithi143.stderr: 88.4%gzip -- replaced with /var/log/ceph/ceph-client.admin.127458.log.gz-5 /var/log/ceph/ceph-client.admin.139707.log: 2023-11-08T19:32:28.368 INFO:teuthology.orchestra.run.smithi143.stderr:--verbose -- /var/log/ceph/ceph-client.admin.120950.log 2023-11-08T19:32:28.368 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.117278.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.139707.log.gz 2023-11-08T19:32:28.368 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131918.log 2023-11-08T19:32:28.369 INFO:teuthology.orchestra.run.smithi143.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.117278.log.gz 2023-11-08T19:32:28.369 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.120950.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124890.log 2023-11-08T19:32:28.369 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.131918.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139040.log 2023-11-08T19:32:28.369 INFO:teuthology.orchestra.run.smithi143.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.120950.log.gz 2023-11-08T19:32:28.370 INFO:teuthology.orchestra.run.smithi143.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.131918.log.gz 2023-11-08T19:32:28.370 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.124890.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118254.log 2023-11-08T19:32:28.370 INFO:teuthology.orchestra.run.smithi143.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.124890.log.gz 2023-11-08T19:32:28.371 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110823.log 2023-11-08T19:32:28.371 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.139040.log: /var/log/ceph/ceph-client.admin.118254.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114324.log 2023-11-08T19:32:28.371 INFO:teuthology.orchestra.run.smithi143.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.139040.log.gz 2023-11-08T19:32:28.371 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.110823.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.118254.log.gz 2023-11-08T19:32:28.372 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137794.log 2023-11-08T19:32:28.372 INFO:teuthology.orchestra.run.smithi143.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.110823.log.gz 2023-11-08T19:32:28.372 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.114324.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134975.log 2023-11-08T19:32:28.373 INFO:teuthology.orchestra.run.smithi143.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.114324.log.gz 2023-11-08T19:32:28.373 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.137794.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128503.log 2023-11-08T19:32:28.373 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.134975.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139270.log 2023-11-08T19:32:28.373 INFO:teuthology.orchestra.run.smithi143.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.137794.log.gz 2023-11-08T19:32:28.374 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.128503.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.134975.log.gz 2023-11-08T19:32:28.374 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119912.log 2023-11-08T19:32:28.374 INFO:teuthology.orchestra.run.smithi143.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.128503.log.gz 2023-11-08T19:32:28.375 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.139270.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-11-08T19:32:28.375 INFO:teuthology.orchestra.run.smithi143.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.139270.log.gz 2023-11-08T19:32:28.375 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120299.log 2023-11-08T19:32:28.375 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.119912.log: /var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134391.log 2023-11-08T19:32:28.376 INFO:teuthology.orchestra.run.smithi143.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.119912.log.gz 2023-11-08T19:32:28.376 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.120299.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110915.log 2023-11-08T19:32:28.376 INFO:teuthology.orchestra.run.smithi143.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.120299.log.gz 2023-11-08T19:32:28.376 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.134391.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127366.log 2023-11-08T19:32:28.377 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122980.log 2023-11-08T19:32:28.377 INFO:teuthology.orchestra.run.smithi143.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.134391.log.gz 2023-11-08T19:32:28.377 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.110915.log: /var/log/ceph/ceph-client.admin.127366.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112709.log 2023-11-08T19:32:28.377 INFO:teuthology.orchestra.run.smithi143.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.110915.log.gz 2023-11-08T19:32:28.378 INFO:teuthology.orchestra.run.smithi143.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.127366.log.gz 2023-11-08T19:32:28.378 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.122980.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125669.log 2023-11-08T19:32:28.378 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.112709.log: 86.2% -- replaced with /var/log/ceph/ceph.log.gz 2023-11-08T19:32:28.379 INFO:teuthology.orchestra.run.smithi143.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.122980.log.gz 2023-11-08T19:32:28.379 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127612.log 2023-11-08T19:32:28.379 INFO:teuthology.orchestra.run.smithi143.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.112709.log.gz 2023-11-08T19:32:28.379 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138473.log 2023-11-08T19:32:28.380 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.125669.log: /var/log/ceph/ceph-client.admin.127612.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131872.log 2023-11-08T19:32:28.380 INFO:teuthology.orchestra.run.smithi143.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.125669.log.gz 2023-11-08T19:32:28.380 INFO:teuthology.orchestra.run.smithi143.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.127612.log.gz 2023-11-08T19:32:28.381 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.138473.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136655.log 2023-11-08T19:32:28.381 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.131872.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.138473.log.gz 2023-11-08T19:32:28.381 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110148.log 2023-11-08T19:32:28.381 INFO:teuthology.orchestra.run.smithi143.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.131872.log.gz 2023-11-08T19:32:28.382 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.136655.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117826.log 2023-11-08T19:32:28.382 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.110148.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114921.log 2023-11-08T19:32:28.382 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.117826.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.110148.log.gz 2023-11-08T19:32:28.382 INFO:teuthology.orchestra.run.smithi143.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.136655.log.gz 2023-11-08T19:32:28.383 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127702.log 2023-11-08T19:32:28.383 INFO:teuthology.orchestra.run.smithi143.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.117826.log.gz 2023-11-08T19:32:28.383 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.114921.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118651.log 2023-11-08T19:32:28.384 INFO:teuthology.orchestra.run.smithi143.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.114921.log.gz 2023-11-08T19:32:28.384 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.127702.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115893.log 2023-11-08T19:32:28.384 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.118651.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.127702.log.gz 2023-11-08T19:32:28.384 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116272.log 2023-11-08T19:32:28.385 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.115893.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121377.log 2023-11-08T19:32:28.385 INFO:teuthology.orchestra.run.smithi143.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.115893.log.gz 2023-11-08T19:32:28.385 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.116272.log: 91.5% -- replaced with /var/log/ceph/ceph-client.admin.118651.log.gz 2023-11-08T19:32:28.385 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128179.log 2023-11-08T19:32:28.386 INFO:teuthology.orchestra.run.smithi143.stderr:gzip 88.9%-5 --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.116272.log.gz/var/log/ceph/ceph.tmp-client.admin.97609.log 2023-11-08T19:32:28.386 INFO:teuthology.orchestra.run.smithi143.stderr: 2023-11-08T19:32:28.386 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.121377.log: /var/log/ceph/ceph-client.admin.128179.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117004.log 2023-11-08T19:32:28.387 INFO:teuthology.orchestra.run.smithi143.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.121377.log.gz 2023-11-08T19:32:28.387 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph.tmp-client.admin.97609.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.128179.log.gz 2023-11-08T19:32:28.387 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111469.log 2023-11-08T19:32:28.387 INFO:teuthology.orchestra.run.smithi143.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.97609.log.gz 2023-11-08T19:32:28.388 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.117004.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.125423.log 2023-11-08T19:32:28.388 INFO:teuthology.orchestra.run.smithi143.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.117004.log.gz 2023-11-08T19:32:28.388 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.111469.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114644.log 2023-11-08T19:32:28.389 INFO:teuthology.orchestra.run.smithi143.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.111469.log.gz 2023-11-08T19:32:28.389 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.0.125423.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130589.log 2023-11-08T19:32:28.389 INFO:teuthology.orchestra.run.smithi143.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.125423.log.gz 2023-11-08T19:32:28.389 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.114644.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115381.log 2023-11-08T19:32:28.390 INFO:teuthology.orchestra.run.smithi143.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.114644.log.gz 2023-11-08T19:32:28.390 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.130589.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111489.log 2023-11-08T19:32:28.390 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.115381.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120437.log 2023-11-08T19:32:28.390 INFO:teuthology.orchestra.run.smithi143.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.130589.log.gz 2023-11-08T19:32:28.391 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.111489.log: 87.8%gzip -- replaced with /var/log/ceph/ceph-client.admin.115381.log.gz-5 2023-11-08T19:32:28.391 INFO:teuthology.orchestra.run.smithi143.stderr:--verbose -- /var/log/ceph/ceph-client.admin.114875.log 2023-11-08T19:32:28.391 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.120437.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.111489.log.gz 2023-11-08T19:32:28.392 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127228.log 2023-11-08T19:32:28.392 INFO:teuthology.orchestra.run.smithi143.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.120437.log.gz 2023-11-08T19:32:28.392 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.114875.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114414.log 2023-11-08T19:32:28.392 INFO:teuthology.orchestra.run.smithi143.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.114875.log.gz 2023-11-08T19:32:28.393 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121906.log 2023-11-08T19:32:28.393 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.127228.log: /var/log/ceph/ceph-client.admin.114414.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138519.log 2023-11-08T19:32:28.393 INFO:teuthology.orchestra.run.smithi143.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.127228.log.gz 2023-11-08T19:32:28.394 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.121906.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.114414.log.gz 2023-11-08T19:32:28.394 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137420.log 2023-11-08T19:32:28.394 INFO:teuthology.orchestra.run.smithi143.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.121906.log.gz 2023-11-08T19:32:28.394 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.138519.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124387.log 2023-11-08T19:32:28.395 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.137420.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.138519.log.gz 2023-11-08T19:32:28.395 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111121.log 2023-11-08T19:32:28.395 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.124387.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112663.log 2023-11-08T19:32:28.395 INFO:teuthology.orchestra.run.smithi143.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.137420.log.gz 2023-11-08T19:32:28.396 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.111121.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.124387.log.gz 2023-11-08T19:32:28.396 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120996.log 2023-11-08T19:32:28.396 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.112663.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.111121.log.gz 2023-11-08T19:32:28.397 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128411.log 2023-11-08T19:32:28.397 INFO:teuthology.orchestra.run.smithi143.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.112663.log.gz 2023-11-08T19:32:28.397 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.120996.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127993.log 2023-11-08T19:32:28.397 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.128411.log: 89.1%gzip -- replaced with /var/log/ceph/ceph-client.admin.120996.log.gz-5 2023-11-08T19:32:28.398 INFO:teuthology.orchestra.run.smithi143.stderr:--verbose -- /var/log/ceph/ceph-client.admin.140482.log 2023-11-08T19:32:28.398 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.127993.log: 88.9%gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.128411.log.gz--verbose 2023-11-08T19:32:28.398 INFO:teuthology.orchestra.run.smithi143.stderr:-- /var/log/ceph/ceph-client.admin.112111.log 2023-11-08T19:32:28.399 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.140482.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.127993.log.gz 2023-11-08T19:32:28.399 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130267.log 2023-11-08T19:32:28.399 INFO:teuthology.orchestra.run.smithi143.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.140482.log.gz 2023-11-08T19:32:28.399 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.112111.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112341.log 2023-11-08T19:32:28.400 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.130267.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.112111.log.gz 2023-11-08T19:32:28.400 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123643.log 2023-11-08T19:32:28.400 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.112341.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.130267.log.gz 2023-11-08T19:32:28.400 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135972.log 2023-11-08T19:32:28.401 INFO:teuthology.orchestra.run.smithi143.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.112341.log.gz 2023-11-08T19:32:28.401 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.123643.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131230.log 2023-11-08T19:32:28.401 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.135972.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.123643.log.gz 2023-11-08T19:32:28.402 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132953.log 2023-11-08T19:32:28.402 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.131230.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.135972.log.gz 2023-11-08T19:32:28.402 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130635.log 2023-11-08T19:32:28.402 INFO:teuthology.orchestra.run.smithi143.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.131230.log.gz 2023-11-08T19:32:28.403 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.132953.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127808.log 2023-11-08T19:32:28.403 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.130635.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.132953.log.gz 2023-11-08T19:32:28.403 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2023-11-08T19:32:28.404 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.127808.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.130635.log.gz 2023-11-08T19:32:28.404 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120142.log 2023-11-08T19:32:28.404 INFO:teuthology.orchestra.run.smithi143.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.127808.log.gz 2023-11-08T19:32:28.404 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119866.log 2023-11-08T19:32:28.405 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.120142.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135478.log 2023-11-08T19:32:28.405 INFO:teuthology.orchestra.run.smithi143.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.120142.log.gz 2023-11-08T19:32:28.405 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.119866.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138856.log 2023-11-08T19:32:28.406 INFO:teuthology.orchestra.run.smithi143.stderr: 88.6%/var/log/ceph/ceph-client.admin.135478.log: gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.119866.log.gz--verbose 2023-11-08T19:32:28.406 INFO:teuthology.orchestra.run.smithi143.stderr:-- /var/log/ceph/ceph-client.admin.125028.log 2023-11-08T19:32:28.406 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.138856.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.135478.log.gz 2023-11-08T19:32:28.406 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123965.log 2023-11-08T19:32:28.407 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121538.log 2023-11-08T19:32:28.407 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.125028.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.138856.log.gz 2023-11-08T19:32:28.407 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.123965.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.125028.log.gz 2023-11-08T19:32:28.407 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118392.log 2023-11-08T19:32:28.408 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131546.log 2023-11-08T19:32:28.408 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.121538.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.123965.log.gz 2023-11-08T19:32:28.408 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.118392.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.121538.log.gz 2023-11-08T19:32:28.409 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138158.log 2023-11-08T19:32:28.409 INFO:teuthology.orchestra.run.smithi143.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.118392.log.gz 2023-11-08T19:32:28.409 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.131546.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125533.log 2023-11-08T19:32:28.409 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.138158.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124750.log 2023-11-08T19:32:28.410 INFO:teuthology.orchestra.run.smithi143.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.131546.log.gz 2023-11-08T19:32:28.410 INFO:teuthology.orchestra.run.smithi143.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.138158.log.gz 2023-11-08T19:32:28.410 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.125533.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111479.log 2023-11-08T19:32:28.410 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140390.log 2023-11-08T19:32:28.411 INFO:teuthology.orchestra.run.smithi143.stderr: 88.9%/var/log/ceph/ceph-client.admin.124750.log: -- replaced with /var/log/ceph/ceph-client.admin.125533.log.gz 2023-11-08T19:32:28.411 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.111479.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114829.log 2023-11-08T19:32:28.411 INFO:teuthology.orchestra.run.smithi143.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.124750.log.gz 2023-11-08T19:32:28.412 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.140390.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.111479.log.gz 2023-11-08T19:32:28.412 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116774.log 2023-11-08T19:32:28.412 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.114829.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.140390.log.gz 2023-11-08T19:32:28.412 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136257.log 2023-11-08T19:32:28.413 INFO:teuthology.orchestra.run.smithi143.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.114829.log.gz 2023-11-08T19:32:28.413 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.116774.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133658.log 2023-11-08T19:32:28.413 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.136257.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.116774.log.gz 2023-11-08T19:32:28.413 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121331.log 2023-11-08T19:32:28.414 INFO:teuthology.orchestra.run.smithi143.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.136257.log.gz 2023-11-08T19:32:28.414 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.133658.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133225.log 2023-11-08T19:32:28.414 INFO:teuthology.orchestra.run.smithi143.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.133658.log.gz 2023-11-08T19:32:28.415 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.121331.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123210.log 2023-11-08T19:32:28.415 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118721.log 2023-11-08T19:32:28.418 INFO:teuthology.orchestra.run.smithi143.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.121331.log.gz/var/log/ceph/ceph-client.admin.133225.log: 2023-11-08T19:32:28.418 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.123210.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.133225.log.gz 2023-11-08T19:32:28.418 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140252.log 2023-11-08T19:32:28.419 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.118721.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2023-11-08T19:32:28.419 INFO:teuthology.orchestra.run.smithi143.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.123210.log.gz 2023-11-08T19:32:28.419 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.140252.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.118721.log.gz 2023-11-08T19:32:28.419 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131686.log 2023-11-08T19:32:28.420 INFO:teuthology.orchestra.run.smithi143.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.140252.log.gz 2023-11-08T19:32:28.420 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117874.log 2023-11-08T19:32:28.420 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-mds.a.log: /var/log/ceph/ceph-client.admin.131686.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121492.log 2023-11-08T19:32:28.420 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.117874.log: 87.5%gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.131686.log.gz--verbose 2023-11-08T19:32:28.421 INFO:teuthology.orchestra.run.smithi143.stderr:-- /var/log/ceph/ceph-client.2.114056.log 2023-11-08T19:32:28.421 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.121492.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.117874.log.gz 2023-11-08T19:32:28.421 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115575.log 2023-11-08T19:32:28.422 INFO:teuthology.orchestra.run.smithi143.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.121492.log.gz 2023-11-08T19:32:28.422 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.2.114056.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141041.log 2023-11-08T19:32:28.422 INFO:teuthology.orchestra.run.smithi143.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.2.114056.log.gz 2023-11-08T19:32:28.422 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134345.log 2023-11-08T19:32:28.423 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.115575.log: /var/log/ceph/ceph-client.admin.141041.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139661.log 2023-11-08T19:32:28.423 INFO:teuthology.orchestra.run.smithi143.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.115575.log.gz 2023-11-08T19:32:28.423 INFO:teuthology.orchestra.run.smithi143.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.141041.log.gz 2023-11-08T19:32:28.424 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.134345.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111855.log 2023-11-08T19:32:28.424 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128365.log 2023-11-08T19:32:28.424 INFO:teuthology.orchestra.run.smithi143.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.134345.log.gz 2023-11-08T19:32:28.424 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.139661.log: /var/log/ceph/ceph-client.admin.111855.log: gzip -5 88.8%--verbose -- /var/log/ceph/ceph-client.admin.133317.log -- replaced with /var/log/ceph/ceph-client.admin.139661.log.gz 2023-11-08T19:32:28.425 INFO:teuthology.orchestra.run.smithi143.stderr: 2023-11-08T19:32:28.425 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.128365.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.111855.log.gz 2023-11-08T19:32:28.425 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118346.log 2023-11-08T19:32:28.426 INFO:teuthology.orchestra.run.smithi143.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.128365.log.gz 2023-11-08T19:32:28.426 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.133317.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120391.log 2023-11-08T19:32:28.426 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.118346.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122389.log 2023-11-08T19:32:28.426 INFO:teuthology.orchestra.run.smithi143.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.133317.log.gz 2023-11-08T19:32:28.427 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.120391.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.118346.log.gz 2023-11-08T19:32:28.427 INFO:teuthology.orchestra.run.smithi143.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.120391.log.gz 2023-11-08T19:32:28.427 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.122389.log: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.122389.log.gz 2023-11-08T19:32:28.428 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126933.log 2023-11-08T19:32:28.428 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116360.log 2023-11-08T19:32:28.428 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.126933.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137146.log 2023-11-08T19:32:28.428 INFO:teuthology.orchestra.run.smithi143.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.126933.log.gz 2023-11-08T19:32:28.429 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.116360.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139490.log 2023-11-08T19:32:28.429 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.137146.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.116360.log.gz 2023-11-08T19:32:28.429 INFO:teuthology.orchestra.run.smithi143.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.137146.log.gz 2023-11-08T19:32:28.430 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.139490.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.139490.log.gz 2023-11-08T19:32:28.430 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110472.log 2023-11-08T19:32:28.430 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137724.log 2023-11-08T19:32:28.430 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.110472.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138042.log 2023-11-08T19:32:28.431 INFO:teuthology.orchestra.run.smithi143.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.110472.log.gz 2023-11-08T19:32:28.431 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.137724.log: /var/log/ceph/ceph-client.admin.138042.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.137724.log.gz 2023-11-08T19:32:28.431 INFO:teuthology.orchestra.run.smithi143.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.138042.log.gz 2023-11-08T19:32:28.432 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111857.log 2023-11-08T19:32:28.432 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135926.log 2023-11-08T19:32:28.432 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.111857.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133384.log 2023-11-08T19:32:28.433 INFO:teuthology.orchestra.run.smithi143.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.111857.log.gz/var/log/ceph/ceph-client.admin.135926.log: 2023-11-08T19:32:28.433 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.133384.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.135926.log.gz 2023-11-08T19:32:28.433 INFO:teuthology.orchestra.run.smithi143.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.133384.log.gz 2023-11-08T19:32:28.433 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130359.log 2023-11-08T19:32:28.434 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122491.log 2023-11-08T19:32:28.434 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.130359.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115243.log 2023-11-08T19:32:28.434 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.122491.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.130359.log.gz 2023-11-08T19:32:28.435 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.115243.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.122491.log.gz 2023-11-08T19:32:28.435 INFO:teuthology.orchestra.run.smithi143.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.115243.log.gz 2023-11-08T19:32:28.435 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121135.log 2023-11-08T19:32:28.435 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124249.log 2023-11-08T19:32:28.436 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.121135.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123164.log 2023-11-08T19:32:28.436 INFO:teuthology.orchestra.run.smithi143.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.121135.log.gz 2023-11-08T19:32:28.436 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.124249.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121952.log 2023-11-08T19:32:28.436 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2023-11-08T19:32:28.437 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.123164.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.124249.log.gz 2023-11-08T19:32:28.437 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.121952.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.123164.log.gz 2023-11-08T19:32:28.437 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130405.log 2023-11-08T19:32:28.438 INFO:teuthology.orchestra.run.smithi143.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.121952.log.gz 2023-11-08T19:32:28.438 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-osd.3.log: /var/log/ceph/ceph-client.admin.130405.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133271.log 2023-11-08T19:32:28.438 INFO:teuthology.orchestra.run.smithi143.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.130405.log.gz 2023-11-08T19:32:28.438 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140804.log 2023-11-08T19:32:28.439 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112525.log 2023-11-08T19:32:28.439 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.133271.log: /var/log/ceph/ceph-client.admin.140804.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.133271.log.gz 2023-11-08T19:32:28.439 INFO:teuthology.orchestra.run.smithi143.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.140804.log.gz 2023-11-08T19:32:28.439 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.112525.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.112525.log.gz 2023-11-08T19:32:28.440 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138088.log 2023-11-08T19:32:28.440 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130313.log 2023-11-08T19:32:28.440 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.138088.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117142.log 2023-11-08T19:32:28.441 INFO:teuthology.orchestra.run.smithi143.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.138088.log.gz 2023-11-08T19:32:28.441 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.130313.log: /var/log/ceph/ceph-client.admin.117142.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.130313.log.gz 2023-11-08T19:32:28.441 INFO:teuthology.orchestra.run.smithi143.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.117142.log.gz 2023-11-08T19:32:28.441 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118513.log 2023-11-08T19:32:28.442 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126136.log 2023-11-08T19:32:28.442 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.118513.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130037.log 2023-11-08T19:32:28.442 INFO:teuthology.orchestra.run.smithi143.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.118513.log.gz 2023-11-08T19:32:28.442 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.126136.log: /var/log/ceph/ceph-client.admin.130037.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.126136.log.gz 2023-11-08T19:32:28.443 INFO:teuthology.orchestra.run.smithi143.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.130037.log.gz 2023-11-08T19:32:28.443 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126841.log 2023-11-08T19:32:28.443 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118446.log 2023-11-08T19:32:28.443 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.126841.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130819.log 2023-11-08T19:32:28.444 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.118446.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.126841.log.gz 2023-11-08T19:32:28.444 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.130819.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.118446.log.gz 2023-11-08T19:32:28.444 INFO:teuthology.orchestra.run.smithi143.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.130819.log.gz 2023-11-08T19:32:28.445 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111490.log 2023-11-08T19:32:28.445 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121423.log 2023-11-08T19:32:28.445 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.111490.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.111490.log.gz 2023-11-08T19:32:28.445 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.121423.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.121423.log.gz 2023-11-08T19:32:28.446 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135716.log 2023-11-08T19:32:28.446 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131011.log 2023-11-08T19:32:28.446 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.135716.log: /var/log/ceph/ceph-client.admin.131011.log: 91.0% 88.7% -- replaced with /var/log/ceph/ceph-client.admin.135716.log.gz -- replaced with /var/log/ceph/ceph-client.admin.131011.log.gz 2023-11-08T19:32:28.447 INFO:teuthology.orchestra.run.smithi143.stderr: 2023-11-08T19:32:28.447 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120050.log 2023-11-08T19:32:28.447 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118208.log 2023-11-08T19:32:28.447 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.120050.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110869.log 2023-11-08T19:32:28.448 INFO:teuthology.orchestra.run.smithi143.stderr: 88.7%/var/log/ceph/ceph-client.admin.118208.log: -- replaced with /var/log/ceph/ceph-client.admin.120050.log.gz 2023-11-08T19:32:28.448 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.110869.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.118208.log.gz 2023-11-08T19:32:28.448 INFO:teuthology.orchestra.run.smithi143.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.110869.log.gz 2023-11-08T19:32:28.448 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136563.log 2023-11-08T19:32:28.449 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120345.log 2023-11-08T19:32:28.449 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.136563.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120483.log 2023-11-08T19:32:28.449 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.120345.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.136563.log.gz 2023-11-08T19:32:28.449 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.120483.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.120345.log.gz 2023-11-08T19:32:28.450 INFO:teuthology.orchestra.run.smithi143.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.120483.log.gz 2023-11-08T19:32:28.450 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116406.log 2023-11-08T19:32:28.450 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131315.log 2023-11-08T19:32:28.451 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.116406.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125581.log 2023-11-08T19:32:28.451 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.131315.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.116406.log.gz 2023-11-08T19:32:28.451 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.125581.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.131315.log.gz 2023-11-08T19:32:28.451 INFO:teuthology.orchestra.run.smithi143.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.125581.log.gz 2023-11-08T19:32:28.452 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111122.log 2023-11-08T19:32:28.452 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126182.log 2023-11-08T19:32:28.452 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.111122.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114690.log 2023-11-08T19:32:28.452 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.126182.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.111122.log.gz 2023-11-08T19:32:28.453 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.114690.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.126182.log.gz 2023-11-08T19:32:28.453 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139132.log 2023-11-08T19:32:28.453 INFO:teuthology.orchestra.run.smithi143.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.114690.log.gz 2023-11-08T19:32:28.454 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115013.log 2023-11-08T19:32:28.454 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125906.log 2023-11-08T19:32:28.454 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.139132.log: /var/log/ceph/ceph-client.admin.115013.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.139132.log.gz 2023-11-08T19:32:28.454 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.125906.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.115013.log.gz 2023-11-08T19:32:28.455 INFO:teuthology.orchestra.run.smithi143.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.125906.log.gz 2023-11-08T19:32:28.455 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109962.log 2023-11-08T19:32:28.455 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138565.log 2023-11-08T19:32:28.456 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.109962.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115335.log 2023-11-08T19:32:28.456 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.138565.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.109962.log.gz 2023-11-08T19:32:28.456 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.115335.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.138565.log.gz 2023-11-08T19:32:28.456 INFO:teuthology.orchestra.run.smithi143.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.115335.log.gz 2023-11-08T19:32:28.457 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135786.log 2023-11-08T19:32:28.457 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135364.log 2023-11-08T19:32:28.457 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.135786.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138810.log 2023-11-08T19:32:28.457 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.135364.log: /var/log/ceph/ceph-client.admin.138810.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.135364.log.gz 2023-11-08T19:32:28.458 INFO:teuthology.orchestra.run.smithi143.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.135786.log.gz 2023-11-08T19:32:28.459 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/ceph-client.admin.104173.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.104173.log.gz 2023-11-08T19:32:28.460 INFO:teuthology.orchestra.run.smithi174.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2023-11-08T19:32:28.460 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104264.log 2023-11-08T19:32:28.460 INFO:teuthology.orchestra.run.smithi174.stderr: 93.3% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2023-11-08T19:32:28.461 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/ceph-client.admin.104264.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.104264.log.gz 2023-11-08T19:32:28.461 INFO:teuthology.orchestra.run.smithi174.stderr: 91.4% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2023-11-08T19:32:28.465 INFO:teuthology.orchestra.run.smithi174.stderr: 94.7% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2023-11-08T19:32:28.538 INFO:teuthology.orchestra.run.smithi143.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.138810.log.gz 2023-11-08T19:32:28.538 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137306.log 2023-11-08T19:32:28.538 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130451.log 2023-11-08T19:32:28.539 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.137306.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135318.log 2023-11-08T19:32:28.539 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.130451.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.137306.log.gz 2023-11-08T19:32:28.539 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.135318.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.130451.log.gz 2023-11-08T19:32:28.540 INFO:teuthology.orchestra.run.smithi143.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.135318.log.gz 2023-11-08T19:32:28.540 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133946.log 2023-11-08T19:32:28.540 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116590.log 2023-11-08T19:32:28.540 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.133946.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122689.log 2023-11-08T19:32:28.541 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.116590.log: /var/log/ceph/ceph-client.admin.122689.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.116590.log.gz 2023-11-08T19:32:28.541 INFO:teuthology.orchestra.run.smithi143.stderr: 91.5% -- replaced with /var/log/ceph/ceph-client.admin.133946.log.gz 2023-11-08T19:32:28.541 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111119.log 88.7% 2023-11-08T19:32:28.542 INFO:teuthology.orchestra.run.smithi143.stderr: -- replaced with /var/log/ceph/ceph-client.admin.122689.log.gz 2023-11-08T19:32:28.542 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104329.log 2023-11-08T19:32:28.542 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128273.log 2023-11-08T19:32:28.542 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.111119.log: /var/log/ceph/ceph-client.admin.104329.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.111119.log.gz 2023-11-08T19:32:28.543 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.128273.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.104329.log.gz 2023-11-08T19:32:28.543 INFO:teuthology.orchestra.run.smithi143.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.128273.log.gz 2023-11-08T19:32:28.543 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133430.log 2023-11-08T19:32:28.543 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117922.log 2023-11-08T19:32:28.544 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.133430.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140068.log 2023-11-08T19:32:28.544 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.117922.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.133430.log.gz 2023-11-08T19:32:28.544 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.140068.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.117922.log.gz 2023-11-08T19:32:28.545 INFO:teuthology.orchestra.run.smithi143.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.140068.log.gz 2023-11-08T19:32:28.545 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.125427.log 2023-11-08T19:32:28.545 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123256.log 2023-11-08T19:32:28.545 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.0.125427.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2023-11-08T19:32:28.546 INFO:teuthology.orchestra.run.smithi143.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.125427.log.gz 2023-11-08T19:32:28.546 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.123256.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115985.log 2023-11-08T19:32:28.546 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-mds.c.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.123256.log.gz 2023-11-08T19:32:28.546 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.115985.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.115985.log.gz 2023-11-08T19:32:28.547 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122643.log 2023-11-08T19:32:28.547 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126044.log 2023-11-08T19:32:28.547 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.122643.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.122643.log.gz 2023-11-08T19:32:28.548 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.126044.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.126044.log.gz 2023-11-08T19:32:28.548 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132494.log 2023-11-08T19:32:28.548 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119958.log 2023-11-08T19:32:28.548 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.132494.log: /var/log/ceph/ceph-client.admin.119958.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.132494.log.gz 2023-11-08T19:32:28.549 INFO:teuthology.orchestra.run.smithi143.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.119958.log.gz 2023-11-08T19:32:28.549 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127320.log 2023-11-08T19:32:28.549 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119091.log 2023-11-08T19:32:28.550 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.127320.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.127320.log.gz 2023-11-08T19:32:28.550 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.119091.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134597.log 2023-11-08T19:32:28.550 INFO:teuthology.orchestra.run.smithi143.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.119091.log.gz 2023-11-08T19:32:28.550 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.134597.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.134597.log.gz 2023-11-08T19:32:28.551 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132378.log 2023-11-08T19:32:28.551 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130911.log 2023-11-08T19:32:28.551 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.132378.log: /var/log/ceph/ceph-client.admin.130911.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.132378.log.gz 2023-11-08T19:32:28.551 INFO:teuthology.orchestra.run.smithi143.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.130911.log.gz 2023-11-08T19:32:28.552 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128457.log 2023-11-08T19:32:28.552 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139178.log 2023-11-08T19:32:28.552 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.128457.log: 90.5% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2023-11-08T19:32:28.553 INFO:teuthology.orchestra.run.smithi143.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.128457.log.gz 2023-11-08T19:32:28.553 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.139178.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.139178.log.gz 2023-11-08T19:32:28.553 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115529.log 2023-11-08T19:32:28.553 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116498.log 2023-11-08T19:32:28.554 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.115529.log: /var/log/ceph/ceph-client.admin.116498.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.115529.log.gz 2023-11-08T19:32:28.554 INFO:teuthology.orchestra.run.smithi143.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.116498.log.gz 2023-11-08T19:32:28.554 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135205.log 2023-11-08T19:32:28.555 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.122305.log 2023-11-08T19:32:28.555 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.135205.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129424.log 2023-11-08T19:32:28.555 INFO:teuthology.orchestra.run.smithi143.stderr: 89.3%/var/log/ceph/ceph-client.0.122305.log: -- replaced with /var/log/ceph/ceph-client.admin.135205.log.gz 2023-11-08T19:32:28.555 INFO:teuthology.orchestra.run.smithi143.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.122305.log.gz 2023-11-08T19:32:28.556 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.129424.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.129424.log.gz 2023-11-08T19:32:28.556 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140666.log 2023-11-08T19:32:28.556 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.118954.log 2023-11-08T19:32:28.556 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.140666.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125952.log 2023-11-08T19:32:28.557 INFO:teuthology.orchestra.run.smithi143.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.140666.log.gz 2023-11-08T19:32:28.557 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.0.118954.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.118954.log.gz 2023-11-08T19:32:28.557 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.125952.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.125952.log.gz 2023-11-08T19:32:28.558 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126749.log 2023-11-08T19:32:28.558 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135670.log 2023-11-08T19:32:28.558 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.126749.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125799.log 2023-11-08T19:32:28.558 INFO:teuthology.orchestra.run.smithi143.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.126749.log.gz 2023-11-08T19:32:28.559 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.135670.log: /var/log/ceph/ceph-client.admin.125799.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.135670.log.gz 2023-11-08T19:32:28.559 INFO:teuthology.orchestra.run.smithi143.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.125799.log.gz 2023-11-08T19:32:28.559 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131361.log 2023-11-08T19:32:28.560 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110288.log 2023-11-08T19:32:28.560 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.131361.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123735.log 2023-11-08T19:32:28.560 INFO:teuthology.orchestra.run.smithi143.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.131361.log.gz 2023-11-08T19:32:28.560 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.110288.log: /var/log/ceph/ceph-client.admin.123735.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.110288.log.gz 2023-11-08T19:32:28.561 INFO:teuthology.orchestra.run.smithi143.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.123735.log.gz 2023-11-08T19:32:28.561 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119345.log 2023-11-08T19:32:28.561 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134437.log 2023-11-08T19:32:28.562 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.119345.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140206.log 2023-11-08T19:32:28.562 INFO:teuthology.orchestra.run.smithi143.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.119345.log.gz 2023-11-08T19:32:28.562 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.134437.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.134437.log.gz 2023-11-08T19:32:28.562 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.140206.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.140206.log.gz 2023-11-08T19:32:28.563 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127854.log 2023-11-08T19:32:28.563 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126473.log 2023-11-08T19:32:28.563 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.127854.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114598.log 2023-11-08T19:32:28.564 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.126473.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.127854.log.gz 2023-11-08T19:32:28.564 INFO:teuthology.orchestra.run.smithi143.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.126473.log.gz 2023-11-08T19:32:28.564 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137677.log 2023-11-08T19:32:28.564 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.114598.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133806.log 2023-11-08T19:32:28.565 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.137677.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.114598.log.gz 2023-11-08T19:32:28.565 INFO:teuthology.orchestra.run.smithi143.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.137677.log.gz 2023-11-08T19:32:28.565 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131734.log 2023-11-08T19:32:28.566 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.133806.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111856.log 2023-11-08T19:32:28.566 INFO:teuthology.orchestra.run.smithi143.stderr: 88.3%/var/log/ceph/ceph-client.admin.131734.log: -- replaced with /var/log/ceph/ceph-client.admin.133806.log.gz 2023-11-08T19:32:28.566 INFO:teuthology.orchestra.run.smithi143.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.131734.log.gz 2023-11-08T19:32:28.566 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.111856.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.111856.log.gz 2023-11-08T19:32:28.567 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109908.log 2023-11-08T19:32:28.567 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134905.log 2023-11-08T19:32:28.567 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.109908.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132102.log 2023-11-08T19:32:28.568 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.134905.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.109908.log.gz 2023-11-08T19:32:28.568 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.132102.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.134905.log.gz 2023-11-08T19:32:28.568 INFO:teuthology.orchestra.run.smithi143.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.132102.log.gz 2023-11-08T19:32:28.568 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110196.log 2023-11-08T19:32:28.569 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114460.log 2023-11-08T19:32:28.569 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.110196.log: /var/log/ceph/ceph-client.admin.114460.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.110196.log.gz 2023-11-08T19:32:28.569 INFO:teuthology.orchestra.run.smithi143.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.114460.log.gz 2023-11-08T19:32:28.570 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119774.log 2023-11-08T19:32:28.570 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132240.log 2023-11-08T19:32:28.570 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.119774.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140114.log 2023-11-08T19:32:28.570 INFO:teuthology.orchestra.run.smithi143.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.119774.log.gz 2023-11-08T19:32:28.571 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.132240.log: /var/log/ceph/ceph-client.admin.140114.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.132240.log.gz 2023-11-08T19:32:28.571 INFO:teuthology.orchestra.run.smithi143.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.140114.log.gz 2023-11-08T19:32:28.571 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130727.log 2023-11-08T19:32:28.572 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123348.log 2023-11-08T19:32:28.572 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.130727.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126228.log 2023-11-08T19:32:28.572 INFO:teuthology.orchestra.run.smithi143.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.130727.log.gz 2023-11-08T19:32:28.572 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.123348.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.123348.log.gz 2023-11-08T19:32:28.573 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123597.log 2023-11-08T19:32:28.573 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.126228.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120667.log 2023-11-08T19:32:28.573 INFO:teuthology.orchestra.run.smithi143.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.126228.log.gz 2023-11-08T19:32:28.573 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.123597.log: /var/log/ceph/ceph-client.admin.120667.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.123597.log.gz 2023-11-08T19:32:28.574 INFO:teuthology.orchestra.run.smithi143.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.120667.log.gz 2023-11-08T19:32:28.574 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124341.log 2023-11-08T19:32:28.574 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138902.log 2023-11-08T19:32:28.575 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.124341.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.124341.log.gz 2023-11-08T19:32:28.575 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.138902.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.138902.log.gz 2023-11-08T19:32:28.575 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137896.log 2023-11-08T19:32:28.575 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111858.log 2023-11-08T19:32:28.576 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.137896.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134835.log 2023-11-08T19:32:28.576 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.111858.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.137896.log.gz 2023-11-08T19:32:28.576 INFO:teuthology.orchestra.run.smithi143.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.111858.log.gz 2023-11-08T19:32:28.576 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.134835.log: 91.5% -- replaced with /var/log/ceph/ceph-client.admin.134835.log.gz 2023-11-08T19:32:28.577 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110008.log 2023-11-08T19:32:28.577 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140022.log 2023-11-08T19:32:28.577 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.110008.log: /var/log/ceph/ceph-client.admin.140022.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.110008.log.gz 2023-11-08T19:32:28.578 INFO:teuthology.orchestra.run.smithi143.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.140022.log.gz 2023-11-08T19:32:28.578 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126519.log 2023-11-08T19:32:28.578 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126887.log 2023-11-08T19:32:28.578 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.126519.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131780.log 2023-11-08T19:32:28.579 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.126887.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.126519.log.gz 2023-11-08T19:32:28.579 INFO:teuthology.orchestra.run.smithi143.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.126887.log.gz 2023-11-08T19:32:28.579 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.131780.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.131780.log.gz 2023-11-08T19:32:28.580 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123486.log 2023-11-08T19:32:28.580 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121722.log 2023-11-08T19:32:28.580 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.123486.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124658.log 2023-11-08T19:32:28.580 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.121722.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.123486.log.gz 2023-11-08T19:32:28.581 INFO:teuthology.orchestra.run.smithi143.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.121722.log.gz 2023-11-08T19:32:28.581 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.124658.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.124658.log.gz 2023-11-08T19:32:28.581 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-11-08T19:32:28.582 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111848.log 2023-11-08T19:32:28.582 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph.audit.log: /var/log/ceph/ceph-client.admin.111848.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.111848.log.gz 2023-11-08T19:32:28.582 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129853.log 2023-11-08T19:32:28.582 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121584.log 2023-11-08T19:32:28.583 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.129853.log: /var/log/ceph/ceph-client.admin.121584.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.129853.log.gz 2023-11-08T19:32:28.583 INFO:teuthology.orchestra.run.smithi143.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.121584.log.gz 2023-11-08T19:32:28.583 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.122301.log 2023-11-08T19:32:28.583 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129700.log 2023-11-08T19:32:28.584 INFO:teuthology.orchestra.run.smithi143.stderr: 89.3% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-11-08T19:32:28.584 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.0.122301.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.122301.log.gz 2023-11-08T19:32:28.584 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127182.log 2023-11-08T19:32:28.585 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.129700.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129654.log 2023-11-08T19:32:28.585 INFO:teuthology.orchestra.run.smithi143.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.129700.log.gz 2023-11-08T19:32:28.585 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.127182.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135556.log 2023-11-08T19:32:28.585 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.129654.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.127182.log.gz 2023-11-08T19:32:28.586 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128227.log 2023-11-08T19:32:28.586 INFO:teuthology.orchestra.run.smithi143.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.129654.log.gz 2023-11-08T19:32:28.586 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.135556.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130221.log 2023-11-08T19:32:28.587 INFO:teuthology.orchestra.run.smithi143.stderr: 89.3%/var/log/ceph/ceph-client.admin.128227.log: -- replaced with /var/log/ceph/ceph-client.admin.135556.log.gz 2023-11-08T19:32:28.587 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.130221.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.128227.log.gz 2023-11-08T19:32:28.587 INFO:teuthology.orchestra.run.smithi143.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.130221.log.gz 2023-11-08T19:32:28.588 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131638.log 2023-11-08T19:32:28.588 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124073.log 2023-11-08T19:32:28.588 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.131638.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123394.log 2023-11-08T19:32:28.588 INFO:teuthology.orchestra.run.smithi143.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.131638.log.gz 2023-11-08T19:32:28.589 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.124073.log: /var/log/ceph/ceph-client.admin.123394.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.124073.log.gz 2023-11-08T19:32:28.589 INFO:teuthology.orchestra.run.smithi143.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.123394.log.gz 2023-11-08T19:32:28.589 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117780.log 2023-11-08T19:32:28.589 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127044.log 2023-11-08T19:32:28.590 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.117780.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126795.log 2023-11-08T19:32:28.590 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.127044.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.117780.log.gz 2023-11-08T19:32:28.590 INFO:teuthology.orchestra.run.smithi143.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.127044.log.gz 2023-11-08T19:32:28.591 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124704.log 2023-11-08T19:32:28.591 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.126795.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116226.log 2023-11-08T19:32:28.591 INFO:teuthology.orchestra.run.smithi143.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.126795.log.gz 2023-11-08T19:32:28.591 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.124704.log: /var/log/ceph/ceph-client.admin.116226.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.124704.log.gz 2023-11-08T19:32:28.592 INFO:teuthology.orchestra.run.smithi143.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.116226.log.gz 2023-11-08T19:32:28.592 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123919.log 2023-11-08T19:32:28.592 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125190.log 2023-11-08T19:32:28.592 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.123919.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137942.log 2023-11-08T19:32:28.593 INFO:teuthology.orchestra.run.smithi143.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.123919.log.gz 2023-11-08T19:32:28.593 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.125190.log: 89.5%/var/log/ceph/ceph-client.admin.137942.log: -- replaced with /var/log/ceph/ceph-client.admin.125190.log.gz 2023-11-08T19:32:28.593 INFO:teuthology.orchestra.run.smithi143.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.137942.log.gz 2023-11-08T19:32:28.594 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123550.log 2023-11-08T19:32:28.594 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129899.log 2023-11-08T19:32:28.594 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.123550.log: /var/log/ceph/ceph-client.admin.129899.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.123550.log.gz 2023-11-08T19:32:28.594 INFO:teuthology.orchestra.run.smithi143.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.129899.log.gz 2023-11-08T19:32:28.595 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118162.log 2023-11-08T19:32:28.595 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132194.log 2023-11-08T19:32:28.595 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.118162.log: /var/log/ceph/ceph-client.admin.132194.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.118162.log.gz 2023-11-08T19:32:28.595 INFO:teuthology.orchestra.run.smithi143.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.132194.log.gz 2023-11-08T19:32:28.596 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112909.log 2023-11-08T19:32:28.596 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117456.log 2023-11-08T19:32:28.596 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.112909.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.112909.log.gz 2023-11-08T19:32:28.597 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.117456.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.117456.log.gz 2023-11-08T19:32:28.597 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112433.log 2023-11-08T19:32:28.597 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119437.log 2023-11-08T19:32:28.597 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.112433.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.112433.log.gz 2023-11-08T19:32:28.598 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.119437.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.119437.log.gz 2023-11-08T19:32:28.598 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129562.log 2023-11-08T19:32:28.599 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120775.log 2023-11-08T19:32:28.599 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.129562.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.129562.log.gz 2023-11-08T19:32:28.600 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.120775.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.120775.log.gz 2023-11-08T19:32:28.600 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127274.log 2023-11-08T19:32:28.600 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140574.log 2023-11-08T19:32:28.601 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.127274.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.127274.log.gz 2023-11-08T19:32:28.601 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.140574.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.140574.log.gz 2023-11-08T19:32:28.601 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136209.log 2023-11-08T19:32:28.602 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120575.log 2023-11-08T19:32:28.602 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.136209.log: /var/log/ceph/ceph-client.admin.120575.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.136209.log.gz 2023-11-08T19:32:28.602 INFO:teuthology.orchestra.run.smithi143.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.120575.log.gz 2023-11-08T19:32:28.603 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122735.log 2023-11-08T19:32:28.603 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136086.log 2023-11-08T19:32:28.603 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.122735.log: /var/log/ceph/ceph-client.admin.136086.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.122735.log.gz 2023-11-08T19:32:28.603 INFO:teuthology.orchestra.run.smithi143.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.136086.log.gz 2023-11-08T19:32:28.604 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.132727.log 2023-11-08T19:32:28.604 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116728.log 2023-11-08T19:32:28.604 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.0.132727.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.132727.log.gz 2023-11-08T19:32:28.605 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.116728.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.116728.log.gz 2023-11-08T19:32:28.605 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.118958.log 2023-11-08T19:32:28.605 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119728.log 2023-11-08T19:32:28.605 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.0.118958.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.118958.log.gz 2023-11-08T19:32:28.606 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.119728.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.119728.log.gz 2023-11-08T19:32:28.606 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140160.log 2023-11-08T19:32:28.606 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133612.log 2023-11-08T19:32:28.607 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.140160.log: /var/log/ceph/ceph-client.admin.133612.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.140160.log.gz 2023-11-08T19:32:28.607 INFO:teuthology.orchestra.run.smithi143.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.133612.log.gz 2023-11-08T19:32:28.607 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110380.log 2023-11-08T19:32:28.607 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112387.log 2023-11-08T19:32:28.610 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.110380.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.110380.log.gz 2023-11-08T19:32:28.610 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.112387.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.112387.log.gz 2023-11-08T19:32:28.610 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136986.log 2023-11-08T19:32:28.611 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139086.log 2023-11-08T19:32:28.611 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.136986.log: /var/log/ceph/ceph-client.admin.139086.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.136986.log.gz 2023-11-08T19:32:28.611 INFO:teuthology.orchestra.run.smithi143.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.139086.log.gz 2023-11-08T19:32:28.612 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115847.log 2023-11-08T19:32:28.612 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126657.log 2023-11-08T19:32:28.612 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.115847.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.115847.log.gz 2023-11-08T19:32:28.612 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.126657.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.126657.log.gz 2023-11-08T19:32:28.613 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122827.log 2023-11-08T19:32:28.613 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132332.log 2023-11-08T19:32:28.613 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.122827.log: /var/log/ceph/ceph-client.admin.132332.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.122827.log.gz 2023-11-08T19:32:28.614 INFO:teuthology.orchestra.run.smithi143.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.132332.log.gz 2023-11-08T19:32:28.614 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118022.log 2023-11-08T19:32:28.614 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126427.log 2023-11-08T19:32:28.614 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.118022.log: /var/log/ceph/ceph-client.admin.126427.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.118022.log.gz 2023-11-08T19:32:28.615 INFO:teuthology.orchestra.run.smithi143.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.126427.log.gz 2023-11-08T19:32:28.615 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138994.log 2023-11-08T19:32:28.615 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128085.log 2023-11-08T19:32:28.616 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.138994.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.138994.log.gz 2023-11-08T19:32:28.616 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.128085.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.128085.log.gz 2023-11-08T19:32:28.616 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129608.log 2023-11-08T19:32:28.616 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131407.log 2023-11-08T19:32:28.617 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.129608.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.129608.log.gz 2023-11-08T19:32:28.617 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.131407.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.131407.log.gz 2023-11-08T19:32:28.617 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123781.log 2023-11-08T19:32:28.618 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118116.log 2023-11-08T19:32:28.618 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.123781.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114278.log 2023-11-08T19:32:28.618 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.118116.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.123781.log.gz 2023-11-08T19:32:28.619 INFO:teuthology.orchestra.run.smithi143.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.118116.log.gz 2023-11-08T19:32:28.619 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.114278.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.114278.log.gz 2023-11-08T19:32:28.619 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136916.log 2023-11-08T19:32:28.619 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136795.log 2023-11-08T19:32:28.620 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.136916.log: /var/log/ceph/ceph-client.admin.136795.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.136916.log.gz 2023-11-08T19:32:28.620 INFO:teuthology.orchestra.run.smithi143.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.136795.log.gz 2023-11-08T19:32:28.620 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132010.log 2023-11-08T19:32:28.621 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129190.log 2023-11-08T19:32:28.621 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.132010.log: /var/log/ceph/ceph-client.admin.129190.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.132010.log.gz 2023-11-08T19:32:28.621 INFO:teuthology.orchestra.run.smithi143.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.129190.log.gz 2023-11-08T19:32:28.621 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110777.log 2023-11-08T19:32:28.622 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115151.log 2023-11-08T19:32:28.622 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.110777.log: /var/log/ceph/ceph-client.admin.115151.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.110777.log.gz 2023-11-08T19:32:28.622 INFO:teuthology.orchestra.run.smithi143.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.115151.log.gz 2023-11-08T19:32:28.622 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121042.log 2023-11-08T19:32:28.623 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110242.log 2023-11-08T19:32:28.623 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.121042.log: /var/log/ceph/ceph-client.admin.110242.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.121042.log.gz 2023-11-08T19:32:28.623 INFO:teuthology.orchestra.run.smithi143.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.110242.log.gz 2023-11-08T19:32:28.624 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132907.log 2023-11-08T19:32:28.624 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123302.log 2023-11-08T19:32:28.624 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.132907.log: /var/log/ceph/ceph-client.admin.123302.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.132907.log.gz 2023-11-08T19:32:28.625 INFO:teuthology.orchestra.run.smithi143.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.123302.log.gz 2023-11-08T19:32:28.625 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112571.log 2023-11-08T19:32:28.625 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121277.log 2023-11-08T19:32:28.625 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.112571.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.112571.log.gz 2023-11-08T19:32:28.626 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.121277.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.121277.log.gz 2023-11-08T19:32:28.626 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.129040.log 2023-11-08T19:32:28.626 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118300.log 2023-11-08T19:32:28.627 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.0.129040.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.129040.log.gz 2023-11-08T19:32:28.627 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.118300.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.118300.log.gz 2023-11-08T19:32:28.627 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140528.log 2023-11-08T19:32:28.628 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122873.log 2023-11-08T19:32:28.628 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.140528.log: /var/log/ceph/ceph-client.admin.122873.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.140528.log.gz 2023-11-08T19:32:28.628 INFO:teuthology.orchestra.run.smithi143.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.122873.log.gz 2023-11-08T19:32:28.628 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112203.log 2023-11-08T19:32:28.629 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138948.log 2023-11-08T19:32:28.629 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.112203.log: /var/log/ceph/ceph-client.admin.138948.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.112203.log.gz 2023-11-08T19:32:28.629 INFO:teuthology.orchestra.run.smithi143.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.138948.log.gz 2023-11-08T19:32:28.630 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136417.log 2023-11-08T19:32:28.630 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136495.log 2023-11-08T19:32:28.630 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.136417.log: /var/log/ceph/ceph-client.admin.136495.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.136417.log.gz 2023-11-08T19:32:28.630 INFO:teuthology.orchestra.run.smithi143.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.136495.log.gz 2023-11-08T19:32:28.631 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110100.log 2023-11-08T19:32:28.631 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131592.log 2023-11-08T19:32:28.631 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.110100.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137850.log 2023-11-08T19:32:28.632 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.131592.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.110100.log.gz 2023-11-08T19:32:28.632 INFO:teuthology.orchestra.run.smithi143.stderr: 88.7%/var/log/ceph/ceph-client.admin.137850.log: -- replaced with /var/log/ceph/ceph-client.admin.131592.log.gz 2023-11-08T19:32:28.632 INFO:teuthology.orchestra.run.smithi143.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.137850.log.gz 2023-11-08T19:32:28.633 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134483.log 2023-11-08T19:32:28.633 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109827.log 2023-11-08T19:32:28.633 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.134483.log: /var/log/ceph/ceph-client.admin.109827.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.134483.log.gz 2023-11-08T19:32:28.634 INFO:teuthology.orchestra.run.smithi143.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.109827.log.gz 2023-11-08T19:32:28.634 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124433.log 2023-11-08T19:32:28.634 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122781.log 2023-11-08T19:32:28.635 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.124433.log: /var/log/ceph/ceph-client.admin.122781.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.124433.log.gz 2023-11-08T19:32:28.635 INFO:teuthology.orchestra.run.smithi143.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.122781.log.gz 2023-11-08T19:32:28.635 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130497.log 2023-11-08T19:32:28.636 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138703.log 2023-11-08T19:32:28.636 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.130497.log: /var/log/ceph/ceph-client.admin.138703.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.130497.log.gz 2023-11-08T19:32:28.636 INFO:teuthology.orchestra.run.smithi143.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.138703.log.gz 2023-11-08T19:32:28.637 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2023-11-08T19:32:28.637 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133900.log 2023-11-08T19:32:28.637 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-osd.1.log: /var/log/ceph/ceph-client.admin.133900.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.133900.log.gz 2023-11-08T19:32:28.637 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124163.log 2023-11-08T19:32:28.638 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139408.log 2023-11-08T19:32:28.638 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.124163.log: /var/log/ceph/ceph-client.admin.139408.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.124163.log.gz 2023-11-08T19:32:28.639 INFO:teuthology.orchestra.run.smithi143.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.139408.log.gz 2023-11-08T19:32:28.639 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111116.log 2023-11-08T19:32:28.639 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119820.log 2023-11-08T19:32:28.640 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.111116.log: /var/log/ceph/ceph-client.admin.119820.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.111116.log.gz 2023-11-08T19:32:28.640 INFO:teuthology.orchestra.run.smithi143.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.119820.log.gz 2023-11-08T19:32:28.640 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111697.log 2023-11-08T19:32:28.641 INFO:teuthology.orchestra.run.smithi143.stderr: 92.9% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2023-11-08T19:32:28.641 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112295.log 2023-11-08T19:32:28.641 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124844.log 2023-11-08T19:32:28.641 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.111697.log: /var/log/ceph/ceph-client.admin.112295.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.111697.log.gz 2023-11-08T19:32:28.642 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.124844.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.112295.log.gz 2023-11-08T19:32:28.642 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110054.log 2023-11-08T19:32:28.643 INFO:teuthology.orchestra.run.smithi143.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.124844.log.gz 2023-11-08T19:32:28.643 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110564.log 2023-11-08T19:32:28.644 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115289.log 2023-11-08T19:32:28.644 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.110054.log: /var/log/ceph/ceph-client.admin.110564.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.110054.log.gz 2023-11-08T19:32:28.644 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.115289.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.110564.log.gz 2023-11-08T19:32:28.645 INFO:teuthology.orchestra.run.smithi143.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.115289.log.gz 2023-11-08T19:32:28.645 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123873.log 2023-11-08T19:32:28.645 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136303.log 2023-11-08T19:32:28.646 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.123873.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139976.log 2023-11-08T19:32:28.646 INFO:teuthology.orchestra.run.smithi143.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.123873.log.gz 2023-11-08T19:32:28.646 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.136303.log: /var/log/ceph/ceph-client.admin.139976.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.136303.log.gz 2023-11-08T19:32:28.647 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125692.log 2023-11-08T19:32:28.647 INFO:teuthology.orchestra.run.smithi143.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.139976.log.gz 2023-11-08T19:32:28.648 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141131.log 2023-11-08T19:32:28.648 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.125692.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129470.log 2023-11-08T19:32:28.648 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.141131.log: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.125692.log.gz 2023-11-08T19:32:28.649 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.129470.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.141131.log.gz 2023-11-08T19:32:28.649 INFO:teuthology.orchestra.run.smithi143.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.129470.log.gz 2023-11-08T19:32:28.649 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115197.log 2023-11-08T19:32:28.649 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116682.log 2023-11-08T19:32:28.650 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.115197.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124982.log 2023-11-08T19:32:28.650 INFO:teuthology.orchestra.run.smithi143.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.115197.log.gz 2023-11-08T19:32:28.650 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.116682.log: /var/log/ceph/ceph-client.admin.124982.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.116682.log.gz 2023-11-08T19:32:28.651 INFO:teuthology.orchestra.run.smithi143.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.124982.log.gz 2023-11-08T19:32:28.651 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140620.log 2023-11-08T19:32:28.651 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120865.log 2023-11-08T19:32:28.651 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.140620.log: /var/log/ceph/ceph-client.admin.120865.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.140620.log.gz 2023-11-08T19:32:28.652 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114783.log 2023-11-08T19:32:28.652 INFO:teuthology.orchestra.run.smithi143.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.120865.log.gz 2023-11-08T19:32:28.652 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129746.log 2023-11-08T19:32:28.652 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.114783.log: /var/log/ceph/ceph-client.admin.129746.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.114783.log.gz 2023-11-08T19:32:28.653 INFO:teuthology.orchestra.run.smithi143.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.129746.log.gz 2023-11-08T19:32:28.653 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111120.log 2023-11-08T19:32:28.653 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112617.log 2023-11-08T19:32:28.654 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.111120.log: /var/log/ceph/ceph-client.admin.112617.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.111120.log.gz 2023-11-08T19:32:28.654 INFO:teuthology.orchestra.run.smithi143.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.112617.log.gz 2023-11-08T19:32:28.654 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124479.log 2023-11-08T19:32:28.654 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124535.log 2023-11-08T19:32:28.655 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.124479.log: /var/log/ceph/ceph-client.admin.124535.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.124479.log.gz 2023-11-08T19:32:28.655 INFO:teuthology.orchestra.run.smithi143.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.124535.log.gz 2023-11-08T19:32:28.655 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131500.log 2023-11-08T19:32:28.655 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137259.log 2023-11-08T19:32:28.656 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.131500.log: /var/log/ceph/ceph-client.admin.137259.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.131500.log.gz 2023-11-08T19:32:28.656 INFO:teuthology.orchestra.run.smithi143.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.137259.log.gz 2023-11-08T19:32:28.656 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128641.log 2023-11-08T19:32:28.657 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111847.log 2023-11-08T19:32:28.657 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.128641.log: /var/log/ceph/ceph-client.admin.111847.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.128641.log.gz 2023-11-08T19:32:28.657 INFO:teuthology.orchestra.run.smithi143.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.111847.log.gz 2023-11-08T19:32:28.658 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111486.log 2023-11-08T19:32:28.658 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121229.log 2023-11-08T19:32:28.658 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.111486.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132999.log 2023-11-08T19:32:28.659 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.121229.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.111486.log.gz 2023-11-08T19:32:28.659 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136609.log 2023-11-08T19:32:28.659 INFO:teuthology.orchestra.run.smithi143.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.121229.log.gz 2023-11-08T19:32:28.660 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.132999.log: /var/log/ceph/ceph-client.admin.136609.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.132999.log.gz 2023-11-08T19:32:28.660 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121181.log 2023-11-08T19:32:28.660 INFO:teuthology.orchestra.run.smithi143.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.136609.log.gz 2023-11-08T19:32:28.661 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131140.log 2023-11-08T19:32:28.661 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.121181.log: /var/log/ceph/ceph-client.admin.131140.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.121181.log.gz 2023-11-08T19:32:28.661 INFO:teuthology.orchestra.run.smithi143.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.131140.log.gz 2023-11-08T19:32:28.662 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132286.log 2023-11-08T19:32:28.662 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117188.log 2023-11-08T19:32:28.662 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.132286.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111111.log 2023-11-08T19:32:28.662 INFO:teuthology.orchestra.run.smithi143.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.132286.log.gz 2023-11-08T19:32:28.663 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.117188.log: /var/log/ceph/ceph-client.admin.111111.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.117188.log.gz 2023-11-08T19:32:28.663 INFO:teuthology.orchestra.run.smithi143.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.111111.log.gz 2023-11-08T19:32:28.663 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134156.log 2023-11-08T19:32:28.664 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129516.log 2023-11-08T19:32:28.664 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.134156.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111481.log 2023-11-08T19:32:28.664 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.129516.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.134156.log.gz 2023-11-08T19:32:28.665 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.111481.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.129516.log.gz 2023-11-08T19:32:28.665 INFO:teuthology.orchestra.run.smithi143.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.111481.log.gz 2023-11-08T19:32:28.665 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127900.log 2023-11-08T19:32:28.666 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128039.log 2023-11-08T19:32:28.666 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.127900.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134789.log 2023-11-08T19:32:28.666 INFO:teuthology.orchestra.run.smithi143.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.127900.log.gz 2023-11-08T19:32:28.667 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.128039.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.128039.log.gz 2023-11-08T19:32:28.667 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137565.log 2023-11-08T19:32:28.667 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.134789.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123072.log 2023-11-08T19:32:28.668 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.137565.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.134789.log.gz 2023-11-08T19:32:28.668 INFO:teuthology.orchestra.run.smithi143.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.137565.log.gz 2023-11-08T19:32:28.668 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118559.log 2023-11-08T19:32:28.669 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.123072.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140850.log 2023-11-08T19:32:28.669 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.118559.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.123072.log.gz 2023-11-08T19:32:28.669 INFO:teuthology.orchestra.run.smithi143.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.118559.log.gz 2023-11-08T19:32:28.669 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121998.log 2023-11-08T19:32:28.670 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.140850.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131454.log 2023-11-08T19:32:28.670 INFO:teuthology.orchestra.run.smithi143.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.140850.log.gz/var/log/ceph/ceph-client.admin.121998.log: 2023-11-08T19:32:28.670 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.131454.log: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.121998.log.gz 2023-11-08T19:32:28.671 INFO:teuthology.orchestra.run.smithi143.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.131454.log.gz 2023-11-08T19:32:28.671 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133476.log 2023-11-08T19:32:28.671 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134210.log 2023-11-08T19:32:28.671 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.133476.log: /var/log/ceph/ceph-client.admin.134210.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.133476.log.gz 2023-11-08T19:32:28.672 INFO:teuthology.orchestra.run.smithi143.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.134210.log.gz 2023-11-08T19:32:28.672 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139584.log 2023-11-08T19:32:28.672 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122068.log 2023-11-08T19:32:28.673 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.139584.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138228.log 2023-11-08T19:32:28.673 INFO:teuthology.orchestra.run.smithi143.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.139584.log.gz 2023-11-08T19:32:28.673 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.122068.log: /var/log/ceph/ceph-client.admin.138228.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.122068.log.gz 2023-11-08T19:32:28.673 INFO:teuthology.orchestra.run.smithi143.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.138228.log.gz 2023-11-08T19:32:28.674 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131964.log 2023-11-08T19:32:28.674 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123689.log 2023-11-08T19:32:28.674 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.131964.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126565.log 2023-11-08T19:32:28.675 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.123689.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.131964.log.gz 2023-11-08T19:32:28.675 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121860.log 2023-11-08T19:32:28.675 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.126565.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.123689.log.gz 2023-11-08T19:32:28.676 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111853.log 2023-11-08T19:32:28.676 INFO:teuthology.orchestra.run.smithi143.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.126565.log.gz 2023-11-08T19:32:28.677 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.121860.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134016.log 2023-11-08T19:32:28.677 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.111853.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.121860.log.gz 2023-11-08T19:32:28.677 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.134016.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.111853.log.gz 2023-11-08T19:32:28.678 INFO:teuthology.orchestra.run.smithi143.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.134016.log.gz 2023-11-08T19:32:28.678 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115939.log 2023-11-08T19:32:28.678 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128131.log 2023-11-08T19:32:28.679 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.115939.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128595.log 2023-11-08T19:32:28.679 INFO:teuthology.orchestra.run.smithi143.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.115939.log.gz 2023-11-08T19:32:28.679 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.128131.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.128131.log.gz 2023-11-08T19:32:28.680 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126381.log 2023-11-08T19:32:28.680 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.128595.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138657.log 2023-11-08T19:32:28.680 INFO:teuthology.orchestra.run.smithi143.stderr: 88.1%/var/log/ceph/ceph-client.admin.126381.log: -- replaced with /var/log/ceph/ceph-client.admin.128595.log.gz 2023-11-08T19:32:28.681 INFO:teuthology.orchestra.run.smithi143.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.126381.log.gz 2023-11-08T19:32:28.681 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117642.log 2023-11-08T19:32:28.681 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.138657.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135272.log 2023-11-08T19:32:28.681 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.117642.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.138657.log.gz 2023-11-08T19:32:28.682 INFO:teuthology.orchestra.run.smithi143.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.117642.log.gz 2023-11-08T19:32:28.682 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.135272.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.135272.log.gz 2023-11-08T19:32:28.682 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117364.log 2023-11-08T19:32:28.683 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124798.log 2023-11-08T19:32:28.683 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.117364.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132148.log 2023-11-08T19:32:28.683 INFO:teuthology.orchestra.run.smithi143.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.117364.log.gz 2023-11-08T19:32:28.683 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.124798.log: /var/log/ceph/ceph-client.admin.132148.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.124798.log.gz 2023-11-08T19:32:28.684 INFO:teuthology.orchestra.run.smithi143.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.132148.log.gz 2023-11-08T19:32:28.684 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123440.log 2023-11-08T19:32:28.684 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128733.log 2023-11-08T19:32:28.684 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.123440.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116452.log 2023-11-08T19:32:28.685 INFO:teuthology.orchestra.run.smithi143.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.123440.log.gz 2023-11-08T19:32:28.685 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.128733.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.128733.log.gz 2023-11-08T19:32:28.685 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127412.log 2023-11-08T19:32:28.686 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.116452.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118068.log 2023-11-08T19:32:28.686 INFO:teuthology.orchestra.run.smithi143.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.116452.log.gz 2023-11-08T19:32:28.686 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.127412.log: /var/log/ceph/ceph-client.admin.118068.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.127412.log.gz 2023-11-08T19:32:28.687 INFO:teuthology.orchestra.run.smithi143.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.118068.log.gz 2023-11-08T19:32:28.687 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112755.log 2023-11-08T19:32:28.687 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117548.log 2023-11-08T19:32:28.687 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.112755.log: /var/log/ceph/ceph-client.admin.117548.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.112755.log.gz 2023-11-08T19:32:28.688 INFO:teuthology.orchestra.run.smithi143.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.117548.log.gz 2023-11-08T19:32:28.688 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116544.log 2023-11-08T19:32:28.689 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110656.log 2023-11-08T19:32:28.689 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.116544.log: /var/log/ceph/ceph-client.admin.110656.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.116544.log.gz 2023-11-08T19:32:28.689 INFO:teuthology.orchestra.run.smithi143.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.110656.log.gz 2023-11-08T19:32:28.689 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133115.log 2023-11-08T19:32:28.690 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117976.log 2023-11-08T19:32:28.690 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.133115.log: /var/log/ceph/ceph-client.admin.117976.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.133115.log.gz 2023-11-08T19:32:28.690 INFO:teuthology.orchestra.run.smithi143.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.117976.log.gz 2023-11-08T19:32:28.691 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114737.log 2023-11-08T19:32:28.691 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125607.log 2023-11-08T19:32:28.691 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.114737.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.114737.log.gz 2023-11-08T19:32:28.692 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.125607.log: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.125607.log.gz 2023-11-08T19:32:28.692 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116180.log 2023-11-08T19:32:28.692 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119153.log 2023-11-08T19:32:28.692 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.116180.log: /var/log/ceph/ceph-client.admin.119153.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.116180.log.gz 2023-11-08T19:32:28.693 INFO:teuthology.orchestra.run.smithi143.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.119153.log.gz 2023-11-08T19:32:28.693 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120529.log 2023-11-08T19:32:28.693 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116958.log 2023-11-08T19:32:28.694 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.120529.log: /var/log/ceph/ceph-client.admin.116958.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.120529.log.gz 2023-11-08T19:32:28.694 INFO:teuthology.orchestra.run.smithi143.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.116958.log.gz 2023-11-08T19:32:28.694 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126090.log 2023-11-08T19:32:28.694 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131826.log 2023-11-08T19:32:28.695 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.126090.log: /var/log/ceph/ceph-client.admin.131826.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.126090.log.gz 2023-11-08T19:32:28.695 INFO:teuthology.orchestra.run.smithi143.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.131826.log.gz 2023-11-08T19:32:28.695 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117595.log 2023-11-08T19:32:28.696 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140712.log 2023-11-08T19:32:28.696 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.117595.log: /var/log/ceph/ceph-client.admin.140712.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.117595.log.gz 2023-11-08T19:32:28.696 INFO:teuthology.orchestra.run.smithi143.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.140712.log.gz 2023-11-08T19:32:28.696 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121814.log 2023-11-08T19:32:28.697 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125754.log 2023-11-08T19:32:28.697 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.121814.log: /var/log/ceph/ceph-client.admin.125754.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.121814.log.gz 2023-11-08T19:32:28.697 INFO:teuthology.orchestra.run.smithi143.stderr: 85.5% -- replaced with /var/log/ceph/ceph-client.admin.125754.log.gz 2023-11-08T19:32:28.698 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109633.log 2023-11-08T19:32:28.698 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116820.log 2023-11-08T19:32:28.698 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.109633.log: /var/log/ceph/ceph-client.admin.116820.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.109633.log.gz 2023-11-08T19:32:28.698 INFO:teuthology.orchestra.run.smithi143.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.116820.log.gz 2023-11-08T19:32:28.699 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123827.log 2023-11-08T19:32:28.699 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119238.log 2023-11-08T19:32:28.699 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.123827.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.123827.log.gz 2023-11-08T19:32:28.700 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.119238.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.119238.log.gz 2023-11-08T19:32:28.700 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133704.log 2023-11-08T19:32:28.700 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115059.log 2023-11-08T19:32:28.700 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.133704.log: /var/log/ceph/ceph-client.admin.115059.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.133704.log.gz 2023-11-08T19:32:28.701 INFO:teuthology.orchestra.run.smithi143.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.115059.log.gz 2023-11-08T19:32:28.701 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135856.log 2023-11-08T19:32:28.701 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132861.log 2023-11-08T19:32:28.702 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.135856.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117734.log 2023-11-08T19:32:28.702 INFO:teuthology.orchestra.run.smithi143.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.135856.log.gz 2023-11-08T19:32:28.702 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.132861.log: /var/log/ceph/ceph-client.admin.117734.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.132861.log.gz 2023-11-08T19:32:28.702 INFO:teuthology.orchestra.run.smithi143.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.117734.log.gz 2023-11-08T19:32:28.703 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133750.log 2023-11-08T19:32:28.703 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130957.log 2023-11-08T19:32:28.703 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.133750.log: /var/log/ceph/ceph-client.admin.130957.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.133750.log.gz 2023-11-08T19:32:28.704 INFO:teuthology.orchestra.run.smithi143.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.130957.log.gz 2023-11-08T19:32:28.704 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137487.log 2023-11-08T19:32:28.704 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136132.log 2023-11-08T19:32:28.704 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.137487.log: /var/log/ceph/ceph-client.admin.136132.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.137487.log.gz 2023-11-08T19:32:28.705 INFO:teuthology.orchestra.run.smithi143.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.136132.log.gz 2023-11-08T19:32:28.705 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126335.log 2023-11-08T19:32:28.705 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126703.log 2023-11-08T19:32:28.705 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.126335.log: /var/log/ceph/ceph-client.admin.126703.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.126335.log.gz 2023-11-08T19:32:28.706 INFO:teuthology.orchestra.run.smithi143.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.126703.log.gz 2023-11-08T19:32:28.706 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139224.log 2023-11-08T19:32:28.706 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110426.log 2023-11-08T19:32:28.707 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.139224.log: /var/log/ceph/ceph-client.admin.110426.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.139224.log.gz 2023-11-08T19:32:28.707 INFO:teuthology.orchestra.run.smithi143.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.110426.log.gz 2023-11-08T19:32:28.707 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125120.log 2023-11-08T19:32:28.707 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114368.log 2023-11-08T19:32:28.708 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.125120.log: /var/log/ceph/ceph-client.admin.114368.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.125120.log.gz 2023-11-08T19:32:28.708 INFO:teuthology.orchestra.run.smithi143.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.114368.log.gz 2023-11-08T19:32:28.709 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127504.log 2023-11-08T19:32:28.709 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125998.log 2023-11-08T19:32:28.709 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.127504.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.127504.log.gz 2023-11-08T19:32:28.709 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.125998.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.125998.log.gz 2023-11-08T19:32:28.710 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118605.log 2023-11-08T19:32:28.710 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116134.log 2023-11-08T19:32:28.710 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.118605.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.118605.log.gz 2023-11-08T19:32:28.711 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.116134.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.116134.log.gz 2023-11-08T19:32:28.711 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141085.log 2023-11-08T19:32:28.711 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135624.log 2023-11-08T19:32:28.712 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.141085.log: /var/log/ceph/ceph-client.admin.135624.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.141085.log.gz 2023-11-08T19:32:28.712 INFO:teuthology.orchestra.run.smithi143.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.135624.log.gz 2023-11-08T19:32:28.713 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124936.log 2023-11-08T19:32:28.713 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134086.log 2023-11-08T19:32:28.713 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.124936.log: /var/log/ceph/ceph-client.admin.134086.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.124936.log.gz 2023-11-08T19:32:28.714 INFO:teuthology.orchestra.run.smithi143.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.134086.log.gz 2023-11-08T19:32:28.714 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115801.log 2023-11-08T19:32:28.714 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104198.log 2023-11-08T19:32:28.715 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.115801.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115621.log 2023-11-08T19:32:28.715 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.104198.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.115801.log.gz 2023-11-08T19:32:28.715 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.115621.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.104198.log.gz 2023-11-08T19:32:28.715 INFO:teuthology.orchestra.run.smithi143.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.115621.log.gz 2023-11-08T19:32:28.716 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110334.log 2023-11-08T19:32:28.716 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.113399.log 2023-11-08T19:32:28.716 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.110334.log: /var/log/ceph/ceph-client.0.113399.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.110334.log.gz 2023-11-08T19:32:28.717 INFO:teuthology.orchestra.run.smithi143.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.113399.log.gz 2023-11-08T19:32:28.717 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115105.log 2023-11-08T19:32:28.717 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112249.log 2023-11-08T19:32:28.718 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.115105.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.115105.log.gz 2023-11-08T19:32:28.718 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.112249.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.112249.log.gz 2023-11-08T19:32:28.718 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139538.log 2023-11-08T19:32:28.719 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140995.log 2023-11-08T19:32:28.719 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.139538.log: /var/log/ceph/ceph-client.admin.140995.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.139538.log.gz 2023-11-08T19:32:28.719 INFO:teuthology.orchestra.run.smithi143.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.140995.log.gz 2023-11-08T19:32:28.719 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119483.log 2023-11-08T19:32:28.720 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117410.log 2023-11-08T19:32:28.720 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.119483.log: /var/log/ceph/ceph-client.admin.117410.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.119483.log.gz 2023-11-08T19:32:28.720 INFO:teuthology.orchestra.run.smithi143.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.117410.log.gz 2023-11-08T19:32:28.720 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117050.log 2023-11-08T19:32:28.721 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120621.log 2023-11-08T19:32:28.721 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.117050.log: /var/log/ceph/ceph-client.admin.120621.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.117050.log.gz 2023-11-08T19:32:28.721 INFO:teuthology.orchestra.run.smithi143.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.120621.log.gz 2023-11-08T19:32:28.722 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130083.log 2023-11-08T19:32:28.723 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119064.log 2023-11-08T19:32:28.724 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.130083.log: /var/log/ceph/ceph-client.admin.119064.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.130083.log.gz 2023-11-08T19:32:28.724 INFO:teuthology.orchestra.run.smithi143.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.119064.log.gz 2023-11-08T19:32:28.724 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132424.log 2023-11-08T19:32:28.724 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123026.log 2023-11-08T19:32:28.725 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.132424.log: /var/log/ceph/ceph-client.admin.123026.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.132424.log.gz 2023-11-08T19:32:28.725 INFO:teuthology.orchestra.run.smithi143.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.123026.log.gz 2023-11-08T19:32:28.725 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112065.log 2023-11-08T19:32:28.726 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111483.log 2023-11-08T19:32:28.726 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.112065.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119176.log 2023-11-08T19:32:28.726 INFO:teuthology.orchestra.run.smithi143.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.112065.log.gz 2023-11-08T19:32:28.726 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.111483.log: 88.8%/var/log/ceph/ceph-client.admin.119176.log: -- replaced with /var/log/ceph/ceph-client.admin.111483.log.gz 2023-11-08T19:32:28.727 INFO:teuthology.orchestra.run.smithi143.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.119176.log.gz 2023-11-08T19:32:28.727 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127090.log 2023-11-08T19:32:28.727 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129256.log 2023-11-08T19:32:28.728 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.127090.log: /var/log/ceph/ceph-client.admin.129256.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.127090.log.gz 2023-11-08T19:32:28.728 INFO:teuthology.orchestra.run.smithi143.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.129256.log.gz 2023-11-08T19:32:28.728 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139362.log 2023-11-08T19:32:28.728 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124295.log 2023-11-08T19:32:28.729 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.139362.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140436.log 2023-11-08T19:32:28.729 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.124295.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.139362.log.gz 2023-11-08T19:32:28.729 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.140436.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.124295.log.gz 2023-11-08T19:32:28.730 INFO:teuthology.orchestra.run.smithi143.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.140436.log.gz 2023-11-08T19:32:28.730 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119391.log 2023-11-08T19:32:28.730 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2023-11-08T19:32:28.731 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.119391.log: /var/log/ceph/ceph-mon.a.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.119391.log.gz 2023-11-08T19:32:28.732 INFO:teuthology.orchestra.run.smithi174.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2023-11-08T19:32:28.732 INFO:teuthology.orchestra.run.smithi174.stderr: 91.2% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2023-11-08T19:32:28.739 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117688.log 2023-11-08T19:32:28.739 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126611.log 2023-11-08T19:32:28.740 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.117688.log: /var/log/ceph/ceph-client.admin.126611.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.117688.log.gz 2023-11-08T19:32:28.740 INFO:teuthology.orchestra.run.smithi143.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.126611.log.gz 2023-11-08T19:32:28.740 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121676.log 2023-11-08T19:32:28.740 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120096.log 2023-11-08T19:32:28.741 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.121676.log: /var/log/ceph/ceph-client.admin.120096.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.121676.log.gz 2023-11-08T19:32:28.741 INFO:teuthology.orchestra.run.smithi143.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.120096.log.gz 2023-11-08T19:32:28.741 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130865.log 2023-11-08T19:32:28.742 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130681.log 2023-11-08T19:32:28.742 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.130865.log: /var/log/ceph/ceph-client.admin.130681.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.130865.log.gz 2023-11-08T19:32:28.742 INFO:teuthology.orchestra.run.smithi143.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.130681.log.gz 2023-11-08T19:32:28.742 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119575.log 2023-11-08T19:32:28.743 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109742.log 2023-11-08T19:32:28.743 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.119575.log: /var/log/ceph/ceph-client.admin.109742.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.119575.log.gz 2023-11-08T19:32:28.743 INFO:teuthology.orchestra.run.smithi143.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.109742.log.gz 2023-11-08T19:32:28.744 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124604.log 2023-11-08T19:32:28.744 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.114052.log 2023-11-08T19:32:28.744 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.124604.log: /var/log/ceph/ceph-client.2.114052.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.124604.log.gz 2023-11-08T19:32:28.744 INFO:teuthology.orchestra.run.smithi143.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.2.114052.log.gz 2023-11-08T19:32:28.745 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117502.log 2023-11-08T19:32:28.745 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.129036.log 2023-11-08T19:32:28.745 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.117502.log: /var/log/ceph/ceph-client.0.129036.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.117502.log.gz 2023-11-08T19:32:28.745 INFO:teuthology.orchestra.run.smithi143.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.129036.log.gz 2023-11-08T19:32:28.746 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119682.log 2023-11-08T19:32:28.746 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110518.log 2023-11-08T19:32:28.746 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.119682.log: /var/log/ceph/ceph-client.admin.110518.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.119682.log.gz 2023-11-08T19:32:28.747 INFO:teuthology.orchestra.run.smithi143.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.110518.log.gz 2023-11-08T19:32:28.747 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121089.log 2023-11-08T19:32:28.747 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130543.log 2023-11-08T19:32:28.748 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.121089.log: /var/log/ceph/ceph-client.admin.130543.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.121089.log.gz 2023-11-08T19:32:28.748 INFO:teuthology.orchestra.run.smithi143.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.130543.log.gz 2023-11-08T19:32:28.748 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140920.log 2023-11-08T19:32:28.748 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139316.log 2023-11-08T19:32:28.749 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.140920.log: /var/log/ceph/ceph-client.admin.139316.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.140920.log.gz 2023-11-08T19:32:28.749 INFO:teuthology.orchestra.run.smithi143.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.139316.log.gz 2023-11-08T19:32:28.749 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128319.log 2023-11-08T19:32:28.750 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133566.log 2023-11-08T19:32:28.750 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.128319.log: /var/log/ceph/ceph-client.admin.133566.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.128319.log.gz 2023-11-08T19:32:28.750 INFO:teuthology.orchestra.run.smithi143.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.133566.log.gz 2023-11-08T19:32:28.750 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2023-11-08T19:32:28.751 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103892.log 2023-11-08T19:32:28.751 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-mgr.y.log: /var/log/ceph/ceph-client.admin.103892.log: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.103892.log.gz 2023-11-08T19:32:28.751 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140298.log 2023-11-08T19:32:28.751 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.140298.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.140298.log.gz 2023-11-08T19:32:28.752 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134743.log 2023-11-08T19:32:28.752 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.134743.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.134743.log.gz 2023-11-08T19:32:28.752 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136725.log 2023-11-08T19:32:28.753 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.136725.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.136725.log.gz 2023-11-08T19:32:28.753 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120004.log 2023-11-08T19:32:28.753 INFO:teuthology.orchestra.run.smithi143.stderr: 94.0% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2023-11-08T19:32:28.753 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.120004.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.120004.log.gz 2023-11-08T19:32:28.754 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128803.log 2023-11-08T19:32:28.754 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135045.log 2023-11-08T19:32:28.754 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.128803.log: /var/log/ceph/ceph-client.admin.135045.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.128803.log.gz 2023-11-08T19:32:28.754 INFO:teuthology.orchestra.run.smithi143.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.135045.log.gz 2023-11-08T19:32:28.755 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116636.log 2023-11-08T19:32:28.755 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112157.log 2023-11-08T19:32:28.755 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.116636.log: /var/log/ceph/ceph-client.admin.112157.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.116636.log.gz 2023-11-08T19:32:28.756 INFO:teuthology.orchestra.run.smithi143.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.112157.log.gz 2023-11-08T19:32:28.756 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134675.log 2023-11-08T19:32:28.756 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116912.log 2023-11-08T19:32:28.756 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.134675.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121630.log 2023-11-08T19:32:28.757 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.116912.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.134675.log.gz 2023-11-08T19:32:28.757 INFO:teuthology.orchestra.run.smithi143.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.116912.log.gz 2023-11-08T19:32:28.757 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.121630.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.121630.log.gz 2023-11-08T19:32:28.757 INFO:teuthology.orchestra.run.smithi143.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2023-11-08T19:32:28.758 INFO:teuthology.orchestra.run.smithi174.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2023-11-08T19:32:28.760 INFO:teuthology.orchestra.run.smithi174.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2023-11-08T19:32:28.802 INFO:teuthology.orchestra.run.smithi174.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2023-11-08T19:32:28.806 INFO:teuthology.orchestra.run.smithi143.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2023-11-08T19:32:28.847 INFO:teuthology.orchestra.run.smithi174.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2023-11-08T19:32:28.849 INFO:teuthology.orchestra.run.smithi174.stderr: 2023-11-08T19:32:28.849 INFO:teuthology.orchestra.run.smithi174.stderr:real 0m0.564s 2023-11-08T19:32:28.850 INFO:teuthology.orchestra.run.smithi174.stderr:user 0m2.656s 2023-11-08T19:32:28.850 INFO:teuthology.orchestra.run.smithi174.stderr:sys 0m0.103s 2023-11-08T19:32:28.853 INFO:teuthology.orchestra.run.smithi143.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2023-11-08T19:32:28.891 INFO:teuthology.orchestra.run.smithi143.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2023-11-08T19:32:29.095 INFO:teuthology.orchestra.run.smithi143.stderr: 90.7% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2023-11-08T19:32:29.097 INFO:teuthology.orchestra.run.smithi143.stderr: 2023-11-08T19:32:29.097 INFO:teuthology.orchestra.run.smithi143.stderr:real 0m0.837s 2023-11-08T19:32:29.098 INFO:teuthology.orchestra.run.smithi143.stderr:user 0m2.804s 2023-11-08T19:32:29.098 INFO:teuthology.orchestra.run.smithi143.stderr:sys 0m0.351s 2023-11-08T19:32:29.098 INFO:tasks.ceph:Archiving logs... 2023-11-08T19:32:29.099 DEBUG:teuthology.misc:Transferring archived files from smithi143:/var/log/ceph to /home/teuthworker/archive/pdonnell-2023-11-08_19:01:12-fs:functional-main-distro-default-smithi/7452105/remote/smithi143/log 2023-11-08T19:32:29.100 DEBUG:teuthology.orchestra.run.smithi143:> sudo tar c -f - -C /var/log/ceph -- . 2023-11-08T19:32:30.072 DEBUG:teuthology.misc:Transferring archived files from smithi174:/var/log/ceph to /home/teuthworker/archive/pdonnell-2023-11-08_19:01:12-fs:functional-main-distro-default-smithi/7452105/remote/smithi174/log 2023-11-08T19:32:30.074 DEBUG:teuthology.orchestra.run.smithi174:> sudo tar c -f - -C /var/log/ceph -- . 2023-11-08T19:32:30.464 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-11-08T19:32:30.474 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-11-08T19:32:30.475 DEBUG:teuthology.orchestra.run.smithi143:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-11-08T19:32:30.478 DEBUG:teuthology.orchestra.run.smithi174:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-11-08T19:32:30.562 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-11-08T19:32:30.572 INFO:teuthology.task.clock:Checking final clock skew... 2023-11-08T19:32:30.572 DEBUG:teuthology.orchestra.run.smithi143:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-08T19:32:30.575 DEBUG:teuthology.orchestra.run.smithi174:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-08T19:32:30.591 INFO:teuthology.orchestra.run.smithi143.stderr:bash: ntpq: command not found 2023-11-08T19:32:30.592 INFO:teuthology.orchestra.run.smithi174.stderr:bash: ntpq: command not found 2023-11-08T19:32:30.596 INFO:teuthology.orchestra.run.smithi143.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-08T19:32:30.597 INFO:teuthology.orchestra.run.smithi143.stdout:=============================================================================== 2023-11-08T19:32:30.597 INFO:teuthology.orchestra.run.smithi143.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 41 +1934us[+1936us] +/- 83ms 2023-11-08T19:32:30.597 INFO:teuthology.orchestra.run.smithi143.stdout:^* hv02.front.sepia.ceph.com 3 6 377 39 -206us[ -203us] +/- 70ms 2023-11-08T19:32:30.597 INFO:teuthology.orchestra.run.smithi143.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 41 -1343us[-1340us] +/- 62ms 2023-11-08T19:32:30.598 INFO:teuthology.orchestra.run.smithi143.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-11-08T19:32:30.599 INFO:teuthology.orchestra.run.smithi174.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-08T19:32:30.600 INFO:teuthology.orchestra.run.smithi174.stdout:=============================================================================== 2023-11-08T19:32:30.600 INFO:teuthology.orchestra.run.smithi174.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 40 +1998us[+1998us] +/- 83ms 2023-11-08T19:32:30.600 INFO:teuthology.orchestra.run.smithi174.stdout:^* hv02.front.sepia.ceph.com 3 6 377 43 -187us[ -325us] +/- 70ms 2023-11-08T19:32:30.600 INFO:teuthology.orchestra.run.smithi174.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 42 -1327us[-1327us] +/- 62ms 2023-11-08T19:32:30.601 INFO:teuthology.orchestra.run.smithi174.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-11-08T19:32:30.601 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-11-08T19:32:30.611 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-11-08T19:32:30.612 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-11-08T19:32:30.625 DEBUG:teuthology.orchestra.run.smithi143:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-11-08T19:32:30.751 DEBUG:teuthology.orchestra.run.smithi174:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-11-08T19:32:30.848 DEBUG:teuthology.orchestra.run.smithi143:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-08T19:32:30.882 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-08T19:32:30.883 DEBUG:teuthology.orchestra.run.smithi174:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-08T19:32:30.918 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-08T19:32:30.919 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-11-08T19:32:30.931 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-11-08T19:32:30.990 INFO:teuthology.task.internal:Duration was 1237.050833 seconds 2023-11-08T19:32:30.991 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-11-08T19:32:31.001 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-11-08T19:32:31.002 DEBUG:teuthology.orchestra.run.smithi143:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-11-08T19:32:31.005 DEBUG:teuthology.orchestra.run.smithi174:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-11-08T19:32:31.049 INFO:teuthology.orchestra.run.smithi143.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-08T19:32:31.051 INFO:teuthology.orchestra.run.smithi174.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-08T19:32:31.401 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-11-08T19:32:31.402 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi143.front.sepia.ceph.com 2023-11-08T19:32:31.402 DEBUG:teuthology.orchestra.run.smithi143:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-11-08T19:32:31.424 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi174.front.sepia.ceph.com 2023-11-08T19:32:31.424 DEBUG:teuthology.orchestra.run.smithi174:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-11-08T19:32:31.446 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-11-08T19:32:31.446 DEBUG:teuthology.orchestra.run.smithi143:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-08T19:32:31.467 DEBUG:teuthology.orchestra.run.smithi174:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-08T19:32:31.527 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-11-08T19:32:31.528 DEBUG:teuthology.orchestra.run.smithi143:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-11-08T19:32:31.530 DEBUG:teuthology.orchestra.run.smithi174:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-11-08T19:32:31.643 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-11-08T19:32:31.654 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-11-08T19:32:31.655 DEBUG:teuthology.orchestra.run.smithi143:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-11-08T19:32:31.682 DEBUG:teuthology.orchestra.run.smithi174:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-11-08T19:32:31.709 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-11-08T19:32:31.721 DEBUG:teuthology.orchestra.run.smithi143:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-11-08T19:32:31.726 DEBUG:teuthology.orchestra.run.smithi174:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-11-08T19:32:31.749 INFO:teuthology.orchestra.run.smithi143.stdout:kernel.core_pattern = core 2023-11-08T19:32:31.775 INFO:teuthology.orchestra.run.smithi174.stdout:kernel.core_pattern = core 2023-11-08T19:32:31.789 DEBUG:teuthology.orchestra.run.smithi143:> test -e /home/ubuntu/cephtest/archive/coredump 2023-11-08T19:32:31.818 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-08T19:32:31.819 DEBUG:teuthology.orchestra.run.smithi174:> test -e /home/ubuntu/cephtest/archive/coredump 2023-11-08T19:32:31.845 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-08T19:32:31.846 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-11-08T19:32:31.857 INFO:teuthology.task.internal:Transferring archived files... 2023-11-08T19:32:31.858 DEBUG:teuthology.misc:Transferring archived files from smithi143:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2023-11-08_19:01:12-fs:functional-main-distro-default-smithi/7452105/remote/smithi143 2023-11-08T19:32:31.858 DEBUG:teuthology.orchestra.run.smithi143:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-11-08T19:32:31.898 DEBUG:teuthology.misc:Transferring archived files from smithi174:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2023-11-08_19:01:12-fs:functional-main-distro-default-smithi/7452105/remote/smithi174 2023-11-08T19:32:31.899 DEBUG:teuthology.orchestra.run.smithi174:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-11-08T19:32:31.936 INFO:teuthology.task.internal:Removing archive directory... 2023-11-08T19:32:31.936 DEBUG:teuthology.orchestra.run.smithi143:> rm -rf -- /home/ubuntu/cephtest/archive 2023-11-08T19:32:31.939 DEBUG:teuthology.orchestra.run.smithi174:> rm -rf -- /home/ubuntu/cephtest/archive 2023-11-08T19:32:31.987 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-11-08T19:32:31.999 INFO:teuthology.task.internal:Not uploading archives. 2023-11-08T19:32:31.999 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-11-08T19:32:32.015 INFO:teuthology.task.internal:Tidying up after the test... 2023-11-08T19:32:32.016 DEBUG:teuthology.orchestra.run.smithi143:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-11-08T19:32:32.018 DEBUG:teuthology.orchestra.run.smithi174:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-11-08T19:32:32.032 INFO:teuthology.orchestra.run.smithi143.stdout: 262320 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Nov 8 19:32 /home/ubuntu/cephtest 2023-11-08T19:32:32.043 INFO:teuthology.orchestra.run.smithi174.stdout: 262320 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Nov 8 19:32 /home/ubuntu/cephtest 2023-11-08T19:32:32.044 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-11-08T19:32:32.055 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-11-08T19:32:32.085 INFO:teuthology.nuke:Checking targets against current locks 2023-11-08T19:32:32.111 DEBUG:teuthology.nuke:shortname: smithi143 2023-11-08T19:32:32.111 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-08T19:32:32.136 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi143.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2023-11-08_19:01:12-fs:functional-main-distro-default-smithi/7452105', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-08 19:06:33.302270', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-11-08T19:32:32.142 DEBUG:teuthology.nuke:shortname: smithi174 2023-11-08T19:32:32.142 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-08T19:32:32.168 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi174.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2023-11-08_19:01:12-fs:functional-main-distro-default-smithi/7452105', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-08 19:06:33.304851', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-11-08T19:32:32.199 INFO:teuthology.orchestra.console.smithi143:Power off 2023-11-08T19:32:32.200 DEBUG:teuthology.orchestra.console.smithi143:pexpect command: ipmitool -H smithi143.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-08T19:32:32.223 DEBUG:teuthology.orchestra.console.smithi143:power off output: Chassis Power Control: Down/Off 2023-11-08T19:32:32.224 DEBUG:teuthology.orchestra.console.smithi143:pexpect command: ipmitool -H smithi143.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-08T19:32:32.242 INFO:teuthology.orchestra.console.smithi174:Power off 2023-11-08T19:32:32.243 DEBUG:teuthology.orchestra.console.smithi174:pexpect command: ipmitool -H smithi174.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-08T19:32:32.265 DEBUG:teuthology.orchestra.console.smithi174:power off output: Chassis Power Control: Down/Off 2023-11-08T19:32:32.266 DEBUG:teuthology.orchestra.console.smithi174:pexpect command: ipmitool -H smithi174.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-08T19:32:32.274 DEBUG:teuthology.orchestra.console.smithi143:check power output: Chassis Power is on 2023-11-08T19:32:32.289 DEBUG:teuthology.orchestra.console.smithi174:check power output: Chassis Power is on 2023-11-08T19:32:36.276 DEBUG:teuthology.orchestra.console.smithi143:pexpect command: ipmitool -H smithi143.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-08T19:32:36.291 DEBUG:teuthology.orchestra.console.smithi174:pexpect command: ipmitool -H smithi174.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-08T19:32:36.388 DEBUG:teuthology.orchestra.console.smithi143:check power output: Chassis Power is on 2023-11-08T19:32:36.402 DEBUG:teuthology.orchestra.console.smithi174:check power output: Chassis Power is on 2023-11-08T19:32:40.390 DEBUG:teuthology.orchestra.console.smithi143:pexpect command: ipmitool -H smithi143.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-08T19:32:40.402 DEBUG:teuthology.orchestra.console.smithi174:pexpect command: ipmitool -H smithi174.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-08T19:32:40.502 DEBUG:teuthology.orchestra.console.smithi143:check power output: Chassis Power is on 2023-11-08T19:32:40.513 DEBUG:teuthology.orchestra.console.smithi174:check power output: Chassis Power is off 2023-11-08T19:32:40.614 INFO:teuthology.orchestra.console.smithi174:Power off completed 2023-11-08T19:32:44.504 DEBUG:teuthology.orchestra.console.smithi143:pexpect command: ipmitool -H smithi143.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-08T19:32:44.615 DEBUG:teuthology.orchestra.console.smithi143:check power output: Chassis Power is off 2023-11-08T19:32:44.716 INFO:teuthology.orchestra.console.smithi143:Power off completed 2023-11-08T19:32:44.820 INFO:teuthology.run:Summary data: description: fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snapshots} duration: 1237.0508329868317 failure_reason: 'Test failure: test_kill_mdstable (tasks.cephfs.test_snapshots.TestSnapshots)' flavor: default owner: scheduled_pdonnell@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=a1594914561d41c4bf17575f28574757 status: fail success: false 2023-11-08T19:32:44.821 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-08T19:32:44.886 INFO:teuthology.run:FAIL